2024-07-31T14:27:05.304 INFO:root:teuthology version: 0.0.1.dev379+g53ce146 2024-07-31T14:27:05.332 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-31T14:27:05.429 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-07-31_13:38:24-fs-wip-vshankar-testing-20240723.143739-debug-testing-default-smithi/7828591 branch: wip-vshankar-testing-20240723.143739-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} email: vshankar@redhat.com first_in_suite: false job_id: '7828591' kernel: client: sha1: distro flavor: default kdb: 1 sha1: 34096973aec0bfbe2bb86f34701acc6cd4133fd9 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: vshankar-2024-07-31_13:38:24-fs-wip-vshankar-testing-20240723.143739-debug-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240723.143739-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug cephsqlite: 20 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_export_ephemeral_random_max: 0.1 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 8c796fca5ff5960f7542e7227bad4a784ee2c804 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 8c796fca5ff5960f7542e7227bad4a784ee2c804 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240723.143739-debug sha1: 8c796fca5ff5960f7542e7227bad4a784ee2c804 owner: scheduled_vshankar@teuthology priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3726 sha1: 8c796fca5ff5960f7542e7227bad4a784ee2c804 sleep_before_teardown: 0 subset: 841/1024 suite: fs suite_branch: wip-vshankar-testing-20240723.143739-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8c796fca5ff5960f7542e7227bad4a784ee2c804/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8c796fca5ff5960f7542e7227bad4a784ee2c804 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGqUEuwY9+9EBNkaevu7m35FNPHTYsDkt56L/lNtNkucMJO7ayXXpDBwc0aQFzoO2977/5RhvjOa3oPMsoYPmy8= smithi073.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJWO8yeotvlVnyCJBB3OYuwJKJ9uPyli5HmrS/P7rcZ75iTRfGt5vfuPrIElHofl+pcDW4HlqnxrLw2b78xKWp8= smithi077.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGaagvAkMGc/QXaGyjv4jc9+P44rESAAOf/bz6GWH1V1FA1var69vjizOZtrq4OrPnQvemZXCJjuj1ufXlMVC6I= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - kernel_untar_build.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 53ce1462e129f6eb4071986336534c740fdebd31 timestamp: 2024-07-31_13:38:24 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.4074525 2024-07-31T14:27:05.429 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8c796fca5ff5960f7542e7227bad4a784ee2c804/qa; will attempt to use it 2024-07-31T14:27:05.430 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8c796fca5ff5960f7542e7227bad4a784ee2c804/qa/tasks 2024-07-31T14:27:05.430 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-07-31T14:27:05.431 INFO:teuthology.task.internal:Checking packages... 2024-07-31T14:27:05.451 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8c796fca5ff5960f7542e7227bad4a784ee2c804' 2024-07-31T14:27:05.452 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-31T14:27:05.452 INFO:teuthology.packaging:ref: None 2024-07-31T14:27:05.452 INFO:teuthology.packaging:tag: None 2024-07-31T14:27:05.452 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240723.143739-debug 2024-07-31T14:27:05.452 INFO:teuthology.packaging:sha1: 8c796fca5ff5960f7542e7227bad4a784ee2c804 2024-07-31T14:27:05.452 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240723.143739-debug 2024-07-31T14:27:05.595 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-5223.g8c796fca 2024-07-31T14:27:05.597 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-07-31T14:27:05.605 INFO:teuthology.task.internal:no buildpackages task found 2024-07-31T14:27:05.605 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-07-31T14:27:05.644 INFO:teuthology.task.internal:Saving configuration 2024-07-31T14:27:05.665 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-07-31T14:27:05.671 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-07-31T14:27:05.698 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-07-31_13:38:24-fs-wip-vshankar-testing-20240723.143739-debug-testing-default-smithi/7828591', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-31 14:22:05.452019', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGqUEuwY9+9EBNkaevu7m35FNPHTYsDkt56L/lNtNkucMJO7ayXXpDBwc0aQFzoO2977/5RhvjOa3oPMsoYPmy8='} 2024-07-31T14:27:05.719 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-07-31_13:38:24-fs-wip-vshankar-testing-20240723.143739-debug-testing-default-smithi/7828591', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-31 14:22:05.453262', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJWO8yeotvlVnyCJBB3OYuwJKJ9uPyli5HmrS/P7rcZ75iTRfGt5vfuPrIElHofl+pcDW4HlqnxrLw2b78xKWp8='} 2024-07-31T14:27:05.739 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-07-31_13:38:24-fs-wip-vshankar-testing-20240723.143739-debug-testing-default-smithi/7828591', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-31 14:22:05.449471', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGaagvAkMGc/QXaGyjv4jc9+P44rESAAOf/bz6GWH1V1FA1var69vjizOZtrq4OrPnQvemZXCJjuj1ufXlMVC6I='} 2024-07-31T14:27:05.740 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-07-31T14:27:05.745 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-07-31T14:27:05.745 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-07-31T14:27:05.745 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-07-31T14:27:05.745 INFO:teuthology.run_tasks:Running task console_log... 2024-07-31T14:27:05.846 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f318140fac0>, signals=[15]) 2024-07-31T14:27:05.847 INFO:teuthology.run_tasks:Running task internal.connect... 2024-07-31T14:27:05.852 INFO:teuthology.task.internal:Opening connections... 2024-07-31T14:27:05.852 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2024-07-31T14:27:05.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-31T14:27:05.932 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi073.front.sepia.ceph.com 2024-07-31T14:27:05.933 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-31T14:27:06.004 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2024-07-31T14:27:06.005 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-31T14:27:06.077 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-07-31T14:27:06.083 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2024-07-31T14:27:06.101 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2024-07-31T14:27:06.102 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2024-07-31T14:27:06.159 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream" 2024-07-31T14:27:06.159 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="9" 2024-07-31T14:27:06.160 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos" 2024-07-31T14:27:06.160 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora" 2024-07-31T14:27:06.160 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="9" 2024-07-31T14:27:06.160 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el9" 2024-07-31T14:27:06.160 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-31T14:27:06.160 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2024-07-31T14:27:06.160 INFO:teuthology.orchestra.run.smithi067.stdout:LOGO="fedora-logo-icon" 2024-07-31T14:27:06.160 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-31T14:27:06.160 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/" 2024-07-31T14:27:06.160 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-31T14:27:06.160 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-31T14:27:06.160 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-31T14:27:06.161 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2024-07-31T14:27:06.197 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2024-07-31T14:27:06.215 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2024-07-31T14:27:06.216 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2024-07-31T14:27:06.273 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="CentOS Stream" 2024-07-31T14:27:06.273 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="9" 2024-07-31T14:27:06.273 INFO:teuthology.orchestra.run.smithi073.stdout:ID="centos" 2024-07-31T14:27:06.273 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="rhel fedora" 2024-07-31T14:27:06.273 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="9" 2024-07-31T14:27:06.273 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el9" 2024-07-31T14:27:06.273 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-31T14:27:06.273 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2024-07-31T14:27:06.274 INFO:teuthology.orchestra.run.smithi073.stdout:LOGO="fedora-logo-icon" 2024-07-31T14:27:06.274 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-31T14:27:06.274 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://centos.org/" 2024-07-31T14:27:06.274 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-31T14:27:06.274 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-31T14:27:06.274 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-31T14:27:06.274 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2024-07-31T14:27:06.362 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2024-07-31T14:27:06.380 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2024-07-31T14:27:06.380 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2024-07-31T14:27:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2024-07-31T14:27:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="9" 2024-07-31T14:27:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2024-07-31T14:27:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2024-07-31T14:27:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="9" 2024-07-31T14:27:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el9" 2024-07-31T14:27:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-31T14:27:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2024-07-31T14:27:06.438 INFO:teuthology.orchestra.run.smithi077.stdout:LOGO="fedora-logo-icon" 2024-07-31T14:27:06.438 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-31T14:27:06.438 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2024-07-31T14:27:06.438 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-31T14:27:06.438 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-31T14:27:06.438 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-31T14:27:06.438 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2024-07-31T14:27:06.455 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-07-31T14:27:06.466 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-07-31T14:27:06.490 INFO:teuthology.task.internal:Checking for old test directory... 2024-07-31T14:27:06.490 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2024-07-31T14:27:06.493 DEBUG:teuthology.orchestra.run.smithi073:> test '!' -e /home/ubuntu/cephtest 2024-07-31T14:27:06.495 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2024-07-31T14:27:06.511 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-07-31T14:27:06.517 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-07-31T14:27:06.517 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2024-07-31T14:27:06.550 DEBUG:teuthology.orchestra.run.smithi073:> test -z $(ls -A /var/lib/ceph) 2024-07-31T14:27:06.552 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2024-07-31T14:27:06.582 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-07-31T14:27:06.649 INFO:teuthology.run_tasks:Running task kernel... 2024-07-31T14:27:06.664 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': 1, 'sha1': '34096973aec0bfbe2bb86f34701acc6cd4133fd9'} 2024-07-31T14:27:06.665 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-07-31T14:27:06.665 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-07-31T14:27:06.665 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-31T14:27:06.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-31T14:27:06.682 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-07-31T14:27:06.739 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-437.el9.x86_64 2024-07-31T14:27:06.740 INFO:teuthology.task.kernel:Running kernel on smithi067: 5.14.0-437.el9.x86_64 2024-07-31T14:27:06.740 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum install -y kernel 2024-07-31T14:27:09.330 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.2 MB 00:00 2024-07-31T14:27:13.331 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 20 MB 00:02 2024-07-31T14:27:19.082 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.6 MB 00:01 2024-07-31T14:27:24.479 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 4.4 kB/s | 18 kB 00:04 2024-07-31T14:27:25.462 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 26 MB/s | 22 MB 00:00 2024-07-31T14:27:31.001 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-07-31T14:27:33.174 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-07-31T14:27:33.174 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-07-31T14:27:33.265 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-07-31T14:27:33.276 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-07-31T14:27:33.276 INFO:teuthology.orchestra.run.smithi067.stdout: Package Architecture Version Repository Size 2024-07-31T14:27:33.276 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-07-31T14:27:33.276 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-07-31T14:27:33.276 INFO:teuthology.orchestra.run.smithi067.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-07-31T14:27:33.276 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-07-31T14:27:33.276 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-core x86_64 5.14.0-480.el9 baseos 17 M 2024-07-31T14:27:33.277 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules x86_64 5.14.0-480.el9 baseos 36 M 2024-07-31T14:27:33.277 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-core x86_64 5.14.0-480.el9 baseos 30 M 2024-07-31T14:27:33.277 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:27:33.277 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-07-31T14:27:33.277 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-07-31T14:27:33.277 INFO:teuthology.orchestra.run.smithi067.stdout:Install 4 Packages 2024-07-31T14:27:33.277 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:27:33.278 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 84 M 2024-07-31T14:27:33.278 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 126 M 2024-07-31T14:27:33.278 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-07-31T14:27:34.528 INFO:teuthology.orchestra.run.smithi067.stdout:(1/4): kernel-5.14.0-480.el9.x86_64.rpm 1.6 MB/s | 1.5 MB 00:00 2024-07-31T14:27:35.036 INFO:teuthology.orchestra.run.smithi067.stdout:(2/4): kernel-core-5.14.0-480.el9.x86_64.rpm 12 MB/s | 17 MB 00:01 2024-07-31T14:27:35.811 INFO:teuthology.orchestra.run.smithi067.stdout:(3/4): kernel-modules-5.14.0-480.el9.x86_64.rpm 16 MB/s | 36 MB 00:02 2024-07-31T14:27:36.762 INFO:teuthology.orchestra.run.smithi067.stdout:(4/4): kernel-modules-core-5.14.0-480.el9.x86_6 13 MB/s | 30 MB 00:02 2024-07-31T14:27:36.762 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-07-31T14:27:36.762 INFO:teuthology.orchestra.run.smithi067.stdout:Total 24 MB/s | 84 MB 00:03 2024-07-31T14:27:37.529 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-07-31T14:27:37.967 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-07-31T14:27:37.968 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-07-31T14:27:39.007 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-07-31T14:27:39.008 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-07-31T14:27:41.589 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-07-31T14:27:42.270 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-modules-core-5.14.0-480.el9.x86_64 1/4 2024-07-31T14:27:42.361 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-core-5.14.0-480.el9.x86_64 2/4 2024-07-31T14:27:45.176 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-core-5.14.0-480.el9.x86_64 2/4 2024-07-31T14:27:45.330 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-07-31T14:27:50.535 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-07-31T14:27:50.590 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-5.14.0-480.el9.x86_64 4/4 2024-07-31T14:27:55.812 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-core-5.14.0-480.el9.x86_64 4/4 2024-07-31T14:28:27.084 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-core-5.14.0-480.el9.x86_64 4/4 2024-07-31T14:28:27.126 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-5.14.0-480.el9.x86_64 4/4 2024-07-31T14:28:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-5.14.0-480.el9.x86_64 4/4 2024-07-31T14:28:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 1/4 2024-07-31T14:28:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-core-5.14.0-480.el9.x86_64 2/4 2024-07-31T14:28:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-07-31T14:28:28.247 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-modules-core-5.14.0-480.el9.x86_64 4/4 2024-07-31T14:28:28.247 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:28:28.247 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-07-31T14:28:28.247 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-5.14.0-480.el9.x86_64 2024-07-31T14:28:28.247 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-core-5.14.0-480.el9.x86_64 2024-07-31T14:28:28.247 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-5.14.0-480.el9.x86_64 2024-07-31T14:28:28.247 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-core-5.14.0-480.el9.x86_64 2024-07-31T14:28:28.247 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:28:28.247 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-07-31T14:28:28.610 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel | sort -rV | head -n 1 2024-07-31T14:28:28.697 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-5.14.0-480.el9.x86_64 2024-07-31T14:28:28.697 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-07-31T14:28:28.697 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-480.el9.x86_64 2024-07-31T14:28:28.698 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-07-31T14:28:28.698 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-07-31T14:28:28.698 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-07-31T14:28:28.698 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi067.front.sepia.ceph.com, path=None, version=distro) 2024-07-31T14:28:28.698 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel | sort -rV | head -n 1 2024-07-31T14:28:28.756 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-5.14.0-480.el9.x86_64 2024-07-31T14:28:28.756 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-07-31T14:28:28.756 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -qi grub2-tools 2024-07-31T14:28:28.790 INFO:teuthology.orchestra.run.smithi067.stdout:Name : grub2-tools 2024-07-31T14:28:28.790 INFO:teuthology.orchestra.run.smithi067.stdout:Epoch : 1 2024-07-31T14:28:28.790 INFO:teuthology.orchestra.run.smithi067.stdout:Version : 2.06 2024-07-31T14:28:28.790 INFO:teuthology.orchestra.run.smithi067.stdout:Release : 68.el9 2024-07-31T14:28:28.790 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: x86_64 2024-07-31T14:28:28.790 INFO:teuthology.orchestra.run.smithi067.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-07-31T14:28:28.790 INFO:teuthology.orchestra.run.smithi067.stdout:Group : Unspecified 2024-07-31T14:28:28.790 INFO:teuthology.orchestra.run.smithi067.stdout:Size : 8264254 2024-07-31T14:28:28.790 INFO:teuthology.orchestra.run.smithi067.stdout:License : GPLv3+ 2024-07-31T14:28:28.791 INFO:teuthology.orchestra.run.smithi067.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-07-31T14:28:28.791 INFO:teuthology.orchestra.run.smithi067.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-07-31T14:28:28.791 INFO:teuthology.orchestra.run.smithi067.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-07-31T14:28:28.791 INFO:teuthology.orchestra.run.smithi067.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-07-31T14:28:28.791 INFO:teuthology.orchestra.run.smithi067.stdout:Packager : builder@centos.org 2024-07-31T14:28:28.791 INFO:teuthology.orchestra.run.smithi067.stdout:Vendor : CentOS 2024-07-31T14:28:28.791 INFO:teuthology.orchestra.run.smithi067.stdout:URL : http://www.gnu.org/software/grub/ 2024-07-31T14:28:28.791 INFO:teuthology.orchestra.run.smithi067.stdout:Summary : Support tools for GRUB. 2024-07-31T14:28:28.791 INFO:teuthology.orchestra.run.smithi067.stdout:Description : 2024-07-31T14:28:28.791 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:28:28.791 INFO:teuthology.orchestra.run.smithi067.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-07-31T14:28:28.792 INFO:teuthology.orchestra.run.smithi067.stdout:customizable bootloader with modular architecture. It supports a rich 2024-07-31T14:28:28.792 INFO:teuthology.orchestra.run.smithi067.stdout:variety of kernel formats, file systems, computer architectures and 2024-07-31T14:28:28.792 INFO:teuthology.orchestra.run.smithi067.stdout:hardware devices. 2024-07-31T14:28:28.792 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:28:28.792 INFO:teuthology.orchestra.run.smithi067.stdout:This subpackage provides tools for support of all platforms. 2024-07-31T14:28:28.793 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-07-31T14:28:28.793 INFO:teuthology.task.kernel:Updating grub on smithi067 to boot 5.14.0-480.el9.x86_64 2024-07-31T14:28:28.793 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-31T14:28:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:Generating grub configuration file ... 2024-07-31T14:28:30.941 INFO:teuthology.orchestra.run.smithi067.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-07-31T14:28:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:done 2024-07-31T14:28:30.978 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-07-31T14:28:30.993 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.zl7rq0mx8M 2024-07-31T14:28:30.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zl7rq0mx8M 2024-07-31T14:28:31.059 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.zl7rq0mx8M 2024-07-31T14:28:31.229 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.zl7rq0mx8M is 6KB 2024-07-31T14:28:31.249 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.zl7rq0mx8M 2024-07-31T14:28:31.264 DEBUG:teuthology.orchestra.run.smithi067:> sudo /bin/ls /boot/loader/entries || true 2024-07-31T14:28:31.326 INFO:teuthology.orchestra.run.smithi067.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-07-31T14:28:31.327 INFO:teuthology.orchestra.run.smithi067.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-07-31T14:28:31.327 INFO:teuthology.orchestra.run.smithi067.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-07-31T14:28:31.327 INFO:teuthology.orchestra.run.smithi067.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-480.el9.x86_64.conf 2024-07-31T14:28:31.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-480.el9.x86_64 2024-07-31T14:28:31.463 DEBUG:teuthology.orchestra.run.smithi067:> sudo shutdown -r now 2024-07-31T14:29:01.466 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-07-31T14:29:01.466 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-07-31T14:29:01.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-31T14:29:19.967 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-07-31T14:29:28.967 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-07-31T14:29:28.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-31T14:29:32.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-07-31T14:29:44.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-07-31T14:29:44.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-31T14:30:02.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-07-31T14:30:17.464 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-07-31T14:30:17.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-31T14:30:17.791 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-07-31T14:30:18.204 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-07-31T14:30:18.204 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-480.el9.x86_64"... 2024-07-31T14:30:18.204 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-07-31T14:30:18.219 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-480.el9.x86_64 2024-07-31T14:30:18.220 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-480.el9.x86_64 vs 5.14.0-480.el9.x86_64 2024-07-31T14:30:18.220 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-07-31T14:30:18.220 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-07-31T14:30:19.221 DEBUG:teuthology.parallel:result is None 2024-07-31T14:30:19.222 INFO:teuthology.run_tasks:Running task internal.base... 2024-07-31T14:30:19.231 INFO:teuthology.task.internal:Creating test directory... 2024-07-31T14:30:19.231 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-31T14:30:19.235 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-31T14:30:19.238 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-31T14:30:19.257 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-07-31T14:30:19.265 INFO:teuthology.run_tasks:Running task internal.archive... 2024-07-31T14:30:19.293 INFO:teuthology.task.internal:Creating archive directory... 2024-07-31T14:30:19.293 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-31T14:30:19.296 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-31T14:30:19.298 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-31T14:30:19.475 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-07-31T14:30:19.482 INFO:teuthology.task.internal:Enabling coredump saving... 2024-07-31T14:30:19.482 DEBUG:teuthology.orchestra.run.smithi073:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-31T14:30:19.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-31T14:30:19.499 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-31T14:30:19.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-31T14:30:19.516 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-31T14:30:19.518 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-31T14:30:19.542 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-31T14:30:19.573 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-31T14:30:19.588 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-31T14:30:19.593 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-31T14:30:19.611 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-31T14:30:19.613 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-31T14:30:19.626 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-31T14:30:19.628 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-07-31T14:30:19.635 INFO:teuthology.task.internal:Configuring sudo... 2024-07-31T14:30:19.635 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-31T14:30:19.637 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-31T14:30:19.656 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-31T14:30:19.695 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-07-31T14:30:19.704 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-07-31T14:30:19.705 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-31T14:30:19.707 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-31T14:30:19.722 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-31T14:30:19.751 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-31T14:30:19.805 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-31T14:30:19.853 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-31T14:30:19.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-31T14:30:19.978 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:30:19.978 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-31T14:30:20.065 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-31T14:30:20.111 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-31T14:30:20.165 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-31T14:30:20.248 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-31T14:30:20.291 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-07-31T14:30:20.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-31T14:30:20.355 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-31T14:30:20.396 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-31T14:30:20.450 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-31T14:30:20.533 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-31T14:30:20.576 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-07-31T14:30:20.576 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-31T14:30:20.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2024-07-31T14:30:20.644 DEBUG:teuthology.orchestra.run.smithi073:> sudo service rsyslog restart 2024-07-31T14:30:20.647 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2024-07-31T14:30:20.677 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-31T14:30:20.689 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-31T14:30:20.725 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-31T14:30:21.180 INFO:teuthology.run_tasks:Running task internal.timer... 2024-07-31T14:30:21.196 INFO:teuthology.task.internal:Starting timer... 2024-07-31T14:30:21.196 INFO:teuthology.run_tasks:Running task pcp... 2024-07-31T14:30:21.247 INFO:teuthology.run_tasks:Running task selinux... 2024-07-31T14:30:21.278 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-07-31T14:30:21.278 DEBUG:teuthology.orchestra.run.smithi067:> sudo service auditd rotate 2024-07-31T14:30:21.389 INFO:teuthology.orchestra.run.smithi067.stdout:Rotating logs: 2024-07-31T14:30:21.391 DEBUG:teuthology.orchestra.run.smithi073:> sudo service auditd rotate 2024-07-31T14:30:21.460 INFO:teuthology.orchestra.run.smithi073.stdout:Rotating logs: 2024-07-31T14:30:21.462 DEBUG:teuthology.orchestra.run.smithi077:> sudo service auditd rotate 2024-07-31T14:30:21.507 INFO:teuthology.orchestra.run.smithi077.stdout:Rotating logs: 2024-07-31T14:30:21.509 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-07-31T14:30:21.510 DEBUG:teuthology.orchestra.run.smithi067:> /usr/sbin/getenforce 2024-07-31T14:30:21.533 INFO:teuthology.orchestra.run.smithi067.stdout:Permissive 2024-07-31T14:30:21.533 DEBUG:teuthology.orchestra.run.smithi073:> /usr/sbin/getenforce 2024-07-31T14:30:21.564 INFO:teuthology.orchestra.run.smithi073.stdout:Permissive 2024-07-31T14:30:21.565 DEBUG:teuthology.orchestra.run.smithi077:> /usr/sbin/getenforce 2024-07-31T14:30:21.590 INFO:teuthology.orchestra.run.smithi077.stdout:Permissive 2024-07-31T14:30:21.590 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi067.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi073.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi077.front.sepia.ceph.com': 'permissive'} 2024-07-31T14:30:21.591 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-31T14:30:21.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-31T14:30:21.617 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-31T14:30:21.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-31T14:30:21.647 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-31T14:30:21.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-31T14:30:21.673 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-07-31T14:30:21.673 DEBUG:teuthology.orchestra.run.smithi067:> sudo /usr/sbin/setenforce permissive 2024-07-31T14:30:21.714 DEBUG:teuthology.orchestra.run.smithi073:> sudo /usr/sbin/setenforce permissive 2024-07-31T14:30:21.750 DEBUG:teuthology.orchestra.run.smithi077:> sudo /usr/sbin/setenforce permissive 2024-07-31T14:30:21.778 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-07-31T14:30:21.787 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-07-31T14:30:21.790 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-07-31T14:30:21.924 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-07-31T14:30:21.935 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-07-31T14:30:21.936 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi067.front.sepia.ceph.com,smithi073.front.sepia.ceph.com,smithi077.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-07-31T14:35:11.483 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi067.front.sepia.ceph.com'), Remote(name='ubuntu@smithi073.front.sepia.ceph.com'), Remote(name='ubuntu@smithi077.front.sepia.ceph.com')] 2024-07-31T14:35:11.485 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-07-31T14:35:11.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-31T14:35:11.561 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-07-31T14:35:11.642 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-07-31T14:35:11.643 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-07-31T14:35:11.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-31T14:35:11.716 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-07-31T14:35:11.799 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-07-31T14:35:11.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-07-31T14:35:11.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-31T14:35:11.879 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-07-31T14:35:11.957 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-07-31T14:35:11.958 INFO:teuthology.run_tasks:Running task clock... 2024-07-31T14:35:11.975 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-07-31T14:35:11.975 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-31T14:35:11.976 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-31T14:35:11.978 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-31T14:35:11.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-31T14:35:11.981 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-31T14:35:11.981 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-31T14:35:12.007 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-31T14:35:12.011 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-31T14:35:12.022 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-31T14:35:12.024 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-31T14:35:12.025 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-31T14:35:12.037 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-31T14:35:12.048 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: ntpd: command not found 2024-07-31T14:35:12.050 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: ntpd: command not found 2024-07-31T14:35:12.060 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: ntpd: command not found 2024-07-31T14:35:12.060 INFO:teuthology.orchestra.run.smithi067.stdout:506 Cannot talk to daemon 2024-07-31T14:35:12.061 INFO:teuthology.orchestra.run.smithi073.stdout:506 Cannot talk to daemon 2024-07-31T14:35:12.070 INFO:teuthology.orchestra.run.smithi077.stdout:506 Cannot talk to daemon 2024-07-31T14:35:12.075 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-31T14:35:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-31T14:35:12.083 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-31T14:35:12.089 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-31T14:35:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-31T14:35:12.096 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-31T14:35:12.150 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-07-31T14:35:12.153 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-31T14:35:12.153 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-07-31T14:35:12.154 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T14:35:12.154 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T14:35:12.154 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T14:35:12.154 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T14:35:12.154 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-07-31T14:35:12.158 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-31T14:35:12.158 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-07-31T14:35:12.158 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T14:35:12.158 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T14:35:12.158 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T14:35:12.158 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T14:35:12.160 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-07-31T14:35:12.164 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-31T14:35:12.165 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-07-31T14:35:12.165 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T14:35:12.165 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T14:35:12.165 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T14:35:12.165 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T14:35:12.166 INFO:teuthology.run_tasks:Running task install... 2024-07-31T14:35:12.178 DEBUG:teuthology.task.install:project ceph 2024-07-31T14:35:12.178 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8c796fca5ff5960f7542e7227bad4a784ee2c804'}} 2024-07-31T14:35:12.178 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8c796fca5ff5960f7542e7227bad4a784ee2c804'} 2024-07-31T14:35:12.179 INFO:teuthology.task.install:Using flavor: default 2024-07-31T14:35:12.186 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-07-31T14:35:12.186 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-07-31T14:35:12.187 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8c796fca5ff5960f7542e7227bad4a784ee2c804', 'tag': None, 'wait_for_package': False} 2024-07-31T14:35:12.187 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8c796fca5ff5960f7542e7227bad4a784ee2c804 2024-07-31T14:35:12.190 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8c796fca5ff5960f7542e7227bad4a784ee2c804', 'tag': None, 'wait_for_package': False} 2024-07-31T14:35:12.190 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8c796fca5ff5960f7542e7227bad4a784ee2c804 2024-07-31T14:35:12.191 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8c796fca5ff5960f7542e7227bad4a784ee2c804', 'tag': None, 'wait_for_package': False} 2024-07-31T14:35:12.191 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8c796fca5ff5960f7542e7227bad4a784ee2c804 2024-07-31T14:35:12.348 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/ 2024-07-31T14:35:12.348 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5223.g8c796fca 2024-07-31T14:35:12.352 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/ 2024-07-31T14:35:12.352 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5223.g8c796fca 2024-07-31T14:35:12.363 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/ 2024-07-31T14:35:12.363 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5223.g8c796fca 2024-07-31T14:35:12.467 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-31T14:35:12.468 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-07-31T14:35:12.468 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-31T14:35:12.480 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-31T14:35:12.481 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-07-31T14:35:12.481 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-31T14:35:12.491 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-31T14:35:12.491 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:35:12.491 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-31T14:35:12.493 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-31T14:35:12.493 DEBUG:teuthology.orchestra.run.smithi077:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8c796fca5ff5960f7542e7227bad4a784ee2c804/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-31T14:35:12.505 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-31T14:35:12.505 DEBUG:teuthology.orchestra.run.smithi073:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8c796fca5ff5960f7542e7227bad4a784ee2c804/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-31T14:35:12.518 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-31T14:35:12.518 DEBUG:teuthology.orchestra.run.smithi067:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8c796fca5ff5960f7542e7227bad4a784ee2c804/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-31T14:35:12.556 DEBUG:teuthology.orchestra.run.smithi077:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-31T14:35:12.569 DEBUG:teuthology.orchestra.run.smithi073:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-31T14:35:12.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-31T14:35:12.628 DEBUG:teuthology.orchestra.run.smithi077:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-31T14:35:12.642 DEBUG:teuthology.orchestra.run.smithi073:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-31T14:35:12.662 DEBUG:teuthology.orchestra.run.smithi067:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-31T14:35:12.691 INFO:teuthology.orchestra.run.smithi077.stdout:check_obsoletes = 1 2024-07-31T14:35:12.692 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum clean all 2024-07-31T14:35:12.704 INFO:teuthology.orchestra.run.smithi073.stdout:check_obsoletes = 1 2024-07-31T14:35:12.706 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum clean all 2024-07-31T14:35:12.726 INFO:teuthology.orchestra.run.smithi067.stdout:check_obsoletes = 1 2024-07-31T14:35:12.728 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum clean all 2024-07-31T14:35:12.915 INFO:teuthology.orchestra.run.smithi077.stdout:45 files removed 2024-07-31T14:35:12.928 INFO:teuthology.orchestra.run.smithi073.stdout:45 files removed 2024-07-31T14:35:12.942 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-31T14:35:12.955 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-31T14:35:12.971 INFO:teuthology.orchestra.run.smithi067.stdout:45 files removed 2024-07-31T14:35:12.998 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-31T14:35:13.526 INFO:teuthology.orchestra.run.smithi077.stdout:ceph packages for x86_64 257 kB/s | 83 kB 00:00 2024-07-31T14:35:13.538 INFO:teuthology.orchestra.run.smithi073.stdout:ceph packages for x86_64 258 kB/s | 83 kB 00:00 2024-07-31T14:35:13.621 INFO:teuthology.orchestra.run.smithi067.stdout:ceph packages for x86_64 253 kB/s | 83 kB 00:00 2024-07-31T14:35:13.869 INFO:teuthology.orchestra.run.smithi077.stdout:ceph noarch packages 80 kB/s | 24 kB 00:00 2024-07-31T14:35:13.879 INFO:teuthology.orchestra.run.smithi073.stdout:ceph noarch packages 80 kB/s | 24 kB 00:00 2024-07-31T14:35:13.988 INFO:teuthology.orchestra.run.smithi067.stdout:ceph noarch packages 77 kB/s | 24 kB 00:00 2024-07-31T14:35:14.195 INFO:teuthology.orchestra.run.smithi077.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-07-31T14:35:14.214 INFO:teuthology.orchestra.run.smithi073.stdout:ceph source packages 6.4 kB/s | 1.9 kB 00:00 2024-07-31T14:35:14.339 INFO:teuthology.orchestra.run.smithi067.stdout:ceph source packages 6.2 kB/s | 1.9 kB 00:00 2024-07-31T14:35:15.858 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.2 MB 00:01 2024-07-31T14:35:16.131 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-07-31T14:35:16.153 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.2 MB 00:01 2024-07-31T14:35:19.399 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 20 MB 00:02 2024-07-31T14:35:20.279 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 20 MB 00:03 2024-07-31T14:35:20.681 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 20 MB 00:03 2024-07-31T14:35:25.341 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.6 MB 00:02 2024-07-31T14:35:25.688 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.6 MB 00:01 2024-07-31T14:35:25.932 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.6 MB 00:01 2024-07-31T14:35:27.376 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 18 kB 00:00 2024-07-31T14:35:28.337 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 27 MB/s | 22 MB 00:00 2024-07-31T14:35:28.385 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 18 kB 00:01 2024-07-31T14:35:29.295 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 28 MB/s | 22 MB 00:00 2024-07-31T14:35:29.752 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - Extras packages 6.4 kB/s | 18 kB 00:02 2024-07-31T14:35:30.653 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 29 MB/s | 22 MB 00:00 2024-07-31T14:35:33.893 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-07-31T14:35:34.585 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-07-31T14:35:36.076 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-07-31T14:35:36.329 INFO:teuthology.orchestra.run.smithi067.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-31T14:35:36.329 INFO:teuthology.orchestra.run.smithi067.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-31T14:35:36.341 INFO:teuthology.orchestra.run.smithi067.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-31T14:35:36.342 INFO:teuthology.orchestra.run.smithi067.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-31T14:35:36.342 INFO:teuthology.orchestra.run.smithi067.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-31T14:35:36.343 INFO:teuthology.orchestra.run.smithi067.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-31T14:35:36.345 INFO:teuthology.orchestra.run.smithi067.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-31T14:35:36.347 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-31T14:35:36.348 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-31T14:35:36.417 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-07-31T14:35:36.438 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-07-31T14:35:36.438 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-07-31T14:35:36.438 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-07-31T14:35:36.438 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-07-31T14:35:36.439 INFO:teuthology.orchestra.run.smithi067.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-31T14:35:36.439 INFO:teuthology.orchestra.run.smithi067.stdout: ceph x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 6.6 k 2024-07-31T14:35:36.439 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 5.5 M 2024-07-31T14:35:36.439 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 875 k 2024-07-31T14:35:36.439 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 148 k 2024-07-31T14:35:36.439 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 1.5 M 2024-07-31T14:35:36.439 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 149 k 2024-07-31T14:35:36.439 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 14 M 2024-07-31T14:35:36.439 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 7.4 M 2024-07-31T14:35:36.439 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 49 k 2024-07-31T14:35:36.439 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 10 M 2024-07-31T14:35:36.440 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 44 M 2024-07-31T14:35:36.440 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 274 k 2024-07-31T14:35:36.440 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 770 k 2024-07-31T14:35:36.440 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 228 k 2024-07-31T14:35:36.440 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-shell x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 27 k 2024-07-31T14:35:36.440 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 18 k 2024-07-31T14:35:36.440 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-31T14:35:36.440 INFO:teuthology.orchestra.run.smithi067.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-31T14:35:36.440 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-31T14:35:36.440 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 32 k 2024-07-31T14:35:36.440 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 768 k 2024-07-31T14:35:36.441 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 127 k 2024-07-31T14:35:36.441 INFO:teuthology.orchestra.run.smithi067.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-31T14:35:36.441 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-31T14:35:36.441 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-31T14:35:36.441 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-31T14:35:36.441 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-31T14:35:36.441 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-31T14:35:36.441 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 162 k 2024-07-31T14:35:36.441 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 321 k 2024-07-31T14:35:36.441 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 303 k 2024-07-31T14:35:36.441 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 99 k 2024-07-31T14:35:36.441 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 86 k 2024-07-31T14:35:36.442 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 3.1 M 2024-07-31T14:35:36.442 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 170 k 2024-07-31T14:35:36.442 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-31T14:35:36.442 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2024-07-31T14:35:36.442 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-31T14:35:36.442 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-31T14:35:36.442 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-31T14:35:36.442 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-31T14:35:36.442 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-31T14:35:36.442 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-31T14:35:36.442 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-31T14:35:36.442 INFO:teuthology.orchestra.run.smithi067.stdout: librados2 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 3.4 M 2024-07-31T14:35:36.443 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 3.1 M 2024-07-31T14:35:36.443 INFO:teuthology.orchestra.run.smithi067.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-31T14:35:36.443 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-31T14:35:36.443 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-31T14:35:36.443 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-07-31T14:35:36.443 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-31T14:35:36.443 INFO:teuthology.orchestra.run.smithi067.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-31T14:35:36.443 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-31T14:35:36.443 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 20 M 2024-07-31T14:35:36.443 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 34 k 2024-07-31T14:35:36.443 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 2.3 M 2024-07-31T14:35:36.443 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 284 k 2024-07-31T14:35:36.444 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 4.9 M 2024-07-31T14:35:36.444 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 18 M 2024-07-31T14:35:36.444 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 16 k 2024-07-31T14:35:36.444 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 25 k 2024-07-31T14:35:36.444 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-31T14:35:36.444 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-31T14:35:36.444 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-31T14:35:36.444 INFO:teuthology.orchestra.run.smithi067.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-31T14:35:36.444 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-31T14:35:36.444 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-31T14:35:36.444 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-31T14:35:36.444 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-31T14:35:36.445 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 158 k 2024-07-31T14:35:36.445 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-31T14:35:36.445 INFO:teuthology.orchestra.run.smithi067.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-31T14:35:36.445 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-31T14:35:36.445 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-31T14:35:36.445 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-31T14:35:36.445 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 506 k 2024-07-31T14:35:36.445 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-31T14:35:36.445 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 5.3 M 2024-07-31T14:35:36.445 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-31T14:35:36.445 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-31T14:35:36.445 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-31T14:35:36.446 INFO:teuthology.orchestra.run.smithi067.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-31T14:35:36.446 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-31T14:35:36.446 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-31T14:35:36.446 INFO:teuthology.orchestra.run.smithi067.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-31T14:35:36.446 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-31T14:35:36.446 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-31T14:35:36.446 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-31T14:35:36.446 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-31T14:35:36.446 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-31T14:35:36.446 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-31T14:35:36.446 INFO:teuthology.orchestra.run.smithi067.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-31T14:35:36.446 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-31T14:35:36.446 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-31T14:35:36.447 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-31T14:35:36.447 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 45 k 2024-07-31T14:35:36.447 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 138 k 2024-07-31T14:35:36.447 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-31T14:35:36.447 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-31T14:35:36.447 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-31T14:35:36.447 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-31T14:35:36.447 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-31T14:35:36.447 INFO:teuthology.orchestra.run.smithi067.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-31T14:35:36.447 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-31T14:35:36.447 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-31T14:35:36.447 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-31T14:35:36.447 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-31T14:35:36.448 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-31T14:35:36.448 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-31T14:35:36.448 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-31T14:35:36.448 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-31T14:35:36.448 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-31T14:35:36.448 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-31T14:35:36.448 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-31T14:35:36.448 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-31T14:35:36.448 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-31T14:35:36.448 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-31T14:35:36.448 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-31T14:35:36.448 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-31T14:35:36.449 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-31T14:35:36.449 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-31T14:35:36.449 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-31T14:35:36.449 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-31T14:35:36.449 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-31T14:35:36.449 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-31T14:35:36.449 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-31T14:35:36.449 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-31T14:35:36.449 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-31T14:35:36.449 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-31T14:35:36.449 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-31T14:35:36.449 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-31T14:35:36.449 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-31T14:35:36.450 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-31T14:35:36.450 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-31T14:35:36.450 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-31T14:35:36.450 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-31T14:35:36.450 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-31T14:35:36.450 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-31T14:35:36.450 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-31T14:35:36.450 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-31T14:35:36.450 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-31T14:35:36.450 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-31T14:35:36.450 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-31T14:35:36.450 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-31T14:35:36.450 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-31T14:35:36.451 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-07-31T14:35:36.451 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-31T14:35:36.451 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-31T14:35:36.451 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-31T14:35:36.451 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-31T14:35:36.451 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-31T14:35:36.451 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-31T14:35:36.451 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-31T14:35:36.451 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-31T14:35:36.451 INFO:teuthology.orchestra.run.smithi067.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-31T14:35:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-31T14:35:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-31T14:35:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-31T14:35:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-31T14:35:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-31T14:35:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:Installing weak dependencies: 2024-07-31T14:35:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-31T14:35:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:35:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-07-31T14:35:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-07-31T14:35:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:Install 148 Packages 2024-07-31T14:35:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 12 Packages 2024-07-31T14:35:36.453 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:35:36.453 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 232 M 2024-07-31T14:35:36.453 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-07-31T14:35:36.878 INFO:teuthology.orchestra.run.smithi073.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-31T14:35:36.879 INFO:teuthology.orchestra.run.smithi073.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-31T14:35:36.890 INFO:teuthology.orchestra.run.smithi073.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-31T14:35:36.891 INFO:teuthology.orchestra.run.smithi073.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-31T14:35:36.891 INFO:teuthology.orchestra.run.smithi073.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-31T14:35:36.892 INFO:teuthology.orchestra.run.smithi073.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-31T14:35:36.894 INFO:teuthology.orchestra.run.smithi073.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-31T14:35:36.896 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-31T14:35:36.897 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-31T14:35:36.962 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-07-31T14:35:36.987 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-07-31T14:35:36.988 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-07-31T14:35:36.988 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-07-31T14:35:36.988 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-07-31T14:35:36.988 INFO:teuthology.orchestra.run.smithi073.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-31T14:35:36.988 INFO:teuthology.orchestra.run.smithi073.stdout: ceph x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 6.6 k 2024-07-31T14:35:36.988 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 5.5 M 2024-07-31T14:35:36.988 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 875 k 2024-07-31T14:35:36.988 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 148 k 2024-07-31T14:35:36.988 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 1.5 M 2024-07-31T14:35:36.988 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 149 k 2024-07-31T14:35:36.988 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 14 M 2024-07-31T14:35:36.989 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 7.4 M 2024-07-31T14:35:36.989 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 49 k 2024-07-31T14:35:36.989 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 10 M 2024-07-31T14:35:36.989 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 44 M 2024-07-31T14:35:36.989 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 274 k 2024-07-31T14:35:36.989 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 770 k 2024-07-31T14:35:36.989 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 228 k 2024-07-31T14:35:36.989 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-shell x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 27 k 2024-07-31T14:35:36.989 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 18 k 2024-07-31T14:35:36.989 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-31T14:35:36.990 INFO:teuthology.orchestra.run.smithi073.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-31T14:35:36.990 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-31T14:35:36.990 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 32 k 2024-07-31T14:35:36.990 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 768 k 2024-07-31T14:35:36.990 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 127 k 2024-07-31T14:35:36.990 INFO:teuthology.orchestra.run.smithi073.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-31T14:35:36.990 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-31T14:35:36.990 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-31T14:35:36.990 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-31T14:35:36.991 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-31T14:35:36.991 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-31T14:35:36.991 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 162 k 2024-07-31T14:35:36.991 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 321 k 2024-07-31T14:35:36.991 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 303 k 2024-07-31T14:35:36.991 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 99 k 2024-07-31T14:35:36.991 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 86 k 2024-07-31T14:35:36.991 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 3.1 M 2024-07-31T14:35:36.991 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 170 k 2024-07-31T14:35:36.991 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-31T14:35:36.991 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrading: 2024-07-31T14:35:36.992 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-31T14:35:36.992 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-31T14:35:36.992 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-31T14:35:36.992 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-31T14:35:36.992 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-31T14:35:36.992 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-31T14:35:36.992 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-31T14:35:36.992 INFO:teuthology.orchestra.run.smithi073.stdout: librados2 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 3.4 M 2024-07-31T14:35:36.992 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 3.1 M 2024-07-31T14:35:36.992 INFO:teuthology.orchestra.run.smithi073.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-31T14:35:36.992 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-31T14:35:36.993 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-31T14:35:36.993 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-07-31T14:35:36.993 INFO:teuthology.orchestra.run.smithi073.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-31T14:35:36.993 INFO:teuthology.orchestra.run.smithi073.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-31T14:35:36.993 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-31T14:35:36.993 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 20 M 2024-07-31T14:35:36.993 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 34 k 2024-07-31T14:35:36.993 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 2.3 M 2024-07-31T14:35:36.993 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 284 k 2024-07-31T14:35:36.993 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 4.9 M 2024-07-31T14:35:36.993 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 18 M 2024-07-31T14:35:36.993 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 16 k 2024-07-31T14:35:36.994 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 25 k 2024-07-31T14:35:36.994 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-31T14:35:36.994 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-31T14:35:36.994 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-31T14:35:36.994 INFO:teuthology.orchestra.run.smithi073.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-31T14:35:36.994 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-31T14:35:36.994 INFO:teuthology.orchestra.run.smithi073.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-31T14:35:36.994 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-31T14:35:36.994 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-31T14:35:36.994 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 158 k 2024-07-31T14:35:36.994 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-31T14:35:36.994 INFO:teuthology.orchestra.run.smithi073.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-31T14:35:36.994 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-31T14:35:36.995 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-31T14:35:36.995 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-31T14:35:36.995 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 506 k 2024-07-31T14:35:36.995 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-31T14:35:36.995 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 5.3 M 2024-07-31T14:35:36.995 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-31T14:35:36.995 INFO:teuthology.orchestra.run.smithi073.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-31T14:35:36.995 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-31T14:35:36.995 INFO:teuthology.orchestra.run.smithi073.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-31T14:35:36.995 INFO:teuthology.orchestra.run.smithi073.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-31T14:35:36.995 INFO:teuthology.orchestra.run.smithi073.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-31T14:35:36.995 INFO:teuthology.orchestra.run.smithi073.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-31T14:35:36.995 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-31T14:35:36.996 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-31T14:35:36.996 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-31T14:35:36.996 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-31T14:35:36.996 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-31T14:35:36.996 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-31T14:35:36.996 INFO:teuthology.orchestra.run.smithi073.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-31T14:35:36.996 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-31T14:35:36.996 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-31T14:35:36.996 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-31T14:35:36.996 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 45 k 2024-07-31T14:35:36.996 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 138 k 2024-07-31T14:35:36.996 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-31T14:35:36.996 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-31T14:35:36.997 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-31T14:35:36.997 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-31T14:35:36.997 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-31T14:35:36.997 INFO:teuthology.orchestra.run.smithi073.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-31T14:35:36.997 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-31T14:35:36.997 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-31T14:35:36.997 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-31T14:35:36.997 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-31T14:35:36.997 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-31T14:35:36.997 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-31T14:35:36.997 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-31T14:35:36.997 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-31T14:35:36.998 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-31T14:35:36.998 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-31T14:35:36.998 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-31T14:35:36.998 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-31T14:35:36.998 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-31T14:35:36.998 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-31T14:35:36.998 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-31T14:35:36.998 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-31T14:35:36.998 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-31T14:35:36.998 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-31T14:35:36.998 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-31T14:35:36.998 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-31T14:35:36.998 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-31T14:35:36.999 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-31T14:35:36.999 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-31T14:35:36.999 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-31T14:35:36.999 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-31T14:35:36.999 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-31T14:35:36.999 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-31T14:35:36.999 INFO:teuthology.orchestra.run.smithi073.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-31T14:35:36.999 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-31T14:35:36.999 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-31T14:35:36.999 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-31T14:35:36.999 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-31T14:35:37.000 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-31T14:35:37.000 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-31T14:35:37.000 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-31T14:35:37.000 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-31T14:35:37.000 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-31T14:35:37.000 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-31T14:35:37.000 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-31T14:35:37.000 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-31T14:35:37.000 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-31T14:35:37.000 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-31T14:35:37.000 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-07-31T14:35:37.000 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-31T14:35:37.000 INFO:teuthology.orchestra.run.smithi073.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-31T14:35:37.000 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-31T14:35:37.001 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-31T14:35:37.001 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-31T14:35:37.001 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-31T14:35:37.001 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-31T14:35:37.001 INFO:teuthology.orchestra.run.smithi073.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-31T14:35:37.001 INFO:teuthology.orchestra.run.smithi073.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-31T14:35:37.001 INFO:teuthology.orchestra.run.smithi073.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-31T14:35:37.001 INFO:teuthology.orchestra.run.smithi073.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-31T14:35:37.001 INFO:teuthology.orchestra.run.smithi073.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-31T14:35:37.001 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-31T14:35:37.001 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-31T14:35:37.001 INFO:teuthology.orchestra.run.smithi073.stdout:Installing weak dependencies: 2024-07-31T14:35:37.001 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-31T14:35:37.002 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:35:37.002 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-07-31T14:35:37.002 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-07-31T14:35:37.002 INFO:teuthology.orchestra.run.smithi073.stdout:Install 148 Packages 2024-07-31T14:35:37.002 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrade 12 Packages 2024-07-31T14:35:37.002 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:35:37.002 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 232 M 2024-07-31T14:35:37.002 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-07-31T14:35:37.020 INFO:teuthology.orchestra.run.smithi067.stdout:(1/160): ceph-19.0.0-5223.g8c796fca.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-07-31T14:35:37.287 INFO:teuthology.orchestra.run.smithi067.stdout:(2/160): ceph-base-19.0.0-5223.g8c796fca.el9.x8 12 MB/s | 5.5 MB 00:00 2024-07-31T14:35:37.320 INFO:teuthology.orchestra.run.smithi067.stdout:(3/160): ceph-fuse-19.0.0-5223.g8c796fca.el9.x8 2.8 MB/s | 875 kB 00:00 2024-07-31T14:35:37.345 INFO:teuthology.orchestra.run.smithi067.stdout:(4/160): ceph-immutable-object-cache-19.0.0-522 2.5 MB/s | 148 kB 00:00 2024-07-31T14:35:37.446 INFO:teuthology.orchestra.run.smithi067.stdout:(5/160): ceph-mgr-19.0.0-5223.g8c796fca.el9.x86 15 MB/s | 1.5 MB 00:00 2024-07-31T14:35:37.588 INFO:teuthology.orchestra.run.smithi067.stdout:(6/160): ceph-mds-19.0.0-5223.g8c796fca.el9.x86 8.5 MB/s | 2.3 MB 00:00 2024-07-31T14:35:37.680 INFO:teuthology.orchestra.run.smithi067.stdout:(7/160): ceph-mon-19.0.0-5223.g8c796fca.el9.x86 21 MB/s | 4.9 MB 00:00 2024-07-31T14:35:37.686 INFO:teuthology.orchestra.run.smithi073.stdout:(1/160): ceph-19.0.0-5223.g8c796fca.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-07-31T14:35:38.080 INFO:teuthology.orchestra.run.smithi067.stdout:(8/160): ceph-common-19.0.0-5223.g8c796fca.el9. 16 MB/s | 20 MB 00:01 2024-07-31T14:35:38.122 INFO:teuthology.orchestra.run.smithi067.stdout:(9/160): ceph-selinux-19.0.0-5223.g8c796fca.el9 607 kB/s | 25 kB 00:00 2024-07-31T14:35:38.127 INFO:teuthology.orchestra.run.smithi073.stdout:(2/160): ceph-fuse-19.0.0-5223.g8c796fca.el9.x8 1.9 MB/s | 875 kB 00:00 2024-07-31T14:35:38.211 INFO:teuthology.orchestra.run.smithi073.stdout:(3/160): ceph-immutable-object-cache-19.0.0-522 1.7 MB/s | 148 kB 00:00 2024-07-31T14:35:38.239 INFO:teuthology.orchestra.run.smithi067.stdout:(10/160): ceph-radosgw-19.0.0-5223.g8c796fca.el 19 MB/s | 10 MB 00:00 2024-07-31T14:35:38.306 INFO:teuthology.orchestra.run.smithi067.stdout:(11/160): cephfs-mirror-19.0.0-5223.g8c796fca.e 3.3 MB/s | 228 kB 00:00 2024-07-31T14:35:38.343 INFO:teuthology.orchestra.run.smithi077.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-31T14:35:38.344 INFO:teuthology.orchestra.run.smithi077.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-31T14:35:38.355 INFO:teuthology.orchestra.run.smithi077.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-31T14:35:38.355 INFO:teuthology.orchestra.run.smithi077.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-31T14:35:38.356 INFO:teuthology.orchestra.run.smithi077.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-31T14:35:38.357 INFO:teuthology.orchestra.run.smithi077.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-31T14:35:38.359 INFO:teuthology.orchestra.run.smithi077.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-31T14:35:38.361 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-31T14:35:38.362 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-31T14:35:38.372 INFO:teuthology.orchestra.run.smithi067.stdout:(12/160): cephfs-shell-19.0.0-5223.g8c796fca.el 406 kB/s | 27 kB 00:00 2024-07-31T14:35:38.426 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-07-31T14:35:38.439 INFO:teuthology.orchestra.run.smithi067.stdout:(13/160): libcephfs-devel-19.0.0-5223.g8c796fca 477 kB/s | 32 kB 00:00 2024-07-31T14:35:38.440 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-07-31T14:35:38.440 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-07-31T14:35:38.440 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-07-31T14:35:38.440 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-07-31T14:35:38.441 INFO:teuthology.orchestra.run.smithi077.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-31T14:35:38.441 INFO:teuthology.orchestra.run.smithi077.stdout: ceph x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 6.6 k 2024-07-31T14:35:38.441 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 5.5 M 2024-07-31T14:35:38.441 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 875 k 2024-07-31T14:35:38.441 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 148 k 2024-07-31T14:35:38.441 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 1.5 M 2024-07-31T14:35:38.441 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 149 k 2024-07-31T14:35:38.441 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 14 M 2024-07-31T14:35:38.441 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 7.4 M 2024-07-31T14:35:38.441 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 49 k 2024-07-31T14:35:38.441 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 10 M 2024-07-31T14:35:38.441 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 44 M 2024-07-31T14:35:38.442 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 274 k 2024-07-31T14:35:38.442 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 770 k 2024-07-31T14:35:38.442 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 228 k 2024-07-31T14:35:38.442 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 27 k 2024-07-31T14:35:38.442 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 18 k 2024-07-31T14:35:38.442 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-31T14:35:38.442 INFO:teuthology.orchestra.run.smithi077.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-31T14:35:38.442 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-31T14:35:38.442 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 32 k 2024-07-31T14:35:38.442 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 768 k 2024-07-31T14:35:38.442 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 127 k 2024-07-31T14:35:38.442 INFO:teuthology.orchestra.run.smithi077.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-31T14:35:38.443 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-31T14:35:38.443 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-31T14:35:38.443 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-31T14:35:38.443 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-31T14:35:38.443 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-31T14:35:38.443 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 162 k 2024-07-31T14:35:38.443 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 321 k 2024-07-31T14:35:38.443 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 303 k 2024-07-31T14:35:38.443 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 99 k 2024-07-31T14:35:38.443 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 86 k 2024-07-31T14:35:38.443 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 3.1 M 2024-07-31T14:35:38.443 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 170 k 2024-07-31T14:35:38.444 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-31T14:35:38.444 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrading: 2024-07-31T14:35:38.444 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-31T14:35:38.444 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-31T14:35:38.444 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-31T14:35:38.444 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-31T14:35:38.444 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-31T14:35:38.444 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-31T14:35:38.444 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-31T14:35:38.444 INFO:teuthology.orchestra.run.smithi077.stdout: librados2 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 3.4 M 2024-07-31T14:35:38.444 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 3.1 M 2024-07-31T14:35:38.444 INFO:teuthology.orchestra.run.smithi077.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-31T14:35:38.445 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-31T14:35:38.445 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-31T14:35:38.445 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-07-31T14:35:38.445 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-31T14:35:38.445 INFO:teuthology.orchestra.run.smithi077.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-31T14:35:38.445 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-31T14:35:38.445 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 20 M 2024-07-31T14:35:38.445 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 34 k 2024-07-31T14:35:38.446 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 2.3 M 2024-07-31T14:35:38.446 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 284 k 2024-07-31T14:35:38.446 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 4.9 M 2024-07-31T14:35:38.446 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 18 M 2024-07-31T14:35:38.446 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5223.g8c796fca.el9 ceph-noarch 16 k 2024-07-31T14:35:38.446 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 25 k 2024-07-31T14:35:38.446 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-31T14:35:38.446 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-31T14:35:38.446 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-31T14:35:38.446 INFO:teuthology.orchestra.run.smithi077.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-31T14:35:38.446 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-31T14:35:38.446 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-31T14:35:38.447 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-31T14:35:38.447 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-31T14:35:38.447 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 158 k 2024-07-31T14:35:38.447 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-31T14:35:38.447 INFO:teuthology.orchestra.run.smithi077.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-31T14:35:38.447 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-31T14:35:38.447 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-31T14:35:38.447 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-31T14:35:38.447 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 506 k 2024-07-31T14:35:38.447 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-31T14:35:38.447 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2 x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 5.3 M 2024-07-31T14:35:38.447 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-31T14:35:38.448 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-31T14:35:38.448 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-31T14:35:38.448 INFO:teuthology.orchestra.run.smithi077.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-31T14:35:38.448 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-31T14:35:38.448 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-31T14:35:38.448 INFO:teuthology.orchestra.run.smithi077.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-31T14:35:38.448 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-31T14:35:38.448 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-31T14:35:38.448 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-31T14:35:38.448 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-31T14:35:38.448 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-31T14:35:38.448 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-31T14:35:38.449 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-31T14:35:38.449 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-31T14:35:38.449 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-31T14:35:38.449 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-31T14:35:38.449 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 45 k 2024-07-31T14:35:38.449 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common x86_64 2:19.0.0-5223.g8c796fca.el9 ceph 138 k 2024-07-31T14:35:38.449 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-31T14:35:38.449 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-31T14:35:38.449 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-31T14:35:38.449 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-31T14:35:38.449 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-31T14:35:38.449 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-31T14:35:38.450 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-31T14:35:38.450 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-31T14:35:38.450 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-31T14:35:38.450 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-31T14:35:38.450 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-31T14:35:38.450 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-31T14:35:38.450 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-31T14:35:38.450 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-31T14:35:38.450 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-31T14:35:38.450 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-31T14:35:38.450 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-31T14:35:38.450 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-31T14:35:38.451 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-31T14:35:38.451 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-31T14:35:38.451 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-31T14:35:38.451 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-31T14:35:38.451 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-31T14:35:38.451 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-31T14:35:38.451 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-31T14:35:38.451 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-31T14:35:38.451 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-31T14:35:38.451 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-31T14:35:38.451 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-31T14:35:38.452 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-31T14:35:38.452 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-31T14:35:38.452 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-31T14:35:38.452 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-31T14:35:38.452 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-31T14:35:38.452 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-31T14:35:38.452 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-31T14:35:38.452 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-31T14:35:38.452 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-31T14:35:38.452 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-31T14:35:38.452 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-31T14:35:38.452 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-31T14:35:38.452 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-31T14:35:38.453 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-31T14:35:38.453 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-31T14:35:38.453 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-31T14:35:38.453 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-31T14:35:38.453 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-31T14:35:38.453 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-31T14:35:38.453 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-07-31T14:35:38.453 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-31T14:35:38.453 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-31T14:35:38.453 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-31T14:35:38.453 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-31T14:35:38.453 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-31T14:35:38.453 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-31T14:35:38.454 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-31T14:35:38.454 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-31T14:35:38.454 INFO:teuthology.orchestra.run.smithi077.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-31T14:35:38.454 INFO:teuthology.orchestra.run.smithi077.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-31T14:35:38.454 INFO:teuthology.orchestra.run.smithi077.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-31T14:35:38.454 INFO:teuthology.orchestra.run.smithi077.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-31T14:35:38.454 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-31T14:35:38.454 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-31T14:35:38.454 INFO:teuthology.orchestra.run.smithi077.stdout:Installing weak dependencies: 2024-07-31T14:35:38.454 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-31T14:35:38.454 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:35:38.454 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-07-31T14:35:38.455 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-07-31T14:35:38.455 INFO:teuthology.orchestra.run.smithi077.stdout:Install 148 Packages 2024-07-31T14:35:38.455 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrade 12 Packages 2024-07-31T14:35:38.455 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:35:38.455 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 232 M 2024-07-31T14:35:38.455 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-07-31T14:35:38.532 INFO:teuthology.orchestra.run.smithi067.stdout:(14/160): libcephfs2-19.0.0-5223.g8c796fca.el9. 8.2 MB/s | 768 kB 00:00 2024-07-31T14:35:38.615 INFO:teuthology.orchestra.run.smithi067.stdout:(15/160): libcephsqlite-19.0.0-5223.g8c796fca.e 1.9 MB/s | 158 kB 00:00 2024-07-31T14:35:38.691 INFO:teuthology.orchestra.run.smithi067.stdout:(16/160): librados-devel-19.0.0-5223.g8c796fca. 1.7 MB/s | 127 kB 00:00 2024-07-31T14:35:38.774 INFO:teuthology.orchestra.run.smithi067.stdout:(17/160): libradosstriper1-19.0.0-5223.g8c796fc 6.0 MB/s | 506 kB 00:00 2024-07-31T14:35:38.828 INFO:teuthology.orchestra.run.smithi073.stdout:(4/160): ceph-mds-19.0.0-5223.g8c796fca.el9.x86 3.7 MB/s | 2.3 MB 00:00 2024-07-31T14:35:38.903 INFO:teuthology.orchestra.run.smithi073.stdout:(5/160): ceph-base-19.0.0-5223.g8c796fca.el9.x8 3.7 MB/s | 5.5 MB 00:01 2024-07-31T14:35:39.158 INFO:teuthology.orchestra.run.smithi067.stdout:(18/160): librgw2-19.0.0-5223.g8c796fca.el9.x86 14 MB/s | 5.3 MB 00:00 2024-07-31T14:35:39.212 INFO:teuthology.orchestra.run.smithi073.stdout:(6/160): ceph-mgr-19.0.0-5223.g8c796fca.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-07-31T14:35:39.233 INFO:teuthology.orchestra.run.smithi067.stdout:(19/160): python3-ceph-argparse-19.0.0-5223.g8c 603 kB/s | 45 kB 00:00 2024-07-31T14:35:39.325 INFO:teuthology.orchestra.run.smithi067.stdout:(20/160): python3-ceph-common-19.0.0-5223.g8c79 1.5 MB/s | 138 kB 00:00 2024-07-31T14:35:39.452 INFO:teuthology.orchestra.run.smithi077.stdout:(1/160): ceph-19.0.0-5223.g8c796fca.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-07-31T14:35:39.542 INFO:teuthology.orchestra.run.smithi067.stdout:(21/160): ceph-osd-19.0.0-5223.g8c796fca.el9.x8 9.1 MB/s | 18 MB 00:01 2024-07-31T14:35:39.559 INFO:teuthology.orchestra.run.smithi067.stdout:(22/160): python3-cephfs-19.0.0-5223.g8c796fca. 693 kB/s | 162 kB 00:00 2024-07-31T14:35:39.609 INFO:teuthology.orchestra.run.smithi067.stdout:(23/160): python3-rados-19.0.0-5223.g8c796fca.e 4.7 MB/s | 321 kB 00:00 2024-07-31T14:35:39.634 INFO:teuthology.orchestra.run.smithi067.stdout:(24/160): python3-rbd-19.0.0-5223.g8c796fca.el9 3.9 MB/s | 303 kB 00:00 2024-07-31T14:35:39.693 INFO:teuthology.orchestra.run.smithi067.stdout:(25/160): python3-rgw-19.0.0-5223.g8c796fca.el9 1.2 MB/s | 99 kB 00:00 2024-07-31T14:35:39.727 INFO:teuthology.orchestra.run.smithi067.stdout:(26/160): rbd-fuse-19.0.0-5223.g8c796fca.el9.x8 938 kB/s | 86 kB 00:00 2024-07-31T14:35:39.827 INFO:teuthology.orchestra.run.smithi067.stdout:(27/160): rbd-nbd-19.0.0-5223.g8c796fca.el9.x86 1.7 MB/s | 170 kB 00:00 2024-07-31T14:35:39.921 INFO:teuthology.orchestra.run.smithi073.stdout:(7/160): ceph-mon-19.0.0-5223.g8c796fca.el9.x86 4.8 MB/s | 4.9 MB 00:01 2024-07-31T14:35:39.927 INFO:teuthology.orchestra.run.smithi067.stdout:(28/160): ceph-grafana-dashboards-19.0.0-5223.g 343 kB/s | 34 kB 00:00 2024-07-31T14:35:40.010 INFO:teuthology.orchestra.run.smithi077.stdout:(2/160): ceph-fuse-19.0.0-5223.g8c796fca.el9.x8 1.5 MB/s | 875 kB 00:00 2024-07-31T14:35:40.102 INFO:teuthology.orchestra.run.smithi077.stdout:(3/160): ceph-immutable-object-cache-19.0.0-522 1.6 MB/s | 148 kB 00:00 2024-07-31T14:35:40.238 INFO:teuthology.orchestra.run.smithi073.stdout:(8/160): ceph-common-19.0.0-5223.g8c796fca.el9. 7.0 MB/s | 20 MB 00:02 2024-07-31T14:35:40.296 INFO:teuthology.orchestra.run.smithi073.stdout:(9/160): ceph-selinux-19.0.0-5223.g8c796fca.el9 432 kB/s | 25 kB 00:00 2024-07-31T14:35:40.336 INFO:teuthology.orchestra.run.smithi067.stdout:(29/160): ceph-test-19.0.0-5223.g8c796fca.el9.x 20 MB/s | 44 MB 00:02 2024-07-31T14:35:40.361 INFO:teuthology.orchestra.run.smithi067.stdout:(30/160): ceph-mgr-cephadm-19.0.0-5223.g8c796fc 345 kB/s | 149 kB 00:00 2024-07-31T14:35:40.420 INFO:teuthology.orchestra.run.smithi067.stdout:(31/160): rbd-mirror-19.0.0-5223.g8c796fca.el9. 4.2 MB/s | 3.1 MB 00:00 2024-07-31T14:35:40.661 INFO:teuthology.orchestra.run.smithi077.stdout:(4/160): ceph-mds-19.0.0-5223.g8c796fca.el9.x86 4.0 MB/s | 2.3 MB 00:00 2024-07-31T14:35:41.163 INFO:teuthology.orchestra.run.smithi073.stdout:(10/160): ceph-osd-19.0.0-5223.g8c796fca.el9.x8 9.1 MB/s | 18 MB 00:01 2024-07-31T14:35:41.212 INFO:teuthology.orchestra.run.smithi067.stdout:(32/160): ceph-mgr-diskprediction-local-19.0.0- 8.7 MB/s | 7.4 MB 00:00 2024-07-31T14:35:41.246 INFO:teuthology.orchestra.run.smithi073.stdout:(11/160): cephfs-mirror-19.0.0-5223.g8c796fca.e 2.7 MB/s | 228 kB 00:00 2024-07-31T14:35:41.262 INFO:teuthology.orchestra.run.smithi067.stdout:(33/160): ceph-mgr-modules-core-19.0.0-5223.g8c 338 kB/s | 284 kB 00:00 2024-07-31T14:35:41.278 INFO:teuthology.orchestra.run.smithi077.stdout:(5/160): ceph-mgr-19.0.0-5223.g8c796fca.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-07-31T14:35:41.295 INFO:teuthology.orchestra.run.smithi067.stdout:(34/160): ceph-mgr-rook-19.0.0-5223.g8c796fca.e 592 kB/s | 49 kB 00:00 2024-07-31T14:35:41.313 INFO:teuthology.orchestra.run.smithi073.stdout:(12/160): cephfs-shell-19.0.0-5223.g8c796fca.el 407 kB/s | 27 kB 00:00 2024-07-31T14:35:41.337 INFO:teuthology.orchestra.run.smithi067.stdout:(35/160): ceph-prometheus-alerts-19.0.0-5223.g8 218 kB/s | 16 kB 00:00 2024-07-31T14:35:41.380 INFO:teuthology.orchestra.run.smithi073.stdout:(13/160): libcephfs-devel-19.0.0-5223.g8c796fca 476 kB/s | 32 kB 00:00 2024-07-31T14:35:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:(36/160): ceph-mgr-dashboard-19.0.0-5223.g8c796 12 MB/s | 14 MB 00:01 2024-07-31T14:35:41.505 INFO:teuthology.orchestra.run.smithi073.stdout:(14/160): libcephfs2-19.0.0-5223.g8c796fca.el9. 6.0 MB/s | 768 kB 00:00 2024-07-31T14:35:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:(37/160): ceph-volume-19.0.0-5223.g8c796fca.el9 1.2 MB/s | 274 kB 00:00 2024-07-31T14:35:41.519 INFO:teuthology.orchestra.run.smithi077.stdout:(6/160): ceph-base-19.0.0-5223.g8c796fca.el9.x8 2.4 MB/s | 5.5 MB 00:02 2024-07-31T14:35:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:(38/160): cephfs-top-19.0.0-5223.g8c796fca.el9. 365 kB/s | 18 kB 00:00 2024-07-31T14:35:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:(39/160): cephadm-19.0.0-5223.g8c796fca.el9.noa 3.2 MB/s | 770 kB 00:00 2024-07-31T14:35:41.572 INFO:teuthology.orchestra.run.smithi073.stdout:(15/160): libcephsqlite-19.0.0-5223.g8c796fca.e 2.3 MB/s | 158 kB 00:00 2024-07-31T14:35:41.639 INFO:teuthology.orchestra.run.smithi073.stdout:(16/160): librados-devel-19.0.0-5223.g8c796fca. 1.9 MB/s | 127 kB 00:00 2024-07-31T14:35:41.723 INFO:teuthology.orchestra.run.smithi073.stdout:(17/160): libradosstriper1-19.0.0-5223.g8c796fc 5.9 MB/s | 506 kB 00:00 2024-07-31T14:35:41.865 INFO:teuthology.orchestra.run.smithi073.stdout:(18/160): ceph-radosgw-19.0.0-5223.g8c796fca.el 5.3 MB/s | 10 MB 00:01 2024-07-31T14:35:41.924 INFO:teuthology.orchestra.run.smithi073.stdout:(19/160): python3-ceph-argparse-19.0.0-5223.g8c 775 kB/s | 45 kB 00:00 2024-07-31T14:35:41.962 INFO:teuthology.orchestra.run.smithi077.stdout:(7/160): ceph-common-19.0.0-5223.g8c796fca.el9. 7.2 MB/s | 20 MB 00:02 2024-07-31T14:35:41.982 INFO:teuthology.orchestra.run.smithi073.stdout:(20/160): python3-ceph-common-19.0.0-5223.g8c79 2.3 MB/s | 138 kB 00:00 2024-07-31T14:35:41.989 INFO:teuthology.orchestra.run.smithi067.stdout:(40/160): fuse-2.9.9-16.el9.x86_64.rpm 168 kB/s | 80 kB 00:00 2024-07-31T14:35:42.058 INFO:teuthology.orchestra.run.smithi073.stdout:(21/160): python3-cephfs-19.0.0-5223.g8c796fca. 2.1 MB/s | 162 kB 00:00 2024-07-31T14:35:42.081 INFO:teuthology.orchestra.run.smithi067.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 384 kB/s | 195 kB 00:00 2024-07-31T14:35:42.139 INFO:teuthology.orchestra.run.smithi067.stdout:(42/160): libgfortran-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 803 kB 00:00 2024-07-31T14:35:42.150 INFO:teuthology.orchestra.run.smithi073.stdout:(22/160): librgw2-19.0.0-5223.g8c796fca.el9.x86 12 MB/s | 5.3 MB 00:00 2024-07-31T14:35:42.183 INFO:teuthology.orchestra.run.smithi073.stdout:(23/160): python3-rados-19.0.0-5223.g8c796fca.e 2.5 MB/s | 321 kB 00:00 2024-07-31T14:35:42.217 INFO:teuthology.orchestra.run.smithi073.stdout:(24/160): python3-rbd-19.0.0-5223.g8c796fca.el9 4.4 MB/s | 303 kB 00:00 2024-07-31T14:35:42.251 INFO:teuthology.orchestra.run.smithi073.stdout:(25/160): python3-rgw-19.0.0-5223.g8c796fca.el9 1.4 MB/s | 99 kB 00:00 2024-07-31T14:35:42.284 INFO:teuthology.orchestra.run.smithi073.stdout:(26/160): rbd-fuse-19.0.0-5223.g8c796fca.el9.x8 1.3 MB/s | 86 kB 00:00 2024-07-31T14:35:42.328 INFO:teuthology.orchestra.run.smithi077.stdout:(8/160): ceph-mon-19.0.0-5223.g8c796fca.el9.x86 4.7 MB/s | 4.9 MB 00:01 2024-07-31T14:35:42.352 INFO:teuthology.orchestra.run.smithi073.stdout:(27/160): rbd-nbd-19.0.0-5223.g8c796fca.el9.x86 2.5 MB/s | 170 kB 00:00 2024-07-31T14:35:42.356 INFO:teuthology.orchestra.run.smithi067.stdout:(43/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 690 kB/s | 253 kB 00:00 2024-07-31T14:35:42.379 INFO:teuthology.orchestra.run.smithi077.stdout:(9/160): ceph-selinux-19.0.0-5223.g8c796fca.el9 502 kB/s | 25 kB 00:00 2024-07-31T14:35:42.418 INFO:teuthology.orchestra.run.smithi073.stdout:(28/160): ceph-grafana-dashboards-19.0.0-5223.g 516 kB/s | 34 kB 00:00 2024-07-31T14:35:42.482 INFO:teuthology.orchestra.run.smithi067.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 338 kB/s | 106 kB 00:00 2024-07-31T14:35:42.486 INFO:teuthology.orchestra.run.smithi073.stdout:(29/160): ceph-mgr-cephadm-19.0.0-5223.g8c796fc 2.2 MB/s | 149 kB 00:00 2024-07-31T14:35:42.590 INFO:teuthology.orchestra.run.smithi067.stdout:(45/160): python3-cryptography-36.0.1-4.el9.x86 2.4 MB/s | 1.2 MB 00:00 2024-07-31T14:35:42.682 INFO:teuthology.orchestra.run.smithi067.stdout:(46/160): python3-pycparser-2.20-6.el9.noarch.r 416 kB/s | 135 kB 00:00 2024-07-31T14:35:42.703 INFO:teuthology.orchestra.run.smithi073.stdout:(30/160): rbd-mirror-19.0.0-5223.g8c796fca.el9. 6.8 MB/s | 3.1 MB 00:00 2024-07-31T14:35:42.829 INFO:teuthology.orchestra.run.smithi077.stdout:(10/160): ceph-radosgw-19.0.0-5223.g8c796fca.el 12 MB/s | 10 MB 00:00 2024-07-31T14:35:42.832 INFO:teuthology.orchestra.run.smithi067.stdout:(47/160): python3-requests-2.25.1-8.el9.noarch. 357 kB/s | 125 kB 00:00 2024-07-31T14:35:42.888 INFO:teuthology.orchestra.run.smithi077.stdout:(11/160): cephfs-mirror-19.0.0-5223.g8c796fca.e 3.8 MB/s | 228 kB 00:00 2024-07-31T14:35:42.930 INFO:teuthology.orchestra.run.smithi077.stdout:(12/160): cephfs-shell-19.0.0-5223.g8c796fca.el 653 kB/s | 27 kB 00:00 2024-07-31T14:35:42.933 INFO:teuthology.orchestra.run.smithi067.stdout:(48/160): python3-urllib3-1.26.5-6.el9.noarch.r 628 kB/s | 215 kB 00:00 2024-07-31T14:35:42.958 INFO:teuthology.orchestra.run.smithi067.stdout:(49/160): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-07-31T14:35:42.972 INFO:teuthology.orchestra.run.smithi077.stdout:(13/160): libcephfs-devel-19.0.0-5223.g8c796fca 760 kB/s | 32 kB 00:00 2024-07-31T14:35:43.008 INFO:teuthology.orchestra.run.smithi067.stdout:(50/160): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-07-31T14:35:43.039 INFO:teuthology.orchestra.run.smithi077.stdout:(14/160): libcephfs2-19.0.0-5223.g8c796fca.el9. 11 MB/s | 768 kB 00:00 2024-07-31T14:35:43.075 INFO:teuthology.orchestra.run.smithi067.stdout:(51/160): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-07-31T14:35:43.089 INFO:teuthology.orchestra.run.smithi077.stdout:(15/160): libcephsqlite-19.0.0-5223.g8c796fca.e 3.1 MB/s | 158 kB 00:00 2024-07-31T14:35:43.103 INFO:teuthology.orchestra.run.smithi073.stdout:(31/160): ceph-test-19.0.0-5223.g8c796fca.el9.x 16 MB/s | 44 MB 00:02 2024-07-31T14:35:43.108 INFO:teuthology.orchestra.run.smithi067.stdout:(52/160): boost-program-options-1.75.0-8.el9.x8 609 kB/s | 107 kB 00:00 2024-07-31T14:35:43.134 INFO:teuthology.orchestra.run.smithi067.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 557 kB/s | 33 kB 00:00 2024-07-31T14:35:43.139 INFO:teuthology.orchestra.run.smithi077.stdout:(16/160): librados-devel-19.0.0-5223.g8c796fca. 2.5 MB/s | 127 kB 00:00 2024-07-31T14:35:43.153 INFO:teuthology.orchestra.run.smithi073.stdout:(32/160): ceph-mgr-modules-core-19.0.0-5223.g8c 5.6 MB/s | 284 kB 00:00 2024-07-31T14:35:43.167 INFO:teuthology.orchestra.run.smithi067.stdout:(54/160): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-07-31T14:35:43.190 INFO:teuthology.orchestra.run.smithi077.stdout:(17/160): libradosstriper1-19.0.0-5223.g8c796fc 9.9 MB/s | 506 kB 00:00 2024-07-31T14:35:43.202 INFO:teuthology.orchestra.run.smithi067.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 267 kB/s | 18 kB 00:00 2024-07-31T14:35:43.211 INFO:teuthology.orchestra.run.smithi073.stdout:(33/160): ceph-mgr-rook-19.0.0-5223.g8c796fca.e 849 kB/s | 49 kB 00:00 2024-07-31T14:35:43.227 INFO:teuthology.orchestra.run.smithi067.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 189 kB/s | 11 kB 00:00 2024-07-31T14:35:43.260 INFO:teuthology.orchestra.run.smithi067.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-07-31T14:35:43.279 INFO:teuthology.orchestra.run.smithi073.stdout:(34/160): ceph-prometheus-alerts-19.0.0-5223.g8 246 kB/s | 16 kB 00:00 2024-07-31T14:35:43.286 INFO:teuthology.orchestra.run.smithi067.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-07-31T14:35:43.327 INFO:teuthology.orchestra.run.smithi067.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-07-31T14:35:43.345 INFO:teuthology.orchestra.run.smithi073.stdout:(35/160): ceph-volume-19.0.0-5223.g8c796fca.el9 4.0 MB/s | 274 kB 00:00 2024-07-31T14:35:43.361 INFO:teuthology.orchestra.run.smithi067.stdout:(60/160): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-07-31T14:35:43.387 INFO:teuthology.orchestra.run.smithi067.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 433 kB/s | 25 kB 00:00 2024-07-31T14:35:43.420 INFO:teuthology.orchestra.run.smithi067.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-07-31T14:35:43.463 INFO:teuthology.orchestra.run.smithi073.stdout:(36/160): cephadm-19.0.0-5223.g8c796fca.el9.noa 6.4 MB/s | 770 kB 00:00 2024-07-31T14:35:43.479 INFO:teuthology.orchestra.run.smithi067.stdout:(63/160): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-07-31T14:35:43.521 INFO:teuthology.orchestra.run.smithi073.stdout:(37/160): cephfs-top-19.0.0-5223.g8c796fca.el9. 314 kB/s | 18 kB 00:00 2024-07-31T14:35:43.529 INFO:teuthology.orchestra.run.smithi067.stdout:(64/160): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-07-31T14:35:43.563 INFO:teuthology.orchestra.run.smithi067.stdout:(65/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-07-31T14:35:43.615 INFO:teuthology.orchestra.run.smithi077.stdout:(18/160): librgw2-19.0.0-5223.g8c796fca.el9.x86 12 MB/s | 5.3 MB 00:00 2024-07-31T14:35:43.665 INFO:teuthology.orchestra.run.smithi077.stdout:(19/160): python3-ceph-argparse-19.0.0-5223.g8c 908 kB/s | 45 kB 00:00 2024-07-31T14:35:43.671 INFO:teuthology.orchestra.run.smithi073.stdout:(38/160): ceph-mgr-dashboard-19.0.0-5223.g8c796 12 MB/s | 14 MB 00:01 2024-07-31T14:35:43.715 INFO:teuthology.orchestra.run.smithi077.stdout:(20/160): python3-ceph-common-19.0.0-5223.g8c79 2.7 MB/s | 138 kB 00:00 2024-07-31T14:35:43.766 INFO:teuthology.orchestra.run.smithi077.stdout:(21/160): python3-cephfs-19.0.0-5223.g8c796fca. 3.2 MB/s | 162 kB 00:00 2024-07-31T14:35:43.788 INFO:teuthology.orchestra.run.smithi073.stdout:(39/160): ceph-mgr-diskprediction-local-19.0.0- 6.8 MB/s | 7.4 MB 00:01 2024-07-31T14:35:43.816 INFO:teuthology.orchestra.run.smithi077.stdout:(22/160): python3-rados-19.0.0-5223.g8c796fca.e 6.3 MB/s | 321 kB 00:00 2024-07-31T14:35:43.866 INFO:teuthology.orchestra.run.smithi077.stdout:(23/160): python3-rbd-19.0.0-5223.g8c796fca.el9 5.9 MB/s | 303 kB 00:00 2024-07-31T14:35:43.917 INFO:teuthology.orchestra.run.smithi077.stdout:(24/160): python3-rgw-19.0.0-5223.g8c796fca.el9 1.9 MB/s | 99 kB 00:00 2024-07-31T14:35:43.963 INFO:teuthology.orchestra.run.smithi067.stdout:(66/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.5 MB/s | 3.0 MB 00:00 2024-07-31T14:35:43.967 INFO:teuthology.orchestra.run.smithi077.stdout:(25/160): rbd-fuse-19.0.0-5223.g8c796fca.el9.x8 1.7 MB/s | 86 kB 00:00 2024-07-31T14:35:44.105 INFO:teuthology.orchestra.run.smithi073.stdout:(40/160): fuse-2.9.9-16.el9.x86_64.rpm 137 kB/s | 80 kB 00:00 2024-07-31T14:35:44.142 INFO:teuthology.orchestra.run.smithi077.stdout:(26/160): rbd-mirror-19.0.0-5223.g8c796fca.el9. 18 MB/s | 3.1 MB 00:00 2024-07-31T14:35:44.193 INFO:teuthology.orchestra.run.smithi077.stdout:(27/160): rbd-nbd-19.0.0-5223.g8c796fca.el9.x86 3.3 MB/s | 170 kB 00:00 2024-07-31T14:35:44.243 INFO:teuthology.orchestra.run.smithi077.stdout:(28/160): ceph-grafana-dashboards-19.0.0-5223.g 685 kB/s | 34 kB 00:00 2024-07-31T14:35:44.256 INFO:teuthology.orchestra.run.smithi073.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 444 kB/s | 195 kB 00:00 2024-07-31T14:35:44.285 INFO:teuthology.orchestra.run.smithi077.stdout:(29/160): ceph-mgr-cephadm-19.0.0-5223.g8c796fc 3.5 MB/s | 149 kB 00:00 2024-07-31T14:35:44.306 INFO:teuthology.orchestra.run.smithi073.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-07-31T14:35:44.481 INFO:teuthology.orchestra.run.smithi073.stdout:(43/160): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-07-31T14:35:44.515 INFO:teuthology.orchestra.run.smithi073.stdout:(44/160): python3-cryptography-36.0.1-4.el9.x86 4.7 MB/s | 1.2 MB 00:00 2024-07-31T14:35:44.540 INFO:teuthology.orchestra.run.smithi073.stdout:(45/160): python3-ply-3.11-14.el9.noarch.rpm 456 kB/s | 106 kB 00:00 2024-07-31T14:35:44.552 INFO:teuthology.orchestra.run.smithi077.stdout:(30/160): ceph-osd-19.0.0-5223.g8c796fca.el9.x8 5.9 MB/s | 18 MB 00:03 2024-07-31T14:35:44.590 INFO:teuthology.orchestra.run.smithi073.stdout:(46/160): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-07-31T14:35:44.649 INFO:teuthology.orchestra.run.smithi073.stdout:(47/160): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-07-31T14:35:44.674 INFO:teuthology.orchestra.run.smithi073.stdout:(48/160): python3-urllib3-1.26.5-6.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-07-31T14:35:44.872 INFO:teuthology.orchestra.run.smithi067.stdout:(67/160): postgresql-13.14-1.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2024-07-31T14:35:45.044 INFO:teuthology.orchestra.run.smithi077.stdout:(31/160): ceph-mgr-dashboard-19.0.0-5223.g8c796 18 MB/s | 14 MB 00:00 2024-07-31T14:35:45.094 INFO:teuthology.orchestra.run.smithi077.stdout:(32/160): ceph-mgr-modules-core-19.0.0-5223.g8c 5.6 MB/s | 284 kB 00:00 2024-07-31T14:35:45.144 INFO:teuthology.orchestra.run.smithi077.stdout:(33/160): ceph-mgr-rook-19.0.0-5223.g8c796fca.e 987 kB/s | 49 kB 00:00 2024-07-31T14:35:45.195 INFO:teuthology.orchestra.run.smithi077.stdout:(34/160): ceph-prometheus-alerts-19.0.0-5223.g8 330 kB/s | 16 kB 00:00 2024-07-31T14:35:45.245 INFO:teuthology.orchestra.run.smithi077.stdout:(35/160): ceph-volume-19.0.0-5223.g8c796fca.el9 5.4 MB/s | 274 kB 00:00 2024-07-31T14:35:45.312 INFO:teuthology.orchestra.run.smithi077.stdout:(36/160): cephadm-19.0.0-5223.g8c796fca.el9.noa 11 MB/s | 770 kB 00:00 2024-07-31T14:35:45.339 INFO:teuthology.orchestra.run.smithi067.stdout:(68/160): postgresql-contrib-13.14-1.el9.x86_64 1.9 MB/s | 911 kB 00:00 2024-07-31T14:35:45.354 INFO:teuthology.orchestra.run.smithi077.stdout:(37/160): cephfs-top-19.0.0-5223.g8c796fca.el9. 436 kB/s | 18 kB 00:00 2024-07-31T14:35:45.366 INFO:teuthology.orchestra.run.smithi073.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 154 kB/s | 107 kB 00:00 2024-07-31T14:35:45.447 INFO:teuthology.orchestra.run.smithi067.stdout:(69/160): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-07-31T14:35:45.463 INFO:teuthology.orchestra.run.smithi077.stdout:(38/160): ceph-mgr-diskprediction-local-19.0.0- 8.1 MB/s | 7.4 MB 00:00 2024-07-31T14:35:45.517 INFO:teuthology.orchestra.run.smithi073.stdout:(50/160): automake-1.16.2-8.el9.noarch.rpm 744 kB/s | 689 kB 00:00 2024-07-31T14:35:45.580 INFO:teuthology.orchestra.run.smithi077.stdout:(39/160): fuse-2.9.9-16.el9.x86_64.rpm 354 kB/s | 80 kB 00:00 2024-07-31T14:35:45.659 INFO:teuthology.orchestra.run.smithi073.stdout:(51/160): bison-3.7.4-5.el9.x86_64.rpm 934 kB/s | 943 kB 00:01 2024-07-31T14:35:45.663 INFO:teuthology.orchestra.run.smithi077.stdout:(40/160): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-07-31T14:35:45.722 INFO:teuthology.orchestra.run.smithi077.stdout:(41/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-07-31T14:35:45.756 INFO:teuthology.orchestra.run.smithi077.stdout:(42/160): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-07-31T14:35:45.772 INFO:teuthology.orchestra.run.smithi067.stdout:(70/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 2.0 MB/s | 4.4 MB 00:02 2024-07-31T14:35:45.814 INFO:teuthology.orchestra.run.smithi077.stdout:(43/160): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-07-31T14:35:45.873 INFO:teuthology.orchestra.run.smithi077.stdout:(44/160): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.2 MB 00:00 2024-07-31T14:35:45.898 INFO:teuthology.orchestra.run.smithi077.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-07-31T14:35:45.925 INFO:teuthology.orchestra.run.smithi073.stdout:(52/160): elfutils-libelf-devel-0.191-4.el9.x86 39 kB/s | 22 kB 00:00 2024-07-31T14:35:45.932 INFO:teuthology.orchestra.run.smithi077.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-07-31T14:35:45.956 INFO:teuthology.orchestra.run.smithi067.stdout:(71/160): openblas-openmp-0.3.21-2.el9.x86_64.r 2.0 MB/s | 4.7 MB 00:02 2024-07-31T14:35:45.957 INFO:teuthology.orchestra.run.smithi077.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-07-31T14:35:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:(72/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-07-31T14:35:46.234 INFO:teuthology.orchestra.run.smithi073.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 57 kB/s | 33 kB 00:00 2024-07-31T14:35:46.290 INFO:teuthology.orchestra.run.smithi067.stdout:(73/160): protobuf-3.14.0-13.el9.x86_64.rpm 2.0 MB/s | 1.0 MB 00:00 2024-07-31T14:35:46.349 INFO:teuthology.orchestra.run.smithi067.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-07-31T14:35:46.351 INFO:teuthology.orchestra.run.smithi073.stdout:(54/160): flex-2.6.4-9.el9.x86_64.rpm 376 kB/s | 313 kB 00:00 2024-07-31T14:35:46.491 INFO:teuthology.orchestra.run.smithi067.stdout:(75/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-07-31T14:35:46.633 INFO:teuthology.orchestra.run.smithi067.stdout:(76/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-07-31T14:35:46.649 INFO:teuthology.orchestra.run.smithi077.stdout:(48/160): automake-1.16.2-8.el9.noarch.rpm 961 kB/s | 689 kB 00:00 2024-07-31T14:35:46.683 INFO:teuthology.orchestra.run.smithi067.stdout:(77/160): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-07-31T14:35:46.717 INFO:teuthology.orchestra.run.smithi077.stdout:(49/160): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-07-31T14:35:46.733 INFO:teuthology.orchestra.run.smithi067.stdout:(78/160): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-07-31T14:35:46.734 INFO:teuthology.orchestra.run.smithi073.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 36 kB/s | 18 kB 00:00 2024-07-31T14:35:46.783 INFO:teuthology.orchestra.run.smithi077.stdout:(50/160): boost-program-options-1.75.0-8.el9.x8 802 kB/s | 107 kB 00:00 2024-07-31T14:35:46.817 INFO:teuthology.orchestra.run.smithi077.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 216 kB/s | 22 kB 00:00 2024-07-31T14:35:46.850 INFO:teuthology.orchestra.run.smithi067.stdout:(79/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-07-31T14:35:46.851 INFO:teuthology.orchestra.run.smithi073.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-07-31T14:35:46.900 INFO:teuthology.orchestra.run.smithi067.stdout:(80/160): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-07-31T14:35:46.909 INFO:teuthology.orchestra.run.smithi077.stdout:(52/160): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-07-31T14:35:46.934 INFO:teuthology.orchestra.run.smithi077.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 279 kB/s | 33 kB 00:00 2024-07-31T14:35:47.027 INFO:teuthology.orchestra.run.smithi073.stdout:(57/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-07-31T14:35:47.343 INFO:teuthology.orchestra.run.smithi077.stdout:(54/160): ceph-test-19.0.0-5223.g8c796fca.el9.x 8.8 MB/s | 44 MB 00:04 2024-07-31T14:35:47.477 INFO:teuthology.orchestra.run.smithi073.stdout:(58/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 216 kB/s | 160 kB 00:00 2024-07-31T14:35:47.510 INFO:teuthology.orchestra.run.smithi073.stdout:(59/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 69 kB/s | 45 kB 00:00 2024-07-31T14:35:47.560 INFO:teuthology.orchestra.run.smithi077.stdout:(55/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.6 MB/s | 3.0 MB 00:00 2024-07-31T14:35:47.576 INFO:teuthology.orchestra.run.smithi077.stdout:(56/160): flexiblas-openblas-openmp-3.0.4-8.el9 28 kB/s | 18 kB 00:00 2024-07-31T14:35:47.602 INFO:teuthology.orchestra.run.smithi077.stdout:(57/160): libaio-devel-0.3.111-13.el9.x86_64.rp 43 kB/s | 11 kB 00:00 2024-07-31T14:35:47.685 INFO:teuthology.orchestra.run.smithi077.stdout:(58/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-07-31T14:35:47.727 INFO:teuthology.orchestra.run.smithi077.stdout:(59/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2024-07-31T14:35:47.752 INFO:teuthology.orchestra.run.smithi077.stdout:(60/160): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-07-31T14:35:47.803 INFO:teuthology.orchestra.run.smithi077.stdout:(61/160): libtool-2.4.6-46.el9.x86_64.rpm 4.8 MB/s | 581 kB 00:00 2024-07-31T14:35:47.828 INFO:teuthology.orchestra.run.smithi077.stdout:(62/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 256 kB/s | 25 kB 00:00 2024-07-31T14:35:47.853 INFO:teuthology.orchestra.run.smithi077.stdout:(63/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 471 kB/s | 47 kB 00:00 2024-07-31T14:35:47.912 INFO:teuthology.orchestra.run.smithi077.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-07-31T14:35:47.919 INFO:teuthology.orchestra.run.smithi073.stdout:(60/160): librdkafka-1.6.1-102.el9.x86_64.rpm 743 kB/s | 662 kB 00:00 2024-07-31T14:35:47.938 INFO:teuthology.orchestra.run.smithi077.stdout:(65/160): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-07-31T14:35:47.963 INFO:teuthology.orchestra.run.smithi077.stdout:(66/160): openblas-0.3.21-2.el9.x86_64.rpm 322 kB/s | 35 kB 00:00 2024-07-31T14:35:48.094 INFO:teuthology.orchestra.run.smithi073.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 44 kB/s | 25 kB 00:00 2024-07-31T14:35:48.189 INFO:teuthology.orchestra.run.smithi077.stdout:(67/160): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-07-31T14:35:48.339 INFO:teuthology.orchestra.run.smithi077.stdout:(68/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-07-31T14:35:48.381 INFO:teuthology.orchestra.run.smithi077.stdout:(69/160): postgresql-13.14-1.el9.x86_64.rpm 3.8 MB/s | 1.6 MB 00:00 2024-07-31T14:35:48.403 INFO:teuthology.orchestra.run.smithi073.stdout:(62/160): libtool-2.4.6-46.el9.x86_64.rpm 628 kB/s | 581 kB 00:00 2024-07-31T14:35:48.414 INFO:teuthology.orchestra.run.smithi077.stdout:(70/160): postgresql-contrib-13.14-1.el9.x86_64 4.0 MB/s | 911 kB 00:00 2024-07-31T14:35:48.476 INFO:teuthology.orchestra.run.smithi067.stdout:(81/160): postgresql-server-13.14-1.el9.x86_64. 1.9 MB/s | 5.8 MB 00:03 2024-07-31T14:35:48.481 INFO:teuthology.orchestra.run.smithi077.stdout:(71/160): postgresql-private-libs-13.14-1.el9.x 952 kB/s | 133 kB 00:00 2024-07-31T14:35:48.562 INFO:teuthology.orchestra.run.smithi073.stdout:(63/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 74 kB/s | 47 kB 00:00 2024-07-31T14:35:48.626 INFO:teuthology.orchestra.run.smithi067.stdout:(82/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-07-31T14:35:48.673 INFO:teuthology.orchestra.run.smithi077.stdout:(72/160): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-07-31T14:35:48.698 INFO:teuthology.orchestra.run.smithi077.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 403 kB/s | 87 kB 00:00 2024-07-31T14:35:48.732 INFO:teuthology.orchestra.run.smithi077.stdout:(74/160): protobuf-3.14.0-13.el9.x86_64.rpm 3.2 MB/s | 1.0 MB 00:00 2024-07-31T14:35:48.776 INFO:teuthology.orchestra.run.smithi067.stdout:(83/160): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-07-31T14:35:48.860 INFO:teuthology.orchestra.run.smithi067.stdout:(84/160): python3-packaging-20.9-5.el9.noarch.r 928 kB/s | 77 kB 00:00 2024-07-31T14:35:48.866 INFO:teuthology.orchestra.run.smithi077.stdout:(75/160): python3-colorama-0.4.6-3.el9.noarch.r 369 kB/s | 61 kB 00:00 2024-07-31T14:35:48.919 INFO:teuthology.orchestra.run.smithi067.stdout:(85/160): python3-prettytable-0.7.2-27.el9.noar 726 kB/s | 42 kB 00:00 2024-07-31T14:35:48.920 INFO:teuthology.orchestra.run.smithi073.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 354 kB/s | 292 kB 00:00 2024-07-31T14:35:48.949 INFO:teuthology.orchestra.run.smithi077.stdout:(76/160): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-07-31T14:35:48.974 INFO:teuthology.orchestra.run.smithi077.stdout:(77/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.0 MB/s | 244 kB 00:00 2024-07-31T14:35:49.000 INFO:teuthology.orchestra.run.smithi077.stdout:(78/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-07-31T14:35:49.061 INFO:teuthology.orchestra.run.smithi067.stdout:(86/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.4 MB/s | 5.1 MB 00:02 2024-07-31T14:35:49.083 INFO:teuthology.orchestra.run.smithi077.stdout:(79/160): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-07-31T14:35:49.108 INFO:teuthology.orchestra.run.smithi077.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 144 kB/s | 19 kB 00:00 2024-07-31T14:35:49.127 INFO:teuthology.orchestra.run.smithi067.stdout:(87/160): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-07-31T14:35:49.134 INFO:teuthology.orchestra.run.smithi077.stdout:(81/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-07-31T14:35:49.178 INFO:teuthology.orchestra.run.smithi067.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-07-31T14:35:49.184 INFO:teuthology.orchestra.run.smithi077.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 347 kB/s | 35 kB 00:00 2024-07-31T14:35:49.196 INFO:teuthology.orchestra.run.smithi073.stdout:(65/160): lua-5.4.4-4.el9.x86_64.rpm 238 kB/s | 188 kB 00:00 2024-07-31T14:35:49.221 INFO:teuthology.orchestra.run.smithi073.stdout:(66/160): openblas-0.3.21-2.el9.x86_64.rpm 53 kB/s | 35 kB 00:00 2024-07-31T14:35:49.228 INFO:teuthology.orchestra.run.smithi067.stdout:(89/160): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-07-31T14:35:49.278 INFO:teuthology.orchestra.run.smithi067.stdout:(90/160): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-07-31T14:35:49.312 INFO:teuthology.orchestra.run.smithi067.stdout:(91/160): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 279 kB 00:00 2024-07-31T14:35:49.354 INFO:teuthology.orchestra.run.smithi067.stdout:(92/160): python3-toml-0.10.2-6.el9.noarch.rpm 998 kB/s | 42 kB 00:00 2024-07-31T14:35:49.376 INFO:teuthology.orchestra.run.smithi077.stdout:(83/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-07-31T14:35:49.402 INFO:teuthology.orchestra.run.smithi077.stdout:(84/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 939 kB/s | 250 kB 00:00 2024-07-31T14:35:49.413 INFO:teuthology.orchestra.run.smithi067.stdout:(93/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-07-31T14:35:49.426 INFO:teuthology.orchestra.run.smithi077.stdout:(85/160): python3-oauthlib-3.1.1-5.el9.noarch.r 918 kB/s | 222 kB 00:00 2024-07-31T14:35:49.485 INFO:teuthology.orchestra.run.smithi077.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 713 kB/s | 77 kB 00:00 2024-07-31T14:35:49.527 INFO:teuthology.orchestra.run.smithi077.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-07-31T14:35:49.552 INFO:teuthology.orchestra.run.smithi077.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-07-31T14:35:49.555 INFO:teuthology.orchestra.run.smithi067.stdout:(94/160): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-07-31T14:35:49.586 INFO:teuthology.orchestra.run.smithi077.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-07-31T14:35:49.597 INFO:teuthology.orchestra.run.smithi067.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 863 kB/s | 36 kB 00:00 2024-07-31T14:35:49.628 INFO:teuthology.orchestra.run.smithi077.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-07-31T14:35:49.638 INFO:teuthology.orchestra.run.smithi067.stdout:(96/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-07-31T14:35:49.645 INFO:teuthology.orchestra.run.smithi077.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 552 kB/s | 51 kB 00:00 2024-07-31T14:35:49.678 INFO:teuthology.orchestra.run.smithi077.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 582 kB/s | 54 kB 00:00 2024-07-31T14:35:49.821 INFO:teuthology.orchestra.run.smithi077.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 237 kB/s | 42 kB 00:00 2024-07-31T14:35:49.822 INFO:teuthology.orchestra.run.smithi067.stdout:(97/160): socat-1.7.4.1-6.el9.x86_64.rpm 1.6 MB/s | 304 kB 00:00 2024-07-31T14:35:49.838 INFO:teuthology.orchestra.run.smithi077.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 278 kB/s | 44 kB 00:00 2024-07-31T14:35:49.923 INFO:teuthology.orchestra.run.smithi067.stdout:(98/160): python3-babel-2.9.1-2.el9.noarch.rpm 1.5 MB/s | 6.0 MB 00:03 2024-07-31T14:35:49.948 INFO:teuthology.orchestra.run.smithi067.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 461 kB/s | 58 kB 00:00 2024-07-31T14:35:49.973 INFO:teuthology.orchestra.run.smithi067.stdout:(100/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-07-31T14:35:50.013 INFO:teuthology.orchestra.run.smithi077.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 208 kB/s | 36 kB 00:00 2024-07-31T14:35:50.032 INFO:teuthology.orchestra.run.smithi067.stdout:(101/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 770 kB/s | 45 kB 00:00 2024-07-31T14:35:50.057 INFO:teuthology.orchestra.run.smithi067.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 585 kB/s | 64 kB 00:00 2024-07-31T14:35:50.082 INFO:teuthology.orchestra.run.smithi067.stdout:(103/160): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-07-31T14:35:50.113 INFO:teuthology.orchestra.run.smithi073.stdout:(67/160): openblas-openmp-0.3.21-2.el9.x86_64.r 4.0 MB/s | 4.7 MB 00:01 2024-07-31T14:35:50.122 INFO:teuthology.orchestra.run.smithi077.stdout:(96/160): qatlib-24.02.0-1.el9.x86_64.rpm 735 kB/s | 221 kB 00:00 2024-07-31T14:35:50.158 INFO:teuthology.orchestra.run.smithi067.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 7.1 MB/s | 548 kB 00:00 2024-07-31T14:35:50.191 INFO:teuthology.orchestra.run.smithi067.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-07-31T14:35:50.216 INFO:teuthology.orchestra.run.smithi067.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 785 kB/s | 19 kB 00:00 2024-07-31T14:35:50.238 INFO:teuthology.orchestra.run.smithi073.stdout:(68/160): postgresql-13.14-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:01 2024-07-31T14:35:50.297 INFO:teuthology.orchestra.run.smithi077.stdout:(97/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-07-31T14:35:50.322 INFO:teuthology.orchestra.run.smithi077.stdout:(98/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 155 kB/s | 47 kB 00:00 2024-07-31T14:35:50.330 INFO:teuthology.orchestra.run.smithi073.stdout:(69/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-07-31T14:35:50.333 INFO:teuthology.orchestra.run.smithi067.stdout:(107/160): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-07-31T14:35:50.347 INFO:teuthology.orchestra.run.smithi077.stdout:(99/160): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-07-31T14:35:50.359 INFO:teuthology.orchestra.run.smithi067.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-07-31T14:35:50.384 INFO:teuthology.orchestra.run.smithi067.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-07-31T14:35:50.397 INFO:teuthology.orchestra.run.smithi077.stdout:(100/160): uuid-1.6.2-55.el9.x86_64.rpm 581 kB/s | 58 kB 00:00 2024-07-31T14:35:50.410 INFO:teuthology.orchestra.run.smithi067.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-07-31T14:35:50.435 INFO:teuthology.orchestra.run.smithi067.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-07-31T14:35:50.448 INFO:teuthology.orchestra.run.smithi077.stdout:(101/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 955 kB/s | 96 kB 00:00 2024-07-31T14:35:50.473 INFO:teuthology.orchestra.run.smithi077.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 508 kB/s | 64 kB 00:00 2024-07-31T14:35:50.477 INFO:teuthology.orchestra.run.smithi067.stdout:(112/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-07-31T14:35:50.498 INFO:teuthology.orchestra.run.smithi077.stdout:(103/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 445 kB/s | 45 kB 00:00 2024-07-31T14:35:50.510 INFO:teuthology.orchestra.run.smithi067.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-07-31T14:35:50.536 INFO:teuthology.orchestra.run.smithi067.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-07-31T14:35:50.549 INFO:teuthology.orchestra.run.smithi077.stdout:(104/160): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-07-31T14:35:50.561 INFO:teuthology.orchestra.run.smithi067.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-07-31T14:35:50.595 INFO:teuthology.orchestra.run.smithi067.stdout:(116/160): python3-certifi-2023.05.07-4.el9.noa 423 kB/s | 14 kB 00:00 2024-07-31T14:35:50.607 INFO:teuthology.orchestra.run.smithi077.stdout:(105/160): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-07-31T14:35:50.620 INFO:teuthology.orchestra.run.smithi067.stdout:(117/160): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-07-31T14:35:50.654 INFO:teuthology.orchestra.run.smithi067.stdout:(118/160): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-07-31T14:35:50.679 INFO:teuthology.orchestra.run.smithi067.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-07-31T14:35:50.697 INFO:teuthology.orchestra.run.smithi077.stdout:(106/160): gperftools-libs-2.9.1-2.el9.x86_64.r 2.1 MB/s | 309 kB 00:00 2024-07-31T14:35:50.724 INFO:teuthology.orchestra.run.smithi077.stdout:(107/160): grpc-data-1.46.7-10.el9.noarch.rpm 170 kB/s | 19 kB 00:00 2024-07-31T14:35:50.729 INFO:teuthology.orchestra.run.smithi067.stdout:(120/160): python3-google-auth-2.32.0-1.el9.noa 9.5 MB/s | 216 kB 00:00 2024-07-31T14:35:50.750 INFO:teuthology.orchestra.run.smithi077.stdout:(108/160): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-07-31T14:35:50.771 INFO:teuthology.orchestra.run.smithi067.stdout:(121/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-07-31T14:35:50.775 INFO:teuthology.orchestra.run.smithi077.stdout:(109/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 516 kB/s | 26 kB 00:00 2024-07-31T14:35:50.796 INFO:teuthology.orchestra.run.smithi067.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-07-31T14:35:50.800 INFO:teuthology.orchestra.run.smithi077.stdout:(110/160): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-07-31T14:35:50.821 INFO:teuthology.orchestra.run.smithi067.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-07-31T14:35:50.826 INFO:teuthology.orchestra.run.smithi077.stdout:(111/160): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-07-31T14:35:50.847 INFO:teuthology.orchestra.run.smithi067.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 704 kB/s | 18 kB 00:00 2024-07-31T14:35:50.851 INFO:teuthology.orchestra.run.smithi077.stdout:(112/160): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-07-31T14:35:50.872 INFO:teuthology.orchestra.run.smithi067.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 932 kB/s | 23 kB 00:00 2024-07-31T14:35:50.893 INFO:teuthology.orchestra.run.smithi077.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-07-31T14:35:50.897 INFO:teuthology.orchestra.run.smithi067.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-07-31T14:35:50.918 INFO:teuthology.orchestra.run.smithi077.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-07-31T14:35:50.922 INFO:teuthology.orchestra.run.smithi067.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 788 kB/s | 20 kB 00:00 2024-07-31T14:35:50.944 INFO:teuthology.orchestra.run.smithi077.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-07-31T14:35:50.969 INFO:teuthology.orchestra.run.smithi077.stdout:(116/160): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-07-31T14:35:50.981 INFO:teuthology.orchestra.run.smithi067.stdout:(128/160): python3-grpcio-1.46.7-10.el9.x86_64. 8.2 MB/s | 2.0 MB 00:00 2024-07-31T14:35:51.005 INFO:teuthology.orchestra.run.smithi073.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 174 kB/s | 133 kB 00:00 2024-07-31T14:35:51.007 INFO:teuthology.orchestra.run.smithi067.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-07-31T14:35:51.011 INFO:teuthology.orchestra.run.smithi077.stdout:(117/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.4 MB/s | 839 kB 00:00 2024-07-31T14:35:51.024 INFO:teuthology.orchestra.run.smithi067.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 4.6 MB/s | 79 kB 00:00 2024-07-31T14:35:51.036 INFO:teuthology.orchestra.run.smithi077.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-07-31T14:35:51.049 INFO:teuthology.orchestra.run.smithi067.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-07-31T14:35:51.070 INFO:teuthology.orchestra.run.smithi077.stdout:(119/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-07-31T14:35:51.072 INFO:teuthology.orchestra.run.smithi073.stdout:(71/160): postgresql-contrib-13.14-1.el9.x86_64 950 kB/s | 911 kB 00:00 2024-07-31T14:35:51.082 INFO:teuthology.orchestra.run.smithi067.stdout:(132/160): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-07-31T14:35:51.095 INFO:teuthology.orchestra.run.smithi077.stdout:(120/160): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-07-31T14:35:51.108 INFO:teuthology.orchestra.run.smithi067.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 654 kB/s | 16 kB 00:00 2024-07-31T14:35:51.121 INFO:teuthology.orchestra.run.smithi077.stdout:(121/160): python3-google-auth-2.32.0-1.el9.noa 4.2 MB/s | 216 kB 00:00 2024-07-31T14:35:51.133 INFO:teuthology.orchestra.run.smithi067.stdout:(134/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-07-31T14:35:51.146 INFO:teuthology.orchestra.run.smithi077.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-07-31T14:35:51.158 INFO:teuthology.orchestra.run.smithi067.stdout:(135/160): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-07-31T14:35:51.175 INFO:teuthology.orchestra.run.smithi067.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 674 kB/s | 28 kB 00:00 2024-07-31T14:35:51.179 INFO:teuthology.orchestra.run.smithi077.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 321 kB/s | 11 kB 00:00 2024-07-31T14:35:51.192 INFO:teuthology.orchestra.run.smithi067.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 1.8 MB/s | 31 kB 00:00 2024-07-31T14:35:51.205 INFO:teuthology.orchestra.run.smithi077.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 703 kB/s | 18 kB 00:00 2024-07-31T14:35:51.210 INFO:teuthology.orchestra.run.smithi067.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-07-31T14:35:51.230 INFO:teuthology.orchestra.run.smithi077.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-07-31T14:35:51.235 INFO:teuthology.orchestra.run.smithi067.stdout:(139/160): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-07-31T14:35:51.256 INFO:teuthology.orchestra.run.smithi077.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 777 kB/s | 19 kB 00:00 2024-07-31T14:35:51.260 INFO:teuthology.orchestra.run.smithi067.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 709 kB/s | 36 kB 00:00 2024-07-31T14:35:51.277 INFO:teuthology.orchestra.run.smithi067.stdout:(141/160): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-07-31T14:35:51.281 INFO:teuthology.orchestra.run.smithi077.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 787 kB/s | 20 kB 00:00 2024-07-31T14:35:51.302 INFO:teuthology.orchestra.run.smithi067.stdout:(142/160): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-07-31T14:35:51.328 INFO:teuthology.orchestra.run.smithi067.stdout:(143/160): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-07-31T14:35:51.353 INFO:teuthology.orchestra.run.smithi067.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 806 kB/s | 20 kB 00:00 2024-07-31T14:35:51.364 INFO:teuthology.orchestra.run.smithi077.stdout:(128/160): libarrow-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-07-31T14:35:51.378 INFO:teuthology.orchestra.run.smithi067.stdout:(145/160): re2-20211101-3.el9.x86_64.rpm 7.5 MB/s | 192 kB 00:00 2024-07-31T14:35:51.398 INFO:teuthology.orchestra.run.smithi077.stdout:(129/160): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-07-31T14:35:51.403 INFO:teuthology.orchestra.run.smithi067.stdout:(146/160): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-07-31T14:35:51.417 INFO:teuthology.orchestra.run.smithi077.stdout:(130/160): python3-logutils-0.3.5-21.el9.noarch 921 kB/s | 46 kB 00:00 2024-07-31T14:35:51.440 INFO:teuthology.orchestra.run.smithi077.stdout:(131/160): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-07-31T14:35:51.457 INFO:teuthology.orchestra.run.smithi077.stdout:(132/160): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-07-31T14:35:51.462 INFO:teuthology.orchestra.run.smithi067.stdout:(147/160): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-07-31T14:35:51.475 INFO:teuthology.orchestra.run.smithi077.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 982 kB/s | 16 kB 00:00 2024-07-31T14:35:51.516 INFO:teuthology.orchestra.run.smithi077.stdout:(134/160): python3-grpcio-1.46.7-10.el9.x86_64. 4.8 MB/s | 2.0 MB 00:00 2024-07-31T14:35:51.542 INFO:teuthology.orchestra.run.smithi077.stdout:(135/160): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-07-31T14:35:51.547 INFO:teuthology.orchestra.run.smithi073.stdout:(72/160): postgresql-server-13.14-1.el9.x86_64. 4.9 MB/s | 5.8 MB 00:01 2024-07-31T14:35:51.559 INFO:teuthology.orchestra.run.smithi077.stdout:(136/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-07-31T14:35:51.584 INFO:teuthology.orchestra.run.smithi077.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 734 kB/s | 31 kB 00:00 2024-07-31T14:35:51.609 INFO:teuthology.orchestra.run.smithi077.stdout:(138/160): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-07-31T14:35:51.635 INFO:teuthology.orchestra.run.smithi077.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-07-31T14:35:51.660 INFO:teuthology.orchestra.run.smithi077.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-07-31T14:35:51.700 INFO:teuthology.orchestra.run.smithi077.stdout:(141/160): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-07-31T14:35:51.711 INFO:teuthology.orchestra.run.smithi077.stdout:(142/160): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-07-31T14:35:51.712 INFO:teuthology.orchestra.run.smithi067.stdout:(148/160): librbd1-19.0.0-5223.g8c796fca.el9.x8 13 MB/s | 3.1 MB 00:00 2024-07-31T14:35:51.736 INFO:teuthology.orchestra.run.smithi077.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-07-31T14:35:51.747 INFO:teuthology.orchestra.run.smithi073.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 129 kB/s | 87 kB 00:00 2024-07-31T14:35:51.761 INFO:teuthology.orchestra.run.smithi077.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 806 kB/s | 20 kB 00:00 2024-07-31T14:35:51.787 INFO:teuthology.orchestra.run.smithi077.stdout:(145/160): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-07-31T14:35:51.812 INFO:teuthology.orchestra.run.smithi077.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-07-31T14:35:51.837 INFO:teuthology.orchestra.run.smithi077.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-07-31T14:35:51.879 INFO:teuthology.orchestra.run.smithi077.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-07-31T14:35:51.896 INFO:teuthology.orchestra.run.smithi067.stdout:(149/160): librados2-19.0.0-5223.g8c796fca.el9. 6.9 MB/s | 3.4 MB 00:00 2024-07-31T14:35:52.015 INFO:teuthology.orchestra.run.smithi073.stdout:(74/160): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-07-31T14:35:52.155 INFO:teuthology.orchestra.run.smithi077.stdout:(149/160): librados2-19.0.0-5223.g8c796fca.el9. 9.9 MB/s | 3.4 MB 00:00 2024-07-31T14:35:52.205 INFO:teuthology.orchestra.run.smithi077.stdout:(150/160): NetworkManager-1.48.2-2.el9.x86_64.r 7.1 MB/s | 2.3 MB 00:00 2024-07-31T14:35:52.238 INFO:teuthology.orchestra.run.smithi067.stdout:(150/160): NetworkManager-1.48.2-2.el9.x86_64.r 4.4 MB/s | 2.3 MB 00:00 2024-07-31T14:35:52.272 INFO:teuthology.orchestra.run.smithi077.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 612 kB/s | 41 kB 00:00 2024-07-31T14:35:52.347 INFO:teuthology.orchestra.run.smithi077.stdout:(152/160): librbd1-19.0.0-5223.g8c796fca.el9.x8 6.2 MB/s | 3.1 MB 00:00 2024-07-31T14:35:52.372 INFO:teuthology.orchestra.run.smithi077.stdout:(153/160): NetworkManager-tui-1.48.2-2.el9.x86_ 2.4 MB/s | 247 kB 00:00 2024-07-31T14:35:52.431 INFO:teuthology.orchestra.run.smithi077.stdout:(154/160): elfutils-libelf-0.191-4.el9.x86_64.r 3.5 MB/s | 207 kB 00:00 2024-07-31T14:35:52.440 INFO:teuthology.orchestra.run.smithi073.stdout:(75/160): python3-colorama-0.4.6-3.el9.noarch.r 88 kB/s | 61 kB 00:00 2024-07-31T14:35:52.465 INFO:teuthology.orchestra.run.smithi077.stdout:(155/160): elfutils-debuginfod-client-0.191-4.e 313 kB/s | 37 kB 00:00 2024-07-31T14:35:52.507 INFO:teuthology.orchestra.run.smithi077.stdout:(156/160): NetworkManager-libnm-1.48.2-2.el9.x8 5.1 MB/s | 1.8 MB 00:00 2024-07-31T14:35:52.532 INFO:teuthology.orchestra.run.smithi077.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 2.5 MB/s | 260 kB 00:00 2024-07-31T14:35:52.538 INFO:teuthology.orchestra.run.smithi067.stdout:(151/160): NetworkManager-libnm-1.48.2-2.el9.x8 2.8 MB/s | 1.8 MB 00:00 2024-07-31T14:35:52.563 INFO:teuthology.orchestra.run.smithi067.stdout:(152/160): NetworkManager-team-1.48.2-2.el9.x86 125 kB/s | 41 kB 00:00 2024-07-31T14:35:52.632 INFO:teuthology.orchestra.run.smithi077.stdout:(158/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 11 MB/s | 1.1 MB 00:00 2024-07-31T14:35:52.682 INFO:teuthology.orchestra.run.smithi077.stdout:(159/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-07-31T14:35:52.715 INFO:teuthology.orchestra.run.smithi073.stdout:(76/160): python3-babel-2.9.1-2.el9.noarch.rpm 5.1 MB/s | 6.0 MB 00:01 2024-07-31T14:35:52.758 INFO:teuthology.orchestra.run.smithi077.stdout:(160/160): openssl-3.2.2-2.el9.x86_64.rpm 4.6 MB/s | 1.4 MB 00:00 2024-07-31T14:35:52.765 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-07-31T14:35:52.765 INFO:teuthology.orchestra.run.smithi077.stdout:Total 16 MB/s | 232 MB 00:14 2024-07-31T14:35:52.839 INFO:teuthology.orchestra.run.smithi067.stdout:(153/160): elfutils-debuginfod-client-0.191-4.e 134 kB/s | 37 kB 00:00 2024-07-31T14:35:52.865 INFO:teuthology.orchestra.run.smithi073.stdout:(77/160): python3-devel-3.9.18-3.el9.x86_64.rpm 296 kB/s | 244 kB 00:00 2024-07-31T14:35:52.880 INFO:teuthology.orchestra.run.smithi067.stdout:(154/160): NetworkManager-tui-1.48.2-2.el9.x86_ 724 kB/s | 247 kB 00:00 2024-07-31T14:35:53.206 INFO:teuthology.orchestra.run.smithi067.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 564 kB/s | 207 kB 00:00 2024-07-31T14:35:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:(78/160): python3-jinja2-2.11.3-6.el9.noarch.rp 292 kB/s | 248 kB 00:00 2024-07-31T14:35:53.306 INFO:teuthology.orchestra.run.smithi067.stdout:(156/160): elfutils-libs-0.191-4.el9.x86_64.rpm 654 kB/s | 260 kB 00:00 2024-07-31T14:35:53.324 INFO:teuthology.orchestra.run.smithi073.stdout:(79/160): python3-jsonpatch-1.21-16.el9.noarch. 44 kB/s | 26 kB 00:00 2024-07-31T14:35:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 38 kB/s | 19 kB 00:00 2024-07-31T14:35:53.648 INFO:teuthology.orchestra.run.smithi067.stdout:(157/160): openssl-3.2.2-2.el9.x86_64.rpm 3.1 MB/s | 1.4 MB 00:00 2024-07-31T14:35:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:(158/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 4.7 MB/s | 2.4 MB 00:00 2024-07-31T14:35:53.950 INFO:teuthology.orchestra.run.smithi073.stdout:(81/160): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-07-31T14:35:53.995 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-07-31T14:35:54.042 INFO:teuthology.orchestra.run.smithi073.stdout:(82/160): python3-mako-1.1.4-6.el9.noarch.rpm 230 kB/s | 172 kB 00:00 2024-07-31T14:35:54.082 INFO:teuthology.orchestra.run.smithi067.stdout:(159/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 2.5 MB/s | 1.1 MB 00:00 2024-07-31T14:35:54.188 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-07-31T14:35:54.188 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-07-31T14:35:54.584 INFO:teuthology.orchestra.run.smithi073.stdout:(83/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-07-31T14:35:54.751 INFO:teuthology.orchestra.run.smithi073.stdout:(84/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 313 kB/s | 250 kB 00:00 2024-07-31T14:35:54.818 INFO:teuthology.orchestra.run.smithi073.stdout:(85/160): python3-oauthlib-3.1.1-5.el9.noarch.r 286 kB/s | 222 kB 00:00 2024-07-31T14:35:55.401 INFO:teuthology.orchestra.run.smithi073.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 94 kB/s | 77 kB 00:00 2024-07-31T14:35:55.476 INFO:teuthology.orchestra.run.smithi073.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 58 kB/s | 42 kB 00:00 2024-07-31T14:35:55.860 INFO:teuthology.orchestra.run.smithi073.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 258 kB/s | 269 kB 00:01 2024-07-31T14:35:56.031 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-07-31T14:35:56.031 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-07-31T14:35:56.377 INFO:teuthology.orchestra.run.smithi073.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 163 kB/s | 159 kB 00:00 2024-07-31T14:35:56.511 INFO:teuthology.orchestra.run.smithi073.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 270 kB/s | 279 kB 00:01 2024-07-31T14:35:56.711 INFO:teuthology.orchestra.run.smithi073.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 60 kB/s | 51 kB 00:00 2024-07-31T14:35:57.153 INFO:teuthology.orchestra.run.smithi073.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 69 kB/s | 54 kB 00:00 2024-07-31T14:35:57.428 INFO:teuthology.orchestra.run.smithi073.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 58 kB/s | 42 kB 00:00 2024-07-31T14:35:57.886 INFO:teuthology.orchestra.run.smithi073.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 60 kB/s | 44 kB 00:00 2024-07-31T14:35:58.320 INFO:teuthology.orchestra.run.smithi073.stdout:(95/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.0 MB/s | 16 MB 00:01 2024-07-31T14:35:58.429 INFO:teuthology.orchestra.run.smithi073.stdout:(96/160): qatlib-24.02.0-1.el9.x86_64.rpm 227 kB/s | 221 kB 00:00 2024-07-31T14:35:58.738 INFO:teuthology.orchestra.run.smithi073.stdout:(97/160): qatlib-service-24.02.0-1.el9.x86_64.r 42 kB/s | 36 kB 00:00 2024-07-31T14:35:59.079 INFO:teuthology.orchestra.run.smithi073.stdout:(98/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 63 kB/s | 47 kB 00:00 2024-07-31T14:35:59.413 INFO:teuthology.orchestra.run.smithi073.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 86 kB/s | 58 kB 00:00 2024-07-31T14:35:59.438 INFO:teuthology.orchestra.run.smithi073.stdout:(100/160): socat-1.7.4.1-6.el9.x86_64.rpm 301 kB/s | 304 kB 00:01 2024-07-31T14:35:59.507 INFO:teuthology.orchestra.run.smithi067.stdout:(160/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.6 MB/s | 16 MB 00:10 2024-07-31T14:35:59.513 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-07-31T14:35:59.513 INFO:teuthology.orchestra.run.smithi067.stdout:Total 10 MB/s | 232 MB 00:23 2024-07-31T14:35:59.813 INFO:teuthology.orchestra.run.smithi073.stdout:(101/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 130 kB/s | 96 kB 00:00 2024-07-31T14:35:59.897 INFO:teuthology.orchestra.run.smithi073.stdout:(102/160): lua-devel-5.4.4-4.el9.x86_64.rpm 269 kB/s | 22 kB 00:00 2024-07-31T14:36:00.047 INFO:teuthology.orchestra.run.smithi073.stdout:(103/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 74 kB/s | 45 kB 00:00 2024-07-31T14:36:00.106 INFO:teuthology.orchestra.run.smithi073.stdout:(104/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 92 kB/s | 64 kB 00:00 2024-07-31T14:36:00.164 INFO:teuthology.orchestra.run.smithi073.stdout:(105/160): abseil-cpp-20211102.0-3.el9.x86_64.r 4.6 MB/s | 548 kB 00:00 2024-07-31T14:36:00.198 INFO:teuthology.orchestra.run.smithi073.stdout:(106/160): protobuf-compiler-3.14.0-13.el9.x86_ 2.8 MB/s | 863 kB 00:00 2024-07-31T14:36:00.232 INFO:teuthology.orchestra.run.smithi073.stdout:(107/160): grpc-data-1.46.7-10.el9.noarch.rpm 300 kB/s | 19 kB 00:00 2024-07-31T14:36:00.282 INFO:teuthology.orchestra.run.smithi073.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-07-31T14:36:00.307 INFO:teuthology.orchestra.run.smithi073.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-07-31T14:36:00.333 INFO:teuthology.orchestra.run.smithi073.stdout:(110/160): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-07-31T14:36:00.358 INFO:teuthology.orchestra.run.smithi073.stdout:(111/160): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-07-31T14:36:00.383 INFO:teuthology.orchestra.run.smithi073.stdout:(112/160): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-07-31T14:36:00.425 INFO:teuthology.orchestra.run.smithi073.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-07-31T14:36:00.451 INFO:teuthology.orchestra.run.smithi073.stdout:(114/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-07-31T14:36:00.476 INFO:teuthology.orchestra.run.smithi073.stdout:(115/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 863 kB/s | 43 kB 00:00 2024-07-31T14:36:00.501 INFO:teuthology.orchestra.run.smithi073.stdout:(116/160): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-07-31T14:36:00.518 INFO:teuthology.orchestra.run.smithi073.stdout:(117/160): python3-certifi-2023.05.07-4.el9.noa 336 kB/s | 14 kB 00:00 2024-07-31T14:36:00.543 INFO:teuthology.orchestra.run.smithi073.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-07-31T14:36:00.577 INFO:teuthology.orchestra.run.smithi073.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-07-31T14:36:00.610 INFO:teuthology.orchestra.run.smithi073.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-07-31T14:36:00.638 INFO:teuthology.orchestra.run.smithi073.stdout:(121/160): python3-google-auth-2.32.0-1.el9.noa 3.6 MB/s | 216 kB 00:00 2024-07-31T14:36:00.661 INFO:teuthology.orchestra.run.smithi073.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-07-31T14:36:00.678 INFO:teuthology.orchestra.run.smithi073.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 649 kB/s | 11 kB 00:00 2024-07-31T14:36:00.703 INFO:teuthology.orchestra.run.smithi073.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 704 kB/s | 18 kB 00:00 2024-07-31T14:36:00.729 INFO:teuthology.orchestra.run.smithi073.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 923 kB/s | 23 kB 00:00 2024-07-31T14:36:00.754 INFO:teuthology.orchestra.run.smithi073.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-07-31T14:36:00.779 INFO:teuthology.orchestra.run.smithi073.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 787 kB/s | 20 kB 00:00 2024-07-31T14:36:00.781 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-07-31T14:36:00.838 INFO:teuthology.orchestra.run.smithi073.stdout:(128/160): python3-grpcio-1.46.7-10.el9.x86_64. 9.0 MB/s | 2.0 MB 00:00 2024-07-31T14:36:00.863 INFO:teuthology.orchestra.run.smithi073.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-07-31T14:36:00.939 INFO:teuthology.orchestra.run.smithi073.stdout:(130/160): libarrow-9.0.0-11.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-07-31T14:36:00.964 INFO:teuthology.orchestra.run.smithi073.stdout:(131/160): python3-more-itertools-8.12.0-2.el9. 785 kB/s | 79 kB 00:00 2024-07-31T14:36:00.989 INFO:teuthology.orchestra.run.smithi073.stdout:(132/160): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-07-31T14:36:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-07-31T14:36:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-07-31T14:36:01.023 INFO:teuthology.orchestra.run.smithi073.stdout:(133/160): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-07-31T14:36:01.057 INFO:teuthology.orchestra.run.smithi073.stdout:(134/160): python3-portend-3.1.0-2.el9.noarch.r 244 kB/s | 16 kB 00:00 2024-07-31T14:36:01.082 INFO:teuthology.orchestra.run.smithi073.stdout:(135/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-07-31T14:36:01.107 INFO:teuthology.orchestra.run.smithi073.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-07-31T14:36:01.141 INFO:teuthology.orchestra.run.smithi073.stdout:(137/160): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-07-31T14:36:01.194 INFO:teuthology.orchestra.run.smithi073.stdout:(138/160): python3-repoze-lru-0.7-16.el9.noarch 356 kB/s | 31 kB 00:00 2024-07-31T14:36:01.225 INFO:teuthology.orchestra.run.smithi073.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 706 kB/s | 59 kB 00:00 2024-07-31T14:36:01.250 INFO:teuthology.orchestra.run.smithi073.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 640 kB/s | 36 kB 00:00 2024-07-31T14:36:01.284 INFO:teuthology.orchestra.run.smithi073.stdout:(141/160): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-07-31T14:36:01.309 INFO:teuthology.orchestra.run.smithi073.stdout:(142/160): python3-routes-2.5.1-5.el9.noarch.rp 937 kB/s | 188 kB 00:00 2024-07-31T14:36:01.334 INFO:teuthology.orchestra.run.smithi073.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-07-31T14:36:01.360 INFO:teuthology.orchestra.run.smithi073.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-07-31T14:36:01.385 INFO:teuthology.orchestra.run.smithi073.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-07-31T14:36:01.410 INFO:teuthology.orchestra.run.smithi073.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-07-31T14:36:01.444 INFO:teuthology.orchestra.run.smithi073.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-07-31T14:36:01.519 INFO:teuthology.orchestra.run.smithi073.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-07-31T14:36:01.695 INFO:teuthology.orchestra.run.smithi073.stdout:(149/160): librados2-19.0.0-5223.g8c796fca.el9. 12 MB/s | 3.4 MB 00:00 2024-07-31T14:36:01.753 INFO:teuthology.orchestra.run.smithi073.stdout:(150/160): NetworkManager-1.48.2-2.el9.x86_64.r 9.9 MB/s | 2.3 MB 00:00 2024-07-31T14:36:01.837 INFO:teuthology.orchestra.run.smithi073.stdout:(151/160): NetworkManager-libnm-1.48.2-2.el9.x8 13 MB/s | 1.8 MB 00:00 2024-07-31T14:36:01.862 INFO:teuthology.orchestra.run.smithi073.stdout:(152/160): NetworkManager-team-1.48.2-2.el9.x86 376 kB/s | 41 kB 00:00 2024-07-31T14:36:01.929 INFO:teuthology.orchestra.run.smithi073.stdout:(153/160): librbd1-19.0.0-5223.g8c796fca.el9.x8 6.5 MB/s | 3.1 MB 00:00 2024-07-31T14:36:01.962 INFO:teuthology.orchestra.run.smithi073.stdout:(154/160): NetworkManager-tui-1.48.2-2.el9.x86_ 1.9 MB/s | 247 kB 00:00 2024-07-31T14:36:01.988 INFO:teuthology.orchestra.run.smithi073.stdout:(155/160): elfutils-debuginfod-client-0.191-4.e 292 kB/s | 37 kB 00:00 2024-07-31T14:36:02.030 INFO:teuthology.orchestra.run.smithi073.stdout:(156/160): elfutils-libelf-0.191-4.el9.x86_64.r 2.0 MB/s | 207 kB 00:00 2024-07-31T14:36:02.065 INFO:teuthology.orchestra.run.smithi073.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 2.5 MB/s | 260 kB 00:00 2024-07-31T14:36:02.122 INFO:teuthology.orchestra.run.smithi073.stdout:(158/160): openssl-3.2.2-2.el9.x86_64.rpm 10 MB/s | 1.4 MB 00:00 2024-07-31T14:36:02.189 INFO:teuthology.orchestra.run.smithi073.stdout:(159/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-07-31T14:36:02.239 INFO:teuthology.orchestra.run.smithi073.stdout:(160/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 6.2 MB/s | 1.1 MB 00:00 2024-07-31T14:36:02.248 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-07-31T14:36:02.248 INFO:teuthology.orchestra.run.smithi073.stdout:Total 9.2 MB/s | 232 MB 00:25 2024-07-31T14:36:02.424 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-07-31T14:36:02.495 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-31T14:36:02.525 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-31T14:36:02.587 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-31T14:36:02.621 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-31T14:36:02.647 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-31T14:36:02.690 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-31T14:36:03.036 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-31T14:36:03.165 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librados2-2:19.0.0-5223.g8c796fca.el9.x86_64 8/172 2024-07-31T14:36:03.293 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:19.0.0-5223.g8c796fca.el9.x86_64 8/172 2024-07-31T14:36:03.324 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs2-2:19.0.0-5223.g8c796fca.el9.x86_64 9/172 2024-07-31T14:36:03.398 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs2-2:19.0.0-5223.g8c796fca.el9.x86_64 9/172 2024-07-31T14:36:03.428 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rados-2:19.0.0-5223.g8c796fca.el9.x86_64 10/172 2024-07-31T14:36:03.461 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-31T14:36:03.478 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-07-31T14:36:03.536 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-07-31T14:36:03.537 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-07-31T14:36:03.646 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-31T14:36:03.670 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-07-31T14:36:03.670 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-07-31T14:36:03.696 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-31T14:36:03.724 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephsqlite-2:19.0.0-5223.g8c796fca.el9.x86_64 14/172 2024-07-31T14:36:03.830 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5223.g8c796fca.el9.x86_64 14/172 2024-07-31T14:36:03.858 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libradosstriper1-2:19.0.0-5223.g8c796fca.el9.x86 15/172 2024-07-31T14:36:03.920 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5223.g8c796fca.el9.x86 15/172 2024-07-31T14:36:03.942 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-31T14:36:03.965 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-31T14:36:04.160 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-31T14:36:04.255 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-31T14:36:04.439 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-31T14:36:04.600 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-31T14:36:04.652 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-31T14:36:04.768 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-31T14:36:04.837 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-31T14:36:05.049 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-31T14:36:05.163 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-31T14:36:05.193 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-31T14:36:05.228 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-31T14:36:05.327 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-31T14:36:05.371 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-31T14:36:05.403 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-31T14:36:05.446 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-31T14:36:05.479 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-31T14:36:05.507 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-31T14:36:05.575 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-31T14:36:05.629 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-31T14:36:05.660 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-31T14:36:05.695 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-31T14:36:05.772 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-31T14:36:05.776 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-07-31T14:36:05.776 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-07-31T14:36:05.814 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-common-2:19.0.0-5223.g8c796fca.el9. 35/172 2024-07-31T14:36:05.849 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-argparse-2:19.0.0-5223.g8c796fca.el 36/172 2024-07-31T14:36:05.878 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cephfs-2:19.0.0-5223.g8c796fca.el9.x86_6 37/172 2024-07-31T14:36:05.949 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-31T14:36:05.992 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-31T14:36:06.067 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-31T14:36:06.202 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-31T14:36:06.262 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-31T14:36:06.817 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-31T14:36:06.899 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-31T14:36:06.928 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-31T14:36:06.964 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-31T14:36:07.004 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-31T14:36:07.043 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librados-devel-2:19.0.0-5223.g8c796fca.el9.x86_6 48/172 2024-07-31T14:36:07.084 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-31T14:36:07.247 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-31T14:36:07.304 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-31T14:36:07.395 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-31T14:36:07.460 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-31T14:36:07.532 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-31T14:36:07.564 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-31T14:36:07.592 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 56/172 2024-07-31T14:36:07.638 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-31T14:36:07.674 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-31T14:36:07.717 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-31T14:36:07.760 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-31T14:36:07.896 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-31T14:36:07.936 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-31T14:36:07.977 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-31T14:36:08.076 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-31T14:36:08.119 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-31T14:36:08.159 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-31T14:36:08.185 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-31T14:36:08.209 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-31T14:36:08.284 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-31T14:36:08.411 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-31T14:36:08.648 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-31T14:36:08.704 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-31T14:36:08.738 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-31T14:36:08.778 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-31T14:36:08.826 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-31T14:36:08.855 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-31T14:36:08.890 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-31T14:36:08.923 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-31T14:36:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-31T14:36:08.999 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-31T14:36:09.090 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-31T14:36:09.137 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-31T14:36:09.168 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-31T14:36:09.272 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-31T14:36:09.335 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-31T14:36:10.064 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-31T14:36:10.108 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-31T14:36:10.141 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-31T14:36:10.242 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-31T14:36:10.248 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-07-31T14:36:10.304 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-31T14:36:10.328 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-31T14:36:10.349 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-31T14:36:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-31T14:36:10.437 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-31T14:36:10.447 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-31T14:36:10.485 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-31T14:36:10.487 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-31T14:36:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-31T14:36:10.564 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-31T14:36:10.662 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-31T14:36:10.691 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-31T14:36:10.769 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-31T14:36:10.923 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-31T14:36:11.047 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librados2-2:19.0.0-5223.g8c796fca.el9.x86_64 8/172 2024-07-31T14:36:11.178 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:19.0.0-5223.g8c796fca.el9.x86_64 8/172 2024-07-31T14:36:11.206 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs2-2:19.0.0-5223.g8c796fca.el9.x86_64 9/172 2024-07-31T14:36:11.299 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs2-2:19.0.0-5223.g8c796fca.el9.x86_64 9/172 2024-07-31T14:36:11.327 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rados-2:19.0.0-5223.g8c796fca.el9.x86_64 10/172 2024-07-31T14:36:11.377 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-31T14:36:11.417 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-31T14:36:11.531 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-31T14:36:11.618 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-31T14:36:11.678 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-31T14:36:11.703 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephsqlite-2:19.0.0-5223.g8c796fca.el9.x86_64 14/172 2024-07-31T14:36:11.795 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5223.g8c796fca.el9.x86_64 14/172 2024-07-31T14:36:11.821 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libradosstriper1-2:19.0.0-5223.g8c796fca.el9.x86 15/172 2024-07-31T14:36:11.867 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5223.g8c796fca.el9.x86 15/172 2024-07-31T14:36:11.889 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-31T14:36:11.915 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-31T14:36:12.108 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-31T14:36:12.207 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-31T14:36:12.413 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-07-31T14:36:12.466 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-31T14:36:12.527 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-31T14:36:12.565 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-31T14:36:12.618 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-31T14:36:12.621 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-31T14:36:12.652 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-31T14:36:12.665 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-31T14:36:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-31T14:36:12.712 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-31T14:36:12.754 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-31T14:36:12.785 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-31T14:36:12.833 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-31T14:36:12.836 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-31T14:36:12.864 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-31T14:36:12.897 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-31T14:36:13.041 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-31T14:36:13.115 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-31T14:36:13.159 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-31T14:36:13.179 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librados2-2:19.0.0-5223.g8c796fca.el9.x86_64 8/172 2024-07-31T14:36:13.198 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-31T14:36:13.223 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-31T14:36:13.247 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-31T14:36:13.268 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librbd1-2:19.0.0-5223.g8c796fca.el9.x86_64 104/172 2024-07-31T14:36:13.304 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:19.0.0-5223.g8c796fca.el9.x86_64 8/172 2024-07-31T14:36:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-31T14:36:13.330 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs2-2:19.0.0-5223.g8c796fca.el9.x86_64 9/172 2024-07-31T14:36:13.359 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:19.0.0-5223.g8c796fca.el9.x86_64 104/172 2024-07-31T14:36:13.374 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-31T14:36:13.390 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rbd-2:19.0.0-5223.g8c796fca.el9.x86_64 105/172 2024-07-31T14:36:13.414 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-31T14:36:13.422 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephfs2-2:19.0.0-5223.g8c796fca.el9.x86_64 9/172 2024-07-31T14:36:13.425 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-31T14:36:13.451 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rados-2:19.0.0-5223.g8c796fca.el9.x86_64 10/172 2024-07-31T14:36:13.457 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-31T14:36:13.483 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-31T14:36:13.490 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-31T14:36:13.517 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-31T14:36:13.589 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-31T14:36:13.657 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-31T14:36:13.687 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-31T14:36:13.690 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-31T14:36:13.731 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-31T14:36:13.740 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-31T14:36:13.768 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephsqlite-2:19.0.0-5223.g8c796fca.el9.x86_64 14/172 2024-07-31T14:36:13.811 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-31T14:36:13.843 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-common-2:19.0.0-5223.g8c796fca.el9. 35/172 2024-07-31T14:36:13.873 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5223.g8c796fca.el9.x86_64 14/172 2024-07-31T14:36:13.887 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-argparse-2:19.0.0-5223.g8c796fca.el 36/172 2024-07-31T14:36:13.903 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libradosstriper1-2:19.0.0-5223.g8c796fca.el9.x86 15/172 2024-07-31T14:36:13.915 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cephfs-2:19.0.0-5223.g8c796fca.el9.x86_6 37/172 2024-07-31T14:36:13.925 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-31T14:36:13.954 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librgw2-2:19.0.0-5223.g8c796fca.el9.x86_64 108/172 2024-07-31T14:36:13.965 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5223.g8c796fca.el9.x86 15/172 2024-07-31T14:36:13.987 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-31T14:36:13.988 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-31T14:36:14.008 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-31T14:36:14.030 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-31T14:36:14.031 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librgw2-2:19.0.0-5223.g8c796fca.el9.x86_64 108/172 2024-07-31T14:36:14.056 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rgw-2:19.0.0-5223.g8c796fca.el9.x86_64 109/172 2024-07-31T14:36:14.107 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-31T14:36:14.211 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-31T14:36:14.244 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-31T14:36:14.300 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-31T14:36:14.309 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-31T14:36:14.494 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-31T14:36:14.653 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-31T14:36:14.713 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-31T14:36:14.811 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-31T14:36:14.854 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-31T14:36:14.892 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-31T14:36:14.938 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-31T14:36:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-31T14:36:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-31T14:36:15.051 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-31T14:36:15.089 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librados-devel-2:19.0.0-5223.g8c796fca.el9.x86_6 48/172 2024-07-31T14:36:15.093 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-31T14:36:15.132 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-31T14:36:15.196 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-31T14:36:15.221 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-31T14:36:15.264 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-31T14:36:15.288 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-31T14:36:15.328 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-31T14:36:15.342 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-31T14:36:15.373 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-31T14:36:15.406 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-31T14:36:15.436 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-31T14:36:15.449 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-31T14:36:15.482 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-31T14:36:15.510 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-31T14:36:15.566 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-31T14:36:15.578 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-31T14:36:15.640 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-31T14:36:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-31T14:36:15.707 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-31T14:36:15.783 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-31T14:36:15.818 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-common-2:19.0.0-5223.g8c796fca.el9. 35/172 2024-07-31T14:36:15.861 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-31T14:36:15.862 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-argparse-2:19.0.0-5223.g8c796fca.el 36/172 2024-07-31T14:36:15.890 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cephfs-2:19.0.0-5223.g8c796fca.el9.x86_6 37/172 2024-07-31T14:36:15.902 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-31T14:36:15.947 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 56/172 2024-07-31T14:36:15.959 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-31T14:36:16.005 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-31T14:36:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-31T14:36:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-31T14:36:16.079 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-31T14:36:16.097 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-31T14:36:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-31T14:36:16.220 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-31T14:36:16.274 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-31T14:36:16.277 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 110/172 2024-07-31T14:36:16.284 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-31T14:36:16.311 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 110/172 2024-07-31T14:36:16.333 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-31T14:36:16.366 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-31T14:36:16.467 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-31T14:36:16.507 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-31T14:36:16.560 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-31T14:36:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-31T14:36:16.614 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-31T14:36:16.706 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-31T14:36:16.789 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-31T14:36:16.847 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-31T14:36:16.870 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-31T14:36:16.875 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 110/172 2024-07-31T14:36:16.880 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-31T14:36:16.907 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-31T14:36:16.908 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-31T14:36:16.915 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-base-2:19.0.0-5223.g8c796fca.el9.x86_64 111/172 2024-07-31T14:36:16.944 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-31T14:36:16.945 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-31T14:36:16.982 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-31T14:36:16.994 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-31T14:36:17.005 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-base-2:19.0.0-5223.g8c796fca.el9.x86_64 111/172 2024-07-31T14:36:17.005 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-31T14:36:17.005 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-31T14:36:17.005 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:36:17.022 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-31T14:36:17.031 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librados-devel-2:19.0.0-5223.g8c796fca.el9.x86_6 48/172 2024-07-31T14:36:17.033 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-selinux-2:19.0.0-5223.g8c796fca.el9.x86_64 112/172 2024-07-31T14:36:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-31T14:36:17.072 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-31T14:36:17.077 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-31T14:36:17.111 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-31T14:36:17.153 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-31T14:36:17.186 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-31T14:36:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-31T14:36:17.276 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-31T14:36:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-31T14:36:17.360 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-31T14:36:17.390 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-31T14:36:17.400 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-31T14:36:17.465 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-31T14:36:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-31T14:36:17.547 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-31T14:36:17.551 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-31T14:36:17.578 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-31T14:36:17.623 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 56/172 2024-07-31T14:36:17.669 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-31T14:36:17.706 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-31T14:36:17.748 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-31T14:36:17.791 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-31T14:36:17.925 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-31T14:36:17.968 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-31T14:36:18.009 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-31T14:36:18.106 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-31T14:36:18.143 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-31T14:36:18.183 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-31T14:36:18.209 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-31T14:36:18.234 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-31T14:36:18.305 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-31T14:36:18.320 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-31T14:36:18.372 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-31T14:36:18.405 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-31T14:36:18.426 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-31T14:36:18.466 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-31T14:36:18.494 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-31T14:36:18.507 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-31T14:36:18.528 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-31T14:36:18.561 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-31T14:36:18.568 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-31T14:36:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-31T14:36:18.607 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-31T14:36:18.637 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-31T14:36:18.672 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-31T14:36:18.705 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-31T14:36:18.706 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-31T14:36:18.741 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-31T14:36:18.747 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-31T14:36:18.782 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-31T14:36:18.876 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-31T14:36:18.926 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-31T14:36:18.929 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-31T14:36:18.955 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-31T14:36:18.960 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-31T14:36:19.036 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-31T14:36:19.063 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-31T14:36:19.110 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-31T14:36:19.726 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-31T14:36:19.824 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-31T14:36:19.849 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-31T14:36:19.875 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-31T14:36:19.909 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-31T14:36:20.006 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-31T14:36:20.064 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-31T14:36:20.108 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-31T14:36:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-31T14:36:20.220 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-31T14:36:20.387 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-31T14:36:20.418 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-31T14:36:20.495 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-31T14:36:21.135 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-31T14:36:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-31T14:36:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-31T14:36:21.256 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-31T14:36:21.279 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-31T14:36:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-31T14:36:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-31T14:36:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librbd1-2:19.0.0-5223.g8c796fca.el9.x86_64 104/172 2024-07-31T14:36:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:19.0.0-5223.g8c796fca.el9.x86_64 104/172 2024-07-31T14:36:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rbd-2:19.0.0-5223.g8c796fca.el9.x86_64 105/172 2024-07-31T14:36:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-31T14:36:22.383 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-31T14:36:22.419 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librgw2-2:19.0.0-5223.g8c796fca.el9.x86_64 108/172 2024-07-31T14:36:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-31T14:36:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librgw2-2:19.0.0-5223.g8c796fca.el9.x86_64 108/172 2024-07-31T14:36:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rgw-2:19.0.0-5223.g8c796fca.el9.x86_64 109/172 2024-07-31T14:36:22.576 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-31T14:36:22.610 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-31T14:36:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-31T14:36:22.971 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-31T14:36:23.023 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librbd1-2:19.0.0-5223.g8c796fca.el9.x86_64 104/172 2024-07-31T14:36:23.114 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:19.0.0-5223.g8c796fca.el9.x86_64 104/172 2024-07-31T14:36:23.144 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rbd-2:19.0.0-5223.g8c796fca.el9.x86_64 105/172 2024-07-31T14:36:23.186 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-31T14:36:23.683 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-31T14:36:23.713 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librgw2-2:19.0.0-5223.g8c796fca.el9.x86_64 108/172 2024-07-31T14:36:23.784 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librgw2-2:19.0.0-5223.g8c796fca.el9.x86_64 108/172 2024-07-31T14:36:23.815 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rgw-2:19.0.0-5223.g8c796fca.el9.x86_64 109/172 2024-07-31T14:36:24.926 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 110/172 2024-07-31T14:36:24.960 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 110/172 2024-07-31T14:36:25.563 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 110/172 2024-07-31T14:36:25.597 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-base-2:19.0.0-5223.g8c796fca.el9.x86_64 111/172 2024-07-31T14:36:25.689 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-base-2:19.0.0-5223.g8c796fca.el9.x86_64 111/172 2024-07-31T14:36:25.689 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-31T14:36:25.689 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-31T14:36:25.689 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:36:25.723 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-selinux-2:19.0.0-5223.g8c796fca.el9.x86_64 112/172 2024-07-31T14:36:26.036 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 110/172 2024-07-31T14:36:26.078 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 110/172 2024-07-31T14:36:26.638 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 110/172 2024-07-31T14:36:26.682 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-base-2:19.0.0-5223.g8c796fca.el9.x86_64 111/172 2024-07-31T14:36:26.773 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-base-2:19.0.0-5223.g8c796fca.el9.x86_64 111/172 2024-07-31T14:36:26.773 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-31T14:36:26.773 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-31T14:36:26.773 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:36:26.799 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-selinux-2:19.0.0-5223.g8c796fca.el9.x86_64 112/172 2024-07-31T14:36:28.640 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5223.g8c796fca.el9.x86_64 112/172 2024-07-31T14:36:28.640 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /sys 2024-07-31T14:36:28.640 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /proc 2024-07-31T14:36:28.641 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /mnt 2024-07-31T14:36:28.641 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /var/tmp 2024-07-31T14:36:28.641 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /home 2024-07-31T14:36:28.641 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /root 2024-07-31T14:36:28.641 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /tmp 2024-07-31T14:36:28.641 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:36:28.822 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-volume-2:19.0.0-5223.g8c796fca.el9.noarch 113/172 2024-07-31T14:36:28.837 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-volume-2:19.0.0-5223.g8c796fca.el9.noarch 113/172 2024-07-31T14:36:28.838 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:28.838 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-31T14:36:28.838 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:36:30.408 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-2:19.0.0-5223.g8c796fca.el9.x86_64 114/172 2024-07-31T14:36:30.434 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-2:19.0.0-5223.g8c796fca.el9.x86_64 114/172 2024-07-31T14:36:30.434 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:30.434 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-31T14:36:30.434 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-31T14:36:30.434 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-31T14:36:30.434 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:36:30.659 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mds-2:19.0.0-5223.g8c796fca.el9.x86_64 115/172 2024-07-31T14:36:30.681 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mds-2:19.0.0-5223.g8c796fca.el9.x86_64 115/172 2024-07-31T14:36:30.681 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:30.681 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-31T14:36:30.682 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-31T14:36:30.682 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-31T14:36:30.682 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:36:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mon-2:19.0.0-5223.g8c796fca.el9.x86_64 116/172 2024-07-31T14:36:31.174 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mon-2:19.0.0-5223.g8c796fca.el9.x86_64 116/172 2024-07-31T14:36:31.174 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:31.174 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-31T14:36:31.174 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-31T14:36:31.174 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-31T14:36:31.174 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:36:31.281 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-31T14:36:31.338 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-31T14:36:31.393 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-31T14:36:31.580 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-31T14:36:31.634 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-31T14:36:31.714 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-31T14:36:31.819 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-31T14:36:31.957 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-31T14:36:32.001 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-31T14:36:32.034 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-31T14:36:32.061 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-31T14:36:32.539 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 128/172 2024-07-31T14:36:32.567 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 128/172 2024-07-31T14:36:32.600 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5223.g8c796fca.e 129/172 2024-07-31T14:36:32.665 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5223.g8c796fca. 130/172 2024-07-31T14:36:32.728 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5223.g8c796fca.el9.noa 131/172 2024-07-31T14:36:35.860 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5223.g8c796fca.el9.noa 131/172 2024-07-31T14:36:35.914 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5223.g8c796fca.el9.n 132/172 2024-07-31T14:36:36.880 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5223.g8c796fca.el9.n 132/172 2024-07-31T14:36:36.905 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5223.g8c7 133/172 2024-07-31T14:36:37.017 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5223.g8c7 133/172 2024-07-31T14:36:37.189 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5223.g8c796fca.el 134/172 2024-07-31T14:36:37.216 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-2:19.0.0-5223.g8c796fca.el9.x86_64 135/172 2024-07-31T14:36:37.239 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5223.g8c796fca.el9.x86_64 135/172 2024-07-31T14:36:37.239 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:37.239 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-31T14:36:37.239 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-31T14:36:37.239 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-31T14:36:37.239 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:36:37.283 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-rook-2:19.0.0-5223.g8c796fca.el9.noarch 136/172 2024-07-31T14:36:37.294 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5223.g8c796fca.el9.noarch 136/172 2024-07-31T14:36:37.394 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-2:19.0.0-5223.g8c796fca.el9.x86_64 137/172 2024-07-31T14:36:37.448 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 138/172 2024-07-31T14:36:37.513 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5223.g8c796fca.el9.x86_64 112/172 2024-07-31T14:36:37.514 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /sys 2024-07-31T14:36:37.514 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /proc 2024-07-31T14:36:37.514 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /mnt 2024-07-31T14:36:37.514 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /var/tmp 2024-07-31T14:36:37.514 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /home 2024-07-31T14:36:37.514 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /root 2024-07-31T14:36:37.514 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /tmp 2024-07-31T14:36:37.514 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:36:37.693 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-volume-2:19.0.0-5223.g8c796fca.el9.noarch 113/172 2024-07-31T14:36:37.709 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-volume-2:19.0.0-5223.g8c796fca.el9.noarch 113/172 2024-07-31T14:36:37.709 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:37.709 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-31T14:36:37.709 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:36:38.411 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-31T14:36:38.444 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-radosgw-2:19.0.0-5223.g8c796fca.el9.x86_64 140/172 2024-07-31T14:36:38.467 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5223.g8c796fca.el9.x86_64 140/172 2024-07-31T14:36:38.467 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:38.467 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-31T14:36:38.467 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-31T14:36:38.467 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-31T14:36:38.468 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:36:38.511 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5223.g8c796 141/172 2024-07-31T14:36:38.533 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5223.g8c796 141/172 2024-07-31T14:36:38.533 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:38.533 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-31T14:36:38.534 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:36:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5223.g8c796fca.el9.x86_64 112/172 2024-07-31T14:36:38.545 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /sys 2024-07-31T14:36:38.545 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /proc 2024-07-31T14:36:38.545 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /mnt 2024-07-31T14:36:38.545 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /var/tmp 2024-07-31T14:36:38.545 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /home 2024-07-31T14:36:38.545 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /root 2024-07-31T14:36:38.545 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /tmp 2024-07-31T14:36:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:36:38.578 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 142/172 2024-07-31T14:36:38.600 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 142/172 2024-07-31T14:36:38.600 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:38.601 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-31T14:36:38.601 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-31T14:36:38.601 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-31T14:36:38.601 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:36:38.714 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-volume-2:19.0.0-5223.g8c796fca.el9.noarch 113/172 2024-07-31T14:36:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-volume-2:19.0.0-5223.g8c796fca.el9.noarch 113/172 2024-07-31T14:36:38.730 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:38.730 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-31T14:36:38.730 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:36:38.946 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 143/172 2024-07-31T14:36:38.968 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 143/172 2024-07-31T14:36:38.968 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:38.968 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-31T14:36:38.968 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-31T14:36:38.968 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-31T14:36:38.969 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:36:39.354 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-osd-2:19.0.0-5223.g8c796fca.el9.x86_64 114/172 2024-07-31T14:36:39.382 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-osd-2:19.0.0-5223.g8c796fca.el9.x86_64 114/172 2024-07-31T14:36:39.383 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:39.383 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-31T14:36:39.383 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-31T14:36:39.383 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-31T14:36:39.383 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:36:39.630 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mds-2:19.0.0-5223.g8c796fca.el9.x86_64 115/172 2024-07-31T14:36:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mds-2:19.0.0-5223.g8c796fca.el9.x86_64 115/172 2024-07-31T14:36:39.654 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:39.654 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-31T14:36:39.654 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-31T14:36:39.654 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-31T14:36:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:36:40.147 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mon-2:19.0.0-5223.g8c796fca.el9.x86_64 116/172 2024-07-31T14:36:40.171 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mon-2:19.0.0-5223.g8c796fca.el9.x86_64 116/172 2024-07-31T14:36:40.172 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:40.172 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-31T14:36:40.172 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-31T14:36:40.172 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-31T14:36:40.172 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:36:40.278 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-31T14:36:40.317 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-osd-2:19.0.0-5223.g8c796fca.el9.x86_64 114/172 2024-07-31T14:36:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-31T14:36:40.343 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-osd-2:19.0.0-5223.g8c796fca.el9.x86_64 114/172 2024-07-31T14:36:40.344 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:40.344 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-31T14:36:40.344 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-31T14:36:40.344 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-31T14:36:40.344 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:36:40.399 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-31T14:36:40.568 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mds-2:19.0.0-5223.g8c796fca.el9.x86_64 115/172 2024-07-31T14:36:40.591 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mds-2:19.0.0-5223.g8c796fca.el9.x86_64 115/172 2024-07-31T14:36:40.591 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:40.591 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-31T14:36:40.591 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-31T14:36:40.591 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-31T14:36:40.591 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:36:40.606 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-31T14:36:40.656 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-31T14:36:40.746 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-31T14:36:40.845 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-31T14:36:40.986 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-31T14:36:41.031 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-31T14:36:41.052 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mon-2:19.0.0-5223.g8c796fca.el9.x86_64 116/172 2024-07-31T14:36:41.064 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-31T14:36:41.075 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mon-2:19.0.0-5223.g8c796fca.el9.x86_64 116/172 2024-07-31T14:36:41.075 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:41.075 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-31T14:36:41.075 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-31T14:36:41.076 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-31T14:36:41.076 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:36:41.090 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-31T14:36:41.190 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-31T14:36:41.247 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-31T14:36:41.302 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-31T14:36:41.497 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-31T14:36:41.538 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 128/172 2024-07-31T14:36:41.544 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-31T14:36:41.563 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 128/172 2024-07-31T14:36:41.596 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5223.g8c796fca.e 129/172 2024-07-31T14:36:41.622 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-31T14:36:41.663 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5223.g8c796fca. 130/172 2024-07-31T14:36:41.686 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5223.g8c796fca.el9.noa 131/172 2024-07-31T14:36:41.719 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-31T14:36:41.857 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-31T14:36:41.903 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-31T14:36:41.944 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-31T14:36:41.972 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-31T14:36:42.366 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 128/172 2024-07-31T14:36:42.395 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 128/172 2024-07-31T14:36:42.435 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5223.g8c796fca.e 129/172 2024-07-31T14:36:42.500 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5223.g8c796fca. 130/172 2024-07-31T14:36:42.527 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5223.g8c796fca.el9.noa 131/172 2024-07-31T14:36:43.283 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-test-2:19.0.0-5223.g8c796fca.el9.x86_64 144/172 2024-07-31T14:36:43.324 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 145/172 2024-07-31T14:36:43.365 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-nbd-2:19.0.0-5223.g8c796fca.el9.x86_64 146/172 2024-07-31T14:36:43.397 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-31T14:36:43.424 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-31T14:36:43.610 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-shell-2:19.0.0-5223.g8c796fca.el9.x86_64 149/172 2024-07-31T14:36:43.661 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-31T14:36:43.688 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs-devel-2:19.0.0-5223.g8c796fca.el9.x86_ 151/172 2024-07-31T14:36:43.730 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-31T14:36:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-31T14:36:43.798 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-31T14:36:44.468 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-31T14:36:44.680 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-31T14:36:44.699 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-31T14:36:45.034 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5223.g8c796fca.el9.noa 131/172 2024-07-31T14:36:45.100 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5223.g8c796fca.el9.n 132/172 2024-07-31T14:36:45.529 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-top-2:19.0.0-5223.g8c796fca.el9.noarch 156/172 2024-07-31T14:36:45.611 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-31T14:36:45.643 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-31T14:36:45.661 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5223.g8c796fca.el9.noa 131/172 2024-07-31T14:36:45.716 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5223.g8c796fca.el9.n 132/172 2024-07-31T14:36:45.735 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-31T14:36:45.773 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-31T14:36:45.798 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-31T14:36:45.824 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-31T14:36:45.850 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-31T14:36:45.850 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-31T14:36:45.934 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-31T14:36:45.967 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-31T14:36:45.967 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-31T14:36:46.060 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-31T14:36:46.085 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-31T14:36:46.091 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-31T14:36:46.091 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-31T14:36:46.117 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5223.g8c796fca.el9.n 132/172 2024-07-31T14:36:46.150 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5223.g8c7 133/172 2024-07-31T14:36:46.270 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5223.g8c7 133/172 2024-07-31T14:36:46.279 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-31T14:36:46.279 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-31T14:36:46.363 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-31T14:36:46.387 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-31T14:36:46.414 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-31T14:36:46.443 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5223.g8c796fca.el 134/172 2024-07-31T14:36:46.478 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-31T14:36:46.486 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-2:19.0.0-5223.g8c796fca.el9.x86_64 135/172 2024-07-31T14:36:46.511 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5223.g8c796fca.el9.x86_64 135/172 2024-07-31T14:36:46.511 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:46.511 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-31T14:36:46.511 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-31T14:36:46.511 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-31T14:36:46.512 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:36:46.562 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-rook-2:19.0.0-5223.g8c796fca.el9.noarch 136/172 2024-07-31T14:36:46.573 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5223.g8c796fca.el9.noarch 136/172 2024-07-31T14:36:46.673 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5223.g8c796fca.el9.n 132/172 2024-07-31T14:36:46.677 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-2:19.0.0-5223.g8c796fca.el9.x86_64 137/172 2024-07-31T14:36:46.700 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5223.g8c7 133/172 2024-07-31T14:36:46.766 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 138/172 2024-07-31T14:36:46.811 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5223.g8c7 133/172 2024-07-31T14:36:46.984 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5223.g8c796fca.el 134/172 2024-07-31T14:36:47.010 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-2:19.0.0-5223.g8c796fca.el9.x86_64 135/172 2024-07-31T14:36:47.034 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5223.g8c796fca.el9.x86_64 135/172 2024-07-31T14:36:47.034 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:47.034 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-31T14:36:47.034 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-31T14:36:47.034 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-31T14:36:47.034 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:36:47.078 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-rook-2:19.0.0-5223.g8c796fca.el9.noarch 136/172 2024-07-31T14:36:47.089 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5223.g8c796fca.el9.noarch 136/172 2024-07-31T14:36:47.188 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-2:19.0.0-5223.g8c796fca.el9.x86_64 137/172 2024-07-31T14:36:47.242 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 138/172 2024-07-31T14:36:47.792 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-31T14:36:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-radosgw-2:19.0.0-5223.g8c796fca.el9.x86_64 140/172 2024-07-31T14:36:47.847 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5223.g8c796fca.el9.x86_64 140/172 2024-07-31T14:36:47.847 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:47.847 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-31T14:36:47.847 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-31T14:36:47.847 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-31T14:36:47.848 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:36:47.889 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5223.g8c796 141/172 2024-07-31T14:36:47.913 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5223.g8c796 141/172 2024-07-31T14:36:47.913 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:47.914 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-31T14:36:47.914 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:36:47.965 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 142/172 2024-07-31T14:36:47.990 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 142/172 2024-07-31T14:36:48.017 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:48.018 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-31T14:36:48.018 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-31T14:36:48.018 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-31T14:36:48.018 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:36:48.197 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-31T14:36:48.230 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-radosgw-2:19.0.0-5223.g8c796fca.el9.x86_64 140/172 2024-07-31T14:36:48.253 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5223.g8c796fca.el9.x86_64 140/172 2024-07-31T14:36:48.253 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:48.253 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-31T14:36:48.253 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-31T14:36:48.253 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-31T14:36:48.253 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:36:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5223.g8c796 141/172 2024-07-31T14:36:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5223.g8c796 141/172 2024-07-31T14:36:48.319 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:48.319 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-31T14:36:48.319 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:36:48.332 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 143/172 2024-07-31T14:36:48.357 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 143/172 2024-07-31T14:36:48.357 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:48.357 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-31T14:36:48.357 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-31T14:36:48.358 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-31T14:36:48.358 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:36:48.364 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 142/172 2024-07-31T14:36:48.387 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 142/172 2024-07-31T14:36:48.387 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:48.387 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-31T14:36:48.387 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-31T14:36:48.387 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-31T14:36:48.387 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:36:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 143/172 2024-07-31T14:36:48.729 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 143/172 2024-07-31T14:36:48.729 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-31T14:36:48.729 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-31T14:36:48.730 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-31T14:36:48.730 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-31T14:36:48.730 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:36:52.640 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-31T14:36:52.640 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-2:19.0.0-5223.g8c796fca.el9.x86_64 1/172 2024-07-31T14:36:52.640 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-base-2:19.0.0-5223.g8c796fca.el9.x86_64 2/172 2024-07-31T14:36:52.640 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 3/172 2024-07-31T14:36:52.640 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 4/172 2024-07-31T14:36:52.641 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5223.g8c796 5/172 2024-07-31T14:36:52.641 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mds-2:19.0.0-5223.g8c796fca.el9.x86_64 6/172 2024-07-31T14:36:52.641 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-2:19.0.0-5223.g8c796fca.el9.x86_64 7/172 2024-07-31T14:36:52.641 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mon-2:19.0.0-5223.g8c796fca.el9.x86_64 8/172 2024-07-31T14:36:52.641 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-osd-2:19.0.0-5223.g8c796fca.el9.x86_64 9/172 2024-07-31T14:36:52.641 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-radosgw-2:19.0.0-5223.g8c796fca.el9.x86_64 10/172 2024-07-31T14:36:52.641 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-selinux-2:19.0.0-5223.g8c796fca.el9.x86_64 11/172 2024-07-31T14:36:52.641 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-test-2:19.0.0-5223.g8c796fca.el9.x86_64 12/172 2024-07-31T14:36:52.641 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 13/172 2024-07-31T14:36:52.641 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-shell-2:19.0.0-5223.g8c796fca.el9.x86_64 14/172 2024-07-31T14:36:52.641 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs-devel-2:19.0.0-5223.g8c796fca.el9.x86_ 15/172 2024-07-31T14:36:52.641 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs2-2:19.0.0-5223.g8c796fca.el9.x86_64 16/172 2024-07-31T14:36:52.642 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephsqlite-2:19.0.0-5223.g8c796fca.el9.x86_64 17/172 2024-07-31T14:36:52.642 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados-devel-2:19.0.0-5223.g8c796fca.el9.x86_6 18/172 2024-07-31T14:36:52.642 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libradosstriper1-2:19.0.0-5223.g8c796fca.el9.x86 19/172 2024-07-31T14:36:52.642 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librgw2-2:19.0.0-5223.g8c796fca.el9.x86_64 20/172 2024-07-31T14:36:52.642 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5223.g8c796fca.el 21/172 2024-07-31T14:36:52.642 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-common-2:19.0.0-5223.g8c796fca.el9. 22/172 2024-07-31T14:36:52.642 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cephfs-2:19.0.0-5223.g8c796fca.el9.x86_6 23/172 2024-07-31T14:36:52.642 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rados-2:19.0.0-5223.g8c796fca.el9.x86_64 24/172 2024-07-31T14:36:52.642 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rbd-2:19.0.0-5223.g8c796fca.el9.x86_64 25/172 2024-07-31T14:36:52.642 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rgw-2:19.0.0-5223.g8c796fca.el9.x86_64 26/172 2024-07-31T14:36:52.642 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 27/172 2024-07-31T14:36:52.643 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 28/172 2024-07-31T14:36:52.643 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-nbd-2:19.0.0-5223.g8c796fca.el9.x86_64 29/172 2024-07-31T14:36:52.643 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5223.g8c796fca. 30/172 2024-07-31T14:36:52.643 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5223.g8c796fca.el9.noa 31/172 2024-07-31T14:36:52.655 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5223.g8c796fca.el9.n 32/172 2024-07-31T14:36:52.656 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5223.g8c7 33/172 2024-07-31T14:36:52.656 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5223.g8c796fca.el 34/172 2024-07-31T14:36:52.656 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5223.g8c796fca.el9.noarch 35/172 2024-07-31T14:36:52.656 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5223.g8c796fca.e 36/172 2024-07-31T14:36:52.656 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-volume-2:19.0.0-5223.g8c796fca.el9.noarch 37/172 2024-07-31T14:36:52.656 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 38/172 2024-07-31T14:36:52.656 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-top-2:19.0.0-5223.g8c796fca.el9.noarch 39/172 2024-07-31T14:36:52.656 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-31T14:36:52.656 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-31T14:36:52.656 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-31T14:36:52.656 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-31T14:36:52.657 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-31T14:36:52.657 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-31T14:36:52.657 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-31T14:36:52.657 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-31T14:36:52.657 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-31T14:36:52.657 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-31T14:36:52.657 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-31T14:36:52.657 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-31T14:36:52.657 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-31T14:36:52.657 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-31T14:36:52.657 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-31T14:36:52.658 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-31T14:36:52.658 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-31T14:36:52.658 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-31T14:36:52.658 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-31T14:36:52.658 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-31T14:36:52.658 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-31T14:36:52.658 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-31T14:36:52.658 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-31T14:36:52.658 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-31T14:36:52.658 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-31T14:36:52.659 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-31T14:36:52.659 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-31T14:36:52.659 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-31T14:36:52.659 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-31T14:36:52.659 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-31T14:36:52.659 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-31T14:36:52.659 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-31T14:36:52.659 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-31T14:36:52.659 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-31T14:36:52.659 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-31T14:36:52.659 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-31T14:36:52.659 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-31T14:36:52.660 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-31T14:36:52.660 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-31T14:36:52.660 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-31T14:36:52.660 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-31T14:36:52.660 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-31T14:36:52.660 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-31T14:36:52.660 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-31T14:36:52.660 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-31T14:36:52.660 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-31T14:36:52.660 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-31T14:36:52.660 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-31T14:36:52.660 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-31T14:36:52.661 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-31T14:36:52.661 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-31T14:36:52.661 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-31T14:36:52.661 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-31T14:36:52.661 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-31T14:36:52.661 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-31T14:36:52.661 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-31T14:36:52.661 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-31T14:36:52.661 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-31T14:36:52.661 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-31T14:36:52.661 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-31T14:36:52.661 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-31T14:36:52.661 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-31T14:36:52.661 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-31T14:36:52.662 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-31T14:36:52.662 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-31T14:36:52.662 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-31T14:36:52.662 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-31T14:36:52.662 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-31T14:36:52.662 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-31T14:36:52.662 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-31T14:36:52.662 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-31T14:36:52.662 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-31T14:36:52.662 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-31T14:36:52.662 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-31T14:36:52.662 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-31T14:36:52.662 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-31T14:36:52.663 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-31T14:36:52.663 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-31T14:36:52.663 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-31T14:36:52.663 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-31T14:36:52.663 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-31T14:36:52.663 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-31T14:36:52.663 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-31T14:36:52.663 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-31T14:36:52.663 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-31T14:36:52.663 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-31T14:36:52.663 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-31T14:36:52.663 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-31T14:36:52.663 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-31T14:36:52.664 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-31T14:36:52.664 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-31T14:36:52.664 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-31T14:36:52.664 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-31T14:36:52.664 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-31T14:36:52.664 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-31T14:36:52.664 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-31T14:36:52.664 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-31T14:36:52.664 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-31T14:36:52.664 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-31T14:36:52.665 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-31T14:36:52.665 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-31T14:36:52.665 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-31T14:36:52.665 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/172 2024-07-31T14:36:52.665 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-31T14:36:52.665 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-31T14:36:52.665 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-31T14:36:52.665 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-31T14:36:52.665 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-31T14:36:52.665 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-31T14:36:52.665 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:19.0.0-5223.g8c796fca.el9.x86_64 149/172 2024-07-31T14:36:52.665 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-31T14:36:52.665 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:19.0.0-5223.g8c796fca.el9.x86_64 151/172 2024-07-31T14:36:52.666 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-31T14:36:52.666 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-31T14:36:52.666 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-31T14:36:52.666 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-31T14:36:52.666 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-31T14:36:52.666 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-31T14:36:52.666 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-31T14:36:52.666 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-31T14:36:52.666 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-31T14:36:52.666 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-31T14:36:52.666 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-31T14:36:52.666 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-31T14:36:52.667 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-31T14:36:52.667 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-31T14:36:52.667 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-31T14:36:52.667 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-31T14:36:52.667 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-31T14:36:52.667 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-31T14:36:52.667 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-31T14:36:52.667 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-31T14:36:52.945 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-test-2:19.0.0-5223.g8c796fca.el9.x86_64 144/172 2024-07-31T14:36:52.987 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 145/172 2024-07-31T14:36:53.027 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-nbd-2:19.0.0-5223.g8c796fca.el9.x86_64 146/172 2024-07-31T14:36:53.052 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-test-2:19.0.0-5223.g8c796fca.el9.x86_64 144/172 2024-07-31T14:36:53.059 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-31T14:36:53.086 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-31T14:36:53.093 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 145/172 2024-07-31T14:36:53.134 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-nbd-2:19.0.0-5223.g8c796fca.el9.x86_64 146/172 2024-07-31T14:36:53.169 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-31T14:36:53.198 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-31T14:36:53.292 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-shell-2:19.0.0-5223.g8c796fca.el9.x86_64 149/172 2024-07-31T14:36:53.360 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-31T14:36:53.380 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-shell-2:19.0.0-5223.g8c796fca.el9.x86_64 149/172 2024-07-31T14:36:53.390 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs-devel-2:19.0.0-5223.g8c796fca.el9.x86_ 151/172 2024-07-31T14:36:53.428 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-31T14:36:53.438 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-31T14:36:53.460 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs-devel-2:19.0.0-5223.g8c796fca.el9.x86_ 151/172 2024-07-31T14:36:53.468 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-31T14:36:53.501 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-31T14:36:53.514 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-31T14:36:53.531 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-31T14:36:53.586 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-31T14:36:54.175 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-31T14:36:54.175 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-31T14:36:54.213 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-31T14:36:54.219 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-31T14:36:54.232 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-31T14:36:54.241 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-31T14:36:55.061 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-top-2:19.0.0-5223.g8c796fca.el9.noarch 156/172 2024-07-31T14:36:55.144 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-31T14:36:55.155 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-top-2:19.0.0-5223.g8c796fca.el9.noarch 156/172 2024-07-31T14:36:55.185 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-31T14:36:55.249 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-31T14:36:55.266 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-31T14:36:55.283 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-31T14:36:55.305 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-31T14:36:55.330 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-31T14:36:55.355 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-31T14:36:55.382 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-31T14:36:55.382 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-31T14:36:55.385 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-31T14:36:55.418 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-31T14:36:55.444 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-31T14:36:55.469 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-31T14:36:55.475 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-31T14:36:55.495 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-31T14:36:55.495 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-31T14:36:55.509 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-31T14:36:55.509 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-31T14:36:55.580 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-31T14:36:55.586 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-31T14:36:55.611 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-31T14:36:55.613 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-31T14:36:55.613 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-31T14:36:55.616 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-31T14:36:55.616 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-31T14:36:55.680 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-31T14:36:55.680 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:36:55.680 INFO:teuthology.orchestra.run.smithi077.stdout:Upgraded: 2024-07-31T14:36:55.680 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-31T14:36:55.680 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-31T14:36:55.681 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-31T14:36:55.681 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-31T14:36:55.681 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-31T14:36:55.681 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-31T14:36:55.681 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-31T14:36:55.681 INFO:teuthology.orchestra.run.smithi077.stdout: librados2-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.681 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.681 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-31T14:36:55.681 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-31T14:36:55.681 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-31T14:36:55.681 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-07-31T14:36:55.681 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-31T14:36:55.681 INFO:teuthology.orchestra.run.smithi077.stdout: automake-1.16.2-8.el9.noarch 2024-07-31T14:36:55.682 INFO:teuthology.orchestra.run.smithi077.stdout: bison-3.7.4-5.el9.x86_64 2024-07-31T14:36:55.682 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-31T14:36:55.682 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.682 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.682 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.682 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.682 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:36:55.682 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.682 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.682 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.682 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:36:55.682 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:36:55.683 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:36:55.683 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:36:55.683 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:36:55.683 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.683 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.683 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:36:55.683 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.683 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.683 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.683 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:36:55.683 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:36:55.683 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.683 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.684 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:36:55.684 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-31T14:36:55.684 INFO:teuthology.orchestra.run.smithi077.stdout: flex-2.6.4-9.el9.x86_64 2024-07-31T14:36:55.684 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-31T14:36:55.684 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-31T14:36:55.684 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-31T14:36:55.684 INFO:teuthology.orchestra.run.smithi077.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-31T14:36:55.684 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-31T14:36:55.684 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-31T14:36:55.684 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-31T14:36:55.684 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-31T14:36:55.684 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-31T14:36:55.684 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.685 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.685 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.685 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-31T14:36:55.685 INFO:teuthology.orchestra.run.smithi077.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-31T14:36:55.685 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-31T14:36:55.685 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-31T14:36:55.685 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-31T14:36:55.685 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.685 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.685 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-31T14:36:55.685 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.685 INFO:teuthology.orchestra.run.smithi077.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-31T14:36:55.686 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-31T14:36:55.686 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-31T14:36:55.686 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-31T14:36:55.686 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-31T14:36:55.686 INFO:teuthology.orchestra.run.smithi077.stdout: lua-5.4.4-4.el9.x86_64 2024-07-31T14:36:55.686 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-31T14:36:55.686 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-31T14:36:55.686 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-31T14:36:55.686 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-31T14:36:55.686 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-31T14:36:55.686 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-31T14:36:55.686 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-13.14-1.el9.x86_64 2024-07-31T14:36:55.686 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-31T14:36:55.687 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-31T14:36:55.687 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-31T14:36:55.687 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-31T14:36:55.687 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-31T14:36:55.687 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-31T14:36:55.687 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-31T14:36:55.687 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-31T14:36:55.687 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-31T14:36:55.687 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-31T14:36:55.687 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.687 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.687 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-31T14:36:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-31T14:36:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-31T14:36:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-31T14:36:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-31T14:36:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-31T14:36:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-31T14:36:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-31T14:36:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-31T14:36:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-31T14:36:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-31T14:36:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-31T14:36:55.688 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-31T14:36:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-31T14:36:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-31T14:36:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-31T14:36:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-31T14:36:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-31T14:36:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-31T14:36:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-31T14:36:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-31T14:36:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-31T14:36:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-31T14:36:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-31T14:36:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-31T14:36:55.689 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-31T14:36:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-31T14:36:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-31T14:36:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-31T14:36:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-31T14:36:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply-3.11-14.el9.noarch 2024-07-31T14:36:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-31T14:36:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-31T14:36:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-31T14:36:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-31T14:36:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-31T14:36:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-31T14:36:55.690 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-31T14:36:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-31T14:36:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-31T14:36:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-31T14:36:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-31T14:36:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-31T14:36:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-31T14:36:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-31T14:36:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-31T14:36:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-31T14:36:55.691 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-31T14:36:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-07-31T14:36:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-31T14:36:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-31T14:36:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-31T14:36:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-31T14:36:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-31T14:36:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-31T14:36:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-31T14:36:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-31T14:36:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-31T14:36:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.692 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:36:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: re2-1:20211101-3.el9.x86_64 2024-07-31T14:36:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-31T14:36:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-31T14:36:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-31T14:36:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-31T14:36:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-31T14:36:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-31T14:36:55.693 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:36:55.693 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-07-31T14:36:55.697 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-31T14:36:55.722 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-31T14:36:55.728 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-31T14:36:55.728 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-31T14:36:55.796 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-31T14:36:55.796 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-31T14:36:55.889 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-31T14:36:55.913 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-31T14:36:55.923 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-31T14:36:55.923 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-31T14:36:55.939 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-31T14:36:56.007 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-31T14:36:56.012 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-31T14:36:56.032 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-31T14:36:56.058 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-31T14:36:56.094 DEBUG:teuthology.parallel:result is None 2024-07-31T14:36:56.143 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-31T14:37:02.275 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-31T14:37:02.276 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-2:19.0.0-5223.g8c796fca.el9.x86_64 1/172 2024-07-31T14:37:02.276 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-base-2:19.0.0-5223.g8c796fca.el9.x86_64 2/172 2024-07-31T14:37:02.276 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 3/172 2024-07-31T14:37:02.276 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 4/172 2024-07-31T14:37:02.276 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5223.g8c796 5/172 2024-07-31T14:37:02.276 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mds-2:19.0.0-5223.g8c796fca.el9.x86_64 6/172 2024-07-31T14:37:02.276 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-2:19.0.0-5223.g8c796fca.el9.x86_64 7/172 2024-07-31T14:37:02.276 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mon-2:19.0.0-5223.g8c796fca.el9.x86_64 8/172 2024-07-31T14:37:02.276 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-osd-2:19.0.0-5223.g8c796fca.el9.x86_64 9/172 2024-07-31T14:37:02.276 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-radosgw-2:19.0.0-5223.g8c796fca.el9.x86_64 10/172 2024-07-31T14:37:02.276 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-selinux-2:19.0.0-5223.g8c796fca.el9.x86_64 11/172 2024-07-31T14:37:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-test-2:19.0.0-5223.g8c796fca.el9.x86_64 12/172 2024-07-31T14:37:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 13/172 2024-07-31T14:37:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-shell-2:19.0.0-5223.g8c796fca.el9.x86_64 14/172 2024-07-31T14:37:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs-devel-2:19.0.0-5223.g8c796fca.el9.x86_ 15/172 2024-07-31T14:37:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs2-2:19.0.0-5223.g8c796fca.el9.x86_64 16/172 2024-07-31T14:37:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephsqlite-2:19.0.0-5223.g8c796fca.el9.x86_64 17/172 2024-07-31T14:37:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados-devel-2:19.0.0-5223.g8c796fca.el9.x86_6 18/172 2024-07-31T14:37:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libradosstriper1-2:19.0.0-5223.g8c796fca.el9.x86 19/172 2024-07-31T14:37:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librgw2-2:19.0.0-5223.g8c796fca.el9.x86_64 20/172 2024-07-31T14:37:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5223.g8c796fca.el 21/172 2024-07-31T14:37:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-common-2:19.0.0-5223.g8c796fca.el9. 22/172 2024-07-31T14:37:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cephfs-2:19.0.0-5223.g8c796fca.el9.x86_6 23/172 2024-07-31T14:37:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rados-2:19.0.0-5223.g8c796fca.el9.x86_64 24/172 2024-07-31T14:37:02.278 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rbd-2:19.0.0-5223.g8c796fca.el9.x86_64 25/172 2024-07-31T14:37:02.278 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rgw-2:19.0.0-5223.g8c796fca.el9.x86_64 26/172 2024-07-31T14:37:02.291 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 27/172 2024-07-31T14:37:02.291 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 28/172 2024-07-31T14:37:02.291 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-nbd-2:19.0.0-5223.g8c796fca.el9.x86_64 29/172 2024-07-31T14:37:02.291 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5223.g8c796fca. 30/172 2024-07-31T14:37:02.291 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5223.g8c796fca.el9.noa 31/172 2024-07-31T14:37:02.291 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5223.g8c796fca.el9.n 32/172 2024-07-31T14:37:02.291 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5223.g8c7 33/172 2024-07-31T14:37:02.292 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5223.g8c796fca.el 34/172 2024-07-31T14:37:02.292 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5223.g8c796fca.el9.noarch 35/172 2024-07-31T14:37:02.292 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5223.g8c796fca.e 36/172 2024-07-31T14:37:02.292 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-volume-2:19.0.0-5223.g8c796fca.el9.noarch 37/172 2024-07-31T14:37:02.292 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 38/172 2024-07-31T14:37:02.292 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-top-2:19.0.0-5223.g8c796fca.el9.noarch 39/172 2024-07-31T14:37:02.292 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-31T14:37:02.292 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-31T14:37:02.292 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-31T14:37:02.292 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-31T14:37:02.292 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-31T14:37:02.293 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-31T14:37:02.293 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-31T14:37:02.293 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-31T14:37:02.293 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-31T14:37:02.293 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-31T14:37:02.293 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-31T14:37:02.293 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-31T14:37:02.293 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-31T14:37:02.293 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-31T14:37:02.293 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-31T14:37:02.293 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-31T14:37:02.293 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-31T14:37:02.294 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-31T14:37:02.294 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-31T14:37:02.294 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-31T14:37:02.294 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-31T14:37:02.294 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-31T14:37:02.294 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-31T14:37:02.294 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-31T14:37:02.294 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-31T14:37:02.294 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-31T14:37:02.294 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-31T14:37:02.294 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-31T14:37:02.295 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-31T14:37:02.295 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-31T14:37:02.295 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-31T14:37:02.295 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-31T14:37:02.295 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-31T14:37:02.295 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-31T14:37:02.295 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-31T14:37:02.295 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-31T14:37:02.295 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-31T14:37:02.295 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-31T14:37:02.295 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-31T14:37:02.295 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-31T14:37:02.295 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-31T14:37:02.296 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-31T14:37:02.296 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-31T14:37:02.296 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-31T14:37:02.296 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-31T14:37:02.296 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-31T14:37:02.296 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-31T14:37:02.296 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-31T14:37:02.296 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-31T14:37:02.296 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-31T14:37:02.296 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-31T14:37:02.296 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-31T14:37:02.296 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-31T14:37:02.297 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-31T14:37:02.297 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-31T14:37:02.297 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-31T14:37:02.297 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-31T14:37:02.297 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-31T14:37:02.297 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-31T14:37:02.297 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-31T14:37:02.298 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-31T14:37:02.298 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-31T14:37:02.298 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-31T14:37:02.298 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-31T14:37:02.298 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-31T14:37:02.298 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-31T14:37:02.298 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-31T14:37:02.298 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-31T14:37:02.298 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-31T14:37:02.298 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-31T14:37:02.298 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-31T14:37:02.298 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-31T14:37:02.299 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-31T14:37:02.299 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-31T14:37:02.299 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-31T14:37:02.299 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-31T14:37:02.299 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-31T14:37:02.299 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-31T14:37:02.299 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-31T14:37:02.299 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-31T14:37:02.299 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-31T14:37:02.299 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-31T14:37:02.299 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-31T14:37:02.299 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-31T14:37:02.300 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-31T14:37:02.300 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-31T14:37:02.300 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-31T14:37:02.300 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-31T14:37:02.300 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-31T14:37:02.300 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-31T14:37:02.300 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-31T14:37:02.300 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-31T14:37:02.300 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-31T14:37:02.300 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-31T14:37:02.300 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-31T14:37:02.300 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-31T14:37:02.300 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-31T14:37:02.301 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-31T14:37:02.301 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-31T14:37:02.301 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-31T14:37:02.301 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-31T14:37:02.301 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-31T14:37:02.301 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/172 2024-07-31T14:37:02.301 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-31T14:37:02.301 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-31T14:37:02.301 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-31T14:37:02.301 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-31T14:37:02.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-31T14:37:02.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-31T14:37:02.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:19.0.0-5223.g8c796fca.el9.x86_64 149/172 2024-07-31T14:37:02.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-31T14:37:02.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:19.0.0-5223.g8c796fca.el9.x86_64 151/172 2024-07-31T14:37:02.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-31T14:37:02.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-31T14:37:02.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-31T14:37:02.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-31T14:37:02.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-31T14:37:02.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-31T14:37:02.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-31T14:37:02.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-31T14:37:02.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-31T14:37:02.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-31T14:37:02.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-31T14:37:02.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-31T14:37:02.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-31T14:37:02.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-31T14:37:02.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-31T14:37:02.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-31T14:37:02.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-31T14:37:02.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-31T14:37:02.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-31T14:37:02.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-31T14:37:02.471 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-31T14:37:02.471 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-2:19.0.0-5223.g8c796fca.el9.x86_64 1/172 2024-07-31T14:37:02.471 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-base-2:19.0.0-5223.g8c796fca.el9.x86_64 2/172 2024-07-31T14:37:02.471 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 3/172 2024-07-31T14:37:02.471 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 4/172 2024-07-31T14:37:02.471 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5223.g8c796 5/172 2024-07-31T14:37:02.471 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mds-2:19.0.0-5223.g8c796fca.el9.x86_64 6/172 2024-07-31T14:37:02.471 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-2:19.0.0-5223.g8c796fca.el9.x86_64 7/172 2024-07-31T14:37:02.471 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mon-2:19.0.0-5223.g8c796fca.el9.x86_64 8/172 2024-07-31T14:37:02.471 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-osd-2:19.0.0-5223.g8c796fca.el9.x86_64 9/172 2024-07-31T14:37:02.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-radosgw-2:19.0.0-5223.g8c796fca.el9.x86_64 10/172 2024-07-31T14:37:02.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-selinux-2:19.0.0-5223.g8c796fca.el9.x86_64 11/172 2024-07-31T14:37:02.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-test-2:19.0.0-5223.g8c796fca.el9.x86_64 12/172 2024-07-31T14:37:02.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 13/172 2024-07-31T14:37:02.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-shell-2:19.0.0-5223.g8c796fca.el9.x86_64 14/172 2024-07-31T14:37:02.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs-devel-2:19.0.0-5223.g8c796fca.el9.x86_ 15/172 2024-07-31T14:37:02.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs2-2:19.0.0-5223.g8c796fca.el9.x86_64 16/172 2024-07-31T14:37:02.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephsqlite-2:19.0.0-5223.g8c796fca.el9.x86_64 17/172 2024-07-31T14:37:02.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados-devel-2:19.0.0-5223.g8c796fca.el9.x86_6 18/172 2024-07-31T14:37:02.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libradosstriper1-2:19.0.0-5223.g8c796fca.el9.x86 19/172 2024-07-31T14:37:02.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librgw2-2:19.0.0-5223.g8c796fca.el9.x86_64 20/172 2024-07-31T14:37:02.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5223.g8c796fca.el 21/172 2024-07-31T14:37:02.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-common-2:19.0.0-5223.g8c796fca.el9. 22/172 2024-07-31T14:37:02.473 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cephfs-2:19.0.0-5223.g8c796fca.el9.x86_6 23/172 2024-07-31T14:37:02.473 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rados-2:19.0.0-5223.g8c796fca.el9.x86_64 24/172 2024-07-31T14:37:02.473 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rbd-2:19.0.0-5223.g8c796fca.el9.x86_64 25/172 2024-07-31T14:37:02.473 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rgw-2:19.0.0-5223.g8c796fca.el9.x86_64 26/172 2024-07-31T14:37:02.473 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 27/172 2024-07-31T14:37:02.473 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 28/172 2024-07-31T14:37:02.473 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-nbd-2:19.0.0-5223.g8c796fca.el9.x86_64 29/172 2024-07-31T14:37:02.486 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5223.g8c796fca. 30/172 2024-07-31T14:37:02.486 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5223.g8c796fca.el9.noa 31/172 2024-07-31T14:37:02.486 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5223.g8c796fca.el9.n 32/172 2024-07-31T14:37:02.486 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5223.g8c7 33/172 2024-07-31T14:37:02.486 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5223.g8c796fca.el 34/172 2024-07-31T14:37:02.486 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5223.g8c796fca.el9.noarch 35/172 2024-07-31T14:37:02.486 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5223.g8c796fca.e 36/172 2024-07-31T14:37:02.487 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-volume-2:19.0.0-5223.g8c796fca.el9.noarch 37/172 2024-07-31T14:37:02.487 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 38/172 2024-07-31T14:37:02.487 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-top-2:19.0.0-5223.g8c796fca.el9.noarch 39/172 2024-07-31T14:37:02.487 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-31T14:37:02.487 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-31T14:37:02.487 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-31T14:37:02.487 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-31T14:37:02.487 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-31T14:37:02.487 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-31T14:37:02.487 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-31T14:37:02.487 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-31T14:37:02.487 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-31T14:37:02.488 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-31T14:37:02.488 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-31T14:37:02.488 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-31T14:37:02.488 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-31T14:37:02.488 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-31T14:37:02.488 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-31T14:37:02.488 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-31T14:37:02.488 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-31T14:37:02.488 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-31T14:37:02.488 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-31T14:37:02.488 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-31T14:37:02.489 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-31T14:37:02.489 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-31T14:37:02.489 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-31T14:37:02.489 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-31T14:37:02.489 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-31T14:37:02.489 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-31T14:37:02.489 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-31T14:37:02.489 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-31T14:37:02.489 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-31T14:37:02.489 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-31T14:37:02.489 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-31T14:37:02.489 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-31T14:37:02.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-31T14:37:02.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-31T14:37:02.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-31T14:37:02.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-31T14:37:02.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-31T14:37:02.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-31T14:37:02.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-31T14:37:02.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-31T14:37:02.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-31T14:37:02.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-31T14:37:02.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-31T14:37:02.490 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-31T14:37:02.491 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-31T14:37:02.491 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-31T14:37:02.491 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-31T14:37:02.491 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-31T14:37:02.491 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-31T14:37:02.491 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-31T14:37:02.491 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-31T14:37:02.491 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-31T14:37:02.491 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-31T14:37:02.491 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-31T14:37:02.491 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-31T14:37:02.491 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-31T14:37:02.492 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-31T14:37:02.492 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-31T14:37:02.492 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-31T14:37:02.492 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-31T14:37:02.492 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-31T14:37:02.492 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-31T14:37:02.492 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-31T14:37:02.492 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-31T14:37:02.492 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-31T14:37:02.492 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-31T14:37:02.492 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-31T14:37:02.492 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-31T14:37:02.492 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-31T14:37:02.493 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-31T14:37:02.493 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-31T14:37:02.493 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-31T14:37:02.493 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-31T14:37:02.493 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-31T14:37:02.493 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-31T14:37:02.493 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-31T14:37:02.493 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-31T14:37:02.493 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-31T14:37:02.493 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-31T14:37:02.493 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-31T14:37:02.493 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-31T14:37:02.494 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-31T14:37:02.494 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-31T14:37:02.494 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-31T14:37:02.494 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-31T14:37:02.494 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-31T14:37:02.494 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-31T14:37:02.494 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-31T14:37:02.494 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-31T14:37:02.494 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-31T14:37:02.494 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-31T14:37:02.494 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-31T14:37:02.494 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-31T14:37:02.494 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-31T14:37:02.495 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-31T14:37:02.495 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-31T14:37:02.495 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-31T14:37:02.495 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-31T14:37:02.495 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-31T14:37:02.495 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-31T14:37:02.495 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-31T14:37:02.495 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-31T14:37:02.495 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/172 2024-07-31T14:37:02.495 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-31T14:37:02.495 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-31T14:37:02.496 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-31T14:37:02.496 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-31T14:37:02.496 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-31T14:37:02.496 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-31T14:37:02.496 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:19.0.0-5223.g8c796fca.el9.x86_64 149/172 2024-07-31T14:37:02.496 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-31T14:37:02.496 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:19.0.0-5223.g8c796fca.el9.x86_64 151/172 2024-07-31T14:37:02.496 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-31T14:37:02.496 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-31T14:37:02.496 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-31T14:37:02.496 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-31T14:37:02.496 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-31T14:37:02.496 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-31T14:37:02.497 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-31T14:37:02.497 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-31T14:37:02.497 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-31T14:37:02.497 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-31T14:37:02.497 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-31T14:37:02.497 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-31T14:37:02.497 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-31T14:37:02.497 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-31T14:37:02.497 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-31T14:37:02.497 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-31T14:37:02.497 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-31T14:37:02.497 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-31T14:37:02.497 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-31T14:37:02.498 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-31T14:37:05.112 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-31T14:37:05.113 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:37:05.113 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2024-07-31T14:37:05.113 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-31T14:37:05.113 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-31T14:37:05.113 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-31T14:37:05.113 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-31T14:37:05.113 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-31T14:37:05.113 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-31T14:37:05.113 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-31T14:37:05.113 INFO:teuthology.orchestra.run.smithi067.stdout: librados2-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.113 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.114 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-31T14:37:05.114 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-31T14:37:05.114 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-31T14:37:05.114 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-07-31T14:37:05.114 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-31T14:37:05.114 INFO:teuthology.orchestra.run.smithi067.stdout: automake-1.16.2-8.el9.noarch 2024-07-31T14:37:05.114 INFO:teuthology.orchestra.run.smithi067.stdout: bison-3.7.4-5.el9.x86_64 2024-07-31T14:37:05.114 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-31T14:37:05.114 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.114 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.114 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.114 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.114 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.115 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.115 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.115 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.115 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.115 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.115 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.115 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.115 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.115 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.115 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.115 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.116 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.116 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.116 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.116 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.116 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.116 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.116 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-shell-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.116 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.116 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-31T14:37:05.116 INFO:teuthology.orchestra.run.smithi067.stdout: flex-2.6.4-9.el9.x86_64 2024-07-31T14:37:05.116 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-31T14:37:05.116 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-31T14:37:05.116 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-31T14:37:05.116 INFO:teuthology.orchestra.run.smithi067.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-31T14:37:05.117 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-31T14:37:05.117 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-31T14:37:05.117 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-31T14:37:05.117 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-31T14:37:05.117 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-31T14:37:05.117 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.117 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.117 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.117 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-31T14:37:05.117 INFO:teuthology.orchestra.run.smithi067.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-31T14:37:05.117 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-31T14:37:05.117 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-31T14:37:05.117 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-31T14:37:05.117 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.118 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.118 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-31T14:37:05.118 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.118 INFO:teuthology.orchestra.run.smithi067.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-31T14:37:05.118 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-31T14:37:05.118 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-31T14:37:05.118 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-31T14:37:05.118 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-31T14:37:05.118 INFO:teuthology.orchestra.run.smithi067.stdout: lua-5.4.4-4.el9.x86_64 2024-07-31T14:37:05.118 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-31T14:37:05.118 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-31T14:37:05.118 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-31T14:37:05.118 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-31T14:37:05.119 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-31T14:37:05.119 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-31T14:37:05.119 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-13.14-1.el9.x86_64 2024-07-31T14:37:05.119 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-31T14:37:05.119 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-31T14:37:05.119 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-31T14:37:05.119 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-31T14:37:05.119 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-31T14:37:05.119 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-31T14:37:05.119 INFO:teuthology.orchestra.run.smithi067.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-31T14:37:05.119 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-31T14:37:05.119 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-31T14:37:05.119 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-31T14:37:05.120 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.120 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.120 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.120 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-31T14:37:05.120 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-31T14:37:05.120 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-31T14:37:05.120 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-31T14:37:05.120 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-31T14:37:05.120 INFO:teuthology.orchestra.run.smithi067.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-31T14:37:05.120 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-31T14:37:05.120 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-31T14:37:05.120 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-31T14:37:05.120 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-31T14:37:05.120 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-31T14:37:05.121 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-31T14:37:05.122 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-31T14:37:05.122 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-31T14:37:05.122 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-31T14:37:05.122 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-31T14:37:05.122 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply-3.11-14.el9.noarch 2024-07-31T14:37:05.122 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-31T14:37:05.122 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-31T14:37:05.122 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-31T14:37:05.122 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-31T14:37:05.122 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-31T14:37:05.122 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-31T14:37:05.122 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-31T14:37:05.123 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-31T14:37:05.123 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-31T14:37:05.123 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.123 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.123 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-31T14:37:05.123 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-31T14:37:05.123 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-31T14:37:05.123 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.123 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-31T14:37:05.123 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-31T14:37:05.123 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-31T14:37:05.123 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-31T14:37:05.123 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-31T14:37:05.124 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-07-31T14:37:05.124 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-31T14:37:05.124 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-31T14:37:05.124 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-31T14:37:05.124 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-31T14:37:05.124 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-31T14:37:05.124 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-31T14:37:05.124 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-31T14:37:05.124 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-31T14:37:05.124 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-31T14:37:05.124 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.124 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.124 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.124 INFO:teuthology.orchestra.run.smithi067.stdout: re2-1:20211101-3.el9.x86_64 2024-07-31T14:37:05.125 INFO:teuthology.orchestra.run.smithi067.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-31T14:37:05.125 INFO:teuthology.orchestra.run.smithi067.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-31T14:37:05.125 INFO:teuthology.orchestra.run.smithi067.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-31T14:37:05.125 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-31T14:37:05.125 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-31T14:37:05.125 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-31T14:37:05.125 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:37:05.125 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-07-31T14:37:05.206 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-31T14:37:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:37:05.207 INFO:teuthology.orchestra.run.smithi073.stdout:Upgraded: 2024-07-31T14:37:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-31T14:37:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-31T14:37:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-31T14:37:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-31T14:37:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-31T14:37:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-31T14:37:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-31T14:37:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: librados2-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-31T14:37:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-31T14:37:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-31T14:37:05.208 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-07-31T14:37:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-31T14:37:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: automake-1.16.2-8.el9.noarch 2024-07-31T14:37:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: bison-3.7.4-5.el9.x86_64 2024-07-31T14:37:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-31T14:37:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-shell-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top-2:19.0.0-5223.g8c796fca.el9.noarch 2024-07-31T14:37:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-31T14:37:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: flex-2.6.4-9.el9.x86_64 2024-07-31T14:37:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-31T14:37:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-31T14:37:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-31T14:37:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-31T14:37:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-31T14:37:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-31T14:37:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-31T14:37:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-31T14:37:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-31T14:37:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-31T14:37:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-31T14:37:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-31T14:37:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-31T14:37:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-31T14:37:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: lua-5.4.4-4.el9.x86_64 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-13.14-1.el9.x86_64 2024-07-31T14:37:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-31T14:37:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-31T14:37:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-31T14:37:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-31T14:37:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-31T14:37:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-31T14:37:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-31T14:37:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-31T14:37:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-31T14:37:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-31T14:37:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-31T14:37:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-31T14:37:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-31T14:37:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-31T14:37:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-31T14:37:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-31T14:37:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-31T14:37:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-31T14:37:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-31T14:37:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-31T14:37:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-31T14:37:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-31T14:37:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-31T14:37:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-31T14:37:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-31T14:37:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-31T14:37:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-31T14:37:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-31T14:37:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-31T14:37:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply-3.11-14.el9.noarch 2024-07-31T14:37:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-31T14:37:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-31T14:37:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-31T14:37:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-31T14:37:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-31T14:37:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-31T14:37:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-31T14:37:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-31T14:37:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-31T14:37:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-31T14:37:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-31T14:37:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-31T14:37:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-31T14:37:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-31T14:37:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-31T14:37:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-31T14:37:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-31T14:37:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-07-31T14:37:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-31T14:37:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-31T14:37:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-31T14:37:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-31T14:37:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-31T14:37:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-31T14:37:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-31T14:37:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-31T14:37:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-31T14:37:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd-2:19.0.0-5223.g8c796fca.el9.x86_64 2024-07-31T14:37:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: re2-1:20211101-3.el9.x86_64 2024-07-31T14:37:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-31T14:37:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-31T14:37:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-31T14:37:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-31T14:37:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-31T14:37:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-31T14:37:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-07-31T14:37:05.219 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-07-31T14:37:05.555 DEBUG:teuthology.parallel:result is None 2024-07-31T14:37:05.648 DEBUG:teuthology.parallel:result is None 2024-07-31T14:37:05.649 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8c796fca5ff5960f7542e7227bad4a784ee2c804 2024-07-31T14:37:05.798 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-31T14:37:05.823 INFO:teuthology.orchestra.run.smithi067.stdout:19.0.0-5223.g8c796fca.el9 2024-07-31T14:37:05.823 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5223.g8c796fca.el9 2024-07-31T14:37:05.823 INFO:teuthology.task.install:The correct ceph version 19.0.0-5223.g8c796fca is installed. 2024-07-31T14:37:05.825 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8c796fca5ff5960f7542e7227bad4a784ee2c804 2024-07-31T14:37:05.967 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-31T14:37:05.989 INFO:teuthology.orchestra.run.smithi073.stdout:19.0.0-5223.g8c796fca.el9 2024-07-31T14:37:05.989 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5223.g8c796fca.el9 2024-07-31T14:37:05.990 INFO:teuthology.task.install:The correct ceph version 19.0.0-5223.g8c796fca is installed. 2024-07-31T14:37:05.992 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8c796fca5ff5960f7542e7227bad4a784ee2c804 2024-07-31T14:37:06.145 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-31T14:37:06.171 INFO:teuthology.orchestra.run.smithi077.stdout:19.0.0-5223.g8c796fca.el9 2024-07-31T14:37:06.171 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5223.g8c796fca.el9 2024-07-31T14:37:06.171 INFO:teuthology.task.install:The correct ceph version 19.0.0-5223.g8c796fca is installed. 2024-07-31T14:37:06.173 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-07-31T14:37:06.173 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:37:06.174 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-31T14:37:06.206 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-07-31T14:37:06.206 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-31T14:37:06.234 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-07-31T14:37:06.234 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-31T14:37:06.264 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-07-31T14:37:06.264 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:37:06.265 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/daemon-helper 2024-07-31T14:37:06.294 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-31T14:37:06.366 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-07-31T14:37:06.366 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/daemon-helper 2024-07-31T14:37:06.394 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-31T14:37:06.458 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-07-31T14:37:06.458 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/daemon-helper 2024-07-31T14:37:06.487 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-31T14:37:06.554 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-07-31T14:37:06.554 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:37:06.554 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-31T14:37:06.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-31T14:37:06.656 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-07-31T14:37:06.656 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-31T14:37:06.680 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-31T14:37:06.744 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-07-31T14:37:06.744 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-31T14:37:06.771 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-31T14:37:06.837 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-07-31T14:37:06.837 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:37:06.837 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/stdin-killer 2024-07-31T14:37:06.869 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-31T14:37:06.939 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-07-31T14:37:06.939 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/stdin-killer 2024-07-31T14:37:06.963 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-31T14:37:07.027 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-07-31T14:37:07.027 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/stdin-killer 2024-07-31T14:37:07.055 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-31T14:37:07.122 INFO:teuthology.run_tasks:Running task cephadm... 2024-07-31T14:37:07.210 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug cephsqlite': 20, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug'}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_export_ephemeral_random_max': 0.1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '8c796fca5ff5960f7542e7227bad4a784ee2c804', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-07-31T14:37:07.211 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 2024-07-31T14:37:07.211 INFO:tasks.cephadm:Cluster fsid is 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:37:07.211 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-07-31T14:37:07.212 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.67', 'mon.b': '172.21.15.73', 'mon.c': '172.21.15.77'} 2024-07-31T14:37:07.212 INFO:tasks.cephadm:First mon is mon.a on smithi067 2024-07-31T14:37:07.212 INFO:tasks.cephadm:First mgr is x 2024-07-31T14:37:07.212 INFO:tasks.cephadm:Normalizing hostnames... 2024-07-31T14:37:07.212 DEBUG:teuthology.orchestra.run.smithi067:> sudo hostname $(hostname -s) 2024-07-31T14:37:07.243 DEBUG:teuthology.orchestra.run.smithi073:> sudo hostname $(hostname -s) 2024-07-31T14:37:07.268 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname $(hostname -s) 2024-07-31T14:37:07.293 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-07-31T14:37:07.293 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8c796fca5ff5960f7542e7227bad4a784ee2c804 2024-07-31T14:37:07.437 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8c796fca5ff5960f7542e7227bad4a784ee2c804', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/81465/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-5223-g8c796fca', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-5223.g8c796fca'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-07-23 16:29:31.862822', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240723.143739-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '8c796fca5ff5960f7542e7227bad4a784ee2c804', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/81466/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-5223-g8c796fca', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-5223.g8c796fca'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-07-23 16:15:12.181713', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240723.143739-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-07-31T14:37:07.580 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-vshankar-testing-20240723.143739-debug, sha1 8c796fca5ff5960f7542e7227bad4a784ee2c804 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8c796fca5ff5960f7542e7227bad4a784ee2c804 2024-07-31T14:37:07.581 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/x86_64/flavors/default/cephadm 2024-07-31T14:37:07.581 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/x86_64/flavors/default/cephadm 2024-07-31T14:37:07.581 DEBUG:teuthology.orchestra.run.smithi067:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-31T14:37:08.117 INFO:teuthology.orchestra.run.smithi067.stdout:-rw-r--r--. 1 ubuntu ubuntu 789832 Jul 31 14:37 /home/ubuntu/cephtest/cephadm 2024-07-31T14:37:08.118 DEBUG:teuthology.orchestra.run.smithi073:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-31T14:37:08.642 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r--. 1 ubuntu ubuntu 789832 Jul 31 14:37 /home/ubuntu/cephtest/cephadm 2024-07-31T14:37:08.642 DEBUG:teuthology.orchestra.run.smithi077:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240723.143739-debug/8c796fca5ff5960f7542e7227bad4a784ee2c804/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-31T14:37:09.167 INFO:teuthology.orchestra.run.smithi077.stdout:-rw-r--r--. 1 ubuntu ubuntu 789832 Jul 31 14:37 /home/ubuntu/cephtest/cephadm 2024-07-31T14:37:09.167 DEBUG:teuthology.orchestra.run.smithi067:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-31T14:37:09.188 DEBUG:teuthology.orchestra.run.smithi073:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-31T14:37:09.207 DEBUG:teuthology.orchestra.run.smithi077:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-31T14:37:09.238 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 on all hosts... 2024-07-31T14:37:09.238 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 pull 2024-07-31T14:37:09.240 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 pull 2024-07-31T14:37:09.249 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 pull 2024-07-31T14:37:10.142 INFO:teuthology.orchestra.run.smithi073.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804... 2024-07-31T14:37:10.160 INFO:teuthology.orchestra.run.smithi077.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804... 2024-07-31T14:37:10.280 INFO:teuthology.orchestra.run.smithi067.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804... 2024-07-31T14:37:46.565 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:37:46.566 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-5223-g8c796fca (8c796fca5ff5960f7542e7227bad4a784ee2c804) squid (dev)", 2024-07-31T14:37:46.566 INFO:teuthology.orchestra.run.smithi067.stdout: "image_id": "2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86", 2024-07-31T14:37:46.566 INFO:teuthology.orchestra.run.smithi067.stdout: "repo_digests": [ 2024-07-31T14:37:46.566 INFO:teuthology.orchestra.run.smithi067.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844" 2024-07-31T14:37:46.566 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-31T14:37:46.566 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:38:08.079 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-07-31T14:38:08.080 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_version": "ceph version 19.0.0-5223-g8c796fca (8c796fca5ff5960f7542e7227bad4a784ee2c804) squid (dev)", 2024-07-31T14:38:08.080 INFO:teuthology.orchestra.run.smithi073.stdout: "image_id": "2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86", 2024-07-31T14:38:08.080 INFO:teuthology.orchestra.run.smithi073.stdout: "repo_digests": [ 2024-07-31T14:38:08.080 INFO:teuthology.orchestra.run.smithi073.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844" 2024-07-31T14:38:08.080 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-07-31T14:38:08.080 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-07-31T14:38:11.136 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-07-31T14:38:11.136 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_version": "ceph version 19.0.0-5223-g8c796fca (8c796fca5ff5960f7542e7227bad4a784ee2c804) squid (dev)", 2024-07-31T14:38:11.137 INFO:teuthology.orchestra.run.smithi077.stdout: "image_id": "2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86", 2024-07-31T14:38:11.137 INFO:teuthology.orchestra.run.smithi077.stdout: "repo_digests": [ 2024-07-31T14:38:11.137 INFO:teuthology.orchestra.run.smithi077.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844" 2024-07-31T14:38:11.137 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-07-31T14:38:11.137 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-07-31T14:38:11.162 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph 2024-07-31T14:38:11.197 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph 2024-07-31T14:38:11.229 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph 2024-07-31T14:38:11.256 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /etc/ceph 2024-07-31T14:38:11.285 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 777 /etc/ceph 2024-07-31T14:38:11.312 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 777 /etc/ceph 2024-07-31T14:38:11.335 INFO:tasks.cephadm:Writing seed config... 2024-07-31T14:38:11.337 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-07-31T14:38:11.337 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-07-31T14:38:11.337 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-07-31T14:38:11.337 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-07-31T14:38:11.337 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-07-31T14:38:11.337 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-07-31T14:38:11.337 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-07-31T14:38:11.338 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-07-31T14:38:11.338 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-07-31T14:38:11.338 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-07-31T14:38:11.338 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-07-31T14:38:11.338 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-07-31T14:38:11.338 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-07-31T14:38:11.338 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-07-31T14:38:11.338 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-07-31T14:38:11.338 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-07-31T14:38:11.338 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-07-31T14:38:11.338 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-07-31T14:38:11.338 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-07-31T14:38:11.338 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-07-31T14:38:11.339 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-07-31T14:38:11.339 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-07-31T14:38:11.339 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-07-31T14:38:11.339 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-07-31T14:38:11.339 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:38:11.340 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-07-31T14:38:11.357 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_export_ephemeral_random_max = 0.1 2024-07-31T14:38:11.358 DEBUG:teuthology.orchestra.run.smithi067:mon.a> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.a.service 2024-07-31T14:38:11.400 DEBUG:teuthology.orchestra.run.smithi067:mgr.x> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mgr.x.service 2024-07-31T14:38:11.443 INFO:tasks.cephadm:Bootstrapping... 2024-07-31T14:38:11.443 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 -v bootstrap --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.67 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-07-31T14:38:11.660 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-07-31T14:38:11.660 INFO:teuthology.orchestra.run.smithi067.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', '-v', 'bootstrap', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.67', '--skip-admin-label'] 2024-07-31T14:38:11.660 INFO:teuthology.orchestra.run.smithi067.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-07-31T14:38:11.660 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying podman|docker is present... 2024-07-31T14:38:11.687 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 5.1.2 2024-07-31T14:38:11.687 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying lvm2 is present... 2024-07-31T14:38:11.687 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying time synchronization is in place... 2024-07-31T14:38:11.695 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-31T14:38:11.695 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-31T14:38:11.702 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-31T14:38:11.702 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-07-31T14:38:11.709 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout enabled 2024-07-31T14:38:11.715 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout active 2024-07-31T14:38:11.715 INFO:teuthology.orchestra.run.smithi067.stdout:Unit chronyd.service is enabled and running 2024-07-31T14:38:11.715 INFO:teuthology.orchestra.run.smithi067.stdout:Repeating the final host check... 2024-07-31T14:38:11.740 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 5.1.2 2024-07-31T14:38:11.741 INFO:teuthology.orchestra.run.smithi067.stdout:podman (/bin/podman) version 5.1.2 is present 2024-07-31T14:38:11.741 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl is present 2024-07-31T14:38:11.741 INFO:teuthology.orchestra.run.smithi067.stdout:lvcreate is present 2024-07-31T14:38:11.747 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-31T14:38:11.748 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-31T14:38:11.754 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-31T14:38:11.754 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-07-31T14:38:11.761 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout enabled 2024-07-31T14:38:11.768 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout active 2024-07-31T14:38:11.768 INFO:teuthology.orchestra.run.smithi067.stdout:Unit chronyd.service is enabled and running 2024-07-31T14:38:11.768 INFO:teuthology.orchestra.run.smithi067.stdout:Host looks OK 2024-07-31T14:38:11.768 INFO:teuthology.orchestra.run.smithi067.stdout:Cluster fsid: 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:38:11.768 INFO:teuthology.orchestra.run.smithi067.stdout:Acquiring lock 140569545605952 on /run/cephadm/5ca2b71e-4f4a-11ef-bcc9-c7b262605968.lock 2024-07-31T14:38:11.768 INFO:teuthology.orchestra.run.smithi067.stdout:Lock 140569545605952 acquired on /run/cephadm/5ca2b71e-4f4a-11ef-bcc9-c7b262605968.lock 2024-07-31T14:38:11.769 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying IP 172.21.15.67 port 3300 ... 2024-07-31T14:38:11.769 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying IP 172.21.15.67 port 6789 ... 2024-07-31T14:38:11.769 INFO:teuthology.orchestra.run.smithi067.stdout:Base mon IP(s) is [172.21.15.67:3300, 172.21.15.67:6789], mon addrv is [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-07-31T14:38:11.772 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.67 metric 100 2024-07-31T14:38:11.772 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.67 metric 100 2024-07-31T14:38:11.775 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-07-31T14:38:11.775 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-07-31T14:38:11.776 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1785sec hoplimit 64 pref medium 2024-07-31T14:38:11.778 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-07-31T14:38:11.778 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-07-31T14:38:11.778 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-31T14:38:11.778 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-07-31T14:38:11.778 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-07-31T14:38:11.778 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-31T14:38:11.779 INFO:teuthology.orchestra.run.smithi067.stdout:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2024-07-31T14:38:11.779 INFO:teuthology.orchestra.run.smithi067.stdout:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2024-07-31T14:38:11.779 INFO:teuthology.orchestra.run.smithi067.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-07-31T14:38:11.780 INFO:teuthology.orchestra.run.smithi067.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-07-31T14:38:11.780 INFO:teuthology.orchestra.run.smithi067.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804... 2024-07-31T14:38:12.252 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 2024-07-31T14:38:12.252 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804... 2024-07-31T14:38:12.252 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Getting image source signatures 2024-07-31T14:38:12.252 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying blob sha256:8c351facb1e0038dad18b6b4a8cc67ebf949a6f1f762b22213af1ebf4574e670 2024-07-31T14:38:12.252 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying blob sha256:a0a261c93a1a3b1a5634697d0569ba314563628d3546bf9326abaa78605cead1 2024-07-31T14:38:12.252 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying config sha256:2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 2024-07-31T14:38:12.252 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Writing manifest to image destination 2024-07-31T14:38:13.231 INFO:teuthology.orchestra.run.smithi067.stdout:ceph: stdout ceph version 19.0.0-5223-g8c796fca (8c796fca5ff5960f7542e7227bad4a784ee2c804) squid (dev) 2024-07-31T14:38:13.231 INFO:teuthology.orchestra.run.smithi067.stdout:Ceph version: ceph version 19.0.0-5223-g8c796fca (8c796fca5ff5960f7542e7227bad4a784ee2c804) squid (dev) 2024-07-31T14:38:13.231 INFO:teuthology.orchestra.run.smithi067.stdout:Extracting ceph user uid/gid from container image... 2024-07-31T14:38:14.096 INFO:teuthology.orchestra.run.smithi067.stdout:stat: stdout 167 167 2024-07-31T14:38:14.096 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial keys... 2024-07-31T14:38:14.901 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQBWTKpmqKXrGRAACcKZlVn8jQ9uTo+mFs9yOQ== 2024-07-31T14:38:15.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQBXTKpm52omDRAA2IRQrC15vQgyL2vUcLdylA== 2024-07-31T14:38:16.661 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQBYTKpmqtOyAhAALRzE1tEdxa8X5T2YAKTsXw== 2024-07-31T14:38:16.661 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial monmap... 2024-07-31T14:38:17.533 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-31T14:38:17.533 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-07-31T14:38:17.533 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:38:17.533 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-31T14:38:17.533 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool for a [v2:172.21.15.67:3300,v1:172.21.15.67:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-31T14:38:17.533 INFO:teuthology.orchestra.run.smithi067.stdout:setting min_mon_release = quincy 2024-07-31T14:38:17.534 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: set fsid to 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:38:17.534 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-31T14:38:17.534 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:38:17.534 INFO:teuthology.orchestra.run.smithi067.stdout:Creating mon... 2024-07-31T14:38:18.472 INFO:teuthology.orchestra.run.smithi067.stdout:create mon.a on 2024-07-31T14:38:18.691 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-07-31T14:38:18.840 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-07-31T14:38:19.023 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968.target → /etc/systemd/system/ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968.target. 2024-07-31T14:38:19.024 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968.target → /etc/systemd/system/ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968.target. 2024-07-31T14:38:19.244 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.a 2024-07-31T14:38:19.245 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.a.service: Unit ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.a.service not loaded. 2024-07-31T14:38:19.412 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968.target.wants/ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.a.service → /etc/systemd/system/ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@.service. 2024-07-31T14:38:19.972 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-31T14:38:19.973 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-07-31T14:38:19.982 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-31T14:38:19.982 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-07-31T14:38:19.982 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-07-31T14:38:19.982 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to enable service . firewalld.service is not available 2024-07-31T14:38:19.982 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mon to start... 2024-07-31T14:38:19.982 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mon... 2024-07-31T14:38:21.017 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout cluster: 2024-07-31T14:38:21.018 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout id: 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:38:21.018 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-07-31T14:38:21.018 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-31T14:38:21.018 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout services: 2024-07-31T14:38:21.018 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.542088s) 2024-07-31T14:38:21.018 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-07-31T14:38:21.018 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-07-31T14:38:21.018 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-31T14:38:21.018 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout data: 2024-07-31T14:38:21.018 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-07-31T14:38:21.018 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-07-31T14:38:21.019 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-07-31T14:38:21.019 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout pgs: 2024-07-31T14:38:21.019 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-31T14:38:21.019 INFO:teuthology.orchestra.run.smithi067.stdout:mon is available 2024-07-31T14:38:21.019 INFO:teuthology.orchestra.run.smithi067.stdout:Assimilating anything we can from ceph.conf... 2024-07-31T14:38:22.091 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-31T14:38:22.091 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [global] 2024-07-31T14:38:22.091 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout fsid = 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:38:22.091 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-07-31T14:38:22.091 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-07-31T14:38:22.091 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-31T14:38:22.091 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-31T14:38:22.091 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-31T14:38:22.091 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-31T14:38:22.092 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-31T14:38:22.092 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-31T14:38:22.092 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-31T14:38:22.092 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-31T14:38:22.092 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [osd] 2024-07-31T14:38:22.092 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-31T14:38:22.092 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-31T14:38:22.092 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-31T14:38:22.092 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-31T14:38:22.092 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-31T14:38:22.092 INFO:teuthology.orchestra.run.smithi067.stdout:Generating new minimal ceph.conf... 2024-07-31T14:38:23.128 INFO:teuthology.orchestra.run.smithi067.stdout:Restarting the monitor... 2024-07-31T14:38:23.405 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:23 smithi067 systemd[1]: Stopping Ceph mon.a for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T14:38:23.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:23 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a[32374]: 2024-07-31T14:38:23.403+0000 7f6ced782640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T14:38:23.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:23 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a[32374]: 2024-07-31T14:38:23.403+0000 7f6ced782640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-07-31T14:38:23.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:23 smithi067 podman[32623]: 2024-07-31 14:38:23.454969094 +0000 UTC m=+0.161032378 container died e6fc3007ad998a9b762f2725bdd96261c7954d2dd27cd0558dadceb0c8f7119c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc) 2024-07-31T14:38:24.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:23 smithi067 podman[32623]: 2024-07-31 14:38:23.810981657 +0000 UTC m=+0.517044949 container cleanup e6fc3007ad998a9b762f2725bdd96261c7954d2dd27cd0558dadceb0c8f7119c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.build-date=20240716) 2024-07-31T14:38:24.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:23 smithi067 bash[32623]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a 2024-07-31T14:38:24.404 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 podman[32635]: 2024-07-31 14:38:24.06187411 +0000 UTC m=+0.598387763 container remove e6fc3007ad998a9b762f2725bdd96261c7954d2dd27cd0558dadceb0c8f7119c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T14:38:24.405 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 systemd[1]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.a.service: Deactivated successfully. 2024-07-31T14:38:24.405 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 systemd[1]: Stopped Ceph mon.a for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:38:24.405 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 systemd[1]: Starting Ceph mon.a for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T14:38:24.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 podman[32730]: 2024-07-31 14:38:24.404305257 +0000 UTC m=+0.108157905 container create 65e2949f2afd31939b9ba7ba8838e0ba2f9657a87212f8737651feaf22b08db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, ceph=True) 2024-07-31T14:38:24.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 podman[32730]: 2024-07-31 14:38:24.313747026 +0000 UTC m=+0.017599688 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 2024-07-31T14:38:24.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 podman[32730]: 2024-07-31 14:38:24.606999201 +0000 UTC m=+0.310851865 container init 65e2949f2afd31939b9ba7ba8838e0ba2f9657a87212f8737651feaf22b08db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS) 2024-07-31T14:38:24.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 podman[32730]: 2024-07-31 14:38:24.610218203 +0000 UTC m=+0.314070851 container start 65e2949f2afd31939b9ba7ba8838e0ba2f9657a87212f8737651feaf22b08db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a, org.label-schema.build-date=20240716, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-07-31T14:38:24.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: set uid:gid to 167:167 (ceph:ceph) 2024-07-31T14:38:24.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: ceph version 19.0.0-5223-g8c796fca (8c796fca5ff5960f7542e7227bad4a784ee2c804) squid (dev), process ceph-mon, pid 2 2024-07-31T14:38:24.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: pidfile_write: ignore empty --pid-file 2024-07-31T14:38:24.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: load: jerasure load: lrc 2024-07-31T14:38:24.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: RocksDB version: 7.9.2 2024-07-31T14:38:24.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Git sha 0 2024-07-31T14:38:24.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Compile date 2024-07-23 15:17:47 2024-07-31T14:38:24.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: DB SUMMARY 2024-07-31T14:38:24.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: DB Session ID: USQK18HKVXRFR2HJYV8D 2024-07-31T14:38:24.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: CURRENT file: CURRENT 2024-07-31T14:38:24.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: IDENTITY file: IDENTITY 2024-07-31T14:38:24.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-07-31T14:38:24.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-07-31T14:38:24.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94445 ; 2024-07-31T14:38:24.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.error_if_exists: 0 2024-07-31T14:38:24.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.create_if_missing: 0 2024-07-31T14:38:24.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.paranoid_checks: 1 2024-07-31T14:38:24.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.flush_verify_memtable_count: 1 2024-07-31T14:38:24.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-31T14:38:24.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-07-31T14:38:24.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.env: 0x55796f2d22e0 2024-07-31T14:38:24.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.fs: PosixFileSystem 2024-07-31T14:38:24.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.info_log: 0x557975b05080 2024-07-31T14:38:24.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_file_opening_threads: 16 2024-07-31T14:38:24.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.statistics: (nil) 2024-07-31T14:38:24.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.use_fsync: 0 2024-07-31T14:38:24.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_log_file_size: 0 2024-07-31T14:38:24.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-31T14:38:24.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-31T14:38:24.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.keep_log_file_num: 1000 2024-07-31T14:38:24.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.recycle_log_file_num: 0 2024-07-31T14:38:24.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.allow_fallocate: 1 2024-07-31T14:38:24.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.allow_mmap_reads: 0 2024-07-31T14:38:24.659 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.allow_mmap_writes: 0 2024-07-31T14:38:24.659 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.use_direct_reads: 0 2024-07-31T14:38:24.659 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-31T14:38:24.659 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.create_missing_column_families: 0 2024-07-31T14:38:24.659 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.db_log_dir: 2024-07-31T14:38:24.659 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.wal_dir: 2024-07-31T14:38:24.659 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-31T14:38:24.659 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-31T14:38:24.659 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-31T14:38:24.659 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-31T14:38:24.659 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-31T14:38:24.659 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-31T14:38:24.660 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.advise_random_on_open: 1 2024-07-31T14:38:24.660 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.db_write_buffer_size: 0 2024-07-31T14:38:24.660 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.write_buffer_manager: 0x557975b095e0 2024-07-31T14:38:24.660 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-31T14:38:24.660 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-31T14:38:24.660 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-31T14:38:24.660 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.rate_limiter: (nil) 2024-07-31T14:38:24.661 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-31T14:38:24.661 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.wal_recovery_mode: 2 2024-07-31T14:38:24.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.enable_thread_tracking: 0 2024-07-31T14:38:24.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.enable_pipelined_write: 0 2024-07-31T14:38:24.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.unordered_write: 0 2024-07-31T14:38:24.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-31T14:38:24.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-31T14:38:24.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-31T14:38:24.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-31T14:38:24.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.row_cache: None 2024-07-31T14:38:24.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.wal_filter: None 2024-07-31T14:38:24.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-31T14:38:24.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.allow_ingest_behind: 0 2024-07-31T14:38:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.two_write_queues: 0 2024-07-31T14:38:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.manual_wal_flush: 0 2024-07-31T14:38:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.wal_compression: 0 2024-07-31T14:38:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.atomic_flush: 0 2024-07-31T14:38:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-31T14:38:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-31T14:38:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-31T14:38:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.log_readahead_size: 0 2024-07-31T14:38:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-31T14:38:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.best_efforts_recovery: 0 2024-07-31T14:38:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-31T14:38:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-31T14:38:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.allow_data_in_errors: 0 2024-07-31T14:38:24.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.db_host_id: __hostname__ 2024-07-31T14:38:24.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-31T14:38:24.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_background_jobs: 2 2024-07-31T14:38:24.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_background_compactions: -1 2024-07-31T14:38:24.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_subcompactions: 1 2024-07-31T14:38:24.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-31T14:38:24.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-31T14:38:24.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-31T14:38:24.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_total_wal_size: 0 2024-07-31T14:38:24.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-31T14:38:24.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-31T14:38:24.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-31T14:38:24.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-31T14:38:24.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_open_files: -1 2024-07-31T14:38:24.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.bytes_per_sync: 0 2024-07-31T14:38:24.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-31T14:38:24.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-31T14:38:24.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compaction_readahead_size: 0 2024-07-31T14:38:24.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_background_flushes: -1 2024-07-31T14:38:24.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Compression algorithms supported: 2024-07-31T14:38:24.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: kZSTD supported: 0 2024-07-31T14:38:24.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: kXpressCompression supported: 0 2024-07-31T14:38:24.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: kBZip2Compression supported: 0 2024-07-31T14:38:24.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-31T14:38:24.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: kLZ4Compression supported: 1 2024-07-31T14:38:24.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: kZlibCompression supported: 1 2024-07-31T14:38:24.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: kLZ4HCCompression supported: 1 2024-07-31T14:38:24.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: kSnappyCompression supported: 1 2024-07-31T14:38:24.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-31T14:38:24.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-31T14:38:24.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-07-31T14:38:24.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-31T14:38:24.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-31T14:38:24.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.merge_operator: 2024-07-31T14:38:24.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compaction_filter: None 2024-07-31T14:38:24.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compaction_filter_factory: None 2024-07-31T14:38:24.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.sst_partitioner_factory: None 2024-07-31T14:38:24.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-31T14:38:24.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-31T14:38:24.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557975b04c80) 2024-07-31T14:38:24.667 INFO:journalctl@ceph.mon.a.smithi067.stdout: cache_index_and_filter_blocks: 1 2024-07-31T14:38:24.667 INFO:journalctl@ceph.mon.a.smithi067.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-31T14:38:24.667 INFO:journalctl@ceph.mon.a.smithi067.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-31T14:38:24.667 INFO:journalctl@ceph.mon.a.smithi067.stdout: pin_top_level_index_and_filter: 1 2024-07-31T14:38:24.667 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_type: 0 2024-07-31T14:38:24.667 INFO:journalctl@ceph.mon.a.smithi067.stdout: data_block_index_type: 0 2024-07-31T14:38:24.668 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_shortening: 1 2024-07-31T14:38:24.668 INFO:journalctl@ceph.mon.a.smithi067.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-31T14:38:24.668 INFO:journalctl@ceph.mon.a.smithi067.stdout: checksum: 4 2024-07-31T14:38:24.668 INFO:journalctl@ceph.mon.a.smithi067.stdout: no_block_cache: 0 2024-07-31T14:38:24.668 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache: 0x557971ac9610 2024-07-31T14:38:24.668 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_name: BinnedLRUCache 2024-07-31T14:38:24.668 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_options: 2024-07-31T14:38:24.668 INFO:journalctl@ceph.mon.a.smithi067.stdout: capacity : 536870912 2024-07-31T14:38:24.668 INFO:journalctl@ceph.mon.a.smithi067.stdout: num_shard_bits : 4 2024-07-31T14:38:24.668 INFO:journalctl@ceph.mon.a.smithi067.stdout: strict_capacity_limit : 0 2024-07-31T14:38:24.668 INFO:journalctl@ceph.mon.a.smithi067.stdout: high_pri_pool_ratio: 0.000 2024-07-31T14:38:24.668 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_compressed: (nil) 2024-07-31T14:38:24.668 INFO:journalctl@ceph.mon.a.smithi067.stdout: persistent_cache: (nil) 2024-07-31T14:38:24.669 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_size: 4096 2024-07-31T14:38:24.669 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_size_deviation: 10 2024-07-31T14:38:24.669 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_restart_interval: 16 2024-07-31T14:38:24.669 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_block_restart_interval: 1 2024-07-31T14:38:24.669 INFO:journalctl@ceph.mon.a.smithi067.stdout: metadata_block_size: 4096 2024-07-31T14:38:24.669 INFO:journalctl@ceph.mon.a.smithi067.stdout: partition_filters: 0 2024-07-31T14:38:24.669 INFO:journalctl@ceph.mon.a.smithi067.stdout: use_delta_encoding: 1 2024-07-31T14:38:24.669 INFO:journalctl@ceph.mon.a.smithi067.stdout: filter_policy: bloomfilter 2024-07-31T14:38:24.669 INFO:journalctl@ceph.mon.a.smithi067.stdout: whole_key_filtering: 1 2024-07-31T14:38:24.669 INFO:journalctl@ceph.mon.a.smithi067.stdout: verify_compression: 0 2024-07-31T14:38:24.669 INFO:journalctl@ceph.mon.a.smithi067.stdout: read_amp_bytes_per_bit: 0 2024-07-31T14:38:24.669 INFO:journalctl@ceph.mon.a.smithi067.stdout: format_version: 5 2024-07-31T14:38:24.670 INFO:journalctl@ceph.mon.a.smithi067.stdout: enable_index_compression: 1 2024-07-31T14:38:24.670 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_align: 0 2024-07-31T14:38:24.670 INFO:journalctl@ceph.mon.a.smithi067.stdout: max_auto_readahead_size: 262144 2024-07-31T14:38:24.670 INFO:journalctl@ceph.mon.a.smithi067.stdout: prepopulate_block_cache: 0 2024-07-31T14:38:24.670 INFO:journalctl@ceph.mon.a.smithi067.stdout: initial_auto_readahead_size: 8192 2024-07-31T14:38:24.670 INFO:journalctl@ceph.mon.a.smithi067.stdout: num_file_reads_for_auto_readahead: 2 2024-07-31T14:38:24.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.write_buffer_size: 33554432 2024-07-31T14:38:24.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_write_buffer_number: 2 2024-07-31T14:38:24.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compression: NoCompression 2024-07-31T14:38:24.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.bottommost_compression: Disabled 2024-07-31T14:38:24.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.prefix_extractor: nullptr 2024-07-31T14:38:24.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-31T14:38:24.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.num_levels: 7 2024-07-31T14:38:24.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-31T14:38:24.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-31T14:38:24.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-31T14:38:24.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-31T14:38:24.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-31T14:38:24.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-31T14:38:24.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-31T14:38:24.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-31T14:38:24.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-31T14:38:24.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-31T14:38:24.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-31T14:38:24.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-31T14:38:24.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-31T14:38:24.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compression_opts.level: 32767 2024-07-31T14:38:24.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compression_opts.strategy: 0 2024-07-31T14:38:24.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-31T14:38:24.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-31T14:38:24.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-31T14:38:24.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-31T14:38:24.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compression_opts.enabled: false 2024-07-31T14:38:24.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-31T14:38:24.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-31T14:38:24.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-31T14:38:24.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-31T14:38:24.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.target_file_size_base: 67108864 2024-07-31T14:38:24.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-31T14:38:24.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-31T14:38:24.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-31T14:38:24.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-31T14:38:24.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-31T14:38:24.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-31T14:38:24.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-31T14:38:24.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-31T14:38:24.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-31T14:38:24.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-31T14:38:24.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-31T14:38:24.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-31T14:38:24.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-31T14:38:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-31T14:38:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.arena_block_size: 1048576 2024-07-31T14:38:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-31T14:38:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-31T14:38:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.disable_auto_compactions: 0 2024-07-31T14:38:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-31T14:38:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-31T14:38:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-31T14:38:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-31T14:38:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-31T14:38:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-31T14:38:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-31T14:38:24.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-31T14:38:24.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-31T14:38:24.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-31T14:38:24.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-07-31T14:38:24.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.inplace_update_support: 0 2024-07-31T14:38:24.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-31T14:38:24.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-31T14:38:24.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-31T14:38:24.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-31T14:38:24.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.bloom_locality: 0 2024-07-31T14:38:24.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.max_successive_merges: 0 2024-07-31T14:38:24.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-31T14:38:24.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.paranoid_file_checks: 0 2024-07-31T14:38:24.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.force_consistency_checks: 1 2024-07-31T14:38:24.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.report_bg_io_stats: 0 2024-07-31T14:38:24.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.ttl: 2592000 2024-07-31T14:38:24.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-31T14:38:24.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-31T14:38:24.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-31T14:38:24.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.enable_blob_files: false 2024-07-31T14:38:24.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.min_blob_size: 0 2024-07-31T14:38:24.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.blob_file_size: 268435456 2024-07-31T14:38:24.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-31T14:38:24.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-31T14:38:24.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-31T14:38:24.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-31T14:38:24.677 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-31T14:38:24.677 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.blob_file_starting_level: 0 2024-07-31T14:38:24.677 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-31T14:38:24.677 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-07-31T14:38:24.677 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-07-31T14:38:24.677 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b5175533-d26d-4905-aa47-89530650cdb8 2024-07-31T14:38:24.677 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722436704647504, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-07-31T14:38:24.677 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-07-31T14:38:24.677 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722436704649196, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90675, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 280, "table_properties": {"data_size": 88716, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11496, "raw_average_key_size": 48, "raw_value_size": 82231, "raw_average_value_size": 344, "num_data_blocks": 12, "num_entries": 239, "num_filter_entries": 239, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1722436704, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b5175533-d26d-4905-aa47-89530650cdb8", "db_session_id": "USQK18HKVXRFR2HJYV8D", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-07-31T14:38:24.677 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722436704649335, "job": 1, "event": "recovery_finished"} 2024-07-31T14:38:24.677 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-07-31T14:38:24.677 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-31T14:38:24.677 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557975b26a80 2024-07-31T14:38:24.678 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: DB pointer 0x5579768c6000 2024-07-31T14:38:24.678 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-31T14:38:24.678 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-31T14:38:24.678 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** DB Stats ** 2024-07-31T14:38:24.678 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-31T14:38:24.678 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-31T14:38:24.678 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-31T14:38:24.678 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-31T14:38:24.678 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-31T14:38:24.678 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-31T14:38:24.678 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-31T14:38:24.678 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-07-31T14:38:24.679 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2024-07-31T14:38:24.679 INFO:journalctl@ceph.mon.a.smithi067.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-31T14:38:24.679 INFO:journalctl@ceph.mon.a.smithi067.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-31T14:38:24.679 INFO:journalctl@ceph.mon.a.smithi067.stdout: L0 2/0 90.41 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-31T14:38:24.679 INFO:journalctl@ceph.mon.a.smithi067.stdout: Sum 2/0 90.41 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-31T14:38:24.679 INFO:journalctl@ceph.mon.a.smithi067.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-31T14:38:24.679 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-07-31T14:38:24.679 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2024-07-31T14:38:24.679 INFO:journalctl@ceph.mon.a.smithi067.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-31T14:38:24.679 INFO:journalctl@ceph.mon.a.smithi067.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-31T14:38:24.679 INFO:journalctl@ceph.mon.a.smithi067.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 97.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-31T14:38:24.679 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-07-31T14:38:24.679 INFO:journalctl@ceph.mon.a.smithi067.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-31T14:38:24.680 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-07-31T14:38:24.680 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-31T14:38:24.680 INFO:journalctl@ceph.mon.a.smithi067.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-31T14:38:24.680 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-31T14:38:24.680 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-31T14:38:24.680 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-31T14:38:24.680 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-31T14:38:24.680 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative compaction: 0.00 GB write, 17.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-31T14:38:24.681 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval compaction: 0.00 GB write, 17.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-31T14:38:24.681 INFO:journalctl@ceph.mon.a.smithi067.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-31T14:38:24.681 INFO:journalctl@ceph.mon.a.smithi067.stdout: Block cache BinnedLRUCache@0x557971ac9610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-07-31T14:38:24.681 INFO:journalctl@ceph.mon.a.smithi067.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-07-31T14:38:24.682 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-07-31T14:38:24.682 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-31T14:38:24.682 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: starting mon.a rank 0 at public addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] at bind addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:38:24.682 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: mon.a@-1(???) e1 preinit fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:38:24.682 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: mon.a@-1(???).mds e1 new map 2024-07-31T14:38:24.682 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: mon.a@-1(???).mds e1 print_map 2024-07-31T14:38:24.682 INFO:journalctl@ceph.mon.a.smithi067.stdout: e1 2024-07-31T14:38:24.682 INFO:journalctl@ceph.mon.a.smithi067.stdout: btime 2024-07-31T14:38:19:925493+0000 2024-07-31T14:38:24.682 INFO:journalctl@ceph.mon.a.smithi067.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-31T14:38:24.682 INFO:journalctl@ceph.mon.a.smithi067.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-31T14:38:24.682 INFO:journalctl@ceph.mon.a.smithi067.stdout: legacy client fscid: -1 2024-07-31T14:38:24.682 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-07-31T14:38:24.683 INFO:journalctl@ceph.mon.a.smithi067.stdout: No filesystems configured 2024-07-31T14:38:24.683 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-31T14:38:24.683 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-31T14:38:24.683 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-31T14:38:24.683 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-31T14:38:24.683 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-07-31T14:38:24.704 INFO:teuthology.orchestra.run.smithi067.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-07-31T14:38:24.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 bash[32730]: 65e2949f2afd31939b9ba7ba8838e0ba2f9657a87212f8737651feaf22b08db2 2024-07-31T14:38:24.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 systemd[1]: Started Ceph mon.a for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:38:24.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: mon.a is new leader, mons a in quorum (ranks 0) 2024-07-31T14:38:24.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: monmap epoch 1 2024-07-31T14:38:24.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:38:24.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: last_changed 2024-07-31T14:38:17.033740+0000 2024-07-31T14:38:24.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: created 2024-07-31T14:38:17.033740+0000 2024-07-31T14:38:24.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: min_mon_release 19 (squid) 2024-07-31T14:38:24.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: election_strategy: 1 2024-07-31T14:38:24.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-31T14:38:24.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: fsmap 2024-07-31T14:38:24.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: osdmap e1: 0 total, 0 up, 0 in 2024-07-31T14:38:24.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:24 smithi067 ceph-mon[32750]: mgrmap e1: no daemons active 2024-07-31T14:38:25.734 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote config to /etc/ceph/ceph.conf 2024-07-31T14:38:25.754 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-07-31T14:38:25.754 INFO:teuthology.orchestra.run.smithi067.stdout:Creating mgr... 2024-07-31T14:38:25.755 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying port 0.0.0.0:9283 ... 2024-07-31T14:38:25.755 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying port 0.0.0.0:8765 ... 2024-07-31T14:38:26.025 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mgr.x 2024-07-31T14:38:26.025 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mgr.x.service: Unit ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mgr.x.service not loaded. 2024-07-31T14:38:26.184 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968.target.wants/ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@.service. 2024-07-31T14:38:26.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:26 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/938228009' entity='client.admin' 2024-07-31T14:38:26.726 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:26 smithi067 podman[32962]: 2024-07-31 14:38:26.644834563 +0000 UTC m=+0.331747717 container start f86146d771dd3601cc78b3c5f3d48a880eafb75d62504e62af847b88a38882fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T14:38:26.726 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:26 smithi067 bash[32962]: f86146d771dd3601cc78b3c5f3d48a880eafb75d62504e62af847b88a38882fc 2024-07-31T14:38:26.743 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-31T14:38:26.743 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-07-31T14:38:26.752 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-31T14:38:26.752 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-07-31T14:38:26.752 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-07-31T14:38:26.752 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to enable service . firewalld.service is not available 2024-07-31T14:38:26.763 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-31T14:38:26.763 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-07-31T14:38:26.772 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-31T14:38:26.772 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-07-31T14:38:26.772 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-07-31T14:38:26.772 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-07-31T14:38:26.772 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr to start... 2024-07-31T14:38:26.772 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr... 2024-07-31T14:38:27.000 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:26 smithi067 systemd[1]: Started Ceph mgr.x for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:38:27.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:27 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2170184402' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-31T14:38:27.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-31T14:38:27.827 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-07-31T14:38:27.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:38:27.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-07-31T14:38:27.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-31T14:38:27.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-31T14:38:27.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-31T14:38:27.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:27.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-31T14:38:27.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-31T14:38:27.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-07-31T14:38:27.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-31T14:38:27.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-31T14:38:27.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-07-31T14:38:27.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-31T14:38:27.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-07-31T14:38:27.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-31T14:38:27.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-31T14:38:27.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-31T14:38:27.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-31T14:38:27.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:27.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-31T14:38:27.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-31T14:38:27.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-31T14:38:27.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-31T14:38:27.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-31T14:38:27.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-31T14:38:27.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-31T14:38:27.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-31T14:38:27.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:27.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-31T14:38:27.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-31T14:38:27.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-31T14:38:27.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-31T14:38:27.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-31T14:38:27.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-31T14:38:27.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-31T14:38:27.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-31T14:38:27.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-31T14:38:27.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:27.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-31T14:38:27.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-31T14:38:27.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "btime": "2024-07-31T14:38:19:925493+0000", 2024-07-31T14:38:27.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-31T14:38:27.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-31T14:38:27.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:27.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-31T14:38:27.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-31T14:38:27.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-31T14:38:27.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-31T14:38:27.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-31T14:38:27.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-31T14:38:27.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-07-31T14:38:27.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-31T14:38:27.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-31T14:38:27.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:27.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-31T14:38:27.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-31T14:38:27.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-07-31T14:38:19.926339+0000", 2024-07-31T14:38:27.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-31T14:38:27.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:27.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-31T14:38:27.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-07-31T14:38:27.833 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (1/15)... 2024-07-31T14:38:30.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:30 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2303087347' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-31T14:38:30.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-31T14:38:30.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-07-31T14:38:30.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:38:30.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-07-31T14:38:30.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-31T14:38:30.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-31T14:38:30.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-31T14:38:30.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:30.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-31T14:38:30.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-31T14:38:30.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-07-31T14:38:30.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-31T14:38:30.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-31T14:38:30.949 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-07-31T14:38:30.949 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-31T14:38:30.949 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-07-31T14:38:30.949 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-31T14:38:30.949 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-31T14:38:30.949 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-31T14:38:30.949 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-31T14:38:30.949 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:30.949 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-31T14:38:30.949 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-31T14:38:30.949 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-31T14:38:30.949 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-31T14:38:30.950 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-31T14:38:30.950 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-31T14:38:30.950 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-31T14:38:30.950 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-31T14:38:30.953 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:30.953 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-31T14:38:30.953 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-31T14:38:30.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-31T14:38:30.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-31T14:38:30.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-31T14:38:30.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-31T14:38:30.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-31T14:38:30.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-31T14:38:30.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-31T14:38:30.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:30.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-31T14:38:30.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-31T14:38:30.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "btime": "2024-07-31T14:38:19:925493+0000", 2024-07-31T14:38:30.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-31T14:38:30.955 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-31T14:38:30.955 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:30.955 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-31T14:38:30.955 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-31T14:38:30.955 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-31T14:38:30.955 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-31T14:38:30.955 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-31T14:38:30.955 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-31T14:38:30.955 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-07-31T14:38:30.955 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-31T14:38:30.955 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-31T14:38:30.955 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:30.956 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-31T14:38:30.956 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-31T14:38:30.956 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-07-31T14:38:19.926339+0000", 2024-07-31T14:38:30.956 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-31T14:38:30.956 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:30.956 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-31T14:38:30.956 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-07-31T14:38:30.956 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (2/15)... 2024-07-31T14:38:32.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:32 smithi067 ceph-mon[32750]: Activating manager daemon x 2024-07-31T14:38:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:32 smithi067 ceph-mon[32750]: mgrmap e2: x(active, starting, since 0.00393739s) 2024-07-31T14:38:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:32 smithi067 ceph-mon[32750]: from='mgr.14100 172.21.15.67:0/2432382190' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-31T14:38:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:32 smithi067 ceph-mon[32750]: from='mgr.14100 172.21.15.67:0/2432382190' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-31T14:38:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:32 smithi067 ceph-mon[32750]: from='mgr.14100 172.21.15.67:0/2432382190' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-31T14:38:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:32 smithi067 ceph-mon[32750]: from='mgr.14100 172.21.15.67:0/2432382190' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:38:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:32 smithi067 ceph-mon[32750]: from='mgr.14100 172.21.15.67:0/2432382190' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-31T14:38:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:32 smithi067 ceph-mon[32750]: Manager daemon x is now available 2024-07-31T14:38:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:32 smithi067 ceph-mon[32750]: from='mgr.14100 172.21.15.67:0/2432382190' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-31T14:38:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:32 smithi067 ceph-mon[32750]: from='mgr.14100 172.21.15.67:0/2432382190' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-31T14:38:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:32 smithi067 ceph-mon[32750]: from='mgr.14100 172.21.15.67:0/2432382190' entity='mgr.x' 2024-07-31T14:38:32.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:32 smithi067 ceph-mon[32750]: from='mgr.14100 172.21.15.67:0/2432382190' entity='mgr.x' 2024-07-31T14:38:32.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:32 smithi067 ceph-mon[32750]: from='mgr.14100 172.21.15.67:0/2432382190' entity='mgr.x' 2024-07-31T14:38:34.264 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:34 smithi067 ceph-mon[32750]: mgrmap e3: x(active, since 1.00829s) 2024-07-31T14:38:34.264 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:34 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3443162733' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-31T14:38:34.269 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-31T14:38:34.269 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-07-31T14:38:34.269 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:38:34.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-07-31T14:38:34.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-31T14:38:34.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-31T14:38:34.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-31T14:38:34.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:34.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-31T14:38:34.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-31T14:38:34.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-07-31T14:38:34.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-31T14:38:34.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-31T14:38:34.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-07-31T14:38:34.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-31T14:38:34.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-07-31T14:38:34.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-31T14:38:34.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-31T14:38:34.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-31T14:38:34.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-31T14:38:34.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:34.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-31T14:38:34.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-31T14:38:34.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-31T14:38:34.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-31T14:38:34.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-31T14:38:34.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-31T14:38:34.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-31T14:38:34.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-31T14:38:34.278 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:34.278 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-31T14:38:34.278 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-31T14:38:34.278 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-31T14:38:34.278 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-31T14:38:34.278 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-31T14:38:34.278 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-31T14:38:34.278 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-31T14:38:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-31T14:38:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-31T14:38:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-31T14:38:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-31T14:38:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "btime": "2024-07-31T14:38:19:925493+0000", 2024-07-31T14:38:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-31T14:38:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-31T14:38:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-31T14:38:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-31T14:38:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-31T14:38:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-31T14:38:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-31T14:38:34.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-31T14:38:34.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-07-31T14:38:34.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-31T14:38:34.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-31T14:38:34.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:34.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-31T14:38:34.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-31T14:38:34.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-07-31T14:38:19.926339+0000", 2024-07-31T14:38:34.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-31T14:38:34.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-31T14:38:34.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-31T14:38:34.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-07-31T14:38:34.280 INFO:teuthology.orchestra.run.smithi067.stdout:mgr is available 2024-07-31T14:38:35.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:35 smithi067 ceph-mon[32750]: mgrmap e4: x(active, since 2s) 2024-07-31T14:38:35.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:35 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2822237682' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-07-31T14:38:35.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-31T14:38:35.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [global] 2024-07-31T14:38:35.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout fsid = 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:38:35.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-07-31T14:38:35.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-07-31T14:38:35.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-31T14:38:35.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-31T14:38:35.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-31T14:38:35.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-31T14:38:35.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-31T14:38:35.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-31T14:38:35.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-31T14:38:35.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-31T14:38:35.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [osd] 2024-07-31T14:38:35.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-31T14:38:35.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-31T14:38:35.360 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-31T14:38:35.360 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-31T14:38:35.360 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-31T14:38:35.360 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling cephadm module... 2024-07-31T14:38:36.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:36 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1599913866' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-07-31T14:38:36.393 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:36 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: ignoring --setuser ceph since I am not root 2024-07-31T14:38:36.394 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:36 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: ignoring --setgroup ceph since I am not root 2024-07-31T14:38:37.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:37 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1599913866' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-07-31T14:38:37.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:37 smithi067 ceph-mon[32750]: mgrmap e5: x(active, since 4s) 2024-07-31T14:38:37.804 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-07-31T14:38:37.804 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-07-31T14:38:37.804 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-31T14:38:37.804 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-07-31T14:38:37.804 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-31T14:38:37.804 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-07-31T14:38:37.804 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for the mgr to restart... 2024-07-31T14:38:37.804 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr epoch 5... 2024-07-31T14:38:38.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:38 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/650338436' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-31T14:38:41.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: Active manager daemon x restarted 2024-07-31T14:38:41.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: Activating manager daemon x 2024-07-31T14:38:41.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: osdmap e2: 0 total, 0 up, 0 in 2024-07-31T14:38:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: mgrmap e6: x(active, starting, since 0.00427569s) 2024-07-31T14:38:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:38:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-31T14:38:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-31T14:38:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-31T14:38:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-31T14:38:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: Manager daemon x is now available 2024-07-31T14:38:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' 2024-07-31T14:38:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' 2024-07-31T14:38:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:38:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:38:41.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-31T14:38:41.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:41 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-31T14:38:43.050 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-07-31T14:38:43.050 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-07-31T14:38:43.051 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-31T14:38:43.051 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-07-31T14:38:43.051 INFO:teuthology.orchestra.run.smithi067.stdout:mgr epoch 5 is available 2024-07-31T14:38:43.051 INFO:teuthology.orchestra.run.smithi067.stdout:Setting orchestrator backend to cephadm... 2024-07-31T14:38:43.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-mon[32750]: Found migration_current of "None". Setting to last migration. 2024-07-31T14:38:43.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' 2024-07-31T14:38:43.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' 2024-07-31T14:38:43.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-mon[32750]: mgrmap e7: x(active, since 1.00587s) 2024-07-31T14:38:44.053 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:38:43.864+0000 7f0bc3cf9640 -1 log_channel(cephadm) log [ERR] : [31/Jul/2024:14:38:43] ENGINE Error in HTTPServer.serve 2024-07-31T14:38:44.053 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: Traceback (most recent call last): 2024-07-31T14:38:44.054 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-31T14:38:44.054 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: self._connections.run(self.expiration_interval) 2024-07-31T14:38:44.054 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-31T14:38:44.054 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: self._run(expiration_interval) 2024-07-31T14:38:44.054 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-31T14:38:44.054 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: new_conn = self._from_server_socket(self.server.socket) 2024-07-31T14:38:44.054 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-31T14:38:44.054 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-31T14:38:44.054 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-31T14:38:44.054 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: s = self.context.wrap_socket( 2024-07-31T14:38:44.054 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-31T14:38:44.054 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: return self.sslsocket_class._create( 2024-07-31T14:38:44.054 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-31T14:38:44.055 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: self.do_handshake() 2024-07-31T14:38:44.055 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-31T14:38:44.055 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: self._sslobj.do_handshake() 2024-07-31T14:38:44.055 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-31T14:38:44.055 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:38:44.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:44 smithi067 ceph-mon[32750]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-07-31T14:38:44.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:44 smithi067 ceph-mon[32750]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-07-31T14:38:44.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:44 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' 2024-07-31T14:38:44.466 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:44 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:38:44.466 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:44 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:38:45.427 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:45 smithi067 ceph-mon[32750]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:38:45.428 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:45 smithi067 ceph-mon[32750]: [31/Jul/2024:14:38:43] ENGINE Bus STARTING 2024-07-31T14:38:45.428 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:45 smithi067 ceph-mon[32750]: [31/Jul/2024:14:38:43] ENGINE Serving on https://172.21.15.67:7150 2024-07-31T14:38:45.428 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:45 smithi067 ceph-mon[32750]: [31/Jul/2024:14:38:43] ENGINE Error in HTTPServer.serve 2024-07-31T14:38:45.428 INFO:journalctl@ceph.mon.a.smithi067.stdout: Traceback (most recent call last): 2024-07-31T14:38:45.428 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-31T14:38:45.428 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._connections.run(self.expiration_interval) 2024-07-31T14:38:45.428 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-31T14:38:45.429 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._run(expiration_interval) 2024-07-31T14:38:45.429 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-31T14:38:45.429 INFO:journalctl@ceph.mon.a.smithi067.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-31T14:38:45.429 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-31T14:38:45.429 INFO:journalctl@ceph.mon.a.smithi067.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-31T14:38:45.429 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-31T14:38:45.429 INFO:journalctl@ceph.mon.a.smithi067.stdout: s = self.context.wrap_socket( 2024-07-31T14:38:45.429 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-31T14:38:45.429 INFO:journalctl@ceph.mon.a.smithi067.stdout: return self.sslsocket_class._create( 2024-07-31T14:38:45.429 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-31T14:38:45.429 INFO:journalctl@ceph.mon.a.smithi067.stdout: self.do_handshake() 2024-07-31T14:38:45.429 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-31T14:38:45.430 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._sslobj.do_handshake() 2024-07-31T14:38:45.430 INFO:journalctl@ceph.mon.a.smithi067.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-31T14:38:45.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:45 smithi067 ceph-mon[32750]: [31/Jul/2024:14:38:43] ENGINE Serving on http://172.21.15.67:8765 2024-07-31T14:38:45.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:45 smithi067 ceph-mon[32750]: [31/Jul/2024:14:38:43] ENGINE Bus STARTED 2024-07-31T14:38:45.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:45 smithi067 ceph-mon[32750]: mgrmap e8: x(active, since 2s) 2024-07-31T14:38:45.431 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout value unchanged 2024-07-31T14:38:45.431 INFO:teuthology.orchestra.run.smithi067.stdout:Generating ssh key... 2024-07-31T14:38:46.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:46 smithi067 ceph-mon[32750]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:38:47.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-mon[32750]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:38:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-mon[32750]: Generating ssh key... 2024-07-31T14:38:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' 2024-07-31T14:38:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' 2024-07-31T14:38:47.304 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: Generating public/private rsa key pair. 2024-07-31T14:38:47.304 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: Your identification has been saved in /tmp/tmpjs2vjoic/key 2024-07-31T14:38:47.304 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: Your public key has been saved in /tmp/tmpjs2vjoic/key.pub 2024-07-31T14:38:47.304 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: The key fingerprint is: 2024-07-31T14:38:47.304 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: SHA256:XDnvlG/GMQo6drNWVJPZR9zHTjeIgmXi+Jnpq+a6rBM ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:38:47.304 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: The key's randomart image is: 2024-07-31T14:38:47.304 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: +---[RSA 3072]----+ 2024-07-31T14:38:47.304 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: | .oo . .Bo| 2024-07-31T14:38:47.305 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: | o.o. o .=.X| 2024-07-31T14:38:47.305 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: | . . = . +=| 2024-07-31T14:38:47.305 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: | ..+. o.. .| 2024-07-31T14:38:47.305 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: | =S ..+ o | 2024-07-31T14:38:47.305 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: | E . . +.+ o | 2024-07-31T14:38:47.305 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: | . .+ o.o = | 2024-07-31T14:38:47.305 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: | .. . ..o.o o | 2024-07-31T14:38:47.305 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: | .o+=o.. .. | 2024-07-31T14:38:47.305 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:38:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: +----[SHA256]-----+ 2024-07-31T14:38:48.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDU+VKZc2gne+A0L9H+u+QDKq/Bwm9nICatRK9r7/KkIUjuANMKY7F4IPVnykL+GZvU/iD1Sst6Rv57djHb+5uZxsK9NoGIuUMaF1X/Fp6IozoEp6hQxI8C/Tf4VFicTPlFfHnjzqMV8PsHFG1zi7+kgnhsnwYFkJDnN2QYMuEXXup4CWxV6RWnK2jkUbgEH/ZSFqhg3hBZ1SCgdZh8WBOCy+1YJt0u2Wxph8uf+DTZugRqEiHq0sgVotraf9iRJbpyzScodUsDSTD0o9Q/lpe5uCGXIi6JngarF5m9gm/YkXoxh+F77qZ4r86msWTLzIXVksf9cRvHI+wwWRZQjkKA05h5NB+dIfl2BefDc3NUzSq9JkUFSRRlH533KZjnEqqdz6taHfWsKd6wkBNFVRdoUBxN/cx7Qv0pT0jq32ZYUyrvfFa8gSv1AOsd+y9kkmQl1+11IeGdizFkTDCCKRJ+N3W1ezthz/10GX3QLT77y/EfNoKECJp1F1jehehqJgk= ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:38:48.752 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-07-31T14:38:48.753 INFO:teuthology.orchestra.run.smithi067.stdout:Adding key to root@localhost authorized_keys... 2024-07-31T14:38:48.753 INFO:teuthology.orchestra.run.smithi067.stdout:Adding host smithi067... 2024-07-31T14:38:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:49 smithi067 ceph-mon[32750]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:38:50.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:50 smithi067 ceph-mon[32750]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi067", "addr": "172.21.15.67", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:38:51.301 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:51 smithi067 ceph-mon[32750]: Deploying cephadm binary to smithi067 2024-07-31T14:38:53.943 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Added host 'smithi067' with addr '172.21.15.67' 2024-07-31T14:38:53.943 INFO:teuthology.orchestra.run.smithi067.stdout:Deploying unmanaged mon service... 2024-07-31T14:38:54.589 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:54 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' 2024-07-31T14:38:54.589 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:54 smithi067 ceph-mon[32750]: Added host smithi067 2024-07-31T14:38:54.589 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:54 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:38:55.889 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:55 smithi067 ceph-mon[32750]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:38:55.890 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:55 smithi067 ceph-mon[32750]: Saving service mon spec with placement count:5 2024-07-31T14:38:55.890 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:55 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' 2024-07-31T14:38:56.142 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-07-31T14:38:56.142 INFO:teuthology.orchestra.run.smithi067.stdout:Deploying unmanaged mgr service... 2024-07-31T14:38:57.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:56 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' 2024-07-31T14:38:57.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:56 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' 2024-07-31T14:38:57.500 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-07-31T14:38:58.019 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:57 smithi067 ceph-mon[32750]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:38:58.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:57 smithi067 ceph-mon[32750]: Saving service mgr spec with placement count:2 2024-07-31T14:38:58.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:57 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' 2024-07-31T14:38:59.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:58 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' 2024-07-31T14:38:59.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:58 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1754822135' entity='client.admin' 2024-07-31T14:38:59.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:38:58 smithi067 ceph-mon[32750]: from='mgr.14118 172.21.15.67:0/4029182389' entity='mgr.x' 2024-07-31T14:39:00.796 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling the dashboard module... 2024-07-31T14:39:01.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:00 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1213235172' entity='client.admin' 2024-07-31T14:39:02.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:02 smithi067 ceph-mon[32750]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:02.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:02 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1769271867' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-07-31T14:39:02.803 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:02 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: ignoring --setuser ceph since I am not root 2024-07-31T14:39:02.804 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:02 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: ignoring --setgroup ceph since I am not root 2024-07-31T14:39:03.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:03 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1769271867' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-07-31T14:39:03.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:03 smithi067 ceph-mon[32750]: mgrmap e9: x(active, since 21s) 2024-07-31T14:39:04.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-07-31T14:39:04.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-07-31T14:39:04.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-31T14:39:04.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-07-31T14:39:04.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-31T14:39:04.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-07-31T14:39:04.404 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for the mgr to restart... 2024-07-31T14:39:04.404 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr epoch 9... 2024-07-31T14:39:04.651 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:04 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3888034940' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-31T14:39:08.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:07 smithi067 ceph-mon[32750]: Active manager daemon x restarted 2024-07-31T14:39:08.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:07 smithi067 ceph-mon[32750]: Activating manager daemon x 2024-07-31T14:39:08.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:07 smithi067 ceph-mon[32750]: osdmap e3: 0 total, 0 up, 0 in 2024-07-31T14:39:08.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:07 smithi067 ceph-mon[32750]: mgrmap e10: x(active, starting, since 0.00425062s) 2024-07-31T14:39:08.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:07 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:39:08.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:07 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-31T14:39:08.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:07 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-31T14:39:08.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:07 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-31T14:39:08.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:07 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-31T14:39:08.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:07 smithi067 ceph-mon[32750]: Manager daemon x is now available 2024-07-31T14:39:08.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:07 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:39:08.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:07 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-31T14:39:08.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:07 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-31T14:39:09.804 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:39:09.530+0000 7f6a38750640 -1 log_channel(cephadm) log [ERR] : [31/Jul/2024:14:39:09] ENGINE Error in HTTPServer.serve 2024-07-31T14:39:09.804 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: Traceback (most recent call last): 2024-07-31T14:39:09.804 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-31T14:39:09.805 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: self._connections.run(self.expiration_interval) 2024-07-31T14:39:09.805 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-31T14:39:09.805 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: self._run(expiration_interval) 2024-07-31T14:39:09.805 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-31T14:39:09.805 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: new_conn = self._from_server_socket(self.server.socket) 2024-07-31T14:39:09.805 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-31T14:39:09.805 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-31T14:39:09.805 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-31T14:39:09.805 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: s = self.context.wrap_socket( 2024-07-31T14:39:09.805 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-31T14:39:09.805 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: return self.sslsocket_class._create( 2024-07-31T14:39:09.805 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-31T14:39:09.806 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: self.do_handshake() 2024-07-31T14:39:09.806 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-31T14:39:09.806 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: self._sslobj.do_handshake() 2024-07-31T14:39:09.806 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-31T14:39:09.806 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:39:09.929 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-07-31T14:39:09.930 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-07-31T14:39:09.930 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-31T14:39:09.930 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-07-31T14:39:09.930 INFO:teuthology.orchestra.run.smithi067.stdout:mgr epoch 9 is available 2024-07-31T14:39:09.930 INFO:teuthology.orchestra.run.smithi067.stdout:Generating a dashboard self-signed certificate... 2024-07-31T14:39:10.218 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-mon[32750]: mgrmap e11: x(active, since 1.0076s) 2024-07-31T14:39:10.218 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-mon[32750]: [31/Jul/2024:14:39:09] ENGINE Bus STARTING 2024-07-31T14:39:10.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:10.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-mon[32750]: [31/Jul/2024:14:39:09] ENGINE Serving on https://172.21.15.67:7150 2024-07-31T14:39:10.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:09 smithi067 ceph-mon[32750]: [31/Jul/2024:14:39:09] ENGINE Error in HTTPServer.serve 2024-07-31T14:39:10.219 INFO:journalctl@ceph.mon.a.smithi067.stdout: Traceback (most recent call last): 2024-07-31T14:39:10.219 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-31T14:39:10.219 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._connections.run(self.expiration_interval) 2024-07-31T14:39:10.219 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-31T14:39:10.219 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._run(expiration_interval) 2024-07-31T14:39:10.219 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-31T14:39:10.219 INFO:journalctl@ceph.mon.a.smithi067.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-31T14:39:10.219 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-31T14:39:10.219 INFO:journalctl@ceph.mon.a.smithi067.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-31T14:39:10.220 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-31T14:39:10.220 INFO:journalctl@ceph.mon.a.smithi067.stdout: s = self.context.wrap_socket( 2024-07-31T14:39:10.220 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-31T14:39:10.220 INFO:journalctl@ceph.mon.a.smithi067.stdout: return self.sslsocket_class._create( 2024-07-31T14:39:10.220 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-31T14:39:10.220 INFO:journalctl@ceph.mon.a.smithi067.stdout: self.do_handshake() 2024-07-31T14:39:10.220 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-31T14:39:10.220 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._sslobj.do_handshake() 2024-07-31T14:39:10.220 INFO:journalctl@ceph.mon.a.smithi067.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-31T14:39:11.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:11 smithi067 ceph-mon[32750]: [31/Jul/2024:14:39:09] ENGINE Serving on http://172.21.15.67:8765 2024-07-31T14:39:11.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:11 smithi067 ceph-mon[32750]: [31/Jul/2024:14:39:09] ENGINE Bus STARTED 2024-07-31T14:39:11.520 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:11 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:11.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:11 smithi067 ceph-mon[32750]: mgrmap e12: x(active, since 2s) 2024-07-31T14:39:11.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:11 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:11.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:11 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:12.135 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-07-31T14:39:12.135 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial admin user... 2024-07-31T14:39:12.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:12 smithi067 ceph-mon[32750]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:39:13.591 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$/D8MJW4McVFyWct/hp0LJOotU/myjFq6V18qp6uCwKZfCkTmMtImS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1722436753, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-07-31T14:39:13.591 INFO:teuthology.orchestra.run.smithi067.stdout:Fetching dashboard port number... 2024-07-31T14:39:14.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:14 smithi067 ceph-mon[32750]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:39:14.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:14 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:14.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 8443 2024-07-31T14:39:14.841 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-31T14:39:14.841 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-07-31T14:39:14.848 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-31T14:39:14.848 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-07-31T14:39:14.849 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-07-31T14:39:14.849 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-07-31T14:39:14.850 INFO:teuthology.orchestra.run.smithi067.stdout:Ceph Dashboard is now available at: 2024-07-31T14:39:14.850 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:39:14.851 INFO:teuthology.orchestra.run.smithi067.stdout: URL: https://smithi067.front.sepia.ceph.com:8443/ 2024-07-31T14:39:14.851 INFO:teuthology.orchestra.run.smithi067.stdout: User: admin 2024-07-31T14:39:14.851 INFO:teuthology.orchestra.run.smithi067.stdout: Password: cv5qd789hl 2024-07-31T14:39:14.851 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:39:14.851 INFO:teuthology.orchestra.run.smithi067.stdout:Saving cluster configuration to /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config directory 2024-07-31T14:39:15.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:15 smithi067 ceph-mon[32750]: mgrmap e13: x(active, since 6s) 2024-07-31T14:39:15.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:15 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/4273516808' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-07-31T14:39:16.158 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-07-31T14:39:16.158 INFO:teuthology.orchestra.run.smithi067.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-07-31T14:39:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:39:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-07-31T14:39:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:39:16.159 INFO:teuthology.orchestra.run.smithi067.stdout:Or, if you are only running a single cluster on this host: 2024-07-31T14:39:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:39:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-07-31T14:39:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:39:16.159 INFO:teuthology.orchestra.run.smithi067.stdout:Please consider enabling telemetry to help improve Ceph: 2024-07-31T14:39:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:39:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: ceph telemetry on 2024-07-31T14:39:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:39:16.159 INFO:teuthology.orchestra.run.smithi067.stdout:For more information see: 2024-07-31T14:39:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:39:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-07-31T14:39:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:39:16.160 INFO:teuthology.orchestra.run.smithi067.stdout:Bootstrap complete. 2024-07-31T14:39:16.204 INFO:tasks.cephadm:Fetching config... 2024-07-31T14:39:16.204 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:39:16.204 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-07-31T14:39:16.222 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-07-31T14:39:16.222 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:39:16.223 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-07-31T14:39:16.279 INFO:tasks.cephadm:Fetching mon keyring... 2024-07-31T14:39:16.280 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:39:16.280 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/keyring of=/dev/stdout 2024-07-31T14:39:16.347 INFO:tasks.cephadm:Fetching pub ssh key... 2024-07-31T14:39:16.348 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:39:16.348 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-07-31T14:39:16.404 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-07-31T14:39:16.404 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDU+VKZc2gne+A0L9H+u+QDKq/Bwm9nICatRK9r7/KkIUjuANMKY7F4IPVnykL+GZvU/iD1Sst6Rv57djHb+5uZxsK9NoGIuUMaF1X/Fp6IozoEp6hQxI8C/Tf4VFicTPlFfHnjzqMV8PsHFG1zi7+kgnhsnwYFkJDnN2QYMuEXXup4CWxV6RWnK2jkUbgEH/ZSFqhg3hBZ1SCgdZh8WBOCy+1YJt0u2Wxph8uf+DTZugRqEiHq0sgVotraf9iRJbpyzScodUsDSTD0o9Q/lpe5uCGXIi6JngarF5m9gm/YkXoxh+F77qZ4r86msWTLzIXVksf9cRvHI+wwWRZQjkKA05h5NB+dIfl2BefDc3NUzSq9JkUFSRRlH533KZjnEqqdz6taHfWsKd6wkBNFVRdoUBxN/cx7Qv0pT0jq32ZYUyrvfFa8gSv1AOsd+y9kkmQl1+11IeGdizFkTDCCKRJ+N3W1ezthz/10GX3QLT77y/EfNoKECJp1F1jehehqJgk= ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-31T14:39:16.483 INFO:teuthology.orchestra.run.smithi067.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDU+VKZc2gne+A0L9H+u+QDKq/Bwm9nICatRK9r7/KkIUjuANMKY7F4IPVnykL+GZvU/iD1Sst6Rv57djHb+5uZxsK9NoGIuUMaF1X/Fp6IozoEp6hQxI8C/Tf4VFicTPlFfHnjzqMV8PsHFG1zi7+kgnhsnwYFkJDnN2QYMuEXXup4CWxV6RWnK2jkUbgEH/ZSFqhg3hBZ1SCgdZh8WBOCy+1YJt0u2Wxph8uf+DTZugRqEiHq0sgVotraf9iRJbpyzScodUsDSTD0o9Q/lpe5uCGXIi6JngarF5m9gm/YkXoxh+F77qZ4r86msWTLzIXVksf9cRvHI+wwWRZQjkKA05h5NB+dIfl2BefDc3NUzSq9JkUFSRRlH533KZjnEqqdz6taHfWsKd6wkBNFVRdoUBxN/cx7Qv0pT0jq32ZYUyrvfFa8gSv1AOsd+y9kkmQl1+11IeGdizFkTDCCKRJ+N3W1ezthz/10GX3QLT77y/EfNoKECJp1F1jehehqJgk= ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:39:16.495 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDU+VKZc2gne+A0L9H+u+QDKq/Bwm9nICatRK9r7/KkIUjuANMKY7F4IPVnykL+GZvU/iD1Sst6Rv57djHb+5uZxsK9NoGIuUMaF1X/Fp6IozoEp6hQxI8C/Tf4VFicTPlFfHnjzqMV8PsHFG1zi7+kgnhsnwYFkJDnN2QYMuEXXup4CWxV6RWnK2jkUbgEH/ZSFqhg3hBZ1SCgdZh8WBOCy+1YJt0u2Wxph8uf+DTZugRqEiHq0sgVotraf9iRJbpyzScodUsDSTD0o9Q/lpe5uCGXIi6JngarF5m9gm/YkXoxh+F77qZ4r86msWTLzIXVksf9cRvHI+wwWRZQjkKA05h5NB+dIfl2BefDc3NUzSq9JkUFSRRlH533KZjnEqqdz6taHfWsKd6wkBNFVRdoUBxN/cx7Qv0pT0jq32ZYUyrvfFa8gSv1AOsd+y9kkmQl1+11IeGdizFkTDCCKRJ+N3W1ezthz/10GX3QLT77y/EfNoKECJp1F1jehehqJgk= ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-31T14:39:16.534 INFO:teuthology.orchestra.run.smithi073.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDU+VKZc2gne+A0L9H+u+QDKq/Bwm9nICatRK9r7/KkIUjuANMKY7F4IPVnykL+GZvU/iD1Sst6Rv57djHb+5uZxsK9NoGIuUMaF1X/Fp6IozoEp6hQxI8C/Tf4VFicTPlFfHnjzqMV8PsHFG1zi7+kgnhsnwYFkJDnN2QYMuEXXup4CWxV6RWnK2jkUbgEH/ZSFqhg3hBZ1SCgdZh8WBOCy+1YJt0u2Wxph8uf+DTZugRqEiHq0sgVotraf9iRJbpyzScodUsDSTD0o9Q/lpe5uCGXIi6JngarF5m9gm/YkXoxh+F77qZ4r86msWTLzIXVksf9cRvHI+wwWRZQjkKA05h5NB+dIfl2BefDc3NUzSq9JkUFSRRlH533KZjnEqqdz6taHfWsKd6wkBNFVRdoUBxN/cx7Qv0pT0jq32ZYUyrvfFa8gSv1AOsd+y9kkmQl1+11IeGdizFkTDCCKRJ+N3W1ezthz/10GX3QLT77y/EfNoKECJp1F1jehehqJgk= ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:39:16.546 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDU+VKZc2gne+A0L9H+u+QDKq/Bwm9nICatRK9r7/KkIUjuANMKY7F4IPVnykL+GZvU/iD1Sst6Rv57djHb+5uZxsK9NoGIuUMaF1X/Fp6IozoEp6hQxI8C/Tf4VFicTPlFfHnjzqMV8PsHFG1zi7+kgnhsnwYFkJDnN2QYMuEXXup4CWxV6RWnK2jkUbgEH/ZSFqhg3hBZ1SCgdZh8WBOCy+1YJt0u2Wxph8uf+DTZugRqEiHq0sgVotraf9iRJbpyzScodUsDSTD0o9Q/lpe5uCGXIi6JngarF5m9gm/YkXoxh+F77qZ4r86msWTLzIXVksf9cRvHI+wwWRZQjkKA05h5NB+dIfl2BefDc3NUzSq9JkUFSRRlH533KZjnEqqdz6taHfWsKd6wkBNFVRdoUBxN/cx7Qv0pT0jq32ZYUyrvfFa8gSv1AOsd+y9kkmQl1+11IeGdizFkTDCCKRJ+N3W1ezthz/10GX3QLT77y/EfNoKECJp1F1jehehqJgk= ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-31T14:39:16.584 INFO:teuthology.orchestra.run.smithi077.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDU+VKZc2gne+A0L9H+u+QDKq/Bwm9nICatRK9r7/KkIUjuANMKY7F4IPVnykL+GZvU/iD1Sst6Rv57djHb+5uZxsK9NoGIuUMaF1X/Fp6IozoEp6hQxI8C/Tf4VFicTPlFfHnjzqMV8PsHFG1zi7+kgnhsnwYFkJDnN2QYMuEXXup4CWxV6RWnK2jkUbgEH/ZSFqhg3hBZ1SCgdZh8WBOCy+1YJt0u2Wxph8uf+DTZugRqEiHq0sgVotraf9iRJbpyzScodUsDSTD0o9Q/lpe5uCGXIi6JngarF5m9gm/YkXoxh+F77qZ4r86msWTLzIXVksf9cRvHI+wwWRZQjkKA05h5NB+dIfl2BefDc3NUzSq9JkUFSRRlH533KZjnEqqdz6taHfWsKd6wkBNFVRdoUBxN/cx7Qv0pT0jq32ZYUyrvfFa8gSv1AOsd+y9kkmQl1+11IeGdizFkTDCCKRJ+N3W1ezthz/10GX3QLT77y/EfNoKECJp1F1jehehqJgk= ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:39:16.595 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-07-31T14:39:16.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:16 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1997082287' entity='client.admin' 2024-07-31T14:39:16.866 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:39:20.238 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-07-31T14:39:20.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-07-31T14:39:20.581 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:39:20.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:20 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1279765199' entity='client.admin' 2024-07-31T14:39:22.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:21 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:22.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:21 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:22.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:21 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:39:22.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:21 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:22.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:21 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:39:22.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:21 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:23.393 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi073 2024-07-31T14:39:23.394 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-07-31T14:39:23.394 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/etc/ceph/ceph.conf 2024-07-31T14:39:23.412 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-07-31T14:39:23.412 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:39:23.469 INFO:tasks.cephadm:Adding host smithi073 to orchestrator... 2024-07-31T14:39:23.469 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch host add smithi073 2024-07-31T14:39:23.733 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:39:23.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:23 smithi067 ceph-mon[32750]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:39:23.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:23 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:23.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:23 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:39:23.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:23 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:39:23.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:23 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:39:23.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:23 smithi067 ceph-mon[32750]: Updating smithi067:/etc/ceph/ceph.conf 2024-07-31T14:39:23.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:23 smithi067 ceph-mon[32750]: Updating smithi067:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:39:23.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:23 smithi067 ceph-mon[32750]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:39:23.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:23 smithi067 ceph-mon[32750]: Updating smithi067:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-07-31T14:39:23.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:23 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:23.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:23 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:23.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:23 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:26.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:25 smithi067 ceph-mon[32750]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi073", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:39:27.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:26 smithi067 ceph-mon[32750]: Deploying cephadm binary to smithi073 2024-07-31T14:39:29.206 INFO:teuthology.orchestra.run.smithi067.stdout:Added host 'smithi073' with addr '172.21.15.73' 2024-07-31T14:39:29.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:29 smithi067 ceph-mon[32750]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:29.885 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch host ls --format=json 2024-07-31T14:39:30.148 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:39:30.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:30 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:30.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:30 smithi067 ceph-mon[32750]: Added host smithi073 2024-07-31T14:39:30.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:30 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:39:30.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:30 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:30.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:30 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:31.695 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:31 smithi067 ceph-mon[32750]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:31.695 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:31 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:31.999 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:39:32.000 INFO:teuthology.orchestra.run.smithi067.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.73", "hostname": "smithi073", "labels": [], "status": ""}] 2024-07-31T14:39:32.544 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi077 2024-07-31T14:39:32.544 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-07-31T14:39:32.545 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/etc/ceph/ceph.conf 2024-07-31T14:39:32.564 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-07-31T14:39:32.564 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:39:32.622 INFO:tasks.cephadm:Adding host smithi077 to orchestrator... 2024-07-31T14:39:32.622 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch host add smithi077 2024-07-31T14:39:32.883 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:39:33.696 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:33 smithi067 ceph-mon[32750]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:33.696 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:33 smithi067 ceph-mon[32750]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:39:33.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:33 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:33.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:33 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:33.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:33 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:33.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:33 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:33.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:33 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi073", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:39:33.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:33 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:39:33.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:33 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:39:34.575 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:34 smithi067 ceph-mon[32750]: Updating smithi073:/etc/ceph/ceph.conf 2024-07-31T14:39:34.576 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:34 smithi067 ceph-mon[32750]: Updating smithi073:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:39:34.576 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:34 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:34.576 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:34 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:34.576 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:34 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:36.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:35 smithi067 ceph-mon[32750]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:39:36.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:35 smithi067 ceph-mon[32750]: Updating smithi073:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-07-31T14:39:36.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:35 smithi067 ceph-mon[32750]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:37.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:36 smithi067 ceph-mon[32750]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi077", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:39:38.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:37 smithi067 ceph-mon[32750]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:38.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:37 smithi067 ceph-mon[32750]: Deploying cephadm binary to smithi077 2024-07-31T14:39:38.800 INFO:teuthology.orchestra.run.smithi067.stdout:Added host 'smithi077' with addr '172.21.15.77' 2024-07-31T14:39:39.534 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch host ls --format=json 2024-07-31T14:39:39.797 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:39:39.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:39 smithi067 ceph-mon[32750]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:39.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:39 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:39.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:39 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:39:39.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:39 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:39.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:39 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:40.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:40 smithi067 ceph-mon[32750]: Added host smithi077 2024-07-31T14:39:40.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:40 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:41.413 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:39:41.414 INFO:teuthology.orchestra.run.smithi067.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.73", "hostname": "smithi073", "labels": [], "status": ""}, {"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}] 2024-07-31T14:39:41.723 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:41 smithi067 ceph-mon[32750]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:42.019 INFO:tasks.cephadm:Setting crush tunables to default 2024-07-31T14:39:42.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd crush tunables default 2024-07-31T14:39:42.278 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:39:42.682 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:42 smithi067 ceph-mon[32750]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:39:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:43 smithi067 ceph-mon[32750]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:43 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:43 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:43 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:43 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:43 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:39:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:43 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:39:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:43 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:39:44.681 INFO:teuthology.orchestra.run.smithi067.stderr:adjusted tunables profile to default 2024-07-31T14:39:44.992 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:44 smithi067 ceph-mon[32750]: Updating smithi077:/etc/ceph/ceph.conf 2024-07-31T14:39:44.992 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:44 smithi067 ceph-mon[32750]: Updating smithi077:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:39:44.992 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:44 smithi067 ceph-mon[32750]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:39:44.992 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:44 smithi067 ceph-mon[32750]: Updating smithi077:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-07-31T14:39:44.992 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:44 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:44.993 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:44 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:44.993 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:44 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:44.993 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:44 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/276398814' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-31T14:39:45.252 INFO:tasks.cephadm:Adding mon.a on smithi067 2024-07-31T14:39:45.252 INFO:tasks.cephadm:Adding mon.b on smithi073 2024-07-31T14:39:45.252 INFO:tasks.cephadm:Adding mon.c on smithi077 2024-07-31T14:39:45.252 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch apply mon '3;smithi067:172.21.15.67=a;smithi073:172.21.15.73=b;smithi077:172.21.15.77=c' 2024-07-31T14:39:45.504 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:39:46.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:45 smithi067 ceph-mon[32750]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:46.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:45 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/276398814' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-31T14:39:46.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:45 smithi067 ceph-mon[32750]: osdmap e4: 0 total, 0 up, 0 in 2024-07-31T14:39:47.263 INFO:teuthology.orchestra.run.smithi077.stdout:Scheduled mon update... 2024-07-31T14:39:47.796 DEBUG:teuthology.orchestra.run.smithi073:mon.b> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.b.service 2024-07-31T14:39:47.799 DEBUG:teuthology.orchestra.run.smithi077:mon.c> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.c.service 2024-07-31T14:39:47.801 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-31T14:39:47.801 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph mon dump -f json 2024-07-31T14:39:47.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:47 smithi067 ceph-mon[32750]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:47.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:47 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:47.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:47 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:39:47.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:47 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:39:47.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:47 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:39:47.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:47 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:47.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:47 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-31T14:39:47.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:47 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:39:49.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:48 smithi067 ceph-mon[32750]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi073:172.21.15.73=b;smithi077:172.21.15.77=c", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:39:49.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:48 smithi067 ceph-mon[32750]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi073:172.21.15.73=b;smithi077:172.21.15.77=c;count:3 2024-07-31T14:39:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:48 smithi067 ceph-mon[32750]: Deploying daemon mon.c on smithi077 2024-07-31T14:39:49.450 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:39:50.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:49 smithi067 ceph-mon[32750]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:51.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:50 smithi067 ceph-mon[32750]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:51.826 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:39:51.826 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":1,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","modified":"2024-07-31T14:38:17.033740Z","created":"2024-07-31T14:38:17.033740Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-31T14:39:51.827 INFO:teuthology.orchestra.run.smithi077.stderr:dumped monmap epoch 1 2024-07-31T14:39:52.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:51 smithi077 ceph-mon[37327]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-07-31T14:39:52.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:51 smithi067 ceph-mon[32750]: from='client.? 172.21.15.77:0/2618527746' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-31T14:39:52.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:52 smithi077 bash[37285]: 31b92f1da4cc83986f33c7864d7e23dd71ae4ca83812ca35b442f6146fd87c3f 2024-07-31T14:39:52.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:52 smithi077 systemd[1]: Started Ceph mon.c for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:39:53.431 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-31T14:39:53.431 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph mon dump -f json 2024-07-31T14:39:53.688 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T14:39:57.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:39:57.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:39:57.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: mon.a calling monitor election 2024-07-31T14:39:57.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:57.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:39:57.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:39:57.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: mon.c calling monitor election 2024-07-31T14:39:57.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:57.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:39:57.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:39:57.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:57.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:39:57.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-31T14:39:57.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: monmap epoch 2 2024-07-31T14:39:57.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:39:57.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: last_changed 2024-07-31T14:39:51.906858+0000 2024-07-31T14:39:57.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: created 2024-07-31T14:38:17.033740+0000 2024-07-31T14:39:57.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: min_mon_release 19 (squid) 2024-07-31T14:39:57.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: election_strategy: 1 2024-07-31T14:39:57.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-31T14:39:57.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: 1: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.c 2024-07-31T14:39:57.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: fsmap 2024-07-31T14:39:57.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: osdmap e4: 0 total, 0 up, 0 in 2024-07-31T14:39:57.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: mgrmap e13: x(active, since 48s) 2024-07-31T14:39:57.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: overall HEALTH_OK 2024-07-31T14:39:57.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:57.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:57.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:57.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-31T14:39:57.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:56 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:39:57.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:39:57.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:39:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: mon.a calling monitor election 2024-07-31T14:39:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:39:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:39:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: mon.c calling monitor election 2024-07-31T14:39:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:39:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:39:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:39:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-31T14:39:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: monmap epoch 2 2024-07-31T14:39:57.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:39:57.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: last_changed 2024-07-31T14:39:51.906858+0000 2024-07-31T14:39:57.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: created 2024-07-31T14:38:17.033740+0000 2024-07-31T14:39:57.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: min_mon_release 19 (squid) 2024-07-31T14:39:57.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: election_strategy: 1 2024-07-31T14:39:57.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-31T14:39:57.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: 1: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.c 2024-07-31T14:39:57.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: fsmap 2024-07-31T14:39:57.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: osdmap e4: 0 total, 0 up, 0 in 2024-07-31T14:39:57.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: mgrmap e13: x(active, since 48s) 2024-07-31T14:39:57.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: overall HEALTH_OK 2024-07-31T14:39:57.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:57.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:57.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:39:57.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-31T14:39:57.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:56 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:39:58.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:57 smithi067 ceph-mon[32750]: Deploying daemon mon.b on smithi073 2024-07-31T14:39:58.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:57 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:39:58.313 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:39:58.313 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":2,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","modified":"2024-07-31T14:39:51.906858Z","created":"2024-07-31T14:38:17.033740Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-07-31T14:39:58.314 INFO:teuthology.orchestra.run.smithi077.stderr:dumped monmap epoch 2 2024-07-31T14:39:58.314 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:57 smithi077 ceph-mon[37327]: Deploying daemon mon.b on smithi073 2024-07-31T14:39:58.314 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:57 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:39:59.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:58 smithi077 ceph-mon[37327]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:59.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:39:58 smithi077 ceph-mon[37327]: from='client.? 172.21.15.77:0/3345077527' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-31T14:39:59.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:58 smithi067 ceph-mon[32750]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:39:59.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:39:58 smithi067 ceph-mon[32750]: from='client.? 172.21.15.77:0/3345077527' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-31T14:39:59.901 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-31T14:39:59.902 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph mon dump -f json 2024-07-31T14:40:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T14:40:00.791 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:00 smithi073 podman[36695]: 2024-07-31 14:40:00.79014406 +0000 UTC m=+0.132543919 container create 0cd68fed9b4c8667597b83db1d7fcad537ceb13b74995b471a11f494d765b8fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T14:40:01.041 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 podman[36695]: 2024-07-31 14:40:01.000565962 +0000 UTC m=+0.342965834 container init 0cd68fed9b4c8667597b83db1d7fcad537ceb13b74995b471a11f494d765b8fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240716, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-07-31T14:40:01.041 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 podman[36695]: 2024-07-31 14:40:01.003410172 +0000 UTC m=+0.345810033 container start 0cd68fed9b4c8667597b83db1d7fcad537ceb13b74995b471a11f494d765b8fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240716) 2024-07-31T14:40:01.041 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: set uid:gid to 167:167 (ceph:ceph) 2024-07-31T14:40:01.042 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: ceph version 19.0.0-5223-g8c796fca (8c796fca5ff5960f7542e7227bad4a784ee2c804) squid (dev), process ceph-mon, pid 2 2024-07-31T14:40:01.042 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: pidfile_write: ignore empty --pid-file 2024-07-31T14:40:01.042 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: load: jerasure load: lrc 2024-07-31T14:40:01.042 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: RocksDB version: 7.9.2 2024-07-31T14:40:01.042 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Git sha 0 2024-07-31T14:40:01.042 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Compile date 2024-07-23 15:17:47 2024-07-31T14:40:01.042 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: DB SUMMARY 2024-07-31T14:40:01.042 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: DB Session ID: DOR9SNTKXUEY7B0K4QTQ 2024-07-31T14:40:01.042 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: CURRENT file: CURRENT 2024-07-31T14:40:01.042 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: IDENTITY file: IDENTITY 2024-07-31T14:40:01.042 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-07-31T14:40:01.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-07-31T14:40:01.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-07-31T14:40:01.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.error_if_exists: 0 2024-07-31T14:40:01.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.create_if_missing: 0 2024-07-31T14:40:01.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.paranoid_checks: 1 2024-07-31T14:40:01.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.flush_verify_memtable_count: 1 2024-07-31T14:40:01.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-31T14:40:01.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-07-31T14:40:01.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.env: 0x562679ca02e0 2024-07-31T14:40:01.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.fs: PosixFileSystem 2024-07-31T14:40:01.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.info_log: 0x56267f3b1080 2024-07-31T14:40:01.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_file_opening_threads: 16 2024-07-31T14:40:01.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.statistics: (nil) 2024-07-31T14:40:01.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.use_fsync: 0 2024-07-31T14:40:01.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_log_file_size: 0 2024-07-31T14:40:01.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-31T14:40:01.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-31T14:40:01.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.keep_log_file_num: 1000 2024-07-31T14:40:01.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.recycle_log_file_num: 0 2024-07-31T14:40:01.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.allow_fallocate: 1 2024-07-31T14:40:01.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.allow_mmap_reads: 0 2024-07-31T14:40:01.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.allow_mmap_writes: 0 2024-07-31T14:40:01.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.use_direct_reads: 0 2024-07-31T14:40:01.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-31T14:40:01.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.create_missing_column_families: 0 2024-07-31T14:40:01.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.db_log_dir: 2024-07-31T14:40:01.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.wal_dir: 2024-07-31T14:40:01.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-31T14:40:01.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-31T14:40:01.047 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-31T14:40:01.047 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-31T14:40:01.047 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-31T14:40:01.047 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-31T14:40:01.047 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.advise_random_on_open: 1 2024-07-31T14:40:01.047 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.db_write_buffer_size: 0 2024-07-31T14:40:01.047 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.write_buffer_manager: 0x56267f3b55e0 2024-07-31T14:40:01.047 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-31T14:40:01.047 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-31T14:40:01.047 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-31T14:40:01.047 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.rate_limiter: (nil) 2024-07-31T14:40:01.048 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-31T14:40:01.048 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.wal_recovery_mode: 2 2024-07-31T14:40:01.048 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.enable_thread_tracking: 0 2024-07-31T14:40:01.048 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.enable_pipelined_write: 0 2024-07-31T14:40:01.048 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.unordered_write: 0 2024-07-31T14:40:01.048 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-31T14:40:01.048 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-31T14:40:01.048 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-31T14:40:01.048 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-31T14:40:01.048 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.row_cache: None 2024-07-31T14:40:01.048 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.wal_filter: None 2024-07-31T14:40:01.048 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-31T14:40:01.049 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.allow_ingest_behind: 0 2024-07-31T14:40:01.049 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.two_write_queues: 0 2024-07-31T14:40:01.049 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.manual_wal_flush: 0 2024-07-31T14:40:01.049 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.wal_compression: 0 2024-07-31T14:40:01.049 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.atomic_flush: 0 2024-07-31T14:40:01.049 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-31T14:40:01.049 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-31T14:40:01.049 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-31T14:40:01.049 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.log_readahead_size: 0 2024-07-31T14:40:01.049 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-31T14:40:01.049 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.best_efforts_recovery: 0 2024-07-31T14:40:01.049 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-31T14:40:01.050 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-31T14:40:01.050 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.allow_data_in_errors: 0 2024-07-31T14:40:01.050 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.db_host_id: __hostname__ 2024-07-31T14:40:01.050 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-31T14:40:01.050 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_background_jobs: 2 2024-07-31T14:40:01.050 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_background_compactions: -1 2024-07-31T14:40:01.050 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_subcompactions: 1 2024-07-31T14:40:01.050 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-31T14:40:01.050 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-31T14:40:01.050 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-31T14:40:01.050 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_total_wal_size: 0 2024-07-31T14:40:01.050 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-31T14:40:01.050 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-31T14:40:01.051 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-31T14:40:01.051 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-31T14:40:01.051 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_open_files: -1 2024-07-31T14:40:01.051 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.bytes_per_sync: 0 2024-07-31T14:40:01.051 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-31T14:40:01.051 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-31T14:40:01.051 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compaction_readahead_size: 0 2024-07-31T14:40:01.051 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_background_flushes: -1 2024-07-31T14:40:01.051 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Compression algorithms supported: 2024-07-31T14:40:01.051 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: kZSTD supported: 0 2024-07-31T14:40:01.051 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: kXpressCompression supported: 0 2024-07-31T14:40:01.051 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: kBZip2Compression supported: 0 2024-07-31T14:40:01.052 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-31T14:40:01.052 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: kLZ4Compression supported: 1 2024-07-31T14:40:01.052 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: kZlibCompression supported: 1 2024-07-31T14:40:01.052 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: kLZ4HCCompression supported: 1 2024-07-31T14:40:01.052 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: kSnappyCompression supported: 1 2024-07-31T14:40:01.052 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-31T14:40:01.052 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-31T14:40:01.052 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-07-31T14:40:01.052 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-31T14:40:01.052 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-31T14:40:01.052 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.merge_operator: 2024-07-31T14:40:01.052 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compaction_filter: None 2024-07-31T14:40:01.052 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compaction_filter_factory: None 2024-07-31T14:40:01.053 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.sst_partitioner_factory: None 2024-07-31T14:40:01.053 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-31T14:40:01.053 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-31T14:40:01.053 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56267f3b0c80) 2024-07-31T14:40:01.053 INFO:journalctl@ceph.mon.b.smithi073.stdout: cache_index_and_filter_blocks: 1 2024-07-31T14:40:01.053 INFO:journalctl@ceph.mon.b.smithi073.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-31T14:40:01.053 INFO:journalctl@ceph.mon.b.smithi073.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-31T14:40:01.053 INFO:journalctl@ceph.mon.b.smithi073.stdout: pin_top_level_index_and_filter: 1 2024-07-31T14:40:01.053 INFO:journalctl@ceph.mon.b.smithi073.stdout: index_type: 0 2024-07-31T14:40:01.053 INFO:journalctl@ceph.mon.b.smithi073.stdout: data_block_index_type: 0 2024-07-31T14:40:01.053 INFO:journalctl@ceph.mon.b.smithi073.stdout: index_shortening: 1 2024-07-31T14:40:01.053 INFO:journalctl@ceph.mon.b.smithi073.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-31T14:40:01.054 INFO:journalctl@ceph.mon.b.smithi073.stdout: checksum: 4 2024-07-31T14:40:01.054 INFO:journalctl@ceph.mon.b.smithi073.stdout: no_block_cache: 0 2024-07-31T14:40:01.054 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_cache: 0x56267b375610 2024-07-31T14:40:01.054 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_cache_name: BinnedLRUCache 2024-07-31T14:40:01.054 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_cache_options: 2024-07-31T14:40:01.054 INFO:journalctl@ceph.mon.b.smithi073.stdout: capacity : 536870912 2024-07-31T14:40:01.054 INFO:journalctl@ceph.mon.b.smithi073.stdout: num_shard_bits : 4 2024-07-31T14:40:01.054 INFO:journalctl@ceph.mon.b.smithi073.stdout: strict_capacity_limit : 0 2024-07-31T14:40:01.054 INFO:journalctl@ceph.mon.b.smithi073.stdout: high_pri_pool_ratio: 0.000 2024-07-31T14:40:01.054 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_cache_compressed: (nil) 2024-07-31T14:40:01.054 INFO:journalctl@ceph.mon.b.smithi073.stdout: persistent_cache: (nil) 2024-07-31T14:40:01.054 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_size: 4096 2024-07-31T14:40:01.054 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_size_deviation: 10 2024-07-31T14:40:01.055 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_restart_interval: 16 2024-07-31T14:40:01.055 INFO:journalctl@ceph.mon.b.smithi073.stdout: index_block_restart_interval: 1 2024-07-31T14:40:01.055 INFO:journalctl@ceph.mon.b.smithi073.stdout: metadata_block_size: 4096 2024-07-31T14:40:01.055 INFO:journalctl@ceph.mon.b.smithi073.stdout: partition_filters: 0 2024-07-31T14:40:01.055 INFO:journalctl@ceph.mon.b.smithi073.stdout: use_delta_encoding: 1 2024-07-31T14:40:01.055 INFO:journalctl@ceph.mon.b.smithi073.stdout: filter_policy: bloomfilter 2024-07-31T14:40:01.055 INFO:journalctl@ceph.mon.b.smithi073.stdout: whole_key_filtering: 1 2024-07-31T14:40:01.055 INFO:journalctl@ceph.mon.b.smithi073.stdout: verify_compression: 0 2024-07-31T14:40:01.055 INFO:journalctl@ceph.mon.b.smithi073.stdout: read_amp_bytes_per_bit: 0 2024-07-31T14:40:01.055 INFO:journalctl@ceph.mon.b.smithi073.stdout: format_version: 5 2024-07-31T14:40:01.055 INFO:journalctl@ceph.mon.b.smithi073.stdout: enable_index_compression: 1 2024-07-31T14:40:01.055 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_align: 0 2024-07-31T14:40:01.056 INFO:journalctl@ceph.mon.b.smithi073.stdout: max_auto_readahead_size: 262144 2024-07-31T14:40:01.056 INFO:journalctl@ceph.mon.b.smithi073.stdout: prepopulate_block_cache: 0 2024-07-31T14:40:01.056 INFO:journalctl@ceph.mon.b.smithi073.stdout: initial_auto_readahead_size: 8192 2024-07-31T14:40:01.056 INFO:journalctl@ceph.mon.b.smithi073.stdout: num_file_reads_for_auto_readahead: 2 2024-07-31T14:40:01.056 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.write_buffer_size: 33554432 2024-07-31T14:40:01.056 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_write_buffer_number: 2 2024-07-31T14:40:01.056 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compression: NoCompression 2024-07-31T14:40:01.056 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.bottommost_compression: Disabled 2024-07-31T14:40:01.056 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.prefix_extractor: nullptr 2024-07-31T14:40:01.056 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-31T14:40:01.056 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.num_levels: 7 2024-07-31T14:40:01.056 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-31T14:40:01.056 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-31T14:40:01.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-31T14:40:01.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-31T14:40:01.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-31T14:40:01.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-31T14:40:01.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-31T14:40:01.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-31T14:40:01.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-31T14:40:01.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-31T14:40:01.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-31T14:40:01.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-31T14:40:01.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-31T14:40:01.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compression_opts.level: 32767 2024-07-31T14:40:01.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compression_opts.strategy: 0 2024-07-31T14:40:01.058 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-31T14:40:01.058 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-31T14:40:01.058 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-31T14:40:01.058 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-31T14:40:01.058 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compression_opts.enabled: false 2024-07-31T14:40:01.058 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-31T14:40:01.058 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-31T14:40:01.058 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-31T14:40:01.058 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-31T14:40:01.058 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.target_file_size_base: 67108864 2024-07-31T14:40:01.058 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-31T14:40:01.058 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-31T14:40:01.058 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-31T14:40:01.059 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-31T14:40:01.059 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-31T14:40:01.059 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-31T14:40:01.059 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-31T14:40:01.059 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-31T14:40:01.059 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-31T14:40:01.059 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-31T14:40:01.059 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-31T14:40:01.059 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-31T14:40:01.059 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-31T14:40:01.059 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-31T14:40:01.059 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.arena_block_size: 1048576 2024-07-31T14:40:01.059 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-31T14:40:01.060 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-31T14:40:01.060 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.disable_auto_compactions: 0 2024-07-31T14:40:01.060 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-31T14:40:01.060 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-31T14:40:01.060 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-31T14:40:01.060 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-31T14:40:01.060 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-31T14:40:01.060 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-31T14:40:01.060 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-31T14:40:01.060 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-31T14:40:01.060 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-31T14:40:01.060 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-31T14:40:01.060 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-07-31T14:40:01.061 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.inplace_update_support: 0 2024-07-31T14:40:01.061 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-31T14:40:01.061 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-31T14:40:01.061 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-31T14:40:01.061 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-31T14:40:01.061 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.bloom_locality: 0 2024-07-31T14:40:01.061 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.max_successive_merges: 0 2024-07-31T14:40:01.061 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-31T14:40:01.061 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.paranoid_file_checks: 0 2024-07-31T14:40:01.061 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.force_consistency_checks: 1 2024-07-31T14:40:01.061 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.report_bg_io_stats: 0 2024-07-31T14:40:01.061 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.ttl: 2592000 2024-07-31T14:40:01.061 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-31T14:40:01.062 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-31T14:40:01.062 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-31T14:40:01.062 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.enable_blob_files: false 2024-07-31T14:40:01.062 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.min_blob_size: 0 2024-07-31T14:40:01.062 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.blob_file_size: 268435456 2024-07-31T14:40:01.062 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-31T14:40:01.062 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-31T14:40:01.062 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-31T14:40:01.062 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-31T14:40:01.062 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-31T14:40:01.062 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.blob_file_starting_level: 0 2024-07-31T14:40:01.062 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-31T14:40:01.062 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-07-31T14:40:01.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-07-31T14:40:01.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3df2d519-a3c6-496d-8e7f-8715608450c1 2024-07-31T14:40:01.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722436801039156, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-07-31T14:40:01.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-07-31T14:40:01.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722436801039575, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1722436801, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3df2d519-a3c6-496d-8e7f-8715608450c1", "db_session_id": "DOR9SNTKXUEY7B0K4QTQ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-07-31T14:40:01.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722436801039667, "job": 1, "event": "recovery_finished"} 2024-07-31T14:40:01.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-07-31T14:40:01.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-31T14:40:01.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56267f3d2a80 2024-07-31T14:40:01.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: DB pointer 0x562680172000 2024-07-31T14:40:01.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-31T14:40:01.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-31T14:40:01.064 INFO:journalctl@ceph.mon.b.smithi073.stdout: ** DB Stats ** 2024-07-31T14:40:01.064 INFO:journalctl@ceph.mon.b.smithi073.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-31T14:40:01.064 INFO:journalctl@ceph.mon.b.smithi073.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-31T14:40:01.064 INFO:journalctl@ceph.mon.b.smithi073.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-31T14:40:01.064 INFO:journalctl@ceph.mon.b.smithi073.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-31T14:40:01.064 INFO:journalctl@ceph.mon.b.smithi073.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-31T14:40:01.064 INFO:journalctl@ceph.mon.b.smithi073.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-31T14:40:01.064 INFO:journalctl@ceph.mon.b.smithi073.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-31T14:40:01.064 INFO:journalctl@ceph.mon.b.smithi073.stdout: 2024-07-31T14:40:01.064 INFO:journalctl@ceph.mon.b.smithi073.stdout: ** Compaction Stats [default] ** 2024-07-31T14:40:01.064 INFO:journalctl@ceph.mon.b.smithi073.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-31T14:40:01.064 INFO:journalctl@ceph.mon.b.smithi073.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-31T14:40:01.064 INFO:journalctl@ceph.mon.b.smithi073.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-07-31T14:40:01.065 INFO:journalctl@ceph.mon.b.smithi073.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-07-31T14:40:01.065 INFO:journalctl@ceph.mon.b.smithi073.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-07-31T14:40:01.065 INFO:journalctl@ceph.mon.b.smithi073.stdout: 2024-07-31T14:40:01.065 INFO:journalctl@ceph.mon.b.smithi073.stdout: ** Compaction Stats [default] ** 2024-07-31T14:40:01.065 INFO:journalctl@ceph.mon.b.smithi073.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-31T14:40:01.065 INFO:journalctl@ceph.mon.b.smithi073.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-31T14:40:01.065 INFO:journalctl@ceph.mon.b.smithi073.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-07-31T14:40:01.065 INFO:journalctl@ceph.mon.b.smithi073.stdout: 2024-07-31T14:40:01.065 INFO:journalctl@ceph.mon.b.smithi073.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-31T14:40:01.065 INFO:journalctl@ceph.mon.b.smithi073.stdout: 2024-07-31T14:40:01.065 INFO:journalctl@ceph.mon.b.smithi073.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-31T14:40:01.065 INFO:journalctl@ceph.mon.b.smithi073.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-31T14:40:01.065 INFO:journalctl@ceph.mon.b.smithi073.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-31T14:40:01.066 INFO:journalctl@ceph.mon.b.smithi073.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-31T14:40:01.066 INFO:journalctl@ceph.mon.b.smithi073.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-31T14:40:01.066 INFO:journalctl@ceph.mon.b.smithi073.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-31T14:40:01.066 INFO:journalctl@ceph.mon.b.smithi073.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-31T14:40:01.066 INFO:journalctl@ceph.mon.b.smithi073.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-31T14:40:01.066 INFO:journalctl@ceph.mon.b.smithi073.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-31T14:40:01.066 INFO:journalctl@ceph.mon.b.smithi073.stdout: Block cache BinnedLRUCache@0x56267b375610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-07-31T14:40:01.066 INFO:journalctl@ceph.mon.b.smithi073.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-07-31T14:40:01.066 INFO:journalctl@ceph.mon.b.smithi073.stdout: 2024-07-31T14:40:01.066 INFO:journalctl@ceph.mon.b.smithi073.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-31T14:40:01.066 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-07-31T14:40:01.066 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: using public_addr v2:172.21.15.73:0/0 -> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] 2024-07-31T14:40:01.066 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: starting mon.b rank -1 at public addrs [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] at bind addrs [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:40:01.067 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b@-1(???) e0 preinit fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:40:01.207 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:00 smithi077 ceph-mon[37327]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.207 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:00 smithi077 ceph-mon[37327]: overall HEALTH_OK 2024-07-31T14:40:01.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:00 smithi067 ceph-mon[32750]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:00 smithi067 ceph-mon[32750]: overall HEALTH_OK 2024-07-31T14:40:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b@-1(synchronizing).mds e1 new map 2024-07-31T14:40:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b@-1(synchronizing).mds e1 print_map 2024-07-31T14:40:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout: e1 2024-07-31T14:40:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout: btime 2024-07-31T14:38:19:925493+0000 2024-07-31T14:40:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-31T14:40:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-31T14:40:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout: legacy client fscid: -1 2024-07-31T14:40:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout: 2024-07-31T14:40:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout: No filesystems configured 2024-07-31T14:40:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-07-31T14:40:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-07-31T14:40:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-07-31T14:40:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-07-31T14:40:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-07-31T14:40:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-07-31T14:40:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-31T14:40:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-31T14:40:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-31T14:40:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-31T14:40:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi077", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:40:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: Deploying cephadm binary to smithi077 2024-07-31T14:40:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: Added host smithi077 2024-07-31T14:40:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:40:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:40:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: Updating smithi077:/etc/ceph/ceph.conf 2024-07-31T14:40:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: Updating smithi077:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:40:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: Updating smithi077:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-07-31T14:40:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/276398814' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-31T14:40:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/276398814' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-31T14:40:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: osdmap e4: 0 total, 0 up, 0 in 2024-07-31T14:40:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-31T14:40:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi073:172.21.15.73=b;smithi077:172.21.15.77=c", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:40:01.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi073:172.21.15.73=b;smithi077:172.21.15.77=c;count:3 2024-07-31T14:40:01.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: Deploying daemon mon.c on smithi077 2024-07-31T14:40:01.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='client.? 172.21.15.77:0/2618527746' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-31T14:40:01.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:40:01.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:40:01.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.a calling monitor election 2024-07-31T14:40:01.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:40:01.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:40:01.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.c calling monitor election 2024-07-31T14:40:01.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:40:01.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:40:01.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:40:01.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-31T14:40:01.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: monmap epoch 2 2024-07-31T14:40:01.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:40:01.400 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: last_changed 2024-07-31T14:39:51.906858+0000 2024-07-31T14:40:01.400 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: created 2024-07-31T14:38:17.033740+0000 2024-07-31T14:40:01.400 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: min_mon_release 19 (squid) 2024-07-31T14:40:01.400 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: election_strategy: 1 2024-07-31T14:40:01.400 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-31T14:40:01.400 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: 1: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.c 2024-07-31T14:40:01.400 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: fsmap 2024-07-31T14:40:01.400 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: osdmap e4: 0 total, 0 up, 0 in 2024-07-31T14:40:01.400 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mgrmap e13: x(active, since 48s) 2024-07-31T14:40:01.400 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: overall HEALTH_OK 2024-07-31T14:40:01.400 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.400 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.401 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:01.401 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-31T14:40:01.401 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:01.401 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: Deploying daemon mon.b on smithi073 2024-07-31T14:40:01.401 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:40:01.401 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.401 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: from='client.? 172.21.15.77:0/3345077527' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-31T14:40:01.401 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:01.401 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: overall HEALTH_OK 2024-07-31T14:40:01.401 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 ceph-mon[36714]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-07-31T14:40:01.401 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 bash[36695]: 0cd68fed9b4c8667597b83db1d7fcad537ceb13b74995b471a11f494d765b8fe 2024-07-31T14:40:01.401 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:01 smithi073 systemd[1]: Started Ceph mon.b for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:40:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:40:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: mon.a calling monitor election 2024-07-31T14:40:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:40:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: mon.c calling monitor election 2024-07-31T14:40:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: mon.b calling monitor election 2024-07-31T14:40:06.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:06.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-31T14:40:06.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: monmap epoch 3 2024-07-31T14:40:06.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:40:06.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: last_changed 2024-07-31T14:40:01.066752+0000 2024-07-31T14:40:06.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: created 2024-07-31T14:38:17.033740+0000 2024-07-31T14:40:06.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: min_mon_release 19 (squid) 2024-07-31T14:40:06.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: election_strategy: 1 2024-07-31T14:40:06.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-31T14:40:06.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: 1: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.c 2024-07-31T14:40:06.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: 2: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.b 2024-07-31T14:40:06.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: fsmap 2024-07-31T14:40:06.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: osdmap e4: 0 total, 0 up, 0 in 2024-07-31T14:40:06.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: mgrmap e13: x(active, since 58s) 2024-07-31T14:40:06.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: overall HEALTH_OK 2024-07-31T14:40:06.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:06.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:06.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:06.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:06 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:06.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:40:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: mon.a calling monitor election 2024-07-31T14:40:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:40:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: mon.c calling monitor election 2024-07-31T14:40:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: mon.b calling monitor election 2024-07-31T14:40:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-31T14:40:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: monmap epoch 3 2024-07-31T14:40:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:40:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: last_changed 2024-07-31T14:40:01.066752+0000 2024-07-31T14:40:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: created 2024-07-31T14:38:17.033740+0000 2024-07-31T14:40:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: min_mon_release 19 (squid) 2024-07-31T14:40:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: election_strategy: 1 2024-07-31T14:40:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-31T14:40:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: 1: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.c 2024-07-31T14:40:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: 2: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.b 2024-07-31T14:40:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: fsmap 2024-07-31T14:40:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: osdmap e4: 0 total, 0 up, 0 in 2024-07-31T14:40:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: mgrmap e13: x(active, since 58s) 2024-07-31T14:40:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: overall HEALTH_OK 2024-07-31T14:40:06.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:06.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:06.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:06.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:06 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:06.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:40:06.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: mon.a calling monitor election 2024-07-31T14:40:06.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:40:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: mon.c calling monitor election 2024-07-31T14:40:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: mon.b calling monitor election 2024-07-31T14:40:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-31T14:40:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: monmap epoch 3 2024-07-31T14:40:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:40:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: last_changed 2024-07-31T14:40:01.066752+0000 2024-07-31T14:40:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: created 2024-07-31T14:38:17.033740+0000 2024-07-31T14:40:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: min_mon_release 19 (squid) 2024-07-31T14:40:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: election_strategy: 1 2024-07-31T14:40:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-31T14:40:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: 1: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.c 2024-07-31T14:40:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: 2: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.b 2024-07-31T14:40:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: fsmap 2024-07-31T14:40:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: osdmap e4: 0 total, 0 up, 0 in 2024-07-31T14:40:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: mgrmap e13: x(active, since 58s) 2024-07-31T14:40:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: overall HEALTH_OK 2024-07-31T14:40:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:06 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:07.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:07 smithi073 ceph-mon[36714]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:07 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:07 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:07 smithi077 ceph-mon[37327]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:07 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:07 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:07.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:07 smithi067 ceph-mon[32750]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:07.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:07 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:07.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:07 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:07.683 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-07-31T14:40:07.692 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":3,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","modified":"2024-07-31T14:40:01.066752Z","created":"2024-07-31T14:38:17.033740Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:3300","nonce":0},{"type":"v1","addr":"172.21.15.73:6789","nonce":0}]},"addr":"172.21.15.73:6789/0","public_addr":"172.21.15.73:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-07-31T14:40:07.692 INFO:teuthology.orchestra.run.smithi077.stderr:dumped monmap epoch 3 2024-07-31T14:40:08.232 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-07-31T14:40:08.232 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config generate-minimal-conf 2024-07-31T14:40:08.345 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.345 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.345 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.346 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.346 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:08.346 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:08.346 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: Updating smithi067:/etc/ceph/ceph.conf 2024-07-31T14:40:08.346 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: Updating smithi073:/etc/ceph/ceph.conf 2024-07-31T14:40:08.346 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: Updating smithi077:/etc/ceph/ceph.conf 2024-07-31T14:40:08.346 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: Updating smithi073:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:08.346 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: Updating smithi077:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:08.346 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: Updating smithi067:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:08.346 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.346 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='client.? 172.21.15.77:0/3048601681' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-31T14:40:08.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-31T14:40:08.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-31T14:40:08.348 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:08.348 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:08 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:08.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:08.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: Updating smithi067:/etc/ceph/ceph.conf 2024-07-31T14:40:08.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: Updating smithi073:/etc/ceph/ceph.conf 2024-07-31T14:40:08.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: Updating smithi077:/etc/ceph/ceph.conf 2024-07-31T14:40:08.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: Updating smithi073:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:08.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: Updating smithi077:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:08.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: Updating smithi067:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:08.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='client.? 172.21.15.77:0/3048601681' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-31T14:40:08.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-31T14:40:08.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-31T14:40:08.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:08.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:08 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: Updating smithi067:/etc/ceph/ceph.conf 2024-07-31T14:40:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: Updating smithi073:/etc/ceph/ceph.conf 2024-07-31T14:40:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: Updating smithi077:/etc/ceph/ceph.conf 2024-07-31T14:40:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: Updating smithi073:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: Updating smithi077:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: Updating smithi067:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='client.? 172.21.15.77:0/3048601681' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-31T14:40:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-31T14:40:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-31T14:40:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:08 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:09.255 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:09.456 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:09 smithi067 ceph-mon[32750]: Reconfiguring mon.a (unknown last config time)... 2024-07-31T14:40:09.457 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:09 smithi067 ceph-mon[32750]: Reconfiguring daemon mon.a on smithi067 2024-07-31T14:40:09.457 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:09 smithi067 ceph-mon[32750]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:09 smithi077 ceph-mon[37327]: Reconfiguring mon.a (unknown last config time)... 2024-07-31T14:40:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:09 smithi077 ceph-mon[37327]: Reconfiguring daemon mon.a on smithi067 2024-07-31T14:40:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:09 smithi077 ceph-mon[37327]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:09.589 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:09 smithi073 ceph-mon[36714]: Reconfiguring mon.a (unknown last config time)... 2024-07-31T14:40:09.589 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:09 smithi073 ceph-mon[36714]: Reconfiguring daemon mon.a on smithi067 2024-07-31T14:40:09.589 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:09 smithi073 ceph-mon[36714]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:10.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:10 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:10.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:10 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:10.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:10 smithi077 ceph-mon[37327]: Reconfiguring mon.b (monmap changed)... 2024-07-31T14:40:10.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:10 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-31T14:40:10.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:10 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-31T14:40:10.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:10 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:10.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:10 smithi077 ceph-mon[37327]: Reconfiguring daemon mon.b on smithi073 2024-07-31T14:40:10.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:10 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:10.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:10 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:10.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:10 smithi067 ceph-mon[32750]: Reconfiguring mon.b (monmap changed)... 2024-07-31T14:40:10.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:10 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-31T14:40:10.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:10 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-31T14:40:10.508 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:10 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:10.508 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:10 smithi067 ceph-mon[32750]: Reconfiguring daemon mon.b on smithi073 2024-07-31T14:40:10.592 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:10 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:10.592 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:10 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:10.593 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:10 smithi073 ceph-mon[36714]: Reconfiguring mon.b (monmap changed)... 2024-07-31T14:40:10.593 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:10 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-31T14:40:10.593 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:10 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-31T14:40:10.593 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:10 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:10.593 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:10 smithi073 ceph-mon[36714]: Reconfiguring daemon mon.b on smithi073 2024-07-31T14:40:10.896 INFO:teuthology.orchestra.run.smithi067.stdout:# minimal ceph.conf for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:40:10.896 INFO:teuthology.orchestra.run.smithi067.stdout:[global] 2024-07-31T14:40:10.896 INFO:teuthology.orchestra.run.smithi067.stdout: fsid = 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:40:10.896 INFO:teuthology.orchestra.run.smithi067.stdout: mon_host = [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] 2024-07-31T14:40:11.468 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-07-31T14:40:11.468 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:40:11.468 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.conf 2024-07-31T14:40:11.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:11 smithi067 ceph-mon[32750]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:11.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:11 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:11.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:11 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:11.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:11 smithi067 ceph-mon[32750]: Reconfiguring mon.c (monmap changed)... 2024-07-31T14:40:11.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:11 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-31T14:40:11.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:11 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-31T14:40:11.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:11 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:11.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:11 smithi067 ceph-mon[32750]: Reconfiguring daemon mon.c on smithi077 2024-07-31T14:40:11.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:11 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3176609086' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:11.499 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:40:11.499 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:40:11.565 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-07-31T14:40:11.565 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.conf 2024-07-31T14:40:11.593 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-07-31T14:40:11.593 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:40:11.657 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-07-31T14:40:11.658 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.conf 2024-07-31T14:40:11.685 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-07-31T14:40:11.685 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:40:11.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:11 smithi077 ceph-mon[37327]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:11.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:11 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:11.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:11 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:11.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:11 smithi077 ceph-mon[37327]: Reconfiguring mon.c (monmap changed)... 2024-07-31T14:40:11.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:11 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-31T14:40:11.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:11 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-31T14:40:11.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:11 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:11.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:11 smithi077 ceph-mon[37327]: Reconfiguring daemon mon.c on smithi077 2024-07-31T14:40:11.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:11 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3176609086' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:11.750 INFO:tasks.cephadm:enabling module snap_schedule 2024-07-31T14:40:11.750 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-07-31T14:40:11.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:11 smithi073 ceph-mon[36714]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:11.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:11 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:11.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:11 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:11.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:11 smithi073 ceph-mon[36714]: Reconfiguring mon.c (monmap changed)... 2024-07-31T14:40:11.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:11 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-31T14:40:11.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:11 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-31T14:40:11.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:11 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:11.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:11 smithi073 ceph-mon[36714]: Reconfiguring daemon mon.c on smithi077 2024-07-31T14:40:11.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:11 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3176609086' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:12.023 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:13.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:12 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:13.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:12 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:13.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:12 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:13.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:12 smithi067 ceph-mon[32750]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:13.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:12 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:13.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:12 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:13.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:12 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:13.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:12 smithi073 ceph-mon[36714]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:13.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:12 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:13.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:12 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:13.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:12 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:13.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:12 smithi077 ceph-mon[37327]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:14.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:13 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:14.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:13 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:14.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:13 smithi073 ceph-mon[36714]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:14.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:13 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3868280175' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-31T14:40:14.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:13 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:14.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:13 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:14.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:13 smithi077 ceph-mon[37327]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:14.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:13 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3868280175' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-31T14:40:14.286 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:13 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:14.286 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:13 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:14.286 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:13 smithi067 ceph-mon[32750]: from='mgr.14150 172.21.15.67:0/643080299' entity='mgr.x' 2024-07-31T14:40:14.286 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:13 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3868280175' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-31T14:40:14.286 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:13 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: ignoring --setuser ceph since I am not root 2024-07-31T14:40:14.286 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:13 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: ignoring --setgroup ceph since I am not root 2024-07-31T14:40:14.463 INFO:tasks.cephadm:Adding mgr.x on smithi067 2024-07-31T14:40:14.464 INFO:tasks.cephadm:Adding mgr.y on smithi073 2024-07-31T14:40:14.464 INFO:tasks.cephadm:Adding mgr.z on smithi077 2024-07-31T14:40:14.464 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch apply mgr '3;smithi067=x;smithi073=y;smithi077=z' 2024-07-31T14:40:14.718 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T14:40:15.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:14 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3868280175' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-31T14:40:15.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:14 smithi073 ceph-mon[36714]: mgrmap e14: x(active, since 65s) 2024-07-31T14:40:15.155 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:14 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3868280175' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-31T14:40:15.155 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:14 smithi077 ceph-mon[37327]: mgrmap e14: x(active, since 65s) 2024-07-31T14:40:15.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:14 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3868280175' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-31T14:40:15.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:14 smithi067 ceph-mon[32750]: mgrmap e14: x(active, since 65s) 2024-07-31T14:40:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:19 smithi073 ceph-mon[36714]: Active manager daemon x restarted 2024-07-31T14:40:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:19 smithi073 ceph-mon[36714]: Activating manager daemon x 2024-07-31T14:40:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:19 smithi073 ceph-mon[36714]: osdmap e5: 0 total, 0 up, 0 in 2024-07-31T14:40:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:19 smithi073 ceph-mon[36714]: mgrmap e15: x(active, starting, since 0.00452274s) 2024-07-31T14:40:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:40:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:40:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-31T14:40:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-31T14:40:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-31T14:40:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-31T14:40:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:19 smithi073 ceph-mon[36714]: Manager daemon x is now available 2024-07-31T14:40:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:19.699 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:19 smithi077 ceph-mon[37327]: Active manager daemon x restarted 2024-07-31T14:40:19.699 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:19 smithi077 ceph-mon[37327]: Activating manager daemon x 2024-07-31T14:40:19.699 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:19 smithi077 ceph-mon[37327]: osdmap e5: 0 total, 0 up, 0 in 2024-07-31T14:40:19.699 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:19 smithi077 ceph-mon[37327]: mgrmap e15: x(active, starting, since 0.00452274s) 2024-07-31T14:40:19.699 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:40:19.699 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:19.699 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:40:19.699 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-31T14:40:19.699 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-31T14:40:19.699 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-31T14:40:19.700 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-31T14:40:19.700 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:19 smithi077 ceph-mon[37327]: Manager daemon x is now available 2024-07-31T14:40:19.700 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:19.707 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:19 smithi067 ceph-mon[32750]: Active manager daemon x restarted 2024-07-31T14:40:19.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:19 smithi067 ceph-mon[32750]: Activating manager daemon x 2024-07-31T14:40:19.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:19 smithi067 ceph-mon[32750]: osdmap e5: 0 total, 0 up, 0 in 2024-07-31T14:40:19.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:19 smithi067 ceph-mon[32750]: mgrmap e15: x(active, starting, since 0.00452274s) 2024-07-31T14:40:19.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:40:19.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:40:19.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:40:19.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-31T14:40:19.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-31T14:40:19.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-31T14:40:19.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-31T14:40:19.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:19 smithi067 ceph-mon[32750]: Manager daemon x is now available 2024-07-31T14:40:19.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:20.300 INFO:teuthology.orchestra.run.smithi077.stdout:Scheduled mgr update... 2024-07-31T14:40:20.581 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-31T14:40:20.582 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-31T14:40:20.582 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:20 smithi077 ceph-mon[37327]: mgrmap e16: x(active, since 1.00846s) 2024-07-31T14:40:20.582 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:20.596 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-31T14:40:20.596 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-31T14:40:20.596 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:20 smithi073 ceph-mon[36714]: mgrmap e16: x(active, since 1.00846s) 2024-07-31T14:40:20.596 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:20.601 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-31T14:40:20.601 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-31T14:40:20.601 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:20 smithi067 ceph-mon[32750]: mgrmap e16: x(active, since 1.00846s) 2024-07-31T14:40:20.601 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:20.836 DEBUG:teuthology.orchestra.run.smithi073:mgr.y> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mgr.y.service 2024-07-31T14:40:20.838 DEBUG:teuthology.orchestra.run.smithi077:mgr.z> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mgr.z.service 2024-07-31T14:40:20.841 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-07-31T14:40:20.841 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-07-31T14:40:20.844 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-07-31T14:40:20.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set client debug_client 20 2024-07-31T14:40:20.887 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-07-31T14:40:20.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set client debug_ms 1 2024-07-31T14:40:20.930 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-07-31T14:40:20.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-07-31T14:40:20.973 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-07-31T14:40:20.973 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-07-31T14:40:21.016 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-07-31T14:40:21.016 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mds debug_mds 20 2024-07-31T14:40:21.060 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-07-31T14:40:21.060 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-07-31T14:40:21.123 INFO:tasks.cephadm:setting config [mds] debug_ms = 1 2024-07-31T14:40:21.123 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mds debug_ms 1 2024-07-31T14:40:21.179 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-07-31T14:40:21.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-07-31T14:40:21.209 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:21.218 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-07-31T14:40:21.219 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-07-31T14:40:21.265 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-07-31T14:40:21.265 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-07-31T14:40:21.319 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-07-31T14:40:21.319 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-07-31T14:40:21.373 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-07-31T14:40:21.373 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-07-31T14:40:21.421 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-07-31T14:40:21.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-07-31T14:40:21.476 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-07-31T14:40:21.476 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-07-31T14:40:21.537 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-07-31T14:40:21.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-07-31T14:40:21.597 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-07-31T14:40:21.598 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-07-31T14:40:21.660 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-07-31T14:40:21.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-07-31T14:40:21.715 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-07-31T14:40:21.715 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-07-31T14:40:21.773 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-07-31T14:40:21.773 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mgr debug_cephsqlite 20 2024-07-31T14:40:21.824 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-07-31T14:40:21.825 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-31T14:40:21.868 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-07-31T14:40:21.868 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-31T14:40:21.914 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-07-31T14:40:21.914 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-07-31T14:40:21.966 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-07-31T14:40:21.966 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-07-31T14:40:22.019 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-07-31T14:40:22.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-07-31T14:40:22.082 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-07-31T14:40:22.082 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-07-31T14:40:22.141 DEBUG:tasks.cephadm:set 26 configs 2024-07-31T14:40:22.141 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi067') 2024-07-31T14:40:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:21 smithi073 ceph-mon[36714]: Saving service mgr spec with placement smithi067=x;smithi073=y;smithi077=z;count:3 2024-07-31T14:40:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:21 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:21 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:21 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:21 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:21 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:21 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:21 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:21 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:21 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi073", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:40:22.161 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:21 smithi067 ceph-mon[32750]: Saving service mgr spec with placement smithi067=x;smithi073=y;smithi077=z;count:3 2024-07-31T14:40:22.161 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:21 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.161 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:21 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.161 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:21 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.161 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:21 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.161 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:21 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.161 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:21 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.161 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:21 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.162 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:21 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.162 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:21 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi073", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:40:22.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:21 smithi077 ceph-mon[37327]: Saving service mgr spec with placement smithi067=x;smithi073=y;smithi077=z;count:3 2024-07-31T14:40:22.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:21 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:21 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:21 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:21 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:21 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:21 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:21 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:21 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:22.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:21 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi073", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:40:23.062 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:23.065 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:23.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:22 smithi073 ceph-mon[36714]: mgrmap e17: x(active, since 2s) 2024-07-31T14:40:23.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:23.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:23.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:40:23.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:23.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:23.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:40:23.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:23.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:23.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-mon[32750]: mgrmap e17: x(active, since 2s) 2024-07-31T14:40:23.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:23.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:23.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:40:23.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:23.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:23.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:40:23.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:23.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:23.173 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:40:22.877+0000 7f4cc1500640 -1 log_channel(cephadm) log [ERR] : [31/Jul/2024:14:40:22] ENGINE Error in HTTPServer.serve 2024-07-31T14:40:23.173 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: Traceback (most recent call last): 2024-07-31T14:40:23.173 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-31T14:40:23.174 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: self._connections.run(self.expiration_interval) 2024-07-31T14:40:23.174 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-31T14:40:23.174 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: self._run(expiration_interval) 2024-07-31T14:40:23.174 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-31T14:40:23.174 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: new_conn = self._from_server_socket(self.server.socket) 2024-07-31T14:40:23.174 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-31T14:40:23.174 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-31T14:40:23.174 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-31T14:40:23.174 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: s = self.context.wrap_socket( 2024-07-31T14:40:23.174 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-31T14:40:23.174 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: return self.sslsocket_class._create( 2024-07-31T14:40:23.174 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-31T14:40:23.174 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: self.do_handshake() 2024-07-31T14:40:23.175 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-31T14:40:23.175 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: self._sslobj.do_handshake() 2024-07-31T14:40:23.175 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-31T14:40:23.175 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:40:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:40:23.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:22 smithi077 ceph-mon[37327]: mgrmap e17: x(active, since 2s) 2024-07-31T14:40:23.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:23.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:23.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:40:23.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:23.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:23.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:40:23.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:23.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:23.727 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:23.734 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:23.738 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:23.952 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:24.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:23 smithi067 ceph-mon[32750]: [31/Jul/2024:14:40:22] ENGINE Bus STARTING 2024-07-31T14:40:24.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:23 smithi067 ceph-mon[32750]: [31/Jul/2024:14:40:22] ENGINE Serving on http://172.21.15.67:8765 2024-07-31T14:40:24.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:23 smithi067 ceph-mon[32750]: Updating smithi067:/etc/ceph/ceph.conf 2024-07-31T14:40:24.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:23 smithi067 ceph-mon[32750]: Updating smithi073:/etc/ceph/ceph.conf 2024-07-31T14:40:24.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:23 smithi067 ceph-mon[32750]: Updating smithi077:/etc/ceph/ceph.conf 2024-07-31T14:40:24.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:23 smithi067 ceph-mon[32750]: [31/Jul/2024:14:40:22] ENGINE Serving on https://172.21.15.67:7150 2024-07-31T14:40:24.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:23 smithi067 ceph-mon[32750]: [31/Jul/2024:14:40:22] ENGINE Bus STARTED 2024-07-31T14:40:24.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:23 smithi067 ceph-mon[32750]: [31/Jul/2024:14:40:22] ENGINE Error in HTTPServer.serve 2024-07-31T14:40:24.054 INFO:journalctl@ceph.mon.a.smithi067.stdout: Traceback (most recent call last): 2024-07-31T14:40:24.054 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-31T14:40:24.054 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._connections.run(self.expiration_interval) 2024-07-31T14:40:24.054 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-31T14:40:24.054 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._run(expiration_interval) 2024-07-31T14:40:24.055 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-31T14:40:24.055 INFO:journalctl@ceph.mon.a.smithi067.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-31T14:40:24.055 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-31T14:40:24.055 INFO:journalctl@ceph.mon.a.smithi067.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-31T14:40:24.055 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-31T14:40:24.055 INFO:journalctl@ceph.mon.a.smithi067.stdout: s = self.context.wrap_socket( 2024-07-31T14:40:24.055 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-31T14:40:24.055 INFO:journalctl@ceph.mon.a.smithi067.stdout: return self.sslsocket_class._create( 2024-07-31T14:40:24.055 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-31T14:40:24.055 INFO:journalctl@ceph.mon.a.smithi067.stdout: self.do_handshake() 2024-07-31T14:40:24.055 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-31T14:40:24.055 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._sslobj.do_handshake() 2024-07-31T14:40:24.055 INFO:journalctl@ceph.mon.a.smithi067.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-31T14:40:24.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:23 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/740562880' entity='client.admin' 2024-07-31T14:40:24.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:24.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:24.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:24.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:24.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:23 smithi073 ceph-mon[36714]: [31/Jul/2024:14:40:22] ENGINE Bus STARTING 2024-07-31T14:40:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:23 smithi073 ceph-mon[36714]: [31/Jul/2024:14:40:22] ENGINE Serving on http://172.21.15.67:8765 2024-07-31T14:40:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:23 smithi073 ceph-mon[36714]: Updating smithi067:/etc/ceph/ceph.conf 2024-07-31T14:40:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:23 smithi073 ceph-mon[36714]: Updating smithi073:/etc/ceph/ceph.conf 2024-07-31T14:40:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:23 smithi073 ceph-mon[36714]: Updating smithi077:/etc/ceph/ceph.conf 2024-07-31T14:40:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:23 smithi073 ceph-mon[36714]: [31/Jul/2024:14:40:22] ENGINE Serving on https://172.21.15.67:7150 2024-07-31T14:40:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:23 smithi073 ceph-mon[36714]: [31/Jul/2024:14:40:22] ENGINE Bus STARTED 2024-07-31T14:40:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:23 smithi073 ceph-mon[36714]: [31/Jul/2024:14:40:22] ENGINE Error in HTTPServer.serve 2024-07-31T14:40:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout: Traceback (most recent call last): 2024-07-31T14:40:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-31T14:40:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout: self._connections.run(self.expiration_interval) 2024-07-31T14:40:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-31T14:40:24.144 INFO:journalctl@ceph.mon.b.smithi073.stdout: self._run(expiration_interval) 2024-07-31T14:40:24.144 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-31T14:40:24.144 INFO:journalctl@ceph.mon.b.smithi073.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-31T14:40:24.144 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-31T14:40:24.144 INFO:journalctl@ceph.mon.b.smithi073.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-31T14:40:24.144 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-31T14:40:24.144 INFO:journalctl@ceph.mon.b.smithi073.stdout: s = self.context.wrap_socket( 2024-07-31T14:40:24.144 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-31T14:40:24.144 INFO:journalctl@ceph.mon.b.smithi073.stdout: return self.sslsocket_class._create( 2024-07-31T14:40:24.144 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-31T14:40:24.144 INFO:journalctl@ceph.mon.b.smithi073.stdout: self.do_handshake() 2024-07-31T14:40:24.144 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-31T14:40:24.145 INFO:journalctl@ceph.mon.b.smithi073.stdout: self._sslobj.do_handshake() 2024-07-31T14:40:24.145 INFO:journalctl@ceph.mon.b.smithi073.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-31T14:40:24.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:23 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/740562880' entity='client.admin' 2024-07-31T14:40:24.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:24.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:24.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:24.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:24.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:23 smithi077 ceph-mon[37327]: [31/Jul/2024:14:40:22] ENGINE Bus STARTING 2024-07-31T14:40:24.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:23 smithi077 ceph-mon[37327]: [31/Jul/2024:14:40:22] ENGINE Serving on http://172.21.15.67:8765 2024-07-31T14:40:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:23 smithi077 ceph-mon[37327]: Updating smithi067:/etc/ceph/ceph.conf 2024-07-31T14:40:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:23 smithi077 ceph-mon[37327]: Updating smithi073:/etc/ceph/ceph.conf 2024-07-31T14:40:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:23 smithi077 ceph-mon[37327]: Updating smithi077:/etc/ceph/ceph.conf 2024-07-31T14:40:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:23 smithi077 ceph-mon[37327]: [31/Jul/2024:14:40:22] ENGINE Serving on https://172.21.15.67:7150 2024-07-31T14:40:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:23 smithi077 ceph-mon[37327]: [31/Jul/2024:14:40:22] ENGINE Bus STARTED 2024-07-31T14:40:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:23 smithi077 ceph-mon[37327]: [31/Jul/2024:14:40:22] ENGINE Error in HTTPServer.serve 2024-07-31T14:40:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout: Traceback (most recent call last): 2024-07-31T14:40:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-31T14:40:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout: self._connections.run(self.expiration_interval) 2024-07-31T14:40:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-31T14:40:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout: self._run(expiration_interval) 2024-07-31T14:40:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-31T14:40:24.239 INFO:journalctl@ceph.mon.c.smithi077.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-31T14:40:24.239 INFO:journalctl@ceph.mon.c.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-31T14:40:24.239 INFO:journalctl@ceph.mon.c.smithi077.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-31T14:40:24.239 INFO:journalctl@ceph.mon.c.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-31T14:40:24.239 INFO:journalctl@ceph.mon.c.smithi077.stdout: s = self.context.wrap_socket( 2024-07-31T14:40:24.239 INFO:journalctl@ceph.mon.c.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-31T14:40:24.239 INFO:journalctl@ceph.mon.c.smithi077.stdout: return self.sslsocket_class._create( 2024-07-31T14:40:24.239 INFO:journalctl@ceph.mon.c.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-31T14:40:24.239 INFO:journalctl@ceph.mon.c.smithi077.stdout: self.do_handshake() 2024-07-31T14:40:24.239 INFO:journalctl@ceph.mon.c.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-31T14:40:24.239 INFO:journalctl@ceph.mon.c.smithi077.stdout: self._sslobj.do_handshake() 2024-07-31T14:40:24.239 INFO:journalctl@ceph.mon.c.smithi077.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-31T14:40:24.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:23 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/740562880' entity='client.admin' 2024-07-31T14:40:24.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:24.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:24.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:24.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:24.446 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi067') 2024-07-31T14:40:25.062 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:25.062 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:25.065 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:25.065 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:25.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: Updating smithi077:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: Updating smithi073:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: Updating smithi067:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:40:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:40:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:40:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: Updating smithi077:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-07-31T14:40:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: Updating smithi073:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-07-31T14:40:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: Updating smithi067:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-07-31T14:40:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-31T14:40:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-31T14:40:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-31T14:40:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: Updating smithi077:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: Updating smithi073:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: Updating smithi067:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:40:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:40:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:40:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: Updating smithi077:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-07-31T14:40:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: Updating smithi073:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-07-31T14:40:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: Updating smithi067:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-07-31T14:40:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-31T14:40:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-31T14:40:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-31T14:40:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:25.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: Updating smithi077:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: Updating smithi073:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: Updating smithi067:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.conf 2024-07-31T14:40:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:40:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:40:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-07-31T14:40:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: Updating smithi077:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-07-31T14:40:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: Updating smithi073:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-07-31T14:40:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: Updating smithi067:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-07-31T14:40:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:25.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-31T14:40:25.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-31T14:40:25.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-31T14:40:25.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:25.902 INFO:journalctl@ceph.mgr.z.smithi077.stdout:Jul 31 14:40:25 smithi077 systemd[1]: Starting Ceph mgr.z for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T14:40:26.170 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:25 smithi077 ceph-mon[37327]: Deploying daemon mgr.z on smithi077 2024-07-31T14:40:26.170 INFO:journalctl@ceph.mgr.z.smithi077.stdout:Jul 31 14:40:25 smithi077 podman[40172]: 2024-07-31 14:40:25.900886078 +0000 UTC m=+0.107792429 container create 550da08080994fa621f23a1f1d0d62911121b0c6b38819ddc53a1d00ca9aa235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, CEPH_POINT_RELEASE=) 2024-07-31T14:40:26.170 INFO:journalctl@ceph.mgr.z.smithi077.stdout:Jul 31 14:40:25 smithi077 podman[40172]: 2024-07-31 14:40:25.810283933 +0000 UTC m=+0.017190229 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T14:40:26.170 INFO:journalctl@ceph.mgr.z.smithi077.stdout:Jul 31 14:40:26 smithi077 podman[40172]: 2024-07-31 14:40:26.09493751 +0000 UTC m=+0.301843799 container init 550da08080994fa621f23a1f1d0d62911121b0c6b38819ddc53a1d00ca9aa235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T14:40:26.171 INFO:journalctl@ceph.mgr.z.smithi077.stdout:Jul 31 14:40:26 smithi077 podman[40172]: 2024-07-31 14:40:26.097362626 +0000 UTC m=+0.304268924 container start 550da08080994fa621f23a1f1d0d62911121b0c6b38819ddc53a1d00ca9aa235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-z, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-07-31T14:40:26.171 INFO:journalctl@ceph.mgr.z.smithi077.stdout:Jul 31 14:40:26 smithi077 ceph-mgr[40191]: -- 172.21.15.77:0/2761296855 <== mon.1 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd65d510e0 con 0x55dd6aaf0480 2024-07-31T14:40:26.171 INFO:journalctl@ceph.mgr.z.smithi077.stdout:Jul 31 14:40:26 smithi077 bash[40172]: 550da08080994fa621f23a1f1d0d62911121b0c6b38819ddc53a1d00ca9aa235 2024-07-31T14:40:26.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:25 smithi067 ceph-mon[32750]: Deploying daemon mgr.z on smithi077 2024-07-31T14:40:26.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:25 smithi073 ceph-mon[36714]: Deploying daemon mgr.z on smithi077 2024-07-31T14:40:26.487 INFO:journalctl@ceph.mgr.z.smithi077.stdout:Jul 31 14:40:26 smithi077 systemd[1]: Started Ceph mgr.z for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:40:27.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:27.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:27.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:27.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-31T14:40:27.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-31T14:40:27.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-31T14:40:27.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:27.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:27 smithi067 ceph-mon[32750]: Deploying daemon mgr.y on smithi073 2024-07-31T14:40:27.599 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:27.599 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:27.599 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:27.599 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-31T14:40:27.599 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-31T14:40:27.599 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-31T14:40:27.599 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:27.599 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:27 smithi073 ceph-mon[36714]: Deploying daemon mgr.y on smithi073 2024-07-31T14:40:27.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:27.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:27.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-31T14:40:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-31T14:40:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-31T14:40:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:27 smithi077 ceph-mon[37327]: Deploying daemon mgr.y on smithi073 2024-07-31T14:40:28.145 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Jul 31 14:40:28 smithi073 systemd[1]: Starting Ceph mgr.y for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T14:40:28.495 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Jul 31 14:40:28 smithi073 podman[39072]: 2024-07-31 14:40:28.144832197 +0000 UTC m=+0.016786378 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T14:40:28.495 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Jul 31 14:40:28 smithi073 podman[39072]: 2024-07-31 14:40:28.242119198 +0000 UTC m=+0.114073411 container create 023c4c6614c00fb5383f6463fb9c2011f012825c1633b63c194d87e8005174b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-07-31T14:40:28.495 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Jul 31 14:40:28 smithi073 podman[39072]: 2024-07-31 14:40:28.41949731 +0000 UTC m=+0.291451480 container init 023c4c6614c00fb5383f6463fb9c2011f012825c1633b63c194d87e8005174b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-y, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T14:40:28.495 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Jul 31 14:40:28 smithi073 podman[39072]: 2024-07-31 14:40:28.422120432 +0000 UTC m=+0.294074604 container start 023c4c6614c00fb5383f6463fb9c2011f012825c1633b63c194d87e8005174b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T14:40:28.495 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Jul 31 14:40:28 smithi073 ceph-mgr[39091]: -- 172.21.15.73:0/2177948179 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56275669f0e0 con 0x56275b43e480 2024-07-31T14:40:28.496 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Jul 31 14:40:28 smithi073 bash[39072]: 023c4c6614c00fb5383f6463fb9c2011f012825c1633b63c194d87e8005174b4 2024-07-31T14:40:28.892 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Jul 31 14:40:28 smithi073 systemd[1]: Started Ceph mgr.y for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:40:29.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:29.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:29.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:29.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:29.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:29.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:29.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:29.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:29.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:29.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:29.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:29.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:29.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:29.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:30.222 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:30.222 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:30.222 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:30.326 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:30.326 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:30.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:30 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:30.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:30 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:30.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:30 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:30 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:30 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:30 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:30 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:30 smithi077 ceph-mon[37327]: Reconfiguring mgr.x (unknown last config time)... 2024-07-31T14:40:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:30 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-31T14:40:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:30 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-31T14:40:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:30 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:30 smithi077 ceph-mon[37327]: Reconfiguring daemon mgr.x on smithi067 2024-07-31T14:40:31.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:30 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:31.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:30 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:31.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:30 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:31.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:30 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:31.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:30 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:31.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:30 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:31.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:30 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:31.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:30 smithi067 ceph-mon[32750]: Reconfiguring mgr.x (unknown last config time)... 2024-07-31T14:40:31.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:30 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-31T14:40:31.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:30 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-31T14:40:31.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:30 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:31.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:30 smithi067 ceph-mon[32750]: Reconfiguring daemon mgr.x on smithi067 2024-07-31T14:40:31.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:30 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:30 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:30 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:30 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:30 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:30 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:30 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:30 smithi073 ceph-mon[36714]: Reconfiguring mgr.x (unknown last config time)... 2024-07-31T14:40:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:30 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-31T14:40:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:30 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-31T14:40:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:30 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:30 smithi073 ceph-mon[36714]: Reconfiguring daemon mgr.x on smithi067 2024-07-31T14:40:31.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:31 smithi067 ceph-mon[32750]: Standby manager daemon z started 2024-07-31T14:40:31.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:31 smithi067 ceph-mon[32750]: from='mgr.? 172.21.15.77:0/3240917245' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-31T14:40:31.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:31 smithi067 ceph-mon[32750]: from='mgr.? 172.21.15.77:0/3240917245' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-31T14:40:31.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:31 smithi067 ceph-mon[32750]: from='mgr.? 172.21.15.77:0/3240917245' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-31T14:40:31.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:31 smithi067 ceph-mon[32750]: from='mgr.? 172.21.15.77:0/3240917245' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-31T14:40:32.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:31 smithi073 ceph-mon[36714]: Standby manager daemon z started 2024-07-31T14:40:32.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:31 smithi073 ceph-mon[36714]: from='mgr.? 172.21.15.77:0/3240917245' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-31T14:40:32.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:31 smithi073 ceph-mon[36714]: from='mgr.? 172.21.15.77:0/3240917245' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-31T14:40:32.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:31 smithi073 ceph-mon[36714]: from='mgr.? 172.21.15.77:0/3240917245' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-31T14:40:32.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:31 smithi073 ceph-mon[36714]: from='mgr.? 172.21.15.77:0/3240917245' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-31T14:40:32.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:31 smithi077 ceph-mon[37327]: Standby manager daemon z started 2024-07-31T14:40:32.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:31 smithi077 ceph-mon[37327]: from='mgr.? 172.21.15.77:0/3240917245' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-31T14:40:32.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:31 smithi077 ceph-mon[37327]: from='mgr.? 172.21.15.77:0/3240917245' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-31T14:40:32.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:31 smithi077 ceph-mon[37327]: from='mgr.? 172.21.15.77:0/3240917245' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-31T14:40:32.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:31 smithi077 ceph-mon[37327]: from='mgr.? 172.21.15.77:0/3240917245' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-31T14:40:33.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:32 smithi067 ceph-mon[32750]: mgrmap e18: x(active, since 12s), standbys: z 2024-07-31T14:40:33.019 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:32 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-31T14:40:33.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:32 smithi073 ceph-mon[36714]: mgrmap e18: x(active, since 12s), standbys: z 2024-07-31T14:40:33.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:32 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-31T14:40:33.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:32 smithi077 ceph-mon[37327]: mgrmap e18: x(active, since 12s), standbys: z 2024-07-31T14:40:33.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:32 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-31T14:40:33.879 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:33.885 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:33.887 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:33.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:33 smithi067 ceph-mon[32750]: Standby manager daemon y started 2024-07-31T14:40:33.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:33 smithi067 ceph-mon[32750]: from='mgr.? 172.21.15.73:0/2112584834' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-31T14:40:33.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:33 smithi067 ceph-mon[32750]: from='mgr.? 172.21.15.73:0/2112584834' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-31T14:40:33.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:33 smithi067 ceph-mon[32750]: from='mgr.? 172.21.15.73:0/2112584834' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-31T14:40:33.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:33 smithi067 ceph-mon[32750]: from='mgr.? 172.21.15.73:0/2112584834' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-31T14:40:33.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:33 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/740208222' entity='client.admin' 2024-07-31T14:40:34.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:33 smithi073 ceph-mon[36714]: Standby manager daemon y started 2024-07-31T14:40:34.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:33 smithi073 ceph-mon[36714]: from='mgr.? 172.21.15.73:0/2112584834' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-31T14:40:34.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:33 smithi073 ceph-mon[36714]: from='mgr.? 172.21.15.73:0/2112584834' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-31T14:40:34.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:33 smithi073 ceph-mon[36714]: from='mgr.? 172.21.15.73:0/2112584834' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-31T14:40:34.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:33 smithi073 ceph-mon[36714]: from='mgr.? 172.21.15.73:0/2112584834' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-31T14:40:34.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:33 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/740208222' entity='client.admin' 2024-07-31T14:40:34.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:33 smithi077 ceph-mon[37327]: Standby manager daemon y started 2024-07-31T14:40:34.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:33 smithi077 ceph-mon[37327]: from='mgr.? 172.21.15.73:0/2112584834' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-31T14:40:34.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:33 smithi077 ceph-mon[37327]: from='mgr.? 172.21.15.73:0/2112584834' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-31T14:40:34.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:33 smithi077 ceph-mon[37327]: from='mgr.? 172.21.15.73:0/2112584834' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-31T14:40:34.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:33 smithi077 ceph-mon[37327]: from='mgr.? 172.21.15.73:0/2112584834' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-31T14:40:34.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:33 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/740208222' entity='client.admin' 2024-07-31T14:40:35.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:34 smithi067 ceph-mon[32750]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-07-31T14:40:35.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-31T14:40:35.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:34 smithi073 ceph-mon[36714]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-07-31T14:40:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-31T14:40:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:34 smithi077 ceph-mon[37327]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-07-31T14:40:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-31T14:40:38.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:38 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1742424598' entity='client.admin' 2024-07-31T14:40:38.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:38.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:38.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:38.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:38.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:38.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:38.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:38 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1742424598' entity='client.admin' 2024-07-31T14:40:38.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:38.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:38.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:38.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:38.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:38.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:38.942 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:38.942 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:38.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:38 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1742424598' entity='client.admin' 2024-07-31T14:40:38.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:38.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:38.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:38.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:38.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:38.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:40.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:40 smithi073 ceph-mon[36714]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:40.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:40 smithi077 ceph-mon[37327]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:41.034 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:40 smithi067 ceph-mon[32750]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:41.974 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:41 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2182886243' entity='client.admin' 2024-07-31T14:40:41.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:41 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2182886243' entity='client.admin' 2024-07-31T14:40:42.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:41 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2182886243' entity='client.admin' 2024-07-31T14:40:42.947 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:42 smithi067 ceph-mon[32750]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:42.947 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:42 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1876235531' entity='client.admin' 2024-07-31T14:40:42.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:42 smithi077 ceph-mon[37327]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:42.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:42 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1876235531' entity='client.admin' 2024-07-31T14:40:43.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:42 smithi073 ceph-mon[36714]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:43.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:42 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1876235531' entity='client.admin' 2024-07-31T14:40:44.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:44 smithi077 ceph-mon[37327]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:45.021 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:44 smithi067 ceph-mon[32750]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:45.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:44 smithi073 ceph-mon[36714]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:46.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.101+0000 7fd1302f6640 1 -- 172.21.15.67:0/3473163846 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fd128104f80 con 0x7fd128107dd0 2024-07-31T14:40:46.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.105+0000 7fd1277fe640 1 -- 172.21.15.67:0/3473163846 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v16)=0 v16) ==== 114+0+0 (secure 0 0 0) 0x7fd11c0ab8e0 con 0x7fd128107dd0 2024-07-31T14:40:46.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.106+0000 7fd1277fe640 1 -- 172.21.15.67:0/3473163846 <== mon.0 v2:172.21.15.67:3300/0 9 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd11c0ab1e0 con 0x7fd128107dd0 2024-07-31T14:40:46.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.108+0000 7fd1302f6640 1 -- 172.21.15.67:0/3473163846 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd1100b3b90 msgr2=0x7fd1100b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:46.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.108+0000 7fd1302f6640 1 --2- 172.21.15.67:0/3473163846 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd1100b3b90 0x7fd1100b60a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd1281056d0 tx=0x7fd118005fd0 comp rx=0 tx=0).stop 2024-07-31T14:40:46.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.108+0000 7fd1302f6640 1 -- 172.21.15.67:0/3473163846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd128107dd0 msgr2=0x7fd1281041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:46.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.108+0000 7fd1302f6640 1 --2- 172.21.15.67:0/3473163846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd128107dd0 0x7fd1281041c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00e9c0 tx=0x7fd11c00ee80 comp rx=0 tx=0).stop 2024-07-31T14:40:46.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.108+0000 7fd1302f6640 1 -- 172.21.15.67:0/3473163846 shutdown_connections 2024-07-31T14:40:46.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.108+0000 7fd1302f6640 1 --2- 172.21.15.67:0/3473163846 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd1100b3b90 0x7fd1100b60a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:46.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.108+0000 7fd1302f6640 1 --2- 172.21.15.67:0/3473163846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd128108720 0x7fd128104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:46.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.108+0000 7fd1302f6640 1 --2- 172.21.15.67:0/3473163846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd128107dd0 0x7fd1281041c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:46.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.108+0000 7fd1302f6640 1 --2- 172.21.15.67:0/3473163846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd128102490 0x7fd128103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:46.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.108+0000 7fd1302f6640 1 -- 172.21.15.67:0/3473163846 >> 172.21.15.67:0/3473163846 conn(0x7fd128079160 msgr2=0x7fd128101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:46.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.108+0000 7fd1302f6640 1 -- 172.21.15.67:0/3473163846 shutdown_connections 2024-07-31T14:40:46.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.108+0000 7fd1302f6640 1 -- 172.21.15.67:0/3473163846 wait complete. 2024-07-31T14:40:46.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:46 smithi067 ceph-mon[32750]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:46.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:46 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1720078350' entity='client.admin' 2024-07-31T14:40:46.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:46 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3473163846' entity='client.admin' 2024-07-31T14:40:46.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:46 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2668803450' entity='client.admin' 2024-07-31T14:40:46.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:46 smithi067 ceph-mon[32750]: from='client.? ' entity='client.admin' 2024-07-31T14:40:46.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:46 smithi077 ceph-mon[37327]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:46.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:46 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1720078350' entity='client.admin' 2024-07-31T14:40:46.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:46 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3473163846' entity='client.admin' 2024-07-31T14:40:46.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:46 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2668803450' entity='client.admin' 2024-07-31T14:40:46.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:46 smithi077 ceph-mon[37327]: from='client.? ' entity='client.admin' 2024-07-31T14:40:47.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:46 smithi073 ceph-mon[36714]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:47.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:46 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1720078350' entity='client.admin' 2024-07-31T14:40:47.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:46 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3473163846' entity='client.admin' 2024-07-31T14:40:47.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:46 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2668803450' entity='client.admin' 2024-07-31T14:40:47.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:46 smithi073 ceph-mon[36714]: from='client.? ' entity='client.admin' 2024-07-31T14:40:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.274+0000 7f8949a9b640 1 -- 172.21.15.67:0/753458740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8944102490 msgr2=0x7f89441028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.274+0000 7f8949a9b640 1 --2- 172.21.15.67:0/753458740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8944102490 0x7f89441028a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8934009ca0 tx=0x7f893402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.274+0000 7f8949a9b640 1 -- 172.21.15.67:0/753458740 shutdown_connections 2024-07-31T14:40:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.274+0000 7f8949a9b640 1 --2- 172.21.15.67:0/753458740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8944108720 0x7f894410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.274+0000 7f8949a9b640 1 --2- 172.21.15.67:0/753458740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8944107dd0 0x7f89441081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.274+0000 7f8949a9b640 1 --2- 172.21.15.67:0/753458740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8944102490 0x7f89441028a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.274+0000 7f8949a9b640 1 -- 172.21.15.67:0/753458740 >> 172.21.15.67:0/753458740 conn(0x7f8944079160 msgr2=0x7f8944079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.274+0000 7f8949a9b640 1 -- 172.21.15.67:0/753458740 shutdown_connections 2024-07-31T14:40:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.274+0000 7f8949a9b640 1 -- 172.21.15.67:0/753458740 wait complete. 2024-07-31T14:40:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.275+0000 7f8949a9b640 1 Processor -- start 2024-07-31T14:40:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.275+0000 7f8949a9b640 1 -- start start 2024-07-31T14:40:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.275+0000 7f8949a9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8944102490 0x7f894419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.275+0000 7f8949a9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8944107dd0 0x7f894419b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.275+0000 7f8949a9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8944108720 0x7f894419b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.275+0000 7f8949a9b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8944112240 con 0x7f8944102490 2024-07-31T14:40:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.275+0000 7f8949a9b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f89441120c0 con 0x7f8944108720 2024-07-31T14:40:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.275+0000 7f8949a9b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f89441123c0 con 0x7f8944107dd0 2024-07-31T14:40:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f89437fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8944102490 0x7f894419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f89437fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8944102490 0x7f894419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35280/0 (socket says 172.21.15.67:35280) 2024-07-31T14:40:47.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f89437fe640 1 -- 172.21.15.67:0/2668803450 learned_addr learned my addr 172.21.15.67:0/2668803450 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:40:47.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f8942ffd640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8944107dd0 0x7f894419b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:47.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f8943fff640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8944108720 0x7f894419b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:47.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f89437fe640 1 -- 172.21.15.67:0/2668803450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8944107dd0 msgr2=0x7f894419b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:47.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f89437fe640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8944107dd0 0x7f894419b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f89437fe640 1 -- 172.21.15.67:0/2668803450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8944108720 msgr2=0x7f894419b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:47.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f89437fe640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8944108720 0x7f894419b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f89437fe640 1 -- 172.21.15.67:0/2668803450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89441a5e40 con 0x7f8944102490 2024-07-31T14:40:47.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f8943fff640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8944108720 0x7f894419b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:40:47.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f89437fe640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8944102490 0x7f894419abb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8934005e30 tx=0x7f89340307f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:47.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f8940ff9640 1 -- 172.21.15.67:0/2668803450 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934031830 con 0x7f8944102490 2024-07-31T14:40:47.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f8949a9b640 1 -- 172.21.15.67:0/2668803450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89441a60d0 con 0x7f8944102490 2024-07-31T14:40:47.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f8940ff9640 1 -- 172.21.15.67:0/2668803450 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f893403b890 con 0x7f8944102490 2024-07-31T14:40:47.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.276+0000 7f8940ff9640 1 -- 172.21.15.67:0/2668803450 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893403a770 con 0x7f8944102490 2024-07-31T14:40:47.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.278+0000 7f8949a9b640 1 -- 172.21.15.67:0/2668803450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89441a6660 con 0x7f8944102490 2024-07-31T14:40:47.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.278+0000 7f8949a9b640 1 -- 172.21.15.67:0/2668803450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89441028a0 con 0x7f8944102490 2024-07-31T14:40:47.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.280+0000 7f8940ff9640 1 -- 172.21.15.67:0/2668803450 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f893403b400 con 0x7f8944102490 2024-07-31T14:40:47.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.282+0000 7f8940ff9640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f89300b3bc0 0x7f89300b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:47.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.282+0000 7f8942ffd640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f89300b3bc0 0x7f89300b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:47.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.282+0000 7f8940ff9640 1 -- 172.21.15.67:0/2668803450 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f89341037e0 con 0x7f8944102490 2024-07-31T14:40:47.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.282+0000 7f8942ffd640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f89300b3bc0 0x7f89300b60d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8944100750 tx=0x7f892c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:47.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.284+0000 7f8940ff9640 1 -- 172.21.15.67:0/2668803450 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f89340cca30 con 0x7f8944102490 2024-07-31T14:40:47.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.495+0000 7f8949a9b640 1 -- 172.21.15.67:0/2668803450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f894410a4f0 con 0x7f8944102490 2024-07-31T14:40:47.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.502+0000 7f8940ff9640 1 -- 172.21.15.67:0/2668803450 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f89340098b0 con 0x7f8944102490 2024-07-31T14:40:47.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.504+0000 7f8949a9b640 1 -- 172.21.15.67:0/2668803450 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f89300b3bc0 msgr2=0x7f89300b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:47.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.504+0000 7f8949a9b640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f89300b3bc0 0x7f89300b60d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8944100750 tx=0x7f892c00b390 comp rx=0 tx=0).stop 2024-07-31T14:40:47.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.504+0000 7f8949a9b640 1 -- 172.21.15.67:0/2668803450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8944102490 msgr2=0x7f894419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:47.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.504+0000 7f8949a9b640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8944102490 0x7f894419abb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8934005e30 tx=0x7f89340307f0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.505+0000 7f8949a9b640 1 -- 172.21.15.67:0/2668803450 shutdown_connections 2024-07-31T14:40:47.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.505+0000 7f8949a9b640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f89300b3bc0 0x7f89300b60d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.505+0000 7f8949a9b640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8944108720 0x7f894419b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.505+0000 7f8949a9b640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8944107dd0 0x7f894419b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.505+0000 7f8949a9b640 1 --2- 172.21.15.67:0/2668803450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8944102490 0x7f894419abb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.505+0000 7f8949a9b640 1 -- 172.21.15.67:0/2668803450 >> 172.21.15.67:0/2668803450 conn(0x7f8944079160 msgr2=0x7f8944100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:47.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.505+0000 7f8949a9b640 1 -- 172.21.15.67:0/2668803450 shutdown_connections 2024-07-31T14:40:47.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.505+0000 7f8949a9b640 1 -- 172.21.15.67:0/2668803450 wait complete. 2024-07-31T14:40:47.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:47 smithi067 ceph-mon[32750]: from='client.? ' entity='client.admin' 2024-07-31T14:40:47.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.489+0000 7f61d4025640 1 -- 172.21.15.67:0/1206576467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61cc108720 msgr2=0x7f61cc10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.489+0000 7f61d4025640 1 --2- 172.21.15.67:0/1206576467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61cc108720 0x7f61cc10ef50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009ca0 tx=0x7f61c402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.489+0000 7f61d4025640 1 -- 172.21.15.67:0/1206576467 shutdown_connections 2024-07-31T14:40:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.489+0000 7f61d4025640 1 --2- 172.21.15.67:0/1206576467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61cc108720 0x7f61cc10ef50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.489+0000 7f61d4025640 1 --2- 172.21.15.67:0/1206576467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc107dd0 0x7f61cc1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.489+0000 7f61d4025640 1 --2- 172.21.15.67:0/1206576467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61cc102490 0x7f61cc1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.489+0000 7f61d4025640 1 -- 172.21.15.67:0/1206576467 >> 172.21.15.67:0/1206576467 conn(0x7f61cc079160 msgr2=0x7f61cc079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.489+0000 7f61d4025640 1 -- 172.21.15.67:0/1206576467 shutdown_connections 2024-07-31T14:40:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.489+0000 7f61d4025640 1 -- 172.21.15.67:0/1206576467 wait complete. 2024-07-31T14:40:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.490+0000 7f61d4025640 1 Processor -- start 2024-07-31T14:40:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.490+0000 7f61d4025640 1 -- start start 2024-07-31T14:40:47.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.490+0000 7f61d4025640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61cc102490 0x7f61cc199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.490+0000 7f61d4025640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc107dd0 0x7f61cc19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.490+0000 7f61d4025640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61cc108720 0x7f61cc19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.490+0000 7f61d4025640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc113390 con 0x7f61cc108720 2024-07-31T14:40:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.490+0000 7f61d4025640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc113210 con 0x7f61cc102490 2024-07-31T14:40:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.490+0000 7f61d4025640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc113510 con 0x7f61cc107dd0 2024-07-31T14:40:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.491+0000 7f61d25c2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61cc108720 0x7f61cc19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.491+0000 7f61d1dc1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61cc102490 0x7f61cc199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.491+0000 7f61d25c2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61cc108720 0x7f61cc19f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35318/0 (socket says 172.21.15.67:35318) 2024-07-31T14:40:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.491+0000 7f61d25c2640 1 -- 172.21.15.67:0/1742130276 learned_addr learned my addr 172.21.15.67:0/1742130276 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:40:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.491+0000 7f61d15c0640 1 --2- 172.21.15.67:0/1742130276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc107dd0 0x7f61cc19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.491+0000 7f61d1dc1640 1 -- 172.21.15.67:0/1742130276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc107dd0 msgr2=0x7f61cc19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.491+0000 7f61d1dc1640 1 --2- 172.21.15.67:0/1742130276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc107dd0 0x7f61cc19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.491+0000 7f61d1dc1640 1 -- 172.21.15.67:0/1742130276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61cc108720 msgr2=0x7f61cc19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.491+0000 7f61d1dc1640 1 --2- 172.21.15.67:0/1742130276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61cc108720 0x7f61cc19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.491+0000 7f61d1dc1640 1 -- 172.21.15.67:0/1742130276 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc19fd30 con 0x7f61cc102490 2024-07-31T14:40:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.491+0000 7f61d25c2640 1 --2- 172.21.15.67:0/1742130276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61cc108720 0x7f61cc19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:40:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.491+0000 7f61d1dc1640 1 --2- 172.21.15.67:0/1742130276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61cc102490 0x7f61cc199dc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61c800e9f0 tx=0x7f61c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.492+0000 7f61c2ffd640 1 -- 172.21.15.67:0/1742130276 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c800cd80 con 0x7f61cc102490 2024-07-31T14:40:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.492+0000 7f61c2ffd640 1 -- 172.21.15.67:0/1742130276 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f61c80035f0 con 0x7f61cc102490 2024-07-31T14:40:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.492+0000 7f61c2ffd640 1 -- 172.21.15.67:0/1742130276 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8010670 con 0x7f61cc102490 2024-07-31T14:40:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.492+0000 7f61d4025640 1 -- 172.21.15.67:0/1742130276 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61cc1a0020 con 0x7f61cc102490 2024-07-31T14:40:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.492+0000 7f61d4025640 1 -- 172.21.15.67:0/1742130276 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61cc10f8c0 con 0x7f61cc102490 2024-07-31T14:40:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.493+0000 7f61d4025640 1 -- 172.21.15.67:0/1742130276 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61cc1028a0 con 0x7f61cc102490 2024-07-31T14:40:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.495+0000 7f61c2ffd640 1 -- 172.21.15.67:0/1742130276 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f61c8004000 con 0x7f61cc102490 2024-07-31T14:40:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.495+0000 7f61c2ffd640 1 --2- 172.21.15.67:0/1742130276 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f61b80b3c10 0x7f61b80b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.496+0000 7f61d15c0640 1 --2- 172.21.15.67:0/1742130276 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f61b80b3c10 0x7f61b80b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.496+0000 7f61c2ffd640 1 -- 172.21.15.67:0/1742130276 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f61c80e2300 con 0x7f61cc102490 2024-07-31T14:40:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.496+0000 7f61d15c0640 1 --2- 172.21.15.67:0/1742130276 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f61b80b3c10 0x7f61b80b6120 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f61cc107430 tx=0x7f61bc00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.499+0000 7f61c2ffd640 1 -- 172.21.15.67:0/1742130276 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f61c80ab5c0 con 0x7f61cc102490 2024-07-31T14:40:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.716+0000 7f61d4025640 1 -- 172.21.15.67:0/1742130276 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f61cc19aef0 con 0x7f61cc102490 2024-07-31T14:40:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.723+0000 7f61c2ffd640 1 -- 172.21.15.67:0/1742130276 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v19)=0 v19) ==== 118+0+0 (secure 0 0 0) 0x7f61cc19aef0 con 0x7f61cc102490 2024-07-31T14:40:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.726+0000 7f61d4025640 1 -- 172.21.15.67:0/1742130276 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f61b80b3c10 msgr2=0x7f61b80b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.726+0000 7f61d4025640 1 --2- 172.21.15.67:0/1742130276 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f61b80b3c10 0x7f61b80b6120 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f61cc107430 tx=0x7f61bc00b390 comp rx=0 tx=0).stop 2024-07-31T14:40:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.726+0000 7f61d4025640 1 -- 172.21.15.67:0/1742130276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61cc102490 msgr2=0x7f61cc199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.726+0000 7f61d4025640 1 --2- 172.21.15.67:0/1742130276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61cc102490 0x7f61cc199dc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61c800e9f0 tx=0x7f61c800eeb0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.726+0000 7f61d4025640 1 -- 172.21.15.67:0/1742130276 shutdown_connections 2024-07-31T14:40:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.726+0000 7f61d4025640 1 --2- 172.21.15.67:0/1742130276 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f61b80b3c10 0x7f61b80b6120 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.726+0000 7f61d4025640 1 --2- 172.21.15.67:0/1742130276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61cc108720 0x7f61cc19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.726+0000 7f61d4025640 1 --2- 172.21.15.67:0/1742130276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61cc107dd0 0x7f61cc19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.726+0000 7f61d4025640 1 --2- 172.21.15.67:0/1742130276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61cc102490 0x7f61cc199dc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:47.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.726+0000 7f61d4025640 1 -- 172.21.15.67:0/1742130276 >> 172.21.15.67:0/1742130276 conn(0x7f61cc079160 msgr2=0x7f61cc104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:47.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.726+0000 7f61d4025640 1 -- 172.21.15.67:0/1742130276 shutdown_connections 2024-07-31T14:40:47.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.726+0000 7f61d4025640 1 -- 172.21.15.67:0/1742130276 wait complete. 2024-07-31T14:40:47.978 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:47 smithi077 ceph-mon[37327]: from='client.? ' entity='client.admin' 2024-07-31T14:40:47.996 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:48.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:47 smithi073 ceph-mon[36714]: from='client.? ' entity='client.admin' 2024-07-31T14:40:48.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.031+0000 7f1f0aaa8640 1 -- 172.21.15.67:0/1720078350 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ef40b3bc0 msgr2=0x7f1ef40b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:48.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.031+0000 7f1f0aaa8640 1 --2- 172.21.15.67:0/1720078350 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ef40b3bc0 0x7f1ef40b60d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009230 tx=0x7f1ef8000bc0 comp rx=0 tx=0).stop 2024-07-31T14:40:48.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.031+0000 7f1f0aaa8640 1 -- 172.21.15.67:0/1720078350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f04108f00 msgr2=0x7f1f0419a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:48.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.031+0000 7f1f0aaa8640 1 --2- 172.21.15.67:0/1720078350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f04108f00 0x7f1f0419a210 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000c8c0 tx=0x7f1ef000cd80 comp rx=0 tx=0).stop 2024-07-31T14:40:48.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.031+0000 7f1f0aaa8640 1 -- 172.21.15.67:0/1720078350 shutdown_connections 2024-07-31T14:40:48.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.031+0000 7f1f0aaa8640 1 --2- 172.21.15.67:0/1720078350 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ef40b3bc0 0x7f1ef40b60d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:48.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.031+0000 7f1f0aaa8640 1 --2- 172.21.15.67:0/1720078350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f04109850 0x7f1f0419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:48.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.031+0000 7f1f0aaa8640 1 --2- 172.21.15.67:0/1720078350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f04108f00 0x7f1f0419a210 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:48.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.031+0000 7f1f0aaa8640 1 --2- 172.21.15.67:0/1720078350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f041035c0 0x7f1f04199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:48.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.031+0000 7f1f0aaa8640 1 -- 172.21.15.67:0/1720078350 >> 172.21.15.67:0/1720078350 conn(0x7f1f040ff010 msgr2=0x7f1f04101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:48.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.031+0000 7f1f0aaa8640 1 -- 172.21.15.67:0/1720078350 shutdown_connections 2024-07-31T14:40:48.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.031+0000 7f1f0aaa8640 1 -- 172.21.15.67:0/1720078350 wait complete. 2024-07-31T14:40:48.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:48 smithi077 ceph-mon[37327]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:49.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.405+0000 7fd97dd95640 1 -- 172.21.15.67:0/3711459912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd978108720 msgr2=0x7fd97810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:49.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.405+0000 7fd97dd95640 1 --2- 172.21.15.67:0/3711459912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd978108720 0x7fd97810ef50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd970009ca0 tx=0x7fd97002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:40:49.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.405+0000 7fd97dd95640 1 -- 172.21.15.67:0/3711459912 shutdown_connections 2024-07-31T14:40:49.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.405+0000 7fd97dd95640 1 --2- 172.21.15.67:0/3711459912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd978108720 0x7fd97810ef50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:49.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.405+0000 7fd97dd95640 1 --2- 172.21.15.67:0/3711459912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd978107dd0 0x7fd9781081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.405+0000 7fd97dd95640 1 --2- 172.21.15.67:0/3711459912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd978102490 0x7fd9781028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.405+0000 7fd97dd95640 1 -- 172.21.15.67:0/3711459912 >> 172.21.15.67:0/3711459912 conn(0x7fd978079160 msgr2=0x7fd978079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.405+0000 7fd97dd95640 1 -- 172.21.15.67:0/3711459912 shutdown_connections 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.405+0000 7fd97dd95640 1 -- 172.21.15.67:0/3711459912 wait complete. 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.406+0000 7fd97dd95640 1 Processor -- start 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.406+0000 7fd97dd95640 1 -- start start 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.407+0000 7fd97dd95640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd978102490 0x7fd978199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.407+0000 7fd97dd95640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd978107dd0 0x7fd97819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.407+0000 7fd97dd95640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd978108720 0x7fd97819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.407+0000 7fd97dd95640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd978113390 con 0x7fd978107dd0 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.407+0000 7fd97dd95640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd978113210 con 0x7fd978102490 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.407+0000 7fd97dd95640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd978113510 con 0x7fd978108720 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.407+0000 7fd976ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd978107dd0 0x7fd97819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.407+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd978102490 0x7fd978199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.407+0000 7fd976ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd978107dd0 0x7fd97819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35302/0 (socket says 172.21.15.67:35302) 2024-07-31T14:40:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.407+0000 7fd976ffd640 1 -- 172.21.15.67:0/2763016545 learned_addr learned my addr 172.21.15.67:0/2763016545 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:40:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.407+0000 7fd977fff640 1 --2- 172.21.15.67:0/2763016545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd978108720 0x7fd97819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.408+0000 7fd977fff640 1 -- 172.21.15.67:0/2763016545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd978102490 msgr2=0x7fd978199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.408+0000 7fd977fff640 1 --2- 172.21.15.67:0/2763016545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd978102490 0x7fd978199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.408+0000 7fd977fff640 1 -- 172.21.15.67:0/2763016545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd978107dd0 msgr2=0x7fd97819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.408+0000 7fd977fff640 1 --2- 172.21.15.67:0/2763016545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd978107dd0 0x7fd97819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.408+0000 7fd977fff640 1 -- 172.21.15.67:0/2763016545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97819fcd0 con 0x7fd978108720 2024-07-31T14:40:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.408+0000 7fd976ffd640 1 --2- 172.21.15.67:0/2763016545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd978107dd0 0x7fd97819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:40:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.408+0000 7fd977fff640 1 --2- 172.21.15.67:0/2763016545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd978108720 0x7fd97819f5a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd970031040 tx=0x7fd970003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.408+0000 7fd974ff9640 1 -- 172.21.15.67:0/2763016545 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970003ce0 con 0x7fd978108720 2024-07-31T14:40:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.408+0000 7fd974ff9640 1 -- 172.21.15.67:0/2763016545 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd9700044d0 con 0x7fd978108720 2024-07-31T14:40:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.408+0000 7fd97dd95640 1 -- 172.21.15.67:0/2763016545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97819ff60 con 0x7fd978108720 2024-07-31T14:40:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.408+0000 7fd974ff9640 1 -- 172.21.15.67:0/2763016545 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970030b70 con 0x7fd978108720 2024-07-31T14:40:49.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.408+0000 7fd97dd95640 1 -- 172.21.15.67:0/2763016545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9781a6720 con 0x7fd978108720 2024-07-31T14:40:49.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.409+0000 7fd97dd95640 1 -- 172.21.15.67:0/2763016545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9781028a0 con 0x7fd978108720 2024-07-31T14:40:49.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.412+0000 7fd974ff9640 1 -- 172.21.15.67:0/2763016545 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fd970030d10 con 0x7fd978108720 2024-07-31T14:40:49.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.412+0000 7fd974ff9640 1 --2- 172.21.15.67:0/2763016545 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9600b3c10 0x7fd9600b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:49.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.413+0000 7fd974ff9640 1 -- 172.21.15.67:0/2763016545 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd970103430 con 0x7fd978108720 2024-07-31T14:40:49.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.413+0000 7fd9777fe640 1 --2- 172.21.15.67:0/2763016545 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9600b3c10 0x7fd9600b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:49.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.413+0000 7fd9777fe640 1 --2- 172.21.15.67:0/2763016545 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9600b3c10 0x7fd9600b6120 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd96800bbe0 tx=0x7fd96800b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:49.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.415+0000 7fd974ff9640 1 -- 172.21.15.67:0/2763016545 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd9700cc680 con 0x7fd978108720 2024-07-31T14:40:49.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.624+0000 7fd97dd95640 1 -- 172.21.15.67:0/2763016545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fd97810a510 con 0x7fd978108720 2024-07-31T14:40:49.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.630+0000 7fd974ff9640 1 -- 172.21.15.67:0/2763016545 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v18)=0 v18) ==== 117+0+0 (secure 0 0 0) 0x7fd9700cbf80 con 0x7fd978108720 2024-07-31T14:40:49.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.633+0000 7fd97dd95640 1 -- 172.21.15.67:0/2763016545 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9600b3c10 msgr2=0x7fd9600b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:49.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.633+0000 7fd97dd95640 1 --2- 172.21.15.67:0/2763016545 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9600b3c10 0x7fd9600b6120 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd96800bbe0 tx=0x7fd96800b390 comp rx=0 tx=0).stop 2024-07-31T14:40:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.633+0000 7fd97dd95640 1 -- 172.21.15.67:0/2763016545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd978108720 msgr2=0x7fd97819f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.633+0000 7fd97dd95640 1 --2- 172.21.15.67:0/2763016545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd978108720 0x7fd97819f5a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd970031040 tx=0x7fd970003970 comp rx=0 tx=0).stop 2024-07-31T14:40:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.633+0000 7fd97dd95640 1 -- 172.21.15.67:0/2763016545 shutdown_connections 2024-07-31T14:40:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.633+0000 7fd97dd95640 1 --2- 172.21.15.67:0/2763016545 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9600b3c10 0x7fd9600b6120 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.633+0000 7fd97dd95640 1 --2- 172.21.15.67:0/2763016545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd978108720 0x7fd97819f5a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.633+0000 7fd97dd95640 1 --2- 172.21.15.67:0/2763016545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd978107dd0 0x7fd97819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.633+0000 7fd97dd95640 1 --2- 172.21.15.67:0/2763016545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd978102490 0x7fd978199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.633+0000 7fd97dd95640 1 -- 172.21.15.67:0/2763016545 >> 172.21.15.67:0/2763016545 conn(0x7fd978079160 msgr2=0x7fd978103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.633+0000 7fd97dd95640 1 -- 172.21.15.67:0/2763016545 shutdown_connections 2024-07-31T14:40:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:46.633+0000 7fd97dd95640 1 -- 172.21.15.67:0/2763016545 wait complete. 2024-07-31T14:40:49.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:48 smithi067 ceph-mon[32750]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:49.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:48 smithi073 ceph-mon[36714]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:50.820 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:50 smithi067 ceph-mon[32750]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:50.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:50 smithi077 ceph-mon[37327]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:51.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:50 smithi073 ceph-mon[36714]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:51.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.759+0000 7f4397489640 1 -- 172.21.15.67:0/1503300813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4390108720 msgr2=0x7f439010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:51.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.759+0000 7f4397489640 1 --2- 172.21.15.67:0/1503300813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4390108720 0x7f439010ef50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4388009ca0 tx=0x7f438802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:40:51.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.759+0000 7f4397489640 1 -- 172.21.15.67:0/1503300813 shutdown_connections 2024-07-31T14:40:51.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.759+0000 7f4397489640 1 --2- 172.21.15.67:0/1503300813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4390108720 0x7f439010ef50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:51.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.759+0000 7f4397489640 1 --2- 172.21.15.67:0/1503300813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4390107dd0 0x7f43901081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:51.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.759+0000 7f4397489640 1 --2- 172.21.15.67:0/1503300813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4390102490 0x7f43901028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:51.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.759+0000 7f4397489640 1 -- 172.21.15.67:0/1503300813 >> 172.21.15.67:0/1503300813 conn(0x7f4390079160 msgr2=0x7f4390079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:51.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.760+0000 7f4397489640 1 -- 172.21.15.67:0/1503300813 shutdown_connections 2024-07-31T14:40:51.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.760+0000 7f4397489640 1 -- 172.21.15.67:0/1503300813 wait complete. 2024-07-31T14:40:51.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.761+0000 7f4397489640 1 Processor -- start 2024-07-31T14:40:51.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.761+0000 7f4397489640 1 -- start start 2024-07-31T14:40:51.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.761+0000 7f4397489640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4390102490 0x7f4390199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:51.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.761+0000 7f4397489640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4390107dd0 0x7f439019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:51.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.761+0000 7f4397489640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4390108720 0x7f439019f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:51.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.761+0000 7f4397489640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4390113380 con 0x7f4390107dd0 2024-07-31T14:40:51.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.761+0000 7f4397489640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4390113200 con 0x7f4390102490 2024-07-31T14:40:51.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.761+0000 7f4397489640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4390113500 con 0x7f4390108720 2024-07-31T14:40:51.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.761+0000 7f4394a24640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4390107dd0 0x7f439019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:51.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.761+0000 7f4394a24640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4390107dd0 0x7f439019a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39294/0 (socket says 172.21.15.67:39294) 2024-07-31T14:40:51.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.761+0000 7f4394a24640 1 -- 172.21.15.67:0/3313945868 learned_addr learned my addr 172.21.15.67:0/3313945868 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:40:51.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.761+0000 7f4395a26640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4390108720 0x7f439019f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:51.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.761+0000 7f4395225640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4390102490 0x7f4390199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:51.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.762+0000 7f4394a24640 1 -- 172.21.15.67:0/3313945868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4390108720 msgr2=0x7f439019f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:51.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.762+0000 7f4394a24640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4390108720 0x7f439019f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:51.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.762+0000 7f4394a24640 1 -- 172.21.15.67:0/3313945868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4390102490 msgr2=0x7f4390199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:51.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.762+0000 7f4394a24640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4390102490 0x7f4390199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:51.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.762+0000 7f4394a24640 1 -- 172.21.15.67:0/3313945868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439019fd00 con 0x7f4390107dd0 2024-07-31T14:40:51.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.762+0000 7f4395225640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4390102490 0x7f4390199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:40:51.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.762+0000 7f4395a26640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4390108720 0x7f439019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:40:51.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.762+0000 7f4394a24640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4390107dd0 0x7f439019a330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f438000d740 tx=0x7f438000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:51.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.762+0000 7f43867fc640 1 -- 172.21.15.67:0/3313945868 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380017070 con 0x7f4390107dd0 2024-07-31T14:40:51.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.762+0000 7f4397489640 1 -- 172.21.15.67:0/3313945868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439019fff0 con 0x7f4390107dd0 2024-07-31T14:40:51.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.762+0000 7f43867fc640 1 -- 172.21.15.67:0/3313945868 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f438000de50 con 0x7f4390107dd0 2024-07-31T14:40:51.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.762+0000 7f43867fc640 1 -- 172.21.15.67:0/3313945868 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380004c70 con 0x7f4390107dd0 2024-07-31T14:40:51.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.762+0000 7f4397489640 1 -- 172.21.15.67:0/3313945868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43901a6750 con 0x7f4390107dd0 2024-07-31T14:40:51.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.763+0000 7f4397489640 1 -- 172.21.15.67:0/3313945868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4374005080 con 0x7f4390107dd0 2024-07-31T14:40:51.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.765+0000 7f43867fc640 1 -- 172.21.15.67:0/3313945868 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f4380009c30 con 0x7f4390107dd0 2024-07-31T14:40:51.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.765+0000 7f43867fc640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f437c0b3b90 0x7f437c0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:51.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.765+0000 7f4395225640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f437c0b3b90 0x7f437c0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:51.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.766+0000 7f43867fc640 1 -- 172.21.15.67:0/3313945868 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f43800e1c20 con 0x7f4390107dd0 2024-07-31T14:40:51.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.766+0000 7f4395225640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f437c0b3b90 0x7f437c0b60a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f438c009c20 tx=0x7f438c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:51.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.768+0000 7f43867fc640 1 -- 172.21.15.67:0/3313945868 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f43800aae70 con 0x7f4390107dd0 2024-07-31T14:40:51.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.972+0000 7f4397489640 1 -- 172.21.15.67:0/3313945868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f4374005370 con 0x7f4390107dd0 2024-07-31T14:40:51.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.979+0000 7f43867fc640 1 -- 172.21.15.67:0/3313945868 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v20)=0 v20) ==== 129+0+0 (secure 0 0 0) 0x7f43800aa770 con 0x7f4390107dd0 2024-07-31T14:40:51.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.982+0000 7f4397489640 1 -- 172.21.15.67:0/3313945868 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f437c0b3b90 msgr2=0x7f437c0b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:51.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.982+0000 7f4397489640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f437c0b3b90 0x7f437c0b60a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f438c009c20 tx=0x7f438c005f90 comp rx=0 tx=0).stop 2024-07-31T14:40:51.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.982+0000 7f4397489640 1 -- 172.21.15.67:0/3313945868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4390107dd0 msgr2=0x7f439019a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:51.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.982+0000 7f4397489640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4390107dd0 0x7f439019a330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f438000d740 tx=0x7f438000dc00 comp rx=0 tx=0).stop 2024-07-31T14:40:51.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.983+0000 7f4397489640 1 -- 172.21.15.67:0/3313945868 shutdown_connections 2024-07-31T14:40:51.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.983+0000 7f4397489640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f437c0b3b90 0x7f437c0b60a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:51.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.983+0000 7f4397489640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4390108720 0x7f439019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:51.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.983+0000 7f4397489640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4390107dd0 0x7f439019a330 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:51.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.983+0000 7f4397489640 1 --2- 172.21.15.67:0/3313945868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4390102490 0x7f4390199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:51.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.983+0000 7f4397489640 1 -- 172.21.15.67:0/3313945868 >> 172.21.15.67:0/3313945868 conn(0x7f4390079160 msgr2=0x7f4390104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:51.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.983+0000 7f4397489640 1 -- 172.21.15.67:0/3313945868 shutdown_connections 2024-07-31T14:40:51.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:51.983+0000 7f4397489640 1 -- 172.21.15.67:0/3313945868 wait complete. 2024-07-31T14:40:52.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.167+0000 7fb540d02640 1 -- 172.21.15.67:0/2001082258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53c108740 msgr2=0x7fb53c10ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:52.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.167+0000 7fb540d02640 1 --2- 172.21.15.67:0/2001082258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53c108740 0x7fb53c10ef70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb534009ca0 tx=0x7fb53402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:40:52.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.167+0000 7fb540d02640 1 -- 172.21.15.67:0/2001082258 shutdown_connections 2024-07-31T14:40:52.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.167+0000 7fb540d02640 1 --2- 172.21.15.67:0/2001082258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53c108740 0x7fb53c10ef70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:52.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.167+0000 7fb540d02640 1 --2- 172.21.15.67:0/2001082258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb53c107df0 0x7fb53c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:52.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.167+0000 7fb540d02640 1 --2- 172.21.15.67:0/2001082258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53c1024b0 0x7fb53c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:52.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.167+0000 7fb540d02640 1 -- 172.21.15.67:0/2001082258 >> 172.21.15.67:0/2001082258 conn(0x7fb53c079170 msgr2=0x7fb53c079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:52.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.167+0000 7fb540d02640 1 -- 172.21.15.67:0/2001082258 shutdown_connections 2024-07-31T14:40:52.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.167+0000 7fb540d02640 1 -- 172.21.15.67:0/2001082258 wait complete. 2024-07-31T14:40:52.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb540d02640 1 Processor -- start 2024-07-31T14:40:52.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb540d02640 1 -- start start 2024-07-31T14:40:52.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb540d02640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53c1024b0 0x7fb53c19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:52.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb540d02640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb53c107df0 0x7fb53c19b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:52.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb540d02640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53c108740 0x7fb53c19b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:52.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb540d02640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c112280 con 0x7fb53c108740 2024-07-31T14:40:52.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb540d02640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c112100 con 0x7fb53c1024b0 2024-07-31T14:40:52.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb540d02640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c112400 con 0x7fb53c107df0 2024-07-31T14:40:52.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb53ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53c108740 0x7fb53c19b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:52.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb53ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53c108740 0x7fb53c19b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39322/0 (socket says 172.21.15.67:39322) 2024-07-31T14:40:52.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb53ad9d640 1 -- 172.21.15.67:0/1682862596 learned_addr learned my addr 172.21.15.67:0/1682862596 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:40:52.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb539d9b640 1 --2- 172.21.15.67:0/1682862596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb53c107df0 0x7fb53c19b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:52.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb53a59c640 1 --2- 172.21.15.67:0/1682862596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53c1024b0 0x7fb53c19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:52.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb53ad9d640 1 -- 172.21.15.67:0/1682862596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb53c107df0 msgr2=0x7fb53c19b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:52.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb53ad9d640 1 --2- 172.21.15.67:0/1682862596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb53c107df0 0x7fb53c19b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:52.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb53ad9d640 1 -- 172.21.15.67:0/1682862596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53c1024b0 msgr2=0x7fb53c19acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:52.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.168+0000 7fb53ad9d640 1 --2- 172.21.15.67:0/1682862596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53c1024b0 0x7fb53c19acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:52.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.169+0000 7fb53ad9d640 1 -- 172.21.15.67:0/1682862596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53c1a5f40 con 0x7fb53c108740 2024-07-31T14:40:52.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.169+0000 7fb53ad9d640 1 --2- 172.21.15.67:0/1682862596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53c108740 0x7fb53c19b730 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb534005e30 tx=0x7fb5340305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:52.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.169+0000 7fb5337fe640 1 -- 172.21.15.67:0/1682862596 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb534003c30 con 0x7fb53c108740 2024-07-31T14:40:52.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.169+0000 7fb540d02640 1 -- 172.21.15.67:0/1682862596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb53c1a61d0 con 0x7fb53c108740 2024-07-31T14:40:52.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.169+0000 7fb5337fe640 1 -- 172.21.15.67:0/1682862596 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb534003dd0 con 0x7fb53c108740 2024-07-31T14:40:52.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.169+0000 7fb5337fe640 1 -- 172.21.15.67:0/1682862596 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb534004710 con 0x7fb53c108740 2024-07-31T14:40:52.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.169+0000 7fb540d02640 1 -- 172.21.15.67:0/1682862596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb53c1a6710 con 0x7fb53c108740 2024-07-31T14:40:52.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.170+0000 7fb540d02640 1 -- 172.21.15.67:0/1682862596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb53c103730 con 0x7fb53c108740 2024-07-31T14:40:52.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.172+0000 7fb5337fe640 1 -- 172.21.15.67:0/1682862596 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fb534004b40 con 0x7fb53c108740 2024-07-31T14:40:52.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.173+0000 7fb5337fe640 1 --2- 172.21.15.67:0/1682862596 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5280b3bc0 0x7fb5280b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:52.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.173+0000 7fb53a59c640 1 --2- 172.21.15.67:0/1682862596 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5280b3bc0 0x7fb5280b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:52.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.173+0000 7fb5337fe640 1 -- 172.21.15.67:0/1682862596 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb5341033c0 con 0x7fb53c108740 2024-07-31T14:40:52.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.173+0000 7fb53a59c640 1 --2- 172.21.15.67:0/1682862596 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5280b3bc0 0x7fb5280b60d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb53c06c0d0 tx=0x7fb52c00b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:52.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.176+0000 7fb5337fe640 1 -- 172.21.15.67:0/1682862596 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb5340cc610 con 0x7fb53c108740 2024-07-31T14:40:52.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.372+0000 7fb540d02640 1 -- 172.21.15.67:0/1682862596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fb53c1028c0 con 0x7fb53c108740 2024-07-31T14:40:52.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.378+0000 7fb5337fe640 1 -- 172.21.15.67:0/1682862596 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v21)=0 v21) ==== 123+0+0 (secure 0 0 0) 0x7fb53c1028c0 con 0x7fb53c108740 2024-07-31T14:40:52.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.380+0000 7fb5337fe640 1 -- 172.21.15.67:0/1682862596 <== mon.0 v2:172.21.15.67:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb534036020 con 0x7fb53c108740 2024-07-31T14:40:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.381+0000 7fb540d02640 1 -- 172.21.15.67:0/1682862596 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5280b3bc0 msgr2=0x7fb5280b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.381+0000 7fb540d02640 1 --2- 172.21.15.67:0/1682862596 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5280b3bc0 0x7fb5280b60d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb53c06c0d0 tx=0x7fb52c00b410 comp rx=0 tx=0).stop 2024-07-31T14:40:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.381+0000 7fb540d02640 1 -- 172.21.15.67:0/1682862596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53c108740 msgr2=0x7fb53c19b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.381+0000 7fb540d02640 1 --2- 172.21.15.67:0/1682862596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53c108740 0x7fb53c19b730 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb534005e30 tx=0x7fb5340305c0 comp rx=0 tx=0).stop 2024-07-31T14:40:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.381+0000 7fb540d02640 1 -- 172.21.15.67:0/1682862596 shutdown_connections 2024-07-31T14:40:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.381+0000 7fb540d02640 1 --2- 172.21.15.67:0/1682862596 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5280b3bc0 0x7fb5280b60d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.381+0000 7fb540d02640 1 --2- 172.21.15.67:0/1682862596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53c108740 0x7fb53c19b730 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.381+0000 7fb540d02640 1 --2- 172.21.15.67:0/1682862596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb53c107df0 0x7fb53c19b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.381+0000 7fb540d02640 1 --2- 172.21.15.67:0/1682862596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53c1024b0 0x7fb53c19acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.381+0000 7fb540d02640 1 -- 172.21.15.67:0/1682862596 >> 172.21.15.67:0/1682862596 conn(0x7fb53c079170 msgr2=0x7fb53c104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.381+0000 7fb540d02640 1 -- 172.21.15.67:0/1682862596 shutdown_connections 2024-07-31T14:40:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:52.381+0000 7fb540d02640 1 -- 172.21.15.67:0/1682862596 wait complete. 2024-07-31T14:40:52.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:52 smithi067 ceph-mon[32750]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:52.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:52 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3313945868' entity='client.admin' 2024-07-31T14:40:52.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:52.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:52.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:52.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:52.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:52 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1682862596' entity='client.admin' 2024-07-31T14:40:52.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:52 smithi077 ceph-mon[37327]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:52.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:52 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3313945868' entity='client.admin' 2024-07-31T14:40:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:52 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1682862596' entity='client.admin' 2024-07-31T14:40:53.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:52 smithi073 ceph-mon[36714]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:52 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3313945868' entity='client.admin' 2024-07-31T14:40:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:52 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1682862596' entity='client.admin' 2024-07-31T14:40:54.682 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi067') 2024-07-31T14:40:54.683 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-07-31T14:40:54.683 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-07-31T14:40:54.683 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi067') 2024-07-31T14:40:54.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.855+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/154284272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e18108990 msgr2=0x7f4e1810f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:54.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.855+0000 7f4e1d7dc640 1 --2- 172.21.15.67:0/154284272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e18108990 0x7f4e1810f1c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009ca0 tx=0x7f4e1002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:40:54.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.856+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/154284272 shutdown_connections 2024-07-31T14:40:54.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.856+0000 7f4e1d7dc640 1 --2- 172.21.15.67:0/154284272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e18108990 0x7f4e1810f1c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:54.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.856+0000 7f4e1d7dc640 1 --2- 172.21.15.67:0/154284272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e18108040 0x7f4e18108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:54.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.856+0000 7f4e1d7dc640 1 --2- 172.21.15.67:0/154284272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e18102c00 0x7f4e18103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:54.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.856+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/154284272 >> 172.21.15.67:0/154284272 conn(0x7f4e18079870 msgr2=0x7f4e18079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:54.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.856+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/154284272 shutdown_connections 2024-07-31T14:40:54.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.856+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/154284272 wait complete. 2024-07-31T14:40:54.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.856+0000 7f4e1d7dc640 1 Processor -- start 2024-07-31T14:40:54.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.856+0000 7f4e1d7dc640 1 -- start start 2024-07-31T14:40:54.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e1d7dc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e18102c00 0x7f4e180fe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:54.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e1d7dc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e18108040 0x7f4e180fede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:54.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e1d7dc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e18108990 0x7f4e180ff320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:54.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e1d7dc640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4e18112660 con 0x7f4e18102c00 2024-07-31T14:40:54.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e1d7dc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4e181124e0 con 0x7f4e18108990 2024-07-31T14:40:54.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e1d7dc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4e181127e0 con 0x7f4e18108040 2024-07-31T14:40:54.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e16ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e18102c00 0x7f4e180fe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:54.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e16ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e18102c00 0x7f4e180fe8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39352/0 (socket says 172.21.15.67:39352) 2024-07-31T14:40:54.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e16ffd640 1 -- 172.21.15.67:0/593048082 learned_addr learned my addr 172.21.15.67:0/593048082 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:40:54.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e177fe640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e18108990 0x7f4e180ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:54.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e0ffff640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e18108040 0x7f4e180fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:54.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e16ffd640 1 -- 172.21.15.67:0/593048082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e18108040 msgr2=0x7f4e180fede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:54.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e16ffd640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e18108040 0x7f4e180fede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:54.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e16ffd640 1 -- 172.21.15.67:0/593048082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e18108990 msgr2=0x7f4e180ff320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:54.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e16ffd640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e18108990 0x7f4e180ff320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:54.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e16ffd640 1 -- 172.21.15.67:0/593048082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e181a8360 con 0x7f4e18102c00 2024-07-31T14:40:54.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e177fe640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e18108990 0x7f4e180ff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:40:54.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e16ffd640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e18102c00 0x7f4e180fe8a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800c890 tx=0x7f4e0800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:54.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.857+0000 7f4e14ff9640 1 -- 172.21.15.67:0/593048082 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08004220 con 0x7f4e18102c00 2024-07-31T14:40:54.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.858+0000 7f4e14ff9640 1 -- 172.21.15.67:0/593048082 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e080043c0 con 0x7f4e18102c00 2024-07-31T14:40:54.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.858+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/593048082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e181a8650 con 0x7f4e18102c00 2024-07-31T14:40:54.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.858+0000 7f4e14ff9640 1 -- 172.21.15.67:0/593048082 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08004a00 con 0x7f4e18102c00 2024-07-31T14:40:54.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.858+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/593048082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e181a8b90 con 0x7f4e18102c00 2024-07-31T14:40:54.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.859+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/593048082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e18103010 con 0x7f4e18102c00 2024-07-31T14:40:54.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.860+0000 7f4e14ff9640 1 -- 172.21.15.67:0/593048082 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f4e080079b0 con 0x7f4e18102c00 2024-07-31T14:40:54.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.861+0000 7f4e14ff9640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4e040b3b90 0x7f4e040b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:54.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.861+0000 7f4e0ffff640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4e040b3b90 0x7f4e040b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:54.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.861+0000 7f4e14ff9640 1 -- 172.21.15.67:0/593048082 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4e080e2250 con 0x7f4e18102c00 2024-07-31T14:40:54.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.862+0000 7f4e0ffff640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4e040b3b90 0x7f4e040b60a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4e00005fd0 tx=0x7f4e00005e20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:54.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:54.864+0000 7f4e14ff9640 1 -- 172.21.15.67:0/593048082 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f4e08014030 con 0x7f4e18102c00 2024-07-31T14:40:54.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:54 smithi067 ceph-mon[32750]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:54.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:54 smithi077 ceph-mon[37327]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:55.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:55.064+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/593048082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f4e1810b790 con 0x7f4e18102c00 2024-07-31T14:40:55.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:55.071+0000 7f4e14ff9640 1 -- 172.21.15.67:0/593048082 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v22)=0 v22) ==== 108+0+0 (secure 0 0 0) 0x7f4e080ab4a0 con 0x7f4e18102c00 2024-07-31T14:40:55.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:55.073+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/593048082 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4e040b3b90 msgr2=0x7f4e040b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:55.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:55.073+0000 7f4e1d7dc640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4e040b3b90 0x7f4e040b60a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4e00005fd0 tx=0x7f4e00005e20 comp rx=0 tx=0).stop 2024-07-31T14:40:55.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:55.073+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/593048082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e18102c00 msgr2=0x7f4e180fe8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:55.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:55.073+0000 7f4e1d7dc640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e18102c00 0x7f4e180fe8a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800c890 tx=0x7f4e0800cd50 comp rx=0 tx=0).stop 2024-07-31T14:40:55.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:55.073+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/593048082 shutdown_connections 2024-07-31T14:40:55.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:55.073+0000 7f4e1d7dc640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4e040b3b90 0x7f4e040b60a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:55.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:55.073+0000 7f4e1d7dc640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e18108990 0x7f4e180ff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:55.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:55.073+0000 7f4e1d7dc640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e18108040 0x7f4e180fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:55.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:55.073+0000 7f4e1d7dc640 1 --2- 172.21.15.67:0/593048082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e18102c00 0x7f4e180fe8a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:55.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:55.073+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/593048082 >> 172.21.15.67:0/593048082 conn(0x7f4e18079870 msgr2=0x7f4e18100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:55.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:55.073+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/593048082 shutdown_connections 2024-07-31T14:40:55.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:55.074+0000 7f4e1d7dc640 1 -- 172.21.15.67:0/593048082 wait complete. 2024-07-31T14:40:55.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:54 smithi073 ceph-mon[36714]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:56.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:56 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/593048082' entity='client.admin' 2024-07-31T14:40:56.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:56 smithi067 ceph-mon[32750]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:56.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:56 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/593048082' entity='client.admin' 2024-07-31T14:40:56.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:56 smithi073 ceph-mon[36714]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:56.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:56 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/593048082' entity='client.admin' 2024-07-31T14:40:56.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:56 smithi077 ceph-mon[37327]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:56.822 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:40:56.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.914+0000 7f9051b26640 1 -- 172.21.15.67:0/1445898695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f904c108f00 msgr2=0x7f904c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:56.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.914+0000 7f9051b26640 1 --2- 172.21.15.67:0/1445898695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f904c108f00 0x7f904c109310 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9040009900 tx=0x7f904002a410 comp rx=0 tx=0).stop 2024-07-31T14:40:56.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.914+0000 7f9051b26640 1 -- 172.21.15.67:0/1445898695 shutdown_connections 2024-07-31T14:40:56.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.914+0000 7f9051b26640 1 --2- 172.21.15.67:0/1445898695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f904c109850 0x7f904c110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:56.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.914+0000 7f9051b26640 1 --2- 172.21.15.67:0/1445898695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f904c108f00 0x7f904c109310 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:56.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.914+0000 7f9051b26640 1 --2- 172.21.15.67:0/1445898695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f904c1035c0 0x7f904c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:56.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.914+0000 7f9051b26640 1 -- 172.21.15.67:0/1445898695 >> 172.21.15.67:0/1445898695 conn(0x7f904c0ff030 msgr2=0x7f904c101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:56.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.914+0000 7f9051b26640 1 -- 172.21.15.67:0/1445898695 shutdown_connections 2024-07-31T14:40:56.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.914+0000 7f9051b26640 1 -- 172.21.15.67:0/1445898695 wait complete. 2024-07-31T14:40:56.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.915+0000 7f9051b26640 1 Processor -- start 2024-07-31T14:40:56.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.915+0000 7f9051b26640 1 -- start start 2024-07-31T14:40:56.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f9051b26640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f904c1035c0 0x7f904c199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:56.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f9051b26640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f904c108f00 0x7f904c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:56.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f9051b26640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f904c109850 0x7f904c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:56.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f9051b26640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1134e0 con 0x7f904c108f00 2024-07-31T14:40:56.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f9051b26640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f904c113360 con 0x7f904c109850 2024-07-31T14:40:56.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f9051b26640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f904c113660 con 0x7f904c1035c0 2024-07-31T14:40:56.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f904affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f904c108f00 0x7f904c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:56.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f904affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f904c108f00 0x7f904c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39384/0 (socket says 172.21.15.67:39384) 2024-07-31T14:40:56.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f904affd640 1 -- 172.21.15.67:0/1639004934 learned_addr learned my addr 172.21.15.67:0/1639004934 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:40:56.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f904bfff640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f904c109850 0x7f904c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:56.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f904b7fe640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f904c1035c0 0x7f904c199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:56.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f904affd640 1 -- 172.21.15.67:0/1639004934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f904c1035c0 msgr2=0x7f904c199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:56.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f904affd640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f904c1035c0 0x7f904c199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:56.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f904affd640 1 -- 172.21.15.67:0/1639004934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f904c109850 msgr2=0x7f904c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:56.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f904affd640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f904c109850 0x7f904c19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:56.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f904affd640 1 -- 172.21.15.67:0/1639004934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c19fe20 con 0x7f904c108f00 2024-07-31T14:40:56.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f904b7fe640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f904c1035c0 0x7f904c199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:40:56.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.916+0000 7f904affd640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f904c108f00 0x7f904c19a450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f904002a930 tx=0x7f9040004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:56.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.917+0000 7f9048ff9640 1 -- 172.21.15.67:0/1639004934 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904003f030 con 0x7f904c108f00 2024-07-31T14:40:56.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.917+0000 7f9051b26640 1 -- 172.21.15.67:0/1639004934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904c1a00b0 con 0x7f904c108f00 2024-07-31T14:40:56.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.917+0000 7f9048ff9640 1 -- 172.21.15.67:0/1639004934 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9040003aa0 con 0x7f904c108f00 2024-07-31T14:40:56.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.917+0000 7f9048ff9640 1 -- 172.21.15.67:0/1639004934 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040035070 con 0x7f904c108f00 2024-07-31T14:40:56.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.917+0000 7f9051b26640 1 -- 172.21.15.67:0/1639004934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904c075020 con 0x7f904c108f00 2024-07-31T14:40:56.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.919+0000 7f9051b26640 1 -- 172.21.15.67:0/1639004934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904c104840 con 0x7f904c108f00 2024-07-31T14:40:56.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.920+0000 7f9048ff9640 1 -- 172.21.15.67:0/1639004934 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f9040042030 con 0x7f904c108f00 2024-07-31T14:40:56.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.921+0000 7f9048ff9640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90340b3b90 0x7f90340b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:56.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.921+0000 7f904b7fe640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90340b3b90 0x7f90340b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:56.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.921+0000 7f9048ff9640 1 -- 172.21.15.67:0/1639004934 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f90401023c0 con 0x7f904c108f00 2024-07-31T14:40:56.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.922+0000 7f904b7fe640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90340b3b90 0x7f90340b60a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f903c0058c0 tx=0x7f903c005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:56.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.926+0000 7f9048ff9640 1 -- 172.21.15.67:0/1639004934 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f90400cb610 con 0x7f904c108f00 2024-07-31T14:40:57.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.126+0000 7f9051b26640 1 -- 172.21.15.67:0/1639004934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f904c1039d0 con 0x7f904c108f00 2024-07-31T14:40:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.132+0000 7f9048ff9640 1 -- 172.21.15.67:0/1639004934 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) ==== 133+0+0 (secure 0 0 0) 0x7f90400caf10 con 0x7f904c108f00 2024-07-31T14:40:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.134+0000 7f9051b26640 1 -- 172.21.15.67:0/1639004934 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90340b3b90 msgr2=0x7f90340b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.134+0000 7f9051b26640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90340b3b90 0x7f90340b60a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f903c0058c0 tx=0x7f903c005850 comp rx=0 tx=0).stop 2024-07-31T14:40:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.134+0000 7f9051b26640 1 -- 172.21.15.67:0/1639004934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f904c108f00 msgr2=0x7f904c19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.134+0000 7f9051b26640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f904c108f00 0x7f904c19a450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f904002a930 tx=0x7f9040004270 comp rx=0 tx=0).stop 2024-07-31T14:40:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.134+0000 7f9051b26640 1 -- 172.21.15.67:0/1639004934 shutdown_connections 2024-07-31T14:40:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.134+0000 7f9051b26640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90340b3b90 0x7f90340b60a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:57.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.134+0000 7f9051b26640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f904c109850 0x7f904c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:57.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.134+0000 7f9051b26640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f904c108f00 0x7f904c19a450 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:57.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.134+0000 7f9051b26640 1 --2- 172.21.15.67:0/1639004934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f904c1035c0 0x7f904c199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:57.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.134+0000 7f9051b26640 1 -- 172.21.15.67:0/1639004934 >> 172.21.15.67:0/1639004934 conn(0x7f904c0ff030 msgr2=0x7f904c101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:57.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.134+0000 7f9051b26640 1 -- 172.21.15.67:0/1639004934 shutdown_connections 2024-07-31T14:40:57.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.134+0000 7f9051b26640 1 -- 172.21.15.67:0/1639004934 wait complete. 2024-07-31T14:40:58.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:58 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3889456504' entity='client.admin' 2024-07-31T14:40:58.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:58 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1639004934' entity='client.admin' 2024-07-31T14:40:58.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:58.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:58.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:58.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:58.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:40:58 smithi067 ceph-mon[32750]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:58.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:58 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3889456504' entity='client.admin' 2024-07-31T14:40:58.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:58 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1639004934' entity='client.admin' 2024-07-31T14:40:58.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:58.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:58.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:58.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:58.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:40:58 smithi073 ceph-mon[36714]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:58.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:58 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3889456504' entity='client.admin' 2024-07-31T14:40:58.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:58 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1639004934' entity='client.admin' 2024-07-31T14:40:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:40:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:40:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:40:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:40:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:40:58 smithi077 ceph-mon[37327]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:40:59.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.802+0000 7f9d09596640 1 -- 172.21.15.67:0/4057760438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d04108ed0 msgr2=0x7f9d041092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:59.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.802+0000 7f9d09596640 1 --2- 172.21.15.67:0/4057760438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d04108ed0 0x7f9d041092e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009900 tx=0x7f9cf802a410 comp rx=0 tx=0).stop 2024-07-31T14:40:59.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.802+0000 7f9d09596640 1 -- 172.21.15.67:0/4057760438 shutdown_connections 2024-07-31T14:40:59.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.802+0000 7f9d09596640 1 --2- 172.21.15.67:0/4057760438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d04109820 0x7f9d04110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:59.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.802+0000 7f9d09596640 1 --2- 172.21.15.67:0/4057760438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d04108ed0 0x7f9d041092e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:59.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.802+0000 7f9d09596640 1 --2- 172.21.15.67:0/4057760438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d04103590 0x7f9d041039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:59.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.802+0000 7f9d09596640 1 -- 172.21.15.67:0/4057760438 >> 172.21.15.67:0/4057760438 conn(0x7f9d0407b280 msgr2=0x7f9d0407b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:59.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.802+0000 7f9d09596640 1 -- 172.21.15.67:0/4057760438 shutdown_connections 2024-07-31T14:40:59.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.802+0000 7f9d09596640 1 -- 172.21.15.67:0/4057760438 wait complete. 2024-07-31T14:40:59.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.803+0000 7f9d09596640 1 Processor -- start 2024-07-31T14:40:59.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.803+0000 7f9d09596640 1 -- start start 2024-07-31T14:40:59.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.803+0000 7f9d09596640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d04103590 0x7f9d04199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:59.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.803+0000 7f9d09596640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d04108ed0 0x7f9d0419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:59.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.803+0000 7f9d09596640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d04109820 0x7f9d0419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:59.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.803+0000 7f9d09596640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9d041134b0 con 0x7f9d04108ed0 2024-07-31T14:40:59.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.803+0000 7f9d09596640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04113330 con 0x7f9d04103590 2024-07-31T14:40:59.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.803+0000 7f9d09596640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04113630 con 0x7f9d04109820 2024-07-31T14:40:59.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.803+0000 7f9d027fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d04108ed0 0x7f9d0419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:59.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.803+0000 7f9d027fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d04108ed0 0x7f9d0419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39368/0 (socket says 172.21.15.67:39368) 2024-07-31T14:40:59.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.803+0000 7f9d027fc640 1 -- 172.21.15.67:0/3889456504 learned_addr learned my addr 172.21.15.67:0/3889456504 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:40:59.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.804+0000 7f9d037fe640 1 --2- 172.21.15.67:0/3889456504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d04109820 0x7f9d0419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:59.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.804+0000 7f9d02ffd640 1 --2- 172.21.15.67:0/3889456504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d04103590 0x7f9d04199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:59.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.804+0000 7f9d027fc640 1 -- 172.21.15.67:0/3889456504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d04109820 msgr2=0x7f9d0419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:59.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.804+0000 7f9d027fc640 1 --2- 172.21.15.67:0/3889456504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d04109820 0x7f9d0419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:59.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.804+0000 7f9d027fc640 1 -- 172.21.15.67:0/3889456504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d04103590 msgr2=0x7f9d04199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:59.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.804+0000 7f9d027fc640 1 --2- 172.21.15.67:0/3889456504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d04103590 0x7f9d04199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:59.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.804+0000 7f9d027fc640 1 -- 172.21.15.67:0/3889456504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0419feb0 con 0x7f9d04108ed0 2024-07-31T14:40:59.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.804+0000 7f9d027fc640 1 --2- 172.21.15.67:0/3889456504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d04108ed0 0x7f9d0419a4e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9cf802a930 tx=0x7f9cf8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:59.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.804+0000 7f9cebfff640 1 -- 172.21.15.67:0/3889456504 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf803f030 con 0x7f9d04108ed0 2024-07-31T14:40:59.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.804+0000 7f9d09596640 1 -- 172.21.15.67:0/3889456504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d041a0140 con 0x7f9d04108ed0 2024-07-31T14:40:59.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.804+0000 7f9cebfff640 1 -- 172.21.15.67:0/3889456504 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cf8003aa0 con 0x7f9d04108ed0 2024-07-31T14:40:59.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.804+0000 7f9cebfff640 1 -- 172.21.15.67:0/3889456504 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8035070 con 0x7f9d04108ed0 2024-07-31T14:40:59.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.804+0000 7f9d09596640 1 -- 172.21.15.67:0/3889456504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d041a6900 con 0x7f9d04108ed0 2024-07-31T14:40:59.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.805+0000 7f9d09596640 1 -- 172.21.15.67:0/3889456504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d041039a0 con 0x7f9d04108ed0 2024-07-31T14:40:59.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.807+0000 7f9cebfff640 1 -- 172.21.15.67:0/3889456504 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f9cf8042030 con 0x7f9d04108ed0 2024-07-31T14:40:59.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.808+0000 7f9cebfff640 1 --2- 172.21.15.67:0/3889456504 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9cec0b3b90 0x7f9cec0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:40:59.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.808+0000 7f9d02ffd640 1 --2- 172.21.15.67:0/3889456504 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9cec0b3b90 0x7f9cec0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:40:59.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.808+0000 7f9cebfff640 1 -- 172.21.15.67:0/3889456504 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9cf8102a20 con 0x7f9d04108ed0 2024-07-31T14:40:59.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.808+0000 7f9d02ffd640 1 --2- 172.21.15.67:0/3889456504 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9cec0b3b90 0x7f9cec0b60a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400bbe0 tx=0x7f9cf400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:40:59.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:56.811+0000 7f9cebfff640 1 -- 172.21.15.67:0/3889456504 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f9cf80cb620 con 0x7f9d04108ed0 2024-07-31T14:40:59.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.023+0000 7f9d09596640 1 -- 172.21.15.67:0/3889456504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f9d0419b070 con 0x7f9d04108ed0 2024-07-31T14:40:59.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.027+0000 7f9cebfff640 1 -- 172.21.15.67:0/3889456504 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v23)=0 v23) ==== 125+0+0 (secure 0 0 0) 0x7f9cf80cfb70 con 0x7f9d04108ed0 2024-07-31T14:40:59.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.029+0000 7f9d09596640 1 -- 172.21.15.67:0/3889456504 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9cec0b3b90 msgr2=0x7f9cec0b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:59.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.029+0000 7f9d09596640 1 --2- 172.21.15.67:0/3889456504 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9cec0b3b90 0x7f9cec0b60a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400bbe0 tx=0x7f9cf400b390 comp rx=0 tx=0).stop 2024-07-31T14:40:59.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.029+0000 7f9d09596640 1 -- 172.21.15.67:0/3889456504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d04108ed0 msgr2=0x7f9d0419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:40:59.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.029+0000 7f9d09596640 1 --2- 172.21.15.67:0/3889456504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d04108ed0 0x7f9d0419a4e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9cf802a930 tx=0x7f9cf8004270 comp rx=0 tx=0).stop 2024-07-31T14:40:59.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.029+0000 7f9d09596640 1 -- 172.21.15.67:0/3889456504 shutdown_connections 2024-07-31T14:40:59.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.029+0000 7f9d09596640 1 --2- 172.21.15.67:0/3889456504 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9cec0b3b90 0x7f9cec0b60a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:59.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.029+0000 7f9d09596640 1 --2- 172.21.15.67:0/3889456504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d04109820 0x7f9d0419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:59.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.029+0000 7f9d09596640 1 --2- 172.21.15.67:0/3889456504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d04108ed0 0x7f9d0419a4e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:59.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.029+0000 7f9d09596640 1 --2- 172.21.15.67:0/3889456504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d04103590 0x7f9d04199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:40:59.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.029+0000 7f9d09596640 1 -- 172.21.15.67:0/3889456504 >> 172.21.15.67:0/3889456504 conn(0x7f9d0407b280 msgr2=0x7f9d04107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:40:59.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.029+0000 7f9d09596640 1 -- 172.21.15.67:0/3889456504 shutdown_connections 2024-07-31T14:40:59.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:40:57.029+0000 7f9d09596640 1 -- 172.21.15.67:0/3889456504 wait complete. 2024-07-31T14:41:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:00 smithi073 ceph-mon[36714]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:00.696 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:00 smithi067 ceph-mon[32750]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:00 smithi077 ceph-mon[37327]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:00.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.910+0000 7f3183b61640 1 -- 172.21.15.67:0/1662346032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f317c105d40 msgr2=0x7f317c106150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:00.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.910+0000 7f3183b61640 1 --2- 172.21.15.67:0/1662346032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f317c105d40 0x7f317c106150 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f316c009900 tx=0x7f316c02a410 comp rx=0 tx=0).stop 2024-07-31T14:41:00.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.910+0000 7f3183b61640 1 -- 172.21.15.67:0/1662346032 shutdown_connections 2024-07-31T14:41:00.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.910+0000 7f3183b61640 1 --2- 172.21.15.67:0/1662346032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f317c106690 0x7f317c110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:00.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.910+0000 7f3183b61640 1 --2- 172.21.15.67:0/1662346032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f317c105d40 0x7f317c106150 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:00.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.910+0000 7f3183b61640 1 --2- 172.21.15.67:0/1662346032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f317c101b50 0x7f317c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:00.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.910+0000 7f3183b61640 1 -- 172.21.15.67:0/1662346032 >> 172.21.15.67:0/1662346032 conn(0x7f317c0798d0 msgr2=0x7f317c079ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:00.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.911+0000 7f3183b61640 1 -- 172.21.15.67:0/1662346032 shutdown_connections 2024-07-31T14:41:00.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.911+0000 7f3183b61640 1 -- 172.21.15.67:0/1662346032 wait complete. 2024-07-31T14:41:00.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.911+0000 7f3183b61640 1 Processor -- start 2024-07-31T14:41:00.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.911+0000 7f3183b61640 1 -- start start 2024-07-31T14:41:00.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f3183b61640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f317c101b50 0x7f317c102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:00.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f3183b61640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f317c105d40 0x7f317c103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:00.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f3183b61640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f317c106690 0x7f317c103570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:00.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f3183b61640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f317c113540 con 0x7f317c106690 2024-07-31T14:41:00.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f3183b61640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f317c1133c0 con 0x7f317c105d40 2024-07-31T14:41:00.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f3183b61640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f317c1136c0 con 0x7f317c101b50 2024-07-31T14:41:00.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f31820fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f317c106690 0x7f317c103570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:00.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f31820fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f317c106690 0x7f317c103570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36226/0 (socket says 172.21.15.67:36226) 2024-07-31T14:41:00.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f31820fe640 1 -- 172.21.15.67:0/1235582817 learned_addr learned my addr 172.21.15.67:0/1235582817 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:41:00.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f31810fc640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f317c105d40 0x7f317c103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:00.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f31818fd640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f317c101b50 0x7f317c102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:00.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f31820fe640 1 -- 172.21.15.67:0/1235582817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f317c101b50 msgr2=0x7f317c102af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:00.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f31820fe640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f317c101b50 0x7f317c102af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:00.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f31820fe640 1 -- 172.21.15.67:0/1235582817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f317c105d40 msgr2=0x7f317c103030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:00.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f31820fe640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f317c105d40 0x7f317c103030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:00.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f31820fe640 1 -- 172.21.15.67:0/1235582817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c1a81f0 con 0x7f317c106690 2024-07-31T14:41:00.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f31810fc640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f317c105d40 0x7f317c103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:41:00.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f31818fd640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f317c101b50 0x7f317c102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:41:00.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f31820fe640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f317c106690 0x7f317c103570 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f317400eb10 tx=0x7f317400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:00.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f3172ffd640 1 -- 172.21.15.67:0/1235582817 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174004b20 con 0x7f317c106690 2024-07-31T14:41:00.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.912+0000 7f3172ffd640 1 -- 172.21.15.67:0/1235582817 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3174004cc0 con 0x7f317c106690 2024-07-31T14:41:00.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.913+0000 7f3183b61640 1 -- 172.21.15.67:0/1235582817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317c1a84e0 con 0x7f317c106690 2024-07-31T14:41:00.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.913+0000 7f3172ffd640 1 -- 172.21.15.67:0/1235582817 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174010670 con 0x7f317c106690 2024-07-31T14:41:00.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.913+0000 7f3183b61640 1 -- 172.21.15.67:0/1235582817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317c1a89f0 con 0x7f317c106690 2024-07-31T14:41:00.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.914+0000 7f3183b61640 1 -- 172.21.15.67:0/1235582817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317c101f60 con 0x7f317c106690 2024-07-31T14:41:00.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.916+0000 7f3172ffd640 1 -- 172.21.15.67:0/1235582817 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f3174005000 con 0x7f317c106690 2024-07-31T14:41:00.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.917+0000 7f3172ffd640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f31680b3c10 0x7f31680b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:00.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.917+0000 7f31818fd640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f31680b3c10 0x7f31680b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:00.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.917+0000 7f3172ffd640 1 -- 172.21.15.67:0/1235582817 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f31740e2780 con 0x7f317c106690 2024-07-31T14:41:00.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.917+0000 7f31818fd640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f31680b3c10 0x7f31680b6120 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3178005d00 tx=0x7f3178005c90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:00.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:00.920+0000 7f3172ffd640 1 -- 172.21.15.67:0/1235582817 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f31740555e0 con 0x7f317c106690 2024-07-31T14:41:01.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.117+0000 7f3183b61640 1 -- 172.21.15.67:0/1235582817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f317c103df0 con 0x7f317c106690 2024-07-31T14:41:01.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.121+0000 7f3172ffd640 1 -- 172.21.15.67:0/1235582817 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v25)=0 v25) ==== 116+0+0 (secure 0 0 0) 0x7f31740ab9d0 con 0x7f317c106690 2024-07-31T14:41:01.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.123+0000 7f3183b61640 1 -- 172.21.15.67:0/1235582817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f31680b3c10 msgr2=0x7f31680b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:01.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.123+0000 7f3183b61640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f31680b3c10 0x7f31680b6120 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3178005d00 tx=0x7f3178005c90 comp rx=0 tx=0).stop 2024-07-31T14:41:01.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.123+0000 7f3183b61640 1 -- 172.21.15.67:0/1235582817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f317c106690 msgr2=0x7f317c103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:01.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.123+0000 7f3183b61640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f317c106690 0x7f317c103570 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f317400eb10 tx=0x7f317400c2d0 comp rx=0 tx=0).stop 2024-07-31T14:41:01.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.123+0000 7f3183b61640 1 -- 172.21.15.67:0/1235582817 shutdown_connections 2024-07-31T14:41:01.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.123+0000 7f3183b61640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f31680b3c10 0x7f31680b6120 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:01.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.123+0000 7f3183b61640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f317c106690 0x7f317c103570 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:01.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.123+0000 7f3183b61640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f317c105d40 0x7f317c103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:01.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.123+0000 7f3183b61640 1 --2- 172.21.15.67:0/1235582817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f317c101b50 0x7f317c102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:01.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.123+0000 7f3183b61640 1 -- 172.21.15.67:0/1235582817 >> 172.21.15.67:0/1235582817 conn(0x7f317c0798d0 msgr2=0x7f317c106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:01.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.123+0000 7f3183b61640 1 -- 172.21.15.67:0/1235582817 shutdown_connections 2024-07-31T14:41:01.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.123+0000 7f3183b61640 1 -- 172.21.15.67:0/1235582817 wait complete. 2024-07-31T14:41:01.513 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi067') 2024-07-31T14:41:01.513 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi067') 2024-07-31T14:41:01.513 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi067') 2024-07-31T14:41:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.904+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2956097207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c8108720 msgr2=0x7f56c810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.904+0000 7f56cfe0e640 1 --2- 172.21.15.67:0/2956097207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c8108720 0x7f56c810ef50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009ca0 tx=0x7f56c002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:41:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.904+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2956097207 shutdown_connections 2024-07-31T14:41:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.904+0000 7f56cfe0e640 1 --2- 172.21.15.67:0/2956097207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c8108720 0x7f56c810ef50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.904+0000 7f56cfe0e640 1 --2- 172.21.15.67:0/2956097207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56c8107dd0 0x7f56c81081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.904+0000 7f56cfe0e640 1 --2- 172.21.15.67:0/2956097207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c8102490 0x7f56c81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.904+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2956097207 >> 172.21.15.67:0/2956097207 conn(0x7f56c8079160 msgr2=0x7f56c8079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.904+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2956097207 shutdown_connections 2024-07-31T14:41:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.904+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2956097207 wait complete. 2024-07-31T14:41:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.905+0000 7f56cfe0e640 1 Processor -- start 2024-07-31T14:41:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.905+0000 7f56cfe0e640 1 -- start start 2024-07-31T14:41:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.905+0000 7f56cfe0e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c8102490 0x7f56c8199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.905+0000 7f56cfe0e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56c8107dd0 0x7f56c819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.905+0000 7f56cfe0e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c8108720 0x7f56c819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.905+0000 7f56cfe0e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f56c8112440 con 0x7f56c8102490 2024-07-31T14:41:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.905+0000 7f56cfe0e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f56c81122c0 con 0x7f56c8108720 2024-07-31T14:41:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.905+0000 7f56cfe0e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f56c81125c0 con 0x7f56c8107dd0 2024-07-31T14:41:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.905+0000 7f56cdbaa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c8102490 0x7f56c8199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.905+0000 7f56cdbaa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c8102490 0x7f56c8199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36256/0 (socket says 172.21.15.67:36256) 2024-07-31T14:41:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.905+0000 7f56cdbaa640 1 -- 172.21.15.67:0/2054279259 learned_addr learned my addr 172.21.15.67:0/2054279259 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:41:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.905+0000 7f56ce3ab640 1 --2- 172.21.15.67:0/2054279259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c8108720 0x7f56c819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.906+0000 7f56cd3a9640 1 --2- 172.21.15.67:0/2054279259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56c8107dd0 0x7f56c819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.906+0000 7f56cdbaa640 1 -- 172.21.15.67:0/2054279259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56c8107dd0 msgr2=0x7f56c819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.906+0000 7f56cdbaa640 1 --2- 172.21.15.67:0/2054279259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56c8107dd0 0x7f56c819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.906+0000 7f56cdbaa640 1 -- 172.21.15.67:0/2054279259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c8108720 msgr2=0x7f56c819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.906+0000 7f56cdbaa640 1 --2- 172.21.15.67:0/2054279259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c8108720 0x7f56c819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.906+0000 7f56cdbaa640 1 -- 172.21.15.67:0/2054279259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c819fe10 con 0x7f56c8102490 2024-07-31T14:41:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.906+0000 7f56cdbaa640 1 --2- 172.21.15.67:0/2054279259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c8102490 0x7f56c8199ea0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f56c400d950 tx=0x7f56c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.906+0000 7f56beffd640 1 -- 172.21.15.67:0/2054279259 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4017030 con 0x7f56c8102490 2024-07-31T14:41:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.906+0000 7f56beffd640 1 -- 172.21.15.67:0/2054279259 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56c4004020 con 0x7f56c8102490 2024-07-31T14:41:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.906+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2054279259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c81a0100 con 0x7f56c8102490 2024-07-31T14:41:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.906+0000 7f56beffd640 1 -- 172.21.15.67:0/2054279259 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c40047a0 con 0x7f56c8102490 2024-07-31T14:41:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.906+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2054279259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c81a6860 con 0x7f56c8102490 2024-07-31T14:41:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.907+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2054279259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c81028a0 con 0x7f56c8102490 2024-07-31T14:41:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.909+0000 7f56beffd640 1 -- 172.21.15.67:0/2054279259 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f56c4009a30 con 0x7f56c8102490 2024-07-31T14:41:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.910+0000 7f56beffd640 1 --2- 172.21.15.67:0/2054279259 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56b40b3c10 0x7f56b40b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.910+0000 7f56cd3a9640 1 --2- 172.21.15.67:0/2054279259 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56b40b3c10 0x7f56b40b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.910+0000 7f56beffd640 1 -- 172.21.15.67:0/2054279259 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f56c40e1a70 con 0x7f56c8102490 2024-07-31T14:41:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.910+0000 7f56cd3a9640 1 --2- 172.21.15.67:0/2054279259 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56b40b3c10 0x7f56b40b6120 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f56c81075e0 tx=0x7f56b8008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:01.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:01.914+0000 7f56beffd640 1 -- 172.21.15.67:0/2054279259 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f56c40aacc0 con 0x7f56c8102490 2024-07-31T14:41:02.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.113+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2054279259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f56c819afd0 con 0x7f56c8102490 2024-07-31T14:41:02.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.118+0000 7f56beffd640 1 -- 172.21.15.67:0/2054279259 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f56c40aa5c0 con 0x7f56c8102490 2024-07-31T14:41:02.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.121+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2054279259 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56b40b3c10 msgr2=0x7f56b40b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:02.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.121+0000 7f56cfe0e640 1 --2- 172.21.15.67:0/2054279259 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56b40b3c10 0x7f56b40b6120 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f56c81075e0 tx=0x7f56b8008040 comp rx=0 tx=0).stop 2024-07-31T14:41:02.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.121+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2054279259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c8102490 msgr2=0x7f56c8199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:02.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.121+0000 7f56cfe0e640 1 --2- 172.21.15.67:0/2054279259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c8102490 0x7f56c8199ea0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f56c400d950 tx=0x7f56c400de10 comp rx=0 tx=0).stop 2024-07-31T14:41:02.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.121+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2054279259 shutdown_connections 2024-07-31T14:41:02.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.121+0000 7f56cfe0e640 1 --2- 172.21.15.67:0/2054279259 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56b40b3c10 0x7f56b40b6120 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:02.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.121+0000 7f56cfe0e640 1 --2- 172.21.15.67:0/2054279259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56c8108720 0x7f56c819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:02.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.121+0000 7f56cfe0e640 1 --2- 172.21.15.67:0/2054279259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56c8107dd0 0x7f56c819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:02.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.121+0000 7f56cfe0e640 1 --2- 172.21.15.67:0/2054279259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c8102490 0x7f56c8199ea0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:02.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.121+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2054279259 >> 172.21.15.67:0/2054279259 conn(0x7f56c8079160 msgr2=0x7f56c806ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:02.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.121+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2054279259 shutdown_connections 2024-07-31T14:41:02.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.121+0000 7f56cfe0e640 1 -- 172.21.15.67:0/2054279259 wait complete. 2024-07-31T14:41:02.325 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:02 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1235582817' entity='client.admin' 2024-07-31T14:41:02.325 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:02 smithi067 ceph-mon[32750]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:02.325 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:02 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2054279259' entity='client.admin' 2024-07-31T14:41:02.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:02 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1235582817' entity='client.admin' 2024-07-31T14:41:02.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:02 smithi073 ceph-mon[36714]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:02.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:02 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2054279259' entity='client.admin' 2024-07-31T14:41:02.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:02 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1235582817' entity='client.admin' 2024-07-31T14:41:02.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:02 smithi077 ceph-mon[37327]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:02.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:02 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2054279259' entity='client.admin' 2024-07-31T14:41:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.167+0000 7f1802d39640 1 -- 172.21.15.67:0/1359927919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17fc108720 msgr2=0x7f17fc10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.167+0000 7f1802d39640 1 --2- 172.21.15.67:0/1359927919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17fc108720 0x7f17fc10ef50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009ca0 tx=0x7f17f402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:41:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.167+0000 7f1802d39640 1 -- 172.21.15.67:0/1359927919 shutdown_connections 2024-07-31T14:41:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.167+0000 7f1802d39640 1 --2- 172.21.15.67:0/1359927919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17fc108720 0x7f17fc10ef50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.167+0000 7f1802d39640 1 --2- 172.21.15.67:0/1359927919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17fc107dd0 0x7f17fc1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.167+0000 7f1802d39640 1 --2- 172.21.15.67:0/1359927919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17fc102490 0x7f17fc1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.167+0000 7f1802d39640 1 -- 172.21.15.67:0/1359927919 >> 172.21.15.67:0/1359927919 conn(0x7f17fc079160 msgr2=0x7f17fc079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.167+0000 7f1802d39640 1 -- 172.21.15.67:0/1359927919 shutdown_connections 2024-07-31T14:41:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.167+0000 7f1802d39640 1 -- 172.21.15.67:0/1359927919 wait complete. 2024-07-31T14:41:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.168+0000 7f1802d39640 1 Processor -- start 2024-07-31T14:41:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.168+0000 7f1802d39640 1 -- start start 2024-07-31T14:41:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.168+0000 7f1802d39640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17fc102490 0x7f17fc199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.168+0000 7f1802d39640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17fc107dd0 0x7f17fc19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f1802d39640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17fc108720 0x7f17fc19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f1802d39640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1123a0 con 0x7f17fc108720 2024-07-31T14:41:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f1802d39640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc112220 con 0x7f17fc107dd0 2024-07-31T14:41:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f1802d39640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc112520 con 0x7f17fc102490 2024-07-31T14:41:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f18012d6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17fc108720 0x7f17fc19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f18012d6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17fc108720 0x7f17fc19f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36282/0 (socket says 172.21.15.67:36282) 2024-07-31T14:41:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f18012d6640 1 -- 172.21.15.67:0/1904400111 learned_addr learned my addr 172.21.15.67:0/1904400111 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:41:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f17fbfff640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17fc107dd0 0x7f17fc19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f1800ad5640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17fc102490 0x7f17fc199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f18012d6640 1 -- 172.21.15.67:0/1904400111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17fc102490 msgr2=0x7f17fc199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f18012d6640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17fc102490 0x7f17fc199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f18012d6640 1 -- 172.21.15.67:0/1904400111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17fc107dd0 msgr2=0x7f17fc19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f18012d6640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17fc107dd0 0x7f17fc19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f18012d6640 1 -- 172.21.15.67:0/1904400111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc19fdd0 con 0x7f17fc108720 2024-07-31T14:41:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f1800ad5640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17fc102490 0x7f17fc199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:41:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f18012d6640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17fc108720 0x7f17fc19f6a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f17f4005cc0 tx=0x7f17f402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f17f9ffb640 1 -- 172.21.15.67:0/1904400111 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4003bf0 con 0x7f17fc108720 2024-07-31T14:41:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f1802d39640 1 -- 172.21.15.67:0/1904400111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc1a0060 con 0x7f17fc108720 2024-07-31T14:41:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f17f9ffb640 1 -- 172.21.15.67:0/1904400111 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f4003d90 con 0x7f17fc108720 2024-07-31T14:41:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.169+0000 7f17f9ffb640 1 -- 172.21.15.67:0/1904400111 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f402fbc0 con 0x7f17fc108720 2024-07-31T14:41:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.170+0000 7f1802d39640 1 -- 172.21.15.67:0/1904400111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc1a6820 con 0x7f17fc108720 2024-07-31T14:41:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.170+0000 7f1802d39640 1 -- 172.21.15.67:0/1904400111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17fc1028a0 con 0x7f17fc108720 2024-07-31T14:41:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.172+0000 7f17f9ffb640 1 -- 172.21.15.67:0/1904400111 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f17f402fd60 con 0x7f17fc108720 2024-07-31T14:41:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.173+0000 7f17f9ffb640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f17ec0b3b90 0x7f17ec0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.173+0000 7f1800ad5640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f17ec0b3b90 0x7f17ec0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.173+0000 7f17f9ffb640 1 -- 172.21.15.67:0/1904400111 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f17f4103980 con 0x7f17fc108720 2024-07-31T14:41:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.173+0000 7f1800ad5640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f17ec0b3b90 0x7f17ec0b60a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009b40 tx=0x7f17f0005e50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.176+0000 7f17f9ffb640 1 -- 172.21.15.67:0/1904400111 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f17f40ccbd0 con 0x7f17fc108720 2024-07-31T14:41:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.375+0000 7f1802d39640 1 -- 172.21.15.67:0/1904400111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f17fc1a6a50 con 0x7f17fc108720 2024-07-31T14:41:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.379+0000 7f17f9ffb640 1 -- 172.21.15.67:0/1904400111 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v27)=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7f17f40cc4d0 con 0x7f17fc108720 2024-07-31T14:41:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.381+0000 7f1802d39640 1 -- 172.21.15.67:0/1904400111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f17ec0b3b90 msgr2=0x7f17ec0b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.381+0000 7f1802d39640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f17ec0b3b90 0x7f17ec0b60a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009b40 tx=0x7f17f0005e50 comp rx=0 tx=0).stop 2024-07-31T14:41:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.381+0000 7f1802d39640 1 -- 172.21.15.67:0/1904400111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17fc108720 msgr2=0x7f17fc19f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.381+0000 7f1802d39640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17fc108720 0x7f17fc19f6a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f17f4005cc0 tx=0x7f17f402f380 comp rx=0 tx=0).stop 2024-07-31T14:41:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.381+0000 7f1802d39640 1 -- 172.21.15.67:0/1904400111 shutdown_connections 2024-07-31T14:41:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.381+0000 7f1802d39640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f17ec0b3b90 0x7f17ec0b60a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.381+0000 7f1802d39640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17fc108720 0x7f17fc19f6a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.381+0000 7f1802d39640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17fc107dd0 0x7f17fc19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.381+0000 7f1802d39640 1 --2- 172.21.15.67:0/1904400111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17fc102490 0x7f17fc199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.382+0000 7f1802d39640 1 -- 172.21.15.67:0/1904400111 >> 172.21.15.67:0/1904400111 conn(0x7f17fc079160 msgr2=0x7f17fc101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.382+0000 7f1802d39640 1 -- 172.21.15.67:0/1904400111 shutdown_connections 2024-07-31T14:41:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:02.382+0000 7f1802d39640 1 -- 172.21.15.67:0/1904400111 wait complete. 2024-07-31T14:41:03.534 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:03 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1904400111' entity='client.admin' 2024-07-31T14:41:03.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:03 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1904400111' entity='client.admin' 2024-07-31T14:41:03.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:03 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1904400111' entity='client.admin' 2024-07-31T14:41:04.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:04 smithi067 ceph-mon[32750]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:04.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:04 smithi073 ceph-mon[36714]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:04.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:04 smithi077 ceph-mon[37327]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:05.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.363+0000 7f4b29f6c640 1 -- 172.21.15.67:0/3114360450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b24107df0 msgr2=0x7f4b24108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:05.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.363+0000 7f4b29f6c640 1 --2- 172.21.15.67:0/3114360450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b24107df0 0x7f4b24108200 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009900 tx=0x7f4b1802a410 comp rx=0 tx=0).stop 2024-07-31T14:41:05.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.363+0000 7f4b29f6c640 1 -- 172.21.15.67:0/3114360450 shutdown_connections 2024-07-31T14:41:05.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.363+0000 7f4b29f6c640 1 --2- 172.21.15.67:0/3114360450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b24108740 0x7f4b2410ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.363+0000 7f4b29f6c640 1 --2- 172.21.15.67:0/3114360450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b24107df0 0x7f4b24108200 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.363+0000 7f4b29f6c640 1 --2- 172.21.15.67:0/3114360450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b241024b0 0x7f4b241028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.363+0000 7f4b29f6c640 1 -- 172.21.15.67:0/3114360450 >> 172.21.15.67:0/3114360450 conn(0x7f4b24079170 msgr2=0x7f4b24079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:05.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.364+0000 7f4b29f6c640 1 -- 172.21.15.67:0/3114360450 shutdown_connections 2024-07-31T14:41:05.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.364+0000 7f4b29f6c640 1 -- 172.21.15.67:0/3114360450 wait complete. 2024-07-31T14:41:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.364+0000 7f4b29f6c640 1 Processor -- start 2024-07-31T14:41:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b29f6c640 1 -- start start 2024-07-31T14:41:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b29f6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b241024b0 0x7f4b2419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b29f6c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b24107df0 0x7f4b2419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b29f6c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b24108740 0x7f4b2419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b29f6c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4b24112370 con 0x7f4b24107df0 2024-07-31T14:41:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b29f6c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241121f0 con 0x7f4b24108740 2024-07-31T14:41:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b29f6c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241124f0 con 0x7f4b241024b0 2024-07-31T14:41:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b22ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b24107df0 0x7f4b2419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b22ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b24107df0 0x7f4b2419b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36298/0 (socket says 172.21.15.67:36298) 2024-07-31T14:41:05.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b22ffd640 1 -- 172.21.15.67:0/4170542371 learned_addr learned my addr 172.21.15.67:0/4170542371 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:41:05.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b237fe640 1 --2- 172.21.15.67:0/4170542371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b241024b0 0x7f4b2419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:05.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b23fff640 1 --2- 172.21.15.67:0/4170542371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b24108740 0x7f4b2419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:05.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b22ffd640 1 -- 172.21.15.67:0/4170542371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b241024b0 msgr2=0x7f4b2419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:05.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b22ffd640 1 --2- 172.21.15.67:0/4170542371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b241024b0 0x7f4b2419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b22ffd640 1 -- 172.21.15.67:0/4170542371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b24108740 msgr2=0x7f4b2419b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:05.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.365+0000 7f4b22ffd640 1 --2- 172.21.15.67:0/4170542371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b24108740 0x7f4b2419b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.366+0000 7f4b22ffd640 1 -- 172.21.15.67:0/4170542371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b241a6020 con 0x7f4b24107df0 2024-07-31T14:41:05.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.366+0000 7f4b22ffd640 1 --2- 172.21.15.67:0/4170542371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b24107df0 0x7f4b2419b2a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4b18032fd0 tx=0x7f4b18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:05.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.366+0000 7f4b20ff9640 1 -- 172.21.15.67:0/4170542371 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1803f030 con 0x7f4b24107df0 2024-07-31T14:41:05.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.366+0000 7f4b20ff9640 1 -- 172.21.15.67:0/4170542371 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b18004490 con 0x7f4b24107df0 2024-07-31T14:41:05.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.366+0000 7f4b29f6c640 1 -- 172.21.15.67:0/4170542371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b241a62b0 con 0x7f4b24107df0 2024-07-31T14:41:05.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.366+0000 7f4b29f6c640 1 -- 172.21.15.67:0/4170542371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b241a6790 con 0x7f4b24107df0 2024-07-31T14:41:05.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.366+0000 7f4b20ff9640 1 -- 172.21.15.67:0/4170542371 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18035070 con 0x7f4b24107df0 2024-07-31T14:41:05.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.367+0000 7f4b29f6c640 1 -- 172.21.15.67:0/4170542371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b24103730 con 0x7f4b24107df0 2024-07-31T14:41:05.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.369+0000 7f4b20ff9640 1 -- 172.21.15.67:0/4170542371 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f4b18042030 con 0x7f4b24107df0 2024-07-31T14:41:05.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.369+0000 7f4b20ff9640 1 --2- 172.21.15.67:0/4170542371 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4b0c0b3b90 0x7f4b0c0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:05.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.370+0000 7f4b237fe640 1 --2- 172.21.15.67:0/4170542371 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4b0c0b3b90 0x7f4b0c0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:05.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.370+0000 7f4b20ff9640 1 -- 172.21.15.67:0/4170542371 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4b18102880 con 0x7f4b24107df0 2024-07-31T14:41:05.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.370+0000 7f4b237fe640 1 --2- 172.21.15.67:0/4170542371 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4b0c0b3b90 0x7f4b0c0b60a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4b2406c110 tx=0x7f4b1400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:05.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.373+0000 7f4b20ff9640 1 -- 172.21.15.67:0/4170542371 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f4b180cb480 con 0x7f4b24107df0 2024-07-31T14:41:05.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.571+0000 7f4b29f6c640 1 -- 172.21.15.67:0/4170542371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f4b2410a560 con 0x7f4b24107df0 2024-07-31T14:41:05.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.579+0000 7f4b20ff9640 1 -- 172.21.15.67:0/4170542371 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v28)=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f4b180cf9d0 con 0x7f4b24107df0 2024-07-31T14:41:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.581+0000 7f4b29f6c640 1 -- 172.21.15.67:0/4170542371 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4b0c0b3b90 msgr2=0x7f4b0c0b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.581+0000 7f4b29f6c640 1 --2- 172.21.15.67:0/4170542371 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4b0c0b3b90 0x7f4b0c0b60a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4b2406c110 tx=0x7f4b1400b390 comp rx=0 tx=0).stop 2024-07-31T14:41:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.582+0000 7f4b29f6c640 1 -- 172.21.15.67:0/4170542371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b24107df0 msgr2=0x7f4b2419b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.582+0000 7f4b29f6c640 1 --2- 172.21.15.67:0/4170542371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b24107df0 0x7f4b2419b2a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4b18032fd0 tx=0x7f4b18004270 comp rx=0 tx=0).stop 2024-07-31T14:41:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.582+0000 7f4b29f6c640 1 -- 172.21.15.67:0/4170542371 shutdown_connections 2024-07-31T14:41:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.582+0000 7f4b29f6c640 1 --2- 172.21.15.67:0/4170542371 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4b0c0b3b90 0x7f4b0c0b60a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.582+0000 7f4b29f6c640 1 --2- 172.21.15.67:0/4170542371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b24108740 0x7f4b2419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.582+0000 7f4b29f6c640 1 --2- 172.21.15.67:0/4170542371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b24107df0 0x7f4b2419b2a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.582+0000 7f4b29f6c640 1 --2- 172.21.15.67:0/4170542371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b241024b0 0x7f4b2419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.582+0000 7f4b29f6c640 1 -- 172.21.15.67:0/4170542371 >> 172.21.15.67:0/4170542371 conn(0x7f4b24079170 msgr2=0x7f4b24103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:05.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.582+0000 7f4b29f6c640 1 -- 172.21.15.67:0/4170542371 shutdown_connections 2024-07-31T14:41:05.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.582+0000 7f4b29f6c640 1 -- 172.21.15.67:0/4170542371 wait complete. 2024-07-31T14:41:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.686+0000 7f12f8806640 1 -- 172.21.15.67:0/1564098544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12f41024a0 msgr2=0x7f12f41028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.686+0000 7f12f8806640 1 --2- 172.21.15.67:0/1564098544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12f41024a0 0x7f12f41028b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009900 tx=0x7f12dc02a410 comp rx=0 tx=0).stop 2024-07-31T14:41:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.686+0000 7f12f8806640 1 -- 172.21.15.67:0/1564098544 shutdown_connections 2024-07-31T14:41:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.686+0000 7f12f8806640 1 --2- 172.21.15.67:0/1564098544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12f4108730 0x7f12f410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.686+0000 7f12f8806640 1 --2- 172.21.15.67:0/1564098544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12f4107de0 0x7f12f41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.686+0000 7f12f8806640 1 --2- 172.21.15.67:0/1564098544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12f41024a0 0x7f12f41028b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.686+0000 7f12f8806640 1 -- 172.21.15.67:0/1564098544 >> 172.21.15.67:0/1564098544 conn(0x7f12f4079160 msgr2=0x7f12f4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.686+0000 7f12f8806640 1 -- 172.21.15.67:0/1564098544 shutdown_connections 2024-07-31T14:41:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.686+0000 7f12f8806640 1 -- 172.21.15.67:0/1564098544 wait complete. 2024-07-31T14:41:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.687+0000 7f12f8806640 1 Processor -- start 2024-07-31T14:41:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.687+0000 7f12f8806640 1 -- start start 2024-07-31T14:41:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.687+0000 7f12f8806640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12f41024a0 0x7f12f4199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.687+0000 7f12f8806640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12f4107de0 0x7f12f419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.687+0000 7f12f8806640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12f4108730 0x7f12f419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.687+0000 7f12f8806640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41123b0 con 0x7f12f4108730 2024-07-31T14:41:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.687+0000 7f12f8806640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f12f4112230 con 0x7f12f41024a0 2024-07-31T14:41:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.687+0000 7f12f8806640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12f4112530 con 0x7f12f4107de0 2024-07-31T14:41:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.687+0000 7f12f259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12f41024a0 0x7f12f4199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.687+0000 7f12f2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12f4108730 0x7f12f419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.687+0000 7f12f259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12f41024a0 0x7f12f4199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46864/0 (socket says 172.21.15.67:46864) 2024-07-31T14:41:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.687+0000 7f12f259c640 1 -- 172.21.15.67:0/1563044116 learned_addr learned my addr 172.21.15.67:0/1563044116 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:41:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.687+0000 7f12f2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12f4108730 0x7f12f419f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36328/0 (socket says 172.21.15.67:36328) 2024-07-31T14:41:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.687+0000 7f12f1d9b640 1 --2- 172.21.15.67:0/1563044116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12f4107de0 0x7f12f419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.688+0000 7f12f259c640 1 -- 172.21.15.67:0/1563044116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12f4107de0 msgr2=0x7f12f419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.688+0000 7f12f259c640 1 --2- 172.21.15.67:0/1563044116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12f4107de0 0x7f12f419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.688+0000 7f12f259c640 1 -- 172.21.15.67:0/1563044116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12f4108730 msgr2=0x7f12f419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.688+0000 7f12f259c640 1 --2- 172.21.15.67:0/1563044116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12f4108730 0x7f12f419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.688+0000 7f12f259c640 1 -- 172.21.15.67:0/1563044116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f419fe40 con 0x7f12f41024a0 2024-07-31T14:41:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.688+0000 7f12f2d9d640 1 --2- 172.21.15.67:0/1563044116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12f4108730 0x7f12f419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:41:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.688+0000 7f12f259c640 1 --2- 172.21.15.67:0/1563044116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12f41024a0 0x7f12f4199ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f12dc005940 tx=0x7f12dc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.688+0000 7f12eb7fe640 1 -- 172.21.15.67:0/1563044116 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc03f030 con 0x7f12f41024a0 2024-07-31T14:41:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.688+0000 7f12f8806640 1 -- 172.21.15.67:0/1563044116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f41a00d0 con 0x7f12f41024a0 2024-07-31T14:41:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.688+0000 7f12f8806640 1 -- 172.21.15.67:0/1563044116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f41a6890 con 0x7f12f41024a0 2024-07-31T14:41:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.688+0000 7f12eb7fe640 1 -- 172.21.15.67:0/1563044116 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12dc003cb0 con 0x7f12f41024a0 2024-07-31T14:41:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.688+0000 7f12eb7fe640 1 -- 172.21.15.67:0/1563044116 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc049060 con 0x7f12f41024a0 2024-07-31T14:41:05.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.689+0000 7f12f8806640 1 -- 172.21.15.67:0/1563044116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12f4103720 con 0x7f12f41024a0 2024-07-31T14:41:05.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.691+0000 7f12eb7fe640 1 -- 172.21.15.67:0/1563044116 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f12dc042030 con 0x7f12f41024a0 2024-07-31T14:41:05.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.692+0000 7f12eb7fe640 1 --2- 172.21.15.67:0/1563044116 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f12e00b3b90 0x7f12e00b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:05.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.692+0000 7f12f1d9b640 1 --2- 172.21.15.67:0/1563044116 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f12e00b3b90 0x7f12e00b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:05.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.692+0000 7f12eb7fe640 1 -- 172.21.15.67:0/1563044116 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f12dc102670 con 0x7f12f41024a0 2024-07-31T14:41:05.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.692+0000 7f12f1d9b640 1 --2- 172.21.15.67:0/1563044116 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f12e00b3b90 0x7f12e00b60a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f12f4100520 tx=0x7f12e400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:05.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.695+0000 7f12eb7fe640 1 -- 172.21.15.67:0/1563044116 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f12dc0cb1d0 con 0x7f12f41024a0 2024-07-31T14:41:05.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.899+0000 7f12f8806640 1 -- 172.21.15.67:0/1563044116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f12f41028b0 con 0x7f12f41024a0 2024-07-31T14:41:05.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.906+0000 7f12eb7fe640 1 -- 172.21.15.67:0/1563044116 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v30)=0 v30) ==== 119+0+0 (secure 0 0 0) 0x7f12dc0cf720 con 0x7f12f41024a0 2024-07-31T14:41:05.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.908+0000 7f12f8806640 1 -- 172.21.15.67:0/1563044116 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f12e00b3b90 msgr2=0x7f12e00b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:05.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.908+0000 7f12f8806640 1 --2- 172.21.15.67:0/1563044116 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f12e00b3b90 0x7f12e00b60a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f12f4100520 tx=0x7f12e400b390 comp rx=0 tx=0).stop 2024-07-31T14:41:05.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.908+0000 7f12f8806640 1 -- 172.21.15.67:0/1563044116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12f41024a0 msgr2=0x7f12f4199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:05.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.908+0000 7f12f8806640 1 --2- 172.21.15.67:0/1563044116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12f41024a0 0x7f12f4199ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f12dc005940 tx=0x7f12dc0042f0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.908+0000 7f12f8806640 1 -- 172.21.15.67:0/1563044116 shutdown_connections 2024-07-31T14:41:05.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.908+0000 7f12f8806640 1 --2- 172.21.15.67:0/1563044116 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f12e00b3b90 0x7f12e00b60a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.908+0000 7f12f8806640 1 --2- 172.21.15.67:0/1563044116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12f4108730 0x7f12f419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.908+0000 7f12f8806640 1 --2- 172.21.15.67:0/1563044116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12f4107de0 0x7f12f419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.908+0000 7f12f8806640 1 --2- 172.21.15.67:0/1563044116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12f41024a0 0x7f12f4199ed0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:05.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.908+0000 7f12f8806640 1 -- 172.21.15.67:0/1563044116 >> 172.21.15.67:0/1563044116 conn(0x7f12f4079160 msgr2=0x7f12f41014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:05.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.909+0000 7f12f8806640 1 -- 172.21.15.67:0/1563044116 shutdown_connections 2024-07-31T14:41:05.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.909+0000 7f12f8806640 1 -- 172.21.15.67:0/1563044116 wait complete. 2024-07-31T14:41:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.567+0000 7efe2874c640 1 -- 172.21.15.67:0/1322192304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe20108720 msgr2=0x7efe2010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.567+0000 7efe2874c640 1 --2- 172.21.15.67:0/1322192304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe20108720 0x7efe2010ef50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efe18009ca0 tx=0x7efe1802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.567+0000 7efe2874c640 1 -- 172.21.15.67:0/1322192304 shutdown_connections 2024-07-31T14:41:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.567+0000 7efe2874c640 1 --2- 172.21.15.67:0/1322192304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe20108720 0x7efe2010ef50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.567+0000 7efe2874c640 1 --2- 172.21.15.67:0/1322192304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe20107dd0 0x7efe201081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.567+0000 7efe2874c640 1 --2- 172.21.15.67:0/1322192304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe20102490 0x7efe201028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.567+0000 7efe2874c640 1 -- 172.21.15.67:0/1322192304 >> 172.21.15.67:0/1322192304 conn(0x7efe20079160 msgr2=0x7efe20079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.567+0000 7efe2874c640 1 -- 172.21.15.67:0/1322192304 shutdown_connections 2024-07-31T14:41:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.567+0000 7efe2874c640 1 -- 172.21.15.67:0/1322192304 wait complete. 2024-07-31T14:41:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.568+0000 7efe2874c640 1 Processor -- start 2024-07-31T14:41:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.568+0000 7efe2874c640 1 -- start start 2024-07-31T14:41:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.568+0000 7efe2874c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe20102490 0x7efe20199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.568+0000 7efe2874c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe20107dd0 0x7efe2019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.568+0000 7efe2874c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe20108720 0x7efe2019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.568+0000 7efe2874c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efe20112380 con 0x7efe20107dd0 2024-07-31T14:41:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.568+0000 7efe2874c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe20112200 con 0x7efe20108720 2024-07-31T14:41:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.568+0000 7efe2874c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efe20112500 con 0x7efe20102490 2024-07-31T14:41:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.568+0000 7efe25ce7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe20107dd0 0x7efe2019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.568+0000 7efe25ce7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe20107dd0 0x7efe2019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36314/0 (socket says 172.21.15.67:36314) 2024-07-31T14:41:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.568+0000 7efe25ce7640 1 -- 172.21.15.67:0/70826174 learned_addr learned my addr 172.21.15.67:0/70826174 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:41:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.568+0000 7efe264e8640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe20102490 0x7efe20199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.568+0000 7efe26ce9640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe20108720 0x7efe2019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.569+0000 7efe25ce7640 1 -- 172.21.15.67:0/70826174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe20102490 msgr2=0x7efe20199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.569+0000 7efe25ce7640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe20102490 0x7efe20199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.569+0000 7efe25ce7640 1 -- 172.21.15.67:0/70826174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe20108720 msgr2=0x7efe2019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.569+0000 7efe25ce7640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe20108720 0x7efe2019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.569+0000 7efe25ce7640 1 -- 172.21.15.67:0/70826174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2019fe10 con 0x7efe20107dd0 2024-07-31T14:41:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.569+0000 7efe26ce9640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe20108720 0x7efe2019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:41:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.569+0000 7efe25ce7640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe20107dd0 0x7efe2019a440 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe1400d770 tx=0x7efe1400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.569+0000 7efe137fe640 1 -- 172.21.15.67:0/70826174 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14017070 con 0x7efe20107dd0 2024-07-31T14:41:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.569+0000 7efe2874c640 1 -- 172.21.15.67:0/70826174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe201a0100 con 0x7efe20107dd0 2024-07-31T14:41:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.569+0000 7efe137fe640 1 -- 172.21.15.67:0/70826174 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe14009c30 con 0x7efe20107dd0 2024-07-31T14:41:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.569+0000 7efe137fe640 1 -- 172.21.15.67:0/70826174 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1400f620 con 0x7efe20107dd0 2024-07-31T14:41:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.569+0000 7efe2874c640 1 -- 172.21.15.67:0/70826174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe201a6860 con 0x7efe20107dd0 2024-07-31T14:41:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.570+0000 7efe2874c640 1 -- 172.21.15.67:0/70826174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe201028a0 con 0x7efe20107dd0 2024-07-31T14:41:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.572+0000 7efe137fe640 1 -- 172.21.15.67:0/70826174 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7efe1401a070 con 0x7efe20107dd0 2024-07-31T14:41:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.572+0000 7efe137fe640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efe0c0b3b90 0x7efe0c0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.573+0000 7efe264e8640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efe0c0b3b90 0x7efe0c0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.573+0000 7efe137fe640 1 -- 172.21.15.67:0/70826174 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7efe140e1b50 con 0x7efe20107dd0 2024-07-31T14:41:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.573+0000 7efe264e8640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efe0c0b3b90 0x7efe0c0b60a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0058c0 tx=0x7efe1c00a4c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.576+0000 7efe137fe640 1 -- 172.21.15.67:0/70826174 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7efe14009de0 con 0x7efe20107dd0 2024-07-31T14:41:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.777+0000 7efe2874c640 1 -- 172.21.15.67:0/70826174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7efe2010b550 con 0x7efe20107dd0 2024-07-31T14:41:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.781+0000 7efe137fe640 1 -- 172.21.15.67:0/70826174 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v29)=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7efe140aada0 con 0x7efe20107dd0 2024-07-31T14:41:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.783+0000 7efe2874c640 1 -- 172.21.15.67:0/70826174 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efe0c0b3b90 msgr2=0x7efe0c0b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.783+0000 7efe2874c640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efe0c0b3b90 0x7efe0c0b60a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0058c0 tx=0x7efe1c00a4c0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.783+0000 7efe2874c640 1 -- 172.21.15.67:0/70826174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe20107dd0 msgr2=0x7efe2019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.783+0000 7efe2874c640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe20107dd0 0x7efe2019a440 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe1400d770 tx=0x7efe1400dc30 comp rx=0 tx=0).stop 2024-07-31T14:41:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.783+0000 7efe2874c640 1 -- 172.21.15.67:0/70826174 shutdown_connections 2024-07-31T14:41:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.783+0000 7efe2874c640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efe0c0b3b90 0x7efe0c0b60a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.783+0000 7efe2874c640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe20108720 0x7efe2019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.783+0000 7efe2874c640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe20107dd0 0x7efe2019a440 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.783+0000 7efe2874c640 1 --2- 172.21.15.67:0/70826174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe20102490 0x7efe20199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.783+0000 7efe2874c640 1 -- 172.21.15.67:0/70826174 >> 172.21.15.67:0/70826174 conn(0x7efe20079160 msgr2=0x7efe2006ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:06.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.784+0000 7efe2874c640 1 -- 172.21.15.67:0/70826174 shutdown_connections 2024-07-31T14:41:06.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:05.784+0000 7efe2874c640 1 -- 172.21.15.67:0/70826174 wait complete. 2024-07-31T14:41:06.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.752+0000 7fd504ea4640 1 -- 172.21.15.67:0/4092681167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5001024b0 msgr2=0x7fd5001028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:06.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.752+0000 7fd504ea4640 1 --2- 172.21.15.67:0/4092681167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5001024b0 0x7fd5001028c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009900 tx=0x7fd4f002a410 comp rx=0 tx=0).stop 2024-07-31T14:41:06.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.752+0000 7fd504ea4640 1 -- 172.21.15.67:0/4092681167 shutdown_connections 2024-07-31T14:41:06.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.752+0000 7fd504ea4640 1 --2- 172.21.15.67:0/4092681167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd500108740 0x7fd50010ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.752+0000 7fd504ea4640 1 --2- 172.21.15.67:0/4092681167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd500107df0 0x7fd500108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.752+0000 7fd504ea4640 1 --2- 172.21.15.67:0/4092681167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5001024b0 0x7fd5001028c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.752+0000 7fd504ea4640 1 -- 172.21.15.67:0/4092681167 >> 172.21.15.67:0/4092681167 conn(0x7fd500079170 msgr2=0x7fd500079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:06.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.752+0000 7fd504ea4640 1 -- 172.21.15.67:0/4092681167 shutdown_connections 2024-07-31T14:41:06.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.752+0000 7fd504ea4640 1 -- 172.21.15.67:0/4092681167 wait complete. 2024-07-31T14:41:06.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.754+0000 7fd504ea4640 1 Processor -- start 2024-07-31T14:41:06.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.754+0000 7fd504ea4640 1 -- start start 2024-07-31T14:41:06.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.754+0000 7fd504ea4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5001024b0 0x7fd500198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:06.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.754+0000 7fd504ea4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd500107df0 0x7fd5001990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:06.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.754+0000 7fd504ea4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd500108740 0x7fd5001995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:06.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.754+0000 7fd504ea4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd5001123a0 con 0x7fd500107df0 2024-07-31T14:41:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.754+0000 7fd504ea4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd500112220 con 0x7fd500108740 2024-07-31T14:41:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.754+0000 7fd504ea4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd500112520 con 0x7fd5001024b0 2024-07-31T14:41:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.754+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd500107df0 0x7fd5001990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.754+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd500107df0 0x7fd5001990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36350/0 (socket says 172.21.15.67:36350) 2024-07-31T14:41:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.754+0000 7fd4f5d9b640 1 -- 172.21.15.67:0/1198716659 learned_addr learned my addr 172.21.15.67:0/1198716659 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:41:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.754+0000 7fd4fe59c640 1 --2- 172.21.15.67:0/1198716659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5001024b0 0x7fd500198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.755+0000 7fd4fed9d640 1 --2- 172.21.15.67:0/1198716659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd500108740 0x7fd5001995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.755+0000 7fd4f5d9b640 1 -- 172.21.15.67:0/1198716659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5001024b0 msgr2=0x7fd500198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.755+0000 7fd4f5d9b640 1 --2- 172.21.15.67:0/1198716659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5001024b0 0x7fd500198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.755+0000 7fd4f5d9b640 1 -- 172.21.15.67:0/1198716659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd500108740 msgr2=0x7fd5001995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.755+0000 7fd4f5d9b640 1 --2- 172.21.15.67:0/1198716659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd500108740 0x7fd5001995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.755+0000 7fd4f5d9b640 1 -- 172.21.15.67:0/1198716659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5001a8210 con 0x7fd500107df0 2024-07-31T14:41:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.755+0000 7fd4f5d9b640 1 --2- 172.21.15.67:0/1198716659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd500107df0 0x7fd5001990a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800c970 tx=0x7fd4e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:06.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.755+0000 7fd4f7fff640 1 -- 172.21.15.67:0/1198716659 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8007bb0 con 0x7fd500107df0 2024-07-31T14:41:06.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.755+0000 7fd4f7fff640 1 -- 172.21.15.67:0/1198716659 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e8007d50 con 0x7fd500107df0 2024-07-31T14:41:06.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.755+0000 7fd504ea4640 1 -- 172.21.15.67:0/1198716659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5001a8500 con 0x7fd500107df0 2024-07-31T14:41:06.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.755+0000 7fd4f7fff640 1 -- 172.21.15.67:0/1198716659 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800f450 con 0x7fd500107df0 2024-07-31T14:41:06.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.755+0000 7fd504ea4640 1 -- 172.21.15.67:0/1198716659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5001a8a90 con 0x7fd500107df0 2024-07-31T14:41:06.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.756+0000 7fd504ea4640 1 -- 172.21.15.67:0/1198716659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd500103730 con 0x7fd500107df0 2024-07-31T14:41:06.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.758+0000 7fd4f7fff640 1 -- 172.21.15.67:0/1198716659 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fd4e8004f20 con 0x7fd500107df0 2024-07-31T14:41:06.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.759+0000 7fd4f7fff640 1 --2- 172.21.15.67:0/1198716659 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd4ec0b3c10 0x7fd4ec0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:06.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.759+0000 7fd4fe59c640 1 --2- 172.21.15.67:0/1198716659 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd4ec0b3c10 0x7fd4ec0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:06.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.759+0000 7fd4f7fff640 1 -- 172.21.15.67:0/1198716659 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd4e80e1f80 con 0x7fd500107df0 2024-07-31T14:41:06.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.759+0000 7fd4fe59c640 1 --2- 172.21.15.67:0/1198716659 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd4ec0b3c10 0x7fd4ec0b6120 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0000c00 tx=0x7fd4f0031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:06.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.762+0000 7fd4f7fff640 1 -- 172.21.15.67:0/1198716659 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd4e8014030 con 0x7fd500107df0 2024-07-31T14:41:06.832 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:06 smithi067 ceph-mon[32750]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:06.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:06 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/4170542371' entity='client.admin' 2024-07-31T14:41:06.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:06 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/70826174' entity='client.admin' 2024-07-31T14:41:06.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:06.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:06.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:41:06.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:06.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:06 smithi067 ceph-mon[32750]: from='client.? ' entity='client.admin' 2024-07-31T14:41:06.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:06 smithi073 ceph-mon[36714]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:06 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/4170542371' entity='client.admin' 2024-07-31T14:41:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:06 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/70826174' entity='client.admin' 2024-07-31T14:41:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:41:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:06 smithi073 ceph-mon[36714]: from='client.? ' entity='client.admin' 2024-07-31T14:41:06.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.957+0000 7fd504ea4640 1 -- 172.21.15.67:0/1198716659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fd5001028c0 con 0x7fd500107df0 2024-07-31T14:41:06.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.961+0000 7fd4f7fff640 1 -- 172.21.15.67:0/1198716659 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7fd4e80aac00 con 0x7fd500107df0 2024-07-31T14:41:06.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.963+0000 7fd504ea4640 1 -- 172.21.15.67:0/1198716659 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd4ec0b3c10 msgr2=0x7fd4ec0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:06.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.963+0000 7fd504ea4640 1 --2- 172.21.15.67:0/1198716659 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd4ec0b3c10 0x7fd4ec0b6120 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0000c00 tx=0x7fd4f0031400 comp rx=0 tx=0).stop 2024-07-31T14:41:06.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.963+0000 7fd504ea4640 1 -- 172.21.15.67:0/1198716659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd500107df0 msgr2=0x7fd5001990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:06.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.963+0000 7fd504ea4640 1 --2- 172.21.15.67:0/1198716659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd500107df0 0x7fd5001990a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800c970 tx=0x7fd4e800ce30 comp rx=0 tx=0).stop 2024-07-31T14:41:06.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.964+0000 7fd504ea4640 1 -- 172.21.15.67:0/1198716659 shutdown_connections 2024-07-31T14:41:06.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.964+0000 7fd504ea4640 1 --2- 172.21.15.67:0/1198716659 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd4ec0b3c10 0x7fd4ec0b6120 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.964+0000 7fd504ea4640 1 --2- 172.21.15.67:0/1198716659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd500108740 0x7fd5001995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.964+0000 7fd504ea4640 1 --2- 172.21.15.67:0/1198716659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd500107df0 0x7fd5001990a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.964+0000 7fd504ea4640 1 --2- 172.21.15.67:0/1198716659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5001024b0 0x7fd500198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:06.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.964+0000 7fd504ea4640 1 -- 172.21.15.67:0/1198716659 >> 172.21.15.67:0/1198716659 conn(0x7fd500079170 msgr2=0x7fd500100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:06.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.964+0000 7fd504ea4640 1 -- 172.21.15.67:0/1198716659 shutdown_connections 2024-07-31T14:41:06.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:06.964+0000 7fd504ea4640 1 -- 172.21.15.67:0/1198716659 wait complete. 2024-07-31T14:41:06.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:06 smithi077 ceph-mon[37327]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:06.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:06 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/4170542371' entity='client.admin' 2024-07-31T14:41:06.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:06 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/70826174' entity='client.admin' 2024-07-31T14:41:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:41:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:06 smithi077 ceph-mon[37327]: from='client.? ' entity='client.admin' 2024-07-31T14:41:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:07 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1198716659' entity='client.admin' 2024-07-31T14:41:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:07 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:07 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:07 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:41:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:07 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:08.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:07 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1198716659' entity='client.admin' 2024-07-31T14:41:08.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:07 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:08.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:07 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:08.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:07 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:41:08.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:07 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:08.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:07 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1198716659' entity='client.admin' 2024-07-31T14:41:08.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:07 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:08.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:07 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:08.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:07 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:41:08.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:07 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:09.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:08 smithi067 ceph-mon[32750]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:09.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:08 smithi077 ceph-mon[37327]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:09.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:08 smithi073 ceph-mon[36714]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:09.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.491+0000 7f7f19e9a640 1 -- 172.21.15.67:0/528896983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f141035e0 msgr2=0x7f7f141039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:09.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.491+0000 7f7f19e9a640 1 --2- 172.21.15.67:0/528896983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f141035e0 0x7f7f141039f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009900 tx=0x7f7f0402a410 comp rx=0 tx=0).stop 2024-07-31T14:41:09.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.491+0000 7f7f19e9a640 1 -- 172.21.15.67:0/528896983 shutdown_connections 2024-07-31T14:41:09.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.491+0000 7f7f19e9a640 1 --2- 172.21.15.67:0/528896983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f14109870 0x7f7f141100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.491+0000 7f7f19e9a640 1 --2- 172.21.15.67:0/528896983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f14108f20 0x7f7f14109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.491+0000 7f7f19e9a640 1 --2- 172.21.15.67:0/528896983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f141035e0 0x7f7f141039f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.491+0000 7f7f19e9a640 1 -- 172.21.15.67:0/528896983 >> 172.21.15.67:0/528896983 conn(0x7f7f140ff030 msgr2=0x7f7f14101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:09.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.491+0000 7f7f19e9a640 1 -- 172.21.15.67:0/528896983 shutdown_connections 2024-07-31T14:41:09.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.491+0000 7f7f19e9a640 1 -- 172.21.15.67:0/528896983 wait complete. 2024-07-31T14:41:09.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.492+0000 7f7f19e9a640 1 Processor -- start 2024-07-31T14:41:09.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.492+0000 7f7f19e9a640 1 -- start start 2024-07-31T14:41:09.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f19e9a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f141035e0 0x7f7f14199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:09.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f19e9a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f14108f20 0x7f7f1419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:09.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f19e9a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f14109870 0x7f7f1419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:09.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f19e9a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7f14113590 con 0x7f7f14109870 2024-07-31T14:41:09.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f19e9a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7f14113410 con 0x7f7f14108f20 2024-07-31T14:41:09.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f19e9a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f14113710 con 0x7f7f141035e0 2024-07-31T14:41:09.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f13fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f14109870 0x7f7f1419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:09.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f13fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f14109870 0x7f7f1419f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36362/0 (socket says 172.21.15.67:36362) 2024-07-31T14:41:09.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f13fff640 1 -- 172.21.15.67:0/4231948500 learned_addr learned my addr 172.21.15.67:0/4231948500 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:41:09.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f12ffd640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f14108f20 0x7f7f1419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:09.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f137fe640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f141035e0 0x7f7f14199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:09.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f13fff640 1 -- 172.21.15.67:0/4231948500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f141035e0 msgr2=0x7f7f14199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:09.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f13fff640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f141035e0 0x7f7f14199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f13fff640 1 -- 172.21.15.67:0/4231948500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f14108f20 msgr2=0x7f7f1419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:09.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f13fff640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f14108f20 0x7f7f1419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f13fff640 1 -- 172.21.15.67:0/4231948500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1419fef0 con 0x7f7f14109870 2024-07-31T14:41:09.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f137fe640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f141035e0 0x7f7f14199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:41:09.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.493+0000 7f7f13fff640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f14109870 0x7f7f1419f7c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00eb10 tx=0x7f7f0c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:09.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.494+0000 7f7f10ff9640 1 -- 172.21.15.67:0/4231948500 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c004b20 con 0x7f7f14109870 2024-07-31T14:41:09.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.494+0000 7f7f19e9a640 1 -- 172.21.15.67:0/4231948500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f141a01e0 con 0x7f7f14109870 2024-07-31T14:41:09.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.494+0000 7f7f10ff9640 1 -- 172.21.15.67:0/4231948500 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f0c004cc0 con 0x7f7f14109870 2024-07-31T14:41:09.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.494+0000 7f7f10ff9640 1 -- 172.21.15.67:0/4231948500 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c010670 con 0x7f7f14109870 2024-07-31T14:41:09.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.494+0000 7f7f19e9a640 1 -- 172.21.15.67:0/4231948500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f141a6940 con 0x7f7f14109870 2024-07-31T14:41:09.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.495+0000 7f7f19e9a640 1 -- 172.21.15.67:0/4231948500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f141039f0 con 0x7f7f14109870 2024-07-31T14:41:09.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.497+0000 7f7f10ff9640 1 -- 172.21.15.67:0/4231948500 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f7f0c005000 con 0x7f7f14109870 2024-07-31T14:41:09.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.498+0000 7f7f10ff9640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7efc0b3b90 0x7f7efc0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:09.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.498+0000 7f7f137fe640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7efc0b3b90 0x7f7efc0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:09.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.498+0000 7f7f10ff9640 1 -- 172.21.15.67:0/4231948500 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7f0c0e2850 con 0x7f7f14109870 2024-07-31T14:41:09.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.499+0000 7f7f137fe640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7efc0b3b90 0x7f7efc0b60a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7f04000c00 tx=0x7f7f04031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:09.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.501+0000 7f7f10ff9640 1 -- 172.21.15.67:0/4231948500 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f7f0c0555e0 con 0x7f7f14109870 2024-07-31T14:41:09.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.706+0000 7f7f19e9a640 1 -- 172.21.15.67:0/4231948500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f7f1419b0b0 con 0x7f7f14109870 2024-07-31T14:41:09.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.710+0000 7f7f10ff9640 1 -- 172.21.15.67:0/4231948500 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v32)=0 v32) ==== 127+0+0 (secure 0 0 0) 0x7f7f1419b0b0 con 0x7f7f14109870 2024-07-31T14:41:09.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.713+0000 7f7f19e9a640 1 -- 172.21.15.67:0/4231948500 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7efc0b3b90 msgr2=0x7f7efc0b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:09.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.713+0000 7f7f19e9a640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7efc0b3b90 0x7f7efc0b60a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7f04000c00 tx=0x7f7f04031400 comp rx=0 tx=0).stop 2024-07-31T14:41:09.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.713+0000 7f7f19e9a640 1 -- 172.21.15.67:0/4231948500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f14109870 msgr2=0x7f7f1419f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:09.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.713+0000 7f7f19e9a640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f14109870 0x7f7f1419f7c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00eb10 tx=0x7f7f0c00c2d0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.713+0000 7f7f19e9a640 1 -- 172.21.15.67:0/4231948500 shutdown_connections 2024-07-31T14:41:09.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.713+0000 7f7f19e9a640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7efc0b3b90 0x7f7efc0b60a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.713+0000 7f7f19e9a640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f14109870 0x7f7f1419f7c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.713+0000 7f7f19e9a640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f14108f20 0x7f7f1419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.713+0000 7f7f19e9a640 1 --2- 172.21.15.67:0/4231948500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f141035e0 0x7f7f14199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.713+0000 7f7f19e9a640 1 -- 172.21.15.67:0/4231948500 >> 172.21.15.67:0/4231948500 conn(0x7f7f140ff030 msgr2=0x7f7f14101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:09.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.713+0000 7f7f19e9a640 1 -- 172.21.15.67:0/4231948500 shutdown_connections 2024-07-31T14:41:09.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.713+0000 7f7f19e9a640 1 -- 172.21.15.67:0/4231948500 wait complete. 2024-07-31T14:41:09.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.800+0000 7ff5a074f640 1 -- 172.21.15.67:0/417058686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff598103590 msgr2=0x7ff5981039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:09.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.800+0000 7ff5a074f640 1 --2- 172.21.15.67:0/417058686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff598103590 0x7ff5981039a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff594009900 tx=0x7ff59402a410 comp rx=0 tx=0).stop 2024-07-31T14:41:09.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.801+0000 7ff5a074f640 1 -- 172.21.15.67:0/417058686 shutdown_connections 2024-07-31T14:41:09.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.801+0000 7ff5a074f640 1 --2- 172.21.15.67:0/417058686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff598109820 0x7ff598110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.801+0000 7ff5a074f640 1 --2- 172.21.15.67:0/417058686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff598108ed0 0x7ff5981092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.801+0000 7ff5a074f640 1 --2- 172.21.15.67:0/417058686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff598103590 0x7ff5981039a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.801+0000 7ff5a074f640 1 -- 172.21.15.67:0/417058686 >> 172.21.15.67:0/417058686 conn(0x7ff59807b280 msgr2=0x7ff59807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:09.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.801+0000 7ff5a074f640 1 -- 172.21.15.67:0/417058686 shutdown_connections 2024-07-31T14:41:09.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.801+0000 7ff5a074f640 1 -- 172.21.15.67:0/417058686 wait complete. 2024-07-31T14:41:09.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.801+0000 7ff5a074f640 1 Processor -- start 2024-07-31T14:41:09.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff5a074f640 1 -- start start 2024-07-31T14:41:09.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff5a074f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff598103590 0x7ff598199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:09.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff5a074f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff598108ed0 0x7ff59819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:09.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff5a074f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff598109820 0x7ff59819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:09.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff5a074f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff5981134f0 con 0x7ff598108ed0 2024-07-31T14:41:09.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff5a074f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff598113370 con 0x7ff598103590 2024-07-31T14:41:09.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff5a074f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff598113670 con 0x7ff598109820 2024-07-31T14:41:09.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff59dcea640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff598108ed0 0x7ff59819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:09.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff59dcea640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff598108ed0 0x7ff59819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36370/0 (socket says 172.21.15.67:36370) 2024-07-31T14:41:09.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff59dcea640 1 -- 172.21.15.67:0/18939290 learned_addr learned my addr 172.21.15.67:0/18939290 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:41:09.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff59e4eb640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff598103590 0x7ff598199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:09.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff59ecec640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff598109820 0x7ff59819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:09.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff59dcea640 1 -- 172.21.15.67:0/18939290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff598109820 msgr2=0x7ff59819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:09.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff59dcea640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff598109820 0x7ff59819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff59dcea640 1 -- 172.21.15.67:0/18939290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff598103590 msgr2=0x7ff598199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:09.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.802+0000 7ff59dcea640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff598103590 0x7ff598199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:09.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.803+0000 7ff59dcea640 1 -- 172.21.15.67:0/18939290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59819feb0 con 0x7ff598108ed0 2024-07-31T14:41:09.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.803+0000 7ff59e4eb640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff598103590 0x7ff598199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:41:09.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.803+0000 7ff59dcea640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff598108ed0 0x7ff59819a4e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff58800d740 tx=0x7ff58800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:09.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.803+0000 7ff58f7fe640 1 -- 172.21.15.67:0/18939290 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588017070 con 0x7ff598108ed0 2024-07-31T14:41:09.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.803+0000 7ff58f7fe640 1 -- 172.21.15.67:0/18939290 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff58800de50 con 0x7ff598108ed0 2024-07-31T14:41:09.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.803+0000 7ff58f7fe640 1 -- 172.21.15.67:0/18939290 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58800f620 con 0x7ff598108ed0 2024-07-31T14:41:09.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.803+0000 7ff5a074f640 1 -- 172.21.15.67:0/18939290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5981a01a0 con 0x7ff598108ed0 2024-07-31T14:41:09.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.803+0000 7ff5a074f640 1 -- 172.21.15.67:0/18939290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5981a6900 con 0x7ff598108ed0 2024-07-31T14:41:09.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.804+0000 7ff5a074f640 1 -- 172.21.15.67:0/18939290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5981039a0 con 0x7ff598108ed0 2024-07-31T14:41:09.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.806+0000 7ff58f7fe640 1 -- 172.21.15.67:0/18939290 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7ff58801a070 con 0x7ff598108ed0 2024-07-31T14:41:09.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.807+0000 7ff58f7fe640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5840b3c10 0x7ff5840b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:09.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.807+0000 7ff59e4eb640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5840b3c10 0x7ff5840b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:09.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.807+0000 7ff58f7fe640 1 -- 172.21.15.67:0/18939290 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff5880e18c0 con 0x7ff598108ed0 2024-07-31T14:41:09.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.807+0000 7ff59e4eb640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5840b3c10 0x7ff5840b6120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff594031440 tx=0x7ff594032ed0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:09.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:09.810+0000 7ff58f7fe640 1 -- 172.21.15.67:0/18939290 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff5880aa520 con 0x7ff598108ed0 2024-07-31T14:41:10.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.006+0000 7ff5a074f640 1 -- 172.21.15.67:0/18939290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7ff59810b5a0 con 0x7ff598108ed0 2024-07-31T14:41:10.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.012+0000 7ff58f7fe640 1 -- 172.21.15.67:0/18939290 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v33)=0 v33) ==== 121+0+0 (secure 0 0 0) 0x7ff5880aea70 con 0x7ff598108ed0 2024-07-31T14:41:10.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.015+0000 7ff5a074f640 1 -- 172.21.15.67:0/18939290 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5840b3c10 msgr2=0x7ff5840b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:10.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.015+0000 7ff5a074f640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5840b3c10 0x7ff5840b6120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff594031440 tx=0x7ff594032ed0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.015+0000 7ff5a074f640 1 -- 172.21.15.67:0/18939290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff598108ed0 msgr2=0x7ff59819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:10.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.015+0000 7ff5a074f640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff598108ed0 0x7ff59819a4e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff58800d740 tx=0x7ff58800dc00 comp rx=0 tx=0).stop 2024-07-31T14:41:10.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.015+0000 7ff5a074f640 1 -- 172.21.15.67:0/18939290 shutdown_connections 2024-07-31T14:41:10.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.015+0000 7ff5a074f640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5840b3c10 0x7ff5840b6120 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.015+0000 7ff5a074f640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff598109820 0x7ff59819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.015+0000 7ff5a074f640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff598108ed0 0x7ff59819a4e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.015+0000 7ff5a074f640 1 --2- 172.21.15.67:0/18939290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff598103590 0x7ff598199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.015+0000 7ff5a074f640 1 -- 172.21.15.67:0/18939290 >> 172.21.15.67:0/18939290 conn(0x7ff59807b280 msgr2=0x7ff598107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:10.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.015+0000 7ff5a074f640 1 -- 172.21.15.67:0/18939290 shutdown_connections 2024-07-31T14:41:10.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.015+0000 7ff5a074f640 1 -- 172.21.15.67:0/18939290 wait complete. 2024-07-31T14:41:10.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.603+0000 7fa23d85d640 1 -- 172.21.15.67:0/216028898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa238103590 msgr2=0x7fa2381039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:10.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.603+0000 7fa23d85d640 1 --2- 172.21.15.67:0/216028898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa238103590 0x7fa2381039a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa228009900 tx=0x7fa22802a410 comp rx=0 tx=0).stop 2024-07-31T14:41:10.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.603+0000 7fa23d85d640 1 -- 172.21.15.67:0/216028898 shutdown_connections 2024-07-31T14:41:10.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.603+0000 7fa23d85d640 1 --2- 172.21.15.67:0/216028898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa238109820 0x7fa238110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.603+0000 7fa23d85d640 1 --2- 172.21.15.67:0/216028898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa238108ed0 0x7fa2381092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.603+0000 7fa23d85d640 1 --2- 172.21.15.67:0/216028898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa238103590 0x7fa2381039a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.603+0000 7fa23d85d640 1 -- 172.21.15.67:0/216028898 >> 172.21.15.67:0/216028898 conn(0x7fa23807b280 msgr2=0x7fa23807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:10.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.604+0000 7fa23d85d640 1 -- 172.21.15.67:0/216028898 shutdown_connections 2024-07-31T14:41:10.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.604+0000 7fa23d85d640 1 -- 172.21.15.67:0/216028898 wait complete. 2024-07-31T14:41:10.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.604+0000 7fa23d85d640 1 Processor -- start 2024-07-31T14:41:10.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.605+0000 7fa23d85d640 1 -- start start 2024-07-31T14:41:10.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.605+0000 7fa23d85d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa238103590 0x7fa238199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:10.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.605+0000 7fa23d85d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa238108ed0 0x7fa23819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:10.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.605+0000 7fa23d85d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa238109820 0x7fa23819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:10.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.605+0000 7fa23d85d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa2381134f0 con 0x7fa238108ed0 2024-07-31T14:41:10.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.605+0000 7fa23d85d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa238113370 con 0x7fa238103590 2024-07-31T14:41:10.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.605+0000 7fa23d85d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa238113670 con 0x7fa238109820 2024-07-31T14:41:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.605+0000 7fa22e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa238108ed0 0x7fa23819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.605+0000 7fa22e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa238108ed0 0x7fa23819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36396/0 (socket says 172.21.15.67:36396) 2024-07-31T14:41:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.605+0000 7fa236ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa238103590 0x7fa238199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.605+0000 7fa22e7fc640 1 -- 172.21.15.67:0/593306945 learned_addr learned my addr 172.21.15.67:0/593306945 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:41:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.605+0000 7fa2377fe640 1 --2- 172.21.15.67:0/593306945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa238109820 0x7fa23819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.606+0000 7fa22e7fc640 1 -- 172.21.15.67:0/593306945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa238109820 msgr2=0x7fa23819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.606+0000 7fa22e7fc640 1 --2- 172.21.15.67:0/593306945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa238109820 0x7fa23819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.606+0000 7fa22e7fc640 1 -- 172.21.15.67:0/593306945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa238103590 msgr2=0x7fa238199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.606+0000 7fa22e7fc640 1 --2- 172.21.15.67:0/593306945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa238103590 0x7fa238199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.606+0000 7fa22e7fc640 1 -- 172.21.15.67:0/593306945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23819feb0 con 0x7fa238108ed0 2024-07-31T14:41:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.606+0000 7fa236ffd640 1 --2- 172.21.15.67:0/593306945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa238103590 0x7fa238199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:41:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.606+0000 7fa22e7fc640 1 --2- 172.21.15.67:0/593306945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa238108ed0 0x7fa23819a4e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa22000e9f0 tx=0x7fa22000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.606+0000 7fa23c85b640 1 -- 172.21.15.67:0/593306945 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22000ccc0 con 0x7fa238108ed0 2024-07-31T14:41:10.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.606+0000 7fa23d85d640 1 -- 172.21.15.67:0/593306945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2381a01a0 con 0x7fa238108ed0 2024-07-31T14:41:10.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.606+0000 7fa23c85b640 1 -- 172.21.15.67:0/593306945 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2200108e0 con 0x7fa238108ed0 2024-07-31T14:41:10.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.606+0000 7fa23c85b640 1 -- 172.21.15.67:0/593306945 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220018860 con 0x7fa238108ed0 2024-07-31T14:41:10.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.606+0000 7fa23d85d640 1 -- 172.21.15.67:0/593306945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2381a6900 con 0x7fa238108ed0 2024-07-31T14:41:10.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.607+0000 7fa23d85d640 1 -- 172.21.15.67:0/593306945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2381039a0 con 0x7fa238108ed0 2024-07-31T14:41:10.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.609+0000 7fa23c85b640 1 -- 172.21.15.67:0/593306945 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fa220010430 con 0x7fa238108ed0 2024-07-31T14:41:10.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.610+0000 7fa23c85b640 1 --2- 172.21.15.67:0/593306945 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2240b3b90 0x7fa2240b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:10.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.610+0000 7fa236ffd640 1 --2- 172.21.15.67:0/593306945 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2240b3b90 0x7fa2240b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:10.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.610+0000 7fa23c85b640 1 -- 172.21.15.67:0/593306945 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa2200e2850 con 0x7fa238108ed0 2024-07-31T14:41:10.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.610+0000 7fa236ffd640 1 --2- 172.21.15.67:0/593306945 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2240b3b90 0x7fa2240b60a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa228031440 tx=0x7fa228032ed0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:10.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.613+0000 7fa23c85b640 1 -- 172.21.15.67:0/593306945 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fa2200abaa0 con 0x7fa238108ed0 2024-07-31T14:41:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.810+0000 7fa23d85d640 1 -- 172.21.15.67:0/593306945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7fa23810b600 con 0x7fa238108ed0 2024-07-31T14:41:10.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.816+0000 7fa23c85b640 1 -- 172.21.15.67:0/593306945 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v34)=0 v34) ==== 116+0+0 (secure 0 0 0) 0x7fa2200ab3a0 con 0x7fa238108ed0 2024-07-31T14:41:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.818+0000 7fa23d85d640 1 -- 172.21.15.67:0/593306945 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2240b3b90 msgr2=0x7fa2240b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.818+0000 7fa23d85d640 1 --2- 172.21.15.67:0/593306945 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2240b3b90 0x7fa2240b60a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa228031440 tx=0x7fa228032ed0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.818+0000 7fa23d85d640 1 -- 172.21.15.67:0/593306945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa238108ed0 msgr2=0x7fa23819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.818+0000 7fa23d85d640 1 --2- 172.21.15.67:0/593306945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa238108ed0 0x7fa23819a4e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa22000e9f0 tx=0x7fa22000eeb0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.818+0000 7fa23d85d640 1 -- 172.21.15.67:0/593306945 shutdown_connections 2024-07-31T14:41:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.818+0000 7fa23d85d640 1 --2- 172.21.15.67:0/593306945 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2240b3b90 0x7fa2240b60a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.818+0000 7fa23d85d640 1 --2- 172.21.15.67:0/593306945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa238109820 0x7fa23819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.818+0000 7fa23d85d640 1 --2- 172.21.15.67:0/593306945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa238108ed0 0x7fa23819a4e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.818+0000 7fa23d85d640 1 --2- 172.21.15.67:0/593306945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa238103590 0x7fa238199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.818+0000 7fa23d85d640 1 -- 172.21.15.67:0/593306945 >> 172.21.15.67:0/593306945 conn(0x7fa23807b280 msgr2=0x7fa238107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.818+0000 7fa23d85d640 1 -- 172.21.15.67:0/593306945 shutdown_connections 2024-07-31T14:41:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:10.818+0000 7fa23d85d640 1 -- 172.21.15.67:0/593306945 wait complete. 2024-07-31T14:41:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:10 smithi077 ceph-mon[37327]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:10 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/4231948500' entity='client.admin' 2024-07-31T14:41:10.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:10 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/18939290' entity='client.admin' 2024-07-31T14:41:11.028 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:10 smithi067 ceph-mon[32750]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:11.028 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:10 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/4231948500' entity='client.admin' 2024-07-31T14:41:11.028 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:10 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/18939290' entity='client.admin' 2024-07-31T14:41:11.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:10 smithi073 ceph-mon[36714]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:11.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:10 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/4231948500' entity='client.admin' 2024-07-31T14:41:11.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:10 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/18939290' entity='client.admin' 2024-07-31T14:41:12.028 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:11 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/593306945' entity='client.admin' 2024-07-31T14:41:12.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:11 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/593306945' entity='client.admin' 2024-07-31T14:41:12.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:11 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/593306945' entity='client.admin' 2024-07-31T14:41:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.259+0000 7f331b6ae640 1 -- 172.21.15.67:0/2394648289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314108720 msgr2=0x7f331410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.259+0000 7f331b6ae640 1 --2- 172.21.15.67:0/2394648289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314108720 0x7f331410ef50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f330c009ca0 tx=0x7f330c02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:41:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.259+0000 7f331b6ae640 1 -- 172.21.15.67:0/2394648289 shutdown_connections 2024-07-31T14:41:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.259+0000 7f331b6ae640 1 --2- 172.21.15.67:0/2394648289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314108720 0x7f331410ef50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.259+0000 7f331b6ae640 1 --2- 172.21.15.67:0/2394648289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3314107dd0 0x7f33141081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.259+0000 7f331b6ae640 1 --2- 172.21.15.67:0/2394648289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3314102490 0x7f33141028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.259+0000 7f331b6ae640 1 -- 172.21.15.67:0/2394648289 >> 172.21.15.67:0/2394648289 conn(0x7f3314079160 msgr2=0x7f3314079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.259+0000 7f331b6ae640 1 -- 172.21.15.67:0/2394648289 shutdown_connections 2024-07-31T14:41:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.259+0000 7f331b6ae640 1 -- 172.21.15.67:0/2394648289 wait complete. 2024-07-31T14:41:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.260+0000 7f331b6ae640 1 Processor -- start 2024-07-31T14:41:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.260+0000 7f331b6ae640 1 -- start start 2024-07-31T14:41:12.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.260+0000 7f331b6ae640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314102490 0x7f331419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:12.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.260+0000 7f331b6ae640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3314107dd0 0x7f331419b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:12.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.260+0000 7f331b6ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3314108720 0x7f331419b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:12.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.260+0000 7f331b6ae640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3314112380 con 0x7f3314102490 2024-07-31T14:41:12.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.260+0000 7f331b6ae640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3314112200 con 0x7f3314107dd0 2024-07-31T14:41:12.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.260+0000 7f331b6ae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3314112500 con 0x7f3314108720 2024-07-31T14:41:12.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.260+0000 7f331944a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314102490 0x7f331419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:12.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.260+0000 7f331944a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314102490 0x7f331419ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41590/0 (socket says 172.21.15.67:41590) 2024-07-31T14:41:12.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.260+0000 7f3319c4b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3314108720 0x7f331419b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:12.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.260+0000 7f331944a640 1 -- 172.21.15.67:0/3526106955 learned_addr learned my addr 172.21.15.67:0/3526106955 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:41:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.260+0000 7f3318c49640 1 --2- 172.21.15.67:0/3526106955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3314107dd0 0x7f331419b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.261+0000 7f3319c4b640 1 -- 172.21.15.67:0/3526106955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3314107dd0 msgr2=0x7f331419b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.261+0000 7f3319c4b640 1 --2- 172.21.15.67:0/3526106955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3314107dd0 0x7f331419b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.261+0000 7f3319c4b640 1 -- 172.21.15.67:0/3526106955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314102490 msgr2=0x7f331419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.261+0000 7f3319c4b640 1 --2- 172.21.15.67:0/3526106955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314102490 0x7f331419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.261+0000 7f3319c4b640 1 -- 172.21.15.67:0/3526106955 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33141a5ef0 con 0x7f3314108720 2024-07-31T14:41:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.261+0000 7f3319c4b640 1 --2- 172.21.15.67:0/3526106955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3314108720 0x7f331419b6b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f330c02ade0 tx=0x7f330c031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.261+0000 7f330a7fc640 1 -- 172.21.15.67:0/3526106955 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c003b70 con 0x7f3314108720 2024-07-31T14:41:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.261+0000 7f330a7fc640 1 -- 172.21.15.67:0/3526106955 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f330c031910 con 0x7f3314108720 2024-07-31T14:41:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.261+0000 7f331b6ae640 1 -- 172.21.15.67:0/3526106955 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33141a6180 con 0x7f3314108720 2024-07-31T14:41:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.261+0000 7f330a7fc640 1 -- 172.21.15.67:0/3526106955 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c02f950 con 0x7f3314108720 2024-07-31T14:41:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.261+0000 7f331b6ae640 1 -- 172.21.15.67:0/3526106955 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33141a66c0 con 0x7f3314108720 2024-07-31T14:41:12.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.262+0000 7f331b6ae640 1 -- 172.21.15.67:0/3526106955 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33141028a0 con 0x7f3314108720 2024-07-31T14:41:12.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.264+0000 7f330a7fc640 1 -- 172.21.15.67:0/3526106955 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f330c02faf0 con 0x7f3314108720 2024-07-31T14:41:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.265+0000 7f330a7fc640 1 --2- 172.21.15.67:0/3526106955 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33000b3b90 0x7f33000b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.265+0000 7f331944a640 1 --2- 172.21.15.67:0/3526106955 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33000b3b90 0x7f33000b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.265+0000 7f330a7fc640 1 -- 172.21.15.67:0/3526106955 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f330c103510 con 0x7f3314108720 2024-07-31T14:41:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.266+0000 7f331944a640 1 --2- 172.21.15.67:0/3526106955 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33000b3b90 0x7f33000b60a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f331406b430 tx=0x7f3310005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:12.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.268+0000 7f330a7fc640 1 -- 172.21.15.67:0/3526106955 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f330c0cc6c0 con 0x7f3314108720 2024-07-31T14:41:12.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.466+0000 7f331b6ae640 1 -- 172.21.15.67:0/3526106955 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f331406ace0 con 0x7f3314108720 2024-07-31T14:41:12.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.472+0000 7f330a7fc640 1 -- 172.21.15.67:0/3526106955 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v35)=0 v35) ==== 120+0+0 (secure 0 0 0) 0x7f330c003d10 con 0x7f3314108720 2024-07-31T14:41:12.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.475+0000 7f331b6ae640 1 -- 172.21.15.67:0/3526106955 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33000b3b90 msgr2=0x7f33000b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:12.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.475+0000 7f331b6ae640 1 --2- 172.21.15.67:0/3526106955 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33000b3b90 0x7f33000b60a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f331406b430 tx=0x7f3310005f90 comp rx=0 tx=0).stop 2024-07-31T14:41:12.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.475+0000 7f331b6ae640 1 -- 172.21.15.67:0/3526106955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3314108720 msgr2=0x7f331419b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:12.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.475+0000 7f331b6ae640 1 --2- 172.21.15.67:0/3526106955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3314108720 0x7f331419b6b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f330c02ade0 tx=0x7f330c031660 comp rx=0 tx=0).stop 2024-07-31T14:41:12.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.475+0000 7f331b6ae640 1 -- 172.21.15.67:0/3526106955 shutdown_connections 2024-07-31T14:41:12.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.475+0000 7f331b6ae640 1 --2- 172.21.15.67:0/3526106955 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33000b3b90 0x7f33000b60a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:12.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.475+0000 7f331b6ae640 1 --2- 172.21.15.67:0/3526106955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3314108720 0x7f331419b6b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:12.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.475+0000 7f331b6ae640 1 --2- 172.21.15.67:0/3526106955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3314107dd0 0x7f331419b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:12.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.475+0000 7f331b6ae640 1 --2- 172.21.15.67:0/3526106955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314102490 0x7f331419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:12.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.475+0000 7f331b6ae640 1 -- 172.21.15.67:0/3526106955 >> 172.21.15.67:0/3526106955 conn(0x7f3314079160 msgr2=0x7f3314104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:12.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.475+0000 7f331b6ae640 1 -- 172.21.15.67:0/3526106955 shutdown_connections 2024-07-31T14:41:12.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:12.475+0000 7f331b6ae640 1 -- 172.21.15.67:0/3526106955 wait complete. 2024-07-31T14:41:12.671 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi067') 2024-07-31T14:41:12.671 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi067') 2024-07-31T14:41:12.671 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi067') 2024-07-31T14:41:12.671 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi067') 2024-07-31T14:41:12.671 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi067') 2024-07-31T14:41:12.672 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi067') 2024-07-31T14:41:13.011 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:12 smithi067 ceph-mon[32750]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:13.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:12 smithi067 ceph-mon[32750]: from='client.? ' entity='client.admin' 2024-07-31T14:41:13.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:12 smithi073 ceph-mon[36714]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:13.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:12 smithi073 ceph-mon[36714]: from='client.? ' entity='client.admin' 2024-07-31T14:41:13.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:12 smithi077 ceph-mon[37327]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:13.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:12 smithi077 ceph-mon[37327]: from='client.? ' entity='client.admin' 2024-07-31T14:41:13.282 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi067') 2024-07-31T14:41:13.282 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi067') 2024-07-31T14:41:13.282 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-07-31T14:41:13.694 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-07-31T14:41:13.694 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi067') 2024-07-31T14:41:13.694 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi067') 2024-07-31T14:41:13.694 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi067') 2024-07-31T14:41:13.695 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi067') 2024-07-31T14:41:13.695 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi067') 2024-07-31T14:41:13.695 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi067') 2024-07-31T14:41:13.695 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5ca2b71e-4f4a-11ef-bcc9-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi067') 2024-07-31T14:41:13.695 INFO:tasks.cephadm:Deploying OSDs... 2024-07-31T14:41:13.696 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:41:13.696 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2024-07-31T14:41:13.713 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-31T14:41:13.713 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_1 2024-07-31T14:41:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-31T14:41:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-31T14:41:13.769 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-07-31T14:41:13.769 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-31T14:41:13.769 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-07-31T14:41:13.770 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-07-31 14:39:19.462710922 +0000 2024-07-31T14:41:13.770 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-07-31 14:34:15.817697490 +0000 2024-07-31T14:41:13.770 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-07-31 14:34:15.817697490 +0000 2024-07-31T14:41:13.770 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-07-31 14:34:15.817697490 +0000 2024-07-31T14:41:13.770 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-31T14:41:13.835 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-07-31T14:41:13.835 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-07-31T14:41:13.835 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.00021065 s, 2.4 MB/s 2024-07-31T14:41:13.836 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-31T14:41:13.894 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_2 2024-07-31T14:41:13.949 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-31T14:41:13.950 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-31T14:41:13.950 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-07-31T14:41:13.950 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-31T14:41:13.950 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-07-31T14:41:13.950 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-07-31 14:39:19.497710195 +0000 2024-07-31T14:41:13.950 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-07-31 14:34:16.181689841 +0000 2024-07-31T14:41:13.950 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-07-31 14:34:16.181689841 +0000 2024-07-31T14:41:13.950 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-07-31 14:34:16.181689841 +0000 2024-07-31T14:41:13.950 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-31T14:41:14.014 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-07-31T14:41:14.014 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-07-31T14:41:14.014 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000216021 s, 2.4 MB/s 2024-07-31T14:41:14.016 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-31T14:41:14.073 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_3 2024-07-31T14:41:14.128 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-31T14:41:14.129 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-31T14:41:14.129 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-07-31T14:41:14.129 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-31T14:41:14.129 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-07-31T14:41:14.129 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-07-31 14:39:19.538709344 +0000 2024-07-31T14:41:14.129 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-07-31 14:34:16.529682527 +0000 2024-07-31T14:41:14.129 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-07-31 14:34:16.529682527 +0000 2024-07-31T14:41:14.129 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-07-31 14:34:16.529682527 +0000 2024-07-31T14:41:14.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-31T14:41:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-07-31T14:41:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-07-31T14:41:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000197612 s, 2.6 MB/s 2024-07-31T14:41:14.195 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-31T14:41:14.254 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_4 2024-07-31T14:41:14.312 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-31T14:41:14.312 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-31T14:41:14.312 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-07-31T14:41:14.312 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-31T14:41:14.313 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-07-31T14:41:14.313 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-07-31 14:39:19.573708618 +0000 2024-07-31T14:41:14.313 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-07-31 14:34:16.863675506 +0000 2024-07-31T14:41:14.313 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-07-31 14:34:16.863675506 +0000 2024-07-31T14:41:14.313 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-07-31 14:34:16.863675506 +0000 2024-07-31T14:41:14.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-31T14:41:14.377 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-07-31T14:41:14.378 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-07-31T14:41:14.378 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000207705 s, 2.5 MB/s 2024-07-31T14:41:14.379 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-31T14:41:14.436 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-07-31T14:41:14.436 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/scratch_devs of=/dev/stdout 2024-07-31T14:41:14.455 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-31T14:41:14.455 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_1 2024-07-31T14:41:14.512 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-31T14:41:14.513 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-31T14:41:14.513 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-07-31T14:41:14.513 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-31T14:41:14.513 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-07-31T14:41:14.513 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-07-31 14:40:27.966371364 +0000 2024-07-31T14:41:14.513 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-07-31 14:34:31.007676548 +0000 2024-07-31T14:41:14.513 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-07-31 14:34:31.007676548 +0000 2024-07-31T14:41:14.513 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-07-31 14:34:31.007676548 +0000 2024-07-31T14:41:14.513 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-31T14:41:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-07-31T14:41:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-07-31T14:41:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000290384 s, 1.8 MB/s 2024-07-31T14:41:14.584 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-31T14:41:14.642 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_2 2024-07-31T14:41:14.699 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-31T14:41:14.699 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-31T14:41:14.699 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-07-31T14:41:14.699 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-31T14:41:14.699 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-07-31T14:41:14.700 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-07-31 14:40:27.966371364 +0000 2024-07-31T14:41:14.700 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-07-31 14:34:31.342668866 +0000 2024-07-31T14:41:14.700 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-07-31 14:34:31.342668866 +0000 2024-07-31T14:41:14.700 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-07-31 14:34:31.342668866 +0000 2024-07-31T14:41:14.700 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-31T14:41:14.766 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-07-31T14:41:14.766 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-07-31T14:41:14.766 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000283453 s, 1.8 MB/s 2024-07-31T14:41:14.768 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-31T14:41:14.826 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_3 2024-07-31T14:41:14.883 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-31T14:41:14.883 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-31T14:41:14.883 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-07-31T14:41:14.883 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-31T14:41:14.884 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-07-31T14:41:14.884 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-07-31 14:40:27.966371364 +0000 2024-07-31T14:41:14.884 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-07-31 14:34:31.678661161 +0000 2024-07-31T14:41:14.884 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-07-31 14:34:31.678661161 +0000 2024-07-31T14:41:14.884 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-07-31 14:34:31.678661161 +0000 2024-07-31T14:41:14.884 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-31T14:41:14.947 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-07-31T14:41:14.948 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-07-31T14:41:14.948 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000227934 s, 2.2 MB/s 2024-07-31T14:41:14.949 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-31T14:41:15.006 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_4 2024-07-31T14:41:15.065 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-31T14:41:15.065 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-31T14:41:15.065 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-07-31T14:41:15.065 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-31T14:41:15.065 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-07-31T14:41:15.065 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-07-31 14:40:27.967371342 +0000 2024-07-31T14:41:15.065 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-07-31 14:34:32.022653272 +0000 2024-07-31T14:41:15.065 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-07-31 14:34:32.022653272 +0000 2024-07-31T14:41:15.065 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-07-31 14:34:32.022653272 +0000 2024-07-31T14:41:15.066 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-31T14:41:15.126 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:14 smithi073 ceph-mon[36714]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:15.129 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-07-31T14:41:15.129 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-07-31T14:41:15.129 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000192361 s, 2.7 MB/s 2024-07-31T14:41:15.130 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-31T14:41:15.186 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-07-31T14:41:15.186 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2024-07-31T14:41:15.206 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-31T14:41:15.206 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_1 2024-07-31T14:41:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:14 smithi077 ceph-mon[37327]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:15.254 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-31T14:41:15.255 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-31T14:41:15.255 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-07-31T14:41:15.255 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-31T14:41:15.255 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-07-31T14:41:15.255 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-07-31 14:40:25.634626531 +0000 2024-07-31T14:41:15.255 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-07-31 14:34:26.474481499 +0000 2024-07-31T14:41:15.255 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-07-31 14:34:26.474481499 +0000 2024-07-31T14:41:15.255 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-07-31 14:34:26.474481499 +0000 2024-07-31T14:41:15.255 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-31T14:41:15.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:14 smithi067 ceph-mon[32750]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:15.323 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-07-31T14:41:15.323 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-07-31T14:41:15.323 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000293265 s, 1.7 MB/s 2024-07-31T14:41:15.324 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-31T14:41:15.382 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_2 2024-07-31T14:41:15.438 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-31T14:41:15.439 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-31T14:41:15.439 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-07-31T14:41:15.439 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-31T14:41:15.439 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-07-31T14:41:15.439 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-07-31 14:40:25.635626510 +0000 2024-07-31T14:41:15.439 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-07-31 14:34:26.802474383 +0000 2024-07-31T14:41:15.439 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-07-31 14:34:26.802474383 +0000 2024-07-31T14:41:15.439 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-07-31 14:34:26.802474383 +0000 2024-07-31T14:41:15.439 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-31T14:41:15.502 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-07-31T14:41:15.502 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-07-31T14:41:15.502 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000206886 s, 2.5 MB/s 2024-07-31T14:41:15.503 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-31T14:41:15.560 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_3 2024-07-31T14:41:15.615 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-31T14:41:15.616 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-31T14:41:15.616 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-07-31T14:41:15.616 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-31T14:41:15.616 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-07-31T14:41:15.616 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-07-31 14:40:25.635626510 +0000 2024-07-31T14:41:15.616 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-07-31 14:34:27.135467158 +0000 2024-07-31T14:41:15.616 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-07-31 14:34:27.135467158 +0000 2024-07-31T14:41:15.616 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-07-31 14:34:27.135467158 +0000 2024-07-31T14:41:15.616 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-31T14:41:15.678 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-07-31T14:41:15.678 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-07-31T14:41:15.678 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000202709 s, 2.5 MB/s 2024-07-31T14:41:15.679 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-31T14:41:15.735 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_4 2024-07-31T14:41:15.790 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-31T14:41:15.790 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-31T14:41:15.791 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-07-31T14:41:15.791 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-31T14:41:15.791 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-07-31T14:41:15.791 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-07-31 14:40:25.635626510 +0000 2024-07-31T14:41:15.791 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-07-31 14:34:27.475459782 +0000 2024-07-31T14:41:15.791 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-07-31 14:34:27.475459782 +0000 2024-07-31T14:41:15.791 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-07-31 14:34:27.475459782 +0000 2024-07-31T14:41:15.791 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-31T14:41:15.852 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-07-31T14:41:15.853 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-07-31T14:41:15.853 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000207282 s, 2.5 MB/s 2024-07-31T14:41:15.854 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-31T14:41:15.909 INFO:tasks.cephadm:Deploying osd.0 on smithi067 with /dev/vg_nvme/lv_4... 2024-07-31T14:41:15.910 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-31T14:41:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:41:17.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:16 smithi067 ceph-mon[32750]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:17.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:16 smithi073 ceph-mon[36714]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:17.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:16 smithi077 ceph-mon[37327]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-31T14:41:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-31T14:41:17.962 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-07-31T14:41:17.962 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-07-31T14:41:17.962 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113806 s, 921 MB/s 2024-07-31T14:41:17.962 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-07-31T14:41:18.680 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi067:vg_nvme/lv_4 2024-07-31T14:41:18.948 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:41:19.046 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:18 smithi067 ceph-mon[32750]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:19.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:18 smithi073 ceph-mon[36714]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:19.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:18 smithi077 ceph-mon[37327]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:20.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.307+0000 7f37d71a0640 1 -- 172.21.15.67:0/1428915869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37d0101ac0 msgr2=0x7f37d0101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:20.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.307+0000 7f37d71a0640 1 --2- 172.21.15.67:0/1428915869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37d0101ac0 0x7f37d0101ed0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009900 tx=0x7f37cc02a3f0 comp rx=0 tx=0).stop 2024-07-31T14:41:20.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.308+0000 7f37d71a0640 1 -- 172.21.15.67:0/1428915869 shutdown_connections 2024-07-31T14:41:20.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.308+0000 7f37d71a0640 1 --2- 172.21.15.67:0/1428915869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37d0107d00 0x7f37d010e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:20.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.308+0000 7f37d71a0640 1 --2- 172.21.15.67:0/1428915869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d01073b0 0x7f37d01077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:20.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.308+0000 7f37d71a0640 1 --2- 172.21.15.67:0/1428915869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37d0101ac0 0x7f37d0101ed0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:20.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.308+0000 7f37d71a0640 1 -- 172.21.15.67:0/1428915869 >> 172.21.15.67:0/1428915869 conn(0x7f37d0077440 msgr2=0x7f37d0077880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:20.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.308+0000 7f37d71a0640 1 -- 172.21.15.67:0/1428915869 shutdown_connections 2024-07-31T14:41:20.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.308+0000 7f37d71a0640 1 -- 172.21.15.67:0/1428915869 wait complete. 2024-07-31T14:41:20.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.308+0000 7f37d71a0640 1 Processor -- start 2024-07-31T14:41:20.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d71a0640 1 -- start start 2024-07-31T14:41:20.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d71a0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37d0101ac0 0x7f37d0199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:20.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d71a0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37d01073b0 0x7f37d019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d71a0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d0107d00 0x7f37d019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d71a0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f37d0112260 con 0x7f37d01073b0 2024-07-31T14:41:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d71a0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f37d01120e0 con 0x7f37d0101ac0 2024-07-31T14:41:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d71a0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f37d01123e0 con 0x7f37d0107d00 2024-07-31T14:41:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d599d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37d01073b0 0x7f37d019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d599d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37d01073b0 0x7f37d019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41606/0 (socket says 172.21.15.67:41606) 2024-07-31T14:41:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d599d640 1 -- 172.21.15.67:0/2258527800 learned_addr learned my addr 172.21.15.67:0/2258527800 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:41:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d619e640 1 --2- 172.21.15.67:0/2258527800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37d0101ac0 0x7f37d0199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d699f640 1 --2- 172.21.15.67:0/2258527800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d0107d00 0x7f37d019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d599d640 1 -- 172.21.15.67:0/2258527800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d0107d00 msgr2=0x7f37d019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d599d640 1 --2- 172.21.15.67:0/2258527800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d0107d00 0x7f37d019f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d599d640 1 -- 172.21.15.67:0/2258527800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37d0101ac0 msgr2=0x7f37d0199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d599d640 1 --2- 172.21.15.67:0/2258527800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37d0101ac0 0x7f37d0199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.309+0000 7f37d599d640 1 -- 172.21.15.67:0/2258527800 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d019fc10 con 0x7f37d01073b0 2024-07-31T14:41:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.310+0000 7f37d599d640 1 --2- 172.21.15.67:0/2258527800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37d01073b0 0x7f37d019a300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f37c000d750 tx=0x7f37c000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.310+0000 7f37c77fe640 1 -- 172.21.15.67:0/2258527800 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0017070 con 0x7f37d01073b0 2024-07-31T14:41:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.310+0000 7f37c77fe640 1 -- 172.21.15.67:0/2258527800 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c00054d0 con 0x7f37d01073b0 2024-07-31T14:41:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.310+0000 7f37d71a0640 1 -- 172.21.15.67:0/2258527800 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d019ff00 con 0x7f37d01073b0 2024-07-31T14:41:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.310+0000 7f37c77fe640 1 -- 172.21.15.67:0/2258527800 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0004c30 con 0x7f37d01073b0 2024-07-31T14:41:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.310+0000 7f37d71a0640 1 -- 172.21.15.67:0/2258527800 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d01a6720 con 0x7f37d01073b0 2024-07-31T14:41:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.311+0000 7f37d71a0640 1 -- 172.21.15.67:0/2258527800 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d0101ed0 con 0x7f37d01073b0 2024-07-31T14:41:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.312+0000 7f37c77fe640 1 -- 172.21.15.67:0/2258527800 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f37c0004dd0 con 0x7f37d01073b0 2024-07-31T14:41:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.313+0000 7f37c77fe640 1 --2- 172.21.15.67:0/2258527800 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37bc0b3c40 0x7f37bc0b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.313+0000 7f37d619e640 1 --2- 172.21.15.67:0/2258527800 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37bc0b3c40 0x7f37bc0b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.313+0000 7f37c77fe640 1 -- 172.21.15.67:0/2258527800 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f37c00e18c0 con 0x7f37d01073b0 2024-07-31T14:41:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.314+0000 7f37d619e640 1 --2- 172.21.15.67:0/2258527800 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37bc0b3c40 0x7f37bc0b6150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f37cc031440 tx=0x7f37cc009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:20.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.318+0000 7f37c77fe640 1 -- 172.21.15.67:0/2258527800 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f37c00aa4c0 con 0x7f37d01073b0 2024-07-31T14:41:20.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:20.522+0000 7f37d71a0640 1 -- 172.21.15.67:0/2258527800 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f37d0063230 con 0x7f37bc0b3c40 2024-07-31T14:41:21.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:20 smithi073 ceph-mon[36714]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:21.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:41:21.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:41:21.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:21.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:20 smithi067 ceph-mon[32750]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:21.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:41:21.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:41:21.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:21.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:20 smithi077 ceph-mon[37327]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:21.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:41:21.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:41:21.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:22.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:21 smithi067 ceph-mon[32750]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:41:22.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:21 smithi073 ceph-mon[36714]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:41:22.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:21 smithi077 ceph-mon[37327]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:41:23.095 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:22 smithi067 ceph-mon[32750]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:23.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:22 smithi073 ceph-mon[36714]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:23.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:22 smithi077 ceph-mon[37327]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:24.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:23 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/4040749411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1469d842-7dd3-4a23-969f-b98d853dbc8a"}]: dispatch 2024-07-31T14:41:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:23 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/4040749411' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1469d842-7dd3-4a23-969f-b98d853dbc8a"}]': finished 2024-07-31T14:41:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:23 smithi073 ceph-mon[36714]: osdmap e6: 1 total, 0 up, 1 in 2024-07-31T14:41:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:23 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1760752401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:41:24.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:23 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/4040749411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1469d842-7dd3-4a23-969f-b98d853dbc8a"}]: dispatch 2024-07-31T14:41:24.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:23 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/4040749411' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1469d842-7dd3-4a23-969f-b98d853dbc8a"}]': finished 2024-07-31T14:41:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:23 smithi077 ceph-mon[37327]: osdmap e6: 1 total, 0 up, 1 in 2024-07-31T14:41:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:23 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1760752401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:41:24.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:23 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/4040749411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1469d842-7dd3-4a23-969f-b98d853dbc8a"}]: dispatch 2024-07-31T14:41:24.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:23 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/4040749411' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1469d842-7dd3-4a23-969f-b98d853dbc8a"}]': finished 2024-07-31T14:41:24.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:23 smithi067 ceph-mon[32750]: osdmap e6: 1 total, 0 up, 1 in 2024-07-31T14:41:24.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:24.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:23 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1760752401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:41:25.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:24 smithi073 ceph-mon[36714]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:25.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:24 smithi077 ceph-mon[37327]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:24 smithi067 ceph-mon[32750]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:27.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:26 smithi073 ceph-mon[36714]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:27.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:26 smithi077 ceph-mon[37327]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:27.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:26 smithi067 ceph-mon[32750]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:29.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:28 smithi067 ceph-mon[32750]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:29.144 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:28 smithi077 ceph-mon[37327]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:29.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:28 smithi073 ceph-mon[36714]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:30.232 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-31T14:41:30.232 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:30.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-31T14:41:30.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:30.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-31T14:41:30.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:31.214 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:30 smithi067 ceph-mon[32750]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:31.214 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:30 smithi067 ceph-mon[32750]: Deploying daemon osd.0 on smithi067 2024-07-31T14:41:31.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:30 smithi077 ceph-mon[37327]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:31.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:30 smithi077 ceph-mon[37327]: Deploying daemon osd.0 on smithi067 2024-07-31T14:41:31.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:30 smithi073 ceph-mon[36714]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:31.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:30 smithi073 ceph-mon[36714]: Deploying daemon osd.0 on smithi067 2024-07-31T14:41:33.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:32 smithi077 ceph-mon[37327]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:33.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:32 smithi067 ceph-mon[32750]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:33.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:32 smithi073 ceph-mon[36714]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:35.046 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:34 smithi067 ceph-mon[32750]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:35.046 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:35.046 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:35.046 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:34 smithi077 ceph-mon[37327]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:35.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:35.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:35.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:35.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:34 smithi073 ceph-mon[36714]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:35.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:35.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:35.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:37.192 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:36 smithi067 ceph-mon[32750]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:37.192 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:36 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:37.192 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:36 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:36 smithi077 ceph-mon[37327]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:36 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:36 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:37.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:36 smithi073 ceph-mon[36714]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:37.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:36 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:37.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:36 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:37.523+0000 7f37c77fe640 1 -- 172.21.15.67:0/2258527800 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f37d0063230 con 0x7f37bc0b3c40 2024-07-31T14:41:37.524 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 0 on host 'smithi067' 2024-07-31T14:41:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:37.525+0000 7f37d71a0640 1 -- 172.21.15.67:0/2258527800 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37bc0b3c40 msgr2=0x7f37bc0b6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:37.525+0000 7f37d71a0640 1 --2- 172.21.15.67:0/2258527800 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37bc0b3c40 0x7f37bc0b6150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f37cc031440 tx=0x7f37cc009560 comp rx=0 tx=0).stop 2024-07-31T14:41:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:37.525+0000 7f37d71a0640 1 -- 172.21.15.67:0/2258527800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37d01073b0 msgr2=0x7f37d019a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:37.525+0000 7f37d71a0640 1 --2- 172.21.15.67:0/2258527800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37d01073b0 0x7f37d019a300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f37c000d750 tx=0x7f37c000dc10 comp rx=0 tx=0).stop 2024-07-31T14:41:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:37.525+0000 7f37d71a0640 1 -- 172.21.15.67:0/2258527800 shutdown_connections 2024-07-31T14:41:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:37.525+0000 7f37d71a0640 1 --2- 172.21.15.67:0/2258527800 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37bc0b3c40 0x7f37bc0b6150 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:37.525+0000 7f37d71a0640 1 --2- 172.21.15.67:0/2258527800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d0107d00 0x7f37d019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:37.525+0000 7f37d71a0640 1 --2- 172.21.15.67:0/2258527800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37d01073b0 0x7f37d019a300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:37.525+0000 7f37d71a0640 1 --2- 172.21.15.67:0/2258527800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37d0101ac0 0x7f37d0199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:37.525+0000 7f37d71a0640 1 -- 172.21.15.67:0/2258527800 >> 172.21.15.67:0/2258527800 conn(0x7f37d0077440 msgr2=0x7f37d010efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:37.526+0000 7f37d71a0640 1 -- 172.21.15.67:0/2258527800 shutdown_connections 2024-07-31T14:41:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:41:37.526+0000 7f37d71a0640 1 -- 172.21.15.67:0/2258527800 wait complete. 2024-07-31T14:41:38.107 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:37 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:38.107 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:37 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:41:38.108 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:37 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:38.108 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:37 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:38.108 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:37 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:38.108 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:37 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:38.109 DEBUG:teuthology.orchestra.run.smithi067:osd.0> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.0.service 2024-07-31T14:41:38.111 INFO:tasks.cephadm:Deploying osd.1 on smithi073 with /dev/vg_nvme/lv_4... 2024-07-31T14:41:38.112 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-31T14:41:38.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:37 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:37 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:41:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:37 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:37 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:37 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:37 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:38.366 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.b/config 2024-07-31T14:41:38.373 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 14:41:38 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0[53057]: 2024-07-31T14:41:38.232+0000 7f6117929640 -1 osd.0 0 log_to_monitors true 2024-07-31T14:41:38.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:37 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:38.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:37 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:41:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:37 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:37 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:37 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:37 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:39.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:38 smithi073 ceph-mon[36714]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:39.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:38 smithi073 ceph-mon[36714]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-31T14:41:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:38 smithi077 ceph-mon[37327]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:38 smithi077 ceph-mon[37327]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-31T14:41:39.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:38 smithi067 ceph-mon[32750]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:39.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:38 smithi067 ceph-mon[32750]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-31T14:41:40.047 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-31T14:41:40.134 INFO:teuthology.orchestra.run.smithi073.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-31T14:41:40.149 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10+0 records in 2024-07-31T14:41:40.149 INFO:teuthology.orchestra.run.smithi073.stderr:10+0 records out 2024-07-31T14:41:40.149 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130335 s, 805 MB/s 2024-07-31T14:41:40.149 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping successful for: 2024-07-31T14:41:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:39 smithi077 ceph-mon[37327]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-31T14:41:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:39 smithi077 ceph-mon[37327]: osdmap e7: 1 total, 0 up, 1 in 2024-07-31T14:41:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:39 smithi077 ceph-mon[37327]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-31T14:41:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:39 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:40.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:39 smithi077 ceph-mon[37327]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-31T14:41:40.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:39 smithi077 ceph-mon[37327]: osdmap e8: 1 total, 0 up, 1 in 2024-07-31T14:41:40.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:39 smithi067 ceph-mon[32750]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-31T14:41:40.278 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:39 smithi067 ceph-mon[32750]: osdmap e7: 1 total, 0 up, 1 in 2024-07-31T14:41:40.278 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:39 smithi067 ceph-mon[32750]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-31T14:41:40.278 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:39 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:40.278 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:39 smithi067 ceph-mon[32750]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-31T14:41:40.278 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:39 smithi067 ceph-mon[32750]: osdmap e8: 1 total, 0 up, 1 in 2024-07-31T14:41:40.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:39 smithi073 ceph-mon[36714]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-31T14:41:40.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:39 smithi073 ceph-mon[36714]: osdmap e7: 1 total, 0 up, 1 in 2024-07-31T14:41:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:39 smithi073 ceph-mon[36714]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-31T14:41:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:39 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:39 smithi073 ceph-mon[36714]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-31T14:41:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:39 smithi073 ceph-mon[36714]: osdmap e8: 1 total, 0 up, 1 in 2024-07-31T14:41:40.830 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_4 2024-07-31T14:41:40.917 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 14:41:40 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0[53057]: 2024-07-31T14:41:40.686+0000 7f61138ab640 -1 osd.0 0 waiting for initial osdmap 2024-07-31T14:41:40.917 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 14:41:40 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0[53057]: 2024-07-31T14:41:40.690+0000 7f610b6a1640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-31T14:41:41.092 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.b/config 2024-07-31T14:41:41.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:40 smithi073 ceph-mon[36714]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:41.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:40 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:40 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:40 smithi073 ceph-mon[36714]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' 2024-07-31T14:41:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:40 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:40 smithi077 ceph-mon[37327]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:40 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:40 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:41.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:40 smithi077 ceph-mon[37327]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' 2024-07-31T14:41:41.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:40 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:40 smithi067 ceph-mon[32750]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:40 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:40 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:40 smithi067 ceph-mon[32750]: from='osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504]' entity='osd.0' 2024-07-31T14:41:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:40 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:42.231 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:41 smithi073 ceph-mon[36714]: purged_snaps scrub starts 2024-07-31T14:41:42.231 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:41 smithi073 ceph-mon[36714]: purged_snaps scrub ok 2024-07-31T14:41:42.231 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:41 smithi073 ceph-mon[36714]: Detected new or changed devices on smithi067 2024-07-31T14:41:42.231 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:41 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:42.231 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:41 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:42.231 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:41 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:41:42.231 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:41 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi067 to 17088M 2024-07-31T14:41:42.232 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:41 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:42.232 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:41 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:42.232 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:41 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:41:42.232 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:41 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:42.232 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:41 smithi073 ceph-mon[36714]: osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504] boot 2024-07-31T14:41:42.232 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:41 smithi073 ceph-mon[36714]: osdmap e9: 1 total, 1 up, 1 in 2024-07-31T14:41:42.232 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:41 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:42.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:41 smithi077 ceph-mon[37327]: purged_snaps scrub starts 2024-07-31T14:41:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:41 smithi077 ceph-mon[37327]: purged_snaps scrub ok 2024-07-31T14:41:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:41 smithi077 ceph-mon[37327]: Detected new or changed devices on smithi067 2024-07-31T14:41:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:41 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:41 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:41 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:41:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:41 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi067 to 17088M 2024-07-31T14:41:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:41 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:41 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:41 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:41:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:41 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:41 smithi077 ceph-mon[37327]: osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504] boot 2024-07-31T14:41:42.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:41 smithi077 ceph-mon[37327]: osdmap e9: 1 total, 1 up, 1 in 2024-07-31T14:41:42.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:41 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:42.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:41 smithi067 ceph-mon[32750]: purged_snaps scrub starts 2024-07-31T14:41:42.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:41 smithi067 ceph-mon[32750]: purged_snaps scrub ok 2024-07-31T14:41:42.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:41 smithi067 ceph-mon[32750]: Detected new or changed devices on smithi067 2024-07-31T14:41:42.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:41 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:42.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:41 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:42.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:41 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:41:42.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:41 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi067 to 17088M 2024-07-31T14:41:42.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:41 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:42.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:41 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:42.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:41 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:41:42.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:41 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:42.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:41 smithi067 ceph-mon[32750]: osd.0 [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504] boot 2024-07-31T14:41:42.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:41 smithi067 ceph-mon[32750]: osdmap e9: 1 total, 1 up, 1 in 2024-07-31T14:41:42.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:41 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-31T14:41:42.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.617+0000 7fd234e9d640 1 -- 172.21.15.73:0/4065431093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd230109820 msgr2=0x7fd230110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:42.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.617+0000 7fd234e9d640 1 --2- 172.21.15.73:0/4065431093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd230109820 0x7fd230110050 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd228009ca0 tx=0x7fd22802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:41:42.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.617+0000 7fd234e9d640 1 -- 172.21.15.73:0/4065431093 shutdown_connections 2024-07-31T14:41:42.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.617+0000 7fd234e9d640 1 --2- 172.21.15.73:0/4065431093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd230109820 0x7fd230110050 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:42.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.617+0000 7fd234e9d640 1 --2- 172.21.15.73:0/4065431093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd230108ed0 0x7fd2301092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:42.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.617+0000 7fd234e9d640 1 --2- 172.21.15.73:0/4065431093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd230103590 0x7fd2301039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:42.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.618+0000 7fd234e9d640 1 -- 172.21.15.73:0/4065431093 >> 172.21.15.73:0/4065431093 conn(0x7fd23007b280 msgr2=0x7fd23007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:41:42.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.618+0000 7fd234e9d640 1 -- 172.21.15.73:0/4065431093 shutdown_connections 2024-07-31T14:41:42.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.618+0000 7fd234e9d640 1 -- 172.21.15.73:0/4065431093 wait complete. 2024-07-31T14:41:42.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.618+0000 7fd234e9d640 1 Processor -- start 2024-07-31T14:41:42.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.618+0000 7fd234e9d640 1 -- start start 2024-07-31T14:41:42.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd234e9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd230103590 0x7fd230199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:42.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd234e9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd230108ed0 0x7fd23019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:42.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd234e9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd230109820 0x7fd23019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:42.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd234e9d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301134f0 con 0x7fd230103590 2024-07-31T14:41:42.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd234e9d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd230113370 con 0x7fd230108ed0 2024-07-31T14:41:42.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd234e9d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd230113670 con 0x7fd230109820 2024-07-31T14:41:42.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd22dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd230108ed0 0x7fd23019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:42.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd22dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd230108ed0 0x7fd23019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53188/0 (socket says 172.21.15.73:53188) 2024-07-31T14:41:42.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd22dd9b640 1 -- 172.21.15.73:0/1858754063 learned_addr learned my addr 172.21.15.73:0/1858754063 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-07-31T14:41:42.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd22e59c640 1 --2- 172.21.15.73:0/1858754063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd230103590 0x7fd230199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:42.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd22ed9d640 1 --2- 172.21.15.73:0/1858754063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd230109820 0x7fd23019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:42.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd22dd9b640 1 -- 172.21.15.73:0/1858754063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd230109820 msgr2=0x7fd23019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:42.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd22dd9b640 1 --2- 172.21.15.73:0/1858754063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd230109820 0x7fd23019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:42.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd22dd9b640 1 -- 172.21.15.73:0/1858754063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd230103590 msgr2=0x7fd230199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:41:42.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd22dd9b640 1 --2- 172.21.15.73:0/1858754063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd230103590 0x7fd230199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:41:42.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd22dd9b640 1 -- 172.21.15.73:0/1858754063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23019fe90 con 0x7fd230108ed0 2024-07-31T14:41:42.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd22dd9b640 1 --2- 172.21.15.73:0/1858754063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd230108ed0 0x7fd23019a4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd22400d750 tx=0x7fd22400dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:42.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.619+0000 7fd21f7fe640 1 -- 172.21.15.73:0/1858754063 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224017070 con 0x7fd230108ed0 2024-07-31T14:41:42.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.620+0000 7fd234e9d640 1 -- 172.21.15.73:0/1858754063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2301a0180 con 0x7fd230108ed0 2024-07-31T14:41:42.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.620+0000 7fd234e9d640 1 -- 172.21.15.73:0/1858754063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2301a68e0 con 0x7fd230108ed0 2024-07-31T14:41:42.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.620+0000 7fd21f7fe640 1 -- 172.21.15.73:0/1858754063 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2240054d0 con 0x7fd230108ed0 2024-07-31T14:41:42.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.620+0000 7fd21f7fe640 1 -- 172.21.15.73:0/1858754063 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224004c30 con 0x7fd230108ed0 2024-07-31T14:41:42.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.621+0000 7fd234e9d640 1 -- 172.21.15.73:0/1858754063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2301039a0 con 0x7fd230108ed0 2024-07-31T14:41:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.622+0000 7fd21f7fe640 1 -- 172.21.15.73:0/1858754063 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fd224004dd0 con 0x7fd230108ed0 2024-07-31T14:41:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.623+0000 7fd21f7fe640 1 --2- 172.21.15.73:0/1858754063 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd2180b3b90 0x7fd2180b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:41:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.623+0000 7fd21f7fe640 1 -- 172.21.15.73:0/1858754063 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1982+0+0 (secure 0 0 0) 0x7fd2240e1f90 con 0x7fd230108ed0 2024-07-31T14:41:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.623+0000 7fd22e59c640 1 --2- 172.21.15.73:0/1858754063 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd2180b3b90 0x7fd2180b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:41:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.624+0000 7fd22e59c640 1 --2- 172.21.15.73:0/1858754063 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd2180b3b90 0x7fd2180b60a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd22000bbe0 tx=0x7fd22000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:41:42.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.626+0000 7fd21f7fe640 1 -- 172.21.15.73:0/1858754063 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd2240aa930 con 0x7fd230108ed0 2024-07-31T14:41:42.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:41:42.817+0000 7fd234e9d640 1 -- 172.21.15.73:0/1858754063 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd230063230 con 0x7fd2180b3b90 2024-07-31T14:41:43.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:42 smithi073 ceph-mon[36714]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:43.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:41:43.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:41:43.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:42 smithi077 ceph-mon[37327]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:41:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:41:43.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:42 smithi067 ceph-mon[32750]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-31T14:41:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:41:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:41:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:44.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:43 smithi073 ceph-mon[36714]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:41:44.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:43 smithi073 ceph-mon[36714]: osdmap e10: 1 total, 1 up, 1 in 2024-07-31T14:41:44.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:43 smithi077 ceph-mon[37327]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:41:44.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:43 smithi077 ceph-mon[37327]: osdmap e10: 1 total, 1 up, 1 in 2024-07-31T14:41:44.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:43 smithi067 ceph-mon[32750]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:41:44.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:43 smithi067 ceph-mon[32750]: osdmap e10: 1 total, 1 up, 1 in 2024-07-31T14:41:45.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:44 smithi073 ceph-mon[36714]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:45.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:44 smithi077 ceph-mon[37327]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:45.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:44 smithi067 ceph-mon[32750]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:46.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:45 smithi073 ceph-mon[36714]: from='client.? 172.21.15.73:0/1337721642' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "329506f6-e103-4178-bcf7-10913f292047"}]: dispatch 2024-07-31T14:41:46.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:45 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "329506f6-e103-4178-bcf7-10913f292047"}]: dispatch 2024-07-31T14:41:46.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:45 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "329506f6-e103-4178-bcf7-10913f292047"}]': finished 2024-07-31T14:41:46.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:45 smithi073 ceph-mon[36714]: osdmap e11: 2 total, 1 up, 2 in 2024-07-31T14:41:46.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:45 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:41:46.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:45 smithi073 ceph-mon[36714]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:46.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:45 smithi073 ceph-mon[36714]: from='client.? 172.21.15.73:0/2595647804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:41:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:45 smithi077 ceph-mon[37327]: from='client.? 172.21.15.73:0/1337721642' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "329506f6-e103-4178-bcf7-10913f292047"}]: dispatch 2024-07-31T14:41:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:45 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "329506f6-e103-4178-bcf7-10913f292047"}]: dispatch 2024-07-31T14:41:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:45 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "329506f6-e103-4178-bcf7-10913f292047"}]': finished 2024-07-31T14:41:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:45 smithi077 ceph-mon[37327]: osdmap e11: 2 total, 1 up, 2 in 2024-07-31T14:41:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:45 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:41:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:45 smithi077 ceph-mon[37327]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:45 smithi077 ceph-mon[37327]: from='client.? 172.21.15.73:0/2595647804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:41:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:45 smithi067 ceph-mon[32750]: from='client.? 172.21.15.73:0/1337721642' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "329506f6-e103-4178-bcf7-10913f292047"}]: dispatch 2024-07-31T14:41:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:45 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "329506f6-e103-4178-bcf7-10913f292047"}]: dispatch 2024-07-31T14:41:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:45 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "329506f6-e103-4178-bcf7-10913f292047"}]': finished 2024-07-31T14:41:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:45 smithi067 ceph-mon[32750]: osdmap e11: 2 total, 1 up, 2 in 2024-07-31T14:41:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:45 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:41:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:45 smithi067 ceph-mon[32750]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:45 smithi067 ceph-mon[32750]: from='client.? 172.21.15.73:0/2595647804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:41:48.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:48 smithi073 ceph-mon[36714]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:48.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:48 smithi077 ceph-mon[37327]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:48.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:48 smithi067 ceph-mon[32750]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:50.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:50 smithi073 ceph-mon[36714]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:50.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:50 smithi077 ceph-mon[37327]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:50.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:50 smithi067 ceph-mon[32750]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:52.582 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:52 smithi073 ceph-mon[36714]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:52.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-31T14:41:52.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:52.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:52 smithi073 ceph-mon[36714]: Deploying daemon osd.1 on smithi073 2024-07-31T14:41:52.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:52 smithi077 ceph-mon[37327]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:52.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-31T14:41:52.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:52.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:52 smithi077 ceph-mon[37327]: Deploying daemon osd.1 on smithi073 2024-07-31T14:41:52.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:52 smithi067 ceph-mon[32750]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:52.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-31T14:41:52.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:41:52.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:52 smithi067 ceph-mon[32750]: Deploying daemon osd.1 on smithi073 2024-07-31T14:41:54.526 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:54 smithi073 ceph-mon[36714]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:54.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:54 smithi077 ceph-mon[37327]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:54.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:54 smithi067 ceph-mon[32750]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:56.480 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:56 smithi073 ceph-mon[36714]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:56.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:56 smithi077 ceph-mon[37327]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:56.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:56 smithi067 ceph-mon[32750]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:57.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:57 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:57.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:57 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:57 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:57.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:57 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:57.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:57 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:57.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:57 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:57.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:57 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:41:57.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:57 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:57.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:57 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:41:58.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:41:58 smithi077 ceph-mon[37327]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:59.018 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:41:58 smithi073 ceph-mon[36714]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:41:59.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:41:58 smithi067 ceph-mon[32750]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:42:00.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:00.002+0000 7fd21f7fe640 1 -- 172.21.15.73:0/1858754063 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd230063230 con 0x7fd2180b3b90 2024-07-31T14:42:00.005 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 1 on host 'smithi073' 2024-07-31T14:42:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:00.005+0000 7fd234e9d640 1 -- 172.21.15.73:0/1858754063 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd2180b3b90 msgr2=0x7fd2180b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:00.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:00.005+0000 7fd234e9d640 1 --2- 172.21.15.73:0/1858754063 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd2180b3b90 0x7fd2180b60a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd22000bbe0 tx=0x7fd22000b390 comp rx=0 tx=0).stop 2024-07-31T14:42:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:00.005+0000 7fd234e9d640 1 -- 172.21.15.73:0/1858754063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd230108ed0 msgr2=0x7fd23019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:00.005+0000 7fd234e9d640 1 --2- 172.21.15.73:0/1858754063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd230108ed0 0x7fd23019a4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd22400d750 tx=0x7fd22400dc10 comp rx=0 tx=0).stop 2024-07-31T14:42:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:00.005+0000 7fd234e9d640 1 -- 172.21.15.73:0/1858754063 shutdown_connections 2024-07-31T14:42:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:00.005+0000 7fd234e9d640 1 --2- 172.21.15.73:0/1858754063 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd2180b3b90 0x7fd2180b60a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:00.005+0000 7fd234e9d640 1 --2- 172.21.15.73:0/1858754063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd230109820 0x7fd23019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:00.005+0000 7fd234e9d640 1 --2- 172.21.15.73:0/1858754063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd230108ed0 0x7fd23019a4c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:00.005+0000 7fd234e9d640 1 --2- 172.21.15.73:0/1858754063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd230103590 0x7fd230199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:00.005+0000 7fd234e9d640 1 -- 172.21.15.73:0/1858754063 >> 172.21.15.73:0/1858754063 conn(0x7fd23007b280 msgr2=0x7fd230106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:42:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:00.005+0000 7fd234e9d640 1 -- 172.21.15.73:0/1858754063 shutdown_connections 2024-07-31T14:42:00.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:00.005+0000 7fd234e9d640 1 -- 172.21.15.73:0/1858754063 wait complete. 2024-07-31T14:42:00.301 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.323 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.323 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:00.323 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:00.323 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.323 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:00 smithi073 ceph-mon[36714]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:42:00.323 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:00.323 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.323 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:00.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:00.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:00 smithi077 ceph-mon[37327]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:42:00.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:00.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:00.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:00.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:00 smithi067 ceph-mon[32750]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:42:00.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:00.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:00.612 DEBUG:teuthology.orchestra.run.smithi073:osd.1> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.1.service 2024-07-31T14:42:00.614 INFO:tasks.cephadm:Deploying osd.2 on smithi077 with /dev/vg_nvme/lv_4... 2024-07-31T14:42:00.614 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-31T14:42:00.871 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T14:42:01.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:01 smithi077 ceph-mon[37327]: from='osd.1 [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-31T14:42:01.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:01 smithi077 ceph-mon[37327]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-31T14:42:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:01 smithi073 ceph-mon[36714]: from='osd.1 [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-31T14:42:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:01 smithi073 ceph-mon[36714]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-31T14:42:01.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:01 smithi067 ceph-mon[32750]: from='osd.1 [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-31T14:42:01.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:01 smithi067 ceph-mon[32750]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-31T14:42:02.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:02 smithi077 ceph-mon[37327]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-31T14:42:02.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:02 smithi077 ceph-mon[37327]: osdmap e12: 2 total, 1 up, 2 in 2024-07-31T14:42:02.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:02.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:02 smithi077 ceph-mon[37327]: from='osd.1 [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:42:02.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:02 smithi077 ceph-mon[37327]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:42:02.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:02 smithi077 ceph-mon[37327]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:42:02.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:02 smithi067 ceph-mon[32750]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-31T14:42:02.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:02 smithi067 ceph-mon[32750]: osdmap e12: 2 total, 1 up, 2 in 2024-07-31T14:42:02.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:02.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:02 smithi067 ceph-mon[32750]: from='osd.1 [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:42:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:02 smithi067 ceph-mon[32750]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:42:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:02 smithi067 ceph-mon[32750]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:42:02.612 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-31T14:42:02.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:02 smithi073 ceph-mon[36714]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-31T14:42:02.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:02 smithi073 ceph-mon[36714]: osdmap e12: 2 total, 1 up, 2 in 2024-07-31T14:42:02.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:02.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:02 smithi073 ceph-mon[36714]: from='osd.1 [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:42:02.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:02 smithi073 ceph-mon[36714]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:42:02.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:02 smithi073 ceph-mon[36714]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:42:02.702 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-31T14:42:02.715 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-07-31T14:42:02.715 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-07-31T14:42:02.715 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112411 s, 933 MB/s 2024-07-31T14:42:02.715 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-07-31T14:42:03.143 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 14:42:02 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-1[42580]: 2024-07-31T14:42:02.865+0000 7f66de9a7640 -1 osd.1 0 waiting for initial osdmap 2024-07-31T14:42:03.143 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 14:42:02 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-1[42580]: 2024-07-31T14:42:02.869+0000 7f66d679d640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-31T14:42:03.323 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:03 smithi077 ceph-mon[37327]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-07-31T14:42:03.323 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:03 smithi077 ceph-mon[37327]: osdmap e13: 2 total, 1 up, 2 in 2024-07-31T14:42:03.323 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:03 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:03.323 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:03 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:03.323 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:03 smithi077 ceph-mon[37327]: from='osd.1 ' entity='osd.1' 2024-07-31T14:42:03.349 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_4 2024-07-31T14:42:03.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:03 smithi067 ceph-mon[32750]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-07-31T14:42:03.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:03 smithi067 ceph-mon[32750]: osdmap e13: 2 total, 1 up, 2 in 2024-07-31T14:42:03.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:03 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:03.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:03 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:03.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:03 smithi067 ceph-mon[32750]: from='osd.1 ' entity='osd.1' 2024-07-31T14:42:03.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:03 smithi073 ceph-mon[36714]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-07-31T14:42:03.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:03 smithi073 ceph-mon[36714]: osdmap e13: 2 total, 1 up, 2 in 2024-07-31T14:42:03.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:03 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:03.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:03 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:03.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:03 smithi073 ceph-mon[36714]: from='osd.1 ' entity='osd.1' 2024-07-31T14:42:03.602 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T14:42:04.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: purged_snaps scrub starts 2024-07-31T14:42:04.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: purged_snaps scrub ok 2024-07-31T14:42:04.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:04.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:42:04.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: Detected new or changed devices on smithi073 2024-07-31T14:42:04.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:04.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:04.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:42:04.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi073 to 17088M 2024-07-31T14:42:04.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:04.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:04.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:04.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:04.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: osd.1 [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318] boot 2024-07-31T14:42:04.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: osdmap e14: 2 total, 2 up, 2 in 2024-07-31T14:42:04.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:04.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: purged_snaps scrub starts 2024-07-31T14:42:04.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: purged_snaps scrub ok 2024-07-31T14:42:04.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:04.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:42:04.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: Detected new or changed devices on smithi073 2024-07-31T14:42:04.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:04.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:04.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:42:04.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi073 to 17088M 2024-07-31T14:42:04.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:04.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:04.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:04.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:04.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: osd.1 [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318] boot 2024-07-31T14:42:04.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: osdmap e14: 2 total, 2 up, 2 in 2024-07-31T14:42:04.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: purged_snaps scrub starts 2024-07-31T14:42:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: purged_snaps scrub ok 2024-07-31T14:42:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-31T14:42:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: Detected new or changed devices on smithi073 2024-07-31T14:42:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:42:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi073 to 17088M 2024-07-31T14:42:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: osd.1 [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318] boot 2024-07-31T14:42:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: osdmap e14: 2 total, 2 up, 2 in 2024-07-31T14:42:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-31T14:42:05.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.037+0000 7f25bac99640 1 -- 172.21.15.77:0/744048999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25b4108760 msgr2=0x7f25b410ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:05.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.037+0000 7f25bac99640 1 --2- 172.21.15.77:0/744048999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25b4108760 0x7f25b410ef90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009ca0 tx=0x7f25ac02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:42:05.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.037+0000 7f25bac99640 1 -- 172.21.15.77:0/744048999 shutdown_connections 2024-07-31T14:42:05.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.037+0000 7f25bac99640 1 --2- 172.21.15.77:0/744048999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25b4108760 0x7f25b410ef90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:05.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.037+0000 7f25bac99640 1 --2- 172.21.15.77:0/744048999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25b4107e10 0x7f25b4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:05.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.037+0000 7f25bac99640 1 --2- 172.21.15.77:0/744048999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25b41024d0 0x7f25b41028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:05.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.037+0000 7f25bac99640 1 -- 172.21.15.77:0/744048999 >> 172.21.15.77:0/744048999 conn(0x7f25b4079180 msgr2=0x7f25b4079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:42:05.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.037+0000 7f25bac99640 1 -- 172.21.15.77:0/744048999 shutdown_connections 2024-07-31T14:42:05.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.037+0000 7f25bac99640 1 -- 172.21.15.77:0/744048999 wait complete. 2024-07-31T14:42:05.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.038+0000 7f25bac99640 1 Processor -- start 2024-07-31T14:42:05.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.038+0000 7f25bac99640 1 -- start start 2024-07-31T14:42:05.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.038+0000 7f25bac99640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25b41024d0 0x7f25b419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:42:05.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.038+0000 7f25bac99640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25b4107e10 0x7f25b419b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:42:05.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.038+0000 7f25bac99640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25b4108760 0x7f25b419b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:42:05.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.038+0000 7f25bac99640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f25b41123e0 con 0x7f25b4107e10 2024-07-31T14:42:05.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.038+0000 7f25bac99640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f25b4112260 con 0x7f25b4108760 2024-07-31T14:42:05.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.038+0000 7f25bac99640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f25b4112560 con 0x7f25b41024d0 2024-07-31T14:42:05.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.038+0000 7f25b8a35640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25b41024d0 0x7f25b419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:42:05.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.038+0000 7f25b8a35640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25b41024d0 0x7f25b419ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51496/0 (socket says 172.21.15.77:51496) 2024-07-31T14:42:05.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.038+0000 7f25b8a35640 1 -- 172.21.15.77:0/2329278935 learned_addr learned my addr 172.21.15.77:0/2329278935 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-07-31T14:42:05.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.038+0000 7f25b3fff640 1 --2- 172.21.15.77:0/2329278935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25b4107e10 0x7f25b419b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:42:05.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.039+0000 7f25b9236640 1 --2- 172.21.15.77:0/2329278935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25b4108760 0x7f25b419b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:42:05.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.039+0000 7f25b8a35640 1 -- 172.21.15.77:0/2329278935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25b4108760 msgr2=0x7f25b419b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:05.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.039+0000 7f25b8a35640 1 --2- 172.21.15.77:0/2329278935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25b4108760 0x7f25b419b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:05.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.039+0000 7f25b8a35640 1 -- 172.21.15.77:0/2329278935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25b4107e10 msgr2=0x7f25b419b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:05.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.039+0000 7f25b8a35640 1 --2- 172.21.15.77:0/2329278935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25b4107e10 0x7f25b419b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:05.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.039+0000 7f25b8a35640 1 -- 172.21.15.77:0/2329278935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b41a5f20 con 0x7f25b41024d0 2024-07-31T14:42:05.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.039+0000 7f25b8a35640 1 --2- 172.21.15.77:0/2329278935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25b41024d0 0x7f25b419ac90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f25a800d770 tx=0x7f25a800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:42:05.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.039+0000 7f25b1ffb640 1 -- 172.21.15.77:0/2329278935 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8017070 con 0x7f25b41024d0 2024-07-31T14:42:05.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.039+0000 7f25bac99640 1 -- 172.21.15.77:0/2329278935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b41a6210 con 0x7f25b41024d0 2024-07-31T14:42:05.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.039+0000 7f25b1ffb640 1 -- 172.21.15.77:0/2329278935 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25a80054d0 con 0x7f25b41024d0 2024-07-31T14:42:05.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.039+0000 7f25b1ffb640 1 -- 172.21.15.77:0/2329278935 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8004c30 con 0x7f25b41024d0 2024-07-31T14:42:05.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.039+0000 7f25bac99640 1 -- 172.21.15.77:0/2329278935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b41a6750 con 0x7f25b41024d0 2024-07-31T14:42:05.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.040+0000 7f25bac99640 1 -- 172.21.15.77:0/2329278935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b4103750 con 0x7f25b41024d0 2024-07-31T14:42:05.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.042+0000 7f25b1ffb640 1 -- 172.21.15.77:0/2329278935 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f25a8004dd0 con 0x7f25b41024d0 2024-07-31T14:42:05.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.042+0000 7f25b1ffb640 1 --2- 172.21.15.77:0/2329278935 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25a40b3bc0 0x7f25a40b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:42:05.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.042+0000 7f25b1ffb640 1 -- 172.21.15.77:0/2329278935 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2524+0+0 (secure 0 0 0) 0x7f25a80e1eb0 con 0x7f25b41024d0 2024-07-31T14:42:05.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.042+0000 7f25b3fff640 1 --2- 172.21.15.77:0/2329278935 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25a40b3bc0 0x7f25a40b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:42:05.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.043+0000 7f25b3fff640 1 --2- 172.21.15.77:0/2329278935 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25a40b3bc0 0x7f25a40b60d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f25b406aca0 tx=0x7f25a000b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:42:05.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.047+0000 7f25b1ffb640 1 -- 172.21.15.77:0/2329278935 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f25a80aa630 con 0x7f25b41024d0 2024-07-31T14:42:05.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:05.241+0000 7f25bac99640 1 -- 172.21.15.77:0/2329278935 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f25b4063230 con 0x7f25a40b3bc0 2024-07-31T14:42:06.204 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:06 smithi077 ceph-mon[37327]: osdmap e15: 2 total, 2 up, 2 in 2024-07-31T14:42:06.204 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:06 smithi077 ceph-mon[37327]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:42:06.204 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:42:06.204 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:42:06.204 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:06.204 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:06 smithi077 ceph-mon[37327]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:06.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:06 smithi067 ceph-mon[32750]: osdmap e15: 2 total, 2 up, 2 in 2024-07-31T14:42:06.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:06 smithi067 ceph-mon[32750]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:42:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:42:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:42:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:06 smithi067 ceph-mon[32750]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:06.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:06 smithi073 ceph-mon[36714]: osdmap e15: 2 total, 2 up, 2 in 2024-07-31T14:42:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:06 smithi073 ceph-mon[36714]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:42:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:42:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:42:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:06 smithi073 ceph-mon[36714]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:08 smithi073 ceph-mon[36714]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:08 smithi073 ceph-mon[36714]: from='client.? 172.21.15.77:0/529990718' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3855f3c9-0985-4dcd-a1f6-97dd54d1d833"}]: dispatch 2024-07-31T14:42:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:08 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3855f3c9-0985-4dcd-a1f6-97dd54d1d833"}]: dispatch 2024-07-31T14:42:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:08 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3855f3c9-0985-4dcd-a1f6-97dd54d1d833"}]': finished 2024-07-31T14:42:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:08 smithi073 ceph-mon[36714]: osdmap e16: 3 total, 2 up, 3 in 2024-07-31T14:42:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:08 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:08 smithi073 ceph-mon[36714]: from='client.? 172.21.15.77:0/517621638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:42:08.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:08 smithi077 ceph-mon[37327]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:08.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:08 smithi077 ceph-mon[37327]: from='client.? 172.21.15.77:0/529990718' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3855f3c9-0985-4dcd-a1f6-97dd54d1d833"}]: dispatch 2024-07-31T14:42:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:08 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3855f3c9-0985-4dcd-a1f6-97dd54d1d833"}]: dispatch 2024-07-31T14:42:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:08 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3855f3c9-0985-4dcd-a1f6-97dd54d1d833"}]': finished 2024-07-31T14:42:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:08 smithi077 ceph-mon[37327]: osdmap e16: 3 total, 2 up, 3 in 2024-07-31T14:42:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:08 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:08 smithi077 ceph-mon[37327]: from='client.? 172.21.15.77:0/517621638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:42:08.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:08 smithi067 ceph-mon[32750]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:08.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:08 smithi067 ceph-mon[32750]: from='client.? 172.21.15.77:0/529990718' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3855f3c9-0985-4dcd-a1f6-97dd54d1d833"}]: dispatch 2024-07-31T14:42:08.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:08 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3855f3c9-0985-4dcd-a1f6-97dd54d1d833"}]: dispatch 2024-07-31T14:42:08.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:08 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3855f3c9-0985-4dcd-a1f6-97dd54d1d833"}]': finished 2024-07-31T14:42:08.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:08 smithi067 ceph-mon[32750]: osdmap e16: 3 total, 2 up, 3 in 2024-07-31T14:42:08.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:08 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:08.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:08 smithi067 ceph-mon[32750]: from='client.? 172.21.15.77:0/517621638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:42:10.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:10 smithi073 ceph-mon[36714]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:10.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:10 smithi077 ceph-mon[37327]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:10.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:10 smithi067 ceph-mon[32750]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:12.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:12 smithi073 ceph-mon[36714]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:12.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:12 smithi077 ceph-mon[37327]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:12.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:12 smithi067 ceph-mon[32750]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:14.626 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:14 smithi077 ceph-mon[37327]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:14.626 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-31T14:42:14.626 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:14.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:14 smithi073 ceph-mon[36714]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:14.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-31T14:42:14.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:14.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:14 smithi067 ceph-mon[32750]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:14.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-31T14:42:14.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:15.630 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:15 smithi077 ceph-mon[37327]: Deploying daemon osd.2 on smithi077 2024-07-31T14:42:15.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:15 smithi073 ceph-mon[36714]: Deploying daemon osd.2 on smithi077 2024-07-31T14:42:15.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:15 smithi067 ceph-mon[32750]: Deploying daemon osd.2 on smithi077 2024-07-31T14:42:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:16 smithi073 ceph-mon[36714]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:16.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:16 smithi077 ceph-mon[37327]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:16.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:16 smithi067 ceph-mon[32750]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:18.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:18 smithi077 ceph-mon[37327]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:18.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:18 smithi073 ceph-mon[36714]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:18.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:18 smithi067 ceph-mon[32750]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:19.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:19.649 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:19.649 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:19.649 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:19.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:19.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:19.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:20.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:20 smithi077 ceph-mon[37327]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:20.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:20 smithi073 ceph-mon[36714]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:20.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:20 smithi067 ceph-mon[32750]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:22.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:22.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:22.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:22.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:22.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:22.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:22 smithi067 ceph-mon[32750]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:22.362 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:22.362 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:22.362 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:22.362 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:22.362 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:22.362 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:22 smithi077 ceph-mon[37327]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:22.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:22.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:22.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:22.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:22.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:22.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:22 smithi073 ceph-mon[36714]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:22.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:22.401+0000 7f25b1ffb640 1 -- 172.21.15.77:0/2329278935 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f25b4063230 con 0x7f25a40b3bc0 2024-07-31T14:42:22.402 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 2 on host 'smithi077' 2024-07-31T14:42:22.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:22.404+0000 7f25bac99640 1 -- 172.21.15.77:0/2329278935 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25a40b3bc0 msgr2=0x7f25a40b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:22.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:22.404+0000 7f25bac99640 1 --2- 172.21.15.77:0/2329278935 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25a40b3bc0 0x7f25a40b60d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f25b406aca0 tx=0x7f25a000b410 comp rx=0 tx=0).stop 2024-07-31T14:42:22.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:22.404+0000 7f25bac99640 1 -- 172.21.15.77:0/2329278935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25b41024d0 msgr2=0x7f25b419ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:22.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:22.404+0000 7f25bac99640 1 --2- 172.21.15.77:0/2329278935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25b41024d0 0x7f25b419ac90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f25a800d770 tx=0x7f25a800dc30 comp rx=0 tx=0).stop 2024-07-31T14:42:22.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:22.404+0000 7f25bac99640 1 -- 172.21.15.77:0/2329278935 shutdown_connections 2024-07-31T14:42:22.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:22.404+0000 7f25bac99640 1 --2- 172.21.15.77:0/2329278935 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25a40b3bc0 0x7f25a40b60d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:22.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:22.404+0000 7f25bac99640 1 --2- 172.21.15.77:0/2329278935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25b4108760 0x7f25b419b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:22.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:22.404+0000 7f25bac99640 1 --2- 172.21.15.77:0/2329278935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25b4107e10 0x7f25b419b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:22.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:22.404+0000 7f25bac99640 1 --2- 172.21.15.77:0/2329278935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25b41024d0 0x7f25b419ac90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:22.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:22.404+0000 7f25bac99640 1 -- 172.21.15.77:0/2329278935 >> 172.21.15.77:0/2329278935 conn(0x7f25b4079180 msgr2=0x7f25b4103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:42:22.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:22.404+0000 7f25bac99640 1 -- 172.21.15.77:0/2329278935 shutdown_connections 2024-07-31T14:42:22.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:42:22.404+0000 7f25bac99640 1 -- 172.21.15.77:0/2329278935 wait complete. 2024-07-31T14:42:23.050 DEBUG:teuthology.orchestra.run.smithi077:osd.2> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.2.service 2024-07-31T14:42:23.052 INFO:tasks.cephadm:Deploying osd.3 on smithi067 with /dev/vg_nvme/lv_3... 2024-07-31T14:42:23.053 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-31T14:42:23.330 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:42:23.368 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:23.368 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:23.368 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:23.368 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:23 smithi077 ceph-mon[37327]: from='osd.2 [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-31T14:42:23.368 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:23 smithi077 ceph-mon[37327]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-31T14:42:23.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:23.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:23.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:23.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:23 smithi073 ceph-mon[36714]: from='osd.2 [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-31T14:42:23.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:23 smithi073 ceph-mon[36714]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-31T14:42:23.451 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:23.451 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:23.451 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:23.451 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:23 smithi067 ceph-mon[32750]: from='osd.2 [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-31T14:42:23.451 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:23 smithi067 ceph-mon[32750]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-31T14:42:24.524 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:24 smithi077 ceph-mon[37327]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:24.524 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:24 smithi077 ceph-mon[37327]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-31T14:42:24.524 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:24 smithi077 ceph-mon[37327]: osdmap e17: 3 total, 2 up, 3 in 2024-07-31T14:42:24.524 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:24.524 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:24 smithi077 ceph-mon[37327]: from='osd.2 [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:42:24.525 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:24 smithi077 ceph-mon[37327]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:42:24.525 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 14:42:24 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-2[43763]: 2024-07-31T14:42:24.410+0000 7f1b1573b640 -1 osd.2 0 waiting for initial osdmap 2024-07-31T14:42:24.525 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 14:42:24 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-2[43763]: 2024-07-31T14:42:24.414+0000 7f1b11539640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-31T14:42:24.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:24 smithi067 ceph-mon[32750]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:24 smithi067 ceph-mon[32750]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-31T14:42:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:24 smithi067 ceph-mon[32750]: osdmap e17: 3 total, 2 up, 3 in 2024-07-31T14:42:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:24 smithi067 ceph-mon[32750]: from='osd.2 [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:42:24.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:24 smithi067 ceph-mon[32750]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:42:24.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:24 smithi073 ceph-mon[36714]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:24 smithi073 ceph-mon[36714]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-31T14:42:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:24 smithi073 ceph-mon[36714]: osdmap e17: 3 total, 2 up, 3 in 2024-07-31T14:42:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:24 smithi073 ceph-mon[36714]: from='osd.2 [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:42:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:24 smithi073 ceph-mon[36714]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:42:25.305 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-31T14:42:25.417 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-31T14:42:25.432 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-07-31T14:42:25.432 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-07-31T14:42:25.432 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133183 s, 787 MB/s 2024-07-31T14:42:25.432 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-07-31T14:42:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:25 smithi077 ceph-mon[37327]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-07-31T14:42:25.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:25 smithi077 ceph-mon[37327]: osdmap e18: 3 total, 2 up, 3 in 2024-07-31T14:42:25.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:25 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:25.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:25 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:25.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:25 smithi067 ceph-mon[32750]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-07-31T14:42:25.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:25 smithi067 ceph-mon[32750]: osdmap e18: 3 total, 2 up, 3 in 2024-07-31T14:42:25.705 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:25 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:25.705 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:25 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:25.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:25 smithi073 ceph-mon[36714]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-07-31T14:42:25.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:25 smithi073 ceph-mon[36714]: osdmap e18: 3 total, 2 up, 3 in 2024-07-31T14:42:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:25 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:25 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:26.016 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi067:vg_nvme/lv_3 2024-07-31T14:42:26.276 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:42:26.692 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: purged_snaps scrub starts 2024-07-31T14:42:26.692 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: purged_snaps scrub ok 2024-07-31T14:42:26.692 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:26.692 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: osd.2 [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750] boot 2024-07-31T14:42:26.692 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: osdmap e19: 3 total, 3 up, 3 in 2024-07-31T14:42:26.692 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:26.692 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: Detected new or changed devices on smithi077 2024-07-31T14:42:26.692 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:26.692 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:26.692 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:42:26.692 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi077 to 17088M 2024-07-31T14:42:26.692 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:26.693 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:26.693 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:26.693 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: purged_snaps scrub starts 2024-07-31T14:42:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: purged_snaps scrub ok 2024-07-31T14:42:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: osd.2 [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750] boot 2024-07-31T14:42:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: osdmap e19: 3 total, 3 up, 3 in 2024-07-31T14:42:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: Detected new or changed devices on smithi077 2024-07-31T14:42:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:42:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi077 to 17088M 2024-07-31T14:42:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: purged_snaps scrub starts 2024-07-31T14:42:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: purged_snaps scrub ok 2024-07-31T14:42:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-31T14:42:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: osd.2 [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750] boot 2024-07-31T14:42:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: osdmap e19: 3 total, 3 up, 3 in 2024-07-31T14:42:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-31T14:42:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: Detected new or changed devices on smithi077 2024-07-31T14:42:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:26.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:42:26.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi077 to 17088M 2024-07-31T14:42:26.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:26.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:26.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:26.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:27.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.775+0000 7f5fb707f640 1 -- 172.21.15.67:0/4086571856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb01024d0 msgr2=0x7f5fb01028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:27.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.775+0000 7f5fb707f640 1 --2- 172.21.15.67:0/4086571856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb01024d0 0x7f5fb01028e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009900 tx=0x7f5f9802a410 comp rx=0 tx=0).stop 2024-07-31T14:42:27.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.775+0000 7f5fb707f640 1 -- 172.21.15.67:0/4086571856 shutdown_connections 2024-07-31T14:42:27.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.775+0000 7f5fb707f640 1 --2- 172.21.15.67:0/4086571856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fb0108760 0x7f5fb010ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:27.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.775+0000 7f5fb707f640 1 --2- 172.21.15.67:0/4086571856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fb0107e10 0x7f5fb0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:27.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.775+0000 7f5fb707f640 1 --2- 172.21.15.67:0/4086571856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb01024d0 0x7f5fb01028e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:27.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.775+0000 7f5fb707f640 1 -- 172.21.15.67:0/4086571856 >> 172.21.15.67:0/4086571856 conn(0x7f5fb0079190 msgr2=0x7f5fb00795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:42:27.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.776+0000 7f5fb707f640 1 -- 172.21.15.67:0/4086571856 shutdown_connections 2024-07-31T14:42:27.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.776+0000 7f5fb707f640 1 -- 172.21.15.67:0/4086571856 wait complete. 2024-07-31T14:42:27.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.776+0000 7f5fb707f640 1 Processor -- start 2024-07-31T14:42:27.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb707f640 1 -- start start 2024-07-31T14:42:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb707f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fb01024d0 0x7f5fb019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:42:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb707f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fb0107e10 0x7f5fb019b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:42:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb707f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb0108760 0x7f5fb019b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:42:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb707f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01123e0 con 0x7f5fb0108760 2024-07-31T14:42:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb707f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0112260 con 0x7f5fb0107e10 2024-07-31T14:42:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb707f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0112560 con 0x7f5fb01024d0 2024-07-31T14:42:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb561c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb0108760 0x7f5fb019b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:42:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb561c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb0108760 0x7f5fb019b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49554/0 (socket says 172.21.15.67:49554) 2024-07-31T14:42:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb561c640 1 -- 172.21.15.67:0/1271213895 learned_addr learned my addr 172.21.15.67:0/1271213895 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:42:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5faffff640 1 --2- 172.21.15.67:0/1271213895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fb0107e10 0x7f5fb019b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:42:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb4e1b640 1 --2- 172.21.15.67:0/1271213895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fb01024d0 0x7f5fb019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:42:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb561c640 1 -- 172.21.15.67:0/1271213895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fb01024d0 msgr2=0x7f5fb019acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb561c640 1 --2- 172.21.15.67:0/1271213895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fb01024d0 0x7f5fb019acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb561c640 1 -- 172.21.15.67:0/1271213895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fb0107e10 msgr2=0x7f5fb019b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb561c640 1 --2- 172.21.15.67:0/1271213895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fb0107e10 0x7f5fb019b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.777+0000 7f5fb561c640 1 -- 172.21.15.67:0/1271213895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb01a5f70 con 0x7f5fb0108760 2024-07-31T14:42:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.778+0000 7f5fb561c640 1 --2- 172.21.15.67:0/1271213895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb0108760 0x7f5fb019b730 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800eaa0 tx=0x7f5fa800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:42:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.778+0000 7f5fadffb640 1 -- 172.21.15.67:0/1271213895 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa800cdb0 con 0x7f5fb0108760 2024-07-31T14:42:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.778+0000 7f5fadffb640 1 -- 172.21.15.67:0/1271213895 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa8005020 con 0x7f5fb0108760 2024-07-31T14:42:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.778+0000 7f5fb707f640 1 -- 172.21.15.67:0/1271213895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb01a6260 con 0x7f5fb0108760 2024-07-31T14:42:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.778+0000 7f5fadffb640 1 -- 172.21.15.67:0/1271213895 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80116d0 con 0x7f5fb0108760 2024-07-31T14:42:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.778+0000 7f5fb707f640 1 -- 172.21.15.67:0/1271213895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb01a67a0 con 0x7f5fb0108760 2024-07-31T14:42:27.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.779+0000 7f5fb707f640 1 -- 172.21.15.67:0/1271213895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb0103750 con 0x7f5fb0108760 2024-07-31T14:42:27.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.781+0000 7f5fadffb640 1 -- 172.21.15.67:0/1271213895 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f5fa80051d0 con 0x7f5fb0108760 2024-07-31T14:42:27.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.782+0000 7f5fadffb640 1 --2- 172.21.15.67:0/1271213895 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5f9c0b3b90 0x7f5f9c0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:42:27.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.782+0000 7f5fadffb640 1 -- 172.21.15.67:0/1271213895 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3066+0+0 (secure 0 0 0) 0x7f5fa80e2cc0 con 0x7f5fb0108760 2024-07-31T14:42:27.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.782+0000 7f5fb4e1b640 1 --2- 172.21.15.67:0/1271213895 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5f9c0b3b90 0x7f5f9c0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:42:27.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.782+0000 7f5fb4e1b640 1 --2- 172.21.15.67:0/1271213895 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5f9c0b3b90 0x7f5f9c0b60a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5f98000c00 tx=0x7f5f98031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:42:27.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.785+0000 7f5fadffb640 1 -- 172.21.15.67:0/1271213895 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5fa8016030 con 0x7f5fb0108760 2024-07-31T14:42:27.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:27.992+0000 7f5fb707f640 1 -- 172.21.15.67:0/1271213895 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5fb0063230 con 0x7f5f9c0b3b90 2024-07-31T14:42:28.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:27 smithi073 ceph-mon[36714]: osdmap e20: 3 total, 3 up, 3 in 2024-07-31T14:42:28.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-31T14:42:28.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:27 smithi077 ceph-mon[37327]: osdmap e20: 3 total, 3 up, 3 in 2024-07-31T14:42:28.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-31T14:42:28.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:27 smithi067 ceph-mon[32750]: osdmap e20: 3 total, 3 up, 3 in 2024-07-31T14:42:28.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-31T14:42:29.053 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 14:42:28 smithi067 sudo[57721]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-31T14:42:29.053 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 14:42:28 smithi067 sudo[57721]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-31T14:42:29.053 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 14:42:28 smithi067 sudo[57721]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-31T14:42:29.053 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 14:42:28 smithi067 sudo[57721]: pam_unix(sudo:session): session closed for user root 2024-07-31T14:42:29.053 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 14:42:28 smithi067 sudo[57724]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-31T14:42:29.053 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 14:42:28 smithi067 sudo[57724]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-31T14:42:29.054 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 14:42:28 smithi067 sudo[57724]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-31T14:42:29.054 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 14:42:28 smithi067 sudo[57724]: pam_unix(sudo:session): session closed for user root 2024-07-31T14:42:29.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:28 smithi067 ceph-mon[32750]: pgmap v72: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:29.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:28 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-31T14:42:29.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:28 smithi067 ceph-mon[32750]: osdmap e21: 3 total, 3 up, 3 in 2024-07-31T14:42:29.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:28 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-31T14:42:29.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:28 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:42:29.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:28 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:42:29.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:28 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:29.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:28 smithi073 ceph-mon[36714]: pgmap v72: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:29.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:28 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-31T14:42:29.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:28 smithi073 ceph-mon[36714]: osdmap e21: 3 total, 3 up, 3 in 2024-07-31T14:42:29.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:28 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-31T14:42:29.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:28 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:42:29.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:28 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:42:29.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:28 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:29.143 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46678]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-31T14:42:29.143 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46678]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-31T14:42:29.143 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46678]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-31T14:42:29.144 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46678]: pam_unix(sudo:session): session closed for user root 2024-07-31T14:42:29.144 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46681]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-31T14:42:29.144 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46681]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-31T14:42:29.144 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46681]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-31T14:42:29.144 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46681]: pam_unix(sudo:session): session closed for user root 2024-07-31T14:42:29.171 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47810]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-31T14:42:29.171 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47810]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-31T14:42:29.171 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47810]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-31T14:42:29.171 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47810]: pam_unix(sudo:session): session closed for user root 2024-07-31T14:42:29.171 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47813]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-31T14:42:29.172 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47813]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-31T14:42:29.172 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47813]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-31T14:42:29.172 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47813]: pam_unix(sudo:session): session closed for user root 2024-07-31T14:42:29.172 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:28 smithi077 ceph-mon[37327]: pgmap v72: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:29.172 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:28 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-31T14:42:29.172 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:28 smithi077 ceph-mon[37327]: osdmap e21: 3 total, 3 up, 3 in 2024-07-31T14:42:29.172 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:28 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-31T14:42:29.172 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:28 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:42:29.172 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:28 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:42:29.172 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:28 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:29.172 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47816]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-31T14:42:29.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 sudo[57729]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-31T14:42:29.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 sudo[57729]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-31T14:42:29.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 sudo[57729]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-31T14:42:29.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 sudo[57729]: pam_unix(sudo:session): session closed for user root 2024-07-31T14:42:29.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 sudo[57732]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-31T14:42:29.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 sudo[57732]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-31T14:42:29.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 sudo[57732]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-31T14:42:29.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 sudo[57732]: pam_unix(sudo:session): session closed for user root 2024-07-31T14:42:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47816]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-31T14:42:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47816]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-31T14:42:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47816]: pam_unix(sudo:session): session closed for user root 2024-07-31T14:42:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47819]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-31T14:42:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47819]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-31T14:42:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47819]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-31T14:42:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 sudo[47819]: pam_unix(sudo:session): session closed for user root 2024-07-31T14:42:29.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46684]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-31T14:42:29.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46684]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-31T14:42:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46684]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-31T14:42:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46684]: pam_unix(sudo:session): session closed for user root 2024-07-31T14:42:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46687]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-31T14:42:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46687]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-31T14:42:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46687]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-31T14:42:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 sudo[46687]: pam_unix(sudo:session): session closed for user root 2024-07-31T14:42:30.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='client.14478 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:42:30.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-31T14:42:30.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: osdmap e22: 3 total, 3 up, 3 in 2024-07-31T14:42:30.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:30.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:30.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:30.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-31T14:42:30.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-31T14:42:30.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:42:30.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:42:30.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:42:30.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-31T14:42:30.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:42:30.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:42:30.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:42:30.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-31T14:42:30.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-31T14:42:30.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:42:30.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:42:30.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:42:30.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:29 smithi067 ceph-mon[32750]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-31T14:42:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:30.129+0000 7f5fadffb640 1 -- 172.21.15.67:0/1271213895 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f5fa80a7400 con 0x7f5fb0108760 2024-07-31T14:42:30.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='client.14478 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:42:30.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-31T14:42:30.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: osdmap e22: 3 total, 3 up, 3 in 2024-07-31T14:42:30.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:30.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:30.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:30.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-31T14:42:30.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-31T14:42:30.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:42:30.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:42:30.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:42:30.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-31T14:42:30.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:42:30.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:42:30.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:42:30.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-31T14:42:30.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-31T14:42:30.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:42:30.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:42:30.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:42:30.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:29 smithi073 ceph-mon[36714]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-31T14:42:30.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='client.14478 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:42:30.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-31T14:42:30.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: osdmap e22: 3 total, 3 up, 3 in 2024-07-31T14:42:30.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:30.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:30.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:30.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-31T14:42:30.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-31T14:42:30.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:42:30.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:42:30.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:42:30.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-31T14:42:30.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:42:30.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:42:30.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:42:30.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-31T14:42:30.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-31T14:42:30.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-31T14:42:30.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-31T14:42:30.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-31T14:42:30.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:29 smithi077 ceph-mon[37327]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-31T14:42:31.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:30 smithi067 ceph-mon[32750]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:31.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:30 smithi067 ceph-mon[32750]: osdmap e23: 3 total, 3 up, 3 in 2024-07-31T14:42:31.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:30 smithi067 ceph-mon[32750]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-07-31T14:42:31.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:30 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1518442573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1693ca0-afc8-4cbd-abef-251f8f4c42f0"}]: dispatch 2024-07-31T14:42:31.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:30 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1518442573' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1693ca0-afc8-4cbd-abef-251f8f4c42f0"}]': finished 2024-07-31T14:42:31.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:30 smithi067 ceph-mon[32750]: osdmap e24: 4 total, 3 up, 4 in 2024-07-31T14:42:31.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:30 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:30 smithi073 ceph-mon[36714]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:30 smithi073 ceph-mon[36714]: osdmap e23: 3 total, 3 up, 3 in 2024-07-31T14:42:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:30 smithi073 ceph-mon[36714]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-07-31T14:42:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:30 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1518442573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1693ca0-afc8-4cbd-abef-251f8f4c42f0"}]: dispatch 2024-07-31T14:42:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:30 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1518442573' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1693ca0-afc8-4cbd-abef-251f8f4c42f0"}]': finished 2024-07-31T14:42:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:30 smithi073 ceph-mon[36714]: osdmap e24: 4 total, 3 up, 4 in 2024-07-31T14:42:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:30 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:31.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:30 smithi077 ceph-mon[37327]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:31.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:30 smithi077 ceph-mon[37327]: osdmap e23: 3 total, 3 up, 3 in 2024-07-31T14:42:31.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:30 smithi077 ceph-mon[37327]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-07-31T14:42:31.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:30 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1518442573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1693ca0-afc8-4cbd-abef-251f8f4c42f0"}]: dispatch 2024-07-31T14:42:31.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:30 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1518442573' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1693ca0-afc8-4cbd-abef-251f8f4c42f0"}]': finished 2024-07-31T14:42:31.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:30 smithi077 ceph-mon[37327]: osdmap e24: 4 total, 3 up, 4 in 2024-07-31T14:42:31.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:30 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:32.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:31 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1825579110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:42:32.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:31 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1825579110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:42:32.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:31 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1825579110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:42:33.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:32 smithi077 ceph-mon[37327]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:33.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:32 smithi067 ceph-mon[32750]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:33.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:32 smithi073 ceph-mon[36714]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:35.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:34 smithi067 ceph-mon[32750]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 93 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:34 smithi077 ceph-mon[37327]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 93 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:35.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:34 smithi073 ceph-mon[36714]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 93 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:37.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:36 smithi067 ceph-mon[32750]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-07-31T14:42:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:36 smithi077 ceph-mon[37327]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-07-31T14:42:37.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:36 smithi073 ceph-mon[36714]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-07-31T14:42:38.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:37 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-31T14:42:38.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:37 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:38.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:37 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-31T14:42:38.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:37 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:38.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:37 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-31T14:42:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:37 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:39.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:38 smithi067 ceph-mon[32750]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-07-31T14:42:39.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:38 smithi067 ceph-mon[32750]: Deploying daemon osd.3 on smithi067 2024-07-31T14:42:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:38 smithi077 ceph-mon[37327]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-07-31T14:42:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:38 smithi077 ceph-mon[37327]: Deploying daemon osd.3 on smithi067 2024-07-31T14:42:39.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:38 smithi073 ceph-mon[36714]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-07-31T14:42:39.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:38 smithi073 ceph-mon[36714]: Deploying daemon osd.3 on smithi067 2024-07-31T14:42:41.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:40 smithi067 ceph-mon[32750]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 2 op/s 2024-07-31T14:42:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:40 smithi077 ceph-mon[37327]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 2 op/s 2024-07-31T14:42:41.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:40 smithi073 ceph-mon[36714]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 2 op/s 2024-07-31T14:42:43.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:42 smithi067 ceph-mon[32750]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 944 B/s rd, 24 KiB/s wr, 2 op/s 2024-07-31T14:42:43.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:43.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:43.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:42 smithi077 ceph-mon[37327]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 944 B/s rd, 24 KiB/s wr, 2 op/s 2024-07-31T14:42:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:43.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:42 smithi073 ceph-mon[36714]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 944 B/s rd, 24 KiB/s wr, 2 op/s 2024-07-31T14:42:43.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:43.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:44.989 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:44 smithi067 ceph-mon[32750]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-31T14:42:45.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:44 smithi077 ceph-mon[37327]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-31T14:42:45.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:44 smithi073 ceph-mon[36714]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-31T14:42:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 3 on host 'smithi067' 2024-07-31T14:42:46.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:46.080+0000 7f5fadffb640 1 -- 172.21.15.67:0/1271213895 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5fb0063230 con 0x7f5f9c0b3b90 2024-07-31T14:42:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:46.083+0000 7f5fb707f640 1 -- 172.21.15.67:0/1271213895 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5f9c0b3b90 msgr2=0x7f5f9c0b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:46.083+0000 7f5fb707f640 1 --2- 172.21.15.67:0/1271213895 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5f9c0b3b90 0x7f5f9c0b60a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5f98000c00 tx=0x7f5f98031400 comp rx=0 tx=0).stop 2024-07-31T14:42:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:46.083+0000 7f5fb707f640 1 -- 172.21.15.67:0/1271213895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb0108760 msgr2=0x7f5fb019b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:46.083+0000 7f5fb707f640 1 --2- 172.21.15.67:0/1271213895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb0108760 0x7f5fb019b730 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800eaa0 tx=0x7f5fa800ef60 comp rx=0 tx=0).stop 2024-07-31T14:42:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:46.083+0000 7f5fb707f640 1 -- 172.21.15.67:0/1271213895 shutdown_connections 2024-07-31T14:42:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:46.083+0000 7f5fb707f640 1 --2- 172.21.15.67:0/1271213895 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5f9c0b3b90 0x7f5f9c0b60a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:46.083+0000 7f5fb707f640 1 --2- 172.21.15.67:0/1271213895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb0108760 0x7f5fb019b730 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:46.083+0000 7f5fb707f640 1 --2- 172.21.15.67:0/1271213895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fb0107e10 0x7f5fb019b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:46.083+0000 7f5fb707f640 1 --2- 172.21.15.67:0/1271213895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fb01024d0 0x7f5fb019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:46.083+0000 7f5fb707f640 1 -- 172.21.15.67:0/1271213895 >> 172.21.15.67:0/1271213895 conn(0x7f5fb0079190 msgr2=0x7f5fb0100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:42:46.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:46.083+0000 7f5fb707f640 1 -- 172.21.15.67:0/1271213895 shutdown_connections 2024-07-31T14:42:46.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:42:46.083+0000 7f5fb707f640 1 -- 172.21.15.67:0/1271213895 wait complete. 2024-07-31T14:42:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:46 smithi067 ceph-mon[32750]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-31T14:42:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:46 smithi077 ceph-mon[37327]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-31T14:42:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:46 smithi073 ceph-mon[36714]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-07-31T14:42:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:42:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:46.769 DEBUG:teuthology.orchestra.run.smithi067:osd.3> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.3.service 2024-07-31T14:42:46.771 INFO:tasks.cephadm:Deploying osd.4 on smithi073 with /dev/vg_nvme/lv_3... 2024-07-31T14:42:46.772 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-31T14:42:47.028 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.b/config 2024-07-31T14:42:47.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:47 smithi067 ceph-mon[32750]: from='osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-31T14:42:47.408 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:47 smithi073 ceph-mon[36714]: from='osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-31T14:42:47.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:47 smithi077 ceph-mon[37327]: from='osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-31T14:42:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:48 smithi073 ceph-mon[36714]: from='osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-31T14:42:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:48 smithi073 ceph-mon[36714]: osdmap e25: 4 total, 3 up, 4 in 2024-07-31T14:42:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:48 smithi073 ceph-mon[36714]: from='osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-31T14:42:48.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:48.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:48 smithi073 ceph-mon[36714]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:48.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:48 smithi067 ceph-mon[32750]: from='osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-31T14:42:48.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:48 smithi067 ceph-mon[32750]: osdmap e25: 4 total, 3 up, 4 in 2024-07-31T14:42:48.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:48 smithi067 ceph-mon[32750]: from='osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-31T14:42:48.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:48.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:48 smithi067 ceph-mon[32750]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:48.554 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 14:42:48 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3[59816]: 2024-07-31T14:42:48.253+0000 7f05ecf6f640 -1 osd.3 0 waiting for initial osdmap 2024-07-31T14:42:48.554 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 14:42:48 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3[59816]: 2024-07-31T14:42:48.259+0000 7f05e8d6d640 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-31T14:42:48.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:48 smithi077 ceph-mon[37327]: from='osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-31T14:42:48.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:48 smithi077 ceph-mon[37327]: osdmap e25: 4 total, 3 up, 4 in 2024-07-31T14:42:48.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:48 smithi077 ceph-mon[37327]: from='osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-31T14:42:48.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:48.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:48 smithi077 ceph-mon[37327]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-31T14:42:48.801 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-31T14:42:48.885 INFO:teuthology.orchestra.run.smithi073.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-31T14:42:48.904 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10+0 records in 2024-07-31T14:42:48.904 INFO:teuthology.orchestra.run.smithi073.stderr:10+0 records out 2024-07-31T14:42:48.904 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.017602 s, 596 MB/s 2024-07-31T14:42:48.904 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping successful for: 2024-07-31T14:42:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:49 smithi067 ceph-mon[32750]: from='osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-31T14:42:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:49 smithi067 ceph-mon[32750]: osdmap e26: 4 total, 3 up, 4 in 2024-07-31T14:42:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:49 smithi067 ceph-mon[32750]: osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409] boot 2024-07-31T14:42:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:49 smithi067 ceph-mon[32750]: osdmap e27: 4 total, 4 up, 4 in 2024-07-31T14:42:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:49.562 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:49 smithi073 ceph-mon[36714]: from='osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-31T14:42:49.562 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:49 smithi073 ceph-mon[36714]: osdmap e26: 4 total, 3 up, 4 in 2024-07-31T14:42:49.562 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:49.562 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:49.562 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:49 smithi073 ceph-mon[36714]: osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409] boot 2024-07-31T14:42:49.562 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:49 smithi073 ceph-mon[36714]: osdmap e27: 4 total, 4 up, 4 in 2024-07-31T14:42:49.562 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:49.596 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_3 2024-07-31T14:42:49.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:49 smithi077 ceph-mon[37327]: from='osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-31T14:42:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:49 smithi077 ceph-mon[37327]: osdmap e26: 4 total, 3 up, 4 in 2024-07-31T14:42:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:49 smithi077 ceph-mon[37327]: osd.3 [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409] boot 2024-07-31T14:42:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:49 smithi077 ceph-mon[37327]: osdmap e27: 4 total, 4 up, 4 in 2024-07-31T14:42:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-31T14:42:49.870 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.b/config 2024-07-31T14:42:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:50 smithi073 ceph-mon[36714]: purged_snaps scrub starts 2024-07-31T14:42:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:50 smithi073 ceph-mon[36714]: purged_snaps scrub ok 2024-07-31T14:42:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:50 smithi073 ceph-mon[36714]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:42:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:50 smithi073 ceph-mon[36714]: Detected new or changed devices on smithi067 2024-07-31T14:42:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:42:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:42:50.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:50 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi067 to 8544M 2024-07-31T14:42:50.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:50.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:50.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:50.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:50.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:50 smithi073 ceph-mon[36714]: osdmap e28: 4 total, 4 up, 4 in 2024-07-31T14:42:50.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:50 smithi067 ceph-mon[32750]: purged_snaps scrub starts 2024-07-31T14:42:50.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:50 smithi067 ceph-mon[32750]: purged_snaps scrub ok 2024-07-31T14:42:50.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:50 smithi067 ceph-mon[32750]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:42:50.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:50 smithi067 ceph-mon[32750]: Detected new or changed devices on smithi067 2024-07-31T14:42:50.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:50.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:50.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:42:50.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:42:50.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:50 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi067 to 8544M 2024-07-31T14:42:50.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:50.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:50.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:50.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:50.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:50 smithi067 ceph-mon[32750]: osdmap e28: 4 total, 4 up, 4 in 2024-07-31T14:42:50.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:50 smithi077 ceph-mon[37327]: purged_snaps scrub starts 2024-07-31T14:42:50.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:50 smithi077 ceph-mon[37327]: purged_snaps scrub ok 2024-07-31T14:42:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:50 smithi077 ceph-mon[37327]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:42:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:50 smithi077 ceph-mon[37327]: Detected new or changed devices on smithi067 2024-07-31T14:42:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:42:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:42:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:50 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi067 to 8544M 2024-07-31T14:42:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:42:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:42:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:50 smithi077 ceph-mon[37327]: osdmap e28: 4 total, 4 up, 4 in 2024-07-31T14:42:51.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.614+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3514509423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ccc108760 msgr2=0x7f4ccc10ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:51.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.614+0000 7f4cd14d4640 1 --2- 172.21.15.73:0/3514509423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ccc108760 0x7f4ccc10ef90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009ca0 tx=0x7f4cc402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:42:51.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.614+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3514509423 shutdown_connections 2024-07-31T14:42:51.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.614+0000 7f4cd14d4640 1 --2- 172.21.15.73:0/3514509423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ccc108760 0x7f4ccc10ef90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:51.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.614+0000 7f4cd14d4640 1 --2- 172.21.15.73:0/3514509423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc107e10 0x7f4ccc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:51.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.614+0000 7f4cd14d4640 1 --2- 172.21.15.73:0/3514509423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ccc1024d0 0x7f4ccc1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:51.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.614+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3514509423 >> 172.21.15.73:0/3514509423 conn(0x7f4ccc079210 msgr2=0x7f4ccc079620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:42:51.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.614+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3514509423 shutdown_connections 2024-07-31T14:42:51.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.614+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3514509423 wait complete. 2024-07-31T14:42:51.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.615+0000 7f4cd14d4640 1 Processor -- start 2024-07-31T14:42:51.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.615+0000 7f4cd14d4640 1 -- start start 2024-07-31T14:42:51.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.615+0000 7f4cd14d4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc1024d0 0x7f4ccc199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:42:51.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.615+0000 7f4cd14d4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ccc107e10 0x7f4ccc19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:42:51.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.615+0000 7f4cd14d4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ccc108760 0x7f4ccc19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:42:51.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.615+0000 7f4cd14d4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc1123c0 con 0x7f4ccc1024d0 2024-07-31T14:42:51.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.615+0000 7f4cd14d4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc112240 con 0x7f4ccc108760 2024-07-31T14:42:51.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.615+0000 7f4cd14d4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc112540 con 0x7f4ccc107e10 2024-07-31T14:42:51.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.615+0000 7f4ccb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ccc108760 0x7f4ccc19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:42:51.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.615+0000 7f4ccb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ccc108760 0x7f4ccc19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59102/0 (socket says 172.21.15.73:59102) 2024-07-31T14:42:51.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.615+0000 7f4ccb7fe640 1 -- 172.21.15.73:0/3915022662 learned_addr learned my addr 172.21.15.73:0/3915022662 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-07-31T14:42:51.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.615+0000 7f4ccaffd640 1 --2- 172.21.15.73:0/3915022662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc1024d0 0x7f4ccc199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:42:51.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.615+0000 7f4cca7fc640 1 --2- 172.21.15.73:0/3915022662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ccc107e10 0x7f4ccc19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:42:51.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.616+0000 7f4ccb7fe640 1 -- 172.21.15.73:0/3915022662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ccc107e10 msgr2=0x7f4ccc19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:51.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.616+0000 7f4ccb7fe640 1 --2- 172.21.15.73:0/3915022662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ccc107e10 0x7f4ccc19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:51.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.616+0000 7f4ccb7fe640 1 -- 172.21.15.73:0/3915022662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc1024d0 msgr2=0x7f4ccc199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:42:51.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.616+0000 7f4ccb7fe640 1 --2- 172.21.15.73:0/3915022662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc1024d0 0x7f4ccc199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:42:51.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.616+0000 7f4ccb7fe640 1 -- 172.21.15.73:0/3915022662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ccc19fd70 con 0x7f4ccc108760 2024-07-31T14:42:51.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.616+0000 7f4ccb7fe640 1 --2- 172.21.15.73:0/3915022662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ccc108760 0x7f4ccc19f640 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4005cc0 tx=0x7f4cc402f380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:42:51.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.616+0000 7f4cb3fff640 1 -- 172.21.15.73:0/3915022662 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4003bf0 con 0x7f4ccc108760 2024-07-31T14:42:51.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.616+0000 7f4cb3fff640 1 -- 172.21.15.73:0/3915022662 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cc4003d90 con 0x7f4ccc108760 2024-07-31T14:42:51.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.616+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3915022662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ccc1a0000 con 0x7f4ccc108760 2024-07-31T14:42:51.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.616+0000 7f4cb3fff640 1 -- 172.21.15.73:0/3915022662 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc402fbc0 con 0x7f4ccc108760 2024-07-31T14:42:51.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.616+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3915022662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ccc1a67c0 con 0x7f4ccc108760 2024-07-31T14:42:51.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.617+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3915022662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ccc1028e0 con 0x7f4ccc108760 2024-07-31T14:42:51.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.619+0000 7f4cb3fff640 1 -- 172.21.15.73:0/3915022662 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f4cc402fd60 con 0x7f4ccc108760 2024-07-31T14:42:51.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.619+0000 7f4cb3fff640 1 --2- 172.21.15.73:0/3915022662 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4cb40b3c20 0x7f4cb40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:42:51.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.620+0000 7f4cb3fff640 1 -- 172.21.15.73:0/3915022662 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3884+0+0 (secure 0 0 0) 0x7f4cc4104340 con 0x7f4ccc108760 2024-07-31T14:42:51.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.620+0000 7f4ccaffd640 1 --2- 172.21.15.73:0/3915022662 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4cb40b3c20 0x7f4cb40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:42:51.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.620+0000 7f4ccaffd640 1 --2- 172.21.15.73:0/3915022662 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4cb40b3c20 0x7f4cb40b6130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00bbe0 tx=0x7f4cbc00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:42:51.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.623+0000 7f4cb3fff640 1 -- 172.21.15.73:0/3915022662 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f4cc40ccae0 con 0x7f4ccc108760 2024-07-31T14:42:51.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:42:51.820+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3915022662 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4ccc1016a0 con 0x7f4cb40b3c20 2024-07-31T14:42:51.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:51 smithi077 ceph-mon[37327]: osdmap e29: 4 total, 4 up, 4 in 2024-07-31T14:42:52.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:51 smithi067 ceph-mon[32750]: osdmap e29: 4 total, 4 up, 4 in 2024-07-31T14:42:52.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:51 smithi073 ceph-mon[36714]: osdmap e29: 4 total, 4 up, 4 in 2024-07-31T14:42:52.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:52 smithi073 ceph-mon[36714]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:42:52.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:52 smithi073 ceph-mon[36714]: from='client.24386 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:42:52.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:42:52.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:42:52.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:52.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:52 smithi077 ceph-mon[37327]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:42:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:52 smithi077 ceph-mon[37327]: from='client.24386 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:42:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:42:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:42:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:53.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:52 smithi067 ceph-mon[32750]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:42:53.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:52 smithi067 ceph-mon[32750]: from='client.24386 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:42:53.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:42:53.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:42:53.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:42:54.838 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:54 smithi073 ceph-mon[36714]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:42:54.838 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:54 smithi073 ceph-mon[36714]: from='client.? 172.21.15.73:0/3885639001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17975063-5f3d-4a52-84f4-6d54ce4fce34"}]: dispatch 2024-07-31T14:42:54.839 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:54 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17975063-5f3d-4a52-84f4-6d54ce4fce34"}]: dispatch 2024-07-31T14:42:54.839 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:54 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17975063-5f3d-4a52-84f4-6d54ce4fce34"}]': finished 2024-07-31T14:42:54.839 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:54 smithi073 ceph-mon[36714]: osdmap e30: 5 total, 4 up, 5 in 2024-07-31T14:42:54.839 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:54 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:42:54.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:54 smithi077 ceph-mon[37327]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:42:54.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:54 smithi077 ceph-mon[37327]: from='client.? 172.21.15.73:0/3885639001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17975063-5f3d-4a52-84f4-6d54ce4fce34"}]: dispatch 2024-07-31T14:42:54.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:54 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17975063-5f3d-4a52-84f4-6d54ce4fce34"}]: dispatch 2024-07-31T14:42:54.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:54 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17975063-5f3d-4a52-84f4-6d54ce4fce34"}]': finished 2024-07-31T14:42:54.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:54 smithi077 ceph-mon[37327]: osdmap e30: 5 total, 4 up, 5 in 2024-07-31T14:42:54.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:54 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:42:55.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:54 smithi067 ceph-mon[32750]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:42:55.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:54 smithi067 ceph-mon[32750]: from='client.? 172.21.15.73:0/3885639001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17975063-5f3d-4a52-84f4-6d54ce4fce34"}]: dispatch 2024-07-31T14:42:55.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:54 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17975063-5f3d-4a52-84f4-6d54ce4fce34"}]: dispatch 2024-07-31T14:42:55.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:54 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17975063-5f3d-4a52-84f4-6d54ce4fce34"}]': finished 2024-07-31T14:42:55.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:54 smithi067 ceph-mon[32750]: osdmap e30: 5 total, 4 up, 5 in 2024-07-31T14:42:55.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:54 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:42:55.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:55 smithi077 ceph-mon[37327]: from='client.? 172.21.15.73:0/1974261326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:42:56.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:55 smithi067 ceph-mon[32750]: from='client.? 172.21.15.73:0/1974261326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:42:56.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:55 smithi073 ceph-mon[36714]: from='client.? 172.21.15.73:0/1974261326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:42:56.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:56 smithi077 ceph-mon[37327]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-31T14:42:57.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:56 smithi067 ceph-mon[32750]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-31T14:42:57.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:56 smithi073 ceph-mon[36714]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-31T14:42:58.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:42:58 smithi077 ceph-mon[37327]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-07-31T14:42:59.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:42:58 smithi073 ceph-mon[36714]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-07-31T14:42:59.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:42:58 smithi067 ceph-mon[32750]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-07-31T14:43:00.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:00 smithi077 ceph-mon[37327]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-07-31T14:43:01.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:00 smithi067 ceph-mon[32750]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-07-31T14:43:01.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:00 smithi073 ceph-mon[36714]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-07-31T14:43:01.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:01 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-31T14:43:01.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:01 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:01.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:01 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-31T14:43:01.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:01 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:02.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:01 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-31T14:43:02.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:01 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:02.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:02 smithi073 ceph-mon[36714]: Deploying daemon osd.4 on smithi073 2024-07-31T14:43:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:02 smithi073 ceph-mon[36714]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:43:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:02 smithi067 ceph-mon[32750]: Deploying daemon osd.4 on smithi073 2024-07-31T14:43:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:02 smithi067 ceph-mon[32750]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:43:03.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:02 smithi077 ceph-mon[37327]: Deploying daemon osd.4 on smithi073 2024-07-31T14:43:03.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:02 smithi077 ceph-mon[37327]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:43:04.882 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:04 smithi073 ceph-mon[36714]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:43:05.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:04 smithi067 ceph-mon[32750]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:43:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:04 smithi077 ceph-mon[37327]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:43:06.997 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:06 smithi073 ceph-mon[36714]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:06.997 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:06.997 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:06.997 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:07.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:06 smithi067 ceph-mon[32750]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:07.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:07.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:07.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:07.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:06 smithi077 ceph-mon[37327]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:07.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:07.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:07.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:09.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:08 smithi067 ceph-mon[32750]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:09.121 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:08 smithi073 ceph-mon[36714]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:09.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:08 smithi077 ceph-mon[37327]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:10.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:43:10.165+0000 7f4cb3fff640 1 -- 172.21.15.73:0/3915022662 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4ccc1016a0 con 0x7f4cb40b3c20 2024-07-31T14:43:10.167 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 4 on host 'smithi073' 2024-07-31T14:43:10.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:43:10.167+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3915022662 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4cb40b3c20 msgr2=0x7f4cb40b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:43:10.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:43:10.167+0000 7f4cd14d4640 1 --2- 172.21.15.73:0/3915022662 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4cb40b3c20 0x7f4cb40b6130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00bbe0 tx=0x7f4cbc00b390 comp rx=0 tx=0).stop 2024-07-31T14:43:10.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:43:10.167+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3915022662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ccc108760 msgr2=0x7f4ccc19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:43:10.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:43:10.167+0000 7f4cd14d4640 1 --2- 172.21.15.73:0/3915022662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ccc108760 0x7f4ccc19f640 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4005cc0 tx=0x7f4cc402f380 comp rx=0 tx=0).stop 2024-07-31T14:43:10.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:43:10.167+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3915022662 shutdown_connections 2024-07-31T14:43:10.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:43:10.167+0000 7f4cd14d4640 1 --2- 172.21.15.73:0/3915022662 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4cb40b3c20 0x7f4cb40b6130 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:10.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:43:10.167+0000 7f4cd14d4640 1 --2- 172.21.15.73:0/3915022662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ccc108760 0x7f4ccc19f640 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:10.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:43:10.167+0000 7f4cd14d4640 1 --2- 172.21.15.73:0/3915022662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ccc107e10 0x7f4ccc19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:10.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:43:10.167+0000 7f4cd14d4640 1 --2- 172.21.15.73:0/3915022662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc1024d0 0x7f4ccc199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:10.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:43:10.167+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3915022662 >> 172.21.15.73:0/3915022662 conn(0x7f4ccc079210 msgr2=0x7f4ccc100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:43:10.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:43:10.167+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3915022662 shutdown_connections 2024-07-31T14:43:10.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:43:10.167+0000 7f4cd14d4640 1 -- 172.21.15.73:0/3915022662 wait complete. 2024-07-31T14:43:10.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:10 smithi073 ceph-mon[36714]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:10 smithi077 ceph-mon[37327]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:10.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:10 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:10 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:10 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:10.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:10 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:10.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:10 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:10 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:10.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:10 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:10 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:10 smithi067 ceph-mon[32750]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:10.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:10 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:10 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:10 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:10.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:10 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:10.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:10 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:10 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:10.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:10 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:10 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:10.872 DEBUG:teuthology.orchestra.run.smithi073:osd.4> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.4.service 2024-07-31T14:43:10.874 INFO:tasks.cephadm:Deploying osd.5 on smithi077 with /dev/vg_nvme/lv_3... 2024-07-31T14:43:10.875 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-31T14:43:11.135 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T14:43:11.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:11 smithi073 ceph-mon[36714]: from='osd.4 [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-31T14:43:11.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:11 smithi073 ceph-mon[36714]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-31T14:43:11.555 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:11 smithi077 ceph-mon[37327]: from='osd.4 [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-31T14:43:11.555 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:11 smithi077 ceph-mon[37327]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-31T14:43:11.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:11 smithi067 ceph-mon[32750]: from='osd.4 [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-31T14:43:11.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:11 smithi067 ceph-mon[32750]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-31T14:43:12.559 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:12 smithi077 ceph-mon[37327]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:12.559 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:12 smithi077 ceph-mon[37327]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-31T14:43:12.559 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:12 smithi077 ceph-mon[37327]: osdmap e31: 5 total, 4 up, 5 in 2024-07-31T14:43:12.559 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:12 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:43:12.559 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:12 smithi077 ceph-mon[37327]: from='osd.4 [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:43:12.559 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:12 smithi077 ceph-mon[37327]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:43:12.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:12 smithi073 ceph-mon[36714]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:12.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:12 smithi073 ceph-mon[36714]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-31T14:43:12.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:12 smithi073 ceph-mon[36714]: osdmap e31: 5 total, 4 up, 5 in 2024-07-31T14:43:12.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:12 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:43:12.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:12 smithi073 ceph-mon[36714]: from='osd.4 [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:43:12.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:12 smithi073 ceph-mon[36714]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:43:12.644 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 14:43:12 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4[49482]: 2024-07-31T14:43:12.344+0000 7f571333e640 -1 osd.4 0 waiting for initial osdmap 2024-07-31T14:43:12.644 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 14:43:12 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4[49482]: 2024-07-31T14:43:12.349+0000 7f570f13c640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-31T14:43:12.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:12 smithi067 ceph-mon[32750]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:12.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:12 smithi067 ceph-mon[32750]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-31T14:43:12.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:12 smithi067 ceph-mon[32750]: osdmap e31: 5 total, 4 up, 5 in 2024-07-31T14:43:12.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:12 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:43:12.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:12 smithi067 ceph-mon[32750]: from='osd.4 [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:43:12.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:12 smithi067 ceph-mon[32750]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:43:12.880 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-31T14:43:12.968 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-31T14:43:12.981 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-07-31T14:43:12.981 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-07-31T14:43:12.981 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011222 s, 934 MB/s 2024-07-31T14:43:12.981 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-07-31T14:43:13.635 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:13 smithi073 ceph-mon[36714]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-07-31T14:43:13.635 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:13 smithi073 ceph-mon[36714]: osdmap e32: 5 total, 4 up, 5 in 2024-07-31T14:43:13.635 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:13 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:43:13.635 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:13 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:43:13.699 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:13 smithi077 ceph-mon[37327]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-07-31T14:43:13.699 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:13 smithi077 ceph-mon[37327]: osdmap e32: 5 total, 4 up, 5 in 2024-07-31T14:43:13.699 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:13 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:43:13.699 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:13 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:43:13.727 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_3 2024-07-31T14:43:13.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:13 smithi067 ceph-mon[32750]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-07-31T14:43:13.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:13 smithi067 ceph-mon[32750]: osdmap e32: 5 total, 4 up, 5 in 2024-07-31T14:43:13.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:13 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:43:13.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:13 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:43:13.978 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T14:43:14.418 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: purged_snaps scrub starts 2024-07-31T14:43:14.419 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: purged_snaps scrub ok 2024-07-31T14:43:14.419 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:14.419 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: osd.4 [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579] boot 2024-07-31T14:43:14.419 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: osdmap e33: 5 total, 5 up, 5 in 2024-07-31T14:43:14.419 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:43:14.419 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: Detected new or changed devices on smithi073 2024-07-31T14:43:14.419 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:14.419 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:14.419 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:43:14.419 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:43:14.419 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi073 to 8544M 2024-07-31T14:43:14.419 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:14.420 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:14.420 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:14.420 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:14.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: purged_snaps scrub starts 2024-07-31T14:43:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: purged_snaps scrub ok 2024-07-31T14:43:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: osd.4 [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579] boot 2024-07-31T14:43:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: osdmap e33: 5 total, 5 up, 5 in 2024-07-31T14:43:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:43:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: Detected new or changed devices on smithi073 2024-07-31T14:43:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:43:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:43:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi073 to 8544M 2024-07-31T14:43:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:14.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: purged_snaps scrub starts 2024-07-31T14:43:14.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: purged_snaps scrub ok 2024-07-31T14:43:14.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-07-31T14:43:14.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: osd.4 [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579] boot 2024-07-31T14:43:14.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: osdmap e33: 5 total, 5 up, 5 in 2024-07-31T14:43:14.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-31T14:43:14.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: Detected new or changed devices on smithi073 2024-07-31T14:43:14.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:14.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:14.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:43:14.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:43:14.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi073 to 8544M 2024-07-31T14:43:14.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:14.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:14.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:14.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:15.403 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:15 smithi077 ceph-mon[37327]: osdmap e34: 5 total, 5 up, 5 in 2024-07-31T14:43:15.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.615+0000 7f506483d640 1 -- 172.21.15.77:0/1789113708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5060101b10 msgr2=0x7f5060101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:43:15.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.615+0000 7f506483d640 1 --2- 172.21.15.77:0/1789113708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5060101b10 0x7f5060101f20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5048009930 tx=0x7f504802a420 comp rx=0 tx=0).stop 2024-07-31T14:43:15.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.615+0000 7f506483d640 1 -- 172.21.15.77:0/1789113708 shutdown_connections 2024-07-31T14:43:15.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.615+0000 7f506483d640 1 --2- 172.21.15.77:0/1789113708 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5060106650 0x7f50601100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:15.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.615+0000 7f506483d640 1 --2- 172.21.15.77:0/1789113708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5060105d00 0x7f5060106110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:15.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.615+0000 7f506483d640 1 --2- 172.21.15.77:0/1789113708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5060101b10 0x7f5060101f20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:15.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.615+0000 7f506483d640 1 -- 172.21.15.77:0/1789113708 >> 172.21.15.77:0/1789113708 conn(0x7f50600799c0 msgr2=0x7f5060079dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:43:15.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.615+0000 7f506483d640 1 -- 172.21.15.77:0/1789113708 shutdown_connections 2024-07-31T14:43:15.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.615+0000 7f506483d640 1 -- 172.21.15.77:0/1789113708 wait complete. 2024-07-31T14:43:15.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.616+0000 7f506483d640 1 Processor -- start 2024-07-31T14:43:15.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.616+0000 7f506483d640 1 -- start start 2024-07-31T14:43:15.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.616+0000 7f506483d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5060101b10 0x7f5060199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:43:15.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.616+0000 7f506483d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5060105d00 0x7f506019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:43:15.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.616+0000 7f506483d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5060106650 0x7f506019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:43:15.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.616+0000 7f506483d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5060113530 con 0x7f5060105d00 2024-07-31T14:43:15.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.616+0000 7f506483d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f50601133b0 con 0x7f5060101b10 2024-07-31T14:43:15.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.616+0000 7f506483d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f50601136b0 con 0x7f5060106650 2024-07-31T14:43:15.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f505ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5060106650 0x7f506019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:43:15.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f505ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5060106650 0x7f506019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42476/0 (socket says 172.21.15.77:42476) 2024-07-31T14:43:15.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f505ed9d640 1 -- 172.21.15.77:0/426095305 learned_addr learned my addr 172.21.15.77:0/426095305 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-07-31T14:43:15.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f505dd9b640 1 --2- 172.21.15.77:0/426095305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5060105d00 0x7f506019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:43:15.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f505e59c640 1 --2- 172.21.15.77:0/426095305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5060101b10 0x7f5060199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:43:15.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f505ed9d640 1 -- 172.21.15.77:0/426095305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5060101b10 msgr2=0x7f5060199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:43:15.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f505ed9d640 1 --2- 172.21.15.77:0/426095305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5060101b10 0x7f5060199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:15.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f505ed9d640 1 -- 172.21.15.77:0/426095305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5060105d00 msgr2=0x7f506019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:43:15.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f505ed9d640 1 --2- 172.21.15.77:0/426095305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5060105d00 0x7f506019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:15.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f505ed9d640 1 -- 172.21.15.77:0/426095305 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506019fdd0 con 0x7f5060106650 2024-07-31T14:43:15.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f505ed9d640 1 --2- 172.21.15.77:0/426095305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5060106650 0x7f506019f760 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f505800cce0 tx=0x7f505800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:43:15.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f50577fe640 1 -- 172.21.15.77:0/426095305 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505800f450 con 0x7f5060106650 2024-07-31T14:43:15.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f50577fe640 1 -- 172.21.15.77:0/426095305 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f505800faa0 con 0x7f5060106650 2024-07-31T14:43:15.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f50577fe640 1 -- 172.21.15.77:0/426095305 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058018a80 con 0x7f5060106650 2024-07-31T14:43:15.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f506483d640 1 -- 172.21.15.77:0/426095305 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50601a00c0 con 0x7f5060106650 2024-07-31T14:43:15.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.617+0000 7f506483d640 1 -- 172.21.15.77:0/426095305 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50601a68e0 con 0x7f5060106650 2024-07-31T14:43:15.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.619+0000 7f506483d640 1 -- 172.21.15.77:0/426095305 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f506006b220 con 0x7f5060106650 2024-07-31T14:43:15.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.620+0000 7f50577fe640 1 -- 172.21.15.77:0/426095305 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f505800fc50 con 0x7f5060106650 2024-07-31T14:43:15.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.621+0000 7f50577fe640 1 --2- 172.21.15.77:0/426095305 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f504c0b3c50 0x7f504c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:43:15.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.621+0000 7f505e59c640 1 --2- 172.21.15.77:0/426095305 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f504c0b3c50 0x7f504c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:43:15.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.621+0000 7f50577fe640 1 -- 172.21.15.77:0/426095305 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4316+0+0 (secure 0 0 0) 0x7f50580e3520 con 0x7f5060106650 2024-07-31T14:43:15.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.622+0000 7f505e59c640 1 --2- 172.21.15.77:0/426095305 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f504c0b3c50 0x7f504c0b6160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5048009900 tx=0x7f5048000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:43:15.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.624+0000 7f50577fe640 1 -- 172.21.15.77:0/426095305 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5058014030 con 0x7f5060106650 2024-07-31T14:43:15.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:15 smithi073 ceph-mon[36714]: osdmap e34: 5 total, 5 up, 5 in 2024-07-31T14:43:15.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:15 smithi067 ceph-mon[32750]: osdmap e34: 5 total, 5 up, 5 in 2024-07-31T14:43:15.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:15.816+0000 7f506483d640 1 -- 172.21.15.77:0/426095305 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f506010b770 con 0x7f504c0b3c50 2024-07-31T14:43:16.486 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:16 smithi077 ceph-mon[37327]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:16.486 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:16 smithi077 ceph-mon[37327]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:43:16.486 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:43:16.486 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:43:16.486 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:16 smithi073 ceph-mon[36714]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:16 smithi073 ceph-mon[36714]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:43:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:43:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:43:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:16.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:16 smithi067 ceph-mon[32750]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:16.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:16 smithi067 ceph-mon[32750]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:43:16.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:43:16.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:43:16.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:18.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:18 smithi073 ceph-mon[36714]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:18.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:18 smithi073 ceph-mon[36714]: from='client.? 172.21.15.77:0/1538372191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e41e714e-62fa-4946-923b-56f8ee66fb86"}]: dispatch 2024-07-31T14:43:18.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:18 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e41e714e-62fa-4946-923b-56f8ee66fb86"}]: dispatch 2024-07-31T14:43:18.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:18 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e41e714e-62fa-4946-923b-56f8ee66fb86"}]': finished 2024-07-31T14:43:18.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:18 smithi073 ceph-mon[36714]: osdmap e35: 6 total, 5 up, 6 in 2024-07-31T14:43:18.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:18 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:18.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:18 smithi077 ceph-mon[37327]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:18 smithi077 ceph-mon[37327]: from='client.? 172.21.15.77:0/1538372191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e41e714e-62fa-4946-923b-56f8ee66fb86"}]: dispatch 2024-07-31T14:43:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:18 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e41e714e-62fa-4946-923b-56f8ee66fb86"}]: dispatch 2024-07-31T14:43:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:18 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e41e714e-62fa-4946-923b-56f8ee66fb86"}]': finished 2024-07-31T14:43:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:18 smithi077 ceph-mon[37327]: osdmap e35: 6 total, 5 up, 6 in 2024-07-31T14:43:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:18 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:18.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:18 smithi067 ceph-mon[32750]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:18.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:18 smithi067 ceph-mon[32750]: from='client.? 172.21.15.77:0/1538372191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e41e714e-62fa-4946-923b-56f8ee66fb86"}]: dispatch 2024-07-31T14:43:18.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:18 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e41e714e-62fa-4946-923b-56f8ee66fb86"}]: dispatch 2024-07-31T14:43:18.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:18 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e41e714e-62fa-4946-923b-56f8ee66fb86"}]': finished 2024-07-31T14:43:18.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:18 smithi067 ceph-mon[32750]: osdmap e35: 6 total, 5 up, 6 in 2024-07-31T14:43:18.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:18 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:19.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:19 smithi073 ceph-mon[36714]: from='client.? 172.21.15.77:0/2794964524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:43:19.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:19 smithi077 ceph-mon[37327]: from='client.? 172.21.15.77:0/2794964524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:43:19.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:19 smithi067 ceph-mon[32750]: from='client.? 172.21.15.77:0/2794964524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:43:20.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:20 smithi073 ceph-mon[36714]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:20.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:20 smithi077 ceph-mon[37327]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:20.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:20 smithi067 ceph-mon[32750]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:22.617 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:22 smithi077 ceph-mon[37327]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:22.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:22 smithi073 ceph-mon[36714]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:22.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:22 smithi067 ceph-mon[32750]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:24.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:24 smithi073 ceph-mon[36714]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:24.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:24 smithi077 ceph-mon[37327]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:24.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:24 smithi067 ceph-mon[32750]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:25 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-31T14:43:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:25 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:25.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:25 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-31T14:43:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:25 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:25.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:25 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-31T14:43:25.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:25 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:26.584 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:26 smithi077 ceph-mon[37327]: Deploying daemon osd.5 on smithi077 2024-07-31T14:43:26.584 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:26 smithi077 ceph-mon[37327]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:26.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:26 smithi073 ceph-mon[36714]: Deploying daemon osd.5 on smithi077 2024-07-31T14:43:26.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:26 smithi073 ceph-mon[36714]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:26.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:26 smithi067 ceph-mon[32750]: Deploying daemon osd.5 on smithi077 2024-07-31T14:43:26.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:26 smithi067 ceph-mon[32750]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:28.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:28 smithi077 ceph-mon[37327]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:28.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:28 smithi073 ceph-mon[36714]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:28.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:28 smithi067 ceph-mon[32750]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:30.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:30 smithi077 ceph-mon[37327]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:30 smithi067 ceph-mon[32750]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:30.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:30 smithi073 ceph-mon[36714]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:31 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:31 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:31 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:31 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:31 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:31 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:31.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:31 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:31.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:31 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:31.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:31 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:32.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:32 smithi067 ceph-mon[32750]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:32.867 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:32 smithi077 ceph-mon[37327]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:32.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:32 smithi073 ceph-mon[36714]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:34.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:34.165+0000 7f50577fe640 1 -- 172.21.15.77:0/426095305 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f506010b770 con 0x7f504c0b3c50 2024-07-31T14:43:34.167 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 5 on host 'smithi077' 2024-07-31T14:43:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:34.168+0000 7f506483d640 1 -- 172.21.15.77:0/426095305 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f504c0b3c50 msgr2=0x7f504c0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:43:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:34.168+0000 7f506483d640 1 --2- 172.21.15.77:0/426095305 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f504c0b3c50 0x7f504c0b6160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5048009900 tx=0x7f5048000bc0 comp rx=0 tx=0).stop 2024-07-31T14:43:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:34.168+0000 7f506483d640 1 -- 172.21.15.77:0/426095305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5060106650 msgr2=0x7f506019f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:43:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:34.168+0000 7f506483d640 1 --2- 172.21.15.77:0/426095305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5060106650 0x7f506019f760 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f505800cce0 tx=0x7f505800a620 comp rx=0 tx=0).stop 2024-07-31T14:43:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:34.168+0000 7f506483d640 1 -- 172.21.15.77:0/426095305 shutdown_connections 2024-07-31T14:43:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:34.168+0000 7f506483d640 1 --2- 172.21.15.77:0/426095305 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f504c0b3c50 0x7f504c0b6160 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:34.168+0000 7f506483d640 1 --2- 172.21.15.77:0/426095305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5060106650 0x7f506019f760 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:34.168+0000 7f506483d640 1 --2- 172.21.15.77:0/426095305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5060105d00 0x7f506019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:34.168+0000 7f506483d640 1 --2- 172.21.15.77:0/426095305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5060101b10 0x7f5060199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:34.168+0000 7f506483d640 1 -- 172.21.15.77:0/426095305 >> 172.21.15.77:0/426095305 conn(0x7f50600799c0 msgr2=0x7f5060106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:43:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:34.168+0000 7f506483d640 1 -- 172.21.15.77:0/426095305 shutdown_connections 2024-07-31T14:43:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:43:34.168+0000 7f506483d640 1 -- 172.21.15.77:0/426095305 wait complete. 2024-07-31T14:43:34.429 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:34.429 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:34.429 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:34.429 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:34.430 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:34.430 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:34 smithi077 ceph-mon[37327]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:34.430 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:34.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:34.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:34.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:34.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:34.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:34.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:34 smithi067 ceph-mon[32750]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:34.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:34 smithi073 ceph-mon[36714]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:34.880 DEBUG:teuthology.orchestra.run.smithi077:osd.5> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.5.service 2024-07-31T14:43:34.883 INFO:tasks.cephadm:Deploying osd.6 on smithi067 with /dev/vg_nvme/lv_2... 2024-07-31T14:43:34.883 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-31T14:43:35.159 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:43:35.283 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:35 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:35.283 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:35 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:35.283 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:35 smithi067 ceph-mon[32750]: from='osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-31T14:43:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:35 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:35 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:35 smithi077 ceph-mon[37327]: from='osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-31T14:43:35.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:35 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:35.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:35 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:35.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:35 smithi073 ceph-mon[36714]: from='osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-31T14:43:36.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:36 smithi077 ceph-mon[37327]: from='osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-31T14:43:36.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:36 smithi077 ceph-mon[37327]: osdmap e36: 6 total, 5 up, 6 in 2024-07-31T14:43:36.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:36 smithi077 ceph-mon[37327]: from='osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:43:36.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:36 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:36.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:36 smithi077 ceph-mon[37327]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:36.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 14:43:36 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T14:43:36.176+0000 7fa2935ca640 -1 osd.5 0 waiting for initial osdmap 2024-07-31T14:43:36.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 14:43:36 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T14:43:36.184+0000 7fa28f3c8640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-31T14:43:36.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:36 smithi067 ceph-mon[32750]: from='osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-31T14:43:36.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:36 smithi067 ceph-mon[32750]: osdmap e36: 6 total, 5 up, 6 in 2024-07-31T14:43:36.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:36 smithi067 ceph-mon[32750]: from='osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:43:36.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:36 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:36.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:36 smithi067 ceph-mon[32750]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:36.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:36 smithi073 ceph-mon[36714]: from='osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-31T14:43:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:36 smithi073 ceph-mon[36714]: osdmap e36: 6 total, 5 up, 6 in 2024-07-31T14:43:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:36 smithi073 ceph-mon[36714]: from='osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:43:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:36 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:36 smithi073 ceph-mon[36714]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:37.109 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-31T14:43:37.201 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-31T14:43:37.214 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-07-31T14:43:37.214 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-07-31T14:43:37.215 INFO:teuthology.orchestra.run.smithi067.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114654 s, 915 MB/s 2024-07-31T14:43:37.215 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-07-31T14:43:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:37 smithi077 ceph-mon[37327]: from='osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-07-31T14:43:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:37 smithi077 ceph-mon[37327]: osdmap e37: 6 total, 5 up, 6 in 2024-07-31T14:43:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:37 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:37 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:37.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:37 smithi067 ceph-mon[32750]: from='osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-07-31T14:43:37.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:37 smithi067 ceph-mon[32750]: osdmap e37: 6 total, 5 up, 6 in 2024-07-31T14:43:37.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:37 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:37.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:37 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:37.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:37 smithi073 ceph-mon[36714]: from='osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-07-31T14:43:37.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:37 smithi073 ceph-mon[36714]: osdmap e37: 6 total, 5 up, 6 in 2024-07-31T14:43:37.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:37 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:37.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:37 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:38.148 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi067:vg_nvme/lv_2 2024-07-31T14:43:38.414 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:43:38.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: purged_snaps scrub starts 2024-07-31T14:43:38.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: purged_snaps scrub ok 2024-07-31T14:43:38.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:38.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212] boot 2024-07-31T14:43:38.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: osdmap e38: 6 total, 6 up, 6 in 2024-07-31T14:43:38.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:38.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:38.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: Detected new or changed devices on smithi077 2024-07-31T14:43:38.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:38.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:38.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:43:38.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:43:38.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi077 to 8544M 2024-07-31T14:43:38.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:38.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:38.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:38.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:38.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: purged_snaps scrub starts 2024-07-31T14:43:38.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: purged_snaps scrub ok 2024-07-31T14:43:38.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:38.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212] boot 2024-07-31T14:43:38.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: osdmap e38: 6 total, 6 up, 6 in 2024-07-31T14:43:38.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:38.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:38.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: Detected new or changed devices on smithi077 2024-07-31T14:43:38.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:38.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:38.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:43:38.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:43:38.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi077 to 8544M 2024-07-31T14:43:38.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:38.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:38.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:38.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: purged_snaps scrub starts 2024-07-31T14:43:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: purged_snaps scrub ok 2024-07-31T14:43:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: osd.5 [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212] boot 2024-07-31T14:43:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: osdmap e38: 6 total, 6 up, 6 in 2024-07-31T14:43:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-31T14:43:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-07-31T14:43:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: Detected new or changed devices on smithi077 2024-07-31T14:43:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:43:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:43:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi077 to 8544M 2024-07-31T14:43:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:39.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:39 smithi077 ceph-mon[37327]: osdmap e39: 6 total, 6 up, 6 in 2024-07-31T14:43:39.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:39 smithi067 ceph-mon[32750]: osdmap e39: 6 total, 6 up, 6 in 2024-07-31T14:43:39.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:39 smithi073 ceph-mon[36714]: osdmap e39: 6 total, 6 up, 6 in 2024-07-31T14:43:40.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.137+0000 7f0133ac1640 1 -- 172.21.15.67:0/1050965914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f012c107e10 msgr2=0x7f012c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:43:40.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.137+0000 7f0133ac1640 1 --2- 172.21.15.67:0/1050965914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f012c107e10 0x7f012c108220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f011c009900 tx=0x7f011c02a410 comp rx=0 tx=0).stop 2024-07-31T14:43:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.138+0000 7f0133ac1640 1 -- 172.21.15.67:0/1050965914 shutdown_connections 2024-07-31T14:43:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.138+0000 7f0133ac1640 1 --2- 172.21.15.67:0/1050965914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f012c108760 0x7f012c10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.138+0000 7f0133ac1640 1 --2- 172.21.15.67:0/1050965914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f012c107e10 0x7f012c108220 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.138+0000 7f0133ac1640 1 --2- 172.21.15.67:0/1050965914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c1024d0 0x7f012c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.138+0000 7f0133ac1640 1 -- 172.21.15.67:0/1050965914 >> 172.21.15.67:0/1050965914 conn(0x7f012c079180 msgr2=0x7f012c079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:43:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.138+0000 7f0133ac1640 1 -- 172.21.15.67:0/1050965914 shutdown_connections 2024-07-31T14:43:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.138+0000 7f0133ac1640 1 -- 172.21.15.67:0/1050965914 wait complete. 2024-07-31T14:43:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.138+0000 7f0133ac1640 1 Processor -- start 2024-07-31T14:43:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f0133ac1640 1 -- start start 2024-07-31T14:43:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f0133ac1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c1024d0 0x7f012c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:43:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f0133ac1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f012c107e10 0x7f012c19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:43:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f0133ac1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f012c108760 0x7f012c19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:43:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f0133ac1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f012c1123d0 con 0x7f012c107e10 2024-07-31T14:43:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f0133ac1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f012c112250 con 0x7f012c1024d0 2024-07-31T14:43:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f0133ac1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f012c112550 con 0x7f012c108760 2024-07-31T14:43:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f013105c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f012c107e10 0x7f012c19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:43:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f013105c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f012c107e10 0x7f012c19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34092/0 (socket says 172.21.15.67:34092) 2024-07-31T14:43:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f013105c640 1 -- 172.21.15.67:0/179031810 learned_addr learned my addr 172.21.15.67:0/179031810 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:43:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f013205e640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f012c108760 0x7f012c19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:43:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f013185d640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c1024d0 0x7f012c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:43:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f013105c640 1 -- 172.21.15.67:0/179031810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f012c108760 msgr2=0x7f012c19b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:43:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f013105c640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f012c108760 0x7f012c19b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f013105c640 1 -- 172.21.15.67:0/179031810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c1024d0 msgr2=0x7f012c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:43:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f013105c640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c1024d0 0x7f012c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f013105c640 1 -- 172.21.15.67:0/179031810 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012c1a5fd0 con 0x7f012c107e10 2024-07-31T14:43:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.139+0000 7f013185d640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c1024d0 0x7f012c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:43:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.140+0000 7f013105c640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f012c107e10 0x7f012c19b280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f011c006010 tx=0x7f011c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:43:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.140+0000 7f0122ffd640 1 -- 172.21.15.67:0/179031810 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c03f030 con 0x7f012c107e10 2024-07-31T14:43:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.140+0000 7f0133ac1640 1 -- 172.21.15.67:0/179031810 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012c1a6260 con 0x7f012c107e10 2024-07-31T14:43:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.140+0000 7f0122ffd640 1 -- 172.21.15.67:0/179031810 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f011c003aa0 con 0x7f012c107e10 2024-07-31T14:43:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.140+0000 7f0122ffd640 1 -- 172.21.15.67:0/179031810 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c035070 con 0x7f012c107e10 2024-07-31T14:43:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.140+0000 7f0133ac1640 1 -- 172.21.15.67:0/179031810 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f012c1a6790 con 0x7f012c107e10 2024-07-31T14:43:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.141+0000 7f0133ac1640 1 -- 172.21.15.67:0/179031810 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f012c1028e0 con 0x7f012c107e10 2024-07-31T14:43:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.142+0000 7f0122ffd640 1 -- 172.21.15.67:0/179031810 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f011c042030 con 0x7f012c107e10 2024-07-31T14:43:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.143+0000 7f0122ffd640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f01180b3c50 0x7f01180b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:43:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.143+0000 7f013185d640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f01180b3c50 0x7f01180b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:43:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.143+0000 7f0122ffd640 1 -- 172.21.15.67:0/179031810 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4748+0+0 (secure 0 0 0) 0x7f011c103750 con 0x7f012c107e10 2024-07-31T14:43:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.144+0000 7f013185d640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f01180b3c50 0x7f01180b6160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f012c06c0d0 tx=0x7f0128005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:43:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.147+0000 7f0122ffd640 1 -- 172.21.15.67:0/179031810 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f011c0cbb90 con 0x7f012c107e10 2024-07-31T14:43:40.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:40.352+0000 7f0133ac1640 1 -- 172.21.15.67:0/179031810 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f012c10a570 con 0x7f01180b3c50 2024-07-31T14:43:40.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:40 smithi077 ceph-mon[37327]: osdmap e40: 6 total, 6 up, 6 in 2024-07-31T14:43:40.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:40 smithi077 ceph-mon[37327]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:43:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:40 smithi067 ceph-mon[32750]: osdmap e40: 6 total, 6 up, 6 in 2024-07-31T14:43:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:40 smithi067 ceph-mon[32750]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:43:40.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:40 smithi073 ceph-mon[36714]: osdmap e40: 6 total, 6 up, 6 in 2024-07-31T14:43:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:40 smithi073 ceph-mon[36714]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:43:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:41 smithi067 ceph-mon[32750]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:43:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:41 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:43:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:41 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:43:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:41 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:41.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:41 smithi077 ceph-mon[37327]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:43:41.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:41 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:43:41.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:41 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:43:41.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:41 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:41.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:41 smithi073 ceph-mon[36714]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:43:41.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:41 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:43:41.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:41 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:43:41.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:41 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:42.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:42 smithi077 ceph-mon[37327]: pgmap v129: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:43:42.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:42 smithi067 ceph-mon[32750]: pgmap v129: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:43:42.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:42 smithi073 ceph-mon[36714]: pgmap v129: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:43:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:43 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1076719111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b20c9a1c-1462-4596-915a-70c38b3b8c76"}]: dispatch 2024-07-31T14:43:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:43 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1076719111' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b20c9a1c-1462-4596-915a-70c38b3b8c76"}]': finished 2024-07-31T14:43:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:43 smithi067 ceph-mon[32750]: osdmap e41: 7 total, 6 up, 7 in 2024-07-31T14:43:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:43 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:43:43.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:43 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1076719111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b20c9a1c-1462-4596-915a-70c38b3b8c76"}]: dispatch 2024-07-31T14:43:43.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:43 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1076719111' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b20c9a1c-1462-4596-915a-70c38b3b8c76"}]': finished 2024-07-31T14:43:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:43 smithi077 ceph-mon[37327]: osdmap e41: 7 total, 6 up, 7 in 2024-07-31T14:43:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:43 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:43:43.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:43 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1076719111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b20c9a1c-1462-4596-915a-70c38b3b8c76"}]: dispatch 2024-07-31T14:43:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:43 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1076719111' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b20c9a1c-1462-4596-915a-70c38b3b8c76"}]': finished 2024-07-31T14:43:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:43 smithi073 ceph-mon[36714]: osdmap e41: 7 total, 6 up, 7 in 2024-07-31T14:43:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:43 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:43:44.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:44 smithi077 ceph-mon[37327]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:43:44.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:44 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/655525394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:43:44.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:44 smithi067 ceph-mon[32750]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:43:44.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:44 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/655525394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:43:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:44 smithi073 ceph-mon[36714]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:43:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:44 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/655525394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:43:46.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:46 smithi073 ceph-mon[36714]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-07-31T14:43:46.702 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:46 smithi067 ceph-mon[32750]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-07-31T14:43:46.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:46 smithi077 ceph-mon[37327]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-07-31T14:43:48.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:48 smithi067 ceph-mon[32750]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-07-31T14:43:48.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:48 smithi073 ceph-mon[36714]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-07-31T14:43:48.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:48 smithi077 ceph-mon[37327]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-07-31T14:43:50.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:50 smithi067 ceph-mon[32750]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:43:50.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-31T14:43:50.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:50.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:50 smithi067 ceph-mon[32750]: Deploying daemon osd.6 on smithi067 2024-07-31T14:43:50.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:50 smithi073 ceph-mon[36714]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:43:50.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-31T14:43:50.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:50.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:50 smithi073 ceph-mon[36714]: Deploying daemon osd.6 on smithi067 2024-07-31T14:43:50.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:50 smithi077 ceph-mon[37327]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:43:50.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-31T14:43:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:50 smithi077 ceph-mon[37327]: Deploying daemon osd.6 on smithi067 2024-07-31T14:43:52.534 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:52 smithi067 ceph-mon[32750]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:43:52.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:52 smithi073 ceph-mon[36714]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:43:52.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:52 smithi077 ceph-mon[37327]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:43:54.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:54 smithi073 ceph-mon[36714]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-07-31T14:43:54.735 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:54 smithi067 ceph-mon[32750]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-07-31T14:43:54.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:54 smithi077 ceph-mon[37327]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-07-31T14:43:55.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:55 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:55.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:55 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:55.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:55 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:55.768 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:55 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:55.768 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:55 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:55.768 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:55 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:55.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:55 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:55.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:55 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:55.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:55 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:56.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:56 smithi077 ceph-mon[37327]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-31T14:43:56.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:56 smithi067 ceph-mon[32750]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-31T14:43:56.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:56 smithi073 ceph-mon[36714]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-31T14:43:58.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:58 smithi077 ceph-mon[37327]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:43:58.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:58.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:58.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:58 smithi067 ceph-mon[32750]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:43:58.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:58.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:58.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:58 smithi073 ceph-mon[36714]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:43:58.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:58.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:59.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:59.240+0000 7f0122ffd640 1 -- 172.21.15.67:0/179031810 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f012c10a570 con 0x7f01180b3c50 2024-07-31T14:43:59.243 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 6 on host 'smithi067' 2024-07-31T14:43:59.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:59.243+0000 7f0133ac1640 1 -- 172.21.15.67:0/179031810 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f01180b3c50 msgr2=0x7f01180b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:43:59.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:59.243+0000 7f0133ac1640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f01180b3c50 0x7f01180b6160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f012c06c0d0 tx=0x7f0128005f90 comp rx=0 tx=0).stop 2024-07-31T14:43:59.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:59.243+0000 7f0133ac1640 1 -- 172.21.15.67:0/179031810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f012c107e10 msgr2=0x7f012c19b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:43:59.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:59.243+0000 7f0133ac1640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f012c107e10 0x7f012c19b280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f011c006010 tx=0x7f011c004060 comp rx=0 tx=0).stop 2024-07-31T14:43:59.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:59.243+0000 7f0133ac1640 1 -- 172.21.15.67:0/179031810 shutdown_connections 2024-07-31T14:43:59.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:59.243+0000 7f0133ac1640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f01180b3c50 0x7f01180b6160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:59.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:59.243+0000 7f0133ac1640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f012c108760 0x7f012c19b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:59.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:59.243+0000 7f0133ac1640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f012c107e10 0x7f012c19b280 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:59.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:59.243+0000 7f0133ac1640 1 --2- 172.21.15.67:0/179031810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c1024d0 0x7f012c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:43:59.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:59.243+0000 7f0133ac1640 1 -- 172.21.15.67:0/179031810 >> 172.21.15.67:0/179031810 conn(0x7f012c079180 msgr2=0x7f012c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:43:59.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:59.243+0000 7f0133ac1640 1 -- 172.21.15.67:0/179031810 shutdown_connections 2024-07-31T14:43:59.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:43:59.243+0000 7f0133ac1640 1 -- 172.21.15.67:0/179031810 wait complete. 2024-07-31T14:43:59.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:59 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:59.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:59 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:59.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:59 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:59.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:59 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:59.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:59 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:59.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:59 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:59.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:43:59 smithi067 ceph-mon[32750]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-31T14:43:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:59 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:59 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:59 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:59 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:59 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:59 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:43:59 smithi077 ceph-mon[37327]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-31T14:43:59.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:59 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:43:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:59 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:43:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:59 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:59 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:43:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:59 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:59 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:43:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:43:59 smithi073 ceph-mon[36714]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-31T14:43:59.977 DEBUG:teuthology.orchestra.run.smithi067:osd.6> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.6.service 2024-07-31T14:43:59.979 INFO:tasks.cephadm:Deploying osd.7 on smithi073 with /dev/vg_nvme/lv_2... 2024-07-31T14:43:59.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-31T14:44:00.240 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.b/config 2024-07-31T14:44:00.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:00 smithi067 ceph-mon[32750]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:44:00.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:00 smithi067 ceph-mon[32750]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-31T14:44:00.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:00 smithi067 ceph-mon[32750]: osdmap e42: 7 total, 6 up, 7 in 2024-07-31T14:44:00.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:00 smithi067 ceph-mon[32750]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-31T14:44:00.518 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:00.618 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:00 smithi073 ceph-mon[36714]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:44:00.619 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:00 smithi073 ceph-mon[36714]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-31T14:44:00.619 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:00 smithi073 ceph-mon[36714]: osdmap e42: 7 total, 6 up, 7 in 2024-07-31T14:44:00.619 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:00 smithi073 ceph-mon[36714]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-31T14:44:00.619 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:00 smithi077 ceph-mon[37327]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:44:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:00 smithi077 ceph-mon[37327]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-31T14:44:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:00 smithi077 ceph-mon[37327]: osdmap e42: 7 total, 6 up, 7 in 2024-07-31T14:44:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:00 smithi077 ceph-mon[37327]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-31T14:44:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:02.004 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-31T14:44:02.090 INFO:teuthology.orchestra.run.smithi073.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-31T14:44:02.105 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10+0 records in 2024-07-31T14:44:02.105 INFO:teuthology.orchestra.run.smithi073.stderr:10+0 records out 2024-07-31T14:44:02.106 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134922 s, 777 MB/s 2024-07-31T14:44:02.106 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping successful for: 2024-07-31T14:44:02.254 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 14:44:01 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T14:44:01.986+0000 7fd9d8775640 -1 osd.6 0 waiting for initial osdmap 2024-07-31T14:44:02.254 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 14:44:01 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T14:44:01.998+0000 7fd9d056b640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-31T14:44:02.404 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:02 smithi073 ceph-mon[36714]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-31T14:44:02.404 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:02 smithi073 ceph-mon[36714]: osdmap e43: 7 total, 6 up, 7 in 2024-07-31T14:44:02.404 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:02.404 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:02.404 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:02 smithi073 ceph-mon[36714]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:44:02.404 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:02 smithi073 ceph-mon[36714]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' 2024-07-31T14:44:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:02 smithi067 ceph-mon[32750]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-31T14:44:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:02 smithi067 ceph-mon[32750]: osdmap e43: 7 total, 6 up, 7 in 2024-07-31T14:44:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:02 smithi067 ceph-mon[32750]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:44:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:02 smithi067 ceph-mon[32750]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' 2024-07-31T14:44:02.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:02 smithi077 ceph-mon[37327]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-31T14:44:02.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:02 smithi077 ceph-mon[37327]: osdmap e43: 7 total, 6 up, 7 in 2024-07-31T14:44:02.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:02.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:02.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:02 smithi077 ceph-mon[37327]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-07-31T14:44:02.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:02 smithi077 ceph-mon[37327]: from='osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157]' entity='osd.6' 2024-07-31T14:44:02.794 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_2 2024-07-31T14:44:03.049 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.b/config 2024-07-31T14:44:03.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:03 smithi067 ceph-mon[32750]: purged_snaps scrub starts 2024-07-31T14:44:03.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:03 smithi067 ceph-mon[32750]: purged_snaps scrub ok 2024-07-31T14:44:03.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:03 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:03.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:03 smithi067 ceph-mon[32750]: osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157] boot 2024-07-31T14:44:03.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:03 smithi067 ceph-mon[32750]: osdmap e44: 7 total, 7 up, 7 in 2024-07-31T14:44:03.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:03 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:03.490 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:03 smithi073 ceph-mon[36714]: purged_snaps scrub starts 2024-07-31T14:44:03.491 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:03 smithi073 ceph-mon[36714]: purged_snaps scrub ok 2024-07-31T14:44:03.491 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:03 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:03.491 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:03 smithi073 ceph-mon[36714]: osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157] boot 2024-07-31T14:44:03.491 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:03 smithi073 ceph-mon[36714]: osdmap e44: 7 total, 7 up, 7 in 2024-07-31T14:44:03.491 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:03 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:03.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:03 smithi077 ceph-mon[37327]: purged_snaps scrub starts 2024-07-31T14:44:03.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:03 smithi077 ceph-mon[37327]: purged_snaps scrub ok 2024-07-31T14:44:03.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:03 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:03.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:03 smithi077 ceph-mon[37327]: osd.6 [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157] boot 2024-07-31T14:44:03.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:03 smithi077 ceph-mon[37327]: osdmap e44: 7 total, 7 up, 7 in 2024-07-31T14:44:03.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:03 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-31T14:44:04.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.664+0000 7f89586b1640 1 -- 172.21.15.73:0/1599722238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89501029d0 msgr2=0x7f8950102de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:44:04.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.664+0000 7f89586b1640 1 --2- 172.21.15.73:0/1599722238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89501029d0 0x7f8950102de0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8940009900 tx=0x7f894002a410 comp rx=0 tx=0).stop 2024-07-31T14:44:04.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.664+0000 7f89586b1640 1 -- 172.21.15.73:0/1599722238 shutdown_connections 2024-07-31T14:44:04.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.664+0000 7f89586b1640 1 --2- 172.21.15.73:0/1599722238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8950103320 0x7f895010eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:04.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.664+0000 7f89586b1640 1 --2- 172.21.15.73:0/1599722238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89501029d0 0x7f8950102de0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:04.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.664+0000 7f89586b1640 1 --2- 172.21.15.73:0/1599722238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8950107060 0x7f8950107470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.664+0000 7f89586b1640 1 -- 172.21.15.73:0/1599722238 >> 172.21.15.73:0/1599722238 conn(0x7f8950079870 msgr2=0x7f8950079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:44:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.664+0000 7f89586b1640 1 -- 172.21.15.73:0/1599722238 shutdown_connections 2024-07-31T14:44:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.664+0000 7f89586b1640 1 -- 172.21.15.73:0/1599722238 wait complete. 2024-07-31T14:44:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.665+0000 7f89586b1640 1 Processor -- start 2024-07-31T14:44:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.665+0000 7f89586b1640 1 -- start start 2024-07-31T14:44:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.665+0000 7f89586b1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89501029d0 0x7f895019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:44:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.665+0000 7f89586b1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8950103320 0x7f895019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:44:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.665+0000 7f89586b1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8950107060 0x7f895019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:44:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.665+0000 7f89586b1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f89501124d0 con 0x7f8950107060 2024-07-31T14:44:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.665+0000 7f89586b1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8950112350 con 0x7f89501029d0 2024-07-31T14:44:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.665+0000 7f89586b1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8950112650 con 0x7f8950103320 2024-07-31T14:44:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f895644d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89501029d0 0x7f895019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:44:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f895644d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89501029d0 0x7f895019ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39192/0 (socket says 172.21.15.73:39192) 2024-07-31T14:44:04.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f895644d640 1 -- 172.21.15.73:0/3816749260 learned_addr learned my addr 172.21.15.73:0/3816749260 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-07-31T14:44:04.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f8955c4c640 1 --2- 172.21.15.73:0/3816749260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8950103320 0x7f895019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:44:04.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f8956c4e640 1 --2- 172.21.15.73:0/3816749260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8950107060 0x7f895019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:44:04.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f895644d640 1 -- 172.21.15.73:0/3816749260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8950103320 msgr2=0x7f895019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:44:04.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f895644d640 1 --2- 172.21.15.73:0/3816749260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8950103320 0x7f895019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:04.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f895644d640 1 -- 172.21.15.73:0/3816749260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8950107060 msgr2=0x7f895019b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:44:04.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f895644d640 1 --2- 172.21.15.73:0/3816749260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8950107060 0x7f895019b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:04.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f895644d640 1 -- 172.21.15.73:0/3816749260 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89501a60f0 con 0x7f89501029d0 2024-07-31T14:44:04.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f895644d640 1 --2- 172.21.15.73:0/3816749260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89501029d0 0x7f895019ae60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f894c00b4a0 tx=0x7f894c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:44:04.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f89477fe640 1 -- 172.21.15.73:0/3816749260 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c014070 con 0x7f89501029d0 2024-07-31T14:44:04.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f89586b1640 1 -- 172.21.15.73:0/3816749260 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89501a63e0 con 0x7f89501029d0 2024-07-31T14:44:04.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f89477fe640 1 -- 172.21.15.73:0/3816749260 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f894c010ac0 con 0x7f89501029d0 2024-07-31T14:44:04.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f89477fe640 1 -- 172.21.15.73:0/3816749260 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c017c30 con 0x7f89501029d0 2024-07-31T14:44:04.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.666+0000 7f89586b1640 1 -- 172.21.15.73:0/3816749260 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89501a6970 con 0x7f89501029d0 2024-07-31T14:44:04.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.667+0000 7f89586b1640 1 -- 172.21.15.73:0/3816749260 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8950102e60 con 0x7f89501029d0 2024-07-31T14:44:04.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.669+0000 7f89477fe640 1 -- 172.21.15.73:0/3816749260 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f894c017070 con 0x7f89501029d0 2024-07-31T14:44:04.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.670+0000 7f89477fe640 1 --2- 172.21.15.73:0/3816749260 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f893c0b3c50 0x7f893c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:44:04.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.670+0000 7f89477fe640 1 -- 172.21.15.73:0/3816749260 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5180+0+0 (secure 0 0 0) 0x7f894c0e27f0 con 0x7f89501029d0 2024-07-31T14:44:04.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.670+0000 7f8955c4c640 1 --2- 172.21.15.73:0/3816749260 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f893c0b3c50 0x7f893c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:44:04.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.670+0000 7f8955c4c640 1 --2- 172.21.15.73:0/3816749260 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f893c0b3c50 0x7f893c0b6160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8940009230 tx=0x7f8940031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:44:04.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.673+0000 7f89477fe640 1 -- 172.21.15.73:0/3816749260 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f894c0aa9e0 con 0x7f89501029d0 2024-07-31T14:44:04.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:04 smithi077 ceph-mon[37327]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:04.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:04 smithi077 ceph-mon[37327]: Detected new or changed devices on smithi067 2024-07-31T14:44:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:04 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi067 to 5696M 2024-07-31T14:44:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:04 smithi077 ceph-mon[37327]: osdmap e45: 7 total, 7 up, 7 in 2024-07-31T14:44:04.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:04 smithi067 ceph-mon[32750]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:04.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:04 smithi067 ceph-mon[32750]: Detected new or changed devices on smithi067 2024-07-31T14:44:04.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:04.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:04 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi067 to 5696M 2024-07-31T14:44:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:04 smithi067 ceph-mon[32750]: osdmap e45: 7 total, 7 up, 7 in 2024-07-31T14:44:04.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:04.865+0000 7f89586b1640 1 -- 172.21.15.73:0/3816749260 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8950063230 con 0x7f893c0b3c50 2024-07-31T14:44:04.867 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:04 smithi073 ceph-mon[36714]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:04.867 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:04 smithi073 ceph-mon[36714]: Detected new or changed devices on smithi067 2024-07-31T14:44:04.867 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:04.868 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:04.868 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:04.868 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:04.868 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:04.868 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:04 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi067 to 5696M 2024-07-31T14:44:04.868 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:04.868 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:04.868 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:04.868 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:04.868 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:04 smithi073 ceph-mon[36714]: osdmap e45: 7 total, 7 up, 7 in 2024-07-31T14:44:05.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:05 smithi073 ceph-mon[36714]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:44:05.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:44:05.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:44:05.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:05 smithi077 ceph-mon[37327]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:44:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:44:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:44:05.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:05.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:05 smithi067 ceph-mon[32750]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:44:05.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:44:05.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:44:05.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:06.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:06 smithi073 ceph-mon[36714]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:06.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:06 smithi077 ceph-mon[37327]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:06.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:06 smithi067 ceph-mon[32750]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:07.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:07 smithi067 ceph-mon[32750]: from='client.? 172.21.15.73:0/3627039945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb608dc-dd03-44a4-8e6f-1fce67757c0d"}]: dispatch 2024-07-31T14:44:07.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:07 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb608dc-dd03-44a4-8e6f-1fce67757c0d"}]: dispatch 2024-07-31T14:44:07.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:07 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cb608dc-dd03-44a4-8e6f-1fce67757c0d"}]': finished 2024-07-31T14:44:07.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:07 smithi067 ceph-mon[32750]: osdmap e46: 8 total, 7 up, 8 in 2024-07-31T14:44:07.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:07 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:07.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:07 smithi073 ceph-mon[36714]: from='client.? 172.21.15.73:0/3627039945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb608dc-dd03-44a4-8e6f-1fce67757c0d"}]: dispatch 2024-07-31T14:44:07.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:07 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb608dc-dd03-44a4-8e6f-1fce67757c0d"}]: dispatch 2024-07-31T14:44:07.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:07 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cb608dc-dd03-44a4-8e6f-1fce67757c0d"}]': finished 2024-07-31T14:44:07.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:07 smithi073 ceph-mon[36714]: osdmap e46: 8 total, 7 up, 8 in 2024-07-31T14:44:07.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:07 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:07.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:07 smithi077 ceph-mon[37327]: from='client.? 172.21.15.73:0/3627039945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb608dc-dd03-44a4-8e6f-1fce67757c0d"}]: dispatch 2024-07-31T14:44:07.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:07 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb608dc-dd03-44a4-8e6f-1fce67757c0d"}]: dispatch 2024-07-31T14:44:07.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:07 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cb608dc-dd03-44a4-8e6f-1fce67757c0d"}]': finished 2024-07-31T14:44:07.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:07 smithi077 ceph-mon[37327]: osdmap e46: 8 total, 7 up, 8 in 2024-07-31T14:44:07.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:07 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:08.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:08 smithi067 ceph-mon[32750]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:08.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:08 smithi067 ceph-mon[32750]: from='client.? 172.21.15.73:0/341981779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:44:08.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:08 smithi073 ceph-mon[36714]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:08.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:08 smithi073 ceph-mon[36714]: from='client.? 172.21.15.73:0/341981779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:44:08.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:08 smithi077 ceph-mon[37327]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:08.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:08 smithi077 ceph-mon[37327]: from='client.? 172.21.15.73:0/341981779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:44:10.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:10 smithi067 ceph-mon[32750]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:10.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:10 smithi073 ceph-mon[36714]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:10 smithi077 ceph-mon[37327]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:12.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:12 smithi073 ceph-mon[36714]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:12.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:12 smithi067 ceph-mon[32750]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:12.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:12 smithi077 ceph-mon[37327]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:14.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:14 smithi073 ceph-mon[36714]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-31T14:44:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:14.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:14 smithi067 ceph-mon[32750]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:14.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-31T14:44:14.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:14.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:14 smithi077 ceph-mon[37327]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:14.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-31T14:44:14.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:15.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:15 smithi067 ceph-mon[32750]: Deploying daemon osd.7 on smithi073 2024-07-31T14:44:15.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:15 smithi073 ceph-mon[36714]: Deploying daemon osd.7 on smithi073 2024-07-31T14:44:15.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:15 smithi077 ceph-mon[37327]: Deploying daemon osd.7 on smithi073 2024-07-31T14:44:16.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:16 smithi067 ceph-mon[32750]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:16.827 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:16 smithi073 ceph-mon[36714]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:16.888 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:16 smithi077 ceph-mon[37327]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:18.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:18 smithi067 ceph-mon[32750]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:18.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:18 smithi073 ceph-mon[36714]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:18.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:18 smithi077 ceph-mon[37327]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:20.664 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:20 smithi073 ceph-mon[36714]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:20.664 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:44:20.664 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:20.664 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:20.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:20 smithi067 ceph-mon[32750]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:20.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:44:20.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:20.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:20.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:20 smithi077 ceph-mon[37327]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:20.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:44:20.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:20.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:22.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:22 smithi067 ceph-mon[32750]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:22.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:22 smithi073 ceph-mon[36714]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:22.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:22 smithi077 ceph-mon[37327]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:23.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:23.386+0000 7f89477fe640 1 -- 172.21.15.73:0/3816749260 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8950063230 con 0x7f893c0b3c50 2024-07-31T14:44:23.388 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 7 on host 'smithi073' 2024-07-31T14:44:23.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:23.388+0000 7f89586b1640 1 -- 172.21.15.73:0/3816749260 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f893c0b3c50 msgr2=0x7f893c0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:44:23.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:23.388+0000 7f89586b1640 1 --2- 172.21.15.73:0/3816749260 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f893c0b3c50 0x7f893c0b6160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8940009230 tx=0x7f8940031400 comp rx=0 tx=0).stop 2024-07-31T14:44:23.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:23.388+0000 7f89586b1640 1 -- 172.21.15.73:0/3816749260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89501029d0 msgr2=0x7f895019ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:44:23.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:23.388+0000 7f89586b1640 1 --2- 172.21.15.73:0/3816749260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89501029d0 0x7f895019ae60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f894c00b4a0 tx=0x7f894c00b960 comp rx=0 tx=0).stop 2024-07-31T14:44:23.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:23.388+0000 7f89586b1640 1 -- 172.21.15.73:0/3816749260 shutdown_connections 2024-07-31T14:44:23.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:23.388+0000 7f89586b1640 1 --2- 172.21.15.73:0/3816749260 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f893c0b3c50 0x7f893c0b6160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:23.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:23.388+0000 7f89586b1640 1 --2- 172.21.15.73:0/3816749260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8950107060 0x7f895019b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:23.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:23.388+0000 7f89586b1640 1 --2- 172.21.15.73:0/3816749260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8950103320 0x7f895019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:23.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:23.388+0000 7f89586b1640 1 --2- 172.21.15.73:0/3816749260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89501029d0 0x7f895019ae60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:23.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:23.388+0000 7f89586b1640 1 -- 172.21.15.73:0/3816749260 >> 172.21.15.73:0/3816749260 conn(0x7f8950079870 msgr2=0x7f8950103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:44:23.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:23.388+0000 7f89586b1640 1 -- 172.21.15.73:0/3816749260 shutdown_connections 2024-07-31T14:44:23.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:44:23.388+0000 7f89586b1640 1 -- 172.21.15.73:0/3816749260 wait complete. 2024-07-31T14:44:23.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:44:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:23.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:23.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:23.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:23.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:23.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:23.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:44:23.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:23.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:23.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:23.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:44:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:24.039 DEBUG:teuthology.orchestra.run.smithi073:osd.7> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.7.service 2024-07-31T14:44:24.041 INFO:tasks.cephadm:Deploying osd.8 on smithi077 with /dev/vg_nvme/lv_2... 2024-07-31T14:44:24.041 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-31T14:44:24.300 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T14:44:24.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:24 smithi077 ceph-mon[37327]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:24.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:24 smithi077 ceph-mon[37327]: from='osd.7 [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-31T14:44:24.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:24 smithi077 ceph-mon[37327]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-31T14:44:24.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:24 smithi067 ceph-mon[32750]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:24.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:24 smithi067 ceph-mon[32750]: from='osd.7 [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-31T14:44:24.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:24 smithi067 ceph-mon[32750]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-31T14:44:24.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:24 smithi073 ceph-mon[36714]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:24 smithi073 ceph-mon[36714]: from='osd.7 [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-31T14:44:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:24 smithi073 ceph-mon[36714]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-31T14:44:25.682 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:25 smithi077 ceph-mon[37327]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-31T14:44:25.682 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:25 smithi077 ceph-mon[37327]: osdmap e47: 8 total, 7 up, 8 in 2024-07-31T14:44:25.682 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:25 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:25.682 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:25 smithi077 ceph-mon[37327]: from='osd.7 [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:44:25.682 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:25 smithi077 ceph-mon[37327]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:44:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:25 smithi073 ceph-mon[36714]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-31T14:44:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:25 smithi073 ceph-mon[36714]: osdmap e47: 8 total, 7 up, 8 in 2024-07-31T14:44:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:25 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:25 smithi073 ceph-mon[36714]: from='osd.7 [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:44:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:25 smithi073 ceph-mon[36714]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:44:26.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:25 smithi067 ceph-mon[32750]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-31T14:44:26.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:25 smithi067 ceph-mon[32750]: osdmap e47: 8 total, 7 up, 8 in 2024-07-31T14:44:26.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:25 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:26.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:25 smithi067 ceph-mon[32750]: from='osd.7 [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:44:26.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:25 smithi067 ceph-mon[32750]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:44:26.057 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-31T14:44:26.141 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-31T14:44:26.154 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-07-31T14:44:26.154 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-07-31T14:44:26.154 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112405 s, 933 MB/s 2024-07-31T14:44:26.154 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-07-31T14:44:26.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:26 smithi073 ceph-mon[36714]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:26 smithi073 ceph-mon[36714]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-07-31T14:44:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:26 smithi073 ceph-mon[36714]: osdmap e48: 8 total, 7 up, 8 in 2024-07-31T14:44:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:26 smithi073 ceph-mon[36714]: from='osd.7 ' entity='osd.7' 2024-07-31T14:44:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:26 smithi073 ceph-mon[36714]: osd.7 [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566] boot 2024-07-31T14:44:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:26 smithi073 ceph-mon[36714]: osdmap e49: 8 total, 8 up, 8 in 2024-07-31T14:44:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:26.644 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 14:44:26 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T14:44:26.361+0000 7fbe428c9640 -1 osd.7 0 waiting for initial osdmap 2024-07-31T14:44:26.644 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 14:44:26 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T14:44:26.369+0000 7fbe3a6bf640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-31T14:44:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:26 smithi077 ceph-mon[37327]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:26 smithi077 ceph-mon[37327]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-07-31T14:44:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:26 smithi077 ceph-mon[37327]: osdmap e48: 8 total, 7 up, 8 in 2024-07-31T14:44:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:26 smithi077 ceph-mon[37327]: from='osd.7 ' entity='osd.7' 2024-07-31T14:44:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:26 smithi077 ceph-mon[37327]: osd.7 [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566] boot 2024-07-31T14:44:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:26 smithi077 ceph-mon[37327]: osdmap e49: 8 total, 8 up, 8 in 2024-07-31T14:44:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:26.837 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_2 2024-07-31T14:44:27.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:26 smithi067 ceph-mon[32750]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-07-31T14:44:27.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:26 smithi067 ceph-mon[32750]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-07-31T14:44:27.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:26 smithi067 ceph-mon[32750]: osdmap e48: 8 total, 7 up, 8 in 2024-07-31T14:44:27.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:27.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:27.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:26 smithi067 ceph-mon[32750]: from='osd.7 ' entity='osd.7' 2024-07-31T14:44:27.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:26 smithi067 ceph-mon[32750]: osd.7 [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566] boot 2024-07-31T14:44:27.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:26 smithi067 ceph-mon[32750]: osdmap e49: 8 total, 8 up, 8 in 2024-07-31T14:44:27.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-31T14:44:27.097 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T14:44:27.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:27 smithi067 ceph-mon[32750]: purged_snaps scrub starts 2024-07-31T14:44:27.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:27 smithi067 ceph-mon[32750]: purged_snaps scrub ok 2024-07-31T14:44:27.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:27.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:27.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:27.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:27.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:27.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:27.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:27.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:27.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:27.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:27 smithi067 ceph-mon[32750]: osdmap e50: 8 total, 8 up, 8 in 2024-07-31T14:44:27.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:27 smithi073 ceph-mon[36714]: purged_snaps scrub starts 2024-07-31T14:44:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:27 smithi073 ceph-mon[36714]: purged_snaps scrub ok 2024-07-31T14:44:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:27 smithi073 ceph-mon[36714]: osdmap e50: 8 total, 8 up, 8 in 2024-07-31T14:44:27.965 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:27 smithi077 ceph-mon[37327]: purged_snaps scrub starts 2024-07-31T14:44:27.966 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:27 smithi077 ceph-mon[37327]: purged_snaps scrub ok 2024-07-31T14:44:27.966 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:27.966 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:27.966 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:27.966 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:27.966 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:27.966 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:27.966 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:27.966 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:27.967 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:27.967 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:27 smithi077 ceph-mon[37327]: osdmap e50: 8 total, 8 up, 8 in 2024-07-31T14:44:28.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.810+0000 7fab0b68c640 1 -- 172.21.15.77:0/1566143888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04108720 msgr2=0x7fab0410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:44:28.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.810+0000 7fab0b68c640 1 --2- 172.21.15.77:0/1566143888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04108720 0x7fab0410ef50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7faafc009ca0 tx=0x7faafc02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:44:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.810+0000 7fab0b68c640 1 -- 172.21.15.77:0/1566143888 shutdown_connections 2024-07-31T14:44:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.810+0000 7fab0b68c640 1 --2- 172.21.15.77:0/1566143888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04108720 0x7fab0410ef50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.810+0000 7fab0b68c640 1 --2- 172.21.15.77:0/1566143888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab04107dd0 0x7fab041081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.810+0000 7fab0b68c640 1 --2- 172.21.15.77:0/1566143888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab04102490 0x7fab041028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.810+0000 7fab0b68c640 1 -- 172.21.15.77:0/1566143888 >> 172.21.15.77:0/1566143888 conn(0x7fab04079160 msgr2=0x7fab04079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:44:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.810+0000 7fab0b68c640 1 -- 172.21.15.77:0/1566143888 shutdown_connections 2024-07-31T14:44:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.810+0000 7fab0b68c640 1 -- 172.21.15.77:0/1566143888 wait complete. 2024-07-31T14:44:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.810+0000 7fab0b68c640 1 Processor -- start 2024-07-31T14:44:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.810+0000 7fab0b68c640 1 -- start start 2024-07-31T14:44:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab0b68c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab04102490 0x7fab0419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:44:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab0b68c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab04107dd0 0x7fab0419b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:44:28.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab0b68c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04108720 0x7fab0419b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:44:28.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab0b68c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fab041123a0 con 0x7fab04107dd0 2024-07-31T14:44:28.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab0b68c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fab04112220 con 0x7fab04102490 2024-07-31T14:44:28.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab0b68c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fab04112520 con 0x7fab04108720 2024-07-31T14:44:28.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab09c29640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04108720 0x7fab0419b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:44:28.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab09c29640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04108720 0x7fab0419b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56752/0 (socket says 172.21.15.77:56752) 2024-07-31T14:44:28.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab09c29640 1 -- 172.21.15.77:0/1111424854 learned_addr learned my addr 172.21.15.77:0/1111424854 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-07-31T14:44:28.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab09428640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab04102490 0x7fab0419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:44:28.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab08c27640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab04107dd0 0x7fab0419b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:44:28.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab09c29640 1 -- 172.21.15.77:0/1111424854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab04102490 msgr2=0x7fab0419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:44:28.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab09c29640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab04102490 0x7fab0419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:28.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab09c29640 1 -- 172.21.15.77:0/1111424854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab04107dd0 msgr2=0x7fab0419b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:44:28.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab09c29640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab04107dd0 0x7fab0419b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:28.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab09c29640 1 -- 172.21.15.77:0/1111424854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab041a5f20 con 0x7fab04108720 2024-07-31T14:44:28.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab08c27640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab04107dd0 0x7fab0419b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:44:28.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7fab09c29640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04108720 0x7fab0419b710 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faafc005cc0 tx=0x7faafc02f380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:44:28.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.811+0000 7faafa7fc640 1 -- 172.21.15.77:0/1111424854 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc003bf0 con 0x7fab04108720 2024-07-31T14:44:28.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.812+0000 7faafa7fc640 1 -- 172.21.15.77:0/1111424854 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faafc003d90 con 0x7fab04108720 2024-07-31T14:44:28.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.812+0000 7fab0b68c640 1 -- 172.21.15.77:0/1111424854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab041a61b0 con 0x7fab04108720 2024-07-31T14:44:28.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.812+0000 7faafa7fc640 1 -- 172.21.15.77:0/1111424854 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc02fbc0 con 0x7fab04108720 2024-07-31T14:44:28.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.812+0000 7fab0b68c640 1 -- 172.21.15.77:0/1111424854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab041a6740 con 0x7fab04108720 2024-07-31T14:44:28.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.813+0000 7fab0b68c640 1 -- 172.21.15.77:0/1111424854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab041028a0 con 0x7fab04108720 2024-07-31T14:44:28.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.815+0000 7faafa7fc640 1 -- 172.21.15.77:0/1111424854 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7faafc02fd60 con 0x7fab04108720 2024-07-31T14:44:28.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.816+0000 7faafa7fc640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faaf00b3cd0 0x7faaf00b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:44:28.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.816+0000 7faafa7fc640 1 -- 172.21.15.77:0/1111424854 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5612+0+0 (secure 0 0 0) 0x7faafc1048f0 con 0x7fab04108720 2024-07-31T14:44:28.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.816+0000 7fab09428640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faaf00b3cd0 0x7faaf00b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:44:28.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.817+0000 7fab09428640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faaf00b3cd0 0x7faaf00b61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fab00005fd0 tx=0x7fab00005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:44:28.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:28.819+0000 7faafa7fc640 1 -- 172.21.15.77:0/1111424854 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7faafc0cc9d0 con 0x7fab04108720 2024-07-31T14:44:28.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:28 smithi073 ceph-mon[36714]: Detected new or changed devices on smithi073 2024-07-31T14:44:28.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:28 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi073 to 5696M 2024-07-31T14:44:28.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:28 smithi073 ceph-mon[36714]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:28.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:28 smithi077 ceph-mon[37327]: Detected new or changed devices on smithi073 2024-07-31T14:44:28.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:28 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi073 to 5696M 2024-07-31T14:44:28.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:28 smithi077 ceph-mon[37327]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:29.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:29.011+0000 7fab0b68c640 1 -- 172.21.15.77:0/1111424854 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fab04063230 con 0x7faaf00b3cd0 2024-07-31T14:44:29.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:28 smithi067 ceph-mon[32750]: Detected new or changed devices on smithi073 2024-07-31T14:44:29.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:28 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi073 to 5696M 2024-07-31T14:44:29.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:28 smithi067 ceph-mon[32750]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:29.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:29 smithi077 ceph-mon[37327]: osdmap e51: 8 total, 8 up, 8 in 2024-07-31T14:44:29.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:44:29.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:44:29.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:29.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:29 smithi073 ceph-mon[36714]: osdmap e51: 8 total, 8 up, 8 in 2024-07-31T14:44:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:44:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:44:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:30.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:29 smithi067 ceph-mon[32750]: osdmap e51: 8 total, 8 up, 8 in 2024-07-31T14:44:30.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:44:30.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:44:30.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:30.688 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:30 smithi077 ceph-mon[37327]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:44:30.688 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:30 smithi077 ceph-mon[37327]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:30.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:30 smithi073 ceph-mon[36714]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:44:30.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:30 smithi073 ceph-mon[36714]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:31.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:30 smithi067 ceph-mon[32750]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:44:31.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:30 smithi067 ceph-mon[32750]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:31.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:31 smithi073 ceph-mon[36714]: from='client.? 172.21.15.77:0/1878901901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b2baf2b-fe02-4048-8adb-de4d19f545d6"}]: dispatch 2024-07-31T14:44:31.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:31 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b2baf2b-fe02-4048-8adb-de4d19f545d6"}]: dispatch 2024-07-31T14:44:31.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:31 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b2baf2b-fe02-4048-8adb-de4d19f545d6"}]': finished 2024-07-31T14:44:31.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:31 smithi073 ceph-mon[36714]: osdmap e52: 9 total, 8 up, 9 in 2024-07-31T14:44:31.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:31 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:31.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:31 smithi077 ceph-mon[37327]: from='client.? 172.21.15.77:0/1878901901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b2baf2b-fe02-4048-8adb-de4d19f545d6"}]: dispatch 2024-07-31T14:44:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:31 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b2baf2b-fe02-4048-8adb-de4d19f545d6"}]: dispatch 2024-07-31T14:44:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:31 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b2baf2b-fe02-4048-8adb-de4d19f545d6"}]': finished 2024-07-31T14:44:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:31 smithi077 ceph-mon[37327]: osdmap e52: 9 total, 8 up, 9 in 2024-07-31T14:44:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:31 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:32.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:31 smithi067 ceph-mon[32750]: from='client.? 172.21.15.77:0/1878901901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b2baf2b-fe02-4048-8adb-de4d19f545d6"}]: dispatch 2024-07-31T14:44:32.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:31 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b2baf2b-fe02-4048-8adb-de4d19f545d6"}]: dispatch 2024-07-31T14:44:32.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:31 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b2baf2b-fe02-4048-8adb-de4d19f545d6"}]': finished 2024-07-31T14:44:32.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:31 smithi067 ceph-mon[32750]: osdmap e52: 9 total, 8 up, 9 in 2024-07-31T14:44:32.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:31 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:32.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:32 smithi073 ceph-mon[36714]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:32.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:32 smithi073 ceph-mon[36714]: from='client.? 172.21.15.77:0/909670226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:44:32.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:32 smithi077 ceph-mon[37327]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:32.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:32 smithi077 ceph-mon[37327]: from='client.? 172.21.15.77:0/909670226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:44:33.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:32 smithi067 ceph-mon[32750]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:33.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:32 smithi067 ceph-mon[32750]: from='client.? 172.21.15.77:0/909670226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:44:34.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:34 smithi073 ceph-mon[36714]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-31T14:44:34.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:34 smithi077 ceph-mon[37327]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-31T14:44:35.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:34 smithi067 ceph-mon[32750]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-31T14:44:36.871 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:36 smithi077 ceph-mon[37327]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-07-31T14:44:36.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:36 smithi073 ceph-mon[36714]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-07-31T14:44:37.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:36 smithi067 ceph-mon[32750]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-07-31T14:44:38.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:38 smithi077 ceph-mon[37327]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-07-31T14:44:38.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-31T14:44:38.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:38.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:38 smithi073 ceph-mon[36714]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-07-31T14:44:38.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-31T14:44:38.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:39.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:38 smithi067 ceph-mon[32750]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-07-31T14:44:39.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-31T14:44:39.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:39.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:39 smithi073 ceph-mon[36714]: Deploying daemon osd.8 on smithi077 2024-07-31T14:44:39.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:39 smithi077 ceph-mon[37327]: Deploying daemon osd.8 on smithi077 2024-07-31T14:44:40.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:39 smithi067 ceph-mon[32750]: Deploying daemon osd.8 on smithi077 2024-07-31T14:44:40.620 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:40 smithi077 ceph-mon[37327]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:44:40.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:40 smithi073 ceph-mon[36714]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:44:41.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:40 smithi067 ceph-mon[32750]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:44:42.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:42 smithi073 ceph-mon[36714]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:44:42.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:42 smithi077 ceph-mon[37327]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:44:43.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:42 smithi067 ceph-mon[32750]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:44:44.869 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:44 smithi077 ceph-mon[37327]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-07-31T14:44:44.869 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:44 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:44:44.869 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:44 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:44.869 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:44 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:44.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:44 smithi073 ceph-mon[36714]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-07-31T14:44:44.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:44 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:44:44.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:44 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:44.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:44 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:45.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:44 smithi067 ceph-mon[32750]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-07-31T14:44:45.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:44 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:44:45.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:44 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:45.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:44 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:46.859 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:46 smithi077 ceph-mon[37327]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:46.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:46 smithi073 ceph-mon[36714]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:47.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:46 smithi067 ceph-mon[32750]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:48.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:48.011+0000 7faafa7fc640 1 -- 172.21.15.77:0/1111424854 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fab04063230 con 0x7faaf00b3cd0 2024-07-31T14:44:48.012 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 8 on host 'smithi077' 2024-07-31T14:44:48.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:48.013+0000 7fab0b68c640 1 -- 172.21.15.77:0/1111424854 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faaf00b3cd0 msgr2=0x7faaf00b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:44:48.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:48.013+0000 7fab0b68c640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faaf00b3cd0 0x7faaf00b61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fab00005fd0 tx=0x7fab00005850 comp rx=0 tx=0).stop 2024-07-31T14:44:48.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:48.013+0000 7fab0b68c640 1 -- 172.21.15.77:0/1111424854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04108720 msgr2=0x7fab0419b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:44:48.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:48.013+0000 7fab0b68c640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04108720 0x7fab0419b710 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faafc005cc0 tx=0x7faafc02f380 comp rx=0 tx=0).stop 2024-07-31T14:44:48.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:48.013+0000 7fab0b68c640 1 -- 172.21.15.77:0/1111424854 shutdown_connections 2024-07-31T14:44:48.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:48.013+0000 7fab0b68c640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faaf00b3cd0 0x7faaf00b61e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:48.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:48.013+0000 7fab0b68c640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04108720 0x7fab0419b710 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:48.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:48.013+0000 7fab0b68c640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab04107dd0 0x7fab0419b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:48.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:48.013+0000 7fab0b68c640 1 --2- 172.21.15.77:0/1111424854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab04102490 0x7fab0419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:48.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:48.013+0000 7fab0b68c640 1 -- 172.21.15.77:0/1111424854 >> 172.21.15.77:0/1111424854 conn(0x7fab04079160 msgr2=0x7fab04104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:44:48.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:48.013+0000 7fab0b68c640 1 -- 172.21.15.77:0/1111424854 shutdown_connections 2024-07-31T14:44:48.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:44:48.013+0000 7fab0b68c640 1 -- 172.21.15.77:0/1111424854 wait complete. 2024-07-31T14:44:48.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:48.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:48.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:48.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:44:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:48.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:48.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:48.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:48.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:48.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:48.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:44:48.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:48.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:48.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:44:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:48.809 DEBUG:teuthology.orchestra.run.smithi077:osd.8> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.8.service 2024-07-31T14:44:48.812 INFO:tasks.cephadm:Deploying osd.9 on smithi067 with /dev/vg_nvme/lv_1... 2024-07-31T14:44:48.812 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-31T14:44:49.090 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:49 smithi077 ceph-mon[37327]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:49.090 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:49.090 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:49 smithi077 ceph-mon[37327]: from='osd.8 [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-31T14:44:49.091 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:49 smithi077 ceph-mon[37327]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-31T14:44:49.093 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:44:49.216 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:49 smithi067 ceph-mon[32750]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:49.217 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:49.217 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:49 smithi067 ceph-mon[32750]: from='osd.8 [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-31T14:44:49.217 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:49 smithi067 ceph-mon[32750]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-31T14:44:49.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:49 smithi073 ceph-mon[36714]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:49.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:49.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:49 smithi073 ceph-mon[36714]: from='osd.8 [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-31T14:44:49.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:49 smithi073 ceph-mon[36714]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-31T14:44:50.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:50 smithi073 ceph-mon[36714]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-31T14:44:50.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:50 smithi073 ceph-mon[36714]: from='osd.8 [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:44:50.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:50 smithi073 ceph-mon[36714]: osdmap e53: 9 total, 8 up, 9 in 2024-07-31T14:44:50.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:50.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:50 smithi073 ceph-mon[36714]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:44:50.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:50 smithi073 ceph-mon[36714]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:50.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:50 smithi067 ceph-mon[32750]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-31T14:44:50.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:50 smithi067 ceph-mon[32750]: from='osd.8 [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:44:50.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:50 smithi067 ceph-mon[32750]: osdmap e53: 9 total, 8 up, 9 in 2024-07-31T14:44:50.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:50.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:50 smithi067 ceph-mon[32750]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:44:50.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:50 smithi067 ceph-mon[32750]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:50.403 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:50 smithi077 ceph-mon[37327]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-31T14:44:50.403 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:50 smithi077 ceph-mon[37327]: from='osd.8 [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:44:50.403 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:50 smithi077 ceph-mon[37327]: osdmap e53: 9 total, 8 up, 9 in 2024-07-31T14:44:50.403 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:50.403 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:50 smithi077 ceph-mon[37327]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:44:50.403 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:50 smithi077 ceph-mon[37327]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:50.404 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 14:44:50 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T14:44:50.032+0000 7fc2cc9b2640 -1 osd.8 0 waiting for initial osdmap 2024-07-31T14:44:50.404 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 14:44:50 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T14:44:50.046+0000 7fc2c87b0640 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-31T14:44:51.187 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-31T14:44:51.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:51 smithi067 ceph-mon[32750]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-07-31T14:44:51.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:51 smithi067 ceph-mon[32750]: osdmap e54: 9 total, 8 up, 9 in 2024-07-31T14:44:51.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:51 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:51.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:51 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-31T14:44:51.279 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-07-31T14:44:51.279 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-07-31T14:44:51.280 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113226 s, 926 MB/s 2024-07-31T14:44:51.280 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-07-31T14:44:51.309 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:51 smithi073 ceph-mon[36714]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-07-31T14:44:51.310 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:51 smithi073 ceph-mon[36714]: osdmap e54: 9 total, 8 up, 9 in 2024-07-31T14:44:51.310 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:51 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:51.310 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:51 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:51.450 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:51 smithi077 ceph-mon[37327]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-07-31T14:44:51.450 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:51 smithi077 ceph-mon[37327]: osdmap e54: 9 total, 8 up, 9 in 2024-07-31T14:44:51.450 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:51 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:51.450 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:51 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:51.876 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi067:vg_nvme/lv_1 2024-07-31T14:44:52.158 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:44:52.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: purged_snaps scrub starts 2024-07-31T14:44:52.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: purged_snaps scrub ok 2024-07-31T14:44:52.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:52.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: osd.8 [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754] boot 2024-07-31T14:44:52.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: osdmap e55: 9 total, 9 up, 9 in 2024-07-31T14:44:52.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:52.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:52.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: Detected new or changed devices on smithi077 2024-07-31T14:44:52.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:52.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:52.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:52.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:52.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:52.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi077 to 5696M 2024-07-31T14:44:52.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:52.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:52.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:52.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:52.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: purged_snaps scrub starts 2024-07-31T14:44:52.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: purged_snaps scrub ok 2024-07-31T14:44:52.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:52.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: osd.8 [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754] boot 2024-07-31T14:44:52.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: osdmap e55: 9 total, 9 up, 9 in 2024-07-31T14:44:52.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:52.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:52.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: Detected new or changed devices on smithi077 2024-07-31T14:44:52.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:52.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:52.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:52.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:52.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:52.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi077 to 5696M 2024-07-31T14:44:52.282 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:52.282 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:52.282 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:52.282 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: purged_snaps scrub starts 2024-07-31T14:44:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: purged_snaps scrub ok 2024-07-31T14:44:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: osd.8 [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754] boot 2024-07-31T14:44:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: osdmap e55: 9 total, 9 up, 9 in 2024-07-31T14:44:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-31T14:44:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-07-31T14:44:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: Detected new or changed devices on smithi077 2024-07-31T14:44:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:52.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:52.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:52.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:44:52.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi077 to 5696M 2024-07-31T14:44:52.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:52.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:52.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:44:52.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:44:53.296 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:53 smithi067 ceph-mon[32750]: osdmap e56: 9 total, 9 up, 9 in 2024-07-31T14:44:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:53 smithi073 ceph-mon[36714]: osdmap e56: 9 total, 9 up, 9 in 2024-07-31T14:44:53.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:53 smithi077 ceph-mon[37327]: osdmap e56: 9 total, 9 up, 9 in 2024-07-31T14:44:53.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.795+0000 7f150e3cc640 1 -- 172.21.15.67:0/2903305872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1508109820 msgr2=0x7f1508110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:44:53.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.795+0000 7f150e3cc640 1 --2- 172.21.15.67:0/2903305872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1508109820 0x7f1508110050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1500009ca0 tx=0x7f150002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:44:53.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.795+0000 7f150e3cc640 1 -- 172.21.15.67:0/2903305872 shutdown_connections 2024-07-31T14:44:53.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.795+0000 7f150e3cc640 1 --2- 172.21.15.67:0/2903305872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1508109820 0x7f1508110050 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:53.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.795+0000 7f150e3cc640 1 --2- 172.21.15.67:0/2903305872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1508108ed0 0x7f15081092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:53.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.795+0000 7f150e3cc640 1 --2- 172.21.15.67:0/2903305872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508103590 0x7f15081039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:53.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.795+0000 7f150e3cc640 1 -- 172.21.15.67:0/2903305872 >> 172.21.15.67:0/2903305872 conn(0x7f150807b280 msgr2=0x7f150807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:44:53.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.795+0000 7f150e3cc640 1 -- 172.21.15.67:0/2903305872 shutdown_connections 2024-07-31T14:44:53.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.795+0000 7f150e3cc640 1 -- 172.21.15.67:0/2903305872 wait complete. 2024-07-31T14:44:53.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.796+0000 7f150e3cc640 1 Processor -- start 2024-07-31T14:44:53.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.796+0000 7f150e3cc640 1 -- start start 2024-07-31T14:44:53.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.796+0000 7f150e3cc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1508103590 0x7f1508199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:44:53.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.796+0000 7f150e3cc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508108ed0 0x7f150819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:44:53.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.796+0000 7f150e3cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1508109820 0x7f150819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:44:53.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.796+0000 7f150e3cc640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f15081134f0 con 0x7f1508103590 2024-07-31T14:44:53.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.796+0000 7f150e3cc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1508113370 con 0x7f1508108ed0 2024-07-31T14:44:53.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.796+0000 7f150e3cc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1508113670 con 0x7f1508109820 2024-07-31T14:44:53.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.796+0000 7f1507fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1508103590 0x7f1508199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:44:53.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.796+0000 7f1507fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1508103590 0x7f1508199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37598/0 (socket says 172.21.15.67:37598) 2024-07-31T14:44:53.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.796+0000 7f1507fff640 1 -- 172.21.15.67:0/3337306938 learned_addr learned my addr 172.21.15.67:0/3337306938 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:44:53.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.796+0000 7f15077fe640 1 --2- 172.21.15.67:0/3337306938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508108ed0 0x7f150819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:44:53.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.796+0000 7f150c969640 1 --2- 172.21.15.67:0/3337306938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1508109820 0x7f150819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:44:53.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.797+0000 7f1507fff640 1 -- 172.21.15.67:0/3337306938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1508109820 msgr2=0x7f150819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:44:53.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.797+0000 7f1507fff640 1 --2- 172.21.15.67:0/3337306938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1508109820 0x7f150819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.797+0000 7f1507fff640 1 -- 172.21.15.67:0/3337306938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508108ed0 msgr2=0x7f150819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.797+0000 7f1507fff640 1 --2- 172.21.15.67:0/3337306938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508108ed0 0x7f150819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.797+0000 7f1507fff640 1 -- 172.21.15.67:0/3337306938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150819fe30 con 0x7f1508103590 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.797+0000 7f1507fff640 1 --2- 172.21.15.67:0/3337306938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1508103590 0x7f1508199ec0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f14f800e9f0 tx=0x7f14f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.797+0000 7f15057fa640 1 -- 172.21.15.67:0/3337306938 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f800cd80 con 0x7f1508103590 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.797+0000 7f150e3cc640 1 -- 172.21.15.67:0/3337306938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15081a0120 con 0x7f1508103590 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.797+0000 7f15057fa640 1 -- 172.21.15.67:0/3337306938 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f80108e0 con 0x7f1508103590 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.797+0000 7f150e3cc640 1 -- 172.21.15.67:0/3337306938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15081a6880 con 0x7f1508103590 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.797+0000 7f15057fa640 1 -- 172.21.15.67:0/3337306938 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8018980 con 0x7f1508103590 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.798+0000 7f150e3cc640 1 -- 172.21.15.67:0/3337306938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15081039a0 con 0x7f1508103590 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.800+0000 7f15057fa640 1 -- 172.21.15.67:0/3337306938 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f14f8010430 con 0x7f1508103590 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.800+0000 7f15057fa640 1 --2- 172.21.15.67:0/3337306938 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14f00b3c20 0x7f14f00b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.800+0000 7f15077fe640 1 --2- 172.21.15.67:0/3337306938 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14f00b3c20 0x7f14f00b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.800+0000 7f15057fa640 1 -- 172.21.15.67:0/3337306938 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6044+0+0 (secure 0 0 0) 0x7f14f80e3c50 con 0x7f1508103590 2024-07-31T14:44:53.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.801+0000 7f15077fe640 1 --2- 172.21.15.67:0/3337306938 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14f00b3c20 0x7f14f00b6130 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f14fc005fd0 tx=0x7f14fc005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:44:53.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:53.803+0000 7f15057fa640 1 -- 172.21.15.67:0/3337306938 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f14f80abb80 con 0x7f1508103590 2024-07-31T14:44:54.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:44:54.009+0000 7f150e3cc640 1 -- 172.21.15.67:0/3337306938 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1508063230 con 0x7f14f00b3c20 2024-07-31T14:44:54.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:54 smithi067 ceph-mon[32750]: osdmap e57: 9 total, 9 up, 9 in 2024-07-31T14:44:54.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:54 smithi067 ceph-mon[32750]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:44:54.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:54 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:44:54.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:54 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:44:54.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:54 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:54.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:54 smithi073 ceph-mon[36714]: osdmap e57: 9 total, 9 up, 9 in 2024-07-31T14:44:54.475 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:54 smithi073 ceph-mon[36714]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:44:54.476 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:54 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:44:54.476 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:54 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:44:54.476 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:54 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:54.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:54 smithi077 ceph-mon[37327]: osdmap e57: 9 total, 9 up, 9 in 2024-07-31T14:44:54.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:54 smithi077 ceph-mon[37327]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:44:54.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:54 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:44:54.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:54 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:44:54.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:54 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:44:55.289 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:55 smithi067 ceph-mon[32750]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:44:55.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:55 smithi073 ceph-mon[36714]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:44:55.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:55 smithi077 ceph-mon[37327]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:44:56.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:56 smithi067 ceph-mon[32750]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:44:56.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:56 smithi073 ceph-mon[36714]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:44:56.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:56 smithi077 ceph-mon[37327]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:44:57.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:57 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2225606820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "246aa076-dae2-448a-a294-2b0ca5ac3368"}]: dispatch 2024-07-31T14:44:57.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:57 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2225606820' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "246aa076-dae2-448a-a294-2b0ca5ac3368"}]': finished 2024-07-31T14:44:57.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:57 smithi067 ceph-mon[32750]: osdmap e58: 10 total, 9 up, 10 in 2024-07-31T14:44:57.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:57 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:44:57.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:57 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3336007312' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:44:57.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:57 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2225606820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "246aa076-dae2-448a-a294-2b0ca5ac3368"}]: dispatch 2024-07-31T14:44:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:57 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2225606820' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "246aa076-dae2-448a-a294-2b0ca5ac3368"}]': finished 2024-07-31T14:44:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:57 smithi073 ceph-mon[36714]: osdmap e58: 10 total, 9 up, 10 in 2024-07-31T14:44:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:57 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:44:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:57 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3336007312' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:44:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:57 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2225606820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "246aa076-dae2-448a-a294-2b0ca5ac3368"}]: dispatch 2024-07-31T14:44:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:57 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2225606820' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "246aa076-dae2-448a-a294-2b0ca5ac3368"}]': finished 2024-07-31T14:44:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:57 smithi077 ceph-mon[37327]: osdmap e58: 10 total, 9 up, 10 in 2024-07-31T14:44:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:57 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:44:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:57 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3336007312' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:44:58.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:44:58 smithi073 ceph-mon[36714]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-31T14:44:58.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:44:58 smithi077 ceph-mon[37327]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-31T14:44:58.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:44:58 smithi067 ceph-mon[32750]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-31T14:45:00.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:00 smithi067 ceph-mon[32750]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-07-31T14:45:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:00 smithi077 ceph-mon[37327]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-07-31T14:45:00.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:00 smithi073 ceph-mon[36714]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-07-31T14:45:02.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:02 smithi067 ceph-mon[32750]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-07-31T14:45:02.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:02 smithi077 ceph-mon[37327]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-07-31T14:45:02.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:02 smithi073 ceph-mon[36714]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-07-31T14:45:04.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:04 smithi077 ceph-mon[37327]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:45:04.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-31T14:45:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:04 smithi077 ceph-mon[37327]: Deploying daemon osd.9 on smithi067 2024-07-31T14:45:04.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:04 smithi067 ceph-mon[32750]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:45:04.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-31T14:45:04.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:04.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:04 smithi067 ceph-mon[32750]: Deploying daemon osd.9 on smithi067 2024-07-31T14:45:04.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:04 smithi073 ceph-mon[36714]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:45:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-31T14:45:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:04 smithi073 ceph-mon[36714]: Deploying daemon osd.9 on smithi067 2024-07-31T14:45:06.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:06 smithi067 ceph-mon[32750]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:45:06.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:06 smithi077 ceph-mon[37327]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:45:06.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:06 smithi073 ceph-mon[36714]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:45:08.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:08 smithi067 ceph-mon[32750]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-07-31T14:45:08.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:08 smithi077 ceph-mon[37327]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-07-31T14:45:08.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:08 smithi073 ceph-mon[36714]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-07-31T14:45:09.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:09 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:09.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:09 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:09.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:09 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:09.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:09 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:09.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:09 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:09.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:09 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:09.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:09 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:09.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:09 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:09.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:09 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:10.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:10 smithi067 ceph-mon[32750]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:45:10.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:10 smithi073 ceph-mon[36714]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:45:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:10 smithi077 ceph-mon[37327]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:45:12.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:45:12.400+0000 7f15057fa640 1 -- 172.21.15.67:0/3337306938 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1508063230 con 0x7f14f00b3c20 2024-07-31T14:45:12.401 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 9 on host 'smithi067' 2024-07-31T14:45:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:45:12.402+0000 7f150e3cc640 1 -- 172.21.15.67:0/3337306938 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14f00b3c20 msgr2=0x7f14f00b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:45:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:45:12.402+0000 7f150e3cc640 1 --2- 172.21.15.67:0/3337306938 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14f00b3c20 0x7f14f00b6130 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f14fc005fd0 tx=0x7f14fc005850 comp rx=0 tx=0).stop 2024-07-31T14:45:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:45:12.402+0000 7f150e3cc640 1 -- 172.21.15.67:0/3337306938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1508103590 msgr2=0x7f1508199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:45:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:45:12.402+0000 7f150e3cc640 1 --2- 172.21.15.67:0/3337306938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1508103590 0x7f1508199ec0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f14f800e9f0 tx=0x7f14f800eeb0 comp rx=0 tx=0).stop 2024-07-31T14:45:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:45:12.402+0000 7f150e3cc640 1 -- 172.21.15.67:0/3337306938 shutdown_connections 2024-07-31T14:45:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:45:12.402+0000 7f150e3cc640 1 --2- 172.21.15.67:0/3337306938 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14f00b3c20 0x7f14f00b6130 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:45:12.402+0000 7f150e3cc640 1 --2- 172.21.15.67:0/3337306938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1508109820 0x7f150819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:45:12.402+0000 7f150e3cc640 1 --2- 172.21.15.67:0/3337306938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508108ed0 0x7f150819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:45:12.402+0000 7f150e3cc640 1 --2- 172.21.15.67:0/3337306938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1508103590 0x7f1508199ec0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:45:12.402+0000 7f150e3cc640 1 -- 172.21.15.67:0/3337306938 >> 172.21.15.67:0/3337306938 conn(0x7f150807b280 msgr2=0x7f1508106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:45:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:45:12.402+0000 7f150e3cc640 1 -- 172.21.15.67:0/3337306938 shutdown_connections 2024-07-31T14:45:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:45:12.403+0000 7f150e3cc640 1 -- 172.21.15.67:0/3337306938 wait complete. 2024-07-31T14:45:12.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:12 smithi077 ceph-mon[37327]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:45:12.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:12 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:12.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:12 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:12.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:12 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:12.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:12 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:45:12.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:12 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:12.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:12 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:12.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:12 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:12.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:12 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:12 smithi067 ceph-mon[32750]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:45:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:12 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:12 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:12 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:12 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:45:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:12 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:12 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:12 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:12 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:12 smithi073 ceph-mon[36714]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:45:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:12 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:12 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:12 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:12 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:45:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:12 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:12 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:12 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:12 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:13.092 DEBUG:teuthology.orchestra.run.smithi067:osd.9> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.9.service 2024-07-31T14:45:13.095 INFO:tasks.cephadm:Deploying osd.10 on smithi073 with /dev/vg_nvme/lv_1... 2024-07-31T14:45:13.095 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-31T14:45:13.351 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.b/config 2024-07-31T14:45:13.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:13 smithi077 ceph-mon[37327]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-31T14:45:13.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:13 smithi067 ceph-mon[32750]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-31T14:45:13.806 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:13 smithi073 ceph-mon[36714]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-31T14:45:14.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:14 smithi067 ceph-mon[32750]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:45:14.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:14 smithi067 ceph-mon[32750]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-31T14:45:14.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:14 smithi067 ceph-mon[32750]: osdmap e59: 10 total, 9 up, 10 in 2024-07-31T14:45:14.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:14 smithi067 ceph-mon[32750]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-31T14:45:14.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:14.734 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:14 smithi073 ceph-mon[36714]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:45:14.734 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:14 smithi073 ceph-mon[36714]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-31T14:45:14.734 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:14 smithi073 ceph-mon[36714]: osdmap e59: 10 total, 9 up, 10 in 2024-07-31T14:45:14.735 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:14 smithi073 ceph-mon[36714]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-31T14:45:14.735 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:14.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:14 smithi077 ceph-mon[37327]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:45:14.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:14 smithi077 ceph-mon[37327]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-31T14:45:14.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:14 smithi077 ceph-mon[37327]: osdmap e59: 10 total, 9 up, 10 in 2024-07-31T14:45:14.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:14 smithi077 ceph-mon[37327]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-31T14:45:14.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:15.226 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-31T14:45:15.321 INFO:teuthology.orchestra.run.smithi073.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-31T14:45:15.334 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10+0 records in 2024-07-31T14:45:15.334 INFO:teuthology.orchestra.run.smithi073.stderr:10+0 records out 2024-07-31T14:45:15.334 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115469 s, 908 MB/s 2024-07-31T14:45:15.334 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping successful for: 2024-07-31T14:45:15.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:15 smithi067 ceph-mon[32750]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-31T14:45:15.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:15 smithi067 ceph-mon[32750]: osdmap e60: 10 total, 9 up, 10 in 2024-07-31T14:45:15.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:15 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:15.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:15 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:15.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:15 smithi067 ceph-mon[32750]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' 2024-07-31T14:45:15.554 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 14:45:15 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T14:45:15.251+0000 7fb8959c1640 -1 osd.9 0 waiting for initial osdmap 2024-07-31T14:45:15.554 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 14:45:15 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T14:45:15.268+0000 7fb88cfa2640 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-31T14:45:15.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:15 smithi073 ceph-mon[36714]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-31T14:45:15.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:15 smithi073 ceph-mon[36714]: osdmap e60: 10 total, 9 up, 10 in 2024-07-31T14:45:15.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:15 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:15.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:15 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:15.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:15 smithi073 ceph-mon[36714]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' 2024-07-31T14:45:15.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:15 smithi077 ceph-mon[37327]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-31T14:45:15.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:15 smithi077 ceph-mon[37327]: osdmap e60: 10 total, 9 up, 10 in 2024-07-31T14:45:15.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:15 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:15.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:15 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:15.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:15 smithi077 ceph-mon[37327]: from='osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060]' entity='osd.9' 2024-07-31T14:45:16.016 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_1 2024-07-31T14:45:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.b/config 2024-07-31T14:45:16.665 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: purged_snaps scrub starts 2024-07-31T14:45:16.665 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: purged_snaps scrub ok 2024-07-31T14:45:16.665 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:45:16.665 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:16.665 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: Detected new or changed devices on smithi067 2024-07-31T14:45:16.665 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:16.665 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:16.665 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:16.665 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:16.665 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:16.666 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:16.666 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi067 to 4272M 2024-07-31T14:45:16.666 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:16.666 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:16.666 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:45:16.666 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:16.666 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060] boot 2024-07-31T14:45:16.666 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: osdmap e61: 10 total, 10 up, 10 in 2024-07-31T14:45:16.666 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:16.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: purged_snaps scrub starts 2024-07-31T14:45:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: purged_snaps scrub ok 2024-07-31T14:45:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:45:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: Detected new or changed devices on smithi067 2024-07-31T14:45:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi067 to 4272M 2024-07-31T14:45:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:45:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060] boot 2024-07-31T14:45:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: osdmap e61: 10 total, 10 up, 10 in 2024-07-31T14:45:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:16.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: purged_snaps scrub starts 2024-07-31T14:45:16.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: purged_snaps scrub ok 2024-07-31T14:45:16.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-07-31T14:45:16.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:16.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: Detected new or changed devices on smithi067 2024-07-31T14:45:16.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:16.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:16.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:16.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:16.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:16.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:16.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi067 to 4272M 2024-07-31T14:45:16.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:16.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:16.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:45:16.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:16.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: osd.9 [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060] boot 2024-07-31T14:45:16.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: osdmap e61: 10 total, 10 up, 10 in 2024-07-31T14:45:16.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-31T14:45:18.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.991+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1902605355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff408101c60 msgr2=0x7ff408102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:45:18.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.991+0000 7ff40eb4b640 1 --2- 172.21.15.73:0/1902605355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff408101c60 0x7ff408102070 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff400009ca0 tx=0x7ff40002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:45:18.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.991+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1902605355 shutdown_connections 2024-07-31T14:45:18.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.991+0000 7ff40eb4b640 1 --2- 172.21.15.73:0/1902605355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff408107ea0 0x7ff40810e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:18.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.991+0000 7ff40eb4b640 1 --2- 172.21.15.73:0/1902605355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff408107550 0x7ff408107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:18.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.991+0000 7ff40eb4b640 1 --2- 172.21.15.73:0/1902605355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff408101c60 0x7ff408102070 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:18.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.991+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1902605355 >> 172.21.15.73:0/1902605355 conn(0x7ff408077690 msgr2=0x7ff408077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:45:18.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.991+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1902605355 shutdown_connections 2024-07-31T14:45:18.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.991+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1902605355 wait complete. 2024-07-31T14:45:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40eb4b640 1 Processor -- start 2024-07-31T14:45:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40eb4b640 1 -- start start 2024-07-31T14:45:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40eb4b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff408101c60 0x7ff408199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:45:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40eb4b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff408107550 0x7ff40819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:45:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40eb4b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff408107ea0 0x7ff40819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:45:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40eb4b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff4081112e0 con 0x7ff408107550 2024-07-31T14:45:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40eb4b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff408111160 con 0x7ff408107ea0 2024-07-31T14:45:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40eb4b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff408111460 con 0x7ff408101c60 2024-07-31T14:45:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40d0e8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff408107ea0 0x7ff40819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:45:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40d0e8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff408107ea0 0x7ff40819f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47074/0 (socket says 172.21.15.73:47074) 2024-07-31T14:45:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40d0e8640 1 -- 172.21.15.73:0/1495284413 learned_addr learned my addr 172.21.15.73:0/1495284413 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-07-31T14:45:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40c8e7640 1 --2- 172.21.15.73:0/1495284413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff408101c60 0x7ff408199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:45:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff407fff640 1 --2- 172.21.15.73:0/1495284413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff408107550 0x7ff40819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:45:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40d0e8640 1 -- 172.21.15.73:0/1495284413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff408101c60 msgr2=0x7ff408199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:45:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40d0e8640 1 --2- 172.21.15.73:0/1495284413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff408101c60 0x7ff408199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40d0e8640 1 -- 172.21.15.73:0/1495284413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff408107550 msgr2=0x7ff40819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:45:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40d0e8640 1 --2- 172.21.15.73:0/1495284413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff408107550 0x7ff40819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40d0e8640 1 -- 172.21.15.73:0/1495284413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40819fe10 con 0x7ff408107ea0 2024-07-31T14:45:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.992+0000 7ff40d0e8640 1 --2- 172.21.15.73:0/1495284413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff408107ea0 0x7ff40819f6e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000e9c0 tx=0x7ff3f000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:45:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.993+0000 7ff405ffb640 1 -- 172.21.15.73:0/1495284413 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f000cce0 con 0x7ff408107ea0 2024-07-31T14:45:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.993+0000 7ff405ffb640 1 -- 172.21.15.73:0/1495284413 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3f00054d0 con 0x7ff408107ea0 2024-07-31T14:45:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.993+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1495284413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4081a0100 con 0x7ff408107ea0 2024-07-31T14:45:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.993+0000 7ff405ffb640 1 -- 172.21.15.73:0/1495284413 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0016670 con 0x7ff408107ea0 2024-07-31T14:45:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.993+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1495284413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4081a6860 con 0x7ff408107ea0 2024-07-31T14:45:18.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.994+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1495284413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff408102070 con 0x7ff408107ea0 2024-07-31T14:45:18.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.995+0000 7ff405ffb640 1 -- 172.21.15.73:0/1495284413 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7ff3f001d020 con 0x7ff408107ea0 2024-07-31T14:45:18.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.996+0000 7ff405ffb640 1 --2- 172.21.15.73:0/1495284413 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff3f40b3c20 0x7ff3f40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:45:18.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.996+0000 7ff405ffb640 1 -- 172.21.15.73:0/1495284413 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6476+0+0 (secure 0 0 0) 0x7ff3f00e3a90 con 0x7ff408107ea0 2024-07-31T14:45:18.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.996+0000 7ff40c8e7640 1 --2- 172.21.15.73:0/1495284413 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff3f40b3c20 0x7ff3f40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:45:18.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:17.997+0000 7ff40c8e7640 1 --2- 172.21.15.73:0/1495284413 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff3f40b3c20 0x7ff3f40b6130 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff400005e30 tx=0x7ff400009360 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:45:18.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:18.000+0000 7ff405ffb640 1 -- 172.21.15.73:0/1495284413 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff3f0024030 con 0x7ff408107ea0 2024-07-31T14:45:18.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:18.194+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1495284413 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff408109cd0 con 0x7ff3f40b3c20 2024-07-31T14:45:18.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:18 smithi067 ceph-mon[32750]: osdmap e62: 10 total, 10 up, 10 in 2024-07-31T14:45:18.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:18 smithi067 ceph-mon[32750]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:18.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:18 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:45:18.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:18 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:45:18.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:18 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:18.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:18 smithi073 ceph-mon[36714]: osdmap e62: 10 total, 10 up, 10 in 2024-07-31T14:45:18.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:18 smithi073 ceph-mon[36714]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:18.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:18 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:45:18.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:18 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:45:18.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:18 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:18.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:18 smithi077 ceph-mon[37327]: osdmap e62: 10 total, 10 up, 10 in 2024-07-31T14:45:18.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:18 smithi077 ceph-mon[37327]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:18.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:18 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:45:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:18 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:45:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:18 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:19.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:19 smithi067 ceph-mon[32750]: from='client.24530 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:45:19.591 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:19 smithi073 ceph-mon[36714]: from='client.24530 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:45:19.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:19 smithi077 ceph-mon[37327]: from='client.24530 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:45:20.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:20 smithi067 ceph-mon[32750]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:20.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:20 smithi073 ceph-mon[36714]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:20.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:20 smithi077 ceph-mon[37327]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:21 smithi067 ceph-mon[32750]: from='client.? 172.21.15.73:0/4123311268' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a6d993-7acb-467d-abf4-92d5e443438e"}]: dispatch 2024-07-31T14:45:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:21 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a6d993-7acb-467d-abf4-92d5e443438e"}]: dispatch 2024-07-31T14:45:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:21 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65a6d993-7acb-467d-abf4-92d5e443438e"}]': finished 2024-07-31T14:45:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:21 smithi067 ceph-mon[32750]: osdmap e63: 11 total, 10 up, 11 in 2024-07-31T14:45:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:21 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:21 smithi067 ceph-mon[32750]: from='client.? 172.21.15.73:0/2795862463' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:45:21.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:21 smithi073 ceph-mon[36714]: from='client.? 172.21.15.73:0/4123311268' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a6d993-7acb-467d-abf4-92d5e443438e"}]: dispatch 2024-07-31T14:45:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:21 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a6d993-7acb-467d-abf4-92d5e443438e"}]: dispatch 2024-07-31T14:45:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:21 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65a6d993-7acb-467d-abf4-92d5e443438e"}]': finished 2024-07-31T14:45:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:21 smithi073 ceph-mon[36714]: osdmap e63: 11 total, 10 up, 11 in 2024-07-31T14:45:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:21 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:21 smithi073 ceph-mon[36714]: from='client.? 172.21.15.73:0/2795862463' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:45:21.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:21 smithi077 ceph-mon[37327]: from='client.? 172.21.15.73:0/4123311268' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a6d993-7acb-467d-abf4-92d5e443438e"}]: dispatch 2024-07-31T14:45:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:21 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a6d993-7acb-467d-abf4-92d5e443438e"}]: dispatch 2024-07-31T14:45:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:21 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65a6d993-7acb-467d-abf4-92d5e443438e"}]': finished 2024-07-31T14:45:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:21 smithi077 ceph-mon[37327]: osdmap e63: 11 total, 10 up, 11 in 2024-07-31T14:45:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:21 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:21 smithi077 ceph-mon[37327]: from='client.? 172.21.15.73:0/2795862463' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:45:22.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:22 smithi067 ceph-mon[32750]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:22.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:22 smithi073 ceph-mon[36714]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:22.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:22 smithi077 ceph-mon[37327]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:24.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:24 smithi073 ceph-mon[36714]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:24.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:24 smithi077 ceph-mon[37327]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:24.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:24 smithi067 ceph-mon[32750]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:26.700 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:26 smithi073 ceph-mon[36714]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:26 smithi077 ceph-mon[37327]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:26.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:26 smithi067 ceph-mon[32750]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:28.513 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:28 smithi073 ceph-mon[36714]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:28.514 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:28 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-31T14:45:28.514 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:28 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:28.514 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:28 smithi073 ceph-mon[36714]: Deploying daemon osd.10 on smithi073 2024-07-31T14:45:28.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:28 smithi077 ceph-mon[37327]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:28.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:28 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-31T14:45:28.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:28 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:28.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:28 smithi077 ceph-mon[37327]: Deploying daemon osd.10 on smithi073 2024-07-31T14:45:28.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:28 smithi067 ceph-mon[32750]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:28.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:28 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-31T14:45:28.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:28 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:28.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:28 smithi067 ceph-mon[32750]: Deploying daemon osd.10 on smithi073 2024-07-31T14:45:30.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:30 smithi073 ceph-mon[36714]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:30.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:30 smithi077 ceph-mon[37327]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:30 smithi067 ceph-mon[32750]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:32 smithi073 ceph-mon[36714]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:32.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:32 smithi077 ceph-mon[37327]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:32.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:32 smithi067 ceph-mon[32750]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:33.671 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:33 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:33.671 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:33 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:33.671 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:33 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:33.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:33 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:33.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:33 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:33.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:33 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:33.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:33 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:33.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:33 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:33.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:33 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:34.729 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:34 smithi073 ceph-mon[36714]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:34.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:34 smithi077 ceph-mon[37327]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:34.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:34 smithi067 ceph-mon[32750]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:36.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:36 smithi077 ceph-mon[37327]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:36.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:36 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:36.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:36 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:36.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:36 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:36.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:36 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:45:36.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:36 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:36.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:36 smithi067 ceph-mon[32750]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:36.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:36 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:36.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:36 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:36.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:36 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:36.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:36 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:45:36.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:36 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:36.880 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:36 smithi073 ceph-mon[36714]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:36.880 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:36 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:36.880 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:36 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:36.881 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:36 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:36.881 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:36 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:45:36.881 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:36 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:36.929 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 10 on host 'smithi073' 2024-07-31T14:45:36.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:36.927+0000 7ff405ffb640 1 -- 172.21.15.73:0/1495284413 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff408109cd0 con 0x7ff3f40b3c20 2024-07-31T14:45:36.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:36.929+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1495284413 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff3f40b3c20 msgr2=0x7ff3f40b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:45:36.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:36.929+0000 7ff40eb4b640 1 --2- 172.21.15.73:0/1495284413 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff3f40b3c20 0x7ff3f40b6130 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff400005e30 tx=0x7ff400009360 comp rx=0 tx=0).stop 2024-07-31T14:45:36.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:36.929+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1495284413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff408107ea0 msgr2=0x7ff40819f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:45:36.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:36.929+0000 7ff40eb4b640 1 --2- 172.21.15.73:0/1495284413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff408107ea0 0x7ff40819f6e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000e9c0 tx=0x7ff3f000ee80 comp rx=0 tx=0).stop 2024-07-31T14:45:36.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:36.930+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1495284413 shutdown_connections 2024-07-31T14:45:36.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:36.930+0000 7ff40eb4b640 1 --2- 172.21.15.73:0/1495284413 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff3f40b3c20 0x7ff3f40b6130 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:36.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:36.930+0000 7ff40eb4b640 1 --2- 172.21.15.73:0/1495284413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff408107ea0 0x7ff40819f6e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:36.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:36.930+0000 7ff40eb4b640 1 --2- 172.21.15.73:0/1495284413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff408107550 0x7ff40819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:36.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:36.930+0000 7ff40eb4b640 1 --2- 172.21.15.73:0/1495284413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff408101c60 0x7ff408199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:36.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:36.930+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1495284413 >> 172.21.15.73:0/1495284413 conn(0x7ff408077690 msgr2=0x7ff408105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:45:36.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:36.930+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1495284413 shutdown_connections 2024-07-31T14:45:36.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-07-31T14:45:36.930+0000 7ff40eb4b640 1 -- 172.21.15.73:0/1495284413 wait complete. 2024-07-31T14:45:37.580 DEBUG:teuthology.orchestra.run.smithi073:osd.10> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.10.service 2024-07-31T14:45:37.583 INFO:tasks.cephadm:Deploying osd.11 on smithi077 with /dev/vg_nvme/lv_1... 2024-07-31T14:45:37.583 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-31T14:45:37.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:37 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:37.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:37 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:37.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:37 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:37.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:37 smithi077 ceph-mon[37327]: from='osd.10 [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-31T14:45:37.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:37 smithi077 ceph-mon[37327]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-31T14:45:37.770 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:37 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:37.770 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:37 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:37.770 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:37 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:37.770 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:37 smithi073 ceph-mon[36714]: from='osd.10 [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-31T14:45:37.770 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:37 smithi073 ceph-mon[36714]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-31T14:45:37.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:37 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:37.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:37 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:37.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:37 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:37.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:37 smithi067 ceph-mon[32750]: from='osd.10 [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-31T14:45:37.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:37 smithi067 ceph-mon[32750]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-31T14:45:37.846 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T14:45:39.082 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:38 smithi073 ceph-mon[36714]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:39.083 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:38 smithi073 ceph-mon[36714]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-31T14:45:39.083 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:38 smithi073 ceph-mon[36714]: osdmap e64: 11 total, 10 up, 11 in 2024-07-31T14:45:39.083 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:38 smithi073 ceph-mon[36714]: from='osd.10 [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:45:39.083 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:39.083 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:38 smithi073 ceph-mon[36714]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:45:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:38 smithi077 ceph-mon[37327]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:38 smithi077 ceph-mon[37327]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-31T14:45:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:38 smithi077 ceph-mon[37327]: osdmap e64: 11 total, 10 up, 11 in 2024-07-31T14:45:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:38 smithi077 ceph-mon[37327]: from='osd.10 [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:45:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:38 smithi077 ceph-mon[37327]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:45:39.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:38 smithi067 ceph-mon[32750]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:39.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:38 smithi067 ceph-mon[32750]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-31T14:45:39.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:38 smithi067 ceph-mon[32750]: osdmap e64: 11 total, 10 up, 11 in 2024-07-31T14:45:39.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:38 smithi067 ceph-mon[32750]: from='osd.10 [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:45:39.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:39.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:38 smithi067 ceph-mon[32750]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-07-31T14:45:39.743 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-31T14:45:39.829 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-31T14:45:39.842 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-07-31T14:45:39.843 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-07-31T14:45:39.843 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118432 s, 885 MB/s 2024-07-31T14:45:39.843 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-07-31T14:45:39.943 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 14:45:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T14:45:39.648+0000 7fbfe3d7a640 -1 osd.10 0 waiting for initial osdmap 2024-07-31T14:45:39.944 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 14:45:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T14:45:39.663+0000 7fbfdb35b640 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-31T14:45:40.170 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:39 smithi077 ceph-mon[37327]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-07-31T14:45:40.171 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:39 smithi077 ceph-mon[37327]: osdmap e65: 11 total, 10 up, 11 in 2024-07-31T14:45:40.171 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:39 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:40.171 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:39 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:40.171 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:39 smithi077 ceph-mon[37327]: from='osd.10 ' entity='osd.10' 2024-07-31T14:45:40.171 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:39 smithi077 ceph-mon[37327]: osd.10 [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997] boot 2024-07-31T14:45:40.171 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:39 smithi077 ceph-mon[37327]: osdmap e66: 11 total, 11 up, 11 in 2024-07-31T14:45:40.171 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:39 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:40.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:39 smithi067 ceph-mon[32750]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-07-31T14:45:40.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:39 smithi067 ceph-mon[32750]: osdmap e65: 11 total, 10 up, 11 in 2024-07-31T14:45:40.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:39 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:40.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:39 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:40.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:39 smithi067 ceph-mon[32750]: from='osd.10 ' entity='osd.10' 2024-07-31T14:45:40.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:39 smithi067 ceph-mon[32750]: osd.10 [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997] boot 2024-07-31T14:45:40.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:39 smithi067 ceph-mon[32750]: osdmap e66: 11 total, 11 up, 11 in 2024-07-31T14:45:40.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:39 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:40.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:39 smithi073 ceph-mon[36714]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-07-31T14:45:40.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:39 smithi073 ceph-mon[36714]: osdmap e65: 11 total, 10 up, 11 in 2024-07-31T14:45:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:39 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:39 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:39 smithi073 ceph-mon[36714]: from='osd.10 ' entity='osd.10' 2024-07-31T14:45:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:39 smithi073 ceph-mon[36714]: osd.10 [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997] boot 2024-07-31T14:45:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:39 smithi073 ceph-mon[36714]: osdmap e66: 11 total, 11 up, 11 in 2024-07-31T14:45:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:39 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-31T14:45:40.501 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_1 2024-07-31T14:45:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T14:45:41.046 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:40 smithi077 ceph-mon[37327]: purged_snaps scrub starts 2024-07-31T14:45:41.046 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:40 smithi077 ceph-mon[37327]: purged_snaps scrub ok 2024-07-31T14:45:41.046 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:40 smithi077 ceph-mon[37327]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:41.046 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:40 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:41.047 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:40 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:41.047 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:40 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:41.047 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:40 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:41.047 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:40 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:41.047 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:40 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:41.047 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:40 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:41.047 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:40 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:41.047 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:40 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:45:41.047 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:40 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:41.047 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:40 smithi077 ceph-mon[37327]: osdmap e67: 11 total, 11 up, 11 in 2024-07-31T14:45:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:40 smithi067 ceph-mon[32750]: purged_snaps scrub starts 2024-07-31T14:45:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:40 smithi067 ceph-mon[32750]: purged_snaps scrub ok 2024-07-31T14:45:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:40 smithi067 ceph-mon[32750]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:40 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:40 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:41.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:40 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:41.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:40 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:41.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:40 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:41.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:40 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:41.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:40 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:41.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:40 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:41.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:40 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:45:41.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:40 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:41.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:40 smithi067 ceph-mon[32750]: osdmap e67: 11 total, 11 up, 11 in 2024-07-31T14:45:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:40 smithi073 ceph-mon[36714]: purged_snaps scrub starts 2024-07-31T14:45:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:40 smithi073 ceph-mon[36714]: purged_snaps scrub ok 2024-07-31T14:45:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:40 smithi073 ceph-mon[36714]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-31T14:45:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:40 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:40 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:40 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:40 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:40 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:41.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:40 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:45:41.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:40 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:41.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:40 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:41.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:40 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:45:41.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:40 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:41.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:40 smithi073 ceph-mon[36714]: osdmap e67: 11 total, 11 up, 11 in 2024-07-31T14:45:42.203 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:41 smithi077 ceph-mon[37327]: Detected new or changed devices on smithi073 2024-07-31T14:45:42.203 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:41 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi073 to 4272M 2024-07-31T14:45:42.204 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:41 smithi077 ceph-mon[37327]: osdmap e68: 11 total, 11 up, 11 in 2024-07-31T14:45:42.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:41 smithi067 ceph-mon[32750]: Detected new or changed devices on smithi073 2024-07-31T14:45:42.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:41 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi073 to 4272M 2024-07-31T14:45:42.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:41 smithi067 ceph-mon[32750]: osdmap e68: 11 total, 11 up, 11 in 2024-07-31T14:45:42.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:41 smithi073 ceph-mon[36714]: Detected new or changed devices on smithi073 2024-07-31T14:45:42.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:41 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi073 to 4272M 2024-07-31T14:45:42.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:41 smithi073 ceph-mon[36714]: osdmap e68: 11 total, 11 up, 11 in 2024-07-31T14:45:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.424+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1457023983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2bc104ce0 msgr2=0x7fe2bc10d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:45:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.424+0000 7fe2c3fdd640 1 --2- 172.21.15.77:0/1457023983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2bc104ce0 0x7fe2bc10d320 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009900 tx=0x7fe2ac02a3f0 comp rx=0 tx=0).stop 2024-07-31T14:45:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.424+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1457023983 shutdown_connections 2024-07-31T14:45:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.424+0000 7fe2c3fdd640 1 --2- 172.21.15.77:0/1457023983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2bc10d860 0x7fe2bc10fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.424+0000 7fe2c3fdd640 1 --2- 172.21.15.77:0/1457023983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2bc104ce0 0x7fe2bc10d320 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.424+0000 7fe2c3fdd640 1 --2- 172.21.15.77:0/1457023983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2bc1042b0 0x7fe2bc1046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:42.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.424+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1457023983 >> 172.21.15.77:0/1457023983 conn(0x7fe2bc079160 msgr2=0x7fe2bc079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:45:42.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.424+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1457023983 shutdown_connections 2024-07-31T14:45:42.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.424+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1457023983 wait complete. 2024-07-31T14:45:42.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.425+0000 7fe2c3fdd640 1 Processor -- start 2024-07-31T14:45:42.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.425+0000 7fe2c3fdd640 1 -- start start 2024-07-31T14:45:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.425+0000 7fe2c3fdd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2bc1042b0 0x7fe2bc199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:45:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.425+0000 7fe2c3fdd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2bc104ce0 0x7fe2bc19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:45:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.425+0000 7fe2c3fdd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2bc10d860 0x7fe2bc19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:45:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.425+0000 7fe2c3fdd640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc113460 con 0x7fe2bc10d860 2024-07-31T14:45:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c3fdd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1132e0 con 0x7fe2bc1042b0 2024-07-31T14:45:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c3fdd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1135e0 con 0x7fe2bc104ce0 2024-07-31T14:45:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c1578640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2bc104ce0 0x7fe2bc19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:45:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c1578640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2bc104ce0 0x7fe2bc19a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57410/0 (socket says 172.21.15.77:57410) 2024-07-31T14:45:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c1578640 1 -- 172.21.15.77:0/1785349965 learned_addr learned my addr 172.21.15.77:0/1785349965 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-07-31T14:45:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c1d79640 1 --2- 172.21.15.77:0/1785349965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2bc1042b0 0x7fe2bc199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:45:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c257a640 1 --2- 172.21.15.77:0/1785349965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2bc10d860 0x7fe2bc19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:45:42.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c1578640 1 -- 172.21.15.77:0/1785349965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2bc1042b0 msgr2=0x7fe2bc199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:45:42.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c1578640 1 --2- 172.21.15.77:0/1785349965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2bc1042b0 0x7fe2bc199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:42.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c1578640 1 -- 172.21.15.77:0/1785349965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2bc10d860 msgr2=0x7fe2bc19f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:45:42.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c1578640 1 --2- 172.21.15.77:0/1785349965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2bc10d860 0x7fe2bc19f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:45:42.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c1578640 1 -- 172.21.15.77:0/1785349965 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2bc19fd00 con 0x7fe2bc104ce0 2024-07-31T14:45:42.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c1578640 1 --2- 172.21.15.77:0/1785349965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2bc104ce0 0x7fe2bc19a3f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac031a90 tx=0x7fe2ac004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:45:42.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2b2ffd640 1 -- 172.21.15.77:0/1785349965 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac03f030 con 0x7fe2bc104ce0 2024-07-31T14:45:42.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2b2ffd640 1 -- 172.21.15.77:0/1785349965 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2ac003cb0 con 0x7fe2bc104ce0 2024-07-31T14:45:42.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1785349965 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2bc19ff90 con 0x7fe2bc104ce0 2024-07-31T14:45:42.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2b2ffd640 1 -- 172.21.15.77:0/1785349965 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac035070 con 0x7fe2bc104ce0 2024-07-31T14:45:42.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.426+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1785349965 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2bc1a0470 con 0x7fe2bc104ce0 2024-07-31T14:45:42.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.427+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1785349965 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2bc1046c0 con 0x7fe2bc104ce0 2024-07-31T14:45:42.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.429+0000 7fe2b2ffd640 1 -- 172.21.15.77:0/1785349965 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fe2ac042030 con 0x7fe2bc104ce0 2024-07-31T14:45:42.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.430+0000 7fe2b2ffd640 1 --2- 172.21.15.77:0/1785349965 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2a80b3c50 0x7fe2a80b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:45:42.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.430+0000 7fe2b2ffd640 1 -- 172.21.15.77:0/1785349965 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6909+0+0 (secure 0 0 0) 0x7fe2ac103dd0 con 0x7fe2bc104ce0 2024-07-31T14:45:42.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.430+0000 7fe2c1d79640 1 --2- 172.21.15.77:0/1785349965 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2a80b3c50 0x7fe2a80b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:45:42.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.431+0000 7fe2c1d79640 1 --2- 172.21.15.77:0/1785349965 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2a80b3c50 0x7fe2a80b6160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8005c40 tx=0x7fe2b8005bd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:45:42.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.433+0000 7fe2b2ffd640 1 -- 172.21.15.77:0/1785349965 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe2ac0cb9a0 con 0x7fe2bc104ce0 2024-07-31T14:45:42.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:45:42.628+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1785349965 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe2bc1069b0 con 0x7fe2a80b3c50 2024-07-31T14:45:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:42 smithi077 ceph-mon[37327]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 651 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:45:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:45:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:45:43.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:42 smithi067 ceph-mon[32750]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 651 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:45:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:45:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:45:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:43.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:42 smithi073 ceph-mon[36714]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 651 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:45:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-31T14:45:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-31T14:45:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:44.048 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:43 smithi077 ceph-mon[37327]: from='client.24553 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:45:44.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:43 smithi067 ceph-mon[32750]: from='client.24553 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:45:44.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:43 smithi073 ceph-mon[36714]: from='client.24553 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:45:45.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:44 smithi077 ceph-mon[37327]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 651 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:45:45.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:44 smithi067 ceph-mon[32750]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 651 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:45:45.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:44 smithi073 ceph-mon[36714]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 651 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:45:46.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:45 smithi067 ceph-mon[32750]: from='client.? 172.21.15.77:0/926263128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "709fb536-23a5-4c14-aed6-97fad837c210"}]: dispatch 2024-07-31T14:45:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:45 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "709fb536-23a5-4c14-aed6-97fad837c210"}]: dispatch 2024-07-31T14:45:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:45 smithi067 ceph-mon[32750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "709fb536-23a5-4c14-aed6-97fad837c210"}]': finished 2024-07-31T14:45:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:45 smithi067 ceph-mon[32750]: osdmap e69: 12 total, 11 up, 12 in 2024-07-31T14:45:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:45 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:45:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:45 smithi067 ceph-mon[32750]: from='client.? 172.21.15.77:0/1125211250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:45:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:45 smithi077 ceph-mon[37327]: from='client.? 172.21.15.77:0/926263128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "709fb536-23a5-4c14-aed6-97fad837c210"}]: dispatch 2024-07-31T14:45:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:45 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "709fb536-23a5-4c14-aed6-97fad837c210"}]: dispatch 2024-07-31T14:45:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:45 smithi077 ceph-mon[37327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "709fb536-23a5-4c14-aed6-97fad837c210"}]': finished 2024-07-31T14:45:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:45 smithi077 ceph-mon[37327]: osdmap e69: 12 total, 11 up, 12 in 2024-07-31T14:45:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:45 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:45:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:45 smithi077 ceph-mon[37327]: from='client.? 172.21.15.77:0/1125211250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:45:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:45 smithi073 ceph-mon[36714]: from='client.? 172.21.15.77:0/926263128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "709fb536-23a5-4c14-aed6-97fad837c210"}]: dispatch 2024-07-31T14:45:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:45 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "709fb536-23a5-4c14-aed6-97fad837c210"}]: dispatch 2024-07-31T14:45:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:45 smithi073 ceph-mon[36714]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "709fb536-23a5-4c14-aed6-97fad837c210"}]': finished 2024-07-31T14:45:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:45 smithi073 ceph-mon[36714]: osdmap e69: 12 total, 11 up, 12 in 2024-07-31T14:45:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:45 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:45:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:45 smithi073 ceph-mon[36714]: from='client.? 172.21.15.77:0/1125211250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-31T14:45:47.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:46 smithi077 ceph-mon[37327]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:45:47.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:46 smithi067 ceph-mon[32750]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:45:47.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:46 smithi073 ceph-mon[36714]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:45:49.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:48 smithi077 ceph-mon[37327]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-07-31T14:45:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:48 smithi067 ceph-mon[32750]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-07-31T14:45:49.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:48 smithi073 ceph-mon[36714]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-07-31T14:45:51.210 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:50 smithi077 ceph-mon[37327]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-31T14:45:51.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:50 smithi067 ceph-mon[32750]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-31T14:45:51.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:50 smithi073 ceph-mon[36714]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-31T14:45:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:52 smithi077 ceph-mon[37327]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-07-31T14:45:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-31T14:45:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:53.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:52 smithi067 ceph-mon[32750]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-07-31T14:45:53.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-31T14:45:53.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:53.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:52 smithi073 ceph-mon[36714]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-07-31T14:45:53.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-31T14:45:53.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:45:54.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:53 smithi077 ceph-mon[37327]: Deploying daemon osd.11 on smithi077 2024-07-31T14:45:54.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:53 smithi067 ceph-mon[32750]: Deploying daemon osd.11 on smithi077 2024-07-31T14:45:54.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:53 smithi073 ceph-mon[36714]: Deploying daemon osd.11 on smithi077 2024-07-31T14:45:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:54 smithi077 ceph-mon[37327]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:45:55.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:54 smithi067 ceph-mon[32750]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:45:55.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:54 smithi073 ceph-mon[36714]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-31T14:45:57.115 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:56 smithi077 ceph-mon[37327]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-07-31T14:45:57.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:56 smithi067 ceph-mon[32750]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-07-31T14:45:57.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:56 smithi073 ceph-mon[36714]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-07-31T14:45:58.280 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:57 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:58.280 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:57 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:58.280 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:57 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:58.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:57 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:58.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:57 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:58.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:57 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:58.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:57 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:45:58.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:57 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:58.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:57 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:45:59.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:45:58 smithi067 ceph-mon[32750]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-31T14:45:59.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:45:58 smithi073 ceph-mon[36714]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-31T14:45:59.436 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:45:58 smithi077 ceph-mon[37327]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-31T14:46:01.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:01.140+0000 7fe2b2ffd640 1 -- 172.21.15.77:0/1785349965 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe2bc1069b0 con 0x7fe2a80b3c50 2024-07-31T14:46:01.142 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 11 on host 'smithi077' 2024-07-31T14:46:01.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:01.142+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1785349965 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2a80b3c50 msgr2=0x7fe2a80b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:01.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:01.142+0000 7fe2c3fdd640 1 --2- 172.21.15.77:0/1785349965 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2a80b3c50 0x7fe2a80b6160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8005c40 tx=0x7fe2b8005bd0 comp rx=0 tx=0).stop 2024-07-31T14:46:01.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:01.142+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1785349965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2bc104ce0 msgr2=0x7fe2bc19a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:01.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:01.142+0000 7fe2c3fdd640 1 --2- 172.21.15.77:0/1785349965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2bc104ce0 0x7fe2bc19a3f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac031a90 tx=0x7fe2ac004270 comp rx=0 tx=0).stop 2024-07-31T14:46:01.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:01.142+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1785349965 shutdown_connections 2024-07-31T14:46:01.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:01.142+0000 7fe2c3fdd640 1 --2- 172.21.15.77:0/1785349965 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2a80b3c50 0x7fe2a80b6160 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:01.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:01.142+0000 7fe2c3fdd640 1 --2- 172.21.15.77:0/1785349965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2bc10d860 0x7fe2bc19f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:01.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:01.142+0000 7fe2c3fdd640 1 --2- 172.21.15.77:0/1785349965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2bc104ce0 0x7fe2bc19a3f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:01.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:01.142+0000 7fe2c3fdd640 1 --2- 172.21.15.77:0/1785349965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2bc1042b0 0x7fe2bc199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:01.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:01.142+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1785349965 >> 172.21.15.77:0/1785349965 conn(0x7fe2bc079160 msgr2=0x7fe2bc1062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:01.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:01.142+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1785349965 shutdown_connections 2024-07-31T14:46:01.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:01.142+0000 7fe2c3fdd640 1 -- 172.21.15.77:0/1785349965 wait complete. 2024-07-31T14:46:01.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:00 smithi067 ceph-mon[32750]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:46:01.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:00 smithi073 ceph-mon[36714]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:46:01.458 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:00 smithi077 ceph-mon[37327]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:46:01.796 DEBUG:teuthology.orchestra.run.smithi077:osd.11> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.11.service 2024-07-31T14:46:01.799 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-07-31T14:46:01.799 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd stat -f json 2024-07-31T14:46:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:46:02.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:02.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:02.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:02 smithi073 ceph-mon[36714]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:46:02.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:02 smithi073 ceph-mon[36714]: from='osd.11 [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-31T14:46:02.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:02 smithi073 ceph-mon[36714]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-31T14:46:02.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:02.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:02.405 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:02.405 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:02.405 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:02 smithi077 ceph-mon[37327]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:46:02.405 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:02 smithi077 ceph-mon[37327]: from='osd.11 [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-31T14:46:02.406 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:02 smithi077 ceph-mon[37327]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-31T14:46:02.406 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:02.406 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:02.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:02.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:02.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:02 smithi067 ceph-mon[32750]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:46:02.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:02 smithi067 ceph-mon[32750]: from='osd.11 [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-31T14:46:02.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:02 smithi067 ceph-mon[32750]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-31T14:46:02.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:02.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:03.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:03 smithi077 ceph-mon[37327]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-31T14:46:03.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:03 smithi077 ceph-mon[37327]: osdmap e70: 12 total, 11 up, 12 in 2024-07-31T14:46:03.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:03 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:46:03.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:03 smithi077 ceph-mon[37327]: from='osd.11 [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:46:03.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:03 smithi077 ceph-mon[37327]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:46:03.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 14:46:03 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T14:46:03.155+0000 7f6b91da3640 -1 osd.11 0 waiting for initial osdmap 2024-07-31T14:46:03.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 14:46:03 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T14:46:03.171+0000 7f6b8dba1640 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-31T14:46:03.443 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:03 smithi067 ceph-mon[32750]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-31T14:46:03.443 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:03 smithi067 ceph-mon[32750]: osdmap e70: 12 total, 11 up, 12 in 2024-07-31T14:46:03.443 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:03 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:46:03.443 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:03 smithi067 ceph-mon[32750]: from='osd.11 [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:46:03.443 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:03 smithi067 ceph-mon[32750]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:46:03.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:03 smithi073 ceph-mon[36714]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-31T14:46:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:03 smithi073 ceph-mon[36714]: osdmap e70: 12 total, 11 up, 12 in 2024-07-31T14:46:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:03 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:46:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:03 smithi073 ceph-mon[36714]: from='osd.11 [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:46:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:03 smithi073 ceph-mon[36714]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-07-31T14:46:03.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.788+0000 7fbf28d86640 1 -- 172.21.15.67:0/866498095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24107dd0 msgr2=0x7fbf241081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:03.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.788+0000 7fbf28d86640 1 --2- 172.21.15.67:0/866498095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24107dd0 0x7fbf241081e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009900 tx=0x7fbf0c02a410 comp rx=0 tx=0).stop 2024-07-31T14:46:03.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.788+0000 7fbf28d86640 1 -- 172.21.15.67:0/866498095 shutdown_connections 2024-07-31T14:46:03.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.788+0000 7fbf28d86640 1 --2- 172.21.15.67:0/866498095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf24108720 0x7fbf2410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:03.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.788+0000 7fbf28d86640 1 --2- 172.21.15.67:0/866498095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24107dd0 0x7fbf241081e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:03.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.788+0000 7fbf28d86640 1 --2- 172.21.15.67:0/866498095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf24102490 0x7fbf241028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:03.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.788+0000 7fbf28d86640 1 -- 172.21.15.67:0/866498095 >> 172.21.15.67:0/866498095 conn(0x7fbf24079160 msgr2=0x7fbf24079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:03.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.788+0000 7fbf28d86640 1 -- 172.21.15.67:0/866498095 shutdown_connections 2024-07-31T14:46:03.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.788+0000 7fbf28d86640 1 -- 172.21.15.67:0/866498095 wait complete. 2024-07-31T14:46:03.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.789+0000 7fbf28d86640 1 Processor -- start 2024-07-31T14:46:03.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.789+0000 7fbf28d86640 1 -- start start 2024-07-31T14:46:03.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.789+0000 7fbf28d86640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf24102490 0x7fbf24198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:03.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.789+0000 7fbf28d86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf24107dd0 0x7fbf24199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:03.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.789+0000 7fbf28d86640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24108720 0x7fbf2419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:03.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf28d86640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbf24112370 con 0x7fbf24108720 2024-07-31T14:46:03.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf28d86640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241121f0 con 0x7fbf24102490 2024-07-31T14:46:03.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf28d86640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241124f0 con 0x7fbf24107dd0 2024-07-31T14:46:03.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf22d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24108720 0x7fbf2419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:03.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf22d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24108720 0x7fbf2419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36820/0 (socket says 172.21.15.67:36820) 2024-07-31T14:46:03.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf22d9d640 1 -- 172.21.15.67:0/1450373952 learned_addr learned my addr 172.21.15.67:0/1450373952 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:46:03.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf21d9b640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf24107dd0 0x7fbf24199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:03.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf2259c640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf24102490 0x7fbf24198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:03.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf22d9d640 1 -- 172.21.15.67:0/1450373952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf24107dd0 msgr2=0x7fbf24199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:03.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf22d9d640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf24107dd0 0x7fbf24199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:03.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf22d9d640 1 -- 172.21.15.67:0/1450373952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf24102490 msgr2=0x7fbf24198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:03.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf22d9d640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf24102490 0x7fbf24198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:03.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf22d9d640 1 -- 172.21.15.67:0/1450373952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2419ed50 con 0x7fbf24108720 2024-07-31T14:46:03.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf21d9b640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf24107dd0 0x7fbf24199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:46:03.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf22d9d640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24108720 0x7fbf2419e620 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00ebb0 tx=0x7fbf1c007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:03.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf1b7fe640 1 -- 172.21.15.67:0/1450373952 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c007ca0 con 0x7fbf24108720 2024-07-31T14:46:03.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf1b7fe640 1 -- 172.21.15.67:0/1450373952 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf1c007e40 con 0x7fbf24108720 2024-07-31T14:46:03.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf28d86640 1 -- 172.21.15.67:0/1450373952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2419efe0 con 0x7fbf24108720 2024-07-31T14:46:03.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.790+0000 7fbf1b7fe640 1 -- 172.21.15.67:0/1450373952 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0057d0 con 0x7fbf24108720 2024-07-31T14:46:03.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.791+0000 7fbf28d86640 1 -- 172.21.15.67:0/1450373952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2406a890 con 0x7fbf24108720 2024-07-31T14:46:03.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.791+0000 7fbf28d86640 1 -- 172.21.15.67:0/1450373952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf24103710 con 0x7fbf24108720 2024-07-31T14:46:03.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.793+0000 7fbf1b7fe640 1 -- 172.21.15.67:0/1450373952 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fbf1c004a40 con 0x7fbf24108720 2024-07-31T14:46:03.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.795+0000 7fbf1b7fe640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbf100b3c20 0x7fbf100b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:03.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.795+0000 7fbf2259c640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbf100b3c20 0x7fbf100b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:03.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.795+0000 7fbf1b7fe640 1 -- 172.21.15.67:0/1450373952 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7062+0+0 (secure 0 0 0) 0x7fbf1c0e5100 con 0x7fbf24108720 2024-07-31T14:46:03.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.795+0000 7fbf2259c640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbf100b3c20 0x7fbf100b6130 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009bd0 tx=0x7fbf14005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:03.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.798+0000 7fbf1b7fe640 1 -- 172.21.15.67:0/1450373952 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fbf1c0ac5c0 con 0x7fbf24108720 2024-07-31T14:46:03.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.994+0000 7fbf28d86640 1 -- 172.21.15.67:0/1450373952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbf2406b2b0 con 0x7fbf24108720 2024-07-31T14:46:03.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.994+0000 7fbf1b7fe640 1 -- 172.21.15.67:0/1450373952 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7fbf1c00e0d0 con 0x7fbf24108720 2024-07-31T14:46:03.995 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:46:03.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.996+0000 7fbf28d86640 1 -- 172.21.15.67:0/1450373952 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbf100b3c20 msgr2=0x7fbf100b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:03.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.996+0000 7fbf28d86640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbf100b3c20 0x7fbf100b6130 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009bd0 tx=0x7fbf14005f90 comp rx=0 tx=0).stop 2024-07-31T14:46:03.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.996+0000 7fbf28d86640 1 -- 172.21.15.67:0/1450373952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24108720 msgr2=0x7fbf2419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:03.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.996+0000 7fbf28d86640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24108720 0x7fbf2419e620 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00ebb0 tx=0x7fbf1c007590 comp rx=0 tx=0).stop 2024-07-31T14:46:03.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.996+0000 7fbf28d86640 1 -- 172.21.15.67:0/1450373952 shutdown_connections 2024-07-31T14:46:03.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.997+0000 7fbf28d86640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbf100b3c20 0x7fbf100b6130 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:03.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.997+0000 7fbf28d86640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24108720 0x7fbf2419e620 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:03.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.997+0000 7fbf28d86640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf24107dd0 0x7fbf24199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:03.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.997+0000 7fbf28d86640 1 --2- 172.21.15.67:0/1450373952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf24102490 0x7fbf24198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:03.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.997+0000 7fbf28d86640 1 -- 172.21.15.67:0/1450373952 >> 172.21.15.67:0/1450373952 conn(0x7fbf24079160 msgr2=0x7fbf24103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:03.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.997+0000 7fbf28d86640 1 -- 172.21.15.67:0/1450373952 shutdown_connections 2024-07-31T14:46:03.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:03.997+0000 7fbf28d86640 1 -- 172.21.15.67:0/1450373952 wait complete. 2024-07-31T14:46:04.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:04 smithi067 ceph-mon[32750]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-07-31T14:46:04.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:04 smithi067 ceph-mon[32750]: osdmap e71: 12 total, 11 up, 12 in 2024-07-31T14:46:04.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:46:04.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:46:04.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:04 smithi067 ceph-mon[32750]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:46:04.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:04 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1450373952' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-31T14:46:04.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:04 smithi077 ceph-mon[37327]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-07-31T14:46:04.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:04 smithi077 ceph-mon[37327]: osdmap e71: 12 total, 11 up, 12 in 2024-07-31T14:46:04.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:46:04.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:46:04.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:04 smithi077 ceph-mon[37327]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:46:04.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:04 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1450373952' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-31T14:46:04.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:04 smithi073 ceph-mon[36714]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-07-31T14:46:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:04 smithi073 ceph-mon[36714]: osdmap e71: 12 total, 11 up, 12 in 2024-07-31T14:46:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:46:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:46:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:04 smithi073 ceph-mon[36714]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-07-31T14:46:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:04 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1450373952' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-31T14:46:04.879 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1722437139,"num_in_osds":12,"osd_in_since":1722437145,"num_remapped_pgs":0} 2024-07-31T14:46:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: purged_snaps scrub starts 2024-07-31T14:46:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: purged_snaps scrub ok 2024-07-31T14:46:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: osd.11 [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961] boot 2024-07-31T14:46:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: osdmap e72: 12 total, 12 up, 12 in 2024-07-31T14:46:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:46:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:46:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:46:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:46:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:46:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:05.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: purged_snaps scrub starts 2024-07-31T14:46:05.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: purged_snaps scrub ok 2024-07-31T14:46:05.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: osd.11 [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961] boot 2024-07-31T14:46:05.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: osdmap e72: 12 total, 12 up, 12 in 2024-07-31T14:46:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:46:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:46:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:46:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:46:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:46:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:05.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: purged_snaps scrub starts 2024-07-31T14:46:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: purged_snaps scrub ok 2024-07-31T14:46:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: osd.11 [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961] boot 2024-07-31T14:46:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: osdmap e72: 12 total, 12 up, 12 in 2024-07-31T14:46:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-31T14:46:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:46:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:46:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:46:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:46:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:05.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:05.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:05.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd stat -f json 2024-07-31T14:46:06.150 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:46:06.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:06 smithi067 ceph-mon[32750]: Detected new or changed devices on smithi077 2024-07-31T14:46:06.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:06 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi077 to 4272M 2024-07-31T14:46:06.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:06 smithi067 ceph-mon[32750]: osdmap e73: 12 total, 12 up, 12 in 2024-07-31T14:46:06.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:46:06.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:06.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:06.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:06.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:06 smithi067 ceph-mon[32750]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:06 smithi077 ceph-mon[37327]: Detected new or changed devices on smithi077 2024-07-31T14:46:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:06 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi077 to 4272M 2024-07-31T14:46:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:06 smithi077 ceph-mon[37327]: osdmap e73: 12 total, 12 up, 12 in 2024-07-31T14:46:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:46:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:06 smithi077 ceph-mon[37327]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:06 smithi073 ceph-mon[36714]: Detected new or changed devices on smithi077 2024-07-31T14:46:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:06 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi077 to 4272M 2024-07-31T14:46:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:06 smithi073 ceph-mon[36714]: osdmap e73: 12 total, 12 up, 12 in 2024-07-31T14:46:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:46:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:06 smithi073 ceph-mon[36714]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:07.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.858+0000 7fcfe7023640 1 -- 172.21.15.67:0/96928720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfe0101cc0 msgr2=0x7fcfe01020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:07.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.858+0000 7fcfe7023640 1 --2- 172.21.15.67:0/96928720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfe0101cc0 0x7fcfe01020d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400aa10 tx=0x7fcfd402b720 comp rx=0 tx=0).stop 2024-07-31T14:46:07.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.858+0000 7fcfe7023640 1 -- 172.21.15.67:0/96928720 shutdown_connections 2024-07-31T14:46:07.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.858+0000 7fcfe7023640 1 --2- 172.21.15.67:0/96928720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcfe0107f50 0x7fcfe010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:07.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.858+0000 7fcfe7023640 1 --2- 172.21.15.67:0/96928720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfe0107600 0x7fcfe0107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:07.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.858+0000 7fcfe7023640 1 --2- 172.21.15.67:0/96928720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfe0101cc0 0x7fcfe01020d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:07.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.858+0000 7fcfe7023640 1 -- 172.21.15.67:0/96928720 >> 172.21.15.67:0/96928720 conn(0x7fcfe00798d0 msgr2=0x7fcfe0079ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:07.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.858+0000 7fcfe7023640 1 -- 172.21.15.67:0/96928720 shutdown_connections 2024-07-31T14:46:07.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.858+0000 7fcfe7023640 1 -- 172.21.15.67:0/96928720 wait complete. 2024-07-31T14:46:07.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.859+0000 7fcfe7023640 1 Processor -- start 2024-07-31T14:46:07.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.859+0000 7fcfe7023640 1 -- start start 2024-07-31T14:46:07.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.859+0000 7fcfe7023640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcfe0101cc0 0x7fcfe0197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:07.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.859+0000 7fcfe7023640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfe0107600 0x7fcfe0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:07.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.859+0000 7fcfe7023640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfe0107f50 0x7fcfe019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:07.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.859+0000 7fcfe7023640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe0111390 con 0x7fcfe0107f50 2024-07-31T14:46:07.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.859+0000 7fcfe7023640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe0111210 con 0x7fcfe0107600 2024-07-31T14:46:07.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.859+0000 7fcfe7023640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe0111510 con 0x7fcfe0101cc0 2024-07-31T14:46:07.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.859+0000 7fcfe55c0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfe0107f50 0x7fcfe019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:07.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.859+0000 7fcfe55c0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfe0107f50 0x7fcfe019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36838/0 (socket says 172.21.15.67:36838) 2024-07-31T14:46:07.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.859+0000 7fcfe55c0640 1 -- 172.21.15.67:0/272897189 learned_addr learned my addr 172.21.15.67:0/272897189 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:46:07.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.859+0000 7fcfe4dbf640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcfe0101cc0 0x7fcfe0197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:07.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.860+0000 7fcfdffff640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfe0107600 0x7fcfe0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:07.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.860+0000 7fcfe55c0640 1 -- 172.21.15.67:0/272897189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcfe0101cc0 msgr2=0x7fcfe0197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:07.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.860+0000 7fcfe55c0640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcfe0101cc0 0x7fcfe0197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:07.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.860+0000 7fcfe55c0640 1 -- 172.21.15.67:0/272897189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfe0107600 msgr2=0x7fcfe0198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:07.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.860+0000 7fcfe55c0640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfe0107600 0x7fcfe0198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:07.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.860+0000 7fcfe55c0640 1 -- 172.21.15.67:0/272897189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe019dd30 con 0x7fcfe0107f50 2024-07-31T14:46:07.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.860+0000 7fcfdffff640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfe0107600 0x7fcfe0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:46:07.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.860+0000 7fcfe55c0640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfe0107f50 0x7fcfe019d600 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800eaa0 tx=0x7fcfd800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:07.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.860+0000 7fcfddffb640 1 -- 172.21.15.67:0/272897189 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd800cdb0 con 0x7fcfe0107f50 2024-07-31T14:46:07.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.860+0000 7fcfddffb640 1 -- 172.21.15.67:0/272897189 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd8005020 con 0x7fcfe0107f50 2024-07-31T14:46:07.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.860+0000 7fcfe7023640 1 -- 172.21.15.67:0/272897189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe019e020 con 0x7fcfe0107f50 2024-07-31T14:46:07.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.860+0000 7fcfddffb640 1 -- 172.21.15.67:0/272897189 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80116d0 con 0x7fcfe0107f50 2024-07-31T14:46:07.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.860+0000 7fcfe7023640 1 -- 172.21.15.67:0/272897189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe01a4780 con 0x7fcfe0107f50 2024-07-31T14:46:07.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.861+0000 7fcfe7023640 1 -- 172.21.15.67:0/272897189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe01020d0 con 0x7fcfe0107f50 2024-07-31T14:46:07.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.863+0000 7fcfddffb640 1 -- 172.21.15.67:0/272897189 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fcfd80051d0 con 0x7fcfe0107f50 2024-07-31T14:46:07.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.864+0000 7fcfddffb640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcfd00b3c20 0x7fcfd00b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:07.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.864+0000 7fcfe4dbf640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcfd00b3c20 0x7fcfd00b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:07.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.864+0000 7fcfddffb640 1 -- 172.21.15.67:0/272897189 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7342+0+0 (secure 0 0 0) 0x7fcfd80e3a40 con 0x7fcfe0107f50 2024-07-31T14:46:07.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.864+0000 7fcfe4dbf640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcfd00b3c20 0x7fcfd00b6130 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4030550 tx=0x7fcfd400a670 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:07.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:07.868+0000 7fcfddffb640 1 -- 172.21.15.67:0/272897189 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fcfd8016030 con 0x7fcfe0107f50 2024-07-31T14:46:08.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:08.064+0000 7fcfe7023640 1 -- 172.21.15.67:0/272897189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fcfe0109d80 con 0x7fcfe0107f50 2024-07-31T14:46:08.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:08.064+0000 7fcfddffb640 1 -- 172.21.15.67:0/272897189 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7fcfd80ab460 con 0x7fcfe0107f50 2024-07-31T14:46:08.065 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:46:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:08.066+0000 7fcfe7023640 1 -- 172.21.15.67:0/272897189 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcfd00b3c20 msgr2=0x7fcfd00b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:08.066+0000 7fcfe7023640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcfd00b3c20 0x7fcfd00b6130 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4030550 tx=0x7fcfd400a670 comp rx=0 tx=0).stop 2024-07-31T14:46:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:08.066+0000 7fcfe7023640 1 -- 172.21.15.67:0/272897189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfe0107f50 msgr2=0x7fcfe019d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:08.066+0000 7fcfe7023640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfe0107f50 0x7fcfe019d600 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800eaa0 tx=0x7fcfd800ef60 comp rx=0 tx=0).stop 2024-07-31T14:46:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:08.066+0000 7fcfe7023640 1 -- 172.21.15.67:0/272897189 shutdown_connections 2024-07-31T14:46:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:08.066+0000 7fcfe7023640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcfd00b3c20 0x7fcfd00b6130 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:08.066+0000 7fcfe7023640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfe0107f50 0x7fcfe019d600 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:08.066+0000 7fcfe7023640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfe0107600 0x7fcfe0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:08.066+0000 7fcfe7023640 1 --2- 172.21.15.67:0/272897189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcfe0101cc0 0x7fcfe0197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:08.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:08.066+0000 7fcfe7023640 1 -- 172.21.15.67:0/272897189 >> 172.21.15.67:0/272897189 conn(0x7fcfe00798d0 msgr2=0x7fcfe01056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:08.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:08.066+0000 7fcfe7023640 1 -- 172.21.15.67:0/272897189 shutdown_connections 2024-07-31T14:46:08.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:08.066+0000 7fcfe7023640 1 -- 172.21.15.67:0/272897189 wait complete. 2024-07-31T14:46:08.622 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1722437164,"num_in_osds":12,"osd_in_since":1722437145,"num_remapped_pgs":0} 2024-07-31T14:46:08.623 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-07-31T14:46:08.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:08 smithi077 ceph-mon[37327]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:08.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:08 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/272897189' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-31T14:46:08.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:08 smithi067 ceph-mon[32750]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:08.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:08 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/272897189' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-31T14:46:08.885 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:46:08.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:08 smithi073 ceph-mon[36714]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:08.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:08 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/272897189' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-31T14:46:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.583+0000 7f6f215ad640 1 -- 172.21.15.67:0/1578712787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f1c105cc0 msgr2=0x7f6f1c1060d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.583+0000 7f6f215ad640 1 --2- 172.21.15.67:0/1578712787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f1c105cc0 0x7f6f1c1060d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009900 tx=0x7f6f0402a410 comp rx=0 tx=0).stop 2024-07-31T14:46:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.583+0000 7f6f215ad640 1 -- 172.21.15.67:0/1578712787 shutdown_connections 2024-07-31T14:46:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.583+0000 7f6f215ad640 1 --2- 172.21.15.67:0/1578712787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f1c106610 0x7f6f1c110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.583+0000 7f6f215ad640 1 --2- 172.21.15.67:0/1578712787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f1c105cc0 0x7f6f1c1060d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.583+0000 7f6f215ad640 1 --2- 172.21.15.67:0/1578712787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f1c1019e0 0x7f6f1c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.583+0000 7f6f215ad640 1 -- 172.21.15.67:0/1578712787 >> 172.21.15.67:0/1578712787 conn(0x7f6f1c077690 msgr2=0x7f6f1c077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.583+0000 7f6f215ad640 1 -- 172.21.15.67:0/1578712787 shutdown_connections 2024-07-31T14:46:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.583+0000 7f6f215ad640 1 -- 172.21.15.67:0/1578712787 wait complete. 2024-07-31T14:46:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.584+0000 7f6f215ad640 1 Processor -- start 2024-07-31T14:46:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.584+0000 7f6f215ad640 1 -- start start 2024-07-31T14:46:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.584+0000 7f6f215ad640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f1c1019e0 0x7f6f1c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.584+0000 7f6f215ad640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f1c105cc0 0x7f6f1c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.584+0000 7f6f215ad640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f1c106610 0x7f6f1c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.584+0000 7f6f215ad640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1134a0 con 0x7f6f1c1019e0 2024-07-31T14:46:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.584+0000 7f6f215ad640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c113320 con 0x7f6f1c105cc0 2024-07-31T14:46:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.584+0000 7f6f215ad640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c113620 con 0x7f6f1c106610 2024-07-31T14:46:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.584+0000 7f6f1affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f1c1019e0 0x7f6f1c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.584+0000 7f6f1affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f1c1019e0 0x7f6f1c199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36854/0 (socket says 172.21.15.67:36854) 2024-07-31T14:46:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.584+0000 7f6f1affd640 1 -- 172.21.15.67:0/3260955308 learned_addr learned my addr 172.21.15.67:0/3260955308 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:46:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.584+0000 7f6f1b7fe640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f1c106610 0x7f6f1c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.585+0000 7f6f1a7fc640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f1c105cc0 0x7f6f1c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.585+0000 7f6f1affd640 1 -- 172.21.15.67:0/3260955308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f1c106610 msgr2=0x7f6f1c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.585+0000 7f6f1affd640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f1c106610 0x7f6f1c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.585+0000 7f6f1affd640 1 -- 172.21.15.67:0/3260955308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f1c105cc0 msgr2=0x7f6f1c19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.585+0000 7f6f1affd640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f1c105cc0 0x7f6f1c19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.585+0000 7f6f1affd640 1 -- 172.21.15.67:0/3260955308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c19ff00 con 0x7f6f1c1019e0 2024-07-31T14:46:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.585+0000 7f6f1a7fc640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f1c105cc0 0x7f6f1c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:46:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.585+0000 7f6f1affd640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f1c1019e0 0x7f6f1c199f90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400c970 tx=0x7f6f1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.585+0000 7f6f13fff640 1 -- 172.21.15.67:0/3260955308 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14007bb0 con 0x7f6f1c1019e0 2024-07-31T14:46:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.585+0000 7f6f13fff640 1 -- 172.21.15.67:0/3260955308 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f14007d50 con 0x7f6f1c1019e0 2024-07-31T14:46:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.585+0000 7f6f215ad640 1 -- 172.21.15.67:0/3260955308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c1a01f0 con 0x7f6f1c1019e0 2024-07-31T14:46:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.585+0000 7f6f13fff640 1 -- 172.21.15.67:0/3260955308 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1400f450 con 0x7f6f1c1019e0 2024-07-31T14:46:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.585+0000 7f6f215ad640 1 -- 172.21.15.67:0/3260955308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c1a6950 con 0x7f6f1c1019e0 2024-07-31T14:46:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.586+0000 7f6f215ad640 1 -- 172.21.15.67:0/3260955308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f1c101df0 con 0x7f6f1c1019e0 2024-07-31T14:46:10.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.588+0000 7f6f13fff640 1 -- 172.21.15.67:0/3260955308 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f6f14004f00 con 0x7f6f1c1019e0 2024-07-31T14:46:10.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.588+0000 7f6f13fff640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6f080b3c20 0x7f6f080b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:10.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.589+0000 7f6f13fff640 1 -- 172.21.15.67:0/3260955308 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7342+0+0 (secure 0 0 0) 0x7f6f140e39d0 con 0x7f6f1c1019e0 2024-07-31T14:46:10.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.589+0000 7f6f1a7fc640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6f080b3c20 0x7f6f080b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:10.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.589+0000 7f6f1a7fc640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6f080b3c20 0x7f6f080b6130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009230 tx=0x7f6f04000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:10.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.592+0000 7f6f13fff640 1 -- 172.21.15.67:0/3260955308 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f6f14014030 con 0x7f6f1c1019e0 2024-07-31T14:46:10.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:10 smithi077 ceph-mon[37327]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:10.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.786+0000 7f6f215ad640 1 -- 172.21.15.67:0/3260955308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f1c02e0c0 con 0x7f6f1c1019e0 2024-07-31T14:46:10.787 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:10 smithi067 ceph-mon[32750]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:10.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.787+0000 7f6f13fff640 1 -- 172.21.15.67:0/3260955308 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18674 (secure 0 0 0) 0x7f6f140ab3f0 con 0x7f6f1c1019e0 2024-07-31T14:46:10.789 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:46:10.789 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":73,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T14:46:05.162233+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T14:46:10.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.789+0000 7f6f215ad640 1 -- 172.21.15.67:0/3260955308 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6f080b3c20 msgr2=0x7f6f080b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:10.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.789+0000 7f6f215ad640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6f080b3c20 0x7f6f080b6130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009230 tx=0x7f6f04000bc0 comp rx=0 tx=0).stop 2024-07-31T14:46:10.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.789+0000 7f6f215ad640 1 -- 172.21.15.67:0/3260955308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f1c1019e0 msgr2=0x7f6f1c199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:10.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.789+0000 7f6f215ad640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f1c1019e0 0x7f6f1c199f90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400c970 tx=0x7f6f1400ce30 comp rx=0 tx=0).stop 2024-07-31T14:46:10.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.789+0000 7f6f215ad640 1 -- 172.21.15.67:0/3260955308 shutdown_connections 2024-07-31T14:46:10.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.789+0000 7f6f215ad640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6f080b3c20 0x7f6f080b6130 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:10.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.789+0000 7f6f215ad640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f1c106610 0x7f6f1c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:10.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.789+0000 7f6f215ad640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f1c105cc0 0x7f6f1c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:10.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.789+0000 7f6f215ad640 1 --2- 172.21.15.67:0/3260955308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f1c1019e0 0x7f6f1c199f90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:10.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.789+0000 7f6f215ad640 1 -- 172.21.15.67:0/3260955308 >> 172.21.15.67:0/3260955308 conn(0x7f6f1c077690 msgr2=0x7f6f1c107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:10.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.789+0000 7f6f215ad640 1 -- 172.21.15.67:0/3260955308 shutdown_connections 2024-07-31T14:46:10.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:10.789+0000 7f6f215ad640 1 -- 172.21.15.67:0/3260955308 wait complete. 2024-07-31T14:46:10.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:10 smithi073 ceph-mon[36714]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:11.535 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-31T14:42:27.359626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-31T14:46:11.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-07-31T14:46:11.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:11 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3260955308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:46:11.801 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:46:11.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:11 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3260955308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:46:11.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:11 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3260955308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:46:12.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:12 smithi067 ceph-mon[32750]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:12.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:12 smithi077 ceph-mon[37327]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:12.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:12 smithi073 ceph-mon[36714]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:13.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.477+0000 7f1937bb6640 1 -- 172.21.15.67:0/3969846400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930108ef0 msgr2=0x7f1930109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:13.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.477+0000 7f1937bb6640 1 --2- 172.21.15.67:0/3969846400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930108ef0 0x7f1930109300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1924009900 tx=0x7f192402a3f0 comp rx=0 tx=0).stop 2024-07-31T14:46:13.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.477+0000 7f1937bb6640 1 -- 172.21.15.67:0/3969846400 shutdown_connections 2024-07-31T14:46:13.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.477+0000 7f1937bb6640 1 --2- 172.21.15.67:0/3969846400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1930109840 0x7f1930110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:13.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.477+0000 7f1937bb6640 1 --2- 172.21.15.67:0/3969846400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930108ef0 0x7f1930109300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:13.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.477+0000 7f1937bb6640 1 --2- 172.21.15.67:0/3969846400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19301035b0 0x7f19301039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:13.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.477+0000 7f1937bb6640 1 -- 172.21.15.67:0/3969846400 >> 172.21.15.67:0/3969846400 conn(0x7f19300ff040 msgr2=0x7f1930101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:13.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.477+0000 7f1937bb6640 1 -- 172.21.15.67:0/3969846400 shutdown_connections 2024-07-31T14:46:13.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.477+0000 7f1937bb6640 1 -- 172.21.15.67:0/3969846400 wait complete. 2024-07-31T14:46:13.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.478+0000 7f1937bb6640 1 Processor -- start 2024-07-31T14:46:13.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.478+0000 7f1937bb6640 1 -- start start 2024-07-31T14:46:13.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.478+0000 7f1937bb6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19301035b0 0x7f1930075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:13.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.478+0000 7f1937bb6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930108ef0 0x7f1930075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:13.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.478+0000 7f1937bb6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1930109840 0x7f1930075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:13.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1937bb6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1930113500 con 0x7f1930108ef0 2024-07-31T14:46:13.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1937bb6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1930113380 con 0x7f1930109840 2024-07-31T14:46:13.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1937bb6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1930113680 con 0x7f19301035b0 2024-07-31T14:46:13.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1935151640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930108ef0 0x7f1930075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:13.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1935151640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930108ef0 0x7f1930075680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41818/0 (socket says 172.21.15.67:41818) 2024-07-31T14:46:13.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1935151640 1 -- 172.21.15.67:0/2393094502 learned_addr learned my addr 172.21.15.67:0/2393094502 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:46:13.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1935952640 1 --2- 172.21.15.67:0/2393094502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19301035b0 0x7f1930075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:13.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1936153640 1 --2- 172.21.15.67:0/2393094502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1930109840 0x7f1930075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:13.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1935151640 1 -- 172.21.15.67:0/2393094502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19301035b0 msgr2=0x7f1930075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:13.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1935151640 1 --2- 172.21.15.67:0/2393094502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19301035b0 0x7f1930075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:13.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1935151640 1 -- 172.21.15.67:0/2393094502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1930109840 msgr2=0x7f1930075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:13.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1935151640 1 --2- 172.21.15.67:0/2393094502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1930109840 0x7f1930075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:13.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1935151640 1 -- 172.21.15.67:0/2393094502 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19301a8160 con 0x7f1930108ef0 2024-07-31T14:46:13.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1935151640 1 --2- 172.21.15.67:0/2393094502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930108ef0 0x7f1930075680 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1924005ab0 tx=0x7f1924003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:13.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f192effd640 1 -- 172.21.15.67:0/2393094502 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192403f070 con 0x7f1930108ef0 2024-07-31T14:46:13.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f192effd640 1 -- 172.21.15.67:0/2393094502 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1924031d60 con 0x7f1930108ef0 2024-07-31T14:46:13.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1937bb6640 1 -- 172.21.15.67:0/2393094502 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19301a83f0 con 0x7f1930108ef0 2024-07-31T14:46:13.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f192effd640 1 -- 172.21.15.67:0/2393094502 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924035030 con 0x7f1930108ef0 2024-07-31T14:46:13.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.479+0000 7f1937bb6640 1 -- 172.21.15.67:0/2393094502 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19301a88d0 con 0x7f1930108ef0 2024-07-31T14:46:13.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.480+0000 7f1937bb6640 1 -- 172.21.15.67:0/2393094502 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1930104830 con 0x7f1930108ef0 2024-07-31T14:46:13.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.482+0000 7f192effd640 1 -- 172.21.15.67:0/2393094502 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f1924042070 con 0x7f1930108ef0 2024-07-31T14:46:13.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.483+0000 7f192effd640 1 --2- 172.21.15.67:0/2393094502 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19180b3c20 0x7f19180b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:13.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.483+0000 7f1935952640 1 --2- 172.21.15.67:0/2393094502 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19180b3c20 0x7f19180b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:13.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.483+0000 7f192effd640 1 -- 172.21.15.67:0/2393094502 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7342+0+0 (secure 0 0 0) 0x7f19241047c0 con 0x7f1930108ef0 2024-07-31T14:46:13.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.483+0000 7f1935952640 1 --2- 172.21.15.67:0/2393094502 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19180b3c20 0x7f19180b6130 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1930076b90 tx=0x7f192800a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:13.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.486+0000 7f192effd640 1 -- 172.21.15.67:0/2393094502 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f19240cc1e0 con 0x7f1930108ef0 2024-07-31T14:46:13.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.680+0000 7f1937bb6640 1 -- 172.21.15.67:0/2393094502 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f193010b650 con 0x7f1930108ef0 2024-07-31T14:46:13.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.680+0000 7f192effd640 1 -- 172.21.15.67:0/2393094502 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f19240cbae0 con 0x7f1930108ef0 2024-07-31T14:46:13.682 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-07-31T14:46:13.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.682+0000 7f1937bb6640 1 -- 172.21.15.67:0/2393094502 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19180b3c20 msgr2=0x7f19180b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:13.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.682+0000 7f1937bb6640 1 --2- 172.21.15.67:0/2393094502 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19180b3c20 0x7f19180b6130 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1930076b90 tx=0x7f192800a2d0 comp rx=0 tx=0).stop 2024-07-31T14:46:13.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.682+0000 7f1937bb6640 1 -- 172.21.15.67:0/2393094502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930108ef0 msgr2=0x7f1930075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:13.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.683+0000 7f1937bb6640 1 --2- 172.21.15.67:0/2393094502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930108ef0 0x7f1930075680 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1924005ab0 tx=0x7f1924003cd0 comp rx=0 tx=0).stop 2024-07-31T14:46:13.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.683+0000 7f1937bb6640 1 -- 172.21.15.67:0/2393094502 shutdown_connections 2024-07-31T14:46:13.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.683+0000 7f1937bb6640 1 --2- 172.21.15.67:0/2393094502 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19180b3c20 0x7f19180b6130 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:13.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.683+0000 7f1937bb6640 1 --2- 172.21.15.67:0/2393094502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1930109840 0x7f1930075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:13.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.683+0000 7f1937bb6640 1 --2- 172.21.15.67:0/2393094502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930108ef0 0x7f1930075680 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:13.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.683+0000 7f1937bb6640 1 --2- 172.21.15.67:0/2393094502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19301035b0 0x7f1930075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:13.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.683+0000 7f1937bb6640 1 -- 172.21.15.67:0/2393094502 >> 172.21.15.67:0/2393094502 conn(0x7f19300ff040 msgr2=0x7f1930101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:13.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.683+0000 7f1937bb6640 1 -- 172.21.15.67:0/2393094502 shutdown_connections 2024-07-31T14:46:13.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:13.683+0000 7f1937bb6640 1 -- 172.21.15.67:0/2393094502 wait complete. 2024-07-31T14:46:14.273 INFO:tasks.cephadm:Adding mds.a on smithi067 2024-07-31T14:46:14.273 INFO:tasks.cephadm:Adding mds.d on smithi067 2024-07-31T14:46:14.273 INFO:tasks.cephadm:Adding mds.g on smithi067 2024-07-31T14:46:14.273 INFO:tasks.cephadm:Adding mds.j on smithi067 2024-07-31T14:46:14.274 INFO:tasks.cephadm:Adding mds.b on smithi073 2024-07-31T14:46:14.274 INFO:tasks.cephadm:Adding mds.e on smithi073 2024-07-31T14:46:14.274 INFO:tasks.cephadm:Adding mds.h on smithi073 2024-07-31T14:46:14.274 INFO:tasks.cephadm:Adding mds.k on smithi073 2024-07-31T14:46:14.274 INFO:tasks.cephadm:Adding mds.c on smithi077 2024-07-31T14:46:14.274 INFO:tasks.cephadm:Adding mds.f on smithi077 2024-07-31T14:46:14.274 INFO:tasks.cephadm:Adding mds.i on smithi077 2024-07-31T14:46:14.274 INFO:tasks.cephadm:Adding mds.l on smithi077 2024-07-31T14:46:14.274 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph orch apply mds all '12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l' 2024-07-31T14:46:14.540 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T14:46:14.662 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:14 smithi077 ceph-mon[37327]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:14.662 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:14 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2393094502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-31T14:46:14.762 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:14 smithi067 ceph-mon[32750]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:14.762 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:14 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2393094502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-31T14:46:14.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:14 smithi073 ceph-mon[36714]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:14.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:14 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2393094502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-31T14:46:16.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.182+0000 7f161126b640 1 -- 172.21.15.77:0/4252724350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c109840 msgr2=0x7f160c110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:16.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.182+0000 7f161126b640 1 --2- 172.21.15.77:0/4252724350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c109840 0x7f160c110070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1604009900 tx=0x7f160402a410 comp rx=0 tx=0).stop 2024-07-31T14:46:16.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.182+0000 7f161126b640 1 -- 172.21.15.77:0/4252724350 shutdown_connections 2024-07-31T14:46:16.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.182+0000 7f161126b640 1 --2- 172.21.15.77:0/4252724350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c109840 0x7f160c110070 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:16.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.182+0000 7f161126b640 1 --2- 172.21.15.77:0/4252724350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f160c108ef0 0x7f160c109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:16.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.182+0000 7f161126b640 1 --2- 172.21.15.77:0/4252724350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c1035b0 0x7f160c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:16.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.182+0000 7f161126b640 1 -- 172.21.15.77:0/4252724350 >> 172.21.15.77:0/4252724350 conn(0x7f160c0ff040 msgr2=0x7f160c101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:16.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.182+0000 7f161126b640 1 -- 172.21.15.77:0/4252724350 shutdown_connections 2024-07-31T14:46:16.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.182+0000 7f161126b640 1 -- 172.21.15.77:0/4252724350 wait complete. 2024-07-31T14:46:16.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.183+0000 7f161126b640 1 Processor -- start 2024-07-31T14:46:16.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.183+0000 7f161126b640 1 -- start start 2024-07-31T14:46:16.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.183+0000 7f161126b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f160c1035b0 0x7f160c199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:16.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.183+0000 7f161126b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c108ef0 0x7f160c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:16.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.183+0000 7f161126b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c109840 0x7f160c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:16.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.183+0000 7f161126b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f160c113540 con 0x7f160c108ef0 2024-07-31T14:46:16.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.183+0000 7f161126b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1133c0 con 0x7f160c1035b0 2024-07-31T14:46:16.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.183+0000 7f161126b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1136c0 con 0x7f160c109840 2024-07-31T14:46:16.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.183+0000 7f160b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c109840 0x7f160c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:16.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.183+0000 7f160b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c109840 0x7f160c19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45394/0 (socket says 172.21.15.77:45394) 2024-07-31T14:46:16.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.183+0000 7f160b7fe640 1 -- 172.21.15.77:0/1430206861 learned_addr learned my addr 172.21.15.77:0/1430206861 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-07-31T14:46:16.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.183+0000 7f160affd640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f160c1035b0 0x7f160c199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:16.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.184+0000 7f160a7fc640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c108ef0 0x7f160c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:16.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.184+0000 7f160b7fe640 1 -- 172.21.15.77:0/1430206861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f160c1035b0 msgr2=0x7f160c199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:16.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.184+0000 7f160b7fe640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f160c1035b0 0x7f160c199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:16.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.184+0000 7f160b7fe640 1 -- 172.21.15.77:0/1430206861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c108ef0 msgr2=0x7f160c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:16.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.184+0000 7f160b7fe640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c108ef0 0x7f160c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:16.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.184+0000 7f160b7fe640 1 -- 172.21.15.77:0/1430206861 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c19fe00 con 0x7f160c109840 2024-07-31T14:46:16.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.184+0000 7f160a7fc640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c108ef0 0x7f160c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:46:16.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.184+0000 7f160b7fe640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c109840 0x7f160c19f6d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1604004770 tx=0x7f1604004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:16.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.184+0000 7f15f3fff640 1 -- 172.21.15.77:0/1430206861 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160403f030 con 0x7f160c109840 2024-07-31T14:46:16.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.184+0000 7f161126b640 1 -- 172.21.15.77:0/1430206861 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160c1a0090 con 0x7f160c109840 2024-07-31T14:46:16.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.184+0000 7f15f3fff640 1 -- 172.21.15.77:0/1430206861 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1604003aa0 con 0x7f160c109840 2024-07-31T14:46:16.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.184+0000 7f15f3fff640 1 -- 172.21.15.77:0/1430206861 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1604035070 con 0x7f160c109840 2024-07-31T14:46:16.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.184+0000 7f161126b640 1 -- 172.21.15.77:0/1430206861 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160c1a6850 con 0x7f160c109840 2024-07-31T14:46:16.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.185+0000 7f161126b640 1 -- 172.21.15.77:0/1430206861 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160c1039c0 con 0x7f160c109840 2024-07-31T14:46:16.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.187+0000 7f15f3fff640 1 -- 172.21.15.77:0/1430206861 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f1604042030 con 0x7f160c109840 2024-07-31T14:46:16.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.188+0000 7f15f3fff640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15f40b3c20 0x7f15f40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:16.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.188+0000 7f15f3fff640 1 -- 172.21.15.77:0/1430206861 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7342+0+0 (secure 0 0 0) 0x7f16041042f0 con 0x7f160c109840 2024-07-31T14:46:16.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.188+0000 7f160affd640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15f40b3c20 0x7f15f40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:16.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.189+0000 7f160affd640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15f40b3c20 0x7f15f40b6130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f15fc005fd0 tx=0x7f15fc009680 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:16.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.191+0000 7f15f3fff640 1 -- 172.21.15.77:0/1430206861 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f16040cbcf0 con 0x7f160c109840 2024-07-31T14:46:16.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.391+0000 7f161126b640 1 -- 172.21.15.77:0/1430206861 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l", "target": ["mon-mgr", ""]}) -- 0x7f160c063230 con 0x7f15f40b3c20 2024-07-31T14:46:16.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.400+0000 7f15f3fff640 1 -- 172.21.15.77:0/1430206861 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f160c063230 con 0x7f15f40b3c20 2024-07-31T14:46:16.401 INFO:teuthology.orchestra.run.smithi077.stdout:Scheduled mds.all update... 2024-07-31T14:46:16.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.402+0000 7f161126b640 1 -- 172.21.15.77:0/1430206861 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15f40b3c20 msgr2=0x7f15f40b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:16.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.402+0000 7f161126b640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15f40b3c20 0x7f15f40b6130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f15fc005fd0 tx=0x7f15fc009680 comp rx=0 tx=0).stop 2024-07-31T14:46:16.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.402+0000 7f161126b640 1 -- 172.21.15.77:0/1430206861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c109840 msgr2=0x7f160c19f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:16.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.402+0000 7f161126b640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c109840 0x7f160c19f6d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1604004770 tx=0x7f1604004270 comp rx=0 tx=0).stop 2024-07-31T14:46:16.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.402+0000 7f161126b640 1 -- 172.21.15.77:0/1430206861 shutdown_connections 2024-07-31T14:46:16.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.402+0000 7f161126b640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15f40b3c20 0x7f15f40b6130 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:16.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.402+0000 7f161126b640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c109840 0x7f160c19f6d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:16.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.402+0000 7f161126b640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c108ef0 0x7f160c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:16.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.402+0000 7f161126b640 1 --2- 172.21.15.77:0/1430206861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f160c1035b0 0x7f160c199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:16.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.402+0000 7f161126b640 1 -- 172.21.15.77:0/1430206861 >> 172.21.15.77:0/1430206861 conn(0x7f160c0ff040 msgr2=0x7f160c108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:16.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.402+0000 7f161126b640 1 -- 172.21.15.77:0/1430206861 shutdown_connections 2024-07-31T14:46:16.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-07-31T14:46:16.402+0000 7f161126b640 1 -- 172.21.15.77:0/1430206861 wait complete. 2024-07-31T14:46:16.684 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:16 smithi077 ceph-mon[37327]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:16.685 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:16.685 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:46:16.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:16 smithi067 ceph-mon[32750]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:16.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:16.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:46:16.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:16 smithi073 ceph-mon[36714]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:16.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:16.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:46:17.136 DEBUG:teuthology.orchestra.run.smithi067:mds.a> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.a.service 2024-07-31T14:46:17.139 DEBUG:teuthology.orchestra.run.smithi067:mds.d> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.d.service 2024-07-31T14:46:17.182 DEBUG:teuthology.orchestra.run.smithi067:mds.g> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.g.service 2024-07-31T14:46:17.225 DEBUG:teuthology.orchestra.run.smithi067:mds.j> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.j.service 2024-07-31T14:46:17.268 DEBUG:teuthology.orchestra.run.smithi073:mds.b> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.b.service 2024-07-31T14:46:17.270 DEBUG:teuthology.orchestra.run.smithi073:mds.e> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.e.service 2024-07-31T14:46:17.313 DEBUG:teuthology.orchestra.run.smithi073:mds.h> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.h.service 2024-07-31T14:46:17.356 DEBUG:teuthology.orchestra.run.smithi073:mds.k> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.k.service 2024-07-31T14:46:17.399 DEBUG:teuthology.orchestra.run.smithi077:mds.c> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.c.service 2024-07-31T14:46:17.401 DEBUG:teuthology.orchestra.run.smithi077:mds.f> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.f.service 2024-07-31T14:46:17.444 DEBUG:teuthology.orchestra.run.smithi077:mds.i> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.i.service 2024-07-31T14:46:17.487 DEBUG:teuthology.orchestra.run.smithi077:mds.l> sudo journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.l.service 2024-07-31T14:46:17.530 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-07-31T14:46:17.530 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:46:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:17 smithi073 ceph-mon[36714]: from='client.24598 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:46:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:17 smithi073 ceph-mon[36714]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l;count:12 2024-07-31T14:46:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:17 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:17 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:17 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:17 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:17 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:17 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:17 smithi073 ceph-mon[36714]: Deploying daemon mds.l on smithi077 2024-07-31T14:46:17.672 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:17 smithi077 ceph-mon[37327]: from='client.24598 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:46:17.672 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:17 smithi077 ceph-mon[37327]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l;count:12 2024-07-31T14:46:17.672 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:17 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:17.673 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:17 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:17.673 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:17 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:17.673 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:17 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:17.673 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:17 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:17.673 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:17 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:17.673 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:17 smithi077 ceph-mon[37327]: Deploying daemon mds.l on smithi077 2024-07-31T14:46:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:46:17.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:17 smithi067 ceph-mon[32750]: from='client.24598 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:46:17.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:17 smithi067 ceph-mon[32750]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l;count:12 2024-07-31T14:46:17.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:17 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:17.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:17 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:17.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:17 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:17.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:17 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:17.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:17 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:17.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:17 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:17.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:17 smithi067 ceph-mon[32750]: Deploying daemon mds.l on smithi077 2024-07-31T14:46:18.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:18 smithi077 ceph-mon[37327]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:19.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:18 smithi073 ceph-mon[36714]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:19.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:18 smithi067 ceph-mon[32750]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.403+0000 7f1735525640 1 -- 172.21.15.67:0/2507525487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1730108740 msgr2=0x7f173010ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.403+0000 7f1735525640 1 --2- 172.21.15.67:0/2507525487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1730108740 0x7f173010ef70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1728009ca0 tx=0x7f172802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:46:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.403+0000 7f1735525640 1 -- 172.21.15.67:0/2507525487 shutdown_connections 2024-07-31T14:46:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.403+0000 7f1735525640 1 --2- 172.21.15.67:0/2507525487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1730108740 0x7f173010ef70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.403+0000 7f1735525640 1 --2- 172.21.15.67:0/2507525487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1730107df0 0x7f1730108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.403+0000 7f1735525640 1 --2- 172.21.15.67:0/2507525487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17301024b0 0x7f17301028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.403+0000 7f1735525640 1 -- 172.21.15.67:0/2507525487 >> 172.21.15.67:0/2507525487 conn(0x7f1730079170 msgr2=0x7f1730079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:19.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.403+0000 7f1735525640 1 -- 172.21.15.67:0/2507525487 shutdown_connections 2024-07-31T14:46:19.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.403+0000 7f1735525640 1 -- 172.21.15.67:0/2507525487 wait complete. 2024-07-31T14:46:19.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.404+0000 7f1735525640 1 Processor -- start 2024-07-31T14:46:19.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.404+0000 7f1735525640 1 -- start start 2024-07-31T14:46:19.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.404+0000 7f1735525640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17301024b0 0x7f173019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:19.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.404+0000 7f1735525640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1730107df0 0x7f173019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:19.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.404+0000 7f1735525640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1730108740 0x7f173019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:19.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.404+0000 7f1735525640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f17301123a0 con 0x7f1730107df0 2024-07-31T14:46:19.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.404+0000 7f1735525640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1730112220 con 0x7f17301024b0 2024-07-31T14:46:19.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.404+0000 7f1735525640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1730112520 con 0x7f1730108740 2024-07-31T14:46:19.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.404+0000 7f172e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1730107df0 0x7f173019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:19.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.404+0000 7f172e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1730107df0 0x7f173019b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41850/0 (socket says 172.21.15.67:41850) 2024-07-31T14:46:19.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.404+0000 7f172e7fc640 1 -- 172.21.15.67:0/2540071094 learned_addr learned my addr 172.21.15.67:0/2540071094 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:46:19.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.404+0000 7f172f7fe640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1730108740 0x7f173019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:19.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.404+0000 7f172effd640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17301024b0 0x7f173019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:19.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.405+0000 7f172e7fc640 1 -- 172.21.15.67:0/2540071094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1730108740 msgr2=0x7f173019b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:19.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.405+0000 7f172e7fc640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1730108740 0x7f173019b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:19.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.405+0000 7f172e7fc640 1 -- 172.21.15.67:0/2540071094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17301024b0 msgr2=0x7f173019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:19.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.405+0000 7f172e7fc640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17301024b0 0x7f173019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:19.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.405+0000 7f172e7fc640 1 -- 172.21.15.67:0/2540071094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17301a5f20 con 0x7f1730107df0 2024-07-31T14:46:19.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.405+0000 7f172effd640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17301024b0 0x7f173019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:46:19.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.405+0000 7f172e7fc640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1730107df0 0x7f173019b1d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1724005fd0 tx=0x7f172400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:19.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.405+0000 7f1717fff640 1 -- 172.21.15.67:0/2540071094 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724017070 con 0x7f1730107df0 2024-07-31T14:46:19.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.405+0000 7f1735525640 1 -- 172.21.15.67:0/2540071094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17301a6210 con 0x7f1730107df0 2024-07-31T14:46:19.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.405+0000 7f1717fff640 1 -- 172.21.15.67:0/2540071094 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1724009c30 con 0x7f1730107df0 2024-07-31T14:46:19.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.405+0000 7f1717fff640 1 -- 172.21.15.67:0/2540071094 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724003d30 con 0x7f1730107df0 2024-07-31T14:46:19.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.405+0000 7f1735525640 1 -- 172.21.15.67:0/2540071094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17301a67a0 con 0x7f1730107df0 2024-07-31T14:46:19.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.406+0000 7f1735525640 1 -- 172.21.15.67:0/2540071094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1730103730 con 0x7f1730107df0 2024-07-31T14:46:19.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.408+0000 7f1717fff640 1 -- 172.21.15.67:0/2540071094 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f1724003ed0 con 0x7f1730107df0 2024-07-31T14:46:19.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.408+0000 7f1717fff640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f17180b3c50 0x7f17180b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:19.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.409+0000 7f172effd640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f17180b3c50 0x7f17180b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:19.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.409+0000 7f1717fff640 1 -- 172.21.15.67:0/2540071094 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7342+0+0 (secure 0 0 0) 0x7f17240e3500 con 0x7f1730107df0 2024-07-31T14:46:19.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.409+0000 7f172effd640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f17180b3c50 0x7f17180b6160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f17200058c0 tx=0x7f1720005d50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:19.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.413+0000 7f1717fff640 1 -- 172.21.15.67:0/2540071094 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f1724020030 con 0x7f1730107df0 2024-07-31T14:46:19.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.664+0000 7f1735525640 1 -- 172.21.15.67:0/2540071094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f173006b980 con 0x7f1730107df0 2024-07-31T14:46:19.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.664+0000 7f1717fff640 1 -- 172.21.15.67:0/2540071094 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f173006b980 con 0x7f1730107df0 2024-07-31T14:46:19.665 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:46:19.666 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":2,"btime":"2024-07-31T14:46:19:342455+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-07-31T14:46:19.666 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 2 2024-07-31T14:46:19.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.666+0000 7f1735525640 1 -- 172.21.15.67:0/2540071094 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f17180b3c50 msgr2=0x7f17180b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:19.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.666+0000 7f1735525640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f17180b3c50 0x7f17180b6160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f17200058c0 tx=0x7f1720005d50 comp rx=0 tx=0).stop 2024-07-31T14:46:19.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.666+0000 7f1735525640 1 -- 172.21.15.67:0/2540071094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1730107df0 msgr2=0x7f173019b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:19.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.666+0000 7f1735525640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1730107df0 0x7f173019b1d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1724005fd0 tx=0x7f172400dde0 comp rx=0 tx=0).stop 2024-07-31T14:46:19.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.666+0000 7f1735525640 1 -- 172.21.15.67:0/2540071094 shutdown_connections 2024-07-31T14:46:19.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.666+0000 7f1735525640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f17180b3c50 0x7f17180b6160 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:19.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.666+0000 7f1735525640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1730108740 0x7f173019b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:19.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.666+0000 7f1735525640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1730107df0 0x7f173019b1d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:19.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.666+0000 7f1735525640 1 --2- 172.21.15.67:0/2540071094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17301024b0 0x7f173019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:19.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.666+0000 7f1735525640 1 -- 172.21.15.67:0/2540071094 >> 172.21.15.67:0/2540071094 conn(0x7f1730079170 msgr2=0x7f1730103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:19.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.666+0000 7f1735525640 1 -- 172.21.15.67:0/2540071094 shutdown_connections 2024-07-31T14:46:19.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:19.666+0000 7f1735525640 1 -- 172.21.15.67:0/2540071094 wait complete. 2024-07-31T14:46:19.737 INFO:journalctl@ceph.mds.l.smithi077.stdout:Jul 31 14:46:19 smithi077 systemd[1]: Started Ceph mds.l for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:46:20.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:20 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.77:6832/2710994774,v1:172.21.15.77:6833/2710994774] up:boot 2024-07-31T14:46:20.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:20 smithi073 ceph-mon[36714]: fsmap 1 up:standby 2024-07-31T14:46:20.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-31T14:46:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:20 smithi073 ceph-mon[36714]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:20 smithi073 ceph-mon[36714]: Deploying daemon mds.k on smithi073 2024-07-31T14:46:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:20 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2540071094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:20.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:20 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.77:6832/2710994774,v1:172.21.15.77:6833/2710994774] up:boot 2024-07-31T14:46:20.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:20 smithi077 ceph-mon[37327]: fsmap 1 up:standby 2024-07-31T14:46:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-31T14:46:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:20 smithi077 ceph-mon[37327]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:20 smithi077 ceph-mon[37327]: Deploying daemon mds.k on smithi073 2024-07-31T14:46:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:20 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2540071094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:20.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:20 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.77:6832/2710994774,v1:172.21.15.77:6833/2710994774] up:boot 2024-07-31T14:46:20.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:20 smithi067 ceph-mon[32750]: fsmap 1 up:standby 2024-07-31T14:46:20.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-31T14:46:20.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:20 smithi067 ceph-mon[32750]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:20.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:20.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:20.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:20.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:20.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:20.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:20.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:20 smithi067 ceph-mon[32750]: Deploying daemon mds.k on smithi073 2024-07-31T14:46:20.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:20 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2540071094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:21.643 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 14:46:21 smithi073 systemd[1]: Starting Ceph mds.k for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T14:46:21.968 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 14:46:21 smithi073 podman[66951]: 2024-07-31 14:46:21.580256338 +0000 UTC m=+0.016661365 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T14:46:21.968 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 14:46:21 smithi073 podman[66951]: 2024-07-31 14:46:21.697702289 +0000 UTC m=+0.134107348 container create 60da7f5aff932b9745617a078643a427780eb967e27ca2b498356bd54ea3637e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-k, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-31T14:46:21.968 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 14:46:21 smithi073 podman[66951]: 2024-07-31 14:46:21.966991308 +0000 UTC m=+0.403396325 container init 60da7f5aff932b9745617a078643a427780eb967e27ca2b498356bd54ea3637e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-k, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-07-31T14:46:22.384 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:46:22.392 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 14:46:21 smithi073 podman[66951]: 2024-07-31 14:46:21.969906863 +0000 UTC m=+0.406311888 container start 60da7f5aff932b9745617a078643a427780eb967e27ca2b498356bd54ea3637e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0) 2024-07-31T14:46:22.392 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 14:46:22 smithi073 ceph-mds[66970]: -- 172.21.15.73:0/1770159415 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af1a001860 con 0x55af1e1a3a80 2024-07-31T14:46:22.392 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 14:46:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-k[66966]: starting mds.k at 2024-07-31T14:46:22.393 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 14:46:22 smithi073 bash[66951]: 60da7f5aff932b9745617a078643a427780eb967e27ca2b498356bd54ea3637e 2024-07-31T14:46:22.393 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 14:46:22 smithi073 systemd[1]: Started Ceph mds.k for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:46:22.689 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:22 smithi067 ceph-mon[32750]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:22.689 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:22.689 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:22.689 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:22.689 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:22.689 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:22.690 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:22 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:22.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:22 smithi077 ceph-mon[37327]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:22.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:22.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:22.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:22.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:22.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:22.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:22 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:22.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:22 smithi073 ceph-mon[36714]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:22.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:22.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:22.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:22.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:22.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:22.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:22 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:23.095 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:46:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:23 smithi067 ceph-mon[32750]: Deploying daemon mds.j on smithi067 2024-07-31T14:46:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:23 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.73:6832/1184537619,v1:172.21.15.73:6833/1184537619] up:boot 2024-07-31T14:46:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:23 smithi067 ceph-mon[32750]: fsmap 2 up:standby 2024-07-31T14:46:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-31T14:46:23.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:23 smithi077 ceph-mon[37327]: Deploying daemon mds.j on smithi067 2024-07-31T14:46:23.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:23 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.73:6832/1184537619,v1:172.21.15.73:6833/1184537619] up:boot 2024-07-31T14:46:23.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:23 smithi077 ceph-mon[37327]: fsmap 2 up:standby 2024-07-31T14:46:23.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-31T14:46:23.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:23 smithi073 ceph-mon[36714]: Deploying daemon mds.j on smithi067 2024-07-31T14:46:23.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:23 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.73:6832/1184537619,v1:172.21.15.73:6833/1184537619] up:boot 2024-07-31T14:46:23.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:23 smithi073 ceph-mon[36714]: fsmap 2 up:standby 2024-07-31T14:46:23.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-31T14:46:24.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:24 smithi067 ceph-mon[32750]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:24.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:24 smithi073 ceph-mon[36714]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:24.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:24 smithi077 ceph-mon[37327]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:25.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.434+0000 7f4b06251640 1 -- 172.21.15.67:0/1671503280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b00107dd0 msgr2=0x7f4b001081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:25.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.434+0000 7f4b06251640 1 --2- 172.21.15.67:0/1671503280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b00107dd0 0x7f4b001081e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009900 tx=0x7f4af402a410 comp rx=0 tx=0).stop 2024-07-31T14:46:25.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.434+0000 7f4b06251640 1 -- 172.21.15.67:0/1671503280 shutdown_connections 2024-07-31T14:46:25.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.434+0000 7f4b06251640 1 --2- 172.21.15.67:0/1671503280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b00108720 0x7f4b0010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:25.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.434+0000 7f4b06251640 1 --2- 172.21.15.67:0/1671503280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b00107dd0 0x7f4b001081e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:25.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.434+0000 7f4b06251640 1 --2- 172.21.15.67:0/1671503280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b00102490 0x7f4b001028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:25.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.434+0000 7f4b06251640 1 -- 172.21.15.67:0/1671503280 >> 172.21.15.67:0/1671503280 conn(0x7f4b00079160 msgr2=0x7f4b00079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:25.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.434+0000 7f4b06251640 1 -- 172.21.15.67:0/1671503280 shutdown_connections 2024-07-31T14:46:25.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.434+0000 7f4b06251640 1 -- 172.21.15.67:0/1671503280 wait complete. 2024-07-31T14:46:25.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.435+0000 7f4b06251640 1 Processor -- start 2024-07-31T14:46:25.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.435+0000 7f4b06251640 1 -- start start 2024-07-31T14:46:25.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.435+0000 7f4b06251640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b00102490 0x7f4b00198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:25.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.435+0000 7f4b06251640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b00107dd0 0x7f4b00199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:25.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.435+0000 7f4b06251640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b00108720 0x7f4b0019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:25.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.435+0000 7f4b06251640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4b00112350 con 0x7f4b00108720 2024-07-31T14:46:25.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.435+0000 7f4b06251640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001121d0 con 0x7f4b00107dd0 2024-07-31T14:46:25.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.435+0000 7f4b06251640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001124d0 con 0x7f4b00102490 2024-07-31T14:46:25.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.435+0000 7f4afffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b00108720 0x7f4b0019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.435+0000 7f4afffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b00108720 0x7f4b0019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53564/0 (socket says 172.21.15.67:53564) 2024-07-31T14:46:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.435+0000 7f4afffff640 1 -- 172.21.15.67:0/2903909511 learned_addr learned my addr 172.21.15.67:0/2903909511 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:46:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.436+0000 7f4afeffd640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b00107dd0 0x7f4b00199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.436+0000 7f4aff7fe640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b00102490 0x7f4b00198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.436+0000 7f4afffff640 1 -- 172.21.15.67:0/2903909511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b00102490 msgr2=0x7f4b00198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.436+0000 7f4afffff640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b00102490 0x7f4b00198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.436+0000 7f4afffff640 1 -- 172.21.15.67:0/2903909511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b00107dd0 msgr2=0x7f4b00199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.436+0000 7f4afffff640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b00107dd0 0x7f4b00199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.436+0000 7f4afffff640 1 -- 172.21.15.67:0/2903909511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0019ec30 con 0x7f4b00108720 2024-07-31T14:46:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.436+0000 7f4aff7fe640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b00102490 0x7f4b00198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:46:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.436+0000 7f4afffff640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b00108720 0x7f4b0019e5c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4af800ebb0 tx=0x7f4af8007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.436+0000 7f4afcff9640 1 -- 172.21.15.67:0/2903909511 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8007ca0 con 0x7f4b00108720 2024-07-31T14:46:25.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.436+0000 7f4afcff9640 1 -- 172.21.15.67:0/2903909511 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4af8007e40 con 0x7f4b00108720 2024-07-31T14:46:25.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.436+0000 7f4afcff9640 1 -- 172.21.15.67:0/2903909511 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af80057d0 con 0x7f4b00108720 2024-07-31T14:46:25.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.436+0000 7f4b06251640 1 -- 172.21.15.67:0/2903909511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0019eec0 con 0x7f4b00108720 2024-07-31T14:46:25.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.436+0000 7f4b06251640 1 -- 172.21.15.67:0/2903909511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0019f3d0 con 0x7f4b00108720 2024-07-31T14:46:25.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.437+0000 7f4b06251640 1 -- 172.21.15.67:0/2903909511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b001028a0 con 0x7f4b00108720 2024-07-31T14:46:25.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.440+0000 7f4afcff9640 1 -- 172.21.15.67:0/2903909511 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f4af8004a40 con 0x7f4b00108720 2024-07-31T14:46:25.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.440+0000 7f4afcff9640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4ae80b3cd0 0x7f4ae80b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:25.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.441+0000 7f4aff7fe640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4ae80b3cd0 0x7f4ae80b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:25.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.441+0000 7f4afcff9640 1 -- 172.21.15.67:0/2903909511 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7342+0+0 (secure 0 0 0) 0x7f4af80e51e0 con 0x7f4b00108720 2024-07-31T14:46:25.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.441+0000 7f4aff7fe640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4ae80b3cd0 0x7f4ae80b61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4af000abe0 tx=0x7f4af000a5c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:25.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.444+0000 7f4afcff9640 1 -- 172.21.15.67:0/2903909511 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f4af80acc00 con 0x7f4b00108720 2024-07-31T14:46:25.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.680+0000 7f4b06251640 1 -- 172.21.15.67:0/2903909511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b0010a090 con 0x7f4b00108720 2024-07-31T14:46:25.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.680+0000 7f4afcff9640 1 -- 172.21.15.67:0/2903909511 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2260 (secure 0 0 0) 0x7f4af800e0d0 con 0x7f4b00108720 2024-07-31T14:46:25.682 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:46:25.682 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":3,"btime":"2024-07-31T14:46:22:411340+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-07-31T14:46:25.682 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 3 2024-07-31T14:46:25.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.683+0000 7f4b06251640 1 -- 172.21.15.67:0/2903909511 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4ae80b3cd0 msgr2=0x7f4ae80b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:25.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.683+0000 7f4b06251640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4ae80b3cd0 0x7f4ae80b61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4af000abe0 tx=0x7f4af000a5c0 comp rx=0 tx=0).stop 2024-07-31T14:46:25.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.683+0000 7f4b06251640 1 -- 172.21.15.67:0/2903909511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b00108720 msgr2=0x7f4b0019e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:25.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.683+0000 7f4b06251640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b00108720 0x7f4b0019e5c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4af800ebb0 tx=0x7f4af8007590 comp rx=0 tx=0).stop 2024-07-31T14:46:25.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.683+0000 7f4b06251640 1 -- 172.21.15.67:0/2903909511 shutdown_connections 2024-07-31T14:46:25.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.683+0000 7f4b06251640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4ae80b3cd0 0x7f4ae80b61e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:25.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.683+0000 7f4b06251640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b00108720 0x7f4b0019e5c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:25.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.683+0000 7f4b06251640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b00107dd0 0x7f4b00199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:25.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.683+0000 7f4b06251640 1 --2- 172.21.15.67:0/2903909511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b00102490 0x7f4b00198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:25.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.683+0000 7f4b06251640 1 -- 172.21.15.67:0/2903909511 >> 172.21.15.67:0/2903909511 conn(0x7f4b00079160 msgr2=0x7f4b00104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:25.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.683+0000 7f4b06251640 1 -- 172.21.15.67:0/2903909511 shutdown_connections 2024-07-31T14:46:25.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:25.683+0000 7f4b06251640 1 -- 172.21.15.67:0/2903909511 wait complete. 2024-07-31T14:46:26.706 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:26 smithi067 ceph-mon[32750]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:26.706 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:26 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2903909511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:26.706 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:26.707 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:26.707 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:26.707 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:26.707 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:26.707 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:26.707 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:26 smithi067 ceph-mon[32750]: Deploying daemon mds.d on smithi067 2024-07-31T14:46:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:26 smithi077 ceph-mon[37327]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:26 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2903909511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:26 smithi077 ceph-mon[37327]: Deploying daemon mds.d on smithi067 2024-07-31T14:46:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:26 smithi073 ceph-mon[36714]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:26 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2903909511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:26 smithi073 ceph-mon[36714]: Deploying daemon mds.d on smithi067 2024-07-31T14:46:27.709 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:27 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.67:6834/952957104,v1:172.21.15.67:6835/952957104] up:boot 2024-07-31T14:46:27.709 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:27 smithi067 ceph-mon[32750]: fsmap 3 up:standby 2024-07-31T14:46:27.709 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-31T14:46:27.736 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:27 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.67:6834/952957104,v1:172.21.15.67:6835/952957104] up:boot 2024-07-31T14:46:27.736 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:27 smithi077 ceph-mon[37327]: fsmap 3 up:standby 2024-07-31T14:46:27.736 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-31T14:46:27.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:27 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.67:6834/952957104,v1:172.21.15.67:6835/952957104] up:boot 2024-07-31T14:46:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:27 smithi073 ceph-mon[36714]: fsmap 3 up:standby 2024-07-31T14:46:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-31T14:46:28.443 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 31 14:46:28 smithi067 podman[79512]: 2024-07-31 14:46:28.248486103 +0000 UTC m=+0.017823459 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T14:46:28.548 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:46:28.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:28 smithi077 ceph-mon[37327]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:28.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:28 smithi067 ceph-mon[32750]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:28.803 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 31 14:46:28 smithi067 podman[79512]: 2024-07-31 14:46:28.552173982 +0000 UTC m=+0.321511336 container init 7062ce67ed1463ad1dd342c447dab5d70a993da00a300e535e486f10c111c649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-d, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-07-31T14:46:28.803 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 31 14:46:28 smithi067 podman[79512]: 2024-07-31 14:46:28.555089765 +0000 UTC m=+0.324427101 container start 7062ce67ed1463ad1dd342c447dab5d70a993da00a300e535e486f10c111c649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc) 2024-07-31T14:46:28.803 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 31 14:46:28 smithi067 ceph-mds[79536]: -- 172.21.15.67:0/4292003513 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5627e3e75860 con 0x5627e8017a80 2024-07-31T14:46:28.804 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 31 14:46:28 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-d[79527]: starting mds.d at 2024-07-31T14:46:28.804 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 31 14:46:28 smithi067 bash[79512]: 7062ce67ed1463ad1dd342c447dab5d70a993da00a300e535e486f10c111c649 2024-07-31T14:46:28.804 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 31 14:46:28 smithi067 systemd[1]: Started Ceph mds.d for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:46:28.829 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:46:28.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:28 smithi073 ceph-mon[36714]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:29.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:29 smithi073 ceph-mon[36714]: Deploying daemon mds.e on smithi073 2024-07-31T14:46:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:29 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.67:6836/505336439,v1:172.21.15.67:6837/505336439] up:boot 2024-07-31T14:46:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:29 smithi073 ceph-mon[36714]: fsmap 4 up:standby 2024-07-31T14:46:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:29 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-31T14:46:29.949 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:29.949 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:29.949 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:29.949 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:29.949 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:29.949 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:29.949 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:29 smithi067 ceph-mon[32750]: Deploying daemon mds.e on smithi073 2024-07-31T14:46:29.949 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:29 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.67:6836/505336439,v1:172.21.15.67:6837/505336439] up:boot 2024-07-31T14:46:29.949 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:29 smithi067 ceph-mon[32750]: fsmap 4 up:standby 2024-07-31T14:46:29.949 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:29 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-31T14:46:29.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:29.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:29.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:29 smithi077 ceph-mon[37327]: Deploying daemon mds.e on smithi073 2024-07-31T14:46:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:29 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.67:6836/505336439,v1:172.21.15.67:6837/505336439] up:boot 2024-07-31T14:46:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:29 smithi077 ceph-mon[37327]: fsmap 4 up:standby 2024-07-31T14:46:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:29 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-31T14:46:30.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.559+0000 7f3fe4810640 1 -- 172.21.15.67:0/4287875972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fe0108ed0 msgr2=0x7f3fe01092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:30.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.559+0000 7f3fe4810640 1 --2- 172.21.15.67:0/4287875972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fe0108ed0 0x7f3fe01092e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009900 tx=0x7f3fd402a410 comp rx=0 tx=0).stop 2024-07-31T14:46:30.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.559+0000 7f3fe4810640 1 -- 172.21.15.67:0/4287875972 shutdown_connections 2024-07-31T14:46:30.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.559+0000 7f3fe4810640 1 --2- 172.21.15.67:0/4287875972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fe0109820 0x7f3fe0110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:30.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.559+0000 7f3fe4810640 1 --2- 172.21.15.67:0/4287875972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fe0108ed0 0x7f3fe01092e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:30.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.559+0000 7f3fe4810640 1 --2- 172.21.15.67:0/4287875972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fe0103590 0x7f3fe01039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:30.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.559+0000 7f3fe4810640 1 -- 172.21.15.67:0/4287875972 >> 172.21.15.67:0/4287875972 conn(0x7f3fe007b280 msgr2=0x7f3fe007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:30.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.559+0000 7f3fe4810640 1 -- 172.21.15.67:0/4287875972 shutdown_connections 2024-07-31T14:46:30.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.559+0000 7f3fe4810640 1 -- 172.21.15.67:0/4287875972 wait complete. 2024-07-31T14:46:30.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.560+0000 7f3fe4810640 1 Processor -- start 2024-07-31T14:46:30.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.560+0000 7f3fe4810640 1 -- start start 2024-07-31T14:46:30.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.560+0000 7f3fe4810640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fe0103590 0x7f3fe019c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.560+0000 7f3fe4810640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fe0108ed0 0x7f3fe019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.560+0000 7f3fe4810640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fe0109820 0x7f3fe01a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fe4810640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0113490 con 0x7f3fe0108ed0 2024-07-31T14:46:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fe4810640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0113310 con 0x7f3fe0109820 2024-07-31T14:46:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fe4810640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0113610 con 0x7f3fe0103590 2024-07-31T14:46:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fddd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fe0108ed0 0x7f3fe019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fde59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fe0103590 0x7f3fe019c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fded9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fe0109820 0x7f3fe01a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fde59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fe0103590 0x7f3fe019c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:40952/0 (socket says 172.21.15.67:40952) 2024-07-31T14:46:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fddd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fe0108ed0 0x7f3fe019c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53618/0 (socket says 172.21.15.67:53618) 2024-07-31T14:46:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fde59c640 1 -- 172.21.15.67:0/45787452 learned_addr learned my addr 172.21.15.67:0/45787452 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:46:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fddd9b640 1 -- 172.21.15.67:0/45787452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fe0103590 msgr2=0x7f3fe019c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fddd9b640 1 --2- 172.21.15.67:0/45787452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fe0103590 0x7f3fe019c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:30.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fddd9b640 1 -- 172.21.15.67:0/45787452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fe0109820 msgr2=0x7f3fe01a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:30.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fddd9b640 1 --2- 172.21.15.67:0/45787452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fe0109820 0x7f3fe01a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:30.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fddd9b640 1 -- 172.21.15.67:0/45787452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe01a1f90 con 0x7f3fe0108ed0 2024-07-31T14:46:30.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fddd9b640 1 --2- 172.21.15.67:0/45787452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fe0108ed0 0x7f3fe019c680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3fd402a930 tx=0x7f3fd4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:30.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fcf7fe640 1 -- 172.21.15.67:0/45787452 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd403f070 con 0x7f3fe0108ed0 2024-07-31T14:46:30.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fcf7fe640 1 -- 172.21.15.67:0/45787452 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd4003ca0 con 0x7f3fe0108ed0 2024-07-31T14:46:30.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fe4810640 1 -- 172.21.15.67:0/45787452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe01a2220 con 0x7f3fe0108ed0 2024-07-31T14:46:30.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.561+0000 7f3fcf7fe640 1 -- 172.21.15.67:0/45787452 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4035070 con 0x7f3fe0108ed0 2024-07-31T14:46:30.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.562+0000 7f3fe4810640 1 -- 172.21.15.67:0/45787452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe01a26d0 con 0x7f3fe0108ed0 2024-07-31T14:46:30.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.563+0000 7f3fe4810640 1 -- 172.21.15.67:0/45787452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe01039a0 con 0x7f3fe0108ed0 2024-07-31T14:46:30.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.564+0000 7f3fcf7fe640 1 -- 172.21.15.67:0/45787452 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f3fd403b460 con 0x7f3fe0108ed0 2024-07-31T14:46:30.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.565+0000 7f3fcf7fe640 1 --2- 172.21.15.67:0/45787452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3fc80b3c50 0x7f3fc80b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:30.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.565+0000 7f3fde59c640 1 --2- 172.21.15.67:0/45787452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3fc80b3c50 0x7f3fc80b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:30.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.565+0000 7f3fcf7fe640 1 -- 172.21.15.67:0/45787452 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7342+0+0 (secure 0 0 0) 0x7f3fd4103ea0 con 0x7f3fe0108ed0 2024-07-31T14:46:30.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.566+0000 7f3fde59c640 1 --2- 172.21.15.67:0/45787452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3fc80b3c50 0x7f3fc80b6160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009c90 tx=0x7f3fd0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:30.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.568+0000 7f3fcf7fe640 1 -- 172.21.15.67:0/45787452 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f3fd40cb270 con 0x7f3fe0108ed0 2024-07-31T14:46:30.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.800+0000 7f3fe4810640 1 -- 172.21.15.67:0/45787452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fe01a8b90 con 0x7f3fe0108ed0 2024-07-31T14:46:30.802 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:30 smithi067 ceph-mon[32750]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:30.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.801+0000 7f3fcf7fe640 1 -- 172.21.15.67:0/45787452 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3904 (secure 0 0 0) 0x7f3fd40cf7c0 con 0x7f3fe0108ed0 2024-07-31T14:46:30.803 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:46:30.803 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"btime":"2024-07-31T14:46:29:442202+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-07-31T14:46:30.803 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 5 2024-07-31T14:46:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.803+0000 7f3fe4810640 1 -- 172.21.15.67:0/45787452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3fc80b3c50 msgr2=0x7f3fc80b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.803+0000 7f3fe4810640 1 --2- 172.21.15.67:0/45787452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3fc80b3c50 0x7f3fc80b6160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009c90 tx=0x7f3fd0005f90 comp rx=0 tx=0).stop 2024-07-31T14:46:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.803+0000 7f3fe4810640 1 -- 172.21.15.67:0/45787452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fe0108ed0 msgr2=0x7f3fe019c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.803+0000 7f3fe4810640 1 --2- 172.21.15.67:0/45787452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fe0108ed0 0x7f3fe019c680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3fd402a930 tx=0x7f3fd4004060 comp rx=0 tx=0).stop 2024-07-31T14:46:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.803+0000 7f3fe4810640 1 -- 172.21.15.67:0/45787452 shutdown_connections 2024-07-31T14:46:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.803+0000 7f3fe4810640 1 --2- 172.21.15.67:0/45787452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3fc80b3c50 0x7f3fc80b6160 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.803+0000 7f3fe4810640 1 --2- 172.21.15.67:0/45787452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fe0109820 0x7f3fe01a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.803+0000 7f3fe4810640 1 --2- 172.21.15.67:0/45787452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fe0108ed0 0x7f3fe019c680 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.803+0000 7f3fe4810640 1 --2- 172.21.15.67:0/45787452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fe0103590 0x7f3fe019c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.803+0000 7f3fe4810640 1 -- 172.21.15.67:0/45787452 >> 172.21.15.67:0/45787452 conn(0x7f3fe007b280 msgr2=0x7f3fe0106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.803+0000 7f3fe4810640 1 -- 172.21.15.67:0/45787452 shutdown_connections 2024-07-31T14:46:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:30.803+0000 7f3fe4810640 1 -- 172.21.15.67:0/45787452 wait complete. 2024-07-31T14:46:30.891 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:30 smithi073 ceph-mon[36714]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:30.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:30 smithi077 ceph-mon[37327]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:31.142 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:46:31 smithi073 podman[67267]: 2024-07-31 14:46:31.126188503 +0000 UTC m=+0.350978225 container start c20528a1cd963bd5ef16627e4053b8850db3b3d57722ec79b687a663de722539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-07-31T14:46:31.642 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:46:31 smithi073 ceph-mds[67286]: -- 172.21.15.73:0/701539933 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e152939860 con 0x55e156adb600 2024-07-31T14:46:31.643 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:46:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: starting mds.e at 2024-07-31T14:46:31.643 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:46:31 smithi073 bash[67267]: c20528a1cd963bd5ef16627e4053b8850db3b3d57722ec79b687a663de722539 2024-07-31T14:46:31.643 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:46:31 smithi073 systemd[1]: Started Ceph mds.e for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:46:31.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:31 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/45787452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:31 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:31 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:31 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:31 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:31 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:31 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:32.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:31 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/45787452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:32.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:31 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:32.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:31 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:32.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:31 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:32.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:31 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:32.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:31 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:32.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:31 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:32.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:31 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/45787452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:32.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:31 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:32.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:31 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:32.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:31 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:32.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:31 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:32.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:31 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:32.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:31 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:32.956 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:32 smithi077 ceph-mon[37327]: Deploying daemon mds.f on smithi077 2024-07-31T14:46:32.956 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:32 smithi077 ceph-mon[37327]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:32.956 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:32 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.73:6834/2854264361,v1:172.21.15.73:6835/2854264361] up:boot 2024-07-31T14:46:32.956 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:32 smithi077 ceph-mon[37327]: fsmap 5 up:standby 2024-07-31T14:46:32.956 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:32 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T14:46:33.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:32 smithi067 ceph-mon[32750]: Deploying daemon mds.f on smithi077 2024-07-31T14:46:33.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:32 smithi067 ceph-mon[32750]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:33.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:32 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.73:6834/2854264361,v1:172.21.15.73:6835/2854264361] up:boot 2024-07-31T14:46:33.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:32 smithi067 ceph-mon[32750]: fsmap 5 up:standby 2024-07-31T14:46:33.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:32 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T14:46:33.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:32 smithi073 ceph-mon[36714]: Deploying daemon mds.f on smithi077 2024-07-31T14:46:33.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:32 smithi073 ceph-mon[36714]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:33.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:32 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.73:6834/2854264361,v1:172.21.15.73:6835/2854264361] up:boot 2024-07-31T14:46:33.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:32 smithi073 ceph-mon[36714]: fsmap 5 up:standby 2024-07-31T14:46:33.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:32 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T14:46:33.237 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 14:46:33 smithi077 systemd[1]: Starting Ceph mds.f for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T14:46:33.479 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:46:33.624 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 14:46:33 smithi077 podman[69283]: 2024-07-31 14:46:33.262921928 +0000 UTC m=+0.016902296 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T14:46:33.624 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 14:46:33 smithi077 podman[69283]: 2024-07-31 14:46:33.379134927 +0000 UTC m=+0.133115296 container create 9519cd747ec15aafa13d69f8751dde29794255449f38c16fcbee8ed50b395ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-07-31T14:46:33.624 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 14:46:33 smithi077 podman[69283]: 2024-07-31 14:46:33.623524169 +0000 UTC m=+0.377504537 container init 9519cd747ec15aafa13d69f8751dde29794255449f38c16fcbee8ed50b395ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-f, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-07-31T14:46:33.757 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:46:33.987 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 14:46:33 smithi077 podman[69283]: 2024-07-31 14:46:33.626204731 +0000 UTC m=+0.380185099 container start 9519cd747ec15aafa13d69f8751dde29794255449f38c16fcbee8ed50b395ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-f, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T14:46:33.987 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 14:46:33 smithi077 ceph-mds[69302]: -- 172.21.15.77:0/3936214310 <== mon.1 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56507f073860 con 0x565083215a80 2024-07-31T14:46:33.988 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 14:46:33 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-f[69298]: starting mds.f at 2024-07-31T14:46:33.988 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 14:46:33 smithi077 bash[69283]: 9519cd747ec15aafa13d69f8751dde29794255449f38c16fcbee8ed50b395ad7 2024-07-31T14:46:33.988 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 14:46:33 smithi077 systemd[1]: Started Ceph mds.f for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:46:34.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:34 smithi077 ceph-mon[37327]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:34.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:34 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] up:boot 2024-07-31T14:46:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:34 smithi077 ceph-mon[37327]: fsmap 6 up:standby 2024-07-31T14:46:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-31T14:46:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:34 smithi077 ceph-mon[37327]: Deploying daemon mds.c on smithi077 2024-07-31T14:46:35.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:34 smithi067 ceph-mon[32750]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:35.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:34 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] up:boot 2024-07-31T14:46:35.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:34 smithi067 ceph-mon[32750]: fsmap 6 up:standby 2024-07-31T14:46:35.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-31T14:46:35.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:35.032 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:35.032 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:35.032 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:35.032 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:35.032 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:35.032 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:34 smithi067 ceph-mon[32750]: Deploying daemon mds.c on smithi077 2024-07-31T14:46:35.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:34 smithi073 ceph-mon[36714]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:34 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] up:boot 2024-07-31T14:46:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:34 smithi073 ceph-mon[36714]: fsmap 6 up:standby 2024-07-31T14:46:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-31T14:46:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:34 smithi073 ceph-mon[36714]: Deploying daemon mds.c on smithi077 2024-07-31T14:46:35.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.552+0000 7f7630032640 1 -- 172.21.15.67:0/3477275543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7628103590 msgr2=0x7f76281039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:35.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.552+0000 7f7630032640 1 --2- 172.21.15.67:0/3477275543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7628103590 0x7f76281039a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7624009900 tx=0x7f762402a410 comp rx=0 tx=0).stop 2024-07-31T14:46:35.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.552+0000 7f7630032640 1 -- 172.21.15.67:0/3477275543 shutdown_connections 2024-07-31T14:46:35.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.552+0000 7f7630032640 1 --2- 172.21.15.67:0/3477275543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7628109820 0x7f7628110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:35.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.552+0000 7f7630032640 1 --2- 172.21.15.67:0/3477275543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628108ed0 0x7f76281092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:35.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.552+0000 7f7630032640 1 --2- 172.21.15.67:0/3477275543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7628103590 0x7f76281039a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:35.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.552+0000 7f7630032640 1 -- 172.21.15.67:0/3477275543 >> 172.21.15.67:0/3477275543 conn(0x7f762807b280 msgr2=0x7f762807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:35.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.552+0000 7f7630032640 1 -- 172.21.15.67:0/3477275543 shutdown_connections 2024-07-31T14:46:35.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.552+0000 7f7630032640 1 -- 172.21.15.67:0/3477275543 wait complete. 2024-07-31T14:46:35.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.553+0000 7f7630032640 1 Processor -- start 2024-07-31T14:46:35.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.553+0000 7f7630032640 1 -- start start 2024-07-31T14:46:35.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.553+0000 7f7630032640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7628103590 0x7f7628199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:35.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.553+0000 7f7630032640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628108ed0 0x7f762819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:35.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.553+0000 7f7630032640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7628109820 0x7f762819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:35.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.553+0000 7f7630032640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f76281134d0 con 0x7f7628103590 2024-07-31T14:46:35.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.553+0000 7f7630032640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7628113350 con 0x7f7628108ed0 2024-07-31T14:46:35.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.553+0000 7f7630032640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7628113650 con 0x7f7628109820 2024-07-31T14:46:35.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.553+0000 7f762ddce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7628103590 0x7f7628199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:35.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.553+0000 7f762ddce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7628103590 0x7f7628199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60932/0 (socket says 172.21.15.67:60932) 2024-07-31T14:46:35.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.553+0000 7f762ddce640 1 -- 172.21.15.67:0/3688873485 learned_addr learned my addr 172.21.15.67:0/3688873485 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:46:35.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.553+0000 7f762e5cf640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7628109820 0x7f762819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:35.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.553+0000 7f762d5cd640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628108ed0 0x7f762819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:35.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.554+0000 7f762ddce640 1 -- 172.21.15.67:0/3688873485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7628109820 msgr2=0x7f762819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:35.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.554+0000 7f762ddce640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7628109820 0x7f762819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:35.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.554+0000 7f762ddce640 1 -- 172.21.15.67:0/3688873485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628108ed0 msgr2=0x7f762819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:35.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.554+0000 7f762ddce640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628108ed0 0x7f762819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:35.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.554+0000 7f762ddce640 1 -- 172.21.15.67:0/3688873485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762819fe50 con 0x7f7628103590 2024-07-31T14:46:35.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.554+0000 7f762e5cf640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7628109820 0x7f762819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:46:35.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.554+0000 7f762d5cd640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628108ed0 0x7f762819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:46:35.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.554+0000 7f762ddce640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7628103590 0x7f7628199ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7624005940 tx=0x7f7624004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:35.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.554+0000 7f761effd640 1 -- 172.21.15.67:0/3688873485 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762403f030 con 0x7f7628103590 2024-07-31T14:46:35.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.554+0000 7f761effd640 1 -- 172.21.15.67:0/3688873485 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7624003aa0 con 0x7f7628103590 2024-07-31T14:46:35.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.554+0000 7f7630032640 1 -- 172.21.15.67:0/3688873485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76281a00e0 con 0x7f7628103590 2024-07-31T14:46:35.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.554+0000 7f761effd640 1 -- 172.21.15.67:0/3688873485 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624035070 con 0x7f7628103590 2024-07-31T14:46:35.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.554+0000 7f7630032640 1 -- 172.21.15.67:0/3688873485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76281a68a0 con 0x7f7628103590 2024-07-31T14:46:35.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.555+0000 7f7630032640 1 -- 172.21.15.67:0/3688873485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76281039a0 con 0x7f7628103590 2024-07-31T14:46:35.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.557+0000 7f761effd640 1 -- 172.21.15.67:0/3688873485 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f7624042030 con 0x7f7628103590 2024-07-31T14:46:35.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.557+0000 7f761effd640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f76140b3c20 0x7f76140b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:35.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.558+0000 7f762d5cd640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f76140b3c20 0x7f76140b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:35.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.558+0000 7f761effd640 1 -- 172.21.15.67:0/3688873485 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7342+0+0 (secure 0 0 0) 0x7f7624104130 con 0x7f7628103590 2024-07-31T14:46:35.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.558+0000 7f762d5cd640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f76140b3c20 0x7f76140b6130 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7618005d00 tx=0x7f7618005c70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:35.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.561+0000 7f761effd640 1 -- 172.21.15.67:0/3688873485 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f76240cbb50 con 0x7f7628103590 2024-07-31T14:46:35.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.797+0000 7f7630032640 1 -- 172.21.15.67:0/3688873485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f762819b010 con 0x7f7628103590 2024-07-31T14:46:35.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.798+0000 7f761effd640 1 -- 172.21.15.67:0/3688873485 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5554 (secure 0 0 0) 0x7f76240cb450 con 0x7f7628103590 2024-07-31T14:46:35.799 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:46:35.799 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":7,"btime":"2024-07-31T14:46:33:725992+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24634,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-07-31T14:46:35.800 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 7 2024-07-31T14:46:35.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.800+0000 7f7630032640 1 -- 172.21.15.67:0/3688873485 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f76140b3c20 msgr2=0x7f76140b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:35.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.800+0000 7f7630032640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f76140b3c20 0x7f76140b6130 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7618005d00 tx=0x7f7618005c70 comp rx=0 tx=0).stop 2024-07-31T14:46:35.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.800+0000 7f7630032640 1 -- 172.21.15.67:0/3688873485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7628103590 msgr2=0x7f7628199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:35.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.800+0000 7f7630032640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7628103590 0x7f7628199ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7624005940 tx=0x7f7624004270 comp rx=0 tx=0).stop 2024-07-31T14:46:35.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.800+0000 7f7630032640 1 -- 172.21.15.67:0/3688873485 shutdown_connections 2024-07-31T14:46:35.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.800+0000 7f7630032640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f76140b3c20 0x7f76140b6130 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:35.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.800+0000 7f7630032640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7628109820 0x7f762819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:35.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.800+0000 7f7630032640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628108ed0 0x7f762819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:35.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.800+0000 7f7630032640 1 --2- 172.21.15.67:0/3688873485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7628103590 0x7f7628199ee0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:35.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.800+0000 7f7630032640 1 -- 172.21.15.67:0/3688873485 >> 172.21.15.67:0/3688873485 conn(0x7f762807b280 msgr2=0x7f7628106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:35.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.800+0000 7f7630032640 1 -- 172.21.15.67:0/3688873485 shutdown_connections 2024-07-31T14:46:35.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:35.800+0000 7f7630032640 1 -- 172.21.15.67:0/3688873485 wait complete. 2024-07-31T14:46:35.914 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 14:46:35 smithi077 systemd[1]: Starting Ceph mds.c for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T14:46:36.212 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 14:46:35 smithi077 podman[69586]: 2024-07-31 14:46:35.815506951 +0000 UTC m=+0.016913894 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T14:46:36.212 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 14:46:35 smithi077 podman[69586]: 2024-07-31 14:46:35.942176711 +0000 UTC m=+0.143583635 container create 71d4c7cc57d35f3086f37369d072b051c5b6f87e0c6a524a24938fb4c8356439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-c, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T14:46:36.212 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 14:46:36 smithi077 podman[69586]: 2024-07-31 14:46:36.211693001 +0000 UTC m=+0.413099924 container init 71d4c7cc57d35f3086f37369d072b051c5b6f87e0c6a524a24938fb4c8356439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716) 2024-07-31T14:46:36.487 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 14:46:36 smithi077 podman[69586]: 2024-07-31 14:46:36.214195891 +0000 UTC m=+0.415602814 container start 71d4c7cc57d35f3086f37369d072b051c5b6f87e0c6a524a24938fb4c8356439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-c, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-07-31T14:46:36.487 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 14:46:36 smithi077 ceph-mds[69606]: -- 172.21.15.77:0/3572422119 <== mon.1 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f0e73d860 con 0x561f128dfa80 2024-07-31T14:46:36.487 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 14:46:36 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-c[69602]: starting mds.c at 2024-07-31T14:46:36.488 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 14:46:36 smithi077 bash[69586]: 71d4c7cc57d35f3086f37369d072b051c5b6f87e0c6a524a24938fb4c8356439 2024-07-31T14:46:36.488 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 14:46:36 smithi077 systemd[1]: Started Ceph mds.c for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:46:36.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:36 smithi067 ceph-mon[32750]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:36.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:36 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3688873485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:36.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:36 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:36.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:36 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:36.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:36 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:36.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:36 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:36.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:36 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:36.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:36 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:37.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:36 smithi073 ceph-mon[36714]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:37.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:36 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3688873485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:37.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:36 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:37.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:36 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:37.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:36 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:37.064 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:36 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:37.064 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:36 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:37.064 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:36 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:36 smithi077 ceph-mon[37327]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:36 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3688873485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:36 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:36 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:36 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:36 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:36 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:36 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:38.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:37 smithi067 ceph-mon[32750]: Deploying daemon mds.h on smithi073 2024-07-31T14:46:38.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:37 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] up:boot 2024-07-31T14:46:38.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:37 smithi067 ceph-mon[32750]: fsmap 7 up:standby 2024-07-31T14:46:38.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:37 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-31T14:46:38.129 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:37 smithi073 ceph-mon[36714]: Deploying daemon mds.h on smithi073 2024-07-31T14:46:38.129 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:37 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] up:boot 2024-07-31T14:46:38.130 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:37 smithi073 ceph-mon[36714]: fsmap 7 up:standby 2024-07-31T14:46:38.130 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:37 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-31T14:46:38.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:37 smithi077 ceph-mon[37327]: Deploying daemon mds.h on smithi073 2024-07-31T14:46:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:37 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] up:boot 2024-07-31T14:46:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:37 smithi077 ceph-mon[37327]: fsmap 7 up:standby 2024-07-31T14:46:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:37 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-31T14:46:38.392 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 14:46:38 smithi073 systemd[1]: Starting Ceph mds.h for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T14:46:38.562 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:46:38.784 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:38 smithi073 ceph-mon[36714]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:38.784 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 14:46:38 smithi073 podman[67569]: 2024-07-31 14:46:38.416625801 +0000 UTC m=+0.017448415 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T14:46:38.785 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 14:46:38 smithi073 podman[67569]: 2024-07-31 14:46:38.546756087 +0000 UTC m=+0.147578747 container create 461ee206e533d2666bca08ded406b1d6f2f0e926b441b7137bca8eb6db47b68e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-07-31T14:46:38.785 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 14:46:38 smithi073 podman[67569]: 2024-07-31 14:46:38.749501475 +0000 UTC m=+0.350324089 container init 461ee206e533d2666bca08ded406b1d6f2f0e926b441b7137bca8eb6db47b68e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-07-31T14:46:38.785 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 14:46:38 smithi073 podman[67569]: 2024-07-31 14:46:38.752188369 +0000 UTC m=+0.353010976 container start 461ee206e533d2666bca08ded406b1d6f2f0e926b441b7137bca8eb6db47b68e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-h, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-07-31T14:46:38.785 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 14:46:38 smithi073 ceph-mds[67589]: -- 172.21.15.73:0/3914067458 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562923bc1860 con 0x562927d63600 2024-07-31T14:46:38.839 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:46:38.961 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:38 smithi067 ceph-mon[32750]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:39.143 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 14:46:38 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-h[67585]: starting mds.h at 2024-07-31T14:46:39.143 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 14:46:38 smithi073 bash[67569]: 461ee206e533d2666bca08ded406b1d6f2f0e926b441b7137bca8eb6db47b68e 2024-07-31T14:46:39.143 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 14:46:38 smithi073 systemd[1]: Started Ceph mds.h for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:46:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:38 smithi077 ceph-mon[37327]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:40.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:39 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:40.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:39 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:40.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:39 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:40.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:39 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:40.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:39 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:40.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:39 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:40.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:39 smithi067 ceph-mon[32750]: Deploying daemon mds.i on smithi077 2024-07-31T14:46:40.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:39 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] up:boot 2024-07-31T14:46:40.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:39 smithi067 ceph-mon[32750]: fsmap 8 up:standby 2024-07-31T14:46:40.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:39 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-31T14:46:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:39 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:39 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:40.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:39 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:40.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:39 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:40.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:39 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:40.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:39 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:40.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:39 smithi077 ceph-mon[37327]: Deploying daemon mds.i on smithi077 2024-07-31T14:46:40.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:39 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] up:boot 2024-07-31T14:46:40.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:39 smithi077 ceph-mon[37327]: fsmap 8 up:standby 2024-07-31T14:46:40.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:39 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-31T14:46:40.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:39 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:39 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:39 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:39 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:39 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:39 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:39 smithi073 ceph-mon[36714]: Deploying daemon mds.i on smithi077 2024-07-31T14:46:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:39 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] up:boot 2024-07-31T14:46:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:39 smithi073 ceph-mon[36714]: fsmap 8 up:standby 2024-07-31T14:46:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:39 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-31T14:46:40.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.557+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/596681424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c88102490 msgr2=0x7f9c881028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:40.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.557+0000 7f9c8d4d0640 1 --2- 172.21.15.67:0/596681424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c88102490 0x7f9c881028a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009ca0 tx=0x7f9c7802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:46:40.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.558+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/596681424 shutdown_connections 2024-07-31T14:46:40.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.558+0000 7f9c8d4d0640 1 --2- 172.21.15.67:0/596681424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c88108720 0x7f9c8810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:40.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.558+0000 7f9c8d4d0640 1 --2- 172.21.15.67:0/596681424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c88107dd0 0x7f9c881081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:40.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.558+0000 7f9c8d4d0640 1 --2- 172.21.15.67:0/596681424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c88102490 0x7f9c881028a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:40.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.558+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/596681424 >> 172.21.15.67:0/596681424 conn(0x7f9c88079160 msgr2=0x7f9c88079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:40.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.558+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/596681424 shutdown_connections 2024-07-31T14:46:40.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.558+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/596681424 wait complete. 2024-07-31T14:46:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.559+0000 7f9c8d4d0640 1 Processor -- start 2024-07-31T14:46:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.559+0000 7f9c8d4d0640 1 -- start start 2024-07-31T14:46:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.559+0000 7f9c8d4d0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c88102490 0x7f9c88198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.559+0000 7f9c8d4d0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c88107dd0 0x7f9c88199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.559+0000 7f9c8d4d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c88108720 0x7f9c8819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.559+0000 7f9c8d4d0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88112380 con 0x7f9c88102490 2024-07-31T14:46:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.559+0000 7f9c8d4d0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88112200 con 0x7f9c88107dd0 2024-07-31T14:46:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.559+0000 7f9c8d4d0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88112500 con 0x7f9c88108720 2024-07-31T14:46:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.559+0000 7f9c86ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c88102490 0x7f9c88198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.559+0000 7f9c86ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c88102490 0x7f9c88198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60948/0 (socket says 172.21.15.67:60948) 2024-07-31T14:46:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.559+0000 7f9c86ffd640 1 -- 172.21.15.67:0/253619089 learned_addr learned my addr 172.21.15.67:0/253619089 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:46:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.559+0000 7f9c877fe640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c88108720 0x7f9c8819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.559+0000 7f9c867fc640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c88107dd0 0x7f9c88199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.560+0000 7f9c86ffd640 1 -- 172.21.15.67:0/253619089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c88108720 msgr2=0x7f9c8819e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.560+0000 7f9c86ffd640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c88108720 0x7f9c8819e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.560+0000 7f9c86ffd640 1 -- 172.21.15.67:0/253619089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c88107dd0 msgr2=0x7f9c88199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.560+0000 7f9c86ffd640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c88107dd0 0x7f9c88199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.560+0000 7f9c86ffd640 1 -- 172.21.15.67:0/253619089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8819ed40 con 0x7f9c88102490 2024-07-31T14:46:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.560+0000 7f9c877fe640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c88108720 0x7f9c8819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:46:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.560+0000 7f9c867fc640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c88107dd0 0x7f9c88199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:46:40.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.560+0000 7f9c86ffd640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c88102490 0x7f9c88198dd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9c7802ade0 tx=0x7f9c78003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:40.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.560+0000 7f9c6ffff640 1 -- 172.21.15.67:0/253619089 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78003d60 con 0x7f9c88102490 2024-07-31T14:46:40.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.560+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/253619089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8819efd0 con 0x7f9c88102490 2024-07-31T14:46:40.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.560+0000 7f9c6ffff640 1 -- 172.21.15.67:0/253619089 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c780044f0 con 0x7f9c88102490 2024-07-31T14:46:40.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.560+0000 7f9c6ffff640 1 -- 172.21.15.67:0/253619089 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7802fd10 con 0x7f9c88102490 2024-07-31T14:46:40.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.560+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/253619089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8806b5f0 con 0x7f9c88102490 2024-07-31T14:46:40.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.561+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/253619089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c88103710 con 0x7f9c88102490 2024-07-31T14:46:40.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.563+0000 7f9c6ffff640 1 -- 172.21.15.67:0/253619089 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f9c780046a0 con 0x7f9c88102490 2024-07-31T14:46:40.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.563+0000 7f9c6ffff640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9c700b3c20 0x7f9c700b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:40.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.564+0000 7f9c867fc640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9c700b3c20 0x7f9c700b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:40.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.564+0000 7f9c6ffff640 1 -- 172.21.15.67:0/253619089 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7342+0+0 (secure 0 0 0) 0x7f9c78105200 con 0x7f9c88102490 2024-07-31T14:46:40.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.564+0000 7f9c867fc640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9c700b3c20 0x7f9c700b6130 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9c88100750 tx=0x7f9c7c005d30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:40.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.567+0000 7f9c6ffff640 1 -- 172.21.15.67:0/253619089 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f9c780ccc20 con 0x7f9c88102490 2024-07-31T14:46:40.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.800+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/253619089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c881028a0 con 0x7f9c88102490 2024-07-31T14:46:40.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.801+0000 7f9c6ffff640 1 -- 172.21.15.67:0/253619089 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7204 (secure 0 0 0) 0x7f9c881028a0 con 0x7f9c88102490 2024-07-31T14:46:40.803 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:46:40.803 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":9,"btime":"2024-07-31T14:46:39:749764+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24634,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24650,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-07-31T14:46:40.803 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 9 2024-07-31T14:46:40.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.804+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/253619089 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9c700b3c20 msgr2=0x7f9c700b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:40.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.804+0000 7f9c8d4d0640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9c700b3c20 0x7f9c700b6130 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9c88100750 tx=0x7f9c7c005d30 comp rx=0 tx=0).stop 2024-07-31T14:46:40.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.804+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/253619089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c88102490 msgr2=0x7f9c88198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:40.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.804+0000 7f9c8d4d0640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c88102490 0x7f9c88198dd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9c7802ade0 tx=0x7f9c78003970 comp rx=0 tx=0).stop 2024-07-31T14:46:40.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.804+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/253619089 shutdown_connections 2024-07-31T14:46:40.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.804+0000 7f9c8d4d0640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9c700b3c20 0x7f9c700b6130 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:40.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.804+0000 7f9c8d4d0640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c88108720 0x7f9c8819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:40.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.804+0000 7f9c8d4d0640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c88107dd0 0x7f9c88199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:40.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.804+0000 7f9c8d4d0640 1 --2- 172.21.15.67:0/253619089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c88102490 0x7f9c88198dd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:40.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.804+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/253619089 >> 172.21.15.67:0/253619089 conn(0x7f9c88079160 msgr2=0x7f9c88100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:40.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.804+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/253619089 shutdown_connections 2024-07-31T14:46:40.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:40.804+0000 7f9c8d4d0640 1 -- 172.21.15.67:0/253619089 wait complete. 2024-07-31T14:46:41.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:40 smithi067 ceph-mon[32750]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:41.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:40 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/253619089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:41.134 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:40 smithi077 ceph-mon[37327]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:41.134 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:40 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/253619089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:41.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:40 smithi073 ceph-mon[36714]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:40 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/253619089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:41.737 INFO:journalctl@ceph.mds.i.smithi077.stdout:Jul 31 14:46:41 smithi077 systemd[1]: Started Ceph mds.i for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:46:42.753 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:42 smithi067 ceph-mon[32750]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:42.753 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:42.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:42.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:42.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:42.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:42.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:42.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:42 smithi067 ceph-mon[32750]: Deploying daemon mds.g on smithi067 2024-07-31T14:46:42.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:42 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] up:boot 2024-07-31T14:46:42.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:42 smithi067 ceph-mon[32750]: fsmap 9 up:standby 2024-07-31T14:46:42.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:42 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-31T14:46:42.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:42 smithi073 ceph-mon[36714]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:42.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:42.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:42.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:42.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:42.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:42.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:42.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:42 smithi073 ceph-mon[36714]: Deploying daemon mds.g on smithi067 2024-07-31T14:46:42.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:42 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] up:boot 2024-07-31T14:46:42.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:42 smithi073 ceph-mon[36714]: fsmap 9 up:standby 2024-07-31T14:46:42.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:42 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-31T14:46:42.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:42 smithi077 ceph-mon[37327]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:42.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:42.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:42.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:42.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:42.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:42.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:42.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:42 smithi077 ceph-mon[37327]: Deploying daemon mds.g on smithi067 2024-07-31T14:46:42.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:42 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] up:boot 2024-07-31T14:46:42.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:42 smithi077 ceph-mon[37327]: fsmap 9 up:standby 2024-07-31T14:46:42.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:42 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-31T14:46:43.555 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:46:44.199 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:46:44.303 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 31 14:46:44 smithi067 systemd[1]: Started Ceph mds.g for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:46:44.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:44 smithi073 ceph-mon[36714]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:44.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:44 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:44.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:44 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:44.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:44 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:44.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:44 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:44.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:44 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:44.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:44 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:44.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:44 smithi077 ceph-mon[37327]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:44.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:44 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:44.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:44 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:44.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:44 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:44.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:44 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:44.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:44 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:44.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:44 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:45.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:44 smithi067 ceph-mon[32750]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:45.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:44 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:45.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:44 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:45.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:44 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:45.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:44 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:45.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:44 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:45.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:44 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:45.848 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:45 smithi067 ceph-mon[32750]: Deploying daemon mds.a on smithi067 2024-07-31T14:46:45.849 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:45 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.67:6838/3532250223,v1:172.21.15.67:6839/3532250223] up:boot 2024-07-31T14:46:45.849 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:45 smithi067 ceph-mon[32750]: fsmap 10 up:standby 2024-07-31T14:46:45.849 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:45 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-31T14:46:45.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:45 smithi073 ceph-mon[36714]: Deploying daemon mds.a on smithi067 2024-07-31T14:46:45.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:45 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.67:6838/3532250223,v1:172.21.15.67:6839/3532250223] up:boot 2024-07-31T14:46:45.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:45 smithi073 ceph-mon[36714]: fsmap 10 up:standby 2024-07-31T14:46:45.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:45 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-31T14:46:45.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:45 smithi077 ceph-mon[37327]: Deploying daemon mds.a on smithi067 2024-07-31T14:46:45.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:45 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.67:6838/3532250223,v1:172.21.15.67:6839/3532250223] up:boot 2024-07-31T14:46:45.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:45 smithi077 ceph-mon[37327]: fsmap 10 up:standby 2024-07-31T14:46:45.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:45 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-31T14:46:46.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.487+0000 7f678e885640 1 -- 172.21.15.67:0/1907710442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67881024a0 msgr2=0x7f67881028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:46.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.487+0000 7f678e885640 1 --2- 172.21.15.67:0/1907710442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67881024a0 0x7f67881028b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6778009900 tx=0x7f677802a410 comp rx=0 tx=0).stop 2024-07-31T14:46:46.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.487+0000 7f678e885640 1 -- 172.21.15.67:0/1907710442 shutdown_connections 2024-07-31T14:46:46.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.487+0000 7f678e885640 1 --2- 172.21.15.67:0/1907710442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6788108730 0x7f678810ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:46.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.487+0000 7f678e885640 1 --2- 172.21.15.67:0/1907710442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6788107de0 0x7f67881081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:46.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.487+0000 7f678e885640 1 --2- 172.21.15.67:0/1907710442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67881024a0 0x7f67881028b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:46.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.487+0000 7f678e885640 1 -- 172.21.15.67:0/1907710442 >> 172.21.15.67:0/1907710442 conn(0x7f6788079140 msgr2=0x7f6788079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:46.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.487+0000 7f678e885640 1 -- 172.21.15.67:0/1907710442 shutdown_connections 2024-07-31T14:46:46.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.487+0000 7f678e885640 1 -- 172.21.15.67:0/1907710442 wait complete. 2024-07-31T14:46:46.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.488+0000 7f678e885640 1 Processor -- start 2024-07-31T14:46:46.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.488+0000 7f678e885640 1 -- start start 2024-07-31T14:46:46.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f678e885640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67881024a0 0x7f6788198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:46.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f678e885640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6788107de0 0x7f6788199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:46.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f678e885640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6788108730 0x7f678819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:46.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f678e885640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f67881123e0 con 0x7f67881024a0 2024-07-31T14:46:46.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f678e885640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6788112260 con 0x7f6788107de0 2024-07-31T14:46:46.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f678e885640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6788112560 con 0x7f6788108730 2024-07-31T14:46:46.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f6787fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67881024a0 0x7f6788198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:46.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f6787fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67881024a0 0x7f6788198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43582/0 (socket says 172.21.15.67:43582) 2024-07-31T14:46:46.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f6787fff640 1 -- 172.21.15.67:0/1344724256 learned_addr learned my addr 172.21.15.67:0/1344724256 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:46:46.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f67877fe640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6788107de0 0x7f6788199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:46.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f678ce22640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6788108730 0x7f678819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:46.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f6787fff640 1 -- 172.21.15.67:0/1344724256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6788108730 msgr2=0x7f678819e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:46.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f6787fff640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6788108730 0x7f678819e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:46.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f6787fff640 1 -- 172.21.15.67:0/1344724256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6788107de0 msgr2=0x7f6788199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:46.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f6787fff640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6788107de0 0x7f6788199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:46.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f6787fff640 1 -- 172.21.15.67:0/1344724256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678819ed40 con 0x7f67881024a0 2024-07-31T14:46:46.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f678ce22640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6788108730 0x7f678819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:46:46.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f6787fff640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67881024a0 0x7f6788198dd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6778005940 tx=0x7f6778004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:46.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f67857fa640 1 -- 172.21.15.67:0/1344724256 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677803f030 con 0x7f67881024a0 2024-07-31T14:46:46.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f67857fa640 1 -- 172.21.15.67:0/1344724256 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6778003aa0 con 0x7f67881024a0 2024-07-31T14:46:46.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f678e885640 1 -- 172.21.15.67:0/1344724256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678819efd0 con 0x7f67881024a0 2024-07-31T14:46:46.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.489+0000 7f67857fa640 1 -- 172.21.15.67:0/1344724256 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778035070 con 0x7f67881024a0 2024-07-31T14:46:46.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.490+0000 7f678e885640 1 -- 172.21.15.67:0/1344724256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67881a5790 con 0x7f67881024a0 2024-07-31T14:46:46.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.490+0000 7f678e885640 1 -- 172.21.15.67:0/1344724256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6788103720 con 0x7f67881024a0 2024-07-31T14:46:46.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.492+0000 7f67857fa640 1 -- 172.21.15.67:0/1344724256 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f6778042030 con 0x7f67881024a0 2024-07-31T14:46:46.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.493+0000 7f67857fa640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f67700b3c20 0x7f67700b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:46.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.494+0000 7f67857fa640 1 -- 172.21.15.67:0/1344724256 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7342+0+0 (secure 0 0 0) 0x7f6778104160 con 0x7f67881024a0 2024-07-31T14:46:46.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.494+0000 7f67877fe640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f67700b3c20 0x7f67700b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:46.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.494+0000 7f67877fe640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f67700b3c20 0x7f67700b6130 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6788104c40 tx=0x7f677c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:46.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.496+0000 7f67857fa640 1 -- 172.21.15.67:0/1344724256 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f67780cbb80 con 0x7f67881024a0 2024-07-31T14:46:46.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.740+0000 7f678e885640 1 -- 172.21.15.67:0/1344724256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67881028b0 con 0x7f67881024a0 2024-07-31T14:46:46.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.741+0000 7f67857fa640 1 -- 172.21.15.67:0/1344724256 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8857 (secure 0 0 0) 0x7f67780cb480 con 0x7f67881024a0 2024-07-31T14:46:46.743 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:46:46.743 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":11,"btime":"2024-07-31T14:46:44:610230+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24634,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24650,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24655,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-07-31T14:46:46.743 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 11 2024-07-31T14:46:46.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.743+0000 7f678e885640 1 -- 172.21.15.67:0/1344724256 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f67700b3c20 msgr2=0x7f67700b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.743+0000 7f678e885640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f67700b3c20 0x7f67700b6130 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6788104c40 tx=0x7f677c00b390 comp rx=0 tx=0).stop 2024-07-31T14:46:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.743+0000 7f678e885640 1 -- 172.21.15.67:0/1344724256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67881024a0 msgr2=0x7f6788198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.743+0000 7f678e885640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67881024a0 0x7f6788198dd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6778005940 tx=0x7f6778004270 comp rx=0 tx=0).stop 2024-07-31T14:46:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.743+0000 7f678e885640 1 -- 172.21.15.67:0/1344724256 shutdown_connections 2024-07-31T14:46:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.743+0000 7f678e885640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f67700b3c20 0x7f67700b6130 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.743+0000 7f678e885640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6788108730 0x7f678819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.743+0000 7f678e885640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6788107de0 0x7f6788199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.743+0000 7f678e885640 1 --2- 172.21.15.67:0/1344724256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67881024a0 0x7f6788198dd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.743+0000 7f678e885640 1 -- 172.21.15.67:0/1344724256 >> 172.21.15.67:0/1344724256 conn(0x7f6788079140 msgr2=0x7f6788105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.743+0000 7f678e885640 1 -- 172.21.15.67:0/1344724256 shutdown_connections 2024-07-31T14:46:46.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:46.743+0000 7f678e885640 1 -- 172.21.15.67:0/1344724256 wait complete. 2024-07-31T14:46:46.888 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:46 smithi077 ceph-mon[37327]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:46.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:46 smithi073 ceph-mon[36714]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:46.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:46 smithi067 ceph-mon[32750]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:47.553 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 14:46:47 smithi067 systemd[1]: Starting Ceph mds.a for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T14:46:47.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:47 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1344724256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:47.977 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:47 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1344724256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:47.977 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 14:46:47 smithi067 podman[80762]: 2024-07-31 14:46:47.731882572 +0000 UTC m=+0.018450255 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T14:46:47.977 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 14:46:47 smithi067 podman[80762]: 2024-07-31 14:46:47.846037132 +0000 UTC m=+0.132608850 container create 143db92221e1558924f6f773c239c984d9721ba3b212d8dc65c7f907c26eb84a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T14:46:47.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:47 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1344724256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:48.553 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 14:46:48 smithi067 podman[80762]: 2024-07-31 14:46:48.090336791 +0000 UTC m=+0.376904488 container init 143db92221e1558924f6f773c239c984d9721ba3b212d8dc65c7f907c26eb84a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-a, io.buildah.version=1.36.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716) 2024-07-31T14:46:48.554 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 14:46:48 smithi067 podman[80762]: 2024-07-31 14:46:48.093282566 +0000 UTC m=+0.379850248 container start 143db92221e1558924f6f773c239c984d9721ba3b212d8dc65c7f907c26eb84a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-a, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-07-31T14:46:48.554 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 14:46:48 smithi067 ceph-mds[80782]: -- 172.21.15.67:0/1062340398 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55981c107860 con 0x5598202a9a80 2024-07-31T14:46:48.554 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 14:46:48 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-a[80778]: starting mds.a at 2024-07-31T14:46:48.554 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 14:46:48 smithi067 bash[80762]: 143db92221e1558924f6f773c239c984d9721ba3b212d8dc65c7f907c26eb84a 2024-07-31T14:46:48.554 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 14:46:48 smithi067 systemd[1]: Started Ceph mds.a for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:46:48.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:48 smithi073 ceph-mon[36714]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:48.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:48.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:48.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:48.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:48.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:48.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:48.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:48 smithi077 ceph-mon[37327]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:48.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:48.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:48.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:48.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:48.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:48.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:49.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:48 smithi067 ceph-mon[32750]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:49.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:49.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:49.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-31T14:46:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-31T14:46:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:49.655 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:46:49.803 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:49 smithi073 ceph-mon[36714]: Deploying daemon mds.b on smithi073 2024-07-31T14:46:49.803 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:49 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.67:6840/1797875282,v1:172.21.15.67:6841/1797875282] up:boot 2024-07-31T14:46:49.803 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:49 smithi073 ceph-mon[36714]: fsmap 11 up:standby 2024-07-31T14:46:49.803 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-31T14:46:49.935 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:46:49.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:49 smithi077 ceph-mon[37327]: Deploying daemon mds.b on smithi073 2024-07-31T14:46:49.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:49 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.67:6840/1797875282,v1:172.21.15.67:6841/1797875282] up:boot 2024-07-31T14:46:49.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:49 smithi077 ceph-mon[37327]: fsmap 11 up:standby 2024-07-31T14:46:49.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-31T14:46:50.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:49 smithi067 ceph-mon[32750]: Deploying daemon mds.b on smithi073 2024-07-31T14:46:50.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:49 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.67:6840/1797875282,v1:172.21.15.67:6841/1797875282] up:boot 2024-07-31T14:46:50.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:49 smithi067 ceph-mon[32750]: fsmap 11 up:standby 2024-07-31T14:46:50.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-31T14:46:50.393 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 14:46:50 smithi073 systemd[1]: Starting Ceph mds.b for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T14:46:50.792 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:50 smithi073 ceph-mon[36714]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:50.792 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 14:46:50 smithi073 podman[67875]: 2024-07-31 14:46:50.419195457 +0000 UTC m=+0.016865004 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T14:46:50.792 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 14:46:50 smithi073 podman[67875]: 2024-07-31 14:46:50.538426588 +0000 UTC m=+0.136096125 container create 121ffbbb3f06f05b9f459fe274b9c2c58ced1a5547843d840829a5250181f8e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T14:46:50.792 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 14:46:50 smithi073 podman[67875]: 2024-07-31 14:46:50.791474066 +0000 UTC m=+0.389143604 container init 121ffbbb3f06f05b9f459fe274b9c2c58ced1a5547843d840829a5250181f8e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T14:46:50.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:50 smithi077 ceph-mon[37327]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:51.041 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:50 smithi067 ceph-mon[32750]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:51.142 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 14:46:50 smithi073 podman[67875]: 2024-07-31 14:46:50.794560777 +0000 UTC m=+0.392230321 container start 121ffbbb3f06f05b9f459fe274b9c2c58ced1a5547843d840829a5250181f8e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS) 2024-07-31T14:46:51.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 14:46:50 smithi073 ceph-mds[67894]: -- 172.21.15.73:0/2204503566 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ce7697b860 con 0x55ce7ab1d600 2024-07-31T14:46:51.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 14:46:50 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: starting mds.b at 2024-07-31T14:46:51.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 14:46:50 smithi073 bash[67875]: 121ffbbb3f06f05b9f459fe274b9c2c58ced1a5547843d840829a5250181f8e2 2024-07-31T14:46:51.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 14:46:50 smithi073 systemd[1]: Started Ceph mds.b for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:46:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.563+0000 7f5ab231c640 1 -- 172.21.15.67:0/3195673368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5aac107df0 msgr2=0x7f5aac108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.563+0000 7f5ab231c640 1 --2- 172.21.15.67:0/3195673368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5aac107df0 0x7f5aac108200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5a94009900 tx=0x7f5a9402a410 comp rx=0 tx=0).stop 2024-07-31T14:46:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.563+0000 7f5ab231c640 1 -- 172.21.15.67:0/3195673368 shutdown_connections 2024-07-31T14:46:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.563+0000 7f5ab231c640 1 --2- 172.21.15.67:0/3195673368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aac108740 0x7f5aac10ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.563+0000 7f5ab231c640 1 --2- 172.21.15.67:0/3195673368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5aac107df0 0x7f5aac108200 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.563+0000 7f5ab231c640 1 --2- 172.21.15.67:0/3195673368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac1024b0 0x7f5aac1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:51.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.563+0000 7f5ab231c640 1 -- 172.21.15.67:0/3195673368 >> 172.21.15.67:0/3195673368 conn(0x7f5aac079170 msgr2=0x7f5aac079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:51.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.564+0000 7f5ab231c640 1 -- 172.21.15.67:0/3195673368 shutdown_connections 2024-07-31T14:46:51.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.564+0000 7f5ab231c640 1 -- 172.21.15.67:0/3195673368 wait complete. 2024-07-31T14:46:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.565+0000 7f5ab231c640 1 Processor -- start 2024-07-31T14:46:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.565+0000 7f5ab231c640 1 -- start start 2024-07-31T14:46:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.565+0000 7f5ab231c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac1024b0 0x7f5aac19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.565+0000 7f5ab231c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5aac107df0 0x7f5aac19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.565+0000 7f5ab231c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aac108740 0x7f5aac19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.565+0000 7f5ab231c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac112370 con 0x7f5aac107df0 2024-07-31T14:46:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.565+0000 7f5ab231c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1121f0 con 0x7f5aac108740 2024-07-31T14:46:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.565+0000 7f5ab231c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1124f0 con 0x7f5aac1024b0 2024-07-31T14:46:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.565+0000 7f5ab08b9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aac108740 0x7f5aac19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.565+0000 7f5aa3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5aac107df0 0x7f5aac19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.565+0000 7f5ab08b9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aac108740 0x7f5aac19b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48648/0 (socket says 172.21.15.67:48648) 2024-07-31T14:46:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.565+0000 7f5ab08b9640 1 -- 172.21.15.67:0/1268971758 learned_addr learned my addr 172.21.15.67:0/1268971758 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:46:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.565+0000 7f5aabfff640 1 --2- 172.21.15.67:0/1268971758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac1024b0 0x7f5aac19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.566+0000 7f5ab08b9640 1 -- 172.21.15.67:0/1268971758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac1024b0 msgr2=0x7f5aac19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.566+0000 7f5ab08b9640 1 --2- 172.21.15.67:0/1268971758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac1024b0 0x7f5aac19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.566+0000 7f5ab08b9640 1 -- 172.21.15.67:0/1268971758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5aac107df0 msgr2=0x7f5aac19b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.566+0000 7f5ab08b9640 1 --2- 172.21.15.67:0/1268971758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5aac107df0 0x7f5aac19b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.566+0000 7f5ab08b9640 1 -- 172.21.15.67:0/1268971758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac1a5f20 con 0x7f5aac108740 2024-07-31T14:46:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.566+0000 7f5ab08b9640 1 --2- 172.21.15.67:0/1268971758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aac108740 0x7f5aac19b7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400eb10 tx=0x7f5aa400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:51.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.566+0000 7f5aa9ffb640 1 -- 172.21.15.67:0/1268971758 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa400ce20 con 0x7f5aac108740 2024-07-31T14:46:51.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.566+0000 7f5ab231c640 1 -- 172.21.15.67:0/1268971758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aac1a6210 con 0x7f5aac108740 2024-07-31T14:46:51.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.566+0000 7f5aa9ffb640 1 -- 172.21.15.67:0/1268971758 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa4005020 con 0x7f5aac108740 2024-07-31T14:46:51.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.566+0000 7f5aa9ffb640 1 -- 172.21.15.67:0/1268971758 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4010690 con 0x7f5aac108740 2024-07-31T14:46:51.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.566+0000 7f5ab231c640 1 -- 172.21.15.67:0/1268971758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aac1a6720 con 0x7f5aac108740 2024-07-31T14:46:51.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.567+0000 7f5ab231c640 1 -- 172.21.15.67:0/1268971758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aac103730 con 0x7f5aac108740 2024-07-31T14:46:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.570+0000 7f5aa9ffb640 1 -- 172.21.15.67:0/1268971758 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f5aa4005510 con 0x7f5aac108740 2024-07-31T14:46:51.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.570+0000 7f5aa9ffb640 1 --2- 172.21.15.67:0/1268971758 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5a980b3c50 0x7f5a980b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:51.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.571+0000 7f5aabfff640 1 --2- 172.21.15.67:0/1268971758 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5a980b3c50 0x7f5a980b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:51.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.571+0000 7f5aa9ffb640 1 -- 172.21.15.67:0/1268971758 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7342+0+0 (secure 0 0 0) 0x7f5aa40e3f70 con 0x7f5aac108740 2024-07-31T14:46:51.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.571+0000 7f5aabfff640 1 --2- 172.21.15.67:0/1268971758 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5a980b3c50 0x7f5a980b6160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5aac06c0d0 tx=0x7f5a9c00a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:51.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.573+0000 7f5aa9ffb640 1 -- 172.21.15.67:0/1268971758 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5aa40555e0 con 0x7f5aac108740 2024-07-31T14:46:51.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.824+0000 7f5ab231c640 1 -- 172.21.15.67:0/1268971758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5aac02e0c0 con 0x7f5aac108740 2024-07-31T14:46:51.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.825+0000 7f5aa9ffb640 1 -- 172.21.15.67:0/1268971758 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10509 (secure 0 0 0) 0x7f5aa40ab990 con 0x7f5aac108740 2024-07-31T14:46:51.826 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:46:51.826 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":13,"btime":"2024-07-31T14:46:51:650186+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14790,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24634,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24650,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24655,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-07-31T14:46:51.827 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 13 2024-07-31T14:46:51.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.827+0000 7f5ab231c640 1 -- 172.21.15.67:0/1268971758 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5a980b3c50 msgr2=0x7f5a980b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:51.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.827+0000 7f5ab231c640 1 --2- 172.21.15.67:0/1268971758 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5a980b3c50 0x7f5a980b6160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5aac06c0d0 tx=0x7f5a9c00a2d0 comp rx=0 tx=0).stop 2024-07-31T14:46:51.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.827+0000 7f5ab231c640 1 -- 172.21.15.67:0/1268971758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aac108740 msgr2=0x7f5aac19b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.827+0000 7f5ab231c640 1 --2- 172.21.15.67:0/1268971758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aac108740 0x7f5aac19b7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400eb10 tx=0x7f5aa400ef10 comp rx=0 tx=0).stop 2024-07-31T14:46:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.827+0000 7f5ab231c640 1 -- 172.21.15.67:0/1268971758 shutdown_connections 2024-07-31T14:46:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.827+0000 7f5ab231c640 1 --2- 172.21.15.67:0/1268971758 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5a980b3c50 0x7f5a980b6160 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.827+0000 7f5ab231c640 1 --2- 172.21.15.67:0/1268971758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aac108740 0x7f5aac19b7a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.827+0000 7f5ab231c640 1 --2- 172.21.15.67:0/1268971758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5aac107df0 0x7f5aac19b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.827+0000 7f5ab231c640 1 --2- 172.21.15.67:0/1268971758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac1024b0 0x7f5aac19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.827+0000 7f5ab231c640 1 -- 172.21.15.67:0/1268971758 >> 172.21.15.67:0/1268971758 conn(0x7f5aac079170 msgr2=0x7f5aac104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.827+0000 7f5ab231c640 1 -- 172.21.15.67:0/1268971758 shutdown_connections 2024-07-31T14:46:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:51.827+0000 7f5ab231c640 1 -- 172.21.15.67:0/1268971758 wait complete. 2024-07-31T14:46:51.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:51 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:51.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:51 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:51 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:51 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:51 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:51 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:46:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:51 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] up:boot 2024-07-31T14:46:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:51 smithi077 ceph-mon[37327]: fsmap 12 up:standby 2024-07-31T14:46:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:51 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-31T14:46:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:51 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1268971758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:52.091 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:51 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:52.091 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:51 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:52.091 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:51 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:52.091 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:51 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:52.091 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:51 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:52.091 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:51 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:46:52.092 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:51 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] up:boot 2024-07-31T14:46:52.092 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:51 smithi067 ceph-mon[32750]: fsmap 12 up:standby 2024-07-31T14:46:52.092 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:51 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-31T14:46:52.092 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:51 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1268971758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:51 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:51 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:51 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:51 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:51 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:51 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:46:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:51 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] up:boot 2024-07-31T14:46:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:51 smithi073 ceph-mon[36714]: fsmap 12 up:standby 2024-07-31T14:46:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:51 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-31T14:46:52.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:51 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1268971758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:46:52.733 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-07-31T14:46:53.112 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:46:53.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:52 smithi067 ceph-mon[32750]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:53.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:53.237 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:53.237 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:53.237 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:52 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:52 smithi077 ceph-mon[37327]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:52 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:52 smithi073 ceph-mon[36714]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:53.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:53.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:53.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:52 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:54.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.786+0000 7f78d5047640 1 -- 172.21.15.67:0/795063350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d01035b0 msgr2=0x7f78d01039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:54.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.786+0000 7f78d5047640 1 --2- 172.21.15.67:0/795063350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d01035b0 0x7f78d01039c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009900 tx=0x7f78c802a410 comp rx=0 tx=0).stop 2024-07-31T14:46:54.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.786+0000 7f78d5047640 1 -- 172.21.15.67:0/795063350 shutdown_connections 2024-07-31T14:46:54.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.786+0000 7f78d5047640 1 --2- 172.21.15.67:0/795063350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d0109840 0x7f78d0110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:54.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.786+0000 7f78d5047640 1 --2- 172.21.15.67:0/795063350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78d0108ef0 0x7f78d0109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:54.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.786+0000 7f78d5047640 1 --2- 172.21.15.67:0/795063350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d01035b0 0x7f78d01039c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:54.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.786+0000 7f78d5047640 1 -- 172.21.15.67:0/795063350 >> 172.21.15.67:0/795063350 conn(0x7f78d00ff040 msgr2=0x7f78d0101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:54.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.786+0000 7f78d5047640 1 -- 172.21.15.67:0/795063350 shutdown_connections 2024-07-31T14:46:54.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.786+0000 7f78d5047640 1 -- 172.21.15.67:0/795063350 wait complete. 2024-07-31T14:46:54.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.787+0000 7f78d5047640 1 Processor -- start 2024-07-31T14:46:54.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.787+0000 7f78d5047640 1 -- start start 2024-07-31T14:46:54.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.787+0000 7f78d5047640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d01035b0 0x7f78d0199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:54.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.787+0000 7f78d5047640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78d0108ef0 0x7f78d019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:54.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.787+0000 7f78d5047640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d0109840 0x7f78d019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:54.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.787+0000 7f78d5047640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f78d01134f0 con 0x7f78d0109840 2024-07-31T14:46:54.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.787+0000 7f78d5047640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0113370 con 0x7f78d01035b0 2024-07-31T14:46:54.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.787+0000 7f78d5047640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0113670 con 0x7f78d0108ef0 2024-07-31T14:46:54.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.787+0000 7f78cf59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d0109840 0x7f78d019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:54.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.787+0000 7f78cf59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d0109840 0x7f78d019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51096/0 (socket says 172.21.15.67:51096) 2024-07-31T14:46:54.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.787+0000 7f78cf59e640 1 -- 172.21.15.67:0/2547257597 learned_addr learned my addr 172.21.15.67:0/2547257597 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:46:54.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.788+0000 7f78ce59c640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78d0108ef0 0x7f78d019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:54.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.788+0000 7f78ced9d640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d01035b0 0x7f78d0199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:54.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.788+0000 7f78cf59e640 1 -- 172.21.15.67:0/2547257597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78d0108ef0 msgr2=0x7f78d019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:54.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.788+0000 7f78cf59e640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78d0108ef0 0x7f78d019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:54.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.788+0000 7f78cf59e640 1 -- 172.21.15.67:0/2547257597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d01035b0 msgr2=0x7f78d0199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:54.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.788+0000 7f78cf59e640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d01035b0 0x7f78d0199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:54.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.788+0000 7f78cf59e640 1 -- 172.21.15.67:0/2547257597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d019fe90 con 0x7f78d0109840 2024-07-31T14:46:54.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.788+0000 7f78ce59c640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78d0108ef0 0x7f78d019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:46:54.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.788+0000 7f78cf59e640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d0109840 0x7f78d019f760 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f78c000cad0 tx=0x7f78c000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:54.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.788+0000 7f78c7fff640 1 -- 172.21.15.67:0/2547257597 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c000ae20 con 0x7f78d0109840 2024-07-31T14:46:54.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.788+0000 7f78c7fff640 1 -- 172.21.15.67:0/2547257597 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c0005020 con 0x7f78d0109840 2024-07-31T14:46:54.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.788+0000 7f78c7fff640 1 -- 172.21.15.67:0/2547257597 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c000f6b0 con 0x7f78d0109840 2024-07-31T14:46:54.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.788+0000 7f78d5047640 1 -- 172.21.15.67:0/2547257597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d01a0180 con 0x7f78d0109840 2024-07-31T14:46:54.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.788+0000 7f78d5047640 1 -- 172.21.15.67:0/2547257597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d01a68e0 con 0x7f78d0109840 2024-07-31T14:46:54.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.789+0000 7f78d5047640 1 -- 172.21.15.67:0/2547257597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d01039c0 con 0x7f78d0109840 2024-07-31T14:46:54.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.791+0000 7f78c7fff640 1 -- 172.21.15.67:0/2547257597 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f78c00051d0 con 0x7f78d0109840 2024-07-31T14:46:54.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.792+0000 7f78c7fff640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78bc0b3c20 0x7f78bc0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:54.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.792+0000 7f78ced9d640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78bc0b3c20 0x7f78bc0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:54.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.792+0000 7f78c7fff640 1 -- 172.21.15.67:0/2547257597 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7342+0+0 (secure 0 0 0) 0x7f78c00e3ca0 con 0x7f78d0109840 2024-07-31T14:46:54.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.793+0000 7f78ced9d640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78bc0b3c20 0x7f78bc0b6130 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f78c8031440 tx=0x7f78c8005d50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:54.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:54.795+0000 7f78c7fff640 1 -- 172.21.15.67:0/2547257597 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f78c0014030 con 0x7f78d0109840 2024-07-31T14:46:55.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:55.021+0000 7f78d5047640 1 -- 172.21.15.67:0/2547257597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f78d002e0c0 con 0x7f78d0109840 2024-07-31T14:46:55.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:55.021+0000 7f78c7fff640 1 -- 172.21.15.67:0/2547257597 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7f78c00ab6c0 con 0x7f78d0109840 2024-07-31T14:46:55.023 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:46:55.023 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-07-31T14:46:55.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T14:46:55.024 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-07-31T14:46:55.024 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-07-31T14:46:55.024 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:46:55.024 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-07-31T14:46:55.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:55.024+0000 7f78d5047640 1 -- 172.21.15.67:0/2547257597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78bc0b3c20 msgr2=0x7f78bc0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:55.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:55.024+0000 7f78d5047640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78bc0b3c20 0x7f78bc0b6130 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f78c8031440 tx=0x7f78c8005d50 comp rx=0 tx=0).stop 2024-07-31T14:46:55.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:55.024+0000 7f78d5047640 1 -- 172.21.15.67:0/2547257597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d0109840 msgr2=0x7f78d019f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:55.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:55.024+0000 7f78d5047640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d0109840 0x7f78d019f760 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f78c000cad0 tx=0x7f78c000cf90 comp rx=0 tx=0).stop 2024-07-31T14:46:55.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:55.024+0000 7f78d5047640 1 -- 172.21.15.67:0/2547257597 shutdown_connections 2024-07-31T14:46:55.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:55.024+0000 7f78d5047640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78bc0b3c20 0x7f78bc0b6130 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:55.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:55.024+0000 7f78d5047640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d0109840 0x7f78d019f760 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:55.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:55.024+0000 7f78d5047640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78d0108ef0 0x7f78d019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:55.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:55.024+0000 7f78d5047640 1 --2- 172.21.15.67:0/2547257597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d01035b0 0x7f78d0199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:55.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:55.024+0000 7f78d5047640 1 -- 172.21.15.67:0/2547257597 >> 172.21.15.67:0/2547257597 conn(0x7f78d00ff040 msgr2=0x7f78d0101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:55.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:55.024+0000 7f78d5047640 1 -- 172.21.15.67:0/2547257597 shutdown_connections 2024-07-31T14:46:55.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:55.024+0000 7f78d5047640 1 -- 172.21.15.67:0/2547257597 wait complete. 2024-07-31T14:46:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:54 smithi077 ceph-mon[37327]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:54 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:54 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:54 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:54 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:55.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:54 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:55.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:54 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:55.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:54 smithi067 ceph-mon[32750]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:55.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:54 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:55.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:54 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:55.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:54 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:55.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:54 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:55.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:54 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:55.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:54 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:55.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:54 smithi073 ceph-mon[36714]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:54 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:54 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:54 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:54 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:54 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:54 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:55.685 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-31T14:46:55.685 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-31T14:46:56.301 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:55 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2547257597' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-31T14:46:56.301 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:55 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:46:56.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:55 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2547257597' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-31T14:46:56.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:55 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:46:56.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:55 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2547257597' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-31T14:46:56.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:55 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:46:56.609 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:46:56.991 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:56 smithi067 ceph-mon[32750]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:56.991 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:56 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:56.991 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:56 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:56.991 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:56 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:56.991 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:56 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:57.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:56 smithi073 ceph-mon[36714]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:57.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:56 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:56 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:56 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:56 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:57.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:56 smithi077 ceph-mon[37327]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:57.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:56 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:57.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:56 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:56 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:56 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:58.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.238+0000 7f6321e9b640 1 -- 172.21.15.67:0/3936200580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f631c105560 msgr2=0x7f631c10efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:58.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.238+0000 7f6321e9b640 1 --2- 172.21.15.67:0/3936200580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f631c105560 0x7f631c10efb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6314009ca0 tx=0x7f631402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:46:58.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.238+0000 7f6321e9b640 1 -- 172.21.15.67:0/3936200580 shutdown_connections 2024-07-31T14:46:58.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.238+0000 7f6321e9b640 1 --2- 172.21.15.67:0/3936200580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f631c105560 0x7f631c10efb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:58.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.238+0000 7f6321e9b640 1 --2- 172.21.15.67:0/3936200580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f631c104c10 0x7f631c105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:58.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.238+0000 7f6321e9b640 1 --2- 172.21.15.67:0/3936200580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f631c102c40 0x7f631c103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:58.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.238+0000 7f6321e9b640 1 -- 172.21.15.67:0/3936200580 >> 172.21.15.67:0/3936200580 conn(0x7f631c0799c0 msgr2=0x7f631c079dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:58.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.238+0000 7f6321e9b640 1 -- 172.21.15.67:0/3936200580 shutdown_connections 2024-07-31T14:46:58.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.238+0000 7f6321e9b640 1 -- 172.21.15.67:0/3936200580 wait complete. 2024-07-31T14:46:58.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.239+0000 7f6321e9b640 1 Processor -- start 2024-07-31T14:46:58.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.239+0000 7f6321e9b640 1 -- start start 2024-07-31T14:46:58.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.239+0000 7f6321e9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f631c102c40 0x7f631c19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:58.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.239+0000 7f6321e9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f631c104c10 0x7f631c19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:58.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.239+0000 7f6321e9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f631c105560 0x7f631c1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:58.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.239+0000 7f6321e9b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f631c113520 con 0x7f631c105560 2024-07-31T14:46:58.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.239+0000 7f6321e9b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1133a0 con 0x7f631c104c10 2024-07-31T14:46:58.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.239+0000 7f6321e9b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1136a0 con 0x7f631c102c40 2024-07-31T14:46:58.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f631bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f631c105560 0x7f631c1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f631bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f631c105560 0x7f631c1a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51114/0 (socket says 172.21.15.67:51114) 2024-07-31T14:46:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f631bfff640 1 -- 172.21.15.67:0/2546785484 learned_addr learned my addr 172.21.15.67:0/2546785484 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:46:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f631b7fe640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f631c102c40 0x7f631c19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f631affd640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f631c104c10 0x7f631c19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f631bfff640 1 -- 172.21.15.67:0/2546785484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f631c102c40 msgr2=0x7f631c19c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f631bfff640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f631c102c40 0x7f631c19c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f631bfff640 1 -- 172.21.15.67:0/2546785484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f631c104c10 msgr2=0x7f631c19c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f631bfff640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f631c104c10 0x7f631c19c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f631bfff640 1 -- 172.21.15.67:0/2546785484 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c1a2100 con 0x7f631c105560 2024-07-31T14:46:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f631b7fe640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f631c102c40 0x7f631c19c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:46:58.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f631bfff640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f631c105560 0x7f631c1a19d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6314005cc0 tx=0x7f631402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:58.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f6318ff9640 1 -- 172.21.15.67:0/2546785484 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314003bf0 con 0x7f631c105560 2024-07-31T14:46:58.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f6318ff9640 1 -- 172.21.15.67:0/2546785484 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6314003d90 con 0x7f631c105560 2024-07-31T14:46:58.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f6321e9b640 1 -- 172.21.15.67:0/2546785484 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631c1a2390 con 0x7f631c105560 2024-07-31T14:46:58.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.240+0000 7f6318ff9640 1 -- 172.21.15.67:0/2546785484 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631402fbc0 con 0x7f631c105560 2024-07-31T14:46:58.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.241+0000 7f6321e9b640 1 -- 172.21.15.67:0/2546785484 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631c1a8b50 con 0x7f631c105560 2024-07-31T14:46:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.241+0000 7f6321e9b640 1 -- 172.21.15.67:0/2546785484 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f631c103050 con 0x7f631c105560 2024-07-31T14:46:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.243+0000 7f6318ff9640 1 -- 172.21.15.67:0/2546785484 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f631402fd60 con 0x7f631c105560 2024-07-31T14:46:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.244+0000 7f6318ff9640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f63040b3c20 0x7f63040b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:46:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.244+0000 7f631b7fe640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f63040b3c20 0x7f63040b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:46:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.244+0000 7f6318ff9640 1 -- 172.21.15.67:0/2546785484 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7342+0+0 (secure 0 0 0) 0x7f63141050c0 con 0x7f631c105560 2024-07-31T14:46:58.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.244+0000 7f631b7fe640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f63040b3c20 0x7f63040b6130 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f630c00bfd0 tx=0x7f630c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:46:58.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.247+0000 7f6318ff9640 1 -- 172.21.15.67:0/2546785484 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f63140ccae0 con 0x7f631c105560 2024-07-31T14:46:58.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:58.454+0000 7f6321e9b640 1 -- 172.21.15.67:0/2546785484 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f631c1a8d80 con 0x7f631c105560 2024-07-31T14:46:59.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:59.001+0000 7f6318ff9640 1 -- 172.21.15.67:0/2546785484 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7f63140098b0 con 0x7f631c105560 2024-07-31T14:46:59.002 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-07-31T14:46:59.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:59.003+0000 7f6321e9b640 1 -- 172.21.15.67:0/2546785484 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f63040b3c20 msgr2=0x7f63040b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:59.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:59.003+0000 7f6321e9b640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f63040b3c20 0x7f63040b6130 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f630c00bfd0 tx=0x7f630c00b390 comp rx=0 tx=0).stop 2024-07-31T14:46:59.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:59.003+0000 7f6321e9b640 1 -- 172.21.15.67:0/2546785484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f631c105560 msgr2=0x7f631c1a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:46:59.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:59.003+0000 7f6321e9b640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f631c105560 0x7f631c1a19d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6314005cc0 tx=0x7f631402f380 comp rx=0 tx=0).stop 2024-07-31T14:46:59.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:59.003+0000 7f6321e9b640 1 -- 172.21.15.67:0/2546785484 shutdown_connections 2024-07-31T14:46:59.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:59.003+0000 7f6321e9b640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f63040b3c20 0x7f63040b6130 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:59.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:59.003+0000 7f6321e9b640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f631c105560 0x7f631c1a19d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:59.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:59.003+0000 7f6321e9b640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f631c104c10 0x7f631c19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:59.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:59.003+0000 7f6321e9b640 1 --2- 172.21.15.67:0/2546785484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f631c102c40 0x7f631c19c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:46:59.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:59.003+0000 7f6321e9b640 1 -- 172.21.15.67:0/2546785484 >> 172.21.15.67:0/2546785484 conn(0x7f631c0799c0 msgr2=0x7f631c105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:46:59.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:59.003+0000 7f6321e9b640 1 -- 172.21.15.67:0/2546785484 shutdown_connections 2024-07-31T14:46:59.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:46:59.004+0000 7f6321e9b640 1 -- 172.21.15.67:0/2546785484 wait complete. 2024-07-31T14:46:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:58 smithi067 ceph-mon[32750]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:59.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:58 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2546785484' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-31T14:46:59.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:59.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:59.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:59.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:59.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:46:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:59.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:58 smithi073 ceph-mon[36714]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:59.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:58 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2546785484' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-31T14:46:59.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:59.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:59.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:59.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:59.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:46:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:59.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:58 smithi077 ceph-mon[37327]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:46:59.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:58 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2546785484' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-31T14:46:59.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:46:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:46:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:46:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:46:59.688 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-07-31T14:46:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:00.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:00 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2546785484' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-31T14:47:00.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:00 smithi067 ceph-mon[32750]: osdmap e74: 12 total, 12 up, 12 in 2024-07-31T14:47:00.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:00 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2546785484' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-31T14:47:00.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:00 smithi073 ceph-mon[36714]: osdmap e74: 12 total, 12 up, 12 in 2024-07-31T14:47:00.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:00 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2546785484' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-31T14:47:00.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:00 smithi077 ceph-mon[37327]: osdmap e74: 12 total, 12 up, 12 in 2024-07-31T14:47:01.037 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:01 smithi067 ceph-mon[32750]: pgmap v262: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:01.037 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:01 smithi067 ceph-mon[32750]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-31T14:47:01.037 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:01 smithi067 ceph-mon[32750]: osdmap e75: 12 total, 12 up, 12 in 2024-07-31T14:47:01.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:01 smithi073 ceph-mon[36714]: pgmap v262: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:01 smithi073 ceph-mon[36714]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-31T14:47:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:01 smithi073 ceph-mon[36714]: osdmap e75: 12 total, 12 up, 12 in 2024-07-31T14:47:01.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:01 smithi077 ceph-mon[37327]: pgmap v262: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:01.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:01 smithi077 ceph-mon[37327]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-31T14:47:01.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:01 smithi077 ceph-mon[37327]: osdmap e75: 12 total, 12 up, 12 in 2024-07-31T14:47:01.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.574+0000 7fabb30a5640 1 -- 172.21.15.67:0/2685646667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac108720 msgr2=0x7fabac10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:01.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.574+0000 7fabb30a5640 1 --2- 172.21.15.67:0/2685646667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac108720 0x7fabac10ef50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faba4009900 tx=0x7faba402a410 comp rx=0 tx=0).stop 2024-07-31T14:47:01.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.574+0000 7fabb30a5640 1 -- 172.21.15.67:0/2685646667 shutdown_connections 2024-07-31T14:47:01.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.574+0000 7fabb30a5640 1 --2- 172.21.15.67:0/2685646667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac108720 0x7fabac10ef50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:01.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.574+0000 7fabb30a5640 1 --2- 172.21.15.67:0/2685646667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabac107dd0 0x7fabac1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:01.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.574+0000 7fabb30a5640 1 --2- 172.21.15.67:0/2685646667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabac102490 0x7fabac1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:01.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.574+0000 7fabb30a5640 1 -- 172.21.15.67:0/2685646667 >> 172.21.15.67:0/2685646667 conn(0x7fabac079160 msgr2=0x7fabac079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:01.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.574+0000 7fabb30a5640 1 -- 172.21.15.67:0/2685646667 shutdown_connections 2024-07-31T14:47:01.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.574+0000 7fabb30a5640 1 -- 172.21.15.67:0/2685646667 wait complete. 2024-07-31T14:47:01.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.575+0000 7fabb30a5640 1 Processor -- start 2024-07-31T14:47:01.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.575+0000 7fabb30a5640 1 -- start start 2024-07-31T14:47:01.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb30a5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabac102490 0x7fabac198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:01.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb30a5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabac107dd0 0x7fabac199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:01.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb30a5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac108720 0x7fabac19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:01.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb30a5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fabac1123a0 con 0x7fabac108720 2024-07-31T14:47:01.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb30a5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fabac112220 con 0x7fabac107dd0 2024-07-31T14:47:01.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb30a5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fabac112520 con 0x7fabac102490 2024-07-31T14:47:01.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb1642640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac108720 0x7fabac19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:01.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb1642640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac108720 0x7fabac19e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33312/0 (socket says 172.21.15.67:33312) 2024-07-31T14:47:01.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb1642640 1 -- 172.21.15.67:0/3407885144 learned_addr learned my addr 172.21.15.67:0/3407885144 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:01.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fababfff640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabac107dd0 0x7fabac199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:01.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb0e41640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabac102490 0x7fabac198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:01.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb1642640 1 -- 172.21.15.67:0/3407885144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabac102490 msgr2=0x7fabac198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:01.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb1642640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabac102490 0x7fabac198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:01.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb1642640 1 -- 172.21.15.67:0/3407885144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabac107dd0 msgr2=0x7fabac199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:01.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb1642640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabac107dd0 0x7fabac199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:01.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb1642640 1 -- 172.21.15.67:0/3407885144 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabac19ed00 con 0x7fabac108720 2024-07-31T14:47:01.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fababfff640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabac107dd0 0x7fabac199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:47:01.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7fabb1642640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac108720 0x7fabac19e5d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faba4004770 tx=0x7faba4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:01.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.576+0000 7faba9ffb640 1 -- 172.21.15.67:0/3407885144 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba403f030 con 0x7fabac108720 2024-07-31T14:47:01.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.577+0000 7fabb30a5640 1 -- 172.21.15.67:0/3407885144 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabac19ef90 con 0x7fabac108720 2024-07-31T14:47:01.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.577+0000 7faba9ffb640 1 -- 172.21.15.67:0/3407885144 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba4003aa0 con 0x7fabac108720 2024-07-31T14:47:01.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.577+0000 7faba9ffb640 1 -- 172.21.15.67:0/3407885144 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4035070 con 0x7fabac108720 2024-07-31T14:47:01.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.577+0000 7fabb30a5640 1 -- 172.21.15.67:0/3407885144 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabac1a5750 con 0x7fabac108720 2024-07-31T14:47:01.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.578+0000 7fabb30a5640 1 -- 172.21.15.67:0/3407885144 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabac1028a0 con 0x7fabac108720 2024-07-31T14:47:01.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.580+0000 7faba9ffb640 1 -- 172.21.15.67:0/3407885144 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7faba4042030 con 0x7fabac108720 2024-07-31T14:47:01.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.580+0000 7faba9ffb640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fab980b3c20 0x7fab980b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:01.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.580+0000 7fabb0e41640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fab980b3c20 0x7fab980b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:01.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.581+0000 7faba9ffb640 1 -- 172.21.15.67:0/3407885144 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7712+0+0 (secure 0 0 0) 0x7faba4104450 con 0x7fabac108720 2024-07-31T14:47:01.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.581+0000 7fabb0e41640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fab980b3c20 0x7fab980b6130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fab9c005fd0 tx=0x7fab9c009870 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:01.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.584+0000 7faba9ffb640 1 -- 172.21.15.67:0/3407885144 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7faba40cbd00 con 0x7fabac108720 2024-07-31T14:47:01.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:01.779+0000 7fabb30a5640 1 -- 172.21.15.67:0/3407885144 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fabac199ec0 con 0x7fabac108720 2024-07-31T14:47:02.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:02.021+0000 7faba9ffb640 1 -- 172.21.15.67:0/3407885144 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v77) ==== 171+0+0 (secure 0 0 0) 0x7fabac199ec0 con 0x7fabac108720 2024-07-31T14:47:02.023 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-07-31T14:47:02.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:02.024+0000 7fabb30a5640 1 -- 172.21.15.67:0/3407885144 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fab980b3c20 msgr2=0x7fab980b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:02.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:02.024+0000 7fabb30a5640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fab980b3c20 0x7fab980b6130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fab9c005fd0 tx=0x7fab9c009870 comp rx=0 tx=0).stop 2024-07-31T14:47:02.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:02.024+0000 7fabb30a5640 1 -- 172.21.15.67:0/3407885144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac108720 msgr2=0x7fabac19e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:02.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:02.024+0000 7fabb30a5640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac108720 0x7fabac19e5d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faba4004770 tx=0x7faba4004270 comp rx=0 tx=0).stop 2024-07-31T14:47:02.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:02.024+0000 7fabb30a5640 1 -- 172.21.15.67:0/3407885144 shutdown_connections 2024-07-31T14:47:02.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:02.024+0000 7fabb30a5640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fab980b3c20 0x7fab980b6130 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:02.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:02.024+0000 7fabb30a5640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac108720 0x7fabac19e5d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:02.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:02.024+0000 7fabb30a5640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabac107dd0 0x7fabac199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:02.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:02.024+0000 7fabb30a5640 1 --2- 172.21.15.67:0/3407885144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabac102490 0x7fabac198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:02.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:02.024+0000 7fabb30a5640 1 -- 172.21.15.67:0/3407885144 >> 172.21.15.67:0/3407885144 conn(0x7fabac079160 msgr2=0x7fabac104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:02.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:02.024+0000 7fabb30a5640 1 -- 172.21.15.67:0/3407885144 shutdown_connections 2024-07-31T14:47:02.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:02.024+0000 7fabb30a5640 1 -- 172.21.15.67:0/3407885144 wait complete. 2024-07-31T14:47:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:02 smithi067 ceph-mon[32750]: osdmap e76: 12 total, 12 up, 12 in 2024-07-31T14:47:02.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:02 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3407885144' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-07-31T14:47:02.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:02 smithi073 ceph-mon[36714]: osdmap e76: 12 total, 12 up, 12 in 2024-07-31T14:47:02.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:02 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3407885144' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-07-31T14:47:02.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:02 smithi077 ceph-mon[37327]: osdmap e76: 12 total, 12 up, 12 in 2024-07-31T14:47:02.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:02 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3407885144' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-07-31T14:47:02.702 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-31T14:47:02.971 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:03.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:03 smithi067 ceph-mon[32750]: pgmap v265: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:03.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:03 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3407885144' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-07-31T14:47:03.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:03 smithi067 ceph-mon[32750]: osdmap e77: 12 total, 12 up, 12 in 2024-07-31T14:47:03.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:03 smithi073 ceph-mon[36714]: pgmap v265: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:03.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:03 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3407885144' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-07-31T14:47:03.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:03 smithi073 ceph-mon[36714]: osdmap e77: 12 total, 12 up, 12 in 2024-07-31T14:47:03.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:03 smithi077 ceph-mon[37327]: pgmap v265: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:03.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:03 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3407885144' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-07-31T14:47:03.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:03 smithi077 ceph-mon[37327]: osdmap e77: 12 total, 12 up, 12 in 2024-07-31T14:47:04.096 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:04 smithi067 ceph-mon[32750]: osdmap e78: 12 total, 12 up, 12 in 2024-07-31T14:47:04.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:04 smithi073 ceph-mon[36714]: osdmap e78: 12 total, 12 up, 12 in 2024-07-31T14:47:04.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:04 smithi077 ceph-mon[37327]: osdmap e78: 12 total, 12 up, 12 in 2024-07-31T14:47:04.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.588+0000 7f649e89f640 1 -- 172.21.15.67:0/408052364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64981035e0 msgr2=0x7f64981039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:04.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.588+0000 7f649e89f640 1 --2- 172.21.15.67:0/408052364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64981035e0 0x7f64981039f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6488009900 tx=0x7f648802a410 comp rx=0 tx=0).stop 2024-07-31T14:47:04.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.588+0000 7f649e89f640 1 -- 172.21.15.67:0/408052364 shutdown_connections 2024-07-31T14:47:04.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.588+0000 7f649e89f640 1 --2- 172.21.15.67:0/408052364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6498109870 0x7f64981100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:04.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.588+0000 7f649e89f640 1 --2- 172.21.15.67:0/408052364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6498108f20 0x7f6498109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:04.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.588+0000 7f649e89f640 1 --2- 172.21.15.67:0/408052364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64981035e0 0x7f64981039f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:04.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.588+0000 7f649e89f640 1 -- 172.21.15.67:0/408052364 >> 172.21.15.67:0/408052364 conn(0x7f64980ff030 msgr2=0x7f6498101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:04.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.588+0000 7f649e89f640 1 -- 172.21.15.67:0/408052364 shutdown_connections 2024-07-31T14:47:04.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.588+0000 7f649e89f640 1 -- 172.21.15.67:0/408052364 wait complete. 2024-07-31T14:47:04.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.588+0000 7f649e89f640 1 Processor -- start 2024-07-31T14:47:04.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.588+0000 7f649e89f640 1 -- start start 2024-07-31T14:47:04.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f649e89f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64981035e0 0x7f6498199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:04.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f649e89f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6498108f20 0x7f649819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:04.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f649e89f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6498109870 0x7f649819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:04.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f649e89f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6498113520 con 0x7f6498108f20 2024-07-31T14:47:04.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f649e89f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f64981133a0 con 0x7f6498109870 2024-07-31T14:47:04.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f649e89f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64981136a0 con 0x7f64981035e0 2024-07-31T14:47:04.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f64977fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6498108f20 0x7f649819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:04.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f64977fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6498108f20 0x7f649819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33330/0 (socket says 172.21.15.67:33330) 2024-07-31T14:47:04.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f64977fe640 1 -- 172.21.15.67:0/2565466359 learned_addr learned my addr 172.21.15.67:0/2565466359 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:04.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f6497fff640 1 --2- 172.21.15.67:0/2565466359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64981035e0 0x7f6498199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:04.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f649ce3c640 1 --2- 172.21.15.67:0/2565466359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6498109870 0x7f649819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:04.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f64977fe640 1 -- 172.21.15.67:0/2565466359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64981035e0 msgr2=0x7f6498199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:04.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f64977fe640 1 --2- 172.21.15.67:0/2565466359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64981035e0 0x7f6498199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:04.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f64977fe640 1 -- 172.21.15.67:0/2565466359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6498109870 msgr2=0x7f649819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:04.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f64977fe640 1 --2- 172.21.15.67:0/2565466359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6498109870 0x7f649819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:04.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.589+0000 7f64977fe640 1 -- 172.21.15.67:0/2565466359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649819fe60 con 0x7f6498108f20 2024-07-31T14:47:04.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.590+0000 7f64977fe640 1 --2- 172.21.15.67:0/2565466359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6498108f20 0x7f649819a490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f648c00e9f0 tx=0x7f648c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:04.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.590+0000 7f64957fa640 1 -- 172.21.15.67:0/2565466359 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c00ccc0 con 0x7f6498108f20 2024-07-31T14:47:04.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.590+0000 7f649e89f640 1 -- 172.21.15.67:0/2565466359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64981a0150 con 0x7f6498108f20 2024-07-31T14:47:04.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.590+0000 7f649e89f640 1 -- 172.21.15.67:0/2565466359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64981a68b0 con 0x7f6498108f20 2024-07-31T14:47:04.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.590+0000 7f64957fa640 1 -- 172.21.15.67:0/2565466359 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f648c0108e0 con 0x7f6498108f20 2024-07-31T14:47:04.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.590+0000 7f64957fa640 1 -- 172.21.15.67:0/2565466359 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c018860 con 0x7f6498108f20 2024-07-31T14:47:04.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.591+0000 7f649e89f640 1 -- 172.21.15.67:0/2565466359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64981039f0 con 0x7f6498108f20 2024-07-31T14:47:04.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.593+0000 7f64957fa640 1 -- 172.21.15.67:0/2565466359 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f648c010430 con 0x7f6498108f20 2024-07-31T14:47:04.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.593+0000 7f64957fa640 1 --2- 172.21.15.67:0/2565466359 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f64800b3ca0 0x7f64800b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:04.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.593+0000 7f6497fff640 1 --2- 172.21.15.67:0/2565466359 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f64800b3ca0 0x7f64800b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:04.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.594+0000 7f64957fa640 1 -- 172.21.15.67:0/2565466359 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8094+0+0 (secure 0 0 0) 0x7f648c0e42f0 con 0x7f6498108f20 2024-07-31T14:47:04.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.594+0000 7f6497fff640 1 --2- 172.21.15.67:0/2565466359 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f64800b3ca0 0x7f64800b61b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6488031440 tx=0x7f6488009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:04.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.597+0000 7f64957fa640 1 -- 172.21.15.67:0/2565466359 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f648c0aba20 con 0x7f6498108f20 2024-07-31T14:47:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:04.831+0000 7f649e89f640 1 -- 172.21.15.67:0/2565466359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f649819b020 con 0x7f6498108f20 2024-07-31T14:47:05.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:05.037+0000 7f64957fa640 1 -- 172.21.15.67:0/2565466359 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f648c0ab320 con 0x7f6498108f20 2024-07-31T14:47:05.039 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 2 and data pool 3 2024-07-31T14:47:05.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:05.040+0000 7f649e89f640 1 -- 172.21.15.67:0/2565466359 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f64800b3ca0 msgr2=0x7f64800b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:05.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:05.040+0000 7f649e89f640 1 --2- 172.21.15.67:0/2565466359 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f64800b3ca0 0x7f64800b61b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6488031440 tx=0x7f6488009560 comp rx=0 tx=0).stop 2024-07-31T14:47:05.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:05.040+0000 7f649e89f640 1 -- 172.21.15.67:0/2565466359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6498108f20 msgr2=0x7f649819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:05.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:05.040+0000 7f649e89f640 1 --2- 172.21.15.67:0/2565466359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6498108f20 0x7f649819a490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f648c00e9f0 tx=0x7f648c00eeb0 comp rx=0 tx=0).stop 2024-07-31T14:47:05.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:05.040+0000 7f649e89f640 1 -- 172.21.15.67:0/2565466359 shutdown_connections 2024-07-31T14:47:05.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:05.040+0000 7f649e89f640 1 --2- 172.21.15.67:0/2565466359 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f64800b3ca0 0x7f64800b61b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:05.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:05.040+0000 7f649e89f640 1 --2- 172.21.15.67:0/2565466359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6498109870 0x7f649819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:05.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:05.040+0000 7f649e89f640 1 --2- 172.21.15.67:0/2565466359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6498108f20 0x7f649819a490 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:05.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:05.040+0000 7f649e89f640 1 --2- 172.21.15.67:0/2565466359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64981035e0 0x7f6498199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:05.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:05.040+0000 7f649e89f640 1 -- 172.21.15.67:0/2565466359 >> 172.21.15.67:0/2565466359 conn(0x7f64980ff030 msgr2=0x7f6498101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:05.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:05.040+0000 7f649e89f640 1 -- 172.21.15.67:0/2565466359 shutdown_connections 2024-07-31T14:47:05.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:05.040+0000 7f649e89f640 1 -- 172.21.15.67:0/2565466359 wait complete. 2024-07-31T14:47:05.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:05 smithi067 ceph-mon[32750]: pgmap v268: 97 pgs: 35 active+clean, 13 creating+peering, 49 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:05.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:05 smithi067 ceph-mon[32750]: osdmap e79: 12 total, 12 up, 12 in 2024-07-31T14:47:05.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:05 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2565466359' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-31T14:47:05.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:05 smithi067 ceph-mon[32750]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-07-31T14:47:05.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:05 smithi073 ceph-mon[36714]: pgmap v268: 97 pgs: 35 active+clean, 13 creating+peering, 49 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:05.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:05 smithi073 ceph-mon[36714]: osdmap e79: 12 total, 12 up, 12 in 2024-07-31T14:47:05.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:05 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2565466359' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-31T14:47:05.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:05 smithi073 ceph-mon[36714]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-07-31T14:47:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:05 smithi077 ceph-mon[37327]: pgmap v268: 97 pgs: 35 active+clean, 13 creating+peering, 49 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:05 smithi077 ceph-mon[37327]: osdmap e79: 12 total, 12 up, 12 in 2024-07-31T14:47:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:05 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2565466359' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-31T14:47:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:05 smithi077 ceph-mon[37327]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-07-31T14:47:05.767 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-07-31T14:47:05.768 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-07-31T14:47:06.036 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:06.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:06 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2565466359' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-31T14:47:06.375 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:06 smithi067 ceph-mon[32750]: osdmap e80: 12 total, 12 up, 12 in 2024-07-31T14:47:06.375 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:06 smithi067 ceph-mon[32750]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-07-31T14:47:06.375 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:06 smithi067 ceph-mon[32750]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-07-31T14:47:06.375 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:06 smithi067 ceph-mon[32750]: pgmap v271: 97 pgs: 71 active+clean, 4 creating+peering, 22 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:06.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:06 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2565466359' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-31T14:47:06.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:06 smithi073 ceph-mon[36714]: osdmap e80: 12 total, 12 up, 12 in 2024-07-31T14:47:06.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:06 smithi073 ceph-mon[36714]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-07-31T14:47:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:06 smithi073 ceph-mon[36714]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-07-31T14:47:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:06 smithi073 ceph-mon[36714]: pgmap v271: 97 pgs: 71 active+clean, 4 creating+peering, 22 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:06.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:06 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2565466359' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-31T14:47:06.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:06 smithi077 ceph-mon[37327]: osdmap e80: 12 total, 12 up, 12 in 2024-07-31T14:47:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:06 smithi077 ceph-mon[37327]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-07-31T14:47:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:06 smithi077 ceph-mon[37327]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-07-31T14:47:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:06 smithi077 ceph-mon[37327]: pgmap v271: 97 pgs: 71 active+clean, 4 creating+peering, 22 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:07.055 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 14:47:06 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T14:47:06.658+0000 7f2e996a6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-31T14:47:07.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:07 smithi067 ceph-mon[32750]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-31T14:47:07.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:07 smithi067 ceph-mon[32750]: Cluster is now healthy 2024-07-31T14:47:07.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:07 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] up:active 2024-07-31T14:47:07.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:07 smithi067 ceph-mon[32750]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:47:07.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:07 smithi067 ceph-mon[32750]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:47:07.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:07 smithi073 ceph-mon[36714]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-31T14:47:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:07 smithi073 ceph-mon[36714]: Cluster is now healthy 2024-07-31T14:47:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:07 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] up:active 2024-07-31T14:47:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:07 smithi073 ceph-mon[36714]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:47:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:07 smithi073 ceph-mon[36714]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:47:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:07 smithi077 ceph-mon[37327]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-31T14:47:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:07 smithi077 ceph-mon[37327]: Cluster is now healthy 2024-07-31T14:47:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:07 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] up:active 2024-07-31T14:47:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:07 smithi077 ceph-mon[37327]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:47:07.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:07 smithi077 ceph-mon[37327]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:47:07.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.657+0000 7f22cd9bc640 1 -- 172.21.15.67:0/1309410917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c8109820 msgr2=0x7f22c8110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:07.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.657+0000 7f22cd9bc640 1 --2- 172.21.15.67:0/1309410917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c8109820 0x7f22c8110050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009ca0 tx=0x7f22c002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:47:07.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.657+0000 7f22cd9bc640 1 -- 172.21.15.67:0/1309410917 shutdown_connections 2024-07-31T14:47:07.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.657+0000 7f22cd9bc640 1 --2- 172.21.15.67:0/1309410917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c8109820 0x7f22c8110050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:07.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.657+0000 7f22cd9bc640 1 --2- 172.21.15.67:0/1309410917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22c8108ed0 0x7f22c81092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:07.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.657+0000 7f22cd9bc640 1 --2- 172.21.15.67:0/1309410917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22c8103590 0x7f22c81039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:07.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.657+0000 7f22cd9bc640 1 -- 172.21.15.67:0/1309410917 >> 172.21.15.67:0/1309410917 conn(0x7f22c807b280 msgr2=0x7f22c807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:07.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.657+0000 7f22cd9bc640 1 -- 172.21.15.67:0/1309410917 shutdown_connections 2024-07-31T14:47:07.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.657+0000 7f22cd9bc640 1 -- 172.21.15.67:0/1309410917 wait complete. 2024-07-31T14:47:07.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.657+0000 7f22cd9bc640 1 Processor -- start 2024-07-31T14:47:07.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22cd9bc640 1 -- start start 2024-07-31T14:47:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22cd9bc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c8103590 0x7f22c8199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22cd9bc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22c8108ed0 0x7f22c819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22cd9bc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22c8109820 0x7f22c819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22cd9bc640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f22c81134f0 con 0x7f22c8103590 2024-07-31T14:47:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22cd9bc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f22c8113370 con 0x7f22c8109820 2024-07-31T14:47:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22cd9bc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f22c8113670 con 0x7f22c8108ed0 2024-07-31T14:47:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22c6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c8103590 0x7f22c8199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22c6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c8103590 0x7f22c8199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33356/0 (socket says 172.21.15.67:33356) 2024-07-31T14:47:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22c6ffd640 1 -- 172.21.15.67:0/53753692 learned_addr learned my addr 172.21.15.67:0/53753692 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22bffff640 1 --2- 172.21.15.67:0/53753692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22c8108ed0 0x7f22c819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22c77fe640 1 --2- 172.21.15.67:0/53753692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22c8109820 0x7f22c819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22c6ffd640 1 -- 172.21.15.67:0/53753692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22c8108ed0 msgr2=0x7f22c819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:07.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22c6ffd640 1 --2- 172.21.15.67:0/53753692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22c8108ed0 0x7f22c819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:07.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22c6ffd640 1 -- 172.21.15.67:0/53753692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22c8109820 msgr2=0x7f22c819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:07.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22c6ffd640 1 --2- 172.21.15.67:0/53753692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22c8109820 0x7f22c819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:07.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.658+0000 7f22c6ffd640 1 -- 172.21.15.67:0/53753692 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c819fd70 con 0x7f22c8103590 2024-07-31T14:47:07.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.659+0000 7f22c6ffd640 1 --2- 172.21.15.67:0/53753692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c8103590 0x7f22c8199ec0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f22b800c920 tx=0x7f22b800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:07.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.659+0000 7f22cc9ba640 1 -- 172.21.15.67:0/53753692 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8007c70 con 0x7f22c8103590 2024-07-31T14:47:07.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.659+0000 7f22cc9ba640 1 -- 172.21.15.67:0/53753692 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22b8007e10 con 0x7f22c8103590 2024-07-31T14:47:07.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.659+0000 7f22cd9bc640 1 -- 172.21.15.67:0/53753692 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c81a0060 con 0x7f22c8103590 2024-07-31T14:47:07.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.659+0000 7f22cc9ba640 1 -- 172.21.15.67:0/53753692 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8018820 con 0x7f22c8103590 2024-07-31T14:47:07.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.659+0000 7f22cd9bc640 1 -- 172.21.15.67:0/53753692 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c81a6880 con 0x7f22c8103590 2024-07-31T14:47:07.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.660+0000 7f22cd9bc640 1 -- 172.21.15.67:0/53753692 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22c81039a0 con 0x7f22c8103590 2024-07-31T14:47:07.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.662+0000 7f22cc9ba640 1 -- 172.21.15.67:0/53753692 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f22b800f450 con 0x7f22c8103590 2024-07-31T14:47:07.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.663+0000 7f22cc9ba640 1 --2- 172.21.15.67:0/53753692 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f22b40b3cd0 0x7f22b40b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:07.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.663+0000 7f22bffff640 1 --2- 172.21.15.67:0/53753692 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f22b40b3cd0 0x7f22b40b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:07.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.663+0000 7f22cc9ba640 1 -- 172.21.15.67:0/53753692 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8194+0+0 (secure 0 0 0) 0x7f22b80e3ea0 con 0x7f22c8103590 2024-07-31T14:47:07.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.664+0000 7f22bffff640 1 --2- 172.21.15.67:0/53753692 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f22b40b3cd0 0x7f22b40b61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f22c8108b20 tx=0x7f22b000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:07.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.667+0000 7f22cc9ba640 1 -- 172.21.15.67:0/53753692 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f22b8014030 con 0x7f22c8103590 2024-07-31T14:47:07.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:07.870+0000 7f22cd9bc640 1 -- 172.21.15.67:0/53753692 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f22c819aff0 con 0x7f22c8103590 2024-07-31T14:47:08.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:08.060+0000 7f22cc9ba640 1 -- 172.21.15.67:0/53753692 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v81) ==== 157+0+0 (secure 0 0 0) 0x7f22b80ab620 con 0x7f22c8103590 2024-07-31T14:47:08.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:08.062+0000 7f22cd9bc640 1 -- 172.21.15.67:0/53753692 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f22b40b3cd0 msgr2=0x7f22b40b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:08.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:08.062+0000 7f22cd9bc640 1 --2- 172.21.15.67:0/53753692 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f22b40b3cd0 0x7f22b40b61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f22c8108b20 tx=0x7f22b000b390 comp rx=0 tx=0).stop 2024-07-31T14:47:08.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:08.062+0000 7f22cd9bc640 1 -- 172.21.15.67:0/53753692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c8103590 msgr2=0x7f22c8199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:08.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:08.062+0000 7f22cd9bc640 1 --2- 172.21.15.67:0/53753692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c8103590 0x7f22c8199ec0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f22b800c920 tx=0x7f22b800cde0 comp rx=0 tx=0).stop 2024-07-31T14:47:08.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:08.062+0000 7f22cd9bc640 1 -- 172.21.15.67:0/53753692 shutdown_connections 2024-07-31T14:47:08.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:08.062+0000 7f22cd9bc640 1 --2- 172.21.15.67:0/53753692 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f22b40b3cd0 0x7f22b40b61e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:08.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:08.062+0000 7f22cd9bc640 1 --2- 172.21.15.67:0/53753692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22c8109820 0x7f22c819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:08.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:08.062+0000 7f22cd9bc640 1 --2- 172.21.15.67:0/53753692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22c8108ed0 0x7f22c819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:08.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:08.062+0000 7f22cd9bc640 1 --2- 172.21.15.67:0/53753692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c8103590 0x7f22c8199ec0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:08.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:08.062+0000 7f22cd9bc640 1 -- 172.21.15.67:0/53753692 >> 172.21.15.67:0/53753692 conn(0x7f22c807b280 msgr2=0x7f22c8106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:08.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:08.062+0000 7f22cd9bc640 1 -- 172.21.15.67:0/53753692 shutdown_connections 2024-07-31T14:47:08.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:08.063+0000 7f22cd9bc640 1 -- 172.21.15.67:0/53753692 wait complete. 2024-07-31T14:47:08.369 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:08 smithi067 ceph-mon[32750]: pgmap v272: 97 pgs: 97 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-31T14:47:08.369 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:08 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/53753692' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-07-31T14:47:08.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:08 smithi073 ceph-mon[36714]: pgmap v272: 97 pgs: 97 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-31T14:47:08.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:08 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/53753692' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-07-31T14:47:08.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:08 smithi077 ceph-mon[37327]: pgmap v272: 97 pgs: 97 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-31T14:47:08.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:08 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/53753692' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-07-31T14:47:08.719 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-07-31T14:47:08.991 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:09.376 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:09 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/53753692' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-07-31T14:47:09.376 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:09 smithi067 ceph-mon[32750]: osdmap e81: 12 total, 12 up, 12 in 2024-07-31T14:47:09.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:09 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/53753692' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-07-31T14:47:09.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:09 smithi073 ceph-mon[36714]: osdmap e81: 12 total, 12 up, 12 in 2024-07-31T14:47:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:09 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/53753692' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-07-31T14:47:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:09 smithi077 ceph-mon[37327]: osdmap e81: 12 total, 12 up, 12 in 2024-07-31T14:47:10.348 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:10 smithi067 ceph-mon[32750]: pgmap v274: 97 pgs: 97 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-07-31T14:47:10.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:10 smithi073 ceph-mon[36714]: pgmap v274: 97 pgs: 97 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-07-31T14:47:10.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:10 smithi077 ceph-mon[37327]: pgmap v274: 97 pgs: 97 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-07-31T14:47:10.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.720+0000 7fb7e0284640 1 -- 172.21.15.67:0/1246768313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7d81035d0 msgr2=0x7fb7d81039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:10.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.720+0000 7fb7e0284640 1 --2- 172.21.15.67:0/1246768313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7d81035d0 0x7fb7d81039e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4009900 tx=0x7fb7d402a410 comp rx=0 tx=0).stop 2024-07-31T14:47:10.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.720+0000 7fb7e0284640 1 -- 172.21.15.67:0/1246768313 shutdown_connections 2024-07-31T14:47:10.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.720+0000 7fb7e0284640 1 --2- 172.21.15.67:0/1246768313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7d8109860 0x7fb7d8110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:10.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.720+0000 7fb7e0284640 1 --2- 172.21.15.67:0/1246768313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7d8108f10 0x7fb7d8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:10.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.720+0000 7fb7e0284640 1 --2- 172.21.15.67:0/1246768313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7d81035d0 0x7fb7d81039e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:10.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.720+0000 7fb7e0284640 1 -- 172.21.15.67:0/1246768313 >> 172.21.15.67:0/1246768313 conn(0x7fb7d80ff040 msgr2=0x7fb7d8101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:10.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.720+0000 7fb7e0284640 1 -- 172.21.15.67:0/1246768313 shutdown_connections 2024-07-31T14:47:10.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.720+0000 7fb7e0284640 1 -- 172.21.15.67:0/1246768313 wait complete. 2024-07-31T14:47:10.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7e0284640 1 Processor -- start 2024-07-31T14:47:10.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7e0284640 1 -- start start 2024-07-31T14:47:10.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7e0284640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7d81035d0 0x7fb7d8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:10.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7e0284640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7d8108f10 0x7fb7d819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:10.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7e0284640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7d8109860 0x7fb7d819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:10.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7e0284640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d8113580 con 0x7fb7d81035d0 2024-07-31T14:47:10.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7e0284640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d8113400 con 0x7fb7d8109860 2024-07-31T14:47:10.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7e0284640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d8113700 con 0x7fb7d8108f10 2024-07-31T14:47:10.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7de020640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7d81035d0 0x7fb7d8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:10.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7de020640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7d81035d0 0x7fb7d8199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33372/0 (socket says 172.21.15.67:33372) 2024-07-31T14:47:10.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7de020640 1 -- 172.21.15.67:0/2723972467 learned_addr learned my addr 172.21.15.67:0/2723972467 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:10.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7de821640 1 --2- 172.21.15.67:0/2723972467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7d8109860 0x7fb7d819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:10.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7dd81f640 1 --2- 172.21.15.67:0/2723972467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7d8108f10 0x7fb7d819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:10.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7de020640 1 -- 172.21.15.67:0/2723972467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7d8108f10 msgr2=0x7fb7d819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:10.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7de020640 1 --2- 172.21.15.67:0/2723972467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7d8108f10 0x7fb7d819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:10.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7de020640 1 -- 172.21.15.67:0/2723972467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7d8109860 msgr2=0x7fb7d819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:10.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7de020640 1 --2- 172.21.15.67:0/2723972467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7d8109860 0x7fb7d819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:10.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.721+0000 7fb7de020640 1 -- 172.21.15.67:0/2723972467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d819fee0 con 0x7fb7d81035d0 2024-07-31T14:47:10.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.722+0000 7fb7de020640 1 --2- 172.21.15.67:0/2723972467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7d81035d0 0x7fb7d8199f70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4006010 tx=0x7fb7d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:10.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.722+0000 7fb7cf7fe640 1 -- 172.21.15.67:0/2723972467 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d403f030 con 0x7fb7d81035d0 2024-07-31T14:47:10.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.722+0000 7fb7cf7fe640 1 -- 172.21.15.67:0/2723972467 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7d4003800 con 0x7fb7d81035d0 2024-07-31T14:47:10.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.722+0000 7fb7e0284640 1 -- 172.21.15.67:0/2723972467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d81a0170 con 0x7fb7d81035d0 2024-07-31T14:47:10.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.722+0000 7fb7cf7fe640 1 -- 172.21.15.67:0/2723972467 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4035070 con 0x7fb7d81035d0 2024-07-31T14:47:10.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.722+0000 7fb7e0284640 1 -- 172.21.15.67:0/2723972467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d81a6930 con 0x7fb7d81035d0 2024-07-31T14:47:10.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.723+0000 7fb7e0284640 1 -- 172.21.15.67:0/2723972467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d81039e0 con 0x7fb7d81035d0 2024-07-31T14:47:10.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.725+0000 7fb7cf7fe640 1 -- 172.21.15.67:0/2723972467 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fb7d4003cb0 con 0x7fb7d81035d0 2024-07-31T14:47:10.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.725+0000 7fb7cf7fe640 1 --2- 172.21.15.67:0/2723972467 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb7c40b3c20 0x7fb7c40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:10.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.725+0000 7fb7dd81f640 1 --2- 172.21.15.67:0/2723972467 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb7c40b3c20 0x7fb7c40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:10.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.725+0000 7fb7cf7fe640 1 -- 172.21.15.67:0/2723972467 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8492+0+0 (secure 0 0 0) 0x7fb7d4104890 con 0x7fb7d81035d0 2024-07-31T14:47:10.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.726+0000 7fb7dd81f640 1 --2- 172.21.15.67:0/2723972467 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb7c40b3c20 0x7fb7c40b6130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8108b50 tx=0x7fb7c800a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:10.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.729+0000 7fb7cf7fe640 1 -- 172.21.15.67:0/2723972467 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb7d40cb790 con 0x7fb7d81035d0 2024-07-31T14:47:10.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:10.923+0000 7fb7e0284640 1 -- 172.21.15.67:0/2723972467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb7d819b0a0 con 0x7fb7d81035d0 2024-07-31T14:47:11.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:11 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2723972467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-31T14:47:11.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:11 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2723972467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-31T14:47:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:11 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2723972467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-31T14:47:12.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:12.079+0000 7fb7cf7fe640 1 -- 172.21.15.67:0/2723972467 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v83) ==== 228+0+0 (secure 0 0 0) 0x7fb7d819b0a0 con 0x7fb7d81035d0 2024-07-31T14:47:12.082 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data_ec' created 2024-07-31T14:47:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:12.082+0000 7fb7e0284640 1 -- 172.21.15.67:0/2723972467 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb7c40b3c20 msgr2=0x7fb7c40b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:12.082+0000 7fb7e0284640 1 --2- 172.21.15.67:0/2723972467 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb7c40b3c20 0x7fb7c40b6130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8108b50 tx=0x7fb7c800a2d0 comp rx=0 tx=0).stop 2024-07-31T14:47:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:12.082+0000 7fb7e0284640 1 -- 172.21.15.67:0/2723972467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7d81035d0 msgr2=0x7fb7d8199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:12.082+0000 7fb7e0284640 1 --2- 172.21.15.67:0/2723972467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7d81035d0 0x7fb7d8199f70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4006010 tx=0x7fb7d40042f0 comp rx=0 tx=0).stop 2024-07-31T14:47:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:12.082+0000 7fb7e0284640 1 -- 172.21.15.67:0/2723972467 shutdown_connections 2024-07-31T14:47:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:12.082+0000 7fb7e0284640 1 --2- 172.21.15.67:0/2723972467 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb7c40b3c20 0x7fb7c40b6130 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:12.082+0000 7fb7e0284640 1 --2- 172.21.15.67:0/2723972467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7d8109860 0x7fb7d819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:12.082+0000 7fb7e0284640 1 --2- 172.21.15.67:0/2723972467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7d8108f10 0x7fb7d819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:12.082+0000 7fb7e0284640 1 --2- 172.21.15.67:0/2723972467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7d81035d0 0x7fb7d8199f70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:12.082+0000 7fb7e0284640 1 -- 172.21.15.67:0/2723972467 >> 172.21.15.67:0/2723972467 conn(0x7fb7d80ff040 msgr2=0x7fb7d8101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:12.082+0000 7fb7e0284640 1 -- 172.21.15.67:0/2723972467 shutdown_connections 2024-07-31T14:47:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:12.082+0000 7fb7e0284640 1 -- 172.21.15.67:0/2723972467 wait complete. 2024-07-31T14:47:12.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:12 smithi073 ceph-mon[36714]: osdmap e82: 12 total, 12 up, 12 in 2024-07-31T14:47:12.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:12 smithi073 ceph-mon[36714]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-07-31T14:47:12.404 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:12 smithi067 ceph-mon[32750]: osdmap e82: 12 total, 12 up, 12 in 2024-07-31T14:47:12.405 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:12 smithi067 ceph-mon[32750]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-07-31T14:47:12.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:12 smithi077 ceph-mon[37327]: osdmap e82: 12 total, 12 up, 12 in 2024-07-31T14:47:12.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:12 smithi077 ceph-mon[37327]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-07-31T14:47:12.692 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-07-31T14:47:12.966 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:13.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:13 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2723972467' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-31T14:47:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:13 smithi073 ceph-mon[36714]: osdmap e83: 12 total, 12 up, 12 in 2024-07-31T14:47:13.444 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:13 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2723972467' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-31T14:47:13.444 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:13 smithi067 ceph-mon[32750]: osdmap e83: 12 total, 12 up, 12 in 2024-07-31T14:47:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:13 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2723972467' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-31T14:47:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:13 smithi077 ceph-mon[37327]: osdmap e83: 12 total, 12 up, 12 in 2024-07-31T14:47:14.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:14 smithi067 ceph-mon[32750]: osdmap e84: 12 total, 12 up, 12 in 2024-07-31T14:47:14.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:14 smithi067 ceph-mon[32750]: pgmap v279: 129 pgs: 2 creating+activating, 25 unknown, 102 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 964 B/s wr, 3 op/s 2024-07-31T14:47:14.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:14 smithi073 ceph-mon[36714]: osdmap e84: 12 total, 12 up, 12 in 2024-07-31T14:47:14.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:14 smithi073 ceph-mon[36714]: pgmap v279: 129 pgs: 2 creating+activating, 25 unknown, 102 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 964 B/s wr, 3 op/s 2024-07-31T14:47:14.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:14 smithi077 ceph-mon[37327]: osdmap e84: 12 total, 12 up, 12 in 2024-07-31T14:47:14.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:14 smithi077 ceph-mon[37327]: pgmap v279: 129 pgs: 2 creating+activating, 25 unknown, 102 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 964 B/s wr, 3 op/s 2024-07-31T14:47:14.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.615+0000 7f7407849640 1 -- 172.21.15.67:0/2297759023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400107dd0 msgr2=0x7f74001081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.615+0000 7f7407849640 1 --2- 172.21.15.67:0/2297759023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400107dd0 0x7f74001081e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009900 tx=0x7f73f402a410 comp rx=0 tx=0).stop 2024-07-31T14:47:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.616+0000 7f7407849640 1 -- 172.21.15.67:0/2297759023 shutdown_connections 2024-07-31T14:47:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.616+0000 7f7407849640 1 --2- 172.21.15.67:0/2297759023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7400108720 0x7f740010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.616+0000 7f7407849640 1 --2- 172.21.15.67:0/2297759023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400107dd0 0x7f74001081e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.616+0000 7f7407849640 1 --2- 172.21.15.67:0/2297759023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7400102490 0x7f74001028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.616+0000 7f7407849640 1 -- 172.21.15.67:0/2297759023 >> 172.21.15.67:0/2297759023 conn(0x7f7400079160 msgr2=0x7f7400079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.616+0000 7f7407849640 1 -- 172.21.15.67:0/2297759023 shutdown_connections 2024-07-31T14:47:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.616+0000 7f7407849640 1 -- 172.21.15.67:0/2297759023 wait complete. 2024-07-31T14:47:14.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.617+0000 7f7407849640 1 Processor -- start 2024-07-31T14:47:14.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.617+0000 7f7407849640 1 -- start start 2024-07-31T14:47:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.617+0000 7f7407849640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7400102490 0x7f740019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.617+0000 7f7407849640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400107dd0 0x7f740019b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.617+0000 7f7407849640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7400108720 0x7f740019b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.617+0000 7f7407849640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7400112350 con 0x7f7400107dd0 2024-07-31T14:47:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.617+0000 7f7407849640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f74001121d0 con 0x7f7400108720 2024-07-31T14:47:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.617+0000 7f7407849640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74001124d0 con 0x7f7400102490 2024-07-31T14:47:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.617+0000 7f7404de4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400107dd0 0x7f740019b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.617+0000 7f7404de4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400107dd0 0x7f740019b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52236/0 (socket says 172.21.15.67:52236) 2024-07-31T14:47:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.617+0000 7f7404de4640 1 -- 172.21.15.67:0/2887091066 learned_addr learned my addr 172.21.15.67:0/2887091066 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.617+0000 7f74055e5640 1 --2- 172.21.15.67:0/2887091066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7400102490 0x7f740019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.617+0000 7f7405de6640 1 --2- 172.21.15.67:0/2887091066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7400108720 0x7f740019b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.618+0000 7f7404de4640 1 -- 172.21.15.67:0/2887091066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7400102490 msgr2=0x7f740019aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.618+0000 7f7404de4640 1 --2- 172.21.15.67:0/2887091066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7400102490 0x7f740019aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.618+0000 7f7404de4640 1 -- 172.21.15.67:0/2887091066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7400108720 msgr2=0x7f740019b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.618+0000 7f7404de4640 1 --2- 172.21.15.67:0/2887091066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7400108720 0x7f740019b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.618+0000 7f7404de4640 1 -- 172.21.15.67:0/2887091066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74001a5f30 con 0x7f7400107dd0 2024-07-31T14:47:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.618+0000 7f7404de4640 1 --2- 172.21.15.67:0/2887091066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400107dd0 0x7f740019b1e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f73f4006010 tx=0x7f73f4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.618+0000 7f73fe7fc640 1 -- 172.21.15.67:0/2887091066 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f403f030 con 0x7f7400107dd0 2024-07-31T14:47:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.618+0000 7f73fe7fc640 1 -- 172.21.15.67:0/2887091066 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73f4003aa0 con 0x7f7400107dd0 2024-07-31T14:47:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.618+0000 7f7407849640 1 -- 172.21.15.67:0/2887091066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74001a61c0 con 0x7f7400107dd0 2024-07-31T14:47:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.618+0000 7f73fe7fc640 1 -- 172.21.15.67:0/2887091066 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f4035070 con 0x7f7400107dd0 2024-07-31T14:47:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.618+0000 7f7407849640 1 -- 172.21.15.67:0/2887091066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74001a66a0 con 0x7f7400107dd0 2024-07-31T14:47:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.619+0000 7f7407849640 1 -- 172.21.15.67:0/2887091066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74001028a0 con 0x7f7400107dd0 2024-07-31T14:47:14.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.621+0000 7f73fe7fc640 1 -- 172.21.15.67:0/2887091066 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f73f4042030 con 0x7f7400107dd0 2024-07-31T14:47:14.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.622+0000 7f73fe7fc640 1 --2- 172.21.15.67:0/2887091066 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f73e80b3c50 0x7f73e80b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:14.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.622+0000 7f74055e5640 1 --2- 172.21.15.67:0/2887091066 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f73e80b3c50 0x7f73e80b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:14.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.622+0000 7f73fe7fc640 1 -- 172.21.15.67:0/2887091066 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8985+0+0 (secure 0 0 0) 0x7f73f41046e0 con 0x7f7400107dd0 2024-07-31T14:47:14.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.622+0000 7f74055e5640 1 --2- 172.21.15.67:0/2887091066 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f73e80b3c50 0x7f73e80b6160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f740006b3f0 tx=0x7f73f800b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:14.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.625+0000 7f73fe7fc640 1 -- 172.21.15.67:0/2887091066 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f73f40cb3f0 con 0x7f7400107dd0 2024-07-31T14:47:14.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:14.819+0000 7f7407849640 1 -- 172.21.15.67:0/2887091066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f740002e0c0 con 0x7f7400107dd0 2024-07-31T14:47:15.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:15.099+0000 7f73fe7fc640 1 -- 172.21.15.67:0/2887091066 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) ==== 169+0+0 (secure 0 0 0) 0x7f73f40cf940 con 0x7f7400107dd0 2024-07-31T14:47:15.101 INFO:teuthology.orchestra.run.smithi067.stderr:set pool 4 allow_ec_overwrites to true 2024-07-31T14:47:15.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:15.102+0000 7f7407849640 1 -- 172.21.15.67:0/2887091066 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f73e80b3c50 msgr2=0x7f73e80b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:15.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:15.102+0000 7f7407849640 1 --2- 172.21.15.67:0/2887091066 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f73e80b3c50 0x7f73e80b6160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f740006b3f0 tx=0x7f73f800b410 comp rx=0 tx=0).stop 2024-07-31T14:47:15.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:15.102+0000 7f7407849640 1 -- 172.21.15.67:0/2887091066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400107dd0 msgr2=0x7f740019b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:15.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:15.102+0000 7f7407849640 1 --2- 172.21.15.67:0/2887091066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400107dd0 0x7f740019b1e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f73f4006010 tx=0x7f73f4004060 comp rx=0 tx=0).stop 2024-07-31T14:47:15.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:15.102+0000 7f7407849640 1 -- 172.21.15.67:0/2887091066 shutdown_connections 2024-07-31T14:47:15.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:15.102+0000 7f7407849640 1 --2- 172.21.15.67:0/2887091066 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f73e80b3c50 0x7f73e80b6160 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:15.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:15.102+0000 7f7407849640 1 --2- 172.21.15.67:0/2887091066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7400108720 0x7f740019b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:15.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:15.102+0000 7f7407849640 1 --2- 172.21.15.67:0/2887091066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400107dd0 0x7f740019b1e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:15.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:15.102+0000 7f7407849640 1 --2- 172.21.15.67:0/2887091066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7400102490 0x7f740019aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:15.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:15.102+0000 7f7407849640 1 -- 172.21.15.67:0/2887091066 >> 172.21.15.67:0/2887091066 conn(0x7f7400079160 msgr2=0x7f7400104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:15.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:15.102+0000 7f7407849640 1 -- 172.21.15.67:0/2887091066 shutdown_connections 2024-07-31T14:47:15.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:15.102+0000 7f7407849640 1 -- 172.21.15.67:0/2887091066 wait complete. 2024-07-31T14:47:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:15 smithi073 ceph-mon[36714]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-31T14:47:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:15 smithi073 ceph-mon[36714]: osdmap e85: 12 total, 12 up, 12 in 2024-07-31T14:47:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:15 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2887091066' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-07-31T14:47:15.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:15 smithi067 ceph-mon[32750]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-31T14:47:15.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:15 smithi067 ceph-mon[32750]: osdmap e85: 12 total, 12 up, 12 in 2024-07-31T14:47:15.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:15 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2887091066' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-07-31T14:47:15.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:15 smithi077 ceph-mon[37327]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-31T14:47:15.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:15 smithi077 ceph-mon[37327]: osdmap e85: 12 total, 12 up, 12 in 2024-07-31T14:47:15.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:15 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2887091066' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-07-31T14:47:15.717 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-07-31T14:47:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:16.349 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:16 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2887091066' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-07-31T14:47:16.349 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:16 smithi067 ceph-mon[32750]: osdmap e86: 12 total, 12 up, 12 in 2024-07-31T14:47:16.349 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:16 smithi067 ceph-mon[32750]: pgmap v282: 129 pgs: 2 creating+activating, 12 unknown, 115 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-07-31T14:47:16.389 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:16 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2887091066' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-07-31T14:47:16.389 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:16 smithi077 ceph-mon[37327]: osdmap e86: 12 total, 12 up, 12 in 2024-07-31T14:47:16.389 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:16 smithi077 ceph-mon[37327]: pgmap v282: 129 pgs: 2 creating+activating, 12 unknown, 115 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-07-31T14:47:16.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:16 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2887091066' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-07-31T14:47:16.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:16 smithi073 ceph-mon[36714]: osdmap e86: 12 total, 12 up, 12 in 2024-07-31T14:47:16.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:16 smithi073 ceph-mon[36714]: pgmap v282: 129 pgs: 2 creating+activating, 12 unknown, 115 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-07-31T14:47:17.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.677+0000 7f032da51640 1 -- 172.21.15.67:0/3398178884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03281019e0 msgr2=0x7f0328101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:17.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.677+0000 7f032da51640 1 --2- 172.21.15.67:0/3398178884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03281019e0 0x7f0328101df0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0320009900 tx=0x7f032002a410 comp rx=0 tx=0).stop 2024-07-31T14:47:17.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.677+0000 7f032da51640 1 -- 172.21.15.67:0/3398178884 shutdown_connections 2024-07-31T14:47:17.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.677+0000 7f032da51640 1 --2- 172.21.15.67:0/3398178884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0328106610 0x7f0328110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:17.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.678+0000 7f032da51640 1 --2- 172.21.15.67:0/3398178884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0328105cc0 0x7f03281060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:17.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.678+0000 7f032da51640 1 --2- 172.21.15.67:0/3398178884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03281019e0 0x7f0328101df0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:17.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.678+0000 7f032da51640 1 -- 172.21.15.67:0/3398178884 >> 172.21.15.67:0/3398178884 conn(0x7f0328077690 msgr2=0x7f0328077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:17.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.678+0000 7f032da51640 1 -- 172.21.15.67:0/3398178884 shutdown_connections 2024-07-31T14:47:17.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.678+0000 7f032da51640 1 -- 172.21.15.67:0/3398178884 wait complete. 2024-07-31T14:47:17.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.678+0000 7f032da51640 1 Processor -- start 2024-07-31T14:47:17.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.679+0000 7f032da51640 1 -- start start 2024-07-31T14:47:17.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.679+0000 7f032da51640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03281019e0 0x7f0328199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:17.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.679+0000 7f032da51640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0328105cc0 0x7f032819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:17.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.679+0000 7f032da51640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0328106610 0x7f032819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:17.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.679+0000 7f032da51640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f03281134c0 con 0x7f0328105cc0 2024-07-31T14:47:17.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.679+0000 7f032da51640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0328113340 con 0x7f03281019e0 2024-07-31T14:47:17.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.679+0000 7f032da51640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0328113640 con 0x7f0328106610 2024-07-31T14:47:17.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.679+0000 7f03267fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0328105cc0 0x7f032819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:17.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.679+0000 7f03267fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0328105cc0 0x7f032819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52260/0 (socket says 172.21.15.67:52260) 2024-07-31T14:47:17.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.679+0000 7f03267fc640 1 -- 172.21.15.67:0/1022923487 learned_addr learned my addr 172.21.15.67:0/1022923487 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:17.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.679+0000 7f0326ffd640 1 --2- 172.21.15.67:0/1022923487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03281019e0 0x7f0328199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:17.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.679+0000 7f03277fe640 1 --2- 172.21.15.67:0/1022923487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0328106610 0x7f032819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:17.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.680+0000 7f03267fc640 1 -- 172.21.15.67:0/1022923487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0328106610 msgr2=0x7f032819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:17.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.680+0000 7f03267fc640 1 --2- 172.21.15.67:0/1022923487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0328106610 0x7f032819f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:17.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.680+0000 7f03267fc640 1 -- 172.21.15.67:0/1022923487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03281019e0 msgr2=0x7f0328199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:17.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.680+0000 7f03267fc640 1 --2- 172.21.15.67:0/1022923487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03281019e0 0x7f0328199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:17.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.680+0000 7f03267fc640 1 -- 172.21.15.67:0/1022923487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032819ff60 con 0x7f0328105cc0 2024-07-31T14:47:17.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.680+0000 7f03267fc640 1 --2- 172.21.15.67:0/1022923487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0328105cc0 0x7f032819a590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f031000d950 tx=0x7f031000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:17.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.680+0000 7f032ca4f640 1 -- 172.21.15.67:0/1022923487 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310017070 con 0x7f0328105cc0 2024-07-31T14:47:17.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.680+0000 7f032da51640 1 -- 172.21.15.67:0/1022923487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03281a0250 con 0x7f0328105cc0 2024-07-31T14:47:17.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.680+0000 7f032ca4f640 1 -- 172.21.15.67:0/1022923487 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0310009c30 con 0x7f0328105cc0 2024-07-31T14:47:17.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.680+0000 7f032ca4f640 1 -- 172.21.15.67:0/1022923487 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03100047a0 con 0x7f0328105cc0 2024-07-31T14:47:17.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.680+0000 7f032da51640 1 -- 172.21.15.67:0/1022923487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03281a69b0 con 0x7f0328105cc0 2024-07-31T14:47:17.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.681+0000 7f032da51640 1 -- 172.21.15.67:0/1022923487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f032819b120 con 0x7f0328105cc0 2024-07-31T14:47:17.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.683+0000 7f032ca4f640 1 -- 172.21.15.67:0/1022923487 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f0310004020 con 0x7f0328105cc0 2024-07-31T14:47:17.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.683+0000 7f032ca4f640 1 --2- 172.21.15.67:0/1022923487 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f03140b3c20 0x7f03140b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:17.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.684+0000 7f0326ffd640 1 --2- 172.21.15.67:0/1022923487 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f03140b3c20 0x7f03140b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:17.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.684+0000 7f032ca4f640 1 -- 172.21.15.67:0/1022923487 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8985+0+0 (secure 0 0 0) 0x7f03100e3f10 con 0x7f0328105cc0 2024-07-31T14:47:17.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.684+0000 7f0326ffd640 1 --2- 172.21.15.67:0/1022923487 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f03140b3c20 0x7f03140b6130 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0320031440 tx=0x7f0320009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:17.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.687+0000 7f032ca4f640 1 -- 172.21.15.67:0/1022923487 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f0310020020 con 0x7f0328105cc0 2024-07-31T14:47:17.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:17.925+0000 7f032da51640 1 -- 172.21.15.67:0/1022923487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f032810b590 con 0x7f0328105cc0 2024-07-31T14:47:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:18.430+0000 7f032ca4f640 1 -- 172.21.15.67:0/1022923487 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f03100ab270 con 0x7f0328105cc0 2024-07-31T14:47:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:added data pool 4 to fsmap 2024-07-31T14:47:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:18.432+0000 7f032da51640 1 -- 172.21.15.67:0/1022923487 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f03140b3c20 msgr2=0x7f03140b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:18.432+0000 7f032da51640 1 --2- 172.21.15.67:0/1022923487 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f03140b3c20 0x7f03140b6130 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0320031440 tx=0x7f0320009560 comp rx=0 tx=0).stop 2024-07-31T14:47:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:18.432+0000 7f032da51640 1 -- 172.21.15.67:0/1022923487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0328105cc0 msgr2=0x7f032819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:18.432+0000 7f032da51640 1 --2- 172.21.15.67:0/1022923487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0328105cc0 0x7f032819a590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f031000d950 tx=0x7f031000dd50 comp rx=0 tx=0).stop 2024-07-31T14:47:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:18.432+0000 7f032da51640 1 -- 172.21.15.67:0/1022923487 shutdown_connections 2024-07-31T14:47:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:18.432+0000 7f032da51640 1 --2- 172.21.15.67:0/1022923487 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f03140b3c20 0x7f03140b6130 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:18.433+0000 7f032da51640 1 --2- 172.21.15.67:0/1022923487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0328106610 0x7f032819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:18.433+0000 7f032da51640 1 --2- 172.21.15.67:0/1022923487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0328105cc0 0x7f032819a590 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:18.433+0000 7f032da51640 1 --2- 172.21.15.67:0/1022923487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03281019e0 0x7f0328199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:18.433+0000 7f032da51640 1 -- 172.21.15.67:0/1022923487 >> 172.21.15.67:0/1022923487 conn(0x7f0328077690 msgr2=0x7f03281070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:18.433+0000 7f032da51640 1 -- 172.21.15.67:0/1022923487 shutdown_connections 2024-07-31T14:47:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:18.433+0000 7f032da51640 1 -- 172.21.15.67:0/1022923487 wait complete. 2024-07-31T14:47:18.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:18 smithi077 ceph-mon[37327]: pgmap v283: 129 pgs: 2 creating+activating, 127 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:18 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1022923487' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-07-31T14:47:18.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:18 smithi067 ceph-mon[32750]: pgmap v283: 129 pgs: 2 creating+activating, 127 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:18.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:18 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1022923487' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-07-31T14:47:18.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:18 smithi073 ceph-mon[36714]: pgmap v283: 129 pgs: 2 creating+activating, 127 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:18.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:18 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1022923487' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-07-31T14:47:19.133 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:47:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:19.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:19 smithi067 ceph-mon[32750]: osdmap e87: 12 total, 12 up, 12 in 2024-07-31T14:47:19.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:19 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1022923487' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-07-31T14:47:19.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:19 smithi067 ceph-mon[32750]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:47:19.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-07-31T14:47:19.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-07-31T14:47:19.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-07-31T14:47:19.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-07-31T14:47:19.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-07-31T14:47:19.700 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-07-31T14:47:19.700 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-07-31T14:47:19.700 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]: dispatch 2024-07-31T14:47:19.700 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-07-31T14:47:19.700 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-07-31T14:47:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:19 smithi077 ceph-mon[37327]: osdmap e87: 12 total, 12 up, 12 in 2024-07-31T14:47:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:19 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1022923487' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-07-31T14:47:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:19 smithi077 ceph-mon[37327]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:47:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-07-31T14:47:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-07-31T14:47:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-07-31T14:47:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-07-31T14:47:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-07-31T14:47:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-07-31T14:47:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-07-31T14:47:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]: dispatch 2024-07-31T14:47:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-07-31T14:47:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-07-31T14:47:19.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:19 smithi073 ceph-mon[36714]: osdmap e87: 12 total, 12 up, 12 in 2024-07-31T14:47:19.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:19 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1022923487' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-07-31T14:47:19.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:19 smithi073 ceph-mon[36714]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:47:19.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-07-31T14:47:19.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-07-31T14:47:19.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-07-31T14:47:19.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-07-31T14:47:19.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-07-31T14:47:19.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-07-31T14:47:19.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-07-31T14:47:19.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]: dispatch 2024-07-31T14:47:19.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-07-31T14:47:19.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-07-31T14:47:20.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:20 smithi067 ceph-mon[32750]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:20.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:20 smithi067 ceph-mon[32750]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-31T14:47:20.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:20 smithi067 ceph-mon[32750]: Cluster is now healthy 2024-07-31T14:47:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-07-31T14:47:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-07-31T14:47:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-07-31T14:47:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-07-31T14:47:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-07-31T14:47:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-07-31T14:47:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-07-31T14:47:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]': finished 2024-07-31T14:47:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-07-31T14:47:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:20 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-07-31T14:47:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:20 smithi067 ceph-mon[32750]: osdmap e88: 12 total, 12 up, 12 in 2024-07-31T14:47:20.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:20 smithi077 ceph-mon[37327]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:20.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:20 smithi077 ceph-mon[37327]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-31T14:47:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:20 smithi077 ceph-mon[37327]: Cluster is now healthy 2024-07-31T14:47:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-07-31T14:47:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-07-31T14:47:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-07-31T14:47:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-07-31T14:47:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-07-31T14:47:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-07-31T14:47:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-07-31T14:47:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]': finished 2024-07-31T14:47:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-07-31T14:47:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:20 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-07-31T14:47:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:20 smithi077 ceph-mon[37327]: osdmap e88: 12 total, 12 up, 12 in 2024-07-31T14:47:20.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:20 smithi073 ceph-mon[36714]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:20.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:20 smithi073 ceph-mon[36714]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-31T14:47:20.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:20 smithi073 ceph-mon[36714]: Cluster is now healthy 2024-07-31T14:47:20.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-07-31T14:47:20.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-07-31T14:47:20.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-07-31T14:47:20.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-07-31T14:47:20.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-07-31T14:47:20.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-07-31T14:47:20.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-07-31T14:47:20.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]': finished 2024-07-31T14:47:20.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-07-31T14:47:20.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:20 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-07-31T14:47:20.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:20 smithi073 ceph-mon[36714]: osdmap e88: 12 total, 12 up, 12 in 2024-07-31T14:47:21.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.066+0000 7fc204d1e640 1 -- 172.21.15.67:0/1066845653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc200102490 msgr2=0x7fc2001028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:21.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.066+0000 7fc204d1e640 1 --2- 172.21.15.67:0/1066845653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc200102490 0x7fc2001028a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009900 tx=0x7fc1f002a3f0 comp rx=0 tx=0).stop 2024-07-31T14:47:21.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.066+0000 7fc204d1e640 1 -- 172.21.15.67:0/1066845653 shutdown_connections 2024-07-31T14:47:21.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.066+0000 7fc204d1e640 1 --2- 172.21.15.67:0/1066845653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc200108720 0x7fc20010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:21.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.066+0000 7fc204d1e640 1 --2- 172.21.15.67:0/1066845653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc200107dd0 0x7fc2001081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:21.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.066+0000 7fc204d1e640 1 --2- 172.21.15.67:0/1066845653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc200102490 0x7fc2001028a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:21.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.066+0000 7fc204d1e640 1 -- 172.21.15.67:0/1066845653 >> 172.21.15.67:0/1066845653 conn(0x7fc200079160 msgr2=0x7fc200079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:21.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.066+0000 7fc204d1e640 1 -- 172.21.15.67:0/1066845653 shutdown_connections 2024-07-31T14:47:21.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.066+0000 7fc204d1e640 1 -- 172.21.15.67:0/1066845653 wait complete. 2024-07-31T14:47:21.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.067+0000 7fc204d1e640 1 Processor -- start 2024-07-31T14:47:21.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.067+0000 7fc204d1e640 1 -- start start 2024-07-31T14:47:21.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc204d1e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc200102490 0x7fc20019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:21.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc204d1e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc200107dd0 0x7fc20019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:21.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc204d1e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc200108720 0x7fc20019b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:21.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc204d1e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc200112380 con 0x7fc200107dd0 2024-07-31T14:47:21.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc204d1e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc200112200 con 0x7fc200108720 2024-07-31T14:47:21.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc204d1e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc200112500 con 0x7fc200102490 2024-07-31T14:47:21.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1fdd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc200107dd0 0x7fc20019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:21.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1fdd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc200107dd0 0x7fc20019b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50100/0 (socket says 172.21.15.67:50100) 2024-07-31T14:47:21.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1fdd9b640 1 -- 172.21.15.67:0/2363562707 learned_addr learned my addr 172.21.15.67:0/2363562707 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:21.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1fe59c640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc200102490 0x7fc20019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:21.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1fed9d640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc200108720 0x7fc20019b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:21.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1fdd9b640 1 -- 172.21.15.67:0/2363562707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc200102490 msgr2=0x7fc20019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:21.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1fdd9b640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc200102490 0x7fc20019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:21.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1fdd9b640 1 -- 172.21.15.67:0/2363562707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc200108720 msgr2=0x7fc20019b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:21.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1fdd9b640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc200108720 0x7fc20019b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:21.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1fdd9b640 1 -- 172.21.15.67:0/2363562707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2001a5e50 con 0x7fc200107dd0 2024-07-31T14:47:21.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1fed9d640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc200108720 0x7fc20019b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:47:21.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1fdd9b640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc200107dd0 0x7fc20019b190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400c970 tx=0x7fc1f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:21.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1ef7fe640 1 -- 172.21.15.67:0/2363562707 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4014070 con 0x7fc200107dd0 2024-07-31T14:47:21.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1ef7fe640 1 -- 172.21.15.67:0/2363562707 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f4004440 con 0x7fc200107dd0 2024-07-31T14:47:21.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc204d1e640 1 -- 172.21.15.67:0/2363562707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2001a6140 con 0x7fc200107dd0 2024-07-31T14:47:21.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.068+0000 7fc1ef7fe640 1 -- 172.21.15.67:0/2363562707 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4003e50 con 0x7fc200107dd0 2024-07-31T14:47:21.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.069+0000 7fc204d1e640 1 -- 172.21.15.67:0/2363562707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2001a66d0 con 0x7fc200107dd0 2024-07-31T14:47:21.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.069+0000 7fc204d1e640 1 -- 172.21.15.67:0/2363562707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2001028a0 con 0x7fc200107dd0 2024-07-31T14:47:21.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.071+0000 7fc1ef7fe640 1 -- 172.21.15.67:0/2363562707 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fc1f4017070 con 0x7fc200107dd0 2024-07-31T14:47:21.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.072+0000 7fc1ef7fe640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc1e80b3e70 0x7fc1e80b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:21.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.072+0000 7fc1fe59c640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc1e80b3e70 0x7fc1e80b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:21.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.072+0000 7fc1ef7fe640 1 -- 172.21.15.67:0/2363562707 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc1f40e4380 con 0x7fc200107dd0 2024-07-31T14:47:21.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.073+0000 7fc1fe59c640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc1e80b3e70 0x7fc1e80b6380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0000c00 tx=0x7fc1f0031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:21.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.076+0000 7fc1ef7fe640 1 -- 172.21.15.67:0/2363562707 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc1f401e030 con 0x7fc200107dd0 2024-07-31T14:47:21.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.311+0000 7fc204d1e640 1 -- 172.21.15.67:0/2363562707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc20006aca0 con 0x7fc200107dd0 2024-07-31T14:47:21.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.312+0000 7fc1ef7fe640 1 -- 172.21.15.67:0/2363562707 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11827 (secure 0 0 0) 0x7fc1f40035f0 con 0x7fc200107dd0 2024-07-31T14:47:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:47:21.314 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"btime":"2024-07-31T14:47:18:423702+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14790,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24634,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24650,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24655,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:47:17.927081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-07-31T14:47:21.314 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-07-31T14:47:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.314+0000 7fc204d1e640 1 -- 172.21.15.67:0/2363562707 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc1e80b3e70 msgr2=0x7fc1e80b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.314+0000 7fc204d1e640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc1e80b3e70 0x7fc1e80b6380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0000c00 tx=0x7fc1f0031400 comp rx=0 tx=0).stop 2024-07-31T14:47:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.314+0000 7fc204d1e640 1 -- 172.21.15.67:0/2363562707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc200107dd0 msgr2=0x7fc20019b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.314+0000 7fc204d1e640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc200107dd0 0x7fc20019b190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400c970 tx=0x7fc1f400ce30 comp rx=0 tx=0).stop 2024-07-31T14:47:21.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.314+0000 7fc204d1e640 1 -- 172.21.15.67:0/2363562707 shutdown_connections 2024-07-31T14:47:21.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.314+0000 7fc204d1e640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc1e80b3e70 0x7fc1e80b6380 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:21.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.314+0000 7fc204d1e640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc200108720 0x7fc20019b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:21.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.314+0000 7fc204d1e640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc200107dd0 0x7fc20019b190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:21.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.314+0000 7fc204d1e640 1 --2- 172.21.15.67:0/2363562707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc200102490 0x7fc20019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:21.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.314+0000 7fc204d1e640 1 -- 172.21.15.67:0/2363562707 >> 172.21.15.67:0/2363562707 conn(0x7fc200079160 msgr2=0x7fc200100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:21.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.314+0000 7fc204d1e640 1 -- 172.21.15.67:0/2363562707 shutdown_connections 2024-07-31T14:47:21.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:21.314+0000 7fc204d1e640 1 -- 172.21.15.67:0/2363562707 wait complete. 2024-07-31T14:47:21.686 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:21 smithi067 ceph-mon[32750]: osdmap e89: 12 total, 12 up, 12 in 2024-07-31T14:47:21.686 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:21 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2363562707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:47:21.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:21 smithi077 ceph-mon[37327]: osdmap e89: 12 total, 12 up, 12 in 2024-07-31T14:47:21.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:21 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2363562707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:47:21.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:21 smithi073 ceph-mon[36714]: osdmap e89: 12 total, 12 up, 12 in 2024-07-31T14:47:21.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:21 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2363562707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:47:22.034 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-07-31T14:47:22.299 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:22.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:22 smithi077 ceph-mon[37327]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:22.758 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:22 smithi067 ceph-mon[32750]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:22.809 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:22 smithi073 ceph-mon[36714]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:23.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.984+0000 7f6556b84640 1 -- 172.21.15.67:0/3623637644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6550107de0 msgr2=0x7f65501081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:23.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.984+0000 7f6556b84640 1 --2- 172.21.15.67:0/3623637644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6550107de0 0x7f65501081f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f654402a410 comp rx=0 tx=0).stop 2024-07-31T14:47:23.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.984+0000 7f6556b84640 1 -- 172.21.15.67:0/3623637644 shutdown_connections 2024-07-31T14:47:23.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.984+0000 7f6556b84640 1 --2- 172.21.15.67:0/3623637644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6550108730 0x7f655010ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:23.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.984+0000 7f6556b84640 1 --2- 172.21.15.67:0/3623637644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6550107de0 0x7f65501081f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:23.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.984+0000 7f6556b84640 1 --2- 172.21.15.67:0/3623637644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65501024a0 0x7f65501028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:23.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.984+0000 7f6556b84640 1 -- 172.21.15.67:0/3623637644 >> 172.21.15.67:0/3623637644 conn(0x7f6550079140 msgr2=0x7f6550079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.984+0000 7f6556b84640 1 -- 172.21.15.67:0/3623637644 shutdown_connections 2024-07-31T14:47:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.984+0000 7f6556b84640 1 -- 172.21.15.67:0/3623637644 wait complete. 2024-07-31T14:47:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.985+0000 7f6556b84640 1 Processor -- start 2024-07-31T14:47:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.985+0000 7f6556b84640 1 -- start start 2024-07-31T14:47:23.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.985+0000 7f6556b84640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65501024a0 0x7f655019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:23.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.985+0000 7f6556b84640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6550107de0 0x7f655019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:23.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.985+0000 7f6556b84640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6550108730 0x7f655019b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:23.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.985+0000 7f6556b84640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f65501123a0 con 0x7f65501024a0 2024-07-31T14:47:23.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.985+0000 7f6556b84640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6550112220 con 0x7f6550107de0 2024-07-31T14:47:23.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.985+0000 7f6556b84640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6550112520 con 0x7f6550108730 2024-07-31T14:47:23.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.985+0000 7f6554920640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65501024a0 0x7f655019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:23.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.985+0000 7f6554920640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65501024a0 0x7f655019ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50122/0 (socket says 172.21.15.67:50122) 2024-07-31T14:47:23.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.985+0000 7f6554920640 1 -- 172.21.15.67:0/527969918 learned_addr learned my addr 172.21.15.67:0/527969918 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:23.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.985+0000 7f654ffff640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6550107de0 0x7f655019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:23.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.986+0000 7f6555121640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6550108730 0x7f655019b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:23.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.986+0000 7f6554920640 1 -- 172.21.15.67:0/527969918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6550108730 msgr2=0x7f655019b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:23.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.986+0000 7f6554920640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6550108730 0x7f655019b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:23.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.986+0000 7f6554920640 1 -- 172.21.15.67:0/527969918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6550107de0 msgr2=0x7f655019b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:23.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.986+0000 7f6554920640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6550107de0 0x7f655019b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:23.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.986+0000 7f6554920640 1 -- 172.21.15.67:0/527969918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65501a5fc0 con 0x7f65501024a0 2024-07-31T14:47:23.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.986+0000 7f654ffff640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6550107de0 0x7f655019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:47:23.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.986+0000 7f6554920640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65501024a0 0x7f655019ad30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f654000e9f0 tx=0x7f654000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:23.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.986+0000 7f654dffb640 1 -- 172.21.15.67:0/527969918 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654000cd80 con 0x7f65501024a0 2024-07-31T14:47:23.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.986+0000 7f654dffb640 1 -- 172.21.15.67:0/527969918 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65400108e0 con 0x7f65501024a0 2024-07-31T14:47:23.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.986+0000 7f6556b84640 1 -- 172.21.15.67:0/527969918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65501a62b0 con 0x7f65501024a0 2024-07-31T14:47:23.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.986+0000 7f654dffb640 1 -- 172.21.15.67:0/527969918 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540018980 con 0x7f65501024a0 2024-07-31T14:47:23.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.986+0000 7f6556b84640 1 -- 172.21.15.67:0/527969918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65501a6810 con 0x7f65501024a0 2024-07-31T14:47:23.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.987+0000 7f6556b84640 1 -- 172.21.15.67:0/527969918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6550103720 con 0x7f65501024a0 2024-07-31T14:47:23.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.990+0000 7f654dffb640 1 -- 172.21.15.67:0/527969918 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f6540010430 con 0x7f65501024a0 2024-07-31T14:47:23.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.991+0000 7f654dffb640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65380b3cd0 0x7f65380b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:23.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.991+0000 7f654ffff640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65380b3cd0 0x7f65380b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:23.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.991+0000 7f654dffb640 1 -- 172.21.15.67:0/527969918 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f65400e48a0 con 0x7f65501024a0 2024-07-31T14:47:23.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.991+0000 7f654ffff640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65380b3cd0 0x7f65380b61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6544009230 tx=0x7f6544031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:23.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:23.994+0000 7f654dffb640 1 -- 172.21.15.67:0/527969918 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f65400aba60 con 0x7f65501024a0 2024-07-31T14:47:24.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:24.192+0000 7f6556b84640 1 -- 172.21.15.67:0/527969918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f655006b9c0 con 0x7f65501024a0 2024-07-31T14:47:24.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:24.193+0000 7f654dffb640 1 -- 172.21.15.67:0/527969918 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24874 (secure 0 0 0) 0x7f65400ab360 con 0x7f65501024a0 2024-07-31T14:47:24.195 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:47:24.195 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T14:47:20.440064+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T14:47:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:24.196+0000 7f6556b84640 1 -- 172.21.15.67:0/527969918 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65380b3cd0 msgr2=0x7f65380b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:24.196+0000 7f6556b84640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65380b3cd0 0x7f65380b61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6544009230 tx=0x7f6544031400 comp rx=0 tx=0).stop 2024-07-31T14:47:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:24.196+0000 7f6556b84640 1 -- 172.21.15.67:0/527969918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65501024a0 msgr2=0x7f655019ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:24.196+0000 7f6556b84640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65501024a0 0x7f655019ad30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f654000e9f0 tx=0x7f654000eeb0 comp rx=0 tx=0).stop 2024-07-31T14:47:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:24.196+0000 7f6556b84640 1 -- 172.21.15.67:0/527969918 shutdown_connections 2024-07-31T14:47:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:24.196+0000 7f6556b84640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65380b3cd0 0x7f65380b61e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:24.196+0000 7f6556b84640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6550108730 0x7f655019b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:24.196+0000 7f6556b84640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6550107de0 0x7f655019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:24.196+0000 7f6556b84640 1 --2- 172.21.15.67:0/527969918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65501024a0 0x7f655019ad30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:24.196+0000 7f6556b84640 1 -- 172.21.15.67:0/527969918 >> 172.21.15.67:0/527969918 conn(0x7f6550079140 msgr2=0x7f6550105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:24.196+0000 7f6556b84640 1 -- 172.21.15.67:0/527969918 shutdown_connections 2024-07-31T14:47:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:24.196+0000 7f6556b84640 1 -- 172.21.15.67:0/527969918 wait complete. 2024-07-31T14:47:24.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:24 smithi077 ceph-mon[37327]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:24.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:24 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/527969918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:24.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:24 smithi067 ceph-mon[32750]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:24.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:24 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/527969918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:24.856 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-07-31T14:47:24.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:24 smithi073 ceph-mon[36714]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:24.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:24 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/527969918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:25.122 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:26.614 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:26 smithi067 ceph-mon[32750]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:26 smithi077 ceph-mon[37327]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:26.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.874+0000 7fe59e17f640 1 -- 172.21.15.67:0/1316220804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe598109870 msgr2=0x7fe5981100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:26.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.874+0000 7fe59e17f640 1 --2- 172.21.15.67:0/1316220804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe598109870 0x7fe5981100a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe590009900 tx=0x7fe59002a410 comp rx=0 tx=0).stop 2024-07-31T14:47:26.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.874+0000 7fe59e17f640 1 -- 172.21.15.67:0/1316220804 shutdown_connections 2024-07-31T14:47:26.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.874+0000 7fe59e17f640 1 --2- 172.21.15.67:0/1316220804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe598109870 0x7fe5981100a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:26.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.874+0000 7fe59e17f640 1 --2- 172.21.15.67:0/1316220804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe598108f20 0x7fe598109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:26.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.874+0000 7fe59e17f640 1 --2- 172.21.15.67:0/1316220804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe598101ad0 0x7fe598101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:26.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.874+0000 7fe59e17f640 1 -- 172.21.15.67:0/1316220804 >> 172.21.15.67:0/1316220804 conn(0x7fe598079870 msgr2=0x7fe598079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:26.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.874+0000 7fe59e17f640 1 -- 172.21.15.67:0/1316220804 shutdown_connections 2024-07-31T14:47:26.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.874+0000 7fe59e17f640 1 -- 172.21.15.67:0/1316220804 wait complete. 2024-07-31T14:47:26.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.874+0000 7fe59e17f640 1 Processor -- start 2024-07-31T14:47:26.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe59e17f640 1 -- start start 2024-07-31T14:47:26.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe59e17f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe598101ad0 0x7fe598102a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:26.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe59e17f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe598108f20 0x7fe598102fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe59e17f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe598109870 0x7fe5981034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe59e17f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe598113540 con 0x7fe598108f20 2024-07-31T14:47:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe59e17f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981133c0 con 0x7fe598109870 2024-07-31T14:47:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe59e17f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981136c0 con 0x7fe598101ad0 2024-07-31T14:47:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe596ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe598108f20 0x7fe598102fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe596ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe598108f20 0x7fe598102fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50142/0 (socket says 172.21.15.67:50142) 2024-07-31T14:47:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe596ffd640 1 -- 172.21.15.67:0/4237420195 learned_addr learned my addr 172.21.15.67:0/4237420195 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe5977fe640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe598101ad0 0x7fe598102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe597fff640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe598109870 0x7fe5981034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe596ffd640 1 -- 172.21.15.67:0/4237420195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe598101ad0 msgr2=0x7fe598102a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:26.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe596ffd640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe598101ad0 0x7fe598102a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:26.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe596ffd640 1 -- 172.21.15.67:0/4237420195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe598109870 msgr2=0x7fe5981034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:26.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe596ffd640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe598109870 0x7fe5981034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:26.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe596ffd640 1 -- 172.21.15.67:0/4237420195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5981a8130 con 0x7fe598108f20 2024-07-31T14:47:26.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.875+0000 7fe597fff640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe598109870 0x7fe5981034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:47:26.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.876+0000 7fe596ffd640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe598108f20 0x7fe598102fb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00d740 tx=0x7fe58c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:26.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.876+0000 7fe594ff9640 1 -- 172.21.15.67:0/4237420195 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c017070 con 0x7fe598108f20 2024-07-31T14:47:26.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.876+0000 7fe594ff9640 1 -- 172.21.15.67:0/4237420195 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe58c00de50 con 0x7fe598108f20 2024-07-31T14:47:26.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.876+0000 7fe59e17f640 1 -- 172.21.15.67:0/4237420195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5981a8420 con 0x7fe598108f20 2024-07-31T14:47:26.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.876+0000 7fe594ff9640 1 -- 172.21.15.67:0/4237420195 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c00f620 con 0x7fe598108f20 2024-07-31T14:47:26.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.876+0000 7fe59e17f640 1 -- 172.21.15.67:0/4237420195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5981a8960 con 0x7fe598108f20 2024-07-31T14:47:26.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.877+0000 7fe59e17f640 1 -- 172.21.15.67:0/4237420195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe598101ee0 con 0x7fe598108f20 2024-07-31T14:47:26.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.879+0000 7fe594ff9640 1 -- 172.21.15.67:0/4237420195 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fe58c01a070 con 0x7fe598108f20 2024-07-31T14:47:26.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.880+0000 7fe594ff9640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe5800b3c20 0x7fe5800b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:26.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.880+0000 7fe5977fe640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe5800b3c20 0x7fe5800b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:26.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.880+0000 7fe594ff9640 1 -- 172.21.15.67:0/4237420195 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fe58c0e3aa0 con 0x7fe598108f20 2024-07-31T14:47:26.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.880+0000 7fe5977fe640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe5800b3c20 0x7fe5800b6130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe5981044c0 tx=0x7fe588009680 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:26.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:26.883+0000 7fe594ff9640 1 -- 172.21.15.67:0/4237420195 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe58c0aacc0 con 0x7fe598108f20 2024-07-31T14:47:26.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:26 smithi073 ceph-mon[36714]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:27.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:27.081+0000 7fe59e17f640 1 -- 172.21.15.67:0/4237420195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe59802e0c0 con 0x7fe598108f20 2024-07-31T14:47:27.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:27.083+0000 7fe594ff9640 1 -- 172.21.15.67:0/4237420195 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24874 (secure 0 0 0) 0x7fe58c0aa5c0 con 0x7fe598108f20 2024-07-31T14:47:27.085 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:47:27.085 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T14:47:20.440064+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T14:47:27.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:27.085+0000 7fe59e17f640 1 -- 172.21.15.67:0/4237420195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe5800b3c20 msgr2=0x7fe5800b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:27.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:27.085+0000 7fe59e17f640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe5800b3c20 0x7fe5800b6130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe5981044c0 tx=0x7fe588009680 comp rx=0 tx=0).stop 2024-07-31T14:47:27.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:27.085+0000 7fe59e17f640 1 -- 172.21.15.67:0/4237420195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe598108f20 msgr2=0x7fe598102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:27.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:27.085+0000 7fe59e17f640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe598108f20 0x7fe598102fb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00d740 tx=0x7fe58c00dc00 comp rx=0 tx=0).stop 2024-07-31T14:47:27.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:27.085+0000 7fe59e17f640 1 -- 172.21.15.67:0/4237420195 shutdown_connections 2024-07-31T14:47:27.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:27.085+0000 7fe59e17f640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe5800b3c20 0x7fe5800b6130 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:27.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:27.085+0000 7fe59e17f640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe598109870 0x7fe5981034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:27.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:27.085+0000 7fe59e17f640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe598108f20 0x7fe598102fb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:27.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:27.085+0000 7fe59e17f640 1 --2- 172.21.15.67:0/4237420195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe598101ad0 0x7fe598102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:27.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:27.085+0000 7fe59e17f640 1 -- 172.21.15.67:0/4237420195 >> 172.21.15.67:0/4237420195 conn(0x7fe598079870 msgr2=0x7fe598106f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:27.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:27.086+0000 7fe59e17f640 1 -- 172.21.15.67:0/4237420195 shutdown_connections 2024-07-31T14:47:27.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:27.086+0000 7fe59e17f640 1 -- 172.21.15.67:0/4237420195 wait complete. 2024-07-31T14:47:27.733 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:27 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/4237420195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:27.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:27 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/4237420195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:27.780 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:47:27.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:27 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/4237420195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:28.046 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:28.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:28 smithi077 ceph-mon[37327]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:28.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:28 smithi067 ceph-mon[32750]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:28.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:28 smithi073 ceph-mon[36714]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:29.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.874+0000 7fbe940bb640 1 -- 172.21.15.67:0/3404556280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe8c107e10 msgr2=0x7fbe8c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:29.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.874+0000 7fbe940bb640 1 --2- 172.21.15.67:0/3404556280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe8c107e10 0x7fbe8c108220 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009900 tx=0x7fbe8802a410 comp rx=0 tx=0).stop 2024-07-31T14:47:29.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.874+0000 7fbe940bb640 1 -- 172.21.15.67:0/3404556280 shutdown_connections 2024-07-31T14:47:29.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.874+0000 7fbe940bb640 1 --2- 172.21.15.67:0/3404556280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe8c108760 0x7fbe8c10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:29.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.874+0000 7fbe940bb640 1 --2- 172.21.15.67:0/3404556280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe8c107e10 0x7fbe8c108220 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:29.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.874+0000 7fbe940bb640 1 --2- 172.21.15.67:0/3404556280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe8c1024d0 0x7fbe8c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:29.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.874+0000 7fbe940bb640 1 -- 172.21.15.67:0/3404556280 >> 172.21.15.67:0/3404556280 conn(0x7fbe8c079190 msgr2=0x7fbe8c0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:29.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.874+0000 7fbe940bb640 1 -- 172.21.15.67:0/3404556280 shutdown_connections 2024-07-31T14:47:29.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.875+0000 7fbe940bb640 1 -- 172.21.15.67:0/3404556280 wait complete. 2024-07-31T14:47:29.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.875+0000 7fbe940bb640 1 Processor -- start 2024-07-31T14:47:29.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.875+0000 7fbe940bb640 1 -- start start 2024-07-31T14:47:29.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.875+0000 7fbe940bb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe8c1024d0 0x7fbe8c198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:29.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.875+0000 7fbe940bb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe8c107e10 0x7fbe8c199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:29.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.875+0000 7fbe940bb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe8c108760 0x7fbe8c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:29.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.875+0000 7fbe940bb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1123b0 con 0x7fbe8c1024d0 2024-07-31T14:47:29.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.875+0000 7fbe940bb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c112230 con 0x7fbe8c108760 2024-07-31T14:47:29.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.875+0000 7fbe940bb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c112530 con 0x7fbe8c107e10 2024-07-31T14:47:29.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe91e57640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe8c1024d0 0x7fbe8c198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:29.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe91e57640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe8c1024d0 0x7fbe8c198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50162/0 (socket says 172.21.15.67:50162) 2024-07-31T14:47:29.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe91e57640 1 -- 172.21.15.67:0/3465411110 learned_addr learned my addr 172.21.15.67:0/3465411110 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:29.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe91656640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe8c107e10 0x7fbe8c199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:29.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe92658640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe8c108760 0x7fbe8c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:29.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe91e57640 1 -- 172.21.15.67:0/3465411110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe8c107e10 msgr2=0x7fbe8c199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:29.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe91e57640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe8c107e10 0x7fbe8c199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:29.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe91e57640 1 -- 172.21.15.67:0/3465411110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe8c108760 msgr2=0x7fbe8c19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:29.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe91e57640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe8c108760 0x7fbe8c19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:29.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe91e57640 1 -- 172.21.15.67:0/3465411110 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c19ed10 con 0x7fbe8c1024d0 2024-07-31T14:47:29.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe92658640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe8c108760 0x7fbe8c19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:47:29.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe91e57640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe8c1024d0 0x7fbe8c198da0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000d770 tx=0x7fbe8000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:29.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe7effd640 1 -- 172.21.15.67:0/3465411110 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80017070 con 0x7fbe8c1024d0 2024-07-31T14:47:29.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe940bb640 1 -- 172.21.15.67:0/3465411110 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe8c19f000 con 0x7fbe8c1024d0 2024-07-31T14:47:29.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe7effd640 1 -- 172.21.15.67:0/3465411110 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe800054d0 con 0x7fbe8c1024d0 2024-07-31T14:47:29.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe7effd640 1 -- 172.21.15.67:0/3465411110 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80004c30 con 0x7fbe8c1024d0 2024-07-31T14:47:29.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.876+0000 7fbe940bb640 1 -- 172.21.15.67:0/3465411110 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe8c1a5760 con 0x7fbe8c1024d0 2024-07-31T14:47:29.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.878+0000 7fbe940bb640 1 -- 172.21.15.67:0/3465411110 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe8c103750 con 0x7fbe8c1024d0 2024-07-31T14:47:29.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.879+0000 7fbe7effd640 1 -- 172.21.15.67:0/3465411110 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fbe80004dd0 con 0x7fbe8c1024d0 2024-07-31T14:47:29.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.880+0000 7fbe7effd640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbe780b3c20 0x7fbe780b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:29.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.881+0000 7fbe91656640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbe780b3c20 0x7fbe780b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:29.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.881+0000 7fbe7effd640 1 -- 172.21.15.67:0/3465411110 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fbe800e3f30 con 0x7fbe8c1024d0 2024-07-31T14:47:29.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.881+0000 7fbe91656640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbe780b3c20 0x7fbe780b6130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbe88000c00 tx=0x7fbe88031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:29.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:29.884+0000 7fbe7effd640 1 -- 172.21.15.67:0/3465411110 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fbe800ab150 con 0x7fbe8c1024d0 2024-07-31T14:47:30.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:30.121+0000 7fbe940bb640 1 -- 172.21.15.67:0/3465411110 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe8c10a5b0 con 0x7fbe8c1024d0 2024-07-31T14:47:30.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:30.122+0000 7fbe7effd640 1 -- 172.21.15.67:0/3465411110 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11827 (secure 0 0 0) 0x7fbe800aaa50 con 0x7fbe8c1024d0 2024-07-31T14:47:30.123 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:47:30.123 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"btime":"2024-07-31T14:47:18:423702+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14790,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24634,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24650,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24655,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:47:17.927081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24671,"qdb_cluster":[24671]},"id":1}]} 2024-07-31T14:47:30.124 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-07-31T14:47:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:30.124+0000 7fbe940bb640 1 -- 172.21.15.67:0/3465411110 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbe780b3c20 msgr2=0x7fbe780b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:30.124+0000 7fbe940bb640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbe780b3c20 0x7fbe780b6130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbe88000c00 tx=0x7fbe88031400 comp rx=0 tx=0).stop 2024-07-31T14:47:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:30.124+0000 7fbe940bb640 1 -- 172.21.15.67:0/3465411110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe8c1024d0 msgr2=0x7fbe8c198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:30.124+0000 7fbe940bb640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe8c1024d0 0x7fbe8c198da0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000d770 tx=0x7fbe8000dc30 comp rx=0 tx=0).stop 2024-07-31T14:47:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:30.124+0000 7fbe940bb640 1 -- 172.21.15.67:0/3465411110 shutdown_connections 2024-07-31T14:47:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:30.124+0000 7fbe940bb640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbe780b3c20 0x7fbe780b6130 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:30.124+0000 7fbe940bb640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe8c108760 0x7fbe8c19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:30.124+0000 7fbe940bb640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe8c107e10 0x7fbe8c199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:30.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:30.124+0000 7fbe940bb640 1 --2- 172.21.15.67:0/3465411110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe8c1024d0 0x7fbe8c198da0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:30.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:30.124+0000 7fbe940bb640 1 -- 172.21.15.67:0/3465411110 >> 172.21.15.67:0/3465411110 conn(0x7fbe8c079190 msgr2=0x7fbe8c104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:30.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:30.124+0000 7fbe940bb640 1 -- 172.21.15.67:0/3465411110 shutdown_connections 2024-07-31T14:47:30.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:30.124+0000 7fbe940bb640 1 -- 172.21.15.67:0/3465411110 wait complete. 2024-07-31T14:47:30.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:30 smithi077 ceph-mon[37327]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:30.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:30 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3465411110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:47:30.802 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:30 smithi067 ceph-mon[32750]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:30.802 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:30 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3465411110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:47:30.832 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-07-31T14:47:30.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:30 smithi073 ceph-mon[36714]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:30.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:30 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3465411110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:47:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:32.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:32 smithi077 ceph-mon[37327]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:32.738 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:32 smithi067 ceph-mon[32750]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:32.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.742+0000 7fc22ff94640 1 -- 172.21.15.67:0/345671224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc228102490 msgr2=0x7fc2281028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:32.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.742+0000 7fc22ff94640 1 --2- 172.21.15.67:0/345671224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc228102490 0x7fc2281028a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc220009900 tx=0x7fc22002a410 comp rx=0 tx=0).stop 2024-07-31T14:47:32.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.742+0000 7fc22ff94640 1 -- 172.21.15.67:0/345671224 shutdown_connections 2024-07-31T14:47:32.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.742+0000 7fc22ff94640 1 --2- 172.21.15.67:0/345671224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc228108720 0x7fc22810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:32.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.742+0000 7fc22ff94640 1 --2- 172.21.15.67:0/345671224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc228107dd0 0x7fc2281081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:32.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.742+0000 7fc22ff94640 1 --2- 172.21.15.67:0/345671224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc228102490 0x7fc2281028a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:32.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.742+0000 7fc22ff94640 1 -- 172.21.15.67:0/345671224 >> 172.21.15.67:0/345671224 conn(0x7fc228079190 msgr2=0x7fc2280795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:32.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.742+0000 7fc22ff94640 1 -- 172.21.15.67:0/345671224 shutdown_connections 2024-07-31T14:47:32.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.742+0000 7fc22ff94640 1 -- 172.21.15.67:0/345671224 wait complete. 2024-07-31T14:47:32.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.743+0000 7fc22ff94640 1 Processor -- start 2024-07-31T14:47:32.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.743+0000 7fc22ff94640 1 -- start start 2024-07-31T14:47:32.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.743+0000 7fc22ff94640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc228102490 0x7fc228199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:32.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.743+0000 7fc22ff94640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc228107dd0 0x7fc22819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:32.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.743+0000 7fc22ff94640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc228108720 0x7fc22819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:32.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.743+0000 7fc22ff94640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc2281123f0 con 0x7fc228102490 2024-07-31T14:47:32.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.743+0000 7fc22ff94640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc228112270 con 0x7fc228108720 2024-07-31T14:47:32.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.743+0000 7fc22ff94640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc228112570 con 0x7fc228107dd0 2024-07-31T14:47:32.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.743+0000 7fc22dd30640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc228102490 0x7fc228199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:32.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.743+0000 7fc22dd30640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc228102490 0x7fc228199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50660/0 (socket says 172.21.15.67:50660) 2024-07-31T14:47:32.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.743+0000 7fc22dd30640 1 -- 172.21.15.67:0/576509150 learned_addr learned my addr 172.21.15.67:0/576509150 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:32.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.744+0000 7fc22d52f640 1 --2- 172.21.15.67:0/576509150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc228107dd0 0x7fc22819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:32.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.744+0000 7fc22e531640 1 --2- 172.21.15.67:0/576509150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc228108720 0x7fc22819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:32.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.744+0000 7fc22dd30640 1 -- 172.21.15.67:0/576509150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc228107dd0 msgr2=0x7fc22819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:32.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.744+0000 7fc22dd30640 1 --2- 172.21.15.67:0/576509150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc228107dd0 0x7fc22819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:32.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.744+0000 7fc22dd30640 1 -- 172.21.15.67:0/576509150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc228108720 msgr2=0x7fc22819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:32.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.744+0000 7fc22dd30640 1 --2- 172.21.15.67:0/576509150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc228108720 0x7fc22819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:32.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.744+0000 7fc22dd30640 1 -- 172.21.15.67:0/576509150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22819fec0 con 0x7fc228102490 2024-07-31T14:47:32.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.744+0000 7fc22dd30640 1 --2- 172.21.15.67:0/576509150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc228102490 0x7fc228199f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc220005940 tx=0x7fc2200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:32.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.744+0000 7fc226ffd640 1 -- 172.21.15.67:0/576509150 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22003f030 con 0x7fc228102490 2024-07-31T14:47:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.744+0000 7fc226ffd640 1 -- 172.21.15.67:0/576509150 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc220003cb0 con 0x7fc228102490 2024-07-31T14:47:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.744+0000 7fc22ff94640 1 -- 172.21.15.67:0/576509150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2281a0150 con 0x7fc228102490 2024-07-31T14:47:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.744+0000 7fc226ffd640 1 -- 172.21.15.67:0/576509150 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220035070 con 0x7fc228102490 2024-07-31T14:47:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.744+0000 7fc22ff94640 1 -- 172.21.15.67:0/576509150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2281a6910 con 0x7fc228102490 2024-07-31T14:47:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.745+0000 7fc22ff94640 1 -- 172.21.15.67:0/576509150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc228103710 con 0x7fc228102490 2024-07-31T14:47:32.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.748+0000 7fc226ffd640 1 -- 172.21.15.67:0/576509150 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fc220042030 con 0x7fc228102490 2024-07-31T14:47:32.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.749+0000 7fc226ffd640 1 --2- 172.21.15.67:0/576509150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc2100b3ca0 0x7fc2100b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:32.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.749+0000 7fc22d52f640 1 --2- 172.21.15.67:0/576509150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc2100b3ca0 0x7fc2100b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:32.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.749+0000 7fc226ffd640 1 -- 172.21.15.67:0/576509150 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc220104be0 con 0x7fc228102490 2024-07-31T14:47:32.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.749+0000 7fc22d52f640 1 --2- 172.21.15.67:0/576509150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc2100b3ca0 0x7fc2100b61b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc21c005fd0 tx=0x7fc21c005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.752+0000 7fc226ffd640 1 -- 172.21.15.67:0/576509150 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc2200cbe00 con 0x7fc228102490 2024-07-31T14:47:32.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:32 smithi073 ceph-mon[36714]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.945+0000 7fc22ff94640 1 -- 172.21.15.67:0/576509150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc22810af20 con 0x7fc228102490 2024-07-31T14:47:32.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.947+0000 7fc226ffd640 1 -- 172.21.15.67:0/576509150 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24874 (secure 0 0 0) 0x7fc2200c5e20 con 0x7fc228102490 2024-07-31T14:47:32.949 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:47:32.949 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T14:47:20.440064+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T14:47:32.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.949+0000 7fc22ff94640 1 -- 172.21.15.67:0/576509150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc2100b3ca0 msgr2=0x7fc2100b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:32.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.949+0000 7fc22ff94640 1 --2- 172.21.15.67:0/576509150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc2100b3ca0 0x7fc2100b61b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc21c005fd0 tx=0x7fc21c005850 comp rx=0 tx=0).stop 2024-07-31T14:47:32.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.949+0000 7fc22ff94640 1 -- 172.21.15.67:0/576509150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc228102490 msgr2=0x7fc228199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:32.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.949+0000 7fc22ff94640 1 --2- 172.21.15.67:0/576509150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc228102490 0x7fc228199f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc220005940 tx=0x7fc2200042f0 comp rx=0 tx=0).stop 2024-07-31T14:47:32.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.949+0000 7fc22ff94640 1 -- 172.21.15.67:0/576509150 shutdown_connections 2024-07-31T14:47:32.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.949+0000 7fc22ff94640 1 --2- 172.21.15.67:0/576509150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc2100b3ca0 0x7fc2100b61b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:32.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.949+0000 7fc22ff94640 1 --2- 172.21.15.67:0/576509150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc228108720 0x7fc22819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:32.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.949+0000 7fc22ff94640 1 --2- 172.21.15.67:0/576509150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc228107dd0 0x7fc22819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:32.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.949+0000 7fc22ff94640 1 --2- 172.21.15.67:0/576509150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc228102490 0x7fc228199f50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:32.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.949+0000 7fc22ff94640 1 -- 172.21.15.67:0/576509150 >> 172.21.15.67:0/576509150 conn(0x7fc228079190 msgr2=0x7fc22806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:32.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.949+0000 7fc22ff94640 1 -- 172.21.15.67:0/576509150 shutdown_connections 2024-07-31T14:47:32.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:32.949+0000 7fc22ff94640 1 -- 172.21.15.67:0/576509150 wait complete. 2024-07-31T14:47:33.633 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-31T14:47:04.833142+0000', 'modified': '2024-07-31T14:47:17.927081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24671}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24671': {'gid': 24671, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.73:6839/4131046421', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 4131046421}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 4131046421}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24671, 'qdb_cluster': [24671]} 2024-07-31T14:47:33.633 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-31T14:47:33.633 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.b status 2024-07-31T14:47:33.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:33 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/576509150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:33.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:33 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/576509150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:33.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:33 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/576509150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:33.899 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:34.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:34 smithi067 ceph-mon[32750]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:34.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:34 smithi077 ceph-mon[37327]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:34.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:34 smithi073 ceph-mon[36714]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:35.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.654+0000 7f8c5d4e8640 1 -- 172.21.15.67:0/1855545318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c58107e10 msgr2=0x7f8c58108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.654+0000 7f8c5d4e8640 1 --2- 172.21.15.67:0/1855545318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c58107e10 0x7f8c58108220 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009900 tx=0x7f8c4c02a410 comp rx=0 tx=0).stop 2024-07-31T14:47:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.654+0000 7f8c5d4e8640 1 -- 172.21.15.67:0/1855545318 shutdown_connections 2024-07-31T14:47:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.654+0000 7f8c5d4e8640 1 --2- 172.21.15.67:0/1855545318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c58108760 0x7f8c5810ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.654+0000 7f8c5d4e8640 1 --2- 172.21.15.67:0/1855545318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c58107e10 0x7f8c58108220 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.654+0000 7f8c5d4e8640 1 --2- 172.21.15.67:0/1855545318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c581024d0 0x7f8c581028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.654+0000 7f8c5d4e8640 1 -- 172.21.15.67:0/1855545318 >> 172.21.15.67:0/1855545318 conn(0x7f8c58079190 msgr2=0x7f8c580795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.654+0000 7f8c5d4e8640 1 -- 172.21.15.67:0/1855545318 shutdown_connections 2024-07-31T14:47:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.655+0000 7f8c5d4e8640 1 -- 172.21.15.67:0/1855545318 wait complete. 2024-07-31T14:47:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.655+0000 7f8c5d4e8640 1 Processor -- start 2024-07-31T14:47:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c5d4e8640 1 -- start start 2024-07-31T14:47:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c5d4e8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c581024d0 0x7f8c58199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c5d4e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c58107e10 0x7f8c5819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c5d4e8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c58108760 0x7f8c5819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c5d4e8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581123b0 con 0x7f8c58108760 2024-07-31T14:47:35.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c5d4e8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8c58112230 con 0x7f8c581024d0 2024-07-31T14:47:35.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c5d4e8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8c58112530 con 0x7f8c58107e10 2024-07-31T14:47:35.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c577fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c58108760 0x7f8c5819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c577fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c58108760 0x7f8c5819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50670/0 (socket says 172.21.15.67:50670) 2024-07-31T14:47:35.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c577fe640 1 -- 172.21.15.67:0/1214698650 learned_addr learned my addr 172.21.15.67:0/1214698650 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:35.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c56ffd640 1 --2- 172.21.15.67:0/1214698650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c581024d0 0x7f8c58199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c567fc640 1 --2- 172.21.15.67:0/1214698650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c58107e10 0x7f8c5819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c577fe640 1 -- 172.21.15.67:0/1214698650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c58107e10 msgr2=0x7f8c5819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:35.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c577fe640 1 --2- 172.21.15.67:0/1214698650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c58107e10 0x7f8c5819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c577fe640 1 -- 172.21.15.67:0/1214698650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c581024d0 msgr2=0x7f8c58199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c577fe640 1 --2- 172.21.15.67:0/1214698650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c581024d0 0x7f8c58199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.656+0000 7f8c577fe640 1 -- 172.21.15.67:0/1214698650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5819fe40 con 0x7f8c58108760 2024-07-31T14:47:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.657+0000 7f8c577fe640 1 --2- 172.21.15.67:0/1214698650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c58108760 0x7f8c5819f710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8c5000eaa0 tx=0x7f8c5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.657+0000 7f8c3ffff640 1 -- 172.21.15.67:0/1214698650 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5000cdb0 con 0x7f8c58108760 2024-07-31T14:47:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.657+0000 7f8c5d4e8640 1 -- 172.21.15.67:0/1214698650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c581a0130 con 0x7f8c58108760 2024-07-31T14:47:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.657+0000 7f8c3ffff640 1 -- 172.21.15.67:0/1214698650 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c50005020 con 0x7f8c58108760 2024-07-31T14:47:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.657+0000 7f8c3ffff640 1 -- 172.21.15.67:0/1214698650 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c500116d0 con 0x7f8c58108760 2024-07-31T14:47:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.657+0000 7f8c5d4e8640 1 -- 172.21.15.67:0/1214698650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c581a6890 con 0x7f8c58108760 2024-07-31T14:47:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.660+0000 7f8c63a39740 1 Processor -- start 2024-07-31T14:47:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.660+0000 7f8c3ffff640 1 -- 172.21.15.67:0/1214698650 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f8c500051d0 con 0x7f8c58108760 2024-07-31T14:47:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.660+0000 7f8c63a39740 1 -- start start 2024-07-31T14:47:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.660+0000 7f8c63a39740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdf0f60 0x55cebd196380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.660+0000 7f8c63a39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdc80a0 0x55cebd1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.660+0000 7f8c63a39740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdd0280 0x55cebd196e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.660+0000 7f8c63a39740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cebd10c920 con 0x55cebcdd0280 2024-07-31T14:47:35.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.660+0000 7f8c63a39740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cebd0fbe20 con 0x55cebcdf0f60 2024-07-31T14:47:35.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.660+0000 7f8c63a39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cebd0ffe60 con 0x55cebcdc80a0 2024-07-31T14:47:35.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.660+0000 7f8c567fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdd0280 0x55cebd196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.660+0000 7f8c567fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdd0280 0x55cebd196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50672/0 (socket says 172.21.15.67:50672) 2024-07-31T14:47:35.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.660+0000 7f8c567fc640 1 -- 172.21.15.67:0/2898504565 learned_addr learned my addr 172.21.15.67:0/2898504565 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:35.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c56ffd640 1 --2- 172.21.15.67:0/2898504565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdc80a0 0x55cebd1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c567fc640 1 --2- 172.21.15.67:0/2898504565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdf0f60 0x55cebd196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c567fc640 1 -- 172.21.15.67:0/2898504565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdc80a0 msgr2=0x55cebd1968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:47:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c567fc640 1 --2- 172.21.15.67:0/2898504565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdc80a0 0x55cebd1968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c567fc640 1 -- 172.21.15.67:0/2898504565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdf0f60 msgr2=0x55cebd196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:47:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c567fc640 1 --2- 172.21.15.67:0/2898504565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdf0f60 0x55cebd196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c567fc640 1 -- 172.21.15.67:0/2898504565 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cebd0a2a40 con 0x55cebcdd0280 2024-07-31T14:47:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c56ffd640 1 --2- 172.21.15.67:0/2898504565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdc80a0 0x55cebd1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:47:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c3ffff640 1 --2- 172.21.15.67:0/1214698650 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c400b3c20 0x7f8c400b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c56ffd640 1 --2- 172.21.15.67:0/1214698650 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c400b3c20 0x7f8c400b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c3ffff640 1 -- 172.21.15.67:0/1214698650 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f8c500e4240 con 0x7f8c58108760 2024-07-31T14:47:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c567fc640 1 --2- 172.21.15.67:0/2898504565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdd0280 0x55cebd196e00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8c5806b0b0 tx=0x7f8c4c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40169e2e197c8ba2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c3d7fa640 1 -- 172.21.15.67:0/2898504565 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c035040 con 0x55cebcdd0280 2024-07-31T14:47:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c3d7fa640 1 -- 172.21.15.67:0/2898504565 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c4c004030 con 0x55cebcdd0280 2024-07-31T14:47:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.661+0000 7f8c3d7fa640 1 -- 172.21.15.67:0/2898504565 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0039b0 con 0x55cebcdd0280 2024-07-31T14:47:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.662+0000 7f8c63a39740 1 -- 172.21.15.67:0/2898504565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdd0280 msgr2=0x55cebd196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.662+0000 7f8c63a39740 1 --2- 172.21.15.67:0/2898504565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdd0280 0x55cebd196e00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8c5806b0b0 tx=0x7f8c4c0058b0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.662+0000 7f8c56ffd640 1 --2- 172.21.15.67:0/1214698650 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c400b3c20 0x7f8c400b6130 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55cebd195000 tx=0x7f8c4800b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.662+0000 7f8c63a39740 1 -- 172.21.15.67:0/2898504565 shutdown_connections 2024-07-31T14:47:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.662+0000 7f8c63a39740 1 --2- 172.21.15.67:0/2898504565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdd0280 0x55cebd196e00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.662+0000 7f8c63a39740 1 --2- 172.21.15.67:0/2898504565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdc80a0 0x55cebd1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.662+0000 7f8c63a39740 1 --2- 172.21.15.67:0/2898504565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdf0f60 0x55cebd196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.662+0000 7f8c63a39740 1 -- 172.21.15.67:0/2898504565 >> 172.21.15.67:0/2898504565 conn(0x55cebcde5350 msgr2=0x55cebd194270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.662+0000 7f8c63a39740 1 -- 172.21.15.67:0/2898504565 shutdown_connections 2024-07-31T14:47:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.662+0000 7f8c63a39740 1 -- 172.21.15.67:0/2898504565 wait complete. 2024-07-31T14:47:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.663+0000 7f8c63a39740 1 -- start start 2024-07-31T14:47:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.663+0000 7f8c63a39740 1 Processor -- start 2024-07-31T14:47:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.663+0000 7f8c63a39740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdc80a0 0x55cebd1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.663+0000 7f8c63a39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdd0280 0x55cebd1a2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.663+0000 7f8c63a39740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdf0f60 0x55cebd1ade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.663+0000 7f8c63a39740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cebcf5f5f0 con 0x55cebcdf0f60 2024-07-31T14:47:35.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.663+0000 7f8c63a39740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cebd09e890 con 0x55cebcdc80a0 2024-07-31T14:47:35.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.663+0000 7f8c63a39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cebd1a4b00 con 0x55cebcdd0280 2024-07-31T14:47:35.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.663+0000 7f8c63a39740 10 client.0 _ll_register_callbacks cb 0x55cebd0a87d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:47:35.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.663+0000 7f8c577fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdf0f60 0x55cebd1ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.663+0000 7f8c577fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdf0f60 0x55cebd1ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50686/0 (socket says 172.21.15.67:50686) 2024-07-31T14:47:35.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.663+0000 7f8c577fe640 1 -- 172.21.15.67:0/3121976645 learned_addr learned my addr 172.21.15.67:0/3121976645 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:35.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.663+0000 7f8c567fc640 1 --2- 172.21.15.67:0/3121976645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdd0280 0x55cebd1a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c56ffd640 1 --2- 172.21.15.67:0/3121976645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdc80a0 0x55cebd1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c577fe640 1 -- 172.21.15.67:0/3121976645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdd0280 msgr2=0x55cebd1a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:47:35.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c577fe640 1 --2- 172.21.15.67:0/3121976645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdd0280 0x55cebd1a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c577fe640 1 -- 172.21.15.67:0/3121976645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdc80a0 msgr2=0x55cebd1a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:47:35.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c577fe640 1 --2- 172.21.15.67:0/3121976645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdc80a0 0x55cebd1a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c577fe640 1 -- 172.21.15.67:0/3121976645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cebce5c540 con 0x55cebcdf0f60 2024-07-31T14:47:35.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c577fe640 1 --2- 172.21.15.67:0/3121976645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdf0f60 0x55cebd1ade50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8c500a7fd0 tx=0x7f8c500aad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e5ea0d133b053d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:35.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:47:35.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c3aff5640 1 -- 172.21.15.67:0/3121976645 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5001f040 con 0x55cebcdf0f60 2024-07-31T14:47:35.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c3aff5640 1 -- 172.21.15.67:0/3121976645 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f8c500a7350 con 0x55cebcdf0f60 2024-07-31T14:47:35.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c63a39740 1 -- 172.21.15.67:0/3121976645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4c004030 con 0x55cebcdf0f60 2024-07-31T14:47:35.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c3aff5640 1 -- 172.21.15.67:0/3121976645 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f8c500b3e10 con 0x55cebcdf0f60 2024-07-31T14:47:35.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c63a39740 10 client.14892 fetch_fsmap learned FSMap version 17 2024-07-31T14:47:35.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c63a39740 1 -- 172.21.15.67:0/3121976645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55cebd1b0b50 con 0x55cebcdf0f60 2024-07-31T14:47:35.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.664+0000 7f8c3aff5640 1 -- 172.21.15.67:0/3121976645 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c4c004030 con 0x55cebcdf0f60 2024-07-31T14:47:35.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.665+0000 7f8c3aff5640 1 -- 172.21.15.67:0/3121976645 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f8c5000f040 con 0x55cebcdf0f60 2024-07-31T14:47:35.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.665+0000 7f8c63a39740 10 client.14892 fetch_fsmap finished waiting for FSMap version 17 2024-07-31T14:47:35.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.665+0000 7f8c63a39740 10 client.14892 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-31T14:47:35.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.665+0000 7f8c63a39740 20 client.14892 populate_metadata read hostname 'smithi067' 2024-07-31T14:47:35.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.665+0000 7f8c63a39740 1 --2- 172.21.15.67:0/3121976645 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cebcdaf8e0 0x55cebd1b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.665+0000 7f8c63a39740 4 client.14892 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:47:35.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.665+0000 7f8c63a39740 1 -- 172.21.15.67:0/3121976645 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cebd10e870 con 0x55cebcdaf8e0 2024-07-31T14:47:35.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.665+0000 7f8c56ffd640 1 --2- 172.21.15.67:0/3121976645 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cebcdaf8e0 0x55cebd1b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.665+0000 7f8c56ffd640 1 --2- 172.21.15.67:0/3121976645 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cebcdaf8e0 0x55cebd1b4630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=184a892ea9bf0f28 server_cookie=ff736760e5465d34 in_seq=0 out_seq=0 2024-07-31T14:47:35.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.665+0000 7f8c3aff5640 10 client.14892 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:47:35.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.666+0000 7f8c3aff5640 1 -- 172.21.15.67:0/3121976645 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cebd10e870 con 0x55cebcdaf8e0 2024-07-31T14:47:35.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.666+0000 7f8c3aff5640 10 client.14892 handle_command_reply: tid=0 2024-07-31T14:47:35.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.666+0000 7f8c63a39740 1 client.14892 shutdown 2024-07-31T14:47:35.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.666+0000 7f8c63a39740 1 -- 172.21.15.67:0/3121976645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdf0f60 msgr2=0x55cebd1ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:35.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.666+0000 7f8c63a39740 1 --2- 172.21.15.67:0/3121976645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdf0f60 0x55cebd1ade50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8c500a7fd0 tx=0x7f8c500aad60 comp rx=0 tx=0).stop 2024-07-31T14:47:35.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.667+0000 7f8c63a39740 1 -- 172.21.15.67:0/3121976645 shutdown_connections 2024-07-31T14:47:35.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.667+0000 7f8c63a39740 1 --2- 172.21.15.67:0/3121976645 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cebcdaf8e0 0x55cebd1b4630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.667+0000 7f8c63a39740 1 --2- 172.21.15.67:0/3121976645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdf0f60 0x55cebd1ade50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.667+0000 7f8c63a39740 1 --2- 172.21.15.67:0/3121976645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdd0280 0x55cebd1a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.667+0000 7f8c63a39740 1 --2- 172.21.15.67:0/3121976645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdc80a0 0x55cebd1a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.667+0000 7f8c3aff5640 0 client.14892 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:47:35.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.667+0000 7f8c63a39740 1 -- 172.21.15.67:0/3121976645 >> 172.21.15.67:0/3121976645 conn(0x55cebcde5350 msgr2=0x55cebd199510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:35.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.667+0000 7f8c63a39740 1 -- 172.21.15.67:0/3121976645 shutdown_connections 2024-07-31T14:47:35.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.667+0000 7f8c63a39740 1 -- 172.21.15.67:0/3121976645 wait complete. 2024-07-31T14:47:35.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.667+0000 7f8c63a39740 20 client.14892 trim_cache size 0 max 16384 2024-07-31T14:47:35.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c63a39740 1 Processor -- start 2024-07-31T14:47:35.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c63a39740 1 -- start start 2024-07-31T14:47:35.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c63a39740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdc80a0 0x55cebd19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c63a39740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdd0280 0x55cebd1addc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c63a39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdf0f60 0x55cebd1ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c63a39740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cebd1b2970 con 0x55cebcdd0280 2024-07-31T14:47:35.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c63a39740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cebd1b3040 con 0x55cebcdc80a0 2024-07-31T14:47:35.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c63a39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cebd1b3710 con 0x55cebcdf0f60 2024-07-31T14:47:35.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c567fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdd0280 0x55cebd1addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c567fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdd0280 0x55cebd1addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50694/0 (socket says 172.21.15.67:50694) 2024-07-31T14:47:35.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c567fc640 1 -- 172.21.15.67:0/3143439126 learned_addr learned my addr 172.21.15.67:0/3143439126 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:35.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c56ffd640 1 --2- 172.21.15.67:0/3143439126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdc80a0 0x55cebd19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c577fe640 1 --2- 172.21.15.67:0/3143439126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdf0f60 0x55cebd1ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c567fc640 1 -- 172.21.15.67:0/3143439126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdf0f60 msgr2=0x55cebd1ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:47:35.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c567fc640 1 --2- 172.21.15.67:0/3143439126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdf0f60 0x55cebd1ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c567fc640 1 -- 172.21.15.67:0/3143439126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdc80a0 msgr2=0x55cebd19c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:47:35.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c567fc640 1 --2- 172.21.15.67:0/3143439126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdc80a0 0x55cebd19c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.686+0000 7f8c567fc640 1 -- 172.21.15.67:0/3143439126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cebd1a2d60 con 0x55cebcdd0280 2024-07-31T14:47:35.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.687+0000 7f8c567fc640 1 --2- 172.21.15.67:0/3143439126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdd0280 0x55cebd1addc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009900 tx=0x7f8c4c005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0e6557fa05b1de server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:35.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.687+0000 7f8c3cff9640 1 -- 172.21.15.67:0/3143439126 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c051070 con 0x55cebcdd0280 2024-07-31T14:47:35.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.687+0000 7f8c3cff9640 1 -- 172.21.15.67:0/3143439126 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c4c031db0 con 0x55cebcdd0280 2024-07-31T14:47:35.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.687+0000 7f8c3cff9640 1 -- 172.21.15.67:0/3143439126 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c035070 con 0x55cebcdd0280 2024-07-31T14:47:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 -- 172.21.15.67:0/3143439126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdd0280 msgr2=0x55cebd1addc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 --2- 172.21.15.67:0/3143439126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdd0280 0x55cebd1addc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009900 tx=0x7f8c4c005750 comp rx=0 tx=0).stop 2024-07-31T14:47:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 -- 172.21.15.67:0/3143439126 shutdown_connections 2024-07-31T14:47:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 --2- 172.21.15.67:0/3143439126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdf0f60 0x55cebd1ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 --2- 172.21.15.67:0/3143439126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdd0280 0x55cebd1addc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 --2- 172.21.15.67:0/3143439126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdc80a0 0x55cebd19c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 -- 172.21.15.67:0/3143439126 >> 172.21.15.67:0/3143439126 conn(0x55cebcde5350 msgr2=0x55cebd1a1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 -- 172.21.15.67:0/3143439126 shutdown_connections 2024-07-31T14:47:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 -- 172.21.15.67:0/3143439126 wait complete. 2024-07-31T14:47:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 -- start start 2024-07-31T14:47:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 Processor -- start 2024-07-31T14:47:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdc80a0 0x55cebd1bcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdd0280 0x55cebd1b9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdf0f60 0x55cebd1b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cebd1b44b0 con 0x55cebcdc80a0 2024-07-31T14:47:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cebd1b3de0 con 0x55cebcdd0280 2024-07-31T14:47:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.688+0000 7f8c63a39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cebd1b4b80 con 0x55cebcdf0f60 2024-07-31T14:47:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c63a39740 10 client.0 _ll_register_callbacks cb 0x7f8c500aa8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:47:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c56ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdc80a0 0x55cebd1bcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c56ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdc80a0 0x55cebd1bcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50698/0 (socket says 172.21.15.67:50698) 2024-07-31T14:47:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c56ffd640 1 -- 172.21.15.67:0/574717780 learned_addr learned my addr 172.21.15.67:0/574717780 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c567fc640 1 --2- 172.21.15.67:0/574717780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdd0280 0x55cebd1b9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c577fe640 1 --2- 172.21.15.67:0/574717780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdf0f60 0x55cebd1b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c56ffd640 1 -- 172.21.15.67:0/574717780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdf0f60 msgr2=0x55cebd1b9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:47:35.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c56ffd640 1 --2- 172.21.15.67:0/574717780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdf0f60 0x55cebd1b9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c56ffd640 1 -- 172.21.15.67:0/574717780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdd0280 msgr2=0x55cebd1b9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:47:35.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c56ffd640 1 --2- 172.21.15.67:0/574717780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdd0280 0x55cebd1b9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c56ffd640 1 -- 172.21.15.67:0/574717780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cebd1ba340 con 0x55cebcdc80a0 2024-07-31T14:47:35.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c577fe640 1 --2- 172.21.15.67:0/574717780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdf0f60 0x55cebd1b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:47:35.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c56ffd640 1 --2- 172.21.15.67:0/574717780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdc80a0 0x55cebd1bcdb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800bfc0 tx=0x7f8c480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189b7a7c2c9f7862 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:35.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:47:35.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c3aff5640 1 -- 172.21.15.67:0/574717780 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48005220 con 0x55cebcdc80a0 2024-07-31T14:47:35.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c3aff5640 1 -- 172.21.15.67:0/574717780 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f8c480053c0 con 0x55cebcdc80a0 2024-07-31T14:47:35.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c63a39740 1 -- 172.21.15.67:0/574717780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4c031db0 con 0x55cebcdc80a0 2024-07-31T14:47:35.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.689+0000 7f8c3aff5640 1 -- 172.21.15.67:0/574717780 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f8c48039050 con 0x55cebcdc80a0 2024-07-31T14:47:35.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.690+0000 7f8c63a39740 10 client.14898 fetch_fsmap learned FSMap version 17 2024-07-31T14:47:35.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.690+0000 7f8c63a39740 1 -- 172.21.15.67:0/574717780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55cebd21c670 con 0x55cebcdc80a0 2024-07-31T14:47:35.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.690+0000 7f8c3aff5640 1 -- 172.21.15.67:0/574717780 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c4c031db0 con 0x55cebcdc80a0 2024-07-31T14:47:35.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.690+0000 7f8c3aff5640 1 -- 172.21.15.67:0/574717780 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f8c48019040 con 0x55cebcdc80a0 2024-07-31T14:47:35.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.690+0000 7f8c63a39740 10 client.14898 fetch_fsmap finished waiting for FSMap version 17 2024-07-31T14:47:35.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.690+0000 7f8c63a39740 10 client.14898 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-31T14:47:35.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.690+0000 7f8c63a39740 20 client.14898 populate_metadata read hostname 'smithi067' 2024-07-31T14:47:35.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.690+0000 7f8c63a39740 1 --2- 172.21.15.67:0/574717780 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cebcd84c90 0x55cebd21ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:35.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.690+0000 7f8c63a39740 4 client.14898 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:47:35.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.690+0000 7f8c63a39740 1 -- 172.21.15.67:0/574717780 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "status"}) -- 0x55cebd10a310 con 0x55cebcd84c90 2024-07-31T14:47:35.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.690+0000 7f8c567fc640 1 --2- 172.21.15.67:0/574717780 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cebcd84c90 0x55cebd21ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:35.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c567fc640 1 --2- 172.21.15.67:0/574717780 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cebcd84c90 0x55cebd21ffa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c430f067c138d6 server_cookie=ad96215d807b3cc6 in_seq=0 out_seq=0 2024-07-31T14:47:35.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c3aff5640 10 client.14898 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:47:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c3aff5640 1 -- 172.21.15.67:0/574717780 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55cebd10a310 con 0x55cebcd84c90 2024-07-31T14:47:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c3aff5640 10 client.14898 handle_command_reply: tid=0 2024-07-31T14:47:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c63a39740 1 client.14898 shutdown 2024-07-31T14:47:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c63a39740 1 -- 172.21.15.67:0/574717780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdc80a0 msgr2=0x55cebd1bcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c63a39740 1 --2- 172.21.15.67:0/574717780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdc80a0 0x55cebd1bcdb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800bfc0 tx=0x7f8c480046c0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c63a39740 1 -- 172.21.15.67:0/574717780 shutdown_connections 2024-07-31T14:47:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c63a39740 1 --2- 172.21.15.67:0/574717780 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cebcd84c90 0x55cebd21ffa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c63a39740 1 --2- 172.21.15.67:0/574717780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebcdf0f60 0x55cebd1b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c63a39740 1 --2- 172.21.15.67:0/574717780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cebcdd0280 0x55cebd1b9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c63a39740 1 --2- 172.21.15.67:0/574717780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cebcdc80a0 0x55cebd1bcdb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c63a39740 1 -- 172.21.15.67:0/574717780 >> 172.21.15.67:0/574717780 conn(0x55cebcde5350 msgr2=0x55cebd193bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c3aff5640 0 client.14898 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:47:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c63a39740 1 -- 172.21.15.67:0/574717780 shutdown_connections 2024-07-31T14:47:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c63a39740 1 -- 172.21.15.67:0/574717780 wait complete. 2024-07-31T14:47:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.691+0000 7f8c63a39740 20 client.14898 trim_cache size 0 max 16384 2024-07-31T14:47:35.699 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:47:35.699 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:47:35.699 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-07-31T14:47:35.699 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24671, 2024-07-31T14:47:35.699 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:47:35.699 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:47:35.699 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:47:35.700 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 30.651282621, 2024-07-31T14:47:35.700 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 17, 2024-07-31T14:47:35.700 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:47:35.700 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 87, 2024-07-31T14:47:35.700 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 44.862820018000001 2024-07-31T14:47:35.700 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:47:35.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.692+0000 7f8c5d4e8640 1 -- 172.21.15.67:0/1214698650 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c400b3c20 msgr2=0x7f8c400b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:35.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.692+0000 7f8c5d4e8640 1 --2- 172.21.15.67:0/1214698650 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c400b3c20 0x7f8c400b6130 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55cebd195000 tx=0x7f8c4800b390 comp rx=0 tx=0).stop 2024-07-31T14:47:35.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.692+0000 7f8c5d4e8640 1 -- 172.21.15.67:0/1214698650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c58108760 msgr2=0x7f8c5819f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:35.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.692+0000 7f8c5d4e8640 1 --2- 172.21.15.67:0/1214698650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c58108760 0x7f8c5819f710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8c5000eaa0 tx=0x7f8c5000ef60 comp rx=0 tx=0).stop 2024-07-31T14:47:35.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.692+0000 7f8c5d4e8640 1 -- 172.21.15.67:0/1214698650 shutdown_connections 2024-07-31T14:47:35.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.692+0000 7f8c5d4e8640 1 --2- 172.21.15.67:0/1214698650 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c400b3c20 0x7f8c400b6130 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.692+0000 7f8c5d4e8640 1 --2- 172.21.15.67:0/1214698650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c58108760 0x7f8c5819f710 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.692+0000 7f8c5d4e8640 1 --2- 172.21.15.67:0/1214698650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c58107e10 0x7f8c5819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.692+0000 7f8c5d4e8640 1 --2- 172.21.15.67:0/1214698650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c581024d0 0x7f8c58199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:35.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.692+0000 7f8c5d4e8640 1 -- 172.21.15.67:0/1214698650 >> 172.21.15.67:0/1214698650 conn(0x7f8c58079190 msgr2=0x7f8c5806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:35.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.692+0000 7f8c5d4e8640 1 -- 172.21.15.67:0/1214698650 shutdown_connections 2024-07-31T14:47:35.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:35.692+0000 7f8c5d4e8640 1 -- 172.21.15.67:0/1214698650 wait complete. 2024-07-31T14:47:36.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:36 smithi067 ceph-mon[32750]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:36.546 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-31T14:47:36.546 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-07-31T14:47:36.546 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs ls 2024-07-31T14:47:36.816 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:36.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:36 smithi073 ceph-mon[36714]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:36.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:36 smithi077 ceph-mon[37327]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-31T14:47:38.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.440+0000 7f9476744640 1 -- 172.21.15.67:0/2394433540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9470105d60 msgr2=0x7f9470106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:38.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.440+0000 7f9476744640 1 --2- 172.21.15.67:0/2394433540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9470105d60 0x7f9470106170 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9464009900 tx=0x7f946402a410 comp rx=0 tx=0).stop 2024-07-31T14:47:38.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.440+0000 7f9476744640 1 -- 172.21.15.67:0/2394433540 shutdown_connections 2024-07-31T14:47:38.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.440+0000 7f9476744640 1 --2- 172.21.15.67:0/2394433540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94701066b0 0x7f9470110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:38.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.440+0000 7f9476744640 1 --2- 172.21.15.67:0/2394433540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9470105d60 0x7f9470106170 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:38.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.440+0000 7f9476744640 1 --2- 172.21.15.67:0/2394433540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9470101b70 0x7f9470101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:38.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.440+0000 7f9476744640 1 -- 172.21.15.67:0/2394433540 >> 172.21.15.67:0/2394433540 conn(0x7f94700798d0 msgr2=0x7f9470079ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:38.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.440+0000 7f9476744640 1 -- 172.21.15.67:0/2394433540 shutdown_connections 2024-07-31T14:47:38.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.440+0000 7f9476744640 1 -- 172.21.15.67:0/2394433540 wait complete. 2024-07-31T14:47:38.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.441+0000 7f9476744640 1 Processor -- start 2024-07-31T14:47:38.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.441+0000 7f9476744640 1 -- start start 2024-07-31T14:47:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.441+0000 7f9476744640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9470101b70 0x7f9470102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.441+0000 7f9476744640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9470105d60 0x7f9470103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.441+0000 7f9476744640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94701066b0 0x7f9470103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.441+0000 7f9476744640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9470113540 con 0x7f9470101b70 2024-07-31T14:47:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.441+0000 7f9476744640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f94701133c0 con 0x7f9470105d60 2024-07-31T14:47:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.441+0000 7f9476744640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f94701136c0 con 0x7f94701066b0 2024-07-31T14:47:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.441+0000 7f946ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9470101b70 0x7f9470102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f946ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9470101b70 0x7f9470102b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50718/0 (socket says 172.21.15.67:50718) 2024-07-31T14:47:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f946ffff640 1 -- 172.21.15.67:0/2170467931 learned_addr learned my addr 172.21.15.67:0/2170467931 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f946f7fe640 1 --2- 172.21.15.67:0/2170467931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9470105d60 0x7f9470103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:38.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f9474ce1640 1 --2- 172.21.15.67:0/2170467931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94701066b0 0x7f9470103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:38.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f946ffff640 1 -- 172.21.15.67:0/2170467931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94701066b0 msgr2=0x7f9470103590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:38.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f946ffff640 1 --2- 172.21.15.67:0/2170467931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94701066b0 0x7f9470103590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:38.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f946ffff640 1 -- 172.21.15.67:0/2170467931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9470105d60 msgr2=0x7f9470103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:38.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f946ffff640 1 --2- 172.21.15.67:0/2170467931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9470105d60 0x7f9470103050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:38.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f946ffff640 1 -- 172.21.15.67:0/2170467931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94701a8120 con 0x7f9470101b70 2024-07-31T14:47:38.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f946ffff640 1 --2- 172.21.15.67:0/2170467931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9470101b70 0x7f9470102b10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f946000c920 tx=0x7f946000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:38.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f946d7fa640 1 -- 172.21.15.67:0/2170467931 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460007c70 con 0x7f9470101b70 2024-07-31T14:47:38.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f946d7fa640 1 -- 172.21.15.67:0/2170467931 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9460007e10 con 0x7f9470101b70 2024-07-31T14:47:38.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f9476744640 1 -- 172.21.15.67:0/2170467931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94701a8410 con 0x7f9470101b70 2024-07-31T14:47:38.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f946d7fa640 1 -- 172.21.15.67:0/2170467931 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460018820 con 0x7f9470101b70 2024-07-31T14:47:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.442+0000 7f9476744640 1 -- 172.21.15.67:0/2170467931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94701a8970 con 0x7f9470101b70 2024-07-31T14:47:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.444+0000 7f9476744640 1 -- 172.21.15.67:0/2170467931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9470101f80 con 0x7f9470101b70 2024-07-31T14:47:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.445+0000 7f946d7fa640 1 -- 172.21.15.67:0/2170467931 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f946000f450 con 0x7f9470101b70 2024-07-31T14:47:38.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.446+0000 7f946d7fa640 1 --2- 172.21.15.67:0/2170467931 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94580b3ca0 0x7f94580b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:38.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.446+0000 7f946f7fe640 1 --2- 172.21.15.67:0/2170467931 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94580b3ca0 0x7f94580b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:38.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.446+0000 7f946d7fa640 1 -- 172.21.15.67:0/2170467931 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f94600e4210 con 0x7f9470101b70 2024-07-31T14:47:38.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.446+0000 7f946f7fe640 1 --2- 172.21.15.67:0/2170467931 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94580b3ca0 0x7f94580b61b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9464000c00 tx=0x7f9464031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.450+0000 7f946d7fa640 1 -- 172.21.15.67:0/2170467931 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f9460014030 con 0x7f9470101b70 2024-07-31T14:47:38.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.689+0000 7f9476744640 1 -- 172.21.15.67:0/2170467931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9470103e10 con 0x7f9470101b70 2024-07-31T14:47:38.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.690+0000 7f946d7fa640 1 -- 172.21.15.67:0/2170467931 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f94600ab430 con 0x7f9470101b70 2024-07-31T14:47:38.691 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-07-31T14:47:38.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.692+0000 7f9476744640 1 -- 172.21.15.67:0/2170467931 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94580b3ca0 msgr2=0x7f94580b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.692+0000 7f9476744640 1 --2- 172.21.15.67:0/2170467931 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94580b3ca0 0x7f94580b61b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9464000c00 tx=0x7f9464031400 comp rx=0 tx=0).stop 2024-07-31T14:47:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.692+0000 7f9476744640 1 -- 172.21.15.67:0/2170467931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9470101b70 msgr2=0x7f9470102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.692+0000 7f9476744640 1 --2- 172.21.15.67:0/2170467931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9470101b70 0x7f9470102b10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f946000c920 tx=0x7f946000cde0 comp rx=0 tx=0).stop 2024-07-31T14:47:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.692+0000 7f9476744640 1 -- 172.21.15.67:0/2170467931 shutdown_connections 2024-07-31T14:47:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.692+0000 7f9476744640 1 --2- 172.21.15.67:0/2170467931 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94580b3ca0 0x7f94580b61b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.692+0000 7f9476744640 1 --2- 172.21.15.67:0/2170467931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94701066b0 0x7f9470103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.692+0000 7f9476744640 1 --2- 172.21.15.67:0/2170467931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9470105d60 0x7f9470103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.692+0000 7f9476744640 1 --2- 172.21.15.67:0/2170467931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9470101b70 0x7f9470102b10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.692+0000 7f9476744640 1 -- 172.21.15.67:0/2170467931 >> 172.21.15.67:0/2170467931 conn(0x7f94700798d0 msgr2=0x7f9470106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.692+0000 7f9476744640 1 -- 172.21.15.67:0/2170467931 shutdown_connections 2024-07-31T14:47:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:38.692+0000 7f9476744640 1 -- 172.21.15.67:0/2170467931 wait complete. 2024-07-31T14:47:38.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:38 smithi067 ceph-mon[32750]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:38.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:38 smithi073 ceph-mon[36714]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:38.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:38 smithi077 ceph-mon[37327]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:39.431 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-31T14:47:39.432 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-31T14:47:39.432 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-07-31T14:47:39.432 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-07-31T14:47:39.432 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-07-31T14:47:39.432 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-31T14:47:39.432 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-31T14:47:39.432 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-31T14:47:39.433 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-07-31T14:47:39.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:47:39.433 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-07-31T14:47:39.456 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-31T14:47:39.456 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-31T14:47:39.456 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-07-31T14:47:39.456 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-07-31T14:47:39.456 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-07-31T14:47:39.456 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-07-31T14:47:39.456 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-31T14:47:39.456 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-07-31T14:47:39.456 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-07-31T14:47:39.457 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-31T14:47:39.457 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-07-31T14:47:39.457 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-07-31T14:47:39.457 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-31T14:47:39.457 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-07-31T14:47:39.457 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-31T14:47:39.457 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-07-31T14:47:39.457 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-07-31T14:47:39.457 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42157sec preferred_lft 42157sec 2024-07-31T14:47:39.457 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-07-31T14:47:39.457 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-07-31T14:47:39.458 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-31T14:47:39.459 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-31T14:47:39.459 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-31T14:47:39.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2024-07-31T14:47:39.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2024-07-31T14:47:39.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2024-07-31T14:47:39.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-31T14:47:39.459 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-31T14:47:39.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-31T14:47:39.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:39 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2170467931' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-31T14:47:39.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:47:39.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:47:39.668 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-31T14:47:39.695 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-07-31T14:47:39.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:47:39.697 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-07-31T14:47:39.764 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-07-31T14:47:39.764 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-31T14:47:39.764 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-07-31T14:47:39.764 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-07-31T14:47:39.764 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-31T14:47:39.765 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-31T14:47:39.765 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-31T14:47:39.765 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-07-31T14:47:39.765 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-07-31T14:47:39.765 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-07-31T14:47:39.765 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-31T14:47:39.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-31T14:47:39.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:39 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2170467931' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-31T14:47:39.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:39 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2170467931' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-31T14:47:40.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:47:40.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:47:40.063 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-07-31T14:47:40.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:47:40.080 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2024-07-31T14:47:40.136 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-31T14:47:40.136 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-31T14:47:40.136 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-31T14:47:40.136 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-07-31T14:47:40.137 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-31T14:47:40.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-31T14:47:40.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:47:40.259 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-07-31T14:47:40.259 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-31T14:47:40.259 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-31T14:47:40.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-07-31T14:47:40.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-31T14:47:40.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-07-31T14:47:40.260 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-07-31T14:47:40.260 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-07-31T14:47:40.260 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-31T14:47:40.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-31T14:47:40.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:47:40.388 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-31T14:47:40.389 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-31T14:47:40.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-07-31T14:47:40.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2024-07-31T14:47:40.389 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-31T14:47:40.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-31T14:47:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:47:40.470 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-07-31T14:47:40.470 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-31T14:47:40.471 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-07-31T14:47:40.487 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-07-31T14:47:40.487 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-31T14:47:40.487 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-07-31T14:47:40.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-07-31T14:47:40.611 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-07-31T14:47:40.669 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-07-31T14:47:40.669 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-07-31T14:47:40.669 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-07-31T14:47:40.669 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-07-31T14:47:40.669 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-07-31T14:47:40.669 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-07-31T14:47:40.669 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-07-31T14:47:40.669 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-07-31T14:47:40.670 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-07-31T14:47:40.670 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-07-31T14:47:40.670 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-07-31T14:47:40.670 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-07-31T14:47:40.670 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-07-31T14:47:40.670 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-07-31T14:47:40.670 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-07-31T14:47:40.670 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-07-31T14:47:40.670 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-07-31T14:47:40.670 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-07-31T14:47:40.670 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-07-31T14:47:40.670 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-07-31T14:47:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-31T14:47:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-31T14:47:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-07-31T14:47:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-07-31T14:47:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-07-31T14:47:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay 2024-07-31T14:47:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/3e51295c991f928777c324004efe79d40527c29bf238dc5b2edc68feca2fd1d5/merged 2024-07-31T14:47:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/960135413c8674d8c7d3ce900e5fde26eec4343567449c6dbf16dcfa6613b6df/merged 2024-07-31T14:47:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/0 2024-07-31T14:47:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-07-31T14:47:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/04b9700cf4b62897feca63b7a3d46f15e56b4841d2b6180fcf1f47d213a9384b/merged 2024-07-31T14:47:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/b7ad35e28aeac3e4c01b37cf123ec218435cf54d7864e4c3382dce732d21e709/merged 2024-07-31T14:47:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/5f8c3be676d21da357b298f960248a21f12ae6f3e4c6198a403da7914139840d/merged 2024-07-31T14:47:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/75ef93bff7378615b42c695b19d7e84ee63615925067534357b0a85025d5a6e5/merged 2024-07-31T14:47:40.672 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/57d76a074936a1eb554829a12891c4c3d3b4f7d00d7d5b3fe5c6cc685d3c9108/merged 2024-07-31T14:47:40.672 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/dea51169dc1449bb67b32ddbb2c571501deca9f360b8213202e97b0e088ed706/merged 2024-07-31T14:47:40.672 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/227f0988408bc964ba64e4788d8aaf28d1ead804f24b102898a40dfbccca6bcf/merged 2024-07-31T14:47:40.673 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/18f95aae4c88e659f2390d08fcbe16e28abf93935403c898a5fb7be14fefcb8d/merged 2024-07-31T14:47:40.673 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-07-31T14:47:40.673 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-31T14:47:40.673 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-31T14:47:40.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:47:40.674 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-07-31T14:47:40.724 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-07-31T14:47:40.725 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-07-31T14:47:40.767 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-07-31T14:47:40.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:40 smithi067 ceph-mon[32750]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:40.793 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-07-31T14:47:40.849 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-07-31T14:47:40.850 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-07-31T14:47:40.850 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-07-31T14:47:40.850 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-07-31T14:47:40.850 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-07-31T14:47:40.850 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-07-31T14:47:40.850 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-07-31T14:47:40.850 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-07-31T14:47:40.850 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-07-31T14:47:40.850 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-07-31T14:47:40.850 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-07-31T14:47:40.850 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-07-31T14:47:40.850 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-07-31T14:47:40.851 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-07-31T14:47:40.851 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-07-31T14:47:40.851 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-07-31T14:47:40.851 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-07-31T14:47:40.851 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-07-31T14:47:40.851 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-07-31T14:47:40.851 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-07-31T14:47:40.851 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-31T14:47:40.851 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-31T14:47:40.851 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-07-31T14:47:40.851 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-07-31T14:47:40.851 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-07-31T14:47:40.851 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay 2024-07-31T14:47:40.852 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/3e51295c991f928777c324004efe79d40527c29bf238dc5b2edc68feca2fd1d5/merged 2024-07-31T14:47:40.852 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/960135413c8674d8c7d3ce900e5fde26eec4343567449c6dbf16dcfa6613b6df/merged 2024-07-31T14:47:40.852 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/0 2024-07-31T14:47:40.852 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-07-31T14:47:40.852 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/04b9700cf4b62897feca63b7a3d46f15e56b4841d2b6180fcf1f47d213a9384b/merged 2024-07-31T14:47:40.852 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/b7ad35e28aeac3e4c01b37cf123ec218435cf54d7864e4c3382dce732d21e709/merged 2024-07-31T14:47:40.852 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/5f8c3be676d21da357b298f960248a21f12ae6f3e4c6198a403da7914139840d/merged 2024-07-31T14:47:40.852 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/75ef93bff7378615b42c695b19d7e84ee63615925067534357b0a85025d5a6e5/merged 2024-07-31T14:47:40.852 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/57d76a074936a1eb554829a12891c4c3d3b4f7d00d7d5b3fe5c6cc685d3c9108/merged 2024-07-31T14:47:40.852 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/dea51169dc1449bb67b32ddbb2c571501deca9f360b8213202e97b0e088ed706/merged 2024-07-31T14:47:40.852 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/227f0988408bc964ba64e4788d8aaf28d1ead804f24b102898a40dfbccca6bcf/merged 2024-07-31T14:47:40.852 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/18f95aae4c88e659f2390d08fcbe16e28abf93935403c898a5fb7be14fefcb8d/merged 2024-07-31T14:47:40.853 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-07-31T14:47:40.853 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-31T14:47:40.853 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-31T14:47:40.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:47:40.853 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-07-31T14:47:40.892 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:2024-07-31T14:47:40.891+0000 7f0bb631b480 -1 init, newargv = 0x7f0b98054370 newargc=15 2024-07-31T14:47:40.893 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[84518]: starting ceph client 2024-07-31T14:47:40.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:40 smithi073 ceph-mon[36714]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:40.899 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[84518]: starting fuse 2024-07-31T14:47:40.907 INFO:teuthology.orchestra.run.smithi067.stdout:119 2024-07-31T14:47:40.907 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-07-31T14:47:40.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-07-31T14:47:40.907 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-07-31T14:47:40.907 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-07-31T14:47:40.907 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-07-31T14:47:40.908 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-07-31T14:47:40.908 DEBUG:teuthology.orchestra.run.smithi067:> 2024-07-31T14:47:40.908 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-07-31T14:47:40.908 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-07-31T14:47:40.908 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-07-31T14:47:40.908 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-07-31T14:47:40.908 DEBUG:teuthology.orchestra.run.smithi067:> 2024-07-31T14:47:40.908 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-07-31T14:47:40.908 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-07-31T14:47:40.908 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-07-31T14:47:40.908 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-07-31T14:47:40.908 DEBUG:teuthology.orchestra.run.smithi067:> 2024-07-31T14:47:40.908 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-07-31T14:47:40.909 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-31T14:47:40.909 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-31T14:47:40.909 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-31T14:47:40.909 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-07-31T14:47:40.909 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-07-31T14:47:40.909 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-07-31T14:47:40.909 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-31T14:47:40.909 DEBUG:teuthology.orchestra.run.smithi067:> 2024-07-31T14:47:40.909 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.admin")) 2024-07-31T14:47:40.909 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-07-31T14:47:40.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:40 smithi077 ceph-mon[37327]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:41.029 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.admin.84518.asok 2024-07-31T14:47:41.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:47:41.039 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.84518.asok 2024-07-31T14:47:41.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:47:41.040 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.84518.asok status 2024-07-31T14:47:41.187 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:47:41.188 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-07-31T14:47:41.188 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "8c796fca5ff5960f7542e7227bad4a784ee2c804", 2024-07-31T14:47:41.188 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-5223-g8c796fca (8c796fca5ff5960f7542e7227bad4a784ee2c804) squid (dev)", 2024-07-31T14:47:41.188 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "admin", 2024-07-31T14:47:41.188 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-07-31T14:47:41.188 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-07-31T14:47:41.188 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "84518", 2024-07-31T14:47:41.188 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-07-31T14:47:41.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:47:41.188 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-07-31T14:47:41.188 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-07-31T14:47:41.188 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 14910, 2024-07-31T14:47:41.188 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-07-31T14:47:41.188 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-07-31T14:47:41.189 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-07-31T14:47:41.189 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 14910 2024-07-31T14:47:41.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:47:41.189 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-07-31T14:47:41.189 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-07-31T14:47:41.189 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-07-31T14:47:41.189 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2766459360 2024-07-31T14:47:41.189 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:47:41.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:47:41.189 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-07-31T14:47:41.189 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-07-31T14:47:41.189 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-07-31T14:47:41.189 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2766459360 2024-07-31T14:47:41.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:47:41.190 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.14910 192.168.0.1:0/2766459360", 2024-07-31T14:47:41.190 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/2766459360", 2024-07-31T14:47:41.190 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-07-31T14:47:41.190 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 17, 2024-07-31T14:47:41.190 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 89, 2024-07-31T14:47:41.190 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-07-31T14:47:41.190 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-07-31T14:47:41.190 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-07-31T14:47:41.190 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:47:41.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:47:41.196 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-07-31T14:47:41.196 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-07-31T14:47:41.212 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-07-31T14:47:41.213 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-07-31T14:47:41.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:47:41.213 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-07-31T14:47:41.279 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-07-31T14:47:41.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-31T14:47:41.394 INFO:teuthology.orchestra.run.smithi067.stdout:# file: . 2024-07-31T14:47:41.394 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-07-31T14:47:41.394 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:47:41.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:47:41.400 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:47:41.400 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-31T14:47:41.418 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd blocklist ls 2024-07-31T14:47:41.723 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:42.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:42 smithi067 ceph-mon[32750]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:42.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:42 smithi073 ceph-mon[36714]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:42.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:42 smithi077 ceph-mon[37327]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:43.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.514+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/55924659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7d0107bf0 msgr2=0x7fd7d0108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:43.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.514+0000 7fd7d4b9d640 1 --2- 172.21.15.67:0/55924659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7d0107bf0 0x7fd7d0108000 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009900 tx=0x7fd7b802a410 comp rx=0 tx=0).stop 2024-07-31T14:47:43.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.514+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/55924659 shutdown_connections 2024-07-31T14:47:43.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.514+0000 7fd7d4b9d640 1 --2- 172.21.15.67:0/55924659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7d0108540 0x7fd7d010ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:43.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.514+0000 7fd7d4b9d640 1 --2- 172.21.15.67:0/55924659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7d0107bf0 0x7fd7d0108000 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:43.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.514+0000 7fd7d4b9d640 1 --2- 172.21.15.67:0/55924659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7d01022b0 0x7fd7d01026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:43.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.514+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/55924659 >> 172.21.15.67:0/55924659 conn(0x7fd7d00fdd00 msgr2=0x7fd7d0100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:43.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.514+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/55924659 shutdown_connections 2024-07-31T14:47:43.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.514+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/55924659 wait complete. 2024-07-31T14:47:43.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.515+0000 7fd7d4b9d640 1 Processor -- start 2024-07-31T14:47:43.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.515+0000 7fd7d4b9d640 1 -- start start 2024-07-31T14:47:43.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.515+0000 7fd7d4b9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7d01022b0 0x7fd7d0199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:43.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.515+0000 7fd7d4b9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7d0107bf0 0x7fd7d019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:43.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.515+0000 7fd7d4b9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7d0108540 0x7fd7d019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:43.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.515+0000 7fd7d4b9d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0113320 con 0x7fd7d01022b0 2024-07-31T14:47:43.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.515+0000 7fd7d4b9d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01131a0 con 0x7fd7d0107bf0 2024-07-31T14:47:43.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.515+0000 7fd7d4b9d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01134a0 con 0x7fd7d0108540 2024-07-31T14:47:43.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7cf7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7d01022b0 0x7fd7d0199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:43.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7cf7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7d01022b0 0x7fd7d0199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48228/0 (socket says 172.21.15.67:48228) 2024-07-31T14:47:43.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7cf7fe640 1 -- 172.21.15.67:0/2756815972 learned_addr learned my addr 172.21.15.67:0/2756815972 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:43.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7cffff640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7d0108540 0x7fd7d019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:43.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7ceffd640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7d0107bf0 0x7fd7d019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:43.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7cf7fe640 1 -- 172.21.15.67:0/2756815972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7d0108540 msgr2=0x7fd7d019f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:43.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7cf7fe640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7d0108540 0x7fd7d019f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:43.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7cf7fe640 1 -- 172.21.15.67:0/2756815972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7d0107bf0 msgr2=0x7fd7d019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:43.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7cf7fe640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7d0107bf0 0x7fd7d019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:43.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7cf7fe640 1 -- 172.21.15.67:0/2756815972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d019fc80 con 0x7fd7d01022b0 2024-07-31T14:47:43.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7ceffd640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7d0107bf0 0x7fd7d019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:47:43.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7cf7fe640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7d01022b0 0x7fd7d0199d10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000d950 tx=0x7fd7c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:43.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7ccff9640 1 -- 172.21.15.67:0/2756815972 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0017070 con 0x7fd7d01022b0 2024-07-31T14:47:43.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/2756815972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d019ff70 con 0x7fd7d01022b0 2024-07-31T14:47:43.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7ccff9640 1 -- 172.21.15.67:0/2756815972 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c0009c30 con 0x7fd7d01022b0 2024-07-31T14:47:43.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.516+0000 7fd7ccff9640 1 -- 172.21.15.67:0/2756815972 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000f620 con 0x7fd7d01022b0 2024-07-31T14:47:43.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.517+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/2756815972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d01a66d0 con 0x7fd7d01022b0 2024-07-31T14:47:43.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.518+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/2756815972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d0103530 con 0x7fd7d01022b0 2024-07-31T14:47:43.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.519+0000 7fd7ccff9640 1 -- 172.21.15.67:0/2756815972 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fd7c0005030 con 0x7fd7d01022b0 2024-07-31T14:47:43.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.520+0000 7fd7ccff9640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7bc0b3c20 0x7fd7bc0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:43.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.520+0000 7fd7ceffd640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7bc0b3c20 0x7fd7bc0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:43.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.520+0000 7fd7ccff9640 1 -- 172.21.15.67:0/2756815972 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fd7c00e3ca0 con 0x7fd7d01022b0 2024-07-31T14:47:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.521+0000 7fd7ceffd640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7bc0b3c20 0x7fd7bc0b6130 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009230 tx=0x7fd7b8000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:43.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.523+0000 7fd7ccff9640 1 -- 172.21.15.67:0/2756815972 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd7c00aaec0 con 0x7fd7d01022b0 2024-07-31T14:47:43.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.731+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/2756815972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd7d010a370 con 0x7fd7d01022b0 2024-07-31T14:47:43.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.732+0000 7fd7ccff9640 1 -- 172.21.15.67:0/2756815972 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v89) ==== 81+0+884 (secure 0 0 0) 0x7fd7c0009de0 con 0x7fd7d01022b0 2024-07-31T14:47:43.733 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/3601135096 2024-08-01T14:40:19.281779+0000 2024-07-31T14:47:43.734 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/3601135096 2024-08-01T14:40:19.281779+0000 2024-07-31T14:47:43.734 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/4249598263 2024-08-01T14:39:07.942187+0000 2024-07-31T14:47:43.734 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/927641174 2024-08-01T14:38:41.486084+0000 2024-07-31T14:47:43.734 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3901988655 2024-08-01T14:38:41.486084+0000 2024-07-31T14:47:43.734 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/1712042445 2024-08-01T14:38:41.486084+0000 2024-07-31T14:47:43.734 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1243394894 2024-08-01T14:39:07.942187+0000 2024-07-31T14:47:43.734 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1563471445 2024-08-01T14:39:07.942187+0000 2024-07-31T14:47:43.734 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/1712042445 2024-08-01T14:38:41.486084+0000 2024-07-31T14:47:43.734 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/76992902 2024-08-01T14:39:07.942187+0000 2024-07-31T14:47:43.734 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3065587341 2024-08-01T14:38:41.486084+0000 2024-07-31T14:47:43.734 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/4249598263 2024-08-01T14:39:07.942187+0000 2024-07-31T14:47:43.734 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3411853619 2024-08-01T14:40:19.281779+0000 2024-07-31T14:47:43.734 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/618685030 2024-08-01T14:40:19.281779+0000 2024-07-31T14:47:43.735 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2614585786 2024-08-01T14:40:19.281779+0000 2024-07-31T14:47:43.735 INFO:teuthology.orchestra.run.smithi067.stderr:listed 15 entries 2024-07-31T14:47:43.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.734+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/2756815972 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7bc0b3c20 msgr2=0x7fd7bc0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:43.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.734+0000 7fd7d4b9d640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7bc0b3c20 0x7fd7bc0b6130 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009230 tx=0x7fd7b8000bc0 comp rx=0 tx=0).stop 2024-07-31T14:47:43.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.734+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/2756815972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7d01022b0 msgr2=0x7fd7d0199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:43.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.734+0000 7fd7d4b9d640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7d01022b0 0x7fd7d0199d10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000d950 tx=0x7fd7c000de10 comp rx=0 tx=0).stop 2024-07-31T14:47:43.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.734+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/2756815972 shutdown_connections 2024-07-31T14:47:43.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.734+0000 7fd7d4b9d640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7bc0b3c20 0x7fd7bc0b6130 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:43.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.734+0000 7fd7d4b9d640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7d0108540 0x7fd7d019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:43.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.734+0000 7fd7d4b9d640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7d0107bf0 0x7fd7d019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:43.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.734+0000 7fd7d4b9d640 1 --2- 172.21.15.67:0/2756815972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7d01022b0 0x7fd7d0199d10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:43.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.734+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/2756815972 >> 172.21.15.67:0/2756815972 conn(0x7fd7d00fdd00 msgr2=0x7fd7d0100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:43.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.734+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/2756815972 shutdown_connections 2024-07-31T14:47:43.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:43.734+0000 7fd7d4b9d640 1 -- 172.21.15.67:0/2756815972 wait complete. 2024-07-31T14:47:44.487 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:47:44.487 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-31T14:47:44.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd blocklist ls 2024-07-31T14:47:44.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:44 smithi067 ceph-mon[32750]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:44.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:44 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2756815972' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-31T14:47:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:44.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:44 smithi073 ceph-mon[36714]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:44.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:44 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2756815972' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-31T14:47:44.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:44 smithi077 ceph-mon[37327]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T14:47:44.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:44 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2756815972' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-31T14:47:46.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.549+0000 7f08960e0640 1 -- 172.21.15.67:0/2077067193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0890107df0 msgr2=0x7f0890108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:46.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.549+0000 7f08960e0640 1 --2- 172.21.15.67:0/2077067193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0890107df0 0x7f0890108200 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0878009900 tx=0x7f087802a410 comp rx=0 tx=0).stop 2024-07-31T14:47:46.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.549+0000 7f08960e0640 1 -- 172.21.15.67:0/2077067193 shutdown_connections 2024-07-31T14:47:46.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.549+0000 7f08960e0640 1 --2- 172.21.15.67:0/2077067193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0890108740 0x7f089010ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.549+0000 7f08960e0640 1 --2- 172.21.15.67:0/2077067193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0890107df0 0x7f0890108200 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.549+0000 7f08960e0640 1 --2- 172.21.15.67:0/2077067193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08901024b0 0x7f08901028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.549+0000 7f08960e0640 1 -- 172.21.15.67:0/2077067193 >> 172.21.15.67:0/2077067193 conn(0x7f0890079170 msgr2=0x7f0890079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.549+0000 7f08960e0640 1 -- 172.21.15.67:0/2077067193 shutdown_connections 2024-07-31T14:47:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.549+0000 7f08960e0640 1 -- 172.21.15.67:0/2077067193 wait complete. 2024-07-31T14:47:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f08960e0640 1 Processor -- start 2024-07-31T14:47:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f08960e0640 1 -- start start 2024-07-31T14:47:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f08960e0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08901024b0 0x7f0890199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f08960e0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0890107df0 0x7f089019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f08960e0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0890108740 0x7f089019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f08960e0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0890112390 con 0x7f08901024b0 2024-07-31T14:47:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f08960e0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0890112210 con 0x7f0890107df0 2024-07-31T14:47:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f08960e0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0890112510 con 0x7f0890108740 2024-07-31T14:47:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f088f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08901024b0 0x7f0890199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:46.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f088f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08901024b0 0x7f0890199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48246/0 (socket says 172.21.15.67:48246) 2024-07-31T14:47:46.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f088f7fe640 1 -- 172.21.15.67:0/2691695598 learned_addr learned my addr 172.21.15.67:0/2691695598 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:46.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f088ffff640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0890108740 0x7f089019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:46.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f0887fff640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0890107df0 0x7f089019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:46.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f088f7fe640 1 -- 172.21.15.67:0/2691695598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0890108740 msgr2=0x7f089019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:46.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f088f7fe640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0890108740 0x7f089019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:46.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f088f7fe640 1 -- 172.21.15.67:0/2691695598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0890107df0 msgr2=0x7f089019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:46.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f088f7fe640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0890107df0 0x7f089019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:46.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f088f7fe640 1 -- 172.21.15.67:0/2691695598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089019fec0 con 0x7f08901024b0 2024-07-31T14:47:46.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.550+0000 7f088ffff640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0890108740 0x7f089019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:47:46.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.551+0000 7f088f7fe640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08901024b0 0x7f0890199f50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f088000e9f0 tx=0x7f088000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:46.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.551+0000 7f088d7fa640 1 -- 172.21.15.67:0/2691695598 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088000cd80 con 0x7f08901024b0 2024-07-31T14:47:46.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.551+0000 7f088d7fa640 1 -- 172.21.15.67:0/2691695598 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08800108e0 con 0x7f08901024b0 2024-07-31T14:47:46.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.551+0000 7f088d7fa640 1 -- 172.21.15.67:0/2691695598 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880018980 con 0x7f08901024b0 2024-07-31T14:47:46.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.551+0000 7f08960e0640 1 -- 172.21.15.67:0/2691695598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08901a01b0 con 0x7f08901024b0 2024-07-31T14:47:46.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.551+0000 7f08960e0640 1 -- 172.21.15.67:0/2691695598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08901a6910 con 0x7f08901024b0 2024-07-31T14:47:46.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.552+0000 7f08960e0640 1 -- 172.21.15.67:0/2691695598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0890103730 con 0x7f08901024b0 2024-07-31T14:47:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:46 smithi067 ceph-mon[32750]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-31T14:47:46.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.555+0000 7f088d7fa640 1 -- 172.21.15.67:0/2691695598 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f0880010430 con 0x7f08901024b0 2024-07-31T14:47:46.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.555+0000 7f088d7fa640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f087c0b3ca0 0x7f087c0b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:46.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.555+0000 7f0887fff640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f087c0b3ca0 0x7f087c0b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:46.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.556+0000 7f088d7fa640 1 -- 172.21.15.67:0/2691695598 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f08800e4910 con 0x7f08901024b0 2024-07-31T14:47:46.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.556+0000 7f0887fff640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f087c0b3ca0 0x7f087c0b61b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0878000c00 tx=0x7f0878031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:46.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.557+0000 7f088d7fa640 1 -- 172.21.15.67:0/2691695598 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f08800abb30 con 0x7f08901024b0 2024-07-31T14:47:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.763+0000 7f08960e0640 1 -- 172.21.15.67:0/2691695598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f08901028c0 con 0x7f08901024b0 2024-07-31T14:47:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.764+0000 7f088d7fa640 1 -- 172.21.15.67:0/2691695598 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v89) ==== 81+0+884 (secure 0 0 0) 0x7f08800ab430 con 0x7f08901024b0 2024-07-31T14:47:46.765 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/3601135096 2024-08-01T14:40:19.281779+0000 2024-07-31T14:47:46.765 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/3601135096 2024-08-01T14:40:19.281779+0000 2024-07-31T14:47:46.765 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/4249598263 2024-08-01T14:39:07.942187+0000 2024-07-31T14:47:46.766 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/927641174 2024-08-01T14:38:41.486084+0000 2024-07-31T14:47:46.766 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3901988655 2024-08-01T14:38:41.486084+0000 2024-07-31T14:47:46.766 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/1712042445 2024-08-01T14:38:41.486084+0000 2024-07-31T14:47:46.766 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1243394894 2024-08-01T14:39:07.942187+0000 2024-07-31T14:47:46.766 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1563471445 2024-08-01T14:39:07.942187+0000 2024-07-31T14:47:46.766 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/1712042445 2024-08-01T14:38:41.486084+0000 2024-07-31T14:47:46.766 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/76992902 2024-08-01T14:39:07.942187+0000 2024-07-31T14:47:46.766 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3065587341 2024-08-01T14:38:41.486084+0000 2024-07-31T14:47:46.766 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/4249598263 2024-08-01T14:39:07.942187+0000 2024-07-31T14:47:46.766 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3411853619 2024-08-01T14:40:19.281779+0000 2024-07-31T14:47:46.766 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/618685030 2024-08-01T14:40:19.281779+0000 2024-07-31T14:47:46.766 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2614585786 2024-08-01T14:40:19.281779+0000 2024-07-31T14:47:46.767 INFO:teuthology.orchestra.run.smithi067.stderr:listed 15 entries 2024-07-31T14:47:46.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.766+0000 7f08960e0640 1 -- 172.21.15.67:0/2691695598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f087c0b3ca0 msgr2=0x7f087c0b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:46.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.766+0000 7f08960e0640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f087c0b3ca0 0x7f087c0b61b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0878000c00 tx=0x7f0878031400 comp rx=0 tx=0).stop 2024-07-31T14:47:46.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.766+0000 7f08960e0640 1 -- 172.21.15.67:0/2691695598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08901024b0 msgr2=0x7f0890199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.766+0000 7f08960e0640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08901024b0 0x7f0890199f50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f088000e9f0 tx=0x7f088000eeb0 comp rx=0 tx=0).stop 2024-07-31T14:47:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.766+0000 7f08960e0640 1 -- 172.21.15.67:0/2691695598 shutdown_connections 2024-07-31T14:47:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.766+0000 7f08960e0640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f087c0b3ca0 0x7f087c0b61b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.766+0000 7f08960e0640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0890108740 0x7f089019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.766+0000 7f08960e0640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0890107df0 0x7f089019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.766+0000 7f08960e0640 1 --2- 172.21.15.67:0/2691695598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08901024b0 0x7f0890199f50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.766+0000 7f08960e0640 1 -- 172.21.15.67:0/2691695598 >> 172.21.15.67:0/2691695598 conn(0x7f0890079170 msgr2=0x7f089006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.766+0000 7f08960e0640 1 -- 172.21.15.67:0/2691695598 shutdown_connections 2024-07-31T14:47:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:46.766+0000 7f08960e0640 1 -- 172.21.15.67:0/2691695598 wait complete. 2024-07-31T14:47:46.789 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:46 smithi077 ceph-mon[37327]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-31T14:47:46.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:46 smithi073 ceph-mon[36714]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-31T14:47:47.589 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-07-31T14:47:47.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:47:47.590 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-07-31T14:47:47.619 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[84518]: fuse finished with error 0 and tester_r 0 2024-07-31T14:47:47.621 INFO:teuthology.orchestra.run:waiting for 300 2024-07-31T14:47:47.671 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-07-31T14:47:47.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:47:47.672 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-07-31T14:47:47.695 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-07-31T14:47:47.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:47 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2691695598' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-31T14:47:47.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:47 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2691695598' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-31T14:47:47.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:47 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2691695598' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-31T14:47:48.001 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:48.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:48 smithi067 ceph-mon[32750]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-31T14:47:48.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:48 smithi073 ceph-mon[36714]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-31T14:47:48.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:48 smithi077 ceph-mon[37327]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-31T14:47:49.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.624+0000 7fe74fdf6640 1 -- 172.21.15.67:0/2817834227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe748107dd0 msgr2=0x7fe7481081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:49.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.624+0000 7fe74fdf6640 1 --2- 172.21.15.67:0/2817834227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe748107dd0 0x7fe7481081e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009900 tx=0x7fe73c02a410 comp rx=0 tx=0).stop 2024-07-31T14:47:49.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.624+0000 7fe74fdf6640 1 -- 172.21.15.67:0/2817834227 shutdown_connections 2024-07-31T14:47:49.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.624+0000 7fe74fdf6640 1 --2- 172.21.15.67:0/2817834227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe748108720 0x7fe74810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:49.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.624+0000 7fe74fdf6640 1 --2- 172.21.15.67:0/2817834227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe748107dd0 0x7fe7481081e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:49.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.624+0000 7fe74fdf6640 1 --2- 172.21.15.67:0/2817834227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe748102490 0x7fe7481028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:49.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.624+0000 7fe74fdf6640 1 -- 172.21.15.67:0/2817834227 >> 172.21.15.67:0/2817834227 conn(0x7fe748079160 msgr2=0x7fe748079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:49.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.624+0000 7fe74fdf6640 1 -- 172.21.15.67:0/2817834227 shutdown_connections 2024-07-31T14:47:49.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.625+0000 7fe74fdf6640 1 -- 172.21.15.67:0/2817834227 wait complete. 2024-07-31T14:47:49.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.625+0000 7fe74fdf6640 1 Processor -- start 2024-07-31T14:47:49.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.625+0000 7fe74fdf6640 1 -- start start 2024-07-31T14:47:49.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.625+0000 7fe74fdf6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe748102490 0x7fe74819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:49.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.625+0000 7fe74fdf6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe748107dd0 0x7fe74819b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:49.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.625+0000 7fe74fdf6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe748108720 0x7fe74819b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:49.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.625+0000 7fe74fdf6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe748112370 con 0x7fe748102490 2024-07-31T14:47:49.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.625+0000 7fe74fdf6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe7481121f0 con 0x7fe748108720 2024-07-31T14:47:49.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.625+0000 7fe74fdf6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe7481124f0 con 0x7fe748107dd0 2024-07-31T14:47:49.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe74db92640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe748102490 0x7fe74819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:49.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe74db92640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe748102490 0x7fe74819ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48262/0 (socket says 172.21.15.67:48262) 2024-07-31T14:47:49.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe74db92640 1 -- 172.21.15.67:0/3544309160 learned_addr learned my addr 172.21.15.67:0/3544309160 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:49.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe74d391640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe748107dd0 0x7fe74819b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:49.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe74e393640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe748108720 0x7fe74819b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:49.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe74db92640 1 -- 172.21.15.67:0/3544309160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe748107dd0 msgr2=0x7fe74819b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe74db92640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe748107dd0 0x7fe74819b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe74db92640 1 -- 172.21.15.67:0/3544309160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe748108720 msgr2=0x7fe74819b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe74db92640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe748108720 0x7fe74819b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe74db92640 1 -- 172.21.15.67:0/3544309160 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7481a5f10 con 0x7fe748102490 2024-07-31T14:47:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe74e393640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe748108720 0x7fe74819b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:47:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe74db92640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe748102490 0x7fe74819ac80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe74400e9f0 tx=0x7fe74400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe73affd640 1 -- 172.21.15.67:0/3544309160 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74400cd80 con 0x7fe748102490 2024-07-31T14:47:49.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe73affd640 1 -- 172.21.15.67:0/3544309160 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7440108e0 con 0x7fe748102490 2024-07-31T14:47:49.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe74fdf6640 1 -- 172.21.15.67:0/3544309160 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7481a6200 con 0x7fe748102490 2024-07-31T14:47:49.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe73affd640 1 -- 172.21.15.67:0/3544309160 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe744018980 con 0x7fe748102490 2024-07-31T14:47:49.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.626+0000 7fe74fdf6640 1 -- 172.21.15.67:0/3544309160 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7481a6760 con 0x7fe748102490 2024-07-31T14:47:49.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.628+0000 7fe74fdf6640 1 -- 172.21.15.67:0/3544309160 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7481028a0 con 0x7fe748102490 2024-07-31T14:47:49.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.629+0000 7fe73affd640 1 -- 172.21.15.67:0/3544309160 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fe744010430 con 0x7fe748102490 2024-07-31T14:47:49.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.630+0000 7fe73affd640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7340b3cd0 0x7fe7340b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:49.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.630+0000 7fe74d391640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7340b3cd0 0x7fe7340b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:49.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.630+0000 7fe73affd640 1 -- 172.21.15.67:0/3544309160 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fe7440e48f0 con 0x7fe748102490 2024-07-31T14:47:49.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.631+0000 7fe74d391640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7340b3cd0 0x7fe7340b61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe73c000c00 tx=0x7fe73c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:49.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.633+0000 7fe73affd640 1 -- 172.21.15.67:0/3544309160 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe7440abb10 con 0x7fe748102490 2024-07-31T14:47:49.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.830+0000 7fe74fdf6640 1 -- 172.21.15.67:0/3544309160 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe74802e0c0 con 0x7fe748102490 2024-07-31T14:47:49.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.832+0000 7fe73affd640 1 -- 172.21.15.67:0/3544309160 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24874 (secure 0 0 0) 0x7fe7440ab410 con 0x7fe748102490 2024-07-31T14:47:49.834 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:47:49.834 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T14:47:20.440064+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T14:47:49.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.834+0000 7fe74fdf6640 1 -- 172.21.15.67:0/3544309160 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7340b3cd0 msgr2=0x7fe7340b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:49.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.834+0000 7fe74fdf6640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7340b3cd0 0x7fe7340b61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe73c000c00 tx=0x7fe73c031400 comp rx=0 tx=0).stop 2024-07-31T14:47:49.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.834+0000 7fe74fdf6640 1 -- 172.21.15.67:0/3544309160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe748102490 msgr2=0x7fe74819ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:49.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.834+0000 7fe74fdf6640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe748102490 0x7fe74819ac80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe74400e9f0 tx=0x7fe74400eeb0 comp rx=0 tx=0).stop 2024-07-31T14:47:49.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.834+0000 7fe74fdf6640 1 -- 172.21.15.67:0/3544309160 shutdown_connections 2024-07-31T14:47:49.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.834+0000 7fe74fdf6640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7340b3cd0 0x7fe7340b61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:49.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.834+0000 7fe74fdf6640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe748108720 0x7fe74819b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:49.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.834+0000 7fe74fdf6640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe748107dd0 0x7fe74819b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:49.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.834+0000 7fe74fdf6640 1 --2- 172.21.15.67:0/3544309160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe748102490 0x7fe74819ac80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:49.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.834+0000 7fe74fdf6640 1 -- 172.21.15.67:0/3544309160 >> 172.21.15.67:0/3544309160 conn(0x7fe748079160 msgr2=0x7fe748104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:49.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.834+0000 7fe74fdf6640 1 -- 172.21.15.67:0/3544309160 shutdown_connections 2024-07-31T14:47:49.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:49.834+0000 7fe74fdf6640 1 -- 172.21.15.67:0/3544309160 wait complete. 2024-07-31T14:47:50.555 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-07-31T14:47:50.579 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:50 smithi067 ceph-mon[32750]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-31T14:47:50.579 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:50 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3544309160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:50.821 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:50.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:50 smithi073 ceph-mon[36714]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-31T14:47:50.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:50 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3544309160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:50.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:50 smithi077 ceph-mon[37327]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-31T14:47:50.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:50 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3544309160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:52.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.543+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/720220152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc881024d0 msgr2=0x7fcc881028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:52.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.543+0000 7fcc8ce8e640 1 --2- 172.21.15.67:0/720220152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc881024d0 0x7fcc881028e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcc78009900 tx=0x7fcc7802a410 comp rx=0 tx=0).stop 2024-07-31T14:47:52.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.543+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/720220152 shutdown_connections 2024-07-31T14:47:52.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.543+0000 7fcc8ce8e640 1 --2- 172.21.15.67:0/720220152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc88108760 0x7fcc8810ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:52.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.543+0000 7fcc8ce8e640 1 --2- 172.21.15.67:0/720220152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc88107e10 0x7fcc88108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:52.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.543+0000 7fcc8ce8e640 1 --2- 172.21.15.67:0/720220152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc881024d0 0x7fcc881028e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:52.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.543+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/720220152 >> 172.21.15.67:0/720220152 conn(0x7fcc88079190 msgr2=0x7fcc880795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:52.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.543+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/720220152 shutdown_connections 2024-07-31T14:47:52.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.543+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/720220152 wait complete. 2024-07-31T14:47:52.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:52 smithi067 ceph-mon[32750]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-31T14:47:52.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.544+0000 7fcc8ce8e640 1 Processor -- start 2024-07-31T14:47:52.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.544+0000 7fcc8ce8e640 1 -- start start 2024-07-31T14:47:52.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc8ce8e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc881024d0 0x7fcc8810fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:52.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc8ce8e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc88107e10 0x7fcc8810ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:52.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc8ce8e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc88108760 0x7fcc881104a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:52.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc8ce8e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcc881134f0 con 0x7fcc881024d0 2024-07-31T14:47:52.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc8ce8e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcc88113370 con 0x7fcc88107e10 2024-07-31T14:47:52.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc8ce8e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc88113670 con 0x7fcc88108760 2024-07-31T14:47:52.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc8659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc881024d0 0x7fcc8810fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:52.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc85d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc88107e10 0x7fcc8810ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:52.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc8659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc881024d0 0x7fcc8810fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48036/0 (socket says 172.21.15.67:48036) 2024-07-31T14:47:52.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc85d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc88107e10 0x7fcc8810ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:33566/0 (socket says 172.21.15.67:33566) 2024-07-31T14:47:52.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc8659c640 1 -- 172.21.15.67:0/4256643822 learned_addr learned my addr 172.21.15.67:0/4256643822 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:52.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc86d9d640 1 --2- 172.21.15.67:0/4256643822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc88108760 0x7fcc881104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:52.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc8659c640 1 -- 172.21.15.67:0/4256643822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc88108760 msgr2=0x7fcc881104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:52.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc8659c640 1 --2- 172.21.15.67:0/4256643822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc88108760 0x7fcc881104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:52.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc8659c640 1 -- 172.21.15.67:0/4256643822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc88107e10 msgr2=0x7fcc8810ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:52.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc8659c640 1 --2- 172.21.15.67:0/4256643822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc88107e10 0x7fcc8810ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:52.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc8659c640 1 -- 172.21.15.67:0/4256643822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc881a81c0 con 0x7fcc881024d0 2024-07-31T14:47:52.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.545+0000 7fcc86d9d640 1 --2- 172.21.15.67:0/4256643822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc88108760 0x7fcc881104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:47:52.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.546+0000 7fcc8659c640 1 --2- 172.21.15.67:0/4256643822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc881024d0 0x7fcc8810fa20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcc78005940 tx=0x7fcc780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:52.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.546+0000 7fcc777fe640 1 -- 172.21.15.67:0/4256643822 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7803f030 con 0x7fcc881024d0 2024-07-31T14:47:52.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.546+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/4256643822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc881a8450 con 0x7fcc881024d0 2024-07-31T14:47:52.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.546+0000 7fcc777fe640 1 -- 172.21.15.67:0/4256643822 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc78003cb0 con 0x7fcc881024d0 2024-07-31T14:47:52.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.546+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/4256643822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc881a8980 con 0x7fcc881024d0 2024-07-31T14:47:52.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.546+0000 7fcc777fe640 1 -- 172.21.15.67:0/4256643822 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78035070 con 0x7fcc881024d0 2024-07-31T14:47:52.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.547+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/4256643822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc88103750 con 0x7fcc881024d0 2024-07-31T14:47:52.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.549+0000 7fcc777fe640 1 -- 172.21.15.67:0/4256643822 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fcc78042030 con 0x7fcc881024d0 2024-07-31T14:47:52.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.550+0000 7fcc777fe640 1 --2- 172.21.15.67:0/4256643822 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc700b3c20 0x7fcc700b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:52.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.550+0000 7fcc85d9b640 1 --2- 172.21.15.67:0/4256643822 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc700b3c20 0x7fcc700b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:52.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.550+0000 7fcc777fe640 1 -- 172.21.15.67:0/4256643822 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fcc78104c30 con 0x7fcc881024d0 2024-07-31T14:47:52.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.551+0000 7fcc85d9b640 1 --2- 172.21.15.67:0/4256643822 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc700b3c20 0x7fcc700b6130 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcc881005a0 tx=0x7fcc7c00b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:52.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.553+0000 7fcc777fe640 1 -- 172.21.15.67:0/4256643822 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fcc780cbdf0 con 0x7fcc881024d0 2024-07-31T14:47:52.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.748+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/4256643822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc88110d20 con 0x7fcc881024d0 2024-07-31T14:47:52.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.750+0000 7fcc777fe640 1 -- 172.21.15.67:0/4256643822 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24874 (secure 0 0 0) 0x7fcc88110d20 con 0x7fcc881024d0 2024-07-31T14:47:52.752 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:47:52.752 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T14:47:20.440064+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T14:47:52.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.752+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/4256643822 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc700b3c20 msgr2=0x7fcc700b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:52.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.752+0000 7fcc8ce8e640 1 --2- 172.21.15.67:0/4256643822 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc700b3c20 0x7fcc700b6130 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcc881005a0 tx=0x7fcc7c00b410 comp rx=0 tx=0).stop 2024-07-31T14:47:52.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.752+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/4256643822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc881024d0 msgr2=0x7fcc8810fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:52.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.752+0000 7fcc8ce8e640 1 --2- 172.21.15.67:0/4256643822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc881024d0 0x7fcc8810fa20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcc78005940 tx=0x7fcc780042f0 comp rx=0 tx=0).stop 2024-07-31T14:47:52.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.752+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/4256643822 shutdown_connections 2024-07-31T14:47:52.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.752+0000 7fcc8ce8e640 1 --2- 172.21.15.67:0/4256643822 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc700b3c20 0x7fcc700b6130 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:52.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.752+0000 7fcc8ce8e640 1 --2- 172.21.15.67:0/4256643822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc88108760 0x7fcc881104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:52.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.752+0000 7fcc8ce8e640 1 --2- 172.21.15.67:0/4256643822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc88107e10 0x7fcc8810ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:52.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.752+0000 7fcc8ce8e640 1 --2- 172.21.15.67:0/4256643822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc881024d0 0x7fcc8810fa20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:52.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.752+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/4256643822 >> 172.21.15.67:0/4256643822 conn(0x7fcc88079190 msgr2=0x7fcc88100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:52.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.752+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/4256643822 shutdown_connections 2024-07-31T14:47:52.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:52.752+0000 7fcc8ce8e640 1 -- 172.21.15.67:0/4256643822 wait complete. 2024-07-31T14:47:52.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:52 smithi073 ceph-mon[36714]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-31T14:47:52.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:52 smithi077 ceph-mon[37327]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-31T14:47:53.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-07-31T14:47:53.759 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:53.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:53 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/4256643822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:53.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:53 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/4256643822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:53.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:53 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/4256643822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:47:54.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:54 smithi067 ceph-mon[32750]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-31T14:47:54.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:54 smithi073 ceph-mon[36714]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-31T14:47:54.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:54 smithi077 ceph-mon[37327]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-31T14:47:55.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.462+0000 7f2db3aec640 1 -- 172.21.15.67:0/3803629506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac102970 msgr2=0x7f2dac10f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:55.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.462+0000 7f2db3aec640 1 --2- 172.21.15.67:0/3803629506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac102970 0x7f2dac10f690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009900 tx=0x7f2d9c02a480 comp rx=0 tx=0).stop 2024-07-31T14:47:55.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.462+0000 7f2db3aec640 1 -- 172.21.15.67:0/3803629506 shutdown_connections 2024-07-31T14:47:55.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.462+0000 7f2db3aec640 1 --2- 172.21.15.67:0/3803629506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dac10fbd0 0x7f2dac111ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:55.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.462+0000 7f2db3aec640 1 --2- 172.21.15.67:0/3803629506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac102970 0x7f2dac10f690 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:55.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.462+0000 7f2db3aec640 1 --2- 172.21.15.67:0/3803629506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2dac102020 0x7f2dac102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:55.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.462+0000 7f2db3aec640 1 -- 172.21.15.67:0/3803629506 >> 172.21.15.67:0/3803629506 conn(0x7f2dac079170 msgr2=0x7f2dac079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:55.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.462+0000 7f2db3aec640 1 -- 172.21.15.67:0/3803629506 shutdown_connections 2024-07-31T14:47:55.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.462+0000 7f2db3aec640 1 -- 172.21.15.67:0/3803629506 wait complete. 2024-07-31T14:47:55.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.462+0000 7f2db3aec640 1 Processor -- start 2024-07-31T14:47:55.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db3aec640 1 -- start start 2024-07-31T14:47:55.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db3aec640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac102020 0x7f2dac19c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:55.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db3aec640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dac102970 0x7f2dac19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:55.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db3aec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2dac10fbd0 0x7f2dac1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:55.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db3aec640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac115660 con 0x7f2dac102020 2024-07-31T14:47:55.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db3aec640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac1154e0 con 0x7f2dac102970 2024-07-31T14:47:55.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db3aec640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac1157e0 con 0x7f2dac10fbd0 2024-07-31T14:47:55.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db1888640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac102020 0x7f2dac19c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:55.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db1888640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac102020 0x7f2dac19c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48046/0 (socket says 172.21.15.67:48046) 2024-07-31T14:47:55.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db1888640 1 -- 172.21.15.67:0/3560156551 learned_addr learned my addr 172.21.15.67:0/3560156551 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:55.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db1087640 1 --2- 172.21.15.67:0/3560156551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dac102970 0x7f2dac19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:55.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db2089640 1 --2- 172.21.15.67:0/3560156551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2dac10fbd0 0x7f2dac1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:55.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db1888640 1 -- 172.21.15.67:0/3560156551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2dac10fbd0 msgr2=0x7f2dac1a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:55.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db1888640 1 --2- 172.21.15.67:0/3560156551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2dac10fbd0 0x7f2dac1a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:55.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db1888640 1 -- 172.21.15.67:0/3560156551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dac102970 msgr2=0x7f2dac19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:55.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db1888640 1 --2- 172.21.15.67:0/3560156551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dac102970 0x7f2dac19c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:55.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.463+0000 7f2db1888640 1 -- 172.21.15.67:0/3560156551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dac1a2000 con 0x7f2dac102020 2024-07-31T14:47:55.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.464+0000 7f2db1888640 1 --2- 172.21.15.67:0/3560156551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac102020 0x7f2dac19c090 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2da800d950 tx=0x7f2da800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:55.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.464+0000 7f2da2ffd640 1 -- 172.21.15.67:0/3560156551 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8017030 con 0x7f2dac102020 2024-07-31T14:47:55.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.464+0000 7f2db3aec640 1 -- 172.21.15.67:0/3560156551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dac1a22f0 con 0x7f2dac102020 2024-07-31T14:47:55.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.464+0000 7f2da2ffd640 1 -- 172.21.15.67:0/3560156551 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2da8004020 con 0x7f2dac102020 2024-07-31T14:47:55.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.464+0000 7f2da2ffd640 1 -- 172.21.15.67:0/3560156551 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da80047a0 con 0x7f2dac102020 2024-07-31T14:47:55.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.464+0000 7f2db3aec640 1 -- 172.21.15.67:0/3560156551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dac1a8a50 con 0x7f2dac102020 2024-07-31T14:47:55.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.465+0000 7f2db3aec640 1 -- 172.21.15.67:0/3560156551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dac06bc70 con 0x7f2dac102020 2024-07-31T14:47:55.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.466+0000 7f2da2ffd640 1 -- 172.21.15.67:0/3560156551 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f2da8009a30 con 0x7f2dac102020 2024-07-31T14:47:55.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.467+0000 7f2da2ffd640 1 --2- 172.21.15.67:0/3560156551 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2d980b3c20 0x7f2d980b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:55.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.467+0000 7f2db1087640 1 --2- 172.21.15.67:0/3560156551 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2d980b3c20 0x7f2d980b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:55.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.467+0000 7f2da2ffd640 1 -- 172.21.15.67:0/3560156551 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f2da80e41b0 con 0x7f2dac102020 2024-07-31T14:47:55.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.468+0000 7f2db1087640 1 --2- 172.21.15.67:0/3560156551 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2d980b3c20 0x7f2d980b6130 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c031440 tx=0x7f2d9c009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:55.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.470+0000 7f2da2ffd640 1 -- 172.21.15.67:0/3560156551 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f2da80ab3d0 con 0x7f2dac102020 2024-07-31T14:47:55.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:55.707+0000 7f2db3aec640 1 -- 172.21.15.67:0/3560156551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2dac102430 con 0x7f2dac102020 2024-07-31T14:47:56.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:56.535+0000 7f2da2ffd640 1 -- 172.21.15.67:0/3560156551 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f2da80aacd0 con 0x7f2dac102020 2024-07-31T14:47:56.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:56.538+0000 7f2db3aec640 1 -- 172.21.15.67:0/3560156551 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2d980b3c20 msgr2=0x7f2d980b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:56.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:56.538+0000 7f2db3aec640 1 --2- 172.21.15.67:0/3560156551 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2d980b3c20 0x7f2d980b6130 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c031440 tx=0x7f2d9c009560 comp rx=0 tx=0).stop 2024-07-31T14:47:56.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:56.538+0000 7f2db3aec640 1 -- 172.21.15.67:0/3560156551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac102020 msgr2=0x7f2dac19c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:56.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:56.538+0000 7f2db3aec640 1 --2- 172.21.15.67:0/3560156551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac102020 0x7f2dac19c090 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2da800d950 tx=0x7f2da800de10 comp rx=0 tx=0).stop 2024-07-31T14:47:56.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:56.538+0000 7f2db3aec640 1 -- 172.21.15.67:0/3560156551 shutdown_connections 2024-07-31T14:47:56.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:56.538+0000 7f2db3aec640 1 --2- 172.21.15.67:0/3560156551 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2d980b3c20 0x7f2d980b6130 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:56.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:56.538+0000 7f2db3aec640 1 --2- 172.21.15.67:0/3560156551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2dac10fbd0 0x7f2dac1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:56.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:56.538+0000 7f2db3aec640 1 --2- 172.21.15.67:0/3560156551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dac102970 0x7f2dac19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:56.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:56.538+0000 7f2db3aec640 1 --2- 172.21.15.67:0/3560156551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac102020 0x7f2dac19c090 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:56.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:56.538+0000 7f2db3aec640 1 -- 172.21.15.67:0/3560156551 >> 172.21.15.67:0/3560156551 conn(0x7f2dac079170 msgr2=0x7f2dac109770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:56.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:56.538+0000 7f2db3aec640 1 -- 172.21.15.67:0/3560156551 shutdown_connections 2024-07-31T14:47:56.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:56.538+0000 7f2db3aec640 1 -- 172.21.15.67:0/3560156551 wait complete. 2024-07-31T14:47:56.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:56 smithi067 ceph-mon[32750]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-31T14:47:56.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:56 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3560156551' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-31T14:47:56.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:56 smithi073 ceph-mon[36714]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-31T14:47:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:56 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3560156551' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-31T14:47:56.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:56 smithi077 ceph-mon[37327]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-31T14:47:56.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:56 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3560156551' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-31T14:47:57.205 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-07-31T14:47:57.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-07-31T14:47:57.476 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:47:57.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:57 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3560156551' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-31T14:47:57.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:57 smithi067 ceph-mon[32750]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:47:57.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:57 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3560156551' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-31T14:47:57.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:57 smithi073 ceph-mon[36714]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:47:57.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:57 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3560156551' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-31T14:47:57.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:57 smithi077 ceph-mon[37327]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:47:58.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:58 smithi067 ceph-mon[32750]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-31T14:47:58.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:58 smithi073 ceph-mon[36714]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-31T14:47:58.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:58 smithi077 ceph-mon[37327]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-31T14:47:59.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.146+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1618457673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d0107df0 msgr2=0x7f41d0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:59.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.146+0000 7f41d6ffb640 1 --2- 172.21.15.67:0/1618457673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d0107df0 0x7f41d0108200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009ca0 tx=0x7f41c002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:47:59.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.147+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1618457673 shutdown_connections 2024-07-31T14:47:59.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.147+0000 7f41d6ffb640 1 --2- 172.21.15.67:0/1618457673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d0108740 0x7f41d010ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:59.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.147+0000 7f41d6ffb640 1 --2- 172.21.15.67:0/1618457673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d0107df0 0x7f41d0108200 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:59.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.147+0000 7f41d6ffb640 1 --2- 172.21.15.67:0/1618457673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41d01024b0 0x7f41d01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:59.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.147+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1618457673 >> 172.21.15.67:0/1618457673 conn(0x7f41d0079170 msgr2=0x7f41d0079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:59.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.147+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1618457673 shutdown_connections 2024-07-31T14:47:59.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.147+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1618457673 wait complete. 2024-07-31T14:47:59.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.147+0000 7f41d6ffb640 1 Processor -- start 2024-07-31T14:47:59.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.147+0000 7f41d6ffb640 1 -- start start 2024-07-31T14:47:59.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d6ffb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d01024b0 0x7f41d0104b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:59.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d6ffb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41d0107df0 0x7f41d0105070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:59.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d6ffb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d0108740 0x7f41d01055b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:59.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d6ffb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f41d0112370 con 0x7f41d01024b0 2024-07-31T14:47:59.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d6ffb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f41d01121f0 con 0x7f41d0107df0 2024-07-31T14:47:59.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d6ffb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41d01124f0 con 0x7f41d0108740 2024-07-31T14:47:59.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d4d97640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d01024b0 0x7f41d0104b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:59.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d4d97640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d01024b0 0x7f41d0104b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48062/0 (socket says 172.21.15.67:48062) 2024-07-31T14:47:59.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d4d97640 1 -- 172.21.15.67:0/1906351773 learned_addr learned my addr 172.21.15.67:0/1906351773 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:47:59.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41cffff640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41d0107df0 0x7f41d0105070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:59.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d5598640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d0108740 0x7f41d01055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:59.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d4d97640 1 -- 172.21.15.67:0/1906351773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d0108740 msgr2=0x7f41d01055b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:59.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d4d97640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d0108740 0x7f41d01055b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:59.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d4d97640 1 -- 172.21.15.67:0/1906351773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41d0107df0 msgr2=0x7f41d0105070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:59.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d4d97640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41d0107df0 0x7f41d0105070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:59.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d4d97640 1 -- 172.21.15.67:0/1906351773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d01a8160 con 0x7f41d01024b0 2024-07-31T14:47:59.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.148+0000 7f41d5598640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d0108740 0x7f41d01055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:47:59.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.149+0000 7f41d4d97640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d01024b0 0x7f41d0104b30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f41b800c950 tx=0x7f41b800ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:59.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.149+0000 7f41cdffb640 1 -- 172.21.15.67:0/1906351773 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b8007c70 con 0x7f41d01024b0 2024-07-31T14:47:59.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.149+0000 7f41cdffb640 1 -- 172.21.15.67:0/1906351773 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41b8007e10 con 0x7f41d01024b0 2024-07-31T14:47:59.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.149+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1906351773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d01a8450 con 0x7f41d01024b0 2024-07-31T14:47:59.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.149+0000 7f41cdffb640 1 -- 172.21.15.67:0/1906351773 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b80187f0 con 0x7f41d01024b0 2024-07-31T14:47:59.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.149+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1906351773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d01a8960 con 0x7f41d01024b0 2024-07-31T14:47:59.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.151+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1906351773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d0103730 con 0x7f41d01024b0 2024-07-31T14:47:59.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.152+0000 7f41cdffb640 1 -- 172.21.15.67:0/1906351773 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f41b8018990 con 0x7f41d01024b0 2024-07-31T14:47:59.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.153+0000 7f41cdffb640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41bc0b3c20 0x7f41bc0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:47:59.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.153+0000 7f41cffff640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41bc0b3c20 0x7f41bc0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:47:59.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.153+0000 7f41cdffb640 1 -- 172.21.15.67:0/1906351773 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f41b80e41e0 con 0x7f41d01024b0 2024-07-31T14:47:59.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.154+0000 7f41cffff640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41bc0b3c20 0x7f41bc0b6130 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f41c0005e30 tx=0x7f41c0009360 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:47:59.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.158+0000 7f41cdffb640 1 -- 172.21.15.67:0/1906351773 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f41b8014030 con 0x7f41d01024b0 2024-07-31T14:47:59.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.399+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1906351773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f41d010b5a0 con 0x7f41d01024b0 2024-07-31T14:47:59.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.548+0000 7f41cdffb640 1 -- 172.21.15.67:0/1906351773 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f41b80ab400 con 0x7f41d01024b0 2024-07-31T14:47:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.550+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1906351773 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41bc0b3c20 msgr2=0x7f41bc0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.550+0000 7f41d6ffb640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41bc0b3c20 0x7f41bc0b6130 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f41c0005e30 tx=0x7f41c0009360 comp rx=0 tx=0).stop 2024-07-31T14:47:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.550+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1906351773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d01024b0 msgr2=0x7f41d0104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:47:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.550+0000 7f41d6ffb640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d01024b0 0x7f41d0104b30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f41b800c950 tx=0x7f41b800ce10 comp rx=0 tx=0).stop 2024-07-31T14:47:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.550+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1906351773 shutdown_connections 2024-07-31T14:47:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.550+0000 7f41d6ffb640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41bc0b3c20 0x7f41bc0b6130 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.550+0000 7f41d6ffb640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d0108740 0x7f41d01055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:59.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.550+0000 7f41d6ffb640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41d0107df0 0x7f41d0105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:59.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.550+0000 7f41d6ffb640 1 --2- 172.21.15.67:0/1906351773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d01024b0 0x7f41d0104b30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:47:59.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.550+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1906351773 >> 172.21.15.67:0/1906351773 conn(0x7f41d0079170 msgr2=0x7f41d006b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:47:59.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.551+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1906351773 shutdown_connections 2024-07-31T14:47:59.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:47:59.551+0000 7f41d6ffb640 1 -- 172.21.15.67:0/1906351773 wait complete. 2024-07-31T14:47:59.753 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:59 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:47:59.753 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:47:59 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1906351773' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-31T14:47:59.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:59 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:47:59.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:47:59 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1906351773' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-31T14:47:59.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:59 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:47:59.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:47:59 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1906351773' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-31T14:48:00.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-07-31T14:48:00.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:00 smithi067 ceph-mon[32750]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-31T14:48:00.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:00 smithi067 ceph-mon[32750]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-31T14:48:00.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:00 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1906351773' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-07-31T14:48:00.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:00 smithi067 ceph-mon[32750]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-31T14:48:00.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:00 smithi067 ceph-mon[32750]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:48:00.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:00 smithi067 ceph-mon[32750]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-07-31T14:48:00.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:00 smithi067 ceph-mon[32750]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-07-31T14:48:00.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:00.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:00.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:00.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:00.839 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:00.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:00 smithi073 ceph-mon[36714]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-31T14:48:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:00 smithi073 ceph-mon[36714]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-31T14:48:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:00 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1906351773' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-07-31T14:48:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:00 smithi073 ceph-mon[36714]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-31T14:48:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:00 smithi073 ceph-mon[36714]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:48:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:00 smithi073 ceph-mon[36714]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-07-31T14:48:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:00 smithi073 ceph-mon[36714]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-07-31T14:48:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:00.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:00 smithi077 ceph-mon[37327]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-31T14:48:00.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:00 smithi077 ceph-mon[37327]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-31T14:48:00.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:00 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1906351773' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-07-31T14:48:00.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:00 smithi077 ceph-mon[37327]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-31T14:48:00.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:00 smithi077 ceph-mon[37327]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-31T14:48:00.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:00 smithi077 ceph-mon[37327]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-07-31T14:48:00.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:00 smithi077 ceph-mon[37327]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-07-31T14:48:00.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:00.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:00.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:00.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:01.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 14:48:01 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T14:48:01.663+0000 7f2e996a6640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-07-31T14:48:01.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:01 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] up:active 2024-07-31T14:48:01.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:01 smithi073 ceph-mon[36714]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-31T14:48:01.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:01 smithi073 ceph-mon[36714]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-07-31T14:48:01.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:01 smithi073 ceph-mon[36714]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-07-31T14:48:01.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:01 smithi073 ceph-mon[36714]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-07-31T14:48:01.959 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:01 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] up:active 2024-07-31T14:48:01.959 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:01 smithi067 ceph-mon[32750]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-31T14:48:01.959 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:01 smithi067 ceph-mon[32750]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-07-31T14:48:01.960 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:01 smithi067 ceph-mon[32750]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-07-31T14:48:01.960 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:01 smithi067 ceph-mon[32750]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-07-31T14:48:01.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:01 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] up:active 2024-07-31T14:48:01.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:01 smithi077 ceph-mon[37327]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-31T14:48:01.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:01 smithi077 ceph-mon[37327]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-07-31T14:48:01.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:01 smithi077 ceph-mon[37327]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-07-31T14:48:01.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:01 smithi077 ceph-mon[37327]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-07-31T14:48:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.483+0000 7ff60f634640 1 -- 172.21.15.67:0/578998343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff608108740 msgr2=0x7ff60810ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.483+0000 7ff60f634640 1 --2- 172.21.15.67:0/578998343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff608108740 0x7ff60810ef70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff600009ca0 tx=0x7ff60002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:48:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.483+0000 7ff60f634640 1 -- 172.21.15.67:0/578998343 shutdown_connections 2024-07-31T14:48:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.483+0000 7ff60f634640 1 --2- 172.21.15.67:0/578998343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff608108740 0x7ff60810ef70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.483+0000 7ff60f634640 1 --2- 172.21.15.67:0/578998343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff608107df0 0x7ff608108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.483+0000 7ff60f634640 1 --2- 172.21.15.67:0/578998343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6081024b0 0x7ff6081028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.483+0000 7ff60f634640 1 -- 172.21.15.67:0/578998343 >> 172.21.15.67:0/578998343 conn(0x7ff608079170 msgr2=0x7ff608079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.483+0000 7ff60f634640 1 -- 172.21.15.67:0/578998343 shutdown_connections 2024-07-31T14:48:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.483+0000 7ff60f634640 1 -- 172.21.15.67:0/578998343 wait complete. 2024-07-31T14:48:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.484+0000 7ff60f634640 1 Processor -- start 2024-07-31T14:48:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.484+0000 7ff60f634640 1 -- start start 2024-07-31T14:48:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.484+0000 7ff60f634640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6081024b0 0x7ff6080ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.484+0000 7ff60f634640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff608107df0 0x7ff6080ff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.484+0000 7ff60f634640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff608108740 0x7ff6080ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.484+0000 7ff60f634640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff6081123a0 con 0x7ff608108740 2024-07-31T14:48:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.484+0000 7ff60f634640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff608112220 con 0x7ff608107df0 2024-07-31T14:48:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.484+0000 7ff60f634640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff608112520 con 0x7ff6081024b0 2024-07-31T14:48:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.484+0000 7ff60dbd1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff608108740 0x7ff6080ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.484+0000 7ff60dbd1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff608108740 0x7ff6080ffb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60286/0 (socket says 172.21.15.67:60286) 2024-07-31T14:48:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.484+0000 7ff60dbd1640 1 -- 172.21.15.67:0/480323609 learned_addr learned my addr 172.21.15.67:0/480323609 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.484+0000 7ff60cbcf640 1 --2- 172.21.15.67:0/480323609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff608107df0 0x7ff6080ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.485+0000 7ff60d3d0640 1 --2- 172.21.15.67:0/480323609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6081024b0 0x7ff6080ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.485+0000 7ff60dbd1640 1 -- 172.21.15.67:0/480323609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6081024b0 msgr2=0x7ff6080ff0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.485+0000 7ff60dbd1640 1 --2- 172.21.15.67:0/480323609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6081024b0 0x7ff6080ff0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.485+0000 7ff60dbd1640 1 -- 172.21.15.67:0/480323609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff608107df0 msgr2=0x7ff6080ff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.485+0000 7ff60dbd1640 1 --2- 172.21.15.67:0/480323609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff608107df0 0x7ff6080ff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.485+0000 7ff60dbd1640 1 -- 172.21.15.67:0/480323609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6081a8120 con 0x7ff608108740 2024-07-31T14:48:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.485+0000 7ff60dbd1640 1 --2- 172.21.15.67:0/480323609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff608108740 0x7ff6080ffb40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff600005cc0 tx=0x7ff60002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:02.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.485+0000 7ff5fe7fc640 1 -- 172.21.15.67:0/480323609 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600003bf0 con 0x7ff608108740 2024-07-31T14:48:02.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.485+0000 7ff60f634640 1 -- 172.21.15.67:0/480323609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6081a83b0 con 0x7ff608108740 2024-07-31T14:48:02.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.485+0000 7ff5fe7fc640 1 -- 172.21.15.67:0/480323609 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff600003d90 con 0x7ff608108740 2024-07-31T14:48:02.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.485+0000 7ff5fe7fc640 1 -- 172.21.15.67:0/480323609 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60002fbc0 con 0x7ff608108740 2024-07-31T14:48:02.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.485+0000 7ff60f634640 1 -- 172.21.15.67:0/480323609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6081a8940 con 0x7ff608108740 2024-07-31T14:48:02.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.486+0000 7ff60f634640 1 -- 172.21.15.67:0/480323609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff608103730 con 0x7ff608108740 2024-07-31T14:48:02.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.489+0000 7ff5fe7fc640 1 -- 172.21.15.67:0/480323609 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7ff60002fd60 con 0x7ff608108740 2024-07-31T14:48:02.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.490+0000 7ff5fe7fc640 1 --2- 172.21.15.67:0/480323609 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5f40b3c20 0x7ff5f40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:02.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.490+0000 7ff60d3d0640 1 --2- 172.21.15.67:0/480323609 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5f40b3c20 0x7ff5f40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:02.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.490+0000 7ff5fe7fc640 1 -- 172.21.15.67:0/480323609 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff600105940 con 0x7ff608108740 2024-07-31T14:48:02.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.490+0000 7ff60d3d0640 1 --2- 172.21.15.67:0/480323609 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5f40b3c20 0x7ff5f40b6130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff608100b10 tx=0x7ff60400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:02.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.493+0000 7ff5fe7fc640 1 -- 172.21.15.67:0/480323609 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff6000ccb00 con 0x7ff608108740 2024-07-31T14:48:02.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:02.735+0000 7ff60f634640 1 -- 172.21.15.67:0/480323609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7ff6081028c0 con 0x7ff608108740 2024-07-31T14:48:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:02 smithi073 ceph-mon[36714]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-07-31T14:48:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:02 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] up:active 2024-07-31T14:48:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:02 smithi073 ceph-mon[36714]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-07-31T14:48:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:02 smithi073 ceph-mon[36714]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-07-31T14:48:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:02 smithi073 ceph-mon[36714]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-07-31T14:48:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:02 smithi073 ceph-mon[36714]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-07-31T14:48:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:48:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:48:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:02.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:02 smithi077 ceph-mon[37327]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-07-31T14:48:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:02 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] up:active 2024-07-31T14:48:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:02 smithi077 ceph-mon[37327]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-07-31T14:48:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:02 smithi077 ceph-mon[37327]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-07-31T14:48:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:02 smithi077 ceph-mon[37327]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-07-31T14:48:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:02 smithi077 ceph-mon[37327]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-07-31T14:48:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:48:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:48:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:02 smithi067 ceph-mon[32750]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-07-31T14:48:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:02 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] up:active 2024-07-31T14:48:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:02 smithi067 ceph-mon[32750]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-07-31T14:48:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:02 smithi067 ceph-mon[32750]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-07-31T14:48:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:02 smithi067 ceph-mon[32750]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-07-31T14:48:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:02 smithi067 ceph-mon[32750]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-07-31T14:48:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:03.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:03.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:48:03.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:48:03.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:48:03.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:03.592+0000 7ff5fe7fc640 1 -- 172.21.15.67:0/480323609 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7ff6081028c0 con 0x7ff608108740 2024-07-31T14:48:03.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:03.595+0000 7ff60f634640 1 -- 172.21.15.67:0/480323609 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5f40b3c20 msgr2=0x7ff5f40b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:03.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:03.595+0000 7ff60f634640 1 --2- 172.21.15.67:0/480323609 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5f40b3c20 0x7ff5f40b6130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff608100b10 tx=0x7ff60400b390 comp rx=0 tx=0).stop 2024-07-31T14:48:03.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:03.595+0000 7ff60f634640 1 -- 172.21.15.67:0/480323609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff608108740 msgr2=0x7ff6080ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:03.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:03.595+0000 7ff60f634640 1 --2- 172.21.15.67:0/480323609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff608108740 0x7ff6080ffb40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff600005cc0 tx=0x7ff60002f380 comp rx=0 tx=0).stop 2024-07-31T14:48:03.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:03.595+0000 7ff60f634640 1 -- 172.21.15.67:0/480323609 shutdown_connections 2024-07-31T14:48:03.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:03.595+0000 7ff60f634640 1 --2- 172.21.15.67:0/480323609 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5f40b3c20 0x7ff5f40b6130 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:03.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:03.595+0000 7ff60f634640 1 --2- 172.21.15.67:0/480323609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff608108740 0x7ff6080ffb40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:03.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:03.595+0000 7ff60f634640 1 --2- 172.21.15.67:0/480323609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff608107df0 0x7ff6080ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:03.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:03.595+0000 7ff60f634640 1 --2- 172.21.15.67:0/480323609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6081024b0 0x7ff6080ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:03.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:03.595+0000 7ff60f634640 1 -- 172.21.15.67:0/480323609 >> 172.21.15.67:0/480323609 conn(0x7ff608079170 msgr2=0x7ff60806b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:03.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:03.595+0000 7ff60f634640 1 -- 172.21.15.67:0/480323609 shutdown_connections 2024-07-31T14:48:03.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:03.595+0000 7ff60f634640 1 -- 172.21.15.67:0/480323609 wait complete. 2024-07-31T14:48:03.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:03 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] up:active 2024-07-31T14:48:03.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:03 smithi073 ceph-mon[36714]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-07-31T14:48:03.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:03 smithi073 ceph-mon[36714]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-31T14:48:03.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:03 smithi073 ceph-mon[36714]: Cluster is now healthy 2024-07-31T14:48:03.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:03 smithi073 ceph-mon[36714]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-07-31T14:48:03.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:03 smithi073 ceph-mon[36714]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-07-31T14:48:03.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:03 smithi073 ceph-mon[36714]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-07-31T14:48:03.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:03 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/480323609' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-31T14:48:03.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:03 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] up:active 2024-07-31T14:48:03.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:03 smithi067 ceph-mon[32750]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-07-31T14:48:03.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:03 smithi067 ceph-mon[32750]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-31T14:48:03.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:03 smithi067 ceph-mon[32750]: Cluster is now healthy 2024-07-31T14:48:03.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:03 smithi067 ceph-mon[32750]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-07-31T14:48:03.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:03 smithi067 ceph-mon[32750]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-07-31T14:48:03.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:03 smithi067 ceph-mon[32750]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-07-31T14:48:03.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:03 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/480323609' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-31T14:48:03.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:03 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] up:active 2024-07-31T14:48:03.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:03 smithi077 ceph-mon[37327]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-07-31T14:48:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:03 smithi077 ceph-mon[37327]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-31T14:48:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:03 smithi077 ceph-mon[37327]: Cluster is now healthy 2024-07-31T14:48:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:03 smithi077 ceph-mon[37327]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-07-31T14:48:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:03 smithi077 ceph-mon[37327]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-07-31T14:48:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:03 smithi077 ceph-mon[37327]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-07-31T14:48:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:03 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/480323609' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-31T14:48:04.214 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-07-31T14:48:04.487 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:04.846 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:04 smithi067 ceph-mon[32750]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-07-31T14:48:04.846 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:04 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/480323609' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-31T14:48:04.846 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:04 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] up:active 2024-07-31T14:48:04.846 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:04 smithi067 ceph-mon[32750]: fsmap cephfs:5 7 up:standby 5 up:active 2024-07-31T14:48:04.846 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:04 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:48:04.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:04 smithi073 ceph-mon[36714]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-07-31T14:48:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:04 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/480323609' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-31T14:48:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:04 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] up:active 2024-07-31T14:48:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:04 smithi073 ceph-mon[36714]: fsmap cephfs:5 7 up:standby 5 up:active 2024-07-31T14:48:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:04 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:48:04.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:04 smithi077 ceph-mon[37327]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-07-31T14:48:04.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:04 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/480323609' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-31T14:48:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:04 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] up:active 2024-07-31T14:48:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:04 smithi077 ceph-mon[37327]: fsmap cephfs:5 7 up:standby 5 up:active 2024-07-31T14:48:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:04 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:48:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.191+0000 7f0951a76640 1 -- 172.21.15.67:0/3907138465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f094c104c70 msgr2=0x7f094c105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.191+0000 7f0951a76640 1 --2- 172.21.15.67:0/3907138465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f094c104c70 0x7f094c105080 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0940009900 tx=0x7f094002a410 comp rx=0 tx=0).stop 2024-07-31T14:48:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.191+0000 7f0951a76640 1 -- 172.21.15.67:0/3907138465 shutdown_connections 2024-07-31T14:48:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.191+0000 7f0951a76640 1 --2- 172.21.15.67:0/3907138465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f094c1055c0 0x7f094c10f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.191+0000 7f0951a76640 1 --2- 172.21.15.67:0/3907138465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f094c104c70 0x7f094c105080 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.191+0000 7f0951a76640 1 --2- 172.21.15.67:0/3907138465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f094c102ca0 0x7f094c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.191+0000 7f0951a76640 1 -- 172.21.15.67:0/3907138465 >> 172.21.15.67:0/3907138465 conn(0x7f094c0798d0 msgr2=0x7f094c079ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.191+0000 7f0951a76640 1 -- 172.21.15.67:0/3907138465 shutdown_connections 2024-07-31T14:48:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.191+0000 7f0951a76640 1 -- 172.21.15.67:0/3907138465 wait complete. 2024-07-31T14:48:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.192+0000 7f0951a76640 1 Processor -- start 2024-07-31T14:48:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.192+0000 7f0951a76640 1 -- start start 2024-07-31T14:48:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f0951a76640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f094c102ca0 0x7f094c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f0951a76640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f094c104c70 0x7f094c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f0951a76640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f094c1055c0 0x7f094c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:06.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f0951a76640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f094c112450 con 0x7f094c104c70 2024-07-31T14:48:06.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f0951a76640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f094c1122d0 con 0x7f094c1055c0 2024-07-31T14:48:06.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f0951a76640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f094c1125d0 con 0x7f094c102ca0 2024-07-31T14:48:06.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f094affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f094c104c70 0x7f094c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:06.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f094affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f094c104c70 0x7f094c19a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60302/0 (socket says 172.21.15.67:60302) 2024-07-31T14:48:06.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f094affd640 1 -- 172.21.15.67:0/3617460403 learned_addr learned my addr 172.21.15.67:0/3617460403 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:06.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f094bfff640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f094c1055c0 0x7f094c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:06.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f094b7fe640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f094c102ca0 0x7f094c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:06.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f094affd640 1 -- 172.21.15.67:0/3617460403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f094c102ca0 msgr2=0x7f094c199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:06.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f094affd640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f094c102ca0 0x7f094c199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:06.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f094affd640 1 -- 172.21.15.67:0/3617460403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f094c1055c0 msgr2=0x7f094c19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f094affd640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f094c1055c0 0x7f094c19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f094affd640 1 -- 172.21.15.67:0/3617460403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094c19ff20 con 0x7f094c104c70 2024-07-31T14:48:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f094bfff640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f094c1055c0 0x7f094c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:48:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.193+0000 7f094affd640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f094c104c70 0x7f094c19a550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0940005940 tx=0x7f09400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.194+0000 7f0948ff9640 1 -- 172.21.15.67:0/3617460403 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094003f030 con 0x7f094c104c70 2024-07-31T14:48:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.194+0000 7f0948ff9640 1 -- 172.21.15.67:0/3617460403 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0940003cb0 con 0x7f094c104c70 2024-07-31T14:48:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.194+0000 7f0951a76640 1 -- 172.21.15.67:0/3617460403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094c1a01b0 con 0x7f094c104c70 2024-07-31T14:48:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.194+0000 7f0948ff9640 1 -- 172.21.15.67:0/3617460403 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0940035070 con 0x7f094c104c70 2024-07-31T14:48:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.194+0000 7f0951a76640 1 -- 172.21.15.67:0/3617460403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f094c1a6970 con 0x7f094c104c70 2024-07-31T14:48:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.195+0000 7f0951a76640 1 -- 172.21.15.67:0/3617460403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f094c06b390 con 0x7f094c104c70 2024-07-31T14:48:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.197+0000 7f0948ff9640 1 -- 172.21.15.67:0/3617460403 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f0940042030 con 0x7f094c104c70 2024-07-31T14:48:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.197+0000 7f0948ff9640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09340b3c20 0x7f09340b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.197+0000 7f094b7fe640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09340b3c20 0x7f09340b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:06.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.197+0000 7f0948ff9640 1 -- 172.21.15.67:0/3617460403 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f0940104800 con 0x7f094c104c70 2024-07-31T14:48:06.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.198+0000 7f094b7fe640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09340b3c20 0x7f09340b6130 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f093c005fd0 tx=0x7f093c005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:06.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.200+0000 7f0948ff9640 1 -- 172.21.15.67:0/3617460403 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f09400cba20 con 0x7f094c104c70 2024-07-31T14:48:06.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.434+0000 7f0951a76640 1 -- 172.21.15.67:0/3617460403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f094c10b690 con 0x7f094c104c70 2024-07-31T14:48:06.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.596+0000 7f0948ff9640 1 -- 172.21.15.67:0/3617460403 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7f09400cb320 con 0x7f094c104c70 2024-07-31T14:48:06.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.599+0000 7f0951a76640 1 -- 172.21.15.67:0/3617460403 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09340b3c20 msgr2=0x7f09340b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:06.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.599+0000 7f0951a76640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09340b3c20 0x7f09340b6130 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f093c005fd0 tx=0x7f093c005850 comp rx=0 tx=0).stop 2024-07-31T14:48:06.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.599+0000 7f0951a76640 1 -- 172.21.15.67:0/3617460403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f094c104c70 msgr2=0x7f094c19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:06.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.599+0000 7f0951a76640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f094c104c70 0x7f094c19a550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0940005940 tx=0x7f09400042f0 comp rx=0 tx=0).stop 2024-07-31T14:48:06.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.599+0000 7f0951a76640 1 -- 172.21.15.67:0/3617460403 shutdown_connections 2024-07-31T14:48:06.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.599+0000 7f0951a76640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09340b3c20 0x7f09340b6130 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:06.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.599+0000 7f0951a76640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f094c1055c0 0x7f094c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:06.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.599+0000 7f0951a76640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f094c104c70 0x7f094c19a550 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:06.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.599+0000 7f0951a76640 1 --2- 172.21.15.67:0/3617460403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f094c102ca0 0x7f094c199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:06.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.599+0000 7f0951a76640 1 -- 172.21.15.67:0/3617460403 >> 172.21.15.67:0/3617460403 conn(0x7f094c0798d0 msgr2=0x7f094c106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:06.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.599+0000 7f0951a76640 1 -- 172.21.15.67:0/3617460403 shutdown_connections 2024-07-31T14:48:06.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:06.599+0000 7f0951a76640 1 -- 172.21.15.67:0/3617460403 wait complete. 2024-07-31T14:48:06.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:06 smithi073 ceph-mon[36714]: pgmap v310: 129 pgs: 129 active+clean; 585 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-07-31T14:48:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:06 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3617460403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-31T14:48:06.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:06 smithi067 ceph-mon[32750]: pgmap v310: 129 pgs: 129 active+clean; 585 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-07-31T14:48:06.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:06 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3617460403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-31T14:48:06.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:06 smithi077 ceph-mon[37327]: pgmap v310: 129 pgs: 129 active+clean; 585 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-07-31T14:48:06.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:06 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3617460403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-31T14:48:07.293 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-07-31T14:48:07.293 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-07-31T14:48:07.563 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:07.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:07 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3617460403' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-31T14:48:07.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:07 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:48:07.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:07 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3617460403' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-31T14:48:07.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:07 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:48:07.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:07 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3617460403' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-31T14:48:07.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:07 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:48:08.705 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:08 smithi067 ceph-mon[32750]: pgmap v311: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-07-31T14:48:08.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:08 smithi073 ceph-mon[36714]: pgmap v311: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-07-31T14:48:08.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:08 smithi077 ceph-mon[37327]: pgmap v311: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-07-31T14:48:09.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.273+0000 7fb6d84aa640 1 -- 172.21.15.67:0/1919392342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d0108740 msgr2=0x7fb6d010ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:09.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.273+0000 7fb6d84aa640 1 --2- 172.21.15.67:0/1919392342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d0108740 0x7fb6d010ef70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009ca0 tx=0x7fb6c802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:48:09.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.273+0000 7fb6d84aa640 1 -- 172.21.15.67:0/1919392342 shutdown_connections 2024-07-31T14:48:09.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.273+0000 7fb6d84aa640 1 --2- 172.21.15.67:0/1919392342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d0108740 0x7fb6d010ef70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:09.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.273+0000 7fb6d84aa640 1 --2- 172.21.15.67:0/1919392342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6d0107df0 0x7fb6d0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:09.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.273+0000 7fb6d84aa640 1 --2- 172.21.15.67:0/1919392342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d01024b0 0x7fb6d01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:09.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.273+0000 7fb6d84aa640 1 -- 172.21.15.67:0/1919392342 >> 172.21.15.67:0/1919392342 conn(0x7fb6d0079170 msgr2=0x7fb6d0079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:09.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.273+0000 7fb6d84aa640 1 -- 172.21.15.67:0/1919392342 shutdown_connections 2024-07-31T14:48:09.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.273+0000 7fb6d84aa640 1 -- 172.21.15.67:0/1919392342 wait complete. 2024-07-31T14:48:09.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.274+0000 7fb6d84aa640 1 Processor -- start 2024-07-31T14:48:09.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.274+0000 7fb6d84aa640 1 -- start start 2024-07-31T14:48:09.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.274+0000 7fb6d84aa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6d01024b0 0x7fb6d019c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:09.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.274+0000 7fb6d84aa640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d0107df0 0x7fb6d019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:09.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.274+0000 7fb6d84aa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d0108740 0x7fb6d01a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:09.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.274+0000 7fb6d84aa640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d01123c0 con 0x7fb6d0108740 2024-07-31T14:48:09.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.274+0000 7fb6d84aa640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d0112240 con 0x7fb6d0107df0 2024-07-31T14:48:09.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.274+0000 7fb6d84aa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d0112540 con 0x7fb6d01024b0 2024-07-31T14:48:09.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.274+0000 7fb6d6a47640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d0108740 0x7fb6d01a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:09.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.275+0000 7fb6d6a47640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d0108740 0x7fb6d01a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60320/0 (socket says 172.21.15.67:60320) 2024-07-31T14:48:09.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.275+0000 7fb6d6a47640 1 -- 172.21.15.67:0/3712512707 learned_addr learned my addr 172.21.15.67:0/3712512707 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:09.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.275+0000 7fb6d6246640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6d01024b0 0x7fb6d019c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:09.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.275+0000 7fb6d5a45640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d0107df0 0x7fb6d019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:09.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.275+0000 7fb6d6a47640 1 -- 172.21.15.67:0/3712512707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6d01024b0 msgr2=0x7fb6d019c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:09.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.275+0000 7fb6d6a47640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6d01024b0 0x7fb6d019c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:09.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.275+0000 7fb6d6a47640 1 -- 172.21.15.67:0/3712512707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d0107df0 msgr2=0x7fb6d019c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:09.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.275+0000 7fb6d6a47640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d0107df0 0x7fb6d019c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:09.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.275+0000 7fb6d6a47640 1 -- 172.21.15.67:0/3712512707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d01a1fa0 con 0x7fb6d0108740 2024-07-31T14:48:09.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.275+0000 7fb6d6246640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6d01024b0 0x7fb6d019c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:48:09.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.275+0000 7fb6d6a47640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d0108740 0x7fb6d01a1870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8005cc0 tx=0x7fb6c802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:09.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.276+0000 7fb6c77fe640 1 -- 172.21.15.67:0/3712512707 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8003bf0 con 0x7fb6d0108740 2024-07-31T14:48:09.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.276+0000 7fb6c77fe640 1 -- 172.21.15.67:0/3712512707 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6c8003d90 con 0x7fb6d0108740 2024-07-31T14:48:09.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.276+0000 7fb6c77fe640 1 -- 172.21.15.67:0/3712512707 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c802fbc0 con 0x7fb6d0108740 2024-07-31T14:48:09.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.276+0000 7fb6d84aa640 1 -- 172.21.15.67:0/3712512707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d01a2230 con 0x7fb6d0108740 2024-07-31T14:48:09.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.276+0000 7fb6d84aa640 1 -- 172.21.15.67:0/3712512707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d01a89f0 con 0x7fb6d0108740 2024-07-31T14:48:09.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.277+0000 7fb6d84aa640 1 -- 172.21.15.67:0/3712512707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d0103730 con 0x7fb6d0108740 2024-07-31T14:48:09.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.279+0000 7fb6c77fe640 1 -- 172.21.15.67:0/3712512707 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fb6c802fd60 con 0x7fb6d0108740 2024-07-31T14:48:09.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.279+0000 7fb6c77fe640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6bc0b3ca0 0x7fb6bc0b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:09.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.280+0000 7fb6d6246640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6bc0b3ca0 0x7fb6bc0b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:09.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.280+0000 7fb6c77fe640 1 -- 172.21.15.67:0/3712512707 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fb6c8105820 con 0x7fb6d0108740 2024-07-31T14:48:09.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.280+0000 7fb6d6246640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6bc0b3ca0 0x7fb6bc0b61b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00bfd0 tx=0x7fb6cc00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:09.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.283+0000 7fb6c77fe640 1 -- 172.21.15.67:0/3712512707 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb6c80cca40 con 0x7fb6d0108740 2024-07-31T14:48:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.508+0000 7fb6d84aa640 1 -- 172.21.15.67:0/3712512707 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb6d01063c0 con 0x7fb6bc0b3ca0 2024-07-31T14:48:09.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.542+0000 7fb6c77fe640 1 -- 172.21.15.67:0/3712512707 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb6d01063c0 con 0x7fb6bc0b3ca0 2024-07-31T14:48:09.543 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:48:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:48:09.513+0000 7f4cce78f640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:48:09.543 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:48:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:48:09.513+0000 7f4cce78f640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:48:09.543 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:48:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:48:09.513+0000 7f4cce78f640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:48:09.543 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:48:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:48:09.513+0000 7f4cce78f640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:48:09.544 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:48:09 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:48:09.513+0000 7f4cce78f640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:48:09.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.544+0000 7fb6d84aa640 1 -- 172.21.15.67:0/3712512707 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6bc0b3ca0 msgr2=0x7fb6bc0b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:09.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.544+0000 7fb6d84aa640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6bc0b3ca0 0x7fb6bc0b61b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00bfd0 tx=0x7fb6cc00b390 comp rx=0 tx=0).stop 2024-07-31T14:48:09.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.544+0000 7fb6d84aa640 1 -- 172.21.15.67:0/3712512707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d0108740 msgr2=0x7fb6d01a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:09.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.544+0000 7fb6d84aa640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d0108740 0x7fb6d01a1870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8005cc0 tx=0x7fb6c802f380 comp rx=0 tx=0).stop 2024-07-31T14:48:09.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.544+0000 7fb6d84aa640 1 -- 172.21.15.67:0/3712512707 shutdown_connections 2024-07-31T14:48:09.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.544+0000 7fb6d84aa640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6bc0b3ca0 0x7fb6bc0b61b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:09.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.544+0000 7fb6d84aa640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d0108740 0x7fb6d01a1870 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:09.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.544+0000 7fb6d84aa640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d0107df0 0x7fb6d019c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:09.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.544+0000 7fb6d84aa640 1 --2- 172.21.15.67:0/3712512707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6d01024b0 0x7fb6d019c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:09.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.544+0000 7fb6d84aa640 1 -- 172.21.15.67:0/3712512707 >> 172.21.15.67:0/3712512707 conn(0x7fb6d0079170 msgr2=0x7fb6d0104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:09.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.544+0000 7fb6d84aa640 1 -- 172.21.15.67:0/3712512707 shutdown_connections 2024-07-31T14:48:09.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:09.544+0000 7fb6d84aa640 1 -- 172.21.15.67:0/3712512707 wait complete. 2024-07-31T14:48:10.161 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-07-31T14:48:10.436 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:10.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:10 smithi073 ceph-mon[36714]: pgmap v312: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-07-31T14:48:10.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:10 smithi073 ceph-mon[36714]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:10.902 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:10 smithi067 ceph-mon[32750]: pgmap v312: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-07-31T14:48:10.902 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:10 smithi067 ceph-mon[32750]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:10 smithi077 ceph-mon[37327]: pgmap v312: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-07-31T14:48:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:10 smithi077 ceph-mon[37327]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:12.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.111+0000 7f62713af640 1 -- 172.21.15.67:0/3611069530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f626c108760 msgr2=0x7f626c10ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.111+0000 7f62713af640 1 --2- 172.21.15.67:0/3611069530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f626c108760 0x7f626c10ef90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6264009900 tx=0x7f626402a3f0 comp rx=0 tx=0).stop 2024-07-31T14:48:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.111+0000 7f62713af640 1 -- 172.21.15.67:0/3611069530 shutdown_connections 2024-07-31T14:48:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.111+0000 7f62713af640 1 --2- 172.21.15.67:0/3611069530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f626c108760 0x7f626c10ef90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.111+0000 7f62713af640 1 --2- 172.21.15.67:0/3611069530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f626c107e10 0x7f626c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.111+0000 7f62713af640 1 --2- 172.21.15.67:0/3611069530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f626c1024d0 0x7f626c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.111+0000 7f62713af640 1 -- 172.21.15.67:0/3611069530 >> 172.21.15.67:0/3611069530 conn(0x7f626c079180 msgr2=0x7f626c079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:12.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.111+0000 7f62713af640 1 -- 172.21.15.67:0/3611069530 shutdown_connections 2024-07-31T14:48:12.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.111+0000 7f62713af640 1 -- 172.21.15.67:0/3611069530 wait complete. 2024-07-31T14:48:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.112+0000 7f62713af640 1 Processor -- start 2024-07-31T14:48:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.112+0000 7f62713af640 1 -- start start 2024-07-31T14:48:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.112+0000 7f62713af640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f626c1024d0 0x7f626c199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.112+0000 7f62713af640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f626c107e10 0x7f626c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.112+0000 7f62713af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f626c108760 0x7f626c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:12.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.112+0000 7f62713af640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1123c0 con 0x7f626c1024d0 2024-07-31T14:48:12.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.112+0000 7f62713af640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f626c112240 con 0x7f626c107e10 2024-07-31T14:48:12.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.112+0000 7f62713af640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f626c112540 con 0x7f626c108760 2024-07-31T14:48:12.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.112+0000 7f626affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f626c1024d0 0x7f626c199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:12.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.112+0000 7f626affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f626c1024d0 0x7f626c199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45476/0 (socket says 172.21.15.67:45476) 2024-07-31T14:48:12.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f626affd640 1 -- 172.21.15.67:0/2995911383 learned_addr learned my addr 172.21.15.67:0/2995911383 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:12.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f626b7fe640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f626c108760 0x7f626c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:12.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f626a7fc640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f626c107e10 0x7f626c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:12.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f626affd640 1 -- 172.21.15.67:0/2995911383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f626c108760 msgr2=0x7f626c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:12.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f626affd640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f626c108760 0x7f626c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:12.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f626affd640 1 -- 172.21.15.67:0/2995911383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f626c107e10 msgr2=0x7f626c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:12.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f626affd640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f626c107e10 0x7f626c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:12.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f626affd640 1 -- 172.21.15.67:0/2995911383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c19fd30 con 0x7f626c1024d0 2024-07-31T14:48:12.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f626a7fc640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f626c107e10 0x7f626c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:48:12.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f626affd640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f626c1024d0 0x7f626c199e80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f625c00cce0 tx=0x7f625c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:12.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f6263fff640 1 -- 172.21.15.67:0/2995911383 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c00f650 con 0x7f626c1024d0 2024-07-31T14:48:12.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f62713af640 1 -- 172.21.15.67:0/2995911383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626c1a0020 con 0x7f626c1024d0 2024-07-31T14:48:12.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f6263fff640 1 -- 172.21.15.67:0/2995911383 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f625c00fca0 con 0x7f626c1024d0 2024-07-31T14:48:12.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f6263fff640 1 -- 172.21.15.67:0/2995911383 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c018b40 con 0x7f626c1024d0 2024-07-31T14:48:12.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.113+0000 7f62713af640 1 -- 172.21.15.67:0/2995911383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626c1a6840 con 0x7f626c1024d0 2024-07-31T14:48:12.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.115+0000 7f62713af640 1 -- 172.21.15.67:0/2995911383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626c103750 con 0x7f626c1024d0 2024-07-31T14:48:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.116+0000 7f6263fff640 1 -- 172.21.15.67:0/2995911383 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f625c018ce0 con 0x7f626c1024d0 2024-07-31T14:48:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.117+0000 7f6263fff640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f62580b3ce0 0x7f62580b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.117+0000 7f626a7fc640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f62580b3ce0 0x7f62580b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:12.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.117+0000 7f6263fff640 1 -- 172.21.15.67:0/2995911383 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f625c0e49a0 con 0x7f626c1024d0 2024-07-31T14:48:12.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.118+0000 7f626a7fc640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f62580b3ce0 0x7f62580b61f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6254005e10 tx=0x7f6254005da0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.122+0000 7f6263fff640 1 -- 172.21.15.67:0/2995911383 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f625c014030 con 0x7f626c1024d0 2024-07-31T14:48:12.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.347+0000 7f62713af640 1 -- 172.21.15.67:0/2995911383 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f626c10b5d0 con 0x7f62580b3ce0 2024-07-31T14:48:12.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.374+0000 7f6263fff640 1 -- 172.21.15.67:0/2995911383 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f625400eb20 con 0x7f62580b3ce0 2024-07-31T14:48:12.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.376+0000 7f62713af640 1 -- 172.21.15.67:0/2995911383 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f62580b3ce0 msgr2=0x7f62580b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:12.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.376+0000 7f62713af640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f62580b3ce0 0x7f62580b61f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6254005e10 tx=0x7f6254005da0 comp rx=0 tx=0).stop 2024-07-31T14:48:12.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.376+0000 7f62713af640 1 -- 172.21.15.67:0/2995911383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f626c1024d0 msgr2=0x7f626c199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:12.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.376+0000 7f62713af640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f626c1024d0 0x7f626c199e80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f625c00cce0 tx=0x7f625c00a620 comp rx=0 tx=0).stop 2024-07-31T14:48:12.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.377+0000 7f62713af640 1 -- 172.21.15.67:0/2995911383 shutdown_connections 2024-07-31T14:48:12.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.377+0000 7f62713af640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f62580b3ce0 0x7f62580b61f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:12.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.377+0000 7f62713af640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f626c108760 0x7f626c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:12.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.377+0000 7f62713af640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f626c107e10 0x7f626c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:12.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.377+0000 7f62713af640 1 --2- 172.21.15.67:0/2995911383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f626c1024d0 0x7f626c199e80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:12.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.377+0000 7f62713af640 1 -- 172.21.15.67:0/2995911383 >> 172.21.15.67:0/2995911383 conn(0x7f626c079180 msgr2=0x7f626c101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:12.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.377+0000 7f62713af640 1 -- 172.21.15.67:0/2995911383 shutdown_connections 2024-07-31T14:48:12.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:12.377+0000 7f62713af640 1 -- 172.21.15.67:0/2995911383 wait complete. 2024-07-31T14:48:12.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:12 smithi067 ceph-mon[32750]: pgmap v313: 129 pgs: 129 active+clean; 593 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-07-31T14:48:12.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:12 smithi067 ceph-mon[32750]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-07-31T14:48:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:12 smithi073 ceph-mon[36714]: pgmap v313: 129 pgs: 129 active+clean; 593 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-07-31T14:48:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:12 smithi073 ceph-mon[36714]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-07-31T14:48:12.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:12 smithi077 ceph-mon[37327]: pgmap v313: 129 pgs: 129 active+clean; 593 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-07-31T14:48:12.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:12 smithi077 ceph-mon[37327]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-07-31T14:48:13.004 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-07-31T14:48:13.273 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:13.648 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:13 smithi067 ceph-mon[32750]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:13.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:13 smithi073 ceph-mon[36714]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:13.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:13 smithi077 ceph-mon[37327]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:14.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:14 smithi067 ceph-mon[32750]: pgmap v314: 129 pgs: 129 active+clean; 594 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.9 KiB/s wr, 22 op/s 2024-07-31T14:48:14.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:14 smithi073 ceph-mon[36714]: pgmap v314: 129 pgs: 129 active+clean; 594 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.9 KiB/s wr, 22 op/s 2024-07-31T14:48:14.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.893+0000 7fd5867a0640 1 -- 172.21.15.67:0/3188631573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd580103590 msgr2=0x7fd5801039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:14.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.893+0000 7fd5867a0640 1 --2- 172.21.15.67:0/3188631573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd580103590 0x7fd5801039a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd570009900 tx=0x7fd57002a410 comp rx=0 tx=0).stop 2024-07-31T14:48:14.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.893+0000 7fd5867a0640 1 -- 172.21.15.67:0/3188631573 shutdown_connections 2024-07-31T14:48:14.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.893+0000 7fd5867a0640 1 --2- 172.21.15.67:0/3188631573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd580109820 0x7fd580110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:14.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.893+0000 7fd5867a0640 1 --2- 172.21.15.67:0/3188631573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd580108ed0 0x7fd5801092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:14.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.893+0000 7fd5867a0640 1 --2- 172.21.15.67:0/3188631573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd580103590 0x7fd5801039a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:14.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.893+0000 7fd5867a0640 1 -- 172.21.15.67:0/3188631573 >> 172.21.15.67:0/3188631573 conn(0x7fd58007b280 msgr2=0x7fd58007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:14.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.893+0000 7fd5867a0640 1 -- 172.21.15.67:0/3188631573 shutdown_connections 2024-07-31T14:48:14.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.893+0000 7fd5867a0640 1 -- 172.21.15.67:0/3188631573 wait complete. 2024-07-31T14:48:14.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.894+0000 7fd5867a0640 1 Processor -- start 2024-07-31T14:48:14.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.894+0000 7fd5867a0640 1 -- start start 2024-07-31T14:48:14.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.894+0000 7fd5867a0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd580103590 0x7fd580199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:14.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.894+0000 7fd5867a0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd580108ed0 0x7fd58019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:14.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.894+0000 7fd5867a0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd580109820 0x7fd58019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:14.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.894+0000 7fd5867a0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd5801134f0 con 0x7fd580103590 2024-07-31T14:48:14.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.894+0000 7fd5867a0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd580113370 con 0x7fd580108ed0 2024-07-31T14:48:14.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.894+0000 7fd5867a0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd580113670 con 0x7fd580109820 2024-07-31T14:48:14.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.894+0000 7fd57ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd580103590 0x7fd580199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:14.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.894+0000 7fd57ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd580103590 0x7fd580199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45498/0 (socket says 172.21.15.67:45498) 2024-07-31T14:48:14.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.894+0000 7fd57ffff640 1 -- 172.21.15.67:0/3220035998 learned_addr learned my addr 172.21.15.67:0/3220035998 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:14.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.894+0000 7fd584d3d640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd580109820 0x7fd58019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:14.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.894+0000 7fd57f7fe640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd580108ed0 0x7fd58019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:14.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.895+0000 7fd57ffff640 1 -- 172.21.15.67:0/3220035998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd580109820 msgr2=0x7fd58019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:14.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.895+0000 7fd57ffff640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd580109820 0x7fd58019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:14.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.895+0000 7fd57ffff640 1 -- 172.21.15.67:0/3220035998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd580108ed0 msgr2=0x7fd58019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:14.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.895+0000 7fd57ffff640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd580108ed0 0x7fd58019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:14.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.895+0000 7fd57ffff640 1 -- 172.21.15.67:0/3220035998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58019feb0 con 0x7fd580103590 2024-07-31T14:48:14.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.895+0000 7fd57f7fe640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd580108ed0 0x7fd58019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:48:14.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.895+0000 7fd57ffff640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd580103590 0x7fd580199f40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd570005940 tx=0x7fd570004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:14.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.895+0000 7fd57d7fa640 1 -- 172.21.15.67:0/3220035998 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57003f030 con 0x7fd580103590 2024-07-31T14:48:14.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.895+0000 7fd57d7fa640 1 -- 172.21.15.67:0/3220035998 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd570003aa0 con 0x7fd580103590 2024-07-31T14:48:14.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.895+0000 7fd57d7fa640 1 -- 172.21.15.67:0/3220035998 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570035070 con 0x7fd580103590 2024-07-31T14:48:14.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.895+0000 7fd5867a0640 1 -- 172.21.15.67:0/3220035998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5801a0140 con 0x7fd580103590 2024-07-31T14:48:14.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.895+0000 7fd5867a0640 1 -- 172.21.15.67:0/3220035998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5801a6900 con 0x7fd580103590 2024-07-31T14:48:14.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.896+0000 7fd5867a0640 1 -- 172.21.15.67:0/3220035998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5801039a0 con 0x7fd580103590 2024-07-31T14:48:14.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.898+0000 7fd57d7fa640 1 -- 172.21.15.67:0/3220035998 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fd570042030 con 0x7fd580103590 2024-07-31T14:48:14.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.899+0000 7fd57d7fa640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd5680b3cb0 0x7fd5680b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:14.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.899+0000 7fd57f7fe640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd5680b3cb0 0x7fd5680b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:14.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.899+0000 7fd57d7fa640 1 -- 172.21.15.67:0/3220035998 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fd570104c60 con 0x7fd580103590 2024-07-31T14:48:14.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.899+0000 7fd57f7fe640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd5680b3cb0 0x7fd5680b61c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd580105e10 tx=0x7fd574005d30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:14.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:14.902+0000 7fd57d7fa640 1 -- 172.21.15.67:0/3220035998 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd5700cbd20 con 0x7fd580103590 2024-07-31T14:48:14.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:14 smithi077 ceph-mon[37327]: pgmap v314: 129 pgs: 129 active+clean; 594 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.9 KiB/s wr, 22 op/s 2024-07-31T14:48:15.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:15.130+0000 7fd5867a0640 1 -- 172.21.15.67:0/3220035998 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fd580063230 con 0x7fd5680b3cb0 2024-07-31T14:48:15.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:15.135+0000 7fd57d7fa640 1 -- 172.21.15.67:0/3220035998 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd580063230 con 0x7fd5680b3cb0 2024-07-31T14:48:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/qa/sv_0/152dbda7-22c7-45e2-a3b4-6584793c0ecb 2024-07-31T14:48:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:15.137+0000 7fd5867a0640 1 -- 172.21.15.67:0/3220035998 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd5680b3cb0 msgr2=0x7fd5680b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:15.137+0000 7fd5867a0640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd5680b3cb0 0x7fd5680b61c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd580105e10 tx=0x7fd574005d30 comp rx=0 tx=0).stop 2024-07-31T14:48:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:15.137+0000 7fd5867a0640 1 -- 172.21.15.67:0/3220035998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd580103590 msgr2=0x7fd580199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:15.137+0000 7fd5867a0640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd580103590 0x7fd580199f40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd570005940 tx=0x7fd570004270 comp rx=0 tx=0).stop 2024-07-31T14:48:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:15.137+0000 7fd5867a0640 1 -- 172.21.15.67:0/3220035998 shutdown_connections 2024-07-31T14:48:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:15.137+0000 7fd5867a0640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd5680b3cb0 0x7fd5680b61c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:15.137+0000 7fd5867a0640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd580109820 0x7fd58019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:15.137+0000 7fd5867a0640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd580108ed0 0x7fd58019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:15.137+0000 7fd5867a0640 1 --2- 172.21.15.67:0/3220035998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd580103590 0x7fd580199f40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:15.137+0000 7fd5867a0640 1 -- 172.21.15.67:0/3220035998 >> 172.21.15.67:0/3220035998 conn(0x7fd58007b280 msgr2=0x7fd580107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:15.137+0000 7fd5867a0640 1 -- 172.21.15.67:0/3220035998 shutdown_connections 2024-07-31T14:48:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:15.137+0000 7fd5867a0640 1 -- 172.21.15.67:0/3220035998 wait complete. 2024-07-31T14:48:15.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-07-31T14:48:16.062 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:16.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:16 smithi067 ceph-mon[32750]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:16.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:16 smithi067 ceph-mon[32750]: pgmap v315: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.3 KiB/s wr, 20 op/s 2024-07-31T14:48:16.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:16 smithi073 ceph-mon[36714]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:16.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:16 smithi073 ceph-mon[36714]: pgmap v315: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.3 KiB/s wr, 20 op/s 2024-07-31T14:48:16.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:16 smithi077 ceph-mon[37327]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:16.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:16 smithi077 ceph-mon[37327]: pgmap v315: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.3 KiB/s wr, 20 op/s 2024-07-31T14:48:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.788+0000 7f492f079640 1 -- 172.21.15.67:0/2937139298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928107dd0 msgr2=0x7f49281081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.788+0000 7f492f079640 1 --2- 172.21.15.67:0/2937139298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928107dd0 0x7f49281081e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4918009ca0 tx=0x7f491802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:48:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.788+0000 7f492f079640 1 -- 172.21.15.67:0/2937139298 shutdown_connections 2024-07-31T14:48:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.788+0000 7f492f079640 1 --2- 172.21.15.67:0/2937139298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4928108720 0x7f492810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.788+0000 7f492f079640 1 --2- 172.21.15.67:0/2937139298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928107dd0 0x7f49281081e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.788+0000 7f492f079640 1 --2- 172.21.15.67:0/2937139298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4928102490 0x7f49281028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.788+0000 7f492f079640 1 -- 172.21.15.67:0/2937139298 >> 172.21.15.67:0/2937139298 conn(0x7f4928079160 msgr2=0x7f4928079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.788+0000 7f492f079640 1 -- 172.21.15.67:0/2937139298 shutdown_connections 2024-07-31T14:48:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.788+0000 7f492f079640 1 -- 172.21.15.67:0/2937139298 wait complete. 2024-07-31T14:48:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.789+0000 7f492f079640 1 Processor -- start 2024-07-31T14:48:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.789+0000 7f492f079640 1 -- start start 2024-07-31T14:48:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.789+0000 7f492f079640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928102490 0x7f4928199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.789+0000 7f492f079640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4928107dd0 0x7f492819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.789+0000 7f492f079640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4928108720 0x7f492819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.789+0000 7f492f079640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f49281134b0 con 0x7f4928102490 2024-07-31T14:48:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.789+0000 7f492f079640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4928113330 con 0x7f4928108720 2024-07-31T14:48:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.789+0000 7f492f079640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4928113630 con 0x7f4928107dd0 2024-07-31T14:48:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.789+0000 7f492ce15640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928102490 0x7f4928199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.789+0000 7f492ce15640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928102490 0x7f4928199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45512/0 (socket says 172.21.15.67:45512) 2024-07-31T14:48:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.789+0000 7f492ce15640 1 -- 172.21.15.67:0/2063318142 learned_addr learned my addr 172.21.15.67:0/2063318142 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.790+0000 7f4927fff640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4928107dd0 0x7f492819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.790+0000 7f492d616640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4928108720 0x7f492819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.790+0000 7f492ce15640 1 -- 172.21.15.67:0/2063318142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4928107dd0 msgr2=0x7f492819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.790+0000 7f492ce15640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4928107dd0 0x7f492819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.790+0000 7f492ce15640 1 -- 172.21.15.67:0/2063318142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4928108720 msgr2=0x7f492819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.790+0000 7f492ce15640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4928108720 0x7f492819f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.790+0000 7f492ce15640 1 -- 172.21.15.67:0/2063318142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492819fdb0 con 0x7f4928102490 2024-07-31T14:48:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.790+0000 7f492d616640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4928108720 0x7f492819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:48:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.790+0000 7f492ce15640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928102490 0x7f4928199f00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f492000c920 tx=0x7f492000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.790+0000 7f4925ffb640 1 -- 172.21.15.67:0/2063318142 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4920007c70 con 0x7f4928102490 2024-07-31T14:48:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.790+0000 7f4925ffb640 1 -- 172.21.15.67:0/2063318142 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4920007e10 con 0x7f4928102490 2024-07-31T14:48:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.790+0000 7f492f079640 1 -- 172.21.15.67:0/2063318142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49281a00a0 con 0x7f4928102490 2024-07-31T14:48:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.790+0000 7f4925ffb640 1 -- 172.21.15.67:0/2063318142 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4920018820 con 0x7f4928102490 2024-07-31T14:48:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.791+0000 7f492f079640 1 -- 172.21.15.67:0/2063318142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49281a68c0 con 0x7f4928102490 2024-07-31T14:48:17.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.792+0000 7f492f079640 1 -- 172.21.15.67:0/2063318142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49281028a0 con 0x7f4928102490 2024-07-31T14:48:17.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.793+0000 7f4925ffb640 1 -- 172.21.15.67:0/2063318142 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f492000f450 con 0x7f4928102490 2024-07-31T14:48:17.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.794+0000 7f4925ffb640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f49140b3ce0 0x7f49140b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:17.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.794+0000 7f4927fff640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f49140b3ce0 0x7f49140b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:17.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.794+0000 7f4925ffb640 1 -- 172.21.15.67:0/2063318142 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f49200e4320 con 0x7f4928102490 2024-07-31T14:48:17.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.794+0000 7f4927fff640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f49140b3ce0 0x7f49140b61f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4918005e30 tx=0x7f4918009360 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:17.798+0000 7f4925ffb640 1 -- 172.21.15.67:0/2063318142 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f4920014030 con 0x7f4928102490 2024-07-31T14:48:18.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:18.022+0000 7f492f079640 1 -- 172.21.15.67:0/2063318142 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4928063230 con 0x7f49140b3ce0 2024-07-31T14:48:18.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:18.049+0000 7f4925ffb640 1 -- 172.21.15.67:0/2063318142 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4928063230 con 0x7f49140b3ce0 2024-07-31T14:48:18.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:18.052+0000 7f492f079640 1 -- 172.21.15.67:0/2063318142 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f49140b3ce0 msgr2=0x7f49140b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:18.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:18.052+0000 7f492f079640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f49140b3ce0 0x7f49140b61f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4918005e30 tx=0x7f4918009360 comp rx=0 tx=0).stop 2024-07-31T14:48:18.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:18.052+0000 7f492f079640 1 -- 172.21.15.67:0/2063318142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928102490 msgr2=0x7f4928199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:18.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:18.052+0000 7f492f079640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928102490 0x7f4928199f00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f492000c920 tx=0x7f492000cde0 comp rx=0 tx=0).stop 2024-07-31T14:48:18.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:18.052+0000 7f492f079640 1 -- 172.21.15.67:0/2063318142 shutdown_connections 2024-07-31T14:48:18.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:18.052+0000 7f492f079640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f49140b3ce0 0x7f49140b61f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:18.052+0000 7f492f079640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4928108720 0x7f492819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:18.052+0000 7f492f079640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4928107dd0 0x7f492819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:18.052+0000 7f492f079640 1 --2- 172.21.15.67:0/2063318142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928102490 0x7f4928199f00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:18.052+0000 7f492f079640 1 -- 172.21.15.67:0/2063318142 >> 172.21.15.67:0/2063318142 conn(0x7f4928079160 msgr2=0x7f4928104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:18.052+0000 7f492f079640 1 -- 172.21.15.67:0/2063318142 shutdown_connections 2024-07-31T14:48:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:18.052+0000 7f492f079640 1 -- 172.21.15.67:0/2063318142 wait complete. 2024-07-31T14:48:18.697 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-07-31T14:48:18.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:18 smithi073 ceph-mon[36714]: pgmap v316: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-07-31T14:48:18.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:18 smithi073 ceph-mon[36714]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:18.963 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:18.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:18 smithi077 ceph-mon[37327]: pgmap v316: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-07-31T14:48:18.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:18 smithi077 ceph-mon[37327]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:19.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:18 smithi067 ceph-mon[32750]: pgmap v316: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-07-31T14:48:19.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:18 smithi067 ceph-mon[32750]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:20.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.613+0000 7fb98cfae640 1 -- 172.21.15.67:0/1934016104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb988102490 msgr2=0x7fb9881028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:20.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.613+0000 7fb98cfae640 1 --2- 172.21.15.67:0/1934016104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb988102490 0x7fb9881028a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb978009900 tx=0x7fb97802a410 comp rx=0 tx=0).stop 2024-07-31T14:48:20.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.613+0000 7fb98cfae640 1 -- 172.21.15.67:0/1934016104 shutdown_connections 2024-07-31T14:48:20.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.613+0000 7fb98cfae640 1 --2- 172.21.15.67:0/1934016104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb988108720 0x7fb98810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:20.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.613+0000 7fb98cfae640 1 --2- 172.21.15.67:0/1934016104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb988107dd0 0x7fb9881081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:20.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.613+0000 7fb98cfae640 1 --2- 172.21.15.67:0/1934016104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb988102490 0x7fb9881028a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:20.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.613+0000 7fb98cfae640 1 -- 172.21.15.67:0/1934016104 >> 172.21.15.67:0/1934016104 conn(0x7fb988079160 msgr2=0x7fb988079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:20.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.614+0000 7fb98cfae640 1 -- 172.21.15.67:0/1934016104 shutdown_connections 2024-07-31T14:48:20.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.614+0000 7fb98cfae640 1 -- 172.21.15.67:0/1934016104 wait complete. 2024-07-31T14:48:20.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.615+0000 7fb98cfae640 1 Processor -- start 2024-07-31T14:48:20.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.615+0000 7fb98cfae640 1 -- start start 2024-07-31T14:48:20.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.615+0000 7fb98cfae640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb988102490 0x7fb98819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:20.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.615+0000 7fb98cfae640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb988107dd0 0x7fb98819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:20.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.615+0000 7fb98cfae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb988108720 0x7fb98819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:20.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.615+0000 7fb98cfae640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb9881123a0 con 0x7fb988107dd0 2024-07-31T14:48:20.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.615+0000 7fb98cfae640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb988112220 con 0x7fb988102490 2024-07-31T14:48:20.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.615+0000 7fb98cfae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb988112520 con 0x7fb988108720 2024-07-31T14:48:20.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.615+0000 7fb985d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb988107dd0 0x7fb98819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:20.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.615+0000 7fb985d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb988107dd0 0x7fb98819b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45528/0 (socket says 172.21.15.67:45528) 2024-07-31T14:48:20.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.615+0000 7fb985d9b640 1 -- 172.21.15.67:0/3993612981 learned_addr learned my addr 172.21.15.67:0/3993612981 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:20.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.616+0000 7fb986d9d640 1 --2- 172.21.15.67:0/3993612981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb988108720 0x7fb98819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:20.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.616+0000 7fb98659c640 1 --2- 172.21.15.67:0/3993612981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb988102490 0x7fb98819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:20.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.616+0000 7fb985d9b640 1 -- 172.21.15.67:0/3993612981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb988108720 msgr2=0x7fb98819b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:20.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.616+0000 7fb985d9b640 1 --2- 172.21.15.67:0/3993612981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb988108720 0x7fb98819b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:20.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.616+0000 7fb985d9b640 1 -- 172.21.15.67:0/3993612981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb988102490 msgr2=0x7fb98819acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:20.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.616+0000 7fb985d9b640 1 --2- 172.21.15.67:0/3993612981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb988102490 0x7fb98819acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:20.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.616+0000 7fb985d9b640 1 -- 172.21.15.67:0/3993612981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9881a5f60 con 0x7fb988107dd0 2024-07-31T14:48:20.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.616+0000 7fb985d9b640 1 --2- 172.21.15.67:0/3993612981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb988107dd0 0x7fb98819b210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00d740 tx=0x7fb97c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:20.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.616+0000 7fb9777fe640 1 -- 172.21.15.67:0/3993612981 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c017070 con 0x7fb988107dd0 2024-07-31T14:48:20.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.616+0000 7fb9777fe640 1 -- 172.21.15.67:0/3993612981 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb97c00de50 con 0x7fb988107dd0 2024-07-31T14:48:20.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.616+0000 7fb98cfae640 1 -- 172.21.15.67:0/3993612981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9881a6250 con 0x7fb988107dd0 2024-07-31T14:48:20.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.616+0000 7fb9777fe640 1 -- 172.21.15.67:0/3993612981 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c004ce0 con 0x7fb988107dd0 2024-07-31T14:48:20.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.616+0000 7fb98cfae640 1 -- 172.21.15.67:0/3993612981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9881a6740 con 0x7fb988107dd0 2024-07-31T14:48:20.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.618+0000 7fb98cfae640 1 -- 172.21.15.67:0/3993612981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9881028a0 con 0x7fb988107dd0 2024-07-31T14:48:20.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.619+0000 7fb9777fe640 1 -- 172.21.15.67:0/3993612981 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb97c004e80 con 0x7fb988107dd0 2024-07-31T14:48:20.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.620+0000 7fb9777fe640 1 --2- 172.21.15.67:0/3993612981 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb9700b3ce0 0x7fb9700b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:20.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.620+0000 7fb98659c640 1 --2- 172.21.15.67:0/3993612981 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb9700b3ce0 0x7fb9700b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:20.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.620+0000 7fb9777fe640 1 -- 172.21.15.67:0/3993612981 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fb97c0e3f00 con 0x7fb988107dd0 2024-07-31T14:48:20.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.620+0000 7fb98659c640 1 --2- 172.21.15.67:0/3993612981 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb9700b3ce0 0x7fb9700b61f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb978000c00 tx=0x7fb978031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:20.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.623+0000 7fb9777fe640 1 -- 172.21.15.67:0/3993612981 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb97c0aa970 con 0x7fb988107dd0 2024-07-31T14:48:20.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.851+0000 7fb98cfae640 1 -- 172.21.15.67:0/3993612981 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb988063230 con 0x7fb9700b3ce0 2024-07-31T14:48:20.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.856+0000 7fb9777fe640 1 -- 172.21.15.67:0/3993612981 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb988063230 con 0x7fb9700b3ce0 2024-07-31T14:48:20.866 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051 2024-07-31T14:48:20.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.858+0000 7fb98cfae640 1 -- 172.21.15.67:0/3993612981 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb9700b3ce0 msgr2=0x7fb9700b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:20.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.858+0000 7fb98cfae640 1 --2- 172.21.15.67:0/3993612981 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb9700b3ce0 0x7fb9700b61f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb978000c00 tx=0x7fb978031400 comp rx=0 tx=0).stop 2024-07-31T14:48:20.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.858+0000 7fb98cfae640 1 -- 172.21.15.67:0/3993612981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb988107dd0 msgr2=0x7fb98819b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:20.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.858+0000 7fb98cfae640 1 --2- 172.21.15.67:0/3993612981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb988107dd0 0x7fb98819b210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00d740 tx=0x7fb97c00dc00 comp rx=0 tx=0).stop 2024-07-31T14:48:20.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.858+0000 7fb98cfae640 1 -- 172.21.15.67:0/3993612981 shutdown_connections 2024-07-31T14:48:20.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.858+0000 7fb98cfae640 1 --2- 172.21.15.67:0/3993612981 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb9700b3ce0 0x7fb9700b61f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:20.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.858+0000 7fb98cfae640 1 --2- 172.21.15.67:0/3993612981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb988108720 0x7fb98819b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:20.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.858+0000 7fb98cfae640 1 --2- 172.21.15.67:0/3993612981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb988107dd0 0x7fb98819b210 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:20.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.858+0000 7fb98cfae640 1 --2- 172.21.15.67:0/3993612981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb988102490 0x7fb98819acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:20.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.858+0000 7fb98cfae640 1 -- 172.21.15.67:0/3993612981 >> 172.21.15.67:0/3993612981 conn(0x7fb988079160 msgr2=0x7fb988100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:20.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.858+0000 7fb98cfae640 1 -- 172.21.15.67:0/3993612981 shutdown_connections 2024-07-31T14:48:20.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:20.858+0000 7fb98cfae640 1 -- 172.21.15.67:0/3993612981 wait complete. 2024-07-31T14:48:20.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:20 smithi067 ceph-mon[32750]: pgmap v317: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-07-31T14:48:20.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:20 smithi073 ceph-mon[36714]: pgmap v317: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-07-31T14:48:20.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:20 smithi077 ceph-mon[37327]: pgmap v317: 129 pgs: 129 active+clean; 608 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-07-31T14:48:21.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:48:21.786 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:21.908 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:21 smithi067 ceph-mon[32750]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:21.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:21 smithi077 ceph-mon[37327]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:22.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:21 smithi073 ceph-mon[36714]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:48:22.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:22 smithi067 ceph-mon[32750]: pgmap v318: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-07-31T14:48:22.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:22 smithi077 ceph-mon[37327]: pgmap v318: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-07-31T14:48:23.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:22 smithi073 ceph-mon[36714]: pgmap v318: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-07-31T14:48:23.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.468+0000 7f3a89c05640 1 -- 172.21.15.67:0/172895548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84109820 msgr2=0x7f3a84110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:23.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.468+0000 7f3a89c05640 1 --2- 172.21.15.67:0/172895548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84109820 0x7f3a84110050 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009ca0 tx=0x7f3a7c02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:48:23.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.468+0000 7f3a89c05640 1 -- 172.21.15.67:0/172895548 shutdown_connections 2024-07-31T14:48:23.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.468+0000 7f3a89c05640 1 --2- 172.21.15.67:0/172895548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84109820 0x7f3a84110050 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:23.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.468+0000 7f3a89c05640 1 --2- 172.21.15.67:0/172895548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a84108ed0 0x7f3a841092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:23.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.468+0000 7f3a89c05640 1 --2- 172.21.15.67:0/172895548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a84103590 0x7f3a841039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.468+0000 7f3a89c05640 1 -- 172.21.15.67:0/172895548 >> 172.21.15.67:0/172895548 conn(0x7f3a8407b280 msgr2=0x7f3a8407b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.468+0000 7f3a89c05640 1 -- 172.21.15.67:0/172895548 shutdown_connections 2024-07-31T14:48:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.468+0000 7f3a89c05640 1 -- 172.21.15.67:0/172895548 wait complete. 2024-07-31T14:48:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.469+0000 7f3a89c05640 1 Processor -- start 2024-07-31T14:48:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.469+0000 7f3a89c05640 1 -- start start 2024-07-31T14:48:23.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.469+0000 7f3a89c05640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a84103590 0x7f3a84199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:23.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.469+0000 7f3a89c05640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84108ed0 0x7f3a8419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.469+0000 7f3a89c05640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a84109820 0x7f3a8419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.469+0000 7f3a89c05640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3a841134f0 con 0x7f3a84108ed0 2024-07-31T14:48:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.469+0000 7f3a89c05640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3a84113370 con 0x7f3a84103590 2024-07-31T14:48:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.469+0000 7f3a89c05640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3a84113670 con 0x7f3a84109820 2024-07-31T14:48:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.469+0000 7f3a82ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84108ed0 0x7f3a8419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a82ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84108ed0 0x7f3a8419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59890/0 (socket says 172.21.15.67:59890) 2024-07-31T14:48:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a82ffd640 1 -- 172.21.15.67:0/3691513452 learned_addr learned my addr 172.21.15.67:0/3691513452 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a837fe640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a84103590 0x7f3a84199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a83fff640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a84109820 0x7f3a8419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a82ffd640 1 -- 172.21.15.67:0/3691513452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a84109820 msgr2=0x7f3a8419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a82ffd640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a84109820 0x7f3a8419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a82ffd640 1 -- 172.21.15.67:0/3691513452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a84103590 msgr2=0x7f3a84199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a82ffd640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a84103590 0x7f3a84199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a82ffd640 1 -- 172.21.15.67:0/3691513452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8419fe30 con 0x7f3a84108ed0 2024-07-31T14:48:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a837fe640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a84103590 0x7f3a84199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:48:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a82ffd640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84108ed0 0x7f3a8419a460 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800e9c0 tx=0x7f3a7800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a80ff9640 1 -- 172.21.15.67:0/3691513452 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800ccc0 con 0x7f3a84108ed0 2024-07-31T14:48:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a89c05640 1 -- 172.21.15.67:0/3691513452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a841a0120 con 0x7f3a84108ed0 2024-07-31T14:48:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a80ff9640 1 -- 172.21.15.67:0/3691513452 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a780054d0 con 0x7f3a84108ed0 2024-07-31T14:48:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a80ff9640 1 -- 172.21.15.67:0/3691513452 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78010430 con 0x7f3a84108ed0 2024-07-31T14:48:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.470+0000 7f3a89c05640 1 -- 172.21.15.67:0/3691513452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a841a6880 con 0x7f3a84108ed0 2024-07-31T14:48:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.471+0000 7f3a89c05640 1 -- 172.21.15.67:0/3691513452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a841039a0 con 0x7f3a84108ed0 2024-07-31T14:48:23.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.473+0000 7f3a80ff9640 1 -- 172.21.15.67:0/3691513452 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f3a78005000 con 0x7f3a84108ed0 2024-07-31T14:48:23.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.474+0000 7f3a80ff9640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3a6c0b3d30 0x7f3a6c0b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:23.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.474+0000 7f3a837fe640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3a6c0b3d30 0x7f3a6c0b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:23.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.474+0000 7f3a80ff9640 1 -- 172.21.15.67:0/3691513452 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f3a780e47a0 con 0x7f3a84108ed0 2024-07-31T14:48:23.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.475+0000 7f3a837fe640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3a6c0b3d30 0x7f3a6c0b6240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3a74005d00 tx=0x7f3a74005c70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:23.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.477+0000 7f3a80ff9640 1 -- 172.21.15.67:0/3691513452 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f3a780555e0 con 0x7f3a84108ed0 2024-07-31T14:48:23.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.709+0000 7f3a89c05640 1 -- 172.21.15.67:0/3691513452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a8419aff0 con 0x7f3a84108ed0 2024-07-31T14:48:23.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.709+0000 7f3a80ff9640 1 -- 172.21.15.67:0/3691513452 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11962 (secure 0 0 0) 0x7f3a780052f0 con 0x7f3a84108ed0 2024-07-31T14:48:23.711 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:48:23.711 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-07-31T14:48:06:596011+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:48:06.435900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T14:48:23.711 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-07-31T14:48:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.711+0000 7f3a89c05640 1 -- 172.21.15.67:0/3691513452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3a6c0b3d30 msgr2=0x7f3a6c0b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.711+0000 7f3a89c05640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3a6c0b3d30 0x7f3a6c0b6240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3a74005d00 tx=0x7f3a74005c70 comp rx=0 tx=0).stop 2024-07-31T14:48:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.711+0000 7f3a89c05640 1 -- 172.21.15.67:0/3691513452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84108ed0 msgr2=0x7f3a8419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.711+0000 7f3a89c05640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84108ed0 0x7f3a8419a460 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800e9c0 tx=0x7f3a7800ee80 comp rx=0 tx=0).stop 2024-07-31T14:48:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.712+0000 7f3a89c05640 1 -- 172.21.15.67:0/3691513452 shutdown_connections 2024-07-31T14:48:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.712+0000 7f3a89c05640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3a6c0b3d30 0x7f3a6c0b6240 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.712+0000 7f3a89c05640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a84109820 0x7f3a8419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.712+0000 7f3a89c05640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84108ed0 0x7f3a8419a460 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.712+0000 7f3a89c05640 1 --2- 172.21.15.67:0/3691513452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a84103590 0x7f3a84199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.712+0000 7f3a89c05640 1 -- 172.21.15.67:0/3691513452 >> 172.21.15.67:0/3691513452 conn(0x7f3a8407b280 msgr2=0x7f3a84106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.712+0000 7f3a89c05640 1 -- 172.21.15.67:0/3691513452 shutdown_connections 2024-07-31T14:48:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:23.712+0000 7f3a89c05640 1 -- 172.21.15.67:0/3691513452 wait complete. 2024-07-31T14:48:24.443 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-07-31T14:48:24.717 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:24.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:24 smithi067 ceph-mon[32750]: pgmap v319: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-07-31T14:48:24.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:24 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3691513452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:48:24.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:24 smithi077 ceph-mon[37327]: pgmap v319: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-07-31T14:48:24.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:24 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3691513452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:48:25.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:24 smithi073 ceph-mon[36714]: pgmap v319: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-07-31T14:48:25.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:24 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3691513452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:48:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.470+0000 7fd92f96e640 1 -- 172.21.15.67:0/472874396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd928102200 msgr2=0x7fd928107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.470+0000 7fd92f96e640 1 --2- 172.21.15.67:0/472874396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd928102200 0x7fd928107600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009900 tx=0x7fd91c02a410 comp rx=0 tx=0).stop 2024-07-31T14:48:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.470+0000 7fd92f96e640 1 -- 172.21.15.67:0/472874396 shutdown_connections 2024-07-31T14:48:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.470+0000 7fd92f96e640 1 --2- 172.21.15.67:0/472874396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928102610 0x7fd928107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.470+0000 7fd92f96e640 1 --2- 172.21.15.67:0/472874396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd928102200 0x7fd928107600 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.470+0000 7fd92f96e640 1 --2- 172.21.15.67:0/472874396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd92806ad60 0x7fd928101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.470+0000 7fd92f96e640 1 -- 172.21.15.67:0/472874396 >> 172.21.15.67:0/472874396 conn(0x7fd9280798d0 msgr2=0x7fd928079ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.471+0000 7fd92f96e640 1 -- 172.21.15.67:0/472874396 shutdown_connections 2024-07-31T14:48:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.471+0000 7fd92f96e640 1 -- 172.21.15.67:0/472874396 wait complete. 2024-07-31T14:48:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.471+0000 7fd92f96e640 1 Processor -- start 2024-07-31T14:48:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.471+0000 7fd92f96e640 1 -- start start 2024-07-31T14:48:26.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92f96e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd92806ad60 0x7fd92819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:26.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92f96e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928102200 0x7fd92819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:26.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92f96e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd928102610 0x7fd92819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:26.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92f96e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd928112450 con 0x7fd928102610 2024-07-31T14:48:26.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92f96e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd9281122d0 con 0x7fd928102200 2024-07-31T14:48:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92f96e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd9281125d0 con 0x7fd92806ad60 2024-07-31T14:48:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92df0b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd928102610 0x7fd92819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92df0b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd928102610 0x7fd92819b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59918/0 (socket says 172.21.15.67:59918) 2024-07-31T14:48:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92df0b640 1 -- 172.21.15.67:0/3370121519 learned_addr learned my addr 172.21.15.67:0/3370121519 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92cf09640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928102200 0x7fd92819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92d70a640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd92806ad60 0x7fd92819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92df0b640 1 -- 172.21.15.67:0/3370121519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd92806ad60 msgr2=0x7fd92819adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92df0b640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd92806ad60 0x7fd92819adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92df0b640 1 -- 172.21.15.67:0/3370121519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928102200 msgr2=0x7fd92819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92df0b640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928102200 0x7fd92819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92df0b640 1 -- 172.21.15.67:0/3370121519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9281a6050 con 0x7fd928102610 2024-07-31T14:48:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92d70a640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd92806ad60 0x7fd92819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:48:26.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.472+0000 7fd92df0b640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd928102610 0x7fd92819b840 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd92000cad0 tx=0x7fd92000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:26.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.473+0000 7fd91a7fc640 1 -- 172.21.15.67:0/3370121519 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92000ae20 con 0x7fd928102610 2024-07-31T14:48:26.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.473+0000 7fd91a7fc640 1 -- 172.21.15.67:0/3370121519 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd920005020 con 0x7fd928102610 2024-07-31T14:48:26.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.473+0000 7fd92f96e640 1 -- 172.21.15.67:0/3370121519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9281a6340 con 0x7fd928102610 2024-07-31T14:48:26.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.473+0000 7fd91a7fc640 1 -- 172.21.15.67:0/3370121519 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92000f6b0 con 0x7fd928102610 2024-07-31T14:48:26.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.473+0000 7fd92f96e640 1 -- 172.21.15.67:0/3370121519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9281a6800 con 0x7fd928102610 2024-07-31T14:48:26.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.474+0000 7fd92f96e640 1 -- 172.21.15.67:0/3370121519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd928101cc0 con 0x7fd928102610 2024-07-31T14:48:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.476+0000 7fd91a7fc640 1 -- 172.21.15.67:0/3370121519 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fd9200051d0 con 0x7fd928102610 2024-07-31T14:48:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.476+0000 7fd91a7fc640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9140b3ce0 0x7fd9140b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.476+0000 7fd92d70a640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9140b3ce0 0x7fd9140b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.477+0000 7fd91a7fc640 1 -- 172.21.15.67:0/3370121519 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fd9200e4630 con 0x7fd928102610 2024-07-31T14:48:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.477+0000 7fd92d70a640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9140b3ce0 0x7fd9140b61f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd924005fd0 tx=0x7fd924005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:26.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.479+0000 7fd91a7fc640 1 -- 172.21.15.67:0/3370121519 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd920014030 con 0x7fd928102610 2024-07-31T14:48:26.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.676+0000 7fd92f96e640 1 -- 172.21.15.67:0/3370121519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9281a6b90 con 0x7fd928102610 2024-07-31T14:48:26.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.678+0000 7fd91a7fc640 1 -- 172.21.15.67:0/3370121519 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24874 (secure 0 0 0) 0x7fd9200ab6f0 con 0x7fd928102610 2024-07-31T14:48:26.679 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:48:26.680 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T14:47:20.440064+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T14:48:26.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.680+0000 7fd92f96e640 1 -- 172.21.15.67:0/3370121519 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9140b3ce0 msgr2=0x7fd9140b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:26.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.680+0000 7fd92f96e640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9140b3ce0 0x7fd9140b61f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd924005fd0 tx=0x7fd924005850 comp rx=0 tx=0).stop 2024-07-31T14:48:26.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.680+0000 7fd92f96e640 1 -- 172.21.15.67:0/3370121519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd928102610 msgr2=0x7fd92819b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:26.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.680+0000 7fd92f96e640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd928102610 0x7fd92819b840 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd92000cad0 tx=0x7fd92000cf90 comp rx=0 tx=0).stop 2024-07-31T14:48:26.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.680+0000 7fd92f96e640 1 -- 172.21.15.67:0/3370121519 shutdown_connections 2024-07-31T14:48:26.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.680+0000 7fd92f96e640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9140b3ce0 0x7fd9140b61f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:26.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.680+0000 7fd92f96e640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd928102610 0x7fd92819b840 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:26.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.680+0000 7fd92f96e640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd928102200 0x7fd92819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:26.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.680+0000 7fd92f96e640 1 --2- 172.21.15.67:0/3370121519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd92806ad60 0x7fd92819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:26.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.680+0000 7fd92f96e640 1 -- 172.21.15.67:0/3370121519 >> 172.21.15.67:0/3370121519 conn(0x7fd9280798d0 msgr2=0x7fd9281007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:26.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.680+0000 7fd92f96e640 1 -- 172.21.15.67:0/3370121519 shutdown_connections 2024-07-31T14:48:26.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:26.680+0000 7fd92f96e640 1 -- 172.21.15.67:0/3370121519 wait complete. 2024-07-31T14:48:26.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:26 smithi077 ceph-mon[37327]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-07-31T14:48:26.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:26 smithi067 ceph-mon[32750]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-07-31T14:48:27.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:26 smithi073 ceph-mon[36714]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-07-31T14:48:27.329 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-31T14:48:27.329 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph pg dump --format=json 2024-07-31T14:48:27.602 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:27.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:27 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3370121519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:48:27.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:27 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3370121519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:48:28.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:27 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3370121519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:48:28.757 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:28 smithi067 ceph-mon[32750]: pgmap v321: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-07-31T14:48:28.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:28 smithi077 ceph-mon[37327]: pgmap v321: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-07-31T14:48:29.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:28 smithi073 ceph-mon[36714]: pgmap v321: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-07-31T14:48:29.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.287+0000 7f06af59e640 1 -- 172.21.15.67:0/358113805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06b01022b0 msgr2=0x7f06b01026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:29.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.287+0000 7f06af59e640 1 --2- 172.21.15.67:0/358113805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06b01022b0 0x7f06b01026c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009900 tx=0x7f06a002a410 comp rx=0 tx=0).stop 2024-07-31T14:48:29.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.287+0000 7f06af59e640 1 -- 172.21.15.67:0/358113805 shutdown_connections 2024-07-31T14:48:29.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.287+0000 7f06af59e640 1 --2- 172.21.15.67:0/358113805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06b0108540 0x7f06b010ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:29.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.287+0000 7f06af59e640 1 --2- 172.21.15.67:0/358113805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06b0107bf0 0x7f06b0108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:29.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.287+0000 7f06af59e640 1 --2- 172.21.15.67:0/358113805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06b01022b0 0x7f06b01026c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:29.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.287+0000 7f06af59e640 1 -- 172.21.15.67:0/358113805 >> 172.21.15.67:0/358113805 conn(0x7f06b00fdd00 msgr2=0x7f06b0100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:29.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.287+0000 7f06af59e640 1 -- 172.21.15.67:0/358113805 shutdown_connections 2024-07-31T14:48:29.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.287+0000 7f06af59e640 1 -- 172.21.15.67:0/358113805 wait complete. 2024-07-31T14:48:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.288+0000 7f06af59e640 1 Processor -- start 2024-07-31T14:48:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.288+0000 7f06af59e640 1 -- start start 2024-07-31T14:48:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.288+0000 7f06af59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06b01022b0 0x7f06b00fe230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.288+0000 7f06af59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06b0107bf0 0x7f06b00fe770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.288+0000 7f06af59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06b0108540 0x7f06b00fecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.288+0000 7f06af59e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f06b0112210 con 0x7f06b0107bf0 2024-07-31T14:48:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.288+0000 7f06af59e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f06b0112090 con 0x7f06b0108540 2024-07-31T14:48:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.288+0000 7f06af59e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06b0112390 con 0x7f06b01022b0 2024-07-31T14:48:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.288+0000 7f06add9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06b0107bf0 0x7f06b00fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.288+0000 7f06add9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06b0107bf0 0x7f06b00fe770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59932/0 (socket says 172.21.15.67:59932) 2024-07-31T14:48:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.288+0000 7f06add9b640 1 -- 172.21.15.67:0/2947221271 learned_addr learned my addr 172.21.15.67:0/2947221271 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.289+0000 7f06ae59c640 1 --2- 172.21.15.67:0/2947221271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06b01022b0 0x7f06b00fe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.289+0000 7f06add9b640 1 -- 172.21.15.67:0/2947221271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06b01022b0 msgr2=0x7f06b00fe230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.289+0000 7f06add9b640 1 --2- 172.21.15.67:0/2947221271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06b01022b0 0x7f06b00fe230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.289+0000 7f06add9b640 1 -- 172.21.15.67:0/2947221271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06b0108540 msgr2=0x7f06b00fecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.289+0000 7f06add9b640 1 --2- 172.21.15.67:0/2947221271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06b0108540 0x7f06b00fecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.289+0000 7f06add9b640 1 -- 172.21.15.67:0/2947221271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b01a7fd0 con 0x7f06b0107bf0 2024-07-31T14:48:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.289+0000 7f06add9b640 1 --2- 172.21.15.67:0/2947221271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06b0107bf0 0x7f06b00fe770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f06a400d750 tx=0x7f06a400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.289+0000 7f069f7fe640 1 -- 172.21.15.67:0/2947221271 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4017070 con 0x7f06b0107bf0 2024-07-31T14:48:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.289+0000 7f06af59e640 1 -- 172.21.15.67:0/2947221271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b01a82c0 con 0x7f06b0107bf0 2024-07-31T14:48:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.289+0000 7f069f7fe640 1 -- 172.21.15.67:0/2947221271 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06a40054d0 con 0x7f06b0107bf0 2024-07-31T14:48:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.289+0000 7f069f7fe640 1 -- 172.21.15.67:0/2947221271 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4004c30 con 0x7f06b0107bf0 2024-07-31T14:48:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.289+0000 7f06af59e640 1 -- 172.21.15.67:0/2947221271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b01a8800 con 0x7f06b0107bf0 2024-07-31T14:48:29.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.290+0000 7f06af59e640 1 -- 172.21.15.67:0/2947221271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b01026c0 con 0x7f06b0107bf0 2024-07-31T14:48:29.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.292+0000 7f069f7fe640 1 -- 172.21.15.67:0/2947221271 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f06a4004dd0 con 0x7f06b0107bf0 2024-07-31T14:48:29.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.293+0000 7f069f7fe640 1 --2- 172.21.15.67:0/2947221271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f06980b3cb0 0x7f06980b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:29.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.293+0000 7f06ae59c640 1 --2- 172.21.15.67:0/2947221271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f06980b3cb0 0x7f06980b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:29.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.293+0000 7f069f7fe640 1 -- 172.21.15.67:0/2947221271 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f06a40e40f0 con 0x7f06b0107bf0 2024-07-31T14:48:29.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.293+0000 7f06ae59c640 1 --2- 172.21.15.67:0/2947221271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f06980b3cb0 0x7f06980b61c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f06a0000c00 tx=0x7f06a0031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.296+0000 7f069f7fe640 1 -- 172.21.15.67:0/2947221271 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f06a40ab1b0 con 0x7f06b0107bf0 2024-07-31T14:48:29.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.489+0000 7f06af59e640 1 -- 172.21.15.67:0/2947221271 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f06b0105af0 con 0x7f06980b3cb0 2024-07-31T14:48:29.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.495+0000 7f069f7fe640 1 -- 172.21.15.67:0/2947221271 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340401 (secure 0 0 0) 0x7f06b0105af0 con 0x7f06980b3cb0 2024-07-31T14:48:29.501 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:48:29.503 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-31T14:48:29.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.499+0000 7f06af59e640 1 -- 172.21.15.67:0/2947221271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f06980b3cb0 msgr2=0x7f06980b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:29.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.499+0000 7f06af59e640 1 --2- 172.21.15.67:0/2947221271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f06980b3cb0 0x7f06980b61c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f06a0000c00 tx=0x7f06a0031400 comp rx=0 tx=0).stop 2024-07-31T14:48:29.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.499+0000 7f06af59e640 1 -- 172.21.15.67:0/2947221271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06b0107bf0 msgr2=0x7f06b00fe770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:29.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.499+0000 7f06af59e640 1 --2- 172.21.15.67:0/2947221271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06b0107bf0 0x7f06b00fe770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f06a400d750 tx=0x7f06a400dc10 comp rx=0 tx=0).stop 2024-07-31T14:48:29.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.499+0000 7f06af59e640 1 -- 172.21.15.67:0/2947221271 shutdown_connections 2024-07-31T14:48:29.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.499+0000 7f06af59e640 1 --2- 172.21.15.67:0/2947221271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f06980b3cb0 0x7f06980b61c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:29.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.499+0000 7f06af59e640 1 --2- 172.21.15.67:0/2947221271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06b0108540 0x7f06b00fecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:29.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.499+0000 7f06af59e640 1 --2- 172.21.15.67:0/2947221271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06b0107bf0 0x7f06b00fe770 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:29.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.499+0000 7f06af59e640 1 --2- 172.21.15.67:0/2947221271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06b01022b0 0x7f06b00fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:29.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.499+0000 7f06af59e640 1 -- 172.21.15.67:0/2947221271 >> 172.21.15.67:0/2947221271 conn(0x7f06b00fdd00 msgr2=0x7f06b0100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:29.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.499+0000 7f06af59e640 1 -- 172.21.15.67:0/2947221271 shutdown_connections 2024-07-31T14:48:29.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:29.499+0000 7f06af59e640 1 -- 172.21.15.67:0/2947221271 wait complete. 2024-07-31T14:48:30.179 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-07-31T14:48:29.394148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654942,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":354,"num_read_kb":274,"num_write":368,"num_write_kb":2286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":303,"ondisk_log_size":303,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379912,"kb_used_data":14120,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561912,"statfs":{"total":1151940427776,"available":1151551397888,"internally_reserved":0,"allocated":14458880,"data_stored":8172472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32286,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":118,"num_read_kb":87,"num_write":18,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003903"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":77,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:24.637701+0000","last_change":"2024-07-31T14:47:00.020067+0000","last_active":"2024-07-31T14:48:24.637701+0000","last_peered":"2024-07-31T14:48:24.637701+0000","last_clean":"2024-07-31T14:48:24.637701+0000","last_became_active":"2024-07-31T14:47:00.019674+0000","last_became_peered":"2024-07-31T14:47:00.019674+0000","last_unstale":"2024-07-31T14:48:24.637701+0000","last_undegraded":"2024-07-31T14:48:24.637701+0000","last_fullsized":"2024-07-31T14:48:24.637701+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:10:08.193786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626661+0000","last_change":"2024-07-31T14:47:03.052069+0000","last_active":"2024-07-31T14:47:20.626661+0000","last_peered":"2024-07-31T14:47:20.626661+0000","last_clean":"2024-07-31T14:47:20.626661+0000","last_became_active":"2024-07-31T14:47:03.051739+0000","last_became_peered":"2024-07-31T14:47:03.051739+0000","last_unstale":"2024-07-31T14:47:20.626661+0000","last_undegraded":"2024-07-31T14:47:20.626661+0000","last_fullsized":"2024-07-31T14:47:20.626661+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:29:19.952794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.628866+0000","last_change":"2024-07-31T14:47:20.628770+0000","last_active":"2024-07-31T14:47:20.628866+0000","last_peered":"2024-07-31T14:47:20.628866+0000","last_clean":"2024-07-31T14:47:20.628866+0000","last_became_active":"2024-07-31T14:47:20.628142+0000","last_became_peered":"2024-07-31T14:47:20.628142+0000","last_unstale":"2024-07-31T14:47:20.628866+0000","last_undegraded":"2024-07-31T14:47:20.628866+0000","last_fullsized":"2024-07-31T14:47:20.628866+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.659670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627157+0000","last_change":"2024-07-31T14:47:00.183893+0000","last_active":"2024-07-31T14:47:20.627157+0000","last_peered":"2024-07-31T14:47:20.627157+0000","last_clean":"2024-07-31T14:47:20.627157+0000","last_became_active":"2024-07-31T14:47:00.183453+0000","last_became_peered":"2024-07-31T14:47:00.183453+0000","last_unstale":"2024-07-31T14:47:20.627157+0000","last_undegraded":"2024-07-31T14:47:20.627157+0000","last_fullsized":"2024-07-31T14:47:20.627157+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:45:08.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449702+0000","last_change":"2024-07-31T14:47:03.057081+0000","last_active":"2024-07-31T14:47:20.449702+0000","last_peered":"2024-07-31T14:47:20.449702+0000","last_clean":"2024-07-31T14:47:20.449702+0000","last_became_active":"2024-07-31T14:47:03.056061+0000","last_became_peered":"2024-07-31T14:47:03.056061+0000","last_unstale":"2024-07-31T14:47:20.449702+0000","last_undegraded":"2024-07-31T14:47:20.449702+0000","last_fullsized":"2024-07-31T14:47:20.449702+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:28:16.760131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604015+0000","last_change":"2024-07-31T14:47:13.105271+0000","last_active":"2024-07-31T14:47:20.604015+0000","last_peered":"2024-07-31T14:47:20.604015+0000","last_clean":"2024-07-31T14:47:20.604015+0000","last_became_active":"2024-07-31T14:47:13.104717+0000","last_became_peered":"2024-07-31T14:47:13.104717+0000","last_unstale":"2024-07-31T14:47:20.604015+0000","last_undegraded":"2024-07-31T14:47:20.604015+0000","last_fullsized":"2024-07-31T14:47:20.604015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:34:21.413957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605719+0000","last_change":"2024-07-31T14:47:20.454745+0000","last_active":"2024-07-31T14:48:03.605719+0000","last_peered":"2024-07-31T14:48:03.605719+0000","last_clean":"2024-07-31T14:48:03.605719+0000","last_became_active":"2024-07-31T14:47:20.449892+0000","last_became_peered":"2024-07-31T14:47:20.449892+0000","last_unstale":"2024-07-31T14:48:03.605719+0000","last_undegraded":"2024-07-31T14:48:03.605719+0000","last_fullsized":"2024-07-31T14:48:03.605719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:59:17.375344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626206+0000","last_change":"2024-07-31T14:47:03.051085+0000","last_active":"2024-07-31T14:47:20.626206+0000","last_peered":"2024-07-31T14:47:20.626206+0000","last_clean":"2024-07-31T14:47:20.626206+0000","last_became_active":"2024-07-31T14:47:03.050686+0000","last_became_peered":"2024-07-31T14:47:03.050686+0000","last_unstale":"2024-07-31T14:47:20.626206+0000","last_undegraded":"2024-07-31T14:47:20.626206+0000","last_fullsized":"2024-07-31T14:47:20.626206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:48:02.494061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604422+0000","last_change":"2024-07-31T14:47:13.104229+0000","last_active":"2024-07-31T14:47:20.604422+0000","last_peered":"2024-07-31T14:47:20.604422+0000","last_clean":"2024-07-31T14:47:20.604422+0000","last_became_active":"2024-07-31T14:47:13.103737+0000","last_became_peered":"2024-07-31T14:47:13.103737+0000","last_unstale":"2024-07-31T14:47:20.604422+0000","last_undegraded":"2024-07-31T14:47:20.604422+0000","last_fullsized":"2024-07-31T14:47:20.604422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:36:00.541345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600473+0000","last_change":"2024-07-31T14:47:00.028169+0000","last_active":"2024-07-31T14:48:02.600473+0000","last_peered":"2024-07-31T14:48:02.600473+0000","last_clean":"2024-07-31T14:48:02.600473+0000","last_became_active":"2024-07-31T14:47:00.027757+0000","last_became_peered":"2024-07-31T14:47:00.027757+0000","last_unstale":"2024-07-31T14:48:02.600473+0000","last_undegraded":"2024-07-31T14:48:02.600473+0000","last_fullsized":"2024-07-31T14:48:02.600473+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:36:04.419551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626884+0000","last_change":"2024-07-31T14:47:03.057400+0000","last_active":"2024-07-31T14:47:20.626884+0000","last_peered":"2024-07-31T14:47:20.626884+0000","last_clean":"2024-07-31T14:47:20.626884+0000","last_became_active":"2024-07-31T14:47:03.057035+0000","last_became_peered":"2024-07-31T14:47:03.057035+0000","last_unstale":"2024-07-31T14:47:20.626884+0000","last_undegraded":"2024-07-31T14:47:20.626884+0000","last_fullsized":"2024-07-31T14:47:20.626884+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:55:07.951076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604150+0000","last_change":"2024-07-31T14:47:13.106093+0000","last_active":"2024-07-31T14:47:20.604150+0000","last_peered":"2024-07-31T14:47:20.604150+0000","last_clean":"2024-07-31T14:47:20.604150+0000","last_became_active":"2024-07-31T14:47:13.105459+0000","last_became_peered":"2024-07-31T14:47:13.105459+0000","last_unstale":"2024-07-31T14:47:20.604150+0000","last_undegraded":"2024-07-31T14:47:20.604150+0000","last_fullsized":"2024-07-31T14:47:20.604150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:07:52.751245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444336+0000","last_change":"2024-07-31T14:47:03.058701+0000","last_active":"2024-07-31T14:47:20.444336+0000","last_peered":"2024-07-31T14:47:20.444336+0000","last_clean":"2024-07-31T14:47:20.444336+0000","last_became_active":"2024-07-31T14:47:03.058359+0000","last_became_peered":"2024-07-31T14:47:03.058359+0000","last_unstale":"2024-07-31T14:47:20.444336+0000","last_undegraded":"2024-07-31T14:47:20.444336+0000","last_fullsized":"2024-07-31T14:47:20.444336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:37:36.149884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:59.565453+0000","last_change":"2024-07-31T14:47:00.022983+0000","last_active":"2024-07-31T14:47:59.565453+0000","last_peered":"2024-07-31T14:47:59.565453+0000","last_clean":"2024-07-31T14:47:59.565453+0000","last_became_active":"2024-07-31T14:47:00.022586+0000","last_became_peered":"2024-07-31T14:47:00.022586+0000","last_unstale":"2024-07-31T14:47:59.565453+0000","last_undegraded":"2024-07-31T14:47:59.565453+0000","last_fullsized":"2024-07-31T14:47:59.565453+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:29:42.641490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604397+0000","last_change":"2024-07-31T14:47:13.363148+0000","last_active":"2024-07-31T14:47:20.604397+0000","last_peered":"2024-07-31T14:47:20.604397+0000","last_clean":"2024-07-31T14:47:20.604397+0000","last_became_active":"2024-07-31T14:47:13.362577+0000","last_became_peered":"2024-07-31T14:47:13.362577+0000","last_unstale":"2024-07-31T14:47:20.604397+0000","last_undegraded":"2024-07-31T14:47:20.604397+0000","last_fullsized":"2024-07-31T14:47:20.604397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:37:59.003260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626394+0000","last_change":"2024-07-31T14:47:03.049586+0000","last_active":"2024-07-31T14:47:20.626394+0000","last_peered":"2024-07-31T14:47:20.626394+0000","last_clean":"2024-07-31T14:47:20.626394+0000","last_became_active":"2024-07-31T14:47:03.049154+0000","last_became_peered":"2024-07-31T14:47:03.049154+0000","last_unstale":"2024-07-31T14:47:20.626394+0000","last_undegraded":"2024-07-31T14:47:20.626394+0000","last_fullsized":"2024-07-31T14:47:20.626394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:51:01.971854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.589394+0000","last_change":"2024-07-31T14:47:00.021254+0000","last_active":"2024-07-31T14:48:01.589394+0000","last_peered":"2024-07-31T14:48:01.589394+0000","last_clean":"2024-07-31T14:48:01.589394+0000","last_became_active":"2024-07-31T14:47:00.020690+0000","last_became_peered":"2024-07-31T14:47:00.020690+0000","last_unstale":"2024-07-31T14:48:01.589394+0000","last_undegraded":"2024-07-31T14:48:01.589394+0000","last_fullsized":"2024-07-31T14:48:01.589394+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:01:27.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604729+0000","last_change":"2024-07-31T14:47:13.102220+0000","last_active":"2024-07-31T14:47:20.604729+0000","last_peered":"2024-07-31T14:47:20.604729+0000","last_clean":"2024-07-31T14:47:20.604729+0000","last_became_active":"2024-07-31T14:47:13.101764+0000","last_became_peered":"2024-07-31T14:47:13.101764+0000","last_unstale":"2024-07-31T14:47:20.604729+0000","last_undegraded":"2024-07-31T14:47:20.604729+0000","last_fullsized":"2024-07-31T14:47:20.604729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:56:44.335340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444911+0000","last_change":"2024-07-31T14:47:03.052348+0000","last_active":"2024-07-31T14:47:20.444911+0000","last_peered":"2024-07-31T14:47:20.444911+0000","last_clean":"2024-07-31T14:47:20.444911+0000","last_became_active":"2024-07-31T14:47:03.051961+0000","last_became_peered":"2024-07-31T14:47:03.051961+0000","last_unstale":"2024-07-31T14:47:20.444911+0000","last_undegraded":"2024-07-31T14:47:20.444911+0000","last_fullsized":"2024-07-31T14:47:20.444911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:19:35.279809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.597015+0000","last_change":"2024-07-31T14:47:00.025013+0000","last_active":"2024-07-31T14:48:02.597015+0000","last_peered":"2024-07-31T14:48:02.597015+0000","last_clean":"2024-07-31T14:48:02.597015+0000","last_became_active":"2024-07-31T14:47:00.024637+0000","last_became_peered":"2024-07-31T14:47:00.024637+0000","last_unstale":"2024-07-31T14:48:02.597015+0000","last_undegraded":"2024-07-31T14:48:02.597015+0000","last_fullsized":"2024-07-31T14:48:02.597015+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:07:21.519549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.629337+0000","last_change":"2024-07-31T14:47:20.629262+0000","last_active":"2024-07-31T14:47:20.629337+0000","last_peered":"2024-07-31T14:47:20.629337+0000","last_clean":"2024-07-31T14:47:20.629337+0000","last_became_active":"2024-07-31T14:47:20.628318+0000","last_became_peered":"2024-07-31T14:47:20.628318+0000","last_unstale":"2024-07-31T14:47:20.629337+0000","last_undegraded":"2024-07-31T14:47:20.629337+0000","last_fullsized":"2024-07-31T14:47:20.629337+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.206705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451066+0000","last_change":"2024-07-31T14:47:03.051527+0000","last_active":"2024-07-31T14:47:20.451066+0000","last_peered":"2024-07-31T14:47:20.451066+0000","last_clean":"2024-07-31T14:47:20.451066+0000","last_became_active":"2024-07-31T14:47:03.051139+0000","last_became_peered":"2024-07-31T14:47:03.051139+0000","last_unstale":"2024-07-31T14:47:20.451066+0000","last_undegraded":"2024-07-31T14:47:20.451066+0000","last_fullsized":"2024-07-31T14:47:20.451066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:53:11.962719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:59.564906+0000","last_change":"2024-07-31T14:47:00.183812+0000","last_active":"2024-07-31T14:47:59.564906+0000","last_peered":"2024-07-31T14:47:59.564906+0000","last_clean":"2024-07-31T14:47:59.564906+0000","last_became_active":"2024-07-31T14:47:00.183440+0000","last_became_peered":"2024-07-31T14:47:00.183440+0000","last_unstale":"2024-07-31T14:47:59.564906+0000","last_undegraded":"2024-07-31T14:47:59.564906+0000","last_fullsized":"2024-07-31T14:47:59.564906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:47:23.166756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604273+0000","last_change":"2024-07-31T14:47:13.363563+0000","last_active":"2024-07-31T14:47:20.604273+0000","last_peered":"2024-07-31T14:47:20.604273+0000","last_clean":"2024-07-31T14:47:20.604273+0000","last_became_active":"2024-07-31T14:47:13.362697+0000","last_became_peered":"2024-07-31T14:47:13.362697+0000","last_unstale":"2024-07-31T14:47:20.604273+0000","last_undegraded":"2024-07-31T14:47:20.604273+0000","last_fullsized":"2024-07-31T14:47:20.604273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:26:38.292073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449255+0000","last_change":"2024-07-31T14:47:03.046763+0000","last_active":"2024-07-31T14:47:20.449255+0000","last_peered":"2024-07-31T14:47:20.449255+0000","last_clean":"2024-07-31T14:47:20.449255+0000","last_became_active":"2024-07-31T14:47:03.046408+0000","last_became_peered":"2024-07-31T14:47:03.046408+0000","last_unstale":"2024-07-31T14:47:20.449255+0000","last_undegraded":"2024-07-31T14:47:20.449255+0000","last_fullsized":"2024-07-31T14:47:20.449255+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:43:22.838495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.599786+0000","last_change":"2024-07-31T14:47:00.236075+0000","last_active":"2024-07-31T14:48:02.599786+0000","last_peered":"2024-07-31T14:48:02.599786+0000","last_clean":"2024-07-31T14:48:02.599786+0000","last_became_active":"2024-07-31T14:47:00.235758+0000","last_became_peered":"2024-07-31T14:47:00.235758+0000","last_unstale":"2024-07-31T14:48:02.599786+0000","last_undegraded":"2024-07-31T14:48:02.599786+0000","last_fullsized":"2024-07-31T14:48:02.599786+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:50.575648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.448155+0000","last_change":"2024-07-31T14:47:13.101876+0000","last_active":"2024-07-31T14:47:20.448155+0000","last_peered":"2024-07-31T14:47:20.448155+0000","last_clean":"2024-07-31T14:47:20.448155+0000","last_became_active":"2024-07-31T14:47:13.101276+0000","last_became_peered":"2024-07-31T14:47:13.101276+0000","last_unstale":"2024-07-31T14:47:20.448155+0000","last_undegraded":"2024-07-31T14:47:20.448155+0000","last_fullsized":"2024-07-31T14:47:20.448155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:53:24.684212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627336+0000","last_change":"2024-07-31T14:47:03.052478+0000","last_active":"2024-07-31T14:47:20.627336+0000","last_peered":"2024-07-31T14:47:20.627336+0000","last_clean":"2024-07-31T14:47:20.627336+0000","last_became_active":"2024-07-31T14:47:03.051877+0000","last_became_peered":"2024-07-31T14:47:03.051877+0000","last_unstale":"2024-07-31T14:47:20.627336+0000","last_undegraded":"2024-07-31T14:47:20.627336+0000","last_fullsized":"2024-07-31T14:47:20.627336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:30:32.666490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:21.058148+0000","last_change":"2024-07-31T14:47:00.020076+0000","last_active":"2024-07-31T14:48:21.058148+0000","last_peered":"2024-07-31T14:48:21.058148+0000","last_clean":"2024-07-31T14:48:21.058148+0000","last_became_active":"2024-07-31T14:47:00.019708+0000","last_became_peered":"2024-07-31T14:47:00.019708+0000","last_unstale":"2024-07-31T14:48:21.058148+0000","last_undegraded":"2024-07-31T14:48:21.058148+0000","last_fullsized":"2024-07-31T14:48:21.058148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:46.520997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625774+0000","last_change":"2024-07-31T14:47:13.362961+0000","last_active":"2024-07-31T14:47:20.625774+0000","last_peered":"2024-07-31T14:47:20.625774+0000","last_clean":"2024-07-31T14:47:20.625774+0000","last_became_active":"2024-07-31T14:47:13.362502+0000","last_became_peered":"2024-07-31T14:47:13.362502+0000","last_unstale":"2024-07-31T14:47:20.625774+0000","last_undegraded":"2024-07-31T14:47:20.625774+0000","last_fullsized":"2024-07-31T14:47:20.625774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:51:45.732477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447467+0000","last_change":"2024-07-31T14:47:03.039418+0000","last_active":"2024-07-31T14:47:20.447467+0000","last_peered":"2024-07-31T14:47:20.447467+0000","last_clean":"2024-07-31T14:47:20.447467+0000","last_became_active":"2024-07-31T14:47:03.039044+0000","last_became_peered":"2024-07-31T14:47:03.039044+0000","last_unstale":"2024-07-31T14:47:20.447467+0000","last_undegraded":"2024-07-31T14:47:20.447467+0000","last_fullsized":"2024-07-31T14:47:20.447467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:40:10.380080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.599492+0000","last_change":"2024-07-31T14:47:00.183778+0000","last_active":"2024-07-31T14:48:02.599492+0000","last_peered":"2024-07-31T14:48:02.599492+0000","last_clean":"2024-07-31T14:48:02.599492+0000","last_became_active":"2024-07-31T14:47:00.183370+0000","last_became_peered":"2024-07-31T14:47:00.183370+0000","last_unstale":"2024-07-31T14:48:02.599492+0000","last_undegraded":"2024-07-31T14:48:02.599492+0000","last_fullsized":"2024-07-31T14:48:02.599492+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:45:27.422216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445075+0000","last_change":"2024-07-31T14:47:13.102583+0000","last_active":"2024-07-31T14:47:20.445075+0000","last_peered":"2024-07-31T14:47:20.445075+0000","last_clean":"2024-07-31T14:47:20.445075+0000","last_became_active":"2024-07-31T14:47:13.102058+0000","last_became_peered":"2024-07-31T14:47:13.102058+0000","last_unstale":"2024-07-31T14:47:20.445075+0000","last_undegraded":"2024-07-31T14:47:20.445075+0000","last_fullsized":"2024-07-31T14:47:20.445075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:17.396173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626564+0000","last_change":"2024-07-31T14:47:03.048334+0000","last_active":"2024-07-31T14:47:20.626564+0000","last_peered":"2024-07-31T14:47:20.626564+0000","last_clean":"2024-07-31T14:47:20.626564+0000","last_became_active":"2024-07-31T14:47:03.047932+0000","last_became_peered":"2024-07-31T14:47:03.047932+0000","last_unstale":"2024-07-31T14:47:20.626564+0000","last_undegraded":"2024-07-31T14:47:20.626564+0000","last_fullsized":"2024-07-31T14:47:20.626564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:14:31.381061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451611+0000","last_change":"2024-07-31T14:47:20.450851+0000","last_active":"2024-07-31T14:47:20.451611+0000","last_peered":"2024-07-31T14:47:20.451611+0000","last_clean":"2024-07-31T14:47:20.451611+0000","last_became_active":"2024-07-31T14:47:20.450435+0000","last_became_peered":"2024-07-31T14:47:20.450435+0000","last_unstale":"2024-07-31T14:47:20.451611+0000","last_undegraded":"2024-07-31T14:47:20.451611+0000","last_fullsized":"2024-07-31T14:47:20.451611+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:17:28.738911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.450787+0000","last_change":"2024-07-31T14:47:13.363054+0000","last_active":"2024-07-31T14:47:20.450787+0000","last_peered":"2024-07-31T14:47:20.450787+0000","last_clean":"2024-07-31T14:47:20.450787+0000","last_became_active":"2024-07-31T14:47:13.362364+0000","last_became_peered":"2024-07-31T14:47:13.362364+0000","last_unstale":"2024-07-31T14:47:20.450787+0000","last_undegraded":"2024-07-31T14:47:20.450787+0000","last_fullsized":"2024-07-31T14:47:20.450787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:02:36.569678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626675+0000","last_change":"2024-07-31T14:47:03.059834+0000","last_active":"2024-07-31T14:47:20.626675+0000","last_peered":"2024-07-31T14:47:20.626675+0000","last_clean":"2024-07-31T14:47:20.626675+0000","last_became_active":"2024-07-31T14:47:03.059127+0000","last_became_peered":"2024-07-31T14:47:03.059127+0000","last_unstale":"2024-07-31T14:47:20.626675+0000","last_undegraded":"2024-07-31T14:47:20.626675+0000","last_fullsized":"2024-07-31T14:47:20.626675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:53:50.308789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":72,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:24.638401+0000","last_change":"2024-07-31T14:47:00.030576+0000","last_active":"2024-07-31T14:48:24.638401+0000","last_peered":"2024-07-31T14:48:24.638401+0000","last_clean":"2024-07-31T14:48:24.638401+0000","last_became_active":"2024-07-31T14:47:00.030174+0000","last_became_peered":"2024-07-31T14:47:00.030174+0000","last_unstale":"2024-07-31T14:48:24.638401+0000","last_undegraded":"2024-07-31T14:48:24.638401+0000","last_fullsized":"2024-07-31T14:48:24.638401+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:13:35.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447207+0000","last_change":"2024-07-31T14:47:13.100180+0000","last_active":"2024-07-31T14:47:20.447207+0000","last_peered":"2024-07-31T14:47:20.447207+0000","last_clean":"2024-07-31T14:47:20.447207+0000","last_became_active":"2024-07-31T14:47:13.099725+0000","last_became_peered":"2024-07-31T14:47:13.099725+0000","last_unstale":"2024-07-31T14:47:20.447207+0000","last_undegraded":"2024-07-31T14:47:20.447207+0000","last_fullsized":"2024-07-31T14:47:20.447207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:25:32.245339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626501+0000","last_change":"2024-07-31T14:47:03.040349+0000","last_active":"2024-07-31T14:47:20.626501+0000","last_peered":"2024-07-31T14:47:20.626501+0000","last_clean":"2024-07-31T14:47:20.626501+0000","last_became_active":"2024-07-31T14:47:03.039887+0000","last_became_peered":"2024-07-31T14:47:03.039887+0000","last_unstale":"2024-07-31T14:47:20.626501+0000","last_undegraded":"2024-07-31T14:47:20.626501+0000","last_fullsized":"2024-07-31T14:47:20.626501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:46.592063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":72,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:26.640694+0000","last_change":"2024-07-31T14:47:00.183988+0000","last_active":"2024-07-31T14:48:26.640694+0000","last_peered":"2024-07-31T14:48:26.640694+0000","last_clean":"2024-07-31T14:48:26.640694+0000","last_became_active":"2024-07-31T14:47:00.183302+0000","last_became_peered":"2024-07-31T14:47:00.183302+0000","last_unstale":"2024-07-31T14:48:26.640694+0000","last_undegraded":"2024-07-31T14:48:26.640694+0000","last_fullsized":"2024-07-31T14:48:26.640694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:40:26.188224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447685+0000","last_change":"2024-07-31T14:47:13.363075+0000","last_active":"2024-07-31T14:47:20.447685+0000","last_peered":"2024-07-31T14:47:20.447685+0000","last_clean":"2024-07-31T14:47:20.447685+0000","last_became_active":"2024-07-31T14:47:13.362599+0000","last_became_peered":"2024-07-31T14:47:13.362599+0000","last_unstale":"2024-07-31T14:47:20.447685+0000","last_undegraded":"2024-07-31T14:47:20.447685+0000","last_fullsized":"2024-07-31T14:47:20.447685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:20:43.617742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.450794+0000","last_change":"2024-07-31T14:47:20.450712+0000","last_active":"2024-07-31T14:47:20.450794+0000","last_peered":"2024-07-31T14:47:20.450794+0000","last_clean":"2024-07-31T14:47:20.450794+0000","last_became_active":"2024-07-31T14:47:20.450229+0000","last_became_peered":"2024-07-31T14:47:20.450229+0000","last_unstale":"2024-07-31T14:47:20.450794+0000","last_undegraded":"2024-07-31T14:47:20.450794+0000","last_fullsized":"2024-07-31T14:47:20.450794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:11:50.631420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:00.569141+0000","last_change":"2024-07-31T14:47:00.021389+0000","last_active":"2024-07-31T14:48:00.569141+0000","last_peered":"2024-07-31T14:48:00.569141+0000","last_clean":"2024-07-31T14:48:00.569141+0000","last_became_active":"2024-07-31T14:47:00.020987+0000","last_became_peered":"2024-07-31T14:47:00.020987+0000","last_unstale":"2024-07-31T14:48:00.569141+0000","last_undegraded":"2024-07-31T14:48:00.569141+0000","last_fullsized":"2024-07-31T14:48:00.569141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:07:24.658463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626950+0000","last_change":"2024-07-31T14:47:13.363105+0000","last_active":"2024-07-31T14:47:20.626950+0000","last_peered":"2024-07-31T14:47:20.626950+0000","last_clean":"2024-07-31T14:47:20.626950+0000","last_became_active":"2024-07-31T14:47:13.362404+0000","last_became_peered":"2024-07-31T14:47:13.362404+0000","last_unstale":"2024-07-31T14:47:20.626950+0000","last_undegraded":"2024-07-31T14:47:20.626950+0000","last_fullsized":"2024-07-31T14:47:20.626950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:48:43.033584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626140+0000","last_change":"2024-07-31T14:47:03.057199+0000","last_active":"2024-07-31T14:47:20.626140+0000","last_peered":"2024-07-31T14:47:20.626140+0000","last_clean":"2024-07-31T14:47:20.626140+0000","last_became_active":"2024-07-31T14:47:03.056555+0000","last_became_peered":"2024-07-31T14:47:03.056555+0000","last_unstale":"2024-07-31T14:47:20.626140+0000","last_undegraded":"2024-07-31T14:47:20.626140+0000","last_fullsized":"2024-07-31T14:47:20.626140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:21:14.263457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605066+0000","last_change":"2024-07-31T14:47:00.032055+0000","last_active":"2024-07-31T14:48:03.605066+0000","last_peered":"2024-07-31T14:48:03.605066+0000","last_clean":"2024-07-31T14:48:03.605066+0000","last_became_active":"2024-07-31T14:47:00.031519+0000","last_became_peered":"2024-07-31T14:47:00.031519+0000","last_unstale":"2024-07-31T14:48:03.605066+0000","last_undegraded":"2024-07-31T14:48:03.605066+0000","last_fullsized":"2024-07-31T14:48:03.605066+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:21:46.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":117,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444372+0000","last_change":"2024-07-31T14:45:42.166503+0000","last_active":"2024-07-31T14:47:20.444372+0000","last_peered":"2024-07-31T14:47:20.444372+0000","last_clean":"2024-07-31T14:47:20.444372+0000","last_became_active":"2024-07-31T14:45:42.160065+0000","last_became_peered":"2024-07-31T14:45:42.160065+0000","last_unstale":"2024-07-31T14:47:20.444372+0000","last_undegraded":"2024-07-31T14:47:20.444372+0000","last_fullsized":"2024-07-31T14:47:20.444372+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_clean_scrub_stamp":"2024-07-31T14:42:27.863113+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:38:33.646022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444767+0000","last_change":"2024-07-31T14:47:13.103812+0000","last_active":"2024-07-31T14:47:20.444767+0000","last_peered":"2024-07-31T14:47:20.444767+0000","last_clean":"2024-07-31T14:47:20.444767+0000","last_became_active":"2024-07-31T14:47:13.102549+0000","last_became_peered":"2024-07-31T14:47:13.102549+0000","last_unstale":"2024-07-31T14:47:20.444767+0000","last_undegraded":"2024-07-31T14:47:20.444767+0000","last_fullsized":"2024-07-31T14:47:20.444767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:18:56.322426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444837+0000","last_change":"2024-07-31T14:47:03.056310+0000","last_active":"2024-07-31T14:47:20.444837+0000","last_peered":"2024-07-31T14:47:20.444837+0000","last_clean":"2024-07-31T14:47:20.444837+0000","last_became_active":"2024-07-31T14:47:03.055695+0000","last_became_peered":"2024-07-31T14:47:03.055695+0000","last_unstale":"2024-07-31T14:47:20.444837+0000","last_undegraded":"2024-07-31T14:47:20.444837+0000","last_fullsized":"2024-07-31T14:47:20.444837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:00:55.124981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":70,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:24.639115+0000","last_change":"2024-07-31T14:47:00.236460+0000","last_active":"2024-07-31T14:48:24.639115+0000","last_peered":"2024-07-31T14:48:24.639115+0000","last_clean":"2024-07-31T14:48:24.639115+0000","last_became_active":"2024-07-31T14:47:00.235924+0000","last_became_peered":"2024-07-31T14:47:00.235924+0000","last_unstale":"2024-07-31T14:48:24.639115+0000","last_undegraded":"2024-07-31T14:48:24.639115+0000","last_fullsized":"2024-07-31T14:48:24.639115+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:05.130958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625884+0000","last_change":"2024-07-31T14:47:13.102341+0000","last_active":"2024-07-31T14:47:20.625884+0000","last_peered":"2024-07-31T14:47:20.625884+0000","last_clean":"2024-07-31T14:47:20.625884+0000","last_became_active":"2024-07-31T14:47:13.101798+0000","last_became_peered":"2024-07-31T14:47:13.101798+0000","last_unstale":"2024-07-31T14:47:20.625884+0000","last_undegraded":"2024-07-31T14:47:20.625884+0000","last_fullsized":"2024-07-31T14:47:20.625884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:34:35.758041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444646+0000","last_change":"2024-07-31T14:47:03.054322+0000","last_active":"2024-07-31T14:47:20.444646+0000","last_peered":"2024-07-31T14:47:20.444646+0000","last_clean":"2024-07-31T14:47:20.444646+0000","last_became_active":"2024-07-31T14:47:03.053112+0000","last_became_peered":"2024-07-31T14:47:03.053112+0000","last_unstale":"2024-07-31T14:47:20.444646+0000","last_undegraded":"2024-07-31T14:47:20.444646+0000","last_fullsized":"2024-07-31T14:47:20.444646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:55:13.542578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600103+0000","last_change":"2024-07-31T14:47:00.026518+0000","last_active":"2024-07-31T14:48:02.600103+0000","last_peered":"2024-07-31T14:48:02.600103+0000","last_clean":"2024-07-31T14:48:02.600103+0000","last_became_active":"2024-07-31T14:47:00.026074+0000","last_became_peered":"2024-07-31T14:47:00.026074+0000","last_unstale":"2024-07-31T14:48:02.600103+0000","last_undegraded":"2024-07-31T14:48:02.600103+0000","last_fullsized":"2024-07-31T14:48:02.600103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:33:36.688353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449307+0000","last_change":"2024-07-31T14:47:13.106365+0000","last_active":"2024-07-31T14:47:20.449307+0000","last_peered":"2024-07-31T14:47:20.449307+0000","last_clean":"2024-07-31T14:47:20.449307+0000","last_became_active":"2024-07-31T14:47:13.105635+0000","last_became_peered":"2024-07-31T14:47:13.105635+0000","last_unstale":"2024-07-31T14:47:20.449307+0000","last_undegraded":"2024-07-31T14:47:20.449307+0000","last_fullsized":"2024-07-31T14:47:20.449307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:52:09.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444806+0000","last_change":"2024-07-31T14:47:03.052522+0000","last_active":"2024-07-31T14:47:20.444806+0000","last_peered":"2024-07-31T14:47:20.444806+0000","last_clean":"2024-07-31T14:47:20.444806+0000","last_became_active":"2024-07-31T14:47:03.051872+0000","last_became_peered":"2024-07-31T14:47:03.051872+0000","last_unstale":"2024-07-31T14:47:20.444806+0000","last_undegraded":"2024-07-31T14:47:20.444806+0000","last_fullsized":"2024-07-31T14:47:20.444806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:29:34.956745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600858+0000","last_change":"2024-07-31T14:47:00.016839+0000","last_active":"2024-07-31T14:48:02.600858+0000","last_peered":"2024-07-31T14:48:02.600858+0000","last_clean":"2024-07-31T14:48:02.600858+0000","last_became_active":"2024-07-31T14:47:00.016438+0000","last_became_peered":"2024-07-31T14:47:00.016438+0000","last_unstale":"2024-07-31T14:48:02.600858+0000","last_undegraded":"2024-07-31T14:48:02.600858+0000","last_fullsized":"2024-07-31T14:48:02.600858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:42:10.875332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626192+0000","last_change":"2024-07-31T14:47:13.105385+0000","last_active":"2024-07-31T14:47:20.626192+0000","last_peered":"2024-07-31T14:47:20.626192+0000","last_clean":"2024-07-31T14:47:20.626192+0000","last_became_active":"2024-07-31T14:47:13.104867+0000","last_became_peered":"2024-07-31T14:47:13.104867+0000","last_unstale":"2024-07-31T14:47:20.626192+0000","last_undegraded":"2024-07-31T14:47:20.626192+0000","last_fullsized":"2024-07-31T14:47:20.626192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:25:21.572830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445617+0000","last_change":"2024-07-31T14:47:03.058304+0000","last_active":"2024-07-31T14:47:20.445617+0000","last_peered":"2024-07-31T14:47:20.445617+0000","last_clean":"2024-07-31T14:47:20.445617+0000","last_became_active":"2024-07-31T14:47:03.057251+0000","last_became_peered":"2024-07-31T14:47:03.057251+0000","last_unstale":"2024-07-31T14:47:20.445617+0000","last_undegraded":"2024-07-31T14:47:20.445617+0000","last_fullsized":"2024-07-31T14:47:20.445617+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:58:20.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.566267+0000","last_change":"2024-07-31T14:47:00.027806+0000","last_active":"2024-07-31T14:48:02.566267+0000","last_peered":"2024-07-31T14:48:02.566267+0000","last_clean":"2024-07-31T14:48:02.566267+0000","last_became_active":"2024-07-31T14:47:00.027403+0000","last_became_peered":"2024-07-31T14:47:00.027403+0000","last_unstale":"2024-07-31T14:48:02.566267+0000","last_undegraded":"2024-07-31T14:48:02.566267+0000","last_fullsized":"2024-07-31T14:48:02.566267+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:57:18.303732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626432+0000","last_change":"2024-07-31T14:47:13.103628+0000","last_active":"2024-07-31T14:47:20.626432+0000","last_peered":"2024-07-31T14:47:20.626432+0000","last_clean":"2024-07-31T14:47:20.626432+0000","last_became_active":"2024-07-31T14:47:13.102296+0000","last_became_peered":"2024-07-31T14:47:13.102296+0000","last_unstale":"2024-07-31T14:47:20.626432+0000","last_undegraded":"2024-07-31T14:47:20.626432+0000","last_fullsized":"2024-07-31T14:47:20.626432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:25:35.109196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626520+0000","last_change":"2024-07-31T14:47:03.052761+0000","last_active":"2024-07-31T14:47:20.626520+0000","last_peered":"2024-07-31T14:47:20.626520+0000","last_clean":"2024-07-31T14:47:20.626520+0000","last_became_active":"2024-07-31T14:47:03.052410+0000","last_became_peered":"2024-07-31T14:47:03.052410+0000","last_unstale":"2024-07-31T14:47:20.626520+0000","last_undegraded":"2024-07-31T14:47:20.626520+0000","last_fullsized":"2024-07-31T14:47:20.626520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:20:47.712491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.599712+0000","last_change":"2024-07-31T14:47:00.183741+0000","last_active":"2024-07-31T14:48:02.599712+0000","last_peered":"2024-07-31T14:48:02.599712+0000","last_clean":"2024-07-31T14:48:02.599712+0000","last_became_active":"2024-07-31T14:47:00.183275+0000","last_became_peered":"2024-07-31T14:47:00.183275+0000","last_unstale":"2024-07-31T14:48:02.599712+0000","last_undegraded":"2024-07-31T14:48:02.599712+0000","last_fullsized":"2024-07-31T14:48:02.599712+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:29:23.205835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.629408+0000","last_change":"2024-07-31T14:47:20.629334+0000","last_active":"2024-07-31T14:47:20.629408+0000","last_peered":"2024-07-31T14:47:20.629408+0000","last_clean":"2024-07-31T14:47:20.629408+0000","last_became_active":"2024-07-31T14:47:20.628865+0000","last_became_peered":"2024-07-31T14:47:20.628865+0000","last_unstale":"2024-07-31T14:47:20.629408+0000","last_undegraded":"2024-07-31T14:47:20.629408+0000","last_fullsized":"2024-07-31T14:47:20.629408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:21:27.199842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444090+0000","last_change":"2024-07-31T14:47:03.054633+0000","last_active":"2024-07-31T14:47:20.444090+0000","last_peered":"2024-07-31T14:47:20.444090+0000","last_clean":"2024-07-31T14:47:20.444090+0000","last_became_active":"2024-07-31T14:47:03.053025+0000","last_became_peered":"2024-07-31T14:47:03.053025+0000","last_unstale":"2024-07-31T14:47:20.444090+0000","last_undegraded":"2024-07-31T14:47:20.444090+0000","last_fullsized":"2024-07-31T14:47:20.444090+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:42:03.392744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:27.642641+0000","last_change":"2024-07-31T14:47:00.027898+0000","last_active":"2024-07-31T14:48:27.642641+0000","last_peered":"2024-07-31T14:48:27.642641+0000","last_clean":"2024-07-31T14:48:27.642641+0000","last_became_active":"2024-07-31T14:47:00.027468+0000","last_became_peered":"2024-07-31T14:47:00.027468+0000","last_unstale":"2024-07-31T14:48:27.642641+0000","last_undegraded":"2024-07-31T14:48:27.642641+0000","last_fullsized":"2024-07-31T14:48:27.642641+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:12:45.922172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444515+0000","last_change":"2024-07-31T14:47:13.104327+0000","last_active":"2024-07-31T14:47:20.444515+0000","last_peered":"2024-07-31T14:47:20.444515+0000","last_clean":"2024-07-31T14:47:20.444515+0000","last_became_active":"2024-07-31T14:47:13.103333+0000","last_became_peered":"2024-07-31T14:47:13.103333+0000","last_unstale":"2024-07-31T14:47:20.444515+0000","last_undegraded":"2024-07-31T14:47:20.444515+0000","last_fullsized":"2024-07-31T14:47:20.444515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:06.226220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444413+0000","last_change":"2024-07-31T14:47:03.044875+0000","last_active":"2024-07-31T14:47:20.444413+0000","last_peered":"2024-07-31T14:47:20.444413+0000","last_clean":"2024-07-31T14:47:20.444413+0000","last_became_active":"2024-07-31T14:47:03.044421+0000","last_became_peered":"2024-07-31T14:47:03.044421+0000","last_unstale":"2024-07-31T14:47:20.444413+0000","last_undegraded":"2024-07-31T14:47:20.444413+0000","last_fullsized":"2024-07-31T14:47:20.444413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:00:02.477300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600649+0000","last_change":"2024-07-31T14:47:00.019790+0000","last_active":"2024-07-31T14:48:02.600649+0000","last_peered":"2024-07-31T14:48:02.600649+0000","last_clean":"2024-07-31T14:48:02.600649+0000","last_became_active":"2024-07-31T14:47:00.019368+0000","last_became_peered":"2024-07-31T14:47:00.019368+0000","last_unstale":"2024-07-31T14:48:02.600649+0000","last_undegraded":"2024-07-31T14:48:02.600649+0000","last_fullsized":"2024-07-31T14:48:02.600649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:15.299109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445210+0000","last_change":"2024-07-31T14:47:13.102618+0000","last_active":"2024-07-31T14:47:20.445210+0000","last_peered":"2024-07-31T14:47:20.445210+0000","last_clean":"2024-07-31T14:47:20.445210+0000","last_became_active":"2024-07-31T14:47:13.102085+0000","last_became_peered":"2024-07-31T14:47:13.102085+0000","last_unstale":"2024-07-31T14:47:20.445210+0000","last_undegraded":"2024-07-31T14:47:20.445210+0000","last_fullsized":"2024-07-31T14:47:20.445210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:05:58.814215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447335+0000","last_change":"2024-07-31T14:47:03.044688+0000","last_active":"2024-07-31T14:47:20.447335+0000","last_peered":"2024-07-31T14:47:20.447335+0000","last_clean":"2024-07-31T14:47:20.447335+0000","last_became_active":"2024-07-31T14:47:03.043751+0000","last_became_peered":"2024-07-31T14:47:03.043751+0000","last_unstale":"2024-07-31T14:47:20.447335+0000","last_undegraded":"2024-07-31T14:47:20.447335+0000","last_fullsized":"2024-07-31T14:47:20.447335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:47:31.450796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604782+0000","last_change":"2024-07-31T14:47:00.026259+0000","last_active":"2024-07-31T14:48:03.604782+0000","last_peered":"2024-07-31T14:48:03.604782+0000","last_clean":"2024-07-31T14:48:03.604782+0000","last_became_active":"2024-07-31T14:47:00.025756+0000","last_became_peered":"2024-07-31T14:47:00.025756+0000","last_unstale":"2024-07-31T14:48:03.604782+0000","last_undegraded":"2024-07-31T14:48:03.604782+0000","last_fullsized":"2024-07-31T14:48:03.604782+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:05:00.258841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626813+0000","last_change":"2024-07-31T14:47:13.104918+0000","last_active":"2024-07-31T14:47:20.626813+0000","last_peered":"2024-07-31T14:47:20.626813+0000","last_clean":"2024-07-31T14:47:20.626813+0000","last_became_active":"2024-07-31T14:47:13.104031+0000","last_became_peered":"2024-07-31T14:47:13.104031+0000","last_unstale":"2024-07-31T14:47:20.626813+0000","last_undegraded":"2024-07-31T14:47:20.626813+0000","last_fullsized":"2024-07-31T14:47:20.626813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:17:15.264474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451830+0000","last_change":"2024-07-31T14:47:03.055159+0000","last_active":"2024-07-31T14:47:20.451830+0000","last_peered":"2024-07-31T14:47:20.451830+0000","last_clean":"2024-07-31T14:47:20.451830+0000","last_became_active":"2024-07-31T14:47:03.052389+0000","last_became_peered":"2024-07-31T14:47:03.052389+0000","last_unstale":"2024-07-31T14:47:20.451830+0000","last_undegraded":"2024-07-31T14:47:20.451830+0000","last_fullsized":"2024-07-31T14:47:20.451830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:16.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'31","reported_seq":123,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:24.638786+0000","last_change":"2024-07-31T14:47:00.030165+0000","last_active":"2024-07-31T14:48:24.638786+0000","last_peered":"2024-07-31T14:48:24.638786+0000","last_clean":"2024-07-31T14:48:24.638786+0000","last_became_active":"2024-07-31T14:47:00.029559+0000","last_became_peered":"2024-07-31T14:47:00.029559+0000","last_unstale":"2024-07-31T14:48:24.638786+0000","last_undegraded":"2024-07-31T14:48:24.638786+0000","last_fullsized":"2024-07-31T14:48:24.638786+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:31:06.115075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56745,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":80,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444173+0000","last_change":"2024-07-31T14:47:13.098517+0000","last_active":"2024-07-31T14:47:20.444173+0000","last_peered":"2024-07-31T14:47:20.444173+0000","last_clean":"2024-07-31T14:47:20.444173+0000","last_became_active":"2024-07-31T14:47:13.098066+0000","last_became_peered":"2024-07-31T14:47:13.098066+0000","last_unstale":"2024-07-31T14:47:20.444173+0000","last_undegraded":"2024-07-31T14:47:20.444173+0000","last_fullsized":"2024-07-31T14:47:20.444173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:08:54.958646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626197+0000","last_change":"2024-07-31T14:47:03.054045+0000","last_active":"2024-07-31T14:47:20.626197+0000","last_peered":"2024-07-31T14:47:20.626197+0000","last_clean":"2024-07-31T14:47:20.626197+0000","last_became_active":"2024-07-31T14:47:03.053658+0000","last_became_peered":"2024-07-31T14:47:03.053658+0000","last_unstale":"2024-07-31T14:47:20.626197+0000","last_undegraded":"2024-07-31T14:47:20.626197+0000","last_fullsized":"2024-07-31T14:47:20.626197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:36:54.563542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.589079+0000","last_change":"2024-07-31T14:47:00.185602+0000","last_active":"2024-07-31T14:48:01.589079+0000","last_peered":"2024-07-31T14:48:01.589079+0000","last_clean":"2024-07-31T14:48:01.589079+0000","last_became_active":"2024-07-31T14:47:00.185230+0000","last_became_peered":"2024-07-31T14:47:00.185230+0000","last_unstale":"2024-07-31T14:48:01.589079+0000","last_undegraded":"2024-07-31T14:48:01.589079+0000","last_fullsized":"2024-07-31T14:48:01.589079+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:21:35.979942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604618+0000","last_change":"2024-07-31T14:47:13.099611+0000","last_active":"2024-07-31T14:47:20.604618+0000","last_peered":"2024-07-31T14:47:20.604618+0000","last_clean":"2024-07-31T14:47:20.604618+0000","last_became_active":"2024-07-31T14:47:13.099127+0000","last_became_peered":"2024-07-31T14:47:13.099127+0000","last_unstale":"2024-07-31T14:47:20.604618+0000","last_undegraded":"2024-07-31T14:47:20.604618+0000","last_fullsized":"2024-07-31T14:47:20.604618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:16:57.192980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449734+0000","last_change":"2024-07-31T14:47:03.056818+0000","last_active":"2024-07-31T14:47:20.449734+0000","last_peered":"2024-07-31T14:47:20.449734+0000","last_clean":"2024-07-31T14:47:20.449734+0000","last_became_active":"2024-07-31T14:47:03.055716+0000","last_became_peered":"2024-07-31T14:47:03.055716+0000","last_unstale":"2024-07-31T14:47:20.449734+0000","last_undegraded":"2024-07-31T14:47:20.449734+0000","last_fullsized":"2024-07-31T14:47:20.449734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:26:33.627362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600183+0000","last_change":"2024-07-31T14:47:00.029651+0000","last_active":"2024-07-31T14:48:02.600183+0000","last_peered":"2024-07-31T14:48:02.600183+0000","last_clean":"2024-07-31T14:48:02.600183+0000","last_became_active":"2024-07-31T14:47:00.029242+0000","last_became_peered":"2024-07-31T14:47:00.029242+0000","last_unstale":"2024-07-31T14:48:02.600183+0000","last_undegraded":"2024-07-31T14:48:02.600183+0000","last_fullsized":"2024-07-31T14:48:02.600183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:23:49.177508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.443789+0000","last_change":"2024-07-31T14:47:13.098862+0000","last_active":"2024-07-31T14:47:20.443789+0000","last_peered":"2024-07-31T14:47:20.443789+0000","last_clean":"2024-07-31T14:47:20.443789+0000","last_became_active":"2024-07-31T14:47:13.098324+0000","last_became_peered":"2024-07-31T14:47:13.098324+0000","last_unstale":"2024-07-31T14:47:20.443789+0000","last_undegraded":"2024-07-31T14:47:20.443789+0000","last_fullsized":"2024-07-31T14:47:20.443789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:50:41.260863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626600+0000","last_change":"2024-07-31T14:47:03.049042+0000","last_active":"2024-07-31T14:47:20.626600+0000","last_peered":"2024-07-31T14:47:20.626600+0000","last_clean":"2024-07-31T14:47:20.626600+0000","last_became_active":"2024-07-31T14:47:03.048535+0000","last_became_peered":"2024-07-31T14:47:03.048535+0000","last_unstale":"2024-07-31T14:47:20.626600+0000","last_undegraded":"2024-07-31T14:47:20.626600+0000","last_fullsized":"2024-07-31T14:47:20.626600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:15.147454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:24.638556+0000","last_change":"2024-07-31T14:47:00.028940+0000","last_active":"2024-07-31T14:48:24.638556+0000","last_peered":"2024-07-31T14:48:24.638556+0000","last_clean":"2024-07-31T14:48:24.638556+0000","last_became_active":"2024-07-31T14:47:00.028472+0000","last_became_peered":"2024-07-31T14:47:00.028472+0000","last_unstale":"2024-07-31T14:48:24.638556+0000","last_undegraded":"2024-07-31T14:48:24.638556+0000","last_fullsized":"2024-07-31T14:48:24.638556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:57:15.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626333+0000","last_change":"2024-07-31T14:47:13.101856+0000","last_active":"2024-07-31T14:47:20.626333+0000","last_peered":"2024-07-31T14:47:20.626333+0000","last_clean":"2024-07-31T14:47:20.626333+0000","last_became_active":"2024-07-31T14:47:13.101346+0000","last_became_peered":"2024-07-31T14:47:13.101346+0000","last_unstale":"2024-07-31T14:47:20.626333+0000","last_undegraded":"2024-07-31T14:47:20.626333+0000","last_fullsized":"2024-07-31T14:47:20.626333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:56:37.071660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449066+0000","last_change":"2024-07-31T14:47:13.105952+0000","last_active":"2024-07-31T14:47:20.449066+0000","last_peered":"2024-07-31T14:47:20.449066+0000","last_clean":"2024-07-31T14:47:20.449066+0000","last_became_active":"2024-07-31T14:47:13.105506+0000","last_became_peered":"2024-07-31T14:47:13.105506+0000","last_unstale":"2024-07-31T14:47:20.449066+0000","last_undegraded":"2024-07-31T14:47:20.449066+0000","last_fullsized":"2024-07-31T14:47:20.449066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:28:33.866216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444136+0000","last_change":"2024-07-31T14:47:03.051152+0000","last_active":"2024-07-31T14:47:20.444136+0000","last_peered":"2024-07-31T14:47:20.444136+0000","last_clean":"2024-07-31T14:47:20.444136+0000","last_became_active":"2024-07-31T14:47:03.050775+0000","last_became_peered":"2024-07-31T14:47:03.050775+0000","last_unstale":"2024-07-31T14:47:20.444136+0000","last_undegraded":"2024-07-31T14:47:20.444136+0000","last_fullsized":"2024-07-31T14:47:20.444136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:47:35.913641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600742+0000","last_change":"2024-07-31T14:47:00.028552+0000","last_active":"2024-07-31T14:48:02.600742+0000","last_peered":"2024-07-31T14:48:02.600742+0000","last_clean":"2024-07-31T14:48:02.600742+0000","last_became_active":"2024-07-31T14:47:00.028143+0000","last_became_peered":"2024-07-31T14:47:00.028143+0000","last_unstale":"2024-07-31T14:48:02.600742+0000","last_undegraded":"2024-07-31T14:48:02.600742+0000","last_fullsized":"2024-07-31T14:48:02.600742+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:09.601635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.606387+0000","last_change":"2024-07-31T14:47:20.606285+0000","last_active":"2024-07-31T14:47:20.606387+0000","last_peered":"2024-07-31T14:47:20.606387+0000","last_clean":"2024-07-31T14:47:20.606387+0000","last_became_active":"2024-07-31T14:47:20.605837+0000","last_became_peered":"2024-07-31T14:47:20.605837+0000","last_unstale":"2024-07-31T14:47:20.606387+0000","last_undegraded":"2024-07-31T14:47:20.606387+0000","last_fullsized":"2024-07-31T14:47:20.606387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:16:35.409790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625946+0000","last_change":"2024-07-31T14:47:03.047442+0000","last_active":"2024-07-31T14:47:20.625946+0000","last_peered":"2024-07-31T14:47:20.625946+0000","last_clean":"2024-07-31T14:47:20.625946+0000","last_became_active":"2024-07-31T14:47:03.047072+0000","last_became_peered":"2024-07-31T14:47:03.047072+0000","last_unstale":"2024-07-31T14:47:20.625946+0000","last_undegraded":"2024-07-31T14:47:20.625946+0000","last_fullsized":"2024-07-31T14:47:20.625946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:56.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:00.572013+0000","last_change":"2024-07-31T14:47:00.027746+0000","last_active":"2024-07-31T14:48:00.572013+0000","last_peered":"2024-07-31T14:48:00.572013+0000","last_clean":"2024-07-31T14:48:00.572013+0000","last_became_active":"2024-07-31T14:47:00.027404+0000","last_became_peered":"2024-07-31T14:47:00.027404+0000","last_unstale":"2024-07-31T14:48:00.572013+0000","last_undegraded":"2024-07-31T14:48:00.572013+0000","last_fullsized":"2024-07-31T14:48:00.572013+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:30:21.011674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.589312+0000","last_change":"2024-07-31T14:47:00.029148+0000","last_active":"2024-07-31T14:48:01.589312+0000","last_peered":"2024-07-31T14:48:01.589312+0000","last_clean":"2024-07-31T14:48:01.589312+0000","last_became_active":"2024-07-31T14:47:00.028718+0000","last_became_peered":"2024-07-31T14:47:00.028718+0000","last_unstale":"2024-07-31T14:48:01.589312+0000","last_undegraded":"2024-07-31T14:48:01.589312+0000","last_fullsized":"2024-07-31T14:48:01.589312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:25:16.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630241+0000","last_change":"2024-07-31T14:47:20.629098+0000","last_active":"2024-07-31T14:47:20.630241+0000","last_peered":"2024-07-31T14:47:20.630241+0000","last_clean":"2024-07-31T14:47:20.630241+0000","last_became_active":"2024-07-31T14:47:20.627775+0000","last_became_peered":"2024-07-31T14:47:20.627775+0000","last_unstale":"2024-07-31T14:47:20.630241+0000","last_undegraded":"2024-07-31T14:47:20.630241+0000","last_fullsized":"2024-07-31T14:47:20.630241+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:22:03.606509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447260+0000","last_change":"2024-07-31T14:47:13.097892+0000","last_active":"2024-07-31T14:47:20.447260+0000","last_peered":"2024-07-31T14:47:20.447260+0000","last_clean":"2024-07-31T14:47:20.447260+0000","last_became_active":"2024-07-31T14:47:13.097419+0000","last_became_peered":"2024-07-31T14:47:13.097419+0000","last_unstale":"2024-07-31T14:47:20.447260+0000","last_undegraded":"2024-07-31T14:47:20.447260+0000","last_fullsized":"2024-07-31T14:47:20.447260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:07:13.456147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:26.642498+0000","last_change":"2024-07-31T14:47:00.019792+0000","last_active":"2024-07-31T14:48:26.642498+0000","last_peered":"2024-07-31T14:48:26.642498+0000","last_clean":"2024-07-31T14:48:26.642498+0000","last_became_active":"2024-07-31T14:47:00.019411+0000","last_became_peered":"2024-07-31T14:47:00.019411+0000","last_unstale":"2024-07-31T14:48:26.642498+0000","last_undegraded":"2024-07-31T14:48:26.642498+0000","last_fullsized":"2024-07-31T14:48:26.642498+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:55:02.148823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.450923+0000","last_change":"2024-07-31T14:47:03.039808+0000","last_active":"2024-07-31T14:47:20.450923+0000","last_peered":"2024-07-31T14:47:20.450923+0000","last_clean":"2024-07-31T14:47:20.450923+0000","last_became_active":"2024-07-31T14:47:03.039382+0000","last_became_peered":"2024-07-31T14:47:03.039382+0000","last_unstale":"2024-07-31T14:47:20.450923+0000","last_undegraded":"2024-07-31T14:47:20.450923+0000","last_fullsized":"2024-07-31T14:47:20.450923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:23:09.751293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.446875+0000","last_change":"2024-07-31T14:47:13.102797+0000","last_active":"2024-07-31T14:47:20.446875+0000","last_peered":"2024-07-31T14:47:20.446875+0000","last_clean":"2024-07-31T14:47:20.446875+0000","last_became_active":"2024-07-31T14:47:13.102320+0000","last_became_peered":"2024-07-31T14:47:13.102320+0000","last_unstale":"2024-07-31T14:47:20.446875+0000","last_undegraded":"2024-07-31T14:47:20.446875+0000","last_fullsized":"2024-07-31T14:47:20.446875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:23.066357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444689+0000","last_change":"2024-07-31T14:47:03.054875+0000","last_active":"2024-07-31T14:47:20.444689+0000","last_peered":"2024-07-31T14:47:20.444689+0000","last_clean":"2024-07-31T14:47:20.444689+0000","last_became_active":"2024-07-31T14:47:03.054529+0000","last_became_peered":"2024-07-31T14:47:03.054529+0000","last_unstale":"2024-07-31T14:47:20.444689+0000","last_undegraded":"2024-07-31T14:47:20.444689+0000","last_fullsized":"2024-07-31T14:47:20.444689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:32:09.802933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.448452+0000","last_change":"2024-07-31T14:47:03.043283+0000","last_active":"2024-07-31T14:47:20.448452+0000","last_peered":"2024-07-31T14:47:20.448452+0000","last_clean":"2024-07-31T14:47:20.448452+0000","last_became_active":"2024-07-31T14:47:03.042851+0000","last_became_peered":"2024-07-31T14:47:03.042851+0000","last_unstale":"2024-07-31T14:47:20.448452+0000","last_undegraded":"2024-07-31T14:47:20.448452+0000","last_fullsized":"2024-07-31T14:47:20.448452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:25:37.408149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447315+0000","last_change":"2024-07-31T14:47:03.047890+0000","last_active":"2024-07-31T14:47:20.447315+0000","last_peered":"2024-07-31T14:47:20.447315+0000","last_clean":"2024-07-31T14:47:20.447315+0000","last_became_active":"2024-07-31T14:47:03.047507+0000","last_became_peered":"2024-07-31T14:47:03.047507+0000","last_unstale":"2024-07-31T14:47:20.447315+0000","last_undegraded":"2024-07-31T14:47:20.447315+0000","last_fullsized":"2024-07-31T14:47:20.447315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:52:45.991323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449437+0000","last_change":"2024-07-31T14:47:03.047903+0000","last_active":"2024-07-31T14:47:20.449437+0000","last_peered":"2024-07-31T14:47:20.449437+0000","last_clean":"2024-07-31T14:47:20.449437+0000","last_became_active":"2024-07-31T14:47:03.047352+0000","last_became_peered":"2024-07-31T14:47:03.047352+0000","last_unstale":"2024-07-31T14:47:20.449437+0000","last_undegraded":"2024-07-31T14:47:20.449437+0000","last_fullsized":"2024-07-31T14:47:20.449437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:06.536023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447343+0000","last_change":"2024-07-31T14:47:03.044199+0000","last_active":"2024-07-31T14:47:20.447343+0000","last_peered":"2024-07-31T14:47:20.447343+0000","last_clean":"2024-07-31T14:47:20.447343+0000","last_became_active":"2024-07-31T14:47:03.043849+0000","last_became_peered":"2024-07-31T14:47:03.043849+0000","last_unstale":"2024-07-31T14:47:20.447343+0000","last_undegraded":"2024-07-31T14:47:20.447343+0000","last_fullsized":"2024-07-31T14:47:20.447343+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:50.474895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445907+0000","last_change":"2024-07-31T14:47:03.060654+0000","last_active":"2024-07-31T14:47:20.445907+0000","last_peered":"2024-07-31T14:47:20.445907+0000","last_clean":"2024-07-31T14:47:20.445907+0000","last_became_active":"2024-07-31T14:47:03.060157+0000","last_became_peered":"2024-07-31T14:47:03.060157+0000","last_unstale":"2024-07-31T14:47:20.445907+0000","last_undegraded":"2024-07-31T14:47:20.445907+0000","last_fullsized":"2024-07-31T14:47:20.445907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:39:50.130831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447018+0000","last_change":"2024-07-31T14:47:03.049910+0000","last_active":"2024-07-31T14:47:20.447018+0000","last_peered":"2024-07-31T14:47:20.447018+0000","last_clean":"2024-07-31T14:47:20.447018+0000","last_became_active":"2024-07-31T14:47:03.049491+0000","last_became_peered":"2024-07-31T14:47:03.049491+0000","last_unstale":"2024-07-31T14:47:20.447018+0000","last_undegraded":"2024-07-31T14:47:20.447018+0000","last_fullsized":"2024-07-31T14:47:20.447018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:18:43.319695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627262+0000","last_change":"2024-07-31T14:47:03.052206+0000","last_active":"2024-07-31T14:47:20.627262+0000","last_peered":"2024-07-31T14:47:20.627262+0000","last_clean":"2024-07-31T14:47:20.627262+0000","last_became_active":"2024-07-31T14:47:03.051175+0000","last_became_peered":"2024-07-31T14:47:03.051175+0000","last_unstale":"2024-07-31T14:47:20.627262+0000","last_undegraded":"2024-07-31T14:47:20.627262+0000","last_fullsized":"2024-07-31T14:47:20.627262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:51:11.583604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626447+0000","last_change":"2024-07-31T14:47:03.052162+0000","last_active":"2024-07-31T14:47:20.626447+0000","last_peered":"2024-07-31T14:47:20.626447+0000","last_clean":"2024-07-31T14:47:20.626447+0000","last_became_active":"2024-07-31T14:47:03.051634+0000","last_became_peered":"2024-07-31T14:47:03.051634+0000","last_unstale":"2024-07-31T14:47:20.626447+0000","last_undegraded":"2024-07-31T14:47:20.626447+0000","last_fullsized":"2024-07-31T14:47:20.626447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:41:59.605192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626439+0000","last_change":"2024-07-31T14:47:03.047475+0000","last_active":"2024-07-31T14:47:20.626439+0000","last_peered":"2024-07-31T14:47:20.626439+0000","last_clean":"2024-07-31T14:47:20.626439+0000","last_became_active":"2024-07-31T14:47:03.047100+0000","last_became_peered":"2024-07-31T14:47:03.047100+0000","last_unstale":"2024-07-31T14:47:20.626439+0000","last_undegraded":"2024-07-31T14:47:20.626439+0000","last_fullsized":"2024-07-31T14:47:20.626439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:08:10.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626240+0000","last_change":"2024-07-31T14:47:03.051281+0000","last_active":"2024-07-31T14:47:20.626240+0000","last_peered":"2024-07-31T14:47:20.626240+0000","last_clean":"2024-07-31T14:47:20.626240+0000","last_became_active":"2024-07-31T14:47:03.050921+0000","last_became_peered":"2024-07-31T14:47:03.050921+0000","last_unstale":"2024-07-31T14:47:20.626240+0000","last_undegraded":"2024-07-31T14:47:20.626240+0000","last_fullsized":"2024-07-31T14:47:20.626240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:24:29.221723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447393+0000","last_change":"2024-07-31T14:47:03.045457+0000","last_active":"2024-07-31T14:47:20.447393+0000","last_peered":"2024-07-31T14:47:20.447393+0000","last_clean":"2024-07-31T14:47:20.447393+0000","last_became_active":"2024-07-31T14:47:03.044466+0000","last_became_peered":"2024-07-31T14:47:03.044466+0000","last_unstale":"2024-07-31T14:47:20.447393+0000","last_undegraded":"2024-07-31T14:47:20.447393+0000","last_fullsized":"2024-07-31T14:47:20.447393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:44:01.197416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447013+0000","last_change":"2024-07-31T14:47:03.053431+0000","last_active":"2024-07-31T14:47:20.447013+0000","last_peered":"2024-07-31T14:47:20.447013+0000","last_clean":"2024-07-31T14:47:20.447013+0000","last_became_active":"2024-07-31T14:47:03.052958+0000","last_became_peered":"2024-07-31T14:47:03.052958+0000","last_unstale":"2024-07-31T14:47:20.447013+0000","last_undegraded":"2024-07-31T14:47:20.447013+0000","last_fullsized":"2024-07-31T14:47:20.447013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:03:28.863589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447102+0000","last_change":"2024-07-31T14:47:03.046641+0000","last_active":"2024-07-31T14:47:20.447102+0000","last_peered":"2024-07-31T14:47:20.447102+0000","last_clean":"2024-07-31T14:47:20.447102+0000","last_became_active":"2024-07-31T14:47:03.046265+0000","last_became_peered":"2024-07-31T14:47:03.046265+0000","last_unstale":"2024-07-31T14:47:20.447102+0000","last_undegraded":"2024-07-31T14:47:20.447102+0000","last_fullsized":"2024-07-31T14:47:20.447102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:20:25.389010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444399+0000","last_change":"2024-07-31T14:47:03.051657+0000","last_active":"2024-07-31T14:47:20.444399+0000","last_peered":"2024-07-31T14:47:20.444399+0000","last_clean":"2024-07-31T14:47:20.444399+0000","last_became_active":"2024-07-31T14:47:03.050722+0000","last_became_peered":"2024-07-31T14:47:03.050722+0000","last_unstale":"2024-07-31T14:47:20.444399+0000","last_undegraded":"2024-07-31T14:47:20.444399+0000","last_fullsized":"2024-07-31T14:47:20.444399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:23:19.387470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447245+0000","last_change":"2024-07-31T14:47:03.059654+0000","last_active":"2024-07-31T14:47:20.447245+0000","last_peered":"2024-07-31T14:47:20.447245+0000","last_clean":"2024-07-31T14:47:20.447245+0000","last_became_active":"2024-07-31T14:47:03.056843+0000","last_became_peered":"2024-07-31T14:47:03.056843+0000","last_unstale":"2024-07-31T14:47:20.447245+0000","last_undegraded":"2024-07-31T14:47:20.447245+0000","last_fullsized":"2024-07-31T14:47:20.447245+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:42:33.696498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625741+0000","last_change":"2024-07-31T14:47:03.052750+0000","last_active":"2024-07-31T14:47:20.625741+0000","last_peered":"2024-07-31T14:47:20.625741+0000","last_clean":"2024-07-31T14:47:20.625741+0000","last_became_active":"2024-07-31T14:47:03.051744+0000","last_became_peered":"2024-07-31T14:47:03.051744+0000","last_unstale":"2024-07-31T14:47:20.625741+0000","last_undegraded":"2024-07-31T14:47:20.625741+0000","last_fullsized":"2024-07-31T14:47:20.625741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:24:33.202918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.452621+0000","last_change":"2024-07-31T14:47:20.452563+0000","last_active":"2024-07-31T14:47:20.452621+0000","last_peered":"2024-07-31T14:47:20.452621+0000","last_clean":"2024-07-31T14:47:20.452621+0000","last_became_active":"2024-07-31T14:47:20.452271+0000","last_became_peered":"2024-07-31T14:47:20.452271+0000","last_unstale":"2024-07-31T14:47:20.452621+0000","last_undegraded":"2024-07-31T14:47:20.452621+0000","last_fullsized":"2024-07-31T14:47:20.452621+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:26:55.070265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.448941+0000","last_change":"2024-07-31T14:47:03.050691+0000","last_active":"2024-07-31T14:47:20.448941+0000","last_peered":"2024-07-31T14:47:20.448941+0000","last_clean":"2024-07-31T14:47:20.448941+0000","last_became_active":"2024-07-31T14:47:03.050320+0000","last_became_peered":"2024-07-31T14:47:03.050320+0000","last_unstale":"2024-07-31T14:47:20.448941+0000","last_undegraded":"2024-07-31T14:47:20.448941+0000","last_fullsized":"2024-07-31T14:47:20.448941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:40:45.767706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447344+0000","last_change":"2024-07-31T14:47:03.048491+0000","last_active":"2024-07-31T14:47:20.447344+0000","last_peered":"2024-07-31T14:47:20.447344+0000","last_clean":"2024-07-31T14:47:20.447344+0000","last_became_active":"2024-07-31T14:47:03.048085+0000","last_became_peered":"2024-07-31T14:47:03.048085+0000","last_unstale":"2024-07-31T14:47:20.447344+0000","last_undegraded":"2024-07-31T14:47:20.447344+0000","last_fullsized":"2024-07-31T14:47:20.447344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:35:01.762365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604826+0000","last_change":"2024-07-31T14:47:03.055502+0000","last_active":"2024-07-31T14:47:20.604826+0000","last_peered":"2024-07-31T14:47:20.604826+0000","last_clean":"2024-07-31T14:47:20.604826+0000","last_became_active":"2024-07-31T14:47:03.055015+0000","last_became_peered":"2024-07-31T14:47:03.055015+0000","last_unstale":"2024-07-31T14:47:20.604826+0000","last_undegraded":"2024-07-31T14:47:20.604826+0000","last_fullsized":"2024-07-31T14:47:20.604826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:50:39.605148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444781+0000","last_change":"2024-07-31T14:47:03.057808+0000","last_active":"2024-07-31T14:47:20.444781+0000","last_peered":"2024-07-31T14:47:20.444781+0000","last_clean":"2024-07-31T14:47:20.444781+0000","last_became_active":"2024-07-31T14:47:03.057443+0000","last_became_peered":"2024-07-31T14:47:03.057443+0000","last_unstale":"2024-07-31T14:47:20.444781+0000","last_undegraded":"2024-07-31T14:47:20.444781+0000","last_fullsized":"2024-07-31T14:47:20.444781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:34:42.915428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.628024+0000","last_change":"2024-07-31T14:47:03.058182+0000","last_active":"2024-07-31T14:47:20.628024+0000","last_peered":"2024-07-31T14:47:20.628024+0000","last_clean":"2024-07-31T14:47:20.628024+0000","last_became_active":"2024-07-31T14:47:03.057802+0000","last_became_peered":"2024-07-31T14:47:03.057802+0000","last_unstale":"2024-07-31T14:47:20.628024+0000","last_undegraded":"2024-07-31T14:47:20.628024+0000","last_fullsized":"2024-07-31T14:47:20.628024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:08:06.608788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626405+0000","last_change":"2024-07-31T14:47:03.058882+0000","last_active":"2024-07-31T14:47:20.626405+0000","last_peered":"2024-07-31T14:47:20.626405+0000","last_clean":"2024-07-31T14:47:20.626405+0000","last_became_active":"2024-07-31T14:47:03.058530+0000","last_became_peered":"2024-07-31T14:47:03.058530+0000","last_unstale":"2024-07-31T14:47:20.626405+0000","last_undegraded":"2024-07-31T14:47:20.626405+0000","last_fullsized":"2024-07-31T14:47:20.626405+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:28:03.631962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625927+0000","last_change":"2024-07-31T14:47:03.052028+0000","last_active":"2024-07-31T14:47:20.625927+0000","last_peered":"2024-07-31T14:47:20.625927+0000","last_clean":"2024-07-31T14:47:20.625927+0000","last_became_active":"2024-07-31T14:47:03.051559+0000","last_became_peered":"2024-07-31T14:47:03.051559+0000","last_unstale":"2024-07-31T14:47:20.625927+0000","last_undegraded":"2024-07-31T14:47:20.625927+0000","last_fullsized":"2024-07-31T14:47:20.625927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:16:40.723795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444190+0000","last_change":"2024-07-31T14:47:03.053884+0000","last_active":"2024-07-31T14:47:20.444190+0000","last_peered":"2024-07-31T14:47:20.444190+0000","last_clean":"2024-07-31T14:47:20.444190+0000","last_became_active":"2024-07-31T14:47:03.053521+0000","last_became_peered":"2024-07-31T14:47:03.053521+0000","last_unstale":"2024-07-31T14:47:20.444190+0000","last_undegraded":"2024-07-31T14:47:20.444190+0000","last_fullsized":"2024-07-31T14:47:20.444190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:16:37.702055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627119+0000","last_change":"2024-07-31T14:47:03.058588+0000","last_active":"2024-07-31T14:47:20.627119+0000","last_peered":"2024-07-31T14:47:20.627119+0000","last_clean":"2024-07-31T14:47:20.627119+0000","last_became_active":"2024-07-31T14:47:03.057820+0000","last_became_peered":"2024-07-31T14:47:03.057820+0000","last_unstale":"2024-07-31T14:47:20.627119+0000","last_undegraded":"2024-07-31T14:47:20.627119+0000","last_fullsized":"2024-07-31T14:47:20.627119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:21.265208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451188+0000","last_change":"2024-07-31T14:47:03.041447+0000","last_active":"2024-07-31T14:47:20.451188+0000","last_peered":"2024-07-31T14:47:20.451188+0000","last_clean":"2024-07-31T14:47:20.451188+0000","last_became_active":"2024-07-31T14:47:03.040935+0000","last_became_peered":"2024-07-31T14:47:03.040935+0000","last_unstale":"2024-07-31T14:47:20.451188+0000","last_undegraded":"2024-07-31T14:47:20.451188+0000","last_fullsized":"2024-07-31T14:47:20.451188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:00:13.624698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.452719+0000","last_change":"2024-07-31T14:47:20.452239+0000","last_active":"2024-07-31T14:47:20.452719+0000","last_peered":"2024-07-31T14:47:20.452719+0000","last_clean":"2024-07-31T14:47:20.452719+0000","last_became_active":"2024-07-31T14:47:20.451924+0000","last_became_peered":"2024-07-31T14:47:20.451924+0000","last_unstale":"2024-07-31T14:47:20.452719+0000","last_undegraded":"2024-07-31T14:47:20.452719+0000","last_fullsized":"2024-07-31T14:47:20.452719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:58:24.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449119+0000","last_change":"2024-07-31T14:47:03.056024+0000","last_active":"2024-07-31T14:47:20.449119+0000","last_peered":"2024-07-31T14:47:20.449119+0000","last_clean":"2024-07-31T14:47:20.449119+0000","last_became_active":"2024-07-31T14:47:03.055630+0000","last_became_peered":"2024-07-31T14:47:03.055630+0000","last_unstale":"2024-07-31T14:47:20.449119+0000","last_undegraded":"2024-07-31T14:47:20.449119+0000","last_fullsized":"2024-07-31T14:47:20.449119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:09:11.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449066+0000","last_change":"2024-07-31T14:47:03.053718+0000","last_active":"2024-07-31T14:47:20.449066+0000","last_peered":"2024-07-31T14:47:20.449066+0000","last_clean":"2024-07-31T14:47:20.449066+0000","last_became_active":"2024-07-31T14:47:03.053361+0000","last_became_peered":"2024-07-31T14:47:03.053361+0000","last_unstale":"2024-07-31T14:47:20.449066+0000","last_undegraded":"2024-07-31T14:47:20.449066+0000","last_fullsized":"2024-07-31T14:47:20.449066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:03:49.220838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447132+0000","last_change":"2024-07-31T14:47:03.056087+0000","last_active":"2024-07-31T14:47:20.447132+0000","last_peered":"2024-07-31T14:47:20.447132+0000","last_clean":"2024-07-31T14:47:20.447132+0000","last_became_active":"2024-07-31T14:47:03.055732+0000","last_became_peered":"2024-07-31T14:47:03.055732+0000","last_unstale":"2024-07-31T14:47:20.447132+0000","last_undegraded":"2024-07-31T14:47:20.447132+0000","last_fullsized":"2024-07-31T14:47:20.447132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:45:01.185666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64340,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":273,"num_read_kb":205,"num_write":243,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":195296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645343,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":519730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841571,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37008,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708144,"statfs":{"total":95995035648,"available":95957139456,"internally_reserved":0,"allocated":1712128,"data_stored":1163682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005097,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":523756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201326,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":520038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397554,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1117317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561080,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":515555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1112117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":519629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117062,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":514872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378699,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":574688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542224,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":574468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":516620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-31T14:48:30.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph pg dump --format=json 2024-07-31T14:48:30.454 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:30.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:30 smithi067 ceph-mon[32750]: pgmap v322: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-07-31T14:48:30.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:30 smithi067 ceph-mon[32750]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:48:31.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:30 smithi073 ceph-mon[36714]: pgmap v322: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-07-31T14:48:31.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:30 smithi073 ceph-mon[36714]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:48:31.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:30 smithi077 ceph-mon[37327]: pgmap v322: 129 pgs: 129 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-07-31T14:48:31.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:30 smithi077 ceph-mon[37327]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:48:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.123+0000 7f41dfce0640 1 -- 172.21.15.67:0/1570842442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d8107dd0 msgr2=0x7f41d81081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.123+0000 7f41dfce0640 1 --2- 172.21.15.67:0/1570842442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d8107dd0 0x7f41d81081e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009900 tx=0x7f41cc02a410 comp rx=0 tx=0).stop 2024-07-31T14:48:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.123+0000 7f41dfce0640 1 -- 172.21.15.67:0/1570842442 shutdown_connections 2024-07-31T14:48:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.123+0000 7f41dfce0640 1 --2- 172.21.15.67:0/1570842442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41d8108720 0x7f41d810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.123+0000 7f41dfce0640 1 --2- 172.21.15.67:0/1570842442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d8107dd0 0x7f41d81081e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.123+0000 7f41dfce0640 1 --2- 172.21.15.67:0/1570842442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8102490 0x7f41d81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.123+0000 7f41dfce0640 1 -- 172.21.15.67:0/1570842442 >> 172.21.15.67:0/1570842442 conn(0x7f41d8079160 msgr2=0x7f41d8079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:32.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.123+0000 7f41dfce0640 1 -- 172.21.15.67:0/1570842442 shutdown_connections 2024-07-31T14:48:32.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.123+0000 7f41dfce0640 1 -- 172.21.15.67:0/1570842442 wait complete. 2024-07-31T14:48:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.124+0000 7f41dfce0640 1 Processor -- start 2024-07-31T14:48:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.124+0000 7f41dfce0640 1 -- start start 2024-07-31T14:48:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.124+0000 7f41dfce0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8102490 0x7f41d810f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41dfce0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41d8107dd0 0x7f41d810ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:32.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41dfce0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d8108720 0x7f41d8110460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:32.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41dfce0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81134b0 con 0x7f41d8108720 2024-07-31T14:48:32.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41dfce0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f41d8113330 con 0x7f41d8107dd0 2024-07-31T14:48:32.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41dfce0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41d8113630 con 0x7f41d8102490 2024-07-31T14:48:32.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41de27d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d8108720 0x7f41d8110460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:32.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41de27d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d8108720 0x7f41d8110460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46076/0 (socket says 172.21.15.67:46076) 2024-07-31T14:48:32.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41de27d640 1 -- 172.21.15.67:0/4185423239 learned_addr learned my addr 172.21.15.67:0/4185423239 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:32.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41dd27b640 1 --2- 172.21.15.67:0/4185423239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41d8107dd0 0x7f41d810ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:32.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41dda7c640 1 --2- 172.21.15.67:0/4185423239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8102490 0x7f41d810f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41de27d640 1 -- 172.21.15.67:0/4185423239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8102490 msgr2=0x7f41d810f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41de27d640 1 --2- 172.21.15.67:0/4185423239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8102490 0x7f41d810f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41de27d640 1 -- 172.21.15.67:0/4185423239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41d8107dd0 msgr2=0x7f41d810ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41de27d640 1 --2- 172.21.15.67:0/4185423239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41d8107dd0 0x7f41d810ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41de27d640 1 -- 172.21.15.67:0/4185423239 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d81a8190 con 0x7f41d8108720 2024-07-31T14:48:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41de27d640 1 --2- 172.21.15.67:0/4185423239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d8108720 0x7f41d8110460 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f41c800cce0 tx=0x7f41c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41d6ffd640 1 -- 172.21.15.67:0/4185423239 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c800f450 con 0x7f41d8108720 2024-07-31T14:48:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.125+0000 7f41d6ffd640 1 -- 172.21.15.67:0/4185423239 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41c800faa0 con 0x7f41d8108720 2024-07-31T14:48:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.126+0000 7f41dfce0640 1 -- 172.21.15.67:0/4185423239 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d81a8480 con 0x7f41d8108720 2024-07-31T14:48:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.126+0000 7f41d6ffd640 1 -- 172.21.15.67:0/4185423239 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8018b70 con 0x7f41d8108720 2024-07-31T14:48:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.126+0000 7f41dfce0640 1 -- 172.21.15.67:0/4185423239 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d81a8940 con 0x7f41d8108720 2024-07-31T14:48:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.127+0000 7f41dfce0640 1 -- 172.21.15.67:0/4185423239 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d8103710 con 0x7f41d8108720 2024-07-31T14:48:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.128+0000 7f41d6ffd640 1 -- 172.21.15.67:0/4185423239 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f41c8018d10 con 0x7f41d8108720 2024-07-31T14:48:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.129+0000 7f41d6ffd640 1 --2- 172.21.15.67:0/4185423239 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41c00b3cb0 0x7f41c00b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.129+0000 7f41dda7c640 1 --2- 172.21.15.67:0/4185423239 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41c00b3cb0 0x7f41c00b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.129+0000 7f41d6ffd640 1 -- 172.21.15.67:0/4185423239 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f41c80e4610 con 0x7f41d8108720 2024-07-31T14:48:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.130+0000 7f41dda7c640 1 --2- 172.21.15.67:0/4185423239 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41c00b3cb0 0x7f41c00b61c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f41d8111430 tx=0x7f41d000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:32.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.132+0000 7f41d6ffd640 1 -- 172.21.15.67:0/4185423239 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f41c8014030 con 0x7f41d8108720 2024-07-31T14:48:32.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.322+0000 7f41dfce0640 1 -- 172.21.15.67:0/4185423239 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41d8063230 con 0x7f41c00b3cb0 2024-07-31T14:48:32.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.328+0000 7f41d6ffd640 1 -- 172.21.15.67:0/4185423239 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340401 (secure 0 0 0) 0x7f41d8063230 con 0x7f41c00b3cb0 2024-07-31T14:48:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:48:32.335 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-31T14:48:32.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.331+0000 7f41dfce0640 1 -- 172.21.15.67:0/4185423239 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41c00b3cb0 msgr2=0x7f41c00b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:32.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.331+0000 7f41dfce0640 1 --2- 172.21.15.67:0/4185423239 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41c00b3cb0 0x7f41c00b61c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f41d8111430 tx=0x7f41d000b390 comp rx=0 tx=0).stop 2024-07-31T14:48:32.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.331+0000 7f41dfce0640 1 -- 172.21.15.67:0/4185423239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d8108720 msgr2=0x7f41d8110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:32.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.331+0000 7f41dfce0640 1 --2- 172.21.15.67:0/4185423239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d8108720 0x7f41d8110460 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f41c800cce0 tx=0x7f41c800a620 comp rx=0 tx=0).stop 2024-07-31T14:48:32.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.332+0000 7f41dfce0640 1 -- 172.21.15.67:0/4185423239 shutdown_connections 2024-07-31T14:48:32.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.332+0000 7f41dfce0640 1 --2- 172.21.15.67:0/4185423239 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41c00b3cb0 0x7f41c00b61c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:32.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.332+0000 7f41dfce0640 1 --2- 172.21.15.67:0/4185423239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41d8108720 0x7f41d8110460 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:32.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.332+0000 7f41dfce0640 1 --2- 172.21.15.67:0/4185423239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41d8107dd0 0x7f41d810ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:32.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.332+0000 7f41dfce0640 1 --2- 172.21.15.67:0/4185423239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8102490 0x7f41d810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:32.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.332+0000 7f41dfce0640 1 -- 172.21.15.67:0/4185423239 >> 172.21.15.67:0/4185423239 conn(0x7f41d8079160 msgr2=0x7f41d8103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:32.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.332+0000 7f41dfce0640 1 -- 172.21.15.67:0/4185423239 shutdown_connections 2024-07-31T14:48:32.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:32.332+0000 7f41dfce0640 1 -- 172.21.15.67:0/4185423239 wait complete. 2024-07-31T14:48:33.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:32 smithi067 ceph-mon[32750]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-07-31T14:48:33.075 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-07-31T14:48:31.394721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656080,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":375,"num_read_kb":291,"num_write":370,"num_write_kb":2289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379912,"kb_used_data":14120,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561912,"statfs":{"total":1151940427776,"available":1151551397888,"internally_reserved":0,"allocated":14458880,"data_stored":8173610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33307,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":99,"num_write":19,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003641"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":77,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:24.637701+0000","last_change":"2024-07-31T14:47:00.020067+0000","last_active":"2024-07-31T14:48:24.637701+0000","last_peered":"2024-07-31T14:48:24.637701+0000","last_clean":"2024-07-31T14:48:24.637701+0000","last_became_active":"2024-07-31T14:47:00.019674+0000","last_became_peered":"2024-07-31T14:47:00.019674+0000","last_unstale":"2024-07-31T14:48:24.637701+0000","last_undegraded":"2024-07-31T14:48:24.637701+0000","last_fullsized":"2024-07-31T14:48:24.637701+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:10:08.193786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626661+0000","last_change":"2024-07-31T14:47:03.052069+0000","last_active":"2024-07-31T14:47:20.626661+0000","last_peered":"2024-07-31T14:47:20.626661+0000","last_clean":"2024-07-31T14:47:20.626661+0000","last_became_active":"2024-07-31T14:47:03.051739+0000","last_became_peered":"2024-07-31T14:47:03.051739+0000","last_unstale":"2024-07-31T14:47:20.626661+0000","last_undegraded":"2024-07-31T14:47:20.626661+0000","last_fullsized":"2024-07-31T14:47:20.626661+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:29:19.952794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.628866+0000","last_change":"2024-07-31T14:47:20.628770+0000","last_active":"2024-07-31T14:47:20.628866+0000","last_peered":"2024-07-31T14:47:20.628866+0000","last_clean":"2024-07-31T14:47:20.628866+0000","last_became_active":"2024-07-31T14:47:20.628142+0000","last_became_peered":"2024-07-31T14:47:20.628142+0000","last_unstale":"2024-07-31T14:47:20.628866+0000","last_undegraded":"2024-07-31T14:47:20.628866+0000","last_fullsized":"2024-07-31T14:47:20.628866+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.659670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627157+0000","last_change":"2024-07-31T14:47:00.183893+0000","last_active":"2024-07-31T14:47:20.627157+0000","last_peered":"2024-07-31T14:47:20.627157+0000","last_clean":"2024-07-31T14:47:20.627157+0000","last_became_active":"2024-07-31T14:47:00.183453+0000","last_became_peered":"2024-07-31T14:47:00.183453+0000","last_unstale":"2024-07-31T14:47:20.627157+0000","last_undegraded":"2024-07-31T14:47:20.627157+0000","last_fullsized":"2024-07-31T14:47:20.627157+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:45:08.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449702+0000","last_change":"2024-07-31T14:47:03.057081+0000","last_active":"2024-07-31T14:47:20.449702+0000","last_peered":"2024-07-31T14:47:20.449702+0000","last_clean":"2024-07-31T14:47:20.449702+0000","last_became_active":"2024-07-31T14:47:03.056061+0000","last_became_peered":"2024-07-31T14:47:03.056061+0000","last_unstale":"2024-07-31T14:47:20.449702+0000","last_undegraded":"2024-07-31T14:47:20.449702+0000","last_fullsized":"2024-07-31T14:47:20.449702+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:28:16.760131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604015+0000","last_change":"2024-07-31T14:47:13.105271+0000","last_active":"2024-07-31T14:47:20.604015+0000","last_peered":"2024-07-31T14:47:20.604015+0000","last_clean":"2024-07-31T14:47:20.604015+0000","last_became_active":"2024-07-31T14:47:13.104717+0000","last_became_peered":"2024-07-31T14:47:13.104717+0000","last_unstale":"2024-07-31T14:47:20.604015+0000","last_undegraded":"2024-07-31T14:47:20.604015+0000","last_fullsized":"2024-07-31T14:47:20.604015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:34:21.413957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605719+0000","last_change":"2024-07-31T14:47:20.454745+0000","last_active":"2024-07-31T14:48:03.605719+0000","last_peered":"2024-07-31T14:48:03.605719+0000","last_clean":"2024-07-31T14:48:03.605719+0000","last_became_active":"2024-07-31T14:47:20.449892+0000","last_became_peered":"2024-07-31T14:47:20.449892+0000","last_unstale":"2024-07-31T14:48:03.605719+0000","last_undegraded":"2024-07-31T14:48:03.605719+0000","last_fullsized":"2024-07-31T14:48:03.605719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:59:17.375344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626206+0000","last_change":"2024-07-31T14:47:03.051085+0000","last_active":"2024-07-31T14:47:20.626206+0000","last_peered":"2024-07-31T14:47:20.626206+0000","last_clean":"2024-07-31T14:47:20.626206+0000","last_became_active":"2024-07-31T14:47:03.050686+0000","last_became_peered":"2024-07-31T14:47:03.050686+0000","last_unstale":"2024-07-31T14:47:20.626206+0000","last_undegraded":"2024-07-31T14:47:20.626206+0000","last_fullsized":"2024-07-31T14:47:20.626206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:48:02.494061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604422+0000","last_change":"2024-07-31T14:47:13.104229+0000","last_active":"2024-07-31T14:47:20.604422+0000","last_peered":"2024-07-31T14:47:20.604422+0000","last_clean":"2024-07-31T14:47:20.604422+0000","last_became_active":"2024-07-31T14:47:13.103737+0000","last_became_peered":"2024-07-31T14:47:13.103737+0000","last_unstale":"2024-07-31T14:47:20.604422+0000","last_undegraded":"2024-07-31T14:47:20.604422+0000","last_fullsized":"2024-07-31T14:47:20.604422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:36:00.541345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600473+0000","last_change":"2024-07-31T14:47:00.028169+0000","last_active":"2024-07-31T14:48:02.600473+0000","last_peered":"2024-07-31T14:48:02.600473+0000","last_clean":"2024-07-31T14:48:02.600473+0000","last_became_active":"2024-07-31T14:47:00.027757+0000","last_became_peered":"2024-07-31T14:47:00.027757+0000","last_unstale":"2024-07-31T14:48:02.600473+0000","last_undegraded":"2024-07-31T14:48:02.600473+0000","last_fullsized":"2024-07-31T14:48:02.600473+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:36:04.419551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626884+0000","last_change":"2024-07-31T14:47:03.057400+0000","last_active":"2024-07-31T14:47:20.626884+0000","last_peered":"2024-07-31T14:47:20.626884+0000","last_clean":"2024-07-31T14:47:20.626884+0000","last_became_active":"2024-07-31T14:47:03.057035+0000","last_became_peered":"2024-07-31T14:47:03.057035+0000","last_unstale":"2024-07-31T14:47:20.626884+0000","last_undegraded":"2024-07-31T14:47:20.626884+0000","last_fullsized":"2024-07-31T14:47:20.626884+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:55:07.951076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604150+0000","last_change":"2024-07-31T14:47:13.106093+0000","last_active":"2024-07-31T14:47:20.604150+0000","last_peered":"2024-07-31T14:47:20.604150+0000","last_clean":"2024-07-31T14:47:20.604150+0000","last_became_active":"2024-07-31T14:47:13.105459+0000","last_became_peered":"2024-07-31T14:47:13.105459+0000","last_unstale":"2024-07-31T14:47:20.604150+0000","last_undegraded":"2024-07-31T14:47:20.604150+0000","last_fullsized":"2024-07-31T14:47:20.604150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:07:52.751245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444336+0000","last_change":"2024-07-31T14:47:03.058701+0000","last_active":"2024-07-31T14:47:20.444336+0000","last_peered":"2024-07-31T14:47:20.444336+0000","last_clean":"2024-07-31T14:47:20.444336+0000","last_became_active":"2024-07-31T14:47:03.058359+0000","last_became_peered":"2024-07-31T14:47:03.058359+0000","last_unstale":"2024-07-31T14:47:20.444336+0000","last_undegraded":"2024-07-31T14:47:20.444336+0000","last_fullsized":"2024-07-31T14:47:20.444336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:37:36.149884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:59.565453+0000","last_change":"2024-07-31T14:47:00.022983+0000","last_active":"2024-07-31T14:47:59.565453+0000","last_peered":"2024-07-31T14:47:59.565453+0000","last_clean":"2024-07-31T14:47:59.565453+0000","last_became_active":"2024-07-31T14:47:00.022586+0000","last_became_peered":"2024-07-31T14:47:00.022586+0000","last_unstale":"2024-07-31T14:47:59.565453+0000","last_undegraded":"2024-07-31T14:47:59.565453+0000","last_fullsized":"2024-07-31T14:47:59.565453+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:29:42.641490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604397+0000","last_change":"2024-07-31T14:47:13.363148+0000","last_active":"2024-07-31T14:47:20.604397+0000","last_peered":"2024-07-31T14:47:20.604397+0000","last_clean":"2024-07-31T14:47:20.604397+0000","last_became_active":"2024-07-31T14:47:13.362577+0000","last_became_peered":"2024-07-31T14:47:13.362577+0000","last_unstale":"2024-07-31T14:47:20.604397+0000","last_undegraded":"2024-07-31T14:47:20.604397+0000","last_fullsized":"2024-07-31T14:47:20.604397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:37:59.003260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626394+0000","last_change":"2024-07-31T14:47:03.049586+0000","last_active":"2024-07-31T14:47:20.626394+0000","last_peered":"2024-07-31T14:47:20.626394+0000","last_clean":"2024-07-31T14:47:20.626394+0000","last_became_active":"2024-07-31T14:47:03.049154+0000","last_became_peered":"2024-07-31T14:47:03.049154+0000","last_unstale":"2024-07-31T14:47:20.626394+0000","last_undegraded":"2024-07-31T14:47:20.626394+0000","last_fullsized":"2024-07-31T14:47:20.626394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:51:01.971854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.589394+0000","last_change":"2024-07-31T14:47:00.021254+0000","last_active":"2024-07-31T14:48:01.589394+0000","last_peered":"2024-07-31T14:48:01.589394+0000","last_clean":"2024-07-31T14:48:01.589394+0000","last_became_active":"2024-07-31T14:47:00.020690+0000","last_became_peered":"2024-07-31T14:47:00.020690+0000","last_unstale":"2024-07-31T14:48:01.589394+0000","last_undegraded":"2024-07-31T14:48:01.589394+0000","last_fullsized":"2024-07-31T14:48:01.589394+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:01:27.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604729+0000","last_change":"2024-07-31T14:47:13.102220+0000","last_active":"2024-07-31T14:47:20.604729+0000","last_peered":"2024-07-31T14:47:20.604729+0000","last_clean":"2024-07-31T14:47:20.604729+0000","last_became_active":"2024-07-31T14:47:13.101764+0000","last_became_peered":"2024-07-31T14:47:13.101764+0000","last_unstale":"2024-07-31T14:47:20.604729+0000","last_undegraded":"2024-07-31T14:47:20.604729+0000","last_fullsized":"2024-07-31T14:47:20.604729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:56:44.335340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444911+0000","last_change":"2024-07-31T14:47:03.052348+0000","last_active":"2024-07-31T14:47:20.444911+0000","last_peered":"2024-07-31T14:47:20.444911+0000","last_clean":"2024-07-31T14:47:20.444911+0000","last_became_active":"2024-07-31T14:47:03.051961+0000","last_became_peered":"2024-07-31T14:47:03.051961+0000","last_unstale":"2024-07-31T14:47:20.444911+0000","last_undegraded":"2024-07-31T14:47:20.444911+0000","last_fullsized":"2024-07-31T14:47:20.444911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:19:35.279809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.597015+0000","last_change":"2024-07-31T14:47:00.025013+0000","last_active":"2024-07-31T14:48:02.597015+0000","last_peered":"2024-07-31T14:48:02.597015+0000","last_clean":"2024-07-31T14:48:02.597015+0000","last_became_active":"2024-07-31T14:47:00.024637+0000","last_became_peered":"2024-07-31T14:47:00.024637+0000","last_unstale":"2024-07-31T14:48:02.597015+0000","last_undegraded":"2024-07-31T14:48:02.597015+0000","last_fullsized":"2024-07-31T14:48:02.597015+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:07:21.519549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.629337+0000","last_change":"2024-07-31T14:47:20.629262+0000","last_active":"2024-07-31T14:47:20.629337+0000","last_peered":"2024-07-31T14:47:20.629337+0000","last_clean":"2024-07-31T14:47:20.629337+0000","last_became_active":"2024-07-31T14:47:20.628318+0000","last_became_peered":"2024-07-31T14:47:20.628318+0000","last_unstale":"2024-07-31T14:47:20.629337+0000","last_undegraded":"2024-07-31T14:47:20.629337+0000","last_fullsized":"2024-07-31T14:47:20.629337+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.206705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451066+0000","last_change":"2024-07-31T14:47:03.051527+0000","last_active":"2024-07-31T14:47:20.451066+0000","last_peered":"2024-07-31T14:47:20.451066+0000","last_clean":"2024-07-31T14:47:20.451066+0000","last_became_active":"2024-07-31T14:47:03.051139+0000","last_became_peered":"2024-07-31T14:47:03.051139+0000","last_unstale":"2024-07-31T14:47:20.451066+0000","last_undegraded":"2024-07-31T14:47:20.451066+0000","last_fullsized":"2024-07-31T14:47:20.451066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:53:11.962719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:59.564906+0000","last_change":"2024-07-31T14:47:00.183812+0000","last_active":"2024-07-31T14:47:59.564906+0000","last_peered":"2024-07-31T14:47:59.564906+0000","last_clean":"2024-07-31T14:47:59.564906+0000","last_became_active":"2024-07-31T14:47:00.183440+0000","last_became_peered":"2024-07-31T14:47:00.183440+0000","last_unstale":"2024-07-31T14:47:59.564906+0000","last_undegraded":"2024-07-31T14:47:59.564906+0000","last_fullsized":"2024-07-31T14:47:59.564906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:47:23.166756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604273+0000","last_change":"2024-07-31T14:47:13.363563+0000","last_active":"2024-07-31T14:47:20.604273+0000","last_peered":"2024-07-31T14:47:20.604273+0000","last_clean":"2024-07-31T14:47:20.604273+0000","last_became_active":"2024-07-31T14:47:13.362697+0000","last_became_peered":"2024-07-31T14:47:13.362697+0000","last_unstale":"2024-07-31T14:47:20.604273+0000","last_undegraded":"2024-07-31T14:47:20.604273+0000","last_fullsized":"2024-07-31T14:47:20.604273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:26:38.292073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449255+0000","last_change":"2024-07-31T14:47:03.046763+0000","last_active":"2024-07-31T14:47:20.449255+0000","last_peered":"2024-07-31T14:47:20.449255+0000","last_clean":"2024-07-31T14:47:20.449255+0000","last_became_active":"2024-07-31T14:47:03.046408+0000","last_became_peered":"2024-07-31T14:47:03.046408+0000","last_unstale":"2024-07-31T14:47:20.449255+0000","last_undegraded":"2024-07-31T14:47:20.449255+0000","last_fullsized":"2024-07-31T14:47:20.449255+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:43:22.838495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.599786+0000","last_change":"2024-07-31T14:47:00.236075+0000","last_active":"2024-07-31T14:48:02.599786+0000","last_peered":"2024-07-31T14:48:02.599786+0000","last_clean":"2024-07-31T14:48:02.599786+0000","last_became_active":"2024-07-31T14:47:00.235758+0000","last_became_peered":"2024-07-31T14:47:00.235758+0000","last_unstale":"2024-07-31T14:48:02.599786+0000","last_undegraded":"2024-07-31T14:48:02.599786+0000","last_fullsized":"2024-07-31T14:48:02.599786+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:50.575648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.448155+0000","last_change":"2024-07-31T14:47:13.101876+0000","last_active":"2024-07-31T14:47:20.448155+0000","last_peered":"2024-07-31T14:47:20.448155+0000","last_clean":"2024-07-31T14:47:20.448155+0000","last_became_active":"2024-07-31T14:47:13.101276+0000","last_became_peered":"2024-07-31T14:47:13.101276+0000","last_unstale":"2024-07-31T14:47:20.448155+0000","last_undegraded":"2024-07-31T14:47:20.448155+0000","last_fullsized":"2024-07-31T14:47:20.448155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:53:24.684212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627336+0000","last_change":"2024-07-31T14:47:03.052478+0000","last_active":"2024-07-31T14:47:20.627336+0000","last_peered":"2024-07-31T14:47:20.627336+0000","last_clean":"2024-07-31T14:47:20.627336+0000","last_became_active":"2024-07-31T14:47:03.051877+0000","last_became_peered":"2024-07-31T14:47:03.051877+0000","last_unstale":"2024-07-31T14:47:20.627336+0000","last_undegraded":"2024-07-31T14:47:20.627336+0000","last_fullsized":"2024-07-31T14:47:20.627336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:30:32.666490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:21.058148+0000","last_change":"2024-07-31T14:47:00.020076+0000","last_active":"2024-07-31T14:48:21.058148+0000","last_peered":"2024-07-31T14:48:21.058148+0000","last_clean":"2024-07-31T14:48:21.058148+0000","last_became_active":"2024-07-31T14:47:00.019708+0000","last_became_peered":"2024-07-31T14:47:00.019708+0000","last_unstale":"2024-07-31T14:48:21.058148+0000","last_undegraded":"2024-07-31T14:48:21.058148+0000","last_fullsized":"2024-07-31T14:48:21.058148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:46.520997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625774+0000","last_change":"2024-07-31T14:47:13.362961+0000","last_active":"2024-07-31T14:47:20.625774+0000","last_peered":"2024-07-31T14:47:20.625774+0000","last_clean":"2024-07-31T14:47:20.625774+0000","last_became_active":"2024-07-31T14:47:13.362502+0000","last_became_peered":"2024-07-31T14:47:13.362502+0000","last_unstale":"2024-07-31T14:47:20.625774+0000","last_undegraded":"2024-07-31T14:47:20.625774+0000","last_fullsized":"2024-07-31T14:47:20.625774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:51:45.732477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447467+0000","last_change":"2024-07-31T14:47:03.039418+0000","last_active":"2024-07-31T14:47:20.447467+0000","last_peered":"2024-07-31T14:47:20.447467+0000","last_clean":"2024-07-31T14:47:20.447467+0000","last_became_active":"2024-07-31T14:47:03.039044+0000","last_became_peered":"2024-07-31T14:47:03.039044+0000","last_unstale":"2024-07-31T14:47:20.447467+0000","last_undegraded":"2024-07-31T14:47:20.447467+0000","last_fullsized":"2024-07-31T14:47:20.447467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:40:10.380080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.599492+0000","last_change":"2024-07-31T14:47:00.183778+0000","last_active":"2024-07-31T14:48:02.599492+0000","last_peered":"2024-07-31T14:48:02.599492+0000","last_clean":"2024-07-31T14:48:02.599492+0000","last_became_active":"2024-07-31T14:47:00.183370+0000","last_became_peered":"2024-07-31T14:47:00.183370+0000","last_unstale":"2024-07-31T14:48:02.599492+0000","last_undegraded":"2024-07-31T14:48:02.599492+0000","last_fullsized":"2024-07-31T14:48:02.599492+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:45:27.422216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445075+0000","last_change":"2024-07-31T14:47:13.102583+0000","last_active":"2024-07-31T14:47:20.445075+0000","last_peered":"2024-07-31T14:47:20.445075+0000","last_clean":"2024-07-31T14:47:20.445075+0000","last_became_active":"2024-07-31T14:47:13.102058+0000","last_became_peered":"2024-07-31T14:47:13.102058+0000","last_unstale":"2024-07-31T14:47:20.445075+0000","last_undegraded":"2024-07-31T14:47:20.445075+0000","last_fullsized":"2024-07-31T14:47:20.445075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:17.396173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626564+0000","last_change":"2024-07-31T14:47:03.048334+0000","last_active":"2024-07-31T14:47:20.626564+0000","last_peered":"2024-07-31T14:47:20.626564+0000","last_clean":"2024-07-31T14:47:20.626564+0000","last_became_active":"2024-07-31T14:47:03.047932+0000","last_became_peered":"2024-07-31T14:47:03.047932+0000","last_unstale":"2024-07-31T14:47:20.626564+0000","last_undegraded":"2024-07-31T14:47:20.626564+0000","last_fullsized":"2024-07-31T14:47:20.626564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:14:31.381061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451611+0000","last_change":"2024-07-31T14:47:20.450851+0000","last_active":"2024-07-31T14:47:20.451611+0000","last_peered":"2024-07-31T14:47:20.451611+0000","last_clean":"2024-07-31T14:47:20.451611+0000","last_became_active":"2024-07-31T14:47:20.450435+0000","last_became_peered":"2024-07-31T14:47:20.450435+0000","last_unstale":"2024-07-31T14:47:20.451611+0000","last_undegraded":"2024-07-31T14:47:20.451611+0000","last_fullsized":"2024-07-31T14:47:20.451611+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:17:28.738911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.450787+0000","last_change":"2024-07-31T14:47:13.363054+0000","last_active":"2024-07-31T14:47:20.450787+0000","last_peered":"2024-07-31T14:47:20.450787+0000","last_clean":"2024-07-31T14:47:20.450787+0000","last_became_active":"2024-07-31T14:47:13.362364+0000","last_became_peered":"2024-07-31T14:47:13.362364+0000","last_unstale":"2024-07-31T14:47:20.450787+0000","last_undegraded":"2024-07-31T14:47:20.450787+0000","last_fullsized":"2024-07-31T14:47:20.450787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:02:36.569678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626675+0000","last_change":"2024-07-31T14:47:03.059834+0000","last_active":"2024-07-31T14:47:20.626675+0000","last_peered":"2024-07-31T14:47:20.626675+0000","last_clean":"2024-07-31T14:47:20.626675+0000","last_became_active":"2024-07-31T14:47:03.059127+0000","last_became_peered":"2024-07-31T14:47:03.059127+0000","last_unstale":"2024-07-31T14:47:20.626675+0000","last_undegraded":"2024-07-31T14:47:20.626675+0000","last_fullsized":"2024-07-31T14:47:20.626675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:53:50.308789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":72,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:24.638401+0000","last_change":"2024-07-31T14:47:00.030576+0000","last_active":"2024-07-31T14:48:24.638401+0000","last_peered":"2024-07-31T14:48:24.638401+0000","last_clean":"2024-07-31T14:48:24.638401+0000","last_became_active":"2024-07-31T14:47:00.030174+0000","last_became_peered":"2024-07-31T14:47:00.030174+0000","last_unstale":"2024-07-31T14:48:24.638401+0000","last_undegraded":"2024-07-31T14:48:24.638401+0000","last_fullsized":"2024-07-31T14:48:24.638401+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:13:35.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447207+0000","last_change":"2024-07-31T14:47:13.100180+0000","last_active":"2024-07-31T14:47:20.447207+0000","last_peered":"2024-07-31T14:47:20.447207+0000","last_clean":"2024-07-31T14:47:20.447207+0000","last_became_active":"2024-07-31T14:47:13.099725+0000","last_became_peered":"2024-07-31T14:47:13.099725+0000","last_unstale":"2024-07-31T14:47:20.447207+0000","last_undegraded":"2024-07-31T14:47:20.447207+0000","last_fullsized":"2024-07-31T14:47:20.447207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:25:32.245339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626501+0000","last_change":"2024-07-31T14:47:03.040349+0000","last_active":"2024-07-31T14:47:20.626501+0000","last_peered":"2024-07-31T14:47:20.626501+0000","last_clean":"2024-07-31T14:47:20.626501+0000","last_became_active":"2024-07-31T14:47:03.039887+0000","last_became_peered":"2024-07-31T14:47:03.039887+0000","last_unstale":"2024-07-31T14:47:20.626501+0000","last_undegraded":"2024-07-31T14:47:20.626501+0000","last_fullsized":"2024-07-31T14:47:20.626501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:46.592063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":72,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:26.640694+0000","last_change":"2024-07-31T14:47:00.183988+0000","last_active":"2024-07-31T14:48:26.640694+0000","last_peered":"2024-07-31T14:48:26.640694+0000","last_clean":"2024-07-31T14:48:26.640694+0000","last_became_active":"2024-07-31T14:47:00.183302+0000","last_became_peered":"2024-07-31T14:47:00.183302+0000","last_unstale":"2024-07-31T14:48:26.640694+0000","last_undegraded":"2024-07-31T14:48:26.640694+0000","last_fullsized":"2024-07-31T14:48:26.640694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:40:26.188224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447685+0000","last_change":"2024-07-31T14:47:13.363075+0000","last_active":"2024-07-31T14:47:20.447685+0000","last_peered":"2024-07-31T14:47:20.447685+0000","last_clean":"2024-07-31T14:47:20.447685+0000","last_became_active":"2024-07-31T14:47:13.362599+0000","last_became_peered":"2024-07-31T14:47:13.362599+0000","last_unstale":"2024-07-31T14:47:20.447685+0000","last_undegraded":"2024-07-31T14:47:20.447685+0000","last_fullsized":"2024-07-31T14:47:20.447685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:20:43.617742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.450794+0000","last_change":"2024-07-31T14:47:20.450712+0000","last_active":"2024-07-31T14:47:20.450794+0000","last_peered":"2024-07-31T14:47:20.450794+0000","last_clean":"2024-07-31T14:47:20.450794+0000","last_became_active":"2024-07-31T14:47:20.450229+0000","last_became_peered":"2024-07-31T14:47:20.450229+0000","last_unstale":"2024-07-31T14:47:20.450794+0000","last_undegraded":"2024-07-31T14:47:20.450794+0000","last_fullsized":"2024-07-31T14:47:20.450794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:11:50.631420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:00.569141+0000","last_change":"2024-07-31T14:47:00.021389+0000","last_active":"2024-07-31T14:48:00.569141+0000","last_peered":"2024-07-31T14:48:00.569141+0000","last_clean":"2024-07-31T14:48:00.569141+0000","last_became_active":"2024-07-31T14:47:00.020987+0000","last_became_peered":"2024-07-31T14:47:00.020987+0000","last_unstale":"2024-07-31T14:48:00.569141+0000","last_undegraded":"2024-07-31T14:48:00.569141+0000","last_fullsized":"2024-07-31T14:48:00.569141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:07:24.658463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626950+0000","last_change":"2024-07-31T14:47:13.363105+0000","last_active":"2024-07-31T14:47:20.626950+0000","last_peered":"2024-07-31T14:47:20.626950+0000","last_clean":"2024-07-31T14:47:20.626950+0000","last_became_active":"2024-07-31T14:47:13.362404+0000","last_became_peered":"2024-07-31T14:47:13.362404+0000","last_unstale":"2024-07-31T14:47:20.626950+0000","last_undegraded":"2024-07-31T14:47:20.626950+0000","last_fullsized":"2024-07-31T14:47:20.626950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:48:43.033584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626140+0000","last_change":"2024-07-31T14:47:03.057199+0000","last_active":"2024-07-31T14:47:20.626140+0000","last_peered":"2024-07-31T14:47:20.626140+0000","last_clean":"2024-07-31T14:47:20.626140+0000","last_became_active":"2024-07-31T14:47:03.056555+0000","last_became_peered":"2024-07-31T14:47:03.056555+0000","last_unstale":"2024-07-31T14:47:20.626140+0000","last_undegraded":"2024-07-31T14:47:20.626140+0000","last_fullsized":"2024-07-31T14:47:20.626140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:21:14.263457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605066+0000","last_change":"2024-07-31T14:47:00.032055+0000","last_active":"2024-07-31T14:48:03.605066+0000","last_peered":"2024-07-31T14:48:03.605066+0000","last_clean":"2024-07-31T14:48:03.605066+0000","last_became_active":"2024-07-31T14:47:00.031519+0000","last_became_peered":"2024-07-31T14:47:00.031519+0000","last_unstale":"2024-07-31T14:48:03.605066+0000","last_undegraded":"2024-07-31T14:48:03.605066+0000","last_fullsized":"2024-07-31T14:48:03.605066+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:21:46.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":117,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444372+0000","last_change":"2024-07-31T14:45:42.166503+0000","last_active":"2024-07-31T14:47:20.444372+0000","last_peered":"2024-07-31T14:47:20.444372+0000","last_clean":"2024-07-31T14:47:20.444372+0000","last_became_active":"2024-07-31T14:45:42.160065+0000","last_became_peered":"2024-07-31T14:45:42.160065+0000","last_unstale":"2024-07-31T14:47:20.444372+0000","last_undegraded":"2024-07-31T14:47:20.444372+0000","last_fullsized":"2024-07-31T14:47:20.444372+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_clean_scrub_stamp":"2024-07-31T14:42:27.863113+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:38:33.646022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444767+0000","last_change":"2024-07-31T14:47:13.103812+0000","last_active":"2024-07-31T14:47:20.444767+0000","last_peered":"2024-07-31T14:47:20.444767+0000","last_clean":"2024-07-31T14:47:20.444767+0000","last_became_active":"2024-07-31T14:47:13.102549+0000","last_became_peered":"2024-07-31T14:47:13.102549+0000","last_unstale":"2024-07-31T14:47:20.444767+0000","last_undegraded":"2024-07-31T14:47:20.444767+0000","last_fullsized":"2024-07-31T14:47:20.444767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:18:56.322426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444837+0000","last_change":"2024-07-31T14:47:03.056310+0000","last_active":"2024-07-31T14:47:20.444837+0000","last_peered":"2024-07-31T14:47:20.444837+0000","last_clean":"2024-07-31T14:47:20.444837+0000","last_became_active":"2024-07-31T14:47:03.055695+0000","last_became_peered":"2024-07-31T14:47:03.055695+0000","last_unstale":"2024-07-31T14:47:20.444837+0000","last_undegraded":"2024-07-31T14:47:20.444837+0000","last_fullsized":"2024-07-31T14:47:20.444837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:00:55.124981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":70,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:24.639115+0000","last_change":"2024-07-31T14:47:00.236460+0000","last_active":"2024-07-31T14:48:24.639115+0000","last_peered":"2024-07-31T14:48:24.639115+0000","last_clean":"2024-07-31T14:48:24.639115+0000","last_became_active":"2024-07-31T14:47:00.235924+0000","last_became_peered":"2024-07-31T14:47:00.235924+0000","last_unstale":"2024-07-31T14:48:24.639115+0000","last_undegraded":"2024-07-31T14:48:24.639115+0000","last_fullsized":"2024-07-31T14:48:24.639115+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:05.130958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625884+0000","last_change":"2024-07-31T14:47:13.102341+0000","last_active":"2024-07-31T14:47:20.625884+0000","last_peered":"2024-07-31T14:47:20.625884+0000","last_clean":"2024-07-31T14:47:20.625884+0000","last_became_active":"2024-07-31T14:47:13.101798+0000","last_became_peered":"2024-07-31T14:47:13.101798+0000","last_unstale":"2024-07-31T14:47:20.625884+0000","last_undegraded":"2024-07-31T14:47:20.625884+0000","last_fullsized":"2024-07-31T14:47:20.625884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:34:35.758041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444646+0000","last_change":"2024-07-31T14:47:03.054322+0000","last_active":"2024-07-31T14:47:20.444646+0000","last_peered":"2024-07-31T14:47:20.444646+0000","last_clean":"2024-07-31T14:47:20.444646+0000","last_became_active":"2024-07-31T14:47:03.053112+0000","last_became_peered":"2024-07-31T14:47:03.053112+0000","last_unstale":"2024-07-31T14:47:20.444646+0000","last_undegraded":"2024-07-31T14:47:20.444646+0000","last_fullsized":"2024-07-31T14:47:20.444646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:55:13.542578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600103+0000","last_change":"2024-07-31T14:47:00.026518+0000","last_active":"2024-07-31T14:48:02.600103+0000","last_peered":"2024-07-31T14:48:02.600103+0000","last_clean":"2024-07-31T14:48:02.600103+0000","last_became_active":"2024-07-31T14:47:00.026074+0000","last_became_peered":"2024-07-31T14:47:00.026074+0000","last_unstale":"2024-07-31T14:48:02.600103+0000","last_undegraded":"2024-07-31T14:48:02.600103+0000","last_fullsized":"2024-07-31T14:48:02.600103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:33:36.688353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449307+0000","last_change":"2024-07-31T14:47:13.106365+0000","last_active":"2024-07-31T14:47:20.449307+0000","last_peered":"2024-07-31T14:47:20.449307+0000","last_clean":"2024-07-31T14:47:20.449307+0000","last_became_active":"2024-07-31T14:47:13.105635+0000","last_became_peered":"2024-07-31T14:47:13.105635+0000","last_unstale":"2024-07-31T14:47:20.449307+0000","last_undegraded":"2024-07-31T14:47:20.449307+0000","last_fullsized":"2024-07-31T14:47:20.449307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:52:09.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444806+0000","last_change":"2024-07-31T14:47:03.052522+0000","last_active":"2024-07-31T14:47:20.444806+0000","last_peered":"2024-07-31T14:47:20.444806+0000","last_clean":"2024-07-31T14:47:20.444806+0000","last_became_active":"2024-07-31T14:47:03.051872+0000","last_became_peered":"2024-07-31T14:47:03.051872+0000","last_unstale":"2024-07-31T14:47:20.444806+0000","last_undegraded":"2024-07-31T14:47:20.444806+0000","last_fullsized":"2024-07-31T14:47:20.444806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:29:34.956745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600858+0000","last_change":"2024-07-31T14:47:00.016839+0000","last_active":"2024-07-31T14:48:02.600858+0000","last_peered":"2024-07-31T14:48:02.600858+0000","last_clean":"2024-07-31T14:48:02.600858+0000","last_became_active":"2024-07-31T14:47:00.016438+0000","last_became_peered":"2024-07-31T14:47:00.016438+0000","last_unstale":"2024-07-31T14:48:02.600858+0000","last_undegraded":"2024-07-31T14:48:02.600858+0000","last_fullsized":"2024-07-31T14:48:02.600858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:42:10.875332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626192+0000","last_change":"2024-07-31T14:47:13.105385+0000","last_active":"2024-07-31T14:47:20.626192+0000","last_peered":"2024-07-31T14:47:20.626192+0000","last_clean":"2024-07-31T14:47:20.626192+0000","last_became_active":"2024-07-31T14:47:13.104867+0000","last_became_peered":"2024-07-31T14:47:13.104867+0000","last_unstale":"2024-07-31T14:47:20.626192+0000","last_undegraded":"2024-07-31T14:47:20.626192+0000","last_fullsized":"2024-07-31T14:47:20.626192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:25:21.572830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445617+0000","last_change":"2024-07-31T14:47:03.058304+0000","last_active":"2024-07-31T14:47:20.445617+0000","last_peered":"2024-07-31T14:47:20.445617+0000","last_clean":"2024-07-31T14:47:20.445617+0000","last_became_active":"2024-07-31T14:47:03.057251+0000","last_became_peered":"2024-07-31T14:47:03.057251+0000","last_unstale":"2024-07-31T14:47:20.445617+0000","last_undegraded":"2024-07-31T14:47:20.445617+0000","last_fullsized":"2024-07-31T14:47:20.445617+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:58:20.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.566267+0000","last_change":"2024-07-31T14:47:00.027806+0000","last_active":"2024-07-31T14:48:02.566267+0000","last_peered":"2024-07-31T14:48:02.566267+0000","last_clean":"2024-07-31T14:48:02.566267+0000","last_became_active":"2024-07-31T14:47:00.027403+0000","last_became_peered":"2024-07-31T14:47:00.027403+0000","last_unstale":"2024-07-31T14:48:02.566267+0000","last_undegraded":"2024-07-31T14:48:02.566267+0000","last_fullsized":"2024-07-31T14:48:02.566267+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:57:18.303732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626432+0000","last_change":"2024-07-31T14:47:13.103628+0000","last_active":"2024-07-31T14:47:20.626432+0000","last_peered":"2024-07-31T14:47:20.626432+0000","last_clean":"2024-07-31T14:47:20.626432+0000","last_became_active":"2024-07-31T14:47:13.102296+0000","last_became_peered":"2024-07-31T14:47:13.102296+0000","last_unstale":"2024-07-31T14:47:20.626432+0000","last_undegraded":"2024-07-31T14:47:20.626432+0000","last_fullsized":"2024-07-31T14:47:20.626432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:25:35.109196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626520+0000","last_change":"2024-07-31T14:47:03.052761+0000","last_active":"2024-07-31T14:47:20.626520+0000","last_peered":"2024-07-31T14:47:20.626520+0000","last_clean":"2024-07-31T14:47:20.626520+0000","last_became_active":"2024-07-31T14:47:03.052410+0000","last_became_peered":"2024-07-31T14:47:03.052410+0000","last_unstale":"2024-07-31T14:47:20.626520+0000","last_undegraded":"2024-07-31T14:47:20.626520+0000","last_fullsized":"2024-07-31T14:47:20.626520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:20:47.712491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.599712+0000","last_change":"2024-07-31T14:47:00.183741+0000","last_active":"2024-07-31T14:48:02.599712+0000","last_peered":"2024-07-31T14:48:02.599712+0000","last_clean":"2024-07-31T14:48:02.599712+0000","last_became_active":"2024-07-31T14:47:00.183275+0000","last_became_peered":"2024-07-31T14:47:00.183275+0000","last_unstale":"2024-07-31T14:48:02.599712+0000","last_undegraded":"2024-07-31T14:48:02.599712+0000","last_fullsized":"2024-07-31T14:48:02.599712+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:29:23.205835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.629408+0000","last_change":"2024-07-31T14:47:20.629334+0000","last_active":"2024-07-31T14:47:20.629408+0000","last_peered":"2024-07-31T14:47:20.629408+0000","last_clean":"2024-07-31T14:47:20.629408+0000","last_became_active":"2024-07-31T14:47:20.628865+0000","last_became_peered":"2024-07-31T14:47:20.628865+0000","last_unstale":"2024-07-31T14:47:20.629408+0000","last_undegraded":"2024-07-31T14:47:20.629408+0000","last_fullsized":"2024-07-31T14:47:20.629408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:21:27.199842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444090+0000","last_change":"2024-07-31T14:47:03.054633+0000","last_active":"2024-07-31T14:47:20.444090+0000","last_peered":"2024-07-31T14:47:20.444090+0000","last_clean":"2024-07-31T14:47:20.444090+0000","last_became_active":"2024-07-31T14:47:03.053025+0000","last_became_peered":"2024-07-31T14:47:03.053025+0000","last_unstale":"2024-07-31T14:47:20.444090+0000","last_undegraded":"2024-07-31T14:47:20.444090+0000","last_fullsized":"2024-07-31T14:47:20.444090+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:42:03.392744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:27.642641+0000","last_change":"2024-07-31T14:47:00.027898+0000","last_active":"2024-07-31T14:48:27.642641+0000","last_peered":"2024-07-31T14:48:27.642641+0000","last_clean":"2024-07-31T14:48:27.642641+0000","last_became_active":"2024-07-31T14:47:00.027468+0000","last_became_peered":"2024-07-31T14:47:00.027468+0000","last_unstale":"2024-07-31T14:48:27.642641+0000","last_undegraded":"2024-07-31T14:48:27.642641+0000","last_fullsized":"2024-07-31T14:48:27.642641+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:12:45.922172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444515+0000","last_change":"2024-07-31T14:47:13.104327+0000","last_active":"2024-07-31T14:47:20.444515+0000","last_peered":"2024-07-31T14:47:20.444515+0000","last_clean":"2024-07-31T14:47:20.444515+0000","last_became_active":"2024-07-31T14:47:13.103333+0000","last_became_peered":"2024-07-31T14:47:13.103333+0000","last_unstale":"2024-07-31T14:47:20.444515+0000","last_undegraded":"2024-07-31T14:47:20.444515+0000","last_fullsized":"2024-07-31T14:47:20.444515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:06.226220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444413+0000","last_change":"2024-07-31T14:47:03.044875+0000","last_active":"2024-07-31T14:47:20.444413+0000","last_peered":"2024-07-31T14:47:20.444413+0000","last_clean":"2024-07-31T14:47:20.444413+0000","last_became_active":"2024-07-31T14:47:03.044421+0000","last_became_peered":"2024-07-31T14:47:03.044421+0000","last_unstale":"2024-07-31T14:47:20.444413+0000","last_undegraded":"2024-07-31T14:47:20.444413+0000","last_fullsized":"2024-07-31T14:47:20.444413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:00:02.477300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600649+0000","last_change":"2024-07-31T14:47:00.019790+0000","last_active":"2024-07-31T14:48:02.600649+0000","last_peered":"2024-07-31T14:48:02.600649+0000","last_clean":"2024-07-31T14:48:02.600649+0000","last_became_active":"2024-07-31T14:47:00.019368+0000","last_became_peered":"2024-07-31T14:47:00.019368+0000","last_unstale":"2024-07-31T14:48:02.600649+0000","last_undegraded":"2024-07-31T14:48:02.600649+0000","last_fullsized":"2024-07-31T14:48:02.600649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:15.299109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445210+0000","last_change":"2024-07-31T14:47:13.102618+0000","last_active":"2024-07-31T14:47:20.445210+0000","last_peered":"2024-07-31T14:47:20.445210+0000","last_clean":"2024-07-31T14:47:20.445210+0000","last_became_active":"2024-07-31T14:47:13.102085+0000","last_became_peered":"2024-07-31T14:47:13.102085+0000","last_unstale":"2024-07-31T14:47:20.445210+0000","last_undegraded":"2024-07-31T14:47:20.445210+0000","last_fullsized":"2024-07-31T14:47:20.445210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:05:58.814215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447335+0000","last_change":"2024-07-31T14:47:03.044688+0000","last_active":"2024-07-31T14:47:20.447335+0000","last_peered":"2024-07-31T14:47:20.447335+0000","last_clean":"2024-07-31T14:47:20.447335+0000","last_became_active":"2024-07-31T14:47:03.043751+0000","last_became_peered":"2024-07-31T14:47:03.043751+0000","last_unstale":"2024-07-31T14:47:20.447335+0000","last_undegraded":"2024-07-31T14:47:20.447335+0000","last_fullsized":"2024-07-31T14:47:20.447335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:47:31.450796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604782+0000","last_change":"2024-07-31T14:47:00.026259+0000","last_active":"2024-07-31T14:48:03.604782+0000","last_peered":"2024-07-31T14:48:03.604782+0000","last_clean":"2024-07-31T14:48:03.604782+0000","last_became_active":"2024-07-31T14:47:00.025756+0000","last_became_peered":"2024-07-31T14:47:00.025756+0000","last_unstale":"2024-07-31T14:48:03.604782+0000","last_undegraded":"2024-07-31T14:48:03.604782+0000","last_fullsized":"2024-07-31T14:48:03.604782+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:05:00.258841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626813+0000","last_change":"2024-07-31T14:47:13.104918+0000","last_active":"2024-07-31T14:47:20.626813+0000","last_peered":"2024-07-31T14:47:20.626813+0000","last_clean":"2024-07-31T14:47:20.626813+0000","last_became_active":"2024-07-31T14:47:13.104031+0000","last_became_peered":"2024-07-31T14:47:13.104031+0000","last_unstale":"2024-07-31T14:47:20.626813+0000","last_undegraded":"2024-07-31T14:47:20.626813+0000","last_fullsized":"2024-07-31T14:47:20.626813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:17:15.264474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451830+0000","last_change":"2024-07-31T14:47:03.055159+0000","last_active":"2024-07-31T14:47:20.451830+0000","last_peered":"2024-07-31T14:47:20.451830+0000","last_clean":"2024-07-31T14:47:20.451830+0000","last_became_active":"2024-07-31T14:47:03.052389+0000","last_became_peered":"2024-07-31T14:47:03.052389+0000","last_unstale":"2024-07-31T14:47:20.451830+0000","last_undegraded":"2024-07-31T14:47:20.451830+0000","last_fullsized":"2024-07-31T14:47:20.451830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:16.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":136,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:29.645868+0000","last_change":"2024-07-31T14:47:00.030165+0000","last_active":"2024-07-31T14:48:29.645868+0000","last_peered":"2024-07-31T14:48:29.645868+0000","last_clean":"2024-07-31T14:48:29.645868+0000","last_became_active":"2024-07-31T14:47:00.029559+0000","last_became_peered":"2024-07-31T14:47:00.029559+0000","last_unstale":"2024-07-31T14:48:29.645868+0000","last_undegraded":"2024-07-31T14:48:29.645868+0000","last_fullsized":"2024-07-31T14:48:29.645868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:31:06.115075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57883,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":62,"num_read_kb":87,"num_write":35,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444173+0000","last_change":"2024-07-31T14:47:13.098517+0000","last_active":"2024-07-31T14:47:20.444173+0000","last_peered":"2024-07-31T14:47:20.444173+0000","last_clean":"2024-07-31T14:47:20.444173+0000","last_became_active":"2024-07-31T14:47:13.098066+0000","last_became_peered":"2024-07-31T14:47:13.098066+0000","last_unstale":"2024-07-31T14:47:20.444173+0000","last_undegraded":"2024-07-31T14:47:20.444173+0000","last_fullsized":"2024-07-31T14:47:20.444173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:08:54.958646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626197+0000","last_change":"2024-07-31T14:47:03.054045+0000","last_active":"2024-07-31T14:47:20.626197+0000","last_peered":"2024-07-31T14:47:20.626197+0000","last_clean":"2024-07-31T14:47:20.626197+0000","last_became_active":"2024-07-31T14:47:03.053658+0000","last_became_peered":"2024-07-31T14:47:03.053658+0000","last_unstale":"2024-07-31T14:47:20.626197+0000","last_undegraded":"2024-07-31T14:47:20.626197+0000","last_fullsized":"2024-07-31T14:47:20.626197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:36:54.563542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.589079+0000","last_change":"2024-07-31T14:47:00.185602+0000","last_active":"2024-07-31T14:48:01.589079+0000","last_peered":"2024-07-31T14:48:01.589079+0000","last_clean":"2024-07-31T14:48:01.589079+0000","last_became_active":"2024-07-31T14:47:00.185230+0000","last_became_peered":"2024-07-31T14:47:00.185230+0000","last_unstale":"2024-07-31T14:48:01.589079+0000","last_undegraded":"2024-07-31T14:48:01.589079+0000","last_fullsized":"2024-07-31T14:48:01.589079+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:21:35.979942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604618+0000","last_change":"2024-07-31T14:47:13.099611+0000","last_active":"2024-07-31T14:47:20.604618+0000","last_peered":"2024-07-31T14:47:20.604618+0000","last_clean":"2024-07-31T14:47:20.604618+0000","last_became_active":"2024-07-31T14:47:13.099127+0000","last_became_peered":"2024-07-31T14:47:13.099127+0000","last_unstale":"2024-07-31T14:47:20.604618+0000","last_undegraded":"2024-07-31T14:47:20.604618+0000","last_fullsized":"2024-07-31T14:47:20.604618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:16:57.192980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449734+0000","last_change":"2024-07-31T14:47:03.056818+0000","last_active":"2024-07-31T14:47:20.449734+0000","last_peered":"2024-07-31T14:47:20.449734+0000","last_clean":"2024-07-31T14:47:20.449734+0000","last_became_active":"2024-07-31T14:47:03.055716+0000","last_became_peered":"2024-07-31T14:47:03.055716+0000","last_unstale":"2024-07-31T14:47:20.449734+0000","last_undegraded":"2024-07-31T14:47:20.449734+0000","last_fullsized":"2024-07-31T14:47:20.449734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:26:33.627362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600183+0000","last_change":"2024-07-31T14:47:00.029651+0000","last_active":"2024-07-31T14:48:02.600183+0000","last_peered":"2024-07-31T14:48:02.600183+0000","last_clean":"2024-07-31T14:48:02.600183+0000","last_became_active":"2024-07-31T14:47:00.029242+0000","last_became_peered":"2024-07-31T14:47:00.029242+0000","last_unstale":"2024-07-31T14:48:02.600183+0000","last_undegraded":"2024-07-31T14:48:02.600183+0000","last_fullsized":"2024-07-31T14:48:02.600183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:23:49.177508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.443789+0000","last_change":"2024-07-31T14:47:13.098862+0000","last_active":"2024-07-31T14:47:20.443789+0000","last_peered":"2024-07-31T14:47:20.443789+0000","last_clean":"2024-07-31T14:47:20.443789+0000","last_became_active":"2024-07-31T14:47:13.098324+0000","last_became_peered":"2024-07-31T14:47:13.098324+0000","last_unstale":"2024-07-31T14:47:20.443789+0000","last_undegraded":"2024-07-31T14:47:20.443789+0000","last_fullsized":"2024-07-31T14:47:20.443789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:50:41.260863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626600+0000","last_change":"2024-07-31T14:47:03.049042+0000","last_active":"2024-07-31T14:47:20.626600+0000","last_peered":"2024-07-31T14:47:20.626600+0000","last_clean":"2024-07-31T14:47:20.626600+0000","last_became_active":"2024-07-31T14:47:03.048535+0000","last_became_peered":"2024-07-31T14:47:03.048535+0000","last_unstale":"2024-07-31T14:47:20.626600+0000","last_undegraded":"2024-07-31T14:47:20.626600+0000","last_fullsized":"2024-07-31T14:47:20.626600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:15.147454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":99,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:29.646074+0000","last_change":"2024-07-31T14:47:00.028940+0000","last_active":"2024-07-31T14:48:29.646074+0000","last_peered":"2024-07-31T14:48:29.646074+0000","last_clean":"2024-07-31T14:48:29.646074+0000","last_became_active":"2024-07-31T14:47:00.028472+0000","last_became_peered":"2024-07-31T14:47:00.028472+0000","last_unstale":"2024-07-31T14:48:29.646074+0000","last_undegraded":"2024-07-31T14:48:29.646074+0000","last_fullsized":"2024-07-31T14:48:29.646074+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:57:15.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626333+0000","last_change":"2024-07-31T14:47:13.101856+0000","last_active":"2024-07-31T14:47:20.626333+0000","last_peered":"2024-07-31T14:47:20.626333+0000","last_clean":"2024-07-31T14:47:20.626333+0000","last_became_active":"2024-07-31T14:47:13.101346+0000","last_became_peered":"2024-07-31T14:47:13.101346+0000","last_unstale":"2024-07-31T14:47:20.626333+0000","last_undegraded":"2024-07-31T14:47:20.626333+0000","last_fullsized":"2024-07-31T14:47:20.626333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:56:37.071660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449066+0000","last_change":"2024-07-31T14:47:13.105952+0000","last_active":"2024-07-31T14:47:20.449066+0000","last_peered":"2024-07-31T14:47:20.449066+0000","last_clean":"2024-07-31T14:47:20.449066+0000","last_became_active":"2024-07-31T14:47:13.105506+0000","last_became_peered":"2024-07-31T14:47:13.105506+0000","last_unstale":"2024-07-31T14:47:20.449066+0000","last_undegraded":"2024-07-31T14:47:20.449066+0000","last_fullsized":"2024-07-31T14:47:20.449066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:28:33.866216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444136+0000","last_change":"2024-07-31T14:47:03.051152+0000","last_active":"2024-07-31T14:47:20.444136+0000","last_peered":"2024-07-31T14:47:20.444136+0000","last_clean":"2024-07-31T14:47:20.444136+0000","last_became_active":"2024-07-31T14:47:03.050775+0000","last_became_peered":"2024-07-31T14:47:03.050775+0000","last_unstale":"2024-07-31T14:47:20.444136+0000","last_undegraded":"2024-07-31T14:47:20.444136+0000","last_fullsized":"2024-07-31T14:47:20.444136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:47:35.913641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600742+0000","last_change":"2024-07-31T14:47:00.028552+0000","last_active":"2024-07-31T14:48:02.600742+0000","last_peered":"2024-07-31T14:48:02.600742+0000","last_clean":"2024-07-31T14:48:02.600742+0000","last_became_active":"2024-07-31T14:47:00.028143+0000","last_became_peered":"2024-07-31T14:47:00.028143+0000","last_unstale":"2024-07-31T14:48:02.600742+0000","last_undegraded":"2024-07-31T14:48:02.600742+0000","last_fullsized":"2024-07-31T14:48:02.600742+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:09.601635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.606387+0000","last_change":"2024-07-31T14:47:20.606285+0000","last_active":"2024-07-31T14:47:20.606387+0000","last_peered":"2024-07-31T14:47:20.606387+0000","last_clean":"2024-07-31T14:47:20.606387+0000","last_became_active":"2024-07-31T14:47:20.605837+0000","last_became_peered":"2024-07-31T14:47:20.605837+0000","last_unstale":"2024-07-31T14:47:20.606387+0000","last_undegraded":"2024-07-31T14:47:20.606387+0000","last_fullsized":"2024-07-31T14:47:20.606387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:16:35.409790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625946+0000","last_change":"2024-07-31T14:47:03.047442+0000","last_active":"2024-07-31T14:47:20.625946+0000","last_peered":"2024-07-31T14:47:20.625946+0000","last_clean":"2024-07-31T14:47:20.625946+0000","last_became_active":"2024-07-31T14:47:03.047072+0000","last_became_peered":"2024-07-31T14:47:03.047072+0000","last_unstale":"2024-07-31T14:47:20.625946+0000","last_undegraded":"2024-07-31T14:47:20.625946+0000","last_fullsized":"2024-07-31T14:47:20.625946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:56.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:00.572013+0000","last_change":"2024-07-31T14:47:00.027746+0000","last_active":"2024-07-31T14:48:00.572013+0000","last_peered":"2024-07-31T14:48:00.572013+0000","last_clean":"2024-07-31T14:48:00.572013+0000","last_became_active":"2024-07-31T14:47:00.027404+0000","last_became_peered":"2024-07-31T14:47:00.027404+0000","last_unstale":"2024-07-31T14:48:00.572013+0000","last_undegraded":"2024-07-31T14:48:00.572013+0000","last_fullsized":"2024-07-31T14:48:00.572013+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:30:21.011674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.589312+0000","last_change":"2024-07-31T14:47:00.029148+0000","last_active":"2024-07-31T14:48:01.589312+0000","last_peered":"2024-07-31T14:48:01.589312+0000","last_clean":"2024-07-31T14:48:01.589312+0000","last_became_active":"2024-07-31T14:47:00.028718+0000","last_became_peered":"2024-07-31T14:47:00.028718+0000","last_unstale":"2024-07-31T14:48:01.589312+0000","last_undegraded":"2024-07-31T14:48:01.589312+0000","last_fullsized":"2024-07-31T14:48:01.589312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:25:16.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630241+0000","last_change":"2024-07-31T14:47:20.629098+0000","last_active":"2024-07-31T14:47:20.630241+0000","last_peered":"2024-07-31T14:47:20.630241+0000","last_clean":"2024-07-31T14:47:20.630241+0000","last_became_active":"2024-07-31T14:47:20.627775+0000","last_became_peered":"2024-07-31T14:47:20.627775+0000","last_unstale":"2024-07-31T14:47:20.630241+0000","last_undegraded":"2024-07-31T14:47:20.630241+0000","last_fullsized":"2024-07-31T14:47:20.630241+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:22:03.606509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447260+0000","last_change":"2024-07-31T14:47:13.097892+0000","last_active":"2024-07-31T14:47:20.447260+0000","last_peered":"2024-07-31T14:47:20.447260+0000","last_clean":"2024-07-31T14:47:20.447260+0000","last_became_active":"2024-07-31T14:47:13.097419+0000","last_became_peered":"2024-07-31T14:47:13.097419+0000","last_unstale":"2024-07-31T14:47:20.447260+0000","last_undegraded":"2024-07-31T14:47:20.447260+0000","last_fullsized":"2024-07-31T14:47:20.447260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:07:13.456147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:26.642498+0000","last_change":"2024-07-31T14:47:00.019792+0000","last_active":"2024-07-31T14:48:26.642498+0000","last_peered":"2024-07-31T14:48:26.642498+0000","last_clean":"2024-07-31T14:48:26.642498+0000","last_became_active":"2024-07-31T14:47:00.019411+0000","last_became_peered":"2024-07-31T14:47:00.019411+0000","last_unstale":"2024-07-31T14:48:26.642498+0000","last_undegraded":"2024-07-31T14:48:26.642498+0000","last_fullsized":"2024-07-31T14:48:26.642498+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:55:02.148823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.450923+0000","last_change":"2024-07-31T14:47:03.039808+0000","last_active":"2024-07-31T14:47:20.450923+0000","last_peered":"2024-07-31T14:47:20.450923+0000","last_clean":"2024-07-31T14:47:20.450923+0000","last_became_active":"2024-07-31T14:47:03.039382+0000","last_became_peered":"2024-07-31T14:47:03.039382+0000","last_unstale":"2024-07-31T14:47:20.450923+0000","last_undegraded":"2024-07-31T14:47:20.450923+0000","last_fullsized":"2024-07-31T14:47:20.450923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:23:09.751293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.446875+0000","last_change":"2024-07-31T14:47:13.102797+0000","last_active":"2024-07-31T14:47:20.446875+0000","last_peered":"2024-07-31T14:47:20.446875+0000","last_clean":"2024-07-31T14:47:20.446875+0000","last_became_active":"2024-07-31T14:47:13.102320+0000","last_became_peered":"2024-07-31T14:47:13.102320+0000","last_unstale":"2024-07-31T14:47:20.446875+0000","last_undegraded":"2024-07-31T14:47:20.446875+0000","last_fullsized":"2024-07-31T14:47:20.446875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:23.066357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444689+0000","last_change":"2024-07-31T14:47:03.054875+0000","last_active":"2024-07-31T14:47:20.444689+0000","last_peered":"2024-07-31T14:47:20.444689+0000","last_clean":"2024-07-31T14:47:20.444689+0000","last_became_active":"2024-07-31T14:47:03.054529+0000","last_became_peered":"2024-07-31T14:47:03.054529+0000","last_unstale":"2024-07-31T14:47:20.444689+0000","last_undegraded":"2024-07-31T14:47:20.444689+0000","last_fullsized":"2024-07-31T14:47:20.444689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:32:09.802933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.448452+0000","last_change":"2024-07-31T14:47:03.043283+0000","last_active":"2024-07-31T14:47:20.448452+0000","last_peered":"2024-07-31T14:47:20.448452+0000","last_clean":"2024-07-31T14:47:20.448452+0000","last_became_active":"2024-07-31T14:47:03.042851+0000","last_became_peered":"2024-07-31T14:47:03.042851+0000","last_unstale":"2024-07-31T14:47:20.448452+0000","last_undegraded":"2024-07-31T14:47:20.448452+0000","last_fullsized":"2024-07-31T14:47:20.448452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:25:37.408149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447315+0000","last_change":"2024-07-31T14:47:03.047890+0000","last_active":"2024-07-31T14:47:20.447315+0000","last_peered":"2024-07-31T14:47:20.447315+0000","last_clean":"2024-07-31T14:47:20.447315+0000","last_became_active":"2024-07-31T14:47:03.047507+0000","last_became_peered":"2024-07-31T14:47:03.047507+0000","last_unstale":"2024-07-31T14:47:20.447315+0000","last_undegraded":"2024-07-31T14:47:20.447315+0000","last_fullsized":"2024-07-31T14:47:20.447315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:52:45.991323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449437+0000","last_change":"2024-07-31T14:47:03.047903+0000","last_active":"2024-07-31T14:47:20.449437+0000","last_peered":"2024-07-31T14:47:20.449437+0000","last_clean":"2024-07-31T14:47:20.449437+0000","last_became_active":"2024-07-31T14:47:03.047352+0000","last_became_peered":"2024-07-31T14:47:03.047352+0000","last_unstale":"2024-07-31T14:47:20.449437+0000","last_undegraded":"2024-07-31T14:47:20.449437+0000","last_fullsized":"2024-07-31T14:47:20.449437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:06.536023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447343+0000","last_change":"2024-07-31T14:47:03.044199+0000","last_active":"2024-07-31T14:47:20.447343+0000","last_peered":"2024-07-31T14:47:20.447343+0000","last_clean":"2024-07-31T14:47:20.447343+0000","last_became_active":"2024-07-31T14:47:03.043849+0000","last_became_peered":"2024-07-31T14:47:03.043849+0000","last_unstale":"2024-07-31T14:47:20.447343+0000","last_undegraded":"2024-07-31T14:47:20.447343+0000","last_fullsized":"2024-07-31T14:47:20.447343+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:50.474895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445907+0000","last_change":"2024-07-31T14:47:03.060654+0000","last_active":"2024-07-31T14:47:20.445907+0000","last_peered":"2024-07-31T14:47:20.445907+0000","last_clean":"2024-07-31T14:47:20.445907+0000","last_became_active":"2024-07-31T14:47:03.060157+0000","last_became_peered":"2024-07-31T14:47:03.060157+0000","last_unstale":"2024-07-31T14:47:20.445907+0000","last_undegraded":"2024-07-31T14:47:20.445907+0000","last_fullsized":"2024-07-31T14:47:20.445907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:39:50.130831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447018+0000","last_change":"2024-07-31T14:47:03.049910+0000","last_active":"2024-07-31T14:47:20.447018+0000","last_peered":"2024-07-31T14:47:20.447018+0000","last_clean":"2024-07-31T14:47:20.447018+0000","last_became_active":"2024-07-31T14:47:03.049491+0000","last_became_peered":"2024-07-31T14:47:03.049491+0000","last_unstale":"2024-07-31T14:47:20.447018+0000","last_undegraded":"2024-07-31T14:47:20.447018+0000","last_fullsized":"2024-07-31T14:47:20.447018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:18:43.319695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627262+0000","last_change":"2024-07-31T14:47:03.052206+0000","last_active":"2024-07-31T14:47:20.627262+0000","last_peered":"2024-07-31T14:47:20.627262+0000","last_clean":"2024-07-31T14:47:20.627262+0000","last_became_active":"2024-07-31T14:47:03.051175+0000","last_became_peered":"2024-07-31T14:47:03.051175+0000","last_unstale":"2024-07-31T14:47:20.627262+0000","last_undegraded":"2024-07-31T14:47:20.627262+0000","last_fullsized":"2024-07-31T14:47:20.627262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:51:11.583604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626447+0000","last_change":"2024-07-31T14:47:03.052162+0000","last_active":"2024-07-31T14:47:20.626447+0000","last_peered":"2024-07-31T14:47:20.626447+0000","last_clean":"2024-07-31T14:47:20.626447+0000","last_became_active":"2024-07-31T14:47:03.051634+0000","last_became_peered":"2024-07-31T14:47:03.051634+0000","last_unstale":"2024-07-31T14:47:20.626447+0000","last_undegraded":"2024-07-31T14:47:20.626447+0000","last_fullsized":"2024-07-31T14:47:20.626447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:41:59.605192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626439+0000","last_change":"2024-07-31T14:47:03.047475+0000","last_active":"2024-07-31T14:47:20.626439+0000","last_peered":"2024-07-31T14:47:20.626439+0000","last_clean":"2024-07-31T14:47:20.626439+0000","last_became_active":"2024-07-31T14:47:03.047100+0000","last_became_peered":"2024-07-31T14:47:03.047100+0000","last_unstale":"2024-07-31T14:47:20.626439+0000","last_undegraded":"2024-07-31T14:47:20.626439+0000","last_fullsized":"2024-07-31T14:47:20.626439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:08:10.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626240+0000","last_change":"2024-07-31T14:47:03.051281+0000","last_active":"2024-07-31T14:47:20.626240+0000","last_peered":"2024-07-31T14:47:20.626240+0000","last_clean":"2024-07-31T14:47:20.626240+0000","last_became_active":"2024-07-31T14:47:03.050921+0000","last_became_peered":"2024-07-31T14:47:03.050921+0000","last_unstale":"2024-07-31T14:47:20.626240+0000","last_undegraded":"2024-07-31T14:47:20.626240+0000","last_fullsized":"2024-07-31T14:47:20.626240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:24:29.221723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447393+0000","last_change":"2024-07-31T14:47:03.045457+0000","last_active":"2024-07-31T14:47:20.447393+0000","last_peered":"2024-07-31T14:47:20.447393+0000","last_clean":"2024-07-31T14:47:20.447393+0000","last_became_active":"2024-07-31T14:47:03.044466+0000","last_became_peered":"2024-07-31T14:47:03.044466+0000","last_unstale":"2024-07-31T14:47:20.447393+0000","last_undegraded":"2024-07-31T14:47:20.447393+0000","last_fullsized":"2024-07-31T14:47:20.447393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:44:01.197416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447013+0000","last_change":"2024-07-31T14:47:03.053431+0000","last_active":"2024-07-31T14:47:20.447013+0000","last_peered":"2024-07-31T14:47:20.447013+0000","last_clean":"2024-07-31T14:47:20.447013+0000","last_became_active":"2024-07-31T14:47:03.052958+0000","last_became_peered":"2024-07-31T14:47:03.052958+0000","last_unstale":"2024-07-31T14:47:20.447013+0000","last_undegraded":"2024-07-31T14:47:20.447013+0000","last_fullsized":"2024-07-31T14:47:20.447013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:03:28.863589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447102+0000","last_change":"2024-07-31T14:47:03.046641+0000","last_active":"2024-07-31T14:47:20.447102+0000","last_peered":"2024-07-31T14:47:20.447102+0000","last_clean":"2024-07-31T14:47:20.447102+0000","last_became_active":"2024-07-31T14:47:03.046265+0000","last_became_peered":"2024-07-31T14:47:03.046265+0000","last_unstale":"2024-07-31T14:47:20.447102+0000","last_undegraded":"2024-07-31T14:47:20.447102+0000","last_fullsized":"2024-07-31T14:47:20.447102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:20:25.389010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444399+0000","last_change":"2024-07-31T14:47:03.051657+0000","last_active":"2024-07-31T14:47:20.444399+0000","last_peered":"2024-07-31T14:47:20.444399+0000","last_clean":"2024-07-31T14:47:20.444399+0000","last_became_active":"2024-07-31T14:47:03.050722+0000","last_became_peered":"2024-07-31T14:47:03.050722+0000","last_unstale":"2024-07-31T14:47:20.444399+0000","last_undegraded":"2024-07-31T14:47:20.444399+0000","last_fullsized":"2024-07-31T14:47:20.444399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:23:19.387470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447245+0000","last_change":"2024-07-31T14:47:03.059654+0000","last_active":"2024-07-31T14:47:20.447245+0000","last_peered":"2024-07-31T14:47:20.447245+0000","last_clean":"2024-07-31T14:47:20.447245+0000","last_became_active":"2024-07-31T14:47:03.056843+0000","last_became_peered":"2024-07-31T14:47:03.056843+0000","last_unstale":"2024-07-31T14:47:20.447245+0000","last_undegraded":"2024-07-31T14:47:20.447245+0000","last_fullsized":"2024-07-31T14:47:20.447245+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:42:33.696498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625741+0000","last_change":"2024-07-31T14:47:03.052750+0000","last_active":"2024-07-31T14:47:20.625741+0000","last_peered":"2024-07-31T14:47:20.625741+0000","last_clean":"2024-07-31T14:47:20.625741+0000","last_became_active":"2024-07-31T14:47:03.051744+0000","last_became_peered":"2024-07-31T14:47:03.051744+0000","last_unstale":"2024-07-31T14:47:20.625741+0000","last_undegraded":"2024-07-31T14:47:20.625741+0000","last_fullsized":"2024-07-31T14:47:20.625741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:24:33.202918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.452621+0000","last_change":"2024-07-31T14:47:20.452563+0000","last_active":"2024-07-31T14:47:20.452621+0000","last_peered":"2024-07-31T14:47:20.452621+0000","last_clean":"2024-07-31T14:47:20.452621+0000","last_became_active":"2024-07-31T14:47:20.452271+0000","last_became_peered":"2024-07-31T14:47:20.452271+0000","last_unstale":"2024-07-31T14:47:20.452621+0000","last_undegraded":"2024-07-31T14:47:20.452621+0000","last_fullsized":"2024-07-31T14:47:20.452621+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:26:55.070265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.448941+0000","last_change":"2024-07-31T14:47:03.050691+0000","last_active":"2024-07-31T14:47:20.448941+0000","last_peered":"2024-07-31T14:47:20.448941+0000","last_clean":"2024-07-31T14:47:20.448941+0000","last_became_active":"2024-07-31T14:47:03.050320+0000","last_became_peered":"2024-07-31T14:47:03.050320+0000","last_unstale":"2024-07-31T14:47:20.448941+0000","last_undegraded":"2024-07-31T14:47:20.448941+0000","last_fullsized":"2024-07-31T14:47:20.448941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:40:45.767706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447344+0000","last_change":"2024-07-31T14:47:03.048491+0000","last_active":"2024-07-31T14:47:20.447344+0000","last_peered":"2024-07-31T14:47:20.447344+0000","last_clean":"2024-07-31T14:47:20.447344+0000","last_became_active":"2024-07-31T14:47:03.048085+0000","last_became_peered":"2024-07-31T14:47:03.048085+0000","last_unstale":"2024-07-31T14:47:20.447344+0000","last_undegraded":"2024-07-31T14:47:20.447344+0000","last_fullsized":"2024-07-31T14:47:20.447344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:35:01.762365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604826+0000","last_change":"2024-07-31T14:47:03.055502+0000","last_active":"2024-07-31T14:47:20.604826+0000","last_peered":"2024-07-31T14:47:20.604826+0000","last_clean":"2024-07-31T14:47:20.604826+0000","last_became_active":"2024-07-31T14:47:03.055015+0000","last_became_peered":"2024-07-31T14:47:03.055015+0000","last_unstale":"2024-07-31T14:47:20.604826+0000","last_undegraded":"2024-07-31T14:47:20.604826+0000","last_fullsized":"2024-07-31T14:47:20.604826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:50:39.605148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444781+0000","last_change":"2024-07-31T14:47:03.057808+0000","last_active":"2024-07-31T14:47:20.444781+0000","last_peered":"2024-07-31T14:47:20.444781+0000","last_clean":"2024-07-31T14:47:20.444781+0000","last_became_active":"2024-07-31T14:47:03.057443+0000","last_became_peered":"2024-07-31T14:47:03.057443+0000","last_unstale":"2024-07-31T14:47:20.444781+0000","last_undegraded":"2024-07-31T14:47:20.444781+0000","last_fullsized":"2024-07-31T14:47:20.444781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:34:42.915428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.628024+0000","last_change":"2024-07-31T14:47:03.058182+0000","last_active":"2024-07-31T14:47:20.628024+0000","last_peered":"2024-07-31T14:47:20.628024+0000","last_clean":"2024-07-31T14:47:20.628024+0000","last_became_active":"2024-07-31T14:47:03.057802+0000","last_became_peered":"2024-07-31T14:47:03.057802+0000","last_unstale":"2024-07-31T14:47:20.628024+0000","last_undegraded":"2024-07-31T14:47:20.628024+0000","last_fullsized":"2024-07-31T14:47:20.628024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:08:06.608788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626405+0000","last_change":"2024-07-31T14:47:03.058882+0000","last_active":"2024-07-31T14:47:20.626405+0000","last_peered":"2024-07-31T14:47:20.626405+0000","last_clean":"2024-07-31T14:47:20.626405+0000","last_became_active":"2024-07-31T14:47:03.058530+0000","last_became_peered":"2024-07-31T14:47:03.058530+0000","last_unstale":"2024-07-31T14:47:20.626405+0000","last_undegraded":"2024-07-31T14:47:20.626405+0000","last_fullsized":"2024-07-31T14:47:20.626405+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:28:03.631962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625927+0000","last_change":"2024-07-31T14:47:03.052028+0000","last_active":"2024-07-31T14:47:20.625927+0000","last_peered":"2024-07-31T14:47:20.625927+0000","last_clean":"2024-07-31T14:47:20.625927+0000","last_became_active":"2024-07-31T14:47:03.051559+0000","last_became_peered":"2024-07-31T14:47:03.051559+0000","last_unstale":"2024-07-31T14:47:20.625927+0000","last_undegraded":"2024-07-31T14:47:20.625927+0000","last_fullsized":"2024-07-31T14:47:20.625927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:16:40.723795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444190+0000","last_change":"2024-07-31T14:47:03.053884+0000","last_active":"2024-07-31T14:47:20.444190+0000","last_peered":"2024-07-31T14:47:20.444190+0000","last_clean":"2024-07-31T14:47:20.444190+0000","last_became_active":"2024-07-31T14:47:03.053521+0000","last_became_peered":"2024-07-31T14:47:03.053521+0000","last_unstale":"2024-07-31T14:47:20.444190+0000","last_undegraded":"2024-07-31T14:47:20.444190+0000","last_fullsized":"2024-07-31T14:47:20.444190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:16:37.702055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627119+0000","last_change":"2024-07-31T14:47:03.058588+0000","last_active":"2024-07-31T14:47:20.627119+0000","last_peered":"2024-07-31T14:47:20.627119+0000","last_clean":"2024-07-31T14:47:20.627119+0000","last_became_active":"2024-07-31T14:47:03.057820+0000","last_became_peered":"2024-07-31T14:47:03.057820+0000","last_unstale":"2024-07-31T14:47:20.627119+0000","last_undegraded":"2024-07-31T14:47:20.627119+0000","last_fullsized":"2024-07-31T14:47:20.627119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:21.265208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451188+0000","last_change":"2024-07-31T14:47:03.041447+0000","last_active":"2024-07-31T14:47:20.451188+0000","last_peered":"2024-07-31T14:47:20.451188+0000","last_clean":"2024-07-31T14:47:20.451188+0000","last_became_active":"2024-07-31T14:47:03.040935+0000","last_became_peered":"2024-07-31T14:47:03.040935+0000","last_unstale":"2024-07-31T14:47:20.451188+0000","last_undegraded":"2024-07-31T14:47:20.451188+0000","last_fullsized":"2024-07-31T14:47:20.451188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:00:13.624698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.452719+0000","last_change":"2024-07-31T14:47:20.452239+0000","last_active":"2024-07-31T14:47:20.452719+0000","last_peered":"2024-07-31T14:47:20.452719+0000","last_clean":"2024-07-31T14:47:20.452719+0000","last_became_active":"2024-07-31T14:47:20.451924+0000","last_became_peered":"2024-07-31T14:47:20.451924+0000","last_unstale":"2024-07-31T14:47:20.452719+0000","last_undegraded":"2024-07-31T14:47:20.452719+0000","last_fullsized":"2024-07-31T14:47:20.452719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:58:24.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449119+0000","last_change":"2024-07-31T14:47:03.056024+0000","last_active":"2024-07-31T14:47:20.449119+0000","last_peered":"2024-07-31T14:47:20.449119+0000","last_clean":"2024-07-31T14:47:20.449119+0000","last_became_active":"2024-07-31T14:47:03.055630+0000","last_became_peered":"2024-07-31T14:47:03.055630+0000","last_unstale":"2024-07-31T14:47:20.449119+0000","last_undegraded":"2024-07-31T14:47:20.449119+0000","last_fullsized":"2024-07-31T14:47:20.449119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:09:11.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449066+0000","last_change":"2024-07-31T14:47:03.053718+0000","last_active":"2024-07-31T14:47:20.449066+0000","last_peered":"2024-07-31T14:47:20.449066+0000","last_clean":"2024-07-31T14:47:20.449066+0000","last_became_active":"2024-07-31T14:47:03.053361+0000","last_became_peered":"2024-07-31T14:47:03.053361+0000","last_unstale":"2024-07-31T14:47:20.449066+0000","last_undegraded":"2024-07-31T14:47:20.449066+0000","last_fullsized":"2024-07-31T14:47:20.449066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:03:49.220838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447132+0000","last_change":"2024-07-31T14:47:03.056087+0000","last_active":"2024-07-31T14:47:20.447132+0000","last_peered":"2024-07-31T14:47:20.447132+0000","last_clean":"2024-07-31T14:47:20.447132+0000","last_became_active":"2024-07-31T14:47:03.055732+0000","last_became_peered":"2024-07-31T14:47:03.055732+0000","last_unstale":"2024-07-31T14:47:20.447132+0000","last_undegraded":"2024-07-31T14:47:20.447132+0000","last_fullsized":"2024-07-31T14:47:20.447132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:45:01.185666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65478,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":294,"num_read_kb":222,"num_write":245,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":196434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645343,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":519730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841572,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37008,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708144,"statfs":{"total":95995035648,"available":95957139456,"internally_reserved":0,"allocated":1712128,"data_stored":1164820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005097,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":523756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201326,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":520038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397554,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1117317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561080,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":515555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1112117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":519629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117062,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":514872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378699,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":574688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542224,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":574468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":516620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":60010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-31T14:48:33.081 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-31T14:48:33.082 INFO:tasks.cephadm:Setting up client nodes... 2024-07-31T14:48:33.082 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-07-31T14:48:33.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:32 smithi073 ceph-mon[36714]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-07-31T14:48:33.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:32 smithi077 ceph-mon[37327]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-07-31T14:48:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:34.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:33 smithi067 ceph-mon[32750]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:48:34.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:33 smithi073 ceph-mon[36714]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:48:34.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:33 smithi077 ceph-mon[37327]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:48:34.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:34 smithi067 ceph-mon[32750]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-07-31T14:48:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.091+0000 7f43ed8a8640 1 -- 172.21.15.67:0/352297353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43e8107570 msgr2=0x7f43e8107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.091+0000 7f43ed8a8640 1 --2- 172.21.15.67:0/352297353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43e8107570 0x7f43e8107980 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009900 tx=0x7f43d002a410 comp rx=0 tx=0).stop 2024-07-31T14:48:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.091+0000 7f43ed8a8640 1 -- 172.21.15.67:0/352297353 shutdown_connections 2024-07-31T14:48:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.091+0000 7f43ed8a8640 1 --2- 172.21.15.67:0/352297353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43e8107ec0 0x7f43e810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.091+0000 7f43ed8a8640 1 --2- 172.21.15.67:0/352297353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43e8107570 0x7f43e8107980 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.091+0000 7f43ed8a8640 1 --2- 172.21.15.67:0/352297353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e8101c30 0x7f43e8102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.091+0000 7f43ed8a8640 1 -- 172.21.15.67:0/352297353 >> 172.21.15.67:0/352297353 conn(0x7f43e8079980 msgr2=0x7f43e8079d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.092+0000 7f43ed8a8640 1 -- 172.21.15.67:0/352297353 shutdown_connections 2024-07-31T14:48:35.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.092+0000 7f43ed8a8640 1 -- 172.21.15.67:0/352297353 wait complete. 2024-07-31T14:48:35.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.092+0000 7f43ed8a8640 1 Processor -- start 2024-07-31T14:48:35.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.092+0000 7f43ed8a8640 1 -- start start 2024-07-31T14:48:35.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43ed8a8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43e8101c30 0x7f43e8199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:35.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43ed8a8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43e8107570 0x7f43e819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43ed8a8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e8107ec0 0x7f43e819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43ed8a8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8111300 con 0x7f43e8101c30 2024-07-31T14:48:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43ed8a8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8111180 con 0x7f43e8107570 2024-07-31T14:48:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43ed8a8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8111480 con 0x7f43e8107ec0 2024-07-31T14:48:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43e6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43e8101c30 0x7f43e8199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43e6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43e8101c30 0x7f43e8199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46092/0 (socket says 172.21.15.67:46092) 2024-07-31T14:48:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43e6ffd640 1 -- 172.21.15.67:0/3285738595 learned_addr learned my addr 172.21.15.67:0/3285738595 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43e77fe640 1 --2- 172.21.15.67:0/3285738595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e8107ec0 0x7f43e819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43dffff640 1 --2- 172.21.15.67:0/3285738595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43e8107570 0x7f43e819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43e6ffd640 1 -- 172.21.15.67:0/3285738595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e8107ec0 msgr2=0x7f43e819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43e6ffd640 1 --2- 172.21.15.67:0/3285738595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e8107ec0 0x7f43e819f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43e6ffd640 1 -- 172.21.15.67:0/3285738595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43e8107570 msgr2=0x7f43e819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43e6ffd640 1 --2- 172.21.15.67:0/3285738595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43e8107570 0x7f43e819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:35.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.093+0000 7f43e6ffd640 1 -- 172.21.15.67:0/3285738595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e819fe70 con 0x7f43e8101c30 2024-07-31T14:48:35.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.094+0000 7f43e6ffd640 1 --2- 172.21.15.67:0/3285738595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43e8101c30 0x7f43e8199f00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f43d800c970 tx=0x7f43d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:35.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.094+0000 7f43ec8a6640 1 -- 172.21.15.67:0/3285738595 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8007bb0 con 0x7f43e8101c30 2024-07-31T14:48:35.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.094+0000 7f43ed8a8640 1 -- 172.21.15.67:0/3285738595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e81a0160 con 0x7f43e8101c30 2024-07-31T14:48:35.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.094+0000 7f43ec8a6640 1 -- 172.21.15.67:0/3285738595 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d8007d50 con 0x7f43e8101c30 2024-07-31T14:48:35.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.094+0000 7f43ec8a6640 1 -- 172.21.15.67:0/3285738595 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d800f450 con 0x7f43e8101c30 2024-07-31T14:48:35.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.094+0000 7f43ed8a8640 1 -- 172.21.15.67:0/3285738595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e81a68c0 con 0x7f43e8101c30 2024-07-31T14:48:35.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.095+0000 7f43ed8a8640 1 -- 172.21.15.67:0/3285738595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e8102040 con 0x7f43e8101c30 2024-07-31T14:48:35.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.096+0000 7f43ec8a6640 1 -- 172.21.15.67:0/3285738595 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f43d8004f20 con 0x7f43e8101c30 2024-07-31T14:48:35.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.097+0000 7f43ec8a6640 1 --2- 172.21.15.67:0/3285738595 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f43d40b3cb0 0x7f43d40b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:35.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.097+0000 7f43dffff640 1 --2- 172.21.15.67:0/3285738595 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f43d40b3cb0 0x7f43d40b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:35.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.097+0000 7f43ec8a6640 1 -- 172.21.15.67:0/3285738595 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f43d80e3ea0 con 0x7f43e8101c30 2024-07-31T14:48:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.098+0000 7f43dffff640 1 --2- 172.21.15.67:0/3285738595 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f43d40b3cb0 0x7f43d40b61c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009230 tx=0x7f43d0031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:35.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.101+0000 7f43ec8a6640 1 -- 172.21.15.67:0/3285738595 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f43d8014030 con 0x7f43e8101c30 2024-07-31T14:48:35.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:34 smithi073 ceph-mon[36714]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-07-31T14:48:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:34 smithi077 ceph-mon[37327]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-07-31T14:48:35.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.370+0000 7f43ed8a8640 1 -- 172.21.15.67:0/3285738595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f43e802e0c0 con 0x7f43e8101c30 2024-07-31T14:48:35.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.373+0000 7f43ec8a6640 1 -- 172.21.15.67:0/3285738595 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f43d80aa910 con 0x7f43e8101c30 2024-07-31T14:48:35.375 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-07-31T14:48:35.375 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDDTqpmHxEwFhAAr9G03HWTlYwaIvQrqlz30Q== 2024-07-31T14:48:35.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.375+0000 7f43ed8a8640 1 -- 172.21.15.67:0/3285738595 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f43d40b3cb0 msgr2=0x7f43d40b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:35.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.375+0000 7f43ed8a8640 1 --2- 172.21.15.67:0/3285738595 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f43d40b3cb0 0x7f43d40b61c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009230 tx=0x7f43d0031400 comp rx=0 tx=0).stop 2024-07-31T14:48:35.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.375+0000 7f43ed8a8640 1 -- 172.21.15.67:0/3285738595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43e8101c30 msgr2=0x7f43e8199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:35.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.375+0000 7f43ed8a8640 1 --2- 172.21.15.67:0/3285738595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43e8101c30 0x7f43e8199f00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f43d800c970 tx=0x7f43d800ce30 comp rx=0 tx=0).stop 2024-07-31T14:48:35.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.375+0000 7f43ed8a8640 1 -- 172.21.15.67:0/3285738595 shutdown_connections 2024-07-31T14:48:35.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.375+0000 7f43ed8a8640 1 --2- 172.21.15.67:0/3285738595 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f43d40b3cb0 0x7f43d40b61c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:35.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.375+0000 7f43ed8a8640 1 --2- 172.21.15.67:0/3285738595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e8107ec0 0x7f43e819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:35.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.375+0000 7f43ed8a8640 1 --2- 172.21.15.67:0/3285738595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43e8107570 0x7f43e819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:35.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.375+0000 7f43ed8a8640 1 --2- 172.21.15.67:0/3285738595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43e8101c30 0x7f43e8199f00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:35.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.375+0000 7f43ed8a8640 1 -- 172.21.15.67:0/3285738595 >> 172.21.15.67:0/3285738595 conn(0x7f43e8079980 msgr2=0x7f43e8103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:35.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.376+0000 7f43ed8a8640 1 -- 172.21.15.67:0/3285738595 shutdown_connections 2024-07-31T14:48:35.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:35.376+0000 7f43ed8a8640 1 -- 172.21.15.67:0/3285738595 wait complete. 2024-07-31T14:48:36.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:35 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3285738595' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-31T14:48:36.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:35 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3285738595' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-31T14:48:36.118 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:48:36.118 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-07-31T14:48:36.118 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-07-31T14:48:36.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:35 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3285738595' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-31T14:48:36.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:35 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3285738595' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-31T14:48:36.161 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph config log 1 --format=json 2024-07-31T14:48:36.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:35 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3285738595' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-31T14:48:36.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:35 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3285738595' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-31T14:48:36.470 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:36.901 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:36 smithi067 ceph-mon[32750]: pgmap v325: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-07-31T14:48:37.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:36 smithi073 ceph-mon[36714]: pgmap v325: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-07-31T14:48:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:36 smithi077 ceph-mon[37327]: pgmap v325: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-07-31T14:48:38.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.327+0000 7f1247a52640 1 -- 172.21.15.67:0/3090232355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240108ed0 msgr2=0x7f12401092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.327+0000 7f1247a52640 1 --2- 172.21.15.67:0/3090232355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240108ed0 0x7f12401092e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1230009900 tx=0x7f123002a410 comp rx=0 tx=0).stop 2024-07-31T14:48:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.327+0000 7f1247a52640 1 -- 172.21.15.67:0/3090232355 shutdown_connections 2024-07-31T14:48:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.327+0000 7f1247a52640 1 --2- 172.21.15.67:0/3090232355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1240109820 0x7f1240110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.327+0000 7f1247a52640 1 --2- 172.21.15.67:0/3090232355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240108ed0 0x7f12401092e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.327+0000 7f1247a52640 1 --2- 172.21.15.67:0/3090232355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1240103590 0x7f12401039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.327+0000 7f1247a52640 1 -- 172.21.15.67:0/3090232355 >> 172.21.15.67:0/3090232355 conn(0x7f124007b280 msgr2=0x7f124007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.327+0000 7f1247a52640 1 -- 172.21.15.67:0/3090232355 shutdown_connections 2024-07-31T14:48:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.327+0000 7f1247a52640 1 -- 172.21.15.67:0/3090232355 wait complete. 2024-07-31T14:48:38.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.328+0000 7f1247a52640 1 Processor -- start 2024-07-31T14:48:38.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.328+0000 7f1247a52640 1 -- start start 2024-07-31T14:48:38.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.328+0000 7f1247a52640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240103590 0x7f124019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:38.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.328+0000 7f1247a52640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1240108ed0 0x7f124019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:38.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.328+0000 7f1247a52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1240109820 0x7f124019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:38.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.328+0000 7f1247a52640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1240113580 con 0x7f1240103590 2024-07-31T14:48:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.328+0000 7f1247a52640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1240113400 con 0x7f1240109820 2024-07-31T14:48:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.328+0000 7f1247a52640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1240113700 con 0x7f1240108ed0 2024-07-31T14:48:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.328+0000 7f12457ee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240103590 0x7f124019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f12457ee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240103590 0x7f124019a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46114/0 (socket says 172.21.15.67:46114) 2024-07-31T14:48:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f12457ee640 1 -- 172.21.15.67:0/293668579 learned_addr learned my addr 172.21.15.67:0/293668579 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f1244fed640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1240108ed0 0x7f124019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f12457ee640 1 -- 172.21.15.67:0/293668579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1240108ed0 msgr2=0x7f124019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f1245fef640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1240109820 0x7f124019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f12457ee640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1240108ed0 0x7f124019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f12457ee640 1 -- 172.21.15.67:0/293668579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1240109820 msgr2=0x7f124019f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f12457ee640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1240109820 0x7f124019f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f12457ee640 1 -- 172.21.15.67:0/293668579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124019ffc0 con 0x7f1240103590 2024-07-31T14:48:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f1244fed640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1240108ed0 0x7f124019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:48:38.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f12457ee640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240103590 0x7f124019a050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f123c00b4a0 tx=0x7f123c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:38.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f12367fc640 1 -- 172.21.15.67:0/293668579 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c014070 con 0x7f1240103590 2024-07-31T14:48:38.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f12367fc640 1 -- 172.21.15.67:0/293668579 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123c010ac0 con 0x7f1240103590 2024-07-31T14:48:38.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f1247a52640 1 -- 172.21.15.67:0/293668579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12401a02b0 con 0x7f1240103590 2024-07-31T14:48:38.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f12367fc640 1 -- 172.21.15.67:0/293668579 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c017c30 con 0x7f1240103590 2024-07-31T14:48:38.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.329+0000 7f1247a52640 1 -- 172.21.15.67:0/293668579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12401a6a10 con 0x7f1240103590 2024-07-31T14:48:38.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.330+0000 7f1247a52640 1 -- 172.21.15.67:0/293668579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1240103a20 con 0x7f1240103590 2024-07-31T14:48:38.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.333+0000 7f12367fc640 1 -- 172.21.15.67:0/293668579 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f123c017070 con 0x7f1240103590 2024-07-31T14:48:38.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.334+0000 7f12367fc640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f122c0b3cb0 0x7f122c0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:38.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.334+0000 7f1244fed640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f122c0b3cb0 0x7f122c0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:38.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.334+0000 7f12367fc640 1 -- 172.21.15.67:0/293668579 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f123c0e3a20 con 0x7f1240103590 2024-07-31T14:48:38.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.334+0000 7f1244fed640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f122c0b3cb0 0x7f122c0b61c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1230000c00 tx=0x7f1230031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:38.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.337+0000 7f12367fc640 1 -- 172.21.15.67:0/293668579 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f123c0aabf0 con 0x7f1240103590 2024-07-31T14:48:38.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.530+0000 7f1247a52640 1 -- 172.21.15.67:0/293668579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f124019b180 con 0x7f1240103590 2024-07-31T14:48:38.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.531+0000 7f12367fc640 1 -- 172.21.15.67:0/293668579 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v54) ==== 86+0+133 (secure 0 0 0) 0x7f123c0aa4f0 con 0x7f1240103590 2024-07-31T14:48:38.533 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:48:38.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.533+0000 7f1247a52640 1 -- 172.21.15.67:0/293668579 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f122c0b3cb0 msgr2=0x7f122c0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:38.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.533+0000 7f1247a52640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f122c0b3cb0 0x7f122c0b61c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1230000c00 tx=0x7f1230031400 comp rx=0 tx=0).stop 2024-07-31T14:48:38.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.533+0000 7f1247a52640 1 -- 172.21.15.67:0/293668579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240103590 msgr2=0x7f124019a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:38.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.533+0000 7f1247a52640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240103590 0x7f124019a050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f123c00b4a0 tx=0x7f123c00b960 comp rx=0 tx=0).stop 2024-07-31T14:48:38.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.534+0000 7f1247a52640 1 -- 172.21.15.67:0/293668579 shutdown_connections 2024-07-31T14:48:38.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.534+0000 7f1247a52640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f122c0b3cb0 0x7f122c0b61c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:38.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.534+0000 7f1247a52640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1240109820 0x7f124019f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:38.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.534+0000 7f1247a52640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1240108ed0 0x7f124019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.534+0000 7f1247a52640 1 --2- 172.21.15.67:0/293668579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240103590 0x7f124019a050 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.534+0000 7f1247a52640 1 -- 172.21.15.67:0/293668579 >> 172.21.15.67:0/293668579 conn(0x7f124007b280 msgr2=0x7f1240107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.534+0000 7f1247a52640 1 -- 172.21.15.67:0/293668579 shutdown_connections 2024-07-31T14:48:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:38.534+0000 7f1247a52640 1 -- 172.21.15.67:0/293668579 wait complete. 2024-07-31T14:48:39.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:38 smithi073 ceph-mon[36714]: pgmap v326: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2024-07-31T14:48:39.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:38 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/293668579' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-31T14:48:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:38 smithi077 ceph-mon[37327]: pgmap v326: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2024-07-31T14:48:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:38 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/293668579' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-31T14:48:39.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:38 smithi067 ceph-mon[32750]: pgmap v326: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2024-07-31T14:48:39.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:38 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/293668579' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-31T14:48:39.328 INFO:teuthology.orchestra.run.smithi067.stdout:[{"version":54,"timestamp":"2024-07-31T14:46:50.968440+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-07-31T14:48:39.328 INFO:tasks.ceph_manager:config epoch is 54 2024-07-31T14:48:39.328 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-07-31T14:48:39.329 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-07-31T14:48:39.329 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph mgr dump --format=json 2024-07-31T14:48:39.595 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:41.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:40 smithi067 ceph-mon[32750]: pgmap v327: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2024-07-31T14:48:41.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:40 smithi073 ceph-mon[36714]: pgmap v327: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2024-07-31T14:48:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:40 smithi077 ceph-mon[37327]: pgmap v327: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2024-07-31T14:48:41.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.287+0000 7fb0faa92640 1 -- 172.21.15.67:0/2555445127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f4108760 msgr2=0x7fb0f410ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:41.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.287+0000 7fb0faa92640 1 --2- 172.21.15.67:0/2555445127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f4108760 0x7fb0f410ef90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009ca0 tx=0x7fb0ec02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:48:41.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.287+0000 7fb0faa92640 1 -- 172.21.15.67:0/2555445127 shutdown_connections 2024-07-31T14:48:41.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.287+0000 7fb0faa92640 1 --2- 172.21.15.67:0/2555445127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f4108760 0x7fb0f410ef90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:41.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.287+0000 7fb0faa92640 1 --2- 172.21.15.67:0/2555445127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0f4107e10 0x7fb0f4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:41.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.287+0000 7fb0faa92640 1 --2- 172.21.15.67:0/2555445127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0f41024d0 0x7fb0f41028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:41.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.287+0000 7fb0faa92640 1 -- 172.21.15.67:0/2555445127 >> 172.21.15.67:0/2555445127 conn(0x7fb0f4079190 msgr2=0x7fb0f40795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:41.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.287+0000 7fb0faa92640 1 -- 172.21.15.67:0/2555445127 shutdown_connections 2024-07-31T14:48:41.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.287+0000 7fb0faa92640 1 -- 172.21.15.67:0/2555445127 wait complete. 2024-07-31T14:48:41.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.288+0000 7fb0faa92640 1 Processor -- start 2024-07-31T14:48:41.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.288+0000 7fb0faa92640 1 -- start start 2024-07-31T14:48:41.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.288+0000 7fb0faa92640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0f41024d0 0x7fb0f419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:41.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.288+0000 7fb0faa92640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f4107e10 0x7fb0f419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:41.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.288+0000 7fb0faa92640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0f4108760 0x7fb0f419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:41.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.288+0000 7fb0faa92640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f41123c0 con 0x7fb0f4107e10 2024-07-31T14:48:41.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.288+0000 7fb0faa92640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f4112240 con 0x7fb0f4108760 2024-07-31T14:48:41.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.288+0000 7fb0faa92640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f4112540 con 0x7fb0f41024d0 2024-07-31T14:48:41.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f4107e10 0x7fb0f419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:41.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f4107e10 0x7fb0f419b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39426/0 (socket says 172.21.15.67:39426) 2024-07-31T14:48:41.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f3fff640 1 -- 172.21.15.67:0/147564978 learned_addr learned my addr 172.21.15.67:0/147564978 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:41.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f882e640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0f41024d0 0x7fb0f419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:41.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f902f640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0f4108760 0x7fb0f419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:41.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f3fff640 1 -- 172.21.15.67:0/147564978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0f41024d0 msgr2=0x7fb0f419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:41.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f3fff640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0f41024d0 0x7fb0f419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:41.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f3fff640 1 -- 172.21.15.67:0/147564978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0f4108760 msgr2=0x7fb0f419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:41.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f3fff640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0f4108760 0x7fb0f419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:41.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f3fff640 1 -- 172.21.15.67:0/147564978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f41a6000 con 0x7fb0f4107e10 2024-07-31T14:48:41.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f902f640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0f4108760 0x7fb0f419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:48:41.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f3fff640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f4107e10 0x7fb0f419b2b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0005fd0 tx=0x7fb0e000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:41.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f1ffb640 1 -- 172.21.15.67:0/147564978 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0017070 con 0x7fb0f4107e10 2024-07-31T14:48:41.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0faa92640 1 -- 172.21.15.67:0/147564978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f41a62f0 con 0x7fb0f4107e10 2024-07-31T14:48:41.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f1ffb640 1 -- 172.21.15.67:0/147564978 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e0009c30 con 0x7fb0f4107e10 2024-07-31T14:48:41.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.289+0000 7fb0f1ffb640 1 -- 172.21.15.67:0/147564978 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0003d30 con 0x7fb0f4107e10 2024-07-31T14:48:41.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.290+0000 7fb0faa92640 1 -- 172.21.15.67:0/147564978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f41a6830 con 0x7fb0f4107e10 2024-07-31T14:48:41.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.291+0000 7fb0faa92640 1 -- 172.21.15.67:0/147564978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f4103750 con 0x7fb0f4107e10 2024-07-31T14:48:41.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.292+0000 7fb0f1ffb640 1 -- 172.21.15.67:0/147564978 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb0e0003ed0 con 0x7fb0f4107e10 2024-07-31T14:48:41.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.293+0000 7fb0f1ffb640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb0e40b3ce0 0x7fb0e40b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:41.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.293+0000 7fb0f882e640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb0e40b3ce0 0x7fb0e40b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:41.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.293+0000 7fb0f1ffb640 1 -- 172.21.15.67:0/147564978 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fb0e00e4660 con 0x7fb0f4107e10 2024-07-31T14:48:41.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.294+0000 7fb0f882e640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb0e40b3ce0 0x7fb0e40b61f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005fd0 tx=0x7fb0e8005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:41.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.296+0000 7fb0f1ffb640 1 -- 172.21.15.67:0/147564978 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb0e0020030 con 0x7fb0f4107e10 2024-07-31T14:48:41.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.547+0000 7fb0faa92640 1 -- 172.21.15.67:0/147564978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb0f41028e0 con 0x7fb0f4107e10 2024-07-31T14:48:41.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.555+0000 7fb0f1ffb640 1 -- 172.21.15.67:0/147564978 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+291034 (secure 0 0 0) 0x7fb0e00ab720 con 0x7fb0f4107e10 2024-07-31T14:48:41.559 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:48:41.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.558+0000 7fb0faa92640 1 -- 172.21.15.67:0/147564978 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb0e40b3ce0 msgr2=0x7fb0e40b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:41.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.558+0000 7fb0faa92640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb0e40b3ce0 0x7fb0e40b61f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005fd0 tx=0x7fb0e8005850 comp rx=0 tx=0).stop 2024-07-31T14:48:41.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.558+0000 7fb0faa92640 1 -- 172.21.15.67:0/147564978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f4107e10 msgr2=0x7fb0f419b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:41.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.558+0000 7fb0faa92640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f4107e10 0x7fb0f419b2b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0005fd0 tx=0x7fb0e000dde0 comp rx=0 tx=0).stop 2024-07-31T14:48:41.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.558+0000 7fb0faa92640 1 -- 172.21.15.67:0/147564978 shutdown_connections 2024-07-31T14:48:41.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.558+0000 7fb0faa92640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb0e40b3ce0 0x7fb0e40b61f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:41.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.558+0000 7fb0faa92640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0f4108760 0x7fb0f419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:41.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.558+0000 7fb0faa92640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f4107e10 0x7fb0f419b2b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:41.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.558+0000 7fb0faa92640 1 --2- 172.21.15.67:0/147564978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0f41024d0 0x7fb0f419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:41.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.558+0000 7fb0faa92640 1 -- 172.21.15.67:0/147564978 >> 172.21.15.67:0/147564978 conn(0x7fb0f4079190 msgr2=0x7fb0f4103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:41.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.558+0000 7fb0faa92640 1 -- 172.21.15.67:0/147564978 shutdown_connections 2024-07-31T14:48:41.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:41.558+0000 7fb0faa92640 1 -- 172.21.15.67:0/147564978 wait complete. 2024-07-31T14:48:42.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:41 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/147564978' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-31T14:48:42.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:41 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/147564978' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-31T14:48:42.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:41 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/147564978' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-31T14:48:42.378 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"flags":0,"active_gid":14214,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2637562983},{"type":"v1","addr":"172.21.15.67:6801","nonce":2637562983}]},"active_addr":"172.21.15.67:6801/2637562983","active_change":"2024-07-31T14:40:19.281908+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24145,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24146,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.67:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":746517180}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":370543409}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":1316333158}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":3221800371}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":2120986485}]}]} 2024-07-31T14:48:42.385 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-07-31T14:48:42.385 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-07-31T14:48:42.385 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-07-31T14:48:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:43.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:42 smithi067 ceph-mon[32750]: pgmap v328: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-07-31T14:48:43.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:42 smithi073 ceph-mon[36714]: pgmap v328: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-07-31T14:48:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:42 smithi077 ceph-mon[37327]: pgmap v328: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-07-31T14:48:44.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.314+0000 7f2f7134d640 1 -- 172.21.15.67:0/535385790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f6c103590 msgr2=0x7f2f6c1039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:44.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.314+0000 7f2f7134d640 1 --2- 172.21.15.67:0/535385790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f6c103590 0x7f2f6c1039a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009900 tx=0x7f2f5c02a410 comp rx=0 tx=0).stop 2024-07-31T14:48:44.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.314+0000 7f2f7134d640 1 -- 172.21.15.67:0/535385790 shutdown_connections 2024-07-31T14:48:44.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.314+0000 7f2f7134d640 1 --2- 172.21.15.67:0/535385790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c109820 0x7f2f6c110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:44.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.314+0000 7f2f7134d640 1 --2- 172.21.15.67:0/535385790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f6c108ed0 0x7f2f6c1092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:44.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.314+0000 7f2f7134d640 1 --2- 172.21.15.67:0/535385790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f6c103590 0x7f2f6c1039a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:44.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.314+0000 7f2f7134d640 1 -- 172.21.15.67:0/535385790 >> 172.21.15.67:0/535385790 conn(0x7f2f6c07b280 msgr2=0x7f2f6c07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:44.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.314+0000 7f2f7134d640 1 -- 172.21.15.67:0/535385790 shutdown_connections 2024-07-31T14:48:44.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.314+0000 7f2f7134d640 1 -- 172.21.15.67:0/535385790 wait complete. 2024-07-31T14:48:44.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.315+0000 7f2f7134d640 1 Processor -- start 2024-07-31T14:48:44.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.315+0000 7f2f7134d640 1 -- start start 2024-07-31T14:48:44.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.315+0000 7f2f7134d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f6c103590 0x7f2f6c075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.315+0000 7f2f7134d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c108ed0 0x7f2f6c075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.315+0000 7f2f7134d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f6c109820 0x7f2f6c075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.315+0000 7f2f7134d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c113540 con 0x7f2f6c103590 2024-07-31T14:48:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.315+0000 7f2f7134d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c1133c0 con 0x7f2f6c109820 2024-07-31T14:48:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.315+0000 7f2f7134d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c1136c0 con 0x7f2f6c108ed0 2024-07-31T14:48:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f6affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f6c103590 0x7f2f6c075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f6affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f6c103590 0x7f2f6c075140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39440/0 (socket says 172.21.15.67:39440) 2024-07-31T14:48:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f6affd640 1 -- 172.21.15.67:0/3427356150 learned_addr learned my addr 172.21.15.67:0/3427356150 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f6a7fc640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c108ed0 0x7f2f6c075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f6b7fe640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f6c109820 0x7f2f6c075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f6affd640 1 -- 172.21.15.67:0/3427356150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c108ed0 msgr2=0x7f2f6c075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f6affd640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c108ed0 0x7f2f6c075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f6affd640 1 -- 172.21.15.67:0/3427356150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f6c109820 msgr2=0x7f2f6c075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:44.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f6affd640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f6c109820 0x7f2f6c075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:44.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f6affd640 1 -- 172.21.15.67:0/3427356150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c1a8290 con 0x7f2f6c103590 2024-07-31T14:48:44.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f6a7fc640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c108ed0 0x7f2f6c075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:48:44.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f6affd640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f6c103590 0x7f2f6c075140 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c006010 tx=0x7f2f5c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:44.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f53fff640 1 -- 172.21.15.67:0/3427356150 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c03f030 con 0x7f2f6c103590 2024-07-31T14:48:44.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f7134d640 1 -- 172.21.15.67:0/3427356150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c1a8520 con 0x7f2f6c103590 2024-07-31T14:48:44.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f53fff640 1 -- 172.21.15.67:0/3427356150 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f5c003800 con 0x7f2f6c103590 2024-07-31T14:48:44.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f7134d640 1 -- 172.21.15.67:0/3427356150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c1a8a50 con 0x7f2f6c103590 2024-07-31T14:48:44.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.316+0000 7f2f53fff640 1 -- 172.21.15.67:0/3427356150 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c035070 con 0x7f2f6c103590 2024-07-31T14:48:44.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.317+0000 7f2f7134d640 1 -- 172.21.15.67:0/3427356150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6c1039a0 con 0x7f2f6c103590 2024-07-31T14:48:44.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.319+0000 7f2f53fff640 1 -- 172.21.15.67:0/3427356150 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f2f5c003cb0 con 0x7f2f6c103590 2024-07-31T14:48:44.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.320+0000 7f2f53fff640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f540b3cb0 0x7f2f540b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:44.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.320+0000 7f2f6a7fc640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f540b3cb0 0x7f2f540b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:44.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.321+0000 7f2f53fff640 1 -- 172.21.15.67:0/3427356150 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f2f5c104ef0 con 0x7f2f6c103590 2024-07-31T14:48:44.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.321+0000 7f2f6a7fc640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f540b3cb0 0x7f2f540b61c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c105ef0 tx=0x7f2f60005ec0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:44.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.324+0000 7f2f53fff640 1 -- 172.21.15.67:0/3427356150 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f2f5c1355e0 con 0x7f2f6c103590 2024-07-31T14:48:44.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.519+0000 7f2f7134d640 1 -- 172.21.15.67:0/3427356150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f6c076440 con 0x7f2f6c103590 2024-07-31T14:48:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.521+0000 7f2f53fff640 1 -- 172.21.15.67:0/3427356150 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24874 (secure 0 0 0) 0x7f2f5c0cbfb0 con 0x7f2f6c103590 2024-07-31T14:48:44.524 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:48:44.524 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T14:47:20.440064+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T14:48:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.523+0000 7f2f7134d640 1 -- 172.21.15.67:0/3427356150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f540b3cb0 msgr2=0x7f2f540b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.523+0000 7f2f7134d640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f540b3cb0 0x7f2f540b61c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c105ef0 tx=0x7f2f60005ec0 comp rx=0 tx=0).stop 2024-07-31T14:48:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.523+0000 7f2f7134d640 1 -- 172.21.15.67:0/3427356150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f6c103590 msgr2=0x7f2f6c075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.523+0000 7f2f7134d640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f6c103590 0x7f2f6c075140 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c006010 tx=0x7f2f5c0042f0 comp rx=0 tx=0).stop 2024-07-31T14:48:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.523+0000 7f2f7134d640 1 -- 172.21.15.67:0/3427356150 shutdown_connections 2024-07-31T14:48:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.523+0000 7f2f7134d640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f540b3cb0 0x7f2f540b61c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.523+0000 7f2f7134d640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f6c109820 0x7f2f6c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.523+0000 7f2f7134d640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c108ed0 0x7f2f6c075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.523+0000 7f2f7134d640 1 --2- 172.21.15.67:0/3427356150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f6c103590 0x7f2f6c075140 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.523+0000 7f2f7134d640 1 -- 172.21.15.67:0/3427356150 >> 172.21.15.67:0/3427356150 conn(0x7f2f6c07b280 msgr2=0x7f2f6c107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.523+0000 7f2f7134d640 1 -- 172.21.15.67:0/3427356150 shutdown_connections 2024-07-31T14:48:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:44.523+0000 7f2f7134d640 1 -- 172.21.15.67:0/3427356150 wait complete. 2024-07-31T14:48:45.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:44 smithi073 ceph-mon[36714]: pgmap v329: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-31T14:48:45.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:44 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3427356150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:48:45.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:44 smithi067 ceph-mon[32750]: pgmap v329: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-31T14:48:45.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:44 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3427356150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:48:45.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:44 smithi077 ceph-mon[37327]: pgmap v329: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-31T14:48:45.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:44 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3427356150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:48:45.240 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-07-31T14:48:45.241 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-07-31T14:48:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:47.016 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:46 smithi067 ceph-mon[32750]: pgmap v330: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-31T14:48:47.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:46 smithi073 ceph-mon[36714]: pgmap v330: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-31T14:48:47.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:46 smithi077 ceph-mon[37327]: pgmap v330: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-31T14:48:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.251+0000 7fa5d4516640 1 -- 172.21.15.67:0/4029559046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5cc1021b0 msgr2=0x7fa5cc10ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.251+0000 7fa5d4516640 1 --2- 172.21.15.67:0/4029559046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5cc1021b0 0x7fa5cc10ffe0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009ca0 tx=0x7fa5c402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:48:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.251+0000 7fa5d4516640 1 -- 172.21.15.67:0/4029559046 shutdown_connections 2024-07-31T14:48:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.251+0000 7fa5d4516640 1 --2- 172.21.15.67:0/4029559046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5cc1021b0 0x7fa5cc10ffe0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.251+0000 7fa5d4516640 1 --2- 172.21.15.67:0/4029559046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5cc101860 0x7fa5cc101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.251+0000 7fa5d4516640 1 --2- 172.21.15.67:0/4029559046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5cc108170 0x7fa5cc108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.251+0000 7fa5d4516640 1 -- 172.21.15.67:0/4029559046 >> 172.21.15.67:0/4029559046 conn(0x7fa5cc0799c0 msgr2=0x7fa5cc079dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.252+0000 7fa5d4516640 1 -- 172.21.15.67:0/4029559046 shutdown_connections 2024-07-31T14:48:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.252+0000 7fa5d4516640 1 -- 172.21.15.67:0/4029559046 wait complete. 2024-07-31T14:48:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.252+0000 7fa5d4516640 1 Processor -- start 2024-07-31T14:48:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.252+0000 7fa5d4516640 1 -- start start 2024-07-31T14:48:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.252+0000 7fa5d4516640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5cc101860 0x7fa5cc199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.252+0000 7fa5d4516640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5cc1021b0 0x7fa5cc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d4516640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5cc108170 0x7fa5cc19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d4516640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc113530 con 0x7fa5cc101860 2024-07-31T14:48:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d4516640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1133b0 con 0x7fa5cc1021b0 2024-07-31T14:48:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d4516640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1136b0 con 0x7fa5cc108170 2024-07-31T14:48:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d22b2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5cc101860 0x7fa5cc199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d22b2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5cc101860 0x7fa5cc199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39454/0 (socket says 172.21.15.67:39454) 2024-07-31T14:48:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d22b2640 1 -- 172.21.15.67:0/4128882817 learned_addr learned my addr 172.21.15.67:0/4128882817 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d1ab1640 1 --2- 172.21.15.67:0/4128882817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5cc1021b0 0x7fa5cc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d22b2640 1 -- 172.21.15.67:0/4128882817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5cc108170 msgr2=0x7fa5cc19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d22b2640 1 --2- 172.21.15.67:0/4128882817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5cc108170 0x7fa5cc19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d22b2640 1 -- 172.21.15.67:0/4128882817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5cc1021b0 msgr2=0x7fa5cc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d22b2640 1 --2- 172.21.15.67:0/4128882817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5cc1021b0 0x7fa5cc19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d22b2640 1 -- 172.21.15.67:0/4128882817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc19fe90 con 0x7fa5cc101860 2024-07-31T14:48:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d1ab1640 1 --2- 172.21.15.67:0/4128882817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5cc1021b0 0x7fa5cc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:48:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d22b2640 1 --2- 172.21.15.67:0/4128882817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5cc101860 0x7fa5cc199f20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000e9f0 tx=0x7fa5c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5bf7fe640 1 -- 172.21.15.67:0/4128882817 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c000cd80 con 0x7fa5cc101860 2024-07-31T14:48:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5bf7fe640 1 -- 172.21.15.67:0/4128882817 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5c00054d0 con 0x7fa5cc101860 2024-07-31T14:48:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d4516640 1 -- 172.21.15.67:0/4128882817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc1a0180 con 0x7fa5cc101860 2024-07-31T14:48:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5bf7fe640 1 -- 172.21.15.67:0/4128882817 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0010690 con 0x7fa5cc101860 2024-07-31T14:48:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.253+0000 7fa5d4516640 1 -- 172.21.15.67:0/4128882817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5cc1a68e0 con 0x7fa5cc101860 2024-07-31T14:48:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.254+0000 7fa5d4516640 1 -- 172.21.15.67:0/4128882817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5cc101c70 con 0x7fa5cc101860 2024-07-31T14:48:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.257+0000 7fa5bf7fe640 1 -- 172.21.15.67:0/4128882817 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fa5c0004e30 con 0x7fa5cc101860 2024-07-31T14:48:47.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.258+0000 7fa5bf7fe640 1 --2- 172.21.15.67:0/4128882817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa5b80b3d30 0x7fa5b80b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:47.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.258+0000 7fa5d1ab1640 1 --2- 172.21.15.67:0/4128882817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa5b80b3d30 0x7fa5b80b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:47.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.258+0000 7fa5bf7fe640 1 -- 172.21.15.67:0/4128882817 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fa5c00e48d0 con 0x7fa5cc101860 2024-07-31T14:48:47.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.258+0000 7fa5d1ab1640 1 --2- 172.21.15.67:0/4128882817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa5b80b3d30 0x7fa5b80b6240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80058c0 tx=0x7fa5c8005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:47.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.261+0000 7fa5bf7fe640 1 -- 172.21.15.67:0/4128882817 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fa5c00555e0 con 0x7fa5cc101860 2024-07-31T14:48:47.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.456+0000 7fa5d4516640 1 -- 172.21.15.67:0/4128882817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5cc19b050 con 0x7fa5cc101860 2024-07-31T14:48:47.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.457+0000 7fa5bf7fe640 1 -- 172.21.15.67:0/4128882817 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24874 (secure 0 0 0) 0x7fa5c00ab930 con 0x7fa5cc101860 2024-07-31T14:48:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:48:47.460 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T14:47:20.440064+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T14:48:47.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.460+0000 7fa5d4516640 1 -- 172.21.15.67:0/4128882817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa5b80b3d30 msgr2=0x7fa5b80b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:47.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.460+0000 7fa5d4516640 1 --2- 172.21.15.67:0/4128882817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa5b80b3d30 0x7fa5b80b6240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80058c0 tx=0x7fa5c8005850 comp rx=0 tx=0).stop 2024-07-31T14:48:47.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.460+0000 7fa5d4516640 1 -- 172.21.15.67:0/4128882817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5cc101860 msgr2=0x7fa5cc199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:47.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.460+0000 7fa5d4516640 1 --2- 172.21.15.67:0/4128882817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5cc101860 0x7fa5cc199f20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000e9f0 tx=0x7fa5c000edf0 comp rx=0 tx=0).stop 2024-07-31T14:48:47.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.460+0000 7fa5d4516640 1 -- 172.21.15.67:0/4128882817 shutdown_connections 2024-07-31T14:48:47.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.460+0000 7fa5d4516640 1 --2- 172.21.15.67:0/4128882817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa5b80b3d30 0x7fa5b80b6240 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:47.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.460+0000 7fa5d4516640 1 --2- 172.21.15.67:0/4128882817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5cc108170 0x7fa5cc19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:47.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.460+0000 7fa5d4516640 1 --2- 172.21.15.67:0/4128882817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5cc1021b0 0x7fa5cc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:47.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.460+0000 7fa5d4516640 1 --2- 172.21.15.67:0/4128882817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5cc101860 0x7fa5cc199f20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:47.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.460+0000 7fa5d4516640 1 -- 172.21.15.67:0/4128882817 >> 172.21.15.67:0/4128882817 conn(0x7fa5cc0799c0 msgr2=0x7fa5cc104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:47.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.460+0000 7fa5d4516640 1 -- 172.21.15.67:0/4128882817 shutdown_connections 2024-07-31T14:48:47.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:47.460+0000 7fa5d4516640 1 -- 172.21.15.67:0/4128882817 wait complete. 2024-07-31T14:48:48.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:47 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/4128882817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:48:48.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:47 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/4128882817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:48:48.229 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-07-31T14:48:48.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-07-31T14:48:48.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-07-31T14:48:48.231 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-07-31T14:48:48.231 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-07-31T14:48:48.231 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-07-31T14:48:48.232 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-07-31T14:48:48.232 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-07-31T14:48:48.233 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-07-31T14:48:48.233 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-07-31T14:48:48.234 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-07-31T14:48:48.234 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-07-31T14:48:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:47 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/4128882817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:48:48.932 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:49.111 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:48 smithi067 ceph-mon[32750]: pgmap v331: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:48:49.115 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:49.117 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:49.117 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:49.126 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:49.127 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:49.132 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:49.139 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:48:49.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:48 smithi073 ceph-mon[36714]: pgmap v331: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:48:49.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:48 smithi077 ceph-mon[37327]: pgmap v331: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:48:51.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:50 smithi073 ceph-mon[36714]: pgmap v332: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:48:51.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:50 smithi067 ceph-mon[32750]: pgmap v332: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:48:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:50 smithi077 ceph-mon[37327]: pgmap v332: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:48:52.841 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:52 smithi067 ceph-mon[32750]: pgmap v333: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:48:53.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:52 smithi073 ceph-mon[36714]: pgmap v333: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:48:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:52 smithi077 ceph-mon[37327]: pgmap v333: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:48:54.990 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:54 smithi067 ceph-mon[32750]: pgmap v334: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:48:55.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:54 smithi073 ceph-mon[36714]: pgmap v334: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:48:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:54 smithi077 ceph-mon[37327]: pgmap v334: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:48:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.619+0000 7f1687163640 1 -- 172.21.15.67:0/2983820418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16801024d0 msgr2=0x7f16801028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.619+0000 7f1687163640 1 --2- 172.21.15.67:0/2983820418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16801024d0 0x7f16801028e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1678009900 tx=0x7f167802a410 comp rx=0 tx=0).stop 2024-07-31T14:48:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.619+0000 7f1687163640 1 -- 172.21.15.67:0/2983820418 shutdown_connections 2024-07-31T14:48:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.619+0000 7f1687163640 1 --2- 172.21.15.67:0/2983820418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1680108760 0x7f168010ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.619+0000 7f1687163640 1 --2- 172.21.15.67:0/2983820418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1680107e10 0x7f1680108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.619+0000 7f1687163640 1 --2- 172.21.15.67:0/2983820418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16801024d0 0x7f16801028e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.619+0000 7f1687163640 1 -- 172.21.15.67:0/2983820418 >> 172.21.15.67:0/2983820418 conn(0x7f1680079180 msgr2=0x7f1680079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.619+0000 7f1687163640 1 -- 172.21.15.67:0/2983820418 shutdown_connections 2024-07-31T14:48:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.619+0000 7f1687163640 1 -- 172.21.15.67:0/2983820418 wait complete. 2024-07-31T14:48:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.620+0000 7f1687163640 1 Processor -- start 2024-07-31T14:48:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.620+0000 7f1687163640 1 -- start start 2024-07-31T14:48:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.621+0000 7f1687163640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16801024d0 0x7f1680199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.621+0000 7f1687163640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1680107e10 0x7f168019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.621+0000 7f1687163640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1680108760 0x7f168019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.621+0000 7f1687163640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f16801123c0 con 0x7f1680107e10 2024-07-31T14:48:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.621+0000 7f1687163640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1680112240 con 0x7f1680108760 2024-07-31T14:48:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.621+0000 7f1687163640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1680112540 con 0x7f16801024d0 2024-07-31T14:48:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.621+0000 7f167ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1680107e10 0x7f168019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.621+0000 7f167ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1680107e10 0x7f168019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55538/0 (socket says 172.21.15.67:55538) 2024-07-31T14:48:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.621+0000 7f167ffff640 1 -- 172.21.15.67:0/103493155 learned_addr learned my addr 172.21.15.67:0/103493155 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.621+0000 7f1685700640 1 --2- 172.21.15.67:0/103493155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1680108760 0x7f168019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.621+0000 7f1684eff640 1 --2- 172.21.15.67:0/103493155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16801024d0 0x7f1680199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.621+0000 7f167ffff640 1 -- 172.21.15.67:0/103493155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16801024d0 msgr2=0x7f1680199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.621+0000 7f167ffff640 1 --2- 172.21.15.67:0/103493155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16801024d0 0x7f1680199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.622+0000 7f167ffff640 1 -- 172.21.15.67:0/103493155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1680108760 msgr2=0x7f168019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.622+0000 7f167ffff640 1 --2- 172.21.15.67:0/103493155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1680108760 0x7f168019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.622+0000 7f167ffff640 1 -- 172.21.15.67:0/103493155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168019fd30 con 0x7f1680107e10 2024-07-31T14:48:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.622+0000 7f167ffff640 1 --2- 172.21.15.67:0/103493155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1680107e10 0x7f168019a420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f167000ccb0 tx=0x7f167000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.622+0000 7f167dffb640 1 -- 172.21.15.67:0/103493155 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167000ce60 con 0x7f1680107e10 2024-07-31T14:48:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.622+0000 7f1687163640 1 -- 172.21.15.67:0/103493155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16801a0020 con 0x7f1680107e10 2024-07-31T14:48:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.622+0000 7f167dffb640 1 -- 172.21.15.67:0/103493155 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f167000fb00 con 0x7f1680107e10 2024-07-31T14:48:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.622+0000 7f167dffb640 1 -- 172.21.15.67:0/103493155 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670018bb0 con 0x7f1680107e10 2024-07-31T14:48:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.622+0000 7f1687163640 1 -- 172.21.15.67:0/103493155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1680103ba0 con 0x7f1680107e10 2024-07-31T14:48:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.623+0000 7f1687163640 1 -- 172.21.15.67:0/103493155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1680101480 con 0x7f1680107e10 2024-07-31T14:48:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.625+0000 7f167dffb640 1 -- 172.21.15.67:0/103493155 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f167000f650 con 0x7f1680107e10 2024-07-31T14:48:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.626+0000 7f167dffb640 1 --2- 172.21.15.67:0/103493155 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f166c0b3ce0 0x7f166c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.626+0000 7f1684eff640 1 --2- 172.21.15.67:0/103493155 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f166c0b3ce0 0x7f166c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.626+0000 7f167dffb640 1 -- 172.21.15.67:0/103493155 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f16700e45d0 con 0x7f1680107e10 2024-07-31T14:48:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.626+0000 7f167dffb640 1 --2- 172.21.15.67:0/103493155 >> [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750] conn(0x7f166c0bf180 0x7f166c0c1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.626+0000 7f167dffb640 1 -- 172.21.15.67:0/103493155 --> [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1670021e50 con 0x7f166c0bf180 2024-07-31T14:48:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.626+0000 7f167dffb640 1 -- 172.21.15.67:0/103493155 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f16700eb050 con 0x7f1680107e10 2024-07-31T14:48:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.627+0000 7f1684eff640 1 --2- 172.21.15.67:0/103493155 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f166c0b3ce0 0x7f166c0b61f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1678031440 tx=0x7f1678009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.627+0000 7f1685700640 1 --2- 172.21.15.67:0/103493155 >> [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750] conn(0x7f166c0bf180 0x7f166c0c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.627+0000 7f1685700640 1 --2- 172.21.15.67:0/103493155 >> [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750] conn(0x7f166c0bf180 0x7f166c0c1670 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.628+0000 7f167dffb640 1 -- 172.21.15.67:0/103493155 <== osd.2 v2:172.21.15.77:6800/1624991750 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1670021e50 con 0x7f166c0bf180 2024-07-31T14:48:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.643+0000 7f1687163640 1 -- 172.21.15.67:0/103493155 --> [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f168006ad80 con 0x7f166c0bf180 2024-07-31T14:48:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.644+0000 7f167dffb640 1 -- 172.21.15.67:0/103493155 <== osd.2 v2:172.21.15.77:6800/1624991750 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f168006ad80 con 0x7f166c0bf180 2024-07-31T14:48:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.644+0000 7f1687163640 1 -- 172.21.15.67:0/103493155 >> [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750] conn(0x7f166c0bf180 msgr2=0x7f166c0c1670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.644+0000 7f1687163640 1 --2- 172.21.15.67:0/103493155 >> [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750] conn(0x7f166c0bf180 0x7f166c0c1670 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.644+0000 7f1687163640 1 -- 172.21.15.67:0/103493155 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f166c0b3ce0 msgr2=0x7f166c0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.644+0000 7f1687163640 1 --2- 172.21.15.67:0/103493155 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f166c0b3ce0 0x7f166c0b61f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1678031440 tx=0x7f1678009560 comp rx=0 tx=0).stop 2024-07-31T14:48:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.644+0000 7f1687163640 1 -- 172.21.15.67:0/103493155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1680107e10 msgr2=0x7f168019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.644+0000 7f1687163640 1 --2- 172.21.15.67:0/103493155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1680107e10 0x7f168019a420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f167000ccb0 tx=0x7f167000a620 comp rx=0 tx=0).stop 2024-07-31T14:48:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.644+0000 7f1685700640 1 -- 172.21.15.67:0/103493155 reap_dead start 2024-07-31T14:48:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.644+0000 7f1687163640 1 -- 172.21.15.67:0/103493155 shutdown_connections 2024-07-31T14:48:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.644+0000 7f1687163640 1 -- 172.21.15.67:0/103493155 >> 172.21.15.67:0/103493155 conn(0x7f1680079180 msgr2=0x7f1680106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.645+0000 7f1687163640 1 -- 172.21.15.67:0/103493155 shutdown_connections 2024-07-31T14:48:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:55.645+0000 7f1687163640 1 -- 172.21.15.67:0/103493155 wait complete. 2024-07-31T14:48:56.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.665+0000 7f3ac07ce640 1 -- 172.21.15.67:0/3330106390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8107e10 msgr2=0x7f3ab8108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:56.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.665+0000 7f3ac07ce640 1 --2- 172.21.15.67:0/3330106390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8107e10 0x7f3ab8108220 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009900 tx=0x7f3ab402a410 comp rx=0 tx=0).stop 2024-07-31T14:48:56.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.665+0000 7f3ac07ce640 1 -- 172.21.15.67:0/3330106390 shutdown_connections 2024-07-31T14:48:56.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.665+0000 7f3ac07ce640 1 --2- 172.21.15.67:0/3330106390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ab8108760 0x7f3ab810ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:56.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.665+0000 7f3ac07ce640 1 --2- 172.21.15.67:0/3330106390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8107e10 0x7f3ab8108220 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:56.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.665+0000 7f3ac07ce640 1 --2- 172.21.15.67:0/3330106390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ab81024d0 0x7f3ab81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:56.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.665+0000 7f3ac07ce640 1 -- 172.21.15.67:0/3330106390 >> 172.21.15.67:0/3330106390 conn(0x7f3ab8079190 msgr2=0x7f3ab80795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:56.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.665+0000 7f3ac07ce640 1 -- 172.21.15.67:0/3330106390 shutdown_connections 2024-07-31T14:48:56.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.665+0000 7f3ac07ce640 1 -- 172.21.15.67:0/3330106390 wait complete. 2024-07-31T14:48:56.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.666+0000 7f3ac07ce640 1 Processor -- start 2024-07-31T14:48:56.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.666+0000 7f3ac07ce640 1 -- start start 2024-07-31T14:48:56.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.666+0000 7f3ac07ce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ab81024d0 0x7f3ab819ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:56.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.666+0000 7f3ac07ce640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ab8107e10 0x7f3ab819b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:56.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.666+0000 7f3ac07ce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8108760 0x7f3ab819b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:56.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.666+0000 7f3ac07ce640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8112390 con 0x7f3ab8108760 2024-07-31T14:48:56.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.666+0000 7f3ac07ce640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8112210 con 0x7f3ab8107e10 2024-07-31T14:48:56.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.666+0000 7f3ac07ce640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8112510 con 0x7f3ab81024d0 2024-07-31T14:48:56.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.666+0000 7f3abed6b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8108760 0x7f3ab819b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.666+0000 7f3abed6b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8108760 0x7f3ab819b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55552/0 (socket says 172.21.15.67:55552) 2024-07-31T14:48:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.666+0000 7f3abe56a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ab81024d0 0x7f3ab819ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.666+0000 7f3abed6b640 1 -- 172.21.15.67:0/1583270317 learned_addr learned my addr 172.21.15.67:0/1583270317 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:48:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.666+0000 7f3abdd69640 1 --2- 172.21.15.67:0/1583270317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ab8107e10 0x7f3ab819b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.667+0000 7f3abed6b640 1 -- 172.21.15.67:0/1583270317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ab81024d0 msgr2=0x7f3ab819ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.667+0000 7f3abed6b640 1 --2- 172.21.15.67:0/1583270317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ab81024d0 0x7f3ab819ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.667+0000 7f3abed6b640 1 -- 172.21.15.67:0/1583270317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ab8107e10 msgr2=0x7f3ab819b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.667+0000 7f3abed6b640 1 --2- 172.21.15.67:0/1583270317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ab8107e10 0x7f3ab819b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.667+0000 7f3abed6b640 1 -- 172.21.15.67:0/1583270317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab81a5f70 con 0x7f3ab8108760 2024-07-31T14:48:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.667+0000 7f3abed6b640 1 --2- 172.21.15.67:0/1583270317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8108760 0x7f3ab819b760 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009fd0 tx=0x7f3ab000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.667+0000 7f3aab7fe640 1 -- 172.21.15.67:0/1583270317 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0017070 con 0x7f3ab8108760 2024-07-31T14:48:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.667+0000 7f3aab7fe640 1 -- 172.21.15.67:0/1583270317 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ab0003e20 con 0x7f3ab8108760 2024-07-31T14:48:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.667+0000 7f3ac07ce640 1 -- 172.21.15.67:0/1583270317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab81a6260 con 0x7f3ab8108760 2024-07-31T14:48:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.667+0000 7f3aab7fe640 1 -- 172.21.15.67:0/1583270317 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0010460 con 0x7f3ab8108760 2024-07-31T14:48:56.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.667+0000 7f3ac07ce640 1 -- 172.21.15.67:0/1583270317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab81a6720 con 0x7f3ab8108760 2024-07-31T14:48:56.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.668+0000 7f3ac07ce640 1 -- 172.21.15.67:0/1583270317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ab8063580 con 0x7f3ab8108760 2024-07-31T14:48:56.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.671+0000 7f3aab7fe640 1 -- 172.21.15.67:0/1583270317 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f3ab0003970 con 0x7f3ab8108760 2024-07-31T14:48:56.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.672+0000 7f3aab7fe640 1 --2- 172.21.15.67:0/1583270317 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aa40b3ce0 0x7f3aa40b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:56.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.672+0000 7f3abe56a640 1 --2- 172.21.15.67:0/1583270317 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aa40b3ce0 0x7f3aa40b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:56.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.672+0000 7f3aab7fe640 1 -- 172.21.15.67:0/1583270317 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f3ab00e42b0 con 0x7f3ab8108760 2024-07-31T14:48:56.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.672+0000 7f3aab7fe640 1 --2- 172.21.15.67:0/1583270317 >> [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212] conn(0x7f3aa40bf180 0x7f3aa40c1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:48:56.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.672+0000 7f3aab7fe640 1 -- 172.21.15.67:0/1583270317 --> [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ab001fe50 con 0x7f3aa40bf180 2024-07-31T14:48:56.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.672+0000 7f3aab7fe640 1 -- 172.21.15.67:0/1583270317 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f3ab00e4700 con 0x7f3ab8108760 2024-07-31T14:48:56.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.672+0000 7f3abe56a640 1 --2- 172.21.15.67:0/1583270317 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aa40b3ce0 0x7f3aa40b61f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3ab806c0d0 tx=0x7f3aac00b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:56.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.672+0000 7f3abdd69640 1 --2- 172.21.15.67:0/1583270317 >> [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212] conn(0x7f3aa40bf180 0x7f3aa40c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:48:56.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.673+0000 7f3abdd69640 1 --2- 172.21.15.67:0/1583270317 >> [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212] conn(0x7f3aa40bf180 0x7f3aa40c1670 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:48:56.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.674+0000 7f3aab7fe640 1 -- 172.21.15.67:0/1583270317 <== osd.5 v2:172.21.15.77:6808/683909212 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ab001fe50 con 0x7f3aa40bf180 2024-07-31T14:48:56.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.689+0000 7f3ac07ce640 1 -- 172.21.15.67:0/1583270317 --> [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3ab8063a00 con 0x7f3aa40bf180 2024-07-31T14:48:56.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.689+0000 7f3aab7fe640 1 -- 172.21.15.67:0/1583270317 <== osd.5 v2:172.21.15.77:6808/683909212 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3ab8063a00 con 0x7f3aa40bf180 2024-07-31T14:48:56.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.689+0000 7f3ac07ce640 1 -- 172.21.15.67:0/1583270317 >> [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212] conn(0x7f3aa40bf180 msgr2=0x7f3aa40c1670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:56.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.689+0000 7f3ac07ce640 1 --2- 172.21.15.67:0/1583270317 >> [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212] conn(0x7f3aa40bf180 0x7f3aa40c1670 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:48:56.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.690+0000 7f3ac07ce640 1 -- 172.21.15.67:0/1583270317 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aa40b3ce0 msgr2=0x7f3aa40b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:56.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.690+0000 7f3ac07ce640 1 --2- 172.21.15.67:0/1583270317 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aa40b3ce0 0x7f3aa40b61f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3ab806c0d0 tx=0x7f3aac00b410 comp rx=0 tx=0).stop 2024-07-31T14:48:56.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.690+0000 7f3ac07ce640 1 -- 172.21.15.67:0/1583270317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8108760 msgr2=0x7f3ab819b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:48:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.690+0000 7f3ac07ce640 1 --2- 172.21.15.67:0/1583270317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8108760 0x7f3ab819b760 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009fd0 tx=0x7f3ab000b260 comp rx=0 tx=0).stop 2024-07-31T14:48:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.690+0000 7f3abed6b640 1 -- 172.21.15.67:0/1583270317 reap_dead start 2024-07-31T14:48:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.690+0000 7f3ac07ce640 1 -- 172.21.15.67:0/1583270317 shutdown_connections 2024-07-31T14:48:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.690+0000 7f3ac07ce640 1 -- 172.21.15.67:0/1583270317 >> 172.21.15.67:0/1583270317 conn(0x7f3ab8079190 msgr2=0x7f3ab8104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:48:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.690+0000 7f3ac07ce640 1 -- 172.21.15.67:0/1583270317 shutdown_connections 2024-07-31T14:48:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:48:56.690+0000 7f3ac07ce640 1 -- 172.21.15.67:0/1583270317 wait complete. 2024-07-31T14:48:57.107 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:56 smithi067 ceph-mon[32750]: pgmap v335: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:48:57.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:56 smithi073 ceph-mon[36714]: pgmap v335: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:48:57.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:56 smithi077 ceph-mon[37327]: pgmap v335: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:48:59.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:48:58 smithi067 ceph-mon[32750]: pgmap v336: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:48:59.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:48:58 smithi073 ceph-mon[36714]: pgmap v336: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:48:59.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:48:58 smithi077 ceph-mon[37327]: pgmap v336: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:00.338 INFO:teuthology.orchestra.run.smithi067.stdout:81604378705 2024-07-31T14:49:00.338 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-07-31T14:49:00.755 INFO:teuthology.orchestra.run.smithi067.stdout:163208757315 2024-07-31T14:49:00.755 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-07-31T14:49:01.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:00 smithi067 ceph-mon[32750]: pgmap v337: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:01.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:00 smithi073 ceph-mon[36714]: pgmap v337: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:01.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:00 smithi077 ceph-mon[37327]: pgmap v337: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:02 smithi067 ceph-mon[32750]: pgmap v338: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:49:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:49:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:03.131 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:02 smithi077 ceph-mon[37327]: pgmap v338: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:49:03.131 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:49:03.131 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:03.132 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:03.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:02 smithi073 ceph-mon[36714]: pgmap v338: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:49:03.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:49:03.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:03.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.951+0000 7f9552295640 1 -- 172.21.15.67:0/2429643361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c108ef0 msgr2=0x7f954c109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.951+0000 7f9552295640 1 --2- 172.21.15.67:0/2429643361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c108ef0 0x7f954c109300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9540009900 tx=0x7f954002a410 comp rx=0 tx=0).stop 2024-07-31T14:49:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.951+0000 7f9552295640 1 -- 172.21.15.67:0/2429643361 shutdown_connections 2024-07-31T14:49:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.951+0000 7f9552295640 1 --2- 172.21.15.67:0/2429643361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f954c109840 0x7f954c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.951+0000 7f9552295640 1 --2- 172.21.15.67:0/2429643361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c108ef0 0x7f954c109300 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.951+0000 7f9552295640 1 --2- 172.21.15.67:0/2429643361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f954c1035b0 0x7f954c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.951+0000 7f9552295640 1 -- 172.21.15.67:0/2429643361 >> 172.21.15.67:0/2429643361 conn(0x7f954c0ff040 msgr2=0x7f954c101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.951+0000 7f9552295640 1 -- 172.21.15.67:0/2429643361 shutdown_connections 2024-07-31T14:49:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.951+0000 7f9552295640 1 -- 172.21.15.67:0/2429643361 wait complete. 2024-07-31T14:49:03.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.952+0000 7f9552295640 1 Processor -- start 2024-07-31T14:49:03.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.952+0000 7f9552295640 1 -- start start 2024-07-31T14:49:03.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.952+0000 7f9552295640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c1035b0 0x7f954c199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:03.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.952+0000 7f9552295640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f954c108ef0 0x7f954c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:03.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.952+0000 7f9552295640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f954c109840 0x7f954c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:03.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.952+0000 7f9552295640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f954c113520 con 0x7f954c1035b0 2024-07-31T14:49:03.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.952+0000 7f9552295640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f954c1133a0 con 0x7f954c109840 2024-07-31T14:49:03.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.952+0000 7f9552295640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f954c1136a0 con 0x7f954c108ef0 2024-07-31T14:49:03.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.952+0000 7f954bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c1035b0 0x7f954c199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:03.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.952+0000 7f954bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c1035b0 0x7f954c199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56878/0 (socket says 172.21.15.67:56878) 2024-07-31T14:49:03.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.952+0000 7f954bfff640 1 -- 172.21.15.67:0/301043598 learned_addr learned my addr 172.21.15.67:0/301043598 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:03.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.952+0000 7f9550832640 1 --2- 172.21.15.67:0/301043598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f954c109840 0x7f954c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:03.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.952+0000 7f954b7fe640 1 --2- 172.21.15.67:0/301043598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f954c108ef0 0x7f954c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:03.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.953+0000 7f954bfff640 1 -- 172.21.15.67:0/301043598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f954c108ef0 msgr2=0x7f954c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:03.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.953+0000 7f954bfff640 1 --2- 172.21.15.67:0/301043598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f954c108ef0 0x7f954c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:03.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.953+0000 7f954bfff640 1 -- 172.21.15.67:0/301043598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f954c109840 msgr2=0x7f954c19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:03.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.953+0000 7f954bfff640 1 --2- 172.21.15.67:0/301043598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f954c109840 0x7f954c19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:03.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.953+0000 7f954bfff640 1 -- 172.21.15.67:0/301043598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c19fe40 con 0x7f954c1035b0 2024-07-31T14:49:03.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.953+0000 7f954bfff640 1 --2- 172.21.15.67:0/301043598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c1035b0 0x7f954c199ed0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f953c00e9f0 tx=0x7f953c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:03.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.953+0000 7f95497fa640 1 -- 172.21.15.67:0/301043598 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c00cd80 con 0x7f954c1035b0 2024-07-31T14:49:03.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.953+0000 7f95497fa640 1 -- 172.21.15.67:0/301043598 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f953c0054d0 con 0x7f954c1035b0 2024-07-31T14:49:03.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.953+0000 7f9552295640 1 -- 172.21.15.67:0/301043598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954c1a0130 con 0x7f954c1035b0 2024-07-31T14:49:03.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.953+0000 7f95497fa640 1 -- 172.21.15.67:0/301043598 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c010690 con 0x7f954c1035b0 2024-07-31T14:49:03.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.953+0000 7f9552295640 1 -- 172.21.15.67:0/301043598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954c075020 con 0x7f954c1035b0 2024-07-31T14:49:03.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.954+0000 7f9552295640 1 -- 172.21.15.67:0/301043598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f954c063500 con 0x7f954c1035b0 2024-07-31T14:49:03.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.956+0000 7f95497fa640 1 -- 172.21.15.67:0/301043598 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f953c004e30 con 0x7f954c1035b0 2024-07-31T14:49:03.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.956+0000 7f95497fa640 1 --2- 172.21.15.67:0/301043598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f95340b3d30 0x7f95340b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:03.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.957+0000 7f954b7fe640 1 --2- 172.21.15.67:0/301043598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f95340b3d30 0x7f95340b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:03.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.957+0000 7f95497fa640 1 -- 172.21.15.67:0/301043598 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f953c0e49b0 con 0x7f954c1035b0 2024-07-31T14:49:03.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.957+0000 7f95497fa640 1 --2- 172.21.15.67:0/301043598 >> [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157] conn(0x7f95340bf1d0 0x7f95340c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:03.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.957+0000 7f95497fa640 1 -- 172.21.15.67:0/301043598 --> [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f953c015e80 con 0x7f95340bf1d0 2024-07-31T14:49:03.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.957+0000 7f95497fa640 1 -- 172.21.15.67:0/301043598 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f953c0e4e00 con 0x7f954c1035b0 2024-07-31T14:49:03.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.957+0000 7f9550832640 1 --2- 172.21.15.67:0/301043598 >> [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157] conn(0x7f95340bf1d0 0x7f95340c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:03.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.957+0000 7f954b7fe640 1 --2- 172.21.15.67:0/301043598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f95340b3d30 0x7f95340b6240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f954002aa00 tx=0x7f9540005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:03.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.957+0000 7f9550832640 1 --2- 172.21.15.67:0/301043598 >> [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157] conn(0x7f95340bf1d0 0x7f95340c16c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:03.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.958+0000 7f95497fa640 1 -- 172.21.15.67:0/301043598 <== osd.6 v2:172.21.15.67:6818/380463157 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f953c015e80 con 0x7f95340bf1d0 2024-07-31T14:49:03.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.972+0000 7f9552295640 1 -- 172.21.15.67:0/301043598 --> [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f954c0639b0 con 0x7f95340bf1d0 2024-07-31T14:49:03.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.973+0000 7f95497fa640 1 -- 172.21.15.67:0/301043598 <== osd.6 v2:172.21.15.67:6818/380463157 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f954c0639b0 con 0x7f95340bf1d0 2024-07-31T14:49:03.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.973+0000 7f9552295640 1 -- 172.21.15.67:0/301043598 >> [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157] conn(0x7f95340bf1d0 msgr2=0x7f95340c16c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:03.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.973+0000 7f9552295640 1 --2- 172.21.15.67:0/301043598 >> [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157] conn(0x7f95340bf1d0 0x7f95340c16c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:03.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.973+0000 7f9552295640 1 -- 172.21.15.67:0/301043598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f95340b3d30 msgr2=0x7f95340b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:03.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.973+0000 7f9552295640 1 --2- 172.21.15.67:0/301043598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f95340b3d30 0x7f95340b6240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f954002aa00 tx=0x7f9540005a20 comp rx=0 tx=0).stop 2024-07-31T14:49:03.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.973+0000 7f9552295640 1 -- 172.21.15.67:0/301043598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c1035b0 msgr2=0x7f954c199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:03.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.973+0000 7f9552295640 1 --2- 172.21.15.67:0/301043598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c1035b0 0x7f954c199ed0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f953c00e9f0 tx=0x7f953c00edf0 comp rx=0 tx=0).stop 2024-07-31T14:49:03.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.973+0000 7f9550832640 1 -- 172.21.15.67:0/301043598 reap_dead start 2024-07-31T14:49:03.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.973+0000 7f9552295640 1 -- 172.21.15.67:0/301043598 shutdown_connections 2024-07-31T14:49:03.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.973+0000 7f9552295640 1 -- 172.21.15.67:0/301043598 >> 172.21.15.67:0/301043598 conn(0x7f954c0ff040 msgr2=0x7f954c101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:03.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.973+0000 7f9552295640 1 -- 172.21.15.67:0/301043598 shutdown_connections 2024-07-31T14:49:03.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:03.973+0000 7f9552295640 1 -- 172.21.15.67:0/301043598 wait complete. 2024-07-31T14:49:05.002 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:04 smithi067 ceph-mon[32750]: pgmap v339: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:49:05.002 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:05.002 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:05.002 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:05.003 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:05.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:04 smithi073 ceph-mon[36714]: pgmap v339: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:49:05.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:04 smithi077 ceph-mon[37327]: pgmap v339: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:49:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:05.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.683+0000 7fd893f9f640 1 -- 172.21.15.67:0/2569773531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd88c1043e0 msgr2=0x7fd88c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:05.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.683+0000 7fd893f9f640 1 --2- 172.21.15.67:0/2569773531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd88c1043e0 0x7fd88c1047f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd884009900 tx=0x7fd88402a410 comp rx=0 tx=0).stop 2024-07-31T14:49:05.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.683+0000 7fd893f9f640 1 -- 172.21.15.67:0/2569773531 shutdown_connections 2024-07-31T14:49:05.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.683+0000 7fd893f9f640 1 --2- 172.21.15.67:0/2569773531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd88c10d840 0x7fd88c10fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:05.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.683+0000 7fd893f9f640 1 --2- 172.21.15.67:0/2569773531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd88c104d30 0x7fd88c10d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:05.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.683+0000 7fd893f9f640 1 --2- 172.21.15.67:0/2569773531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd88c1043e0 0x7fd88c1047f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:05.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.683+0000 7fd893f9f640 1 -- 172.21.15.67:0/2569773531 >> 172.21.15.67:0/2569773531 conn(0x7fd88c079160 msgr2=0x7fd88c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:05.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.683+0000 7fd893f9f640 1 -- 172.21.15.67:0/2569773531 shutdown_connections 2024-07-31T14:49:05.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.683+0000 7fd893f9f640 1 -- 172.21.15.67:0/2569773531 wait complete. 2024-07-31T14:49:05.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.684+0000 7fd893f9f640 1 Processor -- start 2024-07-31T14:49:05.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.684+0000 7fd893f9f640 1 -- start start 2024-07-31T14:49:05.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.684+0000 7fd893f9f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd88c1043e0 0x7fd88c199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:05.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.684+0000 7fd893f9f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd88c104d30 0x7fd88c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:05.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.684+0000 7fd893f9f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd88c10d840 0x7fd88c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:05.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.684+0000 7fd893f9f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1134b0 con 0x7fd88c1043e0 2024-07-31T14:49:05.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.684+0000 7fd893f9f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c113330 con 0x7fd88c104d30 2024-07-31T14:49:05.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.684+0000 7fd893f9f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c113630 con 0x7fd88c10d840 2024-07-31T14:49:05.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.684+0000 7fd891d3b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd88c1043e0 0x7fd88c199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:05.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.684+0000 7fd891d3b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd88c1043e0 0x7fd88c199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56888/0 (socket says 172.21.15.67:56888) 2024-07-31T14:49:05.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.684+0000 7fd891d3b640 1 -- 172.21.15.67:0/115091615 learned_addr learned my addr 172.21.15.67:0/115091615 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:05.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.685+0000 7fd89153a640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd88c104d30 0x7fd88c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:05.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.685+0000 7fd89253c640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd88c10d840 0x7fd88c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:05.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.685+0000 7fd891d3b640 1 -- 172.21.15.67:0/115091615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd88c10d840 msgr2=0x7fd88c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:05.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.685+0000 7fd891d3b640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd88c10d840 0x7fd88c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:05.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.685+0000 7fd891d3b640 1 -- 172.21.15.67:0/115091615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd88c104d30 msgr2=0x7fd88c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:05.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.685+0000 7fd891d3b640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd88c104d30 0x7fd88c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:05.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.685+0000 7fd891d3b640 1 -- 172.21.15.67:0/115091615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c19fe30 con 0x7fd88c1043e0 2024-07-31T14:49:05.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.685+0000 7fd89253c640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd88c10d840 0x7fd88c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:49:05.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.685+0000 7fd891d3b640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd88c1043e0 0x7fd88c199ec0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd884006010 tx=0x7fd884004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:05.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.685+0000 7fd88affd640 1 -- 172.21.15.67:0/115091615 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88403f030 con 0x7fd88c1043e0 2024-07-31T14:49:05.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.685+0000 7fd88affd640 1 -- 172.21.15.67:0/115091615 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd884003aa0 con 0x7fd88c1043e0 2024-07-31T14:49:05.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.685+0000 7fd893f9f640 1 -- 172.21.15.67:0/115091615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88c1a00c0 con 0x7fd88c1043e0 2024-07-31T14:49:05.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.685+0000 7fd88affd640 1 -- 172.21.15.67:0/115091615 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884035070 con 0x7fd88c1043e0 2024-07-31T14:49:05.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.685+0000 7fd893f9f640 1 -- 172.21.15.67:0/115091615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88c1a6880 con 0x7fd88c1043e0 2024-07-31T14:49:05.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.686+0000 7fd893f9f640 1 -- 172.21.15.67:0/115091615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd88c063540 con 0x7fd88c1043e0 2024-07-31T14:49:05.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.688+0000 7fd88affd640 1 -- 172.21.15.67:0/115091615 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fd884042030 con 0x7fd88c1043e0 2024-07-31T14:49:05.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.689+0000 7fd88affd640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd8740b3cb0 0x7fd8740b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:05.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.689+0000 7fd89153a640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd8740b3cb0 0x7fd8740b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:05.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.689+0000 7fd88affd640 1 -- 172.21.15.67:0/115091615 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fd884104b00 con 0x7fd88c1043e0 2024-07-31T14:49:05.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.689+0000 7fd88affd640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579] conn(0x7fd8740bf150 0x7fd8740c1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:05.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.689+0000 7fd88affd640 1 -- 172.21.15.67:0/115091615 --> [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd884046e50 con 0x7fd8740bf150 2024-07-31T14:49:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.689+0000 7fd88affd640 1 -- 172.21.15.67:0/115091615 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fd884104f50 con 0x7fd88c1043e0 2024-07-31T14:49:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.689+0000 7fd89153a640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd8740b3cb0 0x7fd8740b61c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd88c101e20 tx=0x7fd88000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.690+0000 7fd89253c640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579] conn(0x7fd8740bf150 0x7fd8740c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.690+0000 7fd89253c640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579] conn(0x7fd8740bf150 0x7fd8740c1640 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.691+0000 7fd88affd640 1 -- 172.21.15.67:0/115091615 <== osd.4 v2:172.21.15.73:6808/3387924579 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd884046e50 con 0x7fd8740bf150 2024-07-31T14:49:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.705+0000 7fd893f9f640 1 -- 172.21.15.67:0/115091615 --> [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd88c0639c0 con 0x7fd8740bf150 2024-07-31T14:49:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.706+0000 7fd88affd640 1 -- 172.21.15.67:0/115091615 <== osd.4 v2:172.21.15.73:6808/3387924579 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd88c0639c0 con 0x7fd8740bf150 2024-07-31T14:49:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.706+0000 7fd893f9f640 1 -- 172.21.15.67:0/115091615 >> [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579] conn(0x7fd8740bf150 msgr2=0x7fd8740c1640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.706+0000 7fd893f9f640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579] conn(0x7fd8740bf150 0x7fd8740c1640 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.706+0000 7fd893f9f640 1 -- 172.21.15.67:0/115091615 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd8740b3cb0 msgr2=0x7fd8740b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.706+0000 7fd893f9f640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd8740b3cb0 0x7fd8740b61c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd88c101e20 tx=0x7fd88000b390 comp rx=0 tx=0).stop 2024-07-31T14:49:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.706+0000 7fd893f9f640 1 -- 172.21.15.67:0/115091615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd88c1043e0 msgr2=0x7fd88c199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.706+0000 7fd893f9f640 1 --2- 172.21.15.67:0/115091615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd88c1043e0 0x7fd88c199ec0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd884006010 tx=0x7fd884004060 comp rx=0 tx=0).stop 2024-07-31T14:49:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.706+0000 7fd89253c640 1 -- 172.21.15.67:0/115091615 reap_dead start 2024-07-31T14:49:05.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.706+0000 7fd893f9f640 1 -- 172.21.15.67:0/115091615 shutdown_connections 2024-07-31T14:49:05.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.706+0000 7fd893f9f640 1 -- 172.21.15.67:0/115091615 >> 172.21.15.67:0/115091615 conn(0x7fd88c079160 msgr2=0x7fd88c100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:05.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.706+0000 7fd893f9f640 1 -- 172.21.15.67:0/115091615 shutdown_connections 2024-07-31T14:49:05.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:04.706+0000 7fd893f9f640 1 -- 172.21.15.67:0/115091615 wait complete. 2024-07-31T14:49:06.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.415+0000 7fb94b947640 1 -- 172.21.15.67:0/3166411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944108ed0 msgr2=0x7fb9441092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:06.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.415+0000 7fb94b947640 1 --2- 172.21.15.67:0/3166411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944108ed0 0x7fb9441092e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb934009900 tx=0x7fb93402a410 comp rx=0 tx=0).stop 2024-07-31T14:49:06.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.415+0000 7fb94b947640 1 -- 172.21.15.67:0/3166411 shutdown_connections 2024-07-31T14:49:06.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.415+0000 7fb94b947640 1 --2- 172.21.15.67:0/3166411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb944109820 0x7fb944110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:06.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.415+0000 7fb94b947640 1 --2- 172.21.15.67:0/3166411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944108ed0 0x7fb9441092e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:06.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.415+0000 7fb94b947640 1 --2- 172.21.15.67:0/3166411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb944103590 0x7fb9441039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:06.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.415+0000 7fb94b947640 1 -- 172.21.15.67:0/3166411 >> 172.21.15.67:0/3166411 conn(0x7fb9440ff000 msgr2=0x7fb944101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:06.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.415+0000 7fb94b947640 1 -- 172.21.15.67:0/3166411 shutdown_connections 2024-07-31T14:49:06.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.415+0000 7fb94b947640 1 -- 172.21.15.67:0/3166411 wait complete. 2024-07-31T14:49:06.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.416+0000 7fb94b947640 1 Processor -- start 2024-07-31T14:49:06.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.416+0000 7fb94b947640 1 -- start start 2024-07-31T14:49:06.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.416+0000 7fb94b947640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb944103590 0x7fb944075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:06.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.416+0000 7fb94b947640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944108ed0 0x7fb944075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.416+0000 7fb94b947640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb944109820 0x7fb944075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.416+0000 7fb94b947640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb9441134b0 con 0x7fb944108ed0 2024-07-31T14:49:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.416+0000 7fb94b947640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb944113330 con 0x7fb944103590 2024-07-31T14:49:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.416+0000 7fb94b947640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb944113630 con 0x7fb944109820 2024-07-31T14:49:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.416+0000 7fb948ee2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944108ed0 0x7fb944075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.416+0000 7fb948ee2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944108ed0 0x7fb944075680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56932/0 (socket says 172.21.15.67:56932) 2024-07-31T14:49:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.416+0000 7fb948ee2640 1 -- 172.21.15.67:0/3109134732 learned_addr learned my addr 172.21.15.67:0/3109134732 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.416+0000 7fb9496e3640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb944103590 0x7fb944075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.416+0000 7fb949ee4640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb944109820 0x7fb944075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.417+0000 7fb948ee2640 1 -- 172.21.15.67:0/3109134732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb944109820 msgr2=0x7fb944075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.417+0000 7fb948ee2640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb944109820 0x7fb944075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.417+0000 7fb948ee2640 1 -- 172.21.15.67:0/3109134732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb944103590 msgr2=0x7fb944075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.417+0000 7fb948ee2640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb944103590 0x7fb944075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.417+0000 7fb948ee2640 1 -- 172.21.15.67:0/3109134732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9441a81e0 con 0x7fb944108ed0 2024-07-31T14:49:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.417+0000 7fb949ee4640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb944109820 0x7fb944075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:49:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.417+0000 7fb948ee2640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944108ed0 0x7fb944075680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb934006010 tx=0x7fb934004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.417+0000 7fb93a7fc640 1 -- 172.21.15.67:0/3109134732 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93403f030 con 0x7fb944108ed0 2024-07-31T14:49:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.417+0000 7fb93a7fc640 1 -- 172.21.15.67:0/3109134732 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb934003aa0 con 0x7fb944108ed0 2024-07-31T14:49:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.417+0000 7fb94b947640 1 -- 172.21.15.67:0/3109134732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9441a8470 con 0x7fb944108ed0 2024-07-31T14:49:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.417+0000 7fb93a7fc640 1 -- 172.21.15.67:0/3109134732 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934035070 con 0x7fb944108ed0 2024-07-31T14:49:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.417+0000 7fb94b947640 1 -- 172.21.15.67:0/3109134732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9441a8950 con 0x7fb944108ed0 2024-07-31T14:49:06.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.418+0000 7fb94b947640 1 -- 172.21.15.67:0/3109134732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb944063540 con 0x7fb944108ed0 2024-07-31T14:49:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.420+0000 7fb93a7fc640 1 -- 172.21.15.67:0/3109134732 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb934042030 con 0x7fb944108ed0 2024-07-31T14:49:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.421+0000 7fb93a7fc640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb9300b3cb0 0x7fb9300b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.421+0000 7fb9496e3640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb9300b3cb0 0x7fb9300b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.421+0000 7fb93a7fc640 1 -- 172.21.15.67:0/3109134732 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fb9341047f0 con 0x7fb944108ed0 2024-07-31T14:49:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.421+0000 7fb93a7fc640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961] conn(0x7fb9300bf150 0x7fb9300c1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.421+0000 7fb93a7fc640 1 -- 172.21.15.67:0/3109134732 --> [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9300c1d10 con 0x7fb9300bf150 2024-07-31T14:49:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.421+0000 7fb93a7fc640 1 -- 172.21.15.67:0/3109134732 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb934104c40 con 0x7fb944108ed0 2024-07-31T14:49:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.421+0000 7fb9496e3640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb9300b3cb0 0x7fb9300b61c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb944076b90 tx=0x7fb940005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.421+0000 7fb949ee4640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961] conn(0x7fb9300bf150 0x7fb9300c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.422+0000 7fb949ee4640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961] conn(0x7fb9300bf150 0x7fb9300c1640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.423+0000 7fb93a7fc640 1 -- 172.21.15.67:0/3109134732 <== osd.11 v2:172.21.15.77:6824/764558961 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9300c1d10 con 0x7fb9300bf150 2024-07-31T14:49:06.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.437+0000 7fb94b947640 1 -- 172.21.15.67:0/3109134732 --> [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb9440639c0 con 0x7fb9300bf150 2024-07-31T14:49:06.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.438+0000 7fb93a7fc640 1 -- 172.21.15.67:0/3109134732 <== osd.11 v2:172.21.15.77:6824/764558961 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb9440639c0 con 0x7fb9300bf150 2024-07-31T14:49:06.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.438+0000 7fb94b947640 1 -- 172.21.15.67:0/3109134732 >> [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961] conn(0x7fb9300bf150 msgr2=0x7fb9300c1640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:06.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.438+0000 7fb94b947640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961] conn(0x7fb9300bf150 0x7fb9300c1640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:06.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.439+0000 7fb94b947640 1 -- 172.21.15.67:0/3109134732 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb9300b3cb0 msgr2=0x7fb9300b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.439+0000 7fb94b947640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb9300b3cb0 0x7fb9300b61c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb944076b90 tx=0x7fb940005f90 comp rx=0 tx=0).stop 2024-07-31T14:49:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.439+0000 7fb94b947640 1 -- 172.21.15.67:0/3109134732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944108ed0 msgr2=0x7fb944075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.439+0000 7fb94b947640 1 --2- 172.21.15.67:0/3109134732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944108ed0 0x7fb944075680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb934006010 tx=0x7fb934004060 comp rx=0 tx=0).stop 2024-07-31T14:49:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.439+0000 7fb949ee4640 1 -- 172.21.15.67:0/3109134732 reap_dead start 2024-07-31T14:49:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.439+0000 7fb94b947640 1 -- 172.21.15.67:0/3109134732 shutdown_connections 2024-07-31T14:49:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.439+0000 7fb94b947640 1 -- 172.21.15.67:0/3109134732 >> 172.21.15.67:0/3109134732 conn(0x7fb9440ff000 msgr2=0x7fb944101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.439+0000 7fb94b947640 1 -- 172.21.15.67:0/3109134732 shutdown_connections 2024-07-31T14:49:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:06.439+0000 7fb94b947640 1 -- 172.21.15.67:0/3109134732 wait complete. 2024-07-31T14:49:06.978 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:06 smithi067 ceph-mon[32750]: pgmap v340: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-07-31T14:49:07.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.073+0000 7f8b14920640 1 -- 172.21.15.67:0/1822196319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b10107dd0 msgr2=0x7f8b101081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:07.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.073+0000 7f8b14920640 1 --2- 172.21.15.67:0/1822196319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b10107dd0 0x7f8b101081e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009900 tx=0x7f8af802a410 comp rx=0 tx=0).stop 2024-07-31T14:49:07.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.073+0000 7f8b14920640 1 -- 172.21.15.67:0/1822196319 shutdown_connections 2024-07-31T14:49:07.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.073+0000 7f8b14920640 1 --2- 172.21.15.67:0/1822196319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b10108720 0x7f8b1010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:07.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.073+0000 7f8b14920640 1 --2- 172.21.15.67:0/1822196319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b10107dd0 0x7f8b101081e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:07.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.073+0000 7f8b14920640 1 --2- 172.21.15.67:0/1822196319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b10102490 0x7f8b101028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:07.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.073+0000 7f8b14920640 1 -- 172.21.15.67:0/1822196319 >> 172.21.15.67:0/1822196319 conn(0x7f8b10079160 msgr2=0x7f8b10079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:07.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.073+0000 7f8b14920640 1 -- 172.21.15.67:0/1822196319 shutdown_connections 2024-07-31T14:49:07.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.073+0000 7f8b14920640 1 -- 172.21.15.67:0/1822196319 wait complete. 2024-07-31T14:49:07.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.074+0000 7f8b14920640 1 Processor -- start 2024-07-31T14:49:07.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.074+0000 7f8b14920640 1 -- start start 2024-07-31T14:49:07.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.074+0000 7f8b14920640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b10102490 0x7f8b10199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:07.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.074+0000 7f8b14920640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b10107dd0 0x7f8b1019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:07.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.074+0000 7f8b14920640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b10108720 0x7f8b1019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:07.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.074+0000 7f8b14920640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8b10112350 con 0x7f8b10108720 2024-07-31T14:49:07.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.074+0000 7f8b14920640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8b101121d0 con 0x7f8b10107dd0 2024-07-31T14:49:07.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.074+0000 7f8b14920640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8b101124d0 con 0x7f8b10102490 2024-07-31T14:49:07.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b0ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b10108720 0x7f8b1019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:07.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b0ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b10108720 0x7f8b1019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56956/0 (socket says 172.21.15.67:56956) 2024-07-31T14:49:07.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b0ed9d640 1 -- 172.21.15.67:0/926341321 learned_addr learned my addr 172.21.15.67:0/926341321 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:07.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b0dd9b640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b10107dd0 0x7f8b1019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:07.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b0e59c640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b10102490 0x7f8b10199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:07.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b0ed9d640 1 -- 172.21.15.67:0/926341321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b10102490 msgr2=0x7f8b10199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:07.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b0ed9d640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b10102490 0x7f8b10199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:07.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b0ed9d640 1 -- 172.21.15.67:0/926341321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b10107dd0 msgr2=0x7f8b1019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:07.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b0ed9d640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b10107dd0 0x7f8b1019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:07.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b0ed9d640 1 -- 172.21.15.67:0/926341321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1019fe20 con 0x7f8b10108720 2024-07-31T14:49:07.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b0dd9b640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b10107dd0 0x7f8b1019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:49:07.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b0ed9d640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b10108720 0x7f8b1019f7b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800d750 tx=0x7f8b0800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:07.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b077fe640 1 -- 172.21.15.67:0/926341321 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b08017070 con 0x7f8b10108720 2024-07-31T14:49:07.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b14920640 1 -- 172.21.15.67:0/926341321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b101a0110 con 0x7f8b10108720 2024-07-31T14:49:07.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b077fe640 1 -- 172.21.15.67:0/926341321 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b080054d0 con 0x7f8b10108720 2024-07-31T14:49:07.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b077fe640 1 -- 172.21.15.67:0/926341321 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b08004c60 con 0x7f8b10108720 2024-07-31T14:49:07.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.075+0000 7f8b14920640 1 -- 172.21.15.67:0/926341321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b101a05d0 con 0x7f8b10108720 2024-07-31T14:49:07.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.076+0000 7f8b14920640 1 -- 172.21.15.67:0/926341321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b10104cc0 con 0x7f8b10108720 2024-07-31T14:49:07.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.078+0000 7f8b077fe640 1 -- 172.21.15.67:0/926341321 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f8b08009bd0 con 0x7f8b10108720 2024-07-31T14:49:07.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.079+0000 7f8b077fe640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8afc0b3ce0 0x7f8afc0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:07.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.079+0000 7f8b0e59c640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8afc0b3ce0 0x7f8afc0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:07.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.079+0000 7f8b077fe640 1 -- 172.21.15.67:0/926341321 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f8b080e4000 con 0x7f8b10108720 2024-07-31T14:49:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.079+0000 7f8b077fe640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754] conn(0x7f8afc0bf100 0x7f8afc0c15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.079+0000 7f8b077fe640 1 -- 172.21.15.67:0/926341321 --> [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8afc0c1cc0 con 0x7f8afc0bf100 2024-07-31T14:49:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.079+0000 7f8b077fe640 1 -- 172.21.15.67:0/926341321 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f8b080e4450 con 0x7f8b10108720 2024-07-31T14:49:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.080+0000 7f8b0e59c640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8afc0b3ce0 0x7f8afc0b61f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8b00007350 tx=0x7f8b000072e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.080+0000 7f8b0dd9b640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754] conn(0x7f8afc0bf100 0x7f8afc0c15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.080+0000 7f8b0dd9b640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754] conn(0x7f8afc0bf100 0x7f8afc0c15f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.081+0000 7f8b077fe640 1 -- 172.21.15.67:0/926341321 <== osd.8 v2:172.21.15.77:6816/2208692754 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8afc0c1cc0 con 0x7f8afc0bf100 2024-07-31T14:49:07.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.096+0000 7f8b14920640 1 -- 172.21.15.67:0/926341321 --> [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8b10103710 con 0x7f8afc0bf100 2024-07-31T14:49:07.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.096+0000 7f8b077fe640 1 -- 172.21.15.67:0/926341321 <== osd.8 v2:172.21.15.77:6816/2208692754 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8b10103710 con 0x7f8afc0bf100 2024-07-31T14:49:07.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.097+0000 7f8b14920640 1 -- 172.21.15.67:0/926341321 >> [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754] conn(0x7f8afc0bf100 msgr2=0x7f8afc0c15f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:07.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.097+0000 7f8b14920640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754] conn(0x7f8afc0bf100 0x7f8afc0c15f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:07.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.097+0000 7f8b14920640 1 -- 172.21.15.67:0/926341321 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8afc0b3ce0 msgr2=0x7f8afc0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:07.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.097+0000 7f8b14920640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8afc0b3ce0 0x7f8afc0b61f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8b00007350 tx=0x7f8b000072e0 comp rx=0 tx=0).stop 2024-07-31T14:49:07.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.097+0000 7f8b14920640 1 -- 172.21.15.67:0/926341321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b10108720 msgr2=0x7f8b1019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:07.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.097+0000 7f8b14920640 1 --2- 172.21.15.67:0/926341321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b10108720 0x7f8b1019f7b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800d750 tx=0x7f8b0800dc10 comp rx=0 tx=0).stop 2024-07-31T14:49:07.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.097+0000 7f8b0ed9d640 1 -- 172.21.15.67:0/926341321 reap_dead start 2024-07-31T14:49:07.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.097+0000 7f8b14920640 1 -- 172.21.15.67:0/926341321 shutdown_connections 2024-07-31T14:49:07.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.097+0000 7f8b14920640 1 -- 172.21.15.67:0/926341321 >> 172.21.15.67:0/926341321 conn(0x7f8b10079160 msgr2=0x7f8b10101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:07.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.097+0000 7f8b14920640 1 -- 172.21.15.67:0/926341321 shutdown_connections 2024-07-31T14:49:07.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.097+0000 7f8b14920640 1 -- 172.21.15.67:0/926341321 wait complete. 2024-07-31T14:49:07.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:06 smithi073 ceph-mon[36714]: pgmap v340: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-07-31T14:49:07.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:06 smithi077 ceph-mon[37327]: pgmap v340: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-07-31T14:49:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.758+0000 7fa62cbcf640 1 -- 172.21.15.67:0/3205420653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa628108740 msgr2=0x7fa62810ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.758+0000 7fa62cbcf640 1 --2- 172.21.15.67:0/3205420653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa628108740 0x7fa62810ef70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa620009ca0 tx=0x7fa62002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:49:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.758+0000 7fa62cbcf640 1 -- 172.21.15.67:0/3205420653 shutdown_connections 2024-07-31T14:49:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.758+0000 7fa62cbcf640 1 --2- 172.21.15.67:0/3205420653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa628108740 0x7fa62810ef70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.758+0000 7fa62cbcf640 1 --2- 172.21.15.67:0/3205420653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107df0 0x7fa628108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.758+0000 7fa62cbcf640 1 --2- 172.21.15.67:0/3205420653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6281024b0 0x7fa6281028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.758+0000 7fa62cbcf640 1 -- 172.21.15.67:0/3205420653 >> 172.21.15.67:0/3205420653 conn(0x7fa628079170 msgr2=0x7fa628079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.758+0000 7fa62cbcf640 1 -- 172.21.15.67:0/3205420653 shutdown_connections 2024-07-31T14:49:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.758+0000 7fa62cbcf640 1 -- 172.21.15.67:0/3205420653 wait complete. 2024-07-31T14:49:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.759+0000 7fa62cbcf640 1 Processor -- start 2024-07-31T14:49:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.759+0000 7fa62cbcf640 1 -- start start 2024-07-31T14:49:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.759+0000 7fa62cbcf640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6281024b0 0x7fa628199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.759+0000 7fa62cbcf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa628107df0 0x7fa62819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.759+0000 7fa62cbcf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628108740 0x7fa62819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.759+0000 7fa62cbcf640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281134d0 con 0x7fa6281024b0 2024-07-31T14:49:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.759+0000 7fa62cbcf640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa628113350 con 0x7fa628108740 2024-07-31T14:49:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.759+0000 7fa62cbcf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa628113650 con 0x7fa628107df0 2024-07-31T14:49:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.759+0000 7fa62659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6281024b0 0x7fa628199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.759+0000 7fa62659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6281024b0 0x7fa628199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56916/0 (socket says 172.21.15.67:56916) 2024-07-31T14:49:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.759+0000 7fa62659c640 1 -- 172.21.15.67:0/4276588163 learned_addr learned my addr 172.21.15.67:0/4276588163 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.760+0000 7fa626d9d640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628108740 0x7fa62819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.760+0000 7fa625d9b640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa628107df0 0x7fa62819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.760+0000 7fa62659c640 1 -- 172.21.15.67:0/4276588163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa628107df0 msgr2=0x7fa62819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.760+0000 7fa62659c640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa628107df0 0x7fa62819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.760+0000 7fa62659c640 1 -- 172.21.15.67:0/4276588163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628108740 msgr2=0x7fa62819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.760+0000 7fa62659c640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628108740 0x7fa62819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.760+0000 7fa62659c640 1 -- 172.21.15.67:0/4276588163 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62819fe50 con 0x7fa6281024b0 2024-07-31T14:49:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.760+0000 7fa626d9d640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628108740 0x7fa62819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:49:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.760+0000 7fa62659c640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6281024b0 0x7fa628199ee0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa61800c890 tx=0x7fa61800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.760+0000 7fa6177fe640 1 -- 172.21.15.67:0/4276588163 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618004220 con 0x7fa6281024b0 2024-07-31T14:49:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.760+0000 7fa6177fe640 1 -- 172.21.15.67:0/4276588163 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6180043c0 con 0x7fa6281024b0 2024-07-31T14:49:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.760+0000 7fa62cbcf640 1 -- 172.21.15.67:0/4276588163 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6281a0140 con 0x7fa6281024b0 2024-07-31T14:49:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.760+0000 7fa6177fe640 1 -- 172.21.15.67:0/4276588163 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618004a00 con 0x7fa6281024b0 2024-07-31T14:49:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.760+0000 7fa62cbcf640 1 -- 172.21.15.67:0/4276588163 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6281a68a0 con 0x7fa6281024b0 2024-07-31T14:49:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.761+0000 7fa62cbcf640 1 -- 172.21.15.67:0/4276588163 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa628063540 con 0x7fa6281024b0 2024-07-31T14:49:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.763+0000 7fa6177fe640 1 -- 172.21.15.67:0/4276588163 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fa6180079b0 con 0x7fa6281024b0 2024-07-31T14:49:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.764+0000 7fa6177fe640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa6100b3cb0 0x7fa6100b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.764+0000 7fa625d9b640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa6100b3cb0 0x7fa6100b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.764+0000 7fa6177fe640 1 -- 172.21.15.67:0/4276588163 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fa6180e45d0 con 0x7fa6281024b0 2024-07-31T14:49:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.764+0000 7fa6177fe640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504] conn(0x7fa6100bf150 0x7fa6100c1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.764+0000 7fa6177fe640 1 -- 172.21.15.67:0/4276588163 --> [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa618021e50 con 0x7fa6100bf150 2024-07-31T14:49:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.764+0000 7fa6177fe640 1 -- 172.21.15.67:0/4276588163 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa6180e4a20 con 0x7fa6281024b0 2024-07-31T14:49:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.764+0000 7fa626d9d640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504] conn(0x7fa6100bf150 0x7fa6100c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.764+0000 7fa625d9b640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa6100b3cb0 0x7fa6100b61c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa61c005fd0 tx=0x7fa61c005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.765+0000 7fa626d9d640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504] conn(0x7fa6100bf150 0x7fa6100c1640 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.765+0000 7fa6177fe640 1 -- 172.21.15.67:0/4276588163 <== osd.0 v2:172.21.15.67:6802/2442400504 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa618021e50 con 0x7fa6100bf150 2024-07-31T14:49:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.781+0000 7fa62cbcf640 1 -- 172.21.15.67:0/4276588163 --> [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa628063970 con 0x7fa6100bf150 2024-07-31T14:49:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.781+0000 7fa6177fe640 1 -- 172.21.15.67:0/4276588163 <== osd.0 v2:172.21.15.67:6802/2442400504 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa628063970 con 0x7fa6100bf150 2024-07-31T14:49:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.781+0000 7fa62cbcf640 1 -- 172.21.15.67:0/4276588163 >> [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504] conn(0x7fa6100bf150 msgr2=0x7fa6100c1640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.781+0000 7fa62cbcf640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504] conn(0x7fa6100bf150 0x7fa6100c1640 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.782+0000 7fa62cbcf640 1 -- 172.21.15.67:0/4276588163 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa6100b3cb0 msgr2=0x7fa6100b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.782+0000 7fa62cbcf640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa6100b3cb0 0x7fa6100b61c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa61c005fd0 tx=0x7fa61c005850 comp rx=0 tx=0).stop 2024-07-31T14:49:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.782+0000 7fa62cbcf640 1 -- 172.21.15.67:0/4276588163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6281024b0 msgr2=0x7fa628199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.782+0000 7fa62cbcf640 1 --2- 172.21.15.67:0/4276588163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6281024b0 0x7fa628199ee0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa61800c890 tx=0x7fa61800cd50 comp rx=0 tx=0).stop 2024-07-31T14:49:08.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.782+0000 7fa626d9d640 1 -- 172.21.15.67:0/4276588163 reap_dead start 2024-07-31T14:49:08.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.782+0000 7fa62cbcf640 1 -- 172.21.15.67:0/4276588163 shutdown_connections 2024-07-31T14:49:08.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.782+0000 7fa62cbcf640 1 -- 172.21.15.67:0/4276588163 >> 172.21.15.67:0/4276588163 conn(0x7fa628079170 msgr2=0x7fa628104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:08.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.782+0000 7fa62cbcf640 1 -- 172.21.15.67:0/4276588163 shutdown_connections 2024-07-31T14:49:08.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:05.782+0000 7fa62cbcf640 1 -- 172.21.15.67:0/4276588163 wait complete. 2024-07-31T14:49:08.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.289+0000 7f211dd9f640 1 -- 172.21.15.67:0/1525039189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f211810d630 msgr2=0x7f211810fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:08.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.289+0000 7f211dd9f640 1 --2- 172.21.15.67:0/1525039189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f211810d630 0x7f211810fb40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2110009ca0 tx=0x7f211002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:49:08.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.289+0000 7f211dd9f640 1 -- 172.21.15.67:0/1525039189 shutdown_connections 2024-07-31T14:49:08.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.289+0000 7f211dd9f640 1 --2- 172.21.15.67:0/1525039189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f211810d630 0x7f211810fb40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:08.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.289+0000 7f211dd9f640 1 --2- 172.21.15.67:0/1525039189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2118104ab0 0x7f211810d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:08.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.289+0000 7f211dd9f640 1 --2- 172.21.15.67:0/1525039189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2118104080 0x7f2118104490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:08.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.289+0000 7f211dd9f640 1 -- 172.21.15.67:0/1525039189 >> 172.21.15.67:0/1525039189 conn(0x7f21180fdcf0 msgr2=0x7f2118100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:08.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.289+0000 7f211dd9f640 1 -- 172.21.15.67:0/1525039189 shutdown_connections 2024-07-31T14:49:08.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.289+0000 7f211dd9f640 1 -- 172.21.15.67:0/1525039189 wait complete. 2024-07-31T14:49:08.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.290+0000 7f211dd9f640 1 Processor -- start 2024-07-31T14:49:08.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.290+0000 7f211dd9f640 1 -- start start 2024-07-31T14:49:08.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.290+0000 7f211dd9f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2118104080 0x7f2118199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:08.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.290+0000 7f211dd9f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2118104ab0 0x7f211819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:08.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.290+0000 7f211dd9f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211810d630 0x7f211819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:08.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.290+0000 7f211dd9f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f21181132d0 con 0x7f2118104080 2024-07-31T14:49:08.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.290+0000 7f211dd9f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2118113150 con 0x7f211810d630 2024-07-31T14:49:08.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.290+0000 7f211dd9f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2118113450 con 0x7f2118104ab0 2024-07-31T14:49:08.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.290+0000 7f211cd9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2118104080 0x7f2118199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:08.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.290+0000 7f211cd9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2118104080 0x7f2118199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56978/0 (socket says 172.21.15.67:56978) 2024-07-31T14:49:08.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.290+0000 7f211cd9d640 1 -- 172.21.15.67:0/3167881810 learned_addr learned my addr 172.21.15.67:0/3167881810 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:08.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.290+0000 7f2117fff640 1 --2- 172.21.15.67:0/3167881810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2118104ab0 0x7f211819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:08.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.290+0000 7f211d59e640 1 --2- 172.21.15.67:0/3167881810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211810d630 0x7f211819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:08.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.291+0000 7f211cd9d640 1 -- 172.21.15.67:0/3167881810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2118104ab0 msgr2=0x7f211819a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:08.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.291+0000 7f211cd9d640 1 --2- 172.21.15.67:0/3167881810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2118104ab0 0x7f211819a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:08.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.291+0000 7f211cd9d640 1 -- 172.21.15.67:0/3167881810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211810d630 msgr2=0x7f211819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:08.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.291+0000 7f211cd9d640 1 --2- 172.21.15.67:0/3167881810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211810d630 0x7f211819f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:08.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.291+0000 7f211cd9d640 1 -- 172.21.15.67:0/3167881810 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211819fc70 con 0x7f2118104080 2024-07-31T14:49:08.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.291+0000 7f211cd9d640 1 --2- 172.21.15.67:0/3167881810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2118104080 0x7f2118199d00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f210c00d770 tx=0x7f210c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:08.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.291+0000 7f2115ffb640 1 -- 172.21.15.67:0/3167881810 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c017070 con 0x7f2118104080 2024-07-31T14:49:08.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.291+0000 7f2115ffb640 1 -- 172.21.15.67:0/3167881810 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f210c0054d0 con 0x7f2118104080 2024-07-31T14:49:08.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.292+0000 7f2115ffb640 1 -- 172.21.15.67:0/3167881810 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c004c30 con 0x7f2118104080 2024-07-31T14:49:08.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.292+0000 7f211dd9f640 1 -- 172.21.15.67:0/3167881810 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211819ff60 con 0x7f2118104080 2024-07-31T14:49:08.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.292+0000 7f211dd9f640 1 -- 172.21.15.67:0/3167881810 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21181a66c0 con 0x7f2118104080 2024-07-31T14:49:08.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.293+0000 7f211dd9f640 1 -- 172.21.15.67:0/3167881810 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2118107bc0 con 0x7f2118104080 2024-07-31T14:49:08.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.294+0000 7f2115ffb640 1 -- 172.21.15.67:0/3167881810 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f210c004dd0 con 0x7f2118104080 2024-07-31T14:49:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.295+0000 7f2115ffb640 1 --2- 172.21.15.67:0/3167881810 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f21080b3d30 0x7f21080b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.295+0000 7f2117fff640 1 --2- 172.21.15.67:0/3167881810 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f21080b3d30 0x7f21080b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.295+0000 7f2115ffb640 1 -- 172.21.15.67:0/3167881810 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f210c0e3cc0 con 0x7f2118104080 2024-07-31T14:49:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.295+0000 7f2115ffb640 1 --2- 172.21.15.67:0/3167881810 >> [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566] conn(0x7f21080bf1d0 0x7f21080c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.295+0000 7f2115ffb640 1 -- 172.21.15.67:0/3167881810 --> [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21080c1d90 con 0x7f21080bf1d0 2024-07-31T14:49:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.296+0000 7f2115ffb640 1 -- 172.21.15.67:0/3167881810 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f210c0e4110 con 0x7f2118104080 2024-07-31T14:49:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.296+0000 7f211d59e640 1 --2- 172.21.15.67:0/3167881810 >> [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566] conn(0x7f21080bf1d0 0x7f21080c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.296+0000 7f2117fff640 1 --2- 172.21.15.67:0/3167881810 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f21080b3d30 0x7f21080b6240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f211810f9c0 tx=0x7f210400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.296+0000 7f211d59e640 1 --2- 172.21.15.67:0/3167881810 >> [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566] conn(0x7f21080bf1d0 0x7f21080c16c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.297+0000 7f2115ffb640 1 -- 172.21.15.67:0/3167881810 <== osd.7 v2:172.21.15.73:6816/2240196566 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21080c1d90 con 0x7f21080bf1d0 2024-07-31T14:49:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.314+0000 7f211dd9f640 1 -- 172.21.15.67:0/3167881810 --> [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f211806af30 con 0x7f21080bf1d0 2024-07-31T14:49:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.315+0000 7f2115ffb640 1 -- 172.21.15.67:0/3167881810 <== osd.7 v2:172.21.15.73:6816/2240196566 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f211806af30 con 0x7f21080bf1d0 2024-07-31T14:49:08.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.315+0000 7f211dd9f640 1 -- 172.21.15.67:0/3167881810 >> [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566] conn(0x7f21080bf1d0 msgr2=0x7f21080c16c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:08.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.315+0000 7f211dd9f640 1 --2- 172.21.15.67:0/3167881810 >> [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566] conn(0x7f21080bf1d0 0x7f21080c16c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:08.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.316+0000 7f211dd9f640 1 -- 172.21.15.67:0/3167881810 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f21080b3d30 msgr2=0x7f21080b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:08.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.316+0000 7f211dd9f640 1 --2- 172.21.15.67:0/3167881810 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f21080b3d30 0x7f21080b6240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f211810f9c0 tx=0x7f210400b390 comp rx=0 tx=0).stop 2024-07-31T14:49:08.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.316+0000 7f211dd9f640 1 -- 172.21.15.67:0/3167881810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2118104080 msgr2=0x7f2118199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:08.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.316+0000 7f211dd9f640 1 --2- 172.21.15.67:0/3167881810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2118104080 0x7f2118199d00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f210c00d770 tx=0x7f210c00dc30 comp rx=0 tx=0).stop 2024-07-31T14:49:08.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.316+0000 7f211d59e640 1 -- 172.21.15.67:0/3167881810 reap_dead start 2024-07-31T14:49:08.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.316+0000 7f211dd9f640 1 -- 172.21.15.67:0/3167881810 shutdown_connections 2024-07-31T14:49:08.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.316+0000 7f211dd9f640 1 -- 172.21.15.67:0/3167881810 >> 172.21.15.67:0/3167881810 conn(0x7f21180fdcf0 msgr2=0x7f2118107160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:08.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.316+0000 7f211dd9f640 1 -- 172.21.15.67:0/3167881810 shutdown_connections 2024-07-31T14:49:08.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:07.316+0000 7f211dd9f640 1 -- 172.21.15.67:0/3167881810 wait complete. 2024-07-31T14:49:08.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:08 smithi067 ceph-mon[32750]: pgmap v341: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-31T14:49:09.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:08 smithi073 ceph-mon[36714]: pgmap v341: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-31T14:49:09.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:08 smithi077 ceph-mon[37327]: pgmap v341: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-31T14:49:09.263 INFO:teuthology.orchestra.run.smithi067.stdout:210453397563 2024-07-31T14:49:09.264 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-07-31T14:49:09.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.436+0000 7f252424b640 1 -- 172.21.15.67:0/2477773034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c10fbb0 msgr2=0x7f251c111fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:09.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.436+0000 7f252424b640 1 --2- 172.21.15.67:0/2477773034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c10fbb0 0x7f251c111fd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2514009ca0 tx=0x7f251402a950 comp rx=0 tx=0).stop 2024-07-31T14:49:09.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.436+0000 7f252424b640 1 -- 172.21.15.67:0/2477773034 shutdown_connections 2024-07-31T14:49:09.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.436+0000 7f252424b640 1 --2- 172.21.15.67:0/2477773034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c10fbb0 0x7f251c111fd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:09.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.436+0000 7f252424b640 1 --2- 172.21.15.67:0/2477773034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f251c102950 0x7f251c10f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:09.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.436+0000 7f252424b640 1 --2- 172.21.15.67:0/2477773034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f251c102000 0x7f251c102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.436+0000 7f252424b640 1 -- 172.21.15.67:0/2477773034 >> 172.21.15.67:0/2477773034 conn(0x7f251c079160 msgr2=0x7f251c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.437+0000 7f252424b640 1 -- 172.21.15.67:0/2477773034 shutdown_connections 2024-07-31T14:49:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.437+0000 7f252424b640 1 -- 172.21.15.67:0/2477773034 wait complete. 2024-07-31T14:49:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.437+0000 7f252424b640 1 Processor -- start 2024-07-31T14:49:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f252424b640 1 -- start start 2024-07-31T14:49:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f252424b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c102000 0x7f251c19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f252424b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f251c102950 0x7f251c19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f252424b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f251c10fbb0 0x7f251c1a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f252424b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f251c115650 con 0x7f251c102000 2024-07-31T14:49:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f252424b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1154d0 con 0x7f251c10fbb0 2024-07-31T14:49:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f252424b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1157d0 con 0x7f251c102950 2024-07-31T14:49:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f2521fe7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c102000 0x7f251c19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f2521fe7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c102000 0x7f251c19c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57002/0 (socket says 172.21.15.67:57002) 2024-07-31T14:49:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f2521fe7640 1 -- 172.21.15.67:0/414495195 learned_addr learned my addr 172.21.15.67:0/414495195 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f25217e6640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f251c102950 0x7f251c19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f25227e8640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f251c10fbb0 0x7f251c1a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f2521fe7640 1 -- 172.21.15.67:0/414495195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f251c102950 msgr2=0x7f251c19c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f2521fe7640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f251c102950 0x7f251c19c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f2521fe7640 1 -- 172.21.15.67:0/414495195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f251c10fbb0 msgr2=0x7f251c1a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.438+0000 7f2521fe7640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f251c10fbb0 0x7f251c1a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.439+0000 7f2521fe7640 1 -- 172.21.15.67:0/414495195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c1a2110 con 0x7f251c102000 2024-07-31T14:49:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.439+0000 7f25227e8640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f251c10fbb0 0x7f251c1a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:49:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.439+0000 7f2521fe7640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c102000 0x7f251c19c1a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f251000e9f0 tx=0x7f251000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.439+0000 7f250effd640 1 -- 172.21.15.67:0/414495195 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251000cd80 con 0x7f251c102000 2024-07-31T14:49:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.439+0000 7f250effd640 1 -- 172.21.15.67:0/414495195 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25100108e0 con 0x7f251c102000 2024-07-31T14:49:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.439+0000 7f252424b640 1 -- 172.21.15.67:0/414495195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251c1a2400 con 0x7f251c102000 2024-07-31T14:49:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.439+0000 7f250effd640 1 -- 172.21.15.67:0/414495195 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510018980 con 0x7f251c102000 2024-07-31T14:49:09.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.439+0000 7f252424b640 1 -- 172.21.15.67:0/414495195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251c1a8b60 con 0x7f251c102000 2024-07-31T14:49:09.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.440+0000 7f252424b640 1 -- 172.21.15.67:0/414495195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f251c10a160 con 0x7f251c102000 2024-07-31T14:49:09.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.442+0000 7f250effd640 1 -- 172.21.15.67:0/414495195 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f2510010430 con 0x7f251c102000 2024-07-31T14:49:09.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.443+0000 7f250effd640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25080b3cb0 0x7f25080b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:09.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.443+0000 7f25217e6640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25080b3cb0 0x7f25080b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:09.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.443+0000 7f250effd640 1 -- 172.21.15.67:0/414495195 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f25100e4d70 con 0x7f251c102000 2024-07-31T14:49:09.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.444+0000 7f250effd640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318] conn(0x7f25080bf150 0x7f25080c1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:09.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.444+0000 7f250effd640 1 -- 172.21.15.67:0/414495195 --> [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25080c1d10 con 0x7f25080bf150 2024-07-31T14:49:09.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.444+0000 7f250effd640 1 -- 172.21.15.67:0/414495195 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f25100e5120 con 0x7f251c102000 2024-07-31T14:49:09.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.444+0000 7f25227e8640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318] conn(0x7f25080bf150 0x7f25080c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:09.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.444+0000 7f25217e6640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25080b3cb0 0x7f25080b61c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f251c1039c0 tx=0x7f2518005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:09.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.445+0000 7f25227e8640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318] conn(0x7f25080bf150 0x7f25080c1640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:09.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.445+0000 7f250effd640 1 -- 172.21.15.67:0/414495195 <== osd.1 v2:172.21.15.73:6800/1103939318 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f25080c1d10 con 0x7f25080bf150 2024-07-31T14:49:09.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.461+0000 7f252424b640 1 -- 172.21.15.67:0/414495195 --> [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f251c06af90 con 0x7f25080bf150 2024-07-31T14:49:09.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.462+0000 7f250effd640 1 -- 172.21.15.67:0/414495195 <== osd.1 v2:172.21.15.73:6800/1103939318 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f251c06af90 con 0x7f25080bf150 2024-07-31T14:49:09.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.462+0000 7f252424b640 1 -- 172.21.15.67:0/414495195 >> [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318] conn(0x7f25080bf150 msgr2=0x7f25080c1640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:09.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.462+0000 7f252424b640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318] conn(0x7f25080bf150 0x7f25080c1640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:09.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.462+0000 7f252424b640 1 -- 172.21.15.67:0/414495195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25080b3cb0 msgr2=0x7f25080b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:09.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.462+0000 7f252424b640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25080b3cb0 0x7f25080b61c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f251c1039c0 tx=0x7f2518005f90 comp rx=0 tx=0).stop 2024-07-31T14:49:09.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.462+0000 7f252424b640 1 -- 172.21.15.67:0/414495195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c102000 msgr2=0x7f251c19c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:09.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.462+0000 7f252424b640 1 --2- 172.21.15.67:0/414495195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c102000 0x7f251c19c1a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f251000e9f0 tx=0x7f251000eeb0 comp rx=0 tx=0).stop 2024-07-31T14:49:09.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.462+0000 7f25227e8640 1 -- 172.21.15.67:0/414495195 reap_dead start 2024-07-31T14:49:09.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.462+0000 7f252424b640 1 -- 172.21.15.67:0/414495195 shutdown_connections 2024-07-31T14:49:09.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.462+0000 7f252424b640 1 -- 172.21.15.67:0/414495195 >> 172.21.15.67:0/414495195 conn(0x7f251c079160 msgr2=0x7f251c1097c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:09.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.462+0000 7f252424b640 1 -- 172.21.15.67:0/414495195 shutdown_connections 2024-07-31T14:49:09.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:09.462+0000 7f252424b640 1 -- 172.21.15.67:0/414495195 wait complete. 2024-07-31T14:49:10.207 INFO:teuthology.orchestra.run.smithi067.stdout:38654705756 2024-07-31T14:49:10.207 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-07-31T14:49:11.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:10 smithi073 ceph-mon[36714]: pgmap v342: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:11.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:10 smithi077 ceph-mon[37327]: pgmap v342: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:11.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:10 smithi067 ceph-mon[32750]: pgmap v342: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:11.304 INFO:teuthology.orchestra.run.smithi067.stdout:60129542233 2024-07-31T14:49:11.304 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-07-31T14:49:11.543 INFO:teuthology.orchestra.run.smithi067.stdout:141733920842 2024-07-31T14:49:11.543 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-07-31T14:49:11.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.987+0000 7fe13e34b640 1 -- 172.21.15.67:0/1205994833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe138102490 msgr2=0x7fe1381028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:11.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.987+0000 7fe13e34b640 1 --2- 172.21.15.67:0/1205994833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe138102490 0x7fe1381028a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe128009900 tx=0x7fe12802a410 comp rx=0 tx=0).stop 2024-07-31T14:49:11.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.987+0000 7fe13e34b640 1 -- 172.21.15.67:0/1205994833 shutdown_connections 2024-07-31T14:49:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.987+0000 7fe13e34b640 1 --2- 172.21.15.67:0/1205994833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe138108720 0x7fe13810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.987+0000 7fe13e34b640 1 --2- 172.21.15.67:0/1205994833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe138107dd0 0x7fe1381081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.987+0000 7fe13e34b640 1 --2- 172.21.15.67:0/1205994833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe138102490 0x7fe1381028a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.987+0000 7fe13e34b640 1 -- 172.21.15.67:0/1205994833 >> 172.21.15.67:0/1205994833 conn(0x7fe138079160 msgr2=0x7fe138079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.987+0000 7fe13e34b640 1 -- 172.21.15.67:0/1205994833 shutdown_connections 2024-07-31T14:49:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.987+0000 7fe13e34b640 1 -- 172.21.15.67:0/1205994833 wait complete. 2024-07-31T14:49:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.988+0000 7fe13e34b640 1 Processor -- start 2024-07-31T14:49:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.988+0000 7fe13e34b640 1 -- start start 2024-07-31T14:49:11.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.989+0000 7fe13e34b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe138102490 0x7fe138199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:11.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.989+0000 7fe13e34b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe138107dd0 0x7fe13819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:11.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.989+0000 7fe13e34b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe138108720 0x7fe13819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:11.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.989+0000 7fe1377fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe138107dd0 0x7fe13819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:11.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.989+0000 7fe137fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe138102490 0x7fe138199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:11.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.989+0000 7fe1377fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe138107dd0 0x7fe13819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35304/0 (socket says 172.21.15.67:35304) 2024-07-31T14:49:11.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.989+0000 7fe1377fe640 1 -- 172.21.15.67:0/3910306505 learned_addr learned my addr 172.21.15.67:0/3910306505 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:11.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.989+0000 7fe13e34b640 1 -- 172.21.15.67:0/3910306505 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe1381134b0 con 0x7fe138107dd0 2024-07-31T14:49:11.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.989+0000 7fe13e34b640 1 -- 172.21.15.67:0/3910306505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe138113330 con 0x7fe138108720 2024-07-31T14:49:11.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.989+0000 7fe13e34b640 1 -- 172.21.15.67:0/3910306505 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe138113630 con 0x7fe138102490 2024-07-31T14:49:11.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.990+0000 7fe13c8e8640 1 --2- 172.21.15.67:0/3910306505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe138108720 0x7fe13819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:11.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.990+0000 7fe137fff640 1 -- 172.21.15.67:0/3910306505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe138108720 msgr2=0x7fe13819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:11.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.990+0000 7fe137fff640 1 --2- 172.21.15.67:0/3910306505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe138108720 0x7fe13819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:11.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.990+0000 7fe137fff640 1 -- 172.21.15.67:0/3910306505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe138107dd0 msgr2=0x7fe13819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:11.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.990+0000 7fe137fff640 1 --2- 172.21.15.67:0/3910306505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe138107dd0 0x7fe13819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:11.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.990+0000 7fe137fff640 1 -- 172.21.15.67:0/3910306505 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13819fd90 con 0x7fe138102490 2024-07-31T14:49:11.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.990+0000 7fe137fff640 1 --2- 172.21.15.67:0/3910306505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe138102490 0x7fe138199ee0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe12802a930 tx=0x7fe128004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:11.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.991+0000 7fe1357fa640 1 -- 172.21.15.67:0/3910306505 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12803f030 con 0x7fe138102490 2024-07-31T14:49:11.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.991+0000 7fe13e34b640 1 -- 172.21.15.67:0/3910306505 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1381a0020 con 0x7fe138102490 2024-07-31T14:49:11.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.991+0000 7fe1357fa640 1 -- 172.21.15.67:0/3910306505 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe128003cb0 con 0x7fe138102490 2024-07-31T14:49:11.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.991+0000 7fe1357fa640 1 -- 172.21.15.67:0/3910306505 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe128035070 con 0x7fe138102490 2024-07-31T14:49:11.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.991+0000 7fe13e34b640 1 -- 172.21.15.67:0/3910306505 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1381a0500 con 0x7fe138102490 2024-07-31T14:49:11.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.992+0000 7fe13e34b640 1 -- 172.21.15.67:0/3910306505 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe138063500 con 0x7fe138102490 2024-07-31T14:49:11.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.993+0000 7fe1357fa640 1 -- 172.21.15.67:0/3910306505 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fe128042030 con 0x7fe138102490 2024-07-31T14:49:11.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.994+0000 7fe1357fa640 1 --2- 172.21.15.67:0/3910306505 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe1200b3ce0 0x7fe1200b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:11.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.994+0000 7fe1377fe640 1 --2- 172.21.15.67:0/3910306505 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe1200b3ce0 0x7fe1200b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:11.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.994+0000 7fe1357fa640 1 -- 172.21.15.67:0/3910306505 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fe128104c50 con 0x7fe138102490 2024-07-31T14:49:11.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.995+0000 7fe1357fa640 1 --2- 172.21.15.67:0/3910306505 >> [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997] conn(0x7fe1200bf180 0x7fe1200c1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:11.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.995+0000 7fe1357fa640 1 -- 172.21.15.67:0/3910306505 --> [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe128040e70 con 0x7fe1200bf180 2024-07-31T14:49:11.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.995+0000 7fe1357fa640 1 -- 172.21.15.67:0/3910306505 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fe128105000 con 0x7fe138102490 2024-07-31T14:49:11.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.995+0000 7fe13c8e8640 1 --2- 172.21.15.67:0/3910306505 >> [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997] conn(0x7fe1200bf180 0x7fe1200c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:11.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.995+0000 7fe1377fe640 1 --2- 172.21.15.67:0/3910306505 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe1200b3ce0 0x7fe1200b61f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe1381005a0 tx=0x7fe12c00a380 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:11.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.995+0000 7fe13c8e8640 1 --2- 172.21.15.67:0/3910306505 >> [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997] conn(0x7fe1200bf180 0x7fe1200c1670 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:11.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:11.996+0000 7fe1357fa640 1 -- 172.21.15.67:0/3910306505 <== osd.10 v2:172.21.15.73:6824/1494741997 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe128040e70 con 0x7fe1200bf180 2024-07-31T14:49:12.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.011+0000 7fe13e34b640 1 -- 172.21.15.67:0/3910306505 --> [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe138063980 con 0x7fe1200bf180 2024-07-31T14:49:12.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.012+0000 7fe1357fa640 1 -- 172.21.15.67:0/3910306505 <== osd.10 v2:172.21.15.73:6824/1494741997 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe138063980 con 0x7fe1200bf180 2024-07-31T14:49:12.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.012+0000 7fe13e34b640 1 -- 172.21.15.67:0/3910306505 >> [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997] conn(0x7fe1200bf180 msgr2=0x7fe1200c1670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:12.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.012+0000 7fe13e34b640 1 --2- 172.21.15.67:0/3910306505 >> [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997] conn(0x7fe1200bf180 0x7fe1200c1670 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:12.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.012+0000 7fe13e34b640 1 -- 172.21.15.67:0/3910306505 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe1200b3ce0 msgr2=0x7fe1200b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:12.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.012+0000 7fe13e34b640 1 --2- 172.21.15.67:0/3910306505 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe1200b3ce0 0x7fe1200b61f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe1381005a0 tx=0x7fe12c00a380 comp rx=0 tx=0).stop 2024-07-31T14:49:12.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.012+0000 7fe13e34b640 1 -- 172.21.15.67:0/3910306505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe138102490 msgr2=0x7fe138199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:12.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.012+0000 7fe13e34b640 1 --2- 172.21.15.67:0/3910306505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe138102490 0x7fe138199ee0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe12802a930 tx=0x7fe128004270 comp rx=0 tx=0).stop 2024-07-31T14:49:12.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.012+0000 7fe13c8e8640 1 -- 172.21.15.67:0/3910306505 reap_dead start 2024-07-31T14:49:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.012+0000 7fe13e34b640 1 -- 172.21.15.67:0/3910306505 shutdown_connections 2024-07-31T14:49:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.012+0000 7fe13e34b640 1 -- 172.21.15.67:0/3910306505 >> 172.21.15.67:0/3910306505 conn(0x7fe138079160 msgr2=0x7fe138100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.012+0000 7fe13e34b640 1 -- 172.21.15.67:0/3910306505 shutdown_connections 2024-07-31T14:49:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.012+0000 7fe13e34b640 1 -- 172.21.15.67:0/3910306505 wait complete. 2024-07-31T14:49:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.517+0000 7f72a74d6640 1 -- 172.21.15.67:0/800664498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a0108730 msgr2=0x7f72a010ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.517+0000 7f72a74d6640 1 --2- 172.21.15.67:0/800664498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a0108730 0x7f72a010ef60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7298009ca0 tx=0x7f729802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:49:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.517+0000 7f72a74d6640 1 -- 172.21.15.67:0/800664498 shutdown_connections 2024-07-31T14:49:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.517+0000 7f72a74d6640 1 --2- 172.21.15.67:0/800664498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a0108730 0x7f72a010ef60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.517+0000 7f72a74d6640 1 --2- 172.21.15.67:0/800664498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a0107de0 0x7f72a01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.517+0000 7f72a74d6640 1 --2- 172.21.15.67:0/800664498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72a01024a0 0x7f72a01028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.517+0000 7f72a74d6640 1 -- 172.21.15.67:0/800664498 >> 172.21.15.67:0/800664498 conn(0x7f72a0079160 msgr2=0x7f72a0079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.518+0000 7f72a74d6640 1 -- 172.21.15.67:0/800664498 shutdown_connections 2024-07-31T14:49:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.518+0000 7f72a74d6640 1 -- 172.21.15.67:0/800664498 wait complete. 2024-07-31T14:49:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.518+0000 7f72a74d6640 1 Processor -- start 2024-07-31T14:49:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.518+0000 7f72a74d6640 1 -- start start 2024-07-31T14:49:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a74d6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a01024a0 0x7f72a0199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a74d6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72a0107de0 0x7f72a019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a74d6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a0108730 0x7f72a019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a5a73640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a0108730 0x7f72a019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a74d6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0112390 con 0x7f72a0108730 2024-07-31T14:49:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a74d6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0112210 con 0x7f72a01024a0 2024-07-31T14:49:12.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a74d6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0112510 con 0x7f72a0107de0 2024-07-31T14:49:12.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a5a73640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a0108730 0x7f72a019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35328/0 (socket says 172.21.15.67:35328) 2024-07-31T14:49:12.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a5a73640 1 -- 172.21.15.67:0/865307131 learned_addr learned my addr 172.21.15.67:0/865307131 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:12.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a5272640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a01024a0 0x7f72a0199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:12.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a4a71640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72a0107de0 0x7f72a019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:12.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a5a73640 1 -- 172.21.15.67:0/865307131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72a0107de0 msgr2=0x7f72a019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:12.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a5a73640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72a0107de0 0x7f72a019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:12.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a5a73640 1 -- 172.21.15.67:0/865307131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a01024a0 msgr2=0x7f72a0199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:12.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a5a73640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a01024a0 0x7f72a0199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:12.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a5a73640 1 -- 172.21.15.67:0/865307131 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a019fd00 con 0x7f72a0108730 2024-07-31T14:49:12.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.519+0000 7f72a5272640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a01024a0 0x7f72a0199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:49:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.520+0000 7f72a5a73640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a0108730 0x7f72a019f690 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7298005cc0 tx=0x7f729802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.520+0000 7f72967fc640 1 -- 172.21.15.67:0/865307131 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729804c070 con 0x7f72a0108730 2024-07-31T14:49:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.520+0000 7f72a74d6640 1 -- 172.21.15.67:0/865307131 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a019ff90 con 0x7f72a0108730 2024-07-31T14:49:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.520+0000 7f72967fc640 1 -- 172.21.15.67:0/865307131 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f729803fc60 con 0x7f72a0108730 2024-07-31T14:49:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.520+0000 7f72967fc640 1 -- 172.21.15.67:0/865307131 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72980047c0 con 0x7f72a0108730 2024-07-31T14:49:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.520+0000 7f72a74d6640 1 -- 172.21.15.67:0/865307131 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a00fef00 con 0x7f72a0108730 2024-07-31T14:49:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.521+0000 7f72a74d6640 1 -- 172.21.15.67:0/865307131 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f72a0063580 con 0x7f72a0108730 2024-07-31T14:49:12.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.523+0000 7f72967fc640 1 -- 172.21.15.67:0/865307131 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f729804f070 con 0x7f72a0108730 2024-07-31T14:49:12.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.523+0000 7f72967fc640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f728c0b3ce0 0x7f728c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:12.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.524+0000 7f72a5272640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f728c0b3ce0 0x7f728c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:12.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.524+0000 7f72967fc640 1 -- 172.21.15.67:0/865307131 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f7298109660 con 0x7f72a0108730 2024-07-31T14:49:12.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.524+0000 7f72967fc640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409] conn(0x7f728c0bf180 0x7f728c0c1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:12.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.524+0000 7f72967fc640 1 -- 172.21.15.67:0/865307131 --> [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f72980020b0 con 0x7f728c0bf180 2024-07-31T14:49:12.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.524+0000 7f72967fc640 1 -- 172.21.15.67:0/865307131 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f7298109ab0 con 0x7f72a0108730 2024-07-31T14:49:12.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.524+0000 7f72a4a71640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409] conn(0x7f728c0bf180 0x7f728c0c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:12.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.524+0000 7f72a5272640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f728c0b3ce0 0x7f728c0b61f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f729c00afb0 tx=0x7f729c00a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:12.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.524+0000 7f72a4a71640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409] conn(0x7f728c0bf180 0x7f728c0c1670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:12.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.526+0000 7f72967fc640 1 -- 172.21.15.67:0/865307131 <== osd.3 v2:172.21.15.67:6810/3100079409 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72980020b0 con 0x7f728c0bf180 2024-07-31T14:49:12.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.540+0000 7f72a74d6640 1 -- 172.21.15.67:0/865307131 --> [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f72a0063980 con 0x7f728c0bf180 2024-07-31T14:49:12.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.540+0000 7f72967fc640 1 -- 172.21.15.67:0/865307131 <== osd.3 v2:172.21.15.67:6810/3100079409 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f72a0063980 con 0x7f728c0bf180 2024-07-31T14:49:12.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.541+0000 7f72a74d6640 1 -- 172.21.15.67:0/865307131 >> [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409] conn(0x7f728c0bf180 msgr2=0x7f728c0c1670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:12.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.541+0000 7f72a74d6640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409] conn(0x7f728c0bf180 0x7f728c0c1670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:12.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.541+0000 7f72a74d6640 1 -- 172.21.15.67:0/865307131 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f728c0b3ce0 msgr2=0x7f728c0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:12.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.541+0000 7f72a74d6640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f728c0b3ce0 0x7f728c0b61f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f729c00afb0 tx=0x7f729c00a2d0 comp rx=0 tx=0).stop 2024-07-31T14:49:12.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.541+0000 7f72a74d6640 1 -- 172.21.15.67:0/865307131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a0108730 msgr2=0x7f72a019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:12.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.541+0000 7f72a74d6640 1 --2- 172.21.15.67:0/865307131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a0108730 0x7f72a019f690 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7298005cc0 tx=0x7f729802f470 comp rx=0 tx=0).stop 2024-07-31T14:49:12.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.541+0000 7f72a5a73640 1 -- 172.21.15.67:0/865307131 reap_dead start 2024-07-31T14:49:12.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.541+0000 7f72a74d6640 1 -- 172.21.15.67:0/865307131 shutdown_connections 2024-07-31T14:49:12.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.541+0000 7f72a74d6640 1 -- 172.21.15.67:0/865307131 >> 172.21.15.67:0/865307131 conn(0x7f72a0079160 msgr2=0x7f72a0101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:12.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.541+0000 7f72a74d6640 1 -- 172.21.15.67:0/865307131 shutdown_connections 2024-07-31T14:49:12.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:12.541+0000 7f72a74d6640 1 -- 172.21.15.67:0/865307131 wait complete. 2024-07-31T14:49:13.095 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:12 smithi067 ceph-mon[32750]: pgmap v343: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:49:13.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:12 smithi073 ceph-mon[36714]: pgmap v343: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:49:13.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:12 smithi077 ceph-mon[37327]: pgmap v343: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:49:13.418 INFO:teuthology.orchestra.run.smithi067.stdout:283467841581 2024-07-31T14:49:13.418 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-07-31T14:49:13.804 INFO:teuthology.orchestra.run.smithi067.stdout:188978561088 2024-07-31T14:49:13.805 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-07-31T14:49:14.528 INFO:teuthology.orchestra.run.smithi067.stdout:236223201335 2024-07-31T14:49:14.528 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-07-31T14:49:14.992 INFO:teuthology.orchestra.run.smithi067.stdout:115964117071 2024-07-31T14:49:14.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-07-31T14:49:15.086 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:14 smithi067 ceph-mon[32750]: pgmap v344: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-07-31T14:49:15.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:14 smithi073 ceph-mon[36714]: pgmap v344: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-07-31T14:49:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:14 smithi077 ceph-mon[37327]: pgmap v344: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-07-31T14:49:15.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.424+0000 7fc5b3784640 1 -- 172.21.15.67:0/3441040423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5ac102490 msgr2=0x7fc5ac1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:15.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.424+0000 7fc5b3784640 1 --2- 172.21.15.67:0/3441040423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5ac102490 0x7fc5ac1028a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009900 tx=0x7fc5a802a410 comp rx=0 tx=0).stop 2024-07-31T14:49:15.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.424+0000 7fc5b3784640 1 -- 172.21.15.67:0/3441040423 shutdown_connections 2024-07-31T14:49:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.424+0000 7fc5b3784640 1 --2- 172.21.15.67:0/3441040423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5ac108720 0x7fc5ac10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.424+0000 7fc5b3784640 1 --2- 172.21.15.67:0/3441040423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5ac107dd0 0x7fc5ac1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.424+0000 7fc5b3784640 1 --2- 172.21.15.67:0/3441040423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5ac102490 0x7fc5ac1028a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.424+0000 7fc5b3784640 1 -- 172.21.15.67:0/3441040423 >> 172.21.15.67:0/3441040423 conn(0x7fc5ac079160 msgr2=0x7fc5ac079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.425+0000 7fc5b3784640 1 -- 172.21.15.67:0/3441040423 shutdown_connections 2024-07-31T14:49:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.425+0000 7fc5b3784640 1 -- 172.21.15.67:0/3441040423 wait complete. 2024-07-31T14:49:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b3784640 1 Processor -- start 2024-07-31T14:49:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b3784640 1 -- start start 2024-07-31T14:49:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b3784640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5ac102490 0x7fc5ac199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b3784640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5ac107dd0 0x7fc5ac19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b3784640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5ac108720 0x7fc5ac19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b3784640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac112380 con 0x7fc5ac107dd0 2024-07-31T14:49:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b3784640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac112200 con 0x7fc5ac102490 2024-07-31T14:49:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b3784640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac112500 con 0x7fc5ac108720 2024-07-31T14:49:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b0d1f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5ac107dd0 0x7fc5ac19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b0d1f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5ac107dd0 0x7fc5ac19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35348/0 (socket says 172.21.15.67:35348) 2024-07-31T14:49:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b0d1f640 1 -- 172.21.15.67:0/3043764446 learned_addr learned my addr 172.21.15.67:0/3043764446 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b1d21640 1 --2- 172.21.15.67:0/3043764446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5ac108720 0x7fc5ac19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b1520640 1 --2- 172.21.15.67:0/3043764446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5ac102490 0x7fc5ac199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b0d1f640 1 -- 172.21.15.67:0/3043764446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5ac108720 msgr2=0x7fc5ac19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b0d1f640 1 --2- 172.21.15.67:0/3043764446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5ac108720 0x7fc5ac19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b0d1f640 1 -- 172.21.15.67:0/3043764446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5ac102490 msgr2=0x7fc5ac199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b0d1f640 1 --2- 172.21.15.67:0/3043764446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5ac102490 0x7fc5ac199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.426+0000 7fc5b0d1f640 1 -- 172.21.15.67:0/3043764446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac19fdb0 con 0x7fc5ac107dd0 2024-07-31T14:49:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.427+0000 7fc5b0d1f640 1 --2- 172.21.15.67:0/3043764446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5ac107dd0 0x7fc5ac19a3e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00d770 tx=0x7fc59c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.427+0000 7fc5a27fc640 1 -- 172.21.15.67:0/3043764446 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c017070 con 0x7fc5ac107dd0 2024-07-31T14:49:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.427+0000 7fc5b3784640 1 -- 172.21.15.67:0/3043764446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac1a00a0 con 0x7fc5ac107dd0 2024-07-31T14:49:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.427+0000 7fc5a27fc640 1 -- 172.21.15.67:0/3043764446 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc59c009c30 con 0x7fc5ac107dd0 2024-07-31T14:49:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.427+0000 7fc5a27fc640 1 -- 172.21.15.67:0/3043764446 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00f620 con 0x7fc5ac107dd0 2024-07-31T14:49:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.427+0000 7fc5b3784640 1 -- 172.21.15.67:0/3043764446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac1a6800 con 0x7fc5ac107dd0 2024-07-31T14:49:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.428+0000 7fc5b3784640 1 -- 172.21.15.67:0/3043764446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc5ac06abe0 con 0x7fc5ac107dd0 2024-07-31T14:49:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.430+0000 7fc5a27fc640 1 -- 172.21.15.67:0/3043764446 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fc59c01a070 con 0x7fc5ac107dd0 2024-07-31T14:49:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.430+0000 7fc5a27fc640 1 --2- 172.21.15.67:0/3043764446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc5980b3cb0 0x7fc5980b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.430+0000 7fc5b1520640 1 --2- 172.21.15.67:0/3043764446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc5980b3cb0 0x7fc5980b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.430+0000 7fc5a27fc640 1 -- 172.21.15.67:0/3043764446 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc59c0e4260 con 0x7fc5ac107dd0 2024-07-31T14:49:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.431+0000 7fc5a27fc640 1 --2- 172.21.15.67:0/3043764446 >> [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060] conn(0x7fc5980bf150 0x7fc5980c1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.431+0000 7fc5a27fc640 1 -- 172.21.15.67:0/3043764446 --> [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc59c004470 con 0x7fc5980bf150 2024-07-31T14:49:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.431+0000 7fc5a27fc640 1 -- 172.21.15.67:0/3043764446 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc59c0e46b0 con 0x7fc5ac107dd0 2024-07-31T14:49:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.431+0000 7fc5b1d21640 1 --2- 172.21.15.67:0/3043764446 >> [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060] conn(0x7fc5980bf150 0x7fc5980c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.431+0000 7fc5b1520640 1 --2- 172.21.15.67:0/3043764446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc5980b3cb0 0x7fc5980b61c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8031440 tx=0x7fc5a8005d50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.431+0000 7fc5b1d21640 1 --2- 172.21.15.67:0/3043764446 >> [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060] conn(0x7fc5980bf150 0x7fc5980c1640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.432+0000 7fc5a27fc640 1 -- 172.21.15.67:0/3043764446 <== osd.9 v2:172.21.15.67:6826/3242854060 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc59c004470 con 0x7fc5980bf150 2024-07-31T14:49:15.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.447+0000 7fc5b3784640 1 -- 172.21.15.67:0/3043764446 --> [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc5ac103710 con 0x7fc5980bf150 2024-07-31T14:49:15.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.447+0000 7fc5a27fc640 1 -- 172.21.15.67:0/3043764446 <== osd.9 v2:172.21.15.67:6826/3242854060 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc5ac103710 con 0x7fc5980bf150 2024-07-31T14:49:15.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.447+0000 7fc5b3784640 1 -- 172.21.15.67:0/3043764446 >> [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060] conn(0x7fc5980bf150 msgr2=0x7fc5980c1640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:15.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.447+0000 7fc5b3784640 1 --2- 172.21.15.67:0/3043764446 >> [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060] conn(0x7fc5980bf150 0x7fc5980c1640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:15.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.447+0000 7fc5b3784640 1 -- 172.21.15.67:0/3043764446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc5980b3cb0 msgr2=0x7fc5980b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:15.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.448+0000 7fc5b3784640 1 --2- 172.21.15.67:0/3043764446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc5980b3cb0 0x7fc5980b61c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8031440 tx=0x7fc5a8005d50 comp rx=0 tx=0).stop 2024-07-31T14:49:15.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.448+0000 7fc5b3784640 1 -- 172.21.15.67:0/3043764446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5ac107dd0 msgr2=0x7fc5ac19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:15.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.448+0000 7fc5b3784640 1 --2- 172.21.15.67:0/3043764446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5ac107dd0 0x7fc5ac19a3e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00d770 tx=0x7fc59c00dc30 comp rx=0 tx=0).stop 2024-07-31T14:49:15.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.448+0000 7fc5b1d21640 1 -- 172.21.15.67:0/3043764446 reap_dead start 2024-07-31T14:49:15.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.448+0000 7fc5b3784640 1 -- 172.21.15.67:0/3043764446 shutdown_connections 2024-07-31T14:49:15.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.448+0000 7fc5b3784640 1 -- 172.21.15.67:0/3043764446 >> 172.21.15.67:0/3043764446 conn(0x7fc5ac079160 msgr2=0x7fc5ac101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:15.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.448+0000 7fc5b3784640 1 -- 172.21.15.67:0/3043764446 shutdown_connections 2024-07-31T14:49:15.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:15.448+0000 7fc5b3784640 1 -- 172.21.15.67:0/3043764446 wait complete. 2024-07-31T14:49:15.896 INFO:teuthology.orchestra.run.smithi067.stdout:309237645351 2024-07-31T14:49:15.896 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-07-31T14:49:16.643 INFO:teuthology.orchestra.run.smithi067.stdout:261993005107 2024-07-31T14:49:16.644 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-07-31T14:49:16.690 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:16.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:16 smithi067 ceph-mon[32750]: pgmap v345: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:49:17.051 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:17.051 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:17.061 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:17.061 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:17.062 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:17.062 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:17.062 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:17.068 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:17.071 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:17.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:16 smithi073 ceph-mon[36714]: pgmap v345: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:49:17.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:16 smithi077 ceph-mon[37327]: pgmap v345: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:49:17.516 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:19.046 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:18 smithi067 ceph-mon[32750]: pgmap v346: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-31T14:49:19.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:18 smithi073 ceph-mon[36714]: pgmap v346: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-31T14:49:19.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:18 smithi077 ceph-mon[37327]: pgmap v346: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-31T14:49:21.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:20 smithi073 ceph-mon[36714]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-31T14:49:21.147 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:20 smithi067 ceph-mon[32750]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-31T14:49:21.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:20 smithi077 ceph-mon[37327]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-31T14:49:23.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:22 smithi067 ceph-mon[32750]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:49:23.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:22 smithi073 ceph-mon[36714]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:49:23.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.231+0000 7f42fea9f640 1 -- 172.21.15.67:0/2079933216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42f8108720 msgr2=0x7f42f810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:23.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.231+0000 7f42fea9f640 1 --2- 172.21.15.67:0/2079933216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42f8108720 0x7f42f810ef50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009ca0 tx=0x7f42e802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:49:23.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.231+0000 7f42fea9f640 1 -- 172.21.15.67:0/2079933216 shutdown_connections 2024-07-31T14:49:23.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.231+0000 7f42fea9f640 1 --2- 172.21.15.67:0/2079933216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42f8108720 0x7f42f810ef50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:23.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.231+0000 7f42fea9f640 1 --2- 172.21.15.67:0/2079933216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42f8107dd0 0x7f42f81081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:23.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.231+0000 7f42fea9f640 1 --2- 172.21.15.67:0/2079933216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8102490 0x7f42f81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:23.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.231+0000 7f42fea9f640 1 -- 172.21.15.67:0/2079933216 >> 172.21.15.67:0/2079933216 conn(0x7f42f8079160 msgr2=0x7f42f8079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:23.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.232+0000 7f42fea9f640 1 -- 172.21.15.67:0/2079933216 shutdown_connections 2024-07-31T14:49:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.232+0000 7f42fea9f640 1 -- 172.21.15.67:0/2079933216 wait complete. 2024-07-31T14:49:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.232+0000 7f42fea9f640 1 Processor -- start 2024-07-31T14:49:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.232+0000 7f42fea9f640 1 -- start start 2024-07-31T14:49:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fea9f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42f8102490 0x7f42f8199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fea9f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8107dd0 0x7f42f819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fea9f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42f8108720 0x7f42f819c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fea9f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8112380 con 0x7f42f8102490 2024-07-31T14:49:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fea9f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8112200 con 0x7f42f8107dd0 2024-07-31T14:49:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fea9f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8112500 con 0x7f42f8108720 2024-07-31T14:49:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fc83b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42f8102490 0x7f42f8199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fc83b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42f8102490 0x7f42f8199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39896/0 (socket says 172.21.15.67:39896) 2024-07-31T14:49:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fc83b640 1 -- 172.21.15.67:0/1043144064 learned_addr learned my addr 172.21.15.67:0/1043144064 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:23.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42f7fff640 1 --2- 172.21.15.67:0/1043144064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8107dd0 0x7f42f819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:23.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fd03c640 1 --2- 172.21.15.67:0/1043144064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42f8108720 0x7f42f819c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:23.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fc83b640 1 -- 172.21.15.67:0/1043144064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42f8108720 msgr2=0x7f42f819c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:23.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fc83b640 1 --2- 172.21.15.67:0/1043144064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42f8108720 0x7f42f819c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:23.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fc83b640 1 -- 172.21.15.67:0/1043144064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8107dd0 msgr2=0x7f42f819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:23.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fc83b640 1 --2- 172.21.15.67:0/1043144064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8107dd0 0x7f42f819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:23.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.233+0000 7f42fc83b640 1 -- 172.21.15.67:0/1043144064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f819cb40 con 0x7f42f8102490 2024-07-31T14:49:23.378 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:22 smithi077 ceph-mon[37327]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:49:23.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.234+0000 7f42fc83b640 1 --2- 172.21.15.67:0/1043144064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42f8102490 0x7f42f8199e30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f42f000d950 tx=0x7f42f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:23.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.234+0000 7f42f5ffb640 1 -- 172.21.15.67:0/1043144064 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0017030 con 0x7f42f8102490 2024-07-31T14:49:23.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.234+0000 7f42fea9f640 1 -- 172.21.15.67:0/1043144064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f819ce30 con 0x7f42f8102490 2024-07-31T14:49:23.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.234+0000 7f42f5ffb640 1 -- 172.21.15.67:0/1043144064 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f0004020 con 0x7f42f8102490 2024-07-31T14:49:23.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.234+0000 7f42f5ffb640 1 -- 172.21.15.67:0/1043144064 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f00047a0 con 0x7f42f8102490 2024-07-31T14:49:23.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.234+0000 7f42fea9f640 1 -- 172.21.15.67:0/1043144064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f81a67d0 con 0x7f42f8102490 2024-07-31T14:49:23.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.235+0000 7f42fea9f640 1 -- 172.21.15.67:0/1043144064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f81028a0 con 0x7f42f8102490 2024-07-31T14:49:23.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.237+0000 7f42f5ffb640 1 -- 172.21.15.67:0/1043144064 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f42f0009a30 con 0x7f42f8102490 2024-07-31T14:49:23.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.237+0000 7f42f5ffb640 1 --2- 172.21.15.67:0/1043144064 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42e40b3cb0 0x7f42e40b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:23.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.237+0000 7f42f7fff640 1 --2- 172.21.15.67:0/1043144064 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42e40b3cb0 0x7f42e40b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:23.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.238+0000 7f42f5ffb640 1 -- 172.21.15.67:0/1043144064 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f42f00e43a0 con 0x7f42f8102490 2024-07-31T14:49:23.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.238+0000 7f42f7fff640 1 --2- 172.21.15.67:0/1043144064 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42e40b3cb0 0x7f42e40b61c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f42f8079690 tx=0x7f42e0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:23.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.241+0000 7f42f5ffb640 1 -- 172.21.15.67:0/1043144064 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f42f00ab400 con 0x7f42f8102490 2024-07-31T14:49:23.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.442+0000 7f42fea9f640 1 -- 172.21.15.67:0/1043144064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f42f81a6ba0 con 0x7f42f8102490 2024-07-31T14:49:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.442+0000 7f42f5ffb640 1 -- 172.21.15.67:0/1043144064 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f42f00aad00 con 0x7f42f8102490 2024-07-31T14:49:23.447 INFO:teuthology.orchestra.run.smithi067.stdout:261993005108 2024-07-31T14:49:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.444+0000 7f42fea9f640 1 -- 172.21.15.67:0/1043144064 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42e40b3cb0 msgr2=0x7f42e40b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.444+0000 7f42fea9f640 1 --2- 172.21.15.67:0/1043144064 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42e40b3cb0 0x7f42e40b61c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f42f8079690 tx=0x7f42e0005f90 comp rx=0 tx=0).stop 2024-07-31T14:49:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.444+0000 7f42fea9f640 1 -- 172.21.15.67:0/1043144064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42f8102490 msgr2=0x7f42f8199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.444+0000 7f42fea9f640 1 --2- 172.21.15.67:0/1043144064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42f8102490 0x7f42f8199e30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f42f000d950 tx=0x7f42f000de10 comp rx=0 tx=0).stop 2024-07-31T14:49:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.445+0000 7f42fea9f640 1 -- 172.21.15.67:0/1043144064 shutdown_connections 2024-07-31T14:49:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.445+0000 7f42fea9f640 1 --2- 172.21.15.67:0/1043144064 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42e40b3cb0 0x7f42e40b61c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.445+0000 7f42fea9f640 1 --2- 172.21.15.67:0/1043144064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42f8108720 0x7f42f819c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.445+0000 7f42fea9f640 1 --2- 172.21.15.67:0/1043144064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8107dd0 0x7f42f819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.445+0000 7f42fea9f640 1 --2- 172.21.15.67:0/1043144064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42f8102490 0x7f42f8199e30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.445+0000 7f42fea9f640 1 -- 172.21.15.67:0/1043144064 >> 172.21.15.67:0/1043144064 conn(0x7f42f8079160 msgr2=0x7f42f8100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.445+0000 7f42fea9f640 1 -- 172.21.15.67:0/1043144064 shutdown_connections 2024-07-31T14:49:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:23.445+0000 7f42fea9f640 1 -- 172.21.15.67:0/1043144064 wait complete. 2024-07-31T14:49:24.066 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:23 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1043144064' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-31T14:49:24.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:23 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1043144064' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-31T14:49:24.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:23 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1043144064' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-31T14:49:25.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:24 smithi073 ceph-mon[36714]: pgmap v349: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:49:25.161 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:24 smithi067 ceph-mon[32750]: pgmap v349: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:49:25.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:24 smithi077 ceph-mon[37327]: pgmap v349: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:49:27.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:26 smithi067 ceph-mon[32750]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-07-31T14:49:27.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:26 smithi073 ceph-mon[36714]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-07-31T14:49:27.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:26 smithi077 ceph-mon[37327]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-07-31T14:49:27.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.453+0000 7fe6e4f12640 1 -- 172.21.15.67:0/695074866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e0102060 msgr2=0x7fe6e0102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:27.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.453+0000 7fe6e4f12640 1 --2- 172.21.15.67:0/695074866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e0102060 0x7fe6e0102470 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009900 tx=0x7fe6d002a480 comp rx=0 tx=0).stop 2024-07-31T14:49:27.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.453+0000 7fe6e4f12640 1 -- 172.21.15.67:0/695074866 shutdown_connections 2024-07-31T14:49:27.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.453+0000 7fe6e4f12640 1 --2- 172.21.15.67:0/695074866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6e010fc10 0x7fe6e0112030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.453+0000 7fe6e4f12640 1 --2- 172.21.15.67:0/695074866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e01029b0 0x7fe6e010f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.453+0000 7fe6e4f12640 1 --2- 172.21.15.67:0/695074866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e0102060 0x7fe6e0102470 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.453+0000 7fe6e4f12640 1 -- 172.21.15.67:0/695074866 >> 172.21.15.67:0/695074866 conn(0x7fe6e0079160 msgr2=0x7fe6e0079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:27.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.453+0000 7fe6e4f12640 1 -- 172.21.15.67:0/695074866 shutdown_connections 2024-07-31T14:49:27.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.453+0000 7fe6e4f12640 1 -- 172.21.15.67:0/695074866 wait complete. 2024-07-31T14:49:27.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.454+0000 7fe6e4f12640 1 Processor -- start 2024-07-31T14:49:27.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.454+0000 7fe6e4f12640 1 -- start start 2024-07-31T14:49:27.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.455+0000 7fe6e4f12640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e0102060 0x7fe6e019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:27.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.455+0000 7fe6e4f12640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e01029b0 0x7fe6e019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:27.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.455+0000 7fe6e4f12640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6e010fc10 0x7fe6e01a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:27.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.455+0000 7fe6e4f12640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e0115640 con 0x7fe6e01029b0 2024-07-31T14:49:27.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.455+0000 7fe6e4f12640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e01154c0 con 0x7fe6e010fc10 2024-07-31T14:49:27.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.455+0000 7fe6e4f12640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e01157c0 con 0x7fe6e0102060 2024-07-31T14:49:27.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.455+0000 7fe6ddd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e01029b0 0x7fe6e019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:27.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.455+0000 7fe6ddd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e01029b0 0x7fe6e019c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39946/0 (socket says 172.21.15.67:39946) 2024-07-31T14:49:27.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.455+0000 7fe6ddd9b640 1 -- 172.21.15.67:0/1111422275 learned_addr learned my addr 172.21.15.67:0/1111422275 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:27.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.455+0000 7fe6de59c640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e0102060 0x7fe6e019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:27.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.455+0000 7fe6ded9d640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6e010fc10 0x7fe6e01a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:27.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.456+0000 7fe6ddd9b640 1 -- 172.21.15.67:0/1111422275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e0102060 msgr2=0x7fe6e019c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.456+0000 7fe6ddd9b640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e0102060 0x7fe6e019c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.456+0000 7fe6ddd9b640 1 -- 172.21.15.67:0/1111422275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6e010fc10 msgr2=0x7fe6e01a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.456+0000 7fe6ddd9b640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6e010fc10 0x7fe6e01a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.456+0000 7fe6ddd9b640 1 -- 172.21.15.67:0/1111422275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e01a1fc0 con 0x7fe6e01029b0 2024-07-31T14:49:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.456+0000 7fe6de59c640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e0102060 0x7fe6e019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:49:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.456+0000 7fe6ddd9b640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e01029b0 0x7fe6e019c5f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400d950 tx=0x7fe6d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.456+0000 7fe6cf7fe640 1 -- 172.21.15.67:0/1111422275 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4017070 con 0x7fe6e01029b0 2024-07-31T14:49:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.456+0000 7fe6e4f12640 1 -- 172.21.15.67:0/1111422275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e01a22b0 con 0x7fe6e01029b0 2024-07-31T14:49:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.456+0000 7fe6cf7fe640 1 -- 172.21.15.67:0/1111422275 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d4009c30 con 0x7fe6e01029b0 2024-07-31T14:49:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.456+0000 7fe6cf7fe640 1 -- 172.21.15.67:0/1111422275 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d400f620 con 0x7fe6e01029b0 2024-07-31T14:49:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.456+0000 7fe6e4f12640 1 -- 172.21.15.67:0/1111422275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e01a8a10 con 0x7fe6e01029b0 2024-07-31T14:49:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.457+0000 7fe6e4f12640 1 -- 172.21.15.67:0/1111422275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e0102470 con 0x7fe6e01029b0 2024-07-31T14:49:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.460+0000 7fe6cf7fe640 1 -- 172.21.15.67:0/1111422275 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fe6d4005030 con 0x7fe6e01029b0 2024-07-31T14:49:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.461+0000 7fe6cf7fe640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe6c80b3cb0 0x7fe6c80b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:27.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.461+0000 7fe6de59c640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe6c80b3cb0 0x7fe6c80b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:27.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.461+0000 7fe6cf7fe640 1 -- 172.21.15.67:0/1111422275 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fe6d40e3d80 con 0x7fe6e01029b0 2024-07-31T14:49:27.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.462+0000 7fe6de59c640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe6c80b3cb0 0x7fe6c80b61c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0031440 tx=0x7fe6d0032ed0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:27.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.464+0000 7fe6cf7fe640 1 -- 172.21.15.67:0/1111422275 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe6d4009de0 con 0x7fe6e01029b0 2024-07-31T14:49:27.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.677+0000 7fe6e4f12640 1 -- 172.21.15.67:0/1111422275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe6e01a8cf0 con 0x7fe6e01029b0 2024-07-31T14:49:27.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.678+0000 7fe6cf7fe640 1 -- 172.21.15.67:0/1111422275 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe6d40aa7f0 con 0x7fe6e01029b0 2024-07-31T14:49:27.679 INFO:teuthology.orchestra.run.smithi067.stdout:115964117074 2024-07-31T14:49:27.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.680+0000 7fe6e4f12640 1 -- 172.21.15.67:0/1111422275 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe6c80b3cb0 msgr2=0x7fe6c80b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:27.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.680+0000 7fe6e4f12640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe6c80b3cb0 0x7fe6c80b61c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0031440 tx=0x7fe6d0032ed0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.680+0000 7fe6e4f12640 1 -- 172.21.15.67:0/1111422275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e01029b0 msgr2=0x7fe6e019c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:27.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.680+0000 7fe6e4f12640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e01029b0 0x7fe6e019c5f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400d950 tx=0x7fe6d400de10 comp rx=0 tx=0).stop 2024-07-31T14:49:27.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.680+0000 7fe6e4f12640 1 -- 172.21.15.67:0/1111422275 shutdown_connections 2024-07-31T14:49:27.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.680+0000 7fe6e4f12640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe6c80b3cb0 0x7fe6c80b61c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.680+0000 7fe6e4f12640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6e010fc10 0x7fe6e01a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.680+0000 7fe6e4f12640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e01029b0 0x7fe6e019c5f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.680+0000 7fe6e4f12640 1 --2- 172.21.15.67:0/1111422275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e0102060 0x7fe6e019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.680+0000 7fe6e4f12640 1 -- 172.21.15.67:0/1111422275 >> 172.21.15.67:0/1111422275 conn(0x7fe6e0079160 msgr2=0x7fe6e0109790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:27.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.680+0000 7fe6e4f12640 1 -- 172.21.15.67:0/1111422275 shutdown_connections 2024-07-31T14:49:27.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.680+0000 7fe6e4f12640 1 -- 172.21.15.67:0/1111422275 wait complete. 2024-07-31T14:49:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.256+0000 7f7639334640 1 -- 172.21.15.67:0/1172642065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76341024b0 msgr2=0x7f76341028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.256+0000 7f7639334640 1 --2- 172.21.15.67:0/1172642065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76341024b0 0x7f76341028c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7624009900 tx=0x7f762402a410 comp rx=0 tx=0).stop 2024-07-31T14:49:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.256+0000 7f7639334640 1 -- 172.21.15.67:0/1172642065 shutdown_connections 2024-07-31T14:49:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.256+0000 7f7639334640 1 --2- 172.21.15.67:0/1172642065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7634108740 0x7f763410ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.256+0000 7f7639334640 1 --2- 172.21.15.67:0/1172642065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7634107df0 0x7f7634108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.256+0000 7f7639334640 1 --2- 172.21.15.67:0/1172642065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76341024b0 0x7f76341028c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.256+0000 7f7639334640 1 -- 172.21.15.67:0/1172642065 >> 172.21.15.67:0/1172642065 conn(0x7f7634079170 msgr2=0x7f7634079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.257+0000 7f7639334640 1 -- 172.21.15.67:0/1172642065 shutdown_connections 2024-07-31T14:49:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.257+0000 7f7639334640 1 -- 172.21.15.67:0/1172642065 wait complete. 2024-07-31T14:49:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.257+0000 7f7639334640 1 Processor -- start 2024-07-31T14:49:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.257+0000 7f7639334640 1 -- start start 2024-07-31T14:49:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f7639334640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76341024b0 0x7f7634199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f7639334640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7634107df0 0x7f763419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f7639334640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7634108740 0x7f763419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f7639334640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f76341123c0 con 0x7f7634108740 2024-07-31T14:49:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f7639334640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7634112240 con 0x7f7634107df0 2024-07-31T14:49:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f7639334640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7634112540 con 0x7f76341024b0 2024-07-31T14:49:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f76337fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7634108740 0x7f763419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f76337fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7634108740 0x7f763419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39910/0 (socket says 172.21.15.67:39910) 2024-07-31T14:49:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f76337fe640 1 -- 172.21.15.67:0/2216451285 learned_addr learned my addr 172.21.15.67:0/2216451285 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f76327fc640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7634107df0 0x7f763419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f7632ffd640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76341024b0 0x7f7634199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f76337fe640 1 -- 172.21.15.67:0/2216451285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76341024b0 msgr2=0x7f7634199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f76337fe640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76341024b0 0x7f7634199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f76337fe640 1 -- 172.21.15.67:0/2216451285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7634107df0 msgr2=0x7f763419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f76337fe640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7634107df0 0x7f763419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f76337fe640 1 -- 172.21.15.67:0/2216451285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763419fe50 con 0x7f7634108740 2024-07-31T14:49:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f7632ffd640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76341024b0 0x7f7634199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:49:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f76327fc640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7634107df0 0x7f763419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:49:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f76337fe640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7634108740 0x7f763419f720 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f762c00eaa0 tx=0x7f762c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.258+0000 7f761bfff640 1 -- 172.21.15.67:0/2216451285 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c00cdb0 con 0x7f7634108740 2024-07-31T14:49:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.259+0000 7f761bfff640 1 -- 172.21.15.67:0/2216451285 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f762c005020 con 0x7f7634108740 2024-07-31T14:49:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.259+0000 7f7639334640 1 -- 172.21.15.67:0/2216451285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76341a0140 con 0x7f7634108740 2024-07-31T14:49:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.259+0000 7f761bfff640 1 -- 172.21.15.67:0/2216451285 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c0116d0 con 0x7f7634108740 2024-07-31T14:49:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.259+0000 7f7639334640 1 -- 172.21.15.67:0/2216451285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76341a68a0 con 0x7f7634108740 2024-07-31T14:49:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.260+0000 7f7639334640 1 -- 172.21.15.67:0/2216451285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7634103730 con 0x7f7634108740 2024-07-31T14:49:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.261+0000 7f761bfff640 1 -- 172.21.15.67:0/2216451285 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f762c0051d0 con 0x7f7634108740 2024-07-31T14:49:27.991 INFO:teuthology.orchestra.run.smithi067.stdout:38654705760 2024-07-31T14:49:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.262+0000 7f761bfff640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f761c0b3cb0 0x7f761c0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.262+0000 7f7632ffd640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f761c0b3cb0 0x7f761c0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.262+0000 7f761bfff640 1 -- 172.21.15.67:0/2216451285 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f762c0e47b0 con 0x7f7634108740 2024-07-31T14:49:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.263+0000 7f7632ffd640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f761c0b3cb0 0x7f761c0b61c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7624031440 tx=0x7f7624009680 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.265+0000 7f761bfff640 1 -- 172.21.15.67:0/2216451285 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f762c016030 con 0x7f7634108740 2024-07-31T14:49:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.478+0000 7f7639334640 1 -- 172.21.15.67:0/2216451285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f76341028c0 con 0x7f7634108740 2024-07-31T14:49:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.478+0000 7f761bfff640 1 -- 172.21.15.67:0/2216451285 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f762c0ab870 con 0x7f7634108740 2024-07-31T14:49:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.480+0000 7f7639334640 1 -- 172.21.15.67:0/2216451285 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f761c0b3cb0 msgr2=0x7f761c0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.480+0000 7f7639334640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f761c0b3cb0 0x7f761c0b61c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7624031440 tx=0x7f7624009680 comp rx=0 tx=0).stop 2024-07-31T14:49:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.480+0000 7f7639334640 1 -- 172.21.15.67:0/2216451285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7634108740 msgr2=0x7f763419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.480+0000 7f7639334640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7634108740 0x7f763419f720 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f762c00eaa0 tx=0x7f762c00ef60 comp rx=0 tx=0).stop 2024-07-31T14:49:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.481+0000 7f7639334640 1 -- 172.21.15.67:0/2216451285 shutdown_connections 2024-07-31T14:49:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.481+0000 7f7639334640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f761c0b3cb0 0x7f761c0b61c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.481+0000 7f7639334640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7634108740 0x7f763419f720 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.481+0000 7f7639334640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7634107df0 0x7f763419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.481+0000 7f7639334640 1 --2- 172.21.15.67:0/2216451285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76341024b0 0x7f7634199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.481+0000 7f7639334640 1 -- 172.21.15.67:0/2216451285 >> 172.21.15.67:0/2216451285 conn(0x7f7634079170 msgr2=0x7f7634101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:27.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.481+0000 7f7639334640 1 -- 172.21.15.67:0/2216451285 shutdown_connections 2024-07-31T14:49:27.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.481+0000 7f7639334640 1 -- 172.21.15.67:0/2216451285 wait complete. 2024-07-31T14:49:28.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:27 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2216451285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-31T14:49:28.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:27 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1842001790' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-31T14:49:28.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:27 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1111422275' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-31T14:49:28.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:27 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2216451285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-31T14:49:28.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:27 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1842001790' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-31T14:49:28.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:27 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1111422275' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-31T14:49:28.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:27 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2216451285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-31T14:49:28.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:27 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1842001790' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-31T14:49:28.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:27 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1111422275' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-31T14:49:28.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.358+0000 7f58d8a42640 1 -- 172.21.15.67:0/3072179679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58d4108720 msgr2=0x7f58d410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:28.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.358+0000 7f58d8a42640 1 --2- 172.21.15.67:0/3072179679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58d4108720 0x7f58d410ef50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009900 tx=0x7f58c402a410 comp rx=0 tx=0).stop 2024-07-31T14:49:28.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.358+0000 7f58d8a42640 1 -- 172.21.15.67:0/3072179679 shutdown_connections 2024-07-31T14:49:28.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.358+0000 7f58d8a42640 1 --2- 172.21.15.67:0/3072179679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58d4108720 0x7f58d410ef50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:28.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.358+0000 7f58d8a42640 1 --2- 172.21.15.67:0/3072179679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58d4107dd0 0x7f58d41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:28.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.358+0000 7f58d8a42640 1 --2- 172.21.15.67:0/3072179679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58d4102490 0x7f58d41028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.358+0000 7f58d8a42640 1 -- 172.21.15.67:0/3072179679 >> 172.21.15.67:0/3072179679 conn(0x7f58d4079160 msgr2=0x7f58d4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.358+0000 7f58d8a42640 1 -- 172.21.15.67:0/3072179679 shutdown_connections 2024-07-31T14:49:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.358+0000 7f58d8a42640 1 -- 172.21.15.67:0/3072179679 wait complete. 2024-07-31T14:49:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.360+0000 7f58d8a42640 1 Processor -- start 2024-07-31T14:49:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.360+0000 7f58d8a42640 1 -- start start 2024-07-31T14:49:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.360+0000 7f58d8a42640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58d4102490 0x7f58d40ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.360+0000 7f58d8a42640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58d4107dd0 0x7f58d40ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.360+0000 7f58d8a42640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58d4108720 0x7f58d40ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.360+0000 7f58d8a42640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41123d0 con 0x7f58d4107dd0 2024-07-31T14:49:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.360+0000 7f58d8a42640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f58d4112250 con 0x7f58d4108720 2024-07-31T14:49:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.360+0000 7f58d8a42640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58d4112550 con 0x7f58d4102490 2024-07-31T14:49:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.360+0000 7f58d1d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58d4107dd0 0x7f58d40ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.360+0000 7f58d1d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58d4107dd0 0x7f58d40ff5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39928/0 (socket says 172.21.15.67:39928) 2024-07-31T14:49:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.360+0000 7f58d1d9b640 1 -- 172.21.15.67:0/1842001790 learned_addr learned my addr 172.21.15.67:0/1842001790 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.361+0000 7f58d259c640 1 --2- 172.21.15.67:0/1842001790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58d4102490 0x7f58d40ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.361+0000 7f58d2d9d640 1 --2- 172.21.15.67:0/1842001790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58d4108720 0x7f58d40ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.361+0000 7f58d1d9b640 1 -- 172.21.15.67:0/1842001790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58d4102490 msgr2=0x7f58d40ff0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.361+0000 7f58d1d9b640 1 --2- 172.21.15.67:0/1842001790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58d4102490 0x7f58d40ff0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.361+0000 7f58d1d9b640 1 -- 172.21.15.67:0/1842001790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58d4108720 msgr2=0x7f58d40ffb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.361+0000 7f58d1d9b640 1 --2- 172.21.15.67:0/1842001790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58d4108720 0x7f58d40ffb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.361+0000 7f58d1d9b640 1 -- 172.21.15.67:0/1842001790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d41a8130 con 0x7f58d4107dd0 2024-07-31T14:49:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.361+0000 7f58d1d9b640 1 --2- 172.21.15.67:0/1842001790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58d4107dd0 0x7f58d40ff5e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00e9c0 tx=0x7f58bc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.361+0000 7f58cb7fe640 1 -- 172.21.15.67:0/1842001790 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00cd80 con 0x7f58d4107dd0 2024-07-31T14:49:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.361+0000 7f58d8a42640 1 -- 172.21.15.67:0/1842001790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d41a8420 con 0x7f58d4107dd0 2024-07-31T14:49:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.361+0000 7f58cb7fe640 1 -- 172.21.15.67:0/1842001790 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58bc0054d0 con 0x7f58d4107dd0 2024-07-31T14:49:28.393 INFO:teuthology.orchestra.run.smithi067.stdout:141733920846 2024-07-31T14:49:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.361+0000 7f58cb7fe640 1 -- 172.21.15.67:0/1842001790 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc010430 con 0x7f58d4107dd0 2024-07-31T14:49:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.361+0000 7f58d8a42640 1 -- 172.21.15.67:0/1842001790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d41a8960 con 0x7f58d4107dd0 2024-07-31T14:49:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.363+0000 7f58d8a42640 1 -- 172.21.15.67:0/1842001790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d41028a0 con 0x7f58d4107dd0 2024-07-31T14:49:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.365+0000 7f58cb7fe640 1 -- 172.21.15.67:0/1842001790 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f58bc004fb0 con 0x7f58d4107dd0 2024-07-31T14:49:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.366+0000 7f58cb7fe640 1 --2- 172.21.15.67:0/1842001790 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58c00b3cb0 0x7f58c00b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.366+0000 7f58d259c640 1 --2- 172.21.15.67:0/1842001790 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58c00b3cb0 0x7f58c00b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.366+0000 7f58cb7fe640 1 -- 172.21.15.67:0/1842001790 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f58bc0e4820 con 0x7f58d4107dd0 2024-07-31T14:49:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.367+0000 7f58d259c640 1 --2- 172.21.15.67:0/1842001790 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58c00b3cb0 0x7f58c00b61c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f58d4100af0 tx=0x7f58cc009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.373+0000 7f58cb7fe640 1 -- 172.21.15.67:0/1842001790 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f58bc0555e0 con 0x7f58d4107dd0 2024-07-31T14:49:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.594+0000 7f58d8a42640 1 -- 172.21.15.67:0/1842001790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f58d41003a0 con 0x7f58d4107dd0 2024-07-31T14:49:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.594+0000 7f58cb7fe640 1 -- 172.21.15.67:0/1842001790 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f58d41003a0 con 0x7f58d4107dd0 2024-07-31T14:49:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.596+0000 7f58d8a42640 1 -- 172.21.15.67:0/1842001790 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58c00b3cb0 msgr2=0x7f58c00b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.596+0000 7f58d8a42640 1 --2- 172.21.15.67:0/1842001790 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58c00b3cb0 0x7f58c00b61c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f58d4100af0 tx=0x7f58cc009500 comp rx=0 tx=0).stop 2024-07-31T14:49:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.596+0000 7f58d8a42640 1 -- 172.21.15.67:0/1842001790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58d4107dd0 msgr2=0x7f58d40ff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.596+0000 7f58d8a42640 1 --2- 172.21.15.67:0/1842001790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58d4107dd0 0x7f58d40ff5e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00e9c0 tx=0x7f58bc00ee80 comp rx=0 tx=0).stop 2024-07-31T14:49:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.596+0000 7f58d8a42640 1 -- 172.21.15.67:0/1842001790 shutdown_connections 2024-07-31T14:49:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.596+0000 7f58d8a42640 1 --2- 172.21.15.67:0/1842001790 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58c00b3cb0 0x7f58c00b61c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.596+0000 7f58d8a42640 1 --2- 172.21.15.67:0/1842001790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58d4108720 0x7f58d40ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.596+0000 7f58d8a42640 1 --2- 172.21.15.67:0/1842001790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58d4107dd0 0x7f58d40ff5e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.596+0000 7f58d8a42640 1 --2- 172.21.15.67:0/1842001790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58d4102490 0x7f58d40ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.596+0000 7f58d8a42640 1 -- 172.21.15.67:0/1842001790 >> 172.21.15.67:0/1842001790 conn(0x7f58d4079160 msgr2=0x7f58d406abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.596+0000 7f58d8a42640 1 -- 172.21.15.67:0/1842001790 shutdown_connections 2024-07-31T14:49:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:27.597+0000 7f58d8a42640 1 -- 172.21.15.67:0/1842001790 wait complete. 2024-07-31T14:49:29.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:28 smithi067 ceph-mon[32750]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:29.091 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920842 got 141733920846 for osd.4 2024-07-31T14:49:29.092 DEBUG:teuthology.parallel:result is None 2024-07-31T14:49:29.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:28 smithi073 ceph-mon[36714]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:29.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:28 smithi077 ceph-mon[37327]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:30.205 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705756 got 38654705760 for osd.0 2024-07-31T14:49:30.205 DEBUG:teuthology.parallel:result is None 2024-07-31T14:49:31.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:30 smithi067 ceph-mon[32750]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:31.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:30 smithi073 ceph-mon[36714]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:31.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:30 smithi077 ceph-mon[37327]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.811+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3253948398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc101c60 msgr2=0x7f25dc102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.811+0000 7f25e0fd9640 1 --2- 172.21.15.67:0/3253948398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc101c60 0x7f25dc102070 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009900 tx=0x7f25cc02a410 comp rx=0 tx=0).stop 2024-07-31T14:49:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.812+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3253948398 shutdown_connections 2024-07-31T14:49:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.812+0000 7f25e0fd9640 1 --2- 172.21.15.67:0/3253948398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc107ef0 0x7f25dc10e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.812+0000 7f25e0fd9640 1 --2- 172.21.15.67:0/3253948398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25dc1075a0 0x7f25dc1079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.812+0000 7f25e0fd9640 1 --2- 172.21.15.67:0/3253948398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc101c60 0x7f25dc102070 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.812+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3253948398 >> 172.21.15.67:0/3253948398 conn(0x7f25dc079870 msgr2=0x7f25dc079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.812+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3253948398 shutdown_connections 2024-07-31T14:49:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.812+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3253948398 wait complete. 2024-07-31T14:49:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25e0fd9640 1 Processor -- start 2024-07-31T14:49:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25e0fd9640 1 -- start start 2024-07-31T14:49:32.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25e0fd9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc101c60 0x7f25dc110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:32.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25e0fd9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25dc1075a0 0x7f25dc110550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:32.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25e0fd9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc107ef0 0x7f25dc110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:32.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25e0fd9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc113540 con 0x7f25dc101c60 2024-07-31T14:49:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25e0fd9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1133c0 con 0x7f25dc107ef0 2024-07-31T14:49:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25e0fd9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1136c0 con 0x7f25dc1075a0 2024-07-31T14:49:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25da59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc101c60 0x7f25dc110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25da59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc101c60 0x7f25dc110010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53166/0 (socket says 172.21.15.67:53166) 2024-07-31T14:49:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25da59c640 1 -- 172.21.15.67:0/3504606645 learned_addr learned my addr 172.21.15.67:0/3504606645 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25d9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25dc1075a0 0x7f25dc110550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25dad9d640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc107ef0 0x7f25dc110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25da59c640 1 -- 172.21.15.67:0/3504606645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25dc1075a0 msgr2=0x7f25dc110550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25da59c640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25dc1075a0 0x7f25dc110550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25da59c640 1 -- 172.21.15.67:0/3504606645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc107ef0 msgr2=0x7f25dc110a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.813+0000 7f25da59c640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc107ef0 0x7f25dc110a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.814+0000 7f25da59c640 1 -- 172.21.15.67:0/3504606645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25dc1a81b0 con 0x7f25dc101c60 2024-07-31T14:49:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.814+0000 7f25dad9d640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc107ef0 0x7f25dc110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:49:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.814+0000 7f25d9d9b640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25dc1075a0 0x7f25dc110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:49:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.814+0000 7f25da59c640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc101c60 0x7f25dc110010 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f25cc005940 tx=0x7f25cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.814+0000 7f25cb7fe640 1 -- 172.21.15.67:0/3504606645 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc03f030 con 0x7f25dc101c60 2024-07-31T14:49:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.814+0000 7f25cb7fe640 1 -- 172.21.15.67:0/3504606645 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25cc003cb0 con 0x7f25dc101c60 2024-07-31T14:49:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.814+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3504606645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25dc1a8440 con 0x7f25dc101c60 2024-07-31T14:49:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.814+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3504606645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25dc1a8920 con 0x7f25dc101c60 2024-07-31T14:49:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.814+0000 7f25cb7fe640 1 -- 172.21.15.67:0/3504606645 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc035070 con 0x7f25dc101c60 2024-07-31T14:49:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.815+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3504606645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25dc102070 con 0x7f25dc101c60 2024-07-31T14:49:32.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.817+0000 7f25cb7fe640 1 -- 172.21.15.67:0/3504606645 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f25cc042030 con 0x7f25dc101c60 2024-07-31T14:49:32.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.817+0000 7f25cb7fe640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25c40b3cb0 0x7f25c40b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:32.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.818+0000 7f25d9d9b640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25c40b3cb0 0x7f25c40b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:32.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.818+0000 7f25cb7fe640 1 -- 172.21.15.67:0/3504606645 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f25cc104d50 con 0x7f25dc101c60 2024-07-31T14:49:32.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.818+0000 7f25d9d9b640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25c40b3cb0 0x7f25c40b61c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f25dc107300 tx=0x7f25d000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:32.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:32.821+0000 7f25cb7fe640 1 -- 172.21.15.67:0/3504606645 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f25cc0cbe10 con 0x7f25dc101c60 2024-07-31T14:49:33.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:32 smithi067 ceph-mon[32750]: pgmap v353: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:49:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:33.020+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3504606645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f25dc111310 con 0x7f25dc101c60 2024-07-31T14:49:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:33.020+0000 7f25cb7fe640 1 -- 172.21.15.67:0/3504606645 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f25cc0cb710 con 0x7f25dc101c60 2024-07-31T14:49:33.023 INFO:teuthology.orchestra.run.smithi067.stdout:236223201339 2024-07-31T14:49:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:33.022+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3504606645 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25c40b3cb0 msgr2=0x7f25c40b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:33.022+0000 7f25e0fd9640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25c40b3cb0 0x7f25c40b61c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f25dc107300 tx=0x7f25d000b390 comp rx=0 tx=0).stop 2024-07-31T14:49:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:33.022+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3504606645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc101c60 msgr2=0x7f25dc110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:33.022+0000 7f25e0fd9640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc101c60 0x7f25dc110010 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f25cc005940 tx=0x7f25cc0042f0 comp rx=0 tx=0).stop 2024-07-31T14:49:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:33.022+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3504606645 shutdown_connections 2024-07-31T14:49:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:33.022+0000 7f25e0fd9640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f25c40b3cb0 0x7f25c40b61c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:33.022+0000 7f25e0fd9640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc107ef0 0x7f25dc110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:33.022+0000 7f25e0fd9640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25dc1075a0 0x7f25dc110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:33.022+0000 7f25e0fd9640 1 --2- 172.21.15.67:0/3504606645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc101c60 0x7f25dc110010 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:33.022+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3504606645 >> 172.21.15.67:0/3504606645 conn(0x7f25dc079870 msgr2=0x7f25dc105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:33.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:33.023+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3504606645 shutdown_connections 2024-07-31T14:49:33.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:33.023+0000 7f25e0fd9640 1 -- 172.21.15.67:0/3504606645 wait complete. 2024-07-31T14:49:33.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:32 smithi073 ceph-mon[36714]: pgmap v353: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:49:33.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:32 smithi077 ceph-mon[37327]: pgmap v353: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:49:33.977 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:33 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3504606645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-31T14:49:34.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:33 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3504606645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-31T14:49:34.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:33 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3504606645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-31T14:49:35.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:34 smithi067 ceph-mon[32750]: pgmap v354: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:49:35.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:34 smithi073 ceph-mon[36714]: pgmap v354: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:49:35.167 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005107 got 261993005108 for osd.9 2024-07-31T14:49:35.167 DEBUG:teuthology.parallel:result is None 2024-07-31T14:49:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:34 smithi077 ceph-mon[37327]: pgmap v354: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:49:35.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.475+0000 7f2f78232640 1 -- 172.21.15.67:0/2412151145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f70102970 msgr2=0x7f2f7010f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.475+0000 7f2f78232640 1 --2- 172.21.15.67:0/2412151145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f70102970 0x7f2f7010f690 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009900 tx=0x7f2f6002a480 comp rx=0 tx=0).stop 2024-07-31T14:49:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.475+0000 7f2f78232640 1 -- 172.21.15.67:0/2412151145 shutdown_connections 2024-07-31T14:49:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.475+0000 7f2f78232640 1 --2- 172.21.15.67:0/2412151145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f7010fbd0 0x7f2f70111ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.475+0000 7f2f78232640 1 --2- 172.21.15.67:0/2412151145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f70102970 0x7f2f7010f690 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.475+0000 7f2f78232640 1 --2- 172.21.15.67:0/2412151145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f70102020 0x7f2f70102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.475+0000 7f2f78232640 1 -- 172.21.15.67:0/2412151145 >> 172.21.15.67:0/2412151145 conn(0x7f2f70079170 msgr2=0x7f2f70079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.475+0000 7f2f78232640 1 -- 172.21.15.67:0/2412151145 shutdown_connections 2024-07-31T14:49:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.475+0000 7f2f78232640 1 -- 172.21.15.67:0/2412151145 wait complete. 2024-07-31T14:49:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.476+0000 7f2f78232640 1 Processor -- start 2024-07-31T14:49:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.476+0000 7f2f78232640 1 -- start start 2024-07-31T14:49:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.476+0000 7f2f78232640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f70102020 0x7f2f7019c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.476+0000 7f2f78232640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f70102970 0x7f2f7019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.476+0000 7f2f78232640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f7010fbd0 0x7f2f701a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.476+0000 7f2f78232640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2f70115660 con 0x7f2f70102020 2024-07-31T14:49:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.476+0000 7f2f78232640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701154e0 con 0x7f2f70102970 2024-07-31T14:49:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.476+0000 7f2f78232640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701157e0 con 0x7f2f7010fbd0 2024-07-31T14:49:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.476+0000 7f2f75fce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f70102020 0x7f2f7019c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.476+0000 7f2f75fce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f70102020 0x7f2f7019c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53174/0 (socket says 172.21.15.67:53174) 2024-07-31T14:49:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.476+0000 7f2f75fce640 1 -- 172.21.15.67:0/1169927816 learned_addr learned my addr 172.21.15.67:0/1169927816 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.476+0000 7f2f767cf640 1 --2- 172.21.15.67:0/1169927816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f7010fbd0 0x7f2f701a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.477+0000 7f2f757cd640 1 --2- 172.21.15.67:0/1169927816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f70102970 0x7f2f7019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.477+0000 7f2f75fce640 1 -- 172.21.15.67:0/1169927816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f7010fbd0 msgr2=0x7f2f701a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.477+0000 7f2f75fce640 1 --2- 172.21.15.67:0/1169927816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f7010fbd0 0x7f2f701a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.477+0000 7f2f75fce640 1 -- 172.21.15.67:0/1169927816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f70102970 msgr2=0x7f2f7019c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:35.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.477+0000 7f2f75fce640 1 --2- 172.21.15.67:0/1169927816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f70102970 0x7f2f7019c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:35.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.477+0000 7f2f75fce640 1 -- 172.21.15.67:0/1169927816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f701a1f60 con 0x7f2f70102020 2024-07-31T14:49:35.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.477+0000 7f2f75fce640 1 --2- 172.21.15.67:0/1169927816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f70102020 0x7f2f7019c0b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00e9f0 tx=0x7f2f6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:35.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.477+0000 7f2f66ffd640 1 -- 172.21.15.67:0/1169927816 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c00cd80 con 0x7f2f70102020 2024-07-31T14:49:35.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.477+0000 7f2f66ffd640 1 -- 172.21.15.67:0/1169927816 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f6c0108e0 con 0x7f2f70102020 2024-07-31T14:49:35.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.477+0000 7f2f78232640 1 -- 172.21.15.67:0/1169927816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f701a2250 con 0x7f2f70102020 2024-07-31T14:49:35.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.477+0000 7f2f66ffd640 1 -- 172.21.15.67:0/1169927816 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c018980 con 0x7f2f70102020 2024-07-31T14:49:35.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.477+0000 7f2f78232640 1 -- 172.21.15.67:0/1169927816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f701a8a70 con 0x7f2f70102020 2024-07-31T14:49:35.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.479+0000 7f2f78232640 1 -- 172.21.15.67:0/1169927816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f7006bc70 con 0x7f2f70102020 2024-07-31T14:49:35.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.481+0000 7f2f66ffd640 1 -- 172.21.15.67:0/1169927816 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f2f6c010430 con 0x7f2f70102020 2024-07-31T14:49:35.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.481+0000 7f2f66ffd640 1 --2- 172.21.15.67:0/1169927816 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f5c0b3ce0 0x7f2f5c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:35.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.481+0000 7f2f757cd640 1 --2- 172.21.15.67:0/1169927816 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f5c0b3ce0 0x7f2f5c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:35.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.482+0000 7f2f66ffd640 1 -- 172.21.15.67:0/1169927816 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f2f6c0e4a30 con 0x7f2f70102020 2024-07-31T14:49:35.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.482+0000 7f2f757cd640 1 --2- 172.21.15.67:0/1169927816 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f5c0b3ce0 0x7f2f5c0b61f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f60031440 tx=0x7f2f60009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:35.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.485+0000 7f2f66ffd640 1 -- 172.21.15.67:0/1169927816 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f2f6c0ab440 con 0x7f2f70102020 2024-07-31T14:49:35.692 INFO:teuthology.orchestra.run.smithi067.stdout:283467841585 2024-07-31T14:49:35.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.689+0000 7f2f78232640 1 -- 172.21.15.67:0/1169927816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f2f70102430 con 0x7f2f70102020 2024-07-31T14:49:35.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.690+0000 7f2f66ffd640 1 -- 172.21.15.67:0/1169927816 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2f6c0af990 con 0x7f2f70102020 2024-07-31T14:49:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.692+0000 7f2f78232640 1 -- 172.21.15.67:0/1169927816 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f5c0b3ce0 msgr2=0x7f2f5c0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.692+0000 7f2f78232640 1 --2- 172.21.15.67:0/1169927816 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f5c0b3ce0 0x7f2f5c0b61f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f60031440 tx=0x7f2f60009560 comp rx=0 tx=0).stop 2024-07-31T14:49:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.692+0000 7f2f78232640 1 -- 172.21.15.67:0/1169927816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f70102020 msgr2=0x7f2f7019c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.692+0000 7f2f78232640 1 --2- 172.21.15.67:0/1169927816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f70102020 0x7f2f7019c0b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00e9f0 tx=0x7f2f6c00eeb0 comp rx=0 tx=0).stop 2024-07-31T14:49:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.692+0000 7f2f78232640 1 -- 172.21.15.67:0/1169927816 shutdown_connections 2024-07-31T14:49:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.692+0000 7f2f78232640 1 --2- 172.21.15.67:0/1169927816 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f5c0b3ce0 0x7f2f5c0b61f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.692+0000 7f2f78232640 1 --2- 172.21.15.67:0/1169927816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f7010fbd0 0x7f2f701a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.692+0000 7f2f78232640 1 --2- 172.21.15.67:0/1169927816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f70102970 0x7f2f7019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.692+0000 7f2f78232640 1 --2- 172.21.15.67:0/1169927816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f70102020 0x7f2f7019c0b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.692+0000 7f2f78232640 1 -- 172.21.15.67:0/1169927816 >> 172.21.15.67:0/1169927816 conn(0x7f2f70079170 msgr2=0x7f2f70111050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.692+0000 7f2f78232640 1 -- 172.21.15.67:0/1169927816 shutdown_connections 2024-07-31T14:49:35.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:35.692+0000 7f2f78232640 1 -- 172.21.15.67:0/1169927816 wait complete. 2024-07-31T14:49:36.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.096+0000 7f1346ba0640 1 -- 172.21.15.67:0/3798538595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13401022b0 msgr2=0x7f13401026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:36.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.096+0000 7f1346ba0640 1 --2- 172.21.15.67:0/3798538595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13401022b0 0x7f13401026c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1330009900 tx=0x7f133002a410 comp rx=0 tx=0).stop 2024-07-31T14:49:36.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.096+0000 7f1346ba0640 1 -- 172.21.15.67:0/3798538595 shutdown_connections 2024-07-31T14:49:36.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.096+0000 7f1346ba0640 1 --2- 172.21.15.67:0/3798538595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1340108540 0x7f134010ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:36.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.096+0000 7f1346ba0640 1 --2- 172.21.15.67:0/3798538595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1340107bf0 0x7f1340108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:36.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.096+0000 7f1346ba0640 1 --2- 172.21.15.67:0/3798538595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13401022b0 0x7f13401026c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:36.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.096+0000 7f1346ba0640 1 -- 172.21.15.67:0/3798538595 >> 172.21.15.67:0/3798538595 conn(0x7f13400fdd00 msgr2=0x7f1340100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:36.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.096+0000 7f1346ba0640 1 -- 172.21.15.67:0/3798538595 shutdown_connections 2024-07-31T14:49:36.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.096+0000 7f1346ba0640 1 -- 172.21.15.67:0/3798538595 wait complete. 2024-07-31T14:49:36.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.097+0000 7f1346ba0640 1 Processor -- start 2024-07-31T14:49:36.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.097+0000 7f1346ba0640 1 -- start start 2024-07-31T14:49:36.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.097+0000 7f1346ba0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13401022b0 0x7f134019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:36.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.097+0000 7f1346ba0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1340107bf0 0x7f134019b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:36.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.097+0000 7f1346ba0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1340108540 0x7f134019b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:36.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1346ba0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f13401121f0 con 0x7f13401022b0 2024-07-31T14:49:36.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1346ba0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1340112070 con 0x7f1340108540 2024-07-31T14:49:36.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1346ba0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1340112370 con 0x7f1340107bf0 2024-07-31T14:49:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1345b9e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13401022b0 0x7f134019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1345b9e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13401022b0 0x7f134019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53192/0 (socket says 172.21.15.67:53192) 2024-07-31T14:49:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1345b9e640 1 -- 172.21.15.67:0/4160485236 learned_addr learned my addr 172.21.15.67:0/4160485236 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f134639f640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1340108540 0x7f134019b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f134539d640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1340107bf0 0x7f134019b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1345b9e640 1 -- 172.21.15.67:0/4160485236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1340107bf0 msgr2=0x7f134019b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1345b9e640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1340107bf0 0x7f134019b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1345b9e640 1 -- 172.21.15.67:0/4160485236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1340108540 msgr2=0x7f134019b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1345b9e640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1340108540 0x7f134019b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1345b9e640 1 -- 172.21.15.67:0/4160485236 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13401a5dd0 con 0x7f13401022b0 2024-07-31T14:49:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f134539d640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1340107bf0 0x7f134019b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:49:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1345b9e640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13401022b0 0x7f134019ab40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1330005940 tx=0x7f13300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1336ffd640 1 -- 172.21.15.67:0/4160485236 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133003f030 con 0x7f13401022b0 2024-07-31T14:49:36.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1346ba0640 1 -- 172.21.15.67:0/4160485236 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13401a6060 con 0x7f13401022b0 2024-07-31T14:49:36.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1336ffd640 1 -- 172.21.15.67:0/4160485236 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1330003cb0 con 0x7f13401022b0 2024-07-31T14:49:36.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.098+0000 7f1336ffd640 1 -- 172.21.15.67:0/4160485236 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330035070 con 0x7f13401022b0 2024-07-31T14:49:36.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.099+0000 7f1346ba0640 1 -- 172.21.15.67:0/4160485236 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13401a6540 con 0x7f13401022b0 2024-07-31T14:49:36.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.100+0000 7f1346ba0640 1 -- 172.21.15.67:0/4160485236 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1340103530 con 0x7f13401022b0 2024-07-31T14:49:36.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.101+0000 7f1336ffd640 1 -- 172.21.15.67:0/4160485236 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f1330042030 con 0x7f13401022b0 2024-07-31T14:49:36.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.102+0000 7f1336ffd640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f132c0b3ce0 0x7f132c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:36.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.102+0000 7f134539d640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f132c0b3ce0 0x7f132c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:36.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.102+0000 7f1336ffd640 1 -- 172.21.15.67:0/4160485236 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f1330104960 con 0x7f13401022b0 2024-07-31T14:49:36.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.102+0000 7f134539d640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f132c0b3ce0 0x7f132c0b61f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f133c0058c0 tx=0x7f133c005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.105+0000 7f1336ffd640 1 -- 172.21.15.67:0/4160485236 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f13300cba20 con 0x7f13401022b0 2024-07-31T14:49:36.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:35 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1169927816' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-31T14:49:36.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:35 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1169927816' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-31T14:49:36.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:35 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1169927816' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-31T14:49:36.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.302+0000 7f1346ba0640 1 -- 172.21.15.67:0/4160485236 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f134010a320 con 0x7f13401022b0 2024-07-31T14:49:36.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.303+0000 7f1336ffd640 1 -- 172.21.15.67:0/4160485236 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f13300cb320 con 0x7f13401022b0 2024-07-31T14:49:36.304 INFO:teuthology.orchestra.run.smithi067.stdout:81604378713 2024-07-31T14:49:36.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.305+0000 7f1346ba0640 1 -- 172.21.15.67:0/4160485236 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f132c0b3ce0 msgr2=0x7f132c0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.305+0000 7f1346ba0640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f132c0b3ce0 0x7f132c0b61f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f133c0058c0 tx=0x7f133c005850 comp rx=0 tx=0).stop 2024-07-31T14:49:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.305+0000 7f1346ba0640 1 -- 172.21.15.67:0/4160485236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13401022b0 msgr2=0x7f134019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.305+0000 7f1346ba0640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13401022b0 0x7f134019ab40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1330005940 tx=0x7f13300042f0 comp rx=0 tx=0).stop 2024-07-31T14:49:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.305+0000 7f1346ba0640 1 -- 172.21.15.67:0/4160485236 shutdown_connections 2024-07-31T14:49:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.305+0000 7f1346ba0640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f132c0b3ce0 0x7f132c0b61f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.305+0000 7f1346ba0640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1340108540 0x7f134019b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.305+0000 7f1346ba0640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1340107bf0 0x7f134019b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.305+0000 7f1346ba0640 1 --2- 172.21.15.67:0/4160485236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13401022b0 0x7f134019ab40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.305+0000 7f1346ba0640 1 -- 172.21.15.67:0/4160485236 >> 172.21.15.67:0/4160485236 conn(0x7f13400fdd00 msgr2=0x7f1340100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.305+0000 7f1346ba0640 1 -- 172.21.15.67:0/4160485236 shutdown_connections 2024-07-31T14:49:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.305+0000 7f1346ba0640 1 -- 172.21.15.67:0/4160485236 wait complete. 2024-07-31T14:49:37.103 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:36 smithi067 ceph-mon[32750]: pgmap v355: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:49:37.103 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:36 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/4160485236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-31T14:49:37.103 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:36 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/182000350' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-31T14:49:37.135 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117071 got 115964117074 for osd.3 2024-07-31T14:49:37.135 DEBUG:teuthology.parallel:result is None 2024-07-31T14:49:37.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:36 smithi073 ceph-mon[36714]: pgmap v355: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:49:37.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:36 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/4160485236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-31T14:49:37.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:36 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/182000350' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-31T14:49:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:36 smithi077 ceph-mon[37327]: pgmap v355: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:49:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:36 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/4160485236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-31T14:49:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:36 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/182000350' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-31T14:49:37.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.297+0000 7f8c4de1c640 1 -- 172.21.15.67:0/4149724568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48102490 msgr2=0x7f8c481028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:37.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.297+0000 7f8c4de1c640 1 --2- 172.21.15.67:0/4149724568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48102490 0x7f8c481028a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009900 tx=0x7f8c3802a3f0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.297+0000 7f8c4de1c640 1 -- 172.21.15.67:0/4149724568 shutdown_connections 2024-07-31T14:49:37.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.297+0000 7f8c4de1c640 1 --2- 172.21.15.67:0/4149724568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c48108720 0x7f8c4810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.297+0000 7f8c4de1c640 1 --2- 172.21.15.67:0/4149724568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c48107dd0 0x7f8c481081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.297+0000 7f8c4de1c640 1 --2- 172.21.15.67:0/4149724568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48102490 0x7f8c481028a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.297+0000 7f8c4de1c640 1 -- 172.21.15.67:0/4149724568 >> 172.21.15.67:0/4149724568 conn(0x7f8c48079160 msgr2=0x7f8c48079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:37.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.297+0000 7f8c4de1c640 1 -- 172.21.15.67:0/4149724568 shutdown_connections 2024-07-31T14:49:37.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.297+0000 7f8c4de1c640 1 -- 172.21.15.67:0/4149724568 wait complete. 2024-07-31T14:49:37.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.297+0000 7f8c4de1c640 1 Processor -- start 2024-07-31T14:49:37.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c4de1c640 1 -- start start 2024-07-31T14:49:37.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c4de1c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c48102490 0x7f8c48199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:37.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c4de1c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48107dd0 0x7f8c4819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:37.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c4de1c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c48108720 0x7f8c4819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:37.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c4de1c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8c481134e0 con 0x7f8c48107dd0 2024-07-31T14:49:37.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c4de1c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8c48113360 con 0x7f8c48102490 2024-07-31T14:49:37.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c4de1c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8c48113660 con 0x7f8c48108720 2024-07-31T14:49:37.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c46ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48107dd0 0x7f8c4819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:37.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c46ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48107dd0 0x7f8c4819a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53244/0 (socket says 172.21.15.67:53244) 2024-07-31T14:49:37.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c46ffd640 1 -- 172.21.15.67:0/2412179898 learned_addr learned my addr 172.21.15.67:0/2412179898 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:37.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c47fff640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c48108720 0x7f8c4819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:37.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c477fe640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c48102490 0x7f8c48199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:37.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c46ffd640 1 -- 172.21.15.67:0/2412179898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c48108720 msgr2=0x7f8c4819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:37.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c46ffd640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c48108720 0x7f8c4819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c46ffd640 1 -- 172.21.15.67:0/2412179898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c48102490 msgr2=0x7f8c48199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:37.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c46ffd640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c48102490 0x7f8c48199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c46ffd640 1 -- 172.21.15.67:0/2412179898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4819fd20 con 0x7f8c48107dd0 2024-07-31T14:49:37.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.298+0000 7f8c477fe640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c48102490 0x7f8c48199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:49:37.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.299+0000 7f8c46ffd640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48107dd0 0x7f8c4819a410 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00c970 tx=0x7f8c3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:37.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.299+0000 7f8c44ff9640 1 -- 172.21.15.67:0/2412179898 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c014070 con 0x7f8c48107dd0 2024-07-31T14:49:37.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.299+0000 7f8c44ff9640 1 -- 172.21.15.67:0/2412179898 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c3c004440 con 0x7f8c48107dd0 2024-07-31T14:49:37.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.299+0000 7f8c44ff9640 1 -- 172.21.15.67:0/2412179898 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c003e50 con 0x7f8c48107dd0 2024-07-31T14:49:37.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.299+0000 7f8c4de1c640 1 -- 172.21.15.67:0/2412179898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c481a0010 con 0x7f8c48107dd0 2024-07-31T14:49:37.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.299+0000 7f8c4de1c640 1 -- 172.21.15.67:0/2412179898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c481a6830 con 0x7f8c48107dd0 2024-07-31T14:49:37.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.300+0000 7f8c4de1c640 1 -- 172.21.15.67:0/2412179898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c481028a0 con 0x7f8c48107dd0 2024-07-31T14:49:37.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.302+0000 7f8c44ff9640 1 -- 172.21.15.67:0/2412179898 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f8c3c017070 con 0x7f8c48107dd0 2024-07-31T14:49:37.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.302+0000 7f8c44ff9640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c300b3f00 0x7f8c300b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:37.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.302+0000 7f8c477fe640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c300b3f00 0x7f8c300b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:37.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.302+0000 7f8c44ff9640 1 -- 172.21.15.67:0/2412179898 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f8c3c0e4400 con 0x7f8c48107dd0 2024-07-31T14:49:37.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.303+0000 7f8c477fe640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c300b3f00 0x7f8c300b6410 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8c3802a910 tx=0x7f8c38005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:37.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.305+0000 7f8c44ff9640 1 -- 172.21.15.67:0/2412179898 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f8c3c01e030 con 0x7f8c48107dd0 2024-07-31T14:49:37.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.503+0000 7f8c4de1c640 1 -- 172.21.15.67:0/2412179898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8c4810a550 con 0x7f8c48107dd0 2024-07-31T14:49:37.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.504+0000 7f8c44ff9640 1 -- 172.21.15.67:0/2412179898 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8c3c0ab5e0 con 0x7f8c48107dd0 2024-07-31T14:49:37.505 INFO:teuthology.orchestra.run.smithi067.stdout:163208757323 2024-07-31T14:49:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.506+0000 7f8c4de1c640 1 -- 172.21.15.67:0/2412179898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c300b3f00 msgr2=0x7f8c300b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.506+0000 7f8c4de1c640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c300b3f00 0x7f8c300b6410 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8c3802a910 tx=0x7f8c38005fd0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.506+0000 7f8c4de1c640 1 -- 172.21.15.67:0/2412179898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48107dd0 msgr2=0x7f8c4819a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.506+0000 7f8c4de1c640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48107dd0 0x7f8c4819a410 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00c970 tx=0x7f8c3c00ce30 comp rx=0 tx=0).stop 2024-07-31T14:49:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.506+0000 7f8c4de1c640 1 -- 172.21.15.67:0/2412179898 shutdown_connections 2024-07-31T14:49:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.506+0000 7f8c4de1c640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c300b3f00 0x7f8c300b6410 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.506+0000 7f8c4de1c640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c48108720 0x7f8c4819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.506+0000 7f8c4de1c640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48107dd0 0x7f8c4819a410 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.506+0000 7f8c4de1c640 1 --2- 172.21.15.67:0/2412179898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c48102490 0x7f8c48199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.506+0000 7f8c4de1c640 1 -- 172.21.15.67:0/2412179898 >> 172.21.15.67:0/2412179898 conn(0x7f8c48079160 msgr2=0x7f8c48100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.507+0000 7f8c4de1c640 1 -- 172.21.15.67:0/2412179898 shutdown_connections 2024-07-31T14:49:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:37.507+0000 7f8c4de1c640 1 -- 172.21.15.67:0/2412179898 wait complete. 2024-07-31T14:49:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.340+0000 7fe21c4f1640 1 -- 172.21.15.67:0/2098521131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe214107e10 msgr2=0x7fe214108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.340+0000 7fe21c4f1640 1 --2- 172.21.15.67:0/2098521131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe214107e10 0x7fe214108220 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe204009900 tx=0x7fe20402a410 comp rx=0 tx=0).stop 2024-07-31T14:49:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.340+0000 7fe21c4f1640 1 -- 172.21.15.67:0/2098521131 shutdown_connections 2024-07-31T14:49:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.340+0000 7fe21c4f1640 1 --2- 172.21.15.67:0/2098521131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe214108760 0x7fe21410ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.340+0000 7fe21c4f1640 1 --2- 172.21.15.67:0/2098521131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe214107e10 0x7fe214108220 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.340+0000 7fe21c4f1640 1 --2- 172.21.15.67:0/2098521131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2141024d0 0x7fe2141028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.340+0000 7fe21c4f1640 1 -- 172.21.15.67:0/2098521131 >> 172.21.15.67:0/2098521131 conn(0x7fe2140791e0 msgr2=0x7fe2140795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.340+0000 7fe21c4f1640 1 -- 172.21.15.67:0/2098521131 shutdown_connections 2024-07-31T14:49:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.340+0000 7fe21c4f1640 1 -- 172.21.15.67:0/2098521131 wait complete. 2024-07-31T14:49:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.341+0000 7fe21c4f1640 1 Processor -- start 2024-07-31T14:49:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.341+0000 7fe21c4f1640 1 -- start start 2024-07-31T14:49:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe21c4f1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2141024d0 0x7fe21419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe21c4f1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe214107e10 0x7fe21419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe21c4f1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe214108760 0x7fe21419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe21c4f1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe2141123f0 con 0x7fe214107e10 2024-07-31T14:49:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe21c4f1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe214112270 con 0x7fe214108760 2024-07-31T14:49:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe21c4f1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe214112570 con 0x7fe2141024d0 2024-07-31T14:49:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe219a8c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe214107e10 0x7fe21419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe219a8c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe214107e10 0x7fe21419b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53218/0 (socket says 172.21.15.67:53218) 2024-07-31T14:49:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe219a8c640 1 -- 172.21.15.67:0/182000350 learned_addr learned my addr 172.21.15.67:0/182000350 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe21a28d640 1 --2- 172.21.15.67:0/182000350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2141024d0 0x7fe21419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe21aa8e640 1 --2- 172.21.15.67:0/182000350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe214108760 0x7fe21419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe219a8c640 1 -- 172.21.15.67:0/182000350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2141024d0 msgr2=0x7fe21419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe219a8c640 1 --2- 172.21.15.67:0/182000350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2141024d0 0x7fe21419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe219a8c640 1 -- 172.21.15.67:0/182000350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe214108760 msgr2=0x7fe21419b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe219a8c640 1 --2- 172.21.15.67:0/182000350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe214108760 0x7fe21419b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe219a8c640 1 -- 172.21.15.67:0/182000350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2141a5fd0 con 0x7fe214107e10 2024-07-31T14:49:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.342+0000 7fe219a8c640 1 --2- 172.21.15.67:0/182000350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe214107e10 0x7fe21419b280 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe20402a930 tx=0x7fe204031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.343+0000 7fe20b7fe640 1 -- 172.21.15.67:0/182000350 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20403f030 con 0x7fe214107e10 2024-07-31T14:49:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.343+0000 7fe21c4f1640 1 -- 172.21.15.67:0/182000350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2141a6260 con 0x7fe214107e10 2024-07-31T14:49:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.343+0000 7fe20b7fe640 1 -- 172.21.15.67:0/182000350 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe204004490 con 0x7fe214107e10 2024-07-31T14:49:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.343+0000 7fe21c4f1640 1 -- 172.21.15.67:0/182000350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2141a6790 con 0x7fe214107e10 2024-07-31T14:49:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.343+0000 7fe20b7fe640 1 -- 172.21.15.67:0/182000350 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204049060 con 0x7fe214107e10 2024-07-31T14:49:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.345+0000 7fe21c4f1640 1 -- 172.21.15.67:0/182000350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2141028e0 con 0x7fe214107e10 2024-07-31T14:49:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.346+0000 7fe20b7fe640 1 -- 172.21.15.67:0/182000350 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fe204042030 con 0x7fe214107e10 2024-07-31T14:49:37.801 INFO:teuthology.orchestra.run.smithi067.stdout:309237645357 2024-07-31T14:49:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.346+0000 7fe20b7fe640 1 --2- 172.21.15.67:0/182000350 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2000b3ce0 0x7fe2000b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.347+0000 7fe21a28d640 1 --2- 172.21.15.67:0/182000350 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2000b3ce0 0x7fe2000b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.347+0000 7fe20b7fe640 1 -- 172.21.15.67:0/182000350 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fe204104880 con 0x7fe214107e10 2024-07-31T14:49:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.347+0000 7fe21a28d640 1 --2- 172.21.15.67:0/182000350 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2000b3ce0 0x7fe2000b61f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe21406bdb0 tx=0x7fe210005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.352+0000 7fe20b7fe640 1 -- 172.21.15.67:0/182000350 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe2040cb2f0 con 0x7fe214107e10 2024-07-31T14:49:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.550+0000 7fe21c4f1640 1 -- 172.21.15.67:0/182000350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fe214109f80 con 0x7fe214107e10 2024-07-31T14:49:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.551+0000 7fe20b7fe640 1 -- 172.21.15.67:0/182000350 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe2040cf840 con 0x7fe214107e10 2024-07-31T14:49:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.553+0000 7fe21c4f1640 1 -- 172.21.15.67:0/182000350 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2000b3ce0 msgr2=0x7fe2000b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.553+0000 7fe21c4f1640 1 --2- 172.21.15.67:0/182000350 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2000b3ce0 0x7fe2000b61f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe21406bdb0 tx=0x7fe210005f90 comp rx=0 tx=0).stop 2024-07-31T14:49:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.553+0000 7fe21c4f1640 1 -- 172.21.15.67:0/182000350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe214107e10 msgr2=0x7fe21419b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.553+0000 7fe21c4f1640 1 --2- 172.21.15.67:0/182000350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe214107e10 0x7fe21419b280 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe20402a930 tx=0x7fe204031a20 comp rx=0 tx=0).stop 2024-07-31T14:49:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.553+0000 7fe21c4f1640 1 -- 172.21.15.67:0/182000350 shutdown_connections 2024-07-31T14:49:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.553+0000 7fe21c4f1640 1 --2- 172.21.15.67:0/182000350 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2000b3ce0 0x7fe2000b61f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.553+0000 7fe21c4f1640 1 --2- 172.21.15.67:0/182000350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe214108760 0x7fe21419b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.553+0000 7fe21c4f1640 1 --2- 172.21.15.67:0/182000350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe214107e10 0x7fe21419b280 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.553+0000 7fe21c4f1640 1 --2- 172.21.15.67:0/182000350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2141024d0 0x7fe21419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.553+0000 7fe21c4f1640 1 -- 172.21.15.67:0/182000350 >> 172.21.15.67:0/182000350 conn(0x7fe2140791e0 msgr2=0x7fe214105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.553+0000 7fe21c4f1640 1 -- 172.21.15.67:0/182000350 shutdown_connections 2024-07-31T14:49:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:36.553+0000 7fe21c4f1640 1 -- 172.21.15.67:0/182000350 wait complete. 2024-07-31T14:49:38.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:37 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2412179898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-31T14:49:38.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:37 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2412179898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-31T14:49:38.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:37 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2412179898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-31T14:49:38.533 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645351 got 309237645357 for osd.11 2024-07-31T14:49:38.533 DEBUG:teuthology.parallel:result is None 2024-07-31T14:49:39.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:38 smithi073 ceph-mon[36714]: pgmap v356: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:39.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:38 smithi067 ceph-mon[32750]: pgmap v356: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:38 smithi077 ceph-mon[37327]: pgmap v356: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:39.305 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378705 got 81604378713 for osd.2 2024-07-31T14:49:39.305 DEBUG:teuthology.parallel:result is None 2024-07-31T14:49:39.711 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757315 got 163208757323 for osd.5 2024-07-31T14:49:39.712 DEBUG:teuthology.parallel:result is None 2024-07-31T14:49:39.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.984+0000 7fd693dc9640 1 -- 172.21.15.67:0/992405282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd68c109840 msgr2=0x7fd68c110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:39.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.984+0000 7fd693dc9640 1 --2- 172.21.15.67:0/992405282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd68c109840 0x7fd68c110070 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd684009ca0 tx=0x7fd68402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:49:39.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.984+0000 7fd693dc9640 1 -- 172.21.15.67:0/992405282 shutdown_connections 2024-07-31T14:49:39.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.984+0000 7fd693dc9640 1 --2- 172.21.15.67:0/992405282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd68c109840 0x7fd68c110070 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.984+0000 7fd693dc9640 1 --2- 172.21.15.67:0/992405282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd68c108ef0 0x7fd68c109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.984+0000 7fd693dc9640 1 --2- 172.21.15.67:0/992405282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1035b0 0x7fd68c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.984+0000 7fd693dc9640 1 -- 172.21.15.67:0/992405282 >> 172.21.15.67:0/992405282 conn(0x7fd68c0ff040 msgr2=0x7fd68c101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.985+0000 7fd693dc9640 1 -- 172.21.15.67:0/992405282 shutdown_connections 2024-07-31T14:49:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.985+0000 7fd693dc9640 1 -- 172.21.15.67:0/992405282 wait complete. 2024-07-31T14:49:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.985+0000 7fd693dc9640 1 Processor -- start 2024-07-31T14:49:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.985+0000 7fd693dc9640 1 -- start start 2024-07-31T14:49:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd693dc9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1035b0 0x7fd68c199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd693dc9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd68c108ef0 0x7fd68c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd693dc9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd68c109840 0x7fd68c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd693dc9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1134f0 con 0x7fd68c108ef0 2024-07-31T14:49:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd693dc9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c113370 con 0x7fd68c109840 2024-07-31T14:49:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd693dc9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c113670 con 0x7fd68c1035b0 2024-07-31T14:49:39.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd691364640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd68c108ef0 0x7fd68c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:39.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd691364640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd68c108ef0 0x7fd68c19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53254/0 (socket says 172.21.15.67:53254) 2024-07-31T14:49:39.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd691364640 1 -- 172.21.15.67:0/2882894073 learned_addr learned my addr 172.21.15.67:0/2882894073 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:39.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd692366640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd68c109840 0x7fd68c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:39.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd691b65640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1035b0 0x7fd68c199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:39.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd691364640 1 -- 172.21.15.67:0/2882894073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1035b0 msgr2=0x7fd68c199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:39.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd691364640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1035b0 0x7fd68c199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:39.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd691364640 1 -- 172.21.15.67:0/2882894073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd68c109840 msgr2=0x7fd68c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:39.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd691364640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd68c109840 0x7fd68c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:39.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd691364640 1 -- 172.21.15.67:0/2882894073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c19fdd0 con 0x7fd68c108ef0 2024-07-31T14:49:39.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd691b65640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1035b0 0x7fd68c199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:49:39.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd691364640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd68c108ef0 0x7fd68c19a400 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00d740 tx=0x7fd67c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:39.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.986+0000 7fd682ffd640 1 -- 172.21.15.67:0/2882894073 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c017070 con 0x7fd68c108ef0 2024-07-31T14:49:39.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.987+0000 7fd682ffd640 1 -- 172.21.15.67:0/2882894073 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd67c00de50 con 0x7fd68c108ef0 2024-07-31T14:49:39.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.987+0000 7fd693dc9640 1 -- 172.21.15.67:0/2882894073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68c1a00c0 con 0x7fd68c108ef0 2024-07-31T14:49:39.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.987+0000 7fd682ffd640 1 -- 172.21.15.67:0/2882894073 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c00f620 con 0x7fd68c108ef0 2024-07-31T14:49:39.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.987+0000 7fd693dc9640 1 -- 172.21.15.67:0/2882894073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68c1a6820 con 0x7fd68c108ef0 2024-07-31T14:49:39.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.988+0000 7fd693dc9640 1 -- 172.21.15.67:0/2882894073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd68c104830 con 0x7fd68c108ef0 2024-07-31T14:49:39.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.989+0000 7fd682ffd640 1 -- 172.21.15.67:0/2882894073 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fd67c01a070 con 0x7fd68c108ef0 2024-07-31T14:49:39.826 INFO:teuthology.orchestra.run.smithi067.stdout:188978561094 2024-07-31T14:49:39.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.990+0000 7fd682ffd640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd6780b3d30 0x7fd6780b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:39.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.990+0000 7fd691b65640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd6780b3d30 0x7fd6780b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:39.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.990+0000 7fd682ffd640 1 -- 172.21.15.67:0/2882894073 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fd67c0e3bf0 con 0x7fd68c108ef0 2024-07-31T14:49:39.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.990+0000 7fd691b65640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd6780b3d30 0x7fd6780b6240 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd688009c20 tx=0x7fd688005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:39.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:38.994+0000 7fd682ffd640 1 -- 172.21.15.67:0/2882894073 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd67c0aacb0 con 0x7fd68c108ef0 2024-07-31T14:49:39.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:39.190+0000 7fd693dc9640 1 -- 172.21.15.67:0/2882894073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd68c19af90 con 0x7fd68c108ef0 2024-07-31T14:49:39.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:39.191+0000 7fd682ffd640 1 -- 172.21.15.67:0/2882894073 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd67c0aa5b0 con 0x7fd68c108ef0 2024-07-31T14:49:39.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:39.193+0000 7fd693dc9640 1 -- 172.21.15.67:0/2882894073 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd6780b3d30 msgr2=0x7fd6780b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:39.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:39.193+0000 7fd693dc9640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd6780b3d30 0x7fd6780b6240 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd688009c20 tx=0x7fd688005f90 comp rx=0 tx=0).stop 2024-07-31T14:49:39.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:39.193+0000 7fd693dc9640 1 -- 172.21.15.67:0/2882894073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd68c108ef0 msgr2=0x7fd68c19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:39.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:39.193+0000 7fd693dc9640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd68c108ef0 0x7fd68c19a400 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00d740 tx=0x7fd67c00dc00 comp rx=0 tx=0).stop 2024-07-31T14:49:39.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:39.193+0000 7fd693dc9640 1 -- 172.21.15.67:0/2882894073 shutdown_connections 2024-07-31T14:49:39.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:39.193+0000 7fd693dc9640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd6780b3d30 0x7fd6780b6240 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:39.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:39.193+0000 7fd693dc9640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd68c109840 0x7fd68c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:39.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:39.193+0000 7fd693dc9640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd68c108ef0 0x7fd68c19a400 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:39.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:39.193+0000 7fd693dc9640 1 --2- 172.21.15.67:0/2882894073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd68c1035b0 0x7fd68c199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:39.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:39.193+0000 7fd693dc9640 1 -- 172.21.15.67:0/2882894073 >> 172.21.15.67:0/2882894073 conn(0x7fd68c0ff040 msgr2=0x7fd68c101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:39.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:39.193+0000 7fd693dc9640 1 -- 172.21.15.67:0/2882894073 shutdown_connections 2024-07-31T14:49:39.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:39.193+0000 7fd693dc9640 1 -- 172.21.15.67:0/2882894073 wait complete. 2024-07-31T14:49:40.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:39 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2882894073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-31T14:49:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:39 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2882894073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-31T14:49:40.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:39 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2882894073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-31T14:49:40.883 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201335 got 236223201339 for osd.8 2024-07-31T14:49:40.884 DEBUG:teuthology.parallel:result is None 2024-07-31T14:49:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:40 smithi077 ceph-mon[37327]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:41.259 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:40 smithi067 ceph-mon[32750]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:41.298 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841581 got 283467841585 for osd.10 2024-07-31T14:49:41.298 DEBUG:teuthology.parallel:result is None 2024-07-31T14:49:41.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:40 smithi073 ceph-mon[36714]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:42.022 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561088 got 188978561094 for osd.6 2024-07-31T14:49:42.022 DEBUG:teuthology.parallel:result is None 2024-07-31T14:49:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.825+0000 7f2f28556640 1 -- 172.21.15.67:0/2122821308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f20102510 msgr2=0x7f2f20102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.825+0000 7f2f28556640 1 --2- 172.21.15.67:0/2122821308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f20102510 0x7f2f20102920 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2f14009900 tx=0x7f2f1402a410 comp rx=0 tx=0).stop 2024-07-31T14:49:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.825+0000 7f2f28556640 1 -- 172.21.15.67:0/2122821308 shutdown_connections 2024-07-31T14:49:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.825+0000 7f2f28556640 1 --2- 172.21.15.67:0/2122821308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f20108750 0x7f2f2010ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.825+0000 7f2f28556640 1 --2- 172.21.15.67:0/2122821308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f20107e00 0x7f2f20108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.825+0000 7f2f28556640 1 --2- 172.21.15.67:0/2122821308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f20102510 0x7f2f20102920 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.825+0000 7f2f28556640 1 -- 172.21.15.67:0/2122821308 >> 172.21.15.67:0/2122821308 conn(0x7f2f200791c0 msgr2=0x7f2f200795d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.826+0000 7f2f28556640 1 -- 172.21.15.67:0/2122821308 shutdown_connections 2024-07-31T14:49:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.826+0000 7f2f28556640 1 -- 172.21.15.67:0/2122821308 wait complete. 2024-07-31T14:49:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.827+0000 7f2f28556640 1 Processor -- start 2024-07-31T14:49:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.827+0000 7f2f28556640 1 -- start start 2024-07-31T14:49:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.827+0000 7f2f28556640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f20102510 0x7f2f20109070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.827+0000 7f2f28556640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f20107e00 0x7f2f201095b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.827+0000 7f2f28556640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f20108750 0x7f2f20109af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.827+0000 7f2f28556640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20112420 con 0x7f2f20102510 2024-07-31T14:49:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.827+0000 7f2f28556640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201122a0 con 0x7f2f20107e00 2024-07-31T14:49:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.827+0000 7f2f28556640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201125a0 con 0x7f2f20108750 2024-07-31T14:49:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.827+0000 7f2f262f2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f20102510 0x7f2f20109070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.827+0000 7f2f262f2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f20102510 0x7f2f20109070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45090/0 (socket says 172.21.15.67:45090) 2024-07-31T14:49:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.827+0000 7f2f262f2640 1 -- 172.21.15.67:0/2372077392 learned_addr learned my addr 172.21.15.67:0/2372077392 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.828+0000 7f2f26af3640 1 --2- 172.21.15.67:0/2372077392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f20108750 0x7f2f20109af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.828+0000 7f2f25af1640 1 --2- 172.21.15.67:0/2372077392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f20107e00 0x7f2f201095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.828+0000 7f2f262f2640 1 -- 172.21.15.67:0/2372077392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f20108750 msgr2=0x7f2f20109af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:42.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.828+0000 7f2f262f2640 1 --2- 172.21.15.67:0/2372077392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f20108750 0x7f2f20109af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:42.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.828+0000 7f2f262f2640 1 -- 172.21.15.67:0/2372077392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f20107e00 msgr2=0x7f2f201095b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:42.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.828+0000 7f2f262f2640 1 --2- 172.21.15.67:0/2372077392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f20107e00 0x7f2f201095b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:42.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.828+0000 7f2f262f2640 1 -- 172.21.15.67:0/2372077392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f201a8180 con 0x7f2f20102510 2024-07-31T14:49:42.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.828+0000 7f2f262f2640 1 --2- 172.21.15.67:0/2372077392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f20102510 0x7f2f20109070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2f14005940 tx=0x7f2f140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:42.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.828+0000 7f2f137fe640 1 -- 172.21.15.67:0/2372077392 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1403f030 con 0x7f2f20102510 2024-07-31T14:49:42.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.828+0000 7f2f28556640 1 -- 172.21.15.67:0/2372077392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f201a8410 con 0x7f2f20102510 2024-07-31T14:49:42.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.828+0000 7f2f137fe640 1 -- 172.21.15.67:0/2372077392 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f14003cb0 con 0x7f2f20102510 2024-07-31T14:49:42.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.828+0000 7f2f28556640 1 -- 172.21.15.67:0/2372077392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f201a8940 con 0x7f2f20102510 2024-07-31T14:49:42.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.829+0000 7f2f137fe640 1 -- 172.21.15.67:0/2372077392 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14035070 con 0x7f2f20102510 2024-07-31T14:49:42.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.830+0000 7f2f28556640 1 -- 172.21.15.67:0/2372077392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f040052c0 con 0x7f2f20102510 2024-07-31T14:49:42.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.832+0000 7f2f137fe640 1 -- 172.21.15.67:0/2372077392 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f2f14042030 con 0x7f2f20102510 2024-07-31T14:49:42.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.833+0000 7f2f137fe640 1 --2- 172.21.15.67:0/2372077392 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f0c0b3cb0 0x7f2f0c0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:42.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.833+0000 7f2f25af1640 1 --2- 172.21.15.67:0/2372077392 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f0c0b3cb0 0x7f2f0c0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:42.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.833+0000 7f2f137fe640 1 -- 172.21.15.67:0/2372077392 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f2f14104720 con 0x7f2f20102510 2024-07-31T14:49:42.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.834+0000 7f2f25af1640 1 --2- 172.21.15.67:0/2372077392 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f0c0b3cb0 0x7f2f0c0b61c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c005fd0 tx=0x7f2f1c005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:42.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:42.838+0000 7f2f137fe640 1 -- 172.21.15.67:0/2372077392 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f2f140cb7e0 con 0x7f2f20102510 2024-07-31T14:49:43.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.036+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3109926044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc34108120 msgr2=0x7fcc34108530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:43.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.036+0000 7fcc3beb1640 1 --2- 172.21.15.67:0/3109926044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc34108120 0x7fcc34108530 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009890 tx=0x7fcc3002a380 comp rx=0 tx=0).stop 2024-07-31T14:49:43.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.036+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3109926044 shutdown_connections 2024-07-31T14:49:43.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.036+0000 7fcc3beb1640 1 --2- 172.21.15.67:0/3109926044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc34102160 0x7fcc3410ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:43.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.036+0000 7fcc3beb1640 1 --2- 172.21.15.67:0/3109926044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc34101810 0x7fcc34101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:43.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.036+0000 7fcc3beb1640 1 --2- 172.21.15.67:0/3109926044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc34108120 0x7fcc34108530 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:43.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.036+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3109926044 >> 172.21.15.67:0/3109926044 conn(0x7fcc34077690 msgr2=0x7fcc34077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:43.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.036+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3109926044 shutdown_connections 2024-07-31T14:49:43.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.036+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3109926044 wait complete. 2024-07-31T14:49:43.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.037+0000 7fcc3beb1640 1 Processor -- start 2024-07-31T14:49:43.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.037+0000 7fcc3beb1640 1 -- start start 2024-07-31T14:49:43.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.037+0000 7fcc3beb1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc34101810 0x7fcc3419c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:43.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.037+0000 7fcc3beb1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc34102160 0x7fcc3419c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:43.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.037+0000 7fcc3beb1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc34108120 0x7fcc341a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:43.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.037+0000 7fcc3beb1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcc341134c0 con 0x7fcc34108120 2024-07-31T14:49:43.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.037+0000 7fcc3beb1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcc34113340 con 0x7fcc34102160 2024-07-31T14:49:43.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.037+0000 7fcc3beb1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc34113640 con 0x7fcc34101810 2024-07-31T14:49:43.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.037+0000 7fcc3a44e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc34108120 0x7fcc341a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:43.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.037+0000 7fcc3a44e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc34108120 0x7fcc341a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45108/0 (socket says 172.21.15.67:45108) 2024-07-31T14:49:43.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.037+0000 7fcc3a44e640 1 -- 172.21.15.67:0/3489294518 learned_addr learned my addr 172.21.15.67:0/3489294518 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:43.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.038+0000 7fcc39c4d640 1 --2- 172.21.15.67:0/3489294518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc34101810 0x7fcc3419c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:43.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.038+0000 7fcc3a44e640 1 -- 172.21.15.67:0/3489294518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc34101810 msgr2=0x7fcc3419c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:43.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.038+0000 7fcc3a44e640 1 --2- 172.21.15.67:0/3489294518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc34101810 0x7fcc3419c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:43.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.038+0000 7fcc3a44e640 1 -- 172.21.15.67:0/3489294518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc34102160 msgr2=0x7fcc3419c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:43.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.038+0000 7fcc3a44e640 1 --2- 172.21.15.67:0/3489294518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc34102160 0x7fcc3419c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:43.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.038+0000 7fcc3a44e640 1 -- 172.21.15.67:0/3489294518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc341a2100 con 0x7fcc34108120 2024-07-31T14:49:43.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.038+0000 7fcc3a44e640 1 --2- 172.21.15.67:0/3489294518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc34108120 0x7fcc341a19d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009fd0 tx=0x7fcc2c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:43.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.038+0000 7fcc2affd640 1 -- 172.21.15.67:0/3489294518 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c004b20 con 0x7fcc34108120 2024-07-31T14:49:43.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.038+0000 7fcc2affd640 1 -- 172.21.15.67:0/3489294518 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc2c003e20 con 0x7fcc34108120 2024-07-31T14:49:43.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.038+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3489294518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc341a23f0 con 0x7fcc34108120 2024-07-31T14:49:43.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.038+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3489294518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc341a8b50 con 0x7fcc34108120 2024-07-31T14:49:43.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.038+0000 7fcc2affd640 1 -- 172.21.15.67:0/3489294518 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c004820 con 0x7fcc34108120 2024-07-31T14:49:43.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.040+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3489294518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc34101c20 con 0x7fcc34108120 2024-07-31T14:49:43.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.042+0000 7fcc2affd640 1 -- 172.21.15.67:0/3489294518 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fcc2c01d020 con 0x7fcc34108120 2024-07-31T14:49:43.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.043+0000 7f2f28556640 1 -- 172.21.15.67:0/2372077392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2f04005880 con 0x7f2f20102510 2024-07-31T14:49:43.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.043+0000 7fcc2affd640 1 --2- 172.21.15.67:0/3489294518 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc200b3d30 0x7fcc200b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:43.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.044+0000 7fcc2affd640 1 -- 172.21.15.67:0/3489294518 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fcc2c0e4c50 con 0x7fcc34108120 2024-07-31T14:49:43.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.044+0000 7fcc39c4d640 1 --2- 172.21.15.67:0/3489294518 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc200b3d30 0x7fcc200b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:43.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.045+0000 7fcc39c4d640 1 --2- 172.21.15.67:0/3489294518 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc200b3d30 0x7fcc200b6240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcc30000c00 tx=0x7fcc30005d20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:43.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.046+0000 7fcc2affd640 1 -- 172.21.15.67:0/3489294518 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fcc2c024030 con 0x7fcc34108120 2024-07-31T14:49:43.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.046+0000 7f2f137fe640 1 -- 172.21.15.67:0/2372077392 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2f140cb0e0 con 0x7f2f20102510 2024-07-31T14:49:43.047 INFO:teuthology.orchestra.run.smithi067.stdout:60129542239 2024-07-31T14:49:43.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.049+0000 7f2f28556640 1 -- 172.21.15.67:0/2372077392 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f0c0b3cb0 msgr2=0x7f2f0c0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:43.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.049+0000 7f2f28556640 1 --2- 172.21.15.67:0/2372077392 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f0c0b3cb0 0x7f2f0c0b61c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c005fd0 tx=0x7f2f1c005850 comp rx=0 tx=0).stop 2024-07-31T14:49:43.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.049+0000 7f2f28556640 1 -- 172.21.15.67:0/2372077392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f20102510 msgr2=0x7f2f20109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:43.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.049+0000 7f2f28556640 1 --2- 172.21.15.67:0/2372077392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f20102510 0x7f2f20109070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2f14005940 tx=0x7f2f140042f0 comp rx=0 tx=0).stop 2024-07-31T14:49:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.049+0000 7f2f28556640 1 -- 172.21.15.67:0/2372077392 shutdown_connections 2024-07-31T14:49:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.049+0000 7f2f28556640 1 --2- 172.21.15.67:0/2372077392 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2f0c0b3cb0 0x7f2f0c0b61c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.049+0000 7f2f28556640 1 --2- 172.21.15.67:0/2372077392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f20108750 0x7f2f20109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.049+0000 7f2f28556640 1 --2- 172.21.15.67:0/2372077392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f20107e00 0x7f2f201095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.049+0000 7f2f28556640 1 --2- 172.21.15.67:0/2372077392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f20102510 0x7f2f20109070 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.049+0000 7f2f28556640 1 -- 172.21.15.67:0/2372077392 >> 172.21.15.67:0/2372077392 conn(0x7f2f200791c0 msgr2=0x7f2f20100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.049+0000 7f2f28556640 1 -- 172.21.15.67:0/2372077392 shutdown_connections 2024-07-31T14:49:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.049+0000 7f2f28556640 1 -- 172.21.15.67:0/2372077392 wait complete. 2024-07-31T14:49:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:42 smithi077 ceph-mon[37327]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:49:43.260 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:42 smithi067 ceph-mon[32750]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:49:43.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.259+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3489294518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fcc341a8e50 con 0x7fcc34108120 2024-07-31T14:49:43.261 INFO:teuthology.orchestra.run.smithi067.stdout:210453397569 2024-07-31T14:49:43.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.259+0000 7fcc2affd640 1 -- 172.21.15.67:0/3489294518 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcc2c0abcb0 con 0x7fcc34108120 2024-07-31T14:49:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.261+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3489294518 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc200b3d30 msgr2=0x7fcc200b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.261+0000 7fcc3beb1640 1 --2- 172.21.15.67:0/3489294518 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc200b3d30 0x7fcc200b6240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcc30000c00 tx=0x7fcc30005d20 comp rx=0 tx=0).stop 2024-07-31T14:49:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.261+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3489294518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc34108120 msgr2=0x7fcc341a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.261+0000 7fcc3beb1640 1 --2- 172.21.15.67:0/3489294518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc34108120 0x7fcc341a19d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009fd0 tx=0x7fcc2c00c800 comp rx=0 tx=0).stop 2024-07-31T14:49:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.262+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3489294518 shutdown_connections 2024-07-31T14:49:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.262+0000 7fcc3beb1640 1 --2- 172.21.15.67:0/3489294518 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc200b3d30 0x7fcc200b6240 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.262+0000 7fcc3beb1640 1 --2- 172.21.15.67:0/3489294518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc34108120 0x7fcc341a19d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.262+0000 7fcc3beb1640 1 --2- 172.21.15.67:0/3489294518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc34102160 0x7fcc3419c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.262+0000 7fcc3beb1640 1 --2- 172.21.15.67:0/3489294518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc34101810 0x7fcc3419c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.262+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3489294518 >> 172.21.15.67:0/3489294518 conn(0x7fcc34077690 msgr2=0x7fcc34106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:43.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.262+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3489294518 shutdown_connections 2024-07-31T14:49:43.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:43.262+0000 7fcc3beb1640 1 -- 172.21.15.67:0/3489294518 wait complete. 2024-07-31T14:49:43.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:42 smithi073 ceph-mon[36714]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:49:43.886 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397563 got 210453397569 for osd.7 2024-07-31T14:49:43.886 DEBUG:teuthology.parallel:result is None 2024-07-31T14:49:44.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:43 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2372077392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-31T14:49:44.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:43 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3489294518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-31T14:49:44.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:43 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2372077392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-31T14:49:44.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:43 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3489294518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-31T14:49:44.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:43 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2372077392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-31T14:49:44.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:43 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3489294518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-31T14:49:44.398 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542233 got 60129542239 for osd.1 2024-07-31T14:49:44.398 DEBUG:teuthology.parallel:result is None 2024-07-31T14:49:44.398 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-31T14:49:44.399 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph pg dump --format=json 2024-07-31T14:49:44.674 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:45.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:44 smithi067 ceph-mon[32750]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:49:45.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:44 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:45.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:44 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:45.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:44 smithi077 ceph-mon[37327]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:49:45.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:44 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:45.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:44 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:45.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:44 smithi073 ceph-mon[36714]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:49:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:44 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:44 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:46.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:45 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:49:46.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:45 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:49:46.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:45 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:45 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:49:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:45 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:49:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:45 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.373+0000 7f946fb74640 1 -- 172.21.15.67:0/3483936067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f946810fbf0 msgr2=0x7f9468112010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.373+0000 7f946fb74640 1 --2- 172.21.15.67:0/3483936067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f946810fbf0 0x7f9468112010 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9460009ca0 tx=0x7f946002a950 comp rx=0 tx=0).stop 2024-07-31T14:49:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.373+0000 7f946fb74640 1 -- 172.21.15.67:0/3483936067 shutdown_connections 2024-07-31T14:49:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.373+0000 7f946fb74640 1 --2- 172.21.15.67:0/3483936067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f946810fbf0 0x7f9468112010 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.373+0000 7f946fb74640 1 --2- 172.21.15.67:0/3483936067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9468102990 0x7f946810f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.373+0000 7f946fb74640 1 --2- 172.21.15.67:0/3483936067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9468102040 0x7f9468102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.373+0000 7f946fb74640 1 -- 172.21.15.67:0/3483936067 >> 172.21.15.67:0/3483936067 conn(0x7f9468079190 msgr2=0x7f94680795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.373+0000 7f946fb74640 1 -- 172.21.15.67:0/3483936067 shutdown_connections 2024-07-31T14:49:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.373+0000 7f946fb74640 1 -- 172.21.15.67:0/3483936067 wait complete. 2024-07-31T14:49:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.373+0000 7f946fb74640 1 Processor -- start 2024-07-31T14:49:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946fb74640 1 -- start start 2024-07-31T14:49:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946fb74640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9468102040 0x7f946819c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946fb74640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9468102990 0x7f946819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946fb74640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f946810fbf0 0x7f94681a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946fb74640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f94681156a0 con 0x7f9468102040 2024-07-31T14:49:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946fb74640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9468115520 con 0x7f946810fbf0 2024-07-31T14:49:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946fb74640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9468115820 con 0x7f9468102990 2024-07-31T14:49:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946d910640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9468102040 0x7f946819c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946d910640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9468102040 0x7f946819c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45128/0 (socket says 172.21.15.67:45128) 2024-07-31T14:49:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946d910640 1 -- 172.21.15.67:0/168259717 learned_addr learned my addr 172.21.15.67:0/168259717 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946e111640 1 --2- 172.21.15.67:0/168259717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f946810fbf0 0x7f94681a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:46.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946d10f640 1 --2- 172.21.15.67:0/168259717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9468102990 0x7f946819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:46.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946d910640 1 -- 172.21.15.67:0/168259717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9468102990 msgr2=0x7f946819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:46.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946d910640 1 --2- 172.21.15.67:0/168259717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9468102990 0x7f946819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:46.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946d910640 1 -- 172.21.15.67:0/168259717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f946810fbf0 msgr2=0x7f94681a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:46.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946d910640 1 --2- 172.21.15.67:0/168259717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f946810fbf0 0x7f94681a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:46.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946d910640 1 -- 172.21.15.67:0/168259717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94681a2000 con 0x7f9468102040 2024-07-31T14:49:46.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.374+0000 7f946d910640 1 --2- 172.21.15.67:0/168259717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9468102040 0x7f946819c090 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f946400d770 tx=0x7f946400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:46.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.375+0000 7f945effd640 1 -- 172.21.15.67:0/168259717 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464017070 con 0x7f9468102040 2024-07-31T14:49:46.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.375+0000 7f945effd640 1 -- 172.21.15.67:0/168259717 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94640054d0 con 0x7f9468102040 2024-07-31T14:49:46.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.375+0000 7f945effd640 1 -- 172.21.15.67:0/168259717 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464004c30 con 0x7f9468102040 2024-07-31T14:49:46.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.375+0000 7f946fb74640 1 -- 172.21.15.67:0/168259717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94681a22f0 con 0x7f9468102040 2024-07-31T14:49:46.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.375+0000 7f946fb74640 1 -- 172.21.15.67:0/168259717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94681a8a50 con 0x7f9468102040 2024-07-31T14:49:46.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.376+0000 7f946fb74640 1 -- 172.21.15.67:0/168259717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f946806bc70 con 0x7f9468102040 2024-07-31T14:49:46.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.378+0000 7f945effd640 1 -- 172.21.15.67:0/168259717 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f9464004dd0 con 0x7f9468102040 2024-07-31T14:49:46.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.378+0000 7f945effd640 1 --2- 172.21.15.67:0/168259717 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94540b3cb0 0x7f94540b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:46.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.378+0000 7f945effd640 1 -- 172.21.15.67:0/168259717 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f94640e40c0 con 0x7f9468102040 2024-07-31T14:49:46.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.378+0000 7f946d10f640 1 --2- 172.21.15.67:0/168259717 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94540b3cb0 0x7f94540b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:46.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.379+0000 7f946d10f640 1 --2- 172.21.15.67:0/168259717 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94540b3cb0 0x7f94540b61c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9458005d90 tx=0x7f9458005d20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:46.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.381+0000 7f945effd640 1 -- 172.21.15.67:0/168259717 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f94640ab180 con 0x7f9468102040 2024-07-31T14:49:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:45 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:49:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:45 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:49:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:45 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:49:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.572+0000 7f946fb74640 1 -- 172.21.15.67:0/168259717 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9468108d70 con 0x7f94540b3cb0 2024-07-31T14:49:46.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.578+0000 7f945effd640 1 -- 172.21.15.67:0/168259717 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340415 (secure 0 0 0) 0x7f9468108d70 con 0x7f94540b3cb0 2024-07-31T14:49:46.584 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:49:46.586 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-31T14:49:46.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.581+0000 7f946fb74640 1 -- 172.21.15.67:0/168259717 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94540b3cb0 msgr2=0x7f94540b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:46.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.581+0000 7f946fb74640 1 --2- 172.21.15.67:0/168259717 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94540b3cb0 0x7f94540b61c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9458005d90 tx=0x7f9458005d20 comp rx=0 tx=0).stop 2024-07-31T14:49:46.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.581+0000 7f946fb74640 1 -- 172.21.15.67:0/168259717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9468102040 msgr2=0x7f946819c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:46.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.581+0000 7f946fb74640 1 --2- 172.21.15.67:0/168259717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9468102040 0x7f946819c090 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f946400d770 tx=0x7f946400dc30 comp rx=0 tx=0).stop 2024-07-31T14:49:46.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.581+0000 7f946fb74640 1 -- 172.21.15.67:0/168259717 shutdown_connections 2024-07-31T14:49:46.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.581+0000 7f946fb74640 1 --2- 172.21.15.67:0/168259717 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94540b3cb0 0x7f94540b61c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:46.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.581+0000 7f946fb74640 1 --2- 172.21.15.67:0/168259717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f946810fbf0 0x7f94681a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:46.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.581+0000 7f946fb74640 1 --2- 172.21.15.67:0/168259717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9468102990 0x7f946819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:46.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.581+0000 7f946fb74640 1 --2- 172.21.15.67:0/168259717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9468102040 0x7f946819c090 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:46.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.581+0000 7f946fb74640 1 -- 172.21.15.67:0/168259717 >> 172.21.15.67:0/168259717 conn(0x7f9468079190 msgr2=0x7f9468108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:46.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.581+0000 7f946fb74640 1 -- 172.21.15.67:0/168259717 shutdown_connections 2024-07-31T14:49:46.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:46.582+0000 7f946fb74640 1 -- 172.21.15.67:0/168259717 wait complete. 2024-07-31T14:49:47.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:46 smithi067 ceph-mon[32750]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:49:47.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:46 smithi077 ceph-mon[37327]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:49:47.265 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-07-31T14:49:45.417685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662333,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1126,"num_read_kb":673,"num_write":374,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379936,"kb_used_data":14144,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561888,"statfs":{"total":1151940427776,"available":1151551373312,"internally_reserved":0,"allocated":14483456,"data_stored":8192369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003714"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":153,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:39.765232+0000","last_change":"2024-07-31T14:47:00.020067+0000","last_active":"2024-07-31T14:49:39.765232+0000","last_peered":"2024-07-31T14:49:39.765232+0000","last_clean":"2024-07-31T14:49:39.765232+0000","last_became_active":"2024-07-31T14:47:00.019674+0000","last_became_peered":"2024-07-31T14:47:00.019674+0000","last_unstale":"2024-07-31T14:49:39.765232+0000","last_undegraded":"2024-07-31T14:49:39.765232+0000","last_fullsized":"2024-07-31T14:49:39.765232+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:10:08.193786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626715+0000","last_change":"2024-07-31T14:47:03.052069+0000","last_active":"2024-07-31T14:47:20.626715+0000","last_peered":"2024-07-31T14:47:20.626715+0000","last_clean":"2024-07-31T14:47:20.626715+0000","last_became_active":"2024-07-31T14:47:03.051739+0000","last_became_peered":"2024-07-31T14:47:03.051739+0000","last_unstale":"2024-07-31T14:47:20.626715+0000","last_undegraded":"2024-07-31T14:47:20.626715+0000","last_fullsized":"2024-07-31T14:47:20.626715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:29:19.952794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.629384+0000","last_change":"2024-07-31T14:47:20.628770+0000","last_active":"2024-07-31T14:47:20.629384+0000","last_peered":"2024-07-31T14:47:20.629384+0000","last_clean":"2024-07-31T14:47:20.629384+0000","last_became_active":"2024-07-31T14:47:20.628142+0000","last_became_peered":"2024-07-31T14:47:20.628142+0000","last_unstale":"2024-07-31T14:47:20.629384+0000","last_undegraded":"2024-07-31T14:47:20.629384+0000","last_fullsized":"2024-07-31T14:47:20.629384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.659670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:00.571472+0000","last_change":"2024-07-31T14:47:00.183893+0000","last_active":"2024-07-31T14:48:00.571472+0000","last_peered":"2024-07-31T14:48:00.571472+0000","last_clean":"2024-07-31T14:48:00.571472+0000","last_became_active":"2024-07-31T14:47:00.183453+0000","last_became_peered":"2024-07-31T14:47:00.183453+0000","last_unstale":"2024-07-31T14:48:00.571472+0000","last_undegraded":"2024-07-31T14:48:00.571472+0000","last_fullsized":"2024-07-31T14:48:00.571472+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:45:08.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449756+0000","last_change":"2024-07-31T14:47:03.057081+0000","last_active":"2024-07-31T14:47:20.449756+0000","last_peered":"2024-07-31T14:47:20.449756+0000","last_clean":"2024-07-31T14:47:20.449756+0000","last_became_active":"2024-07-31T14:47:03.056061+0000","last_became_peered":"2024-07-31T14:47:03.056061+0000","last_unstale":"2024-07-31T14:47:20.449756+0000","last_undegraded":"2024-07-31T14:47:20.449756+0000","last_fullsized":"2024-07-31T14:47:20.449756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:28:16.760131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604121+0000","last_change":"2024-07-31T14:47:13.105271+0000","last_active":"2024-07-31T14:47:20.604121+0000","last_peered":"2024-07-31T14:47:20.604121+0000","last_clean":"2024-07-31T14:47:20.604121+0000","last_became_active":"2024-07-31T14:47:13.104717+0000","last_became_peered":"2024-07-31T14:47:13.104717+0000","last_unstale":"2024-07-31T14:47:20.604121+0000","last_undegraded":"2024-07-31T14:47:20.604121+0000","last_fullsized":"2024-07-31T14:47:20.604121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:34:21.413957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605719+0000","last_change":"2024-07-31T14:47:20.454745+0000","last_active":"2024-07-31T14:48:03.605719+0000","last_peered":"2024-07-31T14:48:03.605719+0000","last_clean":"2024-07-31T14:48:03.605719+0000","last_became_active":"2024-07-31T14:47:20.449892+0000","last_became_peered":"2024-07-31T14:47:20.449892+0000","last_unstale":"2024-07-31T14:48:03.605719+0000","last_undegraded":"2024-07-31T14:48:03.605719+0000","last_fullsized":"2024-07-31T14:48:03.605719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:59:17.375344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626269+0000","last_change":"2024-07-31T14:47:03.051085+0000","last_active":"2024-07-31T14:47:20.626269+0000","last_peered":"2024-07-31T14:47:20.626269+0000","last_clean":"2024-07-31T14:47:20.626269+0000","last_became_active":"2024-07-31T14:47:03.050686+0000","last_became_peered":"2024-07-31T14:47:03.050686+0000","last_unstale":"2024-07-31T14:47:20.626269+0000","last_undegraded":"2024-07-31T14:47:20.626269+0000","last_fullsized":"2024-07-31T14:47:20.626269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:48:02.494061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604501+0000","last_change":"2024-07-31T14:47:13.104229+0000","last_active":"2024-07-31T14:47:20.604501+0000","last_peered":"2024-07-31T14:47:20.604501+0000","last_clean":"2024-07-31T14:47:20.604501+0000","last_became_active":"2024-07-31T14:47:13.103737+0000","last_became_peered":"2024-07-31T14:47:13.103737+0000","last_unstale":"2024-07-31T14:47:20.604501+0000","last_undegraded":"2024-07-31T14:47:20.604501+0000","last_fullsized":"2024-07-31T14:47:20.604501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:36:00.541345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609858+0000","last_change":"2024-07-31T14:47:00.028169+0000","last_active":"2024-07-31T14:48:03.609858+0000","last_peered":"2024-07-31T14:48:03.609858+0000","last_clean":"2024-07-31T14:48:03.609858+0000","last_became_active":"2024-07-31T14:47:00.027757+0000","last_became_peered":"2024-07-31T14:47:00.027757+0000","last_unstale":"2024-07-31T14:48:03.609858+0000","last_undegraded":"2024-07-31T14:48:03.609858+0000","last_fullsized":"2024-07-31T14:48:03.609858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:36:04.419551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626959+0000","last_change":"2024-07-31T14:47:03.057400+0000","last_active":"2024-07-31T14:47:20.626959+0000","last_peered":"2024-07-31T14:47:20.626959+0000","last_clean":"2024-07-31T14:47:20.626959+0000","last_became_active":"2024-07-31T14:47:03.057035+0000","last_became_peered":"2024-07-31T14:47:03.057035+0000","last_unstale":"2024-07-31T14:47:20.626959+0000","last_undegraded":"2024-07-31T14:47:20.626959+0000","last_fullsized":"2024-07-31T14:47:20.626959+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:55:07.951076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604244+0000","last_change":"2024-07-31T14:47:13.106093+0000","last_active":"2024-07-31T14:47:20.604244+0000","last_peered":"2024-07-31T14:47:20.604244+0000","last_clean":"2024-07-31T14:47:20.604244+0000","last_became_active":"2024-07-31T14:47:13.105459+0000","last_became_peered":"2024-07-31T14:47:13.105459+0000","last_unstale":"2024-07-31T14:47:20.604244+0000","last_undegraded":"2024-07-31T14:47:20.604244+0000","last_fullsized":"2024-07-31T14:47:20.604244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:07:52.751245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444407+0000","last_change":"2024-07-31T14:47:03.058701+0000","last_active":"2024-07-31T14:47:20.444407+0000","last_peered":"2024-07-31T14:47:20.444407+0000","last_clean":"2024-07-31T14:47:20.444407+0000","last_became_active":"2024-07-31T14:47:03.058359+0000","last_became_peered":"2024-07-31T14:47:03.058359+0000","last_unstale":"2024-07-31T14:47:20.444407+0000","last_undegraded":"2024-07-31T14:47:20.444407+0000","last_fullsized":"2024-07-31T14:47:20.444407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:37:36.149884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605126+0000","last_change":"2024-07-31T14:47:00.022983+0000","last_active":"2024-07-31T14:48:03.605126+0000","last_peered":"2024-07-31T14:48:03.605126+0000","last_clean":"2024-07-31T14:48:03.605126+0000","last_became_active":"2024-07-31T14:47:00.022586+0000","last_became_peered":"2024-07-31T14:47:00.022586+0000","last_unstale":"2024-07-31T14:48:03.605126+0000","last_undegraded":"2024-07-31T14:48:03.605126+0000","last_fullsized":"2024-07-31T14:48:03.605126+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:29:42.641490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604474+0000","last_change":"2024-07-31T14:47:13.363148+0000","last_active":"2024-07-31T14:47:20.604474+0000","last_peered":"2024-07-31T14:47:20.604474+0000","last_clean":"2024-07-31T14:47:20.604474+0000","last_became_active":"2024-07-31T14:47:13.362577+0000","last_became_peered":"2024-07-31T14:47:13.362577+0000","last_unstale":"2024-07-31T14:47:20.604474+0000","last_undegraded":"2024-07-31T14:47:20.604474+0000","last_fullsized":"2024-07-31T14:47:20.604474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:37:59.003260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626453+0000","last_change":"2024-07-31T14:47:03.049586+0000","last_active":"2024-07-31T14:47:20.626453+0000","last_peered":"2024-07-31T14:47:20.626453+0000","last_clean":"2024-07-31T14:47:20.626453+0000","last_became_active":"2024-07-31T14:47:03.049154+0000","last_became_peered":"2024-07-31T14:47:03.049154+0000","last_unstale":"2024-07-31T14:47:20.626453+0000","last_undegraded":"2024-07-31T14:47:20.626453+0000","last_fullsized":"2024-07-31T14:47:20.626453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:51:01.971854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.585353+0000","last_change":"2024-07-31T14:47:00.021254+0000","last_active":"2024-07-31T14:48:02.585353+0000","last_peered":"2024-07-31T14:48:02.585353+0000","last_clean":"2024-07-31T14:48:02.585353+0000","last_became_active":"2024-07-31T14:47:00.020690+0000","last_became_peered":"2024-07-31T14:47:00.020690+0000","last_unstale":"2024-07-31T14:48:02.585353+0000","last_undegraded":"2024-07-31T14:48:02.585353+0000","last_fullsized":"2024-07-31T14:48:02.585353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:01:27.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:18.046730+0000","last_change":"2024-07-31T14:47:13.102220+0000","last_active":"2024-07-31T14:48:18.046730+0000","last_peered":"2024-07-31T14:48:18.046730+0000","last_clean":"2024-07-31T14:48:18.046730+0000","last_became_active":"2024-07-31T14:47:13.101764+0000","last_became_peered":"2024-07-31T14:47:13.101764+0000","last_unstale":"2024-07-31T14:48:18.046730+0000","last_undegraded":"2024-07-31T14:48:18.046730+0000","last_fullsized":"2024-07-31T14:48:18.046730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:56:44.335340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444979+0000","last_change":"2024-07-31T14:47:03.052348+0000","last_active":"2024-07-31T14:47:20.444979+0000","last_peered":"2024-07-31T14:47:20.444979+0000","last_clean":"2024-07-31T14:47:20.444979+0000","last_became_active":"2024-07-31T14:47:03.051961+0000","last_became_peered":"2024-07-31T14:47:03.051961+0000","last_unstale":"2024-07-31T14:47:20.444979+0000","last_undegraded":"2024-07-31T14:47:20.444979+0000","last_fullsized":"2024-07-31T14:47:20.444979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:19:35.279809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.597386+0000","last_change":"2024-07-31T14:47:00.025013+0000","last_active":"2024-07-31T14:48:02.597386+0000","last_peered":"2024-07-31T14:48:02.597386+0000","last_clean":"2024-07-31T14:48:02.597386+0000","last_became_active":"2024-07-31T14:47:00.024637+0000","last_became_peered":"2024-07-31T14:47:00.024637+0000","last_unstale":"2024-07-31T14:48:02.597386+0000","last_undegraded":"2024-07-31T14:48:02.597386+0000","last_fullsized":"2024-07-31T14:48:02.597386+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:07:21.519549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630339+0000","last_change":"2024-07-31T14:47:20.629262+0000","last_active":"2024-07-31T14:47:20.630339+0000","last_peered":"2024-07-31T14:47:20.630339+0000","last_clean":"2024-07-31T14:47:20.630339+0000","last_became_active":"2024-07-31T14:47:20.628318+0000","last_became_peered":"2024-07-31T14:47:20.628318+0000","last_unstale":"2024-07-31T14:47:20.630339+0000","last_undegraded":"2024-07-31T14:47:20.630339+0000","last_fullsized":"2024-07-31T14:47:20.630339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.206705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451128+0000","last_change":"2024-07-31T14:47:03.051527+0000","last_active":"2024-07-31T14:47:20.451128+0000","last_peered":"2024-07-31T14:47:20.451128+0000","last_clean":"2024-07-31T14:47:20.451128+0000","last_became_active":"2024-07-31T14:47:03.051139+0000","last_became_peered":"2024-07-31T14:47:03.051139+0000","last_unstale":"2024-07-31T14:47:20.451128+0000","last_undegraded":"2024-07-31T14:47:20.451128+0000","last_fullsized":"2024-07-31T14:47:20.451128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:53:11.962719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:59.566389+0000","last_change":"2024-07-31T14:47:00.183812+0000","last_active":"2024-07-31T14:47:59.566389+0000","last_peered":"2024-07-31T14:47:59.566389+0000","last_clean":"2024-07-31T14:47:59.566389+0000","last_became_active":"2024-07-31T14:47:00.183440+0000","last_became_peered":"2024-07-31T14:47:00.183440+0000","last_unstale":"2024-07-31T14:47:59.566389+0000","last_undegraded":"2024-07-31T14:47:59.566389+0000","last_fullsized":"2024-07-31T14:47:59.566389+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:47:23.166756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604338+0000","last_change":"2024-07-31T14:47:13.363563+0000","last_active":"2024-07-31T14:47:20.604338+0000","last_peered":"2024-07-31T14:47:20.604338+0000","last_clean":"2024-07-31T14:47:20.604338+0000","last_became_active":"2024-07-31T14:47:13.362697+0000","last_became_peered":"2024-07-31T14:47:13.362697+0000","last_unstale":"2024-07-31T14:47:20.604338+0000","last_undegraded":"2024-07-31T14:47:20.604338+0000","last_fullsized":"2024-07-31T14:47:20.604338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:26:38.292073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449305+0000","last_change":"2024-07-31T14:47:03.046763+0000","last_active":"2024-07-31T14:47:20.449305+0000","last_peered":"2024-07-31T14:47:20.449305+0000","last_clean":"2024-07-31T14:47:20.449305+0000","last_became_active":"2024-07-31T14:47:03.046408+0000","last_became_peered":"2024-07-31T14:47:03.046408+0000","last_unstale":"2024-07-31T14:47:20.449305+0000","last_undegraded":"2024-07-31T14:47:20.449305+0000","last_fullsized":"2024-07-31T14:47:20.449305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:43:22.838495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605677+0000","last_change":"2024-07-31T14:47:00.236075+0000","last_active":"2024-07-31T14:48:03.605677+0000","last_peered":"2024-07-31T14:48:03.605677+0000","last_clean":"2024-07-31T14:48:03.605677+0000","last_became_active":"2024-07-31T14:47:00.235758+0000","last_became_peered":"2024-07-31T14:47:00.235758+0000","last_unstale":"2024-07-31T14:48:03.605677+0000","last_undegraded":"2024-07-31T14:48:03.605677+0000","last_fullsized":"2024-07-31T14:48:03.605677+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:50.575648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.448218+0000","last_change":"2024-07-31T14:47:13.101876+0000","last_active":"2024-07-31T14:47:20.448218+0000","last_peered":"2024-07-31T14:47:20.448218+0000","last_clean":"2024-07-31T14:47:20.448218+0000","last_became_active":"2024-07-31T14:47:13.101276+0000","last_became_peered":"2024-07-31T14:47:13.101276+0000","last_unstale":"2024-07-31T14:47:20.448218+0000","last_undegraded":"2024-07-31T14:47:20.448218+0000","last_fullsized":"2024-07-31T14:47:20.448218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:53:24.684212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627393+0000","last_change":"2024-07-31T14:47:03.052478+0000","last_active":"2024-07-31T14:47:20.627393+0000","last_peered":"2024-07-31T14:47:20.627393+0000","last_clean":"2024-07-31T14:47:20.627393+0000","last_became_active":"2024-07-31T14:47:03.051877+0000","last_became_peered":"2024-07-31T14:47:03.051877+0000","last_unstale":"2024-07-31T14:47:20.627393+0000","last_undegraded":"2024-07-31T14:47:20.627393+0000","last_fullsized":"2024-07-31T14:47:20.627393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:30:32.666490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'14","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:27.059299+0000","last_change":"2024-07-31T14:47:00.020076+0000","last_active":"2024-07-31T14:48:27.059299+0000","last_peered":"2024-07-31T14:48:27.059299+0000","last_clean":"2024-07-31T14:48:27.059299+0000","last_became_active":"2024-07-31T14:47:00.019708+0000","last_became_peered":"2024-07-31T14:47:00.019708+0000","last_unstale":"2024-07-31T14:48:27.059299+0000","last_undegraded":"2024-07-31T14:48:27.059299+0000","last_fullsized":"2024-07-31T14:48:27.059299+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:46.520997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625840+0000","last_change":"2024-07-31T14:47:13.362961+0000","last_active":"2024-07-31T14:47:20.625840+0000","last_peered":"2024-07-31T14:47:20.625840+0000","last_clean":"2024-07-31T14:47:20.625840+0000","last_became_active":"2024-07-31T14:47:13.362502+0000","last_became_peered":"2024-07-31T14:47:13.362502+0000","last_unstale":"2024-07-31T14:47:20.625840+0000","last_undegraded":"2024-07-31T14:47:20.625840+0000","last_fullsized":"2024-07-31T14:47:20.625840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:51:45.732477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447518+0000","last_change":"2024-07-31T14:47:03.039418+0000","last_active":"2024-07-31T14:47:20.447518+0000","last_peered":"2024-07-31T14:47:20.447518+0000","last_clean":"2024-07-31T14:47:20.447518+0000","last_became_active":"2024-07-31T14:47:03.039044+0000","last_became_peered":"2024-07-31T14:47:03.039044+0000","last_unstale":"2024-07-31T14:47:20.447518+0000","last_undegraded":"2024-07-31T14:47:20.447518+0000","last_fullsized":"2024-07-31T14:47:20.447518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:40:10.380080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604784+0000","last_change":"2024-07-31T14:47:00.183778+0000","last_active":"2024-07-31T14:48:03.604784+0000","last_peered":"2024-07-31T14:48:03.604784+0000","last_clean":"2024-07-31T14:48:03.604784+0000","last_became_active":"2024-07-31T14:47:00.183370+0000","last_became_peered":"2024-07-31T14:47:00.183370+0000","last_unstale":"2024-07-31T14:48:03.604784+0000","last_undegraded":"2024-07-31T14:48:03.604784+0000","last_fullsized":"2024-07-31T14:48:03.604784+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:45:27.422216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445139+0000","last_change":"2024-07-31T14:47:13.102583+0000","last_active":"2024-07-31T14:47:20.445139+0000","last_peered":"2024-07-31T14:47:20.445139+0000","last_clean":"2024-07-31T14:47:20.445139+0000","last_became_active":"2024-07-31T14:47:13.102058+0000","last_became_peered":"2024-07-31T14:47:13.102058+0000","last_unstale":"2024-07-31T14:47:20.445139+0000","last_undegraded":"2024-07-31T14:47:20.445139+0000","last_fullsized":"2024-07-31T14:47:20.445139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:17.396173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626607+0000","last_change":"2024-07-31T14:47:03.048334+0000","last_active":"2024-07-31T14:47:20.626607+0000","last_peered":"2024-07-31T14:47:20.626607+0000","last_clean":"2024-07-31T14:47:20.626607+0000","last_became_active":"2024-07-31T14:47:03.047932+0000","last_became_peered":"2024-07-31T14:47:03.047932+0000","last_unstale":"2024-07-31T14:47:20.626607+0000","last_undegraded":"2024-07-31T14:47:20.626607+0000","last_fullsized":"2024-07-31T14:47:20.626607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:14:31.381061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.591990+0000","last_change":"2024-07-31T14:47:20.450851+0000","last_active":"2024-07-31T14:48:01.591990+0000","last_peered":"2024-07-31T14:48:01.591990+0000","last_clean":"2024-07-31T14:48:01.591990+0000","last_became_active":"2024-07-31T14:47:20.450435+0000","last_became_peered":"2024-07-31T14:47:20.450435+0000","last_unstale":"2024-07-31T14:48:01.591990+0000","last_undegraded":"2024-07-31T14:48:01.591990+0000","last_fullsized":"2024-07-31T14:48:01.591990+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:17:28.738911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.450860+0000","last_change":"2024-07-31T14:47:13.363054+0000","last_active":"2024-07-31T14:47:20.450860+0000","last_peered":"2024-07-31T14:47:20.450860+0000","last_clean":"2024-07-31T14:47:20.450860+0000","last_became_active":"2024-07-31T14:47:13.362364+0000","last_became_peered":"2024-07-31T14:47:13.362364+0000","last_unstale":"2024-07-31T14:47:20.450860+0000","last_undegraded":"2024-07-31T14:47:20.450860+0000","last_fullsized":"2024-07-31T14:47:20.450860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:02:36.569678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626742+0000","last_change":"2024-07-31T14:47:03.059834+0000","last_active":"2024-07-31T14:47:20.626742+0000","last_peered":"2024-07-31T14:47:20.626742+0000","last_clean":"2024-07-31T14:47:20.626742+0000","last_became_active":"2024-07-31T14:47:03.059127+0000","last_became_peered":"2024-07-31T14:47:03.059127+0000","last_unstale":"2024-07-31T14:47:20.626742+0000","last_undegraded":"2024-07-31T14:47:20.626742+0000","last_fullsized":"2024-07-31T14:47:20.626742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:53:50.308789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:39.765823+0000","last_change":"2024-07-31T14:47:00.030576+0000","last_active":"2024-07-31T14:49:39.765823+0000","last_peered":"2024-07-31T14:49:39.765823+0000","last_clean":"2024-07-31T14:49:39.765823+0000","last_became_active":"2024-07-31T14:47:00.030174+0000","last_became_peered":"2024-07-31T14:47:00.030174+0000","last_unstale":"2024-07-31T14:49:39.765823+0000","last_undegraded":"2024-07-31T14:49:39.765823+0000","last_fullsized":"2024-07-31T14:49:39.765823+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:13:35.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447258+0000","last_change":"2024-07-31T14:47:13.100180+0000","last_active":"2024-07-31T14:47:20.447258+0000","last_peered":"2024-07-31T14:47:20.447258+0000","last_clean":"2024-07-31T14:47:20.447258+0000","last_became_active":"2024-07-31T14:47:13.099725+0000","last_became_peered":"2024-07-31T14:47:13.099725+0000","last_unstale":"2024-07-31T14:47:20.447258+0000","last_undegraded":"2024-07-31T14:47:20.447258+0000","last_fullsized":"2024-07-31T14:47:20.447258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:25:32.245339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626691+0000","last_change":"2024-07-31T14:47:03.040349+0000","last_active":"2024-07-31T14:47:20.626691+0000","last_peered":"2024-07-31T14:47:20.626691+0000","last_clean":"2024-07-31T14:47:20.626691+0000","last_became_active":"2024-07-31T14:47:03.039887+0000","last_became_peered":"2024-07-31T14:47:03.039887+0000","last_unstale":"2024-07-31T14:47:20.626691+0000","last_undegraded":"2024-07-31T14:47:20.626691+0000","last_fullsized":"2024-07-31T14:47:20.626691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:46.592063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:41.750693+0000","last_change":"2024-07-31T14:47:00.183988+0000","last_active":"2024-07-31T14:49:41.750693+0000","last_peered":"2024-07-31T14:49:41.750693+0000","last_clean":"2024-07-31T14:49:41.750693+0000","last_became_active":"2024-07-31T14:47:00.183302+0000","last_became_peered":"2024-07-31T14:47:00.183302+0000","last_unstale":"2024-07-31T14:49:41.750693+0000","last_undegraded":"2024-07-31T14:49:41.750693+0000","last_fullsized":"2024-07-31T14:49:41.750693+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:40:26.188224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447744+0000","last_change":"2024-07-31T14:47:13.363075+0000","last_active":"2024-07-31T14:47:20.447744+0000","last_peered":"2024-07-31T14:47:20.447744+0000","last_clean":"2024-07-31T14:47:20.447744+0000","last_became_active":"2024-07-31T14:47:13.362599+0000","last_became_peered":"2024-07-31T14:47:13.362599+0000","last_unstale":"2024-07-31T14:47:20.447744+0000","last_undegraded":"2024-07-31T14:47:20.447744+0000","last_fullsized":"2024-07-31T14:47:20.447744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:20:43.617742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451262+0000","last_change":"2024-07-31T14:47:20.450712+0000","last_active":"2024-07-31T14:47:20.451262+0000","last_peered":"2024-07-31T14:47:20.451262+0000","last_clean":"2024-07-31T14:47:20.451262+0000","last_became_active":"2024-07-31T14:47:20.450229+0000","last_became_peered":"2024-07-31T14:47:20.450229+0000","last_unstale":"2024-07-31T14:47:20.451262+0000","last_undegraded":"2024-07-31T14:47:20.451262+0000","last_fullsized":"2024-07-31T14:47:20.451262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:11:50.631420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600465+0000","last_change":"2024-07-31T14:47:00.021389+0000","last_active":"2024-07-31T14:48:02.600465+0000","last_peered":"2024-07-31T14:48:02.600465+0000","last_clean":"2024-07-31T14:48:02.600465+0000","last_became_active":"2024-07-31T14:47:00.020987+0000","last_became_peered":"2024-07-31T14:47:00.020987+0000","last_unstale":"2024-07-31T14:48:02.600465+0000","last_undegraded":"2024-07-31T14:48:02.600465+0000","last_fullsized":"2024-07-31T14:48:02.600465+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:07:24.658463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627003+0000","last_change":"2024-07-31T14:47:13.363105+0000","last_active":"2024-07-31T14:47:20.627003+0000","last_peered":"2024-07-31T14:47:20.627003+0000","last_clean":"2024-07-31T14:47:20.627003+0000","last_became_active":"2024-07-31T14:47:13.362404+0000","last_became_peered":"2024-07-31T14:47:13.362404+0000","last_unstale":"2024-07-31T14:47:20.627003+0000","last_undegraded":"2024-07-31T14:47:20.627003+0000","last_fullsized":"2024-07-31T14:47:20.627003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:48:43.033584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626198+0000","last_change":"2024-07-31T14:47:03.057199+0000","last_active":"2024-07-31T14:47:20.626198+0000","last_peered":"2024-07-31T14:47:20.626198+0000","last_clean":"2024-07-31T14:47:20.626198+0000","last_became_active":"2024-07-31T14:47:03.056555+0000","last_became_peered":"2024-07-31T14:47:03.056555+0000","last_unstale":"2024-07-31T14:47:20.626198+0000","last_undegraded":"2024-07-31T14:47:20.626198+0000","last_fullsized":"2024-07-31T14:47:20.626198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:21:14.263457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605605+0000","last_change":"2024-07-31T14:47:00.032055+0000","last_active":"2024-07-31T14:48:03.605605+0000","last_peered":"2024-07-31T14:48:03.605605+0000","last_clean":"2024-07-31T14:48:03.605605+0000","last_became_active":"2024-07-31T14:47:00.031519+0000","last_became_peered":"2024-07-31T14:47:00.031519+0000","last_unstale":"2024-07-31T14:48:03.605605+0000","last_undegraded":"2024-07-31T14:48:03.605605+0000","last_fullsized":"2024-07-31T14:48:03.605605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:21:46.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444468+0000","last_change":"2024-07-31T14:45:42.166503+0000","last_active":"2024-07-31T14:47:20.444468+0000","last_peered":"2024-07-31T14:47:20.444468+0000","last_clean":"2024-07-31T14:47:20.444468+0000","last_became_active":"2024-07-31T14:45:42.160065+0000","last_became_peered":"2024-07-31T14:45:42.160065+0000","last_unstale":"2024-07-31T14:47:20.444468+0000","last_undegraded":"2024-07-31T14:47:20.444468+0000","last_fullsized":"2024-07-31T14:47:20.444468+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_clean_scrub_stamp":"2024-07-31T14:42:27.863113+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:38:33.646022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:12.369486+0000","last_change":"2024-07-31T14:47:13.103812+0000","last_active":"2024-07-31T14:48:12.369486+0000","last_peered":"2024-07-31T14:48:12.369486+0000","last_clean":"2024-07-31T14:48:12.369486+0000","last_became_active":"2024-07-31T14:47:13.102549+0000","last_became_peered":"2024-07-31T14:47:13.102549+0000","last_unstale":"2024-07-31T14:48:12.369486+0000","last_undegraded":"2024-07-31T14:48:12.369486+0000","last_fullsized":"2024-07-31T14:48:12.369486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:18:56.322426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444907+0000","last_change":"2024-07-31T14:47:03.056310+0000","last_active":"2024-07-31T14:47:20.444907+0000","last_peered":"2024-07-31T14:47:20.444907+0000","last_clean":"2024-07-31T14:47:20.444907+0000","last_became_active":"2024-07-31T14:47:03.055695+0000","last_became_peered":"2024-07-31T14:47:03.055695+0000","last_unstale":"2024-07-31T14:47:20.444907+0000","last_undegraded":"2024-07-31T14:47:20.444907+0000","last_fullsized":"2024-07-31T14:47:20.444907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:00:55.124981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:39.748565+0000","last_change":"2024-07-31T14:47:00.236460+0000","last_active":"2024-07-31T14:49:39.748565+0000","last_peered":"2024-07-31T14:49:39.748565+0000","last_clean":"2024-07-31T14:49:39.748565+0000","last_became_active":"2024-07-31T14:47:00.235924+0000","last_became_peered":"2024-07-31T14:47:00.235924+0000","last_unstale":"2024-07-31T14:49:39.748565+0000","last_undegraded":"2024-07-31T14:49:39.748565+0000","last_fullsized":"2024-07-31T14:49:39.748565+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:05.130958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626080+0000","last_change":"2024-07-31T14:47:13.102341+0000","last_active":"2024-07-31T14:47:20.626080+0000","last_peered":"2024-07-31T14:47:20.626080+0000","last_clean":"2024-07-31T14:47:20.626080+0000","last_became_active":"2024-07-31T14:47:13.101798+0000","last_became_peered":"2024-07-31T14:47:13.101798+0000","last_unstale":"2024-07-31T14:47:20.626080+0000","last_undegraded":"2024-07-31T14:47:20.626080+0000","last_fullsized":"2024-07-31T14:47:20.626080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:34:35.758041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444706+0000","last_change":"2024-07-31T14:47:03.054322+0000","last_active":"2024-07-31T14:47:20.444706+0000","last_peered":"2024-07-31T14:47:20.444706+0000","last_clean":"2024-07-31T14:47:20.444706+0000","last_became_active":"2024-07-31T14:47:03.053112+0000","last_became_peered":"2024-07-31T14:47:03.053112+0000","last_unstale":"2024-07-31T14:47:20.444706+0000","last_undegraded":"2024-07-31T14:47:20.444706+0000","last_fullsized":"2024-07-31T14:47:20.444706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:55:13.542578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604832+0000","last_change":"2024-07-31T14:47:00.026518+0000","last_active":"2024-07-31T14:48:03.604832+0000","last_peered":"2024-07-31T14:48:03.604832+0000","last_clean":"2024-07-31T14:48:03.604832+0000","last_became_active":"2024-07-31T14:47:00.026074+0000","last_became_peered":"2024-07-31T14:47:00.026074+0000","last_unstale":"2024-07-31T14:48:03.604832+0000","last_undegraded":"2024-07-31T14:48:03.604832+0000","last_fullsized":"2024-07-31T14:48:03.604832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:33:36.688353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449369+0000","last_change":"2024-07-31T14:47:13.106365+0000","last_active":"2024-07-31T14:47:20.449369+0000","last_peered":"2024-07-31T14:47:20.449369+0000","last_clean":"2024-07-31T14:47:20.449369+0000","last_became_active":"2024-07-31T14:47:13.105635+0000","last_became_peered":"2024-07-31T14:47:13.105635+0000","last_unstale":"2024-07-31T14:47:20.449369+0000","last_undegraded":"2024-07-31T14:47:20.449369+0000","last_fullsized":"2024-07-31T14:47:20.449369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:52:09.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444864+0000","last_change":"2024-07-31T14:47:03.052522+0000","last_active":"2024-07-31T14:47:20.444864+0000","last_peered":"2024-07-31T14:47:20.444864+0000","last_clean":"2024-07-31T14:47:20.444864+0000","last_became_active":"2024-07-31T14:47:03.051872+0000","last_became_peered":"2024-07-31T14:47:03.051872+0000","last_unstale":"2024-07-31T14:47:20.444864+0000","last_undegraded":"2024-07-31T14:47:20.444864+0000","last_fullsized":"2024-07-31T14:47:20.444864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:29:34.956745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600950+0000","last_change":"2024-07-31T14:47:00.016839+0000","last_active":"2024-07-31T14:48:02.600950+0000","last_peered":"2024-07-31T14:48:02.600950+0000","last_clean":"2024-07-31T14:48:02.600950+0000","last_became_active":"2024-07-31T14:47:00.016438+0000","last_became_peered":"2024-07-31T14:47:00.016438+0000","last_unstale":"2024-07-31T14:48:02.600950+0000","last_undegraded":"2024-07-31T14:48:02.600950+0000","last_fullsized":"2024-07-31T14:48:02.600950+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:42:10.875332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626258+0000","last_change":"2024-07-31T14:47:13.105385+0000","last_active":"2024-07-31T14:47:20.626258+0000","last_peered":"2024-07-31T14:47:20.626258+0000","last_clean":"2024-07-31T14:47:20.626258+0000","last_became_active":"2024-07-31T14:47:13.104867+0000","last_became_peered":"2024-07-31T14:47:13.104867+0000","last_unstale":"2024-07-31T14:47:20.626258+0000","last_undegraded":"2024-07-31T14:47:20.626258+0000","last_fullsized":"2024-07-31T14:47:20.626258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:25:21.572830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445679+0000","last_change":"2024-07-31T14:47:03.058304+0000","last_active":"2024-07-31T14:47:20.445679+0000","last_peered":"2024-07-31T14:47:20.445679+0000","last_clean":"2024-07-31T14:47:20.445679+0000","last_became_active":"2024-07-31T14:47:03.057251+0000","last_became_peered":"2024-07-31T14:47:03.057251+0000","last_unstale":"2024-07-31T14:47:20.445679+0000","last_undegraded":"2024-07-31T14:47:20.445679+0000","last_fullsized":"2024-07-31T14:47:20.445679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:58:20.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605301+0000","last_change":"2024-07-31T14:47:00.027806+0000","last_active":"2024-07-31T14:48:03.605301+0000","last_peered":"2024-07-31T14:48:03.605301+0000","last_clean":"2024-07-31T14:48:03.605301+0000","last_became_active":"2024-07-31T14:47:00.027403+0000","last_became_peered":"2024-07-31T14:47:00.027403+0000","last_unstale":"2024-07-31T14:48:03.605301+0000","last_undegraded":"2024-07-31T14:48:03.605301+0000","last_fullsized":"2024-07-31T14:48:03.605301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:57:18.303732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626490+0000","last_change":"2024-07-31T14:47:13.103628+0000","last_active":"2024-07-31T14:47:20.626490+0000","last_peered":"2024-07-31T14:47:20.626490+0000","last_clean":"2024-07-31T14:47:20.626490+0000","last_became_active":"2024-07-31T14:47:13.102296+0000","last_became_peered":"2024-07-31T14:47:13.102296+0000","last_unstale":"2024-07-31T14:47:20.626490+0000","last_undegraded":"2024-07-31T14:47:20.626490+0000","last_fullsized":"2024-07-31T14:47:20.626490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:25:35.109196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626574+0000","last_change":"2024-07-31T14:47:03.052761+0000","last_active":"2024-07-31T14:47:20.626574+0000","last_peered":"2024-07-31T14:47:20.626574+0000","last_clean":"2024-07-31T14:47:20.626574+0000","last_became_active":"2024-07-31T14:47:03.052410+0000","last_became_peered":"2024-07-31T14:47:03.052410+0000","last_unstale":"2024-07-31T14:47:20.626574+0000","last_undegraded":"2024-07-31T14:47:20.626574+0000","last_fullsized":"2024-07-31T14:47:20.626574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:20:47.712491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604937+0000","last_change":"2024-07-31T14:47:00.183741+0000","last_active":"2024-07-31T14:48:03.604937+0000","last_peered":"2024-07-31T14:48:03.604937+0000","last_clean":"2024-07-31T14:48:03.604937+0000","last_became_active":"2024-07-31T14:47:00.183275+0000","last_became_peered":"2024-07-31T14:47:00.183275+0000","last_unstale":"2024-07-31T14:48:03.604937+0000","last_undegraded":"2024-07-31T14:48:03.604937+0000","last_fullsized":"2024-07-31T14:48:03.604937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:29:23.205835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630098+0000","last_change":"2024-07-31T14:47:20.629334+0000","last_active":"2024-07-31T14:47:20.630098+0000","last_peered":"2024-07-31T14:47:20.630098+0000","last_clean":"2024-07-31T14:47:20.630098+0000","last_became_active":"2024-07-31T14:47:20.628865+0000","last_became_peered":"2024-07-31T14:47:20.628865+0000","last_unstale":"2024-07-31T14:47:20.630098+0000","last_undegraded":"2024-07-31T14:47:20.630098+0000","last_fullsized":"2024-07-31T14:47:20.630098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:21:27.199842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444152+0000","last_change":"2024-07-31T14:47:03.054633+0000","last_active":"2024-07-31T14:47:20.444152+0000","last_peered":"2024-07-31T14:47:20.444152+0000","last_clean":"2024-07-31T14:47:20.444152+0000","last_became_active":"2024-07-31T14:47:03.053025+0000","last_became_peered":"2024-07-31T14:47:03.053025+0000","last_unstale":"2024-07-31T14:47:20.444152+0000","last_undegraded":"2024-07-31T14:47:20.444152+0000","last_fullsized":"2024-07-31T14:47:20.444152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:42:03.392744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":152,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:42.752788+0000","last_change":"2024-07-31T14:47:00.027898+0000","last_active":"2024-07-31T14:49:42.752788+0000","last_peered":"2024-07-31T14:49:42.752788+0000","last_clean":"2024-07-31T14:49:42.752788+0000","last_became_active":"2024-07-31T14:47:00.027468+0000","last_became_peered":"2024-07-31T14:47:00.027468+0000","last_unstale":"2024-07-31T14:49:42.752788+0000","last_undegraded":"2024-07-31T14:49:42.752788+0000","last_fullsized":"2024-07-31T14:49:42.752788+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:12:45.922172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444574+0000","last_change":"2024-07-31T14:47:13.104327+0000","last_active":"2024-07-31T14:47:20.444574+0000","last_peered":"2024-07-31T14:47:20.444574+0000","last_clean":"2024-07-31T14:47:20.444574+0000","last_became_active":"2024-07-31T14:47:13.103333+0000","last_became_peered":"2024-07-31T14:47:13.103333+0000","last_unstale":"2024-07-31T14:47:20.444574+0000","last_undegraded":"2024-07-31T14:47:20.444574+0000","last_fullsized":"2024-07-31T14:47:20.444574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:06.226220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444498+0000","last_change":"2024-07-31T14:47:03.044875+0000","last_active":"2024-07-31T14:47:20.444498+0000","last_peered":"2024-07-31T14:47:20.444498+0000","last_clean":"2024-07-31T14:47:20.444498+0000","last_became_active":"2024-07-31T14:47:03.044421+0000","last_became_peered":"2024-07-31T14:47:03.044421+0000","last_unstale":"2024-07-31T14:47:20.444498+0000","last_undegraded":"2024-07-31T14:47:20.444498+0000","last_fullsized":"2024-07-31T14:47:20.444498+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:00:02.477300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605606+0000","last_change":"2024-07-31T14:47:00.019790+0000","last_active":"2024-07-31T14:48:03.605606+0000","last_peered":"2024-07-31T14:48:03.605606+0000","last_clean":"2024-07-31T14:48:03.605606+0000","last_became_active":"2024-07-31T14:47:00.019368+0000","last_became_peered":"2024-07-31T14:47:00.019368+0000","last_unstale":"2024-07-31T14:48:03.605606+0000","last_undegraded":"2024-07-31T14:48:03.605606+0000","last_fullsized":"2024-07-31T14:48:03.605606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:15.299109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445672+0000","last_change":"2024-07-31T14:47:13.102618+0000","last_active":"2024-07-31T14:47:20.445672+0000","last_peered":"2024-07-31T14:47:20.445672+0000","last_clean":"2024-07-31T14:47:20.445672+0000","last_became_active":"2024-07-31T14:47:13.102085+0000","last_became_peered":"2024-07-31T14:47:13.102085+0000","last_unstale":"2024-07-31T14:47:20.445672+0000","last_undegraded":"2024-07-31T14:47:20.445672+0000","last_fullsized":"2024-07-31T14:47:20.445672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:05:58.814215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447389+0000","last_change":"2024-07-31T14:47:03.044688+0000","last_active":"2024-07-31T14:47:20.447389+0000","last_peered":"2024-07-31T14:47:20.447389+0000","last_clean":"2024-07-31T14:47:20.447389+0000","last_became_active":"2024-07-31T14:47:03.043751+0000","last_became_peered":"2024-07-31T14:47:03.043751+0000","last_unstale":"2024-07-31T14:47:20.447389+0000","last_undegraded":"2024-07-31T14:47:20.447389+0000","last_fullsized":"2024-07-31T14:47:20.447389+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:47:31.450796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605830+0000","last_change":"2024-07-31T14:47:00.026259+0000","last_active":"2024-07-31T14:48:03.605830+0000","last_peered":"2024-07-31T14:48:03.605830+0000","last_clean":"2024-07-31T14:48:03.605830+0000","last_became_active":"2024-07-31T14:47:00.025756+0000","last_became_peered":"2024-07-31T14:47:00.025756+0000","last_unstale":"2024-07-31T14:48:03.605830+0000","last_undegraded":"2024-07-31T14:48:03.605830+0000","last_fullsized":"2024-07-31T14:48:03.605830+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:05:00.258841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626888+0000","last_change":"2024-07-31T14:47:13.104918+0000","last_active":"2024-07-31T14:47:20.626888+0000","last_peered":"2024-07-31T14:47:20.626888+0000","last_clean":"2024-07-31T14:47:20.626888+0000","last_became_active":"2024-07-31T14:47:13.104031+0000","last_became_peered":"2024-07-31T14:47:13.104031+0000","last_unstale":"2024-07-31T14:47:20.626888+0000","last_undegraded":"2024-07-31T14:47:20.626888+0000","last_fullsized":"2024-07-31T14:47:20.626888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:17:15.264474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451885+0000","last_change":"2024-07-31T14:47:03.055159+0000","last_active":"2024-07-31T14:47:20.451885+0000","last_peered":"2024-07-31T14:47:20.451885+0000","last_clean":"2024-07-31T14:47:20.451885+0000","last_became_active":"2024-07-31T14:47:03.052389+0000","last_became_peered":"2024-07-31T14:47:03.052389+0000","last_unstale":"2024-07-31T14:47:20.451885+0000","last_undegraded":"2024-07-31T14:47:20.451885+0000","last_fullsized":"2024-07-31T14:47:20.451885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:16.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'35","reported_seq":290,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:44.755220+0000","last_change":"2024-07-31T14:47:00.030165+0000","last_active":"2024-07-31T14:49:44.755220+0000","last_peered":"2024-07-31T14:49:44.755220+0000","last_clean":"2024-07-31T14:49:44.755220+0000","last_became_active":"2024-07-31T14:47:00.029559+0000","last_became_peered":"2024-07-31T14:47:00.029559+0000","last_unstale":"2024-07-31T14:49:44.755220+0000","last_undegraded":"2024-07-31T14:49:44.755220+0000","last_fullsized":"2024-07-31T14:49:44.755220+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:31:06.115075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64136,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":213,"num_read_kb":169,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444234+0000","last_change":"2024-07-31T14:47:13.098517+0000","last_active":"2024-07-31T14:47:20.444234+0000","last_peered":"2024-07-31T14:47:20.444234+0000","last_clean":"2024-07-31T14:47:20.444234+0000","last_became_active":"2024-07-31T14:47:13.098066+0000","last_became_peered":"2024-07-31T14:47:13.098066+0000","last_unstale":"2024-07-31T14:47:20.444234+0000","last_undegraded":"2024-07-31T14:47:20.444234+0000","last_fullsized":"2024-07-31T14:47:20.444234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:08:54.958646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626359+0000","last_change":"2024-07-31T14:47:03.054045+0000","last_active":"2024-07-31T14:47:20.626359+0000","last_peered":"2024-07-31T14:47:20.626359+0000","last_clean":"2024-07-31T14:47:20.626359+0000","last_became_active":"2024-07-31T14:47:03.053658+0000","last_became_peered":"2024-07-31T14:47:03.053658+0000","last_unstale":"2024-07-31T14:47:20.626359+0000","last_undegraded":"2024-07-31T14:47:20.626359+0000","last_fullsized":"2024-07-31T14:47:20.626359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:36:54.563542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.607373+0000","last_change":"2024-07-31T14:47:00.185602+0000","last_active":"2024-07-31T14:48:03.607373+0000","last_peered":"2024-07-31T14:48:03.607373+0000","last_clean":"2024-07-31T14:48:03.607373+0000","last_became_active":"2024-07-31T14:47:00.185230+0000","last_became_peered":"2024-07-31T14:47:00.185230+0000","last_unstale":"2024-07-31T14:48:03.607373+0000","last_undegraded":"2024-07-31T14:48:03.607373+0000","last_fullsized":"2024-07-31T14:48:03.607373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:21:35.979942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604698+0000","last_change":"2024-07-31T14:47:13.099611+0000","last_active":"2024-07-31T14:47:20.604698+0000","last_peered":"2024-07-31T14:47:20.604698+0000","last_clean":"2024-07-31T14:47:20.604698+0000","last_became_active":"2024-07-31T14:47:13.099127+0000","last_became_peered":"2024-07-31T14:47:13.099127+0000","last_unstale":"2024-07-31T14:47:20.604698+0000","last_undegraded":"2024-07-31T14:47:20.604698+0000","last_fullsized":"2024-07-31T14:47:20.604698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:16:57.192980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449785+0000","last_change":"2024-07-31T14:47:03.056818+0000","last_active":"2024-07-31T14:47:20.449785+0000","last_peered":"2024-07-31T14:47:20.449785+0000","last_clean":"2024-07-31T14:47:20.449785+0000","last_became_active":"2024-07-31T14:47:03.055716+0000","last_became_peered":"2024-07-31T14:47:03.055716+0000","last_unstale":"2024-07-31T14:47:20.449785+0000","last_undegraded":"2024-07-31T14:47:20.449785+0000","last_fullsized":"2024-07-31T14:47:20.449785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:26:33.627362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600344+0000","last_change":"2024-07-31T14:47:00.029651+0000","last_active":"2024-07-31T14:48:02.600344+0000","last_peered":"2024-07-31T14:48:02.600344+0000","last_clean":"2024-07-31T14:48:02.600344+0000","last_became_active":"2024-07-31T14:47:00.029242+0000","last_became_peered":"2024-07-31T14:47:00.029242+0000","last_unstale":"2024-07-31T14:48:02.600344+0000","last_undegraded":"2024-07-31T14:48:02.600344+0000","last_fullsized":"2024-07-31T14:48:02.600344+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:23:49.177508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.443852+0000","last_change":"2024-07-31T14:47:13.098862+0000","last_active":"2024-07-31T14:47:20.443852+0000","last_peered":"2024-07-31T14:47:20.443852+0000","last_clean":"2024-07-31T14:47:20.443852+0000","last_became_active":"2024-07-31T14:47:13.098324+0000","last_became_peered":"2024-07-31T14:47:13.098324+0000","last_unstale":"2024-07-31T14:47:20.443852+0000","last_undegraded":"2024-07-31T14:47:20.443852+0000","last_fullsized":"2024-07-31T14:47:20.443852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:50:41.260863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626657+0000","last_change":"2024-07-31T14:47:03.049042+0000","last_active":"2024-07-31T14:47:20.626657+0000","last_peered":"2024-07-31T14:47:20.626657+0000","last_clean":"2024-07-31T14:47:20.626657+0000","last_became_active":"2024-07-31T14:47:03.048535+0000","last_became_peered":"2024-07-31T14:47:03.048535+0000","last_unstale":"2024-07-31T14:47:20.626657+0000","last_undegraded":"2024-07-31T14:47:20.626657+0000","last_fullsized":"2024-07-31T14:47:20.626657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:15.147454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":250,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:44.755287+0000","last_change":"2024-07-31T14:47:00.028940+0000","last_active":"2024-07-31T14:49:44.755287+0000","last_peered":"2024-07-31T14:49:44.755287+0000","last_clean":"2024-07-31T14:49:44.755287+0000","last_became_active":"2024-07-31T14:47:00.028472+0000","last_became_peered":"2024-07-31T14:47:00.028472+0000","last_unstale":"2024-07-31T14:49:44.755287+0000","last_undegraded":"2024-07-31T14:49:44.755287+0000","last_fullsized":"2024-07-31T14:49:44.755287+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:57:15.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626394+0000","last_change":"2024-07-31T14:47:13.101856+0000","last_active":"2024-07-31T14:47:20.626394+0000","last_peered":"2024-07-31T14:47:20.626394+0000","last_clean":"2024-07-31T14:47:20.626394+0000","last_became_active":"2024-07-31T14:47:13.101346+0000","last_became_peered":"2024-07-31T14:47:13.101346+0000","last_unstale":"2024-07-31T14:47:20.626394+0000","last_undegraded":"2024-07-31T14:47:20.626394+0000","last_fullsized":"2024-07-31T14:47:20.626394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:56:37.071660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449190+0000","last_change":"2024-07-31T14:47:13.105952+0000","last_active":"2024-07-31T14:47:20.449190+0000","last_peered":"2024-07-31T14:47:20.449190+0000","last_clean":"2024-07-31T14:47:20.449190+0000","last_became_active":"2024-07-31T14:47:13.105506+0000","last_became_peered":"2024-07-31T14:47:13.105506+0000","last_unstale":"2024-07-31T14:47:20.449190+0000","last_undegraded":"2024-07-31T14:47:20.449190+0000","last_fullsized":"2024-07-31T14:47:20.449190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:28:33.866216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444196+0000","last_change":"2024-07-31T14:47:03.051152+0000","last_active":"2024-07-31T14:47:20.444196+0000","last_peered":"2024-07-31T14:47:20.444196+0000","last_clean":"2024-07-31T14:47:20.444196+0000","last_became_active":"2024-07-31T14:47:03.050775+0000","last_became_peered":"2024-07-31T14:47:03.050775+0000","last_unstale":"2024-07-31T14:47:20.444196+0000","last_undegraded":"2024-07-31T14:47:20.444196+0000","last_fullsized":"2024-07-31T14:47:20.444196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:47:35.913641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609201+0000","last_change":"2024-07-31T14:47:00.028552+0000","last_active":"2024-07-31T14:48:03.609201+0000","last_peered":"2024-07-31T14:48:03.609201+0000","last_clean":"2024-07-31T14:48:03.609201+0000","last_became_active":"2024-07-31T14:47:00.028143+0000","last_became_peered":"2024-07-31T14:47:00.028143+0000","last_unstale":"2024-07-31T14:48:03.609201+0000","last_undegraded":"2024-07-31T14:48:03.609201+0000","last_fullsized":"2024-07-31T14:48:03.609201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:09.601635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.606879+0000","last_change":"2024-07-31T14:47:20.606285+0000","last_active":"2024-07-31T14:47:20.606879+0000","last_peered":"2024-07-31T14:47:20.606879+0000","last_clean":"2024-07-31T14:47:20.606879+0000","last_became_active":"2024-07-31T14:47:20.605837+0000","last_became_peered":"2024-07-31T14:47:20.605837+0000","last_unstale":"2024-07-31T14:47:20.606879+0000","last_undegraded":"2024-07-31T14:47:20.606879+0000","last_fullsized":"2024-07-31T14:47:20.606879+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:16:35.409790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626001+0000","last_change":"2024-07-31T14:47:03.047442+0000","last_active":"2024-07-31T14:47:20.626001+0000","last_peered":"2024-07-31T14:47:20.626001+0000","last_clean":"2024-07-31T14:47:20.626001+0000","last_became_active":"2024-07-31T14:47:03.047072+0000","last_became_peered":"2024-07-31T14:47:03.047072+0000","last_unstale":"2024-07-31T14:47:20.626001+0000","last_undegraded":"2024-07-31T14:47:20.626001+0000","last_fullsized":"2024-07-31T14:47:20.626001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:56.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.599595+0000","last_change":"2024-07-31T14:47:00.027746+0000","last_active":"2024-07-31T14:48:02.599595+0000","last_peered":"2024-07-31T14:48:02.599595+0000","last_clean":"2024-07-31T14:48:02.599595+0000","last_became_active":"2024-07-31T14:47:00.027404+0000","last_became_peered":"2024-07-31T14:47:00.027404+0000","last_unstale":"2024-07-31T14:48:02.599595+0000","last_undegraded":"2024-07-31T14:48:02.599595+0000","last_fullsized":"2024-07-31T14:48:02.599595+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:30:21.011674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.589640+0000","last_change":"2024-07-31T14:47:00.029148+0000","last_active":"2024-07-31T14:48:01.589640+0000","last_peered":"2024-07-31T14:48:01.589640+0000","last_clean":"2024-07-31T14:48:01.589640+0000","last_became_active":"2024-07-31T14:47:00.028718+0000","last_became_peered":"2024-07-31T14:47:00.028718+0000","last_unstale":"2024-07-31T14:48:01.589640+0000","last_undegraded":"2024-07-31T14:48:01.589640+0000","last_fullsized":"2024-07-31T14:48:01.589640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:25:16.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630241+0000","last_change":"2024-07-31T14:47:20.629098+0000","last_active":"2024-07-31T14:47:20.630241+0000","last_peered":"2024-07-31T14:47:20.630241+0000","last_clean":"2024-07-31T14:47:20.630241+0000","last_became_active":"2024-07-31T14:47:20.627775+0000","last_became_peered":"2024-07-31T14:47:20.627775+0000","last_unstale":"2024-07-31T14:47:20.630241+0000","last_undegraded":"2024-07-31T14:47:20.630241+0000","last_fullsized":"2024-07-31T14:47:20.630241+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:22:03.606509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447260+0000","last_change":"2024-07-31T14:47:13.097892+0000","last_active":"2024-07-31T14:47:20.447260+0000","last_peered":"2024-07-31T14:47:20.447260+0000","last_clean":"2024-07-31T14:47:20.447260+0000","last_became_active":"2024-07-31T14:47:13.097419+0000","last_became_peered":"2024-07-31T14:47:13.097419+0000","last_unstale":"2024-07-31T14:47:20.447260+0000","last_undegraded":"2024-07-31T14:47:20.447260+0000","last_fullsized":"2024-07-31T14:47:20.447260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:07:13.456147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:41.751630+0000","last_change":"2024-07-31T14:47:00.019792+0000","last_active":"2024-07-31T14:49:41.751630+0000","last_peered":"2024-07-31T14:49:41.751630+0000","last_clean":"2024-07-31T14:49:41.751630+0000","last_became_active":"2024-07-31T14:47:00.019411+0000","last_became_peered":"2024-07-31T14:47:00.019411+0000","last_unstale":"2024-07-31T14:49:41.751630+0000","last_undegraded":"2024-07-31T14:49:41.751630+0000","last_fullsized":"2024-07-31T14:49:41.751630+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:55:02.148823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.450987+0000","last_change":"2024-07-31T14:47:03.039808+0000","last_active":"2024-07-31T14:47:20.450987+0000","last_peered":"2024-07-31T14:47:20.450987+0000","last_clean":"2024-07-31T14:47:20.450987+0000","last_became_active":"2024-07-31T14:47:03.039382+0000","last_became_peered":"2024-07-31T14:47:03.039382+0000","last_unstale":"2024-07-31T14:47:20.450987+0000","last_undegraded":"2024-07-31T14:47:20.450987+0000","last_fullsized":"2024-07-31T14:47:20.450987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:23:09.751293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.446875+0000","last_change":"2024-07-31T14:47:13.102797+0000","last_active":"2024-07-31T14:47:20.446875+0000","last_peered":"2024-07-31T14:47:20.446875+0000","last_clean":"2024-07-31T14:47:20.446875+0000","last_became_active":"2024-07-31T14:47:13.102320+0000","last_became_peered":"2024-07-31T14:47:13.102320+0000","last_unstale":"2024-07-31T14:47:20.446875+0000","last_undegraded":"2024-07-31T14:47:20.446875+0000","last_fullsized":"2024-07-31T14:47:20.446875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:23.066357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444764+0000","last_change":"2024-07-31T14:47:03.054875+0000","last_active":"2024-07-31T14:47:20.444764+0000","last_peered":"2024-07-31T14:47:20.444764+0000","last_clean":"2024-07-31T14:47:20.444764+0000","last_became_active":"2024-07-31T14:47:03.054529+0000","last_became_peered":"2024-07-31T14:47:03.054529+0000","last_unstale":"2024-07-31T14:47:20.444764+0000","last_undegraded":"2024-07-31T14:47:20.444764+0000","last_fullsized":"2024-07-31T14:47:20.444764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:32:09.802933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.448676+0000","last_change":"2024-07-31T14:47:03.043283+0000","last_active":"2024-07-31T14:47:20.448676+0000","last_peered":"2024-07-31T14:47:20.448676+0000","last_clean":"2024-07-31T14:47:20.448676+0000","last_became_active":"2024-07-31T14:47:03.042851+0000","last_became_peered":"2024-07-31T14:47:03.042851+0000","last_unstale":"2024-07-31T14:47:20.448676+0000","last_undegraded":"2024-07-31T14:47:20.448676+0000","last_fullsized":"2024-07-31T14:47:20.448676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:25:37.408149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447366+0000","last_change":"2024-07-31T14:47:03.047890+0000","last_active":"2024-07-31T14:47:20.447366+0000","last_peered":"2024-07-31T14:47:20.447366+0000","last_clean":"2024-07-31T14:47:20.447366+0000","last_became_active":"2024-07-31T14:47:03.047507+0000","last_became_peered":"2024-07-31T14:47:03.047507+0000","last_unstale":"2024-07-31T14:47:20.447366+0000","last_undegraded":"2024-07-31T14:47:20.447366+0000","last_fullsized":"2024-07-31T14:47:20.447366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:52:45.991323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449508+0000","last_change":"2024-07-31T14:47:03.047903+0000","last_active":"2024-07-31T14:47:20.449508+0000","last_peered":"2024-07-31T14:47:20.449508+0000","last_clean":"2024-07-31T14:47:20.449508+0000","last_became_active":"2024-07-31T14:47:03.047352+0000","last_became_peered":"2024-07-31T14:47:03.047352+0000","last_unstale":"2024-07-31T14:47:20.449508+0000","last_undegraded":"2024-07-31T14:47:20.449508+0000","last_fullsized":"2024-07-31T14:47:20.449508+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:06.536023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447404+0000","last_change":"2024-07-31T14:47:03.044199+0000","last_active":"2024-07-31T14:47:20.447404+0000","last_peered":"2024-07-31T14:47:20.447404+0000","last_clean":"2024-07-31T14:47:20.447404+0000","last_became_active":"2024-07-31T14:47:03.043849+0000","last_became_peered":"2024-07-31T14:47:03.043849+0000","last_unstale":"2024-07-31T14:47:20.447404+0000","last_undegraded":"2024-07-31T14:47:20.447404+0000","last_fullsized":"2024-07-31T14:47:20.447404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:50.474895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445962+0000","last_change":"2024-07-31T14:47:03.060654+0000","last_active":"2024-07-31T14:47:20.445962+0000","last_peered":"2024-07-31T14:47:20.445962+0000","last_clean":"2024-07-31T14:47:20.445962+0000","last_became_active":"2024-07-31T14:47:03.060157+0000","last_became_peered":"2024-07-31T14:47:03.060157+0000","last_unstale":"2024-07-31T14:47:20.445962+0000","last_undegraded":"2024-07-31T14:47:20.445962+0000","last_fullsized":"2024-07-31T14:47:20.445962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:39:50.130831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447079+0000","last_change":"2024-07-31T14:47:03.049910+0000","last_active":"2024-07-31T14:47:20.447079+0000","last_peered":"2024-07-31T14:47:20.447079+0000","last_clean":"2024-07-31T14:47:20.447079+0000","last_became_active":"2024-07-31T14:47:03.049491+0000","last_became_peered":"2024-07-31T14:47:03.049491+0000","last_unstale":"2024-07-31T14:47:20.447079+0000","last_undegraded":"2024-07-31T14:47:20.447079+0000","last_fullsized":"2024-07-31T14:47:20.447079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:18:43.319695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627314+0000","last_change":"2024-07-31T14:47:03.052206+0000","last_active":"2024-07-31T14:47:20.627314+0000","last_peered":"2024-07-31T14:47:20.627314+0000","last_clean":"2024-07-31T14:47:20.627314+0000","last_became_active":"2024-07-31T14:47:03.051175+0000","last_became_peered":"2024-07-31T14:47:03.051175+0000","last_unstale":"2024-07-31T14:47:20.627314+0000","last_undegraded":"2024-07-31T14:47:20.627314+0000","last_fullsized":"2024-07-31T14:47:20.627314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:51:11.583604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626542+0000","last_change":"2024-07-31T14:47:03.052162+0000","last_active":"2024-07-31T14:47:20.626542+0000","last_peered":"2024-07-31T14:47:20.626542+0000","last_clean":"2024-07-31T14:47:20.626542+0000","last_became_active":"2024-07-31T14:47:03.051634+0000","last_became_peered":"2024-07-31T14:47:03.051634+0000","last_unstale":"2024-07-31T14:47:20.626542+0000","last_undegraded":"2024-07-31T14:47:20.626542+0000","last_fullsized":"2024-07-31T14:47:20.626542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:41:59.605192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626490+0000","last_change":"2024-07-31T14:47:03.047475+0000","last_active":"2024-07-31T14:47:20.626490+0000","last_peered":"2024-07-31T14:47:20.626490+0000","last_clean":"2024-07-31T14:47:20.626490+0000","last_became_active":"2024-07-31T14:47:03.047100+0000","last_became_peered":"2024-07-31T14:47:03.047100+0000","last_unstale":"2024-07-31T14:47:20.626490+0000","last_undegraded":"2024-07-31T14:47:20.626490+0000","last_fullsized":"2024-07-31T14:47:20.626490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:08:10.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626514+0000","last_change":"2024-07-31T14:47:03.051281+0000","last_active":"2024-07-31T14:47:20.626514+0000","last_peered":"2024-07-31T14:47:20.626514+0000","last_clean":"2024-07-31T14:47:20.626514+0000","last_became_active":"2024-07-31T14:47:03.050921+0000","last_became_peered":"2024-07-31T14:47:03.050921+0000","last_unstale":"2024-07-31T14:47:20.626514+0000","last_undegraded":"2024-07-31T14:47:20.626514+0000","last_fullsized":"2024-07-31T14:47:20.626514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:24:29.221723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447441+0000","last_change":"2024-07-31T14:47:03.045457+0000","last_active":"2024-07-31T14:47:20.447441+0000","last_peered":"2024-07-31T14:47:20.447441+0000","last_clean":"2024-07-31T14:47:20.447441+0000","last_became_active":"2024-07-31T14:47:03.044466+0000","last_became_peered":"2024-07-31T14:47:03.044466+0000","last_unstale":"2024-07-31T14:47:20.447441+0000","last_undegraded":"2024-07-31T14:47:20.447441+0000","last_fullsized":"2024-07-31T14:47:20.447441+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:44:01.197416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447079+0000","last_change":"2024-07-31T14:47:03.053431+0000","last_active":"2024-07-31T14:47:20.447079+0000","last_peered":"2024-07-31T14:47:20.447079+0000","last_clean":"2024-07-31T14:47:20.447079+0000","last_became_active":"2024-07-31T14:47:03.052958+0000","last_became_peered":"2024-07-31T14:47:03.052958+0000","last_unstale":"2024-07-31T14:47:20.447079+0000","last_undegraded":"2024-07-31T14:47:20.447079+0000","last_fullsized":"2024-07-31T14:47:20.447079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:03:28.863589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447160+0000","last_change":"2024-07-31T14:47:03.046641+0000","last_active":"2024-07-31T14:47:20.447160+0000","last_peered":"2024-07-31T14:47:20.447160+0000","last_clean":"2024-07-31T14:47:20.447160+0000","last_became_active":"2024-07-31T14:47:03.046265+0000","last_became_peered":"2024-07-31T14:47:03.046265+0000","last_unstale":"2024-07-31T14:47:20.447160+0000","last_undegraded":"2024-07-31T14:47:20.447160+0000","last_fullsized":"2024-07-31T14:47:20.447160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:20:25.389010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444454+0000","last_change":"2024-07-31T14:47:03.051657+0000","last_active":"2024-07-31T14:47:20.444454+0000","last_peered":"2024-07-31T14:47:20.444454+0000","last_clean":"2024-07-31T14:47:20.444454+0000","last_became_active":"2024-07-31T14:47:03.050722+0000","last_became_peered":"2024-07-31T14:47:03.050722+0000","last_unstale":"2024-07-31T14:47:20.444454+0000","last_undegraded":"2024-07-31T14:47:20.444454+0000","last_fullsized":"2024-07-31T14:47:20.444454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:23:19.387470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447301+0000","last_change":"2024-07-31T14:47:03.059654+0000","last_active":"2024-07-31T14:47:20.447301+0000","last_peered":"2024-07-31T14:47:20.447301+0000","last_clean":"2024-07-31T14:47:20.447301+0000","last_became_active":"2024-07-31T14:47:03.056843+0000","last_became_peered":"2024-07-31T14:47:03.056843+0000","last_unstale":"2024-07-31T14:47:20.447301+0000","last_undegraded":"2024-07-31T14:47:20.447301+0000","last_fullsized":"2024-07-31T14:47:20.447301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:42:33.696498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625816+0000","last_change":"2024-07-31T14:47:03.052750+0000","last_active":"2024-07-31T14:47:20.625816+0000","last_peered":"2024-07-31T14:47:20.625816+0000","last_clean":"2024-07-31T14:47:20.625816+0000","last_became_active":"2024-07-31T14:47:03.051744+0000","last_became_peered":"2024-07-31T14:47:03.051744+0000","last_unstale":"2024-07-31T14:47:20.625816+0000","last_undegraded":"2024-07-31T14:47:20.625816+0000","last_fullsized":"2024-07-31T14:47:20.625816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:24:33.202918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.453019+0000","last_change":"2024-07-31T14:47:20.452563+0000","last_active":"2024-07-31T14:47:20.453019+0000","last_peered":"2024-07-31T14:47:20.453019+0000","last_clean":"2024-07-31T14:47:20.453019+0000","last_became_active":"2024-07-31T14:47:20.452271+0000","last_became_peered":"2024-07-31T14:47:20.452271+0000","last_unstale":"2024-07-31T14:47:20.453019+0000","last_undegraded":"2024-07-31T14:47:20.453019+0000","last_fullsized":"2024-07-31T14:47:20.453019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:26:55.070265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449004+0000","last_change":"2024-07-31T14:47:03.050691+0000","last_active":"2024-07-31T14:47:20.449004+0000","last_peered":"2024-07-31T14:47:20.449004+0000","last_clean":"2024-07-31T14:47:20.449004+0000","last_became_active":"2024-07-31T14:47:03.050320+0000","last_became_peered":"2024-07-31T14:47:03.050320+0000","last_unstale":"2024-07-31T14:47:20.449004+0000","last_undegraded":"2024-07-31T14:47:20.449004+0000","last_fullsized":"2024-07-31T14:47:20.449004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:40:45.767706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447607+0000","last_change":"2024-07-31T14:47:03.048491+0000","last_active":"2024-07-31T14:47:20.447607+0000","last_peered":"2024-07-31T14:47:20.447607+0000","last_clean":"2024-07-31T14:47:20.447607+0000","last_became_active":"2024-07-31T14:47:03.048085+0000","last_became_peered":"2024-07-31T14:47:03.048085+0000","last_unstale":"2024-07-31T14:47:20.447607+0000","last_undegraded":"2024-07-31T14:47:20.447607+0000","last_fullsized":"2024-07-31T14:47:20.447607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:35:01.762365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604882+0000","last_change":"2024-07-31T14:47:03.055502+0000","last_active":"2024-07-31T14:47:20.604882+0000","last_peered":"2024-07-31T14:47:20.604882+0000","last_clean":"2024-07-31T14:47:20.604882+0000","last_became_active":"2024-07-31T14:47:03.055015+0000","last_became_peered":"2024-07-31T14:47:03.055015+0000","last_unstale":"2024-07-31T14:47:20.604882+0000","last_undegraded":"2024-07-31T14:47:20.604882+0000","last_fullsized":"2024-07-31T14:47:20.604882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:50:39.605148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444836+0000","last_change":"2024-07-31T14:47:03.057808+0000","last_active":"2024-07-31T14:47:20.444836+0000","last_peered":"2024-07-31T14:47:20.444836+0000","last_clean":"2024-07-31T14:47:20.444836+0000","last_became_active":"2024-07-31T14:47:03.057443+0000","last_became_peered":"2024-07-31T14:47:03.057443+0000","last_unstale":"2024-07-31T14:47:20.444836+0000","last_undegraded":"2024-07-31T14:47:20.444836+0000","last_fullsized":"2024-07-31T14:47:20.444836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:34:42.915428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.628089+0000","last_change":"2024-07-31T14:47:03.058182+0000","last_active":"2024-07-31T14:47:20.628089+0000","last_peered":"2024-07-31T14:47:20.628089+0000","last_clean":"2024-07-31T14:47:20.628089+0000","last_became_active":"2024-07-31T14:47:03.057802+0000","last_became_peered":"2024-07-31T14:47:03.057802+0000","last_unstale":"2024-07-31T14:47:20.628089+0000","last_undegraded":"2024-07-31T14:47:20.628089+0000","last_fullsized":"2024-07-31T14:47:20.628089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:08:06.608788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626457+0000","last_change":"2024-07-31T14:47:03.058882+0000","last_active":"2024-07-31T14:47:20.626457+0000","last_peered":"2024-07-31T14:47:20.626457+0000","last_clean":"2024-07-31T14:47:20.626457+0000","last_became_active":"2024-07-31T14:47:03.058530+0000","last_became_peered":"2024-07-31T14:47:03.058530+0000","last_unstale":"2024-07-31T14:47:20.626457+0000","last_undegraded":"2024-07-31T14:47:20.626457+0000","last_fullsized":"2024-07-31T14:47:20.626457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:28:03.631962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626017+0000","last_change":"2024-07-31T14:47:03.052028+0000","last_active":"2024-07-31T14:47:20.626017+0000","last_peered":"2024-07-31T14:47:20.626017+0000","last_clean":"2024-07-31T14:47:20.626017+0000","last_became_active":"2024-07-31T14:47:03.051559+0000","last_became_peered":"2024-07-31T14:47:03.051559+0000","last_unstale":"2024-07-31T14:47:20.626017+0000","last_undegraded":"2024-07-31T14:47:20.626017+0000","last_fullsized":"2024-07-31T14:47:20.626017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:16:40.723795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444253+0000","last_change":"2024-07-31T14:47:03.053884+0000","last_active":"2024-07-31T14:47:20.444253+0000","last_peered":"2024-07-31T14:47:20.444253+0000","last_clean":"2024-07-31T14:47:20.444253+0000","last_became_active":"2024-07-31T14:47:03.053521+0000","last_became_peered":"2024-07-31T14:47:03.053521+0000","last_unstale":"2024-07-31T14:47:20.444253+0000","last_undegraded":"2024-07-31T14:47:20.444253+0000","last_fullsized":"2024-07-31T14:47:20.444253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:16:37.702055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627188+0000","last_change":"2024-07-31T14:47:03.058588+0000","last_active":"2024-07-31T14:47:20.627188+0000","last_peered":"2024-07-31T14:47:20.627188+0000","last_clean":"2024-07-31T14:47:20.627188+0000","last_became_active":"2024-07-31T14:47:03.057820+0000","last_became_peered":"2024-07-31T14:47:03.057820+0000","last_unstale":"2024-07-31T14:47:20.627188+0000","last_undegraded":"2024-07-31T14:47:20.627188+0000","last_fullsized":"2024-07-31T14:47:20.627188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:21.265208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451246+0000","last_change":"2024-07-31T14:47:03.041447+0000","last_active":"2024-07-31T14:47:20.451246+0000","last_peered":"2024-07-31T14:47:20.451246+0000","last_clean":"2024-07-31T14:47:20.451246+0000","last_became_active":"2024-07-31T14:47:03.040935+0000","last_became_peered":"2024-07-31T14:47:03.040935+0000","last_unstale":"2024-07-31T14:47:20.451246+0000","last_undegraded":"2024-07-31T14:47:20.451246+0000","last_fullsized":"2024-07-31T14:47:20.451246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:00:13.624698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:21.483666+0000","last_change":"2024-07-31T14:47:20.452239+0000","last_active":"2024-07-31T14:47:21.483666+0000","last_peered":"2024-07-31T14:47:21.483666+0000","last_clean":"2024-07-31T14:47:21.483666+0000","last_became_active":"2024-07-31T14:47:20.451924+0000","last_became_peered":"2024-07-31T14:47:20.451924+0000","last_unstale":"2024-07-31T14:47:21.483666+0000","last_undegraded":"2024-07-31T14:47:21.483666+0000","last_fullsized":"2024-07-31T14:47:21.483666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:58:24.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449176+0000","last_change":"2024-07-31T14:47:03.056024+0000","last_active":"2024-07-31T14:47:20.449176+0000","last_peered":"2024-07-31T14:47:20.449176+0000","last_clean":"2024-07-31T14:47:20.449176+0000","last_became_active":"2024-07-31T14:47:03.055630+0000","last_became_peered":"2024-07-31T14:47:03.055630+0000","last_unstale":"2024-07-31T14:47:20.449176+0000","last_undegraded":"2024-07-31T14:47:20.449176+0000","last_fullsized":"2024-07-31T14:47:20.449176+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:09:11.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449155+0000","last_change":"2024-07-31T14:47:03.053718+0000","last_active":"2024-07-31T14:47:20.449155+0000","last_peered":"2024-07-31T14:47:20.449155+0000","last_clean":"2024-07-31T14:47:20.449155+0000","last_became_active":"2024-07-31T14:47:03.053361+0000","last_became_peered":"2024-07-31T14:47:03.053361+0000","last_unstale":"2024-07-31T14:47:20.449155+0000","last_undegraded":"2024-07-31T14:47:20.449155+0000","last_fullsized":"2024-07-31T14:47:20.449155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:03:49.220838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447206+0000","last_change":"2024-07-31T14:47:03.056087+0000","last_active":"2024-07-31T14:47:20.447206+0000","last_peered":"2024-07-31T14:47:20.447206+0000","last_clean":"2024-07-31T14:47:20.447206+0000","last_became_active":"2024-07-31T14:47:03.055732+0000","last_became_peered":"2024-07-31T14:47:03.055732+0000","last_unstale":"2024-07-31T14:47:20.447206+0000","last_undegraded":"2024-07-31T14:47:20.447206+0000","last_fullsized":"2024-07-31T14:47:20.447206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:45:01.185666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71731,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1045,"num_read_kb":604,"num_write":249,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":215193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645359,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":519730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841588,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37016,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708136,"statfs":{"total":95995035648,"available":95957131264,"internally_reserved":0,"allocated":1720320,"data_stored":1171073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005113,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":523756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201342,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":520038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397570,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1117317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561096,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":515555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1112117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":519629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117078,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":514872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378715,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":580941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542240,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":580721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705764,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":516620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-31T14:49:47.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph pg dump --format=json 2024-07-31T14:49:47.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:46 smithi073 ceph-mon[36714]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:49:47.538 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:48.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:47 smithi077 ceph-mon[37327]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:49:48.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:47 smithi067 ceph-mon[32750]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:49:48.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:47 smithi073 ceph-mon[36714]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:49:49.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:48 smithi067 ceph-mon[32750]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:49.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:48 smithi077 ceph-mon[37327]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:49.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.275+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2922938294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bdc103590 msgr2=0x7f2bdc1039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:49.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.275+0000 7f2be1b5f640 1 --2- 172.21.15.67:0/2922938294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bdc103590 0x7f2bdc1039a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009900 tx=0x7f2bcc02a410 comp rx=0 tx=0).stop 2024-07-31T14:49:49.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.275+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2922938294 shutdown_connections 2024-07-31T14:49:49.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.275+0000 7f2be1b5f640 1 --2- 172.21.15.67:0/2922938294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bdc109820 0x7f2bdc110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:49.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.275+0000 7f2be1b5f640 1 --2- 172.21.15.67:0/2922938294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bdc108ed0 0x7f2bdc1092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:49.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.275+0000 7f2be1b5f640 1 --2- 172.21.15.67:0/2922938294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bdc103590 0x7f2bdc1039a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:49.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.275+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2922938294 >> 172.21.15.67:0/2922938294 conn(0x7f2bdc07b280 msgr2=0x7f2bdc07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:49.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.275+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2922938294 shutdown_connections 2024-07-31T14:49:49.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.275+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2922938294 wait complete. 2024-07-31T14:49:49.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.275+0000 7f2be1b5f640 1 Processor -- start 2024-07-31T14:49:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2be1b5f640 1 -- start start 2024-07-31T14:49:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2be1b5f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bdc103590 0x7f2bdc199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2be1b5f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bdc108ed0 0x7f2bdc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2be1b5f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bdc109820 0x7f2bdc19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2be1b5f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1134f0 con 0x7f2bdc108ed0 2024-07-31T14:49:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2be1b5f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc113370 con 0x7f2bdc103590 2024-07-31T14:49:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2be1b5f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc113670 con 0x7f2bdc109820 2024-07-31T14:49:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2bdaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bdc108ed0 0x7f2bdc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2bdaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bdc108ed0 0x7f2bdc19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45134/0 (socket says 172.21.15.67:45134) 2024-07-31T14:49:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2bdaffd640 1 -- 172.21.15.67:0/2372611891 learned_addr learned my addr 172.21.15.67:0/2372611891 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2bdbfff640 1 --2- 172.21.15.67:0/2372611891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bdc109820 0x7f2bdc19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2bdb7fe640 1 --2- 172.21.15.67:0/2372611891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bdc103590 0x7f2bdc199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2bdaffd640 1 -- 172.21.15.67:0/2372611891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bdc109820 msgr2=0x7f2bdc19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2bdaffd640 1 --2- 172.21.15.67:0/2372611891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bdc109820 0x7f2bdc19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2bdaffd640 1 -- 172.21.15.67:0/2372611891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bdc103590 msgr2=0x7f2bdc199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2bdaffd640 1 --2- 172.21.15.67:0/2372611891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bdc103590 0x7f2bdc199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.276+0000 7f2bdaffd640 1 -- 172.21.15.67:0/2372611891 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc19feb0 con 0x7f2bdc108ed0 2024-07-31T14:49:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.277+0000 7f2bdaffd640 1 --2- 172.21.15.67:0/2372611891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bdc108ed0 0x7f2bdc19a4e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000d740 tx=0x7f2bd000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.277+0000 7f2bd8ff9640 1 -- 172.21.15.67:0/2372611891 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0017070 con 0x7f2bdc108ed0 2024-07-31T14:49:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.277+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2372611891 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc1a01a0 con 0x7f2bdc108ed0 2024-07-31T14:49:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.277+0000 7f2bd8ff9640 1 -- 172.21.15.67:0/2372611891 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bd000de50 con 0x7f2bdc108ed0 2024-07-31T14:49:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.277+0000 7f2bd8ff9640 1 -- 172.21.15.67:0/2372611891 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd000f620 con 0x7f2bdc108ed0 2024-07-31T14:49:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.277+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2372611891 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc1a6900 con 0x7f2bdc108ed0 2024-07-31T14:49:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.278+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2372611891 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bdc1039a0 con 0x7f2bdc108ed0 2024-07-31T14:49:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.280+0000 7f2bd8ff9640 1 -- 172.21.15.67:0/2372611891 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f2bd001a070 con 0x7f2bdc108ed0 2024-07-31T14:49:49.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.280+0000 7f2bd8ff9640 1 --2- 172.21.15.67:0/2372611891 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2bc40b3cb0 0x7f2bc40b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:49.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.280+0000 7f2bdb7fe640 1 --2- 172.21.15.67:0/2372611891 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2bc40b3cb0 0x7f2bc40b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:49.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.280+0000 7f2bd8ff9640 1 -- 172.21.15.67:0/2372611891 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f2bd00e3b10 con 0x7f2bdc108ed0 2024-07-31T14:49:49.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.281+0000 7f2bdb7fe640 1 --2- 172.21.15.67:0/2372611891 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2bc40b3cb0 0x7f2bc40b61c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc031440 tx=0x7f2bcc009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:49.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.283+0000 7f2bd8ff9640 1 -- 172.21.15.67:0/2372611891 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f2bd00aa580 con 0x7f2bdc108ed0 2024-07-31T14:49:49.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:48 smithi073 ceph-mon[36714]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:49:49.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.505+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2372611891 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2bdc063230 con 0x7f2bc40b3cb0 2024-07-31T14:49:49.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.511+0000 7f2bd8ff9640 1 -- 172.21.15.67:0/2372611891 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340445 (secure 0 0 0) 0x7f2bdc063230 con 0x7f2bc40b3cb0 2024-07-31T14:49:49.515 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:49:49.516 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-31T14:49:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.514+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2372611891 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2bc40b3cb0 msgr2=0x7f2bc40b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.514+0000 7f2be1b5f640 1 --2- 172.21.15.67:0/2372611891 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2bc40b3cb0 0x7f2bc40b61c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc031440 tx=0x7f2bcc009560 comp rx=0 tx=0).stop 2024-07-31T14:49:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.514+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2372611891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bdc108ed0 msgr2=0x7f2bdc19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.514+0000 7f2be1b5f640 1 --2- 172.21.15.67:0/2372611891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bdc108ed0 0x7f2bdc19a4e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000d740 tx=0x7f2bd000dc00 comp rx=0 tx=0).stop 2024-07-31T14:49:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.514+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2372611891 shutdown_connections 2024-07-31T14:49:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.514+0000 7f2be1b5f640 1 --2- 172.21.15.67:0/2372611891 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2bc40b3cb0 0x7f2bc40b61c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.514+0000 7f2be1b5f640 1 --2- 172.21.15.67:0/2372611891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bdc109820 0x7f2bdc19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.514+0000 7f2be1b5f640 1 --2- 172.21.15.67:0/2372611891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bdc108ed0 0x7f2bdc19a4e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.514+0000 7f2be1b5f640 1 --2- 172.21.15.67:0/2372611891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bdc103590 0x7f2bdc199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.514+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2372611891 >> 172.21.15.67:0/2372611891 conn(0x7f2bdc07b280 msgr2=0x7f2bdc107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.514+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2372611891 shutdown_connections 2024-07-31T14:49:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:49.514+0000 7f2be1b5f640 1 -- 172.21.15.67:0/2372611891 wait complete. 2024-07-31T14:49:50.293 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-07-31T14:49:49.418838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662474,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1156,"num_read_kb":683,"num_write":375,"num_write_kb":2297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":316,"ondisk_log_size":316,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379936,"kb_used_data":14144,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561888,"statfs":{"total":1151940427776,"available":1151551373312,"internally_reserved":0,"allocated":14483456,"data_stored":8192369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003721"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:44.772649+0000","last_change":"2024-07-31T14:47:00.020067+0000","last_active":"2024-07-31T14:49:44.772649+0000","last_peered":"2024-07-31T14:49:44.772649+0000","last_clean":"2024-07-31T14:49:44.772649+0000","last_became_active":"2024-07-31T14:47:00.019674+0000","last_became_peered":"2024-07-31T14:47:00.019674+0000","last_unstale":"2024-07-31T14:49:44.772649+0000","last_undegraded":"2024-07-31T14:49:44.772649+0000","last_fullsized":"2024-07-31T14:49:44.772649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:10:08.193786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626715+0000","last_change":"2024-07-31T14:47:03.052069+0000","last_active":"2024-07-31T14:47:20.626715+0000","last_peered":"2024-07-31T14:47:20.626715+0000","last_clean":"2024-07-31T14:47:20.626715+0000","last_became_active":"2024-07-31T14:47:03.051739+0000","last_became_peered":"2024-07-31T14:47:03.051739+0000","last_unstale":"2024-07-31T14:47:20.626715+0000","last_undegraded":"2024-07-31T14:47:20.626715+0000","last_fullsized":"2024-07-31T14:47:20.626715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:29:19.952794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.629384+0000","last_change":"2024-07-31T14:47:20.628770+0000","last_active":"2024-07-31T14:47:20.629384+0000","last_peered":"2024-07-31T14:47:20.629384+0000","last_clean":"2024-07-31T14:47:20.629384+0000","last_became_active":"2024-07-31T14:47:20.628142+0000","last_became_peered":"2024-07-31T14:47:20.628142+0000","last_unstale":"2024-07-31T14:47:20.629384+0000","last_undegraded":"2024-07-31T14:47:20.629384+0000","last_fullsized":"2024-07-31T14:47:20.629384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.659670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:00.571472+0000","last_change":"2024-07-31T14:47:00.183893+0000","last_active":"2024-07-31T14:48:00.571472+0000","last_peered":"2024-07-31T14:48:00.571472+0000","last_clean":"2024-07-31T14:48:00.571472+0000","last_became_active":"2024-07-31T14:47:00.183453+0000","last_became_peered":"2024-07-31T14:47:00.183453+0000","last_unstale":"2024-07-31T14:48:00.571472+0000","last_undegraded":"2024-07-31T14:48:00.571472+0000","last_fullsized":"2024-07-31T14:48:00.571472+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:45:08.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449756+0000","last_change":"2024-07-31T14:47:03.057081+0000","last_active":"2024-07-31T14:47:20.449756+0000","last_peered":"2024-07-31T14:47:20.449756+0000","last_clean":"2024-07-31T14:47:20.449756+0000","last_became_active":"2024-07-31T14:47:03.056061+0000","last_became_peered":"2024-07-31T14:47:03.056061+0000","last_unstale":"2024-07-31T14:47:20.449756+0000","last_undegraded":"2024-07-31T14:47:20.449756+0000","last_fullsized":"2024-07-31T14:47:20.449756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:28:16.760131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604121+0000","last_change":"2024-07-31T14:47:13.105271+0000","last_active":"2024-07-31T14:47:20.604121+0000","last_peered":"2024-07-31T14:47:20.604121+0000","last_clean":"2024-07-31T14:47:20.604121+0000","last_became_active":"2024-07-31T14:47:13.104717+0000","last_became_peered":"2024-07-31T14:47:13.104717+0000","last_unstale":"2024-07-31T14:47:20.604121+0000","last_undegraded":"2024-07-31T14:47:20.604121+0000","last_fullsized":"2024-07-31T14:47:20.604121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:34:21.413957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605719+0000","last_change":"2024-07-31T14:47:20.454745+0000","last_active":"2024-07-31T14:48:03.605719+0000","last_peered":"2024-07-31T14:48:03.605719+0000","last_clean":"2024-07-31T14:48:03.605719+0000","last_became_active":"2024-07-31T14:47:20.449892+0000","last_became_peered":"2024-07-31T14:47:20.449892+0000","last_unstale":"2024-07-31T14:48:03.605719+0000","last_undegraded":"2024-07-31T14:48:03.605719+0000","last_fullsized":"2024-07-31T14:48:03.605719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:59:17.375344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626269+0000","last_change":"2024-07-31T14:47:03.051085+0000","last_active":"2024-07-31T14:47:20.626269+0000","last_peered":"2024-07-31T14:47:20.626269+0000","last_clean":"2024-07-31T14:47:20.626269+0000","last_became_active":"2024-07-31T14:47:03.050686+0000","last_became_peered":"2024-07-31T14:47:03.050686+0000","last_unstale":"2024-07-31T14:47:20.626269+0000","last_undegraded":"2024-07-31T14:47:20.626269+0000","last_fullsized":"2024-07-31T14:47:20.626269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:48:02.494061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604501+0000","last_change":"2024-07-31T14:47:13.104229+0000","last_active":"2024-07-31T14:47:20.604501+0000","last_peered":"2024-07-31T14:47:20.604501+0000","last_clean":"2024-07-31T14:47:20.604501+0000","last_became_active":"2024-07-31T14:47:13.103737+0000","last_became_peered":"2024-07-31T14:47:13.103737+0000","last_unstale":"2024-07-31T14:47:20.604501+0000","last_undegraded":"2024-07-31T14:47:20.604501+0000","last_fullsized":"2024-07-31T14:47:20.604501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:36:00.541345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609858+0000","last_change":"2024-07-31T14:47:00.028169+0000","last_active":"2024-07-31T14:48:03.609858+0000","last_peered":"2024-07-31T14:48:03.609858+0000","last_clean":"2024-07-31T14:48:03.609858+0000","last_became_active":"2024-07-31T14:47:00.027757+0000","last_became_peered":"2024-07-31T14:47:00.027757+0000","last_unstale":"2024-07-31T14:48:03.609858+0000","last_undegraded":"2024-07-31T14:48:03.609858+0000","last_fullsized":"2024-07-31T14:48:03.609858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:36:04.419551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626959+0000","last_change":"2024-07-31T14:47:03.057400+0000","last_active":"2024-07-31T14:47:20.626959+0000","last_peered":"2024-07-31T14:47:20.626959+0000","last_clean":"2024-07-31T14:47:20.626959+0000","last_became_active":"2024-07-31T14:47:03.057035+0000","last_became_peered":"2024-07-31T14:47:03.057035+0000","last_unstale":"2024-07-31T14:47:20.626959+0000","last_undegraded":"2024-07-31T14:47:20.626959+0000","last_fullsized":"2024-07-31T14:47:20.626959+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:55:07.951076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604244+0000","last_change":"2024-07-31T14:47:13.106093+0000","last_active":"2024-07-31T14:47:20.604244+0000","last_peered":"2024-07-31T14:47:20.604244+0000","last_clean":"2024-07-31T14:47:20.604244+0000","last_became_active":"2024-07-31T14:47:13.105459+0000","last_became_peered":"2024-07-31T14:47:13.105459+0000","last_unstale":"2024-07-31T14:47:20.604244+0000","last_undegraded":"2024-07-31T14:47:20.604244+0000","last_fullsized":"2024-07-31T14:47:20.604244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:07:52.751245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444407+0000","last_change":"2024-07-31T14:47:03.058701+0000","last_active":"2024-07-31T14:47:20.444407+0000","last_peered":"2024-07-31T14:47:20.444407+0000","last_clean":"2024-07-31T14:47:20.444407+0000","last_became_active":"2024-07-31T14:47:03.058359+0000","last_became_peered":"2024-07-31T14:47:03.058359+0000","last_unstale":"2024-07-31T14:47:20.444407+0000","last_undegraded":"2024-07-31T14:47:20.444407+0000","last_fullsized":"2024-07-31T14:47:20.444407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:37:36.149884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605126+0000","last_change":"2024-07-31T14:47:00.022983+0000","last_active":"2024-07-31T14:48:03.605126+0000","last_peered":"2024-07-31T14:48:03.605126+0000","last_clean":"2024-07-31T14:48:03.605126+0000","last_became_active":"2024-07-31T14:47:00.022586+0000","last_became_peered":"2024-07-31T14:47:00.022586+0000","last_unstale":"2024-07-31T14:48:03.605126+0000","last_undegraded":"2024-07-31T14:48:03.605126+0000","last_fullsized":"2024-07-31T14:48:03.605126+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:29:42.641490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604474+0000","last_change":"2024-07-31T14:47:13.363148+0000","last_active":"2024-07-31T14:47:20.604474+0000","last_peered":"2024-07-31T14:47:20.604474+0000","last_clean":"2024-07-31T14:47:20.604474+0000","last_became_active":"2024-07-31T14:47:13.362577+0000","last_became_peered":"2024-07-31T14:47:13.362577+0000","last_unstale":"2024-07-31T14:47:20.604474+0000","last_undegraded":"2024-07-31T14:47:20.604474+0000","last_fullsized":"2024-07-31T14:47:20.604474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:37:59.003260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626453+0000","last_change":"2024-07-31T14:47:03.049586+0000","last_active":"2024-07-31T14:47:20.626453+0000","last_peered":"2024-07-31T14:47:20.626453+0000","last_clean":"2024-07-31T14:47:20.626453+0000","last_became_active":"2024-07-31T14:47:03.049154+0000","last_became_peered":"2024-07-31T14:47:03.049154+0000","last_unstale":"2024-07-31T14:47:20.626453+0000","last_undegraded":"2024-07-31T14:47:20.626453+0000","last_fullsized":"2024-07-31T14:47:20.626453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:51:01.971854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.585353+0000","last_change":"2024-07-31T14:47:00.021254+0000","last_active":"2024-07-31T14:48:02.585353+0000","last_peered":"2024-07-31T14:48:02.585353+0000","last_clean":"2024-07-31T14:48:02.585353+0000","last_became_active":"2024-07-31T14:47:00.020690+0000","last_became_peered":"2024-07-31T14:47:00.020690+0000","last_unstale":"2024-07-31T14:48:02.585353+0000","last_undegraded":"2024-07-31T14:48:02.585353+0000","last_fullsized":"2024-07-31T14:48:02.585353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:01:27.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:18.046730+0000","last_change":"2024-07-31T14:47:13.102220+0000","last_active":"2024-07-31T14:48:18.046730+0000","last_peered":"2024-07-31T14:48:18.046730+0000","last_clean":"2024-07-31T14:48:18.046730+0000","last_became_active":"2024-07-31T14:47:13.101764+0000","last_became_peered":"2024-07-31T14:47:13.101764+0000","last_unstale":"2024-07-31T14:48:18.046730+0000","last_undegraded":"2024-07-31T14:48:18.046730+0000","last_fullsized":"2024-07-31T14:48:18.046730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:56:44.335340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444979+0000","last_change":"2024-07-31T14:47:03.052348+0000","last_active":"2024-07-31T14:47:20.444979+0000","last_peered":"2024-07-31T14:47:20.444979+0000","last_clean":"2024-07-31T14:47:20.444979+0000","last_became_active":"2024-07-31T14:47:03.051961+0000","last_became_peered":"2024-07-31T14:47:03.051961+0000","last_unstale":"2024-07-31T14:47:20.444979+0000","last_undegraded":"2024-07-31T14:47:20.444979+0000","last_fullsized":"2024-07-31T14:47:20.444979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:19:35.279809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.597386+0000","last_change":"2024-07-31T14:47:00.025013+0000","last_active":"2024-07-31T14:48:02.597386+0000","last_peered":"2024-07-31T14:48:02.597386+0000","last_clean":"2024-07-31T14:48:02.597386+0000","last_became_active":"2024-07-31T14:47:00.024637+0000","last_became_peered":"2024-07-31T14:47:00.024637+0000","last_unstale":"2024-07-31T14:48:02.597386+0000","last_undegraded":"2024-07-31T14:48:02.597386+0000","last_fullsized":"2024-07-31T14:48:02.597386+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:07:21.519549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630339+0000","last_change":"2024-07-31T14:47:20.629262+0000","last_active":"2024-07-31T14:47:20.630339+0000","last_peered":"2024-07-31T14:47:20.630339+0000","last_clean":"2024-07-31T14:47:20.630339+0000","last_became_active":"2024-07-31T14:47:20.628318+0000","last_became_peered":"2024-07-31T14:47:20.628318+0000","last_unstale":"2024-07-31T14:47:20.630339+0000","last_undegraded":"2024-07-31T14:47:20.630339+0000","last_fullsized":"2024-07-31T14:47:20.630339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.206705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451128+0000","last_change":"2024-07-31T14:47:03.051527+0000","last_active":"2024-07-31T14:47:20.451128+0000","last_peered":"2024-07-31T14:47:20.451128+0000","last_clean":"2024-07-31T14:47:20.451128+0000","last_became_active":"2024-07-31T14:47:03.051139+0000","last_became_peered":"2024-07-31T14:47:03.051139+0000","last_unstale":"2024-07-31T14:47:20.451128+0000","last_undegraded":"2024-07-31T14:47:20.451128+0000","last_fullsized":"2024-07-31T14:47:20.451128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:53:11.962719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:59.566389+0000","last_change":"2024-07-31T14:47:00.183812+0000","last_active":"2024-07-31T14:47:59.566389+0000","last_peered":"2024-07-31T14:47:59.566389+0000","last_clean":"2024-07-31T14:47:59.566389+0000","last_became_active":"2024-07-31T14:47:00.183440+0000","last_became_peered":"2024-07-31T14:47:00.183440+0000","last_unstale":"2024-07-31T14:47:59.566389+0000","last_undegraded":"2024-07-31T14:47:59.566389+0000","last_fullsized":"2024-07-31T14:47:59.566389+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:47:23.166756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604338+0000","last_change":"2024-07-31T14:47:13.363563+0000","last_active":"2024-07-31T14:47:20.604338+0000","last_peered":"2024-07-31T14:47:20.604338+0000","last_clean":"2024-07-31T14:47:20.604338+0000","last_became_active":"2024-07-31T14:47:13.362697+0000","last_became_peered":"2024-07-31T14:47:13.362697+0000","last_unstale":"2024-07-31T14:47:20.604338+0000","last_undegraded":"2024-07-31T14:47:20.604338+0000","last_fullsized":"2024-07-31T14:47:20.604338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:26:38.292073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449305+0000","last_change":"2024-07-31T14:47:03.046763+0000","last_active":"2024-07-31T14:47:20.449305+0000","last_peered":"2024-07-31T14:47:20.449305+0000","last_clean":"2024-07-31T14:47:20.449305+0000","last_became_active":"2024-07-31T14:47:03.046408+0000","last_became_peered":"2024-07-31T14:47:03.046408+0000","last_unstale":"2024-07-31T14:47:20.449305+0000","last_undegraded":"2024-07-31T14:47:20.449305+0000","last_fullsized":"2024-07-31T14:47:20.449305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:43:22.838495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605677+0000","last_change":"2024-07-31T14:47:00.236075+0000","last_active":"2024-07-31T14:48:03.605677+0000","last_peered":"2024-07-31T14:48:03.605677+0000","last_clean":"2024-07-31T14:48:03.605677+0000","last_became_active":"2024-07-31T14:47:00.235758+0000","last_became_peered":"2024-07-31T14:47:00.235758+0000","last_unstale":"2024-07-31T14:48:03.605677+0000","last_undegraded":"2024-07-31T14:48:03.605677+0000","last_fullsized":"2024-07-31T14:48:03.605677+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:50.575648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.448218+0000","last_change":"2024-07-31T14:47:13.101876+0000","last_active":"2024-07-31T14:47:20.448218+0000","last_peered":"2024-07-31T14:47:20.448218+0000","last_clean":"2024-07-31T14:47:20.448218+0000","last_became_active":"2024-07-31T14:47:13.101276+0000","last_became_peered":"2024-07-31T14:47:13.101276+0000","last_unstale":"2024-07-31T14:47:20.448218+0000","last_undegraded":"2024-07-31T14:47:20.448218+0000","last_fullsized":"2024-07-31T14:47:20.448218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:53:24.684212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627393+0000","last_change":"2024-07-31T14:47:03.052478+0000","last_active":"2024-07-31T14:47:20.627393+0000","last_peered":"2024-07-31T14:47:20.627393+0000","last_clean":"2024-07-31T14:47:20.627393+0000","last_became_active":"2024-07-31T14:47:03.051877+0000","last_became_peered":"2024-07-31T14:47:03.051877+0000","last_unstale":"2024-07-31T14:47:20.627393+0000","last_undegraded":"2024-07-31T14:47:20.627393+0000","last_fullsized":"2024-07-31T14:47:20.627393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:30:32.666490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'14","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:27.059299+0000","last_change":"2024-07-31T14:47:00.020076+0000","last_active":"2024-07-31T14:48:27.059299+0000","last_peered":"2024-07-31T14:48:27.059299+0000","last_clean":"2024-07-31T14:48:27.059299+0000","last_became_active":"2024-07-31T14:47:00.019708+0000","last_became_peered":"2024-07-31T14:47:00.019708+0000","last_unstale":"2024-07-31T14:48:27.059299+0000","last_undegraded":"2024-07-31T14:48:27.059299+0000","last_fullsized":"2024-07-31T14:48:27.059299+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:46.520997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625840+0000","last_change":"2024-07-31T14:47:13.362961+0000","last_active":"2024-07-31T14:47:20.625840+0000","last_peered":"2024-07-31T14:47:20.625840+0000","last_clean":"2024-07-31T14:47:20.625840+0000","last_became_active":"2024-07-31T14:47:13.362502+0000","last_became_peered":"2024-07-31T14:47:13.362502+0000","last_unstale":"2024-07-31T14:47:20.625840+0000","last_undegraded":"2024-07-31T14:47:20.625840+0000","last_fullsized":"2024-07-31T14:47:20.625840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:51:45.732477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447518+0000","last_change":"2024-07-31T14:47:03.039418+0000","last_active":"2024-07-31T14:47:20.447518+0000","last_peered":"2024-07-31T14:47:20.447518+0000","last_clean":"2024-07-31T14:47:20.447518+0000","last_became_active":"2024-07-31T14:47:03.039044+0000","last_became_peered":"2024-07-31T14:47:03.039044+0000","last_unstale":"2024-07-31T14:47:20.447518+0000","last_undegraded":"2024-07-31T14:47:20.447518+0000","last_fullsized":"2024-07-31T14:47:20.447518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:40:10.380080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604784+0000","last_change":"2024-07-31T14:47:00.183778+0000","last_active":"2024-07-31T14:48:03.604784+0000","last_peered":"2024-07-31T14:48:03.604784+0000","last_clean":"2024-07-31T14:48:03.604784+0000","last_became_active":"2024-07-31T14:47:00.183370+0000","last_became_peered":"2024-07-31T14:47:00.183370+0000","last_unstale":"2024-07-31T14:48:03.604784+0000","last_undegraded":"2024-07-31T14:48:03.604784+0000","last_fullsized":"2024-07-31T14:48:03.604784+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:45:27.422216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445139+0000","last_change":"2024-07-31T14:47:13.102583+0000","last_active":"2024-07-31T14:47:20.445139+0000","last_peered":"2024-07-31T14:47:20.445139+0000","last_clean":"2024-07-31T14:47:20.445139+0000","last_became_active":"2024-07-31T14:47:13.102058+0000","last_became_peered":"2024-07-31T14:47:13.102058+0000","last_unstale":"2024-07-31T14:47:20.445139+0000","last_undegraded":"2024-07-31T14:47:20.445139+0000","last_fullsized":"2024-07-31T14:47:20.445139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:17.396173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626607+0000","last_change":"2024-07-31T14:47:03.048334+0000","last_active":"2024-07-31T14:47:20.626607+0000","last_peered":"2024-07-31T14:47:20.626607+0000","last_clean":"2024-07-31T14:47:20.626607+0000","last_became_active":"2024-07-31T14:47:03.047932+0000","last_became_peered":"2024-07-31T14:47:03.047932+0000","last_unstale":"2024-07-31T14:47:20.626607+0000","last_undegraded":"2024-07-31T14:47:20.626607+0000","last_fullsized":"2024-07-31T14:47:20.626607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:14:31.381061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.591990+0000","last_change":"2024-07-31T14:47:20.450851+0000","last_active":"2024-07-31T14:48:01.591990+0000","last_peered":"2024-07-31T14:48:01.591990+0000","last_clean":"2024-07-31T14:48:01.591990+0000","last_became_active":"2024-07-31T14:47:20.450435+0000","last_became_peered":"2024-07-31T14:47:20.450435+0000","last_unstale":"2024-07-31T14:48:01.591990+0000","last_undegraded":"2024-07-31T14:48:01.591990+0000","last_fullsized":"2024-07-31T14:48:01.591990+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:17:28.738911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.450860+0000","last_change":"2024-07-31T14:47:13.363054+0000","last_active":"2024-07-31T14:47:20.450860+0000","last_peered":"2024-07-31T14:47:20.450860+0000","last_clean":"2024-07-31T14:47:20.450860+0000","last_became_active":"2024-07-31T14:47:13.362364+0000","last_became_peered":"2024-07-31T14:47:13.362364+0000","last_unstale":"2024-07-31T14:47:20.450860+0000","last_undegraded":"2024-07-31T14:47:20.450860+0000","last_fullsized":"2024-07-31T14:47:20.450860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:02:36.569678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626742+0000","last_change":"2024-07-31T14:47:03.059834+0000","last_active":"2024-07-31T14:47:20.626742+0000","last_peered":"2024-07-31T14:47:20.626742+0000","last_clean":"2024-07-31T14:47:20.626742+0000","last_became_active":"2024-07-31T14:47:03.059127+0000","last_became_peered":"2024-07-31T14:47:03.059127+0000","last_unstale":"2024-07-31T14:47:20.626742+0000","last_undegraded":"2024-07-31T14:47:20.626742+0000","last_fullsized":"2024-07-31T14:47:20.626742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:53:50.308789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":153,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:44.773330+0000","last_change":"2024-07-31T14:47:00.030576+0000","last_active":"2024-07-31T14:49:44.773330+0000","last_peered":"2024-07-31T14:49:44.773330+0000","last_clean":"2024-07-31T14:49:44.773330+0000","last_became_active":"2024-07-31T14:47:00.030174+0000","last_became_peered":"2024-07-31T14:47:00.030174+0000","last_unstale":"2024-07-31T14:49:44.773330+0000","last_undegraded":"2024-07-31T14:49:44.773330+0000","last_fullsized":"2024-07-31T14:49:44.773330+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:13:35.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447258+0000","last_change":"2024-07-31T14:47:13.100180+0000","last_active":"2024-07-31T14:47:20.447258+0000","last_peered":"2024-07-31T14:47:20.447258+0000","last_clean":"2024-07-31T14:47:20.447258+0000","last_became_active":"2024-07-31T14:47:13.099725+0000","last_became_peered":"2024-07-31T14:47:13.099725+0000","last_unstale":"2024-07-31T14:47:20.447258+0000","last_undegraded":"2024-07-31T14:47:20.447258+0000","last_fullsized":"2024-07-31T14:47:20.447258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:25:32.245339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626691+0000","last_change":"2024-07-31T14:47:03.040349+0000","last_active":"2024-07-31T14:47:20.626691+0000","last_peered":"2024-07-31T14:47:20.626691+0000","last_clean":"2024-07-31T14:47:20.626691+0000","last_became_active":"2024-07-31T14:47:03.039887+0000","last_became_peered":"2024-07-31T14:47:03.039887+0000","last_unstale":"2024-07-31T14:47:20.626691+0000","last_undegraded":"2024-07-31T14:47:20.626691+0000","last_fullsized":"2024-07-31T14:47:20.626691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:46.592063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":153,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:46.758091+0000","last_change":"2024-07-31T14:47:00.183988+0000","last_active":"2024-07-31T14:49:46.758091+0000","last_peered":"2024-07-31T14:49:46.758091+0000","last_clean":"2024-07-31T14:49:46.758091+0000","last_became_active":"2024-07-31T14:47:00.183302+0000","last_became_peered":"2024-07-31T14:47:00.183302+0000","last_unstale":"2024-07-31T14:49:46.758091+0000","last_undegraded":"2024-07-31T14:49:46.758091+0000","last_fullsized":"2024-07-31T14:49:46.758091+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:40:26.188224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447744+0000","last_change":"2024-07-31T14:47:13.363075+0000","last_active":"2024-07-31T14:47:20.447744+0000","last_peered":"2024-07-31T14:47:20.447744+0000","last_clean":"2024-07-31T14:47:20.447744+0000","last_became_active":"2024-07-31T14:47:13.362599+0000","last_became_peered":"2024-07-31T14:47:13.362599+0000","last_unstale":"2024-07-31T14:47:20.447744+0000","last_undegraded":"2024-07-31T14:47:20.447744+0000","last_fullsized":"2024-07-31T14:47:20.447744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:20:43.617742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451262+0000","last_change":"2024-07-31T14:47:20.450712+0000","last_active":"2024-07-31T14:47:20.451262+0000","last_peered":"2024-07-31T14:47:20.451262+0000","last_clean":"2024-07-31T14:47:20.451262+0000","last_became_active":"2024-07-31T14:47:20.450229+0000","last_became_peered":"2024-07-31T14:47:20.450229+0000","last_unstale":"2024-07-31T14:47:20.451262+0000","last_undegraded":"2024-07-31T14:47:20.451262+0000","last_fullsized":"2024-07-31T14:47:20.451262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:11:50.631420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600465+0000","last_change":"2024-07-31T14:47:00.021389+0000","last_active":"2024-07-31T14:48:02.600465+0000","last_peered":"2024-07-31T14:48:02.600465+0000","last_clean":"2024-07-31T14:48:02.600465+0000","last_became_active":"2024-07-31T14:47:00.020987+0000","last_became_peered":"2024-07-31T14:47:00.020987+0000","last_unstale":"2024-07-31T14:48:02.600465+0000","last_undegraded":"2024-07-31T14:48:02.600465+0000","last_fullsized":"2024-07-31T14:48:02.600465+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:07:24.658463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627003+0000","last_change":"2024-07-31T14:47:13.363105+0000","last_active":"2024-07-31T14:47:20.627003+0000","last_peered":"2024-07-31T14:47:20.627003+0000","last_clean":"2024-07-31T14:47:20.627003+0000","last_became_active":"2024-07-31T14:47:13.362404+0000","last_became_peered":"2024-07-31T14:47:13.362404+0000","last_unstale":"2024-07-31T14:47:20.627003+0000","last_undegraded":"2024-07-31T14:47:20.627003+0000","last_fullsized":"2024-07-31T14:47:20.627003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:48:43.033584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626198+0000","last_change":"2024-07-31T14:47:03.057199+0000","last_active":"2024-07-31T14:47:20.626198+0000","last_peered":"2024-07-31T14:47:20.626198+0000","last_clean":"2024-07-31T14:47:20.626198+0000","last_became_active":"2024-07-31T14:47:03.056555+0000","last_became_peered":"2024-07-31T14:47:03.056555+0000","last_unstale":"2024-07-31T14:47:20.626198+0000","last_undegraded":"2024-07-31T14:47:20.626198+0000","last_fullsized":"2024-07-31T14:47:20.626198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:21:14.263457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605605+0000","last_change":"2024-07-31T14:47:00.032055+0000","last_active":"2024-07-31T14:48:03.605605+0000","last_peered":"2024-07-31T14:48:03.605605+0000","last_clean":"2024-07-31T14:48:03.605605+0000","last_became_active":"2024-07-31T14:47:00.031519+0000","last_became_peered":"2024-07-31T14:47:00.031519+0000","last_unstale":"2024-07-31T14:48:03.605605+0000","last_undegraded":"2024-07-31T14:48:03.605605+0000","last_fullsized":"2024-07-31T14:48:03.605605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:21:46.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444468+0000","last_change":"2024-07-31T14:45:42.166503+0000","last_active":"2024-07-31T14:47:20.444468+0000","last_peered":"2024-07-31T14:47:20.444468+0000","last_clean":"2024-07-31T14:47:20.444468+0000","last_became_active":"2024-07-31T14:45:42.160065+0000","last_became_peered":"2024-07-31T14:45:42.160065+0000","last_unstale":"2024-07-31T14:47:20.444468+0000","last_undegraded":"2024-07-31T14:47:20.444468+0000","last_fullsized":"2024-07-31T14:47:20.444468+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_clean_scrub_stamp":"2024-07-31T14:42:27.863113+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:38:33.646022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:12.369486+0000","last_change":"2024-07-31T14:47:13.103812+0000","last_active":"2024-07-31T14:48:12.369486+0000","last_peered":"2024-07-31T14:48:12.369486+0000","last_clean":"2024-07-31T14:48:12.369486+0000","last_became_active":"2024-07-31T14:47:13.102549+0000","last_became_peered":"2024-07-31T14:47:13.102549+0000","last_unstale":"2024-07-31T14:48:12.369486+0000","last_undegraded":"2024-07-31T14:48:12.369486+0000","last_fullsized":"2024-07-31T14:48:12.369486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:18:56.322426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444907+0000","last_change":"2024-07-31T14:47:03.056310+0000","last_active":"2024-07-31T14:47:20.444907+0000","last_peered":"2024-07-31T14:47:20.444907+0000","last_clean":"2024-07-31T14:47:20.444907+0000","last_became_active":"2024-07-31T14:47:03.055695+0000","last_became_peered":"2024-07-31T14:47:03.055695+0000","last_unstale":"2024-07-31T14:47:20.444907+0000","last_undegraded":"2024-07-31T14:47:20.444907+0000","last_fullsized":"2024-07-31T14:47:20.444907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:00:55.124981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:44.756005+0000","last_change":"2024-07-31T14:47:00.236460+0000","last_active":"2024-07-31T14:49:44.756005+0000","last_peered":"2024-07-31T14:49:44.756005+0000","last_clean":"2024-07-31T14:49:44.756005+0000","last_became_active":"2024-07-31T14:47:00.235924+0000","last_became_peered":"2024-07-31T14:47:00.235924+0000","last_unstale":"2024-07-31T14:49:44.756005+0000","last_undegraded":"2024-07-31T14:49:44.756005+0000","last_fullsized":"2024-07-31T14:49:44.756005+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:05.130958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626080+0000","last_change":"2024-07-31T14:47:13.102341+0000","last_active":"2024-07-31T14:47:20.626080+0000","last_peered":"2024-07-31T14:47:20.626080+0000","last_clean":"2024-07-31T14:47:20.626080+0000","last_became_active":"2024-07-31T14:47:13.101798+0000","last_became_peered":"2024-07-31T14:47:13.101798+0000","last_unstale":"2024-07-31T14:47:20.626080+0000","last_undegraded":"2024-07-31T14:47:20.626080+0000","last_fullsized":"2024-07-31T14:47:20.626080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:34:35.758041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444706+0000","last_change":"2024-07-31T14:47:03.054322+0000","last_active":"2024-07-31T14:47:20.444706+0000","last_peered":"2024-07-31T14:47:20.444706+0000","last_clean":"2024-07-31T14:47:20.444706+0000","last_became_active":"2024-07-31T14:47:03.053112+0000","last_became_peered":"2024-07-31T14:47:03.053112+0000","last_unstale":"2024-07-31T14:47:20.444706+0000","last_undegraded":"2024-07-31T14:47:20.444706+0000","last_fullsized":"2024-07-31T14:47:20.444706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:55:13.542578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604832+0000","last_change":"2024-07-31T14:47:00.026518+0000","last_active":"2024-07-31T14:48:03.604832+0000","last_peered":"2024-07-31T14:48:03.604832+0000","last_clean":"2024-07-31T14:48:03.604832+0000","last_became_active":"2024-07-31T14:47:00.026074+0000","last_became_peered":"2024-07-31T14:47:00.026074+0000","last_unstale":"2024-07-31T14:48:03.604832+0000","last_undegraded":"2024-07-31T14:48:03.604832+0000","last_fullsized":"2024-07-31T14:48:03.604832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:33:36.688353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449369+0000","last_change":"2024-07-31T14:47:13.106365+0000","last_active":"2024-07-31T14:47:20.449369+0000","last_peered":"2024-07-31T14:47:20.449369+0000","last_clean":"2024-07-31T14:47:20.449369+0000","last_became_active":"2024-07-31T14:47:13.105635+0000","last_became_peered":"2024-07-31T14:47:13.105635+0000","last_unstale":"2024-07-31T14:47:20.449369+0000","last_undegraded":"2024-07-31T14:47:20.449369+0000","last_fullsized":"2024-07-31T14:47:20.449369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:52:09.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444864+0000","last_change":"2024-07-31T14:47:03.052522+0000","last_active":"2024-07-31T14:47:20.444864+0000","last_peered":"2024-07-31T14:47:20.444864+0000","last_clean":"2024-07-31T14:47:20.444864+0000","last_became_active":"2024-07-31T14:47:03.051872+0000","last_became_peered":"2024-07-31T14:47:03.051872+0000","last_unstale":"2024-07-31T14:47:20.444864+0000","last_undegraded":"2024-07-31T14:47:20.444864+0000","last_fullsized":"2024-07-31T14:47:20.444864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:29:34.956745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600950+0000","last_change":"2024-07-31T14:47:00.016839+0000","last_active":"2024-07-31T14:48:02.600950+0000","last_peered":"2024-07-31T14:48:02.600950+0000","last_clean":"2024-07-31T14:48:02.600950+0000","last_became_active":"2024-07-31T14:47:00.016438+0000","last_became_peered":"2024-07-31T14:47:00.016438+0000","last_unstale":"2024-07-31T14:48:02.600950+0000","last_undegraded":"2024-07-31T14:48:02.600950+0000","last_fullsized":"2024-07-31T14:48:02.600950+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:42:10.875332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626258+0000","last_change":"2024-07-31T14:47:13.105385+0000","last_active":"2024-07-31T14:47:20.626258+0000","last_peered":"2024-07-31T14:47:20.626258+0000","last_clean":"2024-07-31T14:47:20.626258+0000","last_became_active":"2024-07-31T14:47:13.104867+0000","last_became_peered":"2024-07-31T14:47:13.104867+0000","last_unstale":"2024-07-31T14:47:20.626258+0000","last_undegraded":"2024-07-31T14:47:20.626258+0000","last_fullsized":"2024-07-31T14:47:20.626258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:25:21.572830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445679+0000","last_change":"2024-07-31T14:47:03.058304+0000","last_active":"2024-07-31T14:47:20.445679+0000","last_peered":"2024-07-31T14:47:20.445679+0000","last_clean":"2024-07-31T14:47:20.445679+0000","last_became_active":"2024-07-31T14:47:03.057251+0000","last_became_peered":"2024-07-31T14:47:03.057251+0000","last_unstale":"2024-07-31T14:47:20.445679+0000","last_undegraded":"2024-07-31T14:47:20.445679+0000","last_fullsized":"2024-07-31T14:47:20.445679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:58:20.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605301+0000","last_change":"2024-07-31T14:47:00.027806+0000","last_active":"2024-07-31T14:48:03.605301+0000","last_peered":"2024-07-31T14:48:03.605301+0000","last_clean":"2024-07-31T14:48:03.605301+0000","last_became_active":"2024-07-31T14:47:00.027403+0000","last_became_peered":"2024-07-31T14:47:00.027403+0000","last_unstale":"2024-07-31T14:48:03.605301+0000","last_undegraded":"2024-07-31T14:48:03.605301+0000","last_fullsized":"2024-07-31T14:48:03.605301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:57:18.303732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626490+0000","last_change":"2024-07-31T14:47:13.103628+0000","last_active":"2024-07-31T14:47:20.626490+0000","last_peered":"2024-07-31T14:47:20.626490+0000","last_clean":"2024-07-31T14:47:20.626490+0000","last_became_active":"2024-07-31T14:47:13.102296+0000","last_became_peered":"2024-07-31T14:47:13.102296+0000","last_unstale":"2024-07-31T14:47:20.626490+0000","last_undegraded":"2024-07-31T14:47:20.626490+0000","last_fullsized":"2024-07-31T14:47:20.626490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:25:35.109196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626574+0000","last_change":"2024-07-31T14:47:03.052761+0000","last_active":"2024-07-31T14:47:20.626574+0000","last_peered":"2024-07-31T14:47:20.626574+0000","last_clean":"2024-07-31T14:47:20.626574+0000","last_became_active":"2024-07-31T14:47:03.052410+0000","last_became_peered":"2024-07-31T14:47:03.052410+0000","last_unstale":"2024-07-31T14:47:20.626574+0000","last_undegraded":"2024-07-31T14:47:20.626574+0000","last_fullsized":"2024-07-31T14:47:20.626574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:20:47.712491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604937+0000","last_change":"2024-07-31T14:47:00.183741+0000","last_active":"2024-07-31T14:48:03.604937+0000","last_peered":"2024-07-31T14:48:03.604937+0000","last_clean":"2024-07-31T14:48:03.604937+0000","last_became_active":"2024-07-31T14:47:00.183275+0000","last_became_peered":"2024-07-31T14:47:00.183275+0000","last_unstale":"2024-07-31T14:48:03.604937+0000","last_undegraded":"2024-07-31T14:48:03.604937+0000","last_fullsized":"2024-07-31T14:48:03.604937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:29:23.205835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630098+0000","last_change":"2024-07-31T14:47:20.629334+0000","last_active":"2024-07-31T14:47:20.630098+0000","last_peered":"2024-07-31T14:47:20.630098+0000","last_clean":"2024-07-31T14:47:20.630098+0000","last_became_active":"2024-07-31T14:47:20.628865+0000","last_became_peered":"2024-07-31T14:47:20.628865+0000","last_unstale":"2024-07-31T14:47:20.630098+0000","last_undegraded":"2024-07-31T14:47:20.630098+0000","last_fullsized":"2024-07-31T14:47:20.630098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:21:27.199842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444152+0000","last_change":"2024-07-31T14:47:03.054633+0000","last_active":"2024-07-31T14:47:20.444152+0000","last_peered":"2024-07-31T14:47:20.444152+0000","last_clean":"2024-07-31T14:47:20.444152+0000","last_became_active":"2024-07-31T14:47:03.053025+0000","last_became_peered":"2024-07-31T14:47:03.053025+0000","last_unstale":"2024-07-31T14:47:20.444152+0000","last_undegraded":"2024-07-31T14:47:20.444152+0000","last_fullsized":"2024-07-31T14:47:20.444152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:42:03.392744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:48.761574+0000","last_change":"2024-07-31T14:47:00.027898+0000","last_active":"2024-07-31T14:49:48.761574+0000","last_peered":"2024-07-31T14:49:48.761574+0000","last_clean":"2024-07-31T14:49:48.761574+0000","last_became_active":"2024-07-31T14:47:00.027468+0000","last_became_peered":"2024-07-31T14:47:00.027468+0000","last_unstale":"2024-07-31T14:49:48.761574+0000","last_undegraded":"2024-07-31T14:49:48.761574+0000","last_fullsized":"2024-07-31T14:49:48.761574+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:12:45.922172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444574+0000","last_change":"2024-07-31T14:47:13.104327+0000","last_active":"2024-07-31T14:47:20.444574+0000","last_peered":"2024-07-31T14:47:20.444574+0000","last_clean":"2024-07-31T14:47:20.444574+0000","last_became_active":"2024-07-31T14:47:13.103333+0000","last_became_peered":"2024-07-31T14:47:13.103333+0000","last_unstale":"2024-07-31T14:47:20.444574+0000","last_undegraded":"2024-07-31T14:47:20.444574+0000","last_fullsized":"2024-07-31T14:47:20.444574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:06.226220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444498+0000","last_change":"2024-07-31T14:47:03.044875+0000","last_active":"2024-07-31T14:47:20.444498+0000","last_peered":"2024-07-31T14:47:20.444498+0000","last_clean":"2024-07-31T14:47:20.444498+0000","last_became_active":"2024-07-31T14:47:03.044421+0000","last_became_peered":"2024-07-31T14:47:03.044421+0000","last_unstale":"2024-07-31T14:47:20.444498+0000","last_undegraded":"2024-07-31T14:47:20.444498+0000","last_fullsized":"2024-07-31T14:47:20.444498+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:00:02.477300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605606+0000","last_change":"2024-07-31T14:47:00.019790+0000","last_active":"2024-07-31T14:48:03.605606+0000","last_peered":"2024-07-31T14:48:03.605606+0000","last_clean":"2024-07-31T14:48:03.605606+0000","last_became_active":"2024-07-31T14:47:00.019368+0000","last_became_peered":"2024-07-31T14:47:00.019368+0000","last_unstale":"2024-07-31T14:48:03.605606+0000","last_undegraded":"2024-07-31T14:48:03.605606+0000","last_fullsized":"2024-07-31T14:48:03.605606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:15.299109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445672+0000","last_change":"2024-07-31T14:47:13.102618+0000","last_active":"2024-07-31T14:47:20.445672+0000","last_peered":"2024-07-31T14:47:20.445672+0000","last_clean":"2024-07-31T14:47:20.445672+0000","last_became_active":"2024-07-31T14:47:13.102085+0000","last_became_peered":"2024-07-31T14:47:13.102085+0000","last_unstale":"2024-07-31T14:47:20.445672+0000","last_undegraded":"2024-07-31T14:47:20.445672+0000","last_fullsized":"2024-07-31T14:47:20.445672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:05:58.814215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447389+0000","last_change":"2024-07-31T14:47:03.044688+0000","last_active":"2024-07-31T14:47:20.447389+0000","last_peered":"2024-07-31T14:47:20.447389+0000","last_clean":"2024-07-31T14:47:20.447389+0000","last_became_active":"2024-07-31T14:47:03.043751+0000","last_became_peered":"2024-07-31T14:47:03.043751+0000","last_unstale":"2024-07-31T14:47:20.447389+0000","last_undegraded":"2024-07-31T14:47:20.447389+0000","last_fullsized":"2024-07-31T14:47:20.447389+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:47:31.450796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605830+0000","last_change":"2024-07-31T14:47:00.026259+0000","last_active":"2024-07-31T14:48:03.605830+0000","last_peered":"2024-07-31T14:48:03.605830+0000","last_clean":"2024-07-31T14:48:03.605830+0000","last_became_active":"2024-07-31T14:47:00.025756+0000","last_became_peered":"2024-07-31T14:47:00.025756+0000","last_unstale":"2024-07-31T14:48:03.605830+0000","last_undegraded":"2024-07-31T14:48:03.605830+0000","last_fullsized":"2024-07-31T14:48:03.605830+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:05:00.258841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626888+0000","last_change":"2024-07-31T14:47:13.104918+0000","last_active":"2024-07-31T14:47:20.626888+0000","last_peered":"2024-07-31T14:47:20.626888+0000","last_clean":"2024-07-31T14:47:20.626888+0000","last_became_active":"2024-07-31T14:47:13.104031+0000","last_became_peered":"2024-07-31T14:47:13.104031+0000","last_unstale":"2024-07-31T14:47:20.626888+0000","last_undegraded":"2024-07-31T14:47:20.626888+0000","last_fullsized":"2024-07-31T14:47:20.626888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:17:15.264474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451885+0000","last_change":"2024-07-31T14:47:03.055159+0000","last_active":"2024-07-31T14:47:20.451885+0000","last_peered":"2024-07-31T14:47:20.451885+0000","last_clean":"2024-07-31T14:47:20.451885+0000","last_became_active":"2024-07-31T14:47:03.052389+0000","last_became_peered":"2024-07-31T14:47:03.052389+0000","last_unstale":"2024-07-31T14:47:20.451885+0000","last_undegraded":"2024-07-31T14:47:20.451885+0000","last_fullsized":"2024-07-31T14:47:20.451885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:16.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'35","reported_seq":290,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:44.755220+0000","last_change":"2024-07-31T14:47:00.030165+0000","last_active":"2024-07-31T14:49:44.755220+0000","last_peered":"2024-07-31T14:49:44.755220+0000","last_clean":"2024-07-31T14:49:44.755220+0000","last_became_active":"2024-07-31T14:47:00.029559+0000","last_became_peered":"2024-07-31T14:47:00.029559+0000","last_unstale":"2024-07-31T14:49:44.755220+0000","last_undegraded":"2024-07-31T14:49:44.755220+0000","last_fullsized":"2024-07-31T14:49:44.755220+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:31:06.115075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64136,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":213,"num_read_kb":169,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444234+0000","last_change":"2024-07-31T14:47:13.098517+0000","last_active":"2024-07-31T14:47:20.444234+0000","last_peered":"2024-07-31T14:47:20.444234+0000","last_clean":"2024-07-31T14:47:20.444234+0000","last_became_active":"2024-07-31T14:47:13.098066+0000","last_became_peered":"2024-07-31T14:47:13.098066+0000","last_unstale":"2024-07-31T14:47:20.444234+0000","last_undegraded":"2024-07-31T14:47:20.444234+0000","last_fullsized":"2024-07-31T14:47:20.444234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:08:54.958646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626359+0000","last_change":"2024-07-31T14:47:03.054045+0000","last_active":"2024-07-31T14:47:20.626359+0000","last_peered":"2024-07-31T14:47:20.626359+0000","last_clean":"2024-07-31T14:47:20.626359+0000","last_became_active":"2024-07-31T14:47:03.053658+0000","last_became_peered":"2024-07-31T14:47:03.053658+0000","last_unstale":"2024-07-31T14:47:20.626359+0000","last_undegraded":"2024-07-31T14:47:20.626359+0000","last_fullsized":"2024-07-31T14:47:20.626359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:36:54.563542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.607373+0000","last_change":"2024-07-31T14:47:00.185602+0000","last_active":"2024-07-31T14:48:03.607373+0000","last_peered":"2024-07-31T14:48:03.607373+0000","last_clean":"2024-07-31T14:48:03.607373+0000","last_became_active":"2024-07-31T14:47:00.185230+0000","last_became_peered":"2024-07-31T14:47:00.185230+0000","last_unstale":"2024-07-31T14:48:03.607373+0000","last_undegraded":"2024-07-31T14:48:03.607373+0000","last_fullsized":"2024-07-31T14:48:03.607373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:21:35.979942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604698+0000","last_change":"2024-07-31T14:47:13.099611+0000","last_active":"2024-07-31T14:47:20.604698+0000","last_peered":"2024-07-31T14:47:20.604698+0000","last_clean":"2024-07-31T14:47:20.604698+0000","last_became_active":"2024-07-31T14:47:13.099127+0000","last_became_peered":"2024-07-31T14:47:13.099127+0000","last_unstale":"2024-07-31T14:47:20.604698+0000","last_undegraded":"2024-07-31T14:47:20.604698+0000","last_fullsized":"2024-07-31T14:47:20.604698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:16:57.192980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449785+0000","last_change":"2024-07-31T14:47:03.056818+0000","last_active":"2024-07-31T14:47:20.449785+0000","last_peered":"2024-07-31T14:47:20.449785+0000","last_clean":"2024-07-31T14:47:20.449785+0000","last_became_active":"2024-07-31T14:47:03.055716+0000","last_became_peered":"2024-07-31T14:47:03.055716+0000","last_unstale":"2024-07-31T14:47:20.449785+0000","last_undegraded":"2024-07-31T14:47:20.449785+0000","last_fullsized":"2024-07-31T14:47:20.449785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:26:33.627362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600344+0000","last_change":"2024-07-31T14:47:00.029651+0000","last_active":"2024-07-31T14:48:02.600344+0000","last_peered":"2024-07-31T14:48:02.600344+0000","last_clean":"2024-07-31T14:48:02.600344+0000","last_became_active":"2024-07-31T14:47:00.029242+0000","last_became_peered":"2024-07-31T14:47:00.029242+0000","last_unstale":"2024-07-31T14:48:02.600344+0000","last_undegraded":"2024-07-31T14:48:02.600344+0000","last_fullsized":"2024-07-31T14:48:02.600344+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:23:49.177508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.443852+0000","last_change":"2024-07-31T14:47:13.098862+0000","last_active":"2024-07-31T14:47:20.443852+0000","last_peered":"2024-07-31T14:47:20.443852+0000","last_clean":"2024-07-31T14:47:20.443852+0000","last_became_active":"2024-07-31T14:47:13.098324+0000","last_became_peered":"2024-07-31T14:47:13.098324+0000","last_unstale":"2024-07-31T14:47:20.443852+0000","last_undegraded":"2024-07-31T14:47:20.443852+0000","last_fullsized":"2024-07-31T14:47:20.443852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:50:41.260863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626657+0000","last_change":"2024-07-31T14:47:03.049042+0000","last_active":"2024-07-31T14:47:20.626657+0000","last_peered":"2024-07-31T14:47:20.626657+0000","last_clean":"2024-07-31T14:47:20.626657+0000","last_became_active":"2024-07-31T14:47:03.048535+0000","last_became_peered":"2024-07-31T14:47:03.048535+0000","last_unstale":"2024-07-31T14:47:20.626657+0000","last_undegraded":"2024-07-31T14:47:20.626657+0000","last_fullsized":"2024-07-31T14:47:20.626657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:15.147454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":250,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:44.755287+0000","last_change":"2024-07-31T14:47:00.028940+0000","last_active":"2024-07-31T14:49:44.755287+0000","last_peered":"2024-07-31T14:49:44.755287+0000","last_clean":"2024-07-31T14:49:44.755287+0000","last_became_active":"2024-07-31T14:47:00.028472+0000","last_became_peered":"2024-07-31T14:47:00.028472+0000","last_unstale":"2024-07-31T14:49:44.755287+0000","last_undegraded":"2024-07-31T14:49:44.755287+0000","last_fullsized":"2024-07-31T14:49:44.755287+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:57:15.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626394+0000","last_change":"2024-07-31T14:47:13.101856+0000","last_active":"2024-07-31T14:47:20.626394+0000","last_peered":"2024-07-31T14:47:20.626394+0000","last_clean":"2024-07-31T14:47:20.626394+0000","last_became_active":"2024-07-31T14:47:13.101346+0000","last_became_peered":"2024-07-31T14:47:13.101346+0000","last_unstale":"2024-07-31T14:47:20.626394+0000","last_undegraded":"2024-07-31T14:47:20.626394+0000","last_fullsized":"2024-07-31T14:47:20.626394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:56:37.071660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449190+0000","last_change":"2024-07-31T14:47:13.105952+0000","last_active":"2024-07-31T14:47:20.449190+0000","last_peered":"2024-07-31T14:47:20.449190+0000","last_clean":"2024-07-31T14:47:20.449190+0000","last_became_active":"2024-07-31T14:47:13.105506+0000","last_became_peered":"2024-07-31T14:47:13.105506+0000","last_unstale":"2024-07-31T14:47:20.449190+0000","last_undegraded":"2024-07-31T14:47:20.449190+0000","last_fullsized":"2024-07-31T14:47:20.449190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:28:33.866216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444196+0000","last_change":"2024-07-31T14:47:03.051152+0000","last_active":"2024-07-31T14:47:20.444196+0000","last_peered":"2024-07-31T14:47:20.444196+0000","last_clean":"2024-07-31T14:47:20.444196+0000","last_became_active":"2024-07-31T14:47:03.050775+0000","last_became_peered":"2024-07-31T14:47:03.050775+0000","last_unstale":"2024-07-31T14:47:20.444196+0000","last_undegraded":"2024-07-31T14:47:20.444196+0000","last_fullsized":"2024-07-31T14:47:20.444196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:47:35.913641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609201+0000","last_change":"2024-07-31T14:47:00.028552+0000","last_active":"2024-07-31T14:48:03.609201+0000","last_peered":"2024-07-31T14:48:03.609201+0000","last_clean":"2024-07-31T14:48:03.609201+0000","last_became_active":"2024-07-31T14:47:00.028143+0000","last_became_peered":"2024-07-31T14:47:00.028143+0000","last_unstale":"2024-07-31T14:48:03.609201+0000","last_undegraded":"2024-07-31T14:48:03.609201+0000","last_fullsized":"2024-07-31T14:48:03.609201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:09.601635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.606879+0000","last_change":"2024-07-31T14:47:20.606285+0000","last_active":"2024-07-31T14:47:20.606879+0000","last_peered":"2024-07-31T14:47:20.606879+0000","last_clean":"2024-07-31T14:47:20.606879+0000","last_became_active":"2024-07-31T14:47:20.605837+0000","last_became_peered":"2024-07-31T14:47:20.605837+0000","last_unstale":"2024-07-31T14:47:20.606879+0000","last_undegraded":"2024-07-31T14:47:20.606879+0000","last_fullsized":"2024-07-31T14:47:20.606879+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:16:35.409790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626001+0000","last_change":"2024-07-31T14:47:03.047442+0000","last_active":"2024-07-31T14:47:20.626001+0000","last_peered":"2024-07-31T14:47:20.626001+0000","last_clean":"2024-07-31T14:47:20.626001+0000","last_became_active":"2024-07-31T14:47:03.047072+0000","last_became_peered":"2024-07-31T14:47:03.047072+0000","last_unstale":"2024-07-31T14:47:20.626001+0000","last_undegraded":"2024-07-31T14:47:20.626001+0000","last_fullsized":"2024-07-31T14:47:20.626001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:56.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.599595+0000","last_change":"2024-07-31T14:47:00.027746+0000","last_active":"2024-07-31T14:48:02.599595+0000","last_peered":"2024-07-31T14:48:02.599595+0000","last_clean":"2024-07-31T14:48:02.599595+0000","last_became_active":"2024-07-31T14:47:00.027404+0000","last_became_peered":"2024-07-31T14:47:00.027404+0000","last_unstale":"2024-07-31T14:48:02.599595+0000","last_undegraded":"2024-07-31T14:48:02.599595+0000","last_fullsized":"2024-07-31T14:48:02.599595+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:30:21.011674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.589640+0000","last_change":"2024-07-31T14:47:00.029148+0000","last_active":"2024-07-31T14:48:01.589640+0000","last_peered":"2024-07-31T14:48:01.589640+0000","last_clean":"2024-07-31T14:48:01.589640+0000","last_became_active":"2024-07-31T14:47:00.028718+0000","last_became_peered":"2024-07-31T14:47:00.028718+0000","last_unstale":"2024-07-31T14:48:01.589640+0000","last_undegraded":"2024-07-31T14:48:01.589640+0000","last_fullsized":"2024-07-31T14:48:01.589640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:25:16.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630241+0000","last_change":"2024-07-31T14:47:20.629098+0000","last_active":"2024-07-31T14:47:20.630241+0000","last_peered":"2024-07-31T14:47:20.630241+0000","last_clean":"2024-07-31T14:47:20.630241+0000","last_became_active":"2024-07-31T14:47:20.627775+0000","last_became_peered":"2024-07-31T14:47:20.627775+0000","last_unstale":"2024-07-31T14:47:20.630241+0000","last_undegraded":"2024-07-31T14:47:20.630241+0000","last_fullsized":"2024-07-31T14:47:20.630241+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:22:03.606509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447331+0000","last_change":"2024-07-31T14:47:13.097892+0000","last_active":"2024-07-31T14:47:20.447331+0000","last_peered":"2024-07-31T14:47:20.447331+0000","last_clean":"2024-07-31T14:47:20.447331+0000","last_became_active":"2024-07-31T14:47:13.097419+0000","last_became_peered":"2024-07-31T14:47:13.097419+0000","last_unstale":"2024-07-31T14:47:20.447331+0000","last_undegraded":"2024-07-31T14:47:20.447331+0000","last_fullsized":"2024-07-31T14:47:20.447331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:07:13.456147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":156,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:46.758911+0000","last_change":"2024-07-31T14:47:00.019792+0000","last_active":"2024-07-31T14:49:46.758911+0000","last_peered":"2024-07-31T14:49:46.758911+0000","last_clean":"2024-07-31T14:49:46.758911+0000","last_became_active":"2024-07-31T14:47:00.019411+0000","last_became_peered":"2024-07-31T14:47:00.019411+0000","last_unstale":"2024-07-31T14:49:46.758911+0000","last_undegraded":"2024-07-31T14:49:46.758911+0000","last_fullsized":"2024-07-31T14:49:46.758911+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:55:02.148823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.450987+0000","last_change":"2024-07-31T14:47:03.039808+0000","last_active":"2024-07-31T14:47:20.450987+0000","last_peered":"2024-07-31T14:47:20.450987+0000","last_clean":"2024-07-31T14:47:20.450987+0000","last_became_active":"2024-07-31T14:47:03.039382+0000","last_became_peered":"2024-07-31T14:47:03.039382+0000","last_unstale":"2024-07-31T14:47:20.450987+0000","last_undegraded":"2024-07-31T14:47:20.450987+0000","last_fullsized":"2024-07-31T14:47:20.450987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:23:09.751293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.446939+0000","last_change":"2024-07-31T14:47:13.102797+0000","last_active":"2024-07-31T14:47:20.446939+0000","last_peered":"2024-07-31T14:47:20.446939+0000","last_clean":"2024-07-31T14:47:20.446939+0000","last_became_active":"2024-07-31T14:47:13.102320+0000","last_became_peered":"2024-07-31T14:47:13.102320+0000","last_unstale":"2024-07-31T14:47:20.446939+0000","last_undegraded":"2024-07-31T14:47:20.446939+0000","last_fullsized":"2024-07-31T14:47:20.446939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:23.066357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444764+0000","last_change":"2024-07-31T14:47:03.054875+0000","last_active":"2024-07-31T14:47:20.444764+0000","last_peered":"2024-07-31T14:47:20.444764+0000","last_clean":"2024-07-31T14:47:20.444764+0000","last_became_active":"2024-07-31T14:47:03.054529+0000","last_became_peered":"2024-07-31T14:47:03.054529+0000","last_unstale":"2024-07-31T14:47:20.444764+0000","last_undegraded":"2024-07-31T14:47:20.444764+0000","last_fullsized":"2024-07-31T14:47:20.444764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:32:09.802933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.448676+0000","last_change":"2024-07-31T14:47:03.043283+0000","last_active":"2024-07-31T14:47:20.448676+0000","last_peered":"2024-07-31T14:47:20.448676+0000","last_clean":"2024-07-31T14:47:20.448676+0000","last_became_active":"2024-07-31T14:47:03.042851+0000","last_became_peered":"2024-07-31T14:47:03.042851+0000","last_unstale":"2024-07-31T14:47:20.448676+0000","last_undegraded":"2024-07-31T14:47:20.448676+0000","last_fullsized":"2024-07-31T14:47:20.448676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:25:37.408149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447366+0000","last_change":"2024-07-31T14:47:03.047890+0000","last_active":"2024-07-31T14:47:20.447366+0000","last_peered":"2024-07-31T14:47:20.447366+0000","last_clean":"2024-07-31T14:47:20.447366+0000","last_became_active":"2024-07-31T14:47:03.047507+0000","last_became_peered":"2024-07-31T14:47:03.047507+0000","last_unstale":"2024-07-31T14:47:20.447366+0000","last_undegraded":"2024-07-31T14:47:20.447366+0000","last_fullsized":"2024-07-31T14:47:20.447366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:52:45.991323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449508+0000","last_change":"2024-07-31T14:47:03.047903+0000","last_active":"2024-07-31T14:47:20.449508+0000","last_peered":"2024-07-31T14:47:20.449508+0000","last_clean":"2024-07-31T14:47:20.449508+0000","last_became_active":"2024-07-31T14:47:03.047352+0000","last_became_peered":"2024-07-31T14:47:03.047352+0000","last_unstale":"2024-07-31T14:47:20.449508+0000","last_undegraded":"2024-07-31T14:47:20.449508+0000","last_fullsized":"2024-07-31T14:47:20.449508+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:06.536023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447404+0000","last_change":"2024-07-31T14:47:03.044199+0000","last_active":"2024-07-31T14:47:20.447404+0000","last_peered":"2024-07-31T14:47:20.447404+0000","last_clean":"2024-07-31T14:47:20.447404+0000","last_became_active":"2024-07-31T14:47:03.043849+0000","last_became_peered":"2024-07-31T14:47:03.043849+0000","last_unstale":"2024-07-31T14:47:20.447404+0000","last_undegraded":"2024-07-31T14:47:20.447404+0000","last_fullsized":"2024-07-31T14:47:20.447404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:50.474895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445962+0000","last_change":"2024-07-31T14:47:03.060654+0000","last_active":"2024-07-31T14:47:20.445962+0000","last_peered":"2024-07-31T14:47:20.445962+0000","last_clean":"2024-07-31T14:47:20.445962+0000","last_became_active":"2024-07-31T14:47:03.060157+0000","last_became_peered":"2024-07-31T14:47:03.060157+0000","last_unstale":"2024-07-31T14:47:20.445962+0000","last_undegraded":"2024-07-31T14:47:20.445962+0000","last_fullsized":"2024-07-31T14:47:20.445962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:39:50.130831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447079+0000","last_change":"2024-07-31T14:47:03.049910+0000","last_active":"2024-07-31T14:47:20.447079+0000","last_peered":"2024-07-31T14:47:20.447079+0000","last_clean":"2024-07-31T14:47:20.447079+0000","last_became_active":"2024-07-31T14:47:03.049491+0000","last_became_peered":"2024-07-31T14:47:03.049491+0000","last_unstale":"2024-07-31T14:47:20.447079+0000","last_undegraded":"2024-07-31T14:47:20.447079+0000","last_fullsized":"2024-07-31T14:47:20.447079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:18:43.319695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627314+0000","last_change":"2024-07-31T14:47:03.052206+0000","last_active":"2024-07-31T14:47:20.627314+0000","last_peered":"2024-07-31T14:47:20.627314+0000","last_clean":"2024-07-31T14:47:20.627314+0000","last_became_active":"2024-07-31T14:47:03.051175+0000","last_became_peered":"2024-07-31T14:47:03.051175+0000","last_unstale":"2024-07-31T14:47:20.627314+0000","last_undegraded":"2024-07-31T14:47:20.627314+0000","last_fullsized":"2024-07-31T14:47:20.627314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:51:11.583604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626542+0000","last_change":"2024-07-31T14:47:03.052162+0000","last_active":"2024-07-31T14:47:20.626542+0000","last_peered":"2024-07-31T14:47:20.626542+0000","last_clean":"2024-07-31T14:47:20.626542+0000","last_became_active":"2024-07-31T14:47:03.051634+0000","last_became_peered":"2024-07-31T14:47:03.051634+0000","last_unstale":"2024-07-31T14:47:20.626542+0000","last_undegraded":"2024-07-31T14:47:20.626542+0000","last_fullsized":"2024-07-31T14:47:20.626542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:41:59.605192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626490+0000","last_change":"2024-07-31T14:47:03.047475+0000","last_active":"2024-07-31T14:47:20.626490+0000","last_peered":"2024-07-31T14:47:20.626490+0000","last_clean":"2024-07-31T14:47:20.626490+0000","last_became_active":"2024-07-31T14:47:03.047100+0000","last_became_peered":"2024-07-31T14:47:03.047100+0000","last_unstale":"2024-07-31T14:47:20.626490+0000","last_undegraded":"2024-07-31T14:47:20.626490+0000","last_fullsized":"2024-07-31T14:47:20.626490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:08:10.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626514+0000","last_change":"2024-07-31T14:47:03.051281+0000","last_active":"2024-07-31T14:47:20.626514+0000","last_peered":"2024-07-31T14:47:20.626514+0000","last_clean":"2024-07-31T14:47:20.626514+0000","last_became_active":"2024-07-31T14:47:03.050921+0000","last_became_peered":"2024-07-31T14:47:03.050921+0000","last_unstale":"2024-07-31T14:47:20.626514+0000","last_undegraded":"2024-07-31T14:47:20.626514+0000","last_fullsized":"2024-07-31T14:47:20.626514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:24:29.221723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447441+0000","last_change":"2024-07-31T14:47:03.045457+0000","last_active":"2024-07-31T14:47:20.447441+0000","last_peered":"2024-07-31T14:47:20.447441+0000","last_clean":"2024-07-31T14:47:20.447441+0000","last_became_active":"2024-07-31T14:47:03.044466+0000","last_became_peered":"2024-07-31T14:47:03.044466+0000","last_unstale":"2024-07-31T14:47:20.447441+0000","last_undegraded":"2024-07-31T14:47:20.447441+0000","last_fullsized":"2024-07-31T14:47:20.447441+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:44:01.197416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447079+0000","last_change":"2024-07-31T14:47:03.053431+0000","last_active":"2024-07-31T14:47:20.447079+0000","last_peered":"2024-07-31T14:47:20.447079+0000","last_clean":"2024-07-31T14:47:20.447079+0000","last_became_active":"2024-07-31T14:47:03.052958+0000","last_became_peered":"2024-07-31T14:47:03.052958+0000","last_unstale":"2024-07-31T14:47:20.447079+0000","last_undegraded":"2024-07-31T14:47:20.447079+0000","last_fullsized":"2024-07-31T14:47:20.447079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:03:28.863589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447160+0000","last_change":"2024-07-31T14:47:03.046641+0000","last_active":"2024-07-31T14:47:20.447160+0000","last_peered":"2024-07-31T14:47:20.447160+0000","last_clean":"2024-07-31T14:47:20.447160+0000","last_became_active":"2024-07-31T14:47:03.046265+0000","last_became_peered":"2024-07-31T14:47:03.046265+0000","last_unstale":"2024-07-31T14:47:20.447160+0000","last_undegraded":"2024-07-31T14:47:20.447160+0000","last_fullsized":"2024-07-31T14:47:20.447160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:20:25.389010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444454+0000","last_change":"2024-07-31T14:47:03.051657+0000","last_active":"2024-07-31T14:47:20.444454+0000","last_peered":"2024-07-31T14:47:20.444454+0000","last_clean":"2024-07-31T14:47:20.444454+0000","last_became_active":"2024-07-31T14:47:03.050722+0000","last_became_peered":"2024-07-31T14:47:03.050722+0000","last_unstale":"2024-07-31T14:47:20.444454+0000","last_undegraded":"2024-07-31T14:47:20.444454+0000","last_fullsized":"2024-07-31T14:47:20.444454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:23:19.387470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447301+0000","last_change":"2024-07-31T14:47:03.059654+0000","last_active":"2024-07-31T14:47:20.447301+0000","last_peered":"2024-07-31T14:47:20.447301+0000","last_clean":"2024-07-31T14:47:20.447301+0000","last_became_active":"2024-07-31T14:47:03.056843+0000","last_became_peered":"2024-07-31T14:47:03.056843+0000","last_unstale":"2024-07-31T14:47:20.447301+0000","last_undegraded":"2024-07-31T14:47:20.447301+0000","last_fullsized":"2024-07-31T14:47:20.447301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:42:33.696498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625816+0000","last_change":"2024-07-31T14:47:03.052750+0000","last_active":"2024-07-31T14:47:20.625816+0000","last_peered":"2024-07-31T14:47:20.625816+0000","last_clean":"2024-07-31T14:47:20.625816+0000","last_became_active":"2024-07-31T14:47:03.051744+0000","last_became_peered":"2024-07-31T14:47:03.051744+0000","last_unstale":"2024-07-31T14:47:20.625816+0000","last_undegraded":"2024-07-31T14:47:20.625816+0000","last_fullsized":"2024-07-31T14:47:20.625816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:24:33.202918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.453019+0000","last_change":"2024-07-31T14:47:20.452563+0000","last_active":"2024-07-31T14:47:20.453019+0000","last_peered":"2024-07-31T14:47:20.453019+0000","last_clean":"2024-07-31T14:47:20.453019+0000","last_became_active":"2024-07-31T14:47:20.452271+0000","last_became_peered":"2024-07-31T14:47:20.452271+0000","last_unstale":"2024-07-31T14:47:20.453019+0000","last_undegraded":"2024-07-31T14:47:20.453019+0000","last_fullsized":"2024-07-31T14:47:20.453019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:26:55.070265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449004+0000","last_change":"2024-07-31T14:47:03.050691+0000","last_active":"2024-07-31T14:47:20.449004+0000","last_peered":"2024-07-31T14:47:20.449004+0000","last_clean":"2024-07-31T14:47:20.449004+0000","last_became_active":"2024-07-31T14:47:03.050320+0000","last_became_peered":"2024-07-31T14:47:03.050320+0000","last_unstale":"2024-07-31T14:47:20.449004+0000","last_undegraded":"2024-07-31T14:47:20.449004+0000","last_fullsized":"2024-07-31T14:47:20.449004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:40:45.767706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447607+0000","last_change":"2024-07-31T14:47:03.048491+0000","last_active":"2024-07-31T14:47:20.447607+0000","last_peered":"2024-07-31T14:47:20.447607+0000","last_clean":"2024-07-31T14:47:20.447607+0000","last_became_active":"2024-07-31T14:47:03.048085+0000","last_became_peered":"2024-07-31T14:47:03.048085+0000","last_unstale":"2024-07-31T14:47:20.447607+0000","last_undegraded":"2024-07-31T14:47:20.447607+0000","last_fullsized":"2024-07-31T14:47:20.447607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:35:01.762365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:29.383050+0000","last_change":"2024-07-31T14:47:03.055502+0000","last_active":"2024-07-31T14:49:29.383050+0000","last_peered":"2024-07-31T14:49:29.383050+0000","last_clean":"2024-07-31T14:49:29.383050+0000","last_became_active":"2024-07-31T14:47:03.055015+0000","last_became_peered":"2024-07-31T14:47:03.055015+0000","last_unstale":"2024-07-31T14:49:29.383050+0000","last_undegraded":"2024-07-31T14:49:29.383050+0000","last_fullsized":"2024-07-31T14:49:29.383050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:50:39.605148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444836+0000","last_change":"2024-07-31T14:47:03.057808+0000","last_active":"2024-07-31T14:47:20.444836+0000","last_peered":"2024-07-31T14:47:20.444836+0000","last_clean":"2024-07-31T14:47:20.444836+0000","last_became_active":"2024-07-31T14:47:03.057443+0000","last_became_peered":"2024-07-31T14:47:03.057443+0000","last_unstale":"2024-07-31T14:47:20.444836+0000","last_undegraded":"2024-07-31T14:47:20.444836+0000","last_fullsized":"2024-07-31T14:47:20.444836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:34:42.915428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.628089+0000","last_change":"2024-07-31T14:47:03.058182+0000","last_active":"2024-07-31T14:47:20.628089+0000","last_peered":"2024-07-31T14:47:20.628089+0000","last_clean":"2024-07-31T14:47:20.628089+0000","last_became_active":"2024-07-31T14:47:03.057802+0000","last_became_peered":"2024-07-31T14:47:03.057802+0000","last_unstale":"2024-07-31T14:47:20.628089+0000","last_undegraded":"2024-07-31T14:47:20.628089+0000","last_fullsized":"2024-07-31T14:47:20.628089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:08:06.608788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626457+0000","last_change":"2024-07-31T14:47:03.058882+0000","last_active":"2024-07-31T14:47:20.626457+0000","last_peered":"2024-07-31T14:47:20.626457+0000","last_clean":"2024-07-31T14:47:20.626457+0000","last_became_active":"2024-07-31T14:47:03.058530+0000","last_became_peered":"2024-07-31T14:47:03.058530+0000","last_unstale":"2024-07-31T14:47:20.626457+0000","last_undegraded":"2024-07-31T14:47:20.626457+0000","last_fullsized":"2024-07-31T14:47:20.626457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:28:03.631962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626017+0000","last_change":"2024-07-31T14:47:03.052028+0000","last_active":"2024-07-31T14:47:20.626017+0000","last_peered":"2024-07-31T14:47:20.626017+0000","last_clean":"2024-07-31T14:47:20.626017+0000","last_became_active":"2024-07-31T14:47:03.051559+0000","last_became_peered":"2024-07-31T14:47:03.051559+0000","last_unstale":"2024-07-31T14:47:20.626017+0000","last_undegraded":"2024-07-31T14:47:20.626017+0000","last_fullsized":"2024-07-31T14:47:20.626017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:16:40.723795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444253+0000","last_change":"2024-07-31T14:47:03.053884+0000","last_active":"2024-07-31T14:47:20.444253+0000","last_peered":"2024-07-31T14:47:20.444253+0000","last_clean":"2024-07-31T14:47:20.444253+0000","last_became_active":"2024-07-31T14:47:03.053521+0000","last_became_peered":"2024-07-31T14:47:03.053521+0000","last_unstale":"2024-07-31T14:47:20.444253+0000","last_undegraded":"2024-07-31T14:47:20.444253+0000","last_fullsized":"2024-07-31T14:47:20.444253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:16:37.702055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627188+0000","last_change":"2024-07-31T14:47:03.058588+0000","last_active":"2024-07-31T14:47:20.627188+0000","last_peered":"2024-07-31T14:47:20.627188+0000","last_clean":"2024-07-31T14:47:20.627188+0000","last_became_active":"2024-07-31T14:47:03.057820+0000","last_became_peered":"2024-07-31T14:47:03.057820+0000","last_unstale":"2024-07-31T14:47:20.627188+0000","last_undegraded":"2024-07-31T14:47:20.627188+0000","last_fullsized":"2024-07-31T14:47:20.627188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:21.265208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.451246+0000","last_change":"2024-07-31T14:47:03.041447+0000","last_active":"2024-07-31T14:47:20.451246+0000","last_peered":"2024-07-31T14:47:20.451246+0000","last_clean":"2024-07-31T14:47:20.451246+0000","last_became_active":"2024-07-31T14:47:03.040935+0000","last_became_peered":"2024-07-31T14:47:03.040935+0000","last_unstale":"2024-07-31T14:47:20.451246+0000","last_undegraded":"2024-07-31T14:47:20.451246+0000","last_fullsized":"2024-07-31T14:47:20.451246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:00:13.624698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:21.483666+0000","last_change":"2024-07-31T14:47:20.452239+0000","last_active":"2024-07-31T14:47:21.483666+0000","last_peered":"2024-07-31T14:47:21.483666+0000","last_clean":"2024-07-31T14:47:21.483666+0000","last_became_active":"2024-07-31T14:47:20.451924+0000","last_became_peered":"2024-07-31T14:47:20.451924+0000","last_unstale":"2024-07-31T14:47:21.483666+0000","last_undegraded":"2024-07-31T14:47:21.483666+0000","last_fullsized":"2024-07-31T14:47:21.483666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:58:24.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449176+0000","last_change":"2024-07-31T14:47:03.056024+0000","last_active":"2024-07-31T14:47:20.449176+0000","last_peered":"2024-07-31T14:47:20.449176+0000","last_clean":"2024-07-31T14:47:20.449176+0000","last_became_active":"2024-07-31T14:47:03.055630+0000","last_became_peered":"2024-07-31T14:47:03.055630+0000","last_unstale":"2024-07-31T14:47:20.449176+0000","last_undegraded":"2024-07-31T14:47:20.449176+0000","last_fullsized":"2024-07-31T14:47:20.449176+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:09:11.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449155+0000","last_change":"2024-07-31T14:47:03.053718+0000","last_active":"2024-07-31T14:47:20.449155+0000","last_peered":"2024-07-31T14:47:20.449155+0000","last_clean":"2024-07-31T14:47:20.449155+0000","last_became_active":"2024-07-31T14:47:03.053361+0000","last_became_peered":"2024-07-31T14:47:03.053361+0000","last_unstale":"2024-07-31T14:47:20.449155+0000","last_undegraded":"2024-07-31T14:47:20.449155+0000","last_fullsized":"2024-07-31T14:47:20.449155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:03:49.220838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447206+0000","last_change":"2024-07-31T14:47:03.056087+0000","last_active":"2024-07-31T14:47:20.447206+0000","last_peered":"2024-07-31T14:47:20.447206+0000","last_clean":"2024-07-31T14:47:20.447206+0000","last_became_active":"2024-07-31T14:47:03.055732+0000","last_became_peered":"2024-07-31T14:47:03.055732+0000","last_unstale":"2024-07-31T14:47:20.447206+0000","last_undegraded":"2024-07-31T14:47:20.447206+0000","last_fullsized":"2024-07-31T14:47:20.447206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:45:01.185666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71872,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1075,"num_read_kb":614,"num_write":250,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":215334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645360,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":519730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841588,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37016,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708136,"statfs":{"total":95995035648,"available":95957131264,"internally_reserved":0,"allocated":1720320,"data_stored":1171073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005114,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":523756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201343,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":520038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397571,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1117317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561097,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":515555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757326,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1112117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920851,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":519629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117079,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":514872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378716,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":580941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542241,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":580721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705765,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":516620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-31T14:49:50.298 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-31T14:49:50.298 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-07-31T14:49:50.299 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-07-31T14:49:50.299 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph health --format=json 2024-07-31T14:49:50.567 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:51.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:50 smithi067 ceph-mon[32750]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-31T14:49:51.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:50 smithi067 ceph-mon[32750]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:49:51.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:50 smithi067 ceph-mon[32750]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-07-31T14:49:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:50 smithi077 ceph-mon[37327]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-31T14:49:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:50 smithi077 ceph-mon[37327]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:49:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:50 smithi077 ceph-mon[37327]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-07-31T14:49:51.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:50 smithi073 ceph-mon[36714]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-31T14:49:51.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:50 smithi073 ceph-mon[36714]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:49:51.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:50 smithi073 ceph-mon[36714]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-07-31T14:49:52.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.360+0000 7fe42a790640 1 -- 172.21.15.67:0/4220690626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4241024b0 msgr2=0x7fe4241028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:52.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.360+0000 7fe42a790640 1 --2- 172.21.15.67:0/4220690626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4241024b0 0x7fe4241028c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe414009900 tx=0x7fe41402a3f0 comp rx=0 tx=0).stop 2024-07-31T14:49:52.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.360+0000 7fe42a790640 1 -- 172.21.15.67:0/4220690626 shutdown_connections 2024-07-31T14:49:52.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.360+0000 7fe42a790640 1 --2- 172.21.15.67:0/4220690626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe424108740 0x7fe42410ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:52.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.360+0000 7fe42a790640 1 --2- 172.21.15.67:0/4220690626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe424107df0 0x7fe424108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:52.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.360+0000 7fe42a790640 1 --2- 172.21.15.67:0/4220690626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4241024b0 0x7fe4241028c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:52.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.360+0000 7fe42a790640 1 -- 172.21.15.67:0/4220690626 >> 172.21.15.67:0/4220690626 conn(0x7fe424079170 msgr2=0x7fe424079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:52.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.360+0000 7fe42a790640 1 -- 172.21.15.67:0/4220690626 shutdown_connections 2024-07-31T14:49:52.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.360+0000 7fe42a790640 1 -- 172.21.15.67:0/4220690626 wait complete. 2024-07-31T14:49:52.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.361+0000 7fe42a790640 1 Processor -- start 2024-07-31T14:49:52.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.361+0000 7fe42a790640 1 -- start start 2024-07-31T14:49:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.361+0000 7fe42a790640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4241024b0 0x7fe424198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.361+0000 7fe42a790640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe424107df0 0x7fe424199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.361+0000 7fe42a790640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe424108740 0x7fe424199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.361+0000 7fe42a790640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241123a0 con 0x7fe4241024b0 2024-07-31T14:49:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.361+0000 7fe42a790640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe424112220 con 0x7fe424107df0 2024-07-31T14:49:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.361+0000 7fe42a790640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe424112520 con 0x7fe424108740 2024-07-31T14:49:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.361+0000 7fe423fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4241024b0 0x7fe424198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.361+0000 7fe423fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4241024b0 0x7fe424198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48664/0 (socket says 172.21.15.67:48664) 2024-07-31T14:49:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.361+0000 7fe423fff640 1 -- 172.21.15.67:0/2989839012 learned_addr learned my addr 172.21.15.67:0/2989839012 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.361+0000 7fe428d2d640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe424108740 0x7fe424199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.362+0000 7fe4237fe640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe424107df0 0x7fe424199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.362+0000 7fe423fff640 1 -- 172.21.15.67:0/2989839012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe424108740 msgr2=0x7fe424199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.362+0000 7fe423fff640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe424108740 0x7fe424199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.362+0000 7fe423fff640 1 -- 172.21.15.67:0/2989839012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe424107df0 msgr2=0x7fe424199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.362+0000 7fe423fff640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe424107df0 0x7fe424199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.362+0000 7fe423fff640 1 -- 172.21.15.67:0/2989839012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4241a8160 con 0x7fe4241024b0 2024-07-31T14:49:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.362+0000 7fe428d2d640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe424108740 0x7fe424199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:49:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.362+0000 7fe423fff640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4241024b0 0x7fe424198ae0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe414005940 tx=0x7fe4140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:52.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.362+0000 7fe4217fa640 1 -- 172.21.15.67:0/2989839012 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41403f030 con 0x7fe4241024b0 2024-07-31T14:49:52.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.362+0000 7fe42a790640 1 -- 172.21.15.67:0/2989839012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4241a8390 con 0x7fe4241024b0 2024-07-31T14:49:52.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.362+0000 7fe4217fa640 1 -- 172.21.15.67:0/2989839012 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe414003cb0 con 0x7fe4241024b0 2024-07-31T14:49:52.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.362+0000 7fe4217fa640 1 -- 172.21.15.67:0/2989839012 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414035070 con 0x7fe4241024b0 2024-07-31T14:49:52.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.362+0000 7fe42a790640 1 -- 172.21.15.67:0/2989839012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4241a88c0 con 0x7fe4241024b0 2024-07-31T14:49:52.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.363+0000 7fe42a790640 1 -- 172.21.15.67:0/2989839012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe424103730 con 0x7fe4241024b0 2024-07-31T14:49:52.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.365+0000 7fe4217fa640 1 -- 172.21.15.67:0/2989839012 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fe414042030 con 0x7fe4241024b0 2024-07-31T14:49:52.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.365+0000 7fe4217fa640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe40c0b3c20 0x7fe40c0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:52.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.366+0000 7fe4237fe640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe40c0b3c20 0x7fe40c0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:52.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.366+0000 7fe4217fa640 1 -- 172.21.15.67:0/2989839012 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fe414104a00 con 0x7fe4241024b0 2024-07-31T14:49:52.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.366+0000 7fe4237fe640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe40c0b3c20 0x7fe40c0b6130 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe418005fd0 tx=0x7fe418005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:52.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.370+0000 7fe4217fa640 1 -- 172.21.15.67:0/2989839012 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe4140cbc20 con 0x7fe4241024b0 2024-07-31T14:49:52.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.606+0000 7fe42a790640 1 -- 172.21.15.67:0/2989839012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe42410a510 con 0x7fe4241024b0 2024-07-31T14:49:52.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.606+0000 7fe4217fa640 1 -- 172.21.15.67:0/2989839012 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe4140cb520 con 0x7fe4241024b0 2024-07-31T14:49:52.608 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:49:52.608 INFO:teuthology.orchestra.run.smithi067.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-07-31T14:49:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.609+0000 7fe42a790640 1 -- 172.21.15.67:0/2989839012 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe40c0b3c20 msgr2=0x7fe40c0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.609+0000 7fe42a790640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe40c0b3c20 0x7fe40c0b6130 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe418005fd0 tx=0x7fe418005850 comp rx=0 tx=0).stop 2024-07-31T14:49:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.609+0000 7fe42a790640 1 -- 172.21.15.67:0/2989839012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4241024b0 msgr2=0x7fe424198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.609+0000 7fe42a790640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4241024b0 0x7fe424198ae0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe414005940 tx=0x7fe4140042f0 comp rx=0 tx=0).stop 2024-07-31T14:49:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.609+0000 7fe42a790640 1 -- 172.21.15.67:0/2989839012 shutdown_connections 2024-07-31T14:49:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.609+0000 7fe42a790640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe40c0b3c20 0x7fe40c0b6130 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.609+0000 7fe42a790640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe424108740 0x7fe424199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.609+0000 7fe42a790640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe424107df0 0x7fe424199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.609+0000 7fe42a790640 1 --2- 172.21.15.67:0/2989839012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4241024b0 0x7fe424198ae0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.609+0000 7fe42a790640 1 -- 172.21.15.67:0/2989839012 >> 172.21.15.67:0/2989839012 conn(0x7fe424079170 msgr2=0x7fe424100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.609+0000 7fe42a790640 1 -- 172.21.15.67:0/2989839012 shutdown_connections 2024-07-31T14:49:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:52.609+0000 7fe42a790640 1 -- 172.21.15.67:0/2989839012 wait complete. 2024-07-31T14:49:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:52 smithi077 ceph-mon[37327]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-07-31T14:49:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:52 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2989839012' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-31T14:49:53.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:52 smithi067 ceph-mon[32750]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-07-31T14:49:53.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:52 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2989839012' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-31T14:49:53.378 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-07-31T14:49:53.379 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:49:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:52 smithi073 ceph-mon[36714]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-07-31T14:49:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:52 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2989839012' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-31T14:49:53.643 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:55.037 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:54 smithi067 ceph-mon[32750]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-07-31T14:49:55.195 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:54 smithi073 ceph-mon[36714]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-07-31T14:49:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:54 smithi077 ceph-mon[37327]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-07-31T14:49:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.426+0000 7f84a0efe640 1 -- 172.21.15.67:0/1915200109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c109850 msgr2=0x7f849c110080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.426+0000 7f84a0efe640 1 --2- 172.21.15.67:0/1915200109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c109850 0x7f849c110080 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8494009900 tx=0x7f849402a410 comp rx=0 tx=0).stop 2024-07-31T14:49:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.427+0000 7f84a0efe640 1 -- 172.21.15.67:0/1915200109 shutdown_connections 2024-07-31T14:49:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.427+0000 7f84a0efe640 1 --2- 172.21.15.67:0/1915200109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c109850 0x7f849c110080 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.427+0000 7f84a0efe640 1 --2- 172.21.15.67:0/1915200109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f849c108f00 0x7f849c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.427+0000 7f84a0efe640 1 --2- 172.21.15.67:0/1915200109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f849c1035c0 0x7f849c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.427+0000 7f84a0efe640 1 -- 172.21.15.67:0/1915200109 >> 172.21.15.67:0/1915200109 conn(0x7f849c0ff030 msgr2=0x7f849c101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.427+0000 7f84a0efe640 1 -- 172.21.15.67:0/1915200109 shutdown_connections 2024-07-31T14:49:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.427+0000 7f84a0efe640 1 -- 172.21.15.67:0/1915200109 wait complete. 2024-07-31T14:49:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.428+0000 7f84a0efe640 1 Processor -- start 2024-07-31T14:49:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.428+0000 7f84a0efe640 1 -- start start 2024-07-31T14:49:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.428+0000 7f84a0efe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f849c1035c0 0x7f849c199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.428+0000 7f84a0efe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f849c108f00 0x7f849c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.428+0000 7f84a0efe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c109850 0x7f849c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.428+0000 7f84a0efe640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f849c113520 con 0x7f849c109850 2024-07-31T14:49:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.428+0000 7f84a0efe640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1133a0 con 0x7f849c1035c0 2024-07-31T14:49:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.428+0000 7f84a0efe640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1136a0 con 0x7f849c108f00 2024-07-31T14:49:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.428+0000 7f849ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c109850 0x7f849c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.428+0000 7f849ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c109850 0x7f849c19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48684/0 (socket says 172.21.15.67:48684) 2024-07-31T14:49:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.428+0000 7f849ad9d640 1 -- 172.21.15.67:0/3750608185 learned_addr learned my addr 172.21.15.67:0/3750608185 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.428+0000 7f8499d9b640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f849c108f00 0x7f849c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:55.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.428+0000 7f849a59c640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f849c1035c0 0x7f849c199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:55.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.429+0000 7f849ad9d640 1 -- 172.21.15.67:0/3750608185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f849c108f00 msgr2=0x7f849c19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:55.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.429+0000 7f849ad9d640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f849c108f00 0x7f849c19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:55.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.429+0000 7f849ad9d640 1 -- 172.21.15.67:0/3750608185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f849c1035c0 msgr2=0x7f849c199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:55.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.429+0000 7f849ad9d640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f849c1035c0 0x7f849c199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:55.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.429+0000 7f849ad9d640 1 -- 172.21.15.67:0/3750608185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c19fda0 con 0x7f849c109850 2024-07-31T14:49:55.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.429+0000 7f849a59c640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f849c1035c0 0x7f849c199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:49:55.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.429+0000 7f849ad9d640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c109850 0x7f849c19f670 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8494004770 tx=0x7f8494004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:55.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.429+0000 7f84937fe640 1 -- 172.21.15.67:0/3750608185 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849403f030 con 0x7f849c109850 2024-07-31T14:49:55.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.429+0000 7f84a0efe640 1 -- 172.21.15.67:0/3750608185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849c1a0030 con 0x7f849c109850 2024-07-31T14:49:55.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.429+0000 7f84937fe640 1 -- 172.21.15.67:0/3750608185 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8494003aa0 con 0x7f849c109850 2024-07-31T14:49:55.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.429+0000 7f84937fe640 1 -- 172.21.15.67:0/3750608185 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494035070 con 0x7f849c109850 2024-07-31T14:49:55.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.429+0000 7f84a0efe640 1 -- 172.21.15.67:0/3750608185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849c1a67f0 con 0x7f849c109850 2024-07-31T14:49:55.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.430+0000 7f84a0efe640 1 -- 172.21.15.67:0/3750608185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f849c1039d0 con 0x7f849c109850 2024-07-31T14:49:55.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.432+0000 7f84937fe640 1 -- 172.21.15.67:0/3750608185 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f8494042030 con 0x7f849c109850 2024-07-31T14:49:55.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.433+0000 7f84937fe640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f84880b3ca0 0x7f84880b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:55.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.433+0000 7f849a59c640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f84880b3ca0 0x7f84880b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:55.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.434+0000 7f84937fe640 1 -- 172.21.15.67:0/3750608185 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f84941048f0 con 0x7f849c109850 2024-07-31T14:49:55.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.434+0000 7f849a59c640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f84880b3ca0 0x7f84880b61b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f848c005fd0 tx=0x7f848c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:55.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.437+0000 7f84937fe640 1 -- 172.21.15.67:0/3750608185 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f84940cbb10 con 0x7f849c109850 2024-07-31T14:49:55.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.670+0000 7f84a0efe640 1 -- 172.21.15.67:0/3750608185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f849c10b640 con 0x7f849c109850 2024-07-31T14:49:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.671+0000 7f84937fe640 1 -- 172.21.15.67:0/3750608185 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11962 (secure 0 0 0) 0x7f84940cb410 con 0x7f849c109850 2024-07-31T14:49:55.673 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:49:55.673 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-07-31T14:48:06:596011+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:48:06.435900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T14:49:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-07-31T14:49:55.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.674+0000 7f84a0efe640 1 -- 172.21.15.67:0/3750608185 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f84880b3ca0 msgr2=0x7f84880b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:55.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.674+0000 7f84a0efe640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f84880b3ca0 0x7f84880b61b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f848c005fd0 tx=0x7f848c009500 comp rx=0 tx=0).stop 2024-07-31T14:49:55.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.674+0000 7f84a0efe640 1 -- 172.21.15.67:0/3750608185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c109850 msgr2=0x7f849c19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:55.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.674+0000 7f84a0efe640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c109850 0x7f849c19f670 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8494004770 tx=0x7f8494004270 comp rx=0 tx=0).stop 2024-07-31T14:49:55.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.675+0000 7f84a0efe640 1 -- 172.21.15.67:0/3750608185 shutdown_connections 2024-07-31T14:49:55.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.675+0000 7f84a0efe640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f84880b3ca0 0x7f84880b61b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:55.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.675+0000 7f84a0efe640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c109850 0x7f849c19f670 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:55.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.675+0000 7f84a0efe640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f849c108f00 0x7f849c19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:55.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.675+0000 7f84a0efe640 1 --2- 172.21.15.67:0/3750608185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f849c1035c0 0x7f849c199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:55.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.675+0000 7f84a0efe640 1 -- 172.21.15.67:0/3750608185 >> 172.21.15.67:0/3750608185 conn(0x7f849c0ff030 msgr2=0x7f849c101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:55.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.675+0000 7f84a0efe640 1 -- 172.21.15.67:0/3750608185 shutdown_connections 2024-07-31T14:49:55.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:55.675+0000 7f84a0efe640 1 -- 172.21.15.67:0/3750608185 wait complete. 2024-07-31T14:49:56.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:55 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3750608185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:49:56.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:55 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3750608185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:49:56.381 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:49:56.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:55 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3750608185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:49:56.650 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:49:57.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:56 smithi067 ceph-mon[32750]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-07-31T14:49:57.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:56 smithi077 ceph-mon[37327]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-07-31T14:49:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:56 smithi073 ceph-mon[36714]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-07-31T14:49:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.313+0000 7f590072a640 1 -- 172.21.15.67:0/3031713930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f8103590 msgr2=0x7f58f81039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.313+0000 7f590072a640 1 --2- 172.21.15.67:0/3031713930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f8103590 0x7f58f81039a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009900 tx=0x7f58f402a410 comp rx=0 tx=0).stop 2024-07-31T14:49:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.313+0000 7f590072a640 1 -- 172.21.15.67:0/3031713930 shutdown_connections 2024-07-31T14:49:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.313+0000 7f590072a640 1 --2- 172.21.15.67:0/3031713930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58f8109820 0x7f58f8110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.313+0000 7f590072a640 1 --2- 172.21.15.67:0/3031713930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58f8108ed0 0x7f58f81092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:58.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.313+0000 7f590072a640 1 --2- 172.21.15.67:0/3031713930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f8103590 0x7f58f81039a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.313+0000 7f590072a640 1 -- 172.21.15.67:0/3031713930 >> 172.21.15.67:0/3031713930 conn(0x7f58f807b280 msgr2=0x7f58f807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.313+0000 7f590072a640 1 -- 172.21.15.67:0/3031713930 shutdown_connections 2024-07-31T14:49:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.313+0000 7f590072a640 1 -- 172.21.15.67:0/3031713930 wait complete. 2024-07-31T14:49:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.314+0000 7f590072a640 1 Processor -- start 2024-07-31T14:49:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.314+0000 7f590072a640 1 -- start start 2024-07-31T14:49:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.314+0000 7f590072a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58f8103590 0x7f58f8199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.314+0000 7f590072a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58f8108ed0 0x7f58f819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.314+0000 7f590072a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f8109820 0x7f58f819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.314+0000 7f590072a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81134f0 con 0x7f58f8109820 2024-07-31T14:49:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.314+0000 7f590072a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f58f8113370 con 0x7f58f8103590 2024-07-31T14:49:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.314+0000 7f590072a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58f8113670 con 0x7f58f8108ed0 2024-07-31T14:49:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.314+0000 7f58fecc7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f8109820 0x7f58f819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:58.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.314+0000 7f58fecc7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f8109820 0x7f58f819f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48700/0 (socket says 172.21.15.67:48700) 2024-07-31T14:49:58.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.314+0000 7f58fecc7640 1 -- 172.21.15.67:0/745581586 learned_addr learned my addr 172.21.15.67:0/745581586 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:49:58.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.315+0000 7f58fdcc5640 1 --2- 172.21.15.67:0/745581586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58f8108ed0 0x7f58f819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:58.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.315+0000 7f58fe4c6640 1 --2- 172.21.15.67:0/745581586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58f8103590 0x7f58f8199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:58.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.315+0000 7f58fecc7640 1 -- 172.21.15.67:0/745581586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58f8108ed0 msgr2=0x7f58f819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:58.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.315+0000 7f58fecc7640 1 --2- 172.21.15.67:0/745581586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58f8108ed0 0x7f58f819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:58.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.315+0000 7f58fecc7640 1 -- 172.21.15.67:0/745581586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58f8103590 msgr2=0x7f58f8199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:58.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.315+0000 7f58fecc7640 1 --2- 172.21.15.67:0/745581586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58f8103590 0x7f58f8199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:58.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.315+0000 7f58fecc7640 1 -- 172.21.15.67:0/745581586 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f819feb0 con 0x7f58f8109820 2024-07-31T14:49:58.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.315+0000 7f58fecc7640 1 --2- 172.21.15.67:0/745581586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f8109820 0x7f58f819f780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009fd0 tx=0x7f58f000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:58.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.315+0000 7f58ef7fe640 1 -- 172.21.15.67:0/745581586 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0017070 con 0x7f58f8109820 2024-07-31T14:49:58.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.315+0000 7f590072a640 1 -- 172.21.15.67:0/745581586 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f81a01a0 con 0x7f58f8109820 2024-07-31T14:49:58.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.315+0000 7f58ef7fe640 1 -- 172.21.15.67:0/745581586 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f0003e20 con 0x7f58f8109820 2024-07-31T14:49:58.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.315+0000 7f58ef7fe640 1 -- 172.21.15.67:0/745581586 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0010460 con 0x7f58f8109820 2024-07-31T14:49:58.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.315+0000 7f590072a640 1 -- 172.21.15.67:0/745581586 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f81a6900 con 0x7f58f8109820 2024-07-31T14:49:58.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.316+0000 7f590072a640 1 -- 172.21.15.67:0/745581586 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f81039a0 con 0x7f58f8109820 2024-07-31T14:49:58.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.318+0000 7f58ef7fe640 1 -- 172.21.15.67:0/745581586 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f58f0003970 con 0x7f58f8109820 2024-07-31T14:49:58.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.319+0000 7f58ef7fe640 1 --2- 172.21.15.67:0/745581586 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58e40b3c20 0x7f58e40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:49:58.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.319+0000 7f58fe4c6640 1 --2- 172.21.15.67:0/745581586 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58e40b3c20 0x7f58e40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:49:58.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.319+0000 7f58ef7fe640 1 -- 172.21.15.67:0/745581586 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f58f00e3e10 con 0x7f58f8109820 2024-07-31T14:49:58.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.319+0000 7f58fe4c6640 1 --2- 172.21.15.67:0/745581586 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58e40b3c20 0x7f58e40b6130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f58f4031440 tx=0x7f58f4005d50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:49:58.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.322+0000 7f58ef7fe640 1 -- 172.21.15.67:0/745581586 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f58f00ab030 con 0x7f58f8109820 2024-07-31T14:49:58.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.556+0000 7f590072a640 1 -- 172.21.15.67:0/745581586 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58f810b600 con 0x7f58f8109820 2024-07-31T14:49:58.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.557+0000 7f58ef7fe640 1 -- 172.21.15.67:0/745581586 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11962 (secure 0 0 0) 0x7f58f00aa930 con 0x7f58f8109820 2024-07-31T14:49:58.558 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:49:58.558 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-07-31T14:48:06:596011+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:48:06.435900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T14:49:58.559 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-07-31T14:49:58.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.559+0000 7f590072a640 1 -- 172.21.15.67:0/745581586 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58e40b3c20 msgr2=0x7f58e40b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:58.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.559+0000 7f590072a640 1 --2- 172.21.15.67:0/745581586 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58e40b3c20 0x7f58e40b6130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f58f4031440 tx=0x7f58f4005d50 comp rx=0 tx=0).stop 2024-07-31T14:49:58.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.559+0000 7f590072a640 1 -- 172.21.15.67:0/745581586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f8109820 msgr2=0x7f58f819f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:49:58.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.559+0000 7f590072a640 1 --2- 172.21.15.67:0/745581586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f8109820 0x7f58f819f780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009fd0 tx=0x7f58f000b260 comp rx=0 tx=0).stop 2024-07-31T14:49:58.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.559+0000 7f590072a640 1 -- 172.21.15.67:0/745581586 shutdown_connections 2024-07-31T14:49:58.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.559+0000 7f590072a640 1 --2- 172.21.15.67:0/745581586 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58e40b3c20 0x7f58e40b6130 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:58.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.559+0000 7f590072a640 1 --2- 172.21.15.67:0/745581586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f8109820 0x7f58f819f780 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:58.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.559+0000 7f590072a640 1 --2- 172.21.15.67:0/745581586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58f8108ed0 0x7f58f819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:58.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.559+0000 7f590072a640 1 --2- 172.21.15.67:0/745581586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58f8103590 0x7f58f8199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:49:58.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.559+0000 7f590072a640 1 -- 172.21.15.67:0/745581586 >> 172.21.15.67:0/745581586 conn(0x7f58f807b280 msgr2=0x7f58f8107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:49:58.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.559+0000 7f590072a640 1 -- 172.21.15.67:0/745581586 shutdown_connections 2024-07-31T14:49:58.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:49:58.559+0000 7f590072a640 1 -- 172.21.15.67:0/745581586 wait complete. 2024-07-31T14:49:59.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:58 smithi067 ceph-mon[32750]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-31T14:49:59.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:49:58 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/745581586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:49:59.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-07-31T14:49:59.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:58 smithi077 ceph-mon[37327]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-31T14:49:59.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:49:58 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/745581586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:49:59.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:58 smithi073 ceph-mon[36714]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-31T14:49:59.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:49:58 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/745581586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:49:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:01.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:00 smithi067 ceph-mon[32750]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-31T14:50:01.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:00 smithi067 ceph-mon[32750]: overall HEALTH_OK 2024-07-31T14:50:01.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:00 smithi077 ceph-mon[37327]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-31T14:50:01.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:00 smithi077 ceph-mon[37327]: overall HEALTH_OK 2024-07-31T14:50:01.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.266+0000 7f4a136ac640 1 -- 172.21.15.67:0/2442692918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c108720 msgr2=0x7f4a0c10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:01.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.266+0000 7f4a136ac640 1 --2- 172.21.15.67:0/2442692918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c108720 0x7f4a0c10ef50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009ca0 tx=0x7f4a0402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:50:01.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.266+0000 7f4a136ac640 1 -- 172.21.15.67:0/2442692918 shutdown_connections 2024-07-31T14:50:01.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.266+0000 7f4a136ac640 1 --2- 172.21.15.67:0/2442692918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c108720 0x7f4a0c10ef50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:01.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.266+0000 7f4a136ac640 1 --2- 172.21.15.67:0/2442692918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c107dd0 0x7f4a0c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:01.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.266+0000 7f4a136ac640 1 --2- 172.21.15.67:0/2442692918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a0c102490 0x7f4a0c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:01.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.266+0000 7f4a136ac640 1 -- 172.21.15.67:0/2442692918 >> 172.21.15.67:0/2442692918 conn(0x7f4a0c079160 msgr2=0x7f4a0c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:01.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.266+0000 7f4a136ac640 1 -- 172.21.15.67:0/2442692918 shutdown_connections 2024-07-31T14:50:01.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.266+0000 7f4a136ac640 1 -- 172.21.15.67:0/2442692918 wait complete. 2024-07-31T14:50:01.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.267+0000 7f4a136ac640 1 Processor -- start 2024-07-31T14:50:01.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.267+0000 7f4a136ac640 1 -- start start 2024-07-31T14:50:01.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.267+0000 7f4a136ac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c102490 0x7f4a0c19c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:01.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.267+0000 7f4a136ac640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c107dd0 0x7f4a0c19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:01.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.267+0000 7f4a136ac640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a0c108720 0x7f4a0c1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:01.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.267+0000 7f4a136ac640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1134a0 con 0x7f4a0c107dd0 2024-07-31T14:50:01.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a136ac640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c113320 con 0x7f4a0c108720 2024-07-31T14:50:01.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a136ac640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c113620 con 0x7f4a0c102490 2024-07-31T14:50:01.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a10c47640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c107dd0 0x7f4a0c19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:01.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a10c47640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c107dd0 0x7f4a0c19c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38544/0 (socket says 172.21.15.67:38544) 2024-07-31T14:50:01.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a10c47640 1 -- 172.21.15.67:0/3903492282 learned_addr learned my addr 172.21.15.67:0/3903492282 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:01.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a11c49640 1 --2- 172.21.15.67:0/3903492282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a0c108720 0x7f4a0c1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:01.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a11448640 1 --2- 172.21.15.67:0/3903492282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c102490 0x7f4a0c19c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:01.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a10c47640 1 -- 172.21.15.67:0/3903492282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c102490 msgr2=0x7f4a0c19c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:01.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a10c47640 1 --2- 172.21.15.67:0/3903492282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c102490 0x7f4a0c19c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:01.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a10c47640 1 -- 172.21.15.67:0/3903492282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a0c108720 msgr2=0x7f4a0c1a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:01.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a10c47640 1 --2- 172.21.15.67:0/3903492282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a0c108720 0x7f4a0c1a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:01.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a10c47640 1 -- 172.21.15.67:0/3903492282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c1a1f40 con 0x7f4a0c107dd0 2024-07-31T14:50:01.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a10c47640 1 --2- 172.21.15.67:0/3903492282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c107dd0 0x7f4a0c19c630 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00d740 tx=0x7f49fc00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:01.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a027fc640 1 -- 172.21.15.67:0/3903492282 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc017070 con 0x7f4a0c107dd0 2024-07-31T14:50:01.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a027fc640 1 -- 172.21.15.67:0/3903492282 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49fc009820 con 0x7f4a0c107dd0 2024-07-31T14:50:01.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a136ac640 1 -- 172.21.15.67:0/3903492282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c1a2230 con 0x7f4a0c107dd0 2024-07-31T14:50:01.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a027fc640 1 -- 172.21.15.67:0/3903492282 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc004ce0 con 0x7f4a0c107dd0 2024-07-31T14:50:01.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.268+0000 7f4a136ac640 1 -- 172.21.15.67:0/3903492282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c1a8a50 con 0x7f4a0c107dd0 2024-07-31T14:50:01.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.270+0000 7f4a136ac640 1 -- 172.21.15.67:0/3903492282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0c1028a0 con 0x7f4a0c107dd0 2024-07-31T14:50:01.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.271+0000 7f4a027fc640 1 -- 172.21.15.67:0/3903492282 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f49fc005220 con 0x7f4a0c107dd0 2024-07-31T14:50:01.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.272+0000 7f4a027fc640 1 --2- 172.21.15.67:0/3903492282 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f49f80b3cd0 0x7f49f80b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:01.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.272+0000 7f4a11448640 1 --2- 172.21.15.67:0/3903492282 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f49f80b3cd0 0x7f49f80b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:01.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.272+0000 7f4a027fc640 1 -- 172.21.15.67:0/3903492282 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f49fc0e3cd0 con 0x7f4a0c107dd0 2024-07-31T14:50:01.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.272+0000 7f4a11448640 1 --2- 172.21.15.67:0/3903492282 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f49f80b3cd0 0x7f49f80b61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800abe0 tx=0x7f4a0800a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:01.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.275+0000 7f4a027fc640 1 -- 172.21.15.67:0/3903492282 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f49fc0aaf70 con 0x7f4a0c107dd0 2024-07-31T14:50:01.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:00 smithi073 ceph-mon[36714]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-31T14:50:01.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:00 smithi073 ceph-mon[36714]: overall HEALTH_OK 2024-07-31T14:50:01.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.470+0000 7f4a136ac640 1 -- 172.21.15.67:0/3903492282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a0c1a8c80 con 0x7f4a0c107dd0 2024-07-31T14:50:01.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.472+0000 7f4a027fc640 1 -- 172.21.15.67:0/3903492282 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24874 (secure 0 0 0) 0x7f49fc0aa870 con 0x7f4a0c107dd0 2024-07-31T14:50:01.473 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:50:01.473 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T14:47:20.440064+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T14:50:01.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.474+0000 7f4a136ac640 1 -- 172.21.15.67:0/3903492282 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f49f80b3cd0 msgr2=0x7f49f80b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:01.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.474+0000 7f4a136ac640 1 --2- 172.21.15.67:0/3903492282 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f49f80b3cd0 0x7f49f80b61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800abe0 tx=0x7f4a0800a2d0 comp rx=0 tx=0).stop 2024-07-31T14:50:01.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.474+0000 7f4a136ac640 1 -- 172.21.15.67:0/3903492282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c107dd0 msgr2=0x7f4a0c19c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:01.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.474+0000 7f4a136ac640 1 --2- 172.21.15.67:0/3903492282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c107dd0 0x7f4a0c19c630 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00d740 tx=0x7f49fc00db40 comp rx=0 tx=0).stop 2024-07-31T14:50:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.474+0000 7f4a136ac640 1 -- 172.21.15.67:0/3903492282 shutdown_connections 2024-07-31T14:50:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.474+0000 7f4a136ac640 1 --2- 172.21.15.67:0/3903492282 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f49f80b3cd0 0x7f49f80b61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.474+0000 7f4a136ac640 1 --2- 172.21.15.67:0/3903492282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a0c108720 0x7f4a0c1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.474+0000 7f4a136ac640 1 --2- 172.21.15.67:0/3903492282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c107dd0 0x7f4a0c19c630 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.474+0000 7f4a136ac640 1 --2- 172.21.15.67:0/3903492282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a0c102490 0x7f4a0c19c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.474+0000 7f4a136ac640 1 -- 172.21.15.67:0/3903492282 >> 172.21.15.67:0/3903492282 conn(0x7f4a0c079160 msgr2=0x7f4a0c103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.474+0000 7f4a136ac640 1 -- 172.21.15.67:0/3903492282 shutdown_connections 2024-07-31T14:50:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:01.474+0000 7f4a136ac640 1 -- 172.21.15.67:0/3903492282 wait complete. 2024-07-31T14:50:02.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:01 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3903492282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:50:02.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:01 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3903492282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:50:02.253 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:50:02.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:01 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3903492282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:50:02.523 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:03.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:02 smithi077 ceph-mon[37327]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-07-31T14:50:03.302 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:02 smithi067 ceph-mon[32750]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-07-31T14:50:03.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:02 smithi073 ceph-mon[36714]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-07-31T14:50:04.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.286+0000 7f9a5d827640 1 -- 172.21.15.67:0/4153891309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a58107dd0 msgr2=0x7f9a581081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:04.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.286+0000 7f9a5d827640 1 --2- 172.21.15.67:0/4153891309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a58107dd0 0x7f9a581081e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009900 tx=0x7f9a4002a410 comp rx=0 tx=0).stop 2024-07-31T14:50:04.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.286+0000 7f9a5d827640 1 -- 172.21.15.67:0/4153891309 shutdown_connections 2024-07-31T14:50:04.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.286+0000 7f9a5d827640 1 --2- 172.21.15.67:0/4153891309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a58108720 0x7f9a5810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:04.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.286+0000 7f9a5d827640 1 --2- 172.21.15.67:0/4153891309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a58107dd0 0x7f9a581081e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:04.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.286+0000 7f9a5d827640 1 --2- 172.21.15.67:0/4153891309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58102490 0x7f9a581028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:04.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.286+0000 7f9a5d827640 1 -- 172.21.15.67:0/4153891309 >> 172.21.15.67:0/4153891309 conn(0x7f9a58079160 msgr2=0x7f9a58079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:04.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.286+0000 7f9a5d827640 1 -- 172.21.15.67:0/4153891309 shutdown_connections 2024-07-31T14:50:04.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.286+0000 7f9a5d827640 1 -- 172.21.15.67:0/4153891309 wait complete. 2024-07-31T14:50:04.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.287+0000 7f9a5d827640 1 Processor -- start 2024-07-31T14:50:04.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.287+0000 7f9a5d827640 1 -- start start 2024-07-31T14:50:04.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.287+0000 7f9a5d827640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a58102490 0x7f9a5819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:04.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.287+0000 7f9a5d827640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a58107dd0 0x7f9a5819b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:04.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.287+0000 7f9a5d827640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58108720 0x7f9a5819b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:04.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.287+0000 7f9a5d827640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58112370 con 0x7f9a58102490 2024-07-31T14:50:04.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.287+0000 7f9a5d827640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581121f0 con 0x7f9a58107dd0 2024-07-31T14:50:04.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.287+0000 7f9a5d827640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581124f0 con 0x7f9a58108720 2024-07-31T14:50:04.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.287+0000 7f9a56ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a58102490 0x7f9a5819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:04.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.287+0000 7f9a56ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a58102490 0x7f9a5819ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38552/0 (socket says 172.21.15.67:38552) 2024-07-31T14:50:04.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.287+0000 7f9a56ffd640 1 -- 172.21.15.67:0/1263654076 learned_addr learned my addr 172.21.15.67:0/1263654076 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:04.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.288+0000 7f9a577fe640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58108720 0x7f9a5819b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:04.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.288+0000 7f9a567fc640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a58107dd0 0x7f9a5819b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:04.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.288+0000 7f9a56ffd640 1 -- 172.21.15.67:0/1263654076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58108720 msgr2=0x7f9a5819b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:04.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.288+0000 7f9a56ffd640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58108720 0x7f9a5819b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:04.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.288+0000 7f9a56ffd640 1 -- 172.21.15.67:0/1263654076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a58107dd0 msgr2=0x7f9a5819b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:04.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.288+0000 7f9a56ffd640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a58107dd0 0x7f9a5819b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:04.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.288+0000 7f9a56ffd640 1 -- 172.21.15.67:0/1263654076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a581a5fe0 con 0x7f9a58102490 2024-07-31T14:50:04.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.288+0000 7f9a577fe640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58108720 0x7f9a5819b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:50:04.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.288+0000 7f9a56ffd640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a58102490 0x7f9a5819ad20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000d770 tx=0x7f9a5000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:04.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.288+0000 7f9a5c825640 1 -- 172.21.15.67:0/1263654076 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50017070 con 0x7f9a58102490 2024-07-31T14:50:04.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.288+0000 7f9a5d827640 1 -- 172.21.15.67:0/1263654076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a581a62d0 con 0x7f9a58102490 2024-07-31T14:50:04.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.288+0000 7f9a5c825640 1 -- 172.21.15.67:0/1263654076 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a500054d0 con 0x7f9a58102490 2024-07-31T14:50:04.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.288+0000 7f9a5c825640 1 -- 172.21.15.67:0/1263654076 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50004c30 con 0x7f9a58102490 2024-07-31T14:50:04.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.288+0000 7f9a5d827640 1 -- 172.21.15.67:0/1263654076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a581a6790 con 0x7f9a58102490 2024-07-31T14:50:04.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.289+0000 7f9a5d827640 1 -- 172.21.15.67:0/1263654076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a58103710 con 0x7f9a58102490 2024-07-31T14:50:04.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.291+0000 7f9a5c825640 1 -- 172.21.15.67:0/1263654076 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f9a50004dd0 con 0x7f9a58102490 2024-07-31T14:50:04.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.292+0000 7f9a5c825640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a440b3c20 0x7f9a440b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:04.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.292+0000 7f9a567fc640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a440b3c20 0x7f9a440b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:04.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.292+0000 7f9a5c825640 1 -- 172.21.15.67:0/1263654076 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f9a500e4020 con 0x7f9a58102490 2024-07-31T14:50:04.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.293+0000 7f9a567fc640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a440b3c20 0x7f9a440b6130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a40000c00 tx=0x7f9a40031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:04.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.295+0000 7f9a5c825640 1 -- 172.21.15.67:0/1263654076 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f9a500ab240 con 0x7f9a58102490 2024-07-31T14:50:04.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.533+0000 7f9a5d827640 1 -- 172.21.15.67:0/1263654076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a5810a540 con 0x7f9a58102490 2024-07-31T14:50:04.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.533+0000 7f9a5c825640 1 -- 172.21.15.67:0/1263654076 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11962 (secure 0 0 0) 0x7f9a500aab40 con 0x7f9a58102490 2024-07-31T14:50:04.535 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:50:04.535 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-07-31T14:48:06:596011+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:48:06.435900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T14:50:04.535 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-07-31T14:50:04.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.536+0000 7f9a5d827640 1 -- 172.21.15.67:0/1263654076 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a440b3c20 msgr2=0x7f9a440b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:04.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.536+0000 7f9a5d827640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a440b3c20 0x7f9a440b6130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a40000c00 tx=0x7f9a40031400 comp rx=0 tx=0).stop 2024-07-31T14:50:04.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.536+0000 7f9a5d827640 1 -- 172.21.15.67:0/1263654076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a58102490 msgr2=0x7f9a5819ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:04.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.536+0000 7f9a5d827640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a58102490 0x7f9a5819ad20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000d770 tx=0x7f9a5000dc30 comp rx=0 tx=0).stop 2024-07-31T14:50:04.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.536+0000 7f9a5d827640 1 -- 172.21.15.67:0/1263654076 shutdown_connections 2024-07-31T14:50:04.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.536+0000 7f9a5d827640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a440b3c20 0x7f9a440b6130 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:04.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.536+0000 7f9a5d827640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58108720 0x7f9a5819b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:04.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.536+0000 7f9a5d827640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a58107dd0 0x7f9a5819b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:04.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.536+0000 7f9a5d827640 1 --2- 172.21.15.67:0/1263654076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a58102490 0x7f9a5819ad20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:04.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.536+0000 7f9a5d827640 1 -- 172.21.15.67:0/1263654076 >> 172.21.15.67:0/1263654076 conn(0x7f9a58079160 msgr2=0x7f9a58103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:04.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.536+0000 7f9a5d827640 1 -- 172.21.15.67:0/1263654076 shutdown_connections 2024-07-31T14:50:04.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:04.536+0000 7f9a5d827640 1 -- 172.21.15.67:0/1263654076 wait complete. 2024-07-31T14:50:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:04 smithi077 ceph-mon[37327]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:50:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:04 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1263654076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:50:05.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:04 smithi067 ceph-mon[32750]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:50:05.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:04 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1263654076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:50:05.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-31T14:47:04.833142+0000', 'modified': '2024-07-31T14:48:06.435900+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24671, 'mds_1': 24655, 'mds_2': 24650, 'mds_3': 24646, 'mds_4': 24634}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14778': {'gid': 14778, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.67:6839/3532250223', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6838', 'nonce': 3532250223}, {'type': 'v1', 'addr': '172.21.15.67:6839', 'nonce': 3532250223}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14790': {'gid': 14790, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.67:6841/1797875282', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6840', 'nonce': 1797875282}, {'type': 'v1', 'addr': '172.21.15.67:6841', 'nonce': 1797875282}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24602': {'gid': 24602, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6833/1184537619', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6832', 'nonce': 1184537619}, {'type': 'v1', 'addr': '172.21.15.73:6833', 'nonce': 1184537619}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24604': {'gid': 24604, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6833/2710994774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6832', 'nonce': 2710994774}, {'type': 'v1', 'addr': '172.21.15.77:6833', 'nonce': 2710994774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24629': {'gid': 24629, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6835/2854264361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 2854264361}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 2854264361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24634': {'gid': 24634, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.77:6835/1866134372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 1866134372}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 1866134372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24646': {'gid': 24646, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.77:6837/3460575274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6836', 'nonce': 3460575274}, {'type': 'v1', 'addr': '172.21.15.77:6837', 'nonce': 3460575274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24650': {'gid': 24650, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.73:6837/3592232039', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 3592232039}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 3592232039}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24655': {'gid': 24655, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.77:6839/2802806156', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 2802806156}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 2802806156}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24671': {'gid': 24671, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.73:6839/4131046421', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 4131046421}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 4131046421}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24671, 'qdb_cluster': [24634, 24646, 24650, 24655, 24671]} 2024-07-31T14:50:05.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-07-31T14:50:05.345 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.f status 2024-07-31T14:50:05.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:04 smithi073 ceph-mon[36714]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:50:05.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:04 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1263654076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:50:05.612 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:07.026 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:06 smithi067 ceph-mon[32750]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-07-31T14:50:07.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:06 smithi077 ceph-mon[37327]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-07-31T14:50:07.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.382+0000 7ff9dbea2640 1 -- 172.21.15.67:0/2217140987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9d4109820 msgr2=0x7ff9d4110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:07.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.382+0000 7ff9dbea2640 1 --2- 172.21.15.67:0/2217140987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9d4109820 0x7ff9d4110050 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009ca0 tx=0x7ff9cc02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.382+0000 7ff9dbea2640 1 -- 172.21.15.67:0/2217140987 shutdown_connections 2024-07-31T14:50:07.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.382+0000 7ff9dbea2640 1 --2- 172.21.15.67:0/2217140987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9d4109820 0x7ff9d4110050 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.382+0000 7ff9dbea2640 1 --2- 172.21.15.67:0/2217140987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4108ed0 0x7ff9d41092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.382+0000 7ff9dbea2640 1 --2- 172.21.15.67:0/2217140987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d4103590 0x7ff9d41039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.382+0000 7ff9dbea2640 1 -- 172.21.15.67:0/2217140987 >> 172.21.15.67:0/2217140987 conn(0x7ff9d407b280 msgr2=0x7ff9d407b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:07.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.382+0000 7ff9dbea2640 1 -- 172.21.15.67:0/2217140987 shutdown_connections 2024-07-31T14:50:07.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.382+0000 7ff9dbea2640 1 -- 172.21.15.67:0/2217140987 wait complete. 2024-07-31T14:50:07.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9dbea2640 1 Processor -- start 2024-07-31T14:50:07.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9dbea2640 1 -- start start 2024-07-31T14:50:07.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9dbea2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d4103590 0x7ff9d4199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9dbea2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4108ed0 0x7ff9d419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9dbea2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9d4109820 0x7ff9d419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9dbea2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d41134d0 con 0x7ff9d4109820 2024-07-31T14:50:07.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9dbea2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d4113350 con 0x7ff9d4103590 2024-07-31T14:50:07.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9dbea2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d4113650 con 0x7ff9d4108ed0 2024-07-31T14:50:07.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9da43f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9d4109820 0x7ff9d419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9d9c3e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d4103590 0x7ff9d4199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9da43f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9d4109820 0x7ff9d419f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38570/0 (socket says 172.21.15.67:38570) 2024-07-31T14:50:07.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9da43f640 1 -- 172.21.15.67:0/1073460820 learned_addr learned my addr 172.21.15.67:0/1073460820 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:07.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9d943d640 1 --2- 172.21.15.67:0/1073460820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4108ed0 0x7ff9d419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9da43f640 1 -- 172.21.15.67:0/1073460820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4108ed0 msgr2=0x7ff9d419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:07.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9da43f640 1 --2- 172.21.15.67:0/1073460820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4108ed0 0x7ff9d419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9da43f640 1 -- 172.21.15.67:0/1073460820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d4103590 msgr2=0x7ff9d4199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:07.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9da43f640 1 --2- 172.21.15.67:0/1073460820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d4103590 0x7ff9d4199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.383+0000 7ff9da43f640 1 -- 172.21.15.67:0/1073460820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d419fd10 con 0x7ff9d4109820 2024-07-31T14:50:07.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.384+0000 7ff9da43f640 1 --2- 172.21.15.67:0/1073460820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9d4109820 0x7ff9d419f6a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc005cc0 tx=0x7ff9cc02f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:07.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.384+0000 7ff9caffd640 1 -- 172.21.15.67:0/1073460820 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc003cf0 con 0x7ff9d4109820 2024-07-31T14:50:07.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.384+0000 7ff9dbea2640 1 -- 172.21.15.67:0/1073460820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d419ffa0 con 0x7ff9d4109820 2024-07-31T14:50:07.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.384+0000 7ff9caffd640 1 -- 172.21.15.67:0/1073460820 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9cc0044f0 con 0x7ff9d4109820 2024-07-31T14:50:07.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.384+0000 7ff9caffd640 1 -- 172.21.15.67:0/1073460820 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc02fc30 con 0x7ff9d4109820 2024-07-31T14:50:07.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.384+0000 7ff9dbea2640 1 -- 172.21.15.67:0/1073460820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d4075020 con 0x7ff9d4109820 2024-07-31T14:50:07.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.387+0000 7ff9e23f3740 1 Processor -- start 2024-07-31T14:50:07.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.387+0000 7ff9caffd640 1 -- 172.21.15.67:0/1073460820 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7ff9cc0305c0 con 0x7ff9d4109820 2024-07-31T14:50:07.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.387+0000 7ff9e23f3740 1 -- start start 2024-07-31T14:50:07.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.387+0000 7ff9e23f3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933649380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.387+0000 7ff9e23f3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56093327b0a0 0x5609336498c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.387+0000 7ff9e23f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560933283280 0x560933649e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.387+0000 7ff9e23f3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5609335bf920 con 0x5609332a3f60 2024-07-31T14:50:07.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.387+0000 7ff9e23f3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5609335aee20 con 0x56093327b0a0 2024-07-31T14:50:07.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.387+0000 7ff9e23f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5609335b2e60 con 0x560933283280 2024-07-31T14:50:07.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9d943d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933649380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9d9c3e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56093327b0a0 0x5609336498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9d943d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933649380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38574/0 (socket says 172.21.15.67:38574) 2024-07-31T14:50:07.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9d943d640 1 -- 172.21.15.67:0/153181973 learned_addr learned my addr 172.21.15.67:0/153181973 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:07.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9d943d640 1 --2- 172.21.15.67:0/153181973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560933283280 0x560933649e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9caffd640 1 --2- 172.21.15.67:0/1073460820 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff9c00b3c50 0x7ff9c00b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9d9c3e640 1 -- 172.21.15.67:0/153181973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560933283280 msgr2=0x560933649e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:07.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9d9c3e640 1 --2- 172.21.15.67:0/153181973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560933283280 0x560933649e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9d9c3e640 1 -- 172.21.15.67:0/153181973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 msgr2=0x560933649380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:07.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9d9c3e640 1 --2- 172.21.15.67:0/153181973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933649380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9d9c3e640 1 -- 172.21.15.67:0/153181973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560933555a40 con 0x56093327b0a0 2024-07-31T14:50:07.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9d943d640 1 --2- 172.21.15.67:0/153181973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933649380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:50:07.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9da43f640 1 --2- 172.21.15.67:0/1073460820 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff9c00b3c50 0x7ff9c00b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9d9c3e640 1 --2- 172.21.15.67:0/153181973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56093327b0a0 0x5609336498c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000ef80 tx=0x7ff9d000e6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cdcff750e9c6ebb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:07.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9caffd640 1 -- 172.21.15.67:0/1073460820 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff9cc1058b0 con 0x7ff9d4109820 2024-07-31T14:50:07.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.388+0000 7ff9bffff640 1 -- 172.21.15.67:0/153181973 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0007e10 con 0x56093327b0a0 2024-07-31T14:50:07.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.389+0000 7ff9bffff640 1 -- 172.21.15.67:0/153181973 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9d000e2e0 con 0x56093327b0a0 2024-07-31T14:50:07.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.389+0000 7ff9bffff640 1 -- 172.21.15.67:0/153181973 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0004cb0 con 0x56093327b0a0 2024-07-31T14:50:07.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.389+0000 7ff9da43f640 1 --2- 172.21.15.67:0/1073460820 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff9c00b3c50 0x7ff9c00b6160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc102fd0 tx=0x7ff9cc040000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:07.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.389+0000 7ff9e23f3740 1 -- 172.21.15.67:0/153181973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56093327b0a0 msgr2=0x5609336498c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:07.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.389+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/153181973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56093327b0a0 0x5609336498c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000ef80 tx=0x7ff9d000e6e0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.389+0000 7ff9e23f3740 1 -- 172.21.15.67:0/153181973 shutdown_connections 2024-07-31T14:50:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.389+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/153181973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560933283280 0x560933649e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.389+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/153181973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56093327b0a0 0x5609336498c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.389+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/153181973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933649380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.389+0000 7ff9e23f3740 1 -- 172.21.15.67:0/153181973 >> 172.21.15.67:0/153181973 conn(0x560933298350 msgr2=0x560933647270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.390+0000 7ff9e23f3740 1 -- 172.21.15.67:0/153181973 shutdown_connections 2024-07-31T14:50:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.390+0000 7ff9e23f3740 1 -- 172.21.15.67:0/153181973 wait complete. 2024-07-31T14:50:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.390+0000 7ff9e23f3740 1 -- start start 2024-07-31T14:50:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.390+0000 7ff9e23f3740 1 Processor -- start 2024-07-31T14:50:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9e23f3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56093327b0a0 0x560933655040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9e23f3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560933283280 0x5609336555a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9e23f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609332a3f60 0x560933660df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9e23f3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5609334125f0 con 0x560933283280 2024-07-31T14:50:07.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9e23f3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560933551890 con 0x56093327b0a0 2024-07-31T14:50:07.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9e23f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560933657b00 con 0x5609332a3f60 2024-07-31T14:50:07.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9e23f3740 10 client.0 _ll_register_callbacks cb 0x56093355b7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:07.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9d9c3e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560933283280 0x5609336555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9d9c3e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560933283280 0x5609336555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38584/0 (socket says 172.21.15.67:38584) 2024-07-31T14:50:07.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9d9c3e640 1 -- 172.21.15.67:0/3284584717 learned_addr learned my addr 172.21.15.67:0/3284584717 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:07.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9d943d640 1 --2- 172.21.15.67:0/3284584717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609332a3f60 0x560933660df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9d943d640 1 --2- 172.21.15.67:0/3284584717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56093327b0a0 0x560933655040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9d9c3e640 1 -- 172.21.15.67:0/3284584717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609332a3f60 msgr2=0x560933660df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:07.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9d9c3e640 1 --2- 172.21.15.67:0/3284584717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609332a3f60 0x560933660df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9d9c3e640 1 -- 172.21.15.67:0/3284584717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56093327b0a0 msgr2=0x560933655040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:07.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9d9c3e640 1 --2- 172.21.15.67:0/3284584717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56093327b0a0 0x560933655040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9d9c3e640 1 -- 172.21.15.67:0/3284584717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56093330f540 con 0x560933283280 2024-07-31T14:50:07.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9d943d640 1 --2- 172.21.15.67:0/3284584717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56093327b0a0 0x560933655040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:50:07.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9d9c3e640 1 --2- 172.21.15.67:0/3284584717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560933283280 0x5609336555a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0000c00 tx=0x7ff9d000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d684a7ebe6c530 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:07.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.391+0000 7ff9bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:07.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.392+0000 7ff9bd7fa640 1 -- 172.21.15.67:0/3284584717 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0026030 con 0x560933283280 2024-07-31T14:50:07.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.392+0000 7ff9bd7fa640 1 -- 172.21.15.67:0/3284584717 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff9d001d040 con 0x560933283280 2024-07-31T14:50:07.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.392+0000 7ff9e23f3740 1 -- 172.21.15.67:0/3284584717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d000e2e0 con 0x560933283280 2024-07-31T14:50:07.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.392+0000 7ff9bd7fa640 1 -- 172.21.15.67:0/3284584717 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9d0033050 con 0x560933283280 2024-07-31T14:50:07.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.392+0000 7ff9e23f3740 10 client.15246 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:07.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.392+0000 7ff9e23f3740 1 -- 172.21.15.67:0/3284584717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560933663b40 con 0x560933283280 2024-07-31T14:50:07.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.392+0000 7ff9bd7fa640 1 -- 172.21.15.67:0/3284584717 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9d000e2e0 con 0x560933283280 2024-07-31T14:50:07.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.392+0000 7ff9bd7fa640 1 -- 172.21.15.67:0/3284584717 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff9d002d4e0 con 0x560933283280 2024-07-31T14:50:07.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.392+0000 7ff9e23f3740 10 client.15246 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:07.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.392+0000 7ff9e23f3740 10 client.15246 resolve_mds: resolved name 'f' to daemon mds.f 2024-07-31T14:50:07.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.392+0000 7ff9e23f3740 20 client.15246 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:07.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.392+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/3284584717 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5609332628e0 0x5609336671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.392+0000 7ff9e23f3740 4 client.15246 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:50:07.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.393+0000 7ff9e23f3740 1 -- 172.21.15.67:0/3284584717 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609335c1870 con 0x5609332628e0 2024-07-31T14:50:07.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.393+0000 7ff9d943d640 1 --2- 172.21.15.67:0/3284584717 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5609332628e0 0x5609336671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.401 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:06 smithi073 ceph-mon[36714]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-07-31T14:50:07.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.393+0000 7ff9d943d640 1 --2- 172.21.15.67:0/3284584717 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5609332628e0 0x5609336671d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9377a110d802a4b9 server_cookie=1c5d4840eef08489 in_seq=0 out_seq=0 2024-07-31T14:50:07.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.393+0000 7ff9bd7fa640 10 client.15246 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:50:07.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.394+0000 7ff9bd7fa640 1 -- 172.21.15.67:0/3284584717 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609335c1870 con 0x5609332628e0 2024-07-31T14:50:07.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.394+0000 7ff9bd7fa640 10 client.15246 handle_command_reply: tid=0 2024-07-31T14:50:07.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.394+0000 7ff9e23f3740 1 client.15246 shutdown 2024-07-31T14:50:07.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.394+0000 7ff9e23f3740 1 -- 172.21.15.67:0/3284584717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560933283280 msgr2=0x5609336555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:07.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.394+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/3284584717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560933283280 0x5609336555a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0000c00 tx=0x7ff9d000cbc0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.395+0000 7ff9e23f3740 1 -- 172.21.15.67:0/3284584717 shutdown_connections 2024-07-31T14:50:07.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.395+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/3284584717 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5609332628e0 0x5609336671d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.395+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/3284584717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609332a3f60 0x560933660df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.395+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/3284584717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560933283280 0x5609336555a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.395+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/3284584717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56093327b0a0 0x560933655040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.395+0000 7ff9bd7fa640 0 client.15246 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:50:07.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.395+0000 7ff9e23f3740 1 -- 172.21.15.67:0/3284584717 >> 172.21.15.67:0/3284584717 conn(0x560933298350 msgr2=0x56093364c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:07.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.395+0000 7ff9e23f3740 1 -- 172.21.15.67:0/3284584717 shutdown_connections 2024-07-31T14:50:07.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.395+0000 7ff9e23f3740 1 -- 172.21.15.67:0/3284584717 wait complete. 2024-07-31T14:50:07.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.395+0000 7ff9e23f3740 20 client.15246 trim_cache size 0 max 16384 2024-07-31T14:50:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9e23f3740 1 Processor -- start 2024-07-31T14:50:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9e23f3740 1 -- start start 2024-07-31T14:50:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9e23f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56093327b0a0 0x560933666b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9e23f3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560933283280 0x560933660d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9e23f3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933655020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9e23f3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560933665510 con 0x5609332a3f60 2024-07-31T14:50:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9e23f3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560933665be0 con 0x560933283280 2024-07-31T14:50:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9e23f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5609336662b0 con 0x56093327b0a0 2024-07-31T14:50:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9d943d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933655020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9d943d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933655020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38592/0 (socket says 172.21.15.67:38592) 2024-07-31T14:50:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9d943d640 1 -- 172.21.15.67:0/3982414768 learned_addr learned my addr 172.21.15.67:0/3982414768 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9d9c3e640 1 --2- 172.21.15.67:0/3982414768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560933283280 0x560933660d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9d943d640 1 --2- 172.21.15.67:0/3982414768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56093327b0a0 0x560933666b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9d943d640 1 -- 172.21.15.67:0/3982414768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56093327b0a0 msgr2=0x560933666b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9d943d640 1 --2- 172.21.15.67:0/3982414768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56093327b0a0 0x560933666b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9d943d640 1 -- 172.21.15.67:0/3982414768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560933283280 msgr2=0x560933660d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9d943d640 1 --2- 172.21.15.67:0/3982414768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560933283280 0x560933660d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.414+0000 7ff9d943d640 1 -- 172.21.15.67:0/3982414768 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609336506d0 con 0x5609332a3f60 2024-07-31T14:50:07.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.415+0000 7ff9d943d640 1 --2- 172.21.15.67:0/3982414768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933655020 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4013520 tx=0x7ff9c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ce4a93e9cec4f1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:07.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.415+0000 7ff9bf7fe640 1 -- 172.21.15.67:0/3982414768 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40151d0 con 0x5609332a3f60 2024-07-31T14:50:07.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.415+0000 7ff9bf7fe640 1 -- 172.21.15.67:0/3982414768 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c400fda0 con 0x5609332a3f60 2024-07-31T14:50:07.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.415+0000 7ff9bf7fe640 1 -- 172.21.15.67:0/3982414768 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40047a0 con 0x5609332a3f60 2024-07-31T14:50:07.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.416+0000 7ff9e23f3740 1 -- 172.21.15.67:0/3982414768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 msgr2=0x560933655020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:07.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.416+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/3982414768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933655020 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4013520 tx=0x7ff9c40046c0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.416+0000 7ff9e23f3740 1 -- 172.21.15.67:0/3982414768 shutdown_connections 2024-07-31T14:50:07.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.416+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/3982414768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933655020 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.416+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/3982414768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560933283280 0x560933660d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.416+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/3982414768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56093327b0a0 0x560933666b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.416+0000 7ff9e23f3740 1 -- 172.21.15.67:0/3982414768 >> 172.21.15.67:0/3982414768 conn(0x560933298350 msgr2=0x5609336536c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:07.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.416+0000 7ff9e23f3740 1 -- 172.21.15.67:0/3982414768 shutdown_connections 2024-07-31T14:50:07.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.416+0000 7ff9e23f3740 1 -- 172.21.15.67:0/3982414768 wait complete. 2024-07-31T14:50:07.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.416+0000 7ff9e23f3740 1 -- start start 2024-07-31T14:50:07.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.416+0000 7ff9e23f3740 1 Processor -- start 2024-07-31T14:50:07.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9e23f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56093327b0a0 0x56093366f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9e23f3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560933283280 0x56093366fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9e23f3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933668af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9e23f3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560933667050 con 0x5609332a3f60 2024-07-31T14:50:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9e23f3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560933666980 con 0x560933283280 2024-07-31T14:50:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9e23f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560933667720 con 0x56093327b0a0 2024-07-31T14:50:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9e23f3740 10 client.0 _ll_register_callbacks cb 0x7ff9d000c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9d943d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933668af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9d943d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933668af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38602/0 (socket says 172.21.15.67:38602) 2024-07-31T14:50:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9d943d640 1 -- 172.21.15.67:0/1335912079 learned_addr learned my addr 172.21.15.67:0/1335912079 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9d943d640 1 --2- 172.21.15.67:0/1335912079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56093327b0a0 0x56093366f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9d9c3e640 1 --2- 172.21.15.67:0/1335912079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560933283280 0x56093366fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9d943d640 1 -- 172.21.15.67:0/1335912079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56093327b0a0 msgr2=0x56093366f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9d943d640 1 --2- 172.21.15.67:0/1335912079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56093327b0a0 0x56093366f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9d943d640 1 -- 172.21.15.67:0/1335912079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560933283280 msgr2=0x56093366fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9d943d640 1 --2- 172.21.15.67:0/1335912079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560933283280 0x56093366fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9d943d640 1 -- 172.21.15.67:0/1335912079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560933669280 con 0x5609332a3f60 2024-07-31T14:50:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.417+0000 7ff9d943d640 1 --2- 172.21.15.67:0/1335912079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933668af0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400a590 tx=0x7ff9c4005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2885541ff28f87 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9bd7fa640 1 -- 172.21.15.67:0/1335912079 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c400faf0 con 0x5609332a3f60 2024-07-31T14:50:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9bd7fa640 1 -- 172.21.15.67:0/1335912079 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff9c40151d0 con 0x5609332a3f60 2024-07-31T14:50:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9e23f3740 1 -- 172.21.15.67:0/1335912079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c400fda0 con 0x5609332a3f60 2024-07-31T14:50:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9bd7fa640 1 -- 172.21.15.67:0/1335912079 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9c40153d0 con 0x5609332a3f60 2024-07-31T14:50:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9e23f3740 10 client.15252 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9e23f3740 1 -- 172.21.15.67:0/1335912079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56093366ec80 con 0x5609332a3f60 2024-07-31T14:50:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9bd7fa640 1 -- 172.21.15.67:0/1335912079 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c400fda0 con 0x5609332a3f60 2024-07-31T14:50:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9bd7fa640 1 -- 172.21.15.67:0/1335912079 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff9c4013770 con 0x5609332a3f60 2024-07-31T14:50:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9e23f3740 10 client.15252 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9e23f3740 10 client.15252 resolve_mds: resolved name 'f' to daemon mds.f 2024-07-31T14:50:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9e23f3740 20 client.15252 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/1335912079 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x560933237c90 0x5609336d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9e23f3740 4 client.15252 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:50:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.418+0000 7ff9e23f3740 1 -- 172.21.15.67:0/1335912079 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "status"}) -- 0x5609335bd310 con 0x560933237c90 2024-07-31T14:50:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.419+0000 7ff9d9c3e640 1 --2- 172.21.15.67:0/1335912079 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x560933237c90 0x5609336d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.419+0000 7ff9d9c3e640 1 --2- 172.21.15.67:0/1335912079 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x560933237c90 0x5609336d3360 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=586b74810baf1195 server_cookie=43df6b5ef287f2ae in_seq=0 out_seq=0 2024-07-31T14:50:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.419+0000 7ff9bd7fa640 10 client.15252 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:50:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.419+0000 7ff9bd7fa640 1 -- 172.21.15.67:0/1335912079 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5609335bd310 con 0x560933237c90 2024-07-31T14:50:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.419+0000 7ff9bd7fa640 10 client.15252 handle_command_reply: tid=0 2024-07-31T14:50:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.419+0000 7ff9e23f3740 1 client.15252 shutdown 2024-07-31T14:50:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.419+0000 7ff9e23f3740 1 -- 172.21.15.67:0/1335912079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 msgr2=0x560933668af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.419+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/1335912079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933668af0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400a590 tx=0x7ff9c4005ea0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9e23f3740 1 -- 172.21.15.67:0/1335912079 shutdown_connections 2024-07-31T14:50:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/1335912079 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x560933237c90 0x5609336d3360 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/1335912079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609332a3f60 0x560933668af0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/1335912079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560933283280 0x56093366fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9e23f3740 1 --2- 172.21.15.67:0/1335912079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56093327b0a0 0x56093366f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9bd7fa640 0 client.15252 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:50:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9e23f3740 1 -- 172.21.15.67:0/1335912079 >> 172.21.15.67:0/1335912079 conn(0x560933298350 msgr2=0x560933653290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9e23f3740 1 -- 172.21.15.67:0/1335912079 shutdown_connections 2024-07-31T14:50:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9e23f3740 1 -- 172.21.15.67:0/1335912079 wait complete. 2024-07-31T14:50:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9e23f3740 20 client.15252 trim_cache size 0 max 16384 2024-07-31T14:50:07.427 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:50:07.427 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:50:07.427 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 4, 2024-07-31T14:50:07.427 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24634, 2024-07-31T14:50:07.427 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:50:07.427 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:50:07.427 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:50:07.427 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 124.83107066399999, 2024-07-31T14:50:07.427 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-31T14:50:07.428 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:50:07.428 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:50:07.428 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 213.75848149000001 2024-07-31T14:50:07.428 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:50:07.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9dbea2640 1 -- 172.21.15.67:0/1073460820 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff9c00b3c50 msgr2=0x7ff9c00b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:07.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9dbea2640 1 --2- 172.21.15.67:0/1073460820 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff9c00b3c50 0x7ff9c00b6160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc102fd0 tx=0x7ff9cc040000 comp rx=0 tx=0).stop 2024-07-31T14:50:07.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9dbea2640 1 -- 172.21.15.67:0/1073460820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9d4109820 msgr2=0x7ff9d419f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:07.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9dbea2640 1 --2- 172.21.15.67:0/1073460820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9d4109820 0x7ff9d419f6a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc005cc0 tx=0x7ff9cc02f470 comp rx=0 tx=0).stop 2024-07-31T14:50:07.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9dbea2640 1 -- 172.21.15.67:0/1073460820 shutdown_connections 2024-07-31T14:50:07.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9dbea2640 1 --2- 172.21.15.67:0/1073460820 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff9c00b3c50 0x7ff9c00b6160 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9dbea2640 1 --2- 172.21.15.67:0/1073460820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9d4109820 0x7ff9d419f6a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9dbea2640 1 --2- 172.21.15.67:0/1073460820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4108ed0 0x7ff9d419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9dbea2640 1 --2- 172.21.15.67:0/1073460820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9d4103590 0x7ff9d4199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:07.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.420+0000 7ff9dbea2640 1 -- 172.21.15.67:0/1073460820 >> 172.21.15.67:0/1073460820 conn(0x7ff9d407b280 msgr2=0x7ff9d4106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:07.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.421+0000 7ff9dbea2640 1 -- 172.21.15.67:0/1073460820 shutdown_connections 2024-07-31T14:50:07.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:07.421+0000 7ff9dbea2640 1 -- 172.21.15.67:0/1073460820 wait complete. 2024-07-31T14:50:08.283 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.c status 2024-07-31T14:50:08.554 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:09.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:08 smithi077 ceph-mon[37327]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:09.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:08 smithi067 ceph-mon[32750]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:09.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:08 smithi073 ceph-mon[36714]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.474+0000 7f9dc0dd4640 1 -- 172.21.15.67:0/2415159861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc103590 msgr2=0x7f9dbc1039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.474+0000 7f9dc0dd4640 1 --2- 172.21.15.67:0/2415159861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc103590 0x7f9dbc1039a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009900 tx=0x7f9dac02a410 comp rx=0 tx=0).stop 2024-07-31T14:50:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.474+0000 7f9dc0dd4640 1 -- 172.21.15.67:0/2415159861 shutdown_connections 2024-07-31T14:50:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.474+0000 7f9dc0dd4640 1 --2- 172.21.15.67:0/2415159861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dbc109820 0x7f9dbc110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.474+0000 7f9dc0dd4640 1 --2- 172.21.15.67:0/2415159861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9dbc108ed0 0x7f9dbc1092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.474+0000 7f9dc0dd4640 1 --2- 172.21.15.67:0/2415159861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc103590 0x7f9dbc1039a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.474+0000 7f9dc0dd4640 1 -- 172.21.15.67:0/2415159861 >> 172.21.15.67:0/2415159861 conn(0x7f9dbc07b280 msgr2=0x7f9dbc07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.474+0000 7f9dc0dd4640 1 -- 172.21.15.67:0/2415159861 shutdown_connections 2024-07-31T14:50:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.474+0000 7f9dc0dd4640 1 -- 172.21.15.67:0/2415159861 wait complete. 2024-07-31T14:50:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.475+0000 7f9dc0dd4640 1 Processor -- start 2024-07-31T14:50:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.475+0000 7f9dc0dd4640 1 -- start start 2024-07-31T14:50:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.475+0000 7f9dc0dd4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc103590 0x7f9dbc199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.475+0000 7f9dc0dd4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9dbc108ed0 0x7f9dbc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.475+0000 7f9dc0dd4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dbc109820 0x7f9dbc19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.475+0000 7f9dc0dd4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc1134f0 con 0x7f9dbc103590 2024-07-31T14:50:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.475+0000 7f9dc0dd4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc113370 con 0x7f9dbc108ed0 2024-07-31T14:50:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.475+0000 7f9dc0dd4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc113670 con 0x7f9dbc109820 2024-07-31T14:50:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.475+0000 7f9dba59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc103590 0x7f9dbc199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dba59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc103590 0x7f9dbc199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38618/0 (socket says 172.21.15.67:38618) 2024-07-31T14:50:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dba59c640 1 -- 172.21.15.67:0/3032689340 learned_addr learned my addr 172.21.15.67:0/3032689340 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9db9d9b640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9dbc108ed0 0x7f9dbc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dbad9d640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dbc109820 0x7f9dbc19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dba59c640 1 -- 172.21.15.67:0/3032689340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dbc109820 msgr2=0x7f9dbc19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dba59c640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dbc109820 0x7f9dbc19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dba59c640 1 -- 172.21.15.67:0/3032689340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9dbc108ed0 msgr2=0x7f9dbc19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dba59c640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9dbc108ed0 0x7f9dbc19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dba59c640 1 -- 172.21.15.67:0/3032689340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc19fed0 con 0x7f9dbc103590 2024-07-31T14:50:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dbad9d640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dbc109820 0x7f9dbc19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:50:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dba59c640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc103590 0x7f9dbc199f60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9dac005940 tx=0x7f9dac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dab7fe640 1 -- 172.21.15.67:0/3032689340 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac03f030 con 0x7f9dbc103590 2024-07-31T14:50:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dc0dd4640 1 -- 172.21.15.67:0/3032689340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dbc1a0160 con 0x7f9dbc103590 2024-07-31T14:50:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dab7fe640 1 -- 172.21.15.67:0/3032689340 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dac003cb0 con 0x7f9dbc103590 2024-07-31T14:50:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dab7fe640 1 -- 172.21.15.67:0/3032689340 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac035070 con 0x7f9dbc103590 2024-07-31T14:50:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.476+0000 7f9dc0dd4640 1 -- 172.21.15.67:0/3032689340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dbc1a6920 con 0x7f9dbc103590 2024-07-31T14:50:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.479+0000 7f9dc7325740 1 Processor -- start 2024-07-31T14:50:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.479+0000 7f9dc7325740 1 -- start start 2024-07-31T14:50:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.479+0000 7f9dab7fe640 1 -- 172.21.15.67:0/3032689340 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f9dac042030 con 0x7f9dbc103590 2024-07-31T14:50:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dc7325740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faed8f60 0x55a9fb27e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dc7325740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb27e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dc7325740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faeb8280 0x55a9fb27ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dc7325740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a9fb1f4920 con 0x55a9faeb00a0 2024-07-31T14:50:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dc7325740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a9fb1e3e20 con 0x55a9faeb8280 2024-07-31T14:50:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dc7325740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a9fb1e7e60 con 0x55a9faed8f60 2024-07-31T14:50:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dba59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb27e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dba59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb27e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38628/0 (socket says 172.21.15.67:38628) 2024-07-31T14:50:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dba59c640 1 -- 172.21.15.67:0/2447650083 learned_addr learned my addr 172.21.15.67:0/2447650083 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dbad9d640 1 --2- 172.21.15.67:0/2447650083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faed8f60 0x55a9fb27e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9db9d9b640 1 --2- 172.21.15.67:0/2447650083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faeb8280 0x55a9fb27ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dba59c640 1 -- 172.21.15.67:0/2447650083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faed8f60 msgr2=0x55a9fb27e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dba59c640 1 --2- 172.21.15.67:0/2447650083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faed8f60 0x55a9fb27e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dba59c640 1 -- 172.21.15.67:0/2447650083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faeb8280 msgr2=0x55a9fb27ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dba59c640 1 --2- 172.21.15.67:0/2447650083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faeb8280 0x55a9fb27ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.480+0000 7f9dba59c640 1 -- 172.21.15.67:0/2447650083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9fb18aa40 con 0x55a9faeb00a0 2024-07-31T14:50:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.481+0000 7f9dab7fe640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9da40b3c20 0x7f9da40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.481+0000 7f9dbad9d640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9da40b3c20 0x7f9da40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.481+0000 7f9dba59c640 1 --2- 172.21.15.67:0/2447650083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb27e8c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9dac10ac10 tx=0x7f9dac10ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1e1e8088535376 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.481+0000 7f9dab7fe640 1 -- 172.21.15.67:0/3032689340 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f9dac1046b0 con 0x7f9dbc103590 2024-07-31T14:50:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.481+0000 7f9da8ff9640 1 -- 172.21.15.67:0/2447650083 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac049030 con 0x55a9faeb00a0 2024-07-31T14:50:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.481+0000 7f9da8ff9640 1 -- 172.21.15.67:0/2447650083 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dac03b460 con 0x55a9faeb00a0 2024-07-31T14:50:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.481+0000 7f9da8ff9640 1 -- 172.21.15.67:0/2447650083 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0c7090 con 0x55a9faeb00a0 2024-07-31T14:50:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.482+0000 7f9dc7325740 1 -- 172.21.15.67:0/2447650083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 msgr2=0x55a9fb27e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.482+0000 7f9dbad9d640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9da40b3c20 0x7f9da40b6130 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9db4005fd0 tx=0x7f9db40085b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.482+0000 7f9dc7325740 1 --2- 172.21.15.67:0/2447650083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb27e8c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9dac10ac10 tx=0x7f9dac10ad00 comp rx=0 tx=0).stop 2024-07-31T14:50:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.482+0000 7f9dc7325740 1 -- 172.21.15.67:0/2447650083 shutdown_connections 2024-07-31T14:50:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.482+0000 7f9dc7325740 1 --2- 172.21.15.67:0/2447650083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faeb8280 0x55a9fb27ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.482+0000 7f9dc7325740 1 --2- 172.21.15.67:0/2447650083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb27e8c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.482+0000 7f9dc7325740 1 --2- 172.21.15.67:0/2447650083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faed8f60 0x55a9fb27e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.482+0000 7f9dc7325740 1 -- 172.21.15.67:0/2447650083 >> 172.21.15.67:0/2447650083 conn(0x55a9faecd350 msgr2=0x55a9fb27c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.482+0000 7f9dc7325740 1 -- 172.21.15.67:0/2447650083 shutdown_connections 2024-07-31T14:50:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.482+0000 7f9dc7325740 1 -- 172.21.15.67:0/2447650083 wait complete. 2024-07-31T14:50:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.483+0000 7f9dc7325740 1 -- start start 2024-07-31T14:50:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.483+0000 7f9dc7325740 1 Processor -- start 2024-07-31T14:50:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.483+0000 7f9dc7325740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb289ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.483+0000 7f9dc7325740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faeb8280 0x55a9fb28a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.483+0000 7f9dc7325740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faed8f60 0x55a9fb295c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.483+0000 7f9dc7325740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a9fb0475f0 con 0x55a9faeb00a0 2024-07-31T14:50:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.483+0000 7f9dc7325740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a9fb186890 con 0x55a9faed8f60 2024-07-31T14:50:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.483+0000 7f9dc7325740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a9fb28c920 con 0x55a9faeb8280 2024-07-31T14:50:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.483+0000 7f9dc7325740 10 client.0 _ll_register_callbacks cb 0x55a9fb1907d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.483+0000 7f9dba59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb289ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.483+0000 7f9dba59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb289ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38630/0 (socket says 172.21.15.67:38630) 2024-07-31T14:50:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.483+0000 7f9dba59c640 1 -- 172.21.15.67:0/15977945 learned_addr learned my addr 172.21.15.67:0/15977945 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.483+0000 7f9db9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faeb8280 0x55a9fb28a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.483+0000 7f9dbad9d640 1 --2- 172.21.15.67:0/15977945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faed8f60 0x55a9fb295c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9dba59c640 1 -- 172.21.15.67:0/15977945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faeb8280 msgr2=0x55a9fb28a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9dba59c640 1 --2- 172.21.15.67:0/15977945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faeb8280 0x55a9fb28a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9dba59c640 1 -- 172.21.15.67:0/15977945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faed8f60 msgr2=0x55a9fb295c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9dba59c640 1 --2- 172.21.15.67:0/15977945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faed8f60 0x55a9fb295c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9dba59c640 1 -- 172.21.15.67:0/15977945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9faf44540 con 0x55a9faeb00a0 2024-07-31T14:50:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9db9d9b640 1 --2- 172.21.15.67:0/15977945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faeb8280 0x55a9fb28a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:50:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9dba59c640 1 --2- 172.21.15.67:0/15977945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb289ff0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9dac031440 tx=0x7f9dac1049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f7a3e72076f604 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9da1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9da1ffb640 1 -- 172.21.15.67:0/15977945 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac049050 con 0x55a9faeb00a0 2024-07-31T14:50:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9da1ffb640 1 -- 172.21.15.67:0/15977945 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f9dac104380 con 0x55a9faeb00a0 2024-07-31T14:50:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9dc7325740 1 -- 172.21.15.67:0/15977945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a9fb1f6870 con 0x55a9faeb00a0 2024-07-31T14:50:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9da1ffb640 1 -- 172.21.15.67:0/15977945 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9dac0ca940 con 0x55a9faeb00a0 2024-07-31T14:50:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9dc7325740 10 client.15264 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9dc7325740 1 -- 172.21.15.67:0/15977945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a9fb298980 con 0x55a9faeb00a0 2024-07-31T14:50:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.484+0000 7f9da1ffb640 1 -- 172.21.15.67:0/15977945 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a9fb1f6870 con 0x55a9faeb00a0 2024-07-31T14:50:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.485+0000 7f9da1ffb640 1 -- 172.21.15.67:0/15977945 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9dac0e2ce0 con 0x55a9faeb00a0 2024-07-31T14:50:10.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.485+0000 7f9dc7325740 10 client.15264 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:10.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.485+0000 7f9dc7325740 10 client.15264 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-31T14:50:10.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.485+0000 7f9dc7325740 20 client.15264 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:10.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.485+0000 7f9dc7325740 1 --2- 172.21.15.67:0/15977945 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a9fae978e0 0x55a9fb29c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.485+0000 7f9dc7325740 4 client.15264 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:50:10.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.485+0000 7f9dc7325740 1 -- 172.21.15.67:0/15977945 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9fb1f2310 con 0x55a9fae978e0 2024-07-31T14:50:10.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.485+0000 7f9db9d9b640 1 --2- 172.21.15.67:0/15977945 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a9fae978e0 0x55a9fb29c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.486+0000 7f9db9d9b640 1 --2- 172.21.15.67:0/15977945 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a9fae978e0 0x55a9fb29c190 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1e6665f0afc42b80 server_cookie=2a94fee68d82511b in_seq=0 out_seq=0 2024-07-31T14:50:10.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.486+0000 7f9da1ffb640 10 client.15264 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:50:10.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.486+0000 7f9da1ffb640 1 -- 172.21.15.67:0/15977945 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9fb1f2310 con 0x55a9fae978e0 2024-07-31T14:50:10.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.486+0000 7f9da1ffb640 10 client.15264 handle_command_reply: tid=0 2024-07-31T14:50:10.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.486+0000 7f9dc7325740 1 client.15264 shutdown 2024-07-31T14:50:10.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.487+0000 7f9dc7325740 1 -- 172.21.15.67:0/15977945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 msgr2=0x55a9fb289ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:10.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.487+0000 7f9dc7325740 1 --2- 172.21.15.67:0/15977945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb289ff0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9dac031440 tx=0x7f9dac1049b0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.487+0000 7f9dc7325740 1 -- 172.21.15.67:0/15977945 shutdown_connections 2024-07-31T14:50:10.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.487+0000 7f9dc7325740 1 --2- 172.21.15.67:0/15977945 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a9fae978e0 0x55a9fb29c190 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.487+0000 7f9dc7325740 1 --2- 172.21.15.67:0/15977945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faed8f60 0x55a9fb295c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.487+0000 7f9dc7325740 1 --2- 172.21.15.67:0/15977945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faeb8280 0x55a9fb28a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.487+0000 7f9da1ffb640 0 client.15264 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:50:10.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.487+0000 7f9dc7325740 1 --2- 172.21.15.67:0/15977945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb289ff0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.487+0000 7f9dc7325740 1 -- 172.21.15.67:0/15977945 >> 172.21.15.67:0/15977945 conn(0x55a9faecd350 msgr2=0x55a9fb27f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:10.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.487+0000 7f9dc7325740 1 -- 172.21.15.67:0/15977945 shutdown_connections 2024-07-31T14:50:10.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.487+0000 7f9dc7325740 1 -- 172.21.15.67:0/15977945 wait complete. 2024-07-31T14:50:10.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.487+0000 7f9dc7325740 20 client.15264 trim_cache size 0 max 16384 2024-07-31T14:50:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.506+0000 7f9dc7325740 1 Processor -- start 2024-07-31T14:50:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.506+0000 7f9dc7325740 1 -- start start 2024-07-31T14:50:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.506+0000 7f9dc7325740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9fae978e0 0x55a9fb284420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.506+0000 7f9dc7325740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faeb00a0 0x55a9fb295b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.506+0000 7f9dc7325740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb8280 0x55a9fb2960c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.506+0000 7f9dc7325740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a9fb29a770 con 0x55a9faeb8280 2024-07-31T14:50:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.506+0000 7f9dc7325740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a9fb29ae40 con 0x55a9faeb00a0 2024-07-31T14:50:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.506+0000 7f9dc7325740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a9fb29b270 con 0x55a9fae978e0 2024-07-31T14:50:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.506+0000 7f9dbad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb8280 0x55a9fb2960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.506+0000 7f9dbad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb8280 0x55a9fb2960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38646/0 (socket says 172.21.15.67:38646) 2024-07-31T14:50:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.506+0000 7f9dbad9d640 1 -- 172.21.15.67:0/1125569629 learned_addr learned my addr 172.21.15.67:0/1125569629 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.506+0000 7f9db9d9b640 1 --2- 172.21.15.67:0/1125569629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faeb00a0 0x55a9fb295b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.506+0000 7f9dba59c640 1 --2- 172.21.15.67:0/1125569629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9fae978e0 0x55a9fb284420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.507+0000 7f9dbad9d640 1 -- 172.21.15.67:0/1125569629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9fae978e0 msgr2=0x55a9fb284420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.507+0000 7f9dbad9d640 1 --2- 172.21.15.67:0/1125569629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9fae978e0 0x55a9fb284420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.507+0000 7f9dbad9d640 1 -- 172.21.15.67:0/1125569629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faeb00a0 msgr2=0x55a9fb295b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:10.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.507+0000 7f9dbad9d640 1 --2- 172.21.15.67:0/1125569629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faeb00a0 0x55a9fb295b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.507+0000 7f9dbad9d640 1 -- 172.21.15.67:0/1125569629 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9fb2882f0 con 0x55a9faeb8280 2024-07-31T14:50:10.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.507+0000 7f9db9d9b640 1 --2- 172.21.15.67:0/1125569629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faeb00a0 0x55a9fb295b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:50:10.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.507+0000 7f9dbad9d640 1 --2- 172.21.15.67:0/1125569629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb8280 0x55a9fb2960c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9db40084d0 tx=0x7f9db4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c20001aa14e47b8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:10.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.507+0000 7f9da3fff640 1 -- 172.21.15.67:0/1125569629 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4027420 con 0x55a9faeb8280 2024-07-31T14:50:10.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.507+0000 7f9da3fff640 1 -- 172.21.15.67:0/1125569629 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9db401d8a0 con 0x55a9faeb8280 2024-07-31T14:50:10.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.507+0000 7f9da3fff640 1 -- 172.21.15.67:0/1125569629 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4026760 con 0x55a9faeb8280 2024-07-31T14:50:10.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.508+0000 7f9dc7325740 1 -- 172.21.15.67:0/1125569629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb8280 msgr2=0x55a9fb2960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:10.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.508+0000 7f9dc7325740 1 --2- 172.21.15.67:0/1125569629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb8280 0x55a9fb2960c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9db40084d0 tx=0x7f9db4013000 comp rx=0 tx=0).stop 2024-07-31T14:50:10.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.508+0000 7f9dc7325740 1 -- 172.21.15.67:0/1125569629 shutdown_connections 2024-07-31T14:50:10.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.508+0000 7f9dc7325740 1 --2- 172.21.15.67:0/1125569629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb8280 0x55a9fb2960c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.508+0000 7f9dc7325740 1 --2- 172.21.15.67:0/1125569629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9faeb00a0 0x55a9fb295b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.508+0000 7f9dc7325740 1 --2- 172.21.15.67:0/1125569629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9fae978e0 0x55a9fb284420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.508+0000 7f9dc7325740 1 -- 172.21.15.67:0/1125569629 >> 172.21.15.67:0/1125569629 conn(0x55a9faecd350 msgr2=0x55a9fb288670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:10.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.508+0000 7f9dc7325740 1 -- 172.21.15.67:0/1125569629 shutdown_connections 2024-07-31T14:50:10.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.508+0000 7f9dc7325740 1 -- 172.21.15.67:0/1125569629 wait complete. 2024-07-31T14:50:10.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.508+0000 7f9dc7325740 1 -- start start 2024-07-31T14:50:10.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9dc7325740 1 Processor -- start 2024-07-31T14:50:10.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9dc7325740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9fae978e0 0x55a9fb29d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9dc7325740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb29dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9dc7325740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faeb8280 0x55a9fb2a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9dc7325740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a9fb29c010 con 0x55a9faeb00a0 2024-07-31T14:50:10.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9dc7325740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a9fb29b940 con 0x55a9fae978e0 2024-07-31T14:50:10.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9dc7325740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a9fb29c6e0 con 0x55a9faeb8280 2024-07-31T14:50:10.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9dc7325740 10 client.0 _ll_register_callbacks cb 0x7f9dac10aa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:10.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9db9d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb29dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9db9d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb29dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38652/0 (socket says 172.21.15.67:38652) 2024-07-31T14:50:10.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9db9d9b640 1 -- 172.21.15.67:0/922769120 learned_addr learned my addr 172.21.15.67:0/922769120 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:10.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9dba59c640 1 --2- 172.21.15.67:0/922769120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9fae978e0 0x55a9fb29d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9dbad9d640 1 --2- 172.21.15.67:0/922769120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faeb8280 0x55a9fb2a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9db9d9b640 1 -- 172.21.15.67:0/922769120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faeb8280 msgr2=0x55a9fb2a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:10.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9db9d9b640 1 --2- 172.21.15.67:0/922769120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faeb8280 0x55a9fb2a2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9db9d9b640 1 -- 172.21.15.67:0/922769120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9fae978e0 msgr2=0x55a9fb29d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:10.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9db9d9b640 1 --2- 172.21.15.67:0/922769120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9fae978e0 0x55a9fb29d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.509+0000 7f9db9d9b640 1 -- 172.21.15.67:0/922769120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9fb2a3370 con 0x55a9faeb00a0 2024-07-31T14:50:10.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.510+0000 7f9db9d9b640 1 --2- 172.21.15.67:0/922769120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb29dcc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc06c420 tx=0x7f9db00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872052b719ade40c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:10.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.510+0000 7f9da1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:10.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.510+0000 7f9da1ffb640 1 -- 172.21.15.67:0/922769120 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db00035f0 con 0x55a9faeb00a0 2024-07-31T14:50:10.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.510+0000 7f9da1ffb640 1 -- 172.21.15.67:0/922769120 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f9db0003790 con 0x55a9faeb00a0 2024-07-31T14:50:10.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.510+0000 7f9dc7325740 1 -- 172.21.15.67:0/922769120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9db401d8a0 con 0x55a9faeb00a0 2024-07-31T14:50:10.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.510+0000 7f9da1ffb640 1 -- 172.21.15.67:0/922769120 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9db002e050 con 0x55a9faeb00a0 2024-07-31T14:50:10.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.510+0000 7f9dc7325740 10 client.15270 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:10.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.510+0000 7f9dc7325740 1 -- 172.21.15.67:0/922769120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a9fb2a4790 con 0x55a9faeb00a0 2024-07-31T14:50:10.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.510+0000 7f9da1ffb640 1 -- 172.21.15.67:0/922769120 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9db401d8a0 con 0x55a9faeb00a0 2024-07-31T14:50:10.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.510+0000 7f9da1ffb640 1 -- 172.21.15.67:0/922769120 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9db001da40 con 0x55a9faeb00a0 2024-07-31T14:50:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.511+0000 7f9dc7325740 10 client.15270 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.511+0000 7f9dc7325740 10 client.15270 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-31T14:50:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.511+0000 7f9dc7325740 20 client.15270 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.511+0000 7f9dc7325740 1 --2- 172.21.15.67:0/922769120 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a9faed8f60 0x55a9fb308250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.511+0000 7f9dc7325740 4 client.15270 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:50:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.511+0000 7f9dc7325740 1 -- 172.21.15.67:0/922769120 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "status"}) -- 0x55a9fb1efa50 con 0x55a9faed8f60 2024-07-31T14:50:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.511+0000 7f9dba59c640 1 --2- 172.21.15.67:0/922769120 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a9faed8f60 0x55a9fb308250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.511+0000 7f9da1ffb640 10 client.15270 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:50:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.511+0000 7f9dba59c640 1 --2- 172.21.15.67:0/922769120 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a9faed8f60 0x55a9fb308250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=804aaf6f4d5ed239 server_cookie=db38f5e7844ffcc6 in_seq=0 out_seq=0 2024-07-31T14:50:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.511+0000 7f9da1ffb640 1 -- 172.21.15.67:0/922769120 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a9fb1efa50 con 0x55a9faed8f60 2024-07-31T14:50:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.511+0000 7f9da1ffb640 10 client.15270 handle_command_reply: tid=0 2024-07-31T14:50:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.511+0000 7f9dc7325740 1 client.15270 shutdown 2024-07-31T14:50:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc7325740 1 -- 172.21.15.67:0/922769120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 msgr2=0x55a9fb29dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:10.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc7325740 1 --2- 172.21.15.67:0/922769120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb29dcc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc06c420 tx=0x7f9db00118d0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc7325740 1 -- 172.21.15.67:0/922769120 shutdown_connections 2024-07-31T14:50:10.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc7325740 1 --2- 172.21.15.67:0/922769120 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a9faed8f60 0x55a9fb308250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc7325740 1 --2- 172.21.15.67:0/922769120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9faeb8280 0x55a9fb2a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc7325740 1 --2- 172.21.15.67:0/922769120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a9faeb00a0 0x55a9fb29dcc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc7325740 1 --2- 172.21.15.67:0/922769120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9fae978e0 0x55a9fb29d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9da1ffb640 0 client.15270 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:50:10.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc7325740 1 -- 172.21.15.67:0/922769120 >> 172.21.15.67:0/922769120 conn(0x55a9faecd350 msgr2=0x55a9fb288500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:10.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc7325740 1 -- 172.21.15.67:0/922769120 shutdown_connections 2024-07-31T14:50:10.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc7325740 1 -- 172.21.15.67:0/922769120 wait complete. 2024-07-31T14:50:10.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc7325740 20 client.15270 trim_cache size 0 max 16384 2024-07-31T14:50:10.519 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:50:10.519 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:50:10.519 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 3, 2024-07-31T14:50:10.519 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24646, 2024-07-31T14:50:10.519 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:50:10.519 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:50:10.519 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:50:10.519 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 128.93302583400001, 2024-07-31T14:50:10.519 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-31T14:50:10.519 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:50:10.519 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:50:10.519 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 214.26405162200001 2024-07-31T14:50:10.520 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:50:10.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc0dd4640 1 -- 172.21.15.67:0/3032689340 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9da40b3c20 msgr2=0x7f9da40b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:10.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc0dd4640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9da40b3c20 0x7f9da40b6130 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9db4005fd0 tx=0x7f9db40085b0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc0dd4640 1 -- 172.21.15.67:0/3032689340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc103590 msgr2=0x7f9dbc199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:10.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.512+0000 7f9dc0dd4640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc103590 0x7f9dbc199f60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9dac005940 tx=0x7f9dac0042f0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.513+0000 7f9dc0dd4640 1 -- 172.21.15.67:0/3032689340 shutdown_connections 2024-07-31T14:50:10.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.513+0000 7f9dc0dd4640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9da40b3c20 0x7f9da40b6130 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.513+0000 7f9dc0dd4640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dbc109820 0x7f9dbc19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.513+0000 7f9dc0dd4640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9dbc108ed0 0x7f9dbc19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.513+0000 7f9dc0dd4640 1 --2- 172.21.15.67:0/3032689340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc103590 0x7f9dbc199f60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:10.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.513+0000 7f9dc0dd4640 1 -- 172.21.15.67:0/3032689340 >> 172.21.15.67:0/3032689340 conn(0x7f9dbc07b280 msgr2=0x7f9dbc107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:10.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.513+0000 7f9dc0dd4640 1 -- 172.21.15.67:0/3032689340 shutdown_connections 2024-07-31T14:50:10.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:10.513+0000 7f9dc0dd4640 1 -- 172.21.15.67:0/3032689340 wait complete. 2024-07-31T14:50:11.231 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:10 smithi067 ceph-mon[32750]: pgmap v372: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:11.235 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.h status 2024-07-31T14:50:11.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:10 smithi077 ceph-mon[37327]: pgmap v372: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:11.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:10 smithi073 ceph-mon[36714]: pgmap v372: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:11.500 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:13.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:12 smithi077 ceph-mon[37327]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:13.265 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:12 smithi067 ceph-mon[32750]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:13.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.363+0000 7fcb04752640 1 -- 172.21.15.67:0/1476545211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcafc1024b0 msgr2=0x7fcafc1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:13.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.363+0000 7fcb04752640 1 --2- 172.21.15.67:0/1476545211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcafc1024b0 0x7fcafc1028c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009900 tx=0x7fcaf802a410 comp rx=0 tx=0).stop 2024-07-31T14:50:13.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.363+0000 7fcb04752640 1 -- 172.21.15.67:0/1476545211 shutdown_connections 2024-07-31T14:50:13.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.363+0000 7fcb04752640 1 --2- 172.21.15.67:0/1476545211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcafc108740 0x7fcafc10ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.363+0000 7fcb04752640 1 --2- 172.21.15.67:0/1476545211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcafc107df0 0x7fcafc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.363+0000 7fcb04752640 1 --2- 172.21.15.67:0/1476545211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcafc1024b0 0x7fcafc1028c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.363+0000 7fcb04752640 1 -- 172.21.15.67:0/1476545211 >> 172.21.15.67:0/1476545211 conn(0x7fcafc079170 msgr2=0x7fcafc079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:13.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.363+0000 7fcb04752640 1 -- 172.21.15.67:0/1476545211 shutdown_connections 2024-07-31T14:50:13.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.363+0000 7fcb04752640 1 -- 172.21.15.67:0/1476545211 wait complete. 2024-07-31T14:50:13.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.364+0000 7fcb04752640 1 Processor -- start 2024-07-31T14:50:13.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.364+0000 7fcb04752640 1 -- start start 2024-07-31T14:50:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.364+0000 7fcb04752640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcafc1024b0 0x7fcafc199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.364+0000 7fcb04752640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcafc107df0 0x7fcafc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.364+0000 7fcb04752640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcafc108740 0x7fcafc19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.364+0000 7fcb04752640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc112420 con 0x7fcafc1024b0 2024-07-31T14:50:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.364+0000 7fcb04752640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc1122a0 con 0x7fcafc107df0 2024-07-31T14:50:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.364+0000 7fcb04752640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc1125a0 con 0x7fcafc108740 2024-07-31T14:50:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.364+0000 7fcb024ee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcafc1024b0 0x7fcafc199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.364+0000 7fcb024ee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcafc1024b0 0x7fcafc199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35652/0 (socket says 172.21.15.67:35652) 2024-07-31T14:50:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.364+0000 7fcb024ee640 1 -- 172.21.15.67:0/1397939111 learned_addr learned my addr 172.21.15.67:0/1397939111 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.364+0000 7fcb02cef640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcafc108740 0x7fcafc19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.365+0000 7fcb01ced640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcafc107df0 0x7fcafc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.365+0000 7fcb024ee640 1 -- 172.21.15.67:0/1397939111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcafc108740 msgr2=0x7fcafc19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.365+0000 7fcb024ee640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcafc108740 0x7fcafc19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.365+0000 7fcb024ee640 1 -- 172.21.15.67:0/1397939111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcafc107df0 msgr2=0x7fcafc19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.365+0000 7fcb024ee640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcafc107df0 0x7fcafc19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.365+0000 7fcb024ee640 1 -- 172.21.15.67:0/1397939111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc19feb0 con 0x7fcafc1024b0 2024-07-31T14:50:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.365+0000 7fcb02cef640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcafc108740 0x7fcafc19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:50:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.365+0000 7fcb01ced640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcafc107df0 0x7fcafc19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:50:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.365+0000 7fcb024ee640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcafc1024b0 0x7fcafc199f40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8005940 tx=0x7fcaf80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.365+0000 7fcaf37fe640 1 -- 172.21.15.67:0/1397939111 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf803f030 con 0x7fcafc1024b0 2024-07-31T14:50:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.365+0000 7fcaf37fe640 1 -- 172.21.15.67:0/1397939111 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf8003cb0 con 0x7fcafc1024b0 2024-07-31T14:50:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.365+0000 7fcb04752640 1 -- 172.21.15.67:0/1397939111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcafc1a0140 con 0x7fcafc1024b0 2024-07-31T14:50:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.365+0000 7fcaf37fe640 1 -- 172.21.15.67:0/1397939111 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8035070 con 0x7fcafc1024b0 2024-07-31T14:50:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.365+0000 7fcb04752640 1 -- 172.21.15.67:0/1397939111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcafc1a6900 con 0x7fcafc1024b0 2024-07-31T14:50:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.368+0000 7fcb0aca3740 1 Processor -- start 2024-07-31T14:50:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.368+0000 7fcaf37fe640 1 -- 172.21.15.67:0/1397939111 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fcaf8042030 con 0x7fcafc1024b0 2024-07-31T14:50:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.368+0000 7fcb0aca3740 1 -- start start 2024-07-31T14:50:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.368+0000 7fcb0aca3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 0x55f35fc643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.368+0000 7fcb0aca3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f8960a0 0x55f35fc648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.368+0000 7fcb0aca3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f89e280 0x55f35fc64e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.368+0000 7fcb0aca3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f35fbda940 con 0x55f35f89e280 2024-07-31T14:50:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.368+0000 7fcb0aca3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f35fbc9e40 con 0x55f35f8bef60 2024-07-31T14:50:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.368+0000 7fcb0aca3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f35fbcde80 con 0x55f35f8960a0 2024-07-31T14:50:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.368+0000 7fcb01ced640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f89e280 0x55f35fc64e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.368+0000 7fcb01ced640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f89e280 0x55f35fc64e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35660/0 (socket says 172.21.15.67:35660) 2024-07-31T14:50:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.368+0000 7fcb01ced640 1 -- 172.21.15.67:0/4070726746 learned_addr learned my addr 172.21.15.67:0/4070726746 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.369+0000 7fcb024ee640 1 --2- 172.21.15.67:0/4070726746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f8960a0 0x55f35fc648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.369+0000 7fcb02cef640 1 --2- 172.21.15.67:0/4070726746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 0x55f35fc643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.369+0000 7fcb01ced640 1 -- 172.21.15.67:0/4070726746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f8960a0 msgr2=0x55f35fc648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.369+0000 7fcb01ced640 1 --2- 172.21.15.67:0/4070726746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f8960a0 0x55f35fc648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.369+0000 7fcb01ced640 1 -- 172.21.15.67:0/4070726746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 msgr2=0x55f35fc643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.369+0000 7fcb01ced640 1 --2- 172.21.15.67:0/4070726746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 0x55f35fc643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.369+0000 7fcaf37fe640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcae80b3c20 0x7fcae80b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.369+0000 7fcb01ced640 1 -- 172.21.15.67:0/4070726746 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f35fb70a60 con 0x55f35f89e280 2024-07-31T14:50:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.369+0000 7fcb02cef640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcae80b3c20 0x7fcae80b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.369+0000 7fcaf37fe640 1 -- 172.21.15.67:0/1397939111 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fcaf81049e0 con 0x7fcafc1024b0 2024-07-31T14:50:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.369+0000 7fcb01ced640 1 --2- 172.21.15.67:0/4070726746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f89e280 0x55f35fc64e20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009bf0 tx=0x7fcaec00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c144ffae2f6aa5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.369+0000 7fcaf0ff9640 1 -- 172.21.15.67:0/4070726746 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec016070 con 0x55f35f89e280 2024-07-31T14:50:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.369+0000 7fcaf0ff9640 1 -- 172.21.15.67:0/4070726746 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaec0054d0 con 0x55f35f89e280 2024-07-31T14:50:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.370+0000 7fcaf0ff9640 1 -- 172.21.15.67:0/4070726746 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec011470 con 0x55f35f89e280 2024-07-31T14:50:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.370+0000 7fcb0aca3740 1 -- 172.21.15.67:0/4070726746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f89e280 msgr2=0x55f35fc64e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.370+0000 7fcb02cef640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcae80b3c20 0x7fcae80b6130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400f400 tx=0x7fcaf4007740 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.370+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/4070726746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f89e280 0x55f35fc64e20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009bf0 tx=0x7fcaec00e6c0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.370+0000 7fcb0aca3740 1 -- 172.21.15.67:0/4070726746 shutdown_connections 2024-07-31T14:50:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.370+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/4070726746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f89e280 0x55f35fc64e20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.370+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/4070726746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f8960a0 0x55f35fc648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.370+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/4070726746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 0x55f35fc643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.370+0000 7fcb0aca3740 1 -- 172.21.15.67:0/4070726746 >> 172.21.15.67:0/4070726746 conn(0x55f35f8b3350 msgr2=0x55f35fc62290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.371+0000 7fcb0aca3740 1 -- 172.21.15.67:0/4070726746 shutdown_connections 2024-07-31T14:50:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.371+0000 7fcb0aca3740 1 -- 172.21.15.67:0/4070726746 wait complete. 2024-07-31T14:50:13.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.371+0000 7fcb0aca3740 1 -- start start 2024-07-31T14:50:13.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.371+0000 7fcb0aca3740 1 Processor -- start 2024-07-31T14:50:13.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.371+0000 7fcb0aca3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f8960a0 0x55f35fc6ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.371+0000 7fcb0aca3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f89e280 0x55f35fc70460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.371+0000 7fcb0aca3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 0x55f35fc7be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.371+0000 7fcb0aca3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f35fa2d5f0 con 0x55f35f89e280 2024-07-31T14:50:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.371+0000 7fcb0aca3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f35fb6c8b0 con 0x55f35f8bef60 2024-07-31T14:50:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.371+0000 7fcb0aca3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f35fc72b80 con 0x55f35f8960a0 2024-07-31T14:50:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.371+0000 7fcb0aca3740 10 client.0 _ll_register_callbacks cb 0x55f35fb767f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcb01ced640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f89e280 0x55f35fc70460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcb01ced640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f89e280 0x55f35fc70460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35672/0 (socket says 172.21.15.67:35672) 2024-07-31T14:50:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcb01ced640 1 -- 172.21.15.67:0/2459275412 learned_addr learned my addr 172.21.15.67:0/2459275412 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcb024ee640 1 --2- 172.21.15.67:0/2459275412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f8960a0 0x55f35fc6ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcb02cef640 1 --2- 172.21.15.67:0/2459275412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 0x55f35fc7be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcb01ced640 1 -- 172.21.15.67:0/2459275412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f8960a0 msgr2=0x55f35fc6ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcb01ced640 1 --2- 172.21.15.67:0/2459275412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f8960a0 0x55f35fc6ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcb01ced640 1 -- 172.21.15.67:0/2459275412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 msgr2=0x55f35fc7be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcb01ced640 1 --2- 172.21.15.67:0/2459275412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 0x55f35fc7be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcb01ced640 1 -- 172.21.15.67:0/2459275412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f35f92a540 con 0x55f35f89e280 2024-07-31T14:50:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcb01ced640 1 --2- 172.21.15.67:0/2459275412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f89e280 0x55f35fc70460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009f10 tx=0x7fcaec009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cb3652578b44d2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcae5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcae5ffb640 1 -- 172.21.15.67:0/2459275412 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec016040 con 0x55f35f89e280 2024-07-31T14:50:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcae5ffb640 1 -- 172.21.15.67:0/2459275412 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fcaec004020 con 0x55f35f89e280 2024-07-31T14:50:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.372+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2459275412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaec0054d0 con 0x55f35f89e280 2024-07-31T14:50:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.373+0000 7fcae5ffb640 1 -- 172.21.15.67:0/2459275412 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcaec004220 con 0x55f35f89e280 2024-07-31T14:50:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.373+0000 7fcb0aca3740 10 client.15282 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.373+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2459275412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f35fc7eb60 con 0x55f35f89e280 2024-07-31T14:50:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.373+0000 7fcae5ffb640 1 -- 172.21.15.67:0/2459275412 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaec0054d0 con 0x55f35f89e280 2024-07-31T14:50:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.373+0000 7fcae5ffb640 1 -- 172.21.15.67:0/2459275412 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcaec00f040 con 0x55f35f89e280 2024-07-31T14:50:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.373+0000 7fcb0aca3740 10 client.15282 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.373+0000 7fcb0aca3740 10 client.15282 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-31T14:50:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.373+0000 7fcb0aca3740 20 client.15282 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.373+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/2459275412 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55f35f87d8e0 0x55f35fc822d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.373+0000 7fcb0aca3740 4 client.15282 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:50:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.373+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2459275412 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f35fbdc890 con 0x55f35f87d8e0 2024-07-31T14:50:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.373+0000 7fcb024ee640 1 --2- 172.21.15.67:0/2459275412 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55f35f87d8e0 0x55f35fc822d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.374+0000 7fcae5ffb640 10 client.15282 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:50:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.374+0000 7fcb024ee640 1 --2- 172.21.15.67:0/2459275412 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55f35f87d8e0 0x55f35fc822d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a372bac574bfc270 server_cookie=c2a12e2d3d3bb823 in_seq=0 out_seq=0 2024-07-31T14:50:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcae5ffb640 1 -- 172.21.15.67:0/2459275412 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f35fbdc890 con 0x55f35f87d8e0 2024-07-31T14:50:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcae5ffb640 10 client.15282 handle_command_reply: tid=0 2024-07-31T14:50:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcb0aca3740 1 client.15282 shutdown 2024-07-31T14:50:13.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2459275412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f89e280 msgr2=0x55f35fc70460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:13.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/2459275412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f89e280 0x55f35fc70460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009f10 tx=0x7fcaec009f40 comp rx=0 tx=0).stop 2024-07-31T14:50:13.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2459275412 shutdown_connections 2024-07-31T14:50:13.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/2459275412 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55f35f87d8e0 0x55f35fc822d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/2459275412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 0x55f35fc7be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/2459275412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f89e280 0x55f35fc70460 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/2459275412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f8960a0 0x55f35fc6ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2459275412 >> 172.21.15.67:0/2459275412 conn(0x55f35f8b3350 msgr2=0x55f35fc67580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:13.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcae5ffb640 0 client.15282 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:50:13.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2459275412 shutdown_connections 2024-07-31T14:50:13.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2459275412 wait complete. 2024-07-31T14:50:13.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.375+0000 7fcb0aca3740 20 client.15282 trim_cache size 0 max 16384 2024-07-31T14:50:13.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:12 smithi073 ceph-mon[36714]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:13.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.394+0000 7fcb0aca3740 1 Processor -- start 2024-07-31T14:50:13.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.394+0000 7fcb0aca3740 1 -- start start 2024-07-31T14:50:13.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.394+0000 7fcb0aca3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f8960a0 0x55f35fc81c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.394+0000 7fcb0aca3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f89e280 0x55f35fc7bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.394+0000 7fcb0aca3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 0x55f35fc61aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.394+0000 7fcb0aca3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f35fc80610 con 0x55f35f8960a0 2024-07-31T14:50:13.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.394+0000 7fcb0aca3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f35fc80ce0 con 0x55f35f8bef60 2024-07-31T14:50:13.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.394+0000 7fcb0aca3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f35fc813b0 con 0x55f35f89e280 2024-07-31T14:50:13.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcb024ee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f8960a0 0x55f35fc81c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcb024ee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f8960a0 0x55f35fc81c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35674/0 (socket says 172.21.15.67:35674) 2024-07-31T14:50:13.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcb024ee640 1 -- 172.21.15.67:0/635793454 learned_addr learned my addr 172.21.15.67:0/635793454 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:13.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcb02cef640 1 --2- 172.21.15.67:0/635793454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 0x55f35fc61aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcb01ced640 1 --2- 172.21.15.67:0/635793454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f89e280 0x55f35fc7bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcb024ee640 1 -- 172.21.15.67:0/635793454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f89e280 msgr2=0x55f35fc7bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:13.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcb024ee640 1 --2- 172.21.15.67:0/635793454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f89e280 0x55f35fc7bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcb024ee640 1 -- 172.21.15.67:0/635793454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 msgr2=0x55f35fc61aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:13.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcb024ee640 1 --2- 172.21.15.67:0/635793454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 0x55f35fc61aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcb024ee640 1 -- 172.21.15.67:0/635793454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f35fc6a9a0 con 0x55f35f8960a0 2024-07-31T14:50:13.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcb02cef640 1 --2- 172.21.15.67:0/635793454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 0x55f35fc61aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:50:13.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcb024ee640 1 --2- 172.21.15.67:0/635793454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f8960a0 0x55f35fc81c00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80cb200 tx=0x7fcaf8101f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3345e029941393b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:13.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcae7fff640 1 -- 172.21.15.67:0/635793454 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8049040 con 0x55f35f8960a0 2024-07-31T14:50:13.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcae7fff640 1 -- 172.21.15.67:0/635793454 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf803b460 con 0x55f35f8960a0 2024-07-31T14:50:13.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.395+0000 7fcae7fff640 1 -- 172.21.15.67:0/635793454 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf80cb500 con 0x55f35f8960a0 2024-07-31T14:50:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.396+0000 7fcb0aca3740 1 -- 172.21.15.67:0/635793454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f8960a0 msgr2=0x55f35fc81c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.396+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/635793454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f8960a0 0x55f35fc81c00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80cb200 tx=0x7fcaf8101f80 comp rx=0 tx=0).stop 2024-07-31T14:50:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.396+0000 7fcb0aca3740 1 -- 172.21.15.67:0/635793454 shutdown_connections 2024-07-31T14:50:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.396+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/635793454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8bef60 0x55f35fc61aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.396+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/635793454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f89e280 0x55f35fc7bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.396+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/635793454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f8960a0 0x55f35fc81c00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.396+0000 7fcb0aca3740 1 -- 172.21.15.67:0/635793454 >> 172.21.15.67:0/635793454 conn(0x55f35f8b3350 msgr2=0x55f35fc6e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.396+0000 7fcb0aca3740 1 -- 172.21.15.67:0/635793454 shutdown_connections 2024-07-31T14:50:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.396+0000 7fcb0aca3740 1 -- 172.21.15.67:0/635793454 wait complete. 2024-07-31T14:50:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb0aca3740 1 -- start start 2024-07-31T14:50:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb0aca3740 1 Processor -- start 2024-07-31T14:50:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb0aca3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8960a0 0x55f35fc60a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb0aca3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f89e280 0x55f35fc61010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb0aca3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f8bef60 0x55f35fc88f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb0aca3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f35fc82150 con 0x55f35f8bef60 2024-07-31T14:50:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb0aca3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f35fc81a80 con 0x55f35f8960a0 2024-07-31T14:50:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb0aca3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f35fc82820 con 0x55f35f89e280 2024-07-31T14:50:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb0aca3740 10 client.0 _ll_register_callbacks cb 0x7fcaec009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb02cef640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f8bef60 0x55f35fc88f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb01ced640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f89e280 0x55f35fc61010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb02cef640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f8bef60 0x55f35fc88f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35678/0 (socket says 172.21.15.67:35678) 2024-07-31T14:50:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb02cef640 1 -- 172.21.15.67:0/2700280201 learned_addr learned my addr 172.21.15.67:0/2700280201 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb024ee640 1 --2- 172.21.15.67:0/2700280201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8960a0 0x55f35fc60a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb02cef640 1 -- 172.21.15.67:0/2700280201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f89e280 msgr2=0x55f35fc61010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb02cef640 1 --2- 172.21.15.67:0/2700280201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f89e280 0x55f35fc61010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb02cef640 1 -- 172.21.15.67:0/2700280201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8960a0 msgr2=0x55f35fc60a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb02cef640 1 --2- 172.21.15.67:0/2700280201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8960a0 0x55f35fc60a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.397+0000 7fcb02cef640 1 -- 172.21.15.67:0/2700280201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f35fc895c0 con 0x55f35f8bef60 2024-07-31T14:50:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.398+0000 7fcb024ee640 1 --2- 172.21.15.67:0/2700280201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8960a0 0x55f35fc60a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:50:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.398+0000 7fcb02cef640 1 --2- 172.21.15.67:0/2700280201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f8bef60 0x55f35fc88f50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcaf401f4f0 tx=0x7fcaf4004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b08121f519d741d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.398+0000 7fcae5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.398+0000 7fcae5ffb640 1 -- 172.21.15.67:0/2700280201 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf402c060 con 0x55f35f8bef60 2024-07-31T14:50:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.398+0000 7fcae5ffb640 1 -- 172.21.15.67:0/2700280201 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fcaf401d900 con 0x55f35f8bef60 2024-07-31T14:50:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.398+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2700280201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f35fbd8330 con 0x55f35f8bef60 2024-07-31T14:50:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.398+0000 7fcae5ffb640 1 -- 172.21.15.67:0/2700280201 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcaf4033050 con 0x55f35f8bef60 2024-07-31T14:50:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.398+0000 7fcb0aca3740 10 client.15288 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.398+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2700280201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f35fc8aa10 con 0x55f35f8bef60 2024-07-31T14:50:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.398+0000 7fcae5ffb640 1 -- 172.21.15.67:0/2700280201 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f35fbd8330 con 0x55f35f8bef60 2024-07-31T14:50:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.399+0000 7fcae5ffb640 1 -- 172.21.15.67:0/2700280201 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcaf4026030 con 0x55f35f8bef60 2024-07-31T14:50:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.399+0000 7fcb0aca3740 10 client.15288 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.399+0000 7fcb0aca3740 10 client.15288 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-31T14:50:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.399+0000 7fcb0aca3740 20 client.15288 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.399+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/2700280201 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55f35f852c90 0x55f35fcee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.399+0000 7fcb0aca3740 4 client.15288 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:50:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.399+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2700280201 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "status"}) -- 0x55f35fbd5a70 con 0x55f35f852c90 2024-07-31T14:50:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.399+0000 7fcb024ee640 1 --2- 172.21.15.67:0/2700280201 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55f35f852c90 0x55f35fcee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.399+0000 7fcae5ffb640 10 client.15288 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:50:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.399+0000 7fcb024ee640 1 --2- 172.21.15.67:0/2700280201 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55f35f852c90 0x55f35fcee4a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c49dfedeaf1ae1fc server_cookie=5e77ec482c94cdf5 in_seq=0 out_seq=0 2024-07-31T14:50:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcae5ffb640 1 -- 172.21.15.67:0/2700280201 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f35fbd5a70 con 0x55f35f852c90 2024-07-31T14:50:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcae5ffb640 10 client.15288 handle_command_reply: tid=0 2024-07-31T14:50:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcb0aca3740 1 client.15288 shutdown 2024-07-31T14:50:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2700280201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f8bef60 msgr2=0x55f35fc88f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/2700280201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f8bef60 0x55f35fc88f50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcaf401f4f0 tx=0x7fcaf4004b50 comp rx=0 tx=0).stop 2024-07-31T14:50:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2700280201 shutdown_connections 2024-07-31T14:50:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/2700280201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f35f8bef60 0x55f35fc88f50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/2700280201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f35f89e280 0x55f35fc61010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/2700280201 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55f35f852c90 0x55f35fcee4a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcb0aca3740 1 --2- 172.21.15.67:0/2700280201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35f8960a0 0x55f35fc60a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcae5ffb640 0 client.15288 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:50:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2700280201 >> 172.21.15.67:0/2700280201 conn(0x55f35f8b3350 msgr2=0x55f35fc61aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2700280201 shutdown_connections 2024-07-31T14:50:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcb0aca3740 1 -- 172.21.15.67:0/2700280201 wait complete. 2024-07-31T14:50:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.400+0000 7fcb0aca3740 20 client.15288 trim_cache size 0 max 16384 2024-07-31T14:50:13.407 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:50:13.407 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:50:13.407 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 2, 2024-07-31T14:50:13.407 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24650, 2024-07-31T14:50:13.407 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:50:13.407 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:50:13.407 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:50:13.407 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 132.83908588700001, 2024-07-31T14:50:13.407 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-31T14:50:13.407 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:50:13.407 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:50:13.407 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 214.613367318 2024-07-31T14:50:13.407 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:50:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.401+0000 7fcb04752640 1 -- 172.21.15.67:0/1397939111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcae80b3c20 msgr2=0x7fcae80b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.401+0000 7fcb04752640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcae80b3c20 0x7fcae80b6130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400f400 tx=0x7fcaf4007740 comp rx=0 tx=0).stop 2024-07-31T14:50:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.401+0000 7fcb04752640 1 -- 172.21.15.67:0/1397939111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcafc1024b0 msgr2=0x7fcafc199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.401+0000 7fcb04752640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcafc1024b0 0x7fcafc199f40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8005940 tx=0x7fcaf80042f0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.401+0000 7fcb04752640 1 -- 172.21.15.67:0/1397939111 shutdown_connections 2024-07-31T14:50:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.401+0000 7fcb04752640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcae80b3c20 0x7fcae80b6130 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.401+0000 7fcb04752640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcafc108740 0x7fcafc19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.401+0000 7fcb04752640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcafc107df0 0x7fcafc19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.401+0000 7fcb04752640 1 --2- 172.21.15.67:0/1397939111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcafc1024b0 0x7fcafc199f40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:13.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.401+0000 7fcb04752640 1 -- 172.21.15.67:0/1397939111 >> 172.21.15.67:0/1397939111 conn(0x7fcafc079170 msgr2=0x7fcafc106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:13.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.401+0000 7fcb04752640 1 -- 172.21.15.67:0/1397939111 shutdown_connections 2024-07-31T14:50:13.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:13.401+0000 7fcb04752640 1 -- 172.21.15.67:0/1397939111 wait complete. 2024-07-31T14:50:14.209 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.i status 2024-07-31T14:50:14.477 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:14 smithi077 ceph-mon[37327]: pgmap v374: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:15.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:14 smithi067 ceph-mon[32750]: pgmap v374: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:15.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:14 smithi073 ceph-mon[36714]: pgmap v374: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:16.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.304+0000 7ff48c627640 1 -- 172.21.15.67:0/2303429226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff484102490 msgr2=0x7ff4841028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:16.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.304+0000 7ff48c627640 1 --2- 172.21.15.67:0/2303429226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff484102490 0x7ff4841028a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff478009900 tx=0x7ff47802a410 comp rx=0 tx=0).stop 2024-07-31T14:50:16.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.305+0000 7ff48c627640 1 -- 172.21.15.67:0/2303429226 shutdown_connections 2024-07-31T14:50:16.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.305+0000 7ff48c627640 1 --2- 172.21.15.67:0/2303429226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff484108720 0x7ff48410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.305+0000 7ff48c627640 1 --2- 172.21.15.67:0/2303429226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff484107dd0 0x7ff4841081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.305+0000 7ff48c627640 1 --2- 172.21.15.67:0/2303429226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff484102490 0x7ff4841028a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.305+0000 7ff48c627640 1 -- 172.21.15.67:0/2303429226 >> 172.21.15.67:0/2303429226 conn(0x7ff484079160 msgr2=0x7ff484079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:16.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.305+0000 7ff48c627640 1 -- 172.21.15.67:0/2303429226 shutdown_connections 2024-07-31T14:50:16.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.305+0000 7ff48c627640 1 -- 172.21.15.67:0/2303429226 wait complete. 2024-07-31T14:50:16.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.305+0000 7ff48c627640 1 Processor -- start 2024-07-31T14:50:16.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.305+0000 7ff48c627640 1 -- start start 2024-07-31T14:50:16.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48c627640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff484102490 0x7ff48419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48c627640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff484107dd0 0x7ff48419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48c627640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff484108720 0x7ff48419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48c627640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841123a0 con 0x7ff484102490 2024-07-31T14:50:16.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48c627640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff484112220 con 0x7ff484108720 2024-07-31T14:50:16.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48c627640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff484112520 con 0x7ff484107dd0 2024-07-31T14:50:16.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48a3c3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff484102490 0x7ff48419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48a3c3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff484102490 0x7ff48419ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35694/0 (socket says 172.21.15.67:35694) 2024-07-31T14:50:16.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48a3c3640 1 -- 172.21.15.67:0/425996425 learned_addr learned my addr 172.21.15.67:0/425996425 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:16.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48abc4640 1 --2- 172.21.15.67:0/425996425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff484108720 0x7ff48419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff489bc2640 1 --2- 172.21.15.67:0/425996425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff484107dd0 0x7ff48419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48a3c3640 1 -- 172.21.15.67:0/425996425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff484107dd0 msgr2=0x7ff48419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:16.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48a3c3640 1 --2- 172.21.15.67:0/425996425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff484107dd0 0x7ff48419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48a3c3640 1 -- 172.21.15.67:0/425996425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff484108720 msgr2=0x7ff48419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:16.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48a3c3640 1 --2- 172.21.15.67:0/425996425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff484108720 0x7ff48419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.306+0000 7ff48a3c3640 1 -- 172.21.15.67:0/425996425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4841a6000 con 0x7ff484102490 2024-07-31T14:50:16.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.307+0000 7ff48a3c3640 1 --2- 172.21.15.67:0/425996425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff484102490 0x7ff48419ad70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff478005940 tx=0x7ff4780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:16.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.307+0000 7ff4777fe640 1 -- 172.21.15.67:0/425996425 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47803f030 con 0x7ff484102490 2024-07-31T14:50:16.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.307+0000 7ff48c627640 1 -- 172.21.15.67:0/425996425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4841a6290 con 0x7ff484102490 2024-07-31T14:50:16.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.307+0000 7ff4777fe640 1 -- 172.21.15.67:0/425996425 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff478003cb0 con 0x7ff484102490 2024-07-31T14:50:16.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.307+0000 7ff4777fe640 1 -- 172.21.15.67:0/425996425 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478035070 con 0x7ff484102490 2024-07-31T14:50:16.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.307+0000 7ff48c627640 1 -- 172.21.15.67:0/425996425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4841a6770 con 0x7ff484102490 2024-07-31T14:50:16.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.310+0000 7ff492b78740 1 Processor -- start 2024-07-31T14:50:16.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.310+0000 7ff492b78740 1 -- start start 2024-07-31T14:50:16.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff492b78740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d625f60 0x55f09d9cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff4777fe640 1 -- 172.21.15.67:0/425996425 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7ff478042030 con 0x7ff484102490 2024-07-31T14:50:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff492b78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d5fd0a0 0x55f09d9cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff492b78740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d605280 0x55f09d9cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff492b78740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f09d941920 con 0x55f09d625f60 2024-07-31T14:50:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff492b78740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f09d930e20 con 0x55f09d605280 2024-07-31T14:50:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff492b78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f09d934e60 con 0x55f09d5fd0a0 2024-07-31T14:50:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff48abc4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d625f60 0x55f09d9cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff48abc4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d625f60 0x55f09d9cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35700/0 (socket says 172.21.15.67:35700) 2024-07-31T14:50:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff48abc4640 1 -- 172.21.15.67:0/4176802333 learned_addr learned my addr 172.21.15.67:0/4176802333 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff489bc2640 1 --2- 172.21.15.67:0/4176802333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d605280 0x55f09d9cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff48a3c3640 1 --2- 172.21.15.67:0/4176802333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d5fd0a0 0x55f09d9cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff48abc4640 1 -- 172.21.15.67:0/4176802333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d5fd0a0 msgr2=0x55f09d9cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff48abc4640 1 --2- 172.21.15.67:0/4176802333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d5fd0a0 0x55f09d9cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff48abc4640 1 -- 172.21.15.67:0/4176802333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d605280 msgr2=0x55f09d9cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:16.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff48abc4640 1 --2- 172.21.15.67:0/4176802333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d605280 0x55f09d9cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.311+0000 7ff48abc4640 1 -- 172.21.15.67:0/4176802333 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f09d8d7a40 con 0x55f09d625f60 2024-07-31T14:50:16.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.312+0000 7ff48abc4640 1 --2- 172.21.15.67:0/4176802333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d625f60 0x55f09d9cb380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009f70 tx=0x7ff47c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28d6339f6a927b3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:16.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.312+0000 7ff4777fe640 1 --2- 172.21.15.67:0/425996425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff4700b3c50 0x7ff4700b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.312+0000 7ff474ff9640 1 -- 172.21.15.67:0/4176802333 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c016070 con 0x55f09d625f60 2024-07-31T14:50:16.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.312+0000 7ff474ff9640 1 -- 172.21.15.67:0/4176802333 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff47c004d20 con 0x55f09d625f60 2024-07-31T14:50:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.312+0000 7ff474ff9640 1 -- 172.21.15.67:0/4176802333 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c012790 con 0x55f09d625f60 2024-07-31T14:50:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.312+0000 7ff48a3c3640 1 --2- 172.21.15.67:0/425996425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff4700b3c50 0x7ff4700b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.312+0000 7ff4777fe640 1 -- 172.21.15.67:0/425996425 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff4781044e0 con 0x7ff484102490 2024-07-31T14:50:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.313+0000 7ff492b78740 1 -- 172.21.15.67:0/4176802333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d625f60 msgr2=0x55f09d9cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.313+0000 7ff48a3c3640 1 --2- 172.21.15.67:0/425996425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff4700b3c50 0x7ff4700b6160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55f09d9ca000 tx=0x7ff47810a060 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.313+0000 7ff492b78740 1 --2- 172.21.15.67:0/4176802333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d625f60 0x55f09d9cb380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009f70 tx=0x7ff47c010800 comp rx=0 tx=0).stop 2024-07-31T14:50:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.313+0000 7ff492b78740 1 -- 172.21.15.67:0/4176802333 shutdown_connections 2024-07-31T14:50:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.313+0000 7ff492b78740 1 --2- 172.21.15.67:0/4176802333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d605280 0x55f09d9cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.313+0000 7ff492b78740 1 --2- 172.21.15.67:0/4176802333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d5fd0a0 0x55f09d9cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.313+0000 7ff492b78740 1 --2- 172.21.15.67:0/4176802333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d625f60 0x55f09d9cb380 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.313+0000 7ff492b78740 1 -- 172.21.15.67:0/4176802333 >> 172.21.15.67:0/4176802333 conn(0x55f09d61a350 msgr2=0x55f09d9c9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.313+0000 7ff492b78740 1 -- 172.21.15.67:0/4176802333 shutdown_connections 2024-07-31T14:50:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.313+0000 7ff492b78740 1 -- 172.21.15.67:0/4176802333 wait complete. 2024-07-31T14:50:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.313+0000 7ff492b78740 1 -- start start 2024-07-31T14:50:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff492b78740 1 Processor -- start 2024-07-31T14:50:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff492b78740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d5fd0a0 0x55f09d9d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff492b78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d605280 0x55f09d9d74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff492b78740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d625f60 0x55f09d9e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff492b78740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f09d7945f0 con 0x55f09d5fd0a0 2024-07-31T14:50:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff492b78740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f09d8d3890 con 0x55f09d625f60 2024-07-31T14:50:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff492b78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f09d9d9b60 con 0x55f09d605280 2024-07-31T14:50:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff492b78740 10 client.0 _ll_register_callbacks cb 0x55f09d8dd7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff48abc4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d5fd0a0 0x55f09d9d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff48abc4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d5fd0a0 0x55f09d9d6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35706/0 (socket says 172.21.15.67:35706) 2024-07-31T14:50:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff48abc4640 1 -- 172.21.15.67:0/81699585 learned_addr learned my addr 172.21.15.67:0/81699585 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff489bc2640 1 --2- 172.21.15.67:0/81699585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d605280 0x55f09d9d74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff48abc4640 1 -- 172.21.15.67:0/81699585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d605280 msgr2=0x55f09d9d74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff48abc4640 1 --2- 172.21.15.67:0/81699585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d605280 0x55f09d9d74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff48abc4640 1 -- 172.21.15.67:0/81699585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d625f60 msgr2=0x55f09d9e2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff48abc4640 1 --2- 172.21.15.67:0/81699585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d625f60 0x55f09d9e2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff48abc4640 1 -- 172.21.15.67:0/81699585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f09d691540 con 0x55f09d5fd0a0 2024-07-31T14:50:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.314+0000 7ff489bc2640 1 --2- 172.21.15.67:0/81699585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d605280 0x55f09d9d74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:50:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.315+0000 7ff48abc4640 1 --2- 172.21.15.67:0/81699585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d5fd0a0 0x55f09d9d6fa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff47c005a80 tx=0x7ff47c004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9287819bb520ce server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.315+0000 7ff46dffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.315+0000 7ff46dffb640 1 -- 172.21.15.67:0/81699585 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c016070 con 0x55f09d5fd0a0 2024-07-31T14:50:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.315+0000 7ff46dffb640 1 -- 172.21.15.67:0/81699585 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff47c0054b0 con 0x55f09d5fd0a0 2024-07-31T14:50:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.315+0000 7ff492b78740 1 -- 172.21.15.67:0/81699585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff47c004d20 con 0x55f09d5fd0a0 2024-07-31T14:50:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.315+0000 7ff46dffb640 1 -- 172.21.15.67:0/81699585 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff47c0056b0 con 0x55f09d5fd0a0 2024-07-31T14:50:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.315+0000 7ff492b78740 10 client.15300 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:16.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.315+0000 7ff492b78740 1 -- 172.21.15.67:0/81699585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f09d9e5bb0 con 0x55f09d5fd0a0 2024-07-31T14:50:16.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.315+0000 7ff46dffb640 1 -- 172.21.15.67:0/81699585 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff47c004d20 con 0x55f09d5fd0a0 2024-07-31T14:50:16.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.315+0000 7ff46dffb640 1 -- 172.21.15.67:0/81699585 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff47c00f040 con 0x55f09d5fd0a0 2024-07-31T14:50:16.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.315+0000 7ff492b78740 10 client.15300 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:16.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.316+0000 7ff492b78740 10 client.15300 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-31T14:50:16.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.316+0000 7ff492b78740 20 client.15300 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:16.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.316+0000 7ff492b78740 1 --2- 172.21.15.67:0/81699585 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f09d5e48e0 0x55f09d9e91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.316+0000 7ff492b78740 4 client.15300 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:50:16.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.316+0000 7ff492b78740 1 -- 172.21.15.67:0/81699585 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f09d943870 con 0x55f09d5e48e0 2024-07-31T14:50:16.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.316+0000 7ff489bc2640 1 --2- 172.21.15.67:0/81699585 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f09d5e48e0 0x55f09d9e91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.316+0000 7ff489bc2640 1 --2- 172.21.15.67:0/81699585 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f09d5e48e0 0x55f09d9e91e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29c63e3205223e9a server_cookie=c42ccba7212cacfd in_seq=0 out_seq=0 2024-07-31T14:50:16.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.316+0000 7ff46dffb640 10 client.15300 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:50:16.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.317+0000 7ff46dffb640 1 -- 172.21.15.67:0/81699585 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f09d943870 con 0x55f09d5e48e0 2024-07-31T14:50:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.317+0000 7ff46dffb640 10 client.15300 handle_command_reply: tid=0 2024-07-31T14:50:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.317+0000 7ff492b78740 1 client.15300 shutdown 2024-07-31T14:50:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.317+0000 7ff492b78740 1 -- 172.21.15.67:0/81699585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d5fd0a0 msgr2=0x55f09d9d6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.317+0000 7ff492b78740 1 --2- 172.21.15.67:0/81699585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d5fd0a0 0x55f09d9d6fa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff47c005a80 tx=0x7ff47c004c50 comp rx=0 tx=0).stop 2024-07-31T14:50:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.317+0000 7ff492b78740 1 -- 172.21.15.67:0/81699585 shutdown_connections 2024-07-31T14:50:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.317+0000 7ff492b78740 1 --2- 172.21.15.67:0/81699585 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f09d5e48e0 0x55f09d9e91e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.317+0000 7ff492b78740 1 --2- 172.21.15.67:0/81699585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d625f60 0x55f09d9e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.317+0000 7ff492b78740 1 --2- 172.21.15.67:0/81699585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d605280 0x55f09d9d74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.317+0000 7ff492b78740 1 --2- 172.21.15.67:0/81699585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d5fd0a0 0x55f09d9d6fa0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.317+0000 7ff46dffb640 0 client.15300 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:50:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.317+0000 7ff492b78740 1 -- 172.21.15.67:0/81699585 >> 172.21.15.67:0/81699585 conn(0x55f09d61a350 msgr2=0x55f09d9ce560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.318+0000 7ff492b78740 1 -- 172.21.15.67:0/81699585 shutdown_connections 2024-07-31T14:50:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.318+0000 7ff492b78740 1 -- 172.21.15.67:0/81699585 wait complete. 2024-07-31T14:50:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.318+0000 7ff492b78740 20 client.15300 trim_cache size 0 max 16384 2024-07-31T14:50:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff492b78740 1 Processor -- start 2024-07-31T14:50:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff492b78740 1 -- start start 2024-07-31T14:50:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff492b78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d5fd0a0 0x55f09d9e7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff492b78740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d605280 0x55f09d9e2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff492b78740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d625f60 0x55f09d9d5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff492b78740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f09d9e7520 con 0x55f09d605280 2024-07-31T14:50:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff492b78740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f09d9e7bf0 con 0x55f09d625f60 2024-07-31T14:50:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff492b78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f09d9e82c0 con 0x55f09d5fd0a0 2024-07-31T14:50:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff489bc2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d605280 0x55f09d9e2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff489bc2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d605280 0x55f09d9e2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35716/0 (socket says 172.21.15.67:35716) 2024-07-31T14:50:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff489bc2640 1 -- 172.21.15.67:0/1428943054 learned_addr learned my addr 172.21.15.67:0/1428943054 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff48abc4640 1 --2- 172.21.15.67:0/1428943054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d5fd0a0 0x55f09d9e7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff48abc4640 1 --2- 172.21.15.67:0/1428943054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d625f60 0x55f09d9d5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff489bc2640 1 -- 172.21.15.67:0/1428943054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d5fd0a0 msgr2=0x55f09d9e7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff489bc2640 1 --2- 172.21.15.67:0/1428943054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d5fd0a0 0x55f09d9e7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff489bc2640 1 -- 172.21.15.67:0/1428943054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d625f60 msgr2=0x55f09d9d5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff489bc2640 1 --2- 172.21.15.67:0/1428943054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d625f60 0x55f09d9d5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff489bc2640 1 -- 172.21.15.67:0/1428943054 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f09d9d7940 con 0x55f09d605280 2024-07-31T14:50:16.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.337+0000 7ff48abc4640 1 --2- 172.21.15.67:0/1428943054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d625f60 0x55f09d9d5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:50:16.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.338+0000 7ff489bc2640 1 --2- 172.21.15.67:0/1428943054 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d605280 0x55f09d9e2df0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff4841007c0 tx=0x7ff480010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdc734e06bc9a94 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:16.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.338+0000 7ff46ffff640 1 -- 172.21.15.67:0/1428943054 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480004220 con 0x55f09d605280 2024-07-31T14:50:16.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.338+0000 7ff46ffff640 1 -- 172.21.15.67:0/1428943054 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4800043c0 con 0x55f09d605280 2024-07-31T14:50:16.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.338+0000 7ff46ffff640 1 -- 172.21.15.67:0/1428943054 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48001e8a0 con 0x55f09d605280 2024-07-31T14:50:16.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.338+0000 7ff492b78740 1 -- 172.21.15.67:0/1428943054 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d605280 msgr2=0x55f09d9e2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.338+0000 7ff492b78740 1 --2- 172.21.15.67:0/1428943054 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d605280 0x55f09d9e2df0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff4841007c0 tx=0x7ff480010800 comp rx=0 tx=0).stop 2024-07-31T14:50:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 -- 172.21.15.67:0/1428943054 shutdown_connections 2024-07-31T14:50:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 --2- 172.21.15.67:0/1428943054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d625f60 0x55f09d9d5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 --2- 172.21.15.67:0/1428943054 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d605280 0x55f09d9e2df0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 --2- 172.21.15.67:0/1428943054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d5fd0a0 0x55f09d9e7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 -- 172.21.15.67:0/1428943054 >> 172.21.15.67:0/1428943054 conn(0x55f09d61a350 msgr2=0x55f09d9d1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 -- 172.21.15.67:0/1428943054 shutdown_connections 2024-07-31T14:50:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 -- 172.21.15.67:0/1428943054 wait complete. 2024-07-31T14:50:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 -- start start 2024-07-31T14:50:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 Processor -- start 2024-07-31T14:50:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d5fd0a0 0x55f09d9ec060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d605280 0x55f09d9ec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d625f60 0x55f09d9f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f09d9e9060 con 0x55f09d625f60 2024-07-31T14:50:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f09d9e8990 con 0x55f09d5fd0a0 2024-07-31T14:50:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f09d9e9730 con 0x55f09d605280 2024-07-31T14:50:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.339+0000 7ff492b78740 10 client.0 _ll_register_callbacks cb 0x7ff47c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff48abc4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d625f60 0x55f09d9f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff48abc4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d625f60 0x55f09d9f00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35726/0 (socket says 172.21.15.67:35726) 2024-07-31T14:50:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff48abc4640 1 -- 172.21.15.67:0/2062485047 learned_addr learned my addr 172.21.15.67:0/2062485047 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff48abc4640 1 --2- 172.21.15.67:0/2062485047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d5fd0a0 0x55f09d9ec060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff489bc2640 1 --2- 172.21.15.67:0/2062485047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d605280 0x55f09d9ec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff48abc4640 1 -- 172.21.15.67:0/2062485047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d605280 msgr2=0x55f09d9ec620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff48abc4640 1 --2- 172.21.15.67:0/2062485047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d605280 0x55f09d9ec620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff48abc4640 1 -- 172.21.15.67:0/2062485047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d5fd0a0 msgr2=0x55f09d9ec060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff48abc4640 1 --2- 172.21.15.67:0/2062485047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d5fd0a0 0x55f09d9ec060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff48abc4640 1 -- 172.21.15.67:0/2062485047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f09d9f0740 con 0x55f09d625f60 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff48abc4640 1 --2- 172.21.15.67:0/2062485047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d625f60 0x55f09d9f00d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009770 tx=0x7ff47c004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf3288e020ac15e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff46dffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff46dffb640 1 -- 172.21.15.67:0/2062485047 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c016040 con 0x55f09d625f60 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff46dffb640 1 -- 172.21.15.67:0/2062485047 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff47c0052b0 con 0x55f09d625f60 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff492b78740 1 -- 172.21.15.67:0/2062485047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4800043c0 con 0x55f09d625f60 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff46dffb640 1 -- 172.21.15.67:0/2062485047 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff47c0054b0 con 0x55f09d625f60 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff492b78740 10 client.15306 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.340+0000 7ff492b78740 1 -- 172.21.15.67:0/2062485047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f09d9f19e0 con 0x55f09d625f60 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.341+0000 7ff46dffb640 1 -- 172.21.15.67:0/2062485047 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4800043c0 con 0x55f09d625f60 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.341+0000 7ff46dffb640 1 -- 172.21.15.67:0/2062485047 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff47c027c60 con 0x55f09d625f60 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.341+0000 7ff492b78740 10 client.15306 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.341+0000 7ff492b78740 10 client.15306 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.341+0000 7ff492b78740 20 client.15306 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.341+0000 7ff492b78740 1 --2- 172.21.15.67:0/2062485047 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f09d5b9c90 0x55f09da55340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.341+0000 7ff492b78740 4 client.15306 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:50:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.341+0000 7ff492b78740 1 -- 172.21.15.67:0/2062485047 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "status"}) -- 0x55f09d93f310 con 0x55f09d5b9c90 2024-07-31T14:50:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.341+0000 7ff489bc2640 1 --2- 172.21.15.67:0/2062485047 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f09d5b9c90 0x55f09da55340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff489bc2640 1 --2- 172.21.15.67:0/2062485047 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f09d5b9c90 0x55f09da55340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae0a79f22b70d5aa server_cookie=ceff4e7b7af94704 in_seq=0 out_seq=0 2024-07-31T14:50:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff46dffb640 10 client.15306 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:50:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff46dffb640 1 -- 172.21.15.67:0/2062485047 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f09d93f310 con 0x55f09d5b9c90 2024-07-31T14:50:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff46dffb640 10 client.15306 handle_command_reply: tid=0 2024-07-31T14:50:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff492b78740 1 client.15306 shutdown 2024-07-31T14:50:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff492b78740 1 -- 172.21.15.67:0/2062485047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d625f60 msgr2=0x55f09d9f00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff492b78740 1 --2- 172.21.15.67:0/2062485047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d625f60 0x55f09d9f00d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009770 tx=0x7ff47c004c50 comp rx=0 tx=0).stop 2024-07-31T14:50:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff492b78740 1 -- 172.21.15.67:0/2062485047 shutdown_connections 2024-07-31T14:50:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff492b78740 1 --2- 172.21.15.67:0/2062485047 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f09d5b9c90 0x55f09da55340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff492b78740 1 --2- 172.21.15.67:0/2062485047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09d625f60 0x55f09d9f00d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff492b78740 1 --2- 172.21.15.67:0/2062485047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f09d605280 0x55f09d9ec620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff46dffb640 0 client.15306 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:50:16.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff492b78740 1 --2- 172.21.15.67:0/2062485047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f09d5fd0a0 0x55f09d9ec060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff492b78740 1 -- 172.21.15.67:0/2062485047 >> 172.21.15.67:0/2062485047 conn(0x55f09d61a350 msgr2=0x55f09d9d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:16.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff492b78740 1 -- 172.21.15.67:0/2062485047 shutdown_connections 2024-07-31T14:50:16.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff492b78740 1 -- 172.21.15.67:0/2062485047 wait complete. 2024-07-31T14:50:16.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.342+0000 7ff492b78740 20 client.15306 trim_cache size 0 max 16384 2024-07-31T14:50:16.349 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:50:16.349 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:50:16.349 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 1, 2024-07-31T14:50:16.349 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24655, 2024-07-31T14:50:16.349 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:50:16.349 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:50:16.350 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:50:16.350 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 136.786114613, 2024-07-31T14:50:16.350 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-31T14:50:16.350 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:50:16.350 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:50:16.350 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 214.83795494200001 2024-07-31T14:50:16.350 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:50:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.343+0000 7ff48c627640 1 -- 172.21.15.67:0/425996425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff4700b3c50 msgr2=0x7ff4700b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.343+0000 7ff48c627640 1 --2- 172.21.15.67:0/425996425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff4700b3c50 0x7ff4700b6160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55f09d9ca000 tx=0x7ff47810a060 comp rx=0 tx=0).stop 2024-07-31T14:50:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.343+0000 7ff48c627640 1 -- 172.21.15.67:0/425996425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff484102490 msgr2=0x7ff48419ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.343+0000 7ff48c627640 1 --2- 172.21.15.67:0/425996425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff484102490 0x7ff48419ad70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff478005940 tx=0x7ff4780042f0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.343+0000 7ff48c627640 1 -- 172.21.15.67:0/425996425 shutdown_connections 2024-07-31T14:50:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.343+0000 7ff48c627640 1 --2- 172.21.15.67:0/425996425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff4700b3c50 0x7ff4700b6160 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.343+0000 7ff48c627640 1 --2- 172.21.15.67:0/425996425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff484108720 0x7ff48419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.343+0000 7ff48c627640 1 --2- 172.21.15.67:0/425996425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff484107dd0 0x7ff48419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.343+0000 7ff48c627640 1 --2- 172.21.15.67:0/425996425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff484102490 0x7ff48419ad70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:16.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.343+0000 7ff48c627640 1 -- 172.21.15.67:0/425996425 >> 172.21.15.67:0/425996425 conn(0x7ff484079160 msgr2=0x7ff484100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:16.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.343+0000 7ff48c627640 1 -- 172.21.15.67:0/425996425 shutdown_connections 2024-07-31T14:50:16.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:16.343+0000 7ff48c627640 1 -- 172.21.15.67:0/425996425 wait complete. 2024-07-31T14:50:17.069 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.b status 2024-07-31T14:50:17.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:16 smithi077 ceph-mon[37327]: pgmap v375: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:17.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:16 smithi067 ceph-mon[32750]: pgmap v375: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:17.341 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:17.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:16 smithi073 ceph-mon[36714]: pgmap v375: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:18.993 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:18 smithi067 ceph-mon[32750]: pgmap v376: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:18.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.987+0000 7ff8f6cba640 1 -- 172.21.15.67:0/2378356223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f0102950 msgr2=0x7ff8f010f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:18.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.987+0000 7ff8f6cba640 1 --2- 172.21.15.67:0/2378356223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f0102950 0x7ff8f010f670 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009900 tx=0x7ff8dc02a480 comp rx=0 tx=0).stop 2024-07-31T14:50:18.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.987+0000 7ff8f6cba640 1 -- 172.21.15.67:0/2378356223 shutdown_connections 2024-07-31T14:50:18.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.987+0000 7ff8f6cba640 1 --2- 172.21.15.67:0/2378356223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8f010fbb0 0x7ff8f0111fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:18.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.987+0000 7ff8f6cba640 1 --2- 172.21.15.67:0/2378356223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f0102950 0x7ff8f010f670 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:18.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.987+0000 7ff8f6cba640 1 --2- 172.21.15.67:0/2378356223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f0102000 0x7ff8f0102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:18.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.987+0000 7ff8f6cba640 1 -- 172.21.15.67:0/2378356223 >> 172.21.15.67:0/2378356223 conn(0x7ff8f0079160 msgr2=0x7ff8f0079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:18.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.987+0000 7ff8f6cba640 1 -- 172.21.15.67:0/2378356223 shutdown_connections 2024-07-31T14:50:18.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.987+0000 7ff8f6cba640 1 -- 172.21.15.67:0/2378356223 wait complete. 2024-07-31T14:50:18.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.988+0000 7ff8f6cba640 1 Processor -- start 2024-07-31T14:50:18.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.988+0000 7ff8f6cba640 1 -- start start 2024-07-31T14:50:18.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.988+0000 7ff8f6cba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8f0102000 0x7ff8f019c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:18.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.988+0000 7ff8f6cba640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f0102950 0x7ff8f019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:18.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.988+0000 7ff8f6cba640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f010fbb0 0x7ff8f01a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:18.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.988+0000 7ff8f6cba640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f0115670 con 0x7ff8f0102950 2024-07-31T14:50:18.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.988+0000 7ff8f6cba640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f01154f0 con 0x7ff8f010fbb0 2024-07-31T14:50:18.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.988+0000 7ff8f6cba640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f01157f0 con 0x7ff8f0102000 2024-07-31T14:50:18.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.988+0000 7ff8effff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f0102950 0x7ff8f019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:18.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.989+0000 7ff8f4a56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8f0102000 0x7ff8f019c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:18.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.989+0000 7ff8effff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f0102950 0x7ff8f019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35754/0 (socket says 172.21.15.67:35754) 2024-07-31T14:50:18.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.989+0000 7ff8effff640 1 -- 172.21.15.67:0/2912015425 learned_addr learned my addr 172.21.15.67:0/2912015425 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:18.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.989+0000 7ff8f5257640 1 --2- 172.21.15.67:0/2912015425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f010fbb0 0x7ff8f01a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.989+0000 7ff8effff640 1 -- 172.21.15.67:0/2912015425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8f0102000 msgr2=0x7ff8f019c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.989+0000 7ff8effff640 1 --2- 172.21.15.67:0/2912015425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8f0102000 0x7ff8f019c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.989+0000 7ff8effff640 1 -- 172.21.15.67:0/2912015425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f010fbb0 msgr2=0x7ff8f01a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.989+0000 7ff8effff640 1 --2- 172.21.15.67:0/2912015425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f010fbb0 0x7ff8f01a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.989+0000 7ff8effff640 1 -- 172.21.15.67:0/2912015425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f01a1ff0 con 0x7ff8f0102950 2024-07-31T14:50:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.989+0000 7ff8effff640 1 --2- 172.21.15.67:0/2912015425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f0102950 0x7ff8f019c620 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc005940 tx=0x7ff8dc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.989+0000 7ff8edffb640 1 -- 172.21.15.67:0/2912015425 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc03f030 con 0x7ff8f0102950 2024-07-31T14:50:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.989+0000 7ff8edffb640 1 -- 172.21.15.67:0/2912015425 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8dc0044c0 con 0x7ff8f0102950 2024-07-31T14:50:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.989+0000 7ff8f6cba640 1 -- 172.21.15.67:0/2912015425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f01a2280 con 0x7ff8f0102950 2024-07-31T14:50:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.989+0000 7ff8edffb640 1 -- 172.21.15.67:0/2912015425 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc035070 con 0x7ff8f0102950 2024-07-31T14:50:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.990+0000 7ff8f6cba640 1 -- 172.21.15.67:0/2912015425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f01a8a40 con 0x7ff8f0102950 2024-07-31T14:50:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.993+0000 7ff8fd20b740 1 Processor -- start 2024-07-31T14:50:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.993+0000 7ff8fd20b740 1 -- start start 2024-07-31T14:50:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.993+0000 7ff8edffb640 1 -- 172.21.15.67:0/2912015425 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7ff8dc042030 con 0x7ff8f0102950 2024-07-31T14:50:19.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.993+0000 7ff8fd20b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7da1f60 0x55f9b81473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.993+0000 7ff8fd20b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d790a0 0x55f9b81478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.993+0000 7ff8fd20b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d81280 0x55f9b8147e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.993+0000 7ff8fd20b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f9b80bd940 con 0x55f9b7d81280 2024-07-31T14:50:19.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.993+0000 7ff8fd20b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f9b80ace40 con 0x55f9b7da1f60 2024-07-31T14:50:19.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.993+0000 7ff8fd20b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f9b80b0e80 con 0x55f9b7d790a0 2024-07-31T14:50:19.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.993+0000 7ff8f4a56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d790a0 0x55f9b81478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.993+0000 7ff8f4a56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d790a0 0x55f9b81478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43340/0 (socket says 172.21.15.67:43340) 2024-07-31T14:50:19.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.993+0000 7ff8f4a56640 1 -- 172.21.15.67:0/1407516028 learned_addr learned my addr 172.21.15.67:0/1407516028 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:19.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.994+0000 7ff8effff640 1 --2- 172.21.15.67:0/1407516028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d81280 0x55f9b8147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.994+0000 7ff8f5257640 1 --2- 172.21.15.67:0/1407516028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7da1f60 0x55f9b81473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.994+0000 7ff8f4a56640 1 -- 172.21.15.67:0/1407516028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7da1f60 msgr2=0x55f9b81473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.994+0000 7ff8f4a56640 1 --2- 172.21.15.67:0/1407516028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7da1f60 0x55f9b81473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.994+0000 7ff8f4a56640 1 -- 172.21.15.67:0/1407516028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d81280 msgr2=0x55f9b8147e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.994+0000 7ff8f4a56640 1 --2- 172.21.15.67:0/1407516028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d81280 0x55f9b8147e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.994+0000 7ff8f4a56640 1 -- 172.21.15.67:0/1407516028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b8053a60 con 0x55f9b7d790a0 2024-07-31T14:50:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.994+0000 7ff8f5257640 1 --2- 172.21.15.67:0/1407516028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7da1f60 0x55f9b81473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:50:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.994+0000 7ff8f4a56640 1 --2- 172.21.15.67:0/1407516028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d790a0 0x55f9b81478e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009cf0 tx=0x7ff8e4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36255ecd46514abe server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.994+0000 7ff8edffb640 1 --2- 172.21.15.67:0/2912015425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e00b3ca0 0x7ff8e00b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.994+0000 7ff8daffd640 1 -- 172.21.15.67:0/1407516028 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4004230 con 0x55f9b7d790a0 2024-07-31T14:50:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.994+0000 7ff8f5257640 1 --2- 172.21.15.67:0/2912015425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e00b3ca0 0x7ff8e00b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.995+0000 7ff8daffd640 1 -- 172.21.15.67:0/1407516028 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8e40043d0 con 0x55f9b7d790a0 2024-07-31T14:50:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.995+0000 7ff8daffd640 1 -- 172.21.15.67:0/1407516028 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4019780 con 0x55f9b7d790a0 2024-07-31T14:50:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.995+0000 7ff8edffb640 1 -- 172.21.15.67:0/2912015425 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff8dc104320 con 0x7ff8f0102950 2024-07-31T14:50:19.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.996+0000 7ff8fd20b740 1 -- 172.21.15.67:0/1407516028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d790a0 msgr2=0x55f9b81478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:19.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.996+0000 7ff8f5257640 1 --2- 172.21.15.67:0/2912015425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e00b3ca0 0x7ff8e00b61b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8005fd0 tx=0x7ff8e8009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:19.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.996+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/1407516028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d790a0 0x55f9b81478e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009cf0 tx=0x7ff8e4011810 comp rx=0 tx=0).stop 2024-07-31T14:50:19.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.996+0000 7ff8fd20b740 1 -- 172.21.15.67:0/1407516028 shutdown_connections 2024-07-31T14:50:19.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.996+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/1407516028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d81280 0x55f9b8147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.996+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/1407516028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d790a0 0x55f9b81478e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.996+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/1407516028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7da1f60 0x55f9b81473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.996+0000 7ff8fd20b740 1 -- 172.21.15.67:0/1407516028 >> 172.21.15.67:0/1407516028 conn(0x55f9b7d96350 msgr2=0x55f9b8145290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:19.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.996+0000 7ff8fd20b740 1 -- 172.21.15.67:0/1407516028 shutdown_connections 2024-07-31T14:50:19.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.996+0000 7ff8fd20b740 1 -- 172.21.15.67:0/1407516028 wait complete. 2024-07-31T14:50:19.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.996+0000 7ff8fd20b740 1 -- start start 2024-07-31T14:50:19.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.996+0000 7ff8fd20b740 1 Processor -- start 2024-07-31T14:50:19.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8fd20b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d790a0 0x55f9b8152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8fd20b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d81280 0x55f9b8153400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8fd20b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7da1f60 0x55f9b815eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8fd20b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f9b7f105f0 con 0x55f9b7da1f60 2024-07-31T14:50:19.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8fd20b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f9b804f8b0 con 0x55f9b7d81280 2024-07-31T14:50:19.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8fd20b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8155b80 con 0x55f9b7d790a0 2024-07-31T14:50:19.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8fd20b740 10 client.0 _ll_register_callbacks cb 0x55f9b80597f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:19.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8f4a56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d790a0 0x55f9b8152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8f4a56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d790a0 0x55f9b8152ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43346/0 (socket says 172.21.15.67:43346) 2024-07-31T14:50:19.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8f4a56640 1 -- 172.21.15.67:0/2835543352 learned_addr learned my addr 172.21.15.67:0/2835543352 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:19.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8effff640 1 --2- 172.21.15.67:0/2835543352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d81280 0x55f9b8153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8f5257640 1 --2- 172.21.15.67:0/2835543352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7da1f60 0x55f9b815eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8f4a56640 1 -- 172.21.15.67:0/2835543352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d81280 msgr2=0x55f9b8153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:19.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8f4a56640 1 --2- 172.21.15.67:0/2835543352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d81280 0x55f9b8153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8f4a56640 1 -- 172.21.15.67:0/2835543352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7da1f60 msgr2=0x55f9b815eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:19.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8f4a56640 1 --2- 172.21.15.67:0/2835543352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7da1f60 0x55f9b815eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.997+0000 7ff8f4a56640 1 -- 172.21.15.67:0/2835543352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b7e0d540 con 0x55f9b7d790a0 2024-07-31T14:50:19.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.998+0000 7ff8f4a56640 1 --2- 172.21.15.67:0/2835543352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d790a0 0x55f9b8152ec0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4005e00 tx=0x7ff8e4003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a9ffd5a50e651b0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:19.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.998+0000 7ff8cffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:50:19.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.998+0000 7ff8cffff640 1 -- 172.21.15.67:0/2835543352 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4027070 con 0x55f9b7d790a0 2024-07-31T14:50:19.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.998+0000 7ff8cffff640 1 -- 172.21.15.67:0/2835543352 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff8e4005240 con 0x55f9b7d790a0 2024-07-31T14:50:19.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.998+0000 7ff8fd20b740 1 -- 172.21.15.67:0/2835543352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e40043d0 con 0x55f9b7d790a0 2024-07-31T14:50:19.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.998+0000 7ff8cffff640 1 -- 172.21.15.67:0/2835543352 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff8e4033050 con 0x55f9b7d790a0 2024-07-31T14:50:19.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.998+0000 7ff8fd20b740 10 client.25105 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:19.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.998+0000 7ff8fd20b740 1 -- 172.21.15.67:0/2835543352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f9b8161b50 con 0x55f9b7d790a0 2024-07-31T14:50:19.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.998+0000 7ff8cffff640 1 -- 172.21.15.67:0/2835543352 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8e40043d0 con 0x55f9b7d790a0 2024-07-31T14:50:19.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.999+0000 7ff8cffff640 1 -- 172.21.15.67:0/2835543352 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff8e4004c00 con 0x55f9b7d790a0 2024-07-31T14:50:19.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.999+0000 7ff8fd20b740 10 client.25105 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:19.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.999+0000 7ff8fd20b740 10 client.25105 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-31T14:50:19.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.999+0000 7ff8fd20b740 20 client.25105 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:19.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.999+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/2835543352 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f9b7d608e0 0x55f9b81653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.999+0000 7ff8fd20b740 4 client.25105 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.999+0000 7ff8fd20b740 1 -- 172.21.15.67:0/2835543352 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9b80bf890 con 0x55f9b7d608e0 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.999+0000 7ff8effff640 1 --2- 172.21.15.67:0/2835543352 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f9b7d608e0 0x55f9b81653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.999+0000 7ff8cffff640 10 client.25105 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:18.999+0000 7ff8effff640 1 --2- 172.21.15.67:0/2835543352 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f9b7d608e0 0x55f9b81653d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec791d5a550b905c server_cookie=1b12ba5868c58481 in_seq=0 out_seq=0 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.000+0000 7ff8cffff640 1 -- 172.21.15.67:0/2835543352 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9b80bf890 con 0x55f9b7d608e0 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.000+0000 7ff8cffff640 10 client.25105 handle_command_reply: tid=0 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.000+0000 7ff8fd20b740 1 client.25105 shutdown 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.000+0000 7ff8fd20b740 1 -- 172.21.15.67:0/2835543352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d790a0 msgr2=0x55f9b8152ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.000+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/2835543352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d790a0 0x55f9b8152ec0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4005e00 tx=0x7ff8e4003820 comp rx=0 tx=0).stop 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.001+0000 7ff8fd20b740 1 -- 172.21.15.67:0/2835543352 shutdown_connections 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.001+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/2835543352 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f9b7d608e0 0x55f9b81653d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.001+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/2835543352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7da1f60 0x55f9b815eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.001+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/2835543352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d81280 0x55f9b8153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.001+0000 7ff8cffff640 0 client.25105 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:50:19.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.001+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/2835543352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d790a0 0x55f9b8152ec0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.001+0000 7ff8fd20b740 1 -- 172.21.15.67:0/2835543352 >> 172.21.15.67:0/2835543352 conn(0x55f9b7d96350 msgr2=0x55f9b814a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:19.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.001+0000 7ff8fd20b740 1 -- 172.21.15.67:0/2835543352 shutdown_connections 2024-07-31T14:50:19.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.001+0000 7ff8fd20b740 1 -- 172.21.15.67:0/2835543352 wait complete. 2024-07-31T14:50:19.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.001+0000 7ff8fd20b740 20 client.25105 trim_cache size 0 max 16384 2024-07-31T14:50:19.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.020+0000 7ff8fd20b740 1 Processor -- start 2024-07-31T14:50:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.020+0000 7ff8fd20b740 1 -- start start 2024-07-31T14:50:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.020+0000 7ff8fd20b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d608e0 0x55f9b81653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.020+0000 7ff8fd20b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d790a0 0x55f9b815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.020+0000 7ff8fd20b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d81280 0x55f9b815f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8fd20b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8163710 con 0x55f9b7d790a0 2024-07-31T14:50:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8fd20b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8163de0 con 0x55f9b7d608e0 2024-07-31T14:50:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8fd20b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f9b81644b0 con 0x55f9b7d81280 2024-07-31T14:50:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8effff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d790a0 0x55f9b815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8effff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d790a0 0x55f9b815ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35782/0 (socket says 172.21.15.67:35782) 2024-07-31T14:50:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8effff640 1 -- 172.21.15.67:0/3858399462 learned_addr learned my addr 172.21.15.67:0/3858399462 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8f5257640 1 --2- 172.21.15.67:0/3858399462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d81280 0x55f9b815f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8f4a56640 1 --2- 172.21.15.67:0/3858399462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d608e0 0x55f9b81653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8effff640 1 -- 172.21.15.67:0/3858399462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d81280 msgr2=0x55f9b815f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8effff640 1 --2- 172.21.15.67:0/3858399462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d81280 0x55f9b815f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8effff640 1 -- 172.21.15.67:0/3858399462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d608e0 msgr2=0x55f9b81653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8effff640 1 --2- 172.21.15.67:0/3858399462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d608e0 0x55f9b81653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8effff640 1 -- 172.21.15.67:0/3858399462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b814e550 con 0x55f9b7d790a0 2024-07-31T14:50:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8effff640 1 --2- 172.21.15.67:0/3858399462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d790a0 0x55f9b815ee40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0cabf0 tx=0x7ff8dc1018c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499c081f9f43c9b5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8da7fc640 1 -- 172.21.15.67:0/3858399462 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0cd070 con 0x55f9b7d790a0 2024-07-31T14:50:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8da7fc640 1 -- 172.21.15.67:0/3858399462 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8dc0caa00 con 0x55f9b7d790a0 2024-07-31T14:50:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.021+0000 7ff8da7fc640 1 -- 172.21.15.67:0/3858399462 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc049050 con 0x55f9b7d790a0 2024-07-31T14:50:19.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.022+0000 7ff8fd20b740 1 -- 172.21.15.67:0/3858399462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d790a0 msgr2=0x55f9b815ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:19.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.022+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/3858399462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d790a0 0x55f9b815ee40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0cabf0 tx=0x7ff8dc1018c0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.022+0000 7ff8fd20b740 1 -- 172.21.15.67:0/3858399462 shutdown_connections 2024-07-31T14:50:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.022+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/3858399462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d81280 0x55f9b815f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.022+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/3858399462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d790a0 0x55f9b815ee40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.022+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/3858399462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d608e0 0x55f9b81653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.022+0000 7ff8fd20b740 1 -- 172.21.15.67:0/3858399462 >> 172.21.15.67:0/3858399462 conn(0x55f9b7d96350 msgr2=0x55f9b8162f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.022+0000 7ff8fd20b740 1 -- 172.21.15.67:0/3858399462 shutdown_connections 2024-07-31T14:50:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.022+0000 7ff8fd20b740 1 -- 172.21.15.67:0/3858399462 wait complete. 2024-07-31T14:50:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.023+0000 7ff8fd20b740 1 -- start start 2024-07-31T14:50:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.023+0000 7ff8fd20b740 1 Processor -- start 2024-07-31T14:50:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.023+0000 7ff8fd20b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d608e0 0x55f9b81665d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.023+0000 7ff8fd20b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d790a0 0x55f9b8166b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.023+0000 7ff8fd20b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d81280 0x55f9b8167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.023+0000 7ff8fd20b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8165250 con 0x55f9b7d790a0 2024-07-31T14:50:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.023+0000 7ff8fd20b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8164b80 con 0x55f9b7d608e0 2024-07-31T14:50:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.023+0000 7ff8fd20b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8165920 con 0x55f9b7d81280 2024-07-31T14:50:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.023+0000 7ff8fd20b740 10 client.0 _ll_register_callbacks cb 0x7ff8e4005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.023+0000 7ff8f5257640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d81280 0x55f9b8167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.023+0000 7ff8effff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d790a0 0x55f9b8166b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.023+0000 7ff8f5257640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d81280 0x55f9b8167050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43368/0 (socket says 172.21.15.67:43368) 2024-07-31T14:50:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.023+0000 7ff8f4a56640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d608e0 0x55f9b81665d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.023+0000 7ff8f5257640 1 -- 172.21.15.67:0/1329789694 learned_addr learned my addr 172.21.15.67:0/1329789694 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.024+0000 7ff8effff640 1 -- 172.21.15.67:0/1329789694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d81280 msgr2=0x55f9b8167050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.024+0000 7ff8effff640 1 --2- 172.21.15.67:0/1329789694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d81280 0x55f9b8167050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.024+0000 7ff8effff640 1 -- 172.21.15.67:0/1329789694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d608e0 msgr2=0x55f9b81665d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.024+0000 7ff8effff640 1 --2- 172.21.15.67:0/1329789694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d608e0 0x55f9b81665d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.024+0000 7ff8effff640 1 -- 172.21.15.67:0/1329789694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b81cc660 con 0x55f9b7d790a0 2024-07-31T14:50:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.024+0000 7ff8f5257640 1 --2- 172.21.15.67:0/1329789694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d81280 0x55f9b8167050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:50:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.024+0000 7ff8effff640 1 --2- 172.21.15.67:0/1329789694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d790a0 0x55f9b8166b10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0cb0b0 tx=0x7ff8dc005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9f4140ba71776 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.024+0000 7ff8cffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.024+0000 7ff8cffff640 1 -- 172.21.15.67:0/1329789694 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0cd070 con 0x55f9b7d790a0 2024-07-31T14:50:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.024+0000 7ff8cffff640 1 -- 172.21.15.67:0/1329789694 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff8dc0c7180 con 0x55f9b7d790a0 2024-07-31T14:50:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.024+0000 7ff8fd20b740 1 -- 172.21.15.67:0/1329789694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8dc0caa00 con 0x55f9b7d790a0 2024-07-31T14:50:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.024+0000 7ff8cffff640 1 -- 172.21.15.67:0/1329789694 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff8dc049070 con 0x55f9b7d790a0 2024-07-31T14:50:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.024+0000 7ff8fd20b740 10 client.15324 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.024+0000 7ff8fd20b740 1 -- 172.21.15.67:0/1329789694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f9b81cd820 con 0x55f9b7d790a0 2024-07-31T14:50:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.025+0000 7ff8cffff640 1 -- 172.21.15.67:0/1329789694 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8dc0caa00 con 0x55f9b7d790a0 2024-07-31T14:50:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.025+0000 7ff8cffff640 1 -- 172.21.15.67:0/1329789694 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff8dc0d3af0 con 0x55f9b7d790a0 2024-07-31T14:50:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.025+0000 7ff8fd20b740 10 client.15324 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.025+0000 7ff8fd20b740 10 client.15324 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-31T14:50:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.025+0000 7ff8fd20b740 20 client.15324 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.025+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/1329789694 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f9b7da1f60 0x55f9b81d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.025+0000 7ff8fd20b740 4 client.15324 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:50:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.025+0000 7ff8fd20b740 1 -- 172.21.15.67:0/1329789694 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "status"}) -- 0x55f9b80bb330 con 0x55f9b7da1f60 2024-07-31T14:50:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.025+0000 7ff8f4a56640 1 --2- 172.21.15.67:0/1329789694 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f9b7da1f60 0x55f9b81d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8f4a56640 1 --2- 172.21.15.67:0/1329789694 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f9b7da1f60 0x55f9b81d0e10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41c6b32a311f9235 server_cookie=7227a685c9ae7213 in_seq=0 out_seq=0 2024-07-31T14:50:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8cffff640 10 client.15324 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:50:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8cffff640 1 -- 172.21.15.67:0/1329789694 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f9b80bb330 con 0x55f9b7da1f60 2024-07-31T14:50:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8cffff640 10 client.15324 handle_command_reply: tid=0 2024-07-31T14:50:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8fd20b740 1 client.15324 shutdown 2024-07-31T14:50:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8fd20b740 1 -- 172.21.15.67:0/1329789694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d790a0 msgr2=0x55f9b8166b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/1329789694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d790a0 0x55f9b8166b10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0cb0b0 tx=0x7ff8dc005a40 comp rx=0 tx=0).stop 2024-07-31T14:50:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8fd20b740 1 -- 172.21.15.67:0/1329789694 shutdown_connections 2024-07-31T14:50:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/1329789694 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f9b7da1f60 0x55f9b81d0e10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/1329789694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9b7d81280 0x55f9b8167050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/1329789694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9b7d790a0 0x55f9b8166b10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8cffff640 0 client.15324 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:50:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8fd20b740 1 --2- 172.21.15.67:0/1329789694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9b7d608e0 0x55f9b81665d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8fd20b740 1 -- 172.21.15.67:0/1329789694 >> 172.21.15.67:0/1329789694 conn(0x55f9b7d96350 msgr2=0x55f9b8151450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:19.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8fd20b740 1 -- 172.21.15.67:0/1329789694 shutdown_connections 2024-07-31T14:50:19.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8fd20b740 1 -- 172.21.15.67:0/1329789694 wait complete. 2024-07-31T14:50:19.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.026+0000 7ff8fd20b740 20 client.15324 trim_cache size 0 max 16384 2024-07-31T14:50:19.034 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:50:19.034 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:50:19.034 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-07-31T14:50:19.034 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24671, 2024-07-31T14:50:19.034 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:50:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:50:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:50:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 193.986173721, 2024-07-31T14:50:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-31T14:50:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:50:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:50:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 208.19771144699999 2024-07-31T14:50:19.035 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:50:19.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.027+0000 7ff8f6cba640 1 -- 172.21.15.67:0/2912015425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e00b3ca0 msgr2=0x7ff8e00b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:19.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.027+0000 7ff8f6cba640 1 --2- 172.21.15.67:0/2912015425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e00b3ca0 0x7ff8e00b61b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8005fd0 tx=0x7ff8e8009500 comp rx=0 tx=0).stop 2024-07-31T14:50:19.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.027+0000 7ff8f6cba640 1 -- 172.21.15.67:0/2912015425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f0102950 msgr2=0x7ff8f019c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:19.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.027+0000 7ff8f6cba640 1 --2- 172.21.15.67:0/2912015425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f0102950 0x7ff8f019c620 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc005940 tx=0x7ff8dc0040c0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.027+0000 7ff8f6cba640 1 -- 172.21.15.67:0/2912015425 shutdown_connections 2024-07-31T14:50:19.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.027+0000 7ff8f6cba640 1 --2- 172.21.15.67:0/2912015425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e00b3ca0 0x7ff8e00b61b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.027+0000 7ff8f6cba640 1 --2- 172.21.15.67:0/2912015425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f010fbb0 0x7ff8f01a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.027+0000 7ff8f6cba640 1 --2- 172.21.15.67:0/2912015425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f0102950 0x7ff8f019c620 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.027+0000 7ff8f6cba640 1 --2- 172.21.15.67:0/2912015425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8f0102000 0x7ff8f019c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:19.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.027+0000 7ff8f6cba640 1 -- 172.21.15.67:0/2912015425 >> 172.21.15.67:0/2912015425 conn(0x7ff8f0079160 msgr2=0x7ff8f0108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:19.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.027+0000 7ff8f6cba640 1 -- 172.21.15.67:0/2912015425 shutdown_connections 2024-07-31T14:50:19.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:19.027+0000 7ff8f6cba640 1 -- 172.21.15.67:0/2912015425 wait complete. 2024-07-31T14:50:19.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:18 smithi077 ceph-mon[37327]: pgmap v376: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:19.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:18 smithi073 ceph-mon[36714]: pgmap v376: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:19.791 INFO:tasks.cephadm:Setup complete, yielding 2024-07-31T14:50:19.792 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-31T14:50:19.803 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-07-31T14:50:19.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- bash -c 'ceph orch status' 2024-07-31T14:50:20.071 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:21.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:20 smithi077 ceph-mon[37327]: pgmap v377: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:21.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:20 smithi067 ceph-mon[32750]: pgmap v377: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:21.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:20 smithi073 ceph-mon[36714]: pgmap v377: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:21.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.739+0000 7f814db3b640 1 -- 172.21.15.67:0/3531520301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8148109820 msgr2=0x7f8148110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:21.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.739+0000 7f814db3b640 1 --2- 172.21.15.67:0/3531520301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8148109820 0x7f8148110050 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8140009ca0 tx=0x7f814002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:50:21.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.739+0000 7f814db3b640 1 -- 172.21.15.67:0/3531520301 shutdown_connections 2024-07-31T14:50:21.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.739+0000 7f814db3b640 1 --2- 172.21.15.67:0/3531520301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8148109820 0x7f8148110050 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:21.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.739+0000 7f814db3b640 1 --2- 172.21.15.67:0/3531520301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8148108ed0 0x7f81481092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:21.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.739+0000 7f814db3b640 1 --2- 172.21.15.67:0/3531520301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8148103590 0x7f81481039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:21.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.739+0000 7f814db3b640 1 -- 172.21.15.67:0/3531520301 >> 172.21.15.67:0/3531520301 conn(0x7f814807b280 msgr2=0x7f814807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:21.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.739+0000 7f814db3b640 1 -- 172.21.15.67:0/3531520301 shutdown_connections 2024-07-31T14:50:21.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.739+0000 7f814db3b640 1 -- 172.21.15.67:0/3531520301 wait complete. 2024-07-31T14:50:21.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.739+0000 7f814db3b640 1 Processor -- start 2024-07-31T14:50:21.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.740+0000 7f814db3b640 1 -- start start 2024-07-31T14:50:21.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.740+0000 7f814db3b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8148103590 0x7f8148075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:21.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.740+0000 7f814db3b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8148108ed0 0x7f8148075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:21.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.740+0000 7f814db3b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8148109820 0x7f8148075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:21.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.740+0000 7f814db3b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f81481134f0 con 0x7f8148108ed0 2024-07-31T14:50:21.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.740+0000 7f814db3b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8148113370 con 0x7f8148103590 2024-07-31T14:50:21.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.740+0000 7f814db3b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8148113670 con 0x7f8148109820 2024-07-31T14:50:21.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.740+0000 7f8146ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8148108ed0 0x7f8148075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:21.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.740+0000 7f8146ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8148108ed0 0x7f8148075680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52122/0 (socket says 172.21.15.67:52122) 2024-07-31T14:50:21.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.740+0000 7f8146ffd640 1 -- 172.21.15.67:0/2785138581 learned_addr learned my addr 172.21.15.67:0/2785138581 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:21.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.740+0000 7f81477fe640 1 --2- 172.21.15.67:0/2785138581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8148103590 0x7f8148075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:21.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.740+0000 7f8147fff640 1 --2- 172.21.15.67:0/2785138581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8148109820 0x7f8148075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:21.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.740+0000 7f8146ffd640 1 -- 172.21.15.67:0/2785138581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8148109820 msgr2=0x7f8148075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:21.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.740+0000 7f8146ffd640 1 --2- 172.21.15.67:0/2785138581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8148109820 0x7f8148075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:21.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.741+0000 7f8146ffd640 1 -- 172.21.15.67:0/2785138581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8148103590 msgr2=0x7f8148075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:21.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.741+0000 7f8146ffd640 1 --2- 172.21.15.67:0/2785138581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8148103590 0x7f8148075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:21.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.741+0000 7f8146ffd640 1 -- 172.21.15.67:0/2785138581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81481a80d0 con 0x7f8148108ed0 2024-07-31T14:50:21.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.741+0000 7f8146ffd640 1 --2- 172.21.15.67:0/2785138581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8148108ed0 0x7f8148075680 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f813c00d750 tx=0x7f813c00db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:21.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.741+0000 7f8144ff9640 1 -- 172.21.15.67:0/2785138581 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c017070 con 0x7f8148108ed0 2024-07-31T14:50:21.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.741+0000 7f8144ff9640 1 -- 172.21.15.67:0/2785138581 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f813c009820 con 0x7f8148108ed0 2024-07-31T14:50:21.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.741+0000 7f8144ff9640 1 -- 172.21.15.67:0/2785138581 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c004c40 con 0x7f8148108ed0 2024-07-31T14:50:21.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.741+0000 7f814db3b640 1 -- 172.21.15.67:0/2785138581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81481a83c0 con 0x7f8148108ed0 2024-07-31T14:50:21.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.741+0000 7f814db3b640 1 -- 172.21.15.67:0/2785138581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81481a88b0 con 0x7f8148108ed0 2024-07-31T14:50:21.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.742+0000 7f814db3b640 1 -- 172.21.15.67:0/2785138581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81481039a0 con 0x7f8148108ed0 2024-07-31T14:50:21.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.744+0000 7f8144ff9640 1 -- 172.21.15.67:0/2785138581 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f813c0099d0 con 0x7f8148108ed0 2024-07-31T14:50:21.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.745+0000 7f8144ff9640 1 --2- 172.21.15.67:0/2785138581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81300b3ca0 0x7f81300b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:21.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.745+0000 7f81477fe640 1 --2- 172.21.15.67:0/2785138581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81300b3ca0 0x7f81300b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:21.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.745+0000 7f8144ff9640 1 -- 172.21.15.67:0/2785138581 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f813c0e3d20 con 0x7f8148108ed0 2024-07-31T14:50:21.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.746+0000 7f81477fe640 1 --2- 172.21.15.67:0/2785138581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81300b3ca0 0x7f81300b61b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8148076b90 tx=0x7f813800a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:21.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.748+0000 7f8144ff9640 1 -- 172.21.15.67:0/2785138581 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f813c0aafc0 con 0x7f8148108ed0 2024-07-31T14:50:21.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.964+0000 7f814db3b640 1 -- 172.21.15.67:0/2785138581 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f8148063230 con 0x7f81300b3ca0 2024-07-31T14:50:21.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.965+0000 7f8144ff9640 1 -- 172.21.15.67:0/2785138581 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f8148063230 con 0x7f81300b3ca0 2024-07-31T14:50:21.968 INFO:teuthology.orchestra.run.smithi067.stdout:Backend: cephadm 2024-07-31T14:50:21.968 INFO:teuthology.orchestra.run.smithi067.stdout:Available: Yes 2024-07-31T14:50:21.968 INFO:teuthology.orchestra.run.smithi067.stdout:Paused: No 2024-07-31T14:50:21.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.967+0000 7f814db3b640 1 -- 172.21.15.67:0/2785138581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81300b3ca0 msgr2=0x7f81300b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:21.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.967+0000 7f814db3b640 1 --2- 172.21.15.67:0/2785138581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81300b3ca0 0x7f81300b61b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8148076b90 tx=0x7f813800a2d0 comp rx=0 tx=0).stop 2024-07-31T14:50:21.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.967+0000 7f814db3b640 1 -- 172.21.15.67:0/2785138581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8148108ed0 msgr2=0x7f8148075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:21.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.967+0000 7f814db3b640 1 --2- 172.21.15.67:0/2785138581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8148108ed0 0x7f8148075680 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f813c00d750 tx=0x7f813c00db50 comp rx=0 tx=0).stop 2024-07-31T14:50:21.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.967+0000 7f814db3b640 1 -- 172.21.15.67:0/2785138581 shutdown_connections 2024-07-31T14:50:21.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.967+0000 7f814db3b640 1 --2- 172.21.15.67:0/2785138581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81300b3ca0 0x7f81300b61b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:21.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.967+0000 7f814db3b640 1 --2- 172.21.15.67:0/2785138581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8148109820 0x7f8148075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.967+0000 7f814db3b640 1 --2- 172.21.15.67:0/2785138581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8148108ed0 0x7f8148075680 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.967+0000 7f814db3b640 1 --2- 172.21.15.67:0/2785138581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8148103590 0x7f8148075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.967+0000 7f814db3b640 1 -- 172.21.15.67:0/2785138581 >> 172.21.15.67:0/2785138581 conn(0x7f814807b280 msgr2=0x7f8148106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.968+0000 7f814db3b640 1 -- 172.21.15.67:0/2785138581 shutdown_connections 2024-07-31T14:50:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:21.968+0000 7f814db3b640 1 -- 172.21.15.67:0/2785138581 wait complete. 2024-07-31T14:50:22.662 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- bash -c 'ceph orch ps' 2024-07-31T14:50:22.929 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:23.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:22 smithi077 ceph-mon[37327]: pgmap v378: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:23.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:22 smithi077 ceph-mon[37327]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:23.252 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:22 smithi067 ceph-mon[32750]: pgmap v378: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:23.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:22 smithi067 ceph-mon[32750]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:23.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:22 smithi073 ceph-mon[36714]: pgmap v378: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:23.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:22 smithi073 ceph-mon[36714]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:24.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.524+0000 7f635e43a640 1 -- 172.21.15.67:0/1565111907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6358107e10 msgr2=0x7f6358108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:24.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.524+0000 7f635e43a640 1 --2- 172.21.15.67:0/1565111907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6358107e10 0x7f6358108220 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6340009900 tx=0x7f634002a410 comp rx=0 tx=0).stop 2024-07-31T14:50:24.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.524+0000 7f635e43a640 1 -- 172.21.15.67:0/1565111907 shutdown_connections 2024-07-31T14:50:24.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.524+0000 7f635e43a640 1 --2- 172.21.15.67:0/1565111907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6358108760 0x7f635810ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:24.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.524+0000 7f635e43a640 1 --2- 172.21.15.67:0/1565111907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6358107e10 0x7f6358108220 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:24.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.524+0000 7f635e43a640 1 --2- 172.21.15.67:0/1565111907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63581024d0 0x7f63581028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:24.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.524+0000 7f635e43a640 1 -- 172.21.15.67:0/1565111907 >> 172.21.15.67:0/1565111907 conn(0x7f6358079190 msgr2=0x7f63580795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:24.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.524+0000 7f635e43a640 1 -- 172.21.15.67:0/1565111907 shutdown_connections 2024-07-31T14:50:24.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.524+0000 7f635e43a640 1 -- 172.21.15.67:0/1565111907 wait complete. 2024-07-31T14:50:24.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.525+0000 7f635e43a640 1 Processor -- start 2024-07-31T14:50:24.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.525+0000 7f635e43a640 1 -- start start 2024-07-31T14:50:24.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.525+0000 7f635e43a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63581024d0 0x7f635819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:24.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.525+0000 7f635e43a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6358107e10 0x7f635819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:24.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.525+0000 7f635e43a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6358108760 0x7f635819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:24.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.525+0000 7f635e43a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f63581123b0 con 0x7f6358108760 2024-07-31T14:50:24.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.525+0000 7f635e43a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6358112230 con 0x7f63581024d0 2024-07-31T14:50:24.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.525+0000 7f635e43a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6358112530 con 0x7f6358107e10 2024-07-31T14:50:24.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.525+0000 7f635c9d7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6358108760 0x7f635819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:24.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.525+0000 7f635c9d7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6358108760 0x7f635819b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52150/0 (socket says 172.21.15.67:52150) 2024-07-31T14:50:24.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.525+0000 7f635c9d7640 1 -- 172.21.15.67:0/1300536585 learned_addr learned my addr 172.21.15.67:0/1300536585 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:24.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.526+0000 7f634ffff640 1 --2- 172.21.15.67:0/1300536585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6358107e10 0x7f635819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:24.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.526+0000 7f6357fff640 1 --2- 172.21.15.67:0/1300536585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63581024d0 0x7f635819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:24.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.526+0000 7f635c9d7640 1 -- 172.21.15.67:0/1300536585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6358107e10 msgr2=0x7f635819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:24.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.526+0000 7f635c9d7640 1 --2- 172.21.15.67:0/1300536585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6358107e10 0x7f635819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:24.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.526+0000 7f635c9d7640 1 -- 172.21.15.67:0/1300536585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63581024d0 msgr2=0x7f635819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:24.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.526+0000 7f635c9d7640 1 --2- 172.21.15.67:0/1300536585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63581024d0 0x7f635819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:24.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.526+0000 7f635c9d7640 1 -- 172.21.15.67:0/1300536585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63581a6030 con 0x7f6358108760 2024-07-31T14:50:24.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.526+0000 7f635c9d7640 1 --2- 172.21.15.67:0/1300536585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6358108760 0x7f635819b820 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f635000eaa0 tx=0x7f635000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:24.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.526+0000 7f6355ffb640 1 -- 172.21.15.67:0/1300536585 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635000cdb0 con 0x7f6358108760 2024-07-31T14:50:24.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.526+0000 7f6355ffb640 1 -- 172.21.15.67:0/1300536585 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6350005020 con 0x7f6358108760 2024-07-31T14:50:24.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.526+0000 7f635e43a640 1 -- 172.21.15.67:0/1300536585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63581a6320 con 0x7f6358108760 2024-07-31T14:50:24.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.526+0000 7f6355ffb640 1 -- 172.21.15.67:0/1300536585 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500116d0 con 0x7f6358108760 2024-07-31T14:50:24.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.526+0000 7f635e43a640 1 -- 172.21.15.67:0/1300536585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63581a6830 con 0x7f6358108760 2024-07-31T14:50:24.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.528+0000 7f635e43a640 1 -- 172.21.15.67:0/1300536585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6358103750 con 0x7f6358108760 2024-07-31T14:50:24.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.529+0000 7f6355ffb640 1 -- 172.21.15.67:0/1300536585 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f63500051d0 con 0x7f6358108760 2024-07-31T14:50:24.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.530+0000 7f6355ffb640 1 --2- 172.21.15.67:0/1300536585 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f63440b3cd0 0x7f63440b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:24.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.530+0000 7f6357fff640 1 --2- 172.21.15.67:0/1300536585 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f63440b3cd0 0x7f63440b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:24.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.531+0000 7f6355ffb640 1 -- 172.21.15.67:0/1300536585 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f63500e45f0 con 0x7f6358108760 2024-07-31T14:50:24.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.531+0000 7f6357fff640 1 --2- 172.21.15.67:0/1300536585 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f63440b3cd0 0x7f63440b61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6348005fd0 tx=0x7f6348005e20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:24.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.534+0000 7f6355ffb640 1 -- 172.21.15.67:0/1300536585 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f6350016030 con 0x7f6358108760 2024-07-31T14:50:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.739+0000 7f635e43a640 1 -- 172.21.15.67:0/1300536585 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6358063230 con 0x7f63440b3cd0 2024-07-31T14:50:24.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.753+0000 7f6355ffb640 1 -- 172.21.15.67:0/1300536585 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f6358063230 con 0x7f63440b3cd0 2024-07-31T14:50:24.754 INFO:teuthology.orchestra.run.smithi067.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-31T14:50:24.754 INFO:teuthology.orchestra.run.smithi067.stdout:mds.a smithi067 running (3m) 40s ago 3m 85.0M - 19.0.0-5223-g8c796fca 2d83fc2832e3 143db92221e1 2024-07-31T14:50:24.754 INFO:teuthology.orchestra.run.smithi067.stdout:mds.b smithi073 running (3m) 80s ago 3m 94.4M - 19.0.0-5223-g8c796fca 2d83fc2832e3 121ffbbb3f06 2024-07-31T14:50:24.754 INFO:teuthology.orchestra.run.smithi067.stdout:mds.c smithi077 running (3m) 80s ago 3m 86.1M - 19.0.0-5223-g8c796fca 2d83fc2832e3 71d4c7cc57d3 2024-07-31T14:50:24.754 INFO:teuthology.orchestra.run.smithi067.stdout:mds.d smithi067 running (3m) 40s ago 3m 82.2M - 19.0.0-5223-g8c796fca 2d83fc2832e3 7062ce67ed14 2024-07-31T14:50:24.755 INFO:teuthology.orchestra.run.smithi067.stdout:mds.e smithi073 running (3m) 80s ago 3m 86.5M - 19.0.0-5223-g8c796fca 2d83fc2832e3 c20528a1cd96 2024-07-31T14:50:24.755 INFO:teuthology.orchestra.run.smithi067.stdout:mds.f smithi077 running (3m) 80s ago 3m 88.2M - 19.0.0-5223-g8c796fca 2d83fc2832e3 9519cd747ec1 2024-07-31T14:50:24.755 INFO:teuthology.orchestra.run.smithi067.stdout:mds.g smithi067 running (3m) 40s ago 3m 85.7M - 19.0.0-5223-g8c796fca 2d83fc2832e3 8117ab345ba3 2024-07-31T14:50:24.755 INFO:teuthology.orchestra.run.smithi067.stdout:mds.h smithi073 running (3m) 80s ago 3m 86.3M - 19.0.0-5223-g8c796fca 2d83fc2832e3 461ee206e533 2024-07-31T14:50:24.755 INFO:teuthology.orchestra.run.smithi067.stdout:mds.i smithi077 running (3m) 80s ago 3m 85.9M - 19.0.0-5223-g8c796fca 2d83fc2832e3 c5369626e9f0 2024-07-31T14:50:24.755 INFO:teuthology.orchestra.run.smithi067.stdout:mds.j smithi067 running (3m) 40s ago 3m 82.4M - 19.0.0-5223-g8c796fca 2d83fc2832e3 956bba6a593a 2024-07-31T14:50:24.755 INFO:teuthology.orchestra.run.smithi067.stdout:mds.k smithi073 running (4m) 80s ago 4m 87.7M - 19.0.0-5223-g8c796fca 2d83fc2832e3 60da7f5aff93 2024-07-31T14:50:24.755 INFO:teuthology.orchestra.run.smithi067.stdout:mds.l smithi077 running (4m) 80s ago 4m 87.3M - 19.0.0-5223-g8c796fca 2d83fc2832e3 40b7e4bfb4ac 2024-07-31T14:50:24.755 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.x smithi067 *:9283,8765 running (11m) 40s ago 11m 585M - 19.0.0-5223-g8c796fca 2d83fc2832e3 f86146d771dd 2024-07-31T14:50:24.755 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.y smithi073 *:8443,8765 running (9m) 80s ago 9m 522M - 19.0.0-5223-g8c796fca 2d83fc2832e3 023c4c6614c0 2024-07-31T14:50:24.755 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.z smithi077 *:8443,8765 running (9m) 80s ago 9m 523M - 19.0.0-5223-g8c796fca 2d83fc2832e3 550da0808099 2024-07-31T14:50:24.756 INFO:teuthology.orchestra.run.smithi067.stdout:mon.a smithi067 running (12m) 40s ago 12m 119M 2048M 19.0.0-5223-g8c796fca 2d83fc2832e3 65e2949f2afd 2024-07-31T14:50:24.756 INFO:teuthology.orchestra.run.smithi067.stdout:mon.b smithi073 running (10m) 80s ago 10m 111M 2048M 19.0.0-5223-g8c796fca 2d83fc2832e3 0cd68fed9b4c 2024-07-31T14:50:24.756 INFO:teuthology.orchestra.run.smithi067.stdout:mon.c smithi077 running (10m) 80s ago 10m 112M 2048M 19.0.0-5223-g8c796fca 2d83fc2832e3 31b92f1da4cc 2024-07-31T14:50:24.756 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 smithi067 running (8m) 40s ago 8m 124M 4272M 19.0.0-5223-g8c796fca 2d83fc2832e3 7fb2494fd29d 2024-07-31T14:50:24.756 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 smithi073 running (8m) 80s ago 8m 124M 4272M 19.0.0-5223-g8c796fca 2d83fc2832e3 9ce8c800be4e 2024-07-31T14:50:24.756 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 smithi077 running (8m) 80s ago 8m 115M 4272M 19.0.0-5223-g8c796fca 2d83fc2832e3 9de1d1ea5b34 2024-07-31T14:50:24.756 INFO:teuthology.orchestra.run.smithi067.stdout:osd.3 smithi067 running (7m) 40s ago 7m 112M 4272M 19.0.0-5223-g8c796fca 2d83fc2832e3 ed7383f8c791 2024-07-31T14:50:24.756 INFO:teuthology.orchestra.run.smithi067.stdout:osd.4 smithi073 running (7m) 80s ago 7m 110M 4272M 19.0.0-5223-g8c796fca 2d83fc2832e3 851b2ea221d2 2024-07-31T14:50:24.756 INFO:teuthology.orchestra.run.smithi067.stdout:osd.5 smithi077 running (6m) 80s ago 6m 113M 4272M 19.0.0-5223-g8c796fca 2d83fc2832e3 fa2fdfb8a062 2024-07-31T14:50:24.756 INFO:teuthology.orchestra.run.smithi067.stdout:osd.6 smithi067 running (6m) 40s ago 6m 119M 4272M 19.0.0-5223-g8c796fca 2d83fc2832e3 8d81460cf622 2024-07-31T14:50:24.756 INFO:teuthology.orchestra.run.smithi067.stdout:osd.7 smithi073 running (6m) 80s ago 6m 122M 4272M 19.0.0-5223-g8c796fca 2d83fc2832e3 0053f8eb312a 2024-07-31T14:50:24.756 INFO:teuthology.orchestra.run.smithi067.stdout:osd.8 smithi077 running (5m) 80s ago 5m 112M 4272M 19.0.0-5223-g8c796fca 2d83fc2832e3 f2833fc5abbe 2024-07-31T14:50:24.757 INFO:teuthology.orchestra.run.smithi067.stdout:osd.9 smithi067 running (5m) 40s ago 5m 119M 4272M 19.0.0-5223-g8c796fca 2d83fc2832e3 17ef155e1888 2024-07-31T14:50:24.757 INFO:teuthology.orchestra.run.smithi067.stdout:osd.10 smithi073 running (4m) 80s ago 4m 120M 4272M 19.0.0-5223-g8c796fca 2d83fc2832e3 2acd3da468f7 2024-07-31T14:50:24.757 INFO:teuthology.orchestra.run.smithi067.stdout:osd.11 smithi077 running (4m) 80s ago 4m 110M 4272M 19.0.0-5223-g8c796fca 2d83fc2832e3 227d5a558e42 2024-07-31T14:50:24.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.755+0000 7f635e43a640 1 -- 172.21.15.67:0/1300536585 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f63440b3cd0 msgr2=0x7f63440b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:24.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.755+0000 7f635e43a640 1 --2- 172.21.15.67:0/1300536585 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f63440b3cd0 0x7f63440b61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6348005fd0 tx=0x7f6348005e20 comp rx=0 tx=0).stop 2024-07-31T14:50:24.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.755+0000 7f635e43a640 1 -- 172.21.15.67:0/1300536585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6358108760 msgr2=0x7f635819b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.755+0000 7f635e43a640 1 --2- 172.21.15.67:0/1300536585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6358108760 0x7f635819b820 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f635000eaa0 tx=0x7f635000ef60 comp rx=0 tx=0).stop 2024-07-31T14:50:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.756+0000 7f635e43a640 1 -- 172.21.15.67:0/1300536585 shutdown_connections 2024-07-31T14:50:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.756+0000 7f635e43a640 1 --2- 172.21.15.67:0/1300536585 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f63440b3cd0 0x7f63440b61e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.756+0000 7f635e43a640 1 --2- 172.21.15.67:0/1300536585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6358108760 0x7f635819b820 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.756+0000 7f635e43a640 1 --2- 172.21.15.67:0/1300536585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6358107e10 0x7f635819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.756+0000 7f635e43a640 1 --2- 172.21.15.67:0/1300536585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63581024d0 0x7f635819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.756+0000 7f635e43a640 1 -- 172.21.15.67:0/1300536585 >> 172.21.15.67:0/1300536585 conn(0x7f6358079190 msgr2=0x7f6358103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.756+0000 7f635e43a640 1 -- 172.21.15.67:0/1300536585 shutdown_connections 2024-07-31T14:50:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:24.756+0000 7f635e43a640 1 -- 172.21.15.67:0/1300536585 wait complete. 2024-07-31T14:50:25.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:24 smithi067 ceph-mon[32750]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:25.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:24 smithi073 ceph-mon[36714]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:25.456 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- bash -c 'ceph orch ls' 2024-07-31T14:50:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:24 smithi077 ceph-mon[37327]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:25.723 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:26.104 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:25 smithi067 ceph-mon[32750]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:26.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:25 smithi073 ceph-mon[36714]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:26.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:25 smithi077 ceph-mon[37327]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:27.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:26 smithi067 ceph-mon[32750]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.375+0000 7fd010874640 1 -- 172.21.15.67:0/2134762003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd00c108ef0 msgr2=0x7fd00c109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.375+0000 7fd010874640 1 --2- 172.21.15.67:0/2134762003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd00c108ef0 0x7fd00c109300 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009900 tx=0x7fcff402a410 comp rx=0 tx=0).stop 2024-07-31T14:50:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.375+0000 7fd010874640 1 -- 172.21.15.67:0/2134762003 shutdown_connections 2024-07-31T14:50:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.375+0000 7fd010874640 1 --2- 172.21.15.67:0/2134762003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd00c109840 0x7fd00c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.375+0000 7fd010874640 1 --2- 172.21.15.67:0/2134762003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd00c108ef0 0x7fd00c109300 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.375+0000 7fd010874640 1 --2- 172.21.15.67:0/2134762003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c1035b0 0x7fd00c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.375+0000 7fd010874640 1 -- 172.21.15.67:0/2134762003 >> 172.21.15.67:0/2134762003 conn(0x7fd00c0ff040 msgr2=0x7fd00c101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.375+0000 7fd010874640 1 -- 172.21.15.67:0/2134762003 shutdown_connections 2024-07-31T14:50:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.375+0000 7fd010874640 1 -- 172.21.15.67:0/2134762003 wait complete. 2024-07-31T14:50:27.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.376+0000 7fd010874640 1 Processor -- start 2024-07-31T14:50:27.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.376+0000 7fd010874640 1 -- start start 2024-07-31T14:50:27.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.376+0000 7fd010874640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd00c1035b0 0x7fd00c199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:27.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.376+0000 7fd010874640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c108ef0 0x7fd00c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:27.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.376+0000 7fd010874640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd00c109840 0x7fd00c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:27.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.377+0000 7fd010874640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c113520 con 0x7fd00c109840 2024-07-31T14:50:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.377+0000 7fd010874640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c1133a0 con 0x7fd00c1035b0 2024-07-31T14:50:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.377+0000 7fd010874640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c1136a0 con 0x7fd00c108ef0 2024-07-31T14:50:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.377+0000 7fd00ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd00c109840 0x7fd00c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.377+0000 7fd00ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd00c109840 0x7fd00c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52180/0 (socket says 172.21.15.67:52180) 2024-07-31T14:50:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.377+0000 7fd00ad9d640 1 -- 172.21.15.67:0/1166312393 learned_addr learned my addr 172.21.15.67:0/1166312393 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.377+0000 7fd00a59c640 1 --2- 172.21.15.67:0/1166312393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd00c1035b0 0x7fd00c199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.377+0000 7fd009d9b640 1 --2- 172.21.15.67:0/1166312393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c108ef0 0x7fd00c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.377+0000 7fd00ad9d640 1 -- 172.21.15.67:0/1166312393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c108ef0 msgr2=0x7fd00c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.377+0000 7fd00ad9d640 1 --2- 172.21.15.67:0/1166312393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c108ef0 0x7fd00c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.377+0000 7fd00ad9d640 1 -- 172.21.15.67:0/1166312393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd00c1035b0 msgr2=0x7fd00c199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.377+0000 7fd00ad9d640 1 --2- 172.21.15.67:0/1166312393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd00c1035b0 0x7fd00c199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.377+0000 7fd00ad9d640 1 -- 172.21.15.67:0/1166312393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c19fdc0 con 0x7fd00c109840 2024-07-31T14:50:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.377+0000 7fd00ad9d640 1 --2- 172.21.15.67:0/1166312393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd00c109840 0x7fd00c19f750 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd004009fd0 tx=0x7fd00400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.378+0000 7fd0037fe640 1 -- 172.21.15.67:0/1166312393 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004011830 con 0x7fd00c109840 2024-07-31T14:50:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.378+0000 7fd010874640 1 -- 172.21.15.67:0/1166312393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00c1a00b0 con 0x7fd00c109840 2024-07-31T14:50:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.378+0000 7fd0037fe640 1 -- 172.21.15.67:0/1166312393 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd004016070 con 0x7fd00c109840 2024-07-31T14:50:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.378+0000 7fd0037fe640 1 -- 172.21.15.67:0/1166312393 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004010de0 con 0x7fd00c109840 2024-07-31T14:50:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.378+0000 7fd010874640 1 -- 172.21.15.67:0/1166312393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd00c1a68d0 con 0x7fd00c109840 2024-07-31T14:50:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.379+0000 7fd010874640 1 -- 172.21.15.67:0/1166312393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00c1039c0 con 0x7fd00c109840 2024-07-31T14:50:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.380+0000 7fd0037fe640 1 -- 172.21.15.67:0/1166312393 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fd0040119d0 con 0x7fd00c109840 2024-07-31T14:50:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.382+0000 7fd0037fe640 1 --2- 172.21.15.67:0/1166312393 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcff80b3c50 0x7fcff80b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.382+0000 7fd00a59c640 1 --2- 172.21.15.67:0/1166312393 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcff80b3c50 0x7fcff80b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.382+0000 7fd0037fe640 1 -- 172.21.15.67:0/1166312393 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fd0040e3d60 con 0x7fd00c109840 2024-07-31T14:50:27.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.383+0000 7fd00a59c640 1 --2- 172.21.15.67:0/1166312393 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcff80b3c50 0x7fcff80b6160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcffc007ec0 tx=0x7fcffc0075b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:27.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.385+0000 7fd0037fe640 1 -- 172.21.15.67:0/1166312393 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd0040aaf60 con 0x7fd00c109840 2024-07-31T14:50:27.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:26 smithi073 ceph-mon[36714]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:26 smithi077 ceph-mon[37327]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:27.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.589+0000 7fd010874640 1 -- 172.21.15.67:0/1166312393 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fd00c063230 con 0x7fcff80b3c50 2024-07-31T14:50:27.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.595+0000 7fd0037fe640 1 -- 172.21.15.67:0/1166312393 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fd00c063230 con 0x7fcff80b3c50 2024-07-31T14:50:27.596 INFO:teuthology.orchestra.run.smithi067.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-07-31T14:50:27.596 INFO:teuthology.orchestra.run.smithi067.stdout:mds.all 12/12 83s ago 4m smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l;count:12 2024-07-31T14:50:27.596 INFO:teuthology.orchestra.run.smithi067.stdout:mgr 3/3 83s ago 10m smithi067=x;smithi073=y;smithi077=z;count:3 2024-07-31T14:50:27.596 INFO:teuthology.orchestra.run.smithi067.stdout:mon 3/3 83s ago 10m smithi067:172.21.15.67=a;smithi073:172.21.15.73=b;smithi077:172.21.15.77=c;count:3 2024-07-31T14:50:27.596 INFO:teuthology.orchestra.run.smithi067.stdout:osd 12 83s ago - 2024-07-31T14:50:27.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.597+0000 7fd010874640 1 -- 172.21.15.67:0/1166312393 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcff80b3c50 msgr2=0x7fcff80b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:27.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.597+0000 7fd010874640 1 --2- 172.21.15.67:0/1166312393 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcff80b3c50 0x7fcff80b6160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcffc007ec0 tx=0x7fcffc0075b0 comp rx=0 tx=0).stop 2024-07-31T14:50:27.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.597+0000 7fd010874640 1 -- 172.21.15.67:0/1166312393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd00c109840 msgr2=0x7fd00c19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:27.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.597+0000 7fd010874640 1 --2- 172.21.15.67:0/1166312393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd00c109840 0x7fd00c19f750 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd004009fd0 tx=0x7fd00400b8e0 comp rx=0 tx=0).stop 2024-07-31T14:50:27.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.597+0000 7fd010874640 1 -- 172.21.15.67:0/1166312393 shutdown_connections 2024-07-31T14:50:27.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.597+0000 7fd010874640 1 --2- 172.21.15.67:0/1166312393 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcff80b3c50 0x7fcff80b6160 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:27.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.597+0000 7fd010874640 1 --2- 172.21.15.67:0/1166312393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd00c109840 0x7fd00c19f750 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:27.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.597+0000 7fd010874640 1 --2- 172.21.15.67:0/1166312393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c108ef0 0x7fd00c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:27.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.597+0000 7fd010874640 1 --2- 172.21.15.67:0/1166312393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd00c1035b0 0x7fd00c199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:27.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.597+0000 7fd010874640 1 -- 172.21.15.67:0/1166312393 >> 172.21.15.67:0/1166312393 conn(0x7fd00c0ff040 msgr2=0x7fd00c101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:27.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.597+0000 7fd010874640 1 -- 172.21.15.67:0/1166312393 shutdown_connections 2024-07-31T14:50:27.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:27.597+0000 7fd010874640 1 -- 172.21.15.67:0/1166312393 wait complete. 2024-07-31T14:50:28.251 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- bash -c 'ceph orch host ls' 2024-07-31T14:50:28.521 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:29.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:29 smithi067 ceph-mon[32750]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:29.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:29 smithi067 ceph-mon[32750]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:29.309 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:29 smithi077 ceph-mon[37327]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:29.310 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:29 smithi077 ceph-mon[37327]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:29.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:29 smithi073 ceph-mon[36714]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:29.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:29 smithi073 ceph-mon[36714]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:30.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.214+0000 7ff87c1ed640 1 -- 172.21.15.67:0/2671805858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff874103590 msgr2=0x7ff8741039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:30.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.214+0000 7ff87c1ed640 1 --2- 172.21.15.67:0/2671805858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff874103590 0x7ff8741039a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff870009900 tx=0x7ff87002a410 comp rx=0 tx=0).stop 2024-07-31T14:50:30.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.214+0000 7ff87c1ed640 1 -- 172.21.15.67:0/2671805858 shutdown_connections 2024-07-31T14:50:30.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.214+0000 7ff87c1ed640 1 --2- 172.21.15.67:0/2671805858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff874109820 0x7ff874110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:30.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.214+0000 7ff87c1ed640 1 --2- 172.21.15.67:0/2671805858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff874108ed0 0x7ff8741092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:30.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.214+0000 7ff87c1ed640 1 --2- 172.21.15.67:0/2671805858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff874103590 0x7ff8741039a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:30.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.214+0000 7ff87c1ed640 1 -- 172.21.15.67:0/2671805858 >> 172.21.15.67:0/2671805858 conn(0x7ff87407b280 msgr2=0x7ff87407b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:30.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.215+0000 7ff87c1ed640 1 -- 172.21.15.67:0/2671805858 shutdown_connections 2024-07-31T14:50:30.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.215+0000 7ff87c1ed640 1 -- 172.21.15.67:0/2671805858 wait complete. 2024-07-31T14:50:30.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.215+0000 7ff87c1ed640 1 Processor -- start 2024-07-31T14:50:30.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.215+0000 7ff87c1ed640 1 -- start start 2024-07-31T14:50:30.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff87c1ed640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff874103590 0x7ff874199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:30.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff87c1ed640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff874108ed0 0x7ff87419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:30.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff87c1ed640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff874109820 0x7ff87419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:30.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff87c1ed640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff8741134f0 con 0x7ff874108ed0 2024-07-31T14:50:30.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff87c1ed640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff874113370 con 0x7ff874109820 2024-07-31T14:50:30.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff87c1ed640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff874113670 con 0x7ff874103590 2024-07-31T14:50:30.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff879788640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff874108ed0 0x7ff87419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:30.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff879788640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff874108ed0 0x7ff87419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52192/0 (socket says 172.21.15.67:52192) 2024-07-31T14:50:30.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff879788640 1 -- 172.21.15.67:0/3432092661 learned_addr learned my addr 172.21.15.67:0/3432092661 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff879f89640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff874103590 0x7ff874199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff87a78a640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff874109820 0x7ff87419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff879788640 1 -- 172.21.15.67:0/3432092661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff874103590 msgr2=0x7ff874199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff879788640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff874103590 0x7ff874199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff879788640 1 -- 172.21.15.67:0/3432092661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff874109820 msgr2=0x7ff87419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff879788640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff874109820 0x7ff87419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff879788640 1 -- 172.21.15.67:0/3432092661 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87419fe70 con 0x7ff874108ed0 2024-07-31T14:50:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff87a78a640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff874109820 0x7ff87419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:50:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.216+0000 7ff879788640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff874108ed0 0x7ff87419a4a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff86400e9f0 tx=0x7ff86400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.217+0000 7ff86affd640 1 -- 172.21.15.67:0/3432092661 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86400ccc0 con 0x7ff874108ed0 2024-07-31T14:50:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.217+0000 7ff87c1ed640 1 -- 172.21.15.67:0/3432092661 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8741a0160 con 0x7ff874108ed0 2024-07-31T14:50:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.217+0000 7ff86affd640 1 -- 172.21.15.67:0/3432092661 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8640108e0 con 0x7ff874108ed0 2024-07-31T14:50:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.217+0000 7ff86affd640 1 -- 172.21.15.67:0/3432092661 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864018860 con 0x7ff874108ed0 2024-07-31T14:50:30.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.217+0000 7ff87c1ed640 1 -- 172.21.15.67:0/3432092661 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8741a68c0 con 0x7ff874108ed0 2024-07-31T14:50:30.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.218+0000 7ff87c1ed640 1 -- 172.21.15.67:0/3432092661 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8741039a0 con 0x7ff874108ed0 2024-07-31T14:50:30.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.220+0000 7ff86affd640 1 -- 172.21.15.67:0/3432092661 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7ff864010430 con 0x7ff874108ed0 2024-07-31T14:50:30.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.221+0000 7ff86affd640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8600b3c20 0x7ff8600b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:30.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.221+0000 7ff879f89640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8600b3c20 0x7ff8600b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:30.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.221+0000 7ff86affd640 1 -- 172.21.15.67:0/3432092661 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff8640e48c0 con 0x7ff874108ed0 2024-07-31T14:50:30.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.221+0000 7ff879f89640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8600b3c20 0x7ff8600b6130 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff870031440 tx=0x7ff870009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:30.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.224+0000 7ff86affd640 1 -- 172.21.15.67:0/3432092661 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff8640abae0 con 0x7ff874108ed0 2024-07-31T14:50:30.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.426+0000 7ff87c1ed640 1 -- 172.21.15.67:0/3432092661 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7ff874063230 con 0x7ff8600b3c20 2024-07-31T14:50:30.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.427+0000 7ff86affd640 1 -- 172.21.15.67:0/3432092661 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7ff874063230 con 0x7ff8600b3c20 2024-07-31T14:50:30.428 INFO:teuthology.orchestra.run.smithi067.stdout:HOST ADDR LABELS STATUS 2024-07-31T14:50:30.428 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 172.21.15.67 2024-07-31T14:50:30.429 INFO:teuthology.orchestra.run.smithi067.stdout:smithi073 172.21.15.73 2024-07-31T14:50:30.429 INFO:teuthology.orchestra.run.smithi067.stdout:smithi077 172.21.15.77 2024-07-31T14:50:30.429 INFO:teuthology.orchestra.run.smithi067.stdout:3 hosts in cluster 2024-07-31T14:50:30.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.429+0000 7ff87c1ed640 1 -- 172.21.15.67:0/3432092661 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8600b3c20 msgr2=0x7ff8600b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:30.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.429+0000 7ff87c1ed640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8600b3c20 0x7ff8600b6130 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff870031440 tx=0x7ff870009560 comp rx=0 tx=0).stop 2024-07-31T14:50:30.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.429+0000 7ff87c1ed640 1 -- 172.21.15.67:0/3432092661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff874108ed0 msgr2=0x7ff87419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:30.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.429+0000 7ff87c1ed640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff874108ed0 0x7ff87419a4a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff86400e9f0 tx=0x7ff86400eeb0 comp rx=0 tx=0).stop 2024-07-31T14:50:30.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.429+0000 7ff87c1ed640 1 -- 172.21.15.67:0/3432092661 shutdown_connections 2024-07-31T14:50:30.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.429+0000 7ff87c1ed640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8600b3c20 0x7ff8600b6130 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:30.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.429+0000 7ff87c1ed640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff874109820 0x7ff87419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:30.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.429+0000 7ff87c1ed640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff874108ed0 0x7ff87419a4a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:30.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.429+0000 7ff87c1ed640 1 --2- 172.21.15.67:0/3432092661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff874103590 0x7ff874199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:30.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.429+0000 7ff87c1ed640 1 -- 172.21.15.67:0/3432092661 >> 172.21.15.67:0/3432092661 conn(0x7ff87407b280 msgr2=0x7ff874106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:30.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.429+0000 7ff87c1ed640 1 -- 172.21.15.67:0/3432092661 shutdown_connections 2024-07-31T14:50:30.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:30.429+0000 7ff87c1ed640 1 -- 172.21.15.67:0/3432092661 wait complete. 2024-07-31T14:50:31.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:31 smithi067 ceph-mon[32750]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:31.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- bash -c 'ceph orch device ls' 2024-07-31T14:50:31.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:31 smithi073 ceph-mon[36714]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:31.396 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:31 smithi077 ceph-mon[37327]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:32.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:32 smithi067 ceph-mon[32750]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:32.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:32 smithi073 ceph-mon[36714]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:32.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:32 smithi077 ceph-mon[37327]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.952+0000 7fd659ddf640 1 -- 172.21.15.67:0/1343577021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd654108720 msgr2=0x7fd65410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.952+0000 7fd659ddf640 1 --2- 172.21.15.67:0/1343577021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd654108720 0x7fd65410ef50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009ca0 tx=0x7fd64c02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:50:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.952+0000 7fd659ddf640 1 -- 172.21.15.67:0/1343577021 shutdown_connections 2024-07-31T14:50:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.952+0000 7fd659ddf640 1 --2- 172.21.15.67:0/1343577021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd654108720 0x7fd65410ef50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.952+0000 7fd659ddf640 1 --2- 172.21.15.67:0/1343577021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd654107dd0 0x7fd6541081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.952+0000 7fd659ddf640 1 --2- 172.21.15.67:0/1343577021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd654102490 0x7fd6541028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:32.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.952+0000 7fd659ddf640 1 -- 172.21.15.67:0/1343577021 >> 172.21.15.67:0/1343577021 conn(0x7fd654079160 msgr2=0x7fd654079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:32.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.952+0000 7fd659ddf640 1 -- 172.21.15.67:0/1343577021 shutdown_connections 2024-07-31T14:50:32.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.952+0000 7fd659ddf640 1 -- 172.21.15.67:0/1343577021 wait complete. 2024-07-31T14:50:32.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.953+0000 7fd659ddf640 1 Processor -- start 2024-07-31T14:50:32.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.953+0000 7fd659ddf640 1 -- start start 2024-07-31T14:50:32.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.953+0000 7fd659ddf640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd654102490 0x7fd654199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:32.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.953+0000 7fd659ddf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd654107dd0 0x7fd65419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:32.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.953+0000 7fd659ddf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd654108720 0x7fd65419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:32.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd659ddf640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd6541123a0 con 0x7fd654102490 2024-07-31T14:50:32.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd659ddf640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd654112220 con 0x7fd654107dd0 2024-07-31T14:50:32.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd659ddf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd654112520 con 0x7fd654108720 2024-07-31T14:50:32.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd6537fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd654102490 0x7fd654199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:32.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd6537fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd654102490 0x7fd654199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49370/0 (socket says 172.21.15.67:49370) 2024-07-31T14:50:32.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd6537fe640 1 -- 172.21.15.67:0/1100516856 learned_addr learned my addr 172.21.15.67:0/1100516856 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:32.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd652ffd640 1 --2- 172.21.15.67:0/1100516856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd654107dd0 0x7fd65419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd653fff640 1 --2- 172.21.15.67:0/1100516856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd654108720 0x7fd65419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd6537fe640 1 -- 172.21.15.67:0/1100516856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd654108720 msgr2=0x7fd65419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd6537fe640 1 --2- 172.21.15.67:0/1100516856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd654108720 0x7fd65419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd6537fe640 1 -- 172.21.15.67:0/1100516856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd654107dd0 msgr2=0x7fd65419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd6537fe640 1 --2- 172.21.15.67:0/1100516856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd654107dd0 0x7fd65419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd6537fe640 1 -- 172.21.15.67:0/1100516856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65419fd50 con 0x7fd654102490 2024-07-31T14:50:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd6537fe640 1 --2- 172.21.15.67:0/1100516856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd654102490 0x7fd654199ea0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd64400d770 tx=0x7fd64400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd650ff9640 1 -- 172.21.15.67:0/1100516856 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd644017070 con 0x7fd654102490 2024-07-31T14:50:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.954+0000 7fd650ff9640 1 -- 172.21.15.67:0/1100516856 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd644009820 con 0x7fd654102490 2024-07-31T14:50:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.955+0000 7fd659ddf640 1 -- 172.21.15.67:0/1100516856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6541a0040 con 0x7fd654102490 2024-07-31T14:50:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.955+0000 7fd650ff9640 1 -- 172.21.15.67:0/1100516856 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd644004bc0 con 0x7fd654102490 2024-07-31T14:50:32.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.955+0000 7fd659ddf640 1 -- 172.21.15.67:0/1100516856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6541a6860 con 0x7fd654102490 2024-07-31T14:50:32.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.956+0000 7fd659ddf640 1 -- 172.21.15.67:0/1100516856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6541028a0 con 0x7fd654102490 2024-07-31T14:50:32.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.957+0000 7fd650ff9640 1 -- 172.21.15.67:0/1100516856 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fd6440099d0 con 0x7fd654102490 2024-07-31T14:50:32.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.958+0000 7fd650ff9640 1 --2- 172.21.15.67:0/1100516856 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd63c0b3cd0 0x7fd63c0b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:32.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.958+0000 7fd652ffd640 1 --2- 172.21.15.67:0/1100516856 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd63c0b3cd0 0x7fd63c0b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:32.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.958+0000 7fd650ff9640 1 -- 172.21.15.67:0/1100516856 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fd6440e3ee0 con 0x7fd654102490 2024-07-31T14:50:32.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.959+0000 7fd652ffd640 1 --2- 172.21.15.67:0/1100516856 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd63c0b3cd0 0x7fd63c0b61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd654069d40 tx=0x7fd64800a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:32.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:32.961+0000 7fd650ff9640 1 -- 172.21.15.67:0/1100516856 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd6440ab180 con 0x7fd654102490 2024-07-31T14:50:33.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:33.166+0000 7fd659ddf640 1 -- 172.21.15.67:0/1100516856 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fd654063230 con 0x7fd63c0b3cd0 2024-07-31T14:50:33.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:33.167+0000 7fd650ff9640 1 -- 172.21.15.67:0/1100516856 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fd654063230 con 0x7fd63c0b3cd0 2024-07-31T14:50:33.169 INFO:teuthology.orchestra.run.smithi067.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-07-31T14:50:33.169 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440042400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-07-31T14:50:33.169 INFO:teuthology.orchestra.run.smithi067.stdout:smithi073 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040115400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-07-31T14:50:33.169 INFO:teuthology.orchestra.run.smithi067.stdout:smithi077 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZJ400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-07-31T14:50:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:33.169+0000 7fd659ddf640 1 -- 172.21.15.67:0/1100516856 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd63c0b3cd0 msgr2=0x7fd63c0b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:33.169+0000 7fd659ddf640 1 --2- 172.21.15.67:0/1100516856 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd63c0b3cd0 0x7fd63c0b61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd654069d40 tx=0x7fd64800a2d0 comp rx=0 tx=0).stop 2024-07-31T14:50:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:33.169+0000 7fd659ddf640 1 -- 172.21.15.67:0/1100516856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd654102490 msgr2=0x7fd654199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:33.169+0000 7fd659ddf640 1 --2- 172.21.15.67:0/1100516856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd654102490 0x7fd654199ea0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd64400d770 tx=0x7fd64400db70 comp rx=0 tx=0).stop 2024-07-31T14:50:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:33.169+0000 7fd659ddf640 1 -- 172.21.15.67:0/1100516856 shutdown_connections 2024-07-31T14:50:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:33.169+0000 7fd659ddf640 1 --2- 172.21.15.67:0/1100516856 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd63c0b3cd0 0x7fd63c0b61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:33.169+0000 7fd659ddf640 1 --2- 172.21.15.67:0/1100516856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd654108720 0x7fd65419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:33.169+0000 7fd659ddf640 1 --2- 172.21.15.67:0/1100516856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd654107dd0 0x7fd65419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:33.169+0000 7fd659ddf640 1 --2- 172.21.15.67:0/1100516856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd654102490 0x7fd654199ea0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:33.169+0000 7fd659ddf640 1 -- 172.21.15.67:0/1100516856 >> 172.21.15.67:0/1100516856 conn(0x7fd654079160 msgr2=0x7fd65406ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:33.170+0000 7fd659ddf640 1 -- 172.21.15.67:0/1100516856 shutdown_connections 2024-07-31T14:50:33.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:33.170+0000 7fd659ddf640 1 -- 172.21.15.67:0/1100516856 wait complete. 2024-07-31T14:50:33.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:33 smithi067 ceph-mon[32750]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:33.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:33 smithi073 ceph-mon[36714]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:33 smithi077 ceph-mon[37327]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:33.884 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-31T14:50:33.892 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-07-31T14:50:33.893 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- bash -c 'ceph fs dump' 2024-07-31T14:50:34.160 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:35.272 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:35 smithi067 ceph-mon[32750]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:35.272 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:35 smithi067 ceph-mon[32750]: pgmap v384: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:35.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:35 smithi073 ceph-mon[36714]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:35.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:35 smithi073 ceph-mon[36714]: pgmap v384: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:35 smithi077 ceph-mon[37327]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:50:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:35 smithi077 ceph-mon[37327]: pgmap v384: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:35.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.801+0000 7fc7231e5640 1 -- 172.21.15.67:0/3416344805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71c102520 msgr2=0x7fc71c102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:35.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.801+0000 7fc7231e5640 1 --2- 172.21.15.67:0/3416344805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71c102520 0x7fc71c102930 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009900 tx=0x7fc70c02a410 comp rx=0 tx=0).stop 2024-07-31T14:50:35.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.801+0000 7fc7231e5640 1 -- 172.21.15.67:0/3416344805 shutdown_connections 2024-07-31T14:50:35.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.801+0000 7fc7231e5640 1 --2- 172.21.15.67:0/3416344805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc71c1087b0 0x7fc71c10efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:35.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.801+0000 7fc7231e5640 1 --2- 172.21.15.67:0/3416344805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc71c107e60 0x7fc71c108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:35.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.801+0000 7fc7231e5640 1 --2- 172.21.15.67:0/3416344805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71c102520 0x7fc71c102930 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:35.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.801+0000 7fc7231e5640 1 -- 172.21.15.67:0/3416344805 >> 172.21.15.67:0/3416344805 conn(0x7fc71c0791a0 msgr2=0x7fc71c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:35.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.801+0000 7fc7231e5640 1 -- 172.21.15.67:0/3416344805 shutdown_connections 2024-07-31T14:50:35.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.801+0000 7fc7231e5640 1 -- 172.21.15.67:0/3416344805 wait complete. 2024-07-31T14:50:35.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.802+0000 7fc7231e5640 1 Processor -- start 2024-07-31T14:50:35.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.802+0000 7fc7231e5640 1 -- start start 2024-07-31T14:50:35.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.802+0000 7fc7231e5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc71c102520 0x7fc71c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:35.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.802+0000 7fc7231e5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71c107e60 0x7fc71c19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:35.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.802+0000 7fc7231e5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc71c1087b0 0x7fc71c19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:35.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.802+0000 7fc7231e5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c112480 con 0x7fc71c107e60 2024-07-31T14:50:35.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.802+0000 7fc7231e5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c112300 con 0x7fc71c102520 2024-07-31T14:50:35.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.802+0000 7fc7231e5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c112600 con 0x7fc71c1087b0 2024-07-31T14:50:35.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.802+0000 7fc71bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71c107e60 0x7fc71c19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:35.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.803+0000 7fc71bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71c107e60 0x7fc71c19b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49380/0 (socket says 172.21.15.67:49380) 2024-07-31T14:50:35.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.803+0000 7fc71bfff640 1 -- 172.21.15.67:0/3436279476 learned_addr learned my addr 172.21.15.67:0/3436279476 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:35.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.803+0000 7fc720f81640 1 --2- 172.21.15.67:0/3436279476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc71c102520 0x7fc71c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.803+0000 7fc721782640 1 --2- 172.21.15.67:0/3436279476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc71c1087b0 0x7fc71c19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.803+0000 7fc71bfff640 1 -- 172.21.15.67:0/3436279476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc71c1087b0 msgr2=0x7fc71c19b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.803+0000 7fc71bfff640 1 --2- 172.21.15.67:0/3436279476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc71c1087b0 0x7fc71c19b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.803+0000 7fc71bfff640 1 -- 172.21.15.67:0/3436279476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc71c102520 msgr2=0x7fc71c19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.803+0000 7fc71bfff640 1 --2- 172.21.15.67:0/3436279476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc71c102520 0x7fc71c19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.803+0000 7fc71bfff640 1 -- 172.21.15.67:0/3436279476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c1a5fe0 con 0x7fc71c107e60 2024-07-31T14:50:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.803+0000 7fc71bfff640 1 --2- 172.21.15.67:0/3436279476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71c107e60 0x7fc71c19b290 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc71000c970 tx=0x7fc71000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.803+0000 7fc719ffb640 1 -- 172.21.15.67:0/3436279476 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710007bb0 con 0x7fc71c107e60 2024-07-31T14:50:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.803+0000 7fc7231e5640 1 -- 172.21.15.67:0/3436279476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71c1a62d0 con 0x7fc71c107e60 2024-07-31T14:50:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.803+0000 7fc719ffb640 1 -- 172.21.15.67:0/3436279476 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc710007d50 con 0x7fc71c107e60 2024-07-31T14:50:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.803+0000 7fc719ffb640 1 -- 172.21.15.67:0/3436279476 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71000f450 con 0x7fc71c107e60 2024-07-31T14:50:35.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.804+0000 7fc7231e5640 1 -- 172.21.15.67:0/3436279476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c1a6810 con 0x7fc71c107e60 2024-07-31T14:50:35.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.805+0000 7fc7231e5640 1 -- 172.21.15.67:0/3436279476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71c1037a0 con 0x7fc71c107e60 2024-07-31T14:50:35.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.806+0000 7fc719ffb640 1 -- 172.21.15.67:0/3436279476 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fc710004eb0 con 0x7fc71c107e60 2024-07-31T14:50:35.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.807+0000 7fc719ffb640 1 --2- 172.21.15.67:0/3436279476 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc7040b3c50 0x7fc7040b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:35.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.807+0000 7fc720f81640 1 --2- 172.21.15.67:0/3436279476 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc7040b3c50 0x7fc7040b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:35.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.807+0000 7fc719ffb640 1 -- 172.21.15.67:0/3436279476 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc7100e3d40 con 0x7fc71c107e60 2024-07-31T14:50:35.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.808+0000 7fc720f81640 1 --2- 172.21.15.67:0/3436279476 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc7040b3c50 0x7fc7040b6160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc70c000c00 tx=0x7fc70c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:35.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:35.812+0000 7fc719ffb640 1 -- 172.21.15.67:0/3436279476 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc710014030 con 0x7fc71c107e60 2024-07-31T14:50:36.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:36.048+0000 7fc7231e5640 1 -- 172.21.15.67:0/3436279476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc71c102930 con 0x7fc71c107e60 2024-07-31T14:50:36.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:36.048+0000 7fc719ffb640 1 -- 172.21.15.67:0/3436279476 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3122 (secure 0 0 0) 0x7fc7100aaf60 con 0x7fc71c107e60 2024-07-31T14:50:36.050 INFO:teuthology.orchestra.run.smithi067.stdout:e29 2024-07-31T14:50:36.050 INFO:teuthology.orchestra.run.smithi067.stdout:btime 2024-07-31T14:48:06:596011+0000 2024-07-31T14:50:36.050 INFO:teuthology.orchestra.run.smithi067.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-31T14:50:36.050 INFO:teuthology.orchestra.run.smithi067.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-31T14:50:36.050 INFO:teuthology.orchestra.run.smithi067.stdout:legacy client fscid: 1 2024-07-31T14:50:36.050 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:50:36.050 INFO:teuthology.orchestra.run.smithi067.stdout:Filesystem 'cephfs' (1) 2024-07-31T14:50:36.050 INFO:teuthology.orchestra.run.smithi067.stdout:fs_name cephfs 2024-07-31T14:50:36.051 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 29 2024-07-31T14:50:36.051 INFO:teuthology.orchestra.run.smithi067.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-31T14:50:36.051 INFO:teuthology.orchestra.run.smithi067.stdout:created 2024-07-31T14:47:04.833142+0000 2024-07-31T14:50:36.051 INFO:teuthology.orchestra.run.smithi067.stdout:modified 2024-07-31T14:48:06.435900+0000 2024-07-31T14:50:36.051 INFO:teuthology.orchestra.run.smithi067.stdout:tableserver 0 2024-07-31T14:50:36.051 INFO:teuthology.orchestra.run.smithi067.stdout:root 0 2024-07-31T14:50:36.051 INFO:teuthology.orchestra.run.smithi067.stdout:session_timeout 300 2024-07-31T14:50:36.051 INFO:teuthology.orchestra.run.smithi067.stdout:session_autoclose 300 2024-07-31T14:50:36.051 INFO:teuthology.orchestra.run.smithi067.stdout:max_file_size 1099511627776 2024-07-31T14:50:36.051 INFO:teuthology.orchestra.run.smithi067.stdout:max_xattr_size 65536 2024-07-31T14:50:36.051 INFO:teuthology.orchestra.run.smithi067.stdout:required_client_features {} 2024-07-31T14:50:36.052 INFO:teuthology.orchestra.run.smithi067.stdout:last_failure 0 2024-07-31T14:50:36.052 INFO:teuthology.orchestra.run.smithi067.stdout:last_failure_osd_epoch 0 2024-07-31T14:50:36.052 INFO:teuthology.orchestra.run.smithi067.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-31T14:50:36.052 INFO:teuthology.orchestra.run.smithi067.stdout:max_mds 5 2024-07-31T14:50:36.052 INFO:teuthology.orchestra.run.smithi067.stdout:in 0,1,2,3,4 2024-07-31T14:50:36.052 INFO:teuthology.orchestra.run.smithi067.stdout:up {0=24671,1=24655,2=24650,3=24646,4=24634} 2024-07-31T14:50:36.052 INFO:teuthology.orchestra.run.smithi067.stdout:failed 2024-07-31T14:50:36.052 INFO:teuthology.orchestra.run.smithi067.stdout:damaged 2024-07-31T14:50:36.052 INFO:teuthology.orchestra.run.smithi067.stdout:stopped 2024-07-31T14:50:36.052 INFO:teuthology.orchestra.run.smithi067.stdout:data_pools [3,4] 2024-07-31T14:50:36.052 INFO:teuthology.orchestra.run.smithi067.stdout:metadata_pool 2 2024-07-31T14:50:36.053 INFO:teuthology.orchestra.run.smithi067.stdout:inline_data disabled 2024-07-31T14:50:36.053 INFO:teuthology.orchestra.run.smithi067.stdout:balancer 2024-07-31T14:50:36.053 INFO:teuthology.orchestra.run.smithi067.stdout:bal_rank_mask -1 2024-07-31T14:50:36.053 INFO:teuthology.orchestra.run.smithi067.stdout:standby_count_wanted 0 2024-07-31T14:50:36.053 INFO:teuthology.orchestra.run.smithi067.stdout:qdb_cluster leader: 24671 members: 24634,24646,24650,24655,24671 2024-07-31T14:50:36.053 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.b{0:24671} state up:active seq 5 addr [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] compat {c=[1],r=[1],i=[1fff]}] 2024-07-31T14:50:36.053 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.e{0:24629} state up:standby-replay seq 1 addr [v2:172.21.15.73:6834/2854264361,v1:172.21.15.73:6835/2854264361] compat {c=[1],r=[1],i=[1fff]}] 2024-07-31T14:50:36.053 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.i{1:24655} state up:active seq 21 addr [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] compat {c=[1],r=[1],i=[1fff]}] 2024-07-31T14:50:36.053 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.l{1:24604} state up:standby-replay seq 1 addr [v2:172.21.15.77:6832/2710994774,v1:172.21.15.77:6833/2710994774] compat {c=[1],r=[1],i=[1fff]}] 2024-07-31T14:50:36.053 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.h{2:24650} state up:active seq 22 addr [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] compat {c=[1],r=[1],i=[1fff]}] 2024-07-31T14:50:36.053 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.k{2:24602} state up:standby-replay seq 1 addr [v2:172.21.15.73:6832/1184537619,v1:172.21.15.73:6833/1184537619] compat {c=[1],r=[1],i=[1fff]}] 2024-07-31T14:50:36.053 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.c{3:24646} state up:active seq 23 addr [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] compat {c=[1],r=[1],i=[1fff]}] 2024-07-31T14:50:36.054 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a{3:14790} state up:standby-replay seq 1 addr [v2:172.21.15.67:6840/1797875282,v1:172.21.15.67:6841/1797875282] compat {c=[1],r=[1],i=[1fff]}] 2024-07-31T14:50:36.054 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.f{4:24634} state up:active seq 24 addr [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] compat {c=[1],r=[1],i=[1fff]}] 2024-07-31T14:50:36.054 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.g{4:14778} state up:standby-replay seq 1 addr [v2:172.21.15.67:6838/3532250223,v1:172.21.15.67:6839/3532250223] compat {c=[1],r=[1],i=[1fff]}] 2024-07-31T14:50:36.054 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:50:36.054 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:50:36.054 INFO:teuthology.orchestra.run.smithi067.stdout:Standby daemons: 2024-07-31T14:50:36.054 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:50:36.054 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.j{-1:14721} state up:standby seq 1 addr [v2:172.21.15.67:6834/952957104,v1:172.21.15.67:6835/952957104] compat {c=[1],r=[1],i=[1fff]}] 2024-07-31T14:50:36.054 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.d{-1:14727} state up:standby seq 1 addr [v2:172.21.15.67:6836/505336439,v1:172.21.15.67:6837/505336439] compat {c=[1],r=[1],i=[1fff]}] 2024-07-31T14:50:36.055 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-07-31T14:50:36.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:36.050+0000 7fc7231e5640 1 -- 172.21.15.67:0/3436279476 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc7040b3c50 msgr2=0x7fc7040b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:36.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:36.050+0000 7fc7231e5640 1 --2- 172.21.15.67:0/3436279476 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc7040b3c50 0x7fc7040b6160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc70c000c00 tx=0x7fc70c031400 comp rx=0 tx=0).stop 2024-07-31T14:50:36.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:36.050+0000 7fc7231e5640 1 -- 172.21.15.67:0/3436279476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71c107e60 msgr2=0x7fc71c19b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:36.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:36.050+0000 7fc7231e5640 1 --2- 172.21.15.67:0/3436279476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71c107e60 0x7fc71c19b290 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc71000c970 tx=0x7fc71000ce30 comp rx=0 tx=0).stop 2024-07-31T14:50:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:36.050+0000 7fc7231e5640 1 -- 172.21.15.67:0/3436279476 shutdown_connections 2024-07-31T14:50:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:36.051+0000 7fc7231e5640 1 --2- 172.21.15.67:0/3436279476 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc7040b3c50 0x7fc7040b6160 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:36.051+0000 7fc7231e5640 1 --2- 172.21.15.67:0/3436279476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc71c1087b0 0x7fc71c19b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:36.051+0000 7fc7231e5640 1 --2- 172.21.15.67:0/3436279476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71c107e60 0x7fc71c19b290 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:36.051+0000 7fc7231e5640 1 --2- 172.21.15.67:0/3436279476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc71c102520 0x7fc71c19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:36.051+0000 7fc7231e5640 1 -- 172.21.15.67:0/3436279476 >> 172.21.15.67:0/3436279476 conn(0x7fc71c0791a0 msgr2=0x7fc71c105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:36.051+0000 7fc7231e5640 1 -- 172.21.15.67:0/3436279476 shutdown_connections 2024-07-31T14:50:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:36.051+0000 7fc7231e5640 1 -- 172.21.15.67:0/3436279476 wait complete. 2024-07-31T14:50:36.744 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- bash -c 'ceph osd dump' 2024-07-31T14:50:37.010 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:37.270 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:37 smithi067 ceph-mon[32750]: pgmap v385: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:37.270 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:37 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3436279476' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-31T14:50:37.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:37 smithi073 ceph-mon[36714]: pgmap v385: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:37.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:37 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3436279476' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-31T14:50:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:37 smithi077 ceph-mon[37327]: pgmap v385: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:37 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3436279476' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-31T14:50:38.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.689+0000 7f5876def640 1 -- 172.21.15.67:0/2198315172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5870102310 msgr2=0x7f5870111fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:38.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.689+0000 7f5876def640 1 --2- 172.21.15.67:0/2198315172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5870102310 0x7f5870111fb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5868009900 tx=0x7f586802a410 comp rx=0 tx=0).stop 2024-07-31T14:50:38.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.689+0000 7f5876def640 1 -- 172.21.15.67:0/2198315172 shutdown_connections 2024-07-31T14:50:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.690+0000 7f5876def640 1 --2- 172.21.15.67:0/2198315172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5870102310 0x7f5870111fb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.690+0000 7f5876def640 1 --2- 172.21.15.67:0/2198315172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5870101820 0x7f5870101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.690+0000 7f5876def640 1 --2- 172.21.15.67:0/2198315172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5870104110 0x7f5870104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.690+0000 7f5876def640 1 -- 172.21.15.67:0/2198315172 >> 172.21.15.67:0/2198315172 conn(0x7f58700799c0 msgr2=0x7f5870079dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.690+0000 7f5876def640 1 -- 172.21.15.67:0/2198315172 shutdown_connections 2024-07-31T14:50:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.690+0000 7f5876def640 1 -- 172.21.15.67:0/2198315172 wait complete. 2024-07-31T14:50:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.690+0000 7f5876def640 1 Processor -- start 2024-07-31T14:50:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.690+0000 7f5876def640 1 -- start start 2024-07-31T14:50:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5876def640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5870101820 0x7f587019c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5876def640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5870102310 0x7f587019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5876def640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5870104110 0x7f58701a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5876def640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5870115730 con 0x7f5870101820 2024-07-31T14:50:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5876def640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f58701155b0 con 0x7f5870104110 2024-07-31T14:50:38.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5876def640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58701158b0 con 0x7f5870102310 2024-07-31T14:50:38.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5874b8b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5870101820 0x7f587019c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:38.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5874b8b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5870101820 0x7f587019c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49390/0 (socket says 172.21.15.67:49390) 2024-07-31T14:50:38.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5874b8b640 1 -- 172.21.15.67:0/2821963693 learned_addr learned my addr 172.21.15.67:0/2821963693 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:38.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f587538c640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5870104110 0x7f58701a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:38.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f586ffff640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5870102310 0x7f587019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:38.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5874b8b640 1 -- 172.21.15.67:0/2821963693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5870102310 msgr2=0x7f587019c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5874b8b640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5870102310 0x7f587019c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5874b8b640 1 -- 172.21.15.67:0/2821963693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5870104110 msgr2=0x7f58701a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5874b8b640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5870104110 0x7f58701a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5874b8b640 1 -- 172.21.15.67:0/2821963693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58701a1f90 con 0x7f5870101820 2024-07-31T14:50:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f587538c640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5870104110 0x7f58701a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:50:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f586ffff640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5870102310 0x7f587019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:50:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f5874b8b640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5870101820 0x7f587019c0e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f585800c920 tx=0x7f585800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.691+0000 7f586dffb640 1 -- 172.21.15.67:0/2821963693 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858007c70 con 0x7f5870101820 2024-07-31T14:50:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.692+0000 7f586dffb640 1 -- 172.21.15.67:0/2821963693 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5858007e10 con 0x7f5870101820 2024-07-31T14:50:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.692+0000 7f5876def640 1 -- 172.21.15.67:0/2821963693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58701a2280 con 0x7f5870101820 2024-07-31T14:50:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.692+0000 7f586dffb640 1 -- 172.21.15.67:0/2821963693 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858018820 con 0x7f5870101820 2024-07-31T14:50:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.692+0000 7f5876def640 1 -- 172.21.15.67:0/2821963693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58701a8aa0 con 0x7f5870101820 2024-07-31T14:50:38.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.693+0000 7f5876def640 1 -- 172.21.15.67:0/2821963693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5870102830 con 0x7f5870101820 2024-07-31T14:50:38.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.694+0000 7f586dffb640 1 -- 172.21.15.67:0/2821963693 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f585800f450 con 0x7f5870101820 2024-07-31T14:50:38.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.695+0000 7f586dffb640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f585c0b3c50 0x7f585c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:38.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.695+0000 7f586ffff640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f585c0b3c50 0x7f585c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:38.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.695+0000 7f586dffb640 1 -- 172.21.15.67:0/2821963693 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f58580e4190 con 0x7f5870101820 2024-07-31T14:50:38.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.696+0000 7f586ffff640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f585c0b3c50 0x7f585c0b6160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f587007a090 tx=0x7f58600082f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:38.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.699+0000 7f586dffb640 1 -- 172.21.15.67:0/2821963693 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5858014030 con 0x7f5870101820 2024-07-31T14:50:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.896+0000 7f5876def640 1 -- 172.21.15.67:0/2821963693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f58701a8d50 con 0x7f5870101820 2024-07-31T14:50:38.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.897+0000 7f586dffb640 1 -- 172.21.15.67:0/2821963693 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5955 (secure 0 0 0) 0x7f58580cd070 con 0x7f5870101820 2024-07-31T14:50:38.899 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 89 2024-07-31T14:50:38.899 INFO:teuthology.orchestra.run.smithi067.stdout:fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T14:50:38.899 INFO:teuthology.orchestra.run.smithi067.stdout:created 2024-07-31T14:38:19.925705+0000 2024-07-31T14:50:38.899 INFO:teuthology.orchestra.run.smithi067.stdout:modified 2024-07-31T14:47:20.440064+0000 2024-07-31T14:50:38.899 INFO:teuthology.orchestra.run.smithi067.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-07-31T14:50:38.899 INFO:teuthology.orchestra.run.smithi067.stdout:crush_version 27 2024-07-31T14:50:38.899 INFO:teuthology.orchestra.run.smithi067.stdout:full_ratio 0.95 2024-07-31T14:50:38.899 INFO:teuthology.orchestra.run.smithi067.stdout:backfillfull_ratio 0.9 2024-07-31T14:50:38.900 INFO:teuthology.orchestra.run.smithi067.stdout:nearfull_ratio 0.85 2024-07-31T14:50:38.900 INFO:teuthology.orchestra.run.smithi067.stdout:require_min_compat_client luminous 2024-07-31T14:50:38.900 INFO:teuthology.orchestra.run.smithi067.stdout:min_compat_client luminous 2024-07-31T14:50:38.900 INFO:teuthology.orchestra.run.smithi067.stdout:require_osd_release squid 2024-07-31T14:50:38.900 INFO:teuthology.orchestra.run.smithi067.stdout:stretch_mode_enabled false 2024-07-31T14:50:38.900 INFO:teuthology.orchestra.run.smithi067.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-07-31T14:50:38.900 INFO:teuthology.orchestra.run.smithi067.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-07-31T14:50:38.900 INFO:teuthology.orchestra.run.smithi067.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-07-31T14:50:38.900 INFO:teuthology.orchestra.run.smithi067.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-07-31T14:50:38.900 INFO:teuthology.orchestra.run.smithi067.stdout:max_osd 12 2024-07-31T14:50:38.900 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 up in weight 1 up_from 9 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6802/2442400504,v1:172.21.15.67:6803/2442400504] [v2:172.21.15.67:6804/2442400504,v1:172.21.15.67:6805/2442400504] exists,up 1469d842-7dd3-4a23-969f-b98d853dbc8a 2024-07-31T14:50:38.900 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 up in weight 1 up_from 14 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6800/1103939318,v1:172.21.15.73:6801/1103939318] [v2:172.21.15.73:6802/1103939318,v1:172.21.15.73:6803/1103939318] exists,up 329506f6-e103-4178-bcf7-10913f292047 2024-07-31T14:50:38.901 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 up in weight 1 up_from 19 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6800/1624991750,v1:172.21.15.77:6801/1624991750] [v2:172.21.15.77:6802/1624991750,v1:172.21.15.77:6803/1624991750] exists,up 3855f3c9-0985-4dcd-a1f6-97dd54d1d833 2024-07-31T14:50:38.901 INFO:teuthology.orchestra.run.smithi067.stdout:osd.3 up in weight 1 up_from 27 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6810/3100079409,v1:172.21.15.67:6811/3100079409] [v2:172.21.15.67:6812/3100079409,v1:172.21.15.67:6813/3100079409] exists,up c1693ca0-afc8-4cbd-abef-251f8f4c42f0 2024-07-31T14:50:38.901 INFO:teuthology.orchestra.run.smithi067.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6808/3387924579,v1:172.21.15.73:6809/3387924579] [v2:172.21.15.73:6810/3387924579,v1:172.21.15.73:6811/3387924579] exists,up 17975063-5f3d-4a52-84f4-6d54ce4fce34 2024-07-31T14:50:38.901 INFO:teuthology.orchestra.run.smithi067.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6808/683909212,v1:172.21.15.77:6809/683909212] [v2:172.21.15.77:6810/683909212,v1:172.21.15.77:6811/683909212] exists,up e41e714e-62fa-4946-923b-56f8ee66fb86 2024-07-31T14:50:38.901 INFO:teuthology.orchestra.run.smithi067.stdout:osd.6 up in weight 1 up_from 44 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6818/380463157,v1:172.21.15.67:6819/380463157] [v2:172.21.15.67:6820/380463157,v1:172.21.15.67:6821/380463157] exists,up b20c9a1c-1462-4596-915a-70c38b3b8c76 2024-07-31T14:50:38.901 INFO:teuthology.orchestra.run.smithi067.stdout:osd.7 up in weight 1 up_from 49 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6816/2240196566,v1:172.21.15.73:6817/2240196566] [v2:172.21.15.73:6818/2240196566,v1:172.21.15.73:6819/2240196566] exists,up 8cb608dc-dd03-44a4-8e6f-1fce67757c0d 2024-07-31T14:50:38.901 INFO:teuthology.orchestra.run.smithi067.stdout:osd.8 up in weight 1 up_from 55 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6816/2208692754,v1:172.21.15.77:6817/2208692754] [v2:172.21.15.77:6818/2208692754,v1:172.21.15.77:6819/2208692754] exists,up 2b2baf2b-fe02-4048-8adb-de4d19f545d6 2024-07-31T14:50:38.901 INFO:teuthology.orchestra.run.smithi067.stdout:osd.9 up in weight 1 up_from 61 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6826/3242854060,v1:172.21.15.67:6827/3242854060] [v2:172.21.15.67:6828/3242854060,v1:172.21.15.67:6829/3242854060] exists,up 246aa076-dae2-448a-a294-2b0ca5ac3368 2024-07-31T14:50:38.901 INFO:teuthology.orchestra.run.smithi067.stdout:osd.10 up in weight 1 up_from 66 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6824/1494741997,v1:172.21.15.73:6825/1494741997] [v2:172.21.15.73:6826/1494741997,v1:172.21.15.73:6827/1494741997] exists,up 65a6d993-7acb-467d-abf4-92d5e443438e 2024-07-31T14:50:38.901 INFO:teuthology.orchestra.run.smithi067.stdout:osd.11 up in weight 1 up_from 72 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6824/764558961,v1:172.21.15.77:6825/764558961] [v2:172.21.15.77:6826/764558961,v1:172.21.15.77:6827/764558961] exists,up 709fb536-23a5-4c14-aed6-97fad837c210 2024-07-31T14:50:38.901 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.4 [2,11] 2024-07-31T14:50:38.901 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.1d [8,11] 2024-07-31T14:50:38.901 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.1 [4,9] 2024-07-31T14:50:38.902 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.1b [5,9] 2024-07-31T14:50:38.902 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.31 [4,8] 2024-07-31T14:50:38.902 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.3c [4,9] 2024-07-31T14:50:38.902 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.f [5,8] 2024-07-31T14:50:38.902 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.16 [9,10] 2024-07-31T14:50:38.902 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.19 [9,8] 2024-07-31T14:50:38.902 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.1f [9,8] 2024-07-31T14:50:38.902 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6801/3601135096 expires 2024-08-01T14:40:19.281779+0000 2024-07-31T14:50:38.902 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6800/3601135096 expires 2024-08-01T14:40:19.281779+0000 2024-07-31T14:50:38.902 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6800/4249598263 expires 2024-08-01T14:39:07.942187+0000 2024-07-31T14:50:38.902 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/927641174 expires 2024-08-01T14:38:41.486084+0000 2024-07-31T14:50:38.902 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3901988655 expires 2024-08-01T14:38:41.486084+0000 2024-07-31T14:50:38.903 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6800/1712042445 expires 2024-08-01T14:38:41.486084+0000 2024-07-31T14:50:38.903 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/1243394894 expires 2024-08-01T14:39:07.942187+0000 2024-07-31T14:50:38.903 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/1563471445 expires 2024-08-01T14:39:07.942187+0000 2024-07-31T14:50:38.903 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6801/1712042445 expires 2024-08-01T14:38:41.486084+0000 2024-07-31T14:50:38.903 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/76992902 expires 2024-08-01T14:39:07.942187+0000 2024-07-31T14:50:38.903 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3065587341 expires 2024-08-01T14:38:41.486084+0000 2024-07-31T14:50:38.903 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6801/4249598263 expires 2024-08-01T14:39:07.942187+0000 2024-07-31T14:50:38.903 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3411853619 expires 2024-08-01T14:40:19.281779+0000 2024-07-31T14:50:38.903 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/618685030 expires 2024-08-01T14:40:19.281779+0000 2024-07-31T14:50:38.903 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/2614585786 expires 2024-08-01T14:40:19.281779+0000 2024-07-31T14:50:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.900+0000 7f5876def640 1 -- 172.21.15.67:0/2821963693 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f585c0b3c50 msgr2=0x7f585c0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.900+0000 7f5876def640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f585c0b3c50 0x7f585c0b6160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f587007a090 tx=0x7f58600082f0 comp rx=0 tx=0).stop 2024-07-31T14:50:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.900+0000 7f5876def640 1 -- 172.21.15.67:0/2821963693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5870101820 msgr2=0x7f587019c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.900+0000 7f5876def640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5870101820 0x7f587019c0e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f585800c920 tx=0x7f585800cde0 comp rx=0 tx=0).stop 2024-07-31T14:50:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.900+0000 7f5876def640 1 -- 172.21.15.67:0/2821963693 shutdown_connections 2024-07-31T14:50:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.900+0000 7f5876def640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f585c0b3c50 0x7f585c0b6160 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.900+0000 7f5876def640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5870104110 0x7f58701a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.900+0000 7f5876def640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5870102310 0x7f587019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:38.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.900+0000 7f5876def640 1 --2- 172.21.15.67:0/2821963693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5870101820 0x7f587019c0e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:38.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.900+0000 7f5876def640 1 -- 172.21.15.67:0/2821963693 >> 172.21.15.67:0/2821963693 conn(0x7f58700799c0 msgr2=0x7f58700ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:38.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.900+0000 7f5876def640 1 -- 172.21.15.67:0/2821963693 shutdown_connections 2024-07-31T14:50:38.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:38.900+0000 7f5876def640 1 -- 172.21.15.67:0/2821963693 wait complete. 2024-07-31T14:50:39.214 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:39 smithi067 ceph-mon[32750]: pgmap v386: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:39.214 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:39 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2821963693' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-31T14:50:39.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:39 smithi073 ceph-mon[36714]: pgmap v386: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:39.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:39 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2821963693' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-31T14:50:39.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:39 smithi077 ceph-mon[37327]: pgmap v386: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:39.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:39 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2821963693' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-31T14:50:39.555 INFO:teuthology.run_tasks:Running task fs.ready... 2024-07-31T14:50:39.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:50:39.832 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:41.228 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:41 smithi067 ceph-mon[32750]: pgmap v387: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:41.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:41 smithi073 ceph-mon[36714]: pgmap v387: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.478+0000 7f16150e3640 1 -- 172.21.15.67:0/3901826452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1610107df0 msgr2=0x7f1610108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.478+0000 7f16150e3640 1 --2- 172.21.15.67:0/3901826452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1610107df0 0x7f1610108200 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1604009900 tx=0x7f160402a410 comp rx=0 tx=0).stop 2024-07-31T14:50:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.478+0000 7f16150e3640 1 -- 172.21.15.67:0/3901826452 shutdown_connections 2024-07-31T14:50:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.478+0000 7f16150e3640 1 --2- 172.21.15.67:0/3901826452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1610108740 0x7f161010ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.478+0000 7f16150e3640 1 --2- 172.21.15.67:0/3901826452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1610107df0 0x7f1610108200 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.478+0000 7f16150e3640 1 --2- 172.21.15.67:0/3901826452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16101024b0 0x7f16101028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.478+0000 7f16150e3640 1 -- 172.21.15.67:0/3901826452 >> 172.21.15.67:0/3901826452 conn(0x7f1610079170 msgr2=0x7f1610079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.478+0000 7f16150e3640 1 -- 172.21.15.67:0/3901826452 shutdown_connections 2024-07-31T14:50:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.479+0000 7f16150e3640 1 -- 172.21.15.67:0/3901826452 wait complete. 2024-07-31T14:50:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.479+0000 7f16150e3640 1 Processor -- start 2024-07-31T14:50:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.480+0000 7f16150e3640 1 -- start start 2024-07-31T14:50:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.480+0000 7f16150e3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16101024b0 0x7f1610199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.480+0000 7f16150e3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1610107df0 0x7f161019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.480+0000 7f16150e3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1610108740 0x7f161019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.480+0000 7f16150e3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f16101123f0 con 0x7f16101024b0 2024-07-31T14:50:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.480+0000 7f16150e3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1610112270 con 0x7f1610107df0 2024-07-31T14:50:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.480+0000 7f16150e3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1610112570 con 0x7f1610108740 2024-07-31T14:50:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.480+0000 7f160ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16101024b0 0x7f1610199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.480+0000 7f160ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16101024b0 0x7f1610199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59230/0 (socket says 172.21.15.67:59230) 2024-07-31T14:50:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.480+0000 7f160ed9d640 1 -- 172.21.15.67:0/3496600356 learned_addr learned my addr 172.21.15.67:0/3496600356 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.480+0000 7f160e59c640 1 --2- 172.21.15.67:0/3496600356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1610107df0 0x7f161019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.480+0000 7f160f59e640 1 --2- 172.21.15.67:0/3496600356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1610108740 0x7f161019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.481+0000 7f160ed9d640 1 -- 172.21.15.67:0/3496600356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1610108740 msgr2=0x7f161019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.481+0000 7f160ed9d640 1 --2- 172.21.15.67:0/3496600356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1610108740 0x7f161019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:41.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.481+0000 7f160ed9d640 1 -- 172.21.15.67:0/3496600356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1610107df0 msgr2=0x7f161019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:41.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.481+0000 7f160ed9d640 1 --2- 172.21.15.67:0/3496600356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1610107df0 0x7f161019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:41.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.481+0000 7f160ed9d640 1 -- 172.21.15.67:0/3496600356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161019fea0 con 0x7f16101024b0 2024-07-31T14:50:41.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.481+0000 7f160ed9d640 1 --2- 172.21.15.67:0/3496600356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16101024b0 0x7f1610199f30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f160000ba50 tx=0x7f160000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:41.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.481+0000 7f15f7fff640 1 -- 172.21.15.67:0/3496600356 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600019070 con 0x7f16101024b0 2024-07-31T14:50:41.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.481+0000 7f16150e3640 1 -- 172.21.15.67:0/3496600356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16101a0190 con 0x7f16101024b0 2024-07-31T14:50:41.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.481+0000 7f15f7fff640 1 -- 172.21.15.67:0/3496600356 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f160000cbe0 con 0x7f16101024b0 2024-07-31T14:50:41.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.481+0000 7f15f7fff640 1 -- 172.21.15.67:0/3496600356 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16000075d0 con 0x7f16101024b0 2024-07-31T14:50:41.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.481+0000 7f16150e3640 1 -- 172.21.15.67:0/3496600356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16101a68f0 con 0x7f16101024b0 2024-07-31T14:50:41.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.482+0000 7f16150e3640 1 -- 172.21.15.67:0/3496600356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1610103730 con 0x7f16101024b0 2024-07-31T14:50:41.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.484+0000 7f15f7fff640 1 -- 172.21.15.67:0/3496600356 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f160001c070 con 0x7f16101024b0 2024-07-31T14:50:41.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.485+0000 7f15f7fff640 1 --2- 172.21.15.67:0/3496600356 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15f80b3c20 0x7f15f80b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:41.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.485+0000 7f160e59c640 1 --2- 172.21.15.67:0/3496600356 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15f80b3c20 0x7f15f80b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:41.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.485+0000 7f15f7fff640 1 -- 172.21.15.67:0/3496600356 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f16000e3670 con 0x7f16101024b0 2024-07-31T14:50:41.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.485+0000 7f160e59c640 1 --2- 172.21.15.67:0/3496600356 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15f80b3c20 0x7f15f80b6130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1604000c00 tx=0x7f1604031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:41.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:41 smithi077 ceph-mon[37327]: pgmap v387: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:41.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.489+0000 7f15f7fff640 1 -- 172.21.15.67:0/3496600356 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f16000aa240 con 0x7f16101024b0 2024-07-31T14:50:41.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.722+0000 7f16150e3640 1 -- 172.21.15.67:0/3496600356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16101028c0 con 0x7f16101024b0 2024-07-31T14:50:41.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.723+0000 7f15f7fff640 1 -- 172.21.15.67:0/3496600356 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11962 (secure 0 0 0) 0x7f16000ae790 con 0x7f16101024b0 2024-07-31T14:50:41.724 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:50:41.724 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-07-31T14:48:06:596011+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:48:06.435900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T14:50:41.725 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-07-31T14:50:41.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.725+0000 7f16150e3640 1 -- 172.21.15.67:0/3496600356 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15f80b3c20 msgr2=0x7f15f80b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:41.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.725+0000 7f16150e3640 1 --2- 172.21.15.67:0/3496600356 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15f80b3c20 0x7f15f80b6130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1604000c00 tx=0x7f1604031400 comp rx=0 tx=0).stop 2024-07-31T14:50:41.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.725+0000 7f16150e3640 1 -- 172.21.15.67:0/3496600356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16101024b0 msgr2=0x7f1610199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:41.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.725+0000 7f16150e3640 1 --2- 172.21.15.67:0/3496600356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16101024b0 0x7f1610199f30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f160000ba50 tx=0x7f160000bf10 comp rx=0 tx=0).stop 2024-07-31T14:50:41.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.725+0000 7f16150e3640 1 -- 172.21.15.67:0/3496600356 shutdown_connections 2024-07-31T14:50:41.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.725+0000 7f16150e3640 1 --2- 172.21.15.67:0/3496600356 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15f80b3c20 0x7f15f80b6130 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:41.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.725+0000 7f16150e3640 1 --2- 172.21.15.67:0/3496600356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1610108740 0x7f161019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:41.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.725+0000 7f16150e3640 1 --2- 172.21.15.67:0/3496600356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1610107df0 0x7f161019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:41.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.725+0000 7f16150e3640 1 --2- 172.21.15.67:0/3496600356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16101024b0 0x7f1610199f30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:41.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.725+0000 7f16150e3640 1 -- 172.21.15.67:0/3496600356 >> 172.21.15.67:0/3496600356 conn(0x7f1610079170 msgr2=0x7f161006b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:41.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.725+0000 7f16150e3640 1 -- 172.21.15.67:0/3496600356 shutdown_connections 2024-07-31T14:50:41.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:41.725+0000 7f16150e3640 1 -- 172.21.15.67:0/3496600356 wait complete. 2024-07-31T14:50:42.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:42 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3496600356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:50:42.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:42 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3496600356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:50:42.457 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:50:42.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:42 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3496600356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:50:42.725 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:43.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:43 smithi067 ceph-mon[32750]: pgmap v388: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:43.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:43 smithi073 ceph-mon[36714]: pgmap v388: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:43.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:43 smithi077 ceph-mon[37327]: pgmap v388: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:44.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.447+0000 7f6ce33dd640 1 -- 172.21.15.67:0/217732936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cdc1024b0 msgr2=0x7f6cdc1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:44.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.447+0000 7f6ce33dd640 1 --2- 172.21.15.67:0/217732936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cdc1024b0 0x7f6cdc1028c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009900 tx=0x7f6cd802a410 comp rx=0 tx=0).stop 2024-07-31T14:50:44.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.447+0000 7f6ce33dd640 1 -- 172.21.15.67:0/217732936 shutdown_connections 2024-07-31T14:50:44.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.447+0000 7f6ce33dd640 1 --2- 172.21.15.67:0/217732936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cdc108740 0x7f6cdc10ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:44.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.447+0000 7f6ce33dd640 1 --2- 172.21.15.67:0/217732936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cdc107df0 0x7f6cdc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:44.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.447+0000 7f6ce33dd640 1 --2- 172.21.15.67:0/217732936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cdc1024b0 0x7f6cdc1028c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:44.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.447+0000 7f6ce33dd640 1 -- 172.21.15.67:0/217732936 >> 172.21.15.67:0/217732936 conn(0x7f6cdc079170 msgr2=0x7f6cdc079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:44.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.447+0000 7f6ce33dd640 1 -- 172.21.15.67:0/217732936 shutdown_connections 2024-07-31T14:50:44.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.447+0000 7f6ce33dd640 1 -- 172.21.15.67:0/217732936 wait complete. 2024-07-31T14:50:44.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.448+0000 7f6ce33dd640 1 Processor -- start 2024-07-31T14:50:44.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.448+0000 7f6ce33dd640 1 -- start start 2024-07-31T14:50:44.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.448+0000 7f6ce33dd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cdc1024b0 0x7f6cdc197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:44.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.448+0000 7f6ce33dd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cdc107df0 0x7f6cdc198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:44.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.448+0000 7f6ce33dd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cdc108740 0x7f6cdc19d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:44.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.448+0000 7f6ce33dd640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc1112b0 con 0x7f6cdc108740 2024-07-31T14:50:44.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.448+0000 7f6ce33dd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc111130 con 0x7f6cdc1024b0 2024-07-31T14:50:44.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.448+0000 7f6ce33dd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc111430 con 0x7f6cdc107df0 2024-07-31T14:50:44.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.448+0000 7f6ce197a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cdc108740 0x7f6cdc19d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:44.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.448+0000 7f6ce197a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cdc108740 0x7f6cdc19d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59258/0 (socket says 172.21.15.67:59258) 2024-07-31T14:50:44.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.448+0000 7f6ce197a640 1 -- 172.21.15.67:0/3656514682 learned_addr learned my addr 172.21.15.67:0/3656514682 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:44.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.448+0000 7f6ce1179640 1 --2- 172.21.15.67:0/3656514682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cdc1024b0 0x7f6cdc197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:44.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.448+0000 7f6ce0978640 1 --2- 172.21.15.67:0/3656514682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cdc107df0 0x7f6cdc198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:44.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.449+0000 7f6ce197a640 1 -- 172.21.15.67:0/3656514682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cdc107df0 msgr2=0x7f6cdc198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:44.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.449+0000 7f6ce197a640 1 --2- 172.21.15.67:0/3656514682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cdc107df0 0x7f6cdc198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:44.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.449+0000 7f6ce197a640 1 -- 172.21.15.67:0/3656514682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cdc1024b0 msgr2=0x7f6cdc197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:44.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.449+0000 7f6ce197a640 1 --2- 172.21.15.67:0/3656514682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cdc1024b0 0x7f6cdc197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:44.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.449+0000 7f6ce197a640 1 -- 172.21.15.67:0/3656514682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc19dc10 con 0x7f6cdc108740 2024-07-31T14:50:44.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.449+0000 7f6ce197a640 1 --2- 172.21.15.67:0/3656514682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cdc108740 0x7f6cdc19d4e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400eaa0 tx=0x7f6cd400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:44.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.449+0000 7f6cd27fc640 1 -- 172.21.15.67:0/3656514682 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd400cdb0 con 0x7f6cdc108740 2024-07-31T14:50:44.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.449+0000 7f6ce33dd640 1 -- 172.21.15.67:0/3656514682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cdc19df00 con 0x7f6cdc108740 2024-07-31T14:50:44.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.449+0000 7f6cd27fc640 1 -- 172.21.15.67:0/3656514682 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cd4005020 con 0x7f6cdc108740 2024-07-31T14:50:44.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.449+0000 7f6cd27fc640 1 -- 172.21.15.67:0/3656514682 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd40116d0 con 0x7f6cdc108740 2024-07-31T14:50:44.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.449+0000 7f6ce33dd640 1 -- 172.21.15.67:0/3656514682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cdc1a4660 con 0x7f6cdc108740 2024-07-31T14:50:44.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.450+0000 7f6ce33dd640 1 -- 172.21.15.67:0/3656514682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cdc103730 con 0x7f6cdc108740 2024-07-31T14:50:44.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.452+0000 7f6cd27fc640 1 -- 172.21.15.67:0/3656514682 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f6cd40054a0 con 0x7f6cdc108740 2024-07-31T14:50:44.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.453+0000 7f6cd27fc640 1 --2- 172.21.15.67:0/3656514682 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6cc80b3c20 0x7f6cc80b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:44.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.453+0000 7f6ce1179640 1 --2- 172.21.15.67:0/3656514682 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6cc80b3c20 0x7f6cc80b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:44.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.453+0000 7f6cd27fc640 1 -- 172.21.15.67:0/3656514682 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f6cd40e4560 con 0x7f6cdc108740 2024-07-31T14:50:44.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.453+0000 7f6ce1179640 1 --2- 172.21.15.67:0/3656514682 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6cc80b3c20 0x7f6cc80b6130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8031440 tx=0x7f6cd8009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:44.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.456+0000 7f6cd27fc640 1 -- 172.21.15.67:0/3656514682 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f6cd4016030 con 0x7f6cdc108740 2024-07-31T14:50:44.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.688+0000 7f6ce33dd640 1 -- 172.21.15.67:0/3656514682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cdc1028c0 con 0x7f6cdc108740 2024-07-31T14:50:44.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.688+0000 7f6cd27fc640 1 -- 172.21.15.67:0/3656514682 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11962 (secure 0 0 0) 0x7f6cd40ab130 con 0x7f6cdc108740 2024-07-31T14:50:44.690 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:50:44.690 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-07-31T14:48:06:596011+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:48:06.435900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T14:50:44.690 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-07-31T14:50:44.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.690+0000 7f6ce33dd640 1 -- 172.21.15.67:0/3656514682 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6cc80b3c20 msgr2=0x7f6cc80b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:44.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.690+0000 7f6ce33dd640 1 --2- 172.21.15.67:0/3656514682 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6cc80b3c20 0x7f6cc80b6130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8031440 tx=0x7f6cd8009560 comp rx=0 tx=0).stop 2024-07-31T14:50:44.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.690+0000 7f6ce33dd640 1 -- 172.21.15.67:0/3656514682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cdc108740 msgr2=0x7f6cdc19d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:44.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.690+0000 7f6ce33dd640 1 --2- 172.21.15.67:0/3656514682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cdc108740 0x7f6cdc19d4e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400eaa0 tx=0x7f6cd400ef60 comp rx=0 tx=0).stop 2024-07-31T14:50:44.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.691+0000 7f6ce33dd640 1 -- 172.21.15.67:0/3656514682 shutdown_connections 2024-07-31T14:50:44.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.691+0000 7f6ce33dd640 1 --2- 172.21.15.67:0/3656514682 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6cc80b3c20 0x7f6cc80b6130 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:44.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.691+0000 7f6ce33dd640 1 --2- 172.21.15.67:0/3656514682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cdc108740 0x7f6cdc19d4e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:44.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.691+0000 7f6ce33dd640 1 --2- 172.21.15.67:0/3656514682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cdc107df0 0x7f6cdc198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:44.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.691+0000 7f6ce33dd640 1 --2- 172.21.15.67:0/3656514682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cdc1024b0 0x7f6cdc197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:44.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.691+0000 7f6ce33dd640 1 -- 172.21.15.67:0/3656514682 >> 172.21.15.67:0/3656514682 conn(0x7f6cdc079170 msgr2=0x7f6cdc106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:44.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.691+0000 7f6ce33dd640 1 -- 172.21.15.67:0/3656514682 shutdown_connections 2024-07-31T14:50:44.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:44.691+0000 7f6ce33dd640 1 -- 172.21.15.67:0/3656514682 wait complete. 2024-07-31T14:50:45.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:45 smithi067 ceph-mon[32750]: pgmap v389: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:45.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:45 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3656514682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:50:45.374 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-07-31T14:50:45.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:45 smithi073 ceph-mon[36714]: pgmap v389: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:45.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:45 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3656514682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:50:45.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:45 smithi077 ceph-mon[37327]: pgmap v389: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:45.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:45 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3656514682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:50:45.663 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:50:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:50:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:50:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:50:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:50:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:50:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:46.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:50:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:50:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:50:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:50:47.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:47 smithi067 ceph-mon[32750]: pgmap v390: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.382+0000 7fd8ba568640 1 -- 172.21.15.67:0/4241205159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b4102950 msgr2=0x7fd8b410f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.382+0000 7fd8ba568640 1 --2- 172.21.15.67:0/4241205159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b4102950 0x7fd8b410f670 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009900 tx=0x7fd8a802a480 comp rx=0 tx=0).stop 2024-07-31T14:50:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.382+0000 7fd8ba568640 1 -- 172.21.15.67:0/4241205159 shutdown_connections 2024-07-31T14:50:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.382+0000 7fd8ba568640 1 --2- 172.21.15.67:0/4241205159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8b410fbb0 0x7fd8b4111fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.382+0000 7fd8ba568640 1 --2- 172.21.15.67:0/4241205159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b4102950 0x7fd8b410f670 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.382+0000 7fd8ba568640 1 --2- 172.21.15.67:0/4241205159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8b4102000 0x7fd8b4102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.382+0000 7fd8ba568640 1 -- 172.21.15.67:0/4241205159 >> 172.21.15.67:0/4241205159 conn(0x7fd8b4079160 msgr2=0x7fd8b4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.382+0000 7fd8ba568640 1 -- 172.21.15.67:0/4241205159 shutdown_connections 2024-07-31T14:50:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.382+0000 7fd8ba568640 1 -- 172.21.15.67:0/4241205159 wait complete. 2024-07-31T14:50:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.383+0000 7fd8ba568640 1 Processor -- start 2024-07-31T14:50:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.383+0000 7fd8ba568640 1 -- start start 2024-07-31T14:50:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.383+0000 7fd8ba568640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8b4102000 0x7fd8b419c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.383+0000 7fd8ba568640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b4102950 0x7fd8b419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.383+0000 7fd8ba568640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8b410fbb0 0x7fd8b41a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.383+0000 7fd8ba568640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b4115640 con 0x7fd8b4102950 2024-07-31T14:50:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.383+0000 7fd8ba568640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b41154c0 con 0x7fd8b410fbb0 2024-07-31T14:50:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.383+0000 7fd8ba568640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b41157c0 con 0x7fd8b4102000 2024-07-31T14:50:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.383+0000 7fd8b37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b4102950 0x7fd8b419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.383+0000 7fd8b37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b4102950 0x7fd8b419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59268/0 (socket says 172.21.15.67:59268) 2024-07-31T14:50:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.383+0000 7fd8b37fe640 1 -- 172.21.15.67:0/3375300927 learned_addr learned my addr 172.21.15.67:0/3375300927 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.383+0000 7fd8b3fff640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8b4102000 0x7fd8b419c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.384+0000 7fd8b8b05640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8b410fbb0 0x7fd8b41a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.384+0000 7fd8b37fe640 1 -- 172.21.15.67:0/3375300927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8b4102000 msgr2=0x7fd8b419c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:47.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.384+0000 7fd8b37fe640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8b4102000 0x7fd8b419c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:47.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.384+0000 7fd8b37fe640 1 -- 172.21.15.67:0/3375300927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8b410fbb0 msgr2=0x7fd8b41a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:47.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.384+0000 7fd8b37fe640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8b410fbb0 0x7fd8b41a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:47.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.384+0000 7fd8b37fe640 1 -- 172.21.15.67:0/3375300927 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b41a2000 con 0x7fd8b4102950 2024-07-31T14:50:47.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.384+0000 7fd8b8b05640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8b410fbb0 0x7fd8b41a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:50:47.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.384+0000 7fd8b37fe640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b4102950 0x7fd8b419c630 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8005940 tx=0x7fd8a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:47.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.384+0000 7fd8b17fa640 1 -- 172.21.15.67:0/3375300927 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a803f030 con 0x7fd8b4102950 2024-07-31T14:50:47.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.384+0000 7fd8b17fa640 1 -- 172.21.15.67:0/3375300927 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8a8003cb0 con 0x7fd8b4102950 2024-07-31T14:50:47.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.384+0000 7fd8ba568640 1 -- 172.21.15.67:0/3375300927 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b41a2290 con 0x7fd8b4102950 2024-07-31T14:50:47.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.384+0000 7fd8b17fa640 1 -- 172.21.15.67:0/3375300927 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8035070 con 0x7fd8b4102950 2024-07-31T14:50:47.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.384+0000 7fd8ba568640 1 -- 172.21.15.67:0/3375300927 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b41a8a50 con 0x7fd8b4102950 2024-07-31T14:50:47.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.385+0000 7fd8ba568640 1 -- 172.21.15.67:0/3375300927 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b4102410 con 0x7fd8b4102950 2024-07-31T14:50:47.389 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:47 smithi077 ceph-mon[37327]: pgmap v390: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:47.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.387+0000 7fd8b17fa640 1 -- 172.21.15.67:0/3375300927 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fd8a8042030 con 0x7fd8b4102950 2024-07-31T14:50:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.388+0000 7fd8b17fa640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd89c0b3ca0 0x7fd89c0b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.388+0000 7fd8b3fff640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd89c0b3ca0 0x7fd89c0b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.388+0000 7fd8b17fa640 1 -- 172.21.15.67:0/3375300927 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fd8a8104be0 con 0x7fd8b4102950 2024-07-31T14:50:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.388+0000 7fd8b3fff640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd89c0b3ca0 0x7fd89c0b61b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4005d00 tx=0x7fd8a4005c50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:47.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:47 smithi073 ceph-mon[36714]: pgmap v390: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.392+0000 7fd8b17fa640 1 -- 172.21.15.67:0/3375300927 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd8a80cbe00 con 0x7fd8b4102950 2024-07-31T14:50:47.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.590+0000 7fd8ba568640 1 -- 172.21.15.67:0/3375300927 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8b419d1c0 con 0x7fd8b4102950 2024-07-31T14:50:47.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.591+0000 7fd8b17fa640 1 -- 172.21.15.67:0/3375300927 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24874 (secure 0 0 0) 0x7fd8a80c5e20 con 0x7fd8b4102950 2024-07-31T14:50:47.594 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:50:47.594 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T14:47:20.440064+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T14:50:47.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.593+0000 7fd8ba568640 1 -- 172.21.15.67:0/3375300927 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd89c0b3ca0 msgr2=0x7fd89c0b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:47.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.593+0000 7fd8ba568640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd89c0b3ca0 0x7fd89c0b61b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4005d00 tx=0x7fd8a4005c50 comp rx=0 tx=0).stop 2024-07-31T14:50:47.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.594+0000 7fd8ba568640 1 -- 172.21.15.67:0/3375300927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b4102950 msgr2=0x7fd8b419c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:47.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.594+0000 7fd8ba568640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b4102950 0x7fd8b419c630 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8005940 tx=0x7fd8a80042f0 comp rx=0 tx=0).stop 2024-07-31T14:50:47.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.594+0000 7fd8ba568640 1 -- 172.21.15.67:0/3375300927 shutdown_connections 2024-07-31T14:50:47.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.594+0000 7fd8ba568640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd89c0b3ca0 0x7fd89c0b61b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:47.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.594+0000 7fd8ba568640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8b410fbb0 0x7fd8b41a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.594+0000 7fd8ba568640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b4102950 0x7fd8b419c630 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.594+0000 7fd8ba568640 1 --2- 172.21.15.67:0/3375300927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8b4102000 0x7fd8b419c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.594+0000 7fd8ba568640 1 -- 172.21.15.67:0/3375300927 >> 172.21.15.67:0/3375300927 conn(0x7fd8b4079160 msgr2=0x7fd8b4111030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.594+0000 7fd8ba568640 1 -- 172.21.15.67:0/3375300927 shutdown_connections 2024-07-31T14:50:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:47.594+0000 7fd8ba568640 1 -- 172.21.15.67:0/3375300927 wait complete. 2024-07-31T14:50:48.259 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:48 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3375300927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:50:48.289 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-31T14:47:04.833142+0000', 'modified': '2024-07-31T14:48:06.435900+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24671, 'mds_1': 24655, 'mds_2': 24650, 'mds_3': 24646, 'mds_4': 24634}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14778': {'gid': 14778, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.67:6839/3532250223', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6838', 'nonce': 3532250223}, {'type': 'v1', 'addr': '172.21.15.67:6839', 'nonce': 3532250223}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14790': {'gid': 14790, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.67:6841/1797875282', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6840', 'nonce': 1797875282}, {'type': 'v1', 'addr': '172.21.15.67:6841', 'nonce': 1797875282}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24602': {'gid': 24602, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6833/1184537619', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6832', 'nonce': 1184537619}, {'type': 'v1', 'addr': '172.21.15.73:6833', 'nonce': 1184537619}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24604': {'gid': 24604, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6833/2710994774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6832', 'nonce': 2710994774}, {'type': 'v1', 'addr': '172.21.15.77:6833', 'nonce': 2710994774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24629': {'gid': 24629, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6835/2854264361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 2854264361}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 2854264361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24634': {'gid': 24634, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.77:6835/1866134372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 1866134372}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 1866134372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24646': {'gid': 24646, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.77:6837/3460575274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6836', 'nonce': 3460575274}, {'type': 'v1', 'addr': '172.21.15.77:6837', 'nonce': 3460575274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24650': {'gid': 24650, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.73:6837/3592232039', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 3592232039}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 3592232039}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24655': {'gid': 24655, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.77:6839/2802806156', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 2802806156}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 2802806156}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24671': {'gid': 24671, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.73:6839/4131046421', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 4131046421}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 4131046421}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24671, 'qdb_cluster': [24634, 24646, 24650, 24655, 24671]} 2024-07-31T14:50:48.289 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-07-31T14:50:48.290 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.f status 2024-07-31T14:50:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:48 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3375300927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:50:48.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:48 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3375300927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:50:48.557 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:49 smithi067 ceph-mon[32750]: pgmap v391: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:49.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:49 smithi073 ceph-mon[36714]: pgmap v391: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:49.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:49 smithi077 ceph-mon[37327]: pgmap v391: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.251+0000 7fc6a9785640 1 -- 172.21.15.67:0/3323955737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a4108720 msgr2=0x7fc6a410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.251+0000 7fc6a9785640 1 --2- 172.21.15.67:0/3323955737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a4108720 0x7fc6a410ef50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc694009ca0 tx=0x7fc69402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.251+0000 7fc6a9785640 1 -- 172.21.15.67:0/3323955737 shutdown_connections 2024-07-31T14:50:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.251+0000 7fc6a9785640 1 --2- 172.21.15.67:0/3323955737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a4108720 0x7fc6a410ef50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.251+0000 7fc6a9785640 1 --2- 172.21.15.67:0/3323955737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6a4107dd0 0x7fc6a41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.251+0000 7fc6a9785640 1 --2- 172.21.15.67:0/3323955737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a4102490 0x7fc6a41028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.251+0000 7fc6a9785640 1 -- 172.21.15.67:0/3323955737 >> 172.21.15.67:0/3323955737 conn(0x7fc6a4079160 msgr2=0x7fc6a4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.251+0000 7fc6a9785640 1 -- 172.21.15.67:0/3323955737 shutdown_connections 2024-07-31T14:50:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.251+0000 7fc6a9785640 1 -- 172.21.15.67:0/3323955737 wait complete. 2024-07-31T14:50:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.252+0000 7fc6a9785640 1 Processor -- start 2024-07-31T14:50:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.252+0000 7fc6a9785640 1 -- start start 2024-07-31T14:50:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a9785640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a4102490 0x7fc6a40ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a9785640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6a4107dd0 0x7fc6a40ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a9785640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a4108720 0x7fc6a40ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a9785640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a41123d0 con 0x7fc6a4108720 2024-07-31T14:50:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a9785640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a4112250 con 0x7fc6a4102490 2024-07-31T14:50:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a9785640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a4112550 con 0x7fc6a4107dd0 2024-07-31T14:50:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a4108720 0x7fc6a40ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a4108720 0x7fc6a40ffb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59280/0 (socket says 172.21.15.67:59280) 2024-07-31T14:50:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a37fe640 1 -- 172.21.15.67:0/2757013161 learned_addr learned my addr 172.21.15.67:0/2757013161 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a2ffd640 1 --2- 172.21.15.67:0/2757013161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a4102490 0x7fc6a40ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a27fc640 1 --2- 172.21.15.67:0/2757013161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6a4107dd0 0x7fc6a40ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a37fe640 1 -- 172.21.15.67:0/2757013161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6a4107dd0 msgr2=0x7fc6a40ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a37fe640 1 --2- 172.21.15.67:0/2757013161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6a4107dd0 0x7fc6a40ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a37fe640 1 -- 172.21.15.67:0/2757013161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a4102490 msgr2=0x7fc6a40ff0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a37fe640 1 --2- 172.21.15.67:0/2757013161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a4102490 0x7fc6a40ff0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.253+0000 7fc6a37fe640 1 -- 172.21.15.67:0/2757013161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a41a8000 con 0x7fc6a4108720 2024-07-31T14:50:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.254+0000 7fc6a37fe640 1 --2- 172.21.15.67:0/2757013161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a4108720 0x7fc6a40ffb20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc694009430 tx=0x7fc6940319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.254+0000 7fc69bfff640 1 -- 172.21.15.67:0/2757013161 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc694003b70 con 0x7fc6a4108720 2024-07-31T14:50:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.254+0000 7fc69bfff640 1 -- 172.21.15.67:0/2757013161 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc694003d10 con 0x7fc6a4108720 2024-07-31T14:50:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.254+0000 7fc6a9785640 1 -- 172.21.15.67:0/2757013161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a41a8290 con 0x7fc6a4108720 2024-07-31T14:50:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.254+0000 7fc69bfff640 1 -- 172.21.15.67:0/2757013161 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69402fa30 con 0x7fc6a4108720 2024-07-31T14:50:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.254+0000 7fc6a9785640 1 -- 172.21.15.67:0/2757013161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a41a87d0 con 0x7fc6a4108720 2024-07-31T14:50:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.257+0000 7fc6afcd6740 1 Processor -- start 2024-07-31T14:50:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.257+0000 7fc69bfff640 1 -- 172.21.15.67:0/2757013161 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fc69402fbd0 con 0x7fc6a4108720 2024-07-31T14:50:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.257+0000 7fc6afcd6740 1 -- start start 2024-07-31T14:50:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6afcd6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0778f60 0x560dd0b1e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6afcd6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd07500a0 0x560dd0b1e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6afcd6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd0758280 0x560dd0b1ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6afcd6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560dd0a94920 con 0x560dd07500a0 2024-07-31T14:50:50.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6afcd6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560dd0a83e20 con 0x560dd0778f60 2024-07-31T14:50:50.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6afcd6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560dd0a87e60 con 0x560dd0758280 2024-07-31T14:50:50.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6a2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd07500a0 0x560dd0b1e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6a2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd07500a0 0x560dd0b1e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59294/0 (socket says 172.21.15.67:59294) 2024-07-31T14:50:50.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6a27fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0778f60 0x560dd0b1e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6a2ffd640 1 -- 172.21.15.67:0/3030514459 learned_addr learned my addr 172.21.15.67:0/3030514459 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:50.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6a27fc640 1 --2- 172.21.15.67:0/3030514459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd0758280 0x560dd0b1ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6a2ffd640 1 -- 172.21.15.67:0/3030514459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd0758280 msgr2=0x560dd0b1ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:50.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6a2ffd640 1 --2- 172.21.15.67:0/3030514459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd0758280 0x560dd0b1ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6a2ffd640 1 -- 172.21.15.67:0/3030514459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0778f60 msgr2=0x560dd0b1e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6a2ffd640 1 --2- 172.21.15.67:0/3030514459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0778f60 0x560dd0b1e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6a2ffd640 1 -- 172.21.15.67:0/3030514459 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd0a2aa40 con 0x560dd07500a0 2024-07-31T14:50:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc69bfff640 1 --2- 172.21.15.67:0/2757013161 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6900b3c20 0x7fc6900b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6a27fc640 1 --2- 172.21.15.67:0/3030514459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0778f60 0x560dd0b1e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:50:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6a27fc640 1 --2- 172.21.15.67:0/2757013161 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6900b3c20 0x7fc6900b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc6a2ffd640 1 --2- 172.21.15.67:0/3030514459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd07500a0 0x560dd0b1e8c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00bcb0 tx=0x7fc69c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7b00d6f6fca30a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.258+0000 7fc69bfff640 1 -- 172.21.15.67:0/2757013161 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc694105520 con 0x7fc6a4108720 2024-07-31T14:50:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.259+0000 7fc6997fa640 1 -- 172.21.15.67:0/3030514459 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c01c480 con 0x560dd07500a0 2024-07-31T14:50:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.259+0000 7fc6997fa640 1 -- 172.21.15.67:0/3030514459 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc69c01cad0 con 0x560dd07500a0 2024-07-31T14:50:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.259+0000 7fc6997fa640 1 -- 172.21.15.67:0/3030514459 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c011c30 con 0x560dd07500a0 2024-07-31T14:50:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.260+0000 7fc6afcd6740 1 -- 172.21.15.67:0/3030514459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd07500a0 msgr2=0x560dd0b1e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.260+0000 7fc6a27fc640 1 --2- 172.21.15.67:0/2757013161 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6900b3c20 0x7fc6900b6130 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4069c70 tx=0x7fc68c00a770 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.260+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/3030514459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd07500a0 0x560dd0b1e8c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00bcb0 tx=0x7fc69c0107a0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.260+0000 7fc6afcd6740 1 -- 172.21.15.67:0/3030514459 shutdown_connections 2024-07-31T14:50:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.260+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/3030514459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd0758280 0x560dd0b1ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.260+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/3030514459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd07500a0 0x560dd0b1e8c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.260+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/3030514459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0778f60 0x560dd0b1e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.260+0000 7fc6afcd6740 1 -- 172.21.15.67:0/3030514459 >> 172.21.15.67:0/3030514459 conn(0x560dd076d350 msgr2=0x560dd0b1c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.260+0000 7fc6afcd6740 1 -- 172.21.15.67:0/3030514459 shutdown_connections 2024-07-31T14:50:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.260+0000 7fc6afcd6740 1 -- 172.21.15.67:0/3030514459 wait complete. 2024-07-31T14:50:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.260+0000 7fc6afcd6740 1 -- start start 2024-07-31T14:50:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.260+0000 7fc6afcd6740 1 Processor -- start 2024-07-31T14:50:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6afcd6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 0x560dd0b29f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6afcd6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0758280 0x560dd0b2a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6afcd6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0778f60 0x560dd0b35e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6afcd6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560dd08e75f0 con 0x560dd0778f60 2024-07-31T14:50:50.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6afcd6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560dd0a26890 con 0x560dd0758280 2024-07-31T14:50:50.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6afcd6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560dd0b2cb60 con 0x560dd07500a0 2024-07-31T14:50:50.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6afcd6740 10 client.0 _ll_register_callbacks cb 0x560dd0a307d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:50.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6a37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0778f60 0x560dd0b35e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6a37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0778f60 0x560dd0b35e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59300/0 (socket says 172.21.15.67:59300) 2024-07-31T14:50:50.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6a37fe640 1 -- 172.21.15.67:0/1451204780 learned_addr learned my addr 172.21.15.67:0/1451204780 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:50.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6a27fc640 1 --2- 172.21.15.67:0/1451204780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0758280 0x560dd0b2a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6a2ffd640 1 --2- 172.21.15.67:0/1451204780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 0x560dd0b29f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6a37fe640 1 -- 172.21.15.67:0/1451204780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 msgr2=0x560dd0b29f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:50.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6a37fe640 1 --2- 172.21.15.67:0/1451204780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 0x560dd0b29f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6a37fe640 1 -- 172.21.15.67:0/1451204780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0758280 msgr2=0x560dd0b2a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6a37fe640 1 --2- 172.21.15.67:0/1451204780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0758280 0x560dd0b2a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6a37fe640 1 -- 172.21.15.67:0/1451204780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd07e4540 con 0x560dd0778f60 2024-07-31T14:50:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.261+0000 7fc6a27fc640 1 --2- 172.21.15.67:0/1451204780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0758280 0x560dd0b2a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:50:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc6a37fe640 1 --2- 172.21.15.67:0/1451204780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0778f60 0x560dd0b35e50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc6940c80c0 tx=0x7fc6940cbb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155702c23190ab44 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc68a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc68a7fc640 1 -- 172.21.15.67:0/1451204780 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6940d0640 con 0x560dd0778f60 2024-07-31T14:50:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc68a7fc640 1 -- 172.21.15.67:0/1451204780 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc6941051b0 con 0x560dd0778f60 2024-07-31T14:50:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc6afcd6740 1 -- 172.21.15.67:0/1451204780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc69c01cad0 con 0x560dd0778f60 2024-07-31T14:50:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc68a7fc640 1 -- 172.21.15.67:0/1451204780 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6940d50f0 con 0x560dd0778f60 2024-07-31T14:50:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc6afcd6740 10 client.15396 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc6afcd6740 1 -- 172.21.15.67:0/1451204780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560dd0b38b40 con 0x560dd0778f60 2024-07-31T14:50:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc68a7fc640 1 -- 172.21.15.67:0/1451204780 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc69c01cad0 con 0x560dd0778f60 2024-07-31T14:50:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc68a7fc640 1 -- 172.21.15.67:0/1451204780 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc6940cb2a0 con 0x560dd0778f60 2024-07-31T14:50:50.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc6afcd6740 10 client.15396 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:50.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc6afcd6740 10 client.15396 resolve_mds: resolved name 'f' to daemon mds.f 2024-07-31T14:50:50.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc6afcd6740 20 client.15396 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:50.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/1451204780 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x560dd07378e0 0x560dd0b3c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc6afcd6740 4 client.15396 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:50:50.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.262+0000 7fc6afcd6740 1 -- 172.21.15.67:0/1451204780 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dd0a96870 con 0x560dd07378e0 2024-07-31T14:50:50.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.263+0000 7fc6a2ffd640 1 --2- 172.21.15.67:0/1451204780 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x560dd07378e0 0x560dd0b3c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.263+0000 7fc6a2ffd640 1 --2- 172.21.15.67:0/1451204780 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x560dd07378e0 0x560dd0b3c5c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=62278ef49d981d1a server_cookie=ba2726cf5830329e in_seq=0 out_seq=0 2024-07-31T14:50:50.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.263+0000 7fc68a7fc640 10 client.15396 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:50:50.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc68a7fc640 1 -- 172.21.15.67:0/1451204780 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560dd0a96870 con 0x560dd07378e0 2024-07-31T14:50:50.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc68a7fc640 10 client.15396 handle_command_reply: tid=0 2024-07-31T14:50:50.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc6afcd6740 1 client.15396 shutdown 2024-07-31T14:50:50.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc6afcd6740 1 -- 172.21.15.67:0/1451204780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0778f60 msgr2=0x560dd0b35e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/1451204780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0778f60 0x560dd0b35e50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc6940c80c0 tx=0x7fc6940cbb50 comp rx=0 tx=0).stop 2024-07-31T14:50:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc6afcd6740 1 -- 172.21.15.67:0/1451204780 shutdown_connections 2024-07-31T14:50:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/1451204780 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x560dd07378e0 0x560dd0b3c5c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/1451204780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0778f60 0x560dd0b35e50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/1451204780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0758280 0x560dd0b2a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/1451204780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 0x560dd0b29f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc68a7fc640 0 client.15396 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:50:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc6afcd6740 1 -- 172.21.15.67:0/1451204780 >> 172.21.15.67:0/1451204780 conn(0x560dd076d350 msgr2=0x560dd0b21560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc6afcd6740 1 -- 172.21.15.67:0/1451204780 shutdown_connections 2024-07-31T14:50:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc6afcd6740 1 -- 172.21.15.67:0/1451204780 wait complete. 2024-07-31T14:50:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.264+0000 7fc6afcd6740 20 client.15396 trim_cache size 0 max 16384 2024-07-31T14:50:50.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.288+0000 7fc6afcd6740 1 Processor -- start 2024-07-31T14:50:50.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6afcd6740 1 -- start start 2024-07-31T14:50:50.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6afcd6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 0x560dd0b35dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6afcd6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0758280 0x560dd0b1b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6afcd6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0778f60 0x560dd0b1bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6afcd6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560dd0b3a900 con 0x560dd0758280 2024-07-31T14:50:50.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6afcd6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560dd0b3afd0 con 0x560dd0778f60 2024-07-31T14:50:50.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6afcd6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560dd0b3b6a0 con 0x560dd07500a0 2024-07-31T14:50:50.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6a27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0758280 0x560dd0b1b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6a27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0758280 0x560dd0b1b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59310/0 (socket says 172.21.15.67:59310) 2024-07-31T14:50:50.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6a27fc640 1 -- 172.21.15.67:0/1684481460 learned_addr learned my addr 172.21.15.67:0/1684481460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:50.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6a2ffd640 1 --2- 172.21.15.67:0/1684481460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 0x560dd0b35dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6a37fe640 1 --2- 172.21.15.67:0/1684481460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0778f60 0x560dd0b1bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6a27fc640 1 -- 172.21.15.67:0/1684481460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 msgr2=0x560dd0b35dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:50.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6a27fc640 1 --2- 172.21.15.67:0/1684481460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 0x560dd0b35dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.289+0000 7fc6a27fc640 1 -- 172.21.15.67:0/1684481460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0778f60 msgr2=0x560dd0b1bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:50.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.290+0000 7fc6a27fc640 1 --2- 172.21.15.67:0/1684481460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0778f60 0x560dd0b1bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.290+0000 7fc6a27fc640 1 -- 172.21.15.67:0/1684481460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd0b2b020 con 0x560dd0758280 2024-07-31T14:50:50.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.290+0000 7fc6a2ffd640 1 --2- 172.21.15.67:0/1684481460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 0x560dd0b35dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:50:50.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.290+0000 7fc6a37fe640 1 --2- 172.21.15.67:0/1684481460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0778f60 0x560dd0b1bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:50:50.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.290+0000 7fc6a27fc640 1 --2- 172.21.15.67:0/1684481460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0758280 0x560dd0b1b840 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0105f0 tx=0x7fc68c012070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1104fbd4f84f1dae server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:50.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.290+0000 7fc698ff9640 1 -- 172.21.15.67:0/1684481460 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c02a640 con 0x560dd0758280 2024-07-31T14:50:50.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.290+0000 7fc698ff9640 1 -- 172.21.15.67:0/1684481460 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc68c0208c0 con 0x560dd0758280 2024-07-31T14:50:50.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.290+0000 7fc698ff9640 1 -- 172.21.15.67:0/1684481460 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0297f0 con 0x560dd0758280 2024-07-31T14:50:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.291+0000 7fc6afcd6740 1 -- 172.21.15.67:0/1684481460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0758280 msgr2=0x560dd0b1b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.291+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/1684481460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0758280 0x560dd0b1b840 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0105f0 tx=0x7fc68c012070 comp rx=0 tx=0).stop 2024-07-31T14:50:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.291+0000 7fc6afcd6740 1 -- 172.21.15.67:0/1684481460 shutdown_connections 2024-07-31T14:50:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.291+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/1684481460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0778f60 0x560dd0b1bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.291+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/1684481460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0758280 0x560dd0b1b840 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.291+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/1684481460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 0x560dd0b35dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.291+0000 7fc6afcd6740 1 -- 172.21.15.67:0/1684481460 >> 172.21.15.67:0/1684481460 conn(0x560dd076d350 msgr2=0x560dd0b285e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.291+0000 7fc6afcd6740 1 -- 172.21.15.67:0/1684481460 shutdown_connections 2024-07-31T14:50:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.291+0000 7fc6afcd6740 1 -- 172.21.15.67:0/1684481460 wait complete. 2024-07-31T14:50:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.291+0000 7fc6afcd6740 1 -- start start 2024-07-31T14:50:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6afcd6740 1 Processor -- start 2024-07-31T14:50:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6afcd6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 0x560dd0b44a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6afcd6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0758280 0x560dd0b45020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6afcd6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0778f60 0x560dd0b3dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6afcd6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560dd0b3c440 con 0x560dd0778f60 2024-07-31T14:50:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6afcd6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560dd0b3bd70 con 0x560dd0758280 2024-07-31T14:50:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6afcd6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560dd0b3cb10 con 0x560dd07500a0 2024-07-31T14:50:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6a37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0778f60 0x560dd0b3dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6a37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0778f60 0x560dd0b3dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59326/0 (socket says 172.21.15.67:59326) 2024-07-31T14:50:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6a37fe640 1 -- 172.21.15.67:0/3777008749 learned_addr learned my addr 172.21.15.67:0/3777008749 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6afcd6740 10 client.0 _ll_register_callbacks cb 0x7fc6940cb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6a2ffd640 1 --2- 172.21.15.67:0/3777008749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 0x560dd0b44a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6a27fc640 1 --2- 172.21.15.67:0/3777008749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0758280 0x560dd0b45020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6a37fe640 1 -- 172.21.15.67:0/3777008749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 msgr2=0x560dd0b44a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6a37fe640 1 --2- 172.21.15.67:0/3777008749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 0x560dd0b44a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6a37fe640 1 -- 172.21.15.67:0/3777008749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0758280 msgr2=0x560dd0b45020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6a37fe640 1 --2- 172.21.15.67:0/3777008749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0758280 0x560dd0b45020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.292+0000 7fc6a37fe640 1 -- 172.21.15.67:0/3777008749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd0b3e760 con 0x560dd0778f60 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.293+0000 7fc6a37fe640 1 --2- 172.21.15.67:0/3777008749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0778f60 0x560dd0b3dfd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc6940cbd40 tx=0x7fc6940d0510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed926379153b71cd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.293+0000 7fc68a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.293+0000 7fc68a7fc640 1 -- 172.21.15.67:0/3777008749 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6940eb060 con 0x560dd0778f60 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.293+0000 7fc68a7fc640 1 -- 172.21.15.67:0/3777008749 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc6941051b0 con 0x560dd0778f60 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.293+0000 7fc6afcd6740 1 -- 172.21.15.67:0/3777008749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc68c0208c0 con 0x560dd0778f60 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.293+0000 7fc68a7fc640 1 -- 172.21.15.67:0/3777008749 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6940d0640 con 0x560dd0778f60 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.293+0000 7fc6afcd6740 10 client.15402 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.293+0000 7fc6afcd6740 1 -- 172.21.15.67:0/3777008749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560dd0b43fa0 con 0x560dd0778f60 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.293+0000 7fc68a7fc640 1 -- 172.21.15.67:0/3777008749 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc68c0208c0 con 0x560dd0778f60 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.294+0000 7fc68a7fc640 1 -- 172.21.15.67:0/3777008749 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc6940cc330 con 0x560dd0778f60 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.294+0000 7fc6afcd6740 10 client.15402 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.294+0000 7fc6afcd6740 10 client.15402 resolve_mds: resolved name 'f' to daemon mds.f 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.294+0000 7fc6afcd6740 20 client.15402 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.294+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/3777008749 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x560dd070cc90 0x560dd0ba8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.294+0000 7fc6afcd6740 4 client.15402 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:50:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.294+0000 7fc6afcd6740 1 -- 172.21.15.67:0/3777008749 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "status"}) -- 0x560dd0a92310 con 0x560dd070cc90 2024-07-31T14:50:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.294+0000 7fc6a2ffd640 1 --2- 172.21.15.67:0/3777008749 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x560dd070cc90 0x560dd0ba8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.294+0000 7fc68a7fc640 10 client.15402 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:50:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.294+0000 7fc6a2ffd640 1 --2- 172.21.15.67:0/3777008749 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x560dd070cc90 0x560dd0ba8830 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=51edefdaf085c0f1 server_cookie=b71effbb4bd5a7ba in_seq=0 out_seq=0 2024-07-31T14:50:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc68a7fc640 1 -- 172.21.15.67:0/3777008749 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560dd0a92310 con 0x560dd070cc90 2024-07-31T14:50:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc68a7fc640 10 client.15402 handle_command_reply: tid=0 2024-07-31T14:50:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc6afcd6740 1 client.15402 shutdown 2024-07-31T14:50:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc6afcd6740 1 -- 172.21.15.67:0/3777008749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0778f60 msgr2=0x560dd0b3dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/3777008749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0778f60 0x560dd0b3dfd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc6940cbd40 tx=0x7fc6940d0510 comp rx=0 tx=0).stop 2024-07-31T14:50:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc6afcd6740 1 -- 172.21.15.67:0/3777008749 shutdown_connections 2024-07-31T14:50:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/3777008749 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x560dd070cc90 0x560dd0ba8830 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/3777008749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560dd0778f60 0x560dd0b3dfd0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/3777008749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd0758280 0x560dd0b45020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc6afcd6740 1 --2- 172.21.15.67:0/3777008749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd07500a0 0x560dd0b44a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc6afcd6740 1 -- 172.21.15.67:0/3777008749 >> 172.21.15.67:0/3777008749 conn(0x560dd076d350 msgr2=0x560dd0b1f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:50.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc68a7fc640 0 client.15402 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:50:50.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc6afcd6740 1 -- 172.21.15.67:0/3777008749 shutdown_connections 2024-07-31T14:50:50.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc6afcd6740 1 -- 172.21.15.67:0/3777008749 wait complete. 2024-07-31T14:50:50.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.295+0000 7fc6afcd6740 20 client.15402 trim_cache size 0 max 16384 2024-07-31T14:50:50.301 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:50:50.301 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:50:50.301 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 4, 2024-07-31T14:50:50.302 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24634, 2024-07-31T14:50:50.302 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:50:50.302 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:50:50.302 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:50:50.302 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 167.70648659099999, 2024-07-31T14:50:50.302 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-31T14:50:50.302 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:50:50.302 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:50:50.302 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 256.63389897500002 2024-07-31T14:50:50.302 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:50:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.296+0000 7fc6a9785640 1 -- 172.21.15.67:0/2757013161 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6900b3c20 msgr2=0x7fc6900b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.296+0000 7fc6a9785640 1 --2- 172.21.15.67:0/2757013161 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6900b3c20 0x7fc6900b6130 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4069c70 tx=0x7fc68c00a770 comp rx=0 tx=0).stop 2024-07-31T14:50:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.296+0000 7fc6a9785640 1 -- 172.21.15.67:0/2757013161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a4108720 msgr2=0x7fc6a40ffb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.296+0000 7fc6a9785640 1 --2- 172.21.15.67:0/2757013161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a4108720 0x7fc6a40ffb20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc694009430 tx=0x7fc6940319f0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.296+0000 7fc6a9785640 1 -- 172.21.15.67:0/2757013161 shutdown_connections 2024-07-31T14:50:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.296+0000 7fc6a9785640 1 --2- 172.21.15.67:0/2757013161 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6900b3c20 0x7fc6900b6130 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.296+0000 7fc6a9785640 1 --2- 172.21.15.67:0/2757013161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a4108720 0x7fc6a40ffb20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.296+0000 7fc6a9785640 1 --2- 172.21.15.67:0/2757013161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6a4107dd0 0x7fc6a40ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.296+0000 7fc6a9785640 1 --2- 172.21.15.67:0/2757013161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a4102490 0x7fc6a40ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:50.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.296+0000 7fc6a9785640 1 -- 172.21.15.67:0/2757013161 >> 172.21.15.67:0/2757013161 conn(0x7fc6a4079160 msgr2=0x7fc6a406ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:50.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.296+0000 7fc6a9785640 1 -- 172.21.15.67:0/2757013161 shutdown_connections 2024-07-31T14:50:50.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:50.296+0000 7fc6a9785640 1 -- 172.21.15.67:0/2757013161 wait complete. 2024-07-31T14:50:50.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.c status 2024-07-31T14:50:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:51.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:51 smithi067 ceph-mon[32750]: pgmap v392: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:51.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:51 smithi073 ceph-mon[36714]: pgmap v392: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:51.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:51 smithi077 ceph-mon[37327]: pgmap v392: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:53.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.035+0000 7fb29774a640 1 -- 172.21.15.67:0/1305053032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290107dd0 msgr2=0x7fb2901081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.035+0000 7fb29774a640 1 --2- 172.21.15.67:0/1305053032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290107dd0 0x7fb2901081e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb280009900 tx=0x7fb28002a3f0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.035+0000 7fb29774a640 1 -- 172.21.15.67:0/1305053032 shutdown_connections 2024-07-31T14:50:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.035+0000 7fb29774a640 1 --2- 172.21.15.67:0/1305053032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb290108720 0x7fb29010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.035+0000 7fb29774a640 1 --2- 172.21.15.67:0/1305053032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290107dd0 0x7fb2901081e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.035+0000 7fb29774a640 1 --2- 172.21.15.67:0/1305053032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb290102490 0x7fb2901028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.035+0000 7fb29774a640 1 -- 172.21.15.67:0/1305053032 >> 172.21.15.67:0/1305053032 conn(0x7fb290079160 msgr2=0x7fb290079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.035+0000 7fb29774a640 1 -- 172.21.15.67:0/1305053032 shutdown_connections 2024-07-31T14:50:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.035+0000 7fb29774a640 1 -- 172.21.15.67:0/1305053032 wait complete. 2024-07-31T14:50:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.035+0000 7fb29774a640 1 Processor -- start 2024-07-31T14:50:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb29774a640 1 -- start start 2024-07-31T14:50:53.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb29774a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb290102490 0x7fb290199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb29774a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290107dd0 0x7fb29019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb29774a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb290108720 0x7fb29019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb29774a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb290112350 con 0x7fb290107dd0 2024-07-31T14:50:53.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb29774a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901121d0 con 0x7fb290108720 2024-07-31T14:50:53.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb29774a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901124d0 con 0x7fb290102490 2024-07-31T14:50:53.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb294ce5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290107dd0 0x7fb29019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb294ce5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290107dd0 0x7fb29019a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46136/0 (socket says 172.21.15.67:46136) 2024-07-31T14:50:53.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb294ce5640 1 -- 172.21.15.67:0/986015065 learned_addr learned my addr 172.21.15.67:0/986015065 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:53.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb2954e6640 1 --2- 172.21.15.67:0/986015065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb290102490 0x7fb290199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb295ce7640 1 --2- 172.21.15.67:0/986015065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb290108720 0x7fb29019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb294ce5640 1 -- 172.21.15.67:0/986015065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb290102490 msgr2=0x7fb290199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:53.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb294ce5640 1 --2- 172.21.15.67:0/986015065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb290102490 0x7fb290199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb294ce5640 1 -- 172.21.15.67:0/986015065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb290108720 msgr2=0x7fb29019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:53.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb294ce5640 1 --2- 172.21.15.67:0/986015065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb290108720 0x7fb29019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.036+0000 7fb294ce5640 1 -- 172.21.15.67:0/986015065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29019fd20 con 0x7fb290107dd0 2024-07-31T14:50:53.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.037+0000 7fb294ce5640 1 --2- 172.21.15.67:0/986015065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290107dd0 0x7fb29019a410 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb280031a90 tx=0x7fb280004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:53.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.037+0000 7fb2867fc640 1 -- 172.21.15.67:0/986015065 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28003f030 con 0x7fb290107dd0 2024-07-31T14:50:53.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.037+0000 7fb29774a640 1 -- 172.21.15.67:0/986015065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29019ffb0 con 0x7fb290107dd0 2024-07-31T14:50:53.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.037+0000 7fb2867fc640 1 -- 172.21.15.67:0/986015065 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb280004490 con 0x7fb290107dd0 2024-07-31T14:50:53.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.037+0000 7fb2867fc640 1 -- 172.21.15.67:0/986015065 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280035070 con 0x7fb290107dd0 2024-07-31T14:50:53.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.037+0000 7fb29774a640 1 -- 172.21.15.67:0/986015065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2901a04e0 con 0x7fb290107dd0 2024-07-31T14:50:53.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.040+0000 7fb29dc9b740 1 Processor -- start 2024-07-31T14:50:53.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.040+0000 7fb29dc9b740 1 -- start start 2024-07-31T14:50:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb29dc9b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534676f60 0x560534a1c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb29dc9b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56053464e0a0 0x560534a1c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb29dc9b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560534656280 0x560534a1ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb29dc9b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560534992920 con 0x560534656280 2024-07-31T14:50:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb29dc9b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560534981e20 con 0x56053464e0a0 2024-07-31T14:50:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb29dc9b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560534985e60 con 0x560534676f60 2024-07-31T14:50:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb2867fc640 1 -- 172.21.15.67:0/986015065 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fb280042030 con 0x7fb290107dd0 2024-07-31T14:50:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb294ce5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560534656280 0x560534a1ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb2954e6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56053464e0a0 0x560534a1c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb294ce5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560534656280 0x560534a1ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46152/0 (socket says 172.21.15.67:46152) 2024-07-31T14:50:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb294ce5640 1 -- 172.21.15.67:0/2479949099 learned_addr learned my addr 172.21.15.67:0/2479949099 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb295ce7640 1 --2- 172.21.15.67:0/2479949099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534676f60 0x560534a1c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb294ce5640 1 -- 172.21.15.67:0/2479949099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534676f60 msgr2=0x560534a1c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb294ce5640 1 --2- 172.21.15.67:0/2479949099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534676f60 0x560534a1c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb294ce5640 1 -- 172.21.15.67:0/2479949099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56053464e0a0 msgr2=0x560534a1c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb294ce5640 1 --2- 172.21.15.67:0/2479949099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56053464e0a0 0x560534a1c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.041+0000 7fb294ce5640 1 -- 172.21.15.67:0/2479949099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560534928a40 con 0x560534656280 2024-07-31T14:50:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.042+0000 7fb294ce5640 1 --2- 172.21.15.67:0/2479949099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560534656280 0x560534a1ce00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb28010c430 tx=0x7fb28010c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a036726a2c6ddbd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.042+0000 7fb2867fc640 1 --2- 172.21.15.67:0/986015065 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb27c0b3cd0 0x7fb27c0b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.042+0000 7fb27b7fe640 1 -- 172.21.15.67:0/2479949099 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2800c73b0 con 0x560534656280 2024-07-31T14:50:53.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.042+0000 7fb295ce7640 1 --2- 172.21.15.67:0/986015065 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb27c0b3cd0 0x7fb27c0b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.042+0000 7fb27b7fe640 1 -- 172.21.15.67:0/2479949099 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb280111030 con 0x560534656280 2024-07-31T14:50:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.042+0000 7fb27b7fe640 1 -- 172.21.15.67:0/2479949099 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2800ca4d0 con 0x560534656280 2024-07-31T14:50:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.042+0000 7fb2867fc640 1 -- 172.21.15.67:0/986015065 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fb280104240 con 0x7fb290107dd0 2024-07-31T14:50:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.043+0000 7fb29dc9b740 1 -- 172.21.15.67:0/2479949099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560534656280 msgr2=0x560534a1ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.043+0000 7fb295ce7640 1 --2- 172.21.15.67:0/986015065 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb27c0b3cd0 0x7fb27c0b61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb288007e50 tx=0x7fb288007600 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.043+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/2479949099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560534656280 0x560534a1ce00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb28010c430 tx=0x7fb28010c520 comp rx=0 tx=0).stop 2024-07-31T14:50:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.043+0000 7fb29dc9b740 1 -- 172.21.15.67:0/2479949099 shutdown_connections 2024-07-31T14:50:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.043+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/2479949099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560534656280 0x560534a1ce00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.043+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/2479949099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56053464e0a0 0x560534a1c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.043+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/2479949099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534676f60 0x560534a1c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.043+0000 7fb29dc9b740 1 -- 172.21.15.67:0/2479949099 >> 172.21.15.67:0/2479949099 conn(0x56053466b350 msgr2=0x560534a1a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.043+0000 7fb29dc9b740 1 -- 172.21.15.67:0/2479949099 shutdown_connections 2024-07-31T14:50:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.043+0000 7fb29dc9b740 1 -- 172.21.15.67:0/2479949099 wait complete. 2024-07-31T14:50:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.044+0000 7fb29dc9b740 1 -- start start 2024-07-31T14:50:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.044+0000 7fb29dc9b740 1 Processor -- start 2024-07-31T14:50:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.044+0000 7fb29dc9b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56053464e0a0 0x560534a27e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.044+0000 7fb29dc9b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560534656280 0x560534a283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.044+0000 7fb29dc9b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 0x560534a33b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.044+0000 7fb29dc9b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5605347e55f0 con 0x560534656280 2024-07-31T14:50:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.044+0000 7fb29dc9b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560534924890 con 0x560534676f60 2024-07-31T14:50:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.044+0000 7fb29dc9b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560534a2a920 con 0x56053464e0a0 2024-07-31T14:50:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.044+0000 7fb29dc9b740 10 client.0 _ll_register_callbacks cb 0x56053492e7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.044+0000 7fb294ce5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560534656280 0x560534a283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.044+0000 7fb294ce5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560534656280 0x560534a283a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46166/0 (socket says 172.21.15.67:46166) 2024-07-31T14:50:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.044+0000 7fb294ce5640 1 -- 172.21.15.67:0/952506548 learned_addr learned my addr 172.21.15.67:0/952506548 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb2954e6640 1 --2- 172.21.15.67:0/952506548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56053464e0a0 0x560534a27e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb295ce7640 1 --2- 172.21.15.67:0/952506548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 0x560534a33b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb294ce5640 1 -- 172.21.15.67:0/952506548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56053464e0a0 msgr2=0x560534a27e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb294ce5640 1 --2- 172.21.15.67:0/952506548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56053464e0a0 0x560534a27e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb294ce5640 1 -- 172.21.15.67:0/952506548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 msgr2=0x560534a33b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb294ce5640 1 --2- 172.21.15.67:0/952506548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 0x560534a33b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb294ce5640 1 -- 172.21.15.67:0/952506548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605346e2540 con 0x560534656280 2024-07-31T14:50:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb294ce5640 1 --2- 172.21.15.67:0/952506548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560534656280 0x560534a283a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb280006010 tx=0x7fb280000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904413d509d61a18 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb278ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb278ff9640 1 -- 172.21.15.67:0/952506548 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280049040 con 0x560534656280 2024-07-31T14:50:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb278ff9640 1 -- 172.21.15.67:0/952506548 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fb2800c7350 con 0x560534656280 2024-07-31T14:50:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb29dc9b740 1 -- 172.21.15.67:0/952506548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb280111030 con 0x560534656280 2024-07-31T14:50:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb278ff9640 1 -- 172.21.15.67:0/952506548 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2800ca6b0 con 0x560534656280 2024-07-31T14:50:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb29dc9b740 10 client.15414 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.045+0000 7fb29dc9b740 1 -- 172.21.15.67:0/952506548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560534a367d0 con 0x560534656280 2024-07-31T14:50:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.046+0000 7fb278ff9640 1 -- 172.21.15.67:0/952506548 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb280111030 con 0x560534656280 2024-07-31T14:50:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.046+0000 7fb278ff9640 1 -- 172.21.15.67:0/952506548 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb28004c060 con 0x560534656280 2024-07-31T14:50:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.046+0000 7fb29dc9b740 10 client.15414 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.046+0000 7fb29dc9b740 10 client.15414 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-31T14:50:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.046+0000 7fb29dc9b740 20 client.15414 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.046+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/952506548 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5605346358e0 0x560534a39fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.046+0000 7fb29dc9b740 4 client.15414 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:50:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.046+0000 7fb29dc9b740 1 -- 172.21.15.67:0/952506548 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560534994870 con 0x5605346358e0 2024-07-31T14:50:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.046+0000 7fb2954e6640 1 --2- 172.21.15.67:0/952506548 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5605346358e0 0x560534a39fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.047+0000 7fb278ff9640 10 client.15414 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:50:53.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.047+0000 7fb2954e6640 1 --2- 172.21.15.67:0/952506548 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5605346358e0 0x560534a39fc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=77207ad30e3f3c6e server_cookie=3a382761906305b3 in_seq=0 out_seq=0 2024-07-31T14:50:53.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.047+0000 7fb278ff9640 1 -- 172.21.15.67:0/952506548 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560534994870 con 0x5605346358e0 2024-07-31T14:50:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.047+0000 7fb278ff9640 10 client.15414 handle_command_reply: tid=0 2024-07-31T14:50:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.047+0000 7fb29dc9b740 1 client.15414 shutdown 2024-07-31T14:50:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.047+0000 7fb29dc9b740 1 -- 172.21.15.67:0/952506548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560534656280 msgr2=0x560534a283a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.047+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/952506548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560534656280 0x560534a283a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb280006010 tx=0x7fb280000bc0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.047+0000 7fb29dc9b740 1 -- 172.21.15.67:0/952506548 shutdown_connections 2024-07-31T14:50:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.048+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/952506548 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5605346358e0 0x560534a39fc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.048+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/952506548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 0x560534a33b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.048+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/952506548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560534656280 0x560534a283a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.048+0000 7fb278ff9640 0 client.15414 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:50:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.048+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/952506548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56053464e0a0 0x560534a27e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.048+0000 7fb29dc9b740 1 -- 172.21.15.67:0/952506548 >> 172.21.15.67:0/952506548 conn(0x56053466b350 msgr2=0x560534a1f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.048+0000 7fb29dc9b740 1 -- 172.21.15.67:0/952506548 shutdown_connections 2024-07-31T14:50:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.048+0000 7fb29dc9b740 1 -- 172.21.15.67:0/952506548 wait complete. 2024-07-31T14:50:53.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.048+0000 7fb29dc9b740 20 client.15414 trim_cache size 0 max 16384 2024-07-31T14:50:53.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.067+0000 7fb29dc9b740 1 Processor -- start 2024-07-31T14:50:53.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.067+0000 7fb29dc9b740 1 -- start start 2024-07-31T14:50:53.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.067+0000 7fb29dc9b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56053464e0a0 0x560534a22200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.067+0000 7fb29dc9b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534656280 0x560534a339d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.067+0000 7fb29dc9b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 0x560534a33f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.067+0000 7fb29dc9b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560534a38300 con 0x56053464e0a0 2024-07-31T14:50:53.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.067+0000 7fb29dc9b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560534a389d0 con 0x560534676f60 2024-07-31T14:50:53.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.067+0000 7fb29dc9b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560534a390a0 con 0x560534656280 2024-07-31T14:50:53.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.067+0000 7fb2954e6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56053464e0a0 0x560534a22200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.067+0000 7fb2954e6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56053464e0a0 0x560534a22200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46168/0 (socket says 172.21.15.67:46168) 2024-07-31T14:50:53.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.067+0000 7fb2954e6640 1 -- 172.21.15.67:0/1156131458 learned_addr learned my addr 172.21.15.67:0/1156131458 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:53.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.068+0000 7fb294ce5640 1 --2- 172.21.15.67:0/1156131458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534656280 0x560534a339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.068+0000 7fb295ce7640 1 --2- 172.21.15.67:0/1156131458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 0x560534a33f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.068+0000 7fb2954e6640 1 -- 172.21.15.67:0/1156131458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534656280 msgr2=0x560534a339d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:53.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.068+0000 7fb2954e6640 1 --2- 172.21.15.67:0/1156131458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534656280 0x560534a339d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.068+0000 7fb2954e6640 1 -- 172.21.15.67:0/1156131458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 msgr2=0x560534a33f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:53.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.068+0000 7fb2954e6640 1 --2- 172.21.15.67:0/1156131458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 0x560534a33f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.068+0000 7fb2954e6640 1 -- 172.21.15.67:0/1156131458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560534a28f20 con 0x56053464e0a0 2024-07-31T14:50:53.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.068+0000 7fb2954e6640 1 --2- 172.21.15.67:0/1156131458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56053464e0a0 0x560534a22200 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x560534a3a910 tx=0x7fb28c00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1041c7ac5f7f0d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:53.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.068+0000 7fb27affd640 1 -- 172.21.15.67:0/1156131458 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c027650 con 0x56053464e0a0 2024-07-31T14:50:53.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.068+0000 7fb27affd640 1 -- 172.21.15.67:0/1156131458 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb28c027ca0 con 0x56053464e0a0 2024-07-31T14:50:53.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.068+0000 7fb27affd640 1 -- 172.21.15.67:0/1156131458 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c01dde0 con 0x56053464e0a0 2024-07-31T14:50:53.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.069+0000 7fb29dc9b740 1 -- 172.21.15.67:0/1156131458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56053464e0a0 msgr2=0x560534a22200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:53.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.069+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/1156131458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56053464e0a0 0x560534a22200 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x560534a3a910 tx=0x7fb28c00a2d0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.069+0000 7fb29dc9b740 1 -- 172.21.15.67:0/1156131458 shutdown_connections 2024-07-31T14:50:53.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.069+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/1156131458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 0x560534a33f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.069+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/1156131458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534656280 0x560534a339d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.069+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/1156131458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56053464e0a0 0x560534a22200 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.069+0000 7fb29dc9b740 1 -- 172.21.15.67:0/1156131458 >> 172.21.15.67:0/1156131458 conn(0x56053466b350 msgr2=0x560534a264e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:53.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.069+0000 7fb29dc9b740 1 -- 172.21.15.67:0/1156131458 shutdown_connections 2024-07-31T14:50:53.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.069+0000 7fb29dc9b740 1 -- 172.21.15.67:0/1156131458 wait complete. 2024-07-31T14:50:53.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.069+0000 7fb29dc9b740 1 -- start start 2024-07-31T14:50:53.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.070+0000 7fb29dc9b740 1 Processor -- start 2024-07-31T14:50:53.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.070+0000 7fb29dc9b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56053464e0a0 0x560534a40100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.070+0000 7fb29dc9b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534656280 0x560534a40680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.070+0000 7fb29dc9b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 0x560534a3b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.070+0000 7fb29dc9b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560534a39e40 con 0x56053464e0a0 2024-07-31T14:50:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.070+0000 7fb29dc9b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560534a39770 con 0x560534676f60 2024-07-31T14:50:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.070+0000 7fb29dc9b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560534a3a510 con 0x560534656280 2024-07-31T14:50:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.070+0000 7fb29dc9b740 10 client.0 _ll_register_callbacks cb 0x7fb28010c300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.070+0000 7fb2954e6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56053464e0a0 0x560534a40100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.070+0000 7fb2954e6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56053464e0a0 0x560534a40100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46170/0 (socket says 172.21.15.67:46170) 2024-07-31T14:50:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.070+0000 7fb2954e6640 1 -- 172.21.15.67:0/503783962 learned_addr learned my addr 172.21.15.67:0/503783962 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.070+0000 7fb294ce5640 1 --2- 172.21.15.67:0/503783962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534656280 0x560534a40680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.070+0000 7fb295ce7640 1 --2- 172.21.15.67:0/503783962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 0x560534a3b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.071+0000 7fb2954e6640 1 -- 172.21.15.67:0/503783962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534656280 msgr2=0x560534a40680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.071+0000 7fb2954e6640 1 --2- 172.21.15.67:0/503783962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534656280 0x560534a40680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.071+0000 7fb2954e6640 1 -- 172.21.15.67:0/503783962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 msgr2=0x560534a3b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.071+0000 7fb2954e6640 1 --2- 172.21.15.67:0/503783962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 0x560534a3b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.071+0000 7fb2954e6640 1 -- 172.21.15.67:0/503783962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560534a3be40 con 0x56053464e0a0 2024-07-31T14:50:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.071+0000 7fb2954e6640 1 --2- 172.21.15.67:0/503783962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56053464e0a0 0x560534a40100 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00a710 tx=0x7fb28c01eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dd31d1e471a15d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.071+0000 7fb278ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.071+0000 7fb278ff9640 1 -- 172.21.15.67:0/503783962 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c038060 con 0x56053464e0a0 2024-07-31T14:50:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.071+0000 7fb278ff9640 1 -- 172.21.15.67:0/503783962 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fb28c01ee40 con 0x56053464e0a0 2024-07-31T14:50:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.071+0000 7fb29dc9b740 1 -- 172.21.15.67:0/503783962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb28c027ca0 con 0x56053464e0a0 2024-07-31T14:50:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.071+0000 7fb278ff9640 1 -- 172.21.15.67:0/503783962 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb28c03f050 con 0x56053464e0a0 2024-07-31T14:50:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.071+0000 7fb29dc9b740 10 client.15420 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.071+0000 7fb29dc9b740 1 -- 172.21.15.67:0/503783962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560534a41830 con 0x56053464e0a0 2024-07-31T14:50:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.071+0000 7fb278ff9640 1 -- 172.21.15.67:0/503783962 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb28c027ca0 con 0x56053464e0a0 2024-07-31T14:50:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.072+0000 7fb278ff9640 1 -- 172.21.15.67:0/503783962 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb28c027940 con 0x56053464e0a0 2024-07-31T14:50:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.072+0000 7fb29dc9b740 10 client.15420 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.072+0000 7fb29dc9b740 10 client.15420 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-31T14:50:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.072+0000 7fb29dc9b740 20 client.15420 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.072+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/503783962 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x56053460ac90 0x560534aa60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.072+0000 7fb29dc9b740 4 client.15420 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:50:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.072+0000 7fb29dc9b740 1 -- 172.21.15.67:0/503783962 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "status"}) -- 0x560534990310 con 0x56053460ac90 2024-07-31T14:50:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.072+0000 7fb294ce5640 1 --2- 172.21.15.67:0/503783962 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x56053460ac90 0x560534aa60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb294ce5640 1 --2- 172.21.15.67:0/503783962 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x56053460ac90 0x560534aa60a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32bf7c40a7efa37f server_cookie=9776b8ea88f3da31 in_seq=0 out_seq=0 2024-07-31T14:50:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb278ff9640 10 client.15420 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:50:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb278ff9640 1 -- 172.21.15.67:0/503783962 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560534990310 con 0x56053460ac90 2024-07-31T14:50:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb278ff9640 10 client.15420 handle_command_reply: tid=0 2024-07-31T14:50:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb29dc9b740 1 client.15420 shutdown 2024-07-31T14:50:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb29dc9b740 1 -- 172.21.15.67:0/503783962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56053464e0a0 msgr2=0x560534a40100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/503783962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56053464e0a0 0x560534a40100 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00a710 tx=0x7fb28c01eca0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb29dc9b740 1 -- 172.21.15.67:0/503783962 shutdown_connections 2024-07-31T14:50:53.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/503783962 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x56053460ac90 0x560534aa60a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/503783962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560534676f60 0x560534a3b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/503783962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560534656280 0x560534a40680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb29dc9b740 1 --2- 172.21.15.67:0/503783962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56053464e0a0 0x560534a40100 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb278ff9640 0 client.15420 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:50:53.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb29dc9b740 1 -- 172.21.15.67:0/503783962 >> 172.21.15.67:0/503783962 conn(0x56053466b350 msgr2=0x560534a18830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:53.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb29dc9b740 1 -- 172.21.15.67:0/503783962 shutdown_connections 2024-07-31T14:50:53.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb29dc9b740 1 -- 172.21.15.67:0/503783962 wait complete. 2024-07-31T14:50:53.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.073+0000 7fb29dc9b740 20 client.15420 trim_cache size 0 max 16384 2024-07-31T14:50:53.079 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:50:53.080 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:50:53.080 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 3, 2024-07-31T14:50:53.080 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24646, 2024-07-31T14:50:53.080 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:50:53.080 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:50:53.080 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:50:53.080 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 171.494341964, 2024-07-31T14:50:53.080 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-31T14:50:53.080 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:50:53.080 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:50:53.080 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 256.82536935899998 2024-07-31T14:50:53.080 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:50:53.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.074+0000 7fb29774a640 1 -- 172.21.15.67:0/986015065 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb27c0b3cd0 msgr2=0x7fb27c0b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:53.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.074+0000 7fb29774a640 1 --2- 172.21.15.67:0/986015065 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb27c0b3cd0 0x7fb27c0b61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb288007e50 tx=0x7fb288007600 comp rx=0 tx=0).stop 2024-07-31T14:50:53.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.074+0000 7fb29774a640 1 -- 172.21.15.67:0/986015065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290107dd0 msgr2=0x7fb29019a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:53.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.074+0000 7fb29774a640 1 --2- 172.21.15.67:0/986015065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290107dd0 0x7fb29019a410 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb280031a90 tx=0x7fb280004270 comp rx=0 tx=0).stop 2024-07-31T14:50:53.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.074+0000 7fb29774a640 1 -- 172.21.15.67:0/986015065 shutdown_connections 2024-07-31T14:50:53.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.074+0000 7fb29774a640 1 --2- 172.21.15.67:0/986015065 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb27c0b3cd0 0x7fb27c0b61e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.074+0000 7fb29774a640 1 --2- 172.21.15.67:0/986015065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb290108720 0x7fb29019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.074+0000 7fb29774a640 1 --2- 172.21.15.67:0/986015065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290107dd0 0x7fb29019a410 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.074+0000 7fb29774a640 1 --2- 172.21.15.67:0/986015065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb290102490 0x7fb290199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:53.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.074+0000 7fb29774a640 1 -- 172.21.15.67:0/986015065 >> 172.21.15.67:0/986015065 conn(0x7fb290079160 msgr2=0x7fb29006ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:53.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.074+0000 7fb29774a640 1 -- 172.21.15.67:0/986015065 shutdown_connections 2024-07-31T14:50:53.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:53.074+0000 7fb29774a640 1 -- 172.21.15.67:0/986015065 wait complete. 2024-07-31T14:50:53.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:53 smithi067 ceph-mon[32750]: pgmap v393: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:53 smithi073 ceph-mon[36714]: pgmap v393: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:53.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:53 smithi077 ceph-mon[37327]: pgmap v393: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:50:53.826 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.h status 2024-07-31T14:50:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:55.230 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:55 smithi067 ceph-mon[32750]: pgmap v394: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:55.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:55 smithi073 ceph-mon[36714]: pgmap v394: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:55.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:55 smithi077 ceph-mon[37327]: pgmap v394: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:50:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.786+0000 7fcc4053f640 1 -- 172.21.15.67:0/1540036009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc38107060 msgr2=0x7fcc38107470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.786+0000 7fcc4053f640 1 --2- 172.21.15.67:0/1540036009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc38107060 0x7fcc38107470 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009900 tx=0x7fcc3402a410 comp rx=0 tx=0).stop 2024-07-31T14:50:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.786+0000 7fcc4053f640 1 -- 172.21.15.67:0/1540036009 shutdown_connections 2024-07-31T14:50:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.786+0000 7fcc4053f640 1 --2- 172.21.15.67:0/1540036009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc38103320 0x7fcc3810eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.786+0000 7fcc4053f640 1 --2- 172.21.15.67:0/1540036009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc381029d0 0x7fcc38102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.786+0000 7fcc4053f640 1 --2- 172.21.15.67:0/1540036009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc38107060 0x7fcc38107470 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.786+0000 7fcc4053f640 1 -- 172.21.15.67:0/1540036009 >> 172.21.15.67:0/1540036009 conn(0x7fcc38079870 msgr2=0x7fcc38079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.786+0000 7fcc4053f640 1 -- 172.21.15.67:0/1540036009 shutdown_connections 2024-07-31T14:50:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.786+0000 7fcc4053f640 1 -- 172.21.15.67:0/1540036009 wait complete. 2024-07-31T14:50:55.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.787+0000 7fcc4053f640 1 Processor -- start 2024-07-31T14:50:55.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.787+0000 7fcc4053f640 1 -- start start 2024-07-31T14:50:55.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.787+0000 7fcc4053f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc381029d0 0x7fcc3810fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.787+0000 7fcc4053f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc38103320 0x7fcc3810ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc4053f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc38107060 0x7fcc381104e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc4053f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38113530 con 0x7fcc381029d0 2024-07-31T14:50:55.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc4053f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381133b0 con 0x7fcc38103320 2024-07-31T14:50:55.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc4053f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381136b0 con 0x7fcc38107060 2024-07-31T14:50:55.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc3e2db640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc381029d0 0x7fcc3810fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc3e2db640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc381029d0 0x7fcc3810fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46184/0 (socket says 172.21.15.67:46184) 2024-07-31T14:50:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc3e2db640 1 -- 172.21.15.67:0/4282880154 learned_addr learned my addr 172.21.15.67:0/4282880154 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc3dada640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc38103320 0x7fcc3810ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc3eadc640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc38107060 0x7fcc381104e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc3e2db640 1 -- 172.21.15.67:0/4282880154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc38107060 msgr2=0x7fcc381104e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc3e2db640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc38107060 0x7fcc381104e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc3e2db640 1 -- 172.21.15.67:0/4282880154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc38103320 msgr2=0x7fcc3810ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc3e2db640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc38103320 0x7fcc3810ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc3e2db640 1 -- 172.21.15.67:0/4282880154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc381a81a0 con 0x7fcc381029d0 2024-07-31T14:50:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc3eadc640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc38107060 0x7fcc381104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:50:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc3e2db640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc381029d0 0x7fcc3810fa60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcc34005940 tx=0x7fcc340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc2f7fe640 1 -- 172.21.15.67:0/4282880154 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3403f030 con 0x7fcc381029d0 2024-07-31T14:50:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc2f7fe640 1 -- 172.21.15.67:0/4282880154 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc34003cb0 con 0x7fcc381029d0 2024-07-31T14:50:55.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.788+0000 7fcc4053f640 1 -- 172.21.15.67:0/4282880154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc381a8430 con 0x7fcc381029d0 2024-07-31T14:50:55.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.789+0000 7fcc2f7fe640 1 -- 172.21.15.67:0/4282880154 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34035070 con 0x7fcc381029d0 2024-07-31T14:50:55.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.789+0000 7fcc4053f640 1 -- 172.21.15.67:0/4282880154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc381a8910 con 0x7fcc381029d0 2024-07-31T14:50:55.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.791+0000 7fcc46a90740 1 Processor -- start 2024-07-31T14:50:55.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.791+0000 7fcc2f7fe640 1 -- 172.21.15.67:0/4282880154 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fcc34042030 con 0x7fcc381029d0 2024-07-31T14:50:55.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.791+0000 7fcc46a90740 1 -- start start 2024-07-31T14:50:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc46a90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af70f60 0x56418b3163a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc46a90740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af480a0 0x56418b3168e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc46a90740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af50280 0x56418b316e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc46a90740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56418b28c940 con 0x56418af480a0 2024-07-31T14:50:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc46a90740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56418b27be40 con 0x56418af50280 2024-07-31T14:50:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc46a90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56418b27fe80 con 0x56418af70f60 2024-07-31T14:50:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc3e2db640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af480a0 0x56418b3168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc3e2db640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af480a0 0x56418b3168e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46196/0 (socket says 172.21.15.67:46196) 2024-07-31T14:50:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc3e2db640 1 -- 172.21.15.67:0/4025157899 learned_addr learned my addr 172.21.15.67:0/4025157899 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc3dada640 1 --2- 172.21.15.67:0/4025157899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af50280 0x56418b316e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc3eadc640 1 --2- 172.21.15.67:0/4025157899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af70f60 0x56418b3163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc2f7fe640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc240b3c20 0x7fcc240b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc3e2db640 1 -- 172.21.15.67:0/4025157899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af70f60 msgr2=0x56418b3163a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc3e2db640 1 --2- 172.21.15.67:0/4025157899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af70f60 0x56418b3163a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc3e2db640 1 -- 172.21.15.67:0/4025157899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af50280 msgr2=0x56418b316e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc3e2db640 1 --2- 172.21.15.67:0/4025157899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af50280 0x56418b316e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc3e2db640 1 -- 172.21.15.67:0/4025157899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418b222a60 con 0x56418af480a0 2024-07-31T14:50:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc3eadc640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc240b3c20 0x7fcc240b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.792+0000 7fcc3dada640 1 --2- 172.21.15.67:0/4025157899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af50280 0x56418b316e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:50:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.793+0000 7fcc2f7fe640 1 -- 172.21.15.67:0/4282880154 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fcc34104950 con 0x7fcc381029d0 2024-07-31T14:50:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.793+0000 7fcc3e2db640 1 --2- 172.21.15.67:0/4025157899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af480a0 0x56418b3168e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcc340b85a0 tx=0x7fcc34090340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da719d054834a4a6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.793+0000 7fcc2cff9640 1 -- 172.21.15.67:0/4025157899 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34106e60 con 0x56418af480a0 2024-07-31T14:50:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.793+0000 7fcc3eadc640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc240b3c20 0x7fcc240b6130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcc30005fd0 tx=0x7fcc30004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.793+0000 7fcc2cff9640 1 -- 172.21.15.67:0/4025157899 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3403b460 con 0x56418af480a0 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.793+0000 7fcc2cff9640 1 -- 172.21.15.67:0/4025157899 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34003e60 con 0x56418af480a0 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.794+0000 7fcc46a90740 1 -- 172.21.15.67:0/4025157899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af480a0 msgr2=0x56418b3168e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.794+0000 7fcc46a90740 1 --2- 172.21.15.67:0/4025157899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af480a0 0x56418b3168e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcc340b85a0 tx=0x7fcc34090340 comp rx=0 tx=0).stop 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.794+0000 7fcc46a90740 1 -- 172.21.15.67:0/4025157899 shutdown_connections 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.794+0000 7fcc46a90740 1 --2- 172.21.15.67:0/4025157899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af50280 0x56418b316e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.794+0000 7fcc46a90740 1 --2- 172.21.15.67:0/4025157899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af480a0 0x56418b3168e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.794+0000 7fcc46a90740 1 --2- 172.21.15.67:0/4025157899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af70f60 0x56418b3163a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.794+0000 7fcc46a90740 1 -- 172.21.15.67:0/4025157899 >> 172.21.15.67:0/4025157899 conn(0x56418af65350 msgr2=0x56418b314290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.794+0000 7fcc46a90740 1 -- 172.21.15.67:0/4025157899 shutdown_connections 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.794+0000 7fcc46a90740 1 -- 172.21.15.67:0/4025157899 wait complete. 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.794+0000 7fcc46a90740 1 -- start start 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc46a90740 1 Processor -- start 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc46a90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af480a0 0x56418b321f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc46a90740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af50280 0x56418b3224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc46a90740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af70f60 0x56418b32db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc46a90740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56418b0df5f0 con 0x56418af50280 2024-07-31T14:50:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc46a90740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56418b21e8b0 con 0x56418af70f60 2024-07-31T14:50:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc46a90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56418b324940 con 0x56418af480a0 2024-07-31T14:50:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc46a90740 10 client.0 _ll_register_callbacks cb 0x56418b2287f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc3dada640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af50280 0x56418b3224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc3dada640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af50280 0x56418b3224e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46206/0 (socket says 172.21.15.67:46206) 2024-07-31T14:50:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc3dada640 1 -- 172.21.15.67:0/3005831895 learned_addr learned my addr 172.21.15.67:0/3005831895 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc3e2db640 1 --2- 172.21.15.67:0/3005831895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af480a0 0x56418b321f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc3eadc640 1 --2- 172.21.15.67:0/3005831895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af70f60 0x56418b32db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc3dada640 1 -- 172.21.15.67:0/3005831895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af480a0 msgr2=0x56418b321f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc3dada640 1 --2- 172.21.15.67:0/3005831895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af480a0 0x56418b321f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc3dada640 1 -- 172.21.15.67:0/3005831895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af70f60 msgr2=0x56418b32db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc3dada640 1 --2- 172.21.15.67:0/3005831895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af70f60 0x56418b32db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc3dada640 1 -- 172.21.15.67:0/3005831895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418afdc540 con 0x56418af50280 2024-07-31T14:50:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.795+0000 7fcc3eadc640 1 --2- 172.21.15.67:0/3005831895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af70f60 0x56418b32db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:50:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.796+0000 7fcc3dada640 1 --2- 172.21.15.67:0/3005831895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af50280 0x56418b3224e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800bf50 tx=0x7fcc280118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aaeb03ebaaf9ca5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.796+0000 7fcc21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.796+0000 7fcc21ffb640 1 -- 172.21.15.67:0/3005831895 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28004030 con 0x56418af50280 2024-07-31T14:50:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.796+0000 7fcc21ffb640 1 -- 172.21.15.67:0/3005831895 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fcc280041d0 con 0x56418af50280 2024-07-31T14:50:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.796+0000 7fcc46a90740 1 -- 172.21.15.67:0/3005831895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56418b28e890 con 0x56418af50280 2024-07-31T14:50:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.796+0000 7fcc21ffb640 1 -- 172.21.15.67:0/3005831895 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc280043d0 con 0x56418af50280 2024-07-31T14:50:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.796+0000 7fcc46a90740 10 client.15432 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.796+0000 7fcc46a90740 1 -- 172.21.15.67:0/3005831895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56418b3308d0 con 0x56418af50280 2024-07-31T14:50:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.796+0000 7fcc21ffb640 1 -- 172.21.15.67:0/3005831895 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56418b28e890 con 0x56418af50280 2024-07-31T14:50:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.797+0000 7fcc21ffb640 1 -- 172.21.15.67:0/3005831895 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcc28003950 con 0x56418af50280 2024-07-31T14:50:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.797+0000 7fcc46a90740 10 client.15432 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.797+0000 7fcc46a90740 10 client.15432 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-31T14:50:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.797+0000 7fcc46a90740 20 client.15432 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.797+0000 7fcc46a90740 1 --2- 172.21.15.67:0/3005831895 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56418af2f8e0 0x56418b3343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.797+0000 7fcc46a90740 4 client.15432 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:50:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.797+0000 7fcc46a90740 1 -- 172.21.15.67:0/3005831895 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56418b28a330 con 0x56418af2f8e0 2024-07-31T14:50:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.797+0000 7fcc3e2db640 1 --2- 172.21.15.67:0/3005831895 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56418af2f8e0 0x56418b3343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.798+0000 7fcc3e2db640 1 --2- 172.21.15.67:0/3005831895 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56418af2f8e0 0x56418b3343b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d761571291b51043 server_cookie=470c5961db500a28 in_seq=0 out_seq=0 2024-07-31T14:50:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.798+0000 7fcc21ffb640 10 client.15432 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:50:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.798+0000 7fcc21ffb640 1 -- 172.21.15.67:0/3005831895 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56418b28a330 con 0x56418af2f8e0 2024-07-31T14:50:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.798+0000 7fcc21ffb640 10 client.15432 handle_command_reply: tid=0 2024-07-31T14:50:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.798+0000 7fcc46a90740 1 client.15432 shutdown 2024-07-31T14:50:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.799+0000 7fcc46a90740 1 -- 172.21.15.67:0/3005831895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af50280 msgr2=0x56418b3224e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.799+0000 7fcc46a90740 1 --2- 172.21.15.67:0/3005831895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af50280 0x56418b3224e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800bf50 tx=0x7fcc280118d0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.799+0000 7fcc46a90740 1 -- 172.21.15.67:0/3005831895 shutdown_connections 2024-07-31T14:50:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.799+0000 7fcc46a90740 1 --2- 172.21.15.67:0/3005831895 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56418af2f8e0 0x56418b3343b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.799+0000 7fcc46a90740 1 --2- 172.21.15.67:0/3005831895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af70f60 0x56418b32db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.799+0000 7fcc46a90740 1 --2- 172.21.15.67:0/3005831895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af50280 0x56418b3224e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.799+0000 7fcc46a90740 1 --2- 172.21.15.67:0/3005831895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af480a0 0x56418b321f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.799+0000 7fcc46a90740 1 -- 172.21.15.67:0/3005831895 >> 172.21.15.67:0/3005831895 conn(0x56418af65350 msgr2=0x56418b3173d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.799+0000 7fcc21ffb640 0 client.15432 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:50:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.799+0000 7fcc46a90740 1 -- 172.21.15.67:0/3005831895 shutdown_connections 2024-07-31T14:50:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.799+0000 7fcc46a90740 1 -- 172.21.15.67:0/3005831895 wait complete. 2024-07-31T14:50:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.799+0000 7fcc46a90740 20 client.15432 trim_cache size 0 max 16384 2024-07-31T14:50:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.818+0000 7fcc46a90740 1 Processor -- start 2024-07-31T14:50:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.818+0000 7fcc46a90740 1 -- start start 2024-07-31T14:50:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.818+0000 7fcc46a90740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af480a0 0x56418b32d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.818+0000 7fcc46a90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af50280 0x56418b321f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.818+0000 7fcc46a90740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af70f60 0x56418b3224a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.818+0000 7fcc46a90740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56418b3326f0 con 0x56418af70f60 2024-07-31T14:50:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.818+0000 7fcc46a90740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56418b332dc0 con 0x56418af480a0 2024-07-31T14:50:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.818+0000 7fcc46a90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56418b333490 con 0x56418af50280 2024-07-31T14:50:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.819+0000 7fcc3eadc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af70f60 0x56418b3224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.819+0000 7fcc3eadc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af70f60 0x56418b3224a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46220/0 (socket says 172.21.15.67:46220) 2024-07-31T14:50:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.819+0000 7fcc3eadc640 1 -- 172.21.15.67:0/732294430 learned_addr learned my addr 172.21.15.67:0/732294430 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.819+0000 7fcc3dada640 1 --2- 172.21.15.67:0/732294430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af50280 0x56418b321f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.819+0000 7fcc3e2db640 1 --2- 172.21.15.67:0/732294430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af480a0 0x56418b32d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.819+0000 7fcc3eadc640 1 -- 172.21.15.67:0/732294430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af50280 msgr2=0x56418b321f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.819+0000 7fcc3eadc640 1 --2- 172.21.15.67:0/732294430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af50280 0x56418b321f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.819+0000 7fcc3eadc640 1 -- 172.21.15.67:0/732294430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af480a0 msgr2=0x56418b32d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.819+0000 7fcc3eadc640 1 --2- 172.21.15.67:0/732294430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af480a0 0x56418b32d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.819+0000 7fcc3eadc640 1 -- 172.21.15.67:0/732294430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418b322c10 con 0x56418af70f60 2024-07-31T14:50:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.819+0000 7fcc3eadc640 1 --2- 172.21.15.67:0/732294430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af70f60 0x56418b3224a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcc3001d750 tx=0x7fcc30014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debcd8050524b70e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.819+0000 7fcc23fff640 1 -- 172.21.15.67:0/732294430 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3002b070 con 0x56418af70f60 2024-07-31T14:50:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.819+0000 7fcc23fff640 1 -- 172.21.15.67:0/732294430 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3001bb50 con 0x56418af70f60 2024-07-31T14:50:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.819+0000 7fcc23fff640 1 -- 172.21.15.67:0/732294430 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30026c30 con 0x56418af70f60 2024-07-31T14:50:55.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.820+0000 7fcc46a90740 1 -- 172.21.15.67:0/732294430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af70f60 msgr2=0x56418b3224a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:55.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.820+0000 7fcc46a90740 1 --2- 172.21.15.67:0/732294430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af70f60 0x56418b3224a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcc3001d750 tx=0x7fcc30014000 comp rx=0 tx=0).stop 2024-07-31T14:50:55.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.820+0000 7fcc46a90740 1 -- 172.21.15.67:0/732294430 shutdown_connections 2024-07-31T14:50:55.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.820+0000 7fcc46a90740 1 --2- 172.21.15.67:0/732294430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af70f60 0x56418b3224a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.820+0000 7fcc46a90740 1 --2- 172.21.15.67:0/732294430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af50280 0x56418b321f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.820+0000 7fcc46a90740 1 --2- 172.21.15.67:0/732294430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af480a0 0x56418b32d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.820+0000 7fcc46a90740 1 -- 172.21.15.67:0/732294430 >> 172.21.15.67:0/732294430 conn(0x56418af65350 msgr2=0x56418b31c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:55.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.820+0000 7fcc46a90740 1 -- 172.21.15.67:0/732294430 shutdown_connections 2024-07-31T14:50:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.820+0000 7fcc46a90740 1 -- 172.21.15.67:0/732294430 wait complete. 2024-07-31T14:50:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.821+0000 7fcc46a90740 1 -- start start 2024-07-31T14:50:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.821+0000 7fcc46a90740 1 Processor -- start 2024-07-31T14:50:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.821+0000 7fcc46a90740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af480a0 0x56418b3357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.821+0000 7fcc46a90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af50280 0x56418b335d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.821+0000 7fcc46a90740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af70f60 0x56418b33ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.821+0000 7fcc46a90740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56418b334230 con 0x56418af480a0 2024-07-31T14:50:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.821+0000 7fcc46a90740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56418b333b60 con 0x56418af70f60 2024-07-31T14:50:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.821+0000 7fcc46a90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56418b334900 con 0x56418af50280 2024-07-31T14:50:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.821+0000 7fcc46a90740 10 client.0 _ll_register_callbacks cb 0x7fcc2800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.821+0000 7fcc3e2db640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af480a0 0x56418b3357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.821+0000 7fcc3e2db640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af480a0 0x56418b3357b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46232/0 (socket says 172.21.15.67:46232) 2024-07-31T14:50:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.821+0000 7fcc3e2db640 1 -- 172.21.15.67:0/3076358271 learned_addr learned my addr 172.21.15.67:0/3076358271 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.821+0000 7fcc3dada640 1 --2- 172.21.15.67:0/3076358271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af50280 0x56418b335d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc3eadc640 1 --2- 172.21.15.67:0/3076358271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af70f60 0x56418b33ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc3e2db640 1 -- 172.21.15.67:0/3076358271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af50280 msgr2=0x56418b335d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc3e2db640 1 --2- 172.21.15.67:0/3076358271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af50280 0x56418b335d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc3e2db640 1 -- 172.21.15.67:0/3076358271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af70f60 msgr2=0x56418b33ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc3e2db640 1 --2- 172.21.15.67:0/3076358271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af70f60 0x56418b33ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc3e2db640 1 -- 172.21.15.67:0/3076358271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418b33b3f0 con 0x56418af480a0 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc3eadc640 1 --2- 172.21.15.67:0/3076358271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af70f60 0x56418b33ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc3e2db640 1 --2- 172.21.15.67:0/3076358271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af480a0 0x56418b3357b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcc340c86e0 tx=0x7fcc3402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ebe118366059bc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc21ffb640 1 -- 172.21.15.67:0/3076358271 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34049040 con 0x56418af480a0 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc21ffb640 1 -- 172.21.15.67:0/3076358271 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fcc34102070 con 0x56418af480a0 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc46a90740 1 -- 172.21.15.67:0/3076358271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc3001bb50 con 0x56418af480a0 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc21ffb640 1 -- 172.21.15.67:0/3076358271 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc34003e60 con 0x56418af480a0 2024-07-31T14:50:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc46a90740 10 client.15438 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc46a90740 1 -- 172.21.15.67:0/3076358271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56418b33c7b0 con 0x56418af480a0 2024-07-31T14:50:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.822+0000 7fcc21ffb640 1 -- 172.21.15.67:0/3076358271 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3001bb50 con 0x56418af480a0 2024-07-31T14:50:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.823+0000 7fcc21ffb640 1 -- 172.21.15.67:0/3076358271 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcc340c7700 con 0x56418af480a0 2024-07-31T14:50:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.823+0000 7fcc46a90740 10 client.15438 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.823+0000 7fcc46a90740 10 client.15438 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-31T14:50:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.823+0000 7fcc46a90740 20 client.15438 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.823+0000 7fcc46a90740 1 --2- 172.21.15.67:0/3076358271 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56418af04c90 0x56418b3a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.823+0000 7fcc46a90740 4 client.15438 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:50:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.823+0000 7fcc46a90740 1 -- 172.21.15.67:0/3076358271 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "status"}) -- 0x56418b287a70 con 0x56418af04c90 2024-07-31T14:50:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.823+0000 7fcc3dada640 1 --2- 172.21.15.67:0/3076358271 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56418af04c90 0x56418b3a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc3dada640 1 --2- 172.21.15.67:0/3076358271 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56418af04c90 0x56418b3a00f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=adb1c20fabb2c656 server_cookie=f15655f3f0f47e38 in_seq=0 out_seq=0 2024-07-31T14:50:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc21ffb640 10 client.15438 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:50:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc21ffb640 1 -- 172.21.15.67:0/3076358271 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56418b287a70 con 0x56418af04c90 2024-07-31T14:50:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc21ffb640 10 client.15438 handle_command_reply: tid=0 2024-07-31T14:50:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc46a90740 1 client.15438 shutdown 2024-07-31T14:50:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc46a90740 1 -- 172.21.15.67:0/3076358271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af480a0 msgr2=0x56418b3357b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc46a90740 1 --2- 172.21.15.67:0/3076358271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af480a0 0x56418b3357b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcc340c86e0 tx=0x7fcc3402a410 comp rx=0 tx=0).stop 2024-07-31T14:50:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc46a90740 1 -- 172.21.15.67:0/3076358271 shutdown_connections 2024-07-31T14:50:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc46a90740 1 --2- 172.21.15.67:0/3076358271 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56418af04c90 0x56418b3a00f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc46a90740 1 --2- 172.21.15.67:0/3076358271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56418af70f60 0x56418b33ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc46a90740 1 --2- 172.21.15.67:0/3076358271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56418af50280 0x56418b335d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc46a90740 1 --2- 172.21.15.67:0/3076358271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56418af480a0 0x56418b3357b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc46a90740 1 -- 172.21.15.67:0/3076358271 >> 172.21.15.67:0/3076358271 conn(0x56418af65350 msgr2=0x56418b320540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc21ffb640 0 client.15438 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:50:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc46a90740 1 -- 172.21.15.67:0/3076358271 shutdown_connections 2024-07-31T14:50:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc46a90740 1 -- 172.21.15.67:0/3076358271 wait complete. 2024-07-31T14:50:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.824+0000 7fcc46a90740 20 client.15438 trim_cache size 0 max 16384 2024-07-31T14:50:55.831 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:50:55.831 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:50:55.831 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 2, 2024-07-31T14:50:55.831 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24650, 2024-07-31T14:50:55.831 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:50:55.831 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:50:55.831 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:50:55.831 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 175.26326582900001, 2024-07-31T14:50:55.831 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-31T14:50:55.831 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:50:55.831 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:50:55.832 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 257.03754821699999 2024-07-31T14:50:55.832 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:50:55.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.825+0000 7fcc4053f640 1 -- 172.21.15.67:0/4282880154 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc240b3c20 msgr2=0x7fcc240b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:55.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.825+0000 7fcc4053f640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc240b3c20 0x7fcc240b6130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcc30005fd0 tx=0x7fcc30004a40 comp rx=0 tx=0).stop 2024-07-31T14:50:55.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.825+0000 7fcc4053f640 1 -- 172.21.15.67:0/4282880154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc381029d0 msgr2=0x7fcc3810fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:55.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.825+0000 7fcc4053f640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc381029d0 0x7fcc3810fa60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcc34005940 tx=0x7fcc340042f0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.825+0000 7fcc4053f640 1 -- 172.21.15.67:0/4282880154 shutdown_connections 2024-07-31T14:50:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.825+0000 7fcc4053f640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcc240b3c20 0x7fcc240b6130 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.825+0000 7fcc4053f640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc38107060 0x7fcc381104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.825+0000 7fcc4053f640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc38103320 0x7fcc3810ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.825+0000 7fcc4053f640 1 --2- 172.21.15.67:0/4282880154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc381029d0 0x7fcc3810fa60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.825+0000 7fcc4053f640 1 -- 172.21.15.67:0/4282880154 >> 172.21.15.67:0/4282880154 conn(0x7fcc38079870 msgr2=0x7fcc38105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.825+0000 7fcc4053f640 1 -- 172.21.15.67:0/4282880154 shutdown_connections 2024-07-31T14:50:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:55.825+0000 7fcc4053f640 1 -- 172.21.15.67:0/4282880154 wait complete. 2024-07-31T14:50:56.522 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.i status 2024-07-31T14:50:56.790 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:50:57.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:57 smithi067 ceph-mon[32750]: pgmap v395: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:57.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:57 smithi073 ceph-mon[36714]: pgmap v395: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:57.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:57 smithi077 ceph-mon[37327]: pgmap v395: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:50:58.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.503+0000 7f9bd70e9640 1 -- 172.21.15.67:0/3998591428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bd0108730 msgr2=0x7f9bd010ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:58.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.503+0000 7f9bd70e9640 1 --2- 172.21.15.67:0/3998591428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bd0108730 0x7f9bd010ef60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009ca0 tx=0x7f9bc802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.503+0000 7f9bd70e9640 1 -- 172.21.15.67:0/3998591428 shutdown_connections 2024-07-31T14:50:58.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.503+0000 7f9bd70e9640 1 --2- 172.21.15.67:0/3998591428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bd0108730 0x7f9bd010ef60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.503+0000 7f9bd70e9640 1 --2- 172.21.15.67:0/3998591428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bd0107de0 0x7f9bd01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.503+0000 7f9bd70e9640 1 --2- 172.21.15.67:0/3998591428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bd01024a0 0x7f9bd01028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.503+0000 7f9bd70e9640 1 -- 172.21.15.67:0/3998591428 >> 172.21.15.67:0/3998591428 conn(0x7f9bd0079140 msgr2=0x7f9bd0079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.503+0000 7f9bd70e9640 1 -- 172.21.15.67:0/3998591428 shutdown_connections 2024-07-31T14:50:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.504+0000 7f9bd70e9640 1 -- 172.21.15.67:0/3998591428 wait complete. 2024-07-31T14:50:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.504+0000 7f9bd70e9640 1 Processor -- start 2024-07-31T14:50:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.504+0000 7f9bd70e9640 1 -- start start 2024-07-31T14:50:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd70e9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bd01024a0 0x7f9bd019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd70e9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bd0107de0 0x7f9bd019b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd70e9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bd0108730 0x7f9bd019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd70e9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0112400 con 0x7f9bd0108730 2024-07-31T14:50:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd70e9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0112280 con 0x7f9bd01024a0 2024-07-31T14:50:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd70e9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0112580 con 0x7f9bd0107de0 2024-07-31T14:50:58.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd5686640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bd0108730 0x7f9bd019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd5686640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bd0108730 0x7f9bd019b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46258/0 (socket says 172.21.15.67:46258) 2024-07-31T14:50:58.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd5686640 1 -- 172.21.15.67:0/2128157922 learned_addr learned my addr 172.21.15.67:0/2128157922 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:58.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd4e85640 1 --2- 172.21.15.67:0/2128157922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bd01024a0 0x7f9bd019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bcffff640 1 --2- 172.21.15.67:0/2128157922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bd0107de0 0x7f9bd019b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd5686640 1 -- 172.21.15.67:0/2128157922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bd0107de0 msgr2=0x7f9bd019b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:58.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd5686640 1 --2- 172.21.15.67:0/2128157922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bd0107de0 0x7f9bd019b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd5686640 1 -- 172.21.15.67:0/2128157922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bd01024a0 msgr2=0x7f9bd019acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:58.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd5686640 1 --2- 172.21.15.67:0/2128157922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bd01024a0 0x7f9bd019acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd5686640 1 -- 172.21.15.67:0/2128157922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd01a5f80 con 0x7f9bd0108730 2024-07-31T14:50:58.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.505+0000 7f9bd5686640 1 --2- 172.21.15.67:0/2128157922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bd0108730 0x7f9bd019b770 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005cc0 tx=0x7f9bc802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:58.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.506+0000 7f9bcdffb640 1 -- 172.21.15.67:0/2128157922 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8003bf0 con 0x7f9bd0108730 2024-07-31T14:50:58.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.506+0000 7f9bcdffb640 1 -- 172.21.15.67:0/2128157922 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc8003d90 con 0x7f9bd0108730 2024-07-31T14:50:58.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.506+0000 7f9bcdffb640 1 -- 172.21.15.67:0/2128157922 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc802fbc0 con 0x7f9bd0108730 2024-07-31T14:50:58.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.506+0000 7f9bd70e9640 1 -- 172.21.15.67:0/2128157922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd01a6210 con 0x7f9bd0108730 2024-07-31T14:50:58.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.506+0000 7f9bd70e9640 1 -- 172.21.15.67:0/2128157922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd01a67a0 con 0x7f9bd0108730 2024-07-31T14:50:58.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.509+0000 7f9bdd63a740 1 Processor -- start 2024-07-31T14:50:58.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.509+0000 7f9bcdffb640 1 -- 172.21.15.67:0/2128157922 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f9bc802fd60 con 0x7f9bd0108730 2024-07-31T14:50:58.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.509+0000 7f9bdd63a740 1 -- start start 2024-07-31T14:50:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.509+0000 7f9bdd63a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c41c4f60 0x55c4c456a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.509+0000 7f9bdd63a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c419c0a0 0x55c4c456a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.509+0000 7f9bdd63a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 0x55c4c456ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.509+0000 7f9bdd63a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c4c44e0940 con 0x55c4c41c4f60 2024-07-31T14:50:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.509+0000 7f9bdd63a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c4c44cfe40 con 0x55c4c41a4280 2024-07-31T14:50:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.509+0000 7f9bdd63a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c4c44d3e80 con 0x55c4c419c0a0 2024-07-31T14:50:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.509+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c41c4f60 0x55c4c456a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c41c4f60 0x55c4c456a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46264/0 (socket says 172.21.15.67:46264) 2024-07-31T14:50:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bcffff640 1 -- 172.21.15.67:0/1406341311 learned_addr learned my addr 172.21.15.67:0/1406341311 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bd4e85640 1 --2- 172.21.15.67:0/1406341311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c419c0a0 0x55c4c456a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bcffff640 1 --2- 172.21.15.67:0/1406341311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 0x55c4c456ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bcdffb640 1 --2- 172.21.15.67:0/2128157922 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9bb80b3cd0 0x7f9bb80b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bcffff640 1 -- 172.21.15.67:0/1406341311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c419c0a0 msgr2=0x55c4c456a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bcffff640 1 --2- 172.21.15.67:0/1406341311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c419c0a0 0x55c4c456a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bcffff640 1 -- 172.21.15.67:0/1406341311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 msgr2=0x55c4c456ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bcffff640 1 --2- 172.21.15.67:0/1406341311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 0x55c4c456ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bcffff640 1 -- 172.21.15.67:0/1406341311 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4c4476a60 con 0x55c4c41c4f60 2024-07-31T14:50:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bd5686640 1 --2- 172.21.15.67:0/2128157922 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9bb80b3cd0 0x7f9bb80b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bcffff640 1 --2- 172.21.15.67:0/1406341311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c41c4f60 0x55c4c456a3a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400bb00 tx=0x7f9bc4004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6375dc388fc44019 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bcdffb640 1 -- 172.21.15.67:0/2128157922 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f9bc8105500 con 0x7f9bd0108730 2024-07-31T14:50:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bb6ffd640 1 -- 172.21.15.67:0/1406341311 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4005020 con 0x55c4c41c4f60 2024-07-31T14:50:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.510+0000 7f9bb6ffd640 1 -- 172.21.15.67:0/1406341311 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc4005670 con 0x55c4c41c4f60 2024-07-31T14:50:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.511+0000 7f9bb6ffd640 1 -- 172.21.15.67:0/1406341311 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4010460 con 0x55c4c41c4f60 2024-07-31T14:50:58.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.511+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1406341311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c41c4f60 msgr2=0x55c4c456a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:58.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.511+0000 7f9bd5686640 1 --2- 172.21.15.67:0/2128157922 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9bb80b3cd0 0x7f9bb80b61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80c80a0 tx=0x7f9bc8099770 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:58.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.511+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/1406341311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c41c4f60 0x55c4c456a3a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400bb00 tx=0x7f9bc4004150 comp rx=0 tx=0).stop 2024-07-31T14:50:58.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.511+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1406341311 shutdown_connections 2024-07-31T14:50:58.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.511+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/1406341311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 0x55c4c456ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.511+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/1406341311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c419c0a0 0x55c4c456a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.511+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/1406341311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c41c4f60 0x55c4c456a3a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.511+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1406341311 >> 172.21.15.67:0/1406341311 conn(0x55c4c41b9350 msgr2=0x55c4c4568290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:58.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.512+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1406341311 shutdown_connections 2024-07-31T14:50:58.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.512+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1406341311 wait complete. 2024-07-31T14:50:58.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.512+0000 7f9bdd63a740 1 -- start start 2024-07-31T14:50:58.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.512+0000 7f9bdd63a740 1 Processor -- start 2024-07-31T14:50:58.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bdd63a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c419c0a0 0x55c4c4575fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bdd63a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 0x55c4c4576520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bdd63a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c41c4f60 0x55c4c4581e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bdd63a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c4c43335f0 con 0x55c4c419c0a0 2024-07-31T14:50:58.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bdd63a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c4c44728b0 con 0x55c4c41a4280 2024-07-31T14:50:58.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bdd63a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c4c4578b20 con 0x55c4c41c4f60 2024-07-31T14:50:58.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bdd63a740 10 client.0 _ll_register_callbacks cb 0x55c4c447c7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:58.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c419c0a0 0x55c4c4575fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c419c0a0 0x55c4c4575fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46276/0 (socket says 172.21.15.67:46276) 2024-07-31T14:50:58.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bcffff640 1 -- 172.21.15.67:0/316806366 learned_addr learned my addr 172.21.15.67:0/316806366 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:58.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bd4e85640 1 --2- 172.21.15.67:0/316806366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 0x55c4c4576520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bcffff640 1 --2- 172.21.15.67:0/316806366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c41c4f60 0x55c4c4581e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bcffff640 1 -- 172.21.15.67:0/316806366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c41c4f60 msgr2=0x55c4c4581e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bcffff640 1 --2- 172.21.15.67:0/316806366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c41c4f60 0x55c4c4581e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bcffff640 1 -- 172.21.15.67:0/316806366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 msgr2=0x55c4c4576520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bcffff640 1 --2- 172.21.15.67:0/316806366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 0x55c4c4576520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bcffff640 1 -- 172.21.15.67:0/316806366 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4c4230540 con 0x55c4c419c0a0 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bd4e85640 1 --2- 172.21.15.67:0/316806366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 0x55c4c4576520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9bcffff640 1 --2- 172.21.15.67:0/316806366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c419c0a0 0x55c4c4575fa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400b5f0 tx=0x7f9bc400b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c767cb6bc1283b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.513+0000 7f9baffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.514+0000 7f9baffff640 1 -- 172.21.15.67:0/316806366 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4005000 con 0x55c4c419c0a0 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.514+0000 7f9bdd63a740 1 -- 172.21.15.67:0/316806366 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc4005670 con 0x55c4c419c0a0 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.514+0000 7f9baffff640 1 -- 172.21.15.67:0/316806366 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f9bc40051a0 con 0x55c4c419c0a0 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.514+0000 7f9baffff640 1 -- 172.21.15.67:0/316806366 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9bc402f050 con 0x55c4c419c0a0 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.514+0000 7f9bdd63a740 10 client.15450 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.514+0000 7f9bdd63a740 1 -- 172.21.15.67:0/316806366 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4c4584b80 con 0x55c4c419c0a0 2024-07-31T14:50:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.514+0000 7f9baffff640 1 -- 172.21.15.67:0/316806366 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc4005670 con 0x55c4c419c0a0 2024-07-31T14:50:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.514+0000 7f9baffff640 1 -- 172.21.15.67:0/316806366 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9bc401d3f0 con 0x55c4c419c0a0 2024-07-31T14:50:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.514+0000 7f9bdd63a740 10 client.15450 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.514+0000 7f9bdd63a740 10 client.15450 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-31T14:50:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.514+0000 7f9bdd63a740 20 client.15450 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.514+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/316806366 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c4c41838e0 0x55c4c4588360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.514+0000 7f9bdd63a740 4 client.15450 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:50:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.514+0000 7f9bdd63a740 1 -- 172.21.15.67:0/316806366 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4c44e2890 con 0x55c4c41838e0 2024-07-31T14:50:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.515+0000 7f9bd4e85640 1 --2- 172.21.15.67:0/316806366 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c4c41838e0 0x55c4c4588360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.515+0000 7f9bd4e85640 1 --2- 172.21.15.67:0/316806366 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c4c41838e0 0x55c4c4588360 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=654004b486eb69c7 server_cookie=f5a440ee0eb85ce6 in_seq=0 out_seq=0 2024-07-31T14:50:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.515+0000 7f9baffff640 10 client.15450 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:50:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.516+0000 7f9baffff640 1 -- 172.21.15.67:0/316806366 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4c44e2890 con 0x55c4c41838e0 2024-07-31T14:50:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.516+0000 7f9baffff640 10 client.15450 handle_command_reply: tid=0 2024-07-31T14:50:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.516+0000 7f9bdd63a740 1 client.15450 shutdown 2024-07-31T14:50:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.516+0000 7f9bdd63a740 1 -- 172.21.15.67:0/316806366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c419c0a0 msgr2=0x55c4c4575fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.516+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/316806366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c419c0a0 0x55c4c4575fa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400b5f0 tx=0x7f9bc400b770 comp rx=0 tx=0).stop 2024-07-31T14:50:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.516+0000 7f9bdd63a740 1 -- 172.21.15.67:0/316806366 shutdown_connections 2024-07-31T14:50:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.516+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/316806366 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c4c41838e0 0x55c4c4588360 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.516+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/316806366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c41c4f60 0x55c4c4581e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.516+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/316806366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 0x55c4c4576520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.516+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/316806366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c419c0a0 0x55c4c4575fa0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.516+0000 7f9baffff640 0 client.15450 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:50:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.516+0000 7f9bdd63a740 1 -- 172.21.15.67:0/316806366 >> 172.21.15.67:0/316806366 conn(0x55c4c41b9350 msgr2=0x55c4c456d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.516+0000 7f9bdd63a740 1 -- 172.21.15.67:0/316806366 shutdown_connections 2024-07-31T14:50:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.516+0000 7f9bdd63a740 1 -- 172.21.15.67:0/316806366 wait complete. 2024-07-31T14:50:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.517+0000 7f9bdd63a740 20 client.15450 trim_cache size 0 max 16384 2024-07-31T14:50:58.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.535+0000 7f9bdd63a740 1 Processor -- start 2024-07-31T14:50:58.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.535+0000 7f9bdd63a740 1 -- start start 2024-07-31T14:50:58.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bdd63a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c419c0a0 0x55c4c45765f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bdd63a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 0x55c4c4581d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bdd63a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c41c4f60 0x55c4c4567860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bdd63a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c4c45866a0 con 0x55c4c41c4f60 2024-07-31T14:50:58.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bdd63a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c4c4586d70 con 0x55c4c41a4280 2024-07-31T14:50:58.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bdd63a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c4c4587440 con 0x55c4c419c0a0 2024-07-31T14:50:58.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c41c4f60 0x55c4c4567860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c41c4f60 0x55c4c4567860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46278/0 (socket says 172.21.15.67:46278) 2024-07-31T14:50:58.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bcffff640 1 -- 172.21.15.67:0/1072157623 learned_addr learned my addr 172.21.15.67:0/1072157623 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:58.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bd4e85640 1 --2- 172.21.15.67:0/1072157623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 0x55c4c4581d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bcffff640 1 --2- 172.21.15.67:0/1072157623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c419c0a0 0x55c4c45765f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bcffff640 1 -- 172.21.15.67:0/1072157623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c419c0a0 msgr2=0x55c4c45765f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:58.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bcffff640 1 --2- 172.21.15.67:0/1072157623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c419c0a0 0x55c4c45765f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bcffff640 1 -- 172.21.15.67:0/1072157623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 msgr2=0x55c4c4581d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:58.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bcffff640 1 --2- 172.21.15.67:0/1072157623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 0x55c4c4581d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bcffff640 1 -- 172.21.15.67:0/1072157623 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4c4577060 con 0x55c4c41c4f60 2024-07-31T14:50:58.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bcffff640 1 --2- 172.21.15.67:0/1072157623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c41c4f60 0x55c4c4567860 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4000c00 tx=0x7f9bc4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfd422cf015fc9f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.536+0000 7f9bb67fc640 1 -- 172.21.15.67:0/1072157623 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc400b8d0 con 0x55c4c41c4f60 2024-07-31T14:50:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.537+0000 7f9bb67fc640 1 -- 172.21.15.67:0/1072157623 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc400b390 con 0x55c4c41c4f60 2024-07-31T14:50:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.537+0000 7f9bb67fc640 1 -- 172.21.15.67:0/1072157623 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4005490 con 0x55c4c41c4f60 2024-07-31T14:50:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.537+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1072157623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c41c4f60 msgr2=0x55c4c4567860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.537+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/1072157623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c41c4f60 0x55c4c4567860 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4000c00 tx=0x7f9bc4001e50 comp rx=0 tx=0).stop 2024-07-31T14:50:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.537+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1072157623 shutdown_connections 2024-07-31T14:50:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.537+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/1072157623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c41c4f60 0x55c4c4567860 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.537+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/1072157623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41a4280 0x55c4c4581d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.537+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/1072157623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c419c0a0 0x55c4c45765f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.537+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1072157623 >> 172.21.15.67:0/1072157623 conn(0x55c4c41b9350 msgr2=0x55c4c4574620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.537+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1072157623 shutdown_connections 2024-07-31T14:50:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.537+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1072157623 wait complete. 2024-07-31T14:50:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.538+0000 7f9bdd63a740 1 -- start start 2024-07-31T14:50:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.538+0000 7f9bdd63a740 1 Processor -- start 2024-07-31T14:50:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.538+0000 7f9bdd63a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c419c0a0 0x55c4c4590720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.538+0000 7f9bdd63a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c41a4280 0x55c4c4590d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.538+0000 7f9bdd63a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41c4f60 0x55c4c4589c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.538+0000 7f9bdd63a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c4c45881e0 con 0x55c4c419c0a0 2024-07-31T14:50:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.538+0000 7f9bdd63a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c4c4587b10 con 0x55c4c41c4f60 2024-07-31T14:50:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.538+0000 7f9bdd63a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c4c45888b0 con 0x55c4c41a4280 2024-07-31T14:50:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.538+0000 7f9bdd63a740 10 client.0 _ll_register_callbacks cb 0x7f9bc4003c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:50:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.538+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c419c0a0 0x55c4c4590720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.538+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c419c0a0 0x55c4c4590720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46290/0 (socket says 172.21.15.67:46290) 2024-07-31T14:50:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.538+0000 7f9bcffff640 1 -- 172.21.15.67:0/1941239328 learned_addr learned my addr 172.21.15.67:0/1941239328 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:50:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9bcffff640 1 --2- 172.21.15.67:0/1941239328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41c4f60 0x55c4c4589c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9bd4e85640 1 --2- 172.21.15.67:0/1941239328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c41a4280 0x55c4c4590d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9bcffff640 1 -- 172.21.15.67:0/1941239328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c41a4280 msgr2=0x55c4c4590d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9bcffff640 1 --2- 172.21.15.67:0/1941239328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c41a4280 0x55c4c4590d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9bcffff640 1 -- 172.21.15.67:0/1941239328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41c4f60 msgr2=0x55c4c4589c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:50:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9bcffff640 1 --2- 172.21.15.67:0/1941239328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41c4f60 0x55c4c4589c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9bcffff640 1 -- 172.21.15.67:0/1941239328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4c458a380 con 0x55c4c419c0a0 2024-07-31T14:50:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9bd4e85640 1 --2- 172.21.15.67:0/1941239328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c41a4280 0x55c4c4590d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:50:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9bcffff640 1 --2- 172.21.15.67:0/1941239328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c419c0a0 0x55c4c4590720 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4004310 tx=0x7f9bc400b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d903d63db3f2b2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:50:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9baffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:50:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9baffff640 1 -- 172.21.15.67:0/1941239328 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc401d460 con 0x55c4c419c0a0 2024-07-31T14:50:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9baffff640 1 -- 172.21.15.67:0/1941239328 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f9bc4015040 con 0x55c4c419c0a0 2024-07-31T14:50:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1941239328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c4c44de330 con 0x55c4c419c0a0 2024-07-31T14:50:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9baffff640 1 -- 172.21.15.67:0/1941239328 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9bc401d600 con 0x55c4c419c0a0 2024-07-31T14:50:58.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9bdd63a740 10 client.15456 fetch_fsmap learned FSMap version 29 2024-07-31T14:50:58.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.539+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1941239328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4c458fce0 con 0x55c4c419c0a0 2024-07-31T14:50:58.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.540+0000 7f9baffff640 1 -- 172.21.15.67:0/1941239328 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c4c44de330 con 0x55c4c419c0a0 2024-07-31T14:50:58.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.540+0000 7f9baffff640 1 -- 172.21.15.67:0/1941239328 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9bc4020640 con 0x55c4c419c0a0 2024-07-31T14:50:58.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.540+0000 7f9bdd63a740 10 client.15456 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:50:58.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.540+0000 7f9bdd63a740 10 client.15456 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-31T14:50:58.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.540+0000 7f9bdd63a740 20 client.15456 populate_metadata read hostname 'smithi067' 2024-07-31T14:50:58.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.540+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/1941239328 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c4c4158c90 0x55c4c45f4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:50:58.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.540+0000 7f9bdd63a740 4 client.15456 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:50:58.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.540+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1941239328 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "status"}) -- 0x55c4c44dba70 con 0x55c4c4158c90 2024-07-31T14:50:58.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.540+0000 7f9bd4e85640 1 --2- 172.21.15.67:0/1941239328 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c4c4158c90 0x55c4c45f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:50:58.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9bd4e85640 1 --2- 172.21.15.67:0/1941239328 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c4c4158c90 0x55c4c45f4700 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d47cf208dbfbfc0d server_cookie=8b8ad5a4e1778b60 in_seq=0 out_seq=0 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9baffff640 10 client.15456 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9baffff640 1 -- 172.21.15.67:0/1941239328 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c4c44dba70 con 0x55c4c4158c90 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9baffff640 10 client.15456 handle_command_reply: tid=0 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9bdd63a740 1 client.15456 shutdown 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1941239328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c419c0a0 msgr2=0x55c4c4590720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/1941239328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c419c0a0 0x55c4c4590720 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4004310 tx=0x7f9bc400b770 comp rx=0 tx=0).stop 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1941239328 shutdown_connections 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/1941239328 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c4c4158c90 0x55c4c45f4700 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/1941239328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4c41c4f60 0x55c4c4589c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/1941239328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4c41a4280 0x55c4c4590d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9bdd63a740 1 --2- 172.21.15.67:0/1941239328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4c419c0a0 0x55c4c4590720 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1941239328 >> 172.21.15.67:0/1941239328 conn(0x55c4c41b9350 msgr2=0x55c4c456d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9baffff640 0 client.15456 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1941239328 shutdown_connections 2024-07-31T14:50:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9bdd63a740 1 -- 172.21.15.67:0/1941239328 wait complete. 2024-07-31T14:50:58.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.541+0000 7f9bdd63a740 20 client.15456 trim_cache size 0 max 16384 2024-07-31T14:50:58.548 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:50:58.548 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:50:58.548 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 1, 2024-07-31T14:50:58.548 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24655, 2024-07-31T14:50:58.548 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:50:58.548 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:50:58.548 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:50:58.548 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 178.985118288, 2024-07-31T14:50:58.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-31T14:50:58.548 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:50:58.549 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:50:58.549 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 257.03695925300002 2024-07-31T14:50:58.549 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:50:58.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.542+0000 7f9bd70e9640 1 -- 172.21.15.67:0/2128157922 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9bb80b3cd0 msgr2=0x7f9bb80b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:58.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.542+0000 7f9bd70e9640 1 --2- 172.21.15.67:0/2128157922 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9bb80b3cd0 0x7f9bb80b61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80c80a0 tx=0x7f9bc8099770 comp rx=0 tx=0).stop 2024-07-31T14:50:58.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.542+0000 7f9bd70e9640 1 -- 172.21.15.67:0/2128157922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bd0108730 msgr2=0x7f9bd019b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:50:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.542+0000 7f9bd70e9640 1 --2- 172.21.15.67:0/2128157922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bd0108730 0x7f9bd019b770 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005cc0 tx=0x7f9bc802f380 comp rx=0 tx=0).stop 2024-07-31T14:50:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.542+0000 7f9bd70e9640 1 -- 172.21.15.67:0/2128157922 shutdown_connections 2024-07-31T14:50:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.542+0000 7f9bd70e9640 1 --2- 172.21.15.67:0/2128157922 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9bb80b3cd0 0x7f9bb80b61e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.542+0000 7f9bd70e9640 1 --2- 172.21.15.67:0/2128157922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bd0108730 0x7f9bd019b770 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.542+0000 7f9bd70e9640 1 --2- 172.21.15.67:0/2128157922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bd0107de0 0x7f9bd019b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.542+0000 7f9bd70e9640 1 --2- 172.21.15.67:0/2128157922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bd01024a0 0x7f9bd019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:50:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.542+0000 7f9bd70e9640 1 -- 172.21.15.67:0/2128157922 >> 172.21.15.67:0/2128157922 conn(0x7f9bd0079140 msgr2=0x7f9bd0105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:50:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.542+0000 7f9bd70e9640 1 -- 172.21.15.67:0/2128157922 shutdown_connections 2024-07-31T14:50:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:50:58.542+0000 7f9bd70e9640 1 -- 172.21.15.67:0/2128157922 wait complete. 2024-07-31T14:50:59.241 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:50:59 smithi067 ceph-mon[32750]: pgmap v396: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:59.272 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.b status 2024-07-31T14:50:59.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:50:59 smithi073 ceph-mon[36714]: pgmap v396: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:59.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:50:59 smithi077 ceph-mon[37327]: pgmap v396: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:50:59.540 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:00.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:00 smithi067 ceph-mon[32750]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:51:00.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:00 smithi073 ceph-mon[36714]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:51:00.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:00 smithi077 ceph-mon[37327]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:51:01.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.197+0000 7fc348052640 1 -- 172.21.15.67:0/2666332405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3401035d0 msgr2=0x7fc3401039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:01.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.197+0000 7fc348052640 1 --2- 172.21.15.67:0/2666332405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3401035d0 0x7fc3401039e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc330009ca0 tx=0x7fc33002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.197+0000 7fc348052640 1 -- 172.21.15.67:0/2666332405 shutdown_connections 2024-07-31T14:51:01.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.197+0000 7fc348052640 1 --2- 172.21.15.67:0/2666332405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc340109860 0x7fc340110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.197+0000 7fc348052640 1 --2- 172.21.15.67:0/2666332405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc340108f10 0x7fc340109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.197+0000 7fc348052640 1 --2- 172.21.15.67:0/2666332405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3401035d0 0x7fc3401039e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.197+0000 7fc348052640 1 -- 172.21.15.67:0/2666332405 >> 172.21.15.67:0/2666332405 conn(0x7fc3400ff060 msgr2=0x7fc3401014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.198+0000 7fc348052640 1 -- 172.21.15.67:0/2666332405 shutdown_connections 2024-07-31T14:51:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.198+0000 7fc348052640 1 -- 172.21.15.67:0/2666332405 wait complete. 2024-07-31T14:51:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.198+0000 7fc348052640 1 Processor -- start 2024-07-31T14:51:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.198+0000 7fc348052640 1 -- start start 2024-07-31T14:51:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc348052640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3401035d0 0x7fc340075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc348052640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc340108f10 0x7fc340075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc348052640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc340109860 0x7fc340075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc348052640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc340113530 con 0x7fc3401035d0 2024-07-31T14:51:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc348052640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc3401133b0 con 0x7fc340108f10 2024-07-31T14:51:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc348052640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc3401136b0 con 0x7fc340109860 2024-07-31T14:51:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc345dee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3401035d0 0x7fc340075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc345dee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3401035d0 0x7fc340075140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33730/0 (socket says 172.21.15.67:33730) 2024-07-31T14:51:01.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc345dee640 1 -- 172.21.15.67:0/1342183716 learned_addr learned my addr 172.21.15.67:0/1342183716 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:01.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc3455ed640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc340108f10 0x7fc340075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc3465ef640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc340109860 0x7fc340075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc345dee640 1 -- 172.21.15.67:0/1342183716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc340109860 msgr2=0x7fc340075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:01.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc345dee640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc340109860 0x7fc340075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc345dee640 1 -- 172.21.15.67:0/1342183716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc340108f10 msgr2=0x7fc340075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:01.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc345dee640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc340108f10 0x7fc340075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc345dee640 1 -- 172.21.15.67:0/1342183716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3401a81c0 con 0x7fc3401035d0 2024-07-31T14:51:01.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc3465ef640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc340109860 0x7fc340075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:51:01.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.199+0000 7fc345dee640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3401035d0 0x7fc340075140 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc330006010 tx=0x7fc3300319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:01.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.200+0000 7fc33effd640 1 -- 172.21.15.67:0/1342183716 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc330003bf0 con 0x7fc3401035d0 2024-07-31T14:51:01.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.200+0000 7fc33effd640 1 -- 172.21.15.67:0/1342183716 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc330003d90 con 0x7fc3401035d0 2024-07-31T14:51:01.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.200+0000 7fc348052640 1 -- 172.21.15.67:0/1342183716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3401a8450 con 0x7fc3401035d0 2024-07-31T14:51:01.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.200+0000 7fc33effd640 1 -- 172.21.15.67:0/1342183716 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33002fc20 con 0x7fc3401035d0 2024-07-31T14:51:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.200+0000 7fc348052640 1 -- 172.21.15.67:0/1342183716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3401a8990 con 0x7fc3401035d0 2024-07-31T14:51:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.203+0000 7fc34e5a3740 1 Processor -- start 2024-07-31T14:51:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.203+0000 7fc34e5a3740 1 -- start start 2024-07-31T14:51:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.203+0000 7fc33effd640 1 -- 172.21.15.67:0/1342183716 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fc3300046a0 con 0x7fc3401035d0 2024-07-31T14:51:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.204+0000 7fc34e5a3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab8cf60 0x55592af323a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.204+0000 7fc34e5a3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af328e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.204+0000 7fc34e5a3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab6c280 0x55592af32e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.204+0000 7fc34e5a3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55592aea8940 con 0x55592ab640a0 2024-07-31T14:51:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.204+0000 7fc34e5a3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55592ae97e40 con 0x55592ab6c280 2024-07-31T14:51:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.204+0000 7fc34e5a3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55592ae9be80 con 0x55592ab8cf60 2024-07-31T14:51:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.204+0000 7fc345dee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.204+0000 7fc345dee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af328e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33744/0 (socket says 172.21.15.67:33744) 2024-07-31T14:51:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.204+0000 7fc345dee640 1 -- 172.21.15.67:0/2181359471 learned_addr learned my addr 172.21.15.67:0/2181359471 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.204+0000 7fc3465ef640 1 --2- 172.21.15.67:0/2181359471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab8cf60 0x55592af323a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.204+0000 7fc3455ed640 1 --2- 172.21.15.67:0/2181359471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab6c280 0x55592af32e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.205+0000 7fc345dee640 1 -- 172.21.15.67:0/2181359471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab8cf60 msgr2=0x55592af323a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.205+0000 7fc345dee640 1 --2- 172.21.15.67:0/2181359471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab8cf60 0x55592af323a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.205+0000 7fc345dee640 1 -- 172.21.15.67:0/2181359471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab6c280 msgr2=0x55592af32e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.205+0000 7fc345dee640 1 --2- 172.21.15.67:0/2181359471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab6c280 0x55592af32e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.205+0000 7fc345dee640 1 -- 172.21.15.67:0/2181359471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55592ae3ea60 con 0x55592ab640a0 2024-07-31T14:51:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.205+0000 7fc33effd640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3280b3c20 0x7fc3280b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.205+0000 7fc345dee640 1 --2- 172.21.15.67:0/2181359471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af328e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc33010ba80 tx=0x7fc33010bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c109f56de58550f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.205+0000 7fc3465ef640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3280b3c20 0x7fc3280b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.205+0000 7fc327fff640 1 -- 172.21.15.67:0/2181359471 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc330114020 con 0x55592ab640a0 2024-07-31T14:51:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.205+0000 7fc33effd640 1 -- 172.21.15.67:0/1342183716 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc3301056d0 con 0x7fc3401035d0 2024-07-31T14:51:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.205+0000 7fc327fff640 1 -- 172.21.15.67:0/2181359471 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3300cbd00 con 0x55592ab640a0 2024-07-31T14:51:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.205+0000 7fc327fff640 1 -- 172.21.15.67:0/2181359471 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3300cbfc0 con 0x55592ab640a0 2024-07-31T14:51:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.206+0000 7fc34e5a3740 1 -- 172.21.15.67:0/2181359471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 msgr2=0x55592af328e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.206+0000 7fc3465ef640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3280b3c20 0x7fc3280b6130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc338005fd0 tx=0x7fc338009420 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.206+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/2181359471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af328e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc33010ba80 tx=0x7fc33010bb70 comp rx=0 tx=0).stop 2024-07-31T14:51:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.206+0000 7fc34e5a3740 1 -- 172.21.15.67:0/2181359471 shutdown_connections 2024-07-31T14:51:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.206+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/2181359471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab6c280 0x55592af32e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.206+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/2181359471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af328e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.206+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/2181359471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab8cf60 0x55592af323a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.206+0000 7fc34e5a3740 1 -- 172.21.15.67:0/2181359471 >> 172.21.15.67:0/2181359471 conn(0x55592ab81350 msgr2=0x55592af30290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.206+0000 7fc34e5a3740 1 -- 172.21.15.67:0/2181359471 shutdown_connections 2024-07-31T14:51:01.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.206+0000 7fc34e5a3740 1 -- 172.21.15.67:0/2181359471 wait complete. 2024-07-31T14:51:01.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.207+0000 7fc34e5a3740 1 -- start start 2024-07-31T14:51:01.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.207+0000 7fc34e5a3740 1 Processor -- start 2024-07-31T14:51:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc34e5a3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab640a0 0x55592af3dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc34e5a3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab6c280 0x55592af3e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc34e5a3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab8cf60 0x55592af49a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc34e5a3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55592acfb5f0 con 0x55592ab8cf60 2024-07-31T14:51:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc34e5a3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55592ae3a8b0 con 0x55592ab640a0 2024-07-31T14:51:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc34e5a3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55592aea8940 con 0x55592ab6c280 2024-07-31T14:51:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc34e5a3740 10 client.0 _ll_register_callbacks cb 0x55592ae447f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc345dee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab640a0 0x55592af3dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc345dee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab640a0 0x55592af3dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:38920/0 (socket says 172.21.15.67:38920) 2024-07-31T14:51:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc345dee640 1 -- 172.21.15.67:0/3476571986 learned_addr learned my addr 172.21.15.67:0/3476571986 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc3465ef640 1 --2- 172.21.15.67:0/3476571986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab8cf60 0x55592af49a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc3455ed640 1 --2- 172.21.15.67:0/3476571986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab6c280 0x55592af3e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc345dee640 1 -- 172.21.15.67:0/3476571986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab6c280 msgr2=0x55592af3e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc345dee640 1 --2- 172.21.15.67:0/3476571986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab6c280 0x55592af3e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc345dee640 1 -- 172.21.15.67:0/3476571986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab8cf60 msgr2=0x55592af49a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc345dee640 1 --2- 172.21.15.67:0/3476571986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab8cf60 0x55592af49a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc345dee640 1 -- 172.21.15.67:0/3476571986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55592abf8540 con 0x55592ab640a0 2024-07-31T14:51:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.208+0000 7fc3465ef640 1 --2- 172.21.15.67:0/3476571986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab8cf60 0x55592af49a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.209+0000 7fc345dee640 1 --2- 172.21.15.67:0/3476571986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab640a0 0x55592af3dee0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc330102df0 tx=0x7fc3300095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bde4470734582ea4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.209+0000 7fc3257fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.209+0000 7fc3257fa640 1 -- 172.21.15.67:0/3476571986 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc330114070 con 0x55592ab640a0 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.209+0000 7fc3257fa640 1 -- 172.21.15.67:0/3476571986 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc3300cc450 con 0x55592ab640a0 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.209+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3476571986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3300cbd00 con 0x55592ab640a0 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.209+0000 7fc3257fa640 1 -- 172.21.15.67:0/3476571986 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc33003d040 con 0x55592ab640a0 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.209+0000 7fc34e5a3740 10 client.25205 fetch_fsmap learned FSMap version 29 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.209+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3476571986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55592af4c6c0 con 0x55592ab640a0 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.209+0000 7fc3257fa640 1 -- 172.21.15.67:0/3476571986 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3300cbd00 con 0x55592ab640a0 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.209+0000 7fc3257fa640 1 -- 172.21.15.67:0/3476571986 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc330105020 con 0x55592ab640a0 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.209+0000 7fc34e5a3740 10 client.25205 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.209+0000 7fc34e5a3740 10 client.25205 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.209+0000 7fc34e5a3740 20 client.25205 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.210+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/3476571986 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55592ab4b8e0 0x55592af4fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.210+0000 7fc34e5a3740 4 client.25205 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:51:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.210+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3476571986 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55592aeaa890 con 0x55592ab4b8e0 2024-07-31T14:51:01.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.210+0000 7fc3455ed640 1 --2- 172.21.15.67:0/3476571986 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55592ab4b8e0 0x55592af4fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.210+0000 7fc3257fa640 10 client.25205 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:01.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.210+0000 7fc3455ed640 1 --2- 172.21.15.67:0/3476571986 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55592ab4b8e0 0x55592af4fe70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77fbda2c8f7399ae server_cookie=3954eb7107079fc5 in_seq=0 out_seq=0 2024-07-31T14:51:01.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc3257fa640 1 -- 172.21.15.67:0/3476571986 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55592aeaa890 con 0x55592ab4b8e0 2024-07-31T14:51:01.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc3257fa640 10 client.25205 handle_command_reply: tid=0 2024-07-31T14:51:01.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc34e5a3740 1 client.25205 shutdown 2024-07-31T14:51:01.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3476571986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab640a0 msgr2=0x55592af3dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:01.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/3476571986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab640a0 0x55592af3dee0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc330102df0 tx=0x7fc3300095c0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3476571986 shutdown_connections 2024-07-31T14:51:01.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/3476571986 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55592ab4b8e0 0x55592af4fe70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/3476571986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab8cf60 0x55592af49a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/3476571986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab6c280 0x55592af3e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/3476571986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab640a0 0x55592af3dee0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc3257fa640 0 client.25205 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3476571986 >> 172.21.15.67:0/3476571986 conn(0x55592ab81350 msgr2=0x55592af35530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3476571986 shutdown_connections 2024-07-31T14:51:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3476571986 wait complete. 2024-07-31T14:51:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.211+0000 7fc34e5a3740 20 client.25205 trim_cache size 0 max 16384 2024-07-31T14:51:01.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.230+0000 7fc34e5a3740 1 Processor -- start 2024-07-31T14:51:01.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.230+0000 7fc34e5a3740 1 -- start start 2024-07-31T14:51:01.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc34e5a3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af49a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc34e5a3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab6c280 0x55592af49f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc34e5a3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab8cf60 0x55592af3c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc34e5a3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55592af4e1b0 con 0x55592ab640a0 2024-07-31T14:51:01.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc34e5a3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55592af4e880 con 0x55592ab6c280 2024-07-31T14:51:01.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc34e5a3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55592af4ef50 con 0x55592ab8cf60 2024-07-31T14:51:01.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc345dee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af49a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc345dee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af49a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33762/0 (socket says 172.21.15.67:33762) 2024-07-31T14:51:01.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc345dee640 1 -- 172.21.15.67:0/2848678487 learned_addr learned my addr 172.21.15.67:0/2848678487 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:01.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc3465ef640 1 --2- 172.21.15.67:0/2848678487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab8cf60 0x55592af3c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc3455ed640 1 --2- 172.21.15.67:0/2848678487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab6c280 0x55592af49f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc345dee640 1 -- 172.21.15.67:0/2848678487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab8cf60 msgr2=0x55592af3c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:01.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc345dee640 1 --2- 172.21.15.67:0/2848678487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab8cf60 0x55592af3c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc345dee640 1 -- 172.21.15.67:0/2848678487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab6c280 msgr2=0x55592af49f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:01.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc345dee640 1 --2- 172.21.15.67:0/2848678487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab6c280 0x55592af49f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc345dee640 1 -- 172.21.15.67:0/2848678487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55592af35200 con 0x55592ab640a0 2024-07-31T14:51:01.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc3465ef640 1 --2- 172.21.15.67:0/2848678487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab8cf60 0x55592af3c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:01.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc345dee640 1 --2- 172.21.15.67:0/2848678487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af49a20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc330004af0 tx=0x7fc330004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b066c4bf97ee9a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:01.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.231+0000 7fc3277fe640 1 -- 172.21.15.67:0/2848678487 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3300cb960 con 0x55592ab640a0 2024-07-31T14:51:01.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.232+0000 7fc3277fe640 1 -- 172.21.15.67:0/2848678487 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc330036030 con 0x55592ab640a0 2024-07-31T14:51:01.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.232+0000 7fc3277fe640 1 -- 172.21.15.67:0/2848678487 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3300cbb40 con 0x55592ab640a0 2024-07-31T14:51:01.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.232+0000 7fc34e5a3740 1 -- 172.21.15.67:0/2848678487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 msgr2=0x55592af49a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:01.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.232+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/2848678487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af49a20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc330004af0 tx=0x7fc330004990 comp rx=0 tx=0).stop 2024-07-31T14:51:01.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.232+0000 7fc34e5a3740 1 -- 172.21.15.67:0/2848678487 shutdown_connections 2024-07-31T14:51:01.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.232+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/2848678487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab8cf60 0x55592af3c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.232+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/2848678487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab6c280 0x55592af49f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.232+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/2848678487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af49a20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.232+0000 7fc34e5a3740 1 -- 172.21.15.67:0/2848678487 >> 172.21.15.67:0/2848678487 conn(0x55592ab81350 msgr2=0x55592af3dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.232+0000 7fc34e5a3740 1 -- 172.21.15.67:0/2848678487 shutdown_connections 2024-07-31T14:51:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.232+0000 7fc34e5a3740 1 -- 172.21.15.67:0/2848678487 wait complete. 2024-07-31T14:51:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.233+0000 7fc34e5a3740 1 -- start start 2024-07-31T14:51:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.233+0000 7fc34e5a3740 1 Processor -- start 2024-07-31T14:51:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.233+0000 7fc34e5a3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af583a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.233+0000 7fc34e5a3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab6c280 0x55592af58980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.233+0000 7fc34e5a3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab8cf60 0x55592af52ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.233+0000 7fc34e5a3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55592af4fcf0 con 0x55592ab640a0 2024-07-31T14:51:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.233+0000 7fc34e5a3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55592af4f620 con 0x55592ab8cf60 2024-07-31T14:51:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.233+0000 7fc34e5a3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55592af503c0 con 0x55592ab6c280 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.233+0000 7fc34e5a3740 10 client.0 _ll_register_callbacks cb 0x7fc33010b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.233+0000 7fc345dee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.233+0000 7fc345dee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33772/0 (socket says 172.21.15.67:33772) 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.233+0000 7fc345dee640 1 -- 172.21.15.67:0/3721328905 learned_addr learned my addr 172.21.15.67:0/3721328905 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc3465ef640 1 --2- 172.21.15.67:0/3721328905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab8cf60 0x55592af52ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc3455ed640 1 --2- 172.21.15.67:0/3721328905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab6c280 0x55592af58980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc345dee640 1 -- 172.21.15.67:0/3721328905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab6c280 msgr2=0x55592af58980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc345dee640 1 --2- 172.21.15.67:0/3721328905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab6c280 0x55592af58980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc345dee640 1 -- 172.21.15.67:0/3721328905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab8cf60 msgr2=0x55592af52ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc345dee640 1 --2- 172.21.15.67:0/3721328905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab8cf60 0x55592af52ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc345dee640 1 -- 172.21.15.67:0/3721328905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55592af53660 con 0x55592ab640a0 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc345dee640 1 --2- 172.21.15.67:0/3721328905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af583a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc3300d52f0 tx=0x7fc33010bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a64c1569035ea2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc3257fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc3257fa640 1 -- 172.21.15.67:0/3721328905 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc330114060 con 0x55592ab640a0 2024-07-31T14:51:01.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc3257fa640 1 -- 172.21.15.67:0/3721328905 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc33010b320 con 0x55592ab640a0 2024-07-31T14:51:01.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3721328905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc330036030 con 0x55592ab640a0 2024-07-31T14:51:01.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc3257fa640 1 -- 172.21.15.67:0/3721328905 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc3300cbb40 con 0x55592ab640a0 2024-07-31T14:51:01.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc34e5a3740 10 client.15474 fetch_fsmap learned FSMap version 29 2024-07-31T14:51:01.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.234+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3721328905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55592af53660 con 0x55592ab640a0 2024-07-31T14:51:01.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.235+0000 7fc3257fa640 1 -- 172.21.15.67:0/3721328905 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc330036030 con 0x55592ab640a0 2024-07-31T14:51:01.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.235+0000 7fc3257fa640 1 -- 172.21.15.67:0/3721328905 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc330105040 con 0x55592ab640a0 2024-07-31T14:51:01.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.235+0000 7fc34e5a3740 10 client.15474 fetch_fsmap finished waiting for FSMap version 29 2024-07-31T14:51:01.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.235+0000 7fc34e5a3740 10 client.15474 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-31T14:51:01.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.235+0000 7fc34e5a3740 20 client.15474 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:01.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.235+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/3721328905 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55592ab20c90 0x55592afbbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:01.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.235+0000 7fc34e5a3740 4 client.15474 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:51:01.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.235+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3721328905 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "status"}) -- 0x55592aea6330 con 0x55592ab20c90 2024-07-31T14:51:01.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.235+0000 7fc3455ed640 1 --2- 172.21.15.67:0/3721328905 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55592ab20c90 0x55592afbbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:01.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc3455ed640 1 --2- 172.21.15.67:0/3721328905 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55592ab20c90 0x55592afbbcb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e0d0a28fbf9a3fb server_cookie=e0fc21048dca91b7 in_seq=0 out_seq=0 2024-07-31T14:51:01.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc3257fa640 10 client.15474 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:01.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc3257fa640 1 -- 172.21.15.67:0/3721328905 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55592aea6330 con 0x55592ab20c90 2024-07-31T14:51:01.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc3257fa640 10 client.15474 handle_command_reply: tid=0 2024-07-31T14:51:01.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc34e5a3740 1 client.15474 shutdown 2024-07-31T14:51:01.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3721328905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 msgr2=0x55592af583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:01.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/3721328905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af583a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc3300d52f0 tx=0x7fc33010bb70 comp rx=0 tx=0).stop 2024-07-31T14:51:01.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3721328905 shutdown_connections 2024-07-31T14:51:01.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/3721328905 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55592ab20c90 0x55592afbbcb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/3721328905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55592ab8cf60 0x55592af52ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/3721328905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55592ab6c280 0x55592af58980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc34e5a3740 1 --2- 172.21.15.67:0/3721328905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55592ab640a0 0x55592af583a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3721328905 >> 172.21.15.67:0/3721328905 conn(0x55592ab81350 msgr2=0x55592af3c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:01.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc3257fa640 0 client.15474 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:01.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3721328905 shutdown_connections 2024-07-31T14:51:01.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc34e5a3740 1 -- 172.21.15.67:0/3721328905 wait complete. 2024-07-31T14:51:01.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.236+0000 7fc34e5a3740 20 client.15474 trim_cache size 0 max 16384 2024-07-31T14:51:01.243 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:51:01.243 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:51:01.243 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-07-31T14:51:01.243 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24671, 2024-07-31T14:51:01.243 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:51:01.243 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:51:01.243 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:51:01.244 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 236.19615226400001, 2024-07-31T14:51:01.244 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-31T14:51:01.244 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:51:01.244 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:51:01.244 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 250.40768933499999 2024-07-31T14:51:01.244 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:51:01.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.237+0000 7fc348052640 1 -- 172.21.15.67:0/1342183716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3280b3c20 msgr2=0x7fc3280b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.237+0000 7fc348052640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3280b3c20 0x7fc3280b6130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc338005fd0 tx=0x7fc338009420 comp rx=0 tx=0).stop 2024-07-31T14:51:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.237+0000 7fc348052640 1 -- 172.21.15.67:0/1342183716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3401035d0 msgr2=0x7fc340075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.237+0000 7fc348052640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3401035d0 0x7fc340075140 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc330006010 tx=0x7fc3300319f0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.237+0000 7fc348052640 1 -- 172.21.15.67:0/1342183716 shutdown_connections 2024-07-31T14:51:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.237+0000 7fc348052640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3280b3c20 0x7fc3280b6130 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.237+0000 7fc348052640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc340109860 0x7fc340075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.237+0000 7fc348052640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc340108f10 0x7fc340075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.237+0000 7fc348052640 1 --2- 172.21.15.67:0/1342183716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3401035d0 0x7fc340075140 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.237+0000 7fc348052640 1 -- 172.21.15.67:0/1342183716 >> 172.21.15.67:0/1342183716 conn(0x7fc3400ff060 msgr2=0x7fc340101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.237+0000 7fc348052640 1 -- 172.21.15.67:0/1342183716 shutdown_connections 2024-07-31T14:51:01.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:01.237+0000 7fc348052640 1 -- 172.21.15.67:0/1342183716 wait complete. 2024-07-31T14:51:01.948 INFO:teuthology.run_tasks:Running task sequential... 2024-07-31T14:51:01.959 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-07-31T14:51:01.959 INFO:teuthology.task.sequential:In sequential, running task print... 2024-07-31T14:51:01.959 INFO:teuthology.task.print:Enabling mgr modules 2024-07-31T14:51:01.960 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-07-31T14:51:01.960 INFO:teuthology.task.exec:Executing custom commands... 2024-07-31T14:51:01.960 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-07-31T14:51:01.961 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-07-31T14:51:02.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.077+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/3284653642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbe410d7e0 msgr2=0x7ffbe410fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:02.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.077+0000 7ffbe9ec2640 1 --2- 172.21.15.67:0/3284653642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbe410d7e0 0x7ffbe410fcc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009ca0 tx=0x7ffbdc02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:51:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.077+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/3284653642 shutdown_connections 2024-07-31T14:51:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.077+0000 7ffbe9ec2640 1 --2- 172.21.15.67:0/3284653642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbe410d7e0 0x7ffbe410fcc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.077+0000 7ffbe9ec2640 1 --2- 172.21.15.67:0/3284653642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbe4104d10 0x7ffbe410d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.077+0000 7ffbe9ec2640 1 --2- 172.21.15.67:0/3284653642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbe41043c0 0x7ffbe41047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.077+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/3284653642 >> 172.21.15.67:0/3284653642 conn(0x7ffbe407a140 msgr2=0x7ffbe407a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.077+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/3284653642 shutdown_connections 2024-07-31T14:51:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.077+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/3284653642 wait complete. 2024-07-31T14:51:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.077+0000 7ffbe9ec2640 1 Processor -- start 2024-07-31T14:51:02.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe9ec2640 1 -- start start 2024-07-31T14:51:02.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe9ec2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbe41043c0 0x7ffbe4199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:02.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe9ec2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbe4104d10 0x7ffbe419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:02.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe9ec2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbe410d7e0 0x7ffbe419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:02.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe9ec2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe4113450 con 0x7ffbe41043c0 2024-07-31T14:51:02.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe9ec2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41132d0 con 0x7ffbe410d7e0 2024-07-31T14:51:02.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe9ec2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41135d0 con 0x7ffbe4104d10 2024-07-31T14:51:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbe41043c0 0x7ffbe4199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbe41043c0 0x7ffbe4199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33788/0 (socket says 172.21.15.67:33788) 2024-07-31T14:51:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe37fe640 1 -- 172.21.15.67:0/2815251081 learned_addr learned my addr 172.21.15.67:0/2815251081 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe2ffd640 1 --2- 172.21.15.67:0/2815251081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbe4104d10 0x7ffbe419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe3fff640 1 --2- 172.21.15.67:0/2815251081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbe410d7e0 0x7ffbe419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe37fe640 1 -- 172.21.15.67:0/2815251081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbe4104d10 msgr2=0x7ffbe419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe37fe640 1 --2- 172.21.15.67:0/2815251081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbe4104d10 0x7ffbe419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe37fe640 1 -- 172.21.15.67:0/2815251081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbe410d7e0 msgr2=0x7ffbe419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe37fe640 1 --2- 172.21.15.67:0/2815251081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbe410d7e0 0x7ffbe419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.078+0000 7ffbe37fe640 1 -- 172.21.15.67:0/2815251081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe419fd60 con 0x7ffbe41043c0 2024-07-31T14:51:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.079+0000 7ffbe37fe640 1 --2- 172.21.15.67:0/2815251081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbe41043c0 0x7ffbe4199e30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400d950 tx=0x7ffbd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.079+0000 7ffbe0ff9640 1 -- 172.21.15.67:0/2815251081 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4017020 con 0x7ffbe41043c0 2024-07-31T14:51:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.079+0000 7ffbe0ff9640 1 -- 172.21.15.67:0/2815251081 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbd4009c30 con 0x7ffbe41043c0 2024-07-31T14:51:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.079+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/2815251081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe41a0050 con 0x7ffbe41043c0 2024-07-31T14:51:02.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.079+0000 7ffbe0ff9640 1 -- 172.21.15.67:0/2815251081 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd400f6b0 con 0x7ffbe41043c0 2024-07-31T14:51:02.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.079+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/2815251081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe41a67e0 con 0x7ffbe41043c0 2024-07-31T14:51:02.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.080+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/2815251081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe406ba20 con 0x7ffbe41043c0 2024-07-31T14:51:02.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.083+0000 7ffbe0ff9640 1 -- 172.21.15.67:0/2815251081 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7ffbd401a020 con 0x7ffbe41043c0 2024-07-31T14:51:02.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.083+0000 7ffbe0ff9640 1 --2- 172.21.15.67:0/2815251081 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffbcc0b3c20 0x7ffbcc0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:02.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.084+0000 7ffbe0ff9640 1 -- 172.21.15.67:0/2815251081 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ffbd40e4120 con 0x7ffbe41043c0 2024-07-31T14:51:02.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.084+0000 7ffbe2ffd640 1 --2- 172.21.15.67:0/2815251081 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffbcc0b3c20 0x7ffbcc0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:02.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.084+0000 7ffbe2ffd640 1 --2- 172.21.15.67:0/2815251081 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffbcc0b3c20 0x7ffbcc0b6130 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffbe410f400 tx=0x7ffbd8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:02.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.087+0000 7ffbe0ff9640 1 -- 172.21.15.67:0/2815251081 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ffbd4009de0 con 0x7ffbe41043c0 2024-07-31T14:51:02.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.332+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/2815251081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7ffbe419af50 con 0x7ffbe41043c0 2024-07-31T14:51:02.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.498+0000 7ffbe0ff9640 1 -- 172.21.15.67:0/2815251081 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 module 'snap_schedule' is already enabled v23) ==== 133+0+0 (secure 0 0 0) 0x7ffbd40ab340 con 0x7ffbe41043c0 2024-07-31T14:51:02.500 INFO:teuthology.orchestra.run.smithi067.stderr:module 'snap_schedule' is already enabled 2024-07-31T14:51:02.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.501+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/2815251081 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffbcc0b3c20 msgr2=0x7ffbcc0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:02.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.501+0000 7ffbe9ec2640 1 --2- 172.21.15.67:0/2815251081 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffbcc0b3c20 0x7ffbcc0b6130 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffbe410f400 tx=0x7ffbd8005f90 comp rx=0 tx=0).stop 2024-07-31T14:51:02.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.501+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/2815251081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbe41043c0 msgr2=0x7ffbe4199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:02.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.501+0000 7ffbe9ec2640 1 --2- 172.21.15.67:0/2815251081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbe41043c0 0x7ffbe4199e30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400d950 tx=0x7ffbd400de10 comp rx=0 tx=0).stop 2024-07-31T14:51:02.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.502+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/2815251081 shutdown_connections 2024-07-31T14:51:02.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.502+0000 7ffbe9ec2640 1 --2- 172.21.15.67:0/2815251081 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffbcc0b3c20 0x7ffbcc0b6130 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:02.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.502+0000 7ffbe9ec2640 1 --2- 172.21.15.67:0/2815251081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbe410d7e0 0x7ffbe419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:02.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.502+0000 7ffbe9ec2640 1 --2- 172.21.15.67:0/2815251081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbe4104d10 0x7ffbe419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:02.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.502+0000 7ffbe9ec2640 1 --2- 172.21.15.67:0/2815251081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbe41043c0 0x7ffbe4199e30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:02.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.502+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/2815251081 >> 172.21.15.67:0/2815251081 conn(0x7ffbe407a140 msgr2=0x7ffbe41072a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:02.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.502+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/2815251081 shutdown_connections 2024-07-31T14:51:02.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:02.503+0000 7ffbe9ec2640 1 -- 172.21.15.67:0/2815251081 wait complete. 2024-07-31T14:51:02.521 INFO:teuthology.run_tasks:Running task kclient... 2024-07-31T14:51:02.534 INFO:tasks.kclient:Mounting kernel clients... 2024-07-31T14:51:02.534 INFO:tasks.kclient:config is {'client.0': {}} 2024-07-31T14:51:02.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:51:02.534 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-07-31T14:51:02.550 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-07-31T14:51:02.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:51:02.551 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-31T14:51:02.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:51:02.617 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-07-31T14:51:02.711 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-07-31T14:51:02.712 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-31T14:51:02.712 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-31T14:51:02.712 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs ls 2024-07-31T14:51:02.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:02 smithi067 ceph-mon[32750]: pgmap v398: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:51:02.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:02 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2815251081' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-31T14:51:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:02 smithi073 ceph-mon[36714]: pgmap v398: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:51:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:02 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2815251081' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-31T14:51:02.979 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:02.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:02 smithi077 ceph-mon[37327]: pgmap v398: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-31T14:51:02.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:02 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2815251081' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-31T14:51:03.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:03 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2815251081' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-31T14:51:03.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:03 smithi067 ceph-mon[32750]: mgrmap e23: x(active, since 10m), standbys: z, y 2024-07-31T14:51:03.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:03 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2815251081' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-31T14:51:03.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:03 smithi073 ceph-mon[36714]: mgrmap e23: x(active, since 10m), standbys: z, y 2024-07-31T14:51:03.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:03 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2815251081' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-31T14:51:03.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:03 smithi077 ceph-mon[37327]: mgrmap e23: x(active, since 10m), standbys: z, y 2024-07-31T14:51:04.725 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:04 smithi067 ceph-mon[32750]: pgmap v399: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:51:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.819+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2169793357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8ac108ef0 msgr2=0x7fc8ac109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.819+0000 7fc8b16d8640 1 --2- 172.21.15.67:0/2169793357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8ac108ef0 0x7fc8ac109300 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc894009900 tx=0x7fc89402a410 comp rx=0 tx=0).stop 2024-07-31T14:51:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.819+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2169793357 shutdown_connections 2024-07-31T14:51:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.819+0000 7fc8b16d8640 1 --2- 172.21.15.67:0/2169793357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ac109840 0x7fc8ac110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.819+0000 7fc8b16d8640 1 --2- 172.21.15.67:0/2169793357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8ac108ef0 0x7fc8ac109300 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.819+0000 7fc8b16d8640 1 --2- 172.21.15.67:0/2169793357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ac1035b0 0x7fc8ac1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.819+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2169793357 >> 172.21.15.67:0/2169793357 conn(0x7fc8ac0ff000 msgr2=0x7fc8ac101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.819+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2169793357 shutdown_connections 2024-07-31T14:51:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.819+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2169793357 wait complete. 2024-07-31T14:51:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.820+0000 7fc8b16d8640 1 Processor -- start 2024-07-31T14:51:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.820+0000 7fc8b16d8640 1 -- start start 2024-07-31T14:51:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8b16d8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ac1035b0 0x7fc8ac075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8b16d8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8ac108ef0 0x7fc8ac075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8b16d8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ac109840 0x7fc8ac075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8b16d8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac1134d0 con 0x7fc8ac108ef0 2024-07-31T14:51:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8b16d8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac113350 con 0x7fc8ac1035b0 2024-07-31T14:51:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8b16d8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac113650 con 0x7fc8ac109840 2024-07-31T14:51:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8aa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8ac108ef0 0x7fc8ac075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8aa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8ac108ef0 0x7fc8ac075680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33816/0 (socket says 172.21.15.67:33816) 2024-07-31T14:51:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8aa7fc640 1 -- 172.21.15.67:0/2220827195 learned_addr learned my addr 172.21.15.67:0/2220827195 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8aaffd640 1 --2- 172.21.15.67:0/2220827195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ac1035b0 0x7fc8ac075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8ab7fe640 1 --2- 172.21.15.67:0/2220827195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ac109840 0x7fc8ac075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8aa7fc640 1 -- 172.21.15.67:0/2220827195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ac109840 msgr2=0x7fc8ac075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8aa7fc640 1 --2- 172.21.15.67:0/2220827195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ac109840 0x7fc8ac075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8aa7fc640 1 -- 172.21.15.67:0/2220827195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ac1035b0 msgr2=0x7fc8ac075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8aa7fc640 1 --2- 172.21.15.67:0/2220827195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ac1035b0 0x7fc8ac075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.821+0000 7fc8aa7fc640 1 -- 172.21.15.67:0/2220827195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ac1a81a0 con 0x7fc8ac108ef0 2024-07-31T14:51:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.822+0000 7fc8aa7fc640 1 --2- 172.21.15.67:0/2220827195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8ac108ef0 0x7fc8ac075680 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc89402a930 tx=0x7fc894031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:04.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.822+0000 7fc8a3fff640 1 -- 172.21.15.67:0/2220827195 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89403f030 con 0x7fc8ac108ef0 2024-07-31T14:51:04.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.822+0000 7fc8a3fff640 1 -- 172.21.15.67:0/2220827195 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc894004490 con 0x7fc8ac108ef0 2024-07-31T14:51:04.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.822+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2220827195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ac1a8430 con 0x7fc8ac108ef0 2024-07-31T14:51:04.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.822+0000 7fc8a3fff640 1 -- 172.21.15.67:0/2220827195 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894035070 con 0x7fc8ac108ef0 2024-07-31T14:51:04.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.822+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2220827195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ac1a8910 con 0x7fc8ac108ef0 2024-07-31T14:51:04.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.823+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2220827195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ac1039c0 con 0x7fc8ac108ef0 2024-07-31T14:51:04.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.824+0000 7fc8a3fff640 1 -- 172.21.15.67:0/2220827195 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7fc894042030 con 0x7fc8ac108ef0 2024-07-31T14:51:04.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.826+0000 7fc8a3fff640 1 --2- 172.21.15.67:0/2220827195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc8980b3c20 0x7fc8980b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:04.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.826+0000 7fc8aaffd640 1 --2- 172.21.15.67:0/2220827195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc8980b3c20 0x7fc8980b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:04.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.826+0000 7fc8a3fff640 1 -- 172.21.15.67:0/2220827195 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc8941047d0 con 0x7fc8ac108ef0 2024-07-31T14:51:04.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.826+0000 7fc8aaffd640 1 --2- 172.21.15.67:0/2220827195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc8980b3c20 0x7fc8980b6130 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac076b90 tx=0x7fc89c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:04.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:04.829+0000 7fc8a3fff640 1 -- 172.21.15.67:0/2220827195 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc8940cb340 con 0x7fc8ac108ef0 2024-07-31T14:51:04.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:04 smithi073 ceph-mon[36714]: pgmap v399: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:51:04.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:04 smithi077 ceph-mon[37327]: pgmap v399: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-07-31T14:51:05.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05.063+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2220827195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc8ac02e0c0 con 0x7fc8ac108ef0 2024-07-31T14:51:05.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05.064+0000 7fc8a3fff640 1 -- 172.21.15.67:0/2220827195 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7fc8940cf890 con 0x7fc8ac108ef0 2024-07-31T14:51:05.065 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-07-31T14:51:05.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05.066+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2220827195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc8980b3c20 msgr2=0x7fc8980b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:05.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05.066+0000 7fc8b16d8640 1 --2- 172.21.15.67:0/2220827195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc8980b3c20 0x7fc8980b6130 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac076b90 tx=0x7fc89c009500 comp rx=0 tx=0).stop 2024-07-31T14:51:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05.066+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2220827195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8ac108ef0 msgr2=0x7fc8ac075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05.066+0000 7fc8b16d8640 1 --2- 172.21.15.67:0/2220827195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8ac108ef0 0x7fc8ac075680 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc89402a930 tx=0x7fc894031a20 comp rx=0 tx=0).stop 2024-07-31T14:51:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05.066+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2220827195 shutdown_connections 2024-07-31T14:51:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05.066+0000 7fc8b16d8640 1 --2- 172.21.15.67:0/2220827195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc8980b3c20 0x7fc8980b6130 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05.066+0000 7fc8b16d8640 1 --2- 172.21.15.67:0/2220827195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ac109840 0x7fc8ac075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05.066+0000 7fc8b16d8640 1 --2- 172.21.15.67:0/2220827195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8ac108ef0 0x7fc8ac075680 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05.066+0000 7fc8b16d8640 1 --2- 172.21.15.67:0/2220827195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ac1035b0 0x7fc8ac075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05.066+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2220827195 >> 172.21.15.67:0/2220827195 conn(0x7fc8ac0ff000 msgr2=0x7fc8ac101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05.066+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2220827195 shutdown_connections 2024-07-31T14:51:05.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05.066+0000 7fc8b16d8640 1 -- 172.21.15.67:0/2220827195 wait complete. 2024-07-31T14:51:05.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:05 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2220827195' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-31T14:51:05.838 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-31T14:51:05.838 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-31T14:51:05.838 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-07-31T14:51:05.838 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-31T14:51:05.839 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-31T14:51:05.839 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-31T14:51:05.839 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-31T14:51:05.839 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-31T14:51:05.839 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-31T14:51:05.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:51:05.839 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-07-31T14:51:05.857 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-31T14:51:05.857 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-31T14:51:05.857 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-07-31T14:51:05.857 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-07-31T14:51:05.858 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-07-31T14:51:05.858 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-07-31T14:51:05.858 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-31T14:51:05.858 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-07-31T14:51:05.858 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-07-31T14:51:05.858 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-31T14:51:05.858 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-07-31T14:51:05.858 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-07-31T14:51:05.858 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-31T14:51:05.858 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-07-31T14:51:05.858 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-31T14:51:05.858 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-07-31T14:51:05.859 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-07-31T14:51:05.859 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 41950sec preferred_lft 41950sec 2024-07-31T14:51:05.859 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-07-31T14:51:05.859 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-07-31T14:51:05.859 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-31T14:51:05.859 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-31T14:51:05.859 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-31T14:51:05.859 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2024-07-31T14:51:05.859 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2024-07-31T14:51:05.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2024-07-31T14:51:05.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-31T14:51:05.860 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-31T14:51:05.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:05 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2220827195' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-31T14:51:05.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-31T14:51:05.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:05 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2220827195' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-31T14:51:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:51:06.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:51:06.007 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-31T14:51:06.032 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-07-31T14:51:06.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:51:06.033 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-07-31T14:51:06.091 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-07-31T14:51:06.091 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-31T14:51:06.091 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-07-31T14:51:06.091 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-07-31T14:51:06.091 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-31T14:51:06.092 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-31T14:51:06.092 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-31T14:51:06.092 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-07-31T14:51:06.092 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-07-31T14:51:06.092 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-07-31T14:51:06.092 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-31T14:51:06.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-31T14:51:06.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:51:06.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:51:06.222 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-07-31T14:51:06.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:51:06.238 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2024-07-31T14:51:06.294 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-31T14:51:06.294 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-31T14:51:06.294 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-31T14:51:06.294 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-07-31T14:51:06.294 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-31T14:51:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-31T14:51:06.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:51:06.415 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-07-31T14:51:06.415 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-31T14:51:06.416 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-31T14:51:06.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-07-31T14:51:06.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-31T14:51:06.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-07-31T14:51:06.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-07-31T14:51:06.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-07-31T14:51:06.416 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-31T14:51:06.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-31T14:51:06.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:51:06.535 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-31T14:51:06.535 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-31T14:51:06.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-07-31T14:51:06.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2024-07-31T14:51:06.535 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-31T14:51:06.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-31T14:51:06.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:51:06.615 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-31T14:51:06.615 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-31T14:51:06.631 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-31T14:51:06.631 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-31T14:51:06.631 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-31T14:51:06.687 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-07-31T14:51:06.687 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-07-31T14:51:06.687 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051 2024-07-31T14:51:06.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:51:06.687 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-07-31T14:51:06.749 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:06 smithi067 ceph-mon[32750]: pgmap v400: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:51:06.782 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-07-31T14:51:06.782 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-07-31T14:51:06.782 INFO:teuthology.orchestra.run.smithi067.stdout:invalid new device string format 2024-07-31T14:51:06.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:06 smithi073 ceph-mon[36714]: pgmap v400: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:51:06.944 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-07-31T14:51:06.944 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-07-31T14:51:06.944 INFO:teuthology.orchestra.run.smithi067.stdout:invalid new device string format 2024-07-31T14:51:06.944 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.73:3300,172.21.15.77:3300" 2024-07-31T14:51:06.944 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.67:3300,172.21.15.73:3300,172.21.15.77:3300:/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051 2024-07-31T14:51:06.945 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=5ca2b71e-4f4a-11ef-bcc9-c7b262605968" will pass to kernel 2024-07-31T14:51:06.945 INFO:teuthology.orchestra.run.smithi067.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-31T14:51:06.945 INFO:teuthology.orchestra.run.smithi067.stdout: You just mounted a file system that supports labels which does not 2024-07-31T14:51:06.945 INFO:teuthology.orchestra.run.smithi067.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-31T14:51:06.945 INFO:teuthology.orchestra.run.smithi067.stdout: applications will generate AVC messages and not be allowed access to 2024-07-31T14:51:06.946 INFO:teuthology.orchestra.run.smithi067.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-31T14:51:06.947 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-31T14:51:06.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T14:51:06.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-31T14:51:06.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:06 smithi077 ceph-mon[37327]: pgmap v400: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-31T14:51:07.018 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-31T14:51:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-31T14:51:07.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:51:07.117 INFO:teuthology.orchestra.run.smithi067.stdout:client15489 2024-07-31T14:51:07.117 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:51:07.117 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-31T14:51:07.136 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-31T14:51:07.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-31T14:51:07.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:51:07.234 INFO:teuthology.orchestra.run.smithi067.stdout:client15489 2024-07-31T14:51:07.234 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-31T14:51:07.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-31T14:51:07.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:51:07.292 INFO:teuthology.orchestra.run.smithi067.stdout:client15489 2024-07-31T14:51:07.292 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968.client15489/status') 2024-07-31T14:51:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-31T14:51:07.353 INFO:teuthology.orchestra.run.smithi067.stdout:instance: client.15489 (3)192.168.0.1:0/4255272500 2024-07-31T14:51:07.353 INFO:teuthology.orchestra.run.smithi067.stdout:blocklisted: false 2024-07-31T14:51:07.354 INFO:teuthology.orchestra.run.smithi067.stderr:0+1 records in 2024-07-31T14:51:07.354 INFO:teuthology.orchestra.run.smithi067.stderr:0+1 records out 2024-07-31T14:51:07.354 INFO:teuthology.orchestra.run.smithi067.stderr:70 bytes copied, 2.7876e-05 s, 2.5 MB/s 2024-07-31T14:51:07.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-31T14:51:07.361 INFO:teuthology.run_tasks:Running task exec... 2024-07-31T14:51:07.372 INFO:teuthology.task.exec:Executing custom commands... 2024-07-31T14:51:07.372 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-07-31T14:51:07.372 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-07-31T14:51:07.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.487+0000 7f0a105a5640 1 -- 172.21.15.67:0/3556350243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a08102420 msgr2=0x7f0a08102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:07.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.487+0000 7f0a105a5640 1 --2- 172.21.15.67:0/3556350243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a08102420 0x7f0a08102830 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009900 tx=0x7f0a0402a410 comp rx=0 tx=0).stop 2024-07-31T14:51:07.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.487+0000 7f0a105a5640 1 -- 172.21.15.67:0/3556350243 shutdown_connections 2024-07-31T14:51:07.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.487+0000 7f0a105a5640 1 --2- 172.21.15.67:0/3556350243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a081086a0 0x7f0a0810eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:07.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.487+0000 7f0a105a5640 1 --2- 172.21.15.67:0/3556350243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a08107d50 0x7f0a08108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.487+0000 7f0a105a5640 1 --2- 172.21.15.67:0/3556350243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a08102420 0x7f0a08102830 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.487+0000 7f0a105a5640 1 -- 172.21.15.67:0/3556350243 >> 172.21.15.67:0/3556350243 conn(0x7f0a080fde20 msgr2=0x7f0a08100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.487+0000 7f0a105a5640 1 -- 172.21.15.67:0/3556350243 shutdown_connections 2024-07-31T14:51:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.487+0000 7f0a105a5640 1 -- 172.21.15.67:0/3556350243 wait complete. 2024-07-31T14:51:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.488+0000 7f0a105a5640 1 Processor -- start 2024-07-31T14:51:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.488+0000 7f0a105a5640 1 -- start start 2024-07-31T14:51:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.488+0000 7f0a105a5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a08102420 0x7f0a0819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.488+0000 7f0a105a5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a08107d50 0x7f0a0819b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.488+0000 7f0a105a5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a081086a0 0x7f0a0819b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.488+0000 7f0a105a5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0a08112350 con 0x7f0a081086a0 2024-07-31T14:51:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.488+0000 7f0a105a5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0a081121d0 con 0x7f0a08107d50 2024-07-31T14:51:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.488+0000 7f0a105a5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0a081124d0 con 0x7f0a08102420 2024-07-31T14:51:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.488+0000 7f0a0eb42640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a081086a0 0x7f0a0819b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:07.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.488+0000 7f0a0eb42640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a081086a0 0x7f0a0819b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33828/0 (socket says 172.21.15.67:33828) 2024-07-31T14:51:07.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.488+0000 7f0a0eb42640 1 -- 172.21.15.67:0/800511225 learned_addr learned my addr 172.21.15.67:0/800511225 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:07.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.488+0000 7f0a0e341640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a08102420 0x7f0a0819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:07.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.488+0000 7f0a0db40640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a08107d50 0x7f0a0819b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:07.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.489+0000 7f0a0eb42640 1 -- 172.21.15.67:0/800511225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a08102420 msgr2=0x7f0a0819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:07.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.489+0000 7f0a0eb42640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a08102420 0x7f0a0819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:07.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.489+0000 7f0a0eb42640 1 -- 172.21.15.67:0/800511225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a08107d50 msgr2=0x7f0a0819b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:07.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.489+0000 7f0a0eb42640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a08107d50 0x7f0a0819b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:07.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.489+0000 7f0a0eb42640 1 -- 172.21.15.67:0/800511225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a081a5ea0 con 0x7f0a081086a0 2024-07-31T14:51:07.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.489+0000 7f0a0db40640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a08107d50 0x7f0a0819b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:51:07.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.489+0000 7f0a0eb42640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a081086a0 0x7f0a0819b720 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000eb10 tx=0x7f0a0000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:07.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.489+0000 7f09ff7fe640 1 -- 172.21.15.67:0/800511225 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00004b20 con 0x7f0a081086a0 2024-07-31T14:51:07.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.489+0000 7f09ff7fe640 1 -- 172.21.15.67:0/800511225 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a00004cc0 con 0x7f0a081086a0 2024-07-31T14:51:07.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.489+0000 7f0a105a5640 1 -- 172.21.15.67:0/800511225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a081a6190 con 0x7f0a081086a0 2024-07-31T14:51:07.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.489+0000 7f09ff7fe640 1 -- 172.21.15.67:0/800511225 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00010670 con 0x7f0a081086a0 2024-07-31T14:51:07.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.489+0000 7f0a105a5640 1 -- 172.21.15.67:0/800511225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a081a66e0 con 0x7f0a081086a0 2024-07-31T14:51:07.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.490+0000 7f0a105a5640 1 -- 172.21.15.67:0/800511225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a08103690 con 0x7f0a081086a0 2024-07-31T14:51:07.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.492+0000 7f09ff7fe640 1 -- 172.21.15.67:0/800511225 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f0a00005000 con 0x7f0a081086a0 2024-07-31T14:51:07.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.492+0000 7f09ff7fe640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09f40b3ca0 0x7f09f40b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:07.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.492+0000 7f0a0e341640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09f40b3ca0 0x7f09f40b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:07.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.493+0000 7f09ff7fe640 1 -- 172.21.15.67:0/800511225 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f0a000e4850 con 0x7f0a081086a0 2024-07-31T14:51:07.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.493+0000 7f0a0e341640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09f40b3ca0 0x7f09f40b61b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0a04000c00 tx=0x7f0a04031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:07.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.495+0000 7f09ff7fe640 1 -- 172.21.15.67:0/800511225 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f0a000555e0 con 0x7f0a081086a0 2024-07-31T14:51:07.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:07.732+0000 7f0a105a5640 1 -- 172.21.15.67:0/800511225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f0a08102830 con 0x7f0a081086a0 2024-07-31T14:51:08.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.515+0000 7f09ff7fe640 1 -- 172.21.15.67:0/800511225 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v30) ==== 118+0+0 (secure 0 0 0) 0x7f0a000aba70 con 0x7f0a081086a0 2024-07-31T14:51:08.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.518+0000 7f0a105a5640 1 -- 172.21.15.67:0/800511225 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09f40b3ca0 msgr2=0x7f09f40b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:08.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.518+0000 7f0a105a5640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09f40b3ca0 0x7f09f40b61b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0a04000c00 tx=0x7f0a04031400 comp rx=0 tx=0).stop 2024-07-31T14:51:08.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.518+0000 7f0a105a5640 1 -- 172.21.15.67:0/800511225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a081086a0 msgr2=0x7f0a0819b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:08.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.518+0000 7f0a105a5640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a081086a0 0x7f0a0819b720 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000eb10 tx=0x7f0a0000c2d0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.518+0000 7f0a105a5640 1 -- 172.21.15.67:0/800511225 shutdown_connections 2024-07-31T14:51:08.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.518+0000 7f0a105a5640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09f40b3ca0 0x7f09f40b61b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.518+0000 7f0a105a5640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a081086a0 0x7f0a0819b720 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.518+0000 7f0a105a5640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a08107d50 0x7f0a0819b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.518+0000 7f0a105a5640 1 --2- 172.21.15.67:0/800511225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a08102420 0x7f0a0819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.518+0000 7f0a105a5640 1 -- 172.21.15.67:0/800511225 >> 172.21.15.67:0/800511225 conn(0x7f0a080fde20 msgr2=0x7f0a08100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:08.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.518+0000 7f0a105a5640 1 -- 172.21.15.67:0/800511225 shutdown_connections 2024-07-31T14:51:08.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.518+0000 7f0a105a5640 1 -- 172.21.15.67:0/800511225 wait complete. 2024-07-31T14:51:08.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-07-31T14:51:08.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.657+0000 7f42d4a58640 1 -- 172.21.15.67:0/518392005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d0102b60 msgr2=0x7f42d0102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:08.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.657+0000 7f42d4a58640 1 --2- 172.21.15.67:0/518392005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d0102b60 0x7f42d0102f70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009900 tx=0x7f42c002a3f0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.657+0000 7f42d4a58640 1 -- 172.21.15.67:0/518392005 shutdown_connections 2024-07-31T14:51:08.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.657+0000 7f42d4a58640 1 --2- 172.21.15.67:0/518392005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d0105470 0x7f42d010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.657+0000 7f42d4a58640 1 --2- 172.21.15.67:0/518392005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42d0104b20 0x7f42d0104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.657+0000 7f42d4a58640 1 --2- 172.21.15.67:0/518392005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d0102b60 0x7f42d0102f70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.657+0000 7f42d4a58640 1 -- 172.21.15.67:0/518392005 >> 172.21.15.67:0/518392005 conn(0x7f42d0079880 msgr2=0x7f42d0079c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:08.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.658+0000 7f42d4a58640 1 -- 172.21.15.67:0/518392005 shutdown_connections 2024-07-31T14:51:08.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.658+0000 7f42d4a58640 1 -- 172.21.15.67:0/518392005 wait complete. 2024-07-31T14:51:08.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.658+0000 7f42d4a58640 1 Processor -- start 2024-07-31T14:51:08.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42d4a58640 1 -- start start 2024-07-31T14:51:08.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42d4a58640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d0102b60 0x7f42d0199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:08.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42d4a58640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d0104b20 0x7f42d019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:08.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42d4a58640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42d0105470 0x7f42d019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:08.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42d4a58640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f42d0112360 con 0x7f42d0104b20 2024-07-31T14:51:08.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42d4a58640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f42d01121e0 con 0x7f42d0105470 2024-07-31T14:51:08.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42d4a58640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f42d01124e0 con 0x7f42d0102b60 2024-07-31T14:51:08.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42c5d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d0104b20 0x7f42d019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:08.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42c5d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d0104b20 0x7f42d019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33848/0 (socket says 172.21.15.67:33848) 2024-07-31T14:51:08.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42c5d9b640 1 -- 172.21.15.67:0/3066010395 learned_addr learned my addr 172.21.15.67:0/3066010395 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:08.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42ced9d640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42d0105470 0x7f42d019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:08.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42ce59c640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d0102b60 0x7f42d0199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:08.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42c5d9b640 1 -- 172.21.15.67:0/3066010395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d0102b60 msgr2=0x7f42d0199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:08.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42c5d9b640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d0102b60 0x7f42d0199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42c5d9b640 1 -- 172.21.15.67:0/3066010395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42d0105470 msgr2=0x7f42d019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:08.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42c5d9b640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42d0105470 0x7f42d019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42c5d9b640 1 -- 172.21.15.67:0/3066010395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d019fd70 con 0x7f42d0104b20 2024-07-31T14:51:08.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42ce59c640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d0102b60 0x7f42d0199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:51:08.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.659+0000 7f42c5d9b640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d0104b20 0x7f42d019a470 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f42b800d950 tx=0x7f42b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:08.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.660+0000 7f42c7fff640 1 -- 172.21.15.67:0/3066010395 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8017070 con 0x7f42d0104b20 2024-07-31T14:51:08.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.660+0000 7f42c7fff640 1 -- 172.21.15.67:0/3066010395 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b8009c30 con 0x7f42d0104b20 2024-07-31T14:51:08.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.660+0000 7f42c7fff640 1 -- 172.21.15.67:0/3066010395 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b80047a0 con 0x7f42d0104b20 2024-07-31T14:51:08.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.660+0000 7f42d4a58640 1 -- 172.21.15.67:0/3066010395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d01a0060 con 0x7f42d0104b20 2024-07-31T14:51:08.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.660+0000 7f42d4a58640 1 -- 172.21.15.67:0/3066010395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d01a6880 con 0x7f42d0104b20 2024-07-31T14:51:08.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.661+0000 7f42d4a58640 1 -- 172.21.15.67:0/3066010395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d019aff0 con 0x7f42d0104b20 2024-07-31T14:51:08.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.663+0000 7f42c7fff640 1 -- 172.21.15.67:0/3066010395 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f42b8004020 con 0x7f42d0104b20 2024-07-31T14:51:08.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.664+0000 7f42c7fff640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42bc0b3c20 0x7f42bc0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:08.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.664+0000 7f42ce59c640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42bc0b3c20 0x7f42bc0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:08.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.664+0000 7f42c7fff640 1 -- 172.21.15.67:0/3066010395 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f42b80e4270 con 0x7f42d0104b20 2024-07-31T14:51:08.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.664+0000 7f42ce59c640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42bc0b3c20 0x7f42bc0b6130 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f42c00096c0 tx=0x7f42c0031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:08.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.666+0000 7f42c7fff640 1 -- 172.21.15.67:0/3066010395 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f42b8020020 con 0x7f42d0104b20 2024-07-31T14:51:08.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:08 smithi067 ceph-mon[32750]: pgmap v401: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:51:08.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:08 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/800511225' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-07-31T14:51:08.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:08 smithi073 ceph-mon[36714]: pgmap v401: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:51:08.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:08 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/800511225' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-07-31T14:51:08.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.891+0000 7f42d4a58640 1 -- 172.21.15.67:0/3066010395 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f42d0063230 con 0x7f42bc0b3c20 2024-07-31T14:51:08.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.913+0000 7f42c7fff640 1 -- 172.21.15.67:0/3066010395 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f42d0063230 con 0x7f42bc0b3c20 2024-07-31T14:51:08.915 INFO:teuthology.orchestra.run.smithi067.stdout:{} 2024-07-31T14:51:08.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.915+0000 7f42d4a58640 1 -- 172.21.15.67:0/3066010395 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42bc0b3c20 msgr2=0x7f42bc0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:08.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.915+0000 7f42d4a58640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42bc0b3c20 0x7f42bc0b6130 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f42c00096c0 tx=0x7f42c0031400 comp rx=0 tx=0).stop 2024-07-31T14:51:08.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.915+0000 7f42d4a58640 1 -- 172.21.15.67:0/3066010395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d0104b20 msgr2=0x7f42d019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:08.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.915+0000 7f42d4a58640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d0104b20 0x7f42d019a470 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f42b800d950 tx=0x7f42b800de10 comp rx=0 tx=0).stop 2024-07-31T14:51:08.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.915+0000 7f42d4a58640 1 -- 172.21.15.67:0/3066010395 shutdown_connections 2024-07-31T14:51:08.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.915+0000 7f42d4a58640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42bc0b3c20 0x7f42bc0b6130 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.915+0000 7f42d4a58640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42d0105470 0x7f42d019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.915+0000 7f42d4a58640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d0104b20 0x7f42d019a470 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.915+0000 7f42d4a58640 1 --2- 172.21.15.67:0/3066010395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d0102b60 0x7f42d0199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:08.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.915+0000 7f42d4a58640 1 -- 172.21.15.67:0/3066010395 >> 172.21.15.67:0/3066010395 conn(0x7f42d0079880 msgr2=0x7f42d0106de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:08.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.915+0000 7f42d4a58640 1 -- 172.21.15.67:0/3066010395 shutdown_connections 2024-07-31T14:51:08.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:08.915+0000 7f42d4a58640 1 -- 172.21.15.67:0/3066010395 wait complete. 2024-07-31T14:51:08.929 INFO:teuthology.run_tasks:Running task check-counter... 2024-07-31T14:51:08.940 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-07-31T14:51:09.000 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-31T14:51:09.001 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:08 smithi077 ceph-mon[37327]: pgmap v401: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-31T14:51:09.001 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:08 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/800511225' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-07-31T14:51:09.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.114+0000 7f9b23fb6640 1 -- 172.21.15.67:0/1129671400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b1c104ca0 msgr2=0x7f9b1c10d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:09.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.114+0000 7f9b23fb6640 1 --2- 172.21.15.67:0/1129671400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b1c104ca0 0x7f9b1c10d1e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9b18009900 tx=0x7f9b1802a410 comp rx=0 tx=0).stop 2024-07-31T14:51:09.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.114+0000 7f9b23fb6640 1 -- 172.21.15.67:0/1129671400 shutdown_connections 2024-07-31T14:51:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.114+0000 7f9b23fb6640 1 --2- 172.21.15.67:0/1129671400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b1c10d7b0 0x7f9b1c10fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.114+0000 7f9b23fb6640 1 --2- 172.21.15.67:0/1129671400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b1c104ca0 0x7f9b1c10d1e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.114+0000 7f9b23fb6640 1 --2- 172.21.15.67:0/1129671400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b1c104350 0x7f9b1c104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.114+0000 7f9b23fb6640 1 -- 172.21.15.67:0/1129671400 >> 172.21.15.67:0/1129671400 conn(0x7f9b1c0fdde0 msgr2=0x7f9b1c100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.114+0000 7f9b23fb6640 1 -- 172.21.15.67:0/1129671400 shutdown_connections 2024-07-31T14:51:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.114+0000 7f9b23fb6640 1 -- 172.21.15.67:0/1129671400 wait complete. 2024-07-31T14:51:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b23fb6640 1 Processor -- start 2024-07-31T14:51:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b23fb6640 1 -- start start 2024-07-31T14:51:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b23fb6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b1c104350 0x7f9b1c102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b23fb6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b1c104ca0 0x7f9b1c1025c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b23fb6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b1c10d7b0 0x7f9b1c102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b23fb6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c113400 con 0x7f9b1c10d7b0 2024-07-31T14:51:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b23fb6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c113280 con 0x7f9b1c104350 2024-07-31T14:51:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b23fb6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c113580 con 0x7f9b1c104ca0 2024-07-31T14:51:09.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b22553640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b1c10d7b0 0x7f9b1c102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:09.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b22553640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b1c10d7b0 0x7f9b1c102b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33886/0 (socket says 172.21.15.67:33886) 2024-07-31T14:51:09.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b21551640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b1c104ca0 0x7f9b1c1025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:09.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b22553640 1 -- 172.21.15.67:0/2992491948 learned_addr learned my addr 172.21.15.67:0/2992491948 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:09.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b21d52640 1 --2- 172.21.15.67:0/2992491948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b1c104350 0x7f9b1c102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:09.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b22553640 1 -- 172.21.15.67:0/2992491948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b1c104ca0 msgr2=0x7f9b1c1025c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:09.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b22553640 1 --2- 172.21.15.67:0/2992491948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b1c104ca0 0x7f9b1c1025c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b22553640 1 -- 172.21.15.67:0/2992491948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b1c104350 msgr2=0x7f9b1c102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:09.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b22553640 1 --2- 172.21.15.67:0/2992491948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b1c104350 0x7f9b1c102080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b22553640 1 -- 172.21.15.67:0/2992491948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1c1a80d0 con 0x7f9b1c10d7b0 2024-07-31T14:51:09.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.115+0000 7f9b21d52640 1 --2- 172.21.15.67:0/2992491948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b1c104350 0x7f9b1c102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:09.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.116+0000 7f9b22553640 1 --2- 172.21.15.67:0/2992491948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b1c10d7b0 0x7f9b1c102b00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400eb10 tx=0x7f9b1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:09.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.116+0000 7f9b12ffd640 1 -- 172.21.15.67:0/2992491948 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14004b20 con 0x7f9b1c10d7b0 2024-07-31T14:51:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.116+0000 7f9b23fb6640 1 -- 172.21.15.67:0/2992491948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1c1a83c0 con 0x7f9b1c10d7b0 2024-07-31T14:51:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.116+0000 7f9b12ffd640 1 -- 172.21.15.67:0/2992491948 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b14004cc0 con 0x7f9b1c10d7b0 2024-07-31T14:51:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.116+0000 7f9b12ffd640 1 -- 172.21.15.67:0/2992491948 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14010690 con 0x7f9b1c10d7b0 2024-07-31T14:51:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.116+0000 7f9b23fb6640 1 -- 172.21.15.67:0/2992491948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b1c1a8930 con 0x7f9b1c10d7b0 2024-07-31T14:51:09.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.117+0000 7f9b23fb6640 1 -- 172.21.15.67:0/2992491948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b1c104760 con 0x7f9b1c10d7b0 2024-07-31T14:51:09.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.118+0000 7f9b12ffd640 1 -- 172.21.15.67:0/2992491948 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f9b14005000 con 0x7f9b1c10d7b0 2024-07-31T14:51:09.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.119+0000 7f9b12ffd640 1 --2- 172.21.15.67:0/2992491948 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9b080b3c50 0x7f9b080b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:09.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.119+0000 7f9b21d52640 1 --2- 172.21.15.67:0/2992491948 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9b080b3c50 0x7f9b080b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:09.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.119+0000 7f9b12ffd640 1 -- 172.21.15.67:0/2992491948 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f9b140e4850 con 0x7f9b1c10d7b0 2024-07-31T14:51:09.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.120+0000 7f9b21d52640 1 --2- 172.21.15.67:0/2992491948 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9b080b3c50 0x7f9b080b6160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c103ad0 tx=0x7f9b0c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:09.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.122+0000 7f9b12ffd640 1 -- 172.21.15.67:0/2992491948 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f9b140555e0 con 0x7f9b1c10d7b0 2024-07-31T14:51:09.303 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:51:08 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:51:08.896+0000 7f4cce78f640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:51:09.303 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:51:08 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:51:08.896+0000 7f4cce78f640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:51:09.303 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:51:08 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:51:08.896+0000 7f4cce78f640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:51:09.303 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:51:08 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:51:08.896+0000 7f4cce78f640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:51:09.303 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:51:08 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:51:08.896+0000 7f4cce78f640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:51:09.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.319+0000 7f9b23fb6640 1 -- 172.21.15.67:0/2992491948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b1c1a8d90 con 0x7f9b1c10d7b0 2024-07-31T14:51:09.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.320+0000 7f9b12ffd640 1 -- 172.21.15.67:0/2992491948 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24874 (secure 0 0 0) 0x7f9b140aba70 con 0x7f9b1c10d7b0 2024-07-31T14:51:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:51:09.323 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T14:47:20.440064+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T14:51:09.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.323+0000 7f9b23fb6640 1 -- 172.21.15.67:0/2992491948 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9b080b3c50 msgr2=0x7f9b080b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:09.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.323+0000 7f9b23fb6640 1 --2- 172.21.15.67:0/2992491948 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9b080b3c50 0x7f9b080b6160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c103ad0 tx=0x7f9b0c005f90 comp rx=0 tx=0).stop 2024-07-31T14:51:09.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.323+0000 7f9b23fb6640 1 -- 172.21.15.67:0/2992491948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b1c10d7b0 msgr2=0x7f9b1c102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:09.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.323+0000 7f9b23fb6640 1 --2- 172.21.15.67:0/2992491948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b1c10d7b0 0x7f9b1c102b00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400eb10 tx=0x7f9b1400c2d0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.323+0000 7f9b23fb6640 1 -- 172.21.15.67:0/2992491948 shutdown_connections 2024-07-31T14:51:09.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.323+0000 7f9b23fb6640 1 --2- 172.21.15.67:0/2992491948 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9b080b3c50 0x7f9b080b6160 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.323+0000 7f9b23fb6640 1 --2- 172.21.15.67:0/2992491948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b1c10d7b0 0x7f9b1c102b00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.323+0000 7f9b23fb6640 1 --2- 172.21.15.67:0/2992491948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b1c104ca0 0x7f9b1c1025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.323+0000 7f9b23fb6640 1 --2- 172.21.15.67:0/2992491948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b1c104350 0x7f9b1c102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.323+0000 7f9b23fb6640 1 -- 172.21.15.67:0/2992491948 >> 172.21.15.67:0/2992491948 conn(0x7f9b1c0fdde0 msgr2=0x7f9b1c1072c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:09.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.323+0000 7f9b23fb6640 1 -- 172.21.15.67:0/2992491948 shutdown_connections 2024-07-31T14:51:09.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.323+0000 7f9b23fb6640 1 -- 172.21.15.67:0/2992491948 wait complete. 2024-07-31T14:51:09.337 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-31T14:42:27.359626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-07-31T14:46:58.459751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-07-31T14:47:01.783728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-07-31T14:47:11.080224+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-07-31T14:51:09.338 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-31T14:51:09.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.454+0000 7ff9b644b640 1 -- 172.21.15.67:0/1436778677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0104330 msgr2=0x7ff9b0104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:09.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.454+0000 7ff9b644b640 1 --2- 172.21.15.67:0/1436778677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0104330 0x7ff9b0104740 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009900 tx=0x7ff9a002a410 comp rx=0 tx=0).stop 2024-07-31T14:51:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.454+0000 7ff9b644b640 1 -- 172.21.15.67:0/1436778677 shutdown_connections 2024-07-31T14:51:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.454+0000 7ff9b644b640 1 --2- 172.21.15.67:0/1436778677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b010d790 0x7ff9b010fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.454+0000 7ff9b644b640 1 --2- 172.21.15.67:0/1436778677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b0104c80 0x7ff9b010d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.454+0000 7ff9b644b640 1 --2- 172.21.15.67:0/1436778677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0104330 0x7ff9b0104740 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.454+0000 7ff9b644b640 1 -- 172.21.15.67:0/1436778677 >> 172.21.15.67:0/1436778677 conn(0x7ff9b00fddc0 msgr2=0x7ff9b0100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.454+0000 7ff9b644b640 1 -- 172.21.15.67:0/1436778677 shutdown_connections 2024-07-31T14:51:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.454+0000 7ff9b644b640 1 -- 172.21.15.67:0/1436778677 wait complete. 2024-07-31T14:51:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9b644b640 1 Processor -- start 2024-07-31T14:51:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9b644b640 1 -- start start 2024-07-31T14:51:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9b644b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0104330 0x7ff9b0199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9b644b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b0104c80 0x7ff9b019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9b644b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b010d790 0x7ff9b019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9b644b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b0113470 con 0x7ff9b0104330 2024-07-31T14:51:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9b644b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b01132f0 con 0x7ff9b010d790 2024-07-31T14:51:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9b644b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b01135f0 con 0x7ff9b0104c80 2024-07-31T14:51:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9affff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0104330 0x7ff9b0199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9affff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0104330 0x7ff9b0199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33914/0 (socket says 172.21.15.67:33914) 2024-07-31T14:51:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9affff640 1 -- 172.21.15.67:0/2092026954 learned_addr learned my addr 172.21.15.67:0/2092026954 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9b49e8640 1 --2- 172.21.15.67:0/2092026954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b010d790 0x7ff9b019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9af7fe640 1 --2- 172.21.15.67:0/2092026954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b0104c80 0x7ff9b019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9affff640 1 -- 172.21.15.67:0/2092026954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b0104c80 msgr2=0x7ff9b019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9affff640 1 --2- 172.21.15.67:0/2092026954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b0104c80 0x7ff9b019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9affff640 1 -- 172.21.15.67:0/2092026954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b010d790 msgr2=0x7ff9b019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9affff640 1 --2- 172.21.15.67:0/2092026954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b010d790 0x7ff9b019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.455+0000 7ff9affff640 1 -- 172.21.15.67:0/2092026954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b019fd90 con 0x7ff9b0104330 2024-07-31T14:51:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.456+0000 7ff9affff640 1 --2- 172.21.15.67:0/2092026954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0104330 0x7ff9b0199ef0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0006010 tx=0x7ff9a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:09.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.456+0000 7ff9ad7fa640 1 -- 172.21.15.67:0/2092026954 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a003f030 con 0x7ff9b0104330 2024-07-31T14:51:09.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.456+0000 7ff9ad7fa640 1 -- 172.21.15.67:0/2092026954 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a0003800 con 0x7ff9b0104330 2024-07-31T14:51:09.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.456+0000 7ff9ad7fa640 1 -- 172.21.15.67:0/2092026954 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0035070 con 0x7ff9b0104330 2024-07-31T14:51:09.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.456+0000 7ff9b644b640 1 -- 172.21.15.67:0/2092026954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b01a0020 con 0x7ff9b0104330 2024-07-31T14:51:09.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.456+0000 7ff9b644b640 1 -- 172.21.15.67:0/2092026954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b01a68a0 con 0x7ff9b0104330 2024-07-31T14:51:09.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.457+0000 7ff9b644b640 1 -- 172.21.15.67:0/2092026954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b0104740 con 0x7ff9b0104330 2024-07-31T14:51:09.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.459+0000 7ff9ad7fa640 1 -- 172.21.15.67:0/2092026954 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7ff9a0003cb0 con 0x7ff9b0104330 2024-07-31T14:51:09.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.459+0000 7ff9ad7fa640 1 --2- 172.21.15.67:0/2092026954 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff9980b3ca0 0x7ff9980b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:09.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.459+0000 7ff9af7fe640 1 --2- 172.21.15.67:0/2092026954 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff9980b3ca0 0x7ff9980b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:09.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.459+0000 7ff9ad7fa640 1 -- 172.21.15.67:0/2092026954 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff9a0104c50 con 0x7ff9b0104330 2024-07-31T14:51:09.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.460+0000 7ff9af7fe640 1 --2- 172.21.15.67:0/2092026954 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff9980b3ca0 0x7ff9980b61b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff9b010eaf0 tx=0x7ff9a4005cc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:09.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.462+0000 7ff9ad7fa640 1 -- 172.21.15.67:0/2092026954 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff9a00cbe70 con 0x7ff9b0104330 2024-07-31T14:51:09.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.543+0000 7ff9ad7fa640 1 -- 172.21.15.67:0/2092026954 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7ff9a00c7a00 con 0x7ff9b0104330 2024-07-31T14:51:09.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.660+0000 7ff9b644b640 1 -- 172.21.15.67:0/2092026954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff9b01a6ad0 con 0x7ff9b0104330 2024-07-31T14:51:09.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.661+0000 7ff9ad7fa640 1 -- 172.21.15.67:0/2092026954 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7ff9a00fb1c0 con 0x7ff9b0104330 2024-07-31T14:51:09.663 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-07-31T14:51:09.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.663+0000 7ff9b644b640 1 -- 172.21.15.67:0/2092026954 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff9980b3ca0 msgr2=0x7ff9980b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:09.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.663+0000 7ff9b644b640 1 --2- 172.21.15.67:0/2092026954 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff9980b3ca0 0x7ff9980b61b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff9b010eaf0 tx=0x7ff9a4005cc0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.663+0000 7ff9b644b640 1 -- 172.21.15.67:0/2092026954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0104330 msgr2=0x7ff9b0199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:09.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.663+0000 7ff9b644b640 1 --2- 172.21.15.67:0/2092026954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0104330 0x7ff9b0199ef0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0006010 tx=0x7ff9a00042f0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.663+0000 7ff9b644b640 1 -- 172.21.15.67:0/2092026954 shutdown_connections 2024-07-31T14:51:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.663+0000 7ff9b644b640 1 --2- 172.21.15.67:0/2092026954 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff9980b3ca0 0x7ff9980b61b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.663+0000 7ff9b644b640 1 --2- 172.21.15.67:0/2092026954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b010d790 0x7ff9b019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.663+0000 7ff9b644b640 1 --2- 172.21.15.67:0/2092026954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b0104c80 0x7ff9b019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.663+0000 7ff9b644b640 1 --2- 172.21.15.67:0/2092026954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0104330 0x7ff9b0199ef0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.663+0000 7ff9b644b640 1 -- 172.21.15.67:0/2092026954 >> 172.21.15.67:0/2092026954 conn(0x7ff9b00fddc0 msgr2=0x7ff9b01072a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.663+0000 7ff9b644b640 1 -- 172.21.15.67:0/2092026954 shutdown_connections 2024-07-31T14:51:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.663+0000 7ff9b644b640 1 -- 172.21.15.67:0/2092026954 wait complete. 2024-07-31T14:51:09.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:09 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/800511225' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-07-31T14:51:09.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:09 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:51:09.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:09 smithi067 ceph-mon[32750]: from='client.15501 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:09.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:09 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2992491948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:51:09.677 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-07-31T14:51:09.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.793+0000 7f3ac9606640 1 -- 172.21.15.67:0/2524744451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ac41023e0 msgr2=0x7f3ac41027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:09.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.793+0000 7f3ac9606640 1 --2- 172.21.15.67:0/2524744451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ac41023e0 0x7f3ac41027f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009900 tx=0x7f3ab402a410 comp rx=0 tx=0).stop 2024-07-31T14:51:09.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.793+0000 7f3ac9606640 1 -- 172.21.15.67:0/2524744451 shutdown_connections 2024-07-31T14:51:09.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.793+0000 7f3ac9606640 1 --2- 172.21.15.67:0/2524744451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ac4108660 0x7f3ac410ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.793+0000 7f3ac9606640 1 --2- 172.21.15.67:0/2524744451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ac4107d10 0x7f3ac4108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.793+0000 7f3ac9606640 1 --2- 172.21.15.67:0/2524744451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ac41023e0 0x7f3ac41027f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.793+0000 7f3ac9606640 1 -- 172.21.15.67:0/2524744451 >> 172.21.15.67:0/2524744451 conn(0x7f3ac40fddc0 msgr2=0x7f3ac4100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:09.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.793+0000 7f3ac9606640 1 -- 172.21.15.67:0/2524744451 shutdown_connections 2024-07-31T14:51:09.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.793+0000 7f3ac9606640 1 -- 172.21.15.67:0/2524744451 wait complete. 2024-07-31T14:51:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.793+0000 7f3ac9606640 1 Processor -- start 2024-07-31T14:51:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac9606640 1 -- start start 2024-07-31T14:51:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac9606640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ac41023e0 0x7f3ac4199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac9606640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ac4107d10 0x7f3ac419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac9606640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ac4108660 0x7f3ac419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac9606640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4112380 con 0x7f3ac4108660 2024-07-31T14:51:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac9606640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4112200 con 0x7f3ac41023e0 2024-07-31T14:51:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac9606640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4112500 con 0x7f3ac4107d10 2024-07-31T14:51:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ac4108660 0x7f3ac419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ac4108660 0x7f3ac419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33940/0 (socket says 172.21.15.67:33940) 2024-07-31T14:51:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac37fe640 1 -- 172.21.15.67:0/2343251079 learned_addr learned my addr 172.21.15.67:0/2343251079 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac27fc640 1 --2- 172.21.15.67:0/2343251079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ac4107d10 0x7f3ac419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac2ffd640 1 --2- 172.21.15.67:0/2343251079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ac41023e0 0x7f3ac4199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac37fe640 1 -- 172.21.15.67:0/2343251079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ac4107d10 msgr2=0x7f3ac419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac37fe640 1 --2- 172.21.15.67:0/2343251079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ac4107d10 0x7f3ac419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac37fe640 1 -- 172.21.15.67:0/2343251079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ac41023e0 msgr2=0x7f3ac4199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac37fe640 1 --2- 172.21.15.67:0/2343251079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ac41023e0 0x7f3ac4199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.794+0000 7f3ac37fe640 1 -- 172.21.15.67:0/2343251079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac419fe00 con 0x7f3ac4108660 2024-07-31T14:51:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.795+0000 7f3ac37fe640 1 --2- 172.21.15.67:0/2343251079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ac4108660 0x7f3ac419f700 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009fd0 tx=0x7f3abc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.795+0000 7f3aabfff640 1 -- 172.21.15.67:0/2343251079 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc017070 con 0x7f3ac4108660 2024-07-31T14:51:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.795+0000 7f3ac9606640 1 -- 172.21.15.67:0/2343251079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac41a00f0 con 0x7f3ac4108660 2024-07-31T14:51:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.795+0000 7f3aabfff640 1 -- 172.21.15.67:0/2343251079 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3abc003e20 con 0x7f3ac4108660 2024-07-31T14:51:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.795+0000 7f3aabfff640 1 -- 172.21.15.67:0/2343251079 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc010460 con 0x7f3ac4108660 2024-07-31T14:51:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.795+0000 7f3ac9606640 1 -- 172.21.15.67:0/2343251079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac41a6880 con 0x7f3ac4108660 2024-07-31T14:51:09.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.796+0000 7f3ac9606640 1 -- 172.21.15.67:0/2343251079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac41027f0 con 0x7f3ac4108660 2024-07-31T14:51:09.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.797+0000 7f3aabfff640 1 -- 172.21.15.67:0/2343251079 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f3abc003970 con 0x7f3ac4108660 2024-07-31T14:51:09.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.798+0000 7f3aabfff640 1 --2- 172.21.15.67:0/2343251079 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aac0b3cb0 0x7f3aac0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:09.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.798+0000 7f3ac2ffd640 1 --2- 172.21.15.67:0/2343251079 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aac0b3cb0 0x7f3aac0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:09.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.798+0000 7f3aabfff640 1 -- 172.21.15.67:0/2343251079 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f3abc0e3e30 con 0x7f3ac4108660 2024-07-31T14:51:09.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.799+0000 7f3ac2ffd640 1 --2- 172.21.15.67:0/2343251079 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aac0b3cb0 0x7f3aac0b61c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4005ab0 tx=0x7f3ab4005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:09.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.801+0000 7f3aabfff640 1 -- 172.21.15.67:0/2343251079 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f3abc0aa8a0 con 0x7f3ac4108660 2024-07-31T14:51:09.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:09 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/800511225' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-07-31T14:51:09.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:09 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:51:09.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:09 smithi073 ceph-mon[36714]: from='client.15501 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:09.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:09 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2992491948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:51:09.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:09 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/800511225' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-07-31T14:51:09.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:09 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:51:09.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:09 smithi077 ceph-mon[37327]: from='client.15501 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:09.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:09 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2992491948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:51:09.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.995+0000 7f3ac9606640 1 -- 172.21.15.67:0/2343251079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3ac410ae60 con 0x7f3ac4108660 2024-07-31T14:51:09.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.996+0000 7f3aabfff640 1 -- 172.21.15.67:0/2343251079 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f3abc0aedf0 con 0x7f3ac4108660 2024-07-31T14:51:09.998 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-07-31T14:51:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.998+0000 7f3ac9606640 1 -- 172.21.15.67:0/2343251079 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aac0b3cb0 msgr2=0x7f3aac0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.998+0000 7f3ac9606640 1 --2- 172.21.15.67:0/2343251079 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aac0b3cb0 0x7f3aac0b61c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4005ab0 tx=0x7f3ab4005a20 comp rx=0 tx=0).stop 2024-07-31T14:51:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.998+0000 7f3ac9606640 1 -- 172.21.15.67:0/2343251079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ac4108660 msgr2=0x7f3ac419f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.998+0000 7f3ac9606640 1 --2- 172.21.15.67:0/2343251079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ac4108660 0x7f3ac419f700 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009fd0 tx=0x7f3abc00b260 comp rx=0 tx=0).stop 2024-07-31T14:51:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.998+0000 7f3ac9606640 1 -- 172.21.15.67:0/2343251079 shutdown_connections 2024-07-31T14:51:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.998+0000 7f3ac9606640 1 --2- 172.21.15.67:0/2343251079 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aac0b3cb0 0x7f3aac0b61c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.998+0000 7f3ac9606640 1 --2- 172.21.15.67:0/2343251079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ac4108660 0x7f3ac419f700 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.998+0000 7f3ac9606640 1 --2- 172.21.15.67:0/2343251079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ac4107d10 0x7f3ac419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.998+0000 7f3ac9606640 1 --2- 172.21.15.67:0/2343251079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ac41023e0 0x7f3ac4199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.998+0000 7f3ac9606640 1 -- 172.21.15.67:0/2343251079 >> 172.21.15.67:0/2343251079 conn(0x7f3ac40fddc0 msgr2=0x7f3ac406b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.998+0000 7f3ac9606640 1 -- 172.21.15.67:0/2343251079 shutdown_connections 2024-07-31T14:51:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:09.998+0000 7f3ac9606640 1 -- 172.21.15.67:0/2343251079 wait complete. 2024-07-31T14:51:10.012 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-07-31T14:51:10.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.128+0000 7feb3d18e640 1 -- 172.21.15.67:0/4160792803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb38104c80 msgr2=0x7feb3810d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:10.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.128+0000 7feb3d18e640 1 --2- 172.21.15.67:0/4160792803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb38104c80 0x7feb3810d1c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009900 tx=0x7feb2c02a410 comp rx=0 tx=0).stop 2024-07-31T14:51:10.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.128+0000 7feb3d18e640 1 -- 172.21.15.67:0/4160792803 shutdown_connections 2024-07-31T14:51:10.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.128+0000 7feb3d18e640 1 --2- 172.21.15.67:0/4160792803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb3810d790 0x7feb3810fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.128+0000 7feb3d18e640 1 --2- 172.21.15.67:0/4160792803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb38104c80 0x7feb3810d1c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.128+0000 7feb3d18e640 1 --2- 172.21.15.67:0/4160792803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb38104330 0x7feb38104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.128+0000 7feb3d18e640 1 -- 172.21.15.67:0/4160792803 >> 172.21.15.67:0/4160792803 conn(0x7feb380fddc0 msgr2=0x7feb38100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:10.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.128+0000 7feb3d18e640 1 -- 172.21.15.67:0/4160792803 shutdown_connections 2024-07-31T14:51:10.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.128+0000 7feb3d18e640 1 -- 172.21.15.67:0/4160792803 wait complete. 2024-07-31T14:51:10.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.128+0000 7feb3d18e640 1 Processor -- start 2024-07-31T14:51:10.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb3d18e640 1 -- start start 2024-07-31T14:51:10.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb3d18e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb38104330 0x7feb38199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:10.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb3d18e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb38104c80 0x7feb3819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:10.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb3d18e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb3810d790 0x7feb3819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:10.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb3d18e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feb381133d0 con 0x7feb38104c80 2024-07-31T14:51:10.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb3d18e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feb38113250 con 0x7feb38104330 2024-07-31T14:51:10.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb3d18e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feb38113550 con 0x7feb3810d790 2024-07-31T14:51:10.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb3659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb38104c80 0x7feb3819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:10.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb3659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb38104c80 0x7feb3819a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33966/0 (socket says 172.21.15.67:33966) 2024-07-31T14:51:10.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb3659c640 1 -- 172.21.15.67:0/3167901587 learned_addr learned my addr 172.21.15.67:0/3167901587 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:10.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb3759e640 1 --2- 172.21.15.67:0/3167901587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb3810d790 0x7feb3819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:10.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb36d9d640 1 --2- 172.21.15.67:0/3167901587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb38104330 0x7feb38199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:10.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb3659c640 1 -- 172.21.15.67:0/3167901587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb3810d790 msgr2=0x7feb3819f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:10.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb3659c640 1 --2- 172.21.15.67:0/3167901587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb3810d790 0x7feb3819f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.129+0000 7feb3659c640 1 -- 172.21.15.67:0/3167901587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb38104330 msgr2=0x7feb38199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:10.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.130+0000 7feb3659c640 1 --2- 172.21.15.67:0/3167901587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb38104330 0x7feb38199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.130+0000 7feb3659c640 1 -- 172.21.15.67:0/3167901587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3819fd20 con 0x7feb38104c80 2024-07-31T14:51:10.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.130+0000 7feb3659c640 1 --2- 172.21.15.67:0/3167901587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb38104c80 0x7feb3819a390 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7feb2c02a930 tx=0x7feb2c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:10.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.130+0000 7feb1ffff640 1 -- 172.21.15.67:0/3167901587 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c03f030 con 0x7feb38104c80 2024-07-31T14:51:10.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.130+0000 7feb1ffff640 1 -- 172.21.15.67:0/3167901587 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb2c003aa0 con 0x7feb38104c80 2024-07-31T14:51:10.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.130+0000 7feb1ffff640 1 -- 172.21.15.67:0/3167901587 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c035070 con 0x7feb38104c80 2024-07-31T14:51:10.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.130+0000 7feb3d18e640 1 -- 172.21.15.67:0/3167901587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3819ffb0 con 0x7feb38104c80 2024-07-31T14:51:10.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.130+0000 7feb3d18e640 1 -- 172.21.15.67:0/3167901587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb381a67a0 con 0x7feb38104c80 2024-07-31T14:51:10.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.131+0000 7feb3d18e640 1 -- 172.21.15.67:0/3167901587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb3806bb40 con 0x7feb38104c80 2024-07-31T14:51:10.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.133+0000 7feb1ffff640 1 -- 172.21.15.67:0/3167901587 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7feb2c042030 con 0x7feb38104c80 2024-07-31T14:51:10.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.134+0000 7feb1ffff640 1 --2- 172.21.15.67:0/3167901587 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feb200b3cb0 0x7feb200b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:10.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.134+0000 7feb36d9d640 1 --2- 172.21.15.67:0/3167901587 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feb200b3cb0 0x7feb200b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:10.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.134+0000 7feb1ffff640 1 -- 172.21.15.67:0/3167901587 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7feb2c104df0 con 0x7feb38104c80 2024-07-31T14:51:10.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.135+0000 7feb36d9d640 1 --2- 172.21.15.67:0/3167901587 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feb200b3cb0 0x7feb200b61c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7feb28005fd0 tx=0x7feb28005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:10.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.137+0000 7feb1ffff640 1 -- 172.21.15.67:0/3167901587 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7feb2c0cbeb0 con 0x7feb38104c80 2024-07-31T14:51:10.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.331+0000 7feb3d18e640 1 -- 172.21.15.67:0/3167901587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feb38108880 con 0x7feb38104c80 2024-07-31T14:51:10.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.331+0000 7feb1ffff640 1 -- 172.21.15.67:0/3167901587 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7feb2c0cb7b0 con 0x7feb38104c80 2024-07-31T14:51:10.334 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-07-31T14:51:10.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.333+0000 7feb3d18e640 1 -- 172.21.15.67:0/3167901587 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feb200b3cb0 msgr2=0x7feb200b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:10.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.333+0000 7feb3d18e640 1 --2- 172.21.15.67:0/3167901587 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feb200b3cb0 0x7feb200b61c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7feb28005fd0 tx=0x7feb28005850 comp rx=0 tx=0).stop 2024-07-31T14:51:10.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.333+0000 7feb3d18e640 1 -- 172.21.15.67:0/3167901587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb38104c80 msgr2=0x7feb3819a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:10.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.333+0000 7feb3d18e640 1 --2- 172.21.15.67:0/3167901587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb38104c80 0x7feb3819a390 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7feb2c02a930 tx=0x7feb2c004270 comp rx=0 tx=0).stop 2024-07-31T14:51:10.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.333+0000 7feb3d18e640 1 -- 172.21.15.67:0/3167901587 shutdown_connections 2024-07-31T14:51:10.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.333+0000 7feb3d18e640 1 --2- 172.21.15.67:0/3167901587 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feb200b3cb0 0x7feb200b61c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.333+0000 7feb3d18e640 1 --2- 172.21.15.67:0/3167901587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb3810d790 0x7feb3819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.333+0000 7feb3d18e640 1 --2- 172.21.15.67:0/3167901587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb38104c80 0x7feb3819a390 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.333+0000 7feb3d18e640 1 --2- 172.21.15.67:0/3167901587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb38104330 0x7feb38199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.333+0000 7feb3d18e640 1 -- 172.21.15.67:0/3167901587 >> 172.21.15.67:0/3167901587 conn(0x7feb380fddc0 msgr2=0x7feb3810faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:10.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.333+0000 7feb3d18e640 1 -- 172.21.15.67:0/3167901587 shutdown_connections 2024-07-31T14:51:10.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.333+0000 7feb3d18e640 1 -- 172.21.15.67:0/3167901587 wait complete. 2024-07-31T14:51:10.347 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-07-31T14:51:10.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.463+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/4273270108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c481023e0 msgr2=0x7f8c481027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:10.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.463+0000 7f8c4d7e8640 1 --2- 172.21.15.67:0/4273270108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c481023e0 0x7f8c481027f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009900 tx=0x7f8c3802a410 comp rx=0 tx=0).stop 2024-07-31T14:51:10.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.463+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/4273270108 shutdown_connections 2024-07-31T14:51:10.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.463+0000 7f8c4d7e8640 1 --2- 172.21.15.67:0/4273270108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c48108680 0x7f8c4810ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.463+0000 7f8c4d7e8640 1 --2- 172.21.15.67:0/4273270108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c48107d30 0x7f8c48108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.463+0000 7f8c4d7e8640 1 --2- 172.21.15.67:0/4273270108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c481023e0 0x7f8c481027f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.463+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/4273270108 >> 172.21.15.67:0/4273270108 conn(0x7f8c48078fa0 msgr2=0x7f8c480793b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:10.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.464+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/4273270108 shutdown_connections 2024-07-31T14:51:10.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.464+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/4273270108 wait complete. 2024-07-31T14:51:10.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.464+0000 7f8c4d7e8640 1 Processor -- start 2024-07-31T14:51:10.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.464+0000 7f8c4d7e8640 1 -- start start 2024-07-31T14:51:10.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c4d7e8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c481023e0 0x7f8c481989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:10.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c4d7e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c48107d30 0x7f8c48198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:10.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c4d7e8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48108680 0x7f8c48199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:10.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c4d7e8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8c481122f0 con 0x7f8c48108680 2024-07-31T14:51:10.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c4d7e8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8c48112170 con 0x7f8c481023e0 2024-07-31T14:51:10.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c4d7e8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8c48112470 con 0x7f8c48107d30 2024-07-31T14:51:10.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c477fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48108680 0x7f8c48199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:10.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c477fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48108680 0x7f8c48199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33976/0 (socket says 172.21.15.67:33976) 2024-07-31T14:51:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c477fe640 1 -- 172.21.15.67:0/3633669643 learned_addr learned my addr 172.21.15.67:0/3633669643 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c46ffd640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c481023e0 0x7f8c481989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c467fc640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c48107d30 0x7f8c48198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c477fe640 1 -- 172.21.15.67:0/3633669643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c48107d30 msgr2=0x7f8c48198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c477fe640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c48107d30 0x7f8c48198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c477fe640 1 -- 172.21.15.67:0/3633669643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c481023e0 msgr2=0x7f8c481989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c477fe640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c481023e0 0x7f8c481989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c477fe640 1 -- 172.21.15.67:0/3633669643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c481a80a0 con 0x7f8c48108680 2024-07-31T14:51:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c46ffd640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c481023e0 0x7f8c481989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c477fe640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48108680 0x7f8c48199470 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000eb10 tx=0x7f8c4000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.465+0000 7f8c2ffff640 1 -- 172.21.15.67:0/3633669643 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40004b20 con 0x7f8c48108680 2024-07-31T14:51:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.466+0000 7f8c2ffff640 1 -- 172.21.15.67:0/3633669643 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c40004cc0 con 0x7f8c48108680 2024-07-31T14:51:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.466+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/3633669643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c481a8390 con 0x7f8c48108680 2024-07-31T14:51:10.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.466+0000 7f8c2ffff640 1 -- 172.21.15.67:0/3633669643 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40010690 con 0x7f8c48108680 2024-07-31T14:51:10.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.466+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/3633669643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c481a8890 con 0x7f8c48108680 2024-07-31T14:51:10.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.467+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/3633669643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c481027f0 con 0x7f8c48108680 2024-07-31T14:51:10.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.468+0000 7f8c2ffff640 1 -- 172.21.15.67:0/3633669643 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f8c40005020 con 0x7f8c48108680 2024-07-31T14:51:10.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.469+0000 7f8c2ffff640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c300b3cb0 0x7f8c300b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:10.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.469+0000 7f8c46ffd640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c300b3cb0 0x7f8c300b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:10.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.469+0000 7f8c2ffff640 1 -- 172.21.15.67:0/3633669643 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f8c400e4980 con 0x7f8c48108680 2024-07-31T14:51:10.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.469+0000 7f8c46ffd640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c300b3cb0 0x7f8c300b61c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8c38000c00 tx=0x7f8c38009680 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:10.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.473+0000 7f8c2ffff640 1 -- 172.21.15.67:0/3633669643 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f8c400aba40 con 0x7f8c48108680 2024-07-31T14:51:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.667+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/3633669643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8c4802e0c0 con 0x7f8c48108680 2024-07-31T14:51:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.668+0000 7f8c2ffff640 1 -- 172.21.15.67:0/3633669643 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f8c400ab340 con 0x7f8c48108680 2024-07-31T14:51:10.669 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-07-31T14:51:10.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.670+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/3633669643 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c300b3cb0 msgr2=0x7f8c300b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:10.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.670+0000 7f8c4d7e8640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c300b3cb0 0x7f8c300b61c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8c38000c00 tx=0x7f8c38009680 comp rx=0 tx=0).stop 2024-07-31T14:51:10.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.670+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/3633669643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48108680 msgr2=0x7f8c48199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:10.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.670+0000 7f8c4d7e8640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48108680 0x7f8c48199470 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000eb10 tx=0x7f8c4000c2d0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.670+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/3633669643 shutdown_connections 2024-07-31T14:51:10.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.670+0000 7f8c4d7e8640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c300b3cb0 0x7f8c300b61c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.670+0000 7f8c4d7e8640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c48108680 0x7f8c48199470 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.670+0000 7f8c4d7e8640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c48107d30 0x7f8c48198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.670+0000 7f8c4d7e8640 1 --2- 172.21.15.67:0/3633669643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c481023e0 0x7f8c481989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:10.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.670+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/3633669643 >> 172.21.15.67:0/3633669643 conn(0x7f8c48078fa0 msgr2=0x7f8c48105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:10.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.670+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/3633669643 shutdown_connections 2024-07-31T14:51:10.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:10.670+0000 7f8c4d7e8640 1 -- 172.21.15.67:0/3633669643 wait complete. 2024-07-31T14:51:10.682 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:10 smithi067 ceph-mon[32750]: pgmap v402: 129 pgs: 129 active+clean; 648 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-31T14:51:10.682 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:10 smithi067 ceph-mon[32750]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-07-31T14:51:10.682 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:10 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2092026954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-31T14:51:10.682 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:10 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2343251079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-31T14:51:10.682 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:10 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3167901587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-31T14:51:10.683 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-07-31T14:51:10.683 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:51:10.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:10 smithi073 ceph-mon[36714]: pgmap v402: 129 pgs: 129 active+clean; 648 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-31T14:51:10.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:10 smithi073 ceph-mon[36714]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-07-31T14:51:10.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:10 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2092026954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-31T14:51:10.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:10 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2343251079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-31T14:51:10.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:10 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3167901587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-31T14:51:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:10 smithi077 ceph-mon[37327]: pgmap v402: 129 pgs: 129 active+clean; 648 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-31T14:51:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:10 smithi077 ceph-mon[37327]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-07-31T14:51:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:10 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2092026954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-31T14:51:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:10 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2343251079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-31T14:51:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:10 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3167901587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-31T14:51:11.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:11 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3633669643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-31T14:51:11.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:11 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3633669643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-31T14:51:11.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:11 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3633669643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-31T14:51:12.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:12 smithi067 ceph-mon[32750]: pgmap v403: 129 pgs: 129 active+clean; 653 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-31T14:51:12.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.750+0000 7f3ac17a9640 1 -- 172.21.15.67:0/3631450420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3abc1024d0 msgr2=0x7f3abc1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:12.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.750+0000 7f3ac17a9640 1 --2- 172.21.15.67:0/3631450420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3abc1024d0 0x7f3abc1028e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009900 tx=0x7f3ab402a410 comp rx=0 tx=0).stop 2024-07-31T14:51:12.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.750+0000 7f3ac17a9640 1 -- 172.21.15.67:0/3631450420 shutdown_connections 2024-07-31T14:51:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.750+0000 7f3ac17a9640 1 --2- 172.21.15.67:0/3631450420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3abc108760 0x7f3abc10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.750+0000 7f3ac17a9640 1 --2- 172.21.15.67:0/3631450420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3abc107e10 0x7f3abc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.750+0000 7f3ac17a9640 1 --2- 172.21.15.67:0/3631450420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3abc1024d0 0x7f3abc1028e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.750+0000 7f3ac17a9640 1 -- 172.21.15.67:0/3631450420 >> 172.21.15.67:0/3631450420 conn(0x7f3abc079210 msgr2=0x7f3abc079620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.750+0000 7f3ac17a9640 1 -- 172.21.15.67:0/3631450420 shutdown_connections 2024-07-31T14:51:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.750+0000 7f3ac17a9640 1 -- 172.21.15.67:0/3631450420 wait complete. 2024-07-31T14:51:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.751+0000 7f3ac17a9640 1 Processor -- start 2024-07-31T14:51:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.751+0000 7f3ac17a9640 1 -- start start 2024-07-31T14:51:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3ac17a9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3abc1024d0 0x7f3abc103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3ac17a9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3abc107e10 0x7f3abc104200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3ac17a9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3abc108760 0x7f3abc104740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3ac17a9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc1123e0 con 0x7f3abc1024d0 2024-07-31T14:51:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3ac17a9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc112260 con 0x7f3abc108760 2024-07-31T14:51:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3ac17a9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc112560 con 0x7f3abc107e10 2024-07-31T14:51:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3abaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3abc1024d0 0x7f3abc103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3abaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3abc1024d0 0x7f3abc103cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52676/0 (socket says 172.21.15.67:52676) 2024-07-31T14:51:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3abaffd640 1 -- 172.21.15.67:0/1208324868 learned_addr learned my addr 172.21.15.67:0/1208324868 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3aba7fc640 1 --2- 172.21.15.67:0/1208324868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3abc107e10 0x7f3abc104200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3abb7fe640 1 --2- 172.21.15.67:0/1208324868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3abc108760 0x7f3abc104740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3abaffd640 1 -- 172.21.15.67:0/1208324868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3abc107e10 msgr2=0x7f3abc104200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3abaffd640 1 --2- 172.21.15.67:0/1208324868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3abc107e10 0x7f3abc104200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3abaffd640 1 -- 172.21.15.67:0/1208324868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3abc108760 msgr2=0x7f3abc104740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3abaffd640 1 --2- 172.21.15.67:0/1208324868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3abc108760 0x7f3abc104740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3abaffd640 1 -- 172.21.15.67:0/1208324868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3abc1a81a0 con 0x7f3abc1024d0 2024-07-31T14:51:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.752+0000 7f3abaffd640 1 --2- 172.21.15.67:0/1208324868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3abc1024d0 0x7f3abc103cc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4005940 tx=0x7f3ab40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.753+0000 7f3ab3fff640 1 -- 172.21.15.67:0/1208324868 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab403f030 con 0x7f3abc1024d0 2024-07-31T14:51:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.753+0000 7f3ab3fff640 1 -- 172.21.15.67:0/1208324868 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ab4003cb0 con 0x7f3abc1024d0 2024-07-31T14:51:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.753+0000 7f3ac17a9640 1 -- 172.21.15.67:0/1208324868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3abc1a8430 con 0x7f3abc1024d0 2024-07-31T14:51:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.753+0000 7f3ab3fff640 1 -- 172.21.15.67:0/1208324868 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4035070 con 0x7f3abc1024d0 2024-07-31T14:51:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.753+0000 7f3ac17a9640 1 -- 172.21.15.67:0/1208324868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3abc1a8910 con 0x7f3abc1024d0 2024-07-31T14:51:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.754+0000 7f3ac17a9640 1 -- 172.21.15.67:0/1208324868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3abc1028e0 con 0x7f3abc1024d0 2024-07-31T14:51:12.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.755+0000 7f3ab3fff640 1 -- 172.21.15.67:0/1208324868 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f3ab4042030 con 0x7f3abc1024d0 2024-07-31T14:51:12.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.756+0000 7f3ab3fff640 1 --2- 172.21.15.67:0/1208324868 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aa80b3cb0 0x7f3aa80b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:12.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.756+0000 7f3aba7fc640 1 --2- 172.21.15.67:0/1208324868 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aa80b3cb0 0x7f3aa80b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:12.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.756+0000 7f3ab3fff640 1 -- 172.21.15.67:0/1208324868 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f3ab4104d80 con 0x7f3abc1024d0 2024-07-31T14:51:12.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.757+0000 7f3aba7fc640 1 --2- 172.21.15.67:0/1208324868 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aa80b3cb0 0x7f3aa80b61c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3abc1008a0 tx=0x7f3aa400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:12.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.759+0000 7f3ab3fff640 1 -- 172.21.15.67:0/1208324868 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f3ab40cb7f0 con 0x7f3abc1024d0 2024-07-31T14:51:12.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:12 smithi073 ceph-mon[36714]: pgmap v403: 129 pgs: 129 active+clean; 653 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-31T14:51:12.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:12 smithi077 ceph-mon[37327]: pgmap v403: 129 pgs: 129 active+clean; 653 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-31T14:51:12.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.994+0000 7f3ac17a9640 1 -- 172.21.15.67:0/1208324868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3abc1a8cc0 con 0x7f3abc1024d0 2024-07-31T14:51:12.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.995+0000 7f3ab3fff640 1 -- 172.21.15.67:0/1208324868 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11962 (secure 0 0 0) 0x7f3ab40d4110 con 0x7f3abc1024d0 2024-07-31T14:51:12.996 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:51:12.997 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-31T14:51:08:515297+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:51:07.734542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:51:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-31T14:51:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.997+0000 7f3ac17a9640 1 -- 172.21.15.67:0/1208324868 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aa80b3cb0 msgr2=0x7f3aa80b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.997+0000 7f3ac17a9640 1 --2- 172.21.15.67:0/1208324868 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aa80b3cb0 0x7f3aa80b61c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3abc1008a0 tx=0x7f3aa400b390 comp rx=0 tx=0).stop 2024-07-31T14:51:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.997+0000 7f3ac17a9640 1 -- 172.21.15.67:0/1208324868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3abc1024d0 msgr2=0x7f3abc103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.997+0000 7f3ac17a9640 1 --2- 172.21.15.67:0/1208324868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3abc1024d0 0x7f3abc103cc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4005940 tx=0x7f3ab40042f0 comp rx=0 tx=0).stop 2024-07-31T14:51:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.997+0000 7f3ac17a9640 1 -- 172.21.15.67:0/1208324868 shutdown_connections 2024-07-31T14:51:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.997+0000 7f3ac17a9640 1 --2- 172.21.15.67:0/1208324868 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3aa80b3cb0 0x7f3aa80b61c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.997+0000 7f3ac17a9640 1 --2- 172.21.15.67:0/1208324868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3abc108760 0x7f3abc104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.997+0000 7f3ac17a9640 1 --2- 172.21.15.67:0/1208324868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3abc107e10 0x7f3abc104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.997+0000 7f3ac17a9640 1 --2- 172.21.15.67:0/1208324868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3abc1024d0 0x7f3abc103cc0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.997+0000 7f3ac17a9640 1 -- 172.21.15.67:0/1208324868 >> 172.21.15.67:0/1208324868 conn(0x7f3abc079210 msgr2=0x7f3abc101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.997+0000 7f3ac17a9640 1 -- 172.21.15.67:0/1208324868 shutdown_connections 2024-07-31T14:51:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:12.997+0000 7f3ac17a9640 1 -- 172.21.15.67:0/1208324868 wait complete. 2024-07-31T14:51:13.706 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-07-31T14:51:13.707 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-07-31T14:51:13.707 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-31T14:51:13.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.823+0000 7fbbd4e25640 1 -- 172.21.15.67:0/394069092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd0101bb0 msgr2=0x7fbbd0101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:13.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.823+0000 7fbbd4e25640 1 --2- 172.21.15.67:0/394069092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd0101bb0 0x7fbbd0101fc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009900 tx=0x7fbbc002a410 comp rx=0 tx=0).stop 2024-07-31T14:51:13.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.823+0000 7fbbd4e25640 1 -- 172.21.15.67:0/394069092 shutdown_connections 2024-07-31T14:51:13.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.823+0000 7fbbd4e25640 1 --2- 172.21.15.67:0/394069092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbd0107e30 0x7fbbd010e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:13.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.823+0000 7fbbd4e25640 1 --2- 172.21.15.67:0/394069092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbd01074e0 0x7fbbd01078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:13.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.823+0000 7fbbd4e25640 1 --2- 172.21.15.67:0/394069092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd0101bb0 0x7fbbd0101fc0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:13.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.823+0000 7fbbd4e25640 1 -- 172.21.15.67:0/394069092 >> 172.21.15.67:0/394069092 conn(0x7fbbd00798a0 msgr2=0x7fbbd0079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:13.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.823+0000 7fbbd4e25640 1 -- 172.21.15.67:0/394069092 shutdown_connections 2024-07-31T14:51:13.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.823+0000 7fbbd4e25640 1 -- 172.21.15.67:0/394069092 wait complete. 2024-07-31T14:51:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.824+0000 7fbbd4e25640 1 Processor -- start 2024-07-31T14:51:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.824+0000 7fbbd4e25640 1 -- start start 2024-07-31T14:51:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.824+0000 7fbbd4e25640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd0101bb0 0x7fbbd0199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.824+0000 7fbbd4e25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbd01074e0 0x7fbbd019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.824+0000 7fbbd4e25640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbd0107e30 0x7fbbd019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.824+0000 7fbbd4e25640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd0113530 con 0x7fbbd0101bb0 2024-07-31T14:51:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.824+0000 7fbbd4e25640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd01133b0 con 0x7fbbd0107e30 2024-07-31T14:51:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.824+0000 7fbbd4e25640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd01136b0 con 0x7fbbd01074e0 2024-07-31T14:51:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.824+0000 7fbbce59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd0101bb0 0x7fbbd0199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.824+0000 7fbbce59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd0101bb0 0x7fbbd0199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52706/0 (socket says 172.21.15.67:52706) 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.824+0000 7fbbce59c640 1 -- 172.21.15.67:0/4212492581 learned_addr learned my addr 172.21.15.67:0/4212492581 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.824+0000 7fbbced9d640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbd0107e30 0x7fbbd019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.824+0000 7fbbcdd9b640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbd01074e0 0x7fbbd019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.824+0000 7fbbce59c640 1 -- 172.21.15.67:0/4212492581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbd01074e0 msgr2=0x7fbbd019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.825+0000 7fbbce59c640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbd01074e0 0x7fbbd019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.825+0000 7fbbce59c640 1 -- 172.21.15.67:0/4212492581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbd0107e30 msgr2=0x7fbbd019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.825+0000 7fbbce59c640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbd0107e30 0x7fbbd019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.825+0000 7fbbce59c640 1 -- 172.21.15.67:0/4212492581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd019fe20 con 0x7fbbd0101bb0 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.825+0000 7fbbcdd9b640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbd01074e0 0x7fbbd019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.825+0000 7fbbce59c640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd0101bb0 0x7fbbd0199ef0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0005ab0 tx=0x7fbbc0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.825+0000 7fbbbf7fe640 1 -- 172.21.15.67:0/4212492581 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc003f070 con 0x7fbbd0101bb0 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.825+0000 7fbbbf7fe640 1 -- 172.21.15.67:0/4212492581 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc0031d60 con 0x7fbbd0101bb0 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.825+0000 7fbbd4e25640 1 -- 172.21.15.67:0/4212492581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd01a00b0 con 0x7fbbd0101bb0 2024-07-31T14:51:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.825+0000 7fbbbf7fe640 1 -- 172.21.15.67:0/4212492581 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0035030 con 0x7fbbd0101bb0 2024-07-31T14:51:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.825+0000 7fbbd4e25640 1 -- 172.21.15.67:0/4212492581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd01a68a0 con 0x7fbbd0101bb0 2024-07-31T14:51:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.826+0000 7fbbd4e25640 1 -- 172.21.15.67:0/4212492581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd0101fc0 con 0x7fbbd0101bb0 2024-07-31T14:51:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.828+0000 7fbbbf7fe640 1 -- 172.21.15.67:0/4212492581 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7fbbc0042070 con 0x7fbbd0101bb0 2024-07-31T14:51:13.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.828+0000 7fbbbf7fe640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbbb80b3cb0 0x7fbbb80b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:13.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.829+0000 7fbbcdd9b640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbbb80b3cb0 0x7fbbb80b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:13.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.829+0000 7fbbbf7fe640 1 -- 172.21.15.67:0/4212492581 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fbbc0105240 con 0x7fbbd0101bb0 2024-07-31T14:51:13.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.829+0000 7fbbcdd9b640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbbb80b3cb0 0x7fbbb80b61c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0107170 tx=0x7fbbc4008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:13.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:13.832+0000 7fbbbf7fe640 1 -- 172.21.15.67:0/4212492581 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fbbc00cc300 con 0x7fbbd0101bb0 2024-07-31T14:51:13.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:13 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1208324868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:13.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:13 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1208324868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:14.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.025+0000 7fbbd4e25640 1 -- 172.21.15.67:0/4212492581 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbbd0109c30 con 0x7fbbb80b3cb0 2024-07-31T14:51:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.031+0000 7fbbbf7fe640 1 -- 172.21.15.67:0/4212492581 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340533 (secure 0 0 0) 0x7fbbd0109c30 con 0x7fbbb80b3cb0 2024-07-31T14:51:14.038 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:51:14.040 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-31T14:51:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.035+0000 7fbbd4e25640 1 -- 172.21.15.67:0/4212492581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbbb80b3cb0 msgr2=0x7fbbb80b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.035+0000 7fbbd4e25640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbbb80b3cb0 0x7fbbb80b61c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0107170 tx=0x7fbbc4008040 comp rx=0 tx=0).stop 2024-07-31T14:51:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.035+0000 7fbbd4e25640 1 -- 172.21.15.67:0/4212492581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd0101bb0 msgr2=0x7fbbd0199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.035+0000 7fbbd4e25640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd0101bb0 0x7fbbd0199ef0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0005ab0 tx=0x7fbbc0003d50 comp rx=0 tx=0).stop 2024-07-31T14:51:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.035+0000 7fbbd4e25640 1 -- 172.21.15.67:0/4212492581 shutdown_connections 2024-07-31T14:51:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.035+0000 7fbbd4e25640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbbb80b3cb0 0x7fbbb80b61c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.035+0000 7fbbd4e25640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbd0107e30 0x7fbbd019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.035+0000 7fbbd4e25640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbd01074e0 0x7fbbd019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.035+0000 7fbbd4e25640 1 --2- 172.21.15.67:0/4212492581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd0101bb0 0x7fbbd0199ef0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.035+0000 7fbbd4e25640 1 -- 172.21.15.67:0/4212492581 >> 172.21.15.67:0/4212492581 conn(0x7fbbd00798a0 msgr2=0x7fbbd0105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.035+0000 7fbbd4e25640 1 -- 172.21.15.67:0/4212492581 shutdown_connections 2024-07-31T14:51:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.035+0000 7fbbd4e25640 1 -- 172.21.15.67:0/4212492581 wait complete. 2024-07-31T14:51:14.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:13 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1208324868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:14.048 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-07-31T14:51:13.443366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669108,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2009,"num_read_kb":1120,"num_write":401,"num_write_kb":2321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":830,"ondisk_log_size":830,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384108,"kb_used_data":14156,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557716,"statfs":{"total":1151940427776,"available":1151547101184,"internally_reserved":0,"allocated":14495744,"data_stored":8209727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":378713403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6103,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":68,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003318"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":245,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:10.893183+0000","last_change":"2024-07-31T14:47:00.020067+0000","last_active":"2024-07-31T14:51:10.893183+0000","last_peered":"2024-07-31T14:51:10.893183+0000","last_clean":"2024-07-31T14:51:10.893183+0000","last_became_active":"2024-07-31T14:47:00.019674+0000","last_became_peered":"2024-07-31T14:47:00.019674+0000","last_unstale":"2024-07-31T14:51:10.893183+0000","last_undegraded":"2024-07-31T14:51:10.893183+0000","last_fullsized":"2024-07-31T14:51:10.893183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:10:08.193786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":189,"num_read_kb":189,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.408739+0000","last_change":"2024-07-31T14:47:03.052069+0000","last_active":"2024-07-31T14:49:49.408739+0000","last_peered":"2024-07-31T14:49:49.408739+0000","last_clean":"2024-07-31T14:49:49.408739+0000","last_became_active":"2024-07-31T14:47:03.051739+0000","last_became_peered":"2024-07-31T14:47:03.051739+0000","last_unstale":"2024-07-31T14:49:49.408739+0000","last_undegraded":"2024-07-31T14:49:49.408739+0000","last_fullsized":"2024-07-31T14:49:49.408739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:29:19.952794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.629384+0000","last_change":"2024-07-31T14:47:20.628770+0000","last_active":"2024-07-31T14:47:20.629384+0000","last_peered":"2024-07-31T14:47:20.629384+0000","last_clean":"2024-07-31T14:47:20.629384+0000","last_became_active":"2024-07-31T14:47:20.628142+0000","last_became_peered":"2024-07-31T14:47:20.628142+0000","last_unstale":"2024-07-31T14:47:20.629384+0000","last_undegraded":"2024-07-31T14:47:20.629384+0000","last_fullsized":"2024-07-31T14:47:20.629384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.659670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:59.291402+0000","last_change":"2024-07-31T14:47:00.183893+0000","last_active":"2024-07-31T14:48:59.291402+0000","last_peered":"2024-07-31T14:48:59.291402+0000","last_clean":"2024-07-31T14:48:59.291402+0000","last_became_active":"2024-07-31T14:47:00.183453+0000","last_became_peered":"2024-07-31T14:47:00.183453+0000","last_unstale":"2024-07-31T14:48:59.291402+0000","last_undegraded":"2024-07-31T14:48:59.291402+0000","last_fullsized":"2024-07-31T14:48:59.291402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:45:08.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404252+0000","last_change":"2024-07-31T14:47:03.057081+0000","last_active":"2024-07-31T14:49:49.404252+0000","last_peered":"2024-07-31T14:49:49.404252+0000","last_clean":"2024-07-31T14:49:49.404252+0000","last_became_active":"2024-07-31T14:47:03.056061+0000","last_became_peered":"2024-07-31T14:47:03.056061+0000","last_unstale":"2024-07-31T14:49:49.404252+0000","last_undegraded":"2024-07-31T14:49:49.404252+0000","last_fullsized":"2024-07-31T14:49:49.404252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:28:16.760131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604121+0000","last_change":"2024-07-31T14:47:13.105271+0000","last_active":"2024-07-31T14:47:20.604121+0000","last_peered":"2024-07-31T14:47:20.604121+0000","last_clean":"2024-07-31T14:47:20.604121+0000","last_became_active":"2024-07-31T14:47:13.104717+0000","last_became_peered":"2024-07-31T14:47:13.104717+0000","last_unstale":"2024-07-31T14:47:20.604121+0000","last_undegraded":"2024-07-31T14:47:20.604121+0000","last_fullsized":"2024-07-31T14:47:20.604121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:34:21.413957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:04.594438+0000","last_change":"2024-07-31T14:47:20.454745+0000","last_active":"2024-07-31T14:48:04.594438+0000","last_peered":"2024-07-31T14:48:04.594438+0000","last_clean":"2024-07-31T14:48:04.594438+0000","last_became_active":"2024-07-31T14:47:20.449892+0000","last_became_peered":"2024-07-31T14:47:20.449892+0000","last_unstale":"2024-07-31T14:48:04.594438+0000","last_undegraded":"2024-07-31T14:48:04.594438+0000","last_fullsized":"2024-07-31T14:48:04.594438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:59:17.375344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.412662+0000","last_change":"2024-07-31T14:47:03.051085+0000","last_active":"2024-07-31T14:49:49.412662+0000","last_peered":"2024-07-31T14:49:49.412662+0000","last_clean":"2024-07-31T14:49:49.412662+0000","last_became_active":"2024-07-31T14:47:03.050686+0000","last_became_peered":"2024-07-31T14:47:03.050686+0000","last_unstale":"2024-07-31T14:49:49.412662+0000","last_undegraded":"2024-07-31T14:49:49.412662+0000","last_fullsized":"2024-07-31T14:49:49.412662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:48:02.494061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604501+0000","last_change":"2024-07-31T14:47:13.104229+0000","last_active":"2024-07-31T14:47:20.604501+0000","last_peered":"2024-07-31T14:47:20.604501+0000","last_clean":"2024-07-31T14:47:20.604501+0000","last_became_active":"2024-07-31T14:47:13.103737+0000","last_became_peered":"2024-07-31T14:47:13.103737+0000","last_unstale":"2024-07-31T14:47:20.604501+0000","last_undegraded":"2024-07-31T14:47:20.604501+0000","last_fullsized":"2024-07-31T14:47:20.604501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:36:00.541345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609858+0000","last_change":"2024-07-31T14:47:00.028169+0000","last_active":"2024-07-31T14:48:03.609858+0000","last_peered":"2024-07-31T14:48:03.609858+0000","last_clean":"2024-07-31T14:48:03.609858+0000","last_became_active":"2024-07-31T14:47:00.027757+0000","last_became_peered":"2024-07-31T14:47:00.027757+0000","last_unstale":"2024-07-31T14:48:03.609858+0000","last_undegraded":"2024-07-31T14:48:03.609858+0000","last_fullsized":"2024-07-31T14:48:03.609858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:36:04.419551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413724+0000","last_change":"2024-07-31T14:47:03.057400+0000","last_active":"2024-07-31T14:49:49.413724+0000","last_peered":"2024-07-31T14:49:49.413724+0000","last_clean":"2024-07-31T14:49:49.413724+0000","last_became_active":"2024-07-31T14:47:03.057035+0000","last_became_peered":"2024-07-31T14:47:03.057035+0000","last_unstale":"2024-07-31T14:49:49.413724+0000","last_undegraded":"2024-07-31T14:49:49.413724+0000","last_fullsized":"2024-07-31T14:49:49.413724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:55:07.951076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604244+0000","last_change":"2024-07-31T14:47:13.106093+0000","last_active":"2024-07-31T14:47:20.604244+0000","last_peered":"2024-07-31T14:47:20.604244+0000","last_clean":"2024-07-31T14:47:20.604244+0000","last_became_active":"2024-07-31T14:47:13.105459+0000","last_became_peered":"2024-07-31T14:47:13.105459+0000","last_unstale":"2024-07-31T14:47:20.604244+0000","last_undegraded":"2024-07-31T14:47:20.604244+0000","last_fullsized":"2024-07-31T14:47:20.604244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:07:52.751245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414377+0000","last_change":"2024-07-31T14:47:03.058701+0000","last_active":"2024-07-31T14:49:49.414377+0000","last_peered":"2024-07-31T14:49:49.414377+0000","last_clean":"2024-07-31T14:49:49.414377+0000","last_became_active":"2024-07-31T14:47:03.058359+0000","last_became_peered":"2024-07-31T14:47:03.058359+0000","last_unstale":"2024-07-31T14:49:49.414377+0000","last_undegraded":"2024-07-31T14:49:49.414377+0000","last_fullsized":"2024-07-31T14:49:49.414377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:37:36.149884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605126+0000","last_change":"2024-07-31T14:47:00.022983+0000","last_active":"2024-07-31T14:48:03.605126+0000","last_peered":"2024-07-31T14:48:03.605126+0000","last_clean":"2024-07-31T14:48:03.605126+0000","last_became_active":"2024-07-31T14:47:00.022586+0000","last_became_peered":"2024-07-31T14:47:00.022586+0000","last_unstale":"2024-07-31T14:48:03.605126+0000","last_undegraded":"2024-07-31T14:48:03.605126+0000","last_fullsized":"2024-07-31T14:48:03.605126+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:29:42.641490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604474+0000","last_change":"2024-07-31T14:47:13.363148+0000","last_active":"2024-07-31T14:47:20.604474+0000","last_peered":"2024-07-31T14:47:20.604474+0000","last_clean":"2024-07-31T14:47:20.604474+0000","last_became_active":"2024-07-31T14:47:13.362577+0000","last_became_peered":"2024-07-31T14:47:13.362577+0000","last_unstale":"2024-07-31T14:47:20.604474+0000","last_undegraded":"2024-07-31T14:47:20.604474+0000","last_fullsized":"2024-07-31T14:47:20.604474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:37:59.003260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409563+0000","last_change":"2024-07-31T14:47:03.049586+0000","last_active":"2024-07-31T14:49:49.409563+0000","last_peered":"2024-07-31T14:49:49.409563+0000","last_clean":"2024-07-31T14:49:49.409563+0000","last_became_active":"2024-07-31T14:47:03.049154+0000","last_became_peered":"2024-07-31T14:47:03.049154+0000","last_unstale":"2024-07-31T14:49:49.409563+0000","last_undegraded":"2024-07-31T14:49:49.409563+0000","last_fullsized":"2024-07-31T14:49:49.409563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:51:01.971854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'5","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:07.609943+0000","last_change":"2024-07-31T14:47:00.021254+0000","last_active":"2024-07-31T14:51:07.609943+0000","last_peered":"2024-07-31T14:51:07.609943+0000","last_clean":"2024-07-31T14:51:07.609943+0000","last_became_active":"2024-07-31T14:47:00.020690+0000","last_became_peered":"2024-07-31T14:47:00.020690+0000","last_unstale":"2024-07-31T14:51:07.609943+0000","last_undegraded":"2024-07-31T14:51:07.609943+0000","last_fullsized":"2024-07-31T14:51:07.609943+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:01:27.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:18.046730+0000","last_change":"2024-07-31T14:47:13.102220+0000","last_active":"2024-07-31T14:48:18.046730+0000","last_peered":"2024-07-31T14:48:18.046730+0000","last_clean":"2024-07-31T14:48:18.046730+0000","last_became_active":"2024-07-31T14:47:13.101764+0000","last_became_peered":"2024-07-31T14:47:13.101764+0000","last_unstale":"2024-07-31T14:48:18.046730+0000","last_undegraded":"2024-07-31T14:48:18.046730+0000","last_fullsized":"2024-07-31T14:48:18.046730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:56:44.335340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.397183+0000","last_change":"2024-07-31T14:47:03.052348+0000","last_active":"2024-07-31T14:49:49.397183+0000","last_peered":"2024-07-31T14:49:49.397183+0000","last_clean":"2024-07-31T14:49:49.397183+0000","last_became_active":"2024-07-31T14:47:03.051961+0000","last_became_peered":"2024-07-31T14:47:03.051961+0000","last_unstale":"2024-07-31T14:49:49.397183+0000","last_undegraded":"2024-07-31T14:49:49.397183+0000","last_fullsized":"2024-07-31T14:49:49.397183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:19:35.279809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.597386+0000","last_change":"2024-07-31T14:47:00.025013+0000","last_active":"2024-07-31T14:48:02.597386+0000","last_peered":"2024-07-31T14:48:02.597386+0000","last_clean":"2024-07-31T14:48:02.597386+0000","last_became_active":"2024-07-31T14:47:00.024637+0000","last_became_peered":"2024-07-31T14:47:00.024637+0000","last_unstale":"2024-07-31T14:48:02.597386+0000","last_undegraded":"2024-07-31T14:48:02.597386+0000","last_fullsized":"2024-07-31T14:48:02.597386+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:07:21.519549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630339+0000","last_change":"2024-07-31T14:47:20.629262+0000","last_active":"2024-07-31T14:47:20.630339+0000","last_peered":"2024-07-31T14:47:20.630339+0000","last_clean":"2024-07-31T14:47:20.630339+0000","last_became_active":"2024-07-31T14:47:20.628318+0000","last_became_peered":"2024-07-31T14:47:20.628318+0000","last_unstale":"2024-07-31T14:47:20.630339+0000","last_undegraded":"2024-07-31T14:47:20.630339+0000","last_fullsized":"2024-07-31T14:47:20.630339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.206705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.361650+0000","last_change":"2024-07-31T14:47:03.051527+0000","last_active":"2024-07-31T14:49:49.361650+0000","last_peered":"2024-07-31T14:49:49.361650+0000","last_clean":"2024-07-31T14:49:49.361650+0000","last_became_active":"2024-07-31T14:47:03.051139+0000","last_became_peered":"2024-07-31T14:47:03.051139+0000","last_unstale":"2024-07-31T14:49:49.361650+0000","last_undegraded":"2024-07-31T14:49:49.361650+0000","last_fullsized":"2024-07-31T14:49:49.361650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:53:11.962719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:59.566389+0000","last_change":"2024-07-31T14:47:00.183812+0000","last_active":"2024-07-31T14:47:59.566389+0000","last_peered":"2024-07-31T14:47:59.566389+0000","last_clean":"2024-07-31T14:47:59.566389+0000","last_became_active":"2024-07-31T14:47:00.183440+0000","last_became_peered":"2024-07-31T14:47:00.183440+0000","last_unstale":"2024-07-31T14:47:59.566389+0000","last_undegraded":"2024-07-31T14:47:59.566389+0000","last_fullsized":"2024-07-31T14:47:59.566389+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:47:23.166756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604338+0000","last_change":"2024-07-31T14:47:13.363563+0000","last_active":"2024-07-31T14:47:20.604338+0000","last_peered":"2024-07-31T14:47:20.604338+0000","last_clean":"2024-07-31T14:47:20.604338+0000","last_became_active":"2024-07-31T14:47:13.362697+0000","last_became_peered":"2024-07-31T14:47:13.362697+0000","last_unstale":"2024-07-31T14:47:20.604338+0000","last_undegraded":"2024-07-31T14:47:20.604338+0000","last_fullsized":"2024-07-31T14:47:20.604338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:26:38.292073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407720+0000","last_change":"2024-07-31T14:47:03.046763+0000","last_active":"2024-07-31T14:49:49.407720+0000","last_peered":"2024-07-31T14:49:49.407720+0000","last_clean":"2024-07-31T14:49:49.407720+0000","last_became_active":"2024-07-31T14:47:03.046408+0000","last_became_peered":"2024-07-31T14:47:03.046408+0000","last_unstale":"2024-07-31T14:49:49.407720+0000","last_undegraded":"2024-07-31T14:49:49.407720+0000","last_fullsized":"2024-07-31T14:49:49.407720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:43:22.838495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605677+0000","last_change":"2024-07-31T14:47:00.236075+0000","last_active":"2024-07-31T14:48:03.605677+0000","last_peered":"2024-07-31T14:48:03.605677+0000","last_clean":"2024-07-31T14:48:03.605677+0000","last_became_active":"2024-07-31T14:47:00.235758+0000","last_became_peered":"2024-07-31T14:47:00.235758+0000","last_unstale":"2024-07-31T14:48:03.605677+0000","last_undegraded":"2024-07-31T14:48:03.605677+0000","last_fullsized":"2024-07-31T14:48:03.605677+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:50.575648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.448218+0000","last_change":"2024-07-31T14:47:13.101876+0000","last_active":"2024-07-31T14:47:20.448218+0000","last_peered":"2024-07-31T14:47:20.448218+0000","last_clean":"2024-07-31T14:47:20.448218+0000","last_became_active":"2024-07-31T14:47:13.101276+0000","last_became_peered":"2024-07-31T14:47:13.101276+0000","last_unstale":"2024-07-31T14:47:20.448218+0000","last_undegraded":"2024-07-31T14:47:20.448218+0000","last_fullsized":"2024-07-31T14:47:20.448218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:53:24.684212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409477+0000","last_change":"2024-07-31T14:47:03.052478+0000","last_active":"2024-07-31T14:49:49.409477+0000","last_peered":"2024-07-31T14:49:49.409477+0000","last_clean":"2024-07-31T14:49:49.409477+0000","last_became_active":"2024-07-31T14:47:03.051877+0000","last_became_peered":"2024-07-31T14:47:03.051877+0000","last_unstale":"2024-07-31T14:49:49.409477+0000","last_undegraded":"2024-07-31T14:49:49.409477+0000","last_fullsized":"2024-07-31T14:49:49.409477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:30:32.666490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:07.082692+0000","last_change":"2024-07-31T14:47:00.020076+0000","last_active":"2024-07-31T14:51:07.082692+0000","last_peered":"2024-07-31T14:51:07.082692+0000","last_clean":"2024-07-31T14:51:07.082692+0000","last_became_active":"2024-07-31T14:47:00.019708+0000","last_became_peered":"2024-07-31T14:47:00.019708+0000","last_unstale":"2024-07-31T14:51:07.082692+0000","last_undegraded":"2024-07-31T14:51:07.082692+0000","last_fullsized":"2024-07-31T14:51:07.082692+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:46.520997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625840+0000","last_change":"2024-07-31T14:47:13.362961+0000","last_active":"2024-07-31T14:47:20.625840+0000","last_peered":"2024-07-31T14:47:20.625840+0000","last_clean":"2024-07-31T14:47:20.625840+0000","last_became_active":"2024-07-31T14:47:13.362502+0000","last_became_peered":"2024-07-31T14:47:13.362502+0000","last_unstale":"2024-07-31T14:47:20.625840+0000","last_undegraded":"2024-07-31T14:47:20.625840+0000","last_fullsized":"2024-07-31T14:47:20.625840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:51:45.732477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399783+0000","last_change":"2024-07-31T14:47:03.039418+0000","last_active":"2024-07-31T14:49:49.399783+0000","last_peered":"2024-07-31T14:49:49.399783+0000","last_clean":"2024-07-31T14:49:49.399783+0000","last_became_active":"2024-07-31T14:47:03.039044+0000","last_became_peered":"2024-07-31T14:47:03.039044+0000","last_unstale":"2024-07-31T14:49:49.399783+0000","last_undegraded":"2024-07-31T14:49:49.399783+0000","last_fullsized":"2024-07-31T14:49:49.399783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:40:10.380080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:07.059440+0000","last_change":"2024-07-31T14:47:00.183778+0000","last_active":"2024-07-31T14:49:07.059440+0000","last_peered":"2024-07-31T14:49:07.059440+0000","last_clean":"2024-07-31T14:49:07.059440+0000","last_became_active":"2024-07-31T14:47:00.183370+0000","last_became_peered":"2024-07-31T14:47:00.183370+0000","last_unstale":"2024-07-31T14:49:07.059440+0000","last_undegraded":"2024-07-31T14:49:07.059440+0000","last_fullsized":"2024-07-31T14:49:07.059440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:45:27.422216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445139+0000","last_change":"2024-07-31T14:47:13.102583+0000","last_active":"2024-07-31T14:47:20.445139+0000","last_peered":"2024-07-31T14:47:20.445139+0000","last_clean":"2024-07-31T14:47:20.445139+0000","last_became_active":"2024-07-31T14:47:13.102058+0000","last_became_peered":"2024-07-31T14:47:13.102058+0000","last_unstale":"2024-07-31T14:47:20.445139+0000","last_undegraded":"2024-07-31T14:47:20.445139+0000","last_fullsized":"2024-07-31T14:47:20.445139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:17.396173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407879+0000","last_change":"2024-07-31T14:47:03.048334+0000","last_active":"2024-07-31T14:49:49.407879+0000","last_peered":"2024-07-31T14:49:49.407879+0000","last_clean":"2024-07-31T14:49:49.407879+0000","last_became_active":"2024-07-31T14:47:03.047932+0000","last_became_peered":"2024-07-31T14:47:03.047932+0000","last_unstale":"2024-07-31T14:49:49.407879+0000","last_undegraded":"2024-07-31T14:49:49.407879+0000","last_fullsized":"2024-07-31T14:49:49.407879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:14:31.381061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:06.494830+0000","last_change":"2024-07-31T14:47:20.450851+0000","last_active":"2024-07-31T14:49:06.494830+0000","last_peered":"2024-07-31T14:49:06.494830+0000","last_clean":"2024-07-31T14:49:06.494830+0000","last_became_active":"2024-07-31T14:47:20.450435+0000","last_became_peered":"2024-07-31T14:47:20.450435+0000","last_unstale":"2024-07-31T14:49:06.494830+0000","last_undegraded":"2024-07-31T14:49:06.494830+0000","last_fullsized":"2024-07-31T14:49:06.494830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:17:28.738911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.450860+0000","last_change":"2024-07-31T14:47:13.363054+0000","last_active":"2024-07-31T14:47:20.450860+0000","last_peered":"2024-07-31T14:47:20.450860+0000","last_clean":"2024-07-31T14:47:20.450860+0000","last_became_active":"2024-07-31T14:47:13.362364+0000","last_became_peered":"2024-07-31T14:47:13.362364+0000","last_unstale":"2024-07-31T14:47:20.450860+0000","last_undegraded":"2024-07-31T14:47:20.450860+0000","last_fullsized":"2024-07-31T14:47:20.450860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:02:36.569678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.393142+0000","last_change":"2024-07-31T14:47:03.059834+0000","last_active":"2024-07-31T14:49:49.393142+0000","last_peered":"2024-07-31T14:49:49.393142+0000","last_clean":"2024-07-31T14:49:49.393142+0000","last_became_active":"2024-07-31T14:47:03.059127+0000","last_became_peered":"2024-07-31T14:47:03.059127+0000","last_unstale":"2024-07-31T14:49:49.393142+0000","last_undegraded":"2024-07-31T14:49:49.393142+0000","last_fullsized":"2024-07-31T14:49:49.393142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:53:50.308789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":241,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:10.893959+0000","last_change":"2024-07-31T14:47:00.030576+0000","last_active":"2024-07-31T14:51:10.893959+0000","last_peered":"2024-07-31T14:51:10.893959+0000","last_clean":"2024-07-31T14:51:10.893959+0000","last_became_active":"2024-07-31T14:47:00.030174+0000","last_became_peered":"2024-07-31T14:47:00.030174+0000","last_unstale":"2024-07-31T14:51:10.893959+0000","last_undegraded":"2024-07-31T14:51:10.893959+0000","last_fullsized":"2024-07-31T14:51:10.893959+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:13:35.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":191,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447258+0000","last_change":"2024-07-31T14:47:13.100180+0000","last_active":"2024-07-31T14:47:20.447258+0000","last_peered":"2024-07-31T14:47:20.447258+0000","last_clean":"2024-07-31T14:47:20.447258+0000","last_became_active":"2024-07-31T14:47:13.099725+0000","last_became_peered":"2024-07-31T14:47:13.099725+0000","last_unstale":"2024-07-31T14:47:20.447258+0000","last_undegraded":"2024-07-31T14:47:20.447258+0000","last_fullsized":"2024-07-31T14:47:20.447258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:25:32.245339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404277+0000","last_change":"2024-07-31T14:47:03.040349+0000","last_active":"2024-07-31T14:49:49.404277+0000","last_peered":"2024-07-31T14:49:49.404277+0000","last_clean":"2024-07-31T14:49:49.404277+0000","last_became_active":"2024-07-31T14:47:03.039887+0000","last_became_peered":"2024-07-31T14:47:03.039887+0000","last_unstale":"2024-07-31T14:49:49.404277+0000","last_undegraded":"2024-07-31T14:49:49.404277+0000","last_fullsized":"2024-07-31T14:49:49.404277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:46.592063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":241,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:12.881992+0000","last_change":"2024-07-31T14:47:00.183988+0000","last_active":"2024-07-31T14:51:12.881992+0000","last_peered":"2024-07-31T14:51:12.881992+0000","last_clean":"2024-07-31T14:51:12.881992+0000","last_became_active":"2024-07-31T14:47:00.183302+0000","last_became_peered":"2024-07-31T14:47:00.183302+0000","last_unstale":"2024-07-31T14:51:12.881992+0000","last_undegraded":"2024-07-31T14:51:12.881992+0000","last_fullsized":"2024-07-31T14:51:12.881992+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:40:26.188224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447744+0000","last_change":"2024-07-31T14:47:13.363075+0000","last_active":"2024-07-31T14:47:20.447744+0000","last_peered":"2024-07-31T14:47:20.447744+0000","last_clean":"2024-07-31T14:47:20.447744+0000","last_became_active":"2024-07-31T14:47:13.362599+0000","last_became_peered":"2024-07-31T14:47:13.362599+0000","last_unstale":"2024-07-31T14:47:20.447744+0000","last_undegraded":"2024-07-31T14:47:20.447744+0000","last_fullsized":"2024-07-31T14:47:20.447744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:20:43.617742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.402186+0000","last_change":"2024-07-31T14:47:20.450712+0000","last_active":"2024-07-31T14:49:49.402186+0000","last_peered":"2024-07-31T14:49:49.402186+0000","last_clean":"2024-07-31T14:49:49.402186+0000","last_became_active":"2024-07-31T14:47:20.450229+0000","last_became_peered":"2024-07-31T14:47:20.450229+0000","last_unstale":"2024-07-31T14:49:49.402186+0000","last_undegraded":"2024-07-31T14:49:49.402186+0000","last_fullsized":"2024-07-31T14:49:49.402186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:11:50.631420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:01.604764+0000","last_change":"2024-07-31T14:47:00.021389+0000","last_active":"2024-07-31T14:49:01.604764+0000","last_peered":"2024-07-31T14:49:01.604764+0000","last_clean":"2024-07-31T14:49:01.604764+0000","last_became_active":"2024-07-31T14:47:00.020987+0000","last_became_peered":"2024-07-31T14:47:00.020987+0000","last_unstale":"2024-07-31T14:49:01.604764+0000","last_undegraded":"2024-07-31T14:49:01.604764+0000","last_fullsized":"2024-07-31T14:49:01.604764+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:07:24.658463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627003+0000","last_change":"2024-07-31T14:47:13.363105+0000","last_active":"2024-07-31T14:47:20.627003+0000","last_peered":"2024-07-31T14:47:20.627003+0000","last_clean":"2024-07-31T14:47:20.627003+0000","last_became_active":"2024-07-31T14:47:13.362404+0000","last_became_peered":"2024-07-31T14:47:13.362404+0000","last_unstale":"2024-07-31T14:47:20.627003+0000","last_undegraded":"2024-07-31T14:47:20.627003+0000","last_fullsized":"2024-07-31T14:47:20.627003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:48:43.033584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404516+0000","last_change":"2024-07-31T14:47:03.057199+0000","last_active":"2024-07-31T14:49:49.404516+0000","last_peered":"2024-07-31T14:49:49.404516+0000","last_clean":"2024-07-31T14:49:49.404516+0000","last_became_active":"2024-07-31T14:47:03.056555+0000","last_became_peered":"2024-07-31T14:47:03.056555+0000","last_unstale":"2024-07-31T14:49:49.404516+0000","last_undegraded":"2024-07-31T14:49:49.404516+0000","last_fullsized":"2024-07-31T14:49:49.404516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:21:14.263457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:04.380224+0000","last_change":"2024-07-31T14:47:00.032055+0000","last_active":"2024-07-31T14:49:04.380224+0000","last_peered":"2024-07-31T14:49:04.380224+0000","last_clean":"2024-07-31T14:49:04.380224+0000","last_became_active":"2024-07-31T14:47:00.031519+0000","last_became_peered":"2024-07-31T14:47:00.031519+0000","last_unstale":"2024-07-31T14:49:04.380224+0000","last_undegraded":"2024-07-31T14:49:04.380224+0000","last_fullsized":"2024-07-31T14:49:04.380224+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:21:46.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:31.600736+0000","last_change":"2024-07-31T14:45:42.166503+0000","last_active":"2024-07-31T14:48:31.600736+0000","last_peered":"2024-07-31T14:48:31.600736+0000","last_clean":"2024-07-31T14:48:31.600736+0000","last_became_active":"2024-07-31T14:45:42.160065+0000","last_became_peered":"2024-07-31T14:45:42.160065+0000","last_unstale":"2024-07-31T14:48:31.600736+0000","last_undegraded":"2024-07-31T14:48:31.600736+0000","last_fullsized":"2024-07-31T14:48:31.600736+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_clean_scrub_stamp":"2024-07-31T14:42:27.863113+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:38:33.646022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:12.369486+0000","last_change":"2024-07-31T14:47:13.103812+0000","last_active":"2024-07-31T14:48:12.369486+0000","last_peered":"2024-07-31T14:48:12.369486+0000","last_clean":"2024-07-31T14:48:12.369486+0000","last_became_active":"2024-07-31T14:47:13.102549+0000","last_became_peered":"2024-07-31T14:47:13.102549+0000","last_unstale":"2024-07-31T14:48:12.369486+0000","last_undegraded":"2024-07-31T14:48:12.369486+0000","last_fullsized":"2024-07-31T14:48:12.369486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:18:56.322426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414083+0000","last_change":"2024-07-31T14:47:03.056310+0000","last_active":"2024-07-31T14:49:49.414083+0000","last_peered":"2024-07-31T14:49:49.414083+0000","last_clean":"2024-07-31T14:49:49.414083+0000","last_became_active":"2024-07-31T14:47:03.055695+0000","last_became_peered":"2024-07-31T14:47:03.055695+0000","last_unstale":"2024-07-31T14:49:49.414083+0000","last_undegraded":"2024-07-31T14:49:49.414083+0000","last_fullsized":"2024-07-31T14:49:49.414083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:00:55.124981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:09.891605+0000","last_change":"2024-07-31T14:47:00.236460+0000","last_active":"2024-07-31T14:51:09.891605+0000","last_peered":"2024-07-31T14:51:09.891605+0000","last_clean":"2024-07-31T14:51:09.891605+0000","last_became_active":"2024-07-31T14:47:00.235924+0000","last_became_peered":"2024-07-31T14:47:00.235924+0000","last_unstale":"2024-07-31T14:51:09.891605+0000","last_undegraded":"2024-07-31T14:51:09.891605+0000","last_fullsized":"2024-07-31T14:51:09.891605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:05.130958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626080+0000","last_change":"2024-07-31T14:47:13.102341+0000","last_active":"2024-07-31T14:47:20.626080+0000","last_peered":"2024-07-31T14:47:20.626080+0000","last_clean":"2024-07-31T14:47:20.626080+0000","last_became_active":"2024-07-31T14:47:13.101798+0000","last_became_peered":"2024-07-31T14:47:13.101798+0000","last_unstale":"2024-07-31T14:47:20.626080+0000","last_undegraded":"2024-07-31T14:47:20.626080+0000","last_fullsized":"2024-07-31T14:47:20.626080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:34:35.758041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409700+0000","last_change":"2024-07-31T14:47:03.054322+0000","last_active":"2024-07-31T14:49:49.409700+0000","last_peered":"2024-07-31T14:49:49.409700+0000","last_clean":"2024-07-31T14:49:49.409700+0000","last_became_active":"2024-07-31T14:47:03.053112+0000","last_became_peered":"2024-07-31T14:47:03.053112+0000","last_unstale":"2024-07-31T14:49:49.409700+0000","last_undegraded":"2024-07-31T14:49:49.409700+0000","last_fullsized":"2024-07-31T14:49:49.409700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:55:13.542578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604832+0000","last_change":"2024-07-31T14:47:00.026518+0000","last_active":"2024-07-31T14:48:03.604832+0000","last_peered":"2024-07-31T14:48:03.604832+0000","last_clean":"2024-07-31T14:48:03.604832+0000","last_became_active":"2024-07-31T14:47:00.026074+0000","last_became_peered":"2024-07-31T14:47:00.026074+0000","last_unstale":"2024-07-31T14:48:03.604832+0000","last_undegraded":"2024-07-31T14:48:03.604832+0000","last_fullsized":"2024-07-31T14:48:03.604832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:33:36.688353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449369+0000","last_change":"2024-07-31T14:47:13.106365+0000","last_active":"2024-07-31T14:47:20.449369+0000","last_peered":"2024-07-31T14:47:20.449369+0000","last_clean":"2024-07-31T14:47:20.449369+0000","last_became_active":"2024-07-31T14:47:13.105635+0000","last_became_peered":"2024-07-31T14:47:13.105635+0000","last_unstale":"2024-07-31T14:47:20.449369+0000","last_undegraded":"2024-07-31T14:47:20.449369+0000","last_fullsized":"2024-07-31T14:47:20.449369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:52:09.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404285+0000","last_change":"2024-07-31T14:47:03.052522+0000","last_active":"2024-07-31T14:49:49.404285+0000","last_peered":"2024-07-31T14:49:49.404285+0000","last_clean":"2024-07-31T14:49:49.404285+0000","last_became_active":"2024-07-31T14:47:03.051872+0000","last_became_peered":"2024-07-31T14:47:03.051872+0000","last_unstale":"2024-07-31T14:49:49.404285+0000","last_undegraded":"2024-07-31T14:49:49.404285+0000","last_fullsized":"2024-07-31T14:49:49.404285+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:29:34.956745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600950+0000","last_change":"2024-07-31T14:47:00.016839+0000","last_active":"2024-07-31T14:48:02.600950+0000","last_peered":"2024-07-31T14:48:02.600950+0000","last_clean":"2024-07-31T14:48:02.600950+0000","last_became_active":"2024-07-31T14:47:00.016438+0000","last_became_peered":"2024-07-31T14:47:00.016438+0000","last_unstale":"2024-07-31T14:48:02.600950+0000","last_undegraded":"2024-07-31T14:48:02.600950+0000","last_fullsized":"2024-07-31T14:48:02.600950+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:42:10.875332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626258+0000","last_change":"2024-07-31T14:47:13.105385+0000","last_active":"2024-07-31T14:47:20.626258+0000","last_peered":"2024-07-31T14:47:20.626258+0000","last_clean":"2024-07-31T14:47:20.626258+0000","last_became_active":"2024-07-31T14:47:13.104867+0000","last_became_peered":"2024-07-31T14:47:13.104867+0000","last_unstale":"2024-07-31T14:47:20.626258+0000","last_undegraded":"2024-07-31T14:47:20.626258+0000","last_fullsized":"2024-07-31T14:47:20.626258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:25:21.572830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399072+0000","last_change":"2024-07-31T14:47:03.058304+0000","last_active":"2024-07-31T14:49:49.399072+0000","last_peered":"2024-07-31T14:49:49.399072+0000","last_clean":"2024-07-31T14:49:49.399072+0000","last_became_active":"2024-07-31T14:47:03.057251+0000","last_became_peered":"2024-07-31T14:47:03.057251+0000","last_unstale":"2024-07-31T14:49:49.399072+0000","last_undegraded":"2024-07-31T14:49:49.399072+0000","last_fullsized":"2024-07-31T14:49:49.399072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:58:20.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605301+0000","last_change":"2024-07-31T14:47:00.027806+0000","last_active":"2024-07-31T14:48:03.605301+0000","last_peered":"2024-07-31T14:48:03.605301+0000","last_clean":"2024-07-31T14:48:03.605301+0000","last_became_active":"2024-07-31T14:47:00.027403+0000","last_became_peered":"2024-07-31T14:47:00.027403+0000","last_unstale":"2024-07-31T14:48:03.605301+0000","last_undegraded":"2024-07-31T14:48:03.605301+0000","last_fullsized":"2024-07-31T14:48:03.605301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:57:18.303732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626490+0000","last_change":"2024-07-31T14:47:13.103628+0000","last_active":"2024-07-31T14:47:20.626490+0000","last_peered":"2024-07-31T14:47:20.626490+0000","last_clean":"2024-07-31T14:47:20.626490+0000","last_became_active":"2024-07-31T14:47:13.102296+0000","last_became_peered":"2024-07-31T14:47:13.102296+0000","last_unstale":"2024-07-31T14:47:20.626490+0000","last_undegraded":"2024-07-31T14:47:20.626490+0000","last_fullsized":"2024-07-31T14:47:20.626490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:25:35.109196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413387+0000","last_change":"2024-07-31T14:47:03.052761+0000","last_active":"2024-07-31T14:49:49.413387+0000","last_peered":"2024-07-31T14:49:49.413387+0000","last_clean":"2024-07-31T14:49:49.413387+0000","last_became_active":"2024-07-31T14:47:03.052410+0000","last_became_peered":"2024-07-31T14:47:03.052410+0000","last_unstale":"2024-07-31T14:49:49.413387+0000","last_undegraded":"2024-07-31T14:49:49.413387+0000","last_fullsized":"2024-07-31T14:49:49.413387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:20:47.712491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604937+0000","last_change":"2024-07-31T14:47:00.183741+0000","last_active":"2024-07-31T14:48:03.604937+0000","last_peered":"2024-07-31T14:48:03.604937+0000","last_clean":"2024-07-31T14:48:03.604937+0000","last_became_active":"2024-07-31T14:47:00.183275+0000","last_became_peered":"2024-07-31T14:47:00.183275+0000","last_unstale":"2024-07-31T14:48:03.604937+0000","last_undegraded":"2024-07-31T14:48:03.604937+0000","last_fullsized":"2024-07-31T14:48:03.604937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:29:23.205835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630098+0000","last_change":"2024-07-31T14:47:20.629334+0000","last_active":"2024-07-31T14:47:20.630098+0000","last_peered":"2024-07-31T14:47:20.630098+0000","last_clean":"2024-07-31T14:47:20.630098+0000","last_became_active":"2024-07-31T14:47:20.628865+0000","last_became_peered":"2024-07-31T14:47:20.628865+0000","last_unstale":"2024-07-31T14:47:20.630098+0000","last_undegraded":"2024-07-31T14:47:20.630098+0000","last_fullsized":"2024-07-31T14:47:20.630098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:21:27.199842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407413+0000","last_change":"2024-07-31T14:47:03.054633+0000","last_active":"2024-07-31T14:49:49.407413+0000","last_peered":"2024-07-31T14:49:49.407413+0000","last_clean":"2024-07-31T14:49:49.407413+0000","last_became_active":"2024-07-31T14:47:03.053025+0000","last_became_peered":"2024-07-31T14:47:03.053025+0000","last_unstale":"2024-07-31T14:49:49.407413+0000","last_undegraded":"2024-07-31T14:49:49.407413+0000","last_fullsized":"2024-07-31T14:49:49.407413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:42:03.392744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:08.876300+0000","last_change":"2024-07-31T14:47:00.027898+0000","last_active":"2024-07-31T14:51:08.876300+0000","last_peered":"2024-07-31T14:51:08.876300+0000","last_clean":"2024-07-31T14:51:08.876300+0000","last_became_active":"2024-07-31T14:47:00.027468+0000","last_became_peered":"2024-07-31T14:47:00.027468+0000","last_unstale":"2024-07-31T14:51:08.876300+0000","last_undegraded":"2024-07-31T14:51:08.876300+0000","last_fullsized":"2024-07-31T14:51:08.876300+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:12:45.922172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444574+0000","last_change":"2024-07-31T14:47:13.104327+0000","last_active":"2024-07-31T14:47:20.444574+0000","last_peered":"2024-07-31T14:47:20.444574+0000","last_clean":"2024-07-31T14:47:20.444574+0000","last_became_active":"2024-07-31T14:47:13.103333+0000","last_became_peered":"2024-07-31T14:47:13.103333+0000","last_unstale":"2024-07-31T14:47:20.444574+0000","last_undegraded":"2024-07-31T14:47:20.444574+0000","last_fullsized":"2024-07-31T14:47:20.444574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:06.226220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.410145+0000","last_change":"2024-07-31T14:47:03.044875+0000","last_active":"2024-07-31T14:49:49.410145+0000","last_peered":"2024-07-31T14:49:49.410145+0000","last_clean":"2024-07-31T14:49:49.410145+0000","last_became_active":"2024-07-31T14:47:03.044421+0000","last_became_peered":"2024-07-31T14:47:03.044421+0000","last_unstale":"2024-07-31T14:49:49.410145+0000","last_undegraded":"2024-07-31T14:49:49.410145+0000","last_fullsized":"2024-07-31T14:49:49.410145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:00:02.477300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605606+0000","last_change":"2024-07-31T14:47:00.019790+0000","last_active":"2024-07-31T14:48:03.605606+0000","last_peered":"2024-07-31T14:48:03.605606+0000","last_clean":"2024-07-31T14:48:03.605606+0000","last_became_active":"2024-07-31T14:47:00.019368+0000","last_became_peered":"2024-07-31T14:47:00.019368+0000","last_unstale":"2024-07-31T14:48:03.605606+0000","last_undegraded":"2024-07-31T14:48:03.605606+0000","last_fullsized":"2024-07-31T14:48:03.605606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:15.299109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445672+0000","last_change":"2024-07-31T14:47:13.102618+0000","last_active":"2024-07-31T14:47:20.445672+0000","last_peered":"2024-07-31T14:47:20.445672+0000","last_clean":"2024-07-31T14:47:20.445672+0000","last_became_active":"2024-07-31T14:47:13.102085+0000","last_became_peered":"2024-07-31T14:47:13.102085+0000","last_unstale":"2024-07-31T14:47:20.445672+0000","last_undegraded":"2024-07-31T14:47:20.445672+0000","last_fullsized":"2024-07-31T14:47:20.445672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:05:58.814215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.406277+0000","last_change":"2024-07-31T14:47:03.044688+0000","last_active":"2024-07-31T14:49:49.406277+0000","last_peered":"2024-07-31T14:49:49.406277+0000","last_clean":"2024-07-31T14:49:49.406277+0000","last_became_active":"2024-07-31T14:47:03.043751+0000","last_became_peered":"2024-07-31T14:47:03.043751+0000","last_unstale":"2024-07-31T14:49:49.406277+0000","last_undegraded":"2024-07-31T14:49:49.406277+0000","last_fullsized":"2024-07-31T14:49:49.406277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:47:31.450796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605830+0000","last_change":"2024-07-31T14:47:00.026259+0000","last_active":"2024-07-31T14:48:03.605830+0000","last_peered":"2024-07-31T14:48:03.605830+0000","last_clean":"2024-07-31T14:48:03.605830+0000","last_became_active":"2024-07-31T14:47:00.025756+0000","last_became_peered":"2024-07-31T14:47:00.025756+0000","last_unstale":"2024-07-31T14:48:03.605830+0000","last_undegraded":"2024-07-31T14:48:03.605830+0000","last_fullsized":"2024-07-31T14:48:03.605830+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:05:00.258841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626888+0000","last_change":"2024-07-31T14:47:13.104918+0000","last_active":"2024-07-31T14:47:20.626888+0000","last_peered":"2024-07-31T14:47:20.626888+0000","last_clean":"2024-07-31T14:47:20.626888+0000","last_became_active":"2024-07-31T14:47:13.104031+0000","last_became_peered":"2024-07-31T14:47:13.104031+0000","last_unstale":"2024-07-31T14:47:20.626888+0000","last_undegraded":"2024-07-31T14:47:20.626888+0000","last_fullsized":"2024-07-31T14:47:20.626888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:17:15.264474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.389841+0000","last_change":"2024-07-31T14:47:03.055159+0000","last_active":"2024-07-31T14:49:49.389841+0000","last_peered":"2024-07-31T14:49:49.389841+0000","last_clean":"2024-07-31T14:49:49.389841+0000","last_became_active":"2024-07-31T14:47:03.052389+0000","last_became_peered":"2024-07-31T14:47:03.052389+0000","last_unstale":"2024-07-31T14:49:49.389841+0000","last_undegraded":"2024-07-31T14:49:49.389841+0000","last_fullsized":"2024-07-31T14:49:49.389841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:16.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'41","reported_seq":469,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:09.878153+0000","last_change":"2024-07-31T14:47:00.030165+0000","last_active":"2024-07-31T14:51:09.878153+0000","last_peered":"2024-07-31T14:51:09.878153+0000","last_clean":"2024-07-31T14:51:09.878153+0000","last_became_active":"2024-07-31T14:47:00.029559+0000","last_became_peered":"2024-07-31T14:47:00.029559+0000","last_unstale":"2024-07-31T14:51:09.878153+0000","last_undegraded":"2024-07-31T14:51:09.878153+0000","last_fullsized":"2024-07-31T14:51:09.878153+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:31:06.115075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69398,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":386,"num_read_kb":261,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444234+0000","last_change":"2024-07-31T14:47:13.098517+0000","last_active":"2024-07-31T14:47:20.444234+0000","last_peered":"2024-07-31T14:47:20.444234+0000","last_clean":"2024-07-31T14:47:20.444234+0000","last_became_active":"2024-07-31T14:47:13.098066+0000","last_became_peered":"2024-07-31T14:47:13.098066+0000","last_unstale":"2024-07-31T14:47:20.444234+0000","last_undegraded":"2024-07-31T14:47:20.444234+0000","last_fullsized":"2024-07-31T14:47:20.444234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:08:54.958646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414451+0000","last_change":"2024-07-31T14:47:03.054045+0000","last_active":"2024-07-31T14:49:49.414451+0000","last_peered":"2024-07-31T14:49:49.414451+0000","last_clean":"2024-07-31T14:49:49.414451+0000","last_became_active":"2024-07-31T14:47:03.053658+0000","last_became_peered":"2024-07-31T14:47:03.053658+0000","last_unstale":"2024-07-31T14:49:49.414451+0000","last_undegraded":"2024-07-31T14:49:49.414451+0000","last_fullsized":"2024-07-31T14:49:49.414451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:36:54.563542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.607373+0000","last_change":"2024-07-31T14:47:00.185602+0000","last_active":"2024-07-31T14:48:03.607373+0000","last_peered":"2024-07-31T14:48:03.607373+0000","last_clean":"2024-07-31T14:48:03.607373+0000","last_became_active":"2024-07-31T14:47:00.185230+0000","last_became_peered":"2024-07-31T14:47:00.185230+0000","last_unstale":"2024-07-31T14:48:03.607373+0000","last_undegraded":"2024-07-31T14:48:03.607373+0000","last_fullsized":"2024-07-31T14:48:03.607373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:21:35.979942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604698+0000","last_change":"2024-07-31T14:47:13.099611+0000","last_active":"2024-07-31T14:47:20.604698+0000","last_peered":"2024-07-31T14:47:20.604698+0000","last_clean":"2024-07-31T14:47:20.604698+0000","last_became_active":"2024-07-31T14:47:13.099127+0000","last_became_peered":"2024-07-31T14:47:13.099127+0000","last_unstale":"2024-07-31T14:47:20.604698+0000","last_undegraded":"2024-07-31T14:47:20.604698+0000","last_fullsized":"2024-07-31T14:47:20.604698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:16:57.192980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400126+0000","last_change":"2024-07-31T14:47:03.056818+0000","last_active":"2024-07-31T14:49:49.400126+0000","last_peered":"2024-07-31T14:49:49.400126+0000","last_clean":"2024-07-31T14:49:49.400126+0000","last_became_active":"2024-07-31T14:47:03.055716+0000","last_became_peered":"2024-07-31T14:47:03.055716+0000","last_unstale":"2024-07-31T14:49:49.400126+0000","last_undegraded":"2024-07-31T14:49:49.400126+0000","last_fullsized":"2024-07-31T14:49:49.400126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:26:33.627362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600344+0000","last_change":"2024-07-31T14:47:00.029651+0000","last_active":"2024-07-31T14:48:02.600344+0000","last_peered":"2024-07-31T14:48:02.600344+0000","last_clean":"2024-07-31T14:48:02.600344+0000","last_became_active":"2024-07-31T14:47:00.029242+0000","last_became_peered":"2024-07-31T14:47:00.029242+0000","last_unstale":"2024-07-31T14:48:02.600344+0000","last_undegraded":"2024-07-31T14:48:02.600344+0000","last_fullsized":"2024-07-31T14:48:02.600344+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:23:49.177508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.443852+0000","last_change":"2024-07-31T14:47:13.098862+0000","last_active":"2024-07-31T14:47:20.443852+0000","last_peered":"2024-07-31T14:47:20.443852+0000","last_clean":"2024-07-31T14:47:20.443852+0000","last_became_active":"2024-07-31T14:47:13.098324+0000","last_became_peered":"2024-07-31T14:47:13.098324+0000","last_unstale":"2024-07-31T14:47:20.443852+0000","last_undegraded":"2024-07-31T14:47:20.443852+0000","last_fullsized":"2024-07-31T14:47:20.443852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:50:41.260863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.412242+0000","last_change":"2024-07-31T14:47:03.049042+0000","last_active":"2024-07-31T14:49:49.412242+0000","last_peered":"2024-07-31T14:49:49.412242+0000","last_clean":"2024-07-31T14:49:49.412242+0000","last_became_active":"2024-07-31T14:47:03.048535+0000","last_became_peered":"2024-07-31T14:47:03.048535+0000","last_unstale":"2024-07-31T14:49:49.412242+0000","last_undegraded":"2024-07-31T14:49:49.412242+0000","last_fullsized":"2024-07-31T14:49:49.412242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:15.147454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":422,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:09.877699+0000","last_change":"2024-07-31T14:47:00.028940+0000","last_active":"2024-07-31T14:51:09.877699+0000","last_peered":"2024-07-31T14:51:09.877699+0000","last_clean":"2024-07-31T14:51:09.877699+0000","last_became_active":"2024-07-31T14:47:00.028472+0000","last_became_peered":"2024-07-31T14:47:00.028472+0000","last_unstale":"2024-07-31T14:51:09.877699+0000","last_undegraded":"2024-07-31T14:51:09.877699+0000","last_fullsized":"2024-07-31T14:51:09.877699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:57:15.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":374,"num_read_kb":374,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626394+0000","last_change":"2024-07-31T14:47:13.101856+0000","last_active":"2024-07-31T14:47:20.626394+0000","last_peered":"2024-07-31T14:47:20.626394+0000","last_clean":"2024-07-31T14:47:20.626394+0000","last_became_active":"2024-07-31T14:47:13.101346+0000","last_became_peered":"2024-07-31T14:47:13.101346+0000","last_unstale":"2024-07-31T14:47:20.626394+0000","last_undegraded":"2024-07-31T14:47:20.626394+0000","last_fullsized":"2024-07-31T14:47:20.626394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:56:37.071660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449190+0000","last_change":"2024-07-31T14:47:13.105952+0000","last_active":"2024-07-31T14:47:20.449190+0000","last_peered":"2024-07-31T14:47:20.449190+0000","last_clean":"2024-07-31T14:47:20.449190+0000","last_became_active":"2024-07-31T14:47:13.105506+0000","last_became_peered":"2024-07-31T14:47:13.105506+0000","last_unstale":"2024-07-31T14:47:20.449190+0000","last_undegraded":"2024-07-31T14:47:20.449190+0000","last_fullsized":"2024-07-31T14:47:20.449190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:28:33.866216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400532+0000","last_change":"2024-07-31T14:47:03.051152+0000","last_active":"2024-07-31T14:49:49.400532+0000","last_peered":"2024-07-31T14:49:49.400532+0000","last_clean":"2024-07-31T14:49:49.400532+0000","last_became_active":"2024-07-31T14:47:03.050775+0000","last_became_peered":"2024-07-31T14:47:03.050775+0000","last_unstale":"2024-07-31T14:49:49.400532+0000","last_undegraded":"2024-07-31T14:49:49.400532+0000","last_fullsized":"2024-07-31T14:49:49.400532+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:47:35.913641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609201+0000","last_change":"2024-07-31T14:47:00.028552+0000","last_active":"2024-07-31T14:48:03.609201+0000","last_peered":"2024-07-31T14:48:03.609201+0000","last_clean":"2024-07-31T14:48:03.609201+0000","last_became_active":"2024-07-31T14:47:00.028143+0000","last_became_peered":"2024-07-31T14:47:00.028143+0000","last_unstale":"2024-07-31T14:48:03.609201+0000","last_undegraded":"2024-07-31T14:48:03.609201+0000","last_fullsized":"2024-07-31T14:48:03.609201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:09.601635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.606879+0000","last_change":"2024-07-31T14:47:20.606285+0000","last_active":"2024-07-31T14:47:20.606879+0000","last_peered":"2024-07-31T14:47:20.606879+0000","last_clean":"2024-07-31T14:47:20.606879+0000","last_became_active":"2024-07-31T14:47:20.605837+0000","last_became_peered":"2024-07-31T14:47:20.605837+0000","last_unstale":"2024-07-31T14:47:20.606879+0000","last_undegraded":"2024-07-31T14:47:20.606879+0000","last_fullsized":"2024-07-31T14:47:20.606879+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:16:35.409790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.398709+0000","last_change":"2024-07-31T14:47:03.047442+0000","last_active":"2024-07-31T14:49:49.398709+0000","last_peered":"2024-07-31T14:49:49.398709+0000","last_clean":"2024-07-31T14:49:49.398709+0000","last_became_active":"2024-07-31T14:47:03.047072+0000","last_became_peered":"2024-07-31T14:47:03.047072+0000","last_unstale":"2024-07-31T14:49:49.398709+0000","last_undegraded":"2024-07-31T14:49:49.398709+0000","last_fullsized":"2024-07-31T14:49:49.398709+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:56.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.599595+0000","last_change":"2024-07-31T14:47:00.027746+0000","last_active":"2024-07-31T14:48:02.599595+0000","last_peered":"2024-07-31T14:48:02.599595+0000","last_clean":"2024-07-31T14:48:02.599595+0000","last_became_active":"2024-07-31T14:47:00.027404+0000","last_became_peered":"2024-07-31T14:47:00.027404+0000","last_unstale":"2024-07-31T14:48:02.599595+0000","last_undegraded":"2024-07-31T14:48:02.599595+0000","last_fullsized":"2024-07-31T14:48:02.599595+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:30:21.011674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.589640+0000","last_change":"2024-07-31T14:47:00.029148+0000","last_active":"2024-07-31T14:48:01.589640+0000","last_peered":"2024-07-31T14:48:01.589640+0000","last_clean":"2024-07-31T14:48:01.589640+0000","last_became_active":"2024-07-31T14:47:00.028718+0000","last_became_peered":"2024-07-31T14:47:00.028718+0000","last_unstale":"2024-07-31T14:48:01.589640+0000","last_undegraded":"2024-07-31T14:48:01.589640+0000","last_fullsized":"2024-07-31T14:48:01.589640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:25:16.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400207+0000","last_change":"2024-07-31T14:47:20.629098+0000","last_active":"2024-07-31T14:49:49.400207+0000","last_peered":"2024-07-31T14:49:49.400207+0000","last_clean":"2024-07-31T14:49:49.400207+0000","last_became_active":"2024-07-31T14:47:20.627775+0000","last_became_peered":"2024-07-31T14:47:20.627775+0000","last_unstale":"2024-07-31T14:49:49.400207+0000","last_undegraded":"2024-07-31T14:49:49.400207+0000","last_fullsized":"2024-07-31T14:49:49.400207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:22:03.606509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447331+0000","last_change":"2024-07-31T14:47:13.097892+0000","last_active":"2024-07-31T14:47:20.447331+0000","last_peered":"2024-07-31T14:47:20.447331+0000","last_clean":"2024-07-31T14:47:20.447331+0000","last_became_active":"2024-07-31T14:47:13.097419+0000","last_became_peered":"2024-07-31T14:47:13.097419+0000","last_unstale":"2024-07-31T14:47:20.447331+0000","last_undegraded":"2024-07-31T14:47:20.447331+0000","last_fullsized":"2024-07-31T14:47:20.447331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:07:13.456147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":243,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:11.881508+0000","last_change":"2024-07-31T14:47:00.019792+0000","last_active":"2024-07-31T14:51:11.881508+0000","last_peered":"2024-07-31T14:51:11.881508+0000","last_clean":"2024-07-31T14:51:11.881508+0000","last_became_active":"2024-07-31T14:47:00.019411+0000","last_became_peered":"2024-07-31T14:47:00.019411+0000","last_unstale":"2024-07-31T14:51:11.881508+0000","last_undegraded":"2024-07-31T14:51:11.881508+0000","last_fullsized":"2024-07-31T14:51:11.881508+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:55:02.148823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":194,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.390707+0000","last_change":"2024-07-31T14:47:03.039808+0000","last_active":"2024-07-31T14:49:49.390707+0000","last_peered":"2024-07-31T14:49:49.390707+0000","last_clean":"2024-07-31T14:49:49.390707+0000","last_became_active":"2024-07-31T14:47:03.039382+0000","last_became_peered":"2024-07-31T14:47:03.039382+0000","last_unstale":"2024-07-31T14:49:49.390707+0000","last_undegraded":"2024-07-31T14:49:49.390707+0000","last_fullsized":"2024-07-31T14:49:49.390707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:23:09.751293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.446939+0000","last_change":"2024-07-31T14:47:13.102797+0000","last_active":"2024-07-31T14:47:20.446939+0000","last_peered":"2024-07-31T14:47:20.446939+0000","last_clean":"2024-07-31T14:47:20.446939+0000","last_became_active":"2024-07-31T14:47:13.102320+0000","last_became_peered":"2024-07-31T14:47:13.102320+0000","last_unstale":"2024-07-31T14:47:20.446939+0000","last_undegraded":"2024-07-31T14:47:20.446939+0000","last_fullsized":"2024-07-31T14:47:20.446939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:23.066357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.397081+0000","last_change":"2024-07-31T14:47:03.054875+0000","last_active":"2024-07-31T14:49:49.397081+0000","last_peered":"2024-07-31T14:49:49.397081+0000","last_clean":"2024-07-31T14:49:49.397081+0000","last_became_active":"2024-07-31T14:47:03.054529+0000","last_became_peered":"2024-07-31T14:47:03.054529+0000","last_unstale":"2024-07-31T14:49:49.397081+0000","last_undegraded":"2024-07-31T14:49:49.397081+0000","last_fullsized":"2024-07-31T14:49:49.397081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:32:09.802933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396215+0000","last_change":"2024-07-31T14:47:03.043283+0000","last_active":"2024-07-31T14:49:49.396215+0000","last_peered":"2024-07-31T14:49:49.396215+0000","last_clean":"2024-07-31T14:49:49.396215+0000","last_became_active":"2024-07-31T14:47:03.042851+0000","last_became_peered":"2024-07-31T14:47:03.042851+0000","last_unstale":"2024-07-31T14:49:49.396215+0000","last_undegraded":"2024-07-31T14:49:49.396215+0000","last_fullsized":"2024-07-31T14:49:49.396215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:25:37.408149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409421+0000","last_change":"2024-07-31T14:47:03.047890+0000","last_active":"2024-07-31T14:49:49.409421+0000","last_peered":"2024-07-31T14:49:49.409421+0000","last_clean":"2024-07-31T14:49:49.409421+0000","last_became_active":"2024-07-31T14:47:03.047507+0000","last_became_peered":"2024-07-31T14:47:03.047507+0000","last_unstale":"2024-07-31T14:49:49.409421+0000","last_undegraded":"2024-07-31T14:49:49.409421+0000","last_fullsized":"2024-07-31T14:49:49.409421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:52:45.991323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403016+0000","last_change":"2024-07-31T14:47:03.047903+0000","last_active":"2024-07-31T14:49:49.403016+0000","last_peered":"2024-07-31T14:49:49.403016+0000","last_clean":"2024-07-31T14:49:49.403016+0000","last_became_active":"2024-07-31T14:47:03.047352+0000","last_became_peered":"2024-07-31T14:47:03.047352+0000","last_unstale":"2024-07-31T14:49:49.403016+0000","last_undegraded":"2024-07-31T14:49:49.403016+0000","last_fullsized":"2024-07-31T14:49:49.403016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:06.536023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403170+0000","last_change":"2024-07-31T14:47:03.044199+0000","last_active":"2024-07-31T14:49:49.403170+0000","last_peered":"2024-07-31T14:49:49.403170+0000","last_clean":"2024-07-31T14:49:49.403170+0000","last_became_active":"2024-07-31T14:47:03.043849+0000","last_became_peered":"2024-07-31T14:47:03.043849+0000","last_unstale":"2024-07-31T14:49:49.403170+0000","last_undegraded":"2024-07-31T14:49:49.403170+0000","last_fullsized":"2024-07-31T14:49:49.403170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:50.474895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.391079+0000","last_change":"2024-07-31T14:47:03.060654+0000","last_active":"2024-07-31T14:49:49.391079+0000","last_peered":"2024-07-31T14:49:49.391079+0000","last_clean":"2024-07-31T14:49:49.391079+0000","last_became_active":"2024-07-31T14:47:03.060157+0000","last_became_peered":"2024-07-31T14:47:03.060157+0000","last_unstale":"2024-07-31T14:49:49.391079+0000","last_undegraded":"2024-07-31T14:49:49.391079+0000","last_fullsized":"2024-07-31T14:49:49.391079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:39:50.130831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414163+0000","last_change":"2024-07-31T14:47:03.049910+0000","last_active":"2024-07-31T14:49:49.414163+0000","last_peered":"2024-07-31T14:49:49.414163+0000","last_clean":"2024-07-31T14:49:49.414163+0000","last_became_active":"2024-07-31T14:47:03.049491+0000","last_became_peered":"2024-07-31T14:47:03.049491+0000","last_unstale":"2024-07-31T14:49:49.414163+0000","last_undegraded":"2024-07-31T14:49:49.414163+0000","last_fullsized":"2024-07-31T14:49:49.414163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:18:43.319695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407340+0000","last_change":"2024-07-31T14:47:03.052206+0000","last_active":"2024-07-31T14:49:49.407340+0000","last_peered":"2024-07-31T14:49:49.407340+0000","last_clean":"2024-07-31T14:49:49.407340+0000","last_became_active":"2024-07-31T14:47:03.051175+0000","last_became_peered":"2024-07-31T14:47:03.051175+0000","last_unstale":"2024-07-31T14:49:49.407340+0000","last_undegraded":"2024-07-31T14:49:49.407340+0000","last_fullsized":"2024-07-31T14:49:49.407340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:51:11.583604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.405579+0000","last_change":"2024-07-31T14:47:03.052162+0000","last_active":"2024-07-31T14:49:49.405579+0000","last_peered":"2024-07-31T14:49:49.405579+0000","last_clean":"2024-07-31T14:49:49.405579+0000","last_became_active":"2024-07-31T14:47:03.051634+0000","last_became_peered":"2024-07-31T14:47:03.051634+0000","last_unstale":"2024-07-31T14:49:49.405579+0000","last_undegraded":"2024-07-31T14:49:49.405579+0000","last_fullsized":"2024-07-31T14:49:49.405579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:41:59.605192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409010+0000","last_change":"2024-07-31T14:47:03.047475+0000","last_active":"2024-07-31T14:49:49.409010+0000","last_peered":"2024-07-31T14:49:49.409010+0000","last_clean":"2024-07-31T14:49:49.409010+0000","last_became_active":"2024-07-31T14:47:03.047100+0000","last_became_peered":"2024-07-31T14:47:03.047100+0000","last_unstale":"2024-07-31T14:49:49.409010+0000","last_undegraded":"2024-07-31T14:49:49.409010+0000","last_fullsized":"2024-07-31T14:49:49.409010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:08:10.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411379+0000","last_change":"2024-07-31T14:47:03.051281+0000","last_active":"2024-07-31T14:49:49.411379+0000","last_peered":"2024-07-31T14:49:49.411379+0000","last_clean":"2024-07-31T14:49:49.411379+0000","last_became_active":"2024-07-31T14:47:03.050921+0000","last_became_peered":"2024-07-31T14:47:03.050921+0000","last_unstale":"2024-07-31T14:49:49.411379+0000","last_undegraded":"2024-07-31T14:49:49.411379+0000","last_fullsized":"2024-07-31T14:49:49.411379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:24:29.221723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403114+0000","last_change":"2024-07-31T14:47:03.045457+0000","last_active":"2024-07-31T14:49:49.403114+0000","last_peered":"2024-07-31T14:49:49.403114+0000","last_clean":"2024-07-31T14:49:49.403114+0000","last_became_active":"2024-07-31T14:47:03.044466+0000","last_became_peered":"2024-07-31T14:47:03.044466+0000","last_unstale":"2024-07-31T14:49:49.403114+0000","last_undegraded":"2024-07-31T14:49:49.403114+0000","last_fullsized":"2024-07-31T14:49:49.403114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:44:01.197416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403872+0000","last_change":"2024-07-31T14:47:03.053431+0000","last_active":"2024-07-31T14:49:49.403872+0000","last_peered":"2024-07-31T14:49:49.403872+0000","last_clean":"2024-07-31T14:49:49.403872+0000","last_became_active":"2024-07-31T14:47:03.052958+0000","last_became_peered":"2024-07-31T14:47:03.052958+0000","last_unstale":"2024-07-31T14:49:49.403872+0000","last_undegraded":"2024-07-31T14:49:49.403872+0000","last_fullsized":"2024-07-31T14:49:49.403872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:03:28.863589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409420+0000","last_change":"2024-07-31T14:47:03.046641+0000","last_active":"2024-07-31T14:49:49.409420+0000","last_peered":"2024-07-31T14:49:49.409420+0000","last_clean":"2024-07-31T14:49:49.409420+0000","last_became_active":"2024-07-31T14:47:03.046265+0000","last_became_peered":"2024-07-31T14:47:03.046265+0000","last_unstale":"2024-07-31T14:49:49.409420+0000","last_undegraded":"2024-07-31T14:49:49.409420+0000","last_fullsized":"2024-07-31T14:49:49.409420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:20:25.389010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.387120+0000","last_change":"2024-07-31T14:47:03.051657+0000","last_active":"2024-07-31T14:49:49.387120+0000","last_peered":"2024-07-31T14:49:49.387120+0000","last_clean":"2024-07-31T14:49:49.387120+0000","last_became_active":"2024-07-31T14:47:03.050722+0000","last_became_peered":"2024-07-31T14:47:03.050722+0000","last_unstale":"2024-07-31T14:49:49.387120+0000","last_undegraded":"2024-07-31T14:49:49.387120+0000","last_fullsized":"2024-07-31T14:49:49.387120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:23:19.387470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411567+0000","last_change":"2024-07-31T14:47:03.059654+0000","last_active":"2024-07-31T14:49:49.411567+0000","last_peered":"2024-07-31T14:49:49.411567+0000","last_clean":"2024-07-31T14:49:49.411567+0000","last_became_active":"2024-07-31T14:47:03.056843+0000","last_became_peered":"2024-07-31T14:47:03.056843+0000","last_unstale":"2024-07-31T14:49:49.411567+0000","last_undegraded":"2024-07-31T14:49:49.411567+0000","last_fullsized":"2024-07-31T14:49:49.411567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:42:33.696498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413200+0000","last_change":"2024-07-31T14:47:03.052750+0000","last_active":"2024-07-31T14:49:49.413200+0000","last_peered":"2024-07-31T14:49:49.413200+0000","last_clean":"2024-07-31T14:49:49.413200+0000","last_became_active":"2024-07-31T14:47:03.051744+0000","last_became_peered":"2024-07-31T14:47:03.051744+0000","last_unstale":"2024-07-31T14:49:49.413200+0000","last_undegraded":"2024-07-31T14:49:49.413200+0000","last_fullsized":"2024-07-31T14:49:49.413200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:24:33.202918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.393222+0000","last_change":"2024-07-31T14:47:20.452563+0000","last_active":"2024-07-31T14:49:49.393222+0000","last_peered":"2024-07-31T14:49:49.393222+0000","last_clean":"2024-07-31T14:49:49.393222+0000","last_became_active":"2024-07-31T14:47:20.452271+0000","last_became_peered":"2024-07-31T14:47:20.452271+0000","last_unstale":"2024-07-31T14:49:49.393222+0000","last_undegraded":"2024-07-31T14:49:49.393222+0000","last_fullsized":"2024-07-31T14:49:49.393222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:26:55.070265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.391684+0000","last_change":"2024-07-31T14:47:03.050691+0000","last_active":"2024-07-31T14:49:49.391684+0000","last_peered":"2024-07-31T14:49:49.391684+0000","last_clean":"2024-07-31T14:49:49.391684+0000","last_became_active":"2024-07-31T14:47:03.050320+0000","last_became_peered":"2024-07-31T14:47:03.050320+0000","last_unstale":"2024-07-31T14:49:49.391684+0000","last_undegraded":"2024-07-31T14:49:49.391684+0000","last_fullsized":"2024-07-31T14:49:49.391684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:40:45.767706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400128+0000","last_change":"2024-07-31T14:47:03.048491+0000","last_active":"2024-07-31T14:49:49.400128+0000","last_peered":"2024-07-31T14:49:49.400128+0000","last_clean":"2024-07-31T14:49:49.400128+0000","last_became_active":"2024-07-31T14:47:03.048085+0000","last_became_peered":"2024-07-31T14:47:03.048085+0000","last_unstale":"2024-07-31T14:49:49.400128+0000","last_undegraded":"2024-07-31T14:49:49.400128+0000","last_fullsized":"2024-07-31T14:49:49.400128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:35:01.762365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.395425+0000","last_change":"2024-07-31T14:47:03.055502+0000","last_active":"2024-07-31T14:49:49.395425+0000","last_peered":"2024-07-31T14:49:49.395425+0000","last_clean":"2024-07-31T14:49:49.395425+0000","last_became_active":"2024-07-31T14:47:03.055015+0000","last_became_peered":"2024-07-31T14:47:03.055015+0000","last_unstale":"2024-07-31T14:49:49.395425+0000","last_undegraded":"2024-07-31T14:49:49.395425+0000","last_fullsized":"2024-07-31T14:49:49.395425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:50:39.605148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413933+0000","last_change":"2024-07-31T14:47:03.057808+0000","last_active":"2024-07-31T14:49:49.413933+0000","last_peered":"2024-07-31T14:49:49.413933+0000","last_clean":"2024-07-31T14:49:49.413933+0000","last_became_active":"2024-07-31T14:47:03.057443+0000","last_became_peered":"2024-07-31T14:47:03.057443+0000","last_unstale":"2024-07-31T14:49:49.413933+0000","last_undegraded":"2024-07-31T14:49:49.413933+0000","last_fullsized":"2024-07-31T14:49:49.413933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:34:42.915428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396116+0000","last_change":"2024-07-31T14:47:03.058182+0000","last_active":"2024-07-31T14:49:49.396116+0000","last_peered":"2024-07-31T14:49:49.396116+0000","last_clean":"2024-07-31T14:49:49.396116+0000","last_became_active":"2024-07-31T14:47:03.057802+0000","last_became_peered":"2024-07-31T14:47:03.057802+0000","last_unstale":"2024-07-31T14:49:49.396116+0000","last_undegraded":"2024-07-31T14:49:49.396116+0000","last_fullsized":"2024-07-31T14:49:49.396116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:08:06.608788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413379+0000","last_change":"2024-07-31T14:47:03.058882+0000","last_active":"2024-07-31T14:49:49.413379+0000","last_peered":"2024-07-31T14:49:49.413379+0000","last_clean":"2024-07-31T14:49:49.413379+0000","last_became_active":"2024-07-31T14:47:03.058530+0000","last_became_peered":"2024-07-31T14:47:03.058530+0000","last_unstale":"2024-07-31T14:49:49.413379+0000","last_undegraded":"2024-07-31T14:49:49.413379+0000","last_fullsized":"2024-07-31T14:49:49.413379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:28:03.631962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404760+0000","last_change":"2024-07-31T14:47:03.052028+0000","last_active":"2024-07-31T14:49:49.404760+0000","last_peered":"2024-07-31T14:49:49.404760+0000","last_clean":"2024-07-31T14:49:49.404760+0000","last_became_active":"2024-07-31T14:47:03.051559+0000","last_became_peered":"2024-07-31T14:47:03.051559+0000","last_unstale":"2024-07-31T14:49:49.404760+0000","last_undegraded":"2024-07-31T14:49:49.404760+0000","last_fullsized":"2024-07-31T14:49:49.404760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:16:40.723795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399075+0000","last_change":"2024-07-31T14:47:03.053884+0000","last_active":"2024-07-31T14:49:49.399075+0000","last_peered":"2024-07-31T14:49:49.399075+0000","last_clean":"2024-07-31T14:49:49.399075+0000","last_became_active":"2024-07-31T14:47:03.053521+0000","last_became_peered":"2024-07-31T14:47:03.053521+0000","last_unstale":"2024-07-31T14:49:49.399075+0000","last_undegraded":"2024-07-31T14:49:49.399075+0000","last_fullsized":"2024-07-31T14:49:49.399075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:16:37.702055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.394011+0000","last_change":"2024-07-31T14:47:03.058588+0000","last_active":"2024-07-31T14:49:49.394011+0000","last_peered":"2024-07-31T14:49:49.394011+0000","last_clean":"2024-07-31T14:49:49.394011+0000","last_became_active":"2024-07-31T14:47:03.057820+0000","last_became_peered":"2024-07-31T14:47:03.057820+0000","last_unstale":"2024-07-31T14:49:49.394011+0000","last_undegraded":"2024-07-31T14:49:49.394011+0000","last_fullsized":"2024-07-31T14:49:49.394011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:21.265208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396703+0000","last_change":"2024-07-31T14:47:03.041447+0000","last_active":"2024-07-31T14:49:49.396703+0000","last_peered":"2024-07-31T14:49:49.396703+0000","last_clean":"2024-07-31T14:49:49.396703+0000","last_became_active":"2024-07-31T14:47:03.040935+0000","last_became_peered":"2024-07-31T14:47:03.040935+0000","last_unstale":"2024-07-31T14:49:49.396703+0000","last_undegraded":"2024-07-31T14:49:49.396703+0000","last_fullsized":"2024-07-31T14:49:49.396703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:00:13.624698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396755+0000","last_change":"2024-07-31T14:47:20.452239+0000","last_active":"2024-07-31T14:49:49.396755+0000","last_peered":"2024-07-31T14:49:49.396755+0000","last_clean":"2024-07-31T14:49:49.396755+0000","last_became_active":"2024-07-31T14:47:20.451924+0000","last_became_peered":"2024-07-31T14:47:20.451924+0000","last_unstale":"2024-07-31T14:49:49.396755+0000","last_undegraded":"2024-07-31T14:49:49.396755+0000","last_fullsized":"2024-07-31T14:49:49.396755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:58:24.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403996+0000","last_change":"2024-07-31T14:47:03.056024+0000","last_active":"2024-07-31T14:49:49.403996+0000","last_peered":"2024-07-31T14:49:49.403996+0000","last_clean":"2024-07-31T14:49:49.403996+0000","last_became_active":"2024-07-31T14:47:03.055630+0000","last_became_peered":"2024-07-31T14:47:03.055630+0000","last_unstale":"2024-07-31T14:49:49.403996+0000","last_undegraded":"2024-07-31T14:49:49.403996+0000","last_fullsized":"2024-07-31T14:49:49.403996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:09:11.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.390933+0000","last_change":"2024-07-31T14:47:03.053718+0000","last_active":"2024-07-31T14:49:49.390933+0000","last_peered":"2024-07-31T14:49:49.390933+0000","last_clean":"2024-07-31T14:49:49.390933+0000","last_became_active":"2024-07-31T14:47:03.053361+0000","last_became_peered":"2024-07-31T14:47:03.053361+0000","last_unstale":"2024-07-31T14:49:49.390933+0000","last_undegraded":"2024-07-31T14:49:49.390933+0000","last_fullsized":"2024-07-31T14:49:49.390933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:03:49.220838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411510+0000","last_change":"2024-07-31T14:47:03.056087+0000","last_active":"2024-07-31T14:49:49.411510+0000","last_peered":"2024-07-31T14:49:49.411510+0000","last_clean":"2024-07-31T14:49:49.411510+0000","last_became_active":"2024-07-31T14:47:03.055732+0000","last_became_peered":"2024-07-31T14:47:03.055732+0000","last_unstale":"2024-07-31T14:49:49.411510+0000","last_undegraded":"2024-07-31T14:49:49.411510+0000","last_fullsized":"2024-07-31T14:49:49.411510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:45:01.185666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78506,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1928,"num_read_kb":1051,"num_write":276,"num_write_kb":169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":238038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645376,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":520038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841605,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37020,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708132,"statfs":{"total":95995035648,"available":95957127168,"internally_reserved":0,"allocated":1724416,"data_stored":1176867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005131,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":523756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201360,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":520038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397588,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1118131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561114,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":515696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757343,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1112258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920867,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":519770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117095,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":514872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378733,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":589563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":581419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705782,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":517319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":72057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-31T14:51:14.054 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-31T14:51:14.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.202+0000 7ff2fe970640 1 -- 172.21.15.67:0/3321863748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f8108670 msgr2=0x7ff2f810eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:14.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.202+0000 7ff2fe970640 1 --2- 172.21.15.67:0/3321863748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f8108670 0x7ff2f810eea0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009ca0 tx=0x7ff2f002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.202+0000 7ff2fe970640 1 -- 172.21.15.67:0/3321863748 shutdown_connections 2024-07-31T14:51:14.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.202+0000 7ff2fe970640 1 --2- 172.21.15.67:0/3321863748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f8108670 0x7ff2f810eea0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.202+0000 7ff2fe970640 1 --2- 172.21.15.67:0/3321863748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2f8107d20 0x7ff2f8108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.202+0000 7ff2fe970640 1 --2- 172.21.15.67:0/3321863748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f81023f0 0x7ff2f8102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.202+0000 7ff2fe970640 1 -- 172.21.15.67:0/3321863748 >> 172.21.15.67:0/3321863748 conn(0x7ff2f807a090 msgr2=0x7ff2f807a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:14.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.202+0000 7ff2fe970640 1 -- 172.21.15.67:0/3321863748 shutdown_connections 2024-07-31T14:51:14.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.202+0000 7ff2fe970640 1 -- 172.21.15.67:0/3321863748 wait complete. 2024-07-31T14:51:14.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.203+0000 7ff2fe970640 1 Processor -- start 2024-07-31T14:51:14.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.203+0000 7ff2fe970640 1 -- start start 2024-07-31T14:51:14.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.203+0000 7ff2fe970640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2f81023f0 0x7ff2f819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:14.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.203+0000 7ff2fe970640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f8107d20 0x7ff2f819b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:14.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.203+0000 7ff2fe970640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f8108670 0x7ff2f819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:14.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.203+0000 7ff2fe970640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f8112320 con 0x7ff2f8108670 2024-07-31T14:51:14.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.203+0000 7ff2fe970640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81121a0 con 0x7ff2f8107d20 2024-07-31T14:51:14.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.203+0000 7ff2fe970640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81124a0 con 0x7ff2f81023f0 2024-07-31T14:51:14.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.203+0000 7ff2fcf0d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f8108670 0x7ff2f819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:14.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.203+0000 7ff2fcf0d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f8108670 0x7ff2f819b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52726/0 (socket says 172.21.15.67:52726) 2024-07-31T14:51:14.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.203+0000 7ff2fcf0d640 1 -- 172.21.15.67:0/756394448 learned_addr learned my addr 172.21.15.67:0/756394448 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:14.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.203+0000 7ff2f77fe640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f8107d20 0x7ff2f819b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:14.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.204+0000 7ff2f7fff640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2f81023f0 0x7ff2f819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:14.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.204+0000 7ff2fcf0d640 1 -- 172.21.15.67:0/756394448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2f81023f0 msgr2=0x7ff2f819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:14.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.204+0000 7ff2fcf0d640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2f81023f0 0x7ff2f819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.204+0000 7ff2fcf0d640 1 -- 172.21.15.67:0/756394448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f8107d20 msgr2=0x7ff2f819b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:14.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.204+0000 7ff2fcf0d640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f8107d20 0x7ff2f819b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.204+0000 7ff2fcf0d640 1 -- 172.21.15.67:0/756394448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f81a5e70 con 0x7ff2f8108670 2024-07-31T14:51:14.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.204+0000 7ff2f7fff640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2f81023f0 0x7ff2f819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:14.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.204+0000 7ff2fcf0d640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f8108670 0x7ff2f819b670 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00098d0 tx=0x7ff2f00317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:14.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.204+0000 7ff2f57fa640 1 -- 172.21.15.67:0/756394448 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0003b70 con 0x7ff2f8108670 2024-07-31T14:51:14.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.204+0000 7ff2fe970640 1 -- 172.21.15.67:0/756394448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f81a6100 con 0x7ff2f8108670 2024-07-31T14:51:14.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.204+0000 7ff2f57fa640 1 -- 172.21.15.67:0/756394448 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f0003d10 con 0x7ff2f8108670 2024-07-31T14:51:14.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.204+0000 7ff2f57fa640 1 -- 172.21.15.67:0/756394448 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f002f9d0 con 0x7ff2f8108670 2024-07-31T14:51:14.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.204+0000 7ff2fe970640 1 -- 172.21.15.67:0/756394448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f81a6650 con 0x7ff2f8108670 2024-07-31T14:51:14.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.205+0000 7ff2fe970640 1 -- 172.21.15.67:0/756394448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f8103660 con 0x7ff2f8108670 2024-07-31T14:51:14.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.207+0000 7ff2f57fa640 1 -- 172.21.15.67:0/756394448 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7ff2f002fb70 con 0x7ff2f8108670 2024-07-31T14:51:14.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.208+0000 7ff2f57fa640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff2e00b3cb0 0x7ff2e00b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:14.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.208+0000 7ff2f7fff640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff2e00b3cb0 0x7ff2e00b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:14.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.208+0000 7ff2f57fa640 1 -- 172.21.15.67:0/756394448 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff2f0105720 con 0x7ff2f8108670 2024-07-31T14:51:14.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.209+0000 7ff2f7fff640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff2e00b3cb0 0x7ff2e00b61c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff2f806ae60 tx=0x7ff2e8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:14.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.211+0000 7ff2f57fa640 1 -- 172.21.15.67:0/756394448 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff2f00cc8f0 con 0x7ff2f8108670 2024-07-31T14:51:14.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.401+0000 7ff2fe970640 1 -- 172.21.15.67:0/756394448 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2f8063230 con 0x7ff2e00b3cb0 2024-07-31T14:51:14.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.407+0000 7ff2f57fa640 1 -- 172.21.15.67:0/756394448 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340533 (secure 0 0 0) 0x7ff2f8063230 con 0x7ff2e00b3cb0 2024-07-31T14:51:14.412 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:51:14.414 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-31T14:51:14.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.411+0000 7ff2fe970640 1 -- 172.21.15.67:0/756394448 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff2e00b3cb0 msgr2=0x7ff2e00b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:14.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.411+0000 7ff2fe970640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff2e00b3cb0 0x7ff2e00b61c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff2f806ae60 tx=0x7ff2e8005f90 comp rx=0 tx=0).stop 2024-07-31T14:51:14.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.411+0000 7ff2fe970640 1 -- 172.21.15.67:0/756394448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f8108670 msgr2=0x7ff2f819b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:14.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.411+0000 7ff2fe970640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f8108670 0x7ff2f819b670 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00098d0 tx=0x7ff2f00317b0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.411+0000 7ff2fe970640 1 -- 172.21.15.67:0/756394448 shutdown_connections 2024-07-31T14:51:14.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.411+0000 7ff2fe970640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff2e00b3cb0 0x7ff2e00b61c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.411+0000 7ff2fe970640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f8108670 0x7ff2f819b670 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.411+0000 7ff2fe970640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f8107d20 0x7ff2f819b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.411+0000 7ff2fe970640 1 --2- 172.21.15.67:0/756394448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2f81023f0 0x7ff2f819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.411+0000 7ff2fe970640 1 -- 172.21.15.67:0/756394448 >> 172.21.15.67:0/756394448 conn(0x7ff2f807a090 msgr2=0x7ff2f8105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:14.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.411+0000 7ff2fe970640 1 -- 172.21.15.67:0/756394448 shutdown_connections 2024-07-31T14:51:14.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.411+0000 7ff2fe970640 1 -- 172.21.15.67:0/756394448 wait complete. 2024-07-31T14:51:14.423 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-07-31T14:51:13.443366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669108,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2009,"num_read_kb":1120,"num_write":401,"num_write_kb":2321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":830,"ondisk_log_size":830,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384108,"kb_used_data":14156,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557716,"statfs":{"total":1151940427776,"available":1151547101184,"internally_reserved":0,"allocated":14495744,"data_stored":8209727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":378713403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6103,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":68,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003318"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":245,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:10.893183+0000","last_change":"2024-07-31T14:47:00.020067+0000","last_active":"2024-07-31T14:51:10.893183+0000","last_peered":"2024-07-31T14:51:10.893183+0000","last_clean":"2024-07-31T14:51:10.893183+0000","last_became_active":"2024-07-31T14:47:00.019674+0000","last_became_peered":"2024-07-31T14:47:00.019674+0000","last_unstale":"2024-07-31T14:51:10.893183+0000","last_undegraded":"2024-07-31T14:51:10.893183+0000","last_fullsized":"2024-07-31T14:51:10.893183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:10:08.193786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":189,"num_read_kb":189,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.408739+0000","last_change":"2024-07-31T14:47:03.052069+0000","last_active":"2024-07-31T14:49:49.408739+0000","last_peered":"2024-07-31T14:49:49.408739+0000","last_clean":"2024-07-31T14:49:49.408739+0000","last_became_active":"2024-07-31T14:47:03.051739+0000","last_became_peered":"2024-07-31T14:47:03.051739+0000","last_unstale":"2024-07-31T14:49:49.408739+0000","last_undegraded":"2024-07-31T14:49:49.408739+0000","last_fullsized":"2024-07-31T14:49:49.408739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:29:19.952794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.629384+0000","last_change":"2024-07-31T14:47:20.628770+0000","last_active":"2024-07-31T14:47:20.629384+0000","last_peered":"2024-07-31T14:47:20.629384+0000","last_clean":"2024-07-31T14:47:20.629384+0000","last_became_active":"2024-07-31T14:47:20.628142+0000","last_became_peered":"2024-07-31T14:47:20.628142+0000","last_unstale":"2024-07-31T14:47:20.629384+0000","last_undegraded":"2024-07-31T14:47:20.629384+0000","last_fullsized":"2024-07-31T14:47:20.629384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.659670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:59.291402+0000","last_change":"2024-07-31T14:47:00.183893+0000","last_active":"2024-07-31T14:48:59.291402+0000","last_peered":"2024-07-31T14:48:59.291402+0000","last_clean":"2024-07-31T14:48:59.291402+0000","last_became_active":"2024-07-31T14:47:00.183453+0000","last_became_peered":"2024-07-31T14:47:00.183453+0000","last_unstale":"2024-07-31T14:48:59.291402+0000","last_undegraded":"2024-07-31T14:48:59.291402+0000","last_fullsized":"2024-07-31T14:48:59.291402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:45:08.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404252+0000","last_change":"2024-07-31T14:47:03.057081+0000","last_active":"2024-07-31T14:49:49.404252+0000","last_peered":"2024-07-31T14:49:49.404252+0000","last_clean":"2024-07-31T14:49:49.404252+0000","last_became_active":"2024-07-31T14:47:03.056061+0000","last_became_peered":"2024-07-31T14:47:03.056061+0000","last_unstale":"2024-07-31T14:49:49.404252+0000","last_undegraded":"2024-07-31T14:49:49.404252+0000","last_fullsized":"2024-07-31T14:49:49.404252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:28:16.760131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604121+0000","last_change":"2024-07-31T14:47:13.105271+0000","last_active":"2024-07-31T14:47:20.604121+0000","last_peered":"2024-07-31T14:47:20.604121+0000","last_clean":"2024-07-31T14:47:20.604121+0000","last_became_active":"2024-07-31T14:47:13.104717+0000","last_became_peered":"2024-07-31T14:47:13.104717+0000","last_unstale":"2024-07-31T14:47:20.604121+0000","last_undegraded":"2024-07-31T14:47:20.604121+0000","last_fullsized":"2024-07-31T14:47:20.604121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:34:21.413957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:04.594438+0000","last_change":"2024-07-31T14:47:20.454745+0000","last_active":"2024-07-31T14:48:04.594438+0000","last_peered":"2024-07-31T14:48:04.594438+0000","last_clean":"2024-07-31T14:48:04.594438+0000","last_became_active":"2024-07-31T14:47:20.449892+0000","last_became_peered":"2024-07-31T14:47:20.449892+0000","last_unstale":"2024-07-31T14:48:04.594438+0000","last_undegraded":"2024-07-31T14:48:04.594438+0000","last_fullsized":"2024-07-31T14:48:04.594438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:59:17.375344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.412662+0000","last_change":"2024-07-31T14:47:03.051085+0000","last_active":"2024-07-31T14:49:49.412662+0000","last_peered":"2024-07-31T14:49:49.412662+0000","last_clean":"2024-07-31T14:49:49.412662+0000","last_became_active":"2024-07-31T14:47:03.050686+0000","last_became_peered":"2024-07-31T14:47:03.050686+0000","last_unstale":"2024-07-31T14:49:49.412662+0000","last_undegraded":"2024-07-31T14:49:49.412662+0000","last_fullsized":"2024-07-31T14:49:49.412662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:48:02.494061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604501+0000","last_change":"2024-07-31T14:47:13.104229+0000","last_active":"2024-07-31T14:47:20.604501+0000","last_peered":"2024-07-31T14:47:20.604501+0000","last_clean":"2024-07-31T14:47:20.604501+0000","last_became_active":"2024-07-31T14:47:13.103737+0000","last_became_peered":"2024-07-31T14:47:13.103737+0000","last_unstale":"2024-07-31T14:47:20.604501+0000","last_undegraded":"2024-07-31T14:47:20.604501+0000","last_fullsized":"2024-07-31T14:47:20.604501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:36:00.541345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609858+0000","last_change":"2024-07-31T14:47:00.028169+0000","last_active":"2024-07-31T14:48:03.609858+0000","last_peered":"2024-07-31T14:48:03.609858+0000","last_clean":"2024-07-31T14:48:03.609858+0000","last_became_active":"2024-07-31T14:47:00.027757+0000","last_became_peered":"2024-07-31T14:47:00.027757+0000","last_unstale":"2024-07-31T14:48:03.609858+0000","last_undegraded":"2024-07-31T14:48:03.609858+0000","last_fullsized":"2024-07-31T14:48:03.609858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:36:04.419551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413724+0000","last_change":"2024-07-31T14:47:03.057400+0000","last_active":"2024-07-31T14:49:49.413724+0000","last_peered":"2024-07-31T14:49:49.413724+0000","last_clean":"2024-07-31T14:49:49.413724+0000","last_became_active":"2024-07-31T14:47:03.057035+0000","last_became_peered":"2024-07-31T14:47:03.057035+0000","last_unstale":"2024-07-31T14:49:49.413724+0000","last_undegraded":"2024-07-31T14:49:49.413724+0000","last_fullsized":"2024-07-31T14:49:49.413724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:55:07.951076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604244+0000","last_change":"2024-07-31T14:47:13.106093+0000","last_active":"2024-07-31T14:47:20.604244+0000","last_peered":"2024-07-31T14:47:20.604244+0000","last_clean":"2024-07-31T14:47:20.604244+0000","last_became_active":"2024-07-31T14:47:13.105459+0000","last_became_peered":"2024-07-31T14:47:13.105459+0000","last_unstale":"2024-07-31T14:47:20.604244+0000","last_undegraded":"2024-07-31T14:47:20.604244+0000","last_fullsized":"2024-07-31T14:47:20.604244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:07:52.751245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414377+0000","last_change":"2024-07-31T14:47:03.058701+0000","last_active":"2024-07-31T14:49:49.414377+0000","last_peered":"2024-07-31T14:49:49.414377+0000","last_clean":"2024-07-31T14:49:49.414377+0000","last_became_active":"2024-07-31T14:47:03.058359+0000","last_became_peered":"2024-07-31T14:47:03.058359+0000","last_unstale":"2024-07-31T14:49:49.414377+0000","last_undegraded":"2024-07-31T14:49:49.414377+0000","last_fullsized":"2024-07-31T14:49:49.414377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:37:36.149884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605126+0000","last_change":"2024-07-31T14:47:00.022983+0000","last_active":"2024-07-31T14:48:03.605126+0000","last_peered":"2024-07-31T14:48:03.605126+0000","last_clean":"2024-07-31T14:48:03.605126+0000","last_became_active":"2024-07-31T14:47:00.022586+0000","last_became_peered":"2024-07-31T14:47:00.022586+0000","last_unstale":"2024-07-31T14:48:03.605126+0000","last_undegraded":"2024-07-31T14:48:03.605126+0000","last_fullsized":"2024-07-31T14:48:03.605126+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:29:42.641490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604474+0000","last_change":"2024-07-31T14:47:13.363148+0000","last_active":"2024-07-31T14:47:20.604474+0000","last_peered":"2024-07-31T14:47:20.604474+0000","last_clean":"2024-07-31T14:47:20.604474+0000","last_became_active":"2024-07-31T14:47:13.362577+0000","last_became_peered":"2024-07-31T14:47:13.362577+0000","last_unstale":"2024-07-31T14:47:20.604474+0000","last_undegraded":"2024-07-31T14:47:20.604474+0000","last_fullsized":"2024-07-31T14:47:20.604474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:37:59.003260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409563+0000","last_change":"2024-07-31T14:47:03.049586+0000","last_active":"2024-07-31T14:49:49.409563+0000","last_peered":"2024-07-31T14:49:49.409563+0000","last_clean":"2024-07-31T14:49:49.409563+0000","last_became_active":"2024-07-31T14:47:03.049154+0000","last_became_peered":"2024-07-31T14:47:03.049154+0000","last_unstale":"2024-07-31T14:49:49.409563+0000","last_undegraded":"2024-07-31T14:49:49.409563+0000","last_fullsized":"2024-07-31T14:49:49.409563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:51:01.971854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'5","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:07.609943+0000","last_change":"2024-07-31T14:47:00.021254+0000","last_active":"2024-07-31T14:51:07.609943+0000","last_peered":"2024-07-31T14:51:07.609943+0000","last_clean":"2024-07-31T14:51:07.609943+0000","last_became_active":"2024-07-31T14:47:00.020690+0000","last_became_peered":"2024-07-31T14:47:00.020690+0000","last_unstale":"2024-07-31T14:51:07.609943+0000","last_undegraded":"2024-07-31T14:51:07.609943+0000","last_fullsized":"2024-07-31T14:51:07.609943+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:01:27.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:18.046730+0000","last_change":"2024-07-31T14:47:13.102220+0000","last_active":"2024-07-31T14:48:18.046730+0000","last_peered":"2024-07-31T14:48:18.046730+0000","last_clean":"2024-07-31T14:48:18.046730+0000","last_became_active":"2024-07-31T14:47:13.101764+0000","last_became_peered":"2024-07-31T14:47:13.101764+0000","last_unstale":"2024-07-31T14:48:18.046730+0000","last_undegraded":"2024-07-31T14:48:18.046730+0000","last_fullsized":"2024-07-31T14:48:18.046730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:56:44.335340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.397183+0000","last_change":"2024-07-31T14:47:03.052348+0000","last_active":"2024-07-31T14:49:49.397183+0000","last_peered":"2024-07-31T14:49:49.397183+0000","last_clean":"2024-07-31T14:49:49.397183+0000","last_became_active":"2024-07-31T14:47:03.051961+0000","last_became_peered":"2024-07-31T14:47:03.051961+0000","last_unstale":"2024-07-31T14:49:49.397183+0000","last_undegraded":"2024-07-31T14:49:49.397183+0000","last_fullsized":"2024-07-31T14:49:49.397183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:19:35.279809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.597386+0000","last_change":"2024-07-31T14:47:00.025013+0000","last_active":"2024-07-31T14:48:02.597386+0000","last_peered":"2024-07-31T14:48:02.597386+0000","last_clean":"2024-07-31T14:48:02.597386+0000","last_became_active":"2024-07-31T14:47:00.024637+0000","last_became_peered":"2024-07-31T14:47:00.024637+0000","last_unstale":"2024-07-31T14:48:02.597386+0000","last_undegraded":"2024-07-31T14:48:02.597386+0000","last_fullsized":"2024-07-31T14:48:02.597386+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:07:21.519549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630339+0000","last_change":"2024-07-31T14:47:20.629262+0000","last_active":"2024-07-31T14:47:20.630339+0000","last_peered":"2024-07-31T14:47:20.630339+0000","last_clean":"2024-07-31T14:47:20.630339+0000","last_became_active":"2024-07-31T14:47:20.628318+0000","last_became_peered":"2024-07-31T14:47:20.628318+0000","last_unstale":"2024-07-31T14:47:20.630339+0000","last_undegraded":"2024-07-31T14:47:20.630339+0000","last_fullsized":"2024-07-31T14:47:20.630339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.206705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.361650+0000","last_change":"2024-07-31T14:47:03.051527+0000","last_active":"2024-07-31T14:49:49.361650+0000","last_peered":"2024-07-31T14:49:49.361650+0000","last_clean":"2024-07-31T14:49:49.361650+0000","last_became_active":"2024-07-31T14:47:03.051139+0000","last_became_peered":"2024-07-31T14:47:03.051139+0000","last_unstale":"2024-07-31T14:49:49.361650+0000","last_undegraded":"2024-07-31T14:49:49.361650+0000","last_fullsized":"2024-07-31T14:49:49.361650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:53:11.962719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:59.566389+0000","last_change":"2024-07-31T14:47:00.183812+0000","last_active":"2024-07-31T14:47:59.566389+0000","last_peered":"2024-07-31T14:47:59.566389+0000","last_clean":"2024-07-31T14:47:59.566389+0000","last_became_active":"2024-07-31T14:47:00.183440+0000","last_became_peered":"2024-07-31T14:47:00.183440+0000","last_unstale":"2024-07-31T14:47:59.566389+0000","last_undegraded":"2024-07-31T14:47:59.566389+0000","last_fullsized":"2024-07-31T14:47:59.566389+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:47:23.166756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604338+0000","last_change":"2024-07-31T14:47:13.363563+0000","last_active":"2024-07-31T14:47:20.604338+0000","last_peered":"2024-07-31T14:47:20.604338+0000","last_clean":"2024-07-31T14:47:20.604338+0000","last_became_active":"2024-07-31T14:47:13.362697+0000","last_became_peered":"2024-07-31T14:47:13.362697+0000","last_unstale":"2024-07-31T14:47:20.604338+0000","last_undegraded":"2024-07-31T14:47:20.604338+0000","last_fullsized":"2024-07-31T14:47:20.604338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:26:38.292073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407720+0000","last_change":"2024-07-31T14:47:03.046763+0000","last_active":"2024-07-31T14:49:49.407720+0000","last_peered":"2024-07-31T14:49:49.407720+0000","last_clean":"2024-07-31T14:49:49.407720+0000","last_became_active":"2024-07-31T14:47:03.046408+0000","last_became_peered":"2024-07-31T14:47:03.046408+0000","last_unstale":"2024-07-31T14:49:49.407720+0000","last_undegraded":"2024-07-31T14:49:49.407720+0000","last_fullsized":"2024-07-31T14:49:49.407720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:43:22.838495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605677+0000","last_change":"2024-07-31T14:47:00.236075+0000","last_active":"2024-07-31T14:48:03.605677+0000","last_peered":"2024-07-31T14:48:03.605677+0000","last_clean":"2024-07-31T14:48:03.605677+0000","last_became_active":"2024-07-31T14:47:00.235758+0000","last_became_peered":"2024-07-31T14:47:00.235758+0000","last_unstale":"2024-07-31T14:48:03.605677+0000","last_undegraded":"2024-07-31T14:48:03.605677+0000","last_fullsized":"2024-07-31T14:48:03.605677+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:50.575648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.448218+0000","last_change":"2024-07-31T14:47:13.101876+0000","last_active":"2024-07-31T14:47:20.448218+0000","last_peered":"2024-07-31T14:47:20.448218+0000","last_clean":"2024-07-31T14:47:20.448218+0000","last_became_active":"2024-07-31T14:47:13.101276+0000","last_became_peered":"2024-07-31T14:47:13.101276+0000","last_unstale":"2024-07-31T14:47:20.448218+0000","last_undegraded":"2024-07-31T14:47:20.448218+0000","last_fullsized":"2024-07-31T14:47:20.448218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:53:24.684212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409477+0000","last_change":"2024-07-31T14:47:03.052478+0000","last_active":"2024-07-31T14:49:49.409477+0000","last_peered":"2024-07-31T14:49:49.409477+0000","last_clean":"2024-07-31T14:49:49.409477+0000","last_became_active":"2024-07-31T14:47:03.051877+0000","last_became_peered":"2024-07-31T14:47:03.051877+0000","last_unstale":"2024-07-31T14:49:49.409477+0000","last_undegraded":"2024-07-31T14:49:49.409477+0000","last_fullsized":"2024-07-31T14:49:49.409477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:30:32.666490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:07.082692+0000","last_change":"2024-07-31T14:47:00.020076+0000","last_active":"2024-07-31T14:51:07.082692+0000","last_peered":"2024-07-31T14:51:07.082692+0000","last_clean":"2024-07-31T14:51:07.082692+0000","last_became_active":"2024-07-31T14:47:00.019708+0000","last_became_peered":"2024-07-31T14:47:00.019708+0000","last_unstale":"2024-07-31T14:51:07.082692+0000","last_undegraded":"2024-07-31T14:51:07.082692+0000","last_fullsized":"2024-07-31T14:51:07.082692+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:46.520997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625840+0000","last_change":"2024-07-31T14:47:13.362961+0000","last_active":"2024-07-31T14:47:20.625840+0000","last_peered":"2024-07-31T14:47:20.625840+0000","last_clean":"2024-07-31T14:47:20.625840+0000","last_became_active":"2024-07-31T14:47:13.362502+0000","last_became_peered":"2024-07-31T14:47:13.362502+0000","last_unstale":"2024-07-31T14:47:20.625840+0000","last_undegraded":"2024-07-31T14:47:20.625840+0000","last_fullsized":"2024-07-31T14:47:20.625840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:51:45.732477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399783+0000","last_change":"2024-07-31T14:47:03.039418+0000","last_active":"2024-07-31T14:49:49.399783+0000","last_peered":"2024-07-31T14:49:49.399783+0000","last_clean":"2024-07-31T14:49:49.399783+0000","last_became_active":"2024-07-31T14:47:03.039044+0000","last_became_peered":"2024-07-31T14:47:03.039044+0000","last_unstale":"2024-07-31T14:49:49.399783+0000","last_undegraded":"2024-07-31T14:49:49.399783+0000","last_fullsized":"2024-07-31T14:49:49.399783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:40:10.380080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:07.059440+0000","last_change":"2024-07-31T14:47:00.183778+0000","last_active":"2024-07-31T14:49:07.059440+0000","last_peered":"2024-07-31T14:49:07.059440+0000","last_clean":"2024-07-31T14:49:07.059440+0000","last_became_active":"2024-07-31T14:47:00.183370+0000","last_became_peered":"2024-07-31T14:47:00.183370+0000","last_unstale":"2024-07-31T14:49:07.059440+0000","last_undegraded":"2024-07-31T14:49:07.059440+0000","last_fullsized":"2024-07-31T14:49:07.059440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:45:27.422216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445139+0000","last_change":"2024-07-31T14:47:13.102583+0000","last_active":"2024-07-31T14:47:20.445139+0000","last_peered":"2024-07-31T14:47:20.445139+0000","last_clean":"2024-07-31T14:47:20.445139+0000","last_became_active":"2024-07-31T14:47:13.102058+0000","last_became_peered":"2024-07-31T14:47:13.102058+0000","last_unstale":"2024-07-31T14:47:20.445139+0000","last_undegraded":"2024-07-31T14:47:20.445139+0000","last_fullsized":"2024-07-31T14:47:20.445139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:17.396173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407879+0000","last_change":"2024-07-31T14:47:03.048334+0000","last_active":"2024-07-31T14:49:49.407879+0000","last_peered":"2024-07-31T14:49:49.407879+0000","last_clean":"2024-07-31T14:49:49.407879+0000","last_became_active":"2024-07-31T14:47:03.047932+0000","last_became_peered":"2024-07-31T14:47:03.047932+0000","last_unstale":"2024-07-31T14:49:49.407879+0000","last_undegraded":"2024-07-31T14:49:49.407879+0000","last_fullsized":"2024-07-31T14:49:49.407879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:14:31.381061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:06.494830+0000","last_change":"2024-07-31T14:47:20.450851+0000","last_active":"2024-07-31T14:49:06.494830+0000","last_peered":"2024-07-31T14:49:06.494830+0000","last_clean":"2024-07-31T14:49:06.494830+0000","last_became_active":"2024-07-31T14:47:20.450435+0000","last_became_peered":"2024-07-31T14:47:20.450435+0000","last_unstale":"2024-07-31T14:49:06.494830+0000","last_undegraded":"2024-07-31T14:49:06.494830+0000","last_fullsized":"2024-07-31T14:49:06.494830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:17:28.738911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.450860+0000","last_change":"2024-07-31T14:47:13.363054+0000","last_active":"2024-07-31T14:47:20.450860+0000","last_peered":"2024-07-31T14:47:20.450860+0000","last_clean":"2024-07-31T14:47:20.450860+0000","last_became_active":"2024-07-31T14:47:13.362364+0000","last_became_peered":"2024-07-31T14:47:13.362364+0000","last_unstale":"2024-07-31T14:47:20.450860+0000","last_undegraded":"2024-07-31T14:47:20.450860+0000","last_fullsized":"2024-07-31T14:47:20.450860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:02:36.569678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.393142+0000","last_change":"2024-07-31T14:47:03.059834+0000","last_active":"2024-07-31T14:49:49.393142+0000","last_peered":"2024-07-31T14:49:49.393142+0000","last_clean":"2024-07-31T14:49:49.393142+0000","last_became_active":"2024-07-31T14:47:03.059127+0000","last_became_peered":"2024-07-31T14:47:03.059127+0000","last_unstale":"2024-07-31T14:49:49.393142+0000","last_undegraded":"2024-07-31T14:49:49.393142+0000","last_fullsized":"2024-07-31T14:49:49.393142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:53:50.308789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":241,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:10.893959+0000","last_change":"2024-07-31T14:47:00.030576+0000","last_active":"2024-07-31T14:51:10.893959+0000","last_peered":"2024-07-31T14:51:10.893959+0000","last_clean":"2024-07-31T14:51:10.893959+0000","last_became_active":"2024-07-31T14:47:00.030174+0000","last_became_peered":"2024-07-31T14:47:00.030174+0000","last_unstale":"2024-07-31T14:51:10.893959+0000","last_undegraded":"2024-07-31T14:51:10.893959+0000","last_fullsized":"2024-07-31T14:51:10.893959+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:13:35.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":191,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447258+0000","last_change":"2024-07-31T14:47:13.100180+0000","last_active":"2024-07-31T14:47:20.447258+0000","last_peered":"2024-07-31T14:47:20.447258+0000","last_clean":"2024-07-31T14:47:20.447258+0000","last_became_active":"2024-07-31T14:47:13.099725+0000","last_became_peered":"2024-07-31T14:47:13.099725+0000","last_unstale":"2024-07-31T14:47:20.447258+0000","last_undegraded":"2024-07-31T14:47:20.447258+0000","last_fullsized":"2024-07-31T14:47:20.447258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:25:32.245339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404277+0000","last_change":"2024-07-31T14:47:03.040349+0000","last_active":"2024-07-31T14:49:49.404277+0000","last_peered":"2024-07-31T14:49:49.404277+0000","last_clean":"2024-07-31T14:49:49.404277+0000","last_became_active":"2024-07-31T14:47:03.039887+0000","last_became_peered":"2024-07-31T14:47:03.039887+0000","last_unstale":"2024-07-31T14:49:49.404277+0000","last_undegraded":"2024-07-31T14:49:49.404277+0000","last_fullsized":"2024-07-31T14:49:49.404277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:46.592063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":241,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:12.881992+0000","last_change":"2024-07-31T14:47:00.183988+0000","last_active":"2024-07-31T14:51:12.881992+0000","last_peered":"2024-07-31T14:51:12.881992+0000","last_clean":"2024-07-31T14:51:12.881992+0000","last_became_active":"2024-07-31T14:47:00.183302+0000","last_became_peered":"2024-07-31T14:47:00.183302+0000","last_unstale":"2024-07-31T14:51:12.881992+0000","last_undegraded":"2024-07-31T14:51:12.881992+0000","last_fullsized":"2024-07-31T14:51:12.881992+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:40:26.188224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447744+0000","last_change":"2024-07-31T14:47:13.363075+0000","last_active":"2024-07-31T14:47:20.447744+0000","last_peered":"2024-07-31T14:47:20.447744+0000","last_clean":"2024-07-31T14:47:20.447744+0000","last_became_active":"2024-07-31T14:47:13.362599+0000","last_became_peered":"2024-07-31T14:47:13.362599+0000","last_unstale":"2024-07-31T14:47:20.447744+0000","last_undegraded":"2024-07-31T14:47:20.447744+0000","last_fullsized":"2024-07-31T14:47:20.447744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:20:43.617742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.402186+0000","last_change":"2024-07-31T14:47:20.450712+0000","last_active":"2024-07-31T14:49:49.402186+0000","last_peered":"2024-07-31T14:49:49.402186+0000","last_clean":"2024-07-31T14:49:49.402186+0000","last_became_active":"2024-07-31T14:47:20.450229+0000","last_became_peered":"2024-07-31T14:47:20.450229+0000","last_unstale":"2024-07-31T14:49:49.402186+0000","last_undegraded":"2024-07-31T14:49:49.402186+0000","last_fullsized":"2024-07-31T14:49:49.402186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:11:50.631420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:01.604764+0000","last_change":"2024-07-31T14:47:00.021389+0000","last_active":"2024-07-31T14:49:01.604764+0000","last_peered":"2024-07-31T14:49:01.604764+0000","last_clean":"2024-07-31T14:49:01.604764+0000","last_became_active":"2024-07-31T14:47:00.020987+0000","last_became_peered":"2024-07-31T14:47:00.020987+0000","last_unstale":"2024-07-31T14:49:01.604764+0000","last_undegraded":"2024-07-31T14:49:01.604764+0000","last_fullsized":"2024-07-31T14:49:01.604764+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:07:24.658463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627003+0000","last_change":"2024-07-31T14:47:13.363105+0000","last_active":"2024-07-31T14:47:20.627003+0000","last_peered":"2024-07-31T14:47:20.627003+0000","last_clean":"2024-07-31T14:47:20.627003+0000","last_became_active":"2024-07-31T14:47:13.362404+0000","last_became_peered":"2024-07-31T14:47:13.362404+0000","last_unstale":"2024-07-31T14:47:20.627003+0000","last_undegraded":"2024-07-31T14:47:20.627003+0000","last_fullsized":"2024-07-31T14:47:20.627003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:48:43.033584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404516+0000","last_change":"2024-07-31T14:47:03.057199+0000","last_active":"2024-07-31T14:49:49.404516+0000","last_peered":"2024-07-31T14:49:49.404516+0000","last_clean":"2024-07-31T14:49:49.404516+0000","last_became_active":"2024-07-31T14:47:03.056555+0000","last_became_peered":"2024-07-31T14:47:03.056555+0000","last_unstale":"2024-07-31T14:49:49.404516+0000","last_undegraded":"2024-07-31T14:49:49.404516+0000","last_fullsized":"2024-07-31T14:49:49.404516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:21:14.263457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:04.380224+0000","last_change":"2024-07-31T14:47:00.032055+0000","last_active":"2024-07-31T14:49:04.380224+0000","last_peered":"2024-07-31T14:49:04.380224+0000","last_clean":"2024-07-31T14:49:04.380224+0000","last_became_active":"2024-07-31T14:47:00.031519+0000","last_became_peered":"2024-07-31T14:47:00.031519+0000","last_unstale":"2024-07-31T14:49:04.380224+0000","last_undegraded":"2024-07-31T14:49:04.380224+0000","last_fullsized":"2024-07-31T14:49:04.380224+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:21:46.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:31.600736+0000","last_change":"2024-07-31T14:45:42.166503+0000","last_active":"2024-07-31T14:48:31.600736+0000","last_peered":"2024-07-31T14:48:31.600736+0000","last_clean":"2024-07-31T14:48:31.600736+0000","last_became_active":"2024-07-31T14:45:42.160065+0000","last_became_peered":"2024-07-31T14:45:42.160065+0000","last_unstale":"2024-07-31T14:48:31.600736+0000","last_undegraded":"2024-07-31T14:48:31.600736+0000","last_fullsized":"2024-07-31T14:48:31.600736+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_clean_scrub_stamp":"2024-07-31T14:42:27.863113+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:38:33.646022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:12.369486+0000","last_change":"2024-07-31T14:47:13.103812+0000","last_active":"2024-07-31T14:48:12.369486+0000","last_peered":"2024-07-31T14:48:12.369486+0000","last_clean":"2024-07-31T14:48:12.369486+0000","last_became_active":"2024-07-31T14:47:13.102549+0000","last_became_peered":"2024-07-31T14:47:13.102549+0000","last_unstale":"2024-07-31T14:48:12.369486+0000","last_undegraded":"2024-07-31T14:48:12.369486+0000","last_fullsized":"2024-07-31T14:48:12.369486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:18:56.322426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414083+0000","last_change":"2024-07-31T14:47:03.056310+0000","last_active":"2024-07-31T14:49:49.414083+0000","last_peered":"2024-07-31T14:49:49.414083+0000","last_clean":"2024-07-31T14:49:49.414083+0000","last_became_active":"2024-07-31T14:47:03.055695+0000","last_became_peered":"2024-07-31T14:47:03.055695+0000","last_unstale":"2024-07-31T14:49:49.414083+0000","last_undegraded":"2024-07-31T14:49:49.414083+0000","last_fullsized":"2024-07-31T14:49:49.414083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:00:55.124981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:09.891605+0000","last_change":"2024-07-31T14:47:00.236460+0000","last_active":"2024-07-31T14:51:09.891605+0000","last_peered":"2024-07-31T14:51:09.891605+0000","last_clean":"2024-07-31T14:51:09.891605+0000","last_became_active":"2024-07-31T14:47:00.235924+0000","last_became_peered":"2024-07-31T14:47:00.235924+0000","last_unstale":"2024-07-31T14:51:09.891605+0000","last_undegraded":"2024-07-31T14:51:09.891605+0000","last_fullsized":"2024-07-31T14:51:09.891605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:05.130958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626080+0000","last_change":"2024-07-31T14:47:13.102341+0000","last_active":"2024-07-31T14:47:20.626080+0000","last_peered":"2024-07-31T14:47:20.626080+0000","last_clean":"2024-07-31T14:47:20.626080+0000","last_became_active":"2024-07-31T14:47:13.101798+0000","last_became_peered":"2024-07-31T14:47:13.101798+0000","last_unstale":"2024-07-31T14:47:20.626080+0000","last_undegraded":"2024-07-31T14:47:20.626080+0000","last_fullsized":"2024-07-31T14:47:20.626080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:34:35.758041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409700+0000","last_change":"2024-07-31T14:47:03.054322+0000","last_active":"2024-07-31T14:49:49.409700+0000","last_peered":"2024-07-31T14:49:49.409700+0000","last_clean":"2024-07-31T14:49:49.409700+0000","last_became_active":"2024-07-31T14:47:03.053112+0000","last_became_peered":"2024-07-31T14:47:03.053112+0000","last_unstale":"2024-07-31T14:49:49.409700+0000","last_undegraded":"2024-07-31T14:49:49.409700+0000","last_fullsized":"2024-07-31T14:49:49.409700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:55:13.542578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604832+0000","last_change":"2024-07-31T14:47:00.026518+0000","last_active":"2024-07-31T14:48:03.604832+0000","last_peered":"2024-07-31T14:48:03.604832+0000","last_clean":"2024-07-31T14:48:03.604832+0000","last_became_active":"2024-07-31T14:47:00.026074+0000","last_became_peered":"2024-07-31T14:47:00.026074+0000","last_unstale":"2024-07-31T14:48:03.604832+0000","last_undegraded":"2024-07-31T14:48:03.604832+0000","last_fullsized":"2024-07-31T14:48:03.604832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:33:36.688353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449369+0000","last_change":"2024-07-31T14:47:13.106365+0000","last_active":"2024-07-31T14:47:20.449369+0000","last_peered":"2024-07-31T14:47:20.449369+0000","last_clean":"2024-07-31T14:47:20.449369+0000","last_became_active":"2024-07-31T14:47:13.105635+0000","last_became_peered":"2024-07-31T14:47:13.105635+0000","last_unstale":"2024-07-31T14:47:20.449369+0000","last_undegraded":"2024-07-31T14:47:20.449369+0000","last_fullsized":"2024-07-31T14:47:20.449369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:52:09.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404285+0000","last_change":"2024-07-31T14:47:03.052522+0000","last_active":"2024-07-31T14:49:49.404285+0000","last_peered":"2024-07-31T14:49:49.404285+0000","last_clean":"2024-07-31T14:49:49.404285+0000","last_became_active":"2024-07-31T14:47:03.051872+0000","last_became_peered":"2024-07-31T14:47:03.051872+0000","last_unstale":"2024-07-31T14:49:49.404285+0000","last_undegraded":"2024-07-31T14:49:49.404285+0000","last_fullsized":"2024-07-31T14:49:49.404285+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:29:34.956745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600950+0000","last_change":"2024-07-31T14:47:00.016839+0000","last_active":"2024-07-31T14:48:02.600950+0000","last_peered":"2024-07-31T14:48:02.600950+0000","last_clean":"2024-07-31T14:48:02.600950+0000","last_became_active":"2024-07-31T14:47:00.016438+0000","last_became_peered":"2024-07-31T14:47:00.016438+0000","last_unstale":"2024-07-31T14:48:02.600950+0000","last_undegraded":"2024-07-31T14:48:02.600950+0000","last_fullsized":"2024-07-31T14:48:02.600950+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:42:10.875332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626258+0000","last_change":"2024-07-31T14:47:13.105385+0000","last_active":"2024-07-31T14:47:20.626258+0000","last_peered":"2024-07-31T14:47:20.626258+0000","last_clean":"2024-07-31T14:47:20.626258+0000","last_became_active":"2024-07-31T14:47:13.104867+0000","last_became_peered":"2024-07-31T14:47:13.104867+0000","last_unstale":"2024-07-31T14:47:20.626258+0000","last_undegraded":"2024-07-31T14:47:20.626258+0000","last_fullsized":"2024-07-31T14:47:20.626258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:25:21.572830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399072+0000","last_change":"2024-07-31T14:47:03.058304+0000","last_active":"2024-07-31T14:49:49.399072+0000","last_peered":"2024-07-31T14:49:49.399072+0000","last_clean":"2024-07-31T14:49:49.399072+0000","last_became_active":"2024-07-31T14:47:03.057251+0000","last_became_peered":"2024-07-31T14:47:03.057251+0000","last_unstale":"2024-07-31T14:49:49.399072+0000","last_undegraded":"2024-07-31T14:49:49.399072+0000","last_fullsized":"2024-07-31T14:49:49.399072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:58:20.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605301+0000","last_change":"2024-07-31T14:47:00.027806+0000","last_active":"2024-07-31T14:48:03.605301+0000","last_peered":"2024-07-31T14:48:03.605301+0000","last_clean":"2024-07-31T14:48:03.605301+0000","last_became_active":"2024-07-31T14:47:00.027403+0000","last_became_peered":"2024-07-31T14:47:00.027403+0000","last_unstale":"2024-07-31T14:48:03.605301+0000","last_undegraded":"2024-07-31T14:48:03.605301+0000","last_fullsized":"2024-07-31T14:48:03.605301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:57:18.303732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626490+0000","last_change":"2024-07-31T14:47:13.103628+0000","last_active":"2024-07-31T14:47:20.626490+0000","last_peered":"2024-07-31T14:47:20.626490+0000","last_clean":"2024-07-31T14:47:20.626490+0000","last_became_active":"2024-07-31T14:47:13.102296+0000","last_became_peered":"2024-07-31T14:47:13.102296+0000","last_unstale":"2024-07-31T14:47:20.626490+0000","last_undegraded":"2024-07-31T14:47:20.626490+0000","last_fullsized":"2024-07-31T14:47:20.626490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:25:35.109196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413387+0000","last_change":"2024-07-31T14:47:03.052761+0000","last_active":"2024-07-31T14:49:49.413387+0000","last_peered":"2024-07-31T14:49:49.413387+0000","last_clean":"2024-07-31T14:49:49.413387+0000","last_became_active":"2024-07-31T14:47:03.052410+0000","last_became_peered":"2024-07-31T14:47:03.052410+0000","last_unstale":"2024-07-31T14:49:49.413387+0000","last_undegraded":"2024-07-31T14:49:49.413387+0000","last_fullsized":"2024-07-31T14:49:49.413387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:20:47.712491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604937+0000","last_change":"2024-07-31T14:47:00.183741+0000","last_active":"2024-07-31T14:48:03.604937+0000","last_peered":"2024-07-31T14:48:03.604937+0000","last_clean":"2024-07-31T14:48:03.604937+0000","last_became_active":"2024-07-31T14:47:00.183275+0000","last_became_peered":"2024-07-31T14:47:00.183275+0000","last_unstale":"2024-07-31T14:48:03.604937+0000","last_undegraded":"2024-07-31T14:48:03.604937+0000","last_fullsized":"2024-07-31T14:48:03.604937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:29:23.205835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630098+0000","last_change":"2024-07-31T14:47:20.629334+0000","last_active":"2024-07-31T14:47:20.630098+0000","last_peered":"2024-07-31T14:47:20.630098+0000","last_clean":"2024-07-31T14:47:20.630098+0000","last_became_active":"2024-07-31T14:47:20.628865+0000","last_became_peered":"2024-07-31T14:47:20.628865+0000","last_unstale":"2024-07-31T14:47:20.630098+0000","last_undegraded":"2024-07-31T14:47:20.630098+0000","last_fullsized":"2024-07-31T14:47:20.630098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:21:27.199842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407413+0000","last_change":"2024-07-31T14:47:03.054633+0000","last_active":"2024-07-31T14:49:49.407413+0000","last_peered":"2024-07-31T14:49:49.407413+0000","last_clean":"2024-07-31T14:49:49.407413+0000","last_became_active":"2024-07-31T14:47:03.053025+0000","last_became_peered":"2024-07-31T14:47:03.053025+0000","last_unstale":"2024-07-31T14:49:49.407413+0000","last_undegraded":"2024-07-31T14:49:49.407413+0000","last_fullsized":"2024-07-31T14:49:49.407413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:42:03.392744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:08.876300+0000","last_change":"2024-07-31T14:47:00.027898+0000","last_active":"2024-07-31T14:51:08.876300+0000","last_peered":"2024-07-31T14:51:08.876300+0000","last_clean":"2024-07-31T14:51:08.876300+0000","last_became_active":"2024-07-31T14:47:00.027468+0000","last_became_peered":"2024-07-31T14:47:00.027468+0000","last_unstale":"2024-07-31T14:51:08.876300+0000","last_undegraded":"2024-07-31T14:51:08.876300+0000","last_fullsized":"2024-07-31T14:51:08.876300+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:12:45.922172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444574+0000","last_change":"2024-07-31T14:47:13.104327+0000","last_active":"2024-07-31T14:47:20.444574+0000","last_peered":"2024-07-31T14:47:20.444574+0000","last_clean":"2024-07-31T14:47:20.444574+0000","last_became_active":"2024-07-31T14:47:13.103333+0000","last_became_peered":"2024-07-31T14:47:13.103333+0000","last_unstale":"2024-07-31T14:47:20.444574+0000","last_undegraded":"2024-07-31T14:47:20.444574+0000","last_fullsized":"2024-07-31T14:47:20.444574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:06.226220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.410145+0000","last_change":"2024-07-31T14:47:03.044875+0000","last_active":"2024-07-31T14:49:49.410145+0000","last_peered":"2024-07-31T14:49:49.410145+0000","last_clean":"2024-07-31T14:49:49.410145+0000","last_became_active":"2024-07-31T14:47:03.044421+0000","last_became_peered":"2024-07-31T14:47:03.044421+0000","last_unstale":"2024-07-31T14:49:49.410145+0000","last_undegraded":"2024-07-31T14:49:49.410145+0000","last_fullsized":"2024-07-31T14:49:49.410145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:00:02.477300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605606+0000","last_change":"2024-07-31T14:47:00.019790+0000","last_active":"2024-07-31T14:48:03.605606+0000","last_peered":"2024-07-31T14:48:03.605606+0000","last_clean":"2024-07-31T14:48:03.605606+0000","last_became_active":"2024-07-31T14:47:00.019368+0000","last_became_peered":"2024-07-31T14:47:00.019368+0000","last_unstale":"2024-07-31T14:48:03.605606+0000","last_undegraded":"2024-07-31T14:48:03.605606+0000","last_fullsized":"2024-07-31T14:48:03.605606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:15.299109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445672+0000","last_change":"2024-07-31T14:47:13.102618+0000","last_active":"2024-07-31T14:47:20.445672+0000","last_peered":"2024-07-31T14:47:20.445672+0000","last_clean":"2024-07-31T14:47:20.445672+0000","last_became_active":"2024-07-31T14:47:13.102085+0000","last_became_peered":"2024-07-31T14:47:13.102085+0000","last_unstale":"2024-07-31T14:47:20.445672+0000","last_undegraded":"2024-07-31T14:47:20.445672+0000","last_fullsized":"2024-07-31T14:47:20.445672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:05:58.814215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.406277+0000","last_change":"2024-07-31T14:47:03.044688+0000","last_active":"2024-07-31T14:49:49.406277+0000","last_peered":"2024-07-31T14:49:49.406277+0000","last_clean":"2024-07-31T14:49:49.406277+0000","last_became_active":"2024-07-31T14:47:03.043751+0000","last_became_peered":"2024-07-31T14:47:03.043751+0000","last_unstale":"2024-07-31T14:49:49.406277+0000","last_undegraded":"2024-07-31T14:49:49.406277+0000","last_fullsized":"2024-07-31T14:49:49.406277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:47:31.450796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605830+0000","last_change":"2024-07-31T14:47:00.026259+0000","last_active":"2024-07-31T14:48:03.605830+0000","last_peered":"2024-07-31T14:48:03.605830+0000","last_clean":"2024-07-31T14:48:03.605830+0000","last_became_active":"2024-07-31T14:47:00.025756+0000","last_became_peered":"2024-07-31T14:47:00.025756+0000","last_unstale":"2024-07-31T14:48:03.605830+0000","last_undegraded":"2024-07-31T14:48:03.605830+0000","last_fullsized":"2024-07-31T14:48:03.605830+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:05:00.258841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626888+0000","last_change":"2024-07-31T14:47:13.104918+0000","last_active":"2024-07-31T14:47:20.626888+0000","last_peered":"2024-07-31T14:47:20.626888+0000","last_clean":"2024-07-31T14:47:20.626888+0000","last_became_active":"2024-07-31T14:47:13.104031+0000","last_became_peered":"2024-07-31T14:47:13.104031+0000","last_unstale":"2024-07-31T14:47:20.626888+0000","last_undegraded":"2024-07-31T14:47:20.626888+0000","last_fullsized":"2024-07-31T14:47:20.626888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:17:15.264474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.389841+0000","last_change":"2024-07-31T14:47:03.055159+0000","last_active":"2024-07-31T14:49:49.389841+0000","last_peered":"2024-07-31T14:49:49.389841+0000","last_clean":"2024-07-31T14:49:49.389841+0000","last_became_active":"2024-07-31T14:47:03.052389+0000","last_became_peered":"2024-07-31T14:47:03.052389+0000","last_unstale":"2024-07-31T14:49:49.389841+0000","last_undegraded":"2024-07-31T14:49:49.389841+0000","last_fullsized":"2024-07-31T14:49:49.389841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:16.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'41","reported_seq":469,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:09.878153+0000","last_change":"2024-07-31T14:47:00.030165+0000","last_active":"2024-07-31T14:51:09.878153+0000","last_peered":"2024-07-31T14:51:09.878153+0000","last_clean":"2024-07-31T14:51:09.878153+0000","last_became_active":"2024-07-31T14:47:00.029559+0000","last_became_peered":"2024-07-31T14:47:00.029559+0000","last_unstale":"2024-07-31T14:51:09.878153+0000","last_undegraded":"2024-07-31T14:51:09.878153+0000","last_fullsized":"2024-07-31T14:51:09.878153+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:31:06.115075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69398,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":386,"num_read_kb":261,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444234+0000","last_change":"2024-07-31T14:47:13.098517+0000","last_active":"2024-07-31T14:47:20.444234+0000","last_peered":"2024-07-31T14:47:20.444234+0000","last_clean":"2024-07-31T14:47:20.444234+0000","last_became_active":"2024-07-31T14:47:13.098066+0000","last_became_peered":"2024-07-31T14:47:13.098066+0000","last_unstale":"2024-07-31T14:47:20.444234+0000","last_undegraded":"2024-07-31T14:47:20.444234+0000","last_fullsized":"2024-07-31T14:47:20.444234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:08:54.958646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414451+0000","last_change":"2024-07-31T14:47:03.054045+0000","last_active":"2024-07-31T14:49:49.414451+0000","last_peered":"2024-07-31T14:49:49.414451+0000","last_clean":"2024-07-31T14:49:49.414451+0000","last_became_active":"2024-07-31T14:47:03.053658+0000","last_became_peered":"2024-07-31T14:47:03.053658+0000","last_unstale":"2024-07-31T14:49:49.414451+0000","last_undegraded":"2024-07-31T14:49:49.414451+0000","last_fullsized":"2024-07-31T14:49:49.414451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:36:54.563542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.607373+0000","last_change":"2024-07-31T14:47:00.185602+0000","last_active":"2024-07-31T14:48:03.607373+0000","last_peered":"2024-07-31T14:48:03.607373+0000","last_clean":"2024-07-31T14:48:03.607373+0000","last_became_active":"2024-07-31T14:47:00.185230+0000","last_became_peered":"2024-07-31T14:47:00.185230+0000","last_unstale":"2024-07-31T14:48:03.607373+0000","last_undegraded":"2024-07-31T14:48:03.607373+0000","last_fullsized":"2024-07-31T14:48:03.607373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:21:35.979942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604698+0000","last_change":"2024-07-31T14:47:13.099611+0000","last_active":"2024-07-31T14:47:20.604698+0000","last_peered":"2024-07-31T14:47:20.604698+0000","last_clean":"2024-07-31T14:47:20.604698+0000","last_became_active":"2024-07-31T14:47:13.099127+0000","last_became_peered":"2024-07-31T14:47:13.099127+0000","last_unstale":"2024-07-31T14:47:20.604698+0000","last_undegraded":"2024-07-31T14:47:20.604698+0000","last_fullsized":"2024-07-31T14:47:20.604698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:16:57.192980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400126+0000","last_change":"2024-07-31T14:47:03.056818+0000","last_active":"2024-07-31T14:49:49.400126+0000","last_peered":"2024-07-31T14:49:49.400126+0000","last_clean":"2024-07-31T14:49:49.400126+0000","last_became_active":"2024-07-31T14:47:03.055716+0000","last_became_peered":"2024-07-31T14:47:03.055716+0000","last_unstale":"2024-07-31T14:49:49.400126+0000","last_undegraded":"2024-07-31T14:49:49.400126+0000","last_fullsized":"2024-07-31T14:49:49.400126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:26:33.627362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600344+0000","last_change":"2024-07-31T14:47:00.029651+0000","last_active":"2024-07-31T14:48:02.600344+0000","last_peered":"2024-07-31T14:48:02.600344+0000","last_clean":"2024-07-31T14:48:02.600344+0000","last_became_active":"2024-07-31T14:47:00.029242+0000","last_became_peered":"2024-07-31T14:47:00.029242+0000","last_unstale":"2024-07-31T14:48:02.600344+0000","last_undegraded":"2024-07-31T14:48:02.600344+0000","last_fullsized":"2024-07-31T14:48:02.600344+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:23:49.177508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.443852+0000","last_change":"2024-07-31T14:47:13.098862+0000","last_active":"2024-07-31T14:47:20.443852+0000","last_peered":"2024-07-31T14:47:20.443852+0000","last_clean":"2024-07-31T14:47:20.443852+0000","last_became_active":"2024-07-31T14:47:13.098324+0000","last_became_peered":"2024-07-31T14:47:13.098324+0000","last_unstale":"2024-07-31T14:47:20.443852+0000","last_undegraded":"2024-07-31T14:47:20.443852+0000","last_fullsized":"2024-07-31T14:47:20.443852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:50:41.260863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.412242+0000","last_change":"2024-07-31T14:47:03.049042+0000","last_active":"2024-07-31T14:49:49.412242+0000","last_peered":"2024-07-31T14:49:49.412242+0000","last_clean":"2024-07-31T14:49:49.412242+0000","last_became_active":"2024-07-31T14:47:03.048535+0000","last_became_peered":"2024-07-31T14:47:03.048535+0000","last_unstale":"2024-07-31T14:49:49.412242+0000","last_undegraded":"2024-07-31T14:49:49.412242+0000","last_fullsized":"2024-07-31T14:49:49.412242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:15.147454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":422,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:09.877699+0000","last_change":"2024-07-31T14:47:00.028940+0000","last_active":"2024-07-31T14:51:09.877699+0000","last_peered":"2024-07-31T14:51:09.877699+0000","last_clean":"2024-07-31T14:51:09.877699+0000","last_became_active":"2024-07-31T14:47:00.028472+0000","last_became_peered":"2024-07-31T14:47:00.028472+0000","last_unstale":"2024-07-31T14:51:09.877699+0000","last_undegraded":"2024-07-31T14:51:09.877699+0000","last_fullsized":"2024-07-31T14:51:09.877699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:57:15.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":374,"num_read_kb":374,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626394+0000","last_change":"2024-07-31T14:47:13.101856+0000","last_active":"2024-07-31T14:47:20.626394+0000","last_peered":"2024-07-31T14:47:20.626394+0000","last_clean":"2024-07-31T14:47:20.626394+0000","last_became_active":"2024-07-31T14:47:13.101346+0000","last_became_peered":"2024-07-31T14:47:13.101346+0000","last_unstale":"2024-07-31T14:47:20.626394+0000","last_undegraded":"2024-07-31T14:47:20.626394+0000","last_fullsized":"2024-07-31T14:47:20.626394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:56:37.071660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449190+0000","last_change":"2024-07-31T14:47:13.105952+0000","last_active":"2024-07-31T14:47:20.449190+0000","last_peered":"2024-07-31T14:47:20.449190+0000","last_clean":"2024-07-31T14:47:20.449190+0000","last_became_active":"2024-07-31T14:47:13.105506+0000","last_became_peered":"2024-07-31T14:47:13.105506+0000","last_unstale":"2024-07-31T14:47:20.449190+0000","last_undegraded":"2024-07-31T14:47:20.449190+0000","last_fullsized":"2024-07-31T14:47:20.449190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:28:33.866216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400532+0000","last_change":"2024-07-31T14:47:03.051152+0000","last_active":"2024-07-31T14:49:49.400532+0000","last_peered":"2024-07-31T14:49:49.400532+0000","last_clean":"2024-07-31T14:49:49.400532+0000","last_became_active":"2024-07-31T14:47:03.050775+0000","last_became_peered":"2024-07-31T14:47:03.050775+0000","last_unstale":"2024-07-31T14:49:49.400532+0000","last_undegraded":"2024-07-31T14:49:49.400532+0000","last_fullsized":"2024-07-31T14:49:49.400532+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:47:35.913641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609201+0000","last_change":"2024-07-31T14:47:00.028552+0000","last_active":"2024-07-31T14:48:03.609201+0000","last_peered":"2024-07-31T14:48:03.609201+0000","last_clean":"2024-07-31T14:48:03.609201+0000","last_became_active":"2024-07-31T14:47:00.028143+0000","last_became_peered":"2024-07-31T14:47:00.028143+0000","last_unstale":"2024-07-31T14:48:03.609201+0000","last_undegraded":"2024-07-31T14:48:03.609201+0000","last_fullsized":"2024-07-31T14:48:03.609201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:09.601635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.606879+0000","last_change":"2024-07-31T14:47:20.606285+0000","last_active":"2024-07-31T14:47:20.606879+0000","last_peered":"2024-07-31T14:47:20.606879+0000","last_clean":"2024-07-31T14:47:20.606879+0000","last_became_active":"2024-07-31T14:47:20.605837+0000","last_became_peered":"2024-07-31T14:47:20.605837+0000","last_unstale":"2024-07-31T14:47:20.606879+0000","last_undegraded":"2024-07-31T14:47:20.606879+0000","last_fullsized":"2024-07-31T14:47:20.606879+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:16:35.409790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.398709+0000","last_change":"2024-07-31T14:47:03.047442+0000","last_active":"2024-07-31T14:49:49.398709+0000","last_peered":"2024-07-31T14:49:49.398709+0000","last_clean":"2024-07-31T14:49:49.398709+0000","last_became_active":"2024-07-31T14:47:03.047072+0000","last_became_peered":"2024-07-31T14:47:03.047072+0000","last_unstale":"2024-07-31T14:49:49.398709+0000","last_undegraded":"2024-07-31T14:49:49.398709+0000","last_fullsized":"2024-07-31T14:49:49.398709+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:56.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.599595+0000","last_change":"2024-07-31T14:47:00.027746+0000","last_active":"2024-07-31T14:48:02.599595+0000","last_peered":"2024-07-31T14:48:02.599595+0000","last_clean":"2024-07-31T14:48:02.599595+0000","last_became_active":"2024-07-31T14:47:00.027404+0000","last_became_peered":"2024-07-31T14:47:00.027404+0000","last_unstale":"2024-07-31T14:48:02.599595+0000","last_undegraded":"2024-07-31T14:48:02.599595+0000","last_fullsized":"2024-07-31T14:48:02.599595+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:30:21.011674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.589640+0000","last_change":"2024-07-31T14:47:00.029148+0000","last_active":"2024-07-31T14:48:01.589640+0000","last_peered":"2024-07-31T14:48:01.589640+0000","last_clean":"2024-07-31T14:48:01.589640+0000","last_became_active":"2024-07-31T14:47:00.028718+0000","last_became_peered":"2024-07-31T14:47:00.028718+0000","last_unstale":"2024-07-31T14:48:01.589640+0000","last_undegraded":"2024-07-31T14:48:01.589640+0000","last_fullsized":"2024-07-31T14:48:01.589640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:25:16.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400207+0000","last_change":"2024-07-31T14:47:20.629098+0000","last_active":"2024-07-31T14:49:49.400207+0000","last_peered":"2024-07-31T14:49:49.400207+0000","last_clean":"2024-07-31T14:49:49.400207+0000","last_became_active":"2024-07-31T14:47:20.627775+0000","last_became_peered":"2024-07-31T14:47:20.627775+0000","last_unstale":"2024-07-31T14:49:49.400207+0000","last_undegraded":"2024-07-31T14:49:49.400207+0000","last_fullsized":"2024-07-31T14:49:49.400207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:22:03.606509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447331+0000","last_change":"2024-07-31T14:47:13.097892+0000","last_active":"2024-07-31T14:47:20.447331+0000","last_peered":"2024-07-31T14:47:20.447331+0000","last_clean":"2024-07-31T14:47:20.447331+0000","last_became_active":"2024-07-31T14:47:13.097419+0000","last_became_peered":"2024-07-31T14:47:13.097419+0000","last_unstale":"2024-07-31T14:47:20.447331+0000","last_undegraded":"2024-07-31T14:47:20.447331+0000","last_fullsized":"2024-07-31T14:47:20.447331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:07:13.456147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":243,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:11.881508+0000","last_change":"2024-07-31T14:47:00.019792+0000","last_active":"2024-07-31T14:51:11.881508+0000","last_peered":"2024-07-31T14:51:11.881508+0000","last_clean":"2024-07-31T14:51:11.881508+0000","last_became_active":"2024-07-31T14:47:00.019411+0000","last_became_peered":"2024-07-31T14:47:00.019411+0000","last_unstale":"2024-07-31T14:51:11.881508+0000","last_undegraded":"2024-07-31T14:51:11.881508+0000","last_fullsized":"2024-07-31T14:51:11.881508+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:55:02.148823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":194,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.390707+0000","last_change":"2024-07-31T14:47:03.039808+0000","last_active":"2024-07-31T14:49:49.390707+0000","last_peered":"2024-07-31T14:49:49.390707+0000","last_clean":"2024-07-31T14:49:49.390707+0000","last_became_active":"2024-07-31T14:47:03.039382+0000","last_became_peered":"2024-07-31T14:47:03.039382+0000","last_unstale":"2024-07-31T14:49:49.390707+0000","last_undegraded":"2024-07-31T14:49:49.390707+0000","last_fullsized":"2024-07-31T14:49:49.390707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:23:09.751293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.446939+0000","last_change":"2024-07-31T14:47:13.102797+0000","last_active":"2024-07-31T14:47:20.446939+0000","last_peered":"2024-07-31T14:47:20.446939+0000","last_clean":"2024-07-31T14:47:20.446939+0000","last_became_active":"2024-07-31T14:47:13.102320+0000","last_became_peered":"2024-07-31T14:47:13.102320+0000","last_unstale":"2024-07-31T14:47:20.446939+0000","last_undegraded":"2024-07-31T14:47:20.446939+0000","last_fullsized":"2024-07-31T14:47:20.446939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:23.066357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.397081+0000","last_change":"2024-07-31T14:47:03.054875+0000","last_active":"2024-07-31T14:49:49.397081+0000","last_peered":"2024-07-31T14:49:49.397081+0000","last_clean":"2024-07-31T14:49:49.397081+0000","last_became_active":"2024-07-31T14:47:03.054529+0000","last_became_peered":"2024-07-31T14:47:03.054529+0000","last_unstale":"2024-07-31T14:49:49.397081+0000","last_undegraded":"2024-07-31T14:49:49.397081+0000","last_fullsized":"2024-07-31T14:49:49.397081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:32:09.802933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396215+0000","last_change":"2024-07-31T14:47:03.043283+0000","last_active":"2024-07-31T14:49:49.396215+0000","last_peered":"2024-07-31T14:49:49.396215+0000","last_clean":"2024-07-31T14:49:49.396215+0000","last_became_active":"2024-07-31T14:47:03.042851+0000","last_became_peered":"2024-07-31T14:47:03.042851+0000","last_unstale":"2024-07-31T14:49:49.396215+0000","last_undegraded":"2024-07-31T14:49:49.396215+0000","last_fullsized":"2024-07-31T14:49:49.396215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:25:37.408149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409421+0000","last_change":"2024-07-31T14:47:03.047890+0000","last_active":"2024-07-31T14:49:49.409421+0000","last_peered":"2024-07-31T14:49:49.409421+0000","last_clean":"2024-07-31T14:49:49.409421+0000","last_became_active":"2024-07-31T14:47:03.047507+0000","last_became_peered":"2024-07-31T14:47:03.047507+0000","last_unstale":"2024-07-31T14:49:49.409421+0000","last_undegraded":"2024-07-31T14:49:49.409421+0000","last_fullsized":"2024-07-31T14:49:49.409421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:52:45.991323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403016+0000","last_change":"2024-07-31T14:47:03.047903+0000","last_active":"2024-07-31T14:49:49.403016+0000","last_peered":"2024-07-31T14:49:49.403016+0000","last_clean":"2024-07-31T14:49:49.403016+0000","last_became_active":"2024-07-31T14:47:03.047352+0000","last_became_peered":"2024-07-31T14:47:03.047352+0000","last_unstale":"2024-07-31T14:49:49.403016+0000","last_undegraded":"2024-07-31T14:49:49.403016+0000","last_fullsized":"2024-07-31T14:49:49.403016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:06.536023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403170+0000","last_change":"2024-07-31T14:47:03.044199+0000","last_active":"2024-07-31T14:49:49.403170+0000","last_peered":"2024-07-31T14:49:49.403170+0000","last_clean":"2024-07-31T14:49:49.403170+0000","last_became_active":"2024-07-31T14:47:03.043849+0000","last_became_peered":"2024-07-31T14:47:03.043849+0000","last_unstale":"2024-07-31T14:49:49.403170+0000","last_undegraded":"2024-07-31T14:49:49.403170+0000","last_fullsized":"2024-07-31T14:49:49.403170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:50.474895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.391079+0000","last_change":"2024-07-31T14:47:03.060654+0000","last_active":"2024-07-31T14:49:49.391079+0000","last_peered":"2024-07-31T14:49:49.391079+0000","last_clean":"2024-07-31T14:49:49.391079+0000","last_became_active":"2024-07-31T14:47:03.060157+0000","last_became_peered":"2024-07-31T14:47:03.060157+0000","last_unstale":"2024-07-31T14:49:49.391079+0000","last_undegraded":"2024-07-31T14:49:49.391079+0000","last_fullsized":"2024-07-31T14:49:49.391079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:39:50.130831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414163+0000","last_change":"2024-07-31T14:47:03.049910+0000","last_active":"2024-07-31T14:49:49.414163+0000","last_peered":"2024-07-31T14:49:49.414163+0000","last_clean":"2024-07-31T14:49:49.414163+0000","last_became_active":"2024-07-31T14:47:03.049491+0000","last_became_peered":"2024-07-31T14:47:03.049491+0000","last_unstale":"2024-07-31T14:49:49.414163+0000","last_undegraded":"2024-07-31T14:49:49.414163+0000","last_fullsized":"2024-07-31T14:49:49.414163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:18:43.319695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407340+0000","last_change":"2024-07-31T14:47:03.052206+0000","last_active":"2024-07-31T14:49:49.407340+0000","last_peered":"2024-07-31T14:49:49.407340+0000","last_clean":"2024-07-31T14:49:49.407340+0000","last_became_active":"2024-07-31T14:47:03.051175+0000","last_became_peered":"2024-07-31T14:47:03.051175+0000","last_unstale":"2024-07-31T14:49:49.407340+0000","last_undegraded":"2024-07-31T14:49:49.407340+0000","last_fullsized":"2024-07-31T14:49:49.407340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:51:11.583604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.405579+0000","last_change":"2024-07-31T14:47:03.052162+0000","last_active":"2024-07-31T14:49:49.405579+0000","last_peered":"2024-07-31T14:49:49.405579+0000","last_clean":"2024-07-31T14:49:49.405579+0000","last_became_active":"2024-07-31T14:47:03.051634+0000","last_became_peered":"2024-07-31T14:47:03.051634+0000","last_unstale":"2024-07-31T14:49:49.405579+0000","last_undegraded":"2024-07-31T14:49:49.405579+0000","last_fullsized":"2024-07-31T14:49:49.405579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:41:59.605192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409010+0000","last_change":"2024-07-31T14:47:03.047475+0000","last_active":"2024-07-31T14:49:49.409010+0000","last_peered":"2024-07-31T14:49:49.409010+0000","last_clean":"2024-07-31T14:49:49.409010+0000","last_became_active":"2024-07-31T14:47:03.047100+0000","last_became_peered":"2024-07-31T14:47:03.047100+0000","last_unstale":"2024-07-31T14:49:49.409010+0000","last_undegraded":"2024-07-31T14:49:49.409010+0000","last_fullsized":"2024-07-31T14:49:49.409010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:08:10.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411379+0000","last_change":"2024-07-31T14:47:03.051281+0000","last_active":"2024-07-31T14:49:49.411379+0000","last_peered":"2024-07-31T14:49:49.411379+0000","last_clean":"2024-07-31T14:49:49.411379+0000","last_became_active":"2024-07-31T14:47:03.050921+0000","last_became_peered":"2024-07-31T14:47:03.050921+0000","last_unstale":"2024-07-31T14:49:49.411379+0000","last_undegraded":"2024-07-31T14:49:49.411379+0000","last_fullsized":"2024-07-31T14:49:49.411379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:24:29.221723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403114+0000","last_change":"2024-07-31T14:47:03.045457+0000","last_active":"2024-07-31T14:49:49.403114+0000","last_peered":"2024-07-31T14:49:49.403114+0000","last_clean":"2024-07-31T14:49:49.403114+0000","last_became_active":"2024-07-31T14:47:03.044466+0000","last_became_peered":"2024-07-31T14:47:03.044466+0000","last_unstale":"2024-07-31T14:49:49.403114+0000","last_undegraded":"2024-07-31T14:49:49.403114+0000","last_fullsized":"2024-07-31T14:49:49.403114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:44:01.197416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403872+0000","last_change":"2024-07-31T14:47:03.053431+0000","last_active":"2024-07-31T14:49:49.403872+0000","last_peered":"2024-07-31T14:49:49.403872+0000","last_clean":"2024-07-31T14:49:49.403872+0000","last_became_active":"2024-07-31T14:47:03.052958+0000","last_became_peered":"2024-07-31T14:47:03.052958+0000","last_unstale":"2024-07-31T14:49:49.403872+0000","last_undegraded":"2024-07-31T14:49:49.403872+0000","last_fullsized":"2024-07-31T14:49:49.403872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:03:28.863589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409420+0000","last_change":"2024-07-31T14:47:03.046641+0000","last_active":"2024-07-31T14:49:49.409420+0000","last_peered":"2024-07-31T14:49:49.409420+0000","last_clean":"2024-07-31T14:49:49.409420+0000","last_became_active":"2024-07-31T14:47:03.046265+0000","last_became_peered":"2024-07-31T14:47:03.046265+0000","last_unstale":"2024-07-31T14:49:49.409420+0000","last_undegraded":"2024-07-31T14:49:49.409420+0000","last_fullsized":"2024-07-31T14:49:49.409420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:20:25.389010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.387120+0000","last_change":"2024-07-31T14:47:03.051657+0000","last_active":"2024-07-31T14:49:49.387120+0000","last_peered":"2024-07-31T14:49:49.387120+0000","last_clean":"2024-07-31T14:49:49.387120+0000","last_became_active":"2024-07-31T14:47:03.050722+0000","last_became_peered":"2024-07-31T14:47:03.050722+0000","last_unstale":"2024-07-31T14:49:49.387120+0000","last_undegraded":"2024-07-31T14:49:49.387120+0000","last_fullsized":"2024-07-31T14:49:49.387120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:23:19.387470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411567+0000","last_change":"2024-07-31T14:47:03.059654+0000","last_active":"2024-07-31T14:49:49.411567+0000","last_peered":"2024-07-31T14:49:49.411567+0000","last_clean":"2024-07-31T14:49:49.411567+0000","last_became_active":"2024-07-31T14:47:03.056843+0000","last_became_peered":"2024-07-31T14:47:03.056843+0000","last_unstale":"2024-07-31T14:49:49.411567+0000","last_undegraded":"2024-07-31T14:49:49.411567+0000","last_fullsized":"2024-07-31T14:49:49.411567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:42:33.696498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413200+0000","last_change":"2024-07-31T14:47:03.052750+0000","last_active":"2024-07-31T14:49:49.413200+0000","last_peered":"2024-07-31T14:49:49.413200+0000","last_clean":"2024-07-31T14:49:49.413200+0000","last_became_active":"2024-07-31T14:47:03.051744+0000","last_became_peered":"2024-07-31T14:47:03.051744+0000","last_unstale":"2024-07-31T14:49:49.413200+0000","last_undegraded":"2024-07-31T14:49:49.413200+0000","last_fullsized":"2024-07-31T14:49:49.413200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:24:33.202918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.393222+0000","last_change":"2024-07-31T14:47:20.452563+0000","last_active":"2024-07-31T14:49:49.393222+0000","last_peered":"2024-07-31T14:49:49.393222+0000","last_clean":"2024-07-31T14:49:49.393222+0000","last_became_active":"2024-07-31T14:47:20.452271+0000","last_became_peered":"2024-07-31T14:47:20.452271+0000","last_unstale":"2024-07-31T14:49:49.393222+0000","last_undegraded":"2024-07-31T14:49:49.393222+0000","last_fullsized":"2024-07-31T14:49:49.393222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:26:55.070265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.391684+0000","last_change":"2024-07-31T14:47:03.050691+0000","last_active":"2024-07-31T14:49:49.391684+0000","last_peered":"2024-07-31T14:49:49.391684+0000","last_clean":"2024-07-31T14:49:49.391684+0000","last_became_active":"2024-07-31T14:47:03.050320+0000","last_became_peered":"2024-07-31T14:47:03.050320+0000","last_unstale":"2024-07-31T14:49:49.391684+0000","last_undegraded":"2024-07-31T14:49:49.391684+0000","last_fullsized":"2024-07-31T14:49:49.391684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:40:45.767706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400128+0000","last_change":"2024-07-31T14:47:03.048491+0000","last_active":"2024-07-31T14:49:49.400128+0000","last_peered":"2024-07-31T14:49:49.400128+0000","last_clean":"2024-07-31T14:49:49.400128+0000","last_became_active":"2024-07-31T14:47:03.048085+0000","last_became_peered":"2024-07-31T14:47:03.048085+0000","last_unstale":"2024-07-31T14:49:49.400128+0000","last_undegraded":"2024-07-31T14:49:49.400128+0000","last_fullsized":"2024-07-31T14:49:49.400128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:35:01.762365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.395425+0000","last_change":"2024-07-31T14:47:03.055502+0000","last_active":"2024-07-31T14:49:49.395425+0000","last_peered":"2024-07-31T14:49:49.395425+0000","last_clean":"2024-07-31T14:49:49.395425+0000","last_became_active":"2024-07-31T14:47:03.055015+0000","last_became_peered":"2024-07-31T14:47:03.055015+0000","last_unstale":"2024-07-31T14:49:49.395425+0000","last_undegraded":"2024-07-31T14:49:49.395425+0000","last_fullsized":"2024-07-31T14:49:49.395425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:50:39.605148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413933+0000","last_change":"2024-07-31T14:47:03.057808+0000","last_active":"2024-07-31T14:49:49.413933+0000","last_peered":"2024-07-31T14:49:49.413933+0000","last_clean":"2024-07-31T14:49:49.413933+0000","last_became_active":"2024-07-31T14:47:03.057443+0000","last_became_peered":"2024-07-31T14:47:03.057443+0000","last_unstale":"2024-07-31T14:49:49.413933+0000","last_undegraded":"2024-07-31T14:49:49.413933+0000","last_fullsized":"2024-07-31T14:49:49.413933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:34:42.915428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396116+0000","last_change":"2024-07-31T14:47:03.058182+0000","last_active":"2024-07-31T14:49:49.396116+0000","last_peered":"2024-07-31T14:49:49.396116+0000","last_clean":"2024-07-31T14:49:49.396116+0000","last_became_active":"2024-07-31T14:47:03.057802+0000","last_became_peered":"2024-07-31T14:47:03.057802+0000","last_unstale":"2024-07-31T14:49:49.396116+0000","last_undegraded":"2024-07-31T14:49:49.396116+0000","last_fullsized":"2024-07-31T14:49:49.396116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:08:06.608788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413379+0000","last_change":"2024-07-31T14:47:03.058882+0000","last_active":"2024-07-31T14:49:49.413379+0000","last_peered":"2024-07-31T14:49:49.413379+0000","last_clean":"2024-07-31T14:49:49.413379+0000","last_became_active":"2024-07-31T14:47:03.058530+0000","last_became_peered":"2024-07-31T14:47:03.058530+0000","last_unstale":"2024-07-31T14:49:49.413379+0000","last_undegraded":"2024-07-31T14:49:49.413379+0000","last_fullsized":"2024-07-31T14:49:49.413379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:28:03.631962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404760+0000","last_change":"2024-07-31T14:47:03.052028+0000","last_active":"2024-07-31T14:49:49.404760+0000","last_peered":"2024-07-31T14:49:49.404760+0000","last_clean":"2024-07-31T14:49:49.404760+0000","last_became_active":"2024-07-31T14:47:03.051559+0000","last_became_peered":"2024-07-31T14:47:03.051559+0000","last_unstale":"2024-07-31T14:49:49.404760+0000","last_undegraded":"2024-07-31T14:49:49.404760+0000","last_fullsized":"2024-07-31T14:49:49.404760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:16:40.723795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399075+0000","last_change":"2024-07-31T14:47:03.053884+0000","last_active":"2024-07-31T14:49:49.399075+0000","last_peered":"2024-07-31T14:49:49.399075+0000","last_clean":"2024-07-31T14:49:49.399075+0000","last_became_active":"2024-07-31T14:47:03.053521+0000","last_became_peered":"2024-07-31T14:47:03.053521+0000","last_unstale":"2024-07-31T14:49:49.399075+0000","last_undegraded":"2024-07-31T14:49:49.399075+0000","last_fullsized":"2024-07-31T14:49:49.399075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:16:37.702055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.394011+0000","last_change":"2024-07-31T14:47:03.058588+0000","last_active":"2024-07-31T14:49:49.394011+0000","last_peered":"2024-07-31T14:49:49.394011+0000","last_clean":"2024-07-31T14:49:49.394011+0000","last_became_active":"2024-07-31T14:47:03.057820+0000","last_became_peered":"2024-07-31T14:47:03.057820+0000","last_unstale":"2024-07-31T14:49:49.394011+0000","last_undegraded":"2024-07-31T14:49:49.394011+0000","last_fullsized":"2024-07-31T14:49:49.394011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:21.265208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396703+0000","last_change":"2024-07-31T14:47:03.041447+0000","last_active":"2024-07-31T14:49:49.396703+0000","last_peered":"2024-07-31T14:49:49.396703+0000","last_clean":"2024-07-31T14:49:49.396703+0000","last_became_active":"2024-07-31T14:47:03.040935+0000","last_became_peered":"2024-07-31T14:47:03.040935+0000","last_unstale":"2024-07-31T14:49:49.396703+0000","last_undegraded":"2024-07-31T14:49:49.396703+0000","last_fullsized":"2024-07-31T14:49:49.396703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:00:13.624698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396755+0000","last_change":"2024-07-31T14:47:20.452239+0000","last_active":"2024-07-31T14:49:49.396755+0000","last_peered":"2024-07-31T14:49:49.396755+0000","last_clean":"2024-07-31T14:49:49.396755+0000","last_became_active":"2024-07-31T14:47:20.451924+0000","last_became_peered":"2024-07-31T14:47:20.451924+0000","last_unstale":"2024-07-31T14:49:49.396755+0000","last_undegraded":"2024-07-31T14:49:49.396755+0000","last_fullsized":"2024-07-31T14:49:49.396755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:58:24.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403996+0000","last_change":"2024-07-31T14:47:03.056024+0000","last_active":"2024-07-31T14:49:49.403996+0000","last_peered":"2024-07-31T14:49:49.403996+0000","last_clean":"2024-07-31T14:49:49.403996+0000","last_became_active":"2024-07-31T14:47:03.055630+0000","last_became_peered":"2024-07-31T14:47:03.055630+0000","last_unstale":"2024-07-31T14:49:49.403996+0000","last_undegraded":"2024-07-31T14:49:49.403996+0000","last_fullsized":"2024-07-31T14:49:49.403996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:09:11.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.390933+0000","last_change":"2024-07-31T14:47:03.053718+0000","last_active":"2024-07-31T14:49:49.390933+0000","last_peered":"2024-07-31T14:49:49.390933+0000","last_clean":"2024-07-31T14:49:49.390933+0000","last_became_active":"2024-07-31T14:47:03.053361+0000","last_became_peered":"2024-07-31T14:47:03.053361+0000","last_unstale":"2024-07-31T14:49:49.390933+0000","last_undegraded":"2024-07-31T14:49:49.390933+0000","last_fullsized":"2024-07-31T14:49:49.390933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:03:49.220838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411510+0000","last_change":"2024-07-31T14:47:03.056087+0000","last_active":"2024-07-31T14:49:49.411510+0000","last_peered":"2024-07-31T14:49:49.411510+0000","last_clean":"2024-07-31T14:49:49.411510+0000","last_became_active":"2024-07-31T14:47:03.055732+0000","last_became_peered":"2024-07-31T14:47:03.055732+0000","last_unstale":"2024-07-31T14:49:49.411510+0000","last_undegraded":"2024-07-31T14:49:49.411510+0000","last_fullsized":"2024-07-31T14:49:49.411510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:45:01.185666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78506,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1928,"num_read_kb":1051,"num_write":276,"num_write_kb":169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":238038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645376,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":520038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841605,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37020,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708132,"statfs":{"total":95995035648,"available":95957127168,"internally_reserved":0,"allocated":1724416,"data_stored":1176867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005131,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":523756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201360,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":520038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397588,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1118131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561114,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":515696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757343,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1112258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920867,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":519770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117095,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":514872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378733,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":589563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":581419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705782,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":517319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":72057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-31T14:51:14.429 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-07-31T14:51:14.429 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-31T14:51:14.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.578+0000 7f5478a06640 1 -- 172.21.15.67:0/3109031235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54741034f0 msgr2=0x7f5474103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:14.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.578+0000 7f5478a06640 1 --2- 172.21.15.67:0/3109031235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54741034f0 0x7f5474103900 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5464009900 tx=0x7f546402a410 comp rx=0 tx=0).stop 2024-07-31T14:51:14.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.578+0000 7f5478a06640 1 -- 172.21.15.67:0/3109031235 shutdown_connections 2024-07-31T14:51:14.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.578+0000 7f5478a06640 1 --2- 172.21.15.67:0/3109031235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5474109770 0x7f547410ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.578+0000 7f5478a06640 1 --2- 172.21.15.67:0/3109031235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5474108e20 0x7f5474109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.578+0000 7f5478a06640 1 --2- 172.21.15.67:0/3109031235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54741034f0 0x7f5474103900 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.578+0000 7f5478a06640 1 -- 172.21.15.67:0/3109031235 >> 172.21.15.67:0/3109031235 conn(0x7f54740fef70 msgr2=0x7f54741013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:14.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.578+0000 7f5478a06640 1 -- 172.21.15.67:0/3109031235 shutdown_connections 2024-07-31T14:51:14.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.578+0000 7f5478a06640 1 -- 172.21.15.67:0/3109031235 wait complete. 2024-07-31T14:51:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f5478a06640 1 Processor -- start 2024-07-31T14:51:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f5478a06640 1 -- start start 2024-07-31T14:51:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f5478a06640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54741034f0 0x7f5474199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f5478a06640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5474108e20 0x7f547419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f5478a06640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5474109770 0x7f547419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f5478a06640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5474113420 con 0x7f54741034f0 2024-07-31T14:51:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f5478a06640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f54741132a0 con 0x7f5474108e20 2024-07-31T14:51:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f5478a06640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54741135a0 con 0x7f5474109770 2024-07-31T14:51:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f547259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54741034f0 0x7f5474199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f547259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54741034f0 0x7f5474199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52748/0 (socket says 172.21.15.67:52748) 2024-07-31T14:51:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f547259c640 1 -- 172.21.15.67:0/3656127930 learned_addr learned my addr 172.21.15.67:0/3656127930 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f5472d9d640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5474109770 0x7f547419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f547259c640 1 -- 172.21.15.67:0/3656127930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5474109770 msgr2=0x7f547419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f5471d9b640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5474108e20 0x7f547419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f547259c640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5474109770 0x7f547419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f547259c640 1 -- 172.21.15.67:0/3656127930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5474108e20 msgr2=0x7f547419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f547259c640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5474108e20 0x7f547419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f547259c640 1 -- 172.21.15.67:0/3656127930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547419fd20 con 0x7f54741034f0 2024-07-31T14:51:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.579+0000 7f5471d9b640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5474108e20 0x7f547419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:51:14.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.580+0000 7f547259c640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54741034f0 0x7f5474199e80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5464006010 tx=0x7f54640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:14.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.580+0000 7f54637fe640 1 -- 172.21.15.67:0/3656127930 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546403f030 con 0x7f54741034f0 2024-07-31T14:51:14.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.580+0000 7f54637fe640 1 -- 172.21.15.67:0/3656127930 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5464003800 con 0x7f54741034f0 2024-07-31T14:51:14.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.580+0000 7f5478a06640 1 -- 172.21.15.67:0/3656127930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547419ffb0 con 0x7f54741034f0 2024-07-31T14:51:14.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.580+0000 7f54637fe640 1 -- 172.21.15.67:0/3656127930 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464035070 con 0x7f54741034f0 2024-07-31T14:51:14.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.580+0000 7f5478a06640 1 -- 172.21.15.67:0/3656127930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54741a04a0 con 0x7f54741034f0 2024-07-31T14:51:14.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.581+0000 7f5478a06640 1 -- 172.21.15.67:0/3656127930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5474103900 con 0x7f54741034f0 2024-07-31T14:51:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.582+0000 7f54637fe640 1 -- 172.21.15.67:0/3656127930 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f5464003cb0 con 0x7f54741034f0 2024-07-31T14:51:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.583+0000 7f54637fe640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f545c0b3cb0 0x7f545c0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.583+0000 7f5471d9b640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f545c0b3cb0 0x7f545c0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.583+0000 7f54637fe640 1 -- 172.21.15.67:0/3656127930 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f5464104eb0 con 0x7f54741034f0 2024-07-31T14:51:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.584+0000 7f5471d9b640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f545c0b3cb0 0x7f545c0b61c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5474069fe0 tx=0x7f5468005d30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:14.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.586+0000 7f54637fe640 1 -- 172.21.15.67:0/3656127930 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f54640cbf70 con 0x7f54741034f0 2024-07-31T14:51:14.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.776+0000 7f5478a06640 1 -- 172.21.15.67:0/3656127930 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f547410b550 con 0x7f545c0b3cb0 2024-07-31T14:51:14.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.781+0000 7f54637fe640 1 -- 172.21.15.67:0/3656127930 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340533 (secure 0 0 0) 0x7f547410b550 con 0x7f545c0b3cb0 2024-07-31T14:51:14.786 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:51:14.788 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-31T14:51:14.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.785+0000 7f5478a06640 1 -- 172.21.15.67:0/3656127930 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f545c0b3cb0 msgr2=0x7f545c0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:14.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.785+0000 7f5478a06640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f545c0b3cb0 0x7f545c0b61c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5474069fe0 tx=0x7f5468005d30 comp rx=0 tx=0).stop 2024-07-31T14:51:14.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.785+0000 7f5478a06640 1 -- 172.21.15.67:0/3656127930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54741034f0 msgr2=0x7f5474199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:14.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.785+0000 7f5478a06640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54741034f0 0x7f5474199e80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5464006010 tx=0x7f54640042f0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.785+0000 7f5478a06640 1 -- 172.21.15.67:0/3656127930 shutdown_connections 2024-07-31T14:51:14.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.785+0000 7f5478a06640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f545c0b3cb0 0x7f545c0b61c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.785+0000 7f5478a06640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5474109770 0x7f547419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.785+0000 7f5478a06640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5474108e20 0x7f547419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.785+0000 7f5478a06640 1 --2- 172.21.15.67:0/3656127930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54741034f0 0x7f5474199e80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:14.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.785+0000 7f5478a06640 1 -- 172.21.15.67:0/3656127930 >> 172.21.15.67:0/3656127930 conn(0x7f54740fef70 msgr2=0x7f5474101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:14.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.785+0000 7f5478a06640 1 -- 172.21.15.67:0/3656127930 shutdown_connections 2024-07-31T14:51:14.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:14.785+0000 7f5478a06640 1 -- 172.21.15.67:0/3656127930 wait complete. 2024-07-31T14:51:14.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:14 smithi067 ceph-mon[32750]: pgmap v404: 129 pgs: 129 active+clean; 653 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-07-31T14:51:14.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:14 smithi067 ceph-mon[32750]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:14.798 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-07-31T14:51:13.443366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669108,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2009,"num_read_kb":1120,"num_write":401,"num_write_kb":2321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":830,"ondisk_log_size":830,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384108,"kb_used_data":14156,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557716,"statfs":{"total":1151940427776,"available":1151547101184,"internally_reserved":0,"allocated":14495744,"data_stored":8209727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":378713403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6103,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":68,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003318"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":245,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:10.893183+0000","last_change":"2024-07-31T14:47:00.020067+0000","last_active":"2024-07-31T14:51:10.893183+0000","last_peered":"2024-07-31T14:51:10.893183+0000","last_clean":"2024-07-31T14:51:10.893183+0000","last_became_active":"2024-07-31T14:47:00.019674+0000","last_became_peered":"2024-07-31T14:47:00.019674+0000","last_unstale":"2024-07-31T14:51:10.893183+0000","last_undegraded":"2024-07-31T14:51:10.893183+0000","last_fullsized":"2024-07-31T14:51:10.893183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:10:08.193786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":189,"num_read_kb":189,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.408739+0000","last_change":"2024-07-31T14:47:03.052069+0000","last_active":"2024-07-31T14:49:49.408739+0000","last_peered":"2024-07-31T14:49:49.408739+0000","last_clean":"2024-07-31T14:49:49.408739+0000","last_became_active":"2024-07-31T14:47:03.051739+0000","last_became_peered":"2024-07-31T14:47:03.051739+0000","last_unstale":"2024-07-31T14:49:49.408739+0000","last_undegraded":"2024-07-31T14:49:49.408739+0000","last_fullsized":"2024-07-31T14:49:49.408739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:29:19.952794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.629384+0000","last_change":"2024-07-31T14:47:20.628770+0000","last_active":"2024-07-31T14:47:20.629384+0000","last_peered":"2024-07-31T14:47:20.629384+0000","last_clean":"2024-07-31T14:47:20.629384+0000","last_became_active":"2024-07-31T14:47:20.628142+0000","last_became_peered":"2024-07-31T14:47:20.628142+0000","last_unstale":"2024-07-31T14:47:20.629384+0000","last_undegraded":"2024-07-31T14:47:20.629384+0000","last_fullsized":"2024-07-31T14:47:20.629384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.659670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:59.291402+0000","last_change":"2024-07-31T14:47:00.183893+0000","last_active":"2024-07-31T14:48:59.291402+0000","last_peered":"2024-07-31T14:48:59.291402+0000","last_clean":"2024-07-31T14:48:59.291402+0000","last_became_active":"2024-07-31T14:47:00.183453+0000","last_became_peered":"2024-07-31T14:47:00.183453+0000","last_unstale":"2024-07-31T14:48:59.291402+0000","last_undegraded":"2024-07-31T14:48:59.291402+0000","last_fullsized":"2024-07-31T14:48:59.291402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:45:08.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404252+0000","last_change":"2024-07-31T14:47:03.057081+0000","last_active":"2024-07-31T14:49:49.404252+0000","last_peered":"2024-07-31T14:49:49.404252+0000","last_clean":"2024-07-31T14:49:49.404252+0000","last_became_active":"2024-07-31T14:47:03.056061+0000","last_became_peered":"2024-07-31T14:47:03.056061+0000","last_unstale":"2024-07-31T14:49:49.404252+0000","last_undegraded":"2024-07-31T14:49:49.404252+0000","last_fullsized":"2024-07-31T14:49:49.404252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:28:16.760131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604121+0000","last_change":"2024-07-31T14:47:13.105271+0000","last_active":"2024-07-31T14:47:20.604121+0000","last_peered":"2024-07-31T14:47:20.604121+0000","last_clean":"2024-07-31T14:47:20.604121+0000","last_became_active":"2024-07-31T14:47:13.104717+0000","last_became_peered":"2024-07-31T14:47:13.104717+0000","last_unstale":"2024-07-31T14:47:20.604121+0000","last_undegraded":"2024-07-31T14:47:20.604121+0000","last_fullsized":"2024-07-31T14:47:20.604121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:34:21.413957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:04.594438+0000","last_change":"2024-07-31T14:47:20.454745+0000","last_active":"2024-07-31T14:48:04.594438+0000","last_peered":"2024-07-31T14:48:04.594438+0000","last_clean":"2024-07-31T14:48:04.594438+0000","last_became_active":"2024-07-31T14:47:20.449892+0000","last_became_peered":"2024-07-31T14:47:20.449892+0000","last_unstale":"2024-07-31T14:48:04.594438+0000","last_undegraded":"2024-07-31T14:48:04.594438+0000","last_fullsized":"2024-07-31T14:48:04.594438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:59:17.375344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.412662+0000","last_change":"2024-07-31T14:47:03.051085+0000","last_active":"2024-07-31T14:49:49.412662+0000","last_peered":"2024-07-31T14:49:49.412662+0000","last_clean":"2024-07-31T14:49:49.412662+0000","last_became_active":"2024-07-31T14:47:03.050686+0000","last_became_peered":"2024-07-31T14:47:03.050686+0000","last_unstale":"2024-07-31T14:49:49.412662+0000","last_undegraded":"2024-07-31T14:49:49.412662+0000","last_fullsized":"2024-07-31T14:49:49.412662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:48:02.494061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604501+0000","last_change":"2024-07-31T14:47:13.104229+0000","last_active":"2024-07-31T14:47:20.604501+0000","last_peered":"2024-07-31T14:47:20.604501+0000","last_clean":"2024-07-31T14:47:20.604501+0000","last_became_active":"2024-07-31T14:47:13.103737+0000","last_became_peered":"2024-07-31T14:47:13.103737+0000","last_unstale":"2024-07-31T14:47:20.604501+0000","last_undegraded":"2024-07-31T14:47:20.604501+0000","last_fullsized":"2024-07-31T14:47:20.604501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:36:00.541345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609858+0000","last_change":"2024-07-31T14:47:00.028169+0000","last_active":"2024-07-31T14:48:03.609858+0000","last_peered":"2024-07-31T14:48:03.609858+0000","last_clean":"2024-07-31T14:48:03.609858+0000","last_became_active":"2024-07-31T14:47:00.027757+0000","last_became_peered":"2024-07-31T14:47:00.027757+0000","last_unstale":"2024-07-31T14:48:03.609858+0000","last_undegraded":"2024-07-31T14:48:03.609858+0000","last_fullsized":"2024-07-31T14:48:03.609858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:36:04.419551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413724+0000","last_change":"2024-07-31T14:47:03.057400+0000","last_active":"2024-07-31T14:49:49.413724+0000","last_peered":"2024-07-31T14:49:49.413724+0000","last_clean":"2024-07-31T14:49:49.413724+0000","last_became_active":"2024-07-31T14:47:03.057035+0000","last_became_peered":"2024-07-31T14:47:03.057035+0000","last_unstale":"2024-07-31T14:49:49.413724+0000","last_undegraded":"2024-07-31T14:49:49.413724+0000","last_fullsized":"2024-07-31T14:49:49.413724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:55:07.951076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604244+0000","last_change":"2024-07-31T14:47:13.106093+0000","last_active":"2024-07-31T14:47:20.604244+0000","last_peered":"2024-07-31T14:47:20.604244+0000","last_clean":"2024-07-31T14:47:20.604244+0000","last_became_active":"2024-07-31T14:47:13.105459+0000","last_became_peered":"2024-07-31T14:47:13.105459+0000","last_unstale":"2024-07-31T14:47:20.604244+0000","last_undegraded":"2024-07-31T14:47:20.604244+0000","last_fullsized":"2024-07-31T14:47:20.604244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:07:52.751245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414377+0000","last_change":"2024-07-31T14:47:03.058701+0000","last_active":"2024-07-31T14:49:49.414377+0000","last_peered":"2024-07-31T14:49:49.414377+0000","last_clean":"2024-07-31T14:49:49.414377+0000","last_became_active":"2024-07-31T14:47:03.058359+0000","last_became_peered":"2024-07-31T14:47:03.058359+0000","last_unstale":"2024-07-31T14:49:49.414377+0000","last_undegraded":"2024-07-31T14:49:49.414377+0000","last_fullsized":"2024-07-31T14:49:49.414377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:37:36.149884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605126+0000","last_change":"2024-07-31T14:47:00.022983+0000","last_active":"2024-07-31T14:48:03.605126+0000","last_peered":"2024-07-31T14:48:03.605126+0000","last_clean":"2024-07-31T14:48:03.605126+0000","last_became_active":"2024-07-31T14:47:00.022586+0000","last_became_peered":"2024-07-31T14:47:00.022586+0000","last_unstale":"2024-07-31T14:48:03.605126+0000","last_undegraded":"2024-07-31T14:48:03.605126+0000","last_fullsized":"2024-07-31T14:48:03.605126+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:29:42.641490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604474+0000","last_change":"2024-07-31T14:47:13.363148+0000","last_active":"2024-07-31T14:47:20.604474+0000","last_peered":"2024-07-31T14:47:20.604474+0000","last_clean":"2024-07-31T14:47:20.604474+0000","last_became_active":"2024-07-31T14:47:13.362577+0000","last_became_peered":"2024-07-31T14:47:13.362577+0000","last_unstale":"2024-07-31T14:47:20.604474+0000","last_undegraded":"2024-07-31T14:47:20.604474+0000","last_fullsized":"2024-07-31T14:47:20.604474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:37:59.003260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409563+0000","last_change":"2024-07-31T14:47:03.049586+0000","last_active":"2024-07-31T14:49:49.409563+0000","last_peered":"2024-07-31T14:49:49.409563+0000","last_clean":"2024-07-31T14:49:49.409563+0000","last_became_active":"2024-07-31T14:47:03.049154+0000","last_became_peered":"2024-07-31T14:47:03.049154+0000","last_unstale":"2024-07-31T14:49:49.409563+0000","last_undegraded":"2024-07-31T14:49:49.409563+0000","last_fullsized":"2024-07-31T14:49:49.409563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:51:01.971854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'5","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:07.609943+0000","last_change":"2024-07-31T14:47:00.021254+0000","last_active":"2024-07-31T14:51:07.609943+0000","last_peered":"2024-07-31T14:51:07.609943+0000","last_clean":"2024-07-31T14:51:07.609943+0000","last_became_active":"2024-07-31T14:47:00.020690+0000","last_became_peered":"2024-07-31T14:47:00.020690+0000","last_unstale":"2024-07-31T14:51:07.609943+0000","last_undegraded":"2024-07-31T14:51:07.609943+0000","last_fullsized":"2024-07-31T14:51:07.609943+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:01:27.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:18.046730+0000","last_change":"2024-07-31T14:47:13.102220+0000","last_active":"2024-07-31T14:48:18.046730+0000","last_peered":"2024-07-31T14:48:18.046730+0000","last_clean":"2024-07-31T14:48:18.046730+0000","last_became_active":"2024-07-31T14:47:13.101764+0000","last_became_peered":"2024-07-31T14:47:13.101764+0000","last_unstale":"2024-07-31T14:48:18.046730+0000","last_undegraded":"2024-07-31T14:48:18.046730+0000","last_fullsized":"2024-07-31T14:48:18.046730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:56:44.335340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.397183+0000","last_change":"2024-07-31T14:47:03.052348+0000","last_active":"2024-07-31T14:49:49.397183+0000","last_peered":"2024-07-31T14:49:49.397183+0000","last_clean":"2024-07-31T14:49:49.397183+0000","last_became_active":"2024-07-31T14:47:03.051961+0000","last_became_peered":"2024-07-31T14:47:03.051961+0000","last_unstale":"2024-07-31T14:49:49.397183+0000","last_undegraded":"2024-07-31T14:49:49.397183+0000","last_fullsized":"2024-07-31T14:49:49.397183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:19:35.279809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.597386+0000","last_change":"2024-07-31T14:47:00.025013+0000","last_active":"2024-07-31T14:48:02.597386+0000","last_peered":"2024-07-31T14:48:02.597386+0000","last_clean":"2024-07-31T14:48:02.597386+0000","last_became_active":"2024-07-31T14:47:00.024637+0000","last_became_peered":"2024-07-31T14:47:00.024637+0000","last_unstale":"2024-07-31T14:48:02.597386+0000","last_undegraded":"2024-07-31T14:48:02.597386+0000","last_fullsized":"2024-07-31T14:48:02.597386+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:07:21.519549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630339+0000","last_change":"2024-07-31T14:47:20.629262+0000","last_active":"2024-07-31T14:47:20.630339+0000","last_peered":"2024-07-31T14:47:20.630339+0000","last_clean":"2024-07-31T14:47:20.630339+0000","last_became_active":"2024-07-31T14:47:20.628318+0000","last_became_peered":"2024-07-31T14:47:20.628318+0000","last_unstale":"2024-07-31T14:47:20.630339+0000","last_undegraded":"2024-07-31T14:47:20.630339+0000","last_fullsized":"2024-07-31T14:47:20.630339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.206705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.361650+0000","last_change":"2024-07-31T14:47:03.051527+0000","last_active":"2024-07-31T14:49:49.361650+0000","last_peered":"2024-07-31T14:49:49.361650+0000","last_clean":"2024-07-31T14:49:49.361650+0000","last_became_active":"2024-07-31T14:47:03.051139+0000","last_became_peered":"2024-07-31T14:47:03.051139+0000","last_unstale":"2024-07-31T14:49:49.361650+0000","last_undegraded":"2024-07-31T14:49:49.361650+0000","last_fullsized":"2024-07-31T14:49:49.361650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:53:11.962719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:59.566389+0000","last_change":"2024-07-31T14:47:00.183812+0000","last_active":"2024-07-31T14:47:59.566389+0000","last_peered":"2024-07-31T14:47:59.566389+0000","last_clean":"2024-07-31T14:47:59.566389+0000","last_became_active":"2024-07-31T14:47:00.183440+0000","last_became_peered":"2024-07-31T14:47:00.183440+0000","last_unstale":"2024-07-31T14:47:59.566389+0000","last_undegraded":"2024-07-31T14:47:59.566389+0000","last_fullsized":"2024-07-31T14:47:59.566389+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:47:23.166756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604338+0000","last_change":"2024-07-31T14:47:13.363563+0000","last_active":"2024-07-31T14:47:20.604338+0000","last_peered":"2024-07-31T14:47:20.604338+0000","last_clean":"2024-07-31T14:47:20.604338+0000","last_became_active":"2024-07-31T14:47:13.362697+0000","last_became_peered":"2024-07-31T14:47:13.362697+0000","last_unstale":"2024-07-31T14:47:20.604338+0000","last_undegraded":"2024-07-31T14:47:20.604338+0000","last_fullsized":"2024-07-31T14:47:20.604338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:26:38.292073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407720+0000","last_change":"2024-07-31T14:47:03.046763+0000","last_active":"2024-07-31T14:49:49.407720+0000","last_peered":"2024-07-31T14:49:49.407720+0000","last_clean":"2024-07-31T14:49:49.407720+0000","last_became_active":"2024-07-31T14:47:03.046408+0000","last_became_peered":"2024-07-31T14:47:03.046408+0000","last_unstale":"2024-07-31T14:49:49.407720+0000","last_undegraded":"2024-07-31T14:49:49.407720+0000","last_fullsized":"2024-07-31T14:49:49.407720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:43:22.838495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605677+0000","last_change":"2024-07-31T14:47:00.236075+0000","last_active":"2024-07-31T14:48:03.605677+0000","last_peered":"2024-07-31T14:48:03.605677+0000","last_clean":"2024-07-31T14:48:03.605677+0000","last_became_active":"2024-07-31T14:47:00.235758+0000","last_became_peered":"2024-07-31T14:47:00.235758+0000","last_unstale":"2024-07-31T14:48:03.605677+0000","last_undegraded":"2024-07-31T14:48:03.605677+0000","last_fullsized":"2024-07-31T14:48:03.605677+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:50.575648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.448218+0000","last_change":"2024-07-31T14:47:13.101876+0000","last_active":"2024-07-31T14:47:20.448218+0000","last_peered":"2024-07-31T14:47:20.448218+0000","last_clean":"2024-07-31T14:47:20.448218+0000","last_became_active":"2024-07-31T14:47:13.101276+0000","last_became_peered":"2024-07-31T14:47:13.101276+0000","last_unstale":"2024-07-31T14:47:20.448218+0000","last_undegraded":"2024-07-31T14:47:20.448218+0000","last_fullsized":"2024-07-31T14:47:20.448218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:53:24.684212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409477+0000","last_change":"2024-07-31T14:47:03.052478+0000","last_active":"2024-07-31T14:49:49.409477+0000","last_peered":"2024-07-31T14:49:49.409477+0000","last_clean":"2024-07-31T14:49:49.409477+0000","last_became_active":"2024-07-31T14:47:03.051877+0000","last_became_peered":"2024-07-31T14:47:03.051877+0000","last_unstale":"2024-07-31T14:49:49.409477+0000","last_undegraded":"2024-07-31T14:49:49.409477+0000","last_fullsized":"2024-07-31T14:49:49.409477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:30:32.666490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:07.082692+0000","last_change":"2024-07-31T14:47:00.020076+0000","last_active":"2024-07-31T14:51:07.082692+0000","last_peered":"2024-07-31T14:51:07.082692+0000","last_clean":"2024-07-31T14:51:07.082692+0000","last_became_active":"2024-07-31T14:47:00.019708+0000","last_became_peered":"2024-07-31T14:47:00.019708+0000","last_unstale":"2024-07-31T14:51:07.082692+0000","last_undegraded":"2024-07-31T14:51:07.082692+0000","last_fullsized":"2024-07-31T14:51:07.082692+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:46.520997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.625840+0000","last_change":"2024-07-31T14:47:13.362961+0000","last_active":"2024-07-31T14:47:20.625840+0000","last_peered":"2024-07-31T14:47:20.625840+0000","last_clean":"2024-07-31T14:47:20.625840+0000","last_became_active":"2024-07-31T14:47:13.362502+0000","last_became_peered":"2024-07-31T14:47:13.362502+0000","last_unstale":"2024-07-31T14:47:20.625840+0000","last_undegraded":"2024-07-31T14:47:20.625840+0000","last_fullsized":"2024-07-31T14:47:20.625840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:51:45.732477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399783+0000","last_change":"2024-07-31T14:47:03.039418+0000","last_active":"2024-07-31T14:49:49.399783+0000","last_peered":"2024-07-31T14:49:49.399783+0000","last_clean":"2024-07-31T14:49:49.399783+0000","last_became_active":"2024-07-31T14:47:03.039044+0000","last_became_peered":"2024-07-31T14:47:03.039044+0000","last_unstale":"2024-07-31T14:49:49.399783+0000","last_undegraded":"2024-07-31T14:49:49.399783+0000","last_fullsized":"2024-07-31T14:49:49.399783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:40:10.380080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:07.059440+0000","last_change":"2024-07-31T14:47:00.183778+0000","last_active":"2024-07-31T14:49:07.059440+0000","last_peered":"2024-07-31T14:49:07.059440+0000","last_clean":"2024-07-31T14:49:07.059440+0000","last_became_active":"2024-07-31T14:47:00.183370+0000","last_became_peered":"2024-07-31T14:47:00.183370+0000","last_unstale":"2024-07-31T14:49:07.059440+0000","last_undegraded":"2024-07-31T14:49:07.059440+0000","last_fullsized":"2024-07-31T14:49:07.059440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:45:27.422216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445139+0000","last_change":"2024-07-31T14:47:13.102583+0000","last_active":"2024-07-31T14:47:20.445139+0000","last_peered":"2024-07-31T14:47:20.445139+0000","last_clean":"2024-07-31T14:47:20.445139+0000","last_became_active":"2024-07-31T14:47:13.102058+0000","last_became_peered":"2024-07-31T14:47:13.102058+0000","last_unstale":"2024-07-31T14:47:20.445139+0000","last_undegraded":"2024-07-31T14:47:20.445139+0000","last_fullsized":"2024-07-31T14:47:20.445139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:17.396173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407879+0000","last_change":"2024-07-31T14:47:03.048334+0000","last_active":"2024-07-31T14:49:49.407879+0000","last_peered":"2024-07-31T14:49:49.407879+0000","last_clean":"2024-07-31T14:49:49.407879+0000","last_became_active":"2024-07-31T14:47:03.047932+0000","last_became_peered":"2024-07-31T14:47:03.047932+0000","last_unstale":"2024-07-31T14:49:49.407879+0000","last_undegraded":"2024-07-31T14:49:49.407879+0000","last_fullsized":"2024-07-31T14:49:49.407879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:14:31.381061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:06.494830+0000","last_change":"2024-07-31T14:47:20.450851+0000","last_active":"2024-07-31T14:49:06.494830+0000","last_peered":"2024-07-31T14:49:06.494830+0000","last_clean":"2024-07-31T14:49:06.494830+0000","last_became_active":"2024-07-31T14:47:20.450435+0000","last_became_peered":"2024-07-31T14:47:20.450435+0000","last_unstale":"2024-07-31T14:49:06.494830+0000","last_undegraded":"2024-07-31T14:49:06.494830+0000","last_fullsized":"2024-07-31T14:49:06.494830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:17:28.738911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.450860+0000","last_change":"2024-07-31T14:47:13.363054+0000","last_active":"2024-07-31T14:47:20.450860+0000","last_peered":"2024-07-31T14:47:20.450860+0000","last_clean":"2024-07-31T14:47:20.450860+0000","last_became_active":"2024-07-31T14:47:13.362364+0000","last_became_peered":"2024-07-31T14:47:13.362364+0000","last_unstale":"2024-07-31T14:47:20.450860+0000","last_undegraded":"2024-07-31T14:47:20.450860+0000","last_fullsized":"2024-07-31T14:47:20.450860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:02:36.569678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.393142+0000","last_change":"2024-07-31T14:47:03.059834+0000","last_active":"2024-07-31T14:49:49.393142+0000","last_peered":"2024-07-31T14:49:49.393142+0000","last_clean":"2024-07-31T14:49:49.393142+0000","last_became_active":"2024-07-31T14:47:03.059127+0000","last_became_peered":"2024-07-31T14:47:03.059127+0000","last_unstale":"2024-07-31T14:49:49.393142+0000","last_undegraded":"2024-07-31T14:49:49.393142+0000","last_fullsized":"2024-07-31T14:49:49.393142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:53:50.308789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":241,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:10.893959+0000","last_change":"2024-07-31T14:47:00.030576+0000","last_active":"2024-07-31T14:51:10.893959+0000","last_peered":"2024-07-31T14:51:10.893959+0000","last_clean":"2024-07-31T14:51:10.893959+0000","last_became_active":"2024-07-31T14:47:00.030174+0000","last_became_peered":"2024-07-31T14:47:00.030174+0000","last_unstale":"2024-07-31T14:51:10.893959+0000","last_undegraded":"2024-07-31T14:51:10.893959+0000","last_fullsized":"2024-07-31T14:51:10.893959+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:13:35.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":191,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447258+0000","last_change":"2024-07-31T14:47:13.100180+0000","last_active":"2024-07-31T14:47:20.447258+0000","last_peered":"2024-07-31T14:47:20.447258+0000","last_clean":"2024-07-31T14:47:20.447258+0000","last_became_active":"2024-07-31T14:47:13.099725+0000","last_became_peered":"2024-07-31T14:47:13.099725+0000","last_unstale":"2024-07-31T14:47:20.447258+0000","last_undegraded":"2024-07-31T14:47:20.447258+0000","last_fullsized":"2024-07-31T14:47:20.447258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:25:32.245339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404277+0000","last_change":"2024-07-31T14:47:03.040349+0000","last_active":"2024-07-31T14:49:49.404277+0000","last_peered":"2024-07-31T14:49:49.404277+0000","last_clean":"2024-07-31T14:49:49.404277+0000","last_became_active":"2024-07-31T14:47:03.039887+0000","last_became_peered":"2024-07-31T14:47:03.039887+0000","last_unstale":"2024-07-31T14:49:49.404277+0000","last_undegraded":"2024-07-31T14:49:49.404277+0000","last_fullsized":"2024-07-31T14:49:49.404277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:46.592063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":241,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:12.881992+0000","last_change":"2024-07-31T14:47:00.183988+0000","last_active":"2024-07-31T14:51:12.881992+0000","last_peered":"2024-07-31T14:51:12.881992+0000","last_clean":"2024-07-31T14:51:12.881992+0000","last_became_active":"2024-07-31T14:47:00.183302+0000","last_became_peered":"2024-07-31T14:47:00.183302+0000","last_unstale":"2024-07-31T14:51:12.881992+0000","last_undegraded":"2024-07-31T14:51:12.881992+0000","last_fullsized":"2024-07-31T14:51:12.881992+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:40:26.188224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447744+0000","last_change":"2024-07-31T14:47:13.363075+0000","last_active":"2024-07-31T14:47:20.447744+0000","last_peered":"2024-07-31T14:47:20.447744+0000","last_clean":"2024-07-31T14:47:20.447744+0000","last_became_active":"2024-07-31T14:47:13.362599+0000","last_became_peered":"2024-07-31T14:47:13.362599+0000","last_unstale":"2024-07-31T14:47:20.447744+0000","last_undegraded":"2024-07-31T14:47:20.447744+0000","last_fullsized":"2024-07-31T14:47:20.447744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:20:43.617742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.402186+0000","last_change":"2024-07-31T14:47:20.450712+0000","last_active":"2024-07-31T14:49:49.402186+0000","last_peered":"2024-07-31T14:49:49.402186+0000","last_clean":"2024-07-31T14:49:49.402186+0000","last_became_active":"2024-07-31T14:47:20.450229+0000","last_became_peered":"2024-07-31T14:47:20.450229+0000","last_unstale":"2024-07-31T14:49:49.402186+0000","last_undegraded":"2024-07-31T14:49:49.402186+0000","last_fullsized":"2024-07-31T14:49:49.402186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:11:50.631420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:01.604764+0000","last_change":"2024-07-31T14:47:00.021389+0000","last_active":"2024-07-31T14:49:01.604764+0000","last_peered":"2024-07-31T14:49:01.604764+0000","last_clean":"2024-07-31T14:49:01.604764+0000","last_became_active":"2024-07-31T14:47:00.020987+0000","last_became_peered":"2024-07-31T14:47:00.020987+0000","last_unstale":"2024-07-31T14:49:01.604764+0000","last_undegraded":"2024-07-31T14:49:01.604764+0000","last_fullsized":"2024-07-31T14:49:01.604764+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:07:24.658463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627003+0000","last_change":"2024-07-31T14:47:13.363105+0000","last_active":"2024-07-31T14:47:20.627003+0000","last_peered":"2024-07-31T14:47:20.627003+0000","last_clean":"2024-07-31T14:47:20.627003+0000","last_became_active":"2024-07-31T14:47:13.362404+0000","last_became_peered":"2024-07-31T14:47:13.362404+0000","last_unstale":"2024-07-31T14:47:20.627003+0000","last_undegraded":"2024-07-31T14:47:20.627003+0000","last_fullsized":"2024-07-31T14:47:20.627003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:48:43.033584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404516+0000","last_change":"2024-07-31T14:47:03.057199+0000","last_active":"2024-07-31T14:49:49.404516+0000","last_peered":"2024-07-31T14:49:49.404516+0000","last_clean":"2024-07-31T14:49:49.404516+0000","last_became_active":"2024-07-31T14:47:03.056555+0000","last_became_peered":"2024-07-31T14:47:03.056555+0000","last_unstale":"2024-07-31T14:49:49.404516+0000","last_undegraded":"2024-07-31T14:49:49.404516+0000","last_fullsized":"2024-07-31T14:49:49.404516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:21:14.263457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:04.380224+0000","last_change":"2024-07-31T14:47:00.032055+0000","last_active":"2024-07-31T14:49:04.380224+0000","last_peered":"2024-07-31T14:49:04.380224+0000","last_clean":"2024-07-31T14:49:04.380224+0000","last_became_active":"2024-07-31T14:47:00.031519+0000","last_became_peered":"2024-07-31T14:47:00.031519+0000","last_unstale":"2024-07-31T14:49:04.380224+0000","last_undegraded":"2024-07-31T14:49:04.380224+0000","last_fullsized":"2024-07-31T14:49:04.380224+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:21:46.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:31.600736+0000","last_change":"2024-07-31T14:45:42.166503+0000","last_active":"2024-07-31T14:48:31.600736+0000","last_peered":"2024-07-31T14:48:31.600736+0000","last_clean":"2024-07-31T14:48:31.600736+0000","last_became_active":"2024-07-31T14:45:42.160065+0000","last_became_peered":"2024-07-31T14:45:42.160065+0000","last_unstale":"2024-07-31T14:48:31.600736+0000","last_undegraded":"2024-07-31T14:48:31.600736+0000","last_fullsized":"2024-07-31T14:48:31.600736+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_clean_scrub_stamp":"2024-07-31T14:42:27.863113+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:38:33.646022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:12.369486+0000","last_change":"2024-07-31T14:47:13.103812+0000","last_active":"2024-07-31T14:48:12.369486+0000","last_peered":"2024-07-31T14:48:12.369486+0000","last_clean":"2024-07-31T14:48:12.369486+0000","last_became_active":"2024-07-31T14:47:13.102549+0000","last_became_peered":"2024-07-31T14:47:13.102549+0000","last_unstale":"2024-07-31T14:48:12.369486+0000","last_undegraded":"2024-07-31T14:48:12.369486+0000","last_fullsized":"2024-07-31T14:48:12.369486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:18:56.322426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414083+0000","last_change":"2024-07-31T14:47:03.056310+0000","last_active":"2024-07-31T14:49:49.414083+0000","last_peered":"2024-07-31T14:49:49.414083+0000","last_clean":"2024-07-31T14:49:49.414083+0000","last_became_active":"2024-07-31T14:47:03.055695+0000","last_became_peered":"2024-07-31T14:47:03.055695+0000","last_unstale":"2024-07-31T14:49:49.414083+0000","last_undegraded":"2024-07-31T14:49:49.414083+0000","last_fullsized":"2024-07-31T14:49:49.414083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:00:55.124981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:09.891605+0000","last_change":"2024-07-31T14:47:00.236460+0000","last_active":"2024-07-31T14:51:09.891605+0000","last_peered":"2024-07-31T14:51:09.891605+0000","last_clean":"2024-07-31T14:51:09.891605+0000","last_became_active":"2024-07-31T14:47:00.235924+0000","last_became_peered":"2024-07-31T14:47:00.235924+0000","last_unstale":"2024-07-31T14:51:09.891605+0000","last_undegraded":"2024-07-31T14:51:09.891605+0000","last_fullsized":"2024-07-31T14:51:09.891605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:05.130958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626080+0000","last_change":"2024-07-31T14:47:13.102341+0000","last_active":"2024-07-31T14:47:20.626080+0000","last_peered":"2024-07-31T14:47:20.626080+0000","last_clean":"2024-07-31T14:47:20.626080+0000","last_became_active":"2024-07-31T14:47:13.101798+0000","last_became_peered":"2024-07-31T14:47:13.101798+0000","last_unstale":"2024-07-31T14:47:20.626080+0000","last_undegraded":"2024-07-31T14:47:20.626080+0000","last_fullsized":"2024-07-31T14:47:20.626080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:34:35.758041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409700+0000","last_change":"2024-07-31T14:47:03.054322+0000","last_active":"2024-07-31T14:49:49.409700+0000","last_peered":"2024-07-31T14:49:49.409700+0000","last_clean":"2024-07-31T14:49:49.409700+0000","last_became_active":"2024-07-31T14:47:03.053112+0000","last_became_peered":"2024-07-31T14:47:03.053112+0000","last_unstale":"2024-07-31T14:49:49.409700+0000","last_undegraded":"2024-07-31T14:49:49.409700+0000","last_fullsized":"2024-07-31T14:49:49.409700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:55:13.542578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604832+0000","last_change":"2024-07-31T14:47:00.026518+0000","last_active":"2024-07-31T14:48:03.604832+0000","last_peered":"2024-07-31T14:48:03.604832+0000","last_clean":"2024-07-31T14:48:03.604832+0000","last_became_active":"2024-07-31T14:47:00.026074+0000","last_became_peered":"2024-07-31T14:47:00.026074+0000","last_unstale":"2024-07-31T14:48:03.604832+0000","last_undegraded":"2024-07-31T14:48:03.604832+0000","last_fullsized":"2024-07-31T14:48:03.604832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:33:36.688353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449369+0000","last_change":"2024-07-31T14:47:13.106365+0000","last_active":"2024-07-31T14:47:20.449369+0000","last_peered":"2024-07-31T14:47:20.449369+0000","last_clean":"2024-07-31T14:47:20.449369+0000","last_became_active":"2024-07-31T14:47:13.105635+0000","last_became_peered":"2024-07-31T14:47:13.105635+0000","last_unstale":"2024-07-31T14:47:20.449369+0000","last_undegraded":"2024-07-31T14:47:20.449369+0000","last_fullsized":"2024-07-31T14:47:20.449369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:52:09.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404285+0000","last_change":"2024-07-31T14:47:03.052522+0000","last_active":"2024-07-31T14:49:49.404285+0000","last_peered":"2024-07-31T14:49:49.404285+0000","last_clean":"2024-07-31T14:49:49.404285+0000","last_became_active":"2024-07-31T14:47:03.051872+0000","last_became_peered":"2024-07-31T14:47:03.051872+0000","last_unstale":"2024-07-31T14:49:49.404285+0000","last_undegraded":"2024-07-31T14:49:49.404285+0000","last_fullsized":"2024-07-31T14:49:49.404285+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:29:34.956745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600950+0000","last_change":"2024-07-31T14:47:00.016839+0000","last_active":"2024-07-31T14:48:02.600950+0000","last_peered":"2024-07-31T14:48:02.600950+0000","last_clean":"2024-07-31T14:48:02.600950+0000","last_became_active":"2024-07-31T14:47:00.016438+0000","last_became_peered":"2024-07-31T14:47:00.016438+0000","last_unstale":"2024-07-31T14:48:02.600950+0000","last_undegraded":"2024-07-31T14:48:02.600950+0000","last_fullsized":"2024-07-31T14:48:02.600950+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:42:10.875332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626258+0000","last_change":"2024-07-31T14:47:13.105385+0000","last_active":"2024-07-31T14:47:20.626258+0000","last_peered":"2024-07-31T14:47:20.626258+0000","last_clean":"2024-07-31T14:47:20.626258+0000","last_became_active":"2024-07-31T14:47:13.104867+0000","last_became_peered":"2024-07-31T14:47:13.104867+0000","last_unstale":"2024-07-31T14:47:20.626258+0000","last_undegraded":"2024-07-31T14:47:20.626258+0000","last_fullsized":"2024-07-31T14:47:20.626258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:25:21.572830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399072+0000","last_change":"2024-07-31T14:47:03.058304+0000","last_active":"2024-07-31T14:49:49.399072+0000","last_peered":"2024-07-31T14:49:49.399072+0000","last_clean":"2024-07-31T14:49:49.399072+0000","last_became_active":"2024-07-31T14:47:03.057251+0000","last_became_peered":"2024-07-31T14:47:03.057251+0000","last_unstale":"2024-07-31T14:49:49.399072+0000","last_undegraded":"2024-07-31T14:49:49.399072+0000","last_fullsized":"2024-07-31T14:49:49.399072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:58:20.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605301+0000","last_change":"2024-07-31T14:47:00.027806+0000","last_active":"2024-07-31T14:48:03.605301+0000","last_peered":"2024-07-31T14:48:03.605301+0000","last_clean":"2024-07-31T14:48:03.605301+0000","last_became_active":"2024-07-31T14:47:00.027403+0000","last_became_peered":"2024-07-31T14:47:00.027403+0000","last_unstale":"2024-07-31T14:48:03.605301+0000","last_undegraded":"2024-07-31T14:48:03.605301+0000","last_fullsized":"2024-07-31T14:48:03.605301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:57:18.303732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626490+0000","last_change":"2024-07-31T14:47:13.103628+0000","last_active":"2024-07-31T14:47:20.626490+0000","last_peered":"2024-07-31T14:47:20.626490+0000","last_clean":"2024-07-31T14:47:20.626490+0000","last_became_active":"2024-07-31T14:47:13.102296+0000","last_became_peered":"2024-07-31T14:47:13.102296+0000","last_unstale":"2024-07-31T14:47:20.626490+0000","last_undegraded":"2024-07-31T14:47:20.626490+0000","last_fullsized":"2024-07-31T14:47:20.626490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:25:35.109196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413387+0000","last_change":"2024-07-31T14:47:03.052761+0000","last_active":"2024-07-31T14:49:49.413387+0000","last_peered":"2024-07-31T14:49:49.413387+0000","last_clean":"2024-07-31T14:49:49.413387+0000","last_became_active":"2024-07-31T14:47:03.052410+0000","last_became_peered":"2024-07-31T14:47:03.052410+0000","last_unstale":"2024-07-31T14:49:49.413387+0000","last_undegraded":"2024-07-31T14:49:49.413387+0000","last_fullsized":"2024-07-31T14:49:49.413387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:20:47.712491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604937+0000","last_change":"2024-07-31T14:47:00.183741+0000","last_active":"2024-07-31T14:48:03.604937+0000","last_peered":"2024-07-31T14:48:03.604937+0000","last_clean":"2024-07-31T14:48:03.604937+0000","last_became_active":"2024-07-31T14:47:00.183275+0000","last_became_peered":"2024-07-31T14:47:00.183275+0000","last_unstale":"2024-07-31T14:48:03.604937+0000","last_undegraded":"2024-07-31T14:48:03.604937+0000","last_fullsized":"2024-07-31T14:48:03.604937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:29:23.205835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630098+0000","last_change":"2024-07-31T14:47:20.629334+0000","last_active":"2024-07-31T14:47:20.630098+0000","last_peered":"2024-07-31T14:47:20.630098+0000","last_clean":"2024-07-31T14:47:20.630098+0000","last_became_active":"2024-07-31T14:47:20.628865+0000","last_became_peered":"2024-07-31T14:47:20.628865+0000","last_unstale":"2024-07-31T14:47:20.630098+0000","last_undegraded":"2024-07-31T14:47:20.630098+0000","last_fullsized":"2024-07-31T14:47:20.630098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:21:27.199842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407413+0000","last_change":"2024-07-31T14:47:03.054633+0000","last_active":"2024-07-31T14:49:49.407413+0000","last_peered":"2024-07-31T14:49:49.407413+0000","last_clean":"2024-07-31T14:49:49.407413+0000","last_became_active":"2024-07-31T14:47:03.053025+0000","last_became_peered":"2024-07-31T14:47:03.053025+0000","last_unstale":"2024-07-31T14:49:49.407413+0000","last_undegraded":"2024-07-31T14:49:49.407413+0000","last_fullsized":"2024-07-31T14:49:49.407413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:42:03.392744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:08.876300+0000","last_change":"2024-07-31T14:47:00.027898+0000","last_active":"2024-07-31T14:51:08.876300+0000","last_peered":"2024-07-31T14:51:08.876300+0000","last_clean":"2024-07-31T14:51:08.876300+0000","last_became_active":"2024-07-31T14:47:00.027468+0000","last_became_peered":"2024-07-31T14:47:00.027468+0000","last_unstale":"2024-07-31T14:51:08.876300+0000","last_undegraded":"2024-07-31T14:51:08.876300+0000","last_fullsized":"2024-07-31T14:51:08.876300+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:12:45.922172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444574+0000","last_change":"2024-07-31T14:47:13.104327+0000","last_active":"2024-07-31T14:47:20.444574+0000","last_peered":"2024-07-31T14:47:20.444574+0000","last_clean":"2024-07-31T14:47:20.444574+0000","last_became_active":"2024-07-31T14:47:13.103333+0000","last_became_peered":"2024-07-31T14:47:13.103333+0000","last_unstale":"2024-07-31T14:47:20.444574+0000","last_undegraded":"2024-07-31T14:47:20.444574+0000","last_fullsized":"2024-07-31T14:47:20.444574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:06.226220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.410145+0000","last_change":"2024-07-31T14:47:03.044875+0000","last_active":"2024-07-31T14:49:49.410145+0000","last_peered":"2024-07-31T14:49:49.410145+0000","last_clean":"2024-07-31T14:49:49.410145+0000","last_became_active":"2024-07-31T14:47:03.044421+0000","last_became_peered":"2024-07-31T14:47:03.044421+0000","last_unstale":"2024-07-31T14:49:49.410145+0000","last_undegraded":"2024-07-31T14:49:49.410145+0000","last_fullsized":"2024-07-31T14:49:49.410145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:00:02.477300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605606+0000","last_change":"2024-07-31T14:47:00.019790+0000","last_active":"2024-07-31T14:48:03.605606+0000","last_peered":"2024-07-31T14:48:03.605606+0000","last_clean":"2024-07-31T14:48:03.605606+0000","last_became_active":"2024-07-31T14:47:00.019368+0000","last_became_peered":"2024-07-31T14:47:00.019368+0000","last_unstale":"2024-07-31T14:48:03.605606+0000","last_undegraded":"2024-07-31T14:48:03.605606+0000","last_fullsized":"2024-07-31T14:48:03.605606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:15.299109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445672+0000","last_change":"2024-07-31T14:47:13.102618+0000","last_active":"2024-07-31T14:47:20.445672+0000","last_peered":"2024-07-31T14:47:20.445672+0000","last_clean":"2024-07-31T14:47:20.445672+0000","last_became_active":"2024-07-31T14:47:13.102085+0000","last_became_peered":"2024-07-31T14:47:13.102085+0000","last_unstale":"2024-07-31T14:47:20.445672+0000","last_undegraded":"2024-07-31T14:47:20.445672+0000","last_fullsized":"2024-07-31T14:47:20.445672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:05:58.814215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.406277+0000","last_change":"2024-07-31T14:47:03.044688+0000","last_active":"2024-07-31T14:49:49.406277+0000","last_peered":"2024-07-31T14:49:49.406277+0000","last_clean":"2024-07-31T14:49:49.406277+0000","last_became_active":"2024-07-31T14:47:03.043751+0000","last_became_peered":"2024-07-31T14:47:03.043751+0000","last_unstale":"2024-07-31T14:49:49.406277+0000","last_undegraded":"2024-07-31T14:49:49.406277+0000","last_fullsized":"2024-07-31T14:49:49.406277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:47:31.450796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605830+0000","last_change":"2024-07-31T14:47:00.026259+0000","last_active":"2024-07-31T14:48:03.605830+0000","last_peered":"2024-07-31T14:48:03.605830+0000","last_clean":"2024-07-31T14:48:03.605830+0000","last_became_active":"2024-07-31T14:47:00.025756+0000","last_became_peered":"2024-07-31T14:47:00.025756+0000","last_unstale":"2024-07-31T14:48:03.605830+0000","last_undegraded":"2024-07-31T14:48:03.605830+0000","last_fullsized":"2024-07-31T14:48:03.605830+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:05:00.258841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626888+0000","last_change":"2024-07-31T14:47:13.104918+0000","last_active":"2024-07-31T14:47:20.626888+0000","last_peered":"2024-07-31T14:47:20.626888+0000","last_clean":"2024-07-31T14:47:20.626888+0000","last_became_active":"2024-07-31T14:47:13.104031+0000","last_became_peered":"2024-07-31T14:47:13.104031+0000","last_unstale":"2024-07-31T14:47:20.626888+0000","last_undegraded":"2024-07-31T14:47:20.626888+0000","last_fullsized":"2024-07-31T14:47:20.626888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:17:15.264474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.389841+0000","last_change":"2024-07-31T14:47:03.055159+0000","last_active":"2024-07-31T14:49:49.389841+0000","last_peered":"2024-07-31T14:49:49.389841+0000","last_clean":"2024-07-31T14:49:49.389841+0000","last_became_active":"2024-07-31T14:47:03.052389+0000","last_became_peered":"2024-07-31T14:47:03.052389+0000","last_unstale":"2024-07-31T14:49:49.389841+0000","last_undegraded":"2024-07-31T14:49:49.389841+0000","last_fullsized":"2024-07-31T14:49:49.389841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:16.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'41","reported_seq":469,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:09.878153+0000","last_change":"2024-07-31T14:47:00.030165+0000","last_active":"2024-07-31T14:51:09.878153+0000","last_peered":"2024-07-31T14:51:09.878153+0000","last_clean":"2024-07-31T14:51:09.878153+0000","last_became_active":"2024-07-31T14:47:00.029559+0000","last_became_peered":"2024-07-31T14:47:00.029559+0000","last_unstale":"2024-07-31T14:51:09.878153+0000","last_undegraded":"2024-07-31T14:51:09.878153+0000","last_fullsized":"2024-07-31T14:51:09.878153+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:31:06.115075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69398,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":386,"num_read_kb":261,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444234+0000","last_change":"2024-07-31T14:47:13.098517+0000","last_active":"2024-07-31T14:47:20.444234+0000","last_peered":"2024-07-31T14:47:20.444234+0000","last_clean":"2024-07-31T14:47:20.444234+0000","last_became_active":"2024-07-31T14:47:13.098066+0000","last_became_peered":"2024-07-31T14:47:13.098066+0000","last_unstale":"2024-07-31T14:47:20.444234+0000","last_undegraded":"2024-07-31T14:47:20.444234+0000","last_fullsized":"2024-07-31T14:47:20.444234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:08:54.958646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414451+0000","last_change":"2024-07-31T14:47:03.054045+0000","last_active":"2024-07-31T14:49:49.414451+0000","last_peered":"2024-07-31T14:49:49.414451+0000","last_clean":"2024-07-31T14:49:49.414451+0000","last_became_active":"2024-07-31T14:47:03.053658+0000","last_became_peered":"2024-07-31T14:47:03.053658+0000","last_unstale":"2024-07-31T14:49:49.414451+0000","last_undegraded":"2024-07-31T14:49:49.414451+0000","last_fullsized":"2024-07-31T14:49:49.414451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:36:54.563542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.607373+0000","last_change":"2024-07-31T14:47:00.185602+0000","last_active":"2024-07-31T14:48:03.607373+0000","last_peered":"2024-07-31T14:48:03.607373+0000","last_clean":"2024-07-31T14:48:03.607373+0000","last_became_active":"2024-07-31T14:47:00.185230+0000","last_became_peered":"2024-07-31T14:47:00.185230+0000","last_unstale":"2024-07-31T14:48:03.607373+0000","last_undegraded":"2024-07-31T14:48:03.607373+0000","last_fullsized":"2024-07-31T14:48:03.607373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:21:35.979942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604698+0000","last_change":"2024-07-31T14:47:13.099611+0000","last_active":"2024-07-31T14:47:20.604698+0000","last_peered":"2024-07-31T14:47:20.604698+0000","last_clean":"2024-07-31T14:47:20.604698+0000","last_became_active":"2024-07-31T14:47:13.099127+0000","last_became_peered":"2024-07-31T14:47:13.099127+0000","last_unstale":"2024-07-31T14:47:20.604698+0000","last_undegraded":"2024-07-31T14:47:20.604698+0000","last_fullsized":"2024-07-31T14:47:20.604698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:16:57.192980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400126+0000","last_change":"2024-07-31T14:47:03.056818+0000","last_active":"2024-07-31T14:49:49.400126+0000","last_peered":"2024-07-31T14:49:49.400126+0000","last_clean":"2024-07-31T14:49:49.400126+0000","last_became_active":"2024-07-31T14:47:03.055716+0000","last_became_peered":"2024-07-31T14:47:03.055716+0000","last_unstale":"2024-07-31T14:49:49.400126+0000","last_undegraded":"2024-07-31T14:49:49.400126+0000","last_fullsized":"2024-07-31T14:49:49.400126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:26:33.627362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600344+0000","last_change":"2024-07-31T14:47:00.029651+0000","last_active":"2024-07-31T14:48:02.600344+0000","last_peered":"2024-07-31T14:48:02.600344+0000","last_clean":"2024-07-31T14:48:02.600344+0000","last_became_active":"2024-07-31T14:47:00.029242+0000","last_became_peered":"2024-07-31T14:47:00.029242+0000","last_unstale":"2024-07-31T14:48:02.600344+0000","last_undegraded":"2024-07-31T14:48:02.600344+0000","last_fullsized":"2024-07-31T14:48:02.600344+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:23:49.177508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.443852+0000","last_change":"2024-07-31T14:47:13.098862+0000","last_active":"2024-07-31T14:47:20.443852+0000","last_peered":"2024-07-31T14:47:20.443852+0000","last_clean":"2024-07-31T14:47:20.443852+0000","last_became_active":"2024-07-31T14:47:13.098324+0000","last_became_peered":"2024-07-31T14:47:13.098324+0000","last_unstale":"2024-07-31T14:47:20.443852+0000","last_undegraded":"2024-07-31T14:47:20.443852+0000","last_fullsized":"2024-07-31T14:47:20.443852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:50:41.260863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.412242+0000","last_change":"2024-07-31T14:47:03.049042+0000","last_active":"2024-07-31T14:49:49.412242+0000","last_peered":"2024-07-31T14:49:49.412242+0000","last_clean":"2024-07-31T14:49:49.412242+0000","last_became_active":"2024-07-31T14:47:03.048535+0000","last_became_peered":"2024-07-31T14:47:03.048535+0000","last_unstale":"2024-07-31T14:49:49.412242+0000","last_undegraded":"2024-07-31T14:49:49.412242+0000","last_fullsized":"2024-07-31T14:49:49.412242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:15.147454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":422,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:09.877699+0000","last_change":"2024-07-31T14:47:00.028940+0000","last_active":"2024-07-31T14:51:09.877699+0000","last_peered":"2024-07-31T14:51:09.877699+0000","last_clean":"2024-07-31T14:51:09.877699+0000","last_became_active":"2024-07-31T14:47:00.028472+0000","last_became_peered":"2024-07-31T14:47:00.028472+0000","last_unstale":"2024-07-31T14:51:09.877699+0000","last_undegraded":"2024-07-31T14:51:09.877699+0000","last_fullsized":"2024-07-31T14:51:09.877699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:57:15.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":374,"num_read_kb":374,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626394+0000","last_change":"2024-07-31T14:47:13.101856+0000","last_active":"2024-07-31T14:47:20.626394+0000","last_peered":"2024-07-31T14:47:20.626394+0000","last_clean":"2024-07-31T14:47:20.626394+0000","last_became_active":"2024-07-31T14:47:13.101346+0000","last_became_peered":"2024-07-31T14:47:13.101346+0000","last_unstale":"2024-07-31T14:47:20.626394+0000","last_undegraded":"2024-07-31T14:47:20.626394+0000","last_fullsized":"2024-07-31T14:47:20.626394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:56:37.071660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449190+0000","last_change":"2024-07-31T14:47:13.105952+0000","last_active":"2024-07-31T14:47:20.449190+0000","last_peered":"2024-07-31T14:47:20.449190+0000","last_clean":"2024-07-31T14:47:20.449190+0000","last_became_active":"2024-07-31T14:47:13.105506+0000","last_became_peered":"2024-07-31T14:47:13.105506+0000","last_unstale":"2024-07-31T14:47:20.449190+0000","last_undegraded":"2024-07-31T14:47:20.449190+0000","last_fullsized":"2024-07-31T14:47:20.449190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:28:33.866216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400532+0000","last_change":"2024-07-31T14:47:03.051152+0000","last_active":"2024-07-31T14:49:49.400532+0000","last_peered":"2024-07-31T14:49:49.400532+0000","last_clean":"2024-07-31T14:49:49.400532+0000","last_became_active":"2024-07-31T14:47:03.050775+0000","last_became_peered":"2024-07-31T14:47:03.050775+0000","last_unstale":"2024-07-31T14:49:49.400532+0000","last_undegraded":"2024-07-31T14:49:49.400532+0000","last_fullsized":"2024-07-31T14:49:49.400532+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:47:35.913641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609201+0000","last_change":"2024-07-31T14:47:00.028552+0000","last_active":"2024-07-31T14:48:03.609201+0000","last_peered":"2024-07-31T14:48:03.609201+0000","last_clean":"2024-07-31T14:48:03.609201+0000","last_became_active":"2024-07-31T14:47:00.028143+0000","last_became_peered":"2024-07-31T14:47:00.028143+0000","last_unstale":"2024-07-31T14:48:03.609201+0000","last_undegraded":"2024-07-31T14:48:03.609201+0000","last_fullsized":"2024-07-31T14:48:03.609201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:09.601635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.606879+0000","last_change":"2024-07-31T14:47:20.606285+0000","last_active":"2024-07-31T14:47:20.606879+0000","last_peered":"2024-07-31T14:47:20.606879+0000","last_clean":"2024-07-31T14:47:20.606879+0000","last_became_active":"2024-07-31T14:47:20.605837+0000","last_became_peered":"2024-07-31T14:47:20.605837+0000","last_unstale":"2024-07-31T14:47:20.606879+0000","last_undegraded":"2024-07-31T14:47:20.606879+0000","last_fullsized":"2024-07-31T14:47:20.606879+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:16:35.409790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.398709+0000","last_change":"2024-07-31T14:47:03.047442+0000","last_active":"2024-07-31T14:49:49.398709+0000","last_peered":"2024-07-31T14:49:49.398709+0000","last_clean":"2024-07-31T14:49:49.398709+0000","last_became_active":"2024-07-31T14:47:03.047072+0000","last_became_peered":"2024-07-31T14:47:03.047072+0000","last_unstale":"2024-07-31T14:49:49.398709+0000","last_undegraded":"2024-07-31T14:49:49.398709+0000","last_fullsized":"2024-07-31T14:49:49.398709+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:56.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.599595+0000","last_change":"2024-07-31T14:47:00.027746+0000","last_active":"2024-07-31T14:48:02.599595+0000","last_peered":"2024-07-31T14:48:02.599595+0000","last_clean":"2024-07-31T14:48:02.599595+0000","last_became_active":"2024-07-31T14:47:00.027404+0000","last_became_peered":"2024-07-31T14:47:00.027404+0000","last_unstale":"2024-07-31T14:48:02.599595+0000","last_undegraded":"2024-07-31T14:48:02.599595+0000","last_fullsized":"2024-07-31T14:48:02.599595+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:30:21.011674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.589640+0000","last_change":"2024-07-31T14:47:00.029148+0000","last_active":"2024-07-31T14:48:01.589640+0000","last_peered":"2024-07-31T14:48:01.589640+0000","last_clean":"2024-07-31T14:48:01.589640+0000","last_became_active":"2024-07-31T14:47:00.028718+0000","last_became_peered":"2024-07-31T14:47:00.028718+0000","last_unstale":"2024-07-31T14:48:01.589640+0000","last_undegraded":"2024-07-31T14:48:01.589640+0000","last_fullsized":"2024-07-31T14:48:01.589640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:25:16.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400207+0000","last_change":"2024-07-31T14:47:20.629098+0000","last_active":"2024-07-31T14:49:49.400207+0000","last_peered":"2024-07-31T14:49:49.400207+0000","last_clean":"2024-07-31T14:49:49.400207+0000","last_became_active":"2024-07-31T14:47:20.627775+0000","last_became_peered":"2024-07-31T14:47:20.627775+0000","last_unstale":"2024-07-31T14:49:49.400207+0000","last_undegraded":"2024-07-31T14:49:49.400207+0000","last_fullsized":"2024-07-31T14:49:49.400207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:22:03.606509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447331+0000","last_change":"2024-07-31T14:47:13.097892+0000","last_active":"2024-07-31T14:47:20.447331+0000","last_peered":"2024-07-31T14:47:20.447331+0000","last_clean":"2024-07-31T14:47:20.447331+0000","last_became_active":"2024-07-31T14:47:13.097419+0000","last_became_peered":"2024-07-31T14:47:13.097419+0000","last_unstale":"2024-07-31T14:47:20.447331+0000","last_undegraded":"2024-07-31T14:47:20.447331+0000","last_fullsized":"2024-07-31T14:47:20.447331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:07:13.456147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":243,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:11.881508+0000","last_change":"2024-07-31T14:47:00.019792+0000","last_active":"2024-07-31T14:51:11.881508+0000","last_peered":"2024-07-31T14:51:11.881508+0000","last_clean":"2024-07-31T14:51:11.881508+0000","last_became_active":"2024-07-31T14:47:00.019411+0000","last_became_peered":"2024-07-31T14:47:00.019411+0000","last_unstale":"2024-07-31T14:51:11.881508+0000","last_undegraded":"2024-07-31T14:51:11.881508+0000","last_fullsized":"2024-07-31T14:51:11.881508+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:55:02.148823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":194,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.390707+0000","last_change":"2024-07-31T14:47:03.039808+0000","last_active":"2024-07-31T14:49:49.390707+0000","last_peered":"2024-07-31T14:49:49.390707+0000","last_clean":"2024-07-31T14:49:49.390707+0000","last_became_active":"2024-07-31T14:47:03.039382+0000","last_became_peered":"2024-07-31T14:47:03.039382+0000","last_unstale":"2024-07-31T14:49:49.390707+0000","last_undegraded":"2024-07-31T14:49:49.390707+0000","last_fullsized":"2024-07-31T14:49:49.390707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:23:09.751293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.446939+0000","last_change":"2024-07-31T14:47:13.102797+0000","last_active":"2024-07-31T14:47:20.446939+0000","last_peered":"2024-07-31T14:47:20.446939+0000","last_clean":"2024-07-31T14:47:20.446939+0000","last_became_active":"2024-07-31T14:47:13.102320+0000","last_became_peered":"2024-07-31T14:47:13.102320+0000","last_unstale":"2024-07-31T14:47:20.446939+0000","last_undegraded":"2024-07-31T14:47:20.446939+0000","last_fullsized":"2024-07-31T14:47:20.446939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:23.066357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.397081+0000","last_change":"2024-07-31T14:47:03.054875+0000","last_active":"2024-07-31T14:49:49.397081+0000","last_peered":"2024-07-31T14:49:49.397081+0000","last_clean":"2024-07-31T14:49:49.397081+0000","last_became_active":"2024-07-31T14:47:03.054529+0000","last_became_peered":"2024-07-31T14:47:03.054529+0000","last_unstale":"2024-07-31T14:49:49.397081+0000","last_undegraded":"2024-07-31T14:49:49.397081+0000","last_fullsized":"2024-07-31T14:49:49.397081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:32:09.802933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396215+0000","last_change":"2024-07-31T14:47:03.043283+0000","last_active":"2024-07-31T14:49:49.396215+0000","last_peered":"2024-07-31T14:49:49.396215+0000","last_clean":"2024-07-31T14:49:49.396215+0000","last_became_active":"2024-07-31T14:47:03.042851+0000","last_became_peered":"2024-07-31T14:47:03.042851+0000","last_unstale":"2024-07-31T14:49:49.396215+0000","last_undegraded":"2024-07-31T14:49:49.396215+0000","last_fullsized":"2024-07-31T14:49:49.396215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:25:37.408149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409421+0000","last_change":"2024-07-31T14:47:03.047890+0000","last_active":"2024-07-31T14:49:49.409421+0000","last_peered":"2024-07-31T14:49:49.409421+0000","last_clean":"2024-07-31T14:49:49.409421+0000","last_became_active":"2024-07-31T14:47:03.047507+0000","last_became_peered":"2024-07-31T14:47:03.047507+0000","last_unstale":"2024-07-31T14:49:49.409421+0000","last_undegraded":"2024-07-31T14:49:49.409421+0000","last_fullsized":"2024-07-31T14:49:49.409421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:52:45.991323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403016+0000","last_change":"2024-07-31T14:47:03.047903+0000","last_active":"2024-07-31T14:49:49.403016+0000","last_peered":"2024-07-31T14:49:49.403016+0000","last_clean":"2024-07-31T14:49:49.403016+0000","last_became_active":"2024-07-31T14:47:03.047352+0000","last_became_peered":"2024-07-31T14:47:03.047352+0000","last_unstale":"2024-07-31T14:49:49.403016+0000","last_undegraded":"2024-07-31T14:49:49.403016+0000","last_fullsized":"2024-07-31T14:49:49.403016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:06.536023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403170+0000","last_change":"2024-07-31T14:47:03.044199+0000","last_active":"2024-07-31T14:49:49.403170+0000","last_peered":"2024-07-31T14:49:49.403170+0000","last_clean":"2024-07-31T14:49:49.403170+0000","last_became_active":"2024-07-31T14:47:03.043849+0000","last_became_peered":"2024-07-31T14:47:03.043849+0000","last_unstale":"2024-07-31T14:49:49.403170+0000","last_undegraded":"2024-07-31T14:49:49.403170+0000","last_fullsized":"2024-07-31T14:49:49.403170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:50.474895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.391079+0000","last_change":"2024-07-31T14:47:03.060654+0000","last_active":"2024-07-31T14:49:49.391079+0000","last_peered":"2024-07-31T14:49:49.391079+0000","last_clean":"2024-07-31T14:49:49.391079+0000","last_became_active":"2024-07-31T14:47:03.060157+0000","last_became_peered":"2024-07-31T14:47:03.060157+0000","last_unstale":"2024-07-31T14:49:49.391079+0000","last_undegraded":"2024-07-31T14:49:49.391079+0000","last_fullsized":"2024-07-31T14:49:49.391079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:39:50.130831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414163+0000","last_change":"2024-07-31T14:47:03.049910+0000","last_active":"2024-07-31T14:49:49.414163+0000","last_peered":"2024-07-31T14:49:49.414163+0000","last_clean":"2024-07-31T14:49:49.414163+0000","last_became_active":"2024-07-31T14:47:03.049491+0000","last_became_peered":"2024-07-31T14:47:03.049491+0000","last_unstale":"2024-07-31T14:49:49.414163+0000","last_undegraded":"2024-07-31T14:49:49.414163+0000","last_fullsized":"2024-07-31T14:49:49.414163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:18:43.319695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407340+0000","last_change":"2024-07-31T14:47:03.052206+0000","last_active":"2024-07-31T14:49:49.407340+0000","last_peered":"2024-07-31T14:49:49.407340+0000","last_clean":"2024-07-31T14:49:49.407340+0000","last_became_active":"2024-07-31T14:47:03.051175+0000","last_became_peered":"2024-07-31T14:47:03.051175+0000","last_unstale":"2024-07-31T14:49:49.407340+0000","last_undegraded":"2024-07-31T14:49:49.407340+0000","last_fullsized":"2024-07-31T14:49:49.407340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:51:11.583604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.405579+0000","last_change":"2024-07-31T14:47:03.052162+0000","last_active":"2024-07-31T14:49:49.405579+0000","last_peered":"2024-07-31T14:49:49.405579+0000","last_clean":"2024-07-31T14:49:49.405579+0000","last_became_active":"2024-07-31T14:47:03.051634+0000","last_became_peered":"2024-07-31T14:47:03.051634+0000","last_unstale":"2024-07-31T14:49:49.405579+0000","last_undegraded":"2024-07-31T14:49:49.405579+0000","last_fullsized":"2024-07-31T14:49:49.405579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:41:59.605192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409010+0000","last_change":"2024-07-31T14:47:03.047475+0000","last_active":"2024-07-31T14:49:49.409010+0000","last_peered":"2024-07-31T14:49:49.409010+0000","last_clean":"2024-07-31T14:49:49.409010+0000","last_became_active":"2024-07-31T14:47:03.047100+0000","last_became_peered":"2024-07-31T14:47:03.047100+0000","last_unstale":"2024-07-31T14:49:49.409010+0000","last_undegraded":"2024-07-31T14:49:49.409010+0000","last_fullsized":"2024-07-31T14:49:49.409010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:08:10.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411379+0000","last_change":"2024-07-31T14:47:03.051281+0000","last_active":"2024-07-31T14:49:49.411379+0000","last_peered":"2024-07-31T14:49:49.411379+0000","last_clean":"2024-07-31T14:49:49.411379+0000","last_became_active":"2024-07-31T14:47:03.050921+0000","last_became_peered":"2024-07-31T14:47:03.050921+0000","last_unstale":"2024-07-31T14:49:49.411379+0000","last_undegraded":"2024-07-31T14:49:49.411379+0000","last_fullsized":"2024-07-31T14:49:49.411379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:24:29.221723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403114+0000","last_change":"2024-07-31T14:47:03.045457+0000","last_active":"2024-07-31T14:49:49.403114+0000","last_peered":"2024-07-31T14:49:49.403114+0000","last_clean":"2024-07-31T14:49:49.403114+0000","last_became_active":"2024-07-31T14:47:03.044466+0000","last_became_peered":"2024-07-31T14:47:03.044466+0000","last_unstale":"2024-07-31T14:49:49.403114+0000","last_undegraded":"2024-07-31T14:49:49.403114+0000","last_fullsized":"2024-07-31T14:49:49.403114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:44:01.197416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403872+0000","last_change":"2024-07-31T14:47:03.053431+0000","last_active":"2024-07-31T14:49:49.403872+0000","last_peered":"2024-07-31T14:49:49.403872+0000","last_clean":"2024-07-31T14:49:49.403872+0000","last_became_active":"2024-07-31T14:47:03.052958+0000","last_became_peered":"2024-07-31T14:47:03.052958+0000","last_unstale":"2024-07-31T14:49:49.403872+0000","last_undegraded":"2024-07-31T14:49:49.403872+0000","last_fullsized":"2024-07-31T14:49:49.403872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:03:28.863589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409420+0000","last_change":"2024-07-31T14:47:03.046641+0000","last_active":"2024-07-31T14:49:49.409420+0000","last_peered":"2024-07-31T14:49:49.409420+0000","last_clean":"2024-07-31T14:49:49.409420+0000","last_became_active":"2024-07-31T14:47:03.046265+0000","last_became_peered":"2024-07-31T14:47:03.046265+0000","last_unstale":"2024-07-31T14:49:49.409420+0000","last_undegraded":"2024-07-31T14:49:49.409420+0000","last_fullsized":"2024-07-31T14:49:49.409420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:20:25.389010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.387120+0000","last_change":"2024-07-31T14:47:03.051657+0000","last_active":"2024-07-31T14:49:49.387120+0000","last_peered":"2024-07-31T14:49:49.387120+0000","last_clean":"2024-07-31T14:49:49.387120+0000","last_became_active":"2024-07-31T14:47:03.050722+0000","last_became_peered":"2024-07-31T14:47:03.050722+0000","last_unstale":"2024-07-31T14:49:49.387120+0000","last_undegraded":"2024-07-31T14:49:49.387120+0000","last_fullsized":"2024-07-31T14:49:49.387120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:23:19.387470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411567+0000","last_change":"2024-07-31T14:47:03.059654+0000","last_active":"2024-07-31T14:49:49.411567+0000","last_peered":"2024-07-31T14:49:49.411567+0000","last_clean":"2024-07-31T14:49:49.411567+0000","last_became_active":"2024-07-31T14:47:03.056843+0000","last_became_peered":"2024-07-31T14:47:03.056843+0000","last_unstale":"2024-07-31T14:49:49.411567+0000","last_undegraded":"2024-07-31T14:49:49.411567+0000","last_fullsized":"2024-07-31T14:49:49.411567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:42:33.696498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413200+0000","last_change":"2024-07-31T14:47:03.052750+0000","last_active":"2024-07-31T14:49:49.413200+0000","last_peered":"2024-07-31T14:49:49.413200+0000","last_clean":"2024-07-31T14:49:49.413200+0000","last_became_active":"2024-07-31T14:47:03.051744+0000","last_became_peered":"2024-07-31T14:47:03.051744+0000","last_unstale":"2024-07-31T14:49:49.413200+0000","last_undegraded":"2024-07-31T14:49:49.413200+0000","last_fullsized":"2024-07-31T14:49:49.413200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:24:33.202918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.393222+0000","last_change":"2024-07-31T14:47:20.452563+0000","last_active":"2024-07-31T14:49:49.393222+0000","last_peered":"2024-07-31T14:49:49.393222+0000","last_clean":"2024-07-31T14:49:49.393222+0000","last_became_active":"2024-07-31T14:47:20.452271+0000","last_became_peered":"2024-07-31T14:47:20.452271+0000","last_unstale":"2024-07-31T14:49:49.393222+0000","last_undegraded":"2024-07-31T14:49:49.393222+0000","last_fullsized":"2024-07-31T14:49:49.393222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:26:55.070265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.391684+0000","last_change":"2024-07-31T14:47:03.050691+0000","last_active":"2024-07-31T14:49:49.391684+0000","last_peered":"2024-07-31T14:49:49.391684+0000","last_clean":"2024-07-31T14:49:49.391684+0000","last_became_active":"2024-07-31T14:47:03.050320+0000","last_became_peered":"2024-07-31T14:47:03.050320+0000","last_unstale":"2024-07-31T14:49:49.391684+0000","last_undegraded":"2024-07-31T14:49:49.391684+0000","last_fullsized":"2024-07-31T14:49:49.391684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:40:45.767706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400128+0000","last_change":"2024-07-31T14:47:03.048491+0000","last_active":"2024-07-31T14:49:49.400128+0000","last_peered":"2024-07-31T14:49:49.400128+0000","last_clean":"2024-07-31T14:49:49.400128+0000","last_became_active":"2024-07-31T14:47:03.048085+0000","last_became_peered":"2024-07-31T14:47:03.048085+0000","last_unstale":"2024-07-31T14:49:49.400128+0000","last_undegraded":"2024-07-31T14:49:49.400128+0000","last_fullsized":"2024-07-31T14:49:49.400128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:35:01.762365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.395425+0000","last_change":"2024-07-31T14:47:03.055502+0000","last_active":"2024-07-31T14:49:49.395425+0000","last_peered":"2024-07-31T14:49:49.395425+0000","last_clean":"2024-07-31T14:49:49.395425+0000","last_became_active":"2024-07-31T14:47:03.055015+0000","last_became_peered":"2024-07-31T14:47:03.055015+0000","last_unstale":"2024-07-31T14:49:49.395425+0000","last_undegraded":"2024-07-31T14:49:49.395425+0000","last_fullsized":"2024-07-31T14:49:49.395425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:50:39.605148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413933+0000","last_change":"2024-07-31T14:47:03.057808+0000","last_active":"2024-07-31T14:49:49.413933+0000","last_peered":"2024-07-31T14:49:49.413933+0000","last_clean":"2024-07-31T14:49:49.413933+0000","last_became_active":"2024-07-31T14:47:03.057443+0000","last_became_peered":"2024-07-31T14:47:03.057443+0000","last_unstale":"2024-07-31T14:49:49.413933+0000","last_undegraded":"2024-07-31T14:49:49.413933+0000","last_fullsized":"2024-07-31T14:49:49.413933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:34:42.915428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396116+0000","last_change":"2024-07-31T14:47:03.058182+0000","last_active":"2024-07-31T14:49:49.396116+0000","last_peered":"2024-07-31T14:49:49.396116+0000","last_clean":"2024-07-31T14:49:49.396116+0000","last_became_active":"2024-07-31T14:47:03.057802+0000","last_became_peered":"2024-07-31T14:47:03.057802+0000","last_unstale":"2024-07-31T14:49:49.396116+0000","last_undegraded":"2024-07-31T14:49:49.396116+0000","last_fullsized":"2024-07-31T14:49:49.396116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:08:06.608788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413379+0000","last_change":"2024-07-31T14:47:03.058882+0000","last_active":"2024-07-31T14:49:49.413379+0000","last_peered":"2024-07-31T14:49:49.413379+0000","last_clean":"2024-07-31T14:49:49.413379+0000","last_became_active":"2024-07-31T14:47:03.058530+0000","last_became_peered":"2024-07-31T14:47:03.058530+0000","last_unstale":"2024-07-31T14:49:49.413379+0000","last_undegraded":"2024-07-31T14:49:49.413379+0000","last_fullsized":"2024-07-31T14:49:49.413379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:28:03.631962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404760+0000","last_change":"2024-07-31T14:47:03.052028+0000","last_active":"2024-07-31T14:49:49.404760+0000","last_peered":"2024-07-31T14:49:49.404760+0000","last_clean":"2024-07-31T14:49:49.404760+0000","last_became_active":"2024-07-31T14:47:03.051559+0000","last_became_peered":"2024-07-31T14:47:03.051559+0000","last_unstale":"2024-07-31T14:49:49.404760+0000","last_undegraded":"2024-07-31T14:49:49.404760+0000","last_fullsized":"2024-07-31T14:49:49.404760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:16:40.723795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399075+0000","last_change":"2024-07-31T14:47:03.053884+0000","last_active":"2024-07-31T14:49:49.399075+0000","last_peered":"2024-07-31T14:49:49.399075+0000","last_clean":"2024-07-31T14:49:49.399075+0000","last_became_active":"2024-07-31T14:47:03.053521+0000","last_became_peered":"2024-07-31T14:47:03.053521+0000","last_unstale":"2024-07-31T14:49:49.399075+0000","last_undegraded":"2024-07-31T14:49:49.399075+0000","last_fullsized":"2024-07-31T14:49:49.399075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:16:37.702055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.394011+0000","last_change":"2024-07-31T14:47:03.058588+0000","last_active":"2024-07-31T14:49:49.394011+0000","last_peered":"2024-07-31T14:49:49.394011+0000","last_clean":"2024-07-31T14:49:49.394011+0000","last_became_active":"2024-07-31T14:47:03.057820+0000","last_became_peered":"2024-07-31T14:47:03.057820+0000","last_unstale":"2024-07-31T14:49:49.394011+0000","last_undegraded":"2024-07-31T14:49:49.394011+0000","last_fullsized":"2024-07-31T14:49:49.394011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:21.265208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396703+0000","last_change":"2024-07-31T14:47:03.041447+0000","last_active":"2024-07-31T14:49:49.396703+0000","last_peered":"2024-07-31T14:49:49.396703+0000","last_clean":"2024-07-31T14:49:49.396703+0000","last_became_active":"2024-07-31T14:47:03.040935+0000","last_became_peered":"2024-07-31T14:47:03.040935+0000","last_unstale":"2024-07-31T14:49:49.396703+0000","last_undegraded":"2024-07-31T14:49:49.396703+0000","last_fullsized":"2024-07-31T14:49:49.396703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:00:13.624698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396755+0000","last_change":"2024-07-31T14:47:20.452239+0000","last_active":"2024-07-31T14:49:49.396755+0000","last_peered":"2024-07-31T14:49:49.396755+0000","last_clean":"2024-07-31T14:49:49.396755+0000","last_became_active":"2024-07-31T14:47:20.451924+0000","last_became_peered":"2024-07-31T14:47:20.451924+0000","last_unstale":"2024-07-31T14:49:49.396755+0000","last_undegraded":"2024-07-31T14:49:49.396755+0000","last_fullsized":"2024-07-31T14:49:49.396755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:58:24.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403996+0000","last_change":"2024-07-31T14:47:03.056024+0000","last_active":"2024-07-31T14:49:49.403996+0000","last_peered":"2024-07-31T14:49:49.403996+0000","last_clean":"2024-07-31T14:49:49.403996+0000","last_became_active":"2024-07-31T14:47:03.055630+0000","last_became_peered":"2024-07-31T14:47:03.055630+0000","last_unstale":"2024-07-31T14:49:49.403996+0000","last_undegraded":"2024-07-31T14:49:49.403996+0000","last_fullsized":"2024-07-31T14:49:49.403996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:09:11.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.390933+0000","last_change":"2024-07-31T14:47:03.053718+0000","last_active":"2024-07-31T14:49:49.390933+0000","last_peered":"2024-07-31T14:49:49.390933+0000","last_clean":"2024-07-31T14:49:49.390933+0000","last_became_active":"2024-07-31T14:47:03.053361+0000","last_became_peered":"2024-07-31T14:47:03.053361+0000","last_unstale":"2024-07-31T14:49:49.390933+0000","last_undegraded":"2024-07-31T14:49:49.390933+0000","last_fullsized":"2024-07-31T14:49:49.390933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:03:49.220838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411510+0000","last_change":"2024-07-31T14:47:03.056087+0000","last_active":"2024-07-31T14:49:49.411510+0000","last_peered":"2024-07-31T14:49:49.411510+0000","last_clean":"2024-07-31T14:49:49.411510+0000","last_became_active":"2024-07-31T14:47:03.055732+0000","last_became_peered":"2024-07-31T14:47:03.055732+0000","last_unstale":"2024-07-31T14:49:49.411510+0000","last_undegraded":"2024-07-31T14:49:49.411510+0000","last_fullsized":"2024-07-31T14:49:49.411510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:45:01.185666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78506,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1928,"num_read_kb":1051,"num_write":276,"num_write_kb":169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":238038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645376,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":520038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841605,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37020,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708132,"statfs":{"total":95995035648,"available":95957127168,"internally_reserved":0,"allocated":1724416,"data_stored":1176867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005131,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":523756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201360,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":520038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397588,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1118131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561114,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":515696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757343,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1112258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920867,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":519770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117095,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":514872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378733,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":589563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":581419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705782,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":517319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":72057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-31T14:51:14.803 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:51:14.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:14 smithi073 ceph-mon[36714]: pgmap v404: 129 pgs: 129 active+clean; 653 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-07-31T14:51:14.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:14 smithi073 ceph-mon[36714]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:14.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:14 smithi077 ceph-mon[37327]: pgmap v404: 129 pgs: 129 active+clean; 653 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-07-31T14:51:14.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:14 smithi077 ceph-mon[37327]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:15.106 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:15.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:15 smithi067 ceph-mon[32750]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:15.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:15 smithi067 ceph-mon[32750]: from='client.15561 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:15.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:15 smithi073 ceph-mon[36714]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:15.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:15 smithi073 ceph-mon[36714]: from='client.15561 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:15.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:15 smithi077 ceph-mon[37327]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:15.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:15 smithi077 ceph-mon[37327]: from='client.15561 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:16.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.766+0000 7f224698d640 1 -- 172.21.15.67:0/422118394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2240109870 msgr2=0x7f22401100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:16.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.766+0000 7f224698d640 1 --2- 172.21.15.67:0/422118394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2240109870 0x7f22401100a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2238009ca0 tx=0x7f223802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:51:16.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.766+0000 7f224698d640 1 -- 172.21.15.67:0/422118394 shutdown_connections 2024-07-31T14:51:16.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.766+0000 7f224698d640 1 --2- 172.21.15.67:0/422118394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2240109870 0x7f22401100a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:16.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.766+0000 7f224698d640 1 --2- 172.21.15.67:0/422118394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2240108f20 0x7f2240109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:16.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.766+0000 7f224698d640 1 --2- 172.21.15.67:0/422118394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22401035e0 0x7f22401039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:16.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.766+0000 7f224698d640 1 -- 172.21.15.67:0/422118394 >> 172.21.15.67:0/422118394 conn(0x7f22400ff030 msgr2=0x7f2240101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:16.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.766+0000 7f224698d640 1 -- 172.21.15.67:0/422118394 shutdown_connections 2024-07-31T14:51:16.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.766+0000 7f224698d640 1 -- 172.21.15.67:0/422118394 wait complete. 2024-07-31T14:51:16.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.767+0000 7f224698d640 1 Processor -- start 2024-07-31T14:51:16.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.767+0000 7f224698d640 1 -- start start 2024-07-31T14:51:16.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.767+0000 7f224698d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22401035e0 0x7f224019c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:16.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.767+0000 7f224698d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2240108f20 0x7f224019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:16.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.767+0000 7f224698d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2240109870 0x7f22401a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:16.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.767+0000 7f224698d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2240113520 con 0x7f22401035e0 2024-07-31T14:51:16.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.767+0000 7f224698d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f22401133a0 con 0x7f2240108f20 2024-07-31T14:51:16.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.767+0000 7f224698d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f22401136a0 con 0x7f2240109870 2024-07-31T14:51:16.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.767+0000 7f223ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22401035e0 0x7f224019c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:16.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.767+0000 7f223ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22401035e0 0x7f224019c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52764/0 (socket says 172.21.15.67:52764) 2024-07-31T14:51:16.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.767+0000 7f223ffff640 1 -- 172.21.15.67:0/3273506984 learned_addr learned my addr 172.21.15.67:0/3273506984 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:16.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.767+0000 7f2244f2a640 1 --2- 172.21.15.67:0/3273506984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2240109870 0x7f22401a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:16.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.767+0000 7f223f7fe640 1 --2- 172.21.15.67:0/3273506984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2240108f20 0x7f224019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:16.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.768+0000 7f223ffff640 1 -- 172.21.15.67:0/3273506984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2240109870 msgr2=0x7f22401a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:16.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.768+0000 7f223ffff640 1 --2- 172.21.15.67:0/3273506984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2240109870 0x7f22401a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:16.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.768+0000 7f223ffff640 1 -- 172.21.15.67:0/3273506984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2240108f20 msgr2=0x7f224019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:16.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.768+0000 7f223ffff640 1 --2- 172.21.15.67:0/3273506984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2240108f20 0x7f224019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:16.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.768+0000 7f223ffff640 1 -- 172.21.15.67:0/3273506984 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22401a2040 con 0x7f22401035e0 2024-07-31T14:51:16.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.768+0000 7f223ffff640 1 --2- 172.21.15.67:0/3273506984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22401035e0 0x7f224019c0d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f223000e9f0 tx=0x7f223000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:16.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.768+0000 7f223d7fa640 1 -- 172.21.15.67:0/3273506984 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223000cd80 con 0x7f22401035e0 2024-07-31T14:51:16.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.768+0000 7f223d7fa640 1 -- 172.21.15.67:0/3273506984 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22300108e0 con 0x7f22401035e0 2024-07-31T14:51:16.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.768+0000 7f224698d640 1 -- 172.21.15.67:0/3273506984 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22401a2330 con 0x7f22401035e0 2024-07-31T14:51:16.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.768+0000 7f223d7fa640 1 -- 172.21.15.67:0/3273506984 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230018980 con 0x7f22401035e0 2024-07-31T14:51:16.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.768+0000 7f224698d640 1 -- 172.21.15.67:0/3273506984 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22401a8a90 con 0x7f22401035e0 2024-07-31T14:51:16.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.769+0000 7f224698d640 1 -- 172.21.15.67:0/3273506984 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22401039f0 con 0x7f22401035e0 2024-07-31T14:51:16.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.771+0000 7f223d7fa640 1 -- 172.21.15.67:0/3273506984 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f2230010430 con 0x7f22401035e0 2024-07-31T14:51:16.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.771+0000 7f223d7fa640 1 --2- 172.21.15.67:0/3273506984 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f22280b3d30 0x7f22280b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:16.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.772+0000 7f223f7fe640 1 --2- 172.21.15.67:0/3273506984 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f22280b3d30 0x7f22280b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:16.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.772+0000 7f223d7fa640 1 -- 172.21.15.67:0/3273506984 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f22300e49d0 con 0x7f22401035e0 2024-07-31T14:51:16.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.772+0000 7f223f7fe640 1 --2- 172.21.15.67:0/3273506984 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f22280b3d30 0x7f22280b6240 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2240106750 tx=0x7f223400b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:16.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:16.774+0000 7f223d7fa640 1 -- 172.21.15.67:0/3273506984 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f22300ab440 con 0x7f22401035e0 2024-07-31T14:51:16.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:16 smithi073 ceph-mon[36714]: pgmap v405: 129 pgs: 129 active+clean; 656 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-07-31T14:51:16.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:16 smithi077 ceph-mon[37327]: pgmap v405: 129 pgs: 129 active+clean; 656 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-07-31T14:51:17.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:17.009+0000 7f224698d640 1 -- 172.21.15.67:0/3273506984 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f224019d200 con 0x7f22401035e0 2024-07-31T14:51:17.010 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:16 smithi067 ceph-mon[32750]: pgmap v405: 129 pgs: 129 active+clean; 656 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-07-31T14:51:17.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:17.010+0000 7f223d7fa640 1 -- 172.21.15.67:0/3273506984 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11962 (secure 0 0 0) 0x7f22300af990 con 0x7f22401035e0 2024-07-31T14:51:17.011 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:51:17.011 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-31T14:51:08:515297+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:51:07.734542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:51:17.011 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-31T14:51:17.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:17.012+0000 7f224698d640 1 -- 172.21.15.67:0/3273506984 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f22280b3d30 msgr2=0x7f22280b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:17.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:17.012+0000 7f224698d640 1 --2- 172.21.15.67:0/3273506984 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f22280b3d30 0x7f22280b6240 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2240106750 tx=0x7f223400b410 comp rx=0 tx=0).stop 2024-07-31T14:51:17.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:17.012+0000 7f224698d640 1 -- 172.21.15.67:0/3273506984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22401035e0 msgr2=0x7f224019c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:17.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:17.012+0000 7f224698d640 1 --2- 172.21.15.67:0/3273506984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22401035e0 0x7f224019c0d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f223000e9f0 tx=0x7f223000eeb0 comp rx=0 tx=0).stop 2024-07-31T14:51:17.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:17.012+0000 7f224698d640 1 -- 172.21.15.67:0/3273506984 shutdown_connections 2024-07-31T14:51:17.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:17.012+0000 7f224698d640 1 --2- 172.21.15.67:0/3273506984 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f22280b3d30 0x7f22280b6240 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:17.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:17.012+0000 7f224698d640 1 --2- 172.21.15.67:0/3273506984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2240109870 0x7f22401a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:17.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:17.012+0000 7f224698d640 1 --2- 172.21.15.67:0/3273506984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2240108f20 0x7f224019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:17.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:17.012+0000 7f224698d640 1 --2- 172.21.15.67:0/3273506984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22401035e0 0x7f224019c0d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:17.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:17.012+0000 7f224698d640 1 -- 172.21.15.67:0/3273506984 >> 172.21.15.67:0/3273506984 conn(0x7f22400ff030 msgr2=0x7f2240107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:17.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:17.012+0000 7f224698d640 1 -- 172.21.15.67:0/3273506984 shutdown_connections 2024-07-31T14:51:17.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:17.012+0000 7f224698d640 1 -- 172.21.15.67:0/3273506984 wait complete. 2024-07-31T14:51:17.702 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-07-31T14:51:17.888 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:17 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3273506984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:17.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:17 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3273506984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:17.981 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:18.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:17 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3273506984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:18.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:18 smithi067 ceph-mon[32750]: pgmap v406: 129 pgs: 129 active+clean; 656 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-07-31T14:51:18.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:18 smithi073 ceph-mon[36714]: pgmap v406: 129 pgs: 129 active+clean; 656 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-07-31T14:51:18.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:18 smithi077 ceph-mon[37327]: pgmap v406: 129 pgs: 129 active+clean; 656 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-07-31T14:51:19.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.837+0000 7f54b990c640 1 -- 172.21.15.67:0/3161414035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b4109830 msgr2=0x7f54b4110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:19.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.837+0000 7f54b990c640 1 --2- 172.21.15.67:0/3161414035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b4109830 0x7f54b4110060 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009900 tx=0x7f54ac02a410 comp rx=0 tx=0).stop 2024-07-31T14:51:19.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.837+0000 7f54b990c640 1 -- 172.21.15.67:0/3161414035 shutdown_connections 2024-07-31T14:51:19.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.837+0000 7f54b990c640 1 --2- 172.21.15.67:0/3161414035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b4109830 0x7f54b4110060 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:19.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.837+0000 7f54b990c640 1 --2- 172.21.15.67:0/3161414035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54b4108ee0 0x7f54b41092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:19.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.837+0000 7f54b990c640 1 --2- 172.21.15.67:0/3161414035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54b41035f0 0x7f54b4103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:19.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.837+0000 7f54b990c640 1 -- 172.21.15.67:0/3161414035 >> 172.21.15.67:0/3161414035 conn(0x7f54b407b340 msgr2=0x7f54b407b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:19.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.837+0000 7f54b990c640 1 -- 172.21.15.67:0/3161414035 shutdown_connections 2024-07-31T14:51:19.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.837+0000 7f54b990c640 1 -- 172.21.15.67:0/3161414035 wait complete. 2024-07-31T14:51:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.838+0000 7f54b990c640 1 Processor -- start 2024-07-31T14:51:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.838+0000 7f54b990c640 1 -- start start 2024-07-31T14:51:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.838+0000 7f54b990c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54b41035f0 0x7f54b4075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.838+0000 7f54b990c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b4108ee0 0x7f54b4075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.838+0000 7f54b990c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54b4109830 0x7f54b4075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.838+0000 7f54b990c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4113500 con 0x7f54b4108ee0 2024-07-31T14:51:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.838+0000 7f54b990c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4113380 con 0x7f54b41035f0 2024-07-31T14:51:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.838+0000 7f54b990c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4113680 con 0x7f54b4109830 2024-07-31T14:51:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b4108ee0 0x7f54b4075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b4108ee0 0x7f54b4075680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52786/0 (socket says 172.21.15.67:52786) 2024-07-31T14:51:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b27fc640 1 -- 172.21.15.67:0/2665583597 learned_addr learned my addr 172.21.15.67:0/2665583597 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b2ffd640 1 --2- 172.21.15.67:0/2665583597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54b41035f0 0x7f54b4075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b37fe640 1 --2- 172.21.15.67:0/2665583597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54b4109830 0x7f54b4075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b27fc640 1 -- 172.21.15.67:0/2665583597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54b4109830 msgr2=0x7f54b4075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b27fc640 1 --2- 172.21.15.67:0/2665583597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54b4109830 0x7f54b4075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b27fc640 1 -- 172.21.15.67:0/2665583597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54b41035f0 msgr2=0x7f54b4075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b27fc640 1 --2- 172.21.15.67:0/2665583597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54b41035f0 0x7f54b4075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b27fc640 1 -- 172.21.15.67:0/2665583597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b41a81f0 con 0x7f54b4108ee0 2024-07-31T14:51:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b27fc640 1 --2- 172.21.15.67:0/2665583597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b4108ee0 0x7f54b4075680 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f549c00d740 tx=0x7f549c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b890a640 1 -- 172.21.15.67:0/2665583597 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c017070 con 0x7f54b4108ee0 2024-07-31T14:51:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b890a640 1 -- 172.21.15.67:0/2665583597 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f549c00de50 con 0x7f54b4108ee0 2024-07-31T14:51:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b990c640 1 -- 172.21.15.67:0/2665583597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b41a84e0 con 0x7f54b4108ee0 2024-07-31T14:51:19.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b890a640 1 -- 172.21.15.67:0/2665583597 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c00f620 con 0x7f54b4108ee0 2024-07-31T14:51:19.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.839+0000 7f54b990c640 1 -- 172.21.15.67:0/2665583597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b41a8a20 con 0x7f54b4108ee0 2024-07-31T14:51:19.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.840+0000 7f54b990c640 1 -- 172.21.15.67:0/2665583597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54b4103a00 con 0x7f54b4108ee0 2024-07-31T14:51:19.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.842+0000 7f54b890a640 1 -- 172.21.15.67:0/2665583597 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f549c01a070 con 0x7f54b4108ee0 2024-07-31T14:51:19.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.843+0000 7f54b890a640 1 --2- 172.21.15.67:0/2665583597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f54a00b3cb0 0x7f54a00b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:19.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.843+0000 7f54b2ffd640 1 --2- 172.21.15.67:0/2665583597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f54a00b3cb0 0x7f54a00b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:19.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.843+0000 7f54b890a640 1 -- 172.21.15.67:0/2665583597 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f549c0e3c40 con 0x7f54b4108ee0 2024-07-31T14:51:19.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.844+0000 7f54b2ffd640 1 --2- 172.21.15.67:0/2665583597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f54a00b3cb0 0x7f54a00b61c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f54b4076b90 tx=0x7f54a40094a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:19.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:19.846+0000 7f54b890a640 1 -- 172.21.15.67:0/2665583597 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f549c0aad00 con 0x7f54b4108ee0 2024-07-31T14:51:20.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.043+0000 7f54b990c640 1 -- 172.21.15.67:0/2665583597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54b4076440 con 0x7f54b4108ee0 2024-07-31T14:51:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.044+0000 7f54b890a640 1 -- 172.21.15.67:0/2665583597 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24874 (secure 0 0 0) 0x7f54b4076440 con 0x7f54b4108ee0 2024-07-31T14:51:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:51:20.046 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T14:47:20.440064+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T14:51:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.046+0000 7f54b990c640 1 -- 172.21.15.67:0/2665583597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f54a00b3cb0 msgr2=0x7f54a00b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.046+0000 7f54b990c640 1 --2- 172.21.15.67:0/2665583597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f54a00b3cb0 0x7f54a00b61c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f54b4076b90 tx=0x7f54a40094a0 comp rx=0 tx=0).stop 2024-07-31T14:51:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.046+0000 7f54b990c640 1 -- 172.21.15.67:0/2665583597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b4108ee0 msgr2=0x7f54b4075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.046+0000 7f54b990c640 1 --2- 172.21.15.67:0/2665583597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b4108ee0 0x7f54b4075680 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f549c00d740 tx=0x7f549c00dc00 comp rx=0 tx=0).stop 2024-07-31T14:51:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.046+0000 7f54b990c640 1 -- 172.21.15.67:0/2665583597 shutdown_connections 2024-07-31T14:51:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.046+0000 7f54b990c640 1 --2- 172.21.15.67:0/2665583597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f54a00b3cb0 0x7f54a00b61c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.046+0000 7f54b990c640 1 --2- 172.21.15.67:0/2665583597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54b4109830 0x7f54b4075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.046+0000 7f54b990c640 1 --2- 172.21.15.67:0/2665583597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b4108ee0 0x7f54b4075680 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.046+0000 7f54b990c640 1 --2- 172.21.15.67:0/2665583597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54b41035f0 0x7f54b4075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.046+0000 7f54b990c640 1 -- 172.21.15.67:0/2665583597 >> 172.21.15.67:0/2665583597 conn(0x7f54b407b340 msgr2=0x7f54b4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.046+0000 7f54b990c640 1 -- 172.21.15.67:0/2665583597 shutdown_connections 2024-07-31T14:51:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.046+0000 7f54b990c640 1 -- 172.21.15.67:0/2665583597 wait complete. 2024-07-31T14:51:20.853 DEBUG:tasks.fwd_scrub:Yielding 2024-07-31T14:51:20.853 INFO:teuthology.run_tasks:Running task exec... 2024-07-31T14:51:20.865 INFO:teuthology.task.exec:Executing custom commands... 2024-07-31T14:51:20.865 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-07-31T14:51:20.865 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-07-31T14:51:20.866 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-07-31T14:51:20.866 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T14:51:20.866 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T14:51:20.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:20 smithi073 ceph-mon[36714]: pgmap v407: 129 pgs: 129 active+clean; 656 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-07-31T14:51:20.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:20 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2665583597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:51:20.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:20 smithi067 ceph-mon[32750]: pgmap v407: 129 pgs: 129 active+clean; 656 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-07-31T14:51:20.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:20 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2665583597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:51:20.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:20 smithi077 ceph-mon[37327]: pgmap v407: 129 pgs: 129 active+clean; 656 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-07-31T14:51:20.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:20 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2665583597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:51:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.995+0000 7f1ee59e8640 1 -- 172.21.15.67:0/192172761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee0108690 msgr2=0x7f1ee010eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.995+0000 7f1ee59e8640 1 --2- 172.21.15.67:0/192172761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee0108690 0x7f1ee010eec0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009ca0 tx=0x7f1ed802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:51:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.995+0000 7f1ee59e8640 1 -- 172.21.15.67:0/192172761 shutdown_connections 2024-07-31T14:51:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.995+0000 7f1ee59e8640 1 --2- 172.21.15.67:0/192172761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee0108690 0x7f1ee010eec0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.995+0000 7f1ee59e8640 1 --2- 172.21.15.67:0/192172761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ee0107d40 0x7f1ee0108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.995+0000 7f1ee59e8640 1 --2- 172.21.15.67:0/192172761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ee0102410 0x7f1ee0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.995+0000 7f1ee59e8640 1 -- 172.21.15.67:0/192172761 >> 172.21.15.67:0/192172761 conn(0x7f1ee00fde10 msgr2=0x7f1ee0100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.995+0000 7f1ee59e8640 1 -- 172.21.15.67:0/192172761 shutdown_connections 2024-07-31T14:51:20.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.995+0000 7f1ee59e8640 1 -- 172.21.15.67:0/192172761 wait complete. 2024-07-31T14:51:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.996+0000 7f1ee59e8640 1 Processor -- start 2024-07-31T14:51:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.996+0000 7f1ee59e8640 1 -- start start 2024-07-31T14:51:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1ee59e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ee0102410 0x7f1ee0198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1ee59e8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee0107d40 0x7f1ee0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1ee59e8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ee0108690 0x7f1ee019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1ee59e8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee0112340 con 0x7f1ee0107d40 2024-07-31T14:51:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1ee59e8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee01121c0 con 0x7f1ee0108690 2024-07-31T14:51:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1ee59e8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee01124c0 con 0x7f1ee0102410 2024-07-31T14:51:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1ede7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee0107d40 0x7f1ee0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1ede7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee0107d40 0x7f1ee0199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45264/0 (socket says 172.21.15.67:45264) 2024-07-31T14:51:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1ede7fc640 1 -- 172.21.15.67:0/124264078 learned_addr learned my addr 172.21.15.67:0/124264078 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1edeffd640 1 --2- 172.21.15.67:0/124264078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ee0102410 0x7f1ee0198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1edf7fe640 1 --2- 172.21.15.67:0/124264078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ee0108690 0x7f1ee019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1ede7fc640 1 -- 172.21.15.67:0/124264078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ee0102410 msgr2=0x7f1ee0198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1ede7fc640 1 --2- 172.21.15.67:0/124264078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ee0102410 0x7f1ee0198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1ede7fc640 1 -- 172.21.15.67:0/124264078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ee0108690 msgr2=0x7f1ee019e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1ede7fc640 1 --2- 172.21.15.67:0/124264078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ee0108690 0x7f1ee019e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.997+0000 7f1ede7fc640 1 -- 172.21.15.67:0/124264078 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee019ec10 con 0x7f1ee0107d40 2024-07-31T14:51:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.998+0000 7f1ede7fc640 1 --2- 172.21.15.67:0/124264078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee0107d40 0x7f1ee0199280 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800e9f0 tx=0x7f1ec800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.998+0000 7f1ee49e6640 1 -- 172.21.15.67:0/124264078 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec800ccc0 con 0x7f1ee0107d40 2024-07-31T14:51:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.998+0000 7f1ee49e6640 1 -- 172.21.15.67:0/124264078 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec80108e0 con 0x7f1ee0107d40 2024-07-31T14:51:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.998+0000 7f1ee59e8640 1 -- 172.21.15.67:0/124264078 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee019ef00 con 0x7f1ee0107d40 2024-07-31T14:51:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.998+0000 7f1ee49e6640 1 -- 172.21.15.67:0/124264078 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8018860 con 0x7f1ee0107d40 2024-07-31T14:51:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.998+0000 7f1ee59e8640 1 -- 172.21.15.67:0/124264078 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee01a5690 con 0x7f1ee0107d40 2024-07-31T14:51:21.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:20.999+0000 7f1ee59e8640 1 -- 172.21.15.67:0/124264078 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee0102820 con 0x7f1ee0107d40 2024-07-31T14:51:21.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.000+0000 7f1ee49e6640 1 -- 172.21.15.67:0/124264078 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f1ec8010430 con 0x7f1ee0107d40 2024-07-31T14:51:21.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.001+0000 7f1ee49e6640 1 --2- 172.21.15.67:0/124264078 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ecc0b3cb0 0x7f1ecc0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:21.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.002+0000 7f1edeffd640 1 --2- 172.21.15.67:0/124264078 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ecc0b3cb0 0x7f1ecc0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:21.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.002+0000 7f1ee49e6640 1 -- 172.21.15.67:0/124264078 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f1ec80e44e0 con 0x7f1ee0107d40 2024-07-31T14:51:21.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.002+0000 7f1edeffd640 1 --2- 172.21.15.67:0/124264078 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ecc0b3cb0 0x7f1ecc0b61c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0005d00 tx=0x7f1ed000b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:21.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.006+0000 7f1ee49e6640 1 -- 172.21.15.67:0/124264078 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f1ec80ab5a0 con 0x7f1ee0107d40 2024-07-31T14:51:21.150 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:21.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.231+0000 7f1ee59e8640 1 -- 172.21.15.67:0/124264078 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f1ee0063230 con 0x7f1ecc0b3cb0 2024-07-31T14:51:21.400 INFO:teuthology.orchestra.run.smithi067.stdout:Schedule set for path / 2024-07-31T14:51:21.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.399+0000 7f1ee49e6640 1 -- 172.21.15.67:0/124264078 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1ee0063230 con 0x7f1ecc0b3cb0 2024-07-31T14:51:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.401+0000 7f1ee59e8640 1 -- 172.21.15.67:0/124264078 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ecc0b3cb0 msgr2=0x7f1ecc0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.401+0000 7f1ee59e8640 1 --2- 172.21.15.67:0/124264078 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ecc0b3cb0 0x7f1ecc0b61c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0005d00 tx=0x7f1ed000b410 comp rx=0 tx=0).stop 2024-07-31T14:51:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.401+0000 7f1ee59e8640 1 -- 172.21.15.67:0/124264078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee0107d40 msgr2=0x7f1ee0199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.401+0000 7f1ee59e8640 1 --2- 172.21.15.67:0/124264078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee0107d40 0x7f1ee0199280 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800e9f0 tx=0x7f1ec800eeb0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.401+0000 7f1ee59e8640 1 -- 172.21.15.67:0/124264078 shutdown_connections 2024-07-31T14:51:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.401+0000 7f1ee59e8640 1 --2- 172.21.15.67:0/124264078 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ecc0b3cb0 0x7f1ecc0b61c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.401+0000 7f1ee59e8640 1 --2- 172.21.15.67:0/124264078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ee0108690 0x7f1ee019e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.401+0000 7f1ee59e8640 1 --2- 172.21.15.67:0/124264078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee0107d40 0x7f1ee0199280 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.401+0000 7f1ee59e8640 1 --2- 172.21.15.67:0/124264078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ee0102410 0x7f1ee0198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.401+0000 7f1ee59e8640 1 -- 172.21.15.67:0/124264078 >> 172.21.15.67:0/124264078 conn(0x7f1ee00fde10 msgr2=0x7f1ee0107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.401+0000 7f1ee59e8640 1 -- 172.21.15.67:0/124264078 shutdown_connections 2024-07-31T14:51:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.401+0000 7f1ee59e8640 1 -- 172.21.15.67:0/124264078 wait complete. 2024-07-31T14:51:21.414 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-07-31T14:51:21.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.532+0000 7f9a08dbb640 1 -- 172.21.15.67:0/1823478572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a04101790 msgr2=0x7f9a04101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:21.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.532+0000 7f9a08dbb640 1 --2- 172.21.15.67:0/1823478572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a04101790 0x7f9a04101ba0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009900 tx=0x7f99f802a480 comp rx=0 tx=0).stop 2024-07-31T14:51:21.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.532+0000 7f9a08dbb640 1 -- 172.21.15.67:0/1823478572 shutdown_connections 2024-07-31T14:51:21.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.532+0000 7f9a08dbb640 1 --2- 172.21.15.67:0/1823478572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a04102260 0x7f9a04111f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.532+0000 7f9a08dbb640 1 --2- 172.21.15.67:0/1823478572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a04101790 0x7f9a04101ba0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.532+0000 7f9a08dbb640 1 --2- 172.21.15.67:0/1823478572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a04104070 0x7f9a04104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.532+0000 7f9a08dbb640 1 -- 172.21.15.67:0/1823478572 >> 172.21.15.67:0/1823478572 conn(0x7f9a04079710 msgr2=0x7f9a04079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:21.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.532+0000 7f9a08dbb640 1 -- 172.21.15.67:0/1823478572 shutdown_connections 2024-07-31T14:51:21.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.532+0000 7f9a08dbb640 1 -- 172.21.15.67:0/1823478572 wait complete. 2024-07-31T14:51:21.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.533+0000 7f9a08dbb640 1 Processor -- start 2024-07-31T14:51:21.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.533+0000 7f9a08dbb640 1 -- start start 2024-07-31T14:51:21.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.533+0000 7f9a08dbb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a04101790 0x7f9a0419c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:21.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.533+0000 7f9a08dbb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a04102260 0x7f9a0419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:21.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.533+0000 7f9a08dbb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a04104070 0x7f9a041a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:21.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.533+0000 7f9a08dbb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a04115620 con 0x7f9a04101790 2024-07-31T14:51:21.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.533+0000 7f9a08dbb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041154a0 con 0x7f9a04102260 2024-07-31T14:51:21.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.533+0000 7f9a08dbb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041157a0 con 0x7f9a04104070 2024-07-31T14:51:21.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.533+0000 7f9a0259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a04101790 0x7f9a0419c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:21.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f9a0259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a04101790 0x7f9a0419c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45306/0 (socket says 172.21.15.67:45306) 2024-07-31T14:51:21.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f9a0259c640 1 -- 172.21.15.67:0/2959591452 learned_addr learned my addr 172.21.15.67:0/2959591452 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:21.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f9a02d9d640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a04104070 0x7f9a041a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:21.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f9a01d9b640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a04102260 0x7f9a0419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:21.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f9a0259c640 1 -- 172.21.15.67:0/2959591452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a04104070 msgr2=0x7f9a041a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:21.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f9a0259c640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a04104070 0x7f9a041a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f9a0259c640 1 -- 172.21.15.67:0/2959591452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a04102260 msgr2=0x7f9a0419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:21.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f9a0259c640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a04102260 0x7f9a0419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f9a0259c640 1 -- 172.21.15.67:0/2959591452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a041a1fd0 con 0x7f9a04101790 2024-07-31T14:51:21.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f9a01d9b640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a04102260 0x7f9a0419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:51:21.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f9a02d9d640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a04104070 0x7f9a041a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:21.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f9a0259c640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a04101790 0x7f9a0419c0a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f99f400d950 tx=0x7f99f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:21.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f99f37fe640 1 -- 172.21.15.67:0/2959591452 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4017030 con 0x7f9a04101790 2024-07-31T14:51:21.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f99f37fe640 1 -- 172.21.15.67:0/2959591452 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99f4004020 con 0x7f9a04101790 2024-07-31T14:51:21.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f99f37fe640 1 -- 172.21.15.67:0/2959591452 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f40047a0 con 0x7f9a04101790 2024-07-31T14:51:21.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f9a08dbb640 1 -- 172.21.15.67:0/2959591452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a041a22c0 con 0x7f9a04101790 2024-07-31T14:51:21.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.534+0000 7f9a08dbb640 1 -- 172.21.15.67:0/2959591452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a041a8a50 con 0x7f9a04101790 2024-07-31T14:51:21.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.535+0000 7f9a08dbb640 1 -- 172.21.15.67:0/2959591452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a04101ba0 con 0x7f9a04101790 2024-07-31T14:51:21.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.537+0000 7f99f37fe640 1 -- 172.21.15.67:0/2959591452 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f99f4009a30 con 0x7f9a04101790 2024-07-31T14:51:21.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.538+0000 7f99f37fe640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99ec0b3cb0 0x7f99ec0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:21.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.538+0000 7f9a01d9b640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99ec0b3cb0 0x7f99ec0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:21.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.538+0000 7f99f37fe640 1 -- 172.21.15.67:0/2959591452 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f99f40e4310 con 0x7f9a04101790 2024-07-31T14:51:21.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.538+0000 7f9a01d9b640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99ec0b3cb0 0x7f99ec0b61c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f99f80096c0 tx=0x7f99f8031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:21.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.541+0000 7f99f37fe640 1 -- 172.21.15.67:0/2959591452 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f99f40ab3d0 con 0x7f9a04101790 2024-07-31T14:51:21.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.633+0000 7f99f37fe640 1 -- 172.21.15.67:0/2959591452 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f99f40a6f60 con 0x7f9a04101790 2024-07-31T14:51:21.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.769+0000 7f9a08dbb640 1 -- 172.21.15.67:0/2959591452 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f9a04063230 con 0x7f99ec0b3cb0 2024-07-31T14:51:21.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.832+0000 7f99f37fe640 1 -- 172.21.15.67:0/2959591452 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f9a04063230 con 0x7f99ec0b3cb0 2024-07-31T14:51:21.833 INFO:teuthology.orchestra.run.smithi067.stdout:Retention added to path / 2024-07-31T14:51:21.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.834+0000 7f9a08dbb640 1 -- 172.21.15.67:0/2959591452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99ec0b3cb0 msgr2=0x7f99ec0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:21.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.834+0000 7f9a08dbb640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99ec0b3cb0 0x7f99ec0b61c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f99f80096c0 tx=0x7f99f8031400 comp rx=0 tx=0).stop 2024-07-31T14:51:21.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.834+0000 7f9a08dbb640 1 -- 172.21.15.67:0/2959591452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a04101790 msgr2=0x7f9a0419c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.834+0000 7f9a08dbb640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a04101790 0x7f9a0419c0a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f99f400d950 tx=0x7f99f400de10 comp rx=0 tx=0).stop 2024-07-31T14:51:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.834+0000 7f9a08dbb640 1 -- 172.21.15.67:0/2959591452 shutdown_connections 2024-07-31T14:51:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.834+0000 7f9a08dbb640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99ec0b3cb0 0x7f99ec0b61c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.834+0000 7f9a08dbb640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a04104070 0x7f9a041a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.834+0000 7f9a08dbb640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a04102260 0x7f9a0419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.834+0000 7f9a08dbb640 1 --2- 172.21.15.67:0/2959591452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a04101790 0x7f9a0419c0a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.834+0000 7f9a08dbb640 1 -- 172.21.15.67:0/2959591452 >> 172.21.15.67:0/2959591452 conn(0x7f9a04079710 msgr2=0x7f9a040ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.834+0000 7f9a08dbb640 1 -- 172.21.15.67:0/2959591452 shutdown_connections 2024-07-31T14:51:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.834+0000 7f9a08dbb640 1 -- 172.21.15.67:0/2959591452 wait complete. 2024-07-31T14:51:21.846 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-07-31T14:51:21.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.965+0000 7f26b9b40640 1 -- 172.21.15.67:0/3169160169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b4108e30 msgr2=0x7f26b4109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:21.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.965+0000 7f26b9b40640 1 --2- 172.21.15.67:0/3169160169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b4108e30 0x7f26b4109240 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009900 tx=0x7f26a802a410 comp rx=0 tx=0).stop 2024-07-31T14:51:21.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.966+0000 7f26b9b40640 1 -- 172.21.15.67:0/3169160169 shutdown_connections 2024-07-31T14:51:21.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.966+0000 7f26b9b40640 1 --2- 172.21.15.67:0/3169160169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26b4109780 0x7f26b410ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.966+0000 7f26b9b40640 1 --2- 172.21.15.67:0/3169160169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b4108e30 0x7f26b4109240 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.966+0000 7f26b9b40640 1 --2- 172.21.15.67:0/3169160169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26b4103500 0x7f26b4103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.966+0000 7f26b9b40640 1 -- 172.21.15.67:0/3169160169 >> 172.21.15.67:0/3169160169 conn(0x7f26b40fef00 msgr2=0x7f26b4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:21.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.966+0000 7f26b9b40640 1 -- 172.21.15.67:0/3169160169 shutdown_connections 2024-07-31T14:51:21.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.966+0000 7f26b9b40640 1 -- 172.21.15.67:0/3169160169 wait complete. 2024-07-31T14:51:21.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.966+0000 7f26b9b40640 1 Processor -- start 2024-07-31T14:51:21.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.966+0000 7f26b9b40640 1 -- start start 2024-07-31T14:51:21.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b9b40640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26b4103500 0x7f26b4199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:21.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b9b40640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b4108e30 0x7f26b419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:21.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b9b40640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26b4109780 0x7f26b419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:21.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b9b40640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4113410 con 0x7f26b4108e30 2024-07-31T14:51:21.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b9b40640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4113290 con 0x7f26b4103500 2024-07-31T14:51:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b9b40640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4113590 con 0x7f26b4109780 2024-07-31T14:51:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b4108e30 0x7f26b419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b4108e30 0x7f26b419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45328/0 (socket says 172.21.15.67:45328) 2024-07-31T14:51:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b2ffd640 1 -- 172.21.15.67:0/1159788006 learned_addr learned my addr 172.21.15.67:0/1159788006 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b3fff640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26b4109780 0x7f26b419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b37fe640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26b4103500 0x7f26b4199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b2ffd640 1 -- 172.21.15.67:0/1159788006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26b4109780 msgr2=0x7f26b419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b2ffd640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26b4109780 0x7f26b419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b2ffd640 1 -- 172.21.15.67:0/1159788006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26b4103500 msgr2=0x7f26b4199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b2ffd640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26b4103500 0x7f26b4199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b2ffd640 1 -- 172.21.15.67:0/1159788006 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b419fd50 con 0x7f26b4108e30 2024-07-31T14:51:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.967+0000 7f26b3fff640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26b4109780 0x7f26b419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:21.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.968+0000 7f26b2ffd640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b4108e30 0x7f26b419a450 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f26a802a930 tx=0x7f26a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:21.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.968+0000 7f26b0ff9640 1 -- 172.21.15.67:0/1159788006 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a803f030 con 0x7f26b4108e30 2024-07-31T14:51:21.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.968+0000 7f26b0ff9640 1 -- 172.21.15.67:0/1159788006 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a8003aa0 con 0x7f26b4108e30 2024-07-31T14:51:21.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.968+0000 7f26b9b40640 1 -- 172.21.15.67:0/1159788006 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b419ffe0 con 0x7f26b4108e30 2024-07-31T14:51:21.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.968+0000 7f26b0ff9640 1 -- 172.21.15.67:0/1159788006 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8035070 con 0x7f26b4108e30 2024-07-31T14:51:21.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.968+0000 7f26b9b40640 1 -- 172.21.15.67:0/1159788006 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b41a04d0 con 0x7f26b4108e30 2024-07-31T14:51:21.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.969+0000 7f26b9b40640 1 -- 172.21.15.67:0/1159788006 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b4104770 con 0x7f26b4108e30 2024-07-31T14:51:21.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.970+0000 7f26b0ff9640 1 -- 172.21.15.67:0/1159788006 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f26a8042030 con 0x7f26b4108e30 2024-07-31T14:51:21.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.971+0000 7f26b0ff9640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f269c0b3d40 0x7f269c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:21.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.971+0000 7f26b37fe640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f269c0b3d40 0x7f269c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:21.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.971+0000 7f26b0ff9640 1 -- 172.21.15.67:0/1159788006 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f26a8104ce0 con 0x7f26b4108e30 2024-07-31T14:51:21.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.972+0000 7f26b37fe640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f269c0b3d40 0x7f269c0b6250 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005fd0 tx=0x7f26a400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:21.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:21.974+0000 7f26b0ff9640 1 -- 172.21.15.67:0/1159788006 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f26a80cb690 con 0x7f26b4108e30 2024-07-31T14:51:22.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.192+0000 7f26b9b40640 1 -- 172.21.15.67:0/1159788006 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f26b4063230 con 0x7f269c0b3d40 2024-07-31T14:51:22.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.198+0000 7f26b0ff9640 1 -- 172.21.15.67:0/1159788006 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f26b4063230 con 0x7f269c0b3d40 2024-07-31T14:51:22.200 INFO:teuthology.orchestra.run.smithi067.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-07-31T00:00:00", "created": "2024-07-31T14:51:21", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-07-31T14:51:22.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.201+0000 7f26b9b40640 1 -- 172.21.15.67:0/1159788006 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f269c0b3d40 msgr2=0x7f269c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:22.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.201+0000 7f26b9b40640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f269c0b3d40 0x7f269c0b6250 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005fd0 tx=0x7f26a400b390 comp rx=0 tx=0).stop 2024-07-31T14:51:22.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.201+0000 7f26b9b40640 1 -- 172.21.15.67:0/1159788006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b4108e30 msgr2=0x7f26b419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:22.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.201+0000 7f26b9b40640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b4108e30 0x7f26b419a450 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f26a802a930 tx=0x7f26a8004270 comp rx=0 tx=0).stop 2024-07-31T14:51:22.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.201+0000 7f26b9b40640 1 -- 172.21.15.67:0/1159788006 shutdown_connections 2024-07-31T14:51:22.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.201+0000 7f26b9b40640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f269c0b3d40 0x7f269c0b6250 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:22.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.201+0000 7f26b9b40640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26b4109780 0x7f26b419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:22.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.201+0000 7f26b9b40640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b4108e30 0x7f26b419a450 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:22.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.201+0000 7f26b9b40640 1 --2- 172.21.15.67:0/1159788006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26b4103500 0x7f26b4199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:22.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.201+0000 7f26b9b40640 1 -- 172.21.15.67:0/1159788006 >> 172.21.15.67:0/1159788006 conn(0x7f26b40fef00 msgr2=0x7f26b4101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:22.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.201+0000 7f26b9b40640 1 -- 172.21.15.67:0/1159788006 shutdown_connections 2024-07-31T14:51:22.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.201+0000 7f26b9b40640 1 -- 172.21.15.67:0/1159788006 wait complete. 2024-07-31T14:51:22.214 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-07-31T14:51:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.328+0000 7f8c593d4640 1 -- 172.21.15.67:0/3899180139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c54101a50 msgr2=0x7f8c54101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.328+0000 7f8c593d4640 1 --2- 172.21.15.67:0/3899180139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c54101a50 0x7f8c54101e60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009900 tx=0x7f8c4402a410 comp rx=0 tx=0).stop 2024-07-31T14:51:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.328+0000 7f8c593d4640 1 -- 172.21.15.67:0/3899180139 shutdown_connections 2024-07-31T14:51:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.328+0000 7f8c593d4640 1 --2- 172.21.15.67:0/3899180139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c54106580 0x7f8c54110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:22.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.328+0000 7f8c593d4640 1 --2- 172.21.15.67:0/3899180139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c54105c30 0x7f8c54106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:22.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.328+0000 7f8c593d4640 1 --2- 172.21.15.67:0/3899180139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c54101a50 0x7f8c54101e60 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:22.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.328+0000 7f8c593d4640 1 -- 172.21.15.67:0/3899180139 >> 172.21.15.67:0/3899180139 conn(0x7f8c54079880 msgr2=0x7f8c54079c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:22.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.328+0000 7f8c593d4640 1 -- 172.21.15.67:0/3899180139 shutdown_connections 2024-07-31T14:51:22.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.328+0000 7f8c593d4640 1 -- 172.21.15.67:0/3899180139 wait complete. 2024-07-31T14:51:22.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.329+0000 7f8c593d4640 1 Processor -- start 2024-07-31T14:51:22.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.329+0000 7f8c593d4640 1 -- start start 2024-07-31T14:51:22.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.329+0000 7f8c593d4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c54101a50 0x7f8c54199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:22.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.329+0000 7f8c593d4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c54105c30 0x7f8c5419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:22.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.329+0000 7f8c593d4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c54106580 0x7f8c5419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:22.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.329+0000 7f8c593d4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54113460 con 0x7f8c54101a50 2024-07-31T14:51:22.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.329+0000 7f8c593d4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541132e0 con 0x7f8c54106580 2024-07-31T14:51:22.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.329+0000 7f8c593d4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541135e0 con 0x7f8c54105c30 2024-07-31T14:51:22.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.329+0000 7f8c52ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c54101a50 0x7f8c54199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:22.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.329+0000 7f8c52ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c54101a50 0x7f8c54199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45336/0 (socket says 172.21.15.67:45336) 2024-07-31T14:51:22.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.329+0000 7f8c52ffd640 1 -- 172.21.15.67:0/436257351 learned_addr learned my addr 172.21.15.67:0/436257351 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:22.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.330+0000 7f8c537fe640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c54106580 0x7f8c5419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:22.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.330+0000 7f8c527fc640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c54105c30 0x7f8c5419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:22.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.330+0000 7f8c52ffd640 1 -- 172.21.15.67:0/436257351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c54105c30 msgr2=0x7f8c5419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:22.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.330+0000 7f8c52ffd640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c54105c30 0x7f8c5419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:22.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.330+0000 7f8c52ffd640 1 -- 172.21.15.67:0/436257351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c54106580 msgr2=0x7f8c5419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:22.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.330+0000 7f8c52ffd640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c54106580 0x7f8c5419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:22.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.330+0000 7f8c52ffd640 1 -- 172.21.15.67:0/436257351 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5419fe50 con 0x7f8c54101a50 2024-07-31T14:51:22.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.330+0000 7f8c527fc640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c54105c30 0x7f8c5419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:51:22.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.330+0000 7f8c52ffd640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c54101a50 0x7f8c54199f20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8c44006010 tx=0x7f8c44004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:22.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.330+0000 7f8c3bfff640 1 -- 172.21.15.67:0/436257351 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4403f030 con 0x7f8c54101a50 2024-07-31T14:51:22.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.330+0000 7f8c3bfff640 1 -- 172.21.15.67:0/436257351 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c44003aa0 con 0x7f8c54101a50 2024-07-31T14:51:22.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.330+0000 7f8c593d4640 1 -- 172.21.15.67:0/436257351 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c541a00e0 con 0x7f8c54101a50 2024-07-31T14:51:22.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.330+0000 7f8c3bfff640 1 -- 172.21.15.67:0/436257351 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44035070 con 0x7f8c54101a50 2024-07-31T14:51:22.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.330+0000 7f8c593d4640 1 -- 172.21.15.67:0/436257351 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c541a68d0 con 0x7f8c54101a50 2024-07-31T14:51:22.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.332+0000 7f8c593d4640 1 -- 172.21.15.67:0/436257351 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c54101e60 con 0x7f8c54101a50 2024-07-31T14:51:22.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.334+0000 7f8c3bfff640 1 -- 172.21.15.67:0/436257351 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f8c44042030 con 0x7f8c54101a50 2024-07-31T14:51:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.335+0000 7f8c3bfff640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c3c0b3d40 0x7f8c3c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.335+0000 7f8c527fc640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c3c0b3d40 0x7f8c3c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.335+0000 7f8c3bfff640 1 -- 172.21.15.67:0/436257351 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f8c44104bf0 con 0x7f8c54101a50 2024-07-31T14:51:22.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.336+0000 7f8c527fc640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c3c0b3d40 0x7f8c3c0b6250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c5406aa50 tx=0x7f8c48005d30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:22.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.338+0000 7f8c3bfff640 1 -- 172.21.15.67:0/436257351 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f8c440cbbf0 con 0x7f8c54101a50 2024-07-31T14:51:22.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.581+0000 7f8c593d4640 1 -- 172.21.15.67:0/436257351 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f8c5410b6a0 con 0x7f8c3c0b3d40 2024-07-31T14:51:22.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.588+0000 7f8c3bfff640 1 -- 172.21.15.67:0/436257351 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f8c5410b6a0 con 0x7f8c3c0b3d40 2024-07-31T14:51:22.589 INFO:teuthology.orchestra.run.smithi067.stdout:/ 1m 6m3h 2024-07-31T14:51:22.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.590+0000 7f8c593d4640 1 -- 172.21.15.67:0/436257351 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c3c0b3d40 msgr2=0x7f8c3c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:22.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.590+0000 7f8c593d4640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c3c0b3d40 0x7f8c3c0b6250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c5406aa50 tx=0x7f8c48005d30 comp rx=0 tx=0).stop 2024-07-31T14:51:22.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.590+0000 7f8c593d4640 1 -- 172.21.15.67:0/436257351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c54101a50 msgr2=0x7f8c54199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:22.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.590+0000 7f8c593d4640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c54101a50 0x7f8c54199f20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8c44006010 tx=0x7f8c44004060 comp rx=0 tx=0).stop 2024-07-31T14:51:22.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.590+0000 7f8c593d4640 1 -- 172.21.15.67:0/436257351 shutdown_connections 2024-07-31T14:51:22.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.590+0000 7f8c593d4640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8c3c0b3d40 0x7f8c3c0b6250 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:22.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.590+0000 7f8c593d4640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c54106580 0x7f8c5419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:22.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.590+0000 7f8c593d4640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c54105c30 0x7f8c5419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:22.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.590+0000 7f8c593d4640 1 --2- 172.21.15.67:0/436257351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c54101a50 0x7f8c54199f20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:22.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.590+0000 7f8c593d4640 1 -- 172.21.15.67:0/436257351 >> 172.21.15.67:0/436257351 conn(0x7f8c54079880 msgr2=0x7f8c54106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:22.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.591+0000 7f8c593d4640 1 -- 172.21.15.67:0/436257351 shutdown_connections 2024-07-31T14:51:22.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:22.591+0000 7f8c593d4640 1 -- 172.21.15.67:0/436257351 wait complete. 2024-07-31T14:51:22.603 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-07-31T14:51:22.632 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-07-31T14:51:22.642 INFO:teuthology.run_tasks:Running task background_exec... 2024-07-31T14:51:22.689 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-07-31T14:51:22.689 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-07-31T14:51:22.692 INFO:teuthology.run_tasks:Running task quiescer... 2024-07-31T14:51:22.744 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-31T14:51:22.744 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph pg dump --format=json 2024-07-31T14:51:22.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:22 smithi073 ceph-mon[36714]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:22.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:22 smithi073 ceph-mon[36714]: pgmap v408: 129 pgs: 129 active+clean; 656 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-07-31T14:51:22.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:22 smithi073 ceph-mon[36714]: mgrmap e25: x(active, since 11m), standbys: z, y 2024-07-31T14:51:22.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:22 smithi073 ceph-mon[36714]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:22.930 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:22 smithi067 ceph-mon[32750]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:22.930 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:22 smithi067 ceph-mon[32750]: pgmap v408: 129 pgs: 129 active+clean; 656 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-07-31T14:51:22.930 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:22 smithi067 ceph-mon[32750]: mgrmap e25: x(active, since 11m), standbys: z, y 2024-07-31T14:51:22.930 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:22 smithi067 ceph-mon[32750]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:22.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:22 smithi077 ceph-mon[37327]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:22.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:22 smithi077 ceph-mon[37327]: pgmap v408: 129 pgs: 129 active+clean; 656 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-07-31T14:51:22.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:22 smithi077 ceph-mon[37327]: mgrmap e25: x(active, since 11m), standbys: z, y 2024-07-31T14:51:22.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:22 smithi077 ceph-mon[37327]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:23.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.029+0000 7efe1436c640 1 -- 172.21.15.67:0/1378668309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe0c108740 msgr2=0x7efe0c10ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:23.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.029+0000 7efe1436c640 1 --2- 172.21.15.67:0/1378668309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe0c108740 0x7efe0c10ef70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efe04009ca0 tx=0x7efe0402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.029+0000 7efe1436c640 1 -- 172.21.15.67:0/1378668309 shutdown_connections 2024-07-31T14:51:23.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.029+0000 7efe1436c640 1 --2- 172.21.15.67:0/1378668309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe0c108740 0x7efe0c10ef70 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.029+0000 7efe1436c640 1 --2- 172.21.15.67:0/1378668309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe0c107df0 0x7efe0c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.029+0000 7efe1436c640 1 --2- 172.21.15.67:0/1378668309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c1024b0 0x7efe0c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.029+0000 7efe1436c640 1 -- 172.21.15.67:0/1378668309 >> 172.21.15.67:0/1378668309 conn(0x7efe0c079170 msgr2=0x7efe0c079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:23.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.029+0000 7efe1436c640 1 -- 172.21.15.67:0/1378668309 shutdown_connections 2024-07-31T14:51:23.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.029+0000 7efe1436c640 1 -- 172.21.15.67:0/1378668309 wait complete. 2024-07-31T14:51:23.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.030+0000 7efe1436c640 1 Processor -- start 2024-07-31T14:51:23.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.030+0000 7efe1436c640 1 -- start start 2024-07-31T14:51:23.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.030+0000 7efe1436c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c1024b0 0x7efe0c199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.030+0000 7efe1436c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe0c107df0 0x7efe0c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.030+0000 7efe1436c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe0c108740 0x7efe0c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.030+0000 7efe1436c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c1134d0 con 0x7efe0c107df0 2024-07-31T14:51:23.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.030+0000 7efe1436c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c113350 con 0x7efe0c1024b0 2024-07-31T14:51:23.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.030+0000 7efe1436c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c113650 con 0x7efe0c108740 2024-07-31T14:51:23.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.030+0000 7efe11907640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe0c107df0 0x7efe0c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.031+0000 7efe11907640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe0c107df0 0x7efe0c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45350/0 (socket says 172.21.15.67:45350) 2024-07-31T14:51:23.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.031+0000 7efe11907640 1 -- 172.21.15.67:0/3291640114 learned_addr learned my addr 172.21.15.67:0/3291640114 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:23.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.031+0000 7efe12108640 1 --2- 172.21.15.67:0/3291640114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c1024b0 0x7efe0c199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.031+0000 7efe12909640 1 --2- 172.21.15.67:0/3291640114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe0c108740 0x7efe0c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.031+0000 7efe11907640 1 -- 172.21.15.67:0/3291640114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe0c108740 msgr2=0x7efe0c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:23.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.031+0000 7efe11907640 1 --2- 172.21.15.67:0/3291640114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe0c108740 0x7efe0c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.031+0000 7efe11907640 1 -- 172.21.15.67:0/3291640114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c1024b0 msgr2=0x7efe0c199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:23.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.031+0000 7efe11907640 1 --2- 172.21.15.67:0/3291640114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c1024b0 0x7efe0c199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.031+0000 7efe11907640 1 -- 172.21.15.67:0/3291640114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0c19fcf0 con 0x7efe0c107df0 2024-07-31T14:51:23.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.031+0000 7efe11907640 1 --2- 172.21.15.67:0/3291640114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe0c107df0 0x7efe0c19a3e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00c970 tx=0x7efdfc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:23.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.031+0000 7efe037fe640 1 -- 172.21.15.67:0/3291640114 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc014070 con 0x7efe0c107df0 2024-07-31T14:51:23.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.031+0000 7efe037fe640 1 -- 172.21.15.67:0/3291640114 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdfc004440 con 0x7efe0c107df0 2024-07-31T14:51:23.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.031+0000 7efe1436c640 1 -- 172.21.15.67:0/3291640114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0c19ffe0 con 0x7efe0c107df0 2024-07-31T14:51:23.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.031+0000 7efe037fe640 1 -- 172.21.15.67:0/3291640114 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc003e50 con 0x7efe0c107df0 2024-07-31T14:51:23.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.032+0000 7efe1436c640 1 -- 172.21.15.67:0/3291640114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe0c1a6800 con 0x7efe0c107df0 2024-07-31T14:51:23.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.035+0000 7efe037fe640 1 -- 172.21.15.67:0/3291640114 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7efdfc017070 con 0x7efe0c107df0 2024-07-31T14:51:23.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.035+0000 7efe1a8bd740 1 Processor -- start 2024-07-31T14:51:23.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.036+0000 7efe1a8bd740 1 -- start start 2024-07-31T14:51:23.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.036+0000 7efe037fe640 1 --2- 172.21.15.67:0/3291640114 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efdf80b3f10 0x7efdf80b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.036+0000 7efe1a8bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c9f350 0x5638a70443b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.036+0000 7efe1a8bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a70448f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.036+0000 7efe1a8bd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c7e670 0x5638a7044e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.036+0000 7efe1a8bd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5638a6f2c2f0 con 0x5638a6c7e670 2024-07-31T14:51:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.036+0000 7efe1a8bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5638a6fbaa80 con 0x5638a6c76490 2024-07-31T14:51:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.036+0000 7efe1a8bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5638a6fa9f80 con 0x5638a6c9f350 2024-07-31T14:51:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.036+0000 7efe12909640 1 --2- 172.21.15.67:0/3291640114 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efdf80b3f10 0x7efdf80b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.036+0000 7efe037fe640 1 -- 172.21.15.67:0/3291640114 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7efdfc0e3b00 con 0x7efe0c107df0 2024-07-31T14:51:23.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.036+0000 7efe12909640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c7e670 0x5638a7044e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.036+0000 7efe12108640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c9f350 0x5638a70443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.036+0000 7efe12909640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c7e670 0x5638a7044e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45354/0 (socket says 172.21.15.67:45354) 2024-07-31T14:51:23.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.037+0000 7efe12909640 1 -- 172.21.15.67:0/2056786158 learned_addr learned my addr 172.21.15.67:0/2056786158 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:23.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.037+0000 7efe11907640 1 --2- 172.21.15.67:0/2056786158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a70448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.037+0000 7efe11907640 1 -- 172.21.15.67:0/2056786158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c9f350 msgr2=0x5638a70443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:23.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.037+0000 7efe11907640 1 --2- 172.21.15.67:0/2056786158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c9f350 0x5638a70443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.037+0000 7efe11907640 1 -- 172.21.15.67:0/2056786158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c7e670 msgr2=0x5638a7044e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:23.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.037+0000 7efe11907640 1 --2- 172.21.15.67:0/2056786158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c7e670 0x5638a7044e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.037+0000 7efe11907640 1 -- 172.21.15.67:0/2056786158 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638a6d0a930 con 0x5638a6c76490 2024-07-31T14:51:23.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.037+0000 7efe12108640 1 --2- 172.21.15.67:0/2056786158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c9f350 0x5638a70443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:23.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.037+0000 7efe12909640 1 --2- 172.21.15.67:0/2056786158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c7e670 0x5638a7044e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:23.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.037+0000 7efe11907640 1 --2- 172.21.15.67:0/2056786158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a70448f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0e0f40 tx=0x7efdfc0a9f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a625d66d63bc606 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:23.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.037+0000 7efe12909640 1 --2- 172.21.15.67:0/3291640114 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efdf80b3f10 0x7efdf80b6420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efe04035000 tx=0x7efe04005c30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:23.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.037+0000 7efe00d99640 1 -- 172.21.15.67:0/2056786158 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0aa400 con 0x5638a6c76490 2024-07-31T14:51:23.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.038+0000 7efe00d99640 1 -- 172.21.15.67:0/2056786158 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdfc0aea00 con 0x5638a6c76490 2024-07-31T14:51:23.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.038+0000 7efe00d99640 1 -- 172.21.15.67:0/2056786158 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0e11c0 con 0x5638a6c76490 2024-07-31T14:51:23.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.038+0000 7efe1a8bd740 1 -- 172.21.15.67:0/2056786158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 msgr2=0x5638a70448f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:23.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.038+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/2056786158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a70448f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0e0f40 tx=0x7efdfc0a9f10 comp rx=0 tx=0).stop 2024-07-31T14:51:23.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.039+0000 7efe1a8bd740 1 -- 172.21.15.67:0/2056786158 shutdown_connections 2024-07-31T14:51:23.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.039+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/2056786158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c7e670 0x5638a7044e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.039+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/2056786158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a70448f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.039+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/2056786158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c9f350 0x5638a70443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.039+0000 7efe1a8bd740 1 -- 172.21.15.67:0/2056786158 >> 172.21.15.67:0/2056786158 conn(0x5638a6c93740 msgr2=0x5638a70422a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:23.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.039+0000 7efe1a8bd740 1 -- 172.21.15.67:0/2056786158 shutdown_connections 2024-07-31T14:51:23.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.039+0000 7efe1a8bd740 1 -- 172.21.15.67:0/2056786158 wait complete. 2024-07-31T14:51:23.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.039+0000 7efe1a8bd740 1 -- start start 2024-07-31T14:51:23.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.039+0000 7efe1a8bd740 1 Processor -- start 2024-07-31T14:51:23.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe1a8bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a704ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe1a8bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c7e670 0x5638a7050520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe1a8bd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c9f350 0x5638a7060110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe1a8bd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5638a6f4c4f0 con 0x5638a6c9f350 2024-07-31T14:51:23.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe1a8bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5638a6fadfc0 con 0x5638a6c76490 2024-07-31T14:51:23.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe1a8bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5638a6e0d890 con 0x5638a6c7e670 2024-07-31T14:51:23.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe12909640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c9f350 0x5638a7060110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe12108640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a704ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe12909640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c9f350 0x5638a7060110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45366/0 (socket says 172.21.15.67:45366) 2024-07-31T14:51:23.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe12909640 1 -- 172.21.15.67:0/3470073155 learned_addr learned my addr 172.21.15.67:0/3470073155 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:23.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe1a8bd740 10 client.0 _ll_register_callbacks cb 0x5638a6f56500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:23.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe11907640 1 --2- 172.21.15.67:0/3470073155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c7e670 0x5638a7050520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe12108640 1 -- 172.21.15.67:0/3470073155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c7e670 msgr2=0x5638a7050520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:23.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe12108640 1 --2- 172.21.15.67:0/3470073155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c7e670 0x5638a7050520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe12108640 1 -- 172.21.15.67:0/3470073155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c9f350 msgr2=0x5638a7060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:23.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe12108640 1 --2- 172.21.15.67:0/3470073155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c9f350 0x5638a7060110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe12108640 1 -- 172.21.15.67:0/3470073155 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638a6f5bc50 con 0x5638a6c76490 2024-07-31T14:51:23.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe11907640 1 --2- 172.21.15.67:0/3470073155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c7e670 0x5638a7050520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:23.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.040+0000 7efe12909640 1 --2- 172.21.15.67:0/3470073155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c9f350 0x5638a7060110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:23.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.041+0000 7efe12108640 1 --2- 172.21.15.67:0/3470073155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a704ffa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efe08012700 tx=0x7efe08012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aad97df6ae591b6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:23.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.041+0000 7efdf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:51:23.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.041+0000 7efdf5ffb640 1 -- 172.21.15.67:0/3470073155 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08004030 con 0x5638a6c76490 2024-07-31T14:51:23.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.041+0000 7efdf5ffb640 1 -- 172.21.15.67:0/3470073155 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7efe080041d0 con 0x5638a6c76490 2024-07-31T14:51:23.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.041+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3470073155 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdfc0aea00 con 0x5638a6c76490 2024-07-31T14:51:23.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.041+0000 7efdf5ffb640 1 -- 172.21.15.67:0/3470073155 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe08025050 con 0x5638a6c76490 2024-07-31T14:51:23.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.041+0000 7efe1a8bd740 10 client.25319 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:23.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.041+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3470073155 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638a7062ea0 con 0x5638a6c76490 2024-07-31T14:51:23.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.042+0000 7efdf5ffb640 1 -- 172.21.15.67:0/3470073155 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdfc0aea00 con 0x5638a6c76490 2024-07-31T14:51:23.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.043+0000 7efdf5ffb640 1 -- 172.21.15.67:0/3470073155 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efe08003a40 con 0x5638a6c76490 2024-07-31T14:51:23.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.043+0000 7efe1a8bd740 10 client.25319 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:23.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.043+0000 7efe1a8bd740 10 client.25319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:51:23.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.043+0000 7efe1a8bd740 20 client.25319 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:23.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.043+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/3470073155 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5638a6c5dcd0 0x5638a7066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.043+0000 7efe1a8bd740 4 client.25319 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:51:23.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.043+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3470073155 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638a6fbc9d0 con 0x5638a6c5dcd0 2024-07-31T14:51:23.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.043+0000 7efe11907640 1 --2- 172.21.15.67:0/3470073155 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5638a6c5dcd0 0x5638a7066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.044+0000 7efe11907640 1 --2- 172.21.15.67:0/3470073155 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5638a6c5dcd0 0x5638a7066450 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e15ac87598d5cbc2 server_cookie=c89b8762933fe83c in_seq=0 out_seq=0 2024-07-31T14:51:23.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.044+0000 7efdf5ffb640 10 client.25319 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:23.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.044+0000 7efdf5ffb640 1 -- 172.21.15.67:0/3470073155 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638a6fbc9d0 con 0x5638a6c5dcd0 2024-07-31T14:51:23.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.044+0000 7efdf5ffb640 10 client.25319 handle_command_reply: tid=0 2024-07-31T14:51:23.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.044+0000 7efe1a8bd740 1 client.25319 shutdown 2024-07-31T14:51:23.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.044+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3470073155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 msgr2=0x5638a704ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:23.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.044+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/3470073155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a704ffa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efe08012700 tx=0x7efe08012bc0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.045+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3470073155 shutdown_connections 2024-07-31T14:51:23.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.045+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/3470073155 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5638a6c5dcd0 0x5638a7066450 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.045+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/3470073155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c9f350 0x5638a7060110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.045+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/3470073155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c7e670 0x5638a7050520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.045+0000 7efdf5ffb640 0 client.25319 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:23.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.045+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/3470073155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a704ffa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.045+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3470073155 >> 172.21.15.67:0/3470073155 conn(0x5638a6c93740 msgr2=0x5638a7045da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:23.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.045+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3470073155 shutdown_connections 2024-07-31T14:51:23.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.045+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3470073155 wait complete. 2024-07-31T14:51:23.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.045+0000 7efe1a8bd740 20 client.25319 trim_cache size 0 max 16384 2024-07-31T14:51:23.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.065+0000 7efe1a8bd740 1 Processor -- start 2024-07-31T14:51:23.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.065+0000 7efe1a8bd740 1 -- start start 2024-07-31T14:51:23.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.065+0000 7efe1a8bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a705ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.065+0000 7efe1a8bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c7e670 0x5638a70519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.065+0000 7efe1a8bd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c9f350 0x5638a7049630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.065+0000 7efe1a8bd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c1ae420 con 0x5638a6c9f350 2024-07-31T14:51:23.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.065+0000 7efe1a8bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c1aff60 con 0x5638a6c76490 2024-07-31T14:51:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.065+0000 7efe1a8bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c1aeaf0 con 0x5638a6c7e670 2024-07-31T14:51:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.066+0000 7efe12909640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c9f350 0x5638a7049630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.066+0000 7efe12909640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c9f350 0x5638a7049630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45382/0 (socket says 172.21.15.67:45382) 2024-07-31T14:51:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.066+0000 7efe12909640 1 -- 172.21.15.67:0/2270522676 learned_addr learned my addr 172.21.15.67:0/2270522676 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.066+0000 7efe11907640 1 --2- 172.21.15.67:0/2270522676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c7e670 0x5638a70519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.066+0000 7efe12108640 1 --2- 172.21.15.67:0/2270522676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a705ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.066+0000 7efe12909640 1 -- 172.21.15.67:0/2270522676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c7e670 msgr2=0x5638a70519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.066+0000 7efe12909640 1 --2- 172.21.15.67:0/2270522676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c7e670 0x5638a70519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.066+0000 7efe12909640 1 -- 172.21.15.67:0/2270522676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 msgr2=0x5638a705ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.066+0000 7efe12909640 1 --2- 172.21.15.67:0/2270522676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a705ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.066+0000 7efe12909640 1 -- 172.21.15.67:0/2270522676 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638a704ed00 con 0x5638a6c9f350 2024-07-31T14:51:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.066+0000 7efe11907640 1 --2- 172.21.15.67:0/2270522676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c7e670 0x5638a70519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.066+0000 7efe12909640 1 --2- 172.21.15.67:0/2270522676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c9f350 0x5638a7049630 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efe04009360 tx=0x7efe04043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4333f80a84f45e11 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:23.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.066+0000 7efdf7fff640 1 -- 172.21.15.67:0/2270522676 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04051070 con 0x5638a6c9f350 2024-07-31T14:51:23.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.067+0000 7efdf7fff640 1 -- 172.21.15.67:0/2270522676 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe04036040 con 0x5638a6c9f350 2024-07-31T14:51:23.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.067+0000 7efdf7fff640 1 -- 172.21.15.67:0/2270522676 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04047040 con 0x5638a6c9f350 2024-07-31T14:51:23.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.067+0000 7efe1a8bd740 1 -- 172.21.15.67:0/2270522676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c9f350 msgr2=0x5638a7049630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:23.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.067+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/2270522676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c9f350 0x5638a7049630 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efe04009360 tx=0x7efe04043000 comp rx=0 tx=0).stop 2024-07-31T14:51:23.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.068+0000 7efe1a8bd740 1 -- 172.21.15.67:0/2270522676 shutdown_connections 2024-07-31T14:51:23.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.068+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/2270522676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c9f350 0x5638a7049630 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.068+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/2270522676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c7e670 0x5638a70519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.068+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/2270522676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a705ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.068+0000 7efe1a8bd740 1 -- 172.21.15.67:0/2270522676 >> 172.21.15.67:0/2270522676 conn(0x5638a6c93740 msgr2=0x5638a70512c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:23.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.068+0000 7efe1a8bd740 1 -- 172.21.15.67:0/2270522676 shutdown_connections 2024-07-31T14:51:23.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.068+0000 7efe1a8bd740 1 -- 172.21.15.67:0/2270522676 wait complete. 2024-07-31T14:51:23.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.068+0000 7efe1a8bd740 1 -- start start 2024-07-31T14:51:23.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.068+0000 7efe1a8bd740 1 Processor -- start 2024-07-31T14:51:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe1a8bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a705c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe1a8bd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c7e670 0x5638a705c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe1a8bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c9f350 0x5638a7057030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe1a8bd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c1af890 con 0x5638a6c7e670 2024-07-31T14:51:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe1a8bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c1af1c0 con 0x5638a6c76490 2024-07-31T14:51:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe1a8bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c1ac400 con 0x5638a6c9f350 2024-07-31T14:51:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe11907640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c7e670 0x5638a705c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe11907640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c7e670 0x5638a705c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45396/0 (socket says 172.21.15.67:45396) 2024-07-31T14:51:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe11907640 1 -- 172.21.15.67:0/3996191727 learned_addr learned my addr 172.21.15.67:0/3996191727 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe1a8bd740 10 client.0 _ll_register_callbacks cb 0x7efe08009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe12909640 1 --2- 172.21.15.67:0/3996191727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c9f350 0x5638a7057030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe12108640 1 --2- 172.21.15.67:0/3996191727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a705c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe11907640 1 -- 172.21.15.67:0/3996191727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c9f350 msgr2=0x5638a7057030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe11907640 1 --2- 172.21.15.67:0/3996191727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c9f350 0x5638a7057030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe11907640 1 -- 172.21.15.67:0/3996191727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 msgr2=0x5638a705c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe11907640 1 --2- 172.21.15.67:0/3996191727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a705c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.069+0000 7efe11907640 1 -- 172.21.15.67:0/3996191727 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638a7057570 con 0x5638a6c7e670 2024-07-31T14:51:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.070+0000 7efe11907640 1 --2- 172.21.15.67:0/3996191727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c7e670 0x5638a705c5c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0e0ff0 tx=0x7efdfc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d673171ce2200c4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.070+0000 7efdf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:51:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.070+0000 7efdf5ffb640 1 -- 172.21.15.67:0/3996191727 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0e13d0 con 0x5638a6c7e670 2024-07-31T14:51:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.070+0000 7efdf5ffb640 1 -- 172.21.15.67:0/3996191727 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7efdfc095260 con 0x5638a6c7e670 2024-07-31T14:51:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.070+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3996191727 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe04036040 con 0x5638a6c7e670 2024-07-31T14:51:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.070+0000 7efdf5ffb640 1 -- 172.21.15.67:0/3996191727 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efdfc0aea00 con 0x5638a6c7e670 2024-07-31T14:51:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.070+0000 7efe1a8bd740 10 client.15621 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.070+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3996191727 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638a70cf3e0 con 0x5638a6c7e670 2024-07-31T14:51:23.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.070+0000 7efdf5ffb640 1 -- 172.21.15.67:0/3996191727 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe04036040 con 0x5638a6c7e670 2024-07-31T14:51:23.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.071+0000 7efdf5ffb640 1 -- 172.21.15.67:0/3996191727 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efdfc0a4c70 con 0x5638a6c7e670 2024-07-31T14:51:23.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.071+0000 7efe1a8bd740 10 client.15621 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:23.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.071+0000 7efe1a8bd740 10 client.15621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:51:23.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.071+0000 7efe1a8bd740 20 client.15621 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:23.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.071+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/3996191727 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5638a6c33080 0x5638a70d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:23.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.071+0000 7efe1a8bd740 4 client.15621 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T14:51:23.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.071+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3996191727 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5638a6fb8470 con 0x5638a6c33080 2024-07-31T14:51:23.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.071+0000 7efe12108640 1 --2- 172.21.15.67:0/3996191727 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5638a6c33080 0x5638a70d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:23.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.071+0000 7efdf5ffb640 10 client.15621 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:23.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.071+0000 7efe12108640 1 --2- 172.21.15.67:0/3996191727 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5638a6c33080 0x5638a70d1620 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b31cb27575b540f server_cookie=bd60538588984488 in_seq=0 out_seq=0 2024-07-31T14:51:23.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.072+0000 7efdf5ffb640 1 -- 172.21.15.67:0/3996191727 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5638a6fb8470 con 0x5638a6c33080 2024-07-31T14:51:23.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.072+0000 7efdf5ffb640 10 client.15621 handle_command_reply: tid=0 2024-07-31T14:51:23.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.072+0000 7efe1a8bd740 1 client.15621 shutdown 2024-07-31T14:51:23.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.073+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3996191727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c7e670 msgr2=0x5638a705c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:23.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.073+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/3996191727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c7e670 0x5638a705c5c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0e0ff0 tx=0x7efdfc000bc0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.073+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3996191727 shutdown_connections 2024-07-31T14:51:23.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.073+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/3996191727 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5638a6c33080 0x5638a70d1620 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.073+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/3996191727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638a6c9f350 0x5638a7057030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.073+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/3996191727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5638a6c7e670 0x5638a705c5c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.073+0000 7efe1a8bd740 1 --2- 172.21.15.67:0/3996191727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638a6c76490 0x5638a705c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.073+0000 7efdf5ffb640 0 client.15621 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:23.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.073+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3996191727 >> 172.21.15.67:0/3996191727 conn(0x5638a6c93740 msgr2=0x5638a70453e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:23.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.073+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3996191727 shutdown_connections 2024-07-31T14:51:23.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.073+0000 7efe1a8bd740 1 -- 172.21.15.67:0/3996191727 wait complete. 2024-07-31T14:51:23.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.073+0000 7efe1a8bd740 20 client.15621 trim_cache size 0 max 16384 2024-07-31T14:51:23.077 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:51:23.078 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T14:51:23.078 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "114ad17e-0968-446c-932a-ee7bc3eeb28f", 2024-07-31T14:51:23.078 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T14:51:23.078 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:51:23.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.074+0000 7efe1436c640 1 -- 172.21.15.67:0/3291640114 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efdf80b3f10 msgr2=0x7efdf80b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:23.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.074+0000 7efe1436c640 1 --2- 172.21.15.67:0/3291640114 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efdf80b3f10 0x7efdf80b6420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efe04035000 tx=0x7efe04005c30 comp rx=0 tx=0).stop 2024-07-31T14:51:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.074+0000 7efe1436c640 1 -- 172.21.15.67:0/3291640114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe0c107df0 msgr2=0x7efe0c19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.074+0000 7efe1436c640 1 --2- 172.21.15.67:0/3291640114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe0c107df0 0x7efe0c19a3e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00c970 tx=0x7efdfc00ce30 comp rx=0 tx=0).stop 2024-07-31T14:51:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.074+0000 7efe1436c640 1 -- 172.21.15.67:0/3291640114 shutdown_connections 2024-07-31T14:51:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.074+0000 7efe1436c640 1 --2- 172.21.15.67:0/3291640114 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efdf80b3f10 0x7efdf80b6420 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.074+0000 7efe1436c640 1 --2- 172.21.15.67:0/3291640114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe0c108740 0x7efe0c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.074+0000 7efe1436c640 1 --2- 172.21.15.67:0/3291640114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe0c107df0 0x7efe0c19a3e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.074+0000 7efe1436c640 1 --2- 172.21.15.67:0/3291640114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c1024b0 0x7efe0c199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.074+0000 7efe1436c640 1 -- 172.21.15.67:0/3291640114 >> 172.21.15.67:0/3291640114 conn(0x7efe0c079170 msgr2=0x7efe0c103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.074+0000 7efe1436c640 1 -- 172.21.15.67:0/3291640114 shutdown_connections 2024-07-31T14:51:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:23.074+0000 7efe1436c640 1 -- 172.21.15.67:0/3291640114 wait complete. 2024-07-31T14:51:23.327 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:23.880 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:23 smithi067 ceph-mon[32750]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:23.880 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:23 smithi067 ceph-mon[32750]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:23.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:51:23.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:23 smithi073 ceph-mon[36714]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:23.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:23 smithi073 ceph-mon[36714]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:23.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:23 smithi077 ceph-mon[37327]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:23.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:23 smithi077 ceph-mon[37327]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T14:51:24.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:24 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T14:51:24.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:51:24.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:51:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:51:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:24 smithi073 ceph-mon[36714]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 164 KiB/s wr, 42 op/s 2024-07-31T14:51:24.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:24 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T14:51:24.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:51:24.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:51:24.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:51:24.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:24 smithi077 ceph-mon[37327]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 164 KiB/s wr, 42 op/s 2024-07-31T14:51:24.994 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:24 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T14:51:24.994 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:51:24.994 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:51:24.995 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:51:24.995 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:24 smithi067 ceph-mon[32750]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 164 KiB/s wr, 42 op/s 2024-07-31T14:51:25.048 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:25.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.727+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3370433189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0ac102490 msgr2=0x7fb0ac1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:25.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.727+0000 7fb0b2a06640 1 --2- 172.21.15.67:0/3370433189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0ac102490 0x7fb0ac1028a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009900 tx=0x7fb09c02a410 comp rx=0 tx=0).stop 2024-07-31T14:51:25.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.727+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3370433189 shutdown_connections 2024-07-31T14:51:25.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.727+0000 7fb0b2a06640 1 --2- 172.21.15.67:0/3370433189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0ac108720 0x7fb0ac10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:25.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.727+0000 7fb0b2a06640 1 --2- 172.21.15.67:0/3370433189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ac107dd0 0x7fb0ac1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:25.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.727+0000 7fb0b2a06640 1 --2- 172.21.15.67:0/3370433189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0ac102490 0x7fb0ac1028a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:25.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.727+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3370433189 >> 172.21.15.67:0/3370433189 conn(0x7fb0ac079160 msgr2=0x7fb0ac079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:25.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.727+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3370433189 shutdown_connections 2024-07-31T14:51:25.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.727+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3370433189 wait complete. 2024-07-31T14:51:25.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.728+0000 7fb0b2a06640 1 Processor -- start 2024-07-31T14:51:25.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.728+0000 7fb0b2a06640 1 -- start start 2024-07-31T14:51:25.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.728+0000 7fb0b2a06640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0ac102490 0x7fb0ac198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:25.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.728+0000 7fb0b2a06640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ac107dd0 0x7fb0ac199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:25.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.728+0000 7fb0b2a06640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0ac108720 0x7fb0ac199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:25.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.728+0000 7fb0b2a06640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1123a0 con 0x7fb0ac108720 2024-07-31T14:51:25.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.728+0000 7fb0b2a06640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac112220 con 0x7fb0ac107dd0 2024-07-31T14:51:25.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.728+0000 7fb0b2a06640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac112520 con 0x7fb0ac102490 2024-07-31T14:51:25.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.728+0000 7fb0b0fa3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0ac108720 0x7fb0ac199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:25.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.728+0000 7fb0b0fa3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0ac108720 0x7fb0ac199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45416/0 (socket says 172.21.15.67:45416) 2024-07-31T14:51:25.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.729+0000 7fb0b0fa3640 1 -- 172.21.15.67:0/3005445842 learned_addr learned my addr 172.21.15.67:0/3005445842 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:25.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.729+0000 7fb0abfff640 1 --2- 172.21.15.67:0/3005445842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0ac102490 0x7fb0ac198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:25.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.729+0000 7fb0a3fff640 1 --2- 172.21.15.67:0/3005445842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ac107dd0 0x7fb0ac199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:25.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.729+0000 7fb0b0fa3640 1 -- 172.21.15.67:0/3005445842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0ac102490 msgr2=0x7fb0ac198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:25.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.729+0000 7fb0b0fa3640 1 --2- 172.21.15.67:0/3005445842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0ac102490 0x7fb0ac198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:25.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.729+0000 7fb0b0fa3640 1 -- 172.21.15.67:0/3005445842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ac107dd0 msgr2=0x7fb0ac199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:25.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.729+0000 7fb0b0fa3640 1 --2- 172.21.15.67:0/3005445842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ac107dd0 0x7fb0ac199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:25.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.729+0000 7fb0b0fa3640 1 -- 172.21.15.67:0/3005445842 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac1a8170 con 0x7fb0ac108720 2024-07-31T14:51:25.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.729+0000 7fb0b0fa3640 1 --2- 172.21.15.67:0/3005445842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0ac108720 0x7fb0ac199540 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400eaa0 tx=0x7fb0a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:25.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.729+0000 7fb0a9ffb640 1 -- 172.21.15.67:0/3005445842 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400cdb0 con 0x7fb0ac108720 2024-07-31T14:51:25.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.729+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3005445842 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ac1a8460 con 0x7fb0ac108720 2024-07-31T14:51:25.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.729+0000 7fb0a9ffb640 1 -- 172.21.15.67:0/3005445842 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a4005020 con 0x7fb0ac108720 2024-07-31T14:51:25.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.729+0000 7fb0a9ffb640 1 -- 172.21.15.67:0/3005445842 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40116d0 con 0x7fb0ac108720 2024-07-31T14:51:25.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.729+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3005445842 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ac1a89a0 con 0x7fb0ac108720 2024-07-31T14:51:25.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.730+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3005445842 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0ac1028a0 con 0x7fb0ac108720 2024-07-31T14:51:25.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.732+0000 7fb0a9ffb640 1 -- 172.21.15.67:0/3005445842 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7fb0a40054a0 con 0x7fb0ac108720 2024-07-31T14:51:25.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.733+0000 7fb0a9ffb640 1 --2- 172.21.15.67:0/3005445842 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb0980b3d40 0x7fb0980b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:25.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.733+0000 7fb0abfff640 1 --2- 172.21.15.67:0/3005445842 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb0980b3d40 0x7fb0980b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:25.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.733+0000 7fb0a9ffb640 1 -- 172.21.15.67:0/3005445842 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fb0a40e43f0 con 0x7fb0ac108720 2024-07-31T14:51:25.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.733+0000 7fb0abfff640 1 --2- 172.21.15.67:0/3005445842 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb0980b3d40 0x7fb0980b6250 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb09c000c00 tx=0x7fb09c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:25.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.737+0000 7fb0a9ffb640 1 -- 172.21.15.67:0/3005445842 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb0a4016030 con 0x7fb0ac108720 2024-07-31T14:51:25.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.930+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3005445842 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb0ac063230 con 0x7fb0980b3d40 2024-07-31T14:51:25.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.936+0000 7fb0a9ffb640 1 -- 172.21.15.67:0/3005445842 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340589 (secure 0 0 0) 0x7fb0ac063230 con 0x7fb0980b3d40 2024-07-31T14:51:25.940 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:51:25.942 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-31T14:51:25.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.939+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3005445842 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb0980b3d40 msgr2=0x7fb0980b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:25.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.939+0000 7fb0b2a06640 1 --2- 172.21.15.67:0/3005445842 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb0980b3d40 0x7fb0980b6250 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb09c000c00 tx=0x7fb09c031400 comp rx=0 tx=0).stop 2024-07-31T14:51:25.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.939+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3005445842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0ac108720 msgr2=0x7fb0ac199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:25.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.939+0000 7fb0b2a06640 1 --2- 172.21.15.67:0/3005445842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0ac108720 0x7fb0ac199540 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400eaa0 tx=0x7fb0a400ef60 comp rx=0 tx=0).stop 2024-07-31T14:51:25.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.939+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3005445842 shutdown_connections 2024-07-31T14:51:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.939+0000 7fb0b2a06640 1 --2- 172.21.15.67:0/3005445842 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb0980b3d40 0x7fb0980b6250 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.939+0000 7fb0b2a06640 1 --2- 172.21.15.67:0/3005445842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0ac108720 0x7fb0ac199540 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.939+0000 7fb0b2a06640 1 --2- 172.21.15.67:0/3005445842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ac107dd0 0x7fb0ac199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.939+0000 7fb0b2a06640 1 --2- 172.21.15.67:0/3005445842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0ac102490 0x7fb0ac198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.939+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3005445842 >> 172.21.15.67:0/3005445842 conn(0x7fb0ac079160 msgr2=0x7fb0ac100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.939+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3005445842 shutdown_connections 2024-07-31T14:51:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:25.939+0000 7fb0b2a06640 1 -- 172.21.15.67:0/3005445842 wait complete. 2024-07-31T14:51:26.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:26 smithi073 ceph-mon[36714]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 164 KiB/s wr, 41 op/s 2024-07-31T14:51:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:26 smithi073 ceph-mon[36714]: from='client.15627 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:26.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:26 smithi077 ceph-mon[37327]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 164 KiB/s wr, 41 op/s 2024-07-31T14:51:26.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:26 smithi077 ceph-mon[37327]: from='client.15627 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:27.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:26 smithi067 ceph-mon[32750]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 164 KiB/s wr, 41 op/s 2024-07-31T14:51:27.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:26 smithi067 ceph-mon[32750]: from='client.15627 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:27.441 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-07-31T14:51:25.447200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1458620,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2341,"num_read_kb":1372,"num_write":572,"num_write_kb":4284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":946,"ondisk_log_size":946,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386232,"kb_used_data":16100,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555592,"statfs":{"total":1151940427776,"available":1151544926208,"internally_reserved":0,"allocated":16486400,"data_stored":10188014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":378713403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":789512,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":332,"num_read_kb":252,"num_write":171,"num_write_kb":1963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003834"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":255,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:19.905520+0000","last_change":"2024-07-31T14:47:00.020067+0000","last_active":"2024-07-31T14:51:19.905520+0000","last_peered":"2024-07-31T14:51:19.905520+0000","last_clean":"2024-07-31T14:51:19.905520+0000","last_became_active":"2024-07-31T14:47:00.019674+0000","last_became_peered":"2024-07-31T14:47:00.019674+0000","last_unstale":"2024-07-31T14:51:19.905520+0000","last_undegraded":"2024-07-31T14:51:19.905520+0000","last_fullsized":"2024-07-31T14:51:19.905520+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:10:08.193786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":199,"num_read_kb":199,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.408739+0000","last_change":"2024-07-31T14:47:03.052069+0000","last_active":"2024-07-31T14:49:49.408739+0000","last_peered":"2024-07-31T14:49:49.408739+0000","last_clean":"2024-07-31T14:49:49.408739+0000","last_became_active":"2024-07-31T14:47:03.051739+0000","last_became_peered":"2024-07-31T14:47:03.051739+0000","last_unstale":"2024-07-31T14:49:49.408739+0000","last_undegraded":"2024-07-31T14:49:49.408739+0000","last_fullsized":"2024-07-31T14:49:49.408739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:29:19.952794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.629384+0000","last_change":"2024-07-31T14:47:20.628770+0000","last_active":"2024-07-31T14:47:20.629384+0000","last_peered":"2024-07-31T14:47:20.629384+0000","last_clean":"2024-07-31T14:47:20.629384+0000","last_became_active":"2024-07-31T14:47:20.628142+0000","last_became_peered":"2024-07-31T14:47:20.628142+0000","last_unstale":"2024-07-31T14:47:20.629384+0000","last_undegraded":"2024-07-31T14:47:20.629384+0000","last_fullsized":"2024-07-31T14:47:20.629384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.659670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:59.291402+0000","last_change":"2024-07-31T14:47:00.183893+0000","last_active":"2024-07-31T14:48:59.291402+0000","last_peered":"2024-07-31T14:48:59.291402+0000","last_clean":"2024-07-31T14:48:59.291402+0000","last_became_active":"2024-07-31T14:47:00.183453+0000","last_became_peered":"2024-07-31T14:47:00.183453+0000","last_unstale":"2024-07-31T14:48:59.291402+0000","last_undegraded":"2024-07-31T14:48:59.291402+0000","last_fullsized":"2024-07-31T14:48:59.291402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:45:08.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404383+0000","last_change":"2024-07-31T14:47:03.057081+0000","last_active":"2024-07-31T14:49:49.404383+0000","last_peered":"2024-07-31T14:49:49.404383+0000","last_clean":"2024-07-31T14:49:49.404383+0000","last_became_active":"2024-07-31T14:47:03.056061+0000","last_became_peered":"2024-07-31T14:47:03.056061+0000","last_unstale":"2024-07-31T14:49:49.404383+0000","last_undegraded":"2024-07-31T14:49:49.404383+0000","last_fullsized":"2024-07-31T14:49:49.404383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:28:16.760131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604121+0000","last_change":"2024-07-31T14:47:13.105271+0000","last_active":"2024-07-31T14:47:20.604121+0000","last_peered":"2024-07-31T14:47:20.604121+0000","last_clean":"2024-07-31T14:47:20.604121+0000","last_became_active":"2024-07-31T14:47:13.104717+0000","last_became_peered":"2024-07-31T14:47:13.104717+0000","last_unstale":"2024-07-31T14:47:20.604121+0000","last_undegraded":"2024-07-31T14:47:20.604121+0000","last_fullsized":"2024-07-31T14:47:20.604121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:34:21.413957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:04.594438+0000","last_change":"2024-07-31T14:47:20.454745+0000","last_active":"2024-07-31T14:48:04.594438+0000","last_peered":"2024-07-31T14:48:04.594438+0000","last_clean":"2024-07-31T14:48:04.594438+0000","last_became_active":"2024-07-31T14:47:20.449892+0000","last_became_peered":"2024-07-31T14:47:20.449892+0000","last_unstale":"2024-07-31T14:48:04.594438+0000","last_undegraded":"2024-07-31T14:48:04.594438+0000","last_fullsized":"2024-07-31T14:48:04.594438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:59:17.375344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.412662+0000","last_change":"2024-07-31T14:47:03.051085+0000","last_active":"2024-07-31T14:49:49.412662+0000","last_peered":"2024-07-31T14:49:49.412662+0000","last_clean":"2024-07-31T14:49:49.412662+0000","last_became_active":"2024-07-31T14:47:03.050686+0000","last_became_peered":"2024-07-31T14:47:03.050686+0000","last_unstale":"2024-07-31T14:49:49.412662+0000","last_undegraded":"2024-07-31T14:49:49.412662+0000","last_fullsized":"2024-07-31T14:49:49.412662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:48:02.494061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604501+0000","last_change":"2024-07-31T14:47:13.104229+0000","last_active":"2024-07-31T14:47:20.604501+0000","last_peered":"2024-07-31T14:47:20.604501+0000","last_clean":"2024-07-31T14:47:20.604501+0000","last_became_active":"2024-07-31T14:47:13.103737+0000","last_became_peered":"2024-07-31T14:47:13.103737+0000","last_unstale":"2024-07-31T14:47:20.604501+0000","last_undegraded":"2024-07-31T14:47:20.604501+0000","last_fullsized":"2024-07-31T14:47:20.604501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:36:00.541345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609858+0000","last_change":"2024-07-31T14:47:00.028169+0000","last_active":"2024-07-31T14:48:03.609858+0000","last_peered":"2024-07-31T14:48:03.609858+0000","last_clean":"2024-07-31T14:48:03.609858+0000","last_became_active":"2024-07-31T14:47:00.027757+0000","last_became_peered":"2024-07-31T14:47:00.027757+0000","last_unstale":"2024-07-31T14:48:03.609858+0000","last_undegraded":"2024-07-31T14:48:03.609858+0000","last_fullsized":"2024-07-31T14:48:03.609858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:36:04.419551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413724+0000","last_change":"2024-07-31T14:47:03.057400+0000","last_active":"2024-07-31T14:49:49.413724+0000","last_peered":"2024-07-31T14:49:49.413724+0000","last_clean":"2024-07-31T14:49:49.413724+0000","last_became_active":"2024-07-31T14:47:03.057035+0000","last_became_peered":"2024-07-31T14:47:03.057035+0000","last_unstale":"2024-07-31T14:49:49.413724+0000","last_undegraded":"2024-07-31T14:49:49.413724+0000","last_fullsized":"2024-07-31T14:49:49.413724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:55:07.951076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604244+0000","last_change":"2024-07-31T14:47:13.106093+0000","last_active":"2024-07-31T14:47:20.604244+0000","last_peered":"2024-07-31T14:47:20.604244+0000","last_clean":"2024-07-31T14:47:20.604244+0000","last_became_active":"2024-07-31T14:47:13.105459+0000","last_became_peered":"2024-07-31T14:47:13.105459+0000","last_unstale":"2024-07-31T14:47:20.604244+0000","last_undegraded":"2024-07-31T14:47:20.604244+0000","last_fullsized":"2024-07-31T14:47:20.604244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:07:52.751245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414377+0000","last_change":"2024-07-31T14:47:03.058701+0000","last_active":"2024-07-31T14:49:49.414377+0000","last_peered":"2024-07-31T14:49:49.414377+0000","last_clean":"2024-07-31T14:49:49.414377+0000","last_became_active":"2024-07-31T14:47:03.058359+0000","last_became_peered":"2024-07-31T14:47:03.058359+0000","last_unstale":"2024-07-31T14:49:49.414377+0000","last_undegraded":"2024-07-31T14:49:49.414377+0000","last_fullsized":"2024-07-31T14:49:49.414377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:37:36.149884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605126+0000","last_change":"2024-07-31T14:47:00.022983+0000","last_active":"2024-07-31T14:48:03.605126+0000","last_peered":"2024-07-31T14:48:03.605126+0000","last_clean":"2024-07-31T14:48:03.605126+0000","last_became_active":"2024-07-31T14:47:00.022586+0000","last_became_peered":"2024-07-31T14:47:00.022586+0000","last_unstale":"2024-07-31T14:48:03.605126+0000","last_undegraded":"2024-07-31T14:48:03.605126+0000","last_fullsized":"2024-07-31T14:48:03.605126+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:29:42.641490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604474+0000","last_change":"2024-07-31T14:47:13.363148+0000","last_active":"2024-07-31T14:47:20.604474+0000","last_peered":"2024-07-31T14:47:20.604474+0000","last_clean":"2024-07-31T14:47:20.604474+0000","last_became_active":"2024-07-31T14:47:13.362577+0000","last_became_peered":"2024-07-31T14:47:13.362577+0000","last_unstale":"2024-07-31T14:47:20.604474+0000","last_undegraded":"2024-07-31T14:47:20.604474+0000","last_fullsized":"2024-07-31T14:47:20.604474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:37:59.003260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409563+0000","last_change":"2024-07-31T14:47:03.049586+0000","last_active":"2024-07-31T14:49:49.409563+0000","last_peered":"2024-07-31T14:49:49.409563+0000","last_clean":"2024-07-31T14:49:49.409563+0000","last_became_active":"2024-07-31T14:47:03.049154+0000","last_became_peered":"2024-07-31T14:47:03.049154+0000","last_unstale":"2024-07-31T14:49:49.409563+0000","last_undegraded":"2024-07-31T14:49:49.409563+0000","last_fullsized":"2024-07-31T14:49:49.409563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:51:01.971854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:09.610387+0000","last_change":"2024-07-31T14:47:00.021254+0000","last_active":"2024-07-31T14:51:09.610387+0000","last_peered":"2024-07-31T14:51:09.610387+0000","last_clean":"2024-07-31T14:51:09.610387+0000","last_became_active":"2024-07-31T14:47:00.020690+0000","last_became_peered":"2024-07-31T14:47:00.020690+0000","last_unstale":"2024-07-31T14:51:09.610387+0000","last_undegraded":"2024-07-31T14:51:09.610387+0000","last_fullsized":"2024-07-31T14:51:09.610387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:01:27.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:18.046730+0000","last_change":"2024-07-31T14:47:13.102220+0000","last_active":"2024-07-31T14:48:18.046730+0000","last_peered":"2024-07-31T14:48:18.046730+0000","last_clean":"2024-07-31T14:48:18.046730+0000","last_became_active":"2024-07-31T14:47:13.101764+0000","last_became_peered":"2024-07-31T14:47:13.101764+0000","last_unstale":"2024-07-31T14:48:18.046730+0000","last_undegraded":"2024-07-31T14:48:18.046730+0000","last_fullsized":"2024-07-31T14:48:18.046730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:56:44.335340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.398408+0000","last_change":"2024-07-31T14:47:03.052348+0000","last_active":"2024-07-31T14:49:49.398408+0000","last_peered":"2024-07-31T14:49:49.398408+0000","last_clean":"2024-07-31T14:49:49.398408+0000","last_became_active":"2024-07-31T14:47:03.051961+0000","last_became_peered":"2024-07-31T14:47:03.051961+0000","last_unstale":"2024-07-31T14:49:49.398408+0000","last_undegraded":"2024-07-31T14:49:49.398408+0000","last_fullsized":"2024-07-31T14:49:49.398408+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:19:35.279809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:09.292622+0000","last_change":"2024-07-31T14:47:00.025013+0000","last_active":"2024-07-31T14:49:09.292622+0000","last_peered":"2024-07-31T14:49:09.292622+0000","last_clean":"2024-07-31T14:49:09.292622+0000","last_became_active":"2024-07-31T14:47:00.024637+0000","last_became_peered":"2024-07-31T14:47:00.024637+0000","last_unstale":"2024-07-31T14:49:09.292622+0000","last_undegraded":"2024-07-31T14:49:09.292622+0000","last_fullsized":"2024-07-31T14:49:09.292622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:07:21.519549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630339+0000","last_change":"2024-07-31T14:47:20.629262+0000","last_active":"2024-07-31T14:47:20.630339+0000","last_peered":"2024-07-31T14:47:20.630339+0000","last_clean":"2024-07-31T14:47:20.630339+0000","last_became_active":"2024-07-31T14:47:20.628318+0000","last_became_peered":"2024-07-31T14:47:20.628318+0000","last_unstale":"2024-07-31T14:47:20.630339+0000","last_undegraded":"2024-07-31T14:47:20.630339+0000","last_fullsized":"2024-07-31T14:47:20.630339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.206705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.387553+0000","last_change":"2024-07-31T14:47:03.051527+0000","last_active":"2024-07-31T14:49:49.387553+0000","last_peered":"2024-07-31T14:49:49.387553+0000","last_clean":"2024-07-31T14:49:49.387553+0000","last_became_active":"2024-07-31T14:47:03.051139+0000","last_became_peered":"2024-07-31T14:47:03.051139+0000","last_unstale":"2024-07-31T14:49:49.387553+0000","last_undegraded":"2024-07-31T14:49:49.387553+0000","last_fullsized":"2024-07-31T14:49:49.387553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:53:11.962719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:09.210377+0000","last_change":"2024-07-31T14:47:00.183812+0000","last_active":"2024-07-31T14:49:09.210377+0000","last_peered":"2024-07-31T14:49:09.210377+0000","last_clean":"2024-07-31T14:49:09.210377+0000","last_became_active":"2024-07-31T14:47:00.183440+0000","last_became_peered":"2024-07-31T14:47:00.183440+0000","last_unstale":"2024-07-31T14:49:09.210377+0000","last_undegraded":"2024-07-31T14:49:09.210377+0000","last_fullsized":"2024-07-31T14:49:09.210377+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:47:23.166756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604338+0000","last_change":"2024-07-31T14:47:13.363563+0000","last_active":"2024-07-31T14:47:20.604338+0000","last_peered":"2024-07-31T14:47:20.604338+0000","last_clean":"2024-07-31T14:47:20.604338+0000","last_became_active":"2024-07-31T14:47:13.362697+0000","last_became_peered":"2024-07-31T14:47:13.362697+0000","last_unstale":"2024-07-31T14:47:20.604338+0000","last_undegraded":"2024-07-31T14:47:20.604338+0000","last_fullsized":"2024-07-31T14:47:20.604338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:26:38.292073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407720+0000","last_change":"2024-07-31T14:47:03.046763+0000","last_active":"2024-07-31T14:49:49.407720+0000","last_peered":"2024-07-31T14:49:49.407720+0000","last_clean":"2024-07-31T14:49:49.407720+0000","last_became_active":"2024-07-31T14:47:03.046408+0000","last_became_peered":"2024-07-31T14:47:03.046408+0000","last_unstale":"2024-07-31T14:49:49.407720+0000","last_undegraded":"2024-07-31T14:49:49.407720+0000","last_fullsized":"2024-07-31T14:49:49.407720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:43:22.838495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:09.380966+0000","last_change":"2024-07-31T14:47:00.236075+0000","last_active":"2024-07-31T14:49:09.380966+0000","last_peered":"2024-07-31T14:49:09.380966+0000","last_clean":"2024-07-31T14:49:09.380966+0000","last_became_active":"2024-07-31T14:47:00.235758+0000","last_became_peered":"2024-07-31T14:47:00.235758+0000","last_unstale":"2024-07-31T14:49:09.380966+0000","last_undegraded":"2024-07-31T14:49:09.380966+0000","last_fullsized":"2024-07-31T14:49:09.380966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:50.575648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.448218+0000","last_change":"2024-07-31T14:47:13.101876+0000","last_active":"2024-07-31T14:47:20.448218+0000","last_peered":"2024-07-31T14:47:20.448218+0000","last_clean":"2024-07-31T14:47:20.448218+0000","last_became_active":"2024-07-31T14:47:13.101276+0000","last_became_peered":"2024-07-31T14:47:13.101276+0000","last_unstale":"2024-07-31T14:47:20.448218+0000","last_undegraded":"2024-07-31T14:47:20.448218+0000","last_fullsized":"2024-07-31T14:47:20.448218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:53:24.684212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409565+0000","last_change":"2024-07-31T14:47:03.052478+0000","last_active":"2024-07-31T14:49:49.409565+0000","last_peered":"2024-07-31T14:49:49.409565+0000","last_clean":"2024-07-31T14:49:49.409565+0000","last_became_active":"2024-07-31T14:47:03.051877+0000","last_became_peered":"2024-07-31T14:47:03.051877+0000","last_unstale":"2024-07-31T14:49:49.409565+0000","last_undegraded":"2024-07-31T14:49:49.409565+0000","last_fullsized":"2024-07-31T14:49:49.409565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:30:32.666490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'73","reported_seq":165,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:21.399348+0000","last_change":"2024-07-31T14:47:00.020076+0000","last_active":"2024-07-31T14:51:21.399348+0000","last_peered":"2024-07-31T14:51:21.399348+0000","last_clean":"2024-07-31T14:51:21.399348+0000","last_became_active":"2024-07-31T14:47:00.019708+0000","last_became_peered":"2024-07-31T14:47:00.019708+0000","last_unstale":"2024-07-31T14:51:21.399348+0000","last_undegraded":"2024-07-31T14:51:21.399348+0000","last_fullsized":"2024-07-31T14:51:21.399348+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:46.520997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":172,"num_read_kb":151,"num_write":132,"num_write_kb":1095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:14.293424+0000","last_change":"2024-07-31T14:47:13.362961+0000","last_active":"2024-07-31T14:49:14.293424+0000","last_peered":"2024-07-31T14:49:14.293424+0000","last_clean":"2024-07-31T14:49:14.293424+0000","last_became_active":"2024-07-31T14:47:13.362502+0000","last_became_peered":"2024-07-31T14:47:13.362502+0000","last_unstale":"2024-07-31T14:49:14.293424+0000","last_undegraded":"2024-07-31T14:49:14.293424+0000","last_fullsized":"2024-07-31T14:49:14.293424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:51:45.732477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399783+0000","last_change":"2024-07-31T14:47:03.039418+0000","last_active":"2024-07-31T14:49:49.399783+0000","last_peered":"2024-07-31T14:49:49.399783+0000","last_clean":"2024-07-31T14:49:49.399783+0000","last_became_active":"2024-07-31T14:47:03.039044+0000","last_became_peered":"2024-07-31T14:47:03.039044+0000","last_unstale":"2024-07-31T14:49:49.399783+0000","last_undegraded":"2024-07-31T14:49:49.399783+0000","last_fullsized":"2024-07-31T14:49:49.399783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:40:10.380080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:07.059440+0000","last_change":"2024-07-31T14:47:00.183778+0000","last_active":"2024-07-31T14:49:07.059440+0000","last_peered":"2024-07-31T14:49:07.059440+0000","last_clean":"2024-07-31T14:49:07.059440+0000","last_became_active":"2024-07-31T14:47:00.183370+0000","last_became_peered":"2024-07-31T14:47:00.183370+0000","last_unstale":"2024-07-31T14:49:07.059440+0000","last_undegraded":"2024-07-31T14:49:07.059440+0000","last_fullsized":"2024-07-31T14:49:07.059440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:45:27.422216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:19.188833+0000","last_change":"2024-07-31T14:47:13.102583+0000","last_active":"2024-07-31T14:49:19.188833+0000","last_peered":"2024-07-31T14:49:19.188833+0000","last_clean":"2024-07-31T14:49:19.188833+0000","last_became_active":"2024-07-31T14:47:13.102058+0000","last_became_peered":"2024-07-31T14:47:13.102058+0000","last_unstale":"2024-07-31T14:49:19.188833+0000","last_undegraded":"2024-07-31T14:49:19.188833+0000","last_fullsized":"2024-07-31T14:49:19.188833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:17.396173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.408095+0000","last_change":"2024-07-31T14:47:03.048334+0000","last_active":"2024-07-31T14:49:49.408095+0000","last_peered":"2024-07-31T14:49:49.408095+0000","last_clean":"2024-07-31T14:49:49.408095+0000","last_became_active":"2024-07-31T14:47:03.047932+0000","last_became_peered":"2024-07-31T14:47:03.047932+0000","last_unstale":"2024-07-31T14:49:49.408095+0000","last_undegraded":"2024-07-31T14:49:49.408095+0000","last_fullsized":"2024-07-31T14:49:49.408095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:14:31.381061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:06.494830+0000","last_change":"2024-07-31T14:47:20.450851+0000","last_active":"2024-07-31T14:49:06.494830+0000","last_peered":"2024-07-31T14:49:06.494830+0000","last_clean":"2024-07-31T14:49:06.494830+0000","last_became_active":"2024-07-31T14:47:20.450435+0000","last_became_peered":"2024-07-31T14:47:20.450435+0000","last_unstale":"2024-07-31T14:49:06.494830+0000","last_undegraded":"2024-07-31T14:49:06.494830+0000","last_fullsized":"2024-07-31T14:49:06.494830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:17:28.738911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.450860+0000","last_change":"2024-07-31T14:47:13.363054+0000","last_active":"2024-07-31T14:47:20.450860+0000","last_peered":"2024-07-31T14:47:20.450860+0000","last_clean":"2024-07-31T14:47:20.450860+0000","last_became_active":"2024-07-31T14:47:13.362364+0000","last_became_peered":"2024-07-31T14:47:13.362364+0000","last_unstale":"2024-07-31T14:47:20.450860+0000","last_undegraded":"2024-07-31T14:47:20.450860+0000","last_fullsized":"2024-07-31T14:47:20.450860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:02:36.569678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.393338+0000","last_change":"2024-07-31T14:47:03.059834+0000","last_active":"2024-07-31T14:49:49.393338+0000","last_peered":"2024-07-31T14:49:49.393338+0000","last_clean":"2024-07-31T14:49:49.393338+0000","last_became_active":"2024-07-31T14:47:03.059127+0000","last_became_peered":"2024-07-31T14:47:03.059127+0000","last_unstale":"2024-07-31T14:49:49.393338+0000","last_undegraded":"2024-07-31T14:49:49.393338+0000","last_fullsized":"2024-07-31T14:49:49.393338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:53:50.308789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":251,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:19.906235+0000","last_change":"2024-07-31T14:47:00.030576+0000","last_active":"2024-07-31T14:51:19.906235+0000","last_peered":"2024-07-31T14:51:19.906235+0000","last_clean":"2024-07-31T14:51:19.906235+0000","last_became_active":"2024-07-31T14:47:00.030174+0000","last_became_peered":"2024-07-31T14:47:00.030174+0000","last_unstale":"2024-07-31T14:51:19.906235+0000","last_undegraded":"2024-07-31T14:51:19.906235+0000","last_fullsized":"2024-07-31T14:51:19.906235+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:13:35.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":201,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:18.289527+0000","last_change":"2024-07-31T14:47:13.100180+0000","last_active":"2024-07-31T14:49:18.289527+0000","last_peered":"2024-07-31T14:49:18.289527+0000","last_clean":"2024-07-31T14:49:18.289527+0000","last_became_active":"2024-07-31T14:47:13.099725+0000","last_became_peered":"2024-07-31T14:47:13.099725+0000","last_unstale":"2024-07-31T14:49:18.289527+0000","last_undegraded":"2024-07-31T14:49:18.289527+0000","last_fullsized":"2024-07-31T14:49:18.289527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:25:32.245339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404277+0000","last_change":"2024-07-31T14:47:03.040349+0000","last_active":"2024-07-31T14:49:49.404277+0000","last_peered":"2024-07-31T14:49:49.404277+0000","last_clean":"2024-07-31T14:49:49.404277+0000","last_became_active":"2024-07-31T14:47:03.039887+0000","last_became_peered":"2024-07-31T14:47:03.039887+0000","last_unstale":"2024-07-31T14:49:49.404277+0000","last_undegraded":"2024-07-31T14:49:49.404277+0000","last_fullsized":"2024-07-31T14:49:49.404277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:46.592063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":251,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:21.895290+0000","last_change":"2024-07-31T14:47:00.183988+0000","last_active":"2024-07-31T14:51:21.895290+0000","last_peered":"2024-07-31T14:51:21.895290+0000","last_clean":"2024-07-31T14:51:21.895290+0000","last_became_active":"2024-07-31T14:47:00.183302+0000","last_became_peered":"2024-07-31T14:47:00.183302+0000","last_unstale":"2024-07-31T14:51:21.895290+0000","last_undegraded":"2024-07-31T14:51:21.895290+0000","last_fullsized":"2024-07-31T14:51:21.895290+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:40:26.188224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":201,"num_read_kb":201,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447744+0000","last_change":"2024-07-31T14:47:13.363075+0000","last_active":"2024-07-31T14:47:20.447744+0000","last_peered":"2024-07-31T14:47:20.447744+0000","last_clean":"2024-07-31T14:47:20.447744+0000","last_became_active":"2024-07-31T14:47:13.362599+0000","last_became_peered":"2024-07-31T14:47:13.362599+0000","last_unstale":"2024-07-31T14:47:20.447744+0000","last_undegraded":"2024-07-31T14:47:20.447744+0000","last_fullsized":"2024-07-31T14:47:20.447744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:20:43.617742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.402186+0000","last_change":"2024-07-31T14:47:20.450712+0000","last_active":"2024-07-31T14:49:49.402186+0000","last_peered":"2024-07-31T14:49:49.402186+0000","last_clean":"2024-07-31T14:49:49.402186+0000","last_became_active":"2024-07-31T14:47:20.450229+0000","last_became_peered":"2024-07-31T14:47:20.450229+0000","last_unstale":"2024-07-31T14:49:49.402186+0000","last_undegraded":"2024-07-31T14:49:49.402186+0000","last_fullsized":"2024-07-31T14:49:49.402186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:11:50.631420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:01.604764+0000","last_change":"2024-07-31T14:47:00.021389+0000","last_active":"2024-07-31T14:49:01.604764+0000","last_peered":"2024-07-31T14:49:01.604764+0000","last_clean":"2024-07-31T14:49:01.604764+0000","last_became_active":"2024-07-31T14:47:00.020987+0000","last_became_peered":"2024-07-31T14:47:00.020987+0000","last_unstale":"2024-07-31T14:49:01.604764+0000","last_undegraded":"2024-07-31T14:49:01.604764+0000","last_fullsized":"2024-07-31T14:49:01.604764+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:07:24.658463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627003+0000","last_change":"2024-07-31T14:47:13.363105+0000","last_active":"2024-07-31T14:47:20.627003+0000","last_peered":"2024-07-31T14:47:20.627003+0000","last_clean":"2024-07-31T14:47:20.627003+0000","last_became_active":"2024-07-31T14:47:13.362404+0000","last_became_peered":"2024-07-31T14:47:13.362404+0000","last_unstale":"2024-07-31T14:47:20.627003+0000","last_undegraded":"2024-07-31T14:47:20.627003+0000","last_fullsized":"2024-07-31T14:47:20.627003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:48:43.033584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404873+0000","last_change":"2024-07-31T14:47:03.057199+0000","last_active":"2024-07-31T14:49:49.404873+0000","last_peered":"2024-07-31T14:49:49.404873+0000","last_clean":"2024-07-31T14:49:49.404873+0000","last_became_active":"2024-07-31T14:47:03.056555+0000","last_became_peered":"2024-07-31T14:47:03.056555+0000","last_unstale":"2024-07-31T14:49:49.404873+0000","last_undegraded":"2024-07-31T14:49:49.404873+0000","last_fullsized":"2024-07-31T14:49:49.404873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:21:14.263457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:04.380224+0000","last_change":"2024-07-31T14:47:00.032055+0000","last_active":"2024-07-31T14:49:04.380224+0000","last_peered":"2024-07-31T14:49:04.380224+0000","last_clean":"2024-07-31T14:49:04.380224+0000","last_became_active":"2024-07-31T14:47:00.031519+0000","last_became_peered":"2024-07-31T14:47:00.031519+0000","last_unstale":"2024-07-31T14:49:04.380224+0000","last_undegraded":"2024-07-31T14:49:04.380224+0000","last_fullsized":"2024-07-31T14:49:04.380224+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:21:46.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:31.600736+0000","last_change":"2024-07-31T14:45:42.166503+0000","last_active":"2024-07-31T14:48:31.600736+0000","last_peered":"2024-07-31T14:48:31.600736+0000","last_clean":"2024-07-31T14:48:31.600736+0000","last_became_active":"2024-07-31T14:45:42.160065+0000","last_became_peered":"2024-07-31T14:45:42.160065+0000","last_unstale":"2024-07-31T14:48:31.600736+0000","last_undegraded":"2024-07-31T14:48:31.600736+0000","last_fullsized":"2024-07-31T14:48:31.600736+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_clean_scrub_stamp":"2024-07-31T14:42:27.863113+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:38:33.646022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:12.369486+0000","last_change":"2024-07-31T14:47:13.103812+0000","last_active":"2024-07-31T14:48:12.369486+0000","last_peered":"2024-07-31T14:48:12.369486+0000","last_clean":"2024-07-31T14:48:12.369486+0000","last_became_active":"2024-07-31T14:47:13.102549+0000","last_became_peered":"2024-07-31T14:47:13.102549+0000","last_unstale":"2024-07-31T14:48:12.369486+0000","last_undegraded":"2024-07-31T14:48:12.369486+0000","last_fullsized":"2024-07-31T14:48:12.369486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:18:56.322426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414083+0000","last_change":"2024-07-31T14:47:03.056310+0000","last_active":"2024-07-31T14:49:49.414083+0000","last_peered":"2024-07-31T14:49:49.414083+0000","last_clean":"2024-07-31T14:49:49.414083+0000","last_became_active":"2024-07-31T14:47:03.055695+0000","last_became_peered":"2024-07-31T14:47:03.055695+0000","last_unstale":"2024-07-31T14:49:49.414083+0000","last_undegraded":"2024-07-31T14:49:49.414083+0000","last_fullsized":"2024-07-31T14:49:49.414083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:00:55.124981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":250,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:19.906620+0000","last_change":"2024-07-31T14:47:00.236460+0000","last_active":"2024-07-31T14:51:19.906620+0000","last_peered":"2024-07-31T14:51:19.906620+0000","last_clean":"2024-07-31T14:51:19.906620+0000","last_became_active":"2024-07-31T14:47:00.235924+0000","last_became_peered":"2024-07-31T14:47:00.235924+0000","last_unstale":"2024-07-31T14:51:19.906620+0000","last_undegraded":"2024-07-31T14:51:19.906620+0000","last_fullsized":"2024-07-31T14:51:19.906620+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:05.130958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":202,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626080+0000","last_change":"2024-07-31T14:47:13.102341+0000","last_active":"2024-07-31T14:47:20.626080+0000","last_peered":"2024-07-31T14:47:20.626080+0000","last_clean":"2024-07-31T14:47:20.626080+0000","last_became_active":"2024-07-31T14:47:13.101798+0000","last_became_peered":"2024-07-31T14:47:13.101798+0000","last_unstale":"2024-07-31T14:47:20.626080+0000","last_undegraded":"2024-07-31T14:47:20.626080+0000","last_fullsized":"2024-07-31T14:47:20.626080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:34:35.758041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409700+0000","last_change":"2024-07-31T14:47:03.054322+0000","last_active":"2024-07-31T14:49:49.409700+0000","last_peered":"2024-07-31T14:49:49.409700+0000","last_clean":"2024-07-31T14:49:49.409700+0000","last_became_active":"2024-07-31T14:47:03.053112+0000","last_became_peered":"2024-07-31T14:47:03.053112+0000","last_unstale":"2024-07-31T14:49:49.409700+0000","last_undegraded":"2024-07-31T14:49:49.409700+0000","last_fullsized":"2024-07-31T14:49:49.409700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:55:13.542578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'62","reported_seq":134,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:22.197100+0000","last_change":"2024-07-31T14:47:00.026518+0000","last_active":"2024-07-31T14:51:22.197100+0000","last_peered":"2024-07-31T14:51:22.197100+0000","last_clean":"2024-07-31T14:51:22.197100+0000","last_became_active":"2024-07-31T14:47:00.026074+0000","last_became_peered":"2024-07-31T14:47:00.026074+0000","last_unstale":"2024-07-31T14:51:22.197100+0000","last_undegraded":"2024-07-31T14:51:22.197100+0000","last_fullsized":"2024-07-31T14:51:22.197100+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:33:36.688353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449369+0000","last_change":"2024-07-31T14:47:13.106365+0000","last_active":"2024-07-31T14:47:20.449369+0000","last_peered":"2024-07-31T14:47:20.449369+0000","last_clean":"2024-07-31T14:47:20.449369+0000","last_became_active":"2024-07-31T14:47:13.105635+0000","last_became_peered":"2024-07-31T14:47:13.105635+0000","last_unstale":"2024-07-31T14:47:20.449369+0000","last_undegraded":"2024-07-31T14:47:20.449369+0000","last_fullsized":"2024-07-31T14:47:20.449369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:52:09.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404285+0000","last_change":"2024-07-31T14:47:03.052522+0000","last_active":"2024-07-31T14:49:49.404285+0000","last_peered":"2024-07-31T14:49:49.404285+0000","last_clean":"2024-07-31T14:49:49.404285+0000","last_became_active":"2024-07-31T14:47:03.051872+0000","last_became_peered":"2024-07-31T14:47:03.051872+0000","last_unstale":"2024-07-31T14:49:49.404285+0000","last_undegraded":"2024-07-31T14:49:49.404285+0000","last_fullsized":"2024-07-31T14:49:49.404285+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:29:34.956745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600950+0000","last_change":"2024-07-31T14:47:00.016839+0000","last_active":"2024-07-31T14:48:02.600950+0000","last_peered":"2024-07-31T14:48:02.600950+0000","last_clean":"2024-07-31T14:48:02.600950+0000","last_became_active":"2024-07-31T14:47:00.016438+0000","last_became_peered":"2024-07-31T14:47:00.016438+0000","last_unstale":"2024-07-31T14:48:02.600950+0000","last_undegraded":"2024-07-31T14:48:02.600950+0000","last_fullsized":"2024-07-31T14:48:02.600950+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:42:10.875332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626258+0000","last_change":"2024-07-31T14:47:13.105385+0000","last_active":"2024-07-31T14:47:20.626258+0000","last_peered":"2024-07-31T14:47:20.626258+0000","last_clean":"2024-07-31T14:47:20.626258+0000","last_became_active":"2024-07-31T14:47:13.104867+0000","last_became_peered":"2024-07-31T14:47:13.104867+0000","last_unstale":"2024-07-31T14:47:20.626258+0000","last_undegraded":"2024-07-31T14:47:20.626258+0000","last_fullsized":"2024-07-31T14:47:20.626258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:25:21.572830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399072+0000","last_change":"2024-07-31T14:47:03.058304+0000","last_active":"2024-07-31T14:49:49.399072+0000","last_peered":"2024-07-31T14:49:49.399072+0000","last_clean":"2024-07-31T14:49:49.399072+0000","last_became_active":"2024-07-31T14:47:03.057251+0000","last_became_peered":"2024-07-31T14:47:03.057251+0000","last_unstale":"2024-07-31T14:49:49.399072+0000","last_undegraded":"2024-07-31T14:49:49.399072+0000","last_fullsized":"2024-07-31T14:49:49.399072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:58:20.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605301+0000","last_change":"2024-07-31T14:47:00.027806+0000","last_active":"2024-07-31T14:48:03.605301+0000","last_peered":"2024-07-31T14:48:03.605301+0000","last_clean":"2024-07-31T14:48:03.605301+0000","last_became_active":"2024-07-31T14:47:00.027403+0000","last_became_peered":"2024-07-31T14:47:00.027403+0000","last_unstale":"2024-07-31T14:48:03.605301+0000","last_undegraded":"2024-07-31T14:48:03.605301+0000","last_fullsized":"2024-07-31T14:48:03.605301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:57:18.303732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626490+0000","last_change":"2024-07-31T14:47:13.103628+0000","last_active":"2024-07-31T14:47:20.626490+0000","last_peered":"2024-07-31T14:47:20.626490+0000","last_clean":"2024-07-31T14:47:20.626490+0000","last_became_active":"2024-07-31T14:47:13.102296+0000","last_became_peered":"2024-07-31T14:47:13.102296+0000","last_unstale":"2024-07-31T14:47:20.626490+0000","last_undegraded":"2024-07-31T14:47:20.626490+0000","last_fullsized":"2024-07-31T14:47:20.626490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:25:35.109196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413387+0000","last_change":"2024-07-31T14:47:03.052761+0000","last_active":"2024-07-31T14:49:49.413387+0000","last_peered":"2024-07-31T14:49:49.413387+0000","last_clean":"2024-07-31T14:49:49.413387+0000","last_became_active":"2024-07-31T14:47:03.052410+0000","last_became_peered":"2024-07-31T14:47:03.052410+0000","last_unstale":"2024-07-31T14:49:49.413387+0000","last_undegraded":"2024-07-31T14:49:49.413387+0000","last_fullsized":"2024-07-31T14:49:49.413387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:20:47.712491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604937+0000","last_change":"2024-07-31T14:47:00.183741+0000","last_active":"2024-07-31T14:48:03.604937+0000","last_peered":"2024-07-31T14:48:03.604937+0000","last_clean":"2024-07-31T14:48:03.604937+0000","last_became_active":"2024-07-31T14:47:00.183275+0000","last_became_peered":"2024-07-31T14:47:00.183275+0000","last_unstale":"2024-07-31T14:48:03.604937+0000","last_undegraded":"2024-07-31T14:48:03.604937+0000","last_fullsized":"2024-07-31T14:48:03.604937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:29:23.205835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630098+0000","last_change":"2024-07-31T14:47:20.629334+0000","last_active":"2024-07-31T14:47:20.630098+0000","last_peered":"2024-07-31T14:47:20.630098+0000","last_clean":"2024-07-31T14:47:20.630098+0000","last_became_active":"2024-07-31T14:47:20.628865+0000","last_became_peered":"2024-07-31T14:47:20.628865+0000","last_unstale":"2024-07-31T14:47:20.630098+0000","last_undegraded":"2024-07-31T14:47:20.630098+0000","last_fullsized":"2024-07-31T14:47:20.630098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:21:27.199842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407413+0000","last_change":"2024-07-31T14:47:03.054633+0000","last_active":"2024-07-31T14:49:49.407413+0000","last_peered":"2024-07-31T14:49:49.407413+0000","last_clean":"2024-07-31T14:49:49.407413+0000","last_became_active":"2024-07-31T14:47:03.053025+0000","last_became_peered":"2024-07-31T14:47:03.053025+0000","last_unstale":"2024-07-31T14:49:49.407413+0000","last_undegraded":"2024-07-31T14:49:49.407413+0000","last_fullsized":"2024-07-31T14:49:49.407413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:42:03.392744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":256,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:22.897205+0000","last_change":"2024-07-31T14:47:00.027898+0000","last_active":"2024-07-31T14:51:22.897205+0000","last_peered":"2024-07-31T14:51:22.897205+0000","last_clean":"2024-07-31T14:51:22.897205+0000","last_became_active":"2024-07-31T14:47:00.027468+0000","last_became_peered":"2024-07-31T14:47:00.027468+0000","last_unstale":"2024-07-31T14:51:22.897205+0000","last_undegraded":"2024-07-31T14:51:22.897205+0000","last_fullsized":"2024-07-31T14:51:22.897205+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:12:45.922172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":207,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444574+0000","last_change":"2024-07-31T14:47:13.104327+0000","last_active":"2024-07-31T14:47:20.444574+0000","last_peered":"2024-07-31T14:47:20.444574+0000","last_clean":"2024-07-31T14:47:20.444574+0000","last_became_active":"2024-07-31T14:47:13.103333+0000","last_became_peered":"2024-07-31T14:47:13.103333+0000","last_unstale":"2024-07-31T14:47:20.444574+0000","last_undegraded":"2024-07-31T14:47:20.444574+0000","last_fullsized":"2024-07-31T14:47:20.444574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:06.226220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.410145+0000","last_change":"2024-07-31T14:47:03.044875+0000","last_active":"2024-07-31T14:49:49.410145+0000","last_peered":"2024-07-31T14:49:49.410145+0000","last_clean":"2024-07-31T14:49:49.410145+0000","last_became_active":"2024-07-31T14:47:03.044421+0000","last_became_peered":"2024-07-31T14:47:03.044421+0000","last_unstale":"2024-07-31T14:49:49.410145+0000","last_undegraded":"2024-07-31T14:49:49.410145+0000","last_fullsized":"2024-07-31T14:49:49.410145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:00:02.477300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605606+0000","last_change":"2024-07-31T14:47:00.019790+0000","last_active":"2024-07-31T14:48:03.605606+0000","last_peered":"2024-07-31T14:48:03.605606+0000","last_clean":"2024-07-31T14:48:03.605606+0000","last_became_active":"2024-07-31T14:47:00.019368+0000","last_became_peered":"2024-07-31T14:47:00.019368+0000","last_unstale":"2024-07-31T14:48:03.605606+0000","last_undegraded":"2024-07-31T14:48:03.605606+0000","last_fullsized":"2024-07-31T14:48:03.605606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:15.299109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445672+0000","last_change":"2024-07-31T14:47:13.102618+0000","last_active":"2024-07-31T14:47:20.445672+0000","last_peered":"2024-07-31T14:47:20.445672+0000","last_clean":"2024-07-31T14:47:20.445672+0000","last_became_active":"2024-07-31T14:47:13.102085+0000","last_became_peered":"2024-07-31T14:47:13.102085+0000","last_unstale":"2024-07-31T14:47:20.445672+0000","last_undegraded":"2024-07-31T14:47:20.445672+0000","last_fullsized":"2024-07-31T14:47:20.445672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:05:58.814215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.406277+0000","last_change":"2024-07-31T14:47:03.044688+0000","last_active":"2024-07-31T14:49:49.406277+0000","last_peered":"2024-07-31T14:49:49.406277+0000","last_clean":"2024-07-31T14:49:49.406277+0000","last_became_active":"2024-07-31T14:47:03.043751+0000","last_became_peered":"2024-07-31T14:47:03.043751+0000","last_unstale":"2024-07-31T14:49:49.406277+0000","last_undegraded":"2024-07-31T14:49:49.406277+0000","last_fullsized":"2024-07-31T14:49:49.406277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:47:31.450796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605830+0000","last_change":"2024-07-31T14:47:00.026259+0000","last_active":"2024-07-31T14:48:03.605830+0000","last_peered":"2024-07-31T14:48:03.605830+0000","last_clean":"2024-07-31T14:48:03.605830+0000","last_became_active":"2024-07-31T14:47:00.025756+0000","last_became_peered":"2024-07-31T14:47:00.025756+0000","last_unstale":"2024-07-31T14:48:03.605830+0000","last_undegraded":"2024-07-31T14:48:03.605830+0000","last_fullsized":"2024-07-31T14:48:03.605830+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:05:00.258841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626888+0000","last_change":"2024-07-31T14:47:13.104918+0000","last_active":"2024-07-31T14:47:20.626888+0000","last_peered":"2024-07-31T14:47:20.626888+0000","last_clean":"2024-07-31T14:47:20.626888+0000","last_became_active":"2024-07-31T14:47:13.104031+0000","last_became_peered":"2024-07-31T14:47:13.104031+0000","last_unstale":"2024-07-31T14:47:20.626888+0000","last_undegraded":"2024-07-31T14:47:20.626888+0000","last_fullsized":"2024-07-31T14:47:20.626888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:17:15.264474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.389841+0000","last_change":"2024-07-31T14:47:03.055159+0000","last_active":"2024-07-31T14:49:49.389841+0000","last_peered":"2024-07-31T14:49:49.389841+0000","last_clean":"2024-07-31T14:49:49.389841+0000","last_became_active":"2024-07-31T14:47:03.052389+0000","last_became_peered":"2024-07-31T14:47:03.052389+0000","last_unstale":"2024-07-31T14:49:49.389841+0000","last_undegraded":"2024-07-31T14:49:49.389841+0000","last_fullsized":"2024-07-31T14:49:49.389841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:16.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'42","reported_seq":502,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:24.899616+0000","last_change":"2024-07-31T14:47:00.030165+0000","last_active":"2024-07-31T14:51:24.899616+0000","last_peered":"2024-07-31T14:51:24.899616+0000","last_clean":"2024-07-31T14:51:24.899616+0000","last_became_active":"2024-07-31T14:47:00.029559+0000","last_became_peered":"2024-07-31T14:47:00.029559+0000","last_unstale":"2024-07-31T14:51:24.899616+0000","last_undegraded":"2024-07-31T14:51:24.899616+0000","last_fullsized":"2024-07-31T14:51:24.899616+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:31:06.115075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71918,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":418,"num_read_kb":280,"num_write":44,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444234+0000","last_change":"2024-07-31T14:47:13.098517+0000","last_active":"2024-07-31T14:47:20.444234+0000","last_peered":"2024-07-31T14:47:20.444234+0000","last_clean":"2024-07-31T14:47:20.444234+0000","last_became_active":"2024-07-31T14:47:13.098066+0000","last_became_peered":"2024-07-31T14:47:13.098066+0000","last_unstale":"2024-07-31T14:47:20.444234+0000","last_undegraded":"2024-07-31T14:47:20.444234+0000","last_fullsized":"2024-07-31T14:47:20.444234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:08:54.958646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414451+0000","last_change":"2024-07-31T14:47:03.054045+0000","last_active":"2024-07-31T14:49:49.414451+0000","last_peered":"2024-07-31T14:49:49.414451+0000","last_clean":"2024-07-31T14:49:49.414451+0000","last_became_active":"2024-07-31T14:47:03.053658+0000","last_became_peered":"2024-07-31T14:47:03.053658+0000","last_unstale":"2024-07-31T14:49:49.414451+0000","last_undegraded":"2024-07-31T14:49:49.414451+0000","last_fullsized":"2024-07-31T14:49:49.414451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:36:54.563542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.607373+0000","last_change":"2024-07-31T14:47:00.185602+0000","last_active":"2024-07-31T14:48:03.607373+0000","last_peered":"2024-07-31T14:48:03.607373+0000","last_clean":"2024-07-31T14:48:03.607373+0000","last_became_active":"2024-07-31T14:47:00.185230+0000","last_became_peered":"2024-07-31T14:47:00.185230+0000","last_unstale":"2024-07-31T14:48:03.607373+0000","last_undegraded":"2024-07-31T14:48:03.607373+0000","last_fullsized":"2024-07-31T14:48:03.607373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:21:35.979942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604698+0000","last_change":"2024-07-31T14:47:13.099611+0000","last_active":"2024-07-31T14:47:20.604698+0000","last_peered":"2024-07-31T14:47:20.604698+0000","last_clean":"2024-07-31T14:47:20.604698+0000","last_became_active":"2024-07-31T14:47:13.099127+0000","last_became_peered":"2024-07-31T14:47:13.099127+0000","last_unstale":"2024-07-31T14:47:20.604698+0000","last_undegraded":"2024-07-31T14:47:20.604698+0000","last_fullsized":"2024-07-31T14:47:20.604698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:16:57.192980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400126+0000","last_change":"2024-07-31T14:47:03.056818+0000","last_active":"2024-07-31T14:49:49.400126+0000","last_peered":"2024-07-31T14:49:49.400126+0000","last_clean":"2024-07-31T14:49:49.400126+0000","last_became_active":"2024-07-31T14:47:03.055716+0000","last_became_peered":"2024-07-31T14:47:03.055716+0000","last_unstale":"2024-07-31T14:49:49.400126+0000","last_undegraded":"2024-07-31T14:49:49.400126+0000","last_fullsized":"2024-07-31T14:49:49.400126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:26:33.627362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600344+0000","last_change":"2024-07-31T14:47:00.029651+0000","last_active":"2024-07-31T14:48:02.600344+0000","last_peered":"2024-07-31T14:48:02.600344+0000","last_clean":"2024-07-31T14:48:02.600344+0000","last_became_active":"2024-07-31T14:47:00.029242+0000","last_became_peered":"2024-07-31T14:47:00.029242+0000","last_unstale":"2024-07-31T14:48:02.600344+0000","last_undegraded":"2024-07-31T14:48:02.600344+0000","last_fullsized":"2024-07-31T14:48:02.600344+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:23:49.177508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.443852+0000","last_change":"2024-07-31T14:47:13.098862+0000","last_active":"2024-07-31T14:47:20.443852+0000","last_peered":"2024-07-31T14:47:20.443852+0000","last_clean":"2024-07-31T14:47:20.443852+0000","last_became_active":"2024-07-31T14:47:13.098324+0000","last_became_peered":"2024-07-31T14:47:13.098324+0000","last_unstale":"2024-07-31T14:47:20.443852+0000","last_undegraded":"2024-07-31T14:47:20.443852+0000","last_fullsized":"2024-07-31T14:47:20.443852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:50:41.260863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.412242+0000","last_change":"2024-07-31T14:47:03.049042+0000","last_active":"2024-07-31T14:49:49.412242+0000","last_peered":"2024-07-31T14:49:49.412242+0000","last_clean":"2024-07-31T14:49:49.412242+0000","last_became_active":"2024-07-31T14:47:03.048535+0000","last_became_peered":"2024-07-31T14:47:03.048535+0000","last_unstale":"2024-07-31T14:49:49.412242+0000","last_undegraded":"2024-07-31T14:49:49.412242+0000","last_fullsized":"2024-07-31T14:49:49.412242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:15.147454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":452,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:24.899853+0000","last_change":"2024-07-31T14:47:00.028940+0000","last_active":"2024-07-31T14:51:24.899853+0000","last_peered":"2024-07-31T14:51:24.899853+0000","last_clean":"2024-07-31T14:51:24.899853+0000","last_became_active":"2024-07-31T14:47:00.028472+0000","last_became_peered":"2024-07-31T14:47:00.028472+0000","last_unstale":"2024-07-31T14:51:24.899853+0000","last_undegraded":"2024-07-31T14:51:24.899853+0000","last_fullsized":"2024-07-31T14:51:24.899853+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:57:15.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":404,"num_read_kb":404,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626394+0000","last_change":"2024-07-31T14:47:13.101856+0000","last_active":"2024-07-31T14:47:20.626394+0000","last_peered":"2024-07-31T14:47:20.626394+0000","last_clean":"2024-07-31T14:47:20.626394+0000","last_became_active":"2024-07-31T14:47:13.101346+0000","last_became_peered":"2024-07-31T14:47:13.101346+0000","last_unstale":"2024-07-31T14:47:20.626394+0000","last_undegraded":"2024-07-31T14:47:20.626394+0000","last_fullsized":"2024-07-31T14:47:20.626394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:56:37.071660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:16.965978+0000","last_change":"2024-07-31T14:47:13.105952+0000","last_active":"2024-07-31T14:49:16.965978+0000","last_peered":"2024-07-31T14:49:16.965978+0000","last_clean":"2024-07-31T14:49:16.965978+0000","last_became_active":"2024-07-31T14:47:13.105506+0000","last_became_peered":"2024-07-31T14:47:13.105506+0000","last_unstale":"2024-07-31T14:49:16.965978+0000","last_undegraded":"2024-07-31T14:49:16.965978+0000","last_fullsized":"2024-07-31T14:49:16.965978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:28:33.866216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400532+0000","last_change":"2024-07-31T14:47:03.051152+0000","last_active":"2024-07-31T14:49:49.400532+0000","last_peered":"2024-07-31T14:49:49.400532+0000","last_clean":"2024-07-31T14:49:49.400532+0000","last_became_active":"2024-07-31T14:47:03.050775+0000","last_became_peered":"2024-07-31T14:47:03.050775+0000","last_unstale":"2024-07-31T14:49:49.400532+0000","last_undegraded":"2024-07-31T14:49:49.400532+0000","last_fullsized":"2024-07-31T14:49:49.400532+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:47:35.913641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609201+0000","last_change":"2024-07-31T14:47:00.028552+0000","last_active":"2024-07-31T14:48:03.609201+0000","last_peered":"2024-07-31T14:48:03.609201+0000","last_clean":"2024-07-31T14:48:03.609201+0000","last_became_active":"2024-07-31T14:47:00.028143+0000","last_became_peered":"2024-07-31T14:47:00.028143+0000","last_unstale":"2024-07-31T14:48:03.609201+0000","last_undegraded":"2024-07-31T14:48:03.609201+0000","last_fullsized":"2024-07-31T14:48:03.609201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:09.601635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:19.188903+0000","last_change":"2024-07-31T14:47:20.606285+0000","last_active":"2024-07-31T14:49:19.188903+0000","last_peered":"2024-07-31T14:49:19.188903+0000","last_clean":"2024-07-31T14:49:19.188903+0000","last_became_active":"2024-07-31T14:47:20.605837+0000","last_became_peered":"2024-07-31T14:47:20.605837+0000","last_unstale":"2024-07-31T14:49:19.188903+0000","last_undegraded":"2024-07-31T14:49:19.188903+0000","last_fullsized":"2024-07-31T14:49:19.188903+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:16:35.409790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.398709+0000","last_change":"2024-07-31T14:47:03.047442+0000","last_active":"2024-07-31T14:49:49.398709+0000","last_peered":"2024-07-31T14:49:49.398709+0000","last_clean":"2024-07-31T14:49:49.398709+0000","last_became_active":"2024-07-31T14:47:03.047072+0000","last_became_peered":"2024-07-31T14:47:03.047072+0000","last_unstale":"2024-07-31T14:49:49.398709+0000","last_undegraded":"2024-07-31T14:49:49.398709+0000","last_fullsized":"2024-07-31T14:49:49.398709+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:56.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.599595+0000","last_change":"2024-07-31T14:47:00.027746+0000","last_active":"2024-07-31T14:48:02.599595+0000","last_peered":"2024-07-31T14:48:02.599595+0000","last_clean":"2024-07-31T14:48:02.599595+0000","last_became_active":"2024-07-31T14:47:00.027404+0000","last_became_peered":"2024-07-31T14:47:00.027404+0000","last_unstale":"2024-07-31T14:48:02.599595+0000","last_undegraded":"2024-07-31T14:48:02.599595+0000","last_fullsized":"2024-07-31T14:48:02.599595+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:30:21.011674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.589640+0000","last_change":"2024-07-31T14:47:00.029148+0000","last_active":"2024-07-31T14:48:01.589640+0000","last_peered":"2024-07-31T14:48:01.589640+0000","last_clean":"2024-07-31T14:48:01.589640+0000","last_became_active":"2024-07-31T14:47:00.028718+0000","last_became_peered":"2024-07-31T14:47:00.028718+0000","last_unstale":"2024-07-31T14:48:01.589640+0000","last_undegraded":"2024-07-31T14:48:01.589640+0000","last_fullsized":"2024-07-31T14:48:01.589640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:25:16.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400207+0000","last_change":"2024-07-31T14:47:20.629098+0000","last_active":"2024-07-31T14:49:49.400207+0000","last_peered":"2024-07-31T14:49:49.400207+0000","last_clean":"2024-07-31T14:49:49.400207+0000","last_became_active":"2024-07-31T14:47:20.627775+0000","last_became_peered":"2024-07-31T14:47:20.627775+0000","last_unstale":"2024-07-31T14:49:49.400207+0000","last_undegraded":"2024-07-31T14:49:49.400207+0000","last_fullsized":"2024-07-31T14:49:49.400207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:22:03.606509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447331+0000","last_change":"2024-07-31T14:47:13.097892+0000","last_active":"2024-07-31T14:47:20.447331+0000","last_peered":"2024-07-31T14:47:20.447331+0000","last_clean":"2024-07-31T14:47:20.447331+0000","last_became_active":"2024-07-31T14:47:13.097419+0000","last_became_peered":"2024-07-31T14:47:13.097419+0000","last_unstale":"2024-07-31T14:47:20.447331+0000","last_undegraded":"2024-07-31T14:47:20.447331+0000","last_fullsized":"2024-07-31T14:47:20.447331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:07:13.456147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":253,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:21.896133+0000","last_change":"2024-07-31T14:47:00.019792+0000","last_active":"2024-07-31T14:51:21.896133+0000","last_peered":"2024-07-31T14:51:21.896133+0000","last_clean":"2024-07-31T14:51:21.896133+0000","last_became_active":"2024-07-31T14:47:00.019411+0000","last_became_peered":"2024-07-31T14:47:00.019411+0000","last_unstale":"2024-07-31T14:51:21.896133+0000","last_undegraded":"2024-07-31T14:51:21.896133+0000","last_fullsized":"2024-07-31T14:51:21.896133+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:55:02.148823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":204,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.390707+0000","last_change":"2024-07-31T14:47:03.039808+0000","last_active":"2024-07-31T14:49:49.390707+0000","last_peered":"2024-07-31T14:49:49.390707+0000","last_clean":"2024-07-31T14:49:49.390707+0000","last_became_active":"2024-07-31T14:47:03.039382+0000","last_became_peered":"2024-07-31T14:47:03.039382+0000","last_unstale":"2024-07-31T14:49:49.390707+0000","last_undegraded":"2024-07-31T14:49:49.390707+0000","last_fullsized":"2024-07-31T14:49:49.390707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:23:09.751293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.446939+0000","last_change":"2024-07-31T14:47:13.102797+0000","last_active":"2024-07-31T14:47:20.446939+0000","last_peered":"2024-07-31T14:47:20.446939+0000","last_clean":"2024-07-31T14:47:20.446939+0000","last_became_active":"2024-07-31T14:47:13.102320+0000","last_became_peered":"2024-07-31T14:47:13.102320+0000","last_unstale":"2024-07-31T14:47:20.446939+0000","last_undegraded":"2024-07-31T14:47:20.446939+0000","last_fullsized":"2024-07-31T14:47:20.446939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:23.066357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.397081+0000","last_change":"2024-07-31T14:47:03.054875+0000","last_active":"2024-07-31T14:49:49.397081+0000","last_peered":"2024-07-31T14:49:49.397081+0000","last_clean":"2024-07-31T14:49:49.397081+0000","last_became_active":"2024-07-31T14:47:03.054529+0000","last_became_peered":"2024-07-31T14:47:03.054529+0000","last_unstale":"2024-07-31T14:49:49.397081+0000","last_undegraded":"2024-07-31T14:49:49.397081+0000","last_fullsized":"2024-07-31T14:49:49.397081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:32:09.802933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396421+0000","last_change":"2024-07-31T14:47:03.043283+0000","last_active":"2024-07-31T14:49:49.396421+0000","last_peered":"2024-07-31T14:49:49.396421+0000","last_clean":"2024-07-31T14:49:49.396421+0000","last_became_active":"2024-07-31T14:47:03.042851+0000","last_became_peered":"2024-07-31T14:47:03.042851+0000","last_unstale":"2024-07-31T14:49:49.396421+0000","last_undegraded":"2024-07-31T14:49:49.396421+0000","last_fullsized":"2024-07-31T14:49:49.396421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:25:37.408149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409526+0000","last_change":"2024-07-31T14:47:03.047890+0000","last_active":"2024-07-31T14:49:49.409526+0000","last_peered":"2024-07-31T14:49:49.409526+0000","last_clean":"2024-07-31T14:49:49.409526+0000","last_became_active":"2024-07-31T14:47:03.047507+0000","last_became_peered":"2024-07-31T14:47:03.047507+0000","last_unstale":"2024-07-31T14:49:49.409526+0000","last_undegraded":"2024-07-31T14:49:49.409526+0000","last_fullsized":"2024-07-31T14:49:49.409526+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:52:45.991323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403192+0000","last_change":"2024-07-31T14:47:03.047903+0000","last_active":"2024-07-31T14:49:49.403192+0000","last_peered":"2024-07-31T14:49:49.403192+0000","last_clean":"2024-07-31T14:49:49.403192+0000","last_became_active":"2024-07-31T14:47:03.047352+0000","last_became_peered":"2024-07-31T14:47:03.047352+0000","last_unstale":"2024-07-31T14:49:49.403192+0000","last_undegraded":"2024-07-31T14:49:49.403192+0000","last_fullsized":"2024-07-31T14:49:49.403192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:06.536023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403277+0000","last_change":"2024-07-31T14:47:03.044199+0000","last_active":"2024-07-31T14:49:49.403277+0000","last_peered":"2024-07-31T14:49:49.403277+0000","last_clean":"2024-07-31T14:49:49.403277+0000","last_became_active":"2024-07-31T14:47:03.043849+0000","last_became_peered":"2024-07-31T14:47:03.043849+0000","last_unstale":"2024-07-31T14:49:49.403277+0000","last_undegraded":"2024-07-31T14:49:49.403277+0000","last_fullsized":"2024-07-31T14:49:49.403277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:50.474895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.391219+0000","last_change":"2024-07-31T14:47:03.060654+0000","last_active":"2024-07-31T14:49:49.391219+0000","last_peered":"2024-07-31T14:49:49.391219+0000","last_clean":"2024-07-31T14:49:49.391219+0000","last_became_active":"2024-07-31T14:47:03.060157+0000","last_became_peered":"2024-07-31T14:47:03.060157+0000","last_unstale":"2024-07-31T14:49:49.391219+0000","last_undegraded":"2024-07-31T14:49:49.391219+0000","last_fullsized":"2024-07-31T14:49:49.391219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:39:50.130831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414163+0000","last_change":"2024-07-31T14:47:03.049910+0000","last_active":"2024-07-31T14:49:49.414163+0000","last_peered":"2024-07-31T14:49:49.414163+0000","last_clean":"2024-07-31T14:49:49.414163+0000","last_became_active":"2024-07-31T14:47:03.049491+0000","last_became_peered":"2024-07-31T14:47:03.049491+0000","last_unstale":"2024-07-31T14:49:49.414163+0000","last_undegraded":"2024-07-31T14:49:49.414163+0000","last_fullsized":"2024-07-31T14:49:49.414163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:18:43.319695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.410414+0000","last_change":"2024-07-31T14:47:03.052206+0000","last_active":"2024-07-31T14:49:49.410414+0000","last_peered":"2024-07-31T14:49:49.410414+0000","last_clean":"2024-07-31T14:49:49.410414+0000","last_became_active":"2024-07-31T14:47:03.051175+0000","last_became_peered":"2024-07-31T14:47:03.051175+0000","last_unstale":"2024-07-31T14:49:49.410414+0000","last_undegraded":"2024-07-31T14:49:49.410414+0000","last_fullsized":"2024-07-31T14:49:49.410414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:51:11.583604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.405714+0000","last_change":"2024-07-31T14:47:03.052162+0000","last_active":"2024-07-31T14:49:49.405714+0000","last_peered":"2024-07-31T14:49:49.405714+0000","last_clean":"2024-07-31T14:49:49.405714+0000","last_became_active":"2024-07-31T14:47:03.051634+0000","last_became_peered":"2024-07-31T14:47:03.051634+0000","last_unstale":"2024-07-31T14:49:49.405714+0000","last_undegraded":"2024-07-31T14:49:49.405714+0000","last_fullsized":"2024-07-31T14:49:49.405714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:41:59.605192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409010+0000","last_change":"2024-07-31T14:47:03.047475+0000","last_active":"2024-07-31T14:49:49.409010+0000","last_peered":"2024-07-31T14:49:49.409010+0000","last_clean":"2024-07-31T14:49:49.409010+0000","last_became_active":"2024-07-31T14:47:03.047100+0000","last_became_peered":"2024-07-31T14:47:03.047100+0000","last_unstale":"2024-07-31T14:49:49.409010+0000","last_undegraded":"2024-07-31T14:49:49.409010+0000","last_fullsized":"2024-07-31T14:49:49.409010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:08:10.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411379+0000","last_change":"2024-07-31T14:47:03.051281+0000","last_active":"2024-07-31T14:49:49.411379+0000","last_peered":"2024-07-31T14:49:49.411379+0000","last_clean":"2024-07-31T14:49:49.411379+0000","last_became_active":"2024-07-31T14:47:03.050921+0000","last_became_peered":"2024-07-31T14:47:03.050921+0000","last_unstale":"2024-07-31T14:49:49.411379+0000","last_undegraded":"2024-07-31T14:49:49.411379+0000","last_fullsized":"2024-07-31T14:49:49.411379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:24:29.221723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403114+0000","last_change":"2024-07-31T14:47:03.045457+0000","last_active":"2024-07-31T14:49:49.403114+0000","last_peered":"2024-07-31T14:49:49.403114+0000","last_clean":"2024-07-31T14:49:49.403114+0000","last_became_active":"2024-07-31T14:47:03.044466+0000","last_became_peered":"2024-07-31T14:47:03.044466+0000","last_unstale":"2024-07-31T14:49:49.403114+0000","last_undegraded":"2024-07-31T14:49:49.403114+0000","last_fullsized":"2024-07-31T14:49:49.403114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:44:01.197416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403872+0000","last_change":"2024-07-31T14:47:03.053431+0000","last_active":"2024-07-31T14:49:49.403872+0000","last_peered":"2024-07-31T14:49:49.403872+0000","last_clean":"2024-07-31T14:49:49.403872+0000","last_became_active":"2024-07-31T14:47:03.052958+0000","last_became_peered":"2024-07-31T14:47:03.052958+0000","last_unstale":"2024-07-31T14:49:49.403872+0000","last_undegraded":"2024-07-31T14:49:49.403872+0000","last_fullsized":"2024-07-31T14:49:49.403872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:03:28.863589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409420+0000","last_change":"2024-07-31T14:47:03.046641+0000","last_active":"2024-07-31T14:49:49.409420+0000","last_peered":"2024-07-31T14:49:49.409420+0000","last_clean":"2024-07-31T14:49:49.409420+0000","last_became_active":"2024-07-31T14:47:03.046265+0000","last_became_peered":"2024-07-31T14:47:03.046265+0000","last_unstale":"2024-07-31T14:49:49.409420+0000","last_undegraded":"2024-07-31T14:49:49.409420+0000","last_fullsized":"2024-07-31T14:49:49.409420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:20:25.389010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.387120+0000","last_change":"2024-07-31T14:47:03.051657+0000","last_active":"2024-07-31T14:49:49.387120+0000","last_peered":"2024-07-31T14:49:49.387120+0000","last_clean":"2024-07-31T14:49:49.387120+0000","last_became_active":"2024-07-31T14:47:03.050722+0000","last_became_peered":"2024-07-31T14:47:03.050722+0000","last_unstale":"2024-07-31T14:49:49.387120+0000","last_undegraded":"2024-07-31T14:49:49.387120+0000","last_fullsized":"2024-07-31T14:49:49.387120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:23:19.387470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411567+0000","last_change":"2024-07-31T14:47:03.059654+0000","last_active":"2024-07-31T14:49:49.411567+0000","last_peered":"2024-07-31T14:49:49.411567+0000","last_clean":"2024-07-31T14:49:49.411567+0000","last_became_active":"2024-07-31T14:47:03.056843+0000","last_became_peered":"2024-07-31T14:47:03.056843+0000","last_unstale":"2024-07-31T14:49:49.411567+0000","last_undegraded":"2024-07-31T14:49:49.411567+0000","last_fullsized":"2024-07-31T14:49:49.411567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:42:33.696498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413200+0000","last_change":"2024-07-31T14:47:03.052750+0000","last_active":"2024-07-31T14:49:49.413200+0000","last_peered":"2024-07-31T14:49:49.413200+0000","last_clean":"2024-07-31T14:49:49.413200+0000","last_became_active":"2024-07-31T14:47:03.051744+0000","last_became_peered":"2024-07-31T14:47:03.051744+0000","last_unstale":"2024-07-31T14:49:49.413200+0000","last_undegraded":"2024-07-31T14:49:49.413200+0000","last_fullsized":"2024-07-31T14:49:49.413200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:24:33.202918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.393222+0000","last_change":"2024-07-31T14:47:20.452563+0000","last_active":"2024-07-31T14:49:49.393222+0000","last_peered":"2024-07-31T14:49:49.393222+0000","last_clean":"2024-07-31T14:49:49.393222+0000","last_became_active":"2024-07-31T14:47:20.452271+0000","last_became_peered":"2024-07-31T14:47:20.452271+0000","last_unstale":"2024-07-31T14:49:49.393222+0000","last_undegraded":"2024-07-31T14:49:49.393222+0000","last_fullsized":"2024-07-31T14:49:49.393222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:26:55.070265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.391684+0000","last_change":"2024-07-31T14:47:03.050691+0000","last_active":"2024-07-31T14:49:49.391684+0000","last_peered":"2024-07-31T14:49:49.391684+0000","last_clean":"2024-07-31T14:49:49.391684+0000","last_became_active":"2024-07-31T14:47:03.050320+0000","last_became_peered":"2024-07-31T14:47:03.050320+0000","last_unstale":"2024-07-31T14:49:49.391684+0000","last_undegraded":"2024-07-31T14:49:49.391684+0000","last_fullsized":"2024-07-31T14:49:49.391684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:40:45.767706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400128+0000","last_change":"2024-07-31T14:47:03.048491+0000","last_active":"2024-07-31T14:49:49.400128+0000","last_peered":"2024-07-31T14:49:49.400128+0000","last_clean":"2024-07-31T14:49:49.400128+0000","last_became_active":"2024-07-31T14:47:03.048085+0000","last_became_peered":"2024-07-31T14:47:03.048085+0000","last_unstale":"2024-07-31T14:49:49.400128+0000","last_undegraded":"2024-07-31T14:49:49.400128+0000","last_fullsized":"2024-07-31T14:49:49.400128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:35:01.762365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.395425+0000","last_change":"2024-07-31T14:47:03.055502+0000","last_active":"2024-07-31T14:49:49.395425+0000","last_peered":"2024-07-31T14:49:49.395425+0000","last_clean":"2024-07-31T14:49:49.395425+0000","last_became_active":"2024-07-31T14:47:03.055015+0000","last_became_peered":"2024-07-31T14:47:03.055015+0000","last_unstale":"2024-07-31T14:49:49.395425+0000","last_undegraded":"2024-07-31T14:49:49.395425+0000","last_fullsized":"2024-07-31T14:49:49.395425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:50:39.605148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413933+0000","last_change":"2024-07-31T14:47:03.057808+0000","last_active":"2024-07-31T14:49:49.413933+0000","last_peered":"2024-07-31T14:49:49.413933+0000","last_clean":"2024-07-31T14:49:49.413933+0000","last_became_active":"2024-07-31T14:47:03.057443+0000","last_became_peered":"2024-07-31T14:47:03.057443+0000","last_unstale":"2024-07-31T14:49:49.413933+0000","last_undegraded":"2024-07-31T14:49:49.413933+0000","last_fullsized":"2024-07-31T14:49:49.413933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:34:42.915428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396116+0000","last_change":"2024-07-31T14:47:03.058182+0000","last_active":"2024-07-31T14:49:49.396116+0000","last_peered":"2024-07-31T14:49:49.396116+0000","last_clean":"2024-07-31T14:49:49.396116+0000","last_became_active":"2024-07-31T14:47:03.057802+0000","last_became_peered":"2024-07-31T14:47:03.057802+0000","last_unstale":"2024-07-31T14:49:49.396116+0000","last_undegraded":"2024-07-31T14:49:49.396116+0000","last_fullsized":"2024-07-31T14:49:49.396116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:08:06.608788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413379+0000","last_change":"2024-07-31T14:47:03.058882+0000","last_active":"2024-07-31T14:49:49.413379+0000","last_peered":"2024-07-31T14:49:49.413379+0000","last_clean":"2024-07-31T14:49:49.413379+0000","last_became_active":"2024-07-31T14:47:03.058530+0000","last_became_peered":"2024-07-31T14:47:03.058530+0000","last_unstale":"2024-07-31T14:49:49.413379+0000","last_undegraded":"2024-07-31T14:49:49.413379+0000","last_fullsized":"2024-07-31T14:49:49.413379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:28:03.631962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404760+0000","last_change":"2024-07-31T14:47:03.052028+0000","last_active":"2024-07-31T14:49:49.404760+0000","last_peered":"2024-07-31T14:49:49.404760+0000","last_clean":"2024-07-31T14:49:49.404760+0000","last_became_active":"2024-07-31T14:47:03.051559+0000","last_became_peered":"2024-07-31T14:47:03.051559+0000","last_unstale":"2024-07-31T14:49:49.404760+0000","last_undegraded":"2024-07-31T14:49:49.404760+0000","last_fullsized":"2024-07-31T14:49:49.404760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:16:40.723795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399075+0000","last_change":"2024-07-31T14:47:03.053884+0000","last_active":"2024-07-31T14:49:49.399075+0000","last_peered":"2024-07-31T14:49:49.399075+0000","last_clean":"2024-07-31T14:49:49.399075+0000","last_became_active":"2024-07-31T14:47:03.053521+0000","last_became_peered":"2024-07-31T14:47:03.053521+0000","last_unstale":"2024-07-31T14:49:49.399075+0000","last_undegraded":"2024-07-31T14:49:49.399075+0000","last_fullsized":"2024-07-31T14:49:49.399075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:16:37.702055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.394011+0000","last_change":"2024-07-31T14:47:03.058588+0000","last_active":"2024-07-31T14:49:49.394011+0000","last_peered":"2024-07-31T14:49:49.394011+0000","last_clean":"2024-07-31T14:49:49.394011+0000","last_became_active":"2024-07-31T14:47:03.057820+0000","last_became_peered":"2024-07-31T14:47:03.057820+0000","last_unstale":"2024-07-31T14:49:49.394011+0000","last_undegraded":"2024-07-31T14:49:49.394011+0000","last_fullsized":"2024-07-31T14:49:49.394011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:21.265208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396703+0000","last_change":"2024-07-31T14:47:03.041447+0000","last_active":"2024-07-31T14:49:49.396703+0000","last_peered":"2024-07-31T14:49:49.396703+0000","last_clean":"2024-07-31T14:49:49.396703+0000","last_became_active":"2024-07-31T14:47:03.040935+0000","last_became_peered":"2024-07-31T14:47:03.040935+0000","last_unstale":"2024-07-31T14:49:49.396703+0000","last_undegraded":"2024-07-31T14:49:49.396703+0000","last_fullsized":"2024-07-31T14:49:49.396703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:00:13.624698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396885+0000","last_change":"2024-07-31T14:47:20.452239+0000","last_active":"2024-07-31T14:49:49.396885+0000","last_peered":"2024-07-31T14:49:49.396885+0000","last_clean":"2024-07-31T14:49:49.396885+0000","last_became_active":"2024-07-31T14:47:20.451924+0000","last_became_peered":"2024-07-31T14:47:20.451924+0000","last_unstale":"2024-07-31T14:49:49.396885+0000","last_undegraded":"2024-07-31T14:49:49.396885+0000","last_fullsized":"2024-07-31T14:49:49.396885+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:58:24.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404120+0000","last_change":"2024-07-31T14:47:03.056024+0000","last_active":"2024-07-31T14:49:49.404120+0000","last_peered":"2024-07-31T14:49:49.404120+0000","last_clean":"2024-07-31T14:49:49.404120+0000","last_became_active":"2024-07-31T14:47:03.055630+0000","last_became_peered":"2024-07-31T14:47:03.055630+0000","last_unstale":"2024-07-31T14:49:49.404120+0000","last_undegraded":"2024-07-31T14:49:49.404120+0000","last_fullsized":"2024-07-31T14:49:49.404120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:09:11.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396693+0000","last_change":"2024-07-31T14:47:03.053718+0000","last_active":"2024-07-31T14:49:49.396693+0000","last_peered":"2024-07-31T14:49:49.396693+0000","last_clean":"2024-07-31T14:49:49.396693+0000","last_became_active":"2024-07-31T14:47:03.053361+0000","last_became_peered":"2024-07-31T14:47:03.053361+0000","last_unstale":"2024-07-31T14:49:49.396693+0000","last_undegraded":"2024-07-31T14:49:49.396693+0000","last_fullsized":"2024-07-31T14:49:49.396693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:03:49.220838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411585+0000","last_change":"2024-07-31T14:47:03.056087+0000","last_active":"2024-07-31T14:49:49.411585+0000","last_peered":"2024-07-31T14:49:49.411585+0000","last_clean":"2024-07-31T14:49:49.411585+0000","last_became_active":"2024-07-31T14:47:03.055732+0000","last_became_peered":"2024-07-31T14:47:03.055732+0000","last_unstale":"2024-07-31T14:49:49.411585+0000","last_undegraded":"2024-07-31T14:49:49.411585+0000","last_fullsized":"2024-07-31T14:49:49.411585+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:45:01.185666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":868018,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2260,"num_read_kb":1303,"num_write":447,"num_write_kb":2132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2973696,"data_stored":2604054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":352,"ondisk_log_size":352,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645379,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":520429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841608,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37472,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707680,"statfs":{"total":95995035648,"available":95956664320,"internally_reserved":0,"allocated":2125824,"data_stored":1573163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005133,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":523756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201362,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1437696,"data_stored":913254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397590,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1118131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561116,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":515696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757345,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29076,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716076,"statfs":{"total":95995035648,"available":95965261824,"internally_reserved":0,"allocated":2048000,"data_stored":1506034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920870,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":520161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117098,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28860,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716292,"statfs":{"total":95995035648,"available":95965483008,"internally_reserved":0,"allocated":1826816,"data_stored":1301864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378735,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":589563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542260,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":588644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705784,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":517319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":819200,"data_stored":787422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":397128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":524288,"data_stored":468353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-31T14:51:27.446 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph pg dump --format=json 2024-07-31T14:51:27.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.873+0000 7f344b472640 1 -- 172.21.15.67:0/4106019142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3444108ed0 msgr2=0x7f34441092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:27.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.873+0000 7f344b472640 1 --2- 172.21.15.67:0/4106019142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3444108ed0 0x7f34441092e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3434009900 tx=0x7f343402a410 comp rx=0 tx=0).stop 2024-07-31T14:51:27.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.873+0000 7f344b472640 1 -- 172.21.15.67:0/4106019142 shutdown_connections 2024-07-31T14:51:27.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.873+0000 7f344b472640 1 --2- 172.21.15.67:0/4106019142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3444109820 0x7f3444110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.873+0000 7f344b472640 1 --2- 172.21.15.67:0/4106019142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3444108ed0 0x7f34441092e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.873+0000 7f344b472640 1 --2- 172.21.15.67:0/4106019142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3444103590 0x7f34441039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.873+0000 7f344b472640 1 -- 172.21.15.67:0/4106019142 >> 172.21.15.67:0/4106019142 conn(0x7f344407b280 msgr2=0x7f344407b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:27.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.873+0000 7f344b472640 1 -- 172.21.15.67:0/4106019142 shutdown_connections 2024-07-31T14:51:27.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.873+0000 7f344b472640 1 -- 172.21.15.67:0/4106019142 wait complete. 2024-07-31T14:51:27.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.874+0000 7f344b472640 1 Processor -- start 2024-07-31T14:51:27.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.874+0000 7f344b472640 1 -- start start 2024-07-31T14:51:27.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.874+0000 7f344b472640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3444103590 0x7f3444199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.874+0000 7f344b472640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3444108ed0 0x7f344419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.874+0000 7f344b472640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3444109820 0x7f344419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.874+0000 7f344b472640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3444113490 con 0x7f3444108ed0 2024-07-31T14:51:27.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.874+0000 7f344b472640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3444113310 con 0x7f3444109820 2024-07-31T14:51:27.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.874+0000 7f344b472640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3444113610 con 0x7f3444103590 2024-07-31T14:51:27.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f3448a0d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3444108ed0 0x7f344419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f3448a0d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3444108ed0 0x7f344419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45432/0 (socket says 172.21.15.67:45432) 2024-07-31T14:51:27.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f3448a0d640 1 -- 172.21.15.67:0/1519264373 learned_addr learned my addr 172.21.15.67:0/1519264373 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:27.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f3449a0f640 1 --2- 172.21.15.67:0/1519264373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3444109820 0x7f344419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f344920e640 1 --2- 172.21.15.67:0/1519264373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3444103590 0x7f3444199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f3448a0d640 1 -- 172.21.15.67:0/1519264373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3444103590 msgr2=0x7f3444199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:27.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f3448a0d640 1 --2- 172.21.15.67:0/1519264373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3444103590 0x7f3444199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f3448a0d640 1 -- 172.21.15.67:0/1519264373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3444109820 msgr2=0x7f344419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:27.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f3448a0d640 1 --2- 172.21.15.67:0/1519264373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3444109820 0x7f344419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f3448a0d640 1 -- 172.21.15.67:0/1519264373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344419fe90 con 0x7f3444108ed0 2024-07-31T14:51:27.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f3448a0d640 1 --2- 172.21.15.67:0/1519264373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3444108ed0 0x7f344419a4c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f343402a930 tx=0x7f3434004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:27.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f343a7fc640 1 -- 172.21.15.67:0/1519264373 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343403f030 con 0x7f3444108ed0 2024-07-31T14:51:27.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f343a7fc640 1 -- 172.21.15.67:0/1519264373 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3434003aa0 con 0x7f3444108ed0 2024-07-31T14:51:27.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f344b472640 1 -- 172.21.15.67:0/1519264373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34441a0120 con 0x7f3444108ed0 2024-07-31T14:51:27.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f343a7fc640 1 -- 172.21.15.67:0/1519264373 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434035070 con 0x7f3444108ed0 2024-07-31T14:51:27.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.875+0000 7f344b472640 1 -- 172.21.15.67:0/1519264373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34441a68e0 con 0x7f3444108ed0 2024-07-31T14:51:27.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.878+0000 7f34519c3740 1 Processor -- start 2024-07-31T14:51:27.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.878+0000 7f343a7fc640 1 -- 172.21.15.67:0/1519264373 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f3434042030 con 0x7f3444108ed0 2024-07-31T14:51:27.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.878+0000 7f34519c3740 1 -- start start 2024-07-31T14:51:27.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f34519c3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ce070 0x55f5fd673670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f34519c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2a51b0 0x55f5fd673bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f34519c3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2ad390 0x55f5fd6740f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f34519c3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f5fd5e9b90 con 0x55f5fd2ad390 2024-07-31T14:51:27.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f34519c3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f5fd5d9090 con 0x55f5fd2ce070 2024-07-31T14:51:27.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f34519c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f5fd5dd0d0 con 0x55f5fd2a51b0 2024-07-31T14:51:27.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f3448a0d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2ad390 0x55f5fd6740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f3448a0d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2ad390 0x55f5fd6740f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45434/0 (socket says 172.21.15.67:45434) 2024-07-31T14:51:27.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f3448a0d640 1 -- 172.21.15.67:0/2094014497 learned_addr learned my addr 172.21.15.67:0/2094014497 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:27.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f3449a0f640 1 --2- 172.21.15.67:0/2094014497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ce070 0x55f5fd673670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f344920e640 1 --2- 172.21.15.67:0/2094014497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2a51b0 0x55f5fd673bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f343a7fc640 1 --2- 172.21.15.67:0/1519264373 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f34300b3d40 0x7f34300b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f3448a0d640 1 -- 172.21.15.67:0/2094014497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2a51b0 msgr2=0x55f5fd673bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:27.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f3448a0d640 1 --2- 172.21.15.67:0/2094014497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2a51b0 0x55f5fd673bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f3448a0d640 1 -- 172.21.15.67:0/2094014497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ce070 msgr2=0x55f5fd673670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:27.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f3448a0d640 1 --2- 172.21.15.67:0/2094014497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ce070 0x55f5fd673670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f3448a0d640 1 -- 172.21.15.67:0/2094014497 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5fd57fb70 con 0x55f5fd2ad390 2024-07-31T14:51:27.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f3449a0f640 1 --2- 172.21.15.67:0/2094014497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ce070 0x55f5fd673670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:27.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f3449a0f640 1 --2- 172.21.15.67:0/1519264373 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f34300b3d40 0x7f34300b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.879+0000 7f343a7fc640 1 -- 172.21.15.67:0/1519264373 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f34341049b0 con 0x7f3444108ed0 2024-07-31T14:51:27.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.880+0000 7f3448a0d640 1 --2- 172.21.15.67:0/2094014497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2ad390 0x55f5fd6740f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3434101d60 tx=0x7f3434090180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7217ba4f231222 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:27.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.880+0000 7f342f7fe640 1 -- 172.21.15.67:0/2094014497 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434049070 con 0x55f5fd2ad390 2024-07-31T14:51:27.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.880+0000 7f3449a0f640 1 --2- 172.21.15.67:0/1519264373 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f34300b3d40 0x7f34300b6250 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f343c0125f0 tx=0x7f343c004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:27.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.880+0000 7f342f7fe640 1 -- 172.21.15.67:0/2094014497 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34340cadc0 con 0x55f5fd2ad390 2024-07-31T14:51:27.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.880+0000 7f342f7fe640 1 -- 172.21.15.67:0/2094014497 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434095670 con 0x55f5fd2ad390 2024-07-31T14:51:27.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.881+0000 7f34519c3740 1 -- 172.21.15.67:0/2094014497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2ad390 msgr2=0x55f5fd6740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:27.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.881+0000 7f34519c3740 1 --2- 172.21.15.67:0/2094014497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2ad390 0x55f5fd6740f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3434101d60 tx=0x7f3434090180 comp rx=0 tx=0).stop 2024-07-31T14:51:27.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.881+0000 7f34519c3740 1 -- 172.21.15.67:0/2094014497 shutdown_connections 2024-07-31T14:51:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.881+0000 7f34519c3740 1 --2- 172.21.15.67:0/2094014497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2ad390 0x55f5fd6740f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.881+0000 7f34519c3740 1 --2- 172.21.15.67:0/2094014497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2a51b0 0x55f5fd673bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.881+0000 7f34519c3740 1 --2- 172.21.15.67:0/2094014497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ce070 0x55f5fd673670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.881+0000 7f34519c3740 1 -- 172.21.15.67:0/2094014497 >> 172.21.15.67:0/2094014497 conn(0x55f5fd2c2460 msgr2=0x55f5fd671560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.881+0000 7f34519c3740 1 -- 172.21.15.67:0/2094014497 shutdown_connections 2024-07-31T14:51:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.881+0000 7f34519c3740 1 -- 172.21.15.67:0/2094014497 wait complete. 2024-07-31T14:51:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.881+0000 7f34519c3740 1 -- start start 2024-07-31T14:51:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.881+0000 7f34519c3740 1 Processor -- start 2024-07-31T14:51:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f34519c3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2a51b0 0x55f5fd67f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f34519c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2ad390 0x55f5fd67f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f34519c3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ce070 0x55f5fd68ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f34519c3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f5fd43c700 con 0x55f5fd2a51b0 2024-07-31T14:51:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f34519c3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f5fd57b9c0 con 0x55f5fd2ce070 2024-07-31T14:51:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f34519c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f5fd681c10 con 0x55f5fd2ad390 2024-07-31T14:51:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f34519c3740 10 client.0 _ll_register_callbacks cb 0x55f5fd5964d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f344920e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2a51b0 0x55f5fd67f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f344920e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2a51b0 0x55f5fd67f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45446/0 (socket says 172.21.15.67:45446) 2024-07-31T14:51:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f344920e640 1 -- 172.21.15.67:0/2428164528 learned_addr learned my addr 172.21.15.67:0/2428164528 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f3448a0d640 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2ad390 0x55f5fd67f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f3449a0f640 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ce070 0x55f5fd68ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f344920e640 1 -- 172.21.15.67:0/2428164528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2ad390 msgr2=0x55f5fd67f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f344920e640 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2ad390 0x55f5fd67f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f344920e640 1 -- 172.21.15.67:0/2428164528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ce070 msgr2=0x55f5fd68ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f344920e640 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ce070 0x55f5fd68ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f344920e640 1 -- 172.21.15.67:0/2428164528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5fd339650 con 0x55f5fd2a51b0 2024-07-31T14:51:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f3449a0f640 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ce070 0x55f5fd68ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f3448a0d640 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2ad390 0x55f5fd67f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.882+0000 7f344920e640 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2a51b0 0x55f5fd67f150 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f344000be60 tx=0x7f3440012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8dbd88b8f5f5958 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f342cff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f342cff9640 1 -- 172.21.15.67:0/2428164528 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440004230 con 0x55f5fd2a51b0 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f342cff9640 1 -- 172.21.15.67:0/2428164528 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f34400043d0 con 0x55f5fd2a51b0 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f34519c3740 1 -- 172.21.15.67:0/2428164528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34340cadc0 con 0x55f5fd2a51b0 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f342cff9640 1 -- 172.21.15.67:0/2428164528 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f34400106a0 con 0x55f5fd2a51b0 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f34519c3740 10 client.15639 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f34519c3740 1 -- 172.21.15.67:0/2428164528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f5fd68db20 con 0x55f5fd2a51b0 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f342cff9640 1 -- 172.21.15.67:0/2428164528 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34340cadc0 con 0x55f5fd2a51b0 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f342cff9640 1 -- 172.21.15.67:0/2428164528 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3440019070 con 0x55f5fd2a51b0 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f34519c3740 10 client.15639 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f34519c3740 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f34519c3740 20 client.15639 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f34519c3740 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f5fd28c9f0 0x55f5fd6910f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f34519c3740 4 client.15639 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:51:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.883+0000 7f34519c3740 1 -- 172.21.15.67:0/2428164528 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5fd5ebae0 con 0x55f5fd28c9f0 2024-07-31T14:51:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.884+0000 7f3448a0d640 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f5fd28c9f0 0x55f5fd6910f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.884+0000 7f342cff9640 10 client.15639 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.884+0000 7f3448a0d640 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f5fd28c9f0 0x55f5fd6910f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3516dcab8b8845d4 server_cookie=9529afa2f2ef92aa in_seq=0 out_seq=0 2024-07-31T14:51:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f342cff9640 1 -- 172.21.15.67:0/2428164528 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5fd5ebae0 con 0x55f5fd28c9f0 2024-07-31T14:51:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f342cff9640 10 client.15639 handle_command_reply: tid=0 2024-07-31T14:51:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f34519c3740 1 client.15639 shutdown 2024-07-31T14:51:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f34519c3740 1 -- 172.21.15.67:0/2428164528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2a51b0 msgr2=0x55f5fd67f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f34519c3740 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2a51b0 0x55f5fd67f150 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f344000be60 tx=0x7f3440012820 comp rx=0 tx=0).stop 2024-07-31T14:51:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f34519c3740 1 -- 172.21.15.67:0/2428164528 shutdown_connections 2024-07-31T14:51:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f34519c3740 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f5fd28c9f0 0x55f5fd6910f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f34519c3740 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ce070 0x55f5fd68ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f34519c3740 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2ad390 0x55f5fd67f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f342cff9640 0 client.15639 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:27.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f34519c3740 1 --2- 172.21.15.67:0/2428164528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2a51b0 0x55f5fd67f150 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f34519c3740 1 -- 172.21.15.67:0/2428164528 >> 172.21.15.67:0/2428164528 conn(0x55f5fd2c2460 msgr2=0x55f5fd66fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:27.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f34519c3740 1 -- 172.21.15.67:0/2428164528 shutdown_connections 2024-07-31T14:51:27.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f34519c3740 1 -- 172.21.15.67:0/2428164528 wait complete. 2024-07-31T14:51:27.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.885+0000 7f34519c3740 20 client.15639 trim_cache size 0 max 16384 2024-07-31T14:51:27.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.904+0000 7f34519c3740 1 Processor -- start 2024-07-31T14:51:27.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.904+0000 7f34519c3740 1 -- start start 2024-07-31T14:51:27.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.904+0000 7f34519c3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd28c9f0 0x55f5fd6715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.904+0000 7f34519c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2a51b0 0x55f5fd679790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.904+0000 7f34519c3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ad390 0x55f5fd68ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.904+0000 7f34519c3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f34441b06d0 con 0x55f5fd28c9f0 2024-07-31T14:51:27.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.904+0000 7f34519c3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f34441b2210 con 0x55f5fd2ad390 2024-07-31T14:51:27.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.904+0000 7f34519c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f34441b0da0 con 0x55f5fd2a51b0 2024-07-31T14:51:27.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.904+0000 7f344920e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd28c9f0 0x55f5fd6715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.904+0000 7f344920e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd28c9f0 0x55f5fd6715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45450/0 (socket says 172.21.15.67:45450) 2024-07-31T14:51:27.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.904+0000 7f344920e640 1 -- 172.21.15.67:0/1794036095 learned_addr learned my addr 172.21.15.67:0/1794036095 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:27.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.905+0000 7f3449a0f640 1 --2- 172.21.15.67:0/1794036095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ad390 0x55f5fd68ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.905+0000 7f3448a0d640 1 --2- 172.21.15.67:0/1794036095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2a51b0 0x55f5fd679790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.905+0000 7f344920e640 1 -- 172.21.15.67:0/1794036095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2a51b0 msgr2=0x55f5fd679790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:27.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.905+0000 7f344920e640 1 --2- 172.21.15.67:0/1794036095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2a51b0 0x55f5fd679790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.905+0000 7f344920e640 1 -- 172.21.15.67:0/1794036095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ad390 msgr2=0x55f5fd68ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:27.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.905+0000 7f344920e640 1 --2- 172.21.15.67:0/1794036095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ad390 0x55f5fd68ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.905+0000 7f344920e640 1 -- 172.21.15.67:0/1794036095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5fd680010 con 0x55f5fd28c9f0 2024-07-31T14:51:27.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.905+0000 7f344920e640 1 --2- 172.21.15.67:0/1794036095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd28c9f0 0x55f5fd6715d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3440000c00 tx=0x7f344001b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8a16dae7a07426 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:27.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.905+0000 7f342effd640 1 -- 172.21.15.67:0/1794036095 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344002d020 con 0x55f5fd28c9f0 2024-07-31T14:51:27.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.905+0000 7f342effd640 1 -- 172.21.15.67:0/1794036095 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34400054f0 con 0x55f5fd28c9f0 2024-07-31T14:51:27.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.905+0000 7f342effd640 1 -- 172.21.15.67:0/1794036095 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440010840 con 0x55f5fd28c9f0 2024-07-31T14:51:27.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.906+0000 7f34519c3740 1 -- 172.21.15.67:0/1794036095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd28c9f0 msgr2=0x55f5fd6715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.906+0000 7f34519c3740 1 --2- 172.21.15.67:0/1794036095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd28c9f0 0x55f5fd6715d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3440000c00 tx=0x7f344001b4c0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.906+0000 7f34519c3740 1 -- 172.21.15.67:0/1794036095 shutdown_connections 2024-07-31T14:51:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.906+0000 7f34519c3740 1 --2- 172.21.15.67:0/1794036095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd2ad390 0x55f5fd68ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.906+0000 7f34519c3740 1 --2- 172.21.15.67:0/1794036095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2a51b0 0x55f5fd679790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.906+0000 7f34519c3740 1 --2- 172.21.15.67:0/1794036095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd28c9f0 0x55f5fd6715d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.906+0000 7f34519c3740 1 -- 172.21.15.67:0/1794036095 >> 172.21.15.67:0/1794036095 conn(0x55f5fd2c2460 msgr2=0x55f5fd670ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.906+0000 7f34519c3740 1 -- 172.21.15.67:0/1794036095 shutdown_connections 2024-07-31T14:51:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.906+0000 7f34519c3740 1 -- 172.21.15.67:0/1794036095 wait complete. 2024-07-31T14:51:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.907+0000 7f34519c3740 1 -- start start 2024-07-31T14:51:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.907+0000 7f34519c3740 1 Processor -- start 2024-07-31T14:51:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.907+0000 7f34519c3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd28c9f0 0x55f5fd682580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.907+0000 7f34519c3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2a51b0 0x55f5fd682b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.907+0000 7f34519c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2ad390 0x55f5fd6f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.907+0000 7f34519c3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f34441b1b40 con 0x55f5fd2a51b0 2024-07-31T14:51:27.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.907+0000 7f34519c3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f34441b1470 con 0x55f5fd28c9f0 2024-07-31T14:51:27.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.907+0000 7f34519c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f34441ae6b0 con 0x55f5fd2ad390 2024-07-31T14:51:27.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.908+0000 7f34519c3740 10 client.0 _ll_register_callbacks cb 0x7f344000b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:27.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.908+0000 7f3448a0d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2a51b0 0x55f5fd682b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.908+0000 7f3448a0d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2a51b0 0x55f5fd682b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45454/0 (socket says 172.21.15.67:45454) 2024-07-31T14:51:27.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.908+0000 7f344920e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd28c9f0 0x55f5fd682580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.908+0000 7f3448a0d640 1 -- 172.21.15.67:0/125332108 learned_addr learned my addr 172.21.15.67:0/125332108 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:27.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.908+0000 7f3449a0f640 1 --2- 172.21.15.67:0/125332108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2ad390 0x55f5fd6f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.908+0000 7f3448a0d640 1 -- 172.21.15.67:0/125332108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2ad390 msgr2=0x55f5fd6f18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:27.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.908+0000 7f3448a0d640 1 --2- 172.21.15.67:0/125332108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2ad390 0x55f5fd6f18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.908+0000 7f3448a0d640 1 -- 172.21.15.67:0/125332108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd28c9f0 msgr2=0x55f5fd682580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:27.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.908+0000 7f3448a0d640 1 --2- 172.21.15.67:0/125332108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd28c9f0 0x55f5fd682580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.908+0000 7f3448a0d640 1 -- 172.21.15.67:0/125332108 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5fd6f1f20 con 0x55f5fd2a51b0 2024-07-31T14:51:27.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.908+0000 7f3448a0d640 1 --2- 172.21.15.67:0/125332108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2a51b0 0x55f5fd682b40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3434101ea0 tx=0x7f34340caa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6848921b47bfc004 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:27.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.908+0000 7f342cff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:51:27.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.908+0000 7f342cff9640 1 -- 172.21.15.67:0/125332108 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434104e60 con 0x55f5fd2a51b0 2024-07-31T14:51:27.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.909+0000 7f342cff9640 1 -- 172.21.15.67:0/125332108 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f343403bdc0 con 0x55f5fd2a51b0 2024-07-31T14:51:27.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.909+0000 7f34519c3740 1 -- 172.21.15.67:0/125332108 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34400054f0 con 0x55f5fd2a51b0 2024-07-31T14:51:27.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.909+0000 7f342cff9640 1 -- 172.21.15.67:0/125332108 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3434049040 con 0x55f5fd2a51b0 2024-07-31T14:51:27.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.909+0000 7f34519c3740 10 client.15645 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:27.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.909+0000 7f34519c3740 1 -- 172.21.15.67:0/125332108 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f5fd6f9230 con 0x55f5fd2a51b0 2024-07-31T14:51:27.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.909+0000 7f342cff9640 1 -- 172.21.15.67:0/125332108 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34400054f0 con 0x55f5fd2a51b0 2024-07-31T14:51:27.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.909+0000 7f342cff9640 1 -- 172.21.15.67:0/125332108 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3434094020 con 0x55f5fd2a51b0 2024-07-31T14:51:27.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.909+0000 7f34519c3740 10 client.15645 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:27.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.909+0000 7f34519c3740 10 client.15645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:51:27.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.909+0000 7f34519c3740 20 client.15645 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:27.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.909+0000 7f34519c3740 1 --2- 172.21.15.67:0/125332108 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f5fd2ce070 0x55f5fd6fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:27.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.909+0000 7f34519c3740 4 client.15645 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:51:27.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.909+0000 7f34519c3740 1 -- 172.21.15.67:0/125332108 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f5fd5e7580 con 0x55f5fd2ce070 2024-07-31T14:51:27.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.909+0000 7f344920e640 1 --2- 172.21.15.67:0/125332108 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f5fd2ce070 0x55f5fd6fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:27.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.910+0000 7f342cff9640 10 client.15645 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:27.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.910+0000 7f344920e640 1 --2- 172.21.15.67:0/125332108 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f5fd2ce070 0x55f5fd6fb5e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87131547cf6a1343 server_cookie=e31490df2a17b886 in_seq=0 out_seq=0 2024-07-31T14:51:27.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f342cff9640 1 -- 172.21.15.67:0/125332108 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f5fd5e7580 con 0x55f5fd2ce070 2024-07-31T14:51:27.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f342cff9640 10 client.15645 handle_command_reply: tid=0 2024-07-31T14:51:27.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f34519c3740 1 client.15645 shutdown 2024-07-31T14:51:27.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f34519c3740 1 -- 172.21.15.67:0/125332108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2a51b0 msgr2=0x55f5fd682b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:27.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f34519c3740 1 --2- 172.21.15.67:0/125332108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2a51b0 0x55f5fd682b40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3434101ea0 tx=0x7f34340caa20 comp rx=0 tx=0).stop 2024-07-31T14:51:27.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f34519c3740 1 -- 172.21.15.67:0/125332108 shutdown_connections 2024-07-31T14:51:27.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f34519c3740 1 --2- 172.21.15.67:0/125332108 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f5fd2ce070 0x55f5fd6fb5e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f34519c3740 1 --2- 172.21.15.67:0/125332108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5fd2ad390 0x55f5fd6f18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f34519c3740 1 --2- 172.21.15.67:0/125332108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5fd2a51b0 0x55f5fd682b40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f34519c3740 1 --2- 172.21.15.67:0/125332108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5fd28c9f0 0x55f5fd682580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f342cff9640 0 client.15645 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:27.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f34519c3740 1 -- 172.21.15.67:0/125332108 >> 172.21.15.67:0/125332108 conn(0x55f5fd2c2460 msgr2=0x55f5fd6746a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:27.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f34519c3740 1 -- 172.21.15.67:0/125332108 shutdown_connections 2024-07-31T14:51:27.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f34519c3740 1 -- 172.21.15.67:0/125332108 wait complete. 2024-07-31T14:51:27.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.911+0000 7f34519c3740 20 client.15645 trim_cache size 0 max 16384 2024-07-31T14:51:27.917 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:51:27.917 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-31T14:51:27.917 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T14:51:27.917 INFO:teuthology.orchestra.run.smithi067.stdout: "114ad17e-0968-446c-932a-ee7bc3eeb28f": { 2024-07-31T14:51:27.918 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T14:51:27.918 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "114ad17e-0968-446c-932a-ee7bc3eeb28f", 2024-07-31T14:51:27.918 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T14:51:27.918 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:51:27.918 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:51:27.918 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:51:27.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.912+0000 7f344b472640 1 -- 172.21.15.67:0/1519264373 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f34300b3d40 msgr2=0x7f34300b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:27.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.912+0000 7f344b472640 1 --2- 172.21.15.67:0/1519264373 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f34300b3d40 0x7f34300b6250 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f343c0125f0 tx=0x7f343c004a40 comp rx=0 tx=0).stop 2024-07-31T14:51:27.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.912+0000 7f344b472640 1 -- 172.21.15.67:0/1519264373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3444108ed0 msgr2=0x7f344419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:27.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.912+0000 7f344b472640 1 --2- 172.21.15.67:0/1519264373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3444108ed0 0x7f344419a4c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f343402a930 tx=0x7f3434004270 comp rx=0 tx=0).stop 2024-07-31T14:51:27.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.912+0000 7f344b472640 1 -- 172.21.15.67:0/1519264373 shutdown_connections 2024-07-31T14:51:27.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.912+0000 7f344b472640 1 --2- 172.21.15.67:0/1519264373 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f34300b3d40 0x7f34300b6250 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.912+0000 7f344b472640 1 --2- 172.21.15.67:0/1519264373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3444109820 0x7f344419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.912+0000 7f344b472640 1 --2- 172.21.15.67:0/1519264373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3444108ed0 0x7f344419a4c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.912+0000 7f344b472640 1 --2- 172.21.15.67:0/1519264373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3444103590 0x7f3444199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:27.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.912+0000 7f344b472640 1 -- 172.21.15.67:0/1519264373 >> 172.21.15.67:0/1519264373 conn(0x7f344407b280 msgr2=0x7f3444106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:27.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.912+0000 7f344b472640 1 -- 172.21.15.67:0/1519264373 shutdown_connections 2024-07-31T14:51:27.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:27.912+0000 7f344b472640 1 -- 172.21.15.67:0/1519264373 wait complete. 2024-07-31T14:51:28.630 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:28.664 INFO:tasks.cephfs.filesystem:scrub status for tag:114ad17e-0968-446c-932a-ee7bc3eeb28f - {'path': '/', 'tag': '114ad17e-0968-446c-932a-ee7bc3eeb28f', 'options': 'recursive,force'} 2024-07-31T14:51:28.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:28 smithi073 ceph-mon[36714]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-31T14:51:28.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:28 smithi077 ceph-mon[37327]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-31T14:51:28.992 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:28 smithi067 ceph-mon[32750]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-31T14:51:30.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.363+0000 7f5401eba640 1 -- 172.21.15.67:0/1880941871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53fc105d00 msgr2=0x7f53fc106110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:30.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.363+0000 7f5401eba640 1 --2- 172.21.15.67:0/1880941871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53fc105d00 0x7f53fc106110 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009900 tx=0x7f53e402a410 comp rx=0 tx=0).stop 2024-07-31T14:51:30.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.363+0000 7f5401eba640 1 -- 172.21.15.67:0/1880941871 shutdown_connections 2024-07-31T14:51:30.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.363+0000 7f5401eba640 1 --2- 172.21.15.67:0/1880941871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53fc106650 0x7f53fc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:30.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.363+0000 7f5401eba640 1 --2- 172.21.15.67:0/1880941871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53fc105d00 0x7f53fc106110 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:30.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.363+0000 7f5401eba640 1 --2- 172.21.15.67:0/1880941871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc101b10 0x7f53fc101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:30.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.363+0000 7f5401eba640 1 -- 172.21.15.67:0/1880941871 >> 172.21.15.67:0/1880941871 conn(0x7f53fc079870 msgr2=0x7f53fc079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:30.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.364+0000 7f5401eba640 1 -- 172.21.15.67:0/1880941871 shutdown_connections 2024-07-31T14:51:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.364+0000 7f5401eba640 1 -- 172.21.15.67:0/1880941871 wait complete. 2024-07-31T14:51:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.364+0000 7f5401eba640 1 Processor -- start 2024-07-31T14:51:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.364+0000 7f5401eba640 1 -- start start 2024-07-31T14:51:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f5401eba640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc101b10 0x7f53fc102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f5401eba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53fc105d00 0x7f53fc102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f5401eba640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53fc106650 0x7f53fc103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f5401eba640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc1134c0 con 0x7f53fc106650 2024-07-31T14:51:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f5401eba640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc113340 con 0x7f53fc101b10 2024-07-31T14:51:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f5401eba640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc113640 con 0x7f53fc105d00 2024-07-31T14:51:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f53fbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53fc106650 0x7f53fc103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f53fbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53fc106650 0x7f53fc103530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45474/0 (socket says 172.21.15.67:45474) 2024-07-31T14:51:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f53fb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc101b10 0x7f53fc102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f53fbfff640 1 -- 172.21.15.67:0/2004572247 learned_addr learned my addr 172.21.15.67:0/2004572247 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f53faffd640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53fc105d00 0x7f53fc102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f53fbfff640 1 -- 172.21.15.67:0/2004572247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53fc105d00 msgr2=0x7f53fc102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f53fbfff640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53fc105d00 0x7f53fc102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f53fbfff640 1 -- 172.21.15.67:0/2004572247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc101b10 msgr2=0x7f53fc102ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f53fbfff640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc101b10 0x7f53fc102ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f53fbfff640 1 -- 172.21.15.67:0/2004572247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53fc1a8190 con 0x7f53fc106650 2024-07-31T14:51:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f53faffd640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53fc105d00 0x7f53fc102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f53fb7fe640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc101b10 0x7f53fc102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.365+0000 7f53fbfff640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53fc106650 0x7f53fc103530 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009fd0 tx=0x7f53f400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.366+0000 7f53f8ff9640 1 -- 172.21.15.67:0/2004572247 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4017070 con 0x7f53fc106650 2024-07-31T14:51:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.366+0000 7f5401eba640 1 -- 172.21.15.67:0/2004572247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53fc1a8480 con 0x7f53fc106650 2024-07-31T14:51:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.366+0000 7f53f8ff9640 1 -- 172.21.15.67:0/2004572247 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53f4003e20 con 0x7f53fc106650 2024-07-31T14:51:30.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.366+0000 7f53f8ff9640 1 -- 172.21.15.67:0/2004572247 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4010460 con 0x7f53fc106650 2024-07-31T14:51:30.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.366+0000 7f5401eba640 1 -- 172.21.15.67:0/2004572247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53fc1a8940 con 0x7f53fc106650 2024-07-31T14:51:30.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.367+0000 7f5401eba640 1 -- 172.21.15.67:0/2004572247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53fc101f20 con 0x7f53fc106650 2024-07-31T14:51:30.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.368+0000 7f53f8ff9640 1 -- 172.21.15.67:0/2004572247 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f53f4003970 con 0x7f53fc106650 2024-07-31T14:51:30.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.369+0000 7f53f8ff9640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f53e80b3d40 0x7f53e80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:30.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.369+0000 7f53fb7fe640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f53e80b3d40 0x7f53e80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:30.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.369+0000 7f53f8ff9640 1 -- 172.21.15.67:0/2004572247 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f53f40e4070 con 0x7f53fc106650 2024-07-31T14:51:30.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.370+0000 7f53fb7fe640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f53e80b3d40 0x7f53e80b6250 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f53fc104500 tx=0x7f53ec005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:30.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.372+0000 7f53f8ff9640 1 -- 172.21.15.67:0/2004572247 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f53f40ab070 con 0x7f53fc106650 2024-07-31T14:51:30.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.562+0000 7f5401eba640 1 -- 172.21.15.67:0/2004572247 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53fc063230 con 0x7f53e80b3d40 2024-07-31T14:51:30.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.567+0000 7f53f8ff9640 1 -- 172.21.15.67:0/2004572247 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340592 (secure 0 0 0) 0x7f53fc063230 con 0x7f53e80b3d40 2024-07-31T14:51:30.572 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:51:30.573 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-31T14:51:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.570+0000 7f5401eba640 1 -- 172.21.15.67:0/2004572247 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f53e80b3d40 msgr2=0x7f53e80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.570+0000 7f5401eba640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f53e80b3d40 0x7f53e80b6250 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f53fc104500 tx=0x7f53ec005f90 comp rx=0 tx=0).stop 2024-07-31T14:51:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.570+0000 7f5401eba640 1 -- 172.21.15.67:0/2004572247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53fc106650 msgr2=0x7f53fc103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.570+0000 7f5401eba640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53fc106650 0x7f53fc103530 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009fd0 tx=0x7f53f400b260 comp rx=0 tx=0).stop 2024-07-31T14:51:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.571+0000 7f5401eba640 1 -- 172.21.15.67:0/2004572247 shutdown_connections 2024-07-31T14:51:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.571+0000 7f5401eba640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f53e80b3d40 0x7f53e80b6250 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.571+0000 7f5401eba640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53fc106650 0x7f53fc103530 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.571+0000 7f5401eba640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53fc105d00 0x7f53fc102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.571+0000 7f5401eba640 1 --2- 172.21.15.67:0/2004572247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc101b10 0x7f53fc102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.571+0000 7f5401eba640 1 -- 172.21.15.67:0/2004572247 >> 172.21.15.67:0/2004572247 conn(0x7f53fc079870 msgr2=0x7f53fc107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.571+0000 7f5401eba640 1 -- 172.21.15.67:0/2004572247 shutdown_connections 2024-07-31T14:51:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:30.571+0000 7f5401eba640 1 -- 172.21.15.67:0/2004572247 wait complete. 2024-07-31T14:51:30.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:30 smithi073 ceph-mon[36714]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-31T14:51:30.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:30 smithi077 ceph-mon[37327]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-31T14:51:31.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:30 smithi067 ceph-mon[32750]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-31T14:51:31.314 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-07-31T14:51:29.448352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1458620,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2502,"num_read_kb":1500,"num_write":626,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":974,"ondisk_log_size":974,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386616,"kb_used_data":16484,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555208,"statfs":{"total":1151940427776,"available":1151544532992,"internally_reserved":0,"allocated":16879616,"data_stored":10581230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":378713403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":786992,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":451,"num_read_kb":356,"num_write":224,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003554"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":261,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:24.912708+0000","last_change":"2024-07-31T14:47:00.020067+0000","last_active":"2024-07-31T14:51:24.912708+0000","last_peered":"2024-07-31T14:51:24.912708+0000","last_clean":"2024-07-31T14:51:24.912708+0000","last_became_active":"2024-07-31T14:47:00.019674+0000","last_became_peered":"2024-07-31T14:47:00.019674+0000","last_unstale":"2024-07-31T14:51:24.912708+0000","last_undegraded":"2024-07-31T14:51:24.912708+0000","last_fullsized":"2024-07-31T14:51:24.912708+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:10:08.193786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":205,"num_read_kb":205,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.408739+0000","last_change":"2024-07-31T14:47:03.052069+0000","last_active":"2024-07-31T14:49:49.408739+0000","last_peered":"2024-07-31T14:49:49.408739+0000","last_clean":"2024-07-31T14:49:49.408739+0000","last_became_active":"2024-07-31T14:47:03.051739+0000","last_became_peered":"2024-07-31T14:47:03.051739+0000","last_unstale":"2024-07-31T14:49:49.408739+0000","last_undegraded":"2024-07-31T14:49:49.408739+0000","last_fullsized":"2024-07-31T14:49:49.408739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:29:19.952794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.629384+0000","last_change":"2024-07-31T14:47:20.628770+0000","last_active":"2024-07-31T14:47:20.629384+0000","last_peered":"2024-07-31T14:47:20.629384+0000","last_clean":"2024-07-31T14:47:20.629384+0000","last_became_active":"2024-07-31T14:47:20.628142+0000","last_became_peered":"2024-07-31T14:47:20.628142+0000","last_unstale":"2024-07-31T14:47:20.629384+0000","last_undegraded":"2024-07-31T14:47:20.629384+0000","last_fullsized":"2024-07-31T14:47:20.629384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.659670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:59.291402+0000","last_change":"2024-07-31T14:47:00.183893+0000","last_active":"2024-07-31T14:48:59.291402+0000","last_peered":"2024-07-31T14:48:59.291402+0000","last_clean":"2024-07-31T14:48:59.291402+0000","last_became_active":"2024-07-31T14:47:00.183453+0000","last_became_peered":"2024-07-31T14:47:00.183453+0000","last_unstale":"2024-07-31T14:48:59.291402+0000","last_undegraded":"2024-07-31T14:48:59.291402+0000","last_fullsized":"2024-07-31T14:48:59.291402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:45:08.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404383+0000","last_change":"2024-07-31T14:47:03.057081+0000","last_active":"2024-07-31T14:49:49.404383+0000","last_peered":"2024-07-31T14:49:49.404383+0000","last_clean":"2024-07-31T14:49:49.404383+0000","last_became_active":"2024-07-31T14:47:03.056061+0000","last_became_peered":"2024-07-31T14:47:03.056061+0000","last_unstale":"2024-07-31T14:49:49.404383+0000","last_undegraded":"2024-07-31T14:49:49.404383+0000","last_fullsized":"2024-07-31T14:49:49.404383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:28:16.760131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604121+0000","last_change":"2024-07-31T14:47:13.105271+0000","last_active":"2024-07-31T14:47:20.604121+0000","last_peered":"2024-07-31T14:47:20.604121+0000","last_clean":"2024-07-31T14:47:20.604121+0000","last_became_active":"2024-07-31T14:47:13.104717+0000","last_became_peered":"2024-07-31T14:47:13.104717+0000","last_unstale":"2024-07-31T14:47:20.604121+0000","last_undegraded":"2024-07-31T14:47:20.604121+0000","last_fullsized":"2024-07-31T14:47:20.604121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:34:21.413957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:04.594438+0000","last_change":"2024-07-31T14:47:20.454745+0000","last_active":"2024-07-31T14:48:04.594438+0000","last_peered":"2024-07-31T14:48:04.594438+0000","last_clean":"2024-07-31T14:48:04.594438+0000","last_became_active":"2024-07-31T14:47:20.449892+0000","last_became_peered":"2024-07-31T14:47:20.449892+0000","last_unstale":"2024-07-31T14:48:04.594438+0000","last_undegraded":"2024-07-31T14:48:04.594438+0000","last_fullsized":"2024-07-31T14:48:04.594438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:59:17.375344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.412662+0000","last_change":"2024-07-31T14:47:03.051085+0000","last_active":"2024-07-31T14:49:49.412662+0000","last_peered":"2024-07-31T14:49:49.412662+0000","last_clean":"2024-07-31T14:49:49.412662+0000","last_became_active":"2024-07-31T14:47:03.050686+0000","last_became_peered":"2024-07-31T14:47:03.050686+0000","last_unstale":"2024-07-31T14:49:49.412662+0000","last_undegraded":"2024-07-31T14:49:49.412662+0000","last_fullsized":"2024-07-31T14:49:49.412662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:48:02.494061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604501+0000","last_change":"2024-07-31T14:47:13.104229+0000","last_active":"2024-07-31T14:47:20.604501+0000","last_peered":"2024-07-31T14:47:20.604501+0000","last_clean":"2024-07-31T14:47:20.604501+0000","last_became_active":"2024-07-31T14:47:13.103737+0000","last_became_peered":"2024-07-31T14:47:13.103737+0000","last_unstale":"2024-07-31T14:47:20.604501+0000","last_undegraded":"2024-07-31T14:47:20.604501+0000","last_fullsized":"2024-07-31T14:47:20.604501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:36:00.541345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609858+0000","last_change":"2024-07-31T14:47:00.028169+0000","last_active":"2024-07-31T14:48:03.609858+0000","last_peered":"2024-07-31T14:48:03.609858+0000","last_clean":"2024-07-31T14:48:03.609858+0000","last_became_active":"2024-07-31T14:47:00.027757+0000","last_became_peered":"2024-07-31T14:47:00.027757+0000","last_unstale":"2024-07-31T14:48:03.609858+0000","last_undegraded":"2024-07-31T14:48:03.609858+0000","last_fullsized":"2024-07-31T14:48:03.609858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:36:04.419551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413724+0000","last_change":"2024-07-31T14:47:03.057400+0000","last_active":"2024-07-31T14:49:49.413724+0000","last_peered":"2024-07-31T14:49:49.413724+0000","last_clean":"2024-07-31T14:49:49.413724+0000","last_became_active":"2024-07-31T14:47:03.057035+0000","last_became_peered":"2024-07-31T14:47:03.057035+0000","last_unstale":"2024-07-31T14:49:49.413724+0000","last_undegraded":"2024-07-31T14:49:49.413724+0000","last_fullsized":"2024-07-31T14:49:49.413724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:55:07.951076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604244+0000","last_change":"2024-07-31T14:47:13.106093+0000","last_active":"2024-07-31T14:47:20.604244+0000","last_peered":"2024-07-31T14:47:20.604244+0000","last_clean":"2024-07-31T14:47:20.604244+0000","last_became_active":"2024-07-31T14:47:13.105459+0000","last_became_peered":"2024-07-31T14:47:13.105459+0000","last_unstale":"2024-07-31T14:47:20.604244+0000","last_undegraded":"2024-07-31T14:47:20.604244+0000","last_fullsized":"2024-07-31T14:47:20.604244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:07:52.751245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414509+0000","last_change":"2024-07-31T14:47:03.058701+0000","last_active":"2024-07-31T14:49:49.414509+0000","last_peered":"2024-07-31T14:49:49.414509+0000","last_clean":"2024-07-31T14:49:49.414509+0000","last_became_active":"2024-07-31T14:47:03.058359+0000","last_became_peered":"2024-07-31T14:47:03.058359+0000","last_unstale":"2024-07-31T14:49:49.414509+0000","last_undegraded":"2024-07-31T14:49:49.414509+0000","last_fullsized":"2024-07-31T14:49:49.414509+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:37:36.149884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605126+0000","last_change":"2024-07-31T14:47:00.022983+0000","last_active":"2024-07-31T14:48:03.605126+0000","last_peered":"2024-07-31T14:48:03.605126+0000","last_clean":"2024-07-31T14:48:03.605126+0000","last_became_active":"2024-07-31T14:47:00.022586+0000","last_became_peered":"2024-07-31T14:47:00.022586+0000","last_unstale":"2024-07-31T14:48:03.605126+0000","last_undegraded":"2024-07-31T14:48:03.605126+0000","last_fullsized":"2024-07-31T14:48:03.605126+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:29:42.641490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604474+0000","last_change":"2024-07-31T14:47:13.363148+0000","last_active":"2024-07-31T14:47:20.604474+0000","last_peered":"2024-07-31T14:47:20.604474+0000","last_clean":"2024-07-31T14:47:20.604474+0000","last_became_active":"2024-07-31T14:47:13.362577+0000","last_became_peered":"2024-07-31T14:47:13.362577+0000","last_unstale":"2024-07-31T14:47:20.604474+0000","last_undegraded":"2024-07-31T14:47:20.604474+0000","last_fullsized":"2024-07-31T14:47:20.604474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:37:59.003260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409563+0000","last_change":"2024-07-31T14:47:03.049586+0000","last_active":"2024-07-31T14:49:49.409563+0000","last_peered":"2024-07-31T14:49:49.409563+0000","last_clean":"2024-07-31T14:49:49.409563+0000","last_became_active":"2024-07-31T14:47:03.049154+0000","last_became_peered":"2024-07-31T14:47:03.049154+0000","last_unstale":"2024-07-31T14:49:49.409563+0000","last_undegraded":"2024-07-31T14:49:49.409563+0000","last_fullsized":"2024-07-31T14:49:49.409563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:51:01.971854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:09.610387+0000","last_change":"2024-07-31T14:47:00.021254+0000","last_active":"2024-07-31T14:51:09.610387+0000","last_peered":"2024-07-31T14:51:09.610387+0000","last_clean":"2024-07-31T14:51:09.610387+0000","last_became_active":"2024-07-31T14:47:00.020690+0000","last_became_peered":"2024-07-31T14:47:00.020690+0000","last_unstale":"2024-07-31T14:51:09.610387+0000","last_undegraded":"2024-07-31T14:51:09.610387+0000","last_fullsized":"2024-07-31T14:51:09.610387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:01:27.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:18.046730+0000","last_change":"2024-07-31T14:47:13.102220+0000","last_active":"2024-07-31T14:48:18.046730+0000","last_peered":"2024-07-31T14:48:18.046730+0000","last_clean":"2024-07-31T14:48:18.046730+0000","last_became_active":"2024-07-31T14:47:13.101764+0000","last_became_peered":"2024-07-31T14:47:13.101764+0000","last_unstale":"2024-07-31T14:48:18.046730+0000","last_undegraded":"2024-07-31T14:48:18.046730+0000","last_fullsized":"2024-07-31T14:48:18.046730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:56:44.335340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.398408+0000","last_change":"2024-07-31T14:47:03.052348+0000","last_active":"2024-07-31T14:49:49.398408+0000","last_peered":"2024-07-31T14:49:49.398408+0000","last_clean":"2024-07-31T14:49:49.398408+0000","last_became_active":"2024-07-31T14:47:03.051961+0000","last_became_peered":"2024-07-31T14:47:03.051961+0000","last_unstale":"2024-07-31T14:49:49.398408+0000","last_undegraded":"2024-07-31T14:49:49.398408+0000","last_fullsized":"2024-07-31T14:49:49.398408+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:19:35.279809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:09.292622+0000","last_change":"2024-07-31T14:47:00.025013+0000","last_active":"2024-07-31T14:49:09.292622+0000","last_peered":"2024-07-31T14:49:09.292622+0000","last_clean":"2024-07-31T14:49:09.292622+0000","last_became_active":"2024-07-31T14:47:00.024637+0000","last_became_peered":"2024-07-31T14:47:00.024637+0000","last_unstale":"2024-07-31T14:49:09.292622+0000","last_undegraded":"2024-07-31T14:49:09.292622+0000","last_fullsized":"2024-07-31T14:49:09.292622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:07:21.519549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630339+0000","last_change":"2024-07-31T14:47:20.629262+0000","last_active":"2024-07-31T14:47:20.630339+0000","last_peered":"2024-07-31T14:47:20.630339+0000","last_clean":"2024-07-31T14:47:20.630339+0000","last_became_active":"2024-07-31T14:47:20.628318+0000","last_became_peered":"2024-07-31T14:47:20.628318+0000","last_unstale":"2024-07-31T14:47:20.630339+0000","last_undegraded":"2024-07-31T14:47:20.630339+0000","last_fullsized":"2024-07-31T14:47:20.630339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.206705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.387553+0000","last_change":"2024-07-31T14:47:03.051527+0000","last_active":"2024-07-31T14:49:49.387553+0000","last_peered":"2024-07-31T14:49:49.387553+0000","last_clean":"2024-07-31T14:49:49.387553+0000","last_became_active":"2024-07-31T14:47:03.051139+0000","last_became_peered":"2024-07-31T14:47:03.051139+0000","last_unstale":"2024-07-31T14:49:49.387553+0000","last_undegraded":"2024-07-31T14:49:49.387553+0000","last_fullsized":"2024-07-31T14:49:49.387553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:53:11.962719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:09.210377+0000","last_change":"2024-07-31T14:47:00.183812+0000","last_active":"2024-07-31T14:49:09.210377+0000","last_peered":"2024-07-31T14:49:09.210377+0000","last_clean":"2024-07-31T14:49:09.210377+0000","last_became_active":"2024-07-31T14:47:00.183440+0000","last_became_peered":"2024-07-31T14:47:00.183440+0000","last_unstale":"2024-07-31T14:49:09.210377+0000","last_undegraded":"2024-07-31T14:49:09.210377+0000","last_fullsized":"2024-07-31T14:49:09.210377+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:47:23.166756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604338+0000","last_change":"2024-07-31T14:47:13.363563+0000","last_active":"2024-07-31T14:47:20.604338+0000","last_peered":"2024-07-31T14:47:20.604338+0000","last_clean":"2024-07-31T14:47:20.604338+0000","last_became_active":"2024-07-31T14:47:13.362697+0000","last_became_peered":"2024-07-31T14:47:13.362697+0000","last_unstale":"2024-07-31T14:47:20.604338+0000","last_undegraded":"2024-07-31T14:47:20.604338+0000","last_fullsized":"2024-07-31T14:47:20.604338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:26:38.292073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407835+0000","last_change":"2024-07-31T14:47:03.046763+0000","last_active":"2024-07-31T14:49:49.407835+0000","last_peered":"2024-07-31T14:49:49.407835+0000","last_clean":"2024-07-31T14:49:49.407835+0000","last_became_active":"2024-07-31T14:47:03.046408+0000","last_became_peered":"2024-07-31T14:47:03.046408+0000","last_unstale":"2024-07-31T14:49:49.407835+0000","last_undegraded":"2024-07-31T14:49:49.407835+0000","last_fullsized":"2024-07-31T14:49:49.407835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:43:22.838495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:09.380966+0000","last_change":"2024-07-31T14:47:00.236075+0000","last_active":"2024-07-31T14:49:09.380966+0000","last_peered":"2024-07-31T14:49:09.380966+0000","last_clean":"2024-07-31T14:49:09.380966+0000","last_became_active":"2024-07-31T14:47:00.235758+0000","last_became_peered":"2024-07-31T14:47:00.235758+0000","last_unstale":"2024-07-31T14:49:09.380966+0000","last_undegraded":"2024-07-31T14:49:09.380966+0000","last_fullsized":"2024-07-31T14:49:09.380966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:50.575648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:21.453373+0000","last_change":"2024-07-31T14:47:13.101876+0000","last_active":"2024-07-31T14:49:21.453373+0000","last_peered":"2024-07-31T14:49:21.453373+0000","last_clean":"2024-07-31T14:49:21.453373+0000","last_became_active":"2024-07-31T14:47:13.101276+0000","last_became_peered":"2024-07-31T14:47:13.101276+0000","last_unstale":"2024-07-31T14:49:21.453373+0000","last_undegraded":"2024-07-31T14:49:21.453373+0000","last_fullsized":"2024-07-31T14:49:21.453373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:53:24.684212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409565+0000","last_change":"2024-07-31T14:47:03.052478+0000","last_active":"2024-07-31T14:49:49.409565+0000","last_peered":"2024-07-31T14:49:49.409565+0000","last_clean":"2024-07-31T14:49:49.409565+0000","last_became_active":"2024-07-31T14:47:03.051877+0000","last_became_peered":"2024-07-31T14:47:03.051877+0000","last_unstale":"2024-07-31T14:49:49.409565+0000","last_undegraded":"2024-07-31T14:49:49.409565+0000","last_fullsized":"2024-07-31T14:49:49.409565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:30:32.666490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'101","reported_seq":232,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:22.587242+0000","last_change":"2024-07-31T14:47:00.020076+0000","last_active":"2024-07-31T14:51:22.587242+0000","last_peered":"2024-07-31T14:51:22.587242+0000","last_clean":"2024-07-31T14:51:22.587242+0000","last_became_active":"2024-07-31T14:47:00.019708+0000","last_became_peered":"2024-07-31T14:47:00.019708+0000","last_unstale":"2024-07-31T14:51:22.587242+0000","last_undegraded":"2024-07-31T14:51:22.587242+0000","last_fullsized":"2024-07-31T14:51:22.587242+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:46.520997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":186,"num_write_kb":1223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:14.293424+0000","last_change":"2024-07-31T14:47:13.362961+0000","last_active":"2024-07-31T14:49:14.293424+0000","last_peered":"2024-07-31T14:49:14.293424+0000","last_clean":"2024-07-31T14:49:14.293424+0000","last_became_active":"2024-07-31T14:47:13.362502+0000","last_became_peered":"2024-07-31T14:47:13.362502+0000","last_unstale":"2024-07-31T14:49:14.293424+0000","last_undegraded":"2024-07-31T14:49:14.293424+0000","last_fullsized":"2024-07-31T14:49:14.293424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:51:45.732477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399783+0000","last_change":"2024-07-31T14:47:03.039418+0000","last_active":"2024-07-31T14:49:49.399783+0000","last_peered":"2024-07-31T14:49:49.399783+0000","last_clean":"2024-07-31T14:49:49.399783+0000","last_became_active":"2024-07-31T14:47:03.039044+0000","last_became_peered":"2024-07-31T14:47:03.039044+0000","last_unstale":"2024-07-31T14:49:49.399783+0000","last_undegraded":"2024-07-31T14:49:49.399783+0000","last_fullsized":"2024-07-31T14:49:49.399783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:40:10.380080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:07.059440+0000","last_change":"2024-07-31T14:47:00.183778+0000","last_active":"2024-07-31T14:49:07.059440+0000","last_peered":"2024-07-31T14:49:07.059440+0000","last_clean":"2024-07-31T14:49:07.059440+0000","last_became_active":"2024-07-31T14:47:00.183370+0000","last_became_peered":"2024-07-31T14:47:00.183370+0000","last_unstale":"2024-07-31T14:49:07.059440+0000","last_undegraded":"2024-07-31T14:49:07.059440+0000","last_fullsized":"2024-07-31T14:49:07.059440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:45:27.422216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:19.188833+0000","last_change":"2024-07-31T14:47:13.102583+0000","last_active":"2024-07-31T14:49:19.188833+0000","last_peered":"2024-07-31T14:49:19.188833+0000","last_clean":"2024-07-31T14:49:19.188833+0000","last_became_active":"2024-07-31T14:47:13.102058+0000","last_became_peered":"2024-07-31T14:47:13.102058+0000","last_unstale":"2024-07-31T14:49:19.188833+0000","last_undegraded":"2024-07-31T14:49:19.188833+0000","last_fullsized":"2024-07-31T14:49:19.188833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:17.396173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.408095+0000","last_change":"2024-07-31T14:47:03.048334+0000","last_active":"2024-07-31T14:49:49.408095+0000","last_peered":"2024-07-31T14:49:49.408095+0000","last_clean":"2024-07-31T14:49:49.408095+0000","last_became_active":"2024-07-31T14:47:03.047932+0000","last_became_peered":"2024-07-31T14:47:03.047932+0000","last_unstale":"2024-07-31T14:49:49.408095+0000","last_undegraded":"2024-07-31T14:49:49.408095+0000","last_fullsized":"2024-07-31T14:49:49.408095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:14:31.381061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:06.494830+0000","last_change":"2024-07-31T14:47:20.450851+0000","last_active":"2024-07-31T14:49:06.494830+0000","last_peered":"2024-07-31T14:49:06.494830+0000","last_clean":"2024-07-31T14:49:06.494830+0000","last_became_active":"2024-07-31T14:47:20.450435+0000","last_became_peered":"2024-07-31T14:47:20.450435+0000","last_unstale":"2024-07-31T14:49:06.494830+0000","last_undegraded":"2024-07-31T14:49:06.494830+0000","last_fullsized":"2024-07-31T14:49:06.494830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:17:28.738911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:22.061505+0000","last_change":"2024-07-31T14:47:13.363054+0000","last_active":"2024-07-31T14:49:22.061505+0000","last_peered":"2024-07-31T14:49:22.061505+0000","last_clean":"2024-07-31T14:49:22.061505+0000","last_became_active":"2024-07-31T14:47:13.362364+0000","last_became_peered":"2024-07-31T14:47:13.362364+0000","last_unstale":"2024-07-31T14:49:22.061505+0000","last_undegraded":"2024-07-31T14:49:22.061505+0000","last_fullsized":"2024-07-31T14:49:22.061505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:02:36.569678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.393338+0000","last_change":"2024-07-31T14:47:03.059834+0000","last_active":"2024-07-31T14:49:49.393338+0000","last_peered":"2024-07-31T14:49:49.393338+0000","last_clean":"2024-07-31T14:49:49.393338+0000","last_became_active":"2024-07-31T14:47:03.059127+0000","last_became_peered":"2024-07-31T14:47:03.059127+0000","last_unstale":"2024-07-31T14:49:49.393338+0000","last_undegraded":"2024-07-31T14:49:49.393338+0000","last_fullsized":"2024-07-31T14:49:49.393338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:53:50.308789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":256,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:24.913236+0000","last_change":"2024-07-31T14:47:00.030576+0000","last_active":"2024-07-31T14:51:24.913236+0000","last_peered":"2024-07-31T14:51:24.913236+0000","last_clean":"2024-07-31T14:51:24.913236+0000","last_became_active":"2024-07-31T14:47:00.030174+0000","last_became_peered":"2024-07-31T14:47:00.030174+0000","last_unstale":"2024-07-31T14:51:24.913236+0000","last_undegraded":"2024-07-31T14:51:24.913236+0000","last_fullsized":"2024-07-31T14:51:24.913236+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:13:35.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":206,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:18.289527+0000","last_change":"2024-07-31T14:47:13.100180+0000","last_active":"2024-07-31T14:49:18.289527+0000","last_peered":"2024-07-31T14:49:18.289527+0000","last_clean":"2024-07-31T14:49:18.289527+0000","last_became_active":"2024-07-31T14:47:13.099725+0000","last_became_peered":"2024-07-31T14:47:13.099725+0000","last_unstale":"2024-07-31T14:49:18.289527+0000","last_undegraded":"2024-07-31T14:49:18.289527+0000","last_fullsized":"2024-07-31T14:49:18.289527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:25:32.245339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404277+0000","last_change":"2024-07-31T14:47:03.040349+0000","last_active":"2024-07-31T14:49:49.404277+0000","last_peered":"2024-07-31T14:49:49.404277+0000","last_clean":"2024-07-31T14:49:49.404277+0000","last_became_active":"2024-07-31T14:47:03.039887+0000","last_became_peered":"2024-07-31T14:47:03.039887+0000","last_unstale":"2024-07-31T14:49:49.404277+0000","last_undegraded":"2024-07-31T14:49:49.404277+0000","last_fullsized":"2024-07-31T14:49:49.404277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:46.592063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":256,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:26.902207+0000","last_change":"2024-07-31T14:47:00.183988+0000","last_active":"2024-07-31T14:51:26.902207+0000","last_peered":"2024-07-31T14:51:26.902207+0000","last_clean":"2024-07-31T14:51:26.902207+0000","last_became_active":"2024-07-31T14:47:00.183302+0000","last_became_peered":"2024-07-31T14:47:00.183302+0000","last_unstale":"2024-07-31T14:51:26.902207+0000","last_undegraded":"2024-07-31T14:51:26.902207+0000","last_fullsized":"2024-07-31T14:51:26.902207+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:40:26.188224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447744+0000","last_change":"2024-07-31T14:47:13.363075+0000","last_active":"2024-07-31T14:47:20.447744+0000","last_peered":"2024-07-31T14:47:20.447744+0000","last_clean":"2024-07-31T14:47:20.447744+0000","last_became_active":"2024-07-31T14:47:13.362599+0000","last_became_peered":"2024-07-31T14:47:13.362599+0000","last_unstale":"2024-07-31T14:47:20.447744+0000","last_undegraded":"2024-07-31T14:47:20.447744+0000","last_fullsized":"2024-07-31T14:47:20.447744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:20:43.617742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.402186+0000","last_change":"2024-07-31T14:47:20.450712+0000","last_active":"2024-07-31T14:49:49.402186+0000","last_peered":"2024-07-31T14:49:49.402186+0000","last_clean":"2024-07-31T14:49:49.402186+0000","last_became_active":"2024-07-31T14:47:20.450229+0000","last_became_peered":"2024-07-31T14:47:20.450229+0000","last_unstale":"2024-07-31T14:49:49.402186+0000","last_undegraded":"2024-07-31T14:49:49.402186+0000","last_fullsized":"2024-07-31T14:49:49.402186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:11:50.631420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:01.604764+0000","last_change":"2024-07-31T14:47:00.021389+0000","last_active":"2024-07-31T14:49:01.604764+0000","last_peered":"2024-07-31T14:49:01.604764+0000","last_clean":"2024-07-31T14:49:01.604764+0000","last_became_active":"2024-07-31T14:47:00.020987+0000","last_became_peered":"2024-07-31T14:47:00.020987+0000","last_unstale":"2024-07-31T14:49:01.604764+0000","last_undegraded":"2024-07-31T14:49:01.604764+0000","last_fullsized":"2024-07-31T14:49:01.604764+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:07:24.658463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627003+0000","last_change":"2024-07-31T14:47:13.363105+0000","last_active":"2024-07-31T14:47:20.627003+0000","last_peered":"2024-07-31T14:47:20.627003+0000","last_clean":"2024-07-31T14:47:20.627003+0000","last_became_active":"2024-07-31T14:47:13.362404+0000","last_became_peered":"2024-07-31T14:47:13.362404+0000","last_unstale":"2024-07-31T14:47:20.627003+0000","last_undegraded":"2024-07-31T14:47:20.627003+0000","last_fullsized":"2024-07-31T14:47:20.627003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:48:43.033584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404873+0000","last_change":"2024-07-31T14:47:03.057199+0000","last_active":"2024-07-31T14:49:49.404873+0000","last_peered":"2024-07-31T14:49:49.404873+0000","last_clean":"2024-07-31T14:49:49.404873+0000","last_became_active":"2024-07-31T14:47:03.056555+0000","last_became_peered":"2024-07-31T14:47:03.056555+0000","last_unstale":"2024-07-31T14:49:49.404873+0000","last_undegraded":"2024-07-31T14:49:49.404873+0000","last_fullsized":"2024-07-31T14:49:49.404873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:21:14.263457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:04.380224+0000","last_change":"2024-07-31T14:47:00.032055+0000","last_active":"2024-07-31T14:49:04.380224+0000","last_peered":"2024-07-31T14:49:04.380224+0000","last_clean":"2024-07-31T14:49:04.380224+0000","last_became_active":"2024-07-31T14:47:00.031519+0000","last_became_peered":"2024-07-31T14:47:00.031519+0000","last_unstale":"2024-07-31T14:49:04.380224+0000","last_undegraded":"2024-07-31T14:49:04.380224+0000","last_fullsized":"2024-07-31T14:49:04.380224+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:21:46.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:31.600736+0000","last_change":"2024-07-31T14:45:42.166503+0000","last_active":"2024-07-31T14:48:31.600736+0000","last_peered":"2024-07-31T14:48:31.600736+0000","last_clean":"2024-07-31T14:48:31.600736+0000","last_became_active":"2024-07-31T14:45:42.160065+0000","last_became_peered":"2024-07-31T14:45:42.160065+0000","last_unstale":"2024-07-31T14:48:31.600736+0000","last_undegraded":"2024-07-31T14:48:31.600736+0000","last_fullsized":"2024-07-31T14:48:31.600736+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_clean_scrub_stamp":"2024-07-31T14:42:27.863113+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:38:33.646022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:21.607394+0000","last_change":"2024-07-31T14:47:13.103812+0000","last_active":"2024-07-31T14:49:21.607394+0000","last_peered":"2024-07-31T14:49:21.607394+0000","last_clean":"2024-07-31T14:49:21.607394+0000","last_became_active":"2024-07-31T14:47:13.102549+0000","last_became_peered":"2024-07-31T14:47:13.102549+0000","last_unstale":"2024-07-31T14:49:21.607394+0000","last_undegraded":"2024-07-31T14:49:21.607394+0000","last_fullsized":"2024-07-31T14:49:21.607394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:18:56.322426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414083+0000","last_change":"2024-07-31T14:47:03.056310+0000","last_active":"2024-07-31T14:49:49.414083+0000","last_peered":"2024-07-31T14:49:49.414083+0000","last_clean":"2024-07-31T14:49:49.414083+0000","last_became_active":"2024-07-31T14:47:03.055695+0000","last_became_peered":"2024-07-31T14:47:03.055695+0000","last_unstale":"2024-07-31T14:49:49.414083+0000","last_undegraded":"2024-07-31T14:49:49.414083+0000","last_fullsized":"2024-07-31T14:49:49.414083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:00:55.124981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":255,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:24.913638+0000","last_change":"2024-07-31T14:47:00.236460+0000","last_active":"2024-07-31T14:51:24.913638+0000","last_peered":"2024-07-31T14:51:24.913638+0000","last_clean":"2024-07-31T14:51:24.913638+0000","last_became_active":"2024-07-31T14:47:00.235924+0000","last_became_peered":"2024-07-31T14:47:00.235924+0000","last_unstale":"2024-07-31T14:51:24.913638+0000","last_undegraded":"2024-07-31T14:51:24.913638+0000","last_fullsized":"2024-07-31T14:51:24.913638+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:05.130958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":207,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626080+0000","last_change":"2024-07-31T14:47:13.102341+0000","last_active":"2024-07-31T14:47:20.626080+0000","last_peered":"2024-07-31T14:47:20.626080+0000","last_clean":"2024-07-31T14:47:20.626080+0000","last_became_active":"2024-07-31T14:47:13.101798+0000","last_became_peered":"2024-07-31T14:47:13.101798+0000","last_unstale":"2024-07-31T14:47:20.626080+0000","last_undegraded":"2024-07-31T14:47:20.626080+0000","last_fullsized":"2024-07-31T14:47:20.626080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:34:35.758041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409700+0000","last_change":"2024-07-31T14:47:03.054322+0000","last_active":"2024-07-31T14:49:49.409700+0000","last_peered":"2024-07-31T14:49:49.409700+0000","last_clean":"2024-07-31T14:49:49.409700+0000","last_became_active":"2024-07-31T14:47:03.053112+0000","last_became_peered":"2024-07-31T14:47:03.053112+0000","last_unstale":"2024-07-31T14:49:49.409700+0000","last_undegraded":"2024-07-31T14:49:49.409700+0000","last_fullsized":"2024-07-31T14:49:49.409700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:55:13.542578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'62","reported_seq":134,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:22.197100+0000","last_change":"2024-07-31T14:47:00.026518+0000","last_active":"2024-07-31T14:51:22.197100+0000","last_peered":"2024-07-31T14:51:22.197100+0000","last_clean":"2024-07-31T14:51:22.197100+0000","last_became_active":"2024-07-31T14:47:00.026074+0000","last_became_peered":"2024-07-31T14:47:00.026074+0000","last_unstale":"2024-07-31T14:51:22.197100+0000","last_undegraded":"2024-07-31T14:51:22.197100+0000","last_fullsized":"2024-07-31T14:51:22.197100+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:33:36.688353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449369+0000","last_change":"2024-07-31T14:47:13.106365+0000","last_active":"2024-07-31T14:47:20.449369+0000","last_peered":"2024-07-31T14:47:20.449369+0000","last_clean":"2024-07-31T14:47:20.449369+0000","last_became_active":"2024-07-31T14:47:13.105635+0000","last_became_peered":"2024-07-31T14:47:13.105635+0000","last_unstale":"2024-07-31T14:47:20.449369+0000","last_undegraded":"2024-07-31T14:47:20.449369+0000","last_fullsized":"2024-07-31T14:47:20.449369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:52:09.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404285+0000","last_change":"2024-07-31T14:47:03.052522+0000","last_active":"2024-07-31T14:49:49.404285+0000","last_peered":"2024-07-31T14:49:49.404285+0000","last_clean":"2024-07-31T14:49:49.404285+0000","last_became_active":"2024-07-31T14:47:03.051872+0000","last_became_peered":"2024-07-31T14:47:03.051872+0000","last_unstale":"2024-07-31T14:49:49.404285+0000","last_undegraded":"2024-07-31T14:49:49.404285+0000","last_fullsized":"2024-07-31T14:49:49.404285+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:29:34.956745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600950+0000","last_change":"2024-07-31T14:47:00.016839+0000","last_active":"2024-07-31T14:48:02.600950+0000","last_peered":"2024-07-31T14:48:02.600950+0000","last_clean":"2024-07-31T14:48:02.600950+0000","last_became_active":"2024-07-31T14:47:00.016438+0000","last_became_peered":"2024-07-31T14:47:00.016438+0000","last_unstale":"2024-07-31T14:48:02.600950+0000","last_undegraded":"2024-07-31T14:48:02.600950+0000","last_fullsized":"2024-07-31T14:48:02.600950+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:42:10.875332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626258+0000","last_change":"2024-07-31T14:47:13.105385+0000","last_active":"2024-07-31T14:47:20.626258+0000","last_peered":"2024-07-31T14:47:20.626258+0000","last_clean":"2024-07-31T14:47:20.626258+0000","last_became_active":"2024-07-31T14:47:13.104867+0000","last_became_peered":"2024-07-31T14:47:13.104867+0000","last_unstale":"2024-07-31T14:47:20.626258+0000","last_undegraded":"2024-07-31T14:47:20.626258+0000","last_fullsized":"2024-07-31T14:47:20.626258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:25:21.572830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399072+0000","last_change":"2024-07-31T14:47:03.058304+0000","last_active":"2024-07-31T14:49:49.399072+0000","last_peered":"2024-07-31T14:49:49.399072+0000","last_clean":"2024-07-31T14:49:49.399072+0000","last_became_active":"2024-07-31T14:47:03.057251+0000","last_became_peered":"2024-07-31T14:47:03.057251+0000","last_unstale":"2024-07-31T14:49:49.399072+0000","last_undegraded":"2024-07-31T14:49:49.399072+0000","last_fullsized":"2024-07-31T14:49:49.399072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:58:20.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605301+0000","last_change":"2024-07-31T14:47:00.027806+0000","last_active":"2024-07-31T14:48:03.605301+0000","last_peered":"2024-07-31T14:48:03.605301+0000","last_clean":"2024-07-31T14:48:03.605301+0000","last_became_active":"2024-07-31T14:47:00.027403+0000","last_became_peered":"2024-07-31T14:47:00.027403+0000","last_unstale":"2024-07-31T14:48:03.605301+0000","last_undegraded":"2024-07-31T14:48:03.605301+0000","last_fullsized":"2024-07-31T14:48:03.605301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:57:18.303732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626490+0000","last_change":"2024-07-31T14:47:13.103628+0000","last_active":"2024-07-31T14:47:20.626490+0000","last_peered":"2024-07-31T14:47:20.626490+0000","last_clean":"2024-07-31T14:47:20.626490+0000","last_became_active":"2024-07-31T14:47:13.102296+0000","last_became_peered":"2024-07-31T14:47:13.102296+0000","last_unstale":"2024-07-31T14:47:20.626490+0000","last_undegraded":"2024-07-31T14:47:20.626490+0000","last_fullsized":"2024-07-31T14:47:20.626490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:25:35.109196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413387+0000","last_change":"2024-07-31T14:47:03.052761+0000","last_active":"2024-07-31T14:49:49.413387+0000","last_peered":"2024-07-31T14:49:49.413387+0000","last_clean":"2024-07-31T14:49:49.413387+0000","last_became_active":"2024-07-31T14:47:03.052410+0000","last_became_peered":"2024-07-31T14:47:03.052410+0000","last_unstale":"2024-07-31T14:49:49.413387+0000","last_undegraded":"2024-07-31T14:49:49.413387+0000","last_fullsized":"2024-07-31T14:49:49.413387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:20:47.712491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604937+0000","last_change":"2024-07-31T14:47:00.183741+0000","last_active":"2024-07-31T14:48:03.604937+0000","last_peered":"2024-07-31T14:48:03.604937+0000","last_clean":"2024-07-31T14:48:03.604937+0000","last_became_active":"2024-07-31T14:47:00.183275+0000","last_became_peered":"2024-07-31T14:47:00.183275+0000","last_unstale":"2024-07-31T14:48:03.604937+0000","last_undegraded":"2024-07-31T14:48:03.604937+0000","last_fullsized":"2024-07-31T14:48:03.604937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:29:23.205835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630098+0000","last_change":"2024-07-31T14:47:20.629334+0000","last_active":"2024-07-31T14:47:20.630098+0000","last_peered":"2024-07-31T14:47:20.630098+0000","last_clean":"2024-07-31T14:47:20.630098+0000","last_became_active":"2024-07-31T14:47:20.628865+0000","last_became_peered":"2024-07-31T14:47:20.628865+0000","last_unstale":"2024-07-31T14:47:20.630098+0000","last_undegraded":"2024-07-31T14:47:20.630098+0000","last_fullsized":"2024-07-31T14:47:20.630098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:21:27.199842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407413+0000","last_change":"2024-07-31T14:47:03.054633+0000","last_active":"2024-07-31T14:49:49.407413+0000","last_peered":"2024-07-31T14:49:49.407413+0000","last_clean":"2024-07-31T14:49:49.407413+0000","last_became_active":"2024-07-31T14:47:03.053025+0000","last_became_peered":"2024-07-31T14:47:03.053025+0000","last_unstale":"2024-07-31T14:49:49.407413+0000","last_undegraded":"2024-07-31T14:49:49.407413+0000","last_fullsized":"2024-07-31T14:49:49.407413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:42:03.392744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":261,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:27.904102+0000","last_change":"2024-07-31T14:47:00.027898+0000","last_active":"2024-07-31T14:51:27.904102+0000","last_peered":"2024-07-31T14:51:27.904102+0000","last_clean":"2024-07-31T14:51:27.904102+0000","last_became_active":"2024-07-31T14:47:00.027468+0000","last_became_peered":"2024-07-31T14:47:00.027468+0000","last_unstale":"2024-07-31T14:51:27.904102+0000","last_undegraded":"2024-07-31T14:51:27.904102+0000","last_fullsized":"2024-07-31T14:51:27.904102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:12:45.922172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":212,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444574+0000","last_change":"2024-07-31T14:47:13.104327+0000","last_active":"2024-07-31T14:47:20.444574+0000","last_peered":"2024-07-31T14:47:20.444574+0000","last_clean":"2024-07-31T14:47:20.444574+0000","last_became_active":"2024-07-31T14:47:13.103333+0000","last_became_peered":"2024-07-31T14:47:13.103333+0000","last_unstale":"2024-07-31T14:47:20.444574+0000","last_undegraded":"2024-07-31T14:47:20.444574+0000","last_fullsized":"2024-07-31T14:47:20.444574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:06.226220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.410145+0000","last_change":"2024-07-31T14:47:03.044875+0000","last_active":"2024-07-31T14:49:49.410145+0000","last_peered":"2024-07-31T14:49:49.410145+0000","last_clean":"2024-07-31T14:49:49.410145+0000","last_became_active":"2024-07-31T14:47:03.044421+0000","last_became_peered":"2024-07-31T14:47:03.044421+0000","last_unstale":"2024-07-31T14:49:49.410145+0000","last_undegraded":"2024-07-31T14:49:49.410145+0000","last_fullsized":"2024-07-31T14:49:49.410145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:00:02.477300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605606+0000","last_change":"2024-07-31T14:47:00.019790+0000","last_active":"2024-07-31T14:48:03.605606+0000","last_peered":"2024-07-31T14:48:03.605606+0000","last_clean":"2024-07-31T14:48:03.605606+0000","last_became_active":"2024-07-31T14:47:00.019368+0000","last_became_peered":"2024-07-31T14:47:00.019368+0000","last_unstale":"2024-07-31T14:48:03.605606+0000","last_undegraded":"2024-07-31T14:48:03.605606+0000","last_fullsized":"2024-07-31T14:48:03.605606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:15.299109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445672+0000","last_change":"2024-07-31T14:47:13.102618+0000","last_active":"2024-07-31T14:47:20.445672+0000","last_peered":"2024-07-31T14:47:20.445672+0000","last_clean":"2024-07-31T14:47:20.445672+0000","last_became_active":"2024-07-31T14:47:13.102085+0000","last_became_peered":"2024-07-31T14:47:13.102085+0000","last_unstale":"2024-07-31T14:47:20.445672+0000","last_undegraded":"2024-07-31T14:47:20.445672+0000","last_fullsized":"2024-07-31T14:47:20.445672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:05:58.814215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.406277+0000","last_change":"2024-07-31T14:47:03.044688+0000","last_active":"2024-07-31T14:49:49.406277+0000","last_peered":"2024-07-31T14:49:49.406277+0000","last_clean":"2024-07-31T14:49:49.406277+0000","last_became_active":"2024-07-31T14:47:03.043751+0000","last_became_peered":"2024-07-31T14:47:03.043751+0000","last_unstale":"2024-07-31T14:49:49.406277+0000","last_undegraded":"2024-07-31T14:49:49.406277+0000","last_fullsized":"2024-07-31T14:49:49.406277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:47:31.450796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605830+0000","last_change":"2024-07-31T14:47:00.026259+0000","last_active":"2024-07-31T14:48:03.605830+0000","last_peered":"2024-07-31T14:48:03.605830+0000","last_clean":"2024-07-31T14:48:03.605830+0000","last_became_active":"2024-07-31T14:47:00.025756+0000","last_became_peered":"2024-07-31T14:47:00.025756+0000","last_unstale":"2024-07-31T14:48:03.605830+0000","last_undegraded":"2024-07-31T14:48:03.605830+0000","last_fullsized":"2024-07-31T14:48:03.605830+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:05:00.258841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626888+0000","last_change":"2024-07-31T14:47:13.104918+0000","last_active":"2024-07-31T14:47:20.626888+0000","last_peered":"2024-07-31T14:47:20.626888+0000","last_clean":"2024-07-31T14:47:20.626888+0000","last_became_active":"2024-07-31T14:47:13.104031+0000","last_became_peered":"2024-07-31T14:47:13.104031+0000","last_unstale":"2024-07-31T14:47:20.626888+0000","last_undegraded":"2024-07-31T14:47:20.626888+0000","last_fullsized":"2024-07-31T14:47:20.626888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:17:15.264474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.389841+0000","last_change":"2024-07-31T14:47:03.055159+0000","last_active":"2024-07-31T14:49:49.389841+0000","last_peered":"2024-07-31T14:49:49.389841+0000","last_clean":"2024-07-31T14:49:49.389841+0000","last_became_active":"2024-07-31T14:47:03.052389+0000","last_became_peered":"2024-07-31T14:47:03.052389+0000","last_unstale":"2024-07-31T14:49:49.389841+0000","last_undegraded":"2024-07-31T14:49:49.389841+0000","last_fullsized":"2024-07-31T14:49:49.389841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:16.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'42","reported_seq":502,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:24.899616+0000","last_change":"2024-07-31T14:47:00.030165+0000","last_active":"2024-07-31T14:51:24.899616+0000","last_peered":"2024-07-31T14:51:24.899616+0000","last_clean":"2024-07-31T14:51:24.899616+0000","last_became_active":"2024-07-31T14:47:00.029559+0000","last_became_peered":"2024-07-31T14:47:00.029559+0000","last_unstale":"2024-07-31T14:51:24.899616+0000","last_undegraded":"2024-07-31T14:51:24.899616+0000","last_fullsized":"2024-07-31T14:51:24.899616+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:31:06.115075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71918,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":418,"num_read_kb":280,"num_write":44,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444234+0000","last_change":"2024-07-31T14:47:13.098517+0000","last_active":"2024-07-31T14:47:20.444234+0000","last_peered":"2024-07-31T14:47:20.444234+0000","last_clean":"2024-07-31T14:47:20.444234+0000","last_became_active":"2024-07-31T14:47:13.098066+0000","last_became_peered":"2024-07-31T14:47:13.098066+0000","last_unstale":"2024-07-31T14:47:20.444234+0000","last_undegraded":"2024-07-31T14:47:20.444234+0000","last_fullsized":"2024-07-31T14:47:20.444234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:08:54.958646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414451+0000","last_change":"2024-07-31T14:47:03.054045+0000","last_active":"2024-07-31T14:49:49.414451+0000","last_peered":"2024-07-31T14:49:49.414451+0000","last_clean":"2024-07-31T14:49:49.414451+0000","last_became_active":"2024-07-31T14:47:03.053658+0000","last_became_peered":"2024-07-31T14:47:03.053658+0000","last_unstale":"2024-07-31T14:49:49.414451+0000","last_undegraded":"2024-07-31T14:49:49.414451+0000","last_fullsized":"2024-07-31T14:49:49.414451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:36:54.563542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.607373+0000","last_change":"2024-07-31T14:47:00.185602+0000","last_active":"2024-07-31T14:48:03.607373+0000","last_peered":"2024-07-31T14:48:03.607373+0000","last_clean":"2024-07-31T14:48:03.607373+0000","last_became_active":"2024-07-31T14:47:00.185230+0000","last_became_peered":"2024-07-31T14:47:00.185230+0000","last_unstale":"2024-07-31T14:48:03.607373+0000","last_undegraded":"2024-07-31T14:48:03.607373+0000","last_fullsized":"2024-07-31T14:48:03.607373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:21:35.979942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604698+0000","last_change":"2024-07-31T14:47:13.099611+0000","last_active":"2024-07-31T14:47:20.604698+0000","last_peered":"2024-07-31T14:47:20.604698+0000","last_clean":"2024-07-31T14:47:20.604698+0000","last_became_active":"2024-07-31T14:47:13.099127+0000","last_became_peered":"2024-07-31T14:47:13.099127+0000","last_unstale":"2024-07-31T14:47:20.604698+0000","last_undegraded":"2024-07-31T14:47:20.604698+0000","last_fullsized":"2024-07-31T14:47:20.604698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:16:57.192980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400126+0000","last_change":"2024-07-31T14:47:03.056818+0000","last_active":"2024-07-31T14:49:49.400126+0000","last_peered":"2024-07-31T14:49:49.400126+0000","last_clean":"2024-07-31T14:49:49.400126+0000","last_became_active":"2024-07-31T14:47:03.055716+0000","last_became_peered":"2024-07-31T14:47:03.055716+0000","last_unstale":"2024-07-31T14:49:49.400126+0000","last_undegraded":"2024-07-31T14:49:49.400126+0000","last_fullsized":"2024-07-31T14:49:49.400126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:26:33.627362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600344+0000","last_change":"2024-07-31T14:47:00.029651+0000","last_active":"2024-07-31T14:48:02.600344+0000","last_peered":"2024-07-31T14:48:02.600344+0000","last_clean":"2024-07-31T14:48:02.600344+0000","last_became_active":"2024-07-31T14:47:00.029242+0000","last_became_peered":"2024-07-31T14:47:00.029242+0000","last_unstale":"2024-07-31T14:48:02.600344+0000","last_undegraded":"2024-07-31T14:48:02.600344+0000","last_fullsized":"2024-07-31T14:48:02.600344+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:23:49.177508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.443852+0000","last_change":"2024-07-31T14:47:13.098862+0000","last_active":"2024-07-31T14:47:20.443852+0000","last_peered":"2024-07-31T14:47:20.443852+0000","last_clean":"2024-07-31T14:47:20.443852+0000","last_became_active":"2024-07-31T14:47:13.098324+0000","last_became_peered":"2024-07-31T14:47:13.098324+0000","last_unstale":"2024-07-31T14:47:20.443852+0000","last_undegraded":"2024-07-31T14:47:20.443852+0000","last_fullsized":"2024-07-31T14:47:20.443852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:50:41.260863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.412242+0000","last_change":"2024-07-31T14:47:03.049042+0000","last_active":"2024-07-31T14:49:49.412242+0000","last_peered":"2024-07-31T14:49:49.412242+0000","last_clean":"2024-07-31T14:49:49.412242+0000","last_became_active":"2024-07-31T14:47:03.048535+0000","last_became_peered":"2024-07-31T14:47:03.048535+0000","last_unstale":"2024-07-31T14:49:49.412242+0000","last_undegraded":"2024-07-31T14:49:49.412242+0000","last_fullsized":"2024-07-31T14:49:49.412242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:15.147454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":452,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:24.899853+0000","last_change":"2024-07-31T14:47:00.028940+0000","last_active":"2024-07-31T14:51:24.899853+0000","last_peered":"2024-07-31T14:51:24.899853+0000","last_clean":"2024-07-31T14:51:24.899853+0000","last_became_active":"2024-07-31T14:47:00.028472+0000","last_became_peered":"2024-07-31T14:47:00.028472+0000","last_unstale":"2024-07-31T14:51:24.899853+0000","last_undegraded":"2024-07-31T14:51:24.899853+0000","last_fullsized":"2024-07-31T14:51:24.899853+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:57:15.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":404,"num_read_kb":404,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626394+0000","last_change":"2024-07-31T14:47:13.101856+0000","last_active":"2024-07-31T14:47:20.626394+0000","last_peered":"2024-07-31T14:47:20.626394+0000","last_clean":"2024-07-31T14:47:20.626394+0000","last_became_active":"2024-07-31T14:47:13.101346+0000","last_became_peered":"2024-07-31T14:47:13.101346+0000","last_unstale":"2024-07-31T14:47:20.626394+0000","last_undegraded":"2024-07-31T14:47:20.626394+0000","last_fullsized":"2024-07-31T14:47:20.626394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:56:37.071660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:16.965978+0000","last_change":"2024-07-31T14:47:13.105952+0000","last_active":"2024-07-31T14:49:16.965978+0000","last_peered":"2024-07-31T14:49:16.965978+0000","last_clean":"2024-07-31T14:49:16.965978+0000","last_became_active":"2024-07-31T14:47:13.105506+0000","last_became_peered":"2024-07-31T14:47:13.105506+0000","last_unstale":"2024-07-31T14:49:16.965978+0000","last_undegraded":"2024-07-31T14:49:16.965978+0000","last_fullsized":"2024-07-31T14:49:16.965978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:28:33.866216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400532+0000","last_change":"2024-07-31T14:47:03.051152+0000","last_active":"2024-07-31T14:49:49.400532+0000","last_peered":"2024-07-31T14:49:49.400532+0000","last_clean":"2024-07-31T14:49:49.400532+0000","last_became_active":"2024-07-31T14:47:03.050775+0000","last_became_peered":"2024-07-31T14:47:03.050775+0000","last_unstale":"2024-07-31T14:49:49.400532+0000","last_undegraded":"2024-07-31T14:49:49.400532+0000","last_fullsized":"2024-07-31T14:49:49.400532+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:47:35.913641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609201+0000","last_change":"2024-07-31T14:47:00.028552+0000","last_active":"2024-07-31T14:48:03.609201+0000","last_peered":"2024-07-31T14:48:03.609201+0000","last_clean":"2024-07-31T14:48:03.609201+0000","last_became_active":"2024-07-31T14:47:00.028143+0000","last_became_peered":"2024-07-31T14:47:00.028143+0000","last_unstale":"2024-07-31T14:48:03.609201+0000","last_undegraded":"2024-07-31T14:48:03.609201+0000","last_fullsized":"2024-07-31T14:48:03.609201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:09.601635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:19.188903+0000","last_change":"2024-07-31T14:47:20.606285+0000","last_active":"2024-07-31T14:49:19.188903+0000","last_peered":"2024-07-31T14:49:19.188903+0000","last_clean":"2024-07-31T14:49:19.188903+0000","last_became_active":"2024-07-31T14:47:20.605837+0000","last_became_peered":"2024-07-31T14:47:20.605837+0000","last_unstale":"2024-07-31T14:49:19.188903+0000","last_undegraded":"2024-07-31T14:49:19.188903+0000","last_fullsized":"2024-07-31T14:49:19.188903+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:16:35.409790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.398709+0000","last_change":"2024-07-31T14:47:03.047442+0000","last_active":"2024-07-31T14:49:49.398709+0000","last_peered":"2024-07-31T14:49:49.398709+0000","last_clean":"2024-07-31T14:49:49.398709+0000","last_became_active":"2024-07-31T14:47:03.047072+0000","last_became_peered":"2024-07-31T14:47:03.047072+0000","last_unstale":"2024-07-31T14:49:49.398709+0000","last_undegraded":"2024-07-31T14:49:49.398709+0000","last_fullsized":"2024-07-31T14:49:49.398709+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:56.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.599595+0000","last_change":"2024-07-31T14:47:00.027746+0000","last_active":"2024-07-31T14:48:02.599595+0000","last_peered":"2024-07-31T14:48:02.599595+0000","last_clean":"2024-07-31T14:48:02.599595+0000","last_became_active":"2024-07-31T14:47:00.027404+0000","last_became_peered":"2024-07-31T14:47:00.027404+0000","last_unstale":"2024-07-31T14:48:02.599595+0000","last_undegraded":"2024-07-31T14:48:02.599595+0000","last_fullsized":"2024-07-31T14:48:02.599595+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:30:21.011674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.589640+0000","last_change":"2024-07-31T14:47:00.029148+0000","last_active":"2024-07-31T14:48:01.589640+0000","last_peered":"2024-07-31T14:48:01.589640+0000","last_clean":"2024-07-31T14:48:01.589640+0000","last_became_active":"2024-07-31T14:47:00.028718+0000","last_became_peered":"2024-07-31T14:47:00.028718+0000","last_unstale":"2024-07-31T14:48:01.589640+0000","last_undegraded":"2024-07-31T14:48:01.589640+0000","last_fullsized":"2024-07-31T14:48:01.589640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:25:16.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400207+0000","last_change":"2024-07-31T14:47:20.629098+0000","last_active":"2024-07-31T14:49:49.400207+0000","last_peered":"2024-07-31T14:49:49.400207+0000","last_clean":"2024-07-31T14:49:49.400207+0000","last_became_active":"2024-07-31T14:47:20.627775+0000","last_became_peered":"2024-07-31T14:47:20.627775+0000","last_unstale":"2024-07-31T14:49:49.400207+0000","last_undegraded":"2024-07-31T14:49:49.400207+0000","last_fullsized":"2024-07-31T14:49:49.400207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:22:03.606509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447331+0000","last_change":"2024-07-31T14:47:13.097892+0000","last_active":"2024-07-31T14:47:20.447331+0000","last_peered":"2024-07-31T14:47:20.447331+0000","last_clean":"2024-07-31T14:47:20.447331+0000","last_became_active":"2024-07-31T14:47:13.097419+0000","last_became_peered":"2024-07-31T14:47:13.097419+0000","last_unstale":"2024-07-31T14:47:20.447331+0000","last_undegraded":"2024-07-31T14:47:20.447331+0000","last_fullsized":"2024-07-31T14:47:20.447331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:07:13.456147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":258,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:26.903505+0000","last_change":"2024-07-31T14:47:00.019792+0000","last_active":"2024-07-31T14:51:26.903505+0000","last_peered":"2024-07-31T14:51:26.903505+0000","last_clean":"2024-07-31T14:51:26.903505+0000","last_became_active":"2024-07-31T14:47:00.019411+0000","last_became_peered":"2024-07-31T14:47:00.019411+0000","last_unstale":"2024-07-31T14:51:26.903505+0000","last_undegraded":"2024-07-31T14:51:26.903505+0000","last_fullsized":"2024-07-31T14:51:26.903505+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:55:02.148823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":209,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.390707+0000","last_change":"2024-07-31T14:47:03.039808+0000","last_active":"2024-07-31T14:49:49.390707+0000","last_peered":"2024-07-31T14:49:49.390707+0000","last_clean":"2024-07-31T14:49:49.390707+0000","last_became_active":"2024-07-31T14:47:03.039382+0000","last_became_peered":"2024-07-31T14:47:03.039382+0000","last_unstale":"2024-07-31T14:49:49.390707+0000","last_undegraded":"2024-07-31T14:49:49.390707+0000","last_fullsized":"2024-07-31T14:49:49.390707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:23:09.751293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.446939+0000","last_change":"2024-07-31T14:47:13.102797+0000","last_active":"2024-07-31T14:47:20.446939+0000","last_peered":"2024-07-31T14:47:20.446939+0000","last_clean":"2024-07-31T14:47:20.446939+0000","last_became_active":"2024-07-31T14:47:13.102320+0000","last_became_peered":"2024-07-31T14:47:13.102320+0000","last_unstale":"2024-07-31T14:47:20.446939+0000","last_undegraded":"2024-07-31T14:47:20.446939+0000","last_fullsized":"2024-07-31T14:47:20.446939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:23.066357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.397081+0000","last_change":"2024-07-31T14:47:03.054875+0000","last_active":"2024-07-31T14:49:49.397081+0000","last_peered":"2024-07-31T14:49:49.397081+0000","last_clean":"2024-07-31T14:49:49.397081+0000","last_became_active":"2024-07-31T14:47:03.054529+0000","last_became_peered":"2024-07-31T14:47:03.054529+0000","last_unstale":"2024-07-31T14:49:49.397081+0000","last_undegraded":"2024-07-31T14:49:49.397081+0000","last_fullsized":"2024-07-31T14:49:49.397081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:32:09.802933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396421+0000","last_change":"2024-07-31T14:47:03.043283+0000","last_active":"2024-07-31T14:49:49.396421+0000","last_peered":"2024-07-31T14:49:49.396421+0000","last_clean":"2024-07-31T14:49:49.396421+0000","last_became_active":"2024-07-31T14:47:03.042851+0000","last_became_peered":"2024-07-31T14:47:03.042851+0000","last_unstale":"2024-07-31T14:49:49.396421+0000","last_undegraded":"2024-07-31T14:49:49.396421+0000","last_fullsized":"2024-07-31T14:49:49.396421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:25:37.408149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409526+0000","last_change":"2024-07-31T14:47:03.047890+0000","last_active":"2024-07-31T14:49:49.409526+0000","last_peered":"2024-07-31T14:49:49.409526+0000","last_clean":"2024-07-31T14:49:49.409526+0000","last_became_active":"2024-07-31T14:47:03.047507+0000","last_became_peered":"2024-07-31T14:47:03.047507+0000","last_unstale":"2024-07-31T14:49:49.409526+0000","last_undegraded":"2024-07-31T14:49:49.409526+0000","last_fullsized":"2024-07-31T14:49:49.409526+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:52:45.991323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403192+0000","last_change":"2024-07-31T14:47:03.047903+0000","last_active":"2024-07-31T14:49:49.403192+0000","last_peered":"2024-07-31T14:49:49.403192+0000","last_clean":"2024-07-31T14:49:49.403192+0000","last_became_active":"2024-07-31T14:47:03.047352+0000","last_became_peered":"2024-07-31T14:47:03.047352+0000","last_unstale":"2024-07-31T14:49:49.403192+0000","last_undegraded":"2024-07-31T14:49:49.403192+0000","last_fullsized":"2024-07-31T14:49:49.403192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:06.536023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403277+0000","last_change":"2024-07-31T14:47:03.044199+0000","last_active":"2024-07-31T14:49:49.403277+0000","last_peered":"2024-07-31T14:49:49.403277+0000","last_clean":"2024-07-31T14:49:49.403277+0000","last_became_active":"2024-07-31T14:47:03.043849+0000","last_became_peered":"2024-07-31T14:47:03.043849+0000","last_unstale":"2024-07-31T14:49:49.403277+0000","last_undegraded":"2024-07-31T14:49:49.403277+0000","last_fullsized":"2024-07-31T14:49:49.403277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:50.474895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.391219+0000","last_change":"2024-07-31T14:47:03.060654+0000","last_active":"2024-07-31T14:49:49.391219+0000","last_peered":"2024-07-31T14:49:49.391219+0000","last_clean":"2024-07-31T14:49:49.391219+0000","last_became_active":"2024-07-31T14:47:03.060157+0000","last_became_peered":"2024-07-31T14:47:03.060157+0000","last_unstale":"2024-07-31T14:49:49.391219+0000","last_undegraded":"2024-07-31T14:49:49.391219+0000","last_fullsized":"2024-07-31T14:49:49.391219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:39:50.130831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414248+0000","last_change":"2024-07-31T14:47:03.049910+0000","last_active":"2024-07-31T14:49:49.414248+0000","last_peered":"2024-07-31T14:49:49.414248+0000","last_clean":"2024-07-31T14:49:49.414248+0000","last_became_active":"2024-07-31T14:47:03.049491+0000","last_became_peered":"2024-07-31T14:47:03.049491+0000","last_unstale":"2024-07-31T14:49:49.414248+0000","last_undegraded":"2024-07-31T14:49:49.414248+0000","last_fullsized":"2024-07-31T14:49:49.414248+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:18:43.319695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.410414+0000","last_change":"2024-07-31T14:47:03.052206+0000","last_active":"2024-07-31T14:49:49.410414+0000","last_peered":"2024-07-31T14:49:49.410414+0000","last_clean":"2024-07-31T14:49:49.410414+0000","last_became_active":"2024-07-31T14:47:03.051175+0000","last_became_peered":"2024-07-31T14:47:03.051175+0000","last_unstale":"2024-07-31T14:49:49.410414+0000","last_undegraded":"2024-07-31T14:49:49.410414+0000","last_fullsized":"2024-07-31T14:49:49.410414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:51:11.583604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.405714+0000","last_change":"2024-07-31T14:47:03.052162+0000","last_active":"2024-07-31T14:49:49.405714+0000","last_peered":"2024-07-31T14:49:49.405714+0000","last_clean":"2024-07-31T14:49:49.405714+0000","last_became_active":"2024-07-31T14:47:03.051634+0000","last_became_peered":"2024-07-31T14:47:03.051634+0000","last_unstale":"2024-07-31T14:49:49.405714+0000","last_undegraded":"2024-07-31T14:49:49.405714+0000","last_fullsized":"2024-07-31T14:49:49.405714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:41:59.605192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409010+0000","last_change":"2024-07-31T14:47:03.047475+0000","last_active":"2024-07-31T14:49:49.409010+0000","last_peered":"2024-07-31T14:49:49.409010+0000","last_clean":"2024-07-31T14:49:49.409010+0000","last_became_active":"2024-07-31T14:47:03.047100+0000","last_became_peered":"2024-07-31T14:47:03.047100+0000","last_unstale":"2024-07-31T14:49:49.409010+0000","last_undegraded":"2024-07-31T14:49:49.409010+0000","last_fullsized":"2024-07-31T14:49:49.409010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:08:10.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411379+0000","last_change":"2024-07-31T14:47:03.051281+0000","last_active":"2024-07-31T14:49:49.411379+0000","last_peered":"2024-07-31T14:49:49.411379+0000","last_clean":"2024-07-31T14:49:49.411379+0000","last_became_active":"2024-07-31T14:47:03.050921+0000","last_became_peered":"2024-07-31T14:47:03.050921+0000","last_unstale":"2024-07-31T14:49:49.411379+0000","last_undegraded":"2024-07-31T14:49:49.411379+0000","last_fullsized":"2024-07-31T14:49:49.411379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:24:29.221723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403114+0000","last_change":"2024-07-31T14:47:03.045457+0000","last_active":"2024-07-31T14:49:49.403114+0000","last_peered":"2024-07-31T14:49:49.403114+0000","last_clean":"2024-07-31T14:49:49.403114+0000","last_became_active":"2024-07-31T14:47:03.044466+0000","last_became_peered":"2024-07-31T14:47:03.044466+0000","last_unstale":"2024-07-31T14:49:49.403114+0000","last_undegraded":"2024-07-31T14:49:49.403114+0000","last_fullsized":"2024-07-31T14:49:49.403114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:44:01.197416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403872+0000","last_change":"2024-07-31T14:47:03.053431+0000","last_active":"2024-07-31T14:49:49.403872+0000","last_peered":"2024-07-31T14:49:49.403872+0000","last_clean":"2024-07-31T14:49:49.403872+0000","last_became_active":"2024-07-31T14:47:03.052958+0000","last_became_peered":"2024-07-31T14:47:03.052958+0000","last_unstale":"2024-07-31T14:49:49.403872+0000","last_undegraded":"2024-07-31T14:49:49.403872+0000","last_fullsized":"2024-07-31T14:49:49.403872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:03:28.863589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409420+0000","last_change":"2024-07-31T14:47:03.046641+0000","last_active":"2024-07-31T14:49:49.409420+0000","last_peered":"2024-07-31T14:49:49.409420+0000","last_clean":"2024-07-31T14:49:49.409420+0000","last_became_active":"2024-07-31T14:47:03.046265+0000","last_became_peered":"2024-07-31T14:47:03.046265+0000","last_unstale":"2024-07-31T14:49:49.409420+0000","last_undegraded":"2024-07-31T14:49:49.409420+0000","last_fullsized":"2024-07-31T14:49:49.409420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:20:25.389010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.387120+0000","last_change":"2024-07-31T14:47:03.051657+0000","last_active":"2024-07-31T14:49:49.387120+0000","last_peered":"2024-07-31T14:49:49.387120+0000","last_clean":"2024-07-31T14:49:49.387120+0000","last_became_active":"2024-07-31T14:47:03.050722+0000","last_became_peered":"2024-07-31T14:47:03.050722+0000","last_unstale":"2024-07-31T14:49:49.387120+0000","last_undegraded":"2024-07-31T14:49:49.387120+0000","last_fullsized":"2024-07-31T14:49:49.387120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:23:19.387470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411567+0000","last_change":"2024-07-31T14:47:03.059654+0000","last_active":"2024-07-31T14:49:49.411567+0000","last_peered":"2024-07-31T14:49:49.411567+0000","last_clean":"2024-07-31T14:49:49.411567+0000","last_became_active":"2024-07-31T14:47:03.056843+0000","last_became_peered":"2024-07-31T14:47:03.056843+0000","last_unstale":"2024-07-31T14:49:49.411567+0000","last_undegraded":"2024-07-31T14:49:49.411567+0000","last_fullsized":"2024-07-31T14:49:49.411567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:42:33.696498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413200+0000","last_change":"2024-07-31T14:47:03.052750+0000","last_active":"2024-07-31T14:49:49.413200+0000","last_peered":"2024-07-31T14:49:49.413200+0000","last_clean":"2024-07-31T14:49:49.413200+0000","last_became_active":"2024-07-31T14:47:03.051744+0000","last_became_peered":"2024-07-31T14:47:03.051744+0000","last_unstale":"2024-07-31T14:49:49.413200+0000","last_undegraded":"2024-07-31T14:49:49.413200+0000","last_fullsized":"2024-07-31T14:49:49.413200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:24:33.202918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.393222+0000","last_change":"2024-07-31T14:47:20.452563+0000","last_active":"2024-07-31T14:49:49.393222+0000","last_peered":"2024-07-31T14:49:49.393222+0000","last_clean":"2024-07-31T14:49:49.393222+0000","last_became_active":"2024-07-31T14:47:20.452271+0000","last_became_peered":"2024-07-31T14:47:20.452271+0000","last_unstale":"2024-07-31T14:49:49.393222+0000","last_undegraded":"2024-07-31T14:49:49.393222+0000","last_fullsized":"2024-07-31T14:49:49.393222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:26:55.070265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.391684+0000","last_change":"2024-07-31T14:47:03.050691+0000","last_active":"2024-07-31T14:49:49.391684+0000","last_peered":"2024-07-31T14:49:49.391684+0000","last_clean":"2024-07-31T14:49:49.391684+0000","last_became_active":"2024-07-31T14:47:03.050320+0000","last_became_peered":"2024-07-31T14:47:03.050320+0000","last_unstale":"2024-07-31T14:49:49.391684+0000","last_undegraded":"2024-07-31T14:49:49.391684+0000","last_fullsized":"2024-07-31T14:49:49.391684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:40:45.767706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400128+0000","last_change":"2024-07-31T14:47:03.048491+0000","last_active":"2024-07-31T14:49:49.400128+0000","last_peered":"2024-07-31T14:49:49.400128+0000","last_clean":"2024-07-31T14:49:49.400128+0000","last_became_active":"2024-07-31T14:47:03.048085+0000","last_became_peered":"2024-07-31T14:47:03.048085+0000","last_unstale":"2024-07-31T14:49:49.400128+0000","last_undegraded":"2024-07-31T14:49:49.400128+0000","last_fullsized":"2024-07-31T14:49:49.400128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:35:01.762365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.395425+0000","last_change":"2024-07-31T14:47:03.055502+0000","last_active":"2024-07-31T14:49:49.395425+0000","last_peered":"2024-07-31T14:49:49.395425+0000","last_clean":"2024-07-31T14:49:49.395425+0000","last_became_active":"2024-07-31T14:47:03.055015+0000","last_became_peered":"2024-07-31T14:47:03.055015+0000","last_unstale":"2024-07-31T14:49:49.395425+0000","last_undegraded":"2024-07-31T14:49:49.395425+0000","last_fullsized":"2024-07-31T14:49:49.395425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:50:39.605148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413933+0000","last_change":"2024-07-31T14:47:03.057808+0000","last_active":"2024-07-31T14:49:49.413933+0000","last_peered":"2024-07-31T14:49:49.413933+0000","last_clean":"2024-07-31T14:49:49.413933+0000","last_became_active":"2024-07-31T14:47:03.057443+0000","last_became_peered":"2024-07-31T14:47:03.057443+0000","last_unstale":"2024-07-31T14:49:49.413933+0000","last_undegraded":"2024-07-31T14:49:49.413933+0000","last_fullsized":"2024-07-31T14:49:49.413933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:34:42.915428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396116+0000","last_change":"2024-07-31T14:47:03.058182+0000","last_active":"2024-07-31T14:49:49.396116+0000","last_peered":"2024-07-31T14:49:49.396116+0000","last_clean":"2024-07-31T14:49:49.396116+0000","last_became_active":"2024-07-31T14:47:03.057802+0000","last_became_peered":"2024-07-31T14:47:03.057802+0000","last_unstale":"2024-07-31T14:49:49.396116+0000","last_undegraded":"2024-07-31T14:49:49.396116+0000","last_fullsized":"2024-07-31T14:49:49.396116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:08:06.608788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413379+0000","last_change":"2024-07-31T14:47:03.058882+0000","last_active":"2024-07-31T14:49:49.413379+0000","last_peered":"2024-07-31T14:49:49.413379+0000","last_clean":"2024-07-31T14:49:49.413379+0000","last_became_active":"2024-07-31T14:47:03.058530+0000","last_became_peered":"2024-07-31T14:47:03.058530+0000","last_unstale":"2024-07-31T14:49:49.413379+0000","last_undegraded":"2024-07-31T14:49:49.413379+0000","last_fullsized":"2024-07-31T14:49:49.413379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:28:03.631962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404760+0000","last_change":"2024-07-31T14:47:03.052028+0000","last_active":"2024-07-31T14:49:49.404760+0000","last_peered":"2024-07-31T14:49:49.404760+0000","last_clean":"2024-07-31T14:49:49.404760+0000","last_became_active":"2024-07-31T14:47:03.051559+0000","last_became_peered":"2024-07-31T14:47:03.051559+0000","last_unstale":"2024-07-31T14:49:49.404760+0000","last_undegraded":"2024-07-31T14:49:49.404760+0000","last_fullsized":"2024-07-31T14:49:49.404760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:16:40.723795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399075+0000","last_change":"2024-07-31T14:47:03.053884+0000","last_active":"2024-07-31T14:49:49.399075+0000","last_peered":"2024-07-31T14:49:49.399075+0000","last_clean":"2024-07-31T14:49:49.399075+0000","last_became_active":"2024-07-31T14:47:03.053521+0000","last_became_peered":"2024-07-31T14:47:03.053521+0000","last_unstale":"2024-07-31T14:49:49.399075+0000","last_undegraded":"2024-07-31T14:49:49.399075+0000","last_fullsized":"2024-07-31T14:49:49.399075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:16:37.702055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.394011+0000","last_change":"2024-07-31T14:47:03.058588+0000","last_active":"2024-07-31T14:49:49.394011+0000","last_peered":"2024-07-31T14:49:49.394011+0000","last_clean":"2024-07-31T14:49:49.394011+0000","last_became_active":"2024-07-31T14:47:03.057820+0000","last_became_peered":"2024-07-31T14:47:03.057820+0000","last_unstale":"2024-07-31T14:49:49.394011+0000","last_undegraded":"2024-07-31T14:49:49.394011+0000","last_fullsized":"2024-07-31T14:49:49.394011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:21.265208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396703+0000","last_change":"2024-07-31T14:47:03.041447+0000","last_active":"2024-07-31T14:49:49.396703+0000","last_peered":"2024-07-31T14:49:49.396703+0000","last_clean":"2024-07-31T14:49:49.396703+0000","last_became_active":"2024-07-31T14:47:03.040935+0000","last_became_peered":"2024-07-31T14:47:03.040935+0000","last_unstale":"2024-07-31T14:49:49.396703+0000","last_undegraded":"2024-07-31T14:49:49.396703+0000","last_fullsized":"2024-07-31T14:49:49.396703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:00:13.624698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396885+0000","last_change":"2024-07-31T14:47:20.452239+0000","last_active":"2024-07-31T14:49:49.396885+0000","last_peered":"2024-07-31T14:49:49.396885+0000","last_clean":"2024-07-31T14:49:49.396885+0000","last_became_active":"2024-07-31T14:47:20.451924+0000","last_became_peered":"2024-07-31T14:47:20.451924+0000","last_unstale":"2024-07-31T14:49:49.396885+0000","last_undegraded":"2024-07-31T14:49:49.396885+0000","last_fullsized":"2024-07-31T14:49:49.396885+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:58:24.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404120+0000","last_change":"2024-07-31T14:47:03.056024+0000","last_active":"2024-07-31T14:49:49.404120+0000","last_peered":"2024-07-31T14:49:49.404120+0000","last_clean":"2024-07-31T14:49:49.404120+0000","last_became_active":"2024-07-31T14:47:03.055630+0000","last_became_peered":"2024-07-31T14:47:03.055630+0000","last_unstale":"2024-07-31T14:49:49.404120+0000","last_undegraded":"2024-07-31T14:49:49.404120+0000","last_fullsized":"2024-07-31T14:49:49.404120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:09:11.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396693+0000","last_change":"2024-07-31T14:47:03.053718+0000","last_active":"2024-07-31T14:49:49.396693+0000","last_peered":"2024-07-31T14:49:49.396693+0000","last_clean":"2024-07-31T14:49:49.396693+0000","last_became_active":"2024-07-31T14:47:03.053361+0000","last_became_peered":"2024-07-31T14:47:03.053361+0000","last_unstale":"2024-07-31T14:49:49.396693+0000","last_undegraded":"2024-07-31T14:49:49.396693+0000","last_fullsized":"2024-07-31T14:49:49.396693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:03:49.220838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411585+0000","last_change":"2024-07-31T14:47:03.056087+0000","last_active":"2024-07-31T14:49:49.411585+0000","last_peered":"2024-07-31T14:49:49.411585+0000","last_clean":"2024-07-31T14:49:49.411585+0000","last_became_active":"2024-07-31T14:47:03.055732+0000","last_became_peered":"2024-07-31T14:47:03.055732+0000","last_unstale":"2024-07-31T14:49:49.411585+0000","last_undegraded":"2024-07-31T14:49:49.411585+0000","last_fullsized":"2024-07-31T14:49:49.411585+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:45:01.185666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":868018,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2421,"num_read_kb":1431,"num_write":501,"num_write_kb":2260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2973696,"data_stored":2604054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":380,"ondisk_log_size":380,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645380,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":520429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841608,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37472,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707680,"statfs":{"total":95995035648,"available":95956664320,"internally_reserved":0,"allocated":2125824,"data_stored":1573163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005134,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36748,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708404,"statfs":{"total":95995035648,"available":95957405696,"internally_reserved":0,"allocated":1445888,"data_stored":916972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201363,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1437696,"data_stored":913254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397591,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1118131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561117,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":515696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757346,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29076,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716076,"statfs":{"total":95995035648,"available":95965261824,"internally_reserved":0,"allocated":2048000,"data_stored":1506034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920871,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":520161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117099,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28860,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716292,"statfs":{"total":95995035648,"available":95965483008,"internally_reserved":0,"allocated":1826816,"data_stored":1301864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378736,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":589563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542261,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":588644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705785,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":517319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":819200,"data_stored":787422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":397128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":524288,"data_stored":468353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-31T14:51:31.319 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-31T14:51:31.319 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph pg dump --format=json 2024-07-31T14:51:31.587 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:31.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:31 smithi067 ceph-mon[32750]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:31.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:31 smithi077 ceph-mon[37327]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:32.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:31 smithi073 ceph-mon[36714]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:32.941 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:32 smithi067 ceph-mon[32750]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-31T14:51:32.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:32 smithi077 ceph-mon[37327]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-31T14:51:33.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:32 smithi073 ceph-mon[36714]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-31T14:51:33.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.181+0000 7f3e66982640 1 -- 172.21.15.67:0/3247343237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e60102490 msgr2=0x7f3e601028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:33.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.181+0000 7f3e66982640 1 --2- 172.21.15.67:0/3247343237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e60102490 0x7f3e601028a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009900 tx=0x7f3e5002a410 comp rx=0 tx=0).stop 2024-07-31T14:51:33.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.181+0000 7f3e66982640 1 -- 172.21.15.67:0/3247343237 shutdown_connections 2024-07-31T14:51:33.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.181+0000 7f3e66982640 1 --2- 172.21.15.67:0/3247343237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e60108720 0x7f3e6010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:33.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.181+0000 7f3e66982640 1 --2- 172.21.15.67:0/3247343237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e60107dd0 0x7f3e601081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:33.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.181+0000 7f3e66982640 1 --2- 172.21.15.67:0/3247343237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e60102490 0x7f3e601028a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:33.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.181+0000 7f3e66982640 1 -- 172.21.15.67:0/3247343237 >> 172.21.15.67:0/3247343237 conn(0x7f3e60079160 msgr2=0x7f3e60079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:33.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.181+0000 7f3e66982640 1 -- 172.21.15.67:0/3247343237 shutdown_connections 2024-07-31T14:51:33.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.181+0000 7f3e66982640 1 -- 172.21.15.67:0/3247343237 wait complete. 2024-07-31T14:51:33.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.182+0000 7f3e66982640 1 Processor -- start 2024-07-31T14:51:33.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.182+0000 7f3e66982640 1 -- start start 2024-07-31T14:51:33.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e66982640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e60102490 0x7f3e60199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:33.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e66982640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e60107dd0 0x7f3e6019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:33.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e66982640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e60108720 0x7f3e6019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:33.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e66982640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3e601134b0 con 0x7f3e60108720 2024-07-31T14:51:33.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e66982640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3e60113330 con 0x7f3e60102490 2024-07-31T14:51:33.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e66982640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3e60113630 con 0x7f3e60107dd0 2024-07-31T14:51:33.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e64f1f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e60108720 0x7f3e6019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:33.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e5ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e60102490 0x7f3e60199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:33.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e5ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e60102490 0x7f3e60199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:42204/0 (socket says 172.21.15.67:42204) 2024-07-31T14:51:33.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e64f1f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e60108720 0x7f3e6019f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44116/0 (socket says 172.21.15.67:44116) 2024-07-31T14:51:33.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e5ffff640 1 -- 172.21.15.67:0/1153558024 learned_addr learned my addr 172.21.15.67:0/1153558024 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:33.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e5f7fe640 1 --2- 172.21.15.67:0/1153558024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e60107dd0 0x7f3e6019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:33.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e64f1f640 1 -- 172.21.15.67:0/1153558024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e60107dd0 msgr2=0x7f3e6019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:33.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e64f1f640 1 --2- 172.21.15.67:0/1153558024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e60107dd0 0x7f3e6019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:33.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e64f1f640 1 -- 172.21.15.67:0/1153558024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e60102490 msgr2=0x7f3e60199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:33.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e64f1f640 1 --2- 172.21.15.67:0/1153558024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e60102490 0x7f3e60199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:33.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e64f1f640 1 -- 172.21.15.67:0/1153558024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6019fe30 con 0x7f3e60108720 2024-07-31T14:51:33.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e5ffff640 1 --2- 172.21.15.67:0/1153558024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e60102490 0x7f3e60199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:33.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.183+0000 7f3e64f1f640 1 --2- 172.21.15.67:0/1153558024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e60108720 0x7f3e6019f700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800e940 tx=0x7f3e5800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:33.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.184+0000 7f3e5d7fa640 1 -- 172.21.15.67:0/1153558024 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58004d20 con 0x7f3e60108720 2024-07-31T14:51:33.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.184+0000 7f3e5d7fa640 1 -- 172.21.15.67:0/1153558024 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e58005020 con 0x7f3e60108720 2024-07-31T14:51:33.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.184+0000 7f3e66982640 1 -- 172.21.15.67:0/1153558024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e601a0120 con 0x7f3e60108720 2024-07-31T14:51:33.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.184+0000 7f3e5d7fa640 1 -- 172.21.15.67:0/1153558024 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e580116d0 con 0x7f3e60108720 2024-07-31T14:51:33.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.184+0000 7f3e66982640 1 -- 172.21.15.67:0/1153558024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e601a6880 con 0x7f3e60108720 2024-07-31T14:51:33.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.185+0000 7f3e66982640 1 -- 172.21.15.67:0/1153558024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e601028a0 con 0x7f3e60108720 2024-07-31T14:51:33.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.186+0000 7f3e5d7fa640 1 -- 172.21.15.67:0/1153558024 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f3e58005460 con 0x7f3e60108720 2024-07-31T14:51:33.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.188+0000 7f3e5d7fa640 1 --2- 172.21.15.67:0/1153558024 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e480b3d40 0x7f3e480b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:33.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.188+0000 7f3e5ffff640 1 --2- 172.21.15.67:0/1153558024 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e480b3d40 0x7f3e480b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:33.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.188+0000 7f3e5d7fa640 1 -- 172.21.15.67:0/1153558024 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f3e580e47d0 con 0x7f3e60108720 2024-07-31T14:51:33.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.188+0000 7f3e5ffff640 1 --2- 172.21.15.67:0/1153558024 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e480b3d40 0x7f3e480b6250 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3e50031440 tx=0x7f3e500058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:33.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.191+0000 7f3e5d7fa640 1 -- 172.21.15.67:0/1153558024 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f3e58016030 con 0x7f3e60108720 2024-07-31T14:51:33.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.384+0000 7f3e66982640 1 -- 172.21.15.67:0/1153558024 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3e60063230 con 0x7f3e480b3d40 2024-07-31T14:51:33.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.390+0000 7f3e5d7fa640 1 -- 172.21.15.67:0/1153558024 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340592 (secure 0 0 0) 0x7f3e60063230 con 0x7f3e480b3d40 2024-07-31T14:51:33.395 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:51:33.397 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-31T14:51:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.394+0000 7f3e66982640 1 -- 172.21.15.67:0/1153558024 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e480b3d40 msgr2=0x7f3e480b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.394+0000 7f3e66982640 1 --2- 172.21.15.67:0/1153558024 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e480b3d40 0x7f3e480b6250 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3e50031440 tx=0x7f3e500058d0 comp rx=0 tx=0).stop 2024-07-31T14:51:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.394+0000 7f3e66982640 1 -- 172.21.15.67:0/1153558024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e60108720 msgr2=0x7f3e6019f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.394+0000 7f3e66982640 1 --2- 172.21.15.67:0/1153558024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e60108720 0x7f3e6019f700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800e940 tx=0x7f3e5800ee00 comp rx=0 tx=0).stop 2024-07-31T14:51:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.394+0000 7f3e66982640 1 -- 172.21.15.67:0/1153558024 shutdown_connections 2024-07-31T14:51:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.394+0000 7f3e66982640 1 --2- 172.21.15.67:0/1153558024 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e480b3d40 0x7f3e480b6250 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.394+0000 7f3e66982640 1 --2- 172.21.15.67:0/1153558024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e60108720 0x7f3e6019f700 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.394+0000 7f3e66982640 1 --2- 172.21.15.67:0/1153558024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e60107dd0 0x7f3e6019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.394+0000 7f3e66982640 1 --2- 172.21.15.67:0/1153558024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e60102490 0x7f3e60199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.394+0000 7f3e66982640 1 -- 172.21.15.67:0/1153558024 >> 172.21.15.67:0/1153558024 conn(0x7f3e60079160 msgr2=0x7f3e60100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.394+0000 7f3e66982640 1 -- 172.21.15.67:0/1153558024 shutdown_connections 2024-07-31T14:51:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:33.394+0000 7f3e66982640 1 -- 172.21.15.67:0/1153558024 wait complete. 2024-07-31T14:51:34.031 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-07-31T14:51:31.448982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1458620,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2522,"num_read_kb":1515,"num_write":626,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":974,"ondisk_log_size":974,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386616,"kb_used_data":16484,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555208,"statfs":{"total":1151940427776,"available":1151544532992,"internally_reserved":0,"allocated":16879616,"data_stored":10581230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":378713403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":786992,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":456,"num_read_kb":366,"num_write":224,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003614"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":261,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:24.912708+0000","last_change":"2024-07-31T14:47:00.020067+0000","last_active":"2024-07-31T14:51:24.912708+0000","last_peered":"2024-07-31T14:51:24.912708+0000","last_clean":"2024-07-31T14:51:24.912708+0000","last_became_active":"2024-07-31T14:47:00.019674+0000","last_became_peered":"2024-07-31T14:47:00.019674+0000","last_unstale":"2024-07-31T14:51:24.912708+0000","last_undegraded":"2024-07-31T14:51:24.912708+0000","last_fullsized":"2024-07-31T14:51:24.912708+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:10:08.193786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":205,"num_read_kb":205,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.408739+0000","last_change":"2024-07-31T14:47:03.052069+0000","last_active":"2024-07-31T14:49:49.408739+0000","last_peered":"2024-07-31T14:49:49.408739+0000","last_clean":"2024-07-31T14:49:49.408739+0000","last_became_active":"2024-07-31T14:47:03.051739+0000","last_became_peered":"2024-07-31T14:47:03.051739+0000","last_unstale":"2024-07-31T14:49:49.408739+0000","last_undegraded":"2024-07-31T14:49:49.408739+0000","last_fullsized":"2024-07-31T14:49:49.408739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:29:19.952794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.629384+0000","last_change":"2024-07-31T14:47:20.628770+0000","last_active":"2024-07-31T14:47:20.629384+0000","last_peered":"2024-07-31T14:47:20.629384+0000","last_clean":"2024-07-31T14:47:20.629384+0000","last_became_active":"2024-07-31T14:47:20.628142+0000","last_became_peered":"2024-07-31T14:47:20.628142+0000","last_unstale":"2024-07-31T14:47:20.629384+0000","last_undegraded":"2024-07-31T14:47:20.629384+0000","last_fullsized":"2024-07-31T14:47:20.629384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.659670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:59.291402+0000","last_change":"2024-07-31T14:47:00.183893+0000","last_active":"2024-07-31T14:48:59.291402+0000","last_peered":"2024-07-31T14:48:59.291402+0000","last_clean":"2024-07-31T14:48:59.291402+0000","last_became_active":"2024-07-31T14:47:00.183453+0000","last_became_peered":"2024-07-31T14:47:00.183453+0000","last_unstale":"2024-07-31T14:48:59.291402+0000","last_undegraded":"2024-07-31T14:48:59.291402+0000","last_fullsized":"2024-07-31T14:48:59.291402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:45:08.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404383+0000","last_change":"2024-07-31T14:47:03.057081+0000","last_active":"2024-07-31T14:49:49.404383+0000","last_peered":"2024-07-31T14:49:49.404383+0000","last_clean":"2024-07-31T14:49:49.404383+0000","last_became_active":"2024-07-31T14:47:03.056061+0000","last_became_peered":"2024-07-31T14:47:03.056061+0000","last_unstale":"2024-07-31T14:49:49.404383+0000","last_undegraded":"2024-07-31T14:49:49.404383+0000","last_fullsized":"2024-07-31T14:49:49.404383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:28:16.760131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604121+0000","last_change":"2024-07-31T14:47:13.105271+0000","last_active":"2024-07-31T14:47:20.604121+0000","last_peered":"2024-07-31T14:47:20.604121+0000","last_clean":"2024-07-31T14:47:20.604121+0000","last_became_active":"2024-07-31T14:47:13.104717+0000","last_became_peered":"2024-07-31T14:47:13.104717+0000","last_unstale":"2024-07-31T14:47:20.604121+0000","last_undegraded":"2024-07-31T14:47:20.604121+0000","last_fullsized":"2024-07-31T14:47:20.604121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:34:21.413957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:04.594438+0000","last_change":"2024-07-31T14:47:20.454745+0000","last_active":"2024-07-31T14:48:04.594438+0000","last_peered":"2024-07-31T14:48:04.594438+0000","last_clean":"2024-07-31T14:48:04.594438+0000","last_became_active":"2024-07-31T14:47:20.449892+0000","last_became_peered":"2024-07-31T14:47:20.449892+0000","last_unstale":"2024-07-31T14:48:04.594438+0000","last_undegraded":"2024-07-31T14:48:04.594438+0000","last_fullsized":"2024-07-31T14:48:04.594438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:59:17.375344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.412662+0000","last_change":"2024-07-31T14:47:03.051085+0000","last_active":"2024-07-31T14:49:49.412662+0000","last_peered":"2024-07-31T14:49:49.412662+0000","last_clean":"2024-07-31T14:49:49.412662+0000","last_became_active":"2024-07-31T14:47:03.050686+0000","last_became_peered":"2024-07-31T14:47:03.050686+0000","last_unstale":"2024-07-31T14:49:49.412662+0000","last_undegraded":"2024-07-31T14:49:49.412662+0000","last_fullsized":"2024-07-31T14:49:49.412662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:48:02.494061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604501+0000","last_change":"2024-07-31T14:47:13.104229+0000","last_active":"2024-07-31T14:47:20.604501+0000","last_peered":"2024-07-31T14:47:20.604501+0000","last_clean":"2024-07-31T14:47:20.604501+0000","last_became_active":"2024-07-31T14:47:13.103737+0000","last_became_peered":"2024-07-31T14:47:13.103737+0000","last_unstale":"2024-07-31T14:47:20.604501+0000","last_undegraded":"2024-07-31T14:47:20.604501+0000","last_fullsized":"2024-07-31T14:47:20.604501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:36:00.541345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609858+0000","last_change":"2024-07-31T14:47:00.028169+0000","last_active":"2024-07-31T14:48:03.609858+0000","last_peered":"2024-07-31T14:48:03.609858+0000","last_clean":"2024-07-31T14:48:03.609858+0000","last_became_active":"2024-07-31T14:47:00.027757+0000","last_became_peered":"2024-07-31T14:47:00.027757+0000","last_unstale":"2024-07-31T14:48:03.609858+0000","last_undegraded":"2024-07-31T14:48:03.609858+0000","last_fullsized":"2024-07-31T14:48:03.609858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:36:04.419551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413724+0000","last_change":"2024-07-31T14:47:03.057400+0000","last_active":"2024-07-31T14:49:49.413724+0000","last_peered":"2024-07-31T14:49:49.413724+0000","last_clean":"2024-07-31T14:49:49.413724+0000","last_became_active":"2024-07-31T14:47:03.057035+0000","last_became_peered":"2024-07-31T14:47:03.057035+0000","last_unstale":"2024-07-31T14:49:49.413724+0000","last_undegraded":"2024-07-31T14:49:49.413724+0000","last_fullsized":"2024-07-31T14:49:49.413724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:55:07.951076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604244+0000","last_change":"2024-07-31T14:47:13.106093+0000","last_active":"2024-07-31T14:47:20.604244+0000","last_peered":"2024-07-31T14:47:20.604244+0000","last_clean":"2024-07-31T14:47:20.604244+0000","last_became_active":"2024-07-31T14:47:13.105459+0000","last_became_peered":"2024-07-31T14:47:13.105459+0000","last_unstale":"2024-07-31T14:47:20.604244+0000","last_undegraded":"2024-07-31T14:47:20.604244+0000","last_fullsized":"2024-07-31T14:47:20.604244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:07:52.751245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414509+0000","last_change":"2024-07-31T14:47:03.058701+0000","last_active":"2024-07-31T14:49:49.414509+0000","last_peered":"2024-07-31T14:49:49.414509+0000","last_clean":"2024-07-31T14:49:49.414509+0000","last_became_active":"2024-07-31T14:47:03.058359+0000","last_became_peered":"2024-07-31T14:47:03.058359+0000","last_unstale":"2024-07-31T14:49:49.414509+0000","last_undegraded":"2024-07-31T14:49:49.414509+0000","last_fullsized":"2024-07-31T14:49:49.414509+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:37:36.149884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605126+0000","last_change":"2024-07-31T14:47:00.022983+0000","last_active":"2024-07-31T14:48:03.605126+0000","last_peered":"2024-07-31T14:48:03.605126+0000","last_clean":"2024-07-31T14:48:03.605126+0000","last_became_active":"2024-07-31T14:47:00.022586+0000","last_became_peered":"2024-07-31T14:47:00.022586+0000","last_unstale":"2024-07-31T14:48:03.605126+0000","last_undegraded":"2024-07-31T14:48:03.605126+0000","last_fullsized":"2024-07-31T14:48:03.605126+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:29:42.641490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604474+0000","last_change":"2024-07-31T14:47:13.363148+0000","last_active":"2024-07-31T14:47:20.604474+0000","last_peered":"2024-07-31T14:47:20.604474+0000","last_clean":"2024-07-31T14:47:20.604474+0000","last_became_active":"2024-07-31T14:47:13.362577+0000","last_became_peered":"2024-07-31T14:47:13.362577+0000","last_unstale":"2024-07-31T14:47:20.604474+0000","last_undegraded":"2024-07-31T14:47:20.604474+0000","last_fullsized":"2024-07-31T14:47:20.604474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:37:59.003260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409563+0000","last_change":"2024-07-31T14:47:03.049586+0000","last_active":"2024-07-31T14:49:49.409563+0000","last_peered":"2024-07-31T14:49:49.409563+0000","last_clean":"2024-07-31T14:49:49.409563+0000","last_became_active":"2024-07-31T14:47:03.049154+0000","last_became_peered":"2024-07-31T14:47:03.049154+0000","last_unstale":"2024-07-31T14:49:49.409563+0000","last_undegraded":"2024-07-31T14:49:49.409563+0000","last_fullsized":"2024-07-31T14:49:49.409563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:51:01.971854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:09.610387+0000","last_change":"2024-07-31T14:47:00.021254+0000","last_active":"2024-07-31T14:51:09.610387+0000","last_peered":"2024-07-31T14:51:09.610387+0000","last_clean":"2024-07-31T14:51:09.610387+0000","last_became_active":"2024-07-31T14:47:00.020690+0000","last_became_peered":"2024-07-31T14:47:00.020690+0000","last_unstale":"2024-07-31T14:51:09.610387+0000","last_undegraded":"2024-07-31T14:51:09.610387+0000","last_fullsized":"2024-07-31T14:51:09.610387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:01:27.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:18.046730+0000","last_change":"2024-07-31T14:47:13.102220+0000","last_active":"2024-07-31T14:48:18.046730+0000","last_peered":"2024-07-31T14:48:18.046730+0000","last_clean":"2024-07-31T14:48:18.046730+0000","last_became_active":"2024-07-31T14:47:13.101764+0000","last_became_peered":"2024-07-31T14:47:13.101764+0000","last_unstale":"2024-07-31T14:48:18.046730+0000","last_undegraded":"2024-07-31T14:48:18.046730+0000","last_fullsized":"2024-07-31T14:48:18.046730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:56:44.335340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.398408+0000","last_change":"2024-07-31T14:47:03.052348+0000","last_active":"2024-07-31T14:49:49.398408+0000","last_peered":"2024-07-31T14:49:49.398408+0000","last_clean":"2024-07-31T14:49:49.398408+0000","last_became_active":"2024-07-31T14:47:03.051961+0000","last_became_peered":"2024-07-31T14:47:03.051961+0000","last_unstale":"2024-07-31T14:49:49.398408+0000","last_undegraded":"2024-07-31T14:49:49.398408+0000","last_fullsized":"2024-07-31T14:49:49.398408+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:19:35.279809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:09.292622+0000","last_change":"2024-07-31T14:47:00.025013+0000","last_active":"2024-07-31T14:49:09.292622+0000","last_peered":"2024-07-31T14:49:09.292622+0000","last_clean":"2024-07-31T14:49:09.292622+0000","last_became_active":"2024-07-31T14:47:00.024637+0000","last_became_peered":"2024-07-31T14:47:00.024637+0000","last_unstale":"2024-07-31T14:49:09.292622+0000","last_undegraded":"2024-07-31T14:49:09.292622+0000","last_fullsized":"2024-07-31T14:49:09.292622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:07:21.519549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630339+0000","last_change":"2024-07-31T14:47:20.629262+0000","last_active":"2024-07-31T14:47:20.630339+0000","last_peered":"2024-07-31T14:47:20.630339+0000","last_clean":"2024-07-31T14:47:20.630339+0000","last_became_active":"2024-07-31T14:47:20.628318+0000","last_became_peered":"2024-07-31T14:47:20.628318+0000","last_unstale":"2024-07-31T14:47:20.630339+0000","last_undegraded":"2024-07-31T14:47:20.630339+0000","last_fullsized":"2024-07-31T14:47:20.630339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:59:08.206705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.387553+0000","last_change":"2024-07-31T14:47:03.051527+0000","last_active":"2024-07-31T14:49:49.387553+0000","last_peered":"2024-07-31T14:49:49.387553+0000","last_clean":"2024-07-31T14:49:49.387553+0000","last_became_active":"2024-07-31T14:47:03.051139+0000","last_became_peered":"2024-07-31T14:47:03.051139+0000","last_unstale":"2024-07-31T14:49:49.387553+0000","last_undegraded":"2024-07-31T14:49:49.387553+0000","last_fullsized":"2024-07-31T14:49:49.387553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:53:11.962719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:09.210377+0000","last_change":"2024-07-31T14:47:00.183812+0000","last_active":"2024-07-31T14:49:09.210377+0000","last_peered":"2024-07-31T14:49:09.210377+0000","last_clean":"2024-07-31T14:49:09.210377+0000","last_became_active":"2024-07-31T14:47:00.183440+0000","last_became_peered":"2024-07-31T14:47:00.183440+0000","last_unstale":"2024-07-31T14:49:09.210377+0000","last_undegraded":"2024-07-31T14:49:09.210377+0000","last_fullsized":"2024-07-31T14:49:09.210377+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:47:23.166756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604338+0000","last_change":"2024-07-31T14:47:13.363563+0000","last_active":"2024-07-31T14:47:20.604338+0000","last_peered":"2024-07-31T14:47:20.604338+0000","last_clean":"2024-07-31T14:47:20.604338+0000","last_became_active":"2024-07-31T14:47:13.362697+0000","last_became_peered":"2024-07-31T14:47:13.362697+0000","last_unstale":"2024-07-31T14:47:20.604338+0000","last_undegraded":"2024-07-31T14:47:20.604338+0000","last_fullsized":"2024-07-31T14:47:20.604338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:26:38.292073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407835+0000","last_change":"2024-07-31T14:47:03.046763+0000","last_active":"2024-07-31T14:49:49.407835+0000","last_peered":"2024-07-31T14:49:49.407835+0000","last_clean":"2024-07-31T14:49:49.407835+0000","last_became_active":"2024-07-31T14:47:03.046408+0000","last_became_peered":"2024-07-31T14:47:03.046408+0000","last_unstale":"2024-07-31T14:49:49.407835+0000","last_undegraded":"2024-07-31T14:49:49.407835+0000","last_fullsized":"2024-07-31T14:49:49.407835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:43:22.838495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:09.380966+0000","last_change":"2024-07-31T14:47:00.236075+0000","last_active":"2024-07-31T14:49:09.380966+0000","last_peered":"2024-07-31T14:49:09.380966+0000","last_clean":"2024-07-31T14:49:09.380966+0000","last_became_active":"2024-07-31T14:47:00.235758+0000","last_became_peered":"2024-07-31T14:47:00.235758+0000","last_unstale":"2024-07-31T14:49:09.380966+0000","last_undegraded":"2024-07-31T14:49:09.380966+0000","last_fullsized":"2024-07-31T14:49:09.380966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:50.575648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:21.453373+0000","last_change":"2024-07-31T14:47:13.101876+0000","last_active":"2024-07-31T14:49:21.453373+0000","last_peered":"2024-07-31T14:49:21.453373+0000","last_clean":"2024-07-31T14:49:21.453373+0000","last_became_active":"2024-07-31T14:47:13.101276+0000","last_became_peered":"2024-07-31T14:47:13.101276+0000","last_unstale":"2024-07-31T14:49:21.453373+0000","last_undegraded":"2024-07-31T14:49:21.453373+0000","last_fullsized":"2024-07-31T14:49:21.453373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:53:24.684212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409565+0000","last_change":"2024-07-31T14:47:03.052478+0000","last_active":"2024-07-31T14:49:49.409565+0000","last_peered":"2024-07-31T14:49:49.409565+0000","last_clean":"2024-07-31T14:49:49.409565+0000","last_became_active":"2024-07-31T14:47:03.051877+0000","last_became_peered":"2024-07-31T14:47:03.051877+0000","last_unstale":"2024-07-31T14:49:49.409565+0000","last_undegraded":"2024-07-31T14:49:49.409565+0000","last_fullsized":"2024-07-31T14:49:49.409565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:30:32.666490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'101","reported_seq":232,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:22.587242+0000","last_change":"2024-07-31T14:47:00.020076+0000","last_active":"2024-07-31T14:51:22.587242+0000","last_peered":"2024-07-31T14:51:22.587242+0000","last_clean":"2024-07-31T14:51:22.587242+0000","last_became_active":"2024-07-31T14:47:00.019708+0000","last_became_peered":"2024-07-31T14:47:00.019708+0000","last_unstale":"2024-07-31T14:51:22.587242+0000","last_undegraded":"2024-07-31T14:51:22.587242+0000","last_fullsized":"2024-07-31T14:51:22.587242+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:46.520997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":186,"num_write_kb":1223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:14.293424+0000","last_change":"2024-07-31T14:47:13.362961+0000","last_active":"2024-07-31T14:49:14.293424+0000","last_peered":"2024-07-31T14:49:14.293424+0000","last_clean":"2024-07-31T14:49:14.293424+0000","last_became_active":"2024-07-31T14:47:13.362502+0000","last_became_peered":"2024-07-31T14:47:13.362502+0000","last_unstale":"2024-07-31T14:49:14.293424+0000","last_undegraded":"2024-07-31T14:49:14.293424+0000","last_fullsized":"2024-07-31T14:49:14.293424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:51:45.732477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399783+0000","last_change":"2024-07-31T14:47:03.039418+0000","last_active":"2024-07-31T14:49:49.399783+0000","last_peered":"2024-07-31T14:49:49.399783+0000","last_clean":"2024-07-31T14:49:49.399783+0000","last_became_active":"2024-07-31T14:47:03.039044+0000","last_became_peered":"2024-07-31T14:47:03.039044+0000","last_unstale":"2024-07-31T14:49:49.399783+0000","last_undegraded":"2024-07-31T14:49:49.399783+0000","last_fullsized":"2024-07-31T14:49:49.399783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:40:10.380080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:07.059440+0000","last_change":"2024-07-31T14:47:00.183778+0000","last_active":"2024-07-31T14:49:07.059440+0000","last_peered":"2024-07-31T14:49:07.059440+0000","last_clean":"2024-07-31T14:49:07.059440+0000","last_became_active":"2024-07-31T14:47:00.183370+0000","last_became_peered":"2024-07-31T14:47:00.183370+0000","last_unstale":"2024-07-31T14:49:07.059440+0000","last_undegraded":"2024-07-31T14:49:07.059440+0000","last_fullsized":"2024-07-31T14:49:07.059440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:45:27.422216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:19.188833+0000","last_change":"2024-07-31T14:47:13.102583+0000","last_active":"2024-07-31T14:49:19.188833+0000","last_peered":"2024-07-31T14:49:19.188833+0000","last_clean":"2024-07-31T14:49:19.188833+0000","last_became_active":"2024-07-31T14:47:13.102058+0000","last_became_peered":"2024-07-31T14:47:13.102058+0000","last_unstale":"2024-07-31T14:49:19.188833+0000","last_undegraded":"2024-07-31T14:49:19.188833+0000","last_fullsized":"2024-07-31T14:49:19.188833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:17.396173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.408095+0000","last_change":"2024-07-31T14:47:03.048334+0000","last_active":"2024-07-31T14:49:49.408095+0000","last_peered":"2024-07-31T14:49:49.408095+0000","last_clean":"2024-07-31T14:49:49.408095+0000","last_became_active":"2024-07-31T14:47:03.047932+0000","last_became_peered":"2024-07-31T14:47:03.047932+0000","last_unstale":"2024-07-31T14:49:49.408095+0000","last_undegraded":"2024-07-31T14:49:49.408095+0000","last_fullsized":"2024-07-31T14:49:49.408095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:14:31.381061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:06.494830+0000","last_change":"2024-07-31T14:47:20.450851+0000","last_active":"2024-07-31T14:49:06.494830+0000","last_peered":"2024-07-31T14:49:06.494830+0000","last_clean":"2024-07-31T14:49:06.494830+0000","last_became_active":"2024-07-31T14:47:20.450435+0000","last_became_peered":"2024-07-31T14:47:20.450435+0000","last_unstale":"2024-07-31T14:49:06.494830+0000","last_undegraded":"2024-07-31T14:49:06.494830+0000","last_fullsized":"2024-07-31T14:49:06.494830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:17:28.738911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:22.061505+0000","last_change":"2024-07-31T14:47:13.363054+0000","last_active":"2024-07-31T14:49:22.061505+0000","last_peered":"2024-07-31T14:49:22.061505+0000","last_clean":"2024-07-31T14:49:22.061505+0000","last_became_active":"2024-07-31T14:47:13.362364+0000","last_became_peered":"2024-07-31T14:47:13.362364+0000","last_unstale":"2024-07-31T14:49:22.061505+0000","last_undegraded":"2024-07-31T14:49:22.061505+0000","last_fullsized":"2024-07-31T14:49:22.061505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:02:36.569678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.393338+0000","last_change":"2024-07-31T14:47:03.059834+0000","last_active":"2024-07-31T14:49:49.393338+0000","last_peered":"2024-07-31T14:49:49.393338+0000","last_clean":"2024-07-31T14:49:49.393338+0000","last_became_active":"2024-07-31T14:47:03.059127+0000","last_became_peered":"2024-07-31T14:47:03.059127+0000","last_unstale":"2024-07-31T14:49:49.393338+0000","last_undegraded":"2024-07-31T14:49:49.393338+0000","last_fullsized":"2024-07-31T14:49:49.393338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:53:50.308789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":256,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:24.913236+0000","last_change":"2024-07-31T14:47:00.030576+0000","last_active":"2024-07-31T14:51:24.913236+0000","last_peered":"2024-07-31T14:51:24.913236+0000","last_clean":"2024-07-31T14:51:24.913236+0000","last_became_active":"2024-07-31T14:47:00.030174+0000","last_became_peered":"2024-07-31T14:47:00.030174+0000","last_unstale":"2024-07-31T14:51:24.913236+0000","last_undegraded":"2024-07-31T14:51:24.913236+0000","last_fullsized":"2024-07-31T14:51:24.913236+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:13:35.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":206,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:18.289527+0000","last_change":"2024-07-31T14:47:13.100180+0000","last_active":"2024-07-31T14:49:18.289527+0000","last_peered":"2024-07-31T14:49:18.289527+0000","last_clean":"2024-07-31T14:49:18.289527+0000","last_became_active":"2024-07-31T14:47:13.099725+0000","last_became_peered":"2024-07-31T14:47:13.099725+0000","last_unstale":"2024-07-31T14:49:18.289527+0000","last_undegraded":"2024-07-31T14:49:18.289527+0000","last_fullsized":"2024-07-31T14:49:18.289527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:25:32.245339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404277+0000","last_change":"2024-07-31T14:47:03.040349+0000","last_active":"2024-07-31T14:49:49.404277+0000","last_peered":"2024-07-31T14:49:49.404277+0000","last_clean":"2024-07-31T14:49:49.404277+0000","last_became_active":"2024-07-31T14:47:03.039887+0000","last_became_peered":"2024-07-31T14:47:03.039887+0000","last_unstale":"2024-07-31T14:49:49.404277+0000","last_undegraded":"2024-07-31T14:49:49.404277+0000","last_fullsized":"2024-07-31T14:49:49.404277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:46.592063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":256,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:26.902207+0000","last_change":"2024-07-31T14:47:00.183988+0000","last_active":"2024-07-31T14:51:26.902207+0000","last_peered":"2024-07-31T14:51:26.902207+0000","last_clean":"2024-07-31T14:51:26.902207+0000","last_became_active":"2024-07-31T14:47:00.183302+0000","last_became_peered":"2024-07-31T14:47:00.183302+0000","last_unstale":"2024-07-31T14:51:26.902207+0000","last_undegraded":"2024-07-31T14:51:26.902207+0000","last_fullsized":"2024-07-31T14:51:26.902207+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:40:26.188224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447744+0000","last_change":"2024-07-31T14:47:13.363075+0000","last_active":"2024-07-31T14:47:20.447744+0000","last_peered":"2024-07-31T14:47:20.447744+0000","last_clean":"2024-07-31T14:47:20.447744+0000","last_became_active":"2024-07-31T14:47:13.362599+0000","last_became_peered":"2024-07-31T14:47:13.362599+0000","last_unstale":"2024-07-31T14:47:20.447744+0000","last_undegraded":"2024-07-31T14:47:20.447744+0000","last_fullsized":"2024-07-31T14:47:20.447744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:20:43.617742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.402186+0000","last_change":"2024-07-31T14:47:20.450712+0000","last_active":"2024-07-31T14:49:49.402186+0000","last_peered":"2024-07-31T14:49:49.402186+0000","last_clean":"2024-07-31T14:49:49.402186+0000","last_became_active":"2024-07-31T14:47:20.450229+0000","last_became_peered":"2024-07-31T14:47:20.450229+0000","last_unstale":"2024-07-31T14:49:49.402186+0000","last_undegraded":"2024-07-31T14:49:49.402186+0000","last_fullsized":"2024-07-31T14:49:49.402186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:11:50.631420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:01.604764+0000","last_change":"2024-07-31T14:47:00.021389+0000","last_active":"2024-07-31T14:49:01.604764+0000","last_peered":"2024-07-31T14:49:01.604764+0000","last_clean":"2024-07-31T14:49:01.604764+0000","last_became_active":"2024-07-31T14:47:00.020987+0000","last_became_peered":"2024-07-31T14:47:00.020987+0000","last_unstale":"2024-07-31T14:49:01.604764+0000","last_undegraded":"2024-07-31T14:49:01.604764+0000","last_fullsized":"2024-07-31T14:49:01.604764+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:07:24.658463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.627003+0000","last_change":"2024-07-31T14:47:13.363105+0000","last_active":"2024-07-31T14:47:20.627003+0000","last_peered":"2024-07-31T14:47:20.627003+0000","last_clean":"2024-07-31T14:47:20.627003+0000","last_became_active":"2024-07-31T14:47:13.362404+0000","last_became_peered":"2024-07-31T14:47:13.362404+0000","last_unstale":"2024-07-31T14:47:20.627003+0000","last_undegraded":"2024-07-31T14:47:20.627003+0000","last_fullsized":"2024-07-31T14:47:20.627003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:48:43.033584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404873+0000","last_change":"2024-07-31T14:47:03.057199+0000","last_active":"2024-07-31T14:49:49.404873+0000","last_peered":"2024-07-31T14:49:49.404873+0000","last_clean":"2024-07-31T14:49:49.404873+0000","last_became_active":"2024-07-31T14:47:03.056555+0000","last_became_peered":"2024-07-31T14:47:03.056555+0000","last_unstale":"2024-07-31T14:49:49.404873+0000","last_undegraded":"2024-07-31T14:49:49.404873+0000","last_fullsized":"2024-07-31T14:49:49.404873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:21:14.263457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:04.380224+0000","last_change":"2024-07-31T14:47:00.032055+0000","last_active":"2024-07-31T14:49:04.380224+0000","last_peered":"2024-07-31T14:49:04.380224+0000","last_clean":"2024-07-31T14:49:04.380224+0000","last_became_active":"2024-07-31T14:47:00.031519+0000","last_became_peered":"2024-07-31T14:47:00.031519+0000","last_unstale":"2024-07-31T14:49:04.380224+0000","last_undegraded":"2024-07-31T14:49:04.380224+0000","last_fullsized":"2024-07-31T14:49:04.380224+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:21:46.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:31.600736+0000","last_change":"2024-07-31T14:45:42.166503+0000","last_active":"2024-07-31T14:48:31.600736+0000","last_peered":"2024-07-31T14:48:31.600736+0000","last_clean":"2024-07-31T14:48:31.600736+0000","last_became_active":"2024-07-31T14:45:42.160065+0000","last_became_peered":"2024-07-31T14:45:42.160065+0000","last_unstale":"2024-07-31T14:48:31.600736+0000","last_undegraded":"2024-07-31T14:48:31.600736+0000","last_fullsized":"2024-07-31T14:48:31.600736+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:42:27.863113+0000","last_clean_scrub_stamp":"2024-07-31T14:42:27.863113+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:38:33.646022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:21.607394+0000","last_change":"2024-07-31T14:47:13.103812+0000","last_active":"2024-07-31T14:49:21.607394+0000","last_peered":"2024-07-31T14:49:21.607394+0000","last_clean":"2024-07-31T14:49:21.607394+0000","last_became_active":"2024-07-31T14:47:13.102549+0000","last_became_peered":"2024-07-31T14:47:13.102549+0000","last_unstale":"2024-07-31T14:49:21.607394+0000","last_undegraded":"2024-07-31T14:49:21.607394+0000","last_fullsized":"2024-07-31T14:49:21.607394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:18:56.322426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414083+0000","last_change":"2024-07-31T14:47:03.056310+0000","last_active":"2024-07-31T14:49:49.414083+0000","last_peered":"2024-07-31T14:49:49.414083+0000","last_clean":"2024-07-31T14:49:49.414083+0000","last_became_active":"2024-07-31T14:47:03.055695+0000","last_became_peered":"2024-07-31T14:47:03.055695+0000","last_unstale":"2024-07-31T14:49:49.414083+0000","last_undegraded":"2024-07-31T14:49:49.414083+0000","last_fullsized":"2024-07-31T14:49:49.414083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:00:55.124981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":255,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:24.913638+0000","last_change":"2024-07-31T14:47:00.236460+0000","last_active":"2024-07-31T14:51:24.913638+0000","last_peered":"2024-07-31T14:51:24.913638+0000","last_clean":"2024-07-31T14:51:24.913638+0000","last_became_active":"2024-07-31T14:47:00.235924+0000","last_became_peered":"2024-07-31T14:47:00.235924+0000","last_unstale":"2024-07-31T14:51:24.913638+0000","last_undegraded":"2024-07-31T14:51:24.913638+0000","last_fullsized":"2024-07-31T14:51:24.913638+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:05.130958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":207,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626080+0000","last_change":"2024-07-31T14:47:13.102341+0000","last_active":"2024-07-31T14:47:20.626080+0000","last_peered":"2024-07-31T14:47:20.626080+0000","last_clean":"2024-07-31T14:47:20.626080+0000","last_became_active":"2024-07-31T14:47:13.101798+0000","last_became_peered":"2024-07-31T14:47:13.101798+0000","last_unstale":"2024-07-31T14:47:20.626080+0000","last_undegraded":"2024-07-31T14:47:20.626080+0000","last_fullsized":"2024-07-31T14:47:20.626080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:34:35.758041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409700+0000","last_change":"2024-07-31T14:47:03.054322+0000","last_active":"2024-07-31T14:49:49.409700+0000","last_peered":"2024-07-31T14:49:49.409700+0000","last_clean":"2024-07-31T14:49:49.409700+0000","last_became_active":"2024-07-31T14:47:03.053112+0000","last_became_peered":"2024-07-31T14:47:03.053112+0000","last_unstale":"2024-07-31T14:49:49.409700+0000","last_undegraded":"2024-07-31T14:49:49.409700+0000","last_fullsized":"2024-07-31T14:49:49.409700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:55:13.542578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'62","reported_seq":134,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:22.197100+0000","last_change":"2024-07-31T14:47:00.026518+0000","last_active":"2024-07-31T14:51:22.197100+0000","last_peered":"2024-07-31T14:51:22.197100+0000","last_clean":"2024-07-31T14:51:22.197100+0000","last_became_active":"2024-07-31T14:47:00.026074+0000","last_became_peered":"2024-07-31T14:47:00.026074+0000","last_unstale":"2024-07-31T14:51:22.197100+0000","last_undegraded":"2024-07-31T14:51:22.197100+0000","last_fullsized":"2024-07-31T14:51:22.197100+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:33:36.688353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.449369+0000","last_change":"2024-07-31T14:47:13.106365+0000","last_active":"2024-07-31T14:47:20.449369+0000","last_peered":"2024-07-31T14:47:20.449369+0000","last_clean":"2024-07-31T14:47:20.449369+0000","last_became_active":"2024-07-31T14:47:13.105635+0000","last_became_peered":"2024-07-31T14:47:13.105635+0000","last_unstale":"2024-07-31T14:47:20.449369+0000","last_undegraded":"2024-07-31T14:47:20.449369+0000","last_fullsized":"2024-07-31T14:47:20.449369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:52:09.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404285+0000","last_change":"2024-07-31T14:47:03.052522+0000","last_active":"2024-07-31T14:49:49.404285+0000","last_peered":"2024-07-31T14:49:49.404285+0000","last_clean":"2024-07-31T14:49:49.404285+0000","last_became_active":"2024-07-31T14:47:03.051872+0000","last_became_peered":"2024-07-31T14:47:03.051872+0000","last_unstale":"2024-07-31T14:49:49.404285+0000","last_undegraded":"2024-07-31T14:49:49.404285+0000","last_fullsized":"2024-07-31T14:49:49.404285+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:29:34.956745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600950+0000","last_change":"2024-07-31T14:47:00.016839+0000","last_active":"2024-07-31T14:48:02.600950+0000","last_peered":"2024-07-31T14:48:02.600950+0000","last_clean":"2024-07-31T14:48:02.600950+0000","last_became_active":"2024-07-31T14:47:00.016438+0000","last_became_peered":"2024-07-31T14:47:00.016438+0000","last_unstale":"2024-07-31T14:48:02.600950+0000","last_undegraded":"2024-07-31T14:48:02.600950+0000","last_fullsized":"2024-07-31T14:48:02.600950+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:42:10.875332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626258+0000","last_change":"2024-07-31T14:47:13.105385+0000","last_active":"2024-07-31T14:47:20.626258+0000","last_peered":"2024-07-31T14:47:20.626258+0000","last_clean":"2024-07-31T14:47:20.626258+0000","last_became_active":"2024-07-31T14:47:13.104867+0000","last_became_peered":"2024-07-31T14:47:13.104867+0000","last_unstale":"2024-07-31T14:47:20.626258+0000","last_undegraded":"2024-07-31T14:47:20.626258+0000","last_fullsized":"2024-07-31T14:47:20.626258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:25:21.572830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399072+0000","last_change":"2024-07-31T14:47:03.058304+0000","last_active":"2024-07-31T14:49:49.399072+0000","last_peered":"2024-07-31T14:49:49.399072+0000","last_clean":"2024-07-31T14:49:49.399072+0000","last_became_active":"2024-07-31T14:47:03.057251+0000","last_became_peered":"2024-07-31T14:47:03.057251+0000","last_unstale":"2024-07-31T14:49:49.399072+0000","last_undegraded":"2024-07-31T14:49:49.399072+0000","last_fullsized":"2024-07-31T14:49:49.399072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:58:20.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605301+0000","last_change":"2024-07-31T14:47:00.027806+0000","last_active":"2024-07-31T14:48:03.605301+0000","last_peered":"2024-07-31T14:48:03.605301+0000","last_clean":"2024-07-31T14:48:03.605301+0000","last_became_active":"2024-07-31T14:47:00.027403+0000","last_became_peered":"2024-07-31T14:47:00.027403+0000","last_unstale":"2024-07-31T14:48:03.605301+0000","last_undegraded":"2024-07-31T14:48:03.605301+0000","last_fullsized":"2024-07-31T14:48:03.605301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:57:18.303732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626490+0000","last_change":"2024-07-31T14:47:13.103628+0000","last_active":"2024-07-31T14:47:20.626490+0000","last_peered":"2024-07-31T14:47:20.626490+0000","last_clean":"2024-07-31T14:47:20.626490+0000","last_became_active":"2024-07-31T14:47:13.102296+0000","last_became_peered":"2024-07-31T14:47:13.102296+0000","last_unstale":"2024-07-31T14:47:20.626490+0000","last_undegraded":"2024-07-31T14:47:20.626490+0000","last_fullsized":"2024-07-31T14:47:20.626490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:25:35.109196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413387+0000","last_change":"2024-07-31T14:47:03.052761+0000","last_active":"2024-07-31T14:49:49.413387+0000","last_peered":"2024-07-31T14:49:49.413387+0000","last_clean":"2024-07-31T14:49:49.413387+0000","last_became_active":"2024-07-31T14:47:03.052410+0000","last_became_peered":"2024-07-31T14:47:03.052410+0000","last_unstale":"2024-07-31T14:49:49.413387+0000","last_undegraded":"2024-07-31T14:49:49.413387+0000","last_fullsized":"2024-07-31T14:49:49.413387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:20:47.712491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.604937+0000","last_change":"2024-07-31T14:47:00.183741+0000","last_active":"2024-07-31T14:48:03.604937+0000","last_peered":"2024-07-31T14:48:03.604937+0000","last_clean":"2024-07-31T14:48:03.604937+0000","last_became_active":"2024-07-31T14:47:00.183275+0000","last_became_peered":"2024-07-31T14:47:00.183275+0000","last_unstale":"2024-07-31T14:48:03.604937+0000","last_undegraded":"2024-07-31T14:48:03.604937+0000","last_fullsized":"2024-07-31T14:48:03.604937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:29:23.205835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.630098+0000","last_change":"2024-07-31T14:47:20.629334+0000","last_active":"2024-07-31T14:47:20.630098+0000","last_peered":"2024-07-31T14:47:20.630098+0000","last_clean":"2024-07-31T14:47:20.630098+0000","last_became_active":"2024-07-31T14:47:20.628865+0000","last_became_peered":"2024-07-31T14:47:20.628865+0000","last_unstale":"2024-07-31T14:47:20.630098+0000","last_undegraded":"2024-07-31T14:47:20.630098+0000","last_fullsized":"2024-07-31T14:47:20.630098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:21:27.199842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.407413+0000","last_change":"2024-07-31T14:47:03.054633+0000","last_active":"2024-07-31T14:49:49.407413+0000","last_peered":"2024-07-31T14:49:49.407413+0000","last_clean":"2024-07-31T14:49:49.407413+0000","last_became_active":"2024-07-31T14:47:03.053025+0000","last_became_peered":"2024-07-31T14:47:03.053025+0000","last_unstale":"2024-07-31T14:49:49.407413+0000","last_undegraded":"2024-07-31T14:49:49.407413+0000","last_fullsized":"2024-07-31T14:49:49.407413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:42:03.392744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":261,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:27.904102+0000","last_change":"2024-07-31T14:47:00.027898+0000","last_active":"2024-07-31T14:51:27.904102+0000","last_peered":"2024-07-31T14:51:27.904102+0000","last_clean":"2024-07-31T14:51:27.904102+0000","last_became_active":"2024-07-31T14:47:00.027468+0000","last_became_peered":"2024-07-31T14:47:00.027468+0000","last_unstale":"2024-07-31T14:51:27.904102+0000","last_undegraded":"2024-07-31T14:51:27.904102+0000","last_fullsized":"2024-07-31T14:51:27.904102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:12:45.922172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":212,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444574+0000","last_change":"2024-07-31T14:47:13.104327+0000","last_active":"2024-07-31T14:47:20.444574+0000","last_peered":"2024-07-31T14:47:20.444574+0000","last_clean":"2024-07-31T14:47:20.444574+0000","last_became_active":"2024-07-31T14:47:13.103333+0000","last_became_peered":"2024-07-31T14:47:13.103333+0000","last_unstale":"2024-07-31T14:47:20.444574+0000","last_undegraded":"2024-07-31T14:47:20.444574+0000","last_fullsized":"2024-07-31T14:47:20.444574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:06.226220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.410145+0000","last_change":"2024-07-31T14:47:03.044875+0000","last_active":"2024-07-31T14:49:49.410145+0000","last_peered":"2024-07-31T14:49:49.410145+0000","last_clean":"2024-07-31T14:49:49.410145+0000","last_became_active":"2024-07-31T14:47:03.044421+0000","last_became_peered":"2024-07-31T14:47:03.044421+0000","last_unstale":"2024-07-31T14:49:49.410145+0000","last_undegraded":"2024-07-31T14:49:49.410145+0000","last_fullsized":"2024-07-31T14:49:49.410145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:00:02.477300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605606+0000","last_change":"2024-07-31T14:47:00.019790+0000","last_active":"2024-07-31T14:48:03.605606+0000","last_peered":"2024-07-31T14:48:03.605606+0000","last_clean":"2024-07-31T14:48:03.605606+0000","last_became_active":"2024-07-31T14:47:00.019368+0000","last_became_peered":"2024-07-31T14:47:00.019368+0000","last_unstale":"2024-07-31T14:48:03.605606+0000","last_undegraded":"2024-07-31T14:48:03.605606+0000","last_fullsized":"2024-07-31T14:48:03.605606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:15.299109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.445672+0000","last_change":"2024-07-31T14:47:13.102618+0000","last_active":"2024-07-31T14:47:20.445672+0000","last_peered":"2024-07-31T14:47:20.445672+0000","last_clean":"2024-07-31T14:47:20.445672+0000","last_became_active":"2024-07-31T14:47:13.102085+0000","last_became_peered":"2024-07-31T14:47:13.102085+0000","last_unstale":"2024-07-31T14:47:20.445672+0000","last_undegraded":"2024-07-31T14:47:20.445672+0000","last_fullsized":"2024-07-31T14:47:20.445672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:05:58.814215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.406277+0000","last_change":"2024-07-31T14:47:03.044688+0000","last_active":"2024-07-31T14:49:49.406277+0000","last_peered":"2024-07-31T14:49:49.406277+0000","last_clean":"2024-07-31T14:49:49.406277+0000","last_became_active":"2024-07-31T14:47:03.043751+0000","last_became_peered":"2024-07-31T14:47:03.043751+0000","last_unstale":"2024-07-31T14:49:49.406277+0000","last_undegraded":"2024-07-31T14:49:49.406277+0000","last_fullsized":"2024-07-31T14:49:49.406277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:47:31.450796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.605830+0000","last_change":"2024-07-31T14:47:00.026259+0000","last_active":"2024-07-31T14:48:03.605830+0000","last_peered":"2024-07-31T14:48:03.605830+0000","last_clean":"2024-07-31T14:48:03.605830+0000","last_became_active":"2024-07-31T14:47:00.025756+0000","last_became_peered":"2024-07-31T14:47:00.025756+0000","last_unstale":"2024-07-31T14:48:03.605830+0000","last_undegraded":"2024-07-31T14:48:03.605830+0000","last_fullsized":"2024-07-31T14:48:03.605830+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:05:00.258841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626888+0000","last_change":"2024-07-31T14:47:13.104918+0000","last_active":"2024-07-31T14:47:20.626888+0000","last_peered":"2024-07-31T14:47:20.626888+0000","last_clean":"2024-07-31T14:47:20.626888+0000","last_became_active":"2024-07-31T14:47:13.104031+0000","last_became_peered":"2024-07-31T14:47:13.104031+0000","last_unstale":"2024-07-31T14:47:20.626888+0000","last_undegraded":"2024-07-31T14:47:20.626888+0000","last_fullsized":"2024-07-31T14:47:20.626888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:17:15.264474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.389841+0000","last_change":"2024-07-31T14:47:03.055159+0000","last_active":"2024-07-31T14:49:49.389841+0000","last_peered":"2024-07-31T14:49:49.389841+0000","last_clean":"2024-07-31T14:49:49.389841+0000","last_became_active":"2024-07-31T14:47:03.052389+0000","last_became_peered":"2024-07-31T14:47:03.052389+0000","last_unstale":"2024-07-31T14:49:49.389841+0000","last_undegraded":"2024-07-31T14:49:49.389841+0000","last_fullsized":"2024-07-31T14:49:49.389841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:16.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'42","reported_seq":512,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:29.906427+0000","last_change":"2024-07-31T14:47:00.030165+0000","last_active":"2024-07-31T14:51:29.906427+0000","last_peered":"2024-07-31T14:51:29.906427+0000","last_clean":"2024-07-31T14:51:29.906427+0000","last_became_active":"2024-07-31T14:47:00.029559+0000","last_became_peered":"2024-07-31T14:47:00.029559+0000","last_unstale":"2024-07-31T14:51:29.906427+0000","last_undegraded":"2024-07-31T14:51:29.906427+0000","last_fullsized":"2024-07-31T14:51:29.906427+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:31:06.115075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71918,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":428,"num_read_kb":285,"num_write":44,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.444234+0000","last_change":"2024-07-31T14:47:13.098517+0000","last_active":"2024-07-31T14:47:20.444234+0000","last_peered":"2024-07-31T14:47:20.444234+0000","last_clean":"2024-07-31T14:47:20.444234+0000","last_became_active":"2024-07-31T14:47:13.098066+0000","last_became_peered":"2024-07-31T14:47:13.098066+0000","last_unstale":"2024-07-31T14:47:20.444234+0000","last_undegraded":"2024-07-31T14:47:20.444234+0000","last_fullsized":"2024-07-31T14:47:20.444234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:08:54.958646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414451+0000","last_change":"2024-07-31T14:47:03.054045+0000","last_active":"2024-07-31T14:49:49.414451+0000","last_peered":"2024-07-31T14:49:49.414451+0000","last_clean":"2024-07-31T14:49:49.414451+0000","last_became_active":"2024-07-31T14:47:03.053658+0000","last_became_peered":"2024-07-31T14:47:03.053658+0000","last_unstale":"2024-07-31T14:49:49.414451+0000","last_undegraded":"2024-07-31T14:49:49.414451+0000","last_fullsized":"2024-07-31T14:49:49.414451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:36:54.563542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.607373+0000","last_change":"2024-07-31T14:47:00.185602+0000","last_active":"2024-07-31T14:48:03.607373+0000","last_peered":"2024-07-31T14:48:03.607373+0000","last_clean":"2024-07-31T14:48:03.607373+0000","last_became_active":"2024-07-31T14:47:00.185230+0000","last_became_peered":"2024-07-31T14:47:00.185230+0000","last_unstale":"2024-07-31T14:48:03.607373+0000","last_undegraded":"2024-07-31T14:48:03.607373+0000","last_fullsized":"2024-07-31T14:48:03.607373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:21:35.979942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.604698+0000","last_change":"2024-07-31T14:47:13.099611+0000","last_active":"2024-07-31T14:47:20.604698+0000","last_peered":"2024-07-31T14:47:20.604698+0000","last_clean":"2024-07-31T14:47:20.604698+0000","last_became_active":"2024-07-31T14:47:13.099127+0000","last_became_peered":"2024-07-31T14:47:13.099127+0000","last_unstale":"2024-07-31T14:47:20.604698+0000","last_undegraded":"2024-07-31T14:47:20.604698+0000","last_fullsized":"2024-07-31T14:47:20.604698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:16:57.192980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400126+0000","last_change":"2024-07-31T14:47:03.056818+0000","last_active":"2024-07-31T14:49:49.400126+0000","last_peered":"2024-07-31T14:49:49.400126+0000","last_clean":"2024-07-31T14:49:49.400126+0000","last_became_active":"2024-07-31T14:47:03.055716+0000","last_became_peered":"2024-07-31T14:47:03.055716+0000","last_unstale":"2024-07-31T14:49:49.400126+0000","last_undegraded":"2024-07-31T14:49:49.400126+0000","last_fullsized":"2024-07-31T14:49:49.400126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:26:33.627362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.600344+0000","last_change":"2024-07-31T14:47:00.029651+0000","last_active":"2024-07-31T14:48:02.600344+0000","last_peered":"2024-07-31T14:48:02.600344+0000","last_clean":"2024-07-31T14:48:02.600344+0000","last_became_active":"2024-07-31T14:47:00.029242+0000","last_became_peered":"2024-07-31T14:47:00.029242+0000","last_unstale":"2024-07-31T14:48:02.600344+0000","last_undegraded":"2024-07-31T14:48:02.600344+0000","last_fullsized":"2024-07-31T14:48:02.600344+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:23:49.177508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.443852+0000","last_change":"2024-07-31T14:47:13.098862+0000","last_active":"2024-07-31T14:47:20.443852+0000","last_peered":"2024-07-31T14:47:20.443852+0000","last_clean":"2024-07-31T14:47:20.443852+0000","last_became_active":"2024-07-31T14:47:13.098324+0000","last_became_peered":"2024-07-31T14:47:13.098324+0000","last_unstale":"2024-07-31T14:47:20.443852+0000","last_undegraded":"2024-07-31T14:47:20.443852+0000","last_fullsized":"2024-07-31T14:47:20.443852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:50:41.260863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.412242+0000","last_change":"2024-07-31T14:47:03.049042+0000","last_active":"2024-07-31T14:49:49.412242+0000","last_peered":"2024-07-31T14:49:49.412242+0000","last_clean":"2024-07-31T14:49:49.412242+0000","last_became_active":"2024-07-31T14:47:03.048535+0000","last_became_peered":"2024-07-31T14:47:03.048535+0000","last_unstale":"2024-07-31T14:49:49.412242+0000","last_undegraded":"2024-07-31T14:49:49.412242+0000","last_fullsized":"2024-07-31T14:49:49.412242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:12:15.147454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":462,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:29.907102+0000","last_change":"2024-07-31T14:47:00.028940+0000","last_active":"2024-07-31T14:51:29.907102+0000","last_peered":"2024-07-31T14:51:29.907102+0000","last_clean":"2024-07-31T14:51:29.907102+0000","last_became_active":"2024-07-31T14:47:00.028472+0000","last_became_peered":"2024-07-31T14:47:00.028472+0000","last_unstale":"2024-07-31T14:51:29.907102+0000","last_undegraded":"2024-07-31T14:51:29.907102+0000","last_fullsized":"2024-07-31T14:51:29.907102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:57:15.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":414,"num_read_kb":414,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.626394+0000","last_change":"2024-07-31T14:47:13.101856+0000","last_active":"2024-07-31T14:47:20.626394+0000","last_peered":"2024-07-31T14:47:20.626394+0000","last_clean":"2024-07-31T14:47:20.626394+0000","last_became_active":"2024-07-31T14:47:13.101346+0000","last_became_peered":"2024-07-31T14:47:13.101346+0000","last_unstale":"2024-07-31T14:47:20.626394+0000","last_undegraded":"2024-07-31T14:47:20.626394+0000","last_fullsized":"2024-07-31T14:47:20.626394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:56:37.071660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:16.965978+0000","last_change":"2024-07-31T14:47:13.105952+0000","last_active":"2024-07-31T14:49:16.965978+0000","last_peered":"2024-07-31T14:49:16.965978+0000","last_clean":"2024-07-31T14:49:16.965978+0000","last_became_active":"2024-07-31T14:47:13.105506+0000","last_became_peered":"2024-07-31T14:47:13.105506+0000","last_unstale":"2024-07-31T14:49:16.965978+0000","last_undegraded":"2024-07-31T14:49:16.965978+0000","last_fullsized":"2024-07-31T14:49:16.965978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:28:33.866216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400532+0000","last_change":"2024-07-31T14:47:03.051152+0000","last_active":"2024-07-31T14:49:49.400532+0000","last_peered":"2024-07-31T14:49:49.400532+0000","last_clean":"2024-07-31T14:49:49.400532+0000","last_became_active":"2024-07-31T14:47:03.050775+0000","last_became_peered":"2024-07-31T14:47:03.050775+0000","last_unstale":"2024-07-31T14:49:49.400532+0000","last_undegraded":"2024-07-31T14:49:49.400532+0000","last_fullsized":"2024-07-31T14:49:49.400532+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:47:35.913641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:03.609201+0000","last_change":"2024-07-31T14:47:00.028552+0000","last_active":"2024-07-31T14:48:03.609201+0000","last_peered":"2024-07-31T14:48:03.609201+0000","last_clean":"2024-07-31T14:48:03.609201+0000","last_became_active":"2024-07-31T14:47:00.028143+0000","last_became_peered":"2024-07-31T14:47:00.028143+0000","last_unstale":"2024-07-31T14:48:03.609201+0000","last_undegraded":"2024-07-31T14:48:03.609201+0000","last_fullsized":"2024-07-31T14:48:03.609201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:38:09.601635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:19.188903+0000","last_change":"2024-07-31T14:47:20.606285+0000","last_active":"2024-07-31T14:49:19.188903+0000","last_peered":"2024-07-31T14:49:19.188903+0000","last_clean":"2024-07-31T14:49:19.188903+0000","last_became_active":"2024-07-31T14:47:20.605837+0000","last_became_peered":"2024-07-31T14:47:20.605837+0000","last_unstale":"2024-07-31T14:49:19.188903+0000","last_undegraded":"2024-07-31T14:49:19.188903+0000","last_fullsized":"2024-07-31T14:49:19.188903+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:16:35.409790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.398709+0000","last_change":"2024-07-31T14:47:03.047442+0000","last_active":"2024-07-31T14:49:49.398709+0000","last_peered":"2024-07-31T14:49:49.398709+0000","last_clean":"2024-07-31T14:49:49.398709+0000","last_became_active":"2024-07-31T14:47:03.047072+0000","last_became_peered":"2024-07-31T14:47:03.047072+0000","last_unstale":"2024-07-31T14:49:49.398709+0000","last_undegraded":"2024-07-31T14:49:49.398709+0000","last_fullsized":"2024-07-31T14:49:49.398709+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:14:56.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:02.599595+0000","last_change":"2024-07-31T14:47:00.027746+0000","last_active":"2024-07-31T14:48:02.599595+0000","last_peered":"2024-07-31T14:48:02.599595+0000","last_clean":"2024-07-31T14:48:02.599595+0000","last_became_active":"2024-07-31T14:47:00.027404+0000","last_became_peered":"2024-07-31T14:47:00.027404+0000","last_unstale":"2024-07-31T14:48:02.599595+0000","last_undegraded":"2024-07-31T14:48:02.599595+0000","last_fullsized":"2024-07-31T14:48:02.599595+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:30:21.011674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:48:01.589640+0000","last_change":"2024-07-31T14:47:00.029148+0000","last_active":"2024-07-31T14:48:01.589640+0000","last_peered":"2024-07-31T14:48:01.589640+0000","last_clean":"2024-07-31T14:48:01.589640+0000","last_became_active":"2024-07-31T14:47:00.028718+0000","last_became_peered":"2024-07-31T14:47:00.028718+0000","last_unstale":"2024-07-31T14:48:01.589640+0000","last_undegraded":"2024-07-31T14:48:01.589640+0000","last_fullsized":"2024-07-31T14:48:01.589640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:25:16.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400207+0000","last_change":"2024-07-31T14:47:20.629098+0000","last_active":"2024-07-31T14:49:49.400207+0000","last_peered":"2024-07-31T14:49:49.400207+0000","last_clean":"2024-07-31T14:49:49.400207+0000","last_became_active":"2024-07-31T14:47:20.627775+0000","last_became_peered":"2024-07-31T14:47:20.627775+0000","last_unstale":"2024-07-31T14:49:49.400207+0000","last_undegraded":"2024-07-31T14:49:49.400207+0000","last_fullsized":"2024-07-31T14:49:49.400207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:22:03.606509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.447331+0000","last_change":"2024-07-31T14:47:13.097892+0000","last_active":"2024-07-31T14:47:20.447331+0000","last_peered":"2024-07-31T14:47:20.447331+0000","last_clean":"2024-07-31T14:47:20.447331+0000","last_became_active":"2024-07-31T14:47:13.097419+0000","last_became_peered":"2024-07-31T14:47:13.097419+0000","last_unstale":"2024-07-31T14:47:20.447331+0000","last_undegraded":"2024-07-31T14:47:20.447331+0000","last_fullsized":"2024-07-31T14:47:20.447331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:07:13.456147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":258,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:51:26.903505+0000","last_change":"2024-07-31T14:47:00.019792+0000","last_active":"2024-07-31T14:51:26.903505+0000","last_peered":"2024-07-31T14:51:26.903505+0000","last_clean":"2024-07-31T14:51:26.903505+0000","last_became_active":"2024-07-31T14:47:00.019411+0000","last_became_peered":"2024-07-31T14:47:00.019411+0000","last_unstale":"2024-07-31T14:51:26.903505+0000","last_undegraded":"2024-07-31T14:51:26.903505+0000","last_fullsized":"2024-07-31T14:51:26.903505+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:46:58.991532+0000","last_clean_scrub_stamp":"2024-07-31T14:46:58.991532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:55:02.148823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":209,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.390707+0000","last_change":"2024-07-31T14:47:03.039808+0000","last_active":"2024-07-31T14:49:49.390707+0000","last_peered":"2024-07-31T14:49:49.390707+0000","last_clean":"2024-07-31T14:49:49.390707+0000","last_became_active":"2024-07-31T14:47:03.039382+0000","last_became_peered":"2024-07-31T14:47:03.039382+0000","last_unstale":"2024-07-31T14:49:49.390707+0000","last_undegraded":"2024-07-31T14:49:49.390707+0000","last_fullsized":"2024-07-31T14:49:49.390707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:23:09.751293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:47:20.446939+0000","last_change":"2024-07-31T14:47:13.102797+0000","last_active":"2024-07-31T14:47:20.446939+0000","last_peered":"2024-07-31T14:47:20.446939+0000","last_clean":"2024-07-31T14:47:20.446939+0000","last_became_active":"2024-07-31T14:47:13.102320+0000","last_became_peered":"2024-07-31T14:47:13.102320+0000","last_unstale":"2024-07-31T14:47:20.446939+0000","last_undegraded":"2024-07-31T14:47:20.446939+0000","last_fullsized":"2024-07-31T14:47:20.446939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:12.074949+0000","last_clean_scrub_stamp":"2024-07-31T14:47:12.074949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:10:23.066357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.397081+0000","last_change":"2024-07-31T14:47:03.054875+0000","last_active":"2024-07-31T14:49:49.397081+0000","last_peered":"2024-07-31T14:49:49.397081+0000","last_clean":"2024-07-31T14:49:49.397081+0000","last_became_active":"2024-07-31T14:47:03.054529+0000","last_became_peered":"2024-07-31T14:47:03.054529+0000","last_unstale":"2024-07-31T14:49:49.397081+0000","last_undegraded":"2024-07-31T14:49:49.397081+0000","last_fullsized":"2024-07-31T14:49:49.397081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:32:09.802933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396421+0000","last_change":"2024-07-31T14:47:03.043283+0000","last_active":"2024-07-31T14:49:49.396421+0000","last_peered":"2024-07-31T14:49:49.396421+0000","last_clean":"2024-07-31T14:49:49.396421+0000","last_became_active":"2024-07-31T14:47:03.042851+0000","last_became_peered":"2024-07-31T14:47:03.042851+0000","last_unstale":"2024-07-31T14:49:49.396421+0000","last_undegraded":"2024-07-31T14:49:49.396421+0000","last_fullsized":"2024-07-31T14:49:49.396421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:25:37.408149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409526+0000","last_change":"2024-07-31T14:47:03.047890+0000","last_active":"2024-07-31T14:49:49.409526+0000","last_peered":"2024-07-31T14:49:49.409526+0000","last_clean":"2024-07-31T14:49:49.409526+0000","last_became_active":"2024-07-31T14:47:03.047507+0000","last_became_peered":"2024-07-31T14:47:03.047507+0000","last_unstale":"2024-07-31T14:49:49.409526+0000","last_undegraded":"2024-07-31T14:49:49.409526+0000","last_fullsized":"2024-07-31T14:49:49.409526+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:52:45.991323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403192+0000","last_change":"2024-07-31T14:47:03.047903+0000","last_active":"2024-07-31T14:49:49.403192+0000","last_peered":"2024-07-31T14:49:49.403192+0000","last_clean":"2024-07-31T14:49:49.403192+0000","last_became_active":"2024-07-31T14:47:03.047352+0000","last_became_peered":"2024-07-31T14:47:03.047352+0000","last_unstale":"2024-07-31T14:49:49.403192+0000","last_undegraded":"2024-07-31T14:49:49.403192+0000","last_fullsized":"2024-07-31T14:49:49.403192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:35:06.536023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403277+0000","last_change":"2024-07-31T14:47:03.044199+0000","last_active":"2024-07-31T14:49:49.403277+0000","last_peered":"2024-07-31T14:49:49.403277+0000","last_clean":"2024-07-31T14:49:49.403277+0000","last_became_active":"2024-07-31T14:47:03.043849+0000","last_became_peered":"2024-07-31T14:47:03.043849+0000","last_unstale":"2024-07-31T14:49:49.403277+0000","last_undegraded":"2024-07-31T14:49:49.403277+0000","last_fullsized":"2024-07-31T14:49:49.403277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:33:50.474895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.391219+0000","last_change":"2024-07-31T14:47:03.060654+0000","last_active":"2024-07-31T14:49:49.391219+0000","last_peered":"2024-07-31T14:49:49.391219+0000","last_clean":"2024-07-31T14:49:49.391219+0000","last_became_active":"2024-07-31T14:47:03.060157+0000","last_became_peered":"2024-07-31T14:47:03.060157+0000","last_unstale":"2024-07-31T14:49:49.391219+0000","last_undegraded":"2024-07-31T14:49:49.391219+0000","last_fullsized":"2024-07-31T14:49:49.391219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:39:50.130831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.414248+0000","last_change":"2024-07-31T14:47:03.049910+0000","last_active":"2024-07-31T14:49:49.414248+0000","last_peered":"2024-07-31T14:49:49.414248+0000","last_clean":"2024-07-31T14:49:49.414248+0000","last_became_active":"2024-07-31T14:47:03.049491+0000","last_became_peered":"2024-07-31T14:47:03.049491+0000","last_unstale":"2024-07-31T14:49:49.414248+0000","last_undegraded":"2024-07-31T14:49:49.414248+0000","last_fullsized":"2024-07-31T14:49:49.414248+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T15:18:43.319695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.410414+0000","last_change":"2024-07-31T14:47:03.052206+0000","last_active":"2024-07-31T14:49:49.410414+0000","last_peered":"2024-07-31T14:49:49.410414+0000","last_clean":"2024-07-31T14:49:49.410414+0000","last_became_active":"2024-07-31T14:47:03.051175+0000","last_became_peered":"2024-07-31T14:47:03.051175+0000","last_unstale":"2024-07-31T14:49:49.410414+0000","last_undegraded":"2024-07-31T14:49:49.410414+0000","last_fullsized":"2024-07-31T14:49:49.410414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:51:11.583604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.405714+0000","last_change":"2024-07-31T14:47:03.052162+0000","last_active":"2024-07-31T14:49:49.405714+0000","last_peered":"2024-07-31T14:49:49.405714+0000","last_clean":"2024-07-31T14:49:49.405714+0000","last_became_active":"2024-07-31T14:47:03.051634+0000","last_became_peered":"2024-07-31T14:47:03.051634+0000","last_unstale":"2024-07-31T14:49:49.405714+0000","last_undegraded":"2024-07-31T14:49:49.405714+0000","last_fullsized":"2024-07-31T14:49:49.405714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:41:59.605192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409010+0000","last_change":"2024-07-31T14:47:03.047475+0000","last_active":"2024-07-31T14:49:49.409010+0000","last_peered":"2024-07-31T14:49:49.409010+0000","last_clean":"2024-07-31T14:49:49.409010+0000","last_became_active":"2024-07-31T14:47:03.047100+0000","last_became_peered":"2024-07-31T14:47:03.047100+0000","last_unstale":"2024-07-31T14:49:49.409010+0000","last_undegraded":"2024-07-31T14:49:49.409010+0000","last_fullsized":"2024-07-31T14:49:49.409010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:08:10.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411379+0000","last_change":"2024-07-31T14:47:03.051281+0000","last_active":"2024-07-31T14:49:49.411379+0000","last_peered":"2024-07-31T14:49:49.411379+0000","last_clean":"2024-07-31T14:49:49.411379+0000","last_became_active":"2024-07-31T14:47:03.050921+0000","last_became_peered":"2024-07-31T14:47:03.050921+0000","last_unstale":"2024-07-31T14:49:49.411379+0000","last_undegraded":"2024-07-31T14:49:49.411379+0000","last_fullsized":"2024-07-31T14:49:49.411379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T20:24:29.221723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403114+0000","last_change":"2024-07-31T14:47:03.045457+0000","last_active":"2024-07-31T14:49:49.403114+0000","last_peered":"2024-07-31T14:49:49.403114+0000","last_clean":"2024-07-31T14:49:49.403114+0000","last_became_active":"2024-07-31T14:47:03.044466+0000","last_became_peered":"2024-07-31T14:47:03.044466+0000","last_unstale":"2024-07-31T14:49:49.403114+0000","last_undegraded":"2024-07-31T14:49:49.403114+0000","last_fullsized":"2024-07-31T14:49:49.403114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:44:01.197416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.403872+0000","last_change":"2024-07-31T14:47:03.053431+0000","last_active":"2024-07-31T14:49:49.403872+0000","last_peered":"2024-07-31T14:49:49.403872+0000","last_clean":"2024-07-31T14:49:49.403872+0000","last_became_active":"2024-07-31T14:47:03.052958+0000","last_became_peered":"2024-07-31T14:47:03.052958+0000","last_unstale":"2024-07-31T14:49:49.403872+0000","last_undegraded":"2024-07-31T14:49:49.403872+0000","last_fullsized":"2024-07-31T14:49:49.403872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:03:28.863589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.409420+0000","last_change":"2024-07-31T14:47:03.046641+0000","last_active":"2024-07-31T14:49:49.409420+0000","last_peered":"2024-07-31T14:49:49.409420+0000","last_clean":"2024-07-31T14:49:49.409420+0000","last_became_active":"2024-07-31T14:47:03.046265+0000","last_became_peered":"2024-07-31T14:47:03.046265+0000","last_unstale":"2024-07-31T14:49:49.409420+0000","last_undegraded":"2024-07-31T14:49:49.409420+0000","last_fullsized":"2024-07-31T14:49:49.409420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:20:25.389010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.387120+0000","last_change":"2024-07-31T14:47:03.051657+0000","last_active":"2024-07-31T14:49:49.387120+0000","last_peered":"2024-07-31T14:49:49.387120+0000","last_clean":"2024-07-31T14:49:49.387120+0000","last_became_active":"2024-07-31T14:47:03.050722+0000","last_became_peered":"2024-07-31T14:47:03.050722+0000","last_unstale":"2024-07-31T14:49:49.387120+0000","last_undegraded":"2024-07-31T14:49:49.387120+0000","last_fullsized":"2024-07-31T14:49:49.387120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:23:19.387470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411567+0000","last_change":"2024-07-31T14:47:03.059654+0000","last_active":"2024-07-31T14:49:49.411567+0000","last_peered":"2024-07-31T14:49:49.411567+0000","last_clean":"2024-07-31T14:49:49.411567+0000","last_became_active":"2024-07-31T14:47:03.056843+0000","last_became_peered":"2024-07-31T14:47:03.056843+0000","last_unstale":"2024-07-31T14:49:49.411567+0000","last_undegraded":"2024-07-31T14:49:49.411567+0000","last_fullsized":"2024-07-31T14:49:49.411567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:42:33.696498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413200+0000","last_change":"2024-07-31T14:47:03.052750+0000","last_active":"2024-07-31T14:49:49.413200+0000","last_peered":"2024-07-31T14:49:49.413200+0000","last_clean":"2024-07-31T14:49:49.413200+0000","last_became_active":"2024-07-31T14:47:03.051744+0000","last_became_peered":"2024-07-31T14:47:03.051744+0000","last_unstale":"2024-07-31T14:49:49.413200+0000","last_undegraded":"2024-07-31T14:49:49.413200+0000","last_fullsized":"2024-07-31T14:49:49.413200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T23:24:33.202918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.393222+0000","last_change":"2024-07-31T14:47:20.452563+0000","last_active":"2024-07-31T14:49:49.393222+0000","last_peered":"2024-07-31T14:49:49.393222+0000","last_clean":"2024-07-31T14:49:49.393222+0000","last_became_active":"2024-07-31T14:47:20.452271+0000","last_became_peered":"2024-07-31T14:47:20.452271+0000","last_unstale":"2024-07-31T14:49:49.393222+0000","last_undegraded":"2024-07-31T14:49:49.393222+0000","last_fullsized":"2024-07-31T14:49:49.393222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T16:26:55.070265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.391684+0000","last_change":"2024-07-31T14:47:03.050691+0000","last_active":"2024-07-31T14:49:49.391684+0000","last_peered":"2024-07-31T14:49:49.391684+0000","last_clean":"2024-07-31T14:49:49.391684+0000","last_became_active":"2024-07-31T14:47:03.050320+0000","last_became_peered":"2024-07-31T14:47:03.050320+0000","last_unstale":"2024-07-31T14:49:49.391684+0000","last_undegraded":"2024-07-31T14:49:49.391684+0000","last_fullsized":"2024-07-31T14:49:49.391684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:40:45.767706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.400128+0000","last_change":"2024-07-31T14:47:03.048491+0000","last_active":"2024-07-31T14:49:49.400128+0000","last_peered":"2024-07-31T14:49:49.400128+0000","last_clean":"2024-07-31T14:49:49.400128+0000","last_became_active":"2024-07-31T14:47:03.048085+0000","last_became_peered":"2024-07-31T14:47:03.048085+0000","last_unstale":"2024-07-31T14:49:49.400128+0000","last_undegraded":"2024-07-31T14:49:49.400128+0000","last_fullsized":"2024-07-31T14:49:49.400128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T22:35:01.762365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.395425+0000","last_change":"2024-07-31T14:47:03.055502+0000","last_active":"2024-07-31T14:49:49.395425+0000","last_peered":"2024-07-31T14:49:49.395425+0000","last_clean":"2024-07-31T14:49:49.395425+0000","last_became_active":"2024-07-31T14:47:03.055015+0000","last_became_peered":"2024-07-31T14:47:03.055015+0000","last_unstale":"2024-07-31T14:49:49.395425+0000","last_undegraded":"2024-07-31T14:49:49.395425+0000","last_fullsized":"2024-07-31T14:49:49.395425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:50:39.605148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413933+0000","last_change":"2024-07-31T14:47:03.057808+0000","last_active":"2024-07-31T14:49:49.413933+0000","last_peered":"2024-07-31T14:49:49.413933+0000","last_clean":"2024-07-31T14:49:49.413933+0000","last_became_active":"2024-07-31T14:47:03.057443+0000","last_became_peered":"2024-07-31T14:47:03.057443+0000","last_unstale":"2024-07-31T14:49:49.413933+0000","last_undegraded":"2024-07-31T14:49:49.413933+0000","last_fullsized":"2024-07-31T14:49:49.413933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T02:34:42.915428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396116+0000","last_change":"2024-07-31T14:47:03.058182+0000","last_active":"2024-07-31T14:49:49.396116+0000","last_peered":"2024-07-31T14:49:49.396116+0000","last_clean":"2024-07-31T14:49:49.396116+0000","last_became_active":"2024-07-31T14:47:03.057802+0000","last_became_peered":"2024-07-31T14:47:03.057802+0000","last_unstale":"2024-07-31T14:49:49.396116+0000","last_undegraded":"2024-07-31T14:49:49.396116+0000","last_fullsized":"2024-07-31T14:49:49.396116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:08:06.608788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.413379+0000","last_change":"2024-07-31T14:47:03.058882+0000","last_active":"2024-07-31T14:49:49.413379+0000","last_peered":"2024-07-31T14:49:49.413379+0000","last_clean":"2024-07-31T14:49:49.413379+0000","last_became_active":"2024-07-31T14:47:03.058530+0000","last_became_peered":"2024-07-31T14:47:03.058530+0000","last_unstale":"2024-07-31T14:49:49.413379+0000","last_undegraded":"2024-07-31T14:49:49.413379+0000","last_fullsized":"2024-07-31T14:49:49.413379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:28:03.631962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404760+0000","last_change":"2024-07-31T14:47:03.052028+0000","last_active":"2024-07-31T14:49:49.404760+0000","last_peered":"2024-07-31T14:49:49.404760+0000","last_clean":"2024-07-31T14:49:49.404760+0000","last_became_active":"2024-07-31T14:47:03.051559+0000","last_became_peered":"2024-07-31T14:47:03.051559+0000","last_unstale":"2024-07-31T14:49:49.404760+0000","last_undegraded":"2024-07-31T14:49:49.404760+0000","last_fullsized":"2024-07-31T14:49:49.404760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:16:40.723795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.399075+0000","last_change":"2024-07-31T14:47:03.053884+0000","last_active":"2024-07-31T14:49:49.399075+0000","last_peered":"2024-07-31T14:49:49.399075+0000","last_clean":"2024-07-31T14:49:49.399075+0000","last_became_active":"2024-07-31T14:47:03.053521+0000","last_became_peered":"2024-07-31T14:47:03.053521+0000","last_unstale":"2024-07-31T14:49:49.399075+0000","last_undegraded":"2024-07-31T14:49:49.399075+0000","last_fullsized":"2024-07-31T14:49:49.399075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T18:16:37.702055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.394011+0000","last_change":"2024-07-31T14:47:03.058588+0000","last_active":"2024-07-31T14:49:49.394011+0000","last_peered":"2024-07-31T14:49:49.394011+0000","last_clean":"2024-07-31T14:49:49.394011+0000","last_became_active":"2024-07-31T14:47:03.057820+0000","last_became_peered":"2024-07-31T14:47:03.057820+0000","last_unstale":"2024-07-31T14:49:49.394011+0000","last_undegraded":"2024-07-31T14:49:49.394011+0000","last_fullsized":"2024-07-31T14:49:49.394011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:36:21.265208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396703+0000","last_change":"2024-07-31T14:47:03.041447+0000","last_active":"2024-07-31T14:49:49.396703+0000","last_peered":"2024-07-31T14:49:49.396703+0000","last_clean":"2024-07-31T14:49:49.396703+0000","last_became_active":"2024-07-31T14:47:03.040935+0000","last_became_peered":"2024-07-31T14:47:03.040935+0000","last_unstale":"2024-07-31T14:49:49.396703+0000","last_undegraded":"2024-07-31T14:49:49.396703+0000","last_fullsized":"2024-07-31T14:49:49.396703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T21:00:13.624698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396885+0000","last_change":"2024-07-31T14:47:20.452239+0000","last_active":"2024-07-31T14:49:49.396885+0000","last_peered":"2024-07-31T14:49:49.396885+0000","last_clean":"2024-07-31T14:49:49.396885+0000","last_became_active":"2024-07-31T14:47:20.451924+0000","last_became_peered":"2024-07-31T14:47:20.451924+0000","last_unstale":"2024-07-31T14:49:49.396885+0000","last_undegraded":"2024-07-31T14:49:49.396885+0000","last_fullsized":"2024-07-31T14:49:49.396885+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T19:58:24.249825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.404120+0000","last_change":"2024-07-31T14:47:03.056024+0000","last_active":"2024-07-31T14:49:49.404120+0000","last_peered":"2024-07-31T14:49:49.404120+0000","last_clean":"2024-07-31T14:49:49.404120+0000","last_became_active":"2024-07-31T14:47:03.055630+0000","last_became_peered":"2024-07-31T14:47:03.055630+0000","last_unstale":"2024-07-31T14:49:49.404120+0000","last_undegraded":"2024-07-31T14:49:49.404120+0000","last_fullsized":"2024-07-31T14:49:49.404120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T01:09:11.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.396693+0000","last_change":"2024-07-31T14:47:03.053718+0000","last_active":"2024-07-31T14:49:49.396693+0000","last_peered":"2024-07-31T14:49:49.396693+0000","last_clean":"2024-07-31T14:49:49.396693+0000","last_became_active":"2024-07-31T14:47:03.053361+0000","last_became_peered":"2024-07-31T14:47:03.053361+0000","last_unstale":"2024-07-31T14:49:49.396693+0000","last_undegraded":"2024-07-31T14:49:49.396693+0000","last_fullsized":"2024-07-31T14:49:49.396693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T00:03:49.220838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-07-31T14:49:49.411585+0000","last_change":"2024-07-31T14:47:03.056087+0000","last_active":"2024-07-31T14:49:49.411585+0000","last_peered":"2024-07-31T14:49:49.411585+0000","last_clean":"2024-07-31T14:49:49.411585+0000","last_became_active":"2024-07-31T14:47:03.055732+0000","last_became_peered":"2024-07-31T14:47:03.055732+0000","last_unstale":"2024-07-31T14:49:49.411585+0000","last_undegraded":"2024-07-31T14:49:49.411585+0000","last_fullsized":"2024-07-31T14:49:49.411585+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-31T14:47:02.015845+0000","last_clean_scrub_stamp":"2024-07-31T14:47:02.015845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-01T17:45:01.185666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":868018,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2441,"num_read_kb":1446,"num_write":501,"num_write_kb":2260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2973696,"data_stored":2604054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":380,"ondisk_log_size":380,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645380,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":520429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841609,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37472,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707680,"statfs":{"total":95995035648,"available":95956664320,"internally_reserved":0,"allocated":2125824,"data_stored":1573163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005134,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36748,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708404,"statfs":{"total":95995035648,"available":95957405696,"internally_reserved":0,"allocated":1445888,"data_stored":916972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201363,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1437696,"data_stored":913254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397591,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1118131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561117,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":515696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757346,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29076,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716076,"statfs":{"total":95995035648,"available":95965261824,"internally_reserved":0,"allocated":2048000,"data_stored":1506034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920871,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":520161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117099,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28860,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716292,"statfs":{"total":95995035648,"available":95965483008,"internally_reserved":0,"allocated":1826816,"data_stored":1301864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378736,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":589563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542261,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":588644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705785,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":517319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":819200,"data_stored":787422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":397128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":524288,"data_stored":468353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-31T14:51:34.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:51:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:34.679 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:34 smithi067 ceph-mon[32750]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:34.680 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:34 smithi067 ceph-mon[32750]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-07-31T14:51:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:34 smithi077 ceph-mon[37327]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:34 smithi077 ceph-mon[37327]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-07-31T14:51:35.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:34 smithi073 ceph-mon[36714]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-31T14:51:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:34 smithi073 ceph-mon[36714]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-07-31T14:51:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.839+0000 7f3144e9a640 1 -- 172.21.15.67:0/2441401695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31401023f0 msgr2=0x7f3140102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.839+0000 7f3144e9a640 1 --2- 172.21.15.67:0/2441401695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31401023f0 0x7f3140102800 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3130009900 tx=0x7f313002a410 comp rx=0 tx=0).stop 2024-07-31T14:51:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.840+0000 7f3144e9a640 1 -- 172.21.15.67:0/2441401695 shutdown_connections 2024-07-31T14:51:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.840+0000 7f3144e9a640 1 --2- 172.21.15.67:0/2441401695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3140108670 0x7f314010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.840+0000 7f3144e9a640 1 --2- 172.21.15.67:0/2441401695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3140107d20 0x7f3140108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.840+0000 7f3144e9a640 1 --2- 172.21.15.67:0/2441401695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31401023f0 0x7f3140102800 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.840+0000 7f3144e9a640 1 -- 172.21.15.67:0/2441401695 >> 172.21.15.67:0/2441401695 conn(0x7f31400fddf0 msgr2=0x7f3140100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.840+0000 7f3144e9a640 1 -- 172.21.15.67:0/2441401695 shutdown_connections 2024-07-31T14:51:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.840+0000 7f3144e9a640 1 -- 172.21.15.67:0/2441401695 wait complete. 2024-07-31T14:51:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.840+0000 7f3144e9a640 1 Processor -- start 2024-07-31T14:51:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f3144e9a640 1 -- start start 2024-07-31T14:51:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f3144e9a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31401023f0 0x7f314019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f3144e9a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3140107d20 0x7f314019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f3144e9a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3140108670 0x7f314019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f3144e9a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3140112340 con 0x7f3140108670 2024-07-31T14:51:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f3144e9a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f31401121c0 con 0x7f3140107d20 2024-07-31T14:51:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f3144e9a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31401124c0 con 0x7f31401023f0 2024-07-31T14:51:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f313ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3140108670 0x7f314019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f313ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3140108670 0x7f314019b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44132/0 (socket says 172.21.15.67:44132) 2024-07-31T14:51:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f313ed9d640 1 -- 172.21.15.67:0/1770263781 learned_addr learned my addr 172.21.15.67:0/1770263781 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f313dd9b640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3140107d20 0x7f314019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f313e59c640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31401023f0 0x7f314019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f313dd9b640 1 -- 172.21.15.67:0/1770263781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31401023f0 msgr2=0x7f314019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f313dd9b640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31401023f0 0x7f314019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f313dd9b640 1 -- 172.21.15.67:0/1770263781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3140108670 msgr2=0x7f314019b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f313dd9b640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3140108670 0x7f314019b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f313dd9b640 1 -- 172.21.15.67:0/1770263781 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31401a5fa0 con 0x7f3140107d20 2024-07-31T14:51:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.841+0000 7f313ed9d640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3140108670 0x7f314019b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.842+0000 7f313dd9b640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3140107d20 0x7f314019b250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f314006c390 tx=0x7f312800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.842+0000 7f31377fe640 1 -- 172.21.15.67:0/1770263781 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128010430 con 0x7f3140107d20 2024-07-31T14:51:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.842+0000 7f3144e9a640 1 -- 172.21.15.67:0/1770263781 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31401a6290 con 0x7f3140107d20 2024-07-31T14:51:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.842+0000 7f31377fe640 1 -- 172.21.15.67:0/1770263781 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3128010a80 con 0x7f3140107d20 2024-07-31T14:51:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.842+0000 7f31377fe640 1 -- 172.21.15.67:0/1770263781 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128017a20 con 0x7f3140107d20 2024-07-31T14:51:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.842+0000 7f3144e9a640 1 -- 172.21.15.67:0/1770263781 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31401a6830 con 0x7f3140107d20 2024-07-31T14:51:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.844+0000 7f314bd8b740 1 Processor -- start 2024-07-31T14:51:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f314bd8b740 1 -- start start 2024-07-31T14:51:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f314bd8b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30adf71b0 0x55c30b199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f314bd8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adce2f0 0x55c30b19a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f314bd8b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30add64d0 0x55c30b19a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f314bd8b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c30b10f3b0 con 0x55c30add64d0 2024-07-31T14:51:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f314bd8b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c30b0fe8b0 con 0x55c30adf71b0 2024-07-31T14:51:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f314bd8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c30b1028f0 con 0x55c30adce2f0 2024-07-31T14:51:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f31377fe640 1 -- 172.21.15.67:0/1770263781 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f3128017c60 con 0x7f3140107d20 2024-07-31T14:51:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f313e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adce2f0 0x55c30b19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f313ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30adf71b0 0x55c30b199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f313e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adce2f0 0x55c30b19a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60330/0 (socket says 172.21.15.67:60330) 2024-07-31T14:51:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f313e59c640 1 -- 172.21.15.67:0/3971826370 learned_addr learned my addr 172.21.15.67:0/3971826370 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f313dd9b640 1 --2- 172.21.15.67:0/3971826370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30add64d0 0x55c30b19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f313e59c640 1 -- 172.21.15.67:0/3971826370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30adf71b0 msgr2=0x55c30b199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f313e59c640 1 --2- 172.21.15.67:0/3971826370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30adf71b0 0x55c30b199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f313e59c640 1 -- 172.21.15.67:0/3971826370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30add64d0 msgr2=0x55c30b19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f313e59c640 1 --2- 172.21.15.67:0/3971826370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30add64d0 0x55c30b19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f313e59c640 1 -- 172.21.15.67:0/3971826370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c30b0a59a0 con 0x55c30adce2f0 2024-07-31T14:51:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.845+0000 7f313dd9b640 1 --2- 172.21.15.67:0/3971826370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30add64d0 0x55c30b19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.846+0000 7f313e59c640 1 --2- 172.21.15.67:0/3971826370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adce2f0 0x55c30b19a220 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3130033000 tx=0x7f3130032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a2f8445e26e77ef server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.846+0000 7f3134ff9640 1 -- 172.21.15.67:0/3971826370 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130035040 con 0x55c30adce2f0 2024-07-31T14:51:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.846+0000 7f31377fe640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f312c0b3d40 0x7f312c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.846+0000 7f3134ff9640 1 -- 172.21.15.67:0/3971826370 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3130004030 con 0x55c30adce2f0 2024-07-31T14:51:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.846+0000 7f3134ff9640 1 -- 172.21.15.67:0/3971826370 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130005490 con 0x55c30adce2f0 2024-07-31T14:51:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.846+0000 7f313ed9d640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f312c0b3d40 0x7f312c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.846+0000 7f31377fe640 1 -- 172.21.15.67:0/1770263781 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f31280e4cd0 con 0x7f3140107d20 2024-07-31T14:51:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.847+0000 7f313ed9d640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f312c0b3d40 0x7f312c0b6250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3138009640 tx=0x7f3138005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.847+0000 7f314bd8b740 1 -- 172.21.15.67:0/3971826370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adce2f0 msgr2=0x55c30b19a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.847+0000 7f314bd8b740 1 --2- 172.21.15.67:0/3971826370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adce2f0 0x55c30b19a220 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3130033000 tx=0x7f3130032b20 comp rx=0 tx=0).stop 2024-07-31T14:51:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.847+0000 7f314bd8b740 1 -- 172.21.15.67:0/3971826370 shutdown_connections 2024-07-31T14:51:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.847+0000 7f314bd8b740 1 --2- 172.21.15.67:0/3971826370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30add64d0 0x55c30b19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.847+0000 7f314bd8b740 1 --2- 172.21.15.67:0/3971826370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adce2f0 0x55c30b19a220 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.847+0000 7f314bd8b740 1 --2- 172.21.15.67:0/3971826370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30adf71b0 0x55c30b199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.847+0000 7f314bd8b740 1 -- 172.21.15.67:0/3971826370 >> 172.21.15.67:0/3971826370 conn(0x55c30adeb300 msgr2=0x55c30b197bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.847+0000 7f314bd8b740 1 -- 172.21.15.67:0/3971826370 shutdown_connections 2024-07-31T14:51:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.847+0000 7f314bd8b740 1 -- 172.21.15.67:0/3971826370 wait complete. 2024-07-31T14:51:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.847+0000 7f314bd8b740 1 -- start start 2024-07-31T14:51:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f314bd8b740 1 Processor -- start 2024-07-31T14:51:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f314bd8b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adce2f0 0x55c30b1a5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f314bd8b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30add64d0 0x55c30b1a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f314bd8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adf71b0 0x55c30b1b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f314bd8b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c30b038ab0 con 0x55c30adce2f0 2024-07-31T14:51:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f314bd8b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c30b086cf0 con 0x55c30add64d0 2024-07-31T14:51:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f314bd8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c30b1a8400 con 0x55c30adf71b0 2024-07-31T14:51:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f314bd8b740 10 client.0 _ll_register_callbacks cb 0x55c30b0f89f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f313e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adce2f0 0x55c30b1a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f313e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adce2f0 0x55c30b1a5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44150/0 (socket says 172.21.15.67:44150) 2024-07-31T14:51:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f313e59c640 1 -- 172.21.15.67:0/4212797438 learned_addr learned my addr 172.21.15.67:0/4212797438 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f313ed9d640 1 --2- 172.21.15.67:0/4212797438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adf71b0 0x55c30b1b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f313dd9b640 1 --2- 172.21.15.67:0/4212797438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30add64d0 0x55c30b1a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f313ed9d640 1 -- 172.21.15.67:0/4212797438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30add64d0 msgr2=0x55c30b1a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f313ed9d640 1 --2- 172.21.15.67:0/4212797438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30add64d0 0x55c30b1a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f313ed9d640 1 -- 172.21.15.67:0/4212797438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adce2f0 msgr2=0x55c30b1a5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f313ed9d640 1 --2- 172.21.15.67:0/4212797438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adce2f0 0x55c30b1a5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f313ed9d640 1 -- 172.21.15.67:0/4212797438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c30ae62790 con 0x55c30adf71b0 2024-07-31T14:51:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.848+0000 7f313e59c640 1 --2- 172.21.15.67:0/4212797438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adce2f0 0x55c30b1a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.849+0000 7f313ed9d640 1 --2- 172.21.15.67:0/4212797438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adf71b0 0x55c30b1b1750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3138012f00 tx=0x7f3138004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c40a1811d53f4322 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.849+0000 7f3125ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:51:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.849+0000 7f3125ffb640 1 -- 172.21.15.67:0/4212797438 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138011850 con 0x55c30adf71b0 2024-07-31T14:51:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.849+0000 7f3125ffb640 1 -- 172.21.15.67:0/4212797438 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f31380119f0 con 0x55c30adf71b0 2024-07-31T14:51:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.849+0000 7f314bd8b740 1 -- 172.21.15.67:0/4212797438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3130004030 con 0x55c30adf71b0 2024-07-31T14:51:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.849+0000 7f3125ffb640 1 -- 172.21.15.67:0/4212797438 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3138011bf0 con 0x55c30adf71b0 2024-07-31T14:51:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.849+0000 7f314bd8b740 10 client.25390 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.849+0000 7f314bd8b740 1 -- 172.21.15.67:0/4212797438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c30aedb620 con 0x55c30adf71b0 2024-07-31T14:51:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.849+0000 7f3125ffb640 1 -- 172.21.15.67:0/4212797438 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3130004030 con 0x55c30adf71b0 2024-07-31T14:51:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.849+0000 7f3125ffb640 1 -- 172.21.15.67:0/4212797438 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3138027070 con 0x55c30adf71b0 2024-07-31T14:51:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.849+0000 7f314bd8b740 10 client.25390 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.849+0000 7f314bd8b740 10 client.25390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:51:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.849+0000 7f314bd8b740 20 client.25390 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.850+0000 7f314bd8b740 1 --2- 172.21.15.67:0/4212797438 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c30adb5d80 0x55c30b1b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.850+0000 7f314bd8b740 4 client.25390 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:51:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.850+0000 7f314bd8b740 1 -- 172.21.15.67:0/4212797438 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c30b111300 con 0x55c30adb5d80 2024-07-31T14:51:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.850+0000 7f313e59c640 1 --2- 172.21.15.67:0/4212797438 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c30adb5d80 0x55c30b1b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.850+0000 7f313e59c640 1 --2- 172.21.15.67:0/4212797438 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c30adb5d80 0x55c30b1b77a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1419718cd4b41d5 server_cookie=6abc8bed561cb0e2 in_seq=0 out_seq=0 2024-07-31T14:51:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.850+0000 7f3125ffb640 10 client.25390 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f3125ffb640 1 -- 172.21.15.67:0/4212797438 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c30b111300 con 0x55c30adb5d80 2024-07-31T14:51:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f3125ffb640 10 client.25390 handle_command_reply: tid=0 2024-07-31T14:51:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f314bd8b740 1 client.25390 shutdown 2024-07-31T14:51:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f314bd8b740 1 -- 172.21.15.67:0/4212797438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adf71b0 msgr2=0x55c30b1b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f314bd8b740 1 --2- 172.21.15.67:0/4212797438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adf71b0 0x55c30b1b1750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3138012f00 tx=0x7f3138004730 comp rx=0 tx=0).stop 2024-07-31T14:51:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f314bd8b740 1 -- 172.21.15.67:0/4212797438 shutdown_connections 2024-07-31T14:51:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f314bd8b740 1 --2- 172.21.15.67:0/4212797438 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c30adb5d80 0x55c30b1b77a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f314bd8b740 1 --2- 172.21.15.67:0/4212797438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adf71b0 0x55c30b1b1750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f314bd8b740 1 --2- 172.21.15.67:0/4212797438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30add64d0 0x55c30b1a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f314bd8b740 1 --2- 172.21.15.67:0/4212797438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adce2f0 0x55c30b1a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f3125ffb640 0 client.25390 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f314bd8b740 1 -- 172.21.15.67:0/4212797438 >> 172.21.15.67:0/4212797438 conn(0x55c30adeb300 msgr2=0x55c30b19ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f314bd8b740 1 -- 172.21.15.67:0/4212797438 shutdown_connections 2024-07-31T14:51:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f314bd8b740 1 -- 172.21.15.67:0/4212797438 wait complete. 2024-07-31T14:51:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.851+0000 7f314bd8b740 20 client.25390 trim_cache size 0 max 16384 2024-07-31T14:51:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.866+0000 7f314bd8b740 1 Processor -- start 2024-07-31T14:51:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.866+0000 7f314bd8b740 1 -- start start 2024-07-31T14:51:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f314bd8b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adce2f0 0x55c30b1b1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f314bd8b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30add64d0 0x55c30b1b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f314bd8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adf71b0 0x55c30b1bcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f314bd8b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f31401b06f0 con 0x55c30adce2f0 2024-07-31T14:51:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f314bd8b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f31401b2230 con 0x55c30add64d0 2024-07-31T14:51:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f314bd8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31401b0dc0 con 0x55c30adf71b0 2024-07-31T14:51:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f313e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adce2f0 0x55c30b1b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f313e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adce2f0 0x55c30b1b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44166/0 (socket says 172.21.15.67:44166) 2024-07-31T14:51:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f313e59c640 1 -- 172.21.15.67:0/1480164481 learned_addr learned my addr 172.21.15.67:0/1480164481 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f313dd9b640 1 --2- 172.21.15.67:0/1480164481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30add64d0 0x55c30b1b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f313e59c640 1 -- 172.21.15.67:0/1480164481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adf71b0 msgr2=0x55c30b1bcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f313e59c640 1 --2- 172.21.15.67:0/1480164481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adf71b0 0x55c30b1bcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f313e59c640 1 -- 172.21.15.67:0/1480164481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30add64d0 msgr2=0x55c30b1b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f313e59c640 1 --2- 172.21.15.67:0/1480164481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30add64d0 0x55c30b1b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f313e59c640 1 -- 172.21.15.67:0/1480164481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c30b1a5300 con 0x55c30adce2f0 2024-07-31T14:51:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f313e59c640 1 --2- 172.21.15.67:0/1480164481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adce2f0 0x55c30b1b1540 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f31300050c0 tx=0x7f3130005190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb784d4ba057bdfe server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.867+0000 7f3127fff640 1 -- 172.21.15.67:0/1480164481 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130035040 con 0x55c30adce2f0 2024-07-31T14:51:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.868+0000 7f3127fff640 1 -- 172.21.15.67:0/1480164481 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3130004460 con 0x55c30adce2f0 2024-07-31T14:51:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.868+0000 7f3127fff640 1 -- 172.21.15.67:0/1480164481 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130032040 con 0x55c30adce2f0 2024-07-31T14:51:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.868+0000 7f314bd8b740 1 -- 172.21.15.67:0/1480164481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adce2f0 msgr2=0x55c30b1b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.868+0000 7f314bd8b740 1 --2- 172.21.15.67:0/1480164481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adce2f0 0x55c30b1b1540 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f31300050c0 tx=0x7f3130005190 comp rx=0 tx=0).stop 2024-07-31T14:51:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.868+0000 7f314bd8b740 1 -- 172.21.15.67:0/1480164481 shutdown_connections 2024-07-31T14:51:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.868+0000 7f314bd8b740 1 --2- 172.21.15.67:0/1480164481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30adf71b0 0x55c30b1bcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.868+0000 7f314bd8b740 1 --2- 172.21.15.67:0/1480164481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30add64d0 0x55c30b1b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.868+0000 7f314bd8b740 1 --2- 172.21.15.67:0/1480164481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adce2f0 0x55c30b1b1540 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.868+0000 7f314bd8b740 1 -- 172.21.15.67:0/1480164481 >> 172.21.15.67:0/1480164481 conn(0x55c30adeb300 msgr2=0x55c30b1a3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.868+0000 7f314bd8b740 1 -- 172.21.15.67:0/1480164481 shutdown_connections 2024-07-31T14:51:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.868+0000 7f314bd8b740 1 -- 172.21.15.67:0/1480164481 wait complete. 2024-07-31T14:51:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f314bd8b740 1 -- start start 2024-07-31T14:51:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f314bd8b740 1 Processor -- start 2024-07-31T14:51:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f314bd8b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30adce2f0 0x55c30b1aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f314bd8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30add64d0 0x55c30b1ab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f314bd8b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adf71b0 0x55c30b21df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f314bd8b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f31401b1b60 con 0x55c30adf71b0 2024-07-31T14:51:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f314bd8b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f31401b1490 con 0x55c30adce2f0 2024-07-31T14:51:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f314bd8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31401ae6d0 con 0x55c30add64d0 2024-07-31T14:51:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f314bd8b740 10 client.0 _ll_register_callbacks cb 0x7f3138009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f313ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adf71b0 0x55c30b21df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f313ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adf71b0 0x55c30b21df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44176/0 (socket says 172.21.15.67:44176) 2024-07-31T14:51:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f313ed9d640 1 -- 172.21.15.67:0/2697219275 learned_addr learned my addr 172.21.15.67:0/2697219275 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f313dd9b640 1 --2- 172.21.15.67:0/2697219275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30add64d0 0x55c30b1ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f313e59c640 1 --2- 172.21.15.67:0/2697219275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30adce2f0 0x55c30b1aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f313ed9d640 1 -- 172.21.15.67:0/2697219275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30add64d0 msgr2=0x55c30b1ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.869+0000 7f313ed9d640 1 --2- 172.21.15.67:0/2697219275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30add64d0 0x55c30b1ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f313ed9d640 1 -- 172.21.15.67:0/2697219275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30adce2f0 msgr2=0x55c30b1aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f313ed9d640 1 --2- 172.21.15.67:0/2697219275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30adce2f0 0x55c30b1aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f313ed9d640 1 -- 172.21.15.67:0/2697219275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c30b21e470 con 0x55c30adf71b0 2024-07-31T14:51:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f313e59c640 1 --2- 172.21.15.67:0/2697219275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30adce2f0 0x55c30b1aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f313ed9d640 1 --2- 172.21.15.67:0/2697219275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adf71b0 0x55c30b21df30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f313800a010 tx=0x7f3138005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d12c25e40e34cc3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f3125ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:51:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f3125ffb640 1 -- 172.21.15.67:0/2697219275 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138011e10 con 0x55c30adf71b0 2024-07-31T14:51:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f3125ffb640 1 -- 172.21.15.67:0/2697219275 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f3138004890 con 0x55c30adf71b0 2024-07-31T14:51:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f314bd8b740 1 -- 172.21.15.67:0/2697219275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3130004460 con 0x55c30adf71b0 2024-07-31T14:51:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f3125ffb640 1 -- 172.21.15.67:0/2697219275 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f313801e6a0 con 0x55c30adf71b0 2024-07-31T14:51:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f314bd8b740 10 client.15675 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f314bd8b740 1 -- 172.21.15.67:0/2697219275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c30b21f690 con 0x55c30adf71b0 2024-07-31T14:51:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f3125ffb640 1 -- 172.21.15.67:0/2697219275 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3130004460 con 0x55c30adf71b0 2024-07-31T14:51:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f3125ffb640 1 -- 172.21.15.67:0/2697219275 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3138035030 con 0x55c30adf71b0 2024-07-31T14:51:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f314bd8b740 10 client.15675 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.870+0000 7f314bd8b740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:51:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.871+0000 7f314bd8b740 20 client.15675 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.871+0000 7f314bd8b740 1 --2- 172.21.15.67:0/2697219275 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c30ad8b1c0 0x55c30b221e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.871+0000 7f314bd8b740 4 client.15675 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:51:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.871+0000 7f314bd8b740 1 -- 172.21.15.67:0/2697219275 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c30b10cda0 con 0x55c30ad8b1c0 2024-07-31T14:51:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.871+0000 7f313e59c640 1 --2- 172.21.15.67:0/2697219275 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c30ad8b1c0 0x55c30b221e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.871+0000 7f3125ffb640 10 client.15675 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.871+0000 7f313e59c640 1 --2- 172.21.15.67:0/2697219275 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c30ad8b1c0 0x55c30b221e80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c0d0770c6935817 server_cookie=bd953f65ac985524 in_seq=0 out_seq=0 2024-07-31T14:51:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.884+0000 7f3125ffb640 1 -- 172.21.15.67:0/2697219275 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c30b10cda0 con 0x55c30ad8b1c0 2024-07-31T14:51:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.884+0000 7f3125ffb640 10 client.15675 handle_command_reply: tid=0 2024-07-31T14:51:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.884+0000 7f314bd8b740 1 client.15675 shutdown 2024-07-31T14:51:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.884+0000 7f314bd8b740 1 -- 172.21.15.67:0/2697219275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adf71b0 msgr2=0x55c30b21df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.884+0000 7f314bd8b740 1 --2- 172.21.15.67:0/2697219275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adf71b0 0x55c30b21df30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f313800a010 tx=0x7f3138005e40 comp rx=0 tx=0).stop 2024-07-31T14:51:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f314bd8b740 1 -- 172.21.15.67:0/2697219275 shutdown_connections 2024-07-31T14:51:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f314bd8b740 1 --2- 172.21.15.67:0/2697219275 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c30ad8b1c0 0x55c30b221e80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f314bd8b740 1 --2- 172.21.15.67:0/2697219275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c30adf71b0 0x55c30b21df30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f314bd8b740 1 --2- 172.21.15.67:0/2697219275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c30add64d0 0x55c30b1ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f314bd8b740 1 --2- 172.21.15.67:0/2697219275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c30adce2f0 0x55c30b1aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f3125ffb640 0 client.15675 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f314bd8b740 1 -- 172.21.15.67:0/2697219275 >> 172.21.15.67:0/2697219275 conn(0x55c30adeb300 msgr2=0x55c30b1a3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f314bd8b740 1 -- 172.21.15.67:0/2697219275 shutdown_connections 2024-07-31T14:51:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f314bd8b740 1 -- 172.21.15.67:0/2697219275 wait complete. 2024-07-31T14:51:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f314bd8b740 20 client.15675 trim_cache size 0 max 16384 2024-07-31T14:51:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:51:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:51:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:51:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:51:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f3144e9a640 1 -- 172.21.15.67:0/1770263781 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f312c0b3d40 msgr2=0x7f312c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f3144e9a640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f312c0b3d40 0x7f312c0b6250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3138009640 tx=0x7f3138005f90 comp rx=0 tx=0).stop 2024-07-31T14:51:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f3144e9a640 1 -- 172.21.15.67:0/1770263781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3140107d20 msgr2=0x7f314019b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f3144e9a640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3140107d20 0x7f314019b250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f314006c390 tx=0x7f312800ef90 comp rx=0 tx=0).stop 2024-07-31T14:51:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f3144e9a640 1 -- 172.21.15.67:0/1770263781 shutdown_connections 2024-07-31T14:51:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f3144e9a640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f312c0b3d40 0x7f312c0b6250 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f3144e9a640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3140108670 0x7f314019b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f3144e9a640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3140107d20 0x7f314019b250 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f3144e9a640 1 --2- 172.21.15.67:0/1770263781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31401023f0 0x7f314019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.885+0000 7f3144e9a640 1 -- 172.21.15.67:0/1770263781 >> 172.21.15.67:0/1770263781 conn(0x7f31400fddf0 msgr2=0x7f31401073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.886+0000 7f3144e9a640 1 -- 172.21.15.67:0/1770263781 shutdown_connections 2024-07-31T14:51:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:35.886+0000 7f3144e9a640 1 -- 172.21.15.67:0/1770263781 wait complete. 2024-07-31T14:51:36.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.041+0000 7f56b29be640 1 -- 172.21.15.67:0/1049785565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56ac10d860 msgr2=0x7f56ac10fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:36.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.041+0000 7f56b29be640 1 --2- 172.21.15.67:0/1049785565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56ac10d860 0x7f56ac10fd30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009900 tx=0x7f56a002a410 comp rx=0 tx=0).stop 2024-07-31T14:51:36.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.042+0000 7f56b29be640 1 -- 172.21.15.67:0/1049785565 shutdown_connections 2024-07-31T14:51:36.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.042+0000 7f56b29be640 1 --2- 172.21.15.67:0/1049785565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56ac10d860 0x7f56ac10fd30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:36.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.042+0000 7f56b29be640 1 --2- 172.21.15.67:0/1049785565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56ac104ce0 0x7f56ac10d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:36.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.042+0000 7f56b29be640 1 --2- 172.21.15.67:0/1049785565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56ac1042b0 0x7f56ac1046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:36.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.042+0000 7f56b29be640 1 -- 172.21.15.67:0/1049785565 >> 172.21.15.67:0/1049785565 conn(0x7f56ac079160 msgr2=0x7f56ac079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:36.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.042+0000 7f56b29be640 1 -- 172.21.15.67:0/1049785565 shutdown_connections 2024-07-31T14:51:36.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.042+0000 7f56b29be640 1 -- 172.21.15.67:0/1049785565 wait complete. 2024-07-31T14:51:36.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.043+0000 7f56b29be640 1 Processor -- start 2024-07-31T14:51:36.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.043+0000 7f56b29be640 1 -- start start 2024-07-31T14:51:36.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.043+0000 7f56b29be640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56ac1042b0 0x7f56ac199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:36.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.043+0000 7f56b29be640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56ac104ce0 0x7f56ac19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:36.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.043+0000 7f56b29be640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56ac10d860 0x7f56ac19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:36.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.043+0000 7f56b29be640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac113540 con 0x7f56ac1042b0 2024-07-31T14:51:36.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.043+0000 7f56b29be640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1133c0 con 0x7f56ac104ce0 2024-07-31T14:51:36.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.043+0000 7f56b29be640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1136c0 con 0x7f56ac10d860 2024-07-31T14:51:36.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.043+0000 7f56abfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56ac1042b0 0x7f56ac199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:36.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.043+0000 7f56abfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56ac1042b0 0x7f56ac199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44188/0 (socket says 172.21.15.67:44188) 2024-07-31T14:51:36.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.043+0000 7f56abfff640 1 -- 172.21.15.67:0/546858456 learned_addr learned my addr 172.21.15.67:0/546858456 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:36.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.043+0000 7f56ab7fe640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56ac104ce0 0x7f56ac19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:36.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.044+0000 7f56b0f5b640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56ac10d860 0x7f56ac19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:36.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.044+0000 7f56abfff640 1 -- 172.21.15.67:0/546858456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56ac10d860 msgr2=0x7f56ac19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:36.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.044+0000 7f56abfff640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56ac10d860 0x7f56ac19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:36.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.044+0000 7f56abfff640 1 -- 172.21.15.67:0/546858456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56ac104ce0 msgr2=0x7f56ac19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:36.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.044+0000 7f56abfff640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56ac104ce0 0x7f56ac19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:36.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.044+0000 7f56abfff640 1 -- 172.21.15.67:0/546858456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac19fe80 con 0x7f56ac1042b0 2024-07-31T14:51:36.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.044+0000 7f56ab7fe640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56ac104ce0 0x7f56ac19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:36.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.044+0000 7f56abfff640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56ac1042b0 0x7f56ac199f10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f569c00b450 tx=0x7f569c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:36.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.044+0000 7f56a97fa640 1 -- 172.21.15.67:0/546858456 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c015020 con 0x7f56ac1042b0 2024-07-31T14:51:36.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.044+0000 7f56b29be640 1 -- 172.21.15.67:0/546858456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ac1a0170 con 0x7f56ac1042b0 2024-07-31T14:51:36.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.044+0000 7f56b29be640 1 -- 172.21.15.67:0/546858456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ac1020a0 con 0x7f56ac1042b0 2024-07-31T14:51:36.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.044+0000 7f56a97fa640 1 -- 172.21.15.67:0/546858456 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f569c011d40 con 0x7f56ac1042b0 2024-07-31T14:51:36.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.044+0000 7f56a97fa640 1 -- 172.21.15.67:0/546858456 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c01adb0 con 0x7f56ac1042b0 2024-07-31T14:51:36.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.046+0000 7f56b29be640 1 -- 172.21.15.67:0/546858456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56980052c0 con 0x7f56ac1042b0 2024-07-31T14:51:36.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.047+0000 7f56a97fa640 1 -- 172.21.15.67:0/546858456 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f569c018020 con 0x7f56ac1042b0 2024-07-31T14:51:36.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.048+0000 7f56a97fa640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56940b3dc0 0x7f56940b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:36.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.048+0000 7f56ab7fe640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56940b3dc0 0x7f56940b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:36.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.048+0000 7f56a97fa640 1 -- 172.21.15.67:0/546858456 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f569c0e3ac0 con 0x7f56ac1042b0 2024-07-31T14:51:36.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.049+0000 7f56ab7fe640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56940b3dc0 0x7f56940b62d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f56ac10f580 tx=0x7f56a4009520 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:36.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.052+0000 7f56a97fa640 1 -- 172.21.15.67:0/546858456 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f569c01f020 con 0x7f56ac1042b0 2024-07-31T14:51:36.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.285+0000 7f56b29be640 1 -- 172.21.15.67:0/546858456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5698005880 con 0x7f56ac1042b0 2024-07-31T14:51:36.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.286+0000 7f56a97fa640 1 -- 172.21.15.67:0/546858456 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11962 (secure 0 0 0) 0x7f569c00d040 con 0x7f56ac1042b0 2024-07-31T14:51:36.287 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:51:36.377 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-31T14:51:08:515297+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:51:07.734542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:51:36.378 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-31T14:51:36.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.288+0000 7f56b29be640 1 -- 172.21.15.67:0/546858456 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56940b3dc0 msgr2=0x7f56940b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:36.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.288+0000 7f56b29be640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56940b3dc0 0x7f56940b62d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f56ac10f580 tx=0x7f56a4009520 comp rx=0 tx=0).stop 2024-07-31T14:51:36.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.288+0000 7f56b29be640 1 -- 172.21.15.67:0/546858456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56ac1042b0 msgr2=0x7f56ac199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:36.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.288+0000 7f56b29be640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56ac1042b0 0x7f56ac199f10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f569c00b450 tx=0x7f569c00b910 comp rx=0 tx=0).stop 2024-07-31T14:51:36.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.288+0000 7f56b29be640 1 -- 172.21.15.67:0/546858456 shutdown_connections 2024-07-31T14:51:36.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.288+0000 7f56b29be640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56940b3dc0 0x7f56940b62d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:36.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.288+0000 7f56b29be640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56ac10d860 0x7f56ac19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:36.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.288+0000 7f56b29be640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56ac104ce0 0x7f56ac19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:36.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.288+0000 7f56b29be640 1 --2- 172.21.15.67:0/546858456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56ac1042b0 0x7f56ac199f10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:36.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.288+0000 7f56b29be640 1 -- 172.21.15.67:0/546858456 >> 172.21.15.67:0/546858456 conn(0x7f56ac079160 msgr2=0x7f56ac107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:36.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.288+0000 7f56b29be640 1 -- 172.21.15.67:0/546858456 shutdown_connections 2024-07-31T14:51:36.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:36.288+0000 7f56b29be640 1 -- 172.21.15.67:0/546858456 wait complete. 2024-07-31T14:51:36.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:36 smithi067 ceph-mon[32750]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 25 op/s 2024-07-31T14:51:36.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:36 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:51:36.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:36 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/546858456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:36.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:51:36.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:36 smithi077 ceph-mon[37327]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 25 op/s 2024-07-31T14:51:36.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:36 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:51:36.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:36 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/546858456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:37.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:36 smithi073 ceph-mon[36714]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 25 op/s 2024-07-31T14:51:37.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:36 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:51:37.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:36 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/546858456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:37.228 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:38.949 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:38 smithi067 ceph-mon[32750]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-07-31T14:51:38.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.958+0000 7f14c2b80640 1 -- 172.21.15.67:0/1361058210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14bc107de0 msgr2=0x7f14bc1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:38.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.958+0000 7f14c2b80640 1 --2- 172.21.15.67:0/1361058210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14bc107de0 0x7f14bc1081f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009ca0 tx=0x7f14ac02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:51:38.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.958+0000 7f14c2b80640 1 -- 172.21.15.67:0/1361058210 shutdown_connections 2024-07-31T14:51:38.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.958+0000 7f14c2b80640 1 --2- 172.21.15.67:0/1361058210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14bc108730 0x7f14bc10ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:38.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.958+0000 7f14c2b80640 1 --2- 172.21.15.67:0/1361058210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14bc107de0 0x7f14bc1081f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:38.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.958+0000 7f14c2b80640 1 --2- 172.21.15.67:0/1361058210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc1024a0 0x7f14bc1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:38.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.958+0000 7f14c2b80640 1 -- 172.21.15.67:0/1361058210 >> 172.21.15.67:0/1361058210 conn(0x7f14bc079140 msgr2=0x7f14bc079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:38.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.959+0000 7f14c2b80640 1 -- 172.21.15.67:0/1361058210 shutdown_connections 2024-07-31T14:51:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.959+0000 7f14c2b80640 1 -- 172.21.15.67:0/1361058210 wait complete. 2024-07-31T14:51:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.959+0000 7f14c2b80640 1 Processor -- start 2024-07-31T14:51:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.959+0000 7f14c2b80640 1 -- start start 2024-07-31T14:51:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14c2b80640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14bc1024a0 0x7f14bc199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14c2b80640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14bc107de0 0x7f14bc19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14c2b80640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc108730 0x7f14bc19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14c2b80640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc113510 con 0x7f14bc107de0 2024-07-31T14:51:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14c2b80640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc113390 con 0x7f14bc108730 2024-07-31T14:51:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14c2b80640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc113690 con 0x7f14bc1024a0 2024-07-31T14:51:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14bbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14bc107de0 0x7f14bc19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14bbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14bc107de0 0x7f14bc19a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44212/0 (socket says 172.21.15.67:44212) 2024-07-31T14:51:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14bbfff640 1 -- 172.21.15.67:0/1583107714 learned_addr learned my addr 172.21.15.67:0/1583107714 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14c111d640 1 --2- 172.21.15.67:0/1583107714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc108730 0x7f14bc19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14c091c640 1 --2- 172.21.15.67:0/1583107714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14bc1024a0 0x7f14bc199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14bbfff640 1 -- 172.21.15.67:0/1583107714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14bc1024a0 msgr2=0x7f14bc199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14bbfff640 1 --2- 172.21.15.67:0/1583107714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14bc1024a0 0x7f14bc199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14bbfff640 1 -- 172.21.15.67:0/1583107714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc108730 msgr2=0x7f14bc19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14bbfff640 1 --2- 172.21.15.67:0/1583107714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc108730 0x7f14bc19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14bbfff640 1 -- 172.21.15.67:0/1583107714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14bc19fe90 con 0x7f14bc107de0 2024-07-31T14:51:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.960+0000 7f14bbfff640 1 --2- 172.21.15.67:0/1583107714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14bc107de0 0x7f14bc19a580 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0319f0 tx=0x7f14ac031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.961+0000 7f14b9ffb640 1 -- 172.21.15.67:0/1583107714 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac003bf0 con 0x7f14bc107de0 2024-07-31T14:51:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.961+0000 7f14b9ffb640 1 -- 172.21.15.67:0/1583107714 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14ac003d90 con 0x7f14bc107de0 2024-07-31T14:51:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.961+0000 7f14c2b80640 1 -- 172.21.15.67:0/1583107714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14bc1a0120 con 0x7f14bc107de0 2024-07-31T14:51:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.961+0000 7f14b9ffb640 1 -- 172.21.15.67:0/1583107714 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac02fb70 con 0x7f14bc107de0 2024-07-31T14:51:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.961+0000 7f14c2b80640 1 -- 172.21.15.67:0/1583107714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14bc1a0660 con 0x7f14bc107de0 2024-07-31T14:51:38.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.962+0000 7f14c2b80640 1 -- 172.21.15.67:0/1583107714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14bc103720 con 0x7f14bc107de0 2024-07-31T14:51:38.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.964+0000 7f14b9ffb640 1 -- 172.21.15.67:0/1583107714 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f14ac02fd10 con 0x7f14bc107de0 2024-07-31T14:51:38.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.964+0000 7f14b9ffb640 1 --2- 172.21.15.67:0/1583107714 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14a80b3d70 0x7f14a80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:38.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.964+0000 7f14c091c640 1 --2- 172.21.15.67:0/1583107714 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14a80b3d70 0x7f14a80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:38.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.965+0000 7f14b9ffb640 1 -- 172.21.15.67:0/1583107714 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f14ac105680 con 0x7f14bc107de0 2024-07-31T14:51:38.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.965+0000 7f14c091c640 1 --2- 172.21.15.67:0/1583107714 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14a80b3d70 0x7f14a80b6280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f14b400abe0 tx=0x7f14b400a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:38.968+0000 7f14b9ffb640 1 -- 172.21.15.67:0/1583107714 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f14ac0cc790 con 0x7f14bc107de0 2024-07-31T14:51:38.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:38 smithi077 ceph-mon[37327]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-07-31T14:51:39.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:38 smithi073 ceph-mon[36714]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-07-31T14:51:39.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:39.199+0000 7f14c2b80640 1 -- 172.21.15.67:0/1583107714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14bc1028b0 con 0x7f14bc107de0 2024-07-31T14:51:39.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:39.200+0000 7f14b9ffb640 1 -- 172.21.15.67:0/1583107714 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11962 (secure 0 0 0) 0x7f14ac0098b0 con 0x7f14bc107de0 2024-07-31T14:51:39.202 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:51:39.202 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-31T14:51:08:515297+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:51:07.734542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:51:39.202 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-31T14:51:39.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:39.203+0000 7f14c2b80640 1 -- 172.21.15.67:0/1583107714 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14a80b3d70 msgr2=0x7f14a80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:39.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:39.203+0000 7f14c2b80640 1 --2- 172.21.15.67:0/1583107714 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14a80b3d70 0x7f14a80b6280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f14b400abe0 tx=0x7f14b400a2d0 comp rx=0 tx=0).stop 2024-07-31T14:51:39.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:39.203+0000 7f14c2b80640 1 -- 172.21.15.67:0/1583107714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14bc107de0 msgr2=0x7f14bc19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:39.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:39.203+0000 7f14c2b80640 1 --2- 172.21.15.67:0/1583107714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14bc107de0 0x7f14bc19a580 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0319f0 tx=0x7f14ac031a20 comp rx=0 tx=0).stop 2024-07-31T14:51:39.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:39.203+0000 7f14c2b80640 1 -- 172.21.15.67:0/1583107714 shutdown_connections 2024-07-31T14:51:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:39.203+0000 7f14c2b80640 1 --2- 172.21.15.67:0/1583107714 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14a80b3d70 0x7f14a80b6280 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:39.203+0000 7f14c2b80640 1 --2- 172.21.15.67:0/1583107714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc108730 0x7f14bc19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:39.203+0000 7f14c2b80640 1 --2- 172.21.15.67:0/1583107714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14bc107de0 0x7f14bc19a580 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:39.203+0000 7f14c2b80640 1 --2- 172.21.15.67:0/1583107714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14bc1024a0 0x7f14bc199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:39.203+0000 7f14c2b80640 1 -- 172.21.15.67:0/1583107714 >> 172.21.15.67:0/1583107714 conn(0x7f14bc079140 msgr2=0x7f14bc105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:39.203+0000 7f14c2b80640 1 -- 172.21.15.67:0/1583107714 shutdown_connections 2024-07-31T14:51:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:39.203+0000 7f14c2b80640 1 -- 172.21.15.67:0/1583107714 wait complete. 2024-07-31T14:51:39.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:39 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1583107714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:39.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:39 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1583107714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:39.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-07-31T14:51:40.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:39 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1583107714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:40.261 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:40.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:40 smithi077 ceph-mon[37327]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 13 op/s 2024-07-31T14:51:41.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:40 smithi067 ceph-mon[32750]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 13 op/s 2024-07-31T14:51:41.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:40 smithi073 ceph-mon[36714]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 13 op/s 2024-07-31T14:51:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.255+0000 7f6c13b12640 1 -- 172.21.15.67:0/3792669758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c0c10fbf0 msgr2=0x7f6c0c112010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.255+0000 7f6c13b12640 1 --2- 172.21.15.67:0/3792669758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c0c10fbf0 0x7f6c0c112010 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6c04009ca0 tx=0x7f6c0402a950 comp rx=0 tx=0).stop 2024-07-31T14:51:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.256+0000 7f6c13b12640 1 -- 172.21.15.67:0/3792669758 shutdown_connections 2024-07-31T14:51:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.256+0000 7f6c13b12640 1 --2- 172.21.15.67:0/3792669758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c0c10fbf0 0x7f6c0c112010 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.256+0000 7f6c13b12640 1 --2- 172.21.15.67:0/3792669758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c0c102990 0x7f6c0c10f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.256+0000 7f6c13b12640 1 --2- 172.21.15.67:0/3792669758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c0c102040 0x7f6c0c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.256+0000 7f6c13b12640 1 -- 172.21.15.67:0/3792669758 >> 172.21.15.67:0/3792669758 conn(0x7f6c0c079190 msgr2=0x7f6c0c0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.256+0000 7f6c13b12640 1 -- 172.21.15.67:0/3792669758 shutdown_connections 2024-07-31T14:51:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.256+0000 7f6c13b12640 1 -- 172.21.15.67:0/3792669758 wait complete. 2024-07-31T14:51:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.256+0000 7f6c13b12640 1 Processor -- start 2024-07-31T14:51:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.256+0000 7f6c13b12640 1 -- start start 2024-07-31T14:51:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c13b12640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c0c102040 0x7f6c0c19c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c13b12640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c0c102990 0x7f6c0c19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c13b12640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c0c10fbf0 0x7f6c0c1a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c13b12640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c1156a0 con 0x7f6c0c102990 2024-07-31T14:51:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c13b12640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c115520 con 0x7f6c0c102040 2024-07-31T14:51:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c13b12640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c115820 con 0x7f6c0c10fbf0 2024-07-31T14:51:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c110ad640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c0c102990 0x7f6c0c19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c110ad640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c0c102990 0x7f6c0c19c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35838/0 (socket says 172.21.15.67:35838) 2024-07-31T14:51:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c110ad640 1 -- 172.21.15.67:0/4116501697 learned_addr learned my addr 172.21.15.67:0/4116501697 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c118ae640 1 --2- 172.21.15.67:0/4116501697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c0c102040 0x7f6c0c19c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c120af640 1 --2- 172.21.15.67:0/4116501697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c0c10fbf0 0x7f6c0c1a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c110ad640 1 -- 172.21.15.67:0/4116501697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c0c10fbf0 msgr2=0x7f6c0c1a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c110ad640 1 --2- 172.21.15.67:0/4116501697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c0c10fbf0 0x7f6c0c1a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c110ad640 1 -- 172.21.15.67:0/4116501697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c0c102040 msgr2=0x7f6c0c19c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c110ad640 1 --2- 172.21.15.67:0/4116501697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c0c102040 0x7f6c0c19c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c110ad640 1 -- 172.21.15.67:0/4116501697 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0c1a1fa0 con 0x7f6c0c102990 2024-07-31T14:51:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c110ad640 1 --2- 172.21.15.67:0/4116501697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c0c102990 0x7f6c0c19c5d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00d950 tx=0x7f6bfc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.257+0000 7f6c02ffd640 1 -- 172.21.15.67:0/4116501697 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc017070 con 0x7f6c0c102990 2024-07-31T14:51:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.258+0000 7f6c02ffd640 1 -- 172.21.15.67:0/4116501697 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bfc009c30 con 0x7f6c0c102990 2024-07-31T14:51:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.258+0000 7f6c13b12640 1 -- 172.21.15.67:0/4116501697 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c0c1a2290 con 0x7f6c0c102990 2024-07-31T14:51:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.258+0000 7f6c02ffd640 1 -- 172.21.15.67:0/4116501697 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00f620 con 0x7f6c0c102990 2024-07-31T14:51:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.258+0000 7f6c13b12640 1 -- 172.21.15.67:0/4116501697 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c0c1a89f0 con 0x7f6c0c102990 2024-07-31T14:51:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.259+0000 7f6c13b12640 1 -- 172.21.15.67:0/4116501697 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c0c06bc70 con 0x7f6c0c102990 2024-07-31T14:51:42.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.260+0000 7f6c02ffd640 1 -- 172.21.15.67:0/4116501697 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f6bfc005030 con 0x7f6c0c102990 2024-07-31T14:51:42.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.261+0000 7f6c02ffd640 1 --2- 172.21.15.67:0/4116501697 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6bf80b3d40 0x7f6bf80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:42.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.261+0000 7f6c118ae640 1 --2- 172.21.15.67:0/4116501697 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6bf80b3d40 0x7f6bf80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:42.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.262+0000 7f6c02ffd640 1 -- 172.21.15.67:0/4116501697 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f6bfc0e3e80 con 0x7f6c0c102990 2024-07-31T14:51:42.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.262+0000 7f6c118ae640 1 --2- 172.21.15.67:0/4116501697 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6bf80b3d40 0x7f6bf80b6250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6c08005fd0 tx=0x7f6c0800b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:42.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.265+0000 7f6c02ffd640 1 -- 172.21.15.67:0/4116501697 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f6bfc009de0 con 0x7f6c0c102990 2024-07-31T14:51:42.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.460+0000 7f6c13b12640 1 -- 172.21.15.67:0/4116501697 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c0c19d160 con 0x7f6c0c102990 2024-07-31T14:51:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.462+0000 7f6c02ffd640 1 -- 172.21.15.67:0/4116501697 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24874 (secure 0 0 0) 0x7f6bfc0aa830 con 0x7f6c0c102990 2024-07-31T14:51:42.464 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:51:42.464 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T14:47:20.440064+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T14:51:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.464+0000 7f6c13b12640 1 -- 172.21.15.67:0/4116501697 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6bf80b3d40 msgr2=0x7f6bf80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.464+0000 7f6c13b12640 1 --2- 172.21.15.67:0/4116501697 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6bf80b3d40 0x7f6bf80b6250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6c08005fd0 tx=0x7f6c0800b390 comp rx=0 tx=0).stop 2024-07-31T14:51:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.464+0000 7f6c13b12640 1 -- 172.21.15.67:0/4116501697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c0c102990 msgr2=0x7f6c0c19c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.464+0000 7f6c13b12640 1 --2- 172.21.15.67:0/4116501697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c0c102990 0x7f6c0c19c5d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00d950 tx=0x7f6bfc00de10 comp rx=0 tx=0).stop 2024-07-31T14:51:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.464+0000 7f6c13b12640 1 -- 172.21.15.67:0/4116501697 shutdown_connections 2024-07-31T14:51:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.464+0000 7f6c13b12640 1 --2- 172.21.15.67:0/4116501697 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6bf80b3d40 0x7f6bf80b6250 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.464+0000 7f6c13b12640 1 --2- 172.21.15.67:0/4116501697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c0c10fbf0 0x7f6c0c1a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.464+0000 7f6c13b12640 1 --2- 172.21.15.67:0/4116501697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c0c102990 0x7f6c0c19c5d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.464+0000 7f6c13b12640 1 --2- 172.21.15.67:0/4116501697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c0c102040 0x7f6c0c19c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.464+0000 7f6c13b12640 1 -- 172.21.15.67:0/4116501697 >> 172.21.15.67:0/4116501697 conn(0x7f6c0c079190 msgr2=0x7f6c0c108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.464+0000 7f6c13b12640 1 -- 172.21.15.67:0/4116501697 shutdown_connections 2024-07-31T14:51:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:42.464+0000 7f6c13b12640 1 -- 172.21.15.67:0/4116501697 wait complete. 2024-07-31T14:51:42.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:42 smithi077 ceph-mon[37327]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-07-31T14:51:42.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:42 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/4116501697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:51:43.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:42 smithi067 ceph-mon[32750]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-07-31T14:51:43.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:42 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/4116501697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:51:43.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:42 smithi073 ceph-mon[36714]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-07-31T14:51:43.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:42 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/4116501697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T14:51:43.214 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-07-31T14:51:43.482 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:44.986 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:44 smithi067 ceph-mon[32750]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-07-31T14:51:44.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:44 smithi077 ceph-mon[37327]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-07-31T14:51:45.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:44 smithi073 ceph-mon[36714]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-07-31T14:51:45.890 INFO:teuthology.orchestra.run.smithi067.stdout:{"run_dir":"/var/run/ceph"} 2024-07-31T14:51:45.890 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-07-31T14:51:45.891 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 3726 2024-07-31T14:51:45.891 DEBUG:tasks.quiescer:Yielding 2024-07-31T14:51:45.891 INFO:teuthology.run_tasks:Running task workunit... 2024-07-31T14:51:45.905 INFO:tasks.workunit:Pulling workunits from ref 8c796fca5ff5960f7542e7227bad4a784ee2c804 2024-07-31T14:51:45.905 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-07-31T14:51:45.905 INFO:tasks.workunit:timeout=3h 2024-07-31T14:51:45.905 INFO:tasks.workunit:cleanup=True 2024-07-31T14:51:45.906 DEBUG:teuthology.orchestra.run.smithi067:> stat -- /home/ubuntu/cephtest/mnt.0 2024-07-31T14:51:45.906 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:51:45.924 INFO:teuthology.orchestra.run.smithi067.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-07-31T14:51:45.924 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-07-31T14:51:45.924 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-07-31T14:51:45.924 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-31T14:51:45.924 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-07-31T14:51:45.925 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-07-31 14:48:18.028117639 +0000 2024-07-31T14:51:45.925 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-07-31 14:48:18.028117639 +0000 2024-07-31T14:51:45.925 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-07-31 14:51:07.009990341 +0000 2024-07-31T14:51:45.925 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-07-31 14:48:18.028117639 +0000 2024-07-31T14:51:45.925 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-07-31T14:51:45.925 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-07-31T14:51:46.001 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8c796fca5ff5960f7542e7227bad4a784ee2c804 2024-07-31T14:51:46.058 INFO:tasks.workunit.client.0.smithi067.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-07-31T14:51:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:46.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:46 smithi077 ceph-mon[37327]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-07-31T14:51:46.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:46 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:51:47.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:46 smithi067 ceph-mon[32750]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-07-31T14:51:47.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:46 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:51:47.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:46 smithi073 ceph-mon[36714]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-07-31T14:51:47.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:46 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:51:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:47 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:51:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:47 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:51:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:47 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:51:47.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:47 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:51:47.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:47 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:51:47.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:47 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:51:48.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.011+0000 7f37a266e640 1 -- 172.21.15.67:0/919869209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f379c102490 msgr2=0x7f379c1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:48.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.011+0000 7f37a266e640 1 --2- 172.21.15.67:0/919869209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f379c102490 0x7f379c1028a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f378c009900 tx=0x7f378c02a410 comp rx=0 tx=0).stop 2024-07-31T14:51:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.011+0000 7f37a266e640 1 -- 172.21.15.67:0/919869209 shutdown_connections 2024-07-31T14:51:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.011+0000 7f37a266e640 1 --2- 172.21.15.67:0/919869209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f379c108720 0x7f379c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.011+0000 7f37a266e640 1 --2- 172.21.15.67:0/919869209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f379c107dd0 0x7f379c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.011+0000 7f37a266e640 1 --2- 172.21.15.67:0/919869209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f379c102490 0x7f379c1028a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.011+0000 7f37a266e640 1 -- 172.21.15.67:0/919869209 >> 172.21.15.67:0/919869209 conn(0x7f379c079160 msgr2=0x7f379c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.011+0000 7f37a266e640 1 -- 172.21.15.67:0/919869209 shutdown_connections 2024-07-31T14:51:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.011+0000 7f37a266e640 1 -- 172.21.15.67:0/919869209 wait complete. 2024-07-31T14:51:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.012+0000 7f37a266e640 1 Processor -- start 2024-07-31T14:51:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.012+0000 7f37a266e640 1 -- start start 2024-07-31T14:51:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a266e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f379c102490 0x7f379c19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a266e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f379c107dd0 0x7f379c19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a266e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f379c108720 0x7f379c19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a266e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f379c112380 con 0x7f379c108720 2024-07-31T14:51:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a266e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f379c112200 con 0x7f379c107dd0 2024-07-31T14:51:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a266e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f379c112500 con 0x7f379c102490 2024-07-31T14:51:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a0c0b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f379c108720 0x7f379c19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a0c0b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f379c108720 0x7f379c19b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35852/0 (socket says 172.21.15.67:35852) 2024-07-31T14:51:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a0c0b640 1 -- 172.21.15.67:0/1143588118 learned_addr learned my addr 172.21.15.67:0/1143588118 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f379b7fe640 1 --2- 172.21.15.67:0/1143588118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f379c107dd0 0x7f379c19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f379bfff640 1 --2- 172.21.15.67:0/1143588118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f379c102490 0x7f379c19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a0c0b640 1 -- 172.21.15.67:0/1143588118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f379c102490 msgr2=0x7f379c19ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a0c0b640 1 --2- 172.21.15.67:0/1143588118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f379c102490 0x7f379c19ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a0c0b640 1 -- 172.21.15.67:0/1143588118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f379c107dd0 msgr2=0x7f379c19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a0c0b640 1 --2- 172.21.15.67:0/1143588118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f379c107dd0 0x7f379c19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a0c0b640 1 -- 172.21.15.67:0/1143588118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c1a6000 con 0x7f379c108720 2024-07-31T14:51:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.013+0000 7f37a0c0b640 1 --2- 172.21.15.67:0/1143588118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f379c108720 0x7f379c19b7f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f379400eaa0 tx=0x7f379400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.014+0000 7f37997fa640 1 -- 172.21.15.67:0/1143588118 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379400cdb0 con 0x7f379c108720 2024-07-31T14:51:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.014+0000 7f37997fa640 1 -- 172.21.15.67:0/1143588118 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3794005020 con 0x7f379c108720 2024-07-31T14:51:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.014+0000 7f37a266e640 1 -- 172.21.15.67:0/1143588118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379c1a62f0 con 0x7f379c108720 2024-07-31T14:51:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.014+0000 7f37997fa640 1 -- 172.21.15.67:0/1143588118 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37940116d0 con 0x7f379c108720 2024-07-31T14:51:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.014+0000 7f37a266e640 1 -- 172.21.15.67:0/1143588118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379c1a6830 con 0x7f379c108720 2024-07-31T14:51:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.015+0000 7f37a266e640 1 -- 172.21.15.67:0/1143588118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379c1028a0 con 0x7f379c108720 2024-07-31T14:51:48.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.017+0000 7f37997fa640 1 -- 172.21.15.67:0/1143588118 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f37940051d0 con 0x7f379c108720 2024-07-31T14:51:48.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.017+0000 7f37997fa640 1 --2- 172.21.15.67:0/1143588118 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37840b3df0 0x7f37840b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:48.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.017+0000 7f379bfff640 1 --2- 172.21.15.67:0/1143588118 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37840b3df0 0x7f37840b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:48.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.018+0000 7f37997fa640 1 -- 172.21.15.67:0/1143588118 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f37940e46e0 con 0x7f379c108720 2024-07-31T14:51:48.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.018+0000 7f379bfff640 1 --2- 172.21.15.67:0/1143588118 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37840b3df0 0x7f37840b6300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f378c000c00 tx=0x7f378c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:48.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.024+0000 7f37997fa640 1 -- 172.21.15.67:0/1143588118 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f3794016030 con 0x7f379c108720 2024-07-31T14:51:48.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:47 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:51:48.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:47 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:51:48.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:47 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:51:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.260+0000 7f37a266e640 1 -- 172.21.15.67:0/1143588118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f379c06aca0 con 0x7f379c108720 2024-07-31T14:51:48.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.261+0000 7f37997fa640 1 -- 172.21.15.67:0/1143588118 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11962 (secure 0 0 0) 0x7f37940ab110 con 0x7f379c108720 2024-07-31T14:51:48.262 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:51:48.263 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-31T14:51:08:515297+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:51:07.734542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:51:48.263 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-31T14:51:48.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.263+0000 7f37a266e640 1 -- 172.21.15.67:0/1143588118 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37840b3df0 msgr2=0x7f37840b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.263+0000 7f37a266e640 1 --2- 172.21.15.67:0/1143588118 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37840b3df0 0x7f37840b6300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f378c000c00 tx=0x7f378c031400 comp rx=0 tx=0).stop 2024-07-31T14:51:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.263+0000 7f37a266e640 1 -- 172.21.15.67:0/1143588118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f379c108720 msgr2=0x7f379c19b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.263+0000 7f37a266e640 1 --2- 172.21.15.67:0/1143588118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f379c108720 0x7f379c19b7f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f379400eaa0 tx=0x7f379400ef60 comp rx=0 tx=0).stop 2024-07-31T14:51:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.263+0000 7f37a266e640 1 -- 172.21.15.67:0/1143588118 shutdown_connections 2024-07-31T14:51:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.263+0000 7f37a266e640 1 --2- 172.21.15.67:0/1143588118 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37840b3df0 0x7f37840b6300 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.263+0000 7f37a266e640 1 --2- 172.21.15.67:0/1143588118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f379c108720 0x7f379c19b7f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.263+0000 7f37a266e640 1 --2- 172.21.15.67:0/1143588118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f379c107dd0 0x7f379c19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.263+0000 7f37a266e640 1 --2- 172.21.15.67:0/1143588118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f379c102490 0x7f379c19ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.263+0000 7f37a266e640 1 -- 172.21.15.67:0/1143588118 >> 172.21.15.67:0/1143588118 conn(0x7f379c079160 msgr2=0x7f379c100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.263+0000 7f37a266e640 1 -- 172.21.15.67:0/1143588118 shutdown_connections 2024-07-31T14:51:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:48.263+0000 7f37a266e640 1 -- 172.21.15.67:0/1143588118 wait complete. 2024-07-31T14:51:48.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:48 smithi067 ceph-mon[32750]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-07-31T14:51:48.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:48 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1143588118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:48.944 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-31T14:47:04.833142+0000', 'modified': '2024-07-31T14:51:07.734542+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24671, 'mds_1': 24655, 'mds_2': 24650, 'mds_3': 24646, 'mds_4': 24634}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14778': {'gid': 14778, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.67:6839/3532250223', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6838', 'nonce': 3532250223}, {'type': 'v1', 'addr': '172.21.15.67:6839', 'nonce': 3532250223}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14790': {'gid': 14790, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.67:6841/1797875282', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6840', 'nonce': 1797875282}, {'type': 'v1', 'addr': '172.21.15.67:6841', 'nonce': 1797875282}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24602': {'gid': 24602, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6833/1184537619', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6832', 'nonce': 1184537619}, {'type': 'v1', 'addr': '172.21.15.73:6833', 'nonce': 1184537619}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24604': {'gid': 24604, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6833/2710994774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6832', 'nonce': 2710994774}, {'type': 'v1', 'addr': '172.21.15.77:6833', 'nonce': 2710994774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24629': {'gid': 24629, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6835/2854264361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 2854264361}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 2854264361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24634': {'gid': 24634, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.77:6835/1866134372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 1866134372}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 1866134372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24646': {'gid': 24646, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.77:6837/3460575274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6836', 'nonce': 3460575274}, {'type': 'v1', 'addr': '172.21.15.77:6837', 'nonce': 3460575274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24650': {'gid': 24650, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.73:6837/3592232039', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 3592232039}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 3592232039}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24655': {'gid': 24655, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.77:6839/2802806156', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 2802806156}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 2802806156}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24671': {'gid': 24671, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.73:6839/4131046421', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 4131046421}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 4131046421}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24671, 'qdb_cluster': [24671, 24655, 24650, 24646, 24634]} 2024-07-31T14:51:48.944 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-07-31T14:51:48.944 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.f status 2024-07-31T14:51:48.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:48 smithi077 ceph-mon[37327]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-07-31T14:51:48.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:48 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1143588118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.991+0000 7fc0345a9640 1 -- 172.21.15.67:0/397009963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc02c101a50 msgr2=0x7fc02c101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.991+0000 7fc0345a9640 1 --2- 172.21.15.67:0/397009963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc02c101a50 0x7fc02c101e60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009ca0 tx=0x7fc01c02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.992+0000 7fc0345a9640 1 -- 172.21.15.67:0/397009963 shutdown_connections 2024-07-31T14:51:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.992+0000 7fc0345a9640 1 --2- 172.21.15.67:0/397009963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc02c1098e0 0x7fc02c1100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.992+0000 7fc0345a9640 1 --2- 172.21.15.67:0/397009963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc02c108f90 0x7fc02c1093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.992+0000 7fc0345a9640 1 --2- 172.21.15.67:0/397009963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc02c101a50 0x7fc02c101e60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.992+0000 7fc0345a9640 1 -- 172.21.15.67:0/397009963 >> 172.21.15.67:0/397009963 conn(0x7fc02c079880 msgr2=0x7fc02c079c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.992+0000 7fc0345a9640 1 -- 172.21.15.67:0/397009963 shutdown_connections 2024-07-31T14:51:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.992+0000 7fc0345a9640 1 -- 172.21.15.67:0/397009963 wait complete. 2024-07-31T14:51:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.992+0000 7fc0345a9640 1 Processor -- start 2024-07-31T14:51:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.992+0000 7fc0345a9640 1 -- start start 2024-07-31T14:51:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc0345a9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc02c101a50 0x7fc02c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc0345a9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc02c108f90 0x7fc02c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc0345a9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc02c1098e0 0x7fc02c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc0345a9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c113540 con 0x7fc02c101a50 2024-07-31T14:51:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc0345a9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1133c0 con 0x7fc02c1098e0 2024-07-31T14:51:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc0345a9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1136c0 con 0x7fc02c108f90 2024-07-31T14:51:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc032345640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc02c101a50 0x7fc02c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc032345640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc02c101a50 0x7fc02c19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35874/0 (socket says 172.21.15.67:35874) 2024-07-31T14:51:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc032345640 1 -- 172.21.15.67:0/920795280 learned_addr learned my addr 172.21.15.67:0/920795280 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc032b46640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc02c1098e0 0x7fc02c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc031b44640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc02c108f90 0x7fc02c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc032345640 1 -- 172.21.15.67:0/920795280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc02c108f90 msgr2=0x7fc02c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc032345640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc02c108f90 0x7fc02c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc032345640 1 -- 172.21.15.67:0/920795280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc02c1098e0 msgr2=0x7fc02c19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc032345640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc02c1098e0 0x7fc02c19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc032345640 1 -- 172.21.15.67:0/920795280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02c19ff50 con 0x7fc02c101a50 2024-07-31T14:51:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.993+0000 7fc031b44640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc02c108f90 0x7fc02c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.994+0000 7fc032345640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc02c101a50 0x7fc02c19a020 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc01c02ade0 tx=0x7fc01c031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.994+0000 7fc02b7fe640 1 -- 172.21.15.67:0/920795280 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c003b70 con 0x7fc02c101a50 2024-07-31T14:51:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.994+0000 7fc0345a9640 1 -- 172.21.15.67:0/920795280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02c1a01e0 con 0x7fc02c101a50 2024-07-31T14:51:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.994+0000 7fc02b7fe640 1 -- 172.21.15.67:0/920795280 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc01c003d10 con 0x7fc02c101a50 2024-07-31T14:51:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.994+0000 7fc02b7fe640 1 -- 172.21.15.67:0/920795280 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c02fa50 con 0x7fc02c101a50 2024-07-31T14:51:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.994+0000 7fc0345a9640 1 -- 172.21.15.67:0/920795280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02c1a69d0 con 0x7fc02c101a50 2024-07-31T14:51:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.996+0000 7fc03b57a740 1 Processor -- start 2024-07-31T14:51:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.997+0000 7fc03b57a740 1 -- start start 2024-07-31T14:51:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.997+0000 7fc03b57a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d362d1b0 0x55f8d39cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.997+0000 7fc03b57a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d36042f0 0x55f8d39d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.997+0000 7fc03b57a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d360c4d0 0x55f8d39d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.997+0000 7fc03b57a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f8d39453b0 con 0x55f8d362d1b0 2024-07-31T14:51:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.997+0000 7fc03b57a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f8d39348b0 con 0x55f8d360c4d0 2024-07-31T14:51:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.997+0000 7fc03b57a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f8d39388f0 con 0x55f8d36042f0 2024-07-31T14:51:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.997+0000 7fc032b46640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d362d1b0 0x55f8d39cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.997+0000 7fc032b46640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d362d1b0 0x55f8d39cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35876/0 (socket says 172.21.15.67:35876) 2024-07-31T14:51:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.997+0000 7fc032b46640 1 -- 172.21.15.67:0/2942778761 learned_addr learned my addr 172.21.15.67:0/2942778761 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.997+0000 7fc02b7fe640 1 -- 172.21.15.67:0/920795280 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7fc01c004580 con 0x7fc02c101a50 2024-07-31T14:51:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.997+0000 7fc031b44640 1 --2- 172.21.15.67:0/2942778761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d360c4d0 0x55f8d39d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.997+0000 7fc032b46640 1 -- 172.21.15.67:0/2942778761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d36042f0 msgr2=0x55f8d39d0220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:51:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.998+0000 7fc032b46640 1 --2- 172.21.15.67:0/2942778761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d36042f0 0x55f8d39d0220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.998+0000 7fc032b46640 1 -- 172.21.15.67:0/2942778761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d360c4d0 msgr2=0x55f8d39d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.998+0000 7fc032b46640 1 --2- 172.21.15.67:0/2942778761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d360c4d0 0x55f8d39d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.998+0000 7fc032b46640 1 -- 172.21.15.67:0/2942778761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8d38db9a0 con 0x55f8d362d1b0 2024-07-31T14:51:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.998+0000 7fc031b44640 1 --2- 172.21.15.67:0/2942778761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d360c4d0 0x55f8d39d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:51:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.998+0000 7fc032b46640 1 --2- 172.21.15.67:0/2942778761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d362d1b0 0x55f8d39cfce0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc02400b6b0 tx=0x7fc024004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948feb0a96e2b477 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.998+0000 7fc028bfe640 1 -- 172.21.15.67:0/2942778761 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024003b20 con 0x55f8d362d1b0 2024-07-31T14:51:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.998+0000 7fc028bfe640 1 -- 172.21.15.67:0/2942778761 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc024015070 con 0x55f8d362d1b0 2024-07-31T14:51:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.998+0000 7fc028bfe640 1 -- 172.21.15.67:0/2942778761 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02401ae60 con 0x55f8d362d1b0 2024-07-31T14:51:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.998+0000 7fc02b7fe640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc0140b3d40 0x7fc0140b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.998+0000 7fc032345640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc0140b3d40 0x7fc0140b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.999+0000 7fc02b7fe640 1 -- 172.21.15.67:0/920795280 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc01c1056b0 con 0x7fc02c101a50 2024-07-31T14:51:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.999+0000 7fc03b57a740 1 -- 172.21.15.67:0/2942778761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d362d1b0 msgr2=0x55f8d39cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.999+0000 7fc032345640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc0140b3d40 0x7fc0140b6250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55f8d39ce960 tx=0x7fc01c009460 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.999+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2942778761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d362d1b0 0x55f8d39cfce0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc02400b6b0 tx=0x7fc024004610 comp rx=0 tx=0).stop 2024-07-31T14:51:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.999+0000 7fc03b57a740 1 -- 172.21.15.67:0/2942778761 shutdown_connections 2024-07-31T14:51:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.999+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2942778761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d360c4d0 0x55f8d39d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.999+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2942778761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d36042f0 0x55f8d39d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.999+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2942778761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d362d1b0 0x55f8d39cfce0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.999+0000 7fc03b57a740 1 -- 172.21.15.67:0/2942778761 >> 172.21.15.67:0/2942778761 conn(0x55f8d3621300 msgr2=0x55f8d39cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.999+0000 7fc03b57a740 1 -- 172.21.15.67:0/2942778761 shutdown_connections 2024-07-31T14:51:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:48.999+0000 7fc03b57a740 1 -- 172.21.15.67:0/2942778761 wait complete. 2024-07-31T14:51:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.000+0000 7fc03b57a740 1 -- start start 2024-07-31T14:51:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.000+0000 7fc03b57a740 1 Processor -- start 2024-07-31T14:51:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.000+0000 7fc03b57a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d36042f0 0x55f8d39db8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.000+0000 7fc03b57a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d360c4d0 0x55f8d39dbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.000+0000 7fc03b57a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d362d1b0 0x55f8d39e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.000+0000 7fc03b57a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f8d386eab0 con 0x55f8d362d1b0 2024-07-31T14:51:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.000+0000 7fc03b57a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f8d38bccf0 con 0x55f8d360c4d0 2024-07-31T14:51:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.000+0000 7fc03b57a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f8d39de460 con 0x55f8d36042f0 2024-07-31T14:51:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.000+0000 7fc03b57a740 10 client.0 _ll_register_callbacks cb 0x55f8d392e9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.000+0000 7fc032b46640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d362d1b0 0x55f8d39e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.000+0000 7fc032b46640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d362d1b0 0x55f8d39e77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35880/0 (socket says 172.21.15.67:35880) 2024-07-31T14:51:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.000+0000 7fc032b46640 1 -- 172.21.15.67:0/2883845132 learned_addr learned my addr 172.21.15.67:0/2883845132 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.000+0000 7fc032b46640 1 --2- 172.21.15.67:0/2883845132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d36042f0 0x55f8d39db8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.000+0000 7fc031b44640 1 --2- 172.21.15.67:0/2883845132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d360c4d0 0x55f8d39dbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.001+0000 7fc032b46640 1 -- 172.21.15.67:0/2883845132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d36042f0 msgr2=0x55f8d39db8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.001+0000 7fc032b46640 1 --2- 172.21.15.67:0/2883845132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d36042f0 0x55f8d39db8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.001+0000 7fc032b46640 1 -- 172.21.15.67:0/2883845132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d360c4d0 msgr2=0x55f8d39dbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.001+0000 7fc032b46640 1 --2- 172.21.15.67:0/2883845132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d360c4d0 0x55f8d39dbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.001+0000 7fc032b46640 1 -- 172.21.15.67:0/2883845132 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8d3698790 con 0x55f8d362d1b0 2024-07-31T14:51:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.001+0000 7fc032b46640 1 --2- 172.21.15.67:0/2883845132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d362d1b0 0x55f8d39e77e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc02401abe0 tx=0x7fc02401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450708a88d78079c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.001+0000 7fc011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:51:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.001+0000 7fc011ffb640 1 -- 172.21.15.67:0/2883845132 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024023040 con 0x55f8d362d1b0 2024-07-31T14:51:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.001+0000 7fc011ffb640 1 -- 172.21.15.67:0/2883845132 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc024003d50 con 0x55f8d362d1b0 2024-07-31T14:51:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.001+0000 7fc03b57a740 1 -- 172.21.15.67:0/2883845132 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc024015070 con 0x55f8d362d1b0 2024-07-31T14:51:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.001+0000 7fc011ffb640 1 -- 172.21.15.67:0/2883845132 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc024032050 con 0x55f8d362d1b0 2024-07-31T14:51:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.001+0000 7fc03b57a740 10 client.15711 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.001+0000 7fc03b57a740 1 -- 172.21.15.67:0/2883845132 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f8d3711620 con 0x55f8d362d1b0 2024-07-31T14:51:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.001+0000 7fc011ffb640 1 -- 172.21.15.67:0/2883845132 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc024015070 con 0x55f8d362d1b0 2024-07-31T14:51:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.002+0000 7fc011ffb640 1 -- 172.21.15.67:0/2883845132 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc02402c500 con 0x55f8d362d1b0 2024-07-31T14:51:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.002+0000 7fc03b57a740 10 client.15711 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.002+0000 7fc03b57a740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:51:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.002+0000 7fc03b57a740 20 client.15711 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.002+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2883845132 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f8d35ebd80 0x55f8d39ed770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.002+0000 7fc03b57a740 4 client.15711 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:51:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.002+0000 7fc03b57a740 1 -- 172.21.15.67:0/2883845132 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8d3947300 con 0x55f8d35ebd80 2024-07-31T14:51:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.002+0000 7fc031b44640 1 --2- 172.21.15.67:0/2883845132 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f8d35ebd80 0x55f8d39ed770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.002+0000 7fc011ffb640 10 client.15711 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.002+0000 7fc031b44640 1 --2- 172.21.15.67:0/2883845132 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f8d35ebd80 0x55f8d39ed770 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8be7bef007b6ab5 server_cookie=1e23f3cc1c267f2f in_seq=0 out_seq=0 2024-07-31T14:51:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.003+0000 7fc011ffb640 1 -- 172.21.15.67:0/2883845132 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8d3947300 con 0x55f8d35ebd80 2024-07-31T14:51:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.003+0000 7fc011ffb640 10 client.15711 handle_command_reply: tid=0 2024-07-31T14:51:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.003+0000 7fc03b57a740 1 client.15711 shutdown 2024-07-31T14:51:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.003+0000 7fc03b57a740 1 -- 172.21.15.67:0/2883845132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d362d1b0 msgr2=0x55f8d39e77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.003+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2883845132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d362d1b0 0x55f8d39e77e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc02401abe0 tx=0x7fc02401ad80 comp rx=0 tx=0).stop 2024-07-31T14:51:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.003+0000 7fc03b57a740 1 -- 172.21.15.67:0/2883845132 shutdown_connections 2024-07-31T14:51:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.003+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2883845132 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f8d35ebd80 0x55f8d39ed770 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.003+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2883845132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d362d1b0 0x55f8d39e77e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.003+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2883845132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d360c4d0 0x55f8d39dbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.003+0000 7fc011ffb640 0 client.15711 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.003+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2883845132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d36042f0 0x55f8d39db8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.003+0000 7fc03b57a740 1 -- 172.21.15.67:0/2883845132 >> 172.21.15.67:0/2883845132 conn(0x55f8d3621300 msgr2=0x55f8d39d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.004+0000 7fc03b57a740 1 -- 172.21.15.67:0/2883845132 shutdown_connections 2024-07-31T14:51:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.004+0000 7fc03b57a740 1 -- 172.21.15.67:0/2883845132 wait complete. 2024-07-31T14:51:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.004+0000 7fc03b57a740 20 client.15711 trim_cache size 0 max 16384 2024-07-31T14:51:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.019+0000 7fc03b57a740 1 Processor -- start 2024-07-31T14:51:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.019+0000 7fc03b57a740 1 -- start start 2024-07-31T14:51:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc03b57a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d35ebd80 0x55f8d39d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc03b57a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d36042f0 0x55f8d39d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc03b57a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d360c4d0 0x55f8d39d4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc03b57a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1b06d0 con 0x55f8d360c4d0 2024-07-31T14:51:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc03b57a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1b2210 con 0x55f8d35ebd80 2024-07-31T14:51:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc03b57a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1b0da0 con 0x55f8d36042f0 2024-07-31T14:51:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc032b46640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d360c4d0 0x55f8d39d4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc032b46640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d360c4d0 0x55f8d39d4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35892/0 (socket says 172.21.15.67:35892) 2024-07-31T14:51:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc032b46640 1 -- 172.21.15.67:0/1319043477 learned_addr learned my addr 172.21.15.67:0/1319043477 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc032b46640 1 --2- 172.21.15.67:0/1319043477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d35ebd80 0x55f8d39d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc031b44640 1 --2- 172.21.15.67:0/1319043477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d36042f0 0x55f8d39d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc032b46640 1 -- 172.21.15.67:0/1319043477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d36042f0 msgr2=0x55f8d39d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc032b46640 1 --2- 172.21.15.67:0/1319043477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d36042f0 0x55f8d39d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc032b46640 1 -- 172.21.15.67:0/1319043477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d35ebd80 msgr2=0x55f8d39d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc032b46640 1 --2- 172.21.15.67:0/1319043477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d35ebd80 0x55f8d39d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.020+0000 7fc032b46640 1 -- 172.21.15.67:0/1319043477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8d39d24f0 con 0x55f8d360c4d0 2024-07-31T14:51:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.021+0000 7fc032b46640 1 --2- 172.21.15.67:0/1319043477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d360c4d0 0x55f8d39d4da0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc024003780 tx=0x7fc024013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb29f0c33056c240 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.021+0000 7fc013fff640 1 -- 172.21.15.67:0/1319043477 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02403a070 con 0x55f8d360c4d0 2024-07-31T14:51:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.021+0000 7fc013fff640 1 -- 172.21.15.67:0/1319043477 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc024015070 con 0x55f8d360c4d0 2024-07-31T14:51:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.021+0000 7fc013fff640 1 -- 172.21.15.67:0/1319043477 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02401ae60 con 0x55f8d360c4d0 2024-07-31T14:51:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.022+0000 7fc03b57a740 1 -- 172.21.15.67:0/1319043477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d360c4d0 msgr2=0x55f8d39d4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.022+0000 7fc03b57a740 1 --2- 172.21.15.67:0/1319043477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d360c4d0 0x55f8d39d4da0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc024003780 tx=0x7fc024013040 comp rx=0 tx=0).stop 2024-07-31T14:51:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.022+0000 7fc03b57a740 1 -- 172.21.15.67:0/1319043477 shutdown_connections 2024-07-31T14:51:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.022+0000 7fc03b57a740 1 --2- 172.21.15.67:0/1319043477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d360c4d0 0x55f8d39d4da0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.022+0000 7fc03b57a740 1 --2- 172.21.15.67:0/1319043477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d36042f0 0x55f8d39d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.022+0000 7fc03b57a740 1 --2- 172.21.15.67:0/1319043477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d35ebd80 0x55f8d39d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.022+0000 7fc03b57a740 1 -- 172.21.15.67:0/1319043477 >> 172.21.15.67:0/1319043477 conn(0x55f8d3621300 msgr2=0x55f8d39d9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.022+0000 7fc03b57a740 1 -- 172.21.15.67:0/1319043477 shutdown_connections 2024-07-31T14:51:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.022+0000 7fc03b57a740 1 -- 172.21.15.67:0/1319043477 wait complete. 2024-07-31T14:51:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.022+0000 7fc03b57a740 1 -- start start 2024-07-31T14:51:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.022+0000 7fc03b57a740 1 Processor -- start 2024-07-31T14:51:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc03b57a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d35ebd80 0x55f8d39e1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc03b57a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d36042f0 0x55f8d39e24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc03b57a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d360c4d0 0x55f8d3a52f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc03b57a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1b1b40 con 0x55f8d360c4d0 2024-07-31T14:51:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc03b57a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1b1470 con 0x55f8d36042f0 2024-07-31T14:51:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc03b57a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1ae6b0 con 0x55f8d35ebd80 2024-07-31T14:51:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc03b57a740 10 client.0 _ll_register_callbacks cb 0x7fc024004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc032b46640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d360c4d0 0x55f8d3a52f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc032b46640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d360c4d0 0x55f8d3a52f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35902/0 (socket says 172.21.15.67:35902) 2024-07-31T14:51:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc032b46640 1 -- 172.21.15.67:0/2493284220 learned_addr learned my addr 172.21.15.67:0/2493284220 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc031b44640 1 --2- 172.21.15.67:0/2493284220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d36042f0 0x55f8d39e24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc032b46640 1 --2- 172.21.15.67:0/2493284220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d35ebd80 0x55f8d39e1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc032b46640 1 -- 172.21.15.67:0/2493284220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d35ebd80 msgr2=0x55f8d39e1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc032b46640 1 --2- 172.21.15.67:0/2493284220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d35ebd80 0x55f8d39e1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc032b46640 1 -- 172.21.15.67:0/2493284220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d36042f0 msgr2=0x55f8d39e24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc032b46640 1 --2- 172.21.15.67:0/2493284220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d36042f0 0x55f8d39e24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc032b46640 1 -- 172.21.15.67:0/2493284220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8d3a535e0 con 0x55f8d360c4d0 2024-07-31T14:51:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc032b46640 1 --2- 172.21.15.67:0/2493284220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d360c4d0 0x55f8d3a52f40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc024003730 tx=0x7fc02401c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20fa7a2334e7dc3e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:51:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc011ffb640 1 -- 172.21.15.67:0/2493284220 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02403a040 con 0x55f8d360c4d0 2024-07-31T14:51:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.023+0000 7fc011ffb640 1 -- 172.21.15.67:0/2493284220 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc024003b30 con 0x55f8d360c4d0 2024-07-31T14:51:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.024+0000 7fc03b57a740 1 -- 172.21.15.67:0/2493284220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc024015070 con 0x55f8d360c4d0 2024-07-31T14:51:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.024+0000 7fc011ffb640 1 -- 172.21.15.67:0/2493284220 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc024003d30 con 0x55f8d360c4d0 2024-07-31T14:51:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.024+0000 7fc03b57a740 10 client.15717 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.024+0000 7fc03b57a740 1 -- 172.21.15.67:0/2493284220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f8d3a54850 con 0x55f8d360c4d0 2024-07-31T14:51:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.024+0000 7fc011ffb640 1 -- 172.21.15.67:0/2493284220 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc024015070 con 0x55f8d360c4d0 2024-07-31T14:51:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.024+0000 7fc011ffb640 1 -- 172.21.15.67:0/2493284220 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc02403e070 con 0x55f8d360c4d0 2024-07-31T14:51:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.024+0000 7fc03b57a740 10 client.15717 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.024+0000 7fc03b57a740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:51:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.024+0000 7fc03b57a740 20 client.15717 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.024+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2493284220 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f8d362d1b0 0x55f8d3a57030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.024+0000 7fc03b57a740 4 client.15717 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:51:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.024+0000 7fc03b57a740 1 -- 172.21.15.67:0/2493284220 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8d3942da0 con 0x55f8d362d1b0 2024-07-31T14:51:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.025+0000 7fc031b44640 1 --2- 172.21.15.67:0/2493284220 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f8d362d1b0 0x55f8d3a57030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.025+0000 7fc011ffb640 10 client.15717 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.025+0000 7fc031b44640 1 --2- 172.21.15.67:0/2493284220 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f8d362d1b0 0x55f8d3a57030 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c1f40caff32f6f4 server_cookie=f2989bb137cca0ca in_seq=0 out_seq=0 2024-07-31T14:51:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.033+0000 7fc011ffb640 1 -- 172.21.15.67:0/2493284220 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f8d3942da0 con 0x55f8d362d1b0 2024-07-31T14:51:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.033+0000 7fc011ffb640 10 client.15717 handle_command_reply: tid=0 2024-07-31T14:51:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.033+0000 7fc03b57a740 1 client.15717 shutdown 2024-07-31T14:51:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.033+0000 7fc03b57a740 1 -- 172.21.15.67:0/2493284220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d360c4d0 msgr2=0x55f8d3a52f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.033+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2493284220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d360c4d0 0x55f8d3a52f40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc024003730 tx=0x7fc02401c040 comp rx=0 tx=0).stop 2024-07-31T14:51:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.033+0000 7fc03b57a740 1 -- 172.21.15.67:0/2493284220 shutdown_connections 2024-07-31T14:51:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.033+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2493284220 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f8d362d1b0 0x55f8d3a57030 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.033+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2493284220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f8d360c4d0 0x55f8d3a52f40 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.033+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2493284220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d36042f0 0x55f8d39e24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.033+0000 7fc03b57a740 1 --2- 172.21.15.67:0/2493284220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8d35ebd80 0x55f8d39e1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.033+0000 7fc011ffb640 0 client.15717 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:51:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.033+0000 7fc03b57a740 1 -- 172.21.15.67:0/2493284220 >> 172.21.15.67:0/2493284220 conn(0x55f8d3621300 msgr2=0x55f8d39d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.033+0000 7fc03b57a740 1 -- 172.21.15.67:0/2493284220 shutdown_connections 2024-07-31T14:51:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.033+0000 7fc03b57a740 1 -- 172.21.15.67:0/2493284220 wait complete. 2024-07-31T14:51:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.034+0000 7fc03b57a740 20 client.15717 trim_cache size 0 max 16384 2024-07-31T14:51:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:51:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:51:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:51:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:51:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.034+0000 7fc0345a9640 1 -- 172.21.15.67:0/920795280 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc0140b3d40 msgr2=0x7fc0140b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.034+0000 7fc0345a9640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc0140b3d40 0x7fc0140b6250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55f8d39ce960 tx=0x7fc01c009460 comp rx=0 tx=0).stop 2024-07-31T14:51:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.034+0000 7fc0345a9640 1 -- 172.21.15.67:0/920795280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc02c101a50 msgr2=0x7fc02c19a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.034+0000 7fc0345a9640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc02c101a50 0x7fc02c19a020 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc01c02ade0 tx=0x7fc01c031770 comp rx=0 tx=0).stop 2024-07-31T14:51:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.034+0000 7fc0345a9640 1 -- 172.21.15.67:0/920795280 shutdown_connections 2024-07-31T14:51:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.034+0000 7fc0345a9640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc0140b3d40 0x7fc0140b6250 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.034+0000 7fc0345a9640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc02c1098e0 0x7fc02c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.034+0000 7fc0345a9640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc02c108f90 0x7fc02c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.034+0000 7fc0345a9640 1 --2- 172.21.15.67:0/920795280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc02c101a50 0x7fc02c19a020 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.034+0000 7fc0345a9640 1 -- 172.21.15.67:0/920795280 >> 172.21.15.67:0/920795280 conn(0x7fc02c079880 msgr2=0x7fc02c107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.034+0000 7fc0345a9640 1 -- 172.21.15.67:0/920795280 shutdown_connections 2024-07-31T14:51:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:51:49.034+0000 7fc0345a9640 1 -- 172.21.15.67:0/920795280 wait complete. 2024-07-31T14:51:49.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:48 smithi073 ceph-mon[36714]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-07-31T14:51:49.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:48 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1143588118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:51:49.257 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:50.774 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:50 smithi067 ceph-mon[32750]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-07-31T14:51:50.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:50 smithi077 ceph-mon[37327]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-07-31T14:51:51.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.014+0000 7ff63f4f3640 1 -- 172.21.15.67:0/2880796727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff638107df0 msgr2=0x7ff638108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:51.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.014+0000 7ff63f4f3640 1 --2- 172.21.15.67:0/2880796727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff638107df0 0x7ff638108200 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff628009900 tx=0x7ff62802a3f0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.014+0000 7ff63f4f3640 1 -- 172.21.15.67:0/2880796727 shutdown_connections 2024-07-31T14:51:51.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.014+0000 7ff63f4f3640 1 --2- 172.21.15.67:0/2880796727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638108740 0x7ff63810ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.014+0000 7ff63f4f3640 1 --2- 172.21.15.67:0/2880796727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff638107df0 0x7ff638108200 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.014+0000 7ff63f4f3640 1 --2- 172.21.15.67:0/2880796727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6381024b0 0x7ff6381028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.014+0000 7ff63f4f3640 1 -- 172.21.15.67:0/2880796727 >> 172.21.15.67:0/2880796727 conn(0x7ff638079170 msgr2=0x7ff638079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:51.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.014+0000 7ff63f4f3640 1 -- 172.21.15.67:0/2880796727 shutdown_connections 2024-07-31T14:51:51.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.014+0000 7ff63f4f3640 1 -- 172.21.15.67:0/2880796727 wait complete. 2024-07-31T14:51:51.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.015+0000 7ff63f4f3640 1 Processor -- start 2024-07-31T14:51:51.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.015+0000 7ff63f4f3640 1 -- start start 2024-07-31T14:51:51.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.015+0000 7ff63f4f3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6381024b0 0x7ff638199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.015+0000 7ff63f4f3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638107df0 0x7ff63819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.015+0000 7ff63f4f3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff638108740 0x7ff63819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.015+0000 7ff63f4f3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff638112390 con 0x7ff6381024b0 2024-07-31T14:51:51.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.015+0000 7ff63f4f3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff638112210 con 0x7ff638107df0 2024-07-31T14:51:51.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.015+0000 7ff63f4f3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff638112510 con 0x7ff638108740 2024-07-31T14:51:51.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.015+0000 7ff63d28f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6381024b0 0x7ff638199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.015+0000 7ff63d28f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6381024b0 0x7ff638199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52570/0 (socket says 172.21.15.67:52570) 2024-07-31T14:51:51.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.015+0000 7ff63d28f640 1 -- 172.21.15.67:0/1656941760 learned_addr learned my addr 172.21.15.67:0/1656941760 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:51.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.016+0000 7ff63da90640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff638108740 0x7ff63819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.016+0000 7ff63ca8e640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638107df0 0x7ff63819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.016+0000 7ff63d28f640 1 -- 172.21.15.67:0/1656941760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff638108740 msgr2=0x7ff63819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:51.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.016+0000 7ff63d28f640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff638108740 0x7ff63819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.016+0000 7ff63d28f640 1 -- 172.21.15.67:0/1656941760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638107df0 msgr2=0x7ff63819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:51.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.016+0000 7ff63d28f640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638107df0 0x7ff63819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.016+0000 7ff63d28f640 1 -- 172.21.15.67:0/1656941760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63819fdc0 con 0x7ff6381024b0 2024-07-31T14:51:51.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.016+0000 7ff63da90640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff638108740 0x7ff63819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:51.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.016+0000 7ff63d28f640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6381024b0 0x7ff638199f10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff63400c970 tx=0x7ff63400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:51.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.016+0000 7ff62e7fc640 1 -- 172.21.15.67:0/1656941760 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634007bb0 con 0x7ff6381024b0 2024-07-31T14:51:51.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.016+0000 7ff62e7fc640 1 -- 172.21.15.67:0/1656941760 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff634007d50 con 0x7ff6381024b0 2024-07-31T14:51:51.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.016+0000 7ff63f4f3640 1 -- 172.21.15.67:0/1656941760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6381a00b0 con 0x7ff6381024b0 2024-07-31T14:51:51.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.016+0000 7ff62e7fc640 1 -- 172.21.15.67:0/1656941760 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63400f450 con 0x7ff6381024b0 2024-07-31T14:51:51.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.016+0000 7ff63f4f3640 1 -- 172.21.15.67:0/1656941760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6381a68d0 con 0x7ff6381024b0 2024-07-31T14:51:51.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.019+0000 7ff645a44740 1 Processor -- start 2024-07-31T14:51:51.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.019+0000 7ff645a44740 1 -- start start 2024-07-31T14:51:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.019+0000 7ff645a44740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106e1df60 0x55f1071c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.019+0000 7ff645a44740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106df50a0 0x55f1071c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.019+0000 7ff645a44740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106dfd280 0x55f1071c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff645a44740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f107139940 con 0x55f106dfd280 2024-07-31T14:51:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff645a44740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f107128e40 con 0x55f106e1df60 2024-07-31T14:51:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff645a44740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f10712ce80 con 0x55f106df50a0 2024-07-31T14:51:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff62e7fc640 1 -- 172.21.15.67:0/1656941760 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7ff634004f20 con 0x7ff6381024b0 2024-07-31T14:51:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff63ca8e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106dfd280 0x55f1071c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff63ca8e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106dfd280 0x55f1071c3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52574/0 (socket says 172.21.15.67:52574) 2024-07-31T14:51:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff63ca8e640 1 -- 172.21.15.67:0/3608441197 learned_addr learned my addr 172.21.15.67:0/3608441197 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff63da90640 1 --2- 172.21.15.67:0/3608441197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106e1df60 0x55f1071c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff63d28f640 1 --2- 172.21.15.67:0/3608441197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106df50a0 0x55f1071c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff63ca8e640 1 -- 172.21.15.67:0/3608441197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106df50a0 msgr2=0x55f1071c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:51.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff63ca8e640 1 --2- 172.21.15.67:0/3608441197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106df50a0 0x55f1071c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff63ca8e640 1 -- 172.21.15.67:0/3608441197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106e1df60 msgr2=0x55f1071c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:51.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff63ca8e640 1 --2- 172.21.15.67:0/3608441197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106e1df60 0x55f1071c33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff63ca8e640 1 -- 172.21.15.67:0/3608441197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1070cfa60 con 0x55f106dfd280 2024-07-31T14:51:51.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff63d28f640 1 --2- 172.21.15.67:0/3608441197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106df50a0 0x55f1071c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:51:51.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.020+0000 7ff63ca8e640 1 --2- 172.21.15.67:0/3608441197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106dfd280 0x55f1071c3e20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff63806b0f0 tx=0x7ff628031c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7accd6465118c54 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:51.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.021+0000 7ff6237fe640 1 -- 172.21.15.67:0/3608441197 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628035050 con 0x55f106dfd280 2024-07-31T14:51:51.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.021+0000 7ff6237fe640 1 -- 172.21.15.67:0/3608441197 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff628004030 con 0x55f106dfd280 2024-07-31T14:51:51.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.021+0000 7ff6237fe640 1 -- 172.21.15.67:0/3608441197 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6280053d0 con 0x55f106dfd280 2024-07-31T14:51:51.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.021+0000 7ff62e7fc640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff6240b3df0 0x7ff6240b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.021+0000 7ff63da90640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff6240b3df0 0x7ff6240b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.021+0000 7ff62e7fc640 1 -- 172.21.15.67:0/1656941760 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff6340e4090 con 0x7ff6381024b0 2024-07-31T14:51:51.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.022+0000 7ff645a44740 1 -- 172.21.15.67:0/3608441197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106dfd280 msgr2=0x55f1071c3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:51.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.022+0000 7ff63da90640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff6240b3df0 0x7ff6240b6300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff630007f90 tx=0x7ff630007720 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:51.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.022+0000 7ff645a44740 1 --2- 172.21.15.67:0/3608441197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106dfd280 0x55f1071c3e20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff63806b0f0 tx=0x7ff628031c50 comp rx=0 tx=0).stop 2024-07-31T14:51:51.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.022+0000 7ff645a44740 1 -- 172.21.15.67:0/3608441197 shutdown_connections 2024-07-31T14:51:51.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.022+0000 7ff645a44740 1 --2- 172.21.15.67:0/3608441197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106dfd280 0x55f1071c3e20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.022+0000 7ff645a44740 1 --2- 172.21.15.67:0/3608441197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106df50a0 0x55f1071c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.022+0000 7ff645a44740 1 --2- 172.21.15.67:0/3608441197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106e1df60 0x55f1071c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.022+0000 7ff645a44740 1 -- 172.21.15.67:0/3608441197 >> 172.21.15.67:0/3608441197 conn(0x55f106e12350 msgr2=0x55f1071c1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:51.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.022+0000 7ff645a44740 1 -- 172.21.15.67:0/3608441197 shutdown_connections 2024-07-31T14:51:51.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.022+0000 7ff645a44740 1 -- 172.21.15.67:0/3608441197 wait complete. 2024-07-31T14:51:51.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.022+0000 7ff645a44740 1 -- start start 2024-07-31T14:51:51.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.022+0000 7ff645a44740 1 Processor -- start 2024-07-31T14:51:51.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff645a44740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106df50a0 0x55f1071cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff645a44740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106dfd280 0x55f1071cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff645a44740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106e1df60 0x55f1071dae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff645a44740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f106f8c5f0 con 0x55f106e1df60 2024-07-31T14:51:51.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff645a44740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f1070cb8b0 con 0x55f106df50a0 2024-07-31T14:51:51.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff645a44740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f1071d1b20 con 0x55f106dfd280 2024-07-31T14:51:51.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff645a44740 10 client.0 _ll_register_callbacks cb 0x55f1070d57f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:51.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff63da90640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106e1df60 0x55f1071dae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff63da90640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106e1df60 0x55f1071dae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52584/0 (socket says 172.21.15.67:52584) 2024-07-31T14:51:51.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff63da90640 1 -- 172.21.15.67:0/507179361 learned_addr learned my addr 172.21.15.67:0/507179361 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:51.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff63ca8e640 1 --2- 172.21.15.67:0/507179361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106dfd280 0x55f1071cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff63d28f640 1 --2- 172.21.15.67:0/507179361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106df50a0 0x55f1071cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff63da90640 1 -- 172.21.15.67:0/507179361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106dfd280 msgr2=0x55f1071cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:51.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff63da90640 1 --2- 172.21.15.67:0/507179361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106dfd280 0x55f1071cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff63da90640 1 -- 172.21.15.67:0/507179361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106df50a0 msgr2=0x55f1071cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:51.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff63da90640 1 --2- 172.21.15.67:0/507179361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106df50a0 0x55f1071cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff63da90640 1 -- 172.21.15.67:0/507179361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f106e89540 con 0x55f106e1df60 2024-07-31T14:51:51.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.023+0000 7ff63ca8e640 1 --2- 172.21.15.67:0/507179361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106dfd280 0x55f1071cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:51.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.024+0000 7ff63da90640 1 --2- 172.21.15.67:0/507179361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106e1df60 0x55f1071dae40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff63000fe80 tx=0x7ff630004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94f17a629fa26f7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:51.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.024+0000 7ff620ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:51:51.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.024+0000 7ff620ff9640 1 -- 172.21.15.67:0/507179361 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63000ec30 con 0x55f106e1df60 2024-07-31T14:51:51.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.024+0000 7ff620ff9640 1 -- 172.21.15.67:0/507179361 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff63000edd0 con 0x55f106e1df60 2024-07-31T14:51:51.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.024+0000 7ff645a44740 1 -- 172.21.15.67:0/507179361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff628004030 con 0x55f106e1df60 2024-07-31T14:51:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.024+0000 7ff620ff9640 1 -- 172.21.15.67:0/507179361 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff630026430 con 0x55f106e1df60 2024-07-31T14:51:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.024+0000 7ff645a44740 10 client.15729 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.024+0000 7ff645a44740 1 -- 172.21.15.67:0/507179361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f1071ddb80 con 0x55f106e1df60 2024-07-31T14:51:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.025+0000 7ff620ff9640 1 -- 172.21.15.67:0/507179361 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff628004030 con 0x55f106e1df60 2024-07-31T14:51:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.025+0000 7ff620ff9640 1 -- 172.21.15.67:0/507179361 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff63001c710 con 0x55f106e1df60 2024-07-31T14:51:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.025+0000 7ff645a44740 10 client.15729 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.025+0000 7ff645a44740 10 client.15729 resolve_mds: resolved name 'f' to daemon mds.f 2024-07-31T14:51:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.025+0000 7ff645a44740 20 client.15729 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.025+0000 7ff645a44740 1 --2- 172.21.15.67:0/507179361 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f106ddc8e0 0x55f1071e16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.025+0000 7ff645a44740 4 client.15729 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:51:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.025+0000 7ff645a44740 1 -- 172.21.15.67:0/507179361 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f10713b890 con 0x55f106ddc8e0 2024-07-31T14:51:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.025+0000 7ff63d28f640 1 --2- 172.21.15.67:0/507179361 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f106ddc8e0 0x55f1071e16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.026+0000 7ff620ff9640 10 client.15729 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:51:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.026+0000 7ff63d28f640 1 --2- 172.21.15.67:0/507179361 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f106ddc8e0 0x55f1071e16c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=53e4159376fa13c9 server_cookie=87d46baaf681f74d in_seq=0 out_seq=0 2024-07-31T14:51:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff620ff9640 1 -- 172.21.15.67:0/507179361 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f10713b890 con 0x55f106ddc8e0 2024-07-31T14:51:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff620ff9640 10 client.15729 handle_command_reply: tid=0 2024-07-31T14:51:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff645a44740 1 client.15729 shutdown 2024-07-31T14:51:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff645a44740 1 -- 172.21.15.67:0/507179361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106e1df60 msgr2=0x55f1071dae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff645a44740 1 --2- 172.21.15.67:0/507179361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106e1df60 0x55f1071dae40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff63000fe80 tx=0x7ff630004a40 comp rx=0 tx=0).stop 2024-07-31T14:51:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff645a44740 1 -- 172.21.15.67:0/507179361 shutdown_connections 2024-07-31T14:51:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff645a44740 1 --2- 172.21.15.67:0/507179361 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f106ddc8e0 0x55f1071e16c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff620ff9640 0 client.15729 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:51:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff645a44740 1 --2- 172.21.15.67:0/507179361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106e1df60 0x55f1071dae40 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff645a44740 1 --2- 172.21.15.67:0/507179361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106dfd280 0x55f1071cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff645a44740 1 --2- 172.21.15.67:0/507179361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106df50a0 0x55f1071cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff645a44740 1 -- 172.21.15.67:0/507179361 >> 172.21.15.67:0/507179361 conn(0x55f106e12350 msgr2=0x55f1071c6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:51.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff645a44740 1 -- 172.21.15.67:0/507179361 shutdown_connections 2024-07-31T14:51:51.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff645a44740 1 -- 172.21.15.67:0/507179361 wait complete. 2024-07-31T14:51:51.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.027+0000 7ff645a44740 20 client.15729 trim_cache size 0 max 16384 2024-07-31T14:51:51.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.046+0000 7ff645a44740 1 Processor -- start 2024-07-31T14:51:51.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.046+0000 7ff645a44740 1 -- start start 2024-07-31T14:51:51.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.046+0000 7ff645a44740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106ddc8e0 0x55f1071e16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.046+0000 7ff645a44740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106df50a0 0x55f1071dadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.046+0000 7ff645a44740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106dfd280 0x55f1071db2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.046+0000 7ff645a44740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f1071dfa00 con 0x55f106ddc8e0 2024-07-31T14:51:51.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.046+0000 7ff645a44740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f1071e00d0 con 0x55f106dfd280 2024-07-31T14:51:51.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.046+0000 7ff645a44740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f1071e07a0 con 0x55f106df50a0 2024-07-31T14:51:51.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.046+0000 7ff63d28f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106ddc8e0 0x55f1071e16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.046+0000 7ff63d28f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106ddc8e0 0x55f1071e16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52592/0 (socket says 172.21.15.67:52592) 2024-07-31T14:51:51.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.046+0000 7ff63d28f640 1 -- 172.21.15.67:0/1409193556 learned_addr learned my addr 172.21.15.67:0/1409193556 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:51.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.047+0000 7ff63da90640 1 --2- 172.21.15.67:0/1409193556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106dfd280 0x55f1071db2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.047+0000 7ff63ca8e640 1 --2- 172.21.15.67:0/1409193556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106df50a0 0x55f1071dadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.047+0000 7ff63d28f640 1 -- 172.21.15.67:0/1409193556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106df50a0 msgr2=0x55f1071dadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:51.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.047+0000 7ff63d28f640 1 --2- 172.21.15.67:0/1409193556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106df50a0 0x55f1071dadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.047+0000 7ff63d28f640 1 -- 172.21.15.67:0/1409193556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106dfd280 msgr2=0x55f1071db2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:51.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.047+0000 7ff63d28f640 1 --2- 172.21.15.67:0/1409193556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106dfd280 0x55f1071db2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.047+0000 7ff63d28f640 1 -- 172.21.15.67:0/1409193556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1071cf890 con 0x55f106ddc8e0 2024-07-31T14:51:51.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.047+0000 7ff63d28f640 1 --2- 172.21.15.67:0/1409193556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106ddc8e0 0x55f1071e16c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff6340a69b0 tx=0x7ff6340e64e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2718e9230cbb706 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:51.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.047+0000 7ff622ffd640 1 -- 172.21.15.67:0/1409193556 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63401d040 con 0x55f106ddc8e0 2024-07-31T14:51:51.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.047+0000 7ff622ffd640 1 -- 172.21.15.67:0/1409193556 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6340aa440 con 0x55f106ddc8e0 2024-07-31T14:51:51.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.047+0000 7ff622ffd640 1 -- 172.21.15.67:0/1409193556 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634004740 con 0x55f106ddc8e0 2024-07-31T14:51:51.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.048+0000 7ff645a44740 1 -- 172.21.15.67:0/1409193556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106ddc8e0 msgr2=0x55f1071e16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:51.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.048+0000 7ff645a44740 1 --2- 172.21.15.67:0/1409193556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106ddc8e0 0x55f1071e16c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff6340a69b0 tx=0x7ff6340e64e0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.048+0000 7ff645a44740 1 -- 172.21.15.67:0/1409193556 shutdown_connections 2024-07-31T14:51:51.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.048+0000 7ff645a44740 1 --2- 172.21.15.67:0/1409193556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106dfd280 0x55f1071db2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.048+0000 7ff645a44740 1 --2- 172.21.15.67:0/1409193556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106df50a0 0x55f1071dadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.048+0000 7ff645a44740 1 --2- 172.21.15.67:0/1409193556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106ddc8e0 0x55f1071e16c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.048+0000 7ff645a44740 1 -- 172.21.15.67:0/1409193556 >> 172.21.15.67:0/1409193556 conn(0x55f106e12350 msgr2=0x55f1071cd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:51.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.048+0000 7ff645a44740 1 -- 172.21.15.67:0/1409193556 shutdown_connections 2024-07-31T14:51:51.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.048+0000 7ff645a44740 1 -- 172.21.15.67:0/1409193556 wait complete. 2024-07-31T14:51:51.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.048+0000 7ff645a44740 1 -- start start 2024-07-31T14:51:51.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.049+0000 7ff645a44740 1 Processor -- start 2024-07-31T14:51:51.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.049+0000 7ff645a44740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106ddc8e0 0x55f1071e8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.049+0000 7ff645a44740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106df50a0 0x55f1071e8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.049+0000 7ff645a44740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106dfd280 0x55f1071e2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.049+0000 7ff645a44740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f1071e1540 con 0x55f106df50a0 2024-07-31T14:51:51.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.049+0000 7ff645a44740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f1071e0e70 con 0x55f106dfd280 2024-07-31T14:51:51.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.049+0000 7ff645a44740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f1071e1c10 con 0x55f106ddc8e0 2024-07-31T14:51:51.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.049+0000 7ff63ca8e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106df50a0 0x55f1071e8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.049+0000 7ff645a44740 10 client.0 _ll_register_callbacks cb 0x7ff63000f820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:51.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.049+0000 7ff63ca8e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106df50a0 0x55f1071e8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52608/0 (socket says 172.21.15.67:52608) 2024-07-31T14:51:51.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.049+0000 7ff63ca8e640 1 -- 172.21.15.67:0/3314642739 learned_addr learned my addr 172.21.15.67:0/3314642739 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:51.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.049+0000 7ff63d28f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106ddc8e0 0x55f1071e8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.050+0000 7ff63da90640 1 --2- 172.21.15.67:0/3314642739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106dfd280 0x55f1071e2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.050+0000 7ff63ca8e640 1 -- 172.21.15.67:0/3314642739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106ddc8e0 msgr2=0x55f1071e8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:51.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.050+0000 7ff63ca8e640 1 --2- 172.21.15.67:0/3314642739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106ddc8e0 0x55f1071e8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.050+0000 7ff63ca8e640 1 -- 172.21.15.67:0/3314642739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106dfd280 msgr2=0x55f1071e2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:51.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.050+0000 7ff63ca8e640 1 --2- 172.21.15.67:0/3314642739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106dfd280 0x55f1071e2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.050+0000 7ff63ca8e640 1 -- 172.21.15.67:0/3314642739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1071e33f0 con 0x55f106df50a0 2024-07-31T14:51:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.050+0000 7ff63d28f640 1 --2- 172.21.15.67:0/3314642739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106ddc8e0 0x55f1071e8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.050+0000 7ff63ca8e640 1 --2- 172.21.15.67:0/3314642739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106df50a0 0x55f1071e8c70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff628005940 tx=0x7ff628031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30173c1fcaa80e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.050+0000 7ff620ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:51:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.050+0000 7ff620ff9640 1 -- 172.21.15.67:0/3314642739 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628051030 con 0x55f106df50a0 2024-07-31T14:51:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.050+0000 7ff620ff9640 1 -- 172.21.15.67:0/3314642739 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff628049070 con 0x55f106df50a0 2024-07-31T14:51:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.050+0000 7ff645a44740 1 -- 172.21.15.67:0/3314642739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6340aa440 con 0x55f106df50a0 2024-07-31T14:51:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.050+0000 7ff620ff9640 1 -- 172.21.15.67:0/3314642739 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff628035030 con 0x55f106df50a0 2024-07-31T14:51:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.051+0000 7ff645a44740 10 client.15735 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.051+0000 7ff645a44740 1 -- 172.21.15.67:0/3314642739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f1071e9d10 con 0x55f106df50a0 2024-07-31T14:51:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.051+0000 7ff620ff9640 1 -- 172.21.15.67:0/3314642739 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6340aa440 con 0x55f106df50a0 2024-07-31T14:51:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.051+0000 7ff620ff9640 1 -- 172.21.15.67:0/3314642739 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff62803b6e0 con 0x55f106df50a0 2024-07-31T14:51:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.051+0000 7ff645a44740 10 client.15735 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:51.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.051+0000 7ff645a44740 10 client.15735 resolve_mds: resolved name 'f' to daemon mds.f 2024-07-31T14:51:51.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.051+0000 7ff645a44740 20 client.15735 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:51.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.051+0000 7ff645a44740 1 --2- 172.21.15.67:0/3314642739 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f106e1df60 0x55f10724d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:51.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.051+0000 7ff645a44740 4 client.15735 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:51:51.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.051+0000 7ff645a44740 1 -- 172.21.15.67:0/3314642739 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "status"}) -- 0x55f107137330 con 0x55f106e1df60 2024-07-31T14:51:51.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.052+0000 7ff63d28f640 1 --2- 172.21.15.67:0/3314642739 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f106e1df60 0x55f10724d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:51.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.052+0000 7ff620ff9640 10 client.15735 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:51:51.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.052+0000 7ff63d28f640 1 --2- 172.21.15.67:0/3314642739 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f106e1df60 0x55f10724d4e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7ac73c40f16e73f9 server_cookie=d19c8758b084d094 in_seq=0 out_seq=0 2024-07-31T14:51:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.052+0000 7ff620ff9640 1 -- 172.21.15.67:0/3314642739 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f107137330 con 0x55f106e1df60 2024-07-31T14:51:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.052+0000 7ff620ff9640 10 client.15735 handle_command_reply: tid=0 2024-07-31T14:51:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.052+0000 7ff645a44740 1 client.15735 shutdown 2024-07-31T14:51:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.053+0000 7ff645a44740 1 -- 172.21.15.67:0/3314642739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106df50a0 msgr2=0x55f1071e8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.053+0000 7ff645a44740 1 --2- 172.21.15.67:0/3314642739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106df50a0 0x55f1071e8c70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff628005940 tx=0x7ff628031950 comp rx=0 tx=0).stop 2024-07-31T14:51:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.053+0000 7ff645a44740 1 -- 172.21.15.67:0/3314642739 shutdown_connections 2024-07-31T14:51:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.053+0000 7ff645a44740 1 --2- 172.21.15.67:0/3314642739 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f106e1df60 0x55f10724d4e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.053+0000 7ff645a44740 1 --2- 172.21.15.67:0/3314642739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f106dfd280 0x55f1071e2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.053+0000 7ff645a44740 1 --2- 172.21.15.67:0/3314642739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f106df50a0 0x55f1071e8c70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.053+0000 7ff620ff9640 0 client.15735 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:51:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.053+0000 7ff645a44740 1 --2- 172.21.15.67:0/3314642739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f106ddc8e0 0x55f1071e8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.053+0000 7ff645a44740 1 -- 172.21.15.67:0/3314642739 >> 172.21.15.67:0/3314642739 conn(0x55f106e12350 msgr2=0x55f1071c43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.053+0000 7ff645a44740 1 -- 172.21.15.67:0/3314642739 shutdown_connections 2024-07-31T14:51:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.053+0000 7ff645a44740 1 -- 172.21.15.67:0/3314642739 wait complete. 2024-07-31T14:51:51.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.053+0000 7ff645a44740 20 client.15735 trim_cache size 0 max 16384 2024-07-31T14:51:51.060 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:51:51.060 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:51:51.060 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 4, 2024-07-31T14:51:51.060 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24634, 2024-07-31T14:51:51.060 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:51:51.060 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:51:51.060 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:51:51.060 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 228.46414586200001, 2024-07-31T14:51:51.060 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 30, 2024-07-31T14:51:51.061 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:51:51.061 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:51:51.061 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 317.391559454 2024-07-31T14:51:51.061 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:51:51.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.054+0000 7ff63f4f3640 1 -- 172.21.15.67:0/1656941760 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff6240b3df0 msgr2=0x7ff6240b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:51.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.054+0000 7ff63f4f3640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff6240b3df0 0x7ff6240b6300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff630007f90 tx=0x7ff630007720 comp rx=0 tx=0).stop 2024-07-31T14:51:51.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.054+0000 7ff63f4f3640 1 -- 172.21.15.67:0/1656941760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6381024b0 msgr2=0x7ff638199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:51.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.054+0000 7ff63f4f3640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6381024b0 0x7ff638199f10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff63400c970 tx=0x7ff63400ce30 comp rx=0 tx=0).stop 2024-07-31T14:51:51.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.054+0000 7ff63f4f3640 1 -- 172.21.15.67:0/1656941760 shutdown_connections 2024-07-31T14:51:51.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.054+0000 7ff63f4f3640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff6240b3df0 0x7ff6240b6300 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.054+0000 7ff63f4f3640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff638108740 0x7ff63819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.054+0000 7ff63f4f3640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638107df0 0x7ff63819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.054+0000 7ff63f4f3640 1 --2- 172.21.15.67:0/1656941760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6381024b0 0x7ff638199f10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:51.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.054+0000 7ff63f4f3640 1 -- 172.21.15.67:0/1656941760 >> 172.21.15.67:0/1656941760 conn(0x7ff638079170 msgr2=0x7ff63806b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:51.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.054+0000 7ff63f4f3640 1 -- 172.21.15.67:0/1656941760 shutdown_connections 2024-07-31T14:51:51.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:51.055+0000 7ff63f4f3640 1 -- 172.21.15.67:0/1656941760 wait complete. 2024-07-31T14:51:51.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:50 smithi073 ceph-mon[36714]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-07-31T14:51:51.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.c status 2024-07-31T14:51:52.124 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:52.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:52 smithi067 ceph-mon[32750]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-07-31T14:51:52.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:52 smithi077 ceph-mon[37327]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-07-31T14:51:53.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:52 smithi073 ceph-mon[36714]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-07-31T14:51:53.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.865+0000 7f4897ce1640 1 -- 172.21.15.67:0/2775149380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890108740 msgr2=0x7f489010ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:53.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.865+0000 7f4897ce1640 1 --2- 172.21.15.67:0/2775149380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890108740 0x7f489010ef70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4888009ca0 tx=0x7f488802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.866+0000 7f4897ce1640 1 -- 172.21.15.67:0/2775149380 shutdown_connections 2024-07-31T14:51:53.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.866+0000 7f4897ce1640 1 --2- 172.21.15.67:0/2775149380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890108740 0x7f489010ef70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.866+0000 7f4897ce1640 1 --2- 172.21.15.67:0/2775149380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4890107df0 0x7f4890108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.866+0000 7f4897ce1640 1 --2- 172.21.15.67:0/2775149380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48901024b0 0x7f48901028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.866+0000 7f4897ce1640 1 -- 172.21.15.67:0/2775149380 >> 172.21.15.67:0/2775149380 conn(0x7f4890079170 msgr2=0x7f4890079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:53.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.866+0000 7f4897ce1640 1 -- 172.21.15.67:0/2775149380 shutdown_connections 2024-07-31T14:51:53.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.866+0000 7f4897ce1640 1 -- 172.21.15.67:0/2775149380 wait complete. 2024-07-31T14:51:53.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.866+0000 7f4897ce1640 1 Processor -- start 2024-07-31T14:51:53.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f4897ce1640 1 -- start start 2024-07-31T14:51:53.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f4897ce1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48901024b0 0x7f489019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f4897ce1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4890107df0 0x7f489019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f4897ce1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890108740 0x7f489019b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f4897ce1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f48901123c0 con 0x7f4890108740 2024-07-31T14:51:53.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f4897ce1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4890112240 con 0x7f48901024b0 2024-07-31T14:51:53.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f4897ce1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4890112540 con 0x7f4890107df0 2024-07-31T14:51:53.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f489627e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890108740 0x7f489019b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f489627e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890108740 0x7f489019b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52626/0 (socket says 172.21.15.67:52626) 2024-07-31T14:51:53.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f489627e640 1 -- 172.21.15.67:0/4038589468 learned_addr learned my addr 172.21.15.67:0/4038589468 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:53.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f4895a7d640 1 --2- 172.21.15.67:0/4038589468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48901024b0 0x7f489019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f489527c640 1 --2- 172.21.15.67:0/4038589468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4890107df0 0x7f489019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f489627e640 1 -- 172.21.15.67:0/4038589468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4890107df0 msgr2=0x7f489019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:53.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f489627e640 1 --2- 172.21.15.67:0/4038589468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4890107df0 0x7f489019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f489627e640 1 -- 172.21.15.67:0/4038589468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48901024b0 msgr2=0x7f489019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:53.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f489627e640 1 --2- 172.21.15.67:0/4038589468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48901024b0 0x7f489019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.867+0000 7f489627e640 1 -- 172.21.15.67:0/4038589468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48901a5fe0 con 0x7f4890108740 2024-07-31T14:51:53.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.868+0000 7f489627e640 1 --2- 172.21.15.67:0/4038589468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890108740 0x7f489019b7d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f48880098d0 tx=0x7f48880317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:53.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.868+0000 7f4886ffd640 1 -- 172.21.15.67:0/4038589468 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888003b70 con 0x7f4890108740 2024-07-31T14:51:53.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.868+0000 7f4886ffd640 1 -- 172.21.15.67:0/4038589468 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4888003d10 con 0x7f4890108740 2024-07-31T14:51:53.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.868+0000 7f4897ce1640 1 -- 172.21.15.67:0/4038589468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48901a6270 con 0x7f4890108740 2024-07-31T14:51:53.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.868+0000 7f4886ffd640 1 -- 172.21.15.67:0/4038589468 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488802fa20 con 0x7f4890108740 2024-07-31T14:51:53.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.868+0000 7f4897ce1640 1 -- 172.21.15.67:0/4038589468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48901a67b0 con 0x7f4890108740 2024-07-31T14:51:53.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.870+0000 7f489e232740 1 Processor -- start 2024-07-31T14:51:53.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.870+0000 7f4886ffd640 1 -- 172.21.15.67:0/4038589468 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f488802fbc0 con 0x7f4890108740 2024-07-31T14:51:53.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.871+0000 7f489e232740 1 -- start start 2024-07-31T14:51:53.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.871+0000 7f489e232740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 0x55a22246c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.871+0000 7f489e232740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a22209e0a0 0x55a22246c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.871+0000 7f489e232740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220a6280 0x55a22246ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.871+0000 7f489e232740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a2223e2940 con 0x55a22209e0a0 2024-07-31T14:51:53.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.871+0000 7f489e232740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a2223d1e40 con 0x55a2220c6f60 2024-07-31T14:51:53.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.871+0000 7f489e232740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a2223d5e80 con 0x55a2220a6280 2024-07-31T14:51:53.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.871+0000 7f4895a7d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a22209e0a0 0x55a22246c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.871+0000 7f4895a7d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a22209e0a0 0x55a22246c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52636/0 (socket says 172.21.15.67:52636) 2024-07-31T14:51:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.871+0000 7f4895a7d640 1 -- 172.21.15.67:0/711613778 learned_addr learned my addr 172.21.15.67:0/711613778 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f489527c640 1 --2- 172.21.15.67:0/711613778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220a6280 0x55a22246ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f489527c640 1 --2- 172.21.15.67:0/711613778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 0x55a22246c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f4886ffd640 1 --2- 172.21.15.67:0/4038589468 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f487c0b3d70 0x7f487c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f4895a7d640 1 -- 172.21.15.67:0/711613778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220a6280 msgr2=0x55a22246ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f4895a7d640 1 --2- 172.21.15.67:0/711613778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220a6280 0x55a22246ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f4895a7d640 1 -- 172.21.15.67:0/711613778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 msgr2=0x55a22246c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f4895a7d640 1 --2- 172.21.15.67:0/711613778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 0x55a22246c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f4895a7d640 1 -- 172.21.15.67:0/711613778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a222378a60 con 0x55a22209e0a0 2024-07-31T14:51:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f489527c640 1 --2- 172.21.15.67:0/711613778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 0x55a22246c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:51:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f489627e640 1 --2- 172.21.15.67:0/4038589468 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f487c0b3d70 0x7f487c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f4895a7d640 1 --2- 172.21.15.67:0/711613778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a22209e0a0 0x55a22246c8e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f488c00bcb0 tx=0x7f488c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916b506b732ea248 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:53.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f4886ffd640 1 -- 172.21.15.67:0/4038589468 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f4888105810 con 0x7f4890108740 2024-07-31T14:51:53.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f487bfff640 1 -- 172.21.15.67:0/711613778 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c01c480 con 0x55a22209e0a0 2024-07-31T14:51:53.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f487bfff640 1 -- 172.21.15.67:0/711613778 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488c01cad0 con 0x55a22209e0a0 2024-07-31T14:51:53.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.872+0000 7f487bfff640 1 -- 172.21.15.67:0/711613778 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c011c30 con 0x55a22209e0a0 2024-07-31T14:51:53.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.873+0000 7f489e232740 1 -- 172.21.15.67:0/711613778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a22209e0a0 msgr2=0x55a22246c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:53.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.873+0000 7f489e232740 1 --2- 172.21.15.67:0/711613778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a22209e0a0 0x55a22246c8e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f488c00bcb0 tx=0x7f488c0107a0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.873+0000 7f489627e640 1 --2- 172.21.15.67:0/4038589468 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f487c0b3d70 0x7f487c0b6280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4888102e10 tx=0x7f4888091590 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.873+0000 7f489e232740 1 -- 172.21.15.67:0/711613778 shutdown_connections 2024-07-31T14:51:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.873+0000 7f489e232740 1 --2- 172.21.15.67:0/711613778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220a6280 0x55a22246ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.873+0000 7f489e232740 1 --2- 172.21.15.67:0/711613778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a22209e0a0 0x55a22246c8e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.873+0000 7f489e232740 1 --2- 172.21.15.67:0/711613778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 0x55a22246c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.873+0000 7f489e232740 1 -- 172.21.15.67:0/711613778 >> 172.21.15.67:0/711613778 conn(0x55a2220bb350 msgr2=0x55a22246a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.873+0000 7f489e232740 1 -- 172.21.15.67:0/711613778 shutdown_connections 2024-07-31T14:51:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.874+0000 7f489e232740 1 -- 172.21.15.67:0/711613778 wait complete. 2024-07-31T14:51:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.874+0000 7f489e232740 1 -- start start 2024-07-31T14:51:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.874+0000 7f489e232740 1 Processor -- start 2024-07-31T14:51:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.874+0000 7f489e232740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a22209e0a0 0x55a222477f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.874+0000 7f489e232740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220a6280 0x55a2224784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.874+0000 7f489e232740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 0x55a222483e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.874+0000 7f489e232740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a2222355f0 con 0x55a22209e0a0 2024-07-31T14:51:53.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.874+0000 7f489e232740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a2223748b0 con 0x55a2220c6f60 2024-07-31T14:51:53.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.874+0000 7f489e232740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a22247ab80 con 0x55a2220a6280 2024-07-31T14:51:53.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f489e232740 10 client.0 _ll_register_callbacks cb 0x55a22237e7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:53.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f489527c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a22209e0a0 0x55a222477f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f489527c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a22209e0a0 0x55a222477f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52650/0 (socket says 172.21.15.67:52650) 2024-07-31T14:51:53.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f489527c640 1 -- 172.21.15.67:0/914055678 learned_addr learned my addr 172.21.15.67:0/914055678 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:53.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f4895a7d640 1 --2- 172.21.15.67:0/914055678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220a6280 0x55a2224784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f489527c640 1 --2- 172.21.15.67:0/914055678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 0x55a222483e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f489527c640 1 -- 172.21.15.67:0/914055678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220a6280 msgr2=0x55a2224784d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f489527c640 1 --2- 172.21.15.67:0/914055678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220a6280 0x55a2224784d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f489527c640 1 -- 172.21.15.67:0/914055678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 msgr2=0x55a222483e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f489527c640 1 --2- 172.21.15.67:0/914055678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 0x55a222483e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f489527c640 1 -- 172.21.15.67:0/914055678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a222132540 con 0x55a22209e0a0 2024-07-31T14:51:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f489527c640 1 --2- 172.21.15.67:0/914055678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a22209e0a0 0x55a222477f90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4880009df0 tx=0x7f4880005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3d750e30aba8f5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f48797fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:51:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f48797fa640 1 -- 172.21.15.67:0/914055678 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880004460 con 0x55a22209e0a0 2024-07-31T14:51:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f48797fa640 1 -- 172.21.15.67:0/914055678 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f488000f980 con 0x55a22209e0a0 2024-07-31T14:51:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.875+0000 7f489e232740 1 -- 172.21.15.67:0/914055678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f488c01cad0 con 0x55a22209e0a0 2024-07-31T14:51:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.876+0000 7f48797fa640 1 -- 172.21.15.67:0/914055678 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4880027050 con 0x55a22209e0a0 2024-07-31T14:51:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.876+0000 7f489e232740 10 client.15747 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.876+0000 7f489e232740 1 -- 172.21.15.67:0/914055678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a222486b30 con 0x55a22209e0a0 2024-07-31T14:51:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.876+0000 7f48797fa640 1 -- 172.21.15.67:0/914055678 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488c01cad0 con 0x55a22209e0a0 2024-07-31T14:51:53.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.876+0000 7f48797fa640 1 -- 172.21.15.67:0/914055678 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f488001f3f0 con 0x55a22209e0a0 2024-07-31T14:51:53.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.876+0000 7f489e232740 10 client.15747 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:53.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.876+0000 7f489e232740 10 client.15747 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-31T14:51:53.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.876+0000 7f489e232740 20 client.15747 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:53.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.876+0000 7f489e232740 1 --2- 172.21.15.67:0/914055678 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a2220858e0 0x55a22248a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.876+0000 7f489e232740 4 client.15747 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:51:53.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.876+0000 7f489e232740 1 -- 172.21.15.67:0/914055678 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2223e4890 con 0x55a2220858e0 2024-07-31T14:51:53.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.877+0000 7f4895a7d640 1 --2- 172.21.15.67:0/914055678 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a2220858e0 0x55a22248a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.877+0000 7f48797fa640 10 client.15747 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:51:53.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.877+0000 7f4895a7d640 1 --2- 172.21.15.67:0/914055678 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a2220858e0 0x55a22248a5a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ee51047dd8bb8d03 server_cookie=bc04a498b994ad47 in_seq=0 out_seq=0 2024-07-31T14:51:53.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f48797fa640 1 -- 172.21.15.67:0/914055678 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2223e4890 con 0x55a2220858e0 2024-07-31T14:51:53.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f48797fa640 10 client.15747 handle_command_reply: tid=0 2024-07-31T14:51:53.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f489e232740 1 client.15747 shutdown 2024-07-31T14:51:53.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f489e232740 1 -- 172.21.15.67:0/914055678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a22209e0a0 msgr2=0x55a222477f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:53.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f489e232740 1 --2- 172.21.15.67:0/914055678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a22209e0a0 0x55a222477f90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4880009df0 tx=0x7f4880005f90 comp rx=0 tx=0).stop 2024-07-31T14:51:53.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f489e232740 1 -- 172.21.15.67:0/914055678 shutdown_connections 2024-07-31T14:51:53.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f489e232740 1 --2- 172.21.15.67:0/914055678 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a2220858e0 0x55a22248a5a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f489e232740 1 --2- 172.21.15.67:0/914055678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 0x55a222483e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f489e232740 1 --2- 172.21.15.67:0/914055678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220a6280 0x55a2224784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f489e232740 1 --2- 172.21.15.67:0/914055678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a22209e0a0 0x55a222477f90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f48797fa640 0 client.15747 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:51:53.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f489e232740 1 -- 172.21.15.67:0/914055678 >> 172.21.15.67:0/914055678 conn(0x55a2220bb350 msgr2=0x55a22246f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:53.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f489e232740 1 -- 172.21.15.67:0/914055678 shutdown_connections 2024-07-31T14:51:53.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f489e232740 1 -- 172.21.15.67:0/914055678 wait complete. 2024-07-31T14:51:53.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.878+0000 7f489e232740 20 client.15747 trim_cache size 0 max 16384 2024-07-31T14:51:53.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.897+0000 7f489e232740 1 Processor -- start 2024-07-31T14:51:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.897+0000 7f489e232740 1 -- start start 2024-07-31T14:51:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.897+0000 7f489e232740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a22209e0a0 0x55a222483de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.897+0000 7f489e232740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2220a6280 0x55a222471620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.897+0000 7f489e232740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 0x55a222471b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.897+0000 7f489e232740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a2224888e0 con 0x55a2220a6280 2024-07-31T14:51:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.897+0000 7f489e232740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a222488fb0 con 0x55a2220c6f60 2024-07-31T14:51:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.897+0000 7f489e232740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a222489680 con 0x55a22209e0a0 2024-07-31T14:51:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f4895a7d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2220a6280 0x55a222471620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f4895a7d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2220a6280 0x55a222471620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52656/0 (socket says 172.21.15.67:52656) 2024-07-31T14:51:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f4895a7d640 1 -- 172.21.15.67:0/896485822 learned_addr learned my addr 172.21.15.67:0/896485822 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f489527c640 1 --2- 172.21.15.67:0/896485822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 0x55a222471b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f489527c640 1 --2- 172.21.15.67:0/896485822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a22209e0a0 0x55a222483de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f4895a7d640 1 -- 172.21.15.67:0/896485822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a22209e0a0 msgr2=0x55a222483de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f4895a7d640 1 --2- 172.21.15.67:0/896485822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a22209e0a0 0x55a222483de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f4895a7d640 1 -- 172.21.15.67:0/896485822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 msgr2=0x55a222471b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f4895a7d640 1 --2- 172.21.15.67:0/896485822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 0x55a222471b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f4895a7d640 1 -- 172.21.15.67:0/896485822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a22246ab80 con 0x55a2220a6280 2024-07-31T14:51:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f489527c640 1 --2- 172.21.15.67:0/896485822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a22209e0a0 0x55a222483de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:51:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f4895a7d640 1 --2- 172.21.15.67:0/896485822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2220a6280 0x55a222471620 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f488c00bcb0 tx=0x7f488c0116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ef1b8e32394170 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f487b7fe640 1 -- 172.21.15.67:0/896485822 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c02c020 con 0x55a2220a6280 2024-07-31T14:51:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f487b7fe640 1 -- 172.21.15.67:0/896485822 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488c0054f0 con 0x55a2220a6280 2024-07-31T14:51:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.898+0000 7f487b7fe640 1 -- 172.21.15.67:0/896485822 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c0035f0 con 0x55a2220a6280 2024-07-31T14:51:53.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.899+0000 7f489e232740 1 -- 172.21.15.67:0/896485822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2220a6280 msgr2=0x55a222471620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.899+0000 7f489e232740 1 --2- 172.21.15.67:0/896485822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2220a6280 0x55a222471620 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f488c00bcb0 tx=0x7f488c0116f0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.899+0000 7f489e232740 1 -- 172.21.15.67:0/896485822 shutdown_connections 2024-07-31T14:51:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.899+0000 7f489e232740 1 --2- 172.21.15.67:0/896485822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2220c6f60 0x55a222471b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.899+0000 7f489e232740 1 --2- 172.21.15.67:0/896485822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2220a6280 0x55a222471620 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.899+0000 7f489e232740 1 --2- 172.21.15.67:0/896485822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a22209e0a0 0x55a222483de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.899+0000 7f489e232740 1 -- 172.21.15.67:0/896485822 >> 172.21.15.67:0/896485822 conn(0x55a2220bb350 msgr2=0x55a222472210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.899+0000 7f489e232740 1 -- 172.21.15.67:0/896485822 shutdown_connections 2024-07-31T14:51:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.899+0000 7f489e232740 1 -- 172.21.15.67:0/896485822 wait complete. 2024-07-31T14:51:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.899+0000 7f489e232740 1 -- start start 2024-07-31T14:51:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.900+0000 7f489e232740 1 Processor -- start 2024-07-31T14:51:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.900+0000 7f489e232740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22209e0a0 0x55a222469040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.900+0000 7f489e232740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2220a6280 0x55a222469620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.900+0000 7f489e232740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220c6f60 0x55a222492820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.900+0000 7f489e232740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a22248a420 con 0x55a2220a6280 2024-07-31T14:51:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.900+0000 7f489e232740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a222489d50 con 0x55a22209e0a0 2024-07-31T14:51:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.900+0000 7f489e232740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a22248aaf0 con 0x55a2220c6f60 2024-07-31T14:51:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.900+0000 7f489e232740 10 client.0 _ll_register_callbacks cb 0x7f4880010970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.900+0000 7f4895a7d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2220a6280 0x55a222469620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.900+0000 7f4895a7d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2220a6280 0x55a222469620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52662/0 (socket says 172.21.15.67:52662) 2024-07-31T14:51:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.900+0000 7f4895a7d640 1 -- 172.21.15.67:0/3651974622 learned_addr learned my addr 172.21.15.67:0/3651974622 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.900+0000 7f489527c640 1 --2- 172.21.15.67:0/3651974622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22209e0a0 0x55a222469040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.900+0000 7f489527c640 1 --2- 172.21.15.67:0/3651974622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220c6f60 0x55a222492820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f4895a7d640 1 -- 172.21.15.67:0/3651974622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220c6f60 msgr2=0x55a222492820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f4895a7d640 1 --2- 172.21.15.67:0/3651974622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220c6f60 0x55a222492820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f4895a7d640 1 -- 172.21.15.67:0/3651974622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22209e0a0 msgr2=0x55a222469040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f4895a7d640 1 --2- 172.21.15.67:0/3651974622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22209e0a0 0x55a222469040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f4895a7d640 1 -- 172.21.15.67:0/3651974622 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a222492ec0 con 0x55a2220a6280 2024-07-31T14:51:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f489527c640 1 --2- 172.21.15.67:0/3651974622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220c6f60 0x55a222492820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:51:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f4895a7d640 1 --2- 172.21.15.67:0/3651974622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2220a6280 0x55a222469620 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f488c011010 tx=0x7f488c0114c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67a7dfc0de3baec server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f48797fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:51:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f48797fa640 1 -- 172.21.15.67:0/3651974622 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c02c030 con 0x55a2220a6280 2024-07-31T14:51:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f489e232740 1 -- 172.21.15.67:0/3651974622 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f488c0054f0 con 0x55a2220a6280 2024-07-31T14:51:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f48797fa640 1 -- 172.21.15.67:0/3651974622 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f488c011760 con 0x55a2220a6280 2024-07-31T14:51:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f48797fa640 1 -- 172.21.15.67:0/3651974622 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f488c03f050 con 0x55a2220a6280 2024-07-31T14:51:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f489e232740 10 client.15753 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f489e232740 1 -- 172.21.15.67:0/3651974622 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a2224901d0 con 0x55a2220a6280 2024-07-31T14:51:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.901+0000 7f48797fa640 1 -- 172.21.15.67:0/3651974622 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488c0054f0 con 0x55a2220a6280 2024-07-31T14:51:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.902+0000 7f48797fa640 1 -- 172.21.15.67:0/3651974622 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f488c003c00 con 0x55a2220a6280 2024-07-31T14:51:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.902+0000 7f489e232740 10 client.15753 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.902+0000 7f489e232740 10 client.15753 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-31T14:51:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.902+0000 7f489e232740 20 client.15753 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.902+0000 7f489e232740 1 --2- 172.21.15.67:0/3651974622 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a22205ac90 0x55a2224f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.902+0000 7f489e232740 4 client.15753 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:51:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.902+0000 7f489e232740 1 -- 172.21.15.67:0/3651974622 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "status"}) -- 0x55a2223e0330 con 0x55a22205ac90 2024-07-31T14:51:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.902+0000 7f489527c640 1 --2- 172.21.15.67:0/3651974622 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a22205ac90 0x55a2224f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.902+0000 7f489527c640 1 --2- 172.21.15.67:0/3651974622 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a22205ac90 0x55a2224f63e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52c023e63deaaf9c server_cookie=58d54d1154199d47 in_seq=0 out_seq=0 2024-07-31T14:51:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.902+0000 7f48797fa640 10 client.15753 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:51:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f48797fa640 1 -- 172.21.15.67:0/3651974622 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a2223e0330 con 0x55a22205ac90 2024-07-31T14:51:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f48797fa640 10 client.15753 handle_command_reply: tid=0 2024-07-31T14:51:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f489e232740 1 client.15753 shutdown 2024-07-31T14:51:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f489e232740 1 -- 172.21.15.67:0/3651974622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2220a6280 msgr2=0x55a222469620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f489e232740 1 --2- 172.21.15.67:0/3651974622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2220a6280 0x55a222469620 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f488c011010 tx=0x7f488c0114c0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f489e232740 1 -- 172.21.15.67:0/3651974622 shutdown_connections 2024-07-31T14:51:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f489e232740 1 --2- 172.21.15.67:0/3651974622 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55a22205ac90 0x55a2224f63e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f489e232740 1 --2- 172.21.15.67:0/3651974622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a2220c6f60 0x55a222492820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f489e232740 1 --2- 172.21.15.67:0/3651974622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2220a6280 0x55a222469620 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f48797fa640 0 client.15753 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:51:53.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f489e232740 1 --2- 172.21.15.67:0/3651974622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22209e0a0 0x55a222469040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f489e232740 1 -- 172.21.15.67:0/3651974622 >> 172.21.15.67:0/3651974622 conn(0x55a2220bb350 msgr2=0x55a222472f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:53.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f489e232740 1 -- 172.21.15.67:0/3651974622 shutdown_connections 2024-07-31T14:51:53.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f489e232740 1 -- 172.21.15.67:0/3651974622 wait complete. 2024-07-31T14:51:53.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.903+0000 7f489e232740 20 client.15753 trim_cache size 0 max 16384 2024-07-31T14:51:53.909 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:51:53.909 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:51:53.910 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 3, 2024-07-31T14:51:53.910 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24646, 2024-07-31T14:51:53.910 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:51:53.910 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:51:53.910 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:51:53.910 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 232.32423061899999, 2024-07-31T14:51:53.910 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 30, 2024-07-31T14:51:53.910 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:51:53.910 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:51:53.910 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 317.65525608299998 2024-07-31T14:51:53.910 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:51:53.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.904+0000 7f4897ce1640 1 -- 172.21.15.67:0/4038589468 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f487c0b3d70 msgr2=0x7f487c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:53.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.904+0000 7f4897ce1640 1 --2- 172.21.15.67:0/4038589468 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f487c0b3d70 0x7f487c0b6280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4888102e10 tx=0x7f4888091590 comp rx=0 tx=0).stop 2024-07-31T14:51:53.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.904+0000 7f4897ce1640 1 -- 172.21.15.67:0/4038589468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890108740 msgr2=0x7f489019b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:53.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.904+0000 7f4897ce1640 1 --2- 172.21.15.67:0/4038589468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890108740 0x7f489019b7d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f48880098d0 tx=0x7f48880317b0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.904+0000 7f4897ce1640 1 -- 172.21.15.67:0/4038589468 shutdown_connections 2024-07-31T14:51:53.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.904+0000 7f4897ce1640 1 --2- 172.21.15.67:0/4038589468 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f487c0b3d70 0x7f487c0b6280 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.904+0000 7f4897ce1640 1 --2- 172.21.15.67:0/4038589468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890108740 0x7f489019b7d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.904+0000 7f4897ce1640 1 --2- 172.21.15.67:0/4038589468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4890107df0 0x7f489019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.904+0000 7f4897ce1640 1 --2- 172.21.15.67:0/4038589468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48901024b0 0x7f489019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:53.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.904+0000 7f4897ce1640 1 -- 172.21.15.67:0/4038589468 >> 172.21.15.67:0/4038589468 conn(0x7f4890079170 msgr2=0x7f4890104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:53.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.904+0000 7f4897ce1640 1 -- 172.21.15.67:0/4038589468 shutdown_connections 2024-07-31T14:51:53.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:53.904+0000 7f4897ce1640 1 -- 172.21.15.67:0/4038589468 wait complete. 2024-07-31T14:51:54.590 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.h status 2024-07-31T14:51:54.858 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:54.980 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:54 smithi067 ceph-mon[32750]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-07-31T14:51:54.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:54 smithi077 ceph-mon[37327]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-07-31T14:51:55.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:54 smithi073 ceph-mon[36714]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-07-31T14:51:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.615+0000 7f9b8133f640 1 -- 172.21.15.67:0/413579171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b7c107dd0 msgr2=0x7f9b7c1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.615+0000 7f9b8133f640 1 --2- 172.21.15.67:0/413579171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b7c107dd0 0x7f9b7c1081e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009900 tx=0x7f9b6402a410 comp rx=0 tx=0).stop 2024-07-31T14:51:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.615+0000 7f9b8133f640 1 -- 172.21.15.67:0/413579171 shutdown_connections 2024-07-31T14:51:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.615+0000 7f9b8133f640 1 --2- 172.21.15.67:0/413579171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b7c108720 0x7f9b7c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.615+0000 7f9b8133f640 1 --2- 172.21.15.67:0/413579171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b7c107dd0 0x7f9b7c1081e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.615+0000 7f9b8133f640 1 --2- 172.21.15.67:0/413579171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b7c102490 0x7f9b7c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.615+0000 7f9b8133f640 1 -- 172.21.15.67:0/413579171 >> 172.21.15.67:0/413579171 conn(0x7f9b7c079160 msgr2=0x7f9b7c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.615+0000 7f9b8133f640 1 -- 172.21.15.67:0/413579171 shutdown_connections 2024-07-31T14:51:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.615+0000 7f9b8133f640 1 -- 172.21.15.67:0/413579171 wait complete. 2024-07-31T14:51:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.616+0000 7f9b8133f640 1 Processor -- start 2024-07-31T14:51:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.616+0000 7f9b8133f640 1 -- start start 2024-07-31T14:51:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.616+0000 7f9b8133f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b7c102490 0x7f9b7c10f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.616+0000 7f9b8133f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b7c107dd0 0x7f9b7c10ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.616+0000 7f9b8133f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b7c108720 0x7f9b7c110460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.616+0000 7f9b8133f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c1134b0 con 0x7f9b7c108720 2024-07-31T14:51:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.616+0000 7f9b8133f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c113330 con 0x7f9b7c102490 2024-07-31T14:51:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.616+0000 7f9b8133f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c113630 con 0x7f9b7c107dd0 2024-07-31T14:51:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.616+0000 7f9b7b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b7c108720 0x7f9b7c110460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.616+0000 7f9b7b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b7c108720 0x7f9b7c110460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52670/0 (socket says 172.21.15.67:52670) 2024-07-31T14:51:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.616+0000 7f9b7b7fe640 1 -- 172.21.15.67:0/2674474199 learned_addr learned my addr 172.21.15.67:0/2674474199 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.617+0000 7f9b7a7fc640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b7c107dd0 0x7f9b7c10ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.617+0000 7f9b7affd640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b7c102490 0x7f9b7c10f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.617+0000 7f9b7b7fe640 1 -- 172.21.15.67:0/2674474199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b7c107dd0 msgr2=0x7f9b7c10ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.617+0000 7f9b7b7fe640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b7c107dd0 0x7f9b7c10ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.617+0000 7f9b7b7fe640 1 -- 172.21.15.67:0/2674474199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b7c102490 msgr2=0x7f9b7c10f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.617+0000 7f9b7b7fe640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b7c102490 0x7f9b7c10f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.617+0000 7f9b7b7fe640 1 -- 172.21.15.67:0/2674474199 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c1a80b0 con 0x7f9b7c108720 2024-07-31T14:51:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.617+0000 7f9b7affd640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b7c102490 0x7f9b7c10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.617+0000 7f9b7b7fe640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b7c108720 0x7f9b7c110460 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00eaa0 tx=0x7f9b6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.617+0000 7f9b73fff640 1 -- 172.21.15.67:0/2674474199 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c00cdb0 con 0x7f9b7c108720 2024-07-31T14:51:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.617+0000 7f9b8133f640 1 -- 172.21.15.67:0/2674474199 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7c1a83a0 con 0x7f9b7c108720 2024-07-31T14:51:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.617+0000 7f9b73fff640 1 -- 172.21.15.67:0/2674474199 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b6c005020 con 0x7f9b7c108720 2024-07-31T14:51:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.617+0000 7f9b73fff640 1 -- 172.21.15.67:0/2674474199 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c0116d0 con 0x7f9b7c108720 2024-07-31T14:51:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.617+0000 7f9b8133f640 1 -- 172.21.15.67:0/2674474199 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7c1a88b0 con 0x7f9b7c108720 2024-07-31T14:51:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.620+0000 7f9b87890740 1 Processor -- start 2024-07-31T14:51:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.620+0000 7f9b73fff640 1 -- 172.21.15.67:0/2674474199 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f9b6c011870 con 0x7f9b7c108720 2024-07-31T14:51:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.620+0000 7f9b87890740 1 -- start start 2024-07-31T14:51:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b87890740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed082f60 0x5574ed4283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b87890740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed05a0a0 0x5574ed4288e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b87890740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed062280 0x5574ed428e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b87890740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5574ed39e940 con 0x5574ed05a0a0 2024-07-31T14:51:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b87890740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5574ed38de40 con 0x5574ed082f60 2024-07-31T14:51:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b87890740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5574ed391e80 con 0x5574ed062280 2024-07-31T14:51:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b7affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed05a0a0 0x5574ed4288e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b7affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed05a0a0 0x5574ed4288e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52678/0 (socket says 172.21.15.67:52678) 2024-07-31T14:51:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b7affd640 1 -- 172.21.15.67:0/846150057 learned_addr learned my addr 172.21.15.67:0/846150057 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b7a7fc640 1 --2- 172.21.15.67:0/846150057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed082f60 0x5574ed4283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b7a7fc640 1 --2- 172.21.15.67:0/846150057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed062280 0x5574ed428e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b7affd640 1 -- 172.21.15.67:0/846150057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed062280 msgr2=0x5574ed428e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b7affd640 1 --2- 172.21.15.67:0/846150057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed062280 0x5574ed428e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b7affd640 1 -- 172.21.15.67:0/846150057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed082f60 msgr2=0x5574ed4283a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b7affd640 1 --2- 172.21.15.67:0/846150057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed082f60 0x5574ed4283a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b7affd640 1 -- 172.21.15.67:0/846150057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574ed334a60 con 0x5574ed05a0a0 2024-07-31T14:51:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b7a7fc640 1 --2- 172.21.15.67:0/846150057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed062280 0x5574ed428e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.621+0000 7f9b73fff640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9b680b3d40 0x7f9b680b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.622+0000 7f9b7affd640 1 --2- 172.21.15.67:0/846150057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed05a0a0 0x5574ed4288e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9b74008c30 tx=0x7f9b7400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527699e48709c4bc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.622+0000 7f9b7a7fc640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9b680b3d40 0x7f9b680b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.622+0000 7f9b73fff640 1 -- 172.21.15.67:0/2674474199 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f9b6c0e4400 con 0x7f9b7c108720 2024-07-31T14:51:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.622+0000 7f9b717fa640 1 -- 172.21.15.67:0/846150057 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74014070 con 0x5574ed05a0a0 2024-07-31T14:51:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.622+0000 7f9b717fa640 1 -- 172.21.15.67:0/846150057 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b740047a0 con 0x5574ed05a0a0 2024-07-31T14:51:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.622+0000 7f9b717fa640 1 -- 172.21.15.67:0/846150057 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74010690 con 0x5574ed05a0a0 2024-07-31T14:51:56.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.623+0000 7f9b87890740 1 -- 172.21.15.67:0/846150057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed05a0a0 msgr2=0x5574ed4288e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:56.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.623+0000 7f9b87890740 1 --2- 172.21.15.67:0/846150057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed05a0a0 0x5574ed4288e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9b74008c30 tx=0x7f9b7400e720 comp rx=0 tx=0).stop 2024-07-31T14:51:56.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.623+0000 7f9b7a7fc640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9b680b3d40 0x7f9b680b6250 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9b64000c00 tx=0x7f9b64005780 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.623+0000 7f9b87890740 1 -- 172.21.15.67:0/846150057 shutdown_connections 2024-07-31T14:51:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.623+0000 7f9b87890740 1 --2- 172.21.15.67:0/846150057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed062280 0x5574ed428e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.623+0000 7f9b87890740 1 --2- 172.21.15.67:0/846150057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed05a0a0 0x5574ed4288e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.623+0000 7f9b87890740 1 --2- 172.21.15.67:0/846150057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed082f60 0x5574ed4283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.623+0000 7f9b87890740 1 -- 172.21.15.67:0/846150057 >> 172.21.15.67:0/846150057 conn(0x5574ed077350 msgr2=0x5574ed426290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.623+0000 7f9b87890740 1 -- 172.21.15.67:0/846150057 shutdown_connections 2024-07-31T14:51:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.623+0000 7f9b87890740 1 -- 172.21.15.67:0/846150057 wait complete. 2024-07-31T14:51:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.624+0000 7f9b87890740 1 -- start start 2024-07-31T14:51:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.624+0000 7f9b87890740 1 Processor -- start 2024-07-31T14:51:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.624+0000 7f9b87890740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed05a0a0 0x5574ed433f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.624+0000 7f9b87890740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed062280 0x5574ed4344c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.624+0000 7f9b87890740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed082f60 0x5574ed43fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.624+0000 7f9b87890740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5574ed1f15f0 con 0x5574ed062280 2024-07-31T14:51:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.624+0000 7f9b87890740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5574ed3308b0 con 0x5574ed05a0a0 2024-07-31T14:51:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.624+0000 7f9b87890740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5574ed436b80 con 0x5574ed082f60 2024-07-31T14:51:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.624+0000 7f9b87890740 10 client.0 _ll_register_callbacks cb 0x5574ed33a7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b7a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed062280 0x5574ed4344c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b7a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed062280 0x5574ed4344c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52688/0 (socket says 172.21.15.67:52688) 2024-07-31T14:51:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b7a7fc640 1 -- 172.21.15.67:0/1480100294 learned_addr learned my addr 172.21.15.67:0/1480100294 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b7affd640 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed05a0a0 0x5574ed433f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b7b7fe640 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed082f60 0x5574ed43fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b7affd640 1 -- 172.21.15.67:0/1480100294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed082f60 msgr2=0x5574ed43fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b7affd640 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed082f60 0x5574ed43fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b7affd640 1 -- 172.21.15.67:0/1480100294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed062280 msgr2=0x5574ed4344c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b7affd640 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed062280 0x5574ed4344c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b7affd640 1 -- 172.21.15.67:0/1480100294 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574ed0ee540 con 0x5574ed05a0a0 2024-07-31T14:51:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b7b7fe640 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed082f60 0x5574ed43fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b7a7fc640 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed062280 0x5574ed4344c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b7affd640 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed05a0a0 0x5574ed433f80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b74005e10 tx=0x7f9b740089a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2be13fd6f32db7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b627fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:51:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.625+0000 7f9b627fc640 1 -- 172.21.15.67:0/1480100294 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74014070 con 0x5574ed05a0a0 2024-07-31T14:51:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.626+0000 7f9b627fc640 1 -- 172.21.15.67:0/1480100294 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f9b74004440 con 0x5574ed05a0a0 2024-07-31T14:51:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.626+0000 7f9b87890740 1 -- 172.21.15.67:0/1480100294 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b740047a0 con 0x5574ed05a0a0 2024-07-31T14:51:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.626+0000 7f9b627fc640 1 -- 172.21.15.67:0/1480100294 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b74005000 con 0x5574ed05a0a0 2024-07-31T14:51:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.626+0000 7f9b87890740 10 client.25448 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.626+0000 7f9b87890740 1 -- 172.21.15.67:0/1480100294 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574ed442b90 con 0x5574ed05a0a0 2024-07-31T14:51:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.626+0000 7f9b627fc640 1 -- 172.21.15.67:0/1480100294 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b740047a0 con 0x5574ed05a0a0 2024-07-31T14:51:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.626+0000 7f9b627fc640 1 -- 172.21.15.67:0/1480100294 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9b74010430 con 0x5574ed05a0a0 2024-07-31T14:51:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.626+0000 7f9b87890740 10 client.25448 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.626+0000 7f9b87890740 10 client.25448 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-31T14:51:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.626+0000 7f9b87890740 20 client.25448 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.626+0000 7f9b87890740 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5574ed0418e0 0x5574ed446370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.626+0000 7f9b87890740 4 client.25448 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:51:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.626+0000 7f9b87890740 1 -- 172.21.15.67:0/1480100294 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574ed3a0890 con 0x5574ed0418e0 2024-07-31T14:51:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.627+0000 7f9b7a7fc640 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5574ed0418e0 0x5574ed446370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.627+0000 7f9b627fc640 10 client.25448 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:51:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.627+0000 7f9b7a7fc640 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5574ed0418e0 0x5574ed446370 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6fe3c689e673888 server_cookie=64041637808e8161 in_seq=0 out_seq=0 2024-07-31T14:51:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.628+0000 7f9b627fc640 1 -- 172.21.15.67:0/1480100294 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574ed3a0890 con 0x5574ed0418e0 2024-07-31T14:51:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.628+0000 7f9b627fc640 10 client.25448 handle_command_reply: tid=0 2024-07-31T14:51:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.628+0000 7f9b87890740 1 client.25448 shutdown 2024-07-31T14:51:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.628+0000 7f9b87890740 1 -- 172.21.15.67:0/1480100294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed05a0a0 msgr2=0x5574ed433f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:56.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.628+0000 7f9b87890740 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed05a0a0 0x5574ed433f80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b74005e10 tx=0x7f9b740089a0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.628+0000 7f9b87890740 1 -- 172.21.15.67:0/1480100294 shutdown_connections 2024-07-31T14:51:56.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.628+0000 7f9b87890740 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed082f60 0x5574ed43fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.628+0000 7f9b87890740 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed062280 0x5574ed4344c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.628+0000 7f9b87890740 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5574ed0418e0 0x5574ed446370 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.628+0000 7f9b87890740 1 --2- 172.21.15.67:0/1480100294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed05a0a0 0x5574ed433f80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.628+0000 7f9b627fc640 0 client.25448 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:51:56.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.628+0000 7f9b87890740 1 -- 172.21.15.67:0/1480100294 >> 172.21.15.67:0/1480100294 conn(0x5574ed077350 msgr2=0x5574ed42b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:56.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.628+0000 7f9b87890740 1 -- 172.21.15.67:0/1480100294 shutdown_connections 2024-07-31T14:51:56.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.628+0000 7f9b87890740 1 -- 172.21.15.67:0/1480100294 wait complete. 2024-07-31T14:51:56.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.629+0000 7f9b87890740 20 client.25448 trim_cache size 0 max 16384 2024-07-31T14:51:56.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.647+0000 7f9b87890740 1 Processor -- start 2024-07-31T14:51:56.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.647+0000 7f9b87890740 1 -- start start 2024-07-31T14:51:56.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b87890740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed05a0a0 0x5574ed434670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b87890740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed062280 0x5574ed43fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b87890740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed082f60 0x5574ed440320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b87890740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5574ed4446b0 con 0x5574ed082f60 2024-07-31T14:51:56.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b87890740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5574ed444d80 con 0x5574ed062280 2024-07-31T14:51:56.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b87890740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5574ed445450 con 0x5574ed05a0a0 2024-07-31T14:51:56.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b7b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed082f60 0x5574ed440320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b7b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed082f60 0x5574ed440320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52694/0 (socket says 172.21.15.67:52694) 2024-07-31T14:51:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b7b7fe640 1 -- 172.21.15.67:0/1660494665 learned_addr learned my addr 172.21.15.67:0/1660494665 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b7affd640 1 --2- 172.21.15.67:0/1660494665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed05a0a0 0x5574ed434670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b7a7fc640 1 --2- 172.21.15.67:0/1660494665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed062280 0x5574ed43fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b7b7fe640 1 -- 172.21.15.67:0/1660494665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed05a0a0 msgr2=0x5574ed434670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b7b7fe640 1 --2- 172.21.15.67:0/1660494665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed05a0a0 0x5574ed434670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b7b7fe640 1 -- 172.21.15.67:0/1660494665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed062280 msgr2=0x5574ed43fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b7b7fe640 1 --2- 172.21.15.67:0/1660494665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed062280 0x5574ed43fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.648+0000 7f9b7b7fe640 1 -- 172.21.15.67:0/1660494665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574ed4366f0 con 0x5574ed082f60 2024-07-31T14:51:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.649+0000 7f9b7b7fe640 1 --2- 172.21.15.67:0/1660494665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed082f60 0x5574ed440320 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0aaaa0 tx=0x7f9b6c0a6d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadcf85e56fb97ce server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.649+0000 7f9b70ff9640 1 -- 172.21.15.67:0/1660494665 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c01f040 con 0x5574ed082f60 2024-07-31T14:51:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.649+0000 7f9b70ff9640 1 -- 172.21.15.67:0/1660494665 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b6c0b3e50 con 0x5574ed082f60 2024-07-31T14:51:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.649+0000 7f9b70ff9640 1 -- 172.21.15.67:0/1660494665 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c0054a0 con 0x5574ed082f60 2024-07-31T14:51:56.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.650+0000 7f9b87890740 1 -- 172.21.15.67:0/1660494665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed082f60 msgr2=0x5574ed440320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:56.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.650+0000 7f9b87890740 1 --2- 172.21.15.67:0/1660494665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed082f60 0x5574ed440320 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0aaaa0 tx=0x7f9b6c0a6d90 comp rx=0 tx=0).stop 2024-07-31T14:51:56.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.650+0000 7f9b87890740 1 -- 172.21.15.67:0/1660494665 shutdown_connections 2024-07-31T14:51:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.650+0000 7f9b87890740 1 --2- 172.21.15.67:0/1660494665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed082f60 0x5574ed440320 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.650+0000 7f9b87890740 1 --2- 172.21.15.67:0/1660494665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed062280 0x5574ed43fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.650+0000 7f9b87890740 1 --2- 172.21.15.67:0/1660494665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed05a0a0 0x5574ed434670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.650+0000 7f9b87890740 1 -- 172.21.15.67:0/1660494665 >> 172.21.15.67:0/1660494665 conn(0x5574ed077350 msgr2=0x5574ed433f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.650+0000 7f9b87890740 1 -- 172.21.15.67:0/1660494665 shutdown_connections 2024-07-31T14:51:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.650+0000 7f9b87890740 1 -- 172.21.15.67:0/1660494665 wait complete. 2024-07-31T14:51:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.650+0000 7f9b87890740 1 -- start start 2024-07-31T14:51:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.650+0000 7f9b87890740 1 Processor -- start 2024-07-31T14:51:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b87890740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed05a0a0 0x5574ed44e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b87890740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed062280 0x5574ed44ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b87890740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed082f60 0x5574ed447c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b87890740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5574ed4461f0 con 0x5574ed062280 2024-07-31T14:51:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b87890740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5574ed445b20 con 0x5574ed05a0a0 2024-07-31T14:51:56.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b87890740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5574ed4468c0 con 0x5574ed082f60 2024-07-31T14:51:56.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b87890740 10 client.0 _ll_register_callbacks cb 0x7f9b74004d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:56.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b7a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed062280 0x5574ed44ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b7a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed062280 0x5574ed44ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52696/0 (socket says 172.21.15.67:52696) 2024-07-31T14:51:56.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b7affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed05a0a0 0x5574ed44e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b7a7fc640 1 -- 172.21.15.67:0/739079961 learned_addr learned my addr 172.21.15.67:0/739079961 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:56.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b7b7fe640 1 --2- 172.21.15.67:0/739079961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed082f60 0x5574ed447c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b7a7fc640 1 -- 172.21.15.67:0/739079961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed082f60 msgr2=0x5574ed447c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:56.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b7a7fc640 1 --2- 172.21.15.67:0/739079961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed082f60 0x5574ed447c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b7a7fc640 1 -- 172.21.15.67:0/739079961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed05a0a0 msgr2=0x5574ed44e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:56.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b7a7fc640 1 --2- 172.21.15.67:0/739079961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed05a0a0 0x5574ed44e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b7a7fc640 1 -- 172.21.15.67:0/739079961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574ed448380 con 0x5574ed062280 2024-07-31T14:51:56.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b7b7fe640 1 --2- 172.21.15.67:0/739079961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed082f60 0x5574ed447c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:56.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b7a7fc640 1 --2- 172.21.15.67:0/739079961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed062280 0x5574ed44ed20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009230 tx=0x7f9b6402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750bf524f3e88c00 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:56.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.651+0000 7f9b627fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:51:56.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.652+0000 7f9b627fc640 1 -- 172.21.15.67:0/739079961 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b64059070 con 0x5574ed062280 2024-07-31T14:51:56.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.652+0000 7f9b627fc640 1 -- 172.21.15.67:0/739079961 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f9b640037f0 con 0x5574ed062280 2024-07-31T14:51:56.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.652+0000 7f9b87890740 1 -- 172.21.15.67:0/739079961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b6c0b3e50 con 0x5574ed062280 2024-07-31T14:51:56.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.652+0000 7f9b627fc640 1 -- 172.21.15.67:0/739079961 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b64064050 con 0x5574ed062280 2024-07-31T14:51:56.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.652+0000 7f9b87890740 10 client.15771 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:56.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.652+0000 7f9b87890740 1 -- 172.21.15.67:0/739079961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574ed44dd40 con 0x5574ed062280 2024-07-31T14:51:56.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.652+0000 7f9b627fc640 1 -- 172.21.15.67:0/739079961 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b6c0b3e50 con 0x5574ed062280 2024-07-31T14:51:56.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.652+0000 7f9b627fc640 1 -- 172.21.15.67:0/739079961 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9b640559b0 con 0x5574ed062280 2024-07-31T14:51:56.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.652+0000 7f9b87890740 10 client.15771 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:56.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.652+0000 7f9b87890740 10 client.15771 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-31T14:51:56.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.652+0000 7f9b87890740 20 client.15771 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:56.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.652+0000 7f9b87890740 1 --2- 172.21.15.67:0/739079961 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5574ed016c90 0x5574ed4b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:56.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.652+0000 7f9b87890740 4 client.15771 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:51:56.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.652+0000 7f9b87890740 1 -- 172.21.15.67:0/739079961 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "status"}) -- 0x5574ed39c330 con 0x5574ed016c90 2024-07-31T14:51:56.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.653+0000 7f9b7affd640 1 --2- 172.21.15.67:0/739079961 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5574ed016c90 0x5574ed4b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:56.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.653+0000 7f9b7affd640 1 --2- 172.21.15.67:0/739079961 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5574ed016c90 0x5574ed4b2880 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c5e999ffbf59697 server_cookie=756e64b96f75cad2 in_seq=0 out_seq=0 2024-07-31T14:51:56.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.653+0000 7f9b627fc640 10 client.15771 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:51:56.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.653+0000 7f9b627fc640 1 -- 172.21.15.67:0/739079961 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5574ed39c330 con 0x5574ed016c90 2024-07-31T14:51:56.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.653+0000 7f9b627fc640 10 client.15771 handle_command_reply: tid=0 2024-07-31T14:51:56.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.653+0000 7f9b87890740 1 client.15771 shutdown 2024-07-31T14:51:56.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b87890740 1 -- 172.21.15.67:0/739079961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed062280 msgr2=0x5574ed44ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:56.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b87890740 1 --2- 172.21.15.67:0/739079961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed062280 0x5574ed44ed20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009230 tx=0x7f9b6402a410 comp rx=0 tx=0).stop 2024-07-31T14:51:56.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b87890740 1 -- 172.21.15.67:0/739079961 shutdown_connections 2024-07-31T14:51:56.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b87890740 1 --2- 172.21.15.67:0/739079961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574ed082f60 0x5574ed447c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b87890740 1 --2- 172.21.15.67:0/739079961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574ed062280 0x5574ed44ed20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b87890740 1 --2- 172.21.15.67:0/739079961 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5574ed016c90 0x5574ed4b2880 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b87890740 1 --2- 172.21.15.67:0/739079961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574ed05a0a0 0x5574ed44e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b87890740 1 -- 172.21.15.67:0/739079961 >> 172.21.15.67:0/739079961 conn(0x5574ed077350 msgr2=0x5574ed42b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:56.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b627fc640 0 client.15771 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:51:56.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b87890740 1 -- 172.21.15.67:0/739079961 shutdown_connections 2024-07-31T14:51:56.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b87890740 1 -- 172.21.15.67:0/739079961 wait complete. 2024-07-31T14:51:56.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b87890740 20 client.15771 trim_cache size 0 max 16384 2024-07-31T14:51:56.661 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:51:56.661 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:51:56.662 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 2, 2024-07-31T14:51:56.662 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24650, 2024-07-31T14:51:56.662 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:51:56.662 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:51:56.662 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:51:56.662 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 236.093917718, 2024-07-31T14:51:56.662 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 30, 2024-07-31T14:51:56.662 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:51:56.662 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:51:56.662 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 317.86819778900002 2024-07-31T14:51:56.662 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:51:56.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b8133f640 1 -- 172.21.15.67:0/2674474199 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9b680b3d40 msgr2=0x7f9b680b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:56.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b8133f640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9b680b3d40 0x7f9b680b6250 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9b64000c00 tx=0x7f9b64005780 comp rx=0 tx=0).stop 2024-07-31T14:51:56.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b8133f640 1 -- 172.21.15.67:0/2674474199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b7c108720 msgr2=0x7f9b7c110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:56.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b8133f640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b7c108720 0x7f9b7c110460 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00eaa0 tx=0x7f9b6c00ef60 comp rx=0 tx=0).stop 2024-07-31T14:51:56.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.654+0000 7f9b8133f640 1 -- 172.21.15.67:0/2674474199 shutdown_connections 2024-07-31T14:51:56.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.655+0000 7f9b8133f640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9b680b3d40 0x7f9b680b6250 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.655+0000 7f9b8133f640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b7c108720 0x7f9b7c110460 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.655+0000 7f9b8133f640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b7c107dd0 0x7f9b7c10ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.655+0000 7f9b8133f640 1 --2- 172.21.15.67:0/2674474199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b7c102490 0x7f9b7c10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:56.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.655+0000 7f9b8133f640 1 -- 172.21.15.67:0/2674474199 >> 172.21.15.67:0/2674474199 conn(0x7f9b7c079160 msgr2=0x7f9b7c104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:56.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.655+0000 7f9b8133f640 1 -- 172.21.15.67:0/2674474199 shutdown_connections 2024-07-31T14:51:56.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:56.655+0000 7f9b8133f640 1 -- 172.21.15.67:0/2674474199 wait complete. 2024-07-31T14:51:56.944 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:56 smithi073 ceph-mon[36714]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-07-31T14:51:57.065 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:56 smithi067 ceph-mon[32750]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-07-31T14:51:57.065 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:56 smithi077 ceph-mon[37327]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-07-31T14:51:57.425 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.i status 2024-07-31T14:51:57.692 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:51:58.665 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:51:58.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:51:58 smithi067 ceph-mon[32750]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-07-31T14:51:58.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:51:58 smithi077 ceph-mon[37327]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-07-31T14:51:59.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:51:58 smithi073 ceph-mon[36714]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-07-31T14:51:59.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.376+0000 7f7003f8b640 1 -- 172.21.15.67:0/1953009492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ffc108740 msgr2=0x7f6ffc10ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:59.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.376+0000 7f7003f8b640 1 --2- 172.21.15.67:0/1953009492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ffc108740 0x7f6ffc10ef70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009900 tx=0x7f6ff402a410 comp rx=0 tx=0).stop 2024-07-31T14:51:59.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.377+0000 7f7003f8b640 1 -- 172.21.15.67:0/1953009492 shutdown_connections 2024-07-31T14:51:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.377+0000 7f7003f8b640 1 --2- 172.21.15.67:0/1953009492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ffc108740 0x7f6ffc10ef70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.377+0000 7f7003f8b640 1 --2- 172.21.15.67:0/1953009492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc107df0 0x7f6ffc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.377+0000 7f7003f8b640 1 --2- 172.21.15.67:0/1953009492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ffc1024b0 0x7f6ffc1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.377+0000 7f7003f8b640 1 -- 172.21.15.67:0/1953009492 >> 172.21.15.67:0/1953009492 conn(0x7f6ffc079170 msgr2=0x7f6ffc079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.377+0000 7f7003f8b640 1 -- 172.21.15.67:0/1953009492 shutdown_connections 2024-07-31T14:51:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.377+0000 7f7003f8b640 1 -- 172.21.15.67:0/1953009492 wait complete. 2024-07-31T14:51:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.377+0000 7f7003f8b640 1 Processor -- start 2024-07-31T14:51:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.377+0000 7f7003f8b640 1 -- start start 2024-07-31T14:51:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7003f8b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ffc1024b0 0x7f6ffc199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7003f8b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ffc107df0 0x7f6ffc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7003f8b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc108740 0x7f6ffc19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7003f8b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1134d0 con 0x7f6ffc107df0 2024-07-31T14:51:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7003f8b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc113350 con 0x7f6ffc1024b0 2024-07-31T14:51:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7003f8b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc113650 con 0x7f6ffc108740 2024-07-31T14:51:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7001526640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ffc107df0 0x7f6ffc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7001526640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ffc107df0 0x7f6ffc19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52712/0 (socket says 172.21.15.67:52712) 2024-07-31T14:51:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7001526640 1 -- 172.21.15.67:0/254559146 learned_addr learned my addr 172.21.15.67:0/254559146 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7002528640 1 --2- 172.21.15.67:0/254559146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc108740 0x7f6ffc19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7001d27640 1 --2- 172.21.15.67:0/254559146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ffc1024b0 0x7f6ffc199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7001526640 1 -- 172.21.15.67:0/254559146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc108740 msgr2=0x7f6ffc19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7001526640 1 --2- 172.21.15.67:0/254559146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc108740 0x7f6ffc19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7001526640 1 -- 172.21.15.67:0/254559146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ffc1024b0 msgr2=0x7f6ffc199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7001526640 1 --2- 172.21.15.67:0/254559146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ffc1024b0 0x7f6ffc199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.378+0000 7f7001526640 1 -- 172.21.15.67:0/254559146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc19fd90 con 0x7f6ffc107df0 2024-07-31T14:51:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.379+0000 7f7001526640 1 --2- 172.21.15.67:0/254559146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ffc107df0 0x7f6ffc19a480 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000ea80 tx=0x7f6ff000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.379+0000 7f6ffaffd640 1 -- 172.21.15.67:0/254559146 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff000cd80 con 0x7f6ffc107df0 2024-07-31T14:51:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.379+0000 7f7003f8b640 1 -- 172.21.15.67:0/254559146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ffc1a0080 con 0x7f6ffc107df0 2024-07-31T14:51:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.379+0000 7f6ffaffd640 1 -- 172.21.15.67:0/254559146 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff0011920 con 0x7f6ffc107df0 2024-07-31T14:51:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.379+0000 7f6ffaffd640 1 -- 172.21.15.67:0/254559146 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff001a960 con 0x7f6ffc107df0 2024-07-31T14:51:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.379+0000 7f7003f8b640 1 -- 172.21.15.67:0/254559146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ffc1a68a0 con 0x7f6ffc107df0 2024-07-31T14:51:59.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.382+0000 7f700a4dc740 1 Processor -- start 2024-07-31T14:51:59.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.382+0000 7f700a4dc740 1 -- start start 2024-07-31T14:51:59.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.383+0000 7f6ffaffd640 1 -- 172.21.15.67:0/254559146 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f6ff0011470 con 0x7f6ffc107df0 2024-07-31T14:51:59.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.383+0000 7f700a4dc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb5ff60 0x55df6ff05380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.383+0000 7f700a4dc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb370a0 0x55df6ff058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.383+0000 7f700a4dc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff05e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.383+0000 7f700a4dc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55df6fe7b920 con 0x55df6fb3f280 2024-07-31T14:51:59.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.383+0000 7f700a4dc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55df6fe6ae20 con 0x55df6fb370a0 2024-07-31T14:51:59.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.383+0000 7f700a4dc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55df6fe6ee60 con 0x55df6fb5ff60 2024-07-31T14:51:59.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.383+0000 7f7002528640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb5ff60 0x55df6ff05380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.383+0000 7f7002528640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb5ff60 0x55df6ff05380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:55888/0 (socket says 172.21.15.67:55888) 2024-07-31T14:51:59.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.383+0000 7f7002528640 1 -- 172.21.15.67:0/3003421969 learned_addr learned my addr 172.21.15.67:0/3003421969 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:59.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.383+0000 7f7001526640 1 --2- 172.21.15.67:0/3003421969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff05e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.383+0000 7f7001d27640 1 --2- 172.21.15.67:0/3003421969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb370a0 0x55df6ff058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.384+0000 7f7002528640 1 -- 172.21.15.67:0/3003421969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb370a0 msgr2=0x55df6ff058c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:59.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.384+0000 7f7002528640 1 --2- 172.21.15.67:0/3003421969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb370a0 0x55df6ff058c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.384+0000 7f7002528640 1 -- 172.21.15.67:0/3003421969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 msgr2=0x55df6ff05e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:59.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.384+0000 7f7002528640 1 --2- 172.21.15.67:0/3003421969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff05e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.384+0000 7f7002528640 1 -- 172.21.15.67:0/3003421969 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6fe11a40 con 0x55df6fb5ff60 2024-07-31T14:51:59.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.384+0000 7f7001d27640 1 --2- 172.21.15.67:0/3003421969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb370a0 0x55df6ff058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:51:59.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.384+0000 7f7002528640 1 --2- 172.21.15.67:0/3003421969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb5ff60 0x55df6ff05380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009900 tx=0x7f6ff4032e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bfa72e25f30d3fa server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:59.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.384+0000 7f6ffaffd640 1 --2- 172.21.15.67:0/254559146 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6fe40b3d70 0x7f6fe40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.384+0000 7f7001d27640 1 --2- 172.21.15.67:0/254559146 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6fe40b3d70 0x7f6fe40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.384+0000 7f6ffaffd640 1 -- 172.21.15.67:0/254559146 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f6ff00e46d0 con 0x7f6ffc107df0 2024-07-31T14:51:59.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.384+0000 7f6fe3fff640 1 -- 172.21.15.67:0/3003421969 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4040070 con 0x55df6fb5ff60 2024-07-31T14:51:59.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.384+0000 7f6fe3fff640 1 -- 172.21.15.67:0/3003421969 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff40316a0 con 0x55df6fb5ff60 2024-07-31T14:51:59.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.384+0000 7f6fe3fff640 1 -- 172.21.15.67:0/3003421969 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4035040 con 0x55df6fb5ff60 2024-07-31T14:51:59.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.385+0000 7f700a4dc740 1 -- 172.21.15.67:0/3003421969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb5ff60 msgr2=0x55df6ff05380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:59.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.385+0000 7f700a4dc740 1 --2- 172.21.15.67:0/3003421969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb5ff60 0x55df6ff05380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009900 tx=0x7f6ff4032e70 comp rx=0 tx=0).stop 2024-07-31T14:51:59.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.385+0000 7f7001d27640 1 --2- 172.21.15.67:0/254559146 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6fe40b3d70 0x7f6fe40b6280 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55df6ff04000 tx=0x7f6fec00a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:59.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.386+0000 7f700a4dc740 1 -- 172.21.15.67:0/3003421969 shutdown_connections 2024-07-31T14:51:59.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.386+0000 7f700a4dc740 1 --2- 172.21.15.67:0/3003421969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff05e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.386+0000 7f700a4dc740 1 --2- 172.21.15.67:0/3003421969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb370a0 0x55df6ff058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.386+0000 7f700a4dc740 1 --2- 172.21.15.67:0/3003421969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb5ff60 0x55df6ff05380 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.386+0000 7f700a4dc740 1 -- 172.21.15.67:0/3003421969 >> 172.21.15.67:0/3003421969 conn(0x55df6fb54350 msgr2=0x55df6ff03270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:59.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.386+0000 7f700a4dc740 1 -- 172.21.15.67:0/3003421969 shutdown_connections 2024-07-31T14:51:59.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.386+0000 7f700a4dc740 1 -- 172.21.15.67:0/3003421969 wait complete. 2024-07-31T14:51:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.386+0000 7f700a4dc740 1 -- start start 2024-07-31T14:51:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.387+0000 7f700a4dc740 1 Processor -- start 2024-07-31T14:51:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.387+0000 7f700a4dc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb370a0 0x55df6ff15370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.387+0000 7f700a4dc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.387+0000 7f700a4dc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb5ff60 0x55df6ff21190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.387+0000 7f700a4dc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55df6fcce5f0 con 0x55df6fb3f280 2024-07-31T14:51:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.387+0000 7f700a4dc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55df6fe0d890 con 0x55df6fb5ff60 2024-07-31T14:51:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.387+0000 7f700a4dc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55df6ff17e60 con 0x55df6fb370a0 2024-07-31T14:51:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.387+0000 7f700a4dc740 10 client.0 _ll_register_callbacks cb 0x55df6fe177d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.387+0000 7f7001526640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff158b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.387+0000 7f7001526640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff158b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52732/0 (socket says 172.21.15.67:52732) 2024-07-31T14:51:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.387+0000 7f7001526640 1 -- 172.21.15.67:0/2662463689 learned_addr learned my addr 172.21.15.67:0/2662463689 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.387+0000 7f7001d27640 1 --2- 172.21.15.67:0/2662463689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb370a0 0x55df6ff15370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.388+0000 7f7002528640 1 --2- 172.21.15.67:0/2662463689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb5ff60 0x55df6ff21190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.388+0000 7f7001d27640 1 -- 172.21.15.67:0/2662463689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb5ff60 msgr2=0x55df6ff21190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.388+0000 7f7001d27640 1 --2- 172.21.15.67:0/2662463689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb5ff60 0x55df6ff21190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.388+0000 7f7001d27640 1 -- 172.21.15.67:0/2662463689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 msgr2=0x55df6ff158b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.388+0000 7f7001d27640 1 --2- 172.21.15.67:0/2662463689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff158b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.388+0000 7f7001d27640 1 -- 172.21.15.67:0/2662463689 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6fbcb540 con 0x55df6fb370a0 2024-07-31T14:51:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.388+0000 7f7001526640 1 --2- 172.21.15.67:0/2662463689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff158b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:51:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.388+0000 7f7001d27640 1 --2- 172.21.15.67:0/2662463689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb370a0 0x55df6ff15370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6fec012e20 tx=0x7f6fec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb7cc848ecaa59d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.388+0000 7f6fe17fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:51:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.388+0000 7f6fe17fa640 1 -- 172.21.15.67:0/2662463689 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec01f420 con 0x55df6fb370a0 2024-07-31T14:51:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.388+0000 7f6fe17fa640 1 -- 172.21.15.67:0/2662463689 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f6fec01f5c0 con 0x55df6fb370a0 2024-07-31T14:51:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.388+0000 7f700a4dc740 1 -- 172.21.15.67:0/2662463689 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff40316a0 con 0x55df6fb370a0 2024-07-31T14:51:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.389+0000 7f6fe17fa640 1 -- 172.21.15.67:0/2662463689 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6fec030050 con 0x55df6fb370a0 2024-07-31T14:51:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.389+0000 7f700a4dc740 10 client.25483 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.389+0000 7f700a4dc740 1 -- 172.21.15.67:0/2662463689 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df6ff23e80 con 0x55df6fb370a0 2024-07-31T14:51:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.389+0000 7f6fe17fa640 1 -- 172.21.15.67:0/2662463689 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff40316a0 con 0x55df6fb370a0 2024-07-31T14:51:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.389+0000 7f6fe17fa640 1 -- 172.21.15.67:0/2662463689 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6fec01e680 con 0x55df6fb370a0 2024-07-31T14:51:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.389+0000 7f700a4dc740 10 client.25483 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.389+0000 7f700a4dc740 10 client.25483 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-31T14:51:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.389+0000 7f700a4dc740 20 client.25483 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.389+0000 7f700a4dc740 1 --2- 172.21.15.67:0/2662463689 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55df6fb1e8e0 0x55df6ff278f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.389+0000 7f700a4dc740 4 client.25483 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:51:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.389+0000 7f700a4dc740 1 -- 172.21.15.67:0/2662463689 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df6fe7d870 con 0x55df6fb1e8e0 2024-07-31T14:51:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.389+0000 7f7001526640 1 --2- 172.21.15.67:0/2662463689 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55df6fb1e8e0 0x55df6ff278f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.390+0000 7f6fe17fa640 10 client.25483 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:51:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.390+0000 7f7001526640 1 --2- 172.21.15.67:0/2662463689 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55df6fb1e8e0 0x55df6ff278f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78454beeb9e503b3 server_cookie=25202eff335abc17 in_seq=0 out_seq=0 2024-07-31T14:51:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f6fe17fa640 1 -- 172.21.15.67:0/2662463689 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df6fe7d870 con 0x55df6fb1e8e0 2024-07-31T14:51:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f6fe17fa640 10 client.25483 handle_command_reply: tid=0 2024-07-31T14:51:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f700a4dc740 1 client.25483 shutdown 2024-07-31T14:51:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f700a4dc740 1 -- 172.21.15.67:0/2662463689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb370a0 msgr2=0x55df6ff15370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f700a4dc740 1 --2- 172.21.15.67:0/2662463689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb370a0 0x55df6ff15370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6fec012e20 tx=0x7f6fec0046c0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f700a4dc740 1 -- 172.21.15.67:0/2662463689 shutdown_connections 2024-07-31T14:51:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f700a4dc740 1 --2- 172.21.15.67:0/2662463689 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55df6fb1e8e0 0x55df6ff278f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f700a4dc740 1 --2- 172.21.15.67:0/2662463689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb5ff60 0x55df6ff21190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f6fe17fa640 0 client.25483 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:51:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f700a4dc740 1 --2- 172.21.15.67:0/2662463689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff158b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f700a4dc740 1 --2- 172.21.15.67:0/2662463689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb370a0 0x55df6ff15370 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f700a4dc740 1 -- 172.21.15.67:0/2662463689 >> 172.21.15.67:0/2662463689 conn(0x55df6fb54350 msgr2=0x55df6ff06d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f700a4dc740 1 -- 172.21.15.67:0/2662463689 shutdown_connections 2024-07-31T14:51:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f700a4dc740 1 -- 172.21.15.67:0/2662463689 wait complete. 2024-07-31T14:51:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.391+0000 7f700a4dc740 20 client.25483 trim_cache size 0 max 16384 2024-07-31T14:51:59.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.421+0000 7f700a4dc740 1 Processor -- start 2024-07-31T14:51:59.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.421+0000 7f700a4dc740 1 -- start start 2024-07-31T14:51:59.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f700a4dc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb370a0 0x55df6ff0b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f700a4dc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff21100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f700a4dc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb5ff60 0x55df6ff21640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f700a4dc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55df6ff25c30 con 0x55df6fb3f280 2024-07-31T14:51:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f700a4dc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55df6ff26300 con 0x55df6fb5ff60 2024-07-31T14:51:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f700a4dc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55df6ff269d0 con 0x55df6fb370a0 2024-07-31T14:51:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f7001526640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff21100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f7001526640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff21100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52736/0 (socket says 172.21.15.67:52736) 2024-07-31T14:51:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f7001526640 1 -- 172.21.15.67:0/3088732748 learned_addr learned my addr 172.21.15.67:0/3088732748 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f7001d27640 1 --2- 172.21.15.67:0/3088732748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb370a0 0x55df6ff0b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f7002528640 1 --2- 172.21.15.67:0/3088732748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb5ff60 0x55df6ff21640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f7001526640 1 -- 172.21.15.67:0/3088732748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb370a0 msgr2=0x55df6ff0b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f7001526640 1 --2- 172.21.15.67:0/3088732748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb370a0 0x55df6ff0b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f7001526640 1 -- 172.21.15.67:0/3088732748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb5ff60 msgr2=0x55df6ff21640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f7001526640 1 --2- 172.21.15.67:0/3088732748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb5ff60 0x55df6ff21640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f7001526640 1 -- 172.21.15.67:0/3088732748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6ff179f0 con 0x55df6fb3f280 2024-07-31T14:51:59.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f7001d27640 1 --2- 172.21.15.67:0/3088732748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb370a0 0x55df6ff0b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:59.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.422+0000 7f7001526640 1 --2- 172.21.15.67:0/3088732748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff21100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00aacd0 tx=0x7f6ff00e6be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd951a16b9ae18 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:59.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.423+0000 7f6fe37fe640 1 -- 172.21.15.67:0/3088732748 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0011760 con 0x55df6fb3f280 2024-07-31T14:51:59.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.423+0000 7f6fe37fe640 1 -- 172.21.15.67:0/3088732748 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff00aaae0 con 0x55df6fb3f280 2024-07-31T14:51:59.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.423+0000 7f6fe37fe640 1 -- 172.21.15.67:0/3088732748 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00a7260 con 0x55df6fb3f280 2024-07-31T14:51:59.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.424+0000 7f700a4dc740 1 -- 172.21.15.67:0/3088732748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 msgr2=0x55df6ff21100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:59.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.424+0000 7f700a4dc740 1 --2- 172.21.15.67:0/3088732748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff21100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00aacd0 tx=0x7f6ff00e6be0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.424+0000 7f700a4dc740 1 -- 172.21.15.67:0/3088732748 shutdown_connections 2024-07-31T14:51:59.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.424+0000 7f700a4dc740 1 --2- 172.21.15.67:0/3088732748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb5ff60 0x55df6ff21640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.424+0000 7f700a4dc740 1 --2- 172.21.15.67:0/3088732748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb3f280 0x55df6ff21100 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.424+0000 7f700a4dc740 1 --2- 172.21.15.67:0/3088732748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb370a0 0x55df6ff0b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.424+0000 7f700a4dc740 1 -- 172.21.15.67:0/3088732748 >> 172.21.15.67:0/3088732748 conn(0x55df6fb54350 msgr2=0x55df6ff139f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:59.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.424+0000 7f700a4dc740 1 -- 172.21.15.67:0/3088732748 shutdown_connections 2024-07-31T14:51:59.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.424+0000 7f700a4dc740 1 -- 172.21.15.67:0/3088732748 wait complete. 2024-07-31T14:51:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.425+0000 7f700a4dc740 1 -- start start 2024-07-31T14:51:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.425+0000 7f700a4dc740 1 Processor -- start 2024-07-31T14:51:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.425+0000 7f700a4dc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb370a0 0x55df6ff2fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.425+0000 7f700a4dc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb3f280 0x55df6ff30330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.425+0000 7f700a4dc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb5ff60 0x55df6ff29220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.425+0000 7f7001d27640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb370a0 0x55df6ff2fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.425+0000 7f700a4dc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55df6ff27770 con 0x55df6fb370a0 2024-07-31T14:51:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f7001d27640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb370a0 0x55df6ff2fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52752/0 (socket says 172.21.15.67:52752) 2024-07-31T14:51:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f700a4dc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55df6ff270a0 con 0x55df6fb3f280 2024-07-31T14:51:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f7001d27640 1 -- 172.21.15.67:0/2013594737 learned_addr learned my addr 172.21.15.67:0/2013594737 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:51:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f700a4dc740 1 -- 172.21.15.67:0/2013594737 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55df6ff27e40 con 0x55df6fb5ff60 2024-07-31T14:51:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f700a4dc740 10 client.0 _ll_register_callbacks cb 0x7f6fec012ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:51:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f7001526640 1 --2- 172.21.15.67:0/2013594737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb3f280 0x55df6ff30330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f7002528640 1 --2- 172.21.15.67:0/2013594737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb5ff60 0x55df6ff29220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f7001d27640 1 -- 172.21.15.67:0/2013594737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb5ff60 msgr2=0x55df6ff29220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f7001d27640 1 --2- 172.21.15.67:0/2013594737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb5ff60 0x55df6ff29220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f7001d27640 1 -- 172.21.15.67:0/2013594737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb3f280 msgr2=0x55df6ff30330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:51:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f7001d27640 1 --2- 172.21.15.67:0/2013594737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb3f280 0x55df6ff30330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f7001d27640 1 -- 172.21.15.67:0/2013594737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6ff299b0 con 0x55df6fb370a0 2024-07-31T14:51:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f7001526640 1 --2- 172.21.15.67:0/2013594737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb3f280 0x55df6ff30330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:51:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f7001d27640 1 --2- 172.21.15.67:0/2013594737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb370a0 0x55df6ff2fd50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6fec000c00 tx=0x7f6fec00fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2ae20dec143838 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:51:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f6fe17fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:51:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f6fe17fa640 1 -- 172.21.15.67:0/2013594737 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec03a030 con 0x55df6fb370a0 2024-07-31T14:51:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.426+0000 7f700a4dc740 1 -- 172.21.15.67:0/2013594737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff00aaae0 con 0x55df6fb370a0 2024-07-31T14:51:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.427+0000 7f6fe17fa640 1 -- 172.21.15.67:0/2013594737 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f6fec01e5c0 con 0x55df6fb370a0 2024-07-31T14:51:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.427+0000 7f6fe17fa640 1 -- 172.21.15.67:0/2013594737 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6fec046050 con 0x55df6fb370a0 2024-07-31T14:51:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.427+0000 7f700a4dc740 10 client.15789 fetch_fsmap learned FSMap version 30 2024-07-31T14:51:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.427+0000 7f700a4dc740 1 -- 172.21.15.67:0/2013594737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df6ff2f320 con 0x55df6fb370a0 2024-07-31T14:51:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.427+0000 7f6fe17fa640 1 -- 172.21.15.67:0/2013594737 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff00aaae0 con 0x55df6fb370a0 2024-07-31T14:51:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.427+0000 7f6fe17fa640 1 -- 172.21.15.67:0/2013594737 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6fec033050 con 0x55df6fb370a0 2024-07-31T14:51:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.427+0000 7f700a4dc740 10 client.15789 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:51:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.427+0000 7f700a4dc740 10 client.15789 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-31T14:51:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.427+0000 7f700a4dc740 20 client.15789 populate_metadata read hostname 'smithi067' 2024-07-31T14:51:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.427+0000 7f700a4dc740 1 --2- 172.21.15.67:0/2013594737 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55df6faf3c90 0x55df6ff93e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:51:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.427+0000 7f700a4dc740 4 client.15789 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:51:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.427+0000 7f700a4dc740 1 -- 172.21.15.67:0/2013594737 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "status"}) -- 0x55df6fe79310 con 0x55df6faf3c90 2024-07-31T14:51:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.428+0000 7f7001526640 1 --2- 172.21.15.67:0/2013594737 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55df6faf3c90 0x55df6ff93e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:51:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.428+0000 7f7001526640 1 --2- 172.21.15.67:0/2013594737 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55df6faf3c90 0x55df6ff93e70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc03c5ac47413769 server_cookie=4ce9406317609b1b in_seq=0 out_seq=0 2024-07-31T14:51:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.428+0000 7f6fe17fa640 10 client.15789 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:51:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.428+0000 7f6fe17fa640 1 -- 172.21.15.67:0/2013594737 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55df6fe79310 con 0x55df6faf3c90 2024-07-31T14:51:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.428+0000 7f6fe17fa640 10 client.15789 handle_command_reply: tid=0 2024-07-31T14:51:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.429+0000 7f700a4dc740 1 client.15789 shutdown 2024-07-31T14:51:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.429+0000 7f700a4dc740 1 -- 172.21.15.67:0/2013594737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb370a0 msgr2=0x55df6ff2fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.429+0000 7f700a4dc740 1 --2- 172.21.15.67:0/2013594737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb370a0 0x55df6ff2fd50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6fec000c00 tx=0x7f6fec00fee0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.429+0000 7f700a4dc740 1 -- 172.21.15.67:0/2013594737 shutdown_connections 2024-07-31T14:51:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.429+0000 7f700a4dc740 1 --2- 172.21.15.67:0/2013594737 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55df6faf3c90 0x55df6ff93e70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.429+0000 7f6fe17fa640 0 client.15789 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:51:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.429+0000 7f700a4dc740 1 --2- 172.21.15.67:0/2013594737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df6fb5ff60 0x55df6ff29220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.429+0000 7f700a4dc740 1 --2- 172.21.15.67:0/2013594737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df6fb3f280 0x55df6ff30330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.429+0000 7f700a4dc740 1 --2- 172.21.15.67:0/2013594737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df6fb370a0 0x55df6ff2fd50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.429+0000 7f700a4dc740 1 -- 172.21.15.67:0/2013594737 >> 172.21.15.67:0/2013594737 conn(0x55df6fb54350 msgr2=0x55df6ff139b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:59.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.429+0000 7f700a4dc740 1 -- 172.21.15.67:0/2013594737 shutdown_connections 2024-07-31T14:51:59.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.429+0000 7f700a4dc740 1 -- 172.21.15.67:0/2013594737 wait complete. 2024-07-31T14:51:59.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.429+0000 7f700a4dc740 20 client.15789 trim_cache size 0 max 16384 2024-07-31T14:51:59.435 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:51:59.435 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:51:59.435 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 1, 2024-07-31T14:51:59.435 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24655, 2024-07-31T14:51:59.435 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:51:59.435 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:51:59.436 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:51:59.436 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 239.87276601100001, 2024-07-31T14:51:59.436 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 30, 2024-07-31T14:51:59.436 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:51:59.436 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:51:59.436 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 317.92460643999999 2024-07-31T14:51:59.436 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:51:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.430+0000 7f7003f8b640 1 -- 172.21.15.67:0/254559146 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6fe40b3d70 msgr2=0x7f6fe40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.430+0000 7f7003f8b640 1 --2- 172.21.15.67:0/254559146 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6fe40b3d70 0x7f6fe40b6280 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55df6ff04000 tx=0x7f6fec00a2d0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.430+0000 7f7003f8b640 1 -- 172.21.15.67:0/254559146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ffc107df0 msgr2=0x7f6ffc19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:51:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.430+0000 7f7003f8b640 1 --2- 172.21.15.67:0/254559146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ffc107df0 0x7f6ffc19a480 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000ea80 tx=0x7f6ff000ee80 comp rx=0 tx=0).stop 2024-07-31T14:51:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.430+0000 7f7003f8b640 1 -- 172.21.15.67:0/254559146 shutdown_connections 2024-07-31T14:51:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.430+0000 7f7003f8b640 1 --2- 172.21.15.67:0/254559146 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6fe40b3d70 0x7f6fe40b6280 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.430+0000 7f7003f8b640 1 --2- 172.21.15.67:0/254559146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ffc108740 0x7f6ffc19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.430+0000 7f7003f8b640 1 --2- 172.21.15.67:0/254559146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ffc107df0 0x7f6ffc19a480 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.430+0000 7f7003f8b640 1 --2- 172.21.15.67:0/254559146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ffc1024b0 0x7f6ffc199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:51:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.430+0000 7f7003f8b640 1 -- 172.21.15.67:0/254559146 >> 172.21.15.67:0/254559146 conn(0x7f6ffc079170 msgr2=0x7f6ffc104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:51:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.430+0000 7f7003f8b640 1 -- 172.21.15.67:0/254559146 shutdown_connections 2024-07-31T14:51:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:51:59.430+0000 7f7003f8b640 1 -- 172.21.15.67:0/254559146 wait complete. 2024-07-31T14:51:59.438 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:00.430 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.b status 2024-07-31T14:52:00.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:00 smithi077 ceph-mon[37327]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-07-31T14:52:01.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:00 smithi067 ceph-mon[32750]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-07-31T14:52:01.006 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:52:00 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:52:00.842+0000 7f4cad358640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:52:01.006 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:52:00 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:52:00.842+0000 7f4cad358640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:52:01.006 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:52:00 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:52:00.842+0000 7f4cad358640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:52:01.006 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:52:00 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:52:00.842+0000 7f4cad358640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:52:01.006 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:52:00 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:52:00.842+0000 7f4cad358640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-31T14:52:01.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:00 smithi073 ceph-mon[36714]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-07-31T14:52:01.851 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:01.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.859+0000 7f6896ec1640 1 -- 172.21.15.67:0/2906893242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890107dd0 msgr2=0x7f68901081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:01.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.859+0000 7f6896ec1640 1 --2- 172.21.15.67:0/2906893242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890107dd0 0x7f68901081e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f687c009900 tx=0x7f687c02a410 comp rx=0 tx=0).stop 2024-07-31T14:52:01.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.859+0000 7f6896ec1640 1 -- 172.21.15.67:0/2906893242 shutdown_connections 2024-07-31T14:52:01.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.859+0000 7f6896ec1640 1 --2- 172.21.15.67:0/2906893242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6890108720 0x7f689010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.859+0000 7f6896ec1640 1 --2- 172.21.15.67:0/2906893242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890107dd0 0x7f68901081e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.859+0000 7f6896ec1640 1 --2- 172.21.15.67:0/2906893242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6890102490 0x7f68901028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.859+0000 7f6896ec1640 1 -- 172.21.15.67:0/2906893242 >> 172.21.15.67:0/2906893242 conn(0x7f6890079160 msgr2=0x7f6890079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:01.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.859+0000 7f6896ec1640 1 -- 172.21.15.67:0/2906893242 shutdown_connections 2024-07-31T14:52:01.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.859+0000 7f6896ec1640 1 -- 172.21.15.67:0/2906893242 wait complete. 2024-07-31T14:52:01.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.860+0000 7f6896ec1640 1 Processor -- start 2024-07-31T14:52:01.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.860+0000 7f6896ec1640 1 -- start start 2024-07-31T14:52:01.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f6896ec1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890102490 0x7f689019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f6896ec1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6890107dd0 0x7f689019b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f6896ec1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6890108720 0x7f689019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f6896ec1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6890112350 con 0x7f6890102490 2024-07-31T14:52:01.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f6896ec1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f68901121d0 con 0x7f6890108720 2024-07-31T14:52:01.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f6896ec1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68901124d0 con 0x7f6890107dd0 2024-07-31T14:52:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f6894c5d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890102490 0x7f689019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f6894c5d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890102490 0x7f689019ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46412/0 (socket says 172.21.15.67:46412) 2024-07-31T14:52:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f6894c5d640 1 -- 172.21.15.67:0/2570031233 learned_addr learned my addr 172.21.15.67:0/2570031233 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f688ffff640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6890107dd0 0x7f689019b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f689545e640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6890108720 0x7f689019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f6894c5d640 1 -- 172.21.15.67:0/2570031233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6890107dd0 msgr2=0x7f689019b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f6894c5d640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6890107dd0 0x7f689019b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f6894c5d640 1 -- 172.21.15.67:0/2570031233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6890108720 msgr2=0x7f689019b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f6894c5d640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6890108720 0x7f689019b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f6894c5d640 1 -- 172.21.15.67:0/2570031233 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68901a5f70 con 0x7f6890102490 2024-07-31T14:52:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.861+0000 7f689545e640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6890108720 0x7f689019b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.862+0000 7f6894c5d640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890102490 0x7f689019ace0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f688400d950 tx=0x7f688400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:01.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.862+0000 7f688dffb640 1 -- 172.21.15.67:0/2570031233 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884018070 con 0x7f6890102490 2024-07-31T14:52:01.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.862+0000 7f6896ec1640 1 -- 172.21.15.67:0/2570031233 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6890198b30 con 0x7f6890102490 2024-07-31T14:52:01.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.862+0000 7f688dffb640 1 -- 172.21.15.67:0/2570031233 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6884009c30 con 0x7f6890102490 2024-07-31T14:52:01.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.862+0000 7f688dffb640 1 -- 172.21.15.67:0/2570031233 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688401b910 con 0x7f6890102490 2024-07-31T14:52:01.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.862+0000 7f6896ec1640 1 -- 172.21.15.67:0/2570031233 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6890199090 con 0x7f6890102490 2024-07-31T14:52:01.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.866+0000 7f689d412740 1 Processor -- start 2024-07-31T14:52:01.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.866+0000 7f689d412740 1 -- start start 2024-07-31T14:52:01.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.866+0000 7f689d412740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e2f070 0x5648f41d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.866+0000 7f689d412740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e061b0 0x5648f41d4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.866+0000 7f689d412740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41d4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.866+0000 7f689d412740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5648f414ab90 con 0x5648f3e0e390 2024-07-31T14:52:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.866+0000 7f689d412740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5648f413a090 con 0x5648f3e2f070 2024-07-31T14:52:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.866+0000 7f689d412740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5648f413e0d0 con 0x5648f3e061b0 2024-07-31T14:52:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.866+0000 7f688dffb640 1 -- 172.21.15.67:0/2570031233 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f6884012760 con 0x7f6890102490 2024-07-31T14:52:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.867+0000 7f688ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41d4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.867+0000 7f689545e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e2f070 0x5648f41d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.867+0000 7f688ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41d4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46428/0 (socket says 172.21.15.67:46428) 2024-07-31T14:52:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.867+0000 7f689545e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e2f070 0x5648f41d4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51520/0 (socket says 172.21.15.67:51520) 2024-07-31T14:52:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.867+0000 7f688ffff640 1 -- 172.21.15.67:0/1515790060 learned_addr learned my addr 172.21.15.67:0/1515790060 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.867+0000 7f6894c5d640 1 --2- 172.21.15.67:0/1515790060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e061b0 0x5648f41d4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.867+0000 7f689545e640 1 -- 172.21.15.67:0/1515790060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e061b0 msgr2=0x5648f41d4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.867+0000 7f689545e640 1 --2- 172.21.15.67:0/1515790060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e061b0 0x5648f41d4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.867+0000 7f689545e640 1 -- 172.21.15.67:0/1515790060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 msgr2=0x5648f41d4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.867+0000 7f689545e640 1 --2- 172.21.15.67:0/1515790060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41d4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.867+0000 7f689545e640 1 -- 172.21.15.67:0/1515790060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f40e0b70 con 0x5648f3e2f070 2024-07-31T14:52:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.868+0000 7f689545e640 1 --2- 172.21.15.67:0/1515790060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e2f070 0x5648f41d4550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6888009ee0 tx=0x7f6888010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25f451cdfe7cb9c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.868+0000 7f688dffb640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68800b3e80 0x7f68800b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.868+0000 7f687affd640 1 -- 172.21.15.67:0/1515790060 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888016070 con 0x5648f3e2f070 2024-07-31T14:52:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.868+0000 7f687affd640 1 -- 172.21.15.67:0/1515790060 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f688801bb10 con 0x5648f3e2f070 2024-07-31T14:52:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.868+0000 7f687affd640 1 -- 172.21.15.67:0/1515790060 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888012c00 con 0x5648f3e2f070 2024-07-31T14:52:01.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.868+0000 7f6894c5d640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68800b3e80 0x7f68800b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.868+0000 7f688dffb640 1 -- 172.21.15.67:0/2570031233 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f68840e3dd0 con 0x7f6890102490 2024-07-31T14:52:01.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.869+0000 7f6894c5d640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68800b3e80 0x7f68800b6390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f68840e12c0 tx=0x7f6884000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:01.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.869+0000 7f689d412740 1 -- 172.21.15.67:0/1515790060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e2f070 msgr2=0x5648f41d4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:01.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.869+0000 7f689d412740 1 --2- 172.21.15.67:0/1515790060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e2f070 0x5648f41d4550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6888009ee0 tx=0x7f6888010740 comp rx=0 tx=0).stop 2024-07-31T14:52:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.869+0000 7f689d412740 1 -- 172.21.15.67:0/1515790060 shutdown_connections 2024-07-31T14:52:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.869+0000 7f689d412740 1 --2- 172.21.15.67:0/1515790060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41d4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.869+0000 7f689d412740 1 --2- 172.21.15.67:0/1515790060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e061b0 0x5648f41d4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.869+0000 7f689d412740 1 --2- 172.21.15.67:0/1515790060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e2f070 0x5648f41d4550 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.869+0000 7f689d412740 1 -- 172.21.15.67:0/1515790060 >> 172.21.15.67:0/1515790060 conn(0x5648f3e23460 msgr2=0x5648f41d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.869+0000 7f689d412740 1 -- 172.21.15.67:0/1515790060 shutdown_connections 2024-07-31T14:52:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.869+0000 7f689d412740 1 -- 172.21.15.67:0/1515790060 wait complete. 2024-07-31T14:52:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.870+0000 7f689d412740 1 -- start start 2024-07-31T14:52:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.870+0000 7f689d412740 1 Processor -- start 2024-07-31T14:52:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f689d412740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e061b0 0x5648f41e88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f689d412740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41e8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f689d412740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e2f070 0x5648f41f4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f688ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41e8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f689d412740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5648f3f9d700 con 0x5648f3e0e390 2024-07-31T14:52:01.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f689d412740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5648f40dc9c0 con 0x5648f3e061b0 2024-07-31T14:52:01.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f688ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41e8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46442/0 (socket says 172.21.15.67:46442) 2024-07-31T14:52:01.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f689d412740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5648f41eb460 con 0x5648f3e2f070 2024-07-31T14:52:01.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f688ffff640 1 -- 172.21.15.67:0/2480597252 learned_addr learned my addr 172.21.15.67:0/2480597252 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:01.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f689d412740 10 client.0 _ll_register_callbacks cb 0x5648f40f74d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:01.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f689545e640 1 --2- 172.21.15.67:0/2480597252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e061b0 0x5648f41e88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f689545e640 1 --2- 172.21.15.67:0/2480597252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e2f070 0x5648f41f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f689545e640 1 -- 172.21.15.67:0/2480597252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e2f070 msgr2=0x5648f41f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:01.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f689545e640 1 --2- 172.21.15.67:0/2480597252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e2f070 0x5648f41f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f689545e640 1 -- 172.21.15.67:0/2480597252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 msgr2=0x5648f41e8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:01.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f689545e640 1 --2- 172.21.15.67:0/2480597252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41e8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.871+0000 7f689545e640 1 -- 172.21.15.67:0/2480597252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f3e9a650 con 0x5648f3e061b0 2024-07-31T14:52:01.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.872+0000 7f688ffff640 1 --2- 172.21.15.67:0/2480597252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41e8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:01.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.872+0000 7f689545e640 1 --2- 172.21.15.67:0/2480597252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e061b0 0x5648f41e88f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f688801a000 tx=0x7f68880127e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=553bacf82cc4875c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:01.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.872+0000 7f686ffff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:52:01.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.872+0000 7f686ffff640 1 -- 172.21.15.67:0/2480597252 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888016030 con 0x5648f3e061b0 2024-07-31T14:52:01.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.872+0000 7f686ffff640 1 -- 172.21.15.67:0/2480597252 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f6888012d90 con 0x5648f3e061b0 2024-07-31T14:52:01.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.872+0000 7f689d412740 1 -- 172.21.15.67:0/2480597252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f688801bb10 con 0x5648f3e061b0 2024-07-31T14:52:01.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.872+0000 7f686ffff640 1 -- 172.21.15.67:0/2480597252 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6888004a40 con 0x5648f3e061b0 2024-07-31T14:52:01.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.872+0000 7f689d412740 10 client.25484 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:01.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.872+0000 7f689d412740 1 -- 172.21.15.67:0/2480597252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5648f41f7460 con 0x5648f3e061b0 2024-07-31T14:52:01.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.873+0000 7f686ffff640 1 -- 172.21.15.67:0/2480597252 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f688801bb10 con 0x5648f3e061b0 2024-07-31T14:52:01.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.873+0000 7f686ffff640 1 -- 172.21.15.67:0/2480597252 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f688800f040 con 0x5648f3e061b0 2024-07-31T14:52:01.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.873+0000 7f689d412740 10 client.25484 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:01.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.873+0000 7f689d412740 10 client.25484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:01.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.873+0000 7f689d412740 20 client.25484 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:01.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.873+0000 7f689d412740 1 --2- 172.21.15.67:0/2480597252 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5648f3ded9f0 0x5648f41fa820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.873+0000 7f689d412740 4 client.25484 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:01.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.873+0000 7f689d412740 1 -- 172.21.15.67:0/2480597252 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648f414cae0 con 0x5648f3ded9f0 2024-07-31T14:52:01.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.874+0000 7f688ffff640 1 --2- 172.21.15.67:0/2480597252 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5648f3ded9f0 0x5648f41fa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.874+0000 7f688ffff640 1 --2- 172.21.15.67:0/2480597252 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5648f3ded9f0 0x5648f41fa820 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1176a7e5c4378e0d server_cookie=64e52cfa7044dfa6 in_seq=0 out_seq=0 2024-07-31T14:52:01.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.874+0000 7f686ffff640 10 client.25484 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:01.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.875+0000 7f686ffff640 1 -- 172.21.15.67:0/2480597252 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648f414cae0 con 0x5648f3ded9f0 2024-07-31T14:52:01.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.875+0000 7f686ffff640 10 client.25484 handle_command_reply: tid=0 2024-07-31T14:52:01.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.875+0000 7f689d412740 1 client.25484 shutdown 2024-07-31T14:52:01.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.875+0000 7f689d412740 1 -- 172.21.15.67:0/2480597252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e061b0 msgr2=0x5648f41e88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:01.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.875+0000 7f689d412740 1 --2- 172.21.15.67:0/2480597252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e061b0 0x5648f41e88f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f688801a000 tx=0x7f68880127e0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.875+0000 7f689d412740 1 -- 172.21.15.67:0/2480597252 shutdown_connections 2024-07-31T14:52:01.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.875+0000 7f689d412740 1 --2- 172.21.15.67:0/2480597252 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5648f3ded9f0 0x5648f41fa820 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.875+0000 7f686ffff640 0 client.25484 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:01.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.875+0000 7f689d412740 1 --2- 172.21.15.67:0/2480597252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e2f070 0x5648f41f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.875+0000 7f689d412740 1 --2- 172.21.15.67:0/2480597252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41e8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.875+0000 7f689d412740 1 --2- 172.21.15.67:0/2480597252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e061b0 0x5648f41e88f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.875+0000 7f689d412740 1 -- 172.21.15.67:0/2480597252 >> 172.21.15.67:0/2480597252 conn(0x5648f3e23460 msgr2=0x5648f41d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:01.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.875+0000 7f689d412740 1 -- 172.21.15.67:0/2480597252 shutdown_connections 2024-07-31T14:52:01.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.876+0000 7f689d412740 1 -- 172.21.15.67:0/2480597252 wait complete. 2024-07-31T14:52:01.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.876+0000 7f689d412740 20 client.25484 trim_cache size 0 max 16384 2024-07-31T14:52:01.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.895+0000 7f689d412740 1 Processor -- start 2024-07-31T14:52:01.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.895+0000 7f689d412740 1 -- start start 2024-07-31T14:52:01.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.895+0000 7f689d412740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e061b0 0x5648f41f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.895+0000 7f689d412740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41d5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.895+0000 7f689d412740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e2f070 0x5648f41ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.895+0000 7f689d412740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f68901ae5f0 con 0x5648f3e0e390 2024-07-31T14:52:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.896+0000 7f689d412740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f68901b0130 con 0x5648f3e061b0 2024-07-31T14:52:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.896+0000 7f689d412740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68901aecc0 con 0x5648f3e2f070 2024-07-31T14:52:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.896+0000 7f688ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.896+0000 7f688ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41d5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46456/0 (socket says 172.21.15.67:46456) 2024-07-31T14:52:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.896+0000 7f688ffff640 1 -- 172.21.15.67:0/3744972846 learned_addr learned my addr 172.21.15.67:0/3744972846 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.896+0000 7f689545e640 1 --2- 172.21.15.67:0/3744972846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e061b0 0x5648f41f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.896+0000 7f689545e640 1 --2- 172.21.15.67:0/3744972846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e2f070 0x5648f41ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.896+0000 7f688ffff640 1 -- 172.21.15.67:0/3744972846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e2f070 msgr2=0x5648f41ffeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.896+0000 7f688ffff640 1 --2- 172.21.15.67:0/3744972846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e2f070 0x5648f41ffeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.896+0000 7f688ffff640 1 -- 172.21.15.67:0/3744972846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e061b0 msgr2=0x5648f41f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.896+0000 7f688ffff640 1 --2- 172.21.15.67:0/3744972846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e061b0 0x5648f41f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.896+0000 7f688ffff640 1 -- 172.21.15.67:0/3744972846 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f41e99b0 con 0x5648f3e0e390 2024-07-31T14:52:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.896+0000 7f689545e640 1 --2- 172.21.15.67:0/3744972846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e061b0 0x5648f41f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.896+0000 7f688ffff640 1 --2- 172.21.15.67:0/3744972846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41d5c90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f687c009230 tx=0x7f687c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e7593515874a1c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.897+0000 7f687a7fc640 1 -- 172.21.15.67:0/3744972846 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c04b070 con 0x5648f3e0e390 2024-07-31T14:52:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.897+0000 7f687a7fc640 1 -- 172.21.15.67:0/3744972846 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f687c0054f0 con 0x5648f3e0e390 2024-07-31T14:52:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.897+0000 7f687a7fc640 1 -- 172.21.15.67:0/3744972846 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c035040 con 0x5648f3e0e390 2024-07-31T14:52:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.898+0000 7f689d412740 1 -- 172.21.15.67:0/3744972846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 msgr2=0x5648f41d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.898+0000 7f689d412740 1 --2- 172.21.15.67:0/3744972846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41d5c90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f687c009230 tx=0x7f687c031400 comp rx=0 tx=0).stop 2024-07-31T14:52:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.898+0000 7f689d412740 1 -- 172.21.15.67:0/3744972846 shutdown_connections 2024-07-31T14:52:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.898+0000 7f689d412740 1 --2- 172.21.15.67:0/3744972846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e2f070 0x5648f41ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.898+0000 7f689d412740 1 --2- 172.21.15.67:0/3744972846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41d5c90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.898+0000 7f689d412740 1 --2- 172.21.15.67:0/3744972846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e061b0 0x5648f41f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.898+0000 7f689d412740 1 -- 172.21.15.67:0/3744972846 >> 172.21.15.67:0/3744972846 conn(0x5648f3e23460 msgr2=0x5648f41d5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.898+0000 7f689d412740 1 -- 172.21.15.67:0/3744972846 shutdown_connections 2024-07-31T14:52:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.898+0000 7f689d412740 1 -- 172.21.15.67:0/3744972846 wait complete. 2024-07-31T14:52:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.898+0000 7f689d412740 1 -- start start 2024-07-31T14:52:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.898+0000 7f689d412740 1 Processor -- start 2024-07-31T14:52:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.899+0000 7f689d412740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e061b0 0x5648f41ec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.899+0000 7f689d412740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41ecd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.899+0000 7f689d412740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e2f070 0x5648f41ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.899+0000 7f689d412740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f68901afa60 con 0x5648f3e0e390 2024-07-31T14:52:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.899+0000 7f689d412740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f68901af390 con 0x5648f3e2f070 2024-07-31T14:52:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.899+0000 7f689d412740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68901ac5d0 con 0x5648f3e061b0 2024-07-31T14:52:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.899+0000 7f689d412740 10 client.0 _ll_register_callbacks cb 0x7f687c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.899+0000 7f688ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41ecd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.899+0000 7f688ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41ecd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46464/0 (socket says 172.21.15.67:46464) 2024-07-31T14:52:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.899+0000 7f688ffff640 1 -- 172.21.15.67:0/863868050 learned_addr learned my addr 172.21.15.67:0/863868050 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.899+0000 7f689545e640 1 --2- 172.21.15.67:0/863868050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e061b0 0x5648f41ec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.899+0000 7f689545e640 1 --2- 172.21.15.67:0/863868050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e2f070 0x5648f41ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.900+0000 7f688ffff640 1 -- 172.21.15.67:0/863868050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e061b0 msgr2=0x5648f41ec7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.900+0000 7f688ffff640 1 --2- 172.21.15.67:0/863868050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e061b0 0x5648f41ec7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.900+0000 7f688ffff640 1 -- 172.21.15.67:0/863868050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e2f070 msgr2=0x5648f41ed2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.900+0000 7f688ffff640 1 --2- 172.21.15.67:0/863868050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e2f070 0x5648f41ed2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.900+0000 7f688ffff640 1 -- 172.21.15.67:0/863868050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f42620d0 con 0x5648f3e0e390 2024-07-31T14:52:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.900+0000 7f688ffff640 1 --2- 172.21.15.67:0/863868050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41ecd30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f687c02a450 tx=0x7f687c005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a577e74430035c93 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.900+0000 7f686ffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.900+0000 7f686ffff640 1 -- 172.21.15.67:0/863868050 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c035070 con 0x5648f3e0e390 2024-07-31T14:52:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.900+0000 7f686ffff640 1 -- 172.21.15.67:0/863868050 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f687c0465d0 con 0x5648f3e0e390 2024-07-31T14:52:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.900+0000 7f689d412740 1 -- 172.21.15.67:0/863868050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f687c0054f0 con 0x5648f3e0e390 2024-07-31T14:52:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.900+0000 7f686ffff640 1 -- 172.21.15.67:0/863868050 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f687c04b040 con 0x5648f3e0e390 2024-07-31T14:52:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.900+0000 7f689d412740 10 client.15813 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.900+0000 7f689d412740 1 -- 172.21.15.67:0/863868050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5648f4263360 con 0x5648f3e0e390 2024-07-31T14:52:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.901+0000 7f686ffff640 1 -- 172.21.15.67:0/863868050 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f687c046c80 con 0x5648f3e0e390 2024-07-31T14:52:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.901+0000 7f686ffff640 1 -- 172.21.15.67:0/863868050 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f687c057030 con 0x5648f3e0e390 2024-07-31T14:52:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.901+0000 7f689d412740 10 client.15813 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.901+0000 7f689d412740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.901+0000 7f689d412740 20 client.15813 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.901+0000 7f689d412740 1 --2- 172.21.15.67:0/863868050 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5648f3dc2da0 0x5648f4265b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.901+0000 7f689d412740 4 client.15813 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:52:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.901+0000 7f689d412740 1 -- 172.21.15.67:0/863868050 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f687c0054f0 con 0x5648f3dc2da0 2024-07-31T14:52:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.901+0000 7f689545e640 1 --2- 172.21.15.67:0/863868050 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5648f3dc2da0 0x5648f4265b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.902+0000 7f686ffff640 10 client.15813 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.902+0000 7f689545e640 1 --2- 172.21.15.67:0/863868050 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5648f3dc2da0 0x5648f4265b10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79f7058a293f3ee3 server_cookie=9594c813df6676c2 in_seq=0 out_seq=0 2024-07-31T14:52:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.902+0000 7f686ffff640 1 -- 172.21.15.67:0/863868050 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f687c0054f0 con 0x5648f3dc2da0 2024-07-31T14:52:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.902+0000 7f686ffff640 10 client.15813 handle_command_reply: tid=0 2024-07-31T14:52:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.902+0000 7f689d412740 1 client.15813 shutdown 2024-07-31T14:52:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.902+0000 7f689d412740 1 -- 172.21.15.67:0/863868050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 msgr2=0x5648f41ecd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.902+0000 7f689d412740 1 --2- 172.21.15.67:0/863868050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41ecd30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f687c02a450 tx=0x7f687c005f30 comp rx=0 tx=0).stop 2024-07-31T14:52:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.902+0000 7f689d412740 1 -- 172.21.15.67:0/863868050 shutdown_connections 2024-07-31T14:52:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.902+0000 7f689d412740 1 --2- 172.21.15.67:0/863868050 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5648f3dc2da0 0x5648f4265b10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.902+0000 7f689d412740 1 --2- 172.21.15.67:0/863868050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648f3e2f070 0x5648f41ed2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.902+0000 7f689d412740 1 --2- 172.21.15.67:0/863868050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f3e0e390 0x5648f41ecd30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.902+0000 7f689d412740 1 --2- 172.21.15.67:0/863868050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f3e061b0 0x5648f41ec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.902+0000 7f689d412740 1 -- 172.21.15.67:0/863868050 >> 172.21.15.67:0/863868050 conn(0x5648f3e23460 msgr2=0x5648f41e9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.902+0000 7f686ffff640 0 client.15813 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.903+0000 7f689d412740 1 -- 172.21.15.67:0/863868050 shutdown_connections 2024-07-31T14:52:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.903+0000 7f689d412740 1 -- 172.21.15.67:0/863868050 wait complete. 2024-07-31T14:52:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.903+0000 7f689d412740 20 client.15813 trim_cache size 0 max 16384 2024-07-31T14:52:01.911 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:52:01.911 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T14:52:01.911 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T14:52:01.911 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:52:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.903+0000 7f6896ec1640 1 -- 172.21.15.67:0/2570031233 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68800b3e80 msgr2=0x7f68800b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.903+0000 7f6896ec1640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68800b3e80 0x7f68800b6390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f68840e12c0 tx=0x7f6884000bc0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.903+0000 7f6896ec1640 1 -- 172.21.15.67:0/2570031233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890102490 msgr2=0x7f689019ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.903+0000 7f6896ec1640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890102490 0x7f689019ace0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f688400d950 tx=0x7f688400de10 comp rx=0 tx=0).stop 2024-07-31T14:52:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.903+0000 7f6896ec1640 1 -- 172.21.15.67:0/2570031233 shutdown_connections 2024-07-31T14:52:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.903+0000 7f6896ec1640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68800b3e80 0x7f68800b6390 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.903+0000 7f6896ec1640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6890108720 0x7f689019b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.903+0000 7f6896ec1640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6890107dd0 0x7f689019b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.903+0000 7f6896ec1640 1 --2- 172.21.15.67:0/2570031233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890102490 0x7f689019ace0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:01.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.903+0000 7f6896ec1640 1 -- 172.21.15.67:0/2570031233 >> 172.21.15.67:0/2570031233 conn(0x7f6890079160 msgr2=0x7f6890104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:01.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.904+0000 7f6896ec1640 1 -- 172.21.15.67:0/2570031233 shutdown_connections 2024-07-31T14:52:01.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:01.904+0000 7f6896ec1640 1 -- 172.21.15.67:0/2570031233 wait complete. 2024-07-31T14:52:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.140+0000 7f2e801d0640 1 -- 172.21.15.67:0/1351728118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e78108670 msgr2=0x7f2e7810eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.140+0000 7f2e801d0640 1 --2- 172.21.15.67:0/1351728118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e78108670 0x7f2e7810eea0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009ca0 tx=0x7f2e7002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.140+0000 7f2e801d0640 1 -- 172.21.15.67:0/1351728118 shutdown_connections 2024-07-31T14:52:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.140+0000 7f2e801d0640 1 --2- 172.21.15.67:0/1351728118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e78108670 0x7f2e7810eea0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.140+0000 7f2e801d0640 1 --2- 172.21.15.67:0/1351728118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e78107d20 0x7f2e78108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.140+0000 7f2e801d0640 1 --2- 172.21.15.67:0/1351728118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e781023f0 0x7f2e78102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.140+0000 7f2e801d0640 1 -- 172.21.15.67:0/1351728118 >> 172.21.15.67:0/1351728118 conn(0x7f2e780fddf0 msgr2=0x7f2e78100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.140+0000 7f2e801d0640 1 -- 172.21.15.67:0/1351728118 shutdown_connections 2024-07-31T14:52:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.140+0000 7f2e801d0640 1 -- 172.21.15.67:0/1351728118 wait complete. 2024-07-31T14:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e801d0640 1 Processor -- start 2024-07-31T14:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e801d0640 1 -- start start 2024-07-31T14:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e801d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e781023f0 0x7f2e7819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e801d0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e78107d20 0x7f2e7819b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e801d0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e78108670 0x7f2e7819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e801d0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2e78112320 con 0x7f2e78108670 2024-07-31T14:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e801d0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781121a0 con 0x7f2e78107d20 2024-07-31T14:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e801d0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781124a0 con 0x7f2e781023f0 2024-07-31T14:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e7e76d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e78108670 0x7f2e7819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e7e76d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e78108670 0x7f2e7819b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46474/0 (socket says 172.21.15.67:46474) 2024-07-31T14:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e7e76d640 1 -- 172.21.15.67:0/2788736677 learned_addr learned my addr 172.21.15.67:0/2788736677 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e7df6c640 1 --2- 172.21.15.67:0/2788736677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e781023f0 0x7f2e7819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e7d76b640 1 --2- 172.21.15.67:0/2788736677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e78107d20 0x7f2e7819b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e7e76d640 1 -- 172.21.15.67:0/2788736677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e781023f0 msgr2=0x7f2e7819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e7e76d640 1 --2- 172.21.15.67:0/2788736677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e781023f0 0x7f2e7819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e7e76d640 1 -- 172.21.15.67:0/2788736677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e78107d20 msgr2=0x7f2e7819b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.141+0000 7f2e7e76d640 1 --2- 172.21.15.67:0/2788736677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e78107d20 0x7f2e7819b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.142+0000 7f2e7e76d640 1 -- 172.21.15.67:0/2788736677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e781a5e80 con 0x7f2e78108670 2024-07-31T14:52:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.142+0000 7f2e7e76d640 1 --2- 172.21.15.67:0/2788736677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e78108670 0x7f2e7819b670 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2e700098d0 tx=0x7f2e700317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.142+0000 7f2e6effd640 1 -- 172.21.15.67:0/2788736677 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70003b70 con 0x7f2e78108670 2024-07-31T14:52:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.142+0000 7f2e801d0640 1 -- 172.21.15.67:0/2788736677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e781a6110 con 0x7f2e78108670 2024-07-31T14:52:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.142+0000 7f2e6effd640 1 -- 172.21.15.67:0/2788736677 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e70003d10 con 0x7f2e78108670 2024-07-31T14:52:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.142+0000 7f2e6effd640 1 -- 172.21.15.67:0/2788736677 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7002fa20 con 0x7f2e78108670 2024-07-31T14:52:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.142+0000 7f2e801d0640 1 -- 172.21.15.67:0/2788736677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e781a6660 con 0x7f2e78108670 2024-07-31T14:52:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.145+0000 7f2e6effd640 1 -- 172.21.15.67:0/2788736677 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f2e7002fc00 con 0x7f2e78108670 2024-07-31T14:52:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.145+0000 7f2e8719c740 1 Processor -- start 2024-07-31T14:52:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.145+0000 7f2e8719c740 1 -- start start 2024-07-31T14:52:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.145+0000 7f2e8719c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d331b0 0x55f3270d5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.145+0000 7f2e8719c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d0a2f0 0x55f3270d6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.145+0000 7f2e8719c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d124d0 0x55f3270d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.145+0000 7f2e8719c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f32704b3b0 con 0x55f326d124d0 2024-07-31T14:52:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.145+0000 7f2e8719c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f32703a8b0 con 0x55f326d331b0 2024-07-31T14:52:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.145+0000 7f2e8719c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f32703e8f0 con 0x55f326d0a2f0 2024-07-31T14:52:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e7d76b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d124d0 0x55f3270d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e7d76b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d124d0 0x55f3270d6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46488/0 (socket says 172.21.15.67:46488) 2024-07-31T14:52:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e7d76b640 1 -- 172.21.15.67:0/4039904661 learned_addr learned my addr 172.21.15.67:0/4039904661 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e7df6c640 1 --2- 172.21.15.67:0/4039904661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d0a2f0 0x55f3270d6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e7d76b640 1 --2- 172.21.15.67:0/4039904661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d331b0 0x55f3270d5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e7d76b640 1 -- 172.21.15.67:0/4039904661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d0a2f0 msgr2=0x55f3270d6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e7d76b640 1 --2- 172.21.15.67:0/4039904661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d0a2f0 0x55f3270d6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e7d76b640 1 -- 172.21.15.67:0/4039904661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d331b0 msgr2=0x55f3270d5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e7d76b640 1 --2- 172.21.15.67:0/4039904661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d331b0 0x55f3270d5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e7d76b640 1 -- 172.21.15.67:0/4039904661 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f326fe19a0 con 0x55f326d124d0 2024-07-31T14:52:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e6effd640 1 --2- 172.21.15.67:0/2788736677 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2e640b3dd0 0x7f2e640b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e7d76b640 1 --2- 172.21.15.67:0/4039904661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d124d0 0x55f3270d6760 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2e680147f0 tx=0x7f2e680095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f859330eeeed4037 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e7df6c640 1 --2- 172.21.15.67:0/2788736677 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2e640b3dd0 0x7f2e640b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e6effd640 1 -- 172.21.15.67:0/2788736677 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f2e701058f0 con 0x7f2e78108670 2024-07-31T14:52:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.146+0000 7f2e63fff640 1 -- 172.21.15.67:0/4039904661 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68014e30 con 0x55f326d124d0 2024-07-31T14:52:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.147+0000 7f2e63fff640 1 -- 172.21.15.67:0/4039904661 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e6800f760 con 0x55f326d124d0 2024-07-31T14:52:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.147+0000 7f2e63fff640 1 -- 172.21.15.67:0/4039904661 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68003850 con 0x55f326d124d0 2024-07-31T14:52:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.147+0000 7f2e8719c740 1 -- 172.21.15.67:0/4039904661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d124d0 msgr2=0x55f3270d6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.147+0000 7f2e8719c740 1 --2- 172.21.15.67:0/4039904661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d124d0 0x55f3270d6760 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2e680147f0 tx=0x7f2e680095d0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.147+0000 7f2e7df6c640 1 --2- 172.21.15.67:0/2788736677 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2e640b3dd0 0x7f2e640b62e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55f3270d4960 tx=0x7f2e7400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.147+0000 7f2e8719c740 1 -- 172.21.15.67:0/4039904661 shutdown_connections 2024-07-31T14:52:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.147+0000 7f2e8719c740 1 --2- 172.21.15.67:0/4039904661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d124d0 0x55f3270d6760 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.147+0000 7f2e8719c740 1 --2- 172.21.15.67:0/4039904661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d0a2f0 0x55f3270d6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.147+0000 7f2e8719c740 1 --2- 172.21.15.67:0/4039904661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d331b0 0x55f3270d5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.147+0000 7f2e8719c740 1 -- 172.21.15.67:0/4039904661 >> 172.21.15.67:0/4039904661 conn(0x55f326d27300 msgr2=0x55f3270d3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.148+0000 7f2e8719c740 1 -- 172.21.15.67:0/4039904661 shutdown_connections 2024-07-31T14:52:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.148+0000 7f2e8719c740 1 -- 172.21.15.67:0/4039904661 wait complete. 2024-07-31T14:52:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.148+0000 7f2e8719c740 1 -- start start 2024-07-31T14:52:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.148+0000 7f2e8719c740 1 Processor -- start 2024-07-31T14:52:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e8719c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270e5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e8719c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d124d0 0x55f3270e61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e8719c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d331b0 0x55f3270f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e8719c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f326f74ab0 con 0x55f326d0a2f0 2024-07-31T14:52:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e8719c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f326fc2cf0 con 0x55f326d124d0 2024-07-31T14:52:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e8719c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f3270e8840 con 0x55f326d331b0 2024-07-31T14:52:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e8719c740 10 client.0 _ll_register_callbacks cb 0x55f3270349f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e7df6c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270e5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e7df6c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270e5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46498/0 (socket says 172.21.15.67:46498) 2024-07-31T14:52:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e7df6c640 1 -- 172.21.15.67:0/2077215643 learned_addr learned my addr 172.21.15.67:0/2077215643 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e7e76d640 1 --2- 172.21.15.67:0/2077215643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d331b0 0x55f3270f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e7d76b640 1 --2- 172.21.15.67:0/2077215643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d124d0 0x55f3270e61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e7df6c640 1 -- 172.21.15.67:0/2077215643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d331b0 msgr2=0x55f3270f1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e7df6c640 1 --2- 172.21.15.67:0/2077215643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d331b0 0x55f3270f1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e7df6c640 1 -- 172.21.15.67:0/2077215643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d124d0 msgr2=0x55f3270e61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e7df6c640 1 --2- 172.21.15.67:0/2077215643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d124d0 0x55f3270e61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.149+0000 7f2e7df6c640 1 -- 172.21.15.67:0/2077215643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f326d9e790 con 0x55f326d0a2f0 2024-07-31T14:52:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.150+0000 7f2e7df6c640 1 --2- 172.21.15.67:0/2077215643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270e5c70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400b620 tx=0x7f2e740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5d5056bf18dc91 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.150+0000 7f2e617fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.150+0000 7f2e617fa640 1 -- 172.21.15.67:0/2077215643 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7402c060 con 0x55f326d0a2f0 2024-07-31T14:52:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.150+0000 7f2e8719c740 1 -- 172.21.15.67:0/2077215643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e6800f760 con 0x55f326d0a2f0 2024-07-31T14:52:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.150+0000 7f2e617fa640 1 -- 172.21.15.67:0/2077215643 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f2e74017030 con 0x55f326d0a2f0 2024-07-31T14:52:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.150+0000 7f2e617fa640 1 -- 172.21.15.67:0/2077215643 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e74033050 con 0x55f326d0a2f0 2024-07-31T14:52:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.150+0000 7f2e8719c740 10 client.15825 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.150+0000 7f2e8719c740 1 -- 172.21.15.67:0/2077215643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f326e17620 con 0x55f326d0a2f0 2024-07-31T14:52:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.150+0000 7f2e617fa640 1 -- 172.21.15.67:0/2077215643 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e6800f760 con 0x55f326d0a2f0 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.151+0000 7f2e617fa640 1 -- 172.21.15.67:0/2077215643 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2e7401c3f0 con 0x55f326d0a2f0 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.151+0000 7f2e8719c740 10 client.15825 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.151+0000 7f2e8719c740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.151+0000 7f2e8719c740 20 client.15825 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.151+0000 7f2e8719c740 1 --2- 172.21.15.67:0/2077215643 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f326cf1d80 0x55f3270f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.151+0000 7f2e8719c740 4 client.15825 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.151+0000 7f2e8719c740 1 -- 172.21.15.67:0/2077215643 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f32704d300 con 0x55f326cf1d80 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.151+0000 7f2e7d76b640 1 --2- 172.21.15.67:0/2077215643 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f326cf1d80 0x55f3270f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.152+0000 7f2e617fa640 10 client.15825 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.152+0000 7f2e7d76b640 1 --2- 172.21.15.67:0/2077215643 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f326cf1d80 0x55f3270f7c60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a20b2cd60f2f2b44 server_cookie=4b6020aefa845380 in_seq=0 out_seq=0 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.152+0000 7f2e617fa640 1 -- 172.21.15.67:0/2077215643 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f32704d300 con 0x55f326cf1d80 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.152+0000 7f2e617fa640 10 client.15825 handle_command_reply: tid=0 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.152+0000 7f2e8719c740 1 client.15825 shutdown 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.153+0000 7f2e8719c740 1 -- 172.21.15.67:0/2077215643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 msgr2=0x55f3270e5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.153+0000 7f2e8719c740 1 --2- 172.21.15.67:0/2077215643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270e5c70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400b620 tx=0x7f2e740046c0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.153+0000 7f2e8719c740 1 -- 172.21.15.67:0/2077215643 shutdown_connections 2024-07-31T14:52:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.153+0000 7f2e8719c740 1 --2- 172.21.15.67:0/2077215643 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f326cf1d80 0x55f3270f7c60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.153+0000 7f2e8719c740 1 --2- 172.21.15.67:0/2077215643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d331b0 0x55f3270f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.153+0000 7f2e617fa640 0 client.15825 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.153+0000 7f2e8719c740 1 --2- 172.21.15.67:0/2077215643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d124d0 0x55f3270e61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.153+0000 7f2e8719c740 1 --2- 172.21.15.67:0/2077215643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270e5c70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.153+0000 7f2e8719c740 1 -- 172.21.15.67:0/2077215643 >> 172.21.15.67:0/2077215643 conn(0x55f326d27300 msgr2=0x55f3270d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.153+0000 7f2e8719c740 1 -- 172.21.15.67:0/2077215643 shutdown_connections 2024-07-31T14:52:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.153+0000 7f2e8719c740 1 -- 172.21.15.67:0/2077215643 wait complete. 2024-07-31T14:52:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.153+0000 7f2e8719c740 20 client.15825 trim_cache size 0 max 16384 2024-07-31T14:52:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.169+0000 7f2e8719c740 1 Processor -- start 2024-07-31T14:52:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.169+0000 7f2e8719c740 1 -- start start 2024-07-31T14:52:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e8719c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326cf1d80 0x55f3270d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e8719c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e8719c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d124d0 0x55f3270d7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e8719c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781ae440 con 0x55f326d0a2f0 2024-07-31T14:52:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e8719c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781aff80 con 0x55f326cf1d80 2024-07-31T14:52:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e8719c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781aeb10 con 0x55f326d124d0 2024-07-31T14:52:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e7d76b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e7d76b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270e5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46508/0 (socket says 172.21.15.67:46508) 2024-07-31T14:52:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e7d76b640 1 -- 172.21.15.67:0/1425813581 learned_addr learned my addr 172.21.15.67:0/1425813581 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e7df6c640 1 --2- 172.21.15.67:0/1425813581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326cf1d80 0x55f3270d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e7e76d640 1 --2- 172.21.15.67:0/1425813581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d124d0 0x55f3270d7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e7d76b640 1 -- 172.21.15.67:0/1425813581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d124d0 msgr2=0x55f3270d7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e7d76b640 1 --2- 172.21.15.67:0/1425813581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d124d0 0x55f3270d7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e7d76b640 1 -- 172.21.15.67:0/1425813581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326cf1d80 msgr2=0x55f3270d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e7d76b640 1 --2- 172.21.15.67:0/1425813581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326cf1d80 0x55f3270d3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e7d76b640 1 -- 172.21.15.67:0/1425813581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3270e69a0 con 0x55f326d0a2f0 2024-07-31T14:52:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e7df6c640 1 --2- 172.21.15.67:0/1425813581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326cf1d80 0x55f3270d3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.170+0000 7f2e7e76d640 1 --2- 172.21.15.67:0/1425813581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d124d0 0x55f3270d7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.171+0000 7f2e7d76b640 1 --2- 172.21.15.67:0/1425813581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270e5f40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2e680147f0 tx=0x7f2e68005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a583cd994a517031 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.171+0000 7f2e637fe640 1 -- 172.21.15.67:0/1425813581 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6802e070 con 0x55f326d0a2f0 2024-07-31T14:52:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.171+0000 7f2e637fe640 1 -- 172.21.15.67:0/1425813581 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e6800f730 con 0x55f326d0a2f0 2024-07-31T14:52:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.171+0000 7f2e637fe640 1 -- 172.21.15.67:0/1425813581 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e680048a0 con 0x55f326d0a2f0 2024-07-31T14:52:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.172+0000 7f2e8719c740 1 -- 172.21.15.67:0/1425813581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 msgr2=0x55f3270e5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.172+0000 7f2e8719c740 1 --2- 172.21.15.67:0/1425813581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270e5f40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2e680147f0 tx=0x7f2e68005f90 comp rx=0 tx=0).stop 2024-07-31T14:52:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.172+0000 7f2e8719c740 1 -- 172.21.15.67:0/1425813581 shutdown_connections 2024-07-31T14:52:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.172+0000 7f2e8719c740 1 --2- 172.21.15.67:0/1425813581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326d124d0 0x55f3270d7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.172+0000 7f2e8719c740 1 --2- 172.21.15.67:0/1425813581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270e5f40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.172+0000 7f2e8719c740 1 --2- 172.21.15.67:0/1425813581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326cf1d80 0x55f3270d3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.172+0000 7f2e8719c740 1 -- 172.21.15.67:0/1425813581 >> 172.21.15.67:0/1425813581 conn(0x55f326d27300 msgr2=0x55f3270d6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.172+0000 7f2e8719c740 1 -- 172.21.15.67:0/1425813581 shutdown_connections 2024-07-31T14:52:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.172+0000 7f2e8719c740 1 -- 172.21.15.67:0/1425813581 wait complete. 2024-07-31T14:52:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.173+0000 7f2e8719c740 1 -- start start 2024-07-31T14:52:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.173+0000 7f2e8719c740 1 Processor -- start 2024-07-31T14:52:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.173+0000 7f2e8719c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326cf1d80 0x55f3270ee3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.173+0000 7f2e8719c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270eab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.173+0000 7f2e8719c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d124d0 0x55f3270eb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.173+0000 7f2e8719c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781af8b0 con 0x55f326d0a2f0 2024-07-31T14:52:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.173+0000 7f2e8719c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781af1e0 con 0x55f326d124d0 2024-07-31T14:52:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.173+0000 7f2e8719c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781ac420 con 0x55f326cf1d80 2024-07-31T14:52:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.173+0000 7f2e8719c740 10 client.0 _ll_register_callbacks cb 0x7f2e74004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.173+0000 7f2e7d76b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270eab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.173+0000 7f2e7d76b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270eab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46516/0 (socket says 172.21.15.67:46516) 2024-07-31T14:52:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.174+0000 7f2e7d76b640 1 -- 172.21.15.67:0/1515062364 learned_addr learned my addr 172.21.15.67:0/1515062364 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.174+0000 7f2e7df6c640 1 --2- 172.21.15.67:0/1515062364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326cf1d80 0x55f3270ee3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.174+0000 7f2e7e76d640 1 --2- 172.21.15.67:0/1515062364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d124d0 0x55f3270eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.174+0000 7f2e7d76b640 1 -- 172.21.15.67:0/1515062364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326cf1d80 msgr2=0x55f3270ee3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.174+0000 7f2e7d76b640 1 --2- 172.21.15.67:0/1515062364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326cf1d80 0x55f3270ee3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.174+0000 7f2e7d76b640 1 -- 172.21.15.67:0/1515062364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d124d0 msgr2=0x55f3270eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.174+0000 7f2e7d76b640 1 --2- 172.21.15.67:0/1515062364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d124d0 0x55f3270eb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.174+0000 7f2e7d76b640 1 -- 172.21.15.67:0/1515062364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f32715e440 con 0x55f326d0a2f0 2024-07-31T14:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.174+0000 7f2e7d76b640 1 --2- 172.21.15.67:0/1515062364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270eab10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2e680058c0 tx=0x7f2e68009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a623ebcbe80e4494 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.174+0000 7f2e617fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.174+0000 7f2e617fa640 1 -- 172.21.15.67:0/1515062364 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68039060 con 0x55f326d0a2f0 2024-07-31T14:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.174+0000 7f2e617fa640 1 -- 172.21.15.67:0/1515062364 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f2e6800fae0 con 0x55f326d0a2f0 2024-07-31T14:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.174+0000 7f2e8719c740 1 -- 172.21.15.67:0/1515062364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e6800f730 con 0x55f326d0a2f0 2024-07-31T14:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.175+0000 7f2e617fa640 1 -- 172.21.15.67:0/1515062364 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e6802e040 con 0x55f326d0a2f0 2024-07-31T14:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.175+0000 7f2e8719c740 10 client.15831 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.175+0000 7f2e8719c740 1 -- 172.21.15.67:0/1515062364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f32715f480 con 0x55f326d0a2f0 2024-07-31T14:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.175+0000 7f2e617fa640 1 -- 172.21.15.67:0/1515062364 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e6800f730 con 0x55f326d0a2f0 2024-07-31T14:52:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.175+0000 7f2e617fa640 1 -- 172.21.15.67:0/1515062364 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2e680048f0 con 0x55f326d0a2f0 2024-07-31T14:52:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.175+0000 7f2e8719c740 10 client.15831 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.175+0000 7f2e8719c740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.175+0000 7f2e8719c740 20 client.15831 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.175+0000 7f2e8719c740 1 --2- 172.21.15.67:0/1515062364 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f326d331b0 0x55f327161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.175+0000 7f2e8719c740 4 client.15831 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:52:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.175+0000 7f2e8719c740 1 -- 172.21.15.67:0/1515062364 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f327048da0 con 0x55f326d331b0 2024-07-31T14:52:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.175+0000 7f2e7df6c640 1 --2- 172.21.15.67:0/1515062364 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f326d331b0 0x55f327161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.176+0000 7f2e7df6c640 1 --2- 172.21.15.67:0/1515062364 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f326d331b0 0x55f327161ae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fc8f2500416a59f server_cookie=f1ede3beaa521adc in_seq=0 out_seq=0 2024-07-31T14:52:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:02.176+0000 7f2e617fa640 10 client.15831 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:02.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:02 smithi077 ceph-mon[37327]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-07-31T14:52:02.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:02 smithi077 ceph-mon[37327]: mgrmap e26: x(active, since 11m), standbys: z, y 2024-07-31T14:52:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:02 smithi067 ceph-mon[32750]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-07-31T14:52:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:02 smithi067 ceph-mon[32750]: mgrmap e26: x(active, since 11m), standbys: z, y 2024-07-31T14:52:03.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:02 smithi073 ceph-mon[36714]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-07-31T14:52:03.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:02 smithi073 ceph-mon[36714]: mgrmap e26: x(active, since 11m), standbys: z, y 2024-07-31T14:52:03.326 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T14:52:03.327 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:52:04.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.385+0000 7f1cc7fd7640 1 -- 172.21.15.67:0/4146478402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0108760 msgr2=0x7f1cc010ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:04.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.385+0000 7f1cc7fd7640 1 --2- 172.21.15.67:0/4146478402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0108760 0x7f1cc010ef90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009ca0 tx=0x7f1cb802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.385+0000 7f1cc7fd7640 1 -- 172.21.15.67:0/4146478402 shutdown_connections 2024-07-31T14:52:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.385+0000 7f1cc7fd7640 1 --2- 172.21.15.67:0/4146478402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0108760 0x7f1cc010ef90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.385+0000 7f1cc7fd7640 1 --2- 172.21.15.67:0/4146478402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc0107e10 0x7f1cc0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.385+0000 7f1cc7fd7640 1 --2- 172.21.15.67:0/4146478402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc01024d0 0x7f1cc01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.385+0000 7f1cc7fd7640 1 -- 172.21.15.67:0/4146478402 >> 172.21.15.67:0/4146478402 conn(0x7f1cc0079190 msgr2=0x7f1cc00795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.385+0000 7f1cc7fd7640 1 -- 172.21.15.67:0/4146478402 shutdown_connections 2024-07-31T14:52:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.386+0000 7f1cc7fd7640 1 -- 172.21.15.67:0/4146478402 wait complete. 2024-07-31T14:52:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.387+0000 7f1cc7fd7640 1 Processor -- start 2024-07-31T14:52:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.387+0000 7f1cc7fd7640 1 -- start start 2024-07-31T14:52:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.387+0000 7f1cc7fd7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc01024d0 0x7f1cc019c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.387+0000 7f1cc7fd7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc0107e10 0x7f1cc019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.387+0000 7f1cc7fd7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0108760 0x7f1cc01a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.387+0000 7f1cc7fd7640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01123c0 con 0x7f1cc0108760 2024-07-31T14:52:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.387+0000 7f1cc7fd7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0112240 con 0x7f1cc01024d0 2024-07-31T14:52:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.387+0000 7f1cc7fd7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0112540 con 0x7f1cc0107e10 2024-07-31T14:52:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.387+0000 7f1cc6574640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0108760 0x7f1cc01a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.387+0000 7f1cc6574640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0108760 0x7f1cc01a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46526/0 (socket says 172.21.15.67:46526) 2024-07-31T14:52:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.387+0000 7f1cc6574640 1 -- 172.21.15.67:0/1874997893 learned_addr learned my addr 172.21.15.67:0/1874997893 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.387+0000 7f1cc5d73640 1 --2- 172.21.15.67:0/1874997893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc01024d0 0x7f1cc019c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.387+0000 7f1cc5572640 1 --2- 172.21.15.67:0/1874997893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc0107e10 0x7f1cc019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.388+0000 7f1cc6574640 1 -- 172.21.15.67:0/1874997893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc0107e10 msgr2=0x7f1cc019c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.388+0000 7f1cc6574640 1 --2- 172.21.15.67:0/1874997893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc0107e10 0x7f1cc019c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.388+0000 7f1cc6574640 1 -- 172.21.15.67:0/1874997893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc01024d0 msgr2=0x7f1cc019c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.388+0000 7f1cc6574640 1 --2- 172.21.15.67:0/1874997893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc01024d0 0x7f1cc019c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.388+0000 7f1cc6574640 1 -- 172.21.15.67:0/1874997893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc01a1fe0 con 0x7f1cc0108760 2024-07-31T14:52:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.388+0000 7f1cc6574640 1 --2- 172.21.15.67:0/1874997893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0108760 0x7f1cc01a18b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80098d0 tx=0x7f1cb80317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:04.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.388+0000 7f1cb6ffd640 1 -- 172.21.15.67:0/1874997893 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8003b70 con 0x7f1cc0108760 2024-07-31T14:52:04.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.388+0000 7f1cb6ffd640 1 -- 172.21.15.67:0/1874997893 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb8003d10 con 0x7f1cc0108760 2024-07-31T14:52:04.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.388+0000 7f1cb6ffd640 1 -- 172.21.15.67:0/1874997893 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb802fa20 con 0x7f1cc0108760 2024-07-31T14:52:04.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.388+0000 7f1cc7fd7640 1 -- 172.21.15.67:0/1874997893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc01a2270 con 0x7f1cc0108760 2024-07-31T14:52:04.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.388+0000 7f1cc7fd7640 1 -- 172.21.15.67:0/1874997893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc01a8a30 con 0x7f1cc0108760 2024-07-31T14:52:04.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.391+0000 7f1cce528740 1 Processor -- start 2024-07-31T14:52:04.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.392+0000 7f1cb6ffd640 1 -- 172.21.15.67:0/1874997893 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f1cb802fc40 con 0x7f1cc0108760 2024-07-31T14:52:04.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.392+0000 7f1cce528740 1 -- start start 2024-07-31T14:52:04.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.392+0000 7f1cce528740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde23f60 0x5652de1c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.392+0000 7f1cce528740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dddfb0a0 0x5652de1c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.392+0000 7f1cce528740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde03280 0x5652de1c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.392+0000 7f1cce528740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5652de13f940 con 0x5652dddfb0a0 2024-07-31T14:52:04.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cce528740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5652de12ee40 con 0x5652dde23f60 2024-07-31T14:52:04.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cce528740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5652de132e80 con 0x5652dde03280 2024-07-31T14:52:04.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cc5d73640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dddfb0a0 0x5652de1c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cc5d73640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dddfb0a0 0x5652de1c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46538/0 (socket says 172.21.15.67:46538) 2024-07-31T14:52:04.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cc5d73640 1 -- 172.21.15.67:0/857042211 learned_addr learned my addr 172.21.15.67:0/857042211 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:04.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cc5572640 1 --2- 172.21.15.67:0/857042211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde23f60 0x5652de1c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cc5572640 1 --2- 172.21.15.67:0/857042211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde03280 0x5652de1c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cb6ffd640 1 --2- 172.21.15.67:0/1874997893 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1cac0b3e50 0x7f1cac0b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cc5d73640 1 -- 172.21.15.67:0/857042211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde03280 msgr2=0x5652de1c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:04.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cc5d73640 1 --2- 172.21.15.67:0/857042211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde03280 0x5652de1c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cc5d73640 1 -- 172.21.15.67:0/857042211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde23f60 msgr2=0x5652de1c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:04.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cc5d73640 1 --2- 172.21.15.67:0/857042211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde23f60 0x5652de1c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cc5d73640 1 -- 172.21.15.67:0/857042211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652de0d5a60 con 0x5652dddfb0a0 2024-07-31T14:52:04.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cc5572640 1 --2- 172.21.15.67:0/857042211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde03280 0x5652de1c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:04.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cc6574640 1 --2- 172.21.15.67:0/1874997893 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1cac0b3e50 0x7f1cac0b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cb6ffd640 1 -- 172.21.15.67:0/1874997893 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f1cb81057a0 con 0x7f1cc0108760 2024-07-31T14:52:04.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.393+0000 7f1cc5d73640 1 --2- 172.21.15.67:0/857042211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dddfb0a0 0x5652de1c98e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009cf0 tx=0x7f1cbc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c001f5780130b221 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:04.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.394+0000 7f1cabfff640 1 -- 172.21.15.67:0/857042211 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc004230 con 0x5652dddfb0a0 2024-07-31T14:52:04.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.394+0000 7f1cabfff640 1 -- 172.21.15.67:0/857042211 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cbc0043d0 con 0x5652dddfb0a0 2024-07-31T14:52:04.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.394+0000 7f1cabfff640 1 -- 172.21.15.67:0/857042211 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc019780 con 0x5652dddfb0a0 2024-07-31T14:52:04.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.395+0000 7f1cce528740 1 -- 172.21.15.67:0/857042211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dddfb0a0 msgr2=0x5652de1c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:04.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.395+0000 7f1cce528740 1 --2- 172.21.15.67:0/857042211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dddfb0a0 0x5652de1c98e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009cf0 tx=0x7f1cbc011810 comp rx=0 tx=0).stop 2024-07-31T14:52:04.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.395+0000 7f1cc6574640 1 --2- 172.21.15.67:0/1874997893 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1cac0b3e50 0x7f1cac0b6360 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8102c90 tx=0x7f1cb80984f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:04.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.395+0000 7f1cce528740 1 -- 172.21.15.67:0/857042211 shutdown_connections 2024-07-31T14:52:04.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.395+0000 7f1cce528740 1 --2- 172.21.15.67:0/857042211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde03280 0x5652de1c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.395+0000 7f1cce528740 1 --2- 172.21.15.67:0/857042211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dddfb0a0 0x5652de1c98e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.395+0000 7f1cce528740 1 --2- 172.21.15.67:0/857042211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde23f60 0x5652de1c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.395+0000 7f1cce528740 1 -- 172.21.15.67:0/857042211 >> 172.21.15.67:0/857042211 conn(0x5652dde18350 msgr2=0x5652de1c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:04.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.395+0000 7f1cce528740 1 -- 172.21.15.67:0/857042211 shutdown_connections 2024-07-31T14:52:04.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.395+0000 7f1cce528740 1 -- 172.21.15.67:0/857042211 wait complete. 2024-07-31T14:52:04.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.395+0000 7f1cce528740 1 -- start start 2024-07-31T14:52:04.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.395+0000 7f1cce528740 1 Processor -- start 2024-07-31T14:52:04.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cce528740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dddfb0a0 0x5652de1d4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cce528740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dde03280 0x5652de1d5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cce528740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde23f60 0x5652de1e0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cce528740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5652ddf925f0 con 0x5652dde03280 2024-07-31T14:52:04.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cce528740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5652de0d18b0 con 0x5652dde23f60 2024-07-31T14:52:04.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cce528740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5652de1d7b80 con 0x5652dddfb0a0 2024-07-31T14:52:04.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cce528740 10 client.0 _ll_register_callbacks cb 0x5652de0db7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:04.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cc5d73640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dde03280 0x5652de1d5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cc5d73640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dde03280 0x5652de1d5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46544/0 (socket says 172.21.15.67:46544) 2024-07-31T14:52:04.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cc5d73640 1 -- 172.21.15.67:0/3100465418 learned_addr learned my addr 172.21.15.67:0/3100465418 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:04.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cc5572640 1 --2- 172.21.15.67:0/3100465418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dddfb0a0 0x5652de1d4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cc5572640 1 --2- 172.21.15.67:0/3100465418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde23f60 0x5652de1e0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cc5d73640 1 -- 172.21.15.67:0/3100465418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dddfb0a0 msgr2=0x5652de1d4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:04.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cc5d73640 1 --2- 172.21.15.67:0/3100465418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dddfb0a0 0x5652de1d4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cc5d73640 1 -- 172.21.15.67:0/3100465418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde23f60 msgr2=0x5652de1e0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:04.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cc5d73640 1 --2- 172.21.15.67:0/3100465418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde23f60 0x5652de1e0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cc5d73640 1 -- 172.21.15.67:0/3100465418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652dde8f540 con 0x5652dde03280 2024-07-31T14:52:04.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.396+0000 7f1cc5572640 1 --2- 172.21.15.67:0/3100465418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde23f60 0x5652de1e0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:04.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.397+0000 7f1cc5d73640 1 --2- 172.21.15.67:0/3100465418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dde03280 0x5652de1d5480 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc005e00 tx=0x7f1cbc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbcda70126132c0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:04.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.397+0000 7f1ca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:04.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.397+0000 7f1ca97fa640 1 -- 172.21.15.67:0/3100465418 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc027070 con 0x5652dde03280 2024-07-31T14:52:04.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.397+0000 7f1ca97fa640 1 -- 172.21.15.67:0/3100465418 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f1cbc005240 con 0x5652dde03280 2024-07-31T14:52:04.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.397+0000 7f1cce528740 1 -- 172.21.15.67:0/3100465418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cbc0043d0 con 0x5652dde03280 2024-07-31T14:52:04.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.397+0000 7f1ca97fa640 1 -- 172.21.15.67:0/3100465418 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1cbc033050 con 0x5652dde03280 2024-07-31T14:52:04.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.397+0000 7f1cce528740 10 client.15843 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:04.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.397+0000 7f1cce528740 1 -- 172.21.15.67:0/3100465418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5652de1e3b40 con 0x5652dde03280 2024-07-31T14:52:04.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.397+0000 7f1ca97fa640 1 -- 172.21.15.67:0/3100465418 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cbc0043d0 con 0x5652dde03280 2024-07-31T14:52:04.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.398+0000 7f1ca97fa640 1 -- 172.21.15.67:0/3100465418 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1cbc004b70 con 0x5652dde03280 2024-07-31T14:52:04.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.398+0000 7f1cce528740 10 client.15843 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:04.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.398+0000 7f1cce528740 10 client.15843 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-31T14:52:04.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.398+0000 7f1cce528740 20 client.15843 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:04.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.398+0000 7f1cce528740 1 --2- 172.21.15.67:0/3100465418 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5652ddde28e0 0x5652de1e7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.398+0000 7f1cce528740 4 client.15843 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:04.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.398+0000 7f1cce528740 1 -- 172.21.15.67:0/3100465418 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652de141890 con 0x5652ddde28e0 2024-07-31T14:52:04.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.398+0000 7f1cc5572640 1 --2- 172.21.15.67:0/3100465418 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5652ddde28e0 0x5652de1e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.398+0000 7f1cc5572640 1 --2- 172.21.15.67:0/3100465418 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5652ddde28e0 0x5652de1e7380 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cf16d8f48dc3473 server_cookie=1a4854f59c62cdb9 in_seq=0 out_seq=0 2024-07-31T14:52:04.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.398+0000 7f1ca97fa640 10 client.15843 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:04.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.399+0000 7f1ca97fa640 1 -- 172.21.15.67:0/3100465418 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652de141890 con 0x5652ddde28e0 2024-07-31T14:52:04.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.399+0000 7f1ca97fa640 10 client.15843 handle_command_reply: tid=0 2024-07-31T14:52:04.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.399+0000 7f1cce528740 1 client.15843 shutdown 2024-07-31T14:52:04.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.399+0000 7f1cce528740 1 -- 172.21.15.67:0/3100465418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dde03280 msgr2=0x5652de1d5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:04.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.399+0000 7f1cce528740 1 --2- 172.21.15.67:0/3100465418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dde03280 0x5652de1d5480 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc005e00 tx=0x7f1cbc003820 comp rx=0 tx=0).stop 2024-07-31T14:52:04.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.400+0000 7f1cce528740 1 -- 172.21.15.67:0/3100465418 shutdown_connections 2024-07-31T14:52:04.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.400+0000 7f1cce528740 1 --2- 172.21.15.67:0/3100465418 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5652ddde28e0 0x5652de1e7380 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.400+0000 7f1cce528740 1 --2- 172.21.15.67:0/3100465418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde23f60 0x5652de1e0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.400+0000 7f1cce528740 1 --2- 172.21.15.67:0/3100465418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dde03280 0x5652de1d5480 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.400+0000 7f1ca97fa640 0 client.15843 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:04.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.400+0000 7f1cce528740 1 --2- 172.21.15.67:0/3100465418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dddfb0a0 0x5652de1d4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.400+0000 7f1cce528740 1 -- 172.21.15.67:0/3100465418 >> 172.21.15.67:0/3100465418 conn(0x5652dde18350 msgr2=0x5652de1cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:04.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.400+0000 7f1cce528740 1 -- 172.21.15.67:0/3100465418 shutdown_connections 2024-07-31T14:52:04.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.400+0000 7f1cce528740 1 -- 172.21.15.67:0/3100465418 wait complete. 2024-07-31T14:52:04.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.400+0000 7f1cce528740 20 client.15843 trim_cache size 0 max 16384 2024-07-31T14:52:04.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.422+0000 7f1cce528740 1 Processor -- start 2024-07-31T14:52:04.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.423+0000 7f1cce528740 1 -- start start 2024-07-31T14:52:04.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.423+0000 7f1cce528740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dddfb0a0 0x5652de1e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.423+0000 7f1cce528740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde03280 0x5652de1e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.423+0000 7f1cce528740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde23f60 0x5652de1e1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.423+0000 7f1cce528740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5652de1e56c0 con 0x5652dddfb0a0 2024-07-31T14:52:04.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.423+0000 7f1cce528740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5652de1e5d90 con 0x5652dde03280 2024-07-31T14:52:04.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.423+0000 7f1cce528740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5652de1e6460 con 0x5652dde23f60 2024-07-31T14:52:04.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.423+0000 7f1cc5572640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dddfb0a0 0x5652de1e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.423+0000 7f1cc5572640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dddfb0a0 0x5652de1e6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46556/0 (socket says 172.21.15.67:46556) 2024-07-31T14:52:04.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.423+0000 7f1cc5572640 1 -- 172.21.15.67:0/3984940261 learned_addr learned my addr 172.21.15.67:0/3984940261 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:04.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.423+0000 7f1cc5d73640 1 --2- 172.21.15.67:0/3984940261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde03280 0x5652de1e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.423+0000 7f1cc5572640 1 --2- 172.21.15.67:0/3984940261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde23f60 0x5652de1e1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.424+0000 7f1cc5572640 1 -- 172.21.15.67:0/3984940261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde23f60 msgr2=0x5652de1e1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:04.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.424+0000 7f1cc5572640 1 --2- 172.21.15.67:0/3984940261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde23f60 0x5652de1e1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.424+0000 7f1cc5572640 1 -- 172.21.15.67:0/3984940261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde03280 msgr2=0x5652de1e0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:04.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.424+0000 7f1cc5572640 1 --2- 172.21.15.67:0/3984940261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde03280 0x5652de1e0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.424+0000 7f1cc5572640 1 -- 172.21.15.67:0/3984940261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652de1e6240 con 0x5652dddfb0a0 2024-07-31T14:52:04.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.424+0000 7f1cc5572640 1 --2- 172.21.15.67:0/3984940261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dddfb0a0 0x5652de1e6cb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0000c00 tx=0x7f1cb000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1fc66161b7041a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:04.431 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:04.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.424+0000 7f1cab7fe640 1 -- 172.21.15.67:0/3984940261 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0025040 con 0x5652dddfb0a0 2024-07-31T14:52:04.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.424+0000 7f1cab7fe640 1 -- 172.21.15.67:0/3984940261 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb0004020 con 0x5652dddfb0a0 2024-07-31T14:52:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.424+0000 7f1cab7fe640 1 -- 172.21.15.67:0/3984940261 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0005520 con 0x5652dddfb0a0 2024-07-31T14:52:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.425+0000 7f1cce528740 1 -- 172.21.15.67:0/3984940261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dddfb0a0 msgr2=0x5652de1e6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.425+0000 7f1cce528740 1 --2- 172.21.15.67:0/3984940261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dddfb0a0 0x5652de1e6cb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0000c00 tx=0x7f1cb000b410 comp rx=0 tx=0).stop 2024-07-31T14:52:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.425+0000 7f1cce528740 1 -- 172.21.15.67:0/3984940261 shutdown_connections 2024-07-31T14:52:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.425+0000 7f1cce528740 1 --2- 172.21.15.67:0/3984940261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde23f60 0x5652de1e1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.425+0000 7f1cce528740 1 --2- 172.21.15.67:0/3984940261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dde03280 0x5652de1e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.425+0000 7f1cce528740 1 --2- 172.21.15.67:0/3984940261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dddfb0a0 0x5652de1e6cb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.425+0000 7f1cce528740 1 -- 172.21.15.67:0/3984940261 >> 172.21.15.67:0/3984940261 conn(0x5652dde18350 msgr2=0x5652de1d35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.425+0000 7f1cce528740 1 -- 172.21.15.67:0/3984940261 shutdown_connections 2024-07-31T14:52:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.425+0000 7f1cce528740 1 -- 172.21.15.67:0/3984940261 wait complete. 2024-07-31T14:52:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.426+0000 7f1cce528740 1 -- start start 2024-07-31T14:52:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.426+0000 7f1cce528740 1 Processor -- start 2024-07-31T14:52:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.426+0000 7f1cce528740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dddfb0a0 0x5652de1e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.426+0000 7f1cce528740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde03280 0x5652de1e9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.426+0000 7f1cce528740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dde23f60 0x5652de1ee190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.426+0000 7f1cce528740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5652de1e7200 con 0x5652dde23f60 2024-07-31T14:52:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.426+0000 7f1cce528740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5652de1e6b30 con 0x5652dddfb0a0 2024-07-31T14:52:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.426+0000 7f1cce528740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5652de1e78d0 con 0x5652dde03280 2024-07-31T14:52:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.426+0000 7f1cce528740 10 client.0 _ll_register_callbacks cb 0x7f1cbc005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.427+0000 7f1cc5572640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dde23f60 0x5652de1ee190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.427+0000 7f1cc5572640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dde23f60 0x5652de1ee190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46560/0 (socket says 172.21.15.67:46560) 2024-07-31T14:52:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.427+0000 7f1cc5572640 1 -- 172.21.15.67:0/2392275348 learned_addr learned my addr 172.21.15.67:0/2392275348 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.427+0000 7f1cc5572640 1 --2- 172.21.15.67:0/2392275348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dddfb0a0 0x5652de1e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.427+0000 7f1cc5d73640 1 --2- 172.21.15.67:0/2392275348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde03280 0x5652de1e9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.427+0000 7f1cc5572640 1 -- 172.21.15.67:0/2392275348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde03280 msgr2=0x5652de1e9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.427+0000 7f1cc5572640 1 --2- 172.21.15.67:0/2392275348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde03280 0x5652de1e9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.427+0000 7f1cc5572640 1 -- 172.21.15.67:0/2392275348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dddfb0a0 msgr2=0x5652de1e8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.427+0000 7f1cc5572640 1 --2- 172.21.15.67:0/2392275348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dddfb0a0 0x5652de1e8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.427+0000 7f1cc5572640 1 -- 172.21.15.67:0/2392275348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652de1ee800 con 0x5652dde23f60 2024-07-31T14:52:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.427+0000 7f1cc5572640 1 --2- 172.21.15.67:0/2392275348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dde23f60 0x5652de1ee190 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0005140 tx=0x7f1cb0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b52583848f329fb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.427+0000 7f1ca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.428+0000 7f1ca97fa640 1 -- 172.21.15.67:0/2392275348 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0033040 con 0x5652dde23f60 2024-07-31T14:52:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.428+0000 7f1ca97fa640 1 -- 172.21.15.67:0/2392275348 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f1cb000faf0 con 0x5652dde23f60 2024-07-31T14:52:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.428+0000 7f1cce528740 1 -- 172.21.15.67:0/2392275348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb0004020 con 0x5652dde23f60 2024-07-31T14:52:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.428+0000 7f1ca97fa640 1 -- 172.21.15.67:0/2392275348 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1cb0042050 con 0x5652dde23f60 2024-07-31T14:52:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.428+0000 7f1cce528740 10 client.15849 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.428+0000 7f1cce528740 1 -- 172.21.15.67:0/2392275348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5652de1efbc0 con 0x5652dde23f60 2024-07-31T14:52:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.428+0000 7f1ca97fa640 1 -- 172.21.15.67:0/2392275348 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb000fcf0 con 0x5652dde23f60 2024-07-31T14:52:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.428+0000 7f1ca97fa640 1 -- 172.21.15.67:0/2392275348 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1cb00049b0 con 0x5652dde23f60 2024-07-31T14:52:04.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.428+0000 7f1cce528740 10 client.15849 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:04.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.429+0000 7f1cce528740 10 client.15849 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-31T14:52:04.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.429+0000 7f1cce528740 20 client.15849 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:04.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.429+0000 7f1cce528740 1 --2- 172.21.15.67:0/2392275348 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5652dddb7c90 0x5652de2532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:04.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.429+0000 7f1cce528740 4 client.15849 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-31T14:52:04.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.429+0000 7f1cce528740 1 -- 172.21.15.67:0/2392275348 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "status"}) -- 0x7f1cb0004020 con 0x5652dddb7c90 2024-07-31T14:52:04.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.429+0000 7f1cc5d73640 1 --2- 172.21.15.67:0/2392275348 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5652dddb7c90 0x5652de2532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:04.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.429+0000 7f1cc5d73640 1 --2- 172.21.15.67:0/2392275348 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5652dddb7c90 0x5652de2532a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee0d5edca76c078e server_cookie=6077198fd4837d2c in_seq=0 out_seq=0 2024-07-31T14:52:04.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.429+0000 7f1ca97fa640 10 client.15849 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:04.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.430+0000 7f1ca97fa640 1 -- 172.21.15.67:0/2392275348 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x7f1cb0004020 con 0x5652dddb7c90 2024-07-31T14:52:04.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.430+0000 7f1ca97fa640 10 client.15849 handle_command_reply: tid=0 2024-07-31T14:52:04.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.430+0000 7f1cce528740 1 client.15849 shutdown 2024-07-31T14:52:04.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.430+0000 7f1cce528740 1 -- 172.21.15.67:0/2392275348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dde23f60 msgr2=0x5652de1ee190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:04.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.430+0000 7f1cce528740 1 --2- 172.21.15.67:0/2392275348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dde23f60 0x5652de1ee190 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0005140 tx=0x7f1cb0005850 comp rx=0 tx=0).stop 2024-07-31T14:52:04.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.430+0000 7f1cce528740 1 -- 172.21.15.67:0/2392275348 shutdown_connections 2024-07-31T14:52:04.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.430+0000 7f1cce528740 1 --2- 172.21.15.67:0/2392275348 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5652dddb7c90 0x5652de2532a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.430+0000 7f1cce528740 1 --2- 172.21.15.67:0/2392275348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652dde23f60 0x5652de1ee190 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.430+0000 7f1cce528740 1 --2- 172.21.15.67:0/2392275348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652dde03280 0x5652de1e9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.430+0000 7f1cce528740 1 --2- 172.21.15.67:0/2392275348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652dddfb0a0 0x5652de1e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.430+0000 7f1ca97fa640 0 client.15849 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:04.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.430+0000 7f1cce528740 1 -- 172.21.15.67:0/2392275348 >> 172.21.15.67:0/2392275348 conn(0x5652dde18350 msgr2=0x5652de1cc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:04.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.430+0000 7f1cce528740 1 -- 172.21.15.67:0/2392275348 shutdown_connections 2024-07-31T14:52:04.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.430+0000 7f1cce528740 1 -- 172.21.15.67:0/2392275348 wait complete. 2024-07-31T14:52:04.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.431+0000 7f1cce528740 20 client.15849 trim_cache size 0 max 16384 2024-07-31T14:52:04.440 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:52:04.440 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5ca2b71e-4f4a-11ef-bcc9-c7b262605968", 2024-07-31T14:52:04.440 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-07-31T14:52:04.440 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24671, 2024-07-31T14:52:04.440 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-31T14:52:04.440 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-31T14:52:04.440 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-31T14:52:04.440 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 299.39010628099999, 2024-07-31T14:52:04.440 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 30, 2024-07-31T14:52:04.440 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-31T14:52:04.440 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-31T14:52:04.440 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 313.60164352599998 2024-07-31T14:52:04.441 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:52:04.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.431+0000 7f1cc7fd7640 1 -- 172.21.15.67:0/1874997893 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1cac0b3e50 msgr2=0x7f1cac0b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:04.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.431+0000 7f1cc7fd7640 1 --2- 172.21.15.67:0/1874997893 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1cac0b3e50 0x7f1cac0b6360 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8102c90 tx=0x7f1cb80984f0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.431+0000 7f1cc7fd7640 1 -- 172.21.15.67:0/1874997893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0108760 msgr2=0x7f1cc01a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.431+0000 7f1cc7fd7640 1 --2- 172.21.15.67:0/1874997893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0108760 0x7f1cc01a18b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80098d0 tx=0x7f1cb80317b0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.432+0000 7f1cc7fd7640 1 -- 172.21.15.67:0/1874997893 shutdown_connections 2024-07-31T14:52:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.432+0000 7f1cc7fd7640 1 --2- 172.21.15.67:0/1874997893 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1cac0b3e50 0x7f1cac0b6360 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.432+0000 7f1cc7fd7640 1 --2- 172.21.15.67:0/1874997893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0108760 0x7f1cc01a18b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.432+0000 7f1cc7fd7640 1 --2- 172.21.15.67:0/1874997893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc0107e10 0x7f1cc019c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.432+0000 7f1cc7fd7640 1 --2- 172.21.15.67:0/1874997893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc01024d0 0x7f1cc019c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.432+0000 7f1cc7fd7640 1 -- 172.21.15.67:0/1874997893 >> 172.21.15.67:0/1874997893 conn(0x7f1cc0079190 msgr2=0x7f1cc006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.432+0000 7f1cc7fd7640 1 -- 172.21.15.67:0/1874997893 shutdown_connections 2024-07-31T14:52:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:04.432+0000 7f1cc7fd7640 1 -- 172.21.15.67:0/1874997893 wait complete. 2024-07-31T14:52:04.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:04 smithi077 ceph-mon[37327]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-07-31T14:52:05.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:04 smithi067 ceph-mon[32750]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-07-31T14:52:05.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:04 smithi073 ceph-mon[36714]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-07-31T14:52:05.167 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-07-31T14:52:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.844+0000 7f2e617fa640 1 -- 172.21.15.67:0/1515062364 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f327048da0 con 0x55f326d331b0 2024-07-31T14:52:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.844+0000 7f2e617fa640 10 client.15831 handle_command_reply: tid=0 2024-07-31T14:52:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.844+0000 7f2e8719c740 1 client.15831 shutdown 2024-07-31T14:52:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.845+0000 7f2e8719c740 1 -- 172.21.15.67:0/1515062364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 msgr2=0x55f3270eab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.845+0000 7f2e8719c740 1 --2- 172.21.15.67:0/1515062364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270eab10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2e680058c0 tx=0x7f2e68009530 comp rx=0 tx=0).stop 2024-07-31T14:52:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.845+0000 7f2e8719c740 1 -- 172.21.15.67:0/1515062364 shutdown_connections 2024-07-31T14:52:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.845+0000 7f2e8719c740 1 --2- 172.21.15.67:0/1515062364 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f326d331b0 0x55f327161ae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.845+0000 7f2e8719c740 1 --2- 172.21.15.67:0/1515062364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f326d124d0 0x55f3270eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.845+0000 7f2e617fa640 0 client.15831 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.845+0000 7f2e8719c740 1 --2- 172.21.15.67:0/1515062364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f326d0a2f0 0x55f3270eab10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.845+0000 7f2e8719c740 1 --2- 172.21.15.67:0/1515062364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f326cf1d80 0x55f3270ee3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.845+0000 7f2e8719c740 1 -- 172.21.15.67:0/1515062364 >> 172.21.15.67:0/1515062364 conn(0x55f326d27300 msgr2=0x55f3270d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.845+0000 7f2e8719c740 1 -- 172.21.15.67:0/1515062364 shutdown_connections 2024-07-31T14:52:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.845+0000 7f2e8719c740 1 -- 172.21.15.67:0/1515062364 wait complete. 2024-07-31T14:52:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.845+0000 7f2e8719c740 20 client.15831 trim_cache size 0 max 16384 2024-07-31T14:52:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:52:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:52:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:52:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:52:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.846+0000 7f2e801d0640 1 -- 172.21.15.67:0/2788736677 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2e640b3dd0 msgr2=0x7f2e640b62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.846+0000 7f2e801d0640 1 --2- 172.21.15.67:0/2788736677 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2e640b3dd0 0x7f2e640b62e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55f3270d4960 tx=0x7f2e7400b390 comp rx=0 tx=0).stop 2024-07-31T14:52:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.846+0000 7f2e801d0640 1 -- 172.21.15.67:0/2788736677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e78108670 msgr2=0x7f2e7819b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.846+0000 7f2e801d0640 1 --2- 172.21.15.67:0/2788736677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e78108670 0x7f2e7819b670 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2e700098d0 tx=0x7f2e700317b0 comp rx=0 tx=0).stop 2024-07-31T14:52:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.846+0000 7f2e801d0640 1 -- 172.21.15.67:0/2788736677 shutdown_connections 2024-07-31T14:52:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.846+0000 7f2e801d0640 1 --2- 172.21.15.67:0/2788736677 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2e640b3dd0 0x7f2e640b62e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.846+0000 7f2e801d0640 1 --2- 172.21.15.67:0/2788736677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e78108670 0x7f2e7819b670 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.846+0000 7f2e801d0640 1 --2- 172.21.15.67:0/2788736677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e78107d20 0x7f2e7819b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.846+0000 7f2e801d0640 1 --2- 172.21.15.67:0/2788736677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e781023f0 0x7f2e7819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.846+0000 7f2e801d0640 1 -- 172.21.15.67:0/2788736677 >> 172.21.15.67:0/2788736677 conn(0x7f2e780fddf0 msgr2=0x7f2e781062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.846+0000 7f2e801d0640 1 -- 172.21.15.67:0/2788736677 shutdown_connections 2024-07-31T14:52:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:05.846+0000 7f2e801d0640 1 -- 172.21.15.67:0/2788736677 wait complete. 2024-07-31T14:52:06.781 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:06 smithi067 ceph-mon[32750]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-31T14:52:06.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:06 smithi077 ceph-mon[37327]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-31T14:52:07.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.036+0000 7f6986a8c640 1 -- 172.21.15.67:0/3968801153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6980102490 msgr2=0x7f69801028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:07.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.036+0000 7f6986a8c640 1 --2- 172.21.15.67:0/3968801153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6980102490 0x7f69801028a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6974009900 tx=0x7f697402a410 comp rx=0 tx=0).stop 2024-07-31T14:52:07.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.036+0000 7f6986a8c640 1 -- 172.21.15.67:0/3968801153 shutdown_connections 2024-07-31T14:52:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.036+0000 7f6986a8c640 1 --2- 172.21.15.67:0/3968801153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6980108720 0x7f698010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.036+0000 7f6986a8c640 1 --2- 172.21.15.67:0/3968801153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6980107dd0 0x7f69801081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.036+0000 7f6986a8c640 1 --2- 172.21.15.67:0/3968801153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6980102490 0x7f69801028a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.036+0000 7f6986a8c640 1 -- 172.21.15.67:0/3968801153 >> 172.21.15.67:0/3968801153 conn(0x7f6980079160 msgr2=0x7f6980079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.036+0000 7f6986a8c640 1 -- 172.21.15.67:0/3968801153 shutdown_connections 2024-07-31T14:52:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.036+0000 7f6986a8c640 1 -- 172.21.15.67:0/3968801153 wait complete. 2024-07-31T14:52:07.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.036+0000 7f6986a8c640 1 Processor -- start 2024-07-31T14:52:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.037+0000 7f6986a8c640 1 -- start start 2024-07-31T14:52:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.037+0000 7f6986a8c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6980102490 0x7f698006a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.037+0000 7f6986a8c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6980107dd0 0x7f698006add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.037+0000 7f6986a8c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6980108720 0x7f698006b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.037+0000 7f6986a8c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f69801123a0 con 0x7f6980107dd0 2024-07-31T14:52:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.037+0000 7f6986a8c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6980112220 con 0x7f6980108720 2024-07-31T14:52:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.037+0000 7f6986a8c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6980112520 con 0x7f6980102490 2024-07-31T14:52:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.037+0000 7f6984828640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6980102490 0x7f698006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.037+0000 7f697ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6980107dd0 0x7f698006add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.037+0000 7f6984828640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6980102490 0x7f698006a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:41462/0 (socket says 172.21.15.67:41462) 2024-07-31T14:52:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.037+0000 7f697ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6980107dd0 0x7f698006add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46572/0 (socket says 172.21.15.67:46572) 2024-07-31T14:52:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.037+0000 7f6984828640 1 -- 172.21.15.67:0/4071074486 learned_addr learned my addr 172.21.15.67:0/4071074486 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.037+0000 7f6985029640 1 --2- 172.21.15.67:0/4071074486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6980108720 0x7f698006b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.038+0000 7f6984828640 1 -- 172.21.15.67:0/4071074486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6980108720 msgr2=0x7f698006b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.038+0000 7f6984828640 1 --2- 172.21.15.67:0/4071074486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6980108720 0x7f698006b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.038+0000 7f6984828640 1 -- 172.21.15.67:0/4071074486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6980107dd0 msgr2=0x7f698006add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.038+0000 7f6984828640 1 --2- 172.21.15.67:0/4071074486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6980107dd0 0x7f698006add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.038+0000 7f6984828640 1 -- 172.21.15.67:0/4071074486 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69801b0950 con 0x7f6980102490 2024-07-31T14:52:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.038+0000 7f6984828640 1 --2- 172.21.15.67:0/4071074486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6980102490 0x7f698006a890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6974005940 tx=0x7f6974004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.038+0000 7f697dffb640 1 -- 172.21.15.67:0/4071074486 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697403f030 con 0x7f6980102490 2024-07-31T14:52:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.038+0000 7f697dffb640 1 -- 172.21.15.67:0/4071074486 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6974003aa0 con 0x7f6980102490 2024-07-31T14:52:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.038+0000 7f6986a8c640 1 -- 172.21.15.67:0/4071074486 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69801b0be0 con 0x7f6980102490 2024-07-31T14:52:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.038+0000 7f697dffb640 1 -- 172.21.15.67:0/4071074486 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974035070 con 0x7f6980102490 2024-07-31T14:52:07.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.039+0000 7f6986a8c640 1 -- 172.21.15.67:0/4071074486 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69801b1110 con 0x7f6980102490 2024-07-31T14:52:07.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.040+0000 7f6986a8c640 1 -- 172.21.15.67:0/4071074486 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69801028a0 con 0x7f6980102490 2024-07-31T14:52:07.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.042+0000 7f697dffb640 1 -- 172.21.15.67:0/4071074486 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f6974042030 con 0x7f6980102490 2024-07-31T14:52:07.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.043+0000 7f697dffb640 1 --2- 172.21.15.67:0/4071074486 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69700b3e50 0x7f69700b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:07.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.043+0000 7f697ffff640 1 --2- 172.21.15.67:0/4071074486 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69700b3e50 0x7f69700b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:07.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.043+0000 7f697dffb640 1 -- 172.21.15.67:0/4071074486 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f6974104d80 con 0x7f6980102490 2024-07-31T14:52:07.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.043+0000 7f697ffff640 1 --2- 172.21.15.67:0/4071074486 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69700b3e50 0x7f69700b6360 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6980100610 tx=0x7f696c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:07.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.045+0000 7f697dffb640 1 -- 172.21.15.67:0/4071074486 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f69740cb560 con 0x7f6980102490 2024-07-31T14:52:07.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:06 smithi073 ceph-mon[36714]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-31T14:52:07.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.287+0000 7f6986a8c640 1 -- 172.21.15.67:0/4071074486 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6980074250 con 0x7f6980102490 2024-07-31T14:52:07.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.288+0000 7f697dffb640 1 -- 172.21.15.67:0/4071074486 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11962 (secure 0 0 0) 0x7f6980074250 con 0x7f6980102490 2024-07-31T14:52:07.290 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:52:07.290 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-31T14:51:08:515297+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:51:07.734542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:52:07.290 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-31T14:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.291+0000 7f6986a8c640 1 -- 172.21.15.67:0/4071074486 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69700b3e50 msgr2=0x7f69700b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.291+0000 7f6986a8c640 1 --2- 172.21.15.67:0/4071074486 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69700b3e50 0x7f69700b6360 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6980100610 tx=0x7f696c00b390 comp rx=0 tx=0).stop 2024-07-31T14:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.291+0000 7f6986a8c640 1 -- 172.21.15.67:0/4071074486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6980102490 msgr2=0x7f698006a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.291+0000 7f6986a8c640 1 --2- 172.21.15.67:0/4071074486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6980102490 0x7f698006a890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6974005940 tx=0x7f6974004270 comp rx=0 tx=0).stop 2024-07-31T14:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.291+0000 7f6986a8c640 1 -- 172.21.15.67:0/4071074486 shutdown_connections 2024-07-31T14:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.291+0000 7f6986a8c640 1 --2- 172.21.15.67:0/4071074486 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69700b3e50 0x7f69700b6360 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.291+0000 7f6986a8c640 1 --2- 172.21.15.67:0/4071074486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6980108720 0x7f698006b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.291+0000 7f6986a8c640 1 --2- 172.21.15.67:0/4071074486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6980107dd0 0x7f698006add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.291+0000 7f6986a8c640 1 --2- 172.21.15.67:0/4071074486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6980102490 0x7f698006a890 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.291+0000 7f6986a8c640 1 -- 172.21.15.67:0/4071074486 >> 172.21.15.67:0/4071074486 conn(0x7f6980079160 msgr2=0x7f6980100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.291+0000 7f6986a8c640 1 -- 172.21.15.67:0/4071074486 shutdown_connections 2024-07-31T14:52:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:07.291+0000 7f6986a8c640 1 -- 172.21.15.67:0/4071074486 wait complete. 2024-07-31T14:52:07.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:07 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/4071074486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:52:07.955 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T14:52:07.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:07 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/4071074486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:52:08.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:07 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/4071074486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:52:08.227 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:08.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:08 smithi077 ceph-mon[37327]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-07-31T14:52:09.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:08 smithi067 ceph-mon[32750]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-07-31T14:52:09.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:08 smithi073 ceph-mon[36714]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-07-31T14:52:10.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.061+0000 7f58e3c2b640 1 -- 172.21.15.67:0/3283722848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58dc109830 msgr2=0x7f58dc110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:10.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.061+0000 7f58e3c2b640 1 --2- 172.21.15.67:0/3283722848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58dc109830 0x7f58dc110060 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009ca0 tx=0x7f58d402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.061+0000 7f58e3c2b640 1 -- 172.21.15.67:0/3283722848 shutdown_connections 2024-07-31T14:52:10.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.061+0000 7f58e3c2b640 1 --2- 172.21.15.67:0/3283722848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58dc109830 0x7f58dc110060 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.061+0000 7f58e3c2b640 1 --2- 172.21.15.67:0/3283722848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58dc108ee0 0x7f58dc1092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.061+0000 7f58e3c2b640 1 --2- 172.21.15.67:0/3283722848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc1035f0 0x7f58dc103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.061+0000 7f58e3c2b640 1 -- 172.21.15.67:0/3283722848 >> 172.21.15.67:0/3283722848 conn(0x7f58dc07b340 msgr2=0x7f58dc07b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:10.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.061+0000 7f58e3c2b640 1 -- 172.21.15.67:0/3283722848 shutdown_connections 2024-07-31T14:52:10.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.061+0000 7f58e3c2b640 1 -- 172.21.15.67:0/3283722848 wait complete. 2024-07-31T14:52:10.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.062+0000 7f58e3c2b640 1 Processor -- start 2024-07-31T14:52:10.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.062+0000 7f58e3c2b640 1 -- start start 2024-07-31T14:52:10.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.062+0000 7f58e3c2b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58dc1035f0 0x7f58dc199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.062+0000 7f58e3c2b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc108ee0 0x7f58dc19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.062+0000 7f58e3c2b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58dc109830 0x7f58dc19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.062+0000 7f58e3c2b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc113500 con 0x7f58dc109830 2024-07-31T14:52:10.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.062+0000 7f58e3c2b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc113380 con 0x7f58dc108ee0 2024-07-31T14:52:10.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.062+0000 7f58e3c2b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc113680 con 0x7f58dc1035f0 2024-07-31T14:52:10.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.062+0000 7f58e21c8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58dc109830 0x7f58dc19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.062+0000 7f58e21c8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58dc109830 0x7f58dc19f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46600/0 (socket says 172.21.15.67:46600) 2024-07-31T14:52:10.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.062+0000 7f58e19c7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58dc1035f0 0x7f58dc199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.062+0000 7f58e21c8640 1 -- 172.21.15.67:0/2685586306 learned_addr learned my addr 172.21.15.67:0/2685586306 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:10.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.062+0000 7f58e11c6640 1 --2- 172.21.15.67:0/2685586306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc108ee0 0x7f58dc19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.063+0000 7f58e21c8640 1 -- 172.21.15.67:0/2685586306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58dc1035f0 msgr2=0x7f58dc199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:10.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.063+0000 7f58e21c8640 1 --2- 172.21.15.67:0/2685586306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58dc1035f0 0x7f58dc199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.063+0000 7f58e21c8640 1 -- 172.21.15.67:0/2685586306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc108ee0 msgr2=0x7f58dc19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:10.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.063+0000 7f58e21c8640 1 --2- 172.21.15.67:0/2685586306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc108ee0 0x7f58dc19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.063+0000 7f58e21c8640 1 -- 172.21.15.67:0/2685586306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc19fdc0 con 0x7f58dc109830 2024-07-31T14:52:10.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.063+0000 7f58e21c8640 1 --2- 172.21.15.67:0/2685586306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58dc109830 0x7f58dc19f690 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f58d40098d0 tx=0x7f58d40317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:10.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.063+0000 7f58d2ffd640 1 -- 172.21.15.67:0/2685586306 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4003b70 con 0x7f58dc109830 2024-07-31T14:52:10.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.063+0000 7f58e3c2b640 1 -- 172.21.15.67:0/2685586306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc1a0050 con 0x7f58dc109830 2024-07-31T14:52:10.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.063+0000 7f58d2ffd640 1 -- 172.21.15.67:0/2685586306 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d4003d10 con 0x7f58dc109830 2024-07-31T14:52:10.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.063+0000 7f58d2ffd640 1 -- 172.21.15.67:0/2685586306 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d402fa20 con 0x7f58dc109830 2024-07-31T14:52:10.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.063+0000 7f58e3c2b640 1 -- 172.21.15.67:0/2685586306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc075020 con 0x7f58dc109830 2024-07-31T14:52:10.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.066+0000 7f58ea17c740 1 Processor -- start 2024-07-31T14:52:10.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.067+0000 7f58d2ffd640 1 -- 172.21.15.67:0/2685586306 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f58d402fc40 con 0x7f58dc109830 2024-07-31T14:52:10.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.067+0000 7f58ea17c740 1 -- start start 2024-07-31T14:52:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.067+0000 7f58ea17c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122f23070 0x55a1232c8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.067+0000 7f58ea17c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122efa1b0 0x55a1232c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.067+0000 7f58ea17c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122f02390 0x55a1232c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.067+0000 7f58ea17c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a12323ebb0 con 0x55a122f23070 2024-07-31T14:52:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.067+0000 7f58ea17c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a12322e0b0 con 0x55a122efa1b0 2024-07-31T14:52:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.067+0000 7f58ea17c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a1232320f0 con 0x55a122f02390 2024-07-31T14:52:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.068+0000 7f58e11c6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122f23070 0x55a1232c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.068+0000 7f58e11c6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122f23070 0x55a1232c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46610/0 (socket says 172.21.15.67:46610) 2024-07-31T14:52:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.068+0000 7f58e11c6640 1 -- 172.21.15.67:0/3171234749 learned_addr learned my addr 172.21.15.67:0/3171234749 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.068+0000 7f58e19c7640 1 --2- 172.21.15.67:0/3171234749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122efa1b0 0x55a1232c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.068+0000 7f58e11c6640 1 --2- 172.21.15.67:0/3171234749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122f02390 0x55a1232c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.068+0000 7f58e11c6640 1 -- 172.21.15.67:0/3171234749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122f02390 msgr2=0x55a1232c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:10.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.068+0000 7f58e11c6640 1 --2- 172.21.15.67:0/3171234749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122f02390 0x55a1232c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.068+0000 7f58d2ffd640 1 --2- 172.21.15.67:0/2685586306 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58c80b3dd0 0x7f58c80b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.068+0000 7f58e11c6640 1 -- 172.21.15.67:0/3171234749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122efa1b0 msgr2=0x55a1232c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:10.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.068+0000 7f58e11c6640 1 --2- 172.21.15.67:0/3171234749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122efa1b0 0x55a1232c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.068+0000 7f58e11c6640 1 -- 172.21.15.67:0/3171234749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1231d4b90 con 0x55a122f23070 2024-07-31T14:52:10.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.068+0000 7f58e21c8640 1 --2- 172.21.15.67:0/2685586306 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58c80b3dd0 0x7f58c80b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.068+0000 7f58e11c6640 1 --2- 172.21.15.67:0/3171234749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122f23070 0x55a1232c8570 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00bb20 tx=0x7f58cc0041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff7acc39b00af83 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:10.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.068+0000 7f58d2ffd640 1 -- 172.21.15.67:0/2685586306 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f58d4105860 con 0x7f58dc109830 2024-07-31T14:52:10.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.069+0000 7f58c7fff640 1 -- 172.21.15.67:0/3171234749 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc005020 con 0x55a122f23070 2024-07-31T14:52:10.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.069+0000 7f58c7fff640 1 -- 172.21.15.67:0/3171234749 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58cc005670 con 0x55a122f23070 2024-07-31T14:52:10.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.069+0000 7f58c7fff640 1 -- 172.21.15.67:0/3171234749 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0106c0 con 0x55a122f23070 2024-07-31T14:52:10.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.069+0000 7f58e21c8640 1 --2- 172.21.15.67:0/2685586306 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58c80b3dd0 0x7f58c80b62e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f58d4102d50 tx=0x7f58d40fbde0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.070+0000 7f58ea17c740 1 -- 172.21.15.67:0/3171234749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122f23070 msgr2=0x55a1232c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.070+0000 7f58ea17c740 1 --2- 172.21.15.67:0/3171234749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122f23070 0x55a1232c8570 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00bb20 tx=0x7f58cc0041d0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.070+0000 7f58ea17c740 1 -- 172.21.15.67:0/3171234749 shutdown_connections 2024-07-31T14:52:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.070+0000 7f58ea17c740 1 --2- 172.21.15.67:0/3171234749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122f02390 0x55a1232c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.070+0000 7f58ea17c740 1 --2- 172.21.15.67:0/3171234749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122efa1b0 0x55a1232c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.070+0000 7f58ea17c740 1 --2- 172.21.15.67:0/3171234749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122f23070 0x55a1232c8570 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.070+0000 7f58ea17c740 1 -- 172.21.15.67:0/3171234749 >> 172.21.15.67:0/3171234749 conn(0x55a122f17460 msgr2=0x55a1232c6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.070+0000 7f58ea17c740 1 -- 172.21.15.67:0/3171234749 shutdown_connections 2024-07-31T14:52:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.070+0000 7f58ea17c740 1 -- 172.21.15.67:0/3171234749 wait complete. 2024-07-31T14:52:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.071+0000 7f58ea17c740 1 -- start start 2024-07-31T14:52:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.071+0000 7f58ea17c740 1 Processor -- start 2024-07-31T14:52:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.071+0000 7f58ea17c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122efa1b0 0x55a1232d4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.071+0000 7f58ea17c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 0x55a1232d47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.071+0000 7f58ea17c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122f23070 0x55a1232dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.071+0000 7f58ea17c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a123091700 con 0x55a122f23070 2024-07-31T14:52:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.071+0000 7f58ea17c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a1231d09e0 con 0x55a122f02390 2024-07-31T14:52:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.071+0000 7f58ea17c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a1232d6cf0 con 0x55a122efa1b0 2024-07-31T14:52:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.071+0000 7f58ea17c740 10 client.0 _ll_register_callbacks cb 0x55a1231eb4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.071+0000 7f58e11c6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122f23070 0x55a1232dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.071+0000 7f58e11c6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122f23070 0x55a1232dffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46620/0 (socket says 172.21.15.67:46620) 2024-07-31T14:52:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.071+0000 7f58e11c6640 1 -- 172.21.15.67:0/1494124247 learned_addr learned my addr 172.21.15.67:0/1494124247 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.072+0000 7f58e19c7640 1 --2- 172.21.15.67:0/1494124247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 0x55a1232d47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.072+0000 7f58e11c6640 1 --2- 172.21.15.67:0/1494124247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122efa1b0 0x55a1232d4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.072+0000 7f58e19c7640 1 -- 172.21.15.67:0/1494124247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122efa1b0 msgr2=0x55a1232d4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.072+0000 7f58e19c7640 1 --2- 172.21.15.67:0/1494124247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122efa1b0 0x55a1232d4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.072+0000 7f58e19c7640 1 -- 172.21.15.67:0/1494124247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122f23070 msgr2=0x55a1232dffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.072+0000 7f58e19c7640 1 --2- 172.21.15.67:0/1494124247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122f23070 0x55a1232dffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.072+0000 7f58e19c7640 1 -- 172.21.15.67:0/1494124247 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a122f8e650 con 0x55a122f02390 2024-07-31T14:52:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.072+0000 7f58e11c6640 1 --2- 172.21.15.67:0/1494124247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122efa1b0 0x55a1232d4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.072+0000 7f58e19c7640 1 --2- 172.21.15.67:0/1494124247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 0x55a1232d47a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009920 tx=0x7f58d8009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8ed6ffe74f465f2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.072+0000 7f58c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:52:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.072+0000 7f58c57fa640 1 -- 172.21.15.67:0/1494124247 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d801f070 con 0x55a122f02390 2024-07-31T14:52:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.072+0000 7f58ea17c740 1 -- 172.21.15.67:0/1494124247 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58cc005670 con 0x55a122f02390 2024-07-31T14:52:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.072+0000 7f58c57fa640 1 -- 172.21.15.67:0/1494124247 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f58d800bd50 con 0x55a122f02390 2024-07-31T14:52:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.073+0000 7f58c57fa640 1 -- 172.21.15.67:0/1494124247 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58d802b050 con 0x55a122f02390 2024-07-31T14:52:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.073+0000 7f58ea17c740 10 client.25535 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.073+0000 7f58ea17c740 1 -- 172.21.15.67:0/1494124247 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1232e2e30 con 0x55a122f02390 2024-07-31T14:52:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.073+0000 7f58c57fa640 1 -- 172.21.15.67:0/1494124247 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58cc005670 con 0x55a122f02390 2024-07-31T14:52:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.073+0000 7f58c57fa640 1 -- 172.21.15.67:0/1494124247 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f58d80233b0 con 0x55a122f02390 2024-07-31T14:52:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.073+0000 7f58ea17c740 10 client.25535 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.073+0000 7f58ea17c740 10 client.25535 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:52:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.073+0000 7f58ea17c740 20 client.25535 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.073+0000 7f58ea17c740 1 --2- 172.21.15.67:0/1494124247 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a122ee19f0 0x55a1232e62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.073+0000 7f58ea17c740 4 client.25535 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.073+0000 7f58ea17c740 1 -- 172.21.15.67:0/1494124247 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a123240b00 con 0x55a122ee19f0 2024-07-31T14:52:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.074+0000 7f58e11c6640 1 --2- 172.21.15.67:0/1494124247 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a122ee19f0 0x55a1232e62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.074+0000 7f58c57fa640 10 client.25535 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:52:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.074+0000 7f58e11c6640 1 --2- 172.21.15.67:0/1494124247 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a122ee19f0 0x55a1232e62f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a521fa5a1ef12297 server_cookie=cada3e99d122056f in_seq=0 out_seq=0 2024-07-31T14:52:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58c57fa640 1 -- 172.21.15.67:0/1494124247 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a123240b00 con 0x55a122ee19f0 2024-07-31T14:52:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58c57fa640 10 client.25535 handle_command_reply: tid=0 2024-07-31T14:52:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58ea17c740 1 client.25535 shutdown 2024-07-31T14:52:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58ea17c740 1 -- 172.21.15.67:0/1494124247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 msgr2=0x55a1232d47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58ea17c740 1 --2- 172.21.15.67:0/1494124247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 0x55a1232d47a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009920 tx=0x7f58d8009de0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58ea17c740 1 -- 172.21.15.67:0/1494124247 shutdown_connections 2024-07-31T14:52:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58ea17c740 1 --2- 172.21.15.67:0/1494124247 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a122ee19f0 0x55a1232e62f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58ea17c740 1 --2- 172.21.15.67:0/1494124247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122f23070 0x55a1232dffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58ea17c740 1 --2- 172.21.15.67:0/1494124247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 0x55a1232d47a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58c57fa640 0 client.25535 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:52:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58ea17c740 1 --2- 172.21.15.67:0/1494124247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122efa1b0 0x55a1232d4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58ea17c740 1 -- 172.21.15.67:0/1494124247 >> 172.21.15.67:0/1494124247 conn(0x55a122f17460 msgr2=0x55a1232cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58ea17c740 1 -- 172.21.15.67:0/1494124247 shutdown_connections 2024-07-31T14:52:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58ea17c740 1 -- 172.21.15.67:0/1494124247 wait complete. 2024-07-31T14:52:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.075+0000 7f58ea17c740 20 client.25535 trim_cache size 0 max 16384 2024-07-31T14:52:10.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.090+0000 7f58ea17c740 1 Processor -- start 2024-07-31T14:52:10.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.090+0000 7f58ea17c740 1 -- start start 2024-07-31T14:52:10.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.090+0000 7f58ea17c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122ee19f0 0x55a1232c6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.090+0000 7f58ea17c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122efa1b0 0x55a1232dfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.090+0000 7f58ea17c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 0x55a1232cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.090+0000 7f58ea17c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc1ae480 con 0x55a122ee19f0 2024-07-31T14:52:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.090+0000 7f58ea17c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc1affc0 con 0x55a122f02390 2024-07-31T14:52:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.090+0000 7f58ea17c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc1aeb50 con 0x55a122efa1b0 2024-07-31T14:52:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.090+0000 7f58e11c6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122ee19f0 0x55a1232c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.090+0000 7f58e11c6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122ee19f0 0x55a1232c6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46630/0 (socket says 172.21.15.67:46630) 2024-07-31T14:52:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.090+0000 7f58e11c6640 1 -- 172.21.15.67:0/496019802 learned_addr learned my addr 172.21.15.67:0/496019802 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.090+0000 7f58e19c7640 1 --2- 172.21.15.67:0/496019802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122efa1b0 0x55a1232dfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.091+0000 7f58e11c6640 1 --2- 172.21.15.67:0/496019802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 0x55a1232cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.091+0000 7f58e11c6640 1 -- 172.21.15.67:0/496019802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122efa1b0 msgr2=0x55a1232dfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.091+0000 7f58e11c6640 1 --2- 172.21.15.67:0/496019802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122efa1b0 0x55a1232dfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.091+0000 7f58e11c6640 1 -- 172.21.15.67:0/496019802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 msgr2=0x55a1232cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:10.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.091+0000 7f58e11c6640 1 --2- 172.21.15.67:0/496019802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 0x55a1232cd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.091+0000 7f58e11c6640 1 -- 172.21.15.67:0/496019802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1232c6dd0 con 0x55a122ee19f0 2024-07-31T14:52:10.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.091+0000 7f58e11c6640 1 --2- 172.21.15.67:0/496019802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122ee19f0 0x55a1232c6490 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f58cc005fd0 tx=0x7f58cc004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1bec06a9dc9d2b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:10.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.091+0000 7f58c77fe640 1 -- 172.21.15.67:0/496019802 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc00b390 con 0x55a122ee19f0 2024-07-31T14:52:10.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.091+0000 7f58c77fe640 1 -- 172.21.15.67:0/496019802 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58cc0054b0 con 0x55a122ee19f0 2024-07-31T14:52:10.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.091+0000 7f58c77fe640 1 -- 172.21.15.67:0/496019802 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc030350 con 0x55a122ee19f0 2024-07-31T14:52:10.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.092+0000 7f58ea17c740 1 -- 172.21.15.67:0/496019802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122ee19f0 msgr2=0x55a1232c6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:10.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.092+0000 7f58ea17c740 1 --2- 172.21.15.67:0/496019802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122ee19f0 0x55a1232c6490 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f58cc005fd0 tx=0x7f58cc004ce0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.092+0000 7f58ea17c740 1 -- 172.21.15.67:0/496019802 shutdown_connections 2024-07-31T14:52:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.092+0000 7f58ea17c740 1 --2- 172.21.15.67:0/496019802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 0x55a1232cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.092+0000 7f58ea17c740 1 --2- 172.21.15.67:0/496019802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122efa1b0 0x55a1232dfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.092+0000 7f58ea17c740 1 --2- 172.21.15.67:0/496019802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122ee19f0 0x55a1232c6490 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.092+0000 7f58ea17c740 1 -- 172.21.15.67:0/496019802 >> 172.21.15.67:0/496019802 conn(0x55a122f17460 msgr2=0x55a1232c5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.092+0000 7f58ea17c740 1 -- 172.21.15.67:0/496019802 shutdown_connections 2024-07-31T14:52:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.092+0000 7f58ea17c740 1 -- 172.21.15.67:0/496019802 wait complete. 2024-07-31T14:52:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.093+0000 7f58ea17c740 1 -- start start 2024-07-31T14:52:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.093+0000 7f58ea17c740 1 Processor -- start 2024-07-31T14:52:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.093+0000 7f58ea17c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122ee19f0 0x55a1232dc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.093+0000 7f58ea17c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122efa1b0 0x55a1232eb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.093+0000 7f58ea17c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 0x55a1232eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.093+0000 7f58ea17c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc1af8f0 con 0x55a122efa1b0 2024-07-31T14:52:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.093+0000 7f58ea17c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc1af220 con 0x55a122f02390 2024-07-31T14:52:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.093+0000 7f58ea17c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc1ac460 con 0x55a122ee19f0 2024-07-31T14:52:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.093+0000 7f58ea17c740 10 client.0 _ll_register_callbacks cb 0x7f58d8010ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58e19c7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122efa1b0 0x55a1232eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58e19c7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122efa1b0 0x55a1232eb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46638/0 (socket says 172.21.15.67:46638) 2024-07-31T14:52:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58e19c7640 1 -- 172.21.15.67:0/1180517441 learned_addr learned my addr 172.21.15.67:0/1180517441 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58e11c6640 1 --2- 172.21.15.67:0/1180517441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 0x55a1232eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58e11c6640 1 --2- 172.21.15.67:0/1180517441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122ee19f0 0x55a1232dc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58e19c7640 1 -- 172.21.15.67:0/1180517441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122ee19f0 msgr2=0x55a1232dc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58e19c7640 1 --2- 172.21.15.67:0/1180517441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122ee19f0 0x55a1232dc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58e19c7640 1 -- 172.21.15.67:0/1180517441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 msgr2=0x55a1232eb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58e19c7640 1 --2- 172.21.15.67:0/1180517441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 0x55a1232eb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58e19c7640 1 -- 172.21.15.67:0/1180517441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1232ebe40 con 0x55a122efa1b0 2024-07-31T14:52:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58e19c7640 1 --2- 172.21.15.67:0/1180517441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122efa1b0 0x55a1232eb340 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009250 tx=0x7f58d80235c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b3246fc7704cdc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58c57fa640 1 -- 172.21.15.67:0/1180517441 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8034060 con 0x55a122efa1b0 2024-07-31T14:52:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58c57fa640 1 -- 172.21.15.67:0/1180517441 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f58d80239f0 con 0x55a122efa1b0 2024-07-31T14:52:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.094+0000 7f58ea17c740 1 -- 172.21.15.67:0/1180517441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58cc0054b0 con 0x55a122efa1b0 2024-07-31T14:52:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.095+0000 7f58c57fa640 1 -- 172.21.15.67:0/1180517441 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58d803b050 con 0x55a122efa1b0 2024-07-31T14:52:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.095+0000 7f58ea17c740 10 client.15873 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.095+0000 7f58ea17c740 1 -- 172.21.15.67:0/1180517441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1232dacd0 con 0x55a122efa1b0 2024-07-31T14:52:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.095+0000 7f58c57fa640 1 -- 172.21.15.67:0/1180517441 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58cc0054b0 con 0x55a122efa1b0 2024-07-31T14:52:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.095+0000 7f58c57fa640 1 -- 172.21.15.67:0/1180517441 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f58d8005150 con 0x55a122efa1b0 2024-07-31T14:52:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.095+0000 7f58ea17c740 10 client.15873 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.095+0000 7f58ea17c740 10 client.15873 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:52:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.095+0000 7f58ea17c740 20 client.15873 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.095+0000 7f58ea17c740 1 --2- 172.21.15.67:0/1180517441 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a122f23070 0x55a123359870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.095+0000 7f58ea17c740 4 client.15873 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:52:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.095+0000 7f58ea17c740 1 -- 172.21.15.67:0/1180517441 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a12323c5a0 con 0x55a122f23070 2024-07-31T14:52:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.096+0000 7f58e11c6640 1 --2- 172.21.15.67:0/1180517441 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a122f23070 0x55a123359870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.096+0000 7f58c57fa640 10 client.15873 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:52:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.096+0000 7f58e11c6640 1 --2- 172.21.15.67:0/1180517441 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a122f23070 0x55a123359870 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2042584b465c8b11 server_cookie=21a5d7441718ab78 in_seq=0 out_seq=0 2024-07-31T14:52:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.096+0000 7f58c57fa640 1 -- 172.21.15.67:0/1180517441 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a12323c5a0 con 0x55a122f23070 2024-07-31T14:52:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.096+0000 7f58c57fa640 10 client.15873 handle_command_reply: tid=0 2024-07-31T14:52:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.096+0000 7f58ea17c740 1 client.15873 shutdown 2024-07-31T14:52:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.097+0000 7f58ea17c740 1 -- 172.21.15.67:0/1180517441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122efa1b0 msgr2=0x55a1232eb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.097+0000 7f58ea17c740 1 --2- 172.21.15.67:0/1180517441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122efa1b0 0x55a1232eb340 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009250 tx=0x7f58d80235c0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.097+0000 7f58ea17c740 1 -- 172.21.15.67:0/1180517441 shutdown_connections 2024-07-31T14:52:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.097+0000 7f58ea17c740 1 --2- 172.21.15.67:0/1180517441 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a122f23070 0x55a123359870 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.097+0000 7f58ea17c740 1 --2- 172.21.15.67:0/1180517441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a122f02390 0x55a1232eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.097+0000 7f58ea17c740 1 --2- 172.21.15.67:0/1180517441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a122efa1b0 0x55a1232eb340 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.097+0000 7f58ea17c740 1 --2- 172.21.15.67:0/1180517441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a122ee19f0 0x55a1232dc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.097+0000 7f58ea17c740 1 -- 172.21.15.67:0/1180517441 >> 172.21.15.67:0/1180517441 conn(0x55a122f17460 msgr2=0x55a1232c95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.097+0000 7f58c57fa640 0 client.15873 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:52:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.097+0000 7f58ea17c740 1 -- 172.21.15.67:0/1180517441 shutdown_connections 2024-07-31T14:52:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.097+0000 7f58ea17c740 1 -- 172.21.15.67:0/1180517441 wait complete. 2024-07-31T14:52:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.097+0000 7f58ea17c740 20 client.15873 trim_cache size 0 max 16384 2024-07-31T14:52:10.104 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:52:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.098+0000 7f58e3c2b640 1 -- 172.21.15.67:0/2685586306 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58c80b3dd0 msgr2=0x7f58c80b62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.098+0000 7f58e3c2b640 1 --2- 172.21.15.67:0/2685586306 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58c80b3dd0 0x7f58c80b62e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f58d4102d50 tx=0x7f58d40fbde0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.098+0000 7f58e3c2b640 1 -- 172.21.15.67:0/2685586306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58dc109830 msgr2=0x7f58dc19f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.098+0000 7f58e3c2b640 1 --2- 172.21.15.67:0/2685586306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58dc109830 0x7f58dc19f690 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f58d40098d0 tx=0x7f58d40317b0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.098+0000 7f58e3c2b640 1 -- 172.21.15.67:0/2685586306 shutdown_connections 2024-07-31T14:52:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.098+0000 7f58e3c2b640 1 --2- 172.21.15.67:0/2685586306 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f58c80b3dd0 0x7f58c80b62e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.098+0000 7f58e3c2b640 1 --2- 172.21.15.67:0/2685586306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58dc109830 0x7f58dc19f690 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.098+0000 7f58e3c2b640 1 --2- 172.21.15.67:0/2685586306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc108ee0 0x7f58dc19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.098+0000 7f58e3c2b640 1 --2- 172.21.15.67:0/2685586306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58dc1035f0 0x7f58dc199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.098+0000 7f58e3c2b640 1 -- 172.21.15.67:0/2685586306 >> 172.21.15.67:0/2685586306 conn(0x7f58dc07b340 msgr2=0x7f58dc106ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.098+0000 7f58e3c2b640 1 -- 172.21.15.67:0/2685586306 shutdown_connections 2024-07-31T14:52:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:10.098+0000 7f58e3c2b640 1 -- 172.21.15.67:0/2685586306 wait complete. 2024-07-31T14:52:10.792 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T14:52:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:10 smithi077 ceph-mon[37327]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-07-31T14:52:11.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:10 smithi067 ceph-mon[32750]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-07-31T14:52:11.072 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:11.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:10 smithi073 ceph-mon[36714]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-07-31T14:52:12.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:12 smithi067 ceph-mon[32750]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-07-31T14:52:12.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:12 smithi077 ceph-mon[37327]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-07-31T14:52:13.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.044+0000 7f6940e2c640 1 -- 172.21.15.67:0/3768708374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f693c107dd0 msgr2=0x7f693c1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:13.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.044+0000 7f6940e2c640 1 --2- 172.21.15.67:0/3768708374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f693c107dd0 0x7f693c1081e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6930009900 tx=0x7f693002a410 comp rx=0 tx=0).stop 2024-07-31T14:52:13.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.044+0000 7f6940e2c640 1 -- 172.21.15.67:0/3768708374 shutdown_connections 2024-07-31T14:52:13.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.044+0000 7f6940e2c640 1 --2- 172.21.15.67:0/3768708374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f693c108720 0x7f693c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.044+0000 7f6940e2c640 1 --2- 172.21.15.67:0/3768708374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f693c107dd0 0x7f693c1081e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.044+0000 7f6940e2c640 1 --2- 172.21.15.67:0/3768708374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f693c102490 0x7f693c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.044+0000 7f6940e2c640 1 -- 172.21.15.67:0/3768708374 >> 172.21.15.67:0/3768708374 conn(0x7f693c079160 msgr2=0x7f693c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:13.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.044+0000 7f6940e2c640 1 -- 172.21.15.67:0/3768708374 shutdown_connections 2024-07-31T14:52:13.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.044+0000 7f6940e2c640 1 -- 172.21.15.67:0/3768708374 wait complete. 2024-07-31T14:52:13.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.045+0000 7f6940e2c640 1 Processor -- start 2024-07-31T14:52:13.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.046+0000 7f6940e2c640 1 -- start start 2024-07-31T14:52:13.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.046+0000 7f6940e2c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f693c102490 0x7f693c199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.046+0000 7f6940e2c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f693c107dd0 0x7f693c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.046+0000 7f6940e2c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f693c108720 0x7f693c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.046+0000 7f6940e2c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f693c112350 con 0x7f693c107dd0 2024-07-31T14:52:13.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.046+0000 7f6940e2c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1121d0 con 0x7f693c102490 2024-07-31T14:52:13.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.046+0000 7f6940e2c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1124d0 con 0x7f693c108720 2024-07-31T14:52:13.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.046+0000 7f6939d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f693c107dd0 0x7f693c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.046+0000 7f6939d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f693c107dd0 0x7f693c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37784/0 (socket says 172.21.15.67:37784) 2024-07-31T14:52:13.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.046+0000 7f6939d9b640 1 -- 172.21.15.67:0/1849649047 learned_addr learned my addr 172.21.15.67:0/1849649047 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:13.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.046+0000 7f693a59c640 1 --2- 172.21.15.67:0/1849649047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f693c102490 0x7f693c199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.047+0000 7f693a59c640 1 -- 172.21.15.67:0/1849649047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f693c108720 msgr2=0x7f693c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:13.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.047+0000 7f693a59c640 1 --2- 172.21.15.67:0/1849649047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f693c108720 0x7f693c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.047+0000 7f693a59c640 1 -- 172.21.15.67:0/1849649047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f693c107dd0 msgr2=0x7f693c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:13.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.047+0000 7f693a59c640 1 --2- 172.21.15.67:0/1849649047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f693c107dd0 0x7f693c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.047+0000 7f693a59c640 1 -- 172.21.15.67:0/1849649047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c19fe80 con 0x7f693c102490 2024-07-31T14:52:13.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.047+0000 7f6939d9b640 1 --2- 172.21.15.67:0/1849649047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f693c107dd0 0x7f693c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:13.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.047+0000 7f693a59c640 1 --2- 172.21.15.67:0/1849649047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f693c102490 0x7f693c199f10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f692c00e9f0 tx=0x7f692c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:13.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.047+0000 7f692b7fe640 1 -- 172.21.15.67:0/1849649047 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c00ccc0 con 0x7f693c102490 2024-07-31T14:52:13.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.047+0000 7f692b7fe640 1 -- 172.21.15.67:0/1849649047 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f692c0108e0 con 0x7f693c102490 2024-07-31T14:52:13.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.047+0000 7f692b7fe640 1 -- 172.21.15.67:0/1849649047 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c018880 con 0x7f693c102490 2024-07-31T14:52:13.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.047+0000 7f6940e2c640 1 -- 172.21.15.67:0/1849649047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c1a0170 con 0x7f693c102490 2024-07-31T14:52:13.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.047+0000 7f6940e2c640 1 -- 172.21.15.67:0/1849649047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c1a68d0 con 0x7f693c102490 2024-07-31T14:52:13.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.051+0000 7f694737d740 1 Processor -- start 2024-07-31T14:52:13.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.051+0000 7f694737d740 1 -- start start 2024-07-31T14:52:13.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.051+0000 7f692b7fe640 1 -- 172.21.15.67:0/1849649047 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f692c010430 con 0x7f693c102490 2024-07-31T14:52:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.052+0000 7f694737d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817d3070 0x563d81b78570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.052+0000 7f694737d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817aa1b0 0x563d81b78ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.052+0000 7f694737d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817b2390 0x563d81b78ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.052+0000 7f694737d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563d81aeebb0 con 0x563d817d3070 2024-07-31T14:52:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.052+0000 7f694737d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563d81ade0b0 con 0x563d817aa1b0 2024-07-31T14:52:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.052+0000 7f694737d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563d81ae20f0 con 0x563d817b2390 2024-07-31T14:52:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.052+0000 7f6939d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817b2390 0x563d81b78ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.052+0000 7f6939d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817b2390 0x563d81b78ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35906/0 (socket says 172.21.15.67:35906) 2024-07-31T14:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.052+0000 7f693a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817aa1b0 0x563d81b78ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.052+0000 7f6939d9b640 1 -- 172.21.15.67:0/199598471 learned_addr learned my addr 172.21.15.67:0/199598471 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.052+0000 7f693ad9d640 1 --2- 172.21.15.67:0/199598471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817d3070 0x563d81b78570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.053+0000 7f693ad9d640 1 -- 172.21.15.67:0/199598471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817b2390 msgr2=0x563d81b78ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.053+0000 7f693ad9d640 1 --2- 172.21.15.67:0/199598471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817b2390 0x563d81b78ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.053+0000 7f693ad9d640 1 -- 172.21.15.67:0/199598471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817aa1b0 msgr2=0x563d81b78ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.053+0000 7f693ad9d640 1 --2- 172.21.15.67:0/199598471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817aa1b0 0x563d81b78ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.053+0000 7f693ad9d640 1 -- 172.21.15.67:0/199598471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d81a84b90 con 0x563d817d3070 2024-07-31T14:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.053+0000 7f692b7fe640 1 --2- 172.21.15.67:0/1849649047 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69240b3dd0 0x7f69240b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.053+0000 7f692b7fe640 1 -- 172.21.15.67:0/1849649047 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f692c0e48a0 con 0x7f693c102490 2024-07-31T14:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.053+0000 7f693a59c640 1 --2- 172.21.15.67:0/1849649047 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69240b3dd0 0x7f69240b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.053+0000 7f693ad9d640 1 --2- 172.21.15.67:0/199598471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817d3070 0x563d81b78570 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6934009c40 tx=0x7f693400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edecd489773b926f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.054+0000 7f6928ff9640 1 -- 172.21.15.67:0/199598471 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934016070 con 0x563d817d3070 2024-07-31T14:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.054+0000 7f693a59c640 1 --2- 172.21.15.67:0/1849649047 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69240b3dd0 0x7f69240b62e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f692c0e1c80 tx=0x7f692c0ea480 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:13.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.054+0000 7f6928ff9640 1 -- 172.21.15.67:0/199598471 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6934005020 con 0x563d817d3070 2024-07-31T14:52:13.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.054+0000 7f6928ff9640 1 -- 172.21.15.67:0/199598471 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69340116d0 con 0x563d817d3070 2024-07-31T14:52:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.056+0000 7f694737d740 1 -- 172.21.15.67:0/199598471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817d3070 msgr2=0x563d81b78570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.056+0000 7f694737d740 1 --2- 172.21.15.67:0/199598471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817d3070 0x563d81b78570 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6934009c40 tx=0x7f693400e6c0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.056+0000 7f694737d740 1 -- 172.21.15.67:0/199598471 shutdown_connections 2024-07-31T14:52:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.056+0000 7f694737d740 1 --2- 172.21.15.67:0/199598471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817b2390 0x563d81b78ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.056+0000 7f694737d740 1 --2- 172.21.15.67:0/199598471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817aa1b0 0x563d81b78ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.056+0000 7f694737d740 1 --2- 172.21.15.67:0/199598471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817d3070 0x563d81b78570 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.056+0000 7f694737d740 1 -- 172.21.15.67:0/199598471 >> 172.21.15.67:0/199598471 conn(0x563d817c7460 msgr2=0x563d81b76460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.056+0000 7f694737d740 1 -- 172.21.15.67:0/199598471 shutdown_connections 2024-07-31T14:52:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.056+0000 7f694737d740 1 -- 172.21.15.67:0/199598471 wait complete. 2024-07-31T14:52:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.057+0000 7f694737d740 1 -- start start 2024-07-31T14:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.057+0000 7f694737d740 1 Processor -- start 2024-07-31T14:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.057+0000 7f694737d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817aa1b0 0x563d81b88550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.057+0000 7f694737d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817b2390 0x563d81b88a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.057+0000 7f694737d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817d3070 0x563d81b818d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f694737d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563d81941700 con 0x563d817b2390 2024-07-31T14:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f694737d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563d81a809e0 con 0x563d817aa1b0 2024-07-31T14:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f694737d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563d81b8b0b0 con 0x563d817d3070 2024-07-31T14:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f694737d740 10 client.0 _ll_register_callbacks cb 0x563d81a9b4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f693ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817aa1b0 0x563d81b88550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f693ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817aa1b0 0x563d81b88550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35594/0 (socket says 172.21.15.67:35594) 2024-07-31T14:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f6939d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817b2390 0x563d81b88a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f693ad9d640 1 -- 172.21.15.67:0/2558636227 learned_addr learned my addr 172.21.15.67:0/2558636227 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f693ad9d640 1 --2- 172.21.15.67:0/2558636227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817d3070 0x563d81b818d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f693ad9d640 1 -- 172.21.15.67:0/2558636227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817d3070 msgr2=0x563d81b818d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f693ad9d640 1 --2- 172.21.15.67:0/2558636227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817d3070 0x563d81b818d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f693ad9d640 1 -- 172.21.15.67:0/2558636227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817b2390 msgr2=0x563d81b88a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f693ad9d640 1 --2- 172.21.15.67:0/2558636227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817b2390 0x563d81b88a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f693ad9d640 1 -- 172.21.15.67:0/2558636227 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d8183e650 con 0x563d817aa1b0 2024-07-31T14:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f693ad9d640 1 --2- 172.21.15.67:0/2558636227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817aa1b0 0x563d81b88550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6934005cc0 tx=0x7f69340059c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=217a61fbcd84aad6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.058+0000 7f6921ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.059+0000 7f6921ffb640 1 -- 172.21.15.67:0/2558636227 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934016040 con 0x563d817aa1b0 2024-07-31T14:52:13.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.059+0000 7f694737d740 1 -- 172.21.15.67:0/2558636227 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6934005020 con 0x563d817aa1b0 2024-07-31T14:52:13.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.059+0000 7f6921ffb640 1 -- 172.21.15.67:0/2558636227 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f6934011890 con 0x563d817aa1b0 2024-07-31T14:52:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.059+0000 7f6921ffb640 1 -- 172.21.15.67:0/2558636227 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6934011a90 con 0x563d817aa1b0 2024-07-31T14:52:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.059+0000 7f694737d740 10 client.25550 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.059+0000 7f694737d740 1 -- 172.21.15.67:0/2558636227 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563d81b9b4e0 con 0x563d817aa1b0 2024-07-31T14:52:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.060+0000 7f6921ffb640 1 -- 172.21.15.67:0/2558636227 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6934005680 con 0x563d817aa1b0 2024-07-31T14:52:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.061+0000 7f6921ffb640 1 -- 172.21.15.67:0/2558636227 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6934026020 con 0x563d817aa1b0 2024-07-31T14:52:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.061+0000 7f694737d740 10 client.25550 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.061+0000 7f694737d740 10 client.25550 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:52:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.061+0000 7f694737d740 20 client.25550 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.061+0000 7f694737d740 1 --2- 172.21.15.67:0/2558636227 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x563d817919f0 0x563d81b9ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.061+0000 7f694737d740 4 client.25550 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.061+0000 7f694737d740 1 -- 172.21.15.67:0/2558636227 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6934005020 con 0x563d817919f0 2024-07-31T14:52:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.061+0000 7f6939d9b640 1 --2- 172.21.15.67:0/2558636227 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x563d817919f0 0x563d81b9ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.062+0000 7f6921ffb640 10 client.25550 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:52:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.062+0000 7f6939d9b640 1 --2- 172.21.15.67:0/2558636227 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x563d817919f0 0x563d81b9ecd0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=101b683aae44f32 server_cookie=638aaeb43d7e4a61 in_seq=0 out_seq=0 2024-07-31T14:52:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.063+0000 7f6921ffb640 1 -- 172.21.15.67:0/2558636227 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6934005020 con 0x563d817919f0 2024-07-31T14:52:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.063+0000 7f6921ffb640 10 client.25550 handle_command_reply: tid=0 2024-07-31T14:52:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.063+0000 7f694737d740 1 client.25550 shutdown 2024-07-31T14:52:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.063+0000 7f694737d740 1 -- 172.21.15.67:0/2558636227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817aa1b0 msgr2=0x563d81b88550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.063+0000 7f694737d740 1 --2- 172.21.15.67:0/2558636227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817aa1b0 0x563d81b88550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6934005cc0 tx=0x7f69340059c0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.063+0000 7f694737d740 1 -- 172.21.15.67:0/2558636227 shutdown_connections 2024-07-31T14:52:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.063+0000 7f694737d740 1 --2- 172.21.15.67:0/2558636227 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x563d817919f0 0x563d81b9ecd0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.063+0000 7f694737d740 1 --2- 172.21.15.67:0/2558636227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817d3070 0x563d81b818d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.063+0000 7f694737d740 1 --2- 172.21.15.67:0/2558636227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817b2390 0x563d81b88a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.063+0000 7f694737d740 1 --2- 172.21.15.67:0/2558636227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817aa1b0 0x563d81b88550 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.063+0000 7f694737d740 1 -- 172.21.15.67:0/2558636227 >> 172.21.15.67:0/2558636227 conn(0x563d817c7460 msgr2=0x563d81b74c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.064+0000 7f6921ffb640 0 client.25550 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:52:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.064+0000 7f694737d740 1 -- 172.21.15.67:0/2558636227 shutdown_connections 2024-07-31T14:52:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.064+0000 7f694737d740 1 -- 172.21.15.67:0/2558636227 wait complete. 2024-07-31T14:52:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.064+0000 7f694737d740 20 client.25550 trim_cache size 0 max 16384 2024-07-31T14:52:13.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.090+0000 7f694737d740 1 Processor -- start 2024-07-31T14:52:13.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.090+0000 7f694737d740 1 -- start start 2024-07-31T14:52:13.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.090+0000 7f694737d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817aa1b0 0x563d81b74c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.090+0000 7f694737d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817b2390 0x563d81b75180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.090+0000 7f694737d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817d3070 0x563d81b75d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.090+0000 7f694737d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1ae610 con 0x563d817b2390 2024-07-31T14:52:13.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.090+0000 7f694737d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1b0150 con 0x563d817d3070 2024-07-31T14:52:13.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.090+0000 7f694737d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1aece0 con 0x563d817aa1b0 2024-07-31T14:52:13.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.090+0000 7f6939d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817b2390 0x563d81b75180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.090+0000 7f6939d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817b2390 0x563d81b75180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37798/0 (socket says 172.21.15.67:37798) 2024-07-31T14:52:13.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.090+0000 7f6939d9b640 1 -- 172.21.15.67:0/843922246 learned_addr learned my addr 172.21.15.67:0/843922246 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:13.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.091+0000 7f693ad9d640 1 --2- 172.21.15.67:0/843922246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817aa1b0 0x563d81b74c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.091+0000 7f693ad9d640 1 --2- 172.21.15.67:0/843922246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817d3070 0x563d81b75d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.091+0000 7f6939d9b640 1 -- 172.21.15.67:0/843922246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817aa1b0 msgr2=0x563d81b74c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:13.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.091+0000 7f6939d9b640 1 --2- 172.21.15.67:0/843922246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817aa1b0 0x563d81b74c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.091+0000 7f6939d9b640 1 -- 172.21.15.67:0/843922246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817d3070 msgr2=0x563d81b75d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:13.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.091+0000 7f6939d9b640 1 --2- 172.21.15.67:0/843922246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817d3070 0x563d81b75d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.091+0000 7f6939d9b640 1 -- 172.21.15.67:0/843922246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d81b718c0 con 0x563d817b2390 2024-07-31T14:52:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.091+0000 7f693ad9d640 1 --2- 172.21.15.67:0/843922246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817aa1b0 0x563d81b74c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.091+0000 7f6939d9b640 1 --2- 172.21.15.67:0/843922246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817b2390 0x563d81b75180 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6930009230 tx=0x7f6930032800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddded7962abfc914 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.091+0000 7f6923fff640 1 -- 172.21.15.67:0/843922246 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930035040 con 0x563d817b2390 2024-07-31T14:52:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.091+0000 7f6923fff640 1 -- 172.21.15.67:0/843922246 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69300035f0 con 0x563d817b2390 2024-07-31T14:52:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.091+0000 7f6923fff640 1 -- 172.21.15.67:0/843922246 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693004a630 con 0x563d817b2390 2024-07-31T14:52:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.093+0000 7f694737d740 1 -- 172.21.15.67:0/843922246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817b2390 msgr2=0x563d81b75180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.093+0000 7f694737d740 1 --2- 172.21.15.67:0/843922246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817b2390 0x563d81b75180 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6930009230 tx=0x7f6930032800 comp rx=0 tx=0).stop 2024-07-31T14:52:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.093+0000 7f694737d740 1 -- 172.21.15.67:0/843922246 shutdown_connections 2024-07-31T14:52:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.093+0000 7f694737d740 1 --2- 172.21.15.67:0/843922246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817d3070 0x563d81b75d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.093+0000 7f694737d740 1 --2- 172.21.15.67:0/843922246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817b2390 0x563d81b75180 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.093+0000 7f694737d740 1 --2- 172.21.15.67:0/843922246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817aa1b0 0x563d81b74c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.093+0000 7f694737d740 1 -- 172.21.15.67:0/843922246 >> 172.21.15.67:0/843922246 conn(0x563d817c7460 msgr2=0x563d81b86bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.093+0000 7f694737d740 1 -- 172.21.15.67:0/843922246 shutdown_connections 2024-07-31T14:52:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.093+0000 7f694737d740 1 -- 172.21.15.67:0/843922246 wait complete. 2024-07-31T14:52:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.093+0000 7f694737d740 1 -- start start 2024-07-31T14:52:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.093+0000 7f694737d740 1 Processor -- start 2024-07-31T14:52:13.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.094+0000 7f694737d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817aa1b0 0x563d81b964a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.094+0000 7f694737d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817b2390 0x563d81b96a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.094+0000 7f694737d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817d3070 0x563d81c054a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.094+0000 7f694737d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1afa80 con 0x563d817aa1b0 2024-07-31T14:52:13.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.094+0000 7f693ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817aa1b0 0x563d81b964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.094+0000 7f693ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817aa1b0 0x563d81b964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37806/0 (socket says 172.21.15.67:37806) 2024-07-31T14:52:13.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.094+0000 7f693ad9d640 1 -- 172.21.15.67:0/1805562963 learned_addr learned my addr 172.21.15.67:0/1805562963 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:13.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.094+0000 7f6939d9b640 1 --2- 172.21.15.67:0/1805562963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817b2390 0x563d81b96a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.094+0000 7f694737d740 1 -- 172.21.15.67:0/1805562963 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1af3b0 con 0x563d817b2390 2024-07-31T14:52:13.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.094+0000 7f694737d740 1 -- 172.21.15.67:0/1805562963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1ac5f0 con 0x563d817d3070 2024-07-31T14:52:13.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.094+0000 7f693ad9d640 1 --2- 172.21.15.67:0/1805562963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817d3070 0x563d81c054a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.095+0000 7f694737d740 10 client.0 _ll_register_callbacks cb 0x7f692c0e1ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.095+0000 7f693ad9d640 1 -- 172.21.15.67:0/1805562963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817d3070 msgr2=0x563d81c054a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.095+0000 7f693ad9d640 1 --2- 172.21.15.67:0/1805562963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817d3070 0x563d81c054a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.095+0000 7f693ad9d640 1 -- 172.21.15.67:0/1805562963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817b2390 msgr2=0x563d81b96a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.095+0000 7f693ad9d640 1 --2- 172.21.15.67:0/1805562963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817b2390 0x563d81b96a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.095+0000 7f693ad9d640 1 -- 172.21.15.67:0/1805562963 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d81c05ab0 con 0x563d817aa1b0 2024-07-31T14:52:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.095+0000 7f693ad9d640 1 --2- 172.21.15.67:0/1805562963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817aa1b0 0x563d81b964a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6934005d70 tx=0x7f69340059c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe417f1303358ed server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.095+0000 7f6921ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.095+0000 7f6921ffb640 1 -- 172.21.15.67:0/1805562963 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934016070 con 0x563d817aa1b0 2024-07-31T14:52:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.095+0000 7f6921ffb640 1 -- 172.21.15.67:0/1805562963 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f6934011890 con 0x563d817aa1b0 2024-07-31T14:52:13.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.096+0000 7f694737d740 1 -- 172.21.15.67:0/1805562963 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69300035f0 con 0x563d817aa1b0 2024-07-31T14:52:13.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.096+0000 7f6921ffb640 1 -- 172.21.15.67:0/1805562963 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f693401a400 con 0x563d817aa1b0 2024-07-31T14:52:13.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.096+0000 7f694737d740 10 client.15891 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:13.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.096+0000 7f694737d740 1 -- 172.21.15.67:0/1805562963 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563d81c06d10 con 0x563d817aa1b0 2024-07-31T14:52:13.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.096+0000 7f6921ffb640 1 -- 172.21.15.67:0/1805562963 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6934004040 con 0x563d817aa1b0 2024-07-31T14:52:13.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.096+0000 7f6921ffb640 1 -- 172.21.15.67:0/1805562963 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6934026040 con 0x563d817aa1b0 2024-07-31T14:52:13.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.096+0000 7f694737d740 10 client.15891 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:13.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.096+0000 7f694737d740 10 client.15891 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:52:13.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.096+0000 7f694737d740 20 client.15891 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:13.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.096+0000 7f694737d740 1 --2- 172.21.15.67:0/1805562963 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x563d81766da0 0x563d81c094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:13.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.097+0000 7f694737d740 4 client.15891 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:52:13.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.097+0000 7f694737d740 1 -- 172.21.15.67:0/1805562963 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f69300035f0 con 0x563d81766da0 2024-07-31T14:52:13.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.097+0000 7f6939d9b640 1 --2- 172.21.15.67:0/1805562963 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x563d81766da0 0x563d81c094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:13.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.098+0000 7f6939d9b640 1 --2- 172.21.15.67:0/1805562963 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x563d81766da0 0x563d81c094f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=19ec7d52027d8bf2 server_cookie=ad9d10678ece73b1 in_seq=0 out_seq=0 2024-07-31T14:52:13.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.098+0000 7f6921ffb640 10 client.15891 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:52:13.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.098+0000 7f6921ffb640 1 -- 172.21.15.67:0/1805562963 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f69300035f0 con 0x563d81766da0 2024-07-31T14:52:13.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.098+0000 7f6921ffb640 10 client.15891 handle_command_reply: tid=0 2024-07-31T14:52:13.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.098+0000 7f694737d740 1 client.15891 shutdown 2024-07-31T14:52:13.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.098+0000 7f694737d740 1 -- 172.21.15.67:0/1805562963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817aa1b0 msgr2=0x563d81b964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:13.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.098+0000 7f694737d740 1 --2- 172.21.15.67:0/1805562963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817aa1b0 0x563d81b964a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6934005d70 tx=0x7f69340059c0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.098+0000 7f694737d740 1 -- 172.21.15.67:0/1805562963 shutdown_connections 2024-07-31T14:52:13.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.098+0000 7f694737d740 1 --2- 172.21.15.67:0/1805562963 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x563d81766da0 0x563d81c094f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.098+0000 7f694737d740 1 --2- 172.21.15.67:0/1805562963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d817d3070 0x563d81c054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.098+0000 7f6921ffb640 0 client.15891 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:52:13.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.098+0000 7f694737d740 1 --2- 172.21.15.67:0/1805562963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d817b2390 0x563d81b96a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.098+0000 7f694737d740 1 --2- 172.21.15.67:0/1805562963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d817aa1b0 0x563d81b964a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.098+0000 7f694737d740 1 -- 172.21.15.67:0/1805562963 >> 172.21.15.67:0/1805562963 conn(0x563d817c7460 msgr2=0x563d81b7d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:13.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.099+0000 7f694737d740 1 -- 172.21.15.67:0/1805562963 shutdown_connections 2024-07-31T14:52:13.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.099+0000 7f694737d740 1 -- 172.21.15.67:0/1805562963 wait complete. 2024-07-31T14:52:13.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.099+0000 7f694737d740 20 client.15891 trim_cache size 0 max 16384 2024-07-31T14:52:13.104 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:52:13.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.099+0000 7f6940e2c640 1 -- 172.21.15.67:0/1849649047 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69240b3dd0 msgr2=0x7f69240b62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:13.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.099+0000 7f6940e2c640 1 --2- 172.21.15.67:0/1849649047 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69240b3dd0 0x7f69240b62e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f692c0e1c80 tx=0x7f692c0ea480 comp rx=0 tx=0).stop 2024-07-31T14:52:13.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.099+0000 7f6940e2c640 1 -- 172.21.15.67:0/1849649047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f693c102490 msgr2=0x7f693c199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:13.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.099+0000 7f6940e2c640 1 --2- 172.21.15.67:0/1849649047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f693c102490 0x7f693c199f10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f692c00e9f0 tx=0x7f692c00eeb0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.100+0000 7f6940e2c640 1 -- 172.21.15.67:0/1849649047 shutdown_connections 2024-07-31T14:52:13.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.100+0000 7f6940e2c640 1 --2- 172.21.15.67:0/1849649047 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69240b3dd0 0x7f69240b62e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.100+0000 7f6940e2c640 1 --2- 172.21.15.67:0/1849649047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f693c108720 0x7f693c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.100+0000 7f6940e2c640 1 --2- 172.21.15.67:0/1849649047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f693c107dd0 0x7f693c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.100+0000 7f6940e2c640 1 --2- 172.21.15.67:0/1849649047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f693c102490 0x7f693c199f10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:13.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.100+0000 7f6940e2c640 1 -- 172.21.15.67:0/1849649047 >> 172.21.15.67:0/1849649047 conn(0x7f693c079160 msgr2=0x7f693c0ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:13.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.100+0000 7f6940e2c640 1 -- 172.21.15.67:0/1849649047 shutdown_connections 2024-07-31T14:52:13.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:13.100+0000 7f6940e2c640 1 -- 172.21.15.67:0/1849649047 wait complete. 2024-07-31T14:52:13.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:12 smithi073 ceph-mon[36714]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-07-31T14:52:13.878 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T14:52:14.160 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:14.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:14 smithi067 ceph-mon[32750]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-07-31T14:52:14.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:14 smithi077 ceph-mon[37327]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-07-31T14:52:15.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:14 smithi073 ceph-mon[36714]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-07-31T14:52:15.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.977+0000 7f1fa12c5640 1 -- 172.21.15.67:0/3162877485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f9c109830 msgr2=0x7f1f9c110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:15.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.977+0000 7f1fa12c5640 1 --2- 172.21.15.67:0/3162877485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f9c109830 0x7f1f9c110060 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009ca0 tx=0x7f1f9402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:52:15.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.977+0000 7f1fa12c5640 1 -- 172.21.15.67:0/3162877485 shutdown_connections 2024-07-31T14:52:15.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.977+0000 7f1fa12c5640 1 --2- 172.21.15.67:0/3162877485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f9c109830 0x7f1f9c110060 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.977+0000 7f1fa12c5640 1 --2- 172.21.15.67:0/3162877485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f9c108ee0 0x7f1f9c1092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.977+0000 7f1fa12c5640 1 --2- 172.21.15.67:0/3162877485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f9c1035f0 0x7f1f9c103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.977+0000 7f1fa12c5640 1 -- 172.21.15.67:0/3162877485 >> 172.21.15.67:0/3162877485 conn(0x7f1f9c07b340 msgr2=0x7f1f9c07b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.977+0000 7f1fa12c5640 1 -- 172.21.15.67:0/3162877485 shutdown_connections 2024-07-31T14:52:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.977+0000 7f1fa12c5640 1 -- 172.21.15.67:0/3162877485 wait complete. 2024-07-31T14:52:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.978+0000 7f1fa12c5640 1 Processor -- start 2024-07-31T14:52:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.979+0000 7f1fa12c5640 1 -- start start 2024-07-31T14:52:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.979+0000 7f1fa12c5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f9c1035f0 0x7f1f9c199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.979+0000 7f1fa12c5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f9c108ee0 0x7f1f9c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.979+0000 7f1fa12c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f9c109830 0x7f1f9c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.979+0000 7f1fa12c5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c113550 con 0x7f1f9c108ee0 2024-07-31T14:52:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.979+0000 7f1fa12c5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1133d0 con 0x7f1f9c1035f0 2024-07-31T14:52:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.979+0000 7f1fa12c5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1136d0 con 0x7f1f9c109830 2024-07-31T14:52:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.979+0000 7f1f9affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f9c1035f0 0x7f1f9c199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.979+0000 7f1f9a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f9c108ee0 0x7f1f9c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.979+0000 7f1f9affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f9c1035f0 0x7f1f9c199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35626/0 (socket says 172.21.15.67:35626) 2024-07-31T14:52:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.979+0000 7f1f9affd640 1 -- 172.21.15.67:0/3828451258 learned_addr learned my addr 172.21.15.67:0/3828451258 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.979+0000 7f1f9b7fe640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f9c109830 0x7f1f9c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.980+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/3828451258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f9c109830 msgr2=0x7f1f9c19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.980+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f9c109830 0x7f1f9c19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.980+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/3828451258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f9c1035f0 msgr2=0x7f1f9c199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.980+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f9c1035f0 0x7f1f9c199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.980+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/3828451258 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c19fde0 con 0x7f1f9c108ee0 2024-07-31T14:52:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.980+0000 7f1f9b7fe640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f9c109830 0x7f1f9c19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.980+0000 7f1f9affd640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f9c1035f0 0x7f1f9c199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.980+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f9c108ee0 0x7f1f9c19a410 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c06b560 tx=0x7f1f9000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.980+0000 7f1f83fff640 1 -- 172.21.15.67:0/3828451258 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9000cc40 con 0x7f1f9c108ee0 2024-07-31T14:52:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.980+0000 7f1f83fff640 1 -- 172.21.15.67:0/3828451258 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f9000cde0 con 0x7f1f9c108ee0 2024-07-31T14:52:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.980+0000 7f1fa12c5640 1 -- 172.21.15.67:0/3828451258 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c1a00d0 con 0x7f1f9c108ee0 2024-07-31T14:52:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.980+0000 7f1f83fff640 1 -- 172.21.15.67:0/3828451258 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90004a00 con 0x7f1f9c108ee0 2024-07-31T14:52:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.980+0000 7f1fa12c5640 1 -- 172.21.15.67:0/3828451258 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c1a6830 con 0x7f1f9c108ee0 2024-07-31T14:52:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.984+0000 7f1f83fff640 1 -- 172.21.15.67:0/3828451258 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f1f90004ba0 con 0x7f1f9c108ee0 2024-07-31T14:52:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.984+0000 7f1fa7816740 1 Processor -- start 2024-07-31T14:52:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.984+0000 7f1fa7816740 1 -- start start 2024-07-31T14:52:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.985+0000 7f1fa7816740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd76e070 0x55e6cdb13690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.985+0000 7f1fa7816740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd7451b0 0x55e6cdb13bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.985+0000 7f1fa7816740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd74d390 0x55e6cdb14110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.985+0000 7f1fa7816740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e6cda89bb0 con 0x55e6cd76e070 2024-07-31T14:52:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.985+0000 7f1fa7816740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e6cda790b0 con 0x55e6cd7451b0 2024-07-31T14:52:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.985+0000 7f1fa7816740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e6cda7d0f0 con 0x55e6cd74d390 2024-07-31T14:52:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.985+0000 7f1f9a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd74d390 0x55e6cdb14110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.985+0000 7f1f9a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd74d390 0x55e6cdb14110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35952/0 (socket says 172.21.15.67:35952) 2024-07-31T14:52:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.985+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/2719358235 learned_addr learned my addr 172.21.15.67:0/2719358235 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.985+0000 7f1f9affd640 1 --2- 172.21.15.67:0/2719358235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd7451b0 0x55e6cdb13bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.985+0000 7f1f9b7fe640 1 --2- 172.21.15.67:0/2719358235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd76e070 0x55e6cdb13690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.986+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/2719358235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd7451b0 msgr2=0x55e6cdb13bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.986+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/2719358235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd7451b0 0x55e6cdb13bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.986+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/2719358235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd76e070 msgr2=0x55e6cdb13690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.986+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/2719358235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd76e070 0x55e6cdb13690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.986+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/2719358235 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6cda1fb90 con 0x55e6cd74d390 2024-07-31T14:52:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.986+0000 7f1f83fff640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1f840b3e50 0x7f1f840b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.986+0000 7f1f9b7fe640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1f840b3e50 0x7f1f840b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.986+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/2719358235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd74d390 0x55e6cdb14110 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f900e1770 tx=0x7f1f900ec660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1885d5cd931850b5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.986+0000 7f1f83fff640 1 -- 172.21.15.67:0/3828451258 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f1f900e4450 con 0x7f1f9c108ee0 2024-07-31T14:52:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.986+0000 7f1f817fa640 1 -- 172.21.15.67:0/2719358235 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f900ece60 con 0x55e6cd74d390 2024-07-31T14:52:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.986+0000 7f1f817fa640 1 -- 172.21.15.67:0/2719358235 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f900a6360 con 0x55e6cd74d390 2024-07-31T14:52:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.986+0000 7f1f817fa640 1 -- 172.21.15.67:0/2719358235 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f900aa350 con 0x55e6cd74d390 2024-07-31T14:52:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.987+0000 7f1fa7816740 1 -- 172.21.15.67:0/2719358235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd74d390 msgr2=0x55e6cdb14110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.987+0000 7f1fa7816740 1 --2- 172.21.15.67:0/2719358235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd74d390 0x55e6cdb14110 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f900e1770 tx=0x7f1f900ec660 comp rx=0 tx=0).stop 2024-07-31T14:52:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.987+0000 7f1f9b7fe640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1f840b3e50 0x7f1f840b6360 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1f94004af0 tx=0x7f1f94004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.988+0000 7f1fa7816740 1 -- 172.21.15.67:0/2719358235 shutdown_connections 2024-07-31T14:52:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.988+0000 7f1fa7816740 1 --2- 172.21.15.67:0/2719358235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd74d390 0x55e6cdb14110 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.988+0000 7f1fa7816740 1 --2- 172.21.15.67:0/2719358235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd7451b0 0x55e6cdb13bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.988+0000 7f1fa7816740 1 --2- 172.21.15.67:0/2719358235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd76e070 0x55e6cdb13690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.988+0000 7f1fa7816740 1 -- 172.21.15.67:0/2719358235 >> 172.21.15.67:0/2719358235 conn(0x55e6cd762460 msgr2=0x55e6cdb11580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.988+0000 7f1fa7816740 1 -- 172.21.15.67:0/2719358235 shutdown_connections 2024-07-31T14:52:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.988+0000 7f1fa7816740 1 -- 172.21.15.67:0/2719358235 wait complete. 2024-07-31T14:52:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.988+0000 7f1fa7816740 1 -- start start 2024-07-31T14:52:15.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.988+0000 7f1fa7816740 1 Processor -- start 2024-07-31T14:52:15.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1fa7816740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd7451b0 0x55e6cdb1f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:15.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1fa7816740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd74d390 0x55e6cdb1f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:15.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1fa7816740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd76e070 0x55e6cdb2b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:15.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1fa7816740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e6cd8dc700 con 0x55e6cd76e070 2024-07-31T14:52:15.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1fa7816740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e6cda1b9e0 con 0x55e6cd7451b0 2024-07-31T14:52:15.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1fa7816740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e6cdb21ec0 con 0x55e6cd74d390 2024-07-31T14:52:15.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1fa7816740 10 client.0 _ll_register_callbacks cb 0x55e6cda364f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:15.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1f9b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd76e070 0x55e6cdb2b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1f9affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd7451b0 0x55e6cdb1f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1f9b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd76e070 0x55e6cdb2b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37840/0 (socket says 172.21.15.67:37840) 2024-07-31T14:52:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1f9b7fe640 1 -- 172.21.15.67:0/3838111526 learned_addr learned my addr 172.21.15.67:0/3838111526 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/3838111526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd74d390 0x55e6cdb1f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1f9affd640 1 -- 172.21.15.67:0/3838111526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd74d390 msgr2=0x55e6cdb1f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1f9affd640 1 --2- 172.21.15.67:0/3838111526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd74d390 0x55e6cdb1f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1f9affd640 1 -- 172.21.15.67:0/3838111526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd76e070 msgr2=0x55e6cdb2b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1f9affd640 1 --2- 172.21.15.67:0/3838111526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd76e070 0x55e6cdb2b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.989+0000 7f1f9affd640 1 -- 172.21.15.67:0/3838111526 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6cd7d9650 con 0x55e6cd7451b0 2024-07-31T14:52:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.990+0000 7f1f9affd640 1 --2- 172.21.15.67:0/3838111526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd7451b0 0x55e6cdb1f320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c01cef0 tx=0x7f1f8c012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a02391567da9b64f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.990+0000 7f1f7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:52:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.990+0000 7f1f7eff5640 1 -- 172.21.15.67:0/3838111526 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c004250 con 0x55e6cd7451b0 2024-07-31T14:52:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.990+0000 7f1fa7816740 1 -- 172.21.15.67:0/3838111526 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f900a6360 con 0x55e6cd7451b0 2024-07-31T14:52:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.990+0000 7f1f7eff5640 1 -- 172.21.15.67:0/3838111526 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f1f8c0043f0 con 0x55e6cd7451b0 2024-07-31T14:52:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.990+0000 7f1f7eff5640 1 -- 172.21.15.67:0/3838111526 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f8c005020 con 0x55e6cd7451b0 2024-07-31T14:52:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.990+0000 7f1fa7816740 10 client.25568 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.990+0000 7f1fa7816740 1 -- 172.21.15.67:0/3838111526 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e6cdb2b820 con 0x55e6cd7451b0 2024-07-31T14:52:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.990+0000 7f1f7eff5640 1 -- 172.21.15.67:0/3838111526 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f900a6360 con 0x55e6cd7451b0 2024-07-31T14:52:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.991+0000 7f1f7eff5640 1 -- 172.21.15.67:0/3838111526 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1f8c023070 con 0x55e6cd7451b0 2024-07-31T14:52:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.991+0000 7f1fa7816740 10 client.25568 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.991+0000 7f1fa7816740 10 client.25568 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:52:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.991+0000 7f1fa7816740 20 client.25568 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.991+0000 7f1fa7816740 1 --2- 172.21.15.67:0/3838111526 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55e6cd72c9f0 0x55e6cdb2ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.991+0000 7f1fa7816740 4 client.25568 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.991+0000 7f1fa7816740 1 -- 172.21.15.67:0/3838111526 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6cda8bb00 con 0x55e6cd72c9f0 2024-07-31T14:52:16.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.991+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/3838111526 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55e6cd72c9f0 0x55e6cdb2ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:16.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.992+0000 7f1f7eff5640 10 client.25568 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:52:16.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.992+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/3838111526 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55e6cd72c9f0 0x55e6cdb2ed90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70d30be21c17d14d server_cookie=135a7146601c2762 in_seq=0 out_seq=0 2024-07-31T14:52:16.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.992+0000 7f1f7eff5640 1 -- 172.21.15.67:0/3838111526 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6cda8bb00 con 0x55e6cd72c9f0 2024-07-31T14:52:16.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.992+0000 7f1f7eff5640 10 client.25568 handle_command_reply: tid=0 2024-07-31T14:52:16.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.992+0000 7f1fa7816740 1 client.25568 shutdown 2024-07-31T14:52:16.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.993+0000 7f1fa7816740 1 -- 172.21.15.67:0/3838111526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd7451b0 msgr2=0x55e6cdb1f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:16.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.993+0000 7f1fa7816740 1 --2- 172.21.15.67:0/3838111526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd7451b0 0x55e6cdb1f320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c01cef0 tx=0x7f1f8c012820 comp rx=0 tx=0).stop 2024-07-31T14:52:16.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.993+0000 7f1fa7816740 1 -- 172.21.15.67:0/3838111526 shutdown_connections 2024-07-31T14:52:16.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.993+0000 7f1fa7816740 1 --2- 172.21.15.67:0/3838111526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd76e070 0x55e6cdb2b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.993+0000 7f1fa7816740 1 --2- 172.21.15.67:0/3838111526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd74d390 0x55e6cdb1f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.993+0000 7f1fa7816740 1 --2- 172.21.15.67:0/3838111526 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55e6cd72c9f0 0x55e6cdb2ed90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.993+0000 7f1fa7816740 1 --2- 172.21.15.67:0/3838111526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd7451b0 0x55e6cdb1f320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.993+0000 7f1f7eff5640 0 client.25568 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:52:16.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.993+0000 7f1fa7816740 1 -- 172.21.15.67:0/3838111526 >> 172.21.15.67:0/3838111526 conn(0x55e6cd762460 msgr2=0x55e6cdb16870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:16.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.993+0000 7f1fa7816740 1 -- 172.21.15.67:0/3838111526 shutdown_connections 2024-07-31T14:52:16.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.993+0000 7f1fa7816740 1 -- 172.21.15.67:0/3838111526 wait complete. 2024-07-31T14:52:16.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:15.993+0000 7f1fa7816740 20 client.25568 trim_cache size 0 max 16384 2024-07-31T14:52:16.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.007+0000 7f1fa7816740 1 Processor -- start 2024-07-31T14:52:16.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.007+0000 7f1fa7816740 1 -- start start 2024-07-31T14:52:16.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.007+0000 7f1fa7816740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd74d390 0x55e6cdb115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.007+0000 7f1fa7816740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd76e070 0x55e6cdb2b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.007+0000 7f1fa7816740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd701da0 0x55e6cdb18910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.007+0000 7f1fa7816740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1ae530 con 0x55e6cd74d390 2024-07-31T14:52:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.007+0000 7f1fa7816740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1b0070 con 0x55e6cd76e070 2024-07-31T14:52:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.007+0000 7f1fa7816740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1aec00 con 0x55e6cd701da0 2024-07-31T14:52:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.008+0000 7f1f9a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd76e070 0x55e6cdb2b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.008+0000 7f1f9affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd74d390 0x55e6cdb115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.008+0000 7f1f9a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd76e070 0x55e6cdb2b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35652/0 (socket says 172.21.15.67:35652) 2024-07-31T14:52:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.008+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/2483429083 learned_addr learned my addr 172.21.15.67:0/2483429083 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.008+0000 7f1f9b7fe640 1 --2- 172.21.15.67:0/2483429083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd701da0 0x55e6cdb18910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.008+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/2483429083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd701da0 msgr2=0x55e6cdb18910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.008+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/2483429083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd701da0 0x55e6cdb18910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.008+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/2483429083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd74d390 msgr2=0x55e6cdb115f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.008+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/2483429083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd74d390 0x55e6cdb115f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.008+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/2483429083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6cdb203e0 con 0x55e6cd76e070 2024-07-31T14:52:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.008+0000 7f1f9b7fe640 1 --2- 172.21.15.67:0/2483429083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd701da0 0x55e6cdb18910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.008+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/2483429083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd76e070 0x55e6cdb2b120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1f900e1e90 tx=0x7f1f900ec150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb1114d4d09bd1a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.009+0000 7f1f80ff9640 1 -- 172.21.15.67:0/2483429083 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9001c040 con 0x55e6cd76e070 2024-07-31T14:52:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.009+0000 7f1f80ff9640 1 -- 172.21.15.67:0/2483429083 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f900a6360 con 0x55e6cd76e070 2024-07-31T14:52:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.009+0000 7f1f80ff9640 1 -- 172.21.15.67:0/2483429083 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f900aa530 con 0x55e6cd76e070 2024-07-31T14:52:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.010+0000 7f1fa7816740 1 -- 172.21.15.67:0/2483429083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd76e070 msgr2=0x55e6cdb2b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.010+0000 7f1fa7816740 1 --2- 172.21.15.67:0/2483429083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd76e070 0x55e6cdb2b120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1f900e1e90 tx=0x7f1f900ec150 comp rx=0 tx=0).stop 2024-07-31T14:52:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.010+0000 7f1fa7816740 1 -- 172.21.15.67:0/2483429083 shutdown_connections 2024-07-31T14:52:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.010+0000 7f1fa7816740 1 --2- 172.21.15.67:0/2483429083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd701da0 0x55e6cdb18910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.010+0000 7f1fa7816740 1 --2- 172.21.15.67:0/2483429083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd76e070 0x55e6cdb2b120 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.010+0000 7f1fa7816740 1 --2- 172.21.15.67:0/2483429083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd74d390 0x55e6cdb115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.010+0000 7f1fa7816740 1 -- 172.21.15.67:0/2483429083 >> 172.21.15.67:0/2483429083 conn(0x55e6cd762460 msgr2=0x55e6cdb10ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.010+0000 7f1fa7816740 1 -- 172.21.15.67:0/2483429083 shutdown_connections 2024-07-31T14:52:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.010+0000 7f1fa7816740 1 -- 172.21.15.67:0/2483429083 wait complete. 2024-07-31T14:52:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.010+0000 7f1fa7816740 1 -- start start 2024-07-31T14:52:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.010+0000 7f1fa7816740 1 Processor -- start 2024-07-31T14:52:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.011+0000 7f1fa7816740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd701da0 0x55e6cdb32610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.011+0000 7f1fa7816740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd74d390 0x55e6cdb32b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.011+0000 7f1fa7816740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd76e070 0x55e6cdb98350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.011+0000 7f1f9a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd74d390 0x55e6cdb32b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.011+0000 7f1f9a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd74d390 0x55e6cdb32b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37854/0 (socket says 172.21.15.67:37854) 2024-07-31T14:52:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.011+0000 7f1f9affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd701da0 0x55e6cdb32610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.011+0000 7f1f9affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd701da0 0x55e6cdb32610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35982/0 (socket says 172.21.15.67:35982) 2024-07-31T14:52:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.011+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/304760039 learned_addr learned my addr 172.21.15.67:0/304760039 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.011+0000 7f1fa7816740 1 -- 172.21.15.67:0/304760039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1af9a0 con 0x55e6cd74d390 2024-07-31T14:52:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.011+0000 7f1fa7816740 1 -- 172.21.15.67:0/304760039 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1af2d0 con 0x55e6cd76e070 2024-07-31T14:52:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.011+0000 7f1fa7816740 1 -- 172.21.15.67:0/304760039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1ac510 con 0x55e6cd701da0 2024-07-31T14:52:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.012+0000 7f1f9b7fe640 1 --2- 172.21.15.67:0/304760039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd76e070 0x55e6cdb98350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.012+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/304760039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd701da0 msgr2=0x55e6cdb32610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.012+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/304760039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd701da0 0x55e6cdb32610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.012+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/304760039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd76e070 msgr2=0x55e6cdb98350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.012+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/304760039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd76e070 0x55e6cdb98350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.012+0000 7f1f9a7fc640 1 -- 172.21.15.67:0/304760039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6cdb33380 con 0x55e6cd74d390 2024-07-31T14:52:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.012+0000 7f1fa7816740 10 client.0 _ll_register_callbacks cb 0x7f1f8c009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.012+0000 7f1fa7816740 1 -- 172.21.15.67:0/304760039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f900a6360 con 0x55e6cd74d390 2024-07-31T14:52:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.012+0000 7f1f9a7fc640 1 --2- 172.21.15.67:0/304760039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd74d390 0x55e6cdb32b50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1f900e18b0 tx=0x7f1f90005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc985768f8e44b9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.012+0000 7f1f7eff5640 10 client.15909 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.012+0000 7f1f7eff5640 1 -- 172.21.15.67:0/304760039 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9008a050 con 0x55e6cd74d390 2024-07-31T14:52:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.012+0000 7f1f7eff5640 1 -- 172.21.15.67:0/304760039 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f1f900e1a80 con 0x55e6cd74d390 2024-07-31T14:52:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.013+0000 7f1f7eff5640 1 -- 172.21.15.67:0/304760039 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f9001c040 con 0x55e6cd74d390 2024-07-31T14:52:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.013+0000 7f1fa7816740 10 client.15909 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.013+0000 7f1fa7816740 1 -- 172.21.15.67:0/304760039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55e6cdb99660 con 0x55e6cd74d390 2024-07-31T14:52:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.013+0000 7f1f7eff5640 1 -- 172.21.15.67:0/304760039 <== mon.0 v2:172.21.15.67:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1f900920c0 con 0x55e6cd74d390 2024-07-31T14:52:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.013+0000 7f1fa7816740 10 client.15909 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:16.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.013+0000 7f1fa7816740 10 client.15909 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:52:16.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.013+0000 7f1fa7816740 20 client.15909 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:16.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.013+0000 7f1fa7816740 1 --2- 172.21.15.67:0/304760039 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55e6cdb99830 0x55e6cdb9bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:16.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.013+0000 7f1fa7816740 4 client.15909 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:52:16.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.013+0000 7f1fa7816740 1 -- 172.21.15.67:0/304760039 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6cda875a0 con 0x55e6cdb99830 2024-07-31T14:52:16.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.014+0000 7f1f9affd640 1 --2- 172.21.15.67:0/304760039 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55e6cdb99830 0x55e6cdb9bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:16.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.014+0000 7f1f9affd640 1 --2- 172.21.15.67:0/304760039 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55e6cdb99830 0x55e6cdb9bc50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5234b4db3bc9f4df server_cookie=9426538fefd93b22 in_seq=0 out_seq=0 2024-07-31T14:52:16.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.014+0000 7f1f7eff5640 10 client.15909 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:52:16.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.014+0000 7f1f7eff5640 1 -- 172.21.15.67:0/304760039 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6cda875a0 con 0x55e6cdb99830 2024-07-31T14:52:16.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.014+0000 7f1f7eff5640 10 client.15909 handle_command_reply: tid=0 2024-07-31T14:52:16.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.014+0000 7f1fa7816740 1 client.15909 shutdown 2024-07-31T14:52:16.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.014+0000 7f1fa7816740 1 -- 172.21.15.67:0/304760039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd74d390 msgr2=0x55e6cdb32b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:16.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.015+0000 7f1fa7816740 1 --2- 172.21.15.67:0/304760039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd74d390 0x55e6cdb32b50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1f900e18b0 tx=0x7f1f90005250 comp rx=0 tx=0).stop 2024-07-31T14:52:16.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.015+0000 7f1fa7816740 1 -- 172.21.15.67:0/304760039 shutdown_connections 2024-07-31T14:52:16.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.015+0000 7f1fa7816740 1 --2- 172.21.15.67:0/304760039 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55e6cdb99830 0x55e6cdb9bc50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.015+0000 7f1fa7816740 1 --2- 172.21.15.67:0/304760039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6cd76e070 0x55e6cdb98350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.015+0000 7f1f7eff5640 0 client.15909 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:52:16.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.015+0000 7f1fa7816740 1 --2- 172.21.15.67:0/304760039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e6cd74d390 0x55e6cdb32b50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.015+0000 7f1fa7816740 1 --2- 172.21.15.67:0/304760039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6cd701da0 0x55e6cdb32610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.015+0000 7f1fa7816740 1 -- 172.21.15.67:0/304760039 >> 172.21.15.67:0/304760039 conn(0x55e6cd762460 msgr2=0x55e6cdb1d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:16.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.015+0000 7f1fa7816740 1 -- 172.21.15.67:0/304760039 shutdown_connections 2024-07-31T14:52:16.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.015+0000 7f1fa7816740 1 -- 172.21.15.67:0/304760039 wait complete. 2024-07-31T14:52:16.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.015+0000 7f1fa7816740 20 client.15909 trim_cache size 0 max 16384 2024-07-31T14:52:16.019 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:52:16.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.015+0000 7f1fa12c5640 1 -- 172.21.15.67:0/3828451258 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1f840b3e50 msgr2=0x7f1f840b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:16.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.015+0000 7f1fa12c5640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1f840b3e50 0x7f1f840b6360 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1f94004af0 tx=0x7f1f94004a40 comp rx=0 tx=0).stop 2024-07-31T14:52:16.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.016+0000 7f1fa12c5640 1 -- 172.21.15.67:0/3828451258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f9c108ee0 msgr2=0x7f1f9c19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:16.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.016+0000 7f1fa12c5640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f9c108ee0 0x7f1f9c19a410 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c06b560 tx=0x7f1f9000ee50 comp rx=0 tx=0).stop 2024-07-31T14:52:16.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.016+0000 7f1fa12c5640 1 -- 172.21.15.67:0/3828451258 shutdown_connections 2024-07-31T14:52:16.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.016+0000 7f1fa12c5640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1f840b3e50 0x7f1f840b6360 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.016+0000 7f1fa12c5640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f9c109830 0x7f1f9c19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.016+0000 7f1fa12c5640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f9c108ee0 0x7f1f9c19a410 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.016+0000 7f1fa12c5640 1 --2- 172.21.15.67:0/3828451258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f9c1035f0 0x7f1f9c199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:16.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.016+0000 7f1fa12c5640 1 -- 172.21.15.67:0/3828451258 >> 172.21.15.67:0/3828451258 conn(0x7f1f9c07b340 msgr2=0x7f1f9c106e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:16.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.016+0000 7f1fa12c5640 1 -- 172.21.15.67:0/3828451258 shutdown_connections 2024-07-31T14:52:16.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:16.016+0000 7f1fa12c5640 1 -- 172.21.15.67:0/3828451258 wait complete. 2024-07-31T14:52:16.924 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T14:52:16.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:16 smithi077 ceph-mon[37327]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-07-31T14:52:17.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:16 smithi067 ceph-mon[32750]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-07-31T14:52:17.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:16 smithi073 ceph-mon[36714]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-07-31T14:52:17.230 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:18.991 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:18 smithi067 ceph-mon[32750]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-07-31T14:52:18.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:18 smithi077 ceph-mon[37327]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-07-31T14:52:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.000+0000 7f5d0bf59640 1 -- 172.21.15.67:0/3071434447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d04108670 msgr2=0x7f5d0410eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.000+0000 7f5d0bf59640 1 --2- 172.21.15.67:0/3071434447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d04108670 0x7f5d0410eea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000b9f0 tx=0x7f5d0002c0b0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.001+0000 7f5d0bf59640 1 -- 172.21.15.67:0/3071434447 shutdown_connections 2024-07-31T14:52:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.001+0000 7f5d0bf59640 1 --2- 172.21.15.67:0/3071434447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d04108670 0x7f5d0410eea0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.001+0000 7f5d0bf59640 1 --2- 172.21.15.67:0/3071434447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d04107d20 0x7f5d04108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.001+0000 7f5d0bf59640 1 --2- 172.21.15.67:0/3071434447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d041023f0 0x7f5d04102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.001+0000 7f5d0bf59640 1 -- 172.21.15.67:0/3071434447 >> 172.21.15.67:0/3071434447 conn(0x7f5d040fddf0 msgr2=0x7f5d04100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.001+0000 7f5d0bf59640 1 -- 172.21.15.67:0/3071434447 shutdown_connections 2024-07-31T14:52:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.001+0000 7f5d0bf59640 1 -- 172.21.15.67:0/3071434447 wait complete. 2024-07-31T14:52:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.002+0000 7f5d0bf59640 1 Processor -- start 2024-07-31T14:52:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.002+0000 7f5d0bf59640 1 -- start start 2024-07-31T14:52:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.002+0000 7f5d0bf59640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d041023f0 0x7f5d0410e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.002+0000 7f5d0bf59640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d04107d20 0x7f5d0410e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.002+0000 7f5d0bf59640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d04108670 0x7f5d041a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.002+0000 7f5d0bf59640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5d04111250 con 0x7f5d04107d20 2024-07-31T14:52:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.002+0000 7f5d0bf59640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5d041110d0 con 0x7f5d041023f0 2024-07-31T14:52:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.002+0000 7f5d0bf59640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5d041113d0 con 0x7f5d04108670 2024-07-31T14:52:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.003+0000 7f5d09cf5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d041023f0 0x7f5d0410e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.003+0000 7f5d094f4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d04107d20 0x7f5d0410e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.003+0000 7f5d09cf5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d041023f0 0x7f5d0410e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35676/0 (socket says 172.21.15.67:35676) 2024-07-31T14:52:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.003+0000 7f5d09cf5640 1 -- 172.21.15.67:0/843374333 learned_addr learned my addr 172.21.15.67:0/843374333 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.003+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d04108670 0x7f5d041a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.003+0000 7f5d09cf5640 1 -- 172.21.15.67:0/843374333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d04108670 msgr2=0x7f5d041a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.003+0000 7f5d09cf5640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d04108670 0x7f5d041a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.003+0000 7f5d09cf5640 1 -- 172.21.15.67:0/843374333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d04107d20 msgr2=0x7f5d0410e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.003+0000 7f5d09cf5640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d04107d20 0x7f5d0410e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.003+0000 7f5d09cf5640 1 -- 172.21.15.67:0/843374333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d041a6340 con 0x7f5d041023f0 2024-07-31T14:52:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.003+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d04108670 0x7f5d041a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.003+0000 7f5d094f4640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d04107d20 0x7f5d0410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.004+0000 7f5d09cf5640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d041023f0 0x7f5d0410e3d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00efb0 tx=0x7f5cfc00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.004+0000 7f5cfaffd640 1 -- 172.21.15.67:0/843374333 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc01b070 con 0x7f5d041023f0 2024-07-31T14:52:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.004+0000 7f5d0bf59640 1 -- 172.21.15.67:0/843374333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d041a6630 con 0x7f5d041023f0 2024-07-31T14:52:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.004+0000 7f5d0bf59640 1 -- 172.21.15.67:0/843374333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d041acdc0 con 0x7f5d041023f0 2024-07-31T14:52:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.005+0000 7f5cfaffd640 1 -- 172.21.15.67:0/843374333 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cfc00ba20 con 0x7f5d041023f0 2024-07-31T14:52:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.005+0000 7f5cfaffd640 1 -- 172.21.15.67:0/843374333 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc01b070 con 0x7f5d041023f0 2024-07-31T14:52:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.008+0000 7f5d12e4c740 1 Processor -- start 2024-07-31T14:52:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.008+0000 7f5d12e4c740 1 -- start start 2024-07-31T14:52:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.008+0000 7f5d12e4c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b4ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.008+0000 7f5d12e4c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a107832f0 0x560a10b4f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.008+0000 7f5d12e4c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1078b4d0 0x560a10b4f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.008+0000 7f5d12e4c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560a10ac43b0 con 0x560a107832f0 2024-07-31T14:52:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.008+0000 7f5d12e4c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560a10ab38b0 con 0x560a107ac1b0 2024-07-31T14:52:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.008+0000 7f5d12e4c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560a10ab78f0 con 0x560a1078b4d0 2024-07-31T14:52:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.009+0000 7f5d0a4f6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b4ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.009+0000 7f5d0a4f6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b4ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35678/0 (socket says 172.21.15.67:35678) 2024-07-31T14:52:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.009+0000 7f5d0a4f6640 1 -- 172.21.15.67:0/3465884748 learned_addr learned my addr 172.21.15.67:0/3465884748 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.009+0000 7f5d094f4640 1 --2- 172.21.15.67:0/3465884748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1078b4d0 0x560a10b4f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.009+0000 7f5d0a4f6640 1 -- 172.21.15.67:0/3465884748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1078b4d0 msgr2=0x560a10b4f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.009+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/3465884748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1078b4d0 0x560a10b4f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.009+0000 7f5d0a4f6640 1 -- 172.21.15.67:0/3465884748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a107832f0 msgr2=0x560a10b4f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.009+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/3465884748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a107832f0 0x560a10b4f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.009+0000 7f5d0a4f6640 1 -- 172.21.15.67:0/3465884748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a10a5a9a0 con 0x560a107ac1b0 2024-07-31T14:52:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.009+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/3465884748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b4ece0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5d00005e20 tx=0x7f5d00005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bfc2fc56358a06b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.010+0000 7f5ceffff640 1 -- 172.21.15.67:0/3465884748 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00042020 con 0x560a107ac1b0 2024-07-31T14:52:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.010+0000 7f5ceffff640 1 -- 172.21.15.67:0/3465884748 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0003a040 con 0x560a107ac1b0 2024-07-31T14:52:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.010+0000 7f5ceffff640 1 -- 172.21.15.67:0/3465884748 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00039070 con 0x560a107ac1b0 2024-07-31T14:52:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.011+0000 7f5d12e4c740 1 -- 172.21.15.67:0/3465884748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 msgr2=0x560a10b4ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.011+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/3465884748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b4ece0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5d00005e20 tx=0x7f5d00005e50 comp rx=0 tx=0).stop 2024-07-31T14:52:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.011+0000 7f5d12e4c740 1 -- 172.21.15.67:0/3465884748 shutdown_connections 2024-07-31T14:52:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.011+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/3465884748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1078b4d0 0x560a10b4f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.011+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/3465884748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a107832f0 0x560a10b4f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.011+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/3465884748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b4ece0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.011+0000 7f5d12e4c740 1 -- 172.21.15.67:0/3465884748 >> 172.21.15.67:0/3465884748 conn(0x560a107a0300 msgr2=0x560a10b4cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.012+0000 7f5cfaffd640 1 -- 172.21.15.67:0/843374333 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f5cfc01e070 con 0x7f5d041023f0 2024-07-31T14:52:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.012+0000 7f5d12e4c740 1 -- 172.21.15.67:0/3465884748 shutdown_connections 2024-07-31T14:52:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.013+0000 7f5d12e4c740 1 -- 172.21.15.67:0/3465884748 wait complete. 2024-07-31T14:52:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.013+0000 7f5cfaffd640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5cf00b3f60 0x7f5cf00b6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.013+0000 7f5cfaffd640 1 -- 172.21.15.67:0/843374333 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f5cfc0e5e90 con 0x7f5d041023f0 2024-07-31T14:52:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.013+0000 7f5d12e4c740 1 -- start start 2024-07-31T14:52:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.013+0000 7f5d12e4c740 1 Processor -- start 2024-07-31T14:52:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.014+0000 7f5d094f4640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5cf00b3f60 0x7f5cf00b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.014+0000 7f5d12e4c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 0x560a10b5a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.014+0000 7f5d12e4c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a107ac1b0 0x560a10b5ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.014+0000 7f5d12e4c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1076ad80 0x560a10b66780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.014+0000 7f5d12e4c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560a109edab0 con 0x560a1078b4d0 2024-07-31T14:52:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.014+0000 7f5d12e4c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560a10a3bcf0 con 0x560a1076ad80 2024-07-31T14:52:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.014+0000 7f5d12e4c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560a10b5d400 con 0x560a107ac1b0 2024-07-31T14:52:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.014+0000 7f5d12e4c740 10 client.0 _ll_register_callbacks cb 0x560a10aad9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.014+0000 7f5d0a4f6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1076ad80 0x560a10b66780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.014+0000 7f5d0a4f6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1076ad80 0x560a10b66780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35694/0 (socket says 172.21.15.67:35694) 2024-07-31T14:52:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.014+0000 7f5d09cf5640 1 --2- 172.21.15.67:0/2940574517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 0x560a10b5a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.014+0000 7f5d0a4f6640 1 -- 172.21.15.67:0/2940574517 learned_addr learned my addr 172.21.15.67:0/2940574517 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.015+0000 7f5d094f4640 1 --2- 172.21.15.67:0/2940574517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a107ac1b0 0x560a10b5ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.015+0000 7f5d0a4f6640 1 -- 172.21.15.67:0/2940574517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a107ac1b0 msgr2=0x560a10b5ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.015+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/2940574517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a107ac1b0 0x560a10b5ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.015+0000 7f5d0a4f6640 1 -- 172.21.15.67:0/2940574517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 msgr2=0x560a10b5a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.015+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/2940574517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 0x560a10b5a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.015+0000 7f5d0a4f6640 1 -- 172.21.15.67:0/2940574517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a10817790 con 0x560a1076ad80 2024-07-31T14:52:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.015+0000 7f5d094f4640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5cf00b3f60 0x7f5cf00b6470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5d041042e0 tx=0x7f5cf400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.015+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/2940574517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1076ad80 0x560a10b66780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009970 tx=0x7f5d00005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=762ea7658927154f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.015+0000 7f5ced7fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:52:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.016+0000 7f5ced7fa640 1 -- 172.21.15.67:0/2940574517 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0004b070 con 0x560a1076ad80 2024-07-31T14:52:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.016+0000 7f5ced7fa640 1 -- 172.21.15.67:0/2940574517 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f5d00004450 con 0x560a1076ad80 2024-07-31T14:52:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.016+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2940574517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d0003a040 con 0x560a1076ad80 2024-07-31T14:52:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.016+0000 7f5ced7fa640 1 -- 172.21.15.67:0/2940574517 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d00039070 con 0x560a1076ad80 2024-07-31T14:52:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.016+0000 7f5d12e4c740 10 client.25586 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.016+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2940574517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560a10890620 con 0x560a1076ad80 2024-07-31T14:52:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.016+0000 7f5ced7fa640 1 -- 172.21.15.67:0/2940574517 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0003a040 con 0x560a1076ad80 2024-07-31T14:52:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.017+0000 7f5ced7fa640 1 -- 172.21.15.67:0/2940574517 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5d000476c0 con 0x560a1076ad80 2024-07-31T14:52:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.017+0000 7f5d12e4c740 10 client.25586 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.017+0000 7f5d12e4c740 10 client.25586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.017+0000 7f5d12e4c740 20 client.25586 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.017+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/2940574517 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a107401c0 0x560a10b6c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.017+0000 7f5d12e4c740 4 client.25586 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.017+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2940574517 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a10ac6300 con 0x560a107401c0 2024-07-31T14:52:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.018+0000 7f5d09cf5640 1 --2- 172.21.15.67:0/2940574517 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a107401c0 0x560a10b6c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.018+0000 7f5d09cf5640 1 --2- 172.21.15.67:0/2940574517 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a107401c0 0x560a10b6c520 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d486999b4f8663f0 server_cookie=ef9754beb21fc276 in_seq=0 out_seq=0 2024-07-31T14:52:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.018+0000 7f5ced7fa640 10 client.25586 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.019+0000 7f5ced7fa640 1 -- 172.21.15.67:0/2940574517 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a10ac6300 con 0x560a107401c0 2024-07-31T14:52:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.019+0000 7f5ced7fa640 10 client.25586 handle_command_reply: tid=0 2024-07-31T14:52:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.019+0000 7f5d12e4c740 1 client.25586 shutdown 2024-07-31T14:52:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.019+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2940574517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1076ad80 msgr2=0x560a10b66780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.019+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/2940574517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1076ad80 0x560a10b66780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009970 tx=0x7f5d00005c90 comp rx=0 tx=0).stop 2024-07-31T14:52:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.019+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2940574517 shutdown_connections 2024-07-31T14:52:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.019+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/2940574517 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a107401c0 0x560a10b6c520 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.019+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/2940574517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1076ad80 0x560a10b66780 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.019+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/2940574517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a107ac1b0 0x560a10b5ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.019+0000 7f5ced7fa640 0 client.25586 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.019+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/2940574517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 0x560a10b5a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.019+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2940574517 >> 172.21.15.67:0/2940574517 conn(0x560a107a0300 msgr2=0x560a10b4fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.019+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2940574517 shutdown_connections 2024-07-31T14:52:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.019+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2940574517 wait complete. 2024-07-31T14:52:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.020+0000 7f5d12e4c740 20 client.25586 trim_cache size 0 max 16384 2024-07-31T14:52:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.050+0000 7f5d12e4c740 1 Processor -- start 2024-07-31T14:52:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.050+0000 7f5d12e4c740 1 -- start start 2024-07-31T14:52:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.050+0000 7f5d12e4c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1076ad80 0x560a10b54e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.050+0000 7f5d12e4c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 0x560a10b50340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.051+0000 7f5d12e4c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b66570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.051+0000 7f5d12e4c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce800fd50 con 0x560a1078b4d0 2024-07-31T14:52:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.051+0000 7f5d12e4c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce8011890 con 0x560a107ac1b0 2024-07-31T14:52:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.051+0000 7f5d094f4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 0x560a10b50340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.051+0000 7f5d094f4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 0x560a10b50340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37876/0 (socket says 172.21.15.67:37876) 2024-07-31T14:52:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.051+0000 7f5d094f4640 1 -- 172.21.15.67:0/95843638 learned_addr learned my addr 172.21.15.67:0/95843638 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.051+0000 7f5d09cf5640 1 --2- 172.21.15.67:0/95843638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1076ad80 0x560a10b54e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.051+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/95843638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b66570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.051+0000 7f5d12e4c740 1 -- 172.21.15.67:0/95843638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce8010420 con 0x560a1076ad80 2024-07-31T14:52:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.051+0000 7f5d0a4f6640 1 -- 172.21.15.67:0/95843638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1076ad80 msgr2=0x560a10b54e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.051+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/95843638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1076ad80 0x560a10b54e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.051+0000 7f5d0a4f6640 1 -- 172.21.15.67:0/95843638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 msgr2=0x560a10b50340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.051+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/95843638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 0x560a10b50340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.051+0000 7f5d0a4f6640 1 -- 172.21.15.67:0/95843638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a10b4da00 con 0x560a107ac1b0 2024-07-31T14:52:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.052+0000 7f5d094f4640 1 --2- 172.21.15.67:0/95843638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 0x560a10b50340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.052+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/95843638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b66570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5d0002c5d0 tx=0x7f5d00005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3551e20d5058fa65 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.052+0000 7f5cef7fe640 1 -- 172.21.15.67:0/95843638 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0004b040 con 0x560a107ac1b0 2024-07-31T14:52:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.052+0000 7f5cef7fe640 1 -- 172.21.15.67:0/95843638 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0000b660 con 0x560a107ac1b0 2024-07-31T14:52:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.052+0000 7f5cef7fe640 1 -- 172.21.15.67:0/95843638 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00042040 con 0x560a107ac1b0 2024-07-31T14:52:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.053+0000 7f5d12e4c740 1 -- 172.21.15.67:0/95843638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 msgr2=0x560a10b66570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.053+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/95843638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b66570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5d0002c5d0 tx=0x7f5d00005730 comp rx=0 tx=0).stop 2024-07-31T14:52:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.053+0000 7f5d12e4c740 1 -- 172.21.15.67:0/95843638 shutdown_connections 2024-07-31T14:52:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.053+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/95843638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b66570 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.053+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/95843638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 0x560a10b50340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.053+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/95843638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1076ad80 0x560a10b54e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.053+0000 7f5d12e4c740 1 -- 172.21.15.67:0/95843638 >> 172.21.15.67:0/95843638 conn(0x560a107a0300 msgr2=0x560a10b4fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.054+0000 7f5d12e4c740 1 -- 172.21.15.67:0/95843638 shutdown_connections 2024-07-31T14:52:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.054+0000 7f5d12e4c740 1 -- 172.21.15.67:0/95843638 wait complete. 2024-07-31T14:52:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.054+0000 7f5d12e4c740 1 -- start start 2024-07-31T14:52:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.054+0000 7f5d12e4c740 1 Processor -- start 2024-07-31T14:52:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.055+0000 7f5d12e4c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1076ad80 0x560a10b62eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.055+0000 7f5d12e4c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 0x560a10b5f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d12e4c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b5f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d12e4c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce80111c0 con 0x560a1078b4d0 2024-07-31T14:52:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d09cf5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1076ad80 0x560a10b62eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d094f4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 0x560a10b5f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d09cf5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1076ad80 0x560a10b62eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:36032/0 (socket says 172.21.15.67:36032) 2024-07-31T14:52:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d09cf5640 1 -- 172.21.15.67:0/2514459047 learned_addr learned my addr 172.21.15.67:0/2514459047 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2514459047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce8010af0 con 0x560a107ac1b0 2024-07-31T14:52:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2514459047 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce800dd30 con 0x560a1076ad80 2024-07-31T14:52:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/2514459047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b5f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d12e4c740 10 client.0 _ll_register_callbacks cb 0x7f5cf400bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d0a4f6640 1 -- 172.21.15.67:0/2514459047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1076ad80 msgr2=0x560a10b62eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/2514459047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1076ad80 0x560a10b62eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d0a4f6640 1 -- 172.21.15.67:0/2514459047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 msgr2=0x560a10b5f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/2514459047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 0x560a10b5f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d0a4f6640 1 -- 172.21.15.67:0/2514459047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a10b60120 con 0x560a107ac1b0 2024-07-31T14:52:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.056+0000 7f5d09cf5640 1 --2- 172.21.15.67:0/2514459047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1076ad80 0x560a10b62eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.057+0000 7f5d0a4f6640 1 --2- 172.21.15.67:0/2514459047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b5f990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5d0002c5d0 tx=0x7f5d00005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=507cb8be1697fa14 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.057+0000 7f5ced7fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:52:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.057+0000 7f5ced7fa640 1 -- 172.21.15.67:0/2514459047 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00068060 con 0x560a107ac1b0 2024-07-31T14:52:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.057+0000 7f5ced7fa640 1 -- 172.21.15.67:0/2514459047 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f5d00003a40 con 0x560a107ac1b0 2024-07-31T14:52:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.057+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2514459047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d0000b660 con 0x560a107ac1b0 2024-07-31T14:52:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.057+0000 7f5ced7fa640 1 -- 172.21.15.67:0/2514459047 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d0004b070 con 0x560a107ac1b0 2024-07-31T14:52:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.057+0000 7f5d12e4c740 10 client.25592 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.057+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2514459047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560a10bd3a90 con 0x560a107ac1b0 2024-07-31T14:52:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.058+0000 7f5ced7fa640 1 -- 172.21.15.67:0/2514459047 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0000b660 con 0x560a107ac1b0 2024-07-31T14:52:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.058+0000 7f5ced7fa640 1 -- 172.21.15.67:0/2514459047 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5d000348f0 con 0x560a107ac1b0 2024-07-31T14:52:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.058+0000 7f5d12e4c740 10 client.25592 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.058+0000 7f5d12e4c740 10 client.25592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.058+0000 7f5d12e4c740 20 client.25592 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.058+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/2514459047 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a10bd3fb0 0x560a10bd63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.058+0000 7f5d12e4c740 4 client.25592 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:52:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.058+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2514459047 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a10ac1da0 con 0x560a10bd3fb0 2024-07-31T14:52:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.059+0000 7f5d09cf5640 1 --2- 172.21.15.67:0/2514459047 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a10bd3fb0 0x560a10bd63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.059+0000 7f5ced7fa640 10 client.25592 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.059+0000 7f5d09cf5640 1 --2- 172.21.15.67:0/2514459047 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a10bd3fb0 0x560a10bd63d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=242661b9971ad480 server_cookie=515fd79a9109bc3a in_seq=0 out_seq=0 2024-07-31T14:52:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.068+0000 7f5ced7fa640 1 -- 172.21.15.67:0/2514459047 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a10ac1da0 con 0x560a10bd3fb0 2024-07-31T14:52:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.068+0000 7f5ced7fa640 10 client.25592 handle_command_reply: tid=0 2024-07-31T14:52:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.068+0000 7f5d12e4c740 1 client.25592 shutdown 2024-07-31T14:52:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.069+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2514459047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 msgr2=0x560a10b5f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.069+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/2514459047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b5f990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5d0002c5d0 tx=0x7f5d00005f90 comp rx=0 tx=0).stop 2024-07-31T14:52:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.069+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2514459047 shutdown_connections 2024-07-31T14:52:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.069+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/2514459047 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a10bd3fb0 0x560a10bd63d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.069+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/2514459047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a107ac1b0 0x560a10b5f990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.069+0000 7f5ced7fa640 0 client.25592 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.069+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/2514459047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1078b4d0 0x560a10b5f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.069+0000 7f5d12e4c740 1 --2- 172.21.15.67:0/2514459047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1076ad80 0x560a10b62eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.069+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2514459047 >> 172.21.15.67:0/2514459047 conn(0x560a107a0300 msgr2=0x560a10b5b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.069+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2514459047 shutdown_connections 2024-07-31T14:52:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.070+0000 7f5d12e4c740 1 -- 172.21.15.67:0/2514459047 wait complete. 2024-07-31T14:52:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.070+0000 7f5d12e4c740 20 client.25592 trim_cache size 0 max 16384 2024-07-31T14:52:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:52:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:52:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:52:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:52:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.070+0000 7f5d0bf59640 1 -- 172.21.15.67:0/843374333 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5cf00b3f60 msgr2=0x7f5cf00b6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.070+0000 7f5d0bf59640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5cf00b3f60 0x7f5cf00b6470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5d041042e0 tx=0x7f5cf400b390 comp rx=0 tx=0).stop 2024-07-31T14:52:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.070+0000 7f5d0bf59640 1 -- 172.21.15.67:0/843374333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d041023f0 msgr2=0x7f5d0410e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.070+0000 7f5d0bf59640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d041023f0 0x7f5d0410e3d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00efb0 tx=0x7f5cfc00c800 comp rx=0 tx=0).stop 2024-07-31T14:52:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.070+0000 7f5d0bf59640 1 -- 172.21.15.67:0/843374333 shutdown_connections 2024-07-31T14:52:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.070+0000 7f5d0bf59640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5cf00b3f60 0x7f5cf00b6470 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.070+0000 7f5d0bf59640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d04108670 0x7f5d041a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.071+0000 7f5d0bf59640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d04107d20 0x7f5d0410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.071+0000 7f5d0bf59640 1 --2- 172.21.15.67:0/843374333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d041023f0 0x7f5d0410e3d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.071+0000 7f5d0bf59640 1 -- 172.21.15.67:0/843374333 >> 172.21.15.67:0/843374333 conn(0x7f5d040fddf0 msgr2=0x7f5d041062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.071+0000 7f5d0bf59640 1 -- 172.21.15.67:0/843374333 shutdown_connections 2024-07-31T14:52:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:19.071+0000 7f5d0bf59640 1 -- 172.21.15.67:0/843374333 wait complete. 2024-07-31T14:52:19.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:18 smithi073 ceph-mon[36714]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-07-31T14:52:19.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.138+0000 7f9eea63d640 1 -- 172.21.15.67:0/4231581361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4102490 msgr2=0x7f9ee41028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.138+0000 7f9eea63d640 1 --2- 172.21.15.67:0/4231581361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4102490 0x7f9ee41028a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009900 tx=0x7f9ed402a410 comp rx=0 tx=0).stop 2024-07-31T14:52:19.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.138+0000 7f9eea63d640 1 -- 172.21.15.67:0/4231581361 shutdown_connections 2024-07-31T14:52:19.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.138+0000 7f9eea63d640 1 --2- 172.21.15.67:0/4231581361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ee4108720 0x7f9ee410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.138+0000 7f9eea63d640 1 --2- 172.21.15.67:0/4231581361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4107dd0 0x7f9ee41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.138+0000 7f9eea63d640 1 --2- 172.21.15.67:0/4231581361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4102490 0x7f9ee41028a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.138+0000 7f9eea63d640 1 -- 172.21.15.67:0/4231581361 >> 172.21.15.67:0/4231581361 conn(0x7f9ee4079160 msgr2=0x7f9ee4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:19.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.138+0000 7f9eea63d640 1 -- 172.21.15.67:0/4231581361 shutdown_connections 2024-07-31T14:52:19.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.138+0000 7f9eea63d640 1 -- 172.21.15.67:0/4231581361 wait complete. 2024-07-31T14:52:19.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.139+0000 7f9eea63d640 1 Processor -- start 2024-07-31T14:52:19.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.139+0000 7f9eea63d640 1 -- start start 2024-07-31T14:52:19.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.140+0000 7f9eea63d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4102490 0x7f9ee4199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.140+0000 7f9eea63d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ee4107dd0 0x7f9ee419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.140+0000 7f9eea63d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4108720 0x7f9ee419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.140+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4102490 0x7f9ee4199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.141+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4102490 0x7f9ee4199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37908/0 (socket says 172.21.15.67:37908) 2024-07-31T14:52:19.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.141+0000 7f9ee3fff640 1 -- 172.21.15.67:0/775764878 learned_addr learned my addr 172.21.15.67:0/775764878 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:19.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.141+0000 7f9eea63d640 1 -- 172.21.15.67:0/775764878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41123f0 con 0x7f9ee4102490 2024-07-31T14:52:19.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.141+0000 7f9eea63d640 1 -- 172.21.15.67:0/775764878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4112270 con 0x7f9ee4108720 2024-07-31T14:52:19.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.141+0000 7f9eea63d640 1 -- 172.21.15.67:0/775764878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4112570 con 0x7f9ee4107dd0 2024-07-31T14:52:19.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.141+0000 7f9ee8bda640 1 --2- 172.21.15.67:0/775764878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4108720 0x7f9ee419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.141+0000 7f9ee37fe640 1 --2- 172.21.15.67:0/775764878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ee4107dd0 0x7f9ee419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.141+0000 7f9ee37fe640 1 -- 172.21.15.67:0/775764878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4108720 msgr2=0x7f9ee419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.141+0000 7f9ee37fe640 1 --2- 172.21.15.67:0/775764878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4108720 0x7f9ee419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.141+0000 7f9ee37fe640 1 -- 172.21.15.67:0/775764878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4102490 msgr2=0x7f9ee4199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.141+0000 7f9ee37fe640 1 --2- 172.21.15.67:0/775764878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4102490 0x7f9ee4199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.141+0000 7f9ee37fe640 1 -- 172.21.15.67:0/775764878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee419fe40 con 0x7f9ee4107dd0 2024-07-31T14:52:19.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.142+0000 7f9ee37fe640 1 --2- 172.21.15.67:0/775764878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ee4107dd0 0x7f9ee419a470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009e60 tx=0x7f9ed800ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:19.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.142+0000 7f9ee17fa640 1 -- 172.21.15.67:0/775764878 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8020070 con 0x7f9ee4107dd0 2024-07-31T14:52:19.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.142+0000 7f9eea63d640 1 -- 172.21.15.67:0/775764878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee41a0130 con 0x7f9ee4107dd0 2024-07-31T14:52:19.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.142+0000 7f9ee17fa640 1 -- 172.21.15.67:0/775764878 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed800ccc0 con 0x7f9ee4107dd0 2024-07-31T14:52:19.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.144+0000 7f9ee17fa640 1 -- 172.21.15.67:0/775764878 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed801ba80 con 0x7f9ee4107dd0 2024-07-31T14:52:19.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.144+0000 7f9eea63d640 1 -- 172.21.15.67:0/775764878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee4108f20 con 0x7f9ee4107dd0 2024-07-31T14:52:19.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.146+0000 7f9ef0b8e740 1 Processor -- start 2024-07-31T14:52:19.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.147+0000 7f9ef0b8e740 1 -- start start 2024-07-31T14:52:19.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.147+0000 7f9ee17fa640 1 -- 172.21.15.67:0/775764878 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f9ed8023070 con 0x7f9ee4107dd0 2024-07-31T14:52:19.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.147+0000 7f9ef0b8e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0a0070 0x56395e445690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.147+0000 7f9ef0b8e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0771b0 0x56395e445bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.147+0000 7f9ef0b8e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e446110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.147+0000 7f9ef0b8e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56395e3bbbb0 con 0x56395e0771b0 2024-07-31T14:52:19.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.147+0000 7f9ef0b8e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56395e3ab0b0 con 0x56395e07f390 2024-07-31T14:52:19.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.147+0000 7f9ef0b8e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56395e3af0f0 con 0x56395e0a0070 2024-07-31T14:52:19.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.147+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0771b0 0x56395e445bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.147+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0771b0 0x56395e445bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37916/0 (socket says 172.21.15.67:37916) 2024-07-31T14:52:19.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.147+0000 7f9ee3fff640 1 -- 172.21.15.67:0/2615316678 learned_addr learned my addr 172.21.15.67:0/2615316678 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:19.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.147+0000 7f9ee8bda640 1 --2- 172.21.15.67:0/2615316678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0a0070 0x56395e445690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.148+0000 7f9ee8bda640 1 -- 172.21.15.67:0/2615316678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 msgr2=0x56395e446110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.148+0000 7f9ee37fe640 1 --2- 172.21.15.67:0/2615316678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e446110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.148+0000 7f9ee8bda640 1 --2- 172.21.15.67:0/2615316678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e446110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.148+0000 7f9ee8bda640 1 -- 172.21.15.67:0/2615316678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0771b0 msgr2=0x56395e445bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.148+0000 7f9ee8bda640 1 --2- 172.21.15.67:0/2615316678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0771b0 0x56395e445bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.148+0000 7f9ee8bda640 1 -- 172.21.15.67:0/2615316678 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56395e351b90 con 0x56395e0a0070 2024-07-31T14:52:19.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.148+0000 7f9ee8bda640 1 --2- 172.21.15.67:0/2615316678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0a0070 0x56395e445690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009f00 tx=0x7f9edc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3916201ad2fcde7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:19.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.148+0000 7f9ee17fa640 1 --2- 172.21.15.67:0/775764878 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9ecc0b3e50 0x7f9ecc0b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.149+0000 7f9eca7fc640 1 -- 172.21.15.67:0/2615316678 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc004d20 con 0x56395e0a0070 2024-07-31T14:52:19.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.149+0000 7f9ee17fa640 1 -- 172.21.15.67:0/775764878 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f9ed80e85f0 con 0x7f9ee4107dd0 2024-07-31T14:52:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.149+0000 7f9eca7fc640 1 -- 172.21.15.67:0/2615316678 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9edc004030 con 0x56395e0a0070 2024-07-31T14:52:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.149+0000 7f9ee3fff640 1 --2- 172.21.15.67:0/775764878 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9ecc0b3e50 0x7f9ecc0b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.149+0000 7f9eca7fc640 1 -- 172.21.15.67:0/2615316678 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc005260 con 0x56395e0a0070 2024-07-31T14:52:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.150+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/2615316678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0a0070 msgr2=0x56395e445690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.150+0000 7f9ee3fff640 1 --2- 172.21.15.67:0/775764878 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9ecc0b3e50 0x7f9ecc0b6360 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40096c0 tx=0x7f9ed4005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.150+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/2615316678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0a0070 0x56395e445690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009f00 tx=0x7f9edc011810 comp rx=0 tx=0).stop 2024-07-31T14:52:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.150+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/2615316678 shutdown_connections 2024-07-31T14:52:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.150+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/2615316678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e446110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.150+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/2615316678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0771b0 0x56395e445bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.150+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/2615316678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0a0070 0x56395e445690 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.150+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/2615316678 >> 172.21.15.67:0/2615316678 conn(0x56395e094460 msgr2=0x56395e443580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.150+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/2615316678 shutdown_connections 2024-07-31T14:52:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.151+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/2615316678 wait complete. 2024-07-31T14:52:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.151+0000 7f9ef0b8e740 1 -- start start 2024-07-31T14:52:19.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.151+0000 7f9ef0b8e740 1 Processor -- start 2024-07-31T14:52:19.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.152+0000 7f9ef0b8e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0771b0 0x56395e451180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.152+0000 7f9ef0b8e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e4516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.152+0000 7f9ef0b8e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0a0070 0x56395e45d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.153+0000 7f9ef0b8e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56395e20e700 con 0x56395e0a0070 2024-07-31T14:52:19.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.153+0000 7f9ef0b8e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56395e34d9e0 con 0x56395e07f390 2024-07-31T14:52:19.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.153+0000 7f9ef0b8e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56395e453f10 con 0x56395e0771b0 2024-07-31T14:52:19.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.153+0000 7f9ee37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e4516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.153+0000 7f9ee37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e4516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35776/0 (socket says 172.21.15.67:35776) 2024-07-31T14:52:19.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.153+0000 7f9ee37fe640 1 -- 172.21.15.67:0/3840031774 learned_addr learned my addr 172.21.15.67:0/3840031774 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:19.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.153+0000 7f9ef0b8e740 10 client.0 _ll_register_callbacks cb 0x56395e3684f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:19.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.153+0000 7f9ee3fff640 1 --2- 172.21.15.67:0/3840031774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0771b0 0x56395e451180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.153+0000 7f9ee8bda640 1 --2- 172.21.15.67:0/3840031774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0a0070 0x56395e45d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.153+0000 7f9ee37fe640 1 -- 172.21.15.67:0/3840031774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0771b0 msgr2=0x56395e451180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.153+0000 7f9ee37fe640 1 --2- 172.21.15.67:0/3840031774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0771b0 0x56395e451180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.153+0000 7f9ee37fe640 1 -- 172.21.15.67:0/3840031774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0a0070 msgr2=0x56395e45d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.153+0000 7f9ee37fe640 1 --2- 172.21.15.67:0/3840031774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0a0070 0x56395e45d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.153+0000 7f9ee37fe640 1 -- 172.21.15.67:0/3840031774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56395e10b650 con 0x56395e07f390 2024-07-31T14:52:19.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.154+0000 7f9ee8bda640 1 --2- 172.21.15.67:0/3840031774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0a0070 0x56395e45d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:19.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.154+0000 7f9ee37fe640 1 --2- 172.21.15.67:0/3840031774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e4516c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80aef20 tx=0x7f9ed80ebef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3766322dcf38c76c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:19.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.154+0000 7f9ec37fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:52:19.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.154+0000 7f9ec37fe640 1 -- 172.21.15.67:0/3840031774 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80aeaa0 con 0x56395e07f390 2024-07-31T14:52:19.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.154+0000 7f9ec37fe640 1 -- 172.21.15.67:0/3840031774 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f9ed80c95c0 con 0x56395e07f390 2024-07-31T14:52:19.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.154+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/3840031774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9edc004030 con 0x56395e07f390 2024-07-31T14:52:19.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.155+0000 7f9ec37fe640 1 -- 172.21.15.67:0/3840031774 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9ed80aef90 con 0x56395e07f390 2024-07-31T14:52:19.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.155+0000 7f9ef0b8e740 10 client.25601 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:19.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.155+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/3840031774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56395e45fe50 con 0x56395e07f390 2024-07-31T14:52:19.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.155+0000 7f9ec37fe640 1 -- 172.21.15.67:0/3840031774 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed80e8070 con 0x56395e07f390 2024-07-31T14:52:19.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.155+0000 7f9ec37fe640 1 -- 172.21.15.67:0/3840031774 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9ed80c9240 con 0x56395e07f390 2024-07-31T14:52:19.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.155+0000 7f9ef0b8e740 10 client.25601 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:19.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.155+0000 7f9ef0b8e740 10 client.25601 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:52:19.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.155+0000 7f9ef0b8e740 20 client.25601 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:19.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.155+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/3840031774 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x56395e05e9f0 0x56395e4637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.155+0000 7f9ef0b8e740 4 client.25601 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:19.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.155+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/3840031774 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9edc004030 con 0x56395e05e9f0 2024-07-31T14:52:19.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.156+0000 7f9ee3fff640 1 --2- 172.21.15.67:0/3840031774 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x56395e05e9f0 0x56395e4637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.157+0000 7f9ec37fe640 10 client.25601 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:52:19.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.157+0000 7f9ee3fff640 1 --2- 172.21.15.67:0/3840031774 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x56395e05e9f0 0x56395e4637c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfa9cb7eca39a180 server_cookie=cb4bbf3886c290da in_seq=0 out_seq=0 2024-07-31T14:52:19.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.158+0000 7f9ec37fe640 1 -- 172.21.15.67:0/3840031774 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9edc004030 con 0x56395e05e9f0 2024-07-31T14:52:19.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.158+0000 7f9ec37fe640 10 client.25601 handle_command_reply: tid=0 2024-07-31T14:52:19.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.158+0000 7f9ef0b8e740 1 client.25601 shutdown 2024-07-31T14:52:19.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.158+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/3840031774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 msgr2=0x56395e4516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.158+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/3840031774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e4516c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80aef20 tx=0x7f9ed80ebef0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.158+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/3840031774 shutdown_connections 2024-07-31T14:52:19.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.158+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/3840031774 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x56395e05e9f0 0x56395e4637c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.158+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/3840031774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0a0070 0x56395e45d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.158+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/3840031774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e4516c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.158+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/3840031774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0771b0 0x56395e451180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.158+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/3840031774 >> 172.21.15.67:0/3840031774 conn(0x56395e094460 msgr2=0x56395e448820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:19.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.158+0000 7f9ec37fe640 0 client.25601 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:52:19.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.158+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/3840031774 shutdown_connections 2024-07-31T14:52:19.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.159+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/3840031774 wait complete. 2024-07-31T14:52:19.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.159+0000 7f9ef0b8e740 20 client.25601 trim_cache size 0 max 16384 2024-07-31T14:52:19.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.186+0000 7f9ef0b8e740 1 Processor -- start 2024-07-31T14:52:19.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.186+0000 7f9ef0b8e740 1 -- start start 2024-07-31T14:52:19.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.186+0000 7f9ef0b8e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0771b0 0x56395e44fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.186+0000 7f9ef0b8e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e448ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.186+0000 7f9ef0b8e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0a0070 0x56395e45cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.186+0000 7f9ef0b8e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ae3d0 con 0x56395e0771b0 2024-07-31T14:52:19.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.186+0000 7f9ef0b8e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41aff10 con 0x56395e07f390 2024-07-31T14:52:19.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.186+0000 7f9ef0b8e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41aeaa0 con 0x56395e0a0070 2024-07-31T14:52:19.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.187+0000 7f9ee37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e448ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.187+0000 7f9ee37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e448ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35788/0 (socket says 172.21.15.67:35788) 2024-07-31T14:52:19.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.187+0000 7f9ee37fe640 1 -- 172.21.15.67:0/1118358970 learned_addr learned my addr 172.21.15.67:0/1118358970 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:19.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.187+0000 7f9ee8bda640 1 --2- 172.21.15.67:0/1118358970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0a0070 0x56395e45cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.187+0000 7f9ee3fff640 1 --2- 172.21.15.67:0/1118358970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0771b0 0x56395e44fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.187+0000 7f9ee37fe640 1 -- 172.21.15.67:0/1118358970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0a0070 msgr2=0x56395e45cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.187+0000 7f9ee37fe640 1 --2- 172.21.15.67:0/1118358970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0a0070 0x56395e45cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.187+0000 7f9ee37fe640 1 -- 172.21.15.67:0/1118358970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0771b0 msgr2=0x56395e44fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.187+0000 7f9ee37fe640 1 --2- 172.21.15.67:0/1118358970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0771b0 0x56395e44fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.187+0000 7f9ee37fe640 1 -- 172.21.15.67:0/1118358970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56395e43efa0 con 0x56395e07f390 2024-07-31T14:52:19.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.187+0000 7f9ee8bda640 1 --2- 172.21.15.67:0/1118358970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0a0070 0x56395e45cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:19.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.187+0000 7f9ee3fff640 1 --2- 172.21.15.67:0/1118358970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0771b0 0x56395e44fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:19.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.187+0000 7f9ee37fe640 1 --2- 172.21.15.67:0/1118358970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e448ea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8000c00 tx=0x7f9ed80b33e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22d03c819f2868f1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:19.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.188+0000 7f9ec9ffb640 1 -- 172.21.15.67:0/1118358970 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80e5e60 con 0x56395e07f390 2024-07-31T14:52:19.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.188+0000 7f9ec9ffb640 1 -- 172.21.15.67:0/1118358970 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed80e8040 con 0x56395e07f390 2024-07-31T14:52:19.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.188+0000 7f9ec9ffb640 1 -- 172.21.15.67:0/1118358970 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80ae8e0 con 0x56395e07f390 2024-07-31T14:52:19.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.189+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/1118358970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 msgr2=0x56395e448ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.189+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/1118358970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e448ea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8000c00 tx=0x7f9ed80b33e0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.189+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/1118358970 shutdown_connections 2024-07-31T14:52:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.189+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/1118358970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0a0070 0x56395e45cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.189+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/1118358970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e07f390 0x56395e448ea0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.189+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/1118358970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e0771b0 0x56395e44fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.189+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/1118358970 >> 172.21.15.67:0/1118358970 conn(0x56395e094460 msgr2=0x56395e448790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.189+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/1118358970 shutdown_connections 2024-07-31T14:52:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.189+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/1118358970 wait complete. 2024-07-31T14:52:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.190+0000 7f9ef0b8e740 1 -- start start 2024-07-31T14:52:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.190+0000 7f9ef0b8e740 1 Processor -- start 2024-07-31T14:52:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.190+0000 7f9ef0b8e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0771b0 0x56395e4548f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.190+0000 7f9ef0b8e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e07f390 0x56395e454eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.190+0000 7f9ef0b8e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e0a0070 0x56395e4c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ef0b8e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41af840 con 0x56395e07f390 2024-07-31T14:52:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ef0b8e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41af170 con 0x56395e0a0070 2024-07-31T14:52:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ef0b8e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ac3b0 con 0x56395e0771b0 2024-07-31T14:52:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0771b0 0x56395e4548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0771b0 0x56395e4548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:36078/0 (socket says 172.21.15.67:36078) 2024-07-31T14:52:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ee3fff640 1 -- 172.21.15.67:0/1190450391 learned_addr learned my addr 172.21.15.67:0/1190450391 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ee37fe640 1 --2- 172.21.15.67:0/1190450391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e07f390 0x56395e454eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ee8bda640 1 --2- 172.21.15.67:0/1190450391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e0a0070 0x56395e4c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ef0b8e740 10 client.0 _ll_register_callbacks cb 0x7f9edc0098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ee3fff640 1 -- 172.21.15.67:0/1190450391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e0a0070 msgr2=0x56395e4c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ee3fff640 1 --2- 172.21.15.67:0/1190450391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e0a0070 0x56395e4c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ee3fff640 1 -- 172.21.15.67:0/1190450391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e07f390 msgr2=0x56395e454eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ee3fff640 1 --2- 172.21.15.67:0/1190450391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e07f390 0x56395e454eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ee3fff640 1 -- 172.21.15.67:0/1190450391 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56395e4ca5a0 con 0x56395e0771b0 2024-07-31T14:52:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ee3fff640 1 --2- 172.21.15.67:0/1190450391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0771b0 0x56395e4548f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x56395e444310 tx=0x7f9ed4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96fdf466997b7f8d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.191+0000 7f9ec37fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:52:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.192+0000 7f9ec37fe640 1 -- 172.21.15.67:0/1190450391 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4035040 con 0x56395e0771b0 2024-07-31T14:52:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.192+0000 7f9ec37fe640 1 -- 172.21.15.67:0/1190450391 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f9ed4051350 con 0x56395e0771b0 2024-07-31T14:52:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.192+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/1190450391 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed80e8040 con 0x56395e0771b0 2024-07-31T14:52:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.192+0000 7f9ec37fe640 1 -- 172.21.15.67:0/1190450391 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9ed4003e60 con 0x56395e0771b0 2024-07-31T14:52:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.192+0000 7f9ef0b8e740 10 client.25633 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.192+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/1190450391 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56395e4cb770 con 0x56395e0771b0 2024-07-31T14:52:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.192+0000 7f9ec37fe640 1 -- 172.21.15.67:0/1190450391 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed80e8040 con 0x56395e0771b0 2024-07-31T14:52:19.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.193+0000 7f9ec37fe640 1 -- 172.21.15.67:0/1190450391 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9ed4051030 con 0x56395e0771b0 2024-07-31T14:52:19.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.193+0000 7f9ef0b8e740 10 client.25633 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.193+0000 7f9ef0b8e740 10 client.25633 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:52:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.193+0000 7f9ef0b8e740 20 client.25633 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.193+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/1190450391 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x56395e033da0 0x56395e4cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.193+0000 7f9ef0b8e740 4 client.25633 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:52:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.193+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/1190450391 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56395e3bdb00 con 0x56395e033da0 2024-07-31T14:52:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.193+0000 7f9ee37fe640 1 --2- 172.21.15.67:0/1190450391 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x56395e033da0 0x56395e4cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.194+0000 7f9ec37fe640 10 client.25633 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:52:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.194+0000 7f9ee37fe640 1 --2- 172.21.15.67:0/1190450391 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x56395e033da0 0x56395e4cdf60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac1b53e1bbd56169 server_cookie=9645755dfb256cdb in_seq=0 out_seq=0 2024-07-31T14:52:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.194+0000 7f9ec37fe640 1 -- 172.21.15.67:0/1190450391 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56395e3bdb00 con 0x56395e033da0 2024-07-31T14:52:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.194+0000 7f9ec37fe640 10 client.25633 handle_command_reply: tid=0 2024-07-31T14:52:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.194+0000 7f9ef0b8e740 1 client.25633 shutdown 2024-07-31T14:52:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.194+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/1190450391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0771b0 msgr2=0x56395e4548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.194+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/1190450391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0771b0 0x56395e4548f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x56395e444310 tx=0x7f9ed4009260 comp rx=0 tx=0).stop 2024-07-31T14:52:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.194+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/1190450391 shutdown_connections 2024-07-31T14:52:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.194+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/1190450391 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x56395e033da0 0x56395e4cdf60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.194+0000 7f9ec37fe640 0 client.25633 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:52:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.194+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/1190450391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56395e0a0070 0x56395e4c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.194+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/1190450391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56395e07f390 0x56395e454eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.194+0000 7f9ef0b8e740 1 --2- 172.21.15.67:0/1190450391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56395e0771b0 0x56395e4548f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.194+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/1190450391 >> 172.21.15.67:0/1190450391 conn(0x56395e094460 msgr2=0x56395e446670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:19.200 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:52:19.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.195+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/1190450391 shutdown_connections 2024-07-31T14:52:19.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.195+0000 7f9ef0b8e740 1 -- 172.21.15.67:0/1190450391 wait complete. 2024-07-31T14:52:19.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.195+0000 7f9ef0b8e740 20 client.25633 trim_cache size 0 max 16384 2024-07-31T14:52:19.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.196+0000 7f9eea63d640 1 -- 172.21.15.67:0/775764878 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9ecc0b3e50 msgr2=0x7f9ecc0b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.196+0000 7f9eea63d640 1 --2- 172.21.15.67:0/775764878 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9ecc0b3e50 0x7f9ecc0b6360 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40096c0 tx=0x7f9ed4005a20 comp rx=0 tx=0).stop 2024-07-31T14:52:19.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.196+0000 7f9eea63d640 1 -- 172.21.15.67:0/775764878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ee4107dd0 msgr2=0x7f9ee419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:19.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.196+0000 7f9eea63d640 1 --2- 172.21.15.67:0/775764878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ee4107dd0 0x7f9ee419a470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009e60 tx=0x7f9ed800ed90 comp rx=0 tx=0).stop 2024-07-31T14:52:19.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.196+0000 7f9eea63d640 1 -- 172.21.15.67:0/775764878 shutdown_connections 2024-07-31T14:52:19.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.196+0000 7f9eea63d640 1 --2- 172.21.15.67:0/775764878 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9ecc0b3e50 0x7f9ecc0b6360 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.196+0000 7f9eea63d640 1 --2- 172.21.15.67:0/775764878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4108720 0x7f9ee419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.196+0000 7f9eea63d640 1 --2- 172.21.15.67:0/775764878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ee4107dd0 0x7f9ee419a470 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.196+0000 7f9eea63d640 1 --2- 172.21.15.67:0/775764878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4102490 0x7f9ee4199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:19.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.196+0000 7f9eea63d640 1 -- 172.21.15.67:0/775764878 >> 172.21.15.67:0/775764878 conn(0x7f9ee4079160 msgr2=0x7f9ee4100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:19.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.196+0000 7f9eea63d640 1 -- 172.21.15.67:0/775764878 shutdown_connections 2024-07-31T14:52:19.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:19.196+0000 7f9eea63d640 1 -- 172.21.15.67:0/775764878 wait complete. 2024-07-31T14:52:20.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T14:52:20.168 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 33.4 2024-07-31T14:52:20.168 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:52:20.427 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:21.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:20 smithi067 ceph-mon[32750]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-31T14:52:21.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:20 smithi067 ceph-mon[32750]: mgrmap e27: x(active, since 12m), standbys: z, y 2024-07-31T14:52:21.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:20 smithi073 ceph-mon[36714]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-31T14:52:21.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:20 smithi073 ceph-mon[36714]: mgrmap e27: x(active, since 12m), standbys: z, y 2024-07-31T14:52:21.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:20 smithi077 ceph-mon[37327]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-31T14:52:21.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:20 smithi077 ceph-mon[37327]: mgrmap e27: x(active, since 12m), standbys: z, y 2024-07-31T14:52:21.755 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:22.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:22 smithi067 ceph-mon[32750]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-07-31T14:52:23.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:22 smithi073 ceph-mon[36714]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-07-31T14:52:23.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:22 smithi077 ceph-mon[37327]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-07-31T14:52:23.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.498+0000 7f62b76e9640 1 -- 172.21.15.67:0/1807318156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b0102490 msgr2=0x7f62b01028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:23.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.498+0000 7f62b76e9640 1 --2- 172.21.15.67:0/1807318156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b0102490 0x7f62b01028a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009900 tx=0x7f62ac02a410 comp rx=0 tx=0).stop 2024-07-31T14:52:23.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.498+0000 7f62b76e9640 1 -- 172.21.15.67:0/1807318156 shutdown_connections 2024-07-31T14:52:23.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.498+0000 7f62b76e9640 1 --2- 172.21.15.67:0/1807318156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b0108720 0x7f62b010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.498+0000 7f62b76e9640 1 --2- 172.21.15.67:0/1807318156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62b0107dd0 0x7f62b01081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.498+0000 7f62b76e9640 1 --2- 172.21.15.67:0/1807318156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b0102490 0x7f62b01028a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.498+0000 7f62b76e9640 1 -- 172.21.15.67:0/1807318156 >> 172.21.15.67:0/1807318156 conn(0x7f62b0079160 msgr2=0x7f62b0079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:23.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.499+0000 7f62b76e9640 1 -- 172.21.15.67:0/1807318156 shutdown_connections 2024-07-31T14:52:23.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.499+0000 7f62b76e9640 1 -- 172.21.15.67:0/1807318156 wait complete. 2024-07-31T14:52:23.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.500+0000 7f62b76e9640 1 Processor -- start 2024-07-31T14:52:23.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.500+0000 7f62b76e9640 1 -- start start 2024-07-31T14:52:23.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.500+0000 7f62b76e9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b0102490 0x7f62b010b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.500+0000 7f62b76e9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62b0107dd0 0x7f62b010bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.500+0000 7f62b76e9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b0108720 0x7f62b01abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.500+0000 7f62b76e9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01112e0 con 0x7f62b0102490 2024-07-31T14:52:23.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.500+0000 7f62b76e9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0111160 con 0x7f62b0107dd0 2024-07-31T14:52:23.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.500+0000 7f62b76e9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0111460 con 0x7f62b0108720 2024-07-31T14:52:23.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.500+0000 7f62b5485640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b0102490 0x7f62b010b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.500+0000 7f62b4c84640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62b0107dd0 0x7f62b010bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.501+0000 7f62b4c84640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62b0107dd0 0x7f62b010bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:54080/0 (socket says 172.21.15.67:54080) 2024-07-31T14:52:23.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.501+0000 7f62b4c84640 1 -- 172.21.15.67:0/561899367 learned_addr learned my addr 172.21.15.67:0/561899367 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.501+0000 7f62b5c86640 1 --2- 172.21.15.67:0/561899367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b0108720 0x7f62b01abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.501+0000 7f62b4c84640 1 -- 172.21.15.67:0/561899367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b0108720 msgr2=0x7f62b01abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.501+0000 7f62b4c84640 1 --2- 172.21.15.67:0/561899367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b0108720 0x7f62b01abfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.501+0000 7f62b4c84640 1 -- 172.21.15.67:0/561899367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b0102490 msgr2=0x7f62b010b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.501+0000 7f62b4c84640 1 --2- 172.21.15.67:0/561899367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b0102490 0x7f62b010b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.501+0000 7f62b4c84640 1 -- 172.21.15.67:0/561899367 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b01ac530 con 0x7f62b0107dd0 2024-07-31T14:52:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.501+0000 7f62b5485640 1 --2- 172.21.15.67:0/561899367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b0102490 0x7f62b010b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.501+0000 7f62b4c84640 1 --2- 172.21.15.67:0/561899367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62b0107dd0 0x7f62b010bf20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f62a000d770 tx=0x7f62a000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.502+0000 7f62a67fc640 1 -- 172.21.15.67:0/561899367 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0017070 con 0x7f62b0107dd0 2024-07-31T14:52:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.502+0000 7f62b76e9640 1 -- 172.21.15.67:0/561899367 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b01ac820 con 0x7f62b0107dd0 2024-07-31T14:52:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.502+0000 7f62a67fc640 1 -- 172.21.15.67:0/561899367 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a0009c30 con 0x7f62b0107dd0 2024-07-31T14:52:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.502+0000 7f62a67fc640 1 -- 172.21.15.67:0/561899367 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a000f620 con 0x7f62b0107dd0 2024-07-31T14:52:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.502+0000 7f62b76e9640 1 -- 172.21.15.67:0/561899367 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b01acd80 con 0x7f62b0107dd0 2024-07-31T14:52:23.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.506+0000 7f62a67fc640 1 -- 172.21.15.67:0/561899367 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f62a001a070 con 0x7f62b0107dd0 2024-07-31T14:52:23.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.506+0000 7f62bdc3a740 1 Processor -- start 2024-07-31T14:52:23.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.507+0000 7f62bdc3a740 1 -- start start 2024-07-31T14:52:23.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.507+0000 7f62bdc3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb3c070 0x564efdee1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.507+0000 7f62bdc3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdb131b0 0x564efdee1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.507+0000 7f62bdc3a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb1b390 0x564efdee1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.507+0000 7f62a67fc640 1 --2- 172.21.15.67:0/561899367 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f629c0b3d40 0x7f629c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.508+0000 7f62b5485640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdb131b0 0x564efdee1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.508+0000 7f62a67fc640 1 -- 172.21.15.67:0/561899367 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f62a00e3e30 con 0x7f62b0107dd0 2024-07-31T14:52:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.508+0000 7f62b5485640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdb131b0 0x564efdee1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:54084/0 (socket says 172.21.15.67:54084) 2024-07-31T14:52:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.508+0000 7f62b5c86640 1 --2- 172.21.15.67:0/561899367 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f629c0b3d40 0x7f629c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.508+0000 7f62bdc3a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564efde57bb0 con 0x564efdb1b390 2024-07-31T14:52:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.508+0000 7f62bdc3a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564efde470b0 con 0x564efdb131b0 2024-07-31T14:52:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.508+0000 7f62bdc3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564efde4b0f0 con 0x564efdb3c070 2024-07-31T14:52:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.508+0000 7f62b4c84640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb1b390 0x564efdee1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.508+0000 7f62b5485640 1 -- 172.21.15.67:0/725483633 learned_addr learned my addr 172.21.15.67:0/725483633 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.508+0000 7f62b5c86640 1 --2- 172.21.15.67:0/725483633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb3c070 0x564efdee1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.508+0000 7f62b5c86640 1 --2- 172.21.15.67:0/561899367 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f629c0b3d40 0x7f629c0b6250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009270 tx=0x7f62a8009200 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:23.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.509+0000 7f62b5485640 1 -- 172.21.15.67:0/725483633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb3c070 msgr2=0x564efdee1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:23.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.509+0000 7f62b5485640 1 --2- 172.21.15.67:0/725483633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb3c070 0x564efdee1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.509+0000 7f62b5485640 1 -- 172.21.15.67:0/725483633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb1b390 msgr2=0x564efdee1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.509+0000 7f62b5485640 1 --2- 172.21.15.67:0/725483633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb1b390 0x564efdee1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.509+0000 7f62b5485640 1 -- 172.21.15.67:0/725483633 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564efddedb90 con 0x564efdb131b0 2024-07-31T14:52:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.509+0000 7f62b4c84640 1 --2- 172.21.15.67:0/725483633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb1b390 0x564efdee1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.509+0000 7f62b5485640 1 --2- 172.21.15.67:0/725483633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdb131b0 0x564efdee1ab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009230 tx=0x7f62ac004520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a62d560e3c26c1b1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.510+0000 7f629b7fe640 1 -- 172.21.15.67:0/725483633 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac035040 con 0x564efdb131b0 2024-07-31T14:52:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.510+0000 7f629b7fe640 1 -- 172.21.15.67:0/725483633 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62ac004c20 con 0x564efdb131b0 2024-07-31T14:52:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.510+0000 7f629b7fe640 1 -- 172.21.15.67:0/725483633 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac003970 con 0x564efdb131b0 2024-07-31T14:52:23.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.511+0000 7f62bdc3a740 1 -- 172.21.15.67:0/725483633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdb131b0 msgr2=0x564efdee1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:23.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.511+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/725483633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdb131b0 0x564efdee1ab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009230 tx=0x7f62ac004520 comp rx=0 tx=0).stop 2024-07-31T14:52:23.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.511+0000 7f62bdc3a740 1 -- 172.21.15.67:0/725483633 shutdown_connections 2024-07-31T14:52:23.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.511+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/725483633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb1b390 0x564efdee1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.511+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/725483633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdb131b0 0x564efdee1ab0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.511+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/725483633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb3c070 0x564efdee1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.511+0000 7f62bdc3a740 1 -- 172.21.15.67:0/725483633 >> 172.21.15.67:0/725483633 conn(0x564efdb30460 msgr2=0x564efdedf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.511+0000 7f62bdc3a740 1 -- 172.21.15.67:0/725483633 shutdown_connections 2024-07-31T14:52:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.512+0000 7f62bdc3a740 1 -- 172.21.15.67:0/725483633 wait complete. 2024-07-31T14:52:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.512+0000 7f62bdc3a740 1 -- start start 2024-07-31T14:52:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.512+0000 7f62bdc3a740 1 Processor -- start 2024-07-31T14:52:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.513+0000 7f62bdc3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdb131b0 0x564efdeed190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.513+0000 7f62bdc3a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb1b390 0x564efdeed730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.513+0000 7f62bdc3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb3c070 0x564efdef8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.513+0000 7f62bdc3a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564efdcaa700 con 0x564efdb1b390 2024-07-31T14:52:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.513+0000 7f62bdc3a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564efdde99e0 con 0x564efdb131b0 2024-07-31T14:52:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.513+0000 7f62bdc3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564efdeefcf0 con 0x564efdb3c070 2024-07-31T14:52:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.513+0000 7f62bdc3a740 10 client.0 _ll_register_callbacks cb 0x564efde044f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.513+0000 7f62b5c86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb3c070 0x564efdef8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.513+0000 7f62b5c86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb3c070 0x564efdef8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50342/0 (socket says 172.21.15.67:50342) 2024-07-31T14:52:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.513+0000 7f62b5c86640 1 -- 172.21.15.67:0/1035434227 learned_addr learned my addr 172.21.15.67:0/1035434227 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.513+0000 7f62b5485640 1 --2- 172.21.15.67:0/1035434227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdb131b0 0x564efdeed190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.514+0000 7f62b5c86640 1 -- 172.21.15.67:0/1035434227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdb131b0 msgr2=0x564efdeed190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.514+0000 7f62b5c86640 1 --2- 172.21.15.67:0/1035434227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdb131b0 0x564efdeed190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.514+0000 7f62b5c86640 1 -- 172.21.15.67:0/1035434227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb1b390 msgr2=0x564efdeed730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.514+0000 7f62b4c84640 1 --2- 172.21.15.67:0/1035434227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb1b390 0x564efdeed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.514+0000 7f62b5c86640 1 --2- 172.21.15.67:0/1035434227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb1b390 0x564efdeed730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.514+0000 7f62b5c86640 1 -- 172.21.15.67:0/1035434227 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564efdba7650 con 0x564efdb3c070 2024-07-31T14:52:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.514+0000 7f62b5c86640 1 --2- 172.21.15.67:0/1035434227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb3c070 0x564efdef8fe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f62a80042d0 tx=0x7f62a8009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c60b127a69ab628b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.514+0000 7f6298ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:52:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.514+0000 7f62b4c84640 1 --2- 172.21.15.67:0/1035434227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb1b390 0x564efdeed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.514+0000 7f6298ff9640 1 -- 172.21.15.67:0/1035434227 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a802a060 con 0x564efdb3c070 2024-07-31T14:52:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.514+0000 7f62bdc3a740 1 -- 172.21.15.67:0/1035434227 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62ac004c20 con 0x564efdb3c070 2024-07-31T14:52:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.514+0000 7f6298ff9640 1 -- 172.21.15.67:0/1035434227 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f62a8010ba0 con 0x564efdb3c070 2024-07-31T14:52:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.515+0000 7f6298ff9640 1 -- 172.21.15.67:0/1035434227 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f62a8031050 con 0x564efdb3c070 2024-07-31T14:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.515+0000 7f62bdc3a740 10 client.25645 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.515+0000 7f62bdc3a740 1 -- 172.21.15.67:0/1035434227 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564efdefbe30 con 0x564efdb3c070 2024-07-31T14:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.515+0000 7f6298ff9640 1 -- 172.21.15.67:0/1035434227 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62ac004c20 con 0x564efdb3c070 2024-07-31T14:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.516+0000 7f6298ff9640 1 -- 172.21.15.67:0/1035434227 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f62a8024070 con 0x564efdb3c070 2024-07-31T14:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.516+0000 7f62bdc3a740 10 client.25645 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.516+0000 7f62bdc3a740 10 client.25645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.516+0000 7f62bdc3a740 20 client.25645 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.516+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/1035434227 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564efdafa9f0 0x564efdeff740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.516+0000 7f62bdc3a740 4 client.25645 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.516+0000 7f62bdc3a740 1 -- 172.21.15.67:0/1035434227 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564efde59b00 con 0x564efdafa9f0 2024-07-31T14:52:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.516+0000 7f62b5485640 1 --2- 172.21.15.67:0/1035434227 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564efdafa9f0 0x564efdeff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.517+0000 7f6298ff9640 10 client.25645 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.517+0000 7f62b5485640 1 --2- 172.21.15.67:0/1035434227 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564efdafa9f0 0x564efdeff740 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f856308be3e58b7 server_cookie=5a64847790d8d45c in_seq=0 out_seq=0 2024-07-31T14:52:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.518+0000 7f6298ff9640 1 -- 172.21.15.67:0/1035434227 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564efde59b00 con 0x564efdafa9f0 2024-07-31T14:52:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.518+0000 7f6298ff9640 10 client.25645 handle_command_reply: tid=0 2024-07-31T14:52:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.518+0000 7f62bdc3a740 1 client.25645 shutdown 2024-07-31T14:52:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.518+0000 7f62bdc3a740 1 -- 172.21.15.67:0/1035434227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb3c070 msgr2=0x564efdef8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.518+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/1035434227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb3c070 0x564efdef8fe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f62a80042d0 tx=0x7f62a8009500 comp rx=0 tx=0).stop 2024-07-31T14:52:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.518+0000 7f62bdc3a740 1 -- 172.21.15.67:0/1035434227 shutdown_connections 2024-07-31T14:52:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.518+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/1035434227 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564efdafa9f0 0x564efdeff740 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.518+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/1035434227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb3c070 0x564efdef8fe0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.518+0000 7f6298ff9640 0 client.25645 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.518+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/1035434227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb1b390 0x564efdeed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.518+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/1035434227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdb131b0 0x564efdeed190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.518+0000 7f62bdc3a740 1 -- 172.21.15.67:0/1035434227 >> 172.21.15.67:0/1035434227 conn(0x564efdb30460 msgr2=0x564efdeddc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.518+0000 7f62bdc3a740 1 -- 172.21.15.67:0/1035434227 shutdown_connections 2024-07-31T14:52:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.519+0000 7f62bdc3a740 1 -- 172.21.15.67:0/1035434227 wait complete. 2024-07-31T14:52:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.519+0000 7f62bdc3a740 20 client.25645 trim_cache size 0 max 16384 2024-07-31T14:52:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.546+0000 7f62bdc3a740 1 Processor -- start 2024-07-31T14:52:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.546+0000 7f62bdc3a740 1 -- start start 2024-07-31T14:52:23.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.547+0000 7f62bdc3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdafa9f0 0x564efdede350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.547+0000 7f62bdc3a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb131b0 0x564efdef8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.547+0000 7f62bdc3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb1b390 0x564efdee67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.547+0000 7f62bdc3a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01b6cc0 con 0x564efdb131b0 2024-07-31T14:52:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.547+0000 7f62bdc3a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01b8800 con 0x564efdafa9f0 2024-07-31T14:52:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.547+0000 7f62bdc3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01b7390 con 0x564efdb1b390 2024-07-31T14:52:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.547+0000 7f62b4c84640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb131b0 0x564efdef8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.547+0000 7f62b5485640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdafa9f0 0x564efdede350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.547+0000 7f62b4c84640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb131b0 0x564efdef8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33730/0 (socket says 172.21.15.67:33730) 2024-07-31T14:52:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.547+0000 7f62b4c84640 1 -- 172.21.15.67:0/4039912786 learned_addr learned my addr 172.21.15.67:0/4039912786 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.547+0000 7f62b5c86640 1 --2- 172.21.15.67:0/4039912786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb1b390 0x564efdee67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.548+0000 7f62b5c86640 1 -- 172.21.15.67:0/4039912786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdafa9f0 msgr2=0x564efdede350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.548+0000 7f62b5c86640 1 --2- 172.21.15.67:0/4039912786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdafa9f0 0x564efdede350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.548+0000 7f62b5c86640 1 -- 172.21.15.67:0/4039912786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb131b0 msgr2=0x564efdef8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.548+0000 7f62b5c86640 1 --2- 172.21.15.67:0/4039912786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb131b0 0x564efdef8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.548+0000 7f62b5c86640 1 -- 172.21.15.67:0/4039912786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564efdee8820 con 0x564efdb1b390 2024-07-31T14:52:23.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.548+0000 7f62b5c86640 1 --2- 172.21.15.67:0/4039912786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb1b390 0x564efdee67f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62a801ae90 tx=0x7f62a80095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a2225afe72a4e1c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:23.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.548+0000 7f629affd640 1 -- 172.21.15.67:0/4039912786 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a803a070 con 0x564efdb1b390 2024-07-31T14:52:23.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.548+0000 7f629affd640 1 -- 172.21.15.67:0/4039912786 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a8025cf0 con 0x564efdb1b390 2024-07-31T14:52:23.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.548+0000 7f629affd640 1 -- 172.21.15.67:0/4039912786 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a801a680 con 0x564efdb1b390 2024-07-31T14:52:23.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.549+0000 7f62bdc3a740 1 -- 172.21.15.67:0/4039912786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb1b390 msgr2=0x564efdee67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:23.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.549+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/4039912786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb1b390 0x564efdee67f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62a801ae90 tx=0x7f62a80095f0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.550+0000 7f62bdc3a740 1 -- 172.21.15.67:0/4039912786 shutdown_connections 2024-07-31T14:52:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.550+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/4039912786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb1b390 0x564efdee67f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.550+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/4039912786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb131b0 0x564efdef8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.550+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/4039912786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdafa9f0 0x564efdede350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.550+0000 7f62bdc3a740 1 -- 172.21.15.67:0/4039912786 >> 172.21.15.67:0/4039912786 conn(0x564efdb30460 msgr2=0x564efdeddc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.550+0000 7f62bdc3a740 1 -- 172.21.15.67:0/4039912786 shutdown_connections 2024-07-31T14:52:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.550+0000 7f62bdc3a740 1 -- 172.21.15.67:0/4039912786 wait complete. 2024-07-31T14:52:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.550+0000 7f62bdc3a740 1 -- start start 2024-07-31T14:52:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.550+0000 7f62bdc3a740 1 Processor -- start 2024-07-31T14:52:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.551+0000 7f62bdc3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdafa9f0 0x564efdf04720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.551+0000 7f62bdc3a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb131b0 0x564efdf04c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.551+0000 7f62b4c84640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb131b0 0x564efdf04c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.551+0000 7f62b4c84640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb131b0 0x564efdf04c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33744/0 (socket says 172.21.15.67:33744) 2024-07-31T14:52:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.551+0000 7f62bdc3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb1b390 0x564efdef3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.551+0000 7f62b4c84640 1 -- 172.21.15.67:0/2133966907 learned_addr learned my addr 172.21.15.67:0/2133966907 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.551+0000 7f62bdc3a740 1 -- 172.21.15.67:0/2133966907 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01b8130 con 0x564efdb131b0 2024-07-31T14:52:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.551+0000 7f62bdc3a740 1 -- 172.21.15.67:0/2133966907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01b7a60 con 0x564efdafa9f0 2024-07-31T14:52:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.551+0000 7f62bdc3a740 1 -- 172.21.15.67:0/2133966907 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01b4ca0 con 0x564efdb1b390 2024-07-31T14:52:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.551+0000 7f62b5485640 1 --2- 172.21.15.67:0/2133966907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdafa9f0 0x564efdf04720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.551+0000 7f62bdc3a740 10 client.0 _ll_register_callbacks cb 0x7f62a8009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.552+0000 7f62b5485640 1 -- 172.21.15.67:0/2133966907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb1b390 msgr2=0x564efdef3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.552+0000 7f62b5485640 1 --2- 172.21.15.67:0/2133966907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb1b390 0x564efdef3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.552+0000 7f62b5485640 1 -- 172.21.15.67:0/2133966907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb131b0 msgr2=0x564efdf04c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.552+0000 7f62b5485640 1 --2- 172.21.15.67:0/2133966907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb131b0 0x564efdf04c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.552+0000 7f62b5485640 1 -- 172.21.15.67:0/2133966907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564efdef4480 con 0x564efdafa9f0 2024-07-31T14:52:23.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.552+0000 7f62b5485640 1 --2- 172.21.15.67:0/2133966907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdafa9f0 0x564efdf04720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f62ac031440 tx=0x7f62ac02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7014550d4d68399d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:23.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.552+0000 7f6298ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:52:23.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.552+0000 7f6298ff9640 1 -- 172.21.15.67:0/2133966907 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac058070 con 0x564efdafa9f0 2024-07-31T14:52:23.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.552+0000 7f6298ff9640 1 -- 172.21.15.67:0/2133966907 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f62ac005400 con 0x564efdafa9f0 2024-07-31T14:52:23.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.552+0000 7f62bdc3a740 1 -- 172.21.15.67:0/2133966907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a8025cf0 con 0x564efdafa9f0 2024-07-31T14:52:23.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.553+0000 7f6298ff9640 1 -- 172.21.15.67:0/2133966907 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f62ac035040 con 0x564efdafa9f0 2024-07-31T14:52:23.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.553+0000 7f62bdc3a740 10 client.25625 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.553+0000 7f62bdc3a740 1 -- 172.21.15.67:0/2133966907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564efdef56e0 con 0x564efdafa9f0 2024-07-31T14:52:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.553+0000 7f6298ff9640 1 -- 172.21.15.67:0/2133966907 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62ac005600 con 0x564efdafa9f0 2024-07-31T14:52:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.554+0000 7f6298ff9640 1 -- 172.21.15.67:0/2133966907 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f62ac03c4d0 con 0x564efdafa9f0 2024-07-31T14:52:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.554+0000 7f62bdc3a740 10 client.25625 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.554+0000 7f62bdc3a740 10 client.25625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.554+0000 7f62bdc3a740 20 client.25625 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.554+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/2133966907 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564efdb3c070 0x564efdf69f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.554+0000 7f62bdc3a740 4 client.25625 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:52:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.554+0000 7f62bdc3a740 1 -- 172.21.15.67:0/2133966907 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f62a8025cf0 con 0x564efdb3c070 2024-07-31T14:52:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.554+0000 7f62b4c84640 1 --2- 172.21.15.67:0/2133966907 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564efdb3c070 0x564efdf69f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.555+0000 7f6298ff9640 10 client.25625 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.555+0000 7f62b4c84640 1 --2- 172.21.15.67:0/2133966907 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564efdb3c070 0x564efdf69f20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6d981d248515929 server_cookie=2f5d861d8caa90e9 in_seq=0 out_seq=0 2024-07-31T14:52:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.555+0000 7f6298ff9640 1 -- 172.21.15.67:0/2133966907 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f62a8025cf0 con 0x564efdb3c070 2024-07-31T14:52:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.555+0000 7f6298ff9640 10 client.25625 handle_command_reply: tid=0 2024-07-31T14:52:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.555+0000 7f62bdc3a740 1 client.25625 shutdown 2024-07-31T14:52:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.556+0000 7f62bdc3a740 1 -- 172.21.15.67:0/2133966907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdafa9f0 msgr2=0x564efdf04720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.556+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/2133966907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdafa9f0 0x564efdf04720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f62ac031440 tx=0x7f62ac02a410 comp rx=0 tx=0).stop 2024-07-31T14:52:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.556+0000 7f62bdc3a740 1 -- 172.21.15.67:0/2133966907 shutdown_connections 2024-07-31T14:52:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.556+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/2133966907 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564efdb3c070 0x564efdf69f20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.556+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/2133966907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564efdb1b390 0x564efdef3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.556+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/2133966907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564efdb131b0 0x564efdf04c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.556+0000 7f62bdc3a740 1 --2- 172.21.15.67:0/2133966907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564efdafa9f0 0x564efdf04720 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.556+0000 7f62bdc3a740 1 -- 172.21.15.67:0/2133966907 >> 172.21.15.67:0/2133966907 conn(0x564efdb30460 msgr2=0x564efdedeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.556+0000 7f6298ff9640 0 client.25625 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.556+0000 7f62bdc3a740 1 -- 172.21.15.67:0/2133966907 shutdown_connections 2024-07-31T14:52:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.556+0000 7f62bdc3a740 1 -- 172.21.15.67:0/2133966907 wait complete. 2024-07-31T14:52:23.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.557+0000 7f62bdc3a740 20 client.25625 trim_cache size 0 max 16384 2024-07-31T14:52:23.561 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:52:23.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.557+0000 7f62b76e9640 1 -- 172.21.15.67:0/561899367 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f629c0b3d40 msgr2=0x7f629c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:23.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.557+0000 7f62b76e9640 1 --2- 172.21.15.67:0/561899367 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f629c0b3d40 0x7f629c0b6250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009270 tx=0x7f62a8009200 comp rx=0 tx=0).stop 2024-07-31T14:52:23.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.557+0000 7f62b76e9640 1 -- 172.21.15.67:0/561899367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62b0107dd0 msgr2=0x7f62b010bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:23.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.557+0000 7f62b76e9640 1 --2- 172.21.15.67:0/561899367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62b0107dd0 0x7f62b010bf20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f62a000d770 tx=0x7f62a000dc30 comp rx=0 tx=0).stop 2024-07-31T14:52:23.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.558+0000 7f62b76e9640 1 -- 172.21.15.67:0/561899367 shutdown_connections 2024-07-31T14:52:23.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.558+0000 7f62b76e9640 1 --2- 172.21.15.67:0/561899367 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f629c0b3d40 0x7f629c0b6250 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.558+0000 7f62b76e9640 1 --2- 172.21.15.67:0/561899367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b0108720 0x7f62b01abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.558+0000 7f62b76e9640 1 --2- 172.21.15.67:0/561899367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62b0107dd0 0x7f62b010bf20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.558+0000 7f62b76e9640 1 --2- 172.21.15.67:0/561899367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b0102490 0x7f62b010b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:23.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.558+0000 7f62b76e9640 1 -- 172.21.15.67:0/561899367 >> 172.21.15.67:0/561899367 conn(0x7f62b0079160 msgr2=0x7f62b0100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:23.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.558+0000 7f62b76e9640 1 -- 172.21.15.67:0/561899367 shutdown_connections 2024-07-31T14:52:23.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:23.558+0000 7f62b76e9640 1 -- 172.21.15.67:0/561899367 wait complete. 2024-07-31T14:52:24.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.244+0000 7feb041cc640 1 -- 172.21.15.67:0/909060958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feafc1019e0 msgr2=0x7feafc101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:24.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.244+0000 7feb041cc640 1 --2- 172.21.15.67:0/909060958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feafc1019e0 0x7feafc101df0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009900 tx=0x7feaf802a410 comp rx=0 tx=0).stop 2024-07-31T14:52:24.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.245+0000 7feb041cc640 1 -- 172.21.15.67:0/909060958 shutdown_connections 2024-07-31T14:52:24.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.245+0000 7feb041cc640 1 --2- 172.21.15.67:0/909060958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feafc106610 0x7feafc110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:24.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.245+0000 7feb041cc640 1 --2- 172.21.15.67:0/909060958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feafc105cc0 0x7feafc1060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:24.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.245+0000 7feb041cc640 1 --2- 172.21.15.67:0/909060958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feafc1019e0 0x7feafc101df0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:24.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.245+0000 7feb041cc640 1 -- 172.21.15.67:0/909060958 >> 172.21.15.67:0/909060958 conn(0x7feafc077690 msgr2=0x7feafc077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:24.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.245+0000 7feb041cc640 1 -- 172.21.15.67:0/909060958 shutdown_connections 2024-07-31T14:52:24.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.246+0000 7feb041cc640 1 -- 172.21.15.67:0/909060958 wait complete. 2024-07-31T14:52:24.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.246+0000 7feb041cc640 1 Processor -- start 2024-07-31T14:52:24.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.247+0000 7feb041cc640 1 -- start start 2024-07-31T14:52:24.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.247+0000 7feb041cc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feafc1019e0 0x7feafc102980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:24.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.247+0000 7feb041cc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feafc105cc0 0x7feafc102ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:24.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.247+0000 7feb041cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feafc106610 0x7feafc103400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:24.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.247+0000 7feb041cc640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feafc113530 con 0x7feafc1019e0 2024-07-31T14:52:24.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.247+0000 7feb041cc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1133b0 con 0x7feafc105cc0 2024-07-31T14:52:24.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.247+0000 7feb041cc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1136b0 con 0x7feafc106610 2024-07-31T14:52:24.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.247+0000 7feb01f68640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feafc1019e0 0x7feafc102980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:24.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.247+0000 7feb01f68640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feafc1019e0 0x7feafc102980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33764/0 (socket says 172.21.15.67:33764) 2024-07-31T14:52:24.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.247+0000 7feb01f68640 1 -- 172.21.15.67:0/940629742 learned_addr learned my addr 172.21.15.67:0/940629742 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:24.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.247+0000 7feb01767640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feafc105cc0 0x7feafc102ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:24.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.248+0000 7feb02769640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feafc106610 0x7feafc103400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:24.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.248+0000 7feb01f68640 1 -- 172.21.15.67:0/940629742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feafc106610 msgr2=0x7feafc103400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:24.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.248+0000 7feb01f68640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feafc106610 0x7feafc103400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.248+0000 7feb01f68640 1 -- 172.21.15.67:0/940629742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feafc105cc0 msgr2=0x7feafc102ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.248+0000 7feb01f68640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feafc105cc0 0x7feafc102ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.248+0000 7feb01f68640 1 -- 172.21.15.67:0/940629742 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feafc1a8260 con 0x7feafc1019e0 2024-07-31T14:52:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.248+0000 7feb01767640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feafc105cc0 0x7feafc102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.248+0000 7feb02769640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feafc106610 0x7feafc103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.248+0000 7feb01f68640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feafc1019e0 0x7feafc102980 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7feaf8005ab0 tx=0x7feaf8003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.248+0000 7feaf2ffd640 1 -- 172.21.15.67:0/940629742 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf803f070 con 0x7feafc1019e0 2024-07-31T14:52:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.249+0000 7feb041cc640 1 -- 172.21.15.67:0/940629742 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feafc1a84f0 con 0x7feafc1019e0 2024-07-31T14:52:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.249+0000 7feb041cc640 1 -- 172.21.15.67:0/940629742 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feafc1a8a20 con 0x7feafc1019e0 2024-07-31T14:52:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.249+0000 7feaf2ffd640 1 -- 172.21.15.67:0/940629742 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf8031d60 con 0x7feafc1019e0 2024-07-31T14:52:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.249+0000 7feaf2ffd640 1 -- 172.21.15.67:0/940629742 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8035030 con 0x7feafc1019e0 2024-07-31T14:52:24.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.250+0000 7feb041cc640 1 -- 172.21.15.67:0/940629742 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae00052c0 con 0x7feafc1019e0 2024-07-31T14:52:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.253+0000 7feaf2ffd640 1 -- 172.21.15.67:0/940629742 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7feaf8042070 con 0x7feafc1019e0 2024-07-31T14:52:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.254+0000 7feaf2ffd640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feae80b3dc0 0x7feae80b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.254+0000 7feaf2ffd640 1 -- 172.21.15.67:0/940629742 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7feaf8104d60 con 0x7feafc1019e0 2024-07-31T14:52:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.254+0000 7feb01767640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feae80b3dc0 0x7feae80b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.255+0000 7feb01767640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feae80b3dc0 0x7feae80b62d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feaec0058c0 tx=0x7feaec005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:24.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.256+0000 7feaf2ffd640 1 -- 172.21.15.67:0/940629742 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7feaf80cbd00 con 0x7feafc1019e0 2024-07-31T14:52:24.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.498+0000 7feb041cc640 1 -- 172.21.15.67:0/940629742 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feae00055b0 con 0x7feafc1019e0 2024-07-31T14:52:24.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.499+0000 7feaf2ffd640 1 -- 172.21.15.67:0/940629742 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11962 (secure 0 0 0) 0x7feaf80cb600 con 0x7feafc1019e0 2024-07-31T14:52:24.500 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:52:24.500 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-31T14:51:08:515297+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:51:07.734542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:52:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-31T14:52:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.501+0000 7feb041cc640 1 -- 172.21.15.67:0/940629742 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feae80b3dc0 msgr2=0x7feae80b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.501+0000 7feb041cc640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feae80b3dc0 0x7feae80b62d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feaec0058c0 tx=0x7feaec005850 comp rx=0 tx=0).stop 2024-07-31T14:52:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.501+0000 7feb041cc640 1 -- 172.21.15.67:0/940629742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feafc1019e0 msgr2=0x7feafc102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.501+0000 7feb041cc640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feafc1019e0 0x7feafc102980 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7feaf8005ab0 tx=0x7feaf8003cd0 comp rx=0 tx=0).stop 2024-07-31T14:52:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.501+0000 7feb041cc640 1 -- 172.21.15.67:0/940629742 shutdown_connections 2024-07-31T14:52:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.501+0000 7feb041cc640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feae80b3dc0 0x7feae80b62d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.501+0000 7feb041cc640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feafc106610 0x7feafc103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.501+0000 7feb041cc640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feafc105cc0 0x7feafc102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:24.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.502+0000 7feb041cc640 1 --2- 172.21.15.67:0/940629742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feafc1019e0 0x7feafc102980 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:24.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.502+0000 7feb041cc640 1 -- 172.21.15.67:0/940629742 >> 172.21.15.67:0/940629742 conn(0x7feafc077690 msgr2=0x7feafc107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:24.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.502+0000 7feb041cc640 1 -- 172.21.15.67:0/940629742 shutdown_connections 2024-07-31T14:52:24.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:24.502+0000 7feb041cc640 1 -- 172.21.15.67:0/940629742 wait complete. 2024-07-31T14:52:24.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:24 smithi067 ceph-mon[32750]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-07-31T14:52:24.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:24 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/940629742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:52:25.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:24 smithi073 ceph-mon[36714]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-07-31T14:52:25.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:24 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/940629742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:52:25.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:24 smithi077 ceph-mon[37327]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-07-31T14:52:25.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:24 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/940629742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:52:25.700 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 153.4 --await-for 45.0' 2024-07-31T14:52:25.700 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 153.4 --await-for 45.0 2024-07-31T14:52:25.931 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T14:52:25.932 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T14:52:26.003 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:26.758 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:26 smithi067 ceph-mon[32750]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-07-31T14:52:27.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:26 smithi073 ceph-mon[36714]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-07-31T14:52:27.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:26 smithi077 ceph-mon[37327]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-07-31T14:52:27.302 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:28.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.589+0000 7f3c48dce640 1 -- 172.21.15.67:0/1640209696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c44107df0 msgr2=0x7f3c44108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:28.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.589+0000 7f3c48dce640 1 --2- 172.21.15.67:0/1640209696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c44107df0 0x7f3c44108200 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009900 tx=0x7f3c2c02a410 comp rx=0 tx=0).stop 2024-07-31T14:52:28.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.589+0000 7f3c48dce640 1 -- 172.21.15.67:0/1640209696 shutdown_connections 2024-07-31T14:52:28.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.589+0000 7f3c48dce640 1 --2- 172.21.15.67:0/1640209696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c44108740 0x7f3c4410ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.589+0000 7f3c48dce640 1 --2- 172.21.15.67:0/1640209696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c44107df0 0x7f3c44108200 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.589+0000 7f3c48dce640 1 --2- 172.21.15.67:0/1640209696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c441024b0 0x7f3c441028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.589+0000 7f3c48dce640 1 -- 172.21.15.67:0/1640209696 >> 172.21.15.67:0/1640209696 conn(0x7f3c44079190 msgr2=0x7f3c440795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:28.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.589+0000 7f3c48dce640 1 -- 172.21.15.67:0/1640209696 shutdown_connections 2024-07-31T14:52:28.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.589+0000 7f3c48dce640 1 -- 172.21.15.67:0/1640209696 wait complete. 2024-07-31T14:52:28.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.591+0000 7f3c48dce640 1 Processor -- start 2024-07-31T14:52:28.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.591+0000 7f3c48dce640 1 -- start start 2024-07-31T14:52:28.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.591+0000 7f3c48dce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c441024b0 0x7f3c441a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.591+0000 7f3c48dce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c44107df0 0x7f3c441a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.591+0000 7f3c48dce640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c44108740 0x7f3c441ac220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.591+0000 7f3c48dce640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3c44113500 con 0x7f3c44107df0 2024-07-31T14:52:28.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.591+0000 7f3c3bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c44107df0 0x7f3c441a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.591+0000 7f3c3bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c44107df0 0x7f3c441a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33788/0 (socket says 172.21.15.67:33788) 2024-07-31T14:52:28.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.591+0000 7f3c4259c640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c441024b0 0x7f3c441a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.591+0000 7f3c3bfff640 1 -- 172.21.15.67:0/2748490394 learned_addr learned my addr 172.21.15.67:0/2748490394 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:28.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.591+0000 7f3c48dce640 1 -- 172.21.15.67:0/2748490394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3c44113380 con 0x7f3c44108740 2024-07-31T14:52:28.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.591+0000 7f3c48dce640 1 -- 172.21.15.67:0/2748490394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c44113680 con 0x7f3c441024b0 2024-07-31T14:52:28.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.592+0000 7f3c42d9d640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c44108740 0x7f3c441ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.592+0000 7f3c4259c640 1 -- 172.21.15.67:0/2748490394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c44108740 msgr2=0x7f3c441ac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:28.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.592+0000 7f3c4259c640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c44108740 0x7f3c441ac220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.592+0000 7f3c4259c640 1 -- 172.21.15.67:0/2748490394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c44107df0 msgr2=0x7f3c441a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:28.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.592+0000 7f3c4259c640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c44107df0 0x7f3c441a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.592+0000 7f3c4259c640 1 -- 172.21.15.67:0/2748490394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c441ac760 con 0x7f3c441024b0 2024-07-31T14:52:28.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.592+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c44107df0 0x7f3c441a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:28.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.593+0000 7f3c4259c640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c441024b0 0x7f3c441a11a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009f40 tx=0x7f3c3400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:28.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.593+0000 7f3c3b7fe640 1 -- 172.21.15.67:0/2748490394 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3400cd00 con 0x7f3c441024b0 2024-07-31T14:52:28.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.593+0000 7f3c48dce640 1 -- 172.21.15.67:0/2748490394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4406c810 con 0x7f3c441024b0 2024-07-31T14:52:28.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.593+0000 7f3c48dce640 1 -- 172.21.15.67:0/2748490394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4406cd70 con 0x7f3c441024b0 2024-07-31T14:52:28.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.593+0000 7f3c3b7fe640 1 -- 172.21.15.67:0/2748490394 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c340035f0 con 0x7f3c441024b0 2024-07-31T14:52:28.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.594+0000 7f3c3b7fe640 1 -- 172.21.15.67:0/2748490394 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c340216a0 con 0x7f3c441024b0 2024-07-31T14:52:28.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.597+0000 7f3c4f31f740 1 Processor -- start 2024-07-31T14:52:28.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.597+0000 7f3c4f31f740 1 -- start start 2024-07-31T14:52:28.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.597+0000 7f3c3b7fe640 1 -- 172.21.15.67:0/2748490394 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3c34021840 con 0x7f3c441024b0 2024-07-31T14:52:28.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.597+0000 7f3c4f31f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc04aa840 0x55dbc0850870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.598+0000 7f3c4f31f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc0481980 0x55dbc0850db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.598+0000 7f3c4f31f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc0489b60 0x55dbc08512f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.598+0000 7f3c4259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc0481980 0x55dbc0850db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.598+0000 7f3c4259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc0481980 0x55dbc0850db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33792/0 (socket says 172.21.15.67:33792) 2024-07-31T14:52:28.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.598+0000 7f3c42d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc04aa840 0x55dbc0850870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.598+0000 7f3c42d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc04aa840 0x55dbc0850870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50412/0 (socket says 172.21.15.67:50412) 2024-07-31T14:52:28.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.598+0000 7f3c4259c640 1 -- 172.21.15.67:0/3709920890 learned_addr learned my addr 172.21.15.67:0/3709920890 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:28.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.598+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3709920890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55dbc07c57b0 con 0x55dbc0481980 2024-07-31T14:52:28.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.598+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3709920890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dbc07b4cb0 con 0x55dbc0489b60 2024-07-31T14:52:28.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.598+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3709920890 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dbc07b8cf0 con 0x55dbc04aa840 2024-07-31T14:52:28.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.598+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/3709920890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc0489b60 0x55dbc08512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.599+0000 7f3c3bfff640 1 -- 172.21.15.67:0/3709920890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc04aa840 msgr2=0x55dbc0850870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:28.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.599+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/3709920890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc04aa840 0x55dbc0850870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.599+0000 7f3c3bfff640 1 -- 172.21.15.67:0/3709920890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc0481980 msgr2=0x55dbc0850db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:28.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.599+0000 7f3c42d9d640 1 --2- 172.21.15.67:0/3709920890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc04aa840 0x55dbc0850870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:28.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.599+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/3709920890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc0481980 0x55dbc0850db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.599+0000 7f3c3bfff640 1 -- 172.21.15.67:0/3709920890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbc074a0b0 con 0x55dbc0489b60 2024-07-31T14:52:28.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.599+0000 7f3c4259c640 1 --2- 172.21.15.67:0/3709920890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc0481980 0x55dbc0850db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:28.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.600+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/3709920890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc0489b60 0x55dbc08512f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c004770 tx=0x7f3c2c004520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb3b64fb63dba6a0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.600+0000 7f3c38ff9640 1 -- 172.21.15.67:0/3709920890 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c040070 con 0x55dbc0489b60 2024-07-31T14:52:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.600+0000 7f3c38ff9640 1 -- 172.21.15.67:0/3709920890 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c2c0047a0 con 0x55dbc0489b60 2024-07-31T14:52:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.600+0000 7f3c38ff9640 1 -- 172.21.15.67:0/3709920890 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c035040 con 0x55dbc0489b60 2024-07-31T14:52:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.601+0000 7f3c3b7fe640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3c300b3dc0 0x7f3c300b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.601+0000 7f3c42d9d640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3c300b3dc0 0x7f3c300b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.601+0000 7f3c3b7fe640 1 -- 172.21.15.67:0/2748490394 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f3c340f5310 con 0x7f3c441024b0 2024-07-31T14:52:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.602+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3709920890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc0489b60 msgr2=0x55dbc08512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.602+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/3709920890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc0489b60 0x55dbc08512f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c004770 tx=0x7f3c2c004520 comp rx=0 tx=0).stop 2024-07-31T14:52:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.602+0000 7f3c42d9d640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3c300b3dc0 0x7f3c300b62d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c005fd0 tx=0x7f3c3c009580 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.602+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3709920890 shutdown_connections 2024-07-31T14:52:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.602+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/3709920890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc0489b60 0x55dbc08512f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.602+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/3709920890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc0481980 0x55dbc0850db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.602+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/3709920890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc04aa840 0x55dbc0850870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.602+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3709920890 >> 172.21.15.67:0/3709920890 conn(0x55dbc049e990 msgr2=0x55dbc084e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.602+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3709920890 shutdown_connections 2024-07-31T14:52:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.602+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3709920890 wait complete. 2024-07-31T14:52:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.603+0000 7f3c4f31f740 1 -- start start 2024-07-31T14:52:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.603+0000 7f3c4f31f740 1 Processor -- start 2024-07-31T14:52:28.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.604+0000 7f3c4f31f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc0481980 0x55dbc085bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.604+0000 7f3c4f31f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc04aa840 0x55dbc085c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.604+0000 7f3c4f31f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc04691c0 0x55dbc0868340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.604+0000 7f3c4f31f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55dbc06187a0 con 0x55dbc04691c0 2024-07-31T14:52:28.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.604+0000 7f3c4f31f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dbc0757320 con 0x55dbc04aa840 2024-07-31T14:52:28.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.604+0000 7f3c4f31f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dbc085eff0 con 0x55dbc0481980 2024-07-31T14:52:28.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.604+0000 7f3c4259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc0481980 0x55dbc085bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.604+0000 7f3c4259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc0481980 0x55dbc085bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50422/0 (socket says 172.21.15.67:50422) 2024-07-31T14:52:28.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.604+0000 7f3c4259c640 1 -- 172.21.15.67:0/2876061636 learned_addr learned my addr 172.21.15.67:0/2876061636 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:28.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.605+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/2876061636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc04aa840 0x55dbc085c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.605+0000 7f3c42d9d640 1 --2- 172.21.15.67:0/2876061636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc04691c0 0x55dbc0868340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.605+0000 7f3c4f31f740 10 client.0 _ll_register_callbacks cb 0x55dbc0761140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:28.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.605+0000 7f3c3bfff640 1 -- 172.21.15.67:0/2876061636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc0481980 msgr2=0x55dbc085bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:28.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.605+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/2876061636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc0481980 0x55dbc085bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.605+0000 7f3c3bfff640 1 -- 172.21.15.67:0/2876061636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc04691c0 msgr2=0x55dbc0868340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:28.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.605+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/2876061636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc04691c0 0x55dbc0868340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.605+0000 7f3c3bfff640 1 -- 172.21.15.67:0/2876061636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbc075b4b0 con 0x55dbc04aa840 2024-07-31T14:52:28.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.605+0000 7f3c42d9d640 1 --2- 172.21.15.67:0/2876061636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc04691c0 0x55dbc0868340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:28.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.606+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/2876061636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc04aa840 0x55dbc085c480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c03e000 tx=0x7f3c2c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75f40932f94182fa server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:28.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.606+0000 7f3c29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:52:28.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.606+0000 7f3c29ffb640 1 -- 172.21.15.67:0/2876061636 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c035070 con 0x55dbc04aa840 2024-07-31T14:52:28.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.606+0000 7f3c4f31f740 1 -- 172.21.15.67:0/2876061636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c2c0047a0 con 0x55dbc04aa840 2024-07-31T14:52:28.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.606+0000 7f3c29ffb640 1 -- 172.21.15.67:0/2876061636 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f3c2c005200 con 0x55dbc04aa840 2024-07-31T14:52:28.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.606+0000 7f3c29ffb640 1 -- 172.21.15.67:0/2876061636 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c2c040040 con 0x55dbc04aa840 2024-07-31T14:52:28.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.606+0000 7f3c4f31f740 10 client.25643 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:28.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.606+0000 7f3c4f31f740 1 -- 172.21.15.67:0/2876061636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dbc0524b90 con 0x55dbc04aa840 2024-07-31T14:52:28.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.607+0000 7f3c29ffb640 1 -- 172.21.15.67:0/2876061636 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c2c0047a0 con 0x55dbc04aa840 2024-07-31T14:52:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.608+0000 7f3c29ffb640 1 -- 172.21.15.67:0/2876061636 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3c2c0314a0 con 0x55dbc04aa840 2024-07-31T14:52:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.608+0000 7f3c4f31f740 10 client.25643 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.608+0000 7f3c4f31f740 10 client.25643 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:52:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.608+0000 7f3c4f31f740 20 client.25643 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.608+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/2876061636 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55dbc043e570 0x55dbc086df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.608+0000 7f3c4f31f740 4 client.25643 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.609+0000 7f3c4f31f740 1 -- 172.21.15.67:0/2876061636 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbc07c7700 con 0x55dbc043e570 2024-07-31T14:52:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.609+0000 7f3c4259c640 1 --2- 172.21.15.67:0/2876061636 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55dbc043e570 0x55dbc086df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.609+0000 7f3c29ffb640 10 client.25643 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:28.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.609+0000 7f3c4259c640 1 --2- 172.21.15.67:0/2876061636 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55dbc043e570 0x55dbc086df60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d3170561cc7d0a3 server_cookie=6ffaca1a33eb749f in_seq=0 out_seq=0 2024-07-31T14:52:28.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.610+0000 7f3c29ffb640 1 -- 172.21.15.67:0/2876061636 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f3c34095e50 con 0x55dbc043e570 2024-07-31T14:52:28.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.610+0000 7f3c29ffb640 10 client.25643 handle_command_reply: tid=0 2024-07-31T14:52:28.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.610+0000 7f3c4f31f740 1 client.25643 shutdown 2024-07-31T14:52:28.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.611+0000 7f3c4f31f740 1 -- 172.21.15.67:0/2876061636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc04aa840 msgr2=0x55dbc085c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:28.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.611+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/2876061636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc04aa840 0x55dbc085c480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c03e000 tx=0x7f3c2c009260 comp rx=0 tx=0).stop 2024-07-31T14:52:28.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.611+0000 7f3c4f31f740 1 -- 172.21.15.67:0/2876061636 shutdown_connections 2024-07-31T14:52:28.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.611+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/2876061636 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55dbc043e570 0x55dbc086df60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.611+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/2876061636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc04691c0 0x55dbc0868340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.611+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/2876061636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc04aa840 0x55dbc085c480 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.611+0000 7f3c29ffb640 0 client.25643 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:28.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.611+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/2876061636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc0481980 0x55dbc085bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.611+0000 7f3c4f31f740 1 -- 172.21.15.67:0/2876061636 >> 172.21.15.67:0/2876061636 conn(0x55dbc049e990 msgr2=0x55dbc0853a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:28.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.611+0000 7f3c4f31f740 1 -- 172.21.15.67:0/2876061636 shutdown_connections 2024-07-31T14:52:28.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.611+0000 7f3c4f31f740 1 -- 172.21.15.67:0/2876061636 wait complete. 2024-07-31T14:52:28.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.612+0000 7f3c4f31f740 20 client.25643 trim_cache size 0 max 16384 2024-07-31T14:52:28.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.649+0000 7f3c4f31f740 1 Processor -- start 2024-07-31T14:52:28.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.649+0000 7f3c4f31f740 1 -- start start 2024-07-31T14:52:28.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.649+0000 7f3c4f31f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc043e570 0x55dbc08576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.649+0000 7f3c4f31f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc04691c0 0x55dbc0855900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.649+0000 7f3c3bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc04691c0 0x55dbc0855900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.649+0000 7f3c3bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc04691c0 0x55dbc0855900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33816/0 (socket says 172.21.15.67:33816) 2024-07-31T14:52:28.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.649+0000 7f3c3bfff640 1 -- 172.21.15.67:0/3840719847 learned_addr learned my addr 172.21.15.67:0/3840719847 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:28.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.650+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/3840719847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc0481980 0x55dbc0855e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.650+0000 7f3c4259c640 1 --2- 172.21.15.67:0/3840719847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc043e570 0x55dbc08576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.650+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3840719847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2400fbc0 con 0x55dbc04691c0 2024-07-31T14:52:28.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.650+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3840719847 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3c24011700 con 0x55dbc0481980 2024-07-31T14:52:28.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.650+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3840719847 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c24010290 con 0x55dbc043e570 2024-07-31T14:52:28.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.650+0000 7f3c42d9d640 1 --2- 172.21.15.67:0/3840719847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc0481980 0x55dbc0855e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.650+0000 7f3c3bfff640 1 -- 172.21.15.67:0/3840719847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc043e570 msgr2=0x55dbc08576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.650+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/3840719847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc043e570 0x55dbc08576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.650+0000 7f3c3bfff640 1 -- 172.21.15.67:0/3840719847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc0481980 msgr2=0x55dbc0855e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.650+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/3840719847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc0481980 0x55dbc0855e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.650+0000 7f3c3bfff640 1 -- 172.21.15.67:0/3840719847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbc085c7f0 con 0x55dbc04691c0 2024-07-31T14:52:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.650+0000 7f3c4259c640 1 --2- 172.21.15.67:0/3840719847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc043e570 0x55dbc08576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.650+0000 7f3c42d9d640 1 --2- 172.21.15.67:0/3840719847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc0481980 0x55dbc0855e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.650+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/3840719847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc04691c0 0x55dbc0855900 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0096c0 tx=0x7f3c2c004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619373d928e9340f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.651+0000 7f3c2bfff640 1 -- 172.21.15.67:0/3840719847 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c035020 con 0x55dbc04691c0 2024-07-31T14:52:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.651+0000 7f3c2bfff640 1 -- 172.21.15.67:0/3840719847 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c2c005850 con 0x55dbc04691c0 2024-07-31T14:52:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.651+0000 7f3c2bfff640 1 -- 172.21.15.67:0/3840719847 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c004e90 con 0x55dbc04691c0 2024-07-31T14:52:28.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.652+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3840719847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc04691c0 msgr2=0x55dbc0855900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:28.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.652+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/3840719847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc04691c0 0x55dbc0855900 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0096c0 tx=0x7f3c2c004520 comp rx=0 tx=0).stop 2024-07-31T14:52:28.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.652+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3840719847 shutdown_connections 2024-07-31T14:52:28.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.652+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/3840719847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc0481980 0x55dbc0855e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.652+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/3840719847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc04691c0 0x55dbc0855900 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.652+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/3840719847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc043e570 0x55dbc08576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.652+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3840719847 >> 172.21.15.67:0/3840719847 conn(0x55dbc049e990 msgr2=0x55dbc085b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:28.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.652+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3840719847 shutdown_connections 2024-07-31T14:52:28.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.652+0000 7f3c4f31f740 1 -- 172.21.15.67:0/3840719847 wait complete. 2024-07-31T14:52:28.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.653+0000 7f3c4f31f740 1 -- start start 2024-07-31T14:52:28.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.653+0000 7f3c4f31f740 1 Processor -- start 2024-07-31T14:52:28.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.653+0000 7f3c4f31f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc043e570 0x55dbc08607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.653+0000 7f3c4f31f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc04691c0 0x55dbc0860d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c4f31f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc0481980 0x55dbc08d57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c4f31f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3c24011030 con 0x55dbc043e570 2024-07-31T14:52:28.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c4f31f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3c24010960 con 0x55dbc04691c0 2024-07-31T14:52:28.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c4f31f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2400dc50 con 0x55dbc0481980 2024-07-31T14:52:28.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c4f31f740 10 client.0 _ll_register_callbacks cb 0x7f3c2c0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:28.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c3bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc04691c0 0x55dbc0860d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c3bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc04691c0 0x55dbc0860d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:54188/0 (socket says 172.21.15.67:54188) 2024-07-31T14:52:28.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c3bfff640 1 -- 172.21.15.67:0/780843453 learned_addr learned my addr 172.21.15.67:0/780843453 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:28.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c42d9d640 1 --2- 172.21.15.67:0/780843453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc0481980 0x55dbc08d57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c4259c640 1 --2- 172.21.15.67:0/780843453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc043e570 0x55dbc08607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c3bfff640 1 -- 172.21.15.67:0/780843453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc0481980 msgr2=0x55dbc08d57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:28.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/780843453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc0481980 0x55dbc08d57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c3bfff640 1 -- 172.21.15.67:0/780843453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc043e570 msgr2=0x55dbc08607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:28.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/780843453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc043e570 0x55dbc08607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c3bfff640 1 -- 172.21.15.67:0/780843453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbc08d5d10 con 0x55dbc04691c0 2024-07-31T14:52:28.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.654+0000 7f3c4259c640 1 --2- 172.21.15.67:0/780843453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc043e570 0x55dbc08607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:28.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.655+0000 7f3c3bfff640 1 --2- 172.21.15.67:0/780843453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc04691c0 0x55dbc0860d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c031870 tx=0x7f3c2c005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a5de9802da4aa88 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:28.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.655+0000 7f3c29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:52:28.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.655+0000 7f3c29ffb640 1 -- 172.21.15.67:0/780843453 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c035070 con 0x55dbc04691c0 2024-07-31T14:52:28.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.655+0000 7f3c29ffb640 1 -- 172.21.15.67:0/780843453 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f3c2c0039b0 con 0x55dbc04691c0 2024-07-31T14:52:28.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.655+0000 7f3c4f31f740 1 -- 172.21.15.67:0/780843453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c2c005850 con 0x55dbc04691c0 2024-07-31T14:52:28.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.655+0000 7f3c29ffb640 1 -- 172.21.15.67:0/780843453 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c2c070050 con 0x55dbc04691c0 2024-07-31T14:52:28.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.655+0000 7f3c4f31f740 10 client.25646 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:28.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.655+0000 7f3c4f31f740 1 -- 172.21.15.67:0/780843453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dbc08d6e20 con 0x55dbc04691c0 2024-07-31T14:52:28.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.656+0000 7f3c29ffb640 1 -- 172.21.15.67:0/780843453 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c2c003bb0 con 0x55dbc04691c0 2024-07-31T14:52:28.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.656+0000 7f3c29ffb640 1 -- 172.21.15.67:0/780843453 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3c2c03c440 con 0x55dbc04691c0 2024-07-31T14:52:28.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.656+0000 7f3c4f31f740 10 client.25646 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:28.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.656+0000 7f3c4f31f740 10 client.25646 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:52:28.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.656+0000 7f3c4f31f740 20 client.25646 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:28.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.656+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/780843453 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55dbc04aa840 0x55dbc08d93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:28.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.656+0000 7f3c4f31f740 4 client.25646 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 153.4, "await_for": 45.0}] 2024-07-31T14:52:28.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.656+0000 7f3c4f31f740 1 -- 172.21.15.67:0/780843453 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 153.4, "await_for": 45.0}) -- 0x7f3c2c005850 con 0x55dbc04aa840 2024-07-31T14:52:28.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.656+0000 7f3c4259c640 1 --2- 172.21.15.67:0/780843453 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55dbc04aa840 0x55dbc08d93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:28.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.657+0000 7f3c29ffb640 10 client.25646 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:28.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.657+0000 7f3c4259c640 1 --2- 172.21.15.67:0/780843453 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55dbc04aa840 0x55dbc08d93c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b41a791b9bbece1 server_cookie=2a5cbf8134cf0d0a in_seq=0 out_seq=0 2024-07-31T14:52:28.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.664+0000 7f3c29ffb640 1 -- 172.21.15.67:0/780843453 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f3c2c005850 con 0x55dbc04aa840 2024-07-31T14:52:28.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.664+0000 7f3c29ffb640 10 client.25646 handle_command_reply: tid=0 2024-07-31T14:52:28.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.664+0000 7f3c4f31f740 1 client.25646 shutdown 2024-07-31T14:52:28.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.665+0000 7f3c4f31f740 1 -- 172.21.15.67:0/780843453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc04691c0 msgr2=0x55dbc0860d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:28.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.665+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/780843453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc04691c0 0x55dbc0860d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c031870 tx=0x7f3c2c005a20 comp rx=0 tx=0).stop 2024-07-31T14:52:28.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.665+0000 7f3c4f31f740 1 -- 172.21.15.67:0/780843453 shutdown_connections 2024-07-31T14:52:28.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.665+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/780843453 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55dbc04aa840 0x55dbc08d93c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.665+0000 7f3c29ffb640 0 client.25646 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:28.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.665+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/780843453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbc0481980 0x55dbc08d57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.665+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/780843453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbc04691c0 0x55dbc0860d30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.665+0000 7f3c4f31f740 1 --2- 172.21.15.67:0/780843453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbc043e570 0x55dbc08607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.665+0000 7f3c4f31f740 1 -- 172.21.15.67:0/780843453 >> 172.21.15.67:0/780843453 conn(0x55dbc049e990 msgr2=0x55dbc0867120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:28.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.665+0000 7f3c4f31f740 1 -- 172.21.15.67:0/780843453 shutdown_connections 2024-07-31T14:52:28.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.665+0000 7f3c4f31f740 1 -- 172.21.15.67:0/780843453 wait complete. 2024-07-31T14:52:28.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.665+0000 7f3c4f31f740 20 client.25646 trim_cache size 0 max 16384 2024-07-31T14:52:28.668 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:52:28.668 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 30, 2024-07-31T14:52:28.668 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T14:52:28.668 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 2, 2024-07-31T14:52:28.668 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T14:52:28.668 INFO:teuthology.orchestra.run.smithi067.stdout: "9b7c4d67": { 2024-07-31T14:52:28.668 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 2, 2024-07-31T14:52:28.668 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T14:52:28.668 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:52:28.669 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T14:52:28.669 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:52:28.669 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:52:28.669 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T14:52:28.669 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 153.4, 2024-07-31T14:52:28.669 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T14:52:28.669 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T14:52:28.669 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T14:52:28.669 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:52:28.669 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T14:52:28.669 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:52:28.669 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:52:28.669 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:52:28.670 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:52:28.670 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:52:28.670 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:52:28.670 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:52:28.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.666+0000 7f3c48dce640 1 -- 172.21.15.67:0/2748490394 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3c300b3dc0 msgr2=0x7f3c300b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:28.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.666+0000 7f3c48dce640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3c300b3dc0 0x7f3c300b62d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c005fd0 tx=0x7f3c3c009580 comp rx=0 tx=0).stop 2024-07-31T14:52:28.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.666+0000 7f3c48dce640 1 -- 172.21.15.67:0/2748490394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c441024b0 msgr2=0x7f3c441a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:28.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.666+0000 7f3c48dce640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c441024b0 0x7f3c441a11a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009f40 tx=0x7f3c3400ee50 comp rx=0 tx=0).stop 2024-07-31T14:52:28.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.667+0000 7f3c48dce640 1 -- 172.21.15.67:0/2748490394 shutdown_connections 2024-07-31T14:52:28.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.667+0000 7f3c48dce640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3c300b3dc0 0x7f3c300b62d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.667+0000 7f3c48dce640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c44108740 0x7f3c441ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.667+0000 7f3c48dce640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c44107df0 0x7f3c441a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.667+0000 7f3c48dce640 1 --2- 172.21.15.67:0/2748490394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c441024b0 0x7f3c441a11a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:28.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.667+0000 7f3c48dce640 1 -- 172.21.15.67:0/2748490394 >> 172.21.15.67:0/2748490394 conn(0x7f3c44079190 msgr2=0x7f3c44103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:28.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.667+0000 7f3c48dce640 1 -- 172.21.15.67:0/2748490394 shutdown_connections 2024-07-31T14:52:28.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:28.667+0000 7f3c48dce640 1 -- 172.21.15.67:0/2748490394 wait complete. 2024-07-31T14:52:29.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:28 smithi067 ceph-mon[32750]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-07-31T14:52:29.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:28 smithi073 ceph-mon[36714]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-07-31T14:52:29.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:28 smithi077 ceph-mon[37327]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-07-31T14:52:29.779 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9b7c4d67', quiesce took 9.6 seconds. Will release after: 23.799999999999997 2024-07-31T14:52:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.287+0000 7f607669e640 1 -- 172.21.15.67:0/2527005163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6070107c10 msgr2=0x7f6070108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.287+0000 7f607669e640 1 --2- 172.21.15.67:0/2527005163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6070107c10 0x7f6070108020 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6060009900 tx=0x7f606002a410 comp rx=0 tx=0).stop 2024-07-31T14:52:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.287+0000 7f607669e640 1 -- 172.21.15.67:0/2527005163 shutdown_connections 2024-07-31T14:52:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.287+0000 7f607669e640 1 --2- 172.21.15.67:0/2527005163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6070108560 0x7f607010ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.287+0000 7f607669e640 1 --2- 172.21.15.67:0/2527005163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6070107c10 0x7f6070108020 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.287+0000 7f607669e640 1 --2- 172.21.15.67:0/2527005163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60701022d0 0x7f60701026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.287+0000 7f607669e640 1 -- 172.21.15.67:0/2527005163 >> 172.21.15.67:0/2527005163 conn(0x7f60700fdd40 msgr2=0x7f6070100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.288+0000 7f607669e640 1 -- 172.21.15.67:0/2527005163 shutdown_connections 2024-07-31T14:52:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.288+0000 7f607669e640 1 -- 172.21.15.67:0/2527005163 wait complete. 2024-07-31T14:52:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.288+0000 7f607669e640 1 Processor -- start 2024-07-31T14:52:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.289+0000 7f607669e640 1 -- start start 2024-07-31T14:52:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.289+0000 7f607669e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60701022d0 0x7f6070199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.289+0000 7f607669e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6070107c10 0x7f607019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.289+0000 7f607669e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6070108560 0x7f607019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.289+0000 7f607669e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f60701122e0 con 0x7f6070107c10 2024-07-31T14:52:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.289+0000 7f607669e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6070112160 con 0x7f60701022d0 2024-07-31T14:52:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.289+0000 7f607669e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6070112460 con 0x7f6070108560 2024-07-31T14:52:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.289+0000 7f6074e9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6070107c10 0x7f607019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.289+0000 7f6074e9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6070107c10 0x7f607019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33846/0 (socket says 172.21.15.67:33846) 2024-07-31T14:52:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.289+0000 7f6074e9b640 1 -- 172.21.15.67:0/4007025848 learned_addr learned my addr 172.21.15.67:0/4007025848 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.289+0000 7f607569c640 1 --2- 172.21.15.67:0/4007025848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60701022d0 0x7f6070199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.290+0000 7f6075e9d640 1 --2- 172.21.15.67:0/4007025848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6070108560 0x7f607019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.290+0000 7f6074e9b640 1 -- 172.21.15.67:0/4007025848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6070108560 msgr2=0x7f607019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.290+0000 7f6074e9b640 1 --2- 172.21.15.67:0/4007025848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6070108560 0x7f607019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.290+0000 7f6074e9b640 1 -- 172.21.15.67:0/4007025848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60701022d0 msgr2=0x7f6070199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.290+0000 7f6074e9b640 1 --2- 172.21.15.67:0/4007025848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60701022d0 0x7f6070199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.290+0000 7f6074e9b640 1 -- 172.21.15.67:0/4007025848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607019fe30 con 0x7f6070107c10 2024-07-31T14:52:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.290+0000 7f6074e9b640 1 --2- 172.21.15.67:0/4007025848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6070107c10 0x7f607019a460 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f606002a930 tx=0x7f6060004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.291+0000 7f60667fc640 1 -- 172.21.15.67:0/4007025848 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606003f030 con 0x7f6070107c10 2024-07-31T14:52:30.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.291+0000 7f60667fc640 1 -- 172.21.15.67:0/4007025848 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6060003cb0 con 0x7f6070107c10 2024-07-31T14:52:30.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.291+0000 7f607669e640 1 -- 172.21.15.67:0/4007025848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60701a00c0 con 0x7f6070107c10 2024-07-31T14:52:30.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.291+0000 7f60667fc640 1 -- 172.21.15.67:0/4007025848 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060035070 con 0x7f6070107c10 2024-07-31T14:52:30.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.291+0000 7f607669e640 1 -- 172.21.15.67:0/4007025848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60700fe150 con 0x7f6070107c10 2024-07-31T14:52:30.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.294+0000 7f60667fc640 1 -- 172.21.15.67:0/4007025848 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6060042030 con 0x7f6070107c10 2024-07-31T14:52:30.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.295+0000 7f607de51740 1 Processor -- start 2024-07-31T14:52:30.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.295+0000 7f607de51740 1 -- start start 2024-07-31T14:52:30.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.295+0000 7f607de51740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6aaf350 0x5584a6e54580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.295+0000 7f607de51740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6a86490 0x5584a6e54ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.295+0000 7f607de51740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e55000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.295+0000 7f607de51740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5584a6d3c2f0 con 0x5584a6a8e670 2024-07-31T14:52:30.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.296+0000 7f607de51740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5584a6dcaa80 con 0x5584a6a86490 2024-07-31T14:52:30.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.296+0000 7f607de51740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5584a6db9f80 con 0x5584a6aaf350 2024-07-31T14:52:30.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.296+0000 7f6074e9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e55000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.296+0000 7f6074e9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e55000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33850/0 (socket says 172.21.15.67:33850) 2024-07-31T14:52:30.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.296+0000 7f6074e9b640 1 -- 172.21.15.67:0/2144185331 learned_addr learned my addr 172.21.15.67:0/2144185331 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:30.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.296+0000 7f6075e9d640 1 --2- 172.21.15.67:0/2144185331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6aaf350 0x5584a6e54580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.296+0000 7f60667fc640 1 --2- 172.21.15.67:0/4007025848 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f605c0b3dc0 0x7f605c0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.296+0000 7f607569c640 1 --2- 172.21.15.67:0/2144185331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6a86490 0x5584a6e54ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.296+0000 7f6075e9d640 1 --2- 172.21.15.67:0/4007025848 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f605c0b3dc0 0x7f605c0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.296+0000 7f60667fc640 1 -- 172.21.15.67:0/4007025848 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f6060104810 con 0x7f6070107c10 2024-07-31T14:52:30.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.297+0000 7f6074e9b640 1 -- 172.21.15.67:0/2144185331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6aaf350 msgr2=0x5584a6e54580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:30.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.297+0000 7f6074e9b640 1 --2- 172.21.15.67:0/2144185331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6aaf350 0x5584a6e54580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.297+0000 7f6074e9b640 1 -- 172.21.15.67:0/2144185331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6a86490 msgr2=0x5584a6e54ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:30.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.297+0000 7f6074e9b640 1 --2- 172.21.15.67:0/2144185331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6a86490 0x5584a6e54ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.297+0000 7f6074e9b640 1 -- 172.21.15.67:0/2144185331 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584a6b1a930 con 0x5584a6a8e670 2024-07-31T14:52:30.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.297+0000 7f6075e9d640 1 --2- 172.21.15.67:0/2144185331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6aaf350 0x5584a6e54580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:30.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.297+0000 7f6074e9b640 1 --2- 172.21.15.67:0/2144185331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e55000 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f60600c8430 tx=0x7f606002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4352419f8d5ce242 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:30.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.297+0000 7f6075e9d640 1 --2- 172.21.15.67:0/4007025848 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f605c0b3dc0 0x7f605c0b62d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6068009fd0 tx=0x7f6068004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:30.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.297+0000 7f605b7fe640 1 -- 172.21.15.67:0/2144185331 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060049070 con 0x5584a6a8e670 2024-07-31T14:52:30.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.298+0000 7f605b7fe640 1 -- 172.21.15.67:0/2144185331 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60600cacd0 con 0x5584a6a8e670 2024-07-31T14:52:30.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.298+0000 7f605b7fe640 1 -- 172.21.15.67:0/2144185331 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600cf710 con 0x5584a6a8e670 2024-07-31T14:52:30.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.299+0000 7f607de51740 1 -- 172.21.15.67:0/2144185331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 msgr2=0x5584a6e55000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:30.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.299+0000 7f607de51740 1 --2- 172.21.15.67:0/2144185331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e55000 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f60600c8430 tx=0x7f606002a410 comp rx=0 tx=0).stop 2024-07-31T14:52:30.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.299+0000 7f607de51740 1 -- 172.21.15.67:0/2144185331 shutdown_connections 2024-07-31T14:52:30.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.299+0000 7f607de51740 1 --2- 172.21.15.67:0/2144185331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e55000 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.299+0000 7f607de51740 1 --2- 172.21.15.67:0/2144185331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6a86490 0x5584a6e54ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.299+0000 7f607de51740 1 --2- 172.21.15.67:0/2144185331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6aaf350 0x5584a6e54580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.299+0000 7f607de51740 1 -- 172.21.15.67:0/2144185331 >> 172.21.15.67:0/2144185331 conn(0x5584a6aa3740 msgr2=0x5584a6e52470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:30.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.299+0000 7f607de51740 1 -- 172.21.15.67:0/2144185331 shutdown_connections 2024-07-31T14:52:30.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.299+0000 7f607de51740 1 -- 172.21.15.67:0/2144185331 wait complete. 2024-07-31T14:52:30.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.299+0000 7f607de51740 1 -- start start 2024-07-31T14:52:30.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.300+0000 7f607de51740 1 Processor -- start 2024-07-31T14:52:30.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.300+0000 7f607de51740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 0x5584a6e60140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.300+0000 7f607de51740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e60680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.300+0000 7f607de51740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6aaf350 0x5584a6e6bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.300+0000 7f607de51740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5584a6d5c4f0 con 0x5584a6a8e670 2024-07-31T14:52:30.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.300+0000 7f607de51740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5584a6dbdfc0 con 0x5584a6aaf350 2024-07-31T14:52:30.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.300+0000 7f607de51740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5584a6c1d890 con 0x5584a6a86490 2024-07-31T14:52:30.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.300+0000 7f607de51740 10 client.0 _ll_register_callbacks cb 0x5584a6d66500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:30.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.301+0000 7f6074e9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e60680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.301+0000 7f6074e9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e60680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33856/0 (socket says 172.21.15.67:33856) 2024-07-31T14:52:30.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.301+0000 7f6074e9b640 1 -- 172.21.15.67:0/735485313 learned_addr learned my addr 172.21.15.67:0/735485313 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:30.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.301+0000 7f607569c640 1 --2- 172.21.15.67:0/735485313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 0x5584a6e60140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.301+0000 7f6075e9d640 1 --2- 172.21.15.67:0/735485313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6aaf350 0x5584a6e6bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.301+0000 7f6074e9b640 1 -- 172.21.15.67:0/735485313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 msgr2=0x5584a6e60140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:30.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.301+0000 7f6074e9b640 1 --2- 172.21.15.67:0/735485313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 0x5584a6e60140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.301+0000 7f6074e9b640 1 -- 172.21.15.67:0/735485313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6aaf350 msgr2=0x5584a6e6bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:30.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.301+0000 7f6074e9b640 1 --2- 172.21.15.67:0/735485313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6aaf350 0x5584a6e6bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.301+0000 7f6074e9b640 1 -- 172.21.15.67:0/735485313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584a6d6bc50 con 0x5584a6a8e670 2024-07-31T14:52:30.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.301+0000 7f6075e9d640 1 --2- 172.21.15.67:0/735485313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6aaf350 0x5584a6e6bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:30.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.302+0000 7f6074e9b640 1 --2- 172.21.15.67:0/735485313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e60680 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f60600096c0 tx=0x7f60600cb240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ad6670698cdf96 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:30.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.302+0000 7f6058ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:30.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.302+0000 7f6058ff9640 1 -- 172.21.15.67:0/735485313 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600e5060 con 0x5584a6a8e670 2024-07-31T14:52:30.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.302+0000 7f607de51740 1 -- 172.21.15.67:0/735485313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60600cacd0 con 0x5584a6a8e670 2024-07-31T14:52:30.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.302+0000 7f6058ff9640 1 -- 172.21.15.67:0/735485313 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f60600c7450 con 0x5584a6a8e670 2024-07-31T14:52:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.302+0000 7f6058ff9640 1 -- 172.21.15.67:0/735485313 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6060049070 con 0x5584a6a8e670 2024-07-31T14:52:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.302+0000 7f607de51740 10 client.15987 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.302+0000 7f607de51740 1 -- 172.21.15.67:0/735485313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5584a6e6e8b0 con 0x5584a6a8e670 2024-07-31T14:52:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.302+0000 7f6058ff9640 1 -- 172.21.15.67:0/735485313 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60600cacd0 con 0x5584a6a8e670 2024-07-31T14:52:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.303+0000 7f6058ff9640 1 -- 172.21.15.67:0/735485313 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f606003bc50 con 0x5584a6a8e670 2024-07-31T14:52:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.303+0000 7f607de51740 10 client.15987 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.303+0000 7f607de51740 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.303+0000 7f607de51740 20 client.15987 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.303+0000 7f607de51740 1 --2- 172.21.15.67:0/735485313 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584a6a6dcd0 0x5584a6e71de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.303+0000 7f607de51740 4 client.15987 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.303+0000 7f607de51740 1 -- 172.21.15.67:0/735485313 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584a6dcc9d0 con 0x5584a6a6dcd0 2024-07-31T14:52:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.303+0000 7f607569c640 1 --2- 172.21.15.67:0/735485313 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584a6a6dcd0 0x5584a6e71de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.304+0000 7f607569c640 1 --2- 172.21.15.67:0/735485313 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584a6a6dcd0 0x5584a6e71de0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c3533d8a0ecdca server_cookie=bf3a069c782ad014 in_seq=0 out_seq=0 2024-07-31T14:52:30.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.304+0000 7f6058ff9640 10 client.15987 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:30.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f6058ff9640 1 -- 172.21.15.67:0/735485313 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584a6dcc9d0 con 0x5584a6a6dcd0 2024-07-31T14:52:30.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f6058ff9640 10 client.15987 handle_command_reply: tid=0 2024-07-31T14:52:30.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f607de51740 1 client.15987 shutdown 2024-07-31T14:52:30.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f607de51740 1 -- 172.21.15.67:0/735485313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 msgr2=0x5584a6e60680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:30.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f607de51740 1 --2- 172.21.15.67:0/735485313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e60680 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f60600096c0 tx=0x7f60600cb240 comp rx=0 tx=0).stop 2024-07-31T14:52:30.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f607de51740 1 -- 172.21.15.67:0/735485313 shutdown_connections 2024-07-31T14:52:30.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f607de51740 1 --2- 172.21.15.67:0/735485313 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584a6a6dcd0 0x5584a6e71de0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f607de51740 1 --2- 172.21.15.67:0/735485313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6aaf350 0x5584a6e6bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f607de51740 1 --2- 172.21.15.67:0/735485313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e60680 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f607de51740 1 --2- 172.21.15.67:0/735485313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 0x5584a6e60140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f6058ff9640 0 client.15987 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:30.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f607de51740 1 -- 172.21.15.67:0/735485313 >> 172.21.15.67:0/735485313 conn(0x5584a6aa3740 msgr2=0x5584a6e57710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:30.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f607de51740 1 -- 172.21.15.67:0/735485313 shutdown_connections 2024-07-31T14:52:30.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f607de51740 1 -- 172.21.15.67:0/735485313 wait complete. 2024-07-31T14:52:30.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.305+0000 7f607de51740 20 client.15987 trim_cache size 0 max 16384 2024-07-31T14:52:30.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.324+0000 7f607de51740 1 Processor -- start 2024-07-31T14:52:30.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.324+0000 7f607de51740 1 -- start start 2024-07-31T14:52:30.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.325+0000 7f607de51740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 0x5584a6e524d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.325+0000 7f607de51740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e61b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.325+0000 7f607de51740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6aaf350 0x5584a6e775a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.325+0000 7f607de51740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f60701b2840 con 0x5584a6a8e670 2024-07-31T14:52:30.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.325+0000 7f607de51740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f60701b4380 con 0x5584a6aaf350 2024-07-31T14:52:30.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.325+0000 7f607de51740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f60701b2f10 con 0x5584a6a86490 2024-07-31T14:52:30.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.325+0000 7f6074e9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e61b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.325+0000 7f6074e9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e61b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33864/0 (socket says 172.21.15.67:33864) 2024-07-31T14:52:30.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.325+0000 7f6074e9b640 1 -- 172.21.15.67:0/211810535 learned_addr learned my addr 172.21.15.67:0/211810535 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:30.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.325+0000 7f607569c640 1 --2- 172.21.15.67:0/211810535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 0x5584a6e524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.325+0000 7f6075e9d640 1 --2- 172.21.15.67:0/211810535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6aaf350 0x5584a6e775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.325+0000 7f6074e9b640 1 -- 172.21.15.67:0/211810535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 msgr2=0x5584a6e524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:30.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.326+0000 7f6074e9b640 1 --2- 172.21.15.67:0/211810535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 0x5584a6e524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.326+0000 7f6074e9b640 1 -- 172.21.15.67:0/211810535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6aaf350 msgr2=0x5584a6e775a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:30.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.326+0000 7f6074e9b640 1 --2- 172.21.15.67:0/211810535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6aaf350 0x5584a6e775a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.326+0000 7f6074e9b640 1 -- 172.21.15.67:0/211810535 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584a6e56e10 con 0x5584a6a8e670 2024-07-31T14:52:30.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.326+0000 7f6074e9b640 1 --2- 172.21.15.67:0/211810535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e61b70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f60600c8430 tx=0x7f606002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538ec2ca34f515f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:30.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.326+0000 7f605affd640 1 -- 172.21.15.67:0/211810535 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600e5040 con 0x5584a6a8e670 2024-07-31T14:52:30.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.326+0000 7f605affd640 1 -- 172.21.15.67:0/211810535 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60600cacd0 con 0x5584a6a8e670 2024-07-31T14:52:30.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.326+0000 7f605affd640 1 -- 172.21.15.67:0/211810535 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600d41c0 con 0x5584a6a8e670 2024-07-31T14:52:30.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.327+0000 7f607de51740 1 -- 172.21.15.67:0/211810535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 msgr2=0x5584a6e61b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:30.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.327+0000 7f607de51740 1 --2- 172.21.15.67:0/211810535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e61b70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f60600c8430 tx=0x7f606002a410 comp rx=0 tx=0).stop 2024-07-31T14:52:30.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.328+0000 7f607de51740 1 -- 172.21.15.67:0/211810535 shutdown_connections 2024-07-31T14:52:30.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.328+0000 7f607de51740 1 --2- 172.21.15.67:0/211810535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6aaf350 0x5584a6e775a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.328+0000 7f607de51740 1 --2- 172.21.15.67:0/211810535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6a8e670 0x5584a6e61b70 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.328+0000 7f607de51740 1 --2- 172.21.15.67:0/211810535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 0x5584a6e524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.328+0000 7f607de51740 1 -- 172.21.15.67:0/211810535 >> 172.21.15.67:0/211810535 conn(0x5584a6aa3740 msgr2=0x5584a6e61460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:30.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.328+0000 7f607de51740 1 -- 172.21.15.67:0/211810535 shutdown_connections 2024-07-31T14:52:30.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.328+0000 7f607de51740 1 -- 172.21.15.67:0/211810535 wait complete. 2024-07-31T14:52:30.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.328+0000 7f607de51740 1 -- start start 2024-07-31T14:52:30.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.329+0000 7f607de51740 1 Processor -- start 2024-07-31T14:52:30.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.329+0000 7f607de51740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 0x5584a6e680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.329+0000 7f607de51740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6a8e670 0x5584a6e635a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.329+0000 7f607de51740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6aaf350 0x5584a6e63b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.329+0000 7f607de51740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f60701b3cb0 con 0x5584a6aaf350 2024-07-31T14:52:30.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.329+0000 7f607de51740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f60701b35e0 con 0x5584a6a8e670 2024-07-31T14:52:30.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.329+0000 7f607de51740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f60701b0820 con 0x5584a6a86490 2024-07-31T14:52:30.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.330+0000 7f607de51740 10 client.0 _ll_register_callbacks cb 0x7f60600c77b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:30.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.330+0000 7f6075e9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6aaf350 0x5584a6e63b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.330+0000 7f607569c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 0x5584a6e680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.330+0000 7f607569c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 0x5584a6e680a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50504/0 (socket says 172.21.15.67:50504) 2024-07-31T14:52:30.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.330+0000 7f607569c640 1 -- 172.21.15.67:0/3693904186 learned_addr learned my addr 172.21.15.67:0/3693904186 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:30.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.330+0000 7f6074e9b640 1 --2- 172.21.15.67:0/3693904186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6a8e670 0x5584a6e635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.330+0000 7f6074e9b640 1 -- 172.21.15.67:0/3693904186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 msgr2=0x5584a6e680a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:30.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.330+0000 7f6074e9b640 1 --2- 172.21.15.67:0/3693904186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 0x5584a6e680a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.330+0000 7f6074e9b640 1 -- 172.21.15.67:0/3693904186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6aaf350 msgr2=0x5584a6e63b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:30.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.330+0000 7f6074e9b640 1 --2- 172.21.15.67:0/3693904186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6aaf350 0x5584a6e63b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.330+0000 7f6074e9b640 1 -- 172.21.15.67:0/3693904186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584a6e642f0 con 0x5584a6a8e670 2024-07-31T14:52:30.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.330+0000 7f6075e9d640 1 --2- 172.21.15.67:0/3693904186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6aaf350 0x5584a6e63b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:30.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.330+0000 7f6074e9b640 1 --2- 172.21.15.67:0/3693904186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6a8e670 0x5584a6e635a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6060006010 tx=0x7f6060102180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c99069bd2a0088 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:30.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.330+0000 7f6058ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:52:30.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.331+0000 7f6058ff9640 1 -- 172.21.15.67:0/3693904186 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600e5050 con 0x5584a6a8e670 2024-07-31T14:52:30.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.331+0000 7f607de51740 1 -- 172.21.15.67:0/3693904186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60600cacd0 con 0x5584a6a8e670 2024-07-31T14:52:30.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.331+0000 7f6058ff9640 1 -- 172.21.15.67:0/3693904186 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f60600d3d30 con 0x5584a6a8e670 2024-07-31T14:52:30.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.331+0000 7f6058ff9640 1 -- 172.21.15.67:0/3693904186 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f60600ca930 con 0x5584a6a8e670 2024-07-31T14:52:30.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.331+0000 7f607de51740 10 client.25658 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:30.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.331+0000 7f607de51740 1 -- 172.21.15.67:0/3693904186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5584a6e642f0 con 0x5584a6a8e670 2024-07-31T14:52:30.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.331+0000 7f6058ff9640 1 -- 172.21.15.67:0/3693904186 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60600cacd0 con 0x5584a6a8e670 2024-07-31T14:52:30.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.333+0000 7f6058ff9640 1 -- 172.21.15.67:0/3693904186 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6060003800 con 0x5584a6a8e670 2024-07-31T14:52:30.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.333+0000 7f607de51740 10 client.25658 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:30.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.333+0000 7f607de51740 10 client.25658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:30.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.333+0000 7f607de51740 20 client.25658 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:30.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.333+0000 7f607de51740 1 --2- 172.21.15.67:0/3693904186 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584a6a43080 0x5584a6edc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:30.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.333+0000 7f607de51740 4 client.25658 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T14:52:30.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.333+0000 7f607de51740 1 -- 172.21.15.67:0/3693904186 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5584a6dc8470 con 0x5584a6a43080 2024-07-31T14:52:30.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.333+0000 7f607569c640 1 --2- 172.21.15.67:0/3693904186 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584a6a43080 0x5584a6edc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:30.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.334+0000 7f6058ff9640 10 client.25658 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:30.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.334+0000 7f607569c640 1 --2- 172.21.15.67:0/3693904186 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584a6a43080 0x5584a6edc8d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2cf8d53e554a17b server_cookie=51dce0333094424f in_seq=0 out_seq=0 2024-07-31T14:52:30.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.335+0000 7f6058ff9640 1 -- 172.21.15.67:0/3693904186 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5584a6dc8470 con 0x5584a6a43080 2024-07-31T14:52:30.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.335+0000 7f6058ff9640 10 client.25658 handle_command_reply: tid=0 2024-07-31T14:52:30.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.335+0000 7f607de51740 1 client.25658 shutdown 2024-07-31T14:52:30.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.335+0000 7f607de51740 1 -- 172.21.15.67:0/3693904186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6a8e670 msgr2=0x5584a6e635a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:30.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.335+0000 7f607de51740 1 --2- 172.21.15.67:0/3693904186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6a8e670 0x5584a6e635a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6060006010 tx=0x7f6060102180 comp rx=0 tx=0).stop 2024-07-31T14:52:30.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.335+0000 7f607de51740 1 -- 172.21.15.67:0/3693904186 shutdown_connections 2024-07-31T14:52:30.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.336+0000 7f607de51740 1 --2- 172.21.15.67:0/3693904186 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584a6a43080 0x5584a6edc8d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.336+0000 7f607de51740 1 --2- 172.21.15.67:0/3693904186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584a6aaf350 0x5584a6e63b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.336+0000 7f6058ff9640 0 client.25658 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:30.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.336+0000 7f607de51740 1 --2- 172.21.15.67:0/3693904186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584a6a8e670 0x5584a6e635a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.336+0000 7f607de51740 1 --2- 172.21.15.67:0/3693904186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584a6a86490 0x5584a6e680a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.336+0000 7f607de51740 1 -- 172.21.15.67:0/3693904186 >> 172.21.15.67:0/3693904186 conn(0x5584a6aa3740 msgr2=0x5584a6e57c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:30.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.336+0000 7f607de51740 1 -- 172.21.15.67:0/3693904186 shutdown_connections 2024-07-31T14:52:30.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.336+0000 7f607de51740 1 -- 172.21.15.67:0/3693904186 wait complete. 2024-07-31T14:52:30.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.336+0000 7f607de51740 20 client.25658 trim_cache size 0 max 16384 2024-07-31T14:52:30.339 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:52:30.339 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T14:52:30.339 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "34cf1f4a-92fa-471c-b0af-f001b51307ea", 2024-07-31T14:52:30.339 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T14:52:30.339 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:52:30.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.337+0000 7f607669e640 1 -- 172.21.15.67:0/4007025848 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f605c0b3dc0 msgr2=0x7f605c0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:30.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.337+0000 7f607669e640 1 --2- 172.21.15.67:0/4007025848 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f605c0b3dc0 0x7f605c0b62d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6068009fd0 tx=0x7f6068004a40 comp rx=0 tx=0).stop 2024-07-31T14:52:30.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.337+0000 7f607669e640 1 -- 172.21.15.67:0/4007025848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6070107c10 msgr2=0x7f607019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:30.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.337+0000 7f607669e640 1 --2- 172.21.15.67:0/4007025848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6070107c10 0x7f607019a460 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f606002a930 tx=0x7f6060004270 comp rx=0 tx=0).stop 2024-07-31T14:52:30.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.337+0000 7f607669e640 1 -- 172.21.15.67:0/4007025848 shutdown_connections 2024-07-31T14:52:30.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.337+0000 7f607669e640 1 --2- 172.21.15.67:0/4007025848 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f605c0b3dc0 0x7f605c0b62d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.337+0000 7f607669e640 1 --2- 172.21.15.67:0/4007025848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6070108560 0x7f607019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.337+0000 7f607669e640 1 --2- 172.21.15.67:0/4007025848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6070107c10 0x7f607019a460 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.337+0000 7f607669e640 1 --2- 172.21.15.67:0/4007025848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60701022d0 0x7f6070199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:30.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.337+0000 7f607669e640 1 -- 172.21.15.67:0/4007025848 >> 172.21.15.67:0/4007025848 conn(0x7f60700fdd40 msgr2=0x7f607006b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:30.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.337+0000 7f607669e640 1 -- 172.21.15.67:0/4007025848 shutdown_connections 2024-07-31T14:52:30.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:30.338+0000 7f607669e640 1 -- 172.21.15.67:0/4007025848 wait complete. 2024-07-31T14:52:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:30 smithi067 ceph-mon[32750]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-07-31T14:52:31.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:30 smithi073 ceph-mon[36714]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-07-31T14:52:31.154 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:52:31.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:30 smithi077 ceph-mon[37327]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-07-31T14:52:31.461 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:31.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:31 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T14:52:31.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:52:31.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:52:31.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:52:31.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:31 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:52:32.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:31 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T14:52:32.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:52:32.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:52:32.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:52:32.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:31 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:52:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.234+0000 7fc2b13eb640 1 -- 172.21.15.67:0/914320229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac10f590 msgr2=0x7fc2ac10f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.234+0000 7fc2b13eb640 1 --2- 172.21.15.67:0/914320229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac10f590 0x7fc2ac10f9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009900 tx=0x7fc2a002a3f0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.235+0000 7fc2b13eb640 1 -- 172.21.15.67:0/914320229 shutdown_connections 2024-07-31T14:52:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.235+0000 7fc2b13eb640 1 --2- 172.21.15.67:0/914320229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2ac10fee0 0x7fc2ac116710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.235+0000 7fc2b13eb640 1 --2- 172.21.15.67:0/914320229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac10f590 0x7fc2ac10f9a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.235+0000 7fc2b13eb640 1 --2- 172.21.15.67:0/914320229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2ac109b50 0x7fc2ac109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.235+0000 7fc2b13eb640 1 -- 172.21.15.67:0/914320229 >> 172.21.15.67:0/914320229 conn(0x7fc2ac079200 msgr2=0x7fc2ac107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.235+0000 7fc2b13eb640 1 -- 172.21.15.67:0/914320229 shutdown_connections 2024-07-31T14:52:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.235+0000 7fc2b13eb640 1 -- 172.21.15.67:0/914320229 wait complete. 2024-07-31T14:52:32.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:31 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T14:52:32.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:52:32.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:52:32.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:52:32.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:31 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:52:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.236+0000 7fc2b13eb640 1 Processor -- start 2024-07-31T14:52:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.236+0000 7fc2b13eb640 1 -- start start 2024-07-31T14:52:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.236+0000 7fc2b13eb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2ac109b50 0x7fc2ac1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.236+0000 7fc2b13eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2ac10f590 0x7fc2ac1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.236+0000 7fc2b13eb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac10fee0 0x7fc2ac1a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.236+0000 7fc2b13eb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac119bc0 con 0x7fc2ac109b50 2024-07-31T14:52:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.236+0000 7fc2b13eb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac119a40 con 0x7fc2ac10fee0 2024-07-31T14:52:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.236+0000 7fc2b13eb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac119d40 con 0x7fc2ac10f590 2024-07-31T14:52:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.237+0000 7fc2aaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2ac109b50 0x7fc2ac1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.237+0000 7fc2aaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2ac109b50 0x7fc2ac1a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42314/0 (socket says 172.21.15.67:42314) 2024-07-31T14:52:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.237+0000 7fc2aaffd640 1 -- 172.21.15.67:0/2005200637 learned_addr learned my addr 172.21.15.67:0/2005200637 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.237+0000 7fc2aa7fc640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2ac10f590 0x7fc2ac1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.237+0000 7fc2ab7fe640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac10fee0 0x7fc2ac1a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.237+0000 7fc2aaffd640 1 -- 172.21.15.67:0/2005200637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2ac10f590 msgr2=0x7fc2ac1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.237+0000 7fc2aaffd640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2ac10f590 0x7fc2ac1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.237+0000 7fc2aaffd640 1 -- 172.21.15.67:0/2005200637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac10fee0 msgr2=0x7fc2ac1a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.237+0000 7fc2aaffd640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac10fee0 0x7fc2ac1a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.237+0000 7fc2aaffd640 1 -- 172.21.15.67:0/2005200637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac1a6480 con 0x7fc2ac109b50 2024-07-31T14:52:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.237+0000 7fc2ab7fe640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac10fee0 0x7fc2ac1a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.237+0000 7fc2aa7fc640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2ac10f590 0x7fc2ac1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.238+0000 7fc2aaffd640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2ac109b50 0x7fc2ac1a05e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00b700 tx=0x7fc29c00bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.238+0000 7fc293fff640 1 -- 172.21.15.67:0/2005200637 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c019070 con 0x7fc2ac109b50 2024-07-31T14:52:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.238+0000 7fc293fff640 1 -- 172.21.15.67:0/2005200637 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc29c00cc90 con 0x7fc2ac109b50 2024-07-31T14:52:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.238+0000 7fc2b13eb640 1 -- 172.21.15.67:0/2005200637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac1a6770 con 0x7fc2ac109b50 2024-07-31T14:52:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.238+0000 7fc2b13eb640 1 -- 172.21.15.67:0/2005200637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac1acf90 con 0x7fc2ac109b50 2024-07-31T14:52:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.238+0000 7fc293fff640 1 -- 172.21.15.67:0/2005200637 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c014df0 con 0x7fc2ac109b50 2024-07-31T14:52:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.242+0000 7fc2b83be740 1 Processor -- start 2024-07-31T14:52:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.243+0000 7fc2b83be740 1 -- start start 2024-07-31T14:52:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.243+0000 7fc2b83be740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 0x557066bb1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.243+0000 7fc2b83be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667e62f0 0x557066bb2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.243+0000 7fc2b83be740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 0x557066bb2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.243+0000 7fc2b83be740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557066b273b0 con 0x5570667ee4d0 2024-07-31T14:52:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.243+0000 7fc2b83be740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557066b168b0 con 0x55706680f1b0 2024-07-31T14:52:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.243+0000 7fc2b83be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557066b1a8f0 con 0x5570667e62f0 2024-07-31T14:52:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.243+0000 7fc2aa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 0x557066bb2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.243+0000 7fc2ab7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 0x557066bb1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.244+0000 7fc2ab7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 0x557066bb1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49692/0 (socket says 172.21.15.67:49692) 2024-07-31T14:52:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.244+0000 7fc2ab7fe640 1 -- 172.21.15.67:0/2874958997 learned_addr learned my addr 172.21.15.67:0/2874958997 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.244+0000 7fc2aa7fc640 1 -- 172.21.15.67:0/2874958997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667e62f0 msgr2=0x557066bb2220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:52:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.244+0000 7fc2aa7fc640 1 --2- 172.21.15.67:0/2874958997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667e62f0 0x557066bb2220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.244+0000 7fc2aa7fc640 1 -- 172.21.15.67:0/2874958997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 msgr2=0x557066bb1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.244+0000 7fc2aa7fc640 1 --2- 172.21.15.67:0/2874958997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 0x557066bb1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.244+0000 7fc2aa7fc640 1 -- 172.21.15.67:0/2874958997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557066abd9a0 con 0x5570667ee4d0 2024-07-31T14:52:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.244+0000 7fc293fff640 1 -- 172.21.15.67:0/2005200637 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc29c01c070 con 0x7fc2ac109b50 2024-07-31T14:52:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.245+0000 7fc2aa7fc640 1 --2- 172.21.15.67:0/2874958997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 0x557066bb2760 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc2a002a910 tx=0x7fc2a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e842436d0bac04e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.245+0000 7fc2917fa640 1 -- 172.21.15.67:0/2874958997 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0035040 con 0x5570667ee4d0 2024-07-31T14:52:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.245+0000 7fc293fff640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc2940b3e40 0x7fc2940b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.245+0000 7fc2917fa640 1 -- 172.21.15.67:0/2874958997 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a00318f0 con 0x5570667ee4d0 2024-07-31T14:52:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.245+0000 7fc2917fa640 1 -- 172.21.15.67:0/2874958997 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0004a90 con 0x5570667ee4d0 2024-07-31T14:52:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.245+0000 7fc2ab7fe640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc2940b3e40 0x7fc2940b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.246+0000 7fc293fff640 1 -- 172.21.15.67:0/2005200637 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc29c0e3010 con 0x7fc2ac109b50 2024-07-31T14:52:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.246+0000 7fc2b83be740 1 -- 172.21.15.67:0/2874958997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 msgr2=0x557066bb2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.246+0000 7fc2ab7fe640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc2940b3e40 0x7fc2940b6350 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4007fd0 tx=0x7fc2a4007520 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.246+0000 7fc2b83be740 1 --2- 172.21.15.67:0/2874958997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 0x557066bb2760 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc2a002a910 tx=0x7fc2a0000bc0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.247+0000 7fc2b83be740 1 -- 172.21.15.67:0/2874958997 shutdown_connections 2024-07-31T14:52:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.247+0000 7fc2b83be740 1 --2- 172.21.15.67:0/2874958997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 0x557066bb2760 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.247+0000 7fc2b83be740 1 --2- 172.21.15.67:0/2874958997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667e62f0 0x557066bb2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.247+0000 7fc2b83be740 1 --2- 172.21.15.67:0/2874958997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 0x557066bb1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.247+0000 7fc2b83be740 1 -- 172.21.15.67:0/2874958997 >> 172.21.15.67:0/2874958997 conn(0x557066803300 msgr2=0x557066bafbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.247+0000 7fc2b83be740 1 -- 172.21.15.67:0/2874958997 shutdown_connections 2024-07-31T14:52:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.247+0000 7fc2b83be740 1 -- 172.21.15.67:0/2874958997 wait complete. 2024-07-31T14:52:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.247+0000 7fc2b83be740 1 -- start start 2024-07-31T14:52:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2b83be740 1 Processor -- start 2024-07-31T14:52:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2b83be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667e62f0 0x557066bbd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2b83be740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 0x557066bbdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2b83be740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 0x557066bc97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2b83be740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557066a50ab0 con 0x5570667ee4d0 2024-07-31T14:52:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2b83be740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557066a9ecf0 con 0x55706680f1b0 2024-07-31T14:52:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2b83be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557066bc0400 con 0x5570667e62f0 2024-07-31T14:52:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2aaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667e62f0 0x557066bbd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2aaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667e62f0 0x557066bbd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:42474/0 (socket says 172.21.15.67:42474) 2024-07-31T14:52:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2aaffd640 1 -- 172.21.15.67:0/388376700 learned_addr learned my addr 172.21.15.67:0/388376700 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2b83be740 10 client.0 _ll_register_callbacks cb 0x557066b109f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2aa7fc640 1 --2- 172.21.15.67:0/388376700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 0x557066bbdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2ab7fe640 1 --2- 172.21.15.67:0/388376700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 0x557066bc97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2aaffd640 1 -- 172.21.15.67:0/388376700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 msgr2=0x557066bc97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.248+0000 7fc2aaffd640 1 --2- 172.21.15.67:0/388376700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 0x557066bc97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.249+0000 7fc2aaffd640 1 -- 172.21.15.67:0/388376700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 msgr2=0x557066bbdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.249+0000 7fc2aaffd640 1 --2- 172.21.15.67:0/388376700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 0x557066bbdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.249+0000 7fc2aaffd640 1 -- 172.21.15.67:0/388376700 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706687a790 con 0x5570667e62f0 2024-07-31T14:52:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.249+0000 7fc2aaffd640 1 --2- 172.21.15.67:0/388376700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667e62f0 0x557066bbd860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00bcf0 tx=0x7fc29c0a91e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0642b39bf16f57c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.249+0000 7fc28eff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:52:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.249+0000 7fc28eff5640 1 -- 172.21.15.67:0/388376700 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0a9a70 con 0x5570667e62f0 2024-07-31T14:52:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.249+0000 7fc28eff5640 1 -- 172.21.15.67:0/388376700 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc29c0e0670 con 0x5570667e62f0 2024-07-31T14:52:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.249+0000 7fc2b83be740 1 -- 172.21.15.67:0/388376700 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2a00318f0 con 0x5570667e62f0 2024-07-31T14:52:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.249+0000 7fc28eff5640 1 -- 172.21.15.67:0/388376700 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc29c0e0870 con 0x5570667e62f0 2024-07-31T14:52:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.249+0000 7fc2b83be740 10 client.25693 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.249+0000 7fc2b83be740 1 -- 172.21.15.67:0/388376700 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5570668f3620 con 0x5570667e62f0 2024-07-31T14:52:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.250+0000 7fc28eff5640 1 -- 172.21.15.67:0/388376700 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a00318f0 con 0x5570667e62f0 2024-07-31T14:52:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.250+0000 7fc28eff5640 1 -- 172.21.15.67:0/388376700 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc29c0e0b40 con 0x5570667e62f0 2024-07-31T14:52:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.250+0000 7fc2b83be740 10 client.25693 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.250+0000 7fc2b83be740 10 client.25693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.250+0000 7fc2b83be740 20 client.25693 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.250+0000 7fc2b83be740 1 --2- 172.21.15.67:0/388376700 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5570667cdd80 0x557066bcf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.250+0000 7fc2b83be740 4 client.25693 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.250+0000 7fc2b83be740 1 -- 172.21.15.67:0/388376700 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557066b29300 con 0x5570667cdd80 2024-07-31T14:52:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.250+0000 7fc2aa7fc640 1 --2- 172.21.15.67:0/388376700 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5570667cdd80 0x557066bcf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.251+0000 7fc28eff5640 10 client.25693 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.251+0000 7fc2aa7fc640 1 --2- 172.21.15.67:0/388376700 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5570667cdd80 0x557066bcf780 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aebe2146cdffb3f5 server_cookie=ae9bc96431192333 in_seq=0 out_seq=0 2024-07-31T14:52:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc28eff5640 1 -- 172.21.15.67:0/388376700 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557066b29300 con 0x5570667cdd80 2024-07-31T14:52:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc28eff5640 10 client.25693 handle_command_reply: tid=0 2024-07-31T14:52:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc2b83be740 1 client.25693 shutdown 2024-07-31T14:52:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc2b83be740 1 -- 172.21.15.67:0/388376700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667e62f0 msgr2=0x557066bbd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc2b83be740 1 --2- 172.21.15.67:0/388376700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667e62f0 0x557066bbd860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00bcf0 tx=0x7fc29c0a91e0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc2b83be740 1 -- 172.21.15.67:0/388376700 shutdown_connections 2024-07-31T14:52:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc2b83be740 1 --2- 172.21.15.67:0/388376700 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5570667cdd80 0x557066bcf780 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc2b83be740 1 --2- 172.21.15.67:0/388376700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 0x557066bc97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc28eff5640 0 client.25693 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc2b83be740 1 --2- 172.21.15.67:0/388376700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 0x557066bbdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc2b83be740 1 --2- 172.21.15.67:0/388376700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667e62f0 0x557066bbd860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc2b83be740 1 -- 172.21.15.67:0/388376700 >> 172.21.15.67:0/388376700 conn(0x557066803300 msgr2=0x557066bb2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc2b83be740 1 -- 172.21.15.67:0/388376700 shutdown_connections 2024-07-31T14:52:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc2b83be740 1 -- 172.21.15.67:0/388376700 wait complete. 2024-07-31T14:52:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.252+0000 7fc2b83be740 20 client.25693 trim_cache size 0 max 16384 2024-07-31T14:52:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.283+0000 7fc2b83be740 1 Processor -- start 2024-07-31T14:52:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.283+0000 7fc2b83be740 1 -- start start 2024-07-31T14:52:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2b83be740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5570667e62f0 0x557066bb74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2b83be740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 0x557066bd4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2b83be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55706680f1b0 0x557066bd5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2b83be740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1b6a90 con 0x5570667ee4d0 2024-07-31T14:52:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2b83be740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1b85d0 con 0x5570667e62f0 2024-07-31T14:52:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2b83be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1b7160 con 0x55706680f1b0 2024-07-31T14:52:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2aa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 0x557066bd4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2ab7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55706680f1b0 0x557066bd5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2aaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5570667e62f0 0x557066bb74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2ab7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55706680f1b0 0x557066bd5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:42478/0 (socket says 172.21.15.67:42478) 2024-07-31T14:52:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2aaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5570667e62f0 0x557066bb74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49720/0 (socket says 172.21.15.67:49720) 2024-07-31T14:52:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2ab7fe640 1 -- 172.21.15.67:0/4204921703 learned_addr learned my addr 172.21.15.67:0/4204921703 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2aaffd640 1 -- 172.21.15.67:0/4204921703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55706680f1b0 msgr2=0x557066bd5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2aaffd640 1 --2- 172.21.15.67:0/4204921703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55706680f1b0 0x557066bd5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.284+0000 7fc2aaffd640 1 -- 172.21.15.67:0/4204921703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 msgr2=0x557066bd4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.285+0000 7fc2aaffd640 1 --2- 172.21.15.67:0/4204921703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 0x557066bd4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.285+0000 7fc2aaffd640 1 -- 172.21.15.67:0/4204921703 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557066bb0570 con 0x5570667e62f0 2024-07-31T14:52:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.285+0000 7fc2aa7fc640 1 --2- 172.21.15.67:0/4204921703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 0x557066bd4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.285+0000 7fc2aaffd640 1 --2- 172.21.15.67:0/4204921703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5570667e62f0 0x557066bb74b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0a6010 tx=0x7fc29c014810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30e71788e3d89e10 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.285+0000 7fc290ff9640 1 -- 172.21.15.67:0/4204921703 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0a5200 con 0x5570667e62f0 2024-07-31T14:52:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.285+0000 7fc290ff9640 1 -- 172.21.15.67:0/4204921703 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc29c0adfc0 con 0x5570667e62f0 2024-07-31T14:52:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.285+0000 7fc290ff9640 1 -- 172.21.15.67:0/4204921703 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0b25e0 con 0x5570667e62f0 2024-07-31T14:52:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.286+0000 7fc2b83be740 1 -- 172.21.15.67:0/4204921703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5570667e62f0 msgr2=0x557066bb74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.286+0000 7fc2b83be740 1 --2- 172.21.15.67:0/4204921703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5570667e62f0 0x557066bb74b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0a6010 tx=0x7fc29c014810 comp rx=0 tx=0).stop 2024-07-31T14:52:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.286+0000 7fc2b83be740 1 -- 172.21.15.67:0/4204921703 shutdown_connections 2024-07-31T14:52:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.286+0000 7fc2b83be740 1 --2- 172.21.15.67:0/4204921703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55706680f1b0 0x557066bd5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.286+0000 7fc2b83be740 1 --2- 172.21.15.67:0/4204921703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667ee4d0 0x557066bd4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.286+0000 7fc2b83be740 1 --2- 172.21.15.67:0/4204921703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5570667e62f0 0x557066bb74b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.286+0000 7fc2b83be740 1 -- 172.21.15.67:0/4204921703 >> 172.21.15.67:0/4204921703 conn(0x557066803300 msgr2=0x557066bb6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.286+0000 7fc2b83be740 1 -- 172.21.15.67:0/4204921703 shutdown_connections 2024-07-31T14:52:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.286+0000 7fc2b83be740 1 -- 172.21.15.67:0/4204921703 wait complete. 2024-07-31T14:52:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.287+0000 7fc2b83be740 1 -- start start 2024-07-31T14:52:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.287+0000 7fc2b83be740 1 Processor -- start 2024-07-31T14:52:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.287+0000 7fc2b83be740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667e62f0 0x557066bc5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.287+0000 7fc2b83be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667ee4d0 0x557066bc5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.287+0000 7fc2b83be740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 0x557066c303f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.287+0000 7fc2b83be740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1b7f00 con 0x5570667e62f0 2024-07-31T14:52:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.287+0000 7fc2b83be740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1b7830 con 0x55706680f1b0 2024-07-31T14:52:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.287+0000 7fc2b83be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1b4a70 con 0x5570667ee4d0 2024-07-31T14:52:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.287+0000 7fc2b83be740 10 client.0 _ll_register_callbacks cb 0x7fc2a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.287+0000 7fc2aaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667e62f0 0x557066bc5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.287+0000 7fc2aaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667e62f0 0x557066bc5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42346/0 (socket says 172.21.15.67:42346) 2024-07-31T14:52:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.287+0000 7fc2aaffd640 1 -- 172.21.15.67:0/2481117220 learned_addr learned my addr 172.21.15.67:0/2481117220 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc2aa7fc640 1 --2- 172.21.15.67:0/2481117220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667ee4d0 0x557066bc5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc2aaffd640 1 -- 172.21.15.67:0/2481117220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667ee4d0 msgr2=0x557066bc5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc2aaffd640 1 --2- 172.21.15.67:0/2481117220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667ee4d0 0x557066bc5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc2aaffd640 1 -- 172.21.15.67:0/2481117220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 msgr2=0x557066c303f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc2ab7fe640 1 --2- 172.21.15.67:0/2481117220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 0x557066c303f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc2aaffd640 1 --2- 172.21.15.67:0/2481117220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 0x557066c303f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc2aaffd640 1 -- 172.21.15.67:0/2481117220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557066bb4210 con 0x5570667e62f0 2024-07-31T14:52:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc2ab7fe640 1 --2- 172.21.15.67:0/2481117220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 0x557066c303f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc2aaffd640 1 --2- 172.21.15.67:0/2481117220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667e62f0 0x557066bc5470 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0a99b0 tx=0x7fc29c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c1f9f8daff6f7a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc28eff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc28eff5640 1 -- 172.21.15.67:0/2481117220 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0c9060 con 0x5570667e62f0 2024-07-31T14:52:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc28eff5640 1 -- 172.21.15.67:0/2481117220 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fc29c0a8e60 con 0x5570667e62f0 2024-07-31T14:52:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc2b83be740 1 -- 172.21.15.67:0/2481117220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc29c0adfc0 con 0x5570667e62f0 2024-07-31T14:52:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc28eff5640 1 -- 172.21.15.67:0/2481117220 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc29c0b2a70 con 0x5570667e62f0 2024-07-31T14:52:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc2b83be740 10 client.16008 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.288+0000 7fc2b83be740 1 -- 172.21.15.67:0/2481117220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557066c37c60 con 0x5570667e62f0 2024-07-31T14:52:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.289+0000 7fc28eff5640 1 -- 172.21.15.67:0/2481117220 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc29c0b25e0 con 0x5570667e62f0 2024-07-31T14:52:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.289+0000 7fc28eff5640 1 -- 172.21.15.67:0/2481117220 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc29c0a5470 con 0x5570667e62f0 2024-07-31T14:52:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.289+0000 7fc2b83be740 10 client.16008 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.289+0000 7fc2b83be740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.289+0000 7fc2b83be740 20 client.16008 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.289+0000 7fc2b83be740 1 --2- 172.21.15.67:0/2481117220 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5570667a31c0 0x557066c3a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.289+0000 7fc2b83be740 4 client.16008 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:52:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.289+0000 7fc2b83be740 1 -- 172.21.15.67:0/2481117220 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc29c0adfc0 con 0x5570667a31c0 2024-07-31T14:52:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.289+0000 7fc2aa7fc640 1 --2- 172.21.15.67:0/2481117220 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5570667a31c0 0x557066c3a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.290+0000 7fc28eff5640 10 client.16008 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.290+0000 7fc2aa7fc640 1 --2- 172.21.15.67:0/2481117220 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5570667a31c0 0x557066c3a1f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ee8d374738314b7 server_cookie=aaa6feb5489efb8e in_seq=0 out_seq=0 2024-07-31T14:52:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.295+0000 7fc28eff5640 1 -- 172.21.15.67:0/2481117220 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc29c0adfc0 con 0x5570667a31c0 2024-07-31T14:52:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.295+0000 7fc28eff5640 10 client.16008 handle_command_reply: tid=0 2024-07-31T14:52:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.295+0000 7fc2b83be740 1 client.16008 shutdown 2024-07-31T14:52:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.296+0000 7fc2b83be740 1 -- 172.21.15.67:0/2481117220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667e62f0 msgr2=0x557066bc5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.296+0000 7fc2b83be740 1 --2- 172.21.15.67:0/2481117220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667e62f0 0x557066bc5470 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0a99b0 tx=0x7fc29c000bc0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.296+0000 7fc2b83be740 1 -- 172.21.15.67:0/2481117220 shutdown_connections 2024-07-31T14:52:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.296+0000 7fc2b83be740 1 --2- 172.21.15.67:0/2481117220 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5570667a31c0 0x557066c3a1f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.296+0000 7fc2b83be740 1 --2- 172.21.15.67:0/2481117220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55706680f1b0 0x557066c303f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.296+0000 7fc2b83be740 1 --2- 172.21.15.67:0/2481117220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570667ee4d0 0x557066bc5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.296+0000 7fc28eff5640 0 client.16008 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.296+0000 7fc2b83be740 1 --2- 172.21.15.67:0/2481117220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570667e62f0 0x557066bc5470 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.296+0000 7fc2b83be740 1 -- 172.21.15.67:0/2481117220 >> 172.21.15.67:0/2481117220 conn(0x557066803300 msgr2=0x557066bbd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.296+0000 7fc2b83be740 1 -- 172.21.15.67:0/2481117220 shutdown_connections 2024-07-31T14:52:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.296+0000 7fc2b83be740 1 -- 172.21.15.67:0/2481117220 wait complete. 2024-07-31T14:52:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.296+0000 7fc2b83be740 20 client.16008 trim_cache size 0 max 16384 2024-07-31T14:52:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:52:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:52:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:52:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:52:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.297+0000 7fc2b13eb640 1 -- 172.21.15.67:0/2005200637 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc2940b3e40 msgr2=0x7fc2940b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.297+0000 7fc2b13eb640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc2940b3e40 0x7fc2940b6350 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4007fd0 tx=0x7fc2a4007520 comp rx=0 tx=0).stop 2024-07-31T14:52:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.297+0000 7fc2b13eb640 1 -- 172.21.15.67:0/2005200637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2ac109b50 msgr2=0x7fc2ac1a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.297+0000 7fc2b13eb640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2ac109b50 0x7fc2ac1a05e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00b700 tx=0x7fc29c00bbc0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.297+0000 7fc2b13eb640 1 -- 172.21.15.67:0/2005200637 shutdown_connections 2024-07-31T14:52:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.297+0000 7fc2b13eb640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc2940b3e40 0x7fc2940b6350 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.297+0000 7fc2b13eb640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2ac10fee0 0x7fc2ac1a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.297+0000 7fc2b13eb640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2ac10f590 0x7fc2ac1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.297+0000 7fc2b13eb640 1 --2- 172.21.15.67:0/2005200637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2ac109b50 0x7fc2ac1a05e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.297+0000 7fc2b13eb640 1 -- 172.21.15.67:0/2005200637 >> 172.21.15.67:0/2005200637 conn(0x7fc2ac079200 msgr2=0x7fc2ac079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.297+0000 7fc2b13eb640 1 -- 172.21.15.67:0/2005200637 shutdown_connections 2024-07-31T14:52:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:32.297+0000 7fc2b13eb640 1 -- 172.21.15.67:0/2005200637 wait complete. 2024-07-31T14:52:33.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:32 smithi067 ceph-mon[32750]: pgmap v443: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-07-31T14:52:33.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:32 smithi073 ceph-mon[36714]: pgmap v443: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-07-31T14:52:33.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:32 smithi077 ceph-mon[37327]: pgmap v443: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-07-31T14:52:35.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:34 smithi067 ceph-mon[32750]: pgmap v444: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 13 op/s 2024-07-31T14:52:35.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:34 smithi073 ceph-mon[36714]: pgmap v444: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 13 op/s 2024-07-31T14:52:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:34 smithi077 ceph-mon[37327]: pgmap v444: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 13 op/s 2024-07-31T14:52:36.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.522+0000 7ffab764a640 1 -- 172.21.15.67:0/1404040679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffab01024d0 msgr2=0x7ffab01028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:36.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.522+0000 7ffab764a640 1 --2- 172.21.15.67:0/1404040679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffab01024d0 0x7ffab01028e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009ca0 tx=0x7ffaac02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.523+0000 7ffab764a640 1 -- 172.21.15.67:0/1404040679 shutdown_connections 2024-07-31T14:52:36.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.523+0000 7ffab764a640 1 --2- 172.21.15.67:0/1404040679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab0108760 0x7ffab010ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.523+0000 7ffab764a640 1 --2- 172.21.15.67:0/1404040679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffab0107e10 0x7ffab0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.523+0000 7ffab764a640 1 --2- 172.21.15.67:0/1404040679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffab01024d0 0x7ffab01028e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.523+0000 7ffab764a640 1 -- 172.21.15.67:0/1404040679 >> 172.21.15.67:0/1404040679 conn(0x7ffab00791e0 msgr2=0x7ffab00795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:36.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.523+0000 7ffab764a640 1 -- 172.21.15.67:0/1404040679 shutdown_connections 2024-07-31T14:52:36.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.523+0000 7ffab764a640 1 -- 172.21.15.67:0/1404040679 wait complete. 2024-07-31T14:52:36.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.524+0000 7ffab764a640 1 Processor -- start 2024-07-31T14:52:36.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.524+0000 7ffab764a640 1 -- start start 2024-07-31T14:52:36.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.524+0000 7ffab764a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffab01024d0 0x7ffab0199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.524+0000 7ffab764a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffab0107e10 0x7ffab019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.524+0000 7ffab764a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab0108760 0x7ffab019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.525+0000 7ffab764a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffab0112460 con 0x7ffab0107e10 2024-07-31T14:52:36.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.525+0000 7ffab764a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffab01122e0 con 0x7ffab01024d0 2024-07-31T14:52:36.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.525+0000 7ffab764a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffab01125e0 con 0x7ffab0108760 2024-07-31T14:52:36.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.525+0000 7ffab53e6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffab01024d0 0x7ffab0199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.525+0000 7ffab53e6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffab01024d0 0x7ffab0199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49742/0 (socket says 172.21.15.67:49742) 2024-07-31T14:52:36.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.525+0000 7ffab53e6640 1 -- 172.21.15.67:0/733346273 learned_addr learned my addr 172.21.15.67:0/733346273 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:36.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.525+0000 7ffab4be5640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffab0107e10 0x7ffab019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.525+0000 7ffab5be7640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab0108760 0x7ffab019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.525+0000 7ffab53e6640 1 -- 172.21.15.67:0/733346273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab0108760 msgr2=0x7ffab019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:36.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.525+0000 7ffab53e6640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab0108760 0x7ffab019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.525+0000 7ffab53e6640 1 -- 172.21.15.67:0/733346273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffab0107e10 msgr2=0x7ffab019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:36.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.525+0000 7ffab53e6640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffab0107e10 0x7ffab019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.525+0000 7ffab53e6640 1 -- 172.21.15.67:0/733346273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab019fe70 con 0x7ffab01024d0 2024-07-31T14:52:36.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.525+0000 7ffab5be7640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab0108760 0x7ffab019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:36.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.526+0000 7ffab53e6640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffab01024d0 0x7ffab0199f10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffaac005e30 tx=0x7ffaac031680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:36.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.526+0000 7ffaa67fc640 1 -- 172.21.15.67:0/733346273 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac051070 con 0x7ffab01024d0 2024-07-31T14:52:36.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.526+0000 7ffab764a640 1 -- 172.21.15.67:0/733346273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab01a0100 con 0x7ffab01024d0 2024-07-31T14:52:36.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.526+0000 7ffaa67fc640 1 -- 172.21.15.67:0/733346273 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaac003e20 con 0x7ffab01024d0 2024-07-31T14:52:36.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.526+0000 7ffaa67fc640 1 -- 172.21.15.67:0/733346273 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac004690 con 0x7ffab01024d0 2024-07-31T14:52:36.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.526+0000 7ffab764a640 1 -- 172.21.15.67:0/733346273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab0112c20 con 0x7ffab01024d0 2024-07-31T14:52:36.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.530+0000 7ffaa67fc640 1 -- 172.21.15.67:0/733346273 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ffaac054070 con 0x7ffab01024d0 2024-07-31T14:52:36.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.530+0000 7ffabdb9b740 1 Processor -- start 2024-07-31T14:52:36.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.530+0000 7ffabdb9b740 1 -- start start 2024-07-31T14:52:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffabdb9b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921fa2070 0x556922347550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffabdb9b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x556922347a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffabdb9b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f81390 0x556922347fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffabdb9b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5569222bdb90 con 0x556921fa2070 2024-07-31T14:52:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffabdb9b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5569222ad090 con 0x556921f791b0 2024-07-31T14:52:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffabdb9b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5569222b10d0 con 0x556921f81390 2024-07-31T14:52:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffaa67fc640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffa9c0b3dc0 0x7ffa9c0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffab53e6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x556922347a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffab53e6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x556922347a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49748/0 (socket says 172.21.15.67:49748) 2024-07-31T14:52:36.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffab53e6640 1 -- 172.21.15.67:0/2364865988 learned_addr learned my addr 172.21.15.67:0/2364865988 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:36.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffab4be5640 1 --2- 172.21.15.67:0/2364865988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f81390 0x556922347fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffab5be7640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffa9c0b3dc0 0x7ffa9c0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffab53e6640 1 -- 172.21.15.67:0/2364865988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f81390 msgr2=0x556922347fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:36.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffab53e6640 1 --2- 172.21.15.67:0/2364865988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f81390 0x556922347fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffab53e6640 1 -- 172.21.15.67:0/2364865988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921fa2070 msgr2=0x556922347550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:36.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffab53e6640 1 --2- 172.21.15.67:0/2364865988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921fa2070 0x556922347550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.531+0000 7ffab53e6640 1 -- 172.21.15.67:0/2364865988 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556922253b70 con 0x556921f791b0 2024-07-31T14:52:36.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.532+0000 7ffab53e6640 1 --2- 172.21.15.67:0/2364865988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x556922347a90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0d0b00 tx=0x7ffaac0dc890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a09716b0bc4b926f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:36.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.532+0000 7ffa9b7fe640 1 -- 172.21.15.67:0/2364865988 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0d92f0 con 0x556921f791b0 2024-07-31T14:52:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.532+0000 7ffa9b7fe640 1 -- 172.21.15.67:0/2364865988 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaac036030 con 0x556921f791b0 2024-07-31T14:52:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.532+0000 7ffa9b7fe640 1 -- 172.21.15.67:0/2364865988 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0d8260 con 0x556921f791b0 2024-07-31T14:52:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.532+0000 7ffaa67fc640 1 -- 172.21.15.67:0/733346273 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ffaac10cdd0 con 0x7ffab01024d0 2024-07-31T14:52:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.533+0000 7ffabdb9b740 1 -- 172.21.15.67:0/2364865988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 msgr2=0x556922347a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.533+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/2364865988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x556922347a90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0d0b00 tx=0x7ffaac0dc890 comp rx=0 tx=0).stop 2024-07-31T14:52:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.533+0000 7ffab5be7640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffa9c0b3dc0 0x7ffa9c0b62d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8005fd0 tx=0x7ffaa800bbf0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.534+0000 7ffabdb9b740 1 -- 172.21.15.67:0/2364865988 shutdown_connections 2024-07-31T14:52:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.534+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/2364865988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f81390 0x556922347fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.534+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/2364865988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x556922347a90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.534+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/2364865988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921fa2070 0x556922347550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.534+0000 7ffabdb9b740 1 -- 172.21.15.67:0/2364865988 >> 172.21.15.67:0/2364865988 conn(0x556921f96460 msgr2=0x556922345440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.534+0000 7ffabdb9b740 1 -- 172.21.15.67:0/2364865988 shutdown_connections 2024-07-31T14:52:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.534+0000 7ffabdb9b740 1 -- 172.21.15.67:0/2364865988 wait complete. 2024-07-31T14:52:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.534+0000 7ffabdb9b740 1 -- start start 2024-07-31T14:52:36.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.534+0000 7ffabdb9b740 1 Processor -- start 2024-07-31T14:52:36.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.535+0000 7ffabdb9b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x556922353090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.535+0000 7ffabdb9b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f81390 0x5569223535d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.535+0000 7ffabdb9b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921fa2070 0x55692235ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.535+0000 7ffab53e6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x556922353090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.535+0000 7ffabdb9b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556922110700 con 0x556921fa2070 2024-07-31T14:52:36.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.535+0000 7ffabdb9b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55692224f9c0 con 0x556921f791b0 2024-07-31T14:52:36.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.535+0000 7ffab53e6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x556922353090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49750/0 (socket says 172.21.15.67:49750) 2024-07-31T14:52:36.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.535+0000 7ffabdb9b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5569222ad090 con 0x556921f81390 2024-07-31T14:52:36.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.535+0000 7ffab53e6640 1 -- 172.21.15.67:0/643396704 learned_addr learned my addr 172.21.15.67:0/643396704 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:36.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.535+0000 7ffab5be7640 1 --2- 172.21.15.67:0/643396704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921fa2070 0x55692235ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.535+0000 7ffabdb9b740 10 client.0 _ll_register_callbacks cb 0x55692226a4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:36.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.536+0000 7ffab53e6640 1 -- 172.21.15.67:0/643396704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f81390 msgr2=0x5569223535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:36.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.536+0000 7ffab53e6640 1 --2- 172.21.15.67:0/643396704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f81390 0x5569223535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.536+0000 7ffab53e6640 1 -- 172.21.15.67:0/643396704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921fa2070 msgr2=0x55692235ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:36.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.536+0000 7ffab53e6640 1 --2- 172.21.15.67:0/643396704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921fa2070 0x55692235ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.536+0000 7ffab53e6640 1 -- 172.21.15.67:0/643396704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692200d650 con 0x556921f791b0 2024-07-31T14:52:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.536+0000 7ffab53e6640 1 --2- 172.21.15.67:0/643396704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x556922353090 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffaac036e30 tx=0x7ffaac10a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8c41886613f48ce server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.536+0000 7ffa98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:52:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.536+0000 7ffa98ff9640 1 -- 172.21.15.67:0/643396704 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0d8480 con 0x556921f791b0 2024-07-31T14:52:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.536+0000 7ffa98ff9640 1 -- 172.21.15.67:0/643396704 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ffaac0f6110 con 0x556921f791b0 2024-07-31T14:52:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.536+0000 7ffabdb9b740 1 -- 172.21.15.67:0/643396704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaac036030 con 0x556921f791b0 2024-07-31T14:52:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.537+0000 7ffa98ff9640 1 -- 172.21.15.67:0/643396704 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffaac0d86a0 con 0x556921f791b0 2024-07-31T14:52:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.537+0000 7ffabdb9b740 10 client.25685 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.537+0000 7ffabdb9b740 1 -- 172.21.15.67:0/643396704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556922361820 con 0x556921f791b0 2024-07-31T14:52:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.537+0000 7ffa98ff9640 1 -- 172.21.15.67:0/643396704 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaac004040 con 0x556921f791b0 2024-07-31T14:52:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.537+0000 7ffa98ff9640 1 -- 172.21.15.67:0/643396704 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ffaac0f4ce0 con 0x556921f791b0 2024-07-31T14:52:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.537+0000 7ffabdb9b740 10 client.25685 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.538+0000 7ffabdb9b740 10 client.25685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.538+0000 7ffabdb9b740 20 client.25685 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.538+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/643396704 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556921f609f0 0x556922364d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.538+0000 7ffabdb9b740 4 client.25685 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.538+0000 7ffabdb9b740 1 -- 172.21.15.67:0/643396704 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffaac036030 con 0x556921f609f0 2024-07-31T14:52:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.538+0000 7ffab4be5640 1 --2- 172.21.15.67:0/643396704 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556921f609f0 0x556922364d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.538+0000 7ffab4be5640 1 --2- 172.21.15.67:0/643396704 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556921f609f0 0x556922364d60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b2cb7159c9e406c server_cookie=51e3570c0143a810 in_seq=0 out_seq=0 2024-07-31T14:52:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.539+0000 7ffa98ff9640 10 client.25685 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.539+0000 7ffa98ff9640 1 -- 172.21.15.67:0/643396704 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ffaac036030 con 0x556921f609f0 2024-07-31T14:52:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.539+0000 7ffa98ff9640 10 client.25685 handle_command_reply: tid=0 2024-07-31T14:52:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.539+0000 7ffabdb9b740 1 client.25685 shutdown 2024-07-31T14:52:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.540+0000 7ffabdb9b740 1 -- 172.21.15.67:0/643396704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 msgr2=0x556922353090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.540+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/643396704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x556922353090 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffaac036e30 tx=0x7ffaac10a6c0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.540+0000 7ffabdb9b740 1 -- 172.21.15.67:0/643396704 shutdown_connections 2024-07-31T14:52:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.540+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/643396704 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556921f609f0 0x556922364d60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.540+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/643396704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921fa2070 0x55692235ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.540+0000 7ffa98ff9640 0 client.25685 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.540+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/643396704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f81390 0x5569223535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.540+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/643396704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x556922353090 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.540+0000 7ffabdb9b740 1 -- 172.21.15.67:0/643396704 >> 172.21.15.67:0/643396704 conn(0x556921f96460 msgr2=0x55692234a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.540+0000 7ffabdb9b740 1 -- 172.21.15.67:0/643396704 shutdown_connections 2024-07-31T14:52:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.540+0000 7ffabdb9b740 1 -- 172.21.15.67:0/643396704 wait complete. 2024-07-31T14:52:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.541+0000 7ffabdb9b740 20 client.25685 trim_cache size 0 max 16384 2024-07-31T14:52:36.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.578+0000 7ffabdb9b740 1 Processor -- start 2024-07-31T14:52:36.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.578+0000 7ffabdb9b740 1 -- start start 2024-07-31T14:52:36.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.578+0000 7ffabdb9b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f609f0 0x5569223454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.578+0000 7ffabdb9b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x55692235e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.578+0000 7ffabdb9b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921f81390 0x55692235ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.578+0000 7ffabdb9b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffab01ae4b0 con 0x556921f81390 2024-07-31T14:52:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.578+0000 7ffabdb9b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffab01afff0 con 0x556921f791b0 2024-07-31T14:52:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.578+0000 7ffabdb9b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffab01aeb80 con 0x556921f609f0 2024-07-31T14:52:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.578+0000 7ffab5be7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921f81390 0x55692235ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.578+0000 7ffab5be7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921f81390 0x55692235ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42392/0 (socket says 172.21.15.67:42392) 2024-07-31T14:52:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.578+0000 7ffab5be7640 1 -- 172.21.15.67:0/149329797 learned_addr learned my addr 172.21.15.67:0/149329797 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.579+0000 7ffab53e6640 1 --2- 172.21.15.67:0/149329797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f609f0 0x5569223454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.579+0000 7ffab5be7640 1 -- 172.21.15.67:0/149329797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f609f0 msgr2=0x5569223454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.579+0000 7ffab5be7640 1 --2- 172.21.15.67:0/149329797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f609f0 0x5569223454b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.579+0000 7ffab5be7640 1 -- 172.21.15.67:0/149329797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 msgr2=0x55692235e9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:52:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.579+0000 7ffab5be7640 1 --2- 172.21.15.67:0/149329797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x55692235e9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.579+0000 7ffab5be7640 1 -- 172.21.15.67:0/149329797 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55692234e720 con 0x556921f81390 2024-07-31T14:52:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.579+0000 7ffab5be7640 1 --2- 172.21.15.67:0/149329797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921f81390 0x55692235ef00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8010610 tx=0x7ffaa8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0102f6a8cf4ec41 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:36.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.579+0000 7ffa9affd640 1 -- 172.21.15.67:0/149329797 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa801d780 con 0x556921f81390 2024-07-31T14:52:36.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.580+0000 7ffa9affd640 1 -- 172.21.15.67:0/149329797 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaa801ddd0 con 0x556921f81390 2024-07-31T14:52:36.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.580+0000 7ffa9affd640 1 -- 172.21.15.67:0/149329797 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8027d00 con 0x556921f81390 2024-07-31T14:52:36.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.581+0000 7ffabdb9b740 1 -- 172.21.15.67:0/149329797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921f81390 msgr2=0x55692235ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:36.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.581+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/149329797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921f81390 0x55692235ef00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8010610 tx=0x7ffaa8015000 comp rx=0 tx=0).stop 2024-07-31T14:52:36.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.581+0000 7ffabdb9b740 1 -- 172.21.15.67:0/149329797 shutdown_connections 2024-07-31T14:52:36.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.581+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/149329797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921f81390 0x55692235ef00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.581+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/149329797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f791b0 0x55692235e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.581+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/149329797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f609f0 0x5569223454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.581+0000 7ffabdb9b740 1 -- 172.21.15.67:0/149329797 >> 172.21.15.67:0/149329797 conn(0x556921f96460 msgr2=0x556922344da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:36.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.581+0000 7ffabdb9b740 1 -- 172.21.15.67:0/149329797 shutdown_connections 2024-07-31T14:52:36.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.581+0000 7ffabdb9b740 1 -- 172.21.15.67:0/149329797 wait complete. 2024-07-31T14:52:36.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.582+0000 7ffabdb9b740 1 -- start start 2024-07-31T14:52:36.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.582+0000 7ffabdb9b740 1 Processor -- start 2024-07-31T14:52:36.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.582+0000 7ffabdb9b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f609f0 0x5569223569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.582+0000 7ffabdb9b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f791b0 0x556922356f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffabdb9b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921f81390 0x556922357440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffab53e6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f609f0 0x5569223569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffab53e6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f609f0 0x5569223569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49776/0 (socket says 172.21.15.67:49776) 2024-07-31T14:52:36.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffab4be5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f791b0 0x556922356f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffab4be5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f791b0 0x556922356f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:42528/0 (socket says 172.21.15.67:42528) 2024-07-31T14:52:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffab5be7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921f81390 0x556922357440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffab5be7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921f81390 0x556922357440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42400/0 (socket says 172.21.15.67:42400) 2024-07-31T14:52:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffab53e6640 1 -- 172.21.15.67:0/3470558848 learned_addr learned my addr 172.21.15.67:0/3470558848 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffabdb9b740 1 -- 172.21.15.67:0/3470558848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffab01af920 con 0x556921f81390 2024-07-31T14:52:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffabdb9b740 1 -- 172.21.15.67:0/3470558848 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffab01af250 con 0x556921f609f0 2024-07-31T14:52:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffabdb9b740 1 -- 172.21.15.67:0/3470558848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffab01ac490 con 0x556921f791b0 2024-07-31T14:52:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffab4be5640 1 -- 172.21.15.67:0/3470558848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f609f0 msgr2=0x5569223569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffab4be5640 1 --2- 172.21.15.67:0/3470558848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f609f0 0x5569223569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffab4be5640 1 -- 172.21.15.67:0/3470558848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921f81390 msgr2=0x556922357440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffab4be5640 1 --2- 172.21.15.67:0/3470558848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921f81390 0x556922357440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffab4be5640 1 -- 172.21.15.67:0/3470558848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569223cc4f0 con 0x556921f791b0 2024-07-31T14:52:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.583+0000 7ffab5be7640 1 --2- 172.21.15.67:0/3470558848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921f81390 0x556922357440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.584+0000 7ffabdb9b740 10 client.0 _ll_register_callbacks cb 0x7ffaa000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.584+0000 7ffabdb9b740 1 -- 172.21.15.67:0/3470558848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaa801ddd0 con 0x556921f791b0 2024-07-31T14:52:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.584+0000 7ffab4be5640 1 --2- 172.21.15.67:0/3470558848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f791b0 0x556922356f00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x556922365490 tx=0x7ffaa000bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88db8bae67f303de server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.584+0000 7ffa98ff9640 10 client.25711 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:52:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.584+0000 7ffa98ff9640 1 -- 172.21.15.67:0/3470558848 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0003d90 con 0x556921f791b0 2024-07-31T14:52:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.584+0000 7ffa98ff9640 1 -- 172.21.15.67:0/3470558848 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ffaa0003f30 con 0x556921f791b0 2024-07-31T14:52:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.584+0000 7ffa98ff9640 1 -- 172.21.15.67:0/3470558848 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffaa0011440 con 0x556921f791b0 2024-07-31T14:52:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.584+0000 7ffabdb9b740 10 client.25711 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.584+0000 7ffabdb9b740 1 -- 172.21.15.67:0/3470558848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5569223cd4d0 con 0x556921f791b0 2024-07-31T14:52:36.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.585+0000 7ffa98ff9640 1 -- 172.21.15.67:0/3470558848 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ffaa00115e0 con 0x556921f791b0 2024-07-31T14:52:36.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.585+0000 7ffabdb9b740 10 client.25711 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:36.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.585+0000 7ffabdb9b740 10 client.25711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:36.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.585+0000 7ffabdb9b740 20 client.25711 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:36.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.585+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/3470558848 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556921fa2070 0x5569223cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.585+0000 7ffabdb9b740 4 client.25711 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:52:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.585+0000 7ffabdb9b740 1 -- 172.21.15.67:0/3470558848 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5569222bfae0 con 0x556921fa2070 2024-07-31T14:52:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.585+0000 7ffab53e6640 1 --2- 172.21.15.67:0/3470558848 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556921fa2070 0x5569223cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffab53e6640 1 --2- 172.21.15.67:0/3470558848 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556921fa2070 0x5569223cfbe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2222321f5d890045 server_cookie=211435c2e75ab6f3 in_seq=0 out_seq=0 2024-07-31T14:52:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffa98ff9640 10 client.25711 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffa98ff9640 1 -- 172.21.15.67:0/3470558848 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5569222bfae0 con 0x556921fa2070 2024-07-31T14:52:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffa98ff9640 10 client.25711 handle_command_reply: tid=0 2024-07-31T14:52:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffabdb9b740 1 client.25711 shutdown 2024-07-31T14:52:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffabdb9b740 1 -- 172.21.15.67:0/3470558848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f791b0 msgr2=0x556922356f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/3470558848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f791b0 0x556922356f00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x556922365490 tx=0x7ffaa000bd90 comp rx=0 tx=0).stop 2024-07-31T14:52:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffabdb9b740 1 -- 172.21.15.67:0/3470558848 shutdown_connections 2024-07-31T14:52:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/3470558848 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556921fa2070 0x5569223cfbe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/3470558848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556921f81390 0x556922357440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/3470558848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556921f791b0 0x556922356f00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffabdb9b740 1 --2- 172.21.15.67:0/3470558848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556921f609f0 0x5569223569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffabdb9b740 1 -- 172.21.15.67:0/3470558848 >> 172.21.15.67:0/3470558848 conn(0x556921f96460 msgr2=0x556922343c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:36.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffa98ff9640 0 client.25711 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:36.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.586+0000 7ffabdb9b740 1 -- 172.21.15.67:0/3470558848 shutdown_connections 2024-07-31T14:52:36.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.587+0000 7ffabdb9b740 1 -- 172.21.15.67:0/3470558848 wait complete. 2024-07-31T14:52:36.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.587+0000 7ffabdb9b740 20 client.25711 trim_cache size 0 max 16384 2024-07-31T14:52:36.590 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:52:36.590 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T14:52:36.590 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T14:52:36.590 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:52:36.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.588+0000 7ffab764a640 1 -- 172.21.15.67:0/733346273 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffa9c0b3dc0 msgr2=0x7ffa9c0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:36.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.588+0000 7ffab764a640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffa9c0b3dc0 0x7ffa9c0b62d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8005fd0 tx=0x7ffaa800bbf0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.588+0000 7ffab764a640 1 -- 172.21.15.67:0/733346273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffab01024d0 msgr2=0x7ffab0199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:36.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.588+0000 7ffab764a640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffab01024d0 0x7ffab0199f10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffaac005e30 tx=0x7ffaac031680 comp rx=0 tx=0).stop 2024-07-31T14:52:36.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.588+0000 7ffab764a640 1 -- 172.21.15.67:0/733346273 shutdown_connections 2024-07-31T14:52:36.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.588+0000 7ffab764a640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffa9c0b3dc0 0x7ffa9c0b62d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.588+0000 7ffab764a640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab0108760 0x7ffab019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.588+0000 7ffab764a640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffab0107e10 0x7ffab019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.588+0000 7ffab764a640 1 --2- 172.21.15.67:0/733346273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffab01024d0 0x7ffab0199f10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:36.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.588+0000 7ffab764a640 1 -- 172.21.15.67:0/733346273 >> 172.21.15.67:0/733346273 conn(0x7ffab00791e0 msgr2=0x7ffab0100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:36.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.588+0000 7ffab764a640 1 -- 172.21.15.67:0/733346273 shutdown_connections 2024-07-31T14:52:36.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:36.588+0000 7ffab764a640 1 -- 172.21.15.67:0/733346273 wait complete. 2024-07-31T14:52:36.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:36 smithi067 ceph-mon[32750]: pgmap v445: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-07-31T14:52:37.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:36 smithi073 ceph-mon[36714]: pgmap v445: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-07-31T14:52:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:36 smithi077 ceph-mon[37327]: pgmap v445: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-07-31T14:52:37.390 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T14:52:37.390 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:52:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:39.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:38 smithi067 ceph-mon[32750]: pgmap v446: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:52:39.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:38 smithi073 ceph-mon[36714]: pgmap v446: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:52:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:38 smithi077 ceph-mon[37327]: pgmap v446: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:52:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.754+0000 7f0fa4380640 1 -- 172.21.15.67:0/1124930236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c108760 msgr2=0x7f0f9c10ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.754+0000 7f0fa4380640 1 --2- 172.21.15.67:0/1124930236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c108760 0x7f0f9c10ef90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009ca0 tx=0x7f0f9402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:52:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.755+0000 7f0fa4380640 1 -- 172.21.15.67:0/1124930236 shutdown_connections 2024-07-31T14:52:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.755+0000 7f0fa4380640 1 --2- 172.21.15.67:0/1124930236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c108760 0x7f0f9c10ef90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.755+0000 7f0fa4380640 1 --2- 172.21.15.67:0/1124930236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c107e10 0x7f0f9c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.755+0000 7f0fa4380640 1 --2- 172.21.15.67:0/1124930236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f9c1024d0 0x7f0f9c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.755+0000 7f0fa4380640 1 -- 172.21.15.67:0/1124930236 >> 172.21.15.67:0/1124930236 conn(0x7f0f9c079190 msgr2=0x7f0f9c0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.755+0000 7f0fa4380640 1 -- 172.21.15.67:0/1124930236 shutdown_connections 2024-07-31T14:52:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.755+0000 7f0fa4380640 1 -- 172.21.15.67:0/1124930236 wait complete. 2024-07-31T14:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.756+0000 7f0fa4380640 1 Processor -- start 2024-07-31T14:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.756+0000 7f0fa4380640 1 -- start start 2024-07-31T14:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.757+0000 7f0fa4380640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c1024d0 0x7f0f9c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.757+0000 7f0fa211c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c1024d0 0x7f0f9c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.757+0000 7f0fa211c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c1024d0 0x7f0f9c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42424/0 (socket says 172.21.15.67:42424) 2024-07-31T14:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.757+0000 7f0fa4380640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c107e10 0x7f0f9c199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.757+0000 7f0fa211c640 1 -- 172.21.15.67:0/106526095 learned_addr learned my addr 172.21.15.67:0/106526095 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.757+0000 7f0fa4380640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f9c108760 0x7f0f9c199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.757+0000 7f0fa4380640 1 -- 172.21.15.67:0/106526095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c112430 con 0x7f0f9c1024d0 2024-07-31T14:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.757+0000 7f0fa4380640 1 -- 172.21.15.67:0/106526095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1122b0 con 0x7f0f9c107e10 2024-07-31T14:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.757+0000 7f0fa4380640 1 -- 172.21.15.67:0/106526095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1125b0 con 0x7f0f9c108760 2024-07-31T14:52:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.757+0000 7f0fa191b640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c107e10 0x7f0f9c199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.757+0000 7f0fa291d640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f9c108760 0x7f0f9c199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.758+0000 7f0fa191b640 1 -- 172.21.15.67:0/106526095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f9c108760 msgr2=0x7f0f9c199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.758+0000 7f0fa191b640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f9c108760 0x7f0f9c199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.758+0000 7f0fa191b640 1 -- 172.21.15.67:0/106526095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c1024d0 msgr2=0x7f0f9c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.758+0000 7f0fa191b640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c1024d0 0x7f0f9c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.758+0000 7f0fa191b640 1 -- 172.21.15.67:0/106526095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c1a81c0 con 0x7f0f9c107e10 2024-07-31T14:52:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.758+0000 7f0fa191b640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c107e10 0x7f0f9c199030 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00e9c0 tx=0x7f0f8c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.758+0000 7f0f937fe640 1 -- 172.21.15.67:0/106526095 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c00ccc0 con 0x7f0f9c107e10 2024-07-31T14:52:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.758+0000 7f0f937fe640 1 -- 172.21.15.67:0/106526095 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f8c0108e0 con 0x7f0f9c107e10 2024-07-31T14:52:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.758+0000 7f0f937fe640 1 -- 172.21.15.67:0/106526095 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0187c0 con 0x7f0f9c107e10 2024-07-31T14:52:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.758+0000 7f0fa4380640 1 -- 172.21.15.67:0/106526095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9c1a84b0 con 0x7f0f9c107e10 2024-07-31T14:52:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.758+0000 7f0fa4380640 1 -- 172.21.15.67:0/106526095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f9c1a89a0 con 0x7f0f9c107e10 2024-07-31T14:52:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.760+0000 7f0fa4380640 1 -- 172.21.15.67:0/106526095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f9c1028e0 con 0x7f0f9c107e10 2024-07-31T14:52:39.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.763+0000 7f0f937fe640 1 -- 172.21.15.67:0/106526095 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0f8c010430 con 0x7f0f9c107e10 2024-07-31T14:52:39.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.764+0000 7f0f937fe640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f880b3d40 0x7f0f880b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:39.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.764+0000 7f0f937fe640 1 -- 172.21.15.67:0/106526095 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f0f8c0e45c0 con 0x7f0f9c107e10 2024-07-31T14:52:39.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.765+0000 7f0fa211c640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f880b3d40 0x7f0f880b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.766+0000 7f0fa211c640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f880b3d40 0x7f0f880b6250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c108510 tx=0x7f0f98005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:39.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:39.769+0000 7f0f937fe640 1 -- 172.21.15.67:0/106526095 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f0f8c0aaf70 con 0x7f0f9c107e10 2024-07-31T14:52:40.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:40.248+0000 7f0fa4380640 1 -- 172.21.15.67:0/106526095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f9c109fc0 con 0x7f0f9c107e10 2024-07-31T14:52:40.251 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:52:40.251 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-31T14:51:08:515297+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:51:07.734542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:52:40.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:40.249+0000 7f0f937fe640 1 -- 172.21.15.67:0/106526095 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11962 (secure 0 0 0) 0x7f0f8c0af4c0 con 0x7f0f9c107e10 2024-07-31T14:52:40.251 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-31T14:52:40.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:40.252+0000 7f0fa4380640 1 -- 172.21.15.67:0/106526095 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f880b3d40 msgr2=0x7f0f880b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:40.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:40.252+0000 7f0fa4380640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f880b3d40 0x7f0f880b6250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c108510 tx=0x7f0f98005f90 comp rx=0 tx=0).stop 2024-07-31T14:52:40.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:40.252+0000 7f0fa4380640 1 -- 172.21.15.67:0/106526095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c107e10 msgr2=0x7f0f9c199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:40.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:40.252+0000 7f0fa4380640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c107e10 0x7f0f9c199030 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00e9c0 tx=0x7f0f8c00ee80 comp rx=0 tx=0).stop 2024-07-31T14:52:40.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:40.252+0000 7f0fa4380640 1 -- 172.21.15.67:0/106526095 shutdown_connections 2024-07-31T14:52:40.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:40.252+0000 7f0fa4380640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f880b3d40 0x7f0f880b6250 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:40.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:40.252+0000 7f0fa4380640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f9c108760 0x7f0f9c199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:40.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:40.252+0000 7f0fa4380640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c107e10 0x7f0f9c199030 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:40.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:40.252+0000 7f0fa4380640 1 --2- 172.21.15.67:0/106526095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c1024d0 0x7f0f9c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:40.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:40.252+0000 7f0fa4380640 1 -- 172.21.15.67:0/106526095 >> 172.21.15.67:0/106526095 conn(0x7f0f9c079190 msgr2=0x7f0f9c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:40.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:40.253+0000 7f0fa4380640 1 -- 172.21.15.67:0/106526095 shutdown_connections 2024-07-31T14:52:40.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:40.253+0000 7f0fa4380640 1 -- 172.21.15.67:0/106526095 wait complete. 2024-07-31T14:52:40.881 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:40 smithi067 ceph-mon[32750]: pgmap v447: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:52:40.882 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:40 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/106526095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:52:40.919 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T14:52:41.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:40 smithi073 ceph-mon[36714]: pgmap v447: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:52:41.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:40 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/106526095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:52:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:40 smithi077 ceph-mon[37327]: pgmap v447: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:52:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:40 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/106526095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:52:43.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:42 smithi067 ceph-mon[32750]: pgmap v448: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:52:43.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:42 smithi073 ceph-mon[36714]: pgmap v448: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:52:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.191+0000 7ff8a33e2640 1 -- 172.21.15.67:0/1085912431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff89c1024e0 msgr2=0x7ff89c1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.191+0000 7ff8a33e2640 1 --2- 172.21.15.67:0/1085912431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff89c1024e0 0x7ff89c1028f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff898009900 tx=0x7ff89802a410 comp rx=0 tx=0).stop 2024-07-31T14:52:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.191+0000 7ff8a33e2640 1 -- 172.21.15.67:0/1085912431 shutdown_connections 2024-07-31T14:52:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.191+0000 7ff8a33e2640 1 --2- 172.21.15.67:0/1085912431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff89c108720 0x7ff89c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.191+0000 7ff8a33e2640 1 --2- 172.21.15.67:0/1085912431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff89c107dd0 0x7ff89c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.191+0000 7ff8a33e2640 1 --2- 172.21.15.67:0/1085912431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff89c1024e0 0x7ff89c1028f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.191+0000 7ff8a33e2640 1 -- 172.21.15.67:0/1085912431 >> 172.21.15.67:0/1085912431 conn(0x7ff89c0790b0 msgr2=0x7ff89c0794c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.192+0000 7ff8a33e2640 1 -- 172.21.15.67:0/1085912431 shutdown_connections 2024-07-31T14:52:43.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.192+0000 7ff8a33e2640 1 -- 172.21.15.67:0/1085912431 wait complete. 2024-07-31T14:52:43.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.192+0000 7ff8a33e2640 1 Processor -- start 2024-07-31T14:52:43.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.193+0000 7ff8a33e2640 1 -- start start 2024-07-31T14:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.193+0000 7ff8a33e2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff89c1024e0 0x7ff89c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.193+0000 7ff8a33e2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff89c107dd0 0x7ff89c199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.193+0000 7ff8a33e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff89c108720 0x7ff89c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.193+0000 7ff8a33e2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1123d0 con 0x7ff89c1024e0 2024-07-31T14:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.193+0000 7ff8a33e2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c112250 con 0x7ff89c107dd0 2024-07-31T14:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.193+0000 7ff8a33e2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c112550 con 0x7ff89c108720 2024-07-31T14:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.193+0000 7ff8a117e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff89c1024e0 0x7ff89c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.193+0000 7ff8a117e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff89c1024e0 0x7ff89c198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46950/0 (socket says 172.21.15.67:46950) 2024-07-31T14:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.193+0000 7ff8a117e640 1 -- 172.21.15.67:0/4015511716 learned_addr learned my addr 172.21.15.67:0/4015511716 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.193+0000 7ff8a197f640 1 --2- 172.21.15.67:0/4015511716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff89c108720 0x7ff89c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.193+0000 7ff8a097d640 1 --2- 172.21.15.67:0/4015511716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff89c107dd0 0x7ff89c199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.194+0000 7ff8a117e640 1 -- 172.21.15.67:0/4015511716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff89c108720 msgr2=0x7ff89c1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.194+0000 7ff8a117e640 1 --2- 172.21.15.67:0/4015511716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff89c108720 0x7ff89c1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.194+0000 7ff8a117e640 1 -- 172.21.15.67:0/4015511716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff89c107dd0 msgr2=0x7ff89c199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.194+0000 7ff8a117e640 1 --2- 172.21.15.67:0/4015511716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff89c107dd0 0x7ff89c199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.194+0000 7ff8a117e640 1 -- 172.21.15.67:0/4015511716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89c1a8300 con 0x7ff89c1024e0 2024-07-31T14:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.194+0000 7ff8a117e640 1 --2- 172.21.15.67:0/4015511716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff89c1024e0 0x7ff89c198c50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff898005940 tx=0x7ff8980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.194+0000 7ff8927fc640 1 -- 172.21.15.67:0/4015511716 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89803f030 con 0x7ff89c1024e0 2024-07-31T14:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.195+0000 7ff8927fc640 1 -- 172.21.15.67:0/4015511716 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff898003cb0 con 0x7ff89c1024e0 2024-07-31T14:52:43.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.196+0000 7ff8927fc640 1 -- 172.21.15.67:0/4015511716 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898035070 con 0x7ff89c1024e0 2024-07-31T14:52:43.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.196+0000 7ff8a33e2640 1 -- 172.21.15.67:0/4015511716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89c19ae60 con 0x7ff89c1024e0 2024-07-31T14:52:43.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.196+0000 7ff8a33e2640 1 -- 172.21.15.67:0/4015511716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89c19b2f0 con 0x7ff89c1024e0 2024-07-31T14:52:43.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.200+0000 7ff8a9933740 1 Processor -- start 2024-07-31T14:52:43.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.200+0000 7ff8a9933740 1 -- start start 2024-07-31T14:52:43.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.201+0000 7ff8927fc640 1 -- 172.21.15.67:0/4015511716 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff898042030 con 0x7ff89c1024e0 2024-07-31T14:52:43.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.201+0000 7ff8a9933740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08d0070 0x55a1a0c75690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.201+0000 7ff8a9933740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a08a71b0 0x55a1a0c75bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.201+0000 7ff8a9933740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08af390 0x55a1a0c76110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.201+0000 7ff8a9933740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a1a0bebbb0 con 0x55a1a08a71b0 2024-07-31T14:52:43.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.201+0000 7ff8a9933740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a1a0bdb0b0 con 0x55a1a08af390 2024-07-31T14:52:43.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.201+0000 7ff8a9933740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a1a0bdf0f0 con 0x55a1a08d0070 2024-07-31T14:52:43.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.201+0000 7ff8a097d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08af390 0x55a1a0c76110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.201+0000 7ff8a117e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a08a71b0 0x55a1a0c75bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.201+0000 7ff8a197f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08d0070 0x55a1a0c75690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.201+0000 7ff8a117e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a08a71b0 0x55a1a0c75bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46958/0 (socket says 172.21.15.67:46958) 2024-07-31T14:52:43.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.201+0000 7ff8a117e640 1 -- 172.21.15.67:0/3347843522 learned_addr learned my addr 172.21.15.67:0/3347843522 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:43.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.202+0000 7ff8a197f640 1 -- 172.21.15.67:0/3347843522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08af390 msgr2=0x55a1a0c76110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:43.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.202+0000 7ff8a197f640 1 --2- 172.21.15.67:0/3347843522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08af390 0x55a1a0c76110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.202+0000 7ff8a197f640 1 -- 172.21.15.67:0/3347843522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a08a71b0 msgr2=0x55a1a0c75bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:43.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.202+0000 7ff8a197f640 1 --2- 172.21.15.67:0/3347843522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a08a71b0 0x55a1a0c75bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.202+0000 7ff8a197f640 1 -- 172.21.15.67:0/3347843522 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a0b81b90 con 0x55a1a08d0070 2024-07-31T14:52:43.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.202+0000 7ff8a117e640 1 --2- 172.21.15.67:0/3347843522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a08a71b0 0x55a1a0c75bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:43.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.202+0000 7ff8a197f640 1 --2- 172.21.15.67:0/3347843522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08d0070 0x55a1a0c75690 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff894009eb0 tx=0x7ff8940107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc9784dd4ccd334a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:43.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.202+0000 7ff8877fe640 1 -- 172.21.15.67:0/3347843522 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894003b70 con 0x55a1a08d0070 2024-07-31T14:52:43.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.202+0000 7ff8877fe640 1 -- 172.21.15.67:0/3347843522 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff894015070 con 0x55a1a08d0070 2024-07-31T14:52:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.202+0000 7ff8877fe640 1 -- 172.21.15.67:0/3347843522 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89401a760 con 0x55a1a08d0070 2024-07-31T14:52:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.203+0000 7ff8927fc640 1 --2- 172.21.15.67:0/4015511716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8880b3e40 0x7ff8880b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.203+0000 7ff8927fc640 1 -- 172.21.15.67:0/4015511716 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff898104290 con 0x7ff89c1024e0 2024-07-31T14:52:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.204+0000 7ff8a117e640 1 --2- 172.21.15.67:0/4015511716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8880b3e40 0x7ff8880b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.204+0000 7ff8a9933740 1 -- 172.21.15.67:0/3347843522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08d0070 msgr2=0x55a1a0c75690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.204+0000 7ff8a9933740 1 --2- 172.21.15.67:0/3347843522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08d0070 0x55a1a0c75690 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff894009eb0 tx=0x7ff8940107a0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.204+0000 7ff8a9933740 1 -- 172.21.15.67:0/3347843522 shutdown_connections 2024-07-31T14:52:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.204+0000 7ff8a9933740 1 --2- 172.21.15.67:0/3347843522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08af390 0x55a1a0c76110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.204+0000 7ff8a9933740 1 --2- 172.21.15.67:0/3347843522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a08a71b0 0x55a1a0c75bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.204+0000 7ff8a9933740 1 --2- 172.21.15.67:0/3347843522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08d0070 0x55a1a0c75690 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.204+0000 7ff8a9933740 1 -- 172.21.15.67:0/3347843522 >> 172.21.15.67:0/3347843522 conn(0x55a1a08c4460 msgr2=0x55a1a0c73580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.204+0000 7ff8a9933740 1 -- 172.21.15.67:0/3347843522 shutdown_connections 2024-07-31T14:52:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.204+0000 7ff8a9933740 1 -- 172.21.15.67:0/3347843522 wait complete. 2024-07-31T14:52:43.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.205+0000 7ff8a117e640 1 --2- 172.21.15.67:0/4015511716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8880b3e40 0x7ff8880b6350 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff898006010 tx=0x7ff898107cc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.205+0000 7ff8a9933740 1 -- start start 2024-07-31T14:52:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.205+0000 7ff8a9933740 1 Processor -- start 2024-07-31T14:52:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a9933740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08a71b0 0x55a1a0c81250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a9933740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08af390 0x55a1a0c81790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a9933740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a08d0070 0x55a1a0c8d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a9933740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a1a0a3e700 con 0x55a1a08d0070 2024-07-31T14:52:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a9933740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a1a0b7d9e0 con 0x55a1a08a71b0 2024-07-31T14:52:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a9933740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a1a0c83e70 con 0x55a1a08af390 2024-07-31T14:52:43.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a9933740 10 client.0 _ll_register_callbacks cb 0x55a1a0b984f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:43.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a197f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a08d0070 0x55a1a0c8d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a197f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a08d0070 0x55a1a0c8d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46970/0 (socket says 172.21.15.67:46970) 2024-07-31T14:52:43.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a197f640 1 -- 172.21.15.67:0/1663861972 learned_addr learned my addr 172.21.15.67:0/1663861972 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:43.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a097d640 1 --2- 172.21.15.67:0/1663861972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08af390 0x55a1a0c81790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a197f640 1 --2- 172.21.15.67:0/1663861972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08a71b0 0x55a1a0c81250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a097d640 1 -- 172.21.15.67:0/1663861972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08a71b0 msgr2=0x55a1a0c81250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:43.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a097d640 1 --2- 172.21.15.67:0/1663861972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08a71b0 0x55a1a0c81250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a097d640 1 -- 172.21.15.67:0/1663861972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a08d0070 msgr2=0x55a1a0c8d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:43.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a097d640 1 --2- 172.21.15.67:0/1663861972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a08d0070 0x55a1a0c8d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.206+0000 7ff8a097d640 1 -- 172.21.15.67:0/1663861972 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a093b650 con 0x55a1a08af390 2024-07-31T14:52:43.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.207+0000 7ff8a097d640 1 --2- 172.21.15.67:0/1663861972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08af390 0x55a1a0c81790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff89c1043c0 tx=0x7ff88c003ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7759dbbf7ba39d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:43.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.207+0000 7ff884ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:52:43.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.207+0000 7ff884ff9640 1 -- 172.21.15.67:0/1663861972 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c023020 con 0x55a1a08af390 2024-07-31T14:52:43.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.207+0000 7ff884ff9640 1 -- 172.21.15.67:0/1663861972 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff88c009ce0 con 0x55a1a08af390 2024-07-31T14:52:43.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.207+0000 7ff8a9933740 1 -- 172.21.15.67:0/1663861972 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff894015070 con 0x55a1a08af390 2024-07-31T14:52:43.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.208+0000 7ff884ff9640 1 -- 172.21.15.67:0/1663861972 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff88c01f450 con 0x55a1a08af390 2024-07-31T14:52:43.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.208+0000 7ff8a9933740 10 client.25729 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:43.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.208+0000 7ff8a9933740 1 -- 172.21.15.67:0/1663861972 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1a0c8fec0 con 0x55a1a08af390 2024-07-31T14:52:43.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.208+0000 7ff884ff9640 1 -- 172.21.15.67:0/1663861972 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff894015070 con 0x55a1a08af390 2024-07-31T14:52:43.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.208+0000 7ff884ff9640 1 -- 172.21.15.67:0/1663861972 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff88c028650 con 0x55a1a08af390 2024-07-31T14:52:43.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.208+0000 7ff8a9933740 10 client.25729 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:43.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.208+0000 7ff8a9933740 10 client.25729 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:52:43.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.208+0000 7ff8a9933740 20 client.25729 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:43.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.208+0000 7ff8a9933740 1 --2- 172.21.15.67:0/1663861972 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a1a088e9f0 0x55a1a0c934a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.208+0000 7ff8a9933740 4 client.25729 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:43.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.208+0000 7ff8a9933740 1 -- 172.21.15.67:0/1663861972 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1a0bedb00 con 0x55a1a088e9f0 2024-07-31T14:52:43.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.208+0000 7ff8a197f640 1 --2- 172.21.15.67:0/1663861972 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a1a088e9f0 0x55a1a0c934a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.209+0000 7ff884ff9640 10 client.25729 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:52:43.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.209+0000 7ff8a197f640 1 --2- 172.21.15.67:0/1663861972 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a1a088e9f0 0x55a1a0c934a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb6d5459bfc78f86 server_cookie=df57f7a2932973c in_seq=0 out_seq=0 2024-07-31T14:52:43.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.210+0000 7ff884ff9640 1 -- 172.21.15.67:0/1663861972 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1a0bedb00 con 0x55a1a088e9f0 2024-07-31T14:52:43.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.210+0000 7ff884ff9640 10 client.25729 handle_command_reply: tid=0 2024-07-31T14:52:43.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.210+0000 7ff8a9933740 1 client.25729 shutdown 2024-07-31T14:52:43.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.210+0000 7ff8a9933740 1 -- 172.21.15.67:0/1663861972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08af390 msgr2=0x55a1a0c81790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:43.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.210+0000 7ff8a9933740 1 --2- 172.21.15.67:0/1663861972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08af390 0x55a1a0c81790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff89c1043c0 tx=0x7ff88c003ed0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.210+0000 7ff8a9933740 1 -- 172.21.15.67:0/1663861972 shutdown_connections 2024-07-31T14:52:43.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.210+0000 7ff8a9933740 1 --2- 172.21.15.67:0/1663861972 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a1a088e9f0 0x55a1a0c934a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.210+0000 7ff884ff9640 0 client.25729 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:52:43.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.210+0000 7ff8a9933740 1 --2- 172.21.15.67:0/1663861972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a08d0070 0x55a1a0c8d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.210+0000 7ff8a9933740 1 --2- 172.21.15.67:0/1663861972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08af390 0x55a1a0c81790 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.210+0000 7ff8a9933740 1 --2- 172.21.15.67:0/1663861972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08a71b0 0x55a1a0c81250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.210+0000 7ff8a9933740 1 -- 172.21.15.67:0/1663861972 >> 172.21.15.67:0/1663861972 conn(0x55a1a08c4460 msgr2=0x55a1a0c71d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:43.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.211+0000 7ff8a9933740 1 -- 172.21.15.67:0/1663861972 shutdown_connections 2024-07-31T14:52:43.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.211+0000 7ff8a9933740 1 -- 172.21.15.67:0/1663861972 wait complete. 2024-07-31T14:52:43.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.211+0000 7ff8a9933740 20 client.25729 trim_cache size 0 max 16384 2024-07-31T14:52:43.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.228+0000 7ff8a9933740 1 Processor -- start 2024-07-31T14:52:43.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.228+0000 7ff8a9933740 1 -- start start 2024-07-31T14:52:43.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a9933740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a088e9f0 0x55a1a0c72370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a9933740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08a71b0 0x55a1a0c73680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a9933740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08af390 0x55a1a0c7a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a9933740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1ae5a0 con 0x55a1a088e9f0 2024-07-31T14:52:43.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a9933740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1b00e0 con 0x55a1a08af390 2024-07-31T14:52:43.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a9933740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1aec70 con 0x55a1a08a71b0 2024-07-31T14:52:43.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a197f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a088e9f0 0x55a1a0c72370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a197f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a088e9f0 0x55a1a0c72370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46982/0 (socket says 172.21.15.67:46982) 2024-07-31T14:52:43.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a197f640 1 -- 172.21.15.67:0/1996972340 learned_addr learned my addr 172.21.15.67:0/1996972340 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:43.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a097d640 1 --2- 172.21.15.67:0/1996972340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08a71b0 0x55a1a0c73680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a197f640 1 --2- 172.21.15.67:0/1996972340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08af390 0x55a1a0c7a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a197f640 1 -- 172.21.15.67:0/1996972340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08a71b0 msgr2=0x55a1a0c73680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:43.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a197f640 1 --2- 172.21.15.67:0/1996972340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08a71b0 0x55a1a0c73680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a197f640 1 -- 172.21.15.67:0/1996972340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08af390 msgr2=0x55a1a0c7a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a197f640 1 --2- 172.21.15.67:0/1996972340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08af390 0x55a1a0c7a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.229+0000 7ff8a197f640 1 -- 172.21.15.67:0/1996972340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a0c82310 con 0x55a1a088e9f0 2024-07-31T14:52:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.230+0000 7ff8a197f640 1 --2- 172.21.15.67:0/1996972340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a088e9f0 0x55a1a0c72370 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff894010ca0 tx=0x7ff894004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84043218cedb578a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.230+0000 7ff886ffd640 1 -- 172.21.15.67:0/1996972340 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894023040 con 0x55a1a088e9f0 2024-07-31T14:52:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.230+0000 7ff886ffd640 1 -- 172.21.15.67:0/1996972340 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff894015050 con 0x55a1a088e9f0 2024-07-31T14:52:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.230+0000 7ff886ffd640 1 -- 172.21.15.67:0/1996972340 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894005280 con 0x55a1a088e9f0 2024-07-31T14:52:43.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.231+0000 7ff8a9933740 1 -- 172.21.15.67:0/1996972340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a088e9f0 msgr2=0x55a1a0c72370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:43.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.231+0000 7ff8a9933740 1 --2- 172.21.15.67:0/1996972340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a088e9f0 0x55a1a0c72370 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff894010ca0 tx=0x7ff894004890 comp rx=0 tx=0).stop 2024-07-31T14:52:43.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.231+0000 7ff8a9933740 1 -- 172.21.15.67:0/1996972340 shutdown_connections 2024-07-31T14:52:43.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.231+0000 7ff8a9933740 1 --2- 172.21.15.67:0/1996972340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08af390 0x55a1a0c7a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.231+0000 7ff8a9933740 1 --2- 172.21.15.67:0/1996972340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08a71b0 0x55a1a0c73680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.231+0000 7ff8a9933740 1 --2- 172.21.15.67:0/1996972340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a088e9f0 0x55a1a0c72370 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.231+0000 7ff8a9933740 1 -- 172.21.15.67:0/1996972340 >> 172.21.15.67:0/1996972340 conn(0x55a1a08c4460 msgr2=0x55a1a0c72f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:43.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.231+0000 7ff8a9933740 1 -- 172.21.15.67:0/1996972340 shutdown_connections 2024-07-31T14:52:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.232+0000 7ff8a9933740 1 -- 172.21.15.67:0/1996972340 wait complete. 2024-07-31T14:52:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.232+0000 7ff8a9933740 1 -- start start 2024-07-31T14:52:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.232+0000 7ff8a9933740 1 Processor -- start 2024-07-31T14:52:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.233+0000 7ff8a9933740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a088e9f0 0x55a1a0c8cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.233+0000 7ff8a197f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a088e9f0 0x55a1a0c8cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.233+0000 7ff8a197f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a088e9f0 0x55a1a0c8cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46984/0 (socket says 172.21.15.67:46984) 2024-07-31T14:52:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.233+0000 7ff8a9933740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08a71b0 0x55a1a0c8d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.233+0000 7ff8a197f640 1 -- 172.21.15.67:0/2391123140 learned_addr learned my addr 172.21.15.67:0/2391123140 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.233+0000 7ff8a9933740 1 --2- 172.21.15.67:0/2391123140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08af390 0x55a1a0c8d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.233+0000 7ff8a9933740 1 -- 172.21.15.67:0/2391123140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1afa10 con 0x55a1a088e9f0 2024-07-31T14:52:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.233+0000 7ff8a9933740 1 -- 172.21.15.67:0/2391123140 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1af340 con 0x55a1a08af390 2024-07-31T14:52:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.233+0000 7ff8a9933740 1 -- 172.21.15.67:0/2391123140 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1ac580 con 0x55a1a08a71b0 2024-07-31T14:52:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.233+0000 7ff8a9933740 10 client.0 _ll_register_callbacks cb 0x7ff88c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.233+0000 7ff8a097d640 1 --2- 172.21.15.67:0/2391123140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08a71b0 0x55a1a0c8d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.234+0000 7ff8a197f640 1 -- 172.21.15.67:0/2391123140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08a71b0 msgr2=0x55a1a0c8d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.234+0000 7ff8a197f640 1 --2- 172.21.15.67:0/2391123140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08a71b0 0x55a1a0c8d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.234+0000 7ff8a197f640 1 -- 172.21.15.67:0/2391123140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08af390 msgr2=0x55a1a0c8d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.234+0000 7ff8a197f640 1 --2- 172.21.15.67:0/2391123140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08af390 0x55a1a0c8d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.234+0000 7ff8a197f640 1 -- 172.21.15.67:0/2391123140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a0c88800 con 0x55a1a088e9f0 2024-07-31T14:52:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.234+0000 7ff8a097d640 1 --2- 172.21.15.67:0/2391123140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08a71b0 0x55a1a0c8d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.234+0000 7ff8a197f640 1 --2- 172.21.15.67:0/2391123140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a088e9f0 0x55a1a0c8cd80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff894009c70 tx=0x7ff894005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f82faab8223903b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.234+0000 7ff884ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.234+0000 7ff884ff9640 1 -- 172.21.15.67:0/2391123140 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894004a00 con 0x55a1a088e9f0 2024-07-31T14:52:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.234+0000 7ff8a9933740 1 -- 172.21.15.67:0/2391123140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff894015050 con 0x55a1a088e9f0 2024-07-31T14:52:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.234+0000 7ff884ff9640 1 -- 172.21.15.67:0/2391123140 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff894003fb0 con 0x55a1a088e9f0 2024-07-31T14:52:43.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.235+0000 7ff884ff9640 1 -- 172.21.15.67:0/2391123140 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff894054050 con 0x55a1a088e9f0 2024-07-31T14:52:43.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.235+0000 7ff8a9933740 10 client.16044 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:43.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.235+0000 7ff8a9933740 1 -- 172.21.15.67:0/2391123140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1a0cfbe70 con 0x55a1a088e9f0 2024-07-31T14:52:43.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.235+0000 7ff884ff9640 1 -- 172.21.15.67:0/2391123140 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff894015050 con 0x55a1a088e9f0 2024-07-31T14:52:43.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:42 smithi077 ceph-mon[37327]: pgmap v448: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:52:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.235+0000 7ff884ff9640 1 -- 172.21.15.67:0/2391123140 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff89401a470 con 0x55a1a088e9f0 2024-07-31T14:52:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.235+0000 7ff8a9933740 10 client.16044 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.235+0000 7ff8a9933740 10 client.16044 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:52:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.235+0000 7ff8a9933740 20 client.16044 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.235+0000 7ff8a9933740 1 --2- 172.21.15.67:0/2391123140 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a1a08d0070 0x55a1a0cfe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.235+0000 7ff8a9933740 4 client.16044 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:52:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.235+0000 7ff8a9933740 1 -- 172.21.15.67:0/2391123140 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1a0be95a0 con 0x55a1a08d0070 2024-07-31T14:52:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.235+0000 7ff8a097d640 1 --2- 172.21.15.67:0/2391123140 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a1a08d0070 0x55a1a0cfe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.236+0000 7ff884ff9640 10 client.16044 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:52:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.236+0000 7ff8a097d640 1 --2- 172.21.15.67:0/2391123140 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a1a08d0070 0x55a1a0cfe3c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9ecdd0d73989a4e server_cookie=90e61f7c24a3ee04 in_seq=0 out_seq=0 2024-07-31T14:52:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff884ff9640 1 -- 172.21.15.67:0/2391123140 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1a0be95a0 con 0x55a1a08d0070 2024-07-31T14:52:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff884ff9640 10 client.16044 handle_command_reply: tid=0 2024-07-31T14:52:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff8a9933740 1 client.16044 shutdown 2024-07-31T14:52:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff8a9933740 1 -- 172.21.15.67:0/2391123140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a088e9f0 msgr2=0x55a1a0c8cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff8a9933740 1 --2- 172.21.15.67:0/2391123140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a088e9f0 0x55a1a0c8cd80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff894009c70 tx=0x7ff894005f10 comp rx=0 tx=0).stop 2024-07-31T14:52:43.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff8a9933740 1 -- 172.21.15.67:0/2391123140 shutdown_connections 2024-07-31T14:52:43.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff8a9933740 1 --2- 172.21.15.67:0/2391123140 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55a1a08d0070 0x55a1a0cfe3c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.244 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:52:43.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff884ff9640 0 client.16044 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:52:43.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff8a9933740 1 --2- 172.21.15.67:0/2391123140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1a08af390 0x55a1a0c8d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff8a9933740 1 --2- 172.21.15.67:0/2391123140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a08a71b0 0x55a1a0c8d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff8a9933740 1 --2- 172.21.15.67:0/2391123140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a1a088e9f0 0x55a1a0c8cd80 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff8a9933740 1 -- 172.21.15.67:0/2391123140 >> 172.21.15.67:0/2391123140 conn(0x55a1a08c4460 msgr2=0x55a1a0c766c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:43.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff8a9933740 1 -- 172.21.15.67:0/2391123140 shutdown_connections 2024-07-31T14:52:43.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff8a9933740 1 -- 172.21.15.67:0/2391123140 wait complete. 2024-07-31T14:52:43.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.237+0000 7ff8a9933740 20 client.16044 trim_cache size 0 max 16384 2024-07-31T14:52:43.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.239+0000 7ff8a33e2640 1 -- 172.21.15.67:0/4015511716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8880b3e40 msgr2=0x7ff8880b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:43.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.239+0000 7ff8a33e2640 1 --2- 172.21.15.67:0/4015511716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8880b3e40 0x7ff8880b6350 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff898006010 tx=0x7ff898107cc0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.239+0000 7ff8a33e2640 1 -- 172.21.15.67:0/4015511716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff89c1024e0 msgr2=0x7ff89c198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:43.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.239+0000 7ff8a33e2640 1 --2- 172.21.15.67:0/4015511716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff89c1024e0 0x7ff89c198c50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff898005940 tx=0x7ff8980042f0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.240+0000 7ff8a33e2640 1 -- 172.21.15.67:0/4015511716 shutdown_connections 2024-07-31T14:52:43.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.240+0000 7ff8a33e2640 1 --2- 172.21.15.67:0/4015511716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8880b3e40 0x7ff8880b6350 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.240+0000 7ff8a33e2640 1 --2- 172.21.15.67:0/4015511716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff89c108720 0x7ff89c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.240+0000 7ff8a33e2640 1 --2- 172.21.15.67:0/4015511716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff89c107dd0 0x7ff89c199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.240+0000 7ff8a33e2640 1 --2- 172.21.15.67:0/4015511716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff89c1024e0 0x7ff89c198c50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:43.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.240+0000 7ff8a33e2640 1 -- 172.21.15.67:0/4015511716 >> 172.21.15.67:0/4015511716 conn(0x7ff89c0790b0 msgr2=0x7ff89c105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:43.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.240+0000 7ff8a33e2640 1 -- 172.21.15.67:0/4015511716 shutdown_connections 2024-07-31T14:52:43.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:43.240+0000 7ff8a33e2640 1 -- 172.21.15.67:0/4015511716 wait complete. 2024-07-31T14:52:44.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T14:52:44.313 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:45.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:44 smithi067 ceph-mon[32750]: pgmap v449: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 12 op/s 2024-07-31T14:52:45.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:44 smithi073 ceph-mon[36714]: pgmap v449: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 12 op/s 2024-07-31T14:52:45.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:44 smithi077 ceph-mon[37327]: pgmap v449: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 12 op/s 2024-07-31T14:52:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.425+0000 7feea8d35640 1 -- 172.21.15.67:0/403443570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea4104340 msgr2=0x7feea4104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.425+0000 7feea8d35640 1 --2- 172.21.15.67:0/403443570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea4104340 0x7feea4104750 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fee94009900 tx=0x7fee9402a410 comp rx=0 tx=0).stop 2024-07-31T14:52:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.425+0000 7feea8d35640 1 -- 172.21.15.67:0/403443570 shutdown_connections 2024-07-31T14:52:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.425+0000 7feea8d35640 1 --2- 172.21.15.67:0/403443570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea410d7a0 0x7feea410fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.425+0000 7feea8d35640 1 --2- 172.21.15.67:0/403443570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feea4104c90 0x7feea410d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.425+0000 7feea8d35640 1 --2- 172.21.15.67:0/403443570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea4104340 0x7feea4104750 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.425+0000 7feea8d35640 1 -- 172.21.15.67:0/403443570 >> 172.21.15.67:0/403443570 conn(0x7feea40fddf0 msgr2=0x7feea4100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.425+0000 7feea8d35640 1 -- 172.21.15.67:0/403443570 shutdown_connections 2024-07-31T14:52:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.425+0000 7feea8d35640 1 -- 172.21.15.67:0/403443570 wait complete. 2024-07-31T14:52:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.426+0000 7feea8d35640 1 Processor -- start 2024-07-31T14:52:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.426+0000 7feea8d35640 1 -- start start 2024-07-31T14:52:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea8d35640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea4104340 0x7feea4199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea8d35640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea4104c90 0x7feea419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea8d35640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feea410d7a0 0x7feea419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea8d35640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feea4113430 con 0x7feea4104c90 2024-07-31T14:52:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea8d35640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feea41132b0 con 0x7feea4104340 2024-07-31T14:52:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea8d35640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feea41135b0 con 0x7feea410d7a0 2024-07-31T14:52:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea1d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea4104c90 0x7feea419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea1d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea4104c90 0x7feea419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47008/0 (socket says 172.21.15.67:47008) 2024-07-31T14:52:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea1d9b640 1 -- 172.21.15.67:0/2450947196 learned_addr learned my addr 172.21.15.67:0/2450947196 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea259c640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea4104340 0x7feea4199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea2d9d640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feea410d7a0 0x7feea419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea1d9b640 1 -- 172.21.15.67:0/2450947196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feea410d7a0 msgr2=0x7feea419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea1d9b640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feea410d7a0 0x7feea419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea1d9b640 1 -- 172.21.15.67:0/2450947196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea4104340 msgr2=0x7feea4199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea1d9b640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea4104340 0x7feea4199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea1d9b640 1 -- 172.21.15.67:0/2450947196 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea419fde0 con 0x7feea4104c90 2024-07-31T14:52:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea2d9d640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feea410d7a0 0x7feea419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea259c640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea4104340 0x7feea4199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7feea1d9b640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea4104c90 0x7feea419a450 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fee9800e9c0 tx=0x7fee9800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.427+0000 7fee937fe640 1 -- 172.21.15.67:0/2450947196 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9800ccc0 con 0x7feea4104c90 2024-07-31T14:52:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.428+0000 7fee937fe640 1 -- 172.21.15.67:0/2450947196 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee980054d0 con 0x7feea4104c90 2024-07-31T14:52:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.428+0000 7feea8d35640 1 -- 172.21.15.67:0/2450947196 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea41a00d0 con 0x7feea4104c90 2024-07-31T14:52:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.428+0000 7fee937fe640 1 -- 172.21.15.67:0/2450947196 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee98010430 con 0x7feea4104c90 2024-07-31T14:52:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.428+0000 7feea8d35640 1 -- 172.21.15.67:0/2450947196 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea41a6860 con 0x7feea4104c90 2024-07-31T14:52:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.431+0000 7feeafc21740 1 Processor -- start 2024-07-31T14:52:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.431+0000 7feeafc21740 1 -- start start 2024-07-31T14:52:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.431+0000 7feeafc21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692aa1b0 0x56036964cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.431+0000 7feeafc21740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692812f0 0x56036964d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.431+0000 7feeafc21740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692894d0 0x56036964d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.431+0000 7feeafc21740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5603695c23b0 con 0x5603692894d0 2024-07-31T14:52:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.431+0000 7feeafc21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5603695b18b0 con 0x5603692aa1b0 2024-07-31T14:52:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.431+0000 7feeafc21740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5603695b58f0 con 0x5603692812f0 2024-07-31T14:52:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.432+0000 7feea259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692812f0 0x56036964d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.432+0000 7feea259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692812f0 0x56036964d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:33672/0 (socket says 172.21.15.67:33672) 2024-07-31T14:52:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.432+0000 7feea259c640 1 -- 172.21.15.67:0/3646172325 learned_addr learned my addr 172.21.15.67:0/3646172325 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.432+0000 7feea2d9d640 1 --2- 172.21.15.67:0/3646172325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692aa1b0 0x56036964cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.432+0000 7fee937fe640 1 -- 172.21.15.67:0/2450947196 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fee98005020 con 0x7feea4104c90 2024-07-31T14:52:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.432+0000 7feea1d9b640 1 --2- 172.21.15.67:0/3646172325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692894d0 0x56036964d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.432+0000 7feea2d9d640 1 -- 172.21.15.67:0/3646172325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692812f0 msgr2=0x56036964d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.432+0000 7feea2d9d640 1 --2- 172.21.15.67:0/3646172325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692812f0 0x56036964d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.432+0000 7feea2d9d640 1 -- 172.21.15.67:0/3646172325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692894d0 msgr2=0x56036964d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.432+0000 7feea2d9d640 1 --2- 172.21.15.67:0/3646172325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692894d0 0x56036964d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.432+0000 7feea2d9d640 1 -- 172.21.15.67:0/3646172325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603695589a0 con 0x5603692aa1b0 2024-07-31T14:52:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.432+0000 7feea2d9d640 1 --2- 172.21.15.67:0/3646172325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692aa1b0 0x56036964cce0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009fb0 tx=0x7fee9c00f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a4f12b9d17156e0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.432+0000 7fee90ff9640 1 -- 172.21.15.67:0/3646172325 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c015070 con 0x5603692aa1b0 2024-07-31T14:52:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.433+0000 7fee90ff9640 1 -- 172.21.15.67:0/3646172325 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c011bb0 con 0x5603692aa1b0 2024-07-31T14:52:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.433+0000 7fee90ff9640 1 -- 172.21.15.67:0/3646172325 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c010b40 con 0x5603692aa1b0 2024-07-31T14:52:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.433+0000 7fee937fe640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fee8c0b3d40 0x7fee8c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.433+0000 7feea259c640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fee8c0b3d40 0x7fee8c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.433+0000 7fee937fe640 1 -- 172.21.15.67:0/2450947196 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fee980e47a0 con 0x7feea4104c90 2024-07-31T14:52:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.434+0000 7feeafc21740 1 -- 172.21.15.67:0/3646172325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692aa1b0 msgr2=0x56036964cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.434+0000 7feeafc21740 1 --2- 172.21.15.67:0/3646172325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692aa1b0 0x56036964cce0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009fb0 tx=0x7fee9c00f7f0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.434+0000 7feea259c640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fee8c0b3d40 0x7fee8c0b6250 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x56036964b960 tx=0x7fee940058b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.434+0000 7feeafc21740 1 -- 172.21.15.67:0/3646172325 shutdown_connections 2024-07-31T14:52:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.434+0000 7feeafc21740 1 --2- 172.21.15.67:0/3646172325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692894d0 0x56036964d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.434+0000 7feeafc21740 1 --2- 172.21.15.67:0/3646172325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692812f0 0x56036964d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.434+0000 7feeafc21740 1 --2- 172.21.15.67:0/3646172325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692aa1b0 0x56036964cce0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.434+0000 7feeafc21740 1 -- 172.21.15.67:0/3646172325 >> 172.21.15.67:0/3646172325 conn(0x56036929e300 msgr2=0x56036964abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.434+0000 7feeafc21740 1 -- 172.21.15.67:0/3646172325 shutdown_connections 2024-07-31T14:52:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.434+0000 7feeafc21740 1 -- 172.21.15.67:0/3646172325 wait complete. 2024-07-31T14:52:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.434+0000 7feeafc21740 1 -- start start 2024-07-31T14:52:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.434+0000 7feeafc21740 1 Processor -- start 2024-07-31T14:52:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feeafc21740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692812f0 0x560369658810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feeafc21740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692894d0 0x560369658d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feeafc21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692aa1b0 0x5603696647e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feeafc21740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5603694ebab0 con 0x5603692894d0 2024-07-31T14:52:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feeafc21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560369539cf0 con 0x5603692aa1b0 2024-07-31T14:52:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feeafc21740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56036965b460 con 0x5603692812f0 2024-07-31T14:52:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feeafc21740 10 client.0 _ll_register_callbacks cb 0x5603695ab9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feea1d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692894d0 0x560369658d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feea1d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692894d0 0x560369658d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47018/0 (socket says 172.21.15.67:47018) 2024-07-31T14:52:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feea1d9b640 1 -- 172.21.15.67:0/163549290 learned_addr learned my addr 172.21.15.67:0/163549290 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feea2d9d640 1 --2- 172.21.15.67:0/163549290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692aa1b0 0x5603696647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feea259c640 1 --2- 172.21.15.67:0/163549290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692812f0 0x560369658810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feea1d9b640 1 -- 172.21.15.67:0/163549290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692812f0 msgr2=0x560369658810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feea1d9b640 1 --2- 172.21.15.67:0/163549290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692812f0 0x560369658810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feea1d9b640 1 -- 172.21.15.67:0/163549290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692aa1b0 msgr2=0x5603696647e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feea1d9b640 1 --2- 172.21.15.67:0/163549290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692aa1b0 0x5603696647e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feea1d9b640 1 -- 172.21.15.67:0/163549290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560369315790 con 0x5603692894d0 2024-07-31T14:52:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feea259c640 1 --2- 172.21.15.67:0/163549290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692812f0 0x560369658810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feea1d9b640 1 --2- 172.21.15.67:0/163549290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692894d0 0x560369658d50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fee9800c6a0 tx=0x7fee980aaab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ff28b52f9917b4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7fee89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7fee89ffb640 1 -- 172.21.15.67:0/163549290 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee98005680 con 0x5603692894d0 2024-07-31T14:52:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7fee89ffb640 1 -- 172.21.15.67:0/163549290 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fee980a7690 con 0x5603692894d0 2024-07-31T14:52:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.435+0000 7feeafc21740 1 -- 172.21.15.67:0/163549290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee9c011bb0 con 0x5603692894d0 2024-07-31T14:52:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.436+0000 7fee89ffb640 1 -- 172.21.15.67:0/163549290 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fee980aabb0 con 0x5603692894d0 2024-07-31T14:52:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.436+0000 7feeafc21740 10 client.16056 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.436+0000 7feeafc21740 1 -- 172.21.15.67:0/163549290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56036938e620 con 0x5603692894d0 2024-07-31T14:52:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.436+0000 7fee89ffb640 1 -- 172.21.15.67:0/163549290 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c011bb0 con 0x5603692894d0 2024-07-31T14:52:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.436+0000 7fee89ffb640 1 -- 172.21.15.67:0/163549290 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fee980e4420 con 0x5603692894d0 2024-07-31T14:52:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.436+0000 7feeafc21740 10 client.16056 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.436+0000 7feeafc21740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.436+0000 7feeafc21740 20 client.16056 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.436+0000 7feeafc21740 1 --2- 172.21.15.67:0/163549290 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560369268d80 0x56036966a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.436+0000 7feeafc21740 4 client.16056 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.436+0000 7feeafc21740 1 -- 172.21.15.67:0/163549290 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603695c4300 con 0x560369268d80 2024-07-31T14:52:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.436+0000 7feea259c640 1 --2- 172.21.15.67:0/163549290 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560369268d80 0x56036966a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.437+0000 7fee89ffb640 10 client.16056 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.437+0000 7feea259c640 1 --2- 172.21.15.67:0/163549290 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560369268d80 0x56036966a820 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=585c8875e4953558 server_cookie=b6000c952276a9f2 in_seq=0 out_seq=0 2024-07-31T14:52:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7fee89ffb640 1 -- 172.21.15.67:0/163549290 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603695c4300 con 0x560369268d80 2024-07-31T14:52:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7fee89ffb640 10 client.16056 handle_command_reply: tid=0 2024-07-31T14:52:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7feeafc21740 1 client.16056 shutdown 2024-07-31T14:52:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7feeafc21740 1 -- 172.21.15.67:0/163549290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692894d0 msgr2=0x560369658d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7feeafc21740 1 --2- 172.21.15.67:0/163549290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692894d0 0x560369658d50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fee9800c6a0 tx=0x7fee980aaab0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7feeafc21740 1 -- 172.21.15.67:0/163549290 shutdown_connections 2024-07-31T14:52:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7feeafc21740 1 --2- 172.21.15.67:0/163549290 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560369268d80 0x56036966a820 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7fee89ffb640 0 client.16056 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7feeafc21740 1 --2- 172.21.15.67:0/163549290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692aa1b0 0x5603696647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7feeafc21740 1 --2- 172.21.15.67:0/163549290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692894d0 0x560369658d50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7feeafc21740 1 --2- 172.21.15.67:0/163549290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692812f0 0x560369658810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7feeafc21740 1 -- 172.21.15.67:0/163549290 >> 172.21.15.67:0/163549290 conn(0x56036929e300 msgr2=0x5603696492b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7feeafc21740 1 -- 172.21.15.67:0/163549290 shutdown_connections 2024-07-31T14:52:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7feeafc21740 1 -- 172.21.15.67:0/163549290 wait complete. 2024-07-31T14:52:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.438+0000 7feeafc21740 20 client.16056 trim_cache size 0 max 16384 2024-07-31T14:52:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.453+0000 7feeafc21740 1 Processor -- start 2024-07-31T14:52:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.453+0000 7feeafc21740 1 -- start start 2024-07-31T14:52:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feeafc21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560369268d80 0x560369652e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feeafc21740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692812f0 0x5603696645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feeafc21740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692894d0 0x560369664b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feeafc21740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feea41b0700 con 0x5603692812f0 2024-07-31T14:52:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feeafc21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feea41b2240 con 0x560369268d80 2024-07-31T14:52:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feeafc21740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feea41b0dd0 con 0x5603692894d0 2024-07-31T14:52:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feea1d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692812f0 0x5603696645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feea1d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692812f0 0x5603696645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47026/0 (socket says 172.21.15.67:47026) 2024-07-31T14:52:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feea1d9b640 1 -- 172.21.15.67:0/2597625447 learned_addr learned my addr 172.21.15.67:0/2597625447 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feea259c640 1 --2- 172.21.15.67:0/2597625447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560369268d80 0x560369652e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feea2d9d640 1 --2- 172.21.15.67:0/2597625447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692894d0 0x560369664b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feea1d9b640 1 -- 172.21.15.67:0/2597625447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692894d0 msgr2=0x560369664b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feea1d9b640 1 --2- 172.21.15.67:0/2597625447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692894d0 0x560369664b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feea1d9b640 1 -- 172.21.15.67:0/2597625447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560369268d80 msgr2=0x560369652e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feea1d9b640 1 --2- 172.21.15.67:0/2597625447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560369268d80 0x560369652e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feea1d9b640 1 -- 172.21.15.67:0/2597625447 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56036964b570 con 0x5603692812f0 2024-07-31T14:52:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feea2d9d640 1 --2- 172.21.15.67:0/2597625447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692894d0 0x560369664b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.454+0000 7feea1d9b640 1 --2- 172.21.15.67:0/2597625447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692812f0 0x5603696645d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fee980ab030 tx=0x7fee980ea240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40be5f8a2226fcc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.455+0000 7fee8bfff640 1 -- 172.21.15.67:0/2597625447 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9800ce60 con 0x5603692812f0 2024-07-31T14:52:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.455+0000 7fee8bfff640 1 -- 172.21.15.67:0/2597625447 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee98005680 con 0x5603692812f0 2024-07-31T14:52:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.455+0000 7fee8bfff640 1 -- 172.21.15.67:0/2597625447 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee980af6a0 con 0x5603692812f0 2024-07-31T14:52:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 -- 172.21.15.67:0/2597625447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692812f0 msgr2=0x5603696645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 --2- 172.21.15.67:0/2597625447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692812f0 0x5603696645d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fee980ab030 tx=0x7fee980ea240 comp rx=0 tx=0).stop 2024-07-31T14:52:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 -- 172.21.15.67:0/2597625447 shutdown_connections 2024-07-31T14:52:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 --2- 172.21.15.67:0/2597625447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692894d0 0x560369664b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 --2- 172.21.15.67:0/2597625447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5603692812f0 0x5603696645d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 --2- 172.21.15.67:0/2597625447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560369268d80 0x560369652e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 -- 172.21.15.67:0/2597625447 >> 172.21.15.67:0/2597625447 conn(0x56036929e300 msgr2=0x560369656e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 -- 172.21.15.67:0/2597625447 shutdown_connections 2024-07-31T14:52:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 -- 172.21.15.67:0/2597625447 wait complete. 2024-07-31T14:52:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 -- start start 2024-07-31T14:52:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 Processor -- start 2024-07-31T14:52:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560369268d80 0x56036965f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692812f0 0x56036965fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692894d0 0x5603696cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feea41b1b70 con 0x560369268d80 2024-07-31T14:52:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feea41b14a0 con 0x5603692812f0 2024-07-31T14:52:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feea41ae6e0 con 0x5603692894d0 2024-07-31T14:52:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.456+0000 7feeafc21740 10 client.0 _ll_register_callbacks cb 0x7fee980096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7feea259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560369268d80 0x56036965f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7feea259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560369268d80 0x56036965f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47032/0 (socket says 172.21.15.67:47032) 2024-07-31T14:52:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7feea259c640 1 -- 172.21.15.67:0/2671421776 learned_addr learned my addr 172.21.15.67:0/2671421776 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7feea2d9d640 1 --2- 172.21.15.67:0/2671421776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692894d0 0x5603696cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7feea1d9b640 1 --2- 172.21.15.67:0/2671421776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692812f0 0x56036965fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7feea259c640 1 -- 172.21.15.67:0/2671421776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692894d0 msgr2=0x5603696cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7feea259c640 1 --2- 172.21.15.67:0/2671421776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692894d0 0x5603696cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7feea259c640 1 -- 172.21.15.67:0/2671421776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692812f0 msgr2=0x56036965fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7feea259c640 1 --2- 172.21.15.67:0/2671421776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692812f0 0x56036965fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7feea259c640 1 -- 172.21.15.67:0/2671421776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56036966ad80 con 0x560369268d80 2024-07-31T14:52:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7feea2d9d640 1 --2- 172.21.15.67:0/2671421776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692894d0 0x5603696cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7feea259c640 1 --2- 172.21.15.67:0/2671421776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560369268d80 0x56036965f4f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fee94031440 tx=0x7fee9403e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3333cb62ce0d2b70 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7fee89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7fee89ffb640 1 -- 172.21.15.67:0/2671421776 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94059060 con 0x560369268d80 2024-07-31T14:52:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7fee89ffb640 1 -- 172.21.15.67:0/2671421776 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fee94005450 con 0x560369268d80 2024-07-31T14:52:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.457+0000 7feeafc21740 1 -- 172.21.15.67:0/2671421776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee98005680 con 0x560369268d80 2024-07-31T14:52:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.458+0000 7fee89ffb640 1 -- 172.21.15.67:0/2671421776 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fee94060050 con 0x560369268d80 2024-07-31T14:52:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.458+0000 7feeafc21740 10 client.16062 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.458+0000 7feeafc21740 1 -- 172.21.15.67:0/2671421776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560369658380 con 0x560369268d80 2024-07-31T14:52:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.458+0000 7fee89ffb640 1 -- 172.21.15.67:0/2671421776 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee98005680 con 0x560369268d80 2024-07-31T14:52:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.458+0000 7fee89ffb640 1 -- 172.21.15.67:0/2671421776 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fee940035f0 con 0x560369268d80 2024-07-31T14:52:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.458+0000 7feeafc21740 10 client.16062 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.458+0000 7feeafc21740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.458+0000 7feeafc21740 20 client.16062 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.458+0000 7feeafc21740 1 --2- 172.21.15.67:0/2671421776 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5603692aa1b0 0x5603696d4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.458+0000 7feeafc21740 4 client.16062 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:52:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.458+0000 7feeafc21740 1 -- 172.21.15.67:0/2671421776 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56036964f8b0 con 0x5603692aa1b0 2024-07-31T14:52:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.458+0000 7feea1d9b640 1 --2- 172.21.15.67:0/2671421776 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5603692aa1b0 0x5603696d4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.459+0000 7feea1d9b640 1 --2- 172.21.15.67:0/2671421776 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5603692aa1b0 0x5603696d4e00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef1679939cb96684 server_cookie=3965acf99ee8ac71 in_seq=0 out_seq=0 2024-07-31T14:52:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.459+0000 7fee89ffb640 10 client.16062 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7fee89ffb640 1 -- 172.21.15.67:0/2671421776 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56036964f8b0 con 0x5603692aa1b0 2024-07-31T14:52:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7fee89ffb640 10 client.16062 handle_command_reply: tid=0 2024-07-31T14:52:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7feeafc21740 1 client.16062 shutdown 2024-07-31T14:52:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7feeafc21740 1 -- 172.21.15.67:0/2671421776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560369268d80 msgr2=0x56036965f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7feeafc21740 1 --2- 172.21.15.67:0/2671421776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560369268d80 0x56036965f4f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fee94031440 tx=0x7fee9403e070 comp rx=0 tx=0).stop 2024-07-31T14:52:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7feeafc21740 1 -- 172.21.15.67:0/2671421776 shutdown_connections 2024-07-31T14:52:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7feeafc21740 1 --2- 172.21.15.67:0/2671421776 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5603692aa1b0 0x5603696d4e00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7feeafc21740 1 --2- 172.21.15.67:0/2671421776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603692894d0 0x5603696cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7fee89ffb640 0 client.16062 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7feeafc21740 1 --2- 172.21.15.67:0/2671421776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603692812f0 0x56036965fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7feeafc21740 1 --2- 172.21.15.67:0/2671421776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560369268d80 0x56036965f4f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7feeafc21740 1 -- 172.21.15.67:0/2671421776 >> 172.21.15.67:0/2671421776 conn(0x56036929e300 msgr2=0x56036964a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7feeafc21740 1 -- 172.21.15.67:0/2671421776 shutdown_connections 2024-07-31T14:52:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7feeafc21740 1 -- 172.21.15.67:0/2671421776 wait complete. 2024-07-31T14:52:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.463+0000 7feeafc21740 20 client.16062 trim_cache size 0 max 16384 2024-07-31T14:52:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:52:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:52:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:52:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:52:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.464+0000 7feea8d35640 1 -- 172.21.15.67:0/2450947196 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fee8c0b3d40 msgr2=0x7fee8c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.464+0000 7feea8d35640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fee8c0b3d40 0x7fee8c0b6250 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x56036964b960 tx=0x7fee940058b0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.464+0000 7feea8d35640 1 -- 172.21.15.67:0/2450947196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea4104c90 msgr2=0x7feea419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.464+0000 7feea8d35640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea4104c90 0x7feea419a450 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fee9800e9c0 tx=0x7fee9800ee80 comp rx=0 tx=0).stop 2024-07-31T14:52:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.464+0000 7feea8d35640 1 -- 172.21.15.67:0/2450947196 shutdown_connections 2024-07-31T14:52:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.464+0000 7feea8d35640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fee8c0b3d40 0x7fee8c0b6250 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.464+0000 7feea8d35640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feea410d7a0 0x7feea419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.464+0000 7feea8d35640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea4104c90 0x7feea419a450 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.464+0000 7feea8d35640 1 --2- 172.21.15.67:0/2450947196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea4104340 0x7feea4199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.464+0000 7feea8d35640 1 -- 172.21.15.67:0/2450947196 >> 172.21.15.67:0/2450947196 conn(0x7feea40fddf0 msgr2=0x7feea41072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.464+0000 7feea8d35640 1 -- 172.21.15.67:0/2450947196 shutdown_connections 2024-07-31T14:52:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:45.464+0000 7feea8d35640 1 -- 172.21.15.67:0/2450947196 wait complete. 2024-07-31T14:52:46.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.054+0000 7faf067c7640 1 -- 172.21.15.67:0/1663268341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf00102490 msgr2=0x7faf001028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:46.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.054+0000 7faf067c7640 1 --2- 172.21.15.67:0/1663268341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf00102490 0x7faf001028a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7faef0009900 tx=0x7faef002a410 comp rx=0 tx=0).stop 2024-07-31T14:52:46.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.054+0000 7faf067c7640 1 -- 172.21.15.67:0/1663268341 shutdown_connections 2024-07-31T14:52:46.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.054+0000 7faf067c7640 1 --2- 172.21.15.67:0/1663268341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf00108720 0x7faf0010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.054+0000 7faf067c7640 1 --2- 172.21.15.67:0/1663268341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf00107dd0 0x7faf001081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.054+0000 7faf067c7640 1 --2- 172.21.15.67:0/1663268341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf00102490 0x7faf001028a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.054+0000 7faf067c7640 1 -- 172.21.15.67:0/1663268341 >> 172.21.15.67:0/1663268341 conn(0x7faf00079160 msgr2=0x7faf00079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:46.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.054+0000 7faf067c7640 1 -- 172.21.15.67:0/1663268341 shutdown_connections 2024-07-31T14:52:46.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.054+0000 7faf067c7640 1 -- 172.21.15.67:0/1663268341 wait complete. 2024-07-31T14:52:46.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.055+0000 7faf067c7640 1 Processor -- start 2024-07-31T14:52:46.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.055+0000 7faf067c7640 1 -- start start 2024-07-31T14:52:46.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.055+0000 7faf067c7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf00102490 0x7faf0019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.055+0000 7faf067c7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf00107dd0 0x7faf0019b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.055+0000 7faf067c7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf00108720 0x7faf0019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.055+0000 7faf067c7640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faf001123d0 con 0x7faf00102490 2024-07-31T14:52:46.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.055+0000 7faf067c7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faf00112250 con 0x7faf00107dd0 2024-07-31T14:52:46.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.055+0000 7faf067c7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faf00112550 con 0x7faf00108720 2024-07-31T14:52:46.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.055+0000 7faefffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf00102490 0x7faf0019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.055+0000 7faefffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf00102490 0x7faf0019acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47042/0 (socket says 172.21.15.67:47042) 2024-07-31T14:52:46.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.055+0000 7faefffff640 1 -- 172.21.15.67:0/353199774 learned_addr learned my addr 172.21.15.67:0/353199774 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:46.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.056+0000 7faf04d64640 1 --2- 172.21.15.67:0/353199774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf00108720 0x7faf0019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.056+0000 7faef7fff640 1 --2- 172.21.15.67:0/353199774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf00107dd0 0x7faf0019b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.056+0000 7faefffff640 1 -- 172.21.15.67:0/353199774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf00108720 msgr2=0x7faf0019b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:46.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.056+0000 7faefffff640 1 --2- 172.21.15.67:0/353199774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf00108720 0x7faf0019b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.056+0000 7faefffff640 1 -- 172.21.15.67:0/353199774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf00107dd0 msgr2=0x7faf0019b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:46.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.056+0000 7faefffff640 1 --2- 172.21.15.67:0/353199774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf00107dd0 0x7faf0019b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.056+0000 7faefffff640 1 -- 172.21.15.67:0/353199774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf001a5f50 con 0x7faf00102490 2024-07-31T14:52:46.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.056+0000 7faefffff640 1 --2- 172.21.15.67:0/353199774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf00102490 0x7faf0019acc0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7faef0005ab0 tx=0x7faef0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:46.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.056+0000 7faefdffb640 1 -- 172.21.15.67:0/353199774 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef003f070 con 0x7faf00102490 2024-07-31T14:52:46.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.056+0000 7faf067c7640 1 -- 172.21.15.67:0/353199774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf001a61e0 con 0x7faf00102490 2024-07-31T14:52:46.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.056+0000 7faefdffb640 1 -- 172.21.15.67:0/353199774 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faef0031d60 con 0x7faf00102490 2024-07-31T14:52:46.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.056+0000 7faefdffb640 1 -- 172.21.15.67:0/353199774 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0035030 con 0x7faf00102490 2024-07-31T14:52:46.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.056+0000 7faf067c7640 1 -- 172.21.15.67:0/353199774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf001a6710 con 0x7faf00102490 2024-07-31T14:52:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.059+0000 7faf0cd18740 1 Processor -- start 2024-07-31T14:52:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.059+0000 7faf0cd18740 1 -- start start 2024-07-31T14:52:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf0cd18740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cde070 0x555dd5083570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf0cd18740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4cb51b0 0x555dd5083ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf0cd18740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cbd390 0x555dd5083ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf0cd18740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555dd4ff9bb0 con 0x555dd4cb51b0 2024-07-31T14:52:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf0cd18740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555dd4fe90b0 con 0x555dd4cde070 2024-07-31T14:52:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf0cd18740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555dd4fed0f0 con 0x555dd4cbd390 2024-07-31T14:52:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf04d64640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cde070 0x555dd5083570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf04d64640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cde070 0x555dd5083570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59798/0 (socket says 172.21.15.67:59798) 2024-07-31T14:52:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf04d64640 1 -- 172.21.15.67:0/955867530 learned_addr learned my addr 172.21.15.67:0/955867530 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faef7fff640 1 --2- 172.21.15.67:0/955867530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cbd390 0x555dd5083ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faefdffb640 1 -- 172.21.15.67:0/353199774 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7faef0042070 con 0x7faf00102490 2024-07-31T14:52:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf04d64640 1 -- 172.21.15.67:0/955867530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cbd390 msgr2=0x555dd5083ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf04d64640 1 --2- 172.21.15.67:0/955867530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cbd390 0x555dd5083ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf04d64640 1 -- 172.21.15.67:0/955867530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4cb51b0 msgr2=0x555dd5083ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faefffff640 1 --2- 172.21.15.67:0/955867530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4cb51b0 0x555dd5083ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf04d64640 1 --2- 172.21.15.67:0/955867530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4cb51b0 0x555dd5083ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf04d64640 1 -- 172.21.15.67:0/955867530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dd4f8fb90 con 0x555dd4cde070 2024-07-31T14:52:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faefffff640 1 --2- 172.21.15.67:0/955867530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4cb51b0 0x555dd5083ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.060+0000 7faf04d64640 1 --2- 172.21.15.67:0/955867530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cde070 0x555dd5083570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faef8009f50 tx=0x7faef80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d29b9d070259618b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.061+0000 7faef67fc640 1 -- 172.21.15.67:0/955867530 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8004d20 con 0x555dd4cde070 2024-07-31T14:52:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.061+0000 7faef67fc640 1 -- 172.21.15.67:0/955867530 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faef8004050 con 0x555dd4cde070 2024-07-31T14:52:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.061+0000 7faef67fc640 1 -- 172.21.15.67:0/955867530 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8005220 con 0x555dd4cde070 2024-07-31T14:52:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.061+0000 7faefdffb640 1 --2- 172.21.15.67:0/353199774 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faeec0b3d70 0x7faeec0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.061+0000 7faefffff640 1 --2- 172.21.15.67:0/353199774 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faeec0b3d70 0x7faeec0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.062+0000 7faefdffb640 1 -- 172.21.15.67:0/353199774 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7faef0104a70 con 0x7faf00102490 2024-07-31T14:52:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.062+0000 7faf0cd18740 1 -- 172.21.15.67:0/955867530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cde070 msgr2=0x555dd5083570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.062+0000 7faf0cd18740 1 --2- 172.21.15.67:0/955867530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cde070 0x555dd5083570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faef8009f50 tx=0x7faef80118d0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.062+0000 7faefffff640 1 --2- 172.21.15.67:0/353199774 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faeec0b3d70 0x7faeec0b6280 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faef0101f60 tx=0x7faef0031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.062+0000 7faf0cd18740 1 -- 172.21.15.67:0/955867530 shutdown_connections 2024-07-31T14:52:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.062+0000 7faf0cd18740 1 --2- 172.21.15.67:0/955867530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cbd390 0x555dd5083ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.062+0000 7faf0cd18740 1 --2- 172.21.15.67:0/955867530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4cb51b0 0x555dd5083ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.062+0000 7faf0cd18740 1 --2- 172.21.15.67:0/955867530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cde070 0x555dd5083570 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.062+0000 7faf0cd18740 1 -- 172.21.15.67:0/955867530 >> 172.21.15.67:0/955867530 conn(0x555dd4cd2460 msgr2=0x555dd5081460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.062+0000 7faf0cd18740 1 -- 172.21.15.67:0/955867530 shutdown_connections 2024-07-31T14:52:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.062+0000 7faf0cd18740 1 -- 172.21.15.67:0/955867530 wait complete. 2024-07-31T14:52:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.062+0000 7faf0cd18740 1 -- start start 2024-07-31T14:52:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.063+0000 7faf0cd18740 1 Processor -- start 2024-07-31T14:52:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.063+0000 7faf0cd18740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cb51b0 0x555dd5093450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.063+0000 7faf0cd18740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4cbd390 0x555dd5093990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.063+0000 7faf0cd18740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cde070 0x555dd509f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.063+0000 7faf0cd18740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555dd4e4c700 con 0x555dd4cbd390 2024-07-31T14:52:46.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.063+0000 7faf0cd18740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555dd4f8b9e0 con 0x555dd4cde070 2024-07-31T14:52:46.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.063+0000 7faf0cd18740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555dd5096080 con 0x555dd4cb51b0 2024-07-31T14:52:46.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.063+0000 7faf0cd18740 10 client.0 _ll_register_callbacks cb 0x555dd4fa64f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:46.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.063+0000 7faef7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4cbd390 0x555dd5093990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.063+0000 7faef7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4cbd390 0x555dd5093990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47062/0 (socket says 172.21.15.67:47062) 2024-07-31T14:52:46.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.063+0000 7faef7fff640 1 -- 172.21.15.67:0/913469472 learned_addr learned my addr 172.21.15.67:0/913469472 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:46.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.063+0000 7faf04d64640 1 --2- 172.21.15.67:0/913469472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cb51b0 0x555dd5093450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.063+0000 7faf04d64640 1 --2- 172.21.15.67:0/913469472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cde070 0x555dd509f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.064+0000 7faef7fff640 1 -- 172.21.15.67:0/913469472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cb51b0 msgr2=0x555dd5093450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:46.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.064+0000 7faef7fff640 1 --2- 172.21.15.67:0/913469472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cb51b0 0x555dd5093450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.064+0000 7faef7fff640 1 -- 172.21.15.67:0/913469472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cde070 msgr2=0x555dd509f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:46.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.064+0000 7faef7fff640 1 --2- 172.21.15.67:0/913469472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cde070 0x555dd509f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.064+0000 7faef7fff640 1 -- 172.21.15.67:0/913469472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dd4d49650 con 0x555dd4cbd390 2024-07-31T14:52:46.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.064+0000 7faf04d64640 1 --2- 172.21.15.67:0/913469472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cb51b0 0x555dd5093450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:46.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.064+0000 7faef7fff640 1 --2- 172.21.15.67:0/913469472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4cbd390 0x555dd5093990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7faee800bf80 tx=0x7faee80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1798d43e085e59 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:46.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.064+0000 7faedf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:46.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.064+0000 7faedf7fe640 1 -- 172.21.15.67:0/913469472 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8004020 con 0x555dd4cbd390 2024-07-31T14:52:46.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.064+0000 7faf0cd18740 1 -- 172.21.15.67:0/913469472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faef8004050 con 0x555dd4cbd390 2024-07-31T14:52:46.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.064+0000 7faedf7fe640 1 -- 172.21.15.67:0/913469472 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7faee80041c0 con 0x555dd4cbd390 2024-07-31T14:52:46.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.064+0000 7faedf7fe640 1 -- 172.21.15.67:0/913469472 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faee8024050 con 0x555dd4cbd390 2024-07-31T14:52:46.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.065+0000 7faf0cd18740 10 client.16071 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:46.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.065+0000 7faf0cd18740 1 -- 172.21.15.67:0/913469472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555dd50a1f80 con 0x555dd4cbd390 2024-07-31T14:52:46.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.065+0000 7faedf7fe640 1 -- 172.21.15.67:0/913469472 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faef8004050 con 0x555dd4cbd390 2024-07-31T14:52:46.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.065+0000 7faedf7fe640 1 -- 172.21.15.67:0/913469472 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7faee80038c0 con 0x555dd4cbd390 2024-07-31T14:52:46.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.065+0000 7faf0cd18740 10 client.16071 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:46.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.065+0000 7faf0cd18740 10 client.16071 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:52:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.065+0000 7faf0cd18740 20 client.16071 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.065+0000 7faf0cd18740 1 --2- 172.21.15.67:0/913469472 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x555dd4c9c9f0 0x555dd50a5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.065+0000 7faf0cd18740 4 client.16071 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.065+0000 7faf0cd18740 1 -- 172.21.15.67:0/913469472 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dd4ffbb00 con 0x555dd4c9c9f0 2024-07-31T14:52:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.065+0000 7faf04d64640 1 --2- 172.21.15.67:0/913469472 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x555dd4c9c9f0 0x555dd50a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.066+0000 7faf04d64640 1 --2- 172.21.15.67:0/913469472 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x555dd4c9c9f0 0x555dd50a5900 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1a03fcd80f89fac6 server_cookie=8df1c842c16047dd in_seq=0 out_seq=0 2024-07-31T14:52:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.066+0000 7faedf7fe640 10 client.16071 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:52:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faedf7fe640 1 -- 172.21.15.67:0/913469472 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555dd4ffbb00 con 0x555dd4c9c9f0 2024-07-31T14:52:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faedf7fe640 10 client.16071 handle_command_reply: tid=0 2024-07-31T14:52:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faf0cd18740 1 client.16071 shutdown 2024-07-31T14:52:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faf0cd18740 1 -- 172.21.15.67:0/913469472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4cbd390 msgr2=0x555dd5093990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faf0cd18740 1 --2- 172.21.15.67:0/913469472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4cbd390 0x555dd5093990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7faee800bf80 tx=0x7faee80118d0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faf0cd18740 1 -- 172.21.15.67:0/913469472 shutdown_connections 2024-07-31T14:52:46.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faf0cd18740 1 --2- 172.21.15.67:0/913469472 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x555dd4c9c9f0 0x555dd50a5900 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faf0cd18740 1 --2- 172.21.15.67:0/913469472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cde070 0x555dd509f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faf0cd18740 1 --2- 172.21.15.67:0/913469472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4cbd390 0x555dd5093990 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faf0cd18740 1 --2- 172.21.15.67:0/913469472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cb51b0 0x555dd5093450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faedf7fe640 0 client.16071 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:52:46.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faf0cd18740 1 -- 172.21.15.67:0/913469472 >> 172.21.15.67:0/913469472 conn(0x555dd4cd2460 msgr2=0x555dd5086610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:46.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faf0cd18740 1 -- 172.21.15.67:0/913469472 shutdown_connections 2024-07-31T14:52:46.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faf0cd18740 1 -- 172.21.15.67:0/913469472 wait complete. 2024-07-31T14:52:46.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.067+0000 7faf0cd18740 20 client.16071 trim_cache size 0 max 16384 2024-07-31T14:52:46.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.082+0000 7faf0cd18740 1 Processor -- start 2024-07-31T14:52:46.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.082+0000 7faf0cd18740 1 -- start start 2024-07-31T14:52:46.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.082+0000 7faf0cd18740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4c9c9f0 0x555dd5086d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.082+0000 7faf0cd18740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cb51b0 0x555dd509f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.082+0000 7faf0cd18740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cbd390 0x555dd509f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.082+0000 7faf0cd18740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faf001ae4c0 con 0x555dd4c9c9f0 2024-07-31T14:52:46.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.082+0000 7faf0cd18740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faf001b0000 con 0x555dd4cb51b0 2024-07-31T14:52:46.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.082+0000 7faf0cd18740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faf001aeb90 con 0x555dd4cbd390 2024-07-31T14:52:46.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.083+0000 7faf04d64640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4c9c9f0 0x555dd5086d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.083+0000 7faf04d64640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4c9c9f0 0x555dd5086d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47070/0 (socket says 172.21.15.67:47070) 2024-07-31T14:52:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.083+0000 7faf04d64640 1 -- 172.21.15.67:0/3042231921 learned_addr learned my addr 172.21.15.67:0/3042231921 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.083+0000 7faf04d64640 1 -- 172.21.15.67:0/3042231921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cbd390 msgr2=0x555dd509f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.083+0000 7faf04d64640 1 --2- 172.21.15.67:0/3042231921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cbd390 0x555dd509f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.083+0000 7faf04d64640 1 -- 172.21.15.67:0/3042231921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cb51b0 msgr2=0x555dd509f1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:52:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.083+0000 7faf04d64640 1 --2- 172.21.15.67:0/3042231921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cb51b0 0x555dd509f1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.083+0000 7faf04d64640 1 -- 172.21.15.67:0/3042231921 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dd5091750 con 0x555dd4c9c9f0 2024-07-31T14:52:46.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.083+0000 7faf04d64640 1 --2- 172.21.15.67:0/3042231921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4c9c9f0 0x555dd5086d20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7faef8002230 tx=0x7faef80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19332a19f755d504 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.083+0000 7faef5ffb640 1 -- 172.21.15.67:0/3042231921 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8003a30 con 0x555dd4c9c9f0 2024-07-31T14:52:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.083+0000 7faef5ffb640 1 -- 172.21.15.67:0/3042231921 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faef8004dc0 con 0x555dd4c9c9f0 2024-07-31T14:52:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.083+0000 7faef5ffb640 1 -- 172.21.15.67:0/3042231921 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8028900 con 0x555dd4c9c9f0 2024-07-31T14:52:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.084+0000 7faf0cd18740 1 -- 172.21.15.67:0/3042231921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4c9c9f0 msgr2=0x555dd5086d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.084+0000 7faf0cd18740 1 --2- 172.21.15.67:0/3042231921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4c9c9f0 0x555dd5086d20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7faef8002230 tx=0x7faef80118d0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.084+0000 7faf0cd18740 1 -- 172.21.15.67:0/3042231921 shutdown_connections 2024-07-31T14:52:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.084+0000 7faf0cd18740 1 --2- 172.21.15.67:0/3042231921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cbd390 0x555dd509f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.084+0000 7faf0cd18740 1 --2- 172.21.15.67:0/3042231921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cb51b0 0x555dd509f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.084+0000 7faf0cd18740 1 --2- 172.21.15.67:0/3042231921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4c9c9f0 0x555dd5086d20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.084+0000 7faf0cd18740 1 -- 172.21.15.67:0/3042231921 >> 172.21.15.67:0/3042231921 conn(0x555dd4cd2460 msgr2=0x555dd5086610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.085+0000 7faf0cd18740 1 -- 172.21.15.67:0/3042231921 shutdown_connections 2024-07-31T14:52:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.085+0000 7faf0cd18740 1 -- 172.21.15.67:0/3042231921 wait complete. 2024-07-31T14:52:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.085+0000 7faf0cd18740 1 -- start start 2024-07-31T14:52:46.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.085+0000 7faf0cd18740 1 Processor -- start 2024-07-31T14:52:46.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faf0cd18740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4c9c9f0 0x555dd5096710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faf0cd18740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cb51b0 0x555dd5096c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faf0cd18740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cbd390 0x555dd510c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faf0cd18740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faf001af930 con 0x555dd4c9c9f0 2024-07-31T14:52:46.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faf0cd18740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faf001af260 con 0x555dd4cbd390 2024-07-31T14:52:46.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faf0cd18740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faf001ac4a0 con 0x555dd4cb51b0 2024-07-31T14:52:46.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faf0cd18740 10 client.0 _ll_register_callbacks cb 0x7faee800bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:46.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faf04d64640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4c9c9f0 0x555dd5096710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faf04d64640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4c9c9f0 0x555dd5096710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47074/0 (socket says 172.21.15.67:47074) 2024-07-31T14:52:46.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faf04d64640 1 -- 172.21.15.67:0/3848196223 learned_addr learned my addr 172.21.15.67:0/3848196223 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:46.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faef7fff640 1 --2- 172.21.15.67:0/3848196223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cb51b0 0x555dd5096c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faf04d64640 1 -- 172.21.15.67:0/3848196223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cb51b0 msgr2=0x555dd5096c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:46.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faf04d64640 1 --2- 172.21.15.67:0/3848196223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cb51b0 0x555dd5096c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faf04d64640 1 -- 172.21.15.67:0/3848196223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cbd390 msgr2=0x555dd510c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:46.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.086+0000 7faf04d64640 1 --2- 172.21.15.67:0/3848196223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cbd390 0x555dd510c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.087+0000 7faf04d64640 1 -- 172.21.15.67:0/3848196223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dd510c600 con 0x555dd4c9c9f0 2024-07-31T14:52:46.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.087+0000 7faf04d64640 1 --2- 172.21.15.67:0/3848196223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4c9c9f0 0x555dd5096710 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7faef80047f0 tx=0x7faef8004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f7006106986d62 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:46.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.087+0000 7faedf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:46.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.087+0000 7faedf7fe640 1 -- 172.21.15.67:0/3848196223 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef80048a0 con 0x555dd4c9c9f0 2024-07-31T14:52:46.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.087+0000 7faedf7fe640 1 -- 172.21.15.67:0/3848196223 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7faef8004a40 con 0x555dd4c9c9f0 2024-07-31T14:52:46.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.087+0000 7faf0cd18740 1 -- 172.21.15.67:0/3848196223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faef8004dc0 con 0x555dd4c9c9f0 2024-07-31T14:52:46.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.087+0000 7faedf7fe640 1 -- 172.21.15.67:0/3848196223 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faef8046050 con 0x555dd4c9c9f0 2024-07-31T14:52:46.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.087+0000 7faf0cd18740 10 client.16077 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:46.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.087+0000 7faf0cd18740 1 -- 172.21.15.67:0/3848196223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555dd510d720 con 0x555dd4c9c9f0 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.088+0000 7faedf7fe640 1 -- 172.21.15.67:0/3848196223 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faef8004dc0 con 0x555dd4c9c9f0 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.088+0000 7faedf7fe640 1 -- 172.21.15.67:0/3848196223 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7faef803f760 con 0x555dd4c9c9f0 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.088+0000 7faf0cd18740 10 client.16077 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.088+0000 7faf0cd18740 10 client.16077 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.088+0000 7faf0cd18740 20 client.16077 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.088+0000 7faf0cd18740 1 --2- 172.21.15.67:0/3848196223 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x555dd4cde070 0x555dd510fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.088+0000 7faf0cd18740 4 client.16077 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.088+0000 7faf0cd18740 1 -- 172.21.15.67:0/3848196223 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555dd4ff75a0 con 0x555dd4cde070 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.088+0000 7faef7fff640 1 --2- 172.21.15.67:0/3848196223 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x555dd4cde070 0x555dd510fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.088+0000 7faedf7fe640 10 client.16077 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faef7fff640 1 --2- 172.21.15.67:0/3848196223 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x555dd4cde070 0x555dd510fc30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dcd7e86d525cab2c server_cookie=19ccd0e44419abab in_seq=0 out_seq=0 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faedf7fe640 1 -- 172.21.15.67:0/3848196223 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555dd4ff75a0 con 0x555dd4cde070 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faedf7fe640 10 client.16077 handle_command_reply: tid=0 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faf0cd18740 1 client.16077 shutdown 2024-07-31T14:52:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faf0cd18740 1 -- 172.21.15.67:0/3848196223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4c9c9f0 msgr2=0x555dd5096710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:46.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faf0cd18740 1 --2- 172.21.15.67:0/3848196223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4c9c9f0 0x555dd5096710 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7faef80047f0 tx=0x7faef8004050 comp rx=0 tx=0).stop 2024-07-31T14:52:46.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faf0cd18740 1 -- 172.21.15.67:0/3848196223 shutdown_connections 2024-07-31T14:52:46.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faf0cd18740 1 --2- 172.21.15.67:0/3848196223 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x555dd4cde070 0x555dd510fc30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faf0cd18740 1 --2- 172.21.15.67:0/3848196223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dd4cbd390 0x555dd510c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faf0cd18740 1 --2- 172.21.15.67:0/3848196223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dd4cb51b0 0x555dd5096c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faf0cd18740 1 --2- 172.21.15.67:0/3848196223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dd4c9c9f0 0x555dd5096710 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faf0cd18740 1 -- 172.21.15.67:0/3848196223 >> 172.21.15.67:0/3848196223 conn(0x555dd4cd2460 msgr2=0x555dd5080d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:46.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faedf7fe640 0 client.16077 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:52:46.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faf0cd18740 1 -- 172.21.15.67:0/3848196223 shutdown_connections 2024-07-31T14:52:46.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faf0cd18740 1 -- 172.21.15.67:0/3848196223 wait complete. 2024-07-31T14:52:46.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.089+0000 7faf0cd18740 20 client.16077 trim_cache size 0 max 16384 2024-07-31T14:52:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:52:46.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.090+0000 7faf067c7640 1 -- 172.21.15.67:0/353199774 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faeec0b3d70 msgr2=0x7faeec0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.090+0000 7faf067c7640 1 --2- 172.21.15.67:0/353199774 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faeec0b3d70 0x7faeec0b6280 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faef0101f60 tx=0x7faef0031400 comp rx=0 tx=0).stop 2024-07-31T14:52:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.090+0000 7faf067c7640 1 -- 172.21.15.67:0/353199774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf00102490 msgr2=0x7faf0019acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.090+0000 7faf067c7640 1 --2- 172.21.15.67:0/353199774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf00102490 0x7faf0019acc0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7faef0005ab0 tx=0x7faef0003c80 comp rx=0 tx=0).stop 2024-07-31T14:52:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.090+0000 7faf067c7640 1 -- 172.21.15.67:0/353199774 shutdown_connections 2024-07-31T14:52:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.090+0000 7faf067c7640 1 --2- 172.21.15.67:0/353199774 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faeec0b3d70 0x7faeec0b6280 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.090+0000 7faf067c7640 1 --2- 172.21.15.67:0/353199774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf00108720 0x7faf0019b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.090+0000 7faf067c7640 1 --2- 172.21.15.67:0/353199774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf00107dd0 0x7faf0019b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.090+0000 7faf067c7640 1 --2- 172.21.15.67:0/353199774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf00102490 0x7faf0019acc0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.090+0000 7faf067c7640 1 -- 172.21.15.67:0/353199774 >> 172.21.15.67:0/353199774 conn(0x7faf00079160 msgr2=0x7faf00100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.090+0000 7faf067c7640 1 -- 172.21.15.67:0/353199774 shutdown_connections 2024-07-31T14:52:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:46.091+0000 7faf067c7640 1 -- 172.21.15.67:0/353199774 wait complete. 2024-07-31T14:52:47.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:46 smithi067 ceph-mon[32750]: pgmap v450: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:52:47.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:46 smithi073 ceph-mon[36714]: pgmap v450: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:52:47.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:46 smithi077 ceph-mon[37327]: pgmap v450: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-07-31T14:52:47.448 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T14:52:47.741 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:48.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:47 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:52:48.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:47 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:52:48.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:47 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:52:48.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:47 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:52:48.137 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:47 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:52:48.138 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:47 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:52:48.138 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:47 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:52:48.138 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:47 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:52:48.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:47 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:52:48.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:47 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:52:48.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:47 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:52:48.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:47 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:52:48.595 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 87% (11057/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-07-31T14:52:49.093 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:48 smithi067 ceph-mon[32750]: pgmap v451: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-31T14:52:49.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:48 smithi073 ceph-mon[36714]: pgmap v451: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-31T14:52:49.206 INFO:tasks.workunit.client.0.smithi067.stderr:Note: switching to '8c796fca5ff5960f7542e7227bad4a784ee2c804'. 2024-07-31T14:52:49.207 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-31T14:52:49.207 INFO:tasks.workunit.client.0.smithi067.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-07-31T14:52:49.207 INFO:tasks.workunit.client.0.smithi067.stderr:changes and commit them, and you can discard any commits you make in this 2024-07-31T14:52:49.207 INFO:tasks.workunit.client.0.smithi067.stderr:state without impacting any branches by switching back to a branch. 2024-07-31T14:52:49.207 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-31T14:52:49.207 INFO:tasks.workunit.client.0.smithi067.stderr:If you want to create a new branch to retain commits you create, you may 2024-07-31T14:52:49.207 INFO:tasks.workunit.client.0.smithi067.stderr:do so (now or later) by using -c with the switch command. Example: 2024-07-31T14:52:49.207 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-31T14:52:49.207 INFO:tasks.workunit.client.0.smithi067.stderr: git switch -c 2024-07-31T14:52:49.207 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-31T14:52:49.207 INFO:tasks.workunit.client.0.smithi067.stderr:Or undo this operation with: 2024-07-31T14:52:49.207 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-31T14:52:49.207 INFO:tasks.workunit.client.0.smithi067.stderr: git switch - 2024-07-31T14:52:49.208 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-31T14:52:49.208 INFO:tasks.workunit.client.0.smithi067.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-07-31T14:52:49.208 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-31T14:52:49.208 INFO:tasks.workunit.client.0.smithi067.stderr:HEAD is now at 8c796fca5ff Merge PR #56602 into wip-vshankar-testing-20240723.143739-debug 2024-07-31T14:52:49.210 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-07-31T14:52:49.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:48 smithi077 ceph-mon[37327]: pgmap v451: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-31T14:52:49.281 INFO:tasks.workunit.client.0.smithi067.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-07-31T14:52:49.283 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-31T14:52:49.283 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-07-31T14:52:49.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.752+0000 7f35c3f62640 1 -- 172.21.15.67:0/2336681353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35bc102490 msgr2=0x7f35bc1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.752+0000 7f35c3f62640 1 --2- 172.21.15.67:0/2336681353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35bc102490 0x7f35bc1028a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009900 tx=0x7f35b402a410 comp rx=0 tx=0).stop 2024-07-31T14:52:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.752+0000 7f35c3f62640 1 -- 172.21.15.67:0/2336681353 shutdown_connections 2024-07-31T14:52:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.752+0000 7f35c3f62640 1 --2- 172.21.15.67:0/2336681353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35bc108720 0x7f35bc10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.752+0000 7f35c3f62640 1 --2- 172.21.15.67:0/2336681353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35bc107dd0 0x7f35bc1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.752+0000 7f35c3f62640 1 --2- 172.21.15.67:0/2336681353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35bc102490 0x7f35bc1028a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.752+0000 7f35c3f62640 1 -- 172.21.15.67:0/2336681353 >> 172.21.15.67:0/2336681353 conn(0x7f35bc079160 msgr2=0x7f35bc079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.752+0000 7f35c3f62640 1 -- 172.21.15.67:0/2336681353 shutdown_connections 2024-07-31T14:52:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.752+0000 7f35c3f62640 1 -- 172.21.15.67:0/2336681353 wait complete. 2024-07-31T14:52:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.753+0000 7f35c3f62640 1 Processor -- start 2024-07-31T14:52:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.753+0000 7f35c3f62640 1 -- start start 2024-07-31T14:52:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.753+0000 7f35c3f62640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35bc102490 0x7f35bc199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.753+0000 7f35c3f62640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35bc107dd0 0x7f35bc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.753+0000 7f35c3f62640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35bc108720 0x7f35bc19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.753+0000 7f35c3f62640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1134b0 con 0x7f35bc102490 2024-07-31T14:52:49.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.754+0000 7f35c3f62640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc113330 con 0x7f35bc107dd0 2024-07-31T14:52:49.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.754+0000 7f35c3f62640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc113630 con 0x7f35bc108720 2024-07-31T14:52:49.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.754+0000 7f35c1cfe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35bc102490 0x7f35bc199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.754+0000 7f35c1cfe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35bc102490 0x7f35bc199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47082/0 (socket says 172.21.15.67:47082) 2024-07-31T14:52:49.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.754+0000 7f35c1cfe640 1 -- 172.21.15.67:0/2291492271 learned_addr learned my addr 172.21.15.67:0/2291492271 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:49.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.754+0000 7f35c14fd640 1 --2- 172.21.15.67:0/2291492271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35bc107dd0 0x7f35bc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.754+0000 7f35c24ff640 1 --2- 172.21.15.67:0/2291492271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35bc108720 0x7f35bc19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.754+0000 7f35c1cfe640 1 -- 172.21.15.67:0/2291492271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35bc108720 msgr2=0x7f35bc19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:49.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.754+0000 7f35c1cfe640 1 --2- 172.21.15.67:0/2291492271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35bc108720 0x7f35bc19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.754+0000 7f35c1cfe640 1 -- 172.21.15.67:0/2291492271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35bc107dd0 msgr2=0x7f35bc19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:49.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.754+0000 7f35c1cfe640 1 --2- 172.21.15.67:0/2291492271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35bc107dd0 0x7f35bc19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.754+0000 7f35c1cfe640 1 -- 172.21.15.67:0/2291492271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35bc19fdf0 con 0x7f35bc102490 2024-07-31T14:52:49.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.754+0000 7f35c1cfe640 1 --2- 172.21.15.67:0/2291492271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35bc102490 0x7f35bc199e80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f35b4005940 tx=0x7f35b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:49.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.754+0000 7f35baffd640 1 -- 172.21.15.67:0/2291492271 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b403f030 con 0x7f35bc102490 2024-07-31T14:52:49.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.755+0000 7f35c3f62640 1 -- 172.21.15.67:0/2291492271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35bc1a0080 con 0x7f35bc102490 2024-07-31T14:52:49.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.755+0000 7f35baffd640 1 -- 172.21.15.67:0/2291492271 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35b4003cb0 con 0x7f35bc102490 2024-07-31T14:52:49.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.755+0000 7f35baffd640 1 -- 172.21.15.67:0/2291492271 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4035070 con 0x7f35bc102490 2024-07-31T14:52:49.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.755+0000 7f35c3f62640 1 -- 172.21.15.67:0/2291492271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35bc1a6840 con 0x7f35bc102490 2024-07-31T14:52:49.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.758+0000 7f35ca4b3740 1 Processor -- start 2024-07-31T14:52:49.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.758+0000 7f35ca4b3740 1 -- start start 2024-07-31T14:52:49.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.758+0000 7f35ca4b3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b0fb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.758+0000 7f35ca4b3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad2d1b0 0x55575b0fbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.758+0000 7f35ca4b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad35390 0x55575b0fbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.758+0000 7f35ca4b3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55575b071bb0 con 0x55575ad56070 2024-07-31T14:52:49.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.758+0000 7f35ca4b3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55575b0610b0 con 0x55575ad2d1b0 2024-07-31T14:52:49.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.758+0000 7f35ca4b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55575b0650f0 con 0x55575ad35390 2024-07-31T14:52:49.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.758+0000 7f35baffd640 1 -- 172.21.15.67:0/2291492271 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f35b4042030 con 0x7f35bc102490 2024-07-31T14:52:49.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.759+0000 7f35c24ff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b0fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.759+0000 7f35c24ff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b0fb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47090/0 (socket says 172.21.15.67:47090) 2024-07-31T14:52:49.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.759+0000 7f35c24ff640 1 -- 172.21.15.67:0/61186743 learned_addr learned my addr 172.21.15.67:0/61186743 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:49.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.759+0000 7f35c1cfe640 1 --2- 172.21.15.67:0/61186743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad2d1b0 0x55575b0fbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.759+0000 7f35c14fd640 1 --2- 172.21.15.67:0/61186743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad35390 0x55575b0fbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.759+0000 7f35c24ff640 1 -- 172.21.15.67:0/61186743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad35390 msgr2=0x55575b0fbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:49.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.759+0000 7f35c24ff640 1 --2- 172.21.15.67:0/61186743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad35390 0x55575b0fbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.759+0000 7f35c24ff640 1 -- 172.21.15.67:0/61186743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad2d1b0 msgr2=0x55575b0fbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:49.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.759+0000 7f35c24ff640 1 --2- 172.21.15.67:0/61186743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad2d1b0 0x55575b0fbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.759+0000 7f35c24ff640 1 -- 172.21.15.67:0/61186743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575b007b90 con 0x55575ad56070 2024-07-31T14:52:49.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.759+0000 7f35c14fd640 1 --2- 172.21.15.67:0/61186743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad35390 0x55575b0fbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:49.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.759+0000 7f35c24ff640 1 --2- 172.21.15.67:0/61186743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b0fb570 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009f70 tx=0x7f35ac010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d1a8f7b0123471 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:49.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.759+0000 7f35baffd640 1 --2- 172.21.15.67:0/2291492271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f35a40b3d40 0x7f35a40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.759+0000 7f35a3fff640 1 -- 172.21.15.67:0/61186743 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac016070 con 0x55575ad56070 2024-07-31T14:52:49.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.760+0000 7f35a3fff640 1 -- 172.21.15.67:0/61186743 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ac004d20 con 0x55575ad56070 2024-07-31T14:52:49.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.760+0000 7f35a3fff640 1 -- 172.21.15.67:0/61186743 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac012790 con 0x55575ad56070 2024-07-31T14:52:49.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.760+0000 7f35c1cfe640 1 --2- 172.21.15.67:0/2291492271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f35a40b3d40 0x7f35a40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.760+0000 7f35baffd640 1 -- 172.21.15.67:0/2291492271 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f35b4104b60 con 0x7f35bc102490 2024-07-31T14:52:49.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.760+0000 7f35ca4b3740 1 -- 172.21.15.67:0/61186743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 msgr2=0x55575b0fb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:49.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.760+0000 7f35c1cfe640 1 --2- 172.21.15.67:0/2291492271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f35a40b3d40 0x7f35a40b6250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55575b0fa1f0 tx=0x7f35b4108610 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:49.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.760+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/61186743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b0fb570 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009f70 tx=0x7f35ac010800 comp rx=0 tx=0).stop 2024-07-31T14:52:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.760+0000 7f35ca4b3740 1 -- 172.21.15.67:0/61186743 shutdown_connections 2024-07-31T14:52:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.760+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/61186743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad35390 0x55575b0fbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.760+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/61186743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad2d1b0 0x55575b0fbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.761+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/61186743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b0fb570 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.761+0000 7f35ca4b3740 1 -- 172.21.15.67:0/61186743 >> 172.21.15.67:0/61186743 conn(0x55575ad4a460 msgr2=0x55575b0f9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.761+0000 7f35ca4b3740 1 -- 172.21.15.67:0/61186743 shutdown_connections 2024-07-31T14:52:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.761+0000 7f35ca4b3740 1 -- 172.21.15.67:0/61186743 wait complete. 2024-07-31T14:52:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.761+0000 7f35ca4b3740 1 -- start start 2024-07-31T14:52:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.761+0000 7f35ca4b3740 1 Processor -- start 2024-07-31T14:52:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35ca4b3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad2d1b0 0x55575b107160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35ca4b3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b1076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35ca4b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad56070 0x55575b113070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35ca4b3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55575aec4700 con 0x55575ad2d1b0 2024-07-31T14:52:49.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35ca4b3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55575b0039e0 con 0x55575ad35390 2024-07-31T14:52:49.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35ca4b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55575b109d50 con 0x55575ad56070 2024-07-31T14:52:49.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35ca4b3740 10 client.0 _ll_register_callbacks cb 0x55575b01e4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:49.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35c24ff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad2d1b0 0x55575b107160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35c24ff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad2d1b0 0x55575b107160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47094/0 (socket says 172.21.15.67:47094) 2024-07-31T14:52:49.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35c24ff640 1 -- 172.21.15.67:0/2415806930 learned_addr learned my addr 172.21.15.67:0/2415806930 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:49.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35c14fd640 1 --2- 172.21.15.67:0/2415806930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b1076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35c24ff640 1 --2- 172.21.15.67:0/2415806930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad56070 0x55575b113070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35c24ff640 1 -- 172.21.15.67:0/2415806930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad56070 msgr2=0x55575b113070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35c24ff640 1 --2- 172.21.15.67:0/2415806930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad56070 0x55575b113070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35c24ff640 1 -- 172.21.15.67:0/2415806930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 msgr2=0x55575b1076a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35c24ff640 1 --2- 172.21.15.67:0/2415806930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b1076a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35c24ff640 1 -- 172.21.15.67:0/2415806930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575adc1650 con 0x55575ad2d1b0 2024-07-31T14:52:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35c14fd640 1 --2- 172.21.15.67:0/2415806930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b1076a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35c24ff640 1 --2- 172.21.15.67:0/2415806930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad2d1b0 0x55575b107160 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0098b0 tx=0x7f35ac009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f67621a3b202c8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.762+0000 7f35a17fa640 1 -- 172.21.15.67:0/2415806930 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac016070 con 0x55575ad2d1b0 2024-07-31T14:52:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.763+0000 7f35a17fa640 1 -- 172.21.15.67:0/2415806930 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f35ac012530 con 0x55575ad2d1b0 2024-07-31T14:52:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.763+0000 7f35ca4b3740 1 -- 172.21.15.67:0/2415806930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35ac004d20 con 0x55575ad2d1b0 2024-07-31T14:52:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.763+0000 7f35a17fa640 1 -- 172.21.15.67:0/2415806930 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f35ac005680 con 0x55575ad2d1b0 2024-07-31T14:52:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.763+0000 7f35ca4b3740 10 client.16089 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.763+0000 7f35ca4b3740 1 -- 172.21.15.67:0/2415806930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55575b115d60 con 0x55575ad2d1b0 2024-07-31T14:52:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.763+0000 7f35a17fa640 1 -- 172.21.15.67:0/2415806930 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ac004d20 con 0x55575ad2d1b0 2024-07-31T14:52:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.763+0000 7f35a17fa640 1 -- 172.21.15.67:0/2415806930 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f35ac00f040 con 0x55575ad2d1b0 2024-07-31T14:52:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.763+0000 7f35ca4b3740 10 client.16089 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.763+0000 7f35ca4b3740 10 client.16089 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:52:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.763+0000 7f35ca4b3740 20 client.16089 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.763+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/2415806930 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55575ad149f0 0x55575b1190c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.763+0000 7f35ca4b3740 4 client.16089 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.763+0000 7f35ca4b3740 1 -- 172.21.15.67:0/2415806930 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55575b073b00 con 0x55575ad149f0 2024-07-31T14:52:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.763+0000 7f35c14fd640 1 --2- 172.21.15.67:0/2415806930 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55575ad149f0 0x55575b1190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.764+0000 7f35a17fa640 10 client.16089 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:52:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.764+0000 7f35c14fd640 1 --2- 172.21.15.67:0/2415806930 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55575ad149f0 0x55575b1190c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a74091d0e34ad22a server_cookie=5305c78e74caf673 in_seq=0 out_seq=0 2024-07-31T14:52:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35a17fa640 1 -- 172.21.15.67:0/2415806930 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55575b073b00 con 0x55575ad149f0 2024-07-31T14:52:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35a17fa640 10 client.16089 handle_command_reply: tid=0 2024-07-31T14:52:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35ca4b3740 1 client.16089 shutdown 2024-07-31T14:52:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35ca4b3740 1 -- 172.21.15.67:0/2415806930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad2d1b0 msgr2=0x55575b107160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/2415806930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad2d1b0 0x55575b107160 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0098b0 tx=0x7f35ac009800 comp rx=0 tx=0).stop 2024-07-31T14:52:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35ca4b3740 1 -- 172.21.15.67:0/2415806930 shutdown_connections 2024-07-31T14:52:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/2415806930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad56070 0x55575b113070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/2415806930 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55575ad149f0 0x55575b1190c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/2415806930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b1076a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35a17fa640 0 client.16089 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:52:49.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/2415806930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad2d1b0 0x55575b107160 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35ca4b3740 1 -- 172.21.15.67:0/2415806930 >> 172.21.15.67:0/2415806930 conn(0x55575ad4a460 msgr2=0x55575b0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:49.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35ca4b3740 1 -- 172.21.15.67:0/2415806930 shutdown_connections 2024-07-31T14:52:49.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35ca4b3740 1 -- 172.21.15.67:0/2415806930 wait complete. 2024-07-31T14:52:49.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.765+0000 7f35ca4b3740 20 client.16089 trim_cache size 0 max 16384 2024-07-31T14:52:49.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.779+0000 7f35ca4b3740 1 Processor -- start 2024-07-31T14:52:49.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.779+0000 7f35ca4b3740 1 -- start start 2024-07-31T14:52:49.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.779+0000 7f35ca4b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad2d1b0 0x55575b0fed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.779+0000 7f35ca4b3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.779+0000 7f35ca4b3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b113520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.779+0000 7f35ca4b3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1b0690 con 0x55575ad56070 2024-07-31T14:52:49.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.779+0000 7f35ca4b3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1b21d0 con 0x55575ad35390 2024-07-31T14:52:49.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.779+0000 7f35ca4b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1b0d60 con 0x55575ad2d1b0 2024-07-31T14:52:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.779+0000 7f35c24ff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.779+0000 7f35c24ff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b113520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47102/0 (socket says 172.21.15.67:47102) 2024-07-31T14:52:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.779+0000 7f35c24ff640 1 -- 172.21.15.67:0/316948960 learned_addr learned my addr 172.21.15.67:0/316948960 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.780+0000 7f35c24ff640 1 --2- 172.21.15.67:0/316948960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad2d1b0 0x55575b0fed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.780+0000 7f35c14fd640 1 --2- 172.21.15.67:0/316948960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.780+0000 7f35c24ff640 1 -- 172.21.15.67:0/316948960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad2d1b0 msgr2=0x55575b0fed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.780+0000 7f35c24ff640 1 --2- 172.21.15.67:0/316948960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad2d1b0 0x55575b0fed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.780+0000 7f35c24ff640 1 -- 172.21.15.67:0/316948960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 msgr2=0x55575b112fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.780+0000 7f35c24ff640 1 --2- 172.21.15.67:0/316948960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b112fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.780+0000 7f35c24ff640 1 -- 172.21.15.67:0/316948960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575b0f4c70 con 0x55575ad56070 2024-07-31T14:52:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.780+0000 7f35c14fd640 1 --2- 172.21.15.67:0/316948960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.780+0000 7f35c24ff640 1 --2- 172.21.15.67:0/316948960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b113520 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009a80 tx=0x7f35ac02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd37f116d2c25a3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.780+0000 7f35a37fe640 1 -- 172.21.15.67:0/316948960 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac016070 con 0x55575ad56070 2024-07-31T14:52:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.780+0000 7f35a37fe640 1 -- 172.21.15.67:0/316948960 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ac01f030 con 0x55575ad56070 2024-07-31T14:52:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.780+0000 7f35a37fe640 1 -- 172.21.15.67:0/316948960 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac01a3e0 con 0x55575ad56070 2024-07-31T14:52:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.781+0000 7f35ca4b3740 1 -- 172.21.15.67:0/316948960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 msgr2=0x55575b113520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.781+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/316948960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b113520 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009a80 tx=0x7f35ac02c040 comp rx=0 tx=0).stop 2024-07-31T14:52:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.781+0000 7f35ca4b3740 1 -- 172.21.15.67:0/316948960 shutdown_connections 2024-07-31T14:52:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.781+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/316948960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b113520 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.781+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/316948960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.781+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/316948960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad2d1b0 0x55575b0fed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.781+0000 7f35ca4b3740 1 -- 172.21.15.67:0/316948960 >> 172.21.15.67:0/316948960 conn(0x55575ad4a460 msgr2=0x55575b0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.781+0000 7f35ca4b3740 1 -- 172.21.15.67:0/316948960 shutdown_connections 2024-07-31T14:52:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.781+0000 7f35ca4b3740 1 -- 172.21.15.67:0/316948960 wait complete. 2024-07-31T14:52:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.781+0000 7f35ca4b3740 1 -- start start 2024-07-31T14:52:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.781+0000 7f35ca4b3740 1 Processor -- start 2024-07-31T14:52:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35ca4b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad2d1b0 0x55575b11c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35ca4b3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b11ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35ca4b3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b117aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35ca4b3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1b1b00 con 0x55575ad56070 2024-07-31T14:52:49.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35ca4b3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1b1430 con 0x55575ad35390 2024-07-31T14:52:49.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35ca4b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1ae670 con 0x55575ad2d1b0 2024-07-31T14:52:49.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35ca4b3740 10 client.0 _ll_register_callbacks cb 0x7f35b41084e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:49.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35c24ff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b117aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35c24ff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b117aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47110/0 (socket says 172.21.15.67:47110) 2024-07-31T14:52:49.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35c24ff640 1 -- 172.21.15.67:0/1135954552 learned_addr learned my addr 172.21.15.67:0/1135954552 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:49.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35c24ff640 1 --2- 172.21.15.67:0/1135954552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad2d1b0 0x55575b11c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35c14fd640 1 --2- 172.21.15.67:0/1135954552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b11ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35c24ff640 1 -- 172.21.15.67:0/1135954552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad2d1b0 msgr2=0x55575b11c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:49.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35c24ff640 1 --2- 172.21.15.67:0/1135954552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad2d1b0 0x55575b11c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35c24ff640 1 -- 172.21.15.67:0/1135954552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 msgr2=0x55575b11ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:49.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35c24ff640 1 --2- 172.21.15.67:0/1135954552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b11ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35c24ff640 1 -- 172.21.15.67:0/1135954552 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575b118230 con 0x55575ad56070 2024-07-31T14:52:49.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35c14fd640 1 --2- 172.21.15.67:0/1135954552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b11ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:49.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35c24ff640 1 --2- 172.21.15.67:0/1135954552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b117aa0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f35ac005240 tx=0x7f35ac005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fdb2d88bdce48e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:49.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.782+0000 7f35a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:49.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.783+0000 7f35a17fa640 1 -- 172.21.15.67:0/1135954552 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac016070 con 0x55575ad56070 2024-07-31T14:52:49.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.783+0000 7f35a17fa640 1 -- 172.21.15.67:0/1135954552 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f35ac012450 con 0x55575ad56070 2024-07-31T14:52:49.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.783+0000 7f35ca4b3740 1 -- 172.21.15.67:0/1135954552 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35ac01f030 con 0x55575ad56070 2024-07-31T14:52:49.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.783+0000 7f35a17fa640 1 -- 172.21.15.67:0/1135954552 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f35ac01a5f0 con 0x55575ad56070 2024-07-31T14:52:49.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.783+0000 7f35ca4b3740 10 client.16095 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:49.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.783+0000 7f35ca4b3740 1 -- 172.21.15.67:0/1135954552 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55575b181130 con 0x55575ad56070 2024-07-31T14:52:49.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.783+0000 7f35a17fa640 1 -- 172.21.15.67:0/1135954552 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ac01f030 con 0x55575ad56070 2024-07-31T14:52:49.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.783+0000 7f35a17fa640 1 -- 172.21.15.67:0/1135954552 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f35ac038bf0 con 0x55575ad56070 2024-07-31T14:52:49.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.783+0000 7f35ca4b3740 10 client.16095 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:49.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.783+0000 7f35ca4b3740 10 client.16095 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:52:49.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.783+0000 7f35ca4b3740 20 client.16095 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:49.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.783+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/1135954552 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55575ace9da0 0x55575b183820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:49.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.783+0000 7f35ca4b3740 4 client.16095 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:52:49.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.783+0000 7f35ca4b3740 1 -- 172.21.15.67:0/1135954552 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55575b102b10 con 0x55575ace9da0 2024-07-31T14:52:49.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.784+0000 7f35c14fd640 1 --2- 172.21.15.67:0/1135954552 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55575ace9da0 0x55575b183820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:49.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.784+0000 7f35c14fd640 1 --2- 172.21.15.67:0/1135954552 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55575ace9da0 0x55575b183820 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=855a27142e10dc1c server_cookie=990237845d0f424b in_seq=0 out_seq=0 2024-07-31T14:52:49.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.784+0000 7f35a17fa640 10 client.16095 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:52:49.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.784+0000 7f35a17fa640 1 -- 172.21.15.67:0/1135954552 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55575b102b10 con 0x55575ace9da0 2024-07-31T14:52:49.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.784+0000 7f35a17fa640 10 client.16095 handle_command_reply: tid=0 2024-07-31T14:52:49.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.784+0000 7f35ca4b3740 1 client.16095 shutdown 2024-07-31T14:52:49.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.784+0000 7f35ca4b3740 1 -- 172.21.15.67:0/1135954552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 msgr2=0x55575b117aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:49.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.784+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/1135954552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b117aa0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f35ac005240 tx=0x7f35ac005d00 comp rx=0 tx=0).stop 2024-07-31T14:52:49.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.784+0000 7f35ca4b3740 1 -- 172.21.15.67:0/1135954552 shutdown_connections 2024-07-31T14:52:49.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/1135954552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55575ad56070 0x55575b117aa0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/1135954552 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55575ace9da0 0x55575b183820 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/1135954552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55575ad35390 0x55575b11ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35ca4b3740 1 --2- 172.21.15.67:0/1135954552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55575ad2d1b0 0x55575b11c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35a17fa640 0 client.16095 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:52:49.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35ca4b3740 1 -- 172.21.15.67:0/1135954552 >> 172.21.15.67:0/1135954552 conn(0x55575ad4a460 msgr2=0x55575b105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:49.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35ca4b3740 1 -- 172.21.15.67:0/1135954552 shutdown_connections 2024-07-31T14:52:49.818 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:52:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35ca4b3740 1 -- 172.21.15.67:0/1135954552 wait complete. 2024-07-31T14:52:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35ca4b3740 20 client.16095 trim_cache size 0 max 16384 2024-07-31T14:52:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35c3f62640 1 -- 172.21.15.67:0/2291492271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f35a40b3d40 msgr2=0x7f35a40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35c3f62640 1 --2- 172.21.15.67:0/2291492271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f35a40b3d40 0x7f35a40b6250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55575b0fa1f0 tx=0x7f35b4108610 comp rx=0 tx=0).stop 2024-07-31T14:52:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35c3f62640 1 -- 172.21.15.67:0/2291492271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35bc102490 msgr2=0x7f35bc199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35c3f62640 1 --2- 172.21.15.67:0/2291492271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35bc102490 0x7f35bc199e80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f35b4005940 tx=0x7f35b40042f0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35c3f62640 1 -- 172.21.15.67:0/2291492271 shutdown_connections 2024-07-31T14:52:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35c3f62640 1 --2- 172.21.15.67:0/2291492271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f35a40b3d40 0x7f35a40b6250 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35c3f62640 1 --2- 172.21.15.67:0/2291492271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35bc108720 0x7f35bc19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35c3f62640 1 --2- 172.21.15.67:0/2291492271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35bc107dd0 0x7f35bc19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35c3f62640 1 --2- 172.21.15.67:0/2291492271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35bc102490 0x7f35bc199e80 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35c3f62640 1 -- 172.21.15.67:0/2291492271 >> 172.21.15.67:0/2291492271 conn(0x7f35bc079160 msgr2=0x7f35bc100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35c3f62640 1 -- 172.21.15.67:0/2291492271 shutdown_connections 2024-07-31T14:52:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:49.785+0000 7f35c3f62640 1 -- 172.21.15.67:0/2291492271 wait complete. 2024-07-31T14:52:50.757 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T14:52:50.911 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-07-31T14:52:50.967 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-07-31T14:52:51.011 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-31T14:52:51.012 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-31T14:52:51.012 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-07-31T14:52:51.035 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:51.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:50 smithi067 ceph-mon[32750]: pgmap v452: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-07-31T14:52:51.057 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-31T14:52:51.060 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T14:52:51.061 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-07-31T14:52:51.078 INFO:tasks.workunit:Running workunits matching kernel_untar_build.sh on client.0... 2024-07-31T14:52:51.079 INFO:tasks.workunit:Running workunit kernel_untar_build.sh... 2024-07-31T14:52:51.079 DEBUG:teuthology.orchestra.run.smithi067:workunit test kernel_untar_build.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8c796fca5ff5960f7542e7227bad4a784ee2c804 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh 2024-07-31T14:52:51.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:50 smithi073 ceph-mon[36714]: pgmap v452: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-07-31T14:52:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:50 smithi077 ceph-mon[37327]: pgmap v452: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-07-31T14:52:52.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.819+0000 7f766d7c8640 1 -- 172.21.15.67:0/372545894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7668107ea0 msgr2=0x7f766810e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:52.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.819+0000 7f766d7c8640 1 --2- 172.21.15.67:0/372545894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7668107ea0 0x7f766810e6d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7660009ca0 tx=0x7f766002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.819+0000 7f766d7c8640 1 -- 172.21.15.67:0/372545894 shutdown_connections 2024-07-31T14:52:52.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.819+0000 7f766d7c8640 1 --2- 172.21.15.67:0/372545894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7668107ea0 0x7f766810e6d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.819+0000 7f766d7c8640 1 --2- 172.21.15.67:0/372545894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7668107550 0x7f7668107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.819+0000 7f766d7c8640 1 --2- 172.21.15.67:0/372545894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7668101c60 0x7f7668102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.819+0000 7f766d7c8640 1 -- 172.21.15.67:0/372545894 >> 172.21.15.67:0/372545894 conn(0x7f7668077690 msgr2=0x7f7668077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.819+0000 7f766d7c8640 1 -- 172.21.15.67:0/372545894 shutdown_connections 2024-07-31T14:52:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.819+0000 7f766d7c8640 1 -- 172.21.15.67:0/372545894 wait complete. 2024-07-31T14:52:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.820+0000 7f766d7c8640 1 Processor -- start 2024-07-31T14:52:52.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.820+0000 7f766d7c8640 1 -- start start 2024-07-31T14:52:52.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f766d7c8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7668101c60 0x7f7668198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f766d7c8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7668107550 0x7f7668199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f766d7c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7668107ea0 0x7f766819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f766d7c8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7668112440 con 0x7f7668101c60 2024-07-31T14:52:52.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f766d7c8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f76681122c0 con 0x7f7668107ea0 2024-07-31T14:52:52.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f766d7c8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f76681125c0 con 0x7f7668107550 2024-07-31T14:52:52.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f7666ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7668101c60 0x7f7668198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f7666ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7668101c60 0x7f7668198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32900/0 (socket says 172.21.15.67:32900) 2024-07-31T14:52:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f7666ffd640 1 -- 172.21.15.67:0/3104053425 learned_addr learned my addr 172.21.15.67:0/3104053425 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f76667fc640 1 --2- 172.21.15.67:0/3104053425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7668107550 0x7f7668199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f76677fe640 1 --2- 172.21.15.67:0/3104053425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7668107ea0 0x7f766819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f7666ffd640 1 -- 172.21.15.67:0/3104053425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7668107550 msgr2=0x7f7668199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f7666ffd640 1 --2- 172.21.15.67:0/3104053425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7668107550 0x7f7668199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f7666ffd640 1 -- 172.21.15.67:0/3104053425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7668107ea0 msgr2=0x7f766819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f7666ffd640 1 --2- 172.21.15.67:0/3104053425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7668107ea0 0x7f766819e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.821+0000 7f7666ffd640 1 -- 172.21.15.67:0/3104053425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766819ece0 con 0x7f7668101c60 2024-07-31T14:52:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.822+0000 7f7666ffd640 1 --2- 172.21.15.67:0/3104053425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7668101c60 0x7f7668198d70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f765800e9f0 tx=0x7f765800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.822+0000 7f764ffff640 1 -- 172.21.15.67:0/3104053425 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765800cd80 con 0x7f7668101c60 2024-07-31T14:52:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.822+0000 7f766d7c8640 1 -- 172.21.15.67:0/3104053425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f766819efd0 con 0x7f7668101c60 2024-07-31T14:52:52.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.822+0000 7f764ffff640 1 -- 172.21.15.67:0/3104053425 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76580054d0 con 0x7f7668101c60 2024-07-31T14:52:52.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.822+0000 7f764ffff640 1 -- 172.21.15.67:0/3104053425 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658010690 con 0x7f7668101c60 2024-07-31T14:52:52.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.822+0000 7f766d7c8640 1 -- 172.21.15.67:0/3104053425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76681a5730 con 0x7f7668101c60 2024-07-31T14:52:52.826 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:52 smithi067 ceph-mon[32750]: pgmap v453: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-31T14:52:52.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.824+0000 7f764ffff640 1 -- 172.21.15.67:0/3104053425 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7658004e90 con 0x7f7668101c60 2024-07-31T14:52:52.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.825+0000 7f764ffff640 1 --2- 172.21.15.67:0/3104053425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f76500b3d40 0x7f76500b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.825+0000 7f76667fc640 1 --2- 172.21.15.67:0/3104053425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f76500b3d40 0x7f76500b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.825+0000 7f7673d19740 1 Processor -- start 2024-07-31T14:52:52.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.825+0000 7f764ffff640 1 -- 172.21.15.67:0/3104053425 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f76580e4750 con 0x7f7668101c60 2024-07-31T14:52:52.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.826+0000 7f7673d19740 1 -- start start 2024-07-31T14:52:52.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.826+0000 7f7673d19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e8a070 0x55f7742341e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.826+0000 7f7673d19740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e611b0 0x55f774231780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.826+0000 7f7673d19740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e69390 0x55f774231e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.826+0000 7f7673d19740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f7741a5bb0 con 0x55f773e611b0 2024-07-31T14:52:52.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.826+0000 7f7673d19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f7741950b0 con 0x55f773e8a070 2024-07-31T14:52:52.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.826+0000 7f7673d19740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f7741990f0 con 0x55f773e69390 2024-07-31T14:52:52.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.826+0000 7f76667fc640 1 --2- 172.21.15.67:0/3104053425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f76500b3d40 0x7f76500b6250 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f76681051c0 tx=0x7f765c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:52.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.826+0000 7f76667fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e611b0 0x55f774231780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.826+0000 7f76667fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e611b0 0x55f774231780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32908/0 (socket says 172.21.15.67:32908) 2024-07-31T14:52:52.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.826+0000 7f76667fc640 1 -- 172.21.15.67:0/2444247847 learned_addr learned my addr 172.21.15.67:0/2444247847 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:52.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.826+0000 7f76677fe640 1 --2- 172.21.15.67:0/2444247847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e69390 0x55f774231e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.826+0000 7f7666ffd640 1 --2- 172.21.15.67:0/2444247847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e8a070 0x55f7742341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.827+0000 7f76667fc640 1 -- 172.21.15.67:0/2444247847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e69390 msgr2=0x55f774231e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:52.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.827+0000 7f76667fc640 1 --2- 172.21.15.67:0/2444247847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e69390 0x55f774231e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.827+0000 7f76667fc640 1 -- 172.21.15.67:0/2444247847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e8a070 msgr2=0x55f7742341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:52.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.827+0000 7f76667fc640 1 --2- 172.21.15.67:0/2444247847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e8a070 0x55f7742341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.827+0000 7f76667fc640 1 -- 172.21.15.67:0/2444247847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f77413bb90 con 0x55f773e611b0 2024-07-31T14:52:52.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.827+0000 7f76677fe640 1 --2- 172.21.15.67:0/2444247847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e69390 0x55f774231e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:52.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.827+0000 7f76667fc640 1 --2- 172.21.15.67:0/2444247847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e611b0 0x55f774231780 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f765c00e610 tx=0x7f765c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3603a5807abf5359 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:52.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.827+0000 7f764d7fa640 1 -- 172.21.15.67:0/2444247847 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c026430 con 0x55f773e611b0 2024-07-31T14:52:52.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.827+0000 7f764d7fa640 1 -- 172.21.15.67:0/2444247847 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f765c01d8c0 con 0x55f773e611b0 2024-07-31T14:52:52.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.827+0000 7f764d7fa640 1 -- 172.21.15.67:0/2444247847 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c025940 con 0x55f773e611b0 2024-07-31T14:52:52.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.828+0000 7f7673d19740 1 -- 172.21.15.67:0/2444247847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e611b0 msgr2=0x55f774231780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:52.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.828+0000 7f7673d19740 1 --2- 172.21.15.67:0/2444247847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e611b0 0x55f774231780 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f765c00e610 tx=0x7f765c014000 comp rx=0 tx=0).stop 2024-07-31T14:52:52.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.828+0000 7f7673d19740 1 -- 172.21.15.67:0/2444247847 shutdown_connections 2024-07-31T14:52:52.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.828+0000 7f7673d19740 1 --2- 172.21.15.67:0/2444247847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e69390 0x55f774231e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.828+0000 7f7673d19740 1 --2- 172.21.15.67:0/2444247847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e611b0 0x55f774231780 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.828+0000 7f7673d19740 1 --2- 172.21.15.67:0/2444247847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e8a070 0x55f7742341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.828+0000 7f7673d19740 1 -- 172.21.15.67:0/2444247847 >> 172.21.15.67:0/2444247847 conn(0x55f773e7e460 msgr2=0x55f77422f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:52.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.828+0000 7f7673d19740 1 -- 172.21.15.67:0/2444247847 shutdown_connections 2024-07-31T14:52:52.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.828+0000 7f7673d19740 1 -- 172.21.15.67:0/2444247847 wait complete. 2024-07-31T14:52:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.829+0000 7f7673d19740 1 -- start start 2024-07-31T14:52:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.829+0000 7f7673d19740 1 Processor -- start 2024-07-31T14:52:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.829+0000 7f7673d19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e611b0 0x55f77423f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.829+0000 7f7673d19740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e69390 0x55f77423fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.829+0000 7f7673d19740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e8a070 0x55f77424b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.829+0000 7f7673d19740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f773ff8700 con 0x55f773e8a070 2024-07-31T14:52:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.829+0000 7f7673d19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f7741379e0 con 0x55f773e611b0 2024-07-31T14:52:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.829+0000 7f7673d19740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f7742420d0 con 0x55f773e69390 2024-07-31T14:52:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.829+0000 7f7673d19740 10 client.0 _ll_register_callbacks cb 0x55f7741524f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.829+0000 7f76677fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e8a070 0x55f77424b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.829+0000 7f76677fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e8a070 0x55f77424b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32914/0 (socket says 172.21.15.67:32914) 2024-07-31T14:52:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.829+0000 7f76677fe640 1 -- 172.21.15.67:0/1337606256 learned_addr learned my addr 172.21.15.67:0/1337606256 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f7666ffd640 1 --2- 172.21.15.67:0/1337606256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e611b0 0x55f77423f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f76667fc640 1 --2- 172.21.15.67:0/1337606256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e69390 0x55f77423fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f76677fe640 1 -- 172.21.15.67:0/1337606256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e69390 msgr2=0x55f77423fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:52.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f76677fe640 1 --2- 172.21.15.67:0/1337606256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e69390 0x55f77423fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f76677fe640 1 -- 172.21.15.67:0/1337606256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e611b0 msgr2=0x55f77423f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:52.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f76677fe640 1 --2- 172.21.15.67:0/1337606256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e611b0 0x55f77423f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f76677fe640 1 -- 172.21.15.67:0/1337606256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f773ef5650 con 0x55f773e8a070 2024-07-31T14:52:52.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f7666ffd640 1 --2- 172.21.15.67:0/1337606256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e611b0 0x55f77423f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:52.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f76677fe640 1 --2- 172.21.15.67:0/1337606256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e8a070 0x55f77424b430 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7660009ca0 tx=0x7f7660009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda74139e8bd5f16 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:52.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f764aff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:52.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f764aff5640 1 -- 172.21.15.67:0/1337606256 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660031720 con 0x55f773e8a070 2024-07-31T14:52:52.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f764aff5640 1 -- 172.21.15.67:0/1337606256 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f76600318c0 con 0x55f773e8a070 2024-07-31T14:52:52.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f7673d19740 1 -- 172.21.15.67:0/1337606256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f765c01d8c0 con 0x55f773e8a070 2024-07-31T14:52:52.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f764aff5640 1 -- 172.21.15.67:0/1337606256 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7660050050 con 0x55f773e8a070 2024-07-31T14:52:52.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f7673d19740 10 client.16107 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:52.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.830+0000 7f7673d19740 1 -- 172.21.15.67:0/1337606256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f77424e100 con 0x55f773e8a070 2024-07-31T14:52:52.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.831+0000 7f764aff5640 1 -- 172.21.15.67:0/1337606256 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f765c01d8c0 con 0x55f773e8a070 2024-07-31T14:52:52.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.831+0000 7f764aff5640 1 -- 172.21.15.67:0/1337606256 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f766002f610 con 0x55f773e8a070 2024-07-31T14:52:52.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.831+0000 7f7673d19740 10 client.16107 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:52.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.831+0000 7f7673d19740 10 client.16107 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:52:52.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.831+0000 7f7673d19740 20 client.16107 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:52.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.831+0000 7f7673d19740 1 --2- 172.21.15.67:0/1337606256 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f773e489f0 0x55f774251630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.831+0000 7f7673d19740 4 client.16107 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:52.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.831+0000 7f7673d19740 1 -- 172.21.15.67:0/1337606256 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7741a7b00 con 0x55f773e489f0 2024-07-31T14:52:52.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.831+0000 7f7666ffd640 1 --2- 172.21.15.67:0/1337606256 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f773e489f0 0x55f774251630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.832+0000 7f7666ffd640 1 --2- 172.21.15.67:0/1337606256 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f773e489f0 0x55f774251630 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9f8bef88ba42fe0 server_cookie=23e712651f35a852 in_seq=0 out_seq=0 2024-07-31T14:52:52.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.832+0000 7f764aff5640 10 client.16107 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:52:52.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.832+0000 7f764aff5640 1 -- 172.21.15.67:0/1337606256 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7741a7b00 con 0x55f773e489f0 2024-07-31T14:52:52.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.832+0000 7f764aff5640 10 client.16107 handle_command_reply: tid=0 2024-07-31T14:52:52.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.832+0000 7f7673d19740 1 client.16107 shutdown 2024-07-31T14:52:52.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.833+0000 7f7673d19740 1 -- 172.21.15.67:0/1337606256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e8a070 msgr2=0x55f77424b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:52.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.833+0000 7f7673d19740 1 --2- 172.21.15.67:0/1337606256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e8a070 0x55f77424b430 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7660009ca0 tx=0x7f7660009910 comp rx=0 tx=0).stop 2024-07-31T14:52:52.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.833+0000 7f7673d19740 1 -- 172.21.15.67:0/1337606256 shutdown_connections 2024-07-31T14:52:52.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.833+0000 7f7673d19740 1 --2- 172.21.15.67:0/1337606256 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f773e489f0 0x55f774251630 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.833+0000 7f7673d19740 1 --2- 172.21.15.67:0/1337606256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e8a070 0x55f77424b430 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.833+0000 7f7673d19740 1 --2- 172.21.15.67:0/1337606256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e69390 0x55f77423fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.833+0000 7f7673d19740 1 --2- 172.21.15.67:0/1337606256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e611b0 0x55f77423f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.833+0000 7f7673d19740 1 -- 172.21.15.67:0/1337606256 >> 172.21.15.67:0/1337606256 conn(0x55f773e7e460 msgr2=0x55f774233170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:52.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.833+0000 7f764aff5640 0 client.16107 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:52:52.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.833+0000 7f7673d19740 1 -- 172.21.15.67:0/1337606256 shutdown_connections 2024-07-31T14:52:52.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.833+0000 7f7673d19740 1 -- 172.21.15.67:0/1337606256 wait complete. 2024-07-31T14:52:52.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.833+0000 7f7673d19740 20 client.16107 trim_cache size 0 max 16384 2024-07-31T14:52:52.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f7673d19740 1 Processor -- start 2024-07-31T14:52:52.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f7673d19740 1 -- start start 2024-07-31T14:52:52.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f7673d19740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e611b0 0x55f7742358b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f7673d19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e69390 0x55f77423ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f7673d19740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e8a070 0x55f774232ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f7673d19740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f76681ae490 con 0x55f773e8a070 2024-07-31T14:52:52.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f7673d19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f76681affd0 con 0x55f773e69390 2024-07-31T14:52:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f7673d19740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f76681aeb60 con 0x55f773e611b0 2024-07-31T14:52:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f76677fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e8a070 0x55f774232ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f76677fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e8a070 0x55f774232ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32928/0 (socket says 172.21.15.67:32928) 2024-07-31T14:52:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f76677fe640 1 -- 172.21.15.67:0/3175954106 learned_addr learned my addr 172.21.15.67:0/3175954106 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f7666ffd640 1 --2- 172.21.15.67:0/3175954106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e611b0 0x55f7742358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f76667fc640 1 --2- 172.21.15.67:0/3175954106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e69390 0x55f77423ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f76677fe640 1 -- 172.21.15.67:0/3175954106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e611b0 msgr2=0x55f7742358b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f76677fe640 1 --2- 172.21.15.67:0/3175954106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e611b0 0x55f7742358b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f76677fe640 1 -- 172.21.15.67:0/3175954106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e69390 msgr2=0x55f77423ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f76677fe640 1 --2- 172.21.15.67:0/3175954106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e69390 0x55f77423ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f76677fe640 1 -- 172.21.15.67:0/3175954106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f77423d410 con 0x55f773e8a070 2024-07-31T14:52:52.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.847+0000 7f76667fc640 1 --2- 172.21.15.67:0/3175954106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e69390 0x55f77423ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:52.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.848+0000 7f76677fe640 1 --2- 172.21.15.67:0/3175954106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e8a070 0x55f774232ec0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7660006010 tx=0x7f7660031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94238460ff06162 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:52.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.848+0000 7f764cff9640 1 -- 172.21.15.67:0/3175954106 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766002f610 con 0x55f773e8a070 2024-07-31T14:52:52.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.848+0000 7f764cff9640 1 -- 172.21.15.67:0/3175954106 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7660036050 con 0x55f773e8a070 2024-07-31T14:52:52.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.848+0000 7f764cff9640 1 -- 172.21.15.67:0/3175954106 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660030d40 con 0x55f773e8a070 2024-07-31T14:52:52.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.848+0000 7f7673d19740 1 -- 172.21.15.67:0/3175954106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e8a070 msgr2=0x55f774232ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:52.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.848+0000 7f7673d19740 1 --2- 172.21.15.67:0/3175954106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e8a070 0x55f774232ec0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7660006010 tx=0x7f7660031950 comp rx=0 tx=0).stop 2024-07-31T14:52:52.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.848+0000 7f7673d19740 1 -- 172.21.15.67:0/3175954106 shutdown_connections 2024-07-31T14:52:52.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.848+0000 7f7673d19740 1 --2- 172.21.15.67:0/3175954106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e8a070 0x55f774232ec0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.848+0000 7f7673d19740 1 --2- 172.21.15.67:0/3175954106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e69390 0x55f77423ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.848+0000 7f7673d19740 1 --2- 172.21.15.67:0/3175954106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e611b0 0x55f7742358b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.849+0000 7f7673d19740 1 -- 172.21.15.67:0/3175954106 >> 172.21.15.67:0/3175954106 conn(0x55f773e7e460 msgr2=0x55f7742327b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:52.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.849+0000 7f7673d19740 1 -- 172.21.15.67:0/3175954106 shutdown_connections 2024-07-31T14:52:52.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.849+0000 7f7673d19740 1 -- 172.21.15.67:0/3175954106 wait complete. 2024-07-31T14:52:52.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.849+0000 7f7673d19740 1 -- start start 2024-07-31T14:52:52.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.849+0000 7f7673d19740 1 Processor -- start 2024-07-31T14:52:52.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.849+0000 7f7673d19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e611b0 0x55f774247390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.849+0000 7f7673d19740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e69390 0x55f774247930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.849+0000 7f7673d19740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e8a070 0x55f77424b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.849+0000 7f7673d19740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f76681af900 con 0x55f773e69390 2024-07-31T14:52:52.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.849+0000 7f7673d19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f76681af230 con 0x55f773e611b0 2024-07-31T14:52:52.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.849+0000 7f7673d19740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f76681ac470 con 0x55f773e8a070 2024-07-31T14:52:52.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.849+0000 7f7673d19740 10 client.0 _ll_register_callbacks cb 0x7f7660009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:52.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.849+0000 7f76667fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e69390 0x55f774247930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f76677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e8a070 0x55f77424b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f76667fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e69390 0x55f774247930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32936/0 (socket says 172.21.15.67:32936) 2024-07-31T14:52:52.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f76667fc640 1 -- 172.21.15.67:0/2687643348 learned_addr learned my addr 172.21.15.67:0/2687643348 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:52.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f7666ffd640 1 --2- 172.21.15.67:0/2687643348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e611b0 0x55f774247390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f76667fc640 1 -- 172.21.15.67:0/2687643348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e8a070 msgr2=0x55f77424b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:52.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f76667fc640 1 --2- 172.21.15.67:0/2687643348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e8a070 0x55f77424b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f76667fc640 1 -- 172.21.15.67:0/2687643348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e611b0 msgr2=0x55f774247390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f76667fc640 1 --2- 172.21.15.67:0/2687643348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e611b0 0x55f774247390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f76667fc640 1 -- 172.21.15.67:0/2687643348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f77424b810 con 0x55f773e69390 2024-07-31T14:52:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f76677fe640 1 --2- 172.21.15.67:0/2687643348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e8a070 0x55f77424b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:52:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f76667fc640 1 --2- 172.21.15.67:0/2687643348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e69390 0x55f774247930 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x55f774230400 tx=0x7f765c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d541b72945bc99bc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f764aff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f764aff5640 1 -- 172.21.15.67:0/2687643348 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c034060 con 0x55f773e69390 2024-07-31T14:52:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f764aff5640 1 -- 172.21.15.67:0/2687643348 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f765c009050 con 0x55f773e69390 2024-07-31T14:52:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f7673d19740 1 -- 172.21.15.67:0/2687643348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7660036050 con 0x55f773e69390 2024-07-31T14:52:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f764aff5640 1 -- 172.21.15.67:0/2687643348 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f765c004020 con 0x55f773e69390 2024-07-31T14:52:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.850+0000 7f7673d19740 10 client.16113 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.851+0000 7f7673d19740 1 -- 172.21.15.67:0/2687643348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f774256ae0 con 0x55f773e69390 2024-07-31T14:52:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.851+0000 7f764aff5640 1 -- 172.21.15.67:0/2687643348 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7660036050 con 0x55f773e69390 2024-07-31T14:52:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.851+0000 7f764aff5640 1 -- 172.21.15.67:0/2687643348 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f765c0047a0 con 0x55f773e69390 2024-07-31T14:52:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.851+0000 7f7673d19740 10 client.16113 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.851+0000 7f7673d19740 10 client.16113 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:52:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.851+0000 7f7673d19740 20 client.16113 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.851+0000 7f7673d19740 1 --2- 172.21.15.67:0/2687643348 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f773e1dda0 0x55f7742bbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.851+0000 7f7673d19740 4 client.16113 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:52:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.851+0000 7f7673d19740 1 -- 172.21.15.67:0/2687643348 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f7741a35a0 con 0x55f773e1dda0 2024-07-31T14:52:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.851+0000 7f7666ffd640 1 --2- 172.21.15.67:0/2687643348 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f773e1dda0 0x55f7742bbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f7666ffd640 1 --2- 172.21.15.67:0/2687643348 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f773e1dda0 0x55f7742bbdd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15a94f22666fdf58 server_cookie=f145f313099ec61d in_seq=0 out_seq=0 2024-07-31T14:52:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f764aff5640 10 client.16113 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:52:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f764aff5640 1 -- 172.21.15.67:0/2687643348 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f7741a35a0 con 0x55f773e1dda0 2024-07-31T14:52:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f764aff5640 10 client.16113 handle_command_reply: tid=0 2024-07-31T14:52:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f7673d19740 1 client.16113 shutdown 2024-07-31T14:52:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f7673d19740 1 -- 172.21.15.67:0/2687643348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e69390 msgr2=0x55f774247930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f7673d19740 1 --2- 172.21.15.67:0/2687643348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e69390 0x55f774247930 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x55f774230400 tx=0x7f765c009900 comp rx=0 tx=0).stop 2024-07-31T14:52:52.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f7673d19740 1 -- 172.21.15.67:0/2687643348 shutdown_connections 2024-07-31T14:52:52.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f7673d19740 1 --2- 172.21.15.67:0/2687643348 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f773e1dda0 0x55f7742bbdd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f7673d19740 1 --2- 172.21.15.67:0/2687643348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f773e8a070 0x55f77424b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f7673d19740 1 --2- 172.21.15.67:0/2687643348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f773e69390 0x55f774247930 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f7673d19740 1 --2- 172.21.15.67:0/2687643348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f773e611b0 0x55f774247390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f764aff5640 0 client.16113 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:52:52.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f7673d19740 1 -- 172.21.15.67:0/2687643348 >> 172.21.15.67:0/2687643348 conn(0x55f773e7e460 msgr2=0x55f774241100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:52.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f7673d19740 1 -- 172.21.15.67:0/2687643348 shutdown_connections 2024-07-31T14:52:52.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f7673d19740 1 -- 172.21.15.67:0/2687643348 wait complete. 2024-07-31T14:52:52.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.852+0000 7f7673d19740 20 client.16113 trim_cache size 0 max 16384 2024-07-31T14:52:52.860 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:52:52.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.853+0000 7f766d7c8640 1 -- 172.21.15.67:0/3104053425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f76500b3d40 msgr2=0x7f76500b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:52.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.853+0000 7f766d7c8640 1 --2- 172.21.15.67:0/3104053425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f76500b3d40 0x7f76500b6250 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f76681051c0 tx=0x7f765c005f90 comp rx=0 tx=0).stop 2024-07-31T14:52:52.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.853+0000 7f766d7c8640 1 -- 172.21.15.67:0/3104053425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7668101c60 msgr2=0x7f7668198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:52.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.853+0000 7f766d7c8640 1 --2- 172.21.15.67:0/3104053425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7668101c60 0x7f7668198d70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f765800e9f0 tx=0x7f765800edf0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.853+0000 7f766d7c8640 1 -- 172.21.15.67:0/3104053425 shutdown_connections 2024-07-31T14:52:52.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.853+0000 7f766d7c8640 1 --2- 172.21.15.67:0/3104053425 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f76500b3d40 0x7f76500b6250 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.853+0000 7f766d7c8640 1 --2- 172.21.15.67:0/3104053425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7668107ea0 0x7f766819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.853+0000 7f766d7c8640 1 --2- 172.21.15.67:0/3104053425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7668107550 0x7f7668199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.853+0000 7f766d7c8640 1 --2- 172.21.15.67:0/3104053425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7668101c60 0x7f7668198d70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.853+0000 7f766d7c8640 1 -- 172.21.15.67:0/3104053425 >> 172.21.15.67:0/3104053425 conn(0x7f7668077690 msgr2=0x7f7668103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.853+0000 7f766d7c8640 1 -- 172.21.15.67:0/3104053425 shutdown_connections 2024-07-31T14:52:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:52.853+0000 7f766d7c8640 1 -- 172.21.15.67:0/3104053425 wait complete. 2024-07-31T14:52:53.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:52 smithi073 ceph-mon[36714]: pgmap v453: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-31T14:52:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:52 smithi077 ceph-mon[37327]: pgmap v453: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-31T14:52:53.580 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:52:53.640 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T14:52:53.870 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:54.553 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:54.943 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:54 smithi067 ceph-mon[32750]: pgmap v454: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-31T14:52:55.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:54 smithi073 ceph-mon[36714]: pgmap v454: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-31T14:52:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:54 smithi077 ceph-mon[37327]: pgmap v454: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-31T14:52:56.881 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:56 smithi067 ceph-mon[32750]: pgmap v455: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-31T14:52:57.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.106+0000 7fd28ac99640 1 -- 172.21.15.67:0/973785187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd284109820 msgr2=0x7fd284110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:57.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.106+0000 7fd28ac99640 1 --2- 172.21.15.67:0/973785187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd284109820 0x7fd284110050 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009ca0 tx=0x7fd27c02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.106+0000 7fd28ac99640 1 -- 172.21.15.67:0/973785187 shutdown_connections 2024-07-31T14:52:57.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.106+0000 7fd28ac99640 1 --2- 172.21.15.67:0/973785187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd284109820 0x7fd284110050 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.106+0000 7fd28ac99640 1 --2- 172.21.15.67:0/973785187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd284108ed0 0x7fd2841092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.106+0000 7fd28ac99640 1 --2- 172.21.15.67:0/973785187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd284103590 0x7fd2841039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.106+0000 7fd28ac99640 1 -- 172.21.15.67:0/973785187 >> 172.21.15.67:0/973785187 conn(0x7fd28407b280 msgr2=0x7fd28407b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:57.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.106+0000 7fd28ac99640 1 -- 172.21.15.67:0/973785187 shutdown_connections 2024-07-31T14:52:57.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.106+0000 7fd28ac99640 1 -- 172.21.15.67:0/973785187 wait complete. 2024-07-31T14:52:57.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd28ac99640 1 Processor -- start 2024-07-31T14:52:57.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd28ac99640 1 -- start start 2024-07-31T14:52:57.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd28ac99640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd284103590 0x7fd284199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd28ac99640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd284108ed0 0x7fd28419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd28ac99640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd284109820 0x7fd28419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd28ac99640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd2841134f0 con 0x7fd284103590 2024-07-31T14:52:57.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd28ac99640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd284113370 con 0x7fd284109820 2024-07-31T14:52:57.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd28ac99640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd284113670 con 0x7fd284108ed0 2024-07-31T14:52:57.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd288a35640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd284103590 0x7fd284199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd288a35640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd284103590 0x7fd284199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32950/0 (socket says 172.21.15.67:32950) 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd288a35640 1 -- 172.21.15.67:0/2992327717 learned_addr learned my addr 172.21.15.67:0/2992327717 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd283fff640 1 --2- 172.21.15.67:0/2992327717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd284108ed0 0x7fd28419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd289236640 1 --2- 172.21.15.67:0/2992327717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd284109820 0x7fd28419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd288a35640 1 -- 172.21.15.67:0/2992327717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd284108ed0 msgr2=0x7fd28419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd288a35640 1 --2- 172.21.15.67:0/2992327717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd284108ed0 0x7fd28419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd288a35640 1 -- 172.21.15.67:0/2992327717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd284109820 msgr2=0x7fd28419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd288a35640 1 --2- 172.21.15.67:0/2992327717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd284109820 0x7fd28419f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.107+0000 7fd288a35640 1 -- 172.21.15.67:0/2992327717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28419fe10 con 0x7fd284103590 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.108+0000 7fd288a35640 1 --2- 172.21.15.67:0/2992327717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd284103590 0x7fd284199ea0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd27800e9f0 tx=0x7fd27800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.108+0000 7fd281ffb640 1 -- 172.21.15.67:0/2992327717 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27800cd80 con 0x7fd284103590 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.108+0000 7fd28ac99640 1 -- 172.21.15.67:0/2992327717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2841a0100 con 0x7fd284103590 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.108+0000 7fd281ffb640 1 -- 172.21.15.67:0/2992327717 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2780054d0 con 0x7fd284103590 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.108+0000 7fd281ffb640 1 -- 172.21.15.67:0/2992327717 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd278010690 con 0x7fd284103590 2024-07-31T14:52:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.108+0000 7fd28ac99640 1 -- 172.21.15.67:0/2992327717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2841a6860 con 0x7fd284103590 2024-07-31T14:52:57.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.109+0000 7fd28ac99640 1 -- 172.21.15.67:0/2992327717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2841039a0 con 0x7fd284103590 2024-07-31T14:52:57.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.111+0000 7fd281ffb640 1 -- 172.21.15.67:0/2992327717 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd278004e30 con 0x7fd284103590 2024-07-31T14:52:57.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.112+0000 7fd281ffb640 1 --2- 172.21.15.67:0/2992327717 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd2740b3d40 0x7fd2740b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.112+0000 7fd281ffb640 1 -- 172.21.15.67:0/2992327717 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fd2780e49c0 con 0x7fd284103590 2024-07-31T14:52:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.112+0000 7fd283fff640 1 --2- 172.21.15.67:0/2992327717 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd2740b3d40 0x7fd2740b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.113+0000 7fd283fff640 1 --2- 172.21.15.67:0/2992327717 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd2740b3d40 0x7fd2740b6250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd2841069f0 tx=0x7fd27000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:57.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.116+0000 7fd281ffb640 1 -- 172.21.15.67:0/2992327717 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd2780555e0 con 0x7fd284103590 2024-07-31T14:52:57.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:56 smithi073 ceph-mon[36714]: pgmap v455: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-31T14:52:57.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:56 smithi077 ceph-mon[37327]: pgmap v455: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-31T14:52:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.315+0000 7fdc5f848640 1 -- 172.21.15.67:0/1193006943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc58108720 msgr2=0x7fdc5810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.315+0000 7fdc5f848640 1 --2- 172.21.15.67:0/1193006943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc58108720 0x7fdc5810ef50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009ca0 tx=0x7fdc5002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.316+0000 7fdc5f848640 1 -- 172.21.15.67:0/1193006943 shutdown_connections 2024-07-31T14:52:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.316+0000 7fdc5f848640 1 --2- 172.21.15.67:0/1193006943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc58108720 0x7fdc5810ef50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.316+0000 7fdc5f848640 1 --2- 172.21.15.67:0/1193006943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc58107dd0 0x7fdc581081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.316+0000 7fdc5f848640 1 --2- 172.21.15.67:0/1193006943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc58102490 0x7fdc581028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.316+0000 7fdc5f848640 1 -- 172.21.15.67:0/1193006943 >> 172.21.15.67:0/1193006943 conn(0x7fdc58079160 msgr2=0x7fdc58079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:57.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.316+0000 7fdc5f848640 1 -- 172.21.15.67:0/1193006943 shutdown_connections 2024-07-31T14:52:57.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.316+0000 7fdc5f848640 1 -- 172.21.15.67:0/1193006943 wait complete. 2024-07-31T14:52:57.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.316+0000 7fdc5f848640 1 Processor -- start 2024-07-31T14:52:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.316+0000 7fdc5f848640 1 -- start start 2024-07-31T14:52:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5f848640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc58102490 0x7fdc58199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5f848640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc58107dd0 0x7fdc5819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5f848640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc58108720 0x7fdc5819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5f848640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdc58112380 con 0x7fdc58108720 2024-07-31T14:52:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5f848640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdc58112200 con 0x7fdc58107dd0 2024-07-31T14:52:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5f848640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdc58112500 con 0x7fdc58102490 2024-07-31T14:52:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5dde5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc58108720 0x7fdc5819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5dde5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc58108720 0x7fdc5819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32976/0 (socket says 172.21.15.67:32976) 2024-07-31T14:52:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5dde5640 1 -- 172.21.15.67:0/2512798020 learned_addr learned my addr 172.21.15.67:0/2512798020 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/2512798020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc58107dd0 0x7fdc5819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5d5e4640 1 --2- 172.21.15.67:0/2512798020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc58102490 0x7fdc58199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5dde5640 1 -- 172.21.15.67:0/2512798020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc58102490 msgr2=0x7fdc58199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5dde5640 1 --2- 172.21.15.67:0/2512798020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc58102490 0x7fdc58199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5dde5640 1 -- 172.21.15.67:0/2512798020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc58107dd0 msgr2=0x7fdc5819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:57.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5dde5640 1 --2- 172.21.15.67:0/2512798020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc58107dd0 0x7fdc5819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.317+0000 7fdc5dde5640 1 -- 172.21.15.67:0/2512798020 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5819fe30 con 0x7fdc58108720 2024-07-31T14:52:57.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.318+0000 7fdc5dde5640 1 --2- 172.21.15.67:0/2512798020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc58108720 0x7fdc5819f700 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdc50005cc0 tx=0x7fdc5002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:57.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.318+0000 7fdc4e7fc640 1 -- 172.21.15.67:0/2512798020 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50003bf0 con 0x7fdc58108720 2024-07-31T14:52:57.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.318+0000 7fdc4e7fc640 1 -- 172.21.15.67:0/2512798020 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc50003d90 con 0x7fdc58108720 2024-07-31T14:52:57.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.318+0000 7fdc4e7fc640 1 -- 172.21.15.67:0/2512798020 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5002fbc0 con 0x7fdc58108720 2024-07-31T14:52:57.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.318+0000 7fdc5f848640 1 -- 172.21.15.67:0/2512798020 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc581a00c0 con 0x7fdc58108720 2024-07-31T14:52:57.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.318+0000 7fdc5f848640 1 -- 172.21.15.67:0/2512798020 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc581049f0 con 0x7fdc58108720 2024-07-31T14:52:57.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.321+0000 7fdc65d99740 1 Processor -- start 2024-07-31T14:52:57.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.321+0000 7fdc4e7fc640 1 -- 172.21.15.67:0/2512798020 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fdc5002fd60 con 0x7fdc58108720 2024-07-31T14:52:57.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.321+0000 7fdc65d99740 1 -- start start 2024-07-31T14:52:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc65d99740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55989172f070 0x559891ad4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc65d99740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598917061b0 0x559891ad4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc65d99740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55989170e390 0x559891ad4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc65d99740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559891a4abb0 con 0x55989172f070 2024-07-31T14:52:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc65d99740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559891a3a0b0 con 0x55989170e390 2024-07-31T14:52:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc65d99740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559891a3e0f0 con 0x5598917061b0 2024-07-31T14:52:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc5cde3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55989172f070 0x559891ad4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc5cde3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55989172f070 0x559891ad4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32984/0 (socket says 172.21.15.67:32984) 2024-07-31T14:52:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc5cde3640 1 -- 172.21.15.67:0/435928570 learned_addr learned my addr 172.21.15.67:0/435928570 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc5d5e4640 1 --2- 172.21.15.67:0/435928570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598917061b0 0x559891ad4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc4e7fc640 1 --2- 172.21.15.67:0/2512798020 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdc440b3d40 0x7fdc440b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc5cde3640 1 -- 172.21.15.67:0/435928570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598917061b0 msgr2=0x559891ad4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:57.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/435928570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598917061b0 0x559891ad4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc5cde3640 1 -- 172.21.15.67:0/435928570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55989170e390 msgr2=0x559891ad4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:57.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/435928570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55989170e390 0x559891ad4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc5cde3640 1 -- 172.21.15.67:0/435928570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598919e0b90 con 0x55989172f070 2024-07-31T14:52:57.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.322+0000 7fdc5dde5640 1 --2- 172.21.15.67:0/2512798020 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdc440b3d40 0x7fdc440b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.323+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/435928570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55989172f070 0x559891ad4570 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800bc90 tx=0x7fdc480127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2651ad785f61374a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:57.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.323+0000 7fdc4e7fc640 1 -- 172.21.15.67:0/2512798020 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fdc50105980 con 0x7fdc58108720 2024-07-31T14:52:57.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.323+0000 7fdc437fe640 1 -- 172.21.15.67:0/435928570 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc480120d0 con 0x55989172f070 2024-07-31T14:52:57.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.323+0000 7fdc5dde5640 1 --2- 172.21.15.67:0/2512798020 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdc440b3d40 0x7fdc440b6250 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdc500c8410 tx=0x7fdc500c6910 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:57.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.323+0000 7fdc437fe640 1 -- 172.21.15.67:0/435928570 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc48016070 con 0x55989172f070 2024-07-31T14:52:57.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.323+0000 7fdc437fe640 1 -- 172.21.15.67:0/435928570 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48010430 con 0x55989172f070 2024-07-31T14:52:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.324+0000 7fdc65d99740 1 -- 172.21.15.67:0/435928570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55989172f070 msgr2=0x559891ad4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.324+0000 7fdc65d99740 1 --2- 172.21.15.67:0/435928570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55989172f070 0x559891ad4570 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800bc90 tx=0x7fdc480127c0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.324+0000 7fdc65d99740 1 -- 172.21.15.67:0/435928570 shutdown_connections 2024-07-31T14:52:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.324+0000 7fdc65d99740 1 --2- 172.21.15.67:0/435928570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55989170e390 0x559891ad4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.324+0000 7fdc65d99740 1 --2- 172.21.15.67:0/435928570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598917061b0 0x559891ad4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.324+0000 7fdc65d99740 1 --2- 172.21.15.67:0/435928570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55989172f070 0x559891ad4570 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.324+0000 7fdc65d99740 1 -- 172.21.15.67:0/435928570 >> 172.21.15.67:0/435928570 conn(0x559891723460 msgr2=0x559891ad2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.324+0000 7fdc65d99740 1 -- 172.21.15.67:0/435928570 shutdown_connections 2024-07-31T14:52:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.324+0000 7fdc65d99740 1 -- 172.21.15.67:0/435928570 wait complete. 2024-07-31T14:52:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.325+0000 7fdc65d99740 1 -- start start 2024-07-31T14:52:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.325+0000 7fdc65d99740 1 Processor -- start 2024-07-31T14:52:57.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc65d99740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598917061b0 0x559891ae0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc65d99740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55989170e390 0x559891ae0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc65d99740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598916ed9f0 0x559891ae6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc65d99740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55989189d700 con 0x5598917061b0 2024-07-31T14:52:57.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc65d99740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5598919dc9e0 con 0x55989170e390 2024-07-31T14:52:57.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc65d99740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559891ae2d50 con 0x5598916ed9f0 2024-07-31T14:52:57.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc5cde3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598917061b0 0x559891ae0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc65d99740 10 client.0 _ll_register_callbacks cb 0x5598919f74f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:57.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc5cde3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598917061b0 0x559891ae0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32990/0 (socket says 172.21.15.67:32990) 2024-07-31T14:52:57.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc5cde3640 1 -- 172.21.15.67:0/4042429564 learned_addr learned my addr 172.21.15.67:0/4042429564 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:57.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc5d5e4640 1 --2- 172.21.15.67:0/4042429564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55989170e390 0x559891ae0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/4042429564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598916ed9f0 0x559891ae6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc5cde3640 1 -- 172.21.15.67:0/4042429564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598916ed9f0 msgr2=0x559891ae6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/4042429564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598916ed9f0 0x559891ae6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc5cde3640 1 -- 172.21.15.67:0/4042429564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55989170e390 msgr2=0x559891ae0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/4042429564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55989170e390 0x559891ae0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc5cde3640 1 -- 172.21.15.67:0/4042429564 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55989179a650 con 0x5598917061b0 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/4042429564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598917061b0 0x559891ae0290 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdc48003b20 tx=0x7fdc48003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c27bccf1920527 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.326+0000 7fdc40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.327+0000 7fdc40ff9640 1 -- 172.21.15.67:0/4042429564 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48005250 con 0x5598917061b0 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.327+0000 7fdc40ff9640 1 -- 172.21.15.67:0/4042429564 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fdc48022040 con 0x5598917061b0 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.327+0000 7fdc65d99740 1 -- 172.21.15.67:0/4042429564 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc48016070 con 0x5598917061b0 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.327+0000 7fdc40ff9640 1 -- 172.21.15.67:0/4042429564 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdc48030050 con 0x5598917061b0 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.327+0000 7fdc65d99740 10 client.16131 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.327+0000 7fdc65d99740 1 -- 172.21.15.67:0/4042429564 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559891aeeee0 con 0x5598917061b0 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.327+0000 7fdc40ff9640 1 -- 172.21.15.67:0/4042429564 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc48016070 con 0x5598917061b0 2024-07-31T14:52:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.327+0000 7fdc40ff9640 1 -- 172.21.15.67:0/4042429564 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdc48003bc0 con 0x5598917061b0 2024-07-31T14:52:57.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.327+0000 7fdc65d99740 10 client.16131 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:57.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.327+0000 7fdc65d99740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:57.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.327+0000 7fdc65d99740 20 client.16131 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:57.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.327+0000 7fdc65d99740 1 --2- 172.21.15.67:0/4042429564 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5598916c2da0 0x559891af2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.327+0000 7fdc65d99740 4 client.16131 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:57.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.327+0000 7fdc65d99740 1 -- 172.21.15.67:0/4042429564 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559891a4cb00 con 0x5598916c2da0 2024-07-31T14:52:57.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.328+0000 7fdc5d5e4640 1 --2- 172.21.15.67:0/4042429564 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5598916c2da0 0x559891af2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.328+0000 7fdc40ff9640 10 client.16131 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:57.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.328+0000 7fdc5d5e4640 1 --2- 172.21.15.67:0/4042429564 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5598916c2da0 0x559891af2240 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=182e527d319a6b3c server_cookie=aa33e359c0594025 in_seq=0 out_seq=0 2024-07-31T14:52:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.329+0000 7fdc40ff9640 1 -- 172.21.15.67:0/4042429564 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559891a4cb00 con 0x5598916c2da0 2024-07-31T14:52:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.329+0000 7fdc40ff9640 10 client.16131 handle_command_reply: tid=0 2024-07-31T14:52:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.329+0000 7fdc65d99740 1 client.16131 shutdown 2024-07-31T14:52:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.329+0000 7fdc65d99740 1 -- 172.21.15.67:0/4042429564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598917061b0 msgr2=0x559891ae0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.329+0000 7fdc65d99740 1 --2- 172.21.15.67:0/4042429564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598917061b0 0x559891ae0290 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdc48003b20 tx=0x7fdc48003840 comp rx=0 tx=0).stop 2024-07-31T14:52:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.329+0000 7fdc65d99740 1 -- 172.21.15.67:0/4042429564 shutdown_connections 2024-07-31T14:52:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.329+0000 7fdc65d99740 1 --2- 172.21.15.67:0/4042429564 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5598916c2da0 0x559891af2240 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.329+0000 7fdc65d99740 1 --2- 172.21.15.67:0/4042429564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598916ed9f0 0x559891ae6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.329+0000 7fdc40ff9640 0 client.16131 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.329+0000 7fdc65d99740 1 --2- 172.21.15.67:0/4042429564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55989170e390 0x559891ae0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.329+0000 7fdc65d99740 1 --2- 172.21.15.67:0/4042429564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598917061b0 0x559891ae0290 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.329+0000 7fdc65d99740 1 -- 172.21.15.67:0/4042429564 >> 172.21.15.67:0/4042429564 conn(0x559891723460 msgr2=0x559891ad0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.329+0000 7fdc65d99740 1 -- 172.21.15.67:0/4042429564 shutdown_connections 2024-07-31T14:52:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.330+0000 7fdc65d99740 1 -- 172.21.15.67:0/4042429564 wait complete. 2024-07-31T14:52:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.330+0000 7fdc65d99740 20 client.16131 trim_cache size 0 max 16384 2024-07-31T14:52:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.358+0000 7fdc65d99740 1 Processor -- start 2024-07-31T14:52:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.358+0000 7fdc65d99740 1 -- start start 2024-07-31T14:52:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.358+0000 7fdc65d99740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598916c2da0 0x559891ad12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.358+0000 7fdc65d99740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598916ed9f0 0x559891ad9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.358+0000 7fdc65d99740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598917061b0 0x559891ae5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.358+0000 7fdc65d99740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdc581ae430 con 0x5598916c2da0 2024-07-31T14:52:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.358+0000 7fdc65d99740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdc581aff70 con 0x5598917061b0 2024-07-31T14:52:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.358+0000 7fdc65d99740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdc581aeb00 con 0x5598916ed9f0 2024-07-31T14:52:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.358+0000 7fdc5cde3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598916c2da0 0x559891ad12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.358+0000 7fdc5cde3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598916c2da0 0x559891ad12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32994/0 (socket says 172.21.15.67:32994) 2024-07-31T14:52:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.358+0000 7fdc5cde3640 1 -- 172.21.15.67:0/1911543325 learned_addr learned my addr 172.21.15.67:0/1911543325 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.359+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/1911543325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598917061b0 0x559891ae5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.359+0000 7fdc5d5e4640 1 --2- 172.21.15.67:0/1911543325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598916ed9f0 0x559891ad9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.359+0000 7fdc5cde3640 1 -- 172.21.15.67:0/1911543325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598916ed9f0 msgr2=0x559891ad9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.359+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/1911543325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598916ed9f0 0x559891ad9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.359+0000 7fdc5cde3640 1 -- 172.21.15.67:0/1911543325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598917061b0 msgr2=0x559891ae5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.359+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/1911543325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598917061b0 0x559891ae5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.359+0000 7fdc5cde3640 1 -- 172.21.15.67:0/1911543325 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559891ad73d0 con 0x5598916c2da0 2024-07-31T14:52:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.359+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/1911543325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598916c2da0 0x559891ad12b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdc48012800 tx=0x7fdc4800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4a43ca3a2ab241 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.359+0000 7fdc42ffd640 1 -- 172.21.15.67:0/1911543325 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48003930 con 0x5598916c2da0 2024-07-31T14:52:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.359+0000 7fdc42ffd640 1 -- 172.21.15.67:0/1911543325 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc48016070 con 0x5598916c2da0 2024-07-31T14:52:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.359+0000 7fdc42ffd640 1 -- 172.21.15.67:0/1911543325 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48004660 con 0x5598916c2da0 2024-07-31T14:52:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.360+0000 7fdc65d99740 1 -- 172.21.15.67:0/1911543325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598916c2da0 msgr2=0x559891ad12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.360+0000 7fdc65d99740 1 --2- 172.21.15.67:0/1911543325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598916c2da0 0x559891ad12b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdc48012800 tx=0x7fdc4800b690 comp rx=0 tx=0).stop 2024-07-31T14:52:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.360+0000 7fdc65d99740 1 -- 172.21.15.67:0/1911543325 shutdown_connections 2024-07-31T14:52:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.360+0000 7fdc65d99740 1 --2- 172.21.15.67:0/1911543325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598917061b0 0x559891ae5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.360+0000 7fdc65d99740 1 --2- 172.21.15.67:0/1911543325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598916ed9f0 0x559891ad9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.360+0000 7fdc65d99740 1 --2- 172.21.15.67:0/1911543325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598916c2da0 0x559891ad12b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.360+0000 7fdc65d99740 1 -- 172.21.15.67:0/1911543325 >> 172.21.15.67:0/1911543325 conn(0x559891723460 msgr2=0x559891ad9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.361+0000 7fdc65d99740 1 -- 172.21.15.67:0/1911543325 shutdown_connections 2024-07-31T14:52:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.361+0000 7fdc65d99740 1 -- 172.21.15.67:0/1911543325 wait complete. 2024-07-31T14:52:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.361+0000 7fdc65d99740 1 -- start start 2024-07-31T14:52:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.361+0000 7fdc65d99740 1 Processor -- start 2024-07-31T14:52:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc65d99740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598916c2da0 0x559891aec8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc65d99740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598916ed9f0 0x559891ae7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc65d99740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598917061b0 0x559891ae8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc65d99740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdc581af8a0 con 0x5598916c2da0 2024-07-31T14:52:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc65d99740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdc581af1d0 con 0x5598916ed9f0 2024-07-31T14:52:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc65d99740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdc581ac410 con 0x5598917061b0 2024-07-31T14:52:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc5cde3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598916c2da0 0x559891aec8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc65d99740 10 client.0 _ll_register_callbacks cb 0x7fdc4800bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc5cde3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598916c2da0 0x559891aec8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33000/0 (socket says 172.21.15.67:33000) 2024-07-31T14:52:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc5cde3640 1 -- 172.21.15.67:0/998965316 learned_addr learned my addr 172.21.15.67:0/998965316 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/998965316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598917061b0 0x559891ae8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc5d5e4640 1 --2- 172.21.15.67:0/998965316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598916ed9f0 0x559891ae7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc5cde3640 1 -- 172.21.15.67:0/998965316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598917061b0 msgr2=0x559891ae8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/998965316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598917061b0 0x559891ae8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc5cde3640 1 -- 172.21.15.67:0/998965316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598916ed9f0 msgr2=0x559891ae7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/998965316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598916ed9f0 0x559891ae7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc5cde3640 1 -- 172.21.15.67:0/998965316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559891ae87d0 con 0x5598916c2da0 2024-07-31T14:52:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.362+0000 7fdc5d5e4640 1 --2- 172.21.15.67:0/998965316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598916ed9f0 0x559891ae7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:52:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.363+0000 7fdc5cde3640 1 --2- 172.21.15.67:0/998965316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598916c2da0 0x559891aec8a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdc48004060 tx=0x7fdc48000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b819fe5104a9c072 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.363+0000 7fdc40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.363+0000 7fdc40ff9640 1 -- 172.21.15.67:0/998965316 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48003c50 con 0x5598916c2da0 2024-07-31T14:52:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.363+0000 7fdc40ff9640 1 -- 172.21.15.67:0/998965316 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fdc48005250 con 0x5598916c2da0 2024-07-31T14:52:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.363+0000 7fdc65d99740 1 -- 172.21.15.67:0/998965316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc48016070 con 0x5598916c2da0 2024-07-31T14:52:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.363+0000 7fdc40ff9640 1 -- 172.21.15.67:0/998965316 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdc4804c050 con 0x5598916c2da0 2024-07-31T14:52:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.363+0000 7fdc65d99740 10 client.16137 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:57.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.363+0000 7fdc65d99740 1 -- 172.21.15.67:0/998965316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559891b5a4d0 con 0x5598916c2da0 2024-07-31T14:52:57.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.363+0000 7fdc40ff9640 1 -- 172.21.15.67:0/998965316 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc48016070 con 0x5598916c2da0 2024-07-31T14:52:57.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.364+0000 7fdc40ff9640 1 -- 172.21.15.67:0/998965316 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdc48040040 con 0x5598916c2da0 2024-07-31T14:52:57.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.364+0000 7fdc65d99740 10 client.16137 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:57.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.364+0000 7fdc65d99740 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:57.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.364+0000 7fdc65d99740 20 client.16137 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:57.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.364+0000 7fdc65d99740 1 --2- 172.21.15.67:0/998965316 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55989170e390 0x559891b5cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:57.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.364+0000 7fdc65d99740 4 client.16137 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:52:57.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.364+0000 7fdc65d99740 1 -- 172.21.15.67:0/998965316 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559891a485a0 con 0x55989170e390 2024-07-31T14:52:57.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.364+0000 7fdc5d5e4640 1 --2- 172.21.15.67:0/998965316 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55989170e390 0x559891b5cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:57.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.364+0000 7fdc5d5e4640 1 --2- 172.21.15.67:0/998965316 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55989170e390 0x559891b5cbb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a908058877292d88 server_cookie=7b7478a1c9678bb5 in_seq=0 out_seq=0 2024-07-31T14:52:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc40ff9640 10 client.16137 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc40ff9640 1 -- 172.21.15.67:0/998965316 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559891a485a0 con 0x55989170e390 2024-07-31T14:52:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc40ff9640 10 client.16137 handle_command_reply: tid=0 2024-07-31T14:52:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc65d99740 1 client.16137 shutdown 2024-07-31T14:52:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc65d99740 1 -- 172.21.15.67:0/998965316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598916c2da0 msgr2=0x559891aec8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc65d99740 1 --2- 172.21.15.67:0/998965316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598916c2da0 0x559891aec8a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdc48004060 tx=0x7fdc48000bc0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc65d99740 1 -- 172.21.15.67:0/998965316 shutdown_connections 2024-07-31T14:52:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc65d99740 1 --2- 172.21.15.67:0/998965316 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55989170e390 0x559891b5cbb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc65d99740 1 --2- 172.21.15.67:0/998965316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598917061b0 0x559891ae8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc65d99740 1 --2- 172.21.15.67:0/998965316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598916ed9f0 0x559891ae7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc40ff9640 0 client.16137 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc65d99740 1 --2- 172.21.15.67:0/998965316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5598916c2da0 0x559891aec8a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc65d99740 1 -- 172.21.15.67:0/998965316 >> 172.21.15.67:0/998965316 conn(0x559891723460 msgr2=0x559891ad1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:57.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc65d99740 1 -- 172.21.15.67:0/998965316 shutdown_connections 2024-07-31T14:52:57.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc65d99740 1 -- 172.21.15.67:0/998965316 wait complete. 2024-07-31T14:52:57.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.365+0000 7fdc65d99740 20 client.16137 trim_cache size 0 max 16384 2024-07-31T14:52:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:52:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.366+0000 7fdc5f848640 1 -- 172.21.15.67:0/2512798020 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdc440b3d40 msgr2=0x7fdc440b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.366+0000 7fdc5f848640 1 --2- 172.21.15.67:0/2512798020 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdc440b3d40 0x7fdc440b6250 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdc500c8410 tx=0x7fdc500c6910 comp rx=0 tx=0).stop 2024-07-31T14:52:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.366+0000 7fdc5f848640 1 -- 172.21.15.67:0/2512798020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc58108720 msgr2=0x7fdc5819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.366+0000 7fdc5f848640 1 --2- 172.21.15.67:0/2512798020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc58108720 0x7fdc5819f700 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdc50005cc0 tx=0x7fdc5002f380 comp rx=0 tx=0).stop 2024-07-31T14:52:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.366+0000 7fdc5f848640 1 -- 172.21.15.67:0/2512798020 shutdown_connections 2024-07-31T14:52:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.366+0000 7fdc5f848640 1 --2- 172.21.15.67:0/2512798020 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdc440b3d40 0x7fdc440b6250 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.366+0000 7fdc5f848640 1 --2- 172.21.15.67:0/2512798020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc58108720 0x7fdc5819f700 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.366+0000 7fdc5f848640 1 --2- 172.21.15.67:0/2512798020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc58107dd0 0x7fdc5819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.366+0000 7fdc5f848640 1 --2- 172.21.15.67:0/2512798020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc58102490 0x7fdc58199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.366+0000 7fdc5f848640 1 -- 172.21.15.67:0/2512798020 >> 172.21.15.67:0/2512798020 conn(0x7fdc58079160 msgr2=0x7fdc5806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.366+0000 7fdc5f848640 1 -- 172.21.15.67:0/2512798020 shutdown_connections 2024-07-31T14:52:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.366+0000 7fdc5f848640 1 -- 172.21.15.67:0/2512798020 wait complete. 2024-07-31T14:52:57.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.372+0000 7fd28ac99640 1 -- 172.21.15.67:0/2992327717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd28419afd0 con 0x7fd284103590 2024-07-31T14:52:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.373+0000 7fd281ffb640 1 -- 172.21.15.67:0/2992327717 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11962 (secure 0 0 0) 0x7fd28419afd0 con 0x7fd284103590 2024-07-31T14:52:57.374 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:52:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-31T14:51:08:515297+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:51:07.734542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:52:57.375 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-31T14:52:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.375+0000 7fd28ac99640 1 -- 172.21.15.67:0/2992327717 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd2740b3d40 msgr2=0x7fd2740b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.375+0000 7fd28ac99640 1 --2- 172.21.15.67:0/2992327717 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd2740b3d40 0x7fd2740b6250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd2841069f0 tx=0x7fd27000b390 comp rx=0 tx=0).stop 2024-07-31T14:52:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.376+0000 7fd28ac99640 1 -- 172.21.15.67:0/2992327717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd284103590 msgr2=0x7fd284199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.376+0000 7fd28ac99640 1 --2- 172.21.15.67:0/2992327717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd284103590 0x7fd284199ea0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd27800e9f0 tx=0x7fd27800edf0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.376+0000 7fd28ac99640 1 -- 172.21.15.67:0/2992327717 shutdown_connections 2024-07-31T14:52:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.376+0000 7fd28ac99640 1 --2- 172.21.15.67:0/2992327717 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd2740b3d40 0x7fd2740b6250 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.376+0000 7fd28ac99640 1 --2- 172.21.15.67:0/2992327717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd284109820 0x7fd28419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.376+0000 7fd28ac99640 1 --2- 172.21.15.67:0/2992327717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd284108ed0 0x7fd28419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.376+0000 7fd28ac99640 1 --2- 172.21.15.67:0/2992327717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd284103590 0x7fd284199ea0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.376+0000 7fd28ac99640 1 -- 172.21.15.67:0/2992327717 >> 172.21.15.67:0/2992327717 conn(0x7fd28407b280 msgr2=0x7fd284104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.376+0000 7fd28ac99640 1 -- 172.21.15.67:0/2992327717 shutdown_connections 2024-07-31T14:52:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:52:57.376+0000 7fd28ac99640 1 -- 172.21.15.67:0/2992327717 wait complete. 2024-07-31T14:52:58.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:57 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2992327717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:52:58.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:57 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2992327717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:52:58.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:57 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2992327717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:52:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.573+0000 7f83d9935640 1 -- 172.21.15.67:0/3986193343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83d4104370 msgr2=0x7f83d4104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.573+0000 7f83d9935640 1 --2- 172.21.15.67:0/3986193343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83d4104370 0x7f83d4104780 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009900 tx=0x7f83c402a410 comp rx=0 tx=0).stop 2024-07-31T14:52:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.573+0000 7f83d9935640 1 -- 172.21.15.67:0/3986193343 shutdown_connections 2024-07-31T14:52:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.573+0000 7f83d9935640 1 --2- 172.21.15.67:0/3986193343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d410d7d0 0x7f83d410fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.573+0000 7f83d9935640 1 --2- 172.21.15.67:0/3986193343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83d4104cc0 0x7f83d410d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.573+0000 7f83d9935640 1 --2- 172.21.15.67:0/3986193343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83d4104370 0x7f83d4104780 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.573+0000 7f83d9935640 1 -- 172.21.15.67:0/3986193343 >> 172.21.15.67:0/3986193343 conn(0x7f83d40fdde0 msgr2=0x7f83d4100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.573+0000 7f83d9935640 1 -- 172.21.15.67:0/3986193343 shutdown_connections 2024-07-31T14:52:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.573+0000 7f83d9935640 1 -- 172.21.15.67:0/3986193343 wait complete. 2024-07-31T14:52:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.574+0000 7f83d9935640 1 Processor -- start 2024-07-31T14:52:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.574+0000 7f83d9935640 1 -- start start 2024-07-31T14:52:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.574+0000 7f83d9935640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83d4104370 0x7f83d4199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.574+0000 7f83d9935640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83d4104cc0 0x7f83d419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.574+0000 7f83d9935640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d410d7d0 0x7f83d419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.574+0000 7f83d9935640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f83d4113440 con 0x7f83d4104cc0 2024-07-31T14:52:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.574+0000 7f83d9935640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41132c0 con 0x7f83d4104370 2024-07-31T14:52:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.574+0000 7f83d9935640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41135c0 con 0x7f83d410d7d0 2024-07-31T14:52:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.574+0000 7f83d27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83d4104cc0 0x7f83d419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.574+0000 7f83d27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83d4104cc0 0x7f83d419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33020/0 (socket says 172.21.15.67:33020) 2024-07-31T14:52:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.574+0000 7f83d27fc640 1 -- 172.21.15.67:0/4203813179 learned_addr learned my addr 172.21.15.67:0/4203813179 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.575+0000 7f83d37fe640 1 --2- 172.21.15.67:0/4203813179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d410d7d0 0x7f83d419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.575+0000 7f83d2ffd640 1 --2- 172.21.15.67:0/4203813179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83d4104370 0x7f83d4199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.575+0000 7f83d27fc640 1 -- 172.21.15.67:0/4203813179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d410d7d0 msgr2=0x7f83d419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.575+0000 7f83d27fc640 1 --2- 172.21.15.67:0/4203813179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d410d7d0 0x7f83d419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.575+0000 7f83d27fc640 1 -- 172.21.15.67:0/4203813179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83d4104370 msgr2=0x7f83d4199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.575+0000 7f83d27fc640 1 --2- 172.21.15.67:0/4203813179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83d4104370 0x7f83d4199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.575+0000 7f83d27fc640 1 -- 172.21.15.67:0/4203813179 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d419fdd0 con 0x7f83d4104cc0 2024-07-31T14:52:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.575+0000 7f83d27fc640 1 --2- 172.21.15.67:0/4203813179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83d4104cc0 0x7f83d419a440 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f83c800e9f0 tx=0x7f83c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.575+0000 7f83d8933640 1 -- 172.21.15.67:0/4203813179 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c800ccc0 con 0x7f83d4104cc0 2024-07-31T14:52:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.575+0000 7f83d8933640 1 -- 172.21.15.67:0/4203813179 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c80108e0 con 0x7f83d4104cc0 2024-07-31T14:52:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.575+0000 7f83d8933640 1 -- 172.21.15.67:0/4203813179 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8018860 con 0x7f83d4104cc0 2024-07-31T14:52:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.575+0000 7f83d9935640 1 -- 172.21.15.67:0/4203813179 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d41a00c0 con 0x7f83d4104cc0 2024-07-31T14:52:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.575+0000 7f83d9935640 1 -- 172.21.15.67:0/4203813179 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d41a6850 con 0x7f83d4104cc0 2024-07-31T14:52:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.578+0000 7f83d8933640 1 -- 172.21.15.67:0/4203813179 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f83c8010430 con 0x7f83d4104cc0 2024-07-31T14:52:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.578+0000 7f83e0822740 1 Processor -- start 2024-07-31T14:52:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.578+0000 7f83e0822740 1 -- start start 2024-07-31T14:52:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.578+0000 7f83e0822740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc391b0 0x55ddbffdbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.578+0000 7f83e0822740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc102f0 0x55ddbffdc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.578+0000 7f83e0822740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc184d0 0x55ddbffdc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.578+0000 7f83e0822740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ddbff513b0 con 0x55ddbfc391b0 2024-07-31T14:52:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.578+0000 7f83e0822740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ddbff408b0 con 0x55ddbfc102f0 2024-07-31T14:52:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.578+0000 7f83e0822740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ddbff448f0 con 0x55ddbfc184d0 2024-07-31T14:52:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.578+0000 7f83d37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc391b0 0x55ddbffdbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.578+0000 7f83d37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc391b0 0x55ddbffdbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33022/0 (socket says 172.21.15.67:33022) 2024-07-31T14:52:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.578+0000 7f83d37fe640 1 -- 172.21.15.67:0/776730470 learned_addr learned my addr 172.21.15.67:0/776730470 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.578+0000 7f83d2ffd640 1 --2- 172.21.15.67:0/776730470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc102f0 0x55ddbffdc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83d27fc640 1 --2- 172.21.15.67:0/776730470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc184d0 0x55ddbffdc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83d37fe640 1 -- 172.21.15.67:0/776730470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc184d0 msgr2=0x55ddbffdc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83d37fe640 1 --2- 172.21.15.67:0/776730470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc184d0 0x55ddbffdc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83d37fe640 1 -- 172.21.15.67:0/776730470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc102f0 msgr2=0x55ddbffdc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83d37fe640 1 --2- 172.21.15.67:0/776730470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc102f0 0x55ddbffdc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83d37fe640 1 -- 172.21.15.67:0/776730470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddbfee79a0 con 0x55ddbfc391b0 2024-07-31T14:52:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83d2ffd640 1 --2- 172.21.15.67:0/776730470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc102f0 0x55ddbffdc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83d27fc640 1 --2- 172.21.15.67:0/776730470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc184d0 0x55ddbffdc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:52:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83d8933640 1 --2- 172.21.15.67:0/4203813179 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f83bc0b3dc0 0x7f83bc0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83d37fe640 1 --2- 172.21.15.67:0/776730470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc391b0 0x55ddbffdbce0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009ee0 tx=0x7f83cc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2128fbcb0eb24efe server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83d2ffd640 1 --2- 172.21.15.67:0/4203813179 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f83bc0b3dc0 0x7f83bc0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83d8933640 1 -- 172.21.15.67:0/4203813179 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f83c80e4680 con 0x7f83d4104cc0 2024-07-31T14:52:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83b9ffb640 1 -- 172.21.15.67:0/776730470 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc016070 con 0x55ddbfc391b0 2024-07-31T14:52:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83b9ffb640 1 -- 172.21.15.67:0/776730470 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83cc004d20 con 0x55ddbfc391b0 2024-07-31T14:52:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.579+0000 7f83b9ffb640 1 -- 172.21.15.67:0/776730470 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0126b0 con 0x55ddbfc391b0 2024-07-31T14:52:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.580+0000 7f83e0822740 1 -- 172.21.15.67:0/776730470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc391b0 msgr2=0x55ddbffdbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.580+0000 7f83d2ffd640 1 --2- 172.21.15.67:0/4203813179 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f83bc0b3dc0 0x7f83bc0b62d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f83c4031440 tx=0x7f83c4005e60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.580+0000 7f83e0822740 1 --2- 172.21.15.67:0/776730470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc391b0 0x55ddbffdbce0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009ee0 tx=0x7f83cc010740 comp rx=0 tx=0).stop 2024-07-31T14:52:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.580+0000 7f83e0822740 1 -- 172.21.15.67:0/776730470 shutdown_connections 2024-07-31T14:52:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.580+0000 7f83e0822740 1 --2- 172.21.15.67:0/776730470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc184d0 0x55ddbffdc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.580+0000 7f83e0822740 1 --2- 172.21.15.67:0/776730470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc102f0 0x55ddbffdc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.580+0000 7f83e0822740 1 --2- 172.21.15.67:0/776730470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc391b0 0x55ddbffdbce0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.580+0000 7f83e0822740 1 -- 172.21.15.67:0/776730470 >> 172.21.15.67:0/776730470 conn(0x55ddbfc2d300 msgr2=0x55ddbffd9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.580+0000 7f83e0822740 1 -- 172.21.15.67:0/776730470 shutdown_connections 2024-07-31T14:52:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.580+0000 7f83e0822740 1 -- 172.21.15.67:0/776730470 wait complete. 2024-07-31T14:52:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.580+0000 7f83e0822740 1 -- start start 2024-07-31T14:52:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.581+0000 7f83e0822740 1 Processor -- start 2024-07-31T14:52:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.581+0000 7f83e0822740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc102f0 0x55ddbffe77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.581+0000 7f83e0822740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc184d0 0x55ddbffe7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.581+0000 7f83e0822740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc391b0 0x55ddbfff37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.581+0000 7f83e0822740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ddbfe7aab0 con 0x55ddbfc102f0 2024-07-31T14:52:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.581+0000 7f83e0822740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ddbfec8cf0 con 0x55ddbfc391b0 2024-07-31T14:52:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.581+0000 7f83e0822740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ddbffea460 con 0x55ddbfc184d0 2024-07-31T14:52:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.581+0000 7f83e0822740 10 client.0 _ll_register_callbacks cb 0x55ddbff3a9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.581+0000 7f83d2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc102f0 0x55ddbffe77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.581+0000 7f83d2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc102f0 0x55ddbffe77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33026/0 (socket says 172.21.15.67:33026) 2024-07-31T14:52:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.581+0000 7f83d2ffd640 1 -- 172.21.15.67:0/997667388 learned_addr learned my addr 172.21.15.67:0/997667388 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.581+0000 7f83d27fc640 1 --2- 172.21.15.67:0/997667388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc184d0 0x55ddbffe7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.581+0000 7f83d37fe640 1 --2- 172.21.15.67:0/997667388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc391b0 0x55ddbfff37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.582+0000 7f83d2ffd640 1 -- 172.21.15.67:0/997667388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc184d0 msgr2=0x55ddbffe7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.582+0000 7f83d2ffd640 1 --2- 172.21.15.67:0/997667388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc184d0 0x55ddbffe7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.582+0000 7f83d2ffd640 1 -- 172.21.15.67:0/997667388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc391b0 msgr2=0x55ddbfff37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.582+0000 7f83d2ffd640 1 --2- 172.21.15.67:0/997667388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc391b0 0x55ddbfff37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.582+0000 7f83d2ffd640 1 -- 172.21.15.67:0/997667388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddbfca4790 con 0x55ddbfc102f0 2024-07-31T14:52:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.582+0000 7f83d2ffd640 1 --2- 172.21.15.67:0/997667388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc102f0 0x55ddbffe77d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009900 tx=0x7f83c402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28a589b075845d8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.582+0000 7f83b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.582+0000 7f83b2ffd640 1 -- 172.21.15.67:0/997667388 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c404b070 con 0x55ddbfc102f0 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.582+0000 7f83e0822740 1 -- 172.21.15.67:0/997667388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83cc004d20 con 0x55ddbfc102f0 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.582+0000 7f83b2ffd640 1 -- 172.21.15.67:0/997667388 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f83c4004390 con 0x55ddbfc102f0 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.582+0000 7f83b2ffd640 1 -- 172.21.15.67:0/997667388 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f83c4057050 con 0x55ddbfc102f0 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.582+0000 7f83e0822740 10 client.16149 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.582+0000 7f83e0822740 1 -- 172.21.15.67:0/997667388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ddbfd1d620 con 0x55ddbfc102f0 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.582+0000 7f83b2ffd640 1 -- 172.21.15.67:0/997667388 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83cc004d20 con 0x55ddbfc102f0 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.583+0000 7f83b2ffd640 1 -- 172.21.15.67:0/997667388 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f83c40476d0 con 0x55ddbfc102f0 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.583+0000 7f83e0822740 10 client.16149 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.583+0000 7f83e0822740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.583+0000 7f83e0822740 20 client.16149 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.583+0000 7f83e0822740 1 --2- 172.21.15.67:0/997667388 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ddbfbf7d80 0x55ddbfff9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.583+0000 7f83e0822740 4 client.16149 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.583+0000 7f83e0822740 1 -- 172.21.15.67:0/997667388 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddbff53300 con 0x55ddbfbf7d80 2024-07-31T14:52:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.583+0000 7f83d27fc640 1 --2- 172.21.15.67:0/997667388 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ddbfbf7d80 0x55ddbfff9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.583+0000 7f83b2ffd640 10 client.16149 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.583+0000 7f83d27fc640 1 --2- 172.21.15.67:0/997667388 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ddbfbf7d80 0x55ddbfff9700 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4a36e9a98da72c3 server_cookie=71a8cb233abf9f98 in_seq=0 out_seq=0 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.584+0000 7f83b2ffd640 1 -- 172.21.15.67:0/997667388 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ddbff53300 con 0x55ddbfbf7d80 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.584+0000 7f83b2ffd640 10 client.16149 handle_command_reply: tid=0 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.584+0000 7f83e0822740 1 client.16149 shutdown 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.584+0000 7f83e0822740 1 -- 172.21.15.67:0/997667388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc102f0 msgr2=0x55ddbffe77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.584+0000 7f83e0822740 1 --2- 172.21.15.67:0/997667388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc102f0 0x55ddbffe77d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009900 tx=0x7f83c402a410 comp rx=0 tx=0).stop 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.584+0000 7f83e0822740 1 -- 172.21.15.67:0/997667388 shutdown_connections 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.584+0000 7f83e0822740 1 --2- 172.21.15.67:0/997667388 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ddbfbf7d80 0x55ddbfff9700 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.584+0000 7f83e0822740 1 --2- 172.21.15.67:0/997667388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc391b0 0x55ddbfff37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.584+0000 7f83e0822740 1 --2- 172.21.15.67:0/997667388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc184d0 0x55ddbffe7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.584+0000 7f83e0822740 1 --2- 172.21.15.67:0/997667388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc102f0 0x55ddbffe77d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.584+0000 7f83b2ffd640 0 client.16149 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.584+0000 7f83e0822740 1 -- 172.21.15.67:0/997667388 >> 172.21.15.67:0/997667388 conn(0x55ddbfc2d300 msgr2=0x55ddbffd82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.585+0000 7f83e0822740 1 -- 172.21.15.67:0/997667388 shutdown_connections 2024-07-31T14:52:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.585+0000 7f83e0822740 1 -- 172.21.15.67:0/997667388 wait complete. 2024-07-31T14:52:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.585+0000 7f83e0822740 20 client.16149 trim_cache size 0 max 16384 2024-07-31T14:52:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.600+0000 7f83e0822740 1 Processor -- start 2024-07-31T14:52:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.600+0000 7f83e0822740 1 -- start start 2024-07-31T14:52:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.600+0000 7f83e0822740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc102f0 0x55ddbffe1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.600+0000 7f83e0822740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc184d0 0x55ddbffdd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.600+0000 7f83e0822740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc391b0 0x55ddbffe91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.600+0000 7f83e0822740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41b0730 con 0x55ddbfc102f0 2024-07-31T14:52:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.600+0000 7f83e0822740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41b2270 con 0x55ddbfc184d0 2024-07-31T14:52:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.600+0000 7f83e0822740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41b0e00 con 0x55ddbfc391b0 2024-07-31T14:52:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.601+0000 7f83d2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc102f0 0x55ddbffe1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.601+0000 7f83d2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc102f0 0x55ddbffe1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33038/0 (socket says 172.21.15.67:33038) 2024-07-31T14:52:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.601+0000 7f83d2ffd640 1 -- 172.21.15.67:0/3526182289 learned_addr learned my addr 172.21.15.67:0/3526182289 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.601+0000 7f83d37fe640 1 --2- 172.21.15.67:0/3526182289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc391b0 0x55ddbffe91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.601+0000 7f83d27fc640 1 --2- 172.21.15.67:0/3526182289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc184d0 0x55ddbffdd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.601+0000 7f83d2ffd640 1 -- 172.21.15.67:0/3526182289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc391b0 msgr2=0x55ddbffe91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.601+0000 7f83d2ffd640 1 --2- 172.21.15.67:0/3526182289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc391b0 0x55ddbffe91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.601+0000 7f83d2ffd640 1 -- 172.21.15.67:0/3526182289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc184d0 msgr2=0x55ddbffdd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.601+0000 7f83d2ffd640 1 --2- 172.21.15.67:0/3526182289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc184d0 0x55ddbffdd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.601+0000 7f83d2ffd640 1 -- 172.21.15.67:0/3526182289 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddbffe5aa0 con 0x55ddbfc102f0 2024-07-31T14:52:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.601+0000 7f83d2ffd640 1 --2- 172.21.15.67:0/3526182289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc102f0 0x55ddbffe1db0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f83c4000c00 tx=0x7f83c40045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e62b60a4bc3dcb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.601+0000 7f83b97fa640 1 -- 172.21.15.67:0/3526182289 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c404b040 con 0x55ddbfc102f0 2024-07-31T14:52:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.601+0000 7f83b97fa640 1 -- 172.21.15.67:0/3526182289 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c4060070 con 0x55ddbfc102f0 2024-07-31T14:52:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.601+0000 7f83b97fa640 1 -- 172.21.15.67:0/3526182289 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4035050 con 0x55ddbfc102f0 2024-07-31T14:52:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.602+0000 7f83e0822740 1 -- 172.21.15.67:0/3526182289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc102f0 msgr2=0x55ddbffe1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.602+0000 7f83e0822740 1 --2- 172.21.15.67:0/3526182289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc102f0 0x55ddbffe1db0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f83c4000c00 tx=0x7f83c40045a0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.602+0000 7f83e0822740 1 -- 172.21.15.67:0/3526182289 shutdown_connections 2024-07-31T14:52:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.602+0000 7f83e0822740 1 --2- 172.21.15.67:0/3526182289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc391b0 0x55ddbffe91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.602+0000 7f83e0822740 1 --2- 172.21.15.67:0/3526182289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc184d0 0x55ddbffdd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.602+0000 7f83e0822740 1 --2- 172.21.15.67:0/3526182289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc102f0 0x55ddbffe1db0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.602+0000 7f83e0822740 1 -- 172.21.15.67:0/3526182289 >> 172.21.15.67:0/3526182289 conn(0x55ddbfc2d300 msgr2=0x55ddbffe8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.602+0000 7f83e0822740 1 -- 172.21.15.67:0/3526182289 shutdown_connections 2024-07-31T14:52:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.602+0000 7f83e0822740 1 -- 172.21.15.67:0/3526182289 wait complete. 2024-07-31T14:52:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.602+0000 7f83e0822740 1 -- start start 2024-07-31T14:52:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83e0822740 1 Processor -- start 2024-07-31T14:52:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83e0822740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc102f0 0x55ddbffeef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83e0822740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc184d0 0x55ddbffef520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83e0822740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc391b0 0x55ddc0059ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83e0822740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41b1ba0 con 0x55ddbfc184d0 2024-07-31T14:52:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83e0822740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41b14d0 con 0x55ddbfc102f0 2024-07-31T14:52:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83e0822740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41ae710 con 0x55ddbfc391b0 2024-07-31T14:52:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83e0822740 10 client.0 _ll_register_callbacks cb 0x7f83c403f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:52:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83d27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc184d0 0x55ddbffef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83d27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc184d0 0x55ddbffef520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33052/0 (socket says 172.21.15.67:33052) 2024-07-31T14:52:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83d27fc640 1 -- 172.21.15.67:0/2286281616 learned_addr learned my addr 172.21.15.67:0/2286281616 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:52:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83d37fe640 1 --2- 172.21.15.67:0/2286281616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc391b0 0x55ddc0059ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83d2ffd640 1 --2- 172.21.15.67:0/2286281616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc102f0 0x55ddbffeef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83d27fc640 1 -- 172.21.15.67:0/2286281616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc391b0 msgr2=0x55ddc0059ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83d27fc640 1 --2- 172.21.15.67:0/2286281616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc391b0 0x55ddc0059ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83d27fc640 1 -- 172.21.15.67:0/2286281616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc102f0 msgr2=0x55ddbffeef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:52:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83d27fc640 1 --2- 172.21.15.67:0/2286281616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc102f0 0x55ddbffeef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.603+0000 7f83d27fc640 1 -- 172.21.15.67:0/2286281616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddc005a550 con 0x55ddbfc184d0 2024-07-31T14:52:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83d27fc640 1 --2- 172.21.15.67:0/2286281616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc184d0 0x55ddbffef520 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f83c800c6a0 tx=0x7f83c80096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eed87575cec3d5a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:52:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:52:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83b2ffd640 1 -- 172.21.15.67:0/2286281616 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c80aaf80 con 0x55ddbfc184d0 2024-07-31T14:52:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83b2ffd640 1 -- 172.21.15.67:0/2286281616 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f83c80a7210 con 0x55ddbfc184d0 2024-07-31T14:52:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83e0822740 1 -- 172.21.15.67:0/2286281616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c4060070 con 0x55ddbfc184d0 2024-07-31T14:52:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83b2ffd640 1 -- 172.21.15.67:0/2286281616 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f83c80af580 con 0x55ddbfc184d0 2024-07-31T14:52:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83e0822740 10 client.16155 fetch_fsmap learned FSMap version 30 2024-07-31T14:52:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83e0822740 1 -- 172.21.15.67:0/2286281616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ddc0061900 con 0x55ddbfc184d0 2024-07-31T14:52:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83b2ffd640 1 -- 172.21.15.67:0/2286281616 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c4060070 con 0x55ddbfc184d0 2024-07-31T14:52:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83b2ffd640 1 -- 172.21.15.67:0/2286281616 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f83c80e7ce0 con 0x55ddbfc184d0 2024-07-31T14:52:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83e0822740 10 client.16155 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:52:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83e0822740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:52:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83e0822740 20 client.16155 populate_metadata read hostname 'smithi067' 2024-07-31T14:52:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83e0822740 1 --2- 172.21.15.67:0/2286281616 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ddbfbcd1c0 0x55ddc00640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:52:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83e0822740 4 client.16155 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:52:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.604+0000 7f83e0822740 1 -- 172.21.15.67:0/2286281616 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ddbff4eda0 con 0x55ddbfbcd1c0 2024-07-31T14:52:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.605+0000 7f83d2ffd640 1 --2- 172.21.15.67:0/2286281616 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ddbfbcd1c0 0x55ddc00640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:52:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.605+0000 7f83b2ffd640 10 client.16155 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.605+0000 7f83d2ffd640 1 --2- 172.21.15.67:0/2286281616 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ddbfbcd1c0 0x55ddc00640c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4811fddc0a59d1c server_cookie=17c324c83a3b957c in_seq=0 out_seq=0 2024-07-31T14:52:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.611+0000 7f83b2ffd640 1 -- 172.21.15.67:0/2286281616 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ddbff4eda0 con 0x55ddbfbcd1c0 2024-07-31T14:52:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.611+0000 7f83b2ffd640 10 client.16155 handle_command_reply: tid=0 2024-07-31T14:52:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.611+0000 7f83e0822740 1 client.16155 shutdown 2024-07-31T14:52:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.611+0000 7f83e0822740 1 -- 172.21.15.67:0/2286281616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc184d0 msgr2=0x55ddbffef520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.611+0000 7f83e0822740 1 --2- 172.21.15.67:0/2286281616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc184d0 0x55ddbffef520 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f83c800c6a0 tx=0x7f83c80096a0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83e0822740 1 -- 172.21.15.67:0/2286281616 shutdown_connections 2024-07-31T14:52:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83e0822740 1 --2- 172.21.15.67:0/2286281616 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ddbfbcd1c0 0x55ddc00640c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83e0822740 1 --2- 172.21.15.67:0/2286281616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ddbfc391b0 0x55ddc0059ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83e0822740 1 --2- 172.21.15.67:0/2286281616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ddbfc184d0 0x55ddbffef520 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83e0822740 1 --2- 172.21.15.67:0/2286281616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddbfc102f0 0x55ddbffeef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83b2ffd640 0 client.16155 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:52:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83e0822740 1 -- 172.21.15.67:0/2286281616 >> 172.21.15.67:0/2286281616 conn(0x55ddbfc2d300 msgr2=0x55ddbffe92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83e0822740 1 -- 172.21.15.67:0/2286281616 shutdown_connections 2024-07-31T14:52:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83e0822740 1 -- 172.21.15.67:0/2286281616 wait complete. 2024-07-31T14:52:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83e0822740 20 client.16155 trim_cache size 0 max 16384 2024-07-31T14:52:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:52:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:52:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:52:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:52:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83d9935640 1 -- 172.21.15.67:0/4203813179 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f83bc0b3dc0 msgr2=0x7f83bc0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83d9935640 1 --2- 172.21.15.67:0/4203813179 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f83bc0b3dc0 0x7f83bc0b62d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f83c4031440 tx=0x7f83c4005e60 comp rx=0 tx=0).stop 2024-07-31T14:52:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83d9935640 1 -- 172.21.15.67:0/4203813179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83d4104cc0 msgr2=0x7f83d419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:52:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83d9935640 1 --2- 172.21.15.67:0/4203813179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83d4104cc0 0x7f83d419a440 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f83c800e9f0 tx=0x7f83c800eeb0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83d9935640 1 -- 172.21.15.67:0/4203813179 shutdown_connections 2024-07-31T14:52:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83d9935640 1 --2- 172.21.15.67:0/4203813179 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f83bc0b3dc0 0x7f83bc0b62d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83d9935640 1 --2- 172.21.15.67:0/4203813179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d410d7d0 0x7f83d419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83d9935640 1 --2- 172.21.15.67:0/4203813179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83d4104cc0 0x7f83d419a440 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83d9935640 1 --2- 172.21.15.67:0/4203813179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83d4104370 0x7f83d4199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:52:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83d9935640 1 -- 172.21.15.67:0/4203813179 >> 172.21.15.67:0/4203813179 conn(0x7f83d40fdde0 msgr2=0x7f83d410fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:52:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83d9935640 1 -- 172.21.15.67:0/4203813179 shutdown_connections 2024-07-31T14:52:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:52:58.612+0000 7f83d9935640 1 -- 172.21.15.67:0/4203813179 wait complete. 2024-07-31T14:52:58.886 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 9b7c4d67 --release --await' 2024-07-31T14:52:58.886 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 9b7c4d67 --release --await 2024-07-31T14:52:59.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:52:58 smithi073 ceph-mon[36714]: pgmap v456: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-31T14:52:59.153 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:52:59.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:52:58 smithi077 ceph-mon[37327]: pgmap v456: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-31T14:52:59.276 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:52:58 smithi067 ceph-mon[32750]: pgmap v456: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-31T14:52:59.440 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T14:52:59.440 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T14:53:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:53:01.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:00 smithi067 ceph-mon[32750]: pgmap v457: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-31T14:53:01.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:00 smithi073 ceph-mon[36714]: pgmap v457: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-31T14:53:01.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:00 smithi077 ceph-mon[37327]: pgmap v457: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-31T14:53:01.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.617+0000 7fb683a83640 1 -- 172.21.15.67:0/3793439554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c102960 msgr2=0x7fb67c10f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:01.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.617+0000 7fb683a83640 1 --2- 172.21.15.67:0/3793439554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c102960 0x7fb67c10f680 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009900 tx=0x7fb66c02a480 comp rx=0 tx=0).stop 2024-07-31T14:53:01.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.617+0000 7fb683a83640 1 -- 172.21.15.67:0/3793439554 shutdown_connections 2024-07-31T14:53:01.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.617+0000 7fb683a83640 1 --2- 172.21.15.67:0/3793439554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c10fbc0 0x7fb67c111fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.617+0000 7fb683a83640 1 --2- 172.21.15.67:0/3793439554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c102960 0x7fb67c10f680 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.617+0000 7fb683a83640 1 --2- 172.21.15.67:0/3793439554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb67c102010 0x7fb67c102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.617+0000 7fb683a83640 1 -- 172.21.15.67:0/3793439554 >> 172.21.15.67:0/3793439554 conn(0x7fb67c079140 msgr2=0x7fb67c079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:01.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.617+0000 7fb683a83640 1 -- 172.21.15.67:0/3793439554 shutdown_connections 2024-07-31T14:53:01.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.617+0000 7fb683a83640 1 -- 172.21.15.67:0/3793439554 wait complete. 2024-07-31T14:53:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.618+0000 7fb683a83640 1 Processor -- start 2024-07-31T14:53:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.618+0000 7fb683a83640 1 -- start start 2024-07-31T14:53:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.618+0000 7fb683a83640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c102010 0x7fb67c19c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.618+0000 7fb683a83640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb67c102960 0x7fb67c19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.618+0000 7fb683a83640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c10fbc0 0x7fb67c1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.618+0000 7fb683a83640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c115680 con 0x7fb67c10fbc0 2024-07-31T14:53:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.618+0000 7fb683a83640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c115500 con 0x7fb67c102010 2024-07-31T14:53:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.618+0000 7fb683a83640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c115800 con 0x7fb67c102960 2024-07-31T14:53:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.618+0000 7fb682020640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c10fbc0 0x7fb67c1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.618+0000 7fb682020640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c10fbc0 0x7fb67c1a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59152/0 (socket says 172.21.15.67:59152) 2024-07-31T14:53:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.618+0000 7fb682020640 1 -- 172.21.15.67:0/4067164849 learned_addr learned my addr 172.21.15.67:0/4067164849 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.618+0000 7fb68101e640 1 --2- 172.21.15.67:0/4067164849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb67c102960 0x7fb67c19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.618+0000 7fb68181f640 1 --2- 172.21.15.67:0/4067164849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c102010 0x7fb67c19c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.619+0000 7fb682020640 1 -- 172.21.15.67:0/4067164849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb67c102960 msgr2=0x7fb67c19c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.619+0000 7fb682020640 1 --2- 172.21.15.67:0/4067164849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb67c102960 0x7fb67c19c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.619+0000 7fb682020640 1 -- 172.21.15.67:0/4067164849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c102010 msgr2=0x7fb67c19c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.619+0000 7fb682020640 1 --2- 172.21.15.67:0/4067164849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c102010 0x7fb67c19c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.619+0000 7fb682020640 1 -- 172.21.15.67:0/4067164849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67c1a1fe0 con 0x7fb67c10fbc0 2024-07-31T14:53:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.619+0000 7fb682020640 1 --2- 172.21.15.67:0/4067164849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c10fbc0 0x7fb67c1a18b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb67400d750 tx=0x7fb67400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.619+0000 7fb672ffd640 1 -- 172.21.15.67:0/4067164849 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674017070 con 0x7fb67c10fbc0 2024-07-31T14:53:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.619+0000 7fb683a83640 1 -- 172.21.15.67:0/4067164849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67c1a22d0 con 0x7fb67c10fbc0 2024-07-31T14:53:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.619+0000 7fb672ffd640 1 -- 172.21.15.67:0/4067164849 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6740054d0 con 0x7fb67c10fbc0 2024-07-31T14:53:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.619+0000 7fb672ffd640 1 -- 172.21.15.67:0/4067164849 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674004c70 con 0x7fb67c10fbc0 2024-07-31T14:53:01.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.619+0000 7fb683a83640 1 -- 172.21.15.67:0/4067164849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb67c1a8a30 con 0x7fb67c10fbc0 2024-07-31T14:53:01.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.622+0000 7fb672ffd640 1 -- 172.21.15.67:0/4067164849 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb674004e10 con 0x7fb67c10fbc0 2024-07-31T14:53:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.622+0000 7fb689fd4740 1 Processor -- start 2024-07-31T14:53:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.622+0000 7fb689fd4740 1 -- start start 2024-07-31T14:53:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb689fd4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856457520 0x55d8567fd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb689fd4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85642e660 0x55d8567fd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb689fd4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d856436840 0x55d8567fdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb689fd4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d856772e70 con 0x55d856457520 2024-07-31T14:53:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb689fd4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d856762370 con 0x55d85642e660 2024-07-31T14:53:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb689fd4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d8567663b0 con 0x55d856436840 2024-07-31T14:53:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb68101e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856457520 0x55d8567fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb68101e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856457520 0x55d8567fd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59168/0 (socket says 172.21.15.67:59168) 2024-07-31T14:53:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb68101e640 1 -- 172.21.15.67:0/1273332606 learned_addr learned my addr 172.21.15.67:0/1273332606 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb68181f640 1 --2- 172.21.15.67:0/1273332606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85642e660 0x55d8567fd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb672ffd640 1 --2- 172.21.15.67:0/4067164849 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6680b3d40 0x7fb6680b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb68101e640 1 --2- 172.21.15.67:0/1273332606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d856436840 0x55d8567fdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb68101e640 1 -- 172.21.15.67:0/1273332606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d856436840 msgr2=0x55d8567fdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb68101e640 1 --2- 172.21.15.67:0/1273332606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d856436840 0x55d8567fdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb68101e640 1 -- 172.21.15.67:0/1273332606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85642e660 msgr2=0x55d8567fd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:01.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb68101e640 1 --2- 172.21.15.67:0/1273332606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85642e660 0x55d8567fd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb68101e640 1 -- 172.21.15.67:0/1273332606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d85672f040 con 0x55d856457520 2024-07-31T14:53:01.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb682020640 1 --2- 172.21.15.67:0/4067164849 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6680b3d40 0x7fb6680b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.623+0000 7fb672ffd640 1 -- 172.21.15.67:0/4067164849 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fb6740e3c40 con 0x7fb67c10fbc0 2024-07-31T14:53:01.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.624+0000 7fb68101e640 1 --2- 172.21.15.67:0/1273332606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856457520 0x55d8567fd070 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb66c02a9a0 tx=0x7fb66c009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc92a396cc2f782 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:01.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.624+0000 7fb667fff640 1 -- 172.21.15.67:0/1273332606 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c044020 con 0x55d856457520 2024-07-31T14:53:01.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.624+0000 7fb667fff640 1 -- 172.21.15.67:0/1273332606 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb66c0317a0 con 0x55d856457520 2024-07-31T14:53:01.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.624+0000 7fb682020640 1 --2- 172.21.15.67:0/4067164849 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6680b3d40 0x7fb6680b6250 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb6740aa2f0 tx=0x7fb674077d40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:01.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.624+0000 7fb667fff640 1 -- 172.21.15.67:0/1273332606 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c035040 con 0x55d856457520 2024-07-31T14:53:01.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.625+0000 7fb689fd4740 1 -- 172.21.15.67:0/1273332606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856457520 msgr2=0x55d8567fd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:01.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.625+0000 7fb689fd4740 1 --2- 172.21.15.67:0/1273332606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856457520 0x55d8567fd070 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb66c02a9a0 tx=0x7fb66c009610 comp rx=0 tx=0).stop 2024-07-31T14:53:01.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.625+0000 7fb689fd4740 1 -- 172.21.15.67:0/1273332606 shutdown_connections 2024-07-31T14:53:01.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.625+0000 7fb689fd4740 1 --2- 172.21.15.67:0/1273332606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d856436840 0x55d8567fdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.625+0000 7fb689fd4740 1 --2- 172.21.15.67:0/1273332606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85642e660 0x55d8567fd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.625+0000 7fb689fd4740 1 --2- 172.21.15.67:0/1273332606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856457520 0x55d8567fd070 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.625+0000 7fb689fd4740 1 -- 172.21.15.67:0/1273332606 >> 172.21.15.67:0/1273332606 conn(0x55d85644b910 msgr2=0x55d8567faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:01.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.625+0000 7fb689fd4740 1 -- 172.21.15.67:0/1273332606 shutdown_connections 2024-07-31T14:53:01.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.625+0000 7fb689fd4740 1 -- 172.21.15.67:0/1273332606 wait complete. 2024-07-31T14:53:01.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.625+0000 7fb689fd4740 1 -- start start 2024-07-31T14:53:01.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.625+0000 7fb689fd4740 1 Processor -- start 2024-07-31T14:53:01.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb689fd4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d85642e660 0x55d856808be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb689fd4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d856436840 0x55d856809120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb689fd4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d856457520 0x55d856814b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb689fd4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d8565c5bb0 con 0x55d85642e660 2024-07-31T14:53:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb689fd4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d856704ac0 con 0x55d856457520 2024-07-31T14:53:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb689fd4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d85680b7f0 con 0x55d856436840 2024-07-31T14:53:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb689fd4740 10 client.0 _ll_register_callbacks cb 0x55d85670d3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb68101e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d85642e660 0x55d856808be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb68101e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d85642e660 0x55d856808be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59170/0 (socket says 172.21.15.67:59170) 2024-07-31T14:53:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb68101e640 1 -- 172.21.15.67:0/944026920 learned_addr learned my addr 172.21.15.67:0/944026920 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb68181f640 1 --2- 172.21.15.67:0/944026920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d856436840 0x55d856809120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb68101e640 1 -- 172.21.15.67:0/944026920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d856436840 msgr2=0x55d856809120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb68101e640 1 --2- 172.21.15.67:0/944026920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d856436840 0x55d856809120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb68101e640 1 -- 172.21.15.67:0/944026920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d856457520 msgr2=0x55d856814b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.626+0000 7fb68101e640 1 --2- 172.21.15.67:0/944026920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d856457520 0x55d856814b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.627+0000 7fb68101e640 1 -- 172.21.15.67:0/944026920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8564c2b00 con 0x55d85642e660 2024-07-31T14:53:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.627+0000 7fb68101e640 1 --2- 172.21.15.67:0/944026920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d85642e660 0x55d856808be0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb66c005940 tx=0x7fb66c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38cd51657c28f4f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.627+0000 7fb6657fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.627+0000 7fb6657fa640 1 -- 172.21.15.67:0/944026920 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c04e020 con 0x55d85642e660 2024-07-31T14:53:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.627+0000 7fb6657fa640 1 -- 172.21.15.67:0/944026920 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fb66c004260 con 0x55d85642e660 2024-07-31T14:53:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.627+0000 7fb689fd4740 1 -- 172.21.15.67:0/944026920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb66c0317a0 con 0x55d85642e660 2024-07-31T14:53:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.627+0000 7fb6657fa640 1 -- 172.21.15.67:0/944026920 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb66c059050 con 0x55d85642e660 2024-07-31T14:53:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.627+0000 7fb689fd4740 10 client.16167 fetch_fsmap learned FSMap version 30 2024-07-31T14:53:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.627+0000 7fb689fd4740 1 -- 172.21.15.67:0/944026920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d856539960 con 0x55d85642e660 2024-07-31T14:53:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.627+0000 7fb6657fa640 1 -- 172.21.15.67:0/944026920 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb66c0317a0 con 0x55d85642e660 2024-07-31T14:53:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.628+0000 7fb6657fa640 1 -- 172.21.15.67:0/944026920 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb66c04a430 con 0x55d85642e660 2024-07-31T14:53:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.628+0000 7fb689fd4740 10 client.16167 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:53:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.628+0000 7fb689fd4740 10 client.16167 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:53:01.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.628+0000 7fb689fd4740 20 client.16167 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:01.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.628+0000 7fb689fd4740 1 --2- 172.21.15.67:0/944026920 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d856415ea0 0x55d85681a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.628+0000 7fb689fd4740 4 client.16167 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:53:01.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.628+0000 7fb689fd4740 1 -- 172.21.15.67:0/944026920 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d856774dc0 con 0x55d856415ea0 2024-07-31T14:53:01.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.628+0000 7fb68181f640 1 --2- 172.21.15.67:0/944026920 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d856415ea0 0x55d85681a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.629+0000 7fb68181f640 1 --2- 172.21.15.67:0/944026920 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d856415ea0 0x55d85681a940 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f52fd6abdab1df5b server_cookie=fec4af054a6e3f8e in_seq=0 out_seq=0 2024-07-31T14:53:01.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.629+0000 7fb6657fa640 10 client.16167 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:01.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.629+0000 7fb6657fa640 1 -- 172.21.15.67:0/944026920 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d856774dc0 con 0x55d856415ea0 2024-07-31T14:53:01.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.629+0000 7fb6657fa640 10 client.16167 handle_command_reply: tid=0 2024-07-31T14:53:01.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.629+0000 7fb689fd4740 1 client.16167 shutdown 2024-07-31T14:53:01.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.630+0000 7fb689fd4740 1 -- 172.21.15.67:0/944026920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d85642e660 msgr2=0x55d856808be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:01.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.630+0000 7fb689fd4740 1 --2- 172.21.15.67:0/944026920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d85642e660 0x55d856808be0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb66c005940 tx=0x7fb66c02a480 comp rx=0 tx=0).stop 2024-07-31T14:53:01.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.630+0000 7fb689fd4740 1 -- 172.21.15.67:0/944026920 shutdown_connections 2024-07-31T14:53:01.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.630+0000 7fb689fd4740 1 --2- 172.21.15.67:0/944026920 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d856415ea0 0x55d85681a940 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.630+0000 7fb689fd4740 1 --2- 172.21.15.67:0/944026920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d856457520 0x55d856814b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.630+0000 7fb689fd4740 1 --2- 172.21.15.67:0/944026920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d856436840 0x55d856809120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.630+0000 7fb6657fa640 0 client.16167 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:01.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.630+0000 7fb689fd4740 1 --2- 172.21.15.67:0/944026920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d85642e660 0x55d856808be0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.630+0000 7fb689fd4740 1 -- 172.21.15.67:0/944026920 >> 172.21.15.67:0/944026920 conn(0x55d85644b910 msgr2=0x55d856800250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:01.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.630+0000 7fb689fd4740 1 -- 172.21.15.67:0/944026920 shutdown_connections 2024-07-31T14:53:01.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.630+0000 7fb689fd4740 1 -- 172.21.15.67:0/944026920 wait complete. 2024-07-31T14:53:01.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.630+0000 7fb689fd4740 20 client.16167 trim_cache size 0 max 16384 2024-07-31T14:53:01.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.648+0000 7fb689fd4740 1 Processor -- start 2024-07-31T14:53:01.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.648+0000 7fb689fd4740 1 -- start start 2024-07-31T14:53:01.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.648+0000 7fb689fd4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85644b910 0x55d856800880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.648+0000 7fb689fd4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d85642e660 0x55d856814ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.648+0000 7fb689fd4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856436840 0x55d856814ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.648+0000 7fb689fd4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c1b27f0 con 0x55d856436840 2024-07-31T14:53:01.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.648+0000 7fb689fd4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c1b4330 con 0x55d85644b910 2024-07-31T14:53:01.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.648+0000 7fb689fd4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c1b2ec0 con 0x55d85642e660 2024-07-31T14:53:01.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.649+0000 7fb68101e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856436840 0x55d856814ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.649+0000 7fb68101e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856436840 0x55d856814ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59184/0 (socket says 172.21.15.67:59184) 2024-07-31T14:53:01.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.649+0000 7fb68101e640 1 -- 172.21.15.67:0/2725926078 learned_addr learned my addr 172.21.15.67:0/2725926078 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:01.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.649+0000 7fb68181f640 1 --2- 172.21.15.67:0/2725926078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d85642e660 0x55d856814ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.649+0000 7fb68101e640 1 --2- 172.21.15.67:0/2725926078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85644b910 0x55d856800880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.649+0000 7fb68101e640 1 -- 172.21.15.67:0/2725926078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d85642e660 msgr2=0x55d856814ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:01.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.649+0000 7fb68101e640 1 --2- 172.21.15.67:0/2725926078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d85642e660 0x55d856814ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.649+0000 7fb68101e640 1 -- 172.21.15.67:0/2725926078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85644b910 msgr2=0x55d856800880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:01.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.649+0000 7fb68101e640 1 --2- 172.21.15.67:0/2725926078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85644b910 0x55d856800880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.649+0000 7fb68101e640 1 -- 172.21.15.67:0/2725926078 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d856809ca0 con 0x55d856436840 2024-07-31T14:53:01.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.649+0000 7fb68101e640 1 --2- 172.21.15.67:0/2725926078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856436840 0x55d856814ff0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0053a0 tx=0x7fb66c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4829fada57b4ab6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.649+0000 7fb6677fe640 1 -- 172.21.15.67:0/2725926078 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c04e040 con 0x55d856436840 2024-07-31T14:53:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.649+0000 7fb6677fe640 1 -- 172.21.15.67:0/2725926078 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb66c04ac70 con 0x55d856436840 2024-07-31T14:53:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.649+0000 7fb6677fe640 1 -- 172.21.15.67:0/2725926078 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c044070 con 0x55d856436840 2024-07-31T14:53:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.650+0000 7fb689fd4740 1 -- 172.21.15.67:0/2725926078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856436840 msgr2=0x55d856814ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.650+0000 7fb689fd4740 1 --2- 172.21.15.67:0/2725926078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856436840 0x55d856814ff0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0053a0 tx=0x7fb66c005fd0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.650+0000 7fb689fd4740 1 -- 172.21.15.67:0/2725926078 shutdown_connections 2024-07-31T14:53:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.650+0000 7fb689fd4740 1 --2- 172.21.15.67:0/2725926078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856436840 0x55d856814ff0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.650+0000 7fb689fd4740 1 --2- 172.21.15.67:0/2725926078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d85642e660 0x55d856814ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.650+0000 7fb689fd4740 1 --2- 172.21.15.67:0/2725926078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85644b910 0x55d856800880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.650+0000 7fb689fd4740 1 -- 172.21.15.67:0/2725926078 >> 172.21.15.67:0/2725926078 conn(0x55d8568092a0 msgr2=0x55d856800170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:01.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.650+0000 7fb689fd4740 1 -- 172.21.15.67:0/2725926078 shutdown_connections 2024-07-31T14:53:01.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.650+0000 7fb689fd4740 1 -- 172.21.15.67:0/2725926078 wait complete. 2024-07-31T14:53:01.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.650+0000 7fb689fd4740 1 -- start start 2024-07-31T14:53:01.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.651+0000 7fb689fd4740 1 Processor -- start 2024-07-31T14:53:01.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.651+0000 7fb689fd4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d85642e660 0x55d85681c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.651+0000 7fb689fd4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856436840 0x55d85681c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.651+0000 7fb689fd4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85644b910 0x55d85687be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.651+0000 7fb689fd4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c1b3c60 con 0x55d856436840 2024-07-31T14:53:01.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.651+0000 7fb689fd4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c1b3590 con 0x55d85644b910 2024-07-31T14:53:01.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.651+0000 7fb689fd4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c1b07d0 con 0x55d85642e660 2024-07-31T14:53:01.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.651+0000 7fb689fd4740 10 client.0 _ll_register_callbacks cb 0x7fb678009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:01.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.651+0000 7fb68181f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856436840 0x55d85681c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.651+0000 7fb68181f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856436840 0x55d85681c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59200/0 (socket says 172.21.15.67:59200) 2024-07-31T14:53:01.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.651+0000 7fb68181f640 1 -- 172.21.15.67:0/387013739 learned_addr learned my addr 172.21.15.67:0/387013739 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:01.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.651+0000 7fb68101e640 1 --2- 172.21.15.67:0/387013739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d85642e660 0x55d85681c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb68101e640 1 --2- 172.21.15.67:0/387013739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85644b910 0x55d85687be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb68181f640 1 -- 172.21.15.67:0/387013739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d85642e660 msgr2=0x55d85681c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:01.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb68181f640 1 --2- 172.21.15.67:0/387013739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d85642e660 0x55d85681c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb68181f640 1 -- 172.21.15.67:0/387013739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85644b910 msgr2=0x55d85687be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:01.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb68181f640 1 --2- 172.21.15.67:0/387013739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85644b910 0x55d85687be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb68181f640 1 -- 172.21.15.67:0/387013739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d85687c4d0 con 0x55d856436840 2024-07-31T14:53:01.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb68181f640 1 --2- 172.21.15.67:0/387013739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856436840 0x55d85681c6e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb678004a80 tx=0x7fb6780128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483da094eafdfac5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:01.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb6657fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:01.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb6657fa640 1 -- 172.21.15.67:0/387013739 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67802a060 con 0x55d856436840 2024-07-31T14:53:01.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb6657fa640 1 -- 172.21.15.67:0/387013739 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fb67801e780 con 0x55d856436840 2024-07-31T14:53:01.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb689fd4740 1 -- 172.21.15.67:0/387013739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb66c04ac70 con 0x55d856436840 2024-07-31T14:53:01.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb6657fa640 1 -- 172.21.15.67:0/387013739 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb678031050 con 0x55d856436840 2024-07-31T14:53:01.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb689fd4740 10 client.16173 fetch_fsmap learned FSMap version 30 2024-07-31T14:53:01.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb689fd4740 1 -- 172.21.15.67:0/387013739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d8568837e0 con 0x55d856436840 2024-07-31T14:53:01.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.652+0000 7fb6657fa640 1 -- 172.21.15.67:0/387013739 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb66c04ac70 con 0x55d856436840 2024-07-31T14:53:01.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.653+0000 7fb6657fa640 1 -- 172.21.15.67:0/387013739 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb678025c40 con 0x55d856436840 2024-07-31T14:53:01.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.653+0000 7fb689fd4740 10 client.16173 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:53:01.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.653+0000 7fb689fd4740 10 client.16173 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:53:01.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.653+0000 7fb689fd4740 20 client.16173 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:01.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.653+0000 7fb689fd4740 1 --2- 172.21.15.67:0/387013739 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d856457520 0x55d856885f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:01.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.653+0000 7fb689fd4740 4 client.16173 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9b7c4d67", "release": true, "await": true}] 2024-07-31T14:53:01.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.653+0000 7fb689fd4740 1 -- 172.21.15.67:0/387013739 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9b7c4d67", "release": true, "await": true}) -- 0x55d856770860 con 0x55d856457520 2024-07-31T14:53:01.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.653+0000 7fb68101e640 1 --2- 172.21.15.67:0/387013739 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d856457520 0x55d856885f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:01.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.653+0000 7fb6657fa640 10 client.16173 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:01.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.653+0000 7fb68101e640 1 --2- 172.21.15.67:0/387013739 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d856457520 0x55d856885f80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7140dd60ee0a5d74 server_cookie=1bb1dcd6856bf790 in_seq=0 out_seq=0 2024-07-31T14:53:01.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb6657fa640 1 -- 172.21.15.67:0/387013739 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55d856770860 con 0x55d856457520 2024-07-31T14:53:01.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb6657fa640 10 client.16173 handle_command_reply: tid=0 2024-07-31T14:53:01.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb689fd4740 1 client.16173 shutdown 2024-07-31T14:53:01.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb689fd4740 1 -- 172.21.15.67:0/387013739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856436840 msgr2=0x55d85681c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:01.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb689fd4740 1 --2- 172.21.15.67:0/387013739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856436840 0x55d85681c6e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb678004a80 tx=0x7fb6780128e0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb689fd4740 1 -- 172.21.15.67:0/387013739 shutdown_connections 2024-07-31T14:53:01.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb689fd4740 1 --2- 172.21.15.67:0/387013739 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d856457520 0x55d856885f80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb689fd4740 1 --2- 172.21.15.67:0/387013739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85644b910 0x55d85687be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb689fd4740 1 --2- 172.21.15.67:0/387013739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d856436840 0x55d85681c6e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb6657fa640 0 client.16173 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:01.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb689fd4740 1 --2- 172.21.15.67:0/387013739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d85642e660 0x55d85681c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb689fd4740 1 -- 172.21.15.67:0/387013739 >> 172.21.15.67:0/387013739 conn(0x55d8568092a0 msgr2=0x55d856807260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:01.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb689fd4740 1 -- 172.21.15.67:0/387013739 shutdown_connections 2024-07-31T14:53:01.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb689fd4740 1 -- 172.21.15.67:0/387013739 wait complete. 2024-07-31T14:53:01.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.657+0000 7fb689fd4740 20 client.16173 trim_cache size 0 max 16384 2024-07-31T14:53:01.661 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:53:01.661 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 30, 2024-07-31T14:53:01.661 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T14:53:01.661 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 4, 2024-07-31T14:53:01.661 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T14:53:01.662 INFO:teuthology.orchestra.run.smithi067.stdout: "9b7c4d67": { 2024-07-31T14:53:01.662 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 4, 2024-07-31T14:53:01.662 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T14:53:01.662 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:53:01.662 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T14:53:01.662 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:53:01.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:53:01.662 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T14:53:01.662 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 153.4, 2024-07-31T14:53:01.662 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T14:53:01.662 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T14:53:01.662 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T14:53:01.663 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:53:01.663 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T14:53:01.663 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:53:01.663 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:53:01.663 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:53:01.663 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:53:01.663 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:53:01.663 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:53:01.663 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:53:01.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.658+0000 7fb683a83640 1 -- 172.21.15.67:0/4067164849 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6680b3d40 msgr2=0x7fb6680b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:01.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.658+0000 7fb683a83640 1 --2- 172.21.15.67:0/4067164849 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6680b3d40 0x7fb6680b6250 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb6740aa2f0 tx=0x7fb674077d40 comp rx=0 tx=0).stop 2024-07-31T14:53:01.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.658+0000 7fb683a83640 1 -- 172.21.15.67:0/4067164849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c10fbc0 msgr2=0x7fb67c1a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:01.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.658+0000 7fb683a83640 1 --2- 172.21.15.67:0/4067164849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c10fbc0 0x7fb67c1a18b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb67400d750 tx=0x7fb67400dc10 comp rx=0 tx=0).stop 2024-07-31T14:53:01.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.659+0000 7fb683a83640 1 -- 172.21.15.67:0/4067164849 shutdown_connections 2024-07-31T14:53:01.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.659+0000 7fb683a83640 1 --2- 172.21.15.67:0/4067164849 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6680b3d40 0x7fb6680b6250 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.659+0000 7fb683a83640 1 --2- 172.21.15.67:0/4067164849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c10fbc0 0x7fb67c1a18b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.659+0000 7fb683a83640 1 --2- 172.21.15.67:0/4067164849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb67c102960 0x7fb67c19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.659+0000 7fb683a83640 1 --2- 172.21.15.67:0/4067164849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c102010 0x7fb67c19c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:01.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.659+0000 7fb683a83640 1 -- 172.21.15.67:0/4067164849 >> 172.21.15.67:0/4067164849 conn(0x7fb67c079140 msgr2=0x7fb67c108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:01.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.659+0000 7fb683a83640 1 -- 172.21.15.67:0/4067164849 shutdown_connections 2024-07-31T14:53:01.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:01.659+0000 7fb683a83640 1 -- 172.21.15.67:0/4067164849 wait complete. 2024-07-31T14:53:01.665 INFO:tasks.workunit.client.0.smithi067.stderr:+ wget -O linux.tar.xz http://download.ceph.com/qa/linux-6.5.11.tar.xz 2024-07-31T14:53:01.731 INFO:tasks.workunit.client.0.smithi067.stderr:--2024-07-31 14:53:01-- http://download.ceph.com/qa/linux-6.5.11.tar.xz 2024-07-31T14:53:01.731 INFO:tasks.workunit.client.0.smithi067.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-07-31T14:53:01.753 INFO:tasks.workunit.client.0.smithi067.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-07-31T14:53:01.774 INFO:tasks.workunit.client.0.smithi067.stderr:HTTP request sent, awaiting response... 200 OK 2024-07-31T14:53:01.774 INFO:tasks.workunit.client.0.smithi067.stderr:Length: 139010916 (133M) [application/octet-stream] 2024-07-31T14:53:01.774 INFO:tasks.workunit.client.0.smithi067.stderr:Saving to: ‘linux.tar.xz’ 2024-07-31T14:53:01.777 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-31T14:53:01.816 INFO:tasks.workunit.client.0.smithi067.stderr: 0K .......... .......... .......... .......... .......... 0% 1.19M 1m51s 2024-07-31T14:53:01.837 INFO:tasks.workunit.client.0.smithi067.stderr: 50K .......... .......... .......... .......... .......... 0% 2.38M 83s 2024-07-31T14:53:01.837 INFO:tasks.workunit.client.0.smithi067.stderr: 100K .......... .......... .......... .......... .......... 0% 237M 56s 2024-07-31T14:53:01.837 INFO:tasks.workunit.client.0.smithi067.stderr: 150K .......... .......... .......... .......... .......... 0% 212M 42s 2024-07-31T14:53:01.858 INFO:tasks.workunit.client.0.smithi067.stderr: 200K .......... .......... .......... .......... .......... 0% 2.42M 44s 2024-07-31T14:53:01.858 INFO:tasks.workunit.client.0.smithi067.stderr: 250K .......... .......... .......... .......... .......... 0% 264M 37s 2024-07-31T14:53:01.858 INFO:tasks.workunit.client.0.smithi067.stderr: 300K .......... .......... .......... .......... .......... 0% 602M 32s 2024-07-31T14:53:01.858 INFO:tasks.workunit.client.0.smithi067.stderr: 350K .......... .......... .......... .......... .......... 0% 539M 28s 2024-07-31T14:53:01.882 INFO:tasks.workunit.client.0.smithi067.stderr: 400K .......... .......... .......... .......... .......... 0% 2.44M 31s 2024-07-31T14:53:01.882 INFO:tasks.workunit.client.0.smithi067.stderr: 450K .......... .......... .......... .......... .......... 0% 334M 28s 2024-07-31T14:53:01.882 INFO:tasks.workunit.client.0.smithi067.stderr: 500K .......... .......... .......... .......... .......... 0% 489M 25s 2024-07-31T14:53:01.882 INFO:tasks.workunit.client.0.smithi067.stderr: 550K .......... .......... .......... .......... .......... 0% 457M 23s 2024-07-31T14:53:01.882 INFO:tasks.workunit.client.0.smithi067.stderr: 600K .......... .......... .......... .......... .......... 0% 510M 21s 2024-07-31T14:53:01.882 INFO:tasks.workunit.client.0.smithi067.stderr: 650K .......... .......... .......... .......... .......... 0% 472M 20s 2024-07-31T14:53:01.882 INFO:tasks.workunit.client.0.smithi067.stderr: 700K .......... .......... .......... .......... .......... 0% 63.6M 19s 2024-07-31T14:53:01.882 INFO:tasks.workunit.client.0.smithi067.stderr: 750K .......... .......... .......... .......... .......... 0% 22.2M 18s 2024-07-31T14:53:01.902 INFO:tasks.workunit.client.0.smithi067.stderr: 800K .......... .......... .......... .......... .......... 0% 2.87M 20s 2024-07-31T14:53:01.902 INFO:tasks.workunit.client.0.smithi067.stderr: 850K .......... .......... .......... .......... .......... 0% 366M 18s 2024-07-31T14:53:01.903 INFO:tasks.workunit.client.0.smithi067.stderr: 900K .......... .......... .......... .......... .......... 0% 622M 17s 2024-07-31T14:53:01.903 INFO:tasks.workunit.client.0.smithi067.stderr: 950K .......... .......... .......... .......... .......... 0% 615M 17s 2024-07-31T14:53:01.903 INFO:tasks.workunit.client.0.smithi067.stderr: 1000K .......... .......... .......... .......... .......... 0% 332M 16s 2024-07-31T14:53:01.903 INFO:tasks.workunit.client.0.smithi067.stderr: 1050K .......... .......... .......... .......... .......... 0% 487M 15s 2024-07-31T14:53:01.903 INFO:tasks.workunit.client.0.smithi067.stderr: 1100K .......... .......... .......... .......... .......... 0% 185M 14s 2024-07-31T14:53:01.903 INFO:tasks.workunit.client.0.smithi067.stderr: 1150K .......... .......... .......... .......... .......... 0% 22.1M 14s 2024-07-31T14:53:01.903 INFO:tasks.workunit.client.0.smithi067.stderr: 1200K .......... .......... .......... .......... .......... 0% 55.8M 14s 2024-07-31T14:53:01.905 INFO:tasks.workunit.client.0.smithi067.stderr: 1250K .......... .......... .......... .......... .......... 0% 23.0M 13s 2024-07-31T14:53:01.905 INFO:tasks.workunit.client.0.smithi067.stderr: 1300K .......... .......... .......... .......... .......... 0% 35.7M 13s 2024-07-31T14:53:01.907 INFO:tasks.workunit.client.0.smithi067.stderr: 1350K .......... .......... .......... .......... .......... 1% 22.1M 13s 2024-07-31T14:53:01.909 INFO:tasks.workunit.client.0.smithi067.stderr: 1400K .......... .......... .......... .......... .......... 1% 23.9M 12s 2024-07-31T14:53:01.911 INFO:tasks.workunit.client.0.smithi067.stderr: 1450K .......... .......... .......... .......... .......... 1% 67.2M 12s 2024-07-31T14:53:01.911 INFO:tasks.workunit.client.0.smithi067.stderr: 1500K .......... .......... .......... .......... .......... 1% 23.2M 12s 2024-07-31T14:53:01.913 INFO:tasks.workunit.client.0.smithi067.stderr: 1550K .......... .......... .......... .......... .......... 1% 33.1M 12s 2024-07-31T14:53:01.915 INFO:tasks.workunit.client.0.smithi067.stderr: 1600K .......... .......... .......... .......... .......... 1% 24.3M 11s 2024-07-31T14:53:01.918 INFO:tasks.workunit.client.0.smithi067.stderr: 1650K .......... .......... .......... .......... .......... 1% 19.3M 11s 2024-07-31T14:53:01.920 INFO:tasks.workunit.client.0.smithi067.stderr: 1700K .......... .......... .......... .......... .......... 1% 27.8M 11s 2024-07-31T14:53:01.920 INFO:tasks.workunit.client.0.smithi067.stderr: 1750K .......... .......... .......... .......... .......... 1% 101M 11s 2024-07-31T14:53:01.922 INFO:tasks.workunit.client.0.smithi067.stderr: 1800K .......... .......... .......... .......... .......... 1% 26.9M 11s 2024-07-31T14:53:01.924 INFO:tasks.workunit.client.0.smithi067.stderr: 1850K .......... .......... .......... .......... .......... 1% 21.4M 11s 2024-07-31T14:53:01.926 INFO:tasks.workunit.client.0.smithi067.stderr: 1900K .......... .......... .......... .......... .......... 1% 23.4M 10s 2024-07-31T14:53:01.928 INFO:tasks.workunit.client.0.smithi067.stderr: 1950K .......... .......... .......... .......... .......... 1% 23.1M 10s 2024-07-31T14:53:01.929 INFO:tasks.workunit.client.0.smithi067.stderr: 2000K .......... .......... .......... .......... .......... 1% 307M 10s 2024-07-31T14:53:01.931 INFO:tasks.workunit.client.0.smithi067.stderr: 2050K .......... .......... .......... .......... .......... 1% 23.8M 10s 2024-07-31T14:53:01.932 INFO:tasks.workunit.client.0.smithi067.stderr: 2100K .......... .......... .......... .......... .......... 1% 23.7M 10s 2024-07-31T14:53:01.933 INFO:tasks.workunit.client.0.smithi067.stderr: 2150K .......... .......... .......... .......... .......... 1% 59.3M 10s 2024-07-31T14:53:01.935 INFO:tasks.workunit.client.0.smithi067.stderr: 2200K .......... .......... .......... .......... .......... 1% 21.7M 10s 2024-07-31T14:53:01.936 INFO:tasks.workunit.client.0.smithi067.stderr: 2250K .......... .......... .......... .......... .......... 1% 40.4M 9s 2024-07-31T14:53:01.938 INFO:tasks.workunit.client.0.smithi067.stderr: 2300K .......... .......... .......... .......... .......... 1% 35.6M 9s 2024-07-31T14:53:01.940 INFO:tasks.workunit.client.0.smithi067.stderr: 2350K .......... .......... .......... .......... .......... 1% 25.5M 9s 2024-07-31T14:53:01.943 INFO:tasks.workunit.client.0.smithi067.stderr: 2400K .......... .......... .......... .......... .......... 1% 20.6M 9s 2024-07-31T14:53:01.944 INFO:tasks.workunit.client.0.smithi067.stderr: 2450K .......... .......... .......... .......... .......... 1% 24.1M 9s 2024-07-31T14:53:01.945 INFO:tasks.workunit.client.0.smithi067.stderr: 2500K .......... .......... .......... .......... .......... 1% 74.6M 9s 2024-07-31T14:53:01.947 INFO:tasks.workunit.client.0.smithi067.stderr: 2550K .......... .......... .......... .......... .......... 1% 22.6M 9s 2024-07-31T14:53:01.949 INFO:tasks.workunit.client.0.smithi067.stderr: 2600K .......... .......... .......... .......... .......... 1% 33.5M 9s 2024-07-31T14:53:01.951 INFO:tasks.workunit.client.0.smithi067.stderr: 2650K .......... .......... .......... .......... .......... 1% 26.3M 9s 2024-07-31T14:53:01.956 INFO:tasks.workunit.client.0.smithi067.stderr: 2700K .......... .......... .......... .......... .......... 2% 20.2M 9s 2024-07-31T14:53:01.956 INFO:tasks.workunit.client.0.smithi067.stderr: 2750K .......... .......... .......... .......... .......... 2% 117M 9s 2024-07-31T14:53:01.956 INFO:tasks.workunit.client.0.smithi067.stderr: 2800K .......... .......... .......... .......... .......... 2% 24.3M 8s 2024-07-31T14:53:01.958 INFO:tasks.workunit.client.0.smithi067.stderr: 2850K .......... .......... .......... .......... .......... 2% 25.8M 8s 2024-07-31T14:53:01.960 INFO:tasks.workunit.client.0.smithi067.stderr: 2900K .......... .......... .......... .......... .......... 2% 23.3M 8s 2024-07-31T14:53:01.962 INFO:tasks.workunit.client.0.smithi067.stderr: 2950K .......... .......... .......... .......... .......... 2% 24.9M 8s 2024-07-31T14:53:01.962 INFO:tasks.workunit.client.0.smithi067.stderr: 3000K .......... .......... .......... .......... .......... 2% 111M 8s 2024-07-31T14:53:01.964 INFO:tasks.workunit.client.0.smithi067.stderr: 3050K .......... .......... .......... .......... .......... 2% 28.3M 8s 2024-07-31T14:53:01.966 INFO:tasks.workunit.client.0.smithi067.stderr: 3100K .......... .......... .......... .......... .......... 2% 20.0M 8s 2024-07-31T14:53:01.968 INFO:tasks.workunit.client.0.smithi067.stderr: 3150K .......... .......... .......... .......... .......... 2% 27.7M 8s 2024-07-31T14:53:01.969 INFO:tasks.workunit.client.0.smithi067.stderr: 3200K .......... .......... .......... .......... .......... 2% 28.9M 8s 2024-07-31T14:53:01.970 INFO:tasks.workunit.client.0.smithi067.stderr: 3250K .......... .......... .......... .......... .......... 2% 53.8M 8s 2024-07-31T14:53:01.972 INFO:tasks.workunit.client.0.smithi067.stderr: 3300K .......... .......... .......... .......... .......... 2% 29.2M 8s 2024-07-31T14:53:01.974 INFO:tasks.workunit.client.0.smithi067.stderr: 3350K .......... .......... .......... .......... .......... 2% 25.8M 8s 2024-07-31T14:53:01.976 INFO:tasks.workunit.client.0.smithi067.stderr: 3400K .......... .......... .......... .......... .......... 2% 26.5M 8s 2024-07-31T14:53:01.977 INFO:tasks.workunit.client.0.smithi067.stderr: 3450K .......... .......... .......... .......... .......... 2% 25.7M 8s 2024-07-31T14:53:01.978 INFO:tasks.workunit.client.0.smithi067.stderr: 3500K .......... .......... .......... .......... .......... 2% 43.6M 8s 2024-07-31T14:53:01.980 INFO:tasks.workunit.client.0.smithi067.stderr: 3550K .......... .......... .......... .......... .......... 2% 23.9M 8s 2024-07-31T14:53:01.982 INFO:tasks.workunit.client.0.smithi067.stderr: 3600K .......... .......... .......... .......... .......... 2% 35.7M 8s 2024-07-31T14:53:01.985 INFO:tasks.workunit.client.0.smithi067.stderr: 3650K .......... .......... .......... .......... .......... 2% 23.9M 7s 2024-07-31T14:53:01.986 INFO:tasks.workunit.client.0.smithi067.stderr: 3700K .......... .......... .......... .......... .......... 2% 24.7M 7s 2024-07-31T14:53:01.987 INFO:tasks.workunit.client.0.smithi067.stderr: 3750K .......... .......... .......... .......... .......... 2% 54.5M 7s 2024-07-31T14:53:01.989 INFO:tasks.workunit.client.0.smithi067.stderr: 3800K .......... .......... .......... .......... .......... 2% 22.8M 7s 2024-07-31T14:53:01.991 INFO:tasks.workunit.client.0.smithi067.stderr: 3850K .......... .......... .......... .......... .......... 2% 35.7M 7s 2024-07-31T14:53:01.993 INFO:tasks.workunit.client.0.smithi067.stderr: 3900K .......... .......... .......... .......... .......... 2% 23.1M 7s 2024-07-31T14:53:01.995 INFO:tasks.workunit.client.0.smithi067.stderr: 3950K .......... .......... .......... .......... .......... 2% 26.7M 7s 2024-07-31T14:53:01.995 INFO:tasks.workunit.client.0.smithi067.stderr: 4000K .......... .......... .......... .......... .......... 2% 49.7M 7s 2024-07-31T14:53:01.997 INFO:tasks.workunit.client.0.smithi067.stderr: 4050K .......... .......... .......... .......... .......... 3% 23.4M 7s 2024-07-31T14:53:01.999 INFO:tasks.workunit.client.0.smithi067.stderr: 4100K .......... .......... .......... .......... .......... 3% 37.2M 7s 2024-07-31T14:53:02.001 INFO:tasks.workunit.client.0.smithi067.stderr: 4150K .......... .......... .......... .......... .......... 3% 23.6M 7s 2024-07-31T14:53:02.002 INFO:tasks.workunit.client.0.smithi067.stderr: 4200K .......... .......... .......... .......... .......... 3% 30.0M 7s 2024-07-31T14:53:02.004 INFO:tasks.workunit.client.0.smithi067.stderr: 4250K .......... .......... .......... .......... .......... 3% 34.3M 7s 2024-07-31T14:53:02.006 INFO:tasks.workunit.client.0.smithi067.stderr: 4300K .......... .......... .......... .......... .......... 3% 25.7M 7s 2024-07-31T14:53:02.007 INFO:tasks.workunit.client.0.smithi067.stderr: 4350K .......... .......... .......... .......... .......... 3% 38.5M 7s 2024-07-31T14:53:02.009 INFO:tasks.workunit.client.0.smithi067.stderr: 4400K .......... .......... .......... .......... .......... 3% 22.4M 7s 2024-07-31T14:53:02.011 INFO:tasks.workunit.client.0.smithi067.stderr: 4450K .......... .......... .......... .......... .......... 3% 24.7M 7s 2024-07-31T14:53:02.012 INFO:tasks.workunit.client.0.smithi067.stderr: 4500K .......... .......... .......... .......... .......... 3% 50.3M 7s 2024-07-31T14:53:02.014 INFO:tasks.workunit.client.0.smithi067.stderr: 4550K .......... .......... .......... .......... .......... 3% 26.5M 7s 2024-07-31T14:53:02.015 INFO:tasks.workunit.client.0.smithi067.stderr: 4600K .......... .......... .......... .......... .......... 3% 32.9M 7s 2024-07-31T14:53:02.018 INFO:tasks.workunit.client.0.smithi067.stderr: 4650K .......... .......... .......... .......... .......... 3% 18.9M 7s 2024-07-31T14:53:02.020 INFO:tasks.workunit.client.0.smithi067.stderr: 4700K .......... .......... .......... .......... .......... 3% 32.8M 7s 2024-07-31T14:53:02.020 INFO:tasks.workunit.client.0.smithi067.stderr: 4750K .......... .......... .......... .......... .......... 3% 50.2M 7s 2024-07-31T14:53:02.022 INFO:tasks.workunit.client.0.smithi067.stderr: 4800K .......... .......... .......... .......... .......... 3% 25.9M 7s 2024-07-31T14:53:02.025 INFO:tasks.workunit.client.0.smithi067.stderr: 4850K .......... .......... .......... .......... .......... 3% 21.7M 7s 2024-07-31T14:53:02.026 INFO:tasks.workunit.client.0.smithi067.stderr: 4900K .......... .......... .......... .......... .......... 3% 45.7M 7s 2024-07-31T14:53:02.028 INFO:tasks.workunit.client.0.smithi067.stderr: 4950K .......... .......... .......... .......... .......... 3% 22.7M 7s 2024-07-31T14:53:02.029 INFO:tasks.workunit.client.0.smithi067.stderr: 5000K .......... .......... .......... .......... .......... 3% 43.1M 7s 2024-07-31T14:53:02.031 INFO:tasks.workunit.client.0.smithi067.stderr: 5050K .......... .......... .......... .......... .......... 3% 25.7M 7s 2024-07-31T14:53:02.032 INFO:tasks.workunit.client.0.smithi067.stderr: 5100K .......... .......... .......... .......... .......... 3% 40.5M 7s 2024-07-31T14:53:02.034 INFO:tasks.workunit.client.0.smithi067.stderr: 5150K .......... .......... .......... .......... .......... 3% 21.7M 7s 2024-07-31T14:53:02.036 INFO:tasks.workunit.client.0.smithi067.stderr: 5200K .......... .......... .......... .......... .......... 3% 23.3M 7s 2024-07-31T14:53:02.037 INFO:tasks.workunit.client.0.smithi067.stderr: 5250K .......... .......... .......... .......... .......... 3% 59.9M 6s 2024-07-31T14:53:02.039 INFO:tasks.workunit.client.0.smithi067.stderr: 5300K .......... .......... .......... .......... .......... 3% 25.0M 6s 2024-07-31T14:53:02.041 INFO:tasks.workunit.client.0.smithi067.stderr: 5350K .......... .......... .......... .......... .......... 3% 34.7M 6s 2024-07-31T14:53:02.043 INFO:tasks.workunit.client.0.smithi067.stderr: 5400K .......... .......... .......... .......... .......... 4% 28.4M 6s 2024-07-31T14:53:02.046 INFO:tasks.workunit.client.0.smithi067.stderr: 5450K .......... .......... .......... .......... .......... 4% 20.1M 6s 2024-07-31T14:53:02.046 INFO:tasks.workunit.client.0.smithi067.stderr: 5500K .......... .......... .......... .......... .......... 4% 64.5M 6s 2024-07-31T14:53:02.047 INFO:tasks.workunit.client.0.smithi067.stderr: 5550K .......... .......... .......... .......... .......... 4% 23.8M 6s 2024-07-31T14:53:02.049 INFO:tasks.workunit.client.0.smithi067.stderr: 5600K .......... .......... .......... .......... .......... 4% 35.3M 6s 2024-07-31T14:53:02.051 INFO:tasks.workunit.client.0.smithi067.stderr: 5650K .......... .......... .......... .......... .......... 4% 23.2M 6s 2024-07-31T14:53:02.053 INFO:tasks.workunit.client.0.smithi067.stderr: 5700K .......... .......... .......... .......... .......... 4% 22.9M 6s 2024-07-31T14:53:02.054 INFO:tasks.workunit.client.0.smithi067.stderr: 5750K .......... .......... .......... .......... .......... 4% 61.7M 6s 2024-07-31T14:53:02.056 INFO:tasks.workunit.client.0.smithi067.stderr: 5800K .......... .......... .......... .......... .......... 4% 23.7M 6s 2024-07-31T14:53:02.057 INFO:tasks.workunit.client.0.smithi067.stderr: 5850K .......... .......... .......... .......... .......... 4% 37.2M 6s 2024-07-31T14:53:02.059 INFO:tasks.workunit.client.0.smithi067.stderr: 5900K .......... .......... .......... .......... .......... 4% 23.3M 6s 2024-07-31T14:53:02.062 INFO:tasks.workunit.client.0.smithi067.stderr: 5950K .......... .......... .......... .......... .......... 4% 23.5M 6s 2024-07-31T14:53:02.062 INFO:tasks.workunit.client.0.smithi067.stderr: 6000K .......... .......... .......... .......... .......... 4% 52.1M 6s 2024-07-31T14:53:02.064 INFO:tasks.workunit.client.0.smithi067.stderr: 6050K .......... .......... .......... .......... .......... 4% 24.9M 6s 2024-07-31T14:53:02.066 INFO:tasks.workunit.client.0.smithi067.stderr: 6100K .......... .......... .......... .......... .......... 4% 38.3M 6s 2024-07-31T14:53:02.068 INFO:tasks.workunit.client.0.smithi067.stderr: 6150K .......... .......... .......... .......... .......... 4% 21.7M 6s 2024-07-31T14:53:02.070 INFO:tasks.workunit.client.0.smithi067.stderr: 6200K .......... .......... .......... .......... .......... 4% 25.0M 6s 2024-07-31T14:53:02.071 INFO:tasks.workunit.client.0.smithi067.stderr: 6250K .......... .......... .......... .......... .......... 4% 53.6M 6s 2024-07-31T14:53:02.073 INFO:tasks.workunit.client.0.smithi067.stderr: 6300K .......... .......... .......... .......... .......... 4% 24.1M 6s 2024-07-31T14:53:02.075 INFO:tasks.workunit.client.0.smithi067.stderr: 6350K .......... .......... .......... .......... .......... 4% 39.3M 6s 2024-07-31T14:53:02.076 INFO:tasks.workunit.client.0.smithi067.stderr: 6400K .......... .......... .......... .......... .......... 4% 22.0M 6s 2024-07-31T14:53:02.079 INFO:tasks.workunit.client.0.smithi067.stderr: 6450K .......... .......... .......... .......... .......... 4% 24.3M 6s 2024-07-31T14:53:02.080 INFO:tasks.workunit.client.0.smithi067.stderr: 6500K .......... .......... .......... .......... .......... 4% 49.4M 6s 2024-07-31T14:53:02.081 INFO:tasks.workunit.client.0.smithi067.stderr: 6550K .......... .......... .......... .......... .......... 4% 30.0M 6s 2024-07-31T14:53:02.083 INFO:tasks.workunit.client.0.smithi067.stderr: 6600K .......... .......... .......... .......... .......... 4% 20.6M 6s 2024-07-31T14:53:02.085 INFO:tasks.workunit.client.0.smithi067.stderr: 6650K .......... .......... .......... .......... .......... 4% 39.5M 6s 2024-07-31T14:53:02.087 INFO:tasks.workunit.client.0.smithi067.stderr: 6700K .......... .......... .......... .......... .......... 4% 21.2M 6s 2024-07-31T14:53:02.089 INFO:tasks.workunit.client.0.smithi067.stderr: 6750K .......... .......... .......... .......... .......... 5% 70.5M 6s 2024-07-31T14:53:02.090 INFO:tasks.workunit.client.0.smithi067.stderr: 6800K .......... .......... .......... .......... .......... 5% 24.8M 6s 2024-07-31T14:53:02.092 INFO:tasks.workunit.client.0.smithi067.stderr: 6850K .......... .......... .......... .......... .......... 5% 21.3M 6s 2024-07-31T14:53:02.094 INFO:tasks.workunit.client.0.smithi067.stderr: 6900K .......... .......... .......... .......... .......... 5% 31.0M 6s 2024-07-31T14:53:02.095 INFO:tasks.workunit.client.0.smithi067.stderr: 6950K .......... .......... .......... .......... .......... 5% 26.9M 6s 2024-07-31T14:53:02.097 INFO:tasks.workunit.client.0.smithi067.stderr: 7000K .......... .......... .......... .......... .......... 5% 23.1M 6s 2024-07-31T14:53:02.098 INFO:tasks.workunit.client.0.smithi067.stderr: 7050K .......... .......... .......... .......... .......... 5% 52.2M 6s 2024-07-31T14:53:02.100 INFO:tasks.workunit.client.0.smithi067.stderr: 7100K .......... .......... .......... .......... .......... 5% 34.4M 6s 2024-07-31T14:53:02.102 INFO:tasks.workunit.client.0.smithi067.stderr: 7150K .......... .......... .......... .......... .......... 5% 25.8M 6s 2024-07-31T14:53:02.103 INFO:tasks.workunit.client.0.smithi067.stderr: 7200K .......... .......... .......... .......... .......... 5% 26.8M 6s 2024-07-31T14:53:02.106 INFO:tasks.workunit.client.0.smithi067.stderr: 7250K .......... .......... .......... .......... .......... 5% 20.5M 6s 2024-07-31T14:53:02.106 INFO:tasks.workunit.client.0.smithi067.stderr: 7300K .......... .......... .......... .......... .......... 5% 56.8M 6s 2024-07-31T14:53:02.108 INFO:tasks.workunit.client.0.smithi067.stderr: 7350K .......... .......... .......... .......... .......... 5% 24.3M 6s 2024-07-31T14:53:02.110 INFO:tasks.workunit.client.0.smithi067.stderr: 7400K .......... .......... .......... .......... .......... 5% 35.0M 6s 2024-07-31T14:53:02.136 INFO:tasks.workunit.client.0.smithi067.stderr: 7450K .......... .......... .......... .......... .......... 5% 27.1M 6s 2024-07-31T14:53:02.136 INFO:tasks.workunit.client.0.smithi067.stderr: 7500K .......... .......... .......... .......... .......... 5% 25.0M 6s 2024-07-31T14:53:02.136 INFO:tasks.workunit.client.0.smithi067.stderr: 7550K .......... .......... .......... .......... .......... 5% 40.0M 6s 2024-07-31T14:53:02.136 INFO:tasks.workunit.client.0.smithi067.stderr: 7600K .......... .......... .......... .......... .......... 5% 24.4M 6s 2024-07-31T14:53:02.136 INFO:tasks.workunit.client.0.smithi067.stderr: 7650K .......... .......... .......... .......... .......... 5% 37.8M 6s 2024-07-31T14:53:02.137 INFO:tasks.workunit.client.0.smithi067.stderr: 7700K .......... .......... .......... .......... .......... 5% 22.2M 6s 2024-07-31T14:53:02.137 INFO:tasks.workunit.client.0.smithi067.stderr: 7750K .......... .......... .......... .......... .......... 5% 25.4M 6s 2024-07-31T14:53:02.137 INFO:tasks.workunit.client.0.smithi067.stderr: 7800K .......... .......... .......... .......... .......... 5% 55.6M 6s 2024-07-31T14:53:02.137 INFO:tasks.workunit.client.0.smithi067.stderr: 7850K .......... .......... .......... .......... .......... 5% 34.6M 6s 2024-07-31T14:53:02.137 INFO:tasks.workunit.client.0.smithi067.stderr: 7900K .......... .......... .......... .......... .......... 5% 23.3M 6s 2024-07-31T14:53:02.137 INFO:tasks.workunit.client.0.smithi067.stderr: 7950K .......... .......... .......... .......... .......... 5% 21.8M 6s 2024-07-31T14:53:02.137 INFO:tasks.workunit.client.0.smithi067.stderr: 8000K .......... .......... .......... .......... .......... 5% 23.1M 6s 2024-07-31T14:53:02.137 INFO:tasks.workunit.client.0.smithi067.stderr: 8050K .......... .......... .......... .......... .......... 5% 65.5M 6s 2024-07-31T14:53:02.137 INFO:tasks.workunit.client.0.smithi067.stderr: 8100K .......... .......... .......... .......... .......... 6% 24.0M 6s 2024-07-31T14:53:02.137 INFO:tasks.workunit.client.0.smithi067.stderr: 8150K .......... .......... .......... .......... .......... 6% 35.3M 6s 2024-07-31T14:53:02.138 INFO:tasks.workunit.client.0.smithi067.stderr: 8200K .......... .......... .......... .......... .......... 6% 24.6M 6s 2024-07-31T14:53:02.139 INFO:tasks.workunit.client.0.smithi067.stderr: 8250K .......... .......... .......... .......... .......... 6% 23.3M 6s 2024-07-31T14:53:02.140 INFO:tasks.workunit.client.0.smithi067.stderr: 8300K .......... .......... .......... .......... .......... 6% 55.1M 6s 2024-07-31T14:53:02.142 INFO:tasks.workunit.client.0.smithi067.stderr: 8350K .......... .......... .......... .......... .......... 6% 23.2M 6s 2024-07-31T14:53:02.145 INFO:tasks.workunit.client.0.smithi067.stderr: 8400K .......... .......... .......... .......... .......... 6% 24.4M 6s 2024-07-31T14:53:02.145 INFO:tasks.workunit.client.0.smithi067.stderr: 8450K .......... .......... .......... .......... .......... 6% 63.7M 6s 2024-07-31T14:53:02.148 INFO:tasks.workunit.client.0.smithi067.stderr: 8500K .......... .......... .......... .......... .......... 6% 18.1M 6s 2024-07-31T14:53:02.149 INFO:tasks.workunit.client.0.smithi067.stderr: 8550K .......... .......... .......... .......... .......... 6% 53.8M 6s 2024-07-31T14:53:02.155 INFO:tasks.workunit.client.0.smithi067.stderr: 8600K .......... .......... .......... .......... .......... 6% 13.3M 6s 2024-07-31T14:53:02.155 INFO:tasks.workunit.client.0.smithi067.stderr: 8650K .......... .......... .......... .......... .......... 6% 146M 6s 2024-07-31T14:53:02.155 INFO:tasks.workunit.client.0.smithi067.stderr: 8700K .......... .......... .......... .......... .......... 6% 31.9M 6s 2024-07-31T14:53:02.156 INFO:tasks.workunit.client.0.smithi067.stderr: 8750K .......... .......... .......... .......... .......... 6% 23.9M 6s 2024-07-31T14:53:02.157 INFO:tasks.workunit.client.0.smithi067.stderr: 8800K .......... .......... .......... .......... .......... 6% 54.6M 5s 2024-07-31T14:53:02.159 INFO:tasks.workunit.client.0.smithi067.stderr: 8850K .......... .......... .......... .......... .......... 6% 24.0M 5s 2024-07-31T14:53:02.161 INFO:tasks.workunit.client.0.smithi067.stderr: 8900K .......... .......... .......... .......... .......... 6% 34.3M 5s 2024-07-31T14:53:02.163 INFO:tasks.workunit.client.0.smithi067.stderr: 8950K .......... .......... .......... .......... .......... 6% 24.0M 5s 2024-07-31T14:53:02.165 INFO:tasks.workunit.client.0.smithi067.stderr: 9000K .......... .......... .......... .......... .......... 6% 22.7M 5s 2024-07-31T14:53:02.165 INFO:tasks.workunit.client.0.smithi067.stderr: 9050K .......... .......... .......... .......... .......... 6% 69.7M 5s 2024-07-31T14:53:02.167 INFO:tasks.workunit.client.0.smithi067.stderr: 9100K .......... .......... .......... .......... .......... 6% 24.0M 5s 2024-07-31T14:53:02.169 INFO:tasks.workunit.client.0.smithi067.stderr: 9150K .......... .......... .......... .......... .......... 6% 37.9M 5s 2024-07-31T14:53:02.171 INFO:tasks.workunit.client.0.smithi067.stderr: 9200K .......... .......... .......... .......... .......... 6% 22.0M 5s 2024-07-31T14:53:02.173 INFO:tasks.workunit.client.0.smithi067.stderr: 9250K .......... .......... .......... .......... .......... 6% 24.5M 5s 2024-07-31T14:53:02.174 INFO:tasks.workunit.client.0.smithi067.stderr: 9300K .......... .......... .......... .......... .......... 6% 56.9M 5s 2024-07-31T14:53:02.176 INFO:tasks.workunit.client.0.smithi067.stderr: 9350K .......... .......... .......... .......... .......... 6% 24.0M 5s 2024-07-31T14:53:02.177 INFO:tasks.workunit.client.0.smithi067.stderr: 9400K .......... .......... .......... .......... .......... 6% 34.3M 5s 2024-07-31T14:53:02.180 INFO:tasks.workunit.client.0.smithi067.stderr: 9450K .......... .......... .......... .......... .......... 6% 23.1M 5s 2024-07-31T14:53:02.181 INFO:tasks.workunit.client.0.smithi067.stderr: 9500K .......... .......... .......... .......... .......... 7% 25.3M 5s 2024-07-31T14:53:02.182 INFO:tasks.workunit.client.0.smithi067.stderr: 9550K .......... .......... .......... .......... .......... 7% 53.6M 5s 2024-07-31T14:53:02.184 INFO:tasks.workunit.client.0.smithi067.stderr: 9600K .......... .......... .......... .......... .......... 7% 23.6M 5s 2024-07-31T14:53:02.186 INFO:tasks.workunit.client.0.smithi067.stderr: 9650K .......... .......... .......... .......... .......... 7% 34.9M 5s 2024-07-31T14:53:02.188 INFO:tasks.workunit.client.0.smithi067.stderr: 9700K .......... .......... .......... .......... .......... 7% 22.4M 5s 2024-07-31T14:53:02.190 INFO:tasks.workunit.client.0.smithi067.stderr: 9750K .......... .......... .......... .......... .......... 7% 28.1M 5s 2024-07-31T14:53:02.191 INFO:tasks.workunit.client.0.smithi067.stderr: 9800K .......... .......... .......... .......... .......... 7% 45.7M 5s 2024-07-31T14:53:02.193 INFO:tasks.workunit.client.0.smithi067.stderr: 9850K .......... .......... .......... .......... .......... 7% 24.2M 5s 2024-07-31T14:53:02.195 INFO:tasks.workunit.client.0.smithi067.stderr: 9900K .......... .......... .......... .......... .......... 7% 27.8M 5s 2024-07-31T14:53:02.195 INFO:tasks.workunit.client.0.smithi067.stderr: 9950K .......... .......... .......... .......... .......... 7% 47.8M 5s 2024-07-31T14:53:02.198 INFO:tasks.workunit.client.0.smithi067.stderr: 10000K .......... .......... .......... .......... .......... 7% 19.2M 5s 2024-07-31T14:53:02.199 INFO:tasks.workunit.client.0.smithi067.stderr: 10050K .......... .......... .......... .......... .......... 7% 46.7M 5s 2024-07-31T14:53:02.201 INFO:tasks.workunit.client.0.smithi067.stderr: 10100K .......... .......... .......... .......... .......... 7% 23.1M 5s 2024-07-31T14:53:02.203 INFO:tasks.workunit.client.0.smithi067.stderr: 10150K .......... .......... .......... .......... .......... 7% 23.8M 5s 2024-07-31T14:53:02.204 INFO:tasks.workunit.client.0.smithi067.stderr: 10200K .......... .......... .......... .......... .......... 7% 40.9M 5s 2024-07-31T14:53:02.207 INFO:tasks.workunit.client.0.smithi067.stderr: 10250K .......... .......... .......... .......... .......... 7% 22.8M 5s 2024-07-31T14:53:02.208 INFO:tasks.workunit.client.0.smithi067.stderr: 10300K .......... .......... .......... .......... .......... 7% 50.5M 5s 2024-07-31T14:53:02.210 INFO:tasks.workunit.client.0.smithi067.stderr: 10350K .......... .......... .......... .......... .......... 7% 24.7M 5s 2024-07-31T14:53:02.211 INFO:tasks.workunit.client.0.smithi067.stderr: 10400K .......... .......... .......... .......... .......... 7% 38.5M 5s 2024-07-31T14:53:02.213 INFO:tasks.workunit.client.0.smithi067.stderr: 10450K .......... .......... .......... .......... .......... 7% 22.5M 5s 2024-07-31T14:53:02.216 INFO:tasks.workunit.client.0.smithi067.stderr: 10500K .......... .......... .......... .......... .......... 7% 23.3M 5s 2024-07-31T14:53:02.216 INFO:tasks.workunit.client.0.smithi067.stderr: 10550K .......... .......... .......... .......... .......... 7% 64.5M 5s 2024-07-31T14:53:02.218 INFO:tasks.workunit.client.0.smithi067.stderr: 10600K .......... .......... .......... .......... .......... 7% 23.7M 5s 2024-07-31T14:53:02.220 INFO:tasks.workunit.client.0.smithi067.stderr: 10650K .......... .......... .......... .......... .......... 7% 22.7M 5s 2024-07-31T14:53:02.222 INFO:tasks.workunit.client.0.smithi067.stderr: 10700K .......... .......... .......... .......... .......... 7% 36.1M 5s 2024-07-31T14:53:02.224 INFO:tasks.workunit.client.0.smithi067.stderr: 10750K .......... .......... .......... .......... .......... 7% 23.8M 5s 2024-07-31T14:53:02.224 INFO:tasks.workunit.client.0.smithi067.stderr: 10800K .......... .......... .......... .......... .......... 7% 61.6M 5s 2024-07-31T14:53:02.226 INFO:tasks.workunit.client.0.smithi067.stderr: 10850K .......... .......... .......... .......... .......... 8% 22.4M 5s 2024-07-31T14:53:02.227 INFO:tasks.workunit.client.0.smithi067.stderr: 10900K .......... .......... .......... .......... .......... 8% 42.0M 5s 2024-07-31T14:53:02.230 INFO:tasks.workunit.client.0.smithi067.stderr: 10950K .......... .......... .......... .......... .......... 8% 21.2M 5s 2024-07-31T14:53:02.233 INFO:tasks.workunit.client.0.smithi067.stderr: 11000K .......... .......... .......... .......... .......... 8% 23.7M 5s 2024-07-31T14:53:02.233 INFO:tasks.workunit.client.0.smithi067.stderr: 11050K .......... .......... .......... .......... .......... 8% 86.7M 5s 2024-07-31T14:53:02.235 INFO:tasks.workunit.client.0.smithi067.stderr: 11100K .......... .......... .......... .......... .......... 8% 22.1M 5s 2024-07-31T14:53:02.237 INFO:tasks.workunit.client.0.smithi067.stderr: 11150K .......... .......... .......... .......... .......... 8% 23.9M 5s 2024-07-31T14:53:02.238 INFO:tasks.workunit.client.0.smithi067.stderr: 11200K .......... .......... .......... .......... .......... 8% 48.0M 5s 2024-07-31T14:53:02.239 INFO:tasks.workunit.client.0.smithi067.stderr: 11250K .......... .......... .......... .......... .......... 8% 29.7M 5s 2024-07-31T14:53:02.241 INFO:tasks.workunit.client.0.smithi067.stderr: 11300K .......... .......... .......... .......... .......... 8% 32.2M 5s 2024-07-31T14:53:02.243 INFO:tasks.workunit.client.0.smithi067.stderr: 11350K .......... .......... .......... .......... .......... 8% 24.9M 5s 2024-07-31T14:53:02.245 INFO:tasks.workunit.client.0.smithi067.stderr: 11400K .......... .......... .......... .......... .......... 8% 35.6M 5s 2024-07-31T14:53:02.247 INFO:tasks.workunit.client.0.smithi067.stderr: 11450K .......... .......... .......... .......... .......... 8% 23.1M 5s 2024-07-31T14:53:02.249 INFO:tasks.workunit.client.0.smithi067.stderr: 11500K .......... .......... .......... .......... .......... 8% 23.8M 5s 2024-07-31T14:53:02.249 INFO:tasks.workunit.client.0.smithi067.stderr: 11550K .......... .......... .......... .......... .......... 8% 58.1M 5s 2024-07-31T14:53:02.251 INFO:tasks.workunit.client.0.smithi067.stderr: 11600K .......... .......... .......... .......... .......... 8% 23.5M 5s 2024-07-31T14:53:02.253 INFO:tasks.workunit.client.0.smithi067.stderr: 11650K .......... .......... .......... .......... .......... 8% 32.6M 5s 2024-07-31T14:53:02.255 INFO:tasks.workunit.client.0.smithi067.stderr: 11700K .......... .......... .......... .......... .......... 8% 26.6M 5s 2024-07-31T14:53:02.257 INFO:tasks.workunit.client.0.smithi067.stderr: 11750K .......... .......... .......... .......... .......... 8% 23.7M 5s 2024-07-31T14:53:02.259 INFO:tasks.workunit.client.0.smithi067.stderr: 11800K .......... .......... .......... .......... .......... 8% 48.7M 5s 2024-07-31T14:53:02.260 INFO:tasks.workunit.client.0.smithi067.stderr: 11850K .......... .......... .......... .......... .......... 8% 25.4M 5s 2024-07-31T14:53:02.262 INFO:tasks.workunit.client.0.smithi067.stderr: 11900K .......... .......... .......... .......... .......... 8% 23.6M 5s 2024-07-31T14:53:02.263 INFO:tasks.workunit.client.0.smithi067.stderr: 11950K .......... .......... .......... .......... .......... 8% 36.5M 5s 2024-07-31T14:53:02.265 INFO:tasks.workunit.client.0.smithi067.stderr: 12000K .......... .......... .......... .......... .......... 8% 23.8M 5s 2024-07-31T14:53:02.267 INFO:tasks.workunit.client.0.smithi067.stderr: 12050K .......... .......... .......... .......... .......... 8% 52.1M 5s 2024-07-31T14:53:02.268 INFO:tasks.workunit.client.0.smithi067.stderr: 12100K .......... .......... .......... .......... .......... 8% 23.4M 5s 2024-07-31T14:53:02.270 INFO:tasks.workunit.client.0.smithi067.stderr: 12150K .......... .......... .......... .......... .......... 8% 23.4M 5s 2024-07-31T14:53:02.272 INFO:tasks.workunit.client.0.smithi067.stderr: 12200K .......... .......... .......... .......... .......... 9% 37.6M 5s 2024-07-31T14:53:02.274 INFO:tasks.workunit.client.0.smithi067.stderr: 12250K .......... .......... .......... .......... .......... 9% 23.8M 5s 2024-07-31T14:53:02.276 INFO:tasks.workunit.client.0.smithi067.stderr: 12300K .......... .......... .......... .......... .......... 9% 23.8M 5s 2024-07-31T14:53:02.277 INFO:tasks.workunit.client.0.smithi067.stderr: 12350K .......... .......... .......... .......... .......... 9% 54.3M 5s 2024-07-31T14:53:02.279 INFO:tasks.workunit.client.0.smithi067.stderr: 12400K .......... .......... .......... .......... .......... 9% 23.5M 5s 2024-07-31T14:53:02.280 INFO:tasks.workunit.client.0.smithi067.stderr: 12450K .......... .......... .......... .......... .......... 9% 37.0M 5s 2024-07-31T14:53:02.282 INFO:tasks.workunit.client.0.smithi067.stderr: 12500K .......... .......... .......... .......... .......... 9% 23.1M 5s 2024-07-31T14:53:02.284 INFO:tasks.workunit.client.0.smithi067.stderr: 12550K .......... .......... .......... .......... .......... 9% 24.0M 5s 2024-07-31T14:53:02.285 INFO:tasks.workunit.client.0.smithi067.stderr: 12600K .......... .......... .......... .......... .......... 9% 51.8M 5s 2024-07-31T14:53:02.287 INFO:tasks.workunit.client.0.smithi067.stderr: 12650K .......... .......... .......... .......... .......... 9% 23.3M 5s 2024-07-31T14:53:02.289 INFO:tasks.workunit.client.0.smithi067.stderr: 12700K .......... .......... .......... .......... .......... 9% 30.6M 5s 2024-07-31T14:53:02.291 INFO:tasks.workunit.client.0.smithi067.stderr: 12750K .......... .......... .......... .......... .......... 9% 23.8M 5s 2024-07-31T14:53:02.294 INFO:tasks.workunit.client.0.smithi067.stderr: 12800K .......... .......... .......... .......... .......... 9% 24.5M 5s 2024-07-31T14:53:02.294 INFO:tasks.workunit.client.0.smithi067.stderr: 12850K .......... .......... .......... .......... .......... 9% 49.2M 5s 2024-07-31T14:53:02.296 INFO:tasks.workunit.client.0.smithi067.stderr: 12900K .......... .......... .......... .......... .......... 9% 24.0M 5s 2024-07-31T14:53:02.297 INFO:tasks.workunit.client.0.smithi067.stderr: 12950K .......... .......... .......... .......... .......... 9% 35.3M 5s 2024-07-31T14:53:02.300 INFO:tasks.workunit.client.0.smithi067.stderr: 13000K .......... .......... .......... .......... .......... 9% 24.1M 5s 2024-07-31T14:53:02.302 INFO:tasks.workunit.client.0.smithi067.stderr: 13050K .......... .......... .......... .......... .......... 9% 24.4M 5s 2024-07-31T14:53:02.302 INFO:tasks.workunit.client.0.smithi067.stderr: 13100K .......... .......... .......... .......... .......... 9% 56.1M 5s 2024-07-31T14:53:02.304 INFO:tasks.workunit.client.0.smithi067.stderr: 13150K .......... .......... .......... .......... .......... 9% 23.2M 5s 2024-07-31T14:53:02.306 INFO:tasks.workunit.client.0.smithi067.stderr: 13200K .......... .......... .......... .......... .......... 9% 36.0M 5s 2024-07-31T14:53:02.308 INFO:tasks.workunit.client.0.smithi067.stderr: 13250K .......... .......... .......... .......... .......... 9% 24.5M 5s 2024-07-31T14:53:02.310 INFO:tasks.workunit.client.0.smithi067.stderr: 13300K .......... .......... .......... .......... .......... 9% 23.4M 5s 2024-07-31T14:53:02.311 INFO:tasks.workunit.client.0.smithi067.stderr: 13350K .......... .......... .......... .......... .......... 9% 54.0M 5s 2024-07-31T14:53:02.313 INFO:tasks.workunit.client.0.smithi067.stderr: 13400K .......... .......... .......... .......... .......... 9% 24.0M 5s 2024-07-31T14:53:02.320 INFO:tasks.workunit.client.0.smithi067.stderr: 13450K .......... .......... .......... .......... .......... 9% 38.2M 5s 2024-07-31T14:53:02.320 INFO:tasks.workunit.client.0.smithi067.stderr: 13500K .......... .......... .......... .......... .......... 9% 24.1M 5s 2024-07-31T14:53:02.320 INFO:tasks.workunit.client.0.smithi067.stderr: 13550K .......... .......... .......... .......... .......... 10% 47.5M 5s 2024-07-31T14:53:02.320 INFO:tasks.workunit.client.0.smithi067.stderr: 13600K .......... .......... .......... .......... .......... 10% 25.4M 5s 2024-07-31T14:53:02.321 INFO:tasks.workunit.client.0.smithi067.stderr: 13650K .......... .......... .......... .......... .......... 10% 24.7M 5s 2024-07-31T14:53:02.322 INFO:tasks.workunit.client.0.smithi067.stderr: 13700K .......... .......... .......... .......... .......... 10% 35.3M 5s 2024-07-31T14:53:02.325 INFO:tasks.workunit.client.0.smithi067.stderr: 13750K .......... .......... .......... .......... .......... 10% 22.9M 5s 2024-07-31T14:53:02.326 INFO:tasks.workunit.client.0.smithi067.stderr: 13800K .......... .......... .......... .......... .......... 10% 33.3M 5s 2024-07-31T14:53:02.328 INFO:tasks.workunit.client.0.smithi067.stderr: 13850K .......... .......... .......... .......... .......... 10% 31.1M 5s 2024-07-31T14:53:02.330 INFO:tasks.workunit.client.0.smithi067.stderr: 13900K .......... .......... .......... .......... .......... 10% 23.9M 5s 2024-07-31T14:53:02.332 INFO:tasks.workunit.client.0.smithi067.stderr: 13950K .......... .......... .......... .......... .......... 10% 43.6M 5s 2024-07-31T14:53:02.333 INFO:tasks.workunit.client.0.smithi067.stderr: 14000K .......... .......... .......... .......... .......... 10% 22.1M 5s 2024-07-31T14:53:02.335 INFO:tasks.workunit.client.0.smithi067.stderr: 14050K .......... .......... .......... .......... .......... 10% 24.1M 5s 2024-07-31T14:53:02.336 INFO:tasks.workunit.client.0.smithi067.stderr: 14100K .......... .......... .......... .......... .......... 10% 59.4M 5s 2024-07-31T14:53:02.338 INFO:tasks.workunit.client.0.smithi067.stderr: 14150K .......... .......... .......... .......... .......... 10% 23.6M 5s 2024-07-31T14:53:02.339 INFO:tasks.workunit.client.0.smithi067.stderr: 14200K .......... .......... .......... .......... .......... 10% 36.7M 5s 2024-07-31T14:53:02.341 INFO:tasks.workunit.client.0.smithi067.stderr: 14250K .......... .......... .......... .......... .......... 10% 22.5M 5s 2024-07-31T14:53:02.344 INFO:tasks.workunit.client.0.smithi067.stderr: 14300K .......... .......... .......... .......... .......... 10% 23.7M 5s 2024-07-31T14:53:02.344 INFO:tasks.workunit.client.0.smithi067.stderr: 14350K .......... .......... .......... .......... .......... 10% 51.9M 5s 2024-07-31T14:53:02.346 INFO:tasks.workunit.client.0.smithi067.stderr: 14400K .......... .......... .......... .......... .......... 10% 29.3M 5s 2024-07-31T14:53:02.348 INFO:tasks.workunit.client.0.smithi067.stderr: 14450K .......... .......... .......... .......... .......... 10% 25.1M 5s 2024-07-31T14:53:02.350 INFO:tasks.workunit.client.0.smithi067.stderr: 14500K .......... .......... .......... .......... .......... 10% 26.9M 5s 2024-07-31T14:53:02.352 INFO:tasks.workunit.client.0.smithi067.stderr: 14550K .......... .......... .......... .......... .......... 10% 22.4M 5s 2024-07-31T14:53:02.353 INFO:tasks.workunit.client.0.smithi067.stderr: 14600K .......... .......... .......... .......... .......... 10% 68.0M 5s 2024-07-31T14:53:02.355 INFO:tasks.workunit.client.0.smithi067.stderr: 14650K .......... .......... .......... .......... .......... 10% 23.3M 5s 2024-07-31T14:53:02.356 INFO:tasks.workunit.client.0.smithi067.stderr: 14700K .......... .......... .......... .......... .......... 10% 33.1M 5s 2024-07-31T14:53:02.359 INFO:tasks.workunit.client.0.smithi067.stderr: 14750K .......... .......... .......... .......... .......... 10% 23.7M 5s 2024-07-31T14:53:02.360 INFO:tasks.workunit.client.0.smithi067.stderr: 14800K .......... .......... .......... .......... .......... 10% 25.6M 5s 2024-07-31T14:53:02.361 INFO:tasks.workunit.client.0.smithi067.stderr: 14850K .......... .......... .......... .......... .......... 10% 48.1M 5s 2024-07-31T14:53:02.363 INFO:tasks.workunit.client.0.smithi067.stderr: 14900K .......... .......... .......... .......... .......... 11% 29.9M 5s 2024-07-31T14:53:02.366 INFO:tasks.workunit.client.0.smithi067.stderr: 14950K .......... .......... .......... .......... .......... 11% 19.0M 5s 2024-07-31T14:53:02.367 INFO:tasks.workunit.client.0.smithi067.stderr: 15000K .......... .......... .......... .......... .......... 11% 37.8M 5s 2024-07-31T14:53:02.368 INFO:tasks.workunit.client.0.smithi067.stderr: 15050K .......... .......... .......... .......... .......... 11% 42.1M 5s 2024-07-31T14:53:02.370 INFO:tasks.workunit.client.0.smithi067.stderr: 15100K .......... .......... .......... .......... .......... 11% 27.9M 5s 2024-07-31T14:53:02.372 INFO:tasks.workunit.client.0.smithi067.stderr: 15150K .......... .......... .......... .......... .......... 11% 22.6M 5s 2024-07-31T14:53:02.373 INFO:tasks.workunit.client.0.smithi067.stderr: 15200K .......... .......... .......... .......... .......... 11% 38.3M 5s 2024-07-31T14:53:02.375 INFO:tasks.workunit.client.0.smithi067.stderr: 15250K .......... .......... .......... .......... .......... 11% 23.1M 5s 2024-07-31T14:53:02.377 INFO:tasks.workunit.client.0.smithi067.stderr: 15300K .......... .......... .......... .......... .......... 11% 32.1M 5s 2024-07-31T14:53:02.378 INFO:tasks.workunit.client.0.smithi067.stderr: 15350K .......... .......... .......... .......... .......... 11% 33.9M 5s 2024-07-31T14:53:02.380 INFO:tasks.workunit.client.0.smithi067.stderr: 15400K .......... .......... .......... .......... .......... 11% 25.4M 5s 2024-07-31T14:53:02.382 INFO:tasks.workunit.client.0.smithi067.stderr: 15450K .......... .......... .......... .......... .......... 11% 24.5M 5s 2024-07-31T14:53:02.383 INFO:tasks.workunit.client.0.smithi067.stderr: 15500K .......... .......... .......... .......... .......... 11% 37.8M 5s 2024-07-31T14:53:02.385 INFO:tasks.workunit.client.0.smithi067.stderr: 15550K .......... .......... .......... .......... .......... 11% 22.8M 5s 2024-07-31T14:53:02.386 INFO:tasks.workunit.client.0.smithi067.stderr: 15600K .......... .......... .......... .......... .......... 11% 56.0M 5s 2024-07-31T14:53:02.389 INFO:tasks.workunit.client.0.smithi067.stderr: 15650K .......... .......... .......... .......... .......... 11% 24.8M 5s 2024-07-31T14:53:02.390 INFO:tasks.workunit.client.0.smithi067.stderr: 15700K .......... .......... .......... .......... .......... 11% 31.6M 5s 2024-07-31T14:53:02.392 INFO:tasks.workunit.client.0.smithi067.stderr: 15750K .......... .......... .......... .......... .......... 11% 23.5M 5s 2024-07-31T14:53:02.395 INFO:tasks.workunit.client.0.smithi067.stderr: 15800K .......... .......... .......... .......... .......... 11% 23.2M 5s 2024-07-31T14:53:02.395 INFO:tasks.workunit.client.0.smithi067.stderr: 15850K .......... .......... .......... .......... .......... 11% 65.3M 5s 2024-07-31T14:53:02.397 INFO:tasks.workunit.client.0.smithi067.stderr: 15900K .......... .......... .......... .......... .......... 11% 22.8M 5s 2024-07-31T14:53:02.399 INFO:tasks.workunit.client.0.smithi067.stderr: 15950K .......... .......... .......... .......... .......... 11% 31.2M 5s 2024-07-31T14:53:02.400 INFO:tasks.workunit.client.0.smithi067.stderr: 16000K .......... .......... .......... .......... .......... 11% 25.9M 5s 2024-07-31T14:53:02.403 INFO:tasks.workunit.client.0.smithi067.stderr: 16050K .......... .......... .......... .......... .......... 11% 23.9M 5s 2024-07-31T14:53:02.403 INFO:tasks.workunit.client.0.smithi067.stderr: 16100K .......... .......... .......... .......... .......... 11% 51.2M 5s 2024-07-31T14:53:02.405 INFO:tasks.workunit.client.0.smithi067.stderr: 16150K .......... .......... .......... .......... .......... 11% 23.5M 5s 2024-07-31T14:53:02.407 INFO:tasks.workunit.client.0.smithi067.stderr: 16200K .......... .......... .......... .......... .......... 11% 28.8M 5s 2024-07-31T14:53:02.409 INFO:tasks.workunit.client.0.smithi067.stderr: 16250K .......... .......... .......... .......... .......... 12% 29.0M 5s 2024-07-31T14:53:02.410 INFO:tasks.workunit.client.0.smithi067.stderr: 16300K .......... .......... .......... .......... .......... 12% 38.4M 5s 2024-07-31T14:53:02.412 INFO:tasks.workunit.client.0.smithi067.stderr: 16350K .......... .......... .......... .......... .......... 12% 30.7M 5s 2024-07-31T14:53:02.414 INFO:tasks.workunit.client.0.smithi067.stderr: 16400K .......... .......... .......... .......... .......... 12% 22.9M 5s 2024-07-31T14:53:02.416 INFO:tasks.workunit.client.0.smithi067.stderr: 16450K .......... .......... .......... .......... .......... 12% 25.4M 5s 2024-07-31T14:53:02.417 INFO:tasks.workunit.client.0.smithi067.stderr: 16500K .......... .......... .......... .......... .......... 12% 46.2M 5s 2024-07-31T14:53:02.419 INFO:tasks.workunit.client.0.smithi067.stderr: 16550K .......... .......... .......... .......... .......... 12% 21.5M 5s 2024-07-31T14:53:02.420 INFO:tasks.workunit.client.0.smithi067.stderr: 16600K .......... .......... .......... .......... .......... 12% 45.2M 5s 2024-07-31T14:53:02.422 INFO:tasks.workunit.client.0.smithi067.stderr: 16650K .......... .......... .......... .......... .......... 12% 22.2M 5s 2024-07-31T14:53:02.424 INFO:tasks.workunit.client.0.smithi067.stderr: 16700K .......... .......... .......... .......... .......... 12% 37.7M 5s 2024-07-31T14:53:02.426 INFO:tasks.workunit.client.0.smithi067.stderr: 16750K .......... .......... .......... .......... .......... 12% 22.9M 5s 2024-07-31T14:53:02.428 INFO:tasks.workunit.client.0.smithi067.stderr: 16800K .......... .......... .......... .......... .......... 12% 26.2M 5s 2024-07-31T14:53:02.428 INFO:tasks.workunit.client.0.smithi067.stderr: 16850K .......... .......... .......... .......... .......... 12% 56.0M 5s 2024-07-31T14:53:02.431 INFO:tasks.workunit.client.0.smithi067.stderr: 16900K .......... .......... .......... .......... .......... 12% 21.0M 5s 2024-07-31T14:53:02.433 INFO:tasks.workunit.client.0.smithi067.stderr: 16950K .......... .......... .......... .......... .......... 12% 39.1M 5s 2024-07-31T14:53:02.434 INFO:tasks.workunit.client.0.smithi067.stderr: 17000K .......... .......... .......... .......... .......... 12% 23.9M 5s 2024-07-31T14:53:02.437 INFO:tasks.workunit.client.0.smithi067.stderr: 17050K .......... .......... .......... .......... .......... 12% 24.6M 5s 2024-07-31T14:53:02.438 INFO:tasks.workunit.client.0.smithi067.stderr: 17100K .......... .......... .......... .......... .......... 12% 58.4M 5s 2024-07-31T14:53:02.439 INFO:tasks.workunit.client.0.smithi067.stderr: 17150K .......... .......... .......... .......... .......... 12% 23.0M 5s 2024-07-31T14:53:02.441 INFO:tasks.workunit.client.0.smithi067.stderr: 17200K .......... .......... .......... .......... .......... 12% 25.1M 5s 2024-07-31T14:53:02.442 INFO:tasks.workunit.client.0.smithi067.stderr: 17250K .......... .......... .......... .......... .......... 12% 33.7M 5s 2024-07-31T14:53:02.444 INFO:tasks.workunit.client.0.smithi067.stderr: 17300K .......... .......... .......... .......... .......... 12% 23.4M 5s 2024-07-31T14:53:02.446 INFO:tasks.workunit.client.0.smithi067.stderr: 17350K .......... .......... .......... .......... .......... 12% 37.2M 5s 2024-07-31T14:53:02.447 INFO:tasks.workunit.client.0.smithi067.stderr: 17400K .......... .......... .......... .......... .......... 12% 34.9M 5s 2024-07-31T14:53:02.449 INFO:tasks.workunit.client.0.smithi067.stderr: 17450K .......... .......... .......... .......... .......... 12% 21.7M 5s 2024-07-31T14:53:02.451 INFO:tasks.workunit.client.0.smithi067.stderr: 17500K .......... .......... .......... .......... .......... 12% 40.6M 5s 2024-07-31T14:53:02.454 INFO:tasks.workunit.client.0.smithi067.stderr: 17550K .......... .......... .......... .......... .......... 12% 23.2M 5s 2024-07-31T14:53:02.454 INFO:tasks.workunit.client.0.smithi067.stderr: 17600K .......... .......... .......... .......... .......... 13% 38.0M 5s 2024-07-31T14:53:02.456 INFO:tasks.workunit.client.0.smithi067.stderr: 17650K .......... .......... .......... .......... .......... 13% 29.2M 5s 2024-07-31T14:53:02.458 INFO:tasks.workunit.client.0.smithi067.stderr: 17700K .......... .......... .......... .......... .......... 13% 23.3M 5s 2024-07-31T14:53:02.459 INFO:tasks.workunit.client.0.smithi067.stderr: 17750K .......... .......... .......... .......... .......... 13% 36.8M 5s 2024-07-31T14:53:02.461 INFO:tasks.workunit.client.0.smithi067.stderr: 17800K .......... .......... .......... .......... .......... 13% 31.6M 5s 2024-07-31T14:53:02.463 INFO:tasks.workunit.client.0.smithi067.stderr: 17850K .......... .......... .......... .......... .......... 13% 18.4M 5s 2024-07-31T14:53:02.464 INFO:tasks.workunit.client.0.smithi067.stderr: 17900K .......... .......... .......... .......... .......... 13% 48.3M 5s 2024-07-31T14:53:02.466 INFO:tasks.workunit.client.0.smithi067.stderr: 17950K .......... .......... .......... .......... .......... 13% 25.7M 5s 2024-07-31T14:53:02.467 INFO:tasks.workunit.client.0.smithi067.stderr: 18000K .......... .......... .......... .......... .......... 13% 45.0M 5s 2024-07-31T14:53:02.470 INFO:tasks.workunit.client.0.smithi067.stderr: 18050K .......... .......... .......... .......... .......... 13% 21.3M 5s 2024-07-31T14:53:02.472 INFO:tasks.workunit.client.0.smithi067.stderr: 18100K .......... .......... .......... .......... .......... 13% 24.3M 5s 2024-07-31T14:53:02.472 INFO:tasks.workunit.client.0.smithi067.stderr: 18150K .......... .......... .......... .......... .......... 13% 52.5M 5s 2024-07-31T14:53:02.475 INFO:tasks.workunit.client.0.smithi067.stderr: 18200K .......... .......... .......... .......... .......... 13% 24.1M 5s 2024-07-31T14:53:02.477 INFO:tasks.workunit.client.0.smithi067.stderr: 18250K .......... .......... .......... .......... .......... 13% 33.0M 5s 2024-07-31T14:53:02.479 INFO:tasks.workunit.client.0.smithi067.stderr: 18300K .......... .......... .......... .......... .......... 13% 24.3M 5s 2024-07-31T14:53:02.479 INFO:tasks.workunit.client.0.smithi067.stderr: 18350K .......... .......... .......... .......... .......... 13% 35.4M 4s 2024-07-31T14:53:02.481 INFO:tasks.workunit.client.0.smithi067.stderr: 18400K .......... .......... .......... .......... .......... 13% 37.7M 4s 2024-07-31T14:53:02.483 INFO:tasks.workunit.client.0.smithi067.stderr: 18450K .......... .......... .......... .......... .......... 13% 20.2M 4s 2024-07-31T14:53:02.484 INFO:tasks.workunit.client.0.smithi067.stderr: 18500K .......... .......... .......... .......... .......... 13% 37.2M 4s 2024-07-31T14:53:02.486 INFO:tasks.workunit.client.0.smithi067.stderr: 18550K .......... .......... .......... .......... .......... 13% 24.5M 4s 2024-07-31T14:53:02.489 INFO:tasks.workunit.client.0.smithi067.stderr: 18600K .......... .......... .......... .......... .......... 13% 24.0M 4s 2024-07-31T14:53:02.489 INFO:tasks.workunit.client.0.smithi067.stderr: 18650K .......... .......... .......... .......... .......... 13% 54.6M 4s 2024-07-31T14:53:02.492 INFO:tasks.workunit.client.0.smithi067.stderr: 18700K .......... .......... .......... .......... .......... 13% 23.4M 4s 2024-07-31T14:53:02.493 INFO:tasks.workunit.client.0.smithi067.stderr: 18750K .......... .......... .......... .......... .......... 13% 37.8M 4s 2024-07-31T14:53:02.495 INFO:tasks.workunit.client.0.smithi067.stderr: 18800K .......... .......... .......... .......... .......... 13% 22.8M 4s 2024-07-31T14:53:02.496 INFO:tasks.workunit.client.0.smithi067.stderr: 18850K .......... .......... .......... .......... .......... 13% 32.0M 4s 2024-07-31T14:53:02.498 INFO:tasks.workunit.client.0.smithi067.stderr: 18900K .......... .......... .......... .......... .......... 13% 36.8M 4s 2024-07-31T14:53:02.500 INFO:tasks.workunit.client.0.smithi067.stderr: 18950K .......... .......... .......... .......... .......... 13% 22.9M 4s 2024-07-31T14:53:02.502 INFO:tasks.workunit.client.0.smithi067.stderr: 19000K .......... .......... .......... .......... .......... 14% 40.4M 4s 2024-07-31T14:53:02.503 INFO:tasks.workunit.client.0.smithi067.stderr: 19050K .......... .......... .......... .......... .......... 14% 23.3M 4s 2024-07-31T14:53:02.505 INFO:tasks.workunit.client.0.smithi067.stderr: 19100K .......... .......... .......... .......... .......... 14% 23.7M 4s 2024-07-31T14:53:02.506 INFO:tasks.workunit.client.0.smithi067.stderr: 19150K .......... .......... .......... .......... .......... 14% 47.8M 4s 2024-07-31T14:53:02.508 INFO:tasks.workunit.client.0.smithi067.stderr: 19200K .......... .......... .......... .......... .......... 14% 24.9M 4s 2024-07-31T14:53:02.510 INFO:tasks.workunit.client.0.smithi067.stderr: 19250K .......... .......... .......... .......... .......... 14% 39.4M 4s 2024-07-31T14:53:02.512 INFO:tasks.workunit.client.0.smithi067.stderr: 19300K .......... .......... .......... .......... .......... 14% 21.9M 4s 2024-07-31T14:53:02.513 INFO:tasks.workunit.client.0.smithi067.stderr: 19350K .......... .......... .......... .......... .......... 14% 26.7M 4s 2024-07-31T14:53:02.514 INFO:tasks.workunit.client.0.smithi067.stderr: 19400K .......... .......... .......... .......... .......... 14% 50.0M 4s 2024-07-31T14:53:02.516 INFO:tasks.workunit.client.0.smithi067.stderr: 19450K .......... .......... .......... .......... .......... 14% 24.0M 4s 2024-07-31T14:53:02.518 INFO:tasks.workunit.client.0.smithi067.stderr: 19500K .......... .......... .......... .......... .......... 14% 36.8M 4s 2024-07-31T14:53:02.520 INFO:tasks.workunit.client.0.smithi067.stderr: 19550K .......... .......... .......... .......... .......... 14% 22.6M 4s 2024-07-31T14:53:02.522 INFO:tasks.workunit.client.0.smithi067.stderr: 19600K .......... .......... .......... .......... .......... 14% 24.1M 4s 2024-07-31T14:53:02.523 INFO:tasks.workunit.client.0.smithi067.stderr: 19650K .......... .......... .......... .......... .......... 14% 48.9M 4s 2024-07-31T14:53:02.525 INFO:tasks.workunit.client.0.smithi067.stderr: 19700K .......... .......... .......... .......... .......... 14% 30.3M 4s 2024-07-31T14:53:02.526 INFO:tasks.workunit.client.0.smithi067.stderr: 19750K .......... .......... .......... .......... .......... 14% 29.8M 4s 2024-07-31T14:53:02.529 INFO:tasks.workunit.client.0.smithi067.stderr: 19800K .......... .......... .......... .......... .......... 14% 24.3M 4s 2024-07-31T14:53:02.531 INFO:tasks.workunit.client.0.smithi067.stderr: 19850K .......... .......... .......... .......... .......... 14% 23.9M 4s 2024-07-31T14:53:02.531 INFO:tasks.workunit.client.0.smithi067.stderr: 19900K .......... .......... .......... .......... .......... 14% 54.2M 4s 2024-07-31T14:53:02.533 INFO:tasks.workunit.client.0.smithi067.stderr: 19950K .......... .......... .......... .......... .......... 14% 23.2M 4s 2024-07-31T14:53:02.535 INFO:tasks.workunit.client.0.smithi067.stderr: 20000K .......... .......... .......... .......... .......... 14% 37.8M 4s 2024-07-31T14:53:02.537 INFO:tasks.workunit.client.0.smithi067.stderr: 20050K .......... .......... .......... .......... .......... 14% 22.5M 4s 2024-07-31T14:53:02.539 INFO:tasks.workunit.client.0.smithi067.stderr: 20100K .......... .......... .......... .......... .......... 14% 27.7M 4s 2024-07-31T14:53:02.540 INFO:tasks.workunit.client.0.smithi067.stderr: 20150K .......... .......... .......... .......... .......... 14% 38.0M 4s 2024-07-31T14:53:02.541 INFO:tasks.workunit.client.0.smithi067.stderr: 20200K .......... .......... .......... .......... .......... 14% 35.4M 4s 2024-07-31T14:53:02.543 INFO:tasks.workunit.client.0.smithi067.stderr: 20250K .......... .......... .......... .......... .......... 14% 26.3M 4s 2024-07-31T14:53:02.546 INFO:tasks.workunit.client.0.smithi067.stderr: 20300K .......... .......... .......... .......... .......... 14% 19.4M 4s 2024-07-31T14:53:02.547 INFO:tasks.workunit.client.0.smithi067.stderr: 20350K .......... .......... .......... .......... .......... 15% 31.5M 4s 2024-07-31T14:53:02.548 INFO:tasks.workunit.client.0.smithi067.stderr: 20400K .......... .......... .......... .......... .......... 15% 53.1M 4s 2024-07-31T14:53:02.550 INFO:tasks.workunit.client.0.smithi067.stderr: 20450K .......... .......... .......... .......... .......... 15% 24.5M 4s 2024-07-31T14:53:02.551 INFO:tasks.workunit.client.0.smithi067.stderr: 20500K .......... .......... .......... .......... .......... 15% 33.7M 4s 2024-07-31T14:53:02.554 INFO:tasks.workunit.client.0.smithi067.stderr: 20550K .......... .......... .......... .......... .......... 15% 25.7M 4s 2024-07-31T14:53:02.556 INFO:tasks.workunit.client.0.smithi067.stderr: 20600K .......... .......... .......... .......... .......... 15% 23.4M 4s 2024-07-31T14:53:02.556 INFO:tasks.workunit.client.0.smithi067.stderr: 20650K .......... .......... .......... .......... .......... 15% 50.1M 4s 2024-07-31T14:53:02.559 INFO:tasks.workunit.client.0.smithi067.stderr: 20700K .......... .......... .......... .......... .......... 15% 23.5M 4s 2024-07-31T14:53:02.560 INFO:tasks.workunit.client.0.smithi067.stderr: 20750K .......... .......... .......... .......... .......... 15% 36.4M 4s 2024-07-31T14:53:02.562 INFO:tasks.workunit.client.0.smithi067.stderr: 20800K .......... .......... .......... .......... .......... 15% 23.5M 4s 2024-07-31T14:53:02.564 INFO:tasks.workunit.client.0.smithi067.stderr: 20850K .......... .......... .......... .......... .......... 15% 28.4M 4s 2024-07-31T14:53:02.565 INFO:tasks.workunit.client.0.smithi067.stderr: 20900K .......... .......... .......... .......... .......... 15% 46.6M 4s 2024-07-31T14:53:02.567 INFO:tasks.workunit.client.0.smithi067.stderr: 20950K .......... .......... .......... .......... .......... 15% 26.3M 4s 2024-07-31T14:53:02.568 INFO:tasks.workunit.client.0.smithi067.stderr: 21000K .......... .......... .......... .......... .......... 15% 31.4M 4s 2024-07-31T14:53:02.571 INFO:tasks.workunit.client.0.smithi067.stderr: 21050K .......... .......... .......... .......... .......... 15% 22.2M 4s 2024-07-31T14:53:02.573 INFO:tasks.workunit.client.0.smithi067.stderr: 21100K .......... .......... .......... .......... .......... 15% 23.9M 4s 2024-07-31T14:53:02.573 INFO:tasks.workunit.client.0.smithi067.stderr: 21150K .......... .......... .......... .......... .......... 15% 68.7M 4s 2024-07-31T14:53:02.575 INFO:tasks.workunit.client.0.smithi067.stderr: 21200K .......... .......... .......... .......... .......... 15% 22.7M 4s 2024-07-31T14:53:02.577 INFO:tasks.workunit.client.0.smithi067.stderr: 21250K .......... .......... .......... .......... .......... 15% 36.2M 4s 2024-07-31T14:53:02.579 INFO:tasks.workunit.client.0.smithi067.stderr: 21300K .......... .......... .......... .......... .......... 15% 22.8M 4s 2024-07-31T14:53:02.581 INFO:tasks.workunit.client.0.smithi067.stderr: 21350K .......... .......... .......... .......... .......... 15% 24.3M 4s 2024-07-31T14:53:02.582 INFO:tasks.workunit.client.0.smithi067.stderr: 21400K .......... .......... .......... .......... .......... 15% 53.8M 4s 2024-07-31T14:53:02.584 INFO:tasks.workunit.client.0.smithi067.stderr: 21450K .......... .......... .......... .......... .......... 15% 23.6M 4s 2024-07-31T14:53:02.585 INFO:tasks.workunit.client.0.smithi067.stderr: 21500K .......... .......... .......... .......... .......... 15% 42.2M 4s 2024-07-31T14:53:02.587 INFO:tasks.workunit.client.0.smithi067.stderr: 21550K .......... .......... .......... .......... .......... 15% 21.8M 4s 2024-07-31T14:53:02.589 INFO:tasks.workunit.client.0.smithi067.stderr: 21600K .......... .......... .......... .......... .......... 15% 25.2M 4s 2024-07-31T14:53:02.590 INFO:tasks.workunit.client.0.smithi067.stderr: 21650K .......... .......... .......... .......... .......... 15% 59.2M 4s 2024-07-31T14:53:02.593 INFO:tasks.workunit.client.0.smithi067.stderr: 21700K .......... .......... .......... .......... .......... 16% 23.8M 4s 2024-07-31T14:53:02.595 INFO:tasks.workunit.client.0.smithi067.stderr: 21750K .......... .......... .......... .......... .......... 16% 22.5M 4s 2024-07-31T14:53:02.595 INFO:tasks.workunit.client.0.smithi067.stderr: 21800K .......... .......... .......... .......... .......... 16% 50.4M 4s 2024-07-31T14:53:02.597 INFO:tasks.workunit.client.0.smithi067.stderr: 21850K .......... .......... .......... .......... .......... 16% 21.6M 4s 2024-07-31T14:53:02.598 INFO:tasks.workunit.client.0.smithi067.stderr: 21900K .......... .......... .......... .......... .......... 16% 54.4M 4s 2024-07-31T14:53:02.600 INFO:tasks.workunit.client.0.smithi067.stderr: 21950K .......... .......... .......... .......... .......... 16% 23.1M 4s 2024-07-31T14:53:02.602 INFO:tasks.workunit.client.0.smithi067.stderr: 22000K .......... .......... .......... .......... .......... 16% 34.2M 4s 2024-07-31T14:53:02.604 INFO:tasks.workunit.client.0.smithi067.stderr: 22050K .......... .......... .......... .......... .......... 16% 25.9M 4s 2024-07-31T14:53:02.606 INFO:tasks.workunit.client.0.smithi067.stderr: 22100K .......... .......... .......... .......... .......... 16% 22.3M 4s 2024-07-31T14:53:02.609 INFO:tasks.workunit.client.0.smithi067.stderr: 22150K .......... .......... .......... .......... .......... 16% 71.5M 4s 2024-07-31T14:53:02.609 INFO:tasks.workunit.client.0.smithi067.stderr: 22200K .......... .......... .......... .......... .......... 16% 26.2M 4s 2024-07-31T14:53:02.610 INFO:tasks.workunit.client.0.smithi067.stderr: 22250K .......... .......... .......... .......... .......... 16% 21.2M 4s 2024-07-31T14:53:02.612 INFO:tasks.workunit.client.0.smithi067.stderr: 22300K .......... .......... .......... .......... .......... 16% 38.0M 4s 2024-07-31T14:53:02.615 INFO:tasks.workunit.client.0.smithi067.stderr: 22350K .......... .......... .......... .......... .......... 16% 21.9M 4s 2024-07-31T14:53:02.616 INFO:tasks.workunit.client.0.smithi067.stderr: 22400K .......... .......... .......... .......... .......... 16% 62.0M 4s 2024-07-31T14:53:02.617 INFO:tasks.workunit.client.0.smithi067.stderr: 22450K .......... .......... .......... .......... .......... 16% 22.3M 4s 2024-07-31T14:53:02.619 INFO:tasks.workunit.client.0.smithi067.stderr: 22500K .......... .......... .......... .......... .......... 16% 24.0M 4s 2024-07-31T14:53:02.621 INFO:tasks.workunit.client.0.smithi067.stderr: 22550K .......... .......... .......... .......... .......... 16% 37.9M 4s 2024-07-31T14:53:02.623 INFO:tasks.workunit.client.0.smithi067.stderr: 22600K .......... .......... .......... .......... .......... 16% 23.7M 4s 2024-07-31T14:53:02.625 INFO:tasks.workunit.client.0.smithi067.stderr: 22650K .......... .......... .......... .......... .......... 16% 23.7M 4s 2024-07-31T14:53:02.625 INFO:tasks.workunit.client.0.smithi067.stderr: 22700K .......... .......... .......... .......... .......... 16% 54.1M 4s 2024-07-31T14:53:02.627 INFO:tasks.workunit.client.0.smithi067.stderr: 22750K .......... .......... .......... .......... .......... 16% 33.7M 4s 2024-07-31T14:53:02.629 INFO:tasks.workunit.client.0.smithi067.stderr: 22800K .......... .......... .......... .......... .......... 16% 19.9M 4s 2024-07-31T14:53:02.631 INFO:tasks.workunit.client.0.smithi067.stderr: 22850K .......... .......... .......... .......... .......... 16% 31.7M 4s 2024-07-31T14:53:02.632 INFO:tasks.workunit.client.0.smithi067.stderr: 22900K .......... .......... .......... .......... .......... 16% 36.7M 4s 2024-07-31T14:53:02.634 INFO:tasks.workunit.client.0.smithi067.stderr: 22950K .......... .......... .......... .......... .......... 16% 37.1M 4s 2024-07-31T14:53:02.636 INFO:tasks.workunit.client.0.smithi067.stderr: 23000K .......... .......... .......... .......... .......... 16% 23.3M 4s 2024-07-31T14:53:02.638 INFO:tasks.workunit.client.0.smithi067.stderr: 23050K .......... .......... .......... .......... .......... 17% 20.9M 4s 2024-07-31T14:53:02.640 INFO:tasks.workunit.client.0.smithi067.stderr: 23100K .......... .......... .......... .......... .......... 17% 33.2M 4s 2024-07-31T14:53:02.641 INFO:tasks.workunit.client.0.smithi067.stderr: 23150K .......... .......... .......... .......... .......... 17% 32.1M 4s 2024-07-31T14:53:02.642 INFO:tasks.workunit.client.0.smithi067.stderr: 23200K .......... .......... .......... .......... .......... 17% 34.7M 4s 2024-07-31T14:53:02.644 INFO:tasks.workunit.client.0.smithi067.stderr: 23250K .......... .......... .......... .......... .......... 17% 25.2M 4s 2024-07-31T14:53:02.646 INFO:tasks.workunit.client.0.smithi067.stderr: 23300K .......... .......... .......... .......... .......... 17% 38.2M 4s 2024-07-31T14:53:02.648 INFO:tasks.workunit.client.0.smithi067.stderr: 23350K .......... .......... .......... .......... .......... 17% 22.7M 4s 2024-07-31T14:53:02.650 INFO:tasks.workunit.client.0.smithi067.stderr: 23400K .......... .......... .......... .......... .......... 17% 23.9M 4s 2024-07-31T14:53:02.651 INFO:tasks.workunit.client.0.smithi067.stderr: 23450K .......... .......... .......... .......... .......... 17% 50.2M 4s 2024-07-31T14:53:02.653 INFO:tasks.workunit.client.0.smithi067.stderr: 23500K .......... .......... .......... .......... .......... 17% 23.8M 4s 2024-07-31T14:53:02.655 INFO:tasks.workunit.client.0.smithi067.stderr: 23550K .......... .......... .......... .......... .......... 17% 38.5M 4s 2024-07-31T14:53:02.657 INFO:tasks.workunit.client.0.smithi067.stderr: 23600K .......... .......... .......... .......... .......... 17% 24.8M 4s 2024-07-31T14:53:02.658 INFO:tasks.workunit.client.0.smithi067.stderr: 23650K .......... .......... .......... .......... .......... 17% 23.9M 4s 2024-07-31T14:53:02.659 INFO:tasks.workunit.client.0.smithi067.stderr: 23700K .......... .......... .......... .......... .......... 17% 50.3M 4s 2024-07-31T14:53:02.661 INFO:tasks.workunit.client.0.smithi067.stderr: 23750K .......... .......... .......... .......... .......... 17% 24.0M 4s 2024-07-31T14:53:02.662 INFO:tasks.workunit.client.0.smithi067.stderr: 23800K .......... .......... .......... .......... .......... 17% 40.0M 4s 2024-07-31T14:53:02.664 INFO:tasks.workunit.client.0.smithi067.stderr: 23850K .......... .......... .......... .......... .......... 17% 29.0M 4s 2024-07-31T14:53:02.667 INFO:tasks.workunit.client.0.smithi067.stderr: 23900K .......... .......... .......... .......... .......... 17% 19.5M 4s 2024-07-31T14:53:02.667 INFO:tasks.workunit.client.0.smithi067.stderr: 23950K .......... .......... .......... .......... .......... 17% 56.5M 4s 2024-07-31T14:53:02.670 INFO:tasks.workunit.client.0.smithi067.stderr: 24000K .......... .......... .......... .......... .......... 17% 23.5M 4s 2024-07-31T14:53:02.671 INFO:tasks.workunit.client.0.smithi067.stderr: 24050K .......... .......... .......... .......... .......... 17% 35.7M 4s 2024-07-31T14:53:02.674 INFO:tasks.workunit.client.0.smithi067.stderr: 24100K .......... .......... .......... .......... .......... 17% 21.0M 4s 2024-07-31T14:53:02.674 INFO:tasks.workunit.client.0.smithi067.stderr: 24150K .......... .......... .......... .......... .......... 17% 47.3M 4s 2024-07-31T14:53:02.676 INFO:tasks.workunit.client.0.smithi067.stderr: 24200K .......... .......... .......... .......... .......... 17% 34.0M 4s 2024-07-31T14:53:02.678 INFO:tasks.workunit.client.0.smithi067.stderr: 24250K .......... .......... .......... .......... .......... 17% 26.4M 4s 2024-07-31T14:53:02.680 INFO:tasks.workunit.client.0.smithi067.stderr: 24300K .......... .......... .......... .......... .......... 17% 20.7M 4s 2024-07-31T14:53:02.681 INFO:tasks.workunit.client.0.smithi067.stderr: 24350K .......... .......... .......... .......... .......... 17% 45.3M 4s 2024-07-31T14:53:02.683 INFO:tasks.workunit.client.0.smithi067.stderr: 24400K .......... .......... .......... .......... .......... 18% 20.5M 4s 2024-07-31T14:53:02.684 INFO:tasks.workunit.client.0.smithi067.stderr: 24450K .......... .......... .......... .......... .......... 18% 54.7M 4s 2024-07-31T14:53:02.686 INFO:tasks.workunit.client.0.smithi067.stderr: 24500K .......... .......... .......... .......... .......... 18% 24.9M 4s 2024-07-31T14:53:02.688 INFO:tasks.workunit.client.0.smithi067.stderr: 24550K .......... .......... .......... .......... .......... 18% 33.9M 4s 2024-07-31T14:53:02.690 INFO:tasks.workunit.client.0.smithi067.stderr: 24600K .......... .......... .......... .......... .......... 18% 25.0M 4s 2024-07-31T14:53:02.692 INFO:tasks.workunit.client.0.smithi067.stderr: 24650K .......... .......... .......... .......... .......... 18% 23.6M 4s 2024-07-31T14:53:02.692 INFO:tasks.workunit.client.0.smithi067.stderr: 24700K .......... .......... .......... .......... .......... 18% 54.5M 4s 2024-07-31T14:53:02.695 INFO:tasks.workunit.client.0.smithi067.stderr: 24750K .......... .......... .......... .......... .......... 18% 23.2M 4s 2024-07-31T14:53:02.697 INFO:tasks.workunit.client.0.smithi067.stderr: 24800K .......... .......... .......... .......... .......... 18% 36.6M 4s 2024-07-31T14:53:02.698 INFO:tasks.workunit.client.0.smithi067.stderr: 24850K .......... .......... .......... .......... .......... 18% 24.0M 4s 2024-07-31T14:53:02.700 INFO:tasks.workunit.client.0.smithi067.stderr: 24900K .......... .......... .......... .......... .......... 18% 23.4M 4s 2024-07-31T14:53:02.701 INFO:tasks.workunit.client.0.smithi067.stderr: 24950K .......... .......... .......... .......... .......... 18% 52.9M 4s 2024-07-31T14:53:02.703 INFO:tasks.workunit.client.0.smithi067.stderr: 25000K .......... .......... .......... .......... .......... 18% 23.1M 4s 2024-07-31T14:53:02.705 INFO:tasks.workunit.client.0.smithi067.stderr: 25050K .......... .......... .......... .......... .......... 18% 25.0M 4s 2024-07-31T14:53:02.707 INFO:tasks.workunit.client.0.smithi067.stderr: 25100K .......... .......... .......... .......... .......... 18% 28.5M 4s 2024-07-31T14:53:02.708 INFO:tasks.workunit.client.0.smithi067.stderr: 25150K .......... .......... .......... .......... .......... 18% 39.9M 4s 2024-07-31T14:53:02.710 INFO:tasks.workunit.client.0.smithi067.stderr: 25200K .......... .......... .......... .......... .......... 18% 32.8M 4s 2024-07-31T14:53:02.712 INFO:tasks.workunit.client.0.smithi067.stderr: 25250K .......... .......... .......... .......... .......... 18% 24.3M 4s 2024-07-31T14:53:02.714 INFO:tasks.workunit.client.0.smithi067.stderr: 25300K .......... .......... .......... .......... .......... 18% 23.8M 4s 2024-07-31T14:53:02.715 INFO:tasks.workunit.client.0.smithi067.stderr: 25350K .......... .......... .......... .......... .......... 18% 33.3M 4s 2024-07-31T14:53:02.716 INFO:tasks.workunit.client.0.smithi067.stderr: 25400K .......... .......... .......... .......... .......... 18% 48.3M 4s 2024-07-31T14:53:02.718 INFO:tasks.workunit.client.0.smithi067.stderr: 25450K .......... .......... .......... .......... .......... 18% 27.6M 4s 2024-07-31T14:53:02.720 INFO:tasks.workunit.client.0.smithi067.stderr: 25500K .......... .......... .......... .......... .......... 18% 23.3M 4s 2024-07-31T14:53:02.722 INFO:tasks.workunit.client.0.smithi067.stderr: 25550K .......... .......... .......... .......... .......... 18% 36.7M 4s 2024-07-31T14:53:02.725 INFO:tasks.workunit.client.0.smithi067.stderr: 25600K .......... .......... .......... .......... .......... 18% 21.7M 4s 2024-07-31T14:53:02.727 INFO:tasks.workunit.client.0.smithi067.stderr: 25650K .......... .......... .......... .......... .......... 18% 27.3M 4s 2024-07-31T14:53:02.727 INFO:tasks.workunit.client.0.smithi067.stderr: 25700K .......... .......... .......... .......... .......... 18% 41.4M 4s 2024-07-31T14:53:02.728 INFO:tasks.workunit.client.0.smithi067.stderr: 25750K .......... .......... .......... .......... .......... 19% 24.4M 4s 2024-07-31T14:53:02.730 INFO:tasks.workunit.client.0.smithi067.stderr: 25800K .......... .......... .......... .......... .......... 19% 37.8M 4s 2024-07-31T14:53:02.732 INFO:tasks.workunit.client.0.smithi067.stderr: 25850K .......... .......... .......... .......... .......... 19% 23.6M 4s 2024-07-31T14:53:02.734 INFO:tasks.workunit.client.0.smithi067.stderr: 25900K .......... .......... .......... .......... .......... 19% 23.9M 4s 2024-07-31T14:53:02.735 INFO:tasks.workunit.client.0.smithi067.stderr: 25950K .......... .......... .......... .......... .......... 19% 54.3M 4s 2024-07-31T14:53:02.737 INFO:tasks.workunit.client.0.smithi067.stderr: 26000K .......... .......... .......... .......... .......... 19% 23.9M 4s 2024-07-31T14:53:02.738 INFO:tasks.workunit.client.0.smithi067.stderr: 26050K .......... .......... .......... .......... .......... 19% 37.0M 4s 2024-07-31T14:53:02.740 INFO:tasks.workunit.client.0.smithi067.stderr: 26100K .......... .......... .......... .......... .......... 19% 23.3M 4s 2024-07-31T14:53:02.742 INFO:tasks.workunit.client.0.smithi067.stderr: 26150K .......... .......... .......... .......... .......... 19% 24.0M 4s 2024-07-31T14:53:02.743 INFO:tasks.workunit.client.0.smithi067.stderr: 26200K .......... .......... .......... .......... .......... 19% 46.6M 4s 2024-07-31T14:53:02.745 INFO:tasks.workunit.client.0.smithi067.stderr: 26250K .......... .......... .......... .......... .......... 19% 25.5M 4s 2024-07-31T14:53:02.747 INFO:tasks.workunit.client.0.smithi067.stderr: 26300K .......... .......... .......... .......... .......... 19% 37.1M 4s 2024-07-31T14:53:02.749 INFO:tasks.workunit.client.0.smithi067.stderr: 26350K .......... .......... .......... .......... .......... 19% 23.2M 4s 2024-07-31T14:53:02.751 INFO:tasks.workunit.client.0.smithi067.stderr: 26400K .......... .......... .......... .......... .......... 19% 24.1M 4s 2024-07-31T14:53:02.752 INFO:tasks.workunit.client.0.smithi067.stderr: 26450K .......... .......... .......... .......... .......... 19% 55.0M 4s 2024-07-31T14:53:02.754 INFO:tasks.workunit.client.0.smithi067.stderr: 26500K .......... .......... .......... .......... .......... 19% 23.8M 4s 2024-07-31T14:53:02.755 INFO:tasks.workunit.client.0.smithi067.stderr: 26550K .......... .......... .......... .......... .......... 19% 37.5M 4s 2024-07-31T14:53:02.757 INFO:tasks.workunit.client.0.smithi067.stderr: 26600K .......... .......... .......... .......... .......... 19% 24.2M 4s 2024-07-31T14:53:02.759 INFO:tasks.workunit.client.0.smithi067.stderr: 26650K .......... .......... .......... .......... .......... 19% 23.3M 4s 2024-07-31T14:53:02.760 INFO:tasks.workunit.client.0.smithi067.stderr: 26700K .......... .......... .......... .......... .......... 19% 55.2M 4s 2024-07-31T14:53:02.762 INFO:tasks.workunit.client.0.smithi067.stderr: 26750K .......... .......... .......... .......... .......... 19% 23.7M 4s 2024-07-31T14:53:02.763 INFO:tasks.workunit.client.0.smithi067.stderr: 26800K .......... .......... .......... .......... .......... 19% 36.4M 4s 2024-07-31T14:53:02.766 INFO:tasks.workunit.client.0.smithi067.stderr: 26850K .......... .......... .......... .......... .......... 19% 24.0M 4s 2024-07-31T14:53:02.768 INFO:tasks.workunit.client.0.smithi067.stderr: 26900K .......... .......... .......... .......... .......... 19% 23.9M 4s 2024-07-31T14:53:02.768 INFO:tasks.workunit.client.0.smithi067.stderr: 26950K .......... .......... .......... .......... .......... 19% 53.4M 4s 2024-07-31T14:53:02.770 INFO:tasks.workunit.client.0.smithi067.stderr: 27000K .......... .......... .......... .......... .......... 19% 24.2M 4s 2024-07-31T14:53:02.772 INFO:tasks.workunit.client.0.smithi067.stderr: 27050K .......... .......... .......... .......... .......... 19% 35.7M 4s 2024-07-31T14:53:02.774 INFO:tasks.workunit.client.0.smithi067.stderr: 27100K .......... .......... .......... .......... .......... 19% 24.1M 4s 2024-07-31T14:53:02.777 INFO:tasks.workunit.client.0.smithi067.stderr: 27150K .......... .......... .......... .......... .......... 20% 23.9M 4s 2024-07-31T14:53:02.777 INFO:tasks.workunit.client.0.smithi067.stderr: 27200K .......... .......... .......... .......... .......... 20% 51.2M 4s 2024-07-31T14:53:02.779 INFO:tasks.workunit.client.0.smithi067.stderr: 27250K .......... .......... .......... .......... .......... 20% 23.8M 4s 2024-07-31T14:53:02.781 INFO:tasks.workunit.client.0.smithi067.stderr: 27300K .......... .......... .......... .......... .......... 20% 34.3M 4s 2024-07-31T14:53:02.783 INFO:tasks.workunit.client.0.smithi067.stderr: 27350K .......... .......... .......... .......... .......... 20% 24.7M 4s 2024-07-31T14:53:02.785 INFO:tasks.workunit.client.0.smithi067.stderr: 27400K .......... .......... .......... .......... .......... 20% 23.8M 4s 2024-07-31T14:53:02.788 INFO:tasks.workunit.client.0.smithi067.stderr: 27450K .......... .......... .......... .......... .......... 20% 53.0M 4s 2024-07-31T14:53:02.788 INFO:tasks.workunit.client.0.smithi067.stderr: 27500K .......... .......... .......... .......... .......... 20% 24.1M 4s 2024-07-31T14:53:02.792 INFO:tasks.workunit.client.0.smithi067.stderr: 27550K .......... .......... .......... .......... .......... 20% 24.8M 4s 2024-07-31T14:53:02.792 INFO:tasks.workunit.client.0.smithi067.stderr: 27600K .......... .......... .......... .......... .......... 20% 32.1M 4s 2024-07-31T14:53:02.794 INFO:tasks.workunit.client.0.smithi067.stderr: 27650K .......... .......... .......... .......... .......... 20% 23.6M 4s 2024-07-31T14:53:02.796 INFO:tasks.workunit.client.0.smithi067.stderr: 27700K .......... .......... .......... .......... .......... 20% 60.8M 4s 2024-07-31T14:53:02.796 INFO:tasks.workunit.client.0.smithi067.stderr: 27750K .......... .......... .......... .......... .......... 20% 23.4M 4s 2024-07-31T14:53:02.798 INFO:tasks.workunit.client.0.smithi067.stderr: 27800K .......... .......... .......... .......... .......... 20% 23.4M 4s 2024-07-31T14:53:02.801 INFO:tasks.workunit.client.0.smithi067.stderr: 27850K .......... .......... .......... .......... .......... 20% 37.3M 4s 2024-07-31T14:53:02.801 INFO:tasks.workunit.client.0.smithi067.stderr: 27900K .......... .......... .......... .......... .......... 20% 24.9M 4s 2024-07-31T14:53:02.803 INFO:tasks.workunit.client.0.smithi067.stderr: 27950K .......... .......... .......... .......... .......... 20% 34.7M 4s 2024-07-31T14:53:02.804 INFO:tasks.workunit.client.0.smithi067.stderr: 28000K .......... .......... .......... .......... .......... 20% 35.0M 4s 2024-07-31T14:53:02.807 INFO:tasks.workunit.client.0.smithi067.stderr: 28050K .......... .......... .......... .......... .......... 20% 27.4M 4s 2024-07-31T14:53:02.809 INFO:tasks.workunit.client.0.smithi067.stderr: 28100K .......... .......... .......... .......... .......... 20% 21.6M 4s 2024-07-31T14:53:02.809 INFO:tasks.workunit.client.0.smithi067.stderr: 28150K .......... .......... .......... .......... .......... 20% 55.4M 4s 2024-07-31T14:53:02.811 INFO:tasks.workunit.client.0.smithi067.stderr: 28200K .......... .......... .......... .......... .......... 20% 23.1M 4s 2024-07-31T14:53:02.813 INFO:tasks.workunit.client.0.smithi067.stderr: 28250K .......... .......... .......... .......... .......... 20% 34.0M 4s 2024-07-31T14:53:02.815 INFO:tasks.workunit.client.0.smithi067.stderr: 28300K .......... .......... .......... .......... .......... 20% 23.5M 4s 2024-07-31T14:53:02.816 INFO:tasks.workunit.client.0.smithi067.stderr: 28350K .......... .......... .......... .......... .......... 20% 35.7M 4s 2024-07-31T14:53:02.817 INFO:tasks.workunit.client.0.smithi067.stderr: 28400K .......... .......... .......... .......... .......... 20% 26.9M 4s 2024-07-31T14:53:02.821 INFO:tasks.workunit.client.0.smithi067.stderr: 28450K .......... .......... .......... .......... .......... 20% 21.0M 4s 2024-07-31T14:53:02.821 INFO:tasks.workunit.client.0.smithi067.stderr: 28500K .......... .......... .......... .......... .......... 21% 56.7M 4s 2024-07-31T14:53:02.823 INFO:tasks.workunit.client.0.smithi067.stderr: 28550K .......... .......... .......... .......... .......... 21% 22.4M 4s 2024-07-31T14:53:02.824 INFO:tasks.workunit.client.0.smithi067.stderr: 28600K .......... .......... .......... .......... .......... 21% 41.1M 4s 2024-07-31T14:53:02.826 INFO:tasks.workunit.client.0.smithi067.stderr: 28650K .......... .......... .......... .......... .......... 21% 22.5M 4s 2024-07-31T14:53:02.829 INFO:tasks.workunit.client.0.smithi067.stderr: 28700K .......... .......... .......... .......... .......... 21% 24.5M 4s 2024-07-31T14:53:02.829 INFO:tasks.workunit.client.0.smithi067.stderr: 28750K .......... .......... .......... .......... .......... 21% 57.8M 4s 2024-07-31T14:53:02.830 INFO:tasks.workunit.client.0.smithi067.stderr: 28800K .......... .......... .......... .......... .......... 21% 33.4M 4s 2024-07-31T14:53:02.833 INFO:tasks.workunit.client.0.smithi067.stderr: 28850K .......... .......... .......... .......... .......... 21% 25.0M 4s 2024-07-31T14:53:02.834 INFO:tasks.workunit.client.0.smithi067.stderr: 28900K .......... .......... .......... .......... .......... 21% 28.2M 4s 2024-07-31T14:53:02.837 INFO:tasks.workunit.client.0.smithi067.stderr: 28950K .......... .......... .......... .......... .......... 21% 21.2M 4s 2024-07-31T14:53:02.837 INFO:tasks.workunit.client.0.smithi067.stderr: 29000K .......... .......... .......... .......... .......... 21% 56.1M 4s 2024-07-31T14:53:02.839 INFO:tasks.workunit.client.0.smithi067.stderr: 29050K .......... .......... .......... .......... .......... 21% 22.7M 4s 2024-07-31T14:53:02.841 INFO:tasks.workunit.client.0.smithi067.stderr: 29100K .......... .......... .......... .......... .......... 21% 35.6M 4s 2024-07-31T14:53:02.843 INFO:tasks.workunit.client.0.smithi067.stderr: 29150K .......... .......... .......... .......... .......... 21% 24.3M 4s 2024-07-31T14:53:02.845 INFO:tasks.workunit.client.0.smithi067.stderr: 29200K .......... .......... .......... .......... .......... 21% 22.5M 4s 2024-07-31T14:53:02.846 INFO:tasks.workunit.client.0.smithi067.stderr: 29250K .......... .......... .......... .......... .......... 21% 64.7M 4s 2024-07-31T14:53:02.848 INFO:tasks.workunit.client.0.smithi067.stderr: 29300K .......... .......... .......... .......... .......... 21% 30.8M 4s 2024-07-31T14:53:02.850 INFO:tasks.workunit.client.0.smithi067.stderr: 29350K .......... .......... .......... .......... .......... 21% 22.3M 4s 2024-07-31T14:53:02.852 INFO:tasks.workunit.client.0.smithi067.stderr: 29400K .......... .......... .......... .......... .......... 21% 22.6M 4s 2024-07-31T14:53:02.853 INFO:tasks.workunit.client.0.smithi067.stderr: 29450K .......... .......... .......... .......... .......... 21% 35.5M 4s 2024-07-31T14:53:02.855 INFO:tasks.workunit.client.0.smithi067.stderr: 29500K .......... .......... .......... .......... .......... 21% 38.0M 4s 2024-07-31T14:53:02.856 INFO:tasks.workunit.client.0.smithi067.stderr: 29550K .......... .......... .......... .......... .......... 21% 25.1M 4s 2024-07-31T14:53:02.858 INFO:tasks.workunit.client.0.smithi067.stderr: 29600K .......... .......... .......... .......... .......... 21% 26.0M 4s 2024-07-31T14:53:02.859 INFO:tasks.workunit.client.0.smithi067.stderr: 29650K .......... .......... .......... .......... .......... 21% 39.0M 4s 2024-07-31T14:53:02.862 INFO:tasks.workunit.client.0.smithi067.stderr: 29700K .......... .......... .......... .......... .......... 21% 21.5M 4s 2024-07-31T14:53:02.863 INFO:tasks.workunit.client.0.smithi067.stderr: 29750K .......... .......... .......... .......... .......... 21% 52.4M 4s 2024-07-31T14:53:02.865 INFO:tasks.workunit.client.0.smithi067.stderr: 29800K .......... .......... .......... .......... .......... 21% 24.5M 4s 2024-07-31T14:53:02.866 INFO:tasks.workunit.client.0.smithi067.stderr: 29850K .......... .......... .......... .......... .......... 22% 36.2M 4s 2024-07-31T14:53:02.868 INFO:tasks.workunit.client.0.smithi067.stderr: 29900K .......... .......... .......... .......... .......... 22% 23.3M 4s 2024-07-31T14:53:02.870 INFO:tasks.workunit.client.0.smithi067.stderr: 29950K .......... .......... .......... .......... .......... 22% 24.9M 4s 2024-07-31T14:53:02.871 INFO:tasks.workunit.client.0.smithi067.stderr: 30000K .......... .......... .......... .......... .......... 22% 54.1M 4s 2024-07-31T14:53:02.873 INFO:tasks.workunit.client.0.smithi067.stderr: 30050K .......... .......... .......... .......... .......... 22% 29.8M 4s 2024-07-31T14:53:02.875 INFO:tasks.workunit.client.0.smithi067.stderr: 30100K .......... .......... .......... .......... .......... 22% 26.2M 4s 2024-07-31T14:53:02.877 INFO:tasks.workunit.client.0.smithi067.stderr: 30150K .......... .......... .......... .......... .......... 22% 23.3M 4s 2024-07-31T14:53:02.879 INFO:tasks.workunit.client.0.smithi067.stderr: 30200K .......... .......... .......... .......... .......... 22% 25.2M 4s 2024-07-31T14:53:02.879 INFO:tasks.workunit.client.0.smithi067.stderr: 30250K .......... .......... .......... .......... .......... 22% 58.2M 4s 2024-07-31T14:53:02.882 INFO:tasks.workunit.client.0.smithi067.stderr: 30300K .......... .......... .......... .......... .......... 22% 23.2M 4s 2024-07-31T14:53:02.883 INFO:tasks.workunit.client.0.smithi067.stderr: 30350K .......... .......... .......... .......... .......... 22% 31.1M 4s 2024-07-31T14:53:02.885 INFO:tasks.workunit.client.0.smithi067.stderr: 30400K .......... .......... .......... .......... .......... 22% 26.1M 4s 2024-07-31T14:53:02.886 INFO:tasks.workunit.client.0.smithi067.stderr: 30450K .......... .......... .......... .......... .......... 22% 36.6M 4s 2024-07-31T14:53:02.888 INFO:tasks.workunit.client.0.smithi067.stderr: 30500K .......... .......... .......... .......... .......... 22% 32.3M 4s 2024-07-31T14:53:02.890 INFO:tasks.workunit.client.0.smithi067.stderr: 30550K .......... .......... .......... .......... .......... 22% 24.3M 4s 2024-07-31T14:53:02.891 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:02 smithi067 ceph-mon[32750]: pgmap v458: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-07-31T14:53:02.892 INFO:tasks.workunit.client.0.smithi067.stderr: 30600K .......... .......... .......... .......... .......... 22% 22.4M 4s 2024-07-31T14:53:02.893 INFO:tasks.workunit.client.0.smithi067.stderr: 30650K .......... .......... .......... .......... .......... 22% 45.4M 4s 2024-07-31T14:53:02.895 INFO:tasks.workunit.client.0.smithi067.stderr: 30700K .......... .......... .......... .......... .......... 22% 35.7M 4s 2024-07-31T14:53:02.896 INFO:tasks.workunit.client.0.smithi067.stderr: 30750K .......... .......... .......... .......... .......... 22% 27.9M 4s 2024-07-31T14:53:02.898 INFO:tasks.workunit.client.0.smithi067.stderr: 30800K .......... .......... .......... .......... .......... 22% 23.6M 4s 2024-07-31T14:53:02.900 INFO:tasks.workunit.client.0.smithi067.stderr: 30850K .......... .......... .......... .......... .......... 22% 24.1M 4s 2024-07-31T14:53:02.902 INFO:tasks.workunit.client.0.smithi067.stderr: 30900K .......... .......... .......... .......... .......... 22% 32.1M 4s 2024-07-31T14:53:02.904 INFO:tasks.workunit.client.0.smithi067.stderr: 30950K .......... .......... .......... .......... .......... 22% 28.5M 4s 2024-07-31T14:53:02.904 INFO:tasks.workunit.client.0.smithi067.stderr: 31000K .......... .......... .......... .......... .......... 22% 47.0M 4s 2024-07-31T14:53:02.907 INFO:tasks.workunit.client.0.smithi067.stderr: 31050K .......... .......... .......... .......... .......... 22% 23.0M 4s 2024-07-31T14:53:02.908 INFO:tasks.workunit.client.0.smithi067.stderr: 31100K .......... .......... .......... .......... .......... 22% 37.8M 4s 2024-07-31T14:53:02.993 INFO:tasks.workunit.client.0.smithi067.stderr: 31150K .......... .......... .......... .......... .......... 22% 598K 4s 2024-07-31T14:53:02.993 INFO:tasks.workunit.client.0.smithi067.stderr: 31200K .......... .......... .......... .......... .......... 23% 469M 4s 2024-07-31T14:53:03.000 INFO:tasks.workunit.client.0.smithi067.stderr: 31250K .......... .......... .......... .......... .......... 23% 17.6M 4s 2024-07-31T14:53:03.000 INFO:tasks.workunit.client.0.smithi067.stderr: 31300K .......... .......... .......... .......... .......... 23% 560M 4s 2024-07-31T14:53:03.001 INFO:tasks.workunit.client.0.smithi067.stderr: 31350K .......... .......... .......... .......... .......... 23% 496M 4s 2024-07-31T14:53:03.001 INFO:tasks.workunit.client.0.smithi067.stderr: 31400K .......... .......... .......... .......... .......... 23% 584M 4s 2024-07-31T14:53:03.001 INFO:tasks.workunit.client.0.smithi067.stderr: 31450K .......... .......... .......... .......... .......... 23% 492M 4s 2024-07-31T14:53:03.001 INFO:tasks.workunit.client.0.smithi067.stderr: 31500K .......... .......... .......... .......... .......... 23% 513M 4s 2024-07-31T14:53:03.001 INFO:tasks.workunit.client.0.smithi067.stderr: 31550K .......... .......... .......... .......... .......... 23% 458M 4s 2024-07-31T14:53:03.001 INFO:tasks.workunit.client.0.smithi067.stderr: 31600K .......... .......... .......... .......... .......... 23% 584M 4s 2024-07-31T14:53:03.001 INFO:tasks.workunit.client.0.smithi067.stderr: 31650K .......... .......... .......... .......... .......... 23% 581M 4s 2024-07-31T14:53:03.054 INFO:tasks.workunit.client.0.smithi067.stderr: 31700K .......... .......... .......... .......... .......... 23% 4.65M 4s 2024-07-31T14:53:03.054 INFO:tasks.workunit.client.0.smithi067.stderr: 31750K .......... .......... .......... .......... .......... 23% 376M 4s 2024-07-31T14:53:03.055 INFO:tasks.workunit.client.0.smithi067.stderr: 31800K .......... .......... .......... .......... .......... 23% 478M 4s 2024-07-31T14:53:03.055 INFO:tasks.workunit.client.0.smithi067.stderr: 31850K .......... .......... .......... .......... .......... 23% 582M 4s 2024-07-31T14:53:03.055 INFO:tasks.workunit.client.0.smithi067.stderr: 31900K .......... .......... .......... .......... .......... 23% 601M 4s 2024-07-31T14:53:03.055 INFO:tasks.workunit.client.0.smithi067.stderr: 31950K .......... .......... .......... .......... .......... 23% 589M 4s 2024-07-31T14:53:03.055 INFO:tasks.workunit.client.0.smithi067.stderr: 32000K .......... .......... .......... .......... .......... 23% 444M 4s 2024-07-31T14:53:03.055 INFO:tasks.workunit.client.0.smithi067.stderr: 32050K .......... .......... .......... .......... .......... 23% 575M 4s 2024-07-31T14:53:03.055 INFO:tasks.workunit.client.0.smithi067.stderr: 32100K .......... .......... .......... .......... .......... 23% 562M 4s 2024-07-31T14:53:03.055 INFO:tasks.workunit.client.0.smithi067.stderr: 32150K .......... .......... .......... .......... .......... 23% 575M 4s 2024-07-31T14:53:03.055 INFO:tasks.workunit.client.0.smithi067.stderr: 32200K .......... .......... .......... .......... .......... 23% 505M 4s 2024-07-31T14:53:03.055 INFO:tasks.workunit.client.0.smithi067.stderr: 32250K .......... .......... .......... .......... .......... 23% 582M 4s 2024-07-31T14:53:03.055 INFO:tasks.workunit.client.0.smithi067.stderr: 32300K .......... .......... .......... .......... .......... 23% 549M 4s 2024-07-31T14:53:03.055 INFO:tasks.workunit.client.0.smithi067.stderr: 32350K .......... .......... .......... .......... .......... 23% 550M 4s 2024-07-31T14:53:03.055 INFO:tasks.workunit.client.0.smithi067.stderr: 32400K .......... .......... .......... .......... .......... 23% 507M 4s 2024-07-31T14:53:03.055 INFO:tasks.workunit.client.0.smithi067.stderr: 32450K .......... .......... .......... .......... .......... 23% 582M 4s 2024-07-31T14:53:03.056 INFO:tasks.workunit.client.0.smithi067.stderr: 32500K .......... .......... .......... .......... .......... 23% 569M 4s 2024-07-31T14:53:03.056 INFO:tasks.workunit.client.0.smithi067.stderr: 32550K .......... .......... .......... .......... .......... 24% 582M 4s 2024-07-31T14:53:03.056 INFO:tasks.workunit.client.0.smithi067.stderr: 32600K .......... .......... .......... .......... .......... 24% 502M 4s 2024-07-31T14:53:03.056 INFO:tasks.workunit.client.0.smithi067.stderr: 32650K .......... .......... .......... .......... .......... 24% 595M 4s 2024-07-31T14:53:03.056 INFO:tasks.workunit.client.0.smithi067.stderr: 32700K .......... .......... .......... .......... .......... 24% 6.53M 4s 2024-07-31T14:53:03.056 INFO:tasks.workunit.client.0.smithi067.stderr: 32750K .......... .......... .......... .......... .......... 24% 544M 4s 2024-07-31T14:53:03.056 INFO:tasks.workunit.client.0.smithi067.stderr: 32800K .......... .......... .......... .......... .......... 24% 595M 4s 2024-07-31T14:53:03.056 INFO:tasks.workunit.client.0.smithi067.stderr: 32850K .......... .......... .......... .......... .......... 24% 506M 4s 2024-07-31T14:53:03.056 INFO:tasks.workunit.client.0.smithi067.stderr: 32900K .......... .......... .......... .......... .......... 24% 594M 4s 2024-07-31T14:53:03.056 INFO:tasks.workunit.client.0.smithi067.stderr: 32950K .......... .......... .......... .......... .......... 24% 584M 4s 2024-07-31T14:53:03.056 INFO:tasks.workunit.client.0.smithi067.stderr: 33000K .......... .......... .......... .......... .......... 24% 572M 4s 2024-07-31T14:53:03.056 INFO:tasks.workunit.client.0.smithi067.stderr: 33050K .......... .......... .......... .......... .......... 24% 488M 4s 2024-07-31T14:53:03.056 INFO:tasks.workunit.client.0.smithi067.stderr: 33100K .......... .......... .......... .......... .......... 24% 594M 4s 2024-07-31T14:53:03.056 INFO:tasks.workunit.client.0.smithi067.stderr: 33150K .......... .......... .......... .......... .......... 24% 576M 4s 2024-07-31T14:53:03.057 INFO:tasks.workunit.client.0.smithi067.stderr: 33200K .......... .......... .......... .......... .......... 24% 578M 4s 2024-07-31T14:53:03.057 INFO:tasks.workunit.client.0.smithi067.stderr: 33250K .......... .......... .......... .......... .......... 24% 160M 4s 2024-07-31T14:53:03.057 INFO:tasks.workunit.client.0.smithi067.stderr: 33300K .......... .......... .......... .......... .......... 24% 466M 4s 2024-07-31T14:53:03.057 INFO:tasks.workunit.client.0.smithi067.stderr: 33350K .......... .......... .......... .......... .......... 24% 576M 4s 2024-07-31T14:53:03.057 INFO:tasks.workunit.client.0.smithi067.stderr: 33400K .......... .......... .......... .......... .......... 24% 590M 4s 2024-07-31T14:53:03.057 INFO:tasks.workunit.client.0.smithi067.stderr: 33450K .......... .......... .......... .......... .......... 24% 572M 4s 2024-07-31T14:53:03.057 INFO:tasks.workunit.client.0.smithi067.stderr: 33500K .......... .......... .......... .......... .......... 24% 496M 4s 2024-07-31T14:53:03.057 INFO:tasks.workunit.client.0.smithi067.stderr: 33550K .......... .......... .......... .......... .......... 24% 575M 4s 2024-07-31T14:53:03.057 INFO:tasks.workunit.client.0.smithi067.stderr: 33600K .......... .......... .......... .......... .......... 24% 583M 4s 2024-07-31T14:53:03.057 INFO:tasks.workunit.client.0.smithi067.stderr: 33650K .......... .......... .......... .......... .......... 24% 581M 4s 2024-07-31T14:53:03.057 INFO:tasks.workunit.client.0.smithi067.stderr: 33700K .......... .......... .......... .......... .......... 24% 413M 4s 2024-07-31T14:53:03.057 INFO:tasks.workunit.client.0.smithi067.stderr: 33750K .......... .......... .......... .......... .......... 24% 576M 4s 2024-07-31T14:53:03.058 INFO:tasks.workunit.client.0.smithi067.stderr: 33800K .......... .......... .......... .......... .......... 24% 556M 4s 2024-07-31T14:53:03.058 INFO:tasks.workunit.client.0.smithi067.stderr: 33850K .......... .......... .......... .......... .......... 24% 575M 4s 2024-07-31T14:53:03.058 INFO:tasks.workunit.client.0.smithi067.stderr: 33900K .......... .......... .......... .......... .......... 25% 511M 4s 2024-07-31T14:53:03.058 INFO:tasks.workunit.client.0.smithi067.stderr: 33950K .......... .......... .......... .......... .......... 25% 499M 4s 2024-07-31T14:53:03.058 INFO:tasks.workunit.client.0.smithi067.stderr: 34000K .......... .......... .......... .......... .......... 25% 581M 4s 2024-07-31T14:53:03.058 INFO:tasks.workunit.client.0.smithi067.stderr: 34050K .......... .......... .......... .......... .......... 25% 556M 4s 2024-07-31T14:53:03.058 INFO:tasks.workunit.client.0.smithi067.stderr: 34100K .......... .......... .......... .......... .......... 25% 509M 4s 2024-07-31T14:53:03.058 INFO:tasks.workunit.client.0.smithi067.stderr: 34150K .......... .......... .......... .......... .......... 25% 597M 4s 2024-07-31T14:53:03.058 INFO:tasks.workunit.client.0.smithi067.stderr: 34200K .......... .......... .......... .......... .......... 25% 602M 4s 2024-07-31T14:53:03.058 INFO:tasks.workunit.client.0.smithi067.stderr: 34250K .......... .......... .......... .......... .......... 25% 553M 4s 2024-07-31T14:53:03.058 INFO:tasks.workunit.client.0.smithi067.stderr: 34300K .......... .......... .......... .......... .......... 25% 503M 4s 2024-07-31T14:53:03.058 INFO:tasks.workunit.client.0.smithi067.stderr: 34350K .......... .......... .......... .......... .......... 25% 624M 4s 2024-07-31T14:53:03.059 INFO:tasks.workunit.client.0.smithi067.stderr: 34400K .......... .......... .......... .......... .......... 25% 36.0M 4s 2024-07-31T14:53:03.059 INFO:tasks.workunit.client.0.smithi067.stderr: 34450K .......... .......... .......... .......... .......... 25% 19.5M 4s 2024-07-31T14:53:03.059 INFO:tasks.workunit.client.0.smithi067.stderr: 34500K .......... .......... .......... .......... .......... 25% 61.5M 4s 2024-07-31T14:53:03.059 INFO:tasks.workunit.client.0.smithi067.stderr: 34550K .......... .......... .......... .......... .......... 25% 22.7M 4s 2024-07-31T14:53:03.059 INFO:tasks.workunit.client.0.smithi067.stderr: 34600K .......... .......... .......... .......... .......... 25% 35.1M 4s 2024-07-31T14:53:03.059 INFO:tasks.workunit.client.0.smithi067.stderr: 34650K .......... .......... .......... .......... .......... 25% 25.4M 4s 2024-07-31T14:53:03.059 INFO:tasks.workunit.client.0.smithi067.stderr: 34700K .......... .......... .......... .......... .......... 25% 23.6M 4s 2024-07-31T14:53:03.059 INFO:tasks.workunit.client.0.smithi067.stderr: 34750K .......... .......... .......... .......... .......... 25% 54.3M 4s 2024-07-31T14:53:03.059 INFO:tasks.workunit.client.0.smithi067.stderr: 34800K .......... .......... .......... .......... .......... 25% 24.4M 4s 2024-07-31T14:53:03.059 INFO:tasks.workunit.client.0.smithi067.stderr: 34850K .......... .......... .......... .......... .......... 25% 33.6M 4s 2024-07-31T14:53:03.059 INFO:tasks.workunit.client.0.smithi067.stderr: 34900K .......... .......... .......... .......... .......... 25% 23.1M 4s 2024-07-31T14:53:03.060 INFO:tasks.workunit.client.0.smithi067.stderr: 34950K .......... .......... .......... .......... .......... 25% 25.3M 4s 2024-07-31T14:53:03.060 INFO:tasks.workunit.client.0.smithi067.stderr: 35000K .......... .......... .......... .......... .......... 25% 59.8M 4s 2024-07-31T14:53:03.060 INFO:tasks.workunit.client.0.smithi067.stderr: 35050K .......... .......... .......... .......... .......... 25% 28.8M 4s 2024-07-31T14:53:03.060 INFO:tasks.workunit.client.0.smithi067.stderr: 35100K .......... .......... .......... .......... .......... 25% 28.7M 4s 2024-07-31T14:53:03.060 INFO:tasks.workunit.client.0.smithi067.stderr: 35150K .......... .......... .......... .......... .......... 25% 24.0M 4s 2024-07-31T14:53:03.060 INFO:tasks.workunit.client.0.smithi067.stderr: 35200K .......... .......... .......... .......... .......... 25% 24.4M 4s 2024-07-31T14:53:03.060 INFO:tasks.workunit.client.0.smithi067.stderr: 35250K .......... .......... .......... .......... .......... 26% 47.8M 4s 2024-07-31T14:53:03.060 INFO:tasks.workunit.client.0.smithi067.stderr: 35300K .......... .......... .......... .......... .......... 26% 24.1M 4s 2024-07-31T14:53:03.060 INFO:tasks.workunit.client.0.smithi067.stderr: 35350K .......... .......... .......... .......... .......... 26% 38.0M 4s 2024-07-31T14:53:03.060 INFO:tasks.workunit.client.0.smithi067.stderr: 35400K .......... .......... .......... .......... .......... 26% 23.6M 4s 2024-07-31T14:53:03.065 INFO:tasks.workunit.client.0.smithi067.stderr: 35450K .......... .......... .......... .......... .......... 26% 24.5M 4s 2024-07-31T14:53:03.065 INFO:tasks.workunit.client.0.smithi067.stderr: 35500K .......... .......... .......... .......... .......... 26% 48.7M 4s 2024-07-31T14:53:03.065 INFO:tasks.workunit.client.0.smithi067.stderr: 35550K .......... .......... .......... .......... .......... 26% 23.9M 4s 2024-07-31T14:53:03.065 INFO:tasks.workunit.client.0.smithi067.stderr: 35600K .......... .......... .......... .......... .......... 26% 35.2M 4s 2024-07-31T14:53:03.065 INFO:tasks.workunit.client.0.smithi067.stderr: 35650K .......... .......... .......... .......... .......... 26% 21.3M 4s 2024-07-31T14:53:03.066 INFO:tasks.workunit.client.0.smithi067.stderr: 35700K .......... .......... .......... .......... .......... 26% 22.3M 4s 2024-07-31T14:53:03.066 INFO:tasks.workunit.client.0.smithi067.stderr: 35750K .......... .......... .......... .......... .......... 26% 83.5M 4s 2024-07-31T14:53:03.067 INFO:tasks.workunit.client.0.smithi067.stderr: 35800K .......... .......... .......... .......... .......... 26% 24.4M 4s 2024-07-31T14:53:03.070 INFO:tasks.workunit.client.0.smithi067.stderr: 35850K .......... .......... .......... .......... .......... 26% 24.9M 4s 2024-07-31T14:53:03.071 INFO:tasks.workunit.client.0.smithi067.stderr: 35900K .......... .......... .......... .......... .......... 26% 22.3M 4s 2024-07-31T14:53:03.073 INFO:tasks.workunit.client.0.smithi067.stderr: 35950K .......... .......... .......... .......... .......... 26% 26.8M 4s 2024-07-31T14:53:03.074 INFO:tasks.workunit.client.0.smithi067.stderr: 36000K .......... .......... .......... .......... .......... 26% 147M 4s 2024-07-31T14:53:03.076 INFO:tasks.workunit.client.0.smithi067.stderr: 36050K .......... .......... .......... .......... .......... 26% 21.6M 4s 2024-07-31T14:53:03.077 INFO:tasks.workunit.client.0.smithi067.stderr: 36100K .......... .......... .......... .......... .......... 26% 26.4M 4s 2024-07-31T14:53:03.079 INFO:tasks.workunit.client.0.smithi067.stderr: 36150K .......... .......... .......... .......... .......... 26% 29.6M 4s 2024-07-31T14:53:03.081 INFO:tasks.workunit.client.0.smithi067.stderr: 36200K .......... .......... .......... .......... .......... 26% 24.9M 4s 2024-07-31T14:53:03.082 INFO:tasks.workunit.client.0.smithi067.stderr: 36250K .......... .......... .......... .......... .......... 26% 63.9M 4s 2024-07-31T14:53:03.083 INFO:tasks.workunit.client.0.smithi067.stderr: 36300K .......... .......... .......... .......... .......... 26% 24.2M 4s 2024-07-31T14:53:03.085 INFO:tasks.workunit.client.0.smithi067.stderr: 36350K .......... .......... .......... .......... .......... 26% 33.3M 4s 2024-07-31T14:53:03.088 INFO:tasks.workunit.client.0.smithi067.stderr: 36400K .......... .......... .......... .......... .......... 26% 25.0M 4s 2024-07-31T14:53:03.088 INFO:tasks.workunit.client.0.smithi067.stderr: 36450K .......... .......... .......... .......... .......... 26% 41.8M 4s 2024-07-31T14:53:03.090 INFO:tasks.workunit.client.0.smithi067.stderr: 36500K .......... .......... .......... .......... .......... 26% 29.5M 4s 2024-07-31T14:53:03.092 INFO:tasks.workunit.client.0.smithi067.stderr: 36550K .......... .......... .......... .......... .......... 26% 24.6M 4s 2024-07-31T14:53:03.094 INFO:tasks.workunit.client.0.smithi067.stderr: 36600K .......... .......... .......... .......... .......... 26% 29.7M 4s 2024-07-31T14:53:03.095 INFO:tasks.workunit.client.0.smithi067.stderr: 36650K .......... .......... .......... .......... .......... 27% 29.1M 4s 2024-07-31T14:53:03.098 INFO:tasks.workunit.client.0.smithi067.stderr: 36700K .......... .......... .......... .......... .......... 27% 20.2M 4s 2024-07-31T14:53:03.098 INFO:tasks.workunit.client.0.smithi067.stderr: 36750K .......... .......... .......... .......... .......... 27% 82.3M 4s 2024-07-31T14:53:03.100 INFO:tasks.workunit.client.0.smithi067.stderr: 36800K .......... .......... .......... .......... .......... 27% 27.7M 4s 2024-07-31T14:53:03.102 INFO:tasks.workunit.client.0.smithi067.stderr: 36850K .......... .......... .......... .......... .......... 27% 20.4M 4s 2024-07-31T14:53:03.103 INFO:tasks.workunit.client.0.smithi067.stderr: 36900K .......... .......... .......... .......... .......... 27% 47.8M 4s 2024-07-31T14:53:03.105 INFO:tasks.workunit.client.0.smithi067.stderr: 36950K .......... .......... .......... .......... .......... 27% 29.3M 4s 2024-07-31T14:53:03.107 INFO:tasks.workunit.client.0.smithi067.stderr: 37000K .......... .......... .......... .......... .......... 27% 31.3M 4s 2024-07-31T14:53:03.109 INFO:tasks.workunit.client.0.smithi067.stderr: 37050K .......... .......... .......... .......... .......... 27% 23.4M 4s 2024-07-31T14:53:03.111 INFO:tasks.workunit.client.0.smithi067.stderr: 37100K .......... .......... .......... .......... .......... 27% 27.7M 4s 2024-07-31T14:53:03.113 INFO:tasks.workunit.client.0.smithi067.stderr: 37150K .......... .......... .......... .......... .......... 27% 20.6M 4s 2024-07-31T14:53:03.114 INFO:tasks.workunit.client.0.smithi067.stderr: 37200K .......... .......... .......... .......... .......... 27% 39.4M 4s 2024-07-31T14:53:03.116 INFO:tasks.workunit.client.0.smithi067.stderr: 37250K .......... .......... .......... .......... .......... 27% 47.3M 4s 2024-07-31T14:53:03.117 INFO:tasks.workunit.client.0.smithi067.stderr: 37300K .......... .......... .......... .......... .......... 27% 23.4M 4s 2024-07-31T14:53:03.119 INFO:tasks.workunit.client.0.smithi067.stderr: 37350K .......... .......... .......... .......... .......... 27% 33.4M 4s 2024-07-31T14:53:03.121 INFO:tasks.workunit.client.0.smithi067.stderr: 37400K .......... .......... .......... .......... .......... 27% 23.4M 4s 2024-07-31T14:53:03.122 INFO:tasks.workunit.client.0.smithi067.stderr: 37450K .......... .......... .......... .......... .......... 27% 36.2M 4s 2024-07-31T14:53:03.124 INFO:tasks.workunit.client.0.smithi067.stderr: 37500K .......... .......... .......... .......... .......... 27% 35.6M 4s 2024-07-31T14:53:03.126 INFO:tasks.workunit.client.0.smithi067.stderr: 37550K .......... .......... .......... .......... .......... 27% 25.2M 4s 2024-07-31T14:53:03.128 INFO:tasks.workunit.client.0.smithi067.stderr: 37600K .......... .......... .......... .......... .......... 27% 20.8M 4s 2024-07-31T14:53:03.129 INFO:tasks.workunit.client.0.smithi067.stderr: 37650K .......... .......... .......... .......... .......... 27% 36.9M 4s 2024-07-31T14:53:03.130 INFO:tasks.workunit.client.0.smithi067.stderr: 37700K .......... .......... .......... .......... .......... 27% 35.3M 4s 2024-07-31T14:53:03.133 INFO:tasks.workunit.client.0.smithi067.stderr: 37750K .......... .......... .......... .......... .......... 27% 18.5M 4s 2024-07-31T14:53:03.134 INFO:tasks.workunit.client.0.smithi067.stderr: 37800K .......... .......... .......... .......... .......... 27% 50.6M 4s 2024-07-31T14:53:03.136 INFO:tasks.workunit.client.0.smithi067.stderr: 37850K .......... .......... .......... .......... .......... 27% 23.8M 4s 2024-07-31T14:53:03.138 INFO:tasks.workunit.client.0.smithi067.stderr: 37900K .......... .......... .......... .......... .......... 27% 25.0M 4s 2024-07-31T14:53:03.140 INFO:tasks.workunit.client.0.smithi067.stderr: 37950K .......... .......... .......... .......... .......... 27% 34.6M 4s 2024-07-31T14:53:03.141 INFO:tasks.workunit.client.0.smithi067.stderr: 38000K .......... .......... .......... .......... .......... 28% 26.8M 4s 2024-07-31T14:53:03.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:02 smithi073 ceph-mon[36714]: pgmap v458: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-07-31T14:53:03.143 INFO:tasks.workunit.client.0.smithi067.stderr: 38050K .......... .......... .......... .......... .......... 28% 38.1M 4s 2024-07-31T14:53:03.144 INFO:tasks.workunit.client.0.smithi067.stderr: 38100K .......... .......... .......... .......... .......... 28% 25.9M 4s 2024-07-31T14:53:03.147 INFO:tasks.workunit.client.0.smithi067.stderr: 38150K .......... .......... .......... .......... .......... 28% 25.0M 4s 2024-07-31T14:53:03.148 INFO:tasks.workunit.client.0.smithi067.stderr: 38200K .......... .......... .......... .......... .......... 28% 32.3M 4s 2024-07-31T14:53:03.153 INFO:tasks.workunit.client.0.smithi067.stderr: 38250K .......... .......... .......... .......... .......... 28% 13.7M 4s 2024-07-31T14:53:03.153 INFO:tasks.workunit.client.0.smithi067.stderr: 38300K .......... .......... .......... .......... .......... 28% 161M 3s 2024-07-31T14:53:03.153 INFO:tasks.workunit.client.0.smithi067.stderr: 38350K .......... .......... .......... .......... .......... 28% 45.9M 3s 2024-07-31T14:53:03.154 INFO:tasks.workunit.client.0.smithi067.stderr: 38400K .......... .......... .......... .......... .......... 28% 42.1M 3s 2024-07-31T14:53:03.156 INFO:tasks.workunit.client.0.smithi067.stderr: 38450K .......... .......... .......... .......... .......... 28% 24.6M 3s 2024-07-31T14:53:03.158 INFO:tasks.workunit.client.0.smithi067.stderr: 38500K .......... .......... .......... .......... .......... 28% 26.4M 3s 2024-07-31T14:53:03.159 INFO:tasks.workunit.client.0.smithi067.stderr: 38550K .......... .......... .......... .......... .......... 28% 35.2M 3s 2024-07-31T14:53:03.161 INFO:tasks.workunit.client.0.smithi067.stderr: 38600K .......... .......... .......... .......... .......... 28% 24.4M 3s 2024-07-31T14:53:03.162 INFO:tasks.workunit.client.0.smithi067.stderr: 38650K .......... .......... .......... .......... .......... 28% 38.3M 3s 2024-07-31T14:53:03.165 INFO:tasks.workunit.client.0.smithi067.stderr: 38700K .......... .......... .......... .......... .......... 28% 22.7M 3s 2024-07-31T14:53:03.167 INFO:tasks.workunit.client.0.smithi067.stderr: 38750K .......... .......... .......... .......... .......... 28% 24.1M 3s 2024-07-31T14:53:03.168 INFO:tasks.workunit.client.0.smithi067.stderr: 38800K .......... .......... .......... .......... .......... 28% 55.4M 3s 2024-07-31T14:53:03.169 INFO:tasks.workunit.client.0.smithi067.stderr: 38850K .......... .......... .......... .......... .......... 28% 25.5M 3s 2024-07-31T14:53:03.171 INFO:tasks.workunit.client.0.smithi067.stderr: 38900K .......... .......... .......... .......... .......... 28% 30.7M 3s 2024-07-31T14:53:03.173 INFO:tasks.workunit.client.0.smithi067.stderr: 38950K .......... .......... .......... .......... .......... 28% 25.4M 3s 2024-07-31T14:53:03.175 INFO:tasks.workunit.client.0.smithi067.stderr: 39000K .......... .......... .......... .......... .......... 28% 22.9M 3s 2024-07-31T14:53:03.176 INFO:tasks.workunit.client.0.smithi067.stderr: 39050K .......... .......... .......... .......... .......... 28% 57.6M 3s 2024-07-31T14:53:03.178 INFO:tasks.workunit.client.0.smithi067.stderr: 39100K .......... .......... .......... .......... .......... 28% 23.1M 3s 2024-07-31T14:53:03.179 INFO:tasks.workunit.client.0.smithi067.stderr: 39150K .......... .......... .......... .......... .......... 28% 51.6M 3s 2024-07-31T14:53:03.182 INFO:tasks.workunit.client.0.smithi067.stderr: 39200K .......... .......... .......... .......... .......... 28% 20.1M 3s 2024-07-31T14:53:03.184 INFO:tasks.workunit.client.0.smithi067.stderr: 39250K .......... .......... .......... .......... .......... 28% 26.3M 3s 2024-07-31T14:53:03.184 INFO:tasks.workunit.client.0.smithi067.stderr: 39300K .......... .......... .......... .......... .......... 28% 51.8M 3s 2024-07-31T14:53:03.186 INFO:tasks.workunit.client.0.smithi067.stderr: 39350K .......... .......... .......... .......... .......... 29% 23.3M 3s 2024-07-31T14:53:03.188 INFO:tasks.workunit.client.0.smithi067.stderr: 39400K .......... .......... .......... .......... .......... 29% 38.7M 3s 2024-07-31T14:53:03.191 INFO:tasks.workunit.client.0.smithi067.stderr: 39450K .......... .......... .......... .......... .......... 29% 21.2M 3s 2024-07-31T14:53:03.192 INFO:tasks.workunit.client.0.smithi067.stderr: 39500K .......... .......... .......... .......... .......... 29% 27.2M 3s 2024-07-31T14:53:03.193 INFO:tasks.workunit.client.0.smithi067.stderr: 39550K .......... .......... .......... .......... .......... 29% 50.6M 3s 2024-07-31T14:53:03.195 INFO:tasks.workunit.client.0.smithi067.stderr: 39600K .......... .......... .......... .......... .......... 29% 22.2M 3s 2024-07-31T14:53:03.196 INFO:tasks.workunit.client.0.smithi067.stderr: 39650K .......... .......... .......... .......... .......... 29% 42.3M 3s 2024-07-31T14:53:03.198 INFO:tasks.workunit.client.0.smithi067.stderr: 39700K .......... .......... .......... .......... .......... 29% 23.7M 3s 2024-07-31T14:53:03.200 INFO:tasks.workunit.client.0.smithi067.stderr: 39750K .......... .......... .......... .......... .......... 29% 23.5M 3s 2024-07-31T14:53:03.201 INFO:tasks.workunit.client.0.smithi067.stderr: 39800K .......... .......... .......... .......... .......... 29% 55.4M 3s 2024-07-31T14:53:03.203 INFO:tasks.workunit.client.0.smithi067.stderr: 39850K .......... .......... .......... .......... .......... 29% 23.4M 3s 2024-07-31T14:53:03.204 INFO:tasks.workunit.client.0.smithi067.stderr: 39900K .......... .......... .......... .......... .......... 29% 38.5M 3s 2024-07-31T14:53:03.207 INFO:tasks.workunit.client.0.smithi067.stderr: 39950K .......... .......... .......... .......... .......... 29% 23.4M 3s 2024-07-31T14:53:03.209 INFO:tasks.workunit.client.0.smithi067.stderr: 40000K .......... .......... .......... .......... .......... 29% 22.9M 3s 2024-07-31T14:53:03.209 INFO:tasks.workunit.client.0.smithi067.stderr: 40050K .......... .......... .......... .......... .......... 29% 56.4M 3s 2024-07-31T14:53:03.211 INFO:tasks.workunit.client.0.smithi067.stderr: 40100K .......... .......... .......... .......... .......... 29% 23.6M 3s 2024-07-31T14:53:03.213 INFO:tasks.workunit.client.0.smithi067.stderr: 40150K .......... .......... .......... .......... .......... 29% 37.8M 3s 2024-07-31T14:53:03.215 INFO:tasks.workunit.client.0.smithi067.stderr: 40200K .......... .......... .......... .......... .......... 29% 24.2M 3s 2024-07-31T14:53:03.217 INFO:tasks.workunit.client.0.smithi067.stderr: 40250K .......... .......... .......... .......... .......... 29% 24.4M 3s 2024-07-31T14:53:03.218 INFO:tasks.workunit.client.0.smithi067.stderr: 40300K .......... .......... .......... .......... .......... 29% 50.5M 3s 2024-07-31T14:53:03.220 INFO:tasks.workunit.client.0.smithi067.stderr: 40350K .......... .......... .......... .......... .......... 29% 23.5M 3s 2024-07-31T14:53:03.222 INFO:tasks.workunit.client.0.smithi067.stderr: 40400K .......... .......... .......... .......... .......... 29% 34.3M 3s 2024-07-31T14:53:03.222 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9b7c4d67', total seconds elapsed: 43.1 2024-07-31T14:53:03.224 INFO:tasks.workunit.client.0.smithi067.stderr: 40450K .......... .......... .......... .......... .......... 29% 20.0M 3s 2024-07-31T14:53:03.225 INFO:tasks.workunit.client.0.smithi067.stderr: 40500K .......... .......... .......... .......... .......... 29% 32.5M 3s 2024-07-31T14:53:03.226 INFO:tasks.workunit.client.0.smithi067.stderr: 40550K .......... .......... .......... .......... .......... 29% 49.9M 3s 2024-07-31T14:53:03.228 INFO:tasks.workunit.client.0.smithi067.stderr: 40600K .......... .......... .......... .......... .......... 29% 32.5M 3s 2024-07-31T14:53:03.230 INFO:tasks.workunit.client.0.smithi067.stderr: 40650K .......... .......... .......... .......... .......... 29% 25.4M 3s 2024-07-31T14:53:03.232 INFO:tasks.workunit.client.0.smithi067.stderr: 40700K .......... .......... .......... .......... .......... 30% 23.1M 3s 2024-07-31T14:53:03.234 INFO:tasks.workunit.client.0.smithi067.stderr: 40750K .......... .......... .......... .......... .......... 30% 25.1M 3s 2024-07-31T14:53:03.235 INFO:tasks.workunit.client.0.smithi067.stderr: 40800K .......... .......... .......... .......... .......... 30% 54.8M 3s 2024-07-31T14:53:03.237 INFO:tasks.workunit.client.0.smithi067.stderr: 40850K .......... .......... .......... .......... .......... 30% 23.6M 3s 2024-07-31T14:53:03.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:02 smithi077 ceph-mon[37327]: pgmap v458: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-07-31T14:53:03.238 INFO:tasks.workunit.client.0.smithi067.stderr: 40900K .......... .......... .......... .......... .......... 30% 39.3M 3s 2024-07-31T14:53:03.241 INFO:tasks.workunit.client.0.smithi067.stderr: 40950K .......... .......... .......... .......... .......... 30% 18.1M 3s 2024-07-31T14:53:03.242 INFO:tasks.workunit.client.0.smithi067.stderr: 41000K .......... .......... .......... .......... .......... 30% 34.5M 3s 2024-07-31T14:53:03.243 INFO:tasks.workunit.client.0.smithi067.stderr: 41050K .......... .......... .......... .......... .......... 30% 47.5M 3s 2024-07-31T14:53:03.245 INFO:tasks.workunit.client.0.smithi067.stderr: 41100K .......... .......... .......... .......... .......... 30% 28.5M 3s 2024-07-31T14:53:03.247 INFO:tasks.workunit.client.0.smithi067.stderr: 41150K .......... .......... .......... .......... .......... 30% 28.8M 3s 2024-07-31T14:53:03.249 INFO:tasks.workunit.client.0.smithi067.stderr: 41200K .......... .......... .......... .......... .......... 30% 25.6M 3s 2024-07-31T14:53:03.251 INFO:tasks.workunit.client.0.smithi067.stderr: 41250K .......... .......... .......... .......... .......... 30% 23.6M 3s 2024-07-31T14:53:03.251 INFO:tasks.workunit.client.0.smithi067.stderr: 41300K .......... .......... .......... .......... .......... 30% 53.4M 3s 2024-07-31T14:53:03.254 INFO:tasks.workunit.client.0.smithi067.stderr: 41350K .......... .......... .......... .......... .......... 30% 23.9M 3s 2024-07-31T14:53:03.255 INFO:tasks.workunit.client.0.smithi067.stderr: 41400K .......... .......... .......... .......... .......... 30% 38.5M 3s 2024-07-31T14:53:03.257 INFO:tasks.workunit.client.0.smithi067.stderr: 41450K .......... .......... .......... .......... .......... 30% 23.8M 3s 2024-07-31T14:53:03.260 INFO:tasks.workunit.client.0.smithi067.stderr: 41500K .......... .......... .......... .......... .......... 30% 22.5M 3s 2024-07-31T14:53:03.260 INFO:tasks.workunit.client.0.smithi067.stderr: 41550K .......... .......... .......... .......... .......... 30% 60.2M 3s 2024-07-31T14:53:03.262 INFO:tasks.workunit.client.0.smithi067.stderr: 41600K .......... .......... .......... .......... .......... 30% 23.6M 3s 2024-07-31T14:53:03.264 INFO:tasks.workunit.client.0.smithi067.stderr: 41650K .......... .......... .......... .......... .......... 30% 35.2M 3s 2024-07-31T14:53:03.265 INFO:tasks.workunit.client.0.smithi067.stderr: 41700K .......... .......... .......... .......... .......... 30% 23.6M 3s 2024-07-31T14:53:03.267 INFO:tasks.workunit.client.0.smithi067.stderr: 41750K .......... .......... .......... .......... .......... 30% 23.6M 3s 2024-07-31T14:53:03.268 INFO:tasks.workunit.client.0.smithi067.stderr: 41800K .......... .......... .......... .......... .......... 30% 53.9M 3s 2024-07-31T14:53:03.270 INFO:tasks.workunit.client.0.smithi067.stderr: 41850K .......... .......... .......... .......... .......... 30% 24.0M 3s 2024-07-31T14:53:03.272 INFO:tasks.workunit.client.0.smithi067.stderr: 41900K .......... .......... .......... .......... .......... 30% 34.9M 3s 2024-07-31T14:53:03.274 INFO:tasks.workunit.client.0.smithi067.stderr: 41950K .......... .......... .......... .......... .......... 30% 24.3M 3s 2024-07-31T14:53:03.276 INFO:tasks.workunit.client.0.smithi067.stderr: 42000K .......... .......... .......... .......... .......... 30% 25.6M 3s 2024-07-31T14:53:03.276 INFO:tasks.workunit.client.0.smithi067.stderr: 42050K .......... .......... .......... .......... .......... 31% 50.6M 3s 2024-07-31T14:53:03.279 INFO:tasks.workunit.client.0.smithi067.stderr: 42100K .......... .......... .......... .......... .......... 31% 23.0M 3s 2024-07-31T14:53:03.281 INFO:tasks.workunit.client.0.smithi067.stderr: 42150K .......... .......... .......... .......... .......... 31% 22.1M 3s 2024-07-31T14:53:03.282 INFO:tasks.workunit.client.0.smithi067.stderr: 42200K .......... .......... .......... .......... .......... 31% 37.8M 3s 2024-07-31T14:53:03.284 INFO:tasks.workunit.client.0.smithi067.stderr: 42250K .......... .......... .......... .......... .......... 31% 24.7M 3s 2024-07-31T14:53:03.285 INFO:tasks.workunit.client.0.smithi067.stderr: 42300K .......... .......... .......... .......... .......... 31% 48.6M 3s 2024-07-31T14:53:03.287 INFO:tasks.workunit.client.0.smithi067.stderr: 42350K .......... .......... .......... .......... .......... 31% 22.3M 3s 2024-07-31T14:53:03.289 INFO:tasks.workunit.client.0.smithi067.stderr: 42400K .......... .......... .......... .......... .......... 31% 26.6M 3s 2024-07-31T14:53:03.291 INFO:tasks.workunit.client.0.smithi067.stderr: 42450K .......... .......... .......... .......... .......... 31% 34.2M 3s 2024-07-31T14:53:03.293 INFO:tasks.workunit.client.0.smithi067.stderr: 42500K .......... .......... .......... .......... .......... 31% 24.1M 3s 2024-07-31T14:53:03.295 INFO:tasks.workunit.client.0.smithi067.stderr: 42550K .......... .......... .......... .......... .......... 31% 56.4M 3s 2024-07-31T14:53:03.295 INFO:tasks.workunit.client.0.smithi067.stderr: 42600K .......... .......... .......... .......... .......... 31% 23.0M 3s 2024-07-31T14:53:03.298 INFO:tasks.workunit.client.0.smithi067.stderr: 42650K .......... .......... .......... .......... .......... 31% 24.3M 3s 2024-07-31T14:53:03.299 INFO:tasks.workunit.client.0.smithi067.stderr: 42700K .......... .......... .......... .......... .......... 31% 37.4M 3s 2024-07-31T14:53:03.301 INFO:tasks.workunit.client.0.smithi067.stderr: 42750K .......... .......... .......... .......... .......... 31% 24.1M 3s 2024-07-31T14:53:03.303 INFO:tasks.workunit.client.0.smithi067.stderr: 42800K .......... .......... .......... .......... .......... 31% 56.4M 3s 2024-07-31T14:53:03.304 INFO:tasks.workunit.client.0.smithi067.stderr: 42850K .......... .......... .......... .......... .......... 31% 23.6M 3s 2024-07-31T14:53:03.306 INFO:tasks.workunit.client.0.smithi067.stderr: 42900K .......... .......... .......... .......... .......... 31% 23.9M 3s 2024-07-31T14:53:03.307 INFO:tasks.workunit.client.0.smithi067.stderr: 42950K .......... .......... .......... .......... .......... 31% 37.5M 3s 2024-07-31T14:53:03.310 INFO:tasks.workunit.client.0.smithi067.stderr: 43000K .......... .......... .......... .......... .......... 31% 21.3M 3s 2024-07-31T14:53:03.310 INFO:tasks.workunit.client.0.smithi067.stderr: 43050K .......... .......... .......... .......... .......... 31% 58.7M 3s 2024-07-31T14:53:03.313 INFO:tasks.workunit.client.0.smithi067.stderr: 43100K .......... .......... .......... .......... .......... 31% 23.6M 3s 2024-07-31T14:53:03.314 INFO:tasks.workunit.client.0.smithi067.stderr: 43150K .......... .......... .......... .......... .......... 31% 29.5M 3s 2024-07-31T14:53:03.315 INFO:tasks.workunit.client.0.smithi067.stderr: 43200K .......... .......... .......... .......... .......... 31% 34.2M 3s 2024-07-31T14:53:03.318 INFO:tasks.workunit.client.0.smithi067.stderr: 43250K .......... .......... .......... .......... .......... 31% 21.9M 3s 2024-07-31T14:53:03.320 INFO:tasks.workunit.client.0.smithi067.stderr: 43300K .......... .......... .......... .......... .......... 31% 25.5M 3s 2024-07-31T14:53:03.321 INFO:tasks.workunit.client.0.smithi067.stderr: 43350K .......... .......... .......... .......... .......... 31% 42.5M 3s 2024-07-31T14:53:03.323 INFO:tasks.workunit.client.0.smithi067.stderr: 43400K .......... .......... .......... .......... .......... 32% 28.0M 3s 2024-07-31T14:53:03.324 INFO:tasks.workunit.client.0.smithi067.stderr: 43450K .......... .......... .......... .......... .......... 32% 31.7M 3s 2024-07-31T14:53:03.327 INFO:tasks.workunit.client.0.smithi067.stderr: 43500K .......... .......... .......... .......... .......... 32% 20.6M 3s 2024-07-31T14:53:03.327 INFO:tasks.workunit.client.0.smithi067.stderr: 43550K .......... .......... .......... .......... .......... 32% 51.7M 3s 2024-07-31T14:53:03.329 INFO:tasks.workunit.client.0.smithi067.stderr: 43600K .......... .......... .......... .......... .......... 32% 28.7M 3s 2024-07-31T14:53:03.331 INFO:tasks.workunit.client.0.smithi067.stderr: 43650K .......... .......... .......... .......... .......... 32% 26.5M 3s 2024-07-31T14:53:03.332 INFO:tasks.workunit.client.0.smithi067.stderr: 43700K .......... .......... .......... .......... .......... 32% 33.1M 3s 2024-07-31T14:53:03.335 INFO:tasks.workunit.client.0.smithi067.stderr: 43750K .......... .......... .......... .......... .......... 32% 20.5M 3s 2024-07-31T14:53:03.337 INFO:tasks.workunit.client.0.smithi067.stderr: 43800K .......... .......... .......... .......... .......... 32% 27.3M 3s 2024-07-31T14:53:03.337 INFO:tasks.workunit.client.0.smithi067.stderr: 43850K .......... .......... .......... .......... .......... 32% 49.7M 3s 2024-07-31T14:53:03.343 INFO:tasks.workunit.client.0.smithi067.stderr: 43900K .......... .......... .......... .......... .......... 32% 24.6M 3s 2024-07-31T14:53:03.343 INFO:tasks.workunit.client.0.smithi067.stderr: 43950K .......... .......... .......... .......... .......... 32% 27.4M 3s 2024-07-31T14:53:03.343 INFO:tasks.workunit.client.0.smithi067.stderr: 44000K .......... .......... .......... .......... .......... 32% 32.3M 3s 2024-07-31T14:53:03.345 INFO:tasks.workunit.client.0.smithi067.stderr: 44050K .......... .......... .......... .......... .......... 32% 23.6M 3s 2024-07-31T14:53:03.346 INFO:tasks.workunit.client.0.smithi067.stderr: 44100K .......... .......... .......... .......... .......... 32% 51.6M 3s 2024-07-31T14:53:03.348 INFO:tasks.workunit.client.0.smithi067.stderr: 44150K .......... .......... .......... .......... .......... 32% 22.9M 3s 2024-07-31T14:53:03.350 INFO:tasks.workunit.client.0.smithi067.stderr: 44200K .......... .......... .......... .......... .......... 32% 25.5M 3s 2024-07-31T14:53:03.351 INFO:tasks.workunit.client.0.smithi067.stderr: 44250K .......... .......... .......... .......... .......... 32% 36.3M 3s 2024-07-31T14:53:03.353 INFO:tasks.workunit.client.0.smithi067.stderr: 44300K .......... .......... .......... .......... .......... 32% 29.2M 3s 2024-07-31T14:53:03.354 INFO:tasks.workunit.client.0.smithi067.stderr: 44350K .......... .......... .......... .......... .......... 32% 38.5M 3s 2024-07-31T14:53:03.356 INFO:tasks.workunit.client.0.smithi067.stderr: 44400K .......... .......... .......... .......... .......... 32% 22.5M 3s 2024-07-31T14:53:03.358 INFO:tasks.workunit.client.0.smithi067.stderr: 44450K .......... .......... .......... .......... .......... 32% 35.4M 3s 2024-07-31T14:53:03.360 INFO:tasks.workunit.client.0.smithi067.stderr: 44500K .......... .......... .......... .......... .......... 32% 23.7M 3s 2024-07-31T14:53:03.362 INFO:tasks.workunit.client.0.smithi067.stderr: 44550K .......... .......... .......... .......... .......... 32% 23.7M 3s 2024-07-31T14:53:03.363 INFO:tasks.workunit.client.0.smithi067.stderr: 44600K .......... .......... .......... .......... .......... 32% 49.5M 3s 2024-07-31T14:53:03.365 INFO:tasks.workunit.client.0.smithi067.stderr: 44650K .......... .......... .......... .......... .......... 32% 24.5M 3s 2024-07-31T14:53:03.366 INFO:tasks.workunit.client.0.smithi067.stderr: 44700K .......... .......... .......... .......... .......... 32% 38.8M 3s 2024-07-31T14:53:03.368 INFO:tasks.workunit.client.0.smithi067.stderr: 44750K .......... .......... .......... .......... .......... 33% 23.8M 3s 2024-07-31T14:53:03.370 INFO:tasks.workunit.client.0.smithi067.stderr: 44800K .......... .......... .......... .......... .......... 33% 23.2M 3s 2024-07-31T14:53:03.371 INFO:tasks.workunit.client.0.smithi067.stderr: 44850K .......... .......... .......... .......... .......... 33% 49.1M 3s 2024-07-31T14:53:03.373 INFO:tasks.workunit.client.0.smithi067.stderr: 44900K .......... .......... .......... .......... .......... 33% 24.2M 3s 2024-07-31T14:53:03.375 INFO:tasks.workunit.client.0.smithi067.stderr: 44950K .......... .......... .......... .......... .......... 33% 39.7M 3s 2024-07-31T14:53:03.376 INFO:tasks.workunit.client.0.smithi067.stderr: 45000K .......... .......... .......... .......... .......... 33% 31.6M 3s 2024-07-31T14:53:03.378 INFO:tasks.workunit.client.0.smithi067.stderr: 45050K .......... .......... .......... .......... .......... 33% 23.4M 3s 2024-07-31T14:53:03.379 INFO:tasks.workunit.client.0.smithi067.stderr: 45100K .......... .......... .......... .......... .......... 33% 37.6M 3s 2024-07-31T14:53:03.382 INFO:tasks.workunit.client.0.smithi067.stderr: 45150K .......... .......... .......... .......... .......... 33% 22.3M 3s 2024-07-31T14:53:03.384 INFO:tasks.workunit.client.0.smithi067.stderr: 45200K .......... .......... .......... .......... .......... 33% 24.3M 3s 2024-07-31T14:53:03.385 INFO:tasks.workunit.client.0.smithi067.stderr: 45250K .......... .......... .......... .......... .......... 33% 38.3M 3s 2024-07-31T14:53:03.387 INFO:tasks.workunit.client.0.smithi067.stderr: 45300K .......... .......... .......... .......... .......... 33% 23.9M 3s 2024-07-31T14:53:03.388 INFO:tasks.workunit.client.0.smithi067.stderr: 45350K .......... .......... .......... .......... .......... 33% 47.0M 3s 2024-07-31T14:53:03.390 INFO:tasks.workunit.client.0.smithi067.stderr: 45400K .......... .......... .......... .......... .......... 33% 23.1M 3s 2024-07-31T14:53:03.392 INFO:tasks.workunit.client.0.smithi067.stderr: 45450K .......... .......... .......... .......... .......... 33% 38.7M 3s 2024-07-31T14:53:03.394 INFO:tasks.workunit.client.0.smithi067.stderr: 45500K .......... .......... .......... .......... .......... 33% 23.0M 3s 2024-07-31T14:53:03.396 INFO:tasks.workunit.client.0.smithi067.stderr: 45550K .......... .......... .......... .......... .......... 33% 23.0M 3s 2024-07-31T14:53:03.396 INFO:tasks.workunit.client.0.smithi067.stderr: 45600K .......... .......... .......... .......... .......... 33% 56.3M 3s 2024-07-31T14:53:03.398 INFO:tasks.workunit.client.0.smithi067.stderr: 45650K .......... .......... .......... .......... .......... 33% 23.3M 3s 2024-07-31T14:53:03.400 INFO:tasks.workunit.client.0.smithi067.stderr: 45700K .......... .......... .......... .......... .......... 33% 40.3M 3s 2024-07-31T14:53:03.402 INFO:tasks.workunit.client.0.smithi067.stderr: 45750K .......... .......... .......... .......... .......... 33% 25.6M 3s 2024-07-31T14:53:03.403 INFO:tasks.workunit.client.0.smithi067.stderr: 45800K .......... .......... .......... .......... .......... 33% 25.4M 3s 2024-07-31T14:53:03.405 INFO:tasks.workunit.client.0.smithi067.stderr: 45850K .......... .......... .......... .......... .......... 33% 38.6M 3s 2024-07-31T14:53:03.407 INFO:tasks.workunit.client.0.smithi067.stderr: 45900K .......... .......... .......... .......... .......... 33% 23.7M 3s 2024-07-31T14:53:03.408 INFO:tasks.workunit.client.0.smithi067.stderr: 45950K .......... .......... .......... .......... .......... 33% 37.6M 3s 2024-07-31T14:53:03.410 INFO:tasks.workunit.client.0.smithi067.stderr: 46000K .......... .......... .......... .......... .......... 33% 25.1M 3s 2024-07-31T14:53:03.412 INFO:tasks.workunit.client.0.smithi067.stderr: 46050K .......... .......... .......... .......... .......... 33% 22.4M 3s 2024-07-31T14:53:03.413 INFO:tasks.workunit.client.0.smithi067.stderr: 46100K .......... .......... .......... .......... .......... 33% 51.6M 3s 2024-07-31T14:53:03.415 INFO:tasks.workunit.client.0.smithi067.stderr: 46150K .......... .......... .......... .......... .......... 34% 23.6M 3s 2024-07-31T14:53:03.417 INFO:tasks.workunit.client.0.smithi067.stderr: 46200K .......... .......... .......... .......... .......... 34% 39.2M 3s 2024-07-31T14:53:03.419 INFO:tasks.workunit.client.0.smithi067.stderr: 46250K .......... .......... .......... .......... .......... 34% 23.9M 3s 2024-07-31T14:53:03.421 INFO:tasks.workunit.client.0.smithi067.stderr: 46300K .......... .......... .......... .......... .......... 34% 23.4M 3s 2024-07-31T14:53:03.422 INFO:tasks.workunit.client.0.smithi067.stderr: 46350K .......... .......... .......... .......... .......... 34% 53.8M 3s 2024-07-31T14:53:03.424 INFO:tasks.workunit.client.0.smithi067.stderr: 46400K .......... .......... .......... .......... .......... 34% 22.5M 3s 2024-07-31T14:53:03.425 INFO:tasks.workunit.client.0.smithi067.stderr: 46450K .......... .......... .......... .......... .......... 34% 36.3M 3s 2024-07-31T14:53:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.424+0000 7fbeef3a5640 1 -- 172.21.15.67:0/3478557329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee8102c30 msgr2=0x7fbee8103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.424+0000 7fbeef3a5640 1 --2- 172.21.15.67:0/3478557329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee8102c30 0x7fbee8103040 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009900 tx=0x7fbee402a410 comp rx=0 tx=0).stop 2024-07-31T14:53:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.424+0000 7fbeef3a5640 1 -- 172.21.15.67:0/3478557329 shutdown_connections 2024-07-31T14:53:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.424+0000 7fbeef3a5640 1 --2- 172.21.15.67:0/3478557329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbee8108720 0x7fbee810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.424+0000 7fbeef3a5640 1 --2- 172.21.15.67:0/3478557329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbee8107dd0 0x7fbee81081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.424+0000 7fbeef3a5640 1 --2- 172.21.15.67:0/3478557329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee8102c30 0x7fbee8103040 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.424+0000 7fbeef3a5640 1 -- 172.21.15.67:0/3478557329 >> 172.21.15.67:0/3478557329 conn(0x7fbee8077690 msgr2=0x7fbee8077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.424+0000 7fbeef3a5640 1 -- 172.21.15.67:0/3478557329 shutdown_connections 2024-07-31T14:53:03.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.424+0000 7fbeef3a5640 1 -- 172.21.15.67:0/3478557329 wait complete. 2024-07-31T14:53:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.425+0000 7fbeef3a5640 1 Processor -- start 2024-07-31T14:53:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.425+0000 7fbeef3a5640 1 -- start start 2024-07-31T14:53:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.425+0000 7fbeef3a5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbee8102c30 0x7fbee8199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.425+0000 7fbeef3a5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbee8107dd0 0x7fbee819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.425+0000 7fbeef3a5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee8108720 0x7fbee819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbeef3a5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbee81134d0 con 0x7fbee8108720 2024-07-31T14:53:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbeef3a5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbee8113350 con 0x7fbee8107dd0 2024-07-31T14:53:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbeef3a5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbee8113650 con 0x7fbee8102c30 2024-07-31T14:53:03.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbeed942640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee8108720 0x7fbee819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbeed942640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee8108720 0x7fbee819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59232/0 (socket says 172.21.15.67:59232) 2024-07-31T14:53:03.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbeed942640 1 -- 172.21.15.67:0/2809116112 learned_addr learned my addr 172.21.15.67:0/2809116112 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:03.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbeec940640 1 --2- 172.21.15.67:0/2809116112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbee8107dd0 0x7fbee819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.430 INFO:tasks.workunit.client.0.smithi067.stderr: 46500K .......... .......... .......... .......... .......... 34% 23.6M 3s 2024-07-31T14:53:03.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbeed141640 1 --2- 172.21.15.67:0/2809116112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbee8102c30 0x7fbee8199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbeed942640 1 -- 172.21.15.67:0/2809116112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbee8102c30 msgr2=0x7fbee8199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbeed942640 1 --2- 172.21.15.67:0/2809116112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbee8102c30 0x7fbee8199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbeed942640 1 -- 172.21.15.67:0/2809116112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbee8107dd0 msgr2=0x7fbee819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbeed942640 1 --2- 172.21.15.67:0/2809116112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbee8107dd0 0x7fbee819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbeed942640 1 -- 172.21.15.67:0/2809116112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee819fd90 con 0x7fbee8108720 2024-07-31T14:53:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbeed942640 1 --2- 172.21.15.67:0/2809116112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee8108720 0x7fbee819f720 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbee000cce0 tx=0x7fbee000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:03.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.426+0000 7fbede7fc640 1 -- 172.21.15.67:0/2809116112 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee000f450 con 0x7fbee8108720 2024-07-31T14:53:03.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.427+0000 7fbeef3a5640 1 -- 172.21.15.67:0/2809116112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee81a0080 con 0x7fbee8108720 2024-07-31T14:53:03.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.427+0000 7fbede7fc640 1 -- 172.21.15.67:0/2809116112 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee000faa0 con 0x7fbee8108720 2024-07-31T14:53:03.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.427+0000 7fbede7fc640 1 -- 172.21.15.67:0/2809116112 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0018b70 con 0x7fbee8108720 2024-07-31T14:53:03.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.427+0000 7fbeef3a5640 1 -- 172.21.15.67:0/2809116112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee81a68a0 con 0x7fbee8108720 2024-07-31T14:53:03.434 INFO:tasks.workunit.client.0.smithi067.stderr: 46550K .......... .......... .......... .......... .......... 34% 24.2M 3s 2024-07-31T14:53:03.434 INFO:tasks.workunit.client.0.smithi067.stderr: 46600K .......... .......... .......... .......... .......... 34% 52.5M 3s 2024-07-31T14:53:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.430+0000 7fbef58f6740 1 Processor -- start 2024-07-31T14:53:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.430+0000 7fbede7fc640 1 -- 172.21.15.67:0/2809116112 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbee0018d10 con 0x7fbee8108720 2024-07-31T14:53:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.431+0000 7fbef58f6740 1 -- start start 2024-07-31T14:53:03.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.431+0000 7fbef58f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a265d350 0x5557a2a023b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.436 INFO:tasks.workunit.client.0.smithi067.stderr: 46650K .......... .......... .......... .......... .......... 34% 22.7M 3s 2024-07-31T14:53:03.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.431+0000 7fbef58f6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a2634490 0x5557a2a028f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.431+0000 7fbef58f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a263c670 0x5557a2a02e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.431+0000 7fbef58f6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5557a28ea2f0 con 0x5557a2634490 2024-07-31T14:53:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.431+0000 7fbef58f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5557a2978a80 con 0x5557a265d350 2024-07-31T14:53:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.431+0000 7fbef58f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5557a2967f80 con 0x5557a263c670 2024-07-31T14:53:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.431+0000 7fbeed141640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a2634490 0x5557a2a028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.431+0000 7fbeed141640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a2634490 0x5557a2a028f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59238/0 (socket says 172.21.15.67:59238) 2024-07-31T14:53:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.431+0000 7fbeed141640 1 -- 172.21.15.67:0/2226003534 learned_addr learned my addr 172.21.15.67:0/2226003534 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.431+0000 7fbeec940640 1 --2- 172.21.15.67:0/2226003534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a263c670 0x5557a2a02e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.431+0000 7fbeec940640 1 --2- 172.21.15.67:0/2226003534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a265d350 0x5557a2a023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.432+0000 7fbede7fc640 1 --2- 172.21.15.67:0/2809116112 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbed40b3d70 0x7fbed40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.432+0000 7fbeed141640 1 -- 172.21.15.67:0/2226003534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a263c670 msgr2=0x5557a2a02e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.432+0000 7fbeed141640 1 --2- 172.21.15.67:0/2226003534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a263c670 0x5557a2a02e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.432+0000 7fbeed141640 1 -- 172.21.15.67:0/2226003534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a265d350 msgr2=0x5557a2a023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:03.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.432+0000 7fbeed141640 1 --2- 172.21.15.67:0/2226003534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a265d350 0x5557a2a023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.432+0000 7fbeed141640 1 -- 172.21.15.67:0/2226003534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a26c8930 con 0x5557a2634490 2024-07-31T14:53:03.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.432+0000 7fbeec940640 1 --2- 172.21.15.67:0/2226003534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a263c670 0x5557a2a02e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:03.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.432+0000 7fbede7fc640 1 -- 172.21.15.67:0/2809116112 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fbee00e4720 con 0x7fbee8108720 2024-07-31T14:53:03.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.432+0000 7fbeed942640 1 --2- 172.21.15.67:0/2809116112 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbed40b3d70 0x7fbed40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.432+0000 7fbeed141640 1 --2- 172.21.15.67:0/2226003534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a2634490 0x5557a2a028f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbee402a930 tx=0x7fbee4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22dee91a25b85695 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:03.440 INFO:tasks.workunit.client.0.smithi067.stderr: 46700K .......... .......... .......... .......... .......... 34% 41.1M 3s 2024-07-31T14:53:03.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.432+0000 7fbed37fe640 1 -- 172.21.15.67:0/2226003534 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee403f070 con 0x5557a2634490 2024-07-31T14:53:03.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.432+0000 7fbed37fe640 1 -- 172.21.15.67:0/2226003534 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee40316a0 con 0x5557a2634490 2024-07-31T14:53:03.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.432+0000 7fbed37fe640 1 -- 172.21.15.67:0/2226003534 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4035070 con 0x5557a2634490 2024-07-31T14:53:03.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.433+0000 7fbef58f6740 1 -- 172.21.15.67:0/2226003534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a2634490 msgr2=0x5557a2a028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:03.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.433+0000 7fbeed942640 1 --2- 172.21.15.67:0/2809116112 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbed40b3d70 0x7fbed40b6280 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbee00a70a0 tx=0x7fbee00774d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:03.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.433+0000 7fbef58f6740 1 --2- 172.21.15.67:0/2226003534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a2634490 0x5557a2a028f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbee402a930 tx=0x7fbee4032ce0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.433+0000 7fbef58f6740 1 -- 172.21.15.67:0/2226003534 shutdown_connections 2024-07-31T14:53:03.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.433+0000 7fbef58f6740 1 --2- 172.21.15.67:0/2226003534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a263c670 0x5557a2a02e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.433+0000 7fbef58f6740 1 --2- 172.21.15.67:0/2226003534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a2634490 0x5557a2a028f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.433+0000 7fbef58f6740 1 --2- 172.21.15.67:0/2226003534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a265d350 0x5557a2a023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.433+0000 7fbef58f6740 1 -- 172.21.15.67:0/2226003534 >> 172.21.15.67:0/2226003534 conn(0x5557a2651740 msgr2=0x5557a2a002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.433+0000 7fbef58f6740 1 -- 172.21.15.67:0/2226003534 shutdown_connections 2024-07-31T14:53:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.433+0000 7fbef58f6740 1 -- 172.21.15.67:0/2226003534 wait complete. 2024-07-31T14:53:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.434+0000 7fbef58f6740 1 -- start start 2024-07-31T14:53:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.434+0000 7fbef58f6740 1 Processor -- start 2024-07-31T14:53:03.444 INFO:tasks.workunit.client.0.smithi067.stderr: 46750K .......... .......... .......... .......... .......... 34% 23.5M 3s 2024-07-31T14:53:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbef58f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a2634490 0x5557a2a0df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbef58f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a263c670 0x5557a2a0e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbef58f6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a265d350 0x5557a2a19ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbef58f6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5557a290a4f0 con 0x5557a265d350 2024-07-31T14:53:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbef58f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5557a296bfc0 con 0x5557a263c670 2024-07-31T14:53:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbef58f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5557a27cb890 con 0x5557a2634490 2024-07-31T14:53:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbef58f6740 10 client.0 _ll_register_callbacks cb 0x5557a2914500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbeec940640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a265d350 0x5557a2a19ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbeec940640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a265d350 0x5557a2a19ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59244/0 (socket says 172.21.15.67:59244) 2024-07-31T14:53:03.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbeec940640 1 -- 172.21.15.67:0/4207563384 learned_addr learned my addr 172.21.15.67:0/4207563384 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:03.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbeed141640 1 --2- 172.21.15.67:0/4207563384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a263c670 0x5557a2a0e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbeec940640 1 --2- 172.21.15.67:0/4207563384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a2634490 0x5557a2a0df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbeec940640 1 -- 172.21.15.67:0/4207563384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a2634490 msgr2=0x5557a2a0df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbeec940640 1 --2- 172.21.15.67:0/4207563384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a2634490 0x5557a2a0df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbeec940640 1 -- 172.21.15.67:0/4207563384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a263c670 msgr2=0x5557a2a0e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbeec940640 1 --2- 172.21.15.67:0/4207563384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a263c670 0x5557a2a0e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbeec940640 1 -- 172.21.15.67:0/4207563384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a2919c50 con 0x5557a265d350 2024-07-31T14:53:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbeed141640 1 --2- 172.21.15.67:0/4207563384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a263c670 0x5557a2a0e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbeec940640 1 --2- 172.21.15.67:0/4207563384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a265d350 0x5557a2a19ca0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbee80754e0 tx=0x7fbed80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b48e4646ff232a9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.435+0000 7fbed0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.436+0000 7fbed0ff9640 1 -- 172.21.15.67:0/4207563384 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8010ce0 con 0x5557a265d350 2024-07-31T14:53:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.436+0000 7fbed0ff9640 1 -- 172.21.15.67:0/4207563384 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fbed8004030 con 0x5557a265d350 2024-07-31T14:53:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.436+0000 7fbef58f6740 1 -- 172.21.15.67:0/4207563384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbee40316a0 con 0x5557a265d350 2024-07-31T14:53:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.436+0000 7fbed0ff9640 1 -- 172.21.15.67:0/4207563384 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbed8004230 con 0x5557a265d350 2024-07-31T14:53:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.436+0000 7fbef58f6740 10 client.16185 fetch_fsmap learned FSMap version 30 2024-07-31T14:53:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.436+0000 7fbef58f6740 1 -- 172.21.15.67:0/4207563384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5557a2a1c9d0 con 0x5557a265d350 2024-07-31T14:53:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.436+0000 7fbed0ff9640 1 -- 172.21.15.67:0/4207563384 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee40316a0 con 0x5557a265d350 2024-07-31T14:53:03.449 INFO:tasks.workunit.client.0.smithi067.stderr: 46800K .......... .......... .......... .......... .......... 34% 24.6M 3s 2024-07-31T14:53:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.436+0000 7fbed0ff9640 1 -- 172.21.15.67:0/4207563384 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbed8020450 con 0x5557a265d350 2024-07-31T14:53:03.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.436+0000 7fbef58f6740 10 client.16185 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:53:03.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.436+0000 7fbef58f6740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:03.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.436+0000 7fbef58f6740 20 client.16185 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:03.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.436+0000 7fbef58f6740 1 --2- 172.21.15.67:0/4207563384 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a261bcd0 0x5557a2a1fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.436+0000 7fbef58f6740 4 client.16185 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:53:03.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.436+0000 7fbef58f6740 1 -- 172.21.15.67:0/4207563384 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557a297a9d0 con 0x5557a261bcd0 2024-07-31T14:53:03.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.437+0000 7fbeed141640 1 --2- 172.21.15.67:0/4207563384 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a261bcd0 0x5557a2a1fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.451 INFO:tasks.workunit.client.0.smithi067.stderr: 46850K .......... .......... .......... .......... .......... 34% 50.4M 3s 2024-07-31T14:53:03.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.437+0000 7fbeed141640 1 --2- 172.21.15.67:0/4207563384 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a261bcd0 0x5557a2a1fec0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2ebbc1c796a4e2b server_cookie=ccc2c9895d7300c5 in_seq=0 out_seq=0 2024-07-31T14:53:03.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.437+0000 7fbed0ff9640 10 client.16185 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.438+0000 7fbed0ff9640 1 -- 172.21.15.67:0/4207563384 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557a297a9d0 con 0x5557a261bcd0 2024-07-31T14:53:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.438+0000 7fbed0ff9640 10 client.16185 handle_command_reply: tid=0 2024-07-31T14:53:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.438+0000 7fbef58f6740 1 client.16185 shutdown 2024-07-31T14:53:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.438+0000 7fbef58f6740 1 -- 172.21.15.67:0/4207563384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a265d350 msgr2=0x5557a2a19ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.438+0000 7fbef58f6740 1 --2- 172.21.15.67:0/4207563384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a265d350 0x5557a2a19ca0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbee80754e0 tx=0x7fbed80046c0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.438+0000 7fbef58f6740 1 -- 172.21.15.67:0/4207563384 shutdown_connections 2024-07-31T14:53:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.438+0000 7fbef58f6740 1 --2- 172.21.15.67:0/4207563384 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a261bcd0 0x5557a2a1fec0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.438+0000 7fbef58f6740 1 --2- 172.21.15.67:0/4207563384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a265d350 0x5557a2a19ca0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.438+0000 7fbef58f6740 1 --2- 172.21.15.67:0/4207563384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a263c670 0x5557a2a0e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.438+0000 7fbed0ff9640 0 client.16185 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.438+0000 7fbef58f6740 1 --2- 172.21.15.67:0/4207563384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a2634490 0x5557a2a0df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.438+0000 7fbef58f6740 1 -- 172.21.15.67:0/4207563384 >> 172.21.15.67:0/4207563384 conn(0x5557a2651740 msgr2=0x5557a2a05590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.438+0000 7fbef58f6740 1 -- 172.21.15.67:0/4207563384 shutdown_connections 2024-07-31T14:53:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.438+0000 7fbef58f6740 1 -- 172.21.15.67:0/4207563384 wait complete. 2024-07-31T14:53:03.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.439+0000 7fbef58f6740 20 client.16185 trim_cache size 0 max 16384 2024-07-31T14:53:03.462 INFO:tasks.workunit.client.0.smithi067.stderr: 46900K .......... .......... .......... .......... .......... 34% 23.8M 3s 2024-07-31T14:53:03.462 INFO:tasks.workunit.client.0.smithi067.stderr: 46950K .......... .......... .......... .......... .......... 34% 38.5M 3s 2024-07-31T14:53:03.462 INFO:tasks.workunit.client.0.smithi067.stderr: 47000K .......... .......... .......... .......... .......... 34% 22.7M 3s 2024-07-31T14:53:03.462 INFO:tasks.workunit.client.0.smithi067.stderr: 47050K .......... .......... .......... .......... .......... 34% 25.2M 3s 2024-07-31T14:53:03.462 INFO:tasks.workunit.client.0.smithi067.stderr: 47100K .......... .......... .......... .......... .......... 34% 54.8M 3s 2024-07-31T14:53:03.462 INFO:tasks.workunit.client.0.smithi067.stderr: 47150K .......... .......... .......... .......... .......... 34% 23.2M 3s 2024-07-31T14:53:03.463 INFO:tasks.workunit.client.0.smithi067.stderr: 47200K .......... .......... .......... .......... .......... 34% 23.7M 3s 2024-07-31T14:53:03.463 INFO:tasks.workunit.client.0.smithi067.stderr: 47250K .......... .......... .......... .......... .......... 34% 48.7M 3s 2024-07-31T14:53:03.463 INFO:tasks.workunit.client.0.smithi067.stderr: 47300K .......... .......... .......... .......... .......... 34% 30.4M 3s 2024-07-31T14:53:03.463 INFO:tasks.workunit.client.0.smithi067.stderr: 47350K .......... .......... .......... .......... .......... 34% 30.7M 3s 2024-07-31T14:53:03.463 INFO:tasks.workunit.client.0.smithi067.stderr: 47400K .......... .......... .......... .......... .......... 34% 22.9M 3s 2024-07-31T14:53:03.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.458+0000 7fbef58f6740 1 Processor -- start 2024-07-31T14:53:03.464 INFO:tasks.workunit.client.0.smithi067.stderr: 47450K .......... .......... .......... .......... .......... 34% 30.1M 3s 2024-07-31T14:53:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.458+0000 7fbef58f6740 1 -- start start 2024-07-31T14:53:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.458+0000 7fbef58f6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a2634490 0x5557a2a0f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.458+0000 7fbef58f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a263c670 0x5557a2a19a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.458+0000 7fbef58f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a265d350 0x5557a2a19fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.458+0000 7fbef58f6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbee81b0620 con 0x5557a2634490 2024-07-31T14:53:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.458+0000 7fbef58f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbee81b2160 con 0x5557a263c670 2024-07-31T14:53:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.458+0000 7fbef58f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbee81b0cf0 con 0x5557a265d350 2024-07-31T14:53:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.458+0000 7fbeec940640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a2634490 0x5557a2a0f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.458+0000 7fbeec940640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a2634490 0x5557a2a0f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59250/0 (socket says 172.21.15.67:59250) 2024-07-31T14:53:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.458+0000 7fbeec940640 1 -- 172.21.15.67:0/2960309947 learned_addr learned my addr 172.21.15.67:0/2960309947 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.458+0000 7fbeed141640 1 --2- 172.21.15.67:0/2960309947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a263c670 0x5557a2a19a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.458+0000 7fbeec940640 1 --2- 172.21.15.67:0/2960309947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a265d350 0x5557a2a19fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.459+0000 7fbeed141640 1 -- 172.21.15.67:0/2960309947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a265d350 msgr2=0x5557a2a19fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.459+0000 7fbeed141640 1 --2- 172.21.15.67:0/2960309947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a265d350 0x5557a2a19fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.459+0000 7fbeed141640 1 -- 172.21.15.67:0/2960309947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a2634490 msgr2=0x5557a2a0f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.459+0000 7fbeed141640 1 --2- 172.21.15.67:0/2960309947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a2634490 0x5557a2a0f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.459+0000 7fbeed141640 1 -- 172.21.15.67:0/2960309947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a2a046a0 con 0x5557a263c670 2024-07-31T14:53:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.459+0000 7fbeec940640 1 --2- 172.21.15.67:0/2960309947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a2634490 0x5557a2a0f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.459+0000 7fbeed141640 1 --2- 172.21.15.67:0/2960309947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a263c670 0x5557a2a19a90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbee4031440 tx=0x7fbee40051c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43d6828860ce9f7e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.459+0000 7fbed2ffd640 1 -- 172.21.15.67:0/2960309947 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4050020 con 0x5557a263c670 2024-07-31T14:53:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.459+0000 7fbed2ffd640 1 -- 172.21.15.67:0/2960309947 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee40035f0 con 0x5557a263c670 2024-07-31T14:53:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.459+0000 7fbed2ffd640 1 -- 172.21.15.67:0/2960309947 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee403f040 con 0x5557a263c670 2024-07-31T14:53:03.468 INFO:tasks.workunit.client.0.smithi067.stderr: 47500K .......... .......... .......... .......... .......... 35% 29.1M 3s 2024-07-31T14:53:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.460+0000 7fbef58f6740 1 -- 172.21.15.67:0/2960309947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a263c670 msgr2=0x5557a2a19a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.460+0000 7fbef58f6740 1 --2- 172.21.15.67:0/2960309947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a263c670 0x5557a2a19a90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbee4031440 tx=0x7fbee40051c0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.460+0000 7fbef58f6740 1 -- 172.21.15.67:0/2960309947 shutdown_connections 2024-07-31T14:53:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.460+0000 7fbef58f6740 1 --2- 172.21.15.67:0/2960309947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a265d350 0x5557a2a19fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.460+0000 7fbef58f6740 1 --2- 172.21.15.67:0/2960309947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a263c670 0x5557a2a19a90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.460+0000 7fbef58f6740 1 --2- 172.21.15.67:0/2960309947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a2634490 0x5557a2a0f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.460+0000 7fbef58f6740 1 -- 172.21.15.67:0/2960309947 >> 172.21.15.67:0/2960309947 conn(0x5557a2651740 msgr2=0x5557a2a0f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:03.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.460+0000 7fbef58f6740 1 -- 172.21.15.67:0/2960309947 shutdown_connections 2024-07-31T14:53:03.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.460+0000 7fbef58f6740 1 -- 172.21.15.67:0/2960309947 wait complete. 2024-07-31T14:53:03.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.461+0000 7fbef58f6740 1 -- start start 2024-07-31T14:53:03.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.461+0000 7fbef58f6740 1 Processor -- start 2024-07-31T14:53:03.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.461+0000 7fbef58f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a2634490 0x5557a2a16480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.461+0000 7fbef58f6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a263c670 0x5557a2a11980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.461+0000 7fbef58f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a265d350 0x5557a2a12070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.461+0000 7fbef58f6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbee81b1a90 con 0x5557a263c670 2024-07-31T14:53:03.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.461+0000 7fbef58f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbee81b13c0 con 0x5557a265d350 2024-07-31T14:53:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.461+0000 7fbef58f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbee81ae600 con 0x5557a2634490 2024-07-31T14:53:03.472 INFO:tasks.workunit.client.0.smithi067.stderr: 47550K .......... .......... .......... .......... .......... 35% 23.7M 3s 2024-07-31T14:53:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.461+0000 7fbef58f6740 10 client.0 _ll_register_callbacks cb 0x7fbed8013530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.461+0000 7fbeed141640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a263c670 0x5557a2a11980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.461+0000 7fbeed141640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a263c670 0x5557a2a11980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59254/0 (socket says 172.21.15.67:59254) 2024-07-31T14:53:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.462+0000 7fbeed141640 1 -- 172.21.15.67:0/500004794 learned_addr learned my addr 172.21.15.67:0/500004794 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.462+0000 7fbeec940640 1 --2- 172.21.15.67:0/500004794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a2634490 0x5557a2a16480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.462+0000 7fbeec940640 1 --2- 172.21.15.67:0/500004794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a265d350 0x5557a2a12070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.462+0000 7fbeed141640 1 -- 172.21.15.67:0/500004794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a2634490 msgr2=0x5557a2a16480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.462+0000 7fbeed141640 1 --2- 172.21.15.67:0/500004794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a2634490 0x5557a2a16480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.462+0000 7fbeed141640 1 -- 172.21.15.67:0/500004794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a265d350 msgr2=0x5557a2a12070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.462+0000 7fbeed141640 1 --2- 172.21.15.67:0/500004794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a265d350 0x5557a2a12070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.462+0000 7fbeed141640 1 -- 172.21.15.67:0/500004794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a2a87650 con 0x5557a263c670 2024-07-31T14:53:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.462+0000 7fbeed141640 1 --2- 172.21.15.67:0/500004794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a263c670 0x5557a2a11980 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbee40095e0 tx=0x7fbee4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ebe1b108ad7d70 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.462+0000 7fbed0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:03.475 INFO:tasks.workunit.client.0.smithi067.stderr: 47600K .......... .......... .......... .......... .......... 35% 54.3M 3s 2024-07-31T14:53:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.462+0000 7fbed0ff9640 1 -- 172.21.15.67:0/500004794 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4050030 con 0x5557a263c670 2024-07-31T14:53:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.462+0000 7fbed0ff9640 1 -- 172.21.15.67:0/500004794 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fbee4004240 con 0x5557a263c670 2024-07-31T14:53:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.462+0000 7fbef58f6740 1 -- 172.21.15.67:0/500004794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbee40035f0 con 0x5557a263c670 2024-07-31T14:53:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.463+0000 7fbed0ff9640 1 -- 172.21.15.67:0/500004794 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbee4067050 con 0x5557a263c670 2024-07-31T14:53:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.463+0000 7fbef58f6740 10 client.16191 fetch_fsmap learned FSMap version 30 2024-07-31T14:53:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.463+0000 7fbef58f6740 1 -- 172.21.15.67:0/500004794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5557a2a88780 con 0x5557a263c670 2024-07-31T14:53:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.463+0000 7fbed0ff9640 1 -- 172.21.15.67:0/500004794 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee40035f0 con 0x5557a263c670 2024-07-31T14:53:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.463+0000 7fbed0ff9640 1 -- 172.21.15.67:0/500004794 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbee40046c0 con 0x5557a263c670 2024-07-31T14:53:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.463+0000 7fbef58f6740 10 client.16191 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:53:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.463+0000 7fbef58f6740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.463+0000 7fbef58f6740 20 client.16191 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.463+0000 7fbef58f6740 1 --2- 172.21.15.67:0/500004794 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a25f1080 0x5557a2a8aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.463+0000 7fbef58f6740 4 client.16191 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T14:53:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.463+0000 7fbef58f6740 1 -- 172.21.15.67:0/500004794 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5557a2976470 con 0x5557a25f1080 2024-07-31T14:53:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.463+0000 7fbeec940640 1 --2- 172.21.15.67:0/500004794 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a25f1080 0x5557a2a8aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:03.479 INFO:tasks.workunit.client.0.smithi067.stderr: 47650K .......... .......... .......... .......... .......... 35% 29.1M 3s 2024-07-31T14:53:03.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.464+0000 7fbed0ff9640 10 client.16191 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:03.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.464+0000 7fbeec940640 1 --2- 172.21.15.67:0/500004794 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a25f1080 0x5557a2a8aa60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afb0c001bb5b0656 server_cookie=901e01ef03ac3b75 in_seq=0 out_seq=0 2024-07-31T14:53:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.465+0000 7fbed0ff9640 1 -- 172.21.15.67:0/500004794 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5557a2976470 con 0x5557a25f1080 2024-07-31T14:53:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.465+0000 7fbed0ff9640 10 client.16191 handle_command_reply: tid=0 2024-07-31T14:53:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.465+0000 7fbef58f6740 1 client.16191 shutdown 2024-07-31T14:53:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.465+0000 7fbef58f6740 1 -- 172.21.15.67:0/500004794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a263c670 msgr2=0x5557a2a11980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.465+0000 7fbef58f6740 1 --2- 172.21.15.67:0/500004794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a263c670 0x5557a2a11980 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbee40095e0 tx=0x7fbee4032ce0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.465+0000 7fbef58f6740 1 -- 172.21.15.67:0/500004794 shutdown_connections 2024-07-31T14:53:03.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.465+0000 7fbef58f6740 1 --2- 172.21.15.67:0/500004794 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a25f1080 0x5557a2a8aa60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.466+0000 7fbef58f6740 1 --2- 172.21.15.67:0/500004794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a265d350 0x5557a2a12070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.466+0000 7fbef58f6740 1 --2- 172.21.15.67:0/500004794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a263c670 0x5557a2a11980 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.466+0000 7fbef58f6740 1 --2- 172.21.15.67:0/500004794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a2634490 0x5557a2a16480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.466+0000 7fbef58f6740 1 -- 172.21.15.67:0/500004794 >> 172.21.15.67:0/500004794 conn(0x5557a2651740 msgr2=0x5557a2a03390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:03.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.466+0000 7fbed0ff9640 0 client.16191 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:03.482 INFO:tasks.workunit.client.0.smithi067.stderr: 47700K .......... .......... .......... .......... .......... 35% 23.5M 3s 2024-07-31T14:53:03.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.466+0000 7fbef58f6740 1 -- 172.21.15.67:0/500004794 shutdown_connections 2024-07-31T14:53:03.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.466+0000 7fbef58f6740 1 -- 172.21.15.67:0/500004794 wait complete. 2024-07-31T14:53:03.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.466+0000 7fbef58f6740 20 client.16191 trim_cache size 0 max 16384 2024-07-31T14:53:03.483 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:53:03.483 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T14:53:03.483 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "f4b83435-0c0a-44b1-9040-db919a275da6", 2024-07-31T14:53:03.484 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T14:53:03.484 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:53:03.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.467+0000 7fbeef3a5640 1 -- 172.21.15.67:0/2809116112 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbed40b3d70 msgr2=0x7fbed40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:03.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.467+0000 7fbeef3a5640 1 --2- 172.21.15.67:0/2809116112 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbed40b3d70 0x7fbed40b6280 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbee00a70a0 tx=0x7fbee00774d0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.467+0000 7fbeef3a5640 1 -- 172.21.15.67:0/2809116112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee8108720 msgr2=0x7fbee819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:03.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.467+0000 7fbeef3a5640 1 --2- 172.21.15.67:0/2809116112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee8108720 0x7fbee819f720 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbee000cce0 tx=0x7fbee000a620 comp rx=0 tx=0).stop 2024-07-31T14:53:03.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.467+0000 7fbeef3a5640 1 -- 172.21.15.67:0/2809116112 shutdown_connections 2024-07-31T14:53:03.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.467+0000 7fbeef3a5640 1 --2- 172.21.15.67:0/2809116112 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbed40b3d70 0x7fbed40b6280 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.467+0000 7fbeef3a5640 1 --2- 172.21.15.67:0/2809116112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee8108720 0x7fbee819f720 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.467+0000 7fbeef3a5640 1 --2- 172.21.15.67:0/2809116112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbee8107dd0 0x7fbee819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.467+0000 7fbeef3a5640 1 --2- 172.21.15.67:0/2809116112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbee8102c30 0x7fbee8199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:03.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.467+0000 7fbeef3a5640 1 -- 172.21.15.67:0/2809116112 >> 172.21.15.67:0/2809116112 conn(0x7fbee8077690 msgr2=0x7fbee8105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:03.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.467+0000 7fbeef3a5640 1 -- 172.21.15.67:0/2809116112 shutdown_connections 2024-07-31T14:53:03.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:03.467+0000 7fbeef3a5640 1 -- 172.21.15.67:0/2809116112 wait complete. 2024-07-31T14:53:03.502 INFO:tasks.workunit.client.0.smithi067.stderr: 47750K .......... .......... .......... .......... .......... 35% 29.8M 3s 2024-07-31T14:53:03.502 INFO:tasks.workunit.client.0.smithi067.stderr: 47800K .......... .......... .......... .......... .......... 35% 23.0M 3s 2024-07-31T14:53:03.502 INFO:tasks.workunit.client.0.smithi067.stderr: 47850K .......... .......... .......... .......... .......... 35% 58.1M 3s 2024-07-31T14:53:03.502 INFO:tasks.workunit.client.0.smithi067.stderr: 47900K .......... .......... .......... .......... .......... 35% 24.2M 3s 2024-07-31T14:53:03.502 INFO:tasks.workunit.client.0.smithi067.stderr: 47950K .......... .......... .......... .......... .......... 35% 30.0M 3s 2024-07-31T14:53:03.502 INFO:tasks.workunit.client.0.smithi067.stderr: 48000K .......... .......... .......... .......... .......... 35% 28.5M 3s 2024-07-31T14:53:03.503 INFO:tasks.workunit.client.0.smithi067.stderr: 48050K .......... .......... .......... .......... .......... 35% 23.4M 3s 2024-07-31T14:53:03.503 INFO:tasks.workunit.client.0.smithi067.stderr: 48100K .......... .......... .......... .......... .......... 35% 53.1M 3s 2024-07-31T14:53:03.503 INFO:tasks.workunit.client.0.smithi067.stderr: 48150K .......... .......... .......... .......... .......... 35% 28.5M 3s 2024-07-31T14:53:03.503 INFO:tasks.workunit.client.0.smithi067.stderr: 48200K .......... .......... .......... .......... .......... 35% 25.2M 3s 2024-07-31T14:53:03.503 INFO:tasks.workunit.client.0.smithi067.stderr: 48250K .......... .......... .......... .......... .......... 35% 28.0M 3s 2024-07-31T14:53:03.503 INFO:tasks.workunit.client.0.smithi067.stderr: 48300K .......... .......... .......... .......... .......... 35% 23.9M 3s 2024-07-31T14:53:03.503 INFO:tasks.workunit.client.0.smithi067.stderr: 48350K .......... .......... .......... .......... .......... 35% 23.2M 3s 2024-07-31T14:53:03.503 INFO:tasks.workunit.client.0.smithi067.stderr: 48400K .......... .......... .......... .......... .......... 35% 58.3M 3s 2024-07-31T14:53:03.503 INFO:tasks.workunit.client.0.smithi067.stderr: 48450K .......... .......... .......... .......... .......... 35% 23.2M 3s 2024-07-31T14:53:03.503 INFO:tasks.workunit.client.0.smithi067.stderr: 48500K .......... .......... .......... .......... .......... 35% 37.1M 3s 2024-07-31T14:53:03.503 INFO:tasks.workunit.client.0.smithi067.stderr: 48550K .......... .......... .......... .......... .......... 35% 24.0M 3s 2024-07-31T14:53:03.503 INFO:tasks.workunit.client.0.smithi067.stderr: 48600K .......... .......... .......... .......... .......... 35% 24.8M 3s 2024-07-31T14:53:03.503 INFO:tasks.workunit.client.0.smithi067.stderr: 48650K .......... .......... .......... .......... .......... 35% 47.5M 3s 2024-07-31T14:53:03.503 INFO:tasks.workunit.client.0.smithi067.stderr: 48700K .......... .......... .......... .......... .......... 35% 23.5M 3s 2024-07-31T14:53:03.506 INFO:tasks.workunit.client.0.smithi067.stderr: 48750K .......... .......... .......... .......... .......... 35% 28.3M 3s 2024-07-31T14:53:03.506 INFO:tasks.workunit.client.0.smithi067.stderr: 48800K .......... .......... .......... .......... .......... 35% 31.2M 3s 2024-07-31T14:53:03.506 INFO:tasks.workunit.client.0.smithi067.stderr: 48850K .......... .......... .......... .......... .......... 36% 44.1M 3s 2024-07-31T14:53:03.507 INFO:tasks.workunit.client.0.smithi067.stderr: 48900K .......... .......... .......... .......... .......... 36% 25.3M 3s 2024-07-31T14:53:03.510 INFO:tasks.workunit.client.0.smithi067.stderr: 48950K .......... .......... .......... .......... .......... 36% 23.5M 3s 2024-07-31T14:53:03.511 INFO:tasks.workunit.client.0.smithi067.stderr: 49000K .......... .......... .......... .......... .......... 36% 41.9M 3s 2024-07-31T14:53:03.513 INFO:tasks.workunit.client.0.smithi067.stderr: 49050K .......... .......... .......... .......... .......... 36% 22.8M 3s 2024-07-31T14:53:03.515 INFO:tasks.workunit.client.0.smithi067.stderr: 49100K .......... .......... .......... .......... .......... 36% 24.3M 3s 2024-07-31T14:53:03.516 INFO:tasks.workunit.client.0.smithi067.stderr: 49150K .......... .......... .......... .......... .......... 36% 48.8M 3s 2024-07-31T14:53:03.518 INFO:tasks.workunit.client.0.smithi067.stderr: 49200K .......... .......... .......... .......... .......... 36% 25.3M 3s 2024-07-31T14:53:03.519 INFO:tasks.workunit.client.0.smithi067.stderr: 49250K .......... .......... .......... .......... .......... 36% 35.5M 3s 2024-07-31T14:53:03.521 INFO:tasks.workunit.client.0.smithi067.stderr: 49300K .......... .......... .......... .......... .......... 36% 24.2M 3s 2024-07-31T14:53:03.523 INFO:tasks.workunit.client.0.smithi067.stderr: 49350K .......... .......... .......... .......... .......... 36% 24.2M 3s 2024-07-31T14:53:03.524 INFO:tasks.workunit.client.0.smithi067.stderr: 49400K .......... .......... .......... .......... .......... 36% 59.7M 3s 2024-07-31T14:53:03.526 INFO:tasks.workunit.client.0.smithi067.stderr: 49450K .......... .......... .......... .......... .......... 36% 22.1M 3s 2024-07-31T14:53:03.528 INFO:tasks.workunit.client.0.smithi067.stderr: 49500K .......... .......... .......... .......... .......... 36% 35.6M 3s 2024-07-31T14:53:03.530 INFO:tasks.workunit.client.0.smithi067.stderr: 49550K .......... .......... .......... .......... .......... 36% 24.1M 3s 2024-07-31T14:53:03.532 INFO:tasks.workunit.client.0.smithi067.stderr: 49600K .......... .......... .......... .......... .......... 36% 23.2M 3s 2024-07-31T14:53:03.533 INFO:tasks.workunit.client.0.smithi067.stderr: 49650K .......... .......... .......... .......... .......... 36% 43.7M 3s 2024-07-31T14:53:03.535 INFO:tasks.workunit.client.0.smithi067.stderr: 49700K .......... .......... .......... .......... .......... 36% 25.7M 3s 2024-07-31T14:53:03.536 INFO:tasks.workunit.client.0.smithi067.stderr: 49750K .......... .......... .......... .......... .......... 36% 37.4M 3s 2024-07-31T14:53:03.538 INFO:tasks.workunit.client.0.smithi067.stderr: 49800K .......... .......... .......... .......... .......... 36% 24.7M 3s 2024-07-31T14:53:03.540 INFO:tasks.workunit.client.0.smithi067.stderr: 49850K .......... .......... .......... .......... .......... 36% 24.4M 3s 2024-07-31T14:53:03.541 INFO:tasks.workunit.client.0.smithi067.stderr: 49900K .......... .......... .......... .......... .......... 36% 48.6M 3s 2024-07-31T14:53:03.543 INFO:tasks.workunit.client.0.smithi067.stderr: 49950K .......... .......... .......... .......... .......... 36% 24.3M 3s 2024-07-31T14:53:03.544 INFO:tasks.workunit.client.0.smithi067.stderr: 50000K .......... .......... .......... .......... .......... 36% 39.3M 3s 2024-07-31T14:53:03.547 INFO:tasks.workunit.client.0.smithi067.stderr: 50050K .......... .......... .......... .......... .......... 36% 23.6M 3s 2024-07-31T14:53:03.548 INFO:tasks.workunit.client.0.smithi067.stderr: 50100K .......... .......... .......... .......... .......... 36% 23.5M 3s 2024-07-31T14:53:03.549 INFO:tasks.workunit.client.0.smithi067.stderr: 50150K .......... .......... .......... .......... .......... 36% 51.8M 3s 2024-07-31T14:53:03.552 INFO:tasks.workunit.client.0.smithi067.stderr: 50200K .......... .......... .......... .......... .......... 37% 22.3M 3s 2024-07-31T14:53:03.553 INFO:tasks.workunit.client.0.smithi067.stderr: 50250K .......... .......... .......... .......... .......... 37% 38.2M 3s 2024-07-31T14:53:03.556 INFO:tasks.workunit.client.0.smithi067.stderr: 50300K .......... .......... .......... .......... .......... 37% 18.8M 3s 2024-07-31T14:53:03.557 INFO:tasks.workunit.client.0.smithi067.stderr: 50350K .......... .......... .......... .......... .......... 37% 83.8M 3s 2024-07-31T14:53:03.558 INFO:tasks.workunit.client.0.smithi067.stderr: 50400K .......... .......... .......... .......... .......... 37% 26.2M 3s 2024-07-31T14:53:03.560 INFO:tasks.workunit.client.0.smithi067.stderr: 50450K .......... .......... .......... .......... .......... 37% 23.8M 3s 2024-07-31T14:53:03.561 INFO:tasks.workunit.client.0.smithi067.stderr: 50500K .......... .......... .......... .......... .......... 37% 36.7M 3s 2024-07-31T14:53:03.564 INFO:tasks.workunit.client.0.smithi067.stderr: 50550K .......... .......... .......... .......... .......... 37% 19.7M 3s 2024-07-31T14:53:03.565 INFO:tasks.workunit.client.0.smithi067.stderr: 50600K .......... .......... .......... .......... .......... 37% 39.3M 3s 2024-07-31T14:53:03.566 INFO:tasks.workunit.client.0.smithi067.stderr: 50650K .......... .......... .......... .......... .......... 37% 39.2M 3s 2024-07-31T14:53:03.568 INFO:tasks.workunit.client.0.smithi067.stderr: 50700K .......... .......... .......... .......... .......... 37% 22.6M 3s 2024-07-31T14:53:03.571 INFO:tasks.workunit.client.0.smithi067.stderr: 50750K .......... .......... .......... .......... .......... 37% 38.6M 3s 2024-07-31T14:53:03.572 INFO:tasks.workunit.client.0.smithi067.stderr: 50800K .......... .......... .......... .......... .......... 37% 20.8M 3s 2024-07-31T14:53:03.574 INFO:tasks.workunit.client.0.smithi067.stderr: 50850K .......... .......... .......... .......... .......... 37% 26.1M 3s 2024-07-31T14:53:03.575 INFO:tasks.workunit.client.0.smithi067.stderr: 50900K .......... .......... .......... .......... .......... 37% 62.1M 3s 2024-07-31T14:53:03.576 INFO:tasks.workunit.client.0.smithi067.stderr: 50950K .......... .......... .......... .......... .......... 37% 29.7M 3s 2024-07-31T14:53:03.579 INFO:tasks.workunit.client.0.smithi067.stderr: 51000K .......... .......... .......... .......... .......... 37% 22.2M 3s 2024-07-31T14:53:03.580 INFO:tasks.workunit.client.0.smithi067.stderr: 51050K .......... .......... .......... .......... .......... 37% 30.0M 3s 2024-07-31T14:53:03.582 INFO:tasks.workunit.client.0.smithi067.stderr: 51100K .......... .......... .......... .......... .......... 37% 32.0M 3s 2024-07-31T14:53:03.583 INFO:tasks.workunit.client.0.smithi067.stderr: 51150K .......... .......... .......... .......... .......... 37% 27.7M 3s 2024-07-31T14:53:03.585 INFO:tasks.workunit.client.0.smithi067.stderr: 51200K .......... .......... .......... .......... .......... 37% 27.1M 3s 2024-07-31T14:53:03.587 INFO:tasks.workunit.client.0.smithi067.stderr: 51250K .......... .......... .......... .......... .......... 37% 35.7M 3s 2024-07-31T14:53:03.589 INFO:tasks.workunit.client.0.smithi067.stderr: 51300K .......... .......... .......... .......... .......... 37% 19.1M 3s 2024-07-31T14:53:03.591 INFO:tasks.workunit.client.0.smithi067.stderr: 51350K .......... .......... .......... .......... .......... 37% 33.3M 3s 2024-07-31T14:53:03.592 INFO:tasks.workunit.client.0.smithi067.stderr: 51400K .......... .......... .......... .......... .......... 37% 49.8M 3s 2024-07-31T14:53:03.594 INFO:tasks.workunit.client.0.smithi067.stderr: 51450K .......... .......... .......... .......... .......... 37% 25.0M 3s 2024-07-31T14:53:03.595 INFO:tasks.workunit.client.0.smithi067.stderr: 51500K .......... .......... .......... .......... .......... 37% 38.9M 3s 2024-07-31T14:53:03.598 INFO:tasks.workunit.client.0.smithi067.stderr: 51550K .......... .......... .......... .......... .......... 38% 18.1M 3s 2024-07-31T14:53:03.599 INFO:tasks.workunit.client.0.smithi067.stderr: 51600K .......... .......... .......... .......... .......... 38% 32.8M 3s 2024-07-31T14:53:03.600 INFO:tasks.workunit.client.0.smithi067.stderr: 51650K .......... .......... .......... .......... .......... 38% 50.6M 3s 2024-07-31T14:53:03.602 INFO:tasks.workunit.client.0.smithi067.stderr: 51700K .......... .......... .......... .......... .......... 38% 26.1M 3s 2024-07-31T14:53:03.603 INFO:tasks.workunit.client.0.smithi067.stderr: 51750K .......... .......... .......... .......... .......... 38% 37.9M 3s 2024-07-31T14:53:03.605 INFO:tasks.workunit.client.0.smithi067.stderr: 51800K .......... .......... .......... .......... .......... 38% 20.7M 3s 2024-07-31T14:53:03.607 INFO:tasks.workunit.client.0.smithi067.stderr: 51850K .......... .......... .......... .......... .......... 38% 36.7M 3s 2024-07-31T14:53:03.608 INFO:tasks.workunit.client.0.smithi067.stderr: 51900K .......... .......... .......... .......... .......... 38% 35.5M 3s 2024-07-31T14:53:03.610 INFO:tasks.workunit.client.0.smithi067.stderr: 51950K .......... .......... .......... .......... .......... 38% 28.9M 3s 2024-07-31T14:53:03.612 INFO:tasks.workunit.client.0.smithi067.stderr: 52000K .......... .......... .......... .......... .......... 38% 19.9M 3s 2024-07-31T14:53:03.613 INFO:tasks.workunit.client.0.smithi067.stderr: 52050K .......... .......... .......... .......... .......... 38% 40.0M 3s 2024-07-31T14:53:03.615 INFO:tasks.workunit.client.0.smithi067.stderr: 52100K .......... .......... .......... .......... .......... 38% 29.9M 3s 2024-07-31T14:53:03.617 INFO:tasks.workunit.client.0.smithi067.stderr: 52150K .......... .......... .......... .......... .......... 38% 33.5M 3s 2024-07-31T14:53:03.619 INFO:tasks.workunit.client.0.smithi067.stderr: 52200K .......... .......... .......... .......... .......... 38% 23.6M 3s 2024-07-31T14:53:03.621 INFO:tasks.workunit.client.0.smithi067.stderr: 52250K .......... .......... .......... .......... .......... 38% 34.8M 3s 2024-07-31T14:53:03.622 INFO:tasks.workunit.client.0.smithi067.stderr: 52300K .......... .......... .......... .......... .......... 38% 23.6M 3s 2024-07-31T14:53:03.624 INFO:tasks.workunit.client.0.smithi067.stderr: 52350K .......... .......... .......... .......... .......... 38% 23.4M 3s 2024-07-31T14:53:03.625 INFO:tasks.workunit.client.0.smithi067.stderr: 52400K .......... .......... .......... .......... .......... 38% 55.1M 3s 2024-07-31T14:53:03.627 INFO:tasks.workunit.client.0.smithi067.stderr: 52450K .......... .......... .......... .......... .......... 38% 24.4M 3s 2024-07-31T14:53:03.629 INFO:tasks.workunit.client.0.smithi067.stderr: 52500K .......... .......... .......... .......... .......... 38% 23.3M 3s 2024-07-31T14:53:03.630 INFO:tasks.workunit.client.0.smithi067.stderr: 52550K .......... .......... .......... .......... .......... 38% 41.3M 3s 2024-07-31T14:53:03.633 INFO:tasks.workunit.client.0.smithi067.stderr: 52600K .......... .......... .......... .......... .......... 38% 21.3M 3s 2024-07-31T14:53:03.634 INFO:tasks.workunit.client.0.smithi067.stderr: 52650K .......... .......... .......... .......... .......... 38% 51.3M 3s 2024-07-31T14:53:03.636 INFO:tasks.workunit.client.0.smithi067.stderr: 52700K .......... .......... .......... .......... .......... 38% 25.6M 3s 2024-07-31T14:53:03.638 INFO:tasks.workunit.client.0.smithi067.stderr: 52750K .......... .......... .......... .......... .......... 38% 23.6M 3s 2024-07-31T14:53:03.639 INFO:tasks.workunit.client.0.smithi067.stderr: 52800K .......... .......... .......... .......... .......... 38% 39.8M 3s 2024-07-31T14:53:03.641 INFO:tasks.workunit.client.0.smithi067.stderr: 52850K .......... .......... .......... .......... .......... 38% 23.4M 3s 2024-07-31T14:53:03.642 INFO:tasks.workunit.client.0.smithi067.stderr: 52900K .......... .......... .......... .......... .......... 39% 47.2M 3s 2024-07-31T14:53:03.644 INFO:tasks.workunit.client.0.smithi067.stderr: 52950K .......... .......... .......... .......... .......... 39% 21.6M 3s 2024-07-31T14:53:03.646 INFO:tasks.workunit.client.0.smithi067.stderr: 53000K .......... .......... .......... .......... .......... 39% 24.3M 3s 2024-07-31T14:53:03.648 INFO:tasks.workunit.client.0.smithi067.stderr: 53050K .......... .......... .......... .......... .......... 39% 37.5M 3s 2024-07-31T14:53:03.649 INFO:tasks.workunit.client.0.smithi067.stderr: 53100K .......... .......... .......... .......... .......... 39% 24.5M 3s 2024-07-31T14:53:03.652 INFO:tasks.workunit.client.0.smithi067.stderr: 53150K .......... .......... .......... .......... .......... 39% 47.6M 3s 2024-07-31T14:53:03.653 INFO:tasks.workunit.client.0.smithi067.stderr: 53200K .......... .......... .......... .......... .......... 39% 24.3M 3s 2024-07-31T14:53:03.655 INFO:tasks.workunit.client.0.smithi067.stderr: 53250K .......... .......... .......... .......... .......... 39% 25.4M 3s 2024-07-31T14:53:03.656 INFO:tasks.workunit.client.0.smithi067.stderr: 53300K .......... .......... .......... .......... .......... 39% 35.1M 3s 2024-07-31T14:53:03.659 INFO:tasks.workunit.client.0.smithi067.stderr: 53350K .......... .......... .......... .......... .......... 39% 26.4M 3s 2024-07-31T14:53:03.660 INFO:tasks.workunit.client.0.smithi067.stderr: 53400K .......... .......... .......... .......... .......... 39% 22.1M 3s 2024-07-31T14:53:03.661 INFO:tasks.workunit.client.0.smithi067.stderr: 53450K .......... .......... .......... .......... .......... 39% 43.3M 3s 2024-07-31T14:53:03.663 INFO:tasks.workunit.client.0.smithi067.stderr: 53500K .......... .......... .......... .......... .......... 39% 25.0M 3s 2024-07-31T14:53:03.664 INFO:tasks.workunit.client.0.smithi067.stderr: 53550K .......... .......... .......... .......... .......... 39% 39.6M 3s 2024-07-31T14:53:03.666 INFO:tasks.workunit.client.0.smithi067.stderr: 53600K .......... .......... .......... .......... .......... 39% 23.6M 3s 2024-07-31T14:53:03.668 INFO:tasks.workunit.client.0.smithi067.stderr: 53650K .......... .......... .......... .......... .......... 39% 24.1M 3s 2024-07-31T14:53:03.669 INFO:tasks.workunit.client.0.smithi067.stderr: 53700K .......... .......... .......... .......... .......... 39% 48.8M 3s 2024-07-31T14:53:03.671 INFO:tasks.workunit.client.0.smithi067.stderr: 53750K .......... .......... .......... .......... .......... 39% 24.6M 3s 2024-07-31T14:53:03.673 INFO:tasks.workunit.client.0.smithi067.stderr: 53800K .......... .......... .......... .......... .......... 39% 36.0M 3s 2024-07-31T14:53:03.675 INFO:tasks.workunit.client.0.smithi067.stderr: 53850K .......... .......... .......... .......... .......... 39% 23.2M 3s 2024-07-31T14:53:03.677 INFO:tasks.workunit.client.0.smithi067.stderr: 53900K .......... .......... .......... .......... .......... 39% 24.7M 3s 2024-07-31T14:53:03.678 INFO:tasks.workunit.client.0.smithi067.stderr: 53950K .......... .......... .......... .......... .......... 39% 57.6M 3s 2024-07-31T14:53:03.680 INFO:tasks.workunit.client.0.smithi067.stderr: 54000K .......... .......... .......... .......... .......... 39% 23.0M 3s 2024-07-31T14:53:03.681 INFO:tasks.workunit.client.0.smithi067.stderr: 54050K .......... .......... .......... .......... .......... 39% 43.3M 3s 2024-07-31T14:53:03.683 INFO:tasks.workunit.client.0.smithi067.stderr: 54100K .......... .......... .......... .......... .......... 39% 21.6M 3s 2024-07-31T14:53:03.685 INFO:tasks.workunit.client.0.smithi067.stderr: 54150K .......... .......... .......... .......... .......... 39% 23.3M 3s 2024-07-31T14:53:03.686 INFO:tasks.workunit.client.0.smithi067.stderr: 54200K .......... .......... .......... .......... .......... 39% 55.8M 3s 2024-07-31T14:53:03.688 INFO:tasks.workunit.client.0.smithi067.stderr: 54250K .......... .......... .......... .......... .......... 39% 23.4M 3s 2024-07-31T14:53:03.689 INFO:tasks.workunit.client.0.smithi067.stderr: 54300K .......... .......... .......... .......... .......... 40% 37.6M 3s 2024-07-31T14:53:03.691 INFO:tasks.workunit.client.0.smithi067.stderr: 54350K .......... .......... .......... .......... .......... 40% 24.5M 3s 2024-07-31T14:53:03.694 INFO:tasks.workunit.client.0.smithi067.stderr: 54400K .......... .......... .......... .......... .......... 40% 23.2M 3s 2024-07-31T14:53:03.695 INFO:tasks.workunit.client.0.smithi067.stderr: 54450K .......... .......... .......... .......... .......... 40% 55.8M 3s 2024-07-31T14:53:03.696 INFO:tasks.workunit.client.0.smithi067.stderr: 54500K .......... .......... .......... .......... .......... 40% 28.8M 3s 2024-07-31T14:53:03.697 INFO:tasks.workunit.client.0.smithi067.stderr: 54550K .......... .......... .......... .......... .......... 40% 35.6M 3s 2024-07-31T14:53:03.700 INFO:tasks.workunit.client.0.smithi067.stderr: 54600K .......... .......... .......... .......... .......... 40% 21.0M 3s 2024-07-31T14:53:03.702 INFO:tasks.workunit.client.0.smithi067.stderr: 54650K .......... .......... .......... .......... .......... 40% 23.3M 3s 2024-07-31T14:53:03.703 INFO:tasks.workunit.client.0.smithi067.stderr: 54700K .......... .......... .......... .......... .......... 40% 57.8M 3s 2024-07-31T14:53:03.704 INFO:tasks.workunit.client.0.smithi067.stderr: 54750K .......... .......... .......... .......... .......... 40% 34.6M 3s 2024-07-31T14:53:03.706 INFO:tasks.workunit.client.0.smithi067.stderr: 54800K .......... .......... .......... .......... .......... 40% 25.1M 3s 2024-07-31T14:53:03.708 INFO:tasks.workunit.client.0.smithi067.stderr: 54850K .......... .......... .......... .......... .......... 40% 23.5M 3s 2024-07-31T14:53:03.711 INFO:tasks.workunit.client.0.smithi067.stderr: 54900K .......... .......... .......... .......... .......... 40% 24.2M 3s 2024-07-31T14:53:03.711 INFO:tasks.workunit.client.0.smithi067.stderr: 54950K .......... .......... .......... .......... .......... 40% 48.6M 3s 2024-07-31T14:53:03.713 INFO:tasks.workunit.client.0.smithi067.stderr: 55000K .......... .......... .......... .......... .......... 40% 23.0M 3s 2024-07-31T14:53:03.715 INFO:tasks.workunit.client.0.smithi067.stderr: 55050K .......... .......... .......... .......... .......... 40% 35.3M 3s 2024-07-31T14:53:03.717 INFO:tasks.workunit.client.0.smithi067.stderr: 55100K .......... .......... .......... .......... .......... 40% 24.3M 3s 2024-07-31T14:53:03.719 INFO:tasks.workunit.client.0.smithi067.stderr: 55150K .......... .......... .......... .......... .......... 40% 24.7M 3s 2024-07-31T14:53:03.720 INFO:tasks.workunit.client.0.smithi067.stderr: 55200K .......... .......... .......... .......... .......... 40% 46.0M 3s 2024-07-31T14:53:03.722 INFO:tasks.workunit.client.0.smithi067.stderr: 55250K .......... .......... .......... .......... .......... 40% 25.5M 3s 2024-07-31T14:53:03.723 INFO:tasks.workunit.client.0.smithi067.stderr: 55300K .......... .......... .......... .......... .......... 40% 40.2M 3s 2024-07-31T14:53:03.725 INFO:tasks.workunit.client.0.smithi067.stderr: 55350K .......... .......... .......... .......... .......... 40% 21.4M 3s 2024-07-31T14:53:03.727 INFO:tasks.workunit.client.0.smithi067.stderr: 55400K .......... .......... .......... .......... .......... 40% 25.2M 3s 2024-07-31T14:53:03.728 INFO:tasks.workunit.client.0.smithi067.stderr: 55450K .......... .......... .......... .......... .......... 40% 52.2M 3s 2024-07-31T14:53:03.730 INFO:tasks.workunit.client.0.smithi067.stderr: 55500K .......... .......... .......... .......... .......... 40% 23.6M 3s 2024-07-31T14:53:03.732 INFO:tasks.workunit.client.0.smithi067.stderr: 55550K .......... .......... .......... .......... .......... 40% 40.2M 3s 2024-07-31T14:53:03.734 INFO:tasks.workunit.client.0.smithi067.stderr: 55600K .......... .......... .......... .......... .......... 40% 23.2M 3s 2024-07-31T14:53:03.736 INFO:tasks.workunit.client.0.smithi067.stderr: 55650K .......... .......... .......... .......... .......... 41% 23.4M 3s 2024-07-31T14:53:03.736 INFO:tasks.workunit.client.0.smithi067.stderr: 55700K .......... .......... .......... .......... .......... 41% 56.0M 3s 2024-07-31T14:53:03.738 INFO:tasks.workunit.client.0.smithi067.stderr: 55750K .......... .......... .......... .......... .......... 41% 24.1M 3s 2024-07-31T14:53:03.740 INFO:tasks.workunit.client.0.smithi067.stderr: 55800K .......... .......... .......... .......... .......... 41% 37.7M 3s 2024-07-31T14:53:03.742 INFO:tasks.workunit.client.0.smithi067.stderr: 55850K .......... .......... .......... .......... .......... 41% 24.0M 3s 2024-07-31T14:53:03.745 INFO:tasks.workunit.client.0.smithi067.stderr: 55900K .......... .......... .......... .......... .......... 41% 24.0M 3s 2024-07-31T14:53:03.745 INFO:tasks.workunit.client.0.smithi067.stderr: 55950K .......... .......... .......... .......... .......... 41% 60.8M 3s 2024-07-31T14:53:03.747 INFO:tasks.workunit.client.0.smithi067.stderr: 56000K .......... .......... .......... .......... .......... 41% 24.2M 3s 2024-07-31T14:53:03.751 INFO:tasks.workunit.client.0.smithi067.stderr: 56050K .......... .......... .......... .......... .......... 41% 30.1M 3s 2024-07-31T14:53:03.751 INFO:tasks.workunit.client.0.smithi067.stderr: 56100K .......... .......... .......... .......... .......... 41% 25.2M 3s 2024-07-31T14:53:03.752 INFO:tasks.workunit.client.0.smithi067.stderr: 56150K .......... .......... .......... .......... .......... 41% 24.4M 3s 2024-07-31T14:53:03.753 INFO:tasks.workunit.client.0.smithi067.stderr: 56200K .......... .......... .......... .......... .......... 41% 54.3M 3s 2024-07-31T14:53:03.755 INFO:tasks.workunit.client.0.smithi067.stderr: 56250K .......... .......... .......... .......... .......... 41% 22.0M 3s 2024-07-31T14:53:03.756 INFO:tasks.workunit.client.0.smithi067.stderr: 56300K .......... .......... .......... .......... .......... 41% 40.5M 3s 2024-07-31T14:53:03.759 INFO:tasks.workunit.client.0.smithi067.stderr: 56350K .......... .......... .......... .......... .......... 41% 22.8M 3s 2024-07-31T14:53:03.761 INFO:tasks.workunit.client.0.smithi067.stderr: 56400K .......... .......... .......... .......... .......... 41% 25.1M 3s 2024-07-31T14:53:03.762 INFO:tasks.workunit.client.0.smithi067.stderr: 56450K .......... .......... .......... .......... .......... 41% 56.0M 3s 2024-07-31T14:53:03.764 INFO:tasks.workunit.client.0.smithi067.stderr: 56500K .......... .......... .......... .......... .......... 41% 23.9M 3s 2024-07-31T14:53:03.765 INFO:tasks.workunit.client.0.smithi067.stderr: 56550K .......... .......... .......... .......... .......... 41% 35.7M 3s 2024-07-31T14:53:03.767 INFO:tasks.workunit.client.0.smithi067.stderr: 56600K .......... .......... .......... .......... .......... 41% 24.7M 3s 2024-07-31T14:53:03.769 INFO:tasks.workunit.client.0.smithi067.stderr: 56650K .......... .......... .......... .......... .......... 41% 23.0M 3s 2024-07-31T14:53:03.770 INFO:tasks.workunit.client.0.smithi067.stderr: 56700K .......... .......... .......... .......... .......... 41% 57.8M 3s 2024-07-31T14:53:03.772 INFO:tasks.workunit.client.0.smithi067.stderr: 56750K .......... .......... .......... .......... .......... 41% 22.0M 3s 2024-07-31T14:53:03.774 INFO:tasks.workunit.client.0.smithi067.stderr: 56800K .......... .......... .......... .......... .......... 41% 37.2M 3s 2024-07-31T14:53:03.775 INFO:tasks.workunit.client.0.smithi067.stderr: 56850K .......... .......... .......... .......... .......... 41% 24.2M 3s 2024-07-31T14:53:03.777 INFO:tasks.workunit.client.0.smithi067.stderr: 56900K .......... .......... .......... .......... .......... 41% 24.0M 3s 2024-07-31T14:53:03.778 INFO:tasks.workunit.client.0.smithi067.stderr: 56950K .......... .......... .......... .......... .......... 41% 59.7M 3s 2024-07-31T14:53:03.780 INFO:tasks.workunit.client.0.smithi067.stderr: 57000K .......... .......... .......... .......... .......... 42% 24.1M 3s 2024-07-31T14:53:03.782 INFO:tasks.workunit.client.0.smithi067.stderr: 57050K .......... .......... .......... .......... .......... 42% 33.0M 3s 2024-07-31T14:53:03.784 INFO:tasks.workunit.client.0.smithi067.stderr: 57100K .......... .......... .......... .......... .......... 42% 24.9M 3s 2024-07-31T14:53:03.786 INFO:tasks.workunit.client.0.smithi067.stderr: 57150K .......... .......... .......... .......... .......... 42% 22.2M 3s 2024-07-31T14:53:03.787 INFO:tasks.workunit.client.0.smithi067.stderr: 57200K .......... .......... .......... .......... .......... 42% 53.4M 3s 2024-07-31T14:53:03.789 INFO:tasks.workunit.client.0.smithi067.stderr: 57250K .......... .......... .......... .......... .......... 42% 24.8M 3s 2024-07-31T14:53:03.791 INFO:tasks.workunit.client.0.smithi067.stderr: 57300K .......... .......... .......... .......... .......... 42% 37.3M 3s 2024-07-31T14:53:03.792 INFO:tasks.workunit.client.0.smithi067.stderr: 57350K .......... .......... .......... .......... .......... 42% 22.6M 3s 2024-07-31T14:53:03.794 INFO:tasks.workunit.client.0.smithi067.stderr: 57400K .......... .......... .......... .......... .......... 42% 25.2M 3s 2024-07-31T14:53:03.795 INFO:tasks.workunit.client.0.smithi067.stderr: 57450K .......... .......... .......... .......... .......... 42% 47.5M 3s 2024-07-31T14:53:03.797 INFO:tasks.workunit.client.0.smithi067.stderr: 57500K .......... .......... .......... .......... .......... 42% 24.4M 3s 2024-07-31T14:53:03.799 INFO:tasks.workunit.client.0.smithi067.stderr: 57550K .......... .......... .......... .......... .......... 42% 37.1M 3s 2024-07-31T14:53:03.801 INFO:tasks.workunit.client.0.smithi067.stderr: 57600K .......... .......... .......... .......... .......... 42% 26.2M 3s 2024-07-31T14:53:03.802 INFO:tasks.workunit.client.0.smithi067.stderr: 57650K .......... .......... .......... .......... .......... 42% 30.3M 3s 2024-07-31T14:53:03.804 INFO:tasks.workunit.client.0.smithi067.stderr: 57700K .......... .......... .......... .......... .......... 42% 34.8M 3s 2024-07-31T14:53:03.805 INFO:tasks.workunit.client.0.smithi067.stderr: 57750K .......... .......... .......... .......... .......... 42% 27.4M 3s 2024-07-31T14:53:03.808 INFO:tasks.workunit.client.0.smithi067.stderr: 57800K .......... .......... .......... .......... .......... 42% 19.6M 3s 2024-07-31T14:53:03.810 INFO:tasks.workunit.client.0.smithi067.stderr: 57850K .......... .......... .......... .......... .......... 42% 32.3M 3s 2024-07-31T14:53:03.812 INFO:tasks.workunit.client.0.smithi067.stderr: 57900K .......... .......... .......... .......... .......... 42% 23.3M 3s 2024-07-31T14:53:03.812 INFO:tasks.workunit.client.0.smithi067.stderr: 57950K .......... .......... .......... .......... .......... 42% 77.4M 3s 2024-07-31T14:53:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 58000K .......... .......... .......... .......... .......... 42% 24.1M 3s 2024-07-31T14:53:03.816 INFO:tasks.workunit.client.0.smithi067.stderr: 58050K .......... .......... .......... .......... .......... 42% 23.1M 3s 2024-07-31T14:53:03.818 INFO:tasks.workunit.client.0.smithi067.stderr: 58100K .......... .......... .......... .......... .......... 42% 28.9M 3s 2024-07-31T14:53:03.819 INFO:tasks.workunit.client.0.smithi067.stderr: 58150K .......... .......... .......... .......... .......... 42% 58.1M 3s 2024-07-31T14:53:03.820 INFO:tasks.workunit.client.0.smithi067.stderr: 58200K .......... .......... .......... .......... .......... 42% 28.4M 3s 2024-07-31T14:53:03.822 INFO:tasks.workunit.client.0.smithi067.stderr: 58250K .......... .......... .......... .......... .......... 42% 23.4M 3s 2024-07-31T14:53:03.825 INFO:tasks.workunit.client.0.smithi067.stderr: 58300K .......... .......... .......... .......... .......... 42% 23.8M 3s 2024-07-31T14:53:03.826 INFO:tasks.workunit.client.0.smithi067.stderr: 58350K .......... .......... .......... .......... .......... 43% 35.9M 3s 2024-07-31T14:53:03.828 INFO:tasks.workunit.client.0.smithi067.stderr: 58400K .......... .......... .......... .......... .......... 43% 23.5M 3s 2024-07-31T14:53:03.829 INFO:tasks.workunit.client.0.smithi067.stderr: 58450K .......... .......... .......... .......... .......... 43% 57.4M 3s 2024-07-31T14:53:03.831 INFO:tasks.workunit.client.0.smithi067.stderr: 58500K .......... .......... .......... .......... .......... 43% 24.1M 3s 2024-07-31T14:53:03.833 INFO:tasks.workunit.client.0.smithi067.stderr: 58550K .......... .......... .......... .......... .......... 43% 22.0M 3s 2024-07-31T14:53:03.834 INFO:tasks.workunit.client.0.smithi067.stderr: 58600K .......... .......... .......... .......... .......... 43% 42.7M 3s 2024-07-31T14:53:03.836 INFO:tasks.workunit.client.0.smithi067.stderr: 58650K .......... .......... .......... .......... .......... 43% 23.4M 3s 2024-07-31T14:53:03.838 INFO:tasks.workunit.client.0.smithi067.stderr: 58700K .......... .......... .......... .......... .......... 43% 36.7M 3s 2024-07-31T14:53:03.839 INFO:tasks.workunit.client.0.smithi067.stderr: 58750K .......... .......... .......... .......... .......... 43% 34.0M 3s 2024-07-31T14:53:03.841 INFO:tasks.workunit.client.0.smithi067.stderr: 58800K .......... .......... .......... .......... .......... 43% 21.4M 3s 2024-07-31T14:53:03.843 INFO:tasks.workunit.client.0.smithi067.stderr: 58850K .......... .......... .......... .......... .......... 43% 34.0M 3s 2024-07-31T14:53:03.845 INFO:tasks.workunit.client.0.smithi067.stderr: 58900K .......... .......... .......... .......... .......... 43% 23.0M 3s 2024-07-31T14:53:03.847 INFO:tasks.workunit.client.0.smithi067.stderr: 58950K .......... .......... .......... .......... .......... 43% 25.6M 3s 2024-07-31T14:53:03.848 INFO:tasks.workunit.client.0.smithi067.stderr: 59000K .......... .......... .......... .......... .......... 43% 51.2M 3s 2024-07-31T14:53:03.850 INFO:tasks.workunit.client.0.smithi067.stderr: 59050K .......... .......... .......... .......... .......... 43% 23.9M 3s 2024-07-31T14:53:03.852 INFO:tasks.workunit.client.0.smithi067.stderr: 59100K .......... .......... .......... .......... .......... 43% 27.1M 3s 2024-07-31T14:53:03.854 INFO:tasks.workunit.client.0.smithi067.stderr: 59150K .......... .......... .......... .......... .......... 43% 30.4M 3s 2024-07-31T14:53:03.855 INFO:tasks.workunit.client.0.smithi067.stderr: 59200K .......... .......... .......... .......... .......... 43% 37.6M 3s 2024-07-31T14:53:03.856 INFO:tasks.workunit.client.0.smithi067.stderr: 59250K .......... .......... .......... .......... .......... 43% 36.9M 3s 2024-07-31T14:53:03.858 INFO:tasks.workunit.client.0.smithi067.stderr: 59300K .......... .......... .......... .......... .......... 43% 20.0M 3s 2024-07-31T14:53:03.859 INFO:tasks.workunit.client.0.smithi067.stderr: 59350K .......... .......... .......... .......... .......... 43% 38.6M 3s 2024-07-31T14:53:03.861 INFO:tasks.workunit.client.0.smithi067.stderr: 59400K .......... .......... .......... .......... .......... 43% 27.1M 3s 2024-07-31T14:53:03.864 INFO:tasks.workunit.client.0.smithi067.stderr: 59450K .......... .......... .......... .......... .......... 43% 21.4M 3s 2024-07-31T14:53:03.864 INFO:tasks.workunit.client.0.smithi067.stderr: 59500K .......... .......... .......... .......... .......... 43% 48.4M 3s 2024-07-31T14:53:03.867 INFO:tasks.workunit.client.0.smithi067.stderr: 59550K .......... .......... .......... .......... .......... 43% 24.6M 3s 2024-07-31T14:53:03.868 INFO:tasks.workunit.client.0.smithi067.stderr: 59600K .......... .......... .......... .......... .......... 43% 41.7M 3s 2024-07-31T14:53:03.870 INFO:tasks.workunit.client.0.smithi067.stderr: 59650K .......... .......... .......... .......... .......... 43% 23.8M 3s 2024-07-31T14:53:03.872 INFO:tasks.workunit.client.0.smithi067.stderr: 59700K .......... .......... .......... .......... .......... 44% 23.3M 3s 2024-07-31T14:53:03.873 INFO:tasks.workunit.client.0.smithi067.stderr: 59750K .......... .......... .......... .......... .......... 44% 53.3M 3s 2024-07-31T14:53:03.875 INFO:tasks.workunit.client.0.smithi067.stderr: 59800K .......... .......... .......... .......... .......... 44% 24.4M 3s 2024-07-31T14:53:03.876 INFO:tasks.workunit.client.0.smithi067.stderr: 59850K .......... .......... .......... .......... .......... 44% 36.2M 3s 2024-07-31T14:53:03.878 INFO:tasks.workunit.client.0.smithi067.stderr: 59900K .......... .......... .......... .......... .......... 44% 24.1M 3s 2024-07-31T14:53:03.880 INFO:tasks.workunit.client.0.smithi067.stderr: 59950K .......... .......... .......... .......... .......... 44% 23.5M 3s 2024-07-31T14:53:03.881 INFO:tasks.workunit.client.0.smithi067.stderr: 60000K .......... .......... .......... .......... .......... 44% 49.7M 3s 2024-07-31T14:53:03.883 INFO:tasks.workunit.client.0.smithi067.stderr: 60050K .......... .......... .......... .......... .......... 44% 25.4M 3s 2024-07-31T14:53:03.885 INFO:tasks.workunit.client.0.smithi067.stderr: 60100K .......... .......... .......... .......... .......... 44% 22.8M 3s 2024-07-31T14:53:03.886 INFO:tasks.workunit.client.0.smithi067.stderr: 60150K .......... .......... .......... .......... .......... 44% 46.1M 3s 2024-07-31T14:53:03.889 INFO:tasks.workunit.client.0.smithi067.stderr: 60200K .......... .......... .......... .......... .......... 44% 20.0M 3s 2024-07-31T14:53:03.890 INFO:tasks.workunit.client.0.smithi067.stderr: 60250K .......... .......... .......... .......... .......... 44% 53.5M 3s 2024-07-31T14:53:03.892 INFO:tasks.workunit.client.0.smithi067.stderr: 60300K .......... .......... .......... .......... .......... 44% 23.8M 3s 2024-07-31T14:53:03.893 INFO:tasks.workunit.client.0.smithi067.stderr: 60350K .......... .......... .......... .......... .......... 44% 34.1M 3s 2024-07-31T14:53:03.895 INFO:tasks.workunit.client.0.smithi067.stderr: 60400K .......... .......... .......... .......... .......... 44% 23.9M 3s 2024-07-31T14:53:03.897 INFO:tasks.workunit.client.0.smithi067.stderr: 60450K .......... .......... .......... .......... .......... 44% 24.6M 3s 2024-07-31T14:53:03.898 INFO:tasks.workunit.client.0.smithi067.stderr: 60500K .......... .......... .......... .......... .......... 44% 53.0M 3s 2024-07-31T14:53:03.900 INFO:tasks.workunit.client.0.smithi067.stderr: 60550K .......... .......... .......... .......... .......... 44% 22.9M 3s 2024-07-31T14:53:03.902 INFO:tasks.workunit.client.0.smithi067.stderr: 60600K .......... .......... .......... .......... .......... 44% 36.4M 3s 2024-07-31T14:53:03.904 INFO:tasks.workunit.client.0.smithi067.stderr: 60650K .......... .......... .......... .......... .......... 44% 23.4M 3s 2024-07-31T14:53:03.906 INFO:tasks.workunit.client.0.smithi067.stderr: 60700K .......... .......... .......... .......... .......... 44% 25.1M 3s 2024-07-31T14:53:03.907 INFO:tasks.workunit.client.0.smithi067.stderr: 60750K .......... .......... .......... .......... .......... 44% 51.0M 3s 2024-07-31T14:53:03.909 INFO:tasks.workunit.client.0.smithi067.stderr: 60800K .......... .......... .......... .......... .......... 44% 24.0M 3s 2024-07-31T14:53:03.910 INFO:tasks.workunit.client.0.smithi067.stderr: 60850K .......... .......... .......... .......... .......... 44% 40.1M 3s 2024-07-31T14:53:03.912 INFO:tasks.workunit.client.0.smithi067.stderr: 60900K .......... .......... .......... .......... .......... 44% 22.8M 3s 2024-07-31T14:53:03.914 INFO:tasks.workunit.client.0.smithi067.stderr: 60950K .......... .......... .......... .......... .......... 44% 24.9M 3s 2024-07-31T14:53:03.915 INFO:tasks.workunit.client.0.smithi067.stderr: 61000K .......... .......... .......... .......... .......... 44% 51.7M 3s 2024-07-31T14:53:03.917 INFO:tasks.workunit.client.0.smithi067.stderr: 61050K .......... .......... .......... .......... .......... 45% 22.8M 3s 2024-07-31T14:53:03.918 INFO:tasks.workunit.client.0.smithi067.stderr: 61100K .......... .......... .......... .......... .......... 45% 40.2M 3s 2024-07-31T14:53:03.921 INFO:tasks.workunit.client.0.smithi067.stderr: 61150K .......... .......... .......... .......... .......... 45% 22.5M 3s 2024-07-31T14:53:03.923 INFO:tasks.workunit.client.0.smithi067.stderr: 61200K .......... .......... .......... .......... .......... 45% 22.5M 3s 2024-07-31T14:53:03.924 INFO:tasks.workunit.client.0.smithi067.stderr: 61250K .......... .......... .......... .......... .......... 45% 48.2M 3s 2024-07-31T14:53:03.926 INFO:tasks.workunit.client.0.smithi067.stderr: 61300K .......... .......... .......... .......... .......... 45% 22.9M 3s 2024-07-31T14:53:03.927 INFO:tasks.workunit.client.0.smithi067.stderr: 61350K .......... .......... .......... .......... .......... 45% 44.8M 3s 2024-07-31T14:53:03.929 INFO:tasks.workunit.client.0.smithi067.stderr: 61400K .......... .......... .......... .......... .......... 45% 22.2M 3s 2024-07-31T14:53:03.931 INFO:tasks.workunit.client.0.smithi067.stderr: 61450K .......... .......... .......... .......... .......... 45% 25.0M 3s 2024-07-31T14:53:03.932 INFO:tasks.workunit.client.0.smithi067.stderr: 61500K .......... .......... .......... .......... .......... 45% 53.5M 3s 2024-07-31T14:53:03.934 INFO:tasks.workunit.client.0.smithi067.stderr: 61550K .......... .......... .......... .......... .......... 45% 22.4M 3s 2024-07-31T14:53:03.936 INFO:tasks.workunit.client.0.smithi067.stderr: 61600K .......... .......... .......... .......... .......... 45% 37.9M 3s 2024-07-31T14:53:03.937 INFO:tasks.workunit.client.0.smithi067.stderr: 61650K .......... .......... .......... .......... .......... 45% 23.7M 3s 2024-07-31T14:53:03.939 INFO:tasks.workunit.client.0.smithi067.stderr: 61700K .......... .......... .......... .......... .......... 45% 25.3M 3s 2024-07-31T14:53:03.940 INFO:tasks.workunit.client.0.smithi067.stderr: 61750K .......... .......... .......... .......... .......... 45% 49.8M 3s 2024-07-31T14:53:03.942 INFO:tasks.workunit.client.0.smithi067.stderr: 61800K .......... .......... .......... .......... .......... 45% 23.6M 3s 2024-07-31T14:53:03.944 INFO:tasks.workunit.client.0.smithi067.stderr: 61850K .......... .......... .......... .......... .......... 45% 34.4M 3s 2024-07-31T14:53:03.946 INFO:tasks.workunit.client.0.smithi067.stderr: 61900K .......... .......... .......... .......... .......... 45% 25.1M 3s 2024-07-31T14:53:03.949 INFO:tasks.workunit.client.0.smithi067.stderr: 61950K .......... .......... .......... .......... .......... 45% 23.9M 3s 2024-07-31T14:53:03.949 INFO:tasks.workunit.client.0.smithi067.stderr: 62000K .......... .......... .......... .......... .......... 45% 60.5M 3s 2024-07-31T14:53:03.953 INFO:tasks.workunit.client.0.smithi067.stderr: 62050K .......... .......... .......... .......... .......... 45% 21.7M 3s 2024-07-31T14:53:03.955 INFO:tasks.workunit.client.0.smithi067.stderr: 62100K .......... .......... .......... .......... .......... 45% 39.6M 3s 2024-07-31T14:53:03.955 INFO:tasks.workunit.client.0.smithi067.stderr: 62150K .......... .......... .......... .......... .......... 45% 23.2M 3s 2024-07-31T14:53:03.956 INFO:tasks.workunit.client.0.smithi067.stderr: 62200K .......... .......... .......... .......... .......... 45% 24.5M 3s 2024-07-31T14:53:03.957 INFO:tasks.workunit.client.0.smithi067.stderr: 62250K .......... .......... .......... .......... .......... 45% 55.7M 3s 2024-07-31T14:53:03.959 INFO:tasks.workunit.client.0.smithi067.stderr: 62300K .......... .......... .......... .......... .......... 45% 23.5M 3s 2024-07-31T14:53:03.960 INFO:tasks.workunit.client.0.smithi067.stderr: 62350K .......... .......... .......... .......... .......... 45% 36.7M 3s 2024-07-31T14:53:03.963 INFO:tasks.workunit.client.0.smithi067.stderr: 62400K .......... .......... .......... .......... .......... 46% 24.4M 3s 2024-07-31T14:53:03.965 INFO:tasks.workunit.client.0.smithi067.stderr: 62450K .......... .......... .......... .......... .......... 46% 23.5M 3s 2024-07-31T14:53:03.965 INFO:tasks.workunit.client.0.smithi067.stderr: 62500K .......... .......... .......... .......... .......... 46% 51.2M 3s 2024-07-31T14:53:03.967 INFO:tasks.workunit.client.0.smithi067.stderr: 62550K .......... .......... .......... .......... .......... 46% 23.9M 3s 2024-07-31T14:53:03.969 INFO:tasks.workunit.client.0.smithi067.stderr: 62600K .......... .......... .......... .......... .......... 46% 32.2M 3s 2024-07-31T14:53:03.971 INFO:tasks.workunit.client.0.smithi067.stderr: 62650K .......... .......... .......... .......... .......... 46% 27.3M 3s 2024-07-31T14:53:03.973 INFO:tasks.workunit.client.0.smithi067.stderr: 62700K .......... .......... .......... .......... .......... 46% 23.7M 3s 2024-07-31T14:53:03.974 INFO:tasks.workunit.client.0.smithi067.stderr: 62750K .......... .......... .......... .......... .......... 46% 50.1M 3s 2024-07-31T14:53:03.976 INFO:tasks.workunit.client.0.smithi067.stderr: 62800K .......... .......... .......... .......... .......... 46% 24.0M 3s 2024-07-31T14:53:03.977 INFO:tasks.workunit.client.0.smithi067.stderr: 62850K .......... .......... .......... .......... .......... 46% 38.0M 3s 2024-07-31T14:53:03.979 INFO:tasks.workunit.client.0.smithi067.stderr: 62900K .......... .......... .......... .......... .......... 46% 22.6M 3s 2024-07-31T14:53:03.982 INFO:tasks.workunit.client.0.smithi067.stderr: 62950K .......... .......... .......... .......... .......... 46% 23.8M 3s 2024-07-31T14:53:03.982 INFO:tasks.workunit.client.0.smithi067.stderr: 63000K .......... .......... .......... .......... .......... 46% 57.2M 3s 2024-07-31T14:53:03.984 INFO:tasks.workunit.client.0.smithi067.stderr: 63050K .......... .......... .......... .......... .......... 46% 24.1M 3s 2024-07-31T14:53:03.987 INFO:tasks.workunit.client.0.smithi067.stderr: 63100K .......... .......... .......... .......... .......... 46% 23.1M 3s 2024-07-31T14:53:03.988 INFO:tasks.workunit.client.0.smithi067.stderr: 63150K .......... .......... .......... .......... .......... 46% 40.4M 3s 2024-07-31T14:53:03.990 INFO:tasks.workunit.client.0.smithi067.stderr: 63200K .......... .......... .......... .......... .......... 46% 22.7M 3s 2024-07-31T14:53:03.990 INFO:tasks.workunit.client.0.smithi067.stderr: 63250K .......... .......... .......... .......... .......... 46% 65.4M 3s 2024-07-31T14:53:03.993 INFO:tasks.workunit.client.0.smithi067.stderr: 63300K .......... .......... .......... .......... .......... 46% 21.7M 3s 2024-07-31T14:53:03.995 INFO:tasks.workunit.client.0.smithi067.stderr: 63350K .......... .......... .......... .......... .......... 46% 25.0M 3s 2024-07-31T14:53:03.996 INFO:tasks.workunit.client.0.smithi067.stderr: 63400K .......... .......... .......... .......... .......... 46% 32.4M 3s 2024-07-31T14:53:03.998 INFO:tasks.workunit.client.0.smithi067.stderr: 63450K .......... .......... .......... .......... .......... 46% 25.0M 3s 2024-07-31T14:53:04.000 INFO:tasks.workunit.client.0.smithi067.stderr: 63500K .......... .......... .......... .......... .......... 46% 57.4M 3s 2024-07-31T14:53:04.001 INFO:tasks.workunit.client.0.smithi067.stderr: 63550K .......... .......... .......... .......... .......... 46% 22.1M 3s 2024-07-31T14:53:04.003 INFO:tasks.workunit.client.0.smithi067.stderr: 63600K .......... .......... .......... .......... .......... 46% 24.6M 3s 2024-07-31T14:53:04.004 INFO:tasks.workunit.client.0.smithi067.stderr: 63650K .......... .......... .......... .......... .......... 46% 37.6M 3s 2024-07-31T14:53:04.007 INFO:tasks.workunit.client.0.smithi067.stderr: 63700K .......... .......... .......... .......... .......... 46% 22.8M 3s 2024-07-31T14:53:04.009 INFO:tasks.workunit.client.0.smithi067.stderr: 63750K .......... .......... .......... .......... .......... 46% 63.3M 3s 2024-07-31T14:53:04.010 INFO:tasks.workunit.client.0.smithi067.stderr: 63800K .......... .......... .......... .......... .......... 47% 21.9M 3s 2024-07-31T14:53:04.012 INFO:tasks.workunit.client.0.smithi067.stderr: 63850K .......... .......... .......... .......... .......... 47% 23.5M 3s 2024-07-31T14:53:04.013 INFO:tasks.workunit.client.0.smithi067.stderr: 63900K .......... .......... .......... .......... .......... 47% 41.5M 3s 2024-07-31T14:53:04.015 INFO:tasks.workunit.client.0.smithi067.stderr: 63950K .......... .......... .......... .......... .......... 47% 22.9M 3s 2024-07-31T14:53:04.017 INFO:tasks.workunit.client.0.smithi067.stderr: 64000K .......... .......... .......... .......... .......... 47% 25.2M 3s 2024-07-31T14:53:04.018 INFO:tasks.workunit.client.0.smithi067.stderr: 64050K .......... .......... .......... .......... .......... 47% 45.9M 3s 2024-07-31T14:53:04.020 INFO:tasks.workunit.client.0.smithi067.stderr: 64100K .......... .......... .......... .......... .......... 47% 25.0M 3s 2024-07-31T14:53:04.022 INFO:tasks.workunit.client.0.smithi067.stderr: 64150K .......... .......... .......... .......... .......... 47% 32.3M 3s 2024-07-31T14:53:04.023 INFO:tasks.workunit.client.0.smithi067.stderr: 64200K .......... .......... .......... .......... .......... 47% 24.9M 3s 2024-07-31T14:53:04.026 INFO:tasks.workunit.client.0.smithi067.stderr: 64250K .......... .......... .......... .......... .......... 47% 23.1M 3s 2024-07-31T14:53:04.027 INFO:tasks.workunit.client.0.smithi067.stderr: 64300K .......... .......... .......... .......... .......... 47% 55.2M 2s 2024-07-31T14:53:04.029 INFO:tasks.workunit.client.0.smithi067.stderr: 64350K .......... .......... .......... .......... .......... 47% 23.9M 2s 2024-07-31T14:53:04.030 INFO:tasks.workunit.client.0.smithi067.stderr: 64400K .......... .......... .......... .......... .......... 47% 37.7M 2s 2024-07-31T14:53:04.032 INFO:tasks.workunit.client.0.smithi067.stderr: 64450K .......... .......... .......... .......... .......... 47% 24.9M 2s 2024-07-31T14:53:04.035 INFO:tasks.workunit.client.0.smithi067.stderr: 64500K .......... .......... .......... .......... .......... 47% 23.5M 2s 2024-07-31T14:53:04.035 INFO:tasks.workunit.client.0.smithi067.stderr: 64550K .......... .......... .......... .......... .......... 47% 47.2M 2s 2024-07-31T14:53:04.037 INFO:tasks.workunit.client.0.smithi067.stderr: 64600K .......... .......... .......... .......... .......... 47% 24.4M 2s 2024-07-31T14:53:04.038 INFO:tasks.workunit.client.0.smithi067.stderr: 64650K .......... .......... .......... .......... .......... 47% 40.3M 2s 2024-07-31T14:53:04.040 INFO:tasks.workunit.client.0.smithi067.stderr: 64700K .......... .......... .......... .......... .......... 47% 23.3M 2s 2024-07-31T14:53:04.043 INFO:tasks.workunit.client.0.smithi067.stderr: 64750K .......... .......... .......... .......... .......... 47% 23.2M 2s 2024-07-31T14:53:04.043 INFO:tasks.workunit.client.0.smithi067.stderr: 64800K .......... .......... .......... .......... .......... 47% 48.4M 2s 2024-07-31T14:53:04.045 INFO:tasks.workunit.client.0.smithi067.stderr: 64850K .......... .......... .......... .......... .......... 47% 24.4M 2s 2024-07-31T14:53:04.047 INFO:tasks.workunit.client.0.smithi067.stderr: 64900K .......... .......... .......... .......... .......... 47% 39.5M 2s 2024-07-31T14:53:04.049 INFO:tasks.workunit.client.0.smithi067.stderr: 64950K .......... .......... .......... .......... .......... 47% 24.5M 2s 2024-07-31T14:53:04.051 INFO:tasks.workunit.client.0.smithi067.stderr: 65000K .......... .......... .......... .......... .......... 47% 24.2M 2s 2024-07-31T14:53:04.051 INFO:tasks.workunit.client.0.smithi067.stderr: 65050K .......... .......... .......... .......... .......... 47% 51.0M 2s 2024-07-31T14:53:04.054 INFO:tasks.workunit.client.0.smithi067.stderr: 65100K .......... .......... .......... .......... .......... 47% 23.6M 2s 2024-07-31T14:53:04.055 INFO:tasks.workunit.client.0.smithi067.stderr: 65150K .......... .......... .......... .......... .......... 48% 35.3M 2s 2024-07-31T14:53:04.057 INFO:tasks.workunit.client.0.smithi067.stderr: 65200K .......... .......... .......... .......... .......... 48% 24.1M 2s 2024-07-31T14:53:04.059 INFO:tasks.workunit.client.0.smithi067.stderr: 65250K .......... .......... .......... .......... .......... 48% 23.2M 2s 2024-07-31T14:53:04.060 INFO:tasks.workunit.client.0.smithi067.stderr: 65300K .......... .......... .......... .......... .......... 48% 53.6M 2s 2024-07-31T14:53:04.062 INFO:tasks.workunit.client.0.smithi067.stderr: 65350K .......... .......... .......... .......... .......... 48% 25.0M 2s 2024-07-31T14:53:04.064 INFO:tasks.workunit.client.0.smithi067.stderr: 65400K .......... .......... .......... .......... .......... 48% 23.6M 2s 2024-07-31T14:53:04.065 INFO:tasks.workunit.client.0.smithi067.stderr: 65450K .......... .......... .......... .......... .......... 48% 55.8M 2s 2024-07-31T14:53:04.067 INFO:tasks.workunit.client.0.smithi067.stderr: 65500K .......... .......... .......... .......... .......... 48% 27.2M 2s 2024-07-31T14:53:04.068 INFO:tasks.workunit.client.0.smithi067.stderr: 65550K .......... .......... .......... .......... .......... 48% 37.7M 2s 2024-07-31T14:53:04.070 INFO:tasks.workunit.client.0.smithi067.stderr: 65600K .......... .......... .......... .......... .......... 48% 27.1M 2s 2024-07-31T14:53:04.073 INFO:tasks.workunit.client.0.smithi067.stderr: 65650K .......... .......... .......... .......... .......... 48% 20.1M 2s 2024-07-31T14:53:04.074 INFO:tasks.workunit.client.0.smithi067.stderr: 65700K .......... .......... .......... .......... .......... 48% 34.7M 2s 2024-07-31T14:53:04.075 INFO:tasks.workunit.client.0.smithi067.stderr: 65750K .......... .......... .......... .......... .......... 48% 45.9M 2s 2024-07-31T14:53:04.077 INFO:tasks.workunit.client.0.smithi067.stderr: 65800K .......... .......... .......... .......... .......... 48% 25.4M 2s 2024-07-31T14:53:04.079 INFO:tasks.workunit.client.0.smithi067.stderr: 65850K .......... .......... .......... .......... .......... 48% 23.3M 2s 2024-07-31T14:53:04.080 INFO:tasks.workunit.client.0.smithi067.stderr: 65900K .......... .......... .......... .......... .......... 48% 40.0M 2s 2024-07-31T14:53:04.083 INFO:tasks.workunit.client.0.smithi067.stderr: 65950K .......... .......... .......... .......... .......... 48% 21.1M 2s 2024-07-31T14:53:04.084 INFO:tasks.workunit.client.0.smithi067.stderr: 66000K .......... .......... .......... .......... .......... 48% 33.2M 2s 2024-07-31T14:53:04.085 INFO:tasks.workunit.client.0.smithi067.stderr: 66050K .......... .......... .......... .......... .......... 48% 38.6M 2s 2024-07-31T14:53:04.087 INFO:tasks.workunit.client.0.smithi067.stderr: 66100K .......... .......... .......... .......... .......... 48% 29.1M 2s 2024-07-31T14:53:04.089 INFO:tasks.workunit.client.0.smithi067.stderr: 66150K .......... .......... .......... .......... .......... 48% 20.3M 2s 2024-07-31T14:53:04.091 INFO:tasks.workunit.client.0.smithi067.stderr: 66200K .......... .......... .......... .......... .......... 48% 36.0M 2s 2024-07-31T14:53:04.092 INFO:tasks.workunit.client.0.smithi067.stderr: 66250K .......... .......... .......... .......... .......... 48% 39.6M 2s 2024-07-31T14:53:04.093 INFO:tasks.workunit.client.0.smithi067.stderr: 66300K .......... .......... .......... .......... .......... 48% 30.7M 2s 2024-07-31T14:53:04.096 INFO:tasks.workunit.client.0.smithi067.stderr: 66350K .......... .......... .......... .......... .......... 48% 22.2M 2s 2024-07-31T14:53:04.097 INFO:tasks.workunit.client.0.smithi067.stderr: 66400K .......... .......... .......... .......... .......... 48% 43.4M 2s 2024-07-31T14:53:04.099 INFO:tasks.workunit.client.0.smithi067.stderr: 66450K .......... .......... .......... .......... .......... 48% 22.9M 2s 2024-07-31T14:53:04.100 INFO:tasks.workunit.client.0.smithi067.stderr: 66500K .......... .......... .......... .......... .......... 49% 37.3M 2s 2024-07-31T14:53:04.102 INFO:tasks.workunit.client.0.smithi067.stderr: 66550K .......... .......... .......... .......... .......... 49% 27.2M 2s 2024-07-31T14:53:04.104 INFO:tasks.workunit.client.0.smithi067.stderr: 66600K .......... .......... .......... .......... .......... 49% 24.4M 2s 2024-07-31T14:53:04.105 INFO:tasks.workunit.client.0.smithi067.stderr: 66650K .......... .......... .......... .......... .......... 49% 32.2M 2s 2024-07-31T14:53:04.107 INFO:tasks.workunit.client.0.smithi067.stderr: 66700K .......... .......... .......... .......... .......... 49% 25.2M 2s 2024-07-31T14:53:04.109 INFO:tasks.workunit.client.0.smithi067.stderr: 66750K .......... .......... .......... .......... .......... 49% 30.5M 2s 2024-07-31T14:53:04.110 INFO:tasks.workunit.client.0.smithi067.stderr: 66800K .......... .......... .......... .......... .......... 49% 33.6M 2s 2024-07-31T14:53:04.112 INFO:tasks.workunit.client.0.smithi067.stderr: 66850K .......... .......... .......... .......... .......... 49% 23.2M 2s 2024-07-31T14:53:04.114 INFO:tasks.workunit.client.0.smithi067.stderr: 66900K .......... .......... .......... .......... .......... 49% 44.7M 2s 2024-07-31T14:53:04.116 INFO:tasks.workunit.client.0.smithi067.stderr: 66950K .......... .......... .......... .......... .......... 49% 21.4M 2s 2024-07-31T14:53:04.118 INFO:tasks.workunit.client.0.smithi067.stderr: 67000K .......... .......... .......... .......... .......... 49% 25.9M 2s 2024-07-31T14:53:04.119 INFO:tasks.workunit.client.0.smithi067.stderr: 67050K .......... .......... .......... .......... .......... 49% 50.3M 2s 2024-07-31T14:53:04.121 INFO:tasks.workunit.client.0.smithi067.stderr: 67100K .......... .......... .......... .......... .......... 49% 24.4M 2s 2024-07-31T14:53:04.122 INFO:tasks.workunit.client.0.smithi067.stderr: 67150K .......... .......... .......... .......... .......... 49% 32.5M 2s 2024-07-31T14:53:04.124 INFO:tasks.workunit.client.0.smithi067.stderr: 67200K .......... .......... .......... .......... .......... 49% 25.3M 2s 2024-07-31T14:53:04.126 INFO:tasks.workunit.client.0.smithi067.stderr: 67250K .......... .......... .......... .......... .......... 49% 29.3M 2s 2024-07-31T14:53:04.127 INFO:tasks.workunit.client.0.smithi067.stderr: 67300K .......... .......... .......... .......... .......... 49% 41.9M 2s 2024-07-31T14:53:04.129 INFO:tasks.workunit.client.0.smithi067.stderr: 67350K .......... .......... .......... .......... .......... 49% 22.0M 2s 2024-07-31T14:53:04.131 INFO:tasks.workunit.client.0.smithi067.stderr: 67400K .......... .......... .......... .......... .......... 49% 39.5M 2s 2024-07-31T14:53:04.132 INFO:tasks.workunit.client.0.smithi067.stderr: 67450K .......... .......... .......... .......... .......... 49% 22.9M 2s 2024-07-31T14:53:04.135 INFO:tasks.workunit.client.0.smithi067.stderr: 67500K .......... .......... .......... .......... .......... 49% 22.6M 2s 2024-07-31T14:53:04.135 INFO:tasks.workunit.client.0.smithi067.stderr: 67550K .......... .......... .......... .......... .......... 49% 59.6M 2s 2024-07-31T14:53:04.137 INFO:tasks.workunit.client.0.smithi067.stderr: 67600K .......... .......... .......... .......... .......... 49% 25.8M 2s 2024-07-31T14:53:04.139 INFO:tasks.workunit.client.0.smithi067.stderr: 67650K .......... .......... .......... .......... .......... 49% 32.5M 2s 2024-07-31T14:53:04.142 INFO:tasks.workunit.client.0.smithi067.stderr: 67700K .......... .......... .......... .......... .......... 49% 31.8M 2s 2024-07-31T14:53:04.143 INFO:tasks.workunit.client.0.smithi067.stderr: 67750K .......... .......... .......... .......... .......... 49% 19.6M 2s 2024-07-31T14:53:04.144 INFO:tasks.workunit.client.0.smithi067.stderr: 67800K .......... .......... .......... .......... .......... 49% 43.8M 2s 2024-07-31T14:53:04.146 INFO:tasks.workunit.client.0.smithi067.stderr: 67850K .......... .......... .......... .......... .......... 50% 25.1M 2s 2024-07-31T14:53:04.148 INFO:tasks.workunit.client.0.smithi067.stderr: 67900K .......... .......... .......... .......... .......... 50% 23.4M 2s 2024-07-31T14:53:04.154 INFO:tasks.workunit.client.0.smithi067.stderr: 67950K .......... .......... .......... .......... .......... 50% 10.5M 2s 2024-07-31T14:53:04.154 INFO:tasks.workunit.client.0.smithi067.stderr: 68000K .......... .......... .......... .......... .......... 50% 316M 2s 2024-07-31T14:53:04.154 INFO:tasks.workunit.client.0.smithi067.stderr: 68050K .......... .......... .......... .......... .......... 50% 119M 2s 2024-07-31T14:53:04.155 INFO:tasks.workunit.client.0.smithi067.stderr: 68100K .......... .......... .......... .......... .......... 50% 35.0M 2s 2024-07-31T14:53:04.156 INFO:tasks.workunit.client.0.smithi067.stderr: 68150K .......... .......... .......... .......... .......... 50% 45.5M 2s 2024-07-31T14:53:04.158 INFO:tasks.workunit.client.0.smithi067.stderr: 68200K .......... .......... .......... .......... .......... 50% 24.2M 2s 2024-07-31T14:53:04.160 INFO:tasks.workunit.client.0.smithi067.stderr: 68250K .......... .......... .......... .......... .......... 50% 24.5M 2s 2024-07-31T14:53:04.161 INFO:tasks.workunit.client.0.smithi067.stderr: 68300K .......... .......... .......... .......... .......... 50% 48.5M 2s 2024-07-31T14:53:04.163 INFO:tasks.workunit.client.0.smithi067.stderr: 68350K .......... .......... .......... .......... .......... 50% 26.8M 2s 2024-07-31T14:53:04.165 INFO:tasks.workunit.client.0.smithi067.stderr: 68400K .......... .......... .......... .......... .......... 50% 23.1M 2s 2024-07-31T14:53:04.166 INFO:tasks.workunit.client.0.smithi067.stderr: 68450K .......... .......... .......... .......... .......... 50% 37.9M 2s 2024-07-31T14:53:04.168 INFO:tasks.workunit.client.0.smithi067.stderr: 68500K .......... .......... .......... .......... .......... 50% 23.4M 2s 2024-07-31T14:53:04.169 INFO:tasks.workunit.client.0.smithi067.stderr: 68550K .......... .......... .......... .......... .......... 50% 44.3M 2s 2024-07-31T14:53:04.171 INFO:tasks.workunit.client.0.smithi067.stderr: 68600K .......... .......... .......... .......... .......... 50% 26.4M 2s 2024-07-31T14:53:04.173 INFO:tasks.workunit.client.0.smithi067.stderr: 68650K .......... .......... .......... .......... .......... 50% 22.6M 2s 2024-07-31T14:53:04.175 INFO:tasks.workunit.client.0.smithi067.stderr: 68700K .......... .......... .......... .......... .......... 50% 38.3M 2s 2024-07-31T14:53:04.177 INFO:tasks.workunit.client.0.smithi067.stderr: 68750K .......... .......... .......... .......... .......... 50% 21.4M 2s 2024-07-31T14:53:04.179 INFO:tasks.workunit.client.0.smithi067.stderr: 68800K .......... .......... .......... .......... .......... 50% 62.1M 2s 2024-07-31T14:53:04.180 INFO:tasks.workunit.client.0.smithi067.stderr: 68850K .......... .......... .......... .......... .......... 50% 22.6M 2s 2024-07-31T14:53:04.182 INFO:tasks.workunit.client.0.smithi067.stderr: 68900K .......... .......... .......... .......... .......... 50% 24.3M 2s 2024-07-31T14:53:04.183 INFO:tasks.workunit.client.0.smithi067.stderr: 68950K .......... .......... .......... .......... .......... 50% 37.1M 2s 2024-07-31T14:53:04.185 INFO:tasks.workunit.client.0.smithi067.stderr: 69000K .......... .......... .......... .......... .......... 50% 23.8M 2s 2024-07-31T14:53:04.187 INFO:tasks.workunit.client.0.smithi067.stderr: 69050K .......... .......... .......... .......... .......... 50% 54.2M 2s 2024-07-31T14:53:04.188 INFO:tasks.workunit.client.0.smithi067.stderr: 69100K .......... .......... .......... .......... .......... 50% 25.1M 2s 2024-07-31T14:53:04.190 INFO:tasks.workunit.client.0.smithi067.stderr: 69150K .......... .......... .......... .......... .......... 50% 22.5M 2s 2024-07-31T14:53:04.191 INFO:tasks.workunit.client.0.smithi067.stderr: 69200K .......... .......... .......... .......... .......... 51% 41.7M 2s 2024-07-31T14:53:04.194 INFO:tasks.workunit.client.0.smithi067.stderr: 69250K .......... .......... .......... .......... .......... 51% 23.1M 2s 2024-07-31T14:53:04.196 INFO:tasks.workunit.client.0.smithi067.stderr: 69300K .......... .......... .......... .......... .......... 51% 25.4M 2s 2024-07-31T14:53:04.196 INFO:tasks.workunit.client.0.smithi067.stderr: 69350K .......... .......... .......... .......... .......... 51% 45.5M 2s 2024-07-31T14:53:04.198 INFO:tasks.workunit.client.0.smithi067.stderr: 69400K .......... .......... .......... .......... .......... 51% 23.8M 2s 2024-07-31T14:53:04.200 INFO:tasks.workunit.client.0.smithi067.stderr: 69450K .......... .......... .......... .......... .......... 51% 36.5M 2s 2024-07-31T14:53:04.202 INFO:tasks.workunit.client.0.smithi067.stderr: 69500K .......... .......... .......... .......... .......... 51% 41.6M 2s 2024-07-31T14:53:04.204 INFO:tasks.workunit.client.0.smithi067.stderr: 69550K .......... .......... .......... .......... .......... 51% 17.7M 2s 2024-07-31T14:53:04.205 INFO:tasks.workunit.client.0.smithi067.stderr: 69600K .......... .......... .......... .......... .......... 51% 46.8M 2s 2024-07-31T14:53:04.207 INFO:tasks.workunit.client.0.smithi067.stderr: 69650K .......... .......... .......... .......... .......... 51% 23.9M 2s 2024-07-31T14:53:04.208 INFO:tasks.workunit.client.0.smithi067.stderr: 69700K .......... .......... .......... .......... .......... 51% 43.7M 2s 2024-07-31T14:53:04.210 INFO:tasks.workunit.client.0.smithi067.stderr: 69750K .......... .......... .......... .......... .......... 51% 22.3M 2s 2024-07-31T14:53:04.212 INFO:tasks.workunit.client.0.smithi067.stderr: 69800K .......... .......... .......... .......... .......... 51% 23.8M 2s 2024-07-31T14:53:04.213 INFO:tasks.workunit.client.0.smithi067.stderr: 69850K .......... .......... .......... .......... .......... 51% 49.0M 2s 2024-07-31T14:53:04.215 INFO:tasks.workunit.client.0.smithi067.stderr: 69900K .......... .......... .......... .......... .......... 51% 23.7M 2s 2024-07-31T14:53:04.217 INFO:tasks.workunit.client.0.smithi067.stderr: 69950K .......... .......... .......... .......... .......... 51% 41.8M 2s 2024-07-31T14:53:04.219 INFO:tasks.workunit.client.0.smithi067.stderr: 70000K .......... .......... .......... .......... .......... 51% 20.6M 2s 2024-07-31T14:53:04.221 INFO:tasks.workunit.client.0.smithi067.stderr: 70050K .......... .......... .......... .......... .......... 51% 23.8M 2s 2024-07-31T14:53:04.222 INFO:tasks.workunit.client.0.smithi067.stderr: 70100K .......... .......... .......... .......... .......... 51% 52.9M 2s 2024-07-31T14:53:04.224 INFO:tasks.workunit.client.0.smithi067.stderr: 70150K .......... .......... .......... .......... .......... 51% 23.4M 2s 2024-07-31T14:53:04.226 INFO:tasks.workunit.client.0.smithi067.stderr: 70200K .......... .......... .......... .......... .......... 51% 25.8M 2s 2024-07-31T14:53:04.227 INFO:tasks.workunit.client.0.smithi067.stderr: 70250K .......... .......... .......... .......... .......... 51% 35.0M 2s 2024-07-31T14:53:04.229 INFO:tasks.workunit.client.0.smithi067.stderr: 70300K .......... .......... .......... .......... .......... 51% 23.3M 2s 2024-07-31T14:53:04.230 INFO:tasks.workunit.client.0.smithi067.stderr: 70350K .......... .......... .......... .......... .......... 51% 75.6M 2s 2024-07-31T14:53:04.232 INFO:tasks.workunit.client.0.smithi067.stderr: 70400K .......... .......... .......... .......... .......... 51% 20.5M 2s 2024-07-31T14:53:04.234 INFO:tasks.workunit.client.0.smithi067.stderr: 70450K .......... .......... .......... .......... .......... 51% 36.4M 2s 2024-07-31T14:53:04.236 INFO:tasks.workunit.client.0.smithi067.stderr: 70500K .......... .......... .......... .......... .......... 51% 23.8M 2s 2024-07-31T14:53:04.238 INFO:tasks.workunit.client.0.smithi067.stderr: 70550K .......... .......... .......... .......... .......... 52% 24.1M 2s 2024-07-31T14:53:04.238 INFO:tasks.workunit.client.0.smithi067.stderr: 70600K .......... .......... .......... .......... .......... 52% 59.5M 2s 2024-07-31T14:53:04.241 INFO:tasks.workunit.client.0.smithi067.stderr: 70650K .......... .......... .......... .......... .......... 52% 23.1M 2s 2024-07-31T14:53:04.242 INFO:tasks.workunit.client.0.smithi067.stderr: 70700K .......... .......... .......... .......... .......... 52% 42.6M 2s 2024-07-31T14:53:04.245 INFO:tasks.workunit.client.0.smithi067.stderr: 70750K .......... .......... .......... .......... .......... 52% 21.2M 2s 2024-07-31T14:53:04.246 INFO:tasks.workunit.client.0.smithi067.stderr: 70800K .......... .......... .......... .......... .......... 52% 25.0M 2s 2024-07-31T14:53:04.247 INFO:tasks.workunit.client.0.smithi067.stderr: 70850K .......... .......... .......... .......... .......... 52% 49.2M 2s 2024-07-31T14:53:04.249 INFO:tasks.workunit.client.0.smithi067.stderr: 70900K .......... .......... .......... .......... .......... 52% 24.6M 2s 2024-07-31T14:53:04.251 INFO:tasks.workunit.client.0.smithi067.stderr: 70950K .......... .......... .......... .......... .......... 52% 35.5M 2s 2024-07-31T14:53:04.252 INFO:tasks.workunit.client.0.smithi067.stderr: 71000K .......... .......... .......... .......... .......... 52% 25.7M 2s 2024-07-31T14:53:04.254 INFO:tasks.workunit.client.0.smithi067.stderr: 71050K .......... .......... .......... .......... .......... 52% 23.3M 2s 2024-07-31T14:53:04.255 INFO:tasks.workunit.client.0.smithi067.stderr: 71100K .......... .......... .......... .......... .......... 52% 53.5M 2s 2024-07-31T14:53:04.257 INFO:tasks.workunit.client.0.smithi067.stderr: 71150K .......... .......... .......... .......... .......... 52% 24.1M 2s 2024-07-31T14:53:04.259 INFO:tasks.workunit.client.0.smithi067.stderr: 71200K .......... .......... .......... .......... .......... 52% 32.7M 2s 2024-07-31T14:53:04.261 INFO:tasks.workunit.client.0.smithi067.stderr: 71250K .......... .......... .......... .......... .......... 52% 27.0M 2s 2024-07-31T14:53:04.263 INFO:tasks.workunit.client.0.smithi067.stderr: 71300K .......... .......... .......... .......... .......... 52% 22.7M 2s 2024-07-31T14:53:04.264 INFO:tasks.workunit.client.0.smithi067.stderr: 71350K .......... .......... .......... .......... .......... 52% 59.5M 2s 2024-07-31T14:53:04.265 INFO:tasks.workunit.client.0.smithi067.stderr: 71400K .......... .......... .......... .......... .......... 52% 27.9M 2s 2024-07-31T14:53:04.268 INFO:tasks.workunit.client.0.smithi067.stderr: 71450K .......... .......... .......... .......... .......... 52% 33.6M 2s 2024-07-31T14:53:04.270 INFO:tasks.workunit.client.0.smithi067.stderr: 71500K .......... .......... .......... .......... .......... 52% 21.7M 2s 2024-07-31T14:53:04.271 INFO:tasks.workunit.client.0.smithi067.stderr: 71550K .......... .......... .......... .......... .......... 52% 24.0M 2s 2024-07-31T14:53:04.272 INFO:tasks.workunit.client.0.smithi067.stderr: 71600K .......... .......... .......... .......... .......... 52% 62.2M 2s 2024-07-31T14:53:04.274 INFO:tasks.workunit.client.0.smithi067.stderr: 71650K .......... .......... .......... .......... .......... 52% 21.9M 2s 2024-07-31T14:53:04.275 INFO:tasks.workunit.client.0.smithi067.stderr: 71700K .......... .......... .......... .......... .......... 52% 40.8M 2s 2024-07-31T14:53:04.277 INFO:tasks.workunit.client.0.smithi067.stderr: 71750K .......... .......... .......... .......... .......... 52% 23.5M 2s 2024-07-31T14:53:04.280 INFO:tasks.workunit.client.0.smithi067.stderr: 71800K .......... .......... .......... .......... .......... 52% 23.8M 2s 2024-07-31T14:53:04.280 INFO:tasks.workunit.client.0.smithi067.stderr: 71850K .......... .......... .......... .......... .......... 52% 52.7M 2s 2024-07-31T14:53:04.282 INFO:tasks.workunit.client.0.smithi067.stderr: 71900K .......... .......... .......... .......... .......... 53% 23.9M 2s 2024-07-31T14:53:04.284 INFO:tasks.workunit.client.0.smithi067.stderr: 71950K .......... .......... .......... .......... .......... 53% 39.2M 2s 2024-07-31T14:53:04.286 INFO:tasks.workunit.client.0.smithi067.stderr: 72000K .......... .......... .......... .......... .......... 53% 23.5M 2s 2024-07-31T14:53:04.288 INFO:tasks.workunit.client.0.smithi067.stderr: 72050K .......... .......... .......... .......... .......... 53% 23.6M 2s 2024-07-31T14:53:04.289 INFO:tasks.workunit.client.0.smithi067.stderr: 72100K .......... .......... .......... .......... .......... 53% 47.7M 2s 2024-07-31T14:53:04.291 INFO:tasks.workunit.client.0.smithi067.stderr: 72150K .......... .......... .......... .......... .......... 53% 23.5M 2s 2024-07-31T14:53:04.292 INFO:tasks.workunit.client.0.smithi067.stderr: 72200K .......... .......... .......... .......... .......... 53% 40.0M 2s 2024-07-31T14:53:04.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:53:04.297 INFO:tasks.workunit.client.0.smithi067.stderr: 72250K .......... .......... .......... .......... .......... 53% 22.8M 2s 2024-07-31T14:53:04.297 INFO:tasks.workunit.client.0.smithi067.stderr: 72300K .......... .......... .......... .......... .......... 53% 23.8M 2s 2024-07-31T14:53:04.299 INFO:tasks.workunit.client.0.smithi067.stderr: 72350K .......... .......... .......... .......... .......... 53% 51.2M 2s 2024-07-31T14:53:04.299 INFO:tasks.workunit.client.0.smithi067.stderr: 72400K .......... .......... .......... .......... .......... 53% 23.5M 2s 2024-07-31T14:53:04.301 INFO:tasks.workunit.client.0.smithi067.stderr: 72450K .......... .......... .......... .......... .......... 53% 36.2M 2s 2024-07-31T14:53:04.303 INFO:tasks.workunit.client.0.smithi067.stderr: 72500K .......... .......... .......... .......... .......... 53% 25.3M 2s 2024-07-31T14:53:04.305 INFO:tasks.workunit.client.0.smithi067.stderr: 72550K .......... .......... .......... .......... .......... 53% 22.7M 2s 2024-07-31T14:53:04.306 INFO:tasks.workunit.client.0.smithi067.stderr: 72600K .......... .......... .......... .......... .......... 53% 52.9M 2s 2024-07-31T14:53:04.308 INFO:tasks.workunit.client.0.smithi067.stderr: 72650K .......... .......... .......... .......... .......... 53% 23.1M 2s 2024-07-31T14:53:04.309 INFO:tasks.workunit.client.0.smithi067.stderr: 72700K .......... .......... .......... .......... .......... 53% 45.5M 2s 2024-07-31T14:53:04.311 INFO:tasks.workunit.client.0.smithi067.stderr: 72750K .......... .......... .......... .......... .......... 53% 23.1M 2s 2024-07-31T14:53:04.313 INFO:tasks.workunit.client.0.smithi067.stderr: 72800K .......... .......... .......... .......... .......... 53% 23.2M 2s 2024-07-31T14:53:04.314 INFO:tasks.workunit.client.0.smithi067.stderr: 72850K .......... .......... .......... .......... .......... 53% 55.0M 2s 2024-07-31T14:53:04.316 INFO:tasks.workunit.client.0.smithi067.stderr: 72900K .......... .......... .......... .......... .......... 53% 24.1M 2s 2024-07-31T14:53:04.317 INFO:tasks.workunit.client.0.smithi067.stderr: 72950K .......... .......... .......... .......... .......... 53% 38.7M 2s 2024-07-31T14:53:04.319 INFO:tasks.workunit.client.0.smithi067.stderr: 73000K .......... .......... .......... .......... .......... 53% 22.6M 2s 2024-07-31T14:53:04.321 INFO:tasks.workunit.client.0.smithi067.stderr: 73050K .......... .......... .......... .......... .......... 53% 26.4M 2s 2024-07-31T14:53:04.322 INFO:tasks.workunit.client.0.smithi067.stderr: 73100K .......... .......... .......... .......... .......... 53% 44.6M 2s 2024-07-31T14:53:04.325 INFO:tasks.workunit.client.0.smithi067.stderr: 73150K .......... .......... .......... .......... .......... 53% 22.7M 2s 2024-07-31T14:53:04.327 INFO:tasks.workunit.client.0.smithi067.stderr: 73200K .......... .......... .......... .......... .......... 53% 38.9M 2s 2024-07-31T14:53:04.328 INFO:tasks.workunit.client.0.smithi067.stderr: 73250K .......... .......... .......... .......... .......... 53% 22.9M 2s 2024-07-31T14:53:04.330 INFO:tasks.workunit.client.0.smithi067.stderr: 73300K .......... .......... .......... .......... .......... 54% 24.1M 2s 2024-07-31T14:53:04.331 INFO:tasks.workunit.client.0.smithi067.stderr: 73350K .......... .......... .......... .......... .......... 54% 55.1M 2s 2024-07-31T14:53:04.333 INFO:tasks.workunit.client.0.smithi067.stderr: 73400K .......... .......... .......... .......... .......... 54% 24.1M 2s 2024-07-31T14:53:04.335 INFO:tasks.workunit.client.0.smithi067.stderr: 73450K .......... .......... .......... .......... .......... 54% 30.6M 2s 2024-07-31T14:53:04.337 INFO:tasks.workunit.client.0.smithi067.stderr: 73500K .......... .......... .......... .......... .......... 54% 22.4M 2s 2024-07-31T14:53:04.338 INFO:tasks.workunit.client.0.smithi067.stderr: 73550K .......... .......... .......... .......... .......... 54% 27.9M 2s 2024-07-31T14:53:04.339 INFO:tasks.workunit.client.0.smithi067.stderr: 73600K .......... .......... .......... .......... .......... 54% 58.6M 2s 2024-07-31T14:53:04.341 INFO:tasks.workunit.client.0.smithi067.stderr: 73650K .......... .......... .......... .......... .......... 54% 24.9M 2s 2024-07-31T14:53:04.343 INFO:tasks.workunit.client.0.smithi067.stderr: 73700K .......... .......... .......... .......... .......... 54% 30.0M 2s 2024-07-31T14:53:04.345 INFO:tasks.workunit.client.0.smithi067.stderr: 73750K .......... .......... .......... .......... .......... 54% 23.8M 2s 2024-07-31T14:53:04.347 INFO:tasks.workunit.client.0.smithi067.stderr: 73800K .......... .......... .......... .......... .......... 54% 27.3M 2s 2024-07-31T14:53:04.347 INFO:tasks.workunit.client.0.smithi067.stderr: 73850K .......... .......... .......... .......... .......... 54% 52.3M 2s 2024-07-31T14:53:04.350 INFO:tasks.workunit.client.0.smithi067.stderr: 73900K .......... .......... .......... .......... .......... 54% 23.5M 2s 2024-07-31T14:53:04.352 INFO:tasks.workunit.client.0.smithi067.stderr: 73950K .......... .......... .......... .......... .......... 54% 24.2M 2s 2024-07-31T14:53:04.353 INFO:tasks.workunit.client.0.smithi067.stderr: 74000K .......... .......... .......... .......... .......... 54% 31.1M 2s 2024-07-31T14:53:04.356 INFO:tasks.workunit.client.0.smithi067.stderr: 74050K .......... .......... .......... .......... .......... 54% 23.8M 2s 2024-07-31T14:53:04.356 INFO:tasks.workunit.client.0.smithi067.stderr: 74100K .......... .......... .......... .......... .......... 54% 75.3M 2s 2024-07-31T14:53:04.358 INFO:tasks.workunit.client.0.smithi067.stderr: 74150K .......... .......... .......... .......... .......... 54% 28.5M 2s 2024-07-31T14:53:04.360 INFO:tasks.workunit.client.0.smithi067.stderr: 74200K .......... .......... .......... .......... .......... 54% 19.8M 2s 2024-07-31T14:53:04.361 INFO:tasks.workunit.client.0.smithi067.stderr: 74250K .......... .......... .......... .......... .......... 54% 39.4M 2s 2024-07-31T14:53:04.363 INFO:tasks.workunit.client.0.smithi067.stderr: 74300K .......... .......... .......... .......... .......... 54% 33.2M 2s 2024-07-31T14:53:04.365 INFO:tasks.workunit.client.0.smithi067.stderr: 74350K .......... .......... .......... .......... .......... 54% 33.2M 2s 2024-07-31T14:53:04.366 INFO:tasks.workunit.client.0.smithi067.stderr: 74400K .......... .......... .......... .......... .......... 54% 26.6M 2s 2024-07-31T14:53:04.368 INFO:tasks.workunit.client.0.smithi067.stderr: 74450K .......... .......... .......... .......... .......... 54% 25.7M 2s 2024-07-31T14:53:04.370 INFO:tasks.workunit.client.0.smithi067.stderr: 74500K .......... .......... .......... .......... .......... 54% 28.7M 2s 2024-07-31T14:53:04.371 INFO:tasks.workunit.client.0.smithi067.stderr: 74550K .......... .......... .......... .......... .......... 54% 32.9M 2s 2024-07-31T14:53:04.373 INFO:tasks.workunit.client.0.smithi067.stderr: 74600K .......... .......... .......... .......... .......... 54% 24.6M 2s 2024-07-31T14:53:04.375 INFO:tasks.workunit.client.0.smithi067.stderr: 74650K .......... .......... .......... .......... .......... 55% 36.4M 2s 2024-07-31T14:53:04.376 INFO:tasks.workunit.client.0.smithi067.stderr: 74700K .......... .......... .......... .......... .......... 55% 24.5M 2s 2024-07-31T14:53:04.379 INFO:tasks.workunit.client.0.smithi067.stderr: 74750K .......... .......... .......... .......... .......... 55% 21.3M 2s 2024-07-31T14:53:04.381 INFO:tasks.workunit.client.0.smithi067.stderr: 74800K .......... .......... .......... .......... .......... 55% 34.0M 2s 2024-07-31T14:53:04.382 INFO:tasks.workunit.client.0.smithi067.stderr: 74850K .......... .......... .......... .......... .......... 55% 33.2M 2s 2024-07-31T14:53:04.383 INFO:tasks.workunit.client.0.smithi067.stderr: 74900K .......... .......... .......... .......... .......... 55% 29.9M 2s 2024-07-31T14:53:04.385 INFO:tasks.workunit.client.0.smithi067.stderr: 74950K .......... .......... .......... .......... .......... 55% 28.9M 2s 2024-07-31T14:53:04.387 INFO:tasks.workunit.client.0.smithi067.stderr: 75000K .......... .......... .......... .......... .......... 55% 33.0M 2s 2024-07-31T14:53:04.388 INFO:tasks.workunit.client.0.smithi067.stderr: 75050K .......... .......... .......... .......... .......... 55% 25.3M 2s 2024-07-31T14:53:04.391 INFO:tasks.workunit.client.0.smithi067.stderr: 75100K .......... .......... .......... .......... .......... 55% 20.7M 2s 2024-07-31T14:53:04.392 INFO:tasks.workunit.client.0.smithi067.stderr: 75150K .......... .......... .......... .......... .......... 55% 51.0M 2s 2024-07-31T14:53:04.394 INFO:tasks.workunit.client.0.smithi067.stderr: 75200K .......... .......... .......... .......... .......... 55% 22.4M 2s 2024-07-31T14:53:04.396 INFO:tasks.workunit.client.0.smithi067.stderr: 75250K .......... .......... .......... .......... .......... 55% 36.9M 2s 2024-07-31T14:53:04.397 INFO:tasks.workunit.client.0.smithi067.stderr: 75300K .......... .......... .......... .......... .......... 55% 23.7M 2s 2024-07-31T14:53:04.400 INFO:tasks.workunit.client.0.smithi067.stderr: 75350K .......... .......... .......... .......... .......... 55% 22.7M 2s 2024-07-31T14:53:04.401 INFO:tasks.workunit.client.0.smithi067.stderr: 75400K .......... .......... .......... .......... .......... 55% 50.8M 2s 2024-07-31T14:53:04.403 INFO:tasks.workunit.client.0.smithi067.stderr: 75450K .......... .......... .......... .......... .......... 55% 27.4M 2s 2024-07-31T14:53:04.404 INFO:tasks.workunit.client.0.smithi067.stderr: 75500K .......... .......... .......... .......... .......... 55% 30.2M 2s 2024-07-31T14:53:04.406 INFO:tasks.workunit.client.0.smithi067.stderr: 75550K .......... .......... .......... .......... .......... 55% 24.5M 2s 2024-07-31T14:53:04.408 INFO:tasks.workunit.client.0.smithi067.stderr: 75600K .......... .......... .......... .......... .......... 55% 24.1M 2s 2024-07-31T14:53:04.409 INFO:tasks.workunit.client.0.smithi067.stderr: 75650K .......... .......... .......... .......... .......... 55% 50.5M 2s 2024-07-31T14:53:04.411 INFO:tasks.workunit.client.0.smithi067.stderr: 75700K .......... .......... .......... .......... .......... 55% 23.3M 2s 2024-07-31T14:53:04.413 INFO:tasks.workunit.client.0.smithi067.stderr: 75750K .......... .......... .......... .......... .......... 55% 36.0M 2s 2024-07-31T14:53:04.414 INFO:tasks.workunit.client.0.smithi067.stderr: 75800K .......... .......... .......... .......... .......... 55% 24.6M 2s 2024-07-31T14:53:04.416 INFO:tasks.workunit.client.0.smithi067.stderr: 75850K .......... .......... .......... .......... .......... 55% 23.3M 2s 2024-07-31T14:53:04.417 INFO:tasks.workunit.client.0.smithi067.stderr: 75900K .......... .......... .......... .......... .......... 55% 51.4M 2s 2024-07-31T14:53:04.419 INFO:tasks.workunit.client.0.smithi067.stderr: 75950K .......... .......... .......... .......... .......... 55% 31.1M 2s 2024-07-31T14:53:04.421 INFO:tasks.workunit.client.0.smithi067.stderr: 76000K .......... .......... .......... .......... .......... 56% 28.4M 2s 2024-07-31T14:53:04.423 INFO:tasks.workunit.client.0.smithi067.stderr: 76050K .......... .......... .......... .......... .......... 56% 23.3M 2s 2024-07-31T14:53:04.425 INFO:tasks.workunit.client.0.smithi067.stderr: 76100K .......... .......... .......... .......... .......... 56% 26.7M 2s 2024-07-31T14:53:04.426 INFO:tasks.workunit.client.0.smithi067.stderr: 76150K .......... .......... .......... .......... .......... 56% 42.6M 2s 2024-07-31T14:53:04.428 INFO:tasks.workunit.client.0.smithi067.stderr: 76200K .......... .......... .......... .......... .......... 56% 25.9M 2s 2024-07-31T14:53:04.429 INFO:tasks.workunit.client.0.smithi067.stderr: 76250K .......... .......... .......... .......... .......... 56% 35.2M 2s 2024-07-31T14:53:04.431 INFO:tasks.workunit.client.0.smithi067.stderr: 76300K .......... .......... .......... .......... .......... 56% 23.5M 2s 2024-07-31T14:53:04.433 INFO:tasks.workunit.client.0.smithi067.stderr: 76350K .......... .......... .......... .......... .......... 56% 23.0M 2s 2024-07-31T14:53:04.434 INFO:tasks.workunit.client.0.smithi067.stderr: 76400K .......... .......... .......... .......... .......... 56% 51.7M 2s 2024-07-31T14:53:04.436 INFO:tasks.workunit.client.0.smithi067.stderr: 76450K .......... .......... .......... .......... .......... 56% 23.7M 2s 2024-07-31T14:53:04.438 INFO:tasks.workunit.client.0.smithi067.stderr: 76500K .......... .......... .......... .......... .......... 56% 39.9M 2s 2024-07-31T14:53:04.439 INFO:tasks.workunit.client.0.smithi067.stderr: 76550K .......... .......... .......... .......... .......... 56% 23.5M 2s 2024-07-31T14:53:04.442 INFO:tasks.workunit.client.0.smithi067.stderr: 76600K .......... .......... .......... .......... .......... 56% 23.9M 2s 2024-07-31T14:53:04.442 INFO:tasks.workunit.client.0.smithi067.stderr: 76650K .......... .......... .......... .......... .......... 56% 51.8M 2s 2024-07-31T14:53:04.445 INFO:tasks.workunit.client.0.smithi067.stderr: 76700K .......... .......... .......... .......... .......... 56% 23.9M 2s 2024-07-31T14:53:04.447 INFO:tasks.workunit.client.0.smithi067.stderr: 76750K .......... .......... .......... .......... .......... 56% 23.1M 2s 2024-07-31T14:53:04.448 INFO:tasks.workunit.client.0.smithi067.stderr: 76800K .......... .......... .......... .......... .......... 56% 39.9M 2s 2024-07-31T14:53:04.450 INFO:tasks.workunit.client.0.smithi067.stderr: 76850K .......... .......... .......... .......... .......... 56% 23.4M 2s 2024-07-31T14:53:04.451 INFO:tasks.workunit.client.0.smithi067.stderr: 76900K .......... .......... .......... .......... .......... 56% 55.1M 2s 2024-07-31T14:53:04.453 INFO:tasks.workunit.client.0.smithi067.stderr: 76950K .......... .......... .......... .......... .......... 56% 22.4M 2s 2024-07-31T14:53:04.454 INFO:tasks.workunit.client.0.smithi067.stderr: 77000K .......... .......... .......... .......... .......... 56% 42.2M 2s 2024-07-31T14:53:04.456 INFO:tasks.workunit.client.0.smithi067.stderr: 77050K .......... .......... .......... .......... .......... 56% 24.3M 2s 2024-07-31T14:53:04.458 INFO:tasks.workunit.client.0.smithi067.stderr: 77100K .......... .......... .......... .......... .......... 56% 23.2M 2s 2024-07-31T14:53:04.459 INFO:tasks.workunit.client.0.smithi067.stderr: 77150K .......... .......... .......... .......... .......... 56% 50.8M 2s 2024-07-31T14:53:04.461 INFO:tasks.workunit.client.0.smithi067.stderr: 77200K .......... .......... .......... .......... .......... 56% 24.1M 2s 2024-07-31T14:53:04.463 INFO:tasks.workunit.client.0.smithi067.stderr: 77250K .......... .......... .......... .......... .......... 56% 40.0M 2s 2024-07-31T14:53:04.465 INFO:tasks.workunit.client.0.smithi067.stderr: 77300K .......... .......... .......... .......... .......... 56% 24.0M 2s 2024-07-31T14:53:04.467 INFO:tasks.workunit.client.0.smithi067.stderr: 77350K .......... .......... .......... .......... .......... 57% 23.8M 2s 2024-07-31T14:53:04.468 INFO:tasks.workunit.client.0.smithi067.stderr: 77400K .......... .......... .......... .......... .......... 57% 45.6M 2s 2024-07-31T14:53:04.470 INFO:tasks.workunit.client.0.smithi067.stderr: 77450K .......... .......... .......... .......... .......... 57% 24.6M 2s 2024-07-31T14:53:04.471 INFO:tasks.workunit.client.0.smithi067.stderr: 77500K .......... .......... .......... .......... .......... 57% 40.4M 2s 2024-07-31T14:53:04.473 INFO:tasks.workunit.client.0.smithi067.stderr: 77550K .......... .......... .......... .......... .......... 57% 25.2M 2s 2024-07-31T14:53:04.475 INFO:tasks.workunit.client.0.smithi067.stderr: 77600K .......... .......... .......... .......... .......... 57% 22.5M 2s 2024-07-31T14:53:04.476 INFO:tasks.workunit.client.0.smithi067.stderr: 77650K .......... .......... .......... .......... .......... 57% 44.9M 2s 2024-07-31T14:53:04.478 INFO:tasks.workunit.client.0.smithi067.stderr: 77700K .......... .......... .......... .......... .......... 57% 24.0M 2s 2024-07-31T14:53:04.480 INFO:tasks.workunit.client.0.smithi067.stderr: 77750K .......... .......... .......... .......... .......... 57% 40.8M 2s 2024-07-31T14:53:04.482 INFO:tasks.workunit.client.0.smithi067.stderr: 77800K .......... .......... .......... .......... .......... 57% 25.2M 2s 2024-07-31T14:53:04.483 INFO:tasks.workunit.client.0.smithi067.stderr: 77850K .......... .......... .......... .......... .......... 57% 22.8M 2s 2024-07-31T14:53:04.484 INFO:tasks.workunit.client.0.smithi067.stderr: 77900K .......... .......... .......... .......... .......... 57% 50.6M 2s 2024-07-31T14:53:04.486 INFO:tasks.workunit.client.0.smithi067.stderr: 77950K .......... .......... .......... .......... .......... 57% 23.5M 2s 2024-07-31T14:53:04.489 INFO:tasks.workunit.client.0.smithi067.stderr: 78000K .......... .......... .......... .......... .......... 57% 41.1M 2s 2024-07-31T14:53:04.490 INFO:tasks.workunit.client.0.smithi067.stderr: 78050K .......... .......... .......... .......... .......... 57% 21.4M 2s 2024-07-31T14:53:04.492 INFO:tasks.workunit.client.0.smithi067.stderr: 78100K .......... .......... .......... .......... .......... 57% 25.2M 2s 2024-07-31T14:53:04.493 INFO:tasks.workunit.client.0.smithi067.stderr: 78150K .......... .......... .......... .......... .......... 57% 48.3M 2s 2024-07-31T14:53:04.495 INFO:tasks.workunit.client.0.smithi067.stderr: 78200K .......... .......... .......... .......... .......... 57% 22.7M 2s 2024-07-31T14:53:04.497 INFO:tasks.workunit.client.0.smithi067.stderr: 78250K .......... .......... .......... .......... .......... 57% 26.7M 2s 2024-07-31T14:53:04.498 INFO:tasks.workunit.client.0.smithi067.stderr: 78300K .......... .......... .......... .......... .......... 57% 31.4M 2s 2024-07-31T14:53:04.501 INFO:tasks.workunit.client.0.smithi067.stderr: 78350K .......... .......... .......... .......... .......... 57% 21.8M 2s 2024-07-31T14:53:04.501 INFO:tasks.workunit.client.0.smithi067.stderr: 78400K .......... .......... .......... .......... .......... 57% 59.0M 2s 2024-07-31T14:53:04.503 INFO:tasks.workunit.client.0.smithi067.stderr: 78450K .......... .......... .......... .......... .......... 57% 23.6M 2s 2024-07-31T14:53:04.505 INFO:tasks.workunit.client.0.smithi067.stderr: 78500K .......... .......... .......... .......... .......... 57% 41.7M 2s 2024-07-31T14:53:04.507 INFO:tasks.workunit.client.0.smithi067.stderr: 78550K .......... .......... .......... .......... .......... 57% 23.4M 2s 2024-07-31T14:53:04.509 INFO:tasks.workunit.client.0.smithi067.stderr: 78600K .......... .......... .......... .......... .......... 57% 23.9M 2s 2024-07-31T14:53:04.510 INFO:tasks.workunit.client.0.smithi067.stderr: 78650K .......... .......... .......... .......... .......... 57% 58.1M 2s 2024-07-31T14:53:04.512 INFO:tasks.workunit.client.0.smithi067.stderr: 78700K .......... .......... .......... .......... .......... 58% 22.6M 2s 2024-07-31T14:53:04.514 INFO:tasks.workunit.client.0.smithi067.stderr: 78750K .......... .......... .......... .......... .......... 58% 23.6M 2s 2024-07-31T14:53:04.515 INFO:tasks.workunit.client.0.smithi067.stderr: 78800K .......... .......... .......... .......... .......... 58% 39.0M 2s 2024-07-31T14:53:04.517 INFO:tasks.workunit.client.0.smithi067.stderr: 78850K .......... .......... .......... .......... .......... 58% 23.4M 2s 2024-07-31T14:53:04.518 INFO:tasks.workunit.client.0.smithi067.stderr: 78900K .......... .......... .......... .......... .......... 58% 59.2M 2s 2024-07-31T14:53:04.520 INFO:tasks.workunit.client.0.smithi067.stderr: 78950K .......... .......... .......... .......... .......... 58% 27.1M 2s 2024-07-31T14:53:04.522 INFO:tasks.workunit.client.0.smithi067.stderr: 79000K .......... .......... .......... .......... .......... 58% 25.0M 2s 2024-07-31T14:53:04.524 INFO:tasks.workunit.client.0.smithi067.stderr: 79050K .......... .......... .......... .......... .......... 58% 28.6M 2s 2024-07-31T14:53:04.526 INFO:tasks.workunit.client.0.smithi067.stderr: 79100K .......... .......... .......... .......... .......... 58% 23.2M 2s 2024-07-31T14:53:04.527 INFO:tasks.workunit.client.0.smithi067.stderr: 79150K .......... .......... .......... .......... .......... 58% 52.2M 2s 2024-07-31T14:53:04.528 INFO:tasks.workunit.client.0.smithi067.stderr: 79200K .......... .......... .......... .......... .......... 58% 24.7M 2s 2024-07-31T14:53:04.531 INFO:tasks.workunit.client.0.smithi067.stderr: 79250K .......... .......... .......... .......... .......... 58% 22.1M 2s 2024-07-31T14:53:04.533 INFO:tasks.workunit.client.0.smithi067.stderr: 79300K .......... .......... .......... .......... .......... 58% 35.7M 2s 2024-07-31T14:53:04.535 INFO:tasks.workunit.client.0.smithi067.stderr: 79350K .......... .......... .......... .......... .......... 58% 20.4M 2s 2024-07-31T14:53:04.537 INFO:tasks.workunit.client.0.smithi067.stderr: 79400K .......... .......... .......... .......... .......... 58% 58.3M 2s 2024-07-31T14:53:04.537 INFO:tasks.workunit.client.0.smithi067.stderr: 79450K .......... .......... .......... .......... .......... 58% 27.4M 2s 2024-07-31T14:53:04.540 INFO:tasks.workunit.client.0.smithi067.stderr: 79500K .......... .......... .......... .......... .......... 58% 19.9M 2s 2024-07-31T14:53:04.541 INFO:tasks.workunit.client.0.smithi067.stderr: 79550K .......... .......... .......... .......... .......... 58% 40.0M 2s 2024-07-31T14:53:04.543 INFO:tasks.workunit.client.0.smithi067.stderr: 79600K .......... .......... .......... .......... .......... 58% 23.2M 2s 2024-07-31T14:53:04.545 INFO:tasks.workunit.client.0.smithi067.stderr: 79650K .......... .......... .......... .......... .......... 58% 28.7M 2s 2024-07-31T14:53:04.545 INFO:tasks.workunit.client.0.smithi067.stderr: 79700K .......... .......... .......... .......... .......... 58% 57.9M 2s 2024-07-31T14:53:04.548 INFO:tasks.workunit.client.0.smithi067.stderr: 79750K .......... .......... .......... .......... .......... 58% 19.8M 2s 2024-07-31T14:53:04.549 INFO:tasks.workunit.client.0.smithi067.stderr: 79800K .......... .......... .......... .......... .......... 58% 37.8M 2s 2024-07-31T14:53:04.551 INFO:tasks.workunit.client.0.smithi067.stderr: 79850K .......... .......... .......... .......... .......... 58% 24.6M 2s 2024-07-31T14:53:04.553 INFO:tasks.workunit.client.0.smithi067.stderr: 79900K .......... .......... .......... .......... .......... 58% 23.9M 2s 2024-07-31T14:53:04.554 INFO:tasks.workunit.client.0.smithi067.stderr: 79950K .......... .......... .......... .......... .......... 58% 77.0M 2s 2024-07-31T14:53:04.556 INFO:tasks.workunit.client.0.smithi067.stderr: 80000K .......... .......... .......... .......... .......... 58% 22.1M 2s 2024-07-31T14:53:04.558 INFO:tasks.workunit.client.0.smithi067.stderr: 80050K .......... .......... .......... .......... .......... 59% 34.7M 2s 2024-07-31T14:53:04.560 INFO:tasks.workunit.client.0.smithi067.stderr: 80100K .......... .......... .......... .......... .......... 59% 23.8M 2s 2024-07-31T14:53:04.562 INFO:tasks.workunit.client.0.smithi067.stderr: 80150K .......... .......... .......... .......... .......... 59% 23.4M 2s 2024-07-31T14:53:04.563 INFO:tasks.workunit.client.0.smithi067.stderr: 80200K .......... .......... .......... .......... .......... 59% 47.9M 2s 2024-07-31T14:53:04.565 INFO:tasks.workunit.client.0.smithi067.stderr: 80250K .......... .......... .......... .......... .......... 59% 25.1M 2s 2024-07-31T14:53:04.566 INFO:tasks.workunit.client.0.smithi067.stderr: 80300K .......... .......... .......... .......... .......... 59% 35.3M 2s 2024-07-31T14:53:04.569 INFO:tasks.workunit.client.0.smithi067.stderr: 80350K .......... .......... .......... .......... .......... 59% 24.3M 2s 2024-07-31T14:53:04.570 INFO:tasks.workunit.client.0.smithi067.stderr: 80400K .......... .......... .......... .......... .......... 59% 24.5M 2s 2024-07-31T14:53:04.570 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:53:04.571 INFO:tasks.workunit.client.0.smithi067.stderr: 80450K .......... .......... .......... .......... .......... 59% 59.5M 2s 2024-07-31T14:53:04.573 INFO:tasks.workunit.client.0.smithi067.stderr: 80500K .......... .......... .......... .......... .......... 59% 21.0M 2s 2024-07-31T14:53:04.574 INFO:tasks.workunit.client.0.smithi067.stderr: 80550K .......... .......... .......... .......... .......... 59% 44.0M 2s 2024-07-31T14:53:04.576 INFO:tasks.workunit.client.0.smithi067.stderr: 80600K .......... .......... .......... .......... .......... 59% 23.1M 2s 2024-07-31T14:53:04.578 INFO:tasks.workunit.client.0.smithi067.stderr: 80650K .......... .......... .......... .......... .......... 59% 23.2M 2s 2024-07-31T14:53:04.579 INFO:tasks.workunit.client.0.smithi067.stderr: 80700K .......... .......... .......... .......... .......... 59% 53.0M 2s 2024-07-31T14:53:04.581 INFO:tasks.workunit.client.0.smithi067.stderr: 80750K .......... .......... .......... .......... .......... 59% 23.5M 2s 2024-07-31T14:53:04.583 INFO:tasks.workunit.client.0.smithi067.stderr: 80800K .......... .......... .......... .......... .......... 59% 40.8M 2s 2024-07-31T14:53:04.585 INFO:tasks.workunit.client.0.smithi067.stderr: 80850K .......... .......... .......... .......... .......... 59% 22.7M 2s 2024-07-31T14:53:04.587 INFO:tasks.workunit.client.0.smithi067.stderr: 80900K .......... .......... .......... .......... .......... 59% 25.3M 2s 2024-07-31T14:53:04.588 INFO:tasks.workunit.client.0.smithi067.stderr: 80950K .......... .......... .......... .......... .......... 59% 53.0M 2s 2024-07-31T14:53:04.590 INFO:tasks.workunit.client.0.smithi067.stderr: 81000K .......... .......... .......... .......... .......... 59% 22.7M 2s 2024-07-31T14:53:04.591 INFO:tasks.workunit.client.0.smithi067.stderr: 81050K .......... .......... .......... .......... .......... 59% 40.4M 2s 2024-07-31T14:53:04.593 INFO:tasks.workunit.client.0.smithi067.stderr: 81100K .......... .......... .......... .......... .......... 59% 22.6M 2s 2024-07-31T14:53:04.595 INFO:tasks.workunit.client.0.smithi067.stderr: 81150K .......... .......... .......... .......... .......... 59% 24.9M 2s 2024-07-31T14:53:04.596 INFO:tasks.workunit.client.0.smithi067.stderr: 81200K .......... .......... .......... .......... .......... 59% 51.6M 2s 2024-07-31T14:53:04.598 INFO:tasks.workunit.client.0.smithi067.stderr: 81250K .......... .......... .......... .......... .......... 59% 23.2M 2s 2024-07-31T14:53:04.599 INFO:tasks.workunit.client.0.smithi067.stderr: 81300K .......... .......... .......... .......... .......... 59% 41.8M 2s 2024-07-31T14:53:04.602 INFO:tasks.workunit.client.0.smithi067.stderr: 81350K .......... .......... .......... .......... .......... 59% 22.8M 2s 2024-07-31T14:53:04.604 INFO:tasks.workunit.client.0.smithi067.stderr: 81400K .......... .......... .......... .......... .......... 59% 21.6M 2s 2024-07-31T14:53:04.606 INFO:tasks.workunit.client.0.smithi067.stderr: 81450K .......... .......... .......... .......... .......... 60% 31.7M 2s 2024-07-31T14:53:04.607 INFO:tasks.workunit.client.0.smithi067.stderr: 81500K .......... .......... .......... .......... .......... 60% 34.4M 2s 2024-07-31T14:53:04.608 INFO:tasks.workunit.client.0.smithi067.stderr: 81550K .......... .......... .......... .......... .......... 60% 34.2M 2s 2024-07-31T14:53:04.610 INFO:tasks.workunit.client.0.smithi067.stderr: 81600K .......... .......... .......... .......... .......... 60% 26.6M 2s 2024-07-31T14:53:04.612 INFO:tasks.workunit.client.0.smithi067.stderr: 81650K .......... .......... .......... .......... .......... 60% 23.9M 2s 2024-07-31T14:53:04.613 INFO:tasks.workunit.client.0.smithi067.stderr: 81700K .......... .......... .......... .......... .......... 60% 46.3M 2s 2024-07-31T14:53:04.615 INFO:tasks.workunit.client.0.smithi067.stderr: 81750K .......... .......... .......... .......... .......... 60% 23.1M 2s 2024-07-31T14:53:04.616 INFO:tasks.workunit.client.0.smithi067.stderr: 81800K .......... .......... .......... .......... .......... 60% 40.4M 2s 2024-07-31T14:53:04.619 INFO:tasks.workunit.client.0.smithi067.stderr: 81850K .......... .......... .......... .......... .......... 60% 21.7M 2s 2024-07-31T14:53:04.620 INFO:tasks.workunit.client.0.smithi067.stderr: 81900K .......... .......... .......... .......... .......... 60% 28.0M 2s 2024-07-31T14:53:04.622 INFO:tasks.workunit.client.0.smithi067.stderr: 81950K .......... .......... .......... .......... .......... 60% 45.7M 2s 2024-07-31T14:53:04.623 INFO:tasks.workunit.client.0.smithi067.stderr: 82000K .......... .......... .......... .......... .......... 60% 32.2M 2s 2024-07-31T14:53:04.625 INFO:tasks.workunit.client.0.smithi067.stderr: 82050K .......... .......... .......... .......... .......... 60% 25.4M 2s 2024-07-31T14:53:04.628 INFO:tasks.workunit.client.0.smithi067.stderr: 82100K .......... .......... .......... .......... .......... 60% 24.8M 2s 2024-07-31T14:53:04.629 INFO:tasks.workunit.client.0.smithi067.stderr: 82150K .......... .......... .......... .......... .......... 60% 50.0M 2s 2024-07-31T14:53:04.630 INFO:tasks.workunit.client.0.smithi067.stderr: 82200K .......... .......... .......... .......... .......... 60% 24.5M 2s 2024-07-31T14:53:04.632 INFO:tasks.workunit.client.0.smithi067.stderr: 82250K .......... .......... .......... .......... .......... 60% 22.6M 2s 2024-07-31T14:53:04.633 INFO:tasks.workunit.client.0.smithi067.stderr: 82300K .......... .......... .......... .......... .......... 60% 43.0M 2s 2024-07-31T14:53:04.635 INFO:tasks.workunit.client.0.smithi067.stderr: 82350K .......... .......... .......... .......... .......... 60% 23.9M 2s 2024-07-31T14:53:04.636 INFO:tasks.workunit.client.0.smithi067.stderr: 82400K .......... .......... .......... .......... .......... 60% 38.8M 2s 2024-07-31T14:53:04.638 INFO:tasks.workunit.client.0.smithi067.stderr: 82450K .......... .......... .......... .......... .......... 60% 26.4M 2s 2024-07-31T14:53:04.640 INFO:tasks.workunit.client.0.smithi067.stderr: 82500K .......... .......... .......... .......... .......... 60% 23.9M 2s 2024-07-31T14:53:04.642 INFO:tasks.workunit.client.0.smithi067.stderr: 82550K .......... .......... .......... .......... .......... 60% 41.8M 2s 2024-07-31T14:53:04.644 INFO:tasks.workunit.client.0.smithi067.stderr: 82600K .......... .......... .......... .......... .......... 60% 19.6M 2s 2024-07-31T14:53:04.645 INFO:tasks.workunit.client.0.smithi067.stderr: 82650K .......... .......... .......... .......... .......... 60% 50.5M 2s 2024-07-31T14:53:04.647 INFO:tasks.workunit.client.0.smithi067.stderr: 82700K .......... .......... .......... .......... .......... 60% 27.8M 2s 2024-07-31T14:53:04.649 INFO:tasks.workunit.client.0.smithi067.stderr: 82750K .......... .......... .......... .......... .......... 60% 21.9M 2s 2024-07-31T14:53:04.650 INFO:tasks.workunit.client.0.smithi067.stderr: 82800K .......... .......... .......... .......... .......... 61% 35.1M 2s 2024-07-31T14:53:04.653 INFO:tasks.workunit.client.0.smithi067.stderr: 82850K .......... .......... .......... .......... .......... 61% 22.2M 2s 2024-07-31T14:53:04.653 INFO:tasks.workunit.client.0.smithi067.stderr: 82900K .......... .......... .......... .......... .......... 61% 56.2M 2s 2024-07-31T14:53:04.655 INFO:tasks.workunit.client.0.smithi067.stderr: 82950K .......... .......... .......... .......... .......... 61% 30.4M 2s 2024-07-31T14:53:04.657 INFO:tasks.workunit.client.0.smithi067.stderr: 83000K .......... .......... .......... .......... .......... 61% 26.3M 2s 2024-07-31T14:53:04.659 INFO:tasks.workunit.client.0.smithi067.stderr: 83050K .......... .......... .......... .......... .......... 61% 24.4M 2s 2024-07-31T14:53:04.660 INFO:tasks.workunit.client.0.smithi067.stderr: 83100K .......... .......... .......... .......... .......... 61% 28.1M 2s 2024-07-31T14:53:04.663 INFO:tasks.workunit.client.0.smithi067.stderr: 83150K .......... .......... .......... .......... .......... 61% 23.2M 2s 2024-07-31T14:53:04.663 INFO:tasks.workunit.client.0.smithi067.stderr: 83200K .......... .......... .......... .......... .......... 61% 46.3M 2s 2024-07-31T14:53:04.665 INFO:tasks.workunit.client.0.smithi067.stderr: 83250K .......... .......... .......... .......... .......... 61% 29.0M 2s 2024-07-31T14:53:04.668 INFO:tasks.workunit.client.0.smithi067.stderr: 83300K .......... .......... .......... .......... .......... 61% 21.9M 2s 2024-07-31T14:53:04.670 INFO:tasks.workunit.client.0.smithi067.stderr: 83350K .......... .......... .......... .......... .......... 61% 24.3M 2s 2024-07-31T14:53:04.671 INFO:tasks.workunit.client.0.smithi067.stderr: 83400K .......... .......... .......... .......... .......... 61% 33.7M 2s 2024-07-31T14:53:04.672 INFO:tasks.workunit.client.0.smithi067.stderr: 83450K .......... .......... .......... .......... .......... 61% 49.3M 2s 2024-07-31T14:53:04.674 INFO:tasks.workunit.client.0.smithi067.stderr: 83500K .......... .......... .......... .......... .......... 61% 22.6M 2s 2024-07-31T14:53:04.676 INFO:tasks.workunit.client.0.smithi067.stderr: 83550K .......... .......... .......... .......... .......... 61% 26.3M 2s 2024-07-31T14:53:04.677 INFO:tasks.workunit.client.0.smithi067.stderr: 83600K .......... .......... .......... .......... .......... 61% 53.9M 2s 2024-07-31T14:53:04.679 INFO:tasks.workunit.client.0.smithi067.stderr: 83650K .......... .......... .......... .......... .......... 61% 22.5M 2s 2024-07-31T14:53:04.680 INFO:tasks.workunit.client.0.smithi067.stderr: 83700K .......... .......... .......... .......... .......... 61% 38.7M 2s 2024-07-31T14:53:04.682 INFO:tasks.workunit.client.0.smithi067.stderr: 83750K .......... .......... .......... .......... .......... 61% 23.0M 2s 2024-07-31T14:53:04.684 INFO:tasks.workunit.client.0.smithi067.stderr: 83800K .......... .......... .......... .......... .......... 61% 44.1M 2s 2024-07-31T14:53:04.686 INFO:tasks.workunit.client.0.smithi067.stderr: 83850K .......... .......... .......... .......... .......... 61% 22.1M 2s 2024-07-31T14:53:04.688 INFO:tasks.workunit.client.0.smithi067.stderr: 83900K .......... .......... .......... .......... .......... 61% 23.1M 2s 2024-07-31T14:53:04.689 INFO:tasks.workunit.client.0.smithi067.stderr: 83950K .......... .......... .......... .......... .......... 61% 66.8M 2s 2024-07-31T14:53:04.691 INFO:tasks.workunit.client.0.smithi067.stderr: 84000K .......... .......... .......... .......... .......... 61% 21.6M 2s 2024-07-31T14:53:04.693 INFO:tasks.workunit.client.0.smithi067.stderr: 84050K .......... .......... .......... .......... .......... 61% 30.8M 2s 2024-07-31T14:53:04.694 INFO:tasks.workunit.client.0.smithi067.stderr: 84100K .......... .......... .......... .......... .......... 61% 33.3M 2s 2024-07-31T14:53:04.696 INFO:tasks.workunit.client.0.smithi067.stderr: 84150K .......... .......... .......... .......... .......... 62% 24.8M 2s 2024-07-31T14:53:04.697 INFO:tasks.workunit.client.0.smithi067.stderr: 84200K .......... .......... .......... .......... .......... 62% 41.2M 2s 2024-07-31T14:53:04.699 INFO:tasks.workunit.client.0.smithi067.stderr: 84250K .......... .......... .......... .......... .......... 62% 27.7M 2s 2024-07-31T14:53:04.702 INFO:tasks.workunit.client.0.smithi067.stderr: 84300K .......... .......... .......... .......... .......... 62% 17.4M 2s 2024-07-31T14:53:04.702 INFO:tasks.workunit.client.0.smithi067.stderr: 84350K .......... .......... .......... .......... .......... 62% 74.9M 2s 2024-07-31T14:53:04.704 INFO:tasks.workunit.client.0.smithi067.stderr: 84400K .......... .......... .......... .......... .......... 62% 23.8M 2s 2024-07-31T14:53:04.705 INFO:tasks.workunit.client.0.smithi067.stderr: 84450K .......... .......... .......... .......... .......... 62% 43.6M 2s 2024-07-31T14:53:04.707 INFO:tasks.workunit.client.0.smithi067.stderr: 84500K .......... .......... .......... .......... .......... 62% 23.9M 2s 2024-07-31T14:53:04.710 INFO:tasks.workunit.client.0.smithi067.stderr: 84550K .......... .......... .......... .......... .......... 62% 23.4M 2s 2024-07-31T14:53:04.711 INFO:tasks.workunit.client.0.smithi067.stderr: 84600K .......... .......... .......... .......... .......... 62% 39.0M 2s 2024-07-31T14:53:04.713 INFO:tasks.workunit.client.0.smithi067.stderr: 84650K .......... .......... .......... .......... .......... 62% 24.9M 2s 2024-07-31T14:53:04.715 INFO:tasks.workunit.client.0.smithi067.stderr: 84700K .......... .......... .......... .......... .......... 62% 45.4M 2s 2024-07-31T14:53:04.716 INFO:tasks.workunit.client.0.smithi067.stderr: 84750K .......... .......... .......... .......... .......... 62% 23.7M 2s 2024-07-31T14:53:04.718 INFO:tasks.workunit.client.0.smithi067.stderr: 84800K .......... .......... .......... .......... .......... 62% 25.1M 2s 2024-07-31T14:53:04.719 INFO:tasks.workunit.client.0.smithi067.stderr: 84850K .......... .......... .......... .......... .......... 62% 39.6M 2s 2024-07-31T14:53:04.721 INFO:tasks.workunit.client.0.smithi067.stderr: 84900K .......... .......... .......... .......... .......... 62% 23.8M 2s 2024-07-31T14:53:04.723 INFO:tasks.workunit.client.0.smithi067.stderr: 84950K .......... .......... .......... .......... .......... 62% 22.7M 2s 2024-07-31T14:53:04.724 INFO:tasks.workunit.client.0.smithi067.stderr: 85000K .......... .......... .......... .......... .......... 62% 49.2M 2s 2024-07-31T14:53:04.726 INFO:tasks.workunit.client.0.smithi067.stderr: 85050K .......... .......... .......... .......... .......... 62% 27.5M 2s 2024-07-31T14:53:04.728 INFO:tasks.workunit.client.0.smithi067.stderr: 85100K .......... .......... .......... .......... .......... 62% 33.7M 2s 2024-07-31T14:53:04.730 INFO:tasks.workunit.client.0.smithi067.stderr: 85150K .......... .......... .......... .......... .......... 62% 21.9M 2s 2024-07-31T14:53:04.732 INFO:tasks.workunit.client.0.smithi067.stderr: 85200K .......... .......... .......... .......... .......... 62% 26.2M 2s 2024-07-31T14:53:04.733 INFO:tasks.workunit.client.0.smithi067.stderr: 85250K .......... .......... .......... .......... .......... 62% 44.1M 2s 2024-07-31T14:53:04.735 INFO:tasks.workunit.client.0.smithi067.stderr: 85300K .......... .......... .......... .......... .......... 62% 30.7M 2s 2024-07-31T14:53:04.736 INFO:tasks.workunit.client.0.smithi067.stderr: 85350K .......... .......... .......... .......... .......... 62% 31.0M 2s 2024-07-31T14:53:04.739 INFO:tasks.workunit.client.0.smithi067.stderr: 85400K .......... .......... .......... .......... .......... 62% 21.3M 2s 2024-07-31T14:53:04.740 INFO:tasks.workunit.client.0.smithi067.stderr: 85450K .......... .......... .......... .......... .......... 62% 24.5M 2s 2024-07-31T14:53:04.741 INFO:tasks.workunit.client.0.smithi067.stderr: 85500K .......... .......... .......... .......... .......... 63% 45.9M 2s 2024-07-31T14:53:04.743 INFO:tasks.workunit.client.0.smithi067.stderr: 85550K .......... .......... .......... .......... .......... 63% 25.8M 2s 2024-07-31T14:53:04.745 INFO:tasks.workunit.client.0.smithi067.stderr: 85600K .......... .......... .......... .......... .......... 63% 34.7M 2s 2024-07-31T14:53:04.747 INFO:tasks.workunit.client.0.smithi067.stderr: 85650K .......... .......... .......... .......... .......... 63% 24.2M 2s 2024-07-31T14:53:04.749 INFO:tasks.workunit.client.0.smithi067.stderr: 85700K .......... .......... .......... .......... .......... 63% 24.5M 2s 2024-07-31T14:53:04.749 INFO:tasks.workunit.client.0.smithi067.stderr: 85750K .......... .......... .......... .......... .......... 63% 55.0M 2s 2024-07-31T14:53:04.752 INFO:tasks.workunit.client.0.smithi067.stderr: 85800K .......... .......... .......... .......... .......... 63% 23.2M 2s 2024-07-31T14:53:04.753 INFO:tasks.workunit.client.0.smithi067.stderr: 85850K .......... .......... .......... .......... .......... 63% 39.5M 2s 2024-07-31T14:53:04.755 INFO:tasks.workunit.client.0.smithi067.stderr: 85900K .......... .......... .......... .......... .......... 63% 23.1M 2s 2024-07-31T14:53:04.757 INFO:tasks.workunit.client.0.smithi067.stderr: 85950K .......... .......... .......... .......... .......... 63% 25.5M 2s 2024-07-31T14:53:04.758 INFO:tasks.workunit.client.0.smithi067.stderr: 86000K .......... .......... .......... .......... .......... 63% 50.1M 2s 2024-07-31T14:53:04.760 INFO:tasks.workunit.client.0.smithi067.stderr: 86050K .......... .......... .......... .......... .......... 63% 23.7M 2s 2024-07-31T14:53:04.761 INFO:tasks.workunit.client.0.smithi067.stderr: 86100K .......... .......... .......... .......... .......... 63% 40.7M 2s 2024-07-31T14:53:04.763 INFO:tasks.workunit.client.0.smithi067.stderr: 86150K .......... .......... .......... .......... .......... 63% 23.2M 2s 2024-07-31T14:53:04.765 INFO:tasks.workunit.client.0.smithi067.stderr: 86200K .......... .......... .......... .......... .......... 63% 23.4M 2s 2024-07-31T14:53:04.766 INFO:tasks.workunit.client.0.smithi067.stderr: 86250K .......... .......... .......... .......... .......... 63% 50.6M 2s 2024-07-31T14:53:04.770 INFO:tasks.workunit.client.0.smithi067.stderr: 86300K .......... .......... .......... .......... .......... 63% 23.6M 2s 2024-07-31T14:53:04.771 INFO:tasks.workunit.client.0.smithi067.stderr: 86350K .......... .......... .......... .......... .......... 63% 34.9M 2s 2024-07-31T14:53:04.772 INFO:tasks.workunit.client.0.smithi067.stderr: 86400K .......... .......... .......... .......... .......... 63% 26.4M 2s 2024-07-31T14:53:04.774 INFO:tasks.workunit.client.0.smithi067.stderr: 86450K .......... .......... .......... .......... .......... 63% 21.8M 2s 2024-07-31T14:53:04.775 INFO:tasks.workunit.client.0.smithi067.stderr: 86500K .......... .......... .......... .......... .......... 63% 51.7M 2s 2024-07-31T14:53:04.777 INFO:tasks.workunit.client.0.smithi067.stderr: 86550K .......... .......... .......... .......... .......... 63% 24.3M 2s 2024-07-31T14:53:04.778 INFO:tasks.workunit.client.0.smithi067.stderr: 86600K .......... .......... .......... .......... .......... 63% 38.7M 2s 2024-07-31T14:53:04.780 INFO:tasks.workunit.client.0.smithi067.stderr: 86650K .......... .......... .......... .......... .......... 63% 22.1M 2s 2024-07-31T14:53:04.782 INFO:tasks.workunit.client.0.smithi067.stderr: 86700K .......... .......... .......... .......... .......... 63% 25.4M 2s 2024-07-31T14:53:04.783 INFO:tasks.workunit.client.0.smithi067.stderr: 86750K .......... .......... .......... .......... .......... 63% 51.8M 2s 2024-07-31T14:53:04.785 INFO:tasks.workunit.client.0.smithi067.stderr: 86800K .......... .......... .......... .......... .......... 63% 25.5M 2s 2024-07-31T14:53:04.786 INFO:tasks.workunit.client.0.smithi067.stderr: 86850K .......... .......... .......... .......... .......... 64% 34.1M 2s 2024-07-31T14:53:04.789 INFO:tasks.workunit.client.0.smithi067.stderr: 86900K .......... .......... .......... .......... .......... 64% 23.5M 2s 2024-07-31T14:53:04.791 INFO:tasks.workunit.client.0.smithi067.stderr: 86950K .......... .......... .......... .......... .......... 64% 20.9M 2s 2024-07-31T14:53:04.792 INFO:tasks.workunit.client.0.smithi067.stderr: 87000K .......... .......... .......... .......... .......... 64% 46.5M 2s 2024-07-31T14:53:04.794 INFO:tasks.workunit.client.0.smithi067.stderr: 87050K .......... .......... .......... .......... .......... 64% 36.7M 2s 2024-07-31T14:53:04.795 INFO:tasks.workunit.client.0.smithi067.stderr: 87100K .......... .......... .......... .......... .......... 64% 27.2M 2s 2024-07-31T14:53:04.797 INFO:tasks.workunit.client.0.smithi067.stderr: 87150K .......... .......... .......... .......... .......... 64% 21.9M 2s 2024-07-31T14:53:04.799 INFO:tasks.workunit.client.0.smithi067.stderr: 87200K .......... .......... .......... .......... .......... 64% 25.4M 2s 2024-07-31T14:53:04.800 INFO:tasks.workunit.client.0.smithi067.stderr: 87250K .......... .......... .......... .......... .......... 64% 45.4M 2s 2024-07-31T14:53:04.802 INFO:tasks.workunit.client.0.smithi067.stderr: 87300K .......... .......... .......... .......... .......... 64% 24.2M 2s 2024-07-31T14:53:04.804 INFO:tasks.workunit.client.0.smithi067.stderr: 87350K .......... .......... .......... .......... .......... 64% 34.8M 2s 2024-07-31T14:53:04.806 INFO:tasks.workunit.client.0.smithi067.stderr: 87400K .......... .......... .......... .......... .......... 64% 24.5M 2s 2024-07-31T14:53:04.808 INFO:tasks.workunit.client.0.smithi067.stderr: 87450K .......... .......... .......... .......... .......... 64% 24.0M 2s 2024-07-31T14:53:04.809 INFO:tasks.workunit.client.0.smithi067.stderr: 87500K .......... .......... .......... .......... .......... 64% 49.1M 2s 2024-07-31T14:53:04.811 INFO:tasks.workunit.client.0.smithi067.stderr: 87550K .......... .......... .......... .......... .......... 64% 24.3M 2s 2024-07-31T14:53:04.812 INFO:tasks.workunit.client.0.smithi067.stderr: 87600K .......... .......... .......... .......... .......... 64% 38.5M 2s 2024-07-31T14:53:04.814 INFO:tasks.workunit.client.0.smithi067.stderr: 87650K .......... .......... .......... .......... .......... 64% 23.8M 2s 2024-07-31T14:53:04.817 INFO:tasks.workunit.client.0.smithi067.stderr: 87700K .......... .......... .......... .......... .......... 64% 20.0M 2s 2024-07-31T14:53:04.817 INFO:tasks.workunit.client.0.smithi067.stderr: 87750K .......... .......... .......... .......... .......... 64% 89.0M 2s 2024-07-31T14:53:04.819 INFO:tasks.workunit.client.0.smithi067.stderr: 87800K .......... .......... .......... .......... .......... 64% 27.3M 2s 2024-07-31T14:53:04.821 INFO:tasks.workunit.client.0.smithi067.stderr: 87850K .......... .......... .......... .......... .......... 64% 28.6M 2s 2024-07-31T14:53:04.823 INFO:tasks.workunit.client.0.smithi067.stderr: 87900K .......... .......... .......... .......... .......... 64% 25.6M 2s 2024-07-31T14:53:04.825 INFO:tasks.workunit.client.0.smithi067.stderr: 87950K .......... .......... .......... .......... .......... 64% 24.5M 2s 2024-07-31T14:53:04.825 INFO:tasks.workunit.client.0.smithi067.stderr: 88000K .......... .......... .......... .......... .......... 64% 49.2M 2s 2024-07-31T14:53:04.827 INFO:tasks.workunit.client.0.smithi067.stderr: 88050K .......... .......... .......... .......... .......... 64% 23.1M 2s 2024-07-31T14:53:04.829 INFO:tasks.workunit.client.0.smithi067.stderr: 88100K .......... .......... .......... .......... .......... 64% 38.9M 2s 2024-07-31T14:53:04.831 INFO:tasks.workunit.client.0.smithi067.stderr: 88150K .......... .......... .......... .......... .......... 64% 23.8M 2s 2024-07-31T14:53:04.833 INFO:tasks.workunit.client.0.smithi067.stderr: 88200K .......... .......... .......... .......... .......... 65% 25.2M 2s 2024-07-31T14:53:04.834 INFO:tasks.workunit.client.0.smithi067.stderr: 88250K .......... .......... .......... .......... .......... 65% 52.1M 2s 2024-07-31T14:53:04.836 INFO:tasks.workunit.client.0.smithi067.stderr: 88300K .......... .......... .......... .......... .......... 65% 23.6M 2s 2024-07-31T14:53:04.837 INFO:tasks.workunit.client.0.smithi067.stderr: 88350K .......... .......... .......... .......... .......... 65% 40.3M 2s 2024-07-31T14:53:04.839 INFO:tasks.workunit.client.0.smithi067.stderr: 88400K .......... .......... .......... .......... .......... 65% 23.8M 2s 2024-07-31T14:53:04.841 INFO:tasks.workunit.client.0.smithi067.stderr: 88450K .......... .......... .......... .......... .......... 65% 23.8M 2s 2024-07-31T14:53:04.842 INFO:tasks.workunit.client.0.smithi067.stderr: 88500K .......... .......... .......... .......... .......... 65% 49.6M 2s 2024-07-31T14:53:04.844 INFO:tasks.workunit.client.0.smithi067.stderr: 88550K .......... .......... .......... .......... .......... 65% 24.1M 2s 2024-07-31T14:53:04.845 INFO:tasks.workunit.client.0.smithi067.stderr: 88600K .......... .......... .......... .......... .......... 65% 38.4M 2s 2024-07-31T14:53:04.848 INFO:tasks.workunit.client.0.smithi067.stderr: 88650K .......... .......... .......... .......... .......... 65% 23.7M 2s 2024-07-31T14:53:04.850 INFO:tasks.workunit.client.0.smithi067.stderr: 88700K .......... .......... .......... .......... .......... 65% 24.0M 2s 2024-07-31T14:53:04.850 INFO:tasks.workunit.client.0.smithi067.stderr: 88750K .......... .......... .......... .......... .......... 65% 56.2M 2s 2024-07-31T14:53:04.852 INFO:tasks.workunit.client.0.smithi067.stderr: 88800K .......... .......... .......... .......... .......... 65% 26.8M 2s 2024-07-31T14:53:04.855 INFO:tasks.workunit.client.0.smithi067.stderr: 88850K .......... .......... .......... .......... .......... 65% 19.9M 2s 2024-07-31T14:53:04.856 INFO:tasks.workunit.client.0.smithi067.stderr: 88900K .......... .......... .......... .......... .......... 65% 38.9M 2s 2024-07-31T14:53:04.859 INFO:tasks.workunit.client.0.smithi067.stderr: 88950K .......... .......... .......... .......... .......... 65% 23.9M 2s 2024-07-31T14:53:04.859 INFO:tasks.workunit.client.0.smithi067.stderr: 89000K .......... .......... .......... .......... .......... 65% 57.1M 2s 2024-07-31T14:53:04.861 INFO:tasks.workunit.client.0.smithi067.stderr: 89050K .......... .......... .......... .......... .......... 65% 21.4M 2s 2024-07-31T14:53:04.863 INFO:tasks.workunit.client.0.smithi067.stderr: 89100K .......... .......... .......... .......... .......... 65% 23.5M 2s 2024-07-31T14:53:04.865 INFO:tasks.workunit.client.0.smithi067.stderr: 89150K .......... .......... .......... .......... .......... 65% 42.3M 2s 2024-07-31T14:53:04.866 INFO:tasks.workunit.client.0.smithi067.stderr: 89200K .......... .......... .......... .......... .......... 65% 23.0M 2s 2024-07-31T14:53:04.867 INFO:tasks.workunit.client.0.smithi067.stderr: 89250K .......... .......... .......... .......... .......... 65% 59.9M 2s 2024-07-31T14:53:04.869 INFO:tasks.workunit.client.0.smithi067.stderr: 89300K .......... .......... .......... .......... .......... 65% 22.6M 2s 2024-07-31T14:53:04.871 INFO:tasks.workunit.client.0.smithi067.stderr: 89350K .......... .......... .......... .......... .......... 65% 23.5M 2s 2024-07-31T14:53:04.873 INFO:tasks.workunit.client.0.smithi067.stderr: 89400K .......... .......... .......... .......... .......... 65% 37.9M 2s 2024-07-31T14:53:04.875 INFO:tasks.workunit.client.0.smithi067.stderr: 89450K .......... .......... .......... .......... .......... 65% 23.8M 2s 2024-07-31T14:53:04.876 INFO:tasks.workunit.client.0.smithi067.stderr: 89500K .......... .......... .......... .......... .......... 65% 44.7M 2s 2024-07-31T14:53:04.878 INFO:tasks.workunit.client.0.smithi067.stderr: 89550K .......... .......... .......... .......... .......... 66% 25.0M 2s 2024-07-31T14:53:04.880 INFO:tasks.workunit.client.0.smithi067.stderr: 89600K .......... .......... .......... .......... .......... 66% 24.5M 2s 2024-07-31T14:53:04.881 INFO:tasks.workunit.client.0.smithi067.stderr: 89650K .......... .......... .......... .......... .......... 66% 39.0M 2s 2024-07-31T14:53:04.884 INFO:tasks.workunit.client.0.smithi067.stderr: 89700K .......... .......... .......... .......... .......... 66% 20.2M 2s 2024-07-31T14:53:04.886 INFO:tasks.workunit.client.0.smithi067.stderr: 89750K .......... .......... .......... .......... .......... 66% 51.7M 2s 2024-07-31T14:53:04.887 INFO:tasks.workunit.client.0.smithi067.stderr: 89800K .......... .......... .......... .......... .......... 66% 24.4M 2s 2024-07-31T14:53:04.889 INFO:tasks.workunit.client.0.smithi067.stderr: 89850K .......... .......... .......... .......... .......... 66% 23.7M 2s 2024-07-31T14:53:04.890 INFO:tasks.workunit.client.0.smithi067.stderr: 89900K .......... .......... .......... .......... .......... 66% 34.6M 2s 2024-07-31T14:53:04.892 INFO:tasks.workunit.client.0.smithi067.stderr: 89950K .......... .......... .......... .......... .......... 66% 25.4M 2s 2024-07-31T14:53:04.894 INFO:tasks.workunit.client.0.smithi067.stderr: 90000K .......... .......... .......... .......... .......... 66% 48.5M 2s 2024-07-31T14:53:04.895 INFO:tasks.workunit.client.0.smithi067.stderr: 90050K .......... .......... .......... .......... .......... 66% 24.8M 2s 2024-07-31T14:53:04.897 INFO:tasks.workunit.client.0.smithi067.stderr: 90100K .......... .......... .......... .......... .......... 66% 23.5M 2s 2024-07-31T14:53:04.898 INFO:tasks.workunit.client.0.smithi067.stderr: 90150K .......... .......... .......... .......... .......... 66% 36.3M 2s 2024-07-31T14:53:04.900 INFO:tasks.workunit.client.0.smithi067.stderr: 90200K .......... .......... .......... .......... .......... 66% 25.2M 2s 2024-07-31T14:53:04.902 INFO:tasks.workunit.client.0.smithi067.stderr: 90250K .......... .......... .......... .......... .......... 66% 23.8M 2s 2024-07-31T14:53:04.903 INFO:tasks.workunit.client.0.smithi067.stderr: 90300K .......... .......... .......... .......... .......... 66% 49.5M 2s 2024-07-31T14:53:04.906 INFO:tasks.workunit.client.0.smithi067.stderr: 90350K .......... .......... .......... .......... .......... 66% 19.0M 2s 2024-07-31T14:53:04.907 INFO:tasks.workunit.client.0.smithi067.stderr: 90400K .......... .......... .......... .......... .......... 66% 35.9M 2s 2024-07-31T14:53:04.909 INFO:tasks.workunit.client.0.smithi067.stderr: 90450K .......... .......... .......... .......... .......... 66% 25.7M 2s 2024-07-31T14:53:04.911 INFO:tasks.workunit.client.0.smithi067.stderr: 90500K .......... .......... .......... .......... .......... 66% 24.3M 2s 2024-07-31T14:53:04.912 INFO:tasks.workunit.client.0.smithi067.stderr: 90550K .......... .......... .......... .......... .......... 66% 48.9M 2s 2024-07-31T14:53:04.914 INFO:tasks.workunit.client.0.smithi067.stderr: 90600K .......... .......... .......... .......... .......... 66% 23.0M 2s 2024-07-31T14:53:04.915 INFO:tasks.workunit.client.0.smithi067.stderr: 90650K .......... .......... .......... .......... .......... 66% 36.0M 2s 2024-07-31T14:53:04.919 INFO:tasks.workunit.client.0.smithi067.stderr: 90700K .......... .......... .......... .......... .......... 66% 22.6M 2s 2024-07-31T14:53:04.919 INFO:tasks.workunit.client.0.smithi067.stderr: 90750K .......... .......... .......... .......... .......... 66% 51.0M 2s 2024-07-31T14:53:04.920 INFO:tasks.workunit.client.0.smithi067.stderr: 90800K .......... .......... .......... .......... .......... 66% 27.9M 2s 2024-07-31T14:53:04.922 INFO:tasks.workunit.client.0.smithi067.stderr: 90850K .......... .......... .......... .......... .......... 66% 35.3M 2s 2024-07-31T14:53:04.924 INFO:tasks.workunit.client.0.smithi067.stderr: 90900K .......... .......... .......... .......... .......... 66% 23.3M 2s 2024-07-31T14:53:04.926 INFO:tasks.workunit.client.0.smithi067.stderr: 90950K .......... .......... .......... .......... .......... 67% 26.7M 2s 2024-07-31T14:53:04.927 INFO:tasks.workunit.client.0.smithi067.stderr: 91000K .......... .......... .......... .......... .......... 67% 38.1M 2s 2024-07-31T14:53:04.929 INFO:tasks.workunit.client.0.smithi067.stderr: 91050K .......... .......... .......... .......... .......... 67% 26.2M 2s 2024-07-31T14:53:04.931 INFO:tasks.workunit.client.0.smithi067.stderr: 91100K .......... .......... .......... .......... .......... 67% 28.8M 2s 2024-07-31T14:53:04.932 INFO:tasks.workunit.client.0.smithi067.stderr: 91150K .......... .......... .......... .......... .......... 67% 30.1M 2s 2024-07-31T14:53:04.934 INFO:tasks.workunit.client.0.smithi067.stderr: 91200K .......... .......... .......... .......... .......... 67% 27.1M 2s 2024-07-31T14:53:04.936 INFO:tasks.workunit.client.0.smithi067.stderr: 91250K .......... .......... .......... .......... .......... 67% 21.3M 2s 2024-07-31T14:53:04.937 INFO:tasks.workunit.client.0.smithi067.stderr: 91300K .......... .......... .......... .......... .......... 67% 49.7M 2s 2024-07-31T14:53:04.939 INFO:tasks.workunit.client.0.smithi067.stderr: 91350K .......... .......... .......... .......... .......... 67% 25.0M 2s 2024-07-31T14:53:04.941 INFO:tasks.workunit.client.0.smithi067.stderr: 91400K .......... .......... .......... .......... .......... 67% 24.5M 2s 2024-07-31T14:53:04.943 INFO:tasks.workunit.client.0.smithi067.stderr: 91450K .......... .......... .......... .......... .......... 67% 30.1M 2s 2024-07-31T14:53:04.944 INFO:tasks.workunit.client.0.smithi067.stderr: 91500K .......... .......... .......... .......... .......... 67% 48.2M 2s 2024-07-31T14:53:04.946 INFO:tasks.workunit.client.0.smithi067.stderr: 91550K .......... .......... .......... .......... .......... 67% 25.3M 2s 2024-07-31T14:53:04.948 INFO:tasks.workunit.client.0.smithi067.stderr: 91600K .......... .......... .......... .......... .......... 67% 23.9M 2s 2024-07-31T14:53:04.949 INFO:tasks.workunit.client.0.smithi067.stderr: 91650K .......... .......... .......... .......... .......... 67% 39.0M 2s 2024-07-31T14:53:04.951 INFO:tasks.workunit.client.0.smithi067.stderr: 91700K .......... .......... .......... .......... .......... 67% 23.2M 2s 2024-07-31T14:53:04.953 INFO:tasks.workunit.client.0.smithi067.stderr: 91750K .......... .......... .......... .......... .......... 67% 34.3M 2s 2024-07-31T14:53:04.954 INFO:tasks.workunit.client.0.smithi067.stderr: 91800K .......... .......... .......... .......... .......... 67% 36.2M 2s 2024-07-31T14:53:04.956 INFO:tasks.workunit.client.0.smithi067.stderr: 91850K .......... .......... .......... .......... .......... 67% 26.4M 2s 2024-07-31T14:53:04.958 INFO:tasks.workunit.client.0.smithi067.stderr: 91900K .......... .......... .......... .......... .......... 67% 18.7M 2s 2024-07-31T14:53:04.961 INFO:tasks.workunit.client.0.smithi067.stderr: 91950K .......... .......... .......... .......... .......... 67% 20.3M 2s 2024-07-31T14:53:04.963 INFO:tasks.workunit.client.0.smithi067.stderr: 92000K .......... .......... .......... .......... .......... 67% 29.2M 2s 2024-07-31T14:53:04.963 INFO:tasks.workunit.client.0.smithi067.stderr: 92050K .......... .......... .......... .......... .......... 67% 50.3M 2s 2024-07-31T14:53:04.965 INFO:tasks.workunit.client.0.smithi067.stderr: 92100K .......... .......... .......... .......... .......... 67% 29.9M 2s 2024-07-31T14:53:04.967 INFO:tasks.workunit.client.0.smithi067.stderr: 92150K .......... .......... .......... .......... .......... 67% 29.4M 2s 2024-07-31T14:53:04.969 INFO:tasks.workunit.client.0.smithi067.stderr: 92200K .......... .......... .......... .......... .......... 67% 33.3M 2s 2024-07-31T14:53:04.970 INFO:tasks.workunit.client.0.smithi067.stderr: 92250K .......... .......... .......... .......... .......... 67% 25.2M 2s 2024-07-31T14:53:04.972 INFO:tasks.workunit.client.0.smithi067.stderr: 92300K .......... .......... .......... .......... .......... 68% 31.1M 2s 2024-07-31T14:53:04.974 INFO:tasks.workunit.client.0.smithi067.stderr: 92350K .......... .......... .......... .......... .......... 68% 22.7M 2s 2024-07-31T14:53:04.975 INFO:tasks.workunit.client.0.smithi067.stderr: 92400K .......... .......... .......... .......... .......... 68% 38.3M 1s 2024-07-31T14:53:04.977 INFO:tasks.workunit.client.0.smithi067.stderr: 92450K .......... .......... .......... .......... .......... 68% 24.7M 1s 2024-07-31T14:53:04.979 INFO:tasks.workunit.client.0.smithi067.stderr: 92500K .......... .......... .......... .......... .......... 68% 27.2M 1s 2024-07-31T14:53:04.980 INFO:tasks.workunit.client.0.smithi067.stderr: 92550K .......... .......... .......... .......... .......... 68% 41.0M 1s 2024-07-31T14:53:04.982 INFO:tasks.workunit.client.0.smithi067.stderr: 92600K .......... .......... .......... .......... .......... 68% 25.6M 1s 2024-07-31T14:53:04.984 INFO:tasks.workunit.client.0.smithi067.stderr: 92650K .......... .......... .......... .......... .......... 68% 30.3M 1s 2024-07-31T14:53:04.986 INFO:tasks.workunit.client.0.smithi067.stderr: 92700K .......... .......... .......... .......... .......... 68% 25.0M 1s 2024-07-31T14:53:04.988 INFO:tasks.workunit.client.0.smithi067.stderr: 92750K .......... .......... .......... .......... .......... 68% 25.0M 1s 2024-07-31T14:53:04.989 INFO:tasks.workunit.client.0.smithi067.stderr: 92800K .......... .......... .......... .......... .......... 68% 39.6M 1s 2024-07-31T14:53:04.991 INFO:tasks.workunit.client.0.smithi067.stderr: 92850K .......... .......... .......... .......... .......... 68% 23.7M 1s 2024-07-31T14:53:04.993 INFO:tasks.workunit.client.0.smithi067.stderr: 92900K .......... .......... .......... .......... .......... 68% 24.6M 1s 2024-07-31T14:53:04.994 INFO:tasks.workunit.client.0.smithi067.stderr: 92950K .......... .......... .......... .......... .......... 68% 42.3M 1s 2024-07-31T14:53:04.996 INFO:tasks.workunit.client.0.smithi067.stderr: 93000K .......... .......... .......... .......... .......... 68% 25.5M 1s 2024-07-31T14:53:04.997 INFO:tasks.workunit.client.0.smithi067.stderr: 93050K .......... .......... .......... .......... .......... 68% 39.5M 1s 2024-07-31T14:53:04.999 INFO:tasks.workunit.client.0.smithi067.stderr: 93100K .......... .......... .......... .......... .......... 68% 25.5M 1s 2024-07-31T14:53:05.000 INFO:tasks.workunit.client.0.smithi067.stderr: 93150K .......... .......... .......... .......... .......... 68% 40.0M 1s 2024-07-31T14:53:05.003 INFO:tasks.workunit.client.0.smithi067.stderr: 93200K .......... .......... .......... .......... .......... 68% 30.6M 1s 2024-07-31T14:53:05.005 INFO:tasks.workunit.client.0.smithi067.stderr: 93250K .......... .......... .......... .......... .......... 68% 18.7M 1s 2024-07-31T14:53:05.005 INFO:tasks.workunit.client.0.smithi067.stderr: 93300K .......... .......... .......... .......... .......... 68% 48.0M 1s 2024-07-31T14:53:05.007 INFO:tasks.workunit.client.0.smithi067.stderr: 93350K .......... .......... .......... .......... .......... 68% 25.9M 1s 2024-07-31T14:53:05.009 INFO:tasks.workunit.client.0.smithi067.stderr: 93400K .......... .......... .......... .......... .......... 68% 39.9M 1s 2024-07-31T14:53:05.012 INFO:tasks.workunit.client.0.smithi067.stderr: 93450K .......... .......... .......... .......... .......... 68% 13.2M 1s 2024-07-31T14:53:05.015 INFO:tasks.workunit.client.0.smithi067.stderr: 93500K .......... .......... .......... .......... .......... 68% 21.0M 1s 2024-07-31T14:53:05.015 INFO:tasks.workunit.client.0.smithi067.stderr: 93550K .......... .......... .......... .......... .......... 68% 46.7M 1s 2024-07-31T14:53:05.017 INFO:tasks.workunit.client.0.smithi067.stderr: 93600K .......... .......... .......... .......... .......... 68% 28.0M 1s 2024-07-31T14:53:05.020 INFO:tasks.workunit.client.0.smithi067.stderr: 93650K .......... .......... .......... .......... .......... 69% 20.4M 1s 2024-07-31T14:53:05.021 INFO:tasks.workunit.client.0.smithi067.stderr: 93700K .......... .......... .......... .......... .......... 69% 35.9M 1s 2024-07-31T14:53:05.023 INFO:tasks.workunit.client.0.smithi067.stderr: 93750K .......... .......... .......... .......... .......... 69% 30.9M 1s 2024-07-31T14:53:05.024 INFO:tasks.workunit.client.0.smithi067.stderr: 93800K .......... .......... .......... .......... .......... 69% 36.0M 1s 2024-07-31T14:53:05.026 INFO:tasks.workunit.client.0.smithi067.stderr: 93850K .......... .......... .......... .......... .......... 69% 23.9M 1s 2024-07-31T14:53:05.028 INFO:tasks.workunit.client.0.smithi067.stderr: 93900K .......... .......... .......... .......... .......... 69% 23.6M 1s 2024-07-31T14:53:05.029 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:04 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T14:53:05.029 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:05.029 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:05.029 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:05.029 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:04 smithi067 ceph-mon[32750]: pgmap v459: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-31T14:53:05.030 INFO:tasks.workunit.client.0.smithi067.stderr: 93950K .......... .......... .......... .......... .......... 69% 41.9M 1s 2024-07-31T14:53:05.031 INFO:tasks.workunit.client.0.smithi067.stderr: 94000K .......... .......... .......... .......... .......... 69% 23.4M 1s 2024-07-31T14:53:05.032 INFO:tasks.workunit.client.0.smithi067.stderr: 94050K .......... .......... .......... .......... .......... 69% 56.0M 1s 2024-07-31T14:53:05.035 INFO:tasks.workunit.client.0.smithi067.stderr: 94100K .......... .......... .......... .......... .......... 69% 23.4M 1s 2024-07-31T14:53:05.037 INFO:tasks.workunit.client.0.smithi067.stderr: 94150K .......... .......... .......... .......... .......... 69% 22.5M 1s 2024-07-31T14:53:05.038 INFO:tasks.workunit.client.0.smithi067.stderr: 94200K .......... .......... .......... .......... .......... 69% 37.8M 1s 2024-07-31T14:53:05.040 INFO:tasks.workunit.client.0.smithi067.stderr: 94250K .......... .......... .......... .......... .......... 69% 24.4M 1s 2024-07-31T14:53:05.040 INFO:tasks.workunit.client.0.smithi067.stderr: 94300K .......... .......... .......... .......... .......... 69% 60.7M 1s 2024-07-31T14:53:05.043 INFO:tasks.workunit.client.0.smithi067.stderr: 94350K .......... .......... .......... .......... .......... 69% 22.0M 1s 2024-07-31T14:53:05.045 INFO:tasks.workunit.client.0.smithi067.stderr: 94400K .......... .......... .......... .......... .......... 69% 23.7M 1s 2024-07-31T14:53:05.046 INFO:tasks.workunit.client.0.smithi067.stderr: 94450K .......... .......... .......... .......... .......... 69% 41.9M 1s 2024-07-31T14:53:05.048 INFO:tasks.workunit.client.0.smithi067.stderr: 94500K .......... .......... .......... .......... .......... 69% 21.7M 1s 2024-07-31T14:53:05.049 INFO:tasks.workunit.client.0.smithi067.stderr: 94550K .......... .......... .......... .......... .......... 69% 48.7M 1s 2024-07-31T14:53:05.052 INFO:tasks.workunit.client.0.smithi067.stderr: 94600K .......... .......... .......... .......... .......... 69% 22.4M 1s 2024-07-31T14:53:05.054 INFO:tasks.workunit.client.0.smithi067.stderr: 94650K .......... .......... .......... .......... .......... 69% 23.0M 1s 2024-07-31T14:53:05.055 INFO:tasks.workunit.client.0.smithi067.stderr: 94700K .......... .......... .......... .......... .......... 69% 37.6M 1s 2024-07-31T14:53:05.057 INFO:tasks.workunit.client.0.smithi067.stderr: 94750K .......... .......... .......... .......... .......... 69% 26.2M 1s 2024-07-31T14:53:05.058 INFO:tasks.workunit.client.0.smithi067.stderr: 94800K .......... .......... .......... .......... .......... 69% 45.5M 1s 2024-07-31T14:53:05.060 INFO:tasks.workunit.client.0.smithi067.stderr: 94850K .......... .......... .......... .......... .......... 69% 24.8M 1s 2024-07-31T14:53:05.062 INFO:tasks.workunit.client.0.smithi067.stderr: 94900K .......... .......... .......... .......... .......... 69% 23.2M 1s 2024-07-31T14:53:05.064 INFO:tasks.workunit.client.0.smithi067.stderr: 94950K .......... .......... .......... .......... .......... 69% 36.3M 1s 2024-07-31T14:53:05.069 INFO:tasks.workunit.client.0.smithi067.stderr: 95000K .......... .......... .......... .......... .......... 70% 9.11M 1s 2024-07-31T14:53:05.071 INFO:tasks.workunit.client.0.smithi067.stderr: 95050K .......... .......... .......... .......... .......... 70% 49.5M 1s 2024-07-31T14:53:05.072 INFO:tasks.workunit.client.0.smithi067.stderr: 95100K .......... .......... .......... .......... .......... 70% 20.1M 1s 2024-07-31T14:53:05.074 INFO:tasks.workunit.client.0.smithi067.stderr: 95150K .......... .......... .......... .......... .......... 70% 23.1M 1s 2024-07-31T14:53:05.075 INFO:tasks.workunit.client.0.smithi067.stderr: 95200K .......... .......... .......... .......... .......... 70% 46.5M 1s 2024-07-31T14:53:05.078 INFO:tasks.workunit.client.0.smithi067.stderr: 95250K .......... .......... .......... .......... .......... 70% 22.0M 1s 2024-07-31T14:53:05.079 INFO:tasks.workunit.client.0.smithi067.stderr: 95300K .......... .......... .......... .......... .......... 70% 50.4M 1s 2024-07-31T14:53:05.080 INFO:tasks.workunit.client.0.smithi067.stderr: 95350K .......... .......... .......... .......... .......... 70% 24.0M 1s 2024-07-31T14:53:05.082 INFO:tasks.workunit.client.0.smithi067.stderr: 95400K .......... .......... .......... .......... .......... 70% 24.6M 1s 2024-07-31T14:53:05.085 INFO:tasks.workunit.client.0.smithi067.stderr: 95450K .......... .......... .......... .......... .......... 70% 25.1M 1s 2024-07-31T14:53:05.086 INFO:tasks.workunit.client.0.smithi067.stderr: 95500K .......... .......... .......... .......... .......... 70% 34.4M 1s 2024-07-31T14:53:05.089 INFO:tasks.workunit.client.0.smithi067.stderr: 95550K .......... .......... .......... .......... .......... 70% 24.5M 1s 2024-07-31T14:53:05.089 INFO:tasks.workunit.client.0.smithi067.stderr: 95600K .......... .......... .......... .......... .......... 70% 51.9M 1s 2024-07-31T14:53:05.091 INFO:tasks.workunit.client.0.smithi067.stderr: 95650K .......... .......... .......... .......... .......... 70% 24.3M 1s 2024-07-31T14:53:05.092 INFO:tasks.workunit.client.0.smithi067.stderr: 95700K .......... .......... .......... .......... .......... 70% 37.9M 1s 2024-07-31T14:53:05.095 INFO:tasks.workunit.client.0.smithi067.stderr: 95750K .......... .......... .......... .......... .......... 70% 23.0M 1s 2024-07-31T14:53:05.096 INFO:tasks.workunit.client.0.smithi067.stderr: 95800K .......... .......... .......... .......... .......... 70% 25.2M 1s 2024-07-31T14:53:05.097 INFO:tasks.workunit.client.0.smithi067.stderr: 95850K .......... .......... .......... .......... .......... 70% 49.1M 1s 2024-07-31T14:53:05.099 INFO:tasks.workunit.client.0.smithi067.stderr: 95900K .......... .......... .......... .......... .......... 70% 35.0M 1s 2024-07-31T14:53:05.101 INFO:tasks.workunit.client.0.smithi067.stderr: 95950K .......... .......... .......... .......... .......... 70% 24.7M 1s 2024-07-31T14:53:05.103 INFO:tasks.workunit.client.0.smithi067.stderr: 96000K .......... .......... .......... .......... .......... 70% 24.0M 1s 2024-07-31T14:53:05.104 INFO:tasks.workunit.client.0.smithi067.stderr: 96050K .......... .......... .......... .......... .......... 70% 24.9M 1s 2024-07-31T14:53:05.105 INFO:tasks.workunit.client.0.smithi067.stderr: 96100K .......... .......... .......... .......... .......... 70% 71.2M 1s 2024-07-31T14:53:05.108 INFO:tasks.workunit.client.0.smithi067.stderr: 96150K .......... .......... .......... .......... .......... 70% 20.3M 1s 2024-07-31T14:53:05.109 INFO:tasks.workunit.client.0.smithi067.stderr: 96200K .......... .......... .......... .......... .......... 70% 37.5M 1s 2024-07-31T14:53:05.111 INFO:tasks.workunit.client.0.smithi067.stderr: 96250K .......... .......... .......... .......... .......... 70% 25.2M 1s 2024-07-31T14:53:05.112 INFO:tasks.workunit.client.0.smithi067.stderr: 96300K .......... .......... .......... .......... .......... 70% 32.7M 1s 2024-07-31T14:53:05.114 INFO:tasks.workunit.client.0.smithi067.stderr: 96350K .......... .......... .......... .......... .......... 71% 32.2M 1s 2024-07-31T14:53:05.116 INFO:tasks.workunit.client.0.smithi067.stderr: 96400K .......... .......... .......... .......... .......... 71% 31.0M 1s 2024-07-31T14:53:05.118 INFO:tasks.workunit.client.0.smithi067.stderr: 96450K .......... .......... .......... .......... .......... 71% 19.6M 1s 2024-07-31T14:53:05.119 INFO:tasks.workunit.client.0.smithi067.stderr: 96500K .......... .......... .......... .......... .......... 71% 35.8M 1s 2024-07-31T14:53:05.123 INFO:tasks.workunit.client.0.smithi067.stderr: 96550K .......... .......... .......... .......... .......... 71% 11.8M 1s 2024-07-31T14:53:05.126 INFO:tasks.workunit.client.0.smithi067.stderr: 96600K .......... .......... .......... .......... .......... 71% 15.0M 1s 2024-07-31T14:53:05.131 INFO:tasks.workunit.client.0.smithi067.stderr: 96650K .......... .......... .......... .......... .......... 71% 10.8M 1s 2024-07-31T14:53:05.134 INFO:tasks.workunit.client.0.smithi067.stderr: 96700K .......... .......... .......... .......... .......... 71% 21.5M 1s 2024-07-31T14:53:05.136 INFO:tasks.workunit.client.0.smithi067.stderr: 96750K .......... .......... .......... .......... .......... 71% 41.3M 1s 2024-07-31T14:53:05.137 INFO:tasks.workunit.client.0.smithi067.stderr: 96800K .......... .......... .......... .......... .......... 71% 19.0M 1s 2024-07-31T14:53:05.138 INFO:tasks.workunit.client.0.smithi067.stderr: 96850K .......... .......... .......... .......... .......... 71% 61.4M 1s 2024-07-31T14:53:05.140 INFO:tasks.workunit.client.0.smithi067.stderr: 96900K .......... .......... .......... .......... .......... 71% 25.0M 1s 2024-07-31T14:53:05.142 INFO:tasks.workunit.client.0.smithi067.stderr: 96950K .......... .......... .......... .......... .......... 71% 31.3M 1s 2024-07-31T14:53:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:04 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T14:53:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:04 smithi073 ceph-mon[36714]: pgmap v459: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-31T14:53:05.143 INFO:tasks.workunit.client.0.smithi067.stderr: 97000K .......... .......... .......... .......... .......... 71% 23.8M 1s 2024-07-31T14:53:05.146 INFO:tasks.workunit.client.0.smithi067.stderr: 97050K .......... .......... .......... .......... .......... 71% 23.9M 1s 2024-07-31T14:53:05.146 INFO:tasks.workunit.client.0.smithi067.stderr: 97100K .......... .......... .......... .......... .......... 71% 51.8M 1s 2024-07-31T14:53:05.156 INFO:tasks.workunit.client.0.smithi067.stderr: 97150K .......... .......... .......... .......... .......... 71% 7.32M 1s 2024-07-31T14:53:05.156 INFO:tasks.workunit.client.0.smithi067.stderr: 97200K .......... .......... .......... .......... .......... 71% 158M 1s 2024-07-31T14:53:05.156 INFO:tasks.workunit.client.0.smithi067.stderr: 97250K .......... .......... .......... .......... .......... 71% 416M 1s 2024-07-31T14:53:05.157 INFO:tasks.workunit.client.0.smithi067.stderr: 97300K .......... .......... .......... .......... .......... 71% 42.5M 1s 2024-07-31T14:53:05.157 INFO:tasks.workunit.client.0.smithi067.stderr: 97350K .......... .......... .......... .......... .......... 71% 92.2M 1s 2024-07-31T14:53:05.157 INFO:tasks.workunit.client.0.smithi067.stderr: 97400K .......... .......... .......... .......... .......... 71% 26.5M 1s 2024-07-31T14:53:05.159 INFO:tasks.workunit.client.0.smithi067.stderr: 97450K .......... .......... .......... .......... .......... 71% 41.9M 1s 2024-07-31T14:53:05.160 INFO:tasks.workunit.client.0.smithi067.stderr: 97500K .......... .......... .......... .......... .......... 71% 24.1M 1s 2024-07-31T14:53:05.163 INFO:tasks.workunit.client.0.smithi067.stderr: 97550K .......... .......... .......... .......... .......... 71% 23.4M 1s 2024-07-31T14:53:05.163 INFO:tasks.workunit.client.0.smithi067.stderr: 97600K .......... .......... .......... .......... .......... 71% 48.3M 1s 2024-07-31T14:53:05.166 INFO:tasks.workunit.client.0.smithi067.stderr: 97650K .......... .......... .......... .......... .......... 71% 21.7M 1s 2024-07-31T14:53:05.167 INFO:tasks.workunit.client.0.smithi067.stderr: 97700K .......... .......... .......... .......... .......... 72% 46.5M 1s 2024-07-31T14:53:05.169 INFO:tasks.workunit.client.0.smithi067.stderr: 97750K .......... .......... .......... .......... .......... 72% 23.7M 1s 2024-07-31T14:53:05.171 INFO:tasks.workunit.client.0.smithi067.stderr: 97800K .......... .......... .......... .......... .......... 72% 25.0M 1s 2024-07-31T14:53:05.172 INFO:tasks.workunit.client.0.smithi067.stderr: 97850K .......... .......... .......... .......... .......... 72% 45.7M 1s 2024-07-31T14:53:05.174 INFO:tasks.workunit.client.0.smithi067.stderr: 97900K .......... .......... .......... .......... .......... 72% 22.9M 1s 2024-07-31T14:53:05.176 INFO:tasks.workunit.client.0.smithi067.stderr: 97950K .......... .......... .......... .......... .......... 72% 36.1M 1s 2024-07-31T14:53:05.177 INFO:tasks.workunit.client.0.smithi067.stderr: 98000K .......... .......... .......... .......... .......... 72% 24.7M 1s 2024-07-31T14:53:05.180 INFO:tasks.workunit.client.0.smithi067.stderr: 98050K .......... .......... .......... .......... .......... 72% 22.7M 1s 2024-07-31T14:53:05.181 INFO:tasks.workunit.client.0.smithi067.stderr: 98100K .......... .......... .......... .......... .......... 72% 46.2M 1s 2024-07-31T14:53:05.183 INFO:tasks.workunit.client.0.smithi067.stderr: 98150K .......... .......... .......... .......... .......... 72% 19.7M 1s 2024-07-31T14:53:05.190 INFO:tasks.workunit.client.0.smithi067.stderr: 98200K .......... .......... .......... .......... .......... 72% 10.8M 1s 2024-07-31T14:53:05.194 INFO:tasks.workunit.client.0.smithi067.stderr: 98250K .......... .......... .......... .......... .......... 72% 7.80M 1s 2024-07-31T14:53:05.198 INFO:tasks.workunit.client.0.smithi067.stderr: 98300K .......... .......... .......... .......... .......... 72% 14.0M 1s 2024-07-31T14:53:05.201 INFO:tasks.workunit.client.0.smithi067.stderr: 98350K .......... .......... .......... .......... .......... 72% 14.8M 1s 2024-07-31T14:53:05.204 INFO:tasks.workunit.client.0.smithi067.stderr: 98400K .......... .......... .......... .......... .......... 72% 14.7M 1s 2024-07-31T14:53:05.206 INFO:tasks.workunit.client.0.smithi067.stderr: 98450K .......... .......... .......... .......... .......... 72% 25.5M 1s 2024-07-31T14:53:05.207 INFO:tasks.workunit.client.0.smithi067.stderr: 98500K .......... .......... .......... .......... .......... 72% 43.1M 1s 2024-07-31T14:53:05.209 INFO:tasks.workunit.client.0.smithi067.stderr: 98550K .......... .......... .......... .......... .......... 72% 24.3M 1s 2024-07-31T14:53:05.210 INFO:tasks.workunit.client.0.smithi067.stderr: 98600K .......... .......... .......... .......... .......... 72% 49.1M 1s 2024-07-31T14:53:05.212 INFO:tasks.workunit.client.0.smithi067.stderr: 98650K .......... .......... .......... .......... .......... 72% 21.9M 1s 2024-07-31T14:53:05.214 INFO:tasks.workunit.client.0.smithi067.stderr: 98700K .......... .......... .......... .......... .......... 72% 23.5M 1s 2024-07-31T14:53:05.215 INFO:tasks.workunit.client.0.smithi067.stderr: 98750K .......... .......... .......... .......... .......... 72% 41.4M 1s 2024-07-31T14:53:05.218 INFO:tasks.workunit.client.0.smithi067.stderr: 98800K .......... .......... .......... .......... .......... 72% 23.5M 1s 2024-07-31T14:53:05.219 INFO:tasks.workunit.client.0.smithi067.stderr: 98850K .......... .......... .......... .......... .......... 72% 41.8M 1s 2024-07-31T14:53:05.221 INFO:tasks.workunit.client.0.smithi067.stderr: 98900K .......... .......... .......... .......... .......... 72% 24.1M 1s 2024-07-31T14:53:05.223 INFO:tasks.workunit.client.0.smithi067.stderr: 98950K .......... .......... .......... .......... .......... 72% 24.3M 1s 2024-07-31T14:53:05.224 INFO:tasks.workunit.client.0.smithi067.stderr: 99000K .......... .......... .......... .......... .......... 72% 40.2M 1s 2024-07-31T14:53:05.226 INFO:tasks.workunit.client.0.smithi067.stderr: 99050K .......... .......... .......... .......... .......... 73% 25.0M 1s 2024-07-31T14:53:05.227 INFO:tasks.workunit.client.0.smithi067.stderr: 99100K .......... .......... .......... .......... .......... 73% 47.7M 1s 2024-07-31T14:53:05.229 INFO:tasks.workunit.client.0.smithi067.stderr: 99150K .......... .......... .......... .......... .......... 73% 23.0M 1s 2024-07-31T14:53:05.231 INFO:tasks.workunit.client.0.smithi067.stderr: 99200K .......... .......... .......... .......... .......... 73% 23.7M 1s 2024-07-31T14:53:05.232 INFO:tasks.workunit.client.0.smithi067.stderr: 99250K .......... .......... .......... .......... .......... 73% 41.0M 1s 2024-07-31T14:53:05.234 INFO:tasks.workunit.client.0.smithi067.stderr: 99300K .......... .......... .......... .......... .......... 73% 24.4M 1s 2024-07-31T14:53:05.235 INFO:tasks.workunit.client.0.smithi067.stderr: 99350K .......... .......... .......... .......... .......... 73% 46.4M 1s 2024-07-31T14:53:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:04 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T14:53:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:04 smithi077 ceph-mon[37327]: pgmap v459: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-31T14:53:05.238 INFO:tasks.workunit.client.0.smithi067.stderr: 99400K .......... .......... .......... .......... .......... 73% 23.1M 1s 2024-07-31T14:53:05.239 INFO:tasks.workunit.client.0.smithi067.stderr: 99450K .......... .......... .......... .......... .......... 73% 38.6M 1s 2024-07-31T14:53:05.241 INFO:tasks.workunit.client.0.smithi067.stderr: 99500K .......... .......... .......... .......... .......... 73% 25.0M 1s 2024-07-31T14:53:05.243 INFO:tasks.workunit.client.0.smithi067.stderr: 99550K .......... .......... .......... .......... .......... 73% 22.8M 1s 2024-07-31T14:53:05.244 INFO:tasks.workunit.client.0.smithi067.stderr: 99600K .......... .......... .......... .......... .......... 73% 45.7M 1s 2024-07-31T14:53:05.246 INFO:tasks.workunit.client.0.smithi067.stderr: 99650K .......... .......... .......... .......... .......... 73% 25.2M 1s 2024-07-31T14:53:05.248 INFO:tasks.workunit.client.0.smithi067.stderr: 99700K .......... .......... .......... .......... .......... 73% 19.9M 1s 2024-07-31T14:53:05.250 INFO:tasks.workunit.client.0.smithi067.stderr: 99750K .......... .......... .......... .......... .......... 73% 30.8M 1s 2024-07-31T14:53:05.253 INFO:tasks.workunit.client.0.smithi067.stderr: 99800K .......... .......... .......... .......... .......... 73% 16.2M 1s 2024-07-31T14:53:05.255 INFO:tasks.workunit.client.0.smithi067.stderr: 99850K .......... .......... .......... .......... .......... 73% 23.0M 1s 2024-07-31T14:53:05.258 INFO:tasks.workunit.client.0.smithi067.stderr: 99900K .......... .......... .......... .......... .......... 73% 15.9M 1s 2024-07-31T14:53:05.264 INFO:tasks.workunit.client.0.smithi067.stderr: 99950K .......... .......... .......... .......... .......... 73% 8.46M 1s 2024-07-31T14:53:05.266 INFO:tasks.workunit.client.0.smithi067.stderr:100000K .......... .......... .......... .......... .......... 73% 24.1M 1s 2024-07-31T14:53:05.271 INFO:tasks.workunit.client.0.smithi067.stderr:100050K .......... .......... .......... .......... .......... 73% 8.52M 1s 2024-07-31T14:53:05.273 INFO:tasks.workunit.client.0.smithi067.stderr:100100K .......... .......... .......... .......... .......... 73% 28.2M 1s 2024-07-31T14:53:05.276 INFO:tasks.workunit.client.0.smithi067.stderr:100150K .......... .......... .......... .......... .......... 73% 17.5M 1s 2024-07-31T14:53:05.277 INFO:tasks.workunit.client.0.smithi067.stderr:100200K .......... .......... .......... .......... .......... 73% 37.8M 1s 2024-07-31T14:53:05.279 INFO:tasks.workunit.client.0.smithi067.stderr:100250K .......... .......... .......... .......... .......... 73% 25.0M 1s 2024-07-31T14:53:05.281 INFO:tasks.workunit.client.0.smithi067.stderr:100300K .......... .......... .......... .......... .......... 73% 24.8M 1s 2024-07-31T14:53:05.283 INFO:tasks.workunit.client.0.smithi067.stderr:100350K .......... .......... .......... .......... .......... 73% 53.6M 1s 2024-07-31T14:53:05.284 INFO:tasks.workunit.client.0.smithi067.stderr:100400K .......... .......... .......... .......... .......... 73% 22.7M 1s 2024-07-31T14:53:05.286 INFO:tasks.workunit.client.0.smithi067.stderr:100450K .......... .......... .......... .......... .......... 74% 34.9M 1s 2024-07-31T14:53:05.288 INFO:tasks.workunit.client.0.smithi067.stderr:100500K .......... .......... .......... .......... .......... 74% 22.1M 1s 2024-07-31T14:53:05.290 INFO:tasks.workunit.client.0.smithi067.stderr:100550K .......... .......... .......... .......... .......... 74% 30.6M 1s 2024-07-31T14:53:05.292 INFO:tasks.workunit.client.0.smithi067.stderr:100600K .......... .......... .......... .......... .......... 74% 47.1M 1s 2024-07-31T14:53:05.293 INFO:tasks.workunit.client.0.smithi067.stderr:100650K .......... .......... .......... .......... .......... 74% 21.0M 1s 2024-07-31T14:53:05.295 INFO:tasks.workunit.client.0.smithi067.stderr:100700K .......... .......... .......... .......... .......... 74% 24.4M 1s 2024-07-31T14:53:05.296 INFO:tasks.workunit.client.0.smithi067.stderr:100750K .......... .......... .......... .......... .......... 74% 42.7M 1s 2024-07-31T14:53:05.298 INFO:tasks.workunit.client.0.smithi067.stderr:100800K .......... .......... .......... .......... .......... 74% 23.7M 1s 2024-07-31T14:53:05.300 INFO:tasks.workunit.client.0.smithi067.stderr:100850K .......... .......... .......... .......... .......... 74% 29.3M 1s 2024-07-31T14:53:05.302 INFO:tasks.workunit.client.0.smithi067.stderr:100900K .......... .......... .......... .......... .......... 74% 30.9M 1s 2024-07-31T14:53:05.303 INFO:tasks.workunit.client.0.smithi067.stderr:100950K .......... .......... .......... .......... .......... 74% 24.0M 1s 2024-07-31T14:53:05.305 INFO:tasks.workunit.client.0.smithi067.stderr:101000K .......... .......... .......... .......... .......... 74% 28.7M 1s 2024-07-31T14:53:05.307 INFO:tasks.workunit.client.0.smithi067.stderr:101050K .......... .......... .......... .......... .......... 74% 33.4M 1s 2024-07-31T14:53:05.309 INFO:tasks.workunit.client.0.smithi067.stderr:101100K .......... .......... .......... .......... .......... 74% 23.1M 1s 2024-07-31T14:53:05.310 INFO:tasks.workunit.client.0.smithi067.stderr:101150K .......... .......... .......... .......... .......... 74% 52.8M 1s 2024-07-31T14:53:05.312 INFO:tasks.workunit.client.0.smithi067.stderr:101200K .......... .......... .......... .......... .......... 74% 23.7M 1s 2024-07-31T14:53:05.313 INFO:tasks.workunit.client.0.smithi067.stderr:101250K .......... .......... .......... .......... .......... 74% 40.5M 1s 2024-07-31T14:53:05.315 INFO:tasks.workunit.client.0.smithi067.stderr:101300K .......... .......... .......... .......... .......... 74% 21.0M 1s 2024-07-31T14:53:05.318 INFO:tasks.workunit.client.0.smithi067.stderr:101350K .......... .......... .......... .......... .......... 74% 18.1M 1s 2024-07-31T14:53:05.319 INFO:tasks.workunit.client.0.smithi067.stderr:101400K .......... .......... .......... .......... .......... 74% 49.5M 1s 2024-07-31T14:53:05.322 INFO:tasks.workunit.client.0.smithi067.stderr:101450K .......... .......... .......... .......... .......... 74% 15.7M 1s 2024-07-31T14:53:05.324 INFO:tasks.workunit.client.0.smithi067.stderr:101500K .......... .......... .......... .......... .......... 74% 26.2M 1s 2024-07-31T14:53:05.328 INFO:tasks.workunit.client.0.smithi067.stderr:101550K .......... .......... .......... .......... .......... 74% 12.1M 1s 2024-07-31T14:53:05.331 INFO:tasks.workunit.client.0.smithi067.stderr:101600K .......... .......... .......... .......... .......... 74% 15.6M 1s 2024-07-31T14:53:05.334 INFO:tasks.workunit.client.0.smithi067.stderr:101650K .......... .......... .......... .......... .......... 74% 15.7M 1s 2024-07-31T14:53:05.338 INFO:tasks.workunit.client.0.smithi067.stderr:101700K .......... .......... .......... .......... .......... 74% 13.9M 1s 2024-07-31T14:53:05.459 INFO:tasks.workunit.client.0.smithi067.stderr:101750K .......... .......... .......... .......... .......... 74% 616K 1s 2024-07-31T14:53:05.459 INFO:tasks.workunit.client.0.smithi067.stderr:101800K .......... .......... .......... .......... .......... 75% 398M 1s 2024-07-31T14:53:05.459 INFO:tasks.workunit.client.0.smithi067.stderr:101850K .......... .......... .......... .......... .......... 75% 453M 1s 2024-07-31T14:53:05.459 INFO:tasks.workunit.client.0.smithi067.stderr:101900K .......... .......... .......... .......... .......... 75% 408M 1s 2024-07-31T14:53:05.459 INFO:tasks.workunit.client.0.smithi067.stderr:101950K .......... .......... .......... .......... .......... 75% 426M 1s 2024-07-31T14:53:05.459 INFO:tasks.workunit.client.0.smithi067.stderr:102000K .......... .......... .......... .......... .......... 75% 413M 1s 2024-07-31T14:53:05.459 INFO:tasks.workunit.client.0.smithi067.stderr:102050K .......... .......... .......... .......... .......... 75% 438M 1s 2024-07-31T14:53:05.459 INFO:tasks.workunit.client.0.smithi067.stderr:102100K .......... .......... .......... .......... .......... 75% 442M 1s 2024-07-31T14:53:05.459 INFO:tasks.workunit.client.0.smithi067.stderr:102150K .......... .......... .......... .......... .......... 75% 359M 1s 2024-07-31T14:53:05.459 INFO:tasks.workunit.client.0.smithi067.stderr:102200K .......... .......... .......... .......... .......... 75% 359M 1s 2024-07-31T14:53:05.459 INFO:tasks.workunit.client.0.smithi067.stderr:102250K .......... .......... .......... .......... .......... 75% 544M 1s 2024-07-31T14:53:05.460 INFO:tasks.workunit.client.0.smithi067.stderr:102300K .......... .......... .......... .......... .......... 75% 469M 1s 2024-07-31T14:53:05.460 INFO:tasks.workunit.client.0.smithi067.stderr:102350K .......... .......... .......... .......... .......... 75% 417M 1s 2024-07-31T14:53:05.460 INFO:tasks.workunit.client.0.smithi067.stderr:102400K .......... .......... .......... .......... .......... 75% 421M 1s 2024-07-31T14:53:05.460 INFO:tasks.workunit.client.0.smithi067.stderr:102450K .......... .......... .......... .......... .......... 75% 482M 1s 2024-07-31T14:53:05.460 INFO:tasks.workunit.client.0.smithi067.stderr:102500K .......... .......... .......... .......... .......... 75% 445M 1s 2024-07-31T14:53:05.460 INFO:tasks.workunit.client.0.smithi067.stderr:102550K .......... .......... .......... .......... .......... 75% 481M 1s 2024-07-31T14:53:05.460 INFO:tasks.workunit.client.0.smithi067.stderr:102600K .......... .......... .......... .......... .......... 75% 365M 1s 2024-07-31T14:53:05.460 INFO:tasks.workunit.client.0.smithi067.stderr:102650K .......... .......... .......... .......... .......... 75% 428M 1s 2024-07-31T14:53:05.460 INFO:tasks.workunit.client.0.smithi067.stderr:102700K .......... .......... .......... .......... .......... 75% 499M 1s 2024-07-31T14:53:05.460 INFO:tasks.workunit.client.0.smithi067.stderr:102750K .......... .......... .......... .......... .......... 75% 449M 1s 2024-07-31T14:53:05.460 INFO:tasks.workunit.client.0.smithi067.stderr:102800K .......... .......... .......... .......... .......... 75% 381M 1s 2024-07-31T14:53:05.460 INFO:tasks.workunit.client.0.smithi067.stderr:102850K .......... .......... .......... .......... .......... 75% 487M 1s 2024-07-31T14:53:05.460 INFO:tasks.workunit.client.0.smithi067.stderr:102900K .......... .......... .......... .......... .......... 75% 563M 1s 2024-07-31T14:53:05.460 INFO:tasks.workunit.client.0.smithi067.stderr:102950K .......... .......... .......... .......... .......... 75% 471M 1s 2024-07-31T14:53:05.461 INFO:tasks.workunit.client.0.smithi067.stderr:103000K .......... .......... .......... .......... .......... 75% 442M 1s 2024-07-31T14:53:05.461 INFO:tasks.workunit.client.0.smithi067.stderr:103050K .......... .......... .......... .......... .......... 75% 371M 1s 2024-07-31T14:53:05.461 INFO:tasks.workunit.client.0.smithi067.stderr:103100K .......... .......... .......... .......... .......... 75% 451M 1s 2024-07-31T14:53:05.461 INFO:tasks.workunit.client.0.smithi067.stderr:103150K .......... .......... .......... .......... .......... 76% 467M 1s 2024-07-31T14:53:05.461 INFO:tasks.workunit.client.0.smithi067.stderr:103200K .......... .......... .......... .......... .......... 76% 373M 1s 2024-07-31T14:53:05.461 INFO:tasks.workunit.client.0.smithi067.stderr:103250K .......... .......... .......... .......... .......... 76% 428M 1s 2024-07-31T14:53:05.461 INFO:tasks.workunit.client.0.smithi067.stderr:103300K .......... .......... .......... .......... .......... 76% 443M 1s 2024-07-31T14:53:05.461 INFO:tasks.workunit.client.0.smithi067.stderr:103350K .......... .......... .......... .......... .......... 76% 457M 1s 2024-07-31T14:53:05.461 INFO:tasks.workunit.client.0.smithi067.stderr:103400K .......... .......... .......... .......... .......... 76% 362M 1s 2024-07-31T14:53:05.461 INFO:tasks.workunit.client.0.smithi067.stderr:103450K .......... .......... .......... .......... .......... 76% 365M 1s 2024-07-31T14:53:05.461 INFO:tasks.workunit.client.0.smithi067.stderr:103500K .......... .......... .......... .......... .......... 76% 457M 1s 2024-07-31T14:53:05.461 INFO:tasks.workunit.client.0.smithi067.stderr:103550K .......... .......... .......... .......... .......... 76% 474M 1s 2024-07-31T14:53:05.461 INFO:tasks.workunit.client.0.smithi067.stderr:103600K .......... .......... .......... .......... .......... 76% 419M 1s 2024-07-31T14:53:05.461 INFO:tasks.workunit.client.0.smithi067.stderr:103650K .......... .......... .......... .......... .......... 76% 462M 1s 2024-07-31T14:53:05.462 INFO:tasks.workunit.client.0.smithi067.stderr:103700K .......... .......... .......... .......... .......... 76% 59.5M 1s 2024-07-31T14:53:05.462 INFO:tasks.workunit.client.0.smithi067.stderr:103750K .......... .......... .......... .......... .......... 76% 22.9M 1s 2024-07-31T14:53:05.462 INFO:tasks.workunit.client.0.smithi067.stderr:103800K .......... .......... .......... .......... .......... 76% 37.5M 1s 2024-07-31T14:53:05.462 INFO:tasks.workunit.client.0.smithi067.stderr:103850K .......... .......... .......... .......... .......... 76% 29.1M 1s 2024-07-31T14:53:05.462 INFO:tasks.workunit.client.0.smithi067.stderr:103900K .......... .......... .......... .......... .......... 76% 35.1M 1s 2024-07-31T14:53:05.462 INFO:tasks.workunit.client.0.smithi067.stderr:103950K .......... .......... .......... .......... .......... 76% 23.6M 1s 2024-07-31T14:53:05.462 INFO:tasks.workunit.client.0.smithi067.stderr:104000K .......... .......... .......... .......... .......... 76% 34.1M 1s 2024-07-31T14:53:05.462 INFO:tasks.workunit.client.0.smithi067.stderr:104050K .......... .......... .......... .......... .......... 76% 27.9M 1s 2024-07-31T14:53:05.462 INFO:tasks.workunit.client.0.smithi067.stderr:104100K .......... .......... .......... .......... .......... 76% 31.8M 1s 2024-07-31T14:53:05.462 INFO:tasks.workunit.client.0.smithi067.stderr:104150K .......... .......... .......... .......... .......... 76% 30.3M 1s 2024-07-31T14:53:05.462 INFO:tasks.workunit.client.0.smithi067.stderr:104200K .......... .......... .......... .......... .......... 76% 24.6M 1s 2024-07-31T14:53:05.462 INFO:tasks.workunit.client.0.smithi067.stderr:104250K .......... .......... .......... .......... .......... 76% 19.4M 1s 2024-07-31T14:53:05.462 INFO:tasks.workunit.client.0.smithi067.stderr:104300K .......... .......... .......... .......... .......... 76% 20.9M 1s 2024-07-31T14:53:05.462 INFO:tasks.workunit.client.0.smithi067.stderr:104350K .......... .......... .......... .......... .......... 76% 23.7M 1s 2024-07-31T14:53:05.463 INFO:tasks.workunit.client.0.smithi067.stderr:104400K .......... .......... .......... .......... .......... 76% 33.8M 1s 2024-07-31T14:53:05.463 INFO:tasks.workunit.client.0.smithi067.stderr:104450K .......... .......... .......... .......... .......... 76% 16.7M 1s 2024-07-31T14:53:05.464 INFO:tasks.workunit.client.0.smithi067.stderr:104500K .......... .......... .......... .......... .......... 77% 4.21M 1s 2024-07-31T14:53:05.465 INFO:tasks.workunit.client.0.smithi067.stderr:104550K .......... .......... .......... .......... .......... 77% 29.3M 1s 2024-07-31T14:53:05.467 INFO:tasks.workunit.client.0.smithi067.stderr:104600K .......... .......... .......... .......... .......... 77% 30.5M 1s 2024-07-31T14:53:05.469 INFO:tasks.workunit.client.0.smithi067.stderr:104650K .......... .......... .......... .......... .......... 77% 23.8M 1s 2024-07-31T14:53:05.470 INFO:tasks.workunit.client.0.smithi067.stderr:104700K .......... .......... .......... .......... .......... 77% 39.8M 1s 2024-07-31T14:53:05.473 INFO:tasks.workunit.client.0.smithi067.stderr:104750K .......... .......... .......... .......... .......... 77% 23.0M 1s 2024-07-31T14:53:05.474 INFO:tasks.workunit.client.0.smithi067.stderr:104800K .......... .......... .......... .......... .......... 77% 33.5M 1s 2024-07-31T14:53:05.475 INFO:tasks.workunit.client.0.smithi067.stderr:104850K .......... .......... .......... .......... .......... 77% 35.7M 1s 2024-07-31T14:53:05.477 INFO:tasks.workunit.client.0.smithi067.stderr:104900K .......... .......... .......... .......... .......... 77% 23.2M 1s 2024-07-31T14:53:05.479 INFO:tasks.workunit.client.0.smithi067.stderr:104950K .......... .......... .......... .......... .......... 77% 36.7M 1s 2024-07-31T14:53:05.481 INFO:tasks.workunit.client.0.smithi067.stderr:105000K .......... .......... .......... .......... .......... 77% 22.3M 1s 2024-07-31T14:53:05.483 INFO:tasks.workunit.client.0.smithi067.stderr:105050K .......... .......... .......... .......... .......... 77% 32.7M 1s 2024-07-31T14:53:05.484 INFO:tasks.workunit.client.0.smithi067.stderr:105100K .......... .......... .......... .......... .......... 77% 37.4M 1s 2024-07-31T14:53:05.486 INFO:tasks.workunit.client.0.smithi067.stderr:105150K .......... .......... .......... .......... .......... 77% 23.7M 1s 2024-07-31T14:53:05.487 INFO:tasks.workunit.client.0.smithi067.stderr:105200K .......... .......... .......... .......... .......... 77% 35.6M 1s 2024-07-31T14:53:05.490 INFO:tasks.workunit.client.0.smithi067.stderr:105250K .......... .......... .......... .......... .......... 77% 22.9M 1s 2024-07-31T14:53:05.491 INFO:tasks.workunit.client.0.smithi067.stderr:105300K .......... .......... .......... .......... .......... 77% 30.8M 1s 2024-07-31T14:53:05.492 INFO:tasks.workunit.client.0.smithi067.stderr:105350K .......... .......... .......... .......... .......... 77% 30.8M 1s 2024-07-31T14:53:05.495 INFO:tasks.workunit.client.0.smithi067.stderr:105400K .......... .......... .......... .......... .......... 77% 21.0M 1s 2024-07-31T14:53:05.497 INFO:tasks.workunit.client.0.smithi067.stderr:105450K .......... .......... .......... .......... .......... 77% 25.1M 1s 2024-07-31T14:53:05.498 INFO:tasks.workunit.client.0.smithi067.stderr:105500K .......... .......... .......... .......... .......... 77% 33.8M 1s 2024-07-31T14:53:05.500 INFO:tasks.workunit.client.0.smithi067.stderr:105550K .......... .......... .......... .......... .......... 77% 22.2M 1s 2024-07-31T14:53:05.503 INFO:tasks.workunit.client.0.smithi067.stderr:105600K .......... .......... .......... .......... .......... 77% 19.2M 1s 2024-07-31T14:53:05.506 INFO:tasks.workunit.client.0.smithi067.stderr:105650K .......... .......... .......... .......... .......... 77% 14.9M 1s 2024-07-31T14:53:05.511 INFO:tasks.workunit.client.0.smithi067.stderr:105700K .......... .......... .......... .......... .......... 77% 9.35M 1s 2024-07-31T14:53:05.513 INFO:tasks.workunit.client.0.smithi067.stderr:105750K .......... .......... .......... .......... .......... 77% 26.5M 1s 2024-07-31T14:53:05.519 INFO:tasks.workunit.client.0.smithi067.stderr:105800K .......... .......... .......... .......... .......... 77% 9.07M 1s 2024-07-31T14:53:05.520 INFO:tasks.workunit.client.0.smithi067.stderr:105850K .......... .......... .......... .......... .......... 78% 38.0M 1s 2024-07-31T14:53:05.522 INFO:tasks.workunit.client.0.smithi067.stderr:105900K .......... .......... .......... .......... .......... 78% 23.0M 1s 2024-07-31T14:53:05.523 INFO:tasks.workunit.client.0.smithi067.stderr:105950K .......... .......... .......... .......... .......... 78% 46.1M 1s 2024-07-31T14:53:05.525 INFO:tasks.workunit.client.0.smithi067.stderr:106000K .......... .......... .......... .......... .......... 78% 22.6M 1s 2024-07-31T14:53:05.527 INFO:tasks.workunit.client.0.smithi067.stderr:106050K .......... .......... .......... .......... .......... 78% 24.8M 1s 2024-07-31T14:53:05.528 INFO:tasks.workunit.client.0.smithi067.stderr:106100K .......... .......... .......... .......... .......... 78% 42.9M 1s 2024-07-31T14:53:05.530 INFO:tasks.workunit.client.0.smithi067.stderr:106150K .......... .......... .......... .......... .......... 78% 24.3M 1s 2024-07-31T14:53:05.532 INFO:tasks.workunit.client.0.smithi067.stderr:106200K .......... .......... .......... .......... .......... 78% 42.6M 1s 2024-07-31T14:53:05.534 INFO:tasks.workunit.client.0.smithi067.stderr:106250K .......... .......... .......... .......... .......... 78% 24.0M 1s 2024-07-31T14:53:05.536 INFO:tasks.workunit.client.0.smithi067.stderr:106300K .......... .......... .......... .......... .......... 78% 25.1M 1s 2024-07-31T14:53:05.537 INFO:tasks.workunit.client.0.smithi067.stderr:106350K .......... .......... .......... .......... .......... 78% 37.9M 1s 2024-07-31T14:53:05.539 INFO:tasks.workunit.client.0.smithi067.stderr:106400K .......... .......... .......... .......... .......... 78% 23.5M 1s 2024-07-31T14:53:05.540 INFO:tasks.workunit.client.0.smithi067.stderr:106450K .......... .......... .......... .......... .......... 78% 37.9M 1s 2024-07-31T14:53:05.542 INFO:tasks.workunit.client.0.smithi067.stderr:106500K .......... .......... .......... .......... .......... 78% 25.3M 1s 2024-07-31T14:53:05.545 INFO:tasks.workunit.client.0.smithi067.stderr:106550K .......... .......... .......... .......... .......... 78% 17.1M 1s 2024-07-31T14:53:05.546 INFO:tasks.workunit.client.0.smithi067.stderr:106600K .......... .......... .......... .......... .......... 78% 39.7M 1s 2024-07-31T14:53:05.549 INFO:tasks.workunit.client.0.smithi067.stderr:106650K .......... .......... .......... .......... .......... 78% 19.5M 1s 2024-07-31T14:53:05.551 INFO:tasks.workunit.client.0.smithi067.stderr:106700K .......... .......... .......... .......... .......... 78% 31.6M 1s 2024-07-31T14:53:05.552 INFO:tasks.workunit.client.0.smithi067.stderr:106750K .......... .......... .......... .......... .......... 78% 20.1M 1s 2024-07-31T14:53:05.556 INFO:tasks.workunit.client.0.smithi067.stderr:106800K .......... .......... .......... .......... .......... 78% 16.8M 1s 2024-07-31T14:53:05.557 INFO:tasks.workunit.client.0.smithi067.stderr:106850K .......... .......... .......... .......... .......... 78% 36.8M 1s 2024-07-31T14:53:05.562 INFO:tasks.workunit.client.0.smithi067.stderr:106900K .......... .......... .......... .......... .......... 78% 9.57M 1s 2024-07-31T14:53:05.564 INFO:tasks.workunit.client.0.smithi067.stderr:106950K .......... .......... .......... .......... .......... 78% 21.2M 1s 2024-07-31T14:53:05.570 INFO:tasks.workunit.client.0.smithi067.stderr:107000K .......... .......... .......... .......... .......... 78% 9.12M 1s 2024-07-31T14:53:05.571 INFO:tasks.workunit.client.0.smithi067.stderr:107050K .......... .......... .......... .......... .......... 78% 30.3M 1s 2024-07-31T14:53:05.573 INFO:tasks.workunit.client.0.smithi067.stderr:107100K .......... .......... .......... .......... .......... 78% 34.3M 1s 2024-07-31T14:53:05.575 INFO:tasks.workunit.client.0.smithi067.stderr:107150K .......... .......... .......... .......... .......... 78% 22.0M 1s 2024-07-31T14:53:05.576 INFO:tasks.workunit.client.0.smithi067.stderr:107200K .......... .......... .......... .......... .......... 79% 40.7M 1s 2024-07-31T14:53:05.579 INFO:tasks.workunit.client.0.smithi067.stderr:107250K .......... .......... .......... .......... .......... 79% 21.4M 1s 2024-07-31T14:53:05.581 INFO:tasks.workunit.client.0.smithi067.stderr:107300K .......... .......... .......... .......... .......... 79% 23.5M 1s 2024-07-31T14:53:05.582 INFO:tasks.workunit.client.0.smithi067.stderr:107350K .......... .......... .......... .......... .......... 79% 40.3M 1s 2024-07-31T14:53:05.584 INFO:tasks.workunit.client.0.smithi067.stderr:107400K .......... .......... .......... .......... .......... 79% 22.3M 1s 2024-07-31T14:53:05.585 INFO:tasks.workunit.client.0.smithi067.stderr:107450K .......... .......... .......... .......... .......... 79% 41.7M 1s 2024-07-31T14:53:05.587 INFO:tasks.workunit.client.0.smithi067.stderr:107500K .......... .......... .......... .......... .......... 79% 25.7M 1s 2024-07-31T14:53:05.589 INFO:tasks.workunit.client.0.smithi067.stderr:107550K .......... .......... .......... .......... .......... 79% 21.8M 1s 2024-07-31T14:53:05.590 INFO:tasks.workunit.client.0.smithi067.stderr:107600K .......... .......... .......... .......... .......... 79% 38.3M 1s 2024-07-31T14:53:05.594 INFO:tasks.workunit.client.0.smithi067.stderr:107650K .......... .......... .......... .......... .......... 79% 16.7M 1s 2024-07-31T14:53:05.594 INFO:tasks.workunit.client.0.smithi067.stderr:107700K .......... .......... .......... .......... .......... 79% 48.1M 1s 2024-07-31T14:53:05.597 INFO:tasks.workunit.client.0.smithi067.stderr:107750K .......... .......... .......... .......... .......... 79% 20.0M 1s 2024-07-31T14:53:05.599 INFO:tasks.workunit.client.0.smithi067.stderr:107800K .......... .......... .......... .......... .......... 79% 20.0M 1s 2024-07-31T14:53:05.600 INFO:tasks.workunit.client.0.smithi067.stderr:107850K .......... .......... .......... .......... .......... 79% 39.2M 1s 2024-07-31T14:53:05.603 INFO:tasks.workunit.client.0.smithi067.stderr:107900K .......... .......... .......... .......... .......... 79% 17.2M 1s 2024-07-31T14:53:05.605 INFO:tasks.workunit.client.0.smithi067.stderr:107950K .......... .......... .......... .......... .......... 79% 26.8M 1s 2024-07-31T14:53:05.609 INFO:tasks.workunit.client.0.smithi067.stderr:108000K .......... .......... .......... .......... .......... 79% 13.6M 1s 2024-07-31T14:53:05.612 INFO:tasks.workunit.client.0.smithi067.stderr:108050K .......... .......... .......... .......... .......... 79% 16.1M 1s 2024-07-31T14:53:05.614 INFO:tasks.workunit.client.0.smithi067.stderr:108100K .......... .......... .......... .......... .......... 79% 22.4M 1s 2024-07-31T14:53:05.617 INFO:tasks.workunit.client.0.smithi067.stderr:108150K .......... .......... .......... .......... .......... 79% 15.5M 1s 2024-07-31T14:53:05.620 INFO:tasks.workunit.client.0.smithi067.stderr:108200K .......... .......... .......... .......... .......... 79% 20.4M 1s 2024-07-31T14:53:05.621 INFO:tasks.workunit.client.0.smithi067.stderr:108250K .......... .......... .......... .......... .......... 79% 25.7M 1s 2024-07-31T14:53:05.624 INFO:tasks.workunit.client.0.smithi067.stderr:108300K .......... .......... .......... .......... .......... 79% 20.9M 1s 2024-07-31T14:53:05.625 INFO:tasks.workunit.client.0.smithi067.stderr:108350K .......... .......... .......... .......... .......... 79% 44.6M 1s 2024-07-31T14:53:05.627 INFO:tasks.workunit.client.0.smithi067.stderr:108400K .......... .......... .......... .......... .......... 79% 23.3M 1s 2024-07-31T14:53:05.629 INFO:tasks.workunit.client.0.smithi067.stderr:108450K .......... .......... .......... .......... .......... 79% 40.2M 1s 2024-07-31T14:53:05.630 INFO:tasks.workunit.client.0.smithi067.stderr:108500K .......... .......... .......... .......... .......... 79% 23.2M 1s 2024-07-31T14:53:05.632 INFO:tasks.workunit.client.0.smithi067.stderr:108550K .......... .......... .......... .......... .......... 79% 24.7M 1s 2024-07-31T14:53:05.633 INFO:tasks.workunit.client.0.smithi067.stderr:108600K .......... .......... .......... .......... .......... 80% 40.9M 1s 2024-07-31T14:53:05.635 INFO:tasks.workunit.client.0.smithi067.stderr:108650K .......... .......... .......... .......... .......... 80% 23.5M 1s 2024-07-31T14:53:05.637 INFO:tasks.workunit.client.0.smithi067.stderr:108700K .......... .......... .......... .......... .......... 80% 49.3M 1s 2024-07-31T14:53:05.639 INFO:tasks.workunit.client.0.smithi067.stderr:108750K .......... .......... .......... .......... .......... 80% 23.9M 1s 2024-07-31T14:53:05.641 INFO:tasks.workunit.client.0.smithi067.stderr:108800K .......... .......... .......... .......... .......... 80% 23.6M 1s 2024-07-31T14:53:05.642 INFO:tasks.workunit.client.0.smithi067.stderr:108850K .......... .......... .......... .......... .......... 80% 40.3M 1s 2024-07-31T14:53:05.644 INFO:tasks.workunit.client.0.smithi067.stderr:108900K .......... .......... .......... .......... .......... 80% 24.8M 1s 2024-07-31T14:53:05.646 INFO:tasks.workunit.client.0.smithi067.stderr:108950K .......... .......... .......... .......... .......... 80% 22.8M 1s 2024-07-31T14:53:05.647 INFO:tasks.workunit.client.0.smithi067.stderr:109000K .......... .......... .......... .......... .......... 80% 49.7M 1s 2024-07-31T14:53:05.649 INFO:tasks.workunit.client.0.smithi067.stderr:109050K .......... .......... .......... .......... .......... 80% 22.8M 1s 2024-07-31T14:53:05.651 INFO:tasks.workunit.client.0.smithi067.stderr:109100K .......... .......... .......... .......... .......... 80% 39.8M 1s 2024-07-31T14:53:05.652 INFO:tasks.workunit.client.0.smithi067.stderr:109150K .......... .......... .......... .......... .......... 80% 25.1M 1s 2024-07-31T14:53:05.654 INFO:tasks.workunit.client.0.smithi067.stderr:109200K .......... .......... .......... .......... .......... 80% 22.7M 1s 2024-07-31T14:53:05.655 INFO:tasks.workunit.client.0.smithi067.stderr:109250K .......... .......... .......... .......... .......... 80% 49.9M 1s 2024-07-31T14:53:05.657 INFO:tasks.workunit.client.0.smithi067.stderr:109300K .......... .......... .......... .......... .......... 80% 27.8M 1s 2024-07-31T14:53:05.660 INFO:tasks.workunit.client.0.smithi067.stderr:109350K .......... .......... .......... .......... .......... 80% 22.0M 1s 2024-07-31T14:53:05.672 INFO:tasks.workunit.client.0.smithi067.stderr:109400K .......... .......... .......... .......... .......... 80% 3.71M 1s 2024-07-31T14:53:05.674 INFO:tasks.workunit.client.0.smithi067.stderr:109450K .......... .......... .......... .......... .......... 80% 23.4M 1s 2024-07-31T14:53:05.675 INFO:tasks.workunit.client.0.smithi067.stderr:109500K .......... .......... .......... .......... .......... 80% 58.7M 1s 2024-07-31T14:53:05.678 INFO:tasks.workunit.client.0.smithi067.stderr:109550K .......... .......... .......... .......... .......... 80% 22.7M 1s 2024-07-31T14:53:05.680 INFO:tasks.workunit.client.0.smithi067.stderr:109600K .......... .......... .......... .......... .......... 80% 35.0M 1s 2024-07-31T14:53:05.681 INFO:tasks.workunit.client.0.smithi067.stderr:109650K .......... .......... .......... .......... .......... 80% 25.1M 1s 2024-07-31T14:53:05.683 INFO:tasks.workunit.client.0.smithi067.stderr:109700K .......... .......... .......... .......... .......... 80% 22.9M 1s 2024-07-31T14:53:05.684 INFO:tasks.workunit.client.0.smithi067.stderr:109750K .......... .......... .......... .......... .......... 80% 50.5M 1s 2024-07-31T14:53:05.686 INFO:tasks.workunit.client.0.smithi067.stderr:109800K .......... .......... .......... .......... .......... 80% 23.4M 1s 2024-07-31T14:53:05.687 INFO:tasks.workunit.client.0.smithi067.stderr:109850K .......... .......... .......... .......... .......... 80% 42.1M 1s 2024-07-31T14:53:05.689 INFO:tasks.workunit.client.0.smithi067.stderr:109900K .......... .......... .......... .......... .......... 80% 22.4M 1s 2024-07-31T14:53:05.691 INFO:tasks.workunit.client.0.smithi067.stderr:109950K .......... .......... .......... .......... .......... 81% 24.9M 1s 2024-07-31T14:53:05.693 INFO:tasks.workunit.client.0.smithi067.stderr:110000K .......... .......... .......... .......... .......... 81% 37.6M 1s 2024-07-31T14:53:05.695 INFO:tasks.workunit.client.0.smithi067.stderr:110050K .......... .......... .......... .......... .......... 81% 26.7M 1s 2024-07-31T14:53:05.697 INFO:tasks.workunit.client.0.smithi067.stderr:110100K .......... .......... .......... .......... .......... 81% 41.6M 1s 2024-07-31T14:53:05.698 INFO:tasks.workunit.client.0.smithi067.stderr:110150K .......... .......... .......... .......... .......... 81% 24.0M 1s 2024-07-31T14:53:05.700 INFO:tasks.workunit.client.0.smithi067.stderr:110200K .......... .......... .......... .......... .......... 81% 23.7M 1s 2024-07-31T14:53:05.701 INFO:tasks.workunit.client.0.smithi067.stderr:110250K .......... .......... .......... .......... .......... 81% 49.0M 1s 2024-07-31T14:53:05.703 INFO:tasks.workunit.client.0.smithi067.stderr:110300K .......... .......... .......... .......... .......... 81% 23.9M 1s 2024-07-31T14:53:05.705 INFO:tasks.workunit.client.0.smithi067.stderr:110350K .......... .......... .......... .......... .......... 81% 23.3M 1s 2024-07-31T14:53:05.706 INFO:tasks.workunit.client.0.smithi067.stderr:110400K .......... .......... .......... .......... .......... 81% 38.8M 1s 2024-07-31T14:53:05.708 INFO:tasks.workunit.client.0.smithi067.stderr:110450K .......... .......... .......... .......... .......... 81% 25.0M 1s 2024-07-31T14:53:05.710 INFO:tasks.workunit.client.0.smithi067.stderr:110500K .......... .......... .......... .......... .......... 81% 32.5M 1s 2024-07-31T14:53:05.714 INFO:tasks.workunit.client.0.smithi067.stderr:110550K .......... .......... .......... .......... .......... 81% 12.3M 1s 2024-07-31T14:53:05.718 INFO:tasks.workunit.client.0.smithi067.stderr:110600K .......... .......... .......... .......... .......... 81% 20.3M 1s 2024-07-31T14:53:05.721 INFO:tasks.workunit.client.0.smithi067.stderr:110650K .......... .......... .......... .......... .......... 81% 9.99M 1s 2024-07-31T14:53:05.730 INFO:tasks.workunit.client.0.smithi067.stderr:110700K .......... .......... .......... .......... .......... 81% 5.52M 1s 2024-07-31T14:53:05.731 INFO:tasks.workunit.client.0.smithi067.stderr:110750K .......... .......... .......... .......... .......... 81% 48.0M 1s 2024-07-31T14:53:05.733 INFO:tasks.workunit.client.0.smithi067.stderr:110800K .......... .......... .......... .......... .......... 81% 26.1M 1s 2024-07-31T14:53:05.735 INFO:tasks.workunit.client.0.smithi067.stderr:110850K .......... .......... .......... .......... .......... 81% 25.4M 1s 2024-07-31T14:53:05.736 INFO:tasks.workunit.client.0.smithi067.stderr:110900K .......... .......... .......... .......... .......... 81% 35.2M 1s 2024-07-31T14:53:05.738 INFO:tasks.workunit.client.0.smithi067.stderr:110950K .......... .......... .......... .......... .......... 81% 22.0M 1s 2024-07-31T14:53:05.739 INFO:tasks.workunit.client.0.smithi067.stderr:111000K .......... .......... .......... .......... .......... 81% 45.6M 1s 2024-07-31T14:53:05.741 INFO:tasks.workunit.client.0.smithi067.stderr:111050K .......... .......... .......... .......... .......... 81% 22.8M 1s 2024-07-31T14:53:05.743 INFO:tasks.workunit.client.0.smithi067.stderr:111100K .......... .......... .......... .......... .......... 81% 25.3M 1s 2024-07-31T14:53:05.744 INFO:tasks.workunit.client.0.smithi067.stderr:111150K .......... .......... .......... .......... .......... 81% 49.0M 1s 2024-07-31T14:53:05.747 INFO:tasks.workunit.client.0.smithi067.stderr:111200K .......... .......... .......... .......... .......... 81% 22.0M 1s 2024-07-31T14:53:05.748 INFO:tasks.workunit.client.0.smithi067.stderr:111250K .......... .......... .......... .......... .......... 81% 45.3M 1s 2024-07-31T14:53:05.750 INFO:tasks.workunit.client.0.smithi067.stderr:111300K .......... .......... .......... .......... .......... 82% 25.3M 1s 2024-07-31T14:53:05.752 INFO:tasks.workunit.client.0.smithi067.stderr:111350K .......... .......... .......... .......... .......... 82% 23.8M 1s 2024-07-31T14:53:05.753 INFO:tasks.workunit.client.0.smithi067.stderr:111400K .......... .......... .......... .......... .......... 82% 40.3M 1s 2024-07-31T14:53:05.755 INFO:tasks.workunit.client.0.smithi067.stderr:111450K .......... .......... .......... .......... .......... 82% 23.8M 1s 2024-07-31T14:53:05.756 INFO:tasks.workunit.client.0.smithi067.stderr:111500K .......... .......... .......... .......... .......... 82% 43.6M 1s 2024-07-31T14:53:05.758 INFO:tasks.workunit.client.0.smithi067.stderr:111550K .......... .......... .......... .......... .......... 82% 21.8M 1s 2024-07-31T14:53:05.760 INFO:tasks.workunit.client.0.smithi067.stderr:111600K .......... .......... .......... .......... .......... 82% 31.1M 1s 2024-07-31T14:53:05.763 INFO:tasks.workunit.client.0.smithi067.stderr:111650K .......... .......... .......... .......... .......... 82% 14.3M 1s 2024-07-31T14:53:05.766 INFO:tasks.workunit.client.0.smithi067.stderr:111700K .......... .......... .......... .......... .......... 82% 21.2M 1s 2024-07-31T14:53:05.767 INFO:tasks.workunit.client.0.smithi067.stderr:111750K .......... .......... .......... .......... .......... 82% 45.8M 1s 2024-07-31T14:53:05.770 INFO:tasks.workunit.client.0.smithi067.stderr:111800K .......... .......... .......... .......... .......... 82% 14.2M 1s 2024-07-31T14:53:05.775 INFO:tasks.workunit.client.0.smithi067.stderr:111850K .......... .......... .......... .......... .......... 82% 14.9M 1s 2024-07-31T14:53:05.776 INFO:tasks.workunit.client.0.smithi067.stderr:111900K .......... .......... .......... .......... .......... 82% 15.8M 1s 2024-07-31T14:53:05.785 INFO:tasks.workunit.client.0.smithi067.stderr:111950K .......... .......... .......... .......... .......... 82% 5.89M 1s 2024-07-31T14:53:05.788 INFO:tasks.workunit.client.0.smithi067.stderr:112000K .......... .......... .......... .......... .......... 82% 14.6M 1s 2024-07-31T14:53:05.790 INFO:tasks.workunit.client.0.smithi067.stderr:112050K .......... .......... .......... .......... .......... 82% 26.2M 1s 2024-07-31T14:53:05.792 INFO:tasks.workunit.client.0.smithi067.stderr:112100K .......... .......... .......... .......... .......... 82% 24.7M 1s 2024-07-31T14:53:05.793 INFO:tasks.workunit.client.0.smithi067.stderr:112150K .......... .......... .......... .......... .......... 82% 40.2M 1s 2024-07-31T14:53:05.794 INFO:tasks.workunit.client.0.smithi067.stderr:112200K .......... .......... .......... .......... .......... 82% 30.9M 1s 2024-07-31T14:53:05.797 INFO:tasks.workunit.client.0.smithi067.stderr:112250K .......... .......... .......... .......... .......... 82% 26.6M 1s 2024-07-31T14:53:05.799 INFO:tasks.workunit.client.0.smithi067.stderr:112300K .......... .......... .......... .......... .......... 82% 24.4M 1s 2024-07-31T14:53:05.800 INFO:tasks.workunit.client.0.smithi067.stderr:112350K .......... .......... .......... .......... .......... 82% 29.7M 1s 2024-07-31T14:53:05.801 INFO:tasks.workunit.client.0.smithi067.stderr:112400K .......... .......... .......... .......... .......... 82% 37.7M 1s 2024-07-31T14:53:05.804 INFO:tasks.workunit.client.0.smithi067.stderr:112450K .......... .......... .......... .......... .......... 82% 23.8M 1s 2024-07-31T14:53:05.805 INFO:tasks.workunit.client.0.smithi067.stderr:112500K .......... .......... .......... .......... .......... 82% 39.0M 1s 2024-07-31T14:53:05.806 INFO:tasks.workunit.client.0.smithi067.stderr:112550K .......... .......... .......... .......... .......... 82% 28.3M 1s 2024-07-31T14:53:05.809 INFO:tasks.workunit.client.0.smithi067.stderr:112600K .......... .......... .......... .......... .......... 82% 22.7M 1s 2024-07-31T14:53:05.810 INFO:tasks.workunit.client.0.smithi067.stderr:112650K .......... .......... .......... .......... .......... 83% 40.4M 1s 2024-07-31T14:53:05.812 INFO:tasks.workunit.client.0.smithi067.stderr:112700K .......... .......... .......... .......... .......... 83% 21.3M 1s 2024-07-31T14:53:05.813 INFO:tasks.workunit.client.0.smithi067.stderr:112750K .......... .......... .......... .......... .......... 83% 41.1M 1s 2024-07-31T14:53:05.817 INFO:tasks.workunit.client.0.smithi067.stderr:112800K .......... .......... .......... .......... .......... 83% 14.3M 1s 2024-07-31T14:53:05.819 INFO:tasks.workunit.client.0.smithi067.stderr:112850K .......... .......... .......... .......... .......... 83% 22.5M 1s 2024-07-31T14:53:05.822 INFO:tasks.workunit.client.0.smithi067.stderr:112900K .......... .......... .......... .......... .......... 83% 20.4M 1s 2024-07-31T14:53:05.824 INFO:tasks.workunit.client.0.smithi067.stderr:112950K .......... .......... .......... .......... .......... 83% 20.3M 1s 2024-07-31T14:53:05.826 INFO:tasks.workunit.client.0.smithi067.stderr:113000K .......... .......... .......... .......... .......... 83% 22.9M 1s 2024-07-31T14:53:05.828 INFO:tasks.workunit.client.0.smithi067.stderr:113050K .......... .......... .......... .......... .......... 83% 18.0M 1s 2024-07-31T14:53:05.831 INFO:tasks.workunit.client.0.smithi067.stderr:113100K .......... .......... .......... .......... .......... 83% 16.7M 1s 2024-07-31T14:53:05.836 INFO:tasks.workunit.client.0.smithi067.stderr:113150K .......... .......... .......... .......... .......... 83% 11.1M 1s 2024-07-31T14:53:05.841 INFO:tasks.workunit.client.0.smithi067.stderr:113200K .......... .......... .......... .......... .......... 83% 9.47M 1s 2024-07-31T14:53:05.844 INFO:tasks.workunit.client.0.smithi067.stderr:113250K .......... .......... .......... .......... .......... 83% 12.8M 1s 2024-07-31T14:53:05.847 INFO:tasks.workunit.client.0.smithi067.stderr:113300K .......... .......... .......... .......... .......... 83% 16.8M 1s 2024-07-31T14:53:05.850 INFO:tasks.workunit.client.0.smithi067.stderr:113350K .......... .......... .......... .......... .......... 83% 28.8M 1s 2024-07-31T14:53:05.851 INFO:tasks.workunit.client.0.smithi067.stderr:113400K .......... .......... .......... .......... .......... 83% 36.2M 1s 2024-07-31T14:53:05.853 INFO:tasks.workunit.client.0.smithi067.stderr:113450K .......... .......... .......... .......... .......... 83% 23.9M 1s 2024-07-31T14:53:05.854 INFO:tasks.workunit.client.0.smithi067.stderr:113500K .......... .......... .......... .......... .......... 83% 41.3M 1s 2024-07-31T14:53:05.856 INFO:tasks.workunit.client.0.smithi067.stderr:113550K .......... .......... .......... .......... .......... 83% 26.7M 1s 2024-07-31T14:53:05.858 INFO:tasks.workunit.client.0.smithi067.stderr:113600K .......... .......... .......... .......... .......... 83% 24.3M 1s 2024-07-31T14:53:05.859 INFO:tasks.workunit.client.0.smithi067.stderr:113650K .......... .......... .......... .......... .......... 83% 38.7M 1s 2024-07-31T14:53:05.861 INFO:tasks.workunit.client.0.smithi067.stderr:113700K .......... .......... .......... .......... .......... 83% 23.3M 1s 2024-07-31T14:53:05.863 INFO:tasks.workunit.client.0.smithi067.stderr:113750K .......... .......... .......... .......... .......... 83% 43.3M 1s 2024-07-31T14:53:05.864 INFO:tasks.workunit.client.0.smithi067.stderr:113800K .......... .......... .......... .......... .......... 83% 26.6M 1s 2024-07-31T14:53:05.867 INFO:tasks.workunit.client.0.smithi067.stderr:113850K .......... .......... .......... .......... .......... 83% 21.7M 1s 2024-07-31T14:53:05.868 INFO:tasks.workunit.client.0.smithi067.stderr:113900K .......... .......... .......... .......... .......... 83% 25.3M 1s 2024-07-31T14:53:05.871 INFO:tasks.workunit.client.0.smithi067.stderr:113950K .......... .......... .......... .......... .......... 83% 17.6M 1s 2024-07-31T14:53:05.873 INFO:tasks.workunit.client.0.smithi067.stderr:114000K .......... .......... .......... .......... .......... 84% 32.9M 1s 2024-07-31T14:53:05.875 INFO:tasks.workunit.client.0.smithi067.stderr:114050K .......... .......... .......... .......... .......... 84% 21.0M 1s 2024-07-31T14:53:05.878 INFO:tasks.workunit.client.0.smithi067.stderr:114100K .......... .......... .......... .......... .......... 84% 15.3M 1s 2024-07-31T14:53:05.881 INFO:tasks.workunit.client.0.smithi067.stderr:114150K .......... .......... .......... .......... .......... 84% 23.3M 1s 2024-07-31T14:53:05.883 INFO:tasks.workunit.client.0.smithi067.stderr:114200K .......... .......... .......... .......... .......... 84% 18.4M 1s 2024-07-31T14:53:05.886 INFO:tasks.workunit.client.0.smithi067.stderr:114250K .......... .......... .......... .......... .......... 84% 14.9M 1s 2024-07-31T14:53:05.888 INFO:tasks.workunit.client.0.smithi067.stderr:114300K .......... .......... .......... .......... .......... 84% 27.6M 1s 2024-07-31T14:53:05.891 INFO:tasks.workunit.client.0.smithi067.stderr:114350K .......... .......... .......... .......... .......... 84% 13.6M 1s 2024-07-31T14:53:05.894 INFO:tasks.workunit.client.0.smithi067.stderr:114400K .......... .......... .......... .......... .......... 84% 16.0M 1s 2024-07-31T14:53:05.899 INFO:tasks.workunit.client.0.smithi067.stderr:114450K .......... .......... .......... .......... .......... 84% 11.3M 1s 2024-07-31T14:53:05.903 INFO:tasks.workunit.client.0.smithi067.stderr:114500K .......... .......... .......... .......... .......... 84% 10.2M 1s 2024-07-31T14:53:05.905 INFO:tasks.workunit.client.0.smithi067.stderr:114550K .......... .......... .......... .......... .......... 84% 31.2M 1s 2024-07-31T14:53:05.907 INFO:tasks.workunit.client.0.smithi067.stderr:114600K .......... .......... .......... .......... .......... 84% 29.6M 1s 2024-07-31T14:53:05.909 INFO:tasks.workunit.client.0.smithi067.stderr:114650K .......... .......... .......... .......... .......... 84% 24.8M 1s 2024-07-31T14:53:05.910 INFO:tasks.workunit.client.0.smithi067.stderr:114700K .......... .......... .......... .......... .......... 84% 43.1M 1s 2024-07-31T14:53:05.912 INFO:tasks.workunit.client.0.smithi067.stderr:114750K .......... .......... .......... .......... .......... 84% 24.1M 1s 2024-07-31T14:53:05.913 INFO:tasks.workunit.client.0.smithi067.stderr:114800K .......... .......... .......... .......... .......... 84% 29.9M 1s 2024-07-31T14:53:05.915 INFO:tasks.workunit.client.0.smithi067.stderr:114850K .......... .......... .......... .......... .......... 84% 28.0M 1s 2024-07-31T14:53:05.917 INFO:tasks.workunit.client.0.smithi067.stderr:114900K .......... .......... .......... .......... .......... 84% 24.2M 1s 2024-07-31T14:53:05.919 INFO:tasks.workunit.client.0.smithi067.stderr:114950K .......... .......... .......... .......... .......... 84% 29.3M 1s 2024-07-31T14:53:05.921 INFO:tasks.workunit.client.0.smithi067.stderr:115000K .......... .......... .......... .......... .......... 84% 19.7M 1s 2024-07-31T14:53:05.923 INFO:tasks.workunit.client.0.smithi067.stderr:115050K .......... .......... .......... .......... .......... 84% 27.8M 1s 2024-07-31T14:53:05.926 INFO:tasks.workunit.client.0.smithi067.stderr:115100K .......... .......... .......... .......... .......... 84% 19.8M 1s 2024-07-31T14:53:05.928 INFO:tasks.workunit.client.0.smithi067.stderr:115150K .......... .......... .......... .......... .......... 84% 22.1M 1s 2024-07-31T14:53:05.929 INFO:tasks.workunit.client.0.smithi067.stderr:115200K .......... .......... .......... .......... .......... 84% 34.7M 1s 2024-07-31T14:53:05.932 INFO:tasks.workunit.client.0.smithi067.stderr:115250K .......... .......... .......... .......... .......... 84% 15.5M 1s 2024-07-31T14:53:05.934 INFO:tasks.workunit.client.0.smithi067.stderr:115300K .......... .......... .......... .......... .......... 84% 24.9M 1s 2024-07-31T14:53:05.937 INFO:tasks.workunit.client.0.smithi067.stderr:115350K .......... .......... .......... .......... .......... 85% 21.7M 1s 2024-07-31T14:53:05.939 INFO:tasks.workunit.client.0.smithi067.stderr:115400K .......... .......... .......... .......... .......... 85% 17.3M 1s 2024-07-31T14:53:05.942 INFO:tasks.workunit.client.0.smithi067.stderr:115450K .......... .......... .......... .......... .......... 85% 17.8M 1s 2024-07-31T14:53:05.946 INFO:tasks.workunit.client.0.smithi067.stderr:115500K .......... .......... .......... .......... .......... 85% 13.6M 1s 2024-07-31T14:53:05.947 INFO:tasks.workunit.client.0.smithi067.stderr:115550K .......... .......... .......... .......... .......... 85% 28.0M 1s 2024-07-31T14:53:05.950 INFO:tasks.workunit.client.0.smithi067.stderr:115600K .......... .......... .......... .......... .......... 85% 17.8M 1s 2024-07-31T14:53:05.955 INFO:tasks.workunit.client.0.smithi067.stderr:115650K .......... .......... .......... .......... .......... 85% 10.3M 1s 2024-07-31T14:53:05.958 INFO:tasks.workunit.client.0.smithi067.stderr:115700K .......... .......... .......... .......... .......... 85% 16.9M 1s 2024-07-31T14:53:05.961 INFO:tasks.workunit.client.0.smithi067.stderr:115750K .......... .......... .......... .......... .......... 85% 13.9M 1s 2024-07-31T14:53:05.963 INFO:tasks.workunit.client.0.smithi067.stderr:115800K .......... .......... .......... .......... .......... 85% 34.5M 1s 2024-07-31T14:53:05.964 INFO:tasks.workunit.client.0.smithi067.stderr:115850K .......... .......... .......... .......... .......... 85% 28.5M 1s 2024-07-31T14:53:05.967 INFO:tasks.workunit.client.0.smithi067.stderr:115900K .......... .......... .......... .......... .......... 85% 24.3M 1s 2024-07-31T14:53:05.968 INFO:tasks.workunit.client.0.smithi067.stderr:115950K .......... .......... .......... .......... .......... 85% 37.8M 1s 2024-07-31T14:53:05.971 INFO:tasks.workunit.client.0.smithi067.stderr:116000K .......... .......... .......... .......... .......... 85% 22.6M 1s 2024-07-31T14:53:05.971 INFO:tasks.workunit.client.0.smithi067.stderr:116050K .......... .......... .......... .......... .......... 85% 40.9M 1s 2024-07-31T14:53:05.973 INFO:tasks.workunit.client.0.smithi067.stderr:116100K .......... .......... .......... .......... .......... 85% 23.2M 1s 2024-07-31T14:53:05.977 INFO:tasks.workunit.client.0.smithi067.stderr:116150K .......... .......... .......... .......... .......... 85% 16.4M 1s 2024-07-31T14:53:05.977 INFO:tasks.workunit.client.0.smithi067.stderr:116200K .......... .......... .......... .......... .......... 85% 40.3M 1s 2024-07-31T14:53:05.980 INFO:tasks.workunit.client.0.smithi067.stderr:116250K .......... .......... .......... .......... .......... 85% 17.8M 1s 2024-07-31T14:53:05.981 INFO:tasks.workunit.client.0.smithi067.stderr:116300K .......... .......... .......... .......... .......... 85% 32.2M 1s 2024-07-31T14:53:05.987 INFO:tasks.workunit.client.0.smithi067.stderr:116350K .......... .......... .......... .......... .......... 85% 27.7M 1s 2024-07-31T14:53:05.987 INFO:tasks.workunit.client.0.smithi067.stderr:116400K .......... .......... .......... .......... .......... 85% 15.5M 1s 2024-07-31T14:53:05.989 INFO:tasks.workunit.client.0.smithi067.stderr:116450K .......... .......... .......... .......... .......... 85% 22.6M 1s 2024-07-31T14:53:05.991 INFO:tasks.workunit.client.0.smithi067.stderr:116500K .......... .......... .......... .......... .......... 85% 19.1M 1s 2024-07-31T14:53:05.993 INFO:tasks.workunit.client.0.smithi067.stderr:116550K .......... .......... .......... .......... .......... 85% 32.1M 1s 2024-07-31T14:53:05.996 INFO:tasks.workunit.client.0.smithi067.stderr:116600K .......... .......... .......... .......... .......... 85% 15.6M 1s 2024-07-31T14:53:05.998 INFO:tasks.workunit.client.0.smithi067.stderr:116650K .......... .......... .......... .......... .......... 85% 20.1M 1s 2024-07-31T14:53:06.001 INFO:tasks.workunit.client.0.smithi067.stderr:116700K .......... .......... .......... .......... .......... 86% 16.7M 1s 2024-07-31T14:53:06.004 INFO:tasks.workunit.client.0.smithi067.stderr:116750K .......... .......... .......... .......... .......... 86% 17.5M 1s 2024-07-31T14:53:06.006 INFO:tasks.workunit.client.0.smithi067.stderr:116800K .......... .......... .......... .......... .......... 86% 22.1M 1s 2024-07-31T14:53:06.010 INFO:tasks.workunit.client.0.smithi067.stderr:116850K .......... .......... .......... .......... .......... 86% 14.0M 1s 2024-07-31T14:53:06.013 INFO:tasks.workunit.client.0.smithi067.stderr:116900K .......... .......... .......... .......... .......... 86% 14.1M 1s 2024-07-31T14:53:06.016 INFO:tasks.workunit.client.0.smithi067.stderr:116950K .......... .......... .......... .......... .......... 86% 19.8M 1s 2024-07-31T14:53:06.020 INFO:tasks.workunit.client.0.smithi067.stderr:117000K .......... .......... .......... .......... .......... 86% 12.7M 1s 2024-07-31T14:53:06.021 INFO:tasks.workunit.client.0.smithi067.stderr:117050K .......... .......... .......... .......... .......... 86% 39.2M 1s 2024-07-31T14:53:06.023 INFO:tasks.workunit.client.0.smithi067.stderr:117100K .......... .......... .......... .......... .......... 86% 22.5M 1s 2024-07-31T14:53:06.025 INFO:tasks.workunit.client.0.smithi067.stderr:117150K .......... .......... .......... .......... .......... 86% 30.0M 1s 2024-07-31T14:53:06.026 INFO:tasks.workunit.client.0.smithi067.stderr:117200K .......... .......... .......... .......... .......... 86% 29.3M 1s 2024-07-31T14:53:06.028 INFO:tasks.workunit.client.0.smithi067.stderr:117250K .......... .......... .......... .......... .......... 86% 24.1M 1s 2024-07-31T14:53:06.030 INFO:tasks.workunit.client.0.smithi067.stderr:117300K .......... .......... .......... .......... .......... 86% 22.3M 1s 2024-07-31T14:53:06.033 INFO:tasks.workunit.client.0.smithi067.stderr:117350K .......... .......... .......... .......... .......... 86% 27.2M 1s 2024-07-31T14:53:06.035 INFO:tasks.workunit.client.0.smithi067.stderr:117400K .......... .......... .......... .......... .......... 86% 17.4M 1s 2024-07-31T14:53:06.037 INFO:tasks.workunit.client.0.smithi067.stderr:117450K .......... .......... .......... .......... .......... 86% 37.5M 1s 2024-07-31T14:53:06.039 INFO:tasks.workunit.client.0.smithi067.stderr:117500K .......... .......... .......... .......... .......... 86% 22.9M 1s 2024-07-31T14:53:06.041 INFO:tasks.workunit.client.0.smithi067.stderr:117550K .......... .......... .......... .......... .......... 86% 19.3M 1s 2024-07-31T14:53:06.043 INFO:tasks.workunit.client.0.smithi067.stderr:117600K .......... .......... .......... .......... .......... 86% 29.4M 1s 2024-07-31T14:53:06.046 INFO:tasks.workunit.client.0.smithi067.stderr:117650K .......... .......... .......... .......... .......... 86% 16.1M 1s 2024-07-31T14:53:06.048 INFO:tasks.workunit.client.0.smithi067.stderr:117700K .......... .......... .......... .......... .......... 86% 30.2M 1s 2024-07-31T14:53:06.051 INFO:tasks.workunit.client.0.smithi067.stderr:117750K .......... .......... .......... .......... .......... 86% 15.8M 1s 2024-07-31T14:53:06.053 INFO:tasks.workunit.client.0.smithi067.stderr:117800K .......... .......... .......... .......... .......... 86% 25.2M 1s 2024-07-31T14:53:06.055 INFO:tasks.workunit.client.0.smithi067.stderr:117850K .......... .......... .......... .......... .......... 86% 17.9M 1s 2024-07-31T14:53:06.058 INFO:tasks.workunit.client.0.smithi067.stderr:117900K .......... .......... .......... .......... .......... 86% 16.3M 1s 2024-07-31T14:53:06.060 INFO:tasks.workunit.client.0.smithi067.stderr:117950K .......... .......... .......... .......... .......... 86% 22.8M 1s 2024-07-31T14:53:06.063 INFO:tasks.workunit.client.0.smithi067.stderr:118000K .......... .......... .......... .......... .......... 86% 17.3M 1s 2024-07-31T14:53:06.066 INFO:tasks.workunit.client.0.smithi067.stderr:118050K .......... .......... .......... .......... .......... 86% 15.1M 1s 2024-07-31T14:53:06.068 INFO:tasks.workunit.client.0.smithi067.stderr:118100K .......... .......... .......... .......... .......... 87% 28.5M 1s 2024-07-31T14:53:06.070 INFO:tasks.workunit.client.0.smithi067.stderr:118150K .......... .......... .......... .......... .......... 87% 23.0M 1s 2024-07-31T14:53:06.076 INFO:tasks.workunit.client.0.smithi067.stderr:118200K .......... .......... .......... .......... .......... 87% 8.86M 1s 2024-07-31T14:53:06.078 INFO:tasks.workunit.client.0.smithi067.stderr:118250K .......... .......... .......... .......... .......... 87% 21.6M 1s 2024-07-31T14:53:06.079 INFO:tasks.workunit.client.0.smithi067.stderr:118300K .......... .......... .......... .......... .......... 87% 39.0M 1s 2024-07-31T14:53:06.081 INFO:tasks.workunit.client.0.smithi067.stderr:118350K .......... .......... .......... .......... .......... 87% 26.3M 1s 2024-07-31T14:53:06.083 INFO:tasks.workunit.client.0.smithi067.stderr:118400K .......... .......... .......... .......... .......... 87% 22.9M 1s 2024-07-31T14:53:06.085 INFO:tasks.workunit.client.0.smithi067.stderr:118450K .......... .......... .......... .......... .......... 87% 25.0M 1s 2024-07-31T14:53:06.088 INFO:tasks.workunit.client.0.smithi067.stderr:118500K .......... .......... .......... .......... .......... 87% 21.3M 1s 2024-07-31T14:53:06.089 INFO:tasks.workunit.client.0.smithi067.stderr:118550K .......... .......... .......... .......... .......... 87% 32.3M 1s 2024-07-31T14:53:06.092 INFO:tasks.workunit.client.0.smithi067.stderr:118600K .......... .......... .......... .......... .......... 87% 22.1M 1s 2024-07-31T14:53:06.094 INFO:tasks.workunit.client.0.smithi067.stderr:118650K .......... .......... .......... .......... .......... 87% 20.5M 1s 2024-07-31T14:53:06.097 INFO:tasks.workunit.client.0.smithi067.stderr:118700K .......... .......... .......... .......... .......... 87% 33.7M 1s 2024-07-31T14:53:06.098 INFO:tasks.workunit.client.0.smithi067.stderr:118750K .......... .......... .......... .......... .......... 87% 16.3M 1s 2024-07-31T14:53:06.101 INFO:tasks.workunit.client.0.smithi067.stderr:118800K .......... .......... .......... .......... .......... 87% 18.6M 1s 2024-07-31T14:53:06.103 INFO:tasks.workunit.client.0.smithi067.stderr:118850K .......... .......... .......... .......... .......... 87% 29.9M 1s 2024-07-31T14:53:06.106 INFO:tasks.workunit.client.0.smithi067.stderr:118900K .......... .......... .......... .......... .......... 87% 14.9M 1s 2024-07-31T14:53:06.109 INFO:tasks.workunit.client.0.smithi067.stderr:118950K .......... .......... .......... .......... .......... 87% 27.3M 1s 2024-07-31T14:53:06.110 INFO:tasks.workunit.client.0.smithi067.stderr:119000K .......... .......... .......... .......... .......... 87% 17.4M 1s 2024-07-31T14:53:06.113 INFO:tasks.workunit.client.0.smithi067.stderr:119050K .......... .......... .......... .......... .......... 87% 29.5M 1s 2024-07-31T14:53:06.115 INFO:tasks.workunit.client.0.smithi067.stderr:119100K .......... .......... .......... .......... .......... 87% 13.9M 1s 2024-07-31T14:53:06.118 INFO:tasks.workunit.client.0.smithi067.stderr:119150K .......... .......... .......... .......... .......... 87% 20.3M 1s 2024-07-31T14:53:06.121 INFO:tasks.workunit.client.0.smithi067.stderr:119200K .......... .......... .......... .......... .......... 87% 17.0M 1s 2024-07-31T14:53:06.124 INFO:tasks.workunit.client.0.smithi067.stderr:119250K .......... .......... .......... .......... .......... 87% 15.9M 1s 2024-07-31T14:53:06.126 INFO:tasks.workunit.client.0.smithi067.stderr:119300K .......... .......... .......... .......... .......... 87% 31.1M 1s 2024-07-31T14:53:06.129 INFO:tasks.workunit.client.0.smithi067.stderr:119350K .......... .......... .......... .......... .......... 87% 14.3M 1s 2024-07-31T14:53:06.133 INFO:tasks.workunit.client.0.smithi067.stderr:119400K .......... .......... .......... .......... .......... 87% 12.8M 1s 2024-07-31T14:53:06.136 INFO:tasks.workunit.client.0.smithi067.stderr:119450K .......... .......... .......... .......... .......... 88% 15.3M 1s 2024-07-31T14:53:06.137 INFO:tasks.workunit.client.0.smithi067.stderr:119500K .......... .......... .......... .......... .......... 88% 31.3M 1s 2024-07-31T14:53:06.140 INFO:tasks.workunit.client.0.smithi067.stderr:119550K .......... .......... .......... .......... .......... 88% 18.8M 1s 2024-07-31T14:53:06.142 INFO:tasks.workunit.client.0.smithi067.stderr:119600K .......... .......... .......... .......... .......... 88% 29.6M 1s 2024-07-31T14:53:06.143 INFO:tasks.workunit.client.0.smithi067.stderr:119650K .......... .......... .......... .......... .......... 88% 28.7M 1s 2024-07-31T14:53:06.146 INFO:tasks.workunit.client.0.smithi067.stderr:119700K .......... .......... .......... .......... .......... 88% 21.3M 1s 2024-07-31T14:53:06.147 INFO:tasks.workunit.client.0.smithi067.stderr:119750K .......... .......... .......... .......... .......... 88% 27.8M 1s 2024-07-31T14:53:06.153 INFO:tasks.workunit.client.0.smithi067.stderr:119800K .......... .......... .......... .......... .......... 88% 12.0M 1s 2024-07-31T14:53:06.153 INFO:tasks.workunit.client.0.smithi067.stderr:119850K .......... .......... .......... .......... .......... 88% 106M 1s 2024-07-31T14:53:06.154 INFO:tasks.workunit.client.0.smithi067.stderr:119900K .......... .......... .......... .......... .......... 88% 21.1M 1s 2024-07-31T14:53:06.156 INFO:tasks.workunit.client.0.smithi067.stderr:119950K .......... .......... .......... .......... .......... 88% 22.9M 1s 2024-07-31T14:53:06.158 INFO:tasks.workunit.client.0.smithi067.stderr:120000K .......... .......... .......... .......... .......... 88% 27.2M 1s 2024-07-31T14:53:06.160 INFO:tasks.workunit.client.0.smithi067.stderr:120050K .......... .......... .......... .......... .......... 88% 26.9M 1s 2024-07-31T14:53:06.163 INFO:tasks.workunit.client.0.smithi067.stderr:120100K .......... .......... .......... .......... .......... 88% 17.7M 1s 2024-07-31T14:53:06.166 INFO:tasks.workunit.client.0.smithi067.stderr:120150K .......... .......... .......... .......... .......... 88% 16.8M 1s 2024-07-31T14:53:06.168 INFO:tasks.workunit.client.0.smithi067.stderr:120200K .......... .......... .......... .......... .......... 88% 20.6M 1s 2024-07-31T14:53:06.169 INFO:tasks.workunit.client.0.smithi067.stderr:120250K .......... .......... .......... .......... .......... 88% 33.0M 1s 2024-07-31T14:53:06.173 INFO:tasks.workunit.client.0.smithi067.stderr:120300K .......... .......... .......... .......... .......... 88% 18.7M 1s 2024-07-31T14:53:06.175 INFO:tasks.workunit.client.0.smithi067.stderr:120350K .......... .......... .......... .......... .......... 88% 17.9M 1s 2024-07-31T14:53:06.178 INFO:tasks.workunit.client.0.smithi067.stderr:120400K .......... .......... .......... .......... .......... 88% 16.4M 1s 2024-07-31T14:53:06.181 INFO:tasks.workunit.client.0.smithi067.stderr:120450K .......... .......... .......... .......... .......... 88% 14.7M 1s 2024-07-31T14:53:06.183 INFO:tasks.workunit.client.0.smithi067.stderr:120500K .......... .......... .......... .......... .......... 88% 27.9M 1s 2024-07-31T14:53:06.185 INFO:tasks.workunit.client.0.smithi067.stderr:120550K .......... .......... .......... .......... .......... 88% 19.2M 1s 2024-07-31T14:53:06.190 INFO:tasks.workunit.client.0.smithi067.stderr:120600K .......... .......... .......... .......... .......... 88% 12.4M 1s 2024-07-31T14:53:06.192 INFO:tasks.workunit.client.0.smithi067.stderr:120650K .......... .......... .......... .......... .......... 88% 19.2M 1s 2024-07-31T14:53:06.194 INFO:tasks.workunit.client.0.smithi067.stderr:120700K .......... .......... .......... .......... .......... 88% 20.9M 1s 2024-07-31T14:53:06.197 INFO:tasks.workunit.client.0.smithi067.stderr:120750K .......... .......... .......... .......... .......... 88% 16.0M 1s 2024-07-31T14:53:06.199 INFO:tasks.workunit.client.0.smithi067.stderr:120800K .......... .......... .......... .......... .......... 89% 25.6M 1s 2024-07-31T14:53:06.201 INFO:tasks.workunit.client.0.smithi067.stderr:120850K .......... .......... .......... .......... .......... 89% 26.3M 1s 2024-07-31T14:53:06.203 INFO:tasks.workunit.client.0.smithi067.stderr:120900K .......... .......... .......... .......... .......... 89% 23.0M 1s 2024-07-31T14:53:06.205 INFO:tasks.workunit.client.0.smithi067.stderr:120950K .......... .......... .......... .......... .......... 89% 24.7M 1s 2024-07-31T14:53:06.207 INFO:tasks.workunit.client.0.smithi067.stderr:121000K .......... .......... .......... .......... .......... 89% 28.7M 1s 2024-07-31T14:53:06.210 INFO:tasks.workunit.client.0.smithi067.stderr:121050K .......... .......... .......... .......... .......... 89% 17.1M 1s 2024-07-31T14:53:06.211 INFO:tasks.workunit.client.0.smithi067.stderr:121100K .......... .......... .......... .......... .......... 89% 27.8M 1s 2024-07-31T14:53:06.214 INFO:tasks.workunit.client.0.smithi067.stderr:121150K .......... .......... .......... .......... .......... 89% 16.4M 1s 2024-07-31T14:53:06.216 INFO:tasks.workunit.client.0.smithi067.stderr:121200K .......... .......... .......... .......... .......... 89% 24.7M 1s 2024-07-31T14:53:06.218 INFO:tasks.workunit.client.0.smithi067.stderr:121250K .......... .......... .......... .......... .......... 89% 28.8M 1s 2024-07-31T14:53:06.220 INFO:tasks.workunit.client.0.smithi067.stderr:121300K .......... .......... .......... .......... .......... 89% 24.8M 1s 2024-07-31T14:53:06.223 INFO:tasks.workunit.client.0.smithi067.stderr:121350K .......... .......... .......... .......... .......... 89% 18.7M 1s 2024-07-31T14:53:06.224 INFO:tasks.workunit.client.0.smithi067.stderr:121400K .......... .......... .......... .......... .......... 89% 26.1M 1s 2024-07-31T14:53:06.227 INFO:tasks.workunit.client.0.smithi067.stderr:121450K .......... .......... .......... .......... .......... 89% 16.8M 1s 2024-07-31T14:53:06.230 INFO:tasks.workunit.client.0.smithi067.stderr:121500K .......... .......... .......... .......... .......... 89% 18.2M 1s 2024-07-31T14:53:06.233 INFO:tasks.workunit.client.0.smithi067.stderr:121550K .......... .......... .......... .......... .......... 89% 18.2M 1s 2024-07-31T14:53:06.235 INFO:tasks.workunit.client.0.smithi067.stderr:121600K .......... .......... .......... .......... .......... 89% 24.9M 1s 2024-07-31T14:53:06.238 INFO:tasks.workunit.client.0.smithi067.stderr:121650K .......... .......... .......... .......... .......... 89% 14.4M 1s 2024-07-31T14:53:06.241 INFO:tasks.workunit.client.0.smithi067.stderr:121700K .......... .......... .......... .......... .......... 89% 17.2M 1s 2024-07-31T14:53:06.242 INFO:tasks.workunit.client.0.smithi067.stderr:121750K .......... .......... .......... .......... .......... 89% 35.7M 1s 2024-07-31T14:53:06.246 INFO:tasks.workunit.client.0.smithi067.stderr:121800K .......... .......... .......... .......... .......... 89% 12.0M 1s 2024-07-31T14:53:06.249 INFO:tasks.workunit.client.0.smithi067.stderr:121850K .......... .......... .......... .......... .......... 89% 20.9M 1s 2024-07-31T14:53:06.251 INFO:tasks.workunit.client.0.smithi067.stderr:121900K .......... .......... .......... .......... .......... 89% 20.5M 1s 2024-07-31T14:53:06.255 INFO:tasks.workunit.client.0.smithi067.stderr:121950K .......... .......... .......... .......... .......... 89% 10.8M 1s 2024-07-31T14:53:06.257 INFO:tasks.workunit.client.0.smithi067.stderr:122000K .......... .......... .......... .......... .......... 89% 37.5M 1s 2024-07-31T14:53:06.260 INFO:tasks.workunit.client.0.smithi067.stderr:122050K .......... .......... .......... .......... .......... 89% 20.2M 1s 2024-07-31T14:53:06.262 INFO:tasks.workunit.client.0.smithi067.stderr:122100K .......... .......... .......... .......... .......... 89% 23.4M 0s 2024-07-31T14:53:06.264 INFO:tasks.workunit.client.0.smithi067.stderr:122150K .......... .......... .......... .......... .......... 90% 24.1M 0s 2024-07-31T14:53:06.265 INFO:tasks.workunit.client.0.smithi067.stderr:122200K .......... .......... .......... .......... .......... 90% 30.1M 0s 2024-07-31T14:53:06.267 INFO:tasks.workunit.client.0.smithi067.stderr:122250K .......... .......... .......... .......... .......... 90% 24.9M 0s 2024-07-31T14:53:06.271 INFO:tasks.workunit.client.0.smithi067.stderr:122300K .......... .......... .......... .......... .......... 90% 13.5M 0s 2024-07-31T14:53:06.272 INFO:tasks.workunit.client.0.smithi067.stderr:122350K .......... .......... .......... .......... .......... 90% 40.8M 0s 2024-07-31T14:53:06.275 INFO:tasks.workunit.client.0.smithi067.stderr:122400K .......... .......... .......... .......... .......... 90% 25.7M 0s 2024-07-31T14:53:06.277 INFO:tasks.workunit.client.0.smithi067.stderr:122450K .......... .......... .......... .......... .......... 90% 21.3M 0s 2024-07-31T14:53:06.278 INFO:tasks.workunit.client.0.smithi067.stderr:122500K .......... .......... .......... .......... .......... 90% 19.8M 0s 2024-07-31T14:53:06.281 INFO:tasks.workunit.client.0.smithi067.stderr:122550K .......... .......... .......... .......... .......... 90% 19.5M 0s 2024-07-31T14:53:06.283 INFO:tasks.workunit.client.0.smithi067.stderr:122600K .......... .......... .......... .......... .......... 90% 31.3M 0s 2024-07-31T14:53:06.285 INFO:tasks.workunit.client.0.smithi067.stderr:122650K .......... .......... .......... .......... .......... 90% 20.6M 0s 2024-07-31T14:53:06.288 INFO:tasks.workunit.client.0.smithi067.stderr:122700K .......... .......... .......... .......... .......... 90% 16.0M 0s 2024-07-31T14:53:06.290 INFO:tasks.workunit.client.0.smithi067.stderr:122750K .......... .......... .......... .......... .......... 90% 28.1M 0s 2024-07-31T14:53:06.293 INFO:tasks.workunit.client.0.smithi067.stderr:122800K .......... .......... .......... .......... .......... 90% 15.2M 0s 2024-07-31T14:53:06.295 INFO:tasks.workunit.client.0.smithi067.stderr:122850K .......... .......... .......... .......... .......... 90% 25.3M 0s 2024-07-31T14:53:06.297 INFO:tasks.workunit.client.0.smithi067.stderr:122900K .......... .......... .......... .......... .......... 90% 20.4M 0s 2024-07-31T14:53:06.300 INFO:tasks.workunit.client.0.smithi067.stderr:122950K .......... .......... .......... .......... .......... 90% 16.3M 0s 2024-07-31T14:53:06.303 INFO:tasks.workunit.client.0.smithi067.stderr:123000K .......... .......... .......... .......... .......... 90% 18.2M 0s 2024-07-31T14:53:06.307 INFO:tasks.workunit.client.0.smithi067.stderr:123050K .......... .......... .......... .......... .......... 90% 12.4M 0s 2024-07-31T14:53:06.308 INFO:tasks.workunit.client.0.smithi067.stderr:123100K .......... .......... .......... .......... .......... 90% 31.6M 0s 2024-07-31T14:53:06.311 INFO:tasks.workunit.client.0.smithi067.stderr:123150K .......... .......... .......... .......... .......... 90% 20.2M 0s 2024-07-31T14:53:06.316 INFO:tasks.workunit.client.0.smithi067.stderr:123200K .......... .......... .......... .......... .......... 90% 10.7M 0s 2024-07-31T14:53:06.318 INFO:tasks.workunit.client.0.smithi067.stderr:123250K .......... .......... .......... .......... .......... 90% 21.5M 0s 2024-07-31T14:53:06.319 INFO:tasks.workunit.client.0.smithi067.stderr:123300K .......... .......... .......... .......... .......... 90% 28.1M 0s 2024-07-31T14:53:06.321 INFO:tasks.workunit.client.0.smithi067.stderr:123350K .......... .......... .......... .......... .......... 90% 27.1M 0s 2024-07-31T14:53:06.324 INFO:tasks.workunit.client.0.smithi067.stderr:123400K .......... .......... .......... .......... .......... 90% 19.4M 0s 2024-07-31T14:53:06.326 INFO:tasks.workunit.client.0.smithi067.stderr:123450K .......... .......... .......... .......... .......... 90% 23.3M 0s 2024-07-31T14:53:06.328 INFO:tasks.workunit.client.0.smithi067.stderr:123500K .......... .......... .......... .......... .......... 91% 18.6M 0s 2024-07-31T14:53:06.331 INFO:tasks.workunit.client.0.smithi067.stderr:123550K .......... .......... .......... .......... .......... 91% 24.5M 0s 2024-07-31T14:53:06.332 INFO:tasks.workunit.client.0.smithi067.stderr:123600K .......... .......... .......... .......... .......... 91% 33.2M 0s 2024-07-31T14:53:06.335 INFO:tasks.workunit.client.0.smithi067.stderr:123650K .......... .......... .......... .......... .......... 91% 15.8M 0s 2024-07-31T14:53:06.338 INFO:tasks.workunit.client.0.smithi067.stderr:123700K .......... .......... .......... .......... .......... 91% 22.2M 0s 2024-07-31T14:53:06.339 INFO:tasks.workunit.client.0.smithi067.stderr:123750K .......... .......... .......... .......... .......... 91% 27.2M 0s 2024-07-31T14:53:06.342 INFO:tasks.workunit.client.0.smithi067.stderr:123800K .......... .......... .......... .......... .......... 91% 21.5M 0s 2024-07-31T14:53:06.343 INFO:tasks.workunit.client.0.smithi067.stderr:123850K .......... .......... .......... .......... .......... 91% 24.5M 0s 2024-07-31T14:53:06.346 INFO:tasks.workunit.client.0.smithi067.stderr:123900K .......... .......... .......... .......... .......... 91% 17.1M 0s 2024-07-31T14:53:06.348 INFO:tasks.workunit.client.0.smithi067.stderr:123950K .......... .......... .......... .......... .......... 91% 21.5M 0s 2024-07-31T14:53:06.353 INFO:tasks.workunit.client.0.smithi067.stderr:124000K .......... .......... .......... .......... .......... 91% 21.1M 0s 2024-07-31T14:53:06.354 INFO:tasks.workunit.client.0.smithi067.stderr:124050K .......... .......... .......... .......... .......... 91% 16.9M 0s 2024-07-31T14:53:06.357 INFO:tasks.workunit.client.0.smithi067.stderr:124100K .......... .......... .......... .......... .......... 91% 24.1M 0s 2024-07-31T14:53:06.359 INFO:tasks.workunit.client.0.smithi067.stderr:124150K .......... .......... .......... .......... .......... 91% 16.9M 0s 2024-07-31T14:53:06.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.359+0000 7eff53fbd640 1 -- 172.21.15.67:0/1052393424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4c107dd0 msgr2=0x7eff4c1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:06.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.359+0000 7eff53fbd640 1 --2- 172.21.15.67:0/1052393424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4c107dd0 0x7eff4c1081e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009900 tx=0x7eff3c02a410 comp rx=0 tx=0).stop 2024-07-31T14:53:06.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.359+0000 7eff53fbd640 1 -- 172.21.15.67:0/1052393424 shutdown_connections 2024-07-31T14:53:06.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.359+0000 7eff53fbd640 1 --2- 172.21.15.67:0/1052393424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff4c108720 0x7eff4c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.359+0000 7eff53fbd640 1 --2- 172.21.15.67:0/1052393424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4c107dd0 0x7eff4c1081e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.359+0000 7eff53fbd640 1 --2- 172.21.15.67:0/1052393424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c102490 0x7eff4c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.359+0000 7eff53fbd640 1 -- 172.21.15.67:0/1052393424 >> 172.21.15.67:0/1052393424 conn(0x7eff4c079160 msgr2=0x7eff4c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:06.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.359+0000 7eff53fbd640 1 -- 172.21.15.67:0/1052393424 shutdown_connections 2024-07-31T14:53:06.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.359+0000 7eff53fbd640 1 -- 172.21.15.67:0/1052393424 wait complete. 2024-07-31T14:53:06.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.360+0000 7eff53fbd640 1 Processor -- start 2024-07-31T14:53:06.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.360+0000 7eff53fbd640 1 -- start start 2024-07-31T14:53:06.363 INFO:tasks.workunit.client.0.smithi067.stderr:124200K .......... .......... .......... .......... .......... 91% 18.7M 0s 2024-07-31T14:53:06.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.360+0000 7eff53fbd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff4c102490 0x7eff4c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.360+0000 7eff53fbd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c107dd0 0x7eff4c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.360+0000 7eff53fbd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4c108720 0x7eff4c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.360+0000 7eff53fbd640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c112350 con 0x7eff4c108720 2024-07-31T14:53:06.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.360+0000 7eff53fbd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c1121d0 con 0x7eff4c107dd0 2024-07-31T14:53:06.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.360+0000 7eff53fbd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c1124d0 con 0x7eff4c102490 2024-07-31T14:53:06.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.360+0000 7eff5255a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4c108720 0x7eff4c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.360+0000 7eff5255a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4c108720 0x7eff4c19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59272/0 (socket says 172.21.15.67:59272) 2024-07-31T14:53:06.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.360+0000 7eff5255a640 1 -- 172.21.15.67:0/4188963749 learned_addr learned my addr 172.21.15.67:0/4188963749 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:06.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.361+0000 7eff51d59640 1 --2- 172.21.15.67:0/4188963749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff4c102490 0x7eff4c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.361+0000 7eff51558640 1 --2- 172.21.15.67:0/4188963749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c107dd0 0x7eff4c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.361+0000 7eff5255a640 1 -- 172.21.15.67:0/4188963749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff4c102490 msgr2=0x7eff4c198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:06.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.361+0000 7eff5255a640 1 --2- 172.21.15.67:0/4188963749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff4c102490 0x7eff4c198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.361+0000 7eff5255a640 1 -- 172.21.15.67:0/4188963749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c107dd0 msgr2=0x7eff4c199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:06.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.361+0000 7eff5255a640 1 --2- 172.21.15.67:0/4188963749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c107dd0 0x7eff4c199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.361+0000 7eff5255a640 1 -- 172.21.15.67:0/4188963749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c19ec70 con 0x7eff4c108720 2024-07-31T14:53:06.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.361+0000 7eff5255a640 1 --2- 172.21.15.67:0/4188963749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4c108720 0x7eff4c19e600 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7eff4400eaa0 tx=0x7eff4400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:06.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.361+0000 7eff42ffd640 1 -- 172.21.15.67:0/4188963749 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4400cdb0 con 0x7eff4c108720 2024-07-31T14:53:06.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.361+0000 7eff42ffd640 1 -- 172.21.15.67:0/4188963749 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff44005020 con 0x7eff4c108720 2024-07-31T14:53:06.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.361+0000 7eff42ffd640 1 -- 172.21.15.67:0/4188963749 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff440116d0 con 0x7eff4c108720 2024-07-31T14:53:06.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.361+0000 7eff53fbd640 1 -- 172.21.15.67:0/4188963749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4c19ef60 con 0x7eff4c108720 2024-07-31T14:53:06.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.361+0000 7eff53fbd640 1 -- 172.21.15.67:0/4188963749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff4c19f420 con 0x7eff4c108720 2024-07-31T14:53:06.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.364+0000 7eff42ffd640 1 -- 172.21.15.67:0/4188963749 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7eff440051d0 con 0x7eff4c108720 2024-07-31T14:53:06.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.364+0000 7eff5a50e740 1 Processor -- start 2024-07-31T14:53:06.372 INFO:tasks.workunit.client.0.smithi067.stderr:124250K .......... .......... .......... .......... .......... 91% 22.9M 0s 2024-07-31T14:53:06.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.364+0000 7eff5a50e740 1 -- start start 2024-07-31T14:53:06.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.364+0000 7eff5a50e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef4670520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.364+0000 7eff5a50e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef4670a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.364+0000 7eff5a50e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 0x55bef4670fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.364+0000 7eff5a50e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bef45e6b60 con 0x55bef42cb070 2024-07-31T14:53:06.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.364+0000 7eff5a50e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bef45d6060 con 0x55bef42aa390 2024-07-31T14:53:06.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.364+0000 7eff5a50e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bef45da0a0 con 0x55bef42a21b0 2024-07-31T14:53:06.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.365+0000 7eff51558640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef4670520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.365+0000 7eff51558640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef4670520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59286/0 (socket says 172.21.15.67:59286) 2024-07-31T14:53:06.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.365+0000 7eff51558640 1 -- 172.21.15.67:0/3609436070 learned_addr learned my addr 172.21.15.67:0/3609436070 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:06.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.365+0000 7eff51558640 1 --2- 172.21.15.67:0/3609436070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 0x55bef4670fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.365+0000 7eff51d59640 1 --2- 172.21.15.67:0/3609436070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef4670a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.365+0000 7eff42ffd640 1 --2- 172.21.15.67:0/4188963749 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7eff380b3df0 0x7eff380b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.365+0000 7eff5255a640 1 --2- 172.21.15.67:0/4188963749 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7eff380b3df0 0x7eff380b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.365+0000 7eff51558640 1 -- 172.21.15.67:0/3609436070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 msgr2=0x55bef4670a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:06.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.365+0000 7eff51558640 1 --2- 172.21.15.67:0/3609436070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef4670a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.365+0000 7eff51558640 1 -- 172.21.15.67:0/3609436070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 msgr2=0x55bef4670fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:06.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.365+0000 7eff51558640 1 --2- 172.21.15.67:0/3609436070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 0x55bef4670fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.365+0000 7eff51558640 1 -- 172.21.15.67:0/3609436070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bef457cb40 con 0x55bef42cb070 2024-07-31T14:53:06.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.365+0000 7eff42ffd640 1 -- 172.21.15.67:0/4188963749 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7eff440e4500 con 0x7eff4c108720 2024-07-31T14:53:06.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.365+0000 7eff51558640 1 --2- 172.21.15.67:0/3609436070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef4670520 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7eff3c005940 tx=0x7eff3c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ff1921f39d10c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:06.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.366+0000 7eff37fff640 1 -- 172.21.15.67:0/3609436070 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c035030 con 0x55bef42cb070 2024-07-31T14:53:06.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.366+0000 7eff37fff640 1 -- 172.21.15.67:0/3609436070 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff3c004230 con 0x55bef42cb070 2024-07-31T14:53:06.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.366+0000 7eff37fff640 1 -- 172.21.15.67:0/3609436070 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c03a480 con 0x55bef42cb070 2024-07-31T14:53:06.378 INFO:tasks.workunit.client.0.smithi067.stderr:124300K .......... .......... .......... .......... .......... 91% 12.4M 0s 2024-07-31T14:53:06.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.367+0000 7eff5a50e740 1 -- 172.21.15.67:0/3609436070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 msgr2=0x55bef4670520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:06.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.367+0000 7eff5255a640 1 --2- 172.21.15.67:0/4188963749 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7eff380b3df0 0x7eff380b6300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7eff440e19f0 tx=0x7eff44015840 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:06.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.367+0000 7eff5a50e740 1 --2- 172.21.15.67:0/3609436070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef4670520 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7eff3c005940 tx=0x7eff3c005a20 comp rx=0 tx=0).stop 2024-07-31T14:53:06.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.367+0000 7eff5a50e740 1 -- 172.21.15.67:0/3609436070 shutdown_connections 2024-07-31T14:53:06.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.367+0000 7eff5a50e740 1 --2- 172.21.15.67:0/3609436070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 0x55bef4670fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.367+0000 7eff5a50e740 1 --2- 172.21.15.67:0/3609436070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef4670a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.367+0000 7eff5a50e740 1 --2- 172.21.15.67:0/3609436070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef4670520 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.367+0000 7eff5a50e740 1 -- 172.21.15.67:0/3609436070 >> 172.21.15.67:0/3609436070 conn(0x55bef42bf460 msgr2=0x55bef466e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:06.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.367+0000 7eff5a50e740 1 -- 172.21.15.67:0/3609436070 shutdown_connections 2024-07-31T14:53:06.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.367+0000 7eff5a50e740 1 -- 172.21.15.67:0/3609436070 wait complete. 2024-07-31T14:53:06.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.367+0000 7eff5a50e740 1 -- start start 2024-07-31T14:53:06.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.367+0000 7eff5a50e740 1 Processor -- start 2024-07-31T14:53:06.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.368+0000 7eff5a50e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef4680500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.368+0000 7eff5a50e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42aa390 0x55bef4680a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.368+0000 7eff5a50e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42cb070 0x55bef468c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.368+0000 7eff5a50e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bef4439700 con 0x55bef42aa390 2024-07-31T14:53:06.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.368+0000 7eff5a50e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bef4578990 con 0x55bef42cb070 2024-07-31T14:53:06.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.368+0000 7eff5a50e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bef4682fd0 con 0x55bef42a21b0 2024-07-31T14:53:06.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.368+0000 7eff5a50e740 10 client.0 _ll_register_callbacks cb 0x55bef45934a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:06.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.368+0000 7eff51d59640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42aa390 0x55bef4680a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.368+0000 7eff51d59640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42aa390 0x55bef4680a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59294/0 (socket says 172.21.15.67:59294) 2024-07-31T14:53:06.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.368+0000 7eff51d59640 1 -- 172.21.15.67:0/1223812126 learned_addr learned my addr 172.21.15.67:0/1223812126 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:06.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.368+0000 7eff51558640 1 --2- 172.21.15.67:0/1223812126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef4680500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.368+0000 7eff51558640 1 --2- 172.21.15.67:0/1223812126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42cb070 0x55bef468c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.369+0000 7eff51d59640 1 -- 172.21.15.67:0/1223812126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 msgr2=0x55bef4680500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.369+0000 7eff51d59640 1 --2- 172.21.15.67:0/1223812126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef4680500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.369+0000 7eff51d59640 1 -- 172.21.15.67:0/1223812126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42cb070 msgr2=0x55bef468c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.369+0000 7eff51d59640 1 --2- 172.21.15.67:0/1223812126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42cb070 0x55bef468c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.369+0000 7eff51d59640 1 -- 172.21.15.67:0/1223812126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bef4336650 con 0x55bef42aa390 2024-07-31T14:53:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.369+0000 7eff51558640 1 --2- 172.21.15.67:0/1223812126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42cb070 0x55bef468c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:53:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.369+0000 7eff51d59640 1 --2- 172.21.15.67:0/1223812126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42aa390 0x55bef4680a40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7eff4800b870 tx=0x7eff4800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2016572f3512e4cc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.369+0000 7eff357fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.369+0000 7eff357fa640 1 -- 172.21.15.67:0/1223812126 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48004230 con 0x55bef42aa390 2024-07-31T14:53:06.384 INFO:tasks.workunit.client.0.smithi067.stderr:124350K .......... .......... .......... .......... .......... 91% 20.7M 0s 2024-07-31T14:53:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.369+0000 7eff5a50e740 1 -- 172.21.15.67:0/1223812126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff3c004230 con 0x55bef42aa390 2024-07-31T14:53:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.369+0000 7eff357fa640 1 -- 172.21.15.67:0/1223812126 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7eff480043d0 con 0x55bef42aa390 2024-07-31T14:53:06.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.369+0000 7eff357fa640 1 -- 172.21.15.67:0/1223812126 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff48024050 con 0x55bef42aa390 2024-07-31T14:53:06.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.369+0000 7eff5a50e740 10 client.16203 fetch_fsmap learned FSMap version 30 2024-07-31T14:53:06.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.369+0000 7eff5a50e740 1 -- 172.21.15.67:0/1223812126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bef468f090 con 0x55bef42aa390 2024-07-31T14:53:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.370+0000 7eff357fa640 1 -- 172.21.15.67:0/1223812126 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff3c004230 con 0x55bef42aa390 2024-07-31T14:53:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.370+0000 7eff357fa640 1 -- 172.21.15.67:0/1223812126 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7eff4801f030 con 0x55bef42aa390 2024-07-31T14:53:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.370+0000 7eff5a50e740 10 client.16203 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:53:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.370+0000 7eff5a50e740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.370+0000 7eff5a50e740 20 client.16203 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.370+0000 7eff5a50e740 1 --2- 172.21.15.67:0/1223812126 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bef42899f0 0x55bef4692a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.370+0000 7eff5a50e740 4 client.16203 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:53:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.370+0000 7eff5a50e740 1 -- 172.21.15.67:0/1223812126 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bef45e8ab0 con 0x55bef42899f0 2024-07-31T14:53:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.370+0000 7eff51558640 1 --2- 172.21.15.67:0/1223812126 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bef42899f0 0x55bef4692a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.371+0000 7eff51558640 1 --2- 172.21.15.67:0/1223812126 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bef42899f0 0x55bef4692a50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27cc1b1372db7a42 server_cookie=53fd6385f12db37c in_seq=0 out_seq=0 2024-07-31T14:53:06.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.371+0000 7eff357fa640 10 client.16203 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:06.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.371+0000 7eff357fa640 1 -- 172.21.15.67:0/1223812126 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bef45e8ab0 con 0x55bef42899f0 2024-07-31T14:53:06.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.371+0000 7eff357fa640 10 client.16203 handle_command_reply: tid=0 2024-07-31T14:53:06.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.372+0000 7eff5a50e740 1 client.16203 shutdown 2024-07-31T14:53:06.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.372+0000 7eff5a50e740 1 -- 172.21.15.67:0/1223812126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42aa390 msgr2=0x55bef4680a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:06.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.372+0000 7eff5a50e740 1 --2- 172.21.15.67:0/1223812126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42aa390 0x55bef4680a40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7eff4800b870 tx=0x7eff4800bd30 comp rx=0 tx=0).stop 2024-07-31T14:53:06.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.372+0000 7eff5a50e740 1 -- 172.21.15.67:0/1223812126 shutdown_connections 2024-07-31T14:53:06.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.372+0000 7eff5a50e740 1 --2- 172.21.15.67:0/1223812126 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bef42899f0 0x55bef4692a50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.372+0000 7eff5a50e740 1 --2- 172.21.15.67:0/1223812126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42cb070 0x55bef468c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.372+0000 7eff5a50e740 1 --2- 172.21.15.67:0/1223812126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42aa390 0x55bef4680a40 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.372+0000 7eff5a50e740 1 --2- 172.21.15.67:0/1223812126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef4680500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.372+0000 7eff357fa640 0 client.16203 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:06.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.372+0000 7eff5a50e740 1 -- 172.21.15.67:0/1223812126 >> 172.21.15.67:0/1223812126 conn(0x55bef42bf460 msgr2=0x55bef466cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:06.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.372+0000 7eff5a50e740 1 -- 172.21.15.67:0/1223812126 shutdown_connections 2024-07-31T14:53:06.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.372+0000 7eff5a50e740 1 -- 172.21.15.67:0/1223812126 wait complete. 2024-07-31T14:53:06.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.372+0000 7eff5a50e740 20 client.16203 trim_cache size 0 max 16384 2024-07-31T14:53:06.396 INFO:tasks.workunit.client.0.smithi067.stderr:124400K .......... .......... .......... .......... .......... 91% 12.2M 0s 2024-07-31T14:53:06.396 INFO:tasks.workunit.client.0.smithi067.stderr:124450K .......... .......... .......... .......... .......... 91% 21.6M 0s 2024-07-31T14:53:06.396 INFO:tasks.workunit.client.0.smithi067.stderr:124500K .......... .......... .......... .......... .......... 91% 17.5M 0s 2024-07-31T14:53:06.396 INFO:tasks.workunit.client.0.smithi067.stderr:124550K .......... .......... .......... .......... .......... 91% 22.3M 0s 2024-07-31T14:53:06.396 INFO:tasks.workunit.client.0.smithi067.stderr:124600K .......... .......... .......... .......... .......... 91% 19.5M 0s 2024-07-31T14:53:06.396 INFO:tasks.workunit.client.0.smithi067.stderr:124650K .......... .......... .......... .......... .......... 91% 30.6M 0s 2024-07-31T14:53:06.396 INFO:tasks.workunit.client.0.smithi067.stderr:124700K .......... .......... .......... .......... .......... 91% 19.3M 0s 2024-07-31T14:53:06.396 INFO:tasks.workunit.client.0.smithi067.stderr:124750K .......... .......... .......... .......... .......... 91% 23.5M 0s 2024-07-31T14:53:06.396 INFO:tasks.workunit.client.0.smithi067.stderr:124800K .......... .......... .......... .......... .......... 91% 28.4M 0s 2024-07-31T14:53:06.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.391+0000 7eff5a50e740 1 Processor -- start 2024-07-31T14:53:06.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.391+0000 7eff5a50e740 1 -- start start 2024-07-31T14:53:06.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.391+0000 7eff5a50e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef466d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.391+0000 7eff5a50e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 0x55bef466dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.391+0000 7eff5a50e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef466e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.391+0000 7eff5a50e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c1ae370 con 0x55bef42cb070 2024-07-31T14:53:06.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.391+0000 7eff5a50e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c1afeb0 con 0x55bef42aa390 2024-07-31T14:53:06.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.391+0000 7eff5a50e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c1aea40 con 0x55bef42a21b0 2024-07-31T14:53:06.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.391+0000 7eff51558640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef466e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.391+0000 7eff51558640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef466e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59310/0 (socket says 172.21.15.67:59310) 2024-07-31T14:53:06.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.391+0000 7eff51558640 1 -- 172.21.15.67:0/512790477 learned_addr learned my addr 172.21.15.67:0/512790477 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:06.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.391+0000 7eff51558640 1 --2- 172.21.15.67:0/512790477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef466d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.392+0000 7eff51d59640 1 --2- 172.21.15.67:0/512790477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 0x55bef466dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.392+0000 7eff51558640 1 -- 172.21.15.67:0/512790477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 msgr2=0x55bef466d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:06.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.392+0000 7eff51558640 1 --2- 172.21.15.67:0/512790477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef466d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.392+0000 7eff51558640 1 -- 172.21.15.67:0/512790477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 msgr2=0x55bef466dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:06.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.392+0000 7eff51558640 1 --2- 172.21.15.67:0/512790477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 0x55bef466dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.392+0000 7eff51558640 1 -- 172.21.15.67:0/512790477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bef467bb60 con 0x55bef42cb070 2024-07-31T14:53:06.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.392+0000 7eff51558640 1 --2- 172.21.15.67:0/512790477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef466e270 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7eff3c02a930 tx=0x7eff3c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb66484e02a75cf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:06.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.392+0000 7eff377fe640 1 -- 172.21.15.67:0/512790477 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c035040 con 0x55bef42cb070 2024-07-31T14:53:06.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.392+0000 7eff377fe640 1 -- 172.21.15.67:0/512790477 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff3c031940 con 0x55bef42cb070 2024-07-31T14:53:06.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.392+0000 7eff377fe640 1 -- 172.21.15.67:0/512790477 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c003950 con 0x55bef42cb070 2024-07-31T14:53:06.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.393+0000 7eff5a50e740 1 -- 172.21.15.67:0/512790477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 msgr2=0x55bef466e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:06.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.393+0000 7eff5a50e740 1 --2- 172.21.15.67:0/512790477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef466e270 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7eff3c02a930 tx=0x7eff3c005a20 comp rx=0 tx=0).stop 2024-07-31T14:53:06.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.393+0000 7eff5a50e740 1 -- 172.21.15.67:0/512790477 shutdown_connections 2024-07-31T14:53:06.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.393+0000 7eff5a50e740 1 --2- 172.21.15.67:0/512790477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef466e270 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.393+0000 7eff5a50e740 1 --2- 172.21.15.67:0/512790477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 0x55bef466dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.393+0000 7eff5a50e740 1 --2- 172.21.15.67:0/512790477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef466d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.393+0000 7eff5a50e740 1 -- 172.21.15.67:0/512790477 >> 172.21.15.67:0/512790477 conn(0x55bef42bf460 msgr2=0x55bef466cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:06.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.393+0000 7eff5a50e740 1 -- 172.21.15.67:0/512790477 shutdown_connections 2024-07-31T14:53:06.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.393+0000 7eff5a50e740 1 -- 172.21.15.67:0/512790477 wait complete. 2024-07-31T14:53:06.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.393+0000 7eff5a50e740 1 -- start start 2024-07-31T14:53:06.402 INFO:tasks.workunit.client.0.smithi067.stderr:124850K .......... .......... .......... .......... .......... 92% 14.7M 0s 2024-07-31T14:53:06.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.394+0000 7eff5a50e740 1 Processor -- start 2024-07-31T14:53:06.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.394+0000 7eff5a50e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef4683630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.394+0000 7eff5a50e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 0x55bef4683bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.394+0000 7eff5a50e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef46f9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.394+0000 7eff5a50e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c1af7e0 con 0x55bef42cb070 2024-07-31T14:53:06.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.394+0000 7eff5a50e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c1af110 con 0x55bef42aa390 2024-07-31T14:53:06.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.394+0000 7eff5a50e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c1ac350 con 0x55bef42a21b0 2024-07-31T14:53:06.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.394+0000 7eff5a50e740 10 client.0 _ll_register_callbacks cb 0x7eff4800c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:06.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.394+0000 7eff51558640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef46f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.394+0000 7eff51558640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef46f9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59312/0 (socket says 172.21.15.67:59312) 2024-07-31T14:53:06.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.394+0000 7eff51558640 1 -- 172.21.15.67:0/29324467 learned_addr learned my addr 172.21.15.67:0/29324467 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:06.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.394+0000 7eff51558640 1 --2- 172.21.15.67:0/29324467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef4683630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.394+0000 7eff51d59640 1 --2- 172.21.15.67:0/29324467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 0x55bef4683bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff51558640 1 -- 172.21.15.67:0/29324467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 msgr2=0x55bef4683630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:06.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff51558640 1 --2- 172.21.15.67:0/29324467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef4683630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff51558640 1 -- 172.21.15.67:0/29324467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 msgr2=0x55bef4683bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:06.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff51558640 1 --2- 172.21.15.67:0/29324467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 0x55bef4683bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff51558640 1 -- 172.21.15.67:0/29324467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bef46f9750 con 0x55bef42cb070 2024-07-31T14:53:06.410 INFO:tasks.workunit.client.0.smithi067.stderr:124900K .......... .......... .......... .......... .......... 92% 37.9M 0s 2024-07-31T14:53:06.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff51558640 1 --2- 172.21.15.67:0/29324467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef46f9210 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7eff3c032d20 tx=0x7eff3c04d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3927bd02db35e9cb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:06.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff357fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:06.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff357fa640 1 -- 172.21.15.67:0/29324467 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c035040 con 0x55bef42cb070 2024-07-31T14:53:06.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff357fa640 1 -- 172.21.15.67:0/29324467 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7eff3c0036d0 con 0x55bef42cb070 2024-07-31T14:53:06.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff5a50e740 1 -- 172.21.15.67:0/29324467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff3c031940 con 0x55bef42cb070 2024-07-31T14:53:06.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff357fa640 1 -- 172.21.15.67:0/29324467 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff3c0038d0 con 0x55bef42cb070 2024-07-31T14:53:06.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff5a50e740 10 client.16209 fetch_fsmap learned FSMap version 30 2024-07-31T14:53:06.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff5a50e740 1 -- 172.21.15.67:0/29324467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bef46fa8e0 con 0x55bef42cb070 2024-07-31T14:53:06.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff357fa640 1 -- 172.21.15.67:0/29324467 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff3c031940 con 0x55bef42cb070 2024-07-31T14:53:06.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.395+0000 7eff357fa640 1 -- 172.21.15.67:0/29324467 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7eff3c03aac0 con 0x55bef42cb070 2024-07-31T14:53:06.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.396+0000 7eff5a50e740 10 client.16209 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:53:06.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.396+0000 7eff5a50e740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:06.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.396+0000 7eff5a50e740 20 client.16209 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:06.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.396+0000 7eff5a50e740 1 --2- 172.21.15.67:0/29324467 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bef425eda0 0x55bef46fcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:06.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.396+0000 7eff5a50e740 4 client.16209 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:53:06.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.396+0000 7eff5a50e740 1 -- 172.21.15.67:0/29324467 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bef45e4550 con 0x55bef425eda0 2024-07-31T14:53:06.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.396+0000 7eff51d59640 1 --2- 172.21.15.67:0/29324467 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bef425eda0 0x55bef46fcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:06.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.396+0000 7eff51d59640 1 --2- 172.21.15.67:0/29324467 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bef425eda0 0x55bef46fcb20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29d9987571eae0b1 server_cookie=d09cf1049cb65f46 in_seq=0 out_seq=0 2024-07-31T14:53:06.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.396+0000 7eff357fa640 10 client.16209 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:06.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff357fa640 1 -- 172.21.15.67:0/29324467 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bef45e4550 con 0x55bef425eda0 2024-07-31T14:53:06.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff357fa640 10 client.16209 handle_command_reply: tid=0 2024-07-31T14:53:06.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff5a50e740 1 client.16209 shutdown 2024-07-31T14:53:06.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff5a50e740 1 -- 172.21.15.67:0/29324467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 msgr2=0x55bef46f9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:06.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff5a50e740 1 --2- 172.21.15.67:0/29324467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef46f9210 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7eff3c032d20 tx=0x7eff3c04d040 comp rx=0 tx=0).stop 2024-07-31T14:53:06.415 INFO:tasks.workunit.client.0.smithi067.stderr:124950K .......... .......... .......... .......... .......... 92% 22.6M 0s 2024-07-31T14:53:06.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff5a50e740 1 -- 172.21.15.67:0/29324467 shutdown_connections 2024-07-31T14:53:06.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff5a50e740 1 --2- 172.21.15.67:0/29324467 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bef425eda0 0x55bef46fcb20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff5a50e740 1 --2- 172.21.15.67:0/29324467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bef42cb070 0x55bef46f9210 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff5a50e740 1 --2- 172.21.15.67:0/29324467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bef42aa390 0x55bef4683bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff357fa640 0 client.16209 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:06.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff5a50e740 1 --2- 172.21.15.67:0/29324467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bef42a21b0 0x55bef4683630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff5a50e740 1 -- 172.21.15.67:0/29324467 >> 172.21.15.67:0/29324467 conn(0x55bef42bf460 msgr2=0x55bef466dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:06.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff5a50e740 1 -- 172.21.15.67:0/29324467 shutdown_connections 2024-07-31T14:53:06.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff5a50e740 1 -- 172.21.15.67:0/29324467 wait complete. 2024-07-31T14:53:06.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.397+0000 7eff5a50e740 20 client.16209 trim_cache size 0 max 16384 2024-07-31T14:53:06.416 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:53:06.417 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T14:53:06.417 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T14:53:06.417 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:53:06.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.398+0000 7eff53fbd640 1 -- 172.21.15.67:0/4188963749 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7eff380b3df0 msgr2=0x7eff380b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:06.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.398+0000 7eff53fbd640 1 --2- 172.21.15.67:0/4188963749 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7eff380b3df0 0x7eff380b6300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7eff440e19f0 tx=0x7eff44015840 comp rx=0 tx=0).stop 2024-07-31T14:53:06.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.398+0000 7eff53fbd640 1 -- 172.21.15.67:0/4188963749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4c108720 msgr2=0x7eff4c19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:06.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.398+0000 7eff53fbd640 1 --2- 172.21.15.67:0/4188963749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4c108720 0x7eff4c19e600 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7eff4400eaa0 tx=0x7eff4400ef60 comp rx=0 tx=0).stop 2024-07-31T14:53:06.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.398+0000 7eff53fbd640 1 -- 172.21.15.67:0/4188963749 shutdown_connections 2024-07-31T14:53:06.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.398+0000 7eff53fbd640 1 --2- 172.21.15.67:0/4188963749 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7eff380b3df0 0x7eff380b6300 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.398+0000 7eff53fbd640 1 --2- 172.21.15.67:0/4188963749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4c108720 0x7eff4c19e600 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.398+0000 7eff53fbd640 1 --2- 172.21.15.67:0/4188963749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c107dd0 0x7eff4c199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.398+0000 7eff53fbd640 1 --2- 172.21.15.67:0/4188963749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff4c102490 0x7eff4c198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:06.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.398+0000 7eff53fbd640 1 -- 172.21.15.67:0/4188963749 >> 172.21.15.67:0/4188963749 conn(0x7eff4c079160 msgr2=0x7eff4c104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:06.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.398+0000 7eff53fbd640 1 -- 172.21.15.67:0/4188963749 shutdown_connections 2024-07-31T14:53:06.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:06.398+0000 7eff53fbd640 1 -- 172.21.15.67:0/4188963749 wait complete. 2024-07-31T14:53:06.426 INFO:tasks.workunit.client.0.smithi067.stderr:125000K .......... .......... .......... .......... .......... 92% 16.4M 0s 2024-07-31T14:53:06.426 INFO:tasks.workunit.client.0.smithi067.stderr:125050K .......... .......... .......... .......... .......... 92% 23.6M 0s 2024-07-31T14:53:06.426 INFO:tasks.workunit.client.0.smithi067.stderr:125100K .......... .......... .......... .......... .......... 92% 18.1M 0s 2024-07-31T14:53:06.426 INFO:tasks.workunit.client.0.smithi067.stderr:125150K .......... .......... .......... .......... .......... 92% 32.2M 0s 2024-07-31T14:53:06.426 INFO:tasks.workunit.client.0.smithi067.stderr:125200K .......... .......... .......... .......... .......... 92% 19.0M 0s 2024-07-31T14:53:06.426 INFO:tasks.workunit.client.0.smithi067.stderr:125250K .......... .......... .......... .......... .......... 92% 16.3M 0s 2024-07-31T14:53:06.426 INFO:tasks.workunit.client.0.smithi067.stderr:125300K .......... .......... .......... .......... .......... 92% 27.5M 0s 2024-07-31T14:53:06.426 INFO:tasks.workunit.client.0.smithi067.stderr:125350K .......... .......... .......... .......... .......... 92% 15.2M 0s 2024-07-31T14:53:06.426 INFO:tasks.workunit.client.0.smithi067.stderr:125400K .......... .......... .......... .......... .......... 92% 22.2M 0s 2024-07-31T14:53:06.427 INFO:tasks.workunit.client.0.smithi067.stderr:125450K .......... .......... .......... .......... .......... 92% 19.8M 0s 2024-07-31T14:53:06.427 INFO:tasks.workunit.client.0.smithi067.stderr:125500K .......... .......... .......... .......... .......... 92% 15.0M 0s 2024-07-31T14:53:06.428 INFO:tasks.workunit.client.0.smithi067.stderr:125550K .......... .......... .......... .......... .......... 92% 20.8M 0s 2024-07-31T14:53:06.432 INFO:tasks.workunit.client.0.smithi067.stderr:125600K .......... .......... .......... .......... .......... 92% 12.0M 0s 2024-07-31T14:53:06.434 INFO:tasks.workunit.client.0.smithi067.stderr:125650K .......... .......... .......... .......... .......... 92% 29.8M 0s 2024-07-31T14:53:06.436 INFO:tasks.workunit.client.0.smithi067.stderr:125700K .......... .......... .......... .......... .......... 92% 20.4M 0s 2024-07-31T14:53:06.439 INFO:tasks.workunit.client.0.smithi067.stderr:125750K .......... .......... .......... .......... .......... 92% 16.5M 0s 2024-07-31T14:53:06.442 INFO:tasks.workunit.client.0.smithi067.stderr:125800K .......... .......... .......... .......... .......... 92% 17.2M 0s 2024-07-31T14:53:06.444 INFO:tasks.workunit.client.0.smithi067.stderr:125850K .......... .......... .......... .......... .......... 92% 19.4M 0s 2024-07-31T14:53:06.447 INFO:tasks.workunit.client.0.smithi067.stderr:125900K .......... .......... .......... .......... .......... 92% 23.8M 0s 2024-07-31T14:53:06.448 INFO:tasks.workunit.client.0.smithi067.stderr:125950K .......... .......... .......... .......... .......... 92% 31.1M 0s 2024-07-31T14:53:06.451 INFO:tasks.workunit.client.0.smithi067.stderr:126000K .......... .......... .......... .......... .......... 92% 16.5M 0s 2024-07-31T14:53:06.453 INFO:tasks.workunit.client.0.smithi067.stderr:126050K .......... .......... .......... .......... .......... 92% 20.9M 0s 2024-07-31T14:53:06.456 INFO:tasks.workunit.client.0.smithi067.stderr:126100K .......... .......... .......... .......... .......... 92% 24.4M 0s 2024-07-31T14:53:06.457 INFO:tasks.workunit.client.0.smithi067.stderr:126150K .......... .......... .......... .......... .......... 92% 27.2M 0s 2024-07-31T14:53:06.461 INFO:tasks.workunit.client.0.smithi067.stderr:126200K .......... .......... .......... .......... .......... 92% 17.4M 0s 2024-07-31T14:53:06.463 INFO:tasks.workunit.client.0.smithi067.stderr:126250K .......... .......... .......... .......... .......... 93% 26.7M 0s 2024-07-31T14:53:06.465 INFO:tasks.workunit.client.0.smithi067.stderr:126300K .......... .......... .......... .......... .......... 93% 15.7M 0s 2024-07-31T14:53:06.467 INFO:tasks.workunit.client.0.smithi067.stderr:126350K .......... .......... .......... .......... .......... 93% 28.8M 0s 2024-07-31T14:53:06.469 INFO:tasks.workunit.client.0.smithi067.stderr:126400K .......... .......... .......... .......... .......... 93% 21.4M 0s 2024-07-31T14:53:06.472 INFO:tasks.workunit.client.0.smithi067.stderr:126450K .......... .......... .......... .......... .......... 93% 17.4M 0s 2024-07-31T14:53:06.475 INFO:tasks.workunit.client.0.smithi067.stderr:126500K .......... .......... .......... .......... .......... 93% 18.8M 0s 2024-07-31T14:53:06.477 INFO:tasks.workunit.client.0.smithi067.stderr:126550K .......... .......... .......... .......... .......... 93% 18.8M 0s 2024-07-31T14:53:06.480 INFO:tasks.workunit.client.0.smithi067.stderr:126600K .......... .......... .......... .......... .......... 93% 28.8M 0s 2024-07-31T14:53:06.481 INFO:tasks.workunit.client.0.smithi067.stderr:126650K .......... .......... .......... .......... .......... 93% 18.4M 0s 2024-07-31T14:53:06.484 INFO:tasks.workunit.client.0.smithi067.stderr:126700K .......... .......... .......... .......... .......... 93% 17.3M 0s 2024-07-31T14:53:06.487 INFO:tasks.workunit.client.0.smithi067.stderr:126750K .......... .......... .......... .......... .......... 93% 21.1M 0s 2024-07-31T14:53:06.489 INFO:tasks.workunit.client.0.smithi067.stderr:126800K .......... .......... .......... .......... .......... 93% 18.0M 0s 2024-07-31T14:53:06.493 INFO:tasks.workunit.client.0.smithi067.stderr:126850K .......... .......... .......... .......... .......... 93% 13.3M 0s 2024-07-31T14:53:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:126900K .......... .......... .......... .......... .......... 93% 19.9M 0s 2024-07-31T14:53:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:126950K .......... .......... .......... .......... .......... 93% 17.0M 0s 2024-07-31T14:53:06.501 INFO:tasks.workunit.client.0.smithi067.stderr:127000K .......... .......... .......... .......... .......... 93% 19.9M 0s 2024-07-31T14:53:06.503 INFO:tasks.workunit.client.0.smithi067.stderr:127050K .......... .......... .......... .......... .......... 93% 19.1M 0s 2024-07-31T14:53:06.506 INFO:tasks.workunit.client.0.smithi067.stderr:127100K .......... .......... .......... .......... .......... 93% 19.5M 0s 2024-07-31T14:53:06.508 INFO:tasks.workunit.client.0.smithi067.stderr:127150K .......... .......... .......... .......... .......... 93% 22.5M 0s 2024-07-31T14:53:06.511 INFO:tasks.workunit.client.0.smithi067.stderr:127200K .......... .......... .......... .......... .......... 93% 17.3M 0s 2024-07-31T14:53:06.513 INFO:tasks.workunit.client.0.smithi067.stderr:127250K .......... .......... .......... .......... .......... 93% 23.1M 0s 2024-07-31T14:53:06.515 INFO:tasks.workunit.client.0.smithi067.stderr:127300K .......... .......... .......... .......... .......... 93% 25.9M 0s 2024-07-31T14:53:06.517 INFO:tasks.workunit.client.0.smithi067.stderr:127350K .......... .......... .......... .......... .......... 93% 19.4M 0s 2024-07-31T14:53:06.519 INFO:tasks.workunit.client.0.smithi067.stderr:127400K .......... .......... .......... .......... .......... 93% 31.0M 0s 2024-07-31T14:53:06.521 INFO:tasks.workunit.client.0.smithi067.stderr:127450K .......... .......... .......... .......... .......... 93% 18.3M 0s 2024-07-31T14:53:06.524 INFO:tasks.workunit.client.0.smithi067.stderr:127500K .......... .......... .......... .......... .......... 93% 22.8M 0s 2024-07-31T14:53:06.527 INFO:tasks.workunit.client.0.smithi067.stderr:127550K .......... .......... .......... .......... .......... 93% 16.0M 0s 2024-07-31T14:53:06.529 INFO:tasks.workunit.client.0.smithi067.stderr:127600K .......... .......... .......... .......... .......... 94% 21.6M 0s 2024-07-31T14:53:06.531 INFO:tasks.workunit.client.0.smithi067.stderr:127650K .......... .......... .......... .......... .......... 94% 22.6M 0s 2024-07-31T14:53:06.534 INFO:tasks.workunit.client.0.smithi067.stderr:127700K .......... .......... .......... .......... .......... 94% 21.0M 0s 2024-07-31T14:53:06.536 INFO:tasks.workunit.client.0.smithi067.stderr:127750K .......... .......... .......... .......... .......... 94% 18.5M 0s 2024-07-31T14:53:06.539 INFO:tasks.workunit.client.0.smithi067.stderr:127800K .......... .......... .......... .......... .......... 94% 16.6M 0s 2024-07-31T14:53:06.541 INFO:tasks.workunit.client.0.smithi067.stderr:127850K .......... .......... .......... .......... .......... 94% 24.0M 0s 2024-07-31T14:53:06.543 INFO:tasks.workunit.client.0.smithi067.stderr:127900K .......... .......... .......... .......... .......... 94% 22.2M 0s 2024-07-31T14:53:06.546 INFO:tasks.workunit.client.0.smithi067.stderr:127950K .......... .......... .......... .......... .......... 94% 17.0M 0s 2024-07-31T14:53:06.550 INFO:tasks.workunit.client.0.smithi067.stderr:128000K .......... .......... .......... .......... .......... 94% 14.5M 0s 2024-07-31T14:53:06.552 INFO:tasks.workunit.client.0.smithi067.stderr:128050K .......... .......... .......... .......... .......... 94% 23.0M 0s 2024-07-31T14:53:06.555 INFO:tasks.workunit.client.0.smithi067.stderr:128100K .......... .......... .......... .......... .......... 94% 12.9M 0s 2024-07-31T14:53:06.557 INFO:tasks.workunit.client.0.smithi067.stderr:128150K .......... .......... .......... .......... .......... 94% 25.1M 0s 2024-07-31T14:53:06.561 INFO:tasks.workunit.client.0.smithi067.stderr:128200K .......... .......... .......... .......... .......... 94% 16.7M 0s 2024-07-31T14:53:06.563 INFO:tasks.workunit.client.0.smithi067.stderr:128250K .......... .......... .......... .......... .......... 94% 20.9M 0s 2024-07-31T14:53:06.565 INFO:tasks.workunit.client.0.smithi067.stderr:128300K .......... .......... .......... .......... .......... 94% 19.8M 0s 2024-07-31T14:53:06.568 INFO:tasks.workunit.client.0.smithi067.stderr:128350K .......... .......... .......... .......... .......... 94% 15.1M 0s 2024-07-31T14:53:06.570 INFO:tasks.workunit.client.0.smithi067.stderr:128400K .......... .......... .......... .......... .......... 94% 29.9M 0s 2024-07-31T14:53:06.572 INFO:tasks.workunit.client.0.smithi067.stderr:128450K .......... .......... .......... .......... .......... 94% 20.1M 0s 2024-07-31T14:53:06.575 INFO:tasks.workunit.client.0.smithi067.stderr:128500K .......... .......... .......... .......... .......... 94% 19.8M 0s 2024-07-31T14:53:06.578 INFO:tasks.workunit.client.0.smithi067.stderr:128550K .......... .......... .......... .......... .......... 94% 23.0M 0s 2024-07-31T14:53:06.579 INFO:tasks.workunit.client.0.smithi067.stderr:128600K .......... .......... .......... .......... .......... 94% 23.7M 0s 2024-07-31T14:53:06.581 INFO:tasks.workunit.client.0.smithi067.stderr:128650K .......... .......... .......... .......... .......... 94% 20.1M 0s 2024-07-31T14:53:06.584 INFO:tasks.workunit.client.0.smithi067.stderr:128700K .......... .......... .......... .......... .......... 94% 19.4M 0s 2024-07-31T14:53:06.586 INFO:tasks.workunit.client.0.smithi067.stderr:128750K .......... .......... .......... .......... .......... 94% 25.7M 0s 2024-07-31T14:53:06.590 INFO:tasks.workunit.client.0.smithi067.stderr:128800K .......... .......... .......... .......... .......... 94% 18.2M 0s 2024-07-31T14:53:06.591 INFO:tasks.workunit.client.0.smithi067.stderr:128850K .......... .......... .......... .......... .......... 94% 19.2M 0s 2024-07-31T14:53:06.593 INFO:tasks.workunit.client.0.smithi067.stderr:128900K .......... .......... .......... .......... .......... 94% 24.2M 0s 2024-07-31T14:53:06.596 INFO:tasks.workunit.client.0.smithi067.stderr:128950K .......... .......... .......... .......... .......... 95% 18.0M 0s 2024-07-31T14:53:06.599 INFO:tasks.workunit.client.0.smithi067.stderr:129000K .......... .......... .......... .......... .......... 95% 16.1M 0s 2024-07-31T14:53:06.601 INFO:tasks.workunit.client.0.smithi067.stderr:129050K .......... .......... .......... .......... .......... 95% 29.4M 0s 2024-07-31T14:53:06.603 INFO:tasks.workunit.client.0.smithi067.stderr:129100K .......... .......... .......... .......... .......... 95% 16.3M 0s 2024-07-31T14:53:06.605 INFO:tasks.workunit.client.0.smithi067.stderr:129150K .......... .......... .......... .......... .......... 95% 26.7M 0s 2024-07-31T14:53:06.611 INFO:tasks.workunit.client.0.smithi067.stderr:129200K .......... .......... .......... .......... .......... 95% 14.3M 0s 2024-07-31T14:53:06.612 INFO:tasks.workunit.client.0.smithi067.stderr:129250K .......... .......... .......... .......... .......... 95% 16.0M 0s 2024-07-31T14:53:06.615 INFO:tasks.workunit.client.0.smithi067.stderr:129300K .......... .......... .......... .......... .......... 95% 18.5M 0s 2024-07-31T14:53:06.617 INFO:tasks.workunit.client.0.smithi067.stderr:129350K .......... .......... .......... .......... .......... 95% 16.7M 0s 2024-07-31T14:53:06.619 INFO:tasks.workunit.client.0.smithi067.stderr:129400K .......... .......... .......... .......... .......... 95% 23.8M 0s 2024-07-31T14:53:06.623 INFO:tasks.workunit.client.0.smithi067.stderr:129450K .......... .......... .......... .......... .......... 95% 14.7M 0s 2024-07-31T14:53:06.626 INFO:tasks.workunit.client.0.smithi067.stderr:129500K .......... .......... .......... .......... .......... 95% 14.9M 0s 2024-07-31T14:53:06.629 INFO:tasks.workunit.client.0.smithi067.stderr:129550K .......... .......... .......... .......... .......... 95% 17.8M 0s 2024-07-31T14:53:06.631 INFO:tasks.workunit.client.0.smithi067.stderr:129600K .......... .......... .......... .......... .......... 95% 21.7M 0s 2024-07-31T14:53:06.633 INFO:tasks.workunit.client.0.smithi067.stderr:129650K .......... .......... .......... .......... .......... 95% 24.6M 0s 2024-07-31T14:53:06.636 INFO:tasks.workunit.client.0.smithi067.stderr:129700K .......... .......... .......... .......... .......... 95% 17.0M 0s 2024-07-31T14:53:06.638 INFO:tasks.workunit.client.0.smithi067.stderr:129750K .......... .......... .......... .......... .......... 95% 21.0M 0s 2024-07-31T14:53:06.641 INFO:tasks.workunit.client.0.smithi067.stderr:129800K .......... .......... .......... .......... .......... 95% 21.9M 0s 2024-07-31T14:53:06.642 INFO:tasks.workunit.client.0.smithi067.stderr:129850K .......... .......... .......... .......... .......... 95% 24.3M 0s 2024-07-31T14:53:06.646 INFO:tasks.workunit.client.0.smithi067.stderr:129900K .......... .......... .......... .......... .......... 95% 16.8M 0s 2024-07-31T14:53:06.647 INFO:tasks.workunit.client.0.smithi067.stderr:129950K .......... .......... .......... .......... .......... 95% 32.6M 0s 2024-07-31T14:53:06.649 INFO:tasks.workunit.client.0.smithi067.stderr:130000K .......... .......... .......... .......... .......... 95% 23.4M 0s 2024-07-31T14:53:06.652 INFO:tasks.workunit.client.0.smithi067.stderr:130050K .......... .......... .......... .......... .......... 95% 15.1M 0s 2024-07-31T14:53:06.654 INFO:tasks.workunit.client.0.smithi067.stderr:130100K .......... .......... .......... .......... .......... 95% 29.5M 0s 2024-07-31T14:53:06.657 INFO:tasks.workunit.client.0.smithi067.stderr:130150K .......... .......... .......... .......... .......... 95% 14.8M 0s 2024-07-31T14:53:06.659 INFO:tasks.workunit.client.0.smithi067.stderr:130200K .......... .......... .......... .......... .......... 95% 29.3M 0s 2024-07-31T14:53:06.661 INFO:tasks.workunit.client.0.smithi067.stderr:130250K .......... .......... .......... .......... .......... 95% 20.4M 0s 2024-07-31T14:53:06.664 INFO:tasks.workunit.client.0.smithi067.stderr:130300K .......... .......... .......... .......... .......... 96% 17.3M 0s 2024-07-31T14:53:06.666 INFO:tasks.workunit.client.0.smithi067.stderr:130350K .......... .......... .......... .......... .......... 96% 22.3M 0s 2024-07-31T14:53:06.670 INFO:tasks.workunit.client.0.smithi067.stderr:130400K .......... .......... .......... .......... .......... 96% 13.3M 0s 2024-07-31T14:53:06.671 INFO:tasks.workunit.client.0.smithi067.stderr:130450K .......... .......... .......... .......... .......... 96% 29.3M 0s 2024-07-31T14:53:06.675 INFO:tasks.workunit.client.0.smithi067.stderr:130500K .......... .......... .......... .......... .......... 96% 16.1M 0s 2024-07-31T14:53:06.678 INFO:tasks.workunit.client.0.smithi067.stderr:130550K .......... .......... .......... .......... .......... 96% 15.9M 0s 2024-07-31T14:53:06.681 INFO:tasks.workunit.client.0.smithi067.stderr:130600K .......... .......... .......... .......... .......... 96% 15.7M 0s 2024-07-31T14:53:06.684 INFO:tasks.workunit.client.0.smithi067.stderr:130650K .......... .......... .......... .......... .......... 96% 17.1M 0s 2024-07-31T14:53:06.685 INFO:tasks.workunit.client.0.smithi067.stderr:130700K .......... .......... .......... .......... .......... 96% 30.5M 0s 2024-07-31T14:53:06.689 INFO:tasks.workunit.client.0.smithi067.stderr:130750K .......... .......... .......... .......... .......... 96% 13.5M 0s 2024-07-31T14:53:06.691 INFO:tasks.workunit.client.0.smithi067.stderr:130800K .......... .......... .......... .......... .......... 96% 19.3M 0s 2024-07-31T14:53:06.693 INFO:tasks.workunit.client.0.smithi067.stderr:130850K .......... .......... .......... .......... .......... 96% 32.7M 0s 2024-07-31T14:53:06.697 INFO:tasks.workunit.client.0.smithi067.stderr:130900K .......... .......... .......... .......... .......... 96% 12.1M 0s 2024-07-31T14:53:06.699 INFO:tasks.workunit.client.0.smithi067.stderr:130950K .......... .......... .......... .......... .......... 96% 25.9M 0s 2024-07-31T14:53:06.701 INFO:tasks.workunit.client.0.smithi067.stderr:131000K .......... .......... .......... .......... .......... 96% 21.3M 0s 2024-07-31T14:53:06.704 INFO:tasks.workunit.client.0.smithi067.stderr:131050K .......... .......... .......... .......... .......... 96% 17.2M 0s 2024-07-31T14:53:06.705 INFO:tasks.workunit.client.0.smithi067.stderr:131100K .......... .......... .......... .......... .......... 96% 28.7M 0s 2024-07-31T14:53:06.708 INFO:tasks.workunit.client.0.smithi067.stderr:131150K .......... .......... .......... .......... .......... 96% 21.7M 0s 2024-07-31T14:53:06.710 INFO:tasks.workunit.client.0.smithi067.stderr:131200K .......... .......... .......... .......... .......... 96% 24.3M 0s 2024-07-31T14:53:06.713 INFO:tasks.workunit.client.0.smithi067.stderr:131250K .......... .......... .......... .......... .......... 96% 20.1M 0s 2024-07-31T14:53:06.716 INFO:tasks.workunit.client.0.smithi067.stderr:131300K .......... .......... .......... .......... .......... 96% 14.4M 0s 2024-07-31T14:53:06.718 INFO:tasks.workunit.client.0.smithi067.stderr:131350K .......... .......... .......... .......... .......... 96% 23.3M 0s 2024-07-31T14:53:06.721 INFO:tasks.workunit.client.0.smithi067.stderr:131400K .......... .......... .......... .......... .......... 96% 17.8M 0s 2024-07-31T14:53:06.722 INFO:tasks.workunit.client.0.smithi067.stderr:131450K .......... .......... .......... .......... .......... 96% 31.7M 0s 2024-07-31T14:53:06.725 INFO:tasks.workunit.client.0.smithi067.stderr:131500K .......... .......... .......... .......... .......... 96% 17.1M 0s 2024-07-31T14:53:06.728 INFO:tasks.workunit.client.0.smithi067.stderr:131550K .......... .......... .......... .......... .......... 96% 16.1M 0s 2024-07-31T14:53:06.730 INFO:tasks.workunit.client.0.smithi067.stderr:131600K .......... .......... .......... .......... .......... 96% 27.8M 0s 2024-07-31T14:53:06.732 INFO:tasks.workunit.client.0.smithi067.stderr:131650K .......... .......... .......... .......... .......... 97% 19.9M 0s 2024-07-31T14:53:06.734 INFO:tasks.workunit.client.0.smithi067.stderr:131700K .......... .......... .......... .......... .......... 97% 21.5M 0s 2024-07-31T14:53:06.738 INFO:tasks.workunit.client.0.smithi067.stderr:131750K .......... .......... .......... .......... .......... 97% 15.6M 0s 2024-07-31T14:53:06.740 INFO:tasks.workunit.client.0.smithi067.stderr:131800K .......... .......... .......... .......... .......... 97% 18.9M 0s 2024-07-31T14:53:06.744 INFO:tasks.workunit.client.0.smithi067.stderr:131850K .......... .......... .......... .......... .......... 97% 13.1M 0s 2024-07-31T14:53:06.747 INFO:tasks.workunit.client.0.smithi067.stderr:131900K .......... .......... .......... .......... .......... 97% 15.6M 0s 2024-07-31T14:53:06.748 INFO:tasks.workunit.client.0.smithi067.stderr:131950K .......... .......... .......... .......... .......... 97% 31.1M 0s 2024-07-31T14:53:06.753 INFO:tasks.workunit.client.0.smithi067.stderr:132000K .......... .......... .......... .......... .......... 97% 12.7M 0s 2024-07-31T14:53:06.755 INFO:tasks.workunit.client.0.smithi067.stderr:132050K .......... .......... .......... .......... .......... 97% 20.6M 0s 2024-07-31T14:53:06.756 INFO:tasks.workunit.client.0.smithi067.stderr:132100K .......... .......... .......... .......... .......... 97% 35.5M 0s 2024-07-31T14:53:06.760 INFO:tasks.workunit.client.0.smithi067.stderr:132150K .......... .......... .......... .......... .......... 97% 12.8M 0s 2024-07-31T14:53:06.762 INFO:tasks.workunit.client.0.smithi067.stderr:132200K .......... .......... .......... .......... .......... 97% 24.2M 0s 2024-07-31T14:53:06.765 INFO:tasks.workunit.client.0.smithi067.stderr:132250K .......... .......... .......... .......... .......... 97% 20.6M 0s 2024-07-31T14:53:06.767 INFO:tasks.workunit.client.0.smithi067.stderr:132300K .......... .......... .......... .......... .......... 97% 17.6M 0s 2024-07-31T14:53:06.769 INFO:tasks.workunit.client.0.smithi067.stderr:132350K .......... .......... .......... .......... .......... 97% 28.1M 0s 2024-07-31T14:53:06.771 INFO:tasks.workunit.client.0.smithi067.stderr:132400K .......... .......... .......... .......... .......... 97% 18.4M 0s 2024-07-31T14:53:06.773 INFO:tasks.workunit.client.0.smithi067.stderr:132450K .......... .......... .......... .......... .......... 97% 26.0M 0s 2024-07-31T14:53:06.775 INFO:tasks.workunit.client.0.smithi067.stderr:132500K .......... .......... .......... .......... .......... 97% 21.4M 0s 2024-07-31T14:53:06.779 INFO:tasks.workunit.client.0.smithi067.stderr:132550K .......... .......... .......... .......... .......... 97% 13.5M 0s 2024-07-31T14:53:06.781 INFO:tasks.workunit.client.0.smithi067.stderr:132600K .......... .......... .......... .......... .......... 97% 29.1M 0s 2024-07-31T14:53:06.784 INFO:tasks.workunit.client.0.smithi067.stderr:132650K .......... .......... .......... .......... .......... 97% 16.6M 0s 2024-07-31T14:53:06.786 INFO:tasks.workunit.client.0.smithi067.stderr:132700K .......... .......... .......... .......... .......... 97% 21.5M 0s 2024-07-31T14:53:06.789 INFO:tasks.workunit.client.0.smithi067.stderr:132750K .......... .......... .......... .......... .......... 97% 17.4M 0s 2024-07-31T14:53:06.792 INFO:tasks.workunit.client.0.smithi067.stderr:132800K .......... .......... .......... .......... .......... 97% 14.9M 0s 2024-07-31T14:53:06.794 INFO:tasks.workunit.client.0.smithi067.stderr:132850K .......... .......... .......... .......... .......... 97% 32.9M 0s 2024-07-31T14:53:06.797 INFO:tasks.workunit.client.0.smithi067.stderr:132900K .......... .......... .......... .......... .......... 97% 15.8M 0s 2024-07-31T14:53:06.799 INFO:tasks.workunit.client.0.smithi067.stderr:132950K .......... .......... .......... .......... .......... 97% 24.1M 0s 2024-07-31T14:53:06.801 INFO:tasks.workunit.client.0.smithi067.stderr:133000K .......... .......... .......... .......... .......... 98% 19.1M 0s 2024-07-31T14:53:06.805 INFO:tasks.workunit.client.0.smithi067.stderr:133050K .......... .......... .......... .......... .......... 98% 15.2M 0s 2024-07-31T14:53:06.807 INFO:tasks.workunit.client.0.smithi067.stderr:133100K .......... .......... .......... .......... .......... 98% 23.8M 0s 2024-07-31T14:53:06.811 INFO:tasks.workunit.client.0.smithi067.stderr:133150K .......... .......... .......... .......... .......... 98% 12.1M 0s 2024-07-31T14:53:06.813 INFO:tasks.workunit.client.0.smithi067.stderr:133200K .......... .......... .......... .......... .......... 98% 23.9M 0s 2024-07-31T14:53:06.818 INFO:tasks.workunit.client.0.smithi067.stderr:133250K .......... .......... .......... .......... .......... 98% 14.7M 0s 2024-07-31T14:53:06.819 INFO:tasks.workunit.client.0.smithi067.stderr:133300K .......... .......... .......... .......... .......... 98% 16.3M 0s 2024-07-31T14:53:06.821 INFO:tasks.workunit.client.0.smithi067.stderr:133350K .......... .......... .......... .......... .......... 98% 33.6M 0s 2024-07-31T14:53:06.825 INFO:tasks.workunit.client.0.smithi067.stderr:133400K .......... .......... .......... .......... .......... 98% 11.1M 0s 2024-07-31T14:53:06.826 INFO:tasks.workunit.client.0.smithi067.stderr:133450K .......... .......... .......... .......... .......... 98% 29.0M 0s 2024-07-31T14:53:06.829 INFO:tasks.workunit.client.0.smithi067.stderr:133500K .......... .......... .......... .......... .......... 98% 30.8M 0s 2024-07-31T14:53:06.832 INFO:tasks.workunit.client.0.smithi067.stderr:133550K .......... .......... .......... .......... .......... 98% 13.0M 0s 2024-07-31T14:53:06.833 INFO:tasks.workunit.client.0.smithi067.stderr:133600K .......... .......... .......... .......... .......... 98% 33.1M 0s 2024-07-31T14:53:06.836 INFO:tasks.workunit.client.0.smithi067.stderr:133650K .......... .......... .......... .......... .......... 98% 19.0M 0s 2024-07-31T14:53:06.838 INFO:tasks.workunit.client.0.smithi067.stderr:133700K .......... .......... .......... .......... .......... 98% 25.3M 0s 2024-07-31T14:53:06.841 INFO:tasks.workunit.client.0.smithi067.stderr:133750K .......... .......... .......... .......... .......... 98% 17.5M 0s 2024-07-31T14:53:06.843 INFO:tasks.workunit.client.0.smithi067.stderr:133800K .......... .......... .......... .......... .......... 98% 19.3M 0s 2024-07-31T14:53:06.847 INFO:tasks.workunit.client.0.smithi067.stderr:133850K .......... .......... .......... .......... .......... 98% 14.7M 0s 2024-07-31T14:53:06.849 INFO:tasks.workunit.client.0.smithi067.stderr:133900K .......... .......... .......... .......... .......... 98% 17.8M 0s 2024-07-31T14:53:06.852 INFO:tasks.workunit.client.0.smithi067.stderr:133950K .......... .......... .......... .......... .......... 98% 19.1M 0s 2024-07-31T14:53:06.855 INFO:tasks.workunit.client.0.smithi067.stderr:134000K .......... .......... .......... .......... .......... 98% 17.2M 0s 2024-07-31T14:53:06.856 INFO:tasks.workunit.client.0.smithi067.stderr:134050K .......... .......... .......... .......... .......... 98% 24.9M 0s 2024-07-31T14:53:06.860 INFO:tasks.workunit.client.0.smithi067.stderr:134100K .......... .......... .......... .......... .......... 98% 22.5M 0s 2024-07-31T14:53:06.862 INFO:tasks.workunit.client.0.smithi067.stderr:134150K .......... .......... .......... .......... .......... 98% 15.6M 0s 2024-07-31T14:53:06.864 INFO:tasks.workunit.client.0.smithi067.stderr:134200K .......... .......... .......... .......... .......... 98% 27.0M 0s 2024-07-31T14:53:06.866 INFO:tasks.workunit.client.0.smithi067.stderr:134250K .......... .......... .......... .......... .......... 98% 23.1M 0s 2024-07-31T14:53:06.868 INFO:tasks.workunit.client.0.smithi067.stderr:134300K .......... .......... .......... .......... .......... 98% 26.0M 0s 2024-07-31T14:53:06.869 INFO:tasks.workunit.client.0.smithi067.stderr:134350K .......... .......... .......... .......... .......... 99% 32.2M 0s 2024-07-31T14:53:06.871 INFO:tasks.workunit.client.0.smithi067.stderr:134400K .......... .......... .......... .......... .......... 99% 31.0M 0s 2024-07-31T14:53:06.872 INFO:tasks.workunit.client.0.smithi067.stderr:134450K .......... .......... .......... .......... .......... 99% 28.1M 0s 2024-07-31T14:53:06.878 INFO:tasks.workunit.client.0.smithi067.stderr:134500K .......... .......... .......... .......... .......... 99% 10.6M 0s 2024-07-31T14:53:06.880 INFO:tasks.workunit.client.0.smithi067.stderr:134550K .......... .......... .......... .......... .......... 99% 21.4M 0s 2024-07-31T14:53:06.882 INFO:tasks.workunit.client.0.smithi067.stderr:134600K .......... .......... .......... .......... .......... 99% 19.6M 0s 2024-07-31T14:53:06.886 INFO:tasks.workunit.client.0.smithi067.stderr:134650K .......... .......... .......... .......... .......... 99% 14.7M 0s 2024-07-31T14:53:06.889 INFO:tasks.workunit.client.0.smithi067.stderr:134700K .......... .......... .......... .......... .......... 99% 29.4M 0s 2024-07-31T14:53:06.891 INFO:tasks.workunit.client.0.smithi067.stderr:134750K .......... .......... .......... .......... .......... 99% 15.0M 0s 2024-07-31T14:53:06.893 INFO:tasks.workunit.client.0.smithi067.stderr:134800K .......... .......... .......... .......... .......... 99% 19.4M 0s 2024-07-31T14:53:06.896 INFO:tasks.workunit.client.0.smithi067.stderr:134850K .......... .......... .......... .......... .......... 99% 18.9M 0s 2024-07-31T14:53:06.897 INFO:tasks.workunit.client.0.smithi067.stderr:134900K .......... .......... .......... .......... .......... 99% 29.6M 0s 2024-07-31T14:53:06.900 INFO:tasks.workunit.client.0.smithi067.stderr:134950K .......... .......... .......... .......... .......... 99% 20.0M 0s 2024-07-31T14:53:06.903 INFO:tasks.workunit.client.0.smithi067.stderr:135000K .......... .......... .......... .......... .......... 99% 19.5M 0s 2024-07-31T14:53:06.906 INFO:tasks.workunit.client.0.smithi067.stderr:135050K .......... .......... .......... .......... .......... 99% 22.5M 0s 2024-07-31T14:53:06.906 INFO:tasks.workunit.client.0.smithi067.stderr:135100K .......... .......... .......... .......... .......... 99% 28.5M 0s 2024-07-31T14:53:06.908 INFO:tasks.workunit.client.0.smithi067.stderr:135150K .......... .......... .......... .......... .......... 99% 24.2M 0s 2024-07-31T14:53:06.915 INFO:tasks.workunit.client.0.smithi067.stderr:135200K .......... .......... .......... .......... .......... 99% 7.03M 0s 2024-07-31T14:53:06.917 INFO:tasks.workunit.client.0.smithi067.stderr:135250K .......... .......... .......... .......... .......... 99% 23.9M 0s 2024-07-31T14:53:06.919 INFO:tasks.workunit.client.0.smithi067.stderr:135300K .......... .......... .......... .......... .......... 99% 29.2M 0s 2024-07-31T14:53:06.920 INFO:tasks.workunit.client.0.smithi067.stderr:135350K .......... .......... .......... .......... .......... 99% 25.8M 0s 2024-07-31T14:53:06.922 INFO:tasks.workunit.client.0.smithi067.stderr:135400K .......... .......... .......... .......... .......... 99% 34.1M 0s 2024-07-31T14:53:06.924 INFO:tasks.workunit.client.0.smithi067.stderr:135450K .......... .......... .......... .......... .......... 99% 18.5M 0s 2024-07-31T14:53:06.935 INFO:tasks.workunit.client.0.smithi067.stderr:135500K .......... .......... .......... .......... .......... 99% 4.60M 0s 2024-07-31T14:53:06.937 INFO:tasks.workunit.client.0.smithi067.stderr:135550K .......... .......... .......... .......... .......... 99% 32.5M 0s 2024-07-31T14:53:06.939 INFO:tasks.workunit.client.0.smithi067.stderr:135600K .......... .......... .......... .......... .......... 99% 23.5M 0s 2024-07-31T14:53:06.941 INFO:tasks.workunit.client.0.smithi067.stderr:135650K .......... .......... .......... .......... .......... 99% 24.7M 0s 2024-07-31T14:53:06.943 INFO:tasks.workunit.client.0.smithi067.stderr:135700K .......... .......... .......... .......... .......... 99% 24.8M 0s 2024-07-31T14:53:06.943 INFO:tasks.workunit.client.0.smithi067.stderr:135750K .. 100% 5.30T=5.2s 2024-07-31T14:53:06.943 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-31T14:53:06.943 INFO:tasks.workunit.client.0.smithi067.stderr:2024-07-31 14:53:06 (25.6 MB/s) - ‘linux.tar.xz’ saved [139010916/139010916] 2024-07-31T14:53:06.943 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-31T14:53:06.943 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir t 2024-07-31T14:53:06.945 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd t 2024-07-31T14:53:06.946 INFO:tasks.workunit.client.0.smithi067.stderr:+ tar xJf ../linux.tar.xz 2024-07-31T14:53:07.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:06 smithi073 ceph-mon[36714]: pgmap v460: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-07-31T14:53:07.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:06 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:53:07.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:06 smithi077 ceph-mon[37327]: pgmap v460: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-07-31T14:53:07.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:06 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:53:07.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:06 smithi067 ceph-mon[32750]: pgmap v460: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-07-31T14:53:07.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:06 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:53:07.328 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T14:53:07.329 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:53:07.636 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:53:09.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:08 smithi073 ceph-mon[36714]: pgmap v461: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-07-31T14:53:09.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:08 smithi067 ceph-mon[32750]: pgmap v461: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-07-31T14:53:09.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:08 smithi077 ceph-mon[37327]: pgmap v461: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-07-31T14:53:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.428+0000 7f2cde018640 1 -- 172.21.15.67:0/1632068663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cd8107df0 msgr2=0x7f2cd8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.428+0000 7f2cde018640 1 --2- 172.21.15.67:0/1632068663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cd8107df0 0x7f2cd8108200 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009900 tx=0x7f2ccc02a410 comp rx=0 tx=0).stop 2024-07-31T14:53:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.429+0000 7f2cde018640 1 -- 172.21.15.67:0/1632068663 shutdown_connections 2024-07-31T14:53:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.429+0000 7f2cde018640 1 --2- 172.21.15.67:0/1632068663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cd8108740 0x7f2cd810ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:09.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.429+0000 7f2cde018640 1 --2- 172.21.15.67:0/1632068663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cd8107df0 0x7f2cd8108200 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:09.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.429+0000 7f2cde018640 1 --2- 172.21.15.67:0/1632068663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cd81024b0 0x7f2cd81028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:09.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.429+0000 7f2cde018640 1 -- 172.21.15.67:0/1632068663 >> 172.21.15.67:0/1632068663 conn(0x7f2cd8079170 msgr2=0x7f2cd8079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:09.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.429+0000 7f2cde018640 1 -- 172.21.15.67:0/1632068663 shutdown_connections 2024-07-31T14:53:09.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.429+0000 7f2cde018640 1 -- 172.21.15.67:0/1632068663 wait complete. 2024-07-31T14:53:09.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.429+0000 7f2cde018640 1 Processor -- start 2024-07-31T14:53:09.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.430+0000 7f2cde018640 1 -- start start 2024-07-31T14:53:09.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.430+0000 7f2cde018640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cd81024b0 0x7f2cd8199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:09.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.430+0000 7f2cde018640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cd8107df0 0x7f2cd819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:09.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.430+0000 7f2cde018640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cd8108740 0x7f2cd819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:09.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.430+0000 7f2cde018640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd8112370 con 0x7f2cd8107df0 2024-07-31T14:53:09.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.430+0000 7f2cde018640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81121f0 con 0x7f2cd81024b0 2024-07-31T14:53:09.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.430+0000 7f2cde018640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81124f0 con 0x7f2cd8108740 2024-07-31T14:53:09.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.430+0000 7f2cd6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cd8107df0 0x7f2cd819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:09.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.430+0000 7f2cd6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cd8107df0 0x7f2cd819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59324/0 (socket says 172.21.15.67:59324) 2024-07-31T14:53:09.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.430+0000 7f2cd6ffd640 1 -- 172.21.15.67:0/3042769955 learned_addr learned my addr 172.21.15.67:0/3042769955 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:09.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.430+0000 7f2cd77fe640 1 --2- 172.21.15.67:0/3042769955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cd81024b0 0x7f2cd8199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:09.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.430+0000 7f2cd7fff640 1 --2- 172.21.15.67:0/3042769955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cd8108740 0x7f2cd819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:09.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.431+0000 7f2cd6ffd640 1 -- 172.21.15.67:0/3042769955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cd8108740 msgr2=0x7f2cd819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:09.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.431+0000 7f2cd6ffd640 1 --2- 172.21.15.67:0/3042769955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cd8108740 0x7f2cd819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:09.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.431+0000 7f2cd6ffd640 1 -- 172.21.15.67:0/3042769955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cd81024b0 msgr2=0x7f2cd8199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:09.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.431+0000 7f2cd6ffd640 1 --2- 172.21.15.67:0/3042769955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cd81024b0 0x7f2cd8199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:09.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.431+0000 7f2cd6ffd640 1 -- 172.21.15.67:0/3042769955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd819fe00 con 0x7f2cd8107df0 2024-07-31T14:53:09.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.431+0000 7f2cd6ffd640 1 --2- 172.21.15.67:0/3042769955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cd8107df0 0x7f2cd819a430 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc032fd0 tx=0x7f2ccc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:09.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.431+0000 7f2cd4ff9640 1 -- 172.21.15.67:0/3042769955 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc03f030 con 0x7f2cd8107df0 2024-07-31T14:53:09.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.431+0000 7f2cde018640 1 -- 172.21.15.67:0/3042769955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd81a0090 con 0x7f2cd8107df0 2024-07-31T14:53:09.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.431+0000 7f2cd4ff9640 1 -- 172.21.15.67:0/3042769955 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ccc004490 con 0x7f2cd8107df0 2024-07-31T14:53:09.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.431+0000 7f2cd4ff9640 1 -- 172.21.15.67:0/3042769955 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc035070 con 0x7f2cd8107df0 2024-07-31T14:53:09.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.431+0000 7f2cde018640 1 -- 172.21.15.67:0/3042769955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd8104a10 con 0x7f2cd8107df0 2024-07-31T14:53:09.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.432+0000 7f2cde018640 1 -- 172.21.15.67:0/3042769955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd8103730 con 0x7f2cd8107df0 2024-07-31T14:53:09.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.435+0000 7f2cd4ff9640 1 -- 172.21.15.67:0/3042769955 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2ccc042030 con 0x7f2cd8107df0 2024-07-31T14:53:09.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.436+0000 7f2cd4ff9640 1 --2- 172.21.15.67:0/3042769955 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2cc00b3d40 0x7f2cc00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:09.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.436+0000 7f2cd77fe640 1 --2- 172.21.15.67:0/3042769955 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2cc00b3d40 0x7f2cc00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:09.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.436+0000 7f2cd4ff9640 1 -- 172.21.15.67:0/3042769955 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f2ccc104b80 con 0x7f2cd8107df0 2024-07-31T14:53:09.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.436+0000 7f2cd77fe640 1 --2- 172.21.15.67:0/3042769955 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2cc00b3d40 0x7f2cc00b6250 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8005d00 tx=0x7f2cc800b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:09.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.439+0000 7f2cd4ff9640 1 -- 172.21.15.67:0/3042769955 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f2ccc0cb530 con 0x7f2cd8107df0 2024-07-31T14:53:09.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.673+0000 7f2cde018640 1 -- 172.21.15.67:0/3042769955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cd81028c0 con 0x7f2cd8107df0 2024-07-31T14:53:09.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.673+0000 7f2cd4ff9640 1 -- 172.21.15.67:0/3042769955 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11962 (secure 0 0 0) 0x7f2ccc0cfa80 con 0x7f2cd8107df0 2024-07-31T14:53:09.678 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:53:09.678 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-31T14:51:08:515297+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:51:07.734542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2854264361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2854264361},{"type":"v1","addr":"172.21.15.73:6835","nonce":2854264361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:53:09.679 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-31T14:53:09.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.675+0000 7f2cde018640 1 -- 172.21.15.67:0/3042769955 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2cc00b3d40 msgr2=0x7f2cc00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:09.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.675+0000 7f2cde018640 1 --2- 172.21.15.67:0/3042769955 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2cc00b3d40 0x7f2cc00b6250 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8005d00 tx=0x7f2cc800b410 comp rx=0 tx=0).stop 2024-07-31T14:53:09.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.675+0000 7f2cde018640 1 -- 172.21.15.67:0/3042769955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cd8107df0 msgr2=0x7f2cd819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:09.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.675+0000 7f2cde018640 1 --2- 172.21.15.67:0/3042769955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cd8107df0 0x7f2cd819a430 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc032fd0 tx=0x7f2ccc004270 comp rx=0 tx=0).stop 2024-07-31T14:53:09.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.675+0000 7f2cde018640 1 -- 172.21.15.67:0/3042769955 shutdown_connections 2024-07-31T14:53:09.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.675+0000 7f2cde018640 1 --2- 172.21.15.67:0/3042769955 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2cc00b3d40 0x7f2cc00b6250 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:09.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.675+0000 7f2cde018640 1 --2- 172.21.15.67:0/3042769955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cd8108740 0x7f2cd819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:09.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.675+0000 7f2cde018640 1 --2- 172.21.15.67:0/3042769955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cd8107df0 0x7f2cd819a430 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:09.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.675+0000 7f2cde018640 1 --2- 172.21.15.67:0/3042769955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cd81024b0 0x7f2cd8199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:09.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.676+0000 7f2cde018640 1 -- 172.21.15.67:0/3042769955 >> 172.21.15.67:0/3042769955 conn(0x7f2cd8079170 msgr2=0x7f2cd806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:09.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.676+0000 7f2cde018640 1 -- 172.21.15.67:0/3042769955 shutdown_connections 2024-07-31T14:53:09.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:09.676+0000 7f2cde018640 1 -- 172.21.15.67:0/3042769955 wait complete. 2024-07-31T14:53:10.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:09 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3042769955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:53:10.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:09 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3042769955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:53:10.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:09 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3042769955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:53:10.536 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T14:53:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:53:11.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:10 smithi067 ceph-mon[32750]: pgmap v462: 129 pgs: 129 active+clean; 86 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 32 op/s 2024-07-31T14:53:11.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:10 smithi073 ceph-mon[36714]: pgmap v462: 129 pgs: 129 active+clean; 86 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 32 op/s 2024-07-31T14:53:11.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:10 smithi077 ceph-mon[37327]: pgmap v462: 129 pgs: 129 active+clean; 86 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 32 op/s 2024-07-31T14:53:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.715+0000 7f1980bb0640 1 -- 172.21.15.67:0/1662744421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f197c108690 msgr2=0x7f197c10eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.715+0000 7f1980bb0640 1 --2- 172.21.15.67:0/1662744421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f197c108690 0x7f197c10eec0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1974009ca0 tx=0x7f197402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.715+0000 7f1980bb0640 1 -- 172.21.15.67:0/1662744421 shutdown_connections 2024-07-31T14:53:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.715+0000 7f1980bb0640 1 --2- 172.21.15.67:0/1662744421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f197c108690 0x7f197c10eec0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.715+0000 7f1980bb0640 1 --2- 172.21.15.67:0/1662744421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f197c107d40 0x7f197c108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.715+0000 7f1980bb0640 1 --2- 172.21.15.67:0/1662744421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c102410 0x7f197c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.715+0000 7f1980bb0640 1 -- 172.21.15.67:0/1662744421 >> 172.21.15.67:0/1662744421 conn(0x7f197c07a140 msgr2=0x7f197c07a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.715+0000 7f1980bb0640 1 -- 172.21.15.67:0/1662744421 shutdown_connections 2024-07-31T14:53:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.715+0000 7f1980bb0640 1 -- 172.21.15.67:0/1662744421 wait complete. 2024-07-31T14:53:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.716+0000 7f1980bb0640 1 Processor -- start 2024-07-31T14:53:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.716+0000 7f1980bb0640 1 -- start start 2024-07-31T14:53:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f1980bb0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f197c102410 0x7f197c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f1980bb0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c107d40 0x7f197c19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f1980bb0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f197c108690 0x7f197c19b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f1980bb0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f197c112300 con 0x7f197c108690 2024-07-31T14:53:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f1980bb0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f197c112180 con 0x7f197c107d40 2024-07-31T14:53:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f1980bb0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f197c112480 con 0x7f197c102410 2024-07-31T14:53:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f197ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f197c108690 0x7f197c19b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f1973fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c107d40 0x7f197c19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f197ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f197c108690 0x7f197c19b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40088/0 (socket says 172.21.15.67:40088) 2024-07-31T14:53:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f1973fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c107d40 0x7f197c19b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49050/0 (socket says 172.21.15.67:49050) 2024-07-31T14:53:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f197ad9d640 1 -- 172.21.15.67:0/1483659971 learned_addr learned my addr 172.21.15.67:0/1483659971 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f197a59c640 1 --2- 172.21.15.67:0/1483659971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f197c102410 0x7f197c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f197ad9d640 1 -- 172.21.15.67:0/1483659971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f197c102410 msgr2=0x7f197c19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f197ad9d640 1 --2- 172.21.15.67:0/1483659971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f197c102410 0x7f197c19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f197ad9d640 1 -- 172.21.15.67:0/1483659971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c107d40 msgr2=0x7f197c19b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f197ad9d640 1 --2- 172.21.15.67:0/1483659971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c107d40 0x7f197c19b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f197ad9d640 1 -- 172.21.15.67:0/1483659971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197c1a5e80 con 0x7f197c108690 2024-07-31T14:53:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.717+0000 7f197ad9d640 1 --2- 172.21.15.67:0/1483659971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f197c108690 0x7f197c19b670 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1974005cc0 tx=0x7f1974031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.718+0000 7f19737fe640 1 -- 172.21.15.67:0/1483659971 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974003b70 con 0x7f197c108690 2024-07-31T14:53:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.718+0000 7f19737fe640 1 -- 172.21.15.67:0/1483659971 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19740318f0 con 0x7f197c108690 2024-07-31T14:53:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.718+0000 7f1980bb0640 1 -- 172.21.15.67:0/1483659971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197c1a6110 con 0x7f197c108690 2024-07-31T14:53:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.718+0000 7f19737fe640 1 -- 172.21.15.67:0/1483659971 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197402f950 con 0x7f197c108690 2024-07-31T14:53:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.718+0000 7f1980bb0640 1 -- 172.21.15.67:0/1483659971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197c1a6660 con 0x7f197c108690 2024-07-31T14:53:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.720+0000 7f1987b78740 1 Processor -- start 2024-07-31T14:53:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.721+0000 7f19737fe640 1 -- 172.21.15.67:0/1483659971 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f197402fb50 con 0x7f197c108690 2024-07-31T14:53:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.721+0000 7f1987b78740 1 -- start start 2024-07-31T14:53:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.721+0000 7f1987b78740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3cad1b0 0x5630d404fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.721+0000 7f1987b78740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 0x5630d4050220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.721+0000 7f1987b78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c8c4d0 0x5630d4050760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.721+0000 7f1987b78740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5630d3fc53b0 con 0x5630d3cad1b0 2024-07-31T14:53:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.721+0000 7f1987b78740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5630d3fb48b0 con 0x5630d3c842f0 2024-07-31T14:53:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.721+0000 7f1987b78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5630d3fb88f0 con 0x5630d3c8c4d0 2024-07-31T14:53:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.721+0000 7f1973fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3cad1b0 0x5630d404fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.721+0000 7f1973fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3cad1b0 0x5630d404fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40102/0 (socket says 172.21.15.67:40102) 2024-07-31T14:53:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.721+0000 7f1973fff640 1 -- 172.21.15.67:0/12444207 learned_addr learned my addr 172.21.15.67:0/12444207 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.721+0000 7f197a59c640 1 --2- 172.21.15.67:0/12444207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 0x5630d4050220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.722+0000 7f1973fff640 1 --2- 172.21.15.67:0/12444207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c8c4d0 0x5630d4050760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.722+0000 7f19737fe640 1 --2- 172.21.15.67:0/1483659971 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19680b3d40 0x7f19680b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.722+0000 7f197a59c640 1 -- 172.21.15.67:0/12444207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c8c4d0 msgr2=0x5630d4050760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.722+0000 7f197a59c640 1 --2- 172.21.15.67:0/12444207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c8c4d0 0x5630d4050760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.722+0000 7f197a59c640 1 -- 172.21.15.67:0/12444207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3cad1b0 msgr2=0x5630d404fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.722+0000 7f197a59c640 1 --2- 172.21.15.67:0/12444207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3cad1b0 0x5630d404fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.722+0000 7f197a59c640 1 -- 172.21.15.67:0/12444207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d3f5b9a0 con 0x5630d3c842f0 2024-07-31T14:53:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.722+0000 7f1973fff640 1 --2- 172.21.15.67:0/12444207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3cad1b0 0x5630d404fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.722+0000 7f197ad9d640 1 --2- 172.21.15.67:0/1483659971 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19680b3d40 0x7f19680b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.722+0000 7f197a59c640 1 --2- 172.21.15.67:0/12444207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 0x5630d4050220 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f196c009c20 tx=0x7f196c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c9e3a643d9ead8f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.722+0000 7f19737fe640 1 -- 172.21.15.67:0/1483659971 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f1974105590 con 0x7f197c108690 2024-07-31T14:53:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.722+0000 7f1970ff9640 1 -- 172.21.15.67:0/12444207 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c004050 con 0x5630d3c842f0 2024-07-31T14:53:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.722+0000 7f1970ff9640 1 -- 172.21.15.67:0/12444207 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f196c0054d0 con 0x5630d3c842f0 2024-07-31T14:53:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.722+0000 7f1970ff9640 1 -- 172.21.15.67:0/12444207 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c004c60 con 0x5630d3c842f0 2024-07-31T14:53:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.723+0000 7f1987b78740 1 -- 172.21.15.67:0/12444207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 msgr2=0x5630d4050220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.723+0000 7f197ad9d640 1 --2- 172.21.15.67:0/1483659971 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19680b3d40 0x7f19680b6250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1974102b90 tx=0x7f19740b3040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.723+0000 7f1987b78740 1 --2- 172.21.15.67:0/12444207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 0x5630d4050220 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f196c009c20 tx=0x7f196c011810 comp rx=0 tx=0).stop 2024-07-31T14:53:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.723+0000 7f1987b78740 1 -- 172.21.15.67:0/12444207 shutdown_connections 2024-07-31T14:53:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.723+0000 7f1987b78740 1 --2- 172.21.15.67:0/12444207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c8c4d0 0x5630d4050760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.723+0000 7f1987b78740 1 --2- 172.21.15.67:0/12444207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 0x5630d4050220 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.723+0000 7f1987b78740 1 --2- 172.21.15.67:0/12444207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3cad1b0 0x5630d404fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.723+0000 7f1987b78740 1 -- 172.21.15.67:0/12444207 >> 172.21.15.67:0/12444207 conn(0x5630d3ca1300 msgr2=0x5630d404dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.723+0000 7f1987b78740 1 -- 172.21.15.67:0/12444207 shutdown_connections 2024-07-31T14:53:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.723+0000 7f1987b78740 1 -- 172.21.15.67:0/12444207 wait complete. 2024-07-31T14:53:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.724+0000 7f1987b78740 1 -- start start 2024-07-31T14:53:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.724+0000 7f1987b78740 1 Processor -- start 2024-07-31T14:53:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.724+0000 7f1987b78740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 0x5630d405b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.724+0000 7f1987b78740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3c8c4d0 0x5630d405bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.724+0000 7f1987b78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3cad1b0 0x5630d4067780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.724+0000 7f1987b78740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5630d3eeeab0 con 0x5630d3c8c4d0 2024-07-31T14:53:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.724+0000 7f1987b78740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5630d3f3ccf0 con 0x5630d3c842f0 2024-07-31T14:53:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.724+0000 7f1987b78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5630d405e460 con 0x5630d3cad1b0 2024-07-31T14:53:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.724+0000 7f1987b78740 10 client.0 _ll_register_callbacks cb 0x5630d3fae9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.724+0000 7f197a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3c8c4d0 0x5630d405bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.724+0000 7f1973fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 0x5630d405b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.724+0000 7f197a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3c8c4d0 0x5630d405bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40118/0 (socket says 172.21.15.67:40118) 2024-07-31T14:53:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.724+0000 7f197a59c640 1 -- 172.21.15.67:0/3186100091 learned_addr learned my addr 172.21.15.67:0/3186100091 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.724+0000 7f1973fff640 1 --2- 172.21.15.67:0/3186100091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3cad1b0 0x5630d4067780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f197a59c640 1 -- 172.21.15.67:0/3186100091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3cad1b0 msgr2=0x5630d4067780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f197a59c640 1 --2- 172.21.15.67:0/3186100091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3cad1b0 0x5630d4067780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f197a59c640 1 -- 172.21.15.67:0/3186100091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 msgr2=0x5630d405b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f197a59c640 1 --2- 172.21.15.67:0/3186100091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 0x5630d405b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f197a59c640 1 -- 172.21.15.67:0/3186100091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d3d18790 con 0x5630d3c8c4d0 2024-07-31T14:53:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f1973fff640 1 --2- 172.21.15.67:0/3186100091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3cad1b0 0x5630d4067780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:53:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f197a59c640 1 --2- 172.21.15.67:0/3186100091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3c8c4d0 0x5630d405bd80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f196c000c00 tx=0x7f196c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d201bb279afc43 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f1961ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f1961ffb640 1 -- 172.21.15.67:0/3186100091 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c027070 con 0x5630d3c8c4d0 2024-07-31T14:53:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f1961ffb640 1 -- 172.21.15.67:0/3186100091 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f196c003a00 con 0x5630d3c8c4d0 2024-07-31T14:53:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f1987b78740 1 -- 172.21.15.67:0/3186100091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f196c0054d0 con 0x5630d3c8c4d0 2024-07-31T14:53:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f1961ffb640 1 -- 172.21.15.67:0/3186100091 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f196c033050 con 0x5630d3c8c4d0 2024-07-31T14:53:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f1987b78740 10 client.16227 fetch_fsmap learned FSMap version 30 2024-07-31T14:53:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f1987b78740 1 -- 172.21.15.67:0/3186100091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5630d3d91620 con 0x5630d3c8c4d0 2024-07-31T14:53:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.725+0000 7f1961ffb640 1 -- 172.21.15.67:0/3186100091 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f196c0054d0 con 0x5630d3c8c4d0 2024-07-31T14:53:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.726+0000 7f1961ffb640 1 -- 172.21.15.67:0/3186100091 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f196c023460 con 0x5630d3c8c4d0 2024-07-31T14:53:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.726+0000 7f1987b78740 10 client.16227 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:53:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.726+0000 7f1987b78740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.726+0000 7f1987b78740 20 client.16227 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.726+0000 7f1987b78740 1 --2- 172.21.15.67:0/3186100091 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5630d3c6bd80 0x5630d406d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.726+0000 7f1987b78740 4 client.16227 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:53:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.726+0000 7f1987b78740 1 -- 172.21.15.67:0/3186100091 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630d3fc7300 con 0x5630d3c6bd80 2024-07-31T14:53:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.726+0000 7f1973fff640 1 --2- 172.21.15.67:0/3186100091 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5630d3c6bd80 0x5630d406d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.726+0000 7f1961ffb640 10 client.16227 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.726+0000 7f1973fff640 1 --2- 172.21.15.67:0/3186100091 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5630d3c6bd80 0x5630d406d620 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed4e023003aef76d server_cookie=826641e6854a516e in_seq=0 out_seq=0 2024-07-31T14:53:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1961ffb640 1 -- 172.21.15.67:0/3186100091 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630d3fc7300 con 0x5630d3c6bd80 2024-07-31T14:53:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1961ffb640 10 client.16227 handle_command_reply: tid=0 2024-07-31T14:53:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1987b78740 1 client.16227 shutdown 2024-07-31T14:53:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1987b78740 1 -- 172.21.15.67:0/3186100091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3c8c4d0 msgr2=0x5630d405bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1987b78740 1 --2- 172.21.15.67:0/3186100091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3c8c4d0 0x5630d405bd80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f196c000c00 tx=0x7f196c0046c0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1987b78740 1 -- 172.21.15.67:0/3186100091 shutdown_connections 2024-07-31T14:53:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1987b78740 1 --2- 172.21.15.67:0/3186100091 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5630d3c6bd80 0x5630d406d620 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1987b78740 1 --2- 172.21.15.67:0/3186100091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3cad1b0 0x5630d4067780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1987b78740 1 --2- 172.21.15.67:0/3186100091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3c8c4d0 0x5630d405bd80 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1987b78740 1 --2- 172.21.15.67:0/3186100091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 0x5630d405b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1961ffb640 0 client.16227 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1987b78740 1 -- 172.21.15.67:0/3186100091 >> 172.21.15.67:0/3186100091 conn(0x5630d3ca1300 msgr2=0x5630d404c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1987b78740 1 -- 172.21.15.67:0/3186100091 shutdown_connections 2024-07-31T14:53:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1987b78740 1 -- 172.21.15.67:0/3186100091 wait complete. 2024-07-31T14:53:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.728+0000 7f1987b78740 20 client.16227 trim_cache size 0 max 16384 2024-07-31T14:53:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1987b78740 1 Processor -- start 2024-07-31T14:53:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1987b78740 1 -- start start 2024-07-31T14:53:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1987b78740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 0x5630d405d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1987b78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c8c4d0 0x5630d405aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1987b78740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3cad1b0 0x5630d4072c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1987b78740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f197c1b0650 con 0x5630d3cad1b0 2024-07-31T14:53:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1987b78740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f197c1b2190 con 0x5630d3c842f0 2024-07-31T14:53:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1987b78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f197c1b0d20 con 0x5630d3c8c4d0 2024-07-31T14:53:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1973fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3cad1b0 0x5630d4072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1973fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3cad1b0 0x5630d4072c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40130/0 (socket says 172.21.15.67:40130) 2024-07-31T14:53:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1973fff640 1 -- 172.21.15.67:0/536068762 learned_addr learned my addr 172.21.15.67:0/536068762 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1973fff640 1 --2- 172.21.15.67:0/536068762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 0x5630d405d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f197a59c640 1 --2- 172.21.15.67:0/536068762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c8c4d0 0x5630d405aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1973fff640 1 -- 172.21.15.67:0/536068762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c8c4d0 msgr2=0x5630d405aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1973fff640 1 --2- 172.21.15.67:0/536068762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c8c4d0 0x5630d405aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1973fff640 1 -- 172.21.15.67:0/536068762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 msgr2=0x5630d405d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1973fff640 1 --2- 172.21.15.67:0/536068762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 0x5630d405d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1973fff640 1 -- 172.21.15.67:0/536068762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d405c4d0 con 0x5630d3cad1b0 2024-07-31T14:53:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.746+0000 7f1973fff640 1 --2- 172.21.15.67:0/536068762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3cad1b0 0x5630d4072c50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1964000c00 tx=0x7f1964017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ce29b3faef8d55 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.747+0000 7f1963fff640 1 -- 172.21.15.67:0/536068762 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964028040 con 0x5630d3cad1b0 2024-07-31T14:53:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.747+0000 7f1963fff640 1 -- 172.21.15.67:0/536068762 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19640035f0 con 0x5630d3cad1b0 2024-07-31T14:53:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.747+0000 7f1963fff640 1 -- 172.21.15.67:0/536068762 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964010690 con 0x5630d3cad1b0 2024-07-31T14:53:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.747+0000 7f1987b78740 1 -- 172.21.15.67:0/536068762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3cad1b0 msgr2=0x5630d4072c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.747+0000 7f1987b78740 1 --2- 172.21.15.67:0/536068762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3cad1b0 0x5630d4072c50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1964000c00 tx=0x7f1964017040 comp rx=0 tx=0).stop 2024-07-31T14:53:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.748+0000 7f1987b78740 1 -- 172.21.15.67:0/536068762 shutdown_connections 2024-07-31T14:53:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.748+0000 7f1987b78740 1 --2- 172.21.15.67:0/536068762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3cad1b0 0x5630d4072c50 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.748+0000 7f1987b78740 1 --2- 172.21.15.67:0/536068762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c8c4d0 0x5630d405aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.748+0000 7f1987b78740 1 --2- 172.21.15.67:0/536068762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3c842f0 0x5630d405d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.748+0000 7f1987b78740 1 -- 172.21.15.67:0/536068762 >> 172.21.15.67:0/536068762 conn(0x5630d3ca1300 msgr2=0x5630d4054e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.748+0000 7f1987b78740 1 -- 172.21.15.67:0/536068762 shutdown_connections 2024-07-31T14:53:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.748+0000 7f1987b78740 1 -- 172.21.15.67:0/536068762 wait complete. 2024-07-31T14:53:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.748+0000 7f1987b78740 1 -- start start 2024-07-31T14:53:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.748+0000 7f1987b78740 1 Processor -- start 2024-07-31T14:53:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1987b78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c842f0 0x5630d4063d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1987b78740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3c8c4d0 0x5630d40606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1987b78740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3cad1b0 0x5630d4060bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1987b78740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f197c1b1ac0 con 0x5630d3c8c4d0 2024-07-31T14:53:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1987b78740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f197c1b13f0 con 0x5630d3cad1b0 2024-07-31T14:53:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1987b78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f197c1ae630 con 0x5630d3c842f0 2024-07-31T14:53:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1987b78740 10 client.0 _ll_register_callbacks cb 0x7f19640097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f197a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3c8c4d0 0x5630d40606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f197a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3c8c4d0 0x5630d40606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40136/0 (socket says 172.21.15.67:40136) 2024-07-31T14:53:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f197a59c640 1 -- 172.21.15.67:0/2782328687 learned_addr learned my addr 172.21.15.67:0/2782328687 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1973fff640 1 --2- 172.21.15.67:0/2782328687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3cad1b0 0x5630d4060bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1973fff640 1 --2- 172.21.15.67:0/2782328687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c842f0 0x5630d4063d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f197a59c640 1 -- 172.21.15.67:0/2782328687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c842f0 msgr2=0x5630d4063d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f197a59c640 1 --2- 172.21.15.67:0/2782328687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c842f0 0x5630d4063d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f197a59c640 1 -- 172.21.15.67:0/2782328687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3cad1b0 msgr2=0x5630d4060bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f197a59c640 1 --2- 172.21.15.67:0/2782328687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3cad1b0 0x5630d4060bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f197a59c640 1 -- 172.21.15.67:0/2782328687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d4052210 con 0x5630d3c8c4d0 2024-07-31T14:53:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1973fff640 1 --2- 172.21.15.67:0/2782328687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c842f0 0x5630d4063d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:53:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f197a59c640 1 --2- 172.21.15.67:0/2782328687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3c8c4d0 0x5630d40606b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f196c023a20 tx=0x7f196c024fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f4ba60ca20edf8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1961ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1961ffb640 1 -- 172.21.15.67:0/2782328687 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c027070 con 0x5630d3c8c4d0 2024-07-31T14:53:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1961ffb640 1 -- 172.21.15.67:0/2782328687 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f196c023aa0 con 0x5630d3c8c4d0 2024-07-31T14:53:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1987b78740 1 -- 172.21.15.67:0/2782328687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19640035f0 con 0x5630d3c8c4d0 2024-07-31T14:53:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.749+0000 7f1961ffb640 1 -- 172.21.15.67:0/2782328687 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f196c005040 con 0x5630d3c8c4d0 2024-07-31T14:53:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.750+0000 7f1987b78740 10 client.16233 fetch_fsmap learned FSMap version 30 2024-07-31T14:53:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.750+0000 7f1987b78740 1 -- 172.21.15.67:0/2782328687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5630d40d4920 con 0x5630d3c8c4d0 2024-07-31T14:53:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.750+0000 7f1961ffb640 1 -- 172.21.15.67:0/2782328687 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19640035f0 con 0x5630d3c8c4d0 2024-07-31T14:53:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.750+0000 7f1961ffb640 1 -- 172.21.15.67:0/2782328687 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f196c004730 con 0x5630d3c8c4d0 2024-07-31T14:53:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.750+0000 7f1987b78740 10 client.16233 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:53:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.750+0000 7f1987b78740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.750+0000 7f1987b78740 20 client.16233 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.750+0000 7f1987b78740 1 --2- 172.21.15.67:0/2782328687 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5630d3c411c0 0x5630d40d7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.750+0000 7f1987b78740 4 client.16233 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:53:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.750+0000 7f1987b78740 1 -- 172.21.15.67:0/2782328687 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630d3fc2da0 con 0x5630d3c411c0 2024-07-31T14:53:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.750+0000 7f1973fff640 1 --2- 172.21.15.67:0/2782328687 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5630d3c411c0 0x5630d40d7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.750+0000 7f1973fff640 1 --2- 172.21.15.67:0/2782328687 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5630d3c411c0 0x5630d40d7070 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b97d2534a83f501 server_cookie=1c4eb7eff6e15a10 in_seq=0 out_seq=0 2024-07-31T14:53:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:11.750+0000 7f1961ffb640 10 client.16233 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:12.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.836+0000 7f9966880640 1 -- 172.21.15.67:0/23141828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9960111970 msgr2=0x7f9960109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:12.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.836+0000 7f9966880640 1 --2- 172.21.15.67:0/23141828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9960111970 0x7f9960109df0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f995801acb0 tx=0x7f995803b8e0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.836+0000 7f9966880640 1 -- 172.21.15.67:0/23141828 shutdown_connections 2024-07-31T14:53:12.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.836+0000 7f9966880640 1 --2- 172.21.15.67:0/23141828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f996011c660 0x7f996011eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.836+0000 7f9966880640 1 --2- 172.21.15.67:0/23141828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f996010a410 0x7f996010a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.836+0000 7f9966880640 1 --2- 172.21.15.67:0/23141828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9960111970 0x7f9960109df0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.836+0000 7f9966880640 1 -- 172.21.15.67:0/23141828 >> 172.21.15.67:0/23141828 conn(0x7f99600708f0 msgr2=0x7f9960070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:12.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.839+0000 7f9966880640 1 -- 172.21.15.67:0/23141828 shutdown_connections 2024-07-31T14:53:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.840+0000 7f9966880640 1 -- 172.21.15.67:0/23141828 wait complete. 2024-07-31T14:53:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.841+0000 7f9966880640 1 Processor -- start 2024-07-31T14:53:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.841+0000 7f9966880640 1 -- start start 2024-07-31T14:53:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.842+0000 7f9966880640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f996010a410 0x7f996011be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.842+0000 7f9966880640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9960111970 0x7f9960118120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.842+0000 7f9966880640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f996011c660 0x7f9960118680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.842+0000 7f9966880640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9960114640 con 0x7f996010a410 2024-07-31T14:53:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.842+0000 7f9966880640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f99601144c0 con 0x7f996011c660 2024-07-31T14:53:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.842+0000 7f9966880640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f99601147c0 con 0x7f9960111970 2024-07-31T14:53:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.843+0000 7f995ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f996010a410 0x7f996011be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.843+0000 7f995ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f996010a410 0x7f996011be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40148/0 (socket says 172.21.15.67:40148) 2024-07-31T14:53:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.843+0000 7f995ffff640 1 -- 172.21.15.67:0/1243936038 learned_addr learned my addr 172.21.15.67:0/1243936038 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:12.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.843+0000 7f9964e1d640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f996011c660 0x7f9960118680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.844+0000 7f995f7fe640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9960111970 0x7f9960118120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.845+0000 7f995f7fe640 1 -- 172.21.15.67:0/1243936038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f996011c660 msgr2=0x7f9960118680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.845+0000 7f995f7fe640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f996011c660 0x7f9960118680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.845+0000 7f995f7fe640 1 -- 172.21.15.67:0/1243936038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f996010a410 msgr2=0x7f996011be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.845+0000 7f995f7fe640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f996010a410 0x7f996011be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.845+0000 7f995f7fe640 1 -- 172.21.15.67:0/1243936038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9960118ea0 con 0x7f9960111970 2024-07-31T14:53:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.845+0000 7f9964e1d640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f996011c660 0x7f9960118680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:53:12.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.847+0000 7f995f7fe640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9960111970 0x7f9960118120 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f996006c110 tx=0x7f995000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:12.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.848+0000 7f995d7fa640 1 -- 172.21.15.67:0/1243936038 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995000cc40 con 0x7f9960111970 2024-07-31T14:53:12.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.848+0000 7f995d7fa640 1 -- 172.21.15.67:0/1243936038 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f995000cde0 con 0x7f9960111970 2024-07-31T14:53:12.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.848+0000 7f995d7fa640 1 -- 172.21.15.67:0/1243936038 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950010430 con 0x7f9960111970 2024-07-31T14:53:12.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.851+0000 7f9966880640 1 -- 172.21.15.67:0/1243936038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99601ac860 con 0x7f9960111970 2024-07-31T14:53:12.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.851+0000 7f9966880640 1 -- 172.21.15.67:0/1243936038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99601acd20 con 0x7f9960111970 2024-07-31T14:53:12.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.855+0000 7f996cdd1740 1 Processor -- start 2024-07-31T14:53:12.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.855+0000 7f996cdd1740 1 -- start start 2024-07-31T14:53:12.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.856+0000 7f996cdd1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6f9070 0x55958aa9e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.856+0000 7f996cdd1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6d01b0 0x55958aa9eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.856+0000 7f996cdd1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d8390 0x55958aa9eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.856+0000 7f996cdd1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55958aa14bb0 con 0x55958a6d01b0 2024-07-31T14:53:12.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.856+0000 7f996cdd1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55958aa040b0 con 0x55958a6f9070 2024-07-31T14:53:12.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.856+0000 7f996cdd1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55958aa080f0 con 0x55958a6d8390 2024-07-31T14:53:12.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.856+0000 7f995ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6d01b0 0x55958aa9eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.856+0000 7f995ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6d01b0 0x55958aa9eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40158/0 (socket says 172.21.15.67:40158) 2024-07-31T14:53:12.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.856+0000 7f995ffff640 1 -- 172.21.15.67:0/2678731785 learned_addr learned my addr 172.21.15.67:0/2678731785 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:12.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.857+0000 7f995ffff640 1 -- 172.21.15.67:0/2678731785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d8390 msgr2=0x55958aa9eff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:53:12.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.857+0000 7f995ffff640 1 --2- 172.21.15.67:0/2678731785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d8390 0x55958aa9eff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.857+0000 7f995ffff640 1 -- 172.21.15.67:0/2678731785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6f9070 msgr2=0x55958aa9e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:12.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.857+0000 7f995ffff640 1 --2- 172.21.15.67:0/2678731785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6f9070 0x55958aa9e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.857+0000 7f995ffff640 1 -- 172.21.15.67:0/2678731785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55958a9aab90 con 0x55958a6d01b0 2024-07-31T14:53:12.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.860+0000 7f995ffff640 1 --2- 172.21.15.67:0/2678731785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6d01b0 0x55958aa9eab0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f995801a7c0 tx=0x7f9958042620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6965c1c58b25e1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:12.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.863+0000 7f99467fc640 1 -- 172.21.15.67:0/2678731785 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958010020 con 0x55958a6d01b0 2024-07-31T14:53:12.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.863+0000 7f99467fc640 1 -- 172.21.15.67:0/2678731785 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9958047070 con 0x55958a6d01b0 2024-07-31T14:53:12.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.863+0000 7f99467fc640 1 -- 172.21.15.67:0/2678731785 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99580043c0 con 0x55958a6d01b0 2024-07-31T14:53:12.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.864+0000 7f996cdd1740 1 -- 172.21.15.67:0/2678731785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6d01b0 msgr2=0x55958aa9eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:12.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.864+0000 7f996cdd1740 1 --2- 172.21.15.67:0/2678731785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6d01b0 0x55958aa9eab0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f995801a7c0 tx=0x7f9958042620 comp rx=0 tx=0).stop 2024-07-31T14:53:12.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.868+0000 7f995d7fa640 1 -- 172.21.15.67:0/1243936038 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f9950003e90 con 0x7f9960111970 2024-07-31T14:53:12.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.870+0000 7f995d7fa640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99480b3f50 0x7f99480b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.870+0000 7f995d7fa640 1 -- 172.21.15.67:0/1243936038 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f99500e4700 con 0x7f9960111970 2024-07-31T14:53:12.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.870+0000 7f9964e1d640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99480b3f50 0x7f99480b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.871+0000 7f996cdd1740 1 -- 172.21.15.67:0/2678731785 shutdown_connections 2024-07-31T14:53:12.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.871+0000 7f996cdd1740 1 --2- 172.21.15.67:0/2678731785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d8390 0x55958aa9eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.871+0000 7f996cdd1740 1 --2- 172.21.15.67:0/2678731785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6d01b0 0x55958aa9eab0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.871+0000 7f996cdd1740 1 --2- 172.21.15.67:0/2678731785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6f9070 0x55958aa9e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.871+0000 7f996cdd1740 1 -- 172.21.15.67:0/2678731785 >> 172.21.15.67:0/2678731785 conn(0x55958a6ed460 msgr2=0x55958aa9c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:12.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.873+0000 7f9964e1d640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99480b3f50 0x7f99480b6460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9954005fd0 tx=0x7f9954005e20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:12.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.878+0000 7f996cdd1740 1 -- 172.21.15.67:0/2678731785 shutdown_connections 2024-07-31T14:53:12.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.878+0000 7f996cdd1740 1 -- 172.21.15.67:0/2678731785 wait complete. 2024-07-31T14:53:12.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.878+0000 7f996cdd1740 1 -- start start 2024-07-31T14:53:12.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.879+0000 7f996cdd1740 1 Processor -- start 2024-07-31T14:53:12.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.879+0000 7f996cdd1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6d01b0 0x55958aaaa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.879+0000 7f996cdd1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d8390 0x55958aaaa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.879+0000 7f996cdd1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6f9070 0x55958aab6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.879+0000 7f996cdd1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55958a867700 con 0x55958a6f9070 2024-07-31T14:53:12.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.879+0000 7f996cdd1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55958a9a69e0 con 0x55958a6d01b0 2024-07-31T14:53:12.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.879+0000 7f996cdd1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55958aaaccf0 con 0x55958a6d8390 2024-07-31T14:53:12.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.879+0000 7f996cdd1740 10 client.0 _ll_register_callbacks cb 0x55958a9c14f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:12.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.880+0000 7f995ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6d01b0 0x55958aaaa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.880+0000 7f9964e1d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6f9070 0x55958aab6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.880+0000 7f9964e1d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6f9070 0x55958aab6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40166/0 (socket says 172.21.15.67:40166) 2024-07-31T14:53:12.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.880+0000 7f9964e1d640 1 -- 172.21.15.67:0/1982016286 learned_addr learned my addr 172.21.15.67:0/1982016286 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:12.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.880+0000 7f995f7fe640 1 --2- 172.21.15.67:0/1982016286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d8390 0x55958aaaa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.882+0000 7f9964e1d640 1 -- 172.21.15.67:0/1982016286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d8390 msgr2=0x55958aaaa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:12.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.882+0000 7f9964e1d640 1 --2- 172.21.15.67:0/1982016286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d8390 0x55958aaaa750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.883+0000 7f9964e1d640 1 -- 172.21.15.67:0/1982016286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6d01b0 msgr2=0x55958aaaa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:12.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.883+0000 7f9964e1d640 1 --2- 172.21.15.67:0/1982016286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6d01b0 0x55958aaaa1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.883+0000 7f9964e1d640 1 -- 172.21.15.67:0/1982016286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55958a764650 con 0x55958a6f9070 2024-07-31T14:53:12.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.883+0000 7f9964e1d640 1 --2- 172.21.15.67:0/1982016286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6f9070 0x55958aab6060 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f99540109b0 tx=0x7f9954004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef141d14e6b5ee71 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:12.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.883+0000 7f993f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:12.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.884+0000 7f993f7fe640 1 -- 172.21.15.67:0/1982016286 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954020070 con 0x55958a6f9070 2024-07-31T14:53:12.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.884+0000 7f993f7fe640 1 -- 172.21.15.67:0/1982016286 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f995400fb40 con 0x55958a6f9070 2024-07-31T14:53:12.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.884+0000 7f996cdd1740 1 -- 172.21.15.67:0/1982016286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9958047070 con 0x55958a6f9070 2024-07-31T14:53:12.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.887+0000 7f993f7fe640 1 -- 172.21.15.67:0/1982016286 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f995400fd40 con 0x55958a6f9070 2024-07-31T14:53:12.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.887+0000 7f996cdd1740 10 client.16245 fetch_fsmap learned FSMap version 30 2024-07-31T14:53:12.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.887+0000 7f996cdd1740 1 -- 172.21.15.67:0/1982016286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55958aab8e30 con 0x55958a6f9070 2024-07-31T14:53:12.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.888+0000 7f993f7fe640 1 -- 172.21.15.67:0/1982016286 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9958047070 con 0x55958a6f9070 2024-07-31T14:53:12.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.889+0000 7f993f7fe640 1 -- 172.21.15.67:0/1982016286 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f995401c580 con 0x55958a6f9070 2024-07-31T14:53:12.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.889+0000 7f996cdd1740 10 client.16245 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:53:12.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.889+0000 7f996cdd1740 10 client.16245 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:53:12.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.889+0000 7f996cdd1740 20 client.16245 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:12.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.889+0000 7f996cdd1740 1 --2- 172.21.15.67:0/1982016286 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55958a6b79f0 0x55958aabc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.889+0000 7f996cdd1740 4 client.16245 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:53:12.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.889+0000 7f996cdd1740 1 -- 172.21.15.67:0/1982016286 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55958aa16b00 con 0x55958a6b79f0 2024-07-31T14:53:12.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.893+0000 7f995ffff640 1 --2- 172.21.15.67:0/1982016286 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55958a6b79f0 0x55958aabc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.898+0000 7f995ffff640 1 --2- 172.21.15.67:0/1982016286 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55958a6b79f0 0x55958aabc280 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dd5545999d47600a server_cookie=f1db6158ac3e374d in_seq=0 out_seq=0 2024-07-31T14:53:12.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.898+0000 7f993f7fe640 10 client.16245 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:53:12.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.901+0000 7f993f7fe640 1 -- 172.21.15.67:0/1982016286 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55958aa16b00 con 0x55958a6b79f0 2024-07-31T14:53:12.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.901+0000 7f993f7fe640 10 client.16245 handle_command_reply: tid=0 2024-07-31T14:53:12.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.901+0000 7f996cdd1740 1 client.16245 shutdown 2024-07-31T14:53:12.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.906+0000 7f996cdd1740 1 -- 172.21.15.67:0/1982016286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6f9070 msgr2=0x55958aab6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:12.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.906+0000 7f996cdd1740 1 --2- 172.21.15.67:0/1982016286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6f9070 0x55958aab6060 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f99540109b0 tx=0x7f9954004730 comp rx=0 tx=0).stop 2024-07-31T14:53:12.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.908+0000 7f996cdd1740 1 -- 172.21.15.67:0/1982016286 shutdown_connections 2024-07-31T14:53:12.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.908+0000 7f996cdd1740 1 --2- 172.21.15.67:0/1982016286 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55958a6b79f0 0x55958aabc280 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.908+0000 7f996cdd1740 1 --2- 172.21.15.67:0/1982016286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6f9070 0x55958aab6060 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.908+0000 7f996cdd1740 1 --2- 172.21.15.67:0/1982016286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d8390 0x55958aaaa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.908+0000 7f996cdd1740 1 --2- 172.21.15.67:0/1982016286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6d01b0 0x55958aaaa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.908+0000 7f996cdd1740 1 -- 172.21.15.67:0/1982016286 >> 172.21.15.67:0/1982016286 conn(0x55958a6ed460 msgr2=0x55958aa9ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:12.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.908+0000 7f993f7fe640 0 client.16245 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:53:12.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.909+0000 7f996cdd1740 1 -- 172.21.15.67:0/1982016286 shutdown_connections 2024-07-31T14:53:12.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.909+0000 7f996cdd1740 1 -- 172.21.15.67:0/1982016286 wait complete. 2024-07-31T14:53:12.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.909+0000 7f996cdd1740 20 client.16245 trim_cache size 0 max 16384 2024-07-31T14:53:12.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.939+0000 7f996cdd1740 1 Processor -- start 2024-07-31T14:53:12.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.939+0000 7f996cdd1740 1 -- start start 2024-07-31T14:53:12.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.939+0000 7f996cdd1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6d01b0 0x55958aa9b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.940+0000 7f996cdd1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d8390 0x55958aab5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.940+0000 7f996cdd1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6f9070 0x55958aaa3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.940+0000 7f996cdd1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f99601b6e10 con 0x55958a6f9070 2024-07-31T14:53:12.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.940+0000 7f996cdd1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f99601b8950 con 0x55958a6d01b0 2024-07-31T14:53:12.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.940+0000 7f995ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6d01b0 0x55958aa9b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.940+0000 7f9964e1d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6f9070 0x55958aaa3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.940+0000 7f9964e1d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6f9070 0x55958aaa3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40174/0 (socket says 172.21.15.67:40174) 2024-07-31T14:53:12.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.940+0000 7f9964e1d640 1 -- 172.21.15.67:0/1815055743 learned_addr learned my addr 172.21.15.67:0/1815055743 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:12.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.941+0000 7f996cdd1740 1 -- 172.21.15.67:0/1815055743 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f99601b74e0 con 0x55958a6d8390 2024-07-31T14:53:12.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.942+0000 7f9964e1d640 1 -- 172.21.15.67:0/1815055743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d8390 msgr2=0x55958aab5e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:53:12.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.942+0000 7f9964e1d640 1 --2- 172.21.15.67:0/1815055743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d8390 0x55958aab5e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.942+0000 7f9964e1d640 1 -- 172.21.15.67:0/1815055743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6d01b0 msgr2=0x55958aa9b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:12.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.942+0000 7f9964e1d640 1 --2- 172.21.15.67:0/1815055743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6d01b0 0x55958aa9b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.942+0000 7f9964e1d640 1 -- 172.21.15.67:0/1815055743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55958aa9cdd0 con 0x55958a6f9070 2024-07-31T14:53:12.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.942+0000 7f995ffff640 1 --2- 172.21.15.67:0/1815055743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6d01b0 0x55958aa9b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:53:12.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.945+0000 7f9964e1d640 1 --2- 172.21.15.67:0/1815055743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6f9070 0x55958aaa3680 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f995401c950 tx=0x7f995400e7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfc9efa6662f2f4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:12.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.950+0000 7f9945ffb640 1 -- 172.21.15.67:0/1815055743 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954020040 con 0x55958a6f9070 2024-07-31T14:53:12.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.950+0000 7f9945ffb640 1 -- 172.21.15.67:0/1815055743 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f995401cd40 con 0x55958a6f9070 2024-07-31T14:53:12.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.950+0000 7f9945ffb640 1 -- 172.21.15.67:0/1815055743 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954005620 con 0x55958a6f9070 2024-07-31T14:53:12.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.951+0000 7f996cdd1740 1 -- 172.21.15.67:0/1815055743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6f9070 msgr2=0x55958aaa3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:12.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.951+0000 7f996cdd1740 1 --2- 172.21.15.67:0/1815055743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6f9070 0x55958aaa3680 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f995401c950 tx=0x7f995400e7a0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.951+0000 7f996cdd1740 1 -- 172.21.15.67:0/1815055743 shutdown_connections 2024-07-31T14:53:12.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.951+0000 7f996cdd1740 1 --2- 172.21.15.67:0/1815055743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6f9070 0x55958aaa3680 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.952+0000 7f996cdd1740 1 --2- 172.21.15.67:0/1815055743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d8390 0x55958aab5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.952+0000 7f996cdd1740 1 --2- 172.21.15.67:0/1815055743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6d01b0 0x55958aa9b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.952+0000 7f996cdd1740 1 -- 172.21.15.67:0/1815055743 >> 172.21.15.67:0/1815055743 conn(0x55958a6ed460 msgr2=0x55958aa9ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:12.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.955+0000 7f996cdd1740 1 -- 172.21.15.67:0/1815055743 shutdown_connections 2024-07-31T14:53:12.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.959+0000 7f996cdd1740 1 -- 172.21.15.67:0/1815055743 wait complete. 2024-07-31T14:53:12.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.960+0000 7f996cdd1740 1 -- start start 2024-07-31T14:53:12.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.960+0000 7f996cdd1740 1 Processor -- start 2024-07-31T14:53:12.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.961+0000 7f996cdd1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d01b0 0x55958aaad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.961+0000 7f996cdd1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6d8390 0x55958aaaddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.961+0000 7f996cdd1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6f9070 0x55958aab1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.961+0000 7f996cdd1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f99601b8280 con 0x55958a6d8390 2024-07-31T14:53:12.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.961+0000 7f996cdd1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f99601b7bb0 con 0x55958a6f9070 2024-07-31T14:53:12.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.961+0000 7f996cdd1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f99601b4df0 con 0x55958a6d01b0 2024-07-31T14:53:12.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.961+0000 7f996cdd1740 10 client.0 _ll_register_callbacks cb 0x7f9954014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:12.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.962+0000 7f995ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d01b0 0x55958aaad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.962+0000 7f995ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d01b0 0x55958aaad7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:47476/0 (socket says 172.21.15.67:47476) 2024-07-31T14:53:12.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.962+0000 7f995ffff640 1 -- 172.21.15.67:0/2056921025 learned_addr learned my addr 172.21.15.67:0/2056921025 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:12.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.962+0000 7f995f7fe640 1 --2- 172.21.15.67:0/2056921025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6d8390 0x55958aaaddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.962+0000 7f9964e1d640 1 --2- 172.21.15.67:0/2056921025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6f9070 0x55958aab1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.964+0000 7f9964e1d640 1 -- 172.21.15.67:0/2056921025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d01b0 msgr2=0x55958aaad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:12.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.964+0000 7f9964e1d640 1 --2- 172.21.15.67:0/2056921025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d01b0 0x55958aaad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.964+0000 7f9964e1d640 1 -- 172.21.15.67:0/2056921025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6d8390 msgr2=0x55958aaaddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:12.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.964+0000 7f9964e1d640 1 --2- 172.21.15.67:0/2056921025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6d8390 0x55958aaaddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.964+0000 7f9964e1d640 1 -- 172.21.15.67:0/2056921025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55958aab2330 con 0x55958a6f9070 2024-07-31T14:53:12.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.970+0000 7f9964e1d640 1 --2- 172.21.15.67:0/2056921025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6f9070 0x55958aab1cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f995401cac0 tx=0x7f995401cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3118850fc0da78 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:12.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.970+0000 7f993f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:53:12.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.972+0000 7f993f7fe640 1 -- 172.21.15.67:0/2056921025 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995403f030 con 0x55958a6f9070 2024-07-31T14:53:12.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.972+0000 7f993f7fe640 1 -- 172.21.15.67:0/2056921025 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f995401c420 con 0x55958a6f9070 2024-07-31T14:53:12.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.972+0000 7f996cdd1740 1 -- 172.21.15.67:0/2056921025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f995401cd40 con 0x55958a6f9070 2024-07-31T14:53:12.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.976+0000 7f993f7fe640 1 -- 172.21.15.67:0/2056921025 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9954048050 con 0x55958a6f9070 2024-07-31T14:53:12.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.979+0000 7f996cdd1740 10 client.25877 fetch_fsmap learned FSMap version 30 2024-07-31T14:53:12.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.979+0000 7f996cdd1740 1 -- 172.21.15.67:0/2056921025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55958ab24380 con 0x55958a6f9070 2024-07-31T14:53:12.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.980+0000 7f993f7fe640 1 -- 172.21.15.67:0/2056921025 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f995401cd40 con 0x55958a6f9070 2024-07-31T14:53:12.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.980+0000 7f993f7fe640 1 -- 172.21.15.67:0/2056921025 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9954004ae0 con 0x55958a6f9070 2024-07-31T14:53:12.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.980+0000 7f996cdd1740 10 client.25877 fetch_fsmap finished waiting for FSMap version 30 2024-07-31T14:53:12.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.980+0000 7f996cdd1740 10 client.25877 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:53:12.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.980+0000 7f996cdd1740 20 client.25877 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:12.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.980+0000 7f996cdd1740 1 --2- 172.21.15.67:0/2056921025 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55958a68cda0 0x55958ab26a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:12.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.980+0000 7f996cdd1740 4 client.25877 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:53:12.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.980+0000 7f996cdd1740 1 -- 172.21.15.67:0/2056921025 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55958aa125a0 con 0x55958a68cda0 2024-07-31T14:53:12.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.983+0000 7f995ffff640 1 --2- 172.21.15.67:0/2056921025 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55958a68cda0 0x55958ab26a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:12.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.984+0000 7f993f7fe640 10 client.25877 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:53:12.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.984+0000 7f995ffff640 1 --2- 172.21.15.67:0/2056921025 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55958a68cda0 0x55958ab26a30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f3f566c398898b63 server_cookie=395247f22c9aa1b2 in_seq=0 out_seq=0 2024-07-31T14:53:12.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.988+0000 7f993f7fe640 1 -- 172.21.15.67:0/2056921025 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55958aa125a0 con 0x55958a68cda0 2024-07-31T14:53:12.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.989+0000 7f993f7fe640 10 client.25877 handle_command_reply: tid=0 2024-07-31T14:53:12.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.989+0000 7f996cdd1740 1 client.25877 shutdown 2024-07-31T14:53:12.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.989+0000 7f996cdd1740 1 -- 172.21.15.67:0/2056921025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6f9070 msgr2=0x55958aab1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.989+0000 7f996cdd1740 1 --2- 172.21.15.67:0/2056921025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6f9070 0x55958aab1cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f995401cac0 tx=0x7f995401cb40 comp rx=0 tx=0).stop 2024-07-31T14:53:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.989+0000 7f996cdd1740 1 -- 172.21.15.67:0/2056921025 shutdown_connections 2024-07-31T14:53:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.989+0000 7f996cdd1740 1 --2- 172.21.15.67:0/2056921025 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55958a68cda0 0x55958ab26a30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.989+0000 7f996cdd1740 1 --2- 172.21.15.67:0/2056921025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55958a6f9070 0x55958aab1cc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.989+0000 7f996cdd1740 1 --2- 172.21.15.67:0/2056921025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55958a6d8390 0x55958aaaddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.989+0000 7f996cdd1740 1 --2- 172.21.15.67:0/2056921025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55958a6d01b0 0x55958aaad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.989+0000 7f993f7fe640 0 client.25877 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:53:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.989+0000 7f996cdd1740 1 -- 172.21.15.67:0/2056921025 >> 172.21.15.67:0/2056921025 conn(0x55958a6ed460 msgr2=0x55958aa9bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:12.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.994+0000 7f996cdd1740 1 -- 172.21.15.67:0/2056921025 shutdown_connections 2024-07-31T14:53:12.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.994+0000 7f996cdd1740 1 -- 172.21.15.67:0/2056921025 wait complete. 2024-07-31T14:53:12.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.994+0000 7f996cdd1740 20 client.25877 trim_cache size 0 max 16384 2024-07-31T14:53:12.995 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:53:12.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.995+0000 7f9966880640 1 -- 172.21.15.67:0/1243936038 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99480b3f50 msgr2=0x7f99480b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:12.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.995+0000 7f9966880640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99480b3f50 0x7f99480b6460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9954005fd0 tx=0x7f9954005e20 comp rx=0 tx=0).stop 2024-07-31T14:53:12.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.995+0000 7f9966880640 1 -- 172.21.15.67:0/1243936038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9960111970 msgr2=0x7f9960118120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:12.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.995+0000 7f9966880640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9960111970 0x7f9960118120 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f996006c110 tx=0x7f995000ee50 comp rx=0 tx=0).stop 2024-07-31T14:53:12.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.995+0000 7f9966880640 1 -- 172.21.15.67:0/1243936038 shutdown_connections 2024-07-31T14:53:12.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.995+0000 7f9966880640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99480b3f50 0x7f99480b6460 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.995+0000 7f9966880640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f996011c660 0x7f9960118680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.995+0000 7f9966880640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9960111970 0x7f9960118120 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.995+0000 7f9966880640 1 --2- 172.21.15.67:0/1243936038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f996010a410 0x7f996011be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.995+0000 7f9966880640 1 -- 172.21.15.67:0/1243936038 >> 172.21.15.67:0/1243936038 conn(0x7f99600708f0 msgr2=0x7f9960073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.995+0000 7f9966880640 1 -- 172.21.15.67:0/1243936038 shutdown_connections 2024-07-31T14:53:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:12.995+0000 7f9966880640 1 -- 172.21.15.67:0/1243936038 wait complete. 2024-07-31T14:53:13.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:12 smithi067 ceph-mon[32750]: pgmap v463: 129 pgs: 129 active+clean; 87 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.1 MiB/s wr, 33 op/s 2024-07-31T14:53:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.082+0000 7f1961ffb640 1 -- 172.21.15.67:0/2782328687 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630d3fc2da0 con 0x5630d3c411c0 2024-07-31T14:53:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.082+0000 7f1961ffb640 10 client.16233 handle_command_reply: tid=0 2024-07-31T14:53:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.082+0000 7f1987b78740 1 client.16233 shutdown 2024-07-31T14:53:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.082+0000 7f1987b78740 1 -- 172.21.15.67:0/2782328687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3c8c4d0 msgr2=0x5630d40606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.082+0000 7f1987b78740 1 --2- 172.21.15.67:0/2782328687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3c8c4d0 0x5630d40606b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f196c023a20 tx=0x7f196c024fc0 comp rx=0 tx=0).stop 2024-07-31T14:53:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.082+0000 7f1987b78740 1 -- 172.21.15.67:0/2782328687 shutdown_connections 2024-07-31T14:53:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.082+0000 7f1987b78740 1 --2- 172.21.15.67:0/2782328687 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5630d3c411c0 0x5630d40d7070 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.082+0000 7f1987b78740 1 --2- 172.21.15.67:0/2782328687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5630d3cad1b0 0x5630d4060bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.082+0000 7f1987b78740 1 --2- 172.21.15.67:0/2782328687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630d3c8c4d0 0x5630d40606b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.082+0000 7f1987b78740 1 --2- 172.21.15.67:0/2782328687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630d3c842f0 0x5630d4063d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.082+0000 7f1987b78740 1 -- 172.21.15.67:0/2782328687 >> 172.21.15.67:0/2782328687 conn(0x5630d3ca1300 msgr2=0x5630d4059e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.083+0000 7f1961ffb640 0 client.16233 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.083+0000 7f1987b78740 1 -- 172.21.15.67:0/2782328687 shutdown_connections 2024-07-31T14:53:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.083+0000 7f1987b78740 1 -- 172.21.15.67:0/2782328687 wait complete. 2024-07-31T14:53:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.083+0000 7f1987b78740 20 client.16233 trim_cache size 0 max 16384 2024-07-31T14:53:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:53:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:53:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:53:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:53:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.084+0000 7f1980bb0640 1 -- 172.21.15.67:0/1483659971 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19680b3d40 msgr2=0x7f19680b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.084+0000 7f1980bb0640 1 --2- 172.21.15.67:0/1483659971 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19680b3d40 0x7f19680b6250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1974102b90 tx=0x7f19740b3040 comp rx=0 tx=0).stop 2024-07-31T14:53:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.084+0000 7f1980bb0640 1 -- 172.21.15.67:0/1483659971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f197c108690 msgr2=0x7f197c19b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.084+0000 7f1980bb0640 1 --2- 172.21.15.67:0/1483659971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f197c108690 0x7f197c19b670 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1974005cc0 tx=0x7f1974031700 comp rx=0 tx=0).stop 2024-07-31T14:53:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.084+0000 7f1980bb0640 1 -- 172.21.15.67:0/1483659971 shutdown_connections 2024-07-31T14:53:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.084+0000 7f1980bb0640 1 --2- 172.21.15.67:0/1483659971 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19680b3d40 0x7f19680b6250 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.084+0000 7f1980bb0640 1 --2- 172.21.15.67:0/1483659971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f197c108690 0x7f197c19b670 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.084+0000 7f1980bb0640 1 --2- 172.21.15.67:0/1483659971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c107d40 0x7f197c19b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.084+0000 7f1980bb0640 1 --2- 172.21.15.67:0/1483659971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f197c102410 0x7f197c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.084+0000 7f1980bb0640 1 -- 172.21.15.67:0/1483659971 >> 172.21.15.67:0/1483659971 conn(0x7f197c07a140 msgr2=0x7f197c103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.084+0000 7f1980bb0640 1 -- 172.21.15.67:0/1483659971 shutdown_connections 2024-07-31T14:53:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:13.084+0000 7f1980bb0640 1 -- 172.21.15.67:0/1483659971 wait complete. 2024-07-31T14:53:13.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:12 smithi073 ceph-mon[36714]: pgmap v463: 129 pgs: 129 active+clean; 87 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.1 MiB/s wr, 33 op/s 2024-07-31T14:53:13.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:12 smithi077 ceph-mon[37327]: pgmap v463: 129 pgs: 129 active+clean; 87 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.1 MiB/s wr, 33 op/s 2024-07-31T14:53:13.562 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: *** Caught signal (Segmentation fault) ** 2024-07-31T14:53:13.562 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: in thread 7f80b41b8640 thread_name:md_log_replay 2024-07-31T14:53:13.562 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: ceph version 19.0.0-5223-g8c796fca (8c796fca5ff5960f7542e7227bad4a784ee2c804) squid (dev) 2024-07-31T14:53:13.563 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f80c159f6f0] 2024-07-31T14:53:13.563 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 2: /usr/bin/ceph-mds(+0x329af2) [0x55e151b03af2] 2024-07-31T14:53:13.563 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 3: /usr/bin/ceph-mds(+0x319cf8) [0x55e151af3cf8] 2024-07-31T14:53:13.563 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 4: (MDCache::finish_uncommitted_fragment(dirfrag_t, int)+0x237) [0x55e151ad14f7] 2024-07-31T14:53:13.563 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 5: (EFragment::replay(MDSRank*)+0x2e6) [0x55e151978628] 2024-07-31T14:53:13.563 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 6: (MDLog::_replay_thread()+0x1c46) [0x55e151c8abb4] 2024-07-31T14:53:13.563 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 7: /usr/bin/ceph-mds(+0x4a1c19) [0x55e151c7bc19] 2024-07-31T14:53:13.563 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 8: (Thread::entry_wrapper()+0x43) [0x7f80c1cf69f5] 2024-07-31T14:53:13.563 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 9: (Thread::_entry_func(void*)+0xd) [0x7f80c1cf6a11] 2024-07-31T14:53:13.563 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 10: /lib64/libc.so.6(+0x89c02) [0x7f80c15eac02] 2024-07-31T14:53:13.564 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 11: /lib64/libc.so.6(+0x10ec40) [0x7f80c166fc40] 2024-07-31T14:53:13.564 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 2024-07-31T14:53:13.188+0000 7f80b41b8640 -1 *** Caught signal (Segmentation fault) ** 2024-07-31T14:53:13.564 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: in thread 7f80b41b8640 thread_name:md_log_replay 2024-07-31T14:53:13.564 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 2024-07-31T14:53:13.564 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: ceph version 19.0.0-5223-g8c796fca (8c796fca5ff5960f7542e7227bad4a784ee2c804) squid (dev) 2024-07-31T14:53:13.564 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f80c159f6f0] 2024-07-31T14:53:13.564 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 2: /usr/bin/ceph-mds(+0x329af2) [0x55e151b03af2] 2024-07-31T14:53:13.564 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 3: /usr/bin/ceph-mds(+0x319cf8) [0x55e151af3cf8] 2024-07-31T14:53:13.564 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 4: (MDCache::finish_uncommitted_fragment(dirfrag_t, int)+0x237) [0x55e151ad14f7] 2024-07-31T14:53:13.564 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 5: (EFragment::replay(MDSRank*)+0x2e6) [0x55e151978628] 2024-07-31T14:53:13.565 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 6: (MDLog::_replay_thread()+0x1c46) [0x55e151c8abb4] 2024-07-31T14:53:13.565 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 7: /usr/bin/ceph-mds(+0x4a1c19) [0x55e151c7bc19] 2024-07-31T14:53:13.565 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 8: (Thread::entry_wrapper()+0x43) [0x7f80c1cf69f5] 2024-07-31T14:53:13.565 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 9: (Thread::_entry_func(void*)+0xd) [0x7f80c1cf6a11] 2024-07-31T14:53:13.565 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 10: /lib64/libc.so.6(+0x89c02) [0x7f80c15eac02] 2024-07-31T14:53:13.565 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 11: /lib64/libc.so.6(+0x10ec40) [0x7f80c166fc40] 2024-07-31T14:53:13.566 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-31T14:53:13.566 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 2024-07-31T14:53:13.566 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 0> 2024-07-31T14:53:13.188+0000 7f80b41b8640 -1 *** Caught signal (Segmentation fault) ** 2024-07-31T14:53:13.566 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: in thread 7f80b41b8640 thread_name:md_log_replay 2024-07-31T14:53:13.566 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 2024-07-31T14:53:13.566 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: ceph version 19.0.0-5223-g8c796fca (8c796fca5ff5960f7542e7227bad4a784ee2c804) squid (dev) 2024-07-31T14:53:13.566 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f80c159f6f0] 2024-07-31T14:53:13.567 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 2: /usr/bin/ceph-mds(+0x329af2) [0x55e151b03af2] 2024-07-31T14:53:13.567 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 3: /usr/bin/ceph-mds(+0x319cf8) [0x55e151af3cf8] 2024-07-31T14:53:13.567 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 4: (MDCache::finish_uncommitted_fragment(dirfrag_t, int)+0x237) [0x55e151ad14f7] 2024-07-31T14:53:13.567 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 5: (EFragment::replay(MDSRank*)+0x2e6) [0x55e151978628] 2024-07-31T14:53:13.567 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 6: (MDLog::_replay_thread()+0x1c46) [0x55e151c8abb4] 2024-07-31T14:53:13.567 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 7: /usr/bin/ceph-mds(+0x4a1c19) [0x55e151c7bc19] 2024-07-31T14:53:13.567 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 8: (Thread::entry_wrapper()+0x43) [0x7f80c1cf69f5] 2024-07-31T14:53:13.567 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 9: (Thread::_entry_func(void*)+0xd) [0x7f80c1cf6a11] 2024-07-31T14:53:13.567 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 10: /lib64/libc.so.6(+0x89c02) [0x7f80c15eac02] 2024-07-31T14:53:13.567 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 11: /lib64/libc.so.6(+0x10ec40) [0x7f80c166fc40] 2024-07-31T14:53:13.567 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-31T14:53:13.568 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 2024-07-31T14:53:13.568 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 0> 2024-07-31T14:53:13.188+0000 7f80b41b8640 -1 *** Caught signal (Segmentation fault) ** 2024-07-31T14:53:13.568 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: in thread 7f80b41b8640 thread_name:md_log_replay 2024-07-31T14:53:13.568 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 2024-07-31T14:53:13.568 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: ceph version 19.0.0-5223-g8c796fca (8c796fca5ff5960f7542e7227bad4a784ee2c804) squid (dev) 2024-07-31T14:53:13.568 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f80c159f6f0] 2024-07-31T14:53:13.568 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 2: /usr/bin/ceph-mds(+0x329af2) [0x55e151b03af2] 2024-07-31T14:53:13.568 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 3: /usr/bin/ceph-mds(+0x319cf8) [0x55e151af3cf8] 2024-07-31T14:53:13.568 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 4: (MDCache::finish_uncommitted_fragment(dirfrag_t, int)+0x237) [0x55e151ad14f7] 2024-07-31T14:53:13.568 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 5: (EFragment::replay(MDSRank*)+0x2e6) [0x55e151978628] 2024-07-31T14:53:13.569 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 6: (MDLog::_replay_thread()+0x1c46) [0x55e151c8abb4] 2024-07-31T14:53:13.569 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 7: /usr/bin/ceph-mds(+0x4a1c19) [0x55e151c7bc19] 2024-07-31T14:53:13.569 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 8: (Thread::entry_wrapper()+0x43) [0x7f80c1cf69f5] 2024-07-31T14:53:13.569 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 9: (Thread::_entry_func(void*)+0xd) [0x7f80c1cf6a11] 2024-07-31T14:53:13.569 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 10: /lib64/libc.so.6(+0x89c02) [0x7f80c15eac02] 2024-07-31T14:53:13.569 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 11: /lib64/libc.so.6(+0x10ec40) [0x7f80c166fc40] 2024-07-31T14:53:13.569 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-31T14:53:13.569 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[67282]: 2024-07-31T14:53:13.569 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 podman[70228]: 2024-07-31 14:53:13.247333451 +0000 UTC m=+0.023827766 container died c20528a1cd963bd5ef16627e4053b8850db3b3d57722ec79b687a663de722539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e, ceph=True, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-07-31T14:53:13.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T14:53:14.073 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:53:14.392 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 podman[70228]: 2024-07-31 14:53:13.962891469 +0000 UTC m=+0.739385769 container remove c20528a1cd963bd5ef16627e4053b8850db3b3d57722ec79b687a663de722539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=) 2024-07-31T14:53:14.393 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:13 smithi073 systemd[1]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.e.service: Main process exited, code=exited, status=139/n/a 2024-07-31T14:53:14.393 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:14 smithi073 systemd[1]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-07-31T14:53:14.393 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:14 smithi073 systemd[1]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.e.service: Consumed 1.389s CPU time. 2024-07-31T14:53:15.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:14 smithi073 ceph-mon[36714]: pgmap v464: 129 pgs: 129 active+clean; 135 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 88 op/s 2024-07-31T14:53:15.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:14 smithi067 ceph-mon[32750]: pgmap v464: 129 pgs: 129 active+clean; 135 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 88 op/s 2024-07-31T14:53:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:14 smithi077 ceph-mon[37327]: pgmap v464: 129 pgs: 129 active+clean; 135 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 88 op/s 2024-07-31T14:53:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.888+0000 7ff8f9456640 1 -- 172.21.15.67:0/511295161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f4108760 msgr2=0x7ff8f410ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.888+0000 7ff8f9456640 1 --2- 172.21.15.67:0/511295161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f4108760 0x7ff8f410ef90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009ca0 tx=0x7ff8ec02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.888+0000 7ff8f9456640 1 -- 172.21.15.67:0/511295161 shutdown_connections 2024-07-31T14:53:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.888+0000 7ff8f9456640 1 --2- 172.21.15.67:0/511295161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f4108760 0x7ff8f410ef90 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.888+0000 7ff8f9456640 1 --2- 172.21.15.67:0/511295161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f4107e10 0x7ff8f4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.888+0000 7ff8f9456640 1 --2- 172.21.15.67:0/511295161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8f41024d0 0x7ff8f41028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.888+0000 7ff8f9456640 1 -- 172.21.15.67:0/511295161 >> 172.21.15.67:0/511295161 conn(0x7ff8f4079190 msgr2=0x7ff8f40795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.888+0000 7ff8f9456640 1 -- 172.21.15.67:0/511295161 shutdown_connections 2024-07-31T14:53:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.888+0000 7ff8f9456640 1 -- 172.21.15.67:0/511295161 wait complete. 2024-07-31T14:53:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.889+0000 7ff8f9456640 1 Processor -- start 2024-07-31T14:53:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.889+0000 7ff8f9456640 1 -- start start 2024-07-31T14:53:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.889+0000 7ff8f9456640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f41024d0 0x7ff8f4199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.889+0000 7ff8f9456640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8f4107e10 0x7ff8f419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.889+0000 7ff8f9456640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f4108760 0x7ff8f419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.889+0000 7ff8f9456640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41134f0 con 0x7ff8f41024d0 2024-07-31T14:53:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.889+0000 7ff8f9456640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4113370 con 0x7ff8f4108760 2024-07-31T14:53:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.889+0000 7ff8f9456640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4113670 con 0x7ff8f4107e10 2024-07-31T14:53:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.889+0000 7ff8f2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f41024d0 0x7ff8f4199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8ebfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8f4107e10 0x7ff8f419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8f2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f41024d0 0x7ff8f4199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40200/0 (socket says 172.21.15.67:40200) 2024-07-31T14:53:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8f2ffd640 1 -- 172.21.15.67:0/2644791183 learned_addr learned my addr 172.21.15.67:0/2644791183 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8f37fe640 1 --2- 172.21.15.67:0/2644791183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f4108760 0x7ff8f419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8f2ffd640 1 -- 172.21.15.67:0/2644791183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8f4107e10 msgr2=0x7ff8f419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8f2ffd640 1 --2- 172.21.15.67:0/2644791183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8f4107e10 0x7ff8f419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8f2ffd640 1 -- 172.21.15.67:0/2644791183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f4108760 msgr2=0x7ff8f419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8f2ffd640 1 --2- 172.21.15.67:0/2644791183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f4108760 0x7ff8f419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8f2ffd640 1 -- 172.21.15.67:0/2644791183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f419fe70 con 0x7ff8f41024d0 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8f2ffd640 1 --2- 172.21.15.67:0/2644791183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f41024d0 0x7ff8f4199f00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400e9f0 tx=0x7ff8e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8f0ff9640 1 -- 172.21.15.67:0/2644791183 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e400cd80 con 0x7ff8f41024d0 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8f0ff9640 1 -- 172.21.15.67:0/2644791183 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8e40054d0 con 0x7ff8f41024d0 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8f9456640 1 -- 172.21.15.67:0/2644791183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f41a0160 con 0x7ff8f41024d0 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8f0ff9640 1 -- 172.21.15.67:0/2644791183 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4010690 con 0x7ff8f41024d0 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.890+0000 7ff8f9456640 1 -- 172.21.15.67:0/2644791183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f41a68c0 con 0x7ff8f41024d0 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.893+0000 7ff8ff9a7740 1 Processor -- start 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.893+0000 7ff8ff9a7740 1 -- start start 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.893+0000 7ff8f0ff9640 1 -- 172.21.15.67:0/2644791183 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff8e4004e90 con 0x7ff8f41024d0 2024-07-31T14:53:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.893+0000 7ff8ff9a7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660bc070 0x561f66461570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.893+0000 7ff8ff9a7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f660931b0 0x561f66461ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8ff9a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 0x561f66461ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8ff9a7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561f663d7bb0 con 0x561f660931b0 2024-07-31T14:53:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8ff9a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561f663c70b0 con 0x561f6609b390 2024-07-31T14:53:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8ff9a7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561f663cb0f0 con 0x561f660bc070 2024-07-31T14:53:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8f2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f660931b0 0x561f66461ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8f37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660bc070 0x561f66461570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8f2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f660931b0 0x561f66461ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40212/0 (socket says 172.21.15.67:40212) 2024-07-31T14:53:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8ebfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 0x561f66461ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8f2ffd640 1 -- 172.21.15.67:0/2701255251 learned_addr learned my addr 172.21.15.67:0/2701255251 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8ebfff640 1 -- 172.21.15.67:0/2701255251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660bc070 msgr2=0x561f66461570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8ebfff640 1 --2- 172.21.15.67:0/2701255251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660bc070 0x561f66461570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8ebfff640 1 -- 172.21.15.67:0/2701255251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f660931b0 msgr2=0x561f66461ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8ebfff640 1 --2- 172.21.15.67:0/2701255251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f660931b0 0x561f66461ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8ebfff640 1 -- 172.21.15.67:0/2701255251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f6636db90 con 0x561f6609b390 2024-07-31T14:53:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8f2ffd640 1 --2- 172.21.15.67:0/2701255251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f660931b0 0x561f66461ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8f0ff9640 1 --2- 172.21.15.67:0/2644791183 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e00b3d40 0x7ff8e00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8ebfff640 1 --2- 172.21.15.67:0/2701255251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 0x561f66461ff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff8f406b0b0 tx=0x7ff8dc011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d94f4bd464bd1852 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.894+0000 7ff8f37fe640 1 --2- 172.21.15.67:0/2644791183 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e00b3d40 0x7ff8e00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.895+0000 7ff8f0ff9640 1 -- 172.21.15.67:0/2644791183 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff8e40e46f0 con 0x7ff8f41024d0 2024-07-31T14:53:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.895+0000 7ff8e97fa640 1 -- 172.21.15.67:0/2701255251 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc004310 con 0x561f6609b390 2024-07-31T14:53:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.895+0000 7ff8e97fa640 1 -- 172.21.15.67:0/2701255251 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8dc0044b0 con 0x561f6609b390 2024-07-31T14:53:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.895+0000 7ff8e97fa640 1 -- 172.21.15.67:0/2701255251 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc019970 con 0x561f6609b390 2024-07-31T14:53:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.896+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/2701255251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 msgr2=0x561f66461ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.896+0000 7ff8f37fe640 1 --2- 172.21.15.67:0/2644791183 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e00b3d40 0x7ff8e00b6250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec02f500 tx=0x7ff8ec02f310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.896+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/2701255251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 0x561f66461ff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff8f406b0b0 tx=0x7ff8dc011440 comp rx=0 tx=0).stop 2024-07-31T14:53:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.896+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/2701255251 shutdown_connections 2024-07-31T14:53:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.896+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/2701255251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 0x561f66461ff0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.896+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/2701255251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f660931b0 0x561f66461ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.896+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/2701255251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660bc070 0x561f66461570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.896+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/2701255251 >> 172.21.15.67:0/2701255251 conn(0x561f660b0460 msgr2=0x561f6645f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.896+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/2701255251 shutdown_connections 2024-07-31T14:53:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.896+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/2701255251 wait complete. 2024-07-31T14:53:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.896+0000 7ff8ff9a7740 1 -- start start 2024-07-31T14:53:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.896+0000 7ff8ff9a7740 1 Processor -- start 2024-07-31T14:53:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ff9a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f660931b0 0x561f6646d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ff9a7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f6609b390 0x561f6646d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ff9a7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660bc070 0x561f66479070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ff9a7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561f6622a700 con 0x561f6609b390 2024-07-31T14:53:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ff9a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561f663699e0 con 0x561f660931b0 2024-07-31T14:53:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ff9a7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561f6646fd50 con 0x561f660bc070 2024-07-31T14:53:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f6609b390 0x561f6646d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f6609b390 0x561f6646d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40222/0 (socket says 172.21.15.67:40222) 2024-07-31T14:53:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ebfff640 1 -- 172.21.15.67:0/4286515635 learned_addr learned my addr 172.21.15.67:0/4286515635 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8f2ffd640 1 --2- 172.21.15.67:0/4286515635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f660931b0 0x561f6646d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ff9a7740 10 client.0 _ll_register_callbacks cb 0x561f663844f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8f37fe640 1 --2- 172.21.15.67:0/4286515635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660bc070 0x561f66479070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ebfff640 1 -- 172.21.15.67:0/4286515635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660bc070 msgr2=0x561f66479070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ebfff640 1 --2- 172.21.15.67:0/4286515635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660bc070 0x561f66479070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ebfff640 1 -- 172.21.15.67:0/4286515635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f660931b0 msgr2=0x561f6646d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ebfff640 1 --2- 172.21.15.67:0/4286515635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f660931b0 0x561f6646d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.897+0000 7ff8ebfff640 1 -- 172.21.15.67:0/4286515635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f66127650 con 0x561f6609b390 2024-07-31T14:53:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.898+0000 7ff8ebfff640 1 --2- 172.21.15.67:0/4286515635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f6609b390 0x561f6646d6f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009460 tx=0x7ff8dc0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3874a9b604c4b5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.898+0000 7ff8da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.898+0000 7ff8da7fc640 1 -- 172.21.15.67:0/4286515635 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc028020 con 0x561f6609b390 2024-07-31T14:53:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.898+0000 7ff8da7fc640 1 -- 172.21.15.67:0/4286515635 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff8dc019710 con 0x561f6609b390 2024-07-31T14:53:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.898+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/4286515635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8dc0044b0 con 0x561f6609b390 2024-07-31T14:53:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.898+0000 7ff8da7fc640 1 -- 172.21.15.67:0/4286515635 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff8dc033050 con 0x561f6609b390 2024-07-31T14:53:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.898+0000 7ff8ff9a7740 10 client.16263 fetch_fsmap learned FSMap version 31 2024-07-31T14:53:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.898+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/4286515635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561f6647bd20 con 0x561f6609b390 2024-07-31T14:53:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.898+0000 7ff8da7fc640 1 -- 172.21.15.67:0/4286515635 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8dc0044b0 con 0x561f6609b390 2024-07-31T14:53:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.898+0000 7ff8da7fc640 1 -- 172.21.15.67:0/4286515635 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7ff8dc018070 con 0x561f6609b390 2024-07-31T14:53:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.899+0000 7ff8ff9a7740 10 client.16263 fetch_fsmap finished waiting for FSMap version 31 2024-07-31T14:53:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.899+0000 7ff8ff9a7740 10 client.16263 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:53:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.899+0000 7ff8ff9a7740 20 client.16263 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.899+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/4286515635 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x561f6607a9f0 0x561f6647f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.899+0000 7ff8ff9a7740 4 client.16263 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:53:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.899+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/4286515635 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f663d9b00 con 0x561f6607a9f0 2024-07-31T14:53:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.899+0000 7ff8f2ffd640 1 --2- 172.21.15.67:0/4286515635 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x561f6607a9f0 0x561f6647f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.899+0000 7ff8f2ffd640 1 --2- 172.21.15.67:0/4286515635 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x561f6607a9f0 0x561f6647f080 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2595e6d41273df22 server_cookie=8228906e61a45ad9 in_seq=0 out_seq=0 2024-07-31T14:53:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.899+0000 7ff8da7fc640 10 client.16263 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:53:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8da7fc640 1 -- 172.21.15.67:0/4286515635 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f663d9b00 con 0x561f6607a9f0 2024-07-31T14:53:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8da7fc640 10 client.16263 handle_command_reply: tid=0 2024-07-31T14:53:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8ff9a7740 1 client.16263 shutdown 2024-07-31T14:53:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/4286515635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f6609b390 msgr2=0x561f6646d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/4286515635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f6609b390 0x561f6646d6f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009460 tx=0x7ff8dc0052f0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/4286515635 shutdown_connections 2024-07-31T14:53:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/4286515635 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x561f6607a9f0 0x561f6647f080 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/4286515635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660bc070 0x561f66479070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/4286515635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f6609b390 0x561f6646d6f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8da7fc640 0 client.16263 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:53:15.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/4286515635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f660931b0 0x561f6646d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/4286515635 >> 172.21.15.67:0/4286515635 conn(0x561f660b0460 msgr2=0x561f66464700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:15.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/4286515635 shutdown_connections 2024-07-31T14:53:15.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/4286515635 wait complete. 2024-07-31T14:53:15.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.900+0000 7ff8ff9a7740 20 client.16263 trim_cache size 0 max 16384 2024-07-31T14:53:15.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.914+0000 7ff8ff9a7740 1 Processor -- start 2024-07-31T14:53:15.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.914+0000 7ff8ff9a7740 1 -- start start 2024-07-31T14:53:15.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ff9a7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f6607a9f0 0x561f66464d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ff9a7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f660931b0 0x561f66466600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ff9a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 0x561f66466b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ff9a7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b06f0 con 0x561f660931b0 2024-07-31T14:53:15.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ff9a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b22c0 con 0x561f6609b390 2024-07-31T14:53:15.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ff9a7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b0dc0 con 0x561f6607a9f0 2024-07-31T14:53:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f660931b0 0x561f66466600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f660931b0 0x561f66466600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40224/0 (socket says 172.21.15.67:40224) 2024-07-31T14:53:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ebfff640 1 -- 172.21.15.67:0/357693269 learned_addr learned my addr 172.21.15.67:0/357693269 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8f2ffd640 1 --2- 172.21.15.67:0/357693269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f6607a9f0 0x561f66464d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8f37fe640 1 --2- 172.21.15.67:0/357693269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 0x561f66466b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ebfff640 1 -- 172.21.15.67:0/357693269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f6607a9f0 msgr2=0x561f66464d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ebfff640 1 --2- 172.21.15.67:0/357693269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f6607a9f0 0x561f66464d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ebfff640 1 -- 172.21.15.67:0/357693269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 msgr2=0x561f66466b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ebfff640 1 --2- 172.21.15.67:0/357693269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 0x561f66466b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ebfff640 1 -- 172.21.15.67:0/357693269 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f6645a8c0 con 0x561f660931b0 2024-07-31T14:53:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8f37fe640 1 --2- 172.21.15.67:0/357693269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 0x561f66466b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:53:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.915+0000 7ff8ebfff640 1 --2- 172.21.15.67:0/357693269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f660931b0 0x561f66466600 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0097e0 tx=0x7ff8dc005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e54f78fff400974 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.916+0000 7ff8e8ff9640 1 -- 172.21.15.67:0/357693269 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc028030 con 0x561f660931b0 2024-07-31T14:53:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.916+0000 7ff8e8ff9640 1 -- 172.21.15.67:0/357693269 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8dc003e10 con 0x561f660931b0 2024-07-31T14:53:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.916+0000 7ff8e8ff9640 1 -- 172.21.15.67:0/357693269 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc01f040 con 0x561f660931b0 2024-07-31T14:53:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.916+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/357693269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f660931b0 msgr2=0x561f66466600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.916+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/357693269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f660931b0 0x561f66466600 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0097e0 tx=0x7ff8dc005b90 comp rx=0 tx=0).stop 2024-07-31T14:53:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/357693269 shutdown_connections 2024-07-31T14:53:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/357693269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 0x561f66466b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/357693269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f660931b0 0x561f66466600 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/357693269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f6607a9f0 0x561f66464d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/357693269 >> 172.21.15.67:0/357693269 conn(0x561f660b0460 msgr2=0x561f66464670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/357693269 shutdown_connections 2024-07-31T14:53:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/357693269 wait complete. 2024-07-31T14:53:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 -- start start 2024-07-31T14:53:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 Processor -- start 2024-07-31T14:53:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f6607a9f0 0x561f664e47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660931b0 0x561f664e4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 0x561f664e5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b1b60 con 0x561f6607a9f0 2024-07-31T14:53:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b1490 con 0x561f6609b390 2024-07-31T14:53:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.917+0000 7ff8ff9a7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41ae6d0 con 0x561f660931b0 2024-07-31T14:53:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8ff9a7740 10 client.0 _ll_register_callbacks cb 0x7ff8e40ea420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8f2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f6607a9f0 0x561f664e47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8f37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 0x561f664e5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8f2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f6607a9f0 0x561f664e47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40232/0 (socket says 172.21.15.67:40232) 2024-07-31T14:53:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8f2ffd640 1 -- 172.21.15.67:0/458888208 learned_addr learned my addr 172.21.15.67:0/458888208 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8ebfff640 1 --2- 172.21.15.67:0/458888208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660931b0 0x561f664e4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8f2ffd640 1 -- 172.21.15.67:0/458888208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660931b0 msgr2=0x561f664e4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8f2ffd640 1 --2- 172.21.15.67:0/458888208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660931b0 0x561f664e4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8f2ffd640 1 -- 172.21.15.67:0/458888208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 msgr2=0x561f664e5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8f2ffd640 1 --2- 172.21.15.67:0/458888208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 0x561f664e5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8f2ffd640 1 -- 172.21.15.67:0/458888208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f66475140 con 0x561f6607a9f0 2024-07-31T14:53:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8f2ffd640 1 --2- 172.21.15.67:0/458888208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f6607a9f0 0x561f664e47f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4004ad0 tx=0x7ff8e40aa9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc78b6064047fcb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8da7fc640 1 -- 172.21.15.67:0/458888208 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e40aab00 con 0x561f6607a9f0 2024-07-31T14:53:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/458888208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8dc003e10 con 0x561f6607a9f0 2024-07-31T14:53:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.918+0000 7ff8da7fc640 1 -- 172.21.15.67:0/458888208 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7ff8e40a7280 con 0x561f6607a9f0 2024-07-31T14:53:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.919+0000 7ff8da7fc640 1 -- 172.21.15.67:0/458888208 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff8e40c0420 con 0x561f6607a9f0 2024-07-31T14:53:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.919+0000 7ff8ff9a7740 10 client.16269 fetch_fsmap learned FSMap version 31 2024-07-31T14:53:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.919+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/458888208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561f6645e340 con 0x561f6607a9f0 2024-07-31T14:53:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.919+0000 7ff8da7fc640 1 -- 172.21.15.67:0/458888208 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8dc003e10 con 0x561f6607a9f0 2024-07-31T14:53:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.919+0000 7ff8da7fc640 1 -- 172.21.15.67:0/458888208 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7ff8e40e4040 con 0x561f6607a9f0 2024-07-31T14:53:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.919+0000 7ff8ff9a7740 10 client.16269 fetch_fsmap finished waiting for FSMap version 31 2024-07-31T14:53:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.919+0000 7ff8ff9a7740 10 client.16269 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:53:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.919+0000 7ff8ff9a7740 20 client.16269 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.919+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/458888208 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x561f660bc070 0x561f664edc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.919+0000 7ff8ff9a7740 4 client.16269 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:53:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.919+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/458888208 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561f663d55a0 con 0x561f660bc070 2024-07-31T14:53:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.919+0000 7ff8ebfff640 1 --2- 172.21.15.67:0/458888208 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x561f660bc070 0x561f664edc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8da7fc640 10 client.16269 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:53:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8ebfff640 1 --2- 172.21.15.67:0/458888208 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x561f660bc070 0x561f664edc10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=17d16e110ddcddff server_cookie=88ea644885146e71 in_seq=0 out_seq=0 2024-07-31T14:53:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8da7fc640 1 -- 172.21.15.67:0/458888208 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561f663d55a0 con 0x561f660bc070 2024-07-31T14:53:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8da7fc640 10 client.16269 handle_command_reply: tid=0 2024-07-31T14:53:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8ff9a7740 1 client.16269 shutdown 2024-07-31T14:53:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/458888208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f6607a9f0 msgr2=0x561f664e47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/458888208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f6607a9f0 0x561f664e47f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4004ad0 tx=0x7ff8e40aa9b0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/458888208 shutdown_connections 2024-07-31T14:53:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/458888208 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x561f660bc070 0x561f664edc10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/458888208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6609b390 0x561f664e5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/458888208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561f660931b0 0x561f664e4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8ff9a7740 1 --2- 172.21.15.67:0/458888208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f6607a9f0 0x561f664e47f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/458888208 >> 172.21.15.67:0/458888208 conn(0x561f660b0460 msgr2=0x561f6646b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8da7fc640 0 client.16269 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:53:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/458888208 shutdown_connections 2024-07-31T14:53:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8ff9a7740 1 -- 172.21.15.67:0/458888208 wait complete. 2024-07-31T14:53:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.920+0000 7ff8ff9a7740 20 client.16269 trim_cache size 0 max 16384 2024-07-31T14:53:15.927 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:53:15.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.921+0000 7ff8f9456640 1 -- 172.21.15.67:0/2644791183 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e00b3d40 msgr2=0x7ff8e00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:15.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.921+0000 7ff8f9456640 1 --2- 172.21.15.67:0/2644791183 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e00b3d40 0x7ff8e00b6250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec02f500 tx=0x7ff8ec02f310 comp rx=0 tx=0).stop 2024-07-31T14:53:15.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.921+0000 7ff8f9456640 1 -- 172.21.15.67:0/2644791183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f41024d0 msgr2=0x7ff8f4199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:15.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.921+0000 7ff8f9456640 1 --2- 172.21.15.67:0/2644791183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f41024d0 0x7ff8f4199f00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400e9f0 tx=0x7ff8e400edf0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.921+0000 7ff8f9456640 1 -- 172.21.15.67:0/2644791183 shutdown_connections 2024-07-31T14:53:15.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.921+0000 7ff8f9456640 1 --2- 172.21.15.67:0/2644791183 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e00b3d40 0x7ff8e00b6250 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.921+0000 7ff8f9456640 1 --2- 172.21.15.67:0/2644791183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f4108760 0x7ff8f419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.921+0000 7ff8f9456640 1 --2- 172.21.15.67:0/2644791183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8f4107e10 0x7ff8f419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.921+0000 7ff8f9456640 1 --2- 172.21.15.67:0/2644791183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8f41024d0 0x7ff8f4199f00 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:15.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.921+0000 7ff8f9456640 1 -- 172.21.15.67:0/2644791183 >> 172.21.15.67:0/2644791183 conn(0x7ff8f4079190 msgr2=0x7ff8f4103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:15.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.921+0000 7ff8f9456640 1 -- 172.21.15.67:0/2644791183 shutdown_connections 2024-07-31T14:53:15.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:15.921+0000 7ff8f9456640 1 -- 172.21.15.67:0/2644791183 wait complete. 2024-07-31T14:53:16.705 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T14:53:16.982 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:53:17.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:16 smithi067 ceph-mon[32750]: pgmap v465: 129 pgs: 129 active+clean; 138 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 11 MiB/s wr, 182 op/s 2024-07-31T14:53:17.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:16 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:17.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:16 smithi073 ceph-mon[36714]: pgmap v465: 129 pgs: 129 active+clean; 138 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 11 MiB/s wr, 182 op/s 2024-07-31T14:53:17.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:16 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:17.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:16 smithi077 ceph-mon[37327]: pgmap v465: 129 pgs: 129 active+clean; 138 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 11 MiB/s wr, 182 op/s 2024-07-31T14:53:17.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:16 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:19.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:18 smithi073 ceph-mon[36714]: pgmap v466: 129 pgs: 129 active+clean; 138 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 11 MiB/s wr, 282 op/s 2024-07-31T14:53:19.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:18 smithi077 ceph-mon[37327]: pgmap v466: 129 pgs: 129 active+clean; 138 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 11 MiB/s wr, 282 op/s 2024-07-31T14:53:19.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:18 smithi067 ceph-mon[32750]: pgmap v466: 129 pgs: 129 active+clean; 138 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 11 MiB/s wr, 282 op/s 2024-07-31T14:53:20.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:19 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:20.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:19 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:20.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:19 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:21.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:20 smithi067 ceph-mon[32750]: pgmap v467: 129 pgs: 129 active+clean; 146 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 12 MiB/s wr, 301 op/s 2024-07-31T14:53:21.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:20 smithi073 ceph-mon[36714]: pgmap v467: 129 pgs: 129 active+clean; 146 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 12 MiB/s wr, 301 op/s 2024-07-31T14:53:21.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:20 smithi077 ceph-mon[37327]: pgmap v467: 129 pgs: 129 active+clean; 146 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 12 MiB/s wr, 301 op/s 2024-07-31T14:53:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.823+0000 7fad97a9a640 1 -- 172.21.15.67:0/569563842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad901024d0 msgr2=0x7fad901028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.823+0000 7fad97a9a640 1 --2- 172.21.15.67:0/569563842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad901024d0 0x7fad901028e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fad80009900 tx=0x7fad8002a410 comp rx=0 tx=0).stop 2024-07-31T14:53:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.823+0000 7fad97a9a640 1 -- 172.21.15.67:0/569563842 shutdown_connections 2024-07-31T14:53:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.823+0000 7fad97a9a640 1 --2- 172.21.15.67:0/569563842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad90108760 0x7fad9010ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.823+0000 7fad97a9a640 1 --2- 172.21.15.67:0/569563842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad90107e10 0x7fad90108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.823+0000 7fad97a9a640 1 --2- 172.21.15.67:0/569563842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad901024d0 0x7fad901028e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.823+0000 7fad97a9a640 1 -- 172.21.15.67:0/569563842 >> 172.21.15.67:0/569563842 conn(0x7fad90079190 msgr2=0x7fad900795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.824+0000 7fad97a9a640 1 -- 172.21.15.67:0/569563842 shutdown_connections 2024-07-31T14:53:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.824+0000 7fad97a9a640 1 -- 172.21.15.67:0/569563842 wait complete. 2024-07-31T14:53:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.824+0000 7fad97a9a640 1 Processor -- start 2024-07-31T14:53:21.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.824+0000 7fad97a9a640 1 -- start start 2024-07-31T14:53:21.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad97a9a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad901024d0 0x7fad9019c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad97a9a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad90107e10 0x7fad9019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad97a9a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad90108760 0x7fad901a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad97a9a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fad901123e0 con 0x7fad90107e10 2024-07-31T14:53:21.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad97a9a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fad90112260 con 0x7fad901024d0 2024-07-31T14:53:21.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad97a9a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad90112560 con 0x7fad90108760 2024-07-31T14:53:21.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad95035640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad90107e10 0x7fad9019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad95035640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad90107e10 0x7fad9019c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45140/0 (socket says 172.21.15.67:45140) 2024-07-31T14:53:21.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad95836640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad901024d0 0x7fad9019c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad95035640 1 -- 172.21.15.67:0/3793694126 learned_addr learned my addr 172.21.15.67:0/3793694126 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:21.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad96037640 1 --2- 172.21.15.67:0/3793694126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad90108760 0x7fad901a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad95035640 1 -- 172.21.15.67:0/3793694126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad90108760 msgr2=0x7fad901a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:21.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad95035640 1 --2- 172.21.15.67:0/3793694126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad90108760 0x7fad901a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad95035640 1 -- 172.21.15.67:0/3793694126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad901024d0 msgr2=0x7fad9019c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:21.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad95035640 1 --2- 172.21.15.67:0/3793694126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad901024d0 0x7fad9019c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad95035640 1 -- 172.21.15.67:0/3793694126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad901a2020 con 0x7fad90107e10 2024-07-31T14:53:21.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad95035640 1 --2- 172.21.15.67:0/3793694126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad90107e10 0x7fad9019c650 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00d770 tx=0x7fad8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:21.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad86ffd640 1 -- 172.21.15.67:0/3793694126 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c017070 con 0x7fad90107e10 2024-07-31T14:53:21.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad86ffd640 1 -- 172.21.15.67:0/3793694126 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8c009c30 con 0x7fad90107e10 2024-07-31T14:53:21.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad97a9a640 1 -- 172.21.15.67:0/3793694126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad901a2310 con 0x7fad90107e10 2024-07-31T14:53:21.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.825+0000 7fad86ffd640 1 -- 172.21.15.67:0/3793694126 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00f620 con 0x7fad90107e10 2024-07-31T14:53:21.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.826+0000 7fad97a9a640 1 -- 172.21.15.67:0/3793694126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad901a8a70 con 0x7fad90107e10 2024-07-31T14:53:21.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.828+0000 7fad86ffd640 1 -- 172.21.15.67:0/3793694126 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fad8c01a070 con 0x7fad90107e10 2024-07-31T14:53:21.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.828+0000 7fad9dfeb740 1 Processor -- start 2024-07-31T14:53:21.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.828+0000 7fad9dfeb740 1 -- start start 2024-07-31T14:53:21.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad9dfeb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 0x56312b075690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad9dfeb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 0x56312b075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad9dfeb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b076110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad9dfeb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56312afebbb0 con 0x56312acaf390 2024-07-31T14:53:21.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad9dfeb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56312afdb0b0 con 0x56312acd0070 2024-07-31T14:53:21.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad9dfeb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56312afdf0f0 con 0x56312aca71b0 2024-07-31T14:53:21.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad95035640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b076110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad96037640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 0x56312b075690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad95035640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b076110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45152/0 (socket says 172.21.15.67:45152) 2024-07-31T14:53:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad95035640 1 -- 172.21.15.67:0/2078780707 learned_addr learned my addr 172.21.15.67:0/2078780707 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad95836640 1 --2- 172.21.15.67:0/2078780707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 0x56312b075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad95035640 1 -- 172.21.15.67:0/2078780707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 msgr2=0x56312b075bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad95035640 1 --2- 172.21.15.67:0/2078780707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 0x56312b075bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad95035640 1 -- 172.21.15.67:0/2078780707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 msgr2=0x56312b075690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad95035640 1 --2- 172.21.15.67:0/2078780707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 0x56312b075690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad95035640 1 -- 172.21.15.67:0/2078780707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312af81b90 con 0x56312acaf390 2024-07-31T14:53:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad86ffd640 1 --2- 172.21.15.67:0/3793694126 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fad7c0b3d40 0x7fad7c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.829+0000 7fad96037640 1 --2- 172.21.15.67:0/2078780707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 0x56312b075690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.830+0000 7fad96037640 1 --2- 172.21.15.67:0/3793694126 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fad7c0b3d40 0x7fad7c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.830+0000 7fad95035640 1 --2- 172.21.15.67:0/2078780707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b076110 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0e11f0 tx=0x7fad8c0ec570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b161d5afd6a39804 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.830+0000 7fad86ffd640 1 -- 172.21.15.67:0/3793694126 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fad8c0e3e40 con 0x7fad90107e10 2024-07-31T14:53:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.830+0000 7fad7bfff640 1 -- 172.21.15.67:0/2078780707 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c021020 con 0x56312acaf390 2024-07-31T14:53:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.830+0000 7fad7bfff640 1 -- 172.21.15.67:0/2078780707 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8c0b3130 con 0x56312acaf390 2024-07-31T14:53:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.830+0000 7fad7bfff640 1 -- 172.21.15.67:0/2078780707 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0a4e60 con 0x56312acaf390 2024-07-31T14:53:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.831+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2078780707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 msgr2=0x56312b076110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.831+0000 7fad96037640 1 --2- 172.21.15.67:0/3793694126 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fad7c0b3d40 0x7fad7c0b6250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fad88005fd0 tx=0x7fad88009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.831+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/2078780707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b076110 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0e11f0 tx=0x7fad8c0ec570 comp rx=0 tx=0).stop 2024-07-31T14:53:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.831+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2078780707 shutdown_connections 2024-07-31T14:53:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.831+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/2078780707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b076110 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.831+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/2078780707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 0x56312b075bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.831+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/2078780707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 0x56312b075690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.831+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2078780707 >> 172.21.15.67:0/2078780707 conn(0x56312acc4460 msgr2=0x56312b073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.831+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2078780707 shutdown_connections 2024-07-31T14:53:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.831+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2078780707 wait complete. 2024-07-31T14:53:21.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.831+0000 7fad9dfeb740 1 -- start start 2024-07-31T14:53:21.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.832+0000 7fad9dfeb740 1 Processor -- start 2024-07-31T14:53:21.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.832+0000 7fad9dfeb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 0x56312b081230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.832+0000 7fad9dfeb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b081770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.832+0000 7fad9dfeb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 0x56312b08d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.832+0000 7fad9dfeb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56312ae3e700 con 0x56312acaf390 2024-07-31T14:53:21.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.832+0000 7fad9dfeb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56312af7d9e0 con 0x56312acd0070 2024-07-31T14:53:21.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.832+0000 7fad9dfeb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56312b083e70 con 0x56312aca71b0 2024-07-31T14:53:21.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.832+0000 7fad95035640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b081770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.832+0000 7fad95836640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 0x56312b081230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.832+0000 7fad95035640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b081770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45160/0 (socket says 172.21.15.67:45160) 2024-07-31T14:53:21.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.832+0000 7fad95035640 1 -- 172.21.15.67:0/271514108 learned_addr learned my addr 172.21.15.67:0/271514108 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:21.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.832+0000 7fad9dfeb740 10 client.0 _ll_register_callbacks cb 0x56312af984f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:21.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.832+0000 7fad96037640 1 --2- 172.21.15.67:0/271514108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 0x56312b08d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.833+0000 7fad95836640 1 -- 172.21.15.67:0/271514108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 msgr2=0x56312b08d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:21.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.833+0000 7fad95836640 1 --2- 172.21.15.67:0/271514108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 0x56312b08d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.833+0000 7fad95836640 1 -- 172.21.15.67:0/271514108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 msgr2=0x56312b081770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:21.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.833+0000 7fad95836640 1 --2- 172.21.15.67:0/271514108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b081770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.833+0000 7fad95836640 1 -- 172.21.15.67:0/271514108 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56312ad3b650 con 0x56312aca71b0 2024-07-31T14:53:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.833+0000 7fad95035640 1 --2- 172.21.15.67:0/271514108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b081770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.833+0000 7fad95836640 1 --2- 172.21.15.67:0/271514108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 0x56312b081230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad80009230 tx=0x7fad800316d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbf5a4642824129b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.833+0000 7fad797fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:53:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.833+0000 7fad797fa640 1 -- 172.21.15.67:0/271514108 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80035040 con 0x56312aca71b0 2024-07-31T14:53:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.833+0000 7fad9dfeb740 1 -- 172.21.15.67:0/271514108 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56312afedb00 con 0x56312aca71b0 2024-07-31T14:53:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.833+0000 7fad797fa640 1 -- 172.21.15.67:0/271514108 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fad80004250 con 0x56312aca71b0 2024-07-31T14:53:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.833+0000 7fad797fa640 1 -- 172.21.15.67:0/271514108 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fad8004e050 con 0x56312aca71b0 2024-07-31T14:53:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.833+0000 7fad9dfeb740 10 client.25951 fetch_fsmap learned FSMap version 32 2024-07-31T14:53:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.833+0000 7fad9dfeb740 1 -- 172.21.15.67:0/271514108 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56312b08fe90 con 0x56312aca71b0 2024-07-31T14:53:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.834+0000 7fad797fa640 1 -- 172.21.15.67:0/271514108 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56312afedb00 con 0x56312aca71b0 2024-07-31T14:53:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.834+0000 7fad797fa640 1 -- 172.21.15.67:0/271514108 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7fad80004870 con 0x56312aca71b0 2024-07-31T14:53:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.834+0000 7fad9dfeb740 10 client.25951 fetch_fsmap finished waiting for FSMap version 32 2024-07-31T14:53:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.834+0000 7fad9dfeb740 10 client.25951 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:53:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.834+0000 7fad9dfeb740 20 client.25951 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:21.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.834+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/271514108 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56312ac8e9f0 0x56312b0934b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.834+0000 7fad9dfeb740 4 client.25951 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:53:21.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.834+0000 7fad9dfeb740 1 -- 172.21.15.67:0/271514108 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56312afe95a0 con 0x56312ac8e9f0 2024-07-31T14:53:21.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.834+0000 7fad95035640 1 --2- 172.21.15.67:0/271514108 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56312ac8e9f0 0x56312b0934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.835+0000 7fad797fa640 10 client.25951 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:53:21.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.835+0000 7fad95035640 1 --2- 172.21.15.67:0/271514108 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56312ac8e9f0 0x56312b0934b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d792321c20056c7 server_cookie=ecfde9456fc7c3d2 in_seq=0 out_seq=0 2024-07-31T14:53:21.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.835+0000 7fad797fa640 1 -- 172.21.15.67:0/271514108 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56312afe95a0 con 0x56312ac8e9f0 2024-07-31T14:53:21.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.835+0000 7fad797fa640 10 client.25951 handle_command_reply: tid=0 2024-07-31T14:53:21.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.835+0000 7fad9dfeb740 1 client.25951 shutdown 2024-07-31T14:53:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.836+0000 7fad9dfeb740 1 -- 172.21.15.67:0/271514108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 msgr2=0x56312b081230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.836+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/271514108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 0x56312b081230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad80009230 tx=0x7fad800316d0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.836+0000 7fad9dfeb740 1 -- 172.21.15.67:0/271514108 shutdown_connections 2024-07-31T14:53:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.836+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/271514108 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56312ac8e9f0 0x56312b0934b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.836+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/271514108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 0x56312b08d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.836+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/271514108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b081770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.836+0000 7fad797fa640 0 client.25951 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:53:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.836+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/271514108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 0x56312b081230 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.836+0000 7fad9dfeb740 1 -- 172.21.15.67:0/271514108 >> 172.21.15.67:0/271514108 conn(0x56312acc4460 msgr2=0x56312b071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.836+0000 7fad9dfeb740 1 -- 172.21.15.67:0/271514108 shutdown_connections 2024-07-31T14:53:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.836+0000 7fad9dfeb740 1 -- 172.21.15.67:0/271514108 wait complete. 2024-07-31T14:53:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.836+0000 7fad9dfeb740 20 client.25951 trim_cache size 0 max 16384 2024-07-31T14:53:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.850+0000 7fad9dfeb740 1 Processor -- start 2024-07-31T14:53:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.850+0000 7fad9dfeb740 1 -- start start 2024-07-31T14:53:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.850+0000 7fad9dfeb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 0x56312b07aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.850+0000 7fad9dfeb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b08cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.850+0000 7fad9dfeb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 0x56312b08d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.850+0000 7fad9dfeb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fad901b2860 con 0x56312acaf390 2024-07-31T14:53:21.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.850+0000 7fad9dfeb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fad901b44c0 con 0x56312acd0070 2024-07-31T14:53:21.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.850+0000 7fad9dfeb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad901b2f30 con 0x56312aca71b0 2024-07-31T14:53:21.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.850+0000 7fad95035640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b08cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.850+0000 7fad95035640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b08cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45174/0 (socket says 172.21.15.67:45174) 2024-07-31T14:53:21.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.850+0000 7fad95035640 1 -- 172.21.15.67:0/2792226372 learned_addr learned my addr 172.21.15.67:0/2792226372 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:21.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.851+0000 7fad95836640 1 --2- 172.21.15.67:0/2792226372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 0x56312b07aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.851+0000 7fad96037640 1 --2- 172.21.15.67:0/2792226372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 0x56312b08d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.851+0000 7fad95035640 1 -- 172.21.15.67:0/2792226372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 msgr2=0x56312b07aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:21.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.851+0000 7fad95035640 1 --2- 172.21.15.67:0/2792226372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 0x56312b07aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.851+0000 7fad95035640 1 -- 172.21.15.67:0/2792226372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 msgr2=0x56312b08d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:21.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.851+0000 7fad95035640 1 --2- 172.21.15.67:0/2792226372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 0x56312b08d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.851+0000 7fad95035640 1 -- 172.21.15.67:0/2792226372 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312b0822f0 con 0x56312acaf390 2024-07-31T14:53:21.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.851+0000 7fad96037640 1 --2- 172.21.15.67:0/2792226372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 0x56312b08d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:53:21.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.851+0000 7fad95035640 1 --2- 172.21.15.67:0/2792226372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b08cf80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0e1880 tx=0x7fad8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88418ed3ef0c1c97 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:21.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.851+0000 7fad7b7fe640 1 -- 172.21.15.67:0/2792226372 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c021040 con 0x56312acaf390 2024-07-31T14:53:21.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.851+0000 7fad7b7fe640 1 -- 172.21.15.67:0/2792226372 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8c0e7760 con 0x56312acaf390 2024-07-31T14:53:21.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.851+0000 7fad7b7fe640 1 -- 172.21.15.67:0/2792226372 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0a4e60 con 0x56312acaf390 2024-07-31T14:53:21.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.852+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2792226372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 msgr2=0x56312b08cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:21.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.852+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/2792226372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b08cf80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0e1880 tx=0x7fad8c0057f0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.852+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2792226372 shutdown_connections 2024-07-31T14:53:21.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.852+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/2792226372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acd0070 0x56312b08d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.852+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/2792226372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312acaf390 0x56312b08cf80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.852+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/2792226372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312aca71b0 0x56312b07aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.852+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2792226372 >> 172.21.15.67:0/2792226372 conn(0x56312acc4460 msgr2=0x56312b07a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:21.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.852+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2792226372 shutdown_connections 2024-07-31T14:53:21.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.852+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2792226372 wait complete. 2024-07-31T14:53:21.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.852+0000 7fad9dfeb740 1 -- start start 2024-07-31T14:53:21.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.853+0000 7fad9dfeb740 1 Processor -- start 2024-07-31T14:53:21.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.853+0000 7fad9dfeb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312aca71b0 0x56312b084a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.853+0000 7fad9dfeb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acaf390 0x56312b085030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.853+0000 7fad9dfeb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312acd0070 0x56312b088f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.853+0000 7fad9dfeb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fad901b3d60 con 0x56312aca71b0 2024-07-31T14:53:21.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.853+0000 7fad9dfeb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fad901b3690 con 0x56312acaf390 2024-07-31T14:53:21.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.853+0000 7fad9dfeb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad901b0840 con 0x56312acd0070 2024-07-31T14:53:21.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.853+0000 7fad9dfeb740 10 client.0 _ll_register_callbacks cb 0x7fad80009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:21.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.853+0000 7fad95836640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312aca71b0 0x56312b084a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.853+0000 7fad95836640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312aca71b0 0x56312b084a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45184/0 (socket says 172.21.15.67:45184) 2024-07-31T14:53:21.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.853+0000 7fad95836640 1 -- 172.21.15.67:0/2951307091 learned_addr learned my addr 172.21.15.67:0/2951307091 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:21.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.853+0000 7fad96037640 1 --2- 172.21.15.67:0/2951307091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312acd0070 0x56312b088f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.853+0000 7fad95035640 1 --2- 172.21.15.67:0/2951307091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acaf390 0x56312b085030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.854+0000 7fad95836640 1 -- 172.21.15.67:0/2951307091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312acd0070 msgr2=0x56312b088f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:21.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.854+0000 7fad95836640 1 --2- 172.21.15.67:0/2951307091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312acd0070 0x56312b088f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.854+0000 7fad95836640 1 -- 172.21.15.67:0/2951307091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acaf390 msgr2=0x56312b085030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:21.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.854+0000 7fad95836640 1 --2- 172.21.15.67:0/2951307091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acaf390 0x56312b085030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.854+0000 7fad95836640 1 -- 172.21.15.67:0/2951307091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56312b089590 con 0x56312aca71b0 2024-07-31T14:53:21.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.854+0000 7fad95836640 1 --2- 172.21.15.67:0/2951307091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312aca71b0 0x56312b084a70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fad80009850 tx=0x7fad80005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e3d106a9d26d0f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:21.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.854+0000 7fad797fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:21.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.854+0000 7fad797fa640 1 -- 172.21.15.67:0/2951307091 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80056020 con 0x56312aca71b0 2024-07-31T14:53:21.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.854+0000 7fad797fa640 1 -- 172.21.15.67:0/2951307091 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fad80032dc0 con 0x56312aca71b0 2024-07-31T14:53:21.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.854+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2951307091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad8c0e7760 con 0x56312aca71b0 2024-07-31T14:53:21.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.854+0000 7fad797fa640 1 -- 172.21.15.67:0/2951307091 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fad80035040 con 0x56312aca71b0 2024-07-31T14:53:21.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.854+0000 7fad9dfeb740 10 client.16287 fetch_fsmap learned FSMap version 32 2024-07-31T14:53:21.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.854+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2951307091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56312b0fb760 con 0x56312aca71b0 2024-07-31T14:53:21.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.854+0000 7fad797fa640 1 -- 172.21.15.67:0/2951307091 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8c0e7760 con 0x56312aca71b0 2024-07-31T14:53:21.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.855+0000 7fad797fa640 1 -- 172.21.15.67:0/2951307091 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7fad80005170 con 0x56312aca71b0 2024-07-31T14:53:21.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.855+0000 7fad9dfeb740 10 client.16287 fetch_fsmap finished waiting for FSMap version 32 2024-07-31T14:53:21.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.855+0000 7fad9dfeb740 10 client.16287 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:53:21.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.855+0000 7fad9dfeb740 20 client.16287 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:21.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.855+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/2951307091 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56312ac63da0 0x56312b0fdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:21.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.855+0000 7fad9dfeb740 4 client.16287 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:53:21.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.855+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2951307091 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56312afe6ce0 con 0x56312ac63da0 2024-07-31T14:53:21.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.855+0000 7fad95035640 1 --2- 172.21.15.67:0/2951307091 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56312ac63da0 0x56312b0fdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:21.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.855+0000 7fad95035640 1 --2- 172.21.15.67:0/2951307091 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56312ac63da0 0x56312b0fdfd0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfbdbe8c255e1e0c server_cookie=ac5305f8cc10a148 in_seq=0 out_seq=0 2024-07-31T14:53:21.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.855+0000 7fad797fa640 10 client.16287 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:53:21.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad797fa640 1 -- 172.21.15.67:0/2951307091 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56312afe6ce0 con 0x56312ac63da0 2024-07-31T14:53:21.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad797fa640 10 client.16287 handle_command_reply: tid=0 2024-07-31T14:53:21.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad9dfeb740 1 client.16287 shutdown 2024-07-31T14:53:21.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2951307091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312aca71b0 msgr2=0x56312b084a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:21.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/2951307091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312aca71b0 0x56312b084a70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fad80009850 tx=0x7fad80005fd0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2951307091 shutdown_connections 2024-07-31T14:53:21.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/2951307091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56312acd0070 0x56312b088f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/2951307091 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56312ac63da0 0x56312b0fdfd0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/2951307091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56312acaf390 0x56312b085030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad9dfeb740 1 --2- 172.21.15.67:0/2951307091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312aca71b0 0x56312b084a70 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad797fa640 0 client.16287 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:53:21.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2951307091 >> 172.21.15.67:0/2951307091 conn(0x56312acc4460 msgr2=0x56312b072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:21.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2951307091 shutdown_connections 2024-07-31T14:53:21.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad9dfeb740 1 -- 172.21.15.67:0/2951307091 wait complete. 2024-07-31T14:53:21.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.856+0000 7fad9dfeb740 20 client.16287 trim_cache size 0 max 16384 2024-07-31T14:53:21.863 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:53:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.857+0000 7fad97a9a640 1 -- 172.21.15.67:0/3793694126 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fad7c0b3d40 msgr2=0x7fad7c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.857+0000 7fad97a9a640 1 --2- 172.21.15.67:0/3793694126 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fad7c0b3d40 0x7fad7c0b6250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fad88005fd0 tx=0x7fad88009500 comp rx=0 tx=0).stop 2024-07-31T14:53:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.857+0000 7fad97a9a640 1 -- 172.21.15.67:0/3793694126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad90107e10 msgr2=0x7fad9019c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.857+0000 7fad97a9a640 1 --2- 172.21.15.67:0/3793694126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad90107e10 0x7fad9019c650 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00d770 tx=0x7fad8c00dc30 comp rx=0 tx=0).stop 2024-07-31T14:53:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.857+0000 7fad97a9a640 1 -- 172.21.15.67:0/3793694126 shutdown_connections 2024-07-31T14:53:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.857+0000 7fad97a9a640 1 --2- 172.21.15.67:0/3793694126 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fad7c0b3d40 0x7fad7c0b6250 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.857+0000 7fad97a9a640 1 --2- 172.21.15.67:0/3793694126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad90108760 0x7fad901a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.857+0000 7fad97a9a640 1 --2- 172.21.15.67:0/3793694126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad90107e10 0x7fad9019c650 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.857+0000 7fad97a9a640 1 --2- 172.21.15.67:0/3793694126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad901024d0 0x7fad9019c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.857+0000 7fad97a9a640 1 -- 172.21.15.67:0/3793694126 >> 172.21.15.67:0/3793694126 conn(0x7fad90079190 msgr2=0x7fad90105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.857+0000 7fad97a9a640 1 -- 172.21.15.67:0/3793694126 shutdown_connections 2024-07-31T14:53:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:21.857+0000 7fad97a9a640 1 -- 172.21.15.67:0/3793694126 wait complete. 2024-07-31T14:53:22.847 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T14:53:23.133 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:53:23.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:22 smithi073 ceph-mon[36714]: pgmap v468: 129 pgs: 129 active+clean; 146 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 5.0 MiB/s wr, 295 op/s 2024-07-31T14:53:23.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:22 smithi077 ceph-mon[37327]: pgmap v468: 129 pgs: 129 active+clean; 146 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 5.0 MiB/s wr, 295 op/s 2024-07-31T14:53:23.257 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:22 smithi067 ceph-mon[32750]: pgmap v468: 129 pgs: 129 active+clean; 146 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 5.0 MiB/s wr, 295 op/s 2024-07-31T14:53:24.650 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:24 smithi073 systemd[1]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 1. 2024-07-31T14:53:24.650 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:24 smithi073 systemd[1]: Stopped Ceph mds.e for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:53:24.650 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:24 smithi073 systemd[1]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.e.service: Consumed 1.389s CPU time. 2024-07-31T14:53:24.650 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:24 smithi073 systemd[1]: Starting Ceph mds.e for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T14:53:24.944 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:24 smithi073 ceph-mon[36714]: pgmap v469: 129 pgs: 129 active+clean; 146 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 5.1 MiB/s wr, 303 op/s 2024-07-31T14:53:24.944 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:24 smithi073 podman[70326]: 2024-07-31 14:53:24.552536622 +0000 UTC m=+0.022153231 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T14:53:24.944 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:24 smithi073 podman[70326]: 2024-07-31 14:53:24.67266218 +0000 UTC m=+0.142278778 container create d2b5d61a47c8fcabd761759224b6df40abfddd34f326ce81debf0d79dbfc294d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T14:53:24.944 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:24 smithi073 podman[70326]: 2024-07-31 14:53:24.943374443 +0000 UTC m=+0.412991068 container init d2b5d61a47c8fcabd761759224b6df40abfddd34f326ce81debf0d79dbfc294d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-07-31T14:53:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.980+0000 7f7f99f12640 1 -- 172.21.15.67:0/3989037889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f9410d860 msgr2=0x7f7f9410fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.980+0000 7f7f99f12640 1 --2- 172.21.15.67:0/3989037889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f9410d860 0x7f7f9410fd30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009ca0 tx=0x7f7f8c02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.980+0000 7f7f99f12640 1 -- 172.21.15.67:0/3989037889 shutdown_connections 2024-07-31T14:53:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.980+0000 7f7f99f12640 1 --2- 172.21.15.67:0/3989037889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f9410d860 0x7f7f9410fd30 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.980+0000 7f7f99f12640 1 --2- 172.21.15.67:0/3989037889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f94104ce0 0x7f7f9410d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.980+0000 7f7f99f12640 1 --2- 172.21.15.67:0/3989037889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f941042b0 0x7f7f941046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.980+0000 7f7f99f12640 1 -- 172.21.15.67:0/3989037889 >> 172.21.15.67:0/3989037889 conn(0x7f7f94079160 msgr2=0x7f7f94079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.980+0000 7f7f99f12640 1 -- 172.21.15.67:0/3989037889 shutdown_connections 2024-07-31T14:53:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.980+0000 7f7f99f12640 1 -- 172.21.15.67:0/3989037889 wait complete. 2024-07-31T14:53:24.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.981+0000 7f7f99f12640 1 Processor -- start 2024-07-31T14:53:24.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.981+0000 7f7f99f12640 1 -- start start 2024-07-31T14:53:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.981+0000 7f7f99f12640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f941042b0 0x7f7f94103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.981+0000 7f7f99f12640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f94104ce0 0x7f7f941021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.981+0000 7f7f99f12640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f9410d860 0x7f7f94102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f99f12640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7f94113480 con 0x7f7f941042b0 2024-07-31T14:53:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f99f12640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7f94113300 con 0x7f7f9410d860 2024-07-31T14:53:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f99f12640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f94113600 con 0x7f7f94104ce0 2024-07-31T14:53:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f937fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f941042b0 0x7f7f94103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f937fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f941042b0 0x7f7f94103bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45204/0 (socket says 172.21.15.67:45204) 2024-07-31T14:53:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f937fe640 1 -- 172.21.15.67:0/3568676481 learned_addr learned my addr 172.21.15.67:0/3568676481 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f93fff640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f9410d860 0x7f7f94102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f8bfff640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f94104ce0 0x7f7f941021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f937fe640 1 -- 172.21.15.67:0/3568676481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f94104ce0 msgr2=0x7f7f941021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:24.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f937fe640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f94104ce0 0x7f7f941021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f937fe640 1 -- 172.21.15.67:0/3568676481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f9410d860 msgr2=0x7f7f94102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:24.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f937fe640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f9410d860 0x7f7f94102700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f937fe640 1 -- 172.21.15.67:0/3568676481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f94102f20 con 0x7f7f941042b0 2024-07-31T14:53:24.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f93fff640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f9410d860 0x7f7f94102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:24.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f937fe640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f941042b0 0x7f7f94103bd0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400d950 tx=0x7f7f8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:24.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f917fa640 1 -- 172.21.15.67:0/3568676481 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84017020 con 0x7f7f941042b0 2024-07-31T14:53:24.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f917fa640 1 -- 172.21.15.67:0/3568676481 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f84009c30 con 0x7f7f941042b0 2024-07-31T14:53:24.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.982+0000 7f7f917fa640 1 -- 172.21.15.67:0/3568676481 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8400f690 con 0x7f7f941042b0 2024-07-31T14:53:24.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.983+0000 7f7f99f12640 1 -- 172.21.15.67:0/3568676481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f941ac820 con 0x7f7f941042b0 2024-07-31T14:53:24.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.983+0000 7f7f99f12640 1 -- 172.21.15.67:0/3568676481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f941acd60 con 0x7f7f941042b0 2024-07-31T14:53:24.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.985+0000 7f7fa0463740 1 Processor -- start 2024-07-31T14:53:24.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.985+0000 7f7f917fa640 1 -- 172.21.15.67:0/3568676481 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7f8401a020 con 0x7f7f941042b0 2024-07-31T14:53:24.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7fa0463740 1 -- start start 2024-07-31T14:53:24.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7fa0463740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 0x55ead50cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7fa0463740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 0x55ead50ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7fa0463740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d06390 0x55ead50ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7fa0463740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ead5042bb0 con 0x55ead4d27070 2024-07-31T14:53:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7fa0463740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ead50320b0 con 0x55ead4d06390 2024-07-31T14:53:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7fa0463740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ead50360f0 con 0x55ead4cfe1b0 2024-07-31T14:53:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7f8bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d06390 0x55ead50ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7f93fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 0x55ead50cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7f8bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d06390 0x55ead50ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:36974/0 (socket says 172.21.15.67:36974) 2024-07-31T14:53:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7f93fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 0x55ead50cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45208/0 (socket says 172.21.15.67:45208) 2024-07-31T14:53:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7f8bfff640 1 -- 172.21.15.67:0/621955378 learned_addr learned my addr 172.21.15.67:0/621955378 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7f937fe640 1 --2- 172.21.15.67:0/621955378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 0x55ead50ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:24.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7f8bfff640 1 -- 172.21.15.67:0/621955378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 msgr2=0x55ead50ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:24.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7f8bfff640 1 --2- 172.21.15.67:0/621955378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 0x55ead50ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7f8bfff640 1 -- 172.21.15.67:0/621955378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 msgr2=0x55ead50cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:24.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7f8bfff640 1 --2- 172.21.15.67:0/621955378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 0x55ead50cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.986+0000 7f7f8bfff640 1 -- 172.21.15.67:0/621955378 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ead4fd8b90 con 0x55ead4d06390 2024-07-31T14:53:24.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.987+0000 7f7f917fa640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7f800b3dc0 0x7f7f800b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:24.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.987+0000 7f7f8bfff640 1 --2- 172.21.15.67:0/621955378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d06390 0x55ead50ccff0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009c20 tx=0x7f7f7c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e25a2c8f92e20a4c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:24.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.987+0000 7f7f93fff640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7f800b3dc0 0x7f7f800b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:24.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.987+0000 7f7f917fa640 1 -- 172.21.15.67:0/3568676481 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f7f840e4160 con 0x7f7f941042b0 2024-07-31T14:53:24.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.987+0000 7f7f89ffb640 1 -- 172.21.15.67:0/621955378 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c004050 con 0x55ead4d06390 2024-07-31T14:53:24.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.987+0000 7f7f89ffb640 1 -- 172.21.15.67:0/621955378 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f7c0054d0 con 0x55ead4d06390 2024-07-31T14:53:24.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.987+0000 7f7f89ffb640 1 -- 172.21.15.67:0/621955378 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c004d40 con 0x55ead4d06390 2024-07-31T14:53:24.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.988+0000 7f7fa0463740 1 -- 172.21.15.67:0/621955378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d06390 msgr2=0x55ead50ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:24.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.988+0000 7f7f93fff640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7f800b3dc0 0x7f7f800b62d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7f94104280 tx=0x7f7f8c032ed0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:24.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.988+0000 7f7fa0463740 1 --2- 172.21.15.67:0/621955378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d06390 0x55ead50ccff0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009c20 tx=0x7f7f7c011810 comp rx=0 tx=0).stop 2024-07-31T14:53:24.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.988+0000 7f7fa0463740 1 -- 172.21.15.67:0/621955378 shutdown_connections 2024-07-31T14:53:24.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.988+0000 7f7fa0463740 1 --2- 172.21.15.67:0/621955378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d06390 0x55ead50ccff0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.988+0000 7f7fa0463740 1 --2- 172.21.15.67:0/621955378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 0x55ead50ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.988+0000 7f7fa0463740 1 --2- 172.21.15.67:0/621955378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 0x55ead50cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.988+0000 7f7fa0463740 1 -- 172.21.15.67:0/621955378 >> 172.21.15.67:0/621955378 conn(0x55ead4d1b460 msgr2=0x55ead50ca460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.988+0000 7f7fa0463740 1 -- 172.21.15.67:0/621955378 shutdown_connections 2024-07-31T14:53:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.988+0000 7f7fa0463740 1 -- 172.21.15.67:0/621955378 wait complete. 2024-07-31T14:53:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.988+0000 7f7fa0463740 1 -- start start 2024-07-31T14:53:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.989+0000 7f7fa0463740 1 Processor -- start 2024-07-31T14:53:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.989+0000 7f7fa0463740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4cfe1b0 0x55ead50d8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.989+0000 7f7fa0463740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4d06390 0x55ead50d86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.989+0000 7f7fa0463740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 0x55ead50e4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.989+0000 7f7fa0463740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ead4e95700 con 0x55ead4d27070 2024-07-31T14:53:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.989+0000 7f7fa0463740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ead4fd49e0 con 0x55ead4cfe1b0 2024-07-31T14:53:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.989+0000 7f7fa0463740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ead50dad50 con 0x55ead4d06390 2024-07-31T14:53:24.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.989+0000 7f7fa0463740 10 client.0 _ll_register_callbacks cb 0x55ead4fef4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:24.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.989+0000 7f7f937fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4cfe1b0 0x55ead50d8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:24.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.989+0000 7f7f93fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 0x55ead50e4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:24.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.989+0000 7f7f937fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4cfe1b0 0x55ead50d8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:36982/0 (socket says 172.21.15.67:36982) 2024-07-31T14:53:24.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.989+0000 7f7f937fe640 1 -- 172.21.15.67:0/3211317901 learned_addr learned my addr 172.21.15.67:0/3211317901 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:24.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.989+0000 7f7f8bfff640 1 --2- 172.21.15.67:0/3211317901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4d06390 0x55ead50d86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:24.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7f937fe640 1 -- 172.21.15.67:0/3211317901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4d06390 msgr2=0x55ead50d86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:24.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7f937fe640 1 --2- 172.21.15.67:0/3211317901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4d06390 0x55ead50d86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7f937fe640 1 -- 172.21.15.67:0/3211317901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 msgr2=0x55ead50e4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:24.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7f937fe640 1 --2- 172.21.15.67:0/3211317901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 0x55ead50e4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7f937fe640 1 -- 172.21.15.67:0/3211317901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ead4d92650 con 0x55ead4cfe1b0 2024-07-31T14:53:24.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7f937fe640 1 --2- 172.21.15.67:0/3211317901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4cfe1b0 0x55ead50d8160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7f84000c00 tx=0x7f7f84010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8a6491d577db390 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7f72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:53:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7f72ffd640 1 -- 172.21.15.67:0/3211317901 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f840aa1d0 con 0x55ead4cfe1b0 2024-07-31T14:53:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7fa0463740 1 -- 172.21.15.67:0/3211317901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7c0054d0 con 0x55ead4cfe1b0 2024-07-31T14:53:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7f72ffd640 1 -- 172.21.15.67:0/3211317901 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f7f840a6cf0 con 0x55ead4cfe1b0 2024-07-31T14:53:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7f72ffd640 1 -- 172.21.15.67:0/3211317901 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7f840a61f0 con 0x55ead4cfe1b0 2024-07-31T14:53:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7fa0463740 10 client.25931 fetch_fsmap learned FSMap version 32 2024-07-31T14:53:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7fa0463740 1 -- 172.21.15.67:0/3211317901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ead50e6db0 con 0x55ead4cfe1b0 2024-07-31T14:53:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7f72ffd640 1 -- 172.21.15.67:0/3211317901 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f7c0054d0 con 0x55ead4cfe1b0 2024-07-31T14:53:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.990+0000 7f7f72ffd640 1 -- 172.21.15.67:0/3211317901 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f7f84090b00 con 0x55ead4cfe1b0 2024-07-31T14:53:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.991+0000 7f7fa0463740 10 client.25931 fetch_fsmap finished waiting for FSMap version 32 2024-07-31T14:53:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.991+0000 7f7fa0463740 10 client.25931 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:53:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.991+0000 7f7fa0463740 20 client.25931 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.991+0000 7f7fa0463740 1 --2- 172.21.15.67:0/3211317901 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ead4ce59f0 0x55ead50ea280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.991+0000 7f7fa0463740 4 client.25931 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:53:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.991+0000 7f7fa0463740 1 -- 172.21.15.67:0/3211317901 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ead5044b00 con 0x55ead4ce59f0 2024-07-31T14:53:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.991+0000 7f7f8bfff640 1 --2- 172.21.15.67:0/3211317901 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ead4ce59f0 0x55ead50ea280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.991+0000 7f7f72ffd640 10 client.25931 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:53:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.991+0000 7f7f8bfff640 1 --2- 172.21.15.67:0/3211317901 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ead4ce59f0 0x55ead50ea280 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9fc15c3e204f655 server_cookie=162210e2547d2154 in_seq=0 out_seq=0 2024-07-31T14:53:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.992+0000 7f7f72ffd640 1 -- 172.21.15.67:0/3211317901 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ead5044b00 con 0x55ead4ce59f0 2024-07-31T14:53:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.992+0000 7f7f72ffd640 10 client.25931 handle_command_reply: tid=0 2024-07-31T14:53:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.992+0000 7f7fa0463740 1 client.25931 shutdown 2024-07-31T14:53:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.992+0000 7f7fa0463740 1 -- 172.21.15.67:0/3211317901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4cfe1b0 msgr2=0x55ead50d8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.992+0000 7f7fa0463740 1 --2- 172.21.15.67:0/3211317901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4cfe1b0 0x55ead50d8160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7f84000c00 tx=0x7f7f84010bd0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.992+0000 7f7fa0463740 1 -- 172.21.15.67:0/3211317901 shutdown_connections 2024-07-31T14:53:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.992+0000 7f7fa0463740 1 --2- 172.21.15.67:0/3211317901 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ead4ce59f0 0x55ead50ea280 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.992+0000 7f7fa0463740 1 --2- 172.21.15.67:0/3211317901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 0x55ead50e4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.992+0000 7f7fa0463740 1 --2- 172.21.15.67:0/3211317901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4d06390 0x55ead50d86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.992+0000 7f7fa0463740 1 --2- 172.21.15.67:0/3211317901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4cfe1b0 0x55ead50d8160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:24.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.992+0000 7f7f72ffd640 0 client.25931 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:53:24.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.992+0000 7f7fa0463740 1 -- 172.21.15.67:0/3211317901 >> 172.21.15.67:0/3211317901 conn(0x55ead4d1b460 msgr2=0x55ead50c8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:24.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.993+0000 7f7fa0463740 1 -- 172.21.15.67:0/3211317901 shutdown_connections 2024-07-31T14:53:24.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.993+0000 7f7fa0463740 1 -- 172.21.15.67:0/3211317901 wait complete. 2024-07-31T14:53:24.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:24.993+0000 7f7fa0463740 20 client.25931 trim_cache size 0 max 16384 2024-07-31T14:53:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.006+0000 7f7fa0463740 1 Processor -- start 2024-07-31T14:53:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7fa0463740 1 -- start start 2024-07-31T14:53:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7fa0463740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 0x55ead50ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7fa0463740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d06390 0x55ead50e3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7fa0463740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d27070 0x55ead50efa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7fa0463740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7f941b6c30 con 0x55ead4d06390 2024-07-31T14:53:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7fa0463740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7f941b8890 con 0x55ead4d27070 2024-07-31T14:53:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7fa0463740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f941b7300 con 0x55ead4cfe1b0 2024-07-31T14:53:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7f8bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d06390 0x55ead50e3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7f8bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d06390 0x55ead50e3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45218/0 (socket says 172.21.15.67:45218) 2024-07-31T14:53:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7f8bfff640 1 -- 172.21.15.67:0/4223124436 learned_addr learned my addr 172.21.15.67:0/4223124436 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7f93fff640 1 --2- 172.21.15.67:0/4223124436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d27070 0x55ead50efa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7f937fe640 1 --2- 172.21.15.67:0/4223124436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 0x55ead50ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7f8bfff640 1 -- 172.21.15.67:0/4223124436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 msgr2=0x55ead50ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7f8bfff640 1 --2- 172.21.15.67:0/4223124436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 0x55ead50ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7f8bfff640 1 -- 172.21.15.67:0/4223124436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d27070 msgr2=0x55ead50efa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7f8bfff640 1 --2- 172.21.15.67:0/4223124436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d27070 0x55ead50efa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7f8bfff640 1 -- 172.21.15.67:0/4223124436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ead50d9220 con 0x55ead4d06390 2024-07-31T14:53:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7f937fe640 1 --2- 172.21.15.67:0/4223124436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 0x55ead50ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:53:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.007+0000 7f7f8bfff640 1 --2- 172.21.15.67:0/4223124436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d06390 0x55ead50e3e30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009bf0 tx=0x7f7f7c0041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226e24d950b25857 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.008+0000 7f7f897fa640 1 -- 172.21.15.67:0/4223124436 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0035f0 con 0x55ead4d06390 2024-07-31T14:53:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.008+0000 7f7f897fa640 1 -- 172.21.15.67:0/4223124436 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f7c003c40 con 0x55ead4d06390 2024-07-31T14:53:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.008+0000 7f7f897fa640 1 -- 172.21.15.67:0/4223124436 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0197f0 con 0x55ead4d06390 2024-07-31T14:53:25.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.008+0000 7f7fa0463740 1 -- 172.21.15.67:0/4223124436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d06390 msgr2=0x55ead50e3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:25.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.008+0000 7f7fa0463740 1 --2- 172.21.15.67:0/4223124436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d06390 0x55ead50e3e30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009bf0 tx=0x7f7f7c0041d0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.009+0000 7f7fa0463740 1 -- 172.21.15.67:0/4223124436 shutdown_connections 2024-07-31T14:53:25.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.009+0000 7f7fa0463740 1 --2- 172.21.15.67:0/4223124436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d27070 0x55ead50efa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.009+0000 7f7fa0463740 1 --2- 172.21.15.67:0/4223124436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d06390 0x55ead50e3e30 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.009+0000 7f7fa0463740 1 --2- 172.21.15.67:0/4223124436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 0x55ead50ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.009+0000 7f7fa0463740 1 -- 172.21.15.67:0/4223124436 >> 172.21.15.67:0/4223124436 conn(0x55ead4d1b460 msgr2=0x55ead50c9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:25.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.009+0000 7f7fa0463740 1 -- 172.21.15.67:0/4223124436 shutdown_connections 2024-07-31T14:53:25.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.009+0000 7f7fa0463740 1 -- 172.21.15.67:0/4223124436 wait complete. 2024-07-31T14:53:25.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.009+0000 7f7fa0463740 1 -- start start 2024-07-31T14:53:25.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.009+0000 7f7fa0463740 1 Processor -- start 2024-07-31T14:53:25.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7fa0463740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 0x55ead50e07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:25.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7fa0463740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d06390 0x55ead50dbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:25.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7fa0463740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 0x55ead50dc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:25.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7fa0463740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7f941b8130 con 0x55ead4d27070 2024-07-31T14:53:25.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7fa0463740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7f941b7a60 con 0x55ead4d06390 2024-07-31T14:53:25.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7fa0463740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f941b4c10 con 0x55ead4cfe1b0 2024-07-31T14:53:25.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7fa0463740 10 client.0 _ll_register_callbacks cb 0x7f7f840e77c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:25.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7f93fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 0x55ead50dc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:25.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7f8bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d06390 0x55ead50dbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:25.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7f93fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 0x55ead50dc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45224/0 (socket says 172.21.15.67:45224) 2024-07-31T14:53:25.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7f93fff640 1 -- 172.21.15.67:0/238535379 learned_addr learned my addr 172.21.15.67:0/238535379 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:25.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7f937fe640 1 --2- 172.21.15.67:0/238535379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 0x55ead50e07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:25.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7f8bfff640 1 -- 172.21.15.67:0/238535379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 msgr2=0x55ead50e07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:25.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7f8bfff640 1 --2- 172.21.15.67:0/238535379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 0x55ead50e07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7f8bfff640 1 -- 172.21.15.67:0/238535379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 msgr2=0x55ead50dc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:25.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7f8bfff640 1 --2- 172.21.15.67:0/238535379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 0x55ead50dc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7f8bfff640 1 -- 172.21.15.67:0/238535379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ead50dcad0 con 0x55ead4d06390 2024-07-31T14:53:25.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7f8bfff640 1 --2- 172.21.15.67:0/238535379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d06390 0x55ead50dbcc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55ead4fd4a20 tx=0x7f7f7c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e145e61be2fdf3f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:25.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7f72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:53:25.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.010+0000 7f7f72ffd640 1 -- 172.21.15.67:0/238535379 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c038060 con 0x55ead4d06390 2024-07-31T14:53:25.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.011+0000 7f7fa0463740 1 -- 172.21.15.67:0/238535379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7c003c40 con 0x55ead4d06390 2024-07-31T14:53:25.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.011+0000 7f7f72ffd640 1 -- 172.21.15.67:0/238535379 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7f7f7c0035f0 con 0x55ead4d06390 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.011+0000 7f7f72ffd640 1 -- 172.21.15.67:0/238535379 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7f7c0037f0 con 0x55ead4d06390 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.011+0000 7f7fa0463740 10 client.25937 fetch_fsmap learned FSMap version 32 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.011+0000 7f7fa0463740 1 -- 172.21.15.67:0/238535379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ead5152a50 con 0x55ead4d06390 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.011+0000 7f7f72ffd640 1 -- 172.21.15.67:0/238535379 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f7c003c40 con 0x55ead4d06390 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.011+0000 7f7f72ffd640 1 -- 172.21.15.67:0/238535379 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f7f7c024410 con 0x55ead4d06390 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.011+0000 7f7fa0463740 10 client.25937 fetch_fsmap finished waiting for FSMap version 32 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.011+0000 7f7fa0463740 10 client.25937 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.011+0000 7f7fa0463740 20 client.25937 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.011+0000 7f7fa0463740 1 --2- 172.21.15.67:0/238535379 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ead4cbada0 0x55ead5154f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.011+0000 7f7fa0463740 4 client.25937 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.011+0000 7f7fa0463740 1 -- 172.21.15.67:0/238535379 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ead50405a0 con 0x55ead4cbada0 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.011+0000 7f7f937fe640 1 --2- 172.21.15.67:0/238535379 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ead4cbada0 0x55ead5154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7f72ffd640 10 client.25937 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7f937fe640 1 --2- 172.21.15.67:0/238535379 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ead4cbada0 0x55ead5154f90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dfe29206283aff85 server_cookie=a93723a7a131bfae in_seq=0 out_seq=0 2024-07-31T14:53:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7f72ffd640 1 -- 172.21.15.67:0/238535379 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ead50405a0 con 0x55ead4cbada0 2024-07-31T14:53:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7f72ffd640 10 client.25937 handle_command_reply: tid=0 2024-07-31T14:53:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7fa0463740 1 client.25937 shutdown 2024-07-31T14:53:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7fa0463740 1 -- 172.21.15.67:0/238535379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d06390 msgr2=0x55ead50dbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7fa0463740 1 --2- 172.21.15.67:0/238535379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d06390 0x55ead50dbcc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55ead4fd4a20 tx=0x7f7f7c004040 comp rx=0 tx=0).stop 2024-07-31T14:53:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7fa0463740 1 -- 172.21.15.67:0/238535379 shutdown_connections 2024-07-31T14:53:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7fa0463740 1 --2- 172.21.15.67:0/238535379 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ead4cbada0 0x55ead5154f90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7f72ffd640 0 client.25937 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:53:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7fa0463740 1 --2- 172.21.15.67:0/238535379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ead4d27070 0x55ead50dc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7fa0463740 1 --2- 172.21.15.67:0/238535379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ead4d06390 0x55ead50dbcc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7fa0463740 1 --2- 172.21.15.67:0/238535379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ead4cfe1b0 0x55ead50e07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7fa0463740 1 -- 172.21.15.67:0/238535379 >> 172.21.15.67:0/238535379 conn(0x55ead4d1b460 msgr2=0x55ead50cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7fa0463740 1 -- 172.21.15.67:0/238535379 shutdown_connections 2024-07-31T14:53:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.012+0000 7f7fa0463740 1 -- 172.21.15.67:0/238535379 wait complete. 2024-07-31T14:53:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.013+0000 7f7fa0463740 20 client.25937 trim_cache size 0 max 16384 2024-07-31T14:53:25.022 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:53:25.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.013+0000 7f7f99f12640 1 -- 172.21.15.67:0/3568676481 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7f800b3dc0 msgr2=0x7f7f800b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:25.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.013+0000 7f7f99f12640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7f800b3dc0 0x7f7f800b62d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7f94104280 tx=0x7f7f8c032ed0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.013+0000 7f7f99f12640 1 -- 172.21.15.67:0/3568676481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f941042b0 msgr2=0x7f7f94103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:25.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.013+0000 7f7f99f12640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f941042b0 0x7f7f94103bd0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400d950 tx=0x7f7f8400de10 comp rx=0 tx=0).stop 2024-07-31T14:53:25.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.013+0000 7f7f99f12640 1 -- 172.21.15.67:0/3568676481 shutdown_connections 2024-07-31T14:53:25.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.013+0000 7f7f99f12640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7f800b3dc0 0x7f7f800b62d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.013+0000 7f7f99f12640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f9410d860 0x7f7f94102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.013+0000 7f7f99f12640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f94104ce0 0x7f7f941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.013+0000 7f7f99f12640 1 --2- 172.21.15.67:0/3568676481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f941042b0 0x7f7f94103bd0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:25.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.013+0000 7f7f99f12640 1 -- 172.21.15.67:0/3568676481 >> 172.21.15.67:0/3568676481 conn(0x7f7f94079160 msgr2=0x7f7f94107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:25.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.013+0000 7f7f99f12640 1 -- 172.21.15.67:0/3568676481 shutdown_connections 2024-07-31T14:53:25.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:25.013+0000 7f7f99f12640 1 -- 172.21.15.67:0/3568676481 wait complete. 2024-07-31T14:53:25.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:24 smithi067 ceph-mon[32750]: pgmap v469: 129 pgs: 129 active+clean; 146 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 5.1 MiB/s wr, 303 op/s 2024-07-31T14:53:25.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:24 smithi077 ceph-mon[37327]: pgmap v469: 129 pgs: 129 active+clean; 146 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 5.1 MiB/s wr, 303 op/s 2024-07-31T14:53:25.393 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:24 smithi073 podman[70326]: 2024-07-31 14:53:24.946906261 +0000 UTC m=+0.416522862 container start d2b5d61a47c8fcabd761759224b6df40abfddd34f326ce81debf0d79dbfc294d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-07-31T14:53:25.393 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:24 smithi073 ceph-mds[70345]: -- 172.21.15.73:0/3834350935 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562ed5751860 con 0x562ed98f3600 2024-07-31T14:53:25.393 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:24 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[70341]: starting mds.e at 2024-07-31T14:53:25.393 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:25 smithi073 bash[70326]: d2b5d61a47c8fcabd761759224b6df40abfddd34f326ce81debf0d79dbfc294d 2024-07-31T14:53:25.393 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 14:53:25 smithi073 systemd[1]: Started Ceph mds.e for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T14:53:25.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T14:53:26.198 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:53:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.200+0000 7f00e52e3640 1 -- 172.21.15.67:0/3529503519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e01074e0 msgr2=0x7f00e01078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.200+0000 7f00e52e3640 1 --2- 172.21.15.67:0/3529503519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e01074e0 0x7f00e01078f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f00d400a970 tx=0x7f00d402b670 comp rx=0 tx=0).stop 2024-07-31T14:53:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.201+0000 7f00e52e3640 1 -- 172.21.15.67:0/3529503519 shutdown_connections 2024-07-31T14:53:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.201+0000 7f00e52e3640 1 --2- 172.21.15.67:0/3529503519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0107e30 0x7f00e010e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.201+0000 7f00e52e3640 1 --2- 172.21.15.67:0/3529503519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e01074e0 0x7f00e01078f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.201+0000 7f00e52e3640 1 --2- 172.21.15.67:0/3529503519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00e0101bb0 0x7f00e0101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.201+0000 7f00e52e3640 1 -- 172.21.15.67:0/3529503519 >> 172.21.15.67:0/3529503519 conn(0x7f00e0079880 msgr2=0x7f00e0079c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.201+0000 7f00e52e3640 1 -- 172.21.15.67:0/3529503519 shutdown_connections 2024-07-31T14:53:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.201+0000 7f00e52e3640 1 -- 172.21.15.67:0/3529503519 wait complete. 2024-07-31T14:53:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.201+0000 7f00e52e3640 1 Processor -- start 2024-07-31T14:53:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.202+0000 7f00e52e3640 1 -- start start 2024-07-31T14:53:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.202+0000 7f00e52e3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e0101bb0 0x7f00e0197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.202+0000 7f00e52e3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00e01074e0 0x7f00e0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.202+0000 7f00e52e3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0107e30 0x7f00e019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.202+0000 7f00e52e3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00e0111270 con 0x7f00e01074e0 2024-07-31T14:53:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.202+0000 7f00e52e3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01110f0 con 0x7f00e0101bb0 2024-07-31T14:53:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.202+0000 7f00e52e3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01113f0 con 0x7f00e0107e30 2024-07-31T14:53:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.202+0000 7f00deffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e0101bb0 0x7f00e0197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.202+0000 7f00de7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00e01074e0 0x7f00e0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.202+0000 7f00df7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0107e30 0x7f00e019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.202+0000 7f00de7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00e01074e0 0x7f00e0198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45236/0 (socket says 172.21.15.67:45236) 2024-07-31T14:53:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.202+0000 7f00de7fc640 1 -- 172.21.15.67:0/705273890 learned_addr learned my addr 172.21.15.67:0/705273890 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.202+0000 7f00deffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e0101bb0 0x7f00e0197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37020/0 (socket says 172.21.15.67:37020) 2024-07-31T14:53:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.203+0000 7f00deffd640 1 -- 172.21.15.67:0/705273890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0107e30 msgr2=0x7f00e019d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.203+0000 7f00deffd640 1 --2- 172.21.15.67:0/705273890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0107e30 0x7f00e019d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.203+0000 7f00deffd640 1 -- 172.21.15.67:0/705273890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00e01074e0 msgr2=0x7f00e0198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.203+0000 7f00deffd640 1 --2- 172.21.15.67:0/705273890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00e01074e0 0x7f00e0198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.203+0000 7f00deffd640 1 -- 172.21.15.67:0/705273890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e019dbc0 con 0x7f00e0101bb0 2024-07-31T14:53:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.203+0000 7f00deffd640 1 --2- 172.21.15.67:0/705273890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e0101bb0 0x7f00e0197c90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f00d0005fd0 tx=0x7f00d000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.203+0000 7f00c7fff640 1 -- 172.21.15.67:0/705273890 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d000cd00 con 0x7f00e0101bb0 2024-07-31T14:53:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.203+0000 7f00e52e3640 1 -- 172.21.15.67:0/705273890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e019deb0 con 0x7f00e0101bb0 2024-07-31T14:53:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.203+0000 7f00c7fff640 1 -- 172.21.15.67:0/705273890 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d00035f0 con 0x7f00e0101bb0 2024-07-31T14:53:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.203+0000 7f00c7fff640 1 -- 172.21.15.67:0/705273890 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0010690 con 0x7f00e0101bb0 2024-07-31T14:53:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.203+0000 7f00e52e3640 1 -- 172.21.15.67:0/705273890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e01a4640 con 0x7f00e0101bb0 2024-07-31T14:53:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.206+0000 7f00ec1d1740 1 Processor -- start 2024-07-31T14:53:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.206+0000 7f00c7fff640 1 -- 172.21.15.67:0/705273890 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f00d0010830 con 0x7f00e0101bb0 2024-07-31T14:53:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00ec1d1740 1 -- start start 2024-07-31T14:53:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00ec1d1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b555081b0 0x559b558aace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00ec1d1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00ec1d1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b554e74d0 0x559b558ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00ec1d1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559b558203b0 con 0x559b554e74d0 2024-07-31T14:53:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00ec1d1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559b5580f8b0 con 0x559b554df2f0 2024-07-31T14:53:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00ec1d1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559b558138f0 con 0x559b555081b0 2024-07-31T14:53:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00deffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00de7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b554e74d0 0x559b558ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00deffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558ab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37022/0 (socket says 172.21.15.67:37022) 2024-07-31T14:53:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00deffd640 1 -- 172.21.15.67:0/662097490 learned_addr learned my addr 172.21.15.67:0/662097490 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00df7fe640 1 --2- 172.21.15.67:0/662097490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b555081b0 0x559b558aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00deffd640 1 -- 172.21.15.67:0/662097490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b555081b0 msgr2=0x559b558aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00deffd640 1 --2- 172.21.15.67:0/662097490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b555081b0 0x559b558aace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00deffd640 1 -- 172.21.15.67:0/662097490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b554e74d0 msgr2=0x559b558ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00deffd640 1 --2- 172.21.15.67:0/662097490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b554e74d0 0x559b558ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00c7fff640 1 --2- 172.21.15.67:0/705273890 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f00c80b3d40 0x7f00c80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00deffd640 1 -- 172.21.15.67:0/662097490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b557b69a0 con 0x559b554df2f0 2024-07-31T14:53:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.207+0000 7f00df7fe640 1 --2- 172.21.15.67:0/662097490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b555081b0 0x559b558aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.208+0000 7f00df7fe640 1 --2- 172.21.15.67:0/705273890 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f00c80b3d40 0x7f00c80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.208+0000 7f00c7fff640 1 -- 172.21.15.67:0/705273890 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f00d00e4550 con 0x7f00e0101bb0 2024-07-31T14:53:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.208+0000 7f00deffd640 1 --2- 172.21.15.67:0/662097490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558ab220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f00d00e18d0 tx=0x7f00d00a7560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b6617dca9bb67d4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.208+0000 7f00c57fa640 1 -- 172.21.15.67:0/662097490 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d00aac20 con 0x559b554df2f0 2024-07-31T14:53:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.208+0000 7f00c57fa640 1 -- 172.21.15.67:0/662097490 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d00bd070 con 0x559b554df2f0 2024-07-31T14:53:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.208+0000 7f00c57fa640 1 -- 172.21.15.67:0/662097490 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d00aa5c0 con 0x559b554df2f0 2024-07-31T14:53:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.209+0000 7f00ec1d1740 1 -- 172.21.15.67:0/662097490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 msgr2=0x559b558ab220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.209+0000 7f00df7fe640 1 --2- 172.21.15.67:0/705273890 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f00c80b3d40 0x7f00c80b6250 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f00d8005fd0 tx=0x7f00d8011040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.209+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/662097490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558ab220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f00d00e18d0 tx=0x7f00d00a7560 comp rx=0 tx=0).stop 2024-07-31T14:53:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.209+0000 7f00ec1d1740 1 -- 172.21.15.67:0/662097490 shutdown_connections 2024-07-31T14:53:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.209+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/662097490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b554e74d0 0x559b558ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.209+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/662097490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558ab220 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.209+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/662097490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b555081b0 0x559b558aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.209+0000 7f00ec1d1740 1 -- 172.21.15.67:0/662097490 >> 172.21.15.67:0/662097490 conn(0x559b554fc300 msgr2=0x559b558a8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.209+0000 7f00ec1d1740 1 -- 172.21.15.67:0/662097490 shutdown_connections 2024-07-31T14:53:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.209+0000 7f00ec1d1740 1 -- 172.21.15.67:0/662097490 wait complete. 2024-07-31T14:53:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.210+0000 7f00ec1d1740 1 -- start start 2024-07-31T14:53:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.210+0000 7f00ec1d1740 1 Processor -- start 2024-07-31T14:53:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.210+0000 7f00ec1d1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b554df2f0 0x559b558b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.210+0000 7f00ec1d1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554e74d0 0x559b558b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.210+0000 7f00ec1d1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558c2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.210+0000 7f00ec1d1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559b55749ab0 con 0x559b555081b0 2024-07-31T14:53:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.210+0000 7f00ec1d1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559b55797cf0 con 0x559b554e74d0 2024-07-31T14:53:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.210+0000 7f00ec1d1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559b558b9460 con 0x559b554df2f0 2024-07-31T14:53:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.210+0000 7f00ec1d1740 10 client.0 _ll_register_callbacks cb 0x559b558099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.210+0000 7f00de7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554e74d0 0x559b558b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.210+0000 7f00df7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558c2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.210+0000 7f00de7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554e74d0 0x559b558b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37028/0 (socket says 172.21.15.67:37028) 2024-07-31T14:53:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.210+0000 7f00de7fc640 1 -- 172.21.15.67:0/2124198105 learned_addr learned my addr 172.21.15.67:0/2124198105 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.211+0000 7f00deffd640 1 --2- 172.21.15.67:0/2124198105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b554df2f0 0x559b558b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.211+0000 7f00de7fc640 1 -- 172.21.15.67:0/2124198105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b554df2f0 msgr2=0x559b558b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.211+0000 7f00de7fc640 1 --2- 172.21.15.67:0/2124198105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b554df2f0 0x559b558b67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.211+0000 7f00de7fc640 1 -- 172.21.15.67:0/2124198105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 msgr2=0x559b558c2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.211+0000 7f00de7fc640 1 --2- 172.21.15.67:0/2124198105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558c2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.211+0000 7f00de7fc640 1 -- 172.21.15.67:0/2124198105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b55573790 con 0x559b554e74d0 2024-07-31T14:53:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.211+0000 7f00df7fe640 1 --2- 172.21.15.67:0/2124198105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558c2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.211+0000 7f00de7fc640 1 --2- 172.21.15.67:0/2124198105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554e74d0 0x559b558b6d20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f00d402bb90 tx=0x7f00d4032630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b6551c444239b3e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.211+0000 7f00c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:53:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.211+0000 7f00c2ff5640 1 -- 172.21.15.67:0/2124198105 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4032930 con 0x559b554e74d0 2024-07-31T14:53:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.211+0000 7f00c2ff5640 1 -- 172.21.15.67:0/2124198105 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f00d4005240 con 0x559b554e74d0 2024-07-31T14:53:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.211+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2124198105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d00bd070 con 0x559b554e74d0 2024-07-31T14:53:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.212+0000 7f00c2ff5640 1 -- 172.21.15.67:0/2124198105 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f00d4051050 con 0x559b554e74d0 2024-07-31T14:53:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.212+0000 7f00ec1d1740 10 client.25949 fetch_fsmap learned FSMap version 34 2024-07-31T14:53:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.212+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2124198105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559b555ec620 con 0x559b554e74d0 2024-07-31T14:53:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.212+0000 7f00c2ff5640 1 -- 172.21.15.67:0/2124198105 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d00bd070 con 0x559b554e74d0 2024-07-31T14:53:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.212+0000 7f00c2ff5640 1 -- 172.21.15.67:0/2124198105 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f00d4046020 con 0x559b554e74d0 2024-07-31T14:53:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.212+0000 7f00ec1d1740 10 client.25949 fetch_fsmap finished waiting for FSMap version 34 2024-07-31T14:53:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.212+0000 7f00ec1d1740 10 client.25949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.212+0000 7f00ec1d1740 20 client.25949 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.212+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2124198105 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559b554c6d80 0x559b558c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.212+0000 7f00ec1d1740 4 client.25949 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:53:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.212+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2124198105 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b55822300 con 0x559b554c6d80 2024-07-31T14:53:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.212+0000 7f00deffd640 1 --2- 172.21.15.67:0/2124198105 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559b554c6d80 0x559b558c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.213+0000 7f00c2ff5640 10 client.25949 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.213+0000 7f00deffd640 1 --2- 172.21.15.67:0/2124198105 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559b554c6d80 0x559b558c87e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3673be1177b8a0b6 server_cookie=249ea994c6a4ab49 in_seq=0 out_seq=0 2024-07-31T14:53:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.213+0000 7f00c2ff5640 1 -- 172.21.15.67:0/2124198105 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b55822300 con 0x559b554c6d80 2024-07-31T14:53:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.213+0000 7f00c2ff5640 10 client.25949 handle_command_reply: tid=0 2024-07-31T14:53:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.213+0000 7f00ec1d1740 1 client.25949 shutdown 2024-07-31T14:53:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.214+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2124198105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554e74d0 msgr2=0x559b558b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.214+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2124198105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554e74d0 0x559b558b6d20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f00d402bb90 tx=0x7f00d4032630 comp rx=0 tx=0).stop 2024-07-31T14:53:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.214+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2124198105 shutdown_connections 2024-07-31T14:53:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.214+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2124198105 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559b554c6d80 0x559b558c87e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.214+0000 7f00c2ff5640 0 client.25949 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.214+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2124198105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558c2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.214+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2124198105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554e74d0 0x559b558b6d20 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f00d402bb90 tx=0x7f00d4032630 comp rx=0 tx=0).stop 2024-07-31T14:53:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.214+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2124198105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b554df2f0 0x559b558b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.214+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2124198105 >> 172.21.15.67:0/2124198105 conn(0x559b554fc300 msgr2=0x559b558abd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.214+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2124198105 shutdown_connections 2024-07-31T14:53:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.214+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2124198105 wait complete. 2024-07-31T14:53:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.214+0000 7f00ec1d1740 20 client.25949 trim_cache size 0 max 16384 2024-07-31T14:53:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.230+0000 7f00ec1d1740 1 Processor -- start 2024-07-31T14:53:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.231+0000 7f00ec1d1740 1 -- start start 2024-07-31T14:53:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.231+0000 7f00ec1d1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.231+0000 7f00ec1d1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.231+0000 7f00ec1d1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b5549c1c0 0x559b558c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.231+0000 7f00ec1d1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01ae4b0 con 0x559b555081b0 2024-07-31T14:53:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.231+0000 7f00ec1d1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b0080 con 0x559b554df2f0 2024-07-31T14:53:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.231+0000 7f00ec1d1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01aeb80 con 0x559b5549c1c0 2024-07-31T14:53:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.231+0000 7f00de7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.231+0000 7f00deffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.231+0000 7f00de7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45270/0 (socket says 172.21.15.67:45270) 2024-07-31T14:53:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.231+0000 7f00deffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558a8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37038/0 (socket says 172.21.15.67:37038) 2024-07-31T14:53:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.231+0000 7f00de7fc640 1 -- 172.21.15.67:0/2197720549 learned_addr learned my addr 172.21.15.67:0/2197720549 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.231+0000 7f00df7fe640 1 --2- 172.21.15.67:0/2197720549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b5549c1c0 0x559b558c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.232+0000 7f00de7fc640 1 -- 172.21.15.67:0/2197720549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b5549c1c0 msgr2=0x559b558c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.232+0000 7f00de7fc640 1 --2- 172.21.15.67:0/2197720549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b5549c1c0 0x559b558c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.232+0000 7f00de7fc640 1 -- 172.21.15.67:0/2197720549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 msgr2=0x559b558a8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.232+0000 7f00de7fc640 1 --2- 172.21.15.67:0/2197720549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558a8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.232+0000 7f00de7fc640 1 -- 172.21.15.67:0/2197720549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b558b6350 con 0x559b555081b0 2024-07-31T14:53:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.232+0000 7f00de7fc640 1 --2- 172.21.15.67:0/2197720549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558b0db0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f00d4032f00 tx=0x7f00d402b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db82b07854181dfc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.232+0000 7f00c4ff9640 1 -- 172.21.15.67:0/2197720549 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d405a070 con 0x559b555081b0 2024-07-31T14:53:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.232+0000 7f00c4ff9640 1 -- 172.21.15.67:0/2197720549 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d4031d30 con 0x559b555081b0 2024-07-31T14:53:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.232+0000 7f00c4ff9640 1 -- 172.21.15.67:0/2197720549 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d40052d0 con 0x559b555081b0 2024-07-31T14:53:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.233+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2197720549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 msgr2=0x559b558b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.233+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2197720549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558b0db0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f00d4032f00 tx=0x7f00d402b670 comp rx=0 tx=0).stop 2024-07-31T14:53:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.233+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2197720549 shutdown_connections 2024-07-31T14:53:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.234+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2197720549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b5549c1c0 0x559b558c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.234+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2197720549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558b0db0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.234+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2197720549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.234+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2197720549 >> 172.21.15.67:0/2197720549 conn(0x559b554fc300 msgr2=0x559b558a84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.234+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2197720549 shutdown_connections 2024-07-31T14:53:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.234+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2197720549 wait complete. 2024-07-31T14:53:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.234+0000 7f00ec1d1740 1 -- start start 2024-07-31T14:53:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.234+0000 7f00ec1d1740 1 Processor -- start 2024-07-31T14:53:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00ec1d1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b5549c1c0 0x559b558bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00ec1d1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00ec1d1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558bbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00ec1d1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01af9b0 con 0x559b555081b0 2024-07-31T14:53:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00ec1d1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01af250 con 0x559b554df2f0 2024-07-31T14:53:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00ec1d1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01ac490 con 0x559b5549c1c0 2024-07-31T14:53:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00ec1d1740 10 client.0 _ll_register_callbacks cb 0x7f00d400a5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00df7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558bbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00df7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558bbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45276/0 (socket says 172.21.15.67:45276) 2024-07-31T14:53:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00df7fe640 1 -- 172.21.15.67:0/2646887534 learned_addr learned my addr 172.21.15.67:0/2646887534 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00de7fc640 1 --2- 172.21.15.67:0/2646887534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00deffd640 1 --2- 172.21.15.67:0/2646887534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b5549c1c0 0x559b558bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00df7fe640 1 -- 172.21.15.67:0/2646887534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b5549c1c0 msgr2=0x559b558bed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00df7fe640 1 --2- 172.21.15.67:0/2646887534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b5549c1c0 0x559b558bed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00df7fe640 1 -- 172.21.15.67:0/2646887534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 msgr2=0x559b558bb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00df7fe640 1 --2- 172.21.15.67:0/2646887534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558bb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00df7fe640 1 -- 172.21.15.67:0/2646887534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b558bc330 con 0x559b555081b0 2024-07-31T14:53:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00de7fc640 1 --2- 172.21.15.67:0/2646887534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558bb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00df7fe640 1 --2- 172.21.15.67:0/2646887534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558bbbf0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f00d801a4d0 tx=0x7f00d8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22be067f486da03 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.235+0000 7f00c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.236+0000 7f00c2ff5640 1 -- 172.21.15.67:0/2646887534 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d802c070 con 0x559b555081b0 2024-07-31T14:53:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.236+0000 7f00c2ff5640 1 -- 172.21.15.67:0/2646887534 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f00d8028650 con 0x559b555081b0 2024-07-31T14:53:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.236+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2646887534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d4031d30 con 0x559b555081b0 2024-07-31T14:53:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.236+0000 7f00c2ff5640 1 -- 172.21.15.67:0/2646887534 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f00d8038050 con 0x559b555081b0 2024-07-31T14:53:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.236+0000 7f00ec1d1740 10 client.16329 fetch_fsmap learned FSMap version 34 2024-07-31T14:53:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.236+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2646887534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559b5592fed0 con 0x559b555081b0 2024-07-31T14:53:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.236+0000 7f00c2ff5640 1 -- 172.21.15.67:0/2646887534 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d4031d30 con 0x559b555081b0 2024-07-31T14:53:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.236+0000 7f00c2ff5640 1 -- 172.21.15.67:0/2646887534 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f00d8019980 con 0x559b555081b0 2024-07-31T14:53:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.236+0000 7f00ec1d1740 10 client.16329 fetch_fsmap finished waiting for FSMap version 34 2024-07-31T14:53:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.236+0000 7f00ec1d1740 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.236+0000 7f00ec1d1740 20 client.16329 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.236+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2646887534 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559b55930740 0x559b55932b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.236+0000 7f00ec1d1740 4 client.16329 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:53:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.236+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2646887534 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b5581dda0 con 0x559b55930740 2024-07-31T14:53:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.237+0000 7f00deffd640 1 --2- 172.21.15.67:0/2646887534 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559b55930740 0x559b55932b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.237+0000 7f00c2ff5640 10 client.16329 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:26.237+0000 7f00deffd640 1 --2- 172.21.15.67:0/2646887534 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559b55930740 0x559b55932b90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28cc28dc43ee9ab server_cookie=9f83d32e913159f in_seq=0 out_seq=0 2024-07-31T14:53:26.245 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:25 smithi077 ceph-mon[37327]: daemon mds.e restarted 2024-07-31T14:53:26.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:25 smithi073 ceph-mon[36714]: daemon mds.e restarted 2024-07-31T14:53:26.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:25 smithi067 ceph-mon[32750]: daemon mds.e restarted 2024-07-31T14:53:27.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:26 smithi077 ceph-mon[37327]: pgmap v470: 129 pgs: 129 active+clean; 148 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.3 MiB/s wr, 252 op/s 2024-07-31T14:53:27.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:26 smithi077 ceph-mon[37327]: osdmap e90: 12 total, 12 up, 12 in 2024-07-31T14:53:27.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:26 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.73:6834/773662454,v1:172.21.15.73:6835/773662454] up:boot 2024-07-31T14:53:27.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:26 smithi077 ceph-mon[37327]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-07-31T14:53:27.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T14:53:27.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:26 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:27.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:26 smithi067 ceph-mon[32750]: pgmap v470: 129 pgs: 129 active+clean; 148 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.3 MiB/s wr, 252 op/s 2024-07-31T14:53:27.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:26 smithi067 ceph-mon[32750]: osdmap e90: 12 total, 12 up, 12 in 2024-07-31T14:53:27.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:26 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.73:6834/773662454,v1:172.21.15.73:6835/773662454] up:boot 2024-07-31T14:53:27.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:26 smithi067 ceph-mon[32750]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-07-31T14:53:27.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T14:53:27.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:26 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:27.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:26 smithi073 ceph-mon[36714]: pgmap v470: 129 pgs: 129 active+clean; 148 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.3 MiB/s wr, 252 op/s 2024-07-31T14:53:27.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:26 smithi073 ceph-mon[36714]: osdmap e90: 12 total, 12 up, 12 in 2024-07-31T14:53:27.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:26 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.73:6834/773662454,v1:172.21.15.73:6835/773662454] up:boot 2024-07-31T14:53:27.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:26 smithi073 ceph-mon[36714]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-07-31T14:53:27.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T14:53:27.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:26 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:27.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.984+0000 7f1159c31640 1 -- 172.21.15.67:0/24716179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1154108720 msgr2=0x7f115410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:27.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.984+0000 7f1159c31640 1 --2- 172.21.15.67:0/24716179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1154108720 0x7f115410ef50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f114c009ca0 tx=0x7f114c02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:53:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.984+0000 7f1159c31640 1 -- 172.21.15.67:0/24716179 shutdown_connections 2024-07-31T14:53:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.984+0000 7f1159c31640 1 --2- 172.21.15.67:0/24716179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1154108720 0x7f115410ef50 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.984+0000 7f1159c31640 1 --2- 172.21.15.67:0/24716179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1154107dd0 0x7f11541081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.984+0000 7f1159c31640 1 --2- 172.21.15.67:0/24716179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1154102490 0x7f11541028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.984+0000 7f1159c31640 1 -- 172.21.15.67:0/24716179 >> 172.21.15.67:0/24716179 conn(0x7f1154079160 msgr2=0x7f1154079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.984+0000 7f1159c31640 1 -- 172.21.15.67:0/24716179 shutdown_connections 2024-07-31T14:53:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.984+0000 7f1159c31640 1 -- 172.21.15.67:0/24716179 wait complete. 2024-07-31T14:53:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.985+0000 7f1159c31640 1 Processor -- start 2024-07-31T14:53:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.985+0000 7f1159c31640 1 -- start start 2024-07-31T14:53:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.985+0000 7f1159c31640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1154102490 0x7f11540ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.985+0000 7f1159c31640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1154107dd0 0x7f11540ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.985+0000 7f1159c31640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1154108720 0x7f11540ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.985+0000 7f1159c31640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1154112380 con 0x7f1154102490 2024-07-31T14:53:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.985+0000 7f1159c31640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1154112200 con 0x7f1154107dd0 2024-07-31T14:53:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.985+0000 7f1159c31640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1154112500 con 0x7f1154108720 2024-07-31T14:53:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.986+0000 7f11537fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1154102490 0x7f11540ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.986+0000 7f11537fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1154102490 0x7f11540ff0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45292/0 (socket says 172.21.15.67:45292) 2024-07-31T14:53:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.986+0000 7f11537fe640 1 -- 172.21.15.67:0/3379829878 learned_addr learned my addr 172.21.15.67:0/3379829878 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.986+0000 7f1152ffd640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1154107dd0 0x7f11540ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.986+0000 7f1153fff640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1154108720 0x7f11540ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.986+0000 7f11537fe640 1 -- 172.21.15.67:0/3379829878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1154108720 msgr2=0x7f11540ffb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.986+0000 7f11537fe640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1154108720 0x7f11540ffb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.986+0000 7f11537fe640 1 -- 172.21.15.67:0/3379829878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1154107dd0 msgr2=0x7f11540ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.986+0000 7f11537fe640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1154107dd0 0x7f11540ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.986+0000 7f11537fe640 1 -- 172.21.15.67:0/3379829878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11541a8160 con 0x7f1154102490 2024-07-31T14:53:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.986+0000 7f1152ffd640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1154107dd0 0x7f11540ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.986+0000 7f11537fe640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1154102490 0x7f11540ff0a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f114400d950 tx=0x7f114400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.987+0000 7f1150ff9640 1 -- 172.21.15.67:0/3379829878 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144017070 con 0x7f1154102490 2024-07-31T14:53:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.987+0000 7f1159c31640 1 -- 172.21.15.67:0/3379829878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11541a8450 con 0x7f1154102490 2024-07-31T14:53:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.987+0000 7f1150ff9640 1 -- 172.21.15.67:0/3379829878 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1144009c30 con 0x7f1154102490 2024-07-31T14:53:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.987+0000 7f1150ff9640 1 -- 172.21.15.67:0/3379829878 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114400f620 con 0x7f1154102490 2024-07-31T14:53:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.987+0000 7f1159c31640 1 -- 172.21.15.67:0/3379829878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11541a89e0 con 0x7f1154102490 2024-07-31T14:53:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.990+0000 7f1160182740 1 Processor -- start 2024-07-31T14:53:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.990+0000 7f1160182740 1 -- start start 2024-07-31T14:53:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.990+0000 7f1160182740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed9546070 0x55fed98eb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.990+0000 7f1160182740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed951d1b0 0x55fed98ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.990+0000 7f1160182740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed9525390 0x55fed98ebff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.990+0000 7f1160182740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fed9861bb0 con 0x55fed9546070 2024-07-31T14:53:27.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.990+0000 7f1160182740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fed98510b0 con 0x55fed9525390 2024-07-31T14:53:27.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.990+0000 7f1160182740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fed98550f0 con 0x55fed951d1b0 2024-07-31T14:53:27.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.990+0000 7f1153fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed9546070 0x55fed98eb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:27.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.990+0000 7f1153fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed9546070 0x55fed98eb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45294/0 (socket says 172.21.15.67:45294) 2024-07-31T14:53:27.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.990+0000 7f1153fff640 1 -- 172.21.15.67:0/1945399049 learned_addr learned my addr 172.21.15.67:0/1945399049 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:27.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.990+0000 7f1152ffd640 1 --2- 172.21.15.67:0/1945399049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed9525390 0x55fed98ebff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:27.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.991+0000 7f1153fff640 1 -- 172.21.15.67:0/1945399049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed951d1b0 msgr2=0x55fed98ebab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:53:27.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.991+0000 7f1153fff640 1 --2- 172.21.15.67:0/1945399049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed951d1b0 0x55fed98ebab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:27.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.991+0000 7f1153fff640 1 -- 172.21.15.67:0/1945399049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed9525390 msgr2=0x55fed98ebff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.991+0000 7f1153fff640 1 --2- 172.21.15.67:0/1945399049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed9525390 0x55fed98ebff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.991+0000 7f1153fff640 1 -- 172.21.15.67:0/1945399049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed97f7b90 con 0x55fed9546070 2024-07-31T14:53:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.991+0000 7f1150ff9640 1 -- 172.21.15.67:0/3379829878 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1144010460 con 0x7f1154102490 2024-07-31T14:53:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.991+0000 7f1153fff640 1 --2- 172.21.15.67:0/1945399049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed9546070 0x55fed98eb570 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f114c005cc0 tx=0x7f114c02f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc9a61c584c508c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.991+0000 7f1149ffb640 1 -- 172.21.15.67:0/1945399049 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c031cf0 con 0x55fed9546070 2024-07-31T14:53:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.991+0000 7f1149ffb640 1 -- 172.21.15.67:0/1945399049 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114c036040 con 0x55fed9546070 2024-07-31T14:53:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.991+0000 7f1149ffb640 1 -- 172.21.15.67:0/1945399049 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c004040 con 0x55fed9546070 2024-07-31T14:53:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.992+0000 7f1150ff9640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11400b3f40 0x7f11400b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.992+0000 7f11537fe640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11400b3f40 0x7f11400b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.992+0000 7f1150ff9640 1 -- 172.21.15.67:0/3379829878 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f114400f3c0 con 0x7f1154102490 2024-07-31T14:53:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.992+0000 7f1160182740 1 -- 172.21.15.67:0/1945399049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed9546070 msgr2=0x55fed98eb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.992+0000 7f1160182740 1 --2- 172.21.15.67:0/1945399049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed9546070 0x55fed98eb570 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f114c005cc0 tx=0x7f114c02f420 comp rx=0 tx=0).stop 2024-07-31T14:53:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.993+0000 7f1160182740 1 -- 172.21.15.67:0/1945399049 shutdown_connections 2024-07-31T14:53:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.993+0000 7f1160182740 1 --2- 172.21.15.67:0/1945399049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed9525390 0x55fed98ebff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.993+0000 7f1160182740 1 --2- 172.21.15.67:0/1945399049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed951d1b0 0x55fed98ebab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.993+0000 7f1160182740 1 --2- 172.21.15.67:0/1945399049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed9546070 0x55fed98eb570 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.993+0000 7f1160182740 1 -- 172.21.15.67:0/1945399049 >> 172.21.15.67:0/1945399049 conn(0x55fed953a460 msgr2=0x55fed98e9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.993+0000 7f1160182740 1 -- 172.21.15.67:0/1945399049 shutdown_connections 2024-07-31T14:53:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.993+0000 7f11537fe640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11400b3f40 0x7f11400b6450 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55fed98ea1f0 tx=0x7f1144005880 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.993+0000 7f1160182740 1 -- 172.21.15.67:0/1945399049 wait complete. 2024-07-31T14:53:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.993+0000 7f1160182740 1 -- start start 2024-07-31T14:53:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.993+0000 7f1160182740 1 Processor -- start 2024-07-31T14:53:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1160182740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed951d1b0 0x55fed98ff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1160182740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed9525390 0x55fed98ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1160182740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed9546070 0x55fed990b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1160182740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fed96b4700 con 0x55fed9546070 2024-07-31T14:53:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1160182740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fed97f39e0 con 0x55fed951d1b0 2024-07-31T14:53:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1160182740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fed9902420 con 0x55fed9525390 2024-07-31T14:53:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1160182740 10 client.0 _ll_register_callbacks cb 0x55fed980e4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1153fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed9546070 0x55fed990b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1152ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed9525390 0x55fed98ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1153fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed9546070 0x55fed990b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45304/0 (socket says 172.21.15.67:45304) 2024-07-31T14:53:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1153fff640 1 -- 172.21.15.67:0/3091366329 learned_addr learned my addr 172.21.15.67:0/3091366329 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1153fff640 1 --2- 172.21.15.67:0/3091366329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed951d1b0 0x55fed98ff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1152ffd640 1 -- 172.21.15.67:0/3091366329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed951d1b0 msgr2=0x55fed98ff8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1152ffd640 1 --2- 172.21.15.67:0/3091366329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed951d1b0 0x55fed98ff8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1152ffd640 1 -- 172.21.15.67:0/3091366329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed9546070 msgr2=0x55fed990b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1152ffd640 1 --2- 172.21.15.67:0/3091366329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed9546070 0x55fed990b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.994+0000 7f1152ffd640 1 -- 172.21.15.67:0/3091366329 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed95b1650 con 0x55fed9525390 2024-07-31T14:53:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.995+0000 7f1153fff640 1 --2- 172.21.15.67:0/3091366329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed9546070 0x55fed990b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.995+0000 7f1152ffd640 1 --2- 172.21.15.67:0/3091366329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed9525390 0x55fed98ffe20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f115406a4d0 tx=0x7f113c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9630a4044e001c67 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.995+0000 7f1132ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:53:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.995+0000 7f1132ffd640 1 -- 172.21.15.67:0/3091366329 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c012110 con 0x55fed9525390 2024-07-31T14:53:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.995+0000 7f1132ffd640 1 -- 172.21.15.67:0/3091366329 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f113c004050 con 0x55fed9525390 2024-07-31T14:53:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.995+0000 7f1160182740 1 -- 172.21.15.67:0/3091366329 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114c036040 con 0x55fed9525390 2024-07-31T14:53:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.995+0000 7f1132ffd640 1 -- 172.21.15.67:0/3091366329 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f113c0122b0 con 0x55fed9525390 2024-07-31T14:53:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.995+0000 7f1160182740 10 client.26005 fetch_fsmap learned FSMap version 34 2024-07-31T14:53:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.995+0000 7f1160182740 1 -- 172.21.15.67:0/3091366329 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fed990e450 con 0x55fed9525390 2024-07-31T14:53:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.995+0000 7f1132ffd640 1 -- 172.21.15.67:0/3091366329 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114c036040 con 0x55fed9525390 2024-07-31T14:53:28.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.996+0000 7f1132ffd640 1 -- 172.21.15.67:0/3091366329 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f113c010430 con 0x55fed9525390 2024-07-31T14:53:28.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.996+0000 7f1160182740 10 client.26005 fetch_fsmap finished waiting for FSMap version 34 2024-07-31T14:53:28.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.996+0000 7f1160182740 10 client.26005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:28.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.996+0000 7f1160182740 20 client.26005 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:28.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.996+0000 7f1160182740 1 --2- 172.21.15.67:0/3091366329 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fed95049f0 0x55fed9911a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:28.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.996+0000 7f1160182740 4 client.26005 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:53:28.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.996+0000 7f1160182740 1 -- 172.21.15.67:0/3091366329 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fed9863b00 con 0x55fed95049f0 2024-07-31T14:53:28.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.996+0000 7f1153fff640 1 --2- 172.21.15.67:0/3091366329 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fed95049f0 0x55fed9911a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:28.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.996+0000 7f1132ffd640 10 client.26005 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:28.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.997+0000 7f1153fff640 1 --2- 172.21.15.67:0/3091366329 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fed95049f0 0x55fed9911a90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5deb6f5520085627 server_cookie=ad77104211bf3833 in_seq=0 out_seq=0 2024-07-31T14:53:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.997+0000 7f1132ffd640 1 -- 172.21.15.67:0/3091366329 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fed9863b00 con 0x55fed95049f0 2024-07-31T14:53:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.997+0000 7f1132ffd640 10 client.26005 handle_command_reply: tid=0 2024-07-31T14:53:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.997+0000 7f1160182740 1 client.26005 shutdown 2024-07-31T14:53:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.998+0000 7f1160182740 1 -- 172.21.15.67:0/3091366329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed9525390 msgr2=0x55fed98ffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.998+0000 7f1160182740 1 --2- 172.21.15.67:0/3091366329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed9525390 0x55fed98ffe20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f115406a4d0 tx=0x7f113c00f790 comp rx=0 tx=0).stop 2024-07-31T14:53:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.998+0000 7f1160182740 1 -- 172.21.15.67:0/3091366329 shutdown_connections 2024-07-31T14:53:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.998+0000 7f1160182740 1 --2- 172.21.15.67:0/3091366329 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fed95049f0 0x55fed9911a90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.998+0000 7f1160182740 1 --2- 172.21.15.67:0/3091366329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed9546070 0x55fed990b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.998+0000 7f1132ffd640 0 client.26005 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.998+0000 7f1160182740 1 --2- 172.21.15.67:0/3091366329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed9525390 0x55fed98ffe20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.998+0000 7f1160182740 1 --2- 172.21.15.67:0/3091366329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed951d1b0 0x55fed98ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.998+0000 7f1160182740 1 -- 172.21.15.67:0/3091366329 >> 172.21.15.67:0/3091366329 conn(0x55fed953a460 msgr2=0x55fed98e7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.998+0000 7f1160182740 1 -- 172.21.15.67:0/3091366329 shutdown_connections 2024-07-31T14:53:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.998+0000 7f1160182740 1 -- 172.21.15.67:0/3091366329 wait complete. 2024-07-31T14:53:28.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:27.998+0000 7f1160182740 20 client.26005 trim_cache size 0 max 16384 2024-07-31T14:53:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.013+0000 7f1160182740 1 Processor -- start 2024-07-31T14:53:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.013+0000 7f1160182740 1 -- start start 2024-07-31T14:53:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1160182740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed95049f0 0x55fed98e8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1160182740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed951d1b0 0x55fed98e8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1160182740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed9525390 0x55fed98e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1160182740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f11541ae5e0 con 0x55fed95049f0 2024-07-31T14:53:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1160182740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11541b01b0 con 0x55fed951d1b0 2024-07-31T14:53:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1160182740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11541aecb0 con 0x55fed9525390 2024-07-31T14:53:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1153fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed95049f0 0x55fed98e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1153fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed95049f0 0x55fed98e8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45306/0 (socket says 172.21.15.67:45306) 2024-07-31T14:53:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1153fff640 1 -- 172.21.15.67:0/3774599442 learned_addr learned my addr 172.21.15.67:0/3774599442 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1152ffd640 1 --2- 172.21.15.67:0/3774599442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed951d1b0 0x55fed98e8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1153fff640 1 --2- 172.21.15.67:0/3774599442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed9525390 0x55fed98e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1153fff640 1 -- 172.21.15.67:0/3774599442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed9525390 msgr2=0x55fed98e92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1153fff640 1 --2- 172.21.15.67:0/3774599442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed9525390 0x55fed98e92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1153fff640 1 -- 172.21.15.67:0/3774599442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed951d1b0 msgr2=0x55fed98e8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1153fff640 1 --2- 172.21.15.67:0/3774599442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed951d1b0 0x55fed98e8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1153fff640 1 -- 172.21.15.67:0/3774599442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed98fb210 con 0x55fed95049f0 2024-07-31T14:53:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1152ffd640 1 --2- 172.21.15.67:0/3774599442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed951d1b0 0x55fed98e8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:53:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.014+0000 7f1153fff640 1 --2- 172.21.15.67:0/3774599442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed95049f0 0x55fed98e8350 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f114c02f490 tx=0x7f114c005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748a49a6f3ab7f0f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.015+0000 7f11497fa640 1 -- 172.21.15.67:0/3774599442 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c05a020 con 0x55fed95049f0 2024-07-31T14:53:28.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.015+0000 7f11497fa640 1 -- 172.21.15.67:0/3774599442 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114c036040 con 0x55fed95049f0 2024-07-31T14:53:28.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.015+0000 7f11497fa640 1 -- 172.21.15.67:0/3774599442 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c04a040 con 0x55fed95049f0 2024-07-31T14:53:28.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.016+0000 7f1160182740 1 -- 172.21.15.67:0/3774599442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed95049f0 msgr2=0x55fed98e8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:28.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.016+0000 7f1160182740 1 --2- 172.21.15.67:0/3774599442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed95049f0 0x55fed98e8350 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f114c02f490 tx=0x7f114c005c30 comp rx=0 tx=0).stop 2024-07-31T14:53:28.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.016+0000 7f1160182740 1 -- 172.21.15.67:0/3774599442 shutdown_connections 2024-07-31T14:53:28.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.016+0000 7f1160182740 1 --2- 172.21.15.67:0/3774599442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed9525390 0x55fed98e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.016+0000 7f1160182740 1 --2- 172.21.15.67:0/3774599442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed951d1b0 0x55fed98e8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.016+0000 7f1160182740 1 --2- 172.21.15.67:0/3774599442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed95049f0 0x55fed98e8350 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.016+0000 7f1160182740 1 -- 172.21.15.67:0/3774599442 >> 172.21.15.67:0/3774599442 conn(0x55fed953a460 msgr2=0x55fed98e7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:28.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.016+0000 7f1160182740 1 -- 172.21.15.67:0/3774599442 shutdown_connections 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.016+0000 7f1160182740 1 -- 172.21.15.67:0/3774599442 wait complete. 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.016+0000 7f1160182740 1 -- start start 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.016+0000 7f1160182740 1 Processor -- start 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1160182740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed95049f0 0x55fed9907be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1160182740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed951d1b0 0x55fed9903090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1160182740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed9525390 0x55fed99035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1160182740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f11541afae0 con 0x55fed95049f0 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1160182740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11541af380 con 0x55fed9525390 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1160182740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11541ac5c0 con 0x55fed951d1b0 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1160182740 10 client.0 _ll_register_callbacks cb 0x7f113c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1153fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed95049f0 0x55fed9907be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1153fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed95049f0 0x55fed9907be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45318/0 (socket says 172.21.15.67:45318) 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1153fff640 1 -- 172.21.15.67:0/3236647303 learned_addr learned my addr 172.21.15.67:0/3236647303 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1152ffd640 1 --2- 172.21.15.67:0/3236647303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed951d1b0 0x55fed9903090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:28.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1153fff640 1 --2- 172.21.15.67:0/3236647303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed9525390 0x55fed99035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:28.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1153fff640 1 -- 172.21.15.67:0/3236647303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed951d1b0 msgr2=0x55fed9903090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:28.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1153fff640 1 --2- 172.21.15.67:0/3236647303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed951d1b0 0x55fed9903090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1153fff640 1 -- 172.21.15.67:0/3236647303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed9525390 msgr2=0x55fed99035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:28.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1153fff640 1 --2- 172.21.15.67:0/3236647303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed9525390 0x55fed99035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1153fff640 1 -- 172.21.15.67:0/3236647303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed9903b90 con 0x55fed95049f0 2024-07-31T14:53:28.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.017+0000 7f1152ffd640 1 --2- 172.21.15.67:0/3236647303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed951d1b0 0x55fed9903090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:28.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1153fff640 1 --2- 172.21.15.67:0/3236647303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed95049f0 0x55fed9907be0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f114c032d20 tx=0x7f114c0042c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec41612f3fa4e90f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:28.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1132ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:28.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1132ffd640 1 -- 172.21.15.67:0/3236647303 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c05a070 con 0x55fed95049f0 2024-07-31T14:53:28.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1132ffd640 1 -- 172.21.15.67:0/3236647303 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f114c0043a0 con 0x55fed95049f0 2024-07-31T14:53:28.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1160182740 1 -- 172.21.15.67:0/3236647303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114c036040 con 0x55fed95049f0 2024-07-31T14:53:28.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1132ffd640 1 -- 172.21.15.67:0/3236647303 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f114c032e00 con 0x55fed95049f0 2024-07-31T14:53:28.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1160182740 10 client.16347 fetch_fsmap learned FSMap version 34 2024-07-31T14:53:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1160182740 1 -- 172.21.15.67:0/3236647303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fed9979f40 con 0x55fed95049f0 2024-07-31T14:53:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1132ffd640 1 -- 172.21.15.67:0/3236647303 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114c036040 con 0x55fed95049f0 2024-07-31T14:53:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1132ffd640 1 -- 172.21.15.67:0/3236647303 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f114c05f040 con 0x55fed95049f0 2024-07-31T14:53:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1160182740 10 client.16347 fetch_fsmap finished waiting for FSMap version 34 2024-07-31T14:53:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1160182740 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1160182740 20 client.16347 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1160182740 1 --2- 172.21.15.67:0/3236647303 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fed9546070 0x55fed997c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1160182740 4 client.16347 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:53:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.018+0000 7f1160182740 1 -- 172.21.15.67:0/3236647303 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fed985f5a0 con 0x55fed9546070 2024-07-31T14:53:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.019+0000 7f1152ffd640 1 --2- 172.21.15.67:0/3236647303 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fed9546070 0x55fed997c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.019+0000 7f1132ffd640 10 client.16347 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.019+0000 7f1152ffd640 1 --2- 172.21.15.67:0/3236647303 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fed9546070 0x55fed997c1d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac1a51a5378b372 server_cookie=6df809ffb6d25b66 in_seq=0 out_seq=0 2024-07-31T14:53:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.019+0000 7f1132ffd640 1 -- 172.21.15.67:0/3236647303 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fed985f5a0 con 0x55fed9546070 2024-07-31T14:53:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.019+0000 7f1132ffd640 10 client.16347 handle_command_reply: tid=0 2024-07-31T14:53:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.019+0000 7f1160182740 1 client.16347 shutdown 2024-07-31T14:53:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1160182740 1 -- 172.21.15.67:0/3236647303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed95049f0 msgr2=0x55fed9907be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1160182740 1 --2- 172.21.15.67:0/3236647303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed95049f0 0x55fed9907be0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f114c032d20 tx=0x7f114c0042c0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1160182740 1 -- 172.21.15.67:0/3236647303 shutdown_connections 2024-07-31T14:53:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1160182740 1 --2- 172.21.15.67:0/3236647303 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fed9546070 0x55fed997c1d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1160182740 1 --2- 172.21.15.67:0/3236647303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fed9525390 0x55fed99035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1160182740 1 --2- 172.21.15.67:0/3236647303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fed951d1b0 0x55fed9903090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1160182740 1 --2- 172.21.15.67:0/3236647303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fed95049f0 0x55fed9907be0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1132ffd640 0 client.16347 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:28.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1160182740 1 -- 172.21.15.67:0/3236647303 >> 172.21.15.67:0/3236647303 conn(0x55fed953a460 msgr2=0x55fed98e8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:28.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1160182740 1 -- 172.21.15.67:0/3236647303 shutdown_connections 2024-07-31T14:53:28.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1160182740 1 -- 172.21.15.67:0/3236647303 wait complete. 2024-07-31T14:53:28.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1160182740 20 client.16347 trim_cache size 0 max 16384 2024-07-31T14:53:28.027 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:53:28.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1159c31640 1 -- 172.21.15.67:0/3379829878 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11400b3f40 msgr2=0x7f11400b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:28.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1159c31640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11400b3f40 0x7f11400b6450 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55fed98ea1f0 tx=0x7f1144005880 comp rx=0 tx=0).stop 2024-07-31T14:53:28.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1159c31640 1 -- 172.21.15.67:0/3379829878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1154102490 msgr2=0x7f11540ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:28.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1159c31640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1154102490 0x7f11540ff0a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f114400d950 tx=0x7f114400de10 comp rx=0 tx=0).stop 2024-07-31T14:53:28.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1159c31640 1 -- 172.21.15.67:0/3379829878 shutdown_connections 2024-07-31T14:53:28.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1159c31640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11400b3f40 0x7f11400b6450 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1159c31640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1154108720 0x7f11540ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1159c31640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1154107dd0 0x7f11540ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1159c31640 1 --2- 172.21.15.67:0/3379829878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1154102490 0x7f11540ff0a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:28.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.020+0000 7f1159c31640 1 -- 172.21.15.67:0/3379829878 >> 172.21.15.67:0/3379829878 conn(0x7f1154079160 msgr2=0x7f115406ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:28.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.021+0000 7f1159c31640 1 -- 172.21.15.67:0/3379829878 shutdown_connections 2024-07-31T14:53:28.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:28.021+0000 7f1159c31640 1 -- 172.21.15.67:0/3379829878 wait complete. 2024-07-31T14:53:29.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:28 smithi067 ceph-mon[32750]: pgmap v472: 129 pgs: 129 active+clean; 152 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 58 op/s 2024-07-31T14:53:29.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:28 smithi077 ceph-mon[37327]: pgmap v472: 129 pgs: 129 active+clean; 152 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 58 op/s 2024-07-31T14:53:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.356+0000 7f00c2ff5640 1 -- 172.21.15.67:0/2646887534 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b5581dda0 con 0x559b55930740 2024-07-31T14:53:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.356+0000 7f00c2ff5640 10 client.16329 handle_command_reply: tid=0 2024-07-31T14:53:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.356+0000 7f00ec1d1740 1 client.16329 shutdown 2024-07-31T14:53:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.356+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2646887534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 msgr2=0x559b558bbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.356+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2646887534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558bbbf0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f00d801a4d0 tx=0x7f00d8004ab0 comp rx=0 tx=0).stop 2024-07-31T14:53:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.356+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2646887534 shutdown_connections 2024-07-31T14:53:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.356+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2646887534 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559b55930740 0x559b55932b90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.356+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2646887534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b555081b0 0x559b558bbbf0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.356+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2646887534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b554df2f0 0x559b558bb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.356+0000 7f00c2ff5640 0 client.16329 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.356+0000 7f00ec1d1740 1 --2- 172.21.15.67:0/2646887534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b5549c1c0 0x559b558bed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.356+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2646887534 >> 172.21.15.67:0/2646887534 conn(0x559b554fc300 msgr2=0x559b558b7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.357+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2646887534 shutdown_connections 2024-07-31T14:53:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.357+0000 7f00ec1d1740 1 -- 172.21.15.67:0/2646887534 wait complete. 2024-07-31T14:53:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.357+0000 7f00ec1d1740 20 client.16329 trim_cache size 0 max 16384 2024-07-31T14:53:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:53:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:53:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:53:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:53:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.358+0000 7f00e52e3640 1 -- 172.21.15.67:0/705273890 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f00c80b3d40 msgr2=0x7f00c80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.358+0000 7f00e52e3640 1 --2- 172.21.15.67:0/705273890 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f00c80b3d40 0x7f00c80b6250 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f00d8005fd0 tx=0x7f00d8011040 comp rx=0 tx=0).stop 2024-07-31T14:53:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.358+0000 7f00e52e3640 1 -- 172.21.15.67:0/705273890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e0101bb0 msgr2=0x7f00e0197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.358+0000 7f00e52e3640 1 --2- 172.21.15.67:0/705273890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e0101bb0 0x7f00e0197c90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f00d0005fd0 tx=0x7f00d000ee50 comp rx=0 tx=0).stop 2024-07-31T14:53:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.358+0000 7f00e52e3640 1 -- 172.21.15.67:0/705273890 shutdown_connections 2024-07-31T14:53:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.358+0000 7f00e52e3640 1 --2- 172.21.15.67:0/705273890 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f00c80b3d40 0x7f00c80b6250 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.358+0000 7f00e52e3640 1 --2- 172.21.15.67:0/705273890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0107e30 0x7f00e019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.358+0000 7f00e52e3640 1 --2- 172.21.15.67:0/705273890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00e01074e0 0x7f00e0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.358+0000 7f00e52e3640 1 --2- 172.21.15.67:0/705273890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e0101bb0 0x7f00e0197c90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.358+0000 7f00e52e3640 1 -- 172.21.15.67:0/705273890 >> 172.21.15.67:0/705273890 conn(0x7f00e0079880 msgr2=0x7f00e01055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.358+0000 7f00e52e3640 1 -- 172.21.15.67:0/705273890 shutdown_connections 2024-07-31T14:53:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:29.358+0000 7f00e52e3640 1 -- 172.21.15.67:0/705273890 wait complete. 2024-07-31T14:53:29.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:28 smithi073 ceph-mon[36714]: pgmap v472: 129 pgs: 129 active+clean; 152 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 58 op/s 2024-07-31T14:53:29.836 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T14:53:29.836 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T14:53:30.114 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:53:30.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:29 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:30.237 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:29 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:30.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:29 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:31.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:30 smithi077 ceph-mon[37327]: pgmap v473: 129 pgs: 129 active+clean; 154 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1021 KiB/s wr, 144 op/s 2024-07-31T14:53:31.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:30 smithi067 ceph-mon[32750]: pgmap v473: 129 pgs: 129 active+clean; 154 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1021 KiB/s wr, 144 op/s 2024-07-31T14:53:31.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:30 smithi073 ceph-mon[36714]: pgmap v473: 129 pgs: 129 active+clean; 154 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1021 KiB/s wr, 144 op/s 2024-07-31T14:53:31.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.927+0000 7ff1fcda1640 1 -- 172.21.15.67:0/591418006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1f8107bf0 msgr2=0x7ff1f8108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:31.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.927+0000 7ff1fcda1640 1 --2- 172.21.15.67:0/591418006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1f8107bf0 0x7ff1f8108000 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009900 tx=0x7ff1e002a410 comp rx=0 tx=0).stop 2024-07-31T14:53:31.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.927+0000 7ff1fcda1640 1 -- 172.21.15.67:0/591418006 shutdown_connections 2024-07-31T14:53:31.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.927+0000 7ff1fcda1640 1 --2- 172.21.15.67:0/591418006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1f8108540 0x7ff1f810ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.927+0000 7ff1fcda1640 1 --2- 172.21.15.67:0/591418006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1f8107bf0 0x7ff1f8108000 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.927+0000 7ff1fcda1640 1 --2- 172.21.15.67:0/591418006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1f81022b0 0x7ff1f81026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.927+0000 7ff1fcda1640 1 -- 172.21.15.67:0/591418006 >> 172.21.15.67:0/591418006 conn(0x7ff1f80fdd00 msgr2=0x7ff1f8100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:31.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.927+0000 7ff1fcda1640 1 -- 172.21.15.67:0/591418006 shutdown_connections 2024-07-31T14:53:31.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.927+0000 7ff1fcda1640 1 -- 172.21.15.67:0/591418006 wait complete. 2024-07-31T14:53:31.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.928+0000 7ff1fcda1640 1 Processor -- start 2024-07-31T14:53:31.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.928+0000 7ff1fcda1640 1 -- start start 2024-07-31T14:53:31.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.928+0000 7ff1fcda1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1f81022b0 0x7ff1f819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.928+0000 7ff1fcda1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1f8107bf0 0x7ff1f819b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1fcda1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1f8108540 0x7ff1f819b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1fcda1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8112220 con 0x7ff1f81022b0 2024-07-31T14:53:31.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1fcda1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81120a0 con 0x7ff1f8107bf0 2024-07-31T14:53:31.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1fcda1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81123a0 con 0x7ff1f8108540 2024-07-31T14:53:31.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1f81022b0 0x7ff1f819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1f81022b0 0x7ff1f819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60104/0 (socket says 172.21.15.67:60104) 2024-07-31T14:53:31.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1f77fe640 1 -- 172.21.15.67:0/2619862998 learned_addr learned my addr 172.21.15.67:0/2619862998 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:31.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1effff640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1f8107bf0 0x7ff1f819b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1f7fff640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1f8108540 0x7ff1f819b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1f77fe640 1 -- 172.21.15.67:0/2619862998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1f8108540 msgr2=0x7ff1f819b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:31.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1f77fe640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1f8108540 0x7ff1f819b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1f77fe640 1 -- 172.21.15.67:0/2619862998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1f8107bf0 msgr2=0x7ff1f819b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1f77fe640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1f8107bf0 0x7ff1f819b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1f77fe640 1 -- 172.21.15.67:0/2619862998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f81a5d80 con 0x7ff1f81022b0 2024-07-31T14:53:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1f7fff640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1f8108540 0x7ff1f819b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:53:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1f77fe640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1f81022b0 0x7ff1f819aaf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800e9f0 tx=0x7ff1e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1f57fa640 1 -- 172.21.15.67:0/2619862998 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e800cd80 con 0x7ff1f81022b0 2024-07-31T14:53:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.929+0000 7ff1f57fa640 1 -- 172.21.15.67:0/2619862998 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e80108e0 con 0x7ff1f81022b0 2024-07-31T14:53:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.930+0000 7ff1fcda1640 1 -- 172.21.15.67:0/2619862998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f81a6070 con 0x7ff1f81022b0 2024-07-31T14:53:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.930+0000 7ff1f57fa640 1 -- 172.21.15.67:0/2619862998 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8018980 con 0x7ff1f81022b0 2024-07-31T14:53:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.930+0000 7ff1fcda1640 1 -- 172.21.15.67:0/2619862998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f81a6580 con 0x7ff1f81022b0 2024-07-31T14:53:31.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.932+0000 7ff204554740 1 Processor -- start 2024-07-31T14:53:31.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.932+0000 7ff204554740 1 -- start start 2024-07-31T14:53:31.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff204554740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce81350 0x55e98d22a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff204554740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce58490 0x55e98d22aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff1f57fa640 1 -- 172.21.15.67:0/2619862998 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff1e8018b20 con 0x7ff1f81022b0 2024-07-31T14:53:31.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff204554740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce60670 0x55e98d22b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff204554740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e98d10e2f0 con 0x55e98ce58490 2024-07-31T14:53:31.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff204554740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e98d19ca80 con 0x55e98ce81350 2024-07-31T14:53:31.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff204554740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e98d18bf80 con 0x55e98ce60670 2024-07-31T14:53:31.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff1f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce58490 0x55e98d22aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce81350 0x55e98d22a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce81350 0x55e98d22a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40512/0 (socket says 172.21.15.67:40512) 2024-07-31T14:53:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff1f7fff640 1 -- 172.21.15.67:0/4157147465 learned_addr learned my addr 172.21.15.67:0/4157147465 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff1effff640 1 --2- 172.21.15.67:0/4157147465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce60670 0x55e98d22b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff1f77fe640 1 -- 172.21.15.67:0/4157147465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce60670 msgr2=0x55e98d22b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff1f77fe640 1 --2- 172.21.15.67:0/4157147465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce60670 0x55e98d22b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff1f77fe640 1 -- 172.21.15.67:0/4157147465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce81350 msgr2=0x55e98d22a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff1f77fe640 1 --2- 172.21.15.67:0/4157147465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce81350 0x55e98d22a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.933+0000 7ff1f77fe640 1 -- 172.21.15.67:0/4157147465 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98ceec930 con 0x55e98ce58490 2024-07-31T14:53:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.934+0000 7ff1f77fe640 1 --2- 172.21.15.67:0/4157147465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce58490 0x55e98d22aed0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80e1b40 tx=0x7ff1e80ec690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f6eb8261e64be0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.934+0000 7ff1f57fa640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff1e40b3d70 0x7ff1e40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.934+0000 7ff1edffb640 1 -- 172.21.15.67:0/4157147465 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80f3070 con 0x55e98ce58490 2024-07-31T14:53:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.934+0000 7ff1f7fff640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff1e40b3d70 0x7ff1e40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.934+0000 7ff1edffb640 1 -- 172.21.15.67:0/4157147465 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e80aab10 con 0x55e98ce58490 2024-07-31T14:53:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.934+0000 7ff1edffb640 1 -- 172.21.15.67:0/4157147465 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80aa770 con 0x55e98ce58490 2024-07-31T14:53:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.934+0000 7ff1f57fa640 1 -- 172.21.15.67:0/2619862998 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff1e80e4700 con 0x7ff1f81022b0 2024-07-31T14:53:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.935+0000 7ff204554740 1 -- 172.21.15.67:0/4157147465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce58490 msgr2=0x55e98d22aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.935+0000 7ff1f7fff640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff1e40b3d70 0x7ff1e40b6280 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0012560 tx=0x7ff1f0009790 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.935+0000 7ff204554740 1 --2- 172.21.15.67:0/4157147465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce58490 0x55e98d22aed0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80e1b40 tx=0x7ff1e80ec690 comp rx=0 tx=0).stop 2024-07-31T14:53:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.935+0000 7ff204554740 1 -- 172.21.15.67:0/4157147465 shutdown_connections 2024-07-31T14:53:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.935+0000 7ff204554740 1 --2- 172.21.15.67:0/4157147465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce60670 0x55e98d22b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.935+0000 7ff204554740 1 --2- 172.21.15.67:0/4157147465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce58490 0x55e98d22aed0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.935+0000 7ff204554740 1 --2- 172.21.15.67:0/4157147465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce81350 0x55e98d22a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.935+0000 7ff204554740 1 -- 172.21.15.67:0/4157147465 >> 172.21.15.67:0/4157147465 conn(0x55e98ce75740 msgr2=0x55e98d228880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.935+0000 7ff204554740 1 -- 172.21.15.67:0/4157147465 shutdown_connections 2024-07-31T14:53:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.935+0000 7ff204554740 1 -- 172.21.15.67:0/4157147465 wait complete. 2024-07-31T14:53:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.935+0000 7ff204554740 1 -- start start 2024-07-31T14:53:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.936+0000 7ff204554740 1 Processor -- start 2024-07-31T14:53:31.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.936+0000 7ff204554740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce58490 0x55e98d2364f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.936+0000 7ff204554740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce60670 0x55e98d236a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.936+0000 7ff204554740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce81350 0x55e98d242310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.936+0000 7ff204554740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e98d12e4f0 con 0x55e98ce81350 2024-07-31T14:53:31.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.936+0000 7ff204554740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e98d18ffc0 con 0x55e98ce60670 2024-07-31T14:53:31.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.936+0000 7ff204554740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e98cfef890 con 0x55e98ce58490 2024-07-31T14:53:31.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.936+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce81350 0x55e98d242310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.936+0000 7ff204554740 10 client.0 _ll_register_callbacks cb 0x55e98d138500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:31.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.936+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce81350 0x55e98d242310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60124/0 (socket says 172.21.15.67:60124) 2024-07-31T14:53:31.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.936+0000 7ff1f7fff640 1 -- 172.21.15.67:0/3465835601 learned_addr learned my addr 172.21.15.67:0/3465835601 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:31.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.936+0000 7ff1f77fe640 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce58490 0x55e98d2364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.936+0000 7ff1effff640 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce60670 0x55e98d236a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff1f7fff640 1 -- 172.21.15.67:0/3465835601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce58490 msgr2=0x55e98d2364f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:31.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff1f7fff640 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce58490 0x55e98d2364f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff1f7fff640 1 -- 172.21.15.67:0/3465835601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce60670 msgr2=0x55e98d236a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff1f7fff640 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce60670 0x55e98d236a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff1f7fff640 1 -- 172.21.15.67:0/3465835601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98d13dc50 con 0x55e98ce81350 2024-07-31T14:53:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff1f77fe640 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce58490 0x55e98d2364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff1effff640 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce60670 0x55e98d236a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff1f7fff640 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce81350 0x55e98d242310 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0013fd0 tx=0x7ff1f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221d44fed6c516 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff1d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff1d6ffd640 1 -- 172.21.15.67:0/3465835601 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f000f700 con 0x55e98ce81350 2024-07-31T14:53:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff1d6ffd640 1 -- 172.21.15.67:0/3465835601 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff1f000f8a0 con 0x55e98ce81350 2024-07-31T14:53:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff204554740 1 -- 172.21.15.67:0/3465835601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e80aab10 con 0x55e98ce81350 2024-07-31T14:53:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff1d6ffd640 1 -- 172.21.15.67:0/3465835601 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff1f000faa0 con 0x55e98ce81350 2024-07-31T14:53:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff204554740 10 client.16359 fetch_fsmap learned FSMap version 36 2024-07-31T14:53:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.937+0000 7ff204554740 1 -- 172.21.15.67:0/3465835601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e98d244fe0 con 0x55e98ce81350 2024-07-31T14:53:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.938+0000 7ff1d6ffd640 1 -- 172.21.15.67:0/3465835601 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e80aab10 con 0x55e98ce81350 2024-07-31T14:53:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.938+0000 7ff1d6ffd640 1 -- 172.21.15.67:0/3465835601 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 36) ==== 8402+0+0 (secure 0 0 0) 0x7ff1f001d410 con 0x55e98ce81350 2024-07-31T14:53:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.938+0000 7ff204554740 10 client.16359 fetch_fsmap finished waiting for FSMap version 36 2024-07-31T14:53:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.938+0000 7ff204554740 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.938+0000 7ff204554740 20 client.16359 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.938+0000 7ff204554740 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e98ce3fcd0 0x55e98d248620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.938+0000 7ff204554740 4 client.16359 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:53:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.938+0000 7ff204554740 1 -- 172.21.15.67:0/3465835601 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e98d19e9d0 con 0x55e98ce3fcd0 2024-07-31T14:53:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.938+0000 7ff1f77fe640 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e98ce3fcd0 0x55e98d248620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.939+0000 7ff1f77fe640 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e98ce3fcd0 0x55e98d248620 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15dabee7c0ec3f22 server_cookie=13e376eba59b38a0 in_seq=0 out_seq=0 2024-07-31T14:53:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.939+0000 7ff1d6ffd640 10 client.16359 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.939+0000 7ff1d6ffd640 1 -- 172.21.15.67:0/3465835601 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e98d19e9d0 con 0x55e98ce3fcd0 2024-07-31T14:53:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.939+0000 7ff1d6ffd640 10 client.16359 handle_command_reply: tid=0 2024-07-31T14:53:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.939+0000 7ff204554740 1 client.16359 shutdown 2024-07-31T14:53:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.939+0000 7ff204554740 1 -- 172.21.15.67:0/3465835601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce81350 msgr2=0x55e98d242310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.939+0000 7ff204554740 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce81350 0x55e98d242310 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0013fd0 tx=0x7ff1f0004a40 comp rx=0 tx=0).stop 2024-07-31T14:53:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.939+0000 7ff204554740 1 -- 172.21.15.67:0/3465835601 shutdown_connections 2024-07-31T14:53:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.939+0000 7ff204554740 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e98ce3fcd0 0x55e98d248620 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.940+0000 7ff204554740 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce81350 0x55e98d242310 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.940+0000 7ff204554740 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce60670 0x55e98d236a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.940+0000 7ff204554740 1 --2- 172.21.15.67:0/3465835601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce58490 0x55e98d2364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.940+0000 7ff1d6ffd640 0 client.16359 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.940+0000 7ff204554740 1 -- 172.21.15.67:0/3465835601 >> 172.21.15.67:0/3465835601 conn(0x55e98ce75740 msgr2=0x55e98d222b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.940+0000 7ff204554740 1 -- 172.21.15.67:0/3465835601 shutdown_connections 2024-07-31T14:53:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.940+0000 7ff204554740 1 -- 172.21.15.67:0/3465835601 wait complete. 2024-07-31T14:53:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.940+0000 7ff204554740 20 client.16359 trim_cache size 0 max 16384 2024-07-31T14:53:31.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.958+0000 7ff204554740 1 Processor -- start 2024-07-31T14:53:31.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.958+0000 7ff204554740 1 -- start start 2024-07-31T14:53:31.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.958+0000 7ff204554740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce58490 0x55e98d2231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.958+0000 7ff204554740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce60670 0x55e98d2288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.958+0000 7ff204554740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce81350 0x55e98d242100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.958+0000 7ff204554740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81ae320 con 0x55e98ce60670 2024-07-31T14:53:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.958+0000 7ff204554740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81aff20 con 0x55e98ce58490 2024-07-31T14:53:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.958+0000 7ff204554740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81ae9f0 con 0x55e98ce81350 2024-07-31T14:53:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.958+0000 7ff1effff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce60670 0x55e98d2288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.959+0000 7ff1effff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce60670 0x55e98d2288b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60136/0 (socket says 172.21.15.67:60136) 2024-07-31T14:53:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.959+0000 7ff1effff640 1 -- 172.21.15.67:0/4034985343 learned_addr learned my addr 172.21.15.67:0/4034985343 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.959+0000 7ff1f77fe640 1 --2- 172.21.15.67:0/4034985343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce58490 0x55e98d2231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.959+0000 7ff1f7fff640 1 --2- 172.21.15.67:0/4034985343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce81350 0x55e98d242100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.959+0000 7ff1effff640 1 -- 172.21.15.67:0/4034985343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce81350 msgr2=0x55e98d242100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.959+0000 7ff1effff640 1 --2- 172.21.15.67:0/4034985343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce81350 0x55e98d242100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.959+0000 7ff1effff640 1 -- 172.21.15.67:0/4034985343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce58490 msgr2=0x55e98d2231e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:31.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.959+0000 7ff1effff640 1 --2- 172.21.15.67:0/4034985343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce58490 0x55e98d2231e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.959+0000 7ff1effff640 1 -- 172.21.15.67:0/4034985343 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98d22cc80 con 0x55e98ce60670 2024-07-31T14:53:31.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.959+0000 7ff1f7fff640 1 --2- 172.21.15.67:0/4034985343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce81350 0x55e98d242100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:53:31.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.959+0000 7ff1effff640 1 --2- 172.21.15.67:0/4034985343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce60670 0x55e98d2288b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0031440 tx=0x7ff1e0004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d95257aff6e9a8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:31.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.959+0000 7ff1ed7fa640 1 -- 172.21.15.67:0/4034985343 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0035040 con 0x55e98ce60670 2024-07-31T14:53:31.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.959+0000 7ff1ed7fa640 1 -- 172.21.15.67:0/4034985343 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e0004750 con 0x55e98ce60670 2024-07-31T14:53:31.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.959+0000 7ff1ed7fa640 1 -- 172.21.15.67:0/4034985343 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0003830 con 0x55e98ce60670 2024-07-31T14:53:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.960+0000 7ff204554740 1 -- 172.21.15.67:0/4034985343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce60670 msgr2=0x55e98d2288b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.960+0000 7ff204554740 1 --2- 172.21.15.67:0/4034985343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce60670 0x55e98d2288b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0031440 tx=0x7ff1e0004310 comp rx=0 tx=0).stop 2024-07-31T14:53:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.960+0000 7ff204554740 1 -- 172.21.15.67:0/4034985343 shutdown_connections 2024-07-31T14:53:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.960+0000 7ff204554740 1 --2- 172.21.15.67:0/4034985343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce81350 0x55e98d242100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.960+0000 7ff204554740 1 --2- 172.21.15.67:0/4034985343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce60670 0x55e98d2288b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.960+0000 7ff204554740 1 --2- 172.21.15.67:0/4034985343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce58490 0x55e98d2231e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.960+0000 7ff204554740 1 -- 172.21.15.67:0/4034985343 >> 172.21.15.67:0/4034985343 conn(0x55e98ce75740 msgr2=0x55e98d2281a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.960+0000 7ff204554740 1 -- 172.21.15.67:0/4034985343 shutdown_connections 2024-07-31T14:53:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.960+0000 7ff204554740 1 -- 172.21.15.67:0/4034985343 wait complete. 2024-07-31T14:53:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.960+0000 7ff204554740 1 -- start start 2024-07-31T14:53:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.961+0000 7ff204554740 1 Processor -- start 2024-07-31T14:53:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.961+0000 7ff204554740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce58490 0x55e98d23e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.961+0000 7ff204554740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce60670 0x55e98d23eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.961+0000 7ff204554740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce81350 0x55e98d239bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.961+0000 7ff204554740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81af850 con 0x55e98ce58490 2024-07-31T14:53:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.961+0000 7ff204554740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81af0c0 con 0x55e98ce60670 2024-07-31T14:53:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.961+0000 7ff204554740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81ac300 con 0x55e98ce81350 2024-07-31T14:53:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.961+0000 7ff204554740 10 client.0 _ll_register_callbacks cb 0x7ff1f00106d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.961+0000 7ff1f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce58490 0x55e98d23e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.961+0000 7ff1f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce58490 0x55e98d23e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60150/0 (socket says 172.21.15.67:60150) 2024-07-31T14:53:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.961+0000 7ff1f77fe640 1 -- 172.21.15.67:0/1371256389 learned_addr learned my addr 172.21.15.67:0/1371256389 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.961+0000 7ff1f7fff640 1 --2- 172.21.15.67:0/1371256389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce81350 0x55e98d239bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.961+0000 7ff1effff640 1 --2- 172.21.15.67:0/1371256389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce60670 0x55e98d23eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff1f77fe640 1 -- 172.21.15.67:0/1371256389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce81350 msgr2=0x55e98d239bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff1f77fe640 1 --2- 172.21.15.67:0/1371256389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce81350 0x55e98d239bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff1f77fe640 1 -- 172.21.15.67:0/1371256389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce60670 msgr2=0x55e98d23eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff1f77fe640 1 --2- 172.21.15.67:0/1371256389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce60670 0x55e98d23eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff1f77fe640 1 -- 172.21.15.67:0/1371256389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98d23a360 con 0x55e98ce58490 2024-07-31T14:53:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff1effff640 1 --2- 172.21.15.67:0/1371256389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce60670 0x55e98d23eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff1f77fe640 1 --2- 172.21.15.67:0/1371256389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce58490 0x55e98d23e590 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80aaf90 tx=0x7ff1e8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d6a3f3ddf24904 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff1d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff1d6ffd640 1 -- 172.21.15.67:0/1371256389 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80952b0 con 0x55e98ce58490 2024-07-31T14:53:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff1d6ffd640 1 -- 172.21.15.67:0/1371256389 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff1e8097820 con 0x55e98ce58490 2024-07-31T14:53:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff204554740 1 -- 172.21.15.67:0/1371256389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e0004750 con 0x55e98ce58490 2024-07-31T14:53:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff1d6ffd640 1 -- 172.21.15.67:0/1371256389 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff1e80954b0 con 0x55e98ce58490 2024-07-31T14:53:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff204554740 10 client.16365 fetch_fsmap learned FSMap version 36 2024-07-31T14:53:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff204554740 1 -- 172.21.15.67:0/1371256389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e98d2b0900 con 0x55e98ce58490 2024-07-31T14:53:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.962+0000 7ff1d6ffd640 1 -- 172.21.15.67:0/1371256389 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e0004750 con 0x55e98ce58490 2024-07-31T14:53:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.963+0000 7ff1d6ffd640 1 -- 172.21.15.67:0/1371256389 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 36) ==== 8402+0+0 (secure 0 0 0) 0x7ff1e80e4070 con 0x55e98ce58490 2024-07-31T14:53:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.963+0000 7ff204554740 10 client.16365 fetch_fsmap finished waiting for FSMap version 36 2024-07-31T14:53:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.963+0000 7ff204554740 10 client.16365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.963+0000 7ff204554740 20 client.16365 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.963+0000 7ff204554740 1 --2- 172.21.15.67:0/1371256389 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e98ce15080 0x55e98d2b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.963+0000 7ff204554740 4 client.16365 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T14:53:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.963+0000 7ff204554740 1 -- 172.21.15.67:0/1371256389 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e98d19a470 con 0x55e98ce15080 2024-07-31T14:53:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.963+0000 7ff1effff640 1 --2- 172.21.15.67:0/1371256389 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e98ce15080 0x55e98d2b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.963+0000 7ff1effff640 1 --2- 172.21.15.67:0/1371256389 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e98ce15080 0x55e98d2b2cb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=299297c7880ba41c server_cookie=c1718145fff98726 in_seq=0 out_seq=0 2024-07-31T14:53:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.963+0000 7ff1d6ffd640 10 client.16365 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.964+0000 7ff1d6ffd640 1 -- 172.21.15.67:0/1371256389 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e98d19a470 con 0x55e98ce15080 2024-07-31T14:53:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.964+0000 7ff1d6ffd640 10 client.16365 handle_command_reply: tid=0 2024-07-31T14:53:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.964+0000 7ff204554740 1 client.16365 shutdown 2024-07-31T14:53:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.964+0000 7ff204554740 1 -- 172.21.15.67:0/1371256389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce58490 msgr2=0x55e98d23e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.964+0000 7ff204554740 1 --2- 172.21.15.67:0/1371256389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce58490 0x55e98d23e590 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80aaf90 tx=0x7ff1e8000bc0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.964+0000 7ff204554740 1 -- 172.21.15.67:0/1371256389 shutdown_connections 2024-07-31T14:53:31.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.964+0000 7ff204554740 1 --2- 172.21.15.67:0/1371256389 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e98ce15080 0x55e98d2b2cb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.964+0000 7ff204554740 1 --2- 172.21.15.67:0/1371256389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e98ce81350 0x55e98d239bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.964+0000 7ff204554740 1 --2- 172.21.15.67:0/1371256389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e98ce60670 0x55e98d23eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.964+0000 7ff204554740 1 --2- 172.21.15.67:0/1371256389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e98ce58490 0x55e98d23e590 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.964+0000 7ff204554740 1 -- 172.21.15.67:0/1371256389 >> 172.21.15.67:0/1371256389 conn(0x55e98ce75740 msgr2=0x55e98d22da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:31.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.964+0000 7ff1d6ffd640 0 client.16365 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:31.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.965+0000 7ff204554740 1 -- 172.21.15.67:0/1371256389 shutdown_connections 2024-07-31T14:53:31.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.965+0000 7ff204554740 1 -- 172.21.15.67:0/1371256389 wait complete. 2024-07-31T14:53:31.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.965+0000 7ff204554740 20 client.16365 trim_cache size 0 max 16384 2024-07-31T14:53:31.976 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:53:31.976 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T14:53:31.976 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "649431d1-591c-4169-a259-86430c9699be", 2024-07-31T14:53:31.976 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T14:53:31.976 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:53:31.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.965+0000 7ff1fcda1640 1 -- 172.21.15.67:0/2619862998 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff1e40b3d70 msgr2=0x7ff1e40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:31.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.965+0000 7ff1fcda1640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff1e40b3d70 0x7ff1e40b6280 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0012560 tx=0x7ff1f0009790 comp rx=0 tx=0).stop 2024-07-31T14:53:31.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.965+0000 7ff1fcda1640 1 -- 172.21.15.67:0/2619862998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1f81022b0 msgr2=0x7ff1f819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:31.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.965+0000 7ff1fcda1640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1f81022b0 0x7ff1f819aaf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800e9f0 tx=0x7ff1e800eeb0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.965+0000 7ff1fcda1640 1 -- 172.21.15.67:0/2619862998 shutdown_connections 2024-07-31T14:53:31.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.965+0000 7ff1fcda1640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff1e40b3d70 0x7ff1e40b6280 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.965+0000 7ff1fcda1640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1f8108540 0x7ff1f819b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.965+0000 7ff1fcda1640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1f8107bf0 0x7ff1f819b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.965+0000 7ff1fcda1640 1 --2- 172.21.15.67:0/2619862998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1f81022b0 0x7ff1f819aaf0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:31.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.966+0000 7ff1fcda1640 1 -- 172.21.15.67:0/2619862998 >> 172.21.15.67:0/2619862998 conn(0x7ff1f80fdd00 msgr2=0x7ff1f8100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:31.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.966+0000 7ff1fcda1640 1 -- 172.21.15.67:0/2619862998 shutdown_connections 2024-07-31T14:53:31.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:31.966+0000 7ff1fcda1640 1 -- 172.21.15.67:0/2619862998 wait complete. 2024-07-31T14:53:32.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:31 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:32.288 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:31 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:32.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:31 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:32.806 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:53:33.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: pgmap v474: 129 pgs: 129 active+clean; 150 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-07-31T14:53:33.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T14:53:33.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: pgmap v474: 129 pgs: 129 active+clean; 150 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-07-31T14:53:33.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T14:53:33.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:33.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.338 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:53:33.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: pgmap v474: 129 pgs: 129 active+clean; 150 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-07-31T14:53:33.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T14:53:33.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:33.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:33.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:35.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:34 smithi067 ceph-mon[32750]: pgmap v475: 129 pgs: 129 active+clean; 157 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 425 op/s 2024-07-31T14:53:35.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:35.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:35.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:35.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:34 smithi077 ceph-mon[37327]: pgmap v475: 129 pgs: 129 active+clean; 157 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 425 op/s 2024-07-31T14:53:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:35.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.299+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/4122861770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a4107dd0 msgr2=0x7fe8a41081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.299+0000 7fe8ac5ea640 1 --2- 172.21.15.67:0/4122861770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a4107dd0 0x7fe8a41081e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe898009900 tx=0x7fe89802a410 comp rx=0 tx=0).stop 2024-07-31T14:53:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.299+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/4122861770 shutdown_connections 2024-07-31T14:53:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.299+0000 7fe8ac5ea640 1 --2- 172.21.15.67:0/4122861770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8a4108720 0x7fe8a410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.299+0000 7fe8ac5ea640 1 --2- 172.21.15.67:0/4122861770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a4107dd0 0x7fe8a41081e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.299+0000 7fe8ac5ea640 1 --2- 172.21.15.67:0/4122861770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a4102490 0x7fe8a41028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.299+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/4122861770 >> 172.21.15.67:0/4122861770 conn(0x7fe8a4079160 msgr2=0x7fe8a4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.299+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/4122861770 shutdown_connections 2024-07-31T14:53:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.300+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/4122861770 wait complete. 2024-07-31T14:53:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.300+0000 7fe8ac5ea640 1 Processor -- start 2024-07-31T14:53:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.301+0000 7fe8ac5ea640 1 -- start start 2024-07-31T14:53:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.302+0000 7fe8ac5ea640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a4102490 0x7fe8a419c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.302+0000 7fe8ac5ea640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a4107dd0 0x7fe8a419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.302+0000 7fe8ac5ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8a4108720 0x7fe8a41a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.302+0000 7fe8ac5ea640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a41134b0 con 0x7fe8a4107dd0 2024-07-31T14:53:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.302+0000 7fe8aa386640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a4102490 0x7fe8a419c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.302+0000 7fe8a9b85640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a4107dd0 0x7fe8a419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.302+0000 7fe8aa386640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a4102490 0x7fe8a419c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40582/0 (socket says 172.21.15.67:40582) 2024-07-31T14:53:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.302+0000 7fe8aa386640 1 -- 172.21.15.67:0/89318997 learned_addr learned my addr 172.21.15.67:0/89318997 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.302+0000 7fe8aab87640 1 --2- 172.21.15.67:0/89318997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8a4108720 0x7fe8a41a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.302+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/89318997 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a4113330 con 0x7fe8a4102490 2024-07-31T14:53:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.302+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/89318997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a4113630 con 0x7fe8a4108720 2024-07-31T14:53:35.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.303+0000 7fe8aa386640 1 -- 172.21.15.67:0/89318997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8a4108720 msgr2=0x7fe8a41a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:35.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.303+0000 7fe8aa386640 1 --2- 172.21.15.67:0/89318997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8a4108720 0x7fe8a41a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.303+0000 7fe8aa386640 1 -- 172.21.15.67:0/89318997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a4107dd0 msgr2=0x7fe8a419c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:35.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.303+0000 7fe8aa386640 1 --2- 172.21.15.67:0/89318997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a4107dd0 0x7fe8a419c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.303+0000 7fe8aa386640 1 -- 172.21.15.67:0/89318997 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a41a20b0 con 0x7fe8a4102490 2024-07-31T14:53:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.303+0000 7fe8aa386640 1 --2- 172.21.15.67:0/89318997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a4102490 0x7fe8a419c140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe89400ebc0 tx=0x7fe89400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.303+0000 7fe8a37fe640 1 -- 172.21.15.67:0/89318997 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894010630 con 0x7fe8a4102490 2024-07-31T14:53:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.303+0000 7fe8a37fe640 1 -- 172.21.15.67:0/89318997 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe894010c80 con 0x7fe8a4102490 2024-07-31T14:53:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.303+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/89318997 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a41a23a0 con 0x7fe8a4102490 2024-07-31T14:53:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.303+0000 7fe8a37fe640 1 -- 172.21.15.67:0/89318997 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894018b60 con 0x7fe8a4102490 2024-07-31T14:53:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.303+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/89318997 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a41a8b00 con 0x7fe8a4102490 2024-07-31T14:53:35.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.306+0000 7fe8b2b3b740 1 Processor -- start 2024-07-31T14:53:35.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.306+0000 7fe8b2b3b740 1 -- start start 2024-07-31T14:53:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.306+0000 7fe8b2b3b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560740215070 0x5607405ba520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.306+0000 7fe8b2b3b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405baa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.306+0000 7fe8b2b3b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607401f4390 0x5607405bafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.306+0000 7fe8b2b3b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560740530b60 con 0x5607401ec1b0 2024-07-31T14:53:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.306+0000 7fe8b2b3b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560740520060 con 0x560740215070 2024-07-31T14:53:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.306+0000 7fe8b2b3b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5607405240a0 con 0x5607401f4390 2024-07-31T14:53:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8a37fe640 1 -- 172.21.15.67:0/89318997 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe894018d00 con 0x7fe8a4102490 2024-07-31T14:53:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8aab87640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560740215070 0x5607405ba520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8aab87640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560740215070 0x5607405ba520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40596/0 (socket says 172.21.15.67:40596) 2024-07-31T14:53:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8aab87640 1 -- 172.21.15.67:0/334882314 learned_addr learned my addr 172.21.15.67:0/334882314 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8aa386640 1 --2- 172.21.15.67:0/334882314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405baa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8a9b85640 1 --2- 172.21.15.67:0/334882314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607401f4390 0x5607405bafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8aab87640 1 -- 172.21.15.67:0/334882314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607401f4390 msgr2=0x5607405bafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8aab87640 1 --2- 172.21.15.67:0/334882314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607401f4390 0x5607405bafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8aab87640 1 -- 172.21.15.67:0/334882314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 msgr2=0x5607405baa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8aab87640 1 --2- 172.21.15.67:0/334882314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405baa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8aab87640 1 -- 172.21.15.67:0/334882314 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607404c6b40 con 0x560740215070 2024-07-31T14:53:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8aab87640 1 --2- 172.21.15.67:0/334882314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560740215070 0x5607405ba520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00b6b0 tx=0x7fe89c004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e24b28af5f8f09d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8a0ff9640 1 -- 172.21.15.67:0/334882314 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c003b20 con 0x560740215070 2024-07-31T14:53:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8a0ff9640 1 -- 172.21.15.67:0/334882314 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe89c015070 con 0x560740215070 2024-07-31T14:53:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.307+0000 7fe8a0ff9640 1 -- 172.21.15.67:0/334882314 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c0116a0 con 0x560740215070 2024-07-31T14:53:35.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.308+0000 7fe8a37fe640 1 --2- 172.21.15.67:0/89318997 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe88c0b3d40 0x7fe88c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.308+0000 7fe8aa386640 1 --2- 172.21.15.67:0/89318997 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe88c0b3d40 0x7fe88c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.308+0000 7fe8a37fe640 1 -- 172.21.15.67:0/89318997 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fe8940e4910 con 0x7fe8a4102490 2024-07-31T14:53:35.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.308+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/334882314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560740215070 msgr2=0x5607405ba520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:35.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.308+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/334882314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560740215070 0x5607405ba520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00b6b0 tx=0x7fe89c004610 comp rx=0 tx=0).stop 2024-07-31T14:53:35.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.308+0000 7fe8aa386640 1 --2- 172.21.15.67:0/89318997 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe88c0b3d40 0x7fe88c0b6250 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe8940e1e00 tx=0x7fe894000f40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:35.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.309+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/334882314 shutdown_connections 2024-07-31T14:53:35.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.309+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/334882314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607401f4390 0x5607405bafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.309+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/334882314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405baa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.309+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/334882314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560740215070 0x5607405ba520 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.309+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/334882314 >> 172.21.15.67:0/334882314 conn(0x560740209460 msgr2=0x5607405b8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.309+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/334882314 shutdown_connections 2024-07-31T14:53:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.309+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/334882314 wait complete. 2024-07-31T14:53:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.309+0000 7fe8b2b3b740 1 -- start start 2024-07-31T14:53:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.309+0000 7fe8b2b3b740 1 Processor -- start 2024-07-31T14:53:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.309+0000 7fe8b2b3b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405c6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8b2b3b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607401f4390 0x5607405c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8b2b3b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560740215070 0x5607405d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8b2b3b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560740383700 con 0x5607401ec1b0 2024-07-31T14:53:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8b2b3b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5607404c2990 con 0x560740215070 2024-07-31T14:53:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8b2b3b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5607405c8d00 con 0x5607401f4390 2024-07-31T14:53:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8b2b3b740 10 client.0 _ll_register_callbacks cb 0x5607404dd4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8aab87640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8aab87640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60202/0 (socket says 172.21.15.67:60202) 2024-07-31T14:53:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8aab87640 1 -- 172.21.15.67:0/1340327043 learned_addr learned my addr 172.21.15.67:0/1340327043 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8aab87640 1 --2- 172.21.15.67:0/1340327043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560740215070 0x5607405d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8a9b85640 1 --2- 172.21.15.67:0/1340327043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607401f4390 0x5607405c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8aab87640 1 -- 172.21.15.67:0/1340327043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607401f4390 msgr2=0x5607405c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8aab87640 1 --2- 172.21.15.67:0/1340327043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607401f4390 0x5607405c66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8aab87640 1 -- 172.21.15.67:0/1340327043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560740215070 msgr2=0x5607405d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8aab87640 1 --2- 172.21.15.67:0/1340327043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560740215070 0x5607405d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8aab87640 1 -- 172.21.15.67:0/1340327043 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560740280650 con 0x5607401ec1b0 2024-07-31T14:53:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe8aab87640 1 --2- 172.21.15.67:0/1340327043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405c6120 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe89c005940 tx=0x7fe89c01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef3886231314c61 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe889ffb640 1 -- 172.21.15.67:0/1340327043 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c023040 con 0x5607401ec1b0 2024-07-31T14:53:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.310+0000 7fe889ffb640 1 -- 172.21.15.67:0/1340327043 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fe89c003d50 con 0x5607401ec1b0 2024-07-31T14:53:35.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.311+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/1340327043 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe89c015070 con 0x5607401ec1b0 2024-07-31T14:53:35.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.311+0000 7fe889ffb640 1 -- 172.21.15.67:0/1340327043 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe89c031050 con 0x5607401ec1b0 2024-07-31T14:53:35.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.311+0000 7fe8b2b3b740 10 client.16377 fetch_fsmap learned FSMap version 36 2024-07-31T14:53:35.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.311+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/1340327043 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5607405d4d60 con 0x5607401ec1b0 2024-07-31T14:53:35.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.311+0000 7fe889ffb640 1 -- 172.21.15.67:0/1340327043 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe89c015070 con 0x5607401ec1b0 2024-07-31T14:53:35.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.311+0000 7fe889ffb640 1 -- 172.21.15.67:0/1340327043 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 36) ==== 8402+0+0 (secure 0 0 0) 0x7fe89c004960 con 0x5607401ec1b0 2024-07-31T14:53:35.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.311+0000 7fe8b2b3b740 10 client.16377 fetch_fsmap finished waiting for FSMap version 36 2024-07-31T14:53:35.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.311+0000 7fe8b2b3b740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:35.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.311+0000 7fe8b2b3b740 20 client.16377 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:35.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.311+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/1340327043 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5607401d39f0 0x5607405d81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.311+0000 7fe8b2b3b740 4 client.16377 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:53:35.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.311+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/1340327043 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560740532ab0 con 0x5607401d39f0 2024-07-31T14:53:35.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.311+0000 7fe8a9b85640 1 --2- 172.21.15.67:0/1340327043 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5607401d39f0 0x5607405d81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.312+0000 7fe8a9b85640 1 --2- 172.21.15.67:0/1340327043 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5607401d39f0 0x5607405d81d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b0e2e6bf4198e7 server_cookie=dadcb304b36abf6f in_seq=0 out_seq=0 2024-07-31T14:53:35.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.312+0000 7fe889ffb640 10 client.16377 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.314+0000 7fe889ffb640 1 -- 172.21.15.67:0/1340327043 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560740532ab0 con 0x5607401d39f0 2024-07-31T14:53:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.314+0000 7fe889ffb640 10 client.16377 handle_command_reply: tid=0 2024-07-31T14:53:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.314+0000 7fe8b2b3b740 1 client.16377 shutdown 2024-07-31T14:53:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.315+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/1340327043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 msgr2=0x5607405c6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.315+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/1340327043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405c6120 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe89c005940 tx=0x7fe89c01abd0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.315+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/1340327043 shutdown_connections 2024-07-31T14:53:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.315+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/1340327043 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5607401d39f0 0x5607405d81d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.315+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/1340327043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560740215070 0x5607405d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.315+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/1340327043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607401f4390 0x5607405c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.315+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/1340327043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405c6120 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.315+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/1340327043 >> 172.21.15.67:0/1340327043 conn(0x560740209460 msgr2=0x5607405b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:35.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.315+0000 7fe889ffb640 0 client.16377 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:35.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.315+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/1340327043 shutdown_connections 2024-07-31T14:53:35.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.315+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/1340327043 wait complete. 2024-07-31T14:53:35.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.315+0000 7fe8b2b3b740 20 client.16377 trim_cache size 0 max 16384 2024-07-31T14:53:35.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8b2b3b740 1 Processor -- start 2024-07-31T14:53:35.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8b2b3b740 1 -- start start 2024-07-31T14:53:35.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8b2b3b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8b2b3b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607401f4390 0x5607405c0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8b2b3b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560740215070 0x5607405d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8b2b3b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a41b29e0 con 0x5607401ec1b0 2024-07-31T14:53:35.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8b2b3b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a41b45e0 con 0x5607401f4390 2024-07-31T14:53:35.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8b2b3b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a41b30b0 con 0x560740215070 2024-07-31T14:53:35.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8aab87640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8aab87640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405b7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60204/0 (socket says 172.21.15.67:60204) 2024-07-31T14:53:35.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8aab87640 1 -- 172.21.15.67:0/4044772360 learned_addr learned my addr 172.21.15.67:0/4044772360 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:35.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8a9b85640 1 --2- 172.21.15.67:0/4044772360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607401f4390 0x5607405c0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8aab87640 1 --2- 172.21.15.67:0/4044772360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560740215070 0x5607405d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8aab87640 1 -- 172.21.15.67:0/4044772360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560740215070 msgr2=0x5607405d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:35.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8aab87640 1 --2- 172.21.15.67:0/4044772360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560740215070 0x5607405d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8aab87640 1 -- 172.21.15.67:0/4044772360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607401f4390 msgr2=0x5607405c0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:35.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8aab87640 1 --2- 172.21.15.67:0/4044772360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607401f4390 0x5607405c0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.336+0000 7fe8aab87640 1 -- 172.21.15.67:0/4044772360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607405c4e80 con 0x5607401ec1b0 2024-07-31T14:53:35.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.337+0000 7fe8aab87640 1 --2- 172.21.15.67:0/4044772360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405b7300 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe89c01ad40 tx=0x7fe89c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf82cc87ea83ce2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:35.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.337+0000 7fe88bfff640 1 -- 172.21.15.67:0/4044772360 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c003b30 con 0x5607401ec1b0 2024-07-31T14:53:35.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.337+0000 7fe88bfff640 1 -- 172.21.15.67:0/4044772360 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe89c015020 con 0x5607401ec1b0 2024-07-31T14:53:35.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.337+0000 7fe88bfff640 1 -- 172.21.15.67:0/4044772360 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c011a10 con 0x5607401ec1b0 2024-07-31T14:53:35.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.338+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/4044772360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 msgr2=0x5607405b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:35.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.338+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/4044772360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405b7300 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe89c01ad40 tx=0x7fe89c01acd0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.338+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/4044772360 shutdown_connections 2024-07-31T14:53:35.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.338+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/4044772360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560740215070 0x5607405d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.338+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/4044772360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607401f4390 0x5607405c0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.338+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/4044772360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405b7300 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.338+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/4044772360 >> 172.21.15.67:0/4044772360 conn(0x560740209460 msgr2=0x5607405b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:35.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.338+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/4044772360 shutdown_connections 2024-07-31T14:53:35.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.338+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/4044772360 wait complete. 2024-07-31T14:53:35.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.338+0000 7fe8b2b3b740 1 -- start start 2024-07-31T14:53:35.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.338+0000 7fe8b2b3b740 1 Processor -- start 2024-07-31T14:53:35.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8b2b3b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8b2b3b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607401f4390 0x5607405c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8b2b3b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560740215070 0x5607406387f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8b2b3b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a41b3f10 con 0x5607401ec1b0 2024-07-31T14:53:35.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8b2b3b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a41b3780 con 0x5607401f4390 2024-07-31T14:53:35.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8b2b3b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a41b09c0 con 0x560740215070 2024-07-31T14:53:35.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8b2b3b740 10 client.0 _ll_register_callbacks cb 0x7fe898009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:35.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8aab87640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8a9b85640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607401f4390 0x5607405c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8aab87640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60216/0 (socket says 172.21.15.67:60216) 2024-07-31T14:53:35.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8aab87640 1 -- 172.21.15.67:0/2611557142 learned_addr learned my addr 172.21.15.67:0/2611557142 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:35.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8aab87640 1 --2- 172.21.15.67:0/2611557142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560740215070 0x5607406387f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8aab87640 1 -- 172.21.15.67:0/2611557142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560740215070 msgr2=0x5607406387f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8aab87640 1 --2- 172.21.15.67:0/2611557142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560740215070 0x5607406387f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8aab87640 1 -- 172.21.15.67:0/2611557142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607401f4390 msgr2=0x5607405c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8aab87640 1 --2- 172.21.15.67:0/2611557142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607401f4390 0x5607405c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8aab87640 1 -- 172.21.15.67:0/2611557142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560740638e60 con 0x5607401ec1b0 2024-07-31T14:53:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8a9b85640 1 --2- 172.21.15.67:0/2611557142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607401f4390 0x5607405c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe8aab87640 1 --2- 172.21.15.67:0/2611557142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405c96c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe89c004450 tx=0x7fe89c0116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89846b088342f2e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.339+0000 7fe889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.340+0000 7fe889ffb640 1 -- 172.21.15.67:0/2611557142 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c0047c0 con 0x5607401ec1b0 2024-07-31T14:53:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.340+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/2611557142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe89c015020 con 0x5607401ec1b0 2024-07-31T14:53:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.340+0000 7fe889ffb640 1 -- 172.21.15.67:0/2611557142 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fe89c004960 con 0x5607401ec1b0 2024-07-31T14:53:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.340+0000 7fe889ffb640 1 -- 172.21.15.67:0/2611557142 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe89c003e70 con 0x5607401ec1b0 2024-07-31T14:53:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.340+0000 7fe8b2b3b740 10 client.16383 fetch_fsmap learned FSMap version 36 2024-07-31T14:53:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.340+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/2611557142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560740640170 con 0x5607401ec1b0 2024-07-31T14:53:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.340+0000 7fe889ffb640 1 -- 172.21.15.67:0/2611557142 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe89c015020 con 0x5607401ec1b0 2024-07-31T14:53:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.340+0000 7fe889ffb640 1 -- 172.21.15.67:0/2611557142 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 36) ==== 8402+0+0 (secure 0 0 0) 0x7fe89c035430 con 0x5607401ec1b0 2024-07-31T14:53:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.340+0000 7fe8b2b3b740 10 client.16383 fetch_fsmap finished waiting for FSMap version 36 2024-07-31T14:53:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.340+0000 7fe8b2b3b740 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.340+0000 7fe8b2b3b740 20 client.16383 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.340+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/2611557142 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5607401a8da0 0x560740642850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.340+0000 7fe8b2b3b740 4 client.16383 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:53:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.340+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/2611557142 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56074052e550 con 0x5607401a8da0 2024-07-31T14:53:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.341+0000 7fe8a9b85640 1 --2- 172.21.15.67:0/2611557142 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5607401a8da0 0x560740642850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.341+0000 7fe889ffb640 10 client.16383 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.341+0000 7fe8a9b85640 1 --2- 172.21.15.67:0/2611557142 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5607401a8da0 0x560740642850 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=726d1163636eaee1 server_cookie=eee410cf428a9003 in_seq=0 out_seq=0 2024-07-31T14:53:35.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.344+0000 7fe889ffb640 1 -- 172.21.15.67:0/2611557142 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x56074052e550 con 0x5607401a8da0 2024-07-31T14:53:35.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.344+0000 7fe889ffb640 10 client.16383 handle_command_reply: tid=0 2024-07-31T14:53:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.344+0000 7fe8b2b3b740 1 client.16383 shutdown 2024-07-31T14:53:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.344+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/2611557142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 msgr2=0x5607405c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.344+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/2611557142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405c96c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe89c004450 tx=0x7fe89c0116c0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.344+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/2611557142 shutdown_connections 2024-07-31T14:53:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.344+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/2611557142 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5607401a8da0 0x560740642850 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.344+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/2611557142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560740215070 0x5607406387f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.344+0000 7fe889ffb640 0 client.16383 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.344+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/2611557142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607401f4390 0x5607405c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.344+0000 7fe8b2b3b740 1 --2- 172.21.15.67:0/2611557142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607401ec1b0 0x5607405c96c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.344+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/2611557142 >> 172.21.15.67:0/2611557142 conn(0x560740209460 msgr2=0x5607405b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.345+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/2611557142 shutdown_connections 2024-07-31T14:53:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.345+0000 7fe8b2b3b740 1 -- 172.21.15.67:0/2611557142 wait complete. 2024-07-31T14:53:35.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.345+0000 7fe8b2b3b740 20 client.16383 trim_cache size 0 max 16384 2024-07-31T14:53:35.349 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:53:35.349 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (1206 inodes in the stack)", 2024-07-31T14:53:35.349 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T14:53:35.349 INFO:teuthology.orchestra.run.smithi067.stdout: "649431d1-591c-4169-a259-86430c9699be": { 2024-07-31T14:53:35.349 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T14:53:35.349 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "649431d1-591c-4169-a259-86430c9699be", 2024-07-31T14:53:35.349 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T14:53:35.349 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:53:35.349 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:53:35.350 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:53:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.345+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/89318997 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe88c0b3d40 msgr2=0x7fe88c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.345+0000 7fe8ac5ea640 1 --2- 172.21.15.67:0/89318997 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe88c0b3d40 0x7fe88c0b6250 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe8940e1e00 tx=0x7fe894000f40 comp rx=0 tx=0).stop 2024-07-31T14:53:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.345+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/89318997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a4102490 msgr2=0x7fe8a419c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.345+0000 7fe8ac5ea640 1 --2- 172.21.15.67:0/89318997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a4102490 0x7fe8a419c140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe89400ebc0 tx=0x7fe89400c6a0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.345+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/89318997 shutdown_connections 2024-07-31T14:53:35.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.345+0000 7fe8ac5ea640 1 --2- 172.21.15.67:0/89318997 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe88c0b3d40 0x7fe88c0b6250 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.345+0000 7fe8ac5ea640 1 --2- 172.21.15.67:0/89318997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8a4108720 0x7fe8a41a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.345+0000 7fe8ac5ea640 1 --2- 172.21.15.67:0/89318997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a4107dd0 0x7fe8a419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.345+0000 7fe8ac5ea640 1 --2- 172.21.15.67:0/89318997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a4102490 0x7fe8a419c140 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:35.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.345+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/89318997 >> 172.21.15.67:0/89318997 conn(0x7fe8a4079160 msgr2=0x7fe8a4104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:35.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.346+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/89318997 shutdown_connections 2024-07-31T14:53:35.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:53:35.346+0000 7fe8ac5ea640 1 -- 172.21.15.67:0/89318997 wait complete. 2024-07-31T14:53:35.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:34 smithi073 ceph-mon[36714]: pgmap v475: 129 pgs: 129 active+clean; 157 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 425 op/s 2024-07-31T14:53:35.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:35.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:35.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:35.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:36.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:36.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:36.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:36.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:36.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:36.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.166 INFO:tasks.cephfs.filesystem:scrub status for tag:649431d1-591c-4169-a259-86430c9699be - {'path': '/', 'tag': '649431d1-591c-4169-a259-86430c9699be', 'options': 'recursive,force'} 2024-07-31T14:53:36.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:36.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:36.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:36.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:36.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:36.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:36.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: pgmap v476: 129 pgs: 129 active+clean; 162 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 549 op/s 2024-07-31T14:53:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:37.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:37.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:37.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: pgmap v476: 129 pgs: 129 active+clean; 162 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 549 op/s 2024-07-31T14:53:37.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:37.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:37.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:37.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:37.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:37.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:37.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:37.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:37.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: pgmap v476: 129 pgs: 129 active+clean; 162 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 549 op/s 2024-07-31T14:53:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:37.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:37.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:37.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:38.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:53:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:38.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:38.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:38.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:38.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:53:38.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:53:38.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:53:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:38 smithi077 ceph-mon[37327]: pgmap v477: 129 pgs: 129 active+clean; 174 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 625 op/s 2024-07-31T14:53:39.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:38 smithi067 ceph-mon[32750]: pgmap v477: 129 pgs: 129 active+clean; 174 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 625 op/s 2024-07-31T14:53:39.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:38 smithi073 ceph-mon[36714]: pgmap v477: 129 pgs: 129 active+clean; 174 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 625 op/s 2024-07-31T14:53:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:40 smithi077 ceph-mon[37327]: pgmap v478: 129 pgs: 129 active+clean; 186 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 936 op/s 2024-07-31T14:53:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:40 smithi067 ceph-mon[32750]: pgmap v478: 129 pgs: 129 active+clean; 186 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 936 op/s 2024-07-31T14:53:41.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:40 smithi073 ceph-mon[36714]: pgmap v478: 129 pgs: 129 active+clean; 186 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 936 op/s 2024-07-31T14:53:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.463+0000 7f5de4c4b640 1 -- 172.21.15.67:0/303505096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de010d7a0 msgr2=0x7f5de010fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.463+0000 7f5de4c4b640 1 --2- 172.21.15.67:0/303505096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de010d7a0 0x7f5de010fc90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009ca0 tx=0x7f5dd802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.463+0000 7f5de4c4b640 1 -- 172.21.15.67:0/303505096 shutdown_connections 2024-07-31T14:53:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.463+0000 7f5de4c4b640 1 --2- 172.21.15.67:0/303505096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de010d7a0 0x7f5de010fc90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.463+0000 7f5de4c4b640 1 --2- 172.21.15.67:0/303505096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5de0104c90 0x7f5de010d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.463+0000 7f5de4c4b640 1 --2- 172.21.15.67:0/303505096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0104340 0x7f5de0104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.463+0000 7f5de4c4b640 1 -- 172.21.15.67:0/303505096 >> 172.21.15.67:0/303505096 conn(0x7f5de00fddf0 msgr2=0x7f5de0100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.463+0000 7f5de4c4b640 1 -- 172.21.15.67:0/303505096 shutdown_connections 2024-07-31T14:53:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.463+0000 7f5de4c4b640 1 -- 172.21.15.67:0/303505096 wait complete. 2024-07-31T14:53:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.464+0000 7f5de4c4b640 1 Processor -- start 2024-07-31T14:53:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.464+0000 7f5de4c4b640 1 -- start start 2024-07-31T14:53:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.464+0000 7f5de4c4b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0104340 0x7f5de0199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.464+0000 7f5de4c4b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5de0104c90 0x7f5de019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.464+0000 7f5de4c4b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de010d7a0 0x7f5de019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.464+0000 7f5de4c4b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0113410 con 0x7f5de010d7a0 2024-07-31T14:53:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.464+0000 7f5de4c4b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0113290 con 0x7f5de0104340 2024-07-31T14:53:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.464+0000 7f5de4c4b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0113590 con 0x7f5de0104c90 2024-07-31T14:53:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.464+0000 7f5dded9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de010d7a0 0x7f5de019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.464+0000 7f5dd7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5de0104c90 0x7f5de019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.464+0000 7f5dded9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de010d7a0 0x7f5de019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54588/0 (socket says 172.21.15.67:54588) 2024-07-31T14:53:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.464+0000 7f5dded9d640 1 -- 172.21.15.67:0/633209762 learned_addr learned my addr 172.21.15.67:0/633209762 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.464+0000 7f5dde59c640 1 --2- 172.21.15.67:0/633209762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0104340 0x7f5de0199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.465+0000 7f5dd7fff640 1 -- 172.21.15.67:0/633209762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0104340 msgr2=0x7f5de0199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.465+0000 7f5dd7fff640 1 --2- 172.21.15.67:0/633209762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0104340 0x7f5de0199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.465+0000 7f5dd7fff640 1 -- 172.21.15.67:0/633209762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de010d7a0 msgr2=0x7f5de019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.465+0000 7f5dd7fff640 1 --2- 172.21.15.67:0/633209762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de010d7a0 0x7f5de019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.465+0000 7f5dd7fff640 1 -- 172.21.15.67:0/633209762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de019fd00 con 0x7f5de0104c90 2024-07-31T14:53:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.465+0000 7f5dded9d640 1 --2- 172.21.15.67:0/633209762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de010d7a0 0x7f5de019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.465+0000 7f5dd7fff640 1 --2- 172.21.15.67:0/633209762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5de0104c90 0x7f5de019a370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5de006c390 tx=0x7f5dc800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.465+0000 7f5dd77fe640 1 -- 172.21.15.67:0/633209762 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc800cc40 con 0x7f5de0104c90 2024-07-31T14:53:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.465+0000 7f5de4c4b640 1 -- 172.21.15.67:0/633209762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de019fff0 con 0x7f5de0104c90 2024-07-31T14:53:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.465+0000 7f5dd77fe640 1 -- 172.21.15.67:0/633209762 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dc800cde0 con 0x7f5de0104c90 2024-07-31T14:53:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.465+0000 7f5dd77fe640 1 -- 172.21.15.67:0/633209762 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8018760 con 0x7f5de0104c90 2024-07-31T14:53:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.465+0000 7f5de4c4b640 1 -- 172.21.15.67:0/633209762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de0101f50 con 0x7f5de0104c90 2024-07-31T14:53:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5debc23740 1 Processor -- start 2024-07-31T14:53:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5debc23740 1 -- start start 2024-07-31T14:53:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5debc23740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb621b0 0x5625cff046b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5debc23740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb392f0 0x5625cff04bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5debc23740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb414d0 0x5625cff05130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5debc23740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5625cfe7a3b0 con 0x5625cfb621b0 2024-07-31T14:53:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5debc23740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5625cfe698b0 con 0x5625cfb414d0 2024-07-31T14:53:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5debc23740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5625cfe6d8f0 con 0x5625cfb392f0 2024-07-31T14:53:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5dded9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb621b0 0x5625cff046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5dded9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb621b0 0x5625cff046b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54594/0 (socket says 172.21.15.67:54594) 2024-07-31T14:53:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5dded9d640 1 -- 172.21.15.67:0/1178469138 learned_addr learned my addr 172.21.15.67:0/1178469138 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5dde59c640 1 --2- 172.21.15.67:0/1178469138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb392f0 0x5625cff04bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5dd77fe640 1 -- 172.21.15.67:0/633209762 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5dc8010a90 con 0x7f5de0104c90 2024-07-31T14:53:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5dded9d640 1 -- 172.21.15.67:0/1178469138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb392f0 msgr2=0x5625cff04bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5dded9d640 1 --2- 172.21.15.67:0/1178469138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb392f0 0x5625cff04bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5dded9d640 1 -- 172.21.15.67:0/1178469138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb414d0 msgr2=0x5625cff05130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:53:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5dded9d640 1 --2- 172.21.15.67:0/1178469138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb414d0 0x5625cff05130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.468+0000 7f5dded9d640 1 -- 172.21.15.67:0/1178469138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625cfe109a0 con 0x5625cfb621b0 2024-07-31T14:53:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.469+0000 7f5dde59c640 1 --2- 172.21.15.67:0/1178469138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb392f0 0x5625cff04bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.469+0000 7f5dded9d640 1 --2- 172.21.15.67:0/1178469138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb621b0 0x5625cff046b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80094e0 tx=0x7f5dd802f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384c7625e6905608 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.469+0000 7f5dd4ff9640 1 -- 172.21.15.67:0/1178469138 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8031650 con 0x5625cfb621b0 2024-07-31T14:53:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.469+0000 7f5dd4ff9640 1 -- 172.21.15.67:0/1178469138 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd8036040 con 0x5625cfb621b0 2024-07-31T14:53:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.469+0000 7f5dd4ff9640 1 -- 172.21.15.67:0/1178469138 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8004560 con 0x5625cfb621b0 2024-07-31T14:53:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.469+0000 7f5dd77fe640 1 --2- 172.21.15.67:0/633209762 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5dcc0b3d40 0x7f5dcc0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.470+0000 7f5dde59c640 1 --2- 172.21.15.67:0/633209762 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5dcc0b3d40 0x7f5dcc0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.470+0000 7f5dd77fe640 1 -- 172.21.15.67:0/633209762 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f5dc80e45c0 con 0x7f5de0104c90 2024-07-31T14:53:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.470+0000 7f5debc23740 1 -- 172.21.15.67:0/1178469138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb621b0 msgr2=0x5625cff046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.470+0000 7f5debc23740 1 --2- 172.21.15.67:0/1178469138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb621b0 0x5625cff046b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80094e0 tx=0x7f5dd802f390 comp rx=0 tx=0).stop 2024-07-31T14:53:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.470+0000 7f5debc23740 1 -- 172.21.15.67:0/1178469138 shutdown_connections 2024-07-31T14:53:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.470+0000 7f5debc23740 1 --2- 172.21.15.67:0/1178469138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb414d0 0x5625cff05130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.470+0000 7f5debc23740 1 --2- 172.21.15.67:0/1178469138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb392f0 0x5625cff04bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.470+0000 7f5debc23740 1 --2- 172.21.15.67:0/1178469138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb621b0 0x5625cff046b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.470+0000 7f5debc23740 1 -- 172.21.15.67:0/1178469138 >> 172.21.15.67:0/1178469138 conn(0x5625cfb56300 msgr2=0x5625cff025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.470+0000 7f5dde59c640 1 --2- 172.21.15.67:0/633209762 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5dcc0b3d40 0x7f5dcc0b6250 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5625cff03330 tx=0x7f5dd000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.470+0000 7f5debc23740 1 -- 172.21.15.67:0/1178469138 shutdown_connections 2024-07-31T14:53:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.470+0000 7f5debc23740 1 -- 172.21.15.67:0/1178469138 wait complete. 2024-07-31T14:53:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.470+0000 7f5debc23740 1 -- start start 2024-07-31T14:53:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.471+0000 7f5debc23740 1 Processor -- start 2024-07-31T14:53:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.471+0000 7f5debc23740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff101e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.471+0000 7f5debc23740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb414d0 0x5625cff10720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.471+0000 7f5debc23740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb621b0 0x5625cff1bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.471+0000 7f5debc23740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5625cfda3ab0 con 0x5625cfb392f0 2024-07-31T14:53:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.471+0000 7f5debc23740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5625cfdf1cf0 con 0x5625cfb414d0 2024-07-31T14:53:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.471+0000 7f5debc23740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5625cff12c50 con 0x5625cfb621b0 2024-07-31T14:53:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.471+0000 7f5debc23740 10 client.0 _ll_register_callbacks cb 0x5625cfe633e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.471+0000 7f5dde59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.471+0000 7f5dded9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb621b0 0x5625cff1bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.471+0000 7f5dde59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff101e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54602/0 (socket says 172.21.15.67:54602) 2024-07-31T14:53:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.471+0000 7f5dde59c640 1 -- 172.21.15.67:0/3014496953 learned_addr learned my addr 172.21.15.67:0/3014496953 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.471+0000 7f5dd7fff640 1 --2- 172.21.15.67:0/3014496953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb414d0 0x5625cff10720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.472+0000 7f5dded9d640 1 -- 172.21.15.67:0/3014496953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb414d0 msgr2=0x5625cff10720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.472+0000 7f5dded9d640 1 --2- 172.21.15.67:0/3014496953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb414d0 0x5625cff10720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.472+0000 7f5dded9d640 1 -- 172.21.15.67:0/3014496953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 msgr2=0x5625cff101e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.472+0000 7f5dded9d640 1 --2- 172.21.15.67:0/3014496953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff101e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.472+0000 7f5dded9d640 1 -- 172.21.15.67:0/3014496953 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625cfbcd790 con 0x5625cfb621b0 2024-07-31T14:53:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.472+0000 7f5dde59c640 1 --2- 172.21.15.67:0/3014496953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.472+0000 7f5dded9d640 1 --2- 172.21.15.67:0/3014496953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb621b0 0x5625cff1bf40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8005e30 tx=0x7f5dd8004650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d97421d148fac0a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.472+0000 7f5dc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:53:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.472+0000 7f5dc5ffb640 1 -- 172.21.15.67:0/3014496953 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8052020 con 0x5625cfb621b0 2024-07-31T14:53:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.472+0000 7f5debc23740 1 -- 172.21.15.67:0/3014496953 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd8036040 con 0x5625cfb621b0 2024-07-31T14:53:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.472+0000 7f5dc5ffb640 1 -- 172.21.15.67:0/3014496953 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f5dd8003b80 con 0x5625cfb621b0 2024-07-31T14:53:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.472+0000 7f5dc5ffb640 1 -- 172.21.15.67:0/3014496953 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5dd80098a0 con 0x5625cfb621b0 2024-07-31T14:53:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.472+0000 7f5debc23740 10 client.26053 fetch_fsmap learned FSMap version 36 2024-07-31T14:53:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.472+0000 7f5debc23740 1 -- 172.21.15.67:0/3014496953 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5625cfc46620 con 0x5625cfb621b0 2024-07-31T14:53:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.473+0000 7f5dc5ffb640 1 -- 172.21.15.67:0/3014496953 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd8036040 con 0x5625cfb621b0 2024-07-31T14:53:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.473+0000 7f5dc5ffb640 1 -- 172.21.15.67:0/3014496953 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8402+0+0 (secure 0 0 0) 0x7f5dd8044070 con 0x5625cfb621b0 2024-07-31T14:53:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.473+0000 7f5debc23740 10 client.26053 fetch_fsmap finished waiting for FSMap version 36 2024-07-31T14:53:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.473+0000 7f5debc23740 10 client.26053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.473+0000 7f5debc23740 20 client.26053 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.473+0000 7f5debc23740 1 --2- 172.21.15.67:0/3014496953 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5625cfb20d80 0x5625cff21d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.473+0000 7f5debc23740 4 client.26053 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:53:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.473+0000 7f5debc23740 1 -- 172.21.15.67:0/3014496953 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625cfe7c300 con 0x5625cfb20d80 2024-07-31T14:53:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.473+0000 7f5dde59c640 1 --2- 172.21.15.67:0/3014496953 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5625cfb20d80 0x5625cff21d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.474+0000 7f5dc5ffb640 10 client.26053 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.474+0000 7f5dde59c640 1 --2- 172.21.15.67:0/3014496953 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5625cfb20d80 0x5625cff21d60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f7d36402d1a82d server_cookie=270a636799df5dc3 in_seq=0 out_seq=0 2024-07-31T14:53:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.474+0000 7f5dc5ffb640 1 -- 172.21.15.67:0/3014496953 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625cfe7c300 con 0x5625cfb20d80 2024-07-31T14:53:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.474+0000 7f5dc5ffb640 10 client.26053 handle_command_reply: tid=0 2024-07-31T14:53:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.474+0000 7f5debc23740 1 client.26053 shutdown 2024-07-31T14:53:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.475+0000 7f5debc23740 1 -- 172.21.15.67:0/3014496953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb621b0 msgr2=0x5625cff1bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.475+0000 7f5debc23740 1 --2- 172.21.15.67:0/3014496953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb621b0 0x5625cff1bf40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8005e30 tx=0x7f5dd8004650 comp rx=0 tx=0).stop 2024-07-31T14:53:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.475+0000 7f5debc23740 1 -- 172.21.15.67:0/3014496953 shutdown_connections 2024-07-31T14:53:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.475+0000 7f5debc23740 1 --2- 172.21.15.67:0/3014496953 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5625cfb20d80 0x5625cff21d60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.475+0000 7f5debc23740 1 --2- 172.21.15.67:0/3014496953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb621b0 0x5625cff1bf40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.475+0000 7f5debc23740 1 --2- 172.21.15.67:0/3014496953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb414d0 0x5625cff10720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.475+0000 7f5debc23740 1 --2- 172.21.15.67:0/3014496953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.475+0000 7f5dc5ffb640 0 client.26053 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.475+0000 7f5debc23740 1 -- 172.21.15.67:0/3014496953 >> 172.21.15.67:0/3014496953 conn(0x5625cfb56300 msgr2=0x5625cff056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.475+0000 7f5debc23740 1 -- 172.21.15.67:0/3014496953 shutdown_connections 2024-07-31T14:53:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.475+0000 7f5debc23740 1 -- 172.21.15.67:0/3014496953 wait complete. 2024-07-31T14:53:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.475+0000 7f5debc23740 20 client.26053 trim_cache size 0 max 16384 2024-07-31T14:53:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.490+0000 7f5debc23740 1 Processor -- start 2024-07-31T14:53:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.490+0000 7f5debc23740 1 -- start start 2024-07-31T14:53:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.490+0000 7f5debc23740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff0fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.490+0000 7f5debc23740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb414d0 0x5625cff0a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.490+0000 7f5debc23740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb621b0 0x5625cff1bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.490+0000 7f5debc23740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01b2850 con 0x5625cfb392f0 2024-07-31T14:53:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.490+0000 7f5debc23740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01b4450 con 0x5625cfb621b0 2024-07-31T14:53:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.490+0000 7f5debc23740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01b2f20 con 0x5625cfb414d0 2024-07-31T14:53:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.490+0000 7f5dde59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff0fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.490+0000 7f5dd7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb414d0 0x5625cff0a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.490+0000 7f5dd7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb414d0 0x5625cff0a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58490/0 (socket says 172.21.15.67:58490) 2024-07-31T14:53:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.490+0000 7f5dde59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff0fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54614/0 (socket says 172.21.15.67:54614) 2024-07-31T14:53:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.490+0000 7f5dd7fff640 1 -- 172.21.15.67:0/2492573471 learned_addr learned my addr 172.21.15.67:0/2492573471 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.491+0000 7f5dded9d640 1 --2- 172.21.15.67:0/2492573471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb621b0 0x5625cff1bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.491+0000 7f5dd7fff640 1 -- 172.21.15.67:0/2492573471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb621b0 msgr2=0x5625cff1bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.491+0000 7f5dd7fff640 1 --2- 172.21.15.67:0/2492573471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb621b0 0x5625cff1bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.491+0000 7f5dd7fff640 1 -- 172.21.15.67:0/2492573471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 msgr2=0x5625cff0fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.491+0000 7f5dd7fff640 1 --2- 172.21.15.67:0/2492573471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff0fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.491+0000 7f5dd7fff640 1 -- 172.21.15.67:0/2492573471 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625cff0ef10 con 0x5625cfb414d0 2024-07-31T14:53:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.491+0000 7f5dde59c640 1 --2- 172.21.15.67:0/2492573471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff0fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.491+0000 7f5dd7fff640 1 --2- 172.21.15.67:0/2492573471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb414d0 0x5625cff0a780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80a81e0 tx=0x7f5dc80ab030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51320f5e1656e50e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.491+0000 7f5dc7fff640 1 -- 172.21.15.67:0/2492573471 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc80c5070 con 0x5625cfb414d0 2024-07-31T14:53:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.491+0000 7f5dc7fff640 1 -- 172.21.15.67:0/2492573471 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dc80af280 con 0x5625cfb414d0 2024-07-31T14:53:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.491+0000 7f5dc7fff640 1 -- 172.21.15.67:0/2492573471 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc80a6b40 con 0x5625cfb414d0 2024-07-31T14:53:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.492+0000 7f5debc23740 1 -- 172.21.15.67:0/2492573471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb414d0 msgr2=0x5625cff0a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.492+0000 7f5debc23740 1 --2- 172.21.15.67:0/2492573471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb414d0 0x5625cff0a780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80a81e0 tx=0x7f5dc80ab030 comp rx=0 tx=0).stop 2024-07-31T14:53:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.492+0000 7f5debc23740 1 -- 172.21.15.67:0/2492573471 shutdown_connections 2024-07-31T14:53:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.492+0000 7f5debc23740 1 --2- 172.21.15.67:0/2492573471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb621b0 0x5625cff1bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.492+0000 7f5debc23740 1 --2- 172.21.15.67:0/2492573471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb414d0 0x5625cff0a780 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.492+0000 7f5debc23740 1 --2- 172.21.15.67:0/2492573471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff0fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.492+0000 7f5debc23740 1 -- 172.21.15.67:0/2492573471 >> 172.21.15.67:0/2492573471 conn(0x5625cfb56300 msgr2=0x5625cff0f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.492+0000 7f5debc23740 1 -- 172.21.15.67:0/2492573471 shutdown_connections 2024-07-31T14:53:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.492+0000 7f5debc23740 1 -- 172.21.15.67:0/2492573471 wait complete. 2024-07-31T14:53:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.492+0000 7f5debc23740 1 -- start start 2024-07-31T14:53:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.492+0000 7f5debc23740 1 Processor -- start 2024-07-31T14:53:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5debc23740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff15290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5debc23740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb414d0 0x5625cff15810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5debc23740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb621b0 0x5625cff88400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5debc23740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01b3d80 con 0x5625cfb392f0 2024-07-31T14:53:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5debc23740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01b35f0 con 0x5625cfb621b0 2024-07-31T14:53:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5debc23740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01b0830 con 0x5625cfb414d0 2024-07-31T14:53:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5dde59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff15290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5dde59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff15290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54622/0 (socket says 172.21.15.67:54622) 2024-07-31T14:53:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5dde59c640 1 -- 172.21.15.67:0/3139966913 learned_addr learned my addr 172.21.15.67:0/3139966913 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5dd7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb414d0 0x5625cff15810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5debc23740 10 client.0 _ll_register_callbacks cb 0x7f5dd000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5dded9d640 1 --2- 172.21.15.67:0/3139966913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb621b0 0x5625cff88400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5dde59c640 1 -- 172.21.15.67:0/3139966913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb414d0 msgr2=0x5625cff15810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5dde59c640 1 --2- 172.21.15.67:0/3139966913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb414d0 0x5625cff15810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5dde59c640 1 -- 172.21.15.67:0/3139966913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb621b0 msgr2=0x5625cff88400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5dde59c640 1 --2- 172.21.15.67:0/3139966913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb621b0 0x5625cff88400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5dde59c640 1 -- 172.21.15.67:0/3139966913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625cff88940 con 0x5625cfb392f0 2024-07-31T14:53:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5dd7fff640 1 --2- 172.21.15.67:0/3139966913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb414d0 0x5625cff15810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5dde59c640 1 --2- 172.21.15.67:0/3139966913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff15290 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x5625cff223d0 tx=0x7f5dd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4712a8ff17e0a636 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.493+0000 7f5dc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.494+0000 7f5dc5ffb640 1 -- 172.21.15.67:0/3139966913 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd001ecf0 con 0x5625cfb392f0 2024-07-31T14:53:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.494+0000 7f5dc5ffb640 1 -- 172.21.15.67:0/3139966913 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f5dd0005020 con 0x5625cfb392f0 2024-07-31T14:53:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.494+0000 7f5debc23740 1 -- 172.21.15.67:0/3139966913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc80af280 con 0x5625cfb392f0 2024-07-31T14:53:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.494+0000 7f5dc5ffb640 1 -- 172.21.15.67:0/3139966913 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5dd003b050 con 0x5625cfb392f0 2024-07-31T14:53:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.494+0000 7f5debc23740 10 client.16401 fetch_fsmap learned FSMap version 36 2024-07-31T14:53:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.494+0000 7f5debc23740 1 -- 172.21.15.67:0/3139966913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5625cff89b10 con 0x5625cfb392f0 2024-07-31T14:53:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.494+0000 7f5dc5ffb640 1 -- 172.21.15.67:0/3139966913 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dc80af280 con 0x5625cfb392f0 2024-07-31T14:53:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.494+0000 7f5dc5ffb640 1 -- 172.21.15.67:0/3139966913 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 36) ==== 8402+0+0 (secure 0 0 0) 0x7f5dd002a380 con 0x5625cfb392f0 2024-07-31T14:53:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.494+0000 7f5debc23740 10 client.16401 fetch_fsmap finished waiting for FSMap version 36 2024-07-31T14:53:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.494+0000 7f5debc23740 10 client.16401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.494+0000 7f5debc23740 20 client.16401 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.494+0000 7f5debc23740 1 --2- 172.21.15.67:0/3139966913 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5625cfaf61c0 0x5625cff8c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.494+0000 7f5debc23740 4 client.16401 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:53:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.494+0000 7f5debc23740 1 -- 172.21.15.67:0/3139966913 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625cfe77da0 con 0x5625cfaf61c0 2024-07-31T14:53:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.495+0000 7f5dd7fff640 1 --2- 172.21.15.67:0/3139966913 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5625cfaf61c0 0x5625cff8c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.495+0000 7f5dc5ffb640 10 client.16401 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:42.495+0000 7f5dd7fff640 1 --2- 172.21.15.67:0/3139966913 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5625cfaf61c0 0x5625cff8c370 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be3b0f7835fff2a7 server_cookie=2fcbc9b419201e5b in_seq=0 out_seq=0 2024-07-31T14:53:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:42 smithi077 ceph-mon[37327]: pgmap v479: 129 pgs: 129 active+clean; 195 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 937 op/s 2024-07-31T14:53:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:42 smithi067 ceph-mon[32750]: pgmap v479: 129 pgs: 129 active+clean; 195 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 937 op/s 2024-07-31T14:53:43.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:42 smithi073 ceph-mon[36714]: pgmap v479: 129 pgs: 129 active+clean; 195 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 937 op/s 2024-07-31T14:53:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.457+0000 7f5dc5ffb640 1 -- 172.21.15.67:0/3139966913 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5625cfe77da0 con 0x5625cfaf61c0 2024-07-31T14:53:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.457+0000 7f5dc5ffb640 10 client.16401 handle_command_reply: tid=0 2024-07-31T14:53:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.457+0000 7f5debc23740 1 client.16401 shutdown 2024-07-31T14:53:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.458+0000 7f5debc23740 1 -- 172.21.15.67:0/3139966913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 msgr2=0x5625cff15290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.458+0000 7f5debc23740 1 --2- 172.21.15.67:0/3139966913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff15290 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x5625cff223d0 tx=0x7f5dd00046c0 comp rx=0 tx=0).stop 2024-07-31T14:53:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.458+0000 7f5debc23740 1 -- 172.21.15.67:0/3139966913 shutdown_connections 2024-07-31T14:53:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.458+0000 7f5debc23740 1 --2- 172.21.15.67:0/3139966913 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5625cfaf61c0 0x5625cff8c370 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.458+0000 7f5debc23740 1 --2- 172.21.15.67:0/3139966913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625cfb621b0 0x5625cff88400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.458+0000 7f5debc23740 1 --2- 172.21.15.67:0/3139966913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5625cfb414d0 0x5625cff15810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.458+0000 7f5dc5ffb640 0 client.16401 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.458+0000 7f5debc23740 1 --2- 172.21.15.67:0/3139966913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5625cfb392f0 0x5625cff15290 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.458+0000 7f5debc23740 1 -- 172.21.15.67:0/3139966913 >> 172.21.15.67:0/3139966913 conn(0x5625cfb56300 msgr2=0x5625cff113f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.458+0000 7f5debc23740 1 -- 172.21.15.67:0/3139966913 shutdown_connections 2024-07-31T14:53:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.458+0000 7f5debc23740 1 -- 172.21.15.67:0/3139966913 wait complete. 2024-07-31T14:53:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.458+0000 7f5debc23740 20 client.16401 trim_cache size 0 max 16384 2024-07-31T14:53:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:53:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:53:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:53:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:53:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.459+0000 7f5de4c4b640 1 -- 172.21.15.67:0/633209762 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5dcc0b3d40 msgr2=0x7f5dcc0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.459+0000 7f5de4c4b640 1 --2- 172.21.15.67:0/633209762 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5dcc0b3d40 0x7f5dcc0b6250 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5625cff03330 tx=0x7f5dd000b390 comp rx=0 tx=0).stop 2024-07-31T14:53:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.459+0000 7f5de4c4b640 1 -- 172.21.15.67:0/633209762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5de0104c90 msgr2=0x7f5de019a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.459+0000 7f5de4c4b640 1 --2- 172.21.15.67:0/633209762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5de0104c90 0x7f5de019a370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5de006c390 tx=0x7f5dc800ee50 comp rx=0 tx=0).stop 2024-07-31T14:53:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.459+0000 7f5de4c4b640 1 -- 172.21.15.67:0/633209762 shutdown_connections 2024-07-31T14:53:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.459+0000 7f5de4c4b640 1 --2- 172.21.15.67:0/633209762 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5dcc0b3d40 0x7f5dcc0b6250 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.459+0000 7f5de4c4b640 1 --2- 172.21.15.67:0/633209762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de010d7a0 0x7f5de019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.459+0000 7f5de4c4b640 1 --2- 172.21.15.67:0/633209762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5de0104c90 0x7f5de019a370 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.459+0000 7f5de4c4b640 1 --2- 172.21.15.67:0/633209762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0104340 0x7f5de0199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.459+0000 7f5de4c4b640 1 -- 172.21.15.67:0/633209762 >> 172.21.15.67:0/633209762 conn(0x7f5de00fddf0 msgr2=0x7f5de0100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.459+0000 7f5de4c4b640 1 -- 172.21.15.67:0/633209762 shutdown_connections 2024-07-31T14:53:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:43.459+0000 7f5de4c4b640 1 -- 172.21.15.67:0/633209762 wait complete. 2024-07-31T14:53:45.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:44 smithi077 ceph-mon[37327]: pgmap v480: 129 pgs: 129 active+clean; 217 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 1.28k op/s 2024-07-31T14:53:45.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:44 smithi067 ceph-mon[32750]: pgmap v480: 129 pgs: 129 active+clean; 217 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 1.28k op/s 2024-07-31T14:53:45.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:44 smithi073 ceph-mon[36714]: pgmap v480: 129 pgs: 129 active+clean; 217 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 1.28k op/s 2024-07-31T14:53:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:45 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:53:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:45 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:53:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:45 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:53:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:45 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:53:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:45 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:53:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:45 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:53:47.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:46 smithi077 ceph-mon[37327]: pgmap v481: 129 pgs: 129 active+clean; 222 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 1.38k op/s 2024-07-31T14:53:47.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:46 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:53:47.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:46 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:53:47.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:46 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:53:47.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:46 smithi067 ceph-mon[32750]: pgmap v481: 129 pgs: 129 active+clean; 222 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 1.38k op/s 2024-07-31T14:53:47.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:46 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:53:47.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:46 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:53:47.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:46 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:53:47.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:46 smithi073 ceph-mon[36714]: pgmap v481: 129 pgs: 129 active+clean; 222 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 1.38k op/s 2024-07-31T14:53:47.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:46 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:53:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:46 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:53:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:46 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:53:48.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:47 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:48.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:47 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:53:48.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:47 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:48.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:47 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:53:48.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:47 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:48.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:47 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:53:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:48 smithi067 ceph-mon[32750]: pgmap v482: 129 pgs: 129 active+clean; 214 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 1.55k op/s 2024-07-31T14:53:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:53:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:53:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:53:49.389 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:48 smithi077 ceph-mon[37327]: pgmap v482: 129 pgs: 129 active+clean; 214 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 1.55k op/s 2024-07-31T14:53:49.389 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:53:49.389 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:53:49.389 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:53:49.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:48 smithi073 ceph-mon[36714]: pgmap v482: 129 pgs: 129 active+clean; 214 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 1.55k op/s 2024-07-31T14:53:49.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:53:49.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:53:49.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:53:51.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:50 smithi067 ceph-mon[32750]: pgmap v483: 129 pgs: 129 active+clean; 212 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 1.42k op/s 2024-07-31T14:53:51.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:50 smithi073 ceph-mon[36714]: pgmap v483: 129 pgs: 129 active+clean; 212 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 1.42k op/s 2024-07-31T14:53:51.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:50 smithi077 ceph-mon[37327]: pgmap v483: 129 pgs: 129 active+clean; 212 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 1.42k op/s 2024-07-31T14:53:53.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:53 smithi067 ceph-mon[32750]: pgmap v484: 129 pgs: 129 active+clean; 212 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 1.08k op/s 2024-07-31T14:53:53.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:53 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:53 smithi073 ceph-mon[36714]: pgmap v484: 129 pgs: 129 active+clean; 212 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 1.08k op/s 2024-07-31T14:53:53.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:53 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:53.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:53 smithi077 ceph-mon[37327]: pgmap v484: 129 pgs: 129 active+clean; 212 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 1.08k op/s 2024-07-31T14:53:53.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:53 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:53:55.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:55 smithi067 ceph-mon[32750]: pgmap v485: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 992 op/s 2024-07-31T14:53:55.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:55 smithi073 ceph-mon[36714]: pgmap v485: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 992 op/s 2024-07-31T14:53:55.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:55 smithi077 ceph-mon[37327]: pgmap v485: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 992 op/s 2024-07-31T14:53:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.576+0000 7fb3e03be640 1 -- 172.21.15.67:0/2581373854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3d8107d20 msgr2=0x7fb3d8108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.576+0000 7fb3e03be640 1 --2- 172.21.15.67:0/2581373854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3d8107d20 0x7fb3d8108130 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009900 tx=0x7fb3c802a410 comp rx=0 tx=0).stop 2024-07-31T14:53:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.576+0000 7fb3e03be640 1 -- 172.21.15.67:0/2581373854 shutdown_connections 2024-07-31T14:53:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.576+0000 7fb3e03be640 1 --2- 172.21.15.67:0/2581373854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3d8108670 0x7fb3d810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.576+0000 7fb3e03be640 1 --2- 172.21.15.67:0/2581373854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3d8107d20 0x7fb3d8108130 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.576+0000 7fb3e03be640 1 --2- 172.21.15.67:0/2581373854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d81023f0 0x7fb3d8102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.576+0000 7fb3e03be640 1 -- 172.21.15.67:0/2581373854 >> 172.21.15.67:0/2581373854 conn(0x7fb3d80fddf0 msgr2=0x7fb3d8100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.576+0000 7fb3e03be640 1 -- 172.21.15.67:0/2581373854 shutdown_connections 2024-07-31T14:53:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.576+0000 7fb3e03be640 1 -- 172.21.15.67:0/2581373854 wait complete. 2024-07-31T14:53:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.577+0000 7fb3e03be640 1 Processor -- start 2024-07-31T14:53:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.577+0000 7fb3e03be640 1 -- start start 2024-07-31T14:53:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.577+0000 7fb3e03be640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d81023f0 0x7fb3d8198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.577+0000 7fb3e03be640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3d8107d20 0x7fb3d8199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.577+0000 7fb3e03be640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3d8108670 0x7fb3d819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.577+0000 7fb3e03be640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81122e0 con 0x7fb3d8107d20 2024-07-31T14:53:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.577+0000 7fb3e03be640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8112160 con 0x7fb3d81023f0 2024-07-31T14:53:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.577+0000 7fb3e03be640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8112460 con 0x7fb3d8108670 2024-07-31T14:53:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.577+0000 7fb3dd959640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3d8107d20 0x7fb3d8199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.577+0000 7fb3dd959640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3d8107d20 0x7fb3d8199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33190/0 (socket says 172.21.15.67:33190) 2024-07-31T14:53:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.577+0000 7fb3dd959640 1 -- 172.21.15.67:0/838997596 learned_addr learned my addr 172.21.15.67:0/838997596 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.577+0000 7fb3de95b640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3d8108670 0x7fb3d819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.577+0000 7fb3de15a640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d81023f0 0x7fb3d8198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.578+0000 7fb3dd959640 1 -- 172.21.15.67:0/838997596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3d8108670 msgr2=0x7fb3d819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.578+0000 7fb3dd959640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3d8108670 0x7fb3d819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.578+0000 7fb3dd959640 1 -- 172.21.15.67:0/838997596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d81023f0 msgr2=0x7fb3d8198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.578+0000 7fb3dd959640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d81023f0 0x7fb3d8198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.578+0000 7fb3dd959640 1 -- 172.21.15.67:0/838997596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d819ec90 con 0x7fb3d8107d20 2024-07-31T14:53:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.578+0000 7fb3de95b640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3d8108670 0x7fb3d819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.578+0000 7fb3dd959640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3d8107d20 0x7fb3d8199300 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb3c802a930 tx=0x7fb3c8031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.578+0000 7fb3cf7fe640 1 -- 172.21.15.67:0/838997596 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c803f030 con 0x7fb3d8107d20 2024-07-31T14:53:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.578+0000 7fb3cf7fe640 1 -- 172.21.15.67:0/838997596 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3c8004490 con 0x7fb3d8107d20 2024-07-31T14:53:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.578+0000 7fb3cf7fe640 1 -- 172.21.15.67:0/838997596 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8035070 con 0x7fb3d8107d20 2024-07-31T14:53:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.578+0000 7fb3e03be640 1 -- 172.21.15.67:0/838997596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d819ef20 con 0x7fb3d8107d20 2024-07-31T14:53:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.578+0000 7fb3e03be640 1 -- 172.21.15.67:0/838997596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d81a5710 con 0x7fb3d8107d20 2024-07-31T14:53:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.581+0000 7fb3cf7fe640 1 -- 172.21.15.67:0/838997596 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb3c8042030 con 0x7fb3d8107d20 2024-07-31T14:53:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.581+0000 7fb3e738a740 1 Processor -- start 2024-07-31T14:53:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.582+0000 7fb3e738a740 1 -- start start 2024-07-31T14:53:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.582+0000 7fb3e738a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43221b0 0x55aeb46c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.582+0000 7fb3e738a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb42f92f0 0x55aeb46c5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.582+0000 7fb3e738a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 0x55aeb46c5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.582+0000 7fb3e738a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55aeb463a3b0 con 0x55aeb43221b0 2024-07-31T14:53:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.582+0000 7fb3e738a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55aeb46298b0 con 0x55aeb43014d0 2024-07-31T14:53:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.582+0000 7fb3e738a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55aeb462d8f0 con 0x55aeb42f92f0 2024-07-31T14:53:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.582+0000 7fb3cf7fe640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb3c40b3d40 0x7fb3c40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.582+0000 7fb3de15a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb42f92f0 0x55aeb46c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.582+0000 7fb3de15a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb42f92f0 0x55aeb46c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:41686/0 (socket says 172.21.15.67:41686) 2024-07-31T14:53:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.582+0000 7fb3de15a640 1 -- 172.21.15.67:0/349839808 learned_addr learned my addr 172.21.15.67:0/349839808 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.582+0000 7fb3de95b640 1 --2- 172.21.15.67:0/349839808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43221b0 0x55aeb46c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.582+0000 7fb3dd959640 1 --2- 172.21.15.67:0/349839808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 0x55aeb46c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.583+0000 7fb3de95b640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb3c40b3d40 0x7fb3c40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.583+0000 7fb3cf7fe640 1 -- 172.21.15.67:0/838997596 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fb3c81042a0 con 0x7fb3d8107d20 2024-07-31T14:53:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.583+0000 7fb3de15a640 1 -- 172.21.15.67:0/349839808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 msgr2=0x55aeb46c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.583+0000 7fb3de15a640 1 --2- 172.21.15.67:0/349839808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 0x55aeb46c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.583+0000 7fb3de15a640 1 -- 172.21.15.67:0/349839808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43221b0 msgr2=0x55aeb46c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.583+0000 7fb3de15a640 1 --2- 172.21.15.67:0/349839808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43221b0 0x55aeb46c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.583+0000 7fb3de15a640 1 -- 172.21.15.67:0/349839808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeb45d09a0 con 0x55aeb42f92f0 2024-07-31T14:53:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.583+0000 7fb3dd959640 1 --2- 172.21.15.67:0/349839808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 0x55aeb46c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:53:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.583+0000 7fb3de15a640 1 --2- 172.21.15.67:0/349839808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb42f92f0 0x55aeb46c5220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009d20 tx=0x7fb3d4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8b2deec0bca9293 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.583+0000 7fb3de95b640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb3c40b3d40 0x7fb3c40b6250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00099e0 tx=0x7fb3d0009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.583+0000 7fb3ccbfe640 1 -- 172.21.15.67:0/349839808 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4004230 con 0x55aeb42f92f0 2024-07-31T14:53:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.583+0000 7fb3ccbfe640 1 -- 172.21.15.67:0/349839808 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3d40043d0 con 0x55aeb42f92f0 2024-07-31T14:53:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.583+0000 7fb3ccbfe640 1 -- 172.21.15.67:0/349839808 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40197c0 con 0x55aeb42f92f0 2024-07-31T14:53:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.584+0000 7fb3e738a740 1 -- 172.21.15.67:0/349839808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb42f92f0 msgr2=0x55aeb46c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.584+0000 7fb3e738a740 1 --2- 172.21.15.67:0/349839808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb42f92f0 0x55aeb46c5220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009d20 tx=0x7fb3d4011810 comp rx=0 tx=0).stop 2024-07-31T14:53:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.584+0000 7fb3e738a740 1 -- 172.21.15.67:0/349839808 shutdown_connections 2024-07-31T14:53:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.584+0000 7fb3e738a740 1 --2- 172.21.15.67:0/349839808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 0x55aeb46c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.584+0000 7fb3e738a740 1 --2- 172.21.15.67:0/349839808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb42f92f0 0x55aeb46c5220 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.584+0000 7fb3e738a740 1 --2- 172.21.15.67:0/349839808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43221b0 0x55aeb46c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.585+0000 7fb3e738a740 1 -- 172.21.15.67:0/349839808 >> 172.21.15.67:0/349839808 conn(0x55aeb4316300 msgr2=0x55aeb46c2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.585+0000 7fb3e738a740 1 -- 172.21.15.67:0/349839808 shutdown_connections 2024-07-31T14:53:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.585+0000 7fb3e738a740 1 -- 172.21.15.67:0/349839808 wait complete. 2024-07-31T14:53:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.585+0000 7fb3e738a740 1 -- start start 2024-07-31T14:53:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.585+0000 7fb3e738a740 1 Processor -- start 2024-07-31T14:53:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.585+0000 7fb3e738a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb42f92f0 0x55aeb46d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.585+0000 7fb3e738a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 0x55aeb46d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.585+0000 7fb3e738a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43221b0 0x55aeb46dc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.585+0000 7fb3e738a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55aeb4563ab0 con 0x55aeb43221b0 2024-07-31T14:53:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.585+0000 7fb3e738a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55aeb45b1cf0 con 0x55aeb43014d0 2024-07-31T14:53:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3e738a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55aeb46d3460 con 0x55aeb42f92f0 2024-07-31T14:53:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3e738a740 10 client.0 _ll_register_callbacks cb 0x55aeb46239f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3de95b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43221b0 0x55aeb46dc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3de15a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb42f92f0 0x55aeb46d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3de95b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43221b0 0x55aeb46dc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33206/0 (socket says 172.21.15.67:33206) 2024-07-31T14:53:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3de15a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb42f92f0 0x55aeb46d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:41694/0 (socket says 172.21.15.67:41694) 2024-07-31T14:53:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3de95b640 1 -- 172.21.15.67:0/707118950 learned_addr learned my addr 172.21.15.67:0/707118950 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3dd959640 1 --2- 172.21.15.67:0/707118950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 0x55aeb46d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3de95b640 1 -- 172.21.15.67:0/707118950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb42f92f0 msgr2=0x55aeb46d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3de95b640 1 --2- 172.21.15.67:0/707118950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb42f92f0 0x55aeb46d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3de95b640 1 -- 172.21.15.67:0/707118950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 msgr2=0x55aeb46d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3de95b640 1 --2- 172.21.15.67:0/707118950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 0x55aeb46d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3de95b640 1 -- 172.21.15.67:0/707118950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeb438d790 con 0x55aeb43221b0 2024-07-31T14:53:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3de15a640 1 --2- 172.21.15.67:0/707118950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb42f92f0 0x55aeb46d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:53:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3de95b640 1 --2- 172.21.15.67:0/707118950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43221b0 0x55aeb46dc7b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0003ea0 tx=0x7fb3d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb289a2260c703d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3c1ffb640 1 -- 172.21.15.67:0/707118950 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d001f670 con 0x55aeb43221b0 2024-07-31T14:53:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3c1ffb640 1 -- 172.21.15.67:0/707118950 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fb3d001f810 con 0x55aeb43221b0 2024-07-31T14:53:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.586+0000 7fb3e738a740 1 -- 172.21.15.67:0/707118950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3d40043d0 con 0x55aeb43221b0 2024-07-31T14:53:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.587+0000 7fb3c1ffb640 1 -- 172.21.15.67:0/707118950 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb3d001fa10 con 0x55aeb43221b0 2024-07-31T14:53:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.587+0000 7fb3e738a740 10 client.16413 fetch_fsmap learned FSMap version 38 2024-07-31T14:53:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.587+0000 7fb3e738a740 1 -- 172.21.15.67:0/707118950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55aeb4406620 con 0x55aeb43221b0 2024-07-31T14:53:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.587+0000 7fb3c1ffb640 1 -- 172.21.15.67:0/707118950 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3d40043d0 con 0x55aeb43221b0 2024-07-31T14:53:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.587+0000 7fb3c1ffb640 1 -- 172.21.15.67:0/707118950 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7fb3d0028070 con 0x55aeb43221b0 2024-07-31T14:53:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.587+0000 7fb3e738a740 10 client.16413 fetch_fsmap finished waiting for FSMap version 38 2024-07-31T14:53:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.587+0000 7fb3e738a740 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.587+0000 7fb3e738a740 20 client.16413 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.587+0000 7fb3e738a740 1 --2- 172.21.15.67:0/707118950 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55aeb42e0d80 0x55aeb46e2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.587+0000 7fb3e738a740 4 client.16413 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:53:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.587+0000 7fb3e738a740 1 -- 172.21.15.67:0/707118950 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aeb463c300 con 0x55aeb42e0d80 2024-07-31T14:53:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.587+0000 7fb3de15a640 1 --2- 172.21.15.67:0/707118950 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55aeb42e0d80 0x55aeb46e2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.588+0000 7fb3c1ffb640 10 client.16413 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.588+0000 7fb3de15a640 1 --2- 172.21.15.67:0/707118950 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55aeb42e0d80 0x55aeb46e2950 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5723f994191e40c3 server_cookie=a3c403952480fd3e in_seq=0 out_seq=0 2024-07-31T14:53:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3c1ffb640 1 -- 172.21.15.67:0/707118950 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aeb463c300 con 0x55aeb42e0d80 2024-07-31T14:53:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3c1ffb640 10 client.16413 handle_command_reply: tid=0 2024-07-31T14:53:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3e738a740 1 client.16413 shutdown 2024-07-31T14:53:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3e738a740 1 -- 172.21.15.67:0/707118950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43221b0 msgr2=0x55aeb46dc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3e738a740 1 --2- 172.21.15.67:0/707118950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43221b0 0x55aeb46dc7b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0003ea0 tx=0x7fb3d0004a40 comp rx=0 tx=0).stop 2024-07-31T14:53:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3e738a740 1 -- 172.21.15.67:0/707118950 shutdown_connections 2024-07-31T14:53:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3e738a740 1 --2- 172.21.15.67:0/707118950 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55aeb42e0d80 0x55aeb46e2950 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3e738a740 1 --2- 172.21.15.67:0/707118950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43221b0 0x55aeb46dc7b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3e738a740 1 --2- 172.21.15.67:0/707118950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 0x55aeb46d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3c1ffb640 0 client.16413 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3e738a740 1 --2- 172.21.15.67:0/707118950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb42f92f0 0x55aeb46d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3e738a740 1 -- 172.21.15.67:0/707118950 >> 172.21.15.67:0/707118950 conn(0x55aeb4316300 msgr2=0x55aeb46c5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3e738a740 1 -- 172.21.15.67:0/707118950 shutdown_connections 2024-07-31T14:53:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3e738a740 1 -- 172.21.15.67:0/707118950 wait complete. 2024-07-31T14:53:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.589+0000 7fb3e738a740 20 client.16413 trim_cache size 0 max 16384 2024-07-31T14:53:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.606+0000 7fb3e738a740 1 Processor -- start 2024-07-31T14:53:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.606+0000 7fb3e738a740 1 -- start start 2024-07-31T14:53:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.606+0000 7fb3e738a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb42f92f0 0x55aeb46c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.606+0000 7fb3e738a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43014d0 0x55aeb46cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.606+0000 7fb3e738a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb43221b0 0x55aeb46c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.606+0000 7fb3e738a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81ae430 con 0x55aeb43014d0 2024-07-31T14:53:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.606+0000 7fb3e738a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81b00c0 con 0x55aeb42f92f0 2024-07-31T14:53:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.606+0000 7fb3e738a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81aeb00 con 0x55aeb43221b0 2024-07-31T14:53:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.606+0000 7fb3dd959640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43014d0 0x55aeb46cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.606+0000 7fb3dd959640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43014d0 0x55aeb46cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33214/0 (socket says 172.21.15.67:33214) 2024-07-31T14:53:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.606+0000 7fb3dd959640 1 -- 172.21.15.67:0/2975375801 learned_addr learned my addr 172.21.15.67:0/2975375801 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.606+0000 7fb3de95b640 1 --2- 172.21.15.67:0/2975375801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb43221b0 0x55aeb46c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.607+0000 7fb3dd959640 1 -- 172.21.15.67:0/2975375801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb43221b0 msgr2=0x55aeb46c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.607+0000 7fb3dd959640 1 --2- 172.21.15.67:0/2975375801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb43221b0 0x55aeb46c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.607+0000 7fb3dd959640 1 -- 172.21.15.67:0/2975375801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb42f92f0 msgr2=0x55aeb46c2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.607+0000 7fb3de15a640 1 --2- 172.21.15.67:0/2975375801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb42f92f0 0x55aeb46c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.607+0000 7fb3dd959640 1 --2- 172.21.15.67:0/2975375801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb42f92f0 0x55aeb46c2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.607+0000 7fb3dd959640 1 -- 172.21.15.67:0/2975375801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeb46d0320 con 0x55aeb43014d0 2024-07-31T14:53:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.607+0000 7fb3de15a640 1 --2- 172.21.15.67:0/2975375801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb42f92f0 0x55aeb46c2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:53:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.607+0000 7fb3dd959640 1 --2- 172.21.15.67:0/2975375801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43014d0 0x55aeb46cae20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8000c00 tx=0x7fb3c80983a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30a196a4bef4b11 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.607+0000 7fb3c3fff640 1 -- 172.21.15.67:0/2975375801 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8049070 con 0x55aeb43014d0 2024-07-31T14:53:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.607+0000 7fb3c3fff640 1 -- 172.21.15.67:0/2975375801 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3c80981e0 con 0x55aeb43014d0 2024-07-31T14:53:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.607+0000 7fb3c3fff640 1 -- 172.21.15.67:0/2975375801 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8098040 con 0x55aeb43014d0 2024-07-31T14:53:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.608+0000 7fb3e738a740 1 -- 172.21.15.67:0/2975375801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43014d0 msgr2=0x55aeb46cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.608+0000 7fb3e738a740 1 --2- 172.21.15.67:0/2975375801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43014d0 0x55aeb46cae20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8000c00 tx=0x7fb3c80983a0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.608+0000 7fb3e738a740 1 -- 172.21.15.67:0/2975375801 shutdown_connections 2024-07-31T14:53:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.608+0000 7fb3e738a740 1 --2- 172.21.15.67:0/2975375801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb43221b0 0x55aeb46c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.608+0000 7fb3e738a740 1 --2- 172.21.15.67:0/2975375801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb43014d0 0x55aeb46cae20 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.608+0000 7fb3e738a740 1 --2- 172.21.15.67:0/2975375801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb42f92f0 0x55aeb46c2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.608+0000 7fb3e738a740 1 -- 172.21.15.67:0/2975375801 >> 172.21.15.67:0/2975375801 conn(0x55aeb4316300 msgr2=0x55aeb46c25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.608+0000 7fb3e738a740 1 -- 172.21.15.67:0/2975375801 shutdown_connections 2024-07-31T14:53:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.608+0000 7fb3e738a740 1 -- 172.21.15.67:0/2975375801 wait complete. 2024-07-31T14:53:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.608+0000 7fb3e738a740 1 -- start start 2024-07-31T14:53:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.608+0000 7fb3e738a740 1 Processor -- start 2024-07-31T14:53:56.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3e738a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb42f92f0 0x55aeb46d8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3e738a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 0x55aeb46d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3e738a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb43221b0 0x55aeb46d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3e738a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81af9f0 con 0x55aeb42f92f0 2024-07-31T14:53:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3e738a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81af260 con 0x55aeb43014d0 2024-07-31T14:53:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3e738a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81ac410 con 0x55aeb43221b0 2024-07-31T14:53:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3e738a740 10 client.0 _ll_register_callbacks cb 0x7fb3d0003d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:53:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3de15a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb42f92f0 0x55aeb46d8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3de15a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb42f92f0 0x55aeb46d8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33230/0 (socket says 172.21.15.67:33230) 2024-07-31T14:53:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3de15a640 1 -- 172.21.15.67:0/3171130728 learned_addr learned my addr 172.21.15.67:0/3171130728 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:53:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3de95b640 1 --2- 172.21.15.67:0/3171130728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb43221b0 0x55aeb46d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3dd959640 1 --2- 172.21.15.67:0/3171130728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 0x55aeb46d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3de15a640 1 -- 172.21.15.67:0/3171130728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb43221b0 msgr2=0x55aeb46d54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3de15a640 1 --2- 172.21.15.67:0/3171130728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb43221b0 0x55aeb46d54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3de15a640 1 -- 172.21.15.67:0/3171130728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 msgr2=0x55aeb46d4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:53:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3de15a640 1 --2- 172.21.15.67:0/3171130728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 0x55aeb46d4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3de15a640 1 -- 172.21.15.67:0/3171130728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeb46d5a20 con 0x55aeb42f92f0 2024-07-31T14:53:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.609+0000 7fb3dd959640 1 --2- 172.21.15.67:0/3171130728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 0x55aeb46d4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:53:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.610+0000 7fb3de15a640 1 --2- 172.21.15.67:0/3171130728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb42f92f0 0x55aeb46d8af0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40050e0 tx=0x7fb3d4019420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bce01aa74692625 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:53:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.610+0000 7fb3c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:53:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.610+0000 7fb3c1ffb640 1 -- 172.21.15.67:0/3171130728 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d402f070 con 0x55aeb42f92f0 2024-07-31T14:53:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.610+0000 7fb3c1ffb640 1 -- 172.21.15.67:0/3171130728 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fb3d4005410 con 0x55aeb42f92f0 2024-07-31T14:53:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.610+0000 7fb3e738a740 1 -- 172.21.15.67:0/3171130728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aeb4637da0 con 0x55aeb42f92f0 2024-07-31T14:53:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.610+0000 7fb3c1ffb640 1 -- 172.21.15.67:0/3171130728 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb3d403b050 con 0x55aeb42f92f0 2024-07-31T14:53:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.610+0000 7fb3e738a740 10 client.16419 fetch_fsmap learned FSMap version 38 2024-07-31T14:53:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.610+0000 7fb3e738a740 1 -- 172.21.15.67:0/3171130728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55aeb4749d40 con 0x55aeb42f92f0 2024-07-31T14:53:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.610+0000 7fb3c1ffb640 1 -- 172.21.15.67:0/3171130728 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55aeb4637da0 con 0x55aeb42f92f0 2024-07-31T14:53:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.610+0000 7fb3c1ffb640 1 -- 172.21.15.67:0/3171130728 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7fb3d4004680 con 0x55aeb42f92f0 2024-07-31T14:53:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.610+0000 7fb3e738a740 10 client.16419 fetch_fsmap finished waiting for FSMap version 38 2024-07-31T14:53:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.610+0000 7fb3e738a740 10 client.16419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:53:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.610+0000 7fb3e738a740 20 client.16419 populate_metadata read hostname 'smithi067' 2024-07-31T14:53:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.610+0000 7fb3e738a740 1 --2- 172.21.15.67:0/3171130728 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55aeb42b61c0 0x55aeb474c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:53:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.611+0000 7fb3e738a740 4 client.16419 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:53:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.611+0000 7fb3e738a740 1 -- 172.21.15.67:0/3171130728 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aeb46354e0 con 0x55aeb42b61c0 2024-07-31T14:53:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.611+0000 7fb3dd959640 1 --2- 172.21.15.67:0/3171130728 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55aeb42b61c0 0x55aeb474c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:53:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.611+0000 7fb3c1ffb640 10 client.16419 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.611+0000 7fb3dd959640 1 --2- 172.21.15.67:0/3171130728 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55aeb42b61c0 0x55aeb474c440 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0393cf3bac0016e server_cookie=73a4f06a54b5bd27 in_seq=0 out_seq=0 2024-07-31T14:53:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.654+0000 7fb3c1ffb640 1 -- 172.21.15.67:0/3171130728 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aeb46354e0 con 0x55aeb42b61c0 2024-07-31T14:53:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.654+0000 7fb3c1ffb640 10 client.16419 handle_command_reply: tid=0 2024-07-31T14:53:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.654+0000 7fb3e738a740 1 client.16419 shutdown 2024-07-31T14:53:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.654+0000 7fb3e738a740 1 -- 172.21.15.67:0/3171130728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb42f92f0 msgr2=0x55aeb46d8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.654+0000 7fb3e738a740 1 --2- 172.21.15.67:0/3171130728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb42f92f0 0x55aeb46d8af0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40050e0 tx=0x7fb3d4019420 comp rx=0 tx=0).stop 2024-07-31T14:53:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.654+0000 7fb3e738a740 1 -- 172.21.15.67:0/3171130728 shutdown_connections 2024-07-31T14:53:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.654+0000 7fb3e738a740 1 --2- 172.21.15.67:0/3171130728 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55aeb42b61c0 0x55aeb474c440 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.654+0000 7fb3e738a740 1 --2- 172.21.15.67:0/3171130728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aeb43221b0 0x55aeb46d54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.654+0000 7fb3c1ffb640 0 client.16419 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:53:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.654+0000 7fb3e738a740 1 --2- 172.21.15.67:0/3171130728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aeb43014d0 0x55aeb46d4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.654+0000 7fb3e738a740 1 --2- 172.21.15.67:0/3171130728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aeb42f92f0 0x55aeb46d8af0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.654+0000 7fb3e738a740 1 -- 172.21.15.67:0/3171130728 >> 172.21.15.67:0/3171130728 conn(0x55aeb4316300 msgr2=0x55aeb46d1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.654+0000 7fb3e738a740 1 -- 172.21.15.67:0/3171130728 shutdown_connections 2024-07-31T14:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.655+0000 7fb3e738a740 1 -- 172.21.15.67:0/3171130728 wait complete. 2024-07-31T14:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.655+0000 7fb3e738a740 20 client.16419 trim_cache size 0 max 16384 2024-07-31T14:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:53:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.655+0000 7fb3e03be640 1 -- 172.21.15.67:0/838997596 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb3c40b3d40 msgr2=0x7fb3c40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.655+0000 7fb3e03be640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb3c40b3d40 0x7fb3c40b6250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00099e0 tx=0x7fb3d0009500 comp rx=0 tx=0).stop 2024-07-31T14:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.655+0000 7fb3e03be640 1 -- 172.21.15.67:0/838997596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3d8107d20 msgr2=0x7fb3d8199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.655+0000 7fb3e03be640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3d8107d20 0x7fb3d8199300 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb3c802a930 tx=0x7fb3c8031a20 comp rx=0 tx=0).stop 2024-07-31T14:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.655+0000 7fb3e03be640 1 -- 172.21.15.67:0/838997596 shutdown_connections 2024-07-31T14:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.655+0000 7fb3e03be640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb3c40b3d40 0x7fb3c40b6250 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.655+0000 7fb3e03be640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3d8108670 0x7fb3d819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.655+0000 7fb3e03be640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3d8107d20 0x7fb3d8199300 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.655+0000 7fb3e03be640 1 --2- 172.21.15.67:0/838997596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d81023f0 0x7fb3d8198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.655+0000 7fb3e03be640 1 -- 172.21.15.67:0/838997596 >> 172.21.15.67:0/838997596 conn(0x7fb3d80fddf0 msgr2=0x7fb3d8100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.655+0000 7fb3e03be640 1 -- 172.21.15.67:0/838997596 shutdown_connections 2024-07-31T14:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:53:56.655+0000 7fb3e03be640 1 -- 172.21.15.67:0/838997596 wait complete. 2024-07-31T14:53:57.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:57 smithi067 ceph-mon[32750]: pgmap v486: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.4 MiB/s wr, 616 op/s 2024-07-31T14:53:57.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:57 smithi073 ceph-mon[36714]: pgmap v486: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.4 MiB/s wr, 616 op/s 2024-07-31T14:53:57.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:57 smithi077 ceph-mon[37327]: pgmap v486: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.4 MiB/s wr, 616 op/s 2024-07-31T14:53:59.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:53:59 smithi067 ceph-mon[32750]: pgmap v487: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 659 KiB/s wr, 318 op/s 2024-07-31T14:53:59.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:53:59 smithi073 ceph-mon[36714]: pgmap v487: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 659 KiB/s wr, 318 op/s 2024-07-31T14:53:59.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:53:59 smithi077 ceph-mon[37327]: pgmap v487: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 659 KiB/s wr, 318 op/s 2024-07-31T14:54:01.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:01 smithi067 ceph-mon[32750]: pgmap v488: 129 pgs: 129 active+clean; 223 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-07-31T14:54:01.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:01 smithi073 ceph-mon[36714]: pgmap v488: 129 pgs: 129 active+clean; 223 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-07-31T14:54:01.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:01 smithi077 ceph-mon[37327]: pgmap v488: 129 pgs: 129 active+clean; 223 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-07-31T14:54:03.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:03 smithi067 ceph-mon[32750]: pgmap v489: 129 pgs: 129 active+clean; 229 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-07-31T14:54:03.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:03 smithi073 ceph-mon[36714]: pgmap v489: 129 pgs: 129 active+clean; 229 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-07-31T14:54:03.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:03 smithi077 ceph-mon[37327]: pgmap v489: 129 pgs: 129 active+clean; 229 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-07-31T14:54:05.252 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.4 2024-07-31T14:54:05.253 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:54:05.282 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:05 smithi067 ceph-mon[32750]: pgmap v490: 129 pgs: 129 active+clean; 250 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 411 op/s 2024-07-31T14:54:05.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:05 smithi073 ceph-mon[36714]: pgmap v490: 129 pgs: 129 active+clean; 250 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 411 op/s 2024-07-31T14:54:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:05 smithi077 ceph-mon[37327]: pgmap v490: 129 pgs: 129 active+clean; 250 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 411 op/s 2024-07-31T14:54:05.524 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:54:06.167 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:54:07.070 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:54:07.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:07 smithi067 ceph-mon[32750]: pgmap v491: 129 pgs: 129 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 569 op/s 2024-07-31T14:54:07.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:07 smithi073 ceph-mon[36714]: pgmap v491: 129 pgs: 129 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 569 op/s 2024-07-31T14:54:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:07 smithi077 ceph-mon[37327]: pgmap v491: 129 pgs: 129 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 569 op/s 2024-07-31T14:54:08.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.522+0000 7f863a6b8640 1 -- 172.21.15.67:0/3576196739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86341024d0 msgr2=0x7f86341028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:08.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.522+0000 7f863a6b8640 1 --2- 172.21.15.67:0/3576196739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86341024d0 0x7f86341028e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8624009900 tx=0x7f862402a410 comp rx=0 tx=0).stop 2024-07-31T14:54:08.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.522+0000 7f863a6b8640 1 -- 172.21.15.67:0/3576196739 shutdown_connections 2024-07-31T14:54:08.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.522+0000 7f863a6b8640 1 --2- 172.21.15.67:0/3576196739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8634108760 0x7f863410ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:08.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.522+0000 7f863a6b8640 1 --2- 172.21.15.67:0/3576196739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8634107e10 0x7f8634108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:08.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.522+0000 7f863a6b8640 1 --2- 172.21.15.67:0/3576196739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86341024d0 0x7f86341028e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:08.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.522+0000 7f863a6b8640 1 -- 172.21.15.67:0/3576196739 >> 172.21.15.67:0/3576196739 conn(0x7f8634079210 msgr2=0x7f8634079620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:08.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.522+0000 7f863a6b8640 1 -- 172.21.15.67:0/3576196739 shutdown_connections 2024-07-31T14:54:08.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.522+0000 7f863a6b8640 1 -- 172.21.15.67:0/3576196739 wait complete. 2024-07-31T14:54:08.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.523+0000 7f863a6b8640 1 Processor -- start 2024-07-31T14:54:08.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.523+0000 7f863a6b8640 1 -- start start 2024-07-31T14:54:08.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f863a6b8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86341024d0 0x7f863419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:08.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f863a6b8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8634107e10 0x7f863419b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:08.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f863a6b8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8634108760 0x7f863419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:08.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f863a6b8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f86341123e0 con 0x7f8634107e10 2024-07-31T14:54:08.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f863a6b8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8634112260 con 0x7f8634108760 2024-07-31T14:54:08.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f863a6b8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8634112560 con 0x7f86341024d0 2024-07-31T14:54:08.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f8633fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86341024d0 0x7f863419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:08.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f8633fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86341024d0 0x7f863419acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:54114/0 (socket says 172.21.15.67:54114) 2024-07-31T14:54:08.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f8633fff640 1 -- 172.21.15.67:0/1944703956 learned_addr learned my addr 172.21.15.67:0/1944703956 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:08.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f862bfff640 1 --2- 172.21.15.67:0/1944703956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8634107e10 0x7f863419b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:08.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f8638c55640 1 --2- 172.21.15.67:0/1944703956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8634108760 0x7f863419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:08.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f8633fff640 1 -- 172.21.15.67:0/1944703956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8634108760 msgr2=0x7f863419b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:08.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f8633fff640 1 --2- 172.21.15.67:0/1944703956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8634108760 0x7f863419b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:08.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f8633fff640 1 -- 172.21.15.67:0/1944703956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8634107e10 msgr2=0x7f863419b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:08.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f8633fff640 1 --2- 172.21.15.67:0/1944703956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8634107e10 0x7f863419b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:08.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f8633fff640 1 -- 172.21.15.67:0/1944703956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86341a5f70 con 0x7f86341024d0 2024-07-31T14:54:08.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.524+0000 7f8633fff640 1 --2- 172.21.15.67:0/1944703956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86341024d0 0x7f863419acb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8624005940 tx=0x7f8624004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:08.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.525+0000 7f8631ffb640 1 -- 172.21.15.67:0/1944703956 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862403f030 con 0x7f86341024d0 2024-07-31T14:54:08.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.525+0000 7f863a6b8640 1 -- 172.21.15.67:0/1944703956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86341a6200 con 0x7f86341024d0 2024-07-31T14:54:08.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.525+0000 7f8631ffb640 1 -- 172.21.15.67:0/1944703956 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8624003aa0 con 0x7f86341024d0 2024-07-31T14:54:08.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.525+0000 7f8631ffb640 1 -- 172.21.15.67:0/1944703956 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624035070 con 0x7f86341024d0 2024-07-31T14:54:08.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.525+0000 7f863a6b8640 1 -- 172.21.15.67:0/1944703956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86341a66e0 con 0x7f86341024d0 2024-07-31T14:54:08.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.526+0000 7f863a6b8640 1 -- 172.21.15.67:0/1944703956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86341028e0 con 0x7f86341024d0 2024-07-31T14:54:08.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.528+0000 7f8631ffb640 1 -- 172.21.15.67:0/1944703956 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8624042030 con 0x7f86341024d0 2024-07-31T14:54:08.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.528+0000 7f8631ffb640 1 --2- 172.21.15.67:0/1944703956 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86200b3d40 0x7f86200b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:08.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.528+0000 7f862bfff640 1 --2- 172.21.15.67:0/1944703956 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86200b3d40 0x7f86200b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:08.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.529+0000 7f8631ffb640 1 -- 172.21.15.67:0/1944703956 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f8624104d20 con 0x7f86341024d0 2024-07-31T14:54:08.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.529+0000 7f862bfff640 1 --2- 172.21.15.67:0/1944703956 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86200b3d40 0x7f86200b6250 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8634107c50 tx=0x7f861c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:08.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.533+0000 7f8631ffb640 1 -- 172.21.15.67:0/1944703956 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f86240cbcc0 con 0x7f86341024d0 2024-07-31T14:54:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.773+0000 7f863a6b8640 1 -- 172.21.15.67:0/1944703956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f863406b6a0 con 0x7f86341024d0 2024-07-31T14:54:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.774+0000 7f8631ffb640 1 -- 172.21.15.67:0/1944703956 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11978 (secure 0 0 0) 0x7f86240cb5c0 con 0x7f86341024d0 2024-07-31T14:54:08.776 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:54:08.776 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":38,"btime":"2024-07-31T14:53:51:995374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:53:51.521398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:54:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 38 2024-07-31T14:54:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.777+0000 7f863a6b8640 1 -- 172.21.15.67:0/1944703956 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86200b3d40 msgr2=0x7f86200b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.777+0000 7f863a6b8640 1 --2- 172.21.15.67:0/1944703956 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86200b3d40 0x7f86200b6250 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8634107c50 tx=0x7f861c00b390 comp rx=0 tx=0).stop 2024-07-31T14:54:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.777+0000 7f863a6b8640 1 -- 172.21.15.67:0/1944703956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86341024d0 msgr2=0x7f863419acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.777+0000 7f863a6b8640 1 --2- 172.21.15.67:0/1944703956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86341024d0 0x7f863419acb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8624005940 tx=0x7f8624004270 comp rx=0 tx=0).stop 2024-07-31T14:54:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.777+0000 7f863a6b8640 1 -- 172.21.15.67:0/1944703956 shutdown_connections 2024-07-31T14:54:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.777+0000 7f863a6b8640 1 --2- 172.21.15.67:0/1944703956 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86200b3d40 0x7f86200b6250 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.777+0000 7f863a6b8640 1 --2- 172.21.15.67:0/1944703956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8634108760 0x7f863419b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.777+0000 7f863a6b8640 1 --2- 172.21.15.67:0/1944703956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8634107e10 0x7f863419b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.777+0000 7f863a6b8640 1 --2- 172.21.15.67:0/1944703956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86341024d0 0x7f863419acb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.777+0000 7f863a6b8640 1 -- 172.21.15.67:0/1944703956 >> 172.21.15.67:0/1944703956 conn(0x7f8634079210 msgr2=0x7f8634100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.777+0000 7f863a6b8640 1 -- 172.21.15.67:0/1944703956 shutdown_connections 2024-07-31T14:54:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:08.777+0000 7f863a6b8640 1 -- 172.21.15.67:0/1944703956 wait complete. 2024-07-31T14:54:09.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:09 smithi067 ceph-mon[32750]: pgmap v492: 129 pgs: 129 active+clean; 283 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 632 op/s 2024-07-31T14:54:09.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:09 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1944703956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:54:09.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:09 smithi073 ceph-mon[36714]: pgmap v492: 129 pgs: 129 active+clean; 283 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 632 op/s 2024-07-31T14:54:09.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:09 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1944703956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:54:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:09 smithi077 ceph-mon[37327]: pgmap v492: 129 pgs: 129 active+clean; 283 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 632 op/s 2024-07-31T14:54:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:09 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1944703956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:54:09.599 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 143.4 --await-for 45.0' 2024-07-31T14:54:09.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 143.4 --await-for 45.0 2024-07-31T14:54:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.758+0000 7f86b7662640 1 -- 172.21.15.67:0/3311798129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b0113830 msgr2=0x7f86b0113c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.758+0000 7f86b7662640 1 --2- 172.21.15.67:0/3311798129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b0113830 0x7f86b0113c40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009900 tx=0x7f86a002a410 comp rx=0 tx=0).stop 2024-07-31T14:54:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.758+0000 7f86b7662640 1 -- 172.21.15.67:0/3311798129 shutdown_connections 2024-07-31T14:54:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.758+0000 7f86b7662640 1 --2- 172.21.15.67:0/3311798129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86b0114180 0x7f86b011a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.758+0000 7f86b7662640 1 --2- 172.21.15.67:0/3311798129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b0113830 0x7f86b0113c40 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.758+0000 7f86b7662640 1 --2- 172.21.15.67:0/3311798129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86b010df00 0x7f86b010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.758+0000 7f86b7662640 1 -- 172.21.15.67:0/3311798129 >> 172.21.15.67:0/3311798129 conn(0x7f86b006e280 msgr2=0x7f86b010be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.758+0000 7f86b7662640 1 -- 172.21.15.67:0/3311798129 shutdown_connections 2024-07-31T14:54:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.758+0000 7f86b7662640 1 -- 172.21.15.67:0/3311798129 wait complete. 2024-07-31T14:54:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.759+0000 7f86b7662640 1 Processor -- start 2024-07-31T14:54:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.759+0000 7f86b7662640 1 -- start start 2024-07-31T14:54:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.759+0000 7f86b7662640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b0113830 0x7f86b01a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.759+0000 7f86b7662640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86b0114180 0x7f86b01a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.759+0000 7f86b7662640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86b01a9d00 0x7f86b01aa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.759+0000 7f86b7662640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f86b011de40 con 0x7f86b0113830 2024-07-31T14:54:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.759+0000 7f86b7662640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86b011dcc0 con 0x7f86b0114180 2024-07-31T14:54:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.759+0000 7f86b7662640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86b011dfc0 con 0x7f86b01a9d00 2024-07-31T14:54:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.760+0000 7f86b53fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b0113830 0x7f86b01a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.760+0000 7f86b53fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b0113830 0x7f86b01a44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39894/0 (socket says 172.21.15.67:39894) 2024-07-31T14:54:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.760+0000 7f86b53fe640 1 -- 172.21.15.67:0/1614692954 learned_addr learned my addr 172.21.15.67:0/1614692954 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.760+0000 7f86b5bff640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86b01a9d00 0x7f86b01aa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.760+0000 7f86b4bfd640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86b0114180 0x7f86b01a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.760+0000 7f86b53fe640 1 -- 172.21.15.67:0/1614692954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86b01a9d00 msgr2=0x7f86b01aa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.760+0000 7f86b53fe640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86b01a9d00 0x7f86b01aa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.760+0000 7f86b53fe640 1 -- 172.21.15.67:0/1614692954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86b0114180 msgr2=0x7f86b01a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.760+0000 7f86b53fe640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86b0114180 0x7f86b01a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.760+0000 7f86b53fe640 1 -- 172.21.15.67:0/1614692954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b01aa810 con 0x7f86b0113830 2024-07-31T14:54:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.760+0000 7f86b5bff640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86b01a9d00 0x7f86b01aa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.760+0000 7f86b53fe640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b0113830 0x7f86b01a44d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00e9f0 tx=0x7f86ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.760+0000 7f86a67fc640 1 -- 172.21.15.67:0/1614692954 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac00cd80 con 0x7f86b0113830 2024-07-31T14:54:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.761+0000 7f86b7662640 1 -- 172.21.15.67:0/1614692954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b01aab00 con 0x7f86b0113830 2024-07-31T14:54:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.761+0000 7f86a67fc640 1 -- 172.21.15.67:0/1614692954 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ac0054d0 con 0x7f86b0113830 2024-07-31T14:54:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.761+0000 7f86a67fc640 1 -- 172.21.15.67:0/1614692954 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac010690 con 0x7f86b0113830 2024-07-31T14:54:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.761+0000 7f86b7662640 1 -- 172.21.15.67:0/1614692954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b01b1180 con 0x7f86b0113830 2024-07-31T14:54:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.763+0000 7f86be639740 1 Processor -- start 2024-07-31T14:54:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86be639740 1 -- start start 2024-07-31T14:54:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86be639740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 0x556bc77a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86be639740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77a2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86be639740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86be639740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556bc77173b0 con 0x556bc73d62f0 2024-07-31T14:54:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86be639740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556bc77068b0 con 0x556bc73ff1b0 2024-07-31T14:54:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86be639740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556bc770a8f0 con 0x556bc73de4d0 2024-07-31T14:54:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86b4bfd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86b4bfd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77a2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:54150/0 (socket says 172.21.15.67:54150) 2024-07-31T14:54:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86b4bfd640 1 -- 172.21.15.67:0/2606807665 learned_addr learned my addr 172.21.15.67:0/2606807665 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86b5bff640 1 --2- 172.21.15.67:0/2606807665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 0x556bc77a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86b4bfd640 1 -- 172.21.15.67:0/2606807665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 msgr2=0x556bc77a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86b4bfd640 1 --2- 172.21.15.67:0/2606807665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 0x556bc77a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86b4bfd640 1 -- 172.21.15.67:0/2606807665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 msgr2=0x556bc77a2220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:54:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86b4bfd640 1 --2- 172.21.15.67:0/2606807665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77a2220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.764+0000 7f86b4bfd640 1 -- 172.21.15.67:0/2606807665 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bc76ad9a0 con 0x556bc73de4d0 2024-07-31T14:54:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.765+0000 7f86b4bfd640 1 --2- 172.21.15.67:0/2606807665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77a2760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f86a0033000 tx=0x7f86a0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aecc590698a2cfde server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.765+0000 7f869b7fe640 1 -- 172.21.15.67:0/2606807665 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0035040 con 0x556bc73de4d0 2024-07-31T14:54:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.765+0000 7f869b7fe640 1 -- 172.21.15.67:0/2606807665 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86a0004030 con 0x556bc73de4d0 2024-07-31T14:54:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.765+0000 7f869b7fe640 1 -- 172.21.15.67:0/2606807665 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0005510 con 0x556bc73de4d0 2024-07-31T14:54:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.765+0000 7f86a67fc640 1 -- 172.21.15.67:0/1614692954 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f86ac004e90 con 0x7f86b0113830 2024-07-31T14:54:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.766+0000 7f86a67fc640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f869c0b3d40 0x7f869c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.766+0000 7f86be639740 1 -- 172.21.15.67:0/2606807665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 msgr2=0x556bc77a2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.766+0000 7f86be639740 1 --2- 172.21.15.67:0/2606807665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77a2760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f86a0033000 tx=0x7f86a0000bc0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.766+0000 7f86be639740 1 -- 172.21.15.67:0/2606807665 shutdown_connections 2024-07-31T14:54:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.766+0000 7f86be639740 1 --2- 172.21.15.67:0/2606807665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77a2760 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.766+0000 7f86be639740 1 --2- 172.21.15.67:0/2606807665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.766+0000 7f86be639740 1 --2- 172.21.15.67:0/2606807665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 0x556bc77a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.766+0000 7f86be639740 1 -- 172.21.15.67:0/2606807665 >> 172.21.15.67:0/2606807665 conn(0x556bc73f3300 msgr2=0x556bc779fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.766+0000 7f86b5bff640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f869c0b3d40 0x7f869c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.766+0000 7f86a67fc640 1 -- 172.21.15.67:0/1614692954 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f86ac0e4720 con 0x7f86b0113830 2024-07-31T14:54:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.766+0000 7f86be639740 1 -- 172.21.15.67:0/2606807665 shutdown_connections 2024-07-31T14:54:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.766+0000 7f86be639740 1 -- 172.21.15.67:0/2606807665 wait complete. 2024-07-31T14:54:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.766+0000 7f86b5bff640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f869c0b3d40 0x7f869c0b6250 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86a8005fd0 tx=0x7f86a80096a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.766+0000 7f86be639740 1 -- start start 2024-07-31T14:54:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.767+0000 7f86be639740 1 Processor -- start 2024-07-31T14:54:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.767+0000 7f86be639740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73d62f0 0x556bc77ad820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.767+0000 7f86be639740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73de4d0 0x556bc77add60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.767+0000 7f86be639740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73ff1b0 0x556bc77b9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.767+0000 7f86be639740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556bc7640ab0 con 0x556bc73de4d0 2024-07-31T14:54:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.767+0000 7f86be639740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556bc768ecf0 con 0x556bc73d62f0 2024-07-31T14:54:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.767+0000 7f86be639740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556bc77b0400 con 0x556bc73ff1b0 2024-07-31T14:54:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.767+0000 7f86be639740 10 client.0 _ll_register_callbacks cb 0x556bc77009f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.767+0000 7f86b5bff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73ff1b0 0x556bc77b9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.767+0000 7f86b4bfd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73de4d0 0x556bc77add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.767+0000 7f86b5bff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73ff1b0 0x556bc77b9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:54162/0 (socket says 172.21.15.67:54162) 2024-07-31T14:54:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.767+0000 7f86b5bff640 1 -- 172.21.15.67:0/598562554 learned_addr learned my addr 172.21.15.67:0/598562554 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f86b53fe640 1 --2- 172.21.15.67:0/598562554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73d62f0 0x556bc77ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f86b5bff640 1 -- 172.21.15.67:0/598562554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73d62f0 msgr2=0x556bc77ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f86b5bff640 1 --2- 172.21.15.67:0/598562554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73d62f0 0x556bc77ad820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f86b5bff640 1 -- 172.21.15.67:0/598562554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73de4d0 msgr2=0x556bc77add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f86b5bff640 1 --2- 172.21.15.67:0/598562554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73de4d0 0x556bc77add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f86b5bff640 1 -- 172.21.15.67:0/598562554 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bc746a790 con 0x556bc73ff1b0 2024-07-31T14:54:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f86b53fe640 1 --2- 172.21.15.67:0/598562554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73d62f0 0x556bc77ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:54:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f86b5bff640 1 --2- 172.21.15.67:0/598562554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73ff1b0 0x556bc77b9750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009970 tx=0x7f86a8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9322a237c107eea8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f8698ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:54:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f8698ff9640 1 -- 172.21.15.67:0/598562554 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a801fc80 con 0x556bc73ff1b0 2024-07-31T14:54:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f86be639740 1 -- 172.21.15.67:0/598562554 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a0004030 con 0x556bc73ff1b0 2024-07-31T14:54:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f8698ff9640 1 -- 172.21.15.67:0/598562554 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f86a8017030 con 0x556bc73ff1b0 2024-07-31T14:54:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f8698ff9640 1 -- 172.21.15.67:0/598562554 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f86a8032050 con 0x556bc73ff1b0 2024-07-31T14:54:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f86be639740 10 client.26095 fetch_fsmap learned FSMap version 38 2024-07-31T14:54:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.768+0000 7f86be639740 1 -- 172.21.15.67:0/598562554 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556bc74e3620 con 0x556bc73ff1b0 2024-07-31T14:54:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.769+0000 7f8698ff9640 1 -- 172.21.15.67:0/598562554 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86a0004030 con 0x556bc73ff1b0 2024-07-31T14:54:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.769+0000 7f8698ff9640 1 -- 172.21.15.67:0/598562554 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7f86a8027c10 con 0x556bc73ff1b0 2024-07-31T14:54:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.769+0000 7f86be639740 10 client.26095 fetch_fsmap finished waiting for FSMap version 38 2024-07-31T14:54:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.769+0000 7f86be639740 10 client.26095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.769+0000 7f86be639740 20 client.26095 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.769+0000 7f86be639740 1 --2- 172.21.15.67:0/598562554 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bc73bdd80 0x556bc77bf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.769+0000 7f86be639740 4 client.26095 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:54:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.769+0000 7f86be639740 1 -- 172.21.15.67:0/598562554 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bc7719300 con 0x556bc73bdd80 2024-07-31T14:54:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.769+0000 7f86b53fe640 1 --2- 172.21.15.67:0/598562554 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bc73bdd80 0x556bc77bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.770+0000 7f8698ff9640 10 client.26095 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.770+0000 7f86b53fe640 1 --2- 172.21.15.67:0/598562554 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bc73bdd80 0x556bc77bf900 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a8338fe501b1f1a server_cookie=6da5aead7a48ff5e in_seq=0 out_seq=0 2024-07-31T14:54:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f8698ff9640 1 -- 172.21.15.67:0/598562554 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556bc7719300 con 0x556bc73bdd80 2024-07-31T14:54:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f8698ff9640 10 client.26095 handle_command_reply: tid=0 2024-07-31T14:54:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f86be639740 1 client.26095 shutdown 2024-07-31T14:54:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f86be639740 1 -- 172.21.15.67:0/598562554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73ff1b0 msgr2=0x556bc77b9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f86be639740 1 --2- 172.21.15.67:0/598562554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73ff1b0 0x556bc77b9750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009970 tx=0x7f86a8004ab0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f86be639740 1 -- 172.21.15.67:0/598562554 shutdown_connections 2024-07-31T14:54:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f86be639740 1 --2- 172.21.15.67:0/598562554 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bc73bdd80 0x556bc77bf900 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f86be639740 1 --2- 172.21.15.67:0/598562554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73ff1b0 0x556bc77b9750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f86be639740 1 --2- 172.21.15.67:0/598562554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73de4d0 0x556bc77add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f8698ff9640 0 client.26095 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f86be639740 1 --2- 172.21.15.67:0/598562554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73d62f0 0x556bc77ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f86be639740 1 -- 172.21.15.67:0/598562554 >> 172.21.15.67:0/598562554 conn(0x556bc73f3300 msgr2=0x556bc77a2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f86be639740 1 -- 172.21.15.67:0/598562554 shutdown_connections 2024-07-31T14:54:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f86be639740 1 -- 172.21.15.67:0/598562554 wait complete. 2024-07-31T14:54:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.771+0000 7f86be639740 20 client.26095 trim_cache size 0 max 16384 2024-07-31T14:54:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.786+0000 7f86be639740 1 Processor -- start 2024-07-31T14:54:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.786+0000 7f86be639740 1 -- start start 2024-07-31T14:54:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86be639740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86be639740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86be639740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 0x556bc77b9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86be639740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01bb040 con 0x556bc73d62f0 2024-07-31T14:54:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86be639740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01bccd0 con 0x556bc73ff1b0 2024-07-31T14:54:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86be639740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01bb710 con 0x556bc73de4d0 2024-07-31T14:54:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86b53fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77a7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86b4bfd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86b53fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77a7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39910/0 (socket says 172.21.15.67:39910) 2024-07-31T14:54:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86b53fe640 1 -- 172.21.15.67:0/2678809511 learned_addr learned my addr 172.21.15.67:0/2678809511 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86b4bfd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:54172/0 (socket says 172.21.15.67:54172) 2024-07-31T14:54:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86b5bff640 1 --2- 172.21.15.67:0/2678809511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 0x556bc77b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86b53fe640 1 -- 172.21.15.67:0/2678809511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 msgr2=0x556bc77af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86b53fe640 1 --2- 172.21.15.67:0/2678809511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86b53fe640 1 -- 172.21.15.67:0/2678809511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 msgr2=0x556bc77b9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86b53fe640 1 --2- 172.21.15.67:0/2678809511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 0x556bc77b9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86b53fe640 1 -- 172.21.15.67:0/2678809511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bc77a36d0 con 0x556bc73d62f0 2024-07-31T14:54:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.787+0000 7f86b53fe640 1 --2- 172.21.15.67:0/2678809511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77a7e20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0a70a0 tx=0x7f86ac005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f138471d8742b7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.788+0000 7f869affd640 1 -- 172.21.15.67:0/2678809511 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0c4070 con 0x556bc73d62f0 2024-07-31T14:54:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.788+0000 7f869affd640 1 -- 172.21.15.67:0/2678809511 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ac0ab020 con 0x556bc73d62f0 2024-07-31T14:54:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.788+0000 7f869affd640 1 -- 172.21.15.67:0/2678809511 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0e1de0 con 0x556bc73d62f0 2024-07-31T14:54:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.789+0000 7f86be639740 1 -- 172.21.15.67:0/2678809511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 msgr2=0x556bc77a7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.789+0000 7f86be639740 1 --2- 172.21.15.67:0/2678809511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77a7e20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0a70a0 tx=0x7f86ac005680 comp rx=0 tx=0).stop 2024-07-31T14:54:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.789+0000 7f86be639740 1 -- 172.21.15.67:0/2678809511 shutdown_connections 2024-07-31T14:54:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.789+0000 7f86be639740 1 --2- 172.21.15.67:0/2678809511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 0x556bc77b9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.789+0000 7f86be639740 1 --2- 172.21.15.67:0/2678809511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.789+0000 7f86be639740 1 --2- 172.21.15.67:0/2678809511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77a7e20 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.789+0000 7f86be639740 1 -- 172.21.15.67:0/2678809511 >> 172.21.15.67:0/2678809511 conn(0x556bc73f3300 msgr2=0x556bc77aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.789+0000 7f86be639740 1 -- 172.21.15.67:0/2678809511 shutdown_connections 2024-07-31T14:54:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.789+0000 7f86be639740 1 -- 172.21.15.67:0/2678809511 wait complete. 2024-07-31T14:54:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.789+0000 7f86be639740 1 -- start start 2024-07-31T14:54:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.789+0000 7f86be639740 1 Processor -- start 2024-07-31T14:54:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.790+0000 7f86be639740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.790+0000 7f86be639740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.790+0000 7f86be639740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 0x556bc77b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.790+0000 7f86be639740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01bc600 con 0x556bc73d62f0 2024-07-31T14:54:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.790+0000 7f86be639740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01bbe70 con 0x556bc73ff1b0 2024-07-31T14:54:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.790+0000 7f86be639740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01b9020 con 0x556bc73de4d0 2024-07-31T14:54:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.790+0000 7f86be639740 10 client.0 _ll_register_callbacks cb 0x7f86a8009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.790+0000 7f86b53fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.790+0000 7f86b53fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39918/0 (socket says 172.21.15.67:39918) 2024-07-31T14:54:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.790+0000 7f86b53fe640 1 -- 172.21.15.67:0/840650607 learned_addr learned my addr 172.21.15.67:0/840650607 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.790+0000 7f86b4bfd640 1 --2- 172.21.15.67:0/840650607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.791+0000 7f86b5bff640 1 --2- 172.21.15.67:0/840650607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 0x556bc77b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.791+0000 7f86b53fe640 1 -- 172.21.15.67:0/840650607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 msgr2=0x556bc77b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.791+0000 7f86b53fe640 1 --2- 172.21.15.67:0/840650607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.791+0000 7f86b53fe640 1 -- 172.21.15.67:0/840650607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 msgr2=0x556bc77b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.791+0000 7f86b53fe640 1 --2- 172.21.15.67:0/840650607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 0x556bc77b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.791+0000 7f86b53fe640 1 -- 172.21.15.67:0/840650607 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bc77b2e70 con 0x556bc73d62f0 2024-07-31T14:54:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.791+0000 7f86b53fe640 1 --2- 172.21.15.67:0/840650607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77b5d40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00e9c0 tx=0x7f86ac0a6ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc8bb1449c39b6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.791+0000 7f8698ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.791+0000 7f8698ff9640 1 -- 172.21.15.67:0/840650607 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0c4040 con 0x556bc73d62f0 2024-07-31T14:54:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.791+0000 7f8698ff9640 1 -- 172.21.15.67:0/840650607 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f86ac0a7610 con 0x556bc73d62f0 2024-07-31T14:54:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.791+0000 7f86be639740 1 -- 172.21.15.67:0/840650607 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ac0ab020 con 0x556bc73d62f0 2024-07-31T14:54:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.791+0000 7f8698ff9640 1 -- 172.21.15.67:0/840650607 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f86ac0af620 con 0x556bc73d62f0 2024-07-31T14:54:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.791+0000 7f86be639740 10 client.16440 fetch_fsmap learned FSMap version 38 2024-07-31T14:54:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.791+0000 7f86be639740 1 -- 172.21.15.67:0/840650607 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556bc7826f90 con 0x556bc73d62f0 2024-07-31T14:54:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.792+0000 7f8698ff9640 1 -- 172.21.15.67:0/840650607 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ac0ab020 con 0x556bc73d62f0 2024-07-31T14:54:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.792+0000 7f8698ff9640 1 -- 172.21.15.67:0/840650607 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7f86ac0ab330 con 0x556bc73d62f0 2024-07-31T14:54:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.792+0000 7f86be639740 10 client.16440 fetch_fsmap finished waiting for FSMap version 38 2024-07-31T14:54:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.792+0000 7f86be639740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.792+0000 7f86be639740 20 client.16440 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.792+0000 7f86be639740 1 --2- 172.21.15.67:0/840650607 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bc73931c0 0x556bc78297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.792+0000 7f86be639740 4 client.16440 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:54:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.792+0000 7f86be639740 1 -- 172.21.15.67:0/840650607 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556bc7714da0 con 0x556bc73931c0 2024-07-31T14:54:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.792+0000 7f86b4bfd640 1 --2- 172.21.15.67:0/840650607 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bc73931c0 0x556bc78297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.793+0000 7f86b4bfd640 1 --2- 172.21.15.67:0/840650607 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bc73931c0 0x556bc78297d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f990cbff3c45190 server_cookie=7578ed21564e70ed in_seq=0 out_seq=0 2024-07-31T14:54:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.793+0000 7f8698ff9640 10 client.16440 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.862+0000 7f8698ff9640 1 -- 172.21.15.67:0/840650607 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556bc7714da0 con 0x556bc73931c0 2024-07-31T14:54:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.862+0000 7f8698ff9640 10 client.16440 handle_command_reply: tid=0 2024-07-31T14:54:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.862+0000 7f86be639740 1 client.16440 shutdown 2024-07-31T14:54:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.862+0000 7f86be639740 1 -- 172.21.15.67:0/840650607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 msgr2=0x556bc77b5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.862+0000 7f86be639740 1 --2- 172.21.15.67:0/840650607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77b5d40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00e9c0 tx=0x7f86ac0a6ff0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.862+0000 7f86be639740 1 -- 172.21.15.67:0/840650607 shutdown_connections 2024-07-31T14:54:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.862+0000 7f86be639740 1 --2- 172.21.15.67:0/840650607 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bc73931c0 0x556bc78297d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.862+0000 7f86be639740 1 --2- 172.21.15.67:0/840650607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bc73ff1b0 0x556bc77b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.862+0000 7f86be639740 1 --2- 172.21.15.67:0/840650607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bc73de4d0 0x556bc77b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.862+0000 7f8698ff9640 0 client.16440 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.862+0000 7f86be639740 1 --2- 172.21.15.67:0/840650607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bc73d62f0 0x556bc77b5d40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.862+0000 7f86be639740 1 -- 172.21.15.67:0/840650607 >> 172.21.15.67:0/840650607 conn(0x556bc73f3300 msgr2=0x556bc77a6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.862+0000 7f86be639740 1 -- 172.21.15.67:0/840650607 shutdown_connections 2024-07-31T14:54:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.863+0000 7f86be639740 1 -- 172.21.15.67:0/840650607 wait complete. 2024-07-31T14:54:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.863+0000 7f86be639740 20 client.16440 trim_cache size 0 max 16384 2024-07-31T14:54:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:54:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:54:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:54:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.863+0000 7f86b7662640 1 -- 172.21.15.67:0/1614692954 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f869c0b3d40 msgr2=0x7f869c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.863+0000 7f86b7662640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f869c0b3d40 0x7f869c0b6250 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86a8005fd0 tx=0x7f86a80096a0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.863+0000 7f86b7662640 1 -- 172.21.15.67:0/1614692954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b0113830 msgr2=0x7f86b01a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.863+0000 7f86b7662640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b0113830 0x7f86b01a44d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00e9f0 tx=0x7f86ac00edf0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.864+0000 7f86b7662640 1 -- 172.21.15.67:0/1614692954 shutdown_connections 2024-07-31T14:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.864+0000 7f86b7662640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f869c0b3d40 0x7f869c0b6250 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.864+0000 7f86b7662640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86b01a9d00 0x7f86b01aa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.864+0000 7f86b7662640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86b0114180 0x7f86b01a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.864+0000 7f86b7662640 1 --2- 172.21.15.67:0/1614692954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b0113830 0x7f86b01a44d0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.864+0000 7f86b7662640 1 -- 172.21.15.67:0/1614692954 >> 172.21.15.67:0/1614692954 conn(0x7f86b010df00 msgr2=0x7f86b010f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.864+0000 7f86b7662640 1 -- 172.21.15.67:0/1614692954 shutdown_connections 2024-07-31T14:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:09.864+0000 7f86b7662640 1 -- 172.21.15.67:0/1614692954 wait complete. 2024-07-31T14:54:11.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.118+0000 7f576c450640 1 -- 172.21.15.67:0/2403778861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5764102490 msgr2=0x7f57641028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:11.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.118+0000 7f576c450640 1 --2- 172.21.15.67:0/2403778861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5764102490 0x7f57641028a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5760009900 tx=0x7f576002a410 comp rx=0 tx=0).stop 2024-07-31T14:54:11.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.118+0000 7f576c450640 1 -- 172.21.15.67:0/2403778861 shutdown_connections 2024-07-31T14:54:11.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.118+0000 7f576c450640 1 --2- 172.21.15.67:0/2403778861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5764108720 0x7f576410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.118+0000 7f576c450640 1 --2- 172.21.15.67:0/2403778861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5764107dd0 0x7f57641081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.118+0000 7f576c450640 1 --2- 172.21.15.67:0/2403778861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5764102490 0x7f57641028a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.118+0000 7f576c450640 1 -- 172.21.15.67:0/2403778861 >> 172.21.15.67:0/2403778861 conn(0x7f5764079160 msgr2=0x7f5764079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:11.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.118+0000 7f576c450640 1 -- 172.21.15.67:0/2403778861 shutdown_connections 2024-07-31T14:54:11.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.118+0000 7f576c450640 1 -- 172.21.15.67:0/2403778861 wait complete. 2024-07-31T14:54:11.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.119+0000 7f576c450640 1 Processor -- start 2024-07-31T14:54:11.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.119+0000 7f576c450640 1 -- start start 2024-07-31T14:54:11.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.119+0000 7f576c450640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5764102490 0x7f5764199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.119+0000 7f576c450640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5764107dd0 0x7f576419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.119+0000 7f576c450640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5764108720 0x7f576419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.119+0000 7f576c450640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5764112380 con 0x7f5764107dd0 2024-07-31T14:54:11.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.119+0000 7f576c450640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5764112200 con 0x7f5764108720 2024-07-31T14:54:11.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.119+0000 7f576c450640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5764112500 con 0x7f5764102490 2024-07-31T14:54:11.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.119+0000 7f576a1ec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5764102490 0x7f5764199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.119+0000 7f576a1ec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5764102490 0x7f5764199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49194/0 (socket says 172.21.15.67:49194) 2024-07-31T14:54:11.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.119+0000 7f576a1ec640 1 -- 172.21.15.67:0/2519085250 learned_addr learned my addr 172.21.15.67:0/2519085250 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:11.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.119+0000 7f57699eb640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5764107dd0 0x7f576419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.120+0000 7f576a9ed640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5764108720 0x7f576419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.120+0000 7f576a1ec640 1 -- 172.21.15.67:0/2519085250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5764108720 msgr2=0x7f576419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:11.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.120+0000 7f576a1ec640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5764108720 0x7f576419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.120+0000 7f576a1ec640 1 -- 172.21.15.67:0/2519085250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5764107dd0 msgr2=0x7f576419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:11.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.120+0000 7f576a1ec640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5764107dd0 0x7f576419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.120+0000 7f576a1ec640 1 -- 172.21.15.67:0/2519085250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576419fe10 con 0x7f5764102490 2024-07-31T14:54:11.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.120+0000 7f576a9ed640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5764108720 0x7f576419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:54:11.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.120+0000 7f576a1ec640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5764102490 0x7f5764199ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5760005940 tx=0x7f57600040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:11.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.120+0000 7f575b7fe640 1 -- 172.21.15.67:0/2519085250 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576003f030 con 0x7f5764102490 2024-07-31T14:54:11.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.120+0000 7f576c450640 1 -- 172.21.15.67:0/2519085250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57641a00a0 con 0x7f5764102490 2024-07-31T14:54:11.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.120+0000 7f575b7fe640 1 -- 172.21.15.67:0/2519085250 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57600044c0 con 0x7f5764102490 2024-07-31T14:54:11.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.120+0000 7f575b7fe640 1 -- 172.21.15.67:0/2519085250 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760035070 con 0x7f5764102490 2024-07-31T14:54:11.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.120+0000 7f576c450640 1 -- 172.21.15.67:0/2519085250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57641a6860 con 0x7f5764102490 2024-07-31T14:54:11.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.123+0000 7f57729a1740 1 Processor -- start 2024-07-31T14:54:11.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.123+0000 7f57729a1740 1 -- start start 2024-07-31T14:54:11.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.123+0000 7f57729a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cd1070 0x555e24076550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.123+0000 7f57729a1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e24076a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.123+0000 7f57729a1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cb0390 0x555e24076fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.123+0000 7f57729a1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555e23fecb90 con 0x555e23ca81b0 2024-07-31T14:54:11.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.123+0000 7f57729a1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555e23fdc090 con 0x555e23cb0390 2024-07-31T14:54:11.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.123+0000 7f575b7fe640 1 -- 172.21.15.67:0/2519085250 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5760042030 con 0x7f5764102490 2024-07-31T14:54:11.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.123+0000 7f57729a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e23fe00d0 con 0x555e23cd1070 2024-07-31T14:54:11.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f576a9ed640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cd1070 0x555e24076550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f576a9ed640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cd1070 0x555e24076550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49206/0 (socket says 172.21.15.67:49206) 2024-07-31T14:54:11.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f576a9ed640 1 -- 172.21.15.67:0/3135336940 learned_addr learned my addr 172.21.15.67:0/3135336940 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:11.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f576a1ec640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e24076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f57699eb640 1 --2- 172.21.15.67:0/3135336940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cb0390 0x555e24076fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f576a9ed640 1 -- 172.21.15.67:0/3135336940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cb0390 msgr2=0x555e24076fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:11.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f576a9ed640 1 --2- 172.21.15.67:0/3135336940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cb0390 0x555e24076fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f576a9ed640 1 -- 172.21.15.67:0/3135336940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 msgr2=0x555e24076a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:11.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f576a9ed640 1 --2- 172.21.15.67:0/3135336940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e24076a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f576a9ed640 1 -- 172.21.15.67:0/3135336940 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e23f82b70 con 0x555e23cd1070 2024-07-31T14:54:11.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f576a9ed640 1 --2- 172.21.15.67:0/3135336940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cd1070 0x555e24076550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f575c009f90 tx=0x7f575c00f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33a235246e15c250 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:11.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f575b7fe640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f57500b3d40 0x7f57500b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f5758d99640 1 -- 172.21.15.67:0/3135336940 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c003b70 con 0x555e23cd1070 2024-07-31T14:54:11.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f5758d99640 1 -- 172.21.15.67:0/3135336940 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f575c003d10 con 0x555e23cd1070 2024-07-31T14:54:11.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.124+0000 7f5758d99640 1 -- 172.21.15.67:0/3135336940 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c011940 con 0x555e23cd1070 2024-07-31T14:54:11.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.125+0000 7f576a1ec640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f57500b3d40 0x7f57500b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.125+0000 7f575b7fe640 1 -- 172.21.15.67:0/2519085250 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f5760104770 con 0x7f5764102490 2024-07-31T14:54:11.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.126+0000 7f57729a1740 1 -- 172.21.15.67:0/3135336940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cd1070 msgr2=0x555e24076550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:11.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.126+0000 7f576a1ec640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f57500b3d40 0x7f57500b6250 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x555e240751d0 tx=0x7f576010a530 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:11.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.126+0000 7f57729a1740 1 --2- 172.21.15.67:0/3135336940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cd1070 0x555e24076550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f575c009f90 tx=0x7f575c00f7f0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.126+0000 7f57729a1740 1 -- 172.21.15.67:0/3135336940 shutdown_connections 2024-07-31T14:54:11.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.126+0000 7f57729a1740 1 --2- 172.21.15.67:0/3135336940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cb0390 0x555e24076fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.126+0000 7f57729a1740 1 --2- 172.21.15.67:0/3135336940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e24076a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.126+0000 7f57729a1740 1 --2- 172.21.15.67:0/3135336940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cd1070 0x555e24076550 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.126+0000 7f57729a1740 1 -- 172.21.15.67:0/3135336940 >> 172.21.15.67:0/3135336940 conn(0x555e23cc5460 msgr2=0x555e24074440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:11.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.126+0000 7f57729a1740 1 -- 172.21.15.67:0/3135336940 shutdown_connections 2024-07-31T14:54:11.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.126+0000 7f57729a1740 1 -- 172.21.15.67:0/3135336940 wait complete. 2024-07-31T14:54:11.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.126+0000 7f57729a1740 1 -- start start 2024-07-31T14:54:11.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.126+0000 7f57729a1740 1 Processor -- start 2024-07-31T14:54:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.127+0000 7f57729a1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e24086c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.127+0000 7f57729a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cb0390 0x555e240871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.127+0000 7f57729a1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cd1070 0x555e2408c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.127+0000 7f57729a1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555e23e3f700 con 0x555e23ca81b0 2024-07-31T14:54:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.127+0000 7f57729a1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555e23f7e9c0 con 0x555e23cd1070 2024-07-31T14:54:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.127+0000 7f57729a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e24084d30 con 0x555e23cb0390 2024-07-31T14:54:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.127+0000 7f576a9ed640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e24086c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.127+0000 7f576a9ed640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e24086c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50570/0 (socket says 172.21.15.67:50570) 2024-07-31T14:54:11.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.127+0000 7f576a9ed640 1 -- 172.21.15.67:0/965191487 learned_addr learned my addr 172.21.15.67:0/965191487 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:11.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.127+0000 7f576a9ed640 1 --2- 172.21.15.67:0/965191487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cd1070 0x555e2408c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.127+0000 7f57729a1740 10 client.0 _ll_register_callbacks cb 0x555e23f994d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:11.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.127+0000 7f57699eb640 1 --2- 172.21.15.67:0/965191487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cb0390 0x555e240871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.128+0000 7f576a9ed640 1 -- 172.21.15.67:0/965191487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cb0390 msgr2=0x555e240871e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:11.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.128+0000 7f576a9ed640 1 --2- 172.21.15.67:0/965191487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cb0390 0x555e240871e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.128+0000 7f576a9ed640 1 -- 172.21.15.67:0/965191487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cd1070 msgr2=0x555e2408c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:11.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.128+0000 7f576a9ed640 1 --2- 172.21.15.67:0/965191487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cd1070 0x555e2408c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.128+0000 7f576a9ed640 1 -- 172.21.15.67:0/965191487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e23d3c650 con 0x555e23ca81b0 2024-07-31T14:54:11.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.128+0000 7f576a9ed640 1 --2- 172.21.15.67:0/965191487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e24086c00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f575c006010 tx=0x7f575c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2ab3c0fe572b1d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:11.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.128+0000 7f574dffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:11.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.129+0000 7f574dffb640 1 -- 172.21.15.67:0/965191487 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c004cd0 con 0x555e23ca81b0 2024-07-31T14:54:11.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.129+0000 7f574dffb640 1 -- 172.21.15.67:0/965191487 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f575c003ec0 con 0x555e23ca81b0 2024-07-31T14:54:11.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.129+0000 7f57729a1740 1 -- 172.21.15.67:0/965191487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f575c003d10 con 0x555e23ca81b0 2024-07-31T14:54:11.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.129+0000 7f574dffb640 1 -- 172.21.15.67:0/965191487 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f575c033050 con 0x555e23ca81b0 2024-07-31T14:54:11.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.129+0000 7f57729a1740 10 client.16452 fetch_fsmap learned FSMap version 38 2024-07-31T14:54:11.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.129+0000 7f57729a1740 1 -- 172.21.15.67:0/965191487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555e24095150 con 0x555e23ca81b0 2024-07-31T14:54:11.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.129+0000 7f574dffb640 1 -- 172.21.15.67:0/965191487 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f575c003d10 con 0x555e23ca81b0 2024-07-31T14:54:11.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.129+0000 7f574dffb640 1 -- 172.21.15.67:0/965191487 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7f575c011ba0 con 0x555e23ca81b0 2024-07-31T14:54:11.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.129+0000 7f57729a1740 10 client.16452 fetch_fsmap finished waiting for FSMap version 38 2024-07-31T14:54:11.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.129+0000 7f57729a1740 10 client.16452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:11.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.129+0000 7f57729a1740 20 client.16452 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:11.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.130+0000 7f57729a1740 1 --2- 172.21.15.67:0/965191487 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e23c8f9f0 0x555e24098570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.130+0000 7f57729a1740 4 client.16452 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:54:11.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.130+0000 7f57729a1740 1 -- 172.21.15.67:0/965191487 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e23feeae0 con 0x555e23c8f9f0 2024-07-31T14:54:11.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.130+0000 7f57699eb640 1 --2- 172.21.15.67:0/965191487 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e23c8f9f0 0x555e24098570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.130+0000 7f574dffb640 10 client.16452 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:11.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.130+0000 7f57699eb640 1 --2- 172.21.15.67:0/965191487 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e23c8f9f0 0x555e24098570 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=500f05c2544b8a71 server_cookie=305476e331413b7d in_seq=0 out_seq=0 2024-07-31T14:54:11.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.131+0000 7f574dffb640 1 -- 172.21.15.67:0/965191487 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e23feeae0 con 0x555e23c8f9f0 2024-07-31T14:54:11.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.131+0000 7f574dffb640 10 client.16452 handle_command_reply: tid=0 2024-07-31T14:54:11.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.131+0000 7f57729a1740 1 client.16452 shutdown 2024-07-31T14:54:11.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.131+0000 7f57729a1740 1 -- 172.21.15.67:0/965191487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 msgr2=0x555e24086c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:11.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.131+0000 7f57729a1740 1 --2- 172.21.15.67:0/965191487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e24086c00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f575c006010 tx=0x7f575c0115a0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.131+0000 7f57729a1740 1 -- 172.21.15.67:0/965191487 shutdown_connections 2024-07-31T14:54:11.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.131+0000 7f57729a1740 1 --2- 172.21.15.67:0/965191487 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e23c8f9f0 0x555e24098570 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.131+0000 7f57729a1740 1 --2- 172.21.15.67:0/965191487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cd1070 0x555e2408c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.131+0000 7f57729a1740 1 --2- 172.21.15.67:0/965191487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cb0390 0x555e240871e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.131+0000 7f57729a1740 1 --2- 172.21.15.67:0/965191487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e24086c00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.131+0000 7f57729a1740 1 -- 172.21.15.67:0/965191487 >> 172.21.15.67:0/965191487 conn(0x555e23cc5460 msgr2=0x555e24072c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:11.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.131+0000 7f574dffb640 0 client.16452 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:11.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.131+0000 7f57729a1740 1 -- 172.21.15.67:0/965191487 shutdown_connections 2024-07-31T14:54:11.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.131+0000 7f57729a1740 1 -- 172.21.15.67:0/965191487 wait complete. 2024-07-31T14:54:11.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.132+0000 7f57729a1740 20 client.16452 trim_cache size 0 max 16384 2024-07-31T14:54:11.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.156+0000 7f57729a1740 1 Processor -- start 2024-07-31T14:54:11.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.156+0000 7f57729a1740 1 -- start start 2024-07-31T14:54:11.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.156+0000 7f57729a1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23ca81b0 0x555e2407fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.156+0000 7f57729a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cb0390 0x555e24080190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.156+0000 7f57729a1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23cd1070 0x555e24092ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.156+0000 7f57729a1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f57641ae460 con 0x555e23cd1070 2024-07-31T14:54:11.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.156+0000 7f57729a1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f57641b00f0 con 0x555e23ca81b0 2024-07-31T14:54:11.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.156+0000 7f57729a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f57641aeb30 con 0x555e23cb0390 2024-07-31T14:54:11.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.157+0000 7f57699eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cb0390 0x555e24080190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.157+0000 7f576a9ed640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23cd1070 0x555e24092ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.157+0000 7f57699eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cb0390 0x555e24080190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49220/0 (socket says 172.21.15.67:49220) 2024-07-31T14:54:11.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.157+0000 7f57699eb640 1 -- 172.21.15.67:0/3680980950 learned_addr learned my addr 172.21.15.67:0/3680980950 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:11.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.157+0000 7f576a9ed640 1 --2- 172.21.15.67:0/3680980950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23ca81b0 0x555e2407fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.157+0000 7f57699eb640 1 -- 172.21.15.67:0/3680980950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23ca81b0 msgr2=0x555e2407fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:11.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.157+0000 7f57699eb640 1 --2- 172.21.15.67:0/3680980950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23ca81b0 0x555e2407fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.157+0000 7f57699eb640 1 -- 172.21.15.67:0/3680980950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23cd1070 msgr2=0x555e24092ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.157+0000 7f57699eb640 1 --2- 172.21.15.67:0/3680980950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23cd1070 0x555e24092ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.157+0000 7f57699eb640 1 -- 172.21.15.67:0/3680980950 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e24081ba0 con 0x555e23cb0390 2024-07-31T14:54:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.157+0000 7f576a9ed640 1 --2- 172.21.15.67:0/3680980950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23cd1070 0x555e24092ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.157+0000 7f57699eb640 1 --2- 172.21.15.67:0/3680980950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cb0390 0x555e24080190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f576406a4d0 tx=0x7f5754012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88c8694d97d370c0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.158+0000 7f574ffff640 1 -- 172.21.15.67:0/3680980950 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754004050 con 0x555e23cb0390 2024-07-31T14:54:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.158+0000 7f574ffff640 1 -- 172.21.15.67:0/3680980950 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57540041f0 con 0x555e23cb0390 2024-07-31T14:54:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.158+0000 7f574ffff640 1 -- 172.21.15.67:0/3680980950 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575401d690 con 0x555e23cb0390 2024-07-31T14:54:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.158+0000 7f57729a1740 1 -- 172.21.15.67:0/3680980950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cb0390 msgr2=0x555e24080190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.158+0000 7f57729a1740 1 --2- 172.21.15.67:0/3680980950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cb0390 0x555e24080190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f576406a4d0 tx=0x7f5754012820 comp rx=0 tx=0).stop 2024-07-31T14:54:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 -- 172.21.15.67:0/3680980950 shutdown_connections 2024-07-31T14:54:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 --2- 172.21.15.67:0/3680980950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23cd1070 0x555e24092ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 --2- 172.21.15.67:0/3680980950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cb0390 0x555e24080190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 --2- 172.21.15.67:0/3680980950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23ca81b0 0x555e2407fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 -- 172.21.15.67:0/3680980950 >> 172.21.15.67:0/3680980950 conn(0x555e23cc5460 msgr2=0x555e24085280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:11.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 -- 172.21.15.67:0/3680980950 shutdown_connections 2024-07-31T14:54:11.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 -- 172.21.15.67:0/3680980950 wait complete. 2024-07-31T14:54:11.162 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:54:11.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 -- start start 2024-07-31T14:54:11.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 Processor -- start 2024-07-31T14:54:11.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e240929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cb0390 0x555e24092fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cd1070 0x555e2408bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f57641afa20 con 0x555e23ca81b0 2024-07-31T14:54:11.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f57641af290 con 0x555e23cb0390 2024-07-31T14:54:11.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.159+0000 7f57729a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f57641ac440 con 0x555e23cd1070 2024-07-31T14:54:11.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f57729a1740 10 client.0 _ll_register_callbacks cb 0x7f575c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:11.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f576a9ed640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e240929d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f576a9ed640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e240929d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50590/0 (socket says 172.21.15.67:50590) 2024-07-31T14:54:11.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f576a9ed640 1 -- 172.21.15.67:0/1446731643 learned_addr learned my addr 172.21.15.67:0/1446731643 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:11.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f576a9ed640 1 --2- 172.21.15.67:0/1446731643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cd1070 0x555e2408bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f57699eb640 1 --2- 172.21.15.67:0/1446731643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cb0390 0x555e24092fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f576a9ed640 1 -- 172.21.15.67:0/1446731643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cd1070 msgr2=0x555e2408bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f576a9ed640 1 --2- 172.21.15.67:0/1446731643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cd1070 0x555e2408bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f576a9ed640 1 -- 172.21.15.67:0/1446731643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cb0390 msgr2=0x555e24092fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f576a9ed640 1 --2- 172.21.15.67:0/1446731643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cb0390 0x555e24092fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f576a9ed640 1 -- 172.21.15.67:0/1446731643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e2408c670 con 0x555e23ca81b0 2024-07-31T14:54:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f57699eb640 1 --2- 172.21.15.67:0/1446731643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cb0390 0x555e24092fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:54:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f576a9ed640 1 --2- 172.21.15.67:0/1446731643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e240929d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f575c000f80 tx=0x7f575c009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98af38934c9c3d4c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f574dffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f574dffb640 1 -- 172.21.15.67:0/1446731643 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c02da40 con 0x555e23ca81b0 2024-07-31T14:54:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f574dffb640 1 -- 172.21.15.67:0/1446731643 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f575c02dbe0 con 0x555e23ca81b0 2024-07-31T14:54:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.160+0000 7f57729a1740 1 -- 172.21.15.67:0/1446731643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57540041f0 con 0x555e23ca81b0 2024-07-31T14:54:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.161+0000 7f574dffb640 1 -- 172.21.15.67:0/1446731643 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f575c02dde0 con 0x555e23ca81b0 2024-07-31T14:54:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.161+0000 7f57729a1740 10 client.16458 fetch_fsmap learned FSMap version 38 2024-07-31T14:54:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.161+0000 7f57729a1740 1 -- 172.21.15.67:0/1446731643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555e24092080 con 0x555e23ca81b0 2024-07-31T14:54:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.161+0000 7f574dffb640 1 -- 172.21.15.67:0/1446731643 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57540041f0 con 0x555e23ca81b0 2024-07-31T14:54:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.161+0000 7f574dffb640 1 -- 172.21.15.67:0/1446731643 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7f575c003dd0 con 0x555e23ca81b0 2024-07-31T14:54:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.161+0000 7f57729a1740 10 client.16458 fetch_fsmap finished waiting for FSMap version 38 2024-07-31T14:54:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.161+0000 7f57729a1740 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.161+0000 7f57729a1740 20 client.16458 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.161+0000 7f57729a1740 1 --2- 172.21.15.67:0/1446731643 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e23c64da0 0x555e24102d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.161+0000 7f57729a1740 4 client.16458 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:54:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.161+0000 7f57729a1740 1 -- 172.21.15.67:0/1446731643 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e23fea580 con 0x555e23c64da0 2024-07-31T14:54:11.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.162+0000 7f57699eb640 1 --2- 172.21.15.67:0/1446731643 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e23c64da0 0x555e24102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:11.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.162+0000 7f574dffb640 10 client.16458 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:11.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.162+0000 7f57699eb640 1 --2- 172.21.15.67:0/1446731643 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e23c64da0 0x555e24102d90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68e547df260f974d server_cookie=18faaa73fdaefcce in_seq=0 out_seq=0 2024-07-31T14:54:11.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f574dffb640 1 -- 172.21.15.67:0/1446731643 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555e23fea580 con 0x555e23c64da0 2024-07-31T14:54:11.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f574dffb640 10 client.16458 handle_command_reply: tid=0 2024-07-31T14:54:11.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f57729a1740 1 client.16458 shutdown 2024-07-31T14:54:11.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f57729a1740 1 -- 172.21.15.67:0/1446731643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 msgr2=0x555e240929d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:11.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f57729a1740 1 --2- 172.21.15.67:0/1446731643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e240929d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f575c000f80 tx=0x7f575c009be0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f57729a1740 1 -- 172.21.15.67:0/1446731643 shutdown_connections 2024-07-31T14:54:11.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f57729a1740 1 --2- 172.21.15.67:0/1446731643 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e23c64da0 0x555e24102d90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f57729a1740 1 --2- 172.21.15.67:0/1446731643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e23cd1070 0x555e2408bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f57729a1740 1 --2- 172.21.15.67:0/1446731643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e23cb0390 0x555e24092fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f57729a1740 1 --2- 172.21.15.67:0/1446731643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e23ca81b0 0x555e240929d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f57729a1740 1 -- 172.21.15.67:0/1446731643 >> 172.21.15.67:0/1446731643 conn(0x555e23cc5460 msgr2=0x555e24073d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f574dffb640 0 client.16458 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f57729a1740 1 -- 172.21.15.67:0/1446731643 shutdown_connections 2024-07-31T14:54:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f57729a1740 1 -- 172.21.15.67:0/1446731643 wait complete. 2024-07-31T14:54:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.163+0000 7f57729a1740 20 client.16458 trim_cache size 0 max 16384 2024-07-31T14:54:11.170 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:54:11.171 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T14:54:11.171 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T14:54:11.171 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:54:11.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.164+0000 7f576c450640 1 -- 172.21.15.67:0/2519085250 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f57500b3d40 msgr2=0x7f57500b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:11.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.164+0000 7f576c450640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f57500b3d40 0x7f57500b6250 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x555e240751d0 tx=0x7f576010a530 comp rx=0 tx=0).stop 2024-07-31T14:54:11.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.164+0000 7f576c450640 1 -- 172.21.15.67:0/2519085250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5764102490 msgr2=0x7f5764199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:11.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.164+0000 7f576c450640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5764102490 0x7f5764199ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5760005940 tx=0x7f57600040c0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.164+0000 7f576c450640 1 -- 172.21.15.67:0/2519085250 shutdown_connections 2024-07-31T14:54:11.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.164+0000 7f576c450640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f57500b3d40 0x7f57500b6250 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.164+0000 7f576c450640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5764108720 0x7f576419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.164+0000 7f576c450640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5764107dd0 0x7f576419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.164+0000 7f576c450640 1 --2- 172.21.15.67:0/2519085250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5764102490 0x7f5764199ea0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.164+0000 7f576c450640 1 -- 172.21.15.67:0/2519085250 >> 172.21.15.67:0/2519085250 conn(0x7f5764079160 msgr2=0x7f5764101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.164+0000 7f576c450640 1 -- 172.21.15.67:0/2519085250 shutdown_connections 2024-07-31T14:54:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:11.164+0000 7f576c450640 1 -- 172.21.15.67:0/2519085250 wait complete. 2024-07-31T14:54:11.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:11 smithi067 ceph-mon[32750]: pgmap v493: 129 pgs: 129 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 670 op/s 2024-07-31T14:54:11.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:11 smithi073 ceph-mon[36714]: pgmap v493: 129 pgs: 129 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 670 op/s 2024-07-31T14:54:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:11 smithi077 ceph-mon[37327]: pgmap v493: 129 pgs: 129 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 670 op/s 2024-07-31T14:54:12.694 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T14:54:12.695 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:54:13.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:13 smithi067 ceph-mon[32750]: pgmap v494: 129 pgs: 129 active+clean; 298 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.7 MiB/s wr, 605 op/s 2024-07-31T14:54:13.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:13 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:54:13.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:13 smithi073 ceph-mon[36714]: pgmap v494: 129 pgs: 129 active+clean; 298 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.7 MiB/s wr, 605 op/s 2024-07-31T14:54:13.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:13 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:54:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:13 smithi077 ceph-mon[37327]: pgmap v494: 129 pgs: 129 active+clean; 298 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.7 MiB/s wr, 605 op/s 2024-07-31T14:54:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:13 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:54:13.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.944+0000 7fa29b935640 1 -- 172.21.15.67:0/1930613844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa294108ee0 msgr2=0x7fa2941092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:13.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.944+0000 7fa29b935640 1 --2- 172.21.15.67:0/1930613844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa294108ee0 0x7fa2941092f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa284009900 tx=0x7fa28402a410 comp rx=0 tx=0).stop 2024-07-31T14:54:13.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.944+0000 7fa29b935640 1 -- 172.21.15.67:0/1930613844 shutdown_connections 2024-07-31T14:54:13.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.944+0000 7fa29b935640 1 --2- 172.21.15.67:0/1930613844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa294109830 0x7fa294110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.944+0000 7fa29b935640 1 --2- 172.21.15.67:0/1930613844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa294108ee0 0x7fa2941092f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.944+0000 7fa29b935640 1 --2- 172.21.15.67:0/1930613844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2941035a0 0x7fa2941039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.944+0000 7fa29b935640 1 -- 172.21.15.67:0/1930613844 >> 172.21.15.67:0/1930613844 conn(0x7fa2940fefd0 msgr2=0x7fa294101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.944+0000 7fa29b935640 1 -- 172.21.15.67:0/1930613844 shutdown_connections 2024-07-31T14:54:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.944+0000 7fa29b935640 1 -- 172.21.15.67:0/1930613844 wait complete. 2024-07-31T14:54:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa29b935640 1 Processor -- start 2024-07-31T14:54:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa29b935640 1 -- start start 2024-07-31T14:54:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa29b935640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2941035a0 0x7fa294199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa29b935640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa294108ee0 0x7fa29419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa29b935640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa294109830 0x7fa29419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa29b935640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941134c0 con 0x7fa2941035a0 2024-07-31T14:54:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa29b935640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa294113340 con 0x7fa294108ee0 2024-07-31T14:54:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa29b935640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa294113640 con 0x7fa294109830 2024-07-31T14:54:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa2996d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2941035a0 0x7fa294199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa2996d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2941035a0 0x7fa294199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50618/0 (socket says 172.21.15.67:50618) 2024-07-31T14:54:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa2996d1640 1 -- 172.21.15.67:0/1781045034 learned_addr learned my addr 172.21.15.67:0/1781045034 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa299ed2640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa294109830 0x7fa29419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa298ed0640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa294108ee0 0x7fa29419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa2996d1640 1 -- 172.21.15.67:0/1781045034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa294109830 msgr2=0x7fa29419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa2996d1640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa294109830 0x7fa29419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa2996d1640 1 -- 172.21.15.67:0/1781045034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa294108ee0 msgr2=0x7fa29419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa2996d1640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa294108ee0 0x7fa29419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa2996d1640 1 -- 172.21.15.67:0/1781045034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29419fe40 con 0x7fa2941035a0 2024-07-31T14:54:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.945+0000 7fa299ed2640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa294109830 0x7fa29419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.946+0000 7fa2996d1640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2941035a0 0x7fa294199ed0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa29000d950 tx=0x7fa29000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.946+0000 7fa28a7fc640 1 -- 172.21.15.67:0/1781045034 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290017070 con 0x7fa2941035a0 2024-07-31T14:54:13.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.946+0000 7fa28a7fc640 1 -- 172.21.15.67:0/1781045034 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa290009c30 con 0x7fa2941035a0 2024-07-31T14:54:13.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.946+0000 7fa28a7fc640 1 -- 172.21.15.67:0/1781045034 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29000f620 con 0x7fa2941035a0 2024-07-31T14:54:13.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.946+0000 7fa29b935640 1 -- 172.21.15.67:0/1781045034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2941a0130 con 0x7fa2941035a0 2024-07-31T14:54:13.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.947+0000 7fa29b935640 1 -- 172.21.15.67:0/1781045034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2941a6890 con 0x7fa2941035a0 2024-07-31T14:54:13.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.949+0000 7fa28a7fc640 1 -- 172.21.15.67:0/1781045034 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fa290010460 con 0x7fa2941035a0 2024-07-31T14:54:13.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.950+0000 7fa2a1e86740 1 Processor -- start 2024-07-31T14:54:13.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.950+0000 7fa2a1e86740 1 -- start start 2024-07-31T14:54:13.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa2a1e86740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76835840 0x563b76bdb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa2a1e86740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b7680c980 0x563b76bdbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa28a7fc640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2800b3e90 0x7fa2800b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa2a1e86740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b76814b60 0x563b76bdc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa2a1e86740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563b76b507b0 con 0x563b76814b60 2024-07-31T14:54:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa2a1e86740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563b76b3fcb0 con 0x563b76835840 2024-07-31T14:54:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa2a1e86740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563b76b43cf0 con 0x563b7680c980 2024-07-31T14:54:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa2996d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b7680c980 0x563b76bdbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa299ed2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b76814b60 0x563b76bdc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa298ed0640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2800b3e90 0x7fa2800b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa299ed2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b76814b60 0x563b76bdc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50620/0 (socket says 172.21.15.67:50620) 2024-07-31T14:54:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa2996d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b7680c980 0x563b76bdbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49258/0 (socket says 172.21.15.67:49258) 2024-07-31T14:54:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa299ed2640 1 -- 172.21.15.67:0/1465465744 learned_addr learned my addr 172.21.15.67:0/1465465744 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa299ed2640 1 --2- 172.21.15.67:0/1465465744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76835840 0x563b76bdb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa28a7fc640 1 -- 172.21.15.67:0/1781045034 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fa2900e3d90 con 0x7fa2941035a0 2024-07-31T14:54:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa2996d1640 1 -- 172.21.15.67:0/1465465744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76835840 msgr2=0x563b76bdb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa2996d1640 1 --2- 172.21.15.67:0/1465465744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76835840 0x563b76bdb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa2996d1640 1 -- 172.21.15.67:0/1465465744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b76814b60 msgr2=0x563b76bdc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa2996d1640 1 --2- 172.21.15.67:0/1465465744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b76814b60 0x563b76bdc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa2996d1640 1 -- 172.21.15.67:0/1465465744 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b76ad50b0 con 0x563b7680c980 2024-07-31T14:54:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.951+0000 7fa299ed2640 1 --2- 172.21.15.67:0/1465465744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b76814b60 0x563b76bdc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.952+0000 7fa2996d1640 1 --2- 172.21.15.67:0/1465465744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b7680c980 0x563b76bdbdb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa2900e11d0 tx=0x7fa29001e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=650aeaedcb61210a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.952+0000 7fa298ed0640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2800b3e90 0x7fa2800b63a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa284009230 tx=0x7fa284031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.952+0000 7fa27f7fe640 1 -- 172.21.15.67:0/1465465744 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900a4e60 con 0x563b7680c980 2024-07-31T14:54:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.952+0000 7fa27f7fe640 1 -- 172.21.15.67:0/1465465744 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2900aa740 con 0x563b7680c980 2024-07-31T14:54:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.952+0000 7fa27f7fe640 1 -- 172.21.15.67:0/1465465744 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290020070 con 0x563b7680c980 2024-07-31T14:54:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.953+0000 7fa2a1e86740 1 -- 172.21.15.67:0/1465465744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b7680c980 msgr2=0x563b76bdbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.953+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/1465465744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b7680c980 0x563b76bdbdb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa2900e11d0 tx=0x7fa29001e000 comp rx=0 tx=0).stop 2024-07-31T14:54:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.953+0000 7fa2a1e86740 1 -- 172.21.15.67:0/1465465744 shutdown_connections 2024-07-31T14:54:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.953+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/1465465744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b76814b60 0x563b76bdc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.953+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/1465465744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b7680c980 0x563b76bdbdb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.953+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/1465465744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76835840 0x563b76bdb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.953+0000 7fa2a1e86740 1 -- 172.21.15.67:0/1465465744 >> 172.21.15.67:0/1465465744 conn(0x563b76829990 msgr2=0x563b76bd9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.953+0000 7fa2a1e86740 1 -- 172.21.15.67:0/1465465744 shutdown_connections 2024-07-31T14:54:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.953+0000 7fa2a1e86740 1 -- 172.21.15.67:0/1465465744 wait complete. 2024-07-31T14:54:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.953+0000 7fa2a1e86740 1 -- start start 2024-07-31T14:54:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.954+0000 7fa2a1e86740 1 Processor -- start 2024-07-31T14:54:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.954+0000 7fa2a1e86740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b7680c980 0x563b76be6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.954+0000 7fa2a1e86740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76814b60 0x563b76be73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.954+0000 7fa2a1e86740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76835840 0x563b76be7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.954+0000 7fa2a1e86740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563b769a37a0 con 0x563b7680c980 2024-07-31T14:54:13.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.954+0000 7fa2a1e86740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563b76ae2320 con 0x563b76814b60 2024-07-31T14:54:13.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.954+0000 7fa2a1e86740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563b76be9f90 con 0x563b76835840 2024-07-31T14:54:13.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.954+0000 7fa2a1e86740 10 client.0 _ll_register_callbacks cb 0x563b76aec140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:13.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.954+0000 7fa2996d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b7680c980 0x563b76be6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.954+0000 7fa2996d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b7680c980 0x563b76be6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50636/0 (socket says 172.21.15.67:50636) 2024-07-31T14:54:13.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.954+0000 7fa2996d1640 1 -- 172.21.15.67:0/749999089 learned_addr learned my addr 172.21.15.67:0/749999089 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.954+0000 7fa299ed2640 1 --2- 172.21.15.67:0/749999089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76835840 0x563b76be7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.954+0000 7fa298ed0640 1 --2- 172.21.15.67:0/749999089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76814b60 0x563b76be73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa2996d1640 1 -- 172.21.15.67:0/749999089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76835840 msgr2=0x563b76be7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa2996d1640 1 --2- 172.21.15.67:0/749999089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76835840 0x563b76be7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa2996d1640 1 -- 172.21.15.67:0/749999089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76814b60 msgr2=0x563b76be73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa2996d1640 1 --2- 172.21.15.67:0/749999089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76814b60 0x563b76be73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa2996d1640 1 -- 172.21.15.67:0/749999089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b76ae64b0 con 0x563b7680c980 2024-07-31T14:54:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa299ed2640 1 --2- 172.21.15.67:0/749999089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76835840 0x563b76be7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa2996d1640 1 --2- 172.21.15.67:0/749999089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b7680c980 0x563b76be6e90 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa2900e3040 tx=0x7fa29001df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1457b81b4491a0f8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa27cff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa27cff9640 1 -- 172.21.15.67:0/749999089 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900c4060 con 0x563b7680c980 2024-07-31T14:54:13.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa27cff9640 1 -- 172.21.15.67:0/749999089 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fa2900aa9a0 con 0x563b7680c980 2024-07-31T14:54:13.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa2a1e86740 1 -- 172.21.15.67:0/749999089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2900aa740 con 0x563b7680c980 2024-07-31T14:54:13.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa27cff9640 1 -- 172.21.15.67:0/749999089 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa2900a4e60 con 0x563b7680c980 2024-07-31T14:54:13.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa2a1e86740 10 client.16470 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:13.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa2a1e86740 1 -- 172.21.15.67:0/749999089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563b768afb90 con 0x563b7680c980 2024-07-31T14:54:13.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.955+0000 7fa27cff9640 1 -- 172.21.15.67:0/749999089 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2900aa740 con 0x563b7680c980 2024-07-31T14:54:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.956+0000 7fa27cff9640 1 -- 172.21.15.67:0/749999089 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7fa2900b32b0 con 0x563b7680c980 2024-07-31T14:54:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.956+0000 7fa2a1e86740 10 client.16470 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.956+0000 7fa2a1e86740 10 client.16470 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:54:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.956+0000 7fa2a1e86740 20 client.16470 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.956+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/749999089 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563b767f41c0 0x563b76bf8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.956+0000 7fa2a1e86740 4 client.16470 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:54:13.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.956+0000 7fa2a1e86740 1 -- 172.21.15.67:0/749999089 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b76b52700 con 0x563b767f41c0 2024-07-31T14:54:13.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.956+0000 7fa298ed0640 1 --2- 172.21.15.67:0/749999089 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563b767f41c0 0x563b76bf8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.956+0000 7fa27cff9640 10 client.16470 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:13.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.956+0000 7fa298ed0640 1 --2- 172.21.15.67:0/749999089 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563b767f41c0 0x563b76bf8da0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d819a28995b53d6 server_cookie=73281aaf4f414399 in_seq=0 out_seq=0 2024-07-31T14:54:13.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.957+0000 7fa27cff9640 1 -- 172.21.15.67:0/749999089 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b76b52700 con 0x563b767f41c0 2024-07-31T14:54:13.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.957+0000 7fa27cff9640 10 client.16470 handle_command_reply: tid=0 2024-07-31T14:54:13.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.957+0000 7fa2a1e86740 1 client.16470 shutdown 2024-07-31T14:54:13.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.957+0000 7fa2a1e86740 1 -- 172.21.15.67:0/749999089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b7680c980 msgr2=0x563b76be6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:13.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.957+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/749999089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b7680c980 0x563b76be6e90 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa2900e3040 tx=0x7fa29001df90 comp rx=0 tx=0).stop 2024-07-31T14:54:13.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.957+0000 7fa2a1e86740 1 -- 172.21.15.67:0/749999089 shutdown_connections 2024-07-31T14:54:13.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.957+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/749999089 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563b767f41c0 0x563b76bf8da0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.957+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/749999089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76835840 0x563b76be7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.957+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/749999089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76814b60 0x563b76be73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.958+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/749999089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b7680c980 0x563b76be6e90 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.958+0000 7fa2a1e86740 1 -- 172.21.15.67:0/749999089 >> 172.21.15.67:0/749999089 conn(0x563b76829990 msgr2=0x563b76bdea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:13.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.958+0000 7fa27cff9640 0 client.16470 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:13.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.958+0000 7fa2a1e86740 1 -- 172.21.15.67:0/749999089 shutdown_connections 2024-07-31T14:54:13.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.958+0000 7fa2a1e86740 1 -- 172.21.15.67:0/749999089 wait complete. 2024-07-31T14:54:13.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.958+0000 7fa2a1e86740 20 client.16470 trim_cache size 0 max 16384 2024-07-31T14:54:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa2a1e86740 1 Processor -- start 2024-07-31T14:54:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa2a1e86740 1 -- start start 2024-07-31T14:54:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa2a1e86740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b7680c980 0x563b76bf28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa2a1e86740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b76814b60 0x563b76be0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa2a1e86740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76835840 0x563b76be4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa2a1e86740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941ae5a0 con 0x563b76814b60 2024-07-31T14:54:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa2a1e86740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941b0260 con 0x563b7680c980 2024-07-31T14:54:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa2a1e86740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941aec70 con 0x563b76835840 2024-07-31T14:54:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa298ed0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b76814b60 0x563b76be0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa299ed2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76835840 0x563b76be4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa299ed2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76835840 0x563b76be4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49276/0 (socket says 172.21.15.67:49276) 2024-07-31T14:54:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa298ed0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b76814b60 0x563b76be0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50642/0 (socket says 172.21.15.67:50642) 2024-07-31T14:54:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa299ed2640 1 -- 172.21.15.67:0/1050932337 learned_addr learned my addr 172.21.15.67:0/1050932337 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa2996d1640 1 --2- 172.21.15.67:0/1050932337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b7680c980 0x563b76bf28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa299ed2640 1 -- 172.21.15.67:0/1050932337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b7680c980 msgr2=0x563b76bf28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:13.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa299ed2640 1 --2- 172.21.15.67:0/1050932337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b7680c980 0x563b76bf28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa299ed2640 1 -- 172.21.15.67:0/1050932337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b76814b60 msgr2=0x563b76be0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:13.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.978+0000 7fa299ed2640 1 --2- 172.21.15.67:0/1050932337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b76814b60 0x563b76be0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.979+0000 7fa299ed2640 1 -- 172.21.15.67:0/1050932337 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b76be8150 con 0x563b76835840 2024-07-31T14:54:13.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.979+0000 7fa298ed0640 1 --2- 172.21.15.67:0/1050932337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b76814b60 0x563b76be0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:13.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.979+0000 7fa299ed2640 1 --2- 172.21.15.67:0/1050932337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76835840 0x563b76be4080 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00e930 tx=0x7fa28c009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33dcf96c118315b7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:13.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.979+0000 7fa27effd640 1 -- 172.21.15.67:0/1050932337 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c029070 con 0x563b76835840 2024-07-31T14:54:13.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.979+0000 7fa27effd640 1 -- 172.21.15.67:0/1050932337 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa28c019e10 con 0x563b76835840 2024-07-31T14:54:13.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.979+0000 7fa27effd640 1 -- 172.21.15.67:0/1050932337 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c004770 con 0x563b76835840 2024-07-31T14:54:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.980+0000 7fa2a1e86740 1 -- 172.21.15.67:0/1050932337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76835840 msgr2=0x563b76be4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.980+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/1050932337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76835840 0x563b76be4080 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00e930 tx=0x7fa28c009f30 comp rx=0 tx=0).stop 2024-07-31T14:54:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.980+0000 7fa2a1e86740 1 -- 172.21.15.67:0/1050932337 shutdown_connections 2024-07-31T14:54:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.980+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/1050932337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76835840 0x563b76be4080 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.980+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/1050932337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b76814b60 0x563b76be0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.980+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/1050932337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b7680c980 0x563b76bf28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.980+0000 7fa2a1e86740 1 -- 172.21.15.67:0/1050932337 >> 172.21.15.67:0/1050932337 conn(0x563b76829990 msgr2=0x563b76bf21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.980+0000 7fa2a1e86740 1 -- 172.21.15.67:0/1050932337 shutdown_connections 2024-07-31T14:54:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.980+0000 7fa2a1e86740 1 -- 172.21.15.67:0/1050932337 wait complete. 2024-07-31T14:54:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.981+0000 7fa2a1e86740 1 -- start start 2024-07-31T14:54:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.981+0000 7fa2a1e86740 1 Processor -- start 2024-07-31T14:54:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.981+0000 7fa2a1e86740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b7680c980 0x563b76bfc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.981+0000 7fa2a1e86740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76814b60 0x563b76bf97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.981+0000 7fa2a1e86740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76835840 0x563b76bf9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.981+0000 7fa2a1e86740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941afb90 con 0x563b7680c980 2024-07-31T14:54:13.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.981+0000 7fa2a1e86740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941af400 con 0x563b76835840 2024-07-31T14:54:13.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.981+0000 7fa2a1e86740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941ac580 con 0x563b76814b60 2024-07-31T14:54:13.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.981+0000 7fa2a1e86740 10 client.0 _ll_register_callbacks cb 0x7fa2900aa140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:13.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.981+0000 7fa2996d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b7680c980 0x563b76bfc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.982+0000 7fa2996d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b7680c980 0x563b76bfc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50650/0 (socket says 172.21.15.67:50650) 2024-07-31T14:54:13.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.982+0000 7fa2996d1640 1 -- 172.21.15.67:0/2799494786 learned_addr learned my addr 172.21.15.67:0/2799494786 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:13.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.982+0000 7fa299ed2640 1 --2- 172.21.15.67:0/2799494786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76835840 0x563b76bf9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.982+0000 7fa298ed0640 1 --2- 172.21.15.67:0/2799494786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76814b60 0x563b76bf97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.982+0000 7fa299ed2640 1 -- 172.21.15.67:0/2799494786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76814b60 msgr2=0x563b76bf97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.982+0000 7fa299ed2640 1 --2- 172.21.15.67:0/2799494786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76814b60 0x563b76bf97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.982+0000 7fa299ed2640 1 -- 172.21.15.67:0/2799494786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b7680c980 msgr2=0x563b76bfc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.982+0000 7fa299ed2640 1 --2- 172.21.15.67:0/2799494786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b7680c980 0x563b76bfc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.982+0000 7fa299ed2640 1 -- 172.21.15.67:0/2799494786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b76bfa370 con 0x563b76835840 2024-07-31T14:54:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.982+0000 7fa2996d1640 1 --2- 172.21.15.67:0/2799494786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b7680c980 0x563b76bfc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.982+0000 7fa299ed2640 1 --2- 172.21.15.67:0/2799494786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76835840 0x563b76bf9d30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00e930 tx=0x7fa28c019cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ba941d69ba4512 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.982+0000 7fa27cff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:54:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.982+0000 7fa27cff9640 1 -- 172.21.15.67:0/2799494786 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c029070 con 0x563b76835840 2024-07-31T14:54:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.982+0000 7fa27cff9640 1 -- 172.21.15.67:0/2799494786 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fa28c0043f0 con 0x563b76835840 2024-07-31T14:54:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.983+0000 7fa2a1e86740 1 -- 172.21.15.67:0/2799494786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa28c019e10 con 0x563b76835840 2024-07-31T14:54:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.983+0000 7fa27cff9640 1 -- 172.21.15.67:0/2799494786 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa28c036050 con 0x563b76835840 2024-07-31T14:54:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.983+0000 7fa2a1e86740 10 client.26087 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.983+0000 7fa2a1e86740 1 -- 172.21.15.67:0/2799494786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563b76c61080 con 0x563b76835840 2024-07-31T14:54:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.983+0000 7fa27cff9640 1 -- 172.21.15.67:0/2799494786 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa28c019e10 con 0x563b76835840 2024-07-31T14:54:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.983+0000 7fa27cff9640 1 -- 172.21.15.67:0/2799494786 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7fa28c003f40 con 0x563b76835840 2024-07-31T14:54:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.983+0000 7fa2a1e86740 10 client.26087 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.983+0000 7fa2a1e86740 10 client.26087 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:54:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.983+0000 7fa2a1e86740 20 client.26087 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.983+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/2799494786 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563b767c9570 0x563b76c63860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.983+0000 7fa2a1e86740 4 client.26087 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.4, "await_for": 45.0}] 2024-07-31T14:54:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.983+0000 7fa2a1e86740 1 -- 172.21.15.67:0/2799494786 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.4, "await_for": 45.0}) -- 0x563b76b4e1a0 con 0x563b767c9570 2024-07-31T14:54:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.983+0000 7fa2996d1640 1 --2- 172.21.15.67:0/2799494786 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563b767c9570 0x563b76c63860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.984+0000 7fa27cff9640 10 client.26087 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:13.984+0000 7fa2996d1640 1 --2- 172.21.15.67:0/2799494786 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563b767c9570 0x563b76c63860 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e627e909f3cd391c server_cookie=c6ded4ff036b6b31 in_seq=0 out_seq=0 2024-07-31T14:54:14.018 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:54:15.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:15 smithi067 ceph-mon[32750]: pgmap v495: 129 pgs: 129 active+clean; 302 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.4 MiB/s wr, 540 op/s 2024-07-31T14:54:15.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:15 smithi073 ceph-mon[36714]: pgmap v495: 129 pgs: 129 active+clean; 302 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.4 MiB/s wr, 540 op/s 2024-07-31T14:54:15.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:15 smithi077 ceph-mon[37327]: pgmap v495: 129 pgs: 129 active+clean; 302 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.4 MiB/s wr, 540 op/s 2024-07-31T14:54:16.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.002+0000 7f07cc72f640 1 -- 172.21.15.67:0/275549433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c4102510 msgr2=0x7f07c4102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:16.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.002+0000 7f07cc72f640 1 --2- 172.21.15.67:0/275549433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c4102510 0x7f07c4102920 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009900 tx=0x7f07c002a410 comp rx=0 tx=0).stop 2024-07-31T14:54:16.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.002+0000 7f07cc72f640 1 -- 172.21.15.67:0/275549433 shutdown_connections 2024-07-31T14:54:16.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.002+0000 7f07cc72f640 1 --2- 172.21.15.67:0/275549433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c4108750 0x7f07c410ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:16.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.002+0000 7f07cc72f640 1 --2- 172.21.15.67:0/275549433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07c4107e00 0x7f07c4108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:16.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.002+0000 7f07cc72f640 1 --2- 172.21.15.67:0/275549433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c4102510 0x7f07c4102920 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:16.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.002+0000 7f07cc72f640 1 -- 172.21.15.67:0/275549433 >> 172.21.15.67:0/275549433 conn(0x7f07c40791c0 msgr2=0x7f07c40795d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:16.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.002+0000 7f07cc72f640 1 -- 172.21.15.67:0/275549433 shutdown_connections 2024-07-31T14:54:16.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.003+0000 7f07cc72f640 1 -- 172.21.15.67:0/275549433 wait complete. 2024-07-31T14:54:16.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.003+0000 7f07cc72f640 1 Processor -- start 2024-07-31T14:54:16.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.003+0000 7f07cc72f640 1 -- start start 2024-07-31T14:54:16.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07cc72f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c4102510 0x7f07c4199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:16.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07cc72f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c4107e00 0x7f07c419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:16.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07cc72f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07c4108750 0x7f07c419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:16.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07cc72f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4112420 con 0x7f07c4102510 2024-07-31T14:54:16.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07cc72f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41122a0 con 0x7f07c4108750 2024-07-31T14:54:16.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07cc72f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41125a0 con 0x7f07c4107e00 2024-07-31T14:54:16.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07ca4cb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c4102510 0x7f07c4199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:16.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07ca4cb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c4102510 0x7f07c4199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50676/0 (socket says 172.21.15.67:50676) 2024-07-31T14:54:16.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07ca4cb640 1 -- 172.21.15.67:0/3417533570 learned_addr learned my addr 172.21.15.67:0/3417533570 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:16.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07c9cca640 1 --2- 172.21.15.67:0/3417533570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c4107e00 0x7f07c419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:16.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07caccc640 1 --2- 172.21.15.67:0/3417533570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07c4108750 0x7f07c419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:16.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07ca4cb640 1 -- 172.21.15.67:0/3417533570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c4107e00 msgr2=0x7f07c419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:16.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07ca4cb640 1 --2- 172.21.15.67:0/3417533570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c4107e00 0x7f07c419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:16.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07ca4cb640 1 -- 172.21.15.67:0/3417533570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07c4108750 msgr2=0x7f07c419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:16.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07ca4cb640 1 --2- 172.21.15.67:0/3417533570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07c4108750 0x7f07c419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:16.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.004+0000 7f07ca4cb640 1 -- 172.21.15.67:0/3417533570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c419feb0 con 0x7f07c4102510 2024-07-31T14:54:16.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.005+0000 7f07ca4cb640 1 --2- 172.21.15.67:0/3417533570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c4102510 0x7f07c4199f40 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f07c0005940 tx=0x7f07c00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.005+0000 7f07bb7fe640 1 -- 172.21.15.67:0/3417533570 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c003f030 con 0x7f07c4102510 2024-07-31T14:54:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.005+0000 7f07bb7fe640 1 -- 172.21.15.67:0/3417533570 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07c00044c0 con 0x7f07c4102510 2024-07-31T14:54:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.005+0000 7f07bb7fe640 1 -- 172.21.15.67:0/3417533570 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0035070 con 0x7f07c4102510 2024-07-31T14:54:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.005+0000 7f07cc72f640 1 -- 172.21.15.67:0/3417533570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c41a0140 con 0x7f07c4102510 2024-07-31T14:54:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.005+0000 7f07cc72f640 1 -- 172.21.15.67:0/3417533570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c41a6900 con 0x7f07c4102510 2024-07-31T14:54:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.006+0000 7f07cc72f640 1 -- 172.21.15.67:0/3417533570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c4103740 con 0x7f07c4102510 2024-07-31T14:54:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.009+0000 7f07bb7fe640 1 -- 172.21.15.67:0/3417533570 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f07c0042030 con 0x7f07c4102510 2024-07-31T14:54:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.010+0000 7f07bb7fe640 1 --2- 172.21.15.67:0/3417533570 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07b00b3d40 0x7f07b00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.010+0000 7f07c9cca640 1 --2- 172.21.15.67:0/3417533570 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07b00b3d40 0x7f07b00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.010+0000 7f07bb7fe640 1 -- 172.21.15.67:0/3417533570 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f07c01048d0 con 0x7f07c4102510 2024-07-31T14:54:16.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.010+0000 7f07c9cca640 1 --2- 172.21.15.67:0/3417533570 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07b00b3d40 0x7f07b00b6250 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f07c4104320 tx=0x7f07b4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.014+0000 7f07bb7fe640 1 -- 172.21.15.67:0/3417533570 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f07c00cb280 con 0x7f07c4102510 2024-07-31T14:54:16.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.311+0000 7f07cc72f640 1 -- 172.21.15.67:0/3417533570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07c4102920 con 0x7f07c4102510 2024-07-31T14:54:16.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.313+0000 7f07bb7fe640 1 -- 172.21.15.67:0/3417533570 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11980 (secure 0 0 0) 0x7f07c00cf7d0 con 0x7f07c4102510 2024-07-31T14:54:16.315 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:54:16.315 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":39,"btime":"2024-07-31T14:54:12:039205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:54:11.524751+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T14:54:16.315 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 39 2024-07-31T14:54:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.316+0000 7f07cc72f640 1 -- 172.21.15.67:0/3417533570 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07b00b3d40 msgr2=0x7f07b00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.316+0000 7f07cc72f640 1 --2- 172.21.15.67:0/3417533570 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07b00b3d40 0x7f07b00b6250 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f07c4104320 tx=0x7f07b4005f90 comp rx=0 tx=0).stop 2024-07-31T14:54:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.317+0000 7f07cc72f640 1 -- 172.21.15.67:0/3417533570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c4102510 msgr2=0x7f07c4199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:16.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.317+0000 7f07cc72f640 1 --2- 172.21.15.67:0/3417533570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c4102510 0x7f07c4199f40 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f07c0005940 tx=0x7f07c00040c0 comp rx=0 tx=0).stop 2024-07-31T14:54:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.317+0000 7f07cc72f640 1 -- 172.21.15.67:0/3417533570 shutdown_connections 2024-07-31T14:54:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.317+0000 7f07cc72f640 1 --2- 172.21.15.67:0/3417533570 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07b00b3d40 0x7f07b00b6250 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.317+0000 7f07cc72f640 1 --2- 172.21.15.67:0/3417533570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07c4108750 0x7f07c419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.317+0000 7f07cc72f640 1 --2- 172.21.15.67:0/3417533570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c4107e00 0x7f07c419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.317+0000 7f07cc72f640 1 --2- 172.21.15.67:0/3417533570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c4102510 0x7f07c4199f40 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.317+0000 7f07cc72f640 1 -- 172.21.15.67:0/3417533570 >> 172.21.15.67:0/3417533570 conn(0x7f07c40791c0 msgr2=0x7f07c4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.317+0000 7f07cc72f640 1 -- 172.21.15.67:0/3417533570 shutdown_connections 2024-07-31T14:54:16.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:16.317+0000 7f07cc72f640 1 -- 172.21.15.67:0/3417533570 wait complete. 2024-07-31T14:54:17.056 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T14:54:17.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:17 smithi067 ceph-mon[32750]: pgmap v496: 129 pgs: 129 active+clean; 305 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.2 MiB/s wr, 371 op/s 2024-07-31T14:54:17.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:17 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3417533570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:54:17.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:17 smithi073 ceph-mon[36714]: pgmap v496: 129 pgs: 129 active+clean; 305 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.2 MiB/s wr, 371 op/s 2024-07-31T14:54:17.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:17 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3417533570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:54:17.470 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:54:17.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:17 smithi077 ceph-mon[37327]: pgmap v496: 129 pgs: 129 active+clean; 305 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.2 MiB/s wr, 371 op/s 2024-07-31T14:54:17.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:17 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3417533570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:54:19.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:19 smithi067 ceph-mon[32750]: pgmap v497: 129 pgs: 129 active+clean; 310 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-07-31T14:54:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.365+0000 7ff076021640 1 -- 172.21.15.67:0/821004073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff07010d840 msgr2=0x7ff07010fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.365+0000 7ff076021640 1 --2- 172.21.15.67:0/821004073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff07010d840 0x7ff07010fd30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff068009900 tx=0x7ff06802a410 comp rx=0 tx=0).stop 2024-07-31T14:54:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.365+0000 7ff076021640 1 -- 172.21.15.67:0/821004073 shutdown_connections 2024-07-31T14:54:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.365+0000 7ff076021640 1 --2- 172.21.15.67:0/821004073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff07010d840 0x7ff07010fd30 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.365+0000 7ff076021640 1 --2- 172.21.15.67:0/821004073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff070104d30 0x7ff07010d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.365+0000 7ff076021640 1 --2- 172.21.15.67:0/821004073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0701043e0 0x7ff0701047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.365+0000 7ff076021640 1 -- 172.21.15.67:0/821004073 >> 172.21.15.67:0/821004073 conn(0x7ff070079160 msgr2=0x7ff070079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.365+0000 7ff076021640 1 -- 172.21.15.67:0/821004073 shutdown_connections 2024-07-31T14:54:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.365+0000 7ff076021640 1 -- 172.21.15.67:0/821004073 wait complete. 2024-07-31T14:54:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.366+0000 7ff076021640 1 Processor -- start 2024-07-31T14:54:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.366+0000 7ff076021640 1 -- start start 2024-07-31T14:54:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.366+0000 7ff076021640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0701043e0 0x7ff070199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.366+0000 7ff076021640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff070104d30 0x7ff07019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.366+0000 7ff076021640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff07010d840 0x7ff07019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.366+0000 7ff076021640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701134a0 con 0x7ff07010d840 2024-07-31T14:54:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.366+0000 7ff076021640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff070113320 con 0x7ff070104d30 2024-07-31T14:54:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.366+0000 7ff076021640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff070113620 con 0x7ff0701043e0 2024-07-31T14:54:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.366+0000 7ff06f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0701043e0 0x7ff070199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.366+0000 7ff06f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0701043e0 0x7ff070199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49322/0 (socket says 172.21.15.67:49322) 2024-07-31T14:54:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.366+0000 7ff06f7fe640 1 -- 172.21.15.67:0/697015150 learned_addr learned my addr 172.21.15.67:0/697015150 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.366+0000 7ff067fff640 1 --2- 172.21.15.67:0/697015150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff070104d30 0x7ff07019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.366+0000 7ff06ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff07010d840 0x7ff07019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.367+0000 7ff06f7fe640 1 -- 172.21.15.67:0/697015150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff070104d30 msgr2=0x7ff07019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.367+0000 7ff06f7fe640 1 --2- 172.21.15.67:0/697015150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff070104d30 0x7ff07019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.367+0000 7ff06f7fe640 1 -- 172.21.15.67:0/697015150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff07010d840 msgr2=0x7ff07019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.367+0000 7ff06f7fe640 1 --2- 172.21.15.67:0/697015150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff07010d840 0x7ff07019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.367+0000 7ff06f7fe640 1 -- 172.21.15.67:0/697015150 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07019fe40 con 0x7ff0701043e0 2024-07-31T14:54:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.367+0000 7ff06f7fe640 1 --2- 172.21.15.67:0/697015150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0701043e0 0x7ff070199ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff06000ebc0 tx=0x7ff06000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.367+0000 7ff06d7fa640 1 -- 172.21.15.67:0/697015150 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060003b70 con 0x7ff0701043e0 2024-07-31T14:54:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.367+0000 7ff076021640 1 -- 172.21.15.67:0/697015150 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0701a0130 con 0x7ff0701043e0 2024-07-31T14:54:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.367+0000 7ff06d7fa640 1 -- 172.21.15.67:0/697015150 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff060003d10 con 0x7ff0701043e0 2024-07-31T14:54:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.367+0000 7ff06d7fa640 1 -- 172.21.15.67:0/697015150 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060019800 con 0x7ff0701043e0 2024-07-31T14:54:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.367+0000 7ff076021640 1 -- 172.21.15.67:0/697015150 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0701a6890 con 0x7ff0701043e0 2024-07-31T14:54:19.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.370+0000 7ff06d7fa640 1 -- 172.21.15.67:0/697015150 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff0600199a0 con 0x7ff0701043e0 2024-07-31T14:54:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.370+0000 7ff07c572740 1 Processor -- start 2024-07-31T14:54:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff07c572740 1 -- start start 2024-07-31T14:54:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff07c572740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed599070 0x5645ed93e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff07c572740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed5701b0 0x5645ed93eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff07c572740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed578390 0x5645ed93eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff07c572740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5645ed8b4bb0 con 0x5645ed599070 2024-07-31T14:54:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff07c572740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5645ed8a40b0 con 0x5645ed5701b0 2024-07-31T14:54:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff07c572740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5645ed8a80f0 con 0x5645ed578390 2024-07-31T14:54:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff06ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed599070 0x5645ed93e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff06f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed5701b0 0x5645ed93eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff06ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed599070 0x5645ed93e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50712/0 (socket says 172.21.15.67:50712) 2024-07-31T14:54:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff06ffff640 1 -- 172.21.15.67:0/3190307836 learned_addr learned my addr 172.21.15.67:0/3190307836 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff067fff640 1 --2- 172.21.15.67:0/3190307836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed578390 0x5645ed93eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff06ffff640 1 -- 172.21.15.67:0/3190307836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed578390 msgr2=0x5645ed93eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff06ffff640 1 --2- 172.21.15.67:0/3190307836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed578390 0x5645ed93eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff06ffff640 1 -- 172.21.15.67:0/3190307836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed5701b0 msgr2=0x5645ed93eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff06d7fa640 1 --2- 172.21.15.67:0/697015150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff0540b3d40 0x7ff0540b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff06ffff640 1 --2- 172.21.15.67:0/3190307836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed5701b0 0x5645ed93eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff06ffff640 1 -- 172.21.15.67:0/3190307836 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ed84ab90 con 0x5645ed599070 2024-07-31T14:54:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.371+0000 7ff06f7fe640 1 --2- 172.21.15.67:0/3190307836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed5701b0 0x5645ed93eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:19.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.372+0000 7ff06ffff640 1 --2- 172.21.15.67:0/697015150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff0540b3d40 0x7ff0540b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.372+0000 7ff06d7fa640 1 -- 172.21.15.67:0/697015150 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff0600e4c40 con 0x7ff0701043e0 2024-07-31T14:54:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.372+0000 7ff06ffff640 1 --2- 172.21.15.67:0/3190307836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed599070 0x5645ed93e570 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff068009900 tx=0x7ff068032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564d6caf9100985a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.372+0000 7ff065ffb640 1 -- 172.21.15.67:0/3190307836 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06804c070 con 0x5645ed599070 2024-07-31T14:54:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.372+0000 7ff065ffb640 1 -- 172.21.15.67:0/3190307836 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff068031880 con 0x5645ed599070 2024-07-31T14:54:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.372+0000 7ff065ffb640 1 -- 172.21.15.67:0/3190307836 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068040070 con 0x5645ed599070 2024-07-31T14:54:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.373+0000 7ff07c572740 1 -- 172.21.15.67:0/3190307836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed599070 msgr2=0x5645ed93e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.373+0000 7ff07c572740 1 --2- 172.21.15.67:0/3190307836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed599070 0x5645ed93e570 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff068009900 tx=0x7ff068032b20 comp rx=0 tx=0).stop 2024-07-31T14:54:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.373+0000 7ff06ffff640 1 --2- 172.21.15.67:0/697015150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff0540b3d40 0x7ff0540b6250 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff068032fd0 tx=0x7ff068004bb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.373+0000 7ff07c572740 1 -- 172.21.15.67:0/3190307836 shutdown_connections 2024-07-31T14:54:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.373+0000 7ff07c572740 1 --2- 172.21.15.67:0/3190307836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed578390 0x5645ed93eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.373+0000 7ff07c572740 1 --2- 172.21.15.67:0/3190307836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed5701b0 0x5645ed93eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.373+0000 7ff07c572740 1 --2- 172.21.15.67:0/3190307836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed599070 0x5645ed93e570 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.373+0000 7ff07c572740 1 -- 172.21.15.67:0/3190307836 >> 172.21.15.67:0/3190307836 conn(0x5645ed58d460 msgr2=0x5645ed93c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.373+0000 7ff07c572740 1 -- 172.21.15.67:0/3190307836 shutdown_connections 2024-07-31T14:54:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.373+0000 7ff07c572740 1 -- 172.21.15.67:0/3190307836 wait complete. 2024-07-31T14:54:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.374+0000 7ff07c572740 1 -- start start 2024-07-31T14:54:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.374+0000 7ff07c572740 1 Processor -- start 2024-07-31T14:54:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.374+0000 7ff07c572740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed94e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.374+0000 7ff07c572740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed578390 0x5645ed94eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.374+0000 7ff07c572740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed599070 0x5645ed95a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.374+0000 7ff07c572740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5645ed707700 con 0x5645ed599070 2024-07-31T14:54:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.374+0000 7ff07c572740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5645ed8469e0 con 0x5645ed578390 2024-07-31T14:54:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.374+0000 7ff07c572740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5645ed951050 con 0x5645ed5701b0 2024-07-31T14:54:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.374+0000 7ff07c572740 10 client.0 _ll_register_callbacks cb 0x5645ed8614f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.374+0000 7ff067fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed578390 0x5645ed94eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.374+0000 7ff06ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed599070 0x5645ed95a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.374+0000 7ff06ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed599070 0x5645ed95a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50716/0 (socket says 172.21.15.67:50716) 2024-07-31T14:54:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.374+0000 7ff06ffff640 1 -- 172.21.15.67:0/320075743 learned_addr learned my addr 172.21.15.67:0/320075743 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.374+0000 7ff067fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed578390 0x5645ed94eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:42592/0 (socket says 172.21.15.67:42592) 2024-07-31T14:54:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.375+0000 7ff06f7fe640 1 --2- 172.21.15.67:0/320075743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed94e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.375+0000 7ff067fff640 1 -- 172.21.15.67:0/320075743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 msgr2=0x5645ed94e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.375+0000 7ff067fff640 1 --2- 172.21.15.67:0/320075743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed94e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.375+0000 7ff067fff640 1 -- 172.21.15.67:0/320075743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed599070 msgr2=0x5645ed95a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.375+0000 7ff067fff640 1 --2- 172.21.15.67:0/320075743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed599070 0x5645ed95a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.375+0000 7ff067fff640 1 -- 172.21.15.67:0/320075743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ed604650 con 0x5645ed578390 2024-07-31T14:54:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.375+0000 7ff06ffff640 1 --2- 172.21.15.67:0/320075743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed599070 0x5645ed95a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.375+0000 7ff067fff640 1 --2- 172.21.15.67:0/320075743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed578390 0x5645ed94eaf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff07006a4d0 tx=0x7ff05c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa10afea32da5346 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.375+0000 7ff052ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:54:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.375+0000 7ff052ffd640 1 -- 172.21.15.67:0/320075743 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c00f890 con 0x5645ed578390 2024-07-31T14:54:19.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.375+0000 7ff07c572740 1 -- 172.21.15.67:0/320075743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff068031880 con 0x5645ed578390 2024-07-31T14:54:19.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.375+0000 7ff052ffd640 1 -- 172.21.15.67:0/320075743 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff05c00fa30 con 0x5645ed578390 2024-07-31T14:54:19.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.376+0000 7ff052ffd640 1 -- 172.21.15.67:0/320075743 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff05c00fc30 con 0x5645ed578390 2024-07-31T14:54:19.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.376+0000 7ff07c572740 10 client.26105 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:19.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.376+0000 7ff07c572740 1 -- 172.21.15.67:0/320075743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5645ed95d1b0 con 0x5645ed578390 2024-07-31T14:54:19.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.376+0000 7ff052ffd640 1 -- 172.21.15.67:0/320075743 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff068031880 con 0x5645ed578390 2024-07-31T14:54:19.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.376+0000 7ff052ffd640 1 -- 172.21.15.67:0/320075743 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7ff05c01c420 con 0x5645ed578390 2024-07-31T14:54:19.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.376+0000 7ff07c572740 10 client.26105 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:19.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.376+0000 7ff07c572740 10 client.26105 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:54:19.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.376+0000 7ff07c572740 20 client.26105 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:19.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.376+0000 7ff07c572740 1 --2- 172.21.15.67:0/320075743 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5645ed5579f0 0x5645ed9606c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.376+0000 7ff07c572740 4 client.26105 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:54:19.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.376+0000 7ff07c572740 1 -- 172.21.15.67:0/320075743 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645ed8b6b00 con 0x5645ed5579f0 2024-07-31T14:54:19.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.376+0000 7ff06f7fe640 1 --2- 172.21.15.67:0/320075743 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5645ed5579f0 0x5645ed9606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.377+0000 7ff052ffd640 10 client.26105 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:54:19.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.377+0000 7ff06f7fe640 1 --2- 172.21.15.67:0/320075743 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5645ed5579f0 0x5645ed9606c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=992f30c46d9b0ef9 server_cookie=8e9676fcb663f84a in_seq=0 out_seq=0 2024-07-31T14:54:19.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff052ffd640 1 -- 172.21.15.67:0/320075743 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645ed8b6b00 con 0x5645ed5579f0 2024-07-31T14:54:19.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff052ffd640 10 client.26105 handle_command_reply: tid=0 2024-07-31T14:54:19.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff07c572740 1 client.26105 shutdown 2024-07-31T14:54:19.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff07c572740 1 -- 172.21.15.67:0/320075743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed578390 msgr2=0x5645ed94eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:19.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff07c572740 1 --2- 172.21.15.67:0/320075743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed578390 0x5645ed94eaf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff07006a4d0 tx=0x7ff05c012b60 comp rx=0 tx=0).stop 2024-07-31T14:54:19.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff07c572740 1 -- 172.21.15.67:0/320075743 shutdown_connections 2024-07-31T14:54:19.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff07c572740 1 --2- 172.21.15.67:0/320075743 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5645ed5579f0 0x5645ed9606c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff07c572740 1 --2- 172.21.15.67:0/320075743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed599070 0x5645ed95a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff07c572740 1 --2- 172.21.15.67:0/320075743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed578390 0x5645ed94eaf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff07c572740 1 --2- 172.21.15.67:0/320075743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed94e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff07c572740 1 -- 172.21.15.67:0/320075743 >> 172.21.15.67:0/320075743 conn(0x5645ed58d460 msgr2=0x5645ed93ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:19.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff052ffd640 0 client.26105 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:54:19.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff07c572740 1 -- 172.21.15.67:0/320075743 shutdown_connections 2024-07-31T14:54:19.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff07c572740 1 -- 172.21.15.67:0/320075743 wait complete. 2024-07-31T14:54:19.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.378+0000 7ff07c572740 20 client.26105 trim_cache size 0 max 16384 2024-07-31T14:54:19.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:19 smithi073 ceph-mon[36714]: pgmap v497: 129 pgs: 129 active+clean; 310 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-07-31T14:54:19.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.393+0000 7ff07c572740 1 Processor -- start 2024-07-31T14:54:19.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.393+0000 7ff07c572740 1 -- start start 2024-07-31T14:54:19.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.393+0000 7ff07c572740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed943cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.393+0000 7ff07c572740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed578390 0x5645ed95a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.393+0000 7ff07c572740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed599070 0x5645ed965e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.393+0000 7ff07c572740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701ae540 con 0x5645ed578390 2024-07-31T14:54:19.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.393+0000 7ff07c572740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701b0200 con 0x5645ed599070 2024-07-31T14:54:19.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.393+0000 7ff07c572740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701aec10 con 0x5645ed5701b0 2024-07-31T14:54:19.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.393+0000 7ff067fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed578390 0x5645ed95a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.393+0000 7ff067fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed578390 0x5645ed95a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50726/0 (socket says 172.21.15.67:50726) 2024-07-31T14:54:19.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.393+0000 7ff067fff640 1 -- 172.21.15.67:0/2348188465 learned_addr learned my addr 172.21.15.67:0/2348188465 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:19.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.393+0000 7ff06f7fe640 1 --2- 172.21.15.67:0/2348188465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed943cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.393+0000 7ff06ffff640 1 --2- 172.21.15.67:0/2348188465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed599070 0x5645ed965e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.394+0000 7ff067fff640 1 -- 172.21.15.67:0/2348188465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 msgr2=0x5645ed943cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:19.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.394+0000 7ff067fff640 1 --2- 172.21.15.67:0/2348188465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed943cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.394+0000 7ff067fff640 1 -- 172.21.15.67:0/2348188465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed599070 msgr2=0x5645ed965e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:19.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.394+0000 7ff067fff640 1 --2- 172.21.15.67:0/2348188465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed599070 0x5645ed965e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.394+0000 7ff067fff640 1 -- 172.21.15.67:0/2348188465 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ed94f650 con 0x5645ed578390 2024-07-31T14:54:19.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.394+0000 7ff06f7fe640 1 --2- 172.21.15.67:0/2348188465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed943cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:19.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.394+0000 7ff067fff640 1 --2- 172.21.15.67:0/2348188465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed578390 0x5645ed95a1a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff05c01ca80 tx=0x7ff05c0096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1faadfe197f5bf77 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:19.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.394+0000 7ff0657fa640 1 -- 172.21.15.67:0/2348188465 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c00fe50 con 0x5645ed578390 2024-07-31T14:54:19.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.394+0000 7ff0657fa640 1 -- 172.21.15.67:0/2348188465 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff05c01c420 con 0x5645ed578390 2024-07-31T14:54:19.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.394+0000 7ff0657fa640 1 -- 172.21.15.67:0/2348188465 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0053f0 con 0x5645ed578390 2024-07-31T14:54:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.395+0000 7ff07c572740 1 -- 172.21.15.67:0/2348188465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed578390 msgr2=0x5645ed95a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.395+0000 7ff07c572740 1 --2- 172.21.15.67:0/2348188465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed578390 0x5645ed95a1a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff05c01ca80 tx=0x7ff05c0096f0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.395+0000 7ff07c572740 1 -- 172.21.15.67:0/2348188465 shutdown_connections 2024-07-31T14:54:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.395+0000 7ff07c572740 1 --2- 172.21.15.67:0/2348188465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed599070 0x5645ed965e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.395+0000 7ff07c572740 1 --2- 172.21.15.67:0/2348188465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed578390 0x5645ed95a1a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.395+0000 7ff07c572740 1 --2- 172.21.15.67:0/2348188465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed943cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.395+0000 7ff07c572740 1 -- 172.21.15.67:0/2348188465 >> 172.21.15.67:0/2348188465 conn(0x5645ed58d460 msgr2=0x5645ed9435e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.395+0000 7ff07c572740 1 -- 172.21.15.67:0/2348188465 shutdown_connections 2024-07-31T14:54:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.395+0000 7ff07c572740 1 -- 172.21.15.67:0/2348188465 wait complete. 2024-07-31T14:54:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.396+0000 7ff07c572740 1 -- start start 2024-07-31T14:54:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.396+0000 7ff07c572740 1 Processor -- start 2024-07-31T14:54:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.396+0000 7ff07c572740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed956840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.396+0000 7ff07c572740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed578390 0x5645ed951d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.396+0000 7ff07c572740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed599070 0x5645ed9522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.396+0000 7ff07c572740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701afb30 con 0x5645ed578390 2024-07-31T14:54:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.396+0000 7ff07c572740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701af3a0 con 0x5645ed599070 2024-07-31T14:54:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.396+0000 7ff07c572740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701ac520 con 0x5645ed5701b0 2024-07-31T14:54:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.396+0000 7ff07c572740 10 client.0 _ll_register_callbacks cb 0x7ff05c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff06f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed956840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff067fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed578390 0x5645ed951d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff06f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed956840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49356/0 (socket says 172.21.15.67:49356) 2024-07-31T14:54:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff06f7fe640 1 -- 172.21.15.67:0/867947478 learned_addr learned my addr 172.21.15.67:0/867947478 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff06ffff640 1 --2- 172.21.15.67:0/867947478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed599070 0x5645ed9522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff067fff640 1 -- 172.21.15.67:0/867947478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 msgr2=0x5645ed956840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff067fff640 1 --2- 172.21.15.67:0/867947478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed956840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff067fff640 1 -- 172.21.15.67:0/867947478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed599070 msgr2=0x5645ed9522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff067fff640 1 --2- 172.21.15.67:0/867947478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed599070 0x5645ed9522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff067fff640 1 -- 172.21.15.67:0/867947478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ed9528c0 con 0x5645ed578390 2024-07-31T14:54:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff06f7fe640 1 --2- 172.21.15.67:0/867947478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed956840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff06ffff640 1 --2- 172.21.15.67:0/867947478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed599070 0x5645ed9522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff067fff640 1 --2- 172.21.15.67:0/867947478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed578390 0x5645ed951d90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009e00 tx=0x7ff05c00fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c63dd0e2efd6500 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff052ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff052ffd640 1 -- 172.21.15.67:0/867947478 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c038060 con 0x5645ed578390 2024-07-31T14:54:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff052ffd640 1 -- 172.21.15.67:0/867947478 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff05c01cab0 con 0x5645ed578390 2024-07-31T14:54:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.397+0000 7ff07c572740 1 -- 172.21.15.67:0/867947478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff05c01c420 con 0x5645ed578390 2024-07-31T14:54:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.398+0000 7ff052ffd640 1 -- 172.21.15.67:0/867947478 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff05c03f050 con 0x5645ed578390 2024-07-31T14:54:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.398+0000 7ff07c572740 10 client.16500 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.398+0000 7ff07c572740 1 -- 172.21.15.67:0/867947478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5645ed9c8bf0 con 0x5645ed578390 2024-07-31T14:54:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.398+0000 7ff052ffd640 1 -- 172.21.15.67:0/867947478 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff05c01c420 con 0x5645ed578390 2024-07-31T14:54:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.398+0000 7ff052ffd640 1 -- 172.21.15.67:0/867947478 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7ff05c032070 con 0x5645ed578390 2024-07-31T14:54:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.398+0000 7ff07c572740 10 client.16500 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.398+0000 7ff07c572740 10 client.16500 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:54:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.398+0000 7ff07c572740 20 client.16500 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.398+0000 7ff07c572740 1 --2- 172.21.15.67:0/867947478 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5645ed52cda0 0x5645ed9cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.398+0000 7ff07c572740 4 client.16500 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:54:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.398+0000 7ff07c572740 1 -- 172.21.15.67:0/867947478 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645ed8b25a0 con 0x5645ed52cda0 2024-07-31T14:54:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.398+0000 7ff06f7fe640 1 --2- 172.21.15.67:0/867947478 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5645ed52cda0 0x5645ed9cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.399+0000 7ff052ffd640 10 client.16500 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:54:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.399+0000 7ff06f7fe640 1 --2- 172.21.15.67:0/867947478 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5645ed52cda0 0x5645ed9cb2b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a735394011b10534 server_cookie=b23c53e5646174db in_seq=0 out_seq=0 2024-07-31T14:54:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.399+0000 7ff052ffd640 1 -- 172.21.15.67:0/867947478 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645ed8b25a0 con 0x5645ed52cda0 2024-07-31T14:54:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.399+0000 7ff052ffd640 10 client.16500 handle_command_reply: tid=0 2024-07-31T14:54:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.399+0000 7ff07c572740 1 client.16500 shutdown 2024-07-31T14:54:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.399+0000 7ff07c572740 1 -- 172.21.15.67:0/867947478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed578390 msgr2=0x5645ed951d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.399+0000 7ff07c572740 1 --2- 172.21.15.67:0/867947478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed578390 0x5645ed951d90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009e00 tx=0x7ff05c00fe50 comp rx=0 tx=0).stop 2024-07-31T14:54:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.399+0000 7ff07c572740 1 -- 172.21.15.67:0/867947478 shutdown_connections 2024-07-31T14:54:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.399+0000 7ff07c572740 1 --2- 172.21.15.67:0/867947478 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5645ed52cda0 0x5645ed9cb2b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.399+0000 7ff07c572740 1 --2- 172.21.15.67:0/867947478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ed599070 0x5645ed9522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.399+0000 7ff07c572740 1 --2- 172.21.15.67:0/867947478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645ed578390 0x5645ed951d90 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.399+0000 7ff07c572740 1 --2- 172.21.15.67:0/867947478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645ed5701b0 0x5645ed956840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.399+0000 7ff07c572740 1 -- 172.21.15.67:0/867947478 >> 172.21.15.67:0/867947478 conn(0x5645ed58d460 msgr2=0x5645ed93f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:19.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.400+0000 7ff052ffd640 0 client.16500 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:54:19.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.400+0000 7ff07c572740 1 -- 172.21.15.67:0/867947478 shutdown_connections 2024-07-31T14:54:19.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.400+0000 7ff07c572740 1 -- 172.21.15.67:0/867947478 wait complete. 2024-07-31T14:54:19.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.400+0000 7ff07c572740 20 client.16500 trim_cache size 0 max 16384 2024-07-31T14:54:19.407 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:54:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.401+0000 7ff076021640 1 -- 172.21.15.67:0/697015150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff0540b3d40 msgr2=0x7ff0540b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.401+0000 7ff076021640 1 --2- 172.21.15.67:0/697015150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff0540b3d40 0x7ff0540b6250 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff068032fd0 tx=0x7ff068004bb0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.401+0000 7ff076021640 1 -- 172.21.15.67:0/697015150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0701043e0 msgr2=0x7ff070199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.401+0000 7ff076021640 1 --2- 172.21.15.67:0/697015150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0701043e0 0x7ff070199ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff06000ebc0 tx=0x7ff06000c6a0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.401+0000 7ff076021640 1 -- 172.21.15.67:0/697015150 shutdown_connections 2024-07-31T14:54:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.401+0000 7ff076021640 1 --2- 172.21.15.67:0/697015150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff0540b3d40 0x7ff0540b6250 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.401+0000 7ff076021640 1 --2- 172.21.15.67:0/697015150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff07010d840 0x7ff07019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.401+0000 7ff076021640 1 --2- 172.21.15.67:0/697015150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff070104d30 0x7ff07019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.401+0000 7ff076021640 1 --2- 172.21.15.67:0/697015150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0701043e0 0x7ff070199ed0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:19.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.401+0000 7ff076021640 1 -- 172.21.15.67:0/697015150 >> 172.21.15.67:0/697015150 conn(0x7ff070079160 msgr2=0x7ff0700fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:19.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.401+0000 7ff076021640 1 -- 172.21.15.67:0/697015150 shutdown_connections 2024-07-31T14:54:19.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:19.401+0000 7ff076021640 1 -- 172.21.15.67:0/697015150 wait complete. 2024-07-31T14:54:19.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:19 smithi077 ceph-mon[37327]: pgmap v497: 129 pgs: 129 active+clean; 310 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-07-31T14:54:20.246 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T14:54:20.700 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:54:21.155 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:21 smithi067 ceph-mon[32750]: pgmap v498: 129 pgs: 129 active+clean; 318 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2024-07-31T14:54:21.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:21 smithi073 ceph-mon[36714]: pgmap v498: 129 pgs: 129 active+clean; 318 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2024-07-31T14:54:21.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:21 smithi077 ceph-mon[37327]: pgmap v498: 129 pgs: 129 active+clean; 318 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2024-07-31T14:54:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.548+0000 7f8a5605f640 1 -- 172.21.15.67:0/2450134769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a50108ed0 msgr2=0x7f8a501092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.548+0000 7f8a5605f640 1 --2- 172.21.15.67:0/2450134769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a50108ed0 0x7f8a501092e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009900 tx=0x7f8a4402a410 comp rx=0 tx=0).stop 2024-07-31T14:54:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.548+0000 7f8a5605f640 1 -- 172.21.15.67:0/2450134769 shutdown_connections 2024-07-31T14:54:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.548+0000 7f8a5605f640 1 --2- 172.21.15.67:0/2450134769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a50109820 0x7f8a50110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.548+0000 7f8a5605f640 1 --2- 172.21.15.67:0/2450134769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a50108ed0 0x7f8a501092e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.548+0000 7f8a5605f640 1 --2- 172.21.15.67:0/2450134769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a50103590 0x7f8a501039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.548+0000 7f8a5605f640 1 -- 172.21.15.67:0/2450134769 >> 172.21.15.67:0/2450134769 conn(0x7f8a5007b280 msgr2=0x7f8a5007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:22.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.548+0000 7f8a5605f640 1 -- 172.21.15.67:0/2450134769 shutdown_connections 2024-07-31T14:54:22.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.548+0000 7f8a5605f640 1 -- 172.21.15.67:0/2450134769 wait complete. 2024-07-31T14:54:22.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.548+0000 7f8a5605f640 1 Processor -- start 2024-07-31T14:54:22.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a5605f640 1 -- start start 2024-07-31T14:54:22.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a5605f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a50103590 0x7f8a50075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a5605f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a50108ed0 0x7f8a50075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a5605f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a50109820 0x7f8a50075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a5605f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501134b0 con 0x7f8a50103590 2024-07-31T14:54:22.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a5605f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50113330 con 0x7f8a50108ed0 2024-07-31T14:54:22.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a5605f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50113630 con 0x7f8a50109820 2024-07-31T14:54:22.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a4f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a50103590 0x7f8a50075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a4f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a50103590 0x7f8a50075140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52886/0 (socket says 172.21.15.67:52886) 2024-07-31T14:54:22.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/2137702049 learned_addr learned my addr 172.21.15.67:0/2137702049 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:22.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a4ffff640 1 --2- 172.21.15.67:0/2137702049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a50109820 0x7f8a50075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a4effd640 1 --2- 172.21.15.67:0/2137702049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a50108ed0 0x7f8a50075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a4ffff640 1 -- 172.21.15.67:0/2137702049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a50108ed0 msgr2=0x7f8a50075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:22.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a4ffff640 1 --2- 172.21.15.67:0/2137702049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a50108ed0 0x7f8a50075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a4ffff640 1 -- 172.21.15.67:0/2137702049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a50103590 msgr2=0x7f8a50075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:22.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a4ffff640 1 --2- 172.21.15.67:0/2137702049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a50103590 0x7f8a50075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.549+0000 7f8a4ffff640 1 -- 172.21.15.67:0/2137702049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a501a8200 con 0x7f8a50109820 2024-07-31T14:54:22.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.550+0000 7f8a4ffff640 1 --2- 172.21.15.67:0/2137702049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a50109820 0x7f8a50075bc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009fd0 tx=0x7f8a4800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:22.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.550+0000 7f8a4cff9640 1 -- 172.21.15.67:0/2137702049 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800ce20 con 0x7f8a50109820 2024-07-31T14:54:22.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.550+0000 7f8a5605f640 1 -- 172.21.15.67:0/2137702049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a501a84f0 con 0x7f8a50109820 2024-07-31T14:54:22.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.550+0000 7f8a4cff9640 1 -- 172.21.15.67:0/2137702049 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a48005020 con 0x7f8a50109820 2024-07-31T14:54:22.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.550+0000 7f8a4cff9640 1 -- 172.21.15.67:0/2137702049 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48010690 con 0x7f8a50109820 2024-07-31T14:54:22.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.550+0000 7f8a5605f640 1 -- 172.21.15.67:0/2137702049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a501a8a50 con 0x7f8a50109820 2024-07-31T14:54:22.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.552+0000 7f8a5c5b0740 1 Processor -- start 2024-07-31T14:54:22.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.552+0000 7f8a4cff9640 1 -- 172.21.15.67:0/2137702049 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8a480054d0 con 0x7f8a50109820 2024-07-31T14:54:22.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a5c5b0740 1 -- start start 2024-07-31T14:54:22.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a5c5b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e3838070 0x55e0e3bdd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a5c5b0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e380f1b0 0x55e0e3bddbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a5c5b0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e3817390 0x55e0e3bde110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a5c5b0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e0e3b53bb0 con 0x55e0e3817390 2024-07-31T14:54:22.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a5c5b0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e0e3b430b0 con 0x55e0e380f1b0 2024-07-31T14:54:22.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a5c5b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e0e3b470f0 con 0x55e0e3838070 2024-07-31T14:54:22.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a4effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e3817390 0x55e0e3bde110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a4effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e3838070 0x55e0e3bdd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a4effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e3838070 0x55e0e3bdd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:39756/0 (socket says 172.21.15.67:39756) 2024-07-31T14:54:22.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a4f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e380f1b0 0x55e0e3bddbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a4effd640 1 -- 172.21.15.67:0/169466823 learned_addr learned my addr 172.21.15.67:0/169466823 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:22.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/169466823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e3838070 msgr2=0x55e0e3bdd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:22.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a4f7fe640 1 --2- 172.21.15.67:0/169466823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e3838070 0x55e0e3bdd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/169466823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e3817390 msgr2=0x55e0e3bde110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:22.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a4f7fe640 1 --2- 172.21.15.67:0/169466823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e3817390 0x55e0e3bde110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a4cff9640 1 --2- 172.21.15.67:0/2137702049 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a380b3d40 0x7f8a380b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.553+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/169466823 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e3ae9b90 con 0x55e0e380f1b0 2024-07-31T14:54:22.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.554+0000 7f8a4effd640 1 --2- 172.21.15.67:0/2137702049 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a380b3d40 0x7f8a380b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.554+0000 7f8a4cff9640 1 -- 172.21.15.67:0/2137702049 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f8a480e4940 con 0x7f8a50109820 2024-07-31T14:54:22.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.554+0000 7f8a4f7fe640 1 --2- 172.21.15.67:0/169466823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e380f1b0 0x55e0e3bddbd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009c80 tx=0x7f8a40011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbf655d5632733b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:22.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.554+0000 7f8a35ffb640 1 -- 172.21.15.67:0/169466823 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4001e070 con 0x55e0e380f1b0 2024-07-31T14:54:22.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.554+0000 7f8a4effd640 1 --2- 172.21.15.67:0/2137702049 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a380b3d40 0x7f8a380b6250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8a440096c0 tx=0x7f8a44000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:22.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.554+0000 7f8a35ffb640 1 -- 172.21.15.67:0/169466823 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a40009e30 con 0x55e0e380f1b0 2024-07-31T14:54:22.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.554+0000 7f8a35ffb640 1 -- 172.21.15.67:0/169466823 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40019410 con 0x55e0e380f1b0 2024-07-31T14:54:22.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.555+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/169466823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e380f1b0 msgr2=0x55e0e3bddbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:22.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.555+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/169466823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e380f1b0 0x55e0e3bddbd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009c80 tx=0x7f8a40011810 comp rx=0 tx=0).stop 2024-07-31T14:54:22.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.555+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/169466823 shutdown_connections 2024-07-31T14:54:22.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.555+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/169466823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e3817390 0x55e0e3bde110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.555+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/169466823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e380f1b0 0x55e0e3bddbd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.555+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/169466823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e3838070 0x55e0e3bdd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.555+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/169466823 >> 172.21.15.67:0/169466823 conn(0x55e0e382c460 msgr2=0x55e0e3bdb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:22.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.555+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/169466823 shutdown_connections 2024-07-31T14:54:22.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.555+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/169466823 wait complete. 2024-07-31T14:54:22.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.555+0000 7f8a5c5b0740 1 -- start start 2024-07-31T14:54:22.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a5c5b0740 1 Processor -- start 2024-07-31T14:54:22.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a5c5b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e380f1b0 0x55e0e3be9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a5c5b0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e3817390 0x55e0e3be9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a5c5b0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3838070 0x55e0e3bf5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a5c5b0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e0e39a6700 con 0x55e0e3817390 2024-07-31T14:54:22.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a5c5b0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e0e3ae59e0 con 0x55e0e3838070 2024-07-31T14:54:22.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a5c5b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e0e3bebe70 con 0x55e0e380f1b0 2024-07-31T14:54:22.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a4f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e380f1b0 0x55e0e3be9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a4effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e3817390 0x55e0e3be9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a5c5b0740 10 client.0 _ll_register_callbacks cb 0x55e0e3b004f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a4f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e380f1b0 0x55e0e3be9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:39766/0 (socket says 172.21.15.67:39766) 2024-07-31T14:54:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/3374586899 learned_addr learned my addr 172.21.15.67:0/3374586899 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a4ffff640 1 --2- 172.21.15.67:0/3374586899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3838070 0x55e0e3bf5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/3374586899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3838070 msgr2=0x55e0e3bf5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.556+0000 7f8a4f7fe640 1 --2- 172.21.15.67:0/3374586899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3838070 0x55e0e3bf5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.557+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/3374586899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e3817390 msgr2=0x55e0e3be9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.557+0000 7f8a4f7fe640 1 --2- 172.21.15.67:0/3374586899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e3817390 0x55e0e3be9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.557+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/3374586899 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e38a3650 con 0x55e0e380f1b0 2024-07-31T14:54:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.557+0000 7f8a4effd640 1 --2- 172.21.15.67:0/3374586899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e3817390 0x55e0e3be9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.557+0000 7f8a4ffff640 1 --2- 172.21.15.67:0/3374586899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3838070 0x55e0e3bf5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:54:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.557+0000 7f8a4f7fe640 1 --2- 172.21.15.67:0/3374586899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e380f1b0 0x55e0e3be9250 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a40012fd0 tx=0x7f8a400052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d49fbca2b3b75ab5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.557+0000 7f8a2effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:54:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.557+0000 7f8a2effd640 1 -- 172.21.15.67:0/3374586899 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4001e040 con 0x55e0e380f1b0 2024-07-31T14:54:22.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.557+0000 7f8a2effd640 1 -- 172.21.15.67:0/3374586899 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f8a400054e0 con 0x55e0e380f1b0 2024-07-31T14:54:22.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.557+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3374586899 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a40009e30 con 0x55e0e380f1b0 2024-07-31T14:54:22.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.557+0000 7f8a2effd640 1 -- 172.21.15.67:0/3374586899 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8a40032050 con 0x55e0e380f1b0 2024-07-31T14:54:22.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.557+0000 7f8a5c5b0740 10 client.26170 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:22.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.557+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3374586899 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e0e3bf7ec0 con 0x55e0e380f1b0 2024-07-31T14:54:22.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.557+0000 7f8a2effd640 1 -- 172.21.15.67:0/3374586899 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a40009e30 con 0x55e0e380f1b0 2024-07-31T14:54:22.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.558+0000 7f8a2effd640 1 -- 172.21.15.67:0/3374586899 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f8a40021030 con 0x55e0e380f1b0 2024-07-31T14:54:22.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.558+0000 7f8a5c5b0740 10 client.26170 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:22.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.558+0000 7f8a5c5b0740 10 client.26170 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:54:22.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.558+0000 7f8a5c5b0740 20 client.26170 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:22.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.558+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/3374586899 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55e0e37f69f0 0x55e0e3bfb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.558+0000 7f8a5c5b0740 4 client.26170 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:54:22.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.558+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3374586899 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0e3b55b00 con 0x55e0e37f69f0 2024-07-31T14:54:22.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.558+0000 7f8a4effd640 1 --2- 172.21.15.67:0/3374586899 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55e0e37f69f0 0x55e0e3bfb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.558+0000 7f8a4effd640 1 --2- 172.21.15.67:0/3374586899 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55e0e37f69f0 0x55e0e3bfb2d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8914b3a6b7ec79c8 server_cookie=4132926db98da5a0 in_seq=0 out_seq=0 2024-07-31T14:54:22.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.558+0000 7f8a2effd640 10 client.26170 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:54:22.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.559+0000 7f8a2effd640 1 -- 172.21.15.67:0/3374586899 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0e3b55b00 con 0x55e0e37f69f0 2024-07-31T14:54:22.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.559+0000 7f8a2effd640 10 client.26170 handle_command_reply: tid=0 2024-07-31T14:54:22.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.559+0000 7f8a5c5b0740 1 client.26170 shutdown 2024-07-31T14:54:22.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.559+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3374586899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e380f1b0 msgr2=0x55e0e3be9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:22.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.559+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/3374586899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e380f1b0 0x55e0e3be9250 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a40012fd0 tx=0x7f8a400052f0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.559+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3374586899 shutdown_connections 2024-07-31T14:54:22.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.559+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/3374586899 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55e0e37f69f0 0x55e0e3bfb2d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.559+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/3374586899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3838070 0x55e0e3bf5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.559+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/3374586899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e3817390 0x55e0e3be9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.559+0000 7f8a2effd640 0 client.26170 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:54:22.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.559+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/3374586899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e380f1b0 0x55e0e3be9250 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.559+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3374586899 >> 172.21.15.67:0/3374586899 conn(0x55e0e382c460 msgr2=0x55e0e3bd9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:22.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.559+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3374586899 shutdown_connections 2024-07-31T14:54:22.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.560+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3374586899 wait complete. 2024-07-31T14:54:22.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.560+0000 7f8a5c5b0740 20 client.26170 trim_cache size 0 max 16384 2024-07-31T14:54:22.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.573+0000 7f8a5c5b0740 1 Processor -- start 2024-07-31T14:54:22.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.573+0000 7f8a5c5b0740 1 -- start start 2024-07-31T14:54:22.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.573+0000 7f8a5c5b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e37f69f0 0x55e0e3bdb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a5c5b0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e380f1b0 0x55e0e3bf4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a5c5b0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3817390 0x55e0e3bf54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a5c5b0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501b29b0 con 0x55e0e380f1b0 2024-07-31T14:54:22.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a5c5b0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501b4670 con 0x55e0e3817390 2024-07-31T14:54:22.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a5c5b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501b3080 con 0x55e0e37f69f0 2024-07-31T14:54:22.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a4effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e380f1b0 0x55e0e3bf4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a4effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e380f1b0 0x55e0e3bf4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52916/0 (socket says 172.21.15.67:52916) 2024-07-31T14:54:22.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a4f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e37f69f0 0x55e0e3bdb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a4effd640 1 -- 172.21.15.67:0/4271832891 learned_addr learned my addr 172.21.15.67:0/4271832891 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:22.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a4ffff640 1 --2- 172.21.15.67:0/4271832891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3817390 0x55e0e3bf54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/4271832891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3817390 msgr2=0x55e0e3bf54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:22.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a4f7fe640 1 --2- 172.21.15.67:0/4271832891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3817390 0x55e0e3bf54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/4271832891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e380f1b0 msgr2=0x55e0e3bf4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:22.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a4f7fe640 1 --2- 172.21.15.67:0/4271832891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e380f1b0 0x55e0e3bf4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/4271832891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e3bea310 con 0x55e0e37f69f0 2024-07-31T14:54:22.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a4effd640 1 --2- 172.21.15.67:0/4271832891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e380f1b0 0x55e0e3bf4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:22.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a4f7fe640 1 --2- 172.21.15.67:0/4271832891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e37f69f0 0x55e0e3bdb300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8a40005e00 tx=0x7f8a40004150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df0c76b5da70e88 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:22.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a357fa640 1 -- 172.21.15.67:0/4271832891 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4001e040 con 0x55e0e37f69f0 2024-07-31T14:54:22.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a357fa640 1 -- 172.21.15.67:0/4271832891 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a40029070 con 0x55e0e37f69f0 2024-07-31T14:54:22.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.574+0000 7f8a357fa640 1 -- 172.21.15.67:0/4271832891 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40000e60 con 0x55e0e37f69f0 2024-07-31T14:54:22.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.575+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/4271832891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e37f69f0 msgr2=0x55e0e3bdb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:22.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.575+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/4271832891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e37f69f0 0x55e0e3bdb300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8a40005e00 tx=0x7f8a40004150 comp rx=0 tx=0).stop 2024-07-31T14:54:22.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.575+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/4271832891 shutdown_connections 2024-07-31T14:54:22.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.575+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/4271832891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3817390 0x55e0e3bf54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.575+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/4271832891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e380f1b0 0x55e0e3bf4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.575+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/4271832891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e37f69f0 0x55e0e3bdb300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.575+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/4271832891 >> 172.21.15.67:0/4271832891 conn(0x55e0e382c460 msgr2=0x55e0e3bdabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:22.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.575+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/4271832891 shutdown_connections 2024-07-31T14:54:22.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.575+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/4271832891 wait complete. 2024-07-31T14:54:22.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.576+0000 7f8a5c5b0740 1 -- start start 2024-07-31T14:54:22.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.576+0000 7f8a5c5b0740 1 Processor -- start 2024-07-31T14:54:22.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.576+0000 7f8a5c5b0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e37f69f0 0x55e0e3bec8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.576+0000 7f8a5c5b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e380f1b0 0x55e0e3becea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.576+0000 7f8a5c5b0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3817390 0x55e0e3c5b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.576+0000 7f8a5c5b0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501b3fa0 con 0x55e0e37f69f0 2024-07-31T14:54:22.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.576+0000 7f8a5c5b0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501b3810 con 0x55e0e3817390 2024-07-31T14:54:22.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.576+0000 7f8a5c5b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501b0990 con 0x55e0e380f1b0 2024-07-31T14:54:22.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.576+0000 7f8a5c5b0740 10 client.0 _ll_register_callbacks cb 0x7f8a4001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:22.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.576+0000 7f8a4f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e37f69f0 0x55e0e3bec8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.576+0000 7f8a4f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e37f69f0 0x55e0e3bec8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52932/0 (socket says 172.21.15.67:52932) 2024-07-31T14:54:22.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.576+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/3651966222 learned_addr learned my addr 172.21.15.67:0/3651966222 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:22.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.576+0000 7f8a4effd640 1 --2- 172.21.15.67:0/3651966222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e380f1b0 0x55e0e3becea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.576+0000 7f8a4ffff640 1 --2- 172.21.15.67:0/3651966222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3817390 0x55e0e3c5b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.577+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/3651966222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e380f1b0 msgr2=0x55e0e3becea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:22.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.577+0000 7f8a4f7fe640 1 --2- 172.21.15.67:0/3651966222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e380f1b0 0x55e0e3becea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.577+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/3651966222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3817390 msgr2=0x55e0e3c5b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:22.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.577+0000 7f8a4f7fe640 1 --2- 172.21.15.67:0/3651966222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3817390 0x55e0e3c5b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.577+0000 7f8a4f7fe640 1 -- 172.21.15.67:0/3651966222 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e3c5bfa0 con 0x55e0e37f69f0 2024-07-31T14:54:22.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.577+0000 7f8a4effd640 1 --2- 172.21.15.67:0/3651966222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e380f1b0 0x55e0e3becea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:22.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.577+0000 7f8a4f7fe640 1 --2- 172.21.15.67:0/3651966222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e37f69f0 0x55e0e3bec8c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006010 tx=0x7f8a40000e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2911c004f464bddf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:22.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.577+0000 7f8a2effd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:22.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.577+0000 7f8a2effd640 1 -- 172.21.15.67:0/3651966222 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4001e070 con 0x55e0e37f69f0 2024-07-31T14:54:22.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.577+0000 7f8a2effd640 1 -- 172.21.15.67:0/3651966222 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f8a40003af0 con 0x55e0e37f69f0 2024-07-31T14:54:22.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.577+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3651966222 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a40029070 con 0x55e0e37f69f0 2024-07-31T14:54:22.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.577+0000 7f8a2effd640 1 -- 172.21.15.67:0/3651966222 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8a40049050 con 0x55e0e37f69f0 2024-07-31T14:54:22.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.577+0000 7f8a5c5b0740 10 client.16518 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:22.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.577+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3651966222 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e0e3c632e0 con 0x55e0e37f69f0 2024-07-31T14:54:22.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.578+0000 7f8a2effd640 1 -- 172.21.15.67:0/3651966222 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a40029070 con 0x55e0e37f69f0 2024-07-31T14:54:22.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.578+0000 7f8a2effd640 1 -- 172.21.15.67:0/3651966222 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f8a40021070 con 0x55e0e37f69f0 2024-07-31T14:54:22.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.578+0000 7f8a5c5b0740 10 client.16518 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:22.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.578+0000 7f8a5c5b0740 10 client.16518 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:54:22.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.578+0000 7f8a5c5b0740 20 client.16518 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:22.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.578+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/3651966222 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55e0e3838070 0x55e0e3c65a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:22.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.578+0000 7f8a5c5b0740 4 client.16518 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:54:22.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.578+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3651966222 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0e3b515a0 con 0x55e0e3838070 2024-07-31T14:54:22.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.578+0000 7f8a4effd640 1 --2- 172.21.15.67:0/3651966222 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55e0e3838070 0x55e0e3c65a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:22.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.578+0000 7f8a2effd640 10 client.16518 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:54:22.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.578+0000 7f8a4effd640 1 --2- 172.21.15.67:0/3651966222 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55e0e3838070 0x55e0e3c65a70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de25bcab06777ec9 server_cookie=cc241aa67c164984 in_seq=0 out_seq=0 2024-07-31T14:54:22.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a2effd640 1 -- 172.21.15.67:0/3651966222 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0e3b515a0 con 0x55e0e3838070 2024-07-31T14:54:22.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a2effd640 10 client.16518 handle_command_reply: tid=0 2024-07-31T14:54:22.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a5c5b0740 1 client.16518 shutdown 2024-07-31T14:54:22.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3651966222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e37f69f0 msgr2=0x55e0e3bec8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:22.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/3651966222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e37f69f0 0x55e0e3bec8c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006010 tx=0x7f8a40000e40 comp rx=0 tx=0).stop 2024-07-31T14:54:22.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3651966222 shutdown_connections 2024-07-31T14:54:22.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/3651966222 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55e0e3838070 0x55e0e3c65a70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/3651966222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0e3817390 0x55e0e3c5b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/3651966222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0e380f1b0 0x55e0e3becea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a5c5b0740 1 --2- 172.21.15.67:0/3651966222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0e37f69f0 0x55e0e3bec8c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3651966222 >> 172.21.15.67:0/3651966222 conn(0x55e0e382c460 msgr2=0x55e0e3bdabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:22.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a2effd640 0 client.16518 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:54:22.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3651966222 shutdown_connections 2024-07-31T14:54:22.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a5c5b0740 1 -- 172.21.15.67:0/3651966222 wait complete. 2024-07-31T14:54:22.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.579+0000 7f8a5c5b0740 20 client.16518 trim_cache size 0 max 16384 2024-07-31T14:54:22.586 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:54:22.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.580+0000 7f8a5605f640 1 -- 172.21.15.67:0/2137702049 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a380b3d40 msgr2=0x7f8a380b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:22.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.580+0000 7f8a5605f640 1 --2- 172.21.15.67:0/2137702049 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a380b3d40 0x7f8a380b6250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8a440096c0 tx=0x7f8a44000bc0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.580+0000 7f8a5605f640 1 -- 172.21.15.67:0/2137702049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a50109820 msgr2=0x7f8a50075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:22.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.580+0000 7f8a5605f640 1 --2- 172.21.15.67:0/2137702049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a50109820 0x7f8a50075bc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009fd0 tx=0x7f8a4800ef10 comp rx=0 tx=0).stop 2024-07-31T14:54:22.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.580+0000 7f8a5605f640 1 -- 172.21.15.67:0/2137702049 shutdown_connections 2024-07-31T14:54:22.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.580+0000 7f8a5605f640 1 --2- 172.21.15.67:0/2137702049 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a380b3d40 0x7f8a380b6250 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.580+0000 7f8a5605f640 1 --2- 172.21.15.67:0/2137702049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a50109820 0x7f8a50075bc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.580+0000 7f8a5605f640 1 --2- 172.21.15.67:0/2137702049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a50108ed0 0x7f8a50075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.580+0000 7f8a5605f640 1 --2- 172.21.15.67:0/2137702049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a50103590 0x7f8a50075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:22.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.580+0000 7f8a5605f640 1 -- 172.21.15.67:0/2137702049 >> 172.21.15.67:0/2137702049 conn(0x7f8a5007b280 msgr2=0x7f8a50107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:22.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.580+0000 7f8a5605f640 1 -- 172.21.15.67:0/2137702049 shutdown_connections 2024-07-31T14:54:22.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:22.580+0000 7f8a5605f640 1 -- 172.21.15.67:0/2137702049 wait complete. 2024-07-31T14:54:23.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.010+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2564995577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4109790 msgr2=0x7ff4c410ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:23.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.010+0000 7ff4c8c92640 1 --2- 172.21.15.67:0/2564995577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4109790 0x7ff4c410ffc0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009ca0 tx=0x7ff4bc02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.010+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2564995577 shutdown_connections 2024-07-31T14:54:23.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.010+0000 7ff4c8c92640 1 --2- 172.21.15.67:0/2564995577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4109790 0x7ff4c410ffc0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.010+0000 7ff4c8c92640 1 --2- 172.21.15.67:0/2564995577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4c4108e40 0x7ff4c4109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.010+0000 7ff4c8c92640 1 --2- 172.21.15.67:0/2564995577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4c4103510 0x7ff4c4103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.010+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2564995577 >> 172.21.15.67:0/2564995577 conn(0x7ff4c40fef10 msgr2=0x7ff4c4101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.011+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2564995577 shutdown_connections 2024-07-31T14:54:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.011+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2564995577 wait complete. 2024-07-31T14:54:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.013+0000 7ff4c8c92640 1 Processor -- start 2024-07-31T14:54:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.013+0000 7ff4c8c92640 1 -- start start 2024-07-31T14:54:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.013+0000 7ff4c8c92640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4c4103510 0x7ff4c41a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.013+0000 7ff4c8c92640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4c4108e40 0x7ff4c41a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.013+0000 7ff4c8c92640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4109790 0x7ff4c41a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.013+0000 7ff4c1d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4c4108e40 0x7ff4c41a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.013+0000 7ff4c259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4c4103510 0x7ff4c41a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.013+0000 7ff4c1d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4c4108e40 0x7ff4c41a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:39794/0 (socket says 172.21.15.67:39794) 2024-07-31T14:54:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.014+0000 7ff4c1d9b640 1 -- 172.21.15.67:0/2311254228 learned_addr learned my addr 172.21.15.67:0/2311254228 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.014+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2311254228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4111260 con 0x7ff4c4109790 2024-07-31T14:54:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.014+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2311254228 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41110e0 con 0x7ff4c4103510 2024-07-31T14:54:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.014+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2311254228 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41113e0 con 0x7ff4c4108e40 2024-07-31T14:54:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.014+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/2311254228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4109790 0x7ff4c41a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.014+0000 7ff4c259c640 1 -- 172.21.15.67:0/2311254228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4c4108e40 msgr2=0x7ff4c41a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.014+0000 7ff4c259c640 1 --2- 172.21.15.67:0/2311254228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4c4108e40 0x7ff4c41a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.014+0000 7ff4c259c640 1 -- 172.21.15.67:0/2311254228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4109790 msgr2=0x7ff4c41a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.014+0000 7ff4c259c640 1 --2- 172.21.15.67:0/2311254228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4109790 0x7ff4c41a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.014+0000 7ff4c259c640 1 -- 172.21.15.67:0/2311254228 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c41a62d0 con 0x7ff4c4103510 2024-07-31T14:54:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.015+0000 7ff4c259c640 1 --2- 172.21.15.67:0/2311254228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4c4103510 0x7ff4c41a0430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400c970 tx=0x7ff4b400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.015+0000 7ff4b37fe640 1 -- 172.21.15.67:0/2311254228 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4007bb0 con 0x7ff4c4103510 2024-07-31T14:54:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.015+0000 7ff4b37fe640 1 -- 172.21.15.67:0/2311254228 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4b4007d50 con 0x7ff4c4103510 2024-07-31T14:54:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.015+0000 7ff4b37fe640 1 -- 172.21.15.67:0/2311254228 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4018780 con 0x7ff4c4103510 2024-07-31T14:54:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.015+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2311254228 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c41a65c0 con 0x7ff4c4103510 2024-07-31T14:54:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.015+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2311254228 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c41acde0 con 0x7ff4c4103510 2024-07-31T14:54:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.019+0000 7ff4cfb84740 1 Processor -- start 2024-07-31T14:54:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.019+0000 7ff4cfb84740 1 -- start start 2024-07-31T14:54:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.019+0000 7ff4cfb84740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c32031b0 0x5628c35a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4cfb84740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35a6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4cfb84740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31e24d0 0x5628c35a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4cfb84740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5628c351b3b0 con 0x5628c32031b0 2024-07-31T14:54:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4cfb84740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5628c350a8b0 con 0x5628c31da2f0 2024-07-31T14:54:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4cfb84740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5628c350e8f0 con 0x5628c31e24d0 2024-07-31T14:54:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4c2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c32031b0 0x5628c35a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4b37fe640 1 -- 172.21.15.67:0/2311254228 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff4b400f450 con 0x7ff4c4103510 2024-07-31T14:54:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4c2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c32031b0 0x5628c35a5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52954/0 (socket says 172.21.15.67:52954) 2024-07-31T14:54:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/3973705913 learned_addr learned my addr 172.21.15.67:0/3973705913 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4c259c640 1 --2- 172.21.15.67:0/3973705913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4c1d9b640 1 --2- 172.21.15.67:0/3973705913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31e24d0 0x5628c35a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/3973705913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31e24d0 msgr2=0x5628c35a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/3973705913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31e24d0 0x5628c35a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/3973705913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 msgr2=0x5628c35a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/3973705913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35a6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/3973705913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c34b19a0 con 0x5628c32031b0 2024-07-31T14:54:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4c1d9b640 1 --2- 172.21.15.67:0/3973705913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31e24d0 0x5628c35a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:54:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.020+0000 7ff4c259c640 1 --2- 172.21.15.67:0/3973705913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:54:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.021+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/3973705913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c32031b0 0x5628c35a5ce0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009900 tx=0x7ff4bc009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f75287e6bb273a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.021+0000 7ff4b0ff9640 1 -- 172.21.15.67:0/3973705913 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc031cf0 con 0x5628c32031b0 2024-07-31T14:54:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.021+0000 7ff4b37fe640 1 --2- 172.21.15.67:0/2311254228 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff4ac0b3e80 0x7ff4ac0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.021+0000 7ff4b0ff9640 1 -- 172.21.15.67:0/3973705913 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4bc036040 con 0x5628c32031b0 2024-07-31T14:54:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.022+0000 7ff4b0ff9640 1 -- 172.21.15.67:0/3973705913 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0042a0 con 0x5628c32031b0 2024-07-31T14:54:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.022+0000 7ff4c259c640 1 --2- 172.21.15.67:0/2311254228 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff4ac0b3e80 0x7ff4ac0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.022+0000 7ff4b37fe640 1 -- 172.21.15.67:0/2311254228 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff4b40e43c0 con 0x7ff4c4103510 2024-07-31T14:54:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.023+0000 7ff4cfb84740 1 -- 172.21.15.67:0/3973705913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c32031b0 msgr2=0x5628c35a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.023+0000 7ff4c259c640 1 --2- 172.21.15.67:0/2311254228 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff4ac0b3e80 0x7ff4ac0b6390 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x5628c35a4960 tx=0x7ff4b409f4e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.023+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/3973705913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c32031b0 0x5628c35a5ce0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009900 tx=0x7ff4bc009930 comp rx=0 tx=0).stop 2024-07-31T14:54:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.023+0000 7ff4cfb84740 1 -- 172.21.15.67:0/3973705913 shutdown_connections 2024-07-31T14:54:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.023+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/3973705913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31e24d0 0x5628c35a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.023+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/3973705913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.023+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/3973705913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c32031b0 0x5628c35a5ce0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.023+0000 7ff4cfb84740 1 -- 172.21.15.67:0/3973705913 >> 172.21.15.67:0/3973705913 conn(0x5628c31f7300 msgr2=0x5628c35a3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.023+0000 7ff4cfb84740 1 -- 172.21.15.67:0/3973705913 shutdown_connections 2024-07-31T14:54:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.023+0000 7ff4cfb84740 1 -- 172.21.15.67:0/3973705913 wait complete. 2024-07-31T14:54:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.024+0000 7ff4cfb84740 1 -- start start 2024-07-31T14:54:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.024+0000 7ff4cfb84740 1 Processor -- start 2024-07-31T14:54:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.025+0000 7ff4cfb84740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.025+0000 7ff4cfb84740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c31e24d0 0x5628c35b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.025+0000 7ff4cfb84740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c32031b0 0x5628c35bd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.025+0000 7ff4cfb84740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5628c3444ab0 con 0x5628c31e24d0 2024-07-31T14:54:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.025+0000 7ff4cfb84740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5628c3492cf0 con 0x5628c31da2f0 2024-07-31T14:54:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.025+0000 7ff4cfb84740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5628c35b44a0 con 0x5628c32031b0 2024-07-31T14:54:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.025+0000 7ff4cfb84740 10 client.0 _ll_register_callbacks cb 0x5628c35049f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.025+0000 7ff4c2d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.025+0000 7ff4c2d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35864/0 (socket says 172.21.15.67:35864) 2024-07-31T14:54:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.025+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/1778204483 learned_addr learned my addr 172.21.15.67:0/1778204483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.026+0000 7ff4c1d9b640 1 --2- 172.21.15.67:0/1778204483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c31e24d0 0x5628c35b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.026+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/1778204483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c32031b0 0x5628c35bd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.026+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/1778204483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c32031b0 msgr2=0x5628c35bd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.026+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/1778204483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c32031b0 0x5628c35bd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.026+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/1778204483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c31e24d0 msgr2=0x5628c35b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.026+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/1778204483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c31e24d0 0x5628c35b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.026+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/1778204483 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c326e790 con 0x5628c31da2f0 2024-07-31T14:54:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.026+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/1778204483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35b1810 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009ca0 tx=0x7ff4bc003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9bd263d1b41a40e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.026+0000 7ff4a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:54:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.027+0000 7ff4a9ffb640 1 -- 172.21.15.67:0/1778204483 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc058060 con 0x5628c31da2f0 2024-07-31T14:54:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.027+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1778204483 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4bc036040 con 0x5628c31da2f0 2024-07-31T14:54:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.027+0000 7ff4a9ffb640 1 -- 172.21.15.67:0/1778204483 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff4bc031860 con 0x5628c31da2f0 2024-07-31T14:54:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.027+0000 7ff4a9ffb640 1 -- 172.21.15.67:0/1778204483 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff4bc04a040 con 0x5628c31da2f0 2024-07-31T14:54:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.027+0000 7ff4cfb84740 10 client.26141 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.027+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1778204483 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5628c32e7620 con 0x5628c31da2f0 2024-07-31T14:54:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.027+0000 7ff4a9ffb640 1 -- 172.21.15.67:0/1778204483 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4bc02fac0 con 0x5628c31da2f0 2024-07-31T14:54:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.028+0000 7ff4a9ffb640 1 -- 172.21.15.67:0/1778204483 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7ff4bc050070 con 0x5628c31da2f0 2024-07-31T14:54:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.028+0000 7ff4cfb84740 10 client.26141 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.028+0000 7ff4cfb84740 10 client.26141 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.028+0000 7ff4cfb84740 20 client.26141 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.028+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1778204483 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c31c1d80 0x5628c35c3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.028+0000 7ff4cfb84740 4 client.26141 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:54:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.028+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1778204483 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff4bc036040 con 0x5628c31c1d80 2024-07-31T14:54:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.028+0000 7ff4c1d9b640 1 --2- 172.21.15.67:0/1778204483 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c31c1d80 0x5628c35c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.029+0000 7ff4c1d9b640 1 --2- 172.21.15.67:0/1778204483 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c31c1d80 0x5628c35c3580 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c59bd2ffdc194ca server_cookie=187fa8516808b6d7 in_seq=0 out_seq=0 2024-07-31T14:54:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.029+0000 7ff4a9ffb640 10 client.26141 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.030+0000 7ff4a9ffb640 1 -- 172.21.15.67:0/1778204483 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff4bc036040 con 0x5628c31c1d80 2024-07-31T14:54:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.030+0000 7ff4a9ffb640 10 client.26141 handle_command_reply: tid=0 2024-07-31T14:54:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.030+0000 7ff4cfb84740 1 client.26141 shutdown 2024-07-31T14:54:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.031+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1778204483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 msgr2=0x5628c35b1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.031+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1778204483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35b1810 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009ca0 tx=0x7ff4bc003e10 comp rx=0 tx=0).stop 2024-07-31T14:54:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.031+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1778204483 shutdown_connections 2024-07-31T14:54:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.031+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1778204483 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c31c1d80 0x5628c35c3580 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.031+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1778204483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c32031b0 0x5628c35bd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.031+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1778204483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c31e24d0 0x5628c35b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.031+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1778204483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35b1810 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.031+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1778204483 >> 172.21.15.67:0/1778204483 conn(0x5628c31f7300 msgr2=0x5628c35a22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.031+0000 7ff4a9ffb640 0 client.26141 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.031+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1778204483 shutdown_connections 2024-07-31T14:54:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.031+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1778204483 wait complete. 2024-07-31T14:54:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.032+0000 7ff4cfb84740 20 client.26141 trim_cache size 0 max 16384 2024-07-31T14:54:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.062+0000 7ff4cfb84740 1 Processor -- start 2024-07-31T14:54:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.062+0000 7ff4cfb84740 1 -- start start 2024-07-31T14:54:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.063+0000 7ff4cfb84740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.063+0000 7ff4cfb84740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31e24d0 0x5628c35bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.063+0000 7ff4cfb84740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c32031b0 0x5628c35bdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.063+0000 7ff4cfb84740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41b6c10 con 0x5628c32031b0 2024-07-31T14:54:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.063+0000 7ff4c2d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.063+0000 7ff4c2d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35880/0 (socket says 172.21.15.67:35880) 2024-07-31T14:54:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.063+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/1924811400 learned_addr learned my addr 172.21.15.67:0/1924811400 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.063+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1924811400 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41b88d0 con 0x5628c31da2f0 2024-07-31T14:54:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.063+0000 7ff4c1d9b640 1 --2- 172.21.15.67:0/1924811400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31e24d0 0x5628c35bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.063+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1924811400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41b72e0 con 0x5628c31e24d0 2024-07-31T14:54:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.063+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/1924811400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c32031b0 0x5628c35bdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.064+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/1924811400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31e24d0 msgr2=0x5628c35bd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.064+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/1924811400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31e24d0 0x5628c35bd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.064+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/1924811400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c32031b0 msgr2=0x5628c35bdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.064+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/1924811400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c32031b0 0x5628c35bdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.064+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/1924811400 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c35a8af0 con 0x5628c31da2f0 2024-07-31T14:54:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.064+0000 7ff4c1d9b640 1 --2- 172.21.15.67:0/1924811400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31e24d0 0x5628c35bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.064+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/1924811400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35b34e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc031b90 tx=0x7ff4bc0021d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f55e48c62c8477f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.064+0000 7ff4abfff640 1 -- 172.21.15.67:0/1924811400 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc058070 con 0x5628c31da2f0 2024-07-31T14:54:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.064+0000 7ff4abfff640 1 -- 172.21.15.67:0/1924811400 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4bc036040 con 0x5628c31da2f0 2024-07-31T14:54:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.064+0000 7ff4abfff640 1 -- 172.21.15.67:0/1924811400 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0318c0 con 0x5628c31da2f0 2024-07-31T14:54:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.065+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1924811400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 msgr2=0x5628c35b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.065+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1924811400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35b34e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc031b90 tx=0x7ff4bc0021d0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.065+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1924811400 shutdown_connections 2024-07-31T14:54:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.065+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1924811400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c32031b0 0x5628c35bdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.066+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1924811400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31e24d0 0x5628c35bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.066+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1924811400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c31da2f0 0x5628c35b34e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.066+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1924811400 >> 172.21.15.67:0/1924811400 conn(0x5628c31f7300 msgr2=0x5628c35aada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.066+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1924811400 shutdown_connections 2024-07-31T14:54:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.066+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1924811400 wait complete. 2024-07-31T14:54:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.066+0000 7ff4cfb84740 1 -- start start 2024-07-31T14:54:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.066+0000 7ff4cfb84740 1 Processor -- start 2024-07-31T14:54:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.067+0000 7ff4cfb84740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31da2f0 0x5628c361dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.067+0000 7ff4cfb84740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c31e24d0 0x5628c361e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.067+0000 7ff4cfb84740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c32031b0 0x5628c3628ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.068+0000 7ff4cfb84740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41b8200 con 0x5628c31e24d0 2024-07-31T14:54:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.068+0000 7ff4cfb84740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41b7a70 con 0x5628c32031b0 2024-07-31T14:54:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.068+0000 7ff4c2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31da2f0 0x5628c361dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.068+0000 7ff4c2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31da2f0 0x5628c361dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:39836/0 (socket says 172.21.15.67:39836) 2024-07-31T14:54:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.068+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/1133959518 learned_addr learned my addr 172.21.15.67:0/1133959518 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.068+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1133959518 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41b4bf0 con 0x5628c31da2f0 2024-07-31T14:54:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.068+0000 7ff4cfb84740 10 client.0 _ll_register_callbacks cb 0x7ff4bc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.068+0000 7ff4c1d9b640 1 --2- 172.21.15.67:0/1133959518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c31e24d0 0x5628c361e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.069+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/1133959518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c32031b0 0x5628c3628ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.069+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/1133959518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c32031b0 msgr2=0x5628c3628ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.069+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/1133959518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c32031b0 0x5628c3628ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.069+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/1133959518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c31e24d0 msgr2=0x5628c361e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.069+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/1133959518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c31e24d0 0x5628c361e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.069+0000 7ff4c2d9d640 1 -- 172.21.15.67:0/1133959518 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c35a4790 con 0x5628c31da2f0 2024-07-31T14:54:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.069+0000 7ff4c2d9d640 1 --2- 172.21.15.67:0/1133959518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31da2f0 0x5628c361dc60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc030be0 tx=0x7ff4bc005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b28c2c6558b633 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.069+0000 7ff4a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:54:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.070+0000 7ff4a9ffb640 1 -- 172.21.15.67:0/1133959518 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc058040 con 0x5628c31da2f0 2024-07-31T14:54:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.070+0000 7ff4a9ffb640 1 -- 172.21.15.67:0/1133959518 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff4bc004b40 con 0x5628c31da2f0 2024-07-31T14:54:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.070+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1133959518 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4bc036040 con 0x5628c31da2f0 2024-07-31T14:54:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.071+0000 7ff4a9ffb640 1 -- 172.21.15.67:0/1133959518 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff4bc02f610 con 0x5628c31da2f0 2024-07-31T14:54:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.071+0000 7ff4cfb84740 10 client.26188 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.071+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1133959518 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5628c362a130 con 0x5628c31da2f0 2024-07-31T14:54:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.072+0000 7ff4a9ffb640 1 -- 172.21.15.67:0/1133959518 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4bc036040 con 0x5628c31da2f0 2024-07-31T14:54:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.072+0000 7ff4a9ffb640 1 -- 172.21.15.67:0/1133959518 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7ff4bc071040 con 0x5628c31da2f0 2024-07-31T14:54:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.072+0000 7ff4cfb84740 10 client.26188 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.072+0000 7ff4cfb84740 10 client.26188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.072+0000 7ff4cfb84740 20 client.26188 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.072+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1133959518 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c31971c0 0x5628c362c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.072+0000 7ff4cfb84740 4 client.26188 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:54:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.072+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1133959518 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628c351d300 con 0x5628c31971c0 2024-07-31T14:54:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.072+0000 7ff4c1d9b640 1 --2- 172.21.15.67:0/1133959518 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c31971c0 0x5628c362c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.073+0000 7ff4a9ffb640 10 client.26188 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.073+0000 7ff4c1d9b640 1 --2- 172.21.15.67:0/1133959518 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c31971c0 0x5628c362c6d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b8c0bbd807fb7f6 server_cookie=6e6dab540a613bcc in_seq=0 out_seq=0 2024-07-31T14:54:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.158+0000 7ff4a9ffb640 1 -- 172.21.15.67:0/1133959518 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5628c351d300 con 0x5628c31971c0 2024-07-31T14:54:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.158+0000 7ff4a9ffb640 10 client.26188 handle_command_reply: tid=0 2024-07-31T14:54:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.159+0000 7ff4cfb84740 1 client.26188 shutdown 2024-07-31T14:54:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.159+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1133959518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31da2f0 msgr2=0x5628c361dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.159+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1133959518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31da2f0 0x5628c361dc60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc030be0 tx=0x7ff4bc005c70 comp rx=0 tx=0).stop 2024-07-31T14:54:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.159+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1133959518 shutdown_connections 2024-07-31T14:54:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.159+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1133959518 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c31971c0 0x5628c362c6d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.159+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1133959518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c32031b0 0x5628c3628ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.159+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1133959518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c31e24d0 0x5628c361e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.159+0000 7ff4cfb84740 1 --2- 172.21.15.67:0/1133959518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c31da2f0 0x5628c361dc60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.159+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1133959518 >> 172.21.15.67:0/1133959518 conn(0x5628c31f7300 msgr2=0x5628c35b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.159+0000 7ff4a9ffb640 0 client.26188 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.159+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1133959518 shutdown_connections 2024-07-31T14:54:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.159+0000 7ff4cfb84740 1 -- 172.21.15.67:0/1133959518 wait complete. 2024-07-31T14:54:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.159+0000 7ff4cfb84740 20 client.26188 trim_cache size 0 max 16384 2024-07-31T14:54:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:54:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:54:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:54:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:54:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.160+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2311254228 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff4ac0b3e80 msgr2=0x7ff4ac0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.160+0000 7ff4c8c92640 1 --2- 172.21.15.67:0/2311254228 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff4ac0b3e80 0x7ff4ac0b6390 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x5628c35a4960 tx=0x7ff4b409f4e0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.160+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2311254228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4c4103510 msgr2=0x7ff4c41a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.160+0000 7ff4c8c92640 1 --2- 172.21.15.67:0/2311254228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4c4103510 0x7ff4c41a0430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400c970 tx=0x7ff4b400ce30 comp rx=0 tx=0).stop 2024-07-31T14:54:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.160+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2311254228 shutdown_connections 2024-07-31T14:54:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.160+0000 7ff4c8c92640 1 --2- 172.21.15.67:0/2311254228 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff4ac0b3e80 0x7ff4ac0b6390 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.160+0000 7ff4c8c92640 1 --2- 172.21.15.67:0/2311254228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4109790 0x7ff4c41a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.160+0000 7ff4c8c92640 1 --2- 172.21.15.67:0/2311254228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4c4108e40 0x7ff4c41a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.160+0000 7ff4c8c92640 1 --2- 172.21.15.67:0/2311254228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4c4103510 0x7ff4c41a0430 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.160+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2311254228 >> 172.21.15.67:0/2311254228 conn(0x7ff4c40fef10 msgr2=0x7ff4c4107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.160+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2311254228 shutdown_connections 2024-07-31T14:54:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:23.160+0000 7ff4c8c92640 1 -- 172.21.15.67:0/2311254228 wait complete. 2024-07-31T14:54:23.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:23 smithi067 ceph-mon[32750]: pgmap v499: 129 pgs: 129 active+clean; 325 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 513 op/s 2024-07-31T14:54:23.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T14:54:23.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:23 smithi073 ceph-mon[36714]: pgmap v499: 129 pgs: 129 active+clean; 325 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 513 op/s 2024-07-31T14:54:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:23 smithi077 ceph-mon[37327]: pgmap v499: 129 pgs: 129 active+clean; 325 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 513 op/s 2024-07-31T14:54:23.657 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:54:23.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.887+0000 7fa27cff9640 1 -- 172.21.15.67:0/2799494786 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x563b76b4e1a0 con 0x563b767c9570 2024-07-31T14:54:23.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.887+0000 7fa27cff9640 10 client.26087 handle_command_reply: tid=0 2024-07-31T14:54:23.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.887+0000 7fa2a1e86740 1 client.26087 shutdown 2024-07-31T14:54:23.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.888+0000 7fa2a1e86740 1 -- 172.21.15.67:0/2799494786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76835840 msgr2=0x563b76bf9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:23.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.888+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/2799494786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76835840 0x563b76bf9d30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00e930 tx=0x7fa28c019cd0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.888+0000 7fa2a1e86740 1 -- 172.21.15.67:0/2799494786 shutdown_connections 2024-07-31T14:54:23.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.888+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/2799494786 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563b767c9570 0x563b76c63860 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.888+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/2799494786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b76835840 0x563b76bf9d30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.888+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/2799494786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563b76814b60 0x563b76bf97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.888+0000 7fa27cff9640 0 client.26087 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:23.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.888+0000 7fa2a1e86740 1 --2- 172.21.15.67:0/2799494786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563b7680c980 0x563b76bfc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.888+0000 7fa2a1e86740 1 -- 172.21.15.67:0/2799494786 >> 172.21.15.67:0/2799494786 conn(0x563b76829990 msgr2=0x563b76bf2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:23.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.888+0000 7fa2a1e86740 1 -- 172.21.15.67:0/2799494786 shutdown_connections 2024-07-31T14:54:23.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.888+0000 7fa2a1e86740 1 -- 172.21.15.67:0/2799494786 wait complete. 2024-07-31T14:54:23.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.888+0000 7fa2a1e86740 20 client.26087 trim_cache size 0 max 16384 2024-07-31T14:54:23.891 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:54:23.891 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 39, 2024-07-31T14:54:23.891 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T14:54:23.891 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 6, 2024-07-31T14:54:23.892 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T14:54:23.892 INFO:teuthology.orchestra.run.smithi067.stdout: "40ba138f": { 2024-07-31T14:54:23.892 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 6, 2024-07-31T14:54:23.892 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T14:54:23.892 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:54:23.892 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T14:54:23.892 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:54:23.892 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:54:23.892 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T14:54:23.892 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 143.4, 2024-07-31T14:54:23.892 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T14:54:23.892 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T14:54:23.892 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T14:54:23.892 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:54:23.893 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T14:54:23.893 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:54:23.893 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:54:23.893 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:54:23.893 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:54:23.893 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:54:23.893 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:54:23.893 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:54:23.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.889+0000 7fa29b935640 1 -- 172.21.15.67:0/1781045034 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2800b3e90 msgr2=0x7fa2800b63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:23.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.889+0000 7fa29b935640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2800b3e90 0x7fa2800b63a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa284009230 tx=0x7fa284031400 comp rx=0 tx=0).stop 2024-07-31T14:54:23.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.889+0000 7fa29b935640 1 -- 172.21.15.67:0/1781045034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2941035a0 msgr2=0x7fa294199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:23.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.889+0000 7fa29b935640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2941035a0 0x7fa294199ed0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa29000d950 tx=0x7fa29000de10 comp rx=0 tx=0).stop 2024-07-31T14:54:23.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.889+0000 7fa29b935640 1 -- 172.21.15.67:0/1781045034 shutdown_connections 2024-07-31T14:54:23.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.889+0000 7fa29b935640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2800b3e90 0x7fa2800b63a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.889+0000 7fa29b935640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa294109830 0x7fa29419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.889+0000 7fa29b935640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa294108ee0 0x7fa29419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.889+0000 7fa29b935640 1 --2- 172.21.15.67:0/1781045034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2941035a0 0x7fa294199ed0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:23.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.889+0000 7fa29b935640 1 -- 172.21.15.67:0/1781045034 >> 172.21.15.67:0/1781045034 conn(0x7fa2940fefd0 msgr2=0x7fa2941085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:23.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.889+0000 7fa29b935640 1 -- 172.21.15.67:0/1781045034 shutdown_connections 2024-07-31T14:54:23.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:23.889+0000 7fa29b935640 1 -- 172.21.15.67:0/1781045034 wait complete. 2024-07-31T14:54:25.119 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '40ba138f', quiesce took 19.9 seconds. Will release after: 3.5 2024-07-31T14:54:25.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:25 smithi073 ceph-mon[36714]: pgmap v500: 129 pgs: 129 active+clean; 332 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 621 op/s 2024-07-31T14:54:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:25 smithi077 ceph-mon[37327]: pgmap v500: 129 pgs: 129 active+clean; 332 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 621 op/s 2024-07-31T14:54:25.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:25 smithi067 ceph-mon[32750]: pgmap v500: 129 pgs: 129 active+clean; 332 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 621 op/s 2024-07-31T14:54:26.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.920+0000 7f86d5028640 1 -- 172.21.15.67:0/3523225725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86d01035b0 msgr2=0x7f86d01039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:26.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.920+0000 7f86d5028640 1 --2- 172.21.15.67:0/3523225725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86d01035b0 0x7f86d01039c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009900 tx=0x7f86c002a410 comp rx=0 tx=0).stop 2024-07-31T14:54:26.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.920+0000 7f86d5028640 1 -- 172.21.15.67:0/3523225725 shutdown_connections 2024-07-31T14:54:26.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.920+0000 7f86d5028640 1 --2- 172.21.15.67:0/3523225725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86d0109840 0x7f86d0110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.920+0000 7f86d5028640 1 --2- 172.21.15.67:0/3523225725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d0108ef0 0x7f86d0109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.920+0000 7f86d5028640 1 --2- 172.21.15.67:0/3523225725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86d01035b0 0x7f86d01039c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.920+0000 7f86d5028640 1 -- 172.21.15.67:0/3523225725 >> 172.21.15.67:0/3523225725 conn(0x7f86d00ff000 msgr2=0x7f86d0101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:26.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.920+0000 7f86d5028640 1 -- 172.21.15.67:0/3523225725 shutdown_connections 2024-07-31T14:54:26.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.920+0000 7f86d5028640 1 -- 172.21.15.67:0/3523225725 wait complete. 2024-07-31T14:54:26.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.921+0000 7f86d5028640 1 Processor -- start 2024-07-31T14:54:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.921+0000 7f86d5028640 1 -- start start 2024-07-31T14:54:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86d5028640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d01035b0 0x7f86d0199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86d5028640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86d0108ef0 0x7f86d019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86d5028640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86d0109840 0x7f86d019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86d5028640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f86d0113510 con 0x7f86d0109840 2024-07-31T14:54:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86d5028640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86d0113390 con 0x7f86d01035b0 2024-07-31T14:54:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86d5028640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86d0113690 con 0x7f86d0108ef0 2024-07-31T14:54:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86cf59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86d0109840 0x7f86d019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86cf59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86d0109840 0x7f86d019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52982/0 (socket says 172.21.15.67:52982) 2024-07-31T14:54:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86cf59e640 1 -- 172.21.15.67:0/2275499351 learned_addr learned my addr 172.21.15.67:0/2275499351 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86ced9d640 1 --2- 172.21.15.67:0/2275499351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d01035b0 0x7f86d0199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86ce59c640 1 --2- 172.21.15.67:0/2275499351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86d0108ef0 0x7f86d019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86cf59e640 1 -- 172.21.15.67:0/2275499351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86d0108ef0 msgr2=0x7f86d019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:26.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86cf59e640 1 --2- 172.21.15.67:0/2275499351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86d0108ef0 0x7f86d019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86cf59e640 1 -- 172.21.15.67:0/2275499351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d01035b0 msgr2=0x7f86d0199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:26.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86cf59e640 1 --2- 172.21.15.67:0/2275499351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d01035b0 0x7f86d0199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.922+0000 7f86cf59e640 1 -- 172.21.15.67:0/2275499351 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d019fed0 con 0x7f86d0109840 2024-07-31T14:54:26.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.923+0000 7f86cf59e640 1 --2- 172.21.15.67:0/2275499351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86d0109840 0x7f86d019f7a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f86c800eb10 tx=0x7f86c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:26.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.923+0000 7f86b7fff640 1 -- 172.21.15.67:0/2275499351 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8004b20 con 0x7f86d0109840 2024-07-31T14:54:26.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.923+0000 7f86b7fff640 1 -- 172.21.15.67:0/2275499351 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86c8004cc0 con 0x7f86d0109840 2024-07-31T14:54:26.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.923+0000 7f86d5028640 1 -- 172.21.15.67:0/2275499351 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d01a01c0 con 0x7f86d0109840 2024-07-31T14:54:26.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.923+0000 7f86b7fff640 1 -- 172.21.15.67:0/2275499351 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8010670 con 0x7f86d0109840 2024-07-31T14:54:26.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.923+0000 7f86d5028640 1 -- 172.21.15.67:0/2275499351 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d01a6920 con 0x7f86d0109840 2024-07-31T14:54:26.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.926+0000 7f86db579740 1 Processor -- start 2024-07-31T14:54:26.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.926+0000 7f86b7fff640 1 -- 172.21.15.67:0/2275499351 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f86c8005020 con 0x7f86d0109840 2024-07-31T14:54:26.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.926+0000 7f86db579740 1 -- start start 2024-07-31T14:54:26.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.926+0000 7f86db579740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771ab7070 0x55c771e5c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.926+0000 7f86db579740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a8e1b0 0x55c771e5cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.926+0000 7f86db579740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a96390 0x55c771e5d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.926+0000 7f86db579740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c771dd2bb0 con 0x55c771ab7070 2024-07-31T14:54:26.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.926+0000 7f86db579740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c771dc20b0 con 0x55c771a8e1b0 2024-07-31T14:54:26.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.926+0000 7f86db579740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c771dc60f0 con 0x55c771a96390 2024-07-31T14:54:26.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86ce59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771ab7070 0x55c771e5c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86ce59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771ab7070 0x55c771e5c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52988/0 (socket says 172.21.15.67:52988) 2024-07-31T14:54:26.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86ce59c640 1 -- 172.21.15.67:0/1045305762 learned_addr learned my addr 172.21.15.67:0/1045305762 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:26.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86ced9d640 1 --2- 172.21.15.67:0/1045305762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a8e1b0 0x55c771e5cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86ce59c640 1 --2- 172.21.15.67:0/1045305762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a96390 0x55c771e5d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86b7fff640 1 --2- 172.21.15.67:0/2275499351 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86b80b3d40 0x7f86b80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86ce59c640 1 -- 172.21.15.67:0/1045305762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a96390 msgr2=0x55c771e5d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:26.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86ce59c640 1 --2- 172.21.15.67:0/1045305762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a96390 0x55c771e5d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86ce59c640 1 -- 172.21.15.67:0/1045305762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a8e1b0 msgr2=0x55c771e5cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:26.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86ce59c640 1 --2- 172.21.15.67:0/1045305762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a8e1b0 0x55c771e5cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86ce59c640 1 -- 172.21.15.67:0/1045305762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c771d68b90 con 0x55c771ab7070 2024-07-31T14:54:26.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86cf59e640 1 --2- 172.21.15.67:0/2275499351 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86b80b3d40 0x7f86b80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86b7fff640 1 -- 172.21.15.67:0/2275499351 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f86c80e47d0 con 0x7f86d0109840 2024-07-31T14:54:26.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86ce59c640 1 --2- 172.21.15.67:0/1045305762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771ab7070 0x55c771e5c690 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f86d006c440 tx=0x7f86c400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b790d9022f2f31 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:26.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.927+0000 7f86b57fa640 1 -- 172.21.15.67:0/1045305762 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4004020 con 0x55c771ab7070 2024-07-31T14:54:26.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.928+0000 7f86b57fa640 1 -- 172.21.15.67:0/1045305762 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86c40054d0 con 0x55c771ab7070 2024-07-31T14:54:26.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.928+0000 7f86b57fa640 1 -- 172.21.15.67:0/1045305762 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4004a90 con 0x55c771ab7070 2024-07-31T14:54:26.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.928+0000 7f86db579740 1 -- 172.21.15.67:0/1045305762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771ab7070 msgr2=0x55c771e5c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:26.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.928+0000 7f86cf59e640 1 --2- 172.21.15.67:0/2275499351 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86b80b3d40 0x7f86b80b6250 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f86c80aadd0 tx=0x7f86c80744f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:26.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.928+0000 7f86db579740 1 --2- 172.21.15.67:0/1045305762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771ab7070 0x55c771e5c690 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f86d006c440 tx=0x7f86c400b650 comp rx=0 tx=0).stop 2024-07-31T14:54:26.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.928+0000 7f86db579740 1 -- 172.21.15.67:0/1045305762 shutdown_connections 2024-07-31T14:54:26.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.928+0000 7f86db579740 1 --2- 172.21.15.67:0/1045305762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a96390 0x55c771e5d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.928+0000 7f86db579740 1 --2- 172.21.15.67:0/1045305762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a8e1b0 0x55c771e5cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.928+0000 7f86db579740 1 --2- 172.21.15.67:0/1045305762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771ab7070 0x55c771e5c690 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.929+0000 7f86db579740 1 -- 172.21.15.67:0/1045305762 >> 172.21.15.67:0/1045305762 conn(0x55c771aab460 msgr2=0x55c771e5a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:26.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.929+0000 7f86db579740 1 -- 172.21.15.67:0/1045305762 shutdown_connections 2024-07-31T14:54:26.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.929+0000 7f86db579740 1 -- 172.21.15.67:0/1045305762 wait complete. 2024-07-31T14:54:26.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.929+0000 7f86db579740 1 -- start start 2024-07-31T14:54:26.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.929+0000 7f86db579740 1 Processor -- start 2024-07-31T14:54:26.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.929+0000 7f86db579740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a8e1b0 0x55c771e682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.929+0000 7f86db579740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771a96390 0x55c771e68800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.929+0000 7f86db579740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771ab7070 0x55c771e741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.929+0000 7f86db579740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c771c25700 con 0x55c771a96390 2024-07-31T14:54:26.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.929+0000 7f86db579740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c771d649e0 con 0x55c771a8e1b0 2024-07-31T14:54:26.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.929+0000 7f86db579740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c771e6ae70 con 0x55c771ab7070 2024-07-31T14:54:26.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86db579740 10 client.0 _ll_register_callbacks cb 0x55c771d7f4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:26.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86ced9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771a96390 0x55c771e68800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86ced9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771a96390 0x55c771e68800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53004/0 (socket says 172.21.15.67:53004) 2024-07-31T14:54:26.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86ced9d640 1 -- 172.21.15.67:0/3290549471 learned_addr learned my addr 172.21.15.67:0/3290549471 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:26.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86ce59c640 1 --2- 172.21.15.67:0/3290549471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a8e1b0 0x55c771e682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86ce59c640 1 --2- 172.21.15.67:0/3290549471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771ab7070 0x55c771e741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86ced9d640 1 -- 172.21.15.67:0/3290549471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771ab7070 msgr2=0x55c771e741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:26.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86ced9d640 1 --2- 172.21.15.67:0/3290549471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771ab7070 0x55c771e741c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86ced9d640 1 -- 172.21.15.67:0/3290549471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a8e1b0 msgr2=0x55c771e682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86ced9d640 1 --2- 172.21.15.67:0/3290549471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a8e1b0 0x55c771e682c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86ced9d640 1 -- 172.21.15.67:0/3290549471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c771b22650 con 0x55c771a96390 2024-07-31T14:54:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86ce59c640 1 --2- 172.21.15.67:0/3290549471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a8e1b0 0x55c771e682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:54:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86ced9d640 1 --2- 172.21.15.67:0/3290549471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771a96390 0x55c771e68800 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009900 tx=0x7f86c00316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49aa07b74d1a320e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86b2ff5640 1 -- 172.21.15.67:0/3290549471 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0035040 con 0x55c771a96390 2024-07-31T14:54:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86db579740 1 -- 172.21.15.67:0/3290549471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86c40054d0 con 0x55c771a96390 2024-07-31T14:54:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.930+0000 7f86b2ff5640 1 -- 172.21.15.67:0/3290549471 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f86c0004040 con 0x55c771a96390 2024-07-31T14:54:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.931+0000 7f86b2ff5640 1 -- 172.21.15.67:0/3290549471 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f86c004f050 con 0x55c771a96390 2024-07-31T14:54:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.931+0000 7f86db579740 10 client.16548 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.931+0000 7f86db579740 1 -- 172.21.15.67:0/3290549471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c771e76f70 con 0x55c771a96390 2024-07-31T14:54:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.931+0000 7f86b2ff5640 1 -- 172.21.15.67:0/3290549471 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86c40054d0 con 0x55c771a96390 2024-07-31T14:54:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.931+0000 7f86b2ff5640 1 -- 172.21.15.67:0/3290549471 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f86c0004750 con 0x55c771a96390 2024-07-31T14:54:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.931+0000 7f86db579740 10 client.16548 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.931+0000 7f86db579740 10 client.16548 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:54:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.931+0000 7f86db579740 20 client.16548 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.931+0000 7f86db579740 1 --2- 172.21.15.67:0/3290549471 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55c771a759f0 0x55c771e7a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.931+0000 7f86db579740 4 client.16548 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:54:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.931+0000 7f86db579740 1 -- 172.21.15.67:0/3290549471 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c771dd4b00 con 0x55c771a759f0 2024-07-31T14:54:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.931+0000 7f86ce59c640 1 --2- 172.21.15.67:0/3290549471 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55c771a759f0 0x55c771e7a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.932+0000 7f86ce59c640 1 --2- 172.21.15.67:0/3290549471 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55c771a759f0 0x55c771e7a590 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a356c75e9f218a5 server_cookie=c1ceb0964a1f24a5 in_seq=0 out_seq=0 2024-07-31T14:54:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.932+0000 7f86b2ff5640 10 client.16548 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:54:26.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86b2ff5640 1 -- 172.21.15.67:0/3290549471 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c771dd4b00 con 0x55c771a759f0 2024-07-31T14:54:26.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86b2ff5640 10 client.16548 handle_command_reply: tid=0 2024-07-31T14:54:26.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86db579740 1 client.16548 shutdown 2024-07-31T14:54:26.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86db579740 1 -- 172.21.15.67:0/3290549471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771a96390 msgr2=0x55c771e68800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:26.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86db579740 1 --2- 172.21.15.67:0/3290549471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771a96390 0x55c771e68800 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009900 tx=0x7f86c00316d0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86db579740 1 -- 172.21.15.67:0/3290549471 shutdown_connections 2024-07-31T14:54:26.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86db579740 1 --2- 172.21.15.67:0/3290549471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771ab7070 0x55c771e741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86db579740 1 --2- 172.21.15.67:0/3290549471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771a96390 0x55c771e68800 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86db579740 1 --2- 172.21.15.67:0/3290549471 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55c771a759f0 0x55c771e7a590 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86db579740 1 --2- 172.21.15.67:0/3290549471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a8e1b0 0x55c771e682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86db579740 1 -- 172.21.15.67:0/3290549471 >> 172.21.15.67:0/3290549471 conn(0x55c771aab460 msgr2=0x55c771e5f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:26.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86b2ff5640 0 client.16548 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:54:26.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86db579740 1 -- 172.21.15.67:0/3290549471 shutdown_connections 2024-07-31T14:54:26.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86db579740 1 -- 172.21.15.67:0/3290549471 wait complete. 2024-07-31T14:54:26.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.933+0000 7f86db579740 20 client.16548 trim_cache size 0 max 16384 2024-07-31T14:54:26.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.947+0000 7f86db579740 1 Processor -- start 2024-07-31T14:54:26.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.947+0000 7f86db579740 1 -- start start 2024-07-31T14:54:26.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.947+0000 7f86db579740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a8e1b0 0x55c771e5fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.947+0000 7f86db579740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a96390 0x55c771e73fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.947+0000 7f86db579740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771ab7070 0x55c771e69cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.947+0000 7f86db579740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01b0690 con 0x55c771ab7070 2024-07-31T14:54:26.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.947+0000 7f86db579740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01b2350 con 0x55c771a96390 2024-07-31T14:54:26.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.947+0000 7f86db579740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01b0d60 con 0x55c771a8e1b0 2024-07-31T14:54:26.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.947+0000 7f86ce59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771ab7070 0x55c771e69cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.947+0000 7f86ce59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771ab7070 0x55c771e69cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53020/0 (socket says 172.21.15.67:53020) 2024-07-31T14:54:26.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.947+0000 7f86ce59c640 1 -- 172.21.15.67:0/4127447691 learned_addr learned my addr 172.21.15.67:0/4127447691 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:26.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.948+0000 7f86ce59c640 1 --2- 172.21.15.67:0/4127447691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a8e1b0 0x55c771e5fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.948+0000 7f86ced9d640 1 --2- 172.21.15.67:0/4127447691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a96390 0x55c771e73fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.948+0000 7f86ce59c640 1 -- 172.21.15.67:0/4127447691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a8e1b0 msgr2=0x55c771e5fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:26.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.948+0000 7f86ce59c640 1 --2- 172.21.15.67:0/4127447691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a8e1b0 0x55c771e5fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.948+0000 7f86ce59c640 1 -- 172.21.15.67:0/4127447691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a96390 msgr2=0x55c771e73fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:26.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.948+0000 7f86ce59c640 1 --2- 172.21.15.67:0/4127447691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a96390 0x55c771e73fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.948+0000 7f86ce59c640 1 -- 172.21.15.67:0/4127447691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c771e5d4c0 con 0x55c771ab7070 2024-07-31T14:54:26.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.948+0000 7f86ce59c640 1 --2- 172.21.15.67:0/4127447691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771ab7070 0x55c771e69cf0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f86c400be90 tx=0x7f86c40044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e547c42a021ae server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:26.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.948+0000 7f86b4ff9640 1 -- 172.21.15.67:0/4127447691 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4027040 con 0x55c771ab7070 2024-07-31T14:54:26.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.948+0000 7f86b4ff9640 1 -- 172.21.15.67:0/4127447691 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86c4012200 con 0x55c771ab7070 2024-07-31T14:54:26.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.948+0000 7f86b4ff9640 1 -- 172.21.15.67:0/4127447691 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c402d620 con 0x55c771ab7070 2024-07-31T14:54:26.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.949+0000 7f86db579740 1 -- 172.21.15.67:0/4127447691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771ab7070 msgr2=0x55c771e69cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:26.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.949+0000 7f86db579740 1 --2- 172.21.15.67:0/4127447691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771ab7070 0x55c771e69cf0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f86c400be90 tx=0x7f86c40044e0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.949+0000 7f86db579740 1 -- 172.21.15.67:0/4127447691 shutdown_connections 2024-07-31T14:54:26.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.949+0000 7f86db579740 1 --2- 172.21.15.67:0/4127447691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771ab7070 0x55c771e69cf0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.949+0000 7f86db579740 1 --2- 172.21.15.67:0/4127447691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771a96390 0x55c771e73fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.949+0000 7f86db579740 1 --2- 172.21.15.67:0/4127447691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a8e1b0 0x55c771e5fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.949+0000 7f86db579740 1 -- 172.21.15.67:0/4127447691 >> 172.21.15.67:0/4127447691 conn(0x55c771aab460 msgr2=0x55c771e695e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:26.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.949+0000 7f86db579740 1 -- 172.21.15.67:0/4127447691 shutdown_connections 2024-07-31T14:54:26.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.949+0000 7f86db579740 1 -- 172.21.15.67:0/4127447691 wait complete. 2024-07-31T14:54:26.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.950+0000 7f86db579740 1 -- start start 2024-07-31T14:54:26.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.950+0000 7f86db579740 1 Processor -- start 2024-07-31T14:54:26.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.950+0000 7f86db579740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a8e1b0 0x55c771edb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.950+0000 7f86db579740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771a96390 0x55c771edb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.950+0000 7f86db579740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771ab7070 0x55c771edbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.950+0000 7f86db579740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01b1c80 con 0x55c771a96390 2024-07-31T14:54:26.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.950+0000 7f86db579740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01b14f0 con 0x55c771ab7070 2024-07-31T14:54:26.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.950+0000 7f86db579740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01ae670 con 0x55c771a8e1b0 2024-07-31T14:54:26.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.950+0000 7f86db579740 10 client.0 _ll_register_callbacks cb 0x7f86c0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:26.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.950+0000 7f86ced9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771a96390 0x55c771edb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.950+0000 7f86ced9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771a96390 0x55c771edb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53024/0 (socket says 172.21.15.67:53024) 2024-07-31T14:54:26.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.950+0000 7f86ced9d640 1 -- 172.21.15.67:0/2864608069 learned_addr learned my addr 172.21.15.67:0/2864608069 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:26.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.951+0000 7f86ce59c640 1 --2- 172.21.15.67:0/2864608069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771ab7070 0x55c771edbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.951+0000 7f86ce59c640 1 --2- 172.21.15.67:0/2864608069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a8e1b0 0x55c771edb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.951+0000 7f86ced9d640 1 -- 172.21.15.67:0/2864608069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a8e1b0 msgr2=0x55c771edb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:26.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.951+0000 7f86ced9d640 1 --2- 172.21.15.67:0/2864608069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a8e1b0 0x55c771edb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.951+0000 7f86ced9d640 1 -- 172.21.15.67:0/2864608069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771ab7070 msgr2=0x55c771edbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:26.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.951+0000 7f86ced9d640 1 --2- 172.21.15.67:0/2864608069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771ab7070 0x55c771edbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.951+0000 7f86ced9d640 1 -- 172.21.15.67:0/2864608069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c771e6f110 con 0x55c771a96390 2024-07-31T14:54:26.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.951+0000 7f86ce59c640 1 --2- 172.21.15.67:0/2864608069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a8e1b0 0x55c771edb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:54:26.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.951+0000 7f86ced9d640 1 --2- 172.21.15.67:0/2864608069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771a96390 0x55c771edb810 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f86c0004b50 tx=0x7f86c00319e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f299c2bed1ae0e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:26.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.951+0000 7f86b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:26.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.951+0000 7f86b2ff5640 1 -- 172.21.15.67:0/2864608069 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0035070 con 0x55c771a96390 2024-07-31T14:54:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.951+0000 7f86b2ff5640 1 -- 172.21.15.67:0/2864608069 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f86c0032b90 con 0x55c771a96390 2024-07-31T14:54:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.951+0000 7f86db579740 1 -- 172.21.15.67:0/2864608069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86c4012200 con 0x55c771a96390 2024-07-31T14:54:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.952+0000 7f86b2ff5640 1 -- 172.21.15.67:0/2864608069 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f86c0061050 con 0x55c771a96390 2024-07-31T14:54:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.952+0000 7f86db579740 10 client.16554 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.952+0000 7f86db579740 1 -- 172.21.15.67:0/2864608069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c771e703c0 con 0x55c771a96390 2024-07-31T14:54:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.952+0000 7f86b2ff5640 1 -- 172.21.15.67:0/2864608069 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86c4012200 con 0x55c771a96390 2024-07-31T14:54:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.952+0000 7f86b2ff5640 1 -- 172.21.15.67:0/2864608069 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f86c0046470 con 0x55c771a96390 2024-07-31T14:54:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.952+0000 7f86db579740 10 client.16554 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.952+0000 7f86db579740 10 client.16554 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:54:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.952+0000 7f86db579740 20 client.16554 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.952+0000 7f86db579740 1 --2- 172.21.15.67:0/2864608069 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55c771a4ada0 0x55c771eed930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.952+0000 7f86db579740 4 client.16554 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:54:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.952+0000 7f86db579740 1 -- 172.21.15.67:0/2864608069 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c771dd05a0 con 0x55c771a4ada0 2024-07-31T14:54:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.952+0000 7f86ce59c640 1 --2- 172.21.15.67:0/2864608069 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55c771a4ada0 0x55c771eed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.953+0000 7f86b2ff5640 10 client.16554 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:54:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.953+0000 7f86ce59c640 1 --2- 172.21.15.67:0/2864608069 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55c771a4ada0 0x55c771eed930 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f56e27a9d0274307 server_cookie=d819f70840ee301d in_seq=0 out_seq=0 2024-07-31T14:54:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.953+0000 7f86b2ff5640 1 -- 172.21.15.67:0/2864608069 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c771dd05a0 con 0x55c771a4ada0 2024-07-31T14:54:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.953+0000 7f86b2ff5640 10 client.16554 handle_command_reply: tid=0 2024-07-31T14:54:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.953+0000 7f86db579740 1 client.16554 shutdown 2024-07-31T14:54:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.953+0000 7f86db579740 1 -- 172.21.15.67:0/2864608069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771a96390 msgr2=0x55c771edb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.953+0000 7f86db579740 1 --2- 172.21.15.67:0/2864608069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771a96390 0x55c771edb810 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f86c0004b50 tx=0x7f86c00319e0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.953+0000 7f86db579740 1 -- 172.21.15.67:0/2864608069 shutdown_connections 2024-07-31T14:54:26.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.953+0000 7f86db579740 1 --2- 172.21.15.67:0/2864608069 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55c771a4ada0 0x55c771eed930 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.953+0000 7f86db579740 1 --2- 172.21.15.67:0/2864608069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c771ab7070 0x55c771edbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.953+0000 7f86db579740 1 --2- 172.21.15.67:0/2864608069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c771a96390 0x55c771edb810 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.953+0000 7f86db579740 1 --2- 172.21.15.67:0/2864608069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c771a8e1b0 0x55c771edb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.953+0000 7f86db579740 1 -- 172.21.15.67:0/2864608069 >> 172.21.15.67:0/2864608069 conn(0x55c771aab460 msgr2=0x55c771e5d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:26.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.953+0000 7f86b2ff5640 0 client.16554 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:54:26.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.954+0000 7f86db579740 1 -- 172.21.15.67:0/2864608069 shutdown_connections 2024-07-31T14:54:26.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.954+0000 7f86db579740 1 -- 172.21.15.67:0/2864608069 wait complete. 2024-07-31T14:54:26.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.954+0000 7f86db579740 20 client.16554 trim_cache size 0 max 16384 2024-07-31T14:54:26.961 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:54:26.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.954+0000 7f86d5028640 1 -- 172.21.15.67:0/2275499351 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86b80b3d40 msgr2=0x7f86b80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:26.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.954+0000 7f86d5028640 1 --2- 172.21.15.67:0/2275499351 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86b80b3d40 0x7f86b80b6250 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f86c80aadd0 tx=0x7f86c80744f0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.954+0000 7f86d5028640 1 -- 172.21.15.67:0/2275499351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86d0109840 msgr2=0x7f86d019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:26.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.954+0000 7f86d5028640 1 --2- 172.21.15.67:0/2275499351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86d0109840 0x7f86d019f7a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f86c800eb10 tx=0x7f86c800c2d0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.954+0000 7f86d5028640 1 -- 172.21.15.67:0/2275499351 shutdown_connections 2024-07-31T14:54:26.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.954+0000 7f86d5028640 1 --2- 172.21.15.67:0/2275499351 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86b80b3d40 0x7f86b80b6250 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.954+0000 7f86d5028640 1 --2- 172.21.15.67:0/2275499351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86d0109840 0x7f86d019f7a0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.954+0000 7f86d5028640 1 --2- 172.21.15.67:0/2275499351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86d0108ef0 0x7f86d019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.954+0000 7f86d5028640 1 --2- 172.21.15.67:0/2275499351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d01035b0 0x7f86d0199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:26.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.954+0000 7f86d5028640 1 -- 172.21.15.67:0/2275499351 >> 172.21.15.67:0/2275499351 conn(0x7f86d00ff000 msgr2=0x7f86d0101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:26.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.955+0000 7f86d5028640 1 -- 172.21.15.67:0/2275499351 shutdown_connections 2024-07-31T14:54:26.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:26.955+0000 7f86d5028640 1 -- 172.21.15.67:0/2275499351 wait complete. 2024-07-31T14:54:27.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:27 smithi067 ceph-mon[32750]: pgmap v501: 129 pgs: 129 active+clean; 339 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 710 op/s 2024-07-31T14:54:27.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:27 smithi073 ceph-mon[36714]: pgmap v501: 129 pgs: 129 active+clean; 339 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 710 op/s 2024-07-31T14:54:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:27 smithi077 ceph-mon[37327]: pgmap v501: 129 pgs: 129 active+clean; 339 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 710 op/s 2024-07-31T14:54:27.979 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T14:54:28.255 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:54:28.620 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:54:29.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:29 smithi073 ceph-mon[36714]: pgmap v502: 129 pgs: 129 active+clean; 342 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 729 op/s 2024-07-31T14:54:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:29 smithi077 ceph-mon[37327]: pgmap v502: 129 pgs: 129 active+clean; 342 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 729 op/s 2024-07-31T14:54:29.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:29 smithi067 ceph-mon[32750]: pgmap v502: 129 pgs: 129 active+clean; 342 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 729 op/s 2024-07-31T14:54:30.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.171+0000 7f7c9f1af640 1 -- 172.21.15.67:0/1255653566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c98108720 msgr2=0x7f7c9810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:30.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.171+0000 7f7c9f1af640 1 --2- 172.21.15.67:0/1255653566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c98108720 0x7f7c9810ef50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009ca0 tx=0x7f7c9002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.171+0000 7f7c9f1af640 1 -- 172.21.15.67:0/1255653566 shutdown_connections 2024-07-31T14:54:30.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.171+0000 7f7c9f1af640 1 --2- 172.21.15.67:0/1255653566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c98108720 0x7f7c9810ef50 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.171+0000 7f7c9f1af640 1 --2- 172.21.15.67:0/1255653566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c98107dd0 0x7f7c981081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.171+0000 7f7c9f1af640 1 --2- 172.21.15.67:0/1255653566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c98102490 0x7f7c981028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.171+0000 7f7c9f1af640 1 -- 172.21.15.67:0/1255653566 >> 172.21.15.67:0/1255653566 conn(0x7f7c98079160 msgr2=0x7f7c98079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:30.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.171+0000 7f7c9f1af640 1 -- 172.21.15.67:0/1255653566 shutdown_connections 2024-07-31T14:54:30.173 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:54:30.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.171+0000 7f7c9f1af640 1 -- 172.21.15.67:0/1255653566 wait complete. 2024-07-31T14:54:30.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.172+0000 7f7c9f1af640 1 Processor -- start 2024-07-31T14:54:30.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.172+0000 7f7c9f1af640 1 -- start start 2024-07-31T14:54:30.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.172+0000 7f7c9f1af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c98102490 0x7f7c98199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.172+0000 7f7c9f1af640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c98107dd0 0x7f7c9819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.172+0000 7f7c9f1af640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c98108720 0x7f7c9819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.172+0000 7f7c9f1af640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c98112380 con 0x7f7c98108720 2024-07-31T14:54:30.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.172+0000 7f7c9f1af640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7c98112200 con 0x7f7c98107dd0 2024-07-31T14:54:30.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.172+0000 7f7c9f1af640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c98112500 con 0x7f7c98102490 2024-07-31T14:54:30.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.172+0000 7f7c9d74c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c98108720 0x7f7c9819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.172+0000 7f7c9cf4b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c98102490 0x7f7c98199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.173+0000 7f7c9d74c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c98108720 0x7f7c9819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53046/0 (socket says 172.21.15.67:53046) 2024-07-31T14:54:30.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.173+0000 7f7c9d74c640 1 -- 172.21.15.67:0/3090281414 learned_addr learned my addr 172.21.15.67:0/3090281414 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:30.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.173+0000 7f7c97fff640 1 --2- 172.21.15.67:0/3090281414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c98107dd0 0x7f7c9819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.173+0000 7f7c9cf4b640 1 -- 172.21.15.67:0/3090281414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c98107dd0 msgr2=0x7f7c9819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:30.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.173+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/3090281414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c98107dd0 0x7f7c9819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.173+0000 7f7c9cf4b640 1 -- 172.21.15.67:0/3090281414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c98108720 msgr2=0x7f7c9819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:30.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.173+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/3090281414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c98108720 0x7f7c9819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.173+0000 7f7c9cf4b640 1 -- 172.21.15.67:0/3090281414 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9819fd70 con 0x7f7c98102490 2024-07-31T14:54:30.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.173+0000 7f7c9d74c640 1 --2- 172.21.15.67:0/3090281414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c98108720 0x7f7c9819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:30.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.173+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/3090281414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c98102490 0x7f7c98199ec0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00d6d0 tx=0x7f7c8c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:30.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.174+0000 7f7c95ffb640 1 -- 172.21.15.67:0/3090281414 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c017070 con 0x7f7c98102490 2024-07-31T14:54:30.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.174+0000 7f7c95ffb640 1 -- 172.21.15.67:0/3090281414 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c8c009820 con 0x7f7c98102490 2024-07-31T14:54:30.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.174+0000 7f7c9f1af640 1 -- 172.21.15.67:0/3090281414 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c981a0060 con 0x7f7c98102490 2024-07-31T14:54:30.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.174+0000 7f7c95ffb640 1 -- 172.21.15.67:0/3090281414 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c004c40 con 0x7f7c98102490 2024-07-31T14:54:30.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.174+0000 7f7c9f1af640 1 -- 172.21.15.67:0/3090281414 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c980fef80 con 0x7f7c98102490 2024-07-31T14:54:30.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.177+0000 7f7c95ffb640 1 -- 172.21.15.67:0/3090281414 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7c8c005220 con 0x7f7c98102490 2024-07-31T14:54:30.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.177+0000 7f7ca5700740 1 Processor -- start 2024-07-31T14:54:30.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.177+0000 7f7ca5700740 1 -- start start 2024-07-31T14:54:30.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.177+0000 7f7ca5700740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542391070 0x55f542736570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.177+0000 7f7ca5700740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f542736ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.177+0000 7f7ca5700740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542370390 0x55f542736ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.177+0000 7f7ca5700740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f5426acbb0 con 0x55f542391070 2024-07-31T14:54:30.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.177+0000 7f7ca5700740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f54269c0b0 con 0x55f542370390 2024-07-31T14:54:30.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.178+0000 7f7ca5700740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f5426a00f0 con 0x55f5423681b0 2024-07-31T14:54:30.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.178+0000 7f7c9d74c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542391070 0x55f542736570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.178+0000 7f7c9d74c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542391070 0x55f542736570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53060/0 (socket says 172.21.15.67:53060) 2024-07-31T14:54:30.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.178+0000 7f7c9d74c640 1 -- 172.21.15.67:0/783097349 learned_addr learned my addr 172.21.15.67:0/783097349 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:30.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.178+0000 7f7c95ffb640 1 --2- 172.21.15.67:0/3090281414 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c880b3d70 0x7f7c880b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.178+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/783097349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f542736ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.178+0000 7f7c97fff640 1 --2- 172.21.15.67:0/783097349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542370390 0x55f542736ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.178+0000 7f7c9d74c640 1 --2- 172.21.15.67:0/3090281414 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c880b3d70 0x7f7c880b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.178+0000 7f7c95ffb640 1 -- 172.21.15.67:0/3090281414 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f7c8c0e3df0 con 0x7f7c98102490 2024-07-31T14:54:30.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.178+0000 7f7c9cf4b640 1 -- 172.21.15.67:0/783097349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542370390 msgr2=0x55f542736ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:30.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.178+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/783097349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542370390 0x55f542736ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.178+0000 7f7c9cf4b640 1 -- 172.21.15.67:0/783097349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542391070 msgr2=0x55f542736570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:30.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.178+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/783097349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542391070 0x55f542736570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.178+0000 7f7c9cf4b640 1 -- 172.21.15.67:0/783097349 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f542642b90 con 0x55f5423681b0 2024-07-31T14:54:30.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.179+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/783097349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f542736ab0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0e1230 tx=0x7f7c8c0da3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb92ed6467f6e0bd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:30.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.179+0000 7f7c9d74c640 1 --2- 172.21.15.67:0/3090281414 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c880b3d70 0x7f7c880b6280 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7c9002f500 tx=0x7f7c9002a8c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:30.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.179+0000 7f7c82ffd640 1 -- 172.21.15.67:0/783097349 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c020020 con 0x55f5423681b0 2024-07-31T14:54:30.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.179+0000 7f7c82ffd640 1 -- 172.21.15.67:0/783097349 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c8c0aa200 con 0x55f5423681b0 2024-07-31T14:54:30.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.179+0000 7f7c82ffd640 1 -- 172.21.15.67:0/783097349 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c009da0 con 0x55f5423681b0 2024-07-31T14:54:30.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.180+0000 7f7ca5700740 1 -- 172.21.15.67:0/783097349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 msgr2=0x55f542736ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:30.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.180+0000 7f7ca5700740 1 --2- 172.21.15.67:0/783097349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f542736ab0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0e1230 tx=0x7f7c8c0da3d0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.180+0000 7f7ca5700740 1 -- 172.21.15.67:0/783097349 shutdown_connections 2024-07-31T14:54:30.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.180+0000 7f7ca5700740 1 --2- 172.21.15.67:0/783097349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542370390 0x55f542736ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.180+0000 7f7ca5700740 1 --2- 172.21.15.67:0/783097349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f542736ab0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.180+0000 7f7ca5700740 1 --2- 172.21.15.67:0/783097349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542391070 0x55f542736570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.180+0000 7f7ca5700740 1 -- 172.21.15.67:0/783097349 >> 172.21.15.67:0/783097349 conn(0x55f542385460 msgr2=0x55f542734460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:30.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.180+0000 7f7ca5700740 1 -- 172.21.15.67:0/783097349 shutdown_connections 2024-07-31T14:54:30.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.180+0000 7f7ca5700740 1 -- 172.21.15.67:0/783097349 wait complete. 2024-07-31T14:54:30.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.180+0000 7f7ca5700740 1 -- start start 2024-07-31T14:54:30.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.181+0000 7f7ca5700740 1 Processor -- start 2024-07-31T14:54:30.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.181+0000 7f7ca5700740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f5427421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.181+0000 7f7ca5700740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542370390 0x55f542742780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.181+0000 7f7ca5700740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542391070 0x55f54274df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.181+0000 7f7ca5700740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f5424ff700 con 0x55f542391070 2024-07-31T14:54:30.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.181+0000 7f7ca5700740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f54263e9e0 con 0x55f542370390 2024-07-31T14:54:30.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.181+0000 7f7ca5700740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f542744c90 con 0x55f5423681b0 2024-07-31T14:54:30.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.181+0000 7f7ca5700740 10 client.0 _ll_register_callbacks cb 0x55f5426594f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:30.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.181+0000 7f7c9d74c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542391070 0x55f54274df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.181+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542370390 0x55f542742780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.181+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542370390 0x55f542742780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35982/0 (socket says 172.21.15.67:35982) 2024-07-31T14:54:30.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.181+0000 7f7c9cf4b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f5427421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.181+0000 7f7c97fff640 1 -- 172.21.15.67:0/263596906 learned_addr learned my addr 172.21.15.67:0/263596906 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:30.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.181+0000 7f7c9d74c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542391070 0x55f54274df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53072/0 (socket says 172.21.15.67:53072) 2024-07-31T14:54:30.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.182+0000 7f7c9cf4b640 1 -- 172.21.15.67:0/263596906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542370390 msgr2=0x55f542742780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:30.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.182+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/263596906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542370390 0x55f542742780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.182+0000 7f7c9cf4b640 1 -- 172.21.15.67:0/263596906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542391070 msgr2=0x55f54274df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:30.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.182+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/263596906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542391070 0x55f54274df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.182+0000 7f7c9cf4b640 1 -- 172.21.15.67:0/263596906 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5423fc650 con 0x55f5423681b0 2024-07-31T14:54:30.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.182+0000 7f7c97fff640 1 --2- 172.21.15.67:0/263596906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542370390 0x55f542742780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:30.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.182+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/263596906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f5427421c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0a9fd0 tx=0x7f7c8c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8875b9bbb046ba1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:30.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.182+0000 7f7c77fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:54:30.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.182+0000 7f7c77fff640 1 -- 172.21.15.67:0/263596906 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c020070 con 0x55f5423681b0 2024-07-31T14:54:30.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.182+0000 7f7ca5700740 1 -- 172.21.15.67:0/263596906 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c8c0aa200 con 0x55f5423681b0 2024-07-31T14:54:30.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.182+0000 7f7c77fff640 1 -- 172.21.15.67:0/263596906 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f7c8c0a6980 con 0x55f5423681b0 2024-07-31T14:54:30.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.183+0000 7f7c77fff640 1 -- 172.21.15.67:0/263596906 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7c8c06e050 con 0x55f5423681b0 2024-07-31T14:54:30.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.183+0000 7f7ca5700740 10 client.26215 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:30.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.183+0000 7f7ca5700740 1 -- 172.21.15.67:0/263596906 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f542750d30 con 0x55f5423681b0 2024-07-31T14:54:30.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.183+0000 7f7c77fff640 1 -- 172.21.15.67:0/263596906 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c8c0aa200 con 0x55f5423681b0 2024-07-31T14:54:30.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.183+0000 7f7c77fff640 1 -- 172.21.15.67:0/263596906 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f7c8c06f240 con 0x55f5423681b0 2024-07-31T14:54:30.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.183+0000 7f7ca5700740 10 client.26215 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:30.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.183+0000 7f7ca5700740 10 client.26215 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:54:30.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.183+0000 7f7ca5700740 20 client.26215 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:30.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.183+0000 7f7ca5700740 1 --2- 172.21.15.67:0/263596906 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f54234f9f0 0x55f542754090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.183+0000 7f7ca5700740 4 client.26215 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:54:30.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.183+0000 7f7ca5700740 1 -- 172.21.15.67:0/263596906 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5426aeb00 con 0x55f54234f9f0 2024-07-31T14:54:30.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.183+0000 7f7c97fff640 1 --2- 172.21.15.67:0/263596906 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f54234f9f0 0x55f542754090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.184+0000 7f7c77fff640 10 client.26215 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:54:30.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.184+0000 7f7c97fff640 1 --2- 172.21.15.67:0/263596906 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f54234f9f0 0x55f542754090 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35891fd5a4ee2e6d server_cookie=b00d100112f9e510 in_seq=0 out_seq=0 2024-07-31T14:54:30.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7c77fff640 1 -- 172.21.15.67:0/263596906 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5426aeb00 con 0x55f54234f9f0 2024-07-31T14:54:30.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7c77fff640 10 client.26215 handle_command_reply: tid=0 2024-07-31T14:54:30.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7ca5700740 1 client.26215 shutdown 2024-07-31T14:54:30.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7ca5700740 1 -- 172.21.15.67:0/263596906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 msgr2=0x55f5427421c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:30.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7ca5700740 1 --2- 172.21.15.67:0/263596906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f5427421c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0a9fd0 tx=0x7f7c8c009c00 comp rx=0 tx=0).stop 2024-07-31T14:54:30.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7ca5700740 1 -- 172.21.15.67:0/263596906 shutdown_connections 2024-07-31T14:54:30.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7ca5700740 1 --2- 172.21.15.67:0/263596906 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f54234f9f0 0x55f542754090 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7ca5700740 1 --2- 172.21.15.67:0/263596906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542391070 0x55f54274df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7ca5700740 1 --2- 172.21.15.67:0/263596906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542370390 0x55f542742780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7c77fff640 0 client.26215 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:54:30.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7ca5700740 1 --2- 172.21.15.67:0/263596906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f5427421c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7ca5700740 1 -- 172.21.15.67:0/263596906 >> 172.21.15.67:0/263596906 conn(0x55f542385460 msgr2=0x55f542739750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:30.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7ca5700740 1 -- 172.21.15.67:0/263596906 shutdown_connections 2024-07-31T14:54:30.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7ca5700740 1 -- 172.21.15.67:0/263596906 wait complete. 2024-07-31T14:54:30.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.185+0000 7f7ca5700740 20 client.26215 trim_cache size 0 max 16384 2024-07-31T14:54:30.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.206+0000 7f7ca5700740 1 Processor -- start 2024-07-31T14:54:30.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.206+0000 7f7ca5700740 1 -- start start 2024-07-31T14:54:30.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.206+0000 7f7ca5700740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f542734560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.206+0000 7f7ca5700740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542370390 0x55f54274def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.206+0000 7f7ca5700740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542391070 0x55f54273b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.206+0000 7f7ca5700740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c981b28e0 con 0x55f542370390 2024-07-31T14:54:30.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.206+0000 7f7ca5700740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7c981b45a0 con 0x55f542391070 2024-07-31T14:54:30.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.206+0000 7f7ca5700740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c981b2fb0 con 0x55f5423681b0 2024-07-31T14:54:30.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.207+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542370390 0x55f54274def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.207+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542370390 0x55f54274def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53080/0 (socket says 172.21.15.67:53080) 2024-07-31T14:54:30.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.207+0000 7f7c97fff640 1 -- 172.21.15.67:0/1299464090 learned_addr learned my addr 172.21.15.67:0/1299464090 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:30.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.207+0000 7f7c9d74c640 1 --2- 172.21.15.67:0/1299464090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542391070 0x55f54273b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.207+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/1299464090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f542734560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.207+0000 7f7c9d74c640 1 -- 172.21.15.67:0/1299464090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 msgr2=0x55f542734560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:30.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.207+0000 7f7c9d74c640 1 --2- 172.21.15.67:0/1299464090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f542734560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.207+0000 7f7c9d74c640 1 -- 172.21.15.67:0/1299464090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542370390 msgr2=0x55f54274def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:30.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.207+0000 7f7c9d74c640 1 --2- 172.21.15.67:0/1299464090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542370390 0x55f54274def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.207+0000 7f7c9d74c640 1 -- 172.21.15.67:0/1299464090 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f54273d850 con 0x55f542391070 2024-07-31T14:54:30.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.207+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/1299464090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f542734560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:30.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.207+0000 7f7c97fff640 1 --2- 172.21.15.67:0/1299464090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542370390 0x55f54274def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:30.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.207+0000 7f7c9d74c640 1 --2- 172.21.15.67:0/1299464090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542391070 0x55f54273b650 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c900094e0 tx=0x7f7c90032ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8900ce9203f2c447 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:30.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.208+0000 7f7c827fc640 1 -- 172.21.15.67:0/1299464090 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c900043c0 con 0x55f542391070 2024-07-31T14:54:30.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.208+0000 7f7c827fc640 1 -- 172.21.15.67:0/1299464090 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c90047040 con 0x55f542391070 2024-07-31T14:54:30.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.208+0000 7f7c827fc640 1 -- 172.21.15.67:0/1299464090 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c900305c0 con 0x55f542391070 2024-07-31T14:54:30.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.209+0000 7f7ca5700740 1 -- 172.21.15.67:0/1299464090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542391070 msgr2=0x55f54273b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:30.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.209+0000 7f7ca5700740 1 --2- 172.21.15.67:0/1299464090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542391070 0x55f54273b650 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c900094e0 tx=0x7f7c90032ed0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.209+0000 7f7ca5700740 1 -- 172.21.15.67:0/1299464090 shutdown_connections 2024-07-31T14:54:30.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.209+0000 7f7ca5700740 1 --2- 172.21.15.67:0/1299464090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542391070 0x55f54273b650 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.209+0000 7f7ca5700740 1 --2- 172.21.15.67:0/1299464090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542370390 0x55f54274def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.209+0000 7f7ca5700740 1 --2- 172.21.15.67:0/1299464090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f542734560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.209+0000 7f7ca5700740 1 -- 172.21.15.67:0/1299464090 >> 172.21.15.67:0/1299464090 conn(0x55f542385460 msgr2=0x55f542733e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:30.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.209+0000 7f7ca5700740 1 -- 172.21.15.67:0/1299464090 shutdown_connections 2024-07-31T14:54:30.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.209+0000 7f7ca5700740 1 -- 172.21.15.67:0/1299464090 wait complete. 2024-07-31T14:54:30.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.210+0000 7f7ca5700740 1 -- start start 2024-07-31T14:54:30.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.210+0000 7f7ca5700740 1 Processor -- start 2024-07-31T14:54:30.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.210+0000 7f7ca5700740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f5427579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7ca5700740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542370390 0x55f542752ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7ca5700740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542391070 0x55f5427535b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7ca5700740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c981b3ed0 con 0x55f542370390 2024-07-31T14:54:30.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7ca5700740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7c981b3740 con 0x55f542391070 2024-07-31T14:54:30.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7ca5700740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c981b08c0 con 0x55f5423681b0 2024-07-31T14:54:30.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7ca5700740 10 client.0 _ll_register_callbacks cb 0x7f7c90009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:30.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7c9cf4b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f5427579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542370390 0x55f542752ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7c9d74c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542391070 0x55f5427535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542370390 0x55f542752ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53084/0 (socket says 172.21.15.67:53084) 2024-07-31T14:54:30.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7c9cf4b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f5427579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:39964/0 (socket says 172.21.15.67:39964) 2024-07-31T14:54:30.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7c97fff640 1 -- 172.21.15.67:0/38538877 learned_addr learned my addr 172.21.15.67:0/38538877 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:30.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7c9cf4b640 1 -- 172.21.15.67:0/38538877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542391070 msgr2=0x55f5427535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:30.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/38538877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542391070 0x55f5427535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7c9cf4b640 1 -- 172.21.15.67:0/38538877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542370390 msgr2=0x55f542752ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:30.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/38538877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542370390 0x55f542752ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7c9cf4b640 1 -- 172.21.15.67:0/38538877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5427bb5d0 con 0x55f5423681b0 2024-07-31T14:54:30.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.211+0000 7f7c97fff640 1 --2- 172.21.15.67:0/38538877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542370390 0x55f542752ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:30.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.212+0000 7f7c9cf4b640 1 --2- 172.21.15.67:0/38538877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f5427579b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c000c00 tx=0x7f7c8c0a6bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81d0df10e8883cf8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:30.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.212+0000 7f7c77fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:54:30.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.212+0000 7f7c77fff640 1 -- 172.21.15.67:0/38538877 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c020040 con 0x55f5423681b0 2024-07-31T14:54:30.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.212+0000 7f7c77fff640 1 -- 172.21.15.67:0/38538877 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f7c8c0cd0b0 con 0x55f5423681b0 2024-07-31T14:54:30.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.212+0000 7f7ca5700740 1 -- 172.21.15.67:0/38538877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c90047040 con 0x55f5423681b0 2024-07-31T14:54:30.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.212+0000 7f7c77fff640 1 -- 172.21.15.67:0/38538877 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7c8c074050 con 0x55f5423681b0 2024-07-31T14:54:30.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.212+0000 7f7ca5700740 10 client.26221 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:30.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.212+0000 7f7ca5700740 1 -- 172.21.15.67:0/38538877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f5427bc720 con 0x55f5423681b0 2024-07-31T14:54:30.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.212+0000 7f7c77fff640 1 -- 172.21.15.67:0/38538877 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c8c0aa200 con 0x55f5423681b0 2024-07-31T14:54:30.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.213+0000 7f7c77fff640 1 -- 172.21.15.67:0/38538877 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f7c8c0a6060 con 0x55f5423681b0 2024-07-31T14:54:30.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.213+0000 7f7ca5700740 10 client.26221 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:30.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.213+0000 7f7ca5700740 10 client.26221 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:54:30.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.213+0000 7f7ca5700740 20 client.26221 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:30.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.213+0000 7f7ca5700740 1 --2- 172.21.15.67:0/38538877 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f542324da0 0x55f5427bebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:30.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.213+0000 7f7ca5700740 4 client.26221 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:54:30.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.213+0000 7f7ca5700740 1 -- 172.21.15.67:0/38538877 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7c90047040 con 0x55f542324da0 2024-07-31T14:54:30.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.213+0000 7f7c97fff640 1 --2- 172.21.15.67:0/38538877 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f542324da0 0x55f5427bebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:30.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.213+0000 7f7c77fff640 10 client.26221 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:54:30.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.213+0000 7f7c97fff640 1 --2- 172.21.15.67:0/38538877 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f542324da0 0x55f5427bebf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9bb50e19a88fccc7 server_cookie=3fd6077179ae0016 in_seq=0 out_seq=0 2024-07-31T14:54:30.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.214+0000 7f7c77fff640 1 -- 172.21.15.67:0/38538877 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7c90047040 con 0x55f542324da0 2024-07-31T14:54:30.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.214+0000 7f7c77fff640 10 client.26221 handle_command_reply: tid=0 2024-07-31T14:54:30.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.214+0000 7f7ca5700740 1 client.26221 shutdown 2024-07-31T14:54:30.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.214+0000 7f7ca5700740 1 -- 172.21.15.67:0/38538877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 msgr2=0x55f5427579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:30.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.214+0000 7f7ca5700740 1 --2- 172.21.15.67:0/38538877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f5427579b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c000c00 tx=0x7f7c8c0a6bb0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.214+0000 7f7ca5700740 1 -- 172.21.15.67:0/38538877 shutdown_connections 2024-07-31T14:54:30.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.214+0000 7f7ca5700740 1 --2- 172.21.15.67:0/38538877 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55f542324da0 0x55f5427bebf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.214+0000 7f7ca5700740 1 --2- 172.21.15.67:0/38538877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f542391070 0x55f5427535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.214+0000 7f7c77fff640 0 client.26221 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:54:30.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.214+0000 7f7ca5700740 1 --2- 172.21.15.67:0/38538877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f542370390 0x55f542752ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.214+0000 7f7ca5700740 1 --2- 172.21.15.67:0/38538877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5423681b0 0x55f5427579b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.214+0000 7f7ca5700740 1 -- 172.21.15.67:0/38538877 >> 172.21.15.67:0/38538877 conn(0x55f542385460 msgr2=0x55f542739670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:30.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.214+0000 7f7ca5700740 1 -- 172.21.15.67:0/38538877 shutdown_connections 2024-07-31T14:54:30.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.214+0000 7f7ca5700740 1 -- 172.21.15.67:0/38538877 wait complete. 2024-07-31T14:54:30.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.215+0000 7f7ca5700740 20 client.26221 trim_cache size 0 max 16384 2024-07-31T14:54:30.221 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:54:30.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.215+0000 7f7c9f1af640 1 -- 172.21.15.67:0/3090281414 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c880b3d70 msgr2=0x7f7c880b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:30.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.215+0000 7f7c9f1af640 1 --2- 172.21.15.67:0/3090281414 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c880b3d70 0x7f7c880b6280 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7c9002f500 tx=0x7f7c9002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.215+0000 7f7c9f1af640 1 -- 172.21.15.67:0/3090281414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c98102490 msgr2=0x7f7c98199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:30.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.215+0000 7f7c9f1af640 1 --2- 172.21.15.67:0/3090281414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c98102490 0x7f7c98199ec0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00d6d0 tx=0x7f7c8c00db90 comp rx=0 tx=0).stop 2024-07-31T14:54:30.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.215+0000 7f7c9f1af640 1 -- 172.21.15.67:0/3090281414 shutdown_connections 2024-07-31T14:54:30.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.215+0000 7f7c9f1af640 1 --2- 172.21.15.67:0/3090281414 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c880b3d70 0x7f7c880b6280 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.215+0000 7f7c9f1af640 1 --2- 172.21.15.67:0/3090281414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c98108720 0x7f7c9819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.215+0000 7f7c9f1af640 1 --2- 172.21.15.67:0/3090281414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c98107dd0 0x7f7c9819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.215+0000 7f7c9f1af640 1 --2- 172.21.15.67:0/3090281414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c98102490 0x7f7c98199ec0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:30.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.215+0000 7f7c9f1af640 1 -- 172.21.15.67:0/3090281414 >> 172.21.15.67:0/3090281414 conn(0x7f7c98079160 msgr2=0x7f7c9806ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:30.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.216+0000 7f7c9f1af640 1 -- 172.21.15.67:0/3090281414 shutdown_connections 2024-07-31T14:54:30.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:30.216+0000 7f7c9f1af640 1 -- 172.21.15.67:0/3090281414 wait complete. 2024-07-31T14:54:31.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:31 smithi067 ceph-mon[32750]: pgmap v503: 129 pgs: 129 active+clean; 342 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 682 op/s 2024-07-31T14:54:31.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:31 smithi073 ceph-mon[36714]: pgmap v503: 129 pgs: 129 active+clean; 342 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 682 op/s 2024-07-31T14:54:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:31 smithi077 ceph-mon[37327]: pgmap v503: 129 pgs: 129 active+clean; 342 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 682 op/s 2024-07-31T14:54:32.482 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T14:54:33.343 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:33 smithi067 ceph-mon[32750]: pgmap v504: 129 pgs: 129 active+clean; 342 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 509 op/s 2024-07-31T14:54:33.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:33 smithi073 ceph-mon[36714]: pgmap v504: 129 pgs: 129 active+clean; 342 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 509 op/s 2024-07-31T14:54:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:33 smithi077 ceph-mon[37327]: pgmap v504: 129 pgs: 129 active+clean; 342 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 509 op/s 2024-07-31T14:54:33.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.580+0000 7f81845c1640 1 -- 172.21.15.67:0/3448612467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f817c103590 msgr2=0x7f817c1039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:33.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.580+0000 7f81845c1640 1 --2- 172.21.15.67:0/3448612467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f817c103590 0x7f817c1039a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8178009900 tx=0x7f817802a410 comp rx=0 tx=0).stop 2024-07-31T14:54:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.580+0000 7f81845c1640 1 -- 172.21.15.67:0/3448612467 shutdown_connections 2024-07-31T14:54:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.580+0000 7f81845c1640 1 --2- 172.21.15.67:0/3448612467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f817c109820 0x7f817c110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.580+0000 7f81845c1640 1 --2- 172.21.15.67:0/3448612467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f817c108ed0 0x7f817c1092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.580+0000 7f81845c1640 1 --2- 172.21.15.67:0/3448612467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f817c103590 0x7f817c1039a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.580+0000 7f81845c1640 1 -- 172.21.15.67:0/3448612467 >> 172.21.15.67:0/3448612467 conn(0x7f817c07b280 msgr2=0x7f817c07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.580+0000 7f81845c1640 1 -- 172.21.15.67:0/3448612467 shutdown_connections 2024-07-31T14:54:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.580+0000 7f81845c1640 1 -- 172.21.15.67:0/3448612467 wait complete. 2024-07-31T14:54:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.581+0000 7f81845c1640 1 Processor -- start 2024-07-31T14:54:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f81845c1640 1 -- start start 2024-07-31T14:54:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f81845c1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f817c103590 0x7f817c199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f81845c1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f817c108ed0 0x7f817c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f81845c1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f817c109820 0x7f817c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f81845c1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1134d0 con 0x7f817c109820 2024-07-31T14:54:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f81845c1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f817c113350 con 0x7f817c108ed0 2024-07-31T14:54:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f81845c1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f817c113650 con 0x7f817c103590 2024-07-31T14:54:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f8182b5e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f817c109820 0x7f817c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f8182b5e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f817c109820 0x7f817c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57914/0 (socket says 172.21.15.67:57914) 2024-07-31T14:54:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f8182b5e640 1 -- 172.21.15.67:0/2337140772 learned_addr learned my addr 172.21.15.67:0/2337140772 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f818235d640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f817c103590 0x7f817c199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f8181b5c640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f817c108ed0 0x7f817c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f8182b5e640 1 -- 172.21.15.67:0/2337140772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f817c103590 msgr2=0x7f817c199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f8182b5e640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f817c103590 0x7f817c199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f8182b5e640 1 -- 172.21.15.67:0/2337140772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f817c108ed0 msgr2=0x7f817c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:33.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f8182b5e640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f817c108ed0 0x7f817c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:33.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f8182b5e640 1 -- 172.21.15.67:0/2337140772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817c19fe50 con 0x7f817c109820 2024-07-31T14:54:33.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.582+0000 7f8181b5c640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f817c108ed0 0x7f817c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:54:33.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.583+0000 7f8182b5e640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f817c109820 0x7f817c19f720 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f817400eaa0 tx=0x7f817400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:33.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.583+0000 7f81737fe640 1 -- 172.21.15.67:0/2337140772 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817400cdb0 con 0x7f817c109820 2024-07-31T14:54:33.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.583+0000 7f81737fe640 1 -- 172.21.15.67:0/2337140772 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8174005020 con 0x7f817c109820 2024-07-31T14:54:33.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.583+0000 7f81845c1640 1 -- 172.21.15.67:0/2337140772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817c1a0140 con 0x7f817c109820 2024-07-31T14:54:33.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.583+0000 7f81737fe640 1 -- 172.21.15.67:0/2337140772 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81740116d0 con 0x7f817c109820 2024-07-31T14:54:33.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.583+0000 7f81845c1640 1 -- 172.21.15.67:0/2337140772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817c1a68a0 con 0x7f817c109820 2024-07-31T14:54:33.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.584+0000 7f81845c1640 1 -- 172.21.15.67:0/2337140772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f817c1039a0 con 0x7f817c109820 2024-07-31T14:54:33.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.586+0000 7f81737fe640 1 -- 172.21.15.67:0/2337140772 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f81740051d0 con 0x7f817c109820 2024-07-31T14:54:33.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.587+0000 7f81737fe640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81680b3d40 0x7f81680b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:33.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.587+0000 7f818235d640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81680b3d40 0x7f81680b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:33.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.588+0000 7f81737fe640 1 -- 172.21.15.67:0/2337140772 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f81740e48b0 con 0x7f817c109820 2024-07-31T14:54:33.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.588+0000 7f818235d640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81680b3d40 0x7f81680b6250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8178031440 tx=0x7f8178009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:33.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.591+0000 7f81737fe640 1 -- 172.21.15.67:0/2337140772 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f8174016030 con 0x7f817c109820 2024-07-31T14:54:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:54:33.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.832+0000 7f81845c1640 1 -- 172.21.15.67:0/2337140772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f817c19b010 con 0x7f817c109820 2024-07-31T14:54:33.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.832+0000 7f81737fe640 1 -- 172.21.15.67:0/2337140772 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11980 (secure 0 0 0) 0x7f817c19b010 con 0x7f817c109820 2024-07-31T14:54:33.834 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:54:33.834 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":39,"btime":"2024-07-31T14:54:12:039205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:54:11.524751+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T14:54:33.834 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 39 2024-07-31T14:54:33.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.834+0000 7f81845c1640 1 -- 172.21.15.67:0/2337140772 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81680b3d40 msgr2=0x7f81680b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:33.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.834+0000 7f81845c1640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81680b3d40 0x7f81680b6250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8178031440 tx=0x7f8178009560 comp rx=0 tx=0).stop 2024-07-31T14:54:33.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.834+0000 7f81845c1640 1 -- 172.21.15.67:0/2337140772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f817c109820 msgr2=0x7f817c19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:33.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.834+0000 7f81845c1640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f817c109820 0x7f817c19f720 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f817400eaa0 tx=0x7f817400ef60 comp rx=0 tx=0).stop 2024-07-31T14:54:33.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.834+0000 7f81845c1640 1 -- 172.21.15.67:0/2337140772 shutdown_connections 2024-07-31T14:54:33.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.834+0000 7f81845c1640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81680b3d40 0x7f81680b6250 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:33.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.834+0000 7f81845c1640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f817c109820 0x7f817c19f720 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:33.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.834+0000 7f81845c1640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f817c108ed0 0x7f817c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:33.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.834+0000 7f81845c1640 1 --2- 172.21.15.67:0/2337140772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f817c103590 0x7f817c199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:33.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.834+0000 7f81845c1640 1 -- 172.21.15.67:0/2337140772 >> 172.21.15.67:0/2337140772 conn(0x7f817c07b280 msgr2=0x7f817c106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:33.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.835+0000 7f81845c1640 1 -- 172.21.15.67:0/2337140772 shutdown_connections 2024-07-31T14:54:33.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:33.835+0000 7f81845c1640 1 -- 172.21.15.67:0/2337140772 wait complete. 2024-07-31T14:54:34.348 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:34 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2337140772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:54:34.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:34 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2337140772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:54:34.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:34 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2337140772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:54:34.667 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 40ba138f --release --await' 2024-07-31T14:54:34.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 40ba138f --release --await 2024-07-31T14:54:35.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:35 smithi067 ceph-mon[32750]: pgmap v505: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.7 MiB/s wr, 319 op/s 2024-07-31T14:54:35.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:35 smithi073 ceph-mon[36714]: pgmap v505: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.7 MiB/s wr, 319 op/s 2024-07-31T14:54:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:35 smithi077 ceph-mon[37327]: pgmap v505: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.7 MiB/s wr, 319 op/s 2024-07-31T14:54:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.288+0000 7f4bd17d4640 1 -- 172.21.15.67:0/2480419184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bcc103530 msgr2=0x7f4bcc103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.288+0000 7f4bd17d4640 1 --2- 172.21.15.67:0/2480419184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bcc103530 0x7f4bcc103940 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009900 tx=0x7f4bbc02a410 comp rx=0 tx=0).stop 2024-07-31T14:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.288+0000 7f4bd17d4640 1 -- 172.21.15.67:0/2480419184 shutdown_connections 2024-07-31T14:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.288+0000 7f4bd17d4640 1 --2- 172.21.15.67:0/2480419184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bcc1097b0 0x7f4bcc10ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.288+0000 7f4bd17d4640 1 --2- 172.21.15.67:0/2480419184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc108e60 0x7f4bcc109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.288+0000 7f4bd17d4640 1 --2- 172.21.15.67:0/2480419184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bcc103530 0x7f4bcc103940 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.288+0000 7f4bd17d4640 1 -- 172.21.15.67:0/2480419184 >> 172.21.15.67:0/2480419184 conn(0x7f4bcc0fef70 msgr2=0x7f4bcc1013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.288+0000 7f4bd17d4640 1 -- 172.21.15.67:0/2480419184 shutdown_connections 2024-07-31T14:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.288+0000 7f4bd17d4640 1 -- 172.21.15.67:0/2480419184 wait complete. 2024-07-31T14:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bd17d4640 1 Processor -- start 2024-07-31T14:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bd17d4640 1 -- start start 2024-07-31T14:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bd17d4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc103530 0x7f4bcc199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bd17d4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bcc108e60 0x7f4bcc19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bd17d4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bcc1097b0 0x7f4bcc19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bd17d4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc113480 con 0x7f4bcc1097b0 2024-07-31T14:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bd17d4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc113300 con 0x7f4bcc108e60 2024-07-31T14:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bd17d4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc113600 con 0x7f4bcc103530 2024-07-31T14:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bcb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bcc1097b0 0x7f4bcc19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bcaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc103530 0x7f4bcc199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bcb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bcc1097b0 0x7f4bcc19f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57932/0 (socket says 172.21.15.67:57932) 2024-07-31T14:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bcb7fe640 1 -- 172.21.15.67:0/2944160511 learned_addr learned my addr 172.21.15.67:0/2944160511 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/2944160511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bcc108e60 0x7f4bcc19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bcb7fe640 1 -- 172.21.15.67:0/2944160511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc103530 msgr2=0x7f4bcc199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bcb7fe640 1 --2- 172.21.15.67:0/2944160511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc103530 0x7f4bcc199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bcb7fe640 1 -- 172.21.15.67:0/2944160511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bcc108e60 msgr2=0x7f4bcc19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bcb7fe640 1 --2- 172.21.15.67:0/2944160511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bcc108e60 0x7f4bcc19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bcb7fe640 1 -- 172.21.15.67:0/2944160511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc19fd90 con 0x7f4bcc1097b0 2024-07-31T14:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.289+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/2944160511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bcc108e60 0x7f4bcc19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.290+0000 7f4bcb7fe640 1 --2- 172.21.15.67:0/2944160511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bcc1097b0 0x7f4bcc19f690 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400eb10 tx=0x7f4bc400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.290+0000 7f4bc8ff9640 1 -- 172.21.15.67:0/2944160511 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4004b20 con 0x7f4bcc1097b0 2024-07-31T14:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.290+0000 7f4bc8ff9640 1 -- 172.21.15.67:0/2944160511 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bc4004cc0 con 0x7f4bcc1097b0 2024-07-31T14:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.290+0000 7f4bd17d4640 1 -- 172.21.15.67:0/2944160511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bcc1a0080 con 0x7f4bcc1097b0 2024-07-31T14:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.290+0000 7f4bc8ff9640 1 -- 172.21.15.67:0/2944160511 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4010670 con 0x7f4bcc1097b0 2024-07-31T14:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.290+0000 7f4bd17d4640 1 -- 172.21.15.67:0/2944160511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bcc1a6810 con 0x7f4bcc1097b0 2024-07-31T14:54:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.293+0000 7f4bd87ab740 1 Processor -- start 2024-07-31T14:54:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.293+0000 7f4bc8ff9640 1 -- 172.21.15.67:0/2944160511 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4bc4010830 con 0x7f4bcc1097b0 2024-07-31T14:54:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.293+0000 7f4bd87ab740 1 -- start start 2024-07-31T14:54:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.293+0000 7f4bd87ab740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184491b0 0x5584187ebce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.293+0000 7f4bd87ab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184202f0 0x5584187ec220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.293+0000 7f4bd87ab740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184284d0 0x5584187ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.293+0000 7f4bd87ab740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5584187613b0 con 0x5584184491b0 2024-07-31T14:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.293+0000 7f4bd87ab740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5584187508b0 con 0x5584184284d0 2024-07-31T14:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.293+0000 7f4bd87ab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5584187548f0 con 0x5584184202f0 2024-07-31T14:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bc3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184491b0 0x5584187ebce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bcaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184202f0 0x5584187ec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bcaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184202f0 0x5584187ec220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:33046/0 (socket says 172.21.15.67:33046) 2024-07-31T14:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bc3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184491b0 0x5584187ebce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57934/0 (socket says 172.21.15.67:57934) 2024-07-31T14:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bcaffd640 1 -- 172.21.15.67:0/596817138 learned_addr learned my addr 172.21.15.67:0/596817138 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bc8ff9640 1 --2- 172.21.15.67:0/2944160511 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4bb80b3dc0 0x7f4bb80b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bcaffd640 1 -- 172.21.15.67:0/596817138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184284d0 msgr2=0x5584187ec760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/596817138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184284d0 0x5584187ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bcaffd640 1 --2- 172.21.15.67:0/596817138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184284d0 0x5584187ec760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bcaffd640 1 -- 172.21.15.67:0/596817138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184491b0 msgr2=0x5584187ebce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bcaffd640 1 --2- 172.21.15.67:0/596817138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184491b0 0x5584187ebce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bcaffd640 1 -- 172.21.15.67:0/596817138 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584186f79a0 con 0x5584184202f0 2024-07-31T14:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/596817138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184284d0 0x5584187ec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bcb7fe640 1 --2- 172.21.15.67:0/2944160511 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4bb80b3dc0 0x7f4bb80b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bcaffd640 1 --2- 172.21.15.67:0/596817138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184202f0 0x5584187ec220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc033000 tx=0x7f4bbc032b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bcaac379eb782e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bc8ff9640 1 -- 172.21.15.67:0/2944160511 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f4bc40e4930 con 0x7f4bcc1097b0 2024-07-31T14:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.294+0000 7f4bc17fa640 1 -- 172.21.15.67:0/596817138 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc035040 con 0x5584184202f0 2024-07-31T14:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.295+0000 7f4bc17fa640 1 -- 172.21.15.67:0/596817138 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bbc004030 con 0x5584184202f0 2024-07-31T14:54:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.295+0000 7f4bc17fa640 1 -- 172.21.15.67:0/596817138 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0055e0 con 0x5584184202f0 2024-07-31T14:54:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.295+0000 7f4bd87ab740 1 -- 172.21.15.67:0/596817138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184202f0 msgr2=0x5584187ec220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.295+0000 7f4bcb7fe640 1 --2- 172.21.15.67:0/2944160511 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4bb80b3dc0 0x7f4bb80b62d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4bc40e1e20 tx=0x7f4bc4075580 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.295+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/596817138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184202f0 0x5584187ec220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc033000 tx=0x7f4bbc032b90 comp rx=0 tx=0).stop 2024-07-31T14:54:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.295+0000 7f4bd87ab740 1 -- 172.21.15.67:0/596817138 shutdown_connections 2024-07-31T14:54:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.295+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/596817138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184284d0 0x5584187ec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.295+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/596817138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184202f0 0x5584187ec220 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.295+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/596817138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184491b0 0x5584187ebce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.295+0000 7f4bd87ab740 1 -- 172.21.15.67:0/596817138 >> 172.21.15.67:0/596817138 conn(0x55841843d300 msgr2=0x5584187e9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.296+0000 7f4bd87ab740 1 -- 172.21.15.67:0/596817138 shutdown_connections 2024-07-31T14:54:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.296+0000 7f4bd87ab740 1 -- 172.21.15.67:0/596817138 wait complete. 2024-07-31T14:54:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.296+0000 7f4bd87ab740 1 -- start start 2024-07-31T14:54:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.296+0000 7f4bd87ab740 1 Processor -- start 2024-07-31T14:54:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.296+0000 7f4bd87ab740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 0x5584187f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.296+0000 7f4bd87ab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184284d0 0x5584187f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.296+0000 7f4bd87ab740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184491b0 0x558418803750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.296+0000 7f4bd87ab740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55841868aab0 con 0x5584184491b0 2024-07-31T14:54:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.296+0000 7f4bd87ab740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5584186d8cf0 con 0x5584184202f0 2024-07-31T14:54:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.296+0000 7f4bd87ab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5584187fa400 con 0x5584184284d0 2024-07-31T14:54:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.296+0000 7f4bd87ab740 10 client.0 _ll_register_callbacks cb 0x55841874a9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bc3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184491b0 0x558418803750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bc3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184491b0 0x558418803750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57940/0 (socket says 172.21.15.67:57940) 2024-07-31T14:54:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bc3fff640 1 -- 172.21.15.67:0/583112893 learned_addr learned my addr 172.21.15.67:0/583112893 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bcaffd640 1 --2- 172.21.15.67:0/583112893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184284d0 0x5584187f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/583112893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 0x5584187f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bc3fff640 1 -- 172.21.15.67:0/583112893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184284d0 msgr2=0x5584187f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/583112893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184284d0 0x5584187f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bc3fff640 1 -- 172.21.15.67:0/583112893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 msgr2=0x5584187f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/583112893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 0x5584187f77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bc3fff640 1 -- 172.21.15.67:0/583112893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584184b4790 con 0x5584184491b0 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/583112893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184491b0 0x558418803750 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009ed0 tx=0x7f4bb4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687f13483b9b1e3e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bb27fc640 1 -- 172.21.15.67:0/583112893 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4013110 con 0x5584184491b0 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bd87ab740 1 -- 172.21.15.67:0/583112893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bbc004030 con 0x5584184491b0 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bb27fc640 1 -- 172.21.15.67:0/583112893 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f4bb4005020 con 0x5584184491b0 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bb27fc640 1 -- 172.21.15.67:0/583112893 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4bb4027050 con 0x5584184491b0 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bd87ab740 10 client.16590 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.297+0000 7f4bd87ab740 1 -- 172.21.15.67:0/583112893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55841852d620 con 0x5584184491b0 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.298+0000 7f4bb27fc640 1 -- 172.21.15.67:0/583112893 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bbc004030 con 0x5584184491b0 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.298+0000 7f4bb27fc640 1 -- 172.21.15.67:0/583112893 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f4bb4004cc0 con 0x5584184491b0 2024-07-31T14:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.298+0000 7f4bd87ab740 10 client.16590 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.298+0000 7f4bd87ab740 10 client.16590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.298+0000 7f4bd87ab740 20 client.16590 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.298+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/583112893 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558418407d80 0x5584188097a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.298+0000 7f4bd87ab740 4 client.16590 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:54:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.298+0000 7f4bd87ab740 1 -- 172.21.15.67:0/583112893 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558418763300 con 0x558418407d80 2024-07-31T14:54:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.298+0000 7f4bcaffd640 1 --2- 172.21.15.67:0/583112893 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558418407d80 0x5584188097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.299+0000 7f4bb27fc640 10 client.16590 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.299+0000 7f4bcaffd640 1 --2- 172.21.15.67:0/583112893 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558418407d80 0x5584188097a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f0e8a2ecaf5d9e server_cookie=a621bf444aa3cd41 in_seq=0 out_seq=0 2024-07-31T14:54:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.300+0000 7f4bb27fc640 1 -- 172.21.15.67:0/583112893 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558418763300 con 0x558418407d80 2024-07-31T14:54:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.300+0000 7f4bb27fc640 10 client.16590 handle_command_reply: tid=0 2024-07-31T14:54:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.300+0000 7f4bd87ab740 1 client.16590 shutdown 2024-07-31T14:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.300+0000 7f4bd87ab740 1 -- 172.21.15.67:0/583112893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184491b0 msgr2=0x558418803750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.300+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/583112893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184491b0 0x558418803750 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009ed0 tx=0x7f4bb4005f90 comp rx=0 tx=0).stop 2024-07-31T14:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.300+0000 7f4bd87ab740 1 -- 172.21.15.67:0/583112893 shutdown_connections 2024-07-31T14:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.300+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/583112893 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558418407d80 0x5584188097a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.300+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/583112893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184491b0 0x558418803750 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.300+0000 7f4bb27fc640 0 client.16590 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.300+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/583112893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184284d0 0x5584187f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.300+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/583112893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 0x5584187f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.300+0000 7f4bd87ab740 1 -- 172.21.15.67:0/583112893 >> 172.21.15.67:0/583112893 conn(0x55841843d300 msgr2=0x5584187ecd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.312+0000 7f4bd87ab740 1 -- 172.21.15.67:0/583112893 shutdown_connections 2024-07-31T14:54:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.312+0000 7f4bd87ab740 1 -- 172.21.15.67:0/583112893 wait complete. 2024-07-31T14:54:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.312+0000 7f4bd87ab740 20 client.16590 trim_cache size 0 max 16384 2024-07-31T14:54:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.327+0000 7f4bd87ab740 1 Processor -- start 2024-07-31T14:54:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.327+0000 7f4bd87ab740 1 -- start start 2024-07-31T14:54:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bd87ab740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 0x5584187f9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bd87ab740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184284d0 0x5584187f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bd87ab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184491b0 0x55841880ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bd87ab740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1b06d0 con 0x5584184284d0 2024-07-31T14:54:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bd87ab740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1b2390 con 0x5584184202f0 2024-07-31T14:54:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bd87ab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1b0da0 con 0x5584184491b0 2024-07-31T14:54:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bcaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184284d0 0x5584187f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bcaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184284d0 0x5584187f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57952/0 (socket says 172.21.15.67:57952) 2024-07-31T14:54:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bcaffd640 1 -- 172.21.15.67:0/1841920362 learned_addr learned my addr 172.21.15.67:0/1841920362 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/1841920362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 0x5584187f9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/1841920362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184491b0 0x55841880ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bcaffd640 1 -- 172.21.15.67:0/1841920362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184491b0 msgr2=0x55841880ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bcaffd640 1 --2- 172.21.15.67:0/1841920362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184491b0 0x55841880ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bcaffd640 1 -- 172.21.15.67:0/1841920362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 msgr2=0x5584187f9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bcaffd640 1 --2- 172.21.15.67:0/1841920362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 0x5584187f9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bcaffd640 1 -- 172.21.15.67:0/1841920362 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584187f9cf0 con 0x5584184284d0 2024-07-31T14:54:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/1841920362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184491b0 0x55841880ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:54:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.328+0000 7f4bcaffd640 1 --2- 172.21.15.67:0/1841920362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184284d0 0x5584187f6cf0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0051d0 tx=0x7f4bbc005200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e0b9f4e01e72c9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.329+0000 7f4bc0ff9640 1 -- 172.21.15.67:0/1841920362 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc035040 con 0x5584184284d0 2024-07-31T14:54:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.329+0000 7f4bc0ff9640 1 -- 172.21.15.67:0/1841920362 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bbc005510 con 0x5584184284d0 2024-07-31T14:54:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.329+0000 7f4bc0ff9640 1 -- 172.21.15.67:0/1841920362 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0043b0 con 0x5584184284d0 2024-07-31T14:54:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.330+0000 7f4bd87ab740 1 -- 172.21.15.67:0/1841920362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184284d0 msgr2=0x5584187f6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.330+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/1841920362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184284d0 0x5584187f6cf0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0051d0 tx=0x7f4bbc005200 comp rx=0 tx=0).stop 2024-07-31T14:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.330+0000 7f4bd87ab740 1 -- 172.21.15.67:0/1841920362 shutdown_connections 2024-07-31T14:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.330+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/1841920362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184491b0 0x55841880ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.330+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/1841920362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184284d0 0x5584187f6cf0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.330+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/1841920362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 0x5584187f9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.330+0000 7f4bd87ab740 1 -- 172.21.15.67:0/1841920362 >> 172.21.15.67:0/1841920362 conn(0x55841843d300 msgr2=0x5584187f0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.330+0000 7f4bd87ab740 1 -- 172.21.15.67:0/1841920362 shutdown_connections 2024-07-31T14:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.330+0000 7f4bd87ab740 1 -- 172.21.15.67:0/1841920362 wait complete. 2024-07-31T14:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.330+0000 7f4bd87ab740 1 -- start start 2024-07-31T14:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.330+0000 7f4bd87ab740 1 Processor -- start 2024-07-31T14:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bd87ab740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 0x5584187fff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bd87ab740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184284d0 0x5584187fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bd87ab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184491b0 0x5584187fcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bd87ab740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1b1cc0 con 0x5584184284d0 2024-07-31T14:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bd87ab740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1b1530 con 0x5584184202f0 2024-07-31T14:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bd87ab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1ae6b0 con 0x5584184491b0 2024-07-31T14:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bd87ab740 10 client.0 _ll_register_callbacks cb 0x7f4bb4009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bcaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184284d0 0x5584187fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bcaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184284d0 0x5584187fc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57958/0 (socket says 172.21.15.67:57958) 2024-07-31T14:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bcaffd640 1 -- 172.21.15.67:0/590437096 learned_addr learned my addr 172.21.15.67:0/590437096 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/590437096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 0x5584187fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/590437096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184491b0 0x5584187fcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bcaffd640 1 -- 172.21.15.67:0/590437096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184491b0 msgr2=0x5584187fcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bcaffd640 1 --2- 172.21.15.67:0/590437096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184491b0 0x5584187fcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bcaffd640 1 -- 172.21.15.67:0/590437096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 msgr2=0x5584187fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bcaffd640 1 --2- 172.21.15.67:0/590437096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 0x5584187fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bcaffd640 1 -- 172.21.15.67:0/590437096 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558418874340 con 0x5584184284d0 2024-07-31T14:54:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/590437096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184491b0 0x5584187fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:54:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bcaffd640 1 --2- 172.21.15.67:0/590437096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184284d0 0x5584187fc6c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc005400 tx=0x7f4bbc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475f66d8f3361589 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.331+0000 7f4bb27fc640 1 -- 172.21.15.67:0/590437096 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc05c070 con 0x5584184284d0 2024-07-31T14:54:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.332+0000 7f4bb27fc640 1 -- 172.21.15.67:0/590437096 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f4bbc004130 con 0x5584184284d0 2024-07-31T14:54:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.332+0000 7f4bd87ab740 1 -- 172.21.15.67:0/590437096 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bbc005510 con 0x5584184284d0 2024-07-31T14:54:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.332+0000 7f4bb27fc640 1 -- 172.21.15.67:0/590437096 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4bbc06a050 con 0x5584184284d0 2024-07-31T14:54:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.332+0000 7f4bd87ab740 10 client.16596 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.332+0000 7f4bd87ab740 1 -- 172.21.15.67:0/590437096 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558418875380 con 0x5584184284d0 2024-07-31T14:54:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.332+0000 7f4bb27fc640 1 -- 172.21.15.67:0/590437096 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bbc005510 con 0x5584184284d0 2024-07-31T14:54:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.332+0000 7f4bb27fc640 1 -- 172.21.15.67:0/590437096 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f4bbc04f790 con 0x5584184284d0 2024-07-31T14:54:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.332+0000 7f4bd87ab740 10 client.16596 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.332+0000 7f4bd87ab740 10 client.16596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.332+0000 7f4bd87ab740 20 client.16596 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.332+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/590437096 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584183dd1c0 0x5584188775c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.332+0000 7f4bd87ab740 4 client.16596 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:54:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.332+0000 7f4bd87ab740 1 -- 172.21.15.67:0/590437096 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55841875eda0 con 0x5584183dd1c0 2024-07-31T14:54:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.333+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/590437096 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584183dd1c0 0x5584188775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.333+0000 7f4bb27fc640 10 client.16596 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.333+0000 7f4bc3fff640 1 --2- 172.21.15.67:0/590437096 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584183dd1c0 0x5584188775c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a54e7f86f4190c5 server_cookie=391f3813fcb92fac in_seq=0 out_seq=0 2024-07-31T14:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.360+0000 7f4bb27fc640 1 -- 172.21.15.67:0/590437096 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55841875eda0 con 0x5584183dd1c0 2024-07-31T14:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.360+0000 7f4bb27fc640 10 client.16596 handle_command_reply: tid=0 2024-07-31T14:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.360+0000 7f4bd87ab740 1 client.16596 shutdown 2024-07-31T14:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.360+0000 7f4bd87ab740 1 -- 172.21.15.67:0/590437096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184284d0 msgr2=0x5584187fc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.360+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/590437096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184284d0 0x5584187fc6c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc005400 tx=0x7f4bbc000bc0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.360+0000 7f4bd87ab740 1 -- 172.21.15.67:0/590437096 shutdown_connections 2024-07-31T14:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.360+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/590437096 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584183dd1c0 0x5584188775c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/590437096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584184491b0 0x5584187fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bb27fc640 0 client.16596 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/590437096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584184284d0 0x5584187fc6c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd87ab740 1 --2- 172.21.15.67:0/590437096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584184202f0 0x5584187fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd87ab740 1 -- 172.21.15.67:0/590437096 >> 172.21.15.67:0/590437096 conn(0x55841843d300 msgr2=0x5584187f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd87ab740 1 -- 172.21.15.67:0/590437096 shutdown_connections 2024-07-31T14:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd87ab740 1 -- 172.21.15.67:0/590437096 wait complete. 2024-07-31T14:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd87ab740 20 client.16596 trim_cache size 0 max 16384 2024-07-31T14:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd17d4640 1 -- 172.21.15.67:0/2944160511 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4bb80b3dc0 msgr2=0x7f4bb80b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd17d4640 1 --2- 172.21.15.67:0/2944160511 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4bb80b3dc0 0x7f4bb80b62d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4bc40e1e20 tx=0x7f4bc4075580 comp rx=0 tx=0).stop 2024-07-31T14:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd17d4640 1 -- 172.21.15.67:0/2944160511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bcc1097b0 msgr2=0x7f4bcc19f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd17d4640 1 --2- 172.21.15.67:0/2944160511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bcc1097b0 0x7f4bcc19f690 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400eb10 tx=0x7f4bc400c2d0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd17d4640 1 -- 172.21.15.67:0/2944160511 shutdown_connections 2024-07-31T14:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd17d4640 1 --2- 172.21.15.67:0/2944160511 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4bb80b3dc0 0x7f4bb80b62d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd17d4640 1 --2- 172.21.15.67:0/2944160511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bcc1097b0 0x7f4bcc19f690 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd17d4640 1 --2- 172.21.15.67:0/2944160511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bcc108e60 0x7f4bcc19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd17d4640 1 --2- 172.21.15.67:0/2944160511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bcc103530 0x7f4bcc199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.361+0000 7f4bd17d4640 1 -- 172.21.15.67:0/2944160511 >> 172.21.15.67:0/2944160511 conn(0x7f4bcc0fef70 msgr2=0x7f4bcc101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.362+0000 7f4bd17d4640 1 -- 172.21.15.67:0/2944160511 shutdown_connections 2024-07-31T14:54:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:36.362+0000 7f4bd17d4640 1 -- 172.21.15.67:0/2944160511 wait complete. 2024-07-31T14:54:36.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.525+0000 7f94a6cd8640 1 -- 172.21.15.67:0/3267308101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a0107de0 msgr2=0x7f94a01081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.526+0000 7f94a6cd8640 1 --2- 172.21.15.67:0/3267308101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a0107de0 0x7f94a01081f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f948c009900 tx=0x7f948c02a410 comp rx=0 tx=0).stop 2024-07-31T14:54:36.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.526+0000 7f94a6cd8640 1 -- 172.21.15.67:0/3267308101 shutdown_connections 2024-07-31T14:54:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.526+0000 7f94a6cd8640 1 --2- 172.21.15.67:0/3267308101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a0108730 0x7f94a010ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.526+0000 7f94a6cd8640 1 --2- 172.21.15.67:0/3267308101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a0107de0 0x7f94a01081f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.526+0000 7f94a6cd8640 1 --2- 172.21.15.67:0/3267308101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a01024a0 0x7f94a01028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.526+0000 7f94a6cd8640 1 -- 172.21.15.67:0/3267308101 >> 172.21.15.67:0/3267308101 conn(0x7f94a0079140 msgr2=0x7f94a0079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.526+0000 7f94a6cd8640 1 -- 172.21.15.67:0/3267308101 shutdown_connections 2024-07-31T14:54:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.526+0000 7f94a6cd8640 1 -- 172.21.15.67:0/3267308101 wait complete. 2024-07-31T14:54:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.526+0000 7f94a6cd8640 1 Processor -- start 2024-07-31T14:54:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.526+0000 7f94a6cd8640 1 -- start start 2024-07-31T14:54:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f94a6cd8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a01024a0 0x7f94a00ff020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f94a6cd8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a0107de0 0x7f94a00ff560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f94a6cd8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a0108730 0x7f94a00ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f94a6cd8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01123c0 con 0x7f94a0107de0 2024-07-31T14:54:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f94a6cd8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f94a0112240 con 0x7f94a0108730 2024-07-31T14:54:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f94a6cd8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f94a0112540 con 0x7f94a01024a0 2024-07-31T14:54:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f949ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a0107de0 0x7f94a00ff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f949ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a0107de0 0x7f94a00ff560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57986/0 (socket says 172.21.15.67:57986) 2024-07-31T14:54:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f94a4a74640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a01024a0 0x7f94a00ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f949ffff640 1 -- 172.21.15.67:0/2265356070 learned_addr learned my addr 172.21.15.67:0/2265356070 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f94a5275640 1 --2- 172.21.15.67:0/2265356070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a0108730 0x7f94a00ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f949ffff640 1 -- 172.21.15.67:0/2265356070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a01024a0 msgr2=0x7f94a00ff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f949ffff640 1 --2- 172.21.15.67:0/2265356070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a01024a0 0x7f94a00ff020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f949ffff640 1 -- 172.21.15.67:0/2265356070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a0108730 msgr2=0x7f94a00ffaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f949ffff640 1 --2- 172.21.15.67:0/2265356070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a0108730 0x7f94a00ffaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f949ffff640 1 -- 172.21.15.67:0/2265356070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a01a8240 con 0x7f94a0107de0 2024-07-31T14:54:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.527+0000 7f94a4a74640 1 --2- 172.21.15.67:0/2265356070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a01024a0 0x7f94a00ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.528+0000 7f949ffff640 1 --2- 172.21.15.67:0/2265356070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a0107de0 0x7f94a00ff560 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f948c006010 tx=0x7f948c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.528+0000 7f949dffb640 1 -- 172.21.15.67:0/2265356070 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c03f030 con 0x7f94a0107de0 2024-07-31T14:54:36.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.528+0000 7f949dffb640 1 -- 172.21.15.67:0/2265356070 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f948c003aa0 con 0x7f94a0107de0 2024-07-31T14:54:36.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.528+0000 7f94a6cd8640 1 -- 172.21.15.67:0/2265356070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a01a84d0 con 0x7f94a0107de0 2024-07-31T14:54:36.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.528+0000 7f949dffb640 1 -- 172.21.15.67:0/2265356070 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c035070 con 0x7f94a0107de0 2024-07-31T14:54:36.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.528+0000 7f94a6cd8640 1 -- 172.21.15.67:0/2265356070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a01a89b0 con 0x7f94a0107de0 2024-07-31T14:54:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.532+0000 7f94ad229740 1 Processor -- start 2024-07-31T14:54:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.532+0000 7f94ad229740 1 -- start start 2024-07-31T14:54:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.532+0000 7f94ad229740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c31e070 0x55e61c6c3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.532+0000 7f94ad229740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2f51b0 0x55e61c6c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.532+0000 7f94ad229740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 0x55e61c6c4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.532+0000 7f94ad229740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e61c639bb0 con 0x55e61c31e070 2024-07-31T14:54:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.532+0000 7f94ad229740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e61c6290b0 con 0x55e61c2f51b0 2024-07-31T14:54:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.532+0000 7f94ad229740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e61c62d0f0 con 0x55e61c2fd390 2024-07-31T14:54:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.533+0000 7f94a4a74640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2f51b0 0x55e61c6c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.533+0000 7f94a4a74640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2f51b0 0x55e61c6c3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:53640/0 (socket says 172.21.15.67:53640) 2024-07-31T14:54:36.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.533+0000 7f94a5275640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c31e070 0x55e61c6c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.533+0000 7f94a4a74640 1 -- 172.21.15.67:0/2203417785 learned_addr learned my addr 172.21.15.67:0/2203417785 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:36.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.534+0000 7f94a5275640 1 -- 172.21.15.67:0/2203417785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 msgr2=0x55e61c6c4110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:54:36.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.534+0000 7f94a5275640 1 --2- 172.21.15.67:0/2203417785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 0x55e61c6c4110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.534+0000 7f94a5275640 1 -- 172.21.15.67:0/2203417785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2f51b0 msgr2=0x55e61c6c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.534+0000 7f94a5275640 1 --2- 172.21.15.67:0/2203417785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2f51b0 0x55e61c6c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.534+0000 7f94a5275640 1 -- 172.21.15.67:0/2203417785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61c5cfb90 con 0x55e61c31e070 2024-07-31T14:54:36.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.534+0000 7f94a5275640 1 --2- 172.21.15.67:0/2203417785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c31e070 0x55e61c6c3690 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9498009eb0 tx=0x7f94980107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd3dd8d9d9e1759 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:36.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.534+0000 7f949dffb640 1 -- 172.21.15.67:0/2265356070 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f948c042030 con 0x7f94a0107de0 2024-07-31T14:54:36.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.534+0000 7f948affd640 1 -- 172.21.15.67:0/2203417785 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498003b70 con 0x55e61c31e070 2024-07-31T14:54:36.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.534+0000 7f948affd640 1 -- 172.21.15.67:0/2203417785 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9498015070 con 0x55e61c31e070 2024-07-31T14:54:36.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.534+0000 7f948affd640 1 -- 172.21.15.67:0/2203417785 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949801a830 con 0x55e61c31e070 2024-07-31T14:54:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.535+0000 7f949dffb640 1 --2- 172.21.15.67:0/2265356070 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94900b3d40 0x7f94900b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.536+0000 7f94a4a74640 1 --2- 172.21.15.67:0/2265356070 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94900b3d40 0x7f94900b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.536+0000 7f94ad229740 1 -- 172.21.15.67:0/2203417785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c31e070 msgr2=0x55e61c6c3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.536+0000 7f94ad229740 1 --2- 172.21.15.67:0/2203417785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c31e070 0x55e61c6c3690 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9498009eb0 tx=0x7f94980107a0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.536+0000 7f949dffb640 1 -- 172.21.15.67:0/2265356070 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f948c104780 con 0x7f94a0107de0 2024-07-31T14:54:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.536+0000 7f94ad229740 1 -- 172.21.15.67:0/2203417785 shutdown_connections 2024-07-31T14:54:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.536+0000 7f94ad229740 1 --2- 172.21.15.67:0/2203417785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 0x55e61c6c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.536+0000 7f94ad229740 1 --2- 172.21.15.67:0/2203417785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2f51b0 0x55e61c6c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.536+0000 7f94ad229740 1 --2- 172.21.15.67:0/2203417785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c31e070 0x55e61c6c3690 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.536+0000 7f94ad229740 1 -- 172.21.15.67:0/2203417785 >> 172.21.15.67:0/2203417785 conn(0x55e61c312460 msgr2=0x55e61c6c1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.536+0000 7f94ad229740 1 -- 172.21.15.67:0/2203417785 shutdown_connections 2024-07-31T14:54:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.536+0000 7f94ad229740 1 -- 172.21.15.67:0/2203417785 wait complete. 2024-07-31T14:54:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.537+0000 7f94a4a74640 1 --2- 172.21.15.67:0/2265356070 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94900b3d40 0x7f94900b6250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55e61c6c2310 tx=0x7f9494005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.537+0000 7f94ad229740 1 -- start start 2024-07-31T14:54:36.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.537+0000 7f94ad229740 1 Processor -- start 2024-07-31T14:54:36.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.537+0000 7f94ad229740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c2f51b0 0x55e61c6cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.537+0000 7f94ad229740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 0x55e61c6cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.537+0000 7f94ad229740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c31e070 0x55e61c6e39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.537+0000 7f94ad229740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e61c48c700 con 0x55e61c2f51b0 2024-07-31T14:54:36.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.537+0000 7f94ad229740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e61c5cb9e0 con 0x55e61c31e070 2024-07-31T14:54:36.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.537+0000 7f94ad229740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e61c6da680 con 0x55e61c2fd390 2024-07-31T14:54:36.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f94a4a74640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c2f51b0 0x55e61c6cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f94ad229740 10 client.0 _ll_register_callbacks cb 0x55e61c5e64f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:36.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f94a4a74640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c2f51b0 0x55e61c6cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58010/0 (socket says 172.21.15.67:58010) 2024-07-31T14:54:36.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f94a4a74640 1 -- 172.21.15.67:0/2262444692 learned_addr learned my addr 172.21.15.67:0/2262444692 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:36.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f949ffff640 1 --2- 172.21.15.67:0/2262444692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 0x55e61c6cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f94a5275640 1 --2- 172.21.15.67:0/2262444692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c31e070 0x55e61c6e39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f949ffff640 1 -- 172.21.15.67:0/2262444692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c31e070 msgr2=0x55e61c6e39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f949ffff640 1 --2- 172.21.15.67:0/2262444692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c31e070 0x55e61c6e39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f949ffff640 1 -- 172.21.15.67:0/2262444692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c2f51b0 msgr2=0x55e61c6cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f949ffff640 1 --2- 172.21.15.67:0/2262444692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c2f51b0 0x55e61c6cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f949ffff640 1 -- 172.21.15.67:0/2262444692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61c389650 con 0x55e61c2fd390 2024-07-31T14:54:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f94a4a74640 1 --2- 172.21.15.67:0/2262444692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c2f51b0 0x55e61c6cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f949ffff640 1 --2- 172.21.15.67:0/2262444692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 0x55e61c6cf8c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f948c101b90 tx=0x7f948c0cab40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fd5d07ebd7fb46e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f947ffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:54:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.538+0000 7f947ffff640 1 -- 172.21.15.67:0/2262444692 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c049040 con 0x55e61c2fd390 2024-07-31T14:54:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.539+0000 7f94ad229740 1 -- 172.21.15.67:0/2262444692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9498015070 con 0x55e61c2fd390 2024-07-31T14:54:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.539+0000 7f947ffff640 1 -- 172.21.15.67:0/2262444692 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f948c0c7720 con 0x55e61c2fd390 2024-07-31T14:54:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.539+0000 7f947ffff640 1 -- 172.21.15.67:0/2262444692 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f948c0cf680 con 0x55e61c2fd390 2024-07-31T14:54:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.539+0000 7f94ad229740 10 client.26248 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.539+0000 7f94ad229740 1 -- 172.21.15.67:0/2262444692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e61c6e6770 con 0x55e61c2fd390 2024-07-31T14:54:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.539+0000 7f947ffff640 1 -- 172.21.15.67:0/2262444692 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9498015070 con 0x55e61c2fd390 2024-07-31T14:54:36.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.539+0000 7f947ffff640 1 -- 172.21.15.67:0/2262444692 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f948c0cb390 con 0x55e61c2fd390 2024-07-31T14:54:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.539+0000 7f94ad229740 10 client.26248 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.539+0000 7f94ad229740 10 client.26248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.539+0000 7f94ad229740 20 client.26248 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.539+0000 7f94ad229740 1 --2- 172.21.15.67:0/2262444692 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e61c2dc9f0 0x55e61c6e9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.539+0000 7f94ad229740 4 client.26248 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:54:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.539+0000 7f94ad229740 1 -- 172.21.15.67:0/2262444692 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e61c63bb00 con 0x55e61c2dc9f0 2024-07-31T14:54:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.540+0000 7f94a4a74640 1 --2- 172.21.15.67:0/2262444692 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e61c2dc9f0 0x55e61c6e9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.540+0000 7f947ffff640 10 client.26248 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:36.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.540+0000 7f94a4a74640 1 --2- 172.21.15.67:0/2262444692 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e61c2dc9f0 0x55e61c6e9b40 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeb200bd95d8f641 server_cookie=4751079f9454225b in_seq=0 out_seq=0 2024-07-31T14:54:36.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f947ffff640 1 -- 172.21.15.67:0/2262444692 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e61c63bb00 con 0x55e61c2dc9f0 2024-07-31T14:54:36.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f947ffff640 10 client.26248 handle_command_reply: tid=0 2024-07-31T14:54:36.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f94ad229740 1 client.26248 shutdown 2024-07-31T14:54:36.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f94ad229740 1 -- 172.21.15.67:0/2262444692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 msgr2=0x55e61c6cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f94ad229740 1 --2- 172.21.15.67:0/2262444692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 0x55e61c6cf8c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f948c101b90 tx=0x7f948c0cab40 comp rx=0 tx=0).stop 2024-07-31T14:54:36.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f94ad229740 1 -- 172.21.15.67:0/2262444692 shutdown_connections 2024-07-31T14:54:36.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f94ad229740 1 --2- 172.21.15.67:0/2262444692 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e61c2dc9f0 0x55e61c6e9b40 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f94ad229740 1 --2- 172.21.15.67:0/2262444692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c31e070 0x55e61c6e39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f94ad229740 1 --2- 172.21.15.67:0/2262444692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 0x55e61c6cf8c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f94ad229740 1 --2- 172.21.15.67:0/2262444692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c2f51b0 0x55e61c6cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f94ad229740 1 -- 172.21.15.67:0/2262444692 >> 172.21.15.67:0/2262444692 conn(0x55e61c312460 msgr2=0x55e61c6bfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:36.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f947ffff640 0 client.26248 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:36.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f94ad229740 1 -- 172.21.15.67:0/2262444692 shutdown_connections 2024-07-31T14:54:36.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f94ad229740 1 -- 172.21.15.67:0/2262444692 wait complete. 2024-07-31T14:54:36.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.541+0000 7f94ad229740 20 client.26248 trim_cache size 0 max 16384 2024-07-31T14:54:36.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.555+0000 7f94ad229740 1 Processor -- start 2024-07-31T14:54:36.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.555+0000 7f94ad229740 1 -- start start 2024-07-31T14:54:36.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.556+0000 7f94ad229740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c2f51b0 0x55e61c6e37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.556+0000 7f94ad229740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2fd390 0x55e61c6bfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.556+0000 7f94ad229740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c31e070 0x55e61c6c0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.556+0000 7f94ad229740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01ae520 con 0x55e61c2f51b0 2024-07-31T14:54:36.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.556+0000 7f94ad229740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01b01e0 con 0x55e61c2fd390 2024-07-31T14:54:36.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.556+0000 7f94ad229740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01aebf0 con 0x55e61c31e070 2024-07-31T14:54:36.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.556+0000 7f94a4a74640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c2f51b0 0x55e61c6e37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.556+0000 7f94a4a74640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c2f51b0 0x55e61c6e37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58020/0 (socket says 172.21.15.67:58020) 2024-07-31T14:54:36.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.556+0000 7f94a4a74640 1 -- 172.21.15.67:0/4157296568 learned_addr learned my addr 172.21.15.67:0/4157296568 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:36.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.556+0000 7f949ffff640 1 --2- 172.21.15.67:0/4157296568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2fd390 0x55e61c6bfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.556+0000 7f94a5275640 1 --2- 172.21.15.67:0/4157296568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c31e070 0x55e61c6c0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.557+0000 7f949ffff640 1 -- 172.21.15.67:0/4157296568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c31e070 msgr2=0x55e61c6c0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.557+0000 7f949ffff640 1 --2- 172.21.15.67:0/4157296568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c31e070 0x55e61c6c0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.557+0000 7f949ffff640 1 -- 172.21.15.67:0/4157296568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c2f51b0 msgr2=0x55e61c6e37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.557+0000 7f949ffff640 1 --2- 172.21.15.67:0/4157296568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c2f51b0 0x55e61c6e37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.557+0000 7f949ffff640 1 -- 172.21.15.67:0/4157296568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61c6d01c0 con 0x55e61c2fd390 2024-07-31T14:54:36.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.557+0000 7f94a4a74640 1 --2- 172.21.15.67:0/4157296568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c2f51b0 0x55e61c6e37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:36.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.557+0000 7f94a5275640 1 --2- 172.21.15.67:0/4157296568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c31e070 0x55e61c6c0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:54:36.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.557+0000 7f949ffff640 1 --2- 172.21.15.67:0/4157296568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2fd390 0x55e61c6bfd00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f948c0c71b0 tx=0x7f948c003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=120dcbffc7614c2c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:36.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.557+0000 7f948a7fc640 1 -- 172.21.15.67:0/4157296568 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c049070 con 0x55e61c2fd390 2024-07-31T14:54:36.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.557+0000 7f948a7fc640 1 -- 172.21.15.67:0/4157296568 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f948c0d4130 con 0x55e61c2fd390 2024-07-31T14:54:36.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.557+0000 7f948a7fc640 1 -- 172.21.15.67:0/4157296568 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c104450 con 0x55e61c2fd390 2024-07-31T14:54:36.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.558+0000 7f94ad229740 1 -- 172.21.15.67:0/4157296568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2fd390 msgr2=0x55e61c6bfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.558+0000 7f94ad229740 1 --2- 172.21.15.67:0/4157296568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2fd390 0x55e61c6bfd00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f948c0c71b0 tx=0x7f948c003f50 comp rx=0 tx=0).stop 2024-07-31T14:54:36.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.558+0000 7f94ad229740 1 -- 172.21.15.67:0/4157296568 shutdown_connections 2024-07-31T14:54:36.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.558+0000 7f94ad229740 1 --2- 172.21.15.67:0/4157296568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c31e070 0x55e61c6c0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.558+0000 7f94ad229740 1 --2- 172.21.15.67:0/4157296568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2fd390 0x55e61c6bfd00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.558+0000 7f94ad229740 1 --2- 172.21.15.67:0/4157296568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c2f51b0 0x55e61c6e37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.558+0000 7f94ad229740 1 -- 172.21.15.67:0/4157296568 >> 172.21.15.67:0/4157296568 conn(0x55e61c312460 msgr2=0x55e61c6cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:36.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.558+0000 7f94ad229740 1 -- 172.21.15.67:0/4157296568 shutdown_connections 2024-07-31T14:54:36.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.558+0000 7f94ad229740 1 -- 172.21.15.67:0/4157296568 wait complete. 2024-07-31T14:54:36.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.559+0000 7f94ad229740 1 -- start start 2024-07-31T14:54:36.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.559+0000 7f94ad229740 1 Processor -- start 2024-07-31T14:54:36.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.559+0000 7f94ad229740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2f51b0 0x55e61c6daf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.559+0000 7f94ad229740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 0x55e61c6db440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.559+0000 7f94ad229740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c31e070 0x55e61c750300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.559+0000 7f94ad229740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01afb10 con 0x55e61c31e070 2024-07-31T14:54:36.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.559+0000 7f94ad229740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01af380 con 0x55e61c2f51b0 2024-07-31T14:54:36.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.559+0000 7f94ad229740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01ac500 con 0x55e61c2fd390 2024-07-31T14:54:36.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.559+0000 7f94ad229740 10 client.0 _ll_register_callbacks cb 0x7f948c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:36.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.560+0000 7f94a5275640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c31e070 0x55e61c750300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.560+0000 7f949ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 0x55e61c6db440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.560+0000 7f94a5275640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c31e070 0x55e61c750300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58032/0 (socket says 172.21.15.67:58032) 2024-07-31T14:54:36.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.560+0000 7f94a5275640 1 -- 172.21.15.67:0/3081013287 learned_addr learned my addr 172.21.15.67:0/3081013287 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:36.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.560+0000 7f94a4a74640 1 --2- 172.21.15.67:0/3081013287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2f51b0 0x55e61c6daf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.560+0000 7f94a5275640 1 -- 172.21.15.67:0/3081013287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 msgr2=0x55e61c6db440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.560+0000 7f94a5275640 1 --2- 172.21.15.67:0/3081013287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 0x55e61c6db440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.560+0000 7f94a5275640 1 -- 172.21.15.67:0/3081013287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2f51b0 msgr2=0x55e61c6daf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:36.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.560+0000 7f94a5275640 1 --2- 172.21.15.67:0/3081013287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2f51b0 0x55e61c6daf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.560+0000 7f94a5275640 1 -- 172.21.15.67:0/3081013287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61c750840 con 0x55e61c31e070 2024-07-31T14:54:36.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.560+0000 7f949ffff640 1 --2- 172.21.15.67:0/3081013287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 0x55e61c6db440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:36.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.560+0000 7f94a5275640 1 --2- 172.21.15.67:0/3081013287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c31e070 0x55e61c750300 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9498009eb0 tx=0x7f9498009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd027696e569624f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:36.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.560+0000 7f947ffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:36.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.560+0000 7f947ffff640 1 -- 172.21.15.67:0/3081013287 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949801aa50 con 0x55e61c31e070 2024-07-31T14:54:36.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.561+0000 7f947ffff640 1 -- 172.21.15.67:0/3081013287 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f949801abf0 con 0x55e61c31e070 2024-07-31T14:54:36.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.561+0000 7f94ad229740 1 -- 172.21.15.67:0/3081013287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f948c0d4130 con 0x55e61c31e070 2024-07-31T14:54:36.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.561+0000 7f947ffff640 1 -- 172.21.15.67:0/3081013287 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9498034050 con 0x55e61c31e070 2024-07-31T14:54:36.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.561+0000 7f94ad229740 10 client.16614 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:36.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.561+0000 7f94ad229740 1 -- 172.21.15.67:0/3081013287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e61c751a50 con 0x55e61c31e070 2024-07-31T14:54:36.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.561+0000 7f947ffff640 1 -- 172.21.15.67:0/3081013287 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f948c0d4130 con 0x55e61c31e070 2024-07-31T14:54:36.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.561+0000 7f947ffff640 1 -- 172.21.15.67:0/3081013287 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f9498026400 con 0x55e61c31e070 2024-07-31T14:54:36.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.561+0000 7f94ad229740 10 client.16614 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:36.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.561+0000 7f94ad229740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:36.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.561+0000 7f94ad229740 20 client.16614 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:36.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.561+0000 7f94ad229740 1 --2- 172.21.15.67:0/3081013287 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e61c2b1da0 0x55e61c753fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:36.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.561+0000 7f94ad229740 4 client.16614 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:54:36.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.561+0000 7f94ad229740 1 -- 172.21.15.67:0/3081013287 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e61c6375a0 con 0x55e61c2b1da0 2024-07-31T14:54:36.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.562+0000 7f94a4a74640 1 --2- 172.21.15.67:0/3081013287 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e61c2b1da0 0x55e61c753fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:36.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.562+0000 7f94a4a74640 1 --2- 172.21.15.67:0/3081013287 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e61c2b1da0 0x55e61c753fc0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697083370dacc74c server_cookie=1f2595c4bc817b8c in_seq=0 out_seq=0 2024-07-31T14:54:36.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.562+0000 7f947ffff640 10 client.16614 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:36.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f947ffff640 1 -- 172.21.15.67:0/3081013287 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e61c6375a0 con 0x55e61c2b1da0 2024-07-31T14:54:36.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f947ffff640 10 client.16614 handle_command_reply: tid=0 2024-07-31T14:54:36.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f94ad229740 1 client.16614 shutdown 2024-07-31T14:54:36.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f94ad229740 1 -- 172.21.15.67:0/3081013287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c31e070 msgr2=0x55e61c750300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f94ad229740 1 --2- 172.21.15.67:0/3081013287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c31e070 0x55e61c750300 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9498009eb0 tx=0x7f9498009c30 comp rx=0 tx=0).stop 2024-07-31T14:54:36.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f94ad229740 1 -- 172.21.15.67:0/3081013287 shutdown_connections 2024-07-31T14:54:36.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f94ad229740 1 --2- 172.21.15.67:0/3081013287 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e61c2b1da0 0x55e61c753fc0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f947ffff640 0 client.16614 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:36.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f94ad229740 1 --2- 172.21.15.67:0/3081013287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e61c31e070 0x55e61c750300 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f94ad229740 1 --2- 172.21.15.67:0/3081013287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e61c2fd390 0x55e61c6db440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f94ad229740 1 --2- 172.21.15.67:0/3081013287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e61c2f51b0 0x55e61c6daf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f94ad229740 1 -- 172.21.15.67:0/3081013287 >> 172.21.15.67:0/3081013287 conn(0x55e61c312460 msgr2=0x55e61c6c0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:36.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f94ad229740 1 -- 172.21.15.67:0/3081013287 shutdown_connections 2024-07-31T14:54:36.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f94ad229740 1 -- 172.21.15.67:0/3081013287 wait complete. 2024-07-31T14:54:36.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.563+0000 7f94ad229740 20 client.16614 trim_cache size 0 max 16384 2024-07-31T14:54:36.569 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:54:36.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.564+0000 7f94a6cd8640 1 -- 172.21.15.67:0/2265356070 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94900b3d40 msgr2=0x7f94900b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.564+0000 7f94a6cd8640 1 --2- 172.21.15.67:0/2265356070 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94900b3d40 0x7f94900b6250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55e61c6c2310 tx=0x7f9494005f90 comp rx=0 tx=0).stop 2024-07-31T14:54:36.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.564+0000 7f94a6cd8640 1 -- 172.21.15.67:0/2265356070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a0107de0 msgr2=0x7f94a00ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:36.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.564+0000 7f94a6cd8640 1 --2- 172.21.15.67:0/2265356070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a0107de0 0x7f94a00ff560 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f948c006010 tx=0x7f948c004060 comp rx=0 tx=0).stop 2024-07-31T14:54:36.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.564+0000 7f94a6cd8640 1 -- 172.21.15.67:0/2265356070 shutdown_connections 2024-07-31T14:54:36.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.564+0000 7f94a6cd8640 1 --2- 172.21.15.67:0/2265356070 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94900b3d40 0x7f94900b6250 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.564+0000 7f94a6cd8640 1 --2- 172.21.15.67:0/2265356070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a0108730 0x7f94a00ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.564+0000 7f94a6cd8640 1 --2- 172.21.15.67:0/2265356070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a0107de0 0x7f94a00ff560 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.564+0000 7f94a6cd8640 1 --2- 172.21.15.67:0/2265356070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94a01024a0 0x7f94a00ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:36.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.564+0000 7f94a6cd8640 1 -- 172.21.15.67:0/2265356070 >> 172.21.15.67:0/2265356070 conn(0x7f94a0079140 msgr2=0x7f94a0101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:36.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.564+0000 7f94a6cd8640 1 -- 172.21.15.67:0/2265356070 shutdown_connections 2024-07-31T14:54:36.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:36.564+0000 7f94a6cd8640 1 -- 172.21.15.67:0/2265356070 wait complete. 2024-07-31T14:54:36.575 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:54:37.146 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:37 smithi067 ceph-mon[32750]: pgmap v506: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.0 MiB/s wr, 201 op/s 2024-07-31T14:54:37.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:37 smithi073 ceph-mon[36714]: pgmap v506: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.0 MiB/s wr, 201 op/s 2024-07-31T14:54:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:37 smithi077 ceph-mon[37327]: pgmap v506: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.0 MiB/s wr, 201 op/s 2024-07-31T14:54:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.583+0000 7f5ac80f5640 1 -- 172.21.15.67:0/3219464789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ac0108760 msgr2=0x7f5ac010ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.583+0000 7f5ac80f5640 1 --2- 172.21.15.67:0/3219464789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ac0108760 0x7f5ac010ef90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009ca0 tx=0x7f5ab802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.583+0000 7f5ac80f5640 1 -- 172.21.15.67:0/3219464789 shutdown_connections 2024-07-31T14:54:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.583+0000 7f5ac80f5640 1 --2- 172.21.15.67:0/3219464789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ac0108760 0x7f5ac010ef90 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.583+0000 7f5ac80f5640 1 --2- 172.21.15.67:0/3219464789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ac0107e10 0x7f5ac0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.583+0000 7f5ac80f5640 1 --2- 172.21.15.67:0/3219464789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ac01024d0 0x7f5ac01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.583+0000 7f5ac80f5640 1 -- 172.21.15.67:0/3219464789 >> 172.21.15.67:0/3219464789 conn(0x7f5ac0079190 msgr2=0x7f5ac00795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:38.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.583+0000 7f5ac80f5640 1 -- 172.21.15.67:0/3219464789 shutdown_connections 2024-07-31T14:54:38.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.583+0000 7f5ac80f5640 1 -- 172.21.15.67:0/3219464789 wait complete. 2024-07-31T14:54:38.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.584+0000 7f5ac80f5640 1 Processor -- start 2024-07-31T14:54:38.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.584+0000 7f5ac80f5640 1 -- start start 2024-07-31T14:54:38.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.584+0000 7f5ac80f5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ac01024d0 0x7f5ac0199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.584+0000 7f5ac80f5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ac0107e10 0x7f5ac019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.584+0000 7f5ac80f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ac0108760 0x7f5ac019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.584+0000 7f5ac80f5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac0112440 con 0x7f5ac01024d0 2024-07-31T14:54:38.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.584+0000 7f5ac80f5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac01122c0 con 0x7f5ac0107e10 2024-07-31T14:54:38.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.584+0000 7f5ac80f5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac01125c0 con 0x7f5ac0108760 2024-07-31T14:54:38.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.584+0000 7f5ac5e91640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ac01024d0 0x7f5ac0199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.584+0000 7f5ac5e91640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ac01024d0 0x7f5ac0199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58056/0 (socket says 172.21.15.67:58056) 2024-07-31T14:54:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.584+0000 7f5ac5e91640 1 -- 172.21.15.67:0/3456235687 learned_addr learned my addr 172.21.15.67:0/3456235687 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.584+0000 7f5ac6692640 1 --2- 172.21.15.67:0/3456235687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ac0108760 0x7f5ac019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.584+0000 7f5ac5690640 1 --2- 172.21.15.67:0/3456235687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ac0107e10 0x7f5ac019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.585+0000 7f5ac5e91640 1 -- 172.21.15.67:0/3456235687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ac0108760 msgr2=0x7f5ac019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.585+0000 7f5ac5e91640 1 --2- 172.21.15.67:0/3456235687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ac0108760 0x7f5ac019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.585+0000 7f5ac5e91640 1 -- 172.21.15.67:0/3456235687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ac0107e10 msgr2=0x7f5ac019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.585+0000 7f5ac5e91640 1 --2- 172.21.15.67:0/3456235687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ac0107e10 0x7f5ac019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.585+0000 7f5ac5e91640 1 -- 172.21.15.67:0/3456235687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac019fe30 con 0x7f5ac01024d0 2024-07-31T14:54:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.585+0000 7f5ac5e91640 1 --2- 172.21.15.67:0/3456235687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ac01024d0 0x7f5ac0199f80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00b4a0 tx=0x7f5abc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:38.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.585+0000 7f5ab2ffd640 1 -- 172.21.15.67:0/3456235687 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc014070 con 0x7f5ac01024d0 2024-07-31T14:54:38.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.585+0000 7f5ab2ffd640 1 -- 172.21.15.67:0/3456235687 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5abc010ac0 con 0x7f5ac01024d0 2024-07-31T14:54:38.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.585+0000 7f5ac80f5640 1 -- 172.21.15.67:0/3456235687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac01a0120 con 0x7f5ac01024d0 2024-07-31T14:54:38.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.585+0000 7f5ab2ffd640 1 -- 172.21.15.67:0/3456235687 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc017c30 con 0x7f5ac01024d0 2024-07-31T14:54:38.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.585+0000 7f5ac80f5640 1 -- 172.21.15.67:0/3456235687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac01a6940 con 0x7f5ac01024d0 2024-07-31T14:54:38.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.588+0000 7f5ace646740 1 Processor -- start 2024-07-31T14:54:38.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.588+0000 7f5ace646740 1 -- start start 2024-07-31T14:54:38.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.588+0000 7f5ab2ffd640 1 -- 172.21.15.67:0/3456235687 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5abc017070 con 0x7f5ac01024d0 2024-07-31T14:54:38.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.588+0000 7f5ace646740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d133520 0x55a74d4d9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.588+0000 7f5ace646740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4d95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.588+0000 7f5ace646740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d112840 0x55a74d4d9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ace646740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a74d44ee70 con 0x55a74d10a660 2024-07-31T14:54:38.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ace646740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a74d43e370 con 0x55a74d133520 2024-07-31T14:54:38.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ace646740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a74d4423b0 con 0x55a74d112840 2024-07-31T14:54:38.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ac5e91640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4d95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ac5e91640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4d95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58068/0 (socket says 172.21.15.67:58068) 2024-07-31T14:54:38.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ac5e91640 1 -- 172.21.15.67:0/458026573 learned_addr learned my addr 172.21.15.67:0/458026573 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:38.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ac6692640 1 --2- 172.21.15.67:0/458026573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d133520 0x55a74d4d9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ac5690640 1 --2- 172.21.15.67:0/458026573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d112840 0x55a74d4d9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ac6692640 1 -- 172.21.15.67:0/458026573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d112840 msgr2=0x55a74d4d9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:38.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ac6692640 1 --2- 172.21.15.67:0/458026573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d112840 0x55a74d4d9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ac6692640 1 -- 172.21.15.67:0/458026573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 msgr2=0x55a74d4d95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:38.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ac6692640 1 --2- 172.21.15.67:0/458026573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4d95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ac6692640 1 -- 172.21.15.67:0/458026573 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a74d40b040 con 0x55a74d133520 2024-07-31T14:54:38.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ac5e91640 1 --2- 172.21.15.67:0/458026573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4d95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:38.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.589+0000 7f5ab2ffd640 1 --2- 172.21.15.67:0/3456235687 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5aac0b3d70 0x7f5aac0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.590+0000 7f5ac5e91640 1 --2- 172.21.15.67:0/3456235687 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5aac0b3d70 0x7f5aac0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.590+0000 7f5ac6692640 1 --2- 172.21.15.67:0/458026573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d133520 0x55a74d4d9070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8006010 tx=0x7f5ab802f4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d833c0067a3c43a3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:38.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.590+0000 7f5ab2ffd640 1 -- 172.21.15.67:0/3456235687 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f5abc0e3540 con 0x7f5ac01024d0 2024-07-31T14:54:38.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.590+0000 7f5aabfff640 1 -- 172.21.15.67:0/458026573 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8031df0 con 0x55a74d133520 2024-07-31T14:54:38.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.590+0000 7f5aabfff640 1 -- 172.21.15.67:0/458026573 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab8036030 con 0x55a74d133520 2024-07-31T14:54:38.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.590+0000 7f5aabfff640 1 -- 172.21.15.67:0/458026573 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80042b0 con 0x55a74d133520 2024-07-31T14:54:38.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.591+0000 7f5ace646740 1 -- 172.21.15.67:0/458026573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d133520 msgr2=0x55a74d4d9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:38.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.591+0000 7f5ace646740 1 --2- 172.21.15.67:0/458026573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d133520 0x55a74d4d9070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8006010 tx=0x7f5ab802f4c0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.591+0000 7f5ac5e91640 1 --2- 172.21.15.67:0/3456235687 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5aac0b3d70 0x7f5aac0b6280 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0a9c50 tx=0x7f5abc0e9110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:38.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.591+0000 7f5ace646740 1 -- 172.21.15.67:0/458026573 shutdown_connections 2024-07-31T14:54:38.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.591+0000 7f5ace646740 1 --2- 172.21.15.67:0/458026573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d112840 0x55a74d4d9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.591+0000 7f5ace646740 1 --2- 172.21.15.67:0/458026573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4d95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.591+0000 7f5ace646740 1 --2- 172.21.15.67:0/458026573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d133520 0x55a74d4d9070 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.591+0000 7f5ace646740 1 -- 172.21.15.67:0/458026573 >> 172.21.15.67:0/458026573 conn(0x55a74d127910 msgr2=0x55a74d4d6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:38.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.591+0000 7f5ace646740 1 -- 172.21.15.67:0/458026573 shutdown_connections 2024-07-31T14:54:38.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.591+0000 7f5ace646740 1 -- 172.21.15.67:0/458026573 wait complete. 2024-07-31T14:54:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.591+0000 7f5ace646740 1 -- start start 2024-07-31T14:54:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.591+0000 7f5ace646740 1 Processor -- start 2024-07-31T14:54:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ace646740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4e9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ace646740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d112840 0x55a74d4e9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ace646740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 0x55a74d4f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ace646740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a74d2a1bb0 con 0x55a74d10a660 2024-07-31T14:54:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ace646740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a74d3e0ac0 con 0x55a74d112840 2024-07-31T14:54:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ace646740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a74d4eb9a0 con 0x55a74d133520 2024-07-31T14:54:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ace646740 10 client.0 _ll_register_callbacks cb 0x55a74d3e93b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ac6692640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ac6692640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4e9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58070/0 (socket says 172.21.15.67:58070) 2024-07-31T14:54:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ac6692640 1 -- 172.21.15.67:0/3719269832 learned_addr learned my addr 172.21.15.67:0/3719269832 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ac6692640 1 --2- 172.21.15.67:0/3719269832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 0x55a74d4f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ac6692640 1 -- 172.21.15.67:0/3719269832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 msgr2=0x55a74d4f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ac6692640 1 --2- 172.21.15.67:0/3719269832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 0x55a74d4f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ac6692640 1 -- 172.21.15.67:0/3719269832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d112840 msgr2=0x55a74d4e9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ac5690640 1 --2- 172.21.15.67:0/3719269832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d112840 0x55a74d4e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ac6692640 1 --2- 172.21.15.67:0/3719269832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d112840 0x55a74d4e9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.592+0000 7f5ac6692640 1 -- 172.21.15.67:0/3719269832 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a74d19eb00 con 0x55a74d10a660 2024-07-31T14:54:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.593+0000 7f5ac5690640 1 --2- 172.21.15.67:0/3719269832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d112840 0x55a74d4e9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:54:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.593+0000 7f5ac6692640 1 --2- 172.21.15.67:0/3719269832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4e9040 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009620 tx=0x7f5ab8009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b93b9f4b0dc8e2c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.593+0000 7f5aa97fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.593+0000 7f5aa97fa640 1 -- 172.21.15.67:0/3719269832 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8058060 con 0x55a74d10a660 2024-07-31T14:54:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.593+0000 7f5aa97fa640 1 -- 172.21.15.67:0/3719269832 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f5ab80042b0 con 0x55a74d10a660 2024-07-31T14:54:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.593+0000 7f5ace646740 1 -- 172.21.15.67:0/3719269832 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab8036030 con 0x55a74d10a660 2024-07-31T14:54:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.593+0000 7f5aa97fa640 1 -- 172.21.15.67:0/3719269832 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5ab805f050 con 0x55a74d10a660 2024-07-31T14:54:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.593+0000 7f5ace646740 10 client.16626 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.593+0000 7f5ace646740 1 -- 172.21.15.67:0/3719269832 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a74d215960 con 0x55a74d10a660 2024-07-31T14:54:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.593+0000 7f5aa97fa640 1 -- 172.21.15.67:0/3719269832 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab8036030 con 0x55a74d10a660 2024-07-31T14:54:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.594+0000 7f5aa97fa640 1 -- 172.21.15.67:0/3719269832 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f5ab8043070 con 0x55a74d10a660 2024-07-31T14:54:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.594+0000 7f5ace646740 10 client.16626 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.594+0000 7f5ace646740 10 client.16626 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:54:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.594+0000 7f5ace646740 20 client.16626 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.594+0000 7f5ace646740 1 --2- 172.21.15.67:0/3719269832 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a74d0f1ea0 0x55a74d4fab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.594+0000 7f5ace646740 4 client.16626 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:54:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.594+0000 7f5ace646740 1 -- 172.21.15.67:0/3719269832 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a74d450dc0 con 0x55a74d0f1ea0 2024-07-31T14:54:38.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.594+0000 7f5ac5690640 1 --2- 172.21.15.67:0/3719269832 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a74d0f1ea0 0x55a74d4fab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.594+0000 7f5ac5690640 1 --2- 172.21.15.67:0/3719269832 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a74d0f1ea0 0x55a74d4fab90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c5f771d8d8ce5c8 server_cookie=9f3408842568b2a6 in_seq=0 out_seq=0 2024-07-31T14:54:38.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.594+0000 7f5aa97fa640 10 client.16626 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:38.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.595+0000 7f5aa97fa640 1 -- 172.21.15.67:0/3719269832 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a74d450dc0 con 0x55a74d0f1ea0 2024-07-31T14:54:38.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.595+0000 7f5aa97fa640 10 client.16626 handle_command_reply: tid=0 2024-07-31T14:54:38.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.595+0000 7f5ace646740 1 client.16626 shutdown 2024-07-31T14:54:38.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.595+0000 7f5ace646740 1 -- 172.21.15.67:0/3719269832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 msgr2=0x55a74d4e9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:38.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.595+0000 7f5ace646740 1 --2- 172.21.15.67:0/3719269832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4e9040 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009620 tx=0x7f5ab8009890 comp rx=0 tx=0).stop 2024-07-31T14:54:38.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.595+0000 7f5ace646740 1 -- 172.21.15.67:0/3719269832 shutdown_connections 2024-07-31T14:54:38.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.595+0000 7f5ace646740 1 --2- 172.21.15.67:0/3719269832 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a74d0f1ea0 0x55a74d4fab90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.595+0000 7f5ace646740 1 --2- 172.21.15.67:0/3719269832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 0x55a74d4f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.595+0000 7f5ace646740 1 --2- 172.21.15.67:0/3719269832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d112840 0x55a74d4e9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.595+0000 7f5ace646740 1 --2- 172.21.15.67:0/3719269832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4e9040 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.595+0000 7f5aa97fa640 0 client.16626 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:38.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.595+0000 7f5ace646740 1 -- 172.21.15.67:0/3719269832 >> 172.21.15.67:0/3719269832 conn(0x55a74d127910 msgr2=0x55a74d4dc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:38.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.595+0000 7f5ace646740 1 -- 172.21.15.67:0/3719269832 shutdown_connections 2024-07-31T14:54:38.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.595+0000 7f5ace646740 1 -- 172.21.15.67:0/3719269832 wait complete. 2024-07-31T14:54:38.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.596+0000 7f5ace646740 20 client.16626 trim_cache size 0 max 16384 2024-07-31T14:54:38.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.613+0000 7f5ace646740 1 Processor -- start 2024-07-31T14:54:38.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.614+0000 7f5ace646740 1 -- start start 2024-07-31T14:54:38.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.614+0000 7f5ace646740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4dc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.614+0000 7f5ace646740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d112840 0x55a74d5002d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.614+0000 7f5ace646740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 0x55a74d500810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.614+0000 7f5ace646740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac01ae480 con 0x55a74d10a660 2024-07-31T14:54:38.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.614+0000 7f5ace646740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac01b0140 con 0x55a74d112840 2024-07-31T14:54:38.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.614+0000 7f5ace646740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac01aeb50 con 0x55a74d133520 2024-07-31T14:54:38.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.614+0000 7f5ac6692640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4dc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.614+0000 7f5ac6692640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4dc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58076/0 (socket says 172.21.15.67:58076) 2024-07-31T14:54:38.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.614+0000 7f5ac6692640 1 -- 172.21.15.67:0/916908011 learned_addr learned my addr 172.21.15.67:0/916908011 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:38.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.614+0000 7f5ac5690640 1 --2- 172.21.15.67:0/916908011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d112840 0x55a74d5002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.614+0000 7f5ac6692640 1 --2- 172.21.15.67:0/916908011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 0x55a74d500810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.615+0000 7f5ac6692640 1 -- 172.21.15.67:0/916908011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 msgr2=0x55a74d500810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:38.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.615+0000 7f5ac6692640 1 --2- 172.21.15.67:0/916908011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 0x55a74d500810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.615+0000 7f5ac6692640 1 -- 172.21.15.67:0/916908011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d112840 msgr2=0x55a74d5002d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:38.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.615+0000 7f5ac6692640 1 --2- 172.21.15.67:0/916908011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d112840 0x55a74d5002d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.615+0000 7f5ac6692640 1 -- 172.21.15.67:0/916908011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a74d4dbd20 con 0x55a74d10a660 2024-07-31T14:54:38.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.615+0000 7f5ac5690640 1 --2- 172.21.15.67:0/916908011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d112840 0x55a74d5002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:38.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.615+0000 7f5ac6692640 1 --2- 172.21.15.67:0/916908011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4dc820 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009860 tx=0x7f5ab802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17049369feec00a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:38.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.615+0000 7f5aab7fe640 1 -- 172.21.15.67:0/916908011 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8058070 con 0x55a74d10a660 2024-07-31T14:54:38.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.615+0000 7f5aab7fe640 1 -- 172.21.15.67:0/916908011 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab8036040 con 0x55a74d10a660 2024-07-31T14:54:38.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.615+0000 7f5aab7fe640 1 -- 172.21.15.67:0/916908011 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab802fac0 con 0x55a74d10a660 2024-07-31T14:54:38.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.616+0000 7f5ace646740 1 -- 172.21.15.67:0/916908011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 msgr2=0x55a74d4dc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:38.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.616+0000 7f5ace646740 1 --2- 172.21.15.67:0/916908011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4dc820 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009860 tx=0x7f5ab802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.616+0000 7f5ace646740 1 -- 172.21.15.67:0/916908011 shutdown_connections 2024-07-31T14:54:38.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.616+0000 7f5ace646740 1 --2- 172.21.15.67:0/916908011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 0x55a74d500810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.616+0000 7f5ace646740 1 --2- 172.21.15.67:0/916908011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d112840 0x55a74d5002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.616+0000 7f5ace646740 1 --2- 172.21.15.67:0/916908011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d10a660 0x55a74d4dc820 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.616+0000 7f5ace646740 1 -- 172.21.15.67:0/916908011 >> 172.21.15.67:0/916908011 conn(0x55a74d127910 msgr2=0x55a74d4dc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.616+0000 7f5ace646740 1 -- 172.21.15.67:0/916908011 shutdown_connections 2024-07-31T14:54:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.616+0000 7f5ace646740 1 -- 172.21.15.67:0/916908011 wait complete. 2024-07-31T14:54:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.617+0000 7f5ace646740 1 -- start start 2024-07-31T14:54:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.617+0000 7f5ace646740 1 Processor -- start 2024-07-31T14:54:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.617+0000 7f5ace646740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d10a660 0x55a74d4eccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.617+0000 7f5ace646740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d112840 0x55a74d4ed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.617+0000 7f5ace646740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 0x55a74d4ed730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.617+0000 7f5ace646740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac01afa70 con 0x55a74d112840 2024-07-31T14:54:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.617+0000 7f5ace646740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac01af2e0 con 0x55a74d10a660 2024-07-31T14:54:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.617+0000 7f5ace646740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac01ac460 con 0x55a74d133520 2024-07-31T14:54:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.617+0000 7f5ace646740 10 client.0 _ll_register_callbacks cb 0x7f5abc0e9060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.617+0000 7f5ac5690640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d112840 0x55a74d4ed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.617+0000 7f5ac5690640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d112840 0x55a74d4ed1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58078/0 (socket says 172.21.15.67:58078) 2024-07-31T14:54:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.617+0000 7f5ac5690640 1 -- 172.21.15.67:0/2126112557 learned_addr learned my addr 172.21.15.67:0/2126112557 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.618+0000 7f5ac6692640 1 --2- 172.21.15.67:0/2126112557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d10a660 0x55a74d4eccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.618+0000 7f5ac6692640 1 --2- 172.21.15.67:0/2126112557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 0x55a74d4ed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.618+0000 7f5ac5690640 1 -- 172.21.15.67:0/2126112557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 msgr2=0x55a74d4ed730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.618+0000 7f5ac5690640 1 --2- 172.21.15.67:0/2126112557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 0x55a74d4ed730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.618+0000 7f5ac5690640 1 -- 172.21.15.67:0/2126112557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d10a660 msgr2=0x55a74d4eccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.618+0000 7f5ac5690640 1 --2- 172.21.15.67:0/2126112557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d10a660 0x55a74d4eccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.618+0000 7f5ac5690640 1 -- 172.21.15.67:0/2126112557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a74d562400 con 0x55a74d112840 2024-07-31T14:54:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.618+0000 7f5ac6692640 1 --2- 172.21.15.67:0/2126112557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d10a660 0x55a74d4eccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:54:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.618+0000 7f5ac5690640 1 --2- 172.21.15.67:0/2126112557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d112840 0x55a74d4ed1f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0107690 tx=0x7f5ab400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cde17a05e4c51ef server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.618+0000 7f5aa97fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.618+0000 7f5aa97fa640 1 -- 172.21.15.67:0/2126112557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4023640 con 0x55a74d112840 2024-07-31T14:54:38.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.618+0000 7f5aa97fa640 1 -- 172.21.15.67:0/2126112557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f5ab40237e0 con 0x55a74d112840 2024-07-31T14:54:38.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.618+0000 7f5ace646740 1 -- 172.21.15.67:0/2126112557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab8036040 con 0x55a74d112840 2024-07-31T14:54:38.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.618+0000 7f5aa97fa640 1 -- 172.21.15.67:0/2126112557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5ab402c050 con 0x55a74d112840 2024-07-31T14:54:38.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.619+0000 7f5ace646740 10 client.16632 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:38.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.619+0000 7f5ace646740 1 -- 172.21.15.67:0/2126112557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a74d563720 con 0x55a74d112840 2024-07-31T14:54:38.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.619+0000 7f5aa97fa640 1 -- 172.21.15.67:0/2126112557 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab8036040 con 0x55a74d112840 2024-07-31T14:54:38.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.619+0000 7f5aa97fa640 1 -- 172.21.15.67:0/2126112557 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f5ab4021020 con 0x55a74d112840 2024-07-31T14:54:38.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.619+0000 7f5ace646740 10 client.16632 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:38.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.619+0000 7f5ace646740 10 client.16632 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:54:38.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.619+0000 7f5ace646740 20 client.16632 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:38.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.619+0000 7f5ace646740 1 --2- 172.21.15.67:0/2126112557 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a74d0c7250 0x55a74d566000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:38.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.619+0000 7f5ace646740 4 client.16632 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "40ba138f", "release": true, "await": true}] 2024-07-31T14:54:38.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.619+0000 7f5ace646740 1 -- 172.21.15.67:0/2126112557 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "40ba138f", "release": true, "await": true}) -- 0x55a74d44c860 con 0x55a74d0c7250 2024-07-31T14:54:38.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.619+0000 7f5ac6692640 1 --2- 172.21.15.67:0/2126112557 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a74d0c7250 0x55a74d566000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:38.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.620+0000 7f5aa97fa640 10 client.16632 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:38.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:38.620+0000 7f5ac6692640 1 --2- 172.21.15.67:0/2126112557 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a74d0c7250 0x55a74d566000 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c2ec007f1c9f5a6 server_cookie=ba5a150976d0bd96 in_seq=0 out_seq=0 2024-07-31T14:54:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.182+0000 7f5aa97fa640 1 -- 172.21.15.67:0/2126112557 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a74d44c860 con 0x55a74d0c7250 2024-07-31T14:54:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.182+0000 7f5aa97fa640 10 client.16632 handle_command_reply: tid=0 2024-07-31T14:54:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.183+0000 7f5ace646740 1 client.16632 shutdown 2024-07-31T14:54:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.183+0000 7f5ace646740 1 -- 172.21.15.67:0/2126112557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d112840 msgr2=0x55a74d4ed1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.183+0000 7f5ace646740 1 --2- 172.21.15.67:0/2126112557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d112840 0x55a74d4ed1f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0107690 tx=0x7f5ab400cef0 comp rx=0 tx=0).stop 2024-07-31T14:54:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.183+0000 7f5ace646740 1 -- 172.21.15.67:0/2126112557 shutdown_connections 2024-07-31T14:54:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.183+0000 7f5ace646740 1 --2- 172.21.15.67:0/2126112557 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a74d0c7250 0x55a74d566000 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.183+0000 7f5ace646740 1 --2- 172.21.15.67:0/2126112557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a74d133520 0x55a74d4ed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.183+0000 7f5ace646740 1 --2- 172.21.15.67:0/2126112557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a74d112840 0x55a74d4ed1f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.183+0000 7f5aa97fa640 0 client.16632 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.183+0000 7f5ace646740 1 --2- 172.21.15.67:0/2126112557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74d10a660 0x55a74d4eccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:39.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.183+0000 7f5ace646740 1 -- 172.21.15.67:0/2126112557 >> 172.21.15.67:0/2126112557 conn(0x55a74d127910 msgr2=0x55a74d4d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:39.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.183+0000 7f5ace646740 1 -- 172.21.15.67:0/2126112557 shutdown_connections 2024-07-31T14:54:39.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.183+0000 7f5ace646740 1 -- 172.21.15.67:0/2126112557 wait complete. 2024-07-31T14:54:39.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.183+0000 7f5ace646740 20 client.16632 trim_cache size 0 max 16384 2024-07-31T14:54:39.186 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:54:39.186 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 39, 2024-07-31T14:54:39.186 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T14:54:39.186 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 8, 2024-07-31T14:54:39.186 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T14:54:39.187 INFO:teuthology.orchestra.run.smithi067.stdout: "40ba138f": { 2024-07-31T14:54:39.187 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 8, 2024-07-31T14:54:39.187 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T14:54:39.187 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:54:39.187 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T14:54:39.187 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:54:39.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:54:39.187 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T14:54:39.187 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 143.4, 2024-07-31T14:54:39.187 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T14:54:39.187 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T14:54:39.187 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T14:54:39.188 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:54:39.188 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T14:54:39.188 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:54:39.188 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:54:39.188 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:54:39.188 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:54:39.188 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:54:39.188 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:54:39.188 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:54:39.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.184+0000 7f5ac80f5640 1 -- 172.21.15.67:0/3456235687 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5aac0b3d70 msgr2=0x7f5aac0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:39.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.184+0000 7f5ac80f5640 1 --2- 172.21.15.67:0/3456235687 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5aac0b3d70 0x7f5aac0b6280 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0a9c50 tx=0x7f5abc0e9110 comp rx=0 tx=0).stop 2024-07-31T14:54:39.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.184+0000 7f5ac80f5640 1 -- 172.21.15.67:0/3456235687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ac01024d0 msgr2=0x7f5ac0199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:39.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.184+0000 7f5ac80f5640 1 --2- 172.21.15.67:0/3456235687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ac01024d0 0x7f5ac0199f80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00b4a0 tx=0x7f5abc00b960 comp rx=0 tx=0).stop 2024-07-31T14:54:39.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.184+0000 7f5ac80f5640 1 -- 172.21.15.67:0/3456235687 shutdown_connections 2024-07-31T14:54:39.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.184+0000 7f5ac80f5640 1 --2- 172.21.15.67:0/3456235687 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5aac0b3d70 0x7f5aac0b6280 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:39.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.184+0000 7f5ac80f5640 1 --2- 172.21.15.67:0/3456235687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ac0108760 0x7f5ac019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:39.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.184+0000 7f5ac80f5640 1 --2- 172.21.15.67:0/3456235687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ac0107e10 0x7f5ac019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:39.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.184+0000 7f5ac80f5640 1 --2- 172.21.15.67:0/3456235687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ac01024d0 0x7f5ac0199f80 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:39.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.184+0000 7f5ac80f5640 1 -- 172.21.15.67:0/3456235687 >> 172.21.15.67:0/3456235687 conn(0x7f5ac0079190 msgr2=0x7f5ac006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.184+0000 7f5ac80f5640 1 -- 172.21.15.67:0/3456235687 shutdown_connections 2024-07-31T14:54:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:39.184+0000 7f5ac80f5640 1 -- 172.21.15.67:0/3456235687 wait complete. 2024-07-31T14:54:39.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:39 smithi073 ceph-mon[36714]: pgmap v507: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 417 KiB/s wr, 94 op/s 2024-07-31T14:54:39.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:39 smithi077 ceph-mon[37327]: pgmap v507: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 417 KiB/s wr, 94 op/s 2024-07-31T14:54:39.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:39 smithi067 ceph-mon[32750]: pgmap v507: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 417 KiB/s wr, 94 op/s 2024-07-31T14:54:39.882 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '40ba138f', total seconds elapsed: 34.6 2024-07-31T14:54:40.233 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T14:54:40.233 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T14:54:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:54:41.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:41 smithi073 ceph-mon[36714]: pgmap v508: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 113 KiB/s wr, 29 op/s 2024-07-31T14:54:41.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:41 smithi077 ceph-mon[37327]: pgmap v508: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 113 KiB/s wr, 29 op/s 2024-07-31T14:54:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:41 smithi067 ceph-mon[32750]: pgmap v508: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 113 KiB/s wr, 29 op/s 2024-07-31T14:54:42.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.608+0000 7fcf890a9640 1 -- 172.21.15.67:0/3633543935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf84107dd0 msgr2=0x7fcf841081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:42.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.608+0000 7fcf890a9640 1 --2- 172.21.15.67:0/3633543935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf84107dd0 0x7fcf841081e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009900 tx=0x7fcf7802a410 comp rx=0 tx=0).stop 2024-07-31T14:54:42.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.609+0000 7fcf890a9640 1 -- 172.21.15.67:0/3633543935 shutdown_connections 2024-07-31T14:54:42.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.609+0000 7fcf890a9640 1 --2- 172.21.15.67:0/3633543935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf84108720 0x7fcf8410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.609+0000 7fcf890a9640 1 --2- 172.21.15.67:0/3633543935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf84107dd0 0x7fcf841081e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.609+0000 7fcf890a9640 1 --2- 172.21.15.67:0/3633543935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf84102490 0x7fcf841028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.609+0000 7fcf890a9640 1 -- 172.21.15.67:0/3633543935 >> 172.21.15.67:0/3633543935 conn(0x7fcf84079160 msgr2=0x7fcf84079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:42.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.609+0000 7fcf890a9640 1 -- 172.21.15.67:0/3633543935 shutdown_connections 2024-07-31T14:54:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.609+0000 7fcf890a9640 1 -- 172.21.15.67:0/3633543935 wait complete. 2024-07-31T14:54:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.609+0000 7fcf890a9640 1 Processor -- start 2024-07-31T14:54:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.610+0000 7fcf890a9640 1 -- start start 2024-07-31T14:54:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.610+0000 7fcf890a9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf84102490 0x7fcf84199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.610+0000 7fcf890a9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf84107dd0 0x7fcf8419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.610+0000 7fcf890a9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf84108720 0x7fcf8419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.610+0000 7fcf890a9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcf84112350 con 0x7fcf84107dd0 2024-07-31T14:54:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.610+0000 7fcf890a9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841121d0 con 0x7fcf84102490 2024-07-31T14:54:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.610+0000 7fcf890a9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841124d0 con 0x7fcf84108720 2024-07-31T14:54:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.610+0000 7fcf8259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf84107dd0 0x7fcf8419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.610+0000 7fcf8259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf84107dd0 0x7fcf8419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36664/0 (socket says 172.21.15.67:36664) 2024-07-31T14:54:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.610+0000 7fcf8259c640 1 -- 172.21.15.67:0/1392041867 learned_addr learned my addr 172.21.15.67:0/1392041867 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.610+0000 7fcf8359e640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf84108720 0x7fcf8419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.610+0000 7fcf82d9d640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf84102490 0x7fcf84199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.611+0000 7fcf8359e640 1 -- 172.21.15.67:0/1392041867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf84102490 msgr2=0x7fcf84199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.611+0000 7fcf8359e640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf84102490 0x7fcf84199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.611+0000 7fcf8359e640 1 -- 172.21.15.67:0/1392041867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf84107dd0 msgr2=0x7fcf8419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.611+0000 7fcf8359e640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf84107dd0 0x7fcf8419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.611+0000 7fcf8359e640 1 -- 172.21.15.67:0/1392041867 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8419fe00 con 0x7fcf84108720 2024-07-31T14:54:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.611+0000 7fcf8259c640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf84107dd0 0x7fcf8419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.611+0000 7fcf8359e640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf84108720 0x7fcf8419f6d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009fd0 tx=0x7fcf7c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.611+0000 7fcf6bfff640 1 -- 172.21.15.67:0/1392041867 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c00ce20 con 0x7fcf84108720 2024-07-31T14:54:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.611+0000 7fcf890a9640 1 -- 172.21.15.67:0/1392041867 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf841a00f0 con 0x7fcf84108720 2024-07-31T14:54:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.611+0000 7fcf6bfff640 1 -- 172.21.15.67:0/1392041867 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf7c005020 con 0x7fcf84108720 2024-07-31T14:54:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.611+0000 7fcf6bfff640 1 -- 172.21.15.67:0/1392041867 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c010690 con 0x7fcf84108720 2024-07-31T14:54:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.611+0000 7fcf890a9640 1 -- 172.21.15.67:0/1392041867 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf841a6850 con 0x7fcf84108720 2024-07-31T14:54:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.614+0000 7fcf6bfff640 1 -- 172.21.15.67:0/1392041867 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fcf7c010830 con 0x7fcf84108720 2024-07-31T14:54:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.614+0000 7fcf8f5fa740 1 Processor -- start 2024-07-31T14:54:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.614+0000 7fcf8f5fa740 1 -- start start 2024-07-31T14:54:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.614+0000 7fcf8f5fa740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a36350 0x5571c5ddb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.614+0000 7fcf8f5fa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5ddb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.614+0000 7fcf8f5fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a15670 0x5571c5ddbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.614+0000 7fcf8f5fa740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5571c5cc32f0 con 0x5571c5a36350 2024-07-31T14:54:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.614+0000 7fcf8f5fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5571c5d51a80 con 0x5571c5a15670 2024-07-31T14:54:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.614+0000 7fcf8f5fa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5571c5d40f80 con 0x5571c5a0d490 2024-07-31T14:54:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.614+0000 7fcf8259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a36350 0x5571c5ddb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf82d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5ddb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf82d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5ddb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35778/0 (socket says 172.21.15.67:35778) 2024-07-31T14:54:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf82d9d640 1 -- 172.21.15.67:0/3368413316 learned_addr learned my addr 172.21.15.67:0/3368413316 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:42.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf8259c640 1 --2- 172.21.15.67:0/3368413316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a15670 0x5571c5ddbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf82d9d640 1 -- 172.21.15.67:0/3368413316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a15670 msgr2=0x5571c5ddbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:42.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf82d9d640 1 --2- 172.21.15.67:0/3368413316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a15670 0x5571c5ddbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf82d9d640 1 -- 172.21.15.67:0/3368413316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a36350 msgr2=0x5571c5ddb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:42.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf82d9d640 1 --2- 172.21.15.67:0/3368413316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a36350 0x5571c5ddb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf82d9d640 1 -- 172.21.15.67:0/3368413316 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c5aa1930 con 0x5571c5a0d490 2024-07-31T14:54:42.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf8259c640 1 --2- 172.21.15.67:0/3368413316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a15670 0x5571c5ddbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:54:42.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf6bfff640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcf6c0b3d40 0x7fcf6c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf82d9d640 1 --2- 172.21.15.67:0/3368413316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5ddb8f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400bd10 tx=0x7fcf740107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a8e600da8819e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:42.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf8259c640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcf6c0b3d40 0x7fcf6c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf6bfff640 1 -- 172.21.15.67:0/1392041867 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fcf7c0e4a00 con 0x7fcf84108720 2024-07-31T14:54:42.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf697fa640 1 -- 172.21.15.67:0/3368413316 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74004030 con 0x5571c5a0d490 2024-07-31T14:54:42.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf697fa640 1 -- 172.21.15.67:0/3368413316 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf740035f0 con 0x5571c5a0d490 2024-07-31T14:54:42.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.615+0000 7fcf697fa640 1 -- 172.21.15.67:0/3368413316 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf740117b0 con 0x5571c5a0d490 2024-07-31T14:54:42.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.616+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/3368413316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 msgr2=0x5571c5ddb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:42.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.616+0000 7fcf8259c640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcf6c0b3d40 0x7fcf6c0b6250 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcf78032fd0 tx=0x7fcf78032f40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:42.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.616+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/3368413316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5ddb8f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400bd10 tx=0x7fcf740107a0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.616+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/3368413316 shutdown_connections 2024-07-31T14:54:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.616+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/3368413316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a15670 0x5571c5ddbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.616+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/3368413316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5ddb8f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.616+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/3368413316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a36350 0x5571c5ddb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.616+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/3368413316 >> 172.21.15.67:0/3368413316 conn(0x5571c5a2a740 msgr2=0x5571c5dd92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.616+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/3368413316 shutdown_connections 2024-07-31T14:54:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.616+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/3368413316 wait complete. 2024-07-31T14:54:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.617+0000 7fcf8f5fa740 1 -- start start 2024-07-31T14:54:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.617+0000 7fcf8f5fa740 1 Processor -- start 2024-07-31T14:54:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.617+0000 7fcf8f5fa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5de6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.617+0000 7fcf8f5fa740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a15670 0x5571c5de7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.617+0000 7fcf8f5fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a36350 0x5571c5df2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.617+0000 7fcf8f5fa740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5571c5ce34f0 con 0x5571c5a15670 2024-07-31T14:54:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.617+0000 7fcf8f5fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5571c5d44fc0 con 0x5571c5a36350 2024-07-31T14:54:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.617+0000 7fcf8f5fa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5571c5ba4890 con 0x5571c5a0d490 2024-07-31T14:54:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.617+0000 7fcf8f5fa740 10 client.0 _ll_register_callbacks cb 0x5571c5ced500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.617+0000 7fcf8259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a15670 0x5571c5de7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.617+0000 7fcf82d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5de6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.617+0000 7fcf8259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a15670 0x5571c5de7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36684/0 (socket says 172.21.15.67:36684) 2024-07-31T14:54:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.617+0000 7fcf8259c640 1 -- 172.21.15.67:0/3506804333 learned_addr learned my addr 172.21.15.67:0/3506804333 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.617+0000 7fcf8359e640 1 --2- 172.21.15.67:0/3506804333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a36350 0x5571c5df2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.618+0000 7fcf8259c640 1 -- 172.21.15.67:0/3506804333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 msgr2=0x5571c5de6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.618+0000 7fcf8259c640 1 --2- 172.21.15.67:0/3506804333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5de6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.618+0000 7fcf8259c640 1 -- 172.21.15.67:0/3506804333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a36350 msgr2=0x5571c5df2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.618+0000 7fcf8259c640 1 --2- 172.21.15.67:0/3506804333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a36350 0x5571c5df2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.618+0000 7fcf8259c640 1 -- 172.21.15.67:0/3506804333 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c5cf2c50 con 0x5571c5a15670 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.618+0000 7fcf8259c640 1 --2- 172.21.15.67:0/3506804333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a15670 0x5571c5de7440 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcf78006010 tx=0x7fcf7802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2759442d8db1a950 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.618+0000 7fcf66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.619+0000 7fcf66ff5640 1 -- 172.21.15.67:0/3506804333 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7804e070 con 0x5571c5a15670 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.619+0000 7fcf66ff5640 1 -- 172.21.15.67:0/3506804333 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fcf78003ac0 con 0x5571c5a15670 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.619+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/3506804333 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf740035f0 con 0x5571c5a15670 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.619+0000 7fcf66ff5640 1 -- 172.21.15.67:0/3506804333 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcf78035040 con 0x5571c5a15670 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.619+0000 7fcf8f5fa740 10 client.16644 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.619+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/3506804333 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5571c5df5920 con 0x5571c5a15670 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.619+0000 7fcf66ff5640 1 -- 172.21.15.67:0/3506804333 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf740035f0 con 0x5571c5a15670 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.619+0000 7fcf66ff5640 1 -- 172.21.15.67:0/3506804333 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7fcf7803c440 con 0x5571c5a15670 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.619+0000 7fcf8f5fa740 10 client.16644 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.619+0000 7fcf8f5fa740 10 client.16644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.619+0000 7fcf8f5fa740 20 client.16644 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.619+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/3506804333 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5571c59f4cd0 0x5571c5df8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.619+0000 7fcf8f5fa740 4 client.16644 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:54:42.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.619+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/3506804333 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571c5d539d0 con 0x5571c59f4cd0 2024-07-31T14:54:42.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.620+0000 7fcf82d9d640 1 --2- 172.21.15.67:0/3506804333 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5571c59f4cd0 0x5571c5df8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.620+0000 7fcf82d9d640 1 --2- 172.21.15.67:0/3506804333 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5571c59f4cd0 0x5571c5df8dd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3174f91527467519 server_cookie=a5f662a68db66f0e in_seq=0 out_seq=0 2024-07-31T14:54:42.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.620+0000 7fcf66ff5640 10 client.16644 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:42.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf66ff5640 1 -- 172.21.15.67:0/3506804333 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571c5d539d0 con 0x5571c59f4cd0 2024-07-31T14:54:42.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf66ff5640 10 client.16644 handle_command_reply: tid=0 2024-07-31T14:54:42.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf8f5fa740 1 client.16644 shutdown 2024-07-31T14:54:42.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/3506804333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a15670 msgr2=0x5571c5de7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:42.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/3506804333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a15670 0x5571c5de7440 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcf78006010 tx=0x7fcf7802a410 comp rx=0 tx=0).stop 2024-07-31T14:54:42.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/3506804333 shutdown_connections 2024-07-31T14:54:42.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/3506804333 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5571c59f4cd0 0x5571c5df8dd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/3506804333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a36350 0x5571c5df2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/3506804333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a15670 0x5571c5de7440 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/3506804333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5de6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf66ff5640 0 client.16644 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:42.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/3506804333 >> 172.21.15.67:0/3506804333 conn(0x5571c5a2a740 msgr2=0x5571c5dde590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:42.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/3506804333 shutdown_connections 2024-07-31T14:54:42.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/3506804333 wait complete. 2024-07-31T14:54:42.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.621+0000 7fcf8f5fa740 20 client.16644 trim_cache size 0 max 16384 2024-07-31T14:54:42.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.641+0000 7fcf8f5fa740 1 Processor -- start 2024-07-31T14:54:42.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.641+0000 7fcf8f5fa740 1 -- start start 2024-07-31T14:54:42.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8f5fa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5ddcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8f5fa740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a15670 0x5571c5de14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8f5fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a36350 0x5571c5df2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8f5fa740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841ae590 con 0x5571c5a15670 2024-07-31T14:54:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8f5fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841b0250 con 0x5571c5a36350 2024-07-31T14:54:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8f5fa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841aec60 con 0x5571c5a0d490 2024-07-31T14:54:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a15670 0x5571c5de14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf82d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5ddcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a15670 0x5571c5de14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36700/0 (socket says 172.21.15.67:36700) 2024-07-31T14:54:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8259c640 1 -- 172.21.15.67:0/763109110 learned_addr learned my addr 172.21.15.67:0/763109110 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8359e640 1 --2- 172.21.15.67:0/763109110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a36350 0x5571c5df2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8259c640 1 -- 172.21.15.67:0/763109110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 msgr2=0x5571c5ddcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8259c640 1 --2- 172.21.15.67:0/763109110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5ddcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8259c640 1 -- 172.21.15.67:0/763109110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a36350 msgr2=0x5571c5df2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8259c640 1 --2- 172.21.15.67:0/763109110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a36350 0x5571c5df2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8259c640 1 -- 172.21.15.67:0/763109110 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c5de7fc0 con 0x5571c5a15670 2024-07-31T14:54:42.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf82d9d640 1 --2- 172.21.15.67:0/763109110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5ddcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.642+0000 7fcf8259c640 1 --2- 172.21.15.67:0/763109110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a15670 0x5571c5de14d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fcf78000c00 tx=0x7fcf78005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39817fad63c56ef0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.643+0000 7fcf68ff9640 1 -- 172.21.15.67:0/763109110 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7804e070 con 0x5571c5a15670 2024-07-31T14:54:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.643+0000 7fcf68ff9640 1 -- 172.21.15.67:0/763109110 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf78003d70 con 0x5571c5a15670 2024-07-31T14:54:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.643+0000 7fcf68ff9640 1 -- 172.21.15.67:0/763109110 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7804a6a0 con 0x5571c5a15670 2024-07-31T14:54:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.644+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/763109110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a15670 msgr2=0x5571c5de14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.644+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/763109110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a15670 0x5571c5de14d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fcf78000c00 tx=0x7fcf78005780 comp rx=0 tx=0).stop 2024-07-31T14:54:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.644+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/763109110 shutdown_connections 2024-07-31T14:54:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.644+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/763109110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a36350 0x5571c5df2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.644+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/763109110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a15670 0x5571c5de14d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.644+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/763109110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a0d490 0x5571c5ddcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.644+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/763109110 >> 172.21.15.67:0/763109110 conn(0x5571c5a2a740 msgr2=0x5571c5dd7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.644+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/763109110 shutdown_connections 2024-07-31T14:54:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.644+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/763109110 wait complete. 2024-07-31T14:54:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.644+0000 7fcf8f5fa740 1 -- start start 2024-07-31T14:54:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.644+0000 7fcf8f5fa740 1 Processor -- start 2024-07-31T14:54:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf8f5fa740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a0d490 0x5571c5def010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf8f5fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a15670 0x5571c5def5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf8f5fa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a36350 0x5571c5dea670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf8f5fa740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841afb80 con 0x5571c5a0d490 2024-07-31T14:54:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf8f5fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841af3f0 con 0x5571c5a15670 2024-07-31T14:54:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf8f5fa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841ac570 con 0x5571c5a36350 2024-07-31T14:54:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf8f5fa740 10 client.0 _ll_register_callbacks cb 0x7fcf78004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf82d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a0d490 0x5571c5def010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf82d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a0d490 0x5571c5def010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36706/0 (socket says 172.21.15.67:36706) 2024-07-31T14:54:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf82d9d640 1 -- 172.21.15.67:0/377763276 learned_addr learned my addr 172.21.15.67:0/377763276 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf8359e640 1 --2- 172.21.15.67:0/377763276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a36350 0x5571c5dea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf8259c640 1 --2- 172.21.15.67:0/377763276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a15670 0x5571c5def5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf82d9d640 1 -- 172.21.15.67:0/377763276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a36350 msgr2=0x5571c5dea670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf82d9d640 1 --2- 172.21.15.67:0/377763276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a36350 0x5571c5dea670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf82d9d640 1 -- 172.21.15.67:0/377763276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a15670 msgr2=0x5571c5def5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf82d9d640 1 --2- 172.21.15.67:0/377763276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a15670 0x5571c5def5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf82d9d640 1 -- 172.21.15.67:0/377763276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c5deae00 con 0x5571c5a0d490 2024-07-31T14:54:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf82d9d640 1 --2- 172.21.15.67:0/377763276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a0d490 0x5571c5def010 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400bd10 tx=0x7fcf74003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d2c86fd97fa3b2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.645+0000 7fcf66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.646+0000 7fcf66ff5640 1 -- 172.21.15.67:0/377763276 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74038060 con 0x5571c5a0d490 2024-07-31T14:54:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.646+0000 7fcf66ff5640 1 -- 172.21.15.67:0/377763276 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fcf74003a00 con 0x5571c5a0d490 2024-07-31T14:54:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.646+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/377763276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf78003d70 con 0x5571c5a0d490 2024-07-31T14:54:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.646+0000 7fcf66ff5640 1 -- 172.21.15.67:0/377763276 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcf7403f050 con 0x5571c5a0d490 2024-07-31T14:54:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.646+0000 7fcf8f5fa740 10 client.16650 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.646+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/377763276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5571c5e611d0 con 0x5571c5a0d490 2024-07-31T14:54:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.646+0000 7fcf66ff5640 1 -- 172.21.15.67:0/377763276 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf78003d70 con 0x5571c5a0d490 2024-07-31T14:54:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.646+0000 7fcf66ff5640 1 -- 172.21.15.67:0/377763276 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7fcf74032070 con 0x5571c5a0d490 2024-07-31T14:54:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.646+0000 7fcf8f5fa740 10 client.16650 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.646+0000 7fcf8f5fa740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.646+0000 7fcf8f5fa740 20 client.16650 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.646+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/377763276 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5571c59ca080 0x5571c5e637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.646+0000 7fcf8f5fa740 4 client.16650 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T14:54:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.646+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/377763276 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5571c5d4f470 con 0x5571c59ca080 2024-07-31T14:54:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.647+0000 7fcf8259c640 1 --2- 172.21.15.67:0/377763276 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5571c59ca080 0x5571c5e637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.647+0000 7fcf8259c640 1 --2- 172.21.15.67:0/377763276 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5571c59ca080 0x5571c5e637b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d494c1981214287b server_cookie=2d6ca0e3b7bb9e27 in_seq=0 out_seq=0 2024-07-31T14:54:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.647+0000 7fcf66ff5640 10 client.16650 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf66ff5640 1 -- 172.21.15.67:0/377763276 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5571c5d4f470 con 0x5571c59ca080 2024-07-31T14:54:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf66ff5640 10 client.16650 handle_command_reply: tid=0 2024-07-31T14:54:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf8f5fa740 1 client.16650 shutdown 2024-07-31T14:54:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/377763276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a0d490 msgr2=0x5571c5def010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/377763276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a0d490 0x5571c5def010 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400bd10 tx=0x7fcf74003da0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/377763276 shutdown_connections 2024-07-31T14:54:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/377763276 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5571c59ca080 0x5571c5e637b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/377763276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571c5a36350 0x5571c5dea670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/377763276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571c5a15670 0x5571c5def5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf8f5fa740 1 --2- 172.21.15.67:0/377763276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571c5a0d490 0x5571c5def010 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/377763276 >> 172.21.15.67:0/377763276 conn(0x5571c5a2a740 msgr2=0x5571c5de8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf66ff5640 0 client.16650 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/377763276 shutdown_connections 2024-07-31T14:54:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf8f5fa740 1 -- 172.21.15.67:0/377763276 wait complete. 2024-07-31T14:54:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.648+0000 7fcf8f5fa740 20 client.16650 trim_cache size 0 max 16384 2024-07-31T14:54:42.654 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:54:42.654 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T14:54:42.655 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "7c6fa637-63b0-4564-ba95-bca6c2bef18d", 2024-07-31T14:54:42.655 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T14:54:42.655 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:54:42.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.649+0000 7fcf890a9640 1 -- 172.21.15.67:0/1392041867 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcf6c0b3d40 msgr2=0x7fcf6c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.649+0000 7fcf890a9640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcf6c0b3d40 0x7fcf6c0b6250 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcf78032fd0 tx=0x7fcf78032f40 comp rx=0 tx=0).stop 2024-07-31T14:54:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.649+0000 7fcf890a9640 1 -- 172.21.15.67:0/1392041867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf84108720 msgr2=0x7fcf8419f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.649+0000 7fcf890a9640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf84108720 0x7fcf8419f6d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009fd0 tx=0x7fcf7c00ef10 comp rx=0 tx=0).stop 2024-07-31T14:54:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.649+0000 7fcf890a9640 1 -- 172.21.15.67:0/1392041867 shutdown_connections 2024-07-31T14:54:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.649+0000 7fcf890a9640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcf6c0b3d40 0x7fcf6c0b6250 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.649+0000 7fcf890a9640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf84108720 0x7fcf8419f6d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.649+0000 7fcf890a9640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf84107dd0 0x7fcf8419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.649+0000 7fcf890a9640 1 --2- 172.21.15.67:0/1392041867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf84102490 0x7fcf84199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.649+0000 7fcf890a9640 1 -- 172.21.15.67:0/1392041867 >> 172.21.15.67:0/1392041867 conn(0x7fcf84079160 msgr2=0x7fcf84101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.649+0000 7fcf890a9640 1 -- 172.21.15.67:0/1392041867 shutdown_connections 2024-07-31T14:54:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:42.650+0000 7fcf890a9640 1 -- 172.21.15.67:0/1392041867 wait complete. 2024-07-31T14:54:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:43 smithi067 ceph-mon[32750]: pgmap v509: 129 pgs: 129 active+clean; 344 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 254 KiB/s wr, 54 op/s 2024-07-31T14:54:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:43 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T14:54:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:43.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:43 smithi073 ceph-mon[36714]: pgmap v509: 129 pgs: 129 active+clean; 344 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 254 KiB/s wr, 54 op/s 2024-07-31T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:43 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:43.487 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:54:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:43 smithi077 ceph-mon[37327]: pgmap v509: 129 pgs: 129 active+clean; 344 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 254 KiB/s wr, 54 op/s 2024-07-31T14:54:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:43 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T14:54:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:43.796 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:54:44.206 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.206 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.206 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:44.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:44.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:44.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:44.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: pgmap v510: 129 pgs: 129 active+clean; 355 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-07-31T14:54:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: pgmap v510: 129 pgs: 129 active+clean; 355 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-07-31T14:54:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:45.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: pgmap v510: 129 pgs: 129 active+clean; 355 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-07-31T14:54:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:45.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:45.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:45.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:45.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:45.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:45.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:45.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.576+0000 7f00f2cac640 1 -- 172.21.15.67:0/1302328734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00ec102490 msgr2=0x7f00ec1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:45.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.576+0000 7f00f2cac640 1 --2- 172.21.15.67:0/1302328734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00ec102490 0x7f00ec1028a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009900 tx=0x7f00d402a410 comp rx=0 tx=0).stop 2024-07-31T14:54:45.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.576+0000 7f00f2cac640 1 -- 172.21.15.67:0/1302328734 shutdown_connections 2024-07-31T14:54:45.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.576+0000 7f00f2cac640 1 --2- 172.21.15.67:0/1302328734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00ec108720 0x7f00ec10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.576+0000 7f00f2cac640 1 --2- 172.21.15.67:0/1302328734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00ec107dd0 0x7f00ec1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.576+0000 7f00f2cac640 1 --2- 172.21.15.67:0/1302328734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00ec102490 0x7f00ec1028a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.576+0000 7f00f2cac640 1 -- 172.21.15.67:0/1302328734 >> 172.21.15.67:0/1302328734 conn(0x7f00ec079160 msgr2=0x7f00ec079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:45.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.576+0000 7f00f2cac640 1 -- 172.21.15.67:0/1302328734 shutdown_connections 2024-07-31T14:54:45.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.576+0000 7f00f2cac640 1 -- 172.21.15.67:0/1302328734 wait complete. 2024-07-31T14:54:45.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.577+0000 7f00f2cac640 1 Processor -- start 2024-07-31T14:54:45.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.577+0000 7f00f2cac640 1 -- start start 2024-07-31T14:54:45.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.577+0000 7f00f2cac640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00ec102490 0x7f00ec19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.577+0000 7f00f2cac640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00ec107dd0 0x7f00ec19b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.577+0000 7f00f2cac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00ec108720 0x7f00ec19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.577+0000 7f00f2cac640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec112380 con 0x7f00ec107dd0 2024-07-31T14:54:45.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.577+0000 7f00f2cac640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec112200 con 0x7f00ec102490 2024-07-31T14:54:45.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.577+0000 7f00f2cac640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec112500 con 0x7f00ec108720 2024-07-31T14:54:45.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.577+0000 7f00ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00ec107dd0 0x7f00ec19b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.578+0000 7f00ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00ec107dd0 0x7f00ec19b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36736/0 (socket says 172.21.15.67:36736) 2024-07-31T14:54:45.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.578+0000 7f00ebfff640 1 -- 172.21.15.67:0/2913977372 learned_addr learned my addr 172.21.15.67:0/2913977372 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:45.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.578+0000 7f00f0a48640 1 --2- 172.21.15.67:0/2913977372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00ec102490 0x7f00ec19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.578+0000 7f00f1249640 1 --2- 172.21.15.67:0/2913977372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00ec108720 0x7f00ec19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.578+0000 7f00ebfff640 1 -- 172.21.15.67:0/2913977372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00ec108720 msgr2=0x7f00ec19b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:45.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.578+0000 7f00ebfff640 1 --2- 172.21.15.67:0/2913977372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00ec108720 0x7f00ec19b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.578+0000 7f00ebfff640 1 -- 172.21.15.67:0/2913977372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00ec102490 msgr2=0x7f00ec19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:45.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.578+0000 7f00ebfff640 1 --2- 172.21.15.67:0/2913977372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00ec102490 0x7f00ec19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.578+0000 7f00ebfff640 1 -- 172.21.15.67:0/2913977372 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec1a5f80 con 0x7f00ec107dd0 2024-07-31T14:54:45.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.578+0000 7f00ebfff640 1 --2- 172.21.15.67:0/2913977372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00ec107dd0 0x7f00ec19b230 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00ea80 tx=0x7f00dc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:45.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.578+0000 7f00e9ffb640 1 -- 172.21.15.67:0/2913977372 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00ce40 con 0x7f00ec107dd0 2024-07-31T14:54:45.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.578+0000 7f00e9ffb640 1 -- 172.21.15.67:0/2913977372 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00dc011920 con 0x7f00ec107dd0 2024-07-31T14:54:45.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.578+0000 7f00f2cac640 1 -- 172.21.15.67:0/2913977372 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ec1a6270 con 0x7f00ec107dd0 2024-07-31T14:54:45.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.578+0000 7f00e9ffb640 1 -- 172.21.15.67:0/2913977372 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc01aa20 con 0x7f00ec107dd0 2024-07-31T14:54:45.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.579+0000 7f00f2cac640 1 -- 172.21.15.67:0/2913977372 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ec1a67b0 con 0x7f00ec107dd0 2024-07-31T14:54:45.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.581+0000 7f00e9ffb640 1 -- 172.21.15.67:0/2913977372 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f00dc01abc0 con 0x7f00ec107dd0 2024-07-31T14:54:45.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.582+0000 7f00e9ffb640 1 --2- 172.21.15.67:0/2913977372 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f00d80b3d70 0x7f00d80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.582+0000 7f00f91fd740 1 Processor -- start 2024-07-31T14:54:45.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.582+0000 7f00f91fd740 1 -- start start 2024-07-31T14:54:45.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.582+0000 7f00f0a48640 1 --2- 172.21.15.67:0/2913977372 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f00d80b3d70 0x7f00d80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.582+0000 7f00e9ffb640 1 -- 172.21.15.67:0/2913977372 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f00dc0e4870 con 0x7f00ec107dd0 2024-07-31T14:54:45.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.582+0000 7f00f91fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952665a070 0x5595269ff550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.582+0000 7f00f91fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x5595269ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.582+0000 7f00f91fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559526639390 0x5595269fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.582+0000 7f00f91fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559526975b90 con 0x5595266311b0 2024-07-31T14:54:45.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.582+0000 7f00f91fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559526965090 con 0x55952665a070 2024-07-31T14:54:45.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.582+0000 7f00f91fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5595269690d0 con 0x559526639390 2024-07-31T14:54:45.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.582+0000 7f00ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x5595269ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.582+0000 7f00ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x5595269ffa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36748/0 (socket says 172.21.15.67:36748) 2024-07-31T14:54:45.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.582+0000 7f00ebfff640 1 -- 172.21.15.67:0/4128311824 learned_addr learned my addr 172.21.15.67:0/4128311824 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:45.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.583+0000 7f00f0a48640 1 --2- 172.21.15.67:0/4128311824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952665a070 0x5595269ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.583+0000 7f00f1249640 1 --2- 172.21.15.67:0/4128311824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559526639390 0x5595269fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.583+0000 7f00f0a48640 1 --2- 172.21.15.67:0/2913977372 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f00d80b3d70 0x7f00d80b6280 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f00d4000c00 tx=0x7f00d4031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:45.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.583+0000 7f00ebfff640 1 -- 172.21.15.67:0/4128311824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559526639390 msgr2=0x5595269fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:45.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.583+0000 7f00ebfff640 1 --2- 172.21.15.67:0/4128311824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559526639390 0x5595269fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.583+0000 7f00ebfff640 1 -- 172.21.15.67:0/4128311824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952665a070 msgr2=0x5595269ff550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:45.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.583+0000 7f00ebfff640 1 --2- 172.21.15.67:0/4128311824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952665a070 0x5595269ff550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.583+0000 7f00ebfff640 1 -- 172.21.15.67:0/4128311824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55952690bb70 con 0x5595266311b0 2024-07-31T14:54:45.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.583+0000 7f00f1249640 1 --2- 172.21.15.67:0/4128311824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559526639390 0x5595269fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:45.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.583+0000 7f00ebfff640 1 --2- 172.21.15.67:0/4128311824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x5595269ffa90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0aae70 tx=0x7f00dc0aadc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d32c4594b28a91a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:45.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.583+0000 7f00e2ffd640 1 -- 172.21.15.67:0/4128311824 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0c4020 con 0x5595266311b0 2024-07-31T14:54:45.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.583+0000 7f00e2ffd640 1 -- 172.21.15.67:0/4128311824 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00dc0af770 con 0x5595266311b0 2024-07-31T14:54:45.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.583+0000 7f00e2ffd640 1 -- 172.21.15.67:0/4128311824 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0aa3d0 con 0x5595266311b0 2024-07-31T14:54:45.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.584+0000 7f00f91fd740 1 -- 172.21.15.67:0/4128311824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 msgr2=0x5595269ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:45.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.584+0000 7f00f91fd740 1 --2- 172.21.15.67:0/4128311824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x5595269ffa90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0aae70 tx=0x7f00dc0aadc0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.584+0000 7f00f91fd740 1 -- 172.21.15.67:0/4128311824 shutdown_connections 2024-07-31T14:54:45.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.584+0000 7f00f91fd740 1 --2- 172.21.15.67:0/4128311824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559526639390 0x5595269fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.584+0000 7f00f91fd740 1 --2- 172.21.15.67:0/4128311824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x5595269ffa90 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.584+0000 7f00f91fd740 1 --2- 172.21.15.67:0/4128311824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952665a070 0x5595269ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.584+0000 7f00f91fd740 1 -- 172.21.15.67:0/4128311824 >> 172.21.15.67:0/4128311824 conn(0x55952664e460 msgr2=0x5595269fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:45.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.584+0000 7f00f91fd740 1 -- 172.21.15.67:0/4128311824 shutdown_connections 2024-07-31T14:54:45.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.584+0000 7f00f91fd740 1 -- 172.21.15.67:0/4128311824 wait complete. 2024-07-31T14:54:45.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.585+0000 7f00f91fd740 1 -- start start 2024-07-31T14:54:45.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.585+0000 7f00f91fd740 1 Processor -- start 2024-07-31T14:54:45.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f91fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a0b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f91fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559526639390 0x559526a0b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f91fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952665a070 0x559526a10ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f91fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5595267c8700 con 0x5595266311b0 2024-07-31T14:54:45.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f91fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5595269079c0 con 0x55952665a070 2024-07-31T14:54:45.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f91fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559526a0dd30 con 0x559526639390 2024-07-31T14:54:45.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f91fd740 10 client.0 _ll_register_callbacks cb 0x5595269224d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:45.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f0a48640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a0b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f0a48640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a0b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36764/0 (socket says 172.21.15.67:36764) 2024-07-31T14:54:45.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f0a48640 1 -- 172.21.15.67:0/3496681786 learned_addr learned my addr 172.21.15.67:0/3496681786 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:45.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00ebfff640 1 --2- 172.21.15.67:0/3496681786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559526639390 0x559526a0b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f1249640 1 --2- 172.21.15.67:0/3496681786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952665a070 0x559526a10ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f0a48640 1 -- 172.21.15.67:0/3496681786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559526639390 msgr2=0x559526a0b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:45.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f0a48640 1 --2- 172.21.15.67:0/3496681786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559526639390 0x559526a0b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f0a48640 1 -- 172.21.15.67:0/3496681786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952665a070 msgr2=0x559526a10ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:45.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f0a48640 1 --2- 172.21.15.67:0/3496681786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952665a070 0x559526a10ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f0a48640 1 -- 172.21.15.67:0/3496681786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595266c5650 con 0x5595266311b0 2024-07-31T14:54:45.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.586+0000 7f00f1249640 1 --2- 172.21.15.67:0/3496681786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952665a070 0x559526a10ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:45.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.587+0000 7f00f0a48640 1 --2- 172.21.15.67:0/3496681786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a0b210 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f00d4033000 tx=0x7f00d4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377832340a6fdb3a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:45.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.587+0000 7f00cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:45.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.587+0000 7f00cbfff640 1 -- 172.21.15.67:0/3496681786 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4035040 con 0x5595266311b0 2024-07-31T14:54:45.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.587+0000 7f00cbfff640 1 -- 172.21.15.67:0/3496681786 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f00d4004230 con 0x5595266311b0 2024-07-31T14:54:45.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.587+0000 7f00f91fd740 1 -- 172.21.15.67:0/3496681786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00dc0af770 con 0x5595266311b0 2024-07-31T14:54:45.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.587+0000 7f00cbfff640 1 -- 172.21.15.67:0/3496681786 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f00d4004430 con 0x5595266311b0 2024-07-31T14:54:45.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.587+0000 7f00f91fd740 10 client.16662 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:45.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.587+0000 7f00f91fd740 1 -- 172.21.15.67:0/3496681786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559526a19e50 con 0x5595266311b0 2024-07-31T14:54:45.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.587+0000 7f00cbfff640 1 -- 172.21.15.67:0/3496681786 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d40054f0 con 0x5595266311b0 2024-07-31T14:54:45.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.588+0000 7f00cbfff640 1 -- 172.21.15.67:0/3496681786 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f00d40035f0 con 0x5595266311b0 2024-07-31T14:54:45.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.588+0000 7f00f91fd740 10 client.16662 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:45.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.588+0000 7f00f91fd740 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:45.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.588+0000 7f00f91fd740 20 client.16662 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:45.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.588+0000 7f00f91fd740 1 --2- 172.21.15.67:0/3496681786 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5595266189f0 0x559526a1d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.588+0000 7f00f91fd740 4 client.16662 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:54:45.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.588+0000 7f00f91fd740 1 -- 172.21.15.67:0/3496681786 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f00dc0af770 con 0x5595266189f0 2024-07-31T14:54:45.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.588+0000 7f00ebfff640 1 --2- 172.21.15.67:0/3496681786 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5595266189f0 0x559526a1d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.589+0000 7f00ebfff640 1 --2- 172.21.15.67:0/3496681786 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5595266189f0 0x559526a1d370 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1f5d7e99e909fe9 server_cookie=d2d80efa8221ecac in_seq=0 out_seq=0 2024-07-31T14:54:45.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.589+0000 7f00cbfff640 10 client.16662 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:45.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.589+0000 7f00cbfff640 1 -- 172.21.15.67:0/3496681786 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f00dc0af770 con 0x5595266189f0 2024-07-31T14:54:45.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.589+0000 7f00cbfff640 10 client.16662 handle_command_reply: tid=0 2024-07-31T14:54:45.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.589+0000 7f00f91fd740 1 client.16662 shutdown 2024-07-31T14:54:45.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.590+0000 7f00f91fd740 1 -- 172.21.15.67:0/3496681786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 msgr2=0x559526a0b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:45.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.590+0000 7f00f91fd740 1 --2- 172.21.15.67:0/3496681786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a0b210 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f00d4033000 tx=0x7f00d4009260 comp rx=0 tx=0).stop 2024-07-31T14:54:45.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.590+0000 7f00f91fd740 1 -- 172.21.15.67:0/3496681786 shutdown_connections 2024-07-31T14:54:45.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.590+0000 7f00f91fd740 1 --2- 172.21.15.67:0/3496681786 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5595266189f0 0x559526a1d370 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.590+0000 7f00f91fd740 1 --2- 172.21.15.67:0/3496681786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952665a070 0x559526a10ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.590+0000 7f00f91fd740 1 --2- 172.21.15.67:0/3496681786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559526639390 0x559526a0b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.590+0000 7f00f91fd740 1 --2- 172.21.15.67:0/3496681786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a0b210 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.590+0000 7f00cbfff640 0 client.16662 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:45.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.590+0000 7f00f91fd740 1 -- 172.21.15.67:0/3496681786 >> 172.21.15.67:0/3496681786 conn(0x55952664e460 msgr2=0x559526a02730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:45.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.590+0000 7f00f91fd740 1 -- 172.21.15.67:0/3496681786 shutdown_connections 2024-07-31T14:54:45.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.590+0000 7f00f91fd740 1 -- 172.21.15.67:0/3496681786 wait complete. 2024-07-31T14:54:45.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.590+0000 7f00f91fd740 20 client.16662 trim_cache size 0 max 16384 2024-07-31T14:54:45.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.608+0000 7f00f91fd740 1 Processor -- start 2024-07-31T14:54:45.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.608+0000 7f00f91fd740 1 -- start start 2024-07-31T14:54:45.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f91fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952664e460 0x559526a00c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f91fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595266189f0 0x559526a04630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f91fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a04b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f91fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec1b06d0 con 0x5595266311b0 2024-07-31T14:54:45.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f91fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec1b2390 con 0x55952664e460 2024-07-31T14:54:45.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f91fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec1b0da0 con 0x5595266189f0 2024-07-31T14:54:45.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f1249640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a04b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f1249640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a04b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36778/0 (socket says 172.21.15.67:36778) 2024-07-31T14:54:45.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f1249640 1 -- 172.21.15.67:0/1089416807 learned_addr learned my addr 172.21.15.67:0/1089416807 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:45.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f0a48640 1 --2- 172.21.15.67:0/1089416807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952664e460 0x559526a00c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00ebfff640 1 --2- 172.21.15.67:0/1089416807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595266189f0 0x559526a04630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f1249640 1 -- 172.21.15.67:0/1089416807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595266189f0 msgr2=0x559526a04630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:45.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f1249640 1 --2- 172.21.15.67:0/1089416807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595266189f0 0x559526a04630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f1249640 1 -- 172.21.15.67:0/1089416807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952664e460 msgr2=0x559526a00c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:45.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f1249640 1 --2- 172.21.15.67:0/1089416807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952664e460 0x559526a00c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f1249640 1 -- 172.21.15.67:0/1089416807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559526a01870 con 0x5595266311b0 2024-07-31T14:54:45.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.609+0000 7f00f0a48640 1 --2- 172.21.15.67:0/1089416807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952664e460 0x559526a00c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:54:45.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.610+0000 7f00f1249640 1 --2- 172.21.15.67:0/1089416807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a04b70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f00e4004620 tx=0x7f00e400dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5e27bdb7afbe server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:45.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.610+0000 7f00e27fc640 1 -- 172.21.15.67:0/1089416807 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4003ad0 con 0x5595266311b0 2024-07-31T14:54:45.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.610+0000 7f00e27fc640 1 -- 172.21.15.67:0/1089416807 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00e40054d0 con 0x5595266311b0 2024-07-31T14:54:45.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.610+0000 7f00e27fc640 1 -- 172.21.15.67:0/1089416807 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4004e10 con 0x5595266311b0 2024-07-31T14:54:45.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.611+0000 7f00f91fd740 1 -- 172.21.15.67:0/1089416807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 msgr2=0x559526a04b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:45.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.611+0000 7f00f91fd740 1 --2- 172.21.15.67:0/1089416807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a04b70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f00e4004620 tx=0x7f00e400dce0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.611+0000 7f00f91fd740 1 -- 172.21.15.67:0/1089416807 shutdown_connections 2024-07-31T14:54:45.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.611+0000 7f00f91fd740 1 --2- 172.21.15.67:0/1089416807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a04b70 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.611+0000 7f00f91fd740 1 --2- 172.21.15.67:0/1089416807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595266189f0 0x559526a04630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.611+0000 7f00f91fd740 1 --2- 172.21.15.67:0/1089416807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55952664e460 0x559526a00c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.611+0000 7f00f91fd740 1 -- 172.21.15.67:0/1089416807 >> 172.21.15.67:0/1089416807 conn(0x559526a1d6e0 msgr2=0x559526a00580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:45.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.611+0000 7f00f91fd740 1 -- 172.21.15.67:0/1089416807 shutdown_connections 2024-07-31T14:54:45.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.611+0000 7f00f91fd740 1 -- 172.21.15.67:0/1089416807 wait complete. 2024-07-31T14:54:45.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.611+0000 7f00f91fd740 1 -- start start 2024-07-31T14:54:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.611+0000 7f00f91fd740 1 Processor -- start 2024-07-31T14:54:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00f91fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595266189f0 0x559526a16c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00f91fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a171b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00f91fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55952664e460 0x559526a176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00f91fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec1b1cc0 con 0x5595266311b0 2024-07-31T14:54:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00f91fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec1b1530 con 0x5595266189f0 2024-07-31T14:54:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00f91fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec1ae6b0 con 0x55952664e460 2024-07-31T14:54:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00f91fd740 10 client.0 _ll_register_callbacks cb 0x7f00d4009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a171b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36792/0 (socket says 172.21.15.67:36792) 2024-07-31T14:54:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00f1249640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55952664e460 0x559526a176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00ebfff640 1 -- 172.21.15.67:0/4036077025 learned_addr learned my addr 172.21.15.67:0/4036077025 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00f0a48640 1 --2- 172.21.15.67:0/4036077025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595266189f0 0x559526a16c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00ebfff640 1 -- 172.21.15.67:0/4036077025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55952664e460 msgr2=0x559526a176f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00ebfff640 1 --2- 172.21.15.67:0/4036077025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55952664e460 0x559526a176f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00ebfff640 1 -- 172.21.15.67:0/4036077025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595266189f0 msgr2=0x559526a16c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00ebfff640 1 --2- 172.21.15.67:0/4036077025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595266189f0 0x559526a16c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00ebfff640 1 -- 172.21.15.67:0/4036077025 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559526a15dd0 con 0x5595266311b0 2024-07-31T14:54:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00f0a48640 1 --2- 172.21.15.67:0/4036077025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595266189f0 0x559526a16c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:54:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.612+0000 7f00f1249640 1 --2- 172.21.15.67:0/4036077025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55952664e460 0x559526a176f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00ebfff640 1 --2- 172.21.15.67:0/4036077025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a171b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f00dc01d000 tx=0x7f00dc0b4390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b426aab7df20de server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:54:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00cbfff640 1 -- 172.21.15.67:0/4036077025 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc06e070 con 0x5595266311b0 2024-07-31T14:54:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00cbfff640 1 -- 172.21.15.67:0/4036077025 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f00dc0a7400 con 0x5595266311b0 2024-07-31T14:54:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00f91fd740 1 -- 172.21.15.67:0/4036077025 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00e40054d0 con 0x5595266311b0 2024-07-31T14:54:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00cbfff640 1 -- 172.21.15.67:0/4036077025 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f00dc0ca030 con 0x5595266311b0 2024-07-31T14:54:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00f91fd740 10 client.16668 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00f91fd740 1 -- 172.21.15.67:0/4036077025 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559526a85fa0 con 0x5595266311b0 2024-07-31T14:54:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00cbfff640 1 -- 172.21.15.67:0/4036077025 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00e40054d0 con 0x5595266311b0 2024-07-31T14:54:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00cbfff640 1 -- 172.21.15.67:0/4036077025 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f00dc022030 con 0x5595266311b0 2024-07-31T14:54:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00f91fd740 10 client.16668 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00f91fd740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00f91fd740 20 client.16668 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00f91fd740 1 --2- 172.21.15.67:0/4036077025 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559526639390 0x559526a88320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00f91fd740 4 client.16668 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:54:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.613+0000 7f00f91fd740 1 -- 172.21.15.67:0/4036077025 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559526977ae0 con 0x559526639390 2024-07-31T14:54:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00f0a48640 1 --2- 172.21.15.67:0/4036077025 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559526639390 0x559526a88320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00f0a48640 1 --2- 172.21.15.67:0/4036077025 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559526639390 0x559526a88320 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=346ae952dc4da40f server_cookie=4b2d796f004a3acc in_seq=0 out_seq=0 2024-07-31T14:54:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00cbfff640 10 client.16668 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00cbfff640 1 -- 172.21.15.67:0/4036077025 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559526977ae0 con 0x559526639390 2024-07-31T14:54:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00cbfff640 10 client.16668 handle_command_reply: tid=0 2024-07-31T14:54:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00f91fd740 1 client.16668 shutdown 2024-07-31T14:54:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00f91fd740 1 -- 172.21.15.67:0/4036077025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 msgr2=0x559526a171b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00f91fd740 1 --2- 172.21.15.67:0/4036077025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a171b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f00dc01d000 tx=0x7f00dc0b4390 comp rx=0 tx=0).stop 2024-07-31T14:54:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00f91fd740 1 -- 172.21.15.67:0/4036077025 shutdown_connections 2024-07-31T14:54:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00f91fd740 1 --2- 172.21.15.67:0/4036077025 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559526639390 0x559526a88320 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00f91fd740 1 --2- 172.21.15.67:0/4036077025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55952664e460 0x559526a176f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00f91fd740 1 --2- 172.21.15.67:0/4036077025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595266311b0 0x559526a171b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00cbfff640 0 client.16668 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00f91fd740 1 --2- 172.21.15.67:0/4036077025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595266189f0 0x559526a16c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.614+0000 7f00f91fd740 1 -- 172.21.15.67:0/4036077025 >> 172.21.15.67:0/4036077025 conn(0x559526a1d6e0 msgr2=0x559526a00eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.615+0000 7f00f91fd740 1 -- 172.21.15.67:0/4036077025 shutdown_connections 2024-07-31T14:54:45.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.615+0000 7f00f91fd740 1 -- 172.21.15.67:0/4036077025 wait complete. 2024-07-31T14:54:45.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.615+0000 7f00f91fd740 20 client.16668 trim_cache size 0 max 16384 2024-07-31T14:54:45.622 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:54:45.622 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-31T14:54:45.622 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T14:54:45.622 INFO:teuthology.orchestra.run.smithi067.stdout: "7c6fa637-63b0-4564-ba95-bca6c2bef18d": { 2024-07-31T14:54:45.622 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T14:54:45.622 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "7c6fa637-63b0-4564-ba95-bca6c2bef18d", 2024-07-31T14:54:45.623 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T14:54:45.623 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:54:45.623 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:54:45.623 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:54:45.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.615+0000 7f00f2cac640 1 -- 172.21.15.67:0/2913977372 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f00d80b3d70 msgr2=0x7f00d80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:45.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.615+0000 7f00f2cac640 1 --2- 172.21.15.67:0/2913977372 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f00d80b3d70 0x7f00d80b6280 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f00d4000c00 tx=0x7f00d4031400 comp rx=0 tx=0).stop 2024-07-31T14:54:45.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.615+0000 7f00f2cac640 1 -- 172.21.15.67:0/2913977372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00ec107dd0 msgr2=0x7f00ec19b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:45.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.615+0000 7f00f2cac640 1 --2- 172.21.15.67:0/2913977372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00ec107dd0 0x7f00ec19b230 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00ea80 tx=0x7f00dc00ef40 comp rx=0 tx=0).stop 2024-07-31T14:54:45.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.616+0000 7f00f2cac640 1 -- 172.21.15.67:0/2913977372 shutdown_connections 2024-07-31T14:54:45.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.616+0000 7f00f2cac640 1 --2- 172.21.15.67:0/2913977372 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f00d80b3d70 0x7f00d80b6280 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.616+0000 7f00f2cac640 1 --2- 172.21.15.67:0/2913977372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00ec108720 0x7f00ec19b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.616+0000 7f00f2cac640 1 --2- 172.21.15.67:0/2913977372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00ec107dd0 0x7f00ec19b230 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.616+0000 7f00f2cac640 1 --2- 172.21.15.67:0/2913977372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00ec102490 0x7f00ec19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:45.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.616+0000 7f00f2cac640 1 -- 172.21.15.67:0/2913977372 >> 172.21.15.67:0/2913977372 conn(0x7f00ec079160 msgr2=0x7f00ec100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:45.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.616+0000 7f00f2cac640 1 -- 172.21.15.67:0/2913977372 shutdown_connections 2024-07-31T14:54:45.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:54:45.616+0000 7f00f2cac640 1 -- 172.21.15.67:0/2913977372 wait complete. 2024-07-31T14:54:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:54:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:46 smithi067 ceph-mon[32750]: pgmap v511: 129 pgs: 129 active+clean; 360 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.6 MiB/s wr, 448 op/s 2024-07-31T14:54:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:54:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:46 smithi073 ceph-mon[36714]: pgmap v511: 129 pgs: 129 active+clean; 360 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.6 MiB/s wr, 448 op/s 2024-07-31T14:54:46.426 INFO:tasks.cephfs.filesystem:scrub status for tag:7c6fa637-63b0-4564-ba95-bca6c2bef18d - {'path': '/', 'tag': '7c6fa637-63b0-4564-ba95-bca6c2bef18d', 'options': 'recursive,force'} 2024-07-31T14:54:46.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:46.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:54:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:54:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:46 smithi077 ceph-mon[37327]: pgmap v511: 129 pgs: 129 active+clean; 360 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.6 MiB/s wr, 448 op/s 2024-07-31T14:54:48.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:48 smithi073 ceph-mon[36714]: pgmap v512: 129 pgs: 129 active+clean; 364 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 606 op/s 2024-07-31T14:54:48.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:54:48.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:48 smithi077 ceph-mon[37327]: pgmap v512: 129 pgs: 129 active+clean; 364 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 606 op/s 2024-07-31T14:54:48.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:54:49.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:48 smithi067 ceph-mon[32750]: pgmap v512: 129 pgs: 129 active+clean; 364 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 606 op/s 2024-07-31T14:54:49.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:54:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.466+0000 7fd6a759d640 1 -- 172.21.15.67:0/483062059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6a0107b50 msgr2=0x7fd6a0107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.466+0000 7fd6a759d640 1 --2- 172.21.15.67:0/483062059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6a0107b50 0x7fd6a0107f60 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd690009900 tx=0x7fd69002a410 comp rx=0 tx=0).stop 2024-07-31T14:54:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.466+0000 7fd6a759d640 1 -- 172.21.15.67:0/483062059 shutdown_connections 2024-07-31T14:54:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.466+0000 7fd6a759d640 1 --2- 172.21.15.67:0/483062059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6a01084a0 0x7fd6a010ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.466+0000 7fd6a759d640 1 --2- 172.21.15.67:0/483062059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6a0107b50 0x7fd6a0107f60 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.466+0000 7fd6a759d640 1 --2- 172.21.15.67:0/483062059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a0102220 0x7fd6a0102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.466+0000 7fd6a759d640 1 -- 172.21.15.67:0/483062059 >> 172.21.15.67:0/483062059 conn(0x7fd6a00fdc40 msgr2=0x7fd6a0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.466+0000 7fd6a759d640 1 -- 172.21.15.67:0/483062059 shutdown_connections 2024-07-31T14:54:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.466+0000 7fd6a759d640 1 -- 172.21.15.67:0/483062059 wait complete. 2024-07-31T14:54:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a759d640 1 Processor -- start 2024-07-31T14:54:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a759d640 1 -- start start 2024-07-31T14:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a759d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6a0102220 0x7fd6a0199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a759d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6a0107b50 0x7fd6a019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a759d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a01084a0 0x7fd6a019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a759d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0113280 con 0x7fd6a0102220 2024-07-31T14:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a759d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0113100 con 0x7fd6a0107b50 2024-07-31T14:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a759d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0113400 con 0x7fd6a01084a0 2024-07-31T14:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a6d9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a01084a0 0x7fd6a019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a659b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6a0102220 0x7fd6a0199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a6d9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a01084a0 0x7fd6a019f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35890/0 (socket says 172.21.15.67:35890) 2024-07-31T14:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a6d9c640 1 -- 172.21.15.67:0/2522945253 learned_addr learned my addr 172.21.15.67:0/2522945253 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a5d9a640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6a0107b50 0x7fd6a019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a6d9c640 1 -- 172.21.15.67:0/2522945253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6a0107b50 msgr2=0x7fd6a019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a6d9c640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6a0107b50 0x7fd6a019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a6d9c640 1 -- 172.21.15.67:0/2522945253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6a0102220 msgr2=0x7fd6a0199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a6d9c640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6a0102220 0x7fd6a0199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a6d9c640 1 -- 172.21.15.67:0/2522945253 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a019fbf0 con 0x7fd6a01084a0 2024-07-31T14:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a5d9a640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6a0107b50 0x7fd6a019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.467+0000 7fd6a659b640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6a0102220 0x7fd6a0199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.468+0000 7fd6a6d9c640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a01084a0 0x7fd6a019f4f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd698009fd0 tx=0x7fd69800b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.468+0000 7fd6977fe640 1 -- 172.21.15.67:0/2522945253 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698017070 con 0x7fd6a01084a0 2024-07-31T14:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.468+0000 7fd6a759d640 1 -- 172.21.15.67:0/2522945253 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a019fee0 con 0x7fd6a01084a0 2024-07-31T14:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.468+0000 7fd6977fe640 1 -- 172.21.15.67:0/2522945253 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd698003e20 con 0x7fd6a01084a0 2024-07-31T14:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.468+0000 7fd6977fe640 1 -- 172.21.15.67:0/2522945253 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698010460 con 0x7fd6a01084a0 2024-07-31T14:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.468+0000 7fd6a759d640 1 -- 172.21.15.67:0/2522945253 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a01a6670 con 0x7fd6a01084a0 2024-07-31T14:54:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.470+0000 7fd6af7cb740 1 Processor -- start 2024-07-31T14:54:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.470+0000 7fd6977fe640 1 -- 172.21.15.67:0/2522945253 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd698003970 con 0x7fd6a01084a0 2024-07-31T14:54:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6af7cb740 1 -- start start 2024-07-31T14:54:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6af7cb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866f141b0 0x5558672b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6af7cb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6af7cb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866ef34d0 0x5558672b7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6af7cb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55586722c3b0 con 0x555866ef34d0 2024-07-31T14:54:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6af7cb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55586721b8b0 con 0x555866f141b0 2024-07-31T14:54:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6af7cb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55586721f8f0 con 0x555866eeb2f0 2024-07-31T14:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6a659b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6a5d9a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866ef34d0 0x5558672b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6a659b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672b6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35892/0 (socket says 172.21.15.67:35892) 2024-07-31T14:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6a659b640 1 -- 172.21.15.67:0/3460709336 learned_addr learned my addr 172.21.15.67:0/3460709336 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6a5d9a640 1 --2- 172.21.15.67:0/3460709336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866f141b0 0x5558672b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6a659b640 1 -- 172.21.15.67:0/3460709336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866f141b0 msgr2=0x5558672b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6a659b640 1 --2- 172.21.15.67:0/3460709336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866f141b0 0x5558672b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6a659b640 1 -- 172.21.15.67:0/3460709336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866ef34d0 msgr2=0x5558672b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6a659b640 1 --2- 172.21.15.67:0/3460709336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866ef34d0 0x5558672b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.471+0000 7fd6a659b640 1 -- 172.21.15.67:0/3460709336 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558671c29a0 con 0x555866eeb2f0 2024-07-31T14:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.472+0000 7fd6a5d9a640 1 --2- 172.21.15.67:0/3460709336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866f141b0 0x5558672b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.472+0000 7fd6977fe640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd68c0b3d30 0x7fd68c0b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.472+0000 7fd6a659b640 1 --2- 172.21.15.67:0/3460709336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672b6df0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0115f0 tx=0x7fd69c011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f90cc2cf857d71a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.472+0000 7fd6a5d9a640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd68c0b3d30 0x7fd68c0b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.472+0000 7fd6977fe640 1 -- 172.21.15.67:0/2522945253 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fd6980e4140 con 0x7fd6a01084a0 2024-07-31T14:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.472+0000 7fd694ff9640 1 -- 172.21.15.67:0/3460709336 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c01e070 con 0x555866eeb2f0 2024-07-31T14:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.472+0000 7fd694ff9640 1 -- 172.21.15.67:0/3460709336 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd69c00faf0 con 0x555866eeb2f0 2024-07-31T14:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.472+0000 7fd694ff9640 1 -- 172.21.15.67:0/3460709336 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c019ae0 con 0x555866eeb2f0 2024-07-31T14:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.473+0000 7fd6af7cb740 1 -- 172.21.15.67:0/3460709336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 msgr2=0x5558672b6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.473+0000 7fd6a5d9a640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd68c0b3d30 0x7fd68c0b6240 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd690009230 tx=0x7fd690040040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.473+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/3460709336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672b6df0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0115f0 tx=0x7fd69c011ab0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.473+0000 7fd6af7cb740 1 -- 172.21.15.67:0/3460709336 shutdown_connections 2024-07-31T14:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.473+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/3460709336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866ef34d0 0x5558672b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.473+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/3460709336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672b6df0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.473+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/3460709336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866f141b0 0x5558672b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.473+0000 7fd6af7cb740 1 -- 172.21.15.67:0/3460709336 >> 172.21.15.67:0/3460709336 conn(0x555866f08300 msgr2=0x5558672b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.473+0000 7fd6af7cb740 1 -- 172.21.15.67:0/3460709336 shutdown_connections 2024-07-31T14:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.473+0000 7fd6af7cb740 1 -- 172.21.15.67:0/3460709336 wait complete. 2024-07-31T14:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.473+0000 7fd6af7cb740 1 -- start start 2024-07-31T14:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.473+0000 7fd6af7cb740 1 Processor -- start 2024-07-31T14:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6af7cb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672c2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6af7cb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 0x5558672c2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6af7cb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866f141b0 0x5558672ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6af7cb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555867155ab0 con 0x555866f141b0 2024-07-31T14:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6af7cb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5558671a3cf0 con 0x555866ef34d0 2024-07-31T14:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6af7cb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5558672c5030 con 0x555866eeb2f0 2024-07-31T14:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6af7cb740 10 client.0 _ll_register_callbacks cb 0x5558672153e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6a6d9c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866f141b0 0x5558672ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6a659b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6a659b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672c2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35898/0 (socket says 172.21.15.67:35898) 2024-07-31T14:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6a659b640 1 -- 172.21.15.67:0/1159600291 learned_addr learned my addr 172.21.15.67:0/1159600291 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6a6d9c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866f141b0 0x5558672ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36818/0 (socket says 172.21.15.67:36818) 2024-07-31T14:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6a5d9a640 1 --2- 172.21.15.67:0/1159600291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 0x5558672c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6a659b640 1 -- 172.21.15.67:0/1159600291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 msgr2=0x5558672c2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6a659b640 1 --2- 172.21.15.67:0/1159600291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 0x5558672c2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6a659b640 1 -- 172.21.15.67:0/1159600291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866f141b0 msgr2=0x5558672ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6a659b640 1 --2- 172.21.15.67:0/1159600291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866f141b0 0x5558672ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6a659b640 1 -- 172.21.15.67:0/1159600291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555866f7f790 con 0x555866eeb2f0 2024-07-31T14:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.474+0000 7fd6a6d9c640 1 --2- 172.21.15.67:0/1159600291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866f141b0 0x5558672ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd6a659b640 1 --2- 172.21.15.67:0/1159600291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672c2400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009ce0 tx=0x7fd69c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0b79996e8b02ee server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd689ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd689ffb640 1 -- 172.21.15.67:0/1159600291 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c01e040 con 0x555866eeb2f0 2024-07-31T14:54:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd689ffb640 1 -- 172.21.15.67:0/1159600291 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fd69c00f780 con 0x555866eeb2f0 2024-07-31T14:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd6af7cb740 1 -- 172.21.15.67:0/1159600291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd69c00faf0 con 0x555866eeb2f0 2024-07-31T14:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd689ffb640 1 -- 172.21.15.67:0/1159600291 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd69c030050 con 0x555866eeb2f0 2024-07-31T14:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd6af7cb740 10 client.26314 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd6af7cb740 1 -- 172.21.15.67:0/1159600291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555866ff8620 con 0x555866eeb2f0 2024-07-31T14:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd689ffb640 1 -- 172.21.15.67:0/1159600291 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd69c00faf0 con 0x555866eeb2f0 2024-07-31T14:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd689ffb640 1 -- 172.21.15.67:0/1159600291 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7fd69c021070 con 0x555866eeb2f0 2024-07-31T14:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd6af7cb740 10 client.26314 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd6af7cb740 10 client.26314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd6af7cb740 20 client.26314 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/1159600291 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555866ed2d80 0x5558672d4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd6af7cb740 4 client.26314 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.475+0000 7fd6af7cb740 1 -- 172.21.15.67:0/1159600291 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55586722e300 con 0x555866ed2d80 2024-07-31T14:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.476+0000 7fd6a5d9a640 1 --2- 172.21.15.67:0/1159600291 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555866ed2d80 0x5558672d4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.476+0000 7fd6a5d9a640 1 --2- 172.21.15.67:0/1159600291 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555866ed2d80 0x5558672d4210 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af1eaca25137cf4b server_cookie=efb8d85ee22af07e in_seq=0 out_seq=0 2024-07-31T14:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.476+0000 7fd689ffb640 10 client.26314 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd689ffb640 1 -- 172.21.15.67:0/1159600291 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55586722e300 con 0x555866ed2d80 2024-07-31T14:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd689ffb640 10 client.26314 handle_command_reply: tid=0 2024-07-31T14:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd6af7cb740 1 client.26314 shutdown 2024-07-31T14:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd6af7cb740 1 -- 172.21.15.67:0/1159600291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 msgr2=0x5558672c2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/1159600291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672c2400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009ce0 tx=0x7fd69c005740 comp rx=0 tx=0).stop 2024-07-31T14:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd6af7cb740 1 -- 172.21.15.67:0/1159600291 shutdown_connections 2024-07-31T14:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/1159600291 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555866ed2d80 0x5558672d4210 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/1159600291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866f141b0 0x5558672ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd689ffb640 0 client.26314 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/1159600291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 0x5558672c2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/1159600291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672c2400 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd6af7cb740 1 -- 172.21.15.67:0/1159600291 >> 172.21.15.67:0/1159600291 conn(0x555866f08300 msgr2=0x5558672b78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd6af7cb740 1 -- 172.21.15.67:0/1159600291 shutdown_connections 2024-07-31T14:54:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd6af7cb740 1 -- 172.21.15.67:0/1159600291 wait complete. 2024-07-31T14:54:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.477+0000 7fd6af7cb740 20 client.26314 trim_cache size 0 max 16384 2024-07-31T14:54:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.493+0000 7fd6af7cb740 1 Processor -- start 2024-07-31T14:54:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.493+0000 7fd6af7cb740 1 -- start start 2024-07-31T14:54:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.493+0000 7fd6af7cb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866eeb2f0 0x5558672bc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.493+0000 7fd6af7cb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 0x5558672c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.493+0000 7fd6af7cb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866f141b0 0x5558672b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.493+0000 7fd6af7cb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01b0440 con 0x555866eeb2f0 2024-07-31T14:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.493+0000 7fd6af7cb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01b2100 con 0x555866ef34d0 2024-07-31T14:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.493+0000 7fd6af7cb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01b0b10 con 0x555866f141b0 2024-07-31T14:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.493+0000 7fd6a659b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866eeb2f0 0x5558672bc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.493+0000 7fd6a659b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866eeb2f0 0x5558672bc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36820/0 (socket says 172.21.15.67:36820) 2024-07-31T14:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.493+0000 7fd6a659b640 1 -- 172.21.15.67:0/4123243292 learned_addr learned my addr 172.21.15.67:0/4123243292 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.493+0000 7fd6a6d9c640 1 --2- 172.21.15.67:0/4123243292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866f141b0 0x5558672b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.494+0000 7fd6a5d9a640 1 --2- 172.21.15.67:0/4123243292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 0x5558672c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.494+0000 7fd6a659b640 1 -- 172.21.15.67:0/4123243292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866f141b0 msgr2=0x5558672b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.494+0000 7fd6a659b640 1 --2- 172.21.15.67:0/4123243292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866f141b0 0x5558672b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.494+0000 7fd6a659b640 1 -- 172.21.15.67:0/4123243292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 msgr2=0x5558672c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.494+0000 7fd6a659b640 1 --2- 172.21.15.67:0/4123243292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 0x5558672c40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.494+0000 7fd6a659b640 1 -- 172.21.15.67:0/4123243292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558672b96c0 con 0x555866eeb2f0 2024-07-31T14:54:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.494+0000 7fd6a5d9a640 1 --2- 172.21.15.67:0/4123243292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 0x5558672c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:54:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.494+0000 7fd6a659b640 1 --2- 172.21.15.67:0/4123243292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866eeb2f0 0x5558672bc980 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009710 tx=0x7fd69c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333367739accb9b4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.494+0000 7fd68bfff640 1 -- 172.21.15.67:0/4123243292 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c01e070 con 0x555866eeb2f0 2024-07-31T14:54:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.494+0000 7fd68bfff640 1 -- 172.21.15.67:0/4123243292 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd69c027070 con 0x555866eeb2f0 2024-07-31T14:54:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.494+0000 7fd68bfff640 1 -- 172.21.15.67:0/4123243292 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c003de0 con 0x555866eeb2f0 2024-07-31T14:54:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.495+0000 7fd6af7cb740 1 -- 172.21.15.67:0/4123243292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866eeb2f0 msgr2=0x5558672bc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.495+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/4123243292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866eeb2f0 0x5558672bc980 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009710 tx=0x7fd69c00f3c0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.495+0000 7fd6af7cb740 1 -- 172.21.15.67:0/4123243292 shutdown_connections 2024-07-31T14:54:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.495+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/4123243292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866f141b0 0x5558672b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.495+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/4123243292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 0x5558672c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.495+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/4123243292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866eeb2f0 0x5558672bc980 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.495+0000 7fd6af7cb740 1 -- 172.21.15.67:0/4123243292 >> 172.21.15.67:0/4123243292 conn(0x555866f08300 msgr2=0x5558672b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.495+0000 7fd6af7cb740 1 -- 172.21.15.67:0/4123243292 shutdown_connections 2024-07-31T14:54:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.495+0000 7fd6af7cb740 1 -- 172.21.15.67:0/4123243292 wait complete. 2024-07-31T14:54:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.495+0000 7fd6af7cb740 1 -- start start 2024-07-31T14:54:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.495+0000 7fd6af7cb740 1 Processor -- start 2024-07-31T14:54:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6af7cb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6af7cb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 0x5558672c9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6af7cb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866f141b0 0x555867333920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6af7cb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01b1a30 con 0x555866f141b0 2024-07-31T14:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6af7cb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01b12a0 con 0x555866ef34d0 2024-07-31T14:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6af7cb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01ae420 con 0x555866eeb2f0 2024-07-31T14:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6af7cb740 10 client.0 _ll_register_callbacks cb 0x7fd69c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6a659b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6a659b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35922/0 (socket says 172.21.15.67:35922) 2024-07-31T14:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6a6d9c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866f141b0 0x555867333920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6a659b640 1 -- 172.21.15.67:0/252862302 learned_addr learned my addr 172.21.15.67:0/252862302 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6a5d9a640 1 --2- 172.21.15.67:0/252862302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 0x5558672c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6a659b640 1 -- 172.21.15.67:0/252862302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 msgr2=0x5558672c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6a659b640 1 --2- 172.21.15.67:0/252862302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 0x5558672c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6a659b640 1 -- 172.21.15.67:0/252862302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866f141b0 msgr2=0x555867333920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6a659b640 1 --2- 172.21.15.67:0/252862302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866f141b0 0x555867333920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6a659b640 1 -- 172.21.15.67:0/252862302 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555867333f90 con 0x555866eeb2f0 2024-07-31T14:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6a6d9c640 1 --2- 172.21.15.67:0/252862302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866f141b0 0x555867333920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6a659b640 1 --2- 172.21.15.67:0/252862302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672c8b90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009ce0 tx=0x7fd69c00fa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc33048c82160459 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd689ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd689ffb640 1 -- 172.21.15.67:0/252862302 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c039060 con 0x555866eeb2f0 2024-07-31T14:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd689ffb640 1 -- 172.21.15.67:0/252862302 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fd69c0196b0 con 0x555866eeb2f0 2024-07-31T14:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.496+0000 7fd6af7cb740 1 -- 172.21.15.67:0/252862302 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd69c027070 con 0x555866eeb2f0 2024-07-31T14:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.497+0000 7fd689ffb640 1 -- 172.21.15.67:0/252862302 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd69c01e040 con 0x555866eeb2f0 2024-07-31T14:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.497+0000 7fd6af7cb740 10 client.26320 fetch_fsmap learned FSMap version 39 2024-07-31T14:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.497+0000 7fd6af7cb740 1 -- 172.21.15.67:0/252862302 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55586733b390 con 0x555866eeb2f0 2024-07-31T14:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.497+0000 7fd689ffb640 1 -- 172.21.15.67:0/252862302 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd69c027070 con 0x555866eeb2f0 2024-07-31T14:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.497+0000 7fd689ffb640 1 -- 172.21.15.67:0/252862302 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7fd69c021040 con 0x555866eeb2f0 2024-07-31T14:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.497+0000 7fd6af7cb740 10 client.26320 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.497+0000 7fd6af7cb740 10 client.26320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.497+0000 7fd6af7cb740 20 client.26320 populate_metadata read hostname 'smithi067' 2024-07-31T14:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.497+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/252862302 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555866ea81c0 0x55586733d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.497+0000 7fd6af7cb740 4 client.26320 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.497+0000 7fd6af7cb740 1 -- 172.21.15.67:0/252862302 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555867229da0 con 0x555866ea81c0 2024-07-31T14:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.497+0000 7fd6a5d9a640 1 --2- 172.21.15.67:0/252862302 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555866ea81c0 0x55586733d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.498+0000 7fd689ffb640 10 client.26320 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.498+0000 7fd6a5d9a640 1 --2- 172.21.15.67:0/252862302 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555866ea81c0 0x55586733d990 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b539e3d5026aaf7 server_cookie=7a0d1375f8bec78c in_seq=0 out_seq=0 2024-07-31T14:54:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.701+0000 7fd689ffb640 1 -- 172.21.15.67:0/252862302 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555867229da0 con 0x555866ea81c0 2024-07-31T14:54:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.701+0000 7fd689ffb640 10 client.26320 handle_command_reply: tid=0 2024-07-31T14:54:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.701+0000 7fd6af7cb740 1 client.26320 shutdown 2024-07-31T14:54:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.701+0000 7fd6af7cb740 1 -- 172.21.15.67:0/252862302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 msgr2=0x5558672c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.701+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/252862302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672c8b90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009ce0 tx=0x7fd69c00fa30 comp rx=0 tx=0).stop 2024-07-31T14:54:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.701+0000 7fd6af7cb740 1 -- 172.21.15.67:0/252862302 shutdown_connections 2024-07-31T14:54:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.701+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/252862302 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555866ea81c0 0x55586733d990 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.701+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/252862302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555866f141b0 0x555867333920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.701+0000 7fd689ffb640 0 client.26320 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:54:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.701+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/252862302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555866ef34d0 0x5558672c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.701+0000 7fd6af7cb740 1 --2- 172.21.15.67:0/252862302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555866eeb2f0 0x5558672c8b90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.701+0000 7fd6af7cb740 1 -- 172.21.15.67:0/252862302 >> 172.21.15.67:0/252862302 conn(0x555866f08300 msgr2=0x5558672b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6af7cb740 1 -- 172.21.15.67:0/252862302 shutdown_connections 2024-07-31T14:54:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6af7cb740 1 -- 172.21.15.67:0/252862302 wait complete. 2024-07-31T14:54:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6af7cb740 20 client.26320 trim_cache size 0 max 16384 2024-07-31T14:54:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:54:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:54:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:54:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:54:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6a759d640 1 -- 172.21.15.67:0/2522945253 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd68c0b3d30 msgr2=0x7fd68c0b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6a759d640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd68c0b3d30 0x7fd68c0b6240 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd690009230 tx=0x7fd690040040 comp rx=0 tx=0).stop 2024-07-31T14:54:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6a759d640 1 -- 172.21.15.67:0/2522945253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a01084a0 msgr2=0x7fd6a019f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:54:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6a759d640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a01084a0 0x7fd6a019f4f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd698009fd0 tx=0x7fd69800b260 comp rx=0 tx=0).stop 2024-07-31T14:54:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6a759d640 1 -- 172.21.15.67:0/2522945253 shutdown_connections 2024-07-31T14:54:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6a759d640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd68c0b3d30 0x7fd68c0b6240 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6a759d640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a01084a0 0x7fd6a019f4f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6a759d640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6a0107b50 0x7fd6a019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6a759d640 1 --2- 172.21.15.67:0/2522945253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6a0102220 0x7fd6a0199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:54:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6a759d640 1 -- 172.21.15.67:0/2522945253 >> 172.21.15.67:0/2522945253 conn(0x7fd6a00fdc40 msgr2=0x7fd6a0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:54:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6a759d640 1 -- 172.21.15.67:0/2522945253 shutdown_connections 2024-07-31T14:54:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:54:49.702+0000 7fd6a759d640 1 -- 172.21.15.67:0/2522945253 wait complete. 2024-07-31T14:54:49.888 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:54:49.889 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:54:49.889 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:54:49.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:54:49.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:54:49.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:54:50.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:54:50.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:54:50.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:54:51.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:50 smithi073 ceph-mon[36714]: pgmap v513: 129 pgs: 129 active+clean; 364 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 614 op/s 2024-07-31T14:54:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:50 smithi077 ceph-mon[37327]: pgmap v513: 129 pgs: 129 active+clean; 364 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 614 op/s 2024-07-31T14:54:51.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:50 smithi067 ceph-mon[32750]: pgmap v513: 129 pgs: 129 active+clean; 364 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 614 op/s 2024-07-31T14:54:53.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:52 smithi073 ceph-mon[36714]: pgmap v514: 129 pgs: 129 active+clean; 366 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 646 op/s 2024-07-31T14:54:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:52 smithi077 ceph-mon[37327]: pgmap v514: 129 pgs: 129 active+clean; 366 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 646 op/s 2024-07-31T14:54:53.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:52 smithi067 ceph-mon[32750]: pgmap v514: 129 pgs: 129 active+clean; 366 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 646 op/s 2024-07-31T14:54:55.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:54 smithi073 ceph-mon[36714]: pgmap v515: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 689 op/s 2024-07-31T14:54:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:54 smithi077 ceph-mon[37327]: pgmap v515: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 689 op/s 2024-07-31T14:54:55.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:54 smithi067 ceph-mon[32750]: pgmap v515: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 689 op/s 2024-07-31T14:54:57.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:56 smithi073 ceph-mon[36714]: pgmap v516: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 556 op/s 2024-07-31T14:54:57.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:56 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:54:57.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:56 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:54:57.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:56 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:54:57.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:56 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:54:57.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:56 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:54:57.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:56 smithi077 ceph-mon[37327]: pgmap v516: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 556 op/s 2024-07-31T14:54:57.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:56 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:54:57.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:56 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:54:57.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:56 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:54:57.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:56 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:54:57.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:56 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:54:57.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:56 smithi067 ceph-mon[32750]: pgmap v516: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 556 op/s 2024-07-31T14:54:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:56 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:54:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:56 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:54:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:56 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:54:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:56 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:54:57.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:56 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:54:59.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:54:58 smithi073 ceph-mon[36714]: pgmap v517: 129 pgs: 129 active+clean; 372 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 335 op/s 2024-07-31T14:54:59.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:54:58 smithi077 ceph-mon[37327]: pgmap v517: 129 pgs: 129 active+clean; 372 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 335 op/s 2024-07-31T14:54:59.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:54:58 smithi067 ceph-mon[32750]: pgmap v517: 129 pgs: 129 active+clean; 372 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 335 op/s 2024-07-31T14:55:01.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:00 smithi073 ceph-mon[36714]: pgmap v518: 129 pgs: 129 active+clean; 374 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.0 MiB/s wr, 184 op/s 2024-07-31T14:55:01.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:00 smithi077 ceph-mon[37327]: pgmap v518: 129 pgs: 129 active+clean; 374 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.0 MiB/s wr, 184 op/s 2024-07-31T14:55:01.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:00 smithi067 ceph-mon[32750]: pgmap v518: 129 pgs: 129 active+clean; 374 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.0 MiB/s wr, 184 op/s 2024-07-31T14:55:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.886+0000 7f71c09b1640 1 -- 172.21.15.67:0/2300182844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71bc10a330 msgr2=0x7f71bc10a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.886+0000 7f71c09b1640 1 --2- 172.21.15.67:0/2300182844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71bc10a330 0x7f71bc10a740 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009900 tx=0x7f71b002a4d0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.892+0000 7f71c09b1640 1 -- 172.21.15.67:0/2300182844 shutdown_connections 2024-07-31T14:55:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.892+0000 7f71c09b1640 1 --2- 172.21.15.67:0/2300182844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71bc11c5d0 0x7f71bc11eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.892+0000 7f71c09b1640 1 --2- 172.21.15.67:0/2300182844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71bc10a330 0x7f71bc10a740 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.892+0000 7f71c09b1640 1 --2- 172.21.15.67:0/2300182844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71bc1118d0 0x7f71bc109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.892+0000 7f71c09b1640 1 -- 172.21.15.67:0/2300182844 >> 172.21.15.67:0/2300182844 conn(0x7f71bc0707c0 msgr2=0x7f71bc070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.896+0000 7f71c09b1640 1 -- 172.21.15.67:0/2300182844 shutdown_connections 2024-07-31T14:55:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.899+0000 7f71c09b1640 1 -- 172.21.15.67:0/2300182844 wait complete. 2024-07-31T14:55:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.900+0000 7f71c09b1640 1 Processor -- start 2024-07-31T14:55:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.903+0000 7f71c09b1640 1 -- start start 2024-07-31T14:55:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.903+0000 7f71c09b1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71bc10a330 0x7f71bc1a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.903+0000 7f71c09b1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71bc1118d0 0x7f71bc1a9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.903+0000 7f71c09b1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71bc11c5d0 0x7f71bc1ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.903+0000 7f71c09b1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1222e0 con 0x7f71bc11c5d0 2024-07-31T14:55:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.903+0000 7f71c09b1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc122160 con 0x7f71bc1118d0 2024-07-31T14:55:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.903+0000 7f71c09b1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc122460 con 0x7f71bc10a330 2024-07-31T14:55:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.905+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71bc1118d0 0x7f71bc1a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.905+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71bc1118d0 0x7f71bc1a9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52762/0 (socket says 172.21.15.67:52762) 2024-07-31T14:55:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.905+0000 7f71b9d9b640 1 -- 172.21.15.67:0/793781564 learned_addr learned my addr 172.21.15.67:0/793781564 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.906+0000 7f71bad9d640 1 --2- 172.21.15.67:0/793781564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71bc11c5d0 0x7f71bc1ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.907+0000 7f71bad9d640 1 -- 172.21.15.67:0/793781564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71bc10a330 msgr2=0x7f71bc1a8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.907+0000 7f71bad9d640 1 --2- 172.21.15.67:0/793781564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71bc10a330 0x7f71bc1a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.907+0000 7f71bad9d640 1 -- 172.21.15.67:0/793781564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71bc1118d0 msgr2=0x7f71bc1a9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.907+0000 7f71bad9d640 1 --2- 172.21.15.67:0/793781564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71bc1118d0 0x7f71bc1a9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.907+0000 7f71bad9d640 1 -- 172.21.15.67:0/793781564 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc1aebf0 con 0x7f71bc11c5d0 2024-07-31T14:55:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.907+0000 7f71bad9d640 1 --2- 172.21.15.67:0/793781564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71bc11c5d0 0x7f71bc1ae4f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f71b400ca30 tx=0x7f71b400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.908+0000 7f71ab7fe640 1 -- 172.21.15.67:0/793781564 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4007d90 con 0x7f71bc11c5d0 2024-07-31T14:55:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.908+0000 7f71ab7fe640 1 -- 172.21.15.67:0/793781564 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b4005020 con 0x7f71bc11c5d0 2024-07-31T14:55:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.908+0000 7f71c09b1640 1 -- 172.21.15.67:0/793781564 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71bc1aeee0 con 0x7f71bc11c5d0 2024-07-31T14:55:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.908+0000 7f71c09b1640 1 -- 172.21.15.67:0/793781564 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71bc1b5670 con 0x7f71bc11c5d0 2024-07-31T14:55:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.910+0000 7f71ab7fe640 1 -- 172.21.15.67:0/793781564 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b400f6b0 con 0x7f71bc11c5d0 2024-07-31T14:55:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.915+0000 7f71ab7fe640 1 -- 172.21.15.67:0/793781564 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f71b400f850 con 0x7f71bc11c5d0 2024-07-31T14:55:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.916+0000 7f71c7982740 1 Processor -- start 2024-07-31T14:55:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.918+0000 7f71c7982740 1 -- start start 2024-07-31T14:55:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.918+0000 7f71c7982740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f5a31b0 0x55643f945ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.918+0000 7f71c7982740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f57a2f0 0x55643f946220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.918+0000 7f71c7982740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f946760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.918+0000 7f71c7982740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55643f8bb3b0 con 0x55643f5a31b0 2024-07-31T14:55:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.918+0000 7f71c7982740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55643f8aa8b0 con 0x55643f5824d0 2024-07-31T14:55:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.918+0000 7f71c7982740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55643f8ae8f0 con 0x55643f57a2f0 2024-07-31T14:55:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.922+0000 7f71ab7fe640 1 --2- 172.21.15.67:0/793781564 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f71a40b3e70 0x7f71a40b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.923+0000 7f71ab7fe640 1 -- 172.21.15.67:0/793781564 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f71b40e4510 con 0x7f71bc11c5d0 2024-07-31T14:55:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.923+0000 7f71ba59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f57a2f0 0x55643f946220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.923+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f946760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.923+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f5a31b0 0x55643f945ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.923+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f5a31b0 0x55643f945ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38596/0 (socket says 172.21.15.67:38596) 2024-07-31T14:55:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.923+0000 7f71b9d9b640 1 -- 172.21.15.67:0/2000084320 learned_addr learned my addr 172.21.15.67:0/2000084320 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.923+0000 7f71bad9d640 1 --2- 172.21.15.67:0/793781564 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f71a40b3e70 0x7f71a40b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.924+0000 7f71b9d9b640 1 -- 172.21.15.67:0/2000084320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f57a2f0 msgr2=0x55643f946220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.924+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/2000084320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f57a2f0 0x55643f946220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.924+0000 7f71b9d9b640 1 -- 172.21.15.67:0/2000084320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 msgr2=0x55643f946760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.924+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/2000084320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f946760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.924+0000 7f71b9d9b640 1 -- 172.21.15.67:0/2000084320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55643f8519a0 con 0x55643f5a31b0 2024-07-31T14:55:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.924+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/2000084320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f5a31b0 0x55643f945ce0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f71b00099f0 tx=0x7f71b0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd45832248366567 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.927+0000 7f71a8ff9640 1 -- 172.21.15.67:0/2000084320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0040270 con 0x55643f5a31b0 2024-07-31T14:55:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.927+0000 7f71a8ff9640 1 -- 172.21.15.67:0/2000084320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b0040ca0 con 0x55643f5a31b0 2024-07-31T14:55:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.927+0000 7f71a8ff9640 1 -- 172.21.15.67:0/2000084320 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b003cb80 con 0x55643f5a31b0 2024-07-31T14:55:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.929+0000 7f71c7982740 1 -- 172.21.15.67:0/2000084320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f5a31b0 msgr2=0x55643f945ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.929+0000 7f71c7982740 1 --2- 172.21.15.67:0/2000084320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f5a31b0 0x55643f945ce0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f71b00099f0 tx=0x7f71b0005fd0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.932+0000 7f71bad9d640 1 --2- 172.21.15.67:0/793781564 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f71a40b3e70 0x7f71a40b6380 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f71b4007fd0 tx=0x7f71b40aa790 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.933+0000 7f71c7982740 1 -- 172.21.15.67:0/2000084320 shutdown_connections 2024-07-31T14:55:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.933+0000 7f71c7982740 1 --2- 172.21.15.67:0/2000084320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f946760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.933+0000 7f71c7982740 1 --2- 172.21.15.67:0/2000084320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f57a2f0 0x55643f946220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.933+0000 7f71c7982740 1 --2- 172.21.15.67:0/2000084320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f5a31b0 0x55643f945ce0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.933+0000 7f71c7982740 1 -- 172.21.15.67:0/2000084320 >> 172.21.15.67:0/2000084320 conn(0x55643f597300 msgr2=0x55643f943bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.936+0000 7f71c7982740 1 -- 172.21.15.67:0/2000084320 shutdown_connections 2024-07-31T14:55:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.937+0000 7f71c7982740 1 -- 172.21.15.67:0/2000084320 wait complete. 2024-07-31T14:55:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.938+0000 7f71c7982740 1 -- start start 2024-07-31T14:55:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.938+0000 7f71c7982740 1 Processor -- start 2024-07-31T14:55:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.940+0000 7f71c7982740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f57a2f0 0x55643f951860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.940+0000 7f71c7982740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f951da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.940+0000 7f71c7982740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f5a31b0 0x55643f95d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.940+0000 7f71c7982740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55643f7e4ab0 con 0x55643f57a2f0 2024-07-31T14:55:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.940+0000 7f71c7982740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55643f832cf0 con 0x55643f5824d0 2024-07-31T14:55:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.940+0000 7f71c7982740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55643f954400 con 0x55643f5a31b0 2024-07-31T14:55:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.940+0000 7f71c7982740 10 client.0 _ll_register_callbacks cb 0x55643f8a49f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.940+0000 7f71ba59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f951da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.940+0000 7f71ba59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f951da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52774/0 (socket says 172.21.15.67:52774) 2024-07-31T14:55:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.940+0000 7f71ba59c640 1 -- 172.21.15.67:0/1017701155 learned_addr learned my addr 172.21.15.67:0/1017701155 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.940+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/1017701155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f57a2f0 0x55643f951860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.941+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/1017701155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f5a31b0 0x55643f95d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.941+0000 7f71b9d9b640 1 -- 172.21.15.67:0/1017701155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f5a31b0 msgr2=0x55643f95d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.941+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/1017701155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f5a31b0 0x55643f95d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.941+0000 7f71b9d9b640 1 -- 172.21.15.67:0/1017701155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 msgr2=0x55643f951da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.941+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/1017701155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f951da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.941+0000 7f71b9d9b640 1 -- 172.21.15.67:0/1017701155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55643f60e790 con 0x55643f57a2f0 2024-07-31T14:55:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.942+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/1017701155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f57a2f0 0x55643f951860 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f71b00092a0 tx=0x7f71b003c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a634c15b0a50625 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.942+0000 7f71a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:55:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.944+0000 7f71a1ffb640 1 -- 172.21.15.67:0/1017701155 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0067060 con 0x55643f57a2f0 2024-07-31T14:55:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.945+0000 7f71a1ffb640 1 -- 172.21.15.67:0/1017701155 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f71b00409f0 con 0x55643f57a2f0 2024-07-31T14:55:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.948+0000 7f71c7982740 1 -- 172.21.15.67:0/1017701155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b0040ca0 con 0x55643f57a2f0 2024-07-31T14:55:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.949+0000 7f71a1ffb640 1 -- 172.21.15.67:0/1017701155 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f71b003c900 con 0x55643f57a2f0 2024-07-31T14:55:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.949+0000 7f71c7982740 10 client.16698 fetch_fsmap learned FSMap version 39 2024-07-31T14:55:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.949+0000 7f71c7982740 1 -- 172.21.15.67:0/1017701155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55643f687620 con 0x55643f57a2f0 2024-07-31T14:55:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.950+0000 7f71a1ffb640 1 -- 172.21.15.67:0/1017701155 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b0040ca0 con 0x55643f57a2f0 2024-07-31T14:55:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.950+0000 7f71a1ffb640 1 -- 172.21.15.67:0/1017701155 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f71b0031910 con 0x55643f57a2f0 2024-07-31T14:55:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.950+0000 7f71c7982740 10 client.16698 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:55:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.950+0000 7f71c7982740 10 client.16698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.950+0000 7f71c7982740 20 client.16698 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.950+0000 7f71c7982740 1 --2- 172.21.15.67:0/1017701155 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55643f561d80 0x55643f963690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.950+0000 7f71c7982740 4 client.16698 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:55:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.950+0000 7f71c7982740 1 -- 172.21.15.67:0/1017701155 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55643f8bd300 con 0x55643f561d80 2024-07-31T14:55:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.953+0000 7f71ba59c640 1 --2- 172.21.15.67:0/1017701155 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55643f561d80 0x55643f963690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.961+0000 7f71ba59c640 1 --2- 172.21.15.67:0/1017701155 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55643f561d80 0x55643f963690 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1abe64b528925a0d server_cookie=d0a5e0ff3218ed38 in_seq=0 out_seq=0 2024-07-31T14:55:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.963+0000 7f71a1ffb640 10 client.16698 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.965+0000 7f71a1ffb640 1 -- 172.21.15.67:0/1017701155 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55643f8bd300 con 0x55643f561d80 2024-07-31T14:55:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.965+0000 7f71a1ffb640 10 client.16698 handle_command_reply: tid=0 2024-07-31T14:55:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.965+0000 7f71c7982740 1 client.16698 shutdown 2024-07-31T14:55:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.968+0000 7f71c7982740 1 -- 172.21.15.67:0/1017701155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f57a2f0 msgr2=0x55643f951860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.968+0000 7f71c7982740 1 --2- 172.21.15.67:0/1017701155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f57a2f0 0x55643f951860 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f71b00092a0 tx=0x7f71b003c760 comp rx=0 tx=0).stop 2024-07-31T14:55:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.974+0000 7f71c7982740 1 -- 172.21.15.67:0/1017701155 shutdown_connections 2024-07-31T14:55:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.974+0000 7f71c7982740 1 --2- 172.21.15.67:0/1017701155 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55643f561d80 0x55643f963690 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.974+0000 7f71c7982740 1 --2- 172.21.15.67:0/1017701155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f5a31b0 0x55643f95d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.974+0000 7f71c7982740 1 --2- 172.21.15.67:0/1017701155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f951da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.974+0000 7f71c7982740 1 --2- 172.21.15.67:0/1017701155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f57a2f0 0x55643f951860 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.974+0000 7f71c7982740 1 -- 172.21.15.67:0/1017701155 >> 172.21.15.67:0/1017701155 conn(0x55643f597300 msgr2=0x55643f9422b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.974+0000 7f71a1ffb640 0 client.16698 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.975+0000 7f71c7982740 1 -- 172.21.15.67:0/1017701155 shutdown_connections 2024-07-31T14:55:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.977+0000 7f71c7982740 1 -- 172.21.15.67:0/1017701155 wait complete. 2024-07-31T14:55:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:02.978+0000 7f71c7982740 20 client.16698 trim_cache size 0 max 16384 2024-07-31T14:55:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.017+0000 7f71c7982740 1 Processor -- start 2024-07-31T14:55:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.021+0000 7f71c7982740 1 -- start start 2024-07-31T14:55:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.021+0000 7f71c7982740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f57a2f0 0x55643f943c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.021+0000 7f71c7982740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f94bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.021+0000 7f71c7982740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f5a31b0 0x55643f95d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.021+0000 7f71c7982740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f719c00fe70 con 0x55643f57a2f0 2024-07-31T14:55:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.021+0000 7f71c7982740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f719c011b30 con 0x55643f5824d0 2024-07-31T14:55:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.021+0000 7f71c7982740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f719c010540 con 0x55643f5a31b0 2024-07-31T14:55:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.022+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f5a31b0 0x55643f95d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.022+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f5a31b0 0x55643f95d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:33744/0 (socket says 172.21.15.67:33744) 2024-07-31T14:55:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.022+0000 7f71b9d9b640 1 -- 172.21.15.67:0/1311729846 learned_addr learned my addr 172.21.15.67:0/1311729846 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.023+0000 7f71b9d9b640 1 -- 172.21.15.67:0/1311729846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 msgr2=0x55643f94bd80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:55:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.023+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/1311729846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f94bd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.023+0000 7f71b9d9b640 1 -- 172.21.15.67:0/1311729846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f57a2f0 msgr2=0x55643f943c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.023+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/1311729846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f57a2f0 0x55643f943c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.023+0000 7f71b9d9b640 1 -- 172.21.15.67:0/1311729846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55643f948af0 con 0x55643f5a31b0 2024-07-31T14:55:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.027+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/1311729846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f5a31b0 0x55643f95d570 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71b0031e80 tx=0x7f71b002a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc32931d67ea00f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.031+0000 7f71a3fff640 1 -- 172.21.15.67:0/1311729846 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0067040 con 0x55643f5a31b0 2024-07-31T14:55:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.032+0000 7f71a3fff640 1 -- 172.21.15.67:0/1311729846 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b0040e50 con 0x55643f5a31b0 2024-07-31T14:55:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.032+0000 7f71a3fff640 1 -- 172.21.15.67:0/1311729846 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0071080 con 0x55643f5a31b0 2024-07-31T14:55:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.033+0000 7f71c7982740 1 -- 172.21.15.67:0/1311729846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f5a31b0 msgr2=0x55643f95d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.033+0000 7f71c7982740 1 --2- 172.21.15.67:0/1311729846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f5a31b0 0x55643f95d570 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71b0031e80 tx=0x7f71b002a4d0 comp rx=0 tx=0).stop 2024-07-31T14:55:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.035+0000 7f71c7982740 1 -- 172.21.15.67:0/1311729846 shutdown_connections 2024-07-31T14:55:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.035+0000 7f71c7982740 1 --2- 172.21.15.67:0/1311729846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f5a31b0 0x55643f95d570 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.035+0000 7f71c7982740 1 --2- 172.21.15.67:0/1311729846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f94bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.035+0000 7f71c7982740 1 --2- 172.21.15.67:0/1311729846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f57a2f0 0x55643f943c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.035+0000 7f71c7982740 1 -- 172.21.15.67:0/1311729846 >> 172.21.15.67:0/1311729846 conn(0x55643f597300 msgr2=0x55643f9434f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.038+0000 7f71c7982740 1 -- 172.21.15.67:0/1311729846 shutdown_connections 2024-07-31T14:55:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.040+0000 7f71c7982740 1 -- 172.21.15.67:0/1311729846 wait complete. 2024-07-31T14:55:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.041+0000 7f71c7982740 1 -- start start 2024-07-31T14:55:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.041+0000 7f71c7982740 1 Processor -- start 2024-07-31T14:55:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.045+0000 7f71c7982740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f57a2f0 0x55643f9566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.045+0000 7f71c7982740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f956bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.045+0000 7f71c7982740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f5a31b0 0x55643f957130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.046+0000 7f71c7982740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f719c011460 con 0x55643f5a31b0 2024-07-31T14:55:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.046+0000 7f71c7982740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f719c010cd0 con 0x55643f5824d0 2024-07-31T14:55:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.046+0000 7f71c7982740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f719c00de50 con 0x55643f57a2f0 2024-07-31T14:55:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.046+0000 7f71c7982740 10 client.0 _ll_register_callbacks cb 0x7f71b0004960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.049+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f5a31b0 0x55643f957130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.049+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f5a31b0 0x55643f957130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38620/0 (socket says 172.21.15.67:38620) 2024-07-31T14:55:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.049+0000 7f71b9d9b640 1 -- 172.21.15.67:0/2336261961 learned_addr learned my addr 172.21.15.67:0/2336261961 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.049+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/2336261961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f57a2f0 0x55643f9566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.050+0000 7f71ba59c640 1 --2- 172.21.15.67:0/2336261961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f956bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.052+0000 7f71b9d9b640 1 -- 172.21.15.67:0/2336261961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 msgr2=0x55643f956bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.052+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/2336261961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f956bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.052+0000 7f71b9d9b640 1 -- 172.21.15.67:0/2336261961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f5a31b0 msgr2=0x55643f957130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.052+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/2336261961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f5a31b0 0x55643f957130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.052+0000 7f71b9d9b640 1 -- 172.21.15.67:0/2336261961 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55643f9c9e60 con 0x55643f57a2f0 2024-07-31T14:55:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.056+0000 7f71b9d9b640 1 --2- 172.21.15.67:0/2336261961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f57a2f0 0x55643f9566b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f71b00092a0 tx=0x7f71b0005990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85b545c8691dd881 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.056+0000 7f71a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.057+0000 7f71a1ffb640 1 -- 172.21.15.67:0/2336261961 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b00408f0 con 0x55643f57a2f0 2024-07-31T14:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.057+0000 7f71a1ffb640 1 -- 172.21.15.67:0/2336261961 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f71b0040a90 con 0x55643f57a2f0 2024-07-31T14:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.057+0000 7f71c7982740 1 -- 172.21.15.67:0/2336261961 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b0040e50 con 0x55643f57a2f0 2024-07-31T14:55:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.063+0000 7f71a1ffb640 1 -- 172.21.15.67:0/2336261961 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f71b0040c90 con 0x55643f57a2f0 2024-07-31T14:55:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.064+0000 7f71c7982740 10 client.26332 fetch_fsmap learned FSMap version 39 2024-07-31T14:55:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.064+0000 7f71c7982740 1 -- 172.21.15.67:0/2336261961 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55643f9cb0c0 con 0x55643f57a2f0 2024-07-31T14:55:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.067+0000 7f71a1ffb640 1 -- 172.21.15.67:0/2336261961 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b0040e50 con 0x55643f57a2f0 2024-07-31T14:55:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.067+0000 7f71a1ffb640 1 -- 172.21.15.67:0/2336261961 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f71b003e9f0 con 0x55643f57a2f0 2024-07-31T14:55:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.067+0000 7f71c7982740 10 client.26332 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:55:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.067+0000 7f71c7982740 10 client.26332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.067+0000 7f71c7982740 20 client.26332 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.067+0000 7f71c7982740 1 --2- 172.21.15.67:0/2336261961 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55643f5371c0 0x55643f9cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.067+0000 7f71c7982740 4 client.26332 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:55:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.067+0000 7f71c7982740 1 -- 172.21.15.67:0/2336261961 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55643f8b8da0 con 0x55643f5371c0 2024-07-31T14:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.069+0000 7f71ba59c640 1 --2- 172.21.15.67:0/2336261961 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55643f5371c0 0x55643f9cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.086+0000 7f71ba59c640 1 --2- 172.21.15.67:0/2336261961 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55643f5371c0 0x55643f9cd660 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90a0bed610210bc1 server_cookie=99034a2f69ec28ba in_seq=0 out_seq=0 2024-07-31T14:55:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:03.086+0000 7f71a1ffb640 10 client.26332 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:03.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:02 smithi073 ceph-mon[36714]: pgmap v519: 129 pgs: 129 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2024-07-31T14:55:03.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:02 smithi077 ceph-mon[37327]: pgmap v519: 129 pgs: 129 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2024-07-31T14:55:03.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:02 smithi067 ceph-mon[32750]: pgmap v519: 129 pgs: 129 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2024-07-31T14:55:05.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:04 smithi073 ceph-mon[36714]: pgmap v520: 129 pgs: 129 active+clean; 378 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-07-31T14:55:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:04 smithi077 ceph-mon[37327]: pgmap v520: 129 pgs: 129 active+clean; 378 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-07-31T14:55:05.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:04 smithi067 ceph-mon[32750]: pgmap v520: 129 pgs: 129 active+clean; 378 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-07-31T14:55:07.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:06 smithi073 ceph-mon[36714]: pgmap v521: 129 pgs: 129 active+clean; 408 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 3.5 MiB/s wr, 378 op/s 2024-07-31T14:55:07.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:06 smithi077 ceph-mon[37327]: pgmap v521: 129 pgs: 129 active+clean; 408 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 3.5 MiB/s wr, 378 op/s 2024-07-31T14:55:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71a1ffb640 1 -- 172.21.15.67:0/2336261961 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55643f8b8da0 con 0x55643f5371c0 2024-07-31T14:55:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71a1ffb640 10 client.26332 handle_command_reply: tid=0 2024-07-31T14:55:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71c7982740 1 client.26332 shutdown 2024-07-31T14:55:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71c7982740 1 -- 172.21.15.67:0/2336261961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f57a2f0 msgr2=0x55643f9566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71c7982740 1 --2- 172.21.15.67:0/2336261961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f57a2f0 0x55643f9566b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f71b00092a0 tx=0x7f71b0005990 comp rx=0 tx=0).stop 2024-07-31T14:55:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71c7982740 1 -- 172.21.15.67:0/2336261961 shutdown_connections 2024-07-31T14:55:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71c7982740 1 --2- 172.21.15.67:0/2336261961 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55643f5371c0 0x55643f9cd660 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71c7982740 1 --2- 172.21.15.67:0/2336261961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55643f5a31b0 0x55643f957130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71a1ffb640 0 client.26332 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71c7982740 1 --2- 172.21.15.67:0/2336261961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55643f5824d0 0x55643f956bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71c7982740 1 --2- 172.21.15.67:0/2336261961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55643f57a2f0 0x55643f9566b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71c7982740 1 -- 172.21.15.67:0/2336261961 >> 172.21.15.67:0/2336261961 conn(0x55643f597300 msgr2=0x55643f946bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71c7982740 1 -- 172.21.15.67:0/2336261961 shutdown_connections 2024-07-31T14:55:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71c7982740 1 -- 172.21.15.67:0/2336261961 wait complete. 2024-07-31T14:55:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.240+0000 7f71c7982740 20 client.26332 trim_cache size 0 max 16384 2024-07-31T14:55:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:55:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:55:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:55:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:55:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.241+0000 7f71c09b1640 1 -- 172.21.15.67:0/793781564 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f71a40b3e70 msgr2=0x7f71a40b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.241+0000 7f71c09b1640 1 --2- 172.21.15.67:0/793781564 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f71a40b3e70 0x7f71a40b6380 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f71b4007fd0 tx=0x7f71b40aa790 comp rx=0 tx=0).stop 2024-07-31T14:55:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.241+0000 7f71c09b1640 1 -- 172.21.15.67:0/793781564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71bc11c5d0 msgr2=0x7f71bc1ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.241+0000 7f71c09b1640 1 --2- 172.21.15.67:0/793781564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71bc11c5d0 0x7f71bc1ae4f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f71b400ca30 tx=0x7f71b400cef0 comp rx=0 tx=0).stop 2024-07-31T14:55:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.241+0000 7f71c09b1640 1 -- 172.21.15.67:0/793781564 shutdown_connections 2024-07-31T14:55:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.241+0000 7f71c09b1640 1 --2- 172.21.15.67:0/793781564 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f71a40b3e70 0x7f71a40b6380 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.241+0000 7f71c09b1640 1 --2- 172.21.15.67:0/793781564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71bc11c5d0 0x7f71bc1ae4f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.241+0000 7f71c09b1640 1 --2- 172.21.15.67:0/793781564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71bc1118d0 0x7f71bc1a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.241+0000 7f71c09b1640 1 --2- 172.21.15.67:0/793781564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71bc10a330 0x7f71bc1a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.241+0000 7f71c09b1640 1 -- 172.21.15.67:0/793781564 >> 172.21.15.67:0/793781564 conn(0x7f71bc0707c0 msgr2=0x7f71bc11a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.241+0000 7f71c09b1640 1 -- 172.21.15.67:0/793781564 shutdown_connections 2024-07-31T14:55:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:07.241+0000 7f71c09b1640 1 -- 172.21.15.67:0/793781564 wait complete. 2024-07-31T14:55:07.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:06 smithi067 ceph-mon[32750]: pgmap v521: 129 pgs: 129 active+clean; 408 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 3.5 MiB/s wr, 378 op/s 2024-07-31T14:55:09.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:08 smithi073 ceph-mon[36714]: pgmap v522: 129 pgs: 129 active+clean; 460 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.0 MiB/s wr, 688 op/s 2024-07-31T14:55:09.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:08 smithi077 ceph-mon[37327]: pgmap v522: 129 pgs: 129 active+clean; 460 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.0 MiB/s wr, 688 op/s 2024-07-31T14:55:09.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:08 smithi067 ceph-mon[32750]: pgmap v522: 129 pgs: 129 active+clean; 460 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.0 MiB/s wr, 688 op/s 2024-07-31T14:55:11.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:10 smithi073 ceph-mon[36714]: pgmap v523: 129 pgs: 129 active+clean; 475 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.2 MiB/s wr, 998 op/s 2024-07-31T14:55:11.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:10 smithi077 ceph-mon[37327]: pgmap v523: 129 pgs: 129 active+clean; 475 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.2 MiB/s wr, 998 op/s 2024-07-31T14:55:11.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:10 smithi067 ceph-mon[32750]: pgmap v523: 129 pgs: 129 active+clean; 475 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.2 MiB/s wr, 998 op/s 2024-07-31T14:55:13.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:12 smithi073 ceph-mon[36714]: pgmap v524: 129 pgs: 129 active+clean; 471 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.3 MiB/s wr, 1.05k op/s 2024-07-31T14:55:13.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:12 smithi077 ceph-mon[37327]: pgmap v524: 129 pgs: 129 active+clean; 471 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.3 MiB/s wr, 1.05k op/s 2024-07-31T14:55:13.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:12 smithi067 ceph-mon[32750]: pgmap v524: 129 pgs: 129 active+clean; 471 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.3 MiB/s wr, 1.05k op/s 2024-07-31T14:55:15.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:14 smithi073 ceph-mon[36714]: pgmap v525: 129 pgs: 129 active+clean; 477 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.8 MiB/s wr, 1.15k op/s 2024-07-31T14:55:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:14 smithi077 ceph-mon[37327]: pgmap v525: 129 pgs: 129 active+clean; 477 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.8 MiB/s wr, 1.15k op/s 2024-07-31T14:55:15.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:14 smithi067 ceph-mon[32750]: pgmap v525: 129 pgs: 129 active+clean; 477 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.8 MiB/s wr, 1.15k op/s 2024-07-31T14:55:16.427 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:55:16.713 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:55:17.083 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:16 smithi067 ceph-mon[32750]: pgmap v526: 129 pgs: 129 active+clean; 478 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.8 MiB/s wr, 1.12k op/s 2024-07-31T14:55:17.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:16 smithi073 ceph-mon[36714]: pgmap v526: 129 pgs: 129 active+clean; 478 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.8 MiB/s wr, 1.12k op/s 2024-07-31T14:55:17.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:16 smithi077 ceph-mon[37327]: pgmap v526: 129 pgs: 129 active+clean; 478 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.8 MiB/s wr, 1.12k op/s 2024-07-31T14:55:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.600+0000 7f182f625640 1 -- 172.21.15.67:0/3782270199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1828102960 msgr2=0x7f182810f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.600+0000 7f182f625640 1 --2- 172.21.15.67:0/3782270199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1828102960 0x7f182810f680 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1824009900 tx=0x7f182402a480 comp rx=0 tx=0).stop 2024-07-31T14:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.600+0000 7f182f625640 1 -- 172.21.15.67:0/3782270199 shutdown_connections 2024-07-31T14:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.600+0000 7f182f625640 1 --2- 172.21.15.67:0/3782270199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f182810fbc0 0x7f1828111fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.600+0000 7f182f625640 1 --2- 172.21.15.67:0/3782270199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1828102960 0x7f182810f680 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.600+0000 7f182f625640 1 --2- 172.21.15.67:0/3782270199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1828102010 0x7f1828102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.600+0000 7f182f625640 1 -- 172.21.15.67:0/3782270199 >> 172.21.15.67:0/3782270199 conn(0x7f1828079140 msgr2=0x7f1828079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.600+0000 7f182f625640 1 -- 172.21.15.67:0/3782270199 shutdown_connections 2024-07-31T14:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.600+0000 7f182f625640 1 -- 172.21.15.67:0/3782270199 wait complete. 2024-07-31T14:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182f625640 1 Processor -- start 2024-07-31T14:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182f625640 1 -- start start 2024-07-31T14:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182f625640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1828102010 0x7f182819c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182f625640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1828102960 0x7f182819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182f625640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f182810fbc0 0x7f18281a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182f625640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1828115690 con 0x7f182810fbc0 2024-07-31T14:55:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182f625640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1828115510 con 0x7f1828102960 2024-07-31T14:55:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182f625640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1828115810 con 0x7f1828102010 2024-07-31T14:55:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182dbc2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f182810fbc0 0x7f18281a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182dbc2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f182810fbc0 0x7f18281a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43158/0 (socket says 172.21.15.67:43158) 2024-07-31T14:55:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182dbc2640 1 -- 172.21.15.67:0/726353835 learned_addr learned my addr 172.21.15.67:0/726353835 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182cbc0640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1828102960 0x7f182819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182d3c1640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1828102010 0x7f182819c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182dbc2640 1 -- 172.21.15.67:0/726353835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1828102010 msgr2=0x7f182819c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182dbc2640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1828102010 0x7f182819c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182dbc2640 1 -- 172.21.15.67:0/726353835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1828102960 msgr2=0x7f182819c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182dbc2640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1828102960 0x7f182819c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182dbc2640 1 -- 172.21.15.67:0/726353835 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18281a2050 con 0x7f182810fbc0 2024-07-31T14:55:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182cbc0640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1828102960 0x7f182819c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:55:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.601+0000 7f182d3c1640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1828102010 0x7f182819c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:55:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.602+0000 7f182dbc2640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f182810fbc0 0x7f18281a1920 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f182000eaa0 tx=0x7f182000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.602+0000 7f181e7fc640 1 -- 172.21.15.67:0/726353835 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182000cdb0 con 0x7f182810fbc0 2024-07-31T14:55:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.602+0000 7f181e7fc640 1 -- 172.21.15.67:0/726353835 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1820005020 con 0x7f182810fbc0 2024-07-31T14:55:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.602+0000 7f182f625640 1 -- 172.21.15.67:0/726353835 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18281a2340 con 0x7f182810fbc0 2024-07-31T14:55:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.602+0000 7f181e7fc640 1 -- 172.21.15.67:0/726353835 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18200116d0 con 0x7f182810fbc0 2024-07-31T14:55:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.602+0000 7f182f625640 1 -- 172.21.15.67:0/726353835 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18281a8aa0 con 0x7f182810fbc0 2024-07-31T14:55:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.605+0000 7f1835b76740 1 Processor -- start 2024-07-31T14:55:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.605+0000 7f181e7fc640 1 -- 172.21.15.67:0/726353835 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f18200051d0 con 0x7f182810fbc0 2024-07-31T14:55:18.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.605+0000 7f1835b76740 1 -- start start 2024-07-31T14:55:18.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.605+0000 7f1835b76740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523372070 0x557523717640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.605+0000 7f1835b76740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233491b0 0x557523717b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.605+0000 7f1835b76740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523351390 0x5575237180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.605+0000 7f1835b76740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55752368db60 con 0x557523351390 2024-07-31T14:55:18.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.605+0000 7f1835b76740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55752367d060 con 0x5575233491b0 2024-07-31T14:55:18.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.605+0000 7f1835b76740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5575236810a0 con 0x557523372070 2024-07-31T14:55:18.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.605+0000 7f182cbc0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523351390 0x5575237180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.605+0000 7f182cbc0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523351390 0x5575237180c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43174/0 (socket says 172.21.15.67:43174) 2024-07-31T14:55:18.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.605+0000 7f182cbc0640 1 -- 172.21.15.67:0/2209476132 learned_addr learned my addr 172.21.15.67:0/2209476132 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:18.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.605+0000 7f182cbc0640 1 --2- 172.21.15.67:0/2209476132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523372070 0x557523717640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.605+0000 7f182d3c1640 1 --2- 172.21.15.67:0/2209476132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233491b0 0x557523717b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.606+0000 7f182cbc0640 1 -- 172.21.15.67:0/2209476132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523372070 msgr2=0x557523717640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:18.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.606+0000 7f182cbc0640 1 --2- 172.21.15.67:0/2209476132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523372070 0x557523717640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.606+0000 7f182cbc0640 1 -- 172.21.15.67:0/2209476132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233491b0 msgr2=0x557523717b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:18.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.606+0000 7f182cbc0640 1 --2- 172.21.15.67:0/2209476132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233491b0 0x557523717b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.606+0000 7f182cbc0640 1 -- 172.21.15.67:0/2209476132 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557523623b40 con 0x557523351390 2024-07-31T14:55:18.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.606+0000 7f181e7fc640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18140b3d40 0x7f18140b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.606+0000 7f182d3c1640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18140b3d40 0x7f18140b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.606+0000 7f182cbc0640 1 --2- 172.21.15.67:0/2209476132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523351390 0x5575237180c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1824005f70 tx=0x7f1824031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a8b72cb682be05 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:18.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.606+0000 7f181e7fc640 1 -- 172.21.15.67:0/726353835 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f18200e4550 con 0x7f182810fbc0 2024-07-31T14:55:18.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.606+0000 7f18137fe640 1 -- 172.21.15.67:0/2209476132 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1824035040 con 0x557523351390 2024-07-31T14:55:18.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.606+0000 7f18137fe640 1 -- 172.21.15.67:0/2209476132 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18240035f0 con 0x557523351390 2024-07-31T14:55:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.606+0000 7f18137fe640 1 -- 172.21.15.67:0/2209476132 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18240044e0 con 0x557523351390 2024-07-31T14:55:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.607+0000 7f1835b76740 1 -- 172.21.15.67:0/2209476132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523351390 msgr2=0x5575237180c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.607+0000 7f182d3c1640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18140b3d40 0x7f18140b6250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5575237162c0 tx=0x7f181800b3f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.607+0000 7f1835b76740 1 --2- 172.21.15.67:0/2209476132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523351390 0x5575237180c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1824005f70 tx=0x7f1824031400 comp rx=0 tx=0).stop 2024-07-31T14:55:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.607+0000 7f1835b76740 1 -- 172.21.15.67:0/2209476132 shutdown_connections 2024-07-31T14:55:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.607+0000 7f1835b76740 1 --2- 172.21.15.67:0/2209476132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523351390 0x5575237180c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.607+0000 7f1835b76740 1 --2- 172.21.15.67:0/2209476132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233491b0 0x557523717b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.607+0000 7f1835b76740 1 --2- 172.21.15.67:0/2209476132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523372070 0x557523717640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.607+0000 7f1835b76740 1 -- 172.21.15.67:0/2209476132 >> 172.21.15.67:0/2209476132 conn(0x557523366460 msgr2=0x557523715530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.607+0000 7f1835b76740 1 -- 172.21.15.67:0/2209476132 shutdown_connections 2024-07-31T14:55:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.607+0000 7f1835b76740 1 -- 172.21.15.67:0/2209476132 wait complete. 2024-07-31T14:55:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.608+0000 7f1835b76740 1 -- start start 2024-07-31T14:55:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.608+0000 7f1835b76740 1 Processor -- start 2024-07-31T14:55:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.608+0000 7f1835b76740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233491b0 0x557523723200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.608+0000 7f1835b76740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523351390 0x557523723740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.608+0000 7f1835b76740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523372070 0x55752372f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.608+0000 7f1835b76740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5575234e0700 con 0x557523372070 2024-07-31T14:55:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.608+0000 7f1835b76740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55752361f990 con 0x5575233491b0 2024-07-31T14:55:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.608+0000 7f1835b76740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557523725dc0 con 0x557523351390 2024-07-31T14:55:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.608+0000 7f1835b76740 10 client.0 _ll_register_callbacks cb 0x55752363a4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.608+0000 7f182dbc2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523372070 0x55752372f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.608+0000 7f182dbc2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523372070 0x55752372f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43180/0 (socket says 172.21.15.67:43180) 2024-07-31T14:55:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.608+0000 7f182dbc2640 1 -- 172.21.15.67:0/2723043492 learned_addr learned my addr 172.21.15.67:0/2723043492 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.608+0000 7f182d3c1640 1 --2- 172.21.15.67:0/2723043492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233491b0 0x557523723200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f182cbc0640 1 --2- 172.21.15.67:0/2723043492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523351390 0x557523723740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f182dbc2640 1 -- 172.21.15.67:0/2723043492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523351390 msgr2=0x557523723740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f182dbc2640 1 --2- 172.21.15.67:0/2723043492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523351390 0x557523723740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f182dbc2640 1 -- 172.21.15.67:0/2723043492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233491b0 msgr2=0x557523723200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f182dbc2640 1 --2- 172.21.15.67:0/2723043492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233491b0 0x557523723200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f182dbc2640 1 -- 172.21.15.67:0/2723043492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575233dd650 con 0x557523372070 2024-07-31T14:55:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f182cbc0640 1 --2- 172.21.15.67:0/2723043492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523351390 0x557523723740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:55:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f182dbc2640 1 --2- 172.21.15.67:0/2723043492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523372070 0x55752372f0e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f18200e1f90 tx=0x7f18200ab690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5473223f53d3179 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f1810ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:55:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f1810ff9640 1 -- 172.21.15.67:0/2723043492 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182001f040 con 0x557523372070 2024-07-31T14:55:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f1810ff9640 1 -- 172.21.15.67:0/2723043492 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f18200054f0 con 0x557523372070 2024-07-31T14:55:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f1835b76740 1 -- 172.21.15.67:0/2723043492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18240035f0 con 0x557523372070 2024-07-31T14:55:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f1810ff9640 1 -- 172.21.15.67:0/2723043492 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f18200b3f00 con 0x557523372070 2024-07-31T14:55:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f1835b76740 10 client.16713 fetch_fsmap learned FSMap version 39 2024-07-31T14:55:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f1835b76740 1 -- 172.21.15.67:0/2723043492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557523731e70 con 0x557523372070 2024-07-31T14:55:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.609+0000 7f1810ff9640 1 -- 172.21.15.67:0/2723043492 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18240035f0 con 0x557523372070 2024-07-31T14:55:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.610+0000 7f1810ff9640 1 -- 172.21.15.67:0/2723043492 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f182000f040 con 0x557523372070 2024-07-31T14:55:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.610+0000 7f1835b76740 10 client.16713 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:55:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.610+0000 7f1835b76740 10 client.16713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.610+0000 7f1835b76740 20 client.16713 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.610+0000 7f1835b76740 1 --2- 172.21.15.67:0/2723043492 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5575233309f0 0x557523735440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.610+0000 7f1835b76740 4 client.16713 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:55:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.610+0000 7f1835b76740 1 -- 172.21.15.67:0/2723043492 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55752368fab0 con 0x5575233309f0 2024-07-31T14:55:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.610+0000 7f182d3c1640 1 --2- 172.21.15.67:0/2723043492 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5575233309f0 0x557523735440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.610+0000 7f1810ff9640 10 client.16713 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.610+0000 7f182d3c1640 1 --2- 172.21.15.67:0/2723043492 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5575233309f0 0x557523735440 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72a6571342a5e3fc server_cookie=47b7323379a62d8c in_seq=0 out_seq=0 2024-07-31T14:55:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.611+0000 7f1810ff9640 1 -- 172.21.15.67:0/2723043492 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55752368fab0 con 0x5575233309f0 2024-07-31T14:55:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.611+0000 7f1810ff9640 10 client.16713 handle_command_reply: tid=0 2024-07-31T14:55:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.611+0000 7f1835b76740 1 client.16713 shutdown 2024-07-31T14:55:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.611+0000 7f1835b76740 1 -- 172.21.15.67:0/2723043492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523372070 msgr2=0x55752372f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.611+0000 7f1835b76740 1 --2- 172.21.15.67:0/2723043492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523372070 0x55752372f0e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f18200e1f90 tx=0x7f18200ab690 comp rx=0 tx=0).stop 2024-07-31T14:55:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.611+0000 7f1835b76740 1 -- 172.21.15.67:0/2723043492 shutdown_connections 2024-07-31T14:55:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.611+0000 7f1835b76740 1 --2- 172.21.15.67:0/2723043492 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5575233309f0 0x557523735440 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.611+0000 7f1810ff9640 0 client.16713 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.611+0000 7f1835b76740 1 --2- 172.21.15.67:0/2723043492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523372070 0x55752372f0e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.611+0000 7f1835b76740 1 --2- 172.21.15.67:0/2723043492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523351390 0x557523723740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.611+0000 7f1835b76740 1 --2- 172.21.15.67:0/2723043492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233491b0 0x557523723200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.611+0000 7f1835b76740 1 -- 172.21.15.67:0/2723043492 >> 172.21.15.67:0/2723043492 conn(0x557523366460 msgr2=0x557523713cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.612+0000 7f1835b76740 1 -- 172.21.15.67:0/2723043492 shutdown_connections 2024-07-31T14:55:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.612+0000 7f1835b76740 1 -- 172.21.15.67:0/2723043492 wait complete. 2024-07-31T14:55:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.612+0000 7f1835b76740 20 client.16713 trim_cache size 0 max 16384 2024-07-31T14:55:18.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.630+0000 7f1835b76740 1 Processor -- start 2024-07-31T14:55:18.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.630+0000 7f1835b76740 1 -- start start 2024-07-31T14:55:18.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.630+0000 7f1835b76740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233309f0 0x55752371ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.630+0000 7f1835b76740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575233491b0 0x55752372eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.630+0000 7f1835b76740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523351390 0x557523714e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.630+0000 7f1835b76740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f18281b2950 con 0x557523351390 2024-07-31T14:55:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.630+0000 7f1835b76740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f18281b4610 con 0x5575233309f0 2024-07-31T14:55:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.630+0000 7f1835b76740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f18281b3020 con 0x5575233491b0 2024-07-31T14:55:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.631+0000 7f182dbc2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523351390 0x557523714e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.631+0000 7f182dbc2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523351390 0x557523714e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43188/0 (socket says 172.21.15.67:43188) 2024-07-31T14:55:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.631+0000 7f182dbc2640 1 -- 172.21.15.67:0/3843283954 learned_addr learned my addr 172.21.15.67:0/3843283954 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.631+0000 7f182d3c1640 1 --2- 172.21.15.67:0/3843283954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233309f0 0x55752371ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.631+0000 7f182cbc0640 1 --2- 172.21.15.67:0/3843283954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575233491b0 0x55752372eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.631+0000 7f182dbc2640 1 -- 172.21.15.67:0/3843283954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575233491b0 msgr2=0x55752372eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.631+0000 7f182dbc2640 1 --2- 172.21.15.67:0/3843283954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575233491b0 0x55752372eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.631+0000 7f182dbc2640 1 -- 172.21.15.67:0/3843283954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233309f0 msgr2=0x55752371ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.631+0000 7f182dbc2640 1 --2- 172.21.15.67:0/3843283954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233309f0 0x55752371ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.631+0000 7f182dbc2640 1 -- 172.21.15.67:0/3843283954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557523719d30 con 0x557523351390 2024-07-31T14:55:18.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.631+0000 7f182dbc2640 1 --2- 172.21.15.67:0/3843283954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523351390 0x557523714e50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f18200e1a40 tx=0x7f18200aa860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac551cd25e5cca4f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:18.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.631+0000 7f1812ffd640 1 -- 172.21.15.67:0/3843283954 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182001f040 con 0x557523351390 2024-07-31T14:55:18.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.631+0000 7f1812ffd640 1 -- 172.21.15.67:0/3843283954 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18200ad030 con 0x557523351390 2024-07-31T14:55:18.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.631+0000 7f1812ffd640 1 -- 172.21.15.67:0/3843283954 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18200a9ec0 con 0x557523351390 2024-07-31T14:55:18.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.632+0000 7f1835b76740 1 -- 172.21.15.67:0/3843283954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523351390 msgr2=0x557523714e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:18.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.632+0000 7f1835b76740 1 --2- 172.21.15.67:0/3843283954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523351390 0x557523714e50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f18200e1a40 tx=0x7f18200aa860 comp rx=0 tx=0).stop 2024-07-31T14:55:18.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.632+0000 7f1835b76740 1 -- 172.21.15.67:0/3843283954 shutdown_connections 2024-07-31T14:55:18.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.632+0000 7f1835b76740 1 --2- 172.21.15.67:0/3843283954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557523351390 0x557523714e50 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.632+0000 7f1835b76740 1 --2- 172.21.15.67:0/3843283954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575233491b0 0x55752372eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.632+0000 7f1835b76740 1 --2- 172.21.15.67:0/3843283954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233309f0 0x55752371ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.632+0000 7f1835b76740 1 -- 172.21.15.67:0/3843283954 >> 172.21.15.67:0/3843283954 conn(0x557523366460 msgr2=0x55752371c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:18.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.632+0000 7f1835b76740 1 -- 172.21.15.67:0/3843283954 shutdown_connections 2024-07-31T14:55:18.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.632+0000 7f1835b76740 1 -- 172.21.15.67:0/3843283954 wait complete. 2024-07-31T14:55:18.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.633+0000 7f1835b76740 1 -- start start 2024-07-31T14:55:18.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.633+0000 7f1835b76740 1 Processor -- start 2024-07-31T14:55:18.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.633+0000 7f1835b76740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233309f0 0x557523727330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.633+0000 7f1835b76740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575233491b0 0x5575237278d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.633+0000 7f1835b76740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523351390 0x55752379cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.633+0000 7f1835b76740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f18281b3f40 con 0x5575233491b0 2024-07-31T14:55:18.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.633+0000 7f1835b76740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f18281b37b0 con 0x5575233309f0 2024-07-31T14:55:18.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.633+0000 7f1835b76740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f18281b0930 con 0x557523351390 2024-07-31T14:55:18.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.633+0000 7f1835b76740 10 client.0 _ll_register_callbacks cb 0x7f18200aa9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:18.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.633+0000 7f182cbc0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575233491b0 0x5575237278d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.633+0000 7f182d3c1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233309f0 0x557523727330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.633+0000 7f182cbc0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575233491b0 0x5575237278d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43204/0 (socket says 172.21.15.67:43204) 2024-07-31T14:55:18.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.633+0000 7f182cbc0640 1 -- 172.21.15.67:0/128707305 learned_addr learned my addr 172.21.15.67:0/128707305 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:18.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f182dbc2640 1 --2- 172.21.15.67:0/128707305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523351390 0x55752379cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f182cbc0640 1 -- 172.21.15.67:0/128707305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523351390 msgr2=0x55752379cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:18.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f182cbc0640 1 --2- 172.21.15.67:0/128707305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523351390 0x55752379cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f182cbc0640 1 -- 172.21.15.67:0/128707305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233309f0 msgr2=0x557523727330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:18.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f182cbc0640 1 --2- 172.21.15.67:0/128707305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233309f0 0x557523727330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f182cbc0640 1 -- 172.21.15.67:0/128707305 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55752379d110 con 0x5575233491b0 2024-07-31T14:55:18.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f182d3c1640 1 --2- 172.21.15.67:0/128707305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233309f0 0x557523727330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:18.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f182cbc0640 1 --2- 172.21.15.67:0/128707305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575233491b0 0x5575237278d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1824005f70 tx=0x7f1824005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3d2ef444b8c7df server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:18.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f1810ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:55:18.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f1810ff9640 1 -- 172.21.15.67:0/128707305 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182404d020 con 0x5575233491b0 2024-07-31T14:55:18.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f1810ff9640 1 -- 172.21.15.67:0/128707305 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f1824005550 con 0x5575233491b0 2024-07-31T14:55:18.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f1835b76740 1 -- 172.21.15.67:0/128707305 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18200ad030 con 0x5575233491b0 2024-07-31T14:55:18.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f1810ff9640 1 -- 172.21.15.67:0/128707305 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1824058050 con 0x5575233491b0 2024-07-31T14:55:18.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f1835b76740 10 client.16719 fetch_fsmap learned FSMap version 39 2024-07-31T14:55:18.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.634+0000 7f1835b76740 1 -- 172.21.15.67:0/128707305 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55752379e2a0 con 0x5575233491b0 2024-07-31T14:55:18.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.635+0000 7f1810ff9640 1 -- 172.21.15.67:0/128707305 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18200ad030 con 0x5575233491b0 2024-07-31T14:55:18.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.635+0000 7f1810ff9640 1 -- 172.21.15.67:0/128707305 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f182403a040 con 0x5575233491b0 2024-07-31T14:55:18.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.635+0000 7f1835b76740 10 client.16719 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:55:18.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.635+0000 7f1835b76740 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:18.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.635+0000 7f1835b76740 20 client.16719 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:18.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.635+0000 7f1835b76740 1 --2- 172.21.15.67:0/128707305 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557523372070 0x5575237a0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:18.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.635+0000 7f1835b76740 4 client.16719 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:55:18.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.635+0000 7f1835b76740 1 -- 172.21.15.67:0/128707305 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55752368b550 con 0x557523372070 2024-07-31T14:55:18.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.635+0000 7f182d3c1640 1 --2- 172.21.15.67:0/128707305 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557523372070 0x5575237a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:18.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.636+0000 7f182d3c1640 1 --2- 172.21.15.67:0/128707305 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557523372070 0x5575237a0600 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e749695c6c26899e server_cookie=8a744fbf0a7023c6 in_seq=0 out_seq=0 2024-07-31T14:55:18.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.636+0000 7f1810ff9640 10 client.16719 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:18.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.636+0000 7f1810ff9640 1 -- 172.21.15.67:0/128707305 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55752368b550 con 0x557523372070 2024-07-31T14:55:18.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.636+0000 7f1810ff9640 10 client.16719 handle_command_reply: tid=0 2024-07-31T14:55:18.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.636+0000 7f1835b76740 1 client.16719 shutdown 2024-07-31T14:55:18.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.636+0000 7f1835b76740 1 -- 172.21.15.67:0/128707305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575233491b0 msgr2=0x5575237278d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:18.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.636+0000 7f1835b76740 1 --2- 172.21.15.67:0/128707305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575233491b0 0x5575237278d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1824005f70 tx=0x7f1824005a20 comp rx=0 tx=0).stop 2024-07-31T14:55:18.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.636+0000 7f1835b76740 1 -- 172.21.15.67:0/128707305 shutdown_connections 2024-07-31T14:55:18.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.636+0000 7f1835b76740 1 --2- 172.21.15.67:0/128707305 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557523372070 0x5575237a0600 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f1835b76740 1 --2- 172.21.15.67:0/128707305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557523351390 0x55752379cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f1835b76740 1 --2- 172.21.15.67:0/128707305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575233491b0 0x5575237278d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f1810ff9640 0 client.16719 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:18.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f1835b76740 1 --2- 172.21.15.67:0/128707305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575233309f0 0x557523727330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f1835b76740 1 -- 172.21.15.67:0/128707305 >> 172.21.15.67:0/128707305 conn(0x557523366460 msgr2=0x557523714e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:18.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f1835b76740 1 -- 172.21.15.67:0/128707305 shutdown_connections 2024-07-31T14:55:18.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f1835b76740 1 -- 172.21.15.67:0/128707305 wait complete. 2024-07-31T14:55:18.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f1835b76740 20 client.16719 trim_cache size 0 max 16384 2024-07-31T14:55:18.643 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:55:18.644 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T14:55:18.644 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T14:55:18.644 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:55:18.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f182f625640 1 -- 172.21.15.67:0/726353835 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18140b3d40 msgr2=0x7f18140b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:18.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f182f625640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18140b3d40 0x7f18140b6250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5575237162c0 tx=0x7f181800b3f0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f182f625640 1 -- 172.21.15.67:0/726353835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f182810fbc0 msgr2=0x7f18281a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:18.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f182f625640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f182810fbc0 0x7f18281a1920 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f182000eaa0 tx=0x7f182000ef60 comp rx=0 tx=0).stop 2024-07-31T14:55:18.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f182f625640 1 -- 172.21.15.67:0/726353835 shutdown_connections 2024-07-31T14:55:18.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f182f625640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18140b3d40 0x7f18140b6250 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f182f625640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f182810fbc0 0x7f18281a1920 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f182f625640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1828102960 0x7f182819c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f182f625640 1 --2- 172.21.15.67:0/726353835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1828102010 0x7f182819c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:18.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f182f625640 1 -- 172.21.15.67:0/726353835 >> 172.21.15.67:0/726353835 conn(0x7f1828079140 msgr2=0x7f1828108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:18.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.637+0000 7f182f625640 1 -- 172.21.15.67:0/726353835 shutdown_connections 2024-07-31T14:55:18.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:18.638+0000 7f182f625640 1 -- 172.21.15.67:0/726353835 wait complete. 2024-07-31T14:55:19.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:18 smithi073 ceph-mon[36714]: pgmap v527: 129 pgs: 129 active+clean; 484 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.6 MiB/s wr, 866 op/s 2024-07-31T14:55:19.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:18 smithi077 ceph-mon[37327]: pgmap v527: 129 pgs: 129 active+clean; 484 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.6 MiB/s wr, 866 op/s 2024-07-31T14:55:19.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:18 smithi067 ceph-mon[32750]: pgmap v527: 129 pgs: 129 active+clean; 484 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.6 MiB/s wr, 866 op/s 2024-07-31T14:55:19.463 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T14:55:19.464 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:55:19.737 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:55:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.352+0000 7f997aa46640 1 -- 172.21.15.67:0/2649549460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9974107d20 msgr2=0x7f9974108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.352+0000 7f997aa46640 1 --2- 172.21.15.67:0/2649549460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9974107d20 0x7f9974108130 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f995c009900 tx=0x7f995c02a410 comp rx=0 tx=0).stop 2024-07-31T14:55:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.353+0000 7f997aa46640 1 -- 172.21.15.67:0/2649549460 shutdown_connections 2024-07-31T14:55:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.353+0000 7f997aa46640 1 --2- 172.21.15.67:0/2649549460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974108670 0x7f997410eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.353+0000 7f997aa46640 1 --2- 172.21.15.67:0/2649549460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9974107d20 0x7f9974108130 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.353+0000 7f997aa46640 1 --2- 172.21.15.67:0/2649549460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99741023f0 0x7f9974102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.353+0000 7f997aa46640 1 -- 172.21.15.67:0/2649549460 >> 172.21.15.67:0/2649549460 conn(0x7f99740fddf0 msgr2=0x7f9974100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.353+0000 7f997aa46640 1 -- 172.21.15.67:0/2649549460 shutdown_connections 2024-07-31T14:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.353+0000 7f997aa46640 1 -- 172.21.15.67:0/2649549460 wait complete. 2024-07-31T14:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.353+0000 7f997aa46640 1 Processor -- start 2024-07-31T14:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.353+0000 7f997aa46640 1 -- start start 2024-07-31T14:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f997aa46640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99741023f0 0x7f997419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f997aa46640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974107d20 0x7f997419b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f997aa46640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9974108670 0x7f997419b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f997aa46640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9974112300 con 0x7f99741023f0 2024-07-31T14:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f997aa46640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9974112180 con 0x7f9974108670 2024-07-31T14:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f997aa46640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9974112480 con 0x7f9974107d20 2024-07-31T14:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f9973fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99741023f0 0x7f997419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f9973fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99741023f0 0x7f997419ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43226/0 (socket says 172.21.15.67:43226) 2024-07-31T14:55:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f9973fff640 1 -- 172.21.15.67:0/2278257833 learned_addr learned my addr 172.21.15.67:0/2278257833 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f9978fe3640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9974108670 0x7f997419b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f996bfff640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974107d20 0x7f997419b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f9973fff640 1 -- 172.21.15.67:0/2278257833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974107d20 msgr2=0x7f997419b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f9973fff640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974107d20 0x7f997419b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f9973fff640 1 -- 172.21.15.67:0/2278257833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9974108670 msgr2=0x7f997419b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f9973fff640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9974108670 0x7f997419b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f9973fff640 1 -- 172.21.15.67:0/2278257833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99741a5f40 con 0x7f99741023f0 2024-07-31T14:55:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f9978fe3640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9974108670 0x7f997419b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f9973fff640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99741023f0 0x7f997419ac30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f996400ea60 tx=0x7f996400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.354+0000 7f9971ffb640 1 -- 172.21.15.67:0/2278257833 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996400cde0 con 0x7f99741023f0 2024-07-31T14:55:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.355+0000 7f9971ffb640 1 -- 172.21.15.67:0/2278257833 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99640035f0 con 0x7f99741023f0 2024-07-31T14:55:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.355+0000 7f997aa46640 1 -- 172.21.15.67:0/2278257833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99741a6230 con 0x7f99741023f0 2024-07-31T14:55:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.355+0000 7f9971ffb640 1 -- 172.21.15.67:0/2278257833 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964010690 con 0x7f99741023f0 2024-07-31T14:55:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.355+0000 7f997aa46640 1 -- 172.21.15.67:0/2278257833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99741a6750 con 0x7f99741023f0 2024-07-31T14:55:20.365 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.357+0000 7f9971ffb640 1 -- 172.21.15.67:0/2278257833 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f99640037a0 con 0x7f99741023f0 2024-07-31T14:55:20.365 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.357+0000 7f9981a15740 1 Processor -- start 2024-07-31T14:55:20.365 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9981a15740 1 -- start start 2024-07-31T14:55:20.365 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9981a15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ed11b0 0x561678273ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9981a15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677ea82f0 0x561678274220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9981a15740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677eb04d0 0x561678274760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9981a15740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5616781e93b0 con 0x561677eb04d0 2024-07-31T14:55:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9981a15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5616781d88b0 con 0x561677ea82f0 2024-07-31T14:55:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9981a15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5616781dc8f0 con 0x561677ed11b0 2024-07-31T14:55:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f996bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677eb04d0 0x561678274760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9973fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677ea82f0 0x561678274220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9978fe3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ed11b0 0x561678273ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9978fe3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ed11b0 0x561678273ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37580/0 (socket says 172.21.15.67:37580) 2024-07-31T14:55:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f996bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677eb04d0 0x561678274760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43236/0 (socket says 172.21.15.67:43236) 2024-07-31T14:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9978fe3640 1 -- 172.21.15.67:0/4213513941 learned_addr learned my addr 172.21.15.67:0/4213513941 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9971ffb640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99600b3d40 0x7f99600b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9973fff640 1 -- 172.21.15.67:0/4213513941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ed11b0 msgr2=0x561678273ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9973fff640 1 --2- 172.21.15.67:0/4213513941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ed11b0 0x561678273ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9973fff640 1 -- 172.21.15.67:0/4213513941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677eb04d0 msgr2=0x561678274760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9973fff640 1 --2- 172.21.15.67:0/4213513941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677eb04d0 0x561678274760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9973fff640 1 -- 172.21.15.67:0/4213513941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56167817f9a0 con 0x561677ea82f0 2024-07-31T14:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.358+0000 7f9978fe3640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99600b3d40 0x7f99600b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.359+0000 7f9971ffb640 1 -- 172.21.15.67:0/2278257833 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f99640e4690 con 0x7f99741023f0 2024-07-31T14:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.359+0000 7f9973fff640 1 --2- 172.21.15.67:0/4213513941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677ea82f0 0x561678274220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f99640e1ad0 tx=0x7f996406fdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a3a4fb32ba5aa02 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.359+0000 7f996a7fc640 1 -- 172.21.15.67:0/4213513941 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99640aad60 con 0x561677ea82f0 2024-07-31T14:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.359+0000 7f9978fe3640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99600b3d40 0x7f99600b6250 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f996c009fd0 tx=0x7f996c004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.359+0000 7f996a7fc640 1 -- 172.21.15.67:0/4213513941 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99640aaaa0 con 0x561677ea82f0 2024-07-31T14:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.359+0000 7f996a7fc640 1 -- 172.21.15.67:0/4213513941 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99640aa700 con 0x561677ea82f0 2024-07-31T14:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.360+0000 7f9981a15740 1 -- 172.21.15.67:0/4213513941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677ea82f0 msgr2=0x561678274220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.360+0000 7f9981a15740 1 --2- 172.21.15.67:0/4213513941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677ea82f0 0x561678274220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f99640e1ad0 tx=0x7f996406fdb0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.360+0000 7f9981a15740 1 -- 172.21.15.67:0/4213513941 shutdown_connections 2024-07-31T14:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.360+0000 7f9981a15740 1 --2- 172.21.15.67:0/4213513941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677eb04d0 0x561678274760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.360+0000 7f9981a15740 1 --2- 172.21.15.67:0/4213513941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677ea82f0 0x561678274220 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.360+0000 7f9981a15740 1 --2- 172.21.15.67:0/4213513941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ed11b0 0x561678273ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.360+0000 7f9981a15740 1 -- 172.21.15.67:0/4213513941 >> 172.21.15.67:0/4213513941 conn(0x561677ec5300 msgr2=0x561678271bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.360+0000 7f9981a15740 1 -- 172.21.15.67:0/4213513941 shutdown_connections 2024-07-31T14:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.360+0000 7f9981a15740 1 -- 172.21.15.67:0/4213513941 wait complete. 2024-07-31T14:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.360+0000 7f9981a15740 1 -- start start 2024-07-31T14:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9981a15740 1 Processor -- start 2024-07-31T14:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9981a15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 0x561678283980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9981a15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677eb04d0 0x561678283f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9981a15740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677ed11b0 0x56167828fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9981a15740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561678112ab0 con 0x561677ed11b0 2024-07-31T14:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9981a15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561678160cf0 con 0x561677eb04d0 2024-07-31T14:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9981a15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5616782867e0 con 0x561677ea82f0 2024-07-31T14:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9981a15740 10 client.0 _ll_register_callbacks cb 0x5616781d29f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9978fe3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677ed11b0 0x56167828fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f996bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677eb04d0 0x561678283f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9978fe3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677ed11b0 0x56167828fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43238/0 (socket says 172.21.15.67:43238) 2024-07-31T14:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9978fe3640 1 -- 172.21.15.67:0/3068619475 learned_addr learned my addr 172.21.15.67:0/3068619475 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9973fff640 1 --2- 172.21.15.67:0/3068619475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 0x561678283980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9978fe3640 1 -- 172.21.15.67:0/3068619475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 msgr2=0x561678283980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9978fe3640 1 --2- 172.21.15.67:0/3068619475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 0x561678283980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9978fe3640 1 -- 172.21.15.67:0/3068619475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677eb04d0 msgr2=0x561678283f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9978fe3640 1 --2- 172.21.15.67:0/3068619475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677eb04d0 0x561678283f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.361+0000 7f9978fe3640 1 -- 172.21.15.67:0/3068619475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561677f3c790 con 0x561677ed11b0 2024-07-31T14:55:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.362+0000 7f9978fe3640 1 --2- 172.21.15.67:0/3068619475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677ed11b0 0x56167828fb70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f996c011dc0 tx=0x7f996c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395f1df141d1fa75 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.362+0000 7f99537fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:55:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.362+0000 7f99537fe640 1 -- 172.21.15.67:0/3068619475 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c02d060 con 0x561677ed11b0 2024-07-31T14:55:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.362+0000 7f99537fe640 1 -- 172.21.15.67:0/3068619475 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f996c028600 con 0x561677ed11b0 2024-07-31T14:55:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.362+0000 7f9981a15740 1 -- 172.21.15.67:0/3068619475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99640aaaa0 con 0x561677ed11b0 2024-07-31T14:55:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.362+0000 7f99537fe640 1 -- 172.21.15.67:0/3068619475 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f996c034050 con 0x561677ed11b0 2024-07-31T14:55:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.362+0000 7f9981a15740 10 client.16731 fetch_fsmap learned FSMap version 39 2024-07-31T14:55:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.362+0000 7f9981a15740 1 -- 172.21.15.67:0/3068619475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561677fb5620 con 0x561677ed11b0 2024-07-31T14:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.362+0000 7f99537fe640 1 -- 172.21.15.67:0/3068619475 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99640aaaa0 con 0x561677ed11b0 2024-07-31T14:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.362+0000 7f99537fe640 1 -- 172.21.15.67:0/3068619475 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f996c00f990 con 0x561677ed11b0 2024-07-31T14:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.363+0000 7f9981a15740 10 client.16731 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.363+0000 7f9981a15740 10 client.16731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.363+0000 7f9981a15740 20 client.16731 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.363+0000 7f9981a15740 1 --2- 172.21.15.67:0/3068619475 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561677e8fd80 0x561678295c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.363+0000 7f9981a15740 4 client.16731 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.363+0000 7f9981a15740 1 -- 172.21.15.67:0/3068619475 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616781eb300 con 0x561677e8fd80 2024-07-31T14:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.363+0000 7f9973fff640 1 --2- 172.21.15.67:0/3068619475 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561677e8fd80 0x561678295c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.363+0000 7f99537fe640 10 client.16731 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.363+0000 7f9973fff640 1 --2- 172.21.15.67:0/3068619475 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561677e8fd80 0x561678295c30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86768f41899642e7 server_cookie=5ed5b2ef4457af9b in_seq=0 out_seq=0 2024-07-31T14:55:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.364+0000 7f99537fe640 1 -- 172.21.15.67:0/3068619475 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616781eb300 con 0x561677e8fd80 2024-07-31T14:55:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.364+0000 7f99537fe640 10 client.16731 handle_command_reply: tid=0 2024-07-31T14:55:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.365+0000 7f9981a15740 1 client.16731 shutdown 2024-07-31T14:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.365+0000 7f9981a15740 1 -- 172.21.15.67:0/3068619475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677ed11b0 msgr2=0x56167828fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.365+0000 7f9981a15740 1 --2- 172.21.15.67:0/3068619475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677ed11b0 0x56167828fb70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f996c011dc0 tx=0x7f996c004ab0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.365+0000 7f9981a15740 1 -- 172.21.15.67:0/3068619475 shutdown_connections 2024-07-31T14:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.365+0000 7f9981a15740 1 --2- 172.21.15.67:0/3068619475 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561677e8fd80 0x561678295c30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.365+0000 7f9981a15740 1 --2- 172.21.15.67:0/3068619475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677ed11b0 0x56167828fb70 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.365+0000 7f9981a15740 1 --2- 172.21.15.67:0/3068619475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677eb04d0 0x561678283f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.365+0000 7f9981a15740 1 --2- 172.21.15.67:0/3068619475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 0x561678283980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.365+0000 7f99537fe640 0 client.16731 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.365+0000 7f9981a15740 1 -- 172.21.15.67:0/3068619475 >> 172.21.15.67:0/3068619475 conn(0x561677ec5300 msgr2=0x5616782756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.365+0000 7f9981a15740 1 -- 172.21.15.67:0/3068619475 shutdown_connections 2024-07-31T14:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.365+0000 7f9981a15740 1 -- 172.21.15.67:0/3068619475 wait complete. 2024-07-31T14:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.365+0000 7f9981a15740 20 client.16731 trim_cache size 0 max 16384 2024-07-31T14:55:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9981a15740 1 Processor -- start 2024-07-31T14:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9981a15740 1 -- start start 2024-07-31T14:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9981a15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 0x56167828f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9981a15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677eb04d0 0x56167828fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9981a15740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677ed11b0 0x56167829b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9981a15740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f99741ae4a0 con 0x561677ed11b0 2024-07-31T14:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9981a15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f99741b0160 con 0x561677eb04d0 2024-07-31T14:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9981a15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f99741aeb70 con 0x561677ea82f0 2024-07-31T14:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9978fe3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677ed11b0 0x56167829b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f996bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677eb04d0 0x56167828fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9978fe3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677ed11b0 0x56167829b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43252/0 (socket says 172.21.15.67:43252) 2024-07-31T14:55:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9978fe3640 1 -- 172.21.15.67:0/1695803777 learned_addr learned my addr 172.21.15.67:0/1695803777 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9973fff640 1 --2- 172.21.15.67:0/1695803777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 0x56167828f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9978fe3640 1 -- 172.21.15.67:0/1695803777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 msgr2=0x56167828f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9978fe3640 1 --2- 172.21.15.67:0/1695803777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 0x56167828f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9978fe3640 1 -- 172.21.15.67:0/1695803777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677eb04d0 msgr2=0x56167828fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9978fe3640 1 --2- 172.21.15.67:0/1695803777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677eb04d0 0x56167828fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9978fe3640 1 -- 172.21.15.67:0/1695803777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56167827f160 con 0x561677ed11b0 2024-07-31T14:55:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.382+0000 7f9973fff640 1 --2- 172.21.15.67:0/1695803777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 0x56167828f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:55:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.383+0000 7f9978fe3640 1 --2- 172.21.15.67:0/1695803777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677ed11b0 0x56167829b430 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f996c005cc0 tx=0x7f996c003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980ec781845f5773 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.383+0000 7f9969ffb640 1 -- 172.21.15.67:0/1695803777 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c03d070 con 0x561677ed11b0 2024-07-31T14:55:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.383+0000 7f9969ffb640 1 -- 172.21.15.67:0/1695803777 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f996c00fa70 con 0x561677ed11b0 2024-07-31T14:55:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.383+0000 7f9969ffb640 1 -- 172.21.15.67:0/1695803777 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c01beb0 con 0x561677ed11b0 2024-07-31T14:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.384+0000 7f9981a15740 1 -- 172.21.15.67:0/1695803777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677ed11b0 msgr2=0x56167829b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.384+0000 7f9981a15740 1 --2- 172.21.15.67:0/1695803777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677ed11b0 0x56167829b430 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f996c005cc0 tx=0x7f996c003fd0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.384+0000 7f9981a15740 1 -- 172.21.15.67:0/1695803777 shutdown_connections 2024-07-31T14:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.384+0000 7f9981a15740 1 --2- 172.21.15.67:0/1695803777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677ed11b0 0x56167829b430 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.384+0000 7f9981a15740 1 --2- 172.21.15.67:0/1695803777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677eb04d0 0x56167828fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.384+0000 7f9981a15740 1 --2- 172.21.15.67:0/1695803777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 0x56167828f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.384+0000 7f9981a15740 1 -- 172.21.15.67:0/1695803777 >> 172.21.15.67:0/1695803777 conn(0x561677ec5300 msgr2=0x5616782821e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.384+0000 7f9981a15740 1 -- 172.21.15.67:0/1695803777 shutdown_connections 2024-07-31T14:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.384+0000 7f9981a15740 1 -- 172.21.15.67:0/1695803777 wait complete. 2024-07-31T14:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.384+0000 7f9981a15740 1 -- start start 2024-07-31T14:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f9981a15740 1 Processor -- start 2024-07-31T14:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f9981a15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 0x5616782892c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f9981a15740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677eb04d0 0x561678289840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f9981a15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677ed11b0 0x5616782fc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f9981a15740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f99741afa90 con 0x561677eb04d0 2024-07-31T14:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f9981a15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f99741af300 con 0x561677ed11b0 2024-07-31T14:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f9981a15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f99741ac480 con 0x561677ea82f0 2024-07-31T14:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f9981a15740 10 client.0 _ll_register_callbacks cb 0x7f996c003c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f996bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677eb04d0 0x561678289840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f996bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677eb04d0 0x561678289840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43268/0 (socket says 172.21.15.67:43268) 2024-07-31T14:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f996bfff640 1 -- 172.21.15.67:0/2121672483 learned_addr learned my addr 172.21.15.67:0/2121672483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f9978fe3640 1 --2- 172.21.15.67:0/2121672483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677ed11b0 0x5616782fc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f9973fff640 1 --2- 172.21.15.67:0/2121672483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 0x5616782892c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f996bfff640 1 -- 172.21.15.67:0/2121672483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 msgr2=0x5616782892c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f996bfff640 1 --2- 172.21.15.67:0/2121672483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 0x5616782892c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f996bfff640 1 -- 172.21.15.67:0/2121672483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677ed11b0 msgr2=0x5616782fc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f996bfff640 1 --2- 172.21.15.67:0/2121672483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677ed11b0 0x5616782fc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f996bfff640 1 -- 172.21.15.67:0/2121672483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616782fca40 con 0x561677eb04d0 2024-07-31T14:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.385+0000 7f9978fe3640 1 --2- 172.21.15.67:0/2121672483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677ed11b0 0x5616782fc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.386+0000 7f996bfff640 1 --2- 172.21.15.67:0/2121672483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677eb04d0 0x561678289840 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f995c031440 tx=0x7f995c031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b960e1b02c1ad0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.386+0000 7f99537fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.386+0000 7f99537fe640 1 -- 172.21.15.67:0/2121672483 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c035040 con 0x561677eb04d0 2024-07-31T14:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.386+0000 7f99537fe640 1 -- 172.21.15.67:0/2121672483 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f995c004310 con 0x561677eb04d0 2024-07-31T14:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.386+0000 7f9981a15740 1 -- 172.21.15.67:0/2121672483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f996c00fa70 con 0x561677eb04d0 2024-07-31T14:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.386+0000 7f99537fe640 1 -- 172.21.15.67:0/2121672483 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f995c005030 con 0x561677eb04d0 2024-07-31T14:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.386+0000 7f9981a15740 10 client.16737 fetch_fsmap learned FSMap version 39 2024-07-31T14:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.386+0000 7f9981a15740 1 -- 172.21.15.67:0/2121672483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5616782fdce0 con 0x561677eb04d0 2024-07-31T14:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.386+0000 7f99537fe640 1 -- 172.21.15.67:0/2121672483 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f996c00fa70 con 0x561677eb04d0 2024-07-31T14:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.386+0000 7f99537fe640 1 -- 172.21.15.67:0/2121672483 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f995c004ae0 con 0x561677eb04d0 2024-07-31T14:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.386+0000 7f9981a15740 10 client.16737 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.386+0000 7f9981a15740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.386+0000 7f9981a15740 20 client.16737 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.386+0000 7f9981a15740 1 --2- 172.21.15.67:0/2121672483 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561677e651c0 0x5616783005e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.387+0000 7f9981a15740 4 client.16737 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:55:20.394 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.387+0000 7f9981a15740 1 -- 172.21.15.67:0/2121672483 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5616781e6da0 con 0x561677e651c0 2024-07-31T14:55:20.394 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.387+0000 7f9973fff640 1 --2- 172.21.15.67:0/2121672483 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561677e651c0 0x5616783005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:20.394 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.387+0000 7f9973fff640 1 --2- 172.21.15.67:0/2121672483 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561677e651c0 0x5616783005e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52aeac02c8849d47 server_cookie=f9a9cc798ebb19d9 in_seq=0 out_seq=0 2024-07-31T14:55:20.394 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:20.387+0000 7f99537fe640 10 client.16737 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:21.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:20 smithi073 ceph-mon[36714]: pgmap v528: 129 pgs: 129 active+clean; 484 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 528 op/s 2024-07-31T14:55:21.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:20 smithi067 ceph-mon[32750]: pgmap v528: 129 pgs: 129 active+clean; 484 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 528 op/s 2024-07-31T14:55:21.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:20 smithi077 ceph-mon[37327]: pgmap v528: 129 pgs: 129 active+clean; 484 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 528 op/s 2024-07-31T14:55:21.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.623+0000 7efe8bfff640 1 -- 172.21.15.67:0/2907579347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8c10a080 msgr2=0x7efe8c10a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:21.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.623+0000 7efe8bfff640 1 --2- 172.21.15.67:0/2907579347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8c10a080 0x7efe8c10a490 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0074e0 tx=0x7efe7c02b070 comp rx=0 tx=0).stop 2024-07-31T14:55:21.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.624+0000 7efe8bfff640 1 -- 172.21.15.67:0/2907579347 shutdown_connections 2024-07-31T14:55:21.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.624+0000 7efe8bfff640 1 --2- 172.21.15.67:0/2907579347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe8c110090 0x7efe8c1168c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:21.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.624+0000 7efe8bfff640 1 --2- 172.21.15.67:0/2907579347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe8c10f740 0x7efe8c10fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:21.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.624+0000 7efe8bfff640 1 --2- 172.21.15.67:0/2907579347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8c10a080 0x7efe8c10a490 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:21.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.624+0000 7efe8bfff640 1 -- 172.21.15.67:0/2907579347 >> 172.21.15.67:0/2907579347 conn(0x7efe8c06e420 msgr2=0x7efe8c06e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:21.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.624+0000 7efe8bfff640 1 -- 172.21.15.67:0/2907579347 shutdown_connections 2024-07-31T14:55:21.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.624+0000 7efe8bfff640 1 -- 172.21.15.67:0/2907579347 wait complete. 2024-07-31T14:55:21.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.626+0000 7efe8bfff640 1 Processor -- start 2024-07-31T14:55:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.627+0000 7efe8bfff640 1 -- start start 2024-07-31T14:55:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.628+0000 7efe8bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8c10a080 0x7efe8c1a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.628+0000 7efe8bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe8c10f740 0x7efe8c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.628+0000 7efe8bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe8c110090 0x7efe8c1a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.628+0000 7efe8bfff640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c119950 con 0x7efe8c110090 2024-07-31T14:55:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.628+0000 7efe8bfff640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c1197d0 con 0x7efe8c10a080 2024-07-31T14:55:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.628+0000 7efe8bfff640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c119ad0 con 0x7efe8c10f740 2024-07-31T14:55:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.629+0000 7efe8a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe8c10f740 0x7efe8c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.629+0000 7efe8a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe8c10f740 0x7efe8c1a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44270/0 (socket says 172.21.15.67:44270) 2024-07-31T14:55:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.629+0000 7efe8a7fc640 1 -- 172.21.15.67:0/1688670927 learned_addr learned my addr 172.21.15.67:0/1688670927 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:21.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.630+0000 7efe8affd640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8c10a080 0x7efe8c1a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:21.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.630+0000 7efe8a7fc640 1 -- 172.21.15.67:0/1688670927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8c10a080 msgr2=0x7efe8c1a02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:21.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.630+0000 7efe8a7fc640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8c10a080 0x7efe8c1a02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:21.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.630+0000 7efe8a7fc640 1 -- 172.21.15.67:0/1688670927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe8c110090 msgr2=0x7efe8c1a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:21.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.629+0000 7efe8b7fe640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe8c110090 0x7efe8c1a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:21.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.633+0000 7efe8a7fc640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe8c110090 0x7efe8c1a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:21.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.633+0000 7efe8a7fc640 1 -- 172.21.15.67:0/1688670927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8c1a6190 con 0x7efe8c10f740 2024-07-31T14:55:21.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.634+0000 7efe8a7fc640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe8c10f740 0x7efe8c1a0880 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe8000c8c0 tx=0x7efe8000cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:21.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.634+0000 7efe8b7fe640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe8c110090 0x7efe8c1a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:55:21.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.636+0000 7efe73fff640 1 -- 172.21.15.67:0/1688670927 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80022070 con 0x7efe8c10f740 2024-07-31T14:55:21.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.636+0000 7efe73fff640 1 -- 172.21.15.67:0/1688670927 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe80007c50 con 0x7efe8c10f740 2024-07-31T14:55:21.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.636+0000 7efe73fff640 1 -- 172.21.15.67:0/1688670927 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8001dd60 con 0x7efe8c10f740 2024-07-31T14:55:21.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.636+0000 7efe8bfff640 1 -- 172.21.15.67:0/1688670927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8c1a6480 con 0x7efe8c10f740 2024-07-31T14:55:21.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.636+0000 7efe8bfff640 1 -- 172.21.15.67:0/1688670927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8c1acca0 con 0x7efe8c10f740 2024-07-31T14:55:21.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.640+0000 7efe8bfff640 1 -- 172.21.15.67:0/1688670927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8c10a490 con 0x7efe8c10f740 2024-07-31T14:55:21.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.651+0000 7efe73fff640 1 -- 172.21.15.67:0/1688670927 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7efe8001d440 con 0x7efe8c10f740 2024-07-31T14:55:21.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.653+0000 7efe73fff640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efe740b3f40 0x7efe740b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:21.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.653+0000 7efe73fff640 1 -- 172.21.15.67:0/1688670927 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7efe800ec430 con 0x7efe8c10f740 2024-07-31T14:55:21.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.656+0000 7efe8affd640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efe740b3f40 0x7efe740b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:21.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.656+0000 7efe73fff640 1 -- 172.21.15.67:0/1688670927 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7efe800b3560 con 0x7efe8c10f740 2024-07-31T14:55:21.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:21.657+0000 7efe8affd640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efe740b3f40 0x7efe740b6450 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efe7c02b660 tx=0x7efe7c02d040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:22.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:22.155+0000 7efe8bfff640 1 -- 172.21.15.67:0/1688670927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe8c1a1410 con 0x7efe8c10f740 2024-07-31T14:55:22.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:22.161+0000 7efe73fff640 1 -- 172.21.15.67:0/1688670927 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11980 (secure 0 0 0) 0x7efe8c1a1410 con 0x7efe8c10f740 2024-07-31T14:55:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:55:22.163 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":39,"btime":"2024-07-31T14:54:12:039205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:54:11.524751+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T14:55:22.163 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 39 2024-07-31T14:55:22.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:22.166+0000 7efe8bfff640 1 -- 172.21.15.67:0/1688670927 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efe740b3f40 msgr2=0x7efe740b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:22.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:22.166+0000 7efe8bfff640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efe740b3f40 0x7efe740b6450 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efe7c02b660 tx=0x7efe7c02d040 comp rx=0 tx=0).stop 2024-07-31T14:55:22.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:22.167+0000 7efe8bfff640 1 -- 172.21.15.67:0/1688670927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe8c10f740 msgr2=0x7efe8c1a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:22.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:22.167+0000 7efe8bfff640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe8c10f740 0x7efe8c1a0880 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe8000c8c0 tx=0x7efe8000cd80 comp rx=0 tx=0).stop 2024-07-31T14:55:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:22.168+0000 7efe8bfff640 1 -- 172.21.15.67:0/1688670927 shutdown_connections 2024-07-31T14:55:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:22.168+0000 7efe8bfff640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efe740b3f40 0x7efe740b6450 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:22.168+0000 7efe8bfff640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe8c110090 0x7efe8c1a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:22.169+0000 7efe8bfff640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe8c10f740 0x7efe8c1a0880 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:22.169+0000 7efe8bfff640 1 --2- 172.21.15.67:0/1688670927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe8c10a080 0x7efe8c1a02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:22.169+0000 7efe8bfff640 1 -- 172.21.15.67:0/1688670927 >> 172.21.15.67:0/1688670927 conn(0x7efe8c06e420 msgr2=0x7efe8c10d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:22.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:22.172+0000 7efe8bfff640 1 -- 172.21.15.67:0/1688670927 shutdown_connections 2024-07-31T14:55:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:22.174+0000 7efe8bfff640 1 -- 172.21.15.67:0/1688670927 wait complete. 2024-07-31T14:55:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.677+0000 7f99537fe640 1 -- 172.21.15.67:0/2121672483 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5616781e6da0 con 0x561677e651c0 2024-07-31T14:55:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.677+0000 7f99537fe640 10 client.16737 handle_command_reply: tid=0 2024-07-31T14:55:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.677+0000 7f9981a15740 1 client.16737 shutdown 2024-07-31T14:55:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.677+0000 7f9981a15740 1 -- 172.21.15.67:0/2121672483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677eb04d0 msgr2=0x561678289840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.677+0000 7f9981a15740 1 --2- 172.21.15.67:0/2121672483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677eb04d0 0x561678289840 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f995c031440 tx=0x7f995c031950 comp rx=0 tx=0).stop 2024-07-31T14:55:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.677+0000 7f9981a15740 1 -- 172.21.15.67:0/2121672483 shutdown_connections 2024-07-31T14:55:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.677+0000 7f9981a15740 1 --2- 172.21.15.67:0/2121672483 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561677e651c0 0x5616783005e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.677+0000 7f9981a15740 1 --2- 172.21.15.67:0/2121672483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561677ed11b0 0x5616782fc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.677+0000 7f9981a15740 1 --2- 172.21.15.67:0/2121672483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561677eb04d0 0x561678289840 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.677+0000 7f9981a15740 1 --2- 172.21.15.67:0/2121672483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561677ea82f0 0x5616782892c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.677+0000 7f9981a15740 1 -- 172.21.15.67:0/2121672483 >> 172.21.15.67:0/2121672483 conn(0x561677ec5300 msgr2=0x561678284e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.677+0000 7f99537fe640 0 client.16737 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.677+0000 7f9981a15740 1 -- 172.21.15.67:0/2121672483 shutdown_connections 2024-07-31T14:55:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.677+0000 7f9981a15740 1 -- 172.21.15.67:0/2121672483 wait complete. 2024-07-31T14:55:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.678+0000 7f9981a15740 20 client.16737 trim_cache size 0 max 16384 2024-07-31T14:55:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:55:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:55:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:55:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:55:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.678+0000 7f997aa46640 1 -- 172.21.15.67:0/2278257833 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99600b3d40 msgr2=0x7f99600b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.678+0000 7f997aa46640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99600b3d40 0x7f99600b6250 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f996c009fd0 tx=0x7f996c004a40 comp rx=0 tx=0).stop 2024-07-31T14:55:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.678+0000 7f997aa46640 1 -- 172.21.15.67:0/2278257833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99741023f0 msgr2=0x7f997419ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.678+0000 7f997aa46640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99741023f0 0x7f997419ac30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f996400ea60 tx=0x7f996400ef20 comp rx=0 tx=0).stop 2024-07-31T14:55:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.678+0000 7f997aa46640 1 -- 172.21.15.67:0/2278257833 shutdown_connections 2024-07-31T14:55:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.678+0000 7f997aa46640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99600b3d40 0x7f99600b6250 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.678+0000 7f997aa46640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9974108670 0x7f997419b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.678+0000 7f997aa46640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974107d20 0x7f997419b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.678+0000 7f997aa46640 1 --2- 172.21.15.67:0/2278257833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99741023f0 0x7f997419ac30 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.678+0000 7f997aa46640 1 -- 172.21.15.67:0/2278257833 >> 172.21.15.67:0/2278257833 conn(0x7f99740fddf0 msgr2=0x7f9974100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.679+0000 7f997aa46640 1 -- 172.21.15.67:0/2278257833 shutdown_connections 2024-07-31T14:55:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:22.679+0000 7f997aa46640 1 -- 172.21.15.67:0/2278257833 wait complete. 2024-07-31T14:55:22.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T14:55:23.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:22 smithi073 ceph-mon[36714]: pgmap v529: 129 pgs: 129 active+clean; 484 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-07-31T14:55:23.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:22 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1688670927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:55:23.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:22 smithi077 ceph-mon[37327]: pgmap v529: 129 pgs: 129 active+clean; 484 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-07-31T14:55:23.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:22 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1688670927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:55:23.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:22 smithi067 ceph-mon[32750]: pgmap v529: 129 pgs: 129 active+clean; 484 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-07-31T14:55:23.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:22 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1688670927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:55:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:55:23.340 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.4 2024-07-31T14:55:23.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:55:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:55:25.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:24 smithi067 ceph-mon[32750]: pgmap v530: 129 pgs: 129 active+clean; 486 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 322 op/s 2024-07-31T14:55:25.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:24 smithi077 ceph-mon[37327]: pgmap v530: 129 pgs: 129 active+clean; 486 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 322 op/s 2024-07-31T14:55:25.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:24 smithi073 ceph-mon[36714]: pgmap v530: 129 pgs: 129 active+clean; 486 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 322 op/s 2024-07-31T14:55:26.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.465+0000 7f558c08b640 1 -- 172.21.15.67:0/2479058486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584109850 msgr2=0x7f5584110080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:26.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.465+0000 7f558c08b640 1 --2- 172.21.15.67:0/2479058486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584109850 0x7f5584110080 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f557c009ca0 tx=0x7f557c02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.465+0000 7f558c08b640 1 -- 172.21.15.67:0/2479058486 shutdown_connections 2024-07-31T14:55:26.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.465+0000 7f558c08b640 1 --2- 172.21.15.67:0/2479058486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584109850 0x7f5584110080 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.465+0000 7f558c08b640 1 --2- 172.21.15.67:0/2479058486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584108f00 0x7f5584109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.465+0000 7f558c08b640 1 --2- 172.21.15.67:0/2479058486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55841035c0 0x7f55841039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.465+0000 7f558c08b640 1 -- 172.21.15.67:0/2479058486 >> 172.21.15.67:0/2479058486 conn(0x7f55840ff010 msgr2=0x7f5584101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:26.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.466+0000 7f558c08b640 1 -- 172.21.15.67:0/2479058486 shutdown_connections 2024-07-31T14:55:26.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.466+0000 7f558c08b640 1 -- 172.21.15.67:0/2479058486 wait complete. 2024-07-31T14:55:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.466+0000 7f558c08b640 1 Processor -- start 2024-07-31T14:55:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.466+0000 7f558c08b640 1 -- start start 2024-07-31T14:55:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f558c08b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55841035c0 0x7f5584199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f558c08b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584108f00 0x7f558419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f558c08b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584109850 0x7f558419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f558c08b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5584113500 con 0x7f5584108f00 2024-07-31T14:55:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f558c08b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5584113380 con 0x7f5584109850 2024-07-31T14:55:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f558c08b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5584113680 con 0x7f55841035c0 2024-07-31T14:55:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f5589626640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584108f00 0x7f558419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f5589626640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584108f00 0x7f558419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43418/0 (socket says 172.21.15.67:43418) 2024-07-31T14:55:26.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f5589626640 1 -- 172.21.15.67:0/2944442919 learned_addr learned my addr 172.21.15.67:0/2944442919 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:26.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f558a628640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584109850 0x7f558419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f5589e27640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55841035c0 0x7f5584199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f5589626640 1 -- 172.21.15.67:0/2944442919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55841035c0 msgr2=0x7f5584199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:26.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f5589626640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55841035c0 0x7f5584199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f5589626640 1 -- 172.21.15.67:0/2944442919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584109850 msgr2=0x7f558419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:26.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f5589626640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584109850 0x7f558419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f5589626640 1 -- 172.21.15.67:0/2944442919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558419fde0 con 0x7f5584108f00 2024-07-31T14:55:26.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f558a628640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584109850 0x7f558419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:26.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.467+0000 7f5589626640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584108f00 0x7f558419a410 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f557400e970 tx=0x7f557400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:26.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.468+0000 7f557affd640 1 -- 172.21.15.67:0/2944442919 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557400cd80 con 0x7f5584108f00 2024-07-31T14:55:26.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.468+0000 7f557affd640 1 -- 172.21.15.67:0/2944442919 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55740035f0 con 0x7f5584108f00 2024-07-31T14:55:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.468+0000 7f558c08b640 1 -- 172.21.15.67:0/2944442919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55841a00d0 con 0x7f5584108f00 2024-07-31T14:55:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.468+0000 7f557affd640 1 -- 172.21.15.67:0/2944442919 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574010690 con 0x7f5584108f00 2024-07-31T14:55:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.468+0000 7f558c08b640 1 -- 172.21.15.67:0/2944442919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55841a6830 con 0x7f5584108f00 2024-07-31T14:55:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.471+0000 7f55925dc740 1 Processor -- start 2024-07-31T14:55:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.472+0000 7f55925dc740 1 -- start start 2024-07-31T14:55:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.472+0000 7f557affd640 1 -- 172.21.15.67:0/2944442919 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5574010830 con 0x7f5584108f00 2024-07-31T14:55:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.472+0000 7f55925dc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba9e070 0x562d1be43570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.472+0000 7f55925dc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba751b0 0x562d1be43ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.472+0000 7f55925dc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 0x562d1be43ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.472+0000 7f55925dc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562d1bdb9bb0 con 0x562d1ba751b0 2024-07-31T14:55:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.472+0000 7f55925dc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562d1bda90b0 con 0x562d1ba9e070 2024-07-31T14:55:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.472+0000 7f55925dc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562d1bdad0f0 con 0x562d1ba7d390 2024-07-31T14:55:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.472+0000 7f558a628640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba9e070 0x562d1be43570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.472+0000 7f5589626640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 0x562d1be43ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.472+0000 7f558a628640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba9e070 0x562d1be43570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45130/0 (socket says 172.21.15.67:45130) 2024-07-31T14:55:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.472+0000 7f558a628640 1 -- 172.21.15.67:0/4182454209 learned_addr learned my addr 172.21.15.67:0/4182454209 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.472+0000 7f5589e27640 1 --2- 172.21.15.67:0/4182454209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba751b0 0x562d1be43ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.473+0000 7f558a628640 1 -- 172.21.15.67:0/4182454209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 msgr2=0x562d1be43ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:26.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.473+0000 7f558a628640 1 --2- 172.21.15.67:0/4182454209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 0x562d1be43ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.473+0000 7f558a628640 1 -- 172.21.15.67:0/4182454209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba751b0 msgr2=0x562d1be43ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.473+0000 7f558a628640 1 --2- 172.21.15.67:0/4182454209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba751b0 0x562d1be43ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.473+0000 7f558a628640 1 -- 172.21.15.67:0/4182454209 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1bd4fb90 con 0x562d1ba9e070 2024-07-31T14:55:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.473+0000 7f557affd640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f55700b3d40 0x7f55700b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.473+0000 7f5589626640 1 --2- 172.21.15.67:0/4182454209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 0x562d1be43ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.473+0000 7f5589e27640 1 --2- 172.21.15.67:0/4182454209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba751b0 0x562d1be43ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.473+0000 7f5589e27640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f55700b3d40 0x7f55700b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.473+0000 7f558a628640 1 --2- 172.21.15.67:0/4182454209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba9e070 0x562d1be43570 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f557c033040 tx=0x7f557c02f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24c5c1b6d34b0b45 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.473+0000 7f557affd640 1 -- 172.21.15.67:0/2944442919 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f55740e4580 con 0x7f5584108f00 2024-07-31T14:55:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.474+0000 7f556ffff640 1 -- 172.21.15.67:0/4182454209 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c031650 con 0x562d1ba9e070 2024-07-31T14:55:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.474+0000 7f556ffff640 1 -- 172.21.15.67:0/4182454209 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f557c036040 con 0x562d1ba9e070 2024-07-31T14:55:26.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.474+0000 7f556ffff640 1 -- 172.21.15.67:0/4182454209 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c004540 con 0x562d1ba9e070 2024-07-31T14:55:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.475+0000 7f55925dc740 1 -- 172.21.15.67:0/4182454209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba9e070 msgr2=0x562d1be43570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.475+0000 7f5589e27640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f55700b3d40 0x7f55700b6250 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x562d1be421f0 tx=0x7f558000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.475+0000 7f55925dc740 1 --2- 172.21.15.67:0/4182454209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba9e070 0x562d1be43570 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f557c033040 tx=0x7f557c02f3c0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.475+0000 7f55925dc740 1 -- 172.21.15.67:0/4182454209 shutdown_connections 2024-07-31T14:55:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.475+0000 7f55925dc740 1 --2- 172.21.15.67:0/4182454209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 0x562d1be43ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.475+0000 7f55925dc740 1 --2- 172.21.15.67:0/4182454209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba751b0 0x562d1be43ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.475+0000 7f55925dc740 1 --2- 172.21.15.67:0/4182454209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba9e070 0x562d1be43570 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.475+0000 7f55925dc740 1 -- 172.21.15.67:0/4182454209 >> 172.21.15.67:0/4182454209 conn(0x562d1ba92460 msgr2=0x562d1be41460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.475+0000 7f55925dc740 1 -- 172.21.15.67:0/4182454209 shutdown_connections 2024-07-31T14:55:26.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.475+0000 7f55925dc740 1 -- 172.21.15.67:0/4182454209 wait complete. 2024-07-31T14:55:26.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.475+0000 7f55925dc740 1 -- start start 2024-07-31T14:55:26.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.476+0000 7f55925dc740 1 Processor -- start 2024-07-31T14:55:26.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.476+0000 7f55925dc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.476+0000 7f55925dc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 0x562d1be53a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.476+0000 7f55925dc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba9e070 0x562d1be5f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.476+0000 7f55925dc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562d1bc0c700 con 0x562d1ba9e070 2024-07-31T14:55:26.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.476+0000 7f55925dc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562d1bd4b9e0 con 0x562d1ba751b0 2024-07-31T14:55:26.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.476+0000 7f55925dc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562d1be55e70 con 0x562d1ba7d390 2024-07-31T14:55:26.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.476+0000 7f55925dc740 10 client.0 _ll_register_callbacks cb 0x562d1bd664f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:26.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.476+0000 7f558a628640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba9e070 0x562d1be5f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.476+0000 7f5589e27640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.476+0000 7f558a628640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba9e070 0x562d1be5f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43444/0 (socket says 172.21.15.67:43444) 2024-07-31T14:55:26.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.476+0000 7f558a628640 1 -- 172.21.15.67:0/1768104912 learned_addr learned my addr 172.21.15.67:0/1768104912 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.477+0000 7f558a628640 1 -- 172.21.15.67:0/1768104912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 msgr2=0x562d1be53a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.477+0000 7f558a628640 1 --2- 172.21.15.67:0/1768104912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 0x562d1be53a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.477+0000 7f558a628640 1 -- 172.21.15.67:0/1768104912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 msgr2=0x562d1be534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.477+0000 7f558a628640 1 --2- 172.21.15.67:0/1768104912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.477+0000 7f558a628640 1 -- 172.21.15.67:0/1768104912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1bb09650 con 0x562d1ba9e070 2024-07-31T14:55:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.477+0000 7f5589e27640 1 --2- 172.21.15.67:0/1768104912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.477+0000 7f558a628640 1 --2- 172.21.15.67:0/1768104912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba9e070 0x562d1be5f200 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f557c005e30 tx=0x7f557c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cfd4d507ffdff7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.477+0000 7f556d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:55:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.478+0000 7f556d7fa640 1 -- 172.21.15.67:0/1768104912 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c004260 con 0x562d1ba9e070 2024-07-31T14:55:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.478+0000 7f556d7fa640 1 -- 172.21.15.67:0/1768104912 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f557c004400 con 0x562d1ba9e070 2024-07-31T14:55:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.478+0000 7f55925dc740 1 -- 172.21.15.67:0/1768104912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f557c036040 con 0x562d1ba9e070 2024-07-31T14:55:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.478+0000 7f556d7fa640 1 -- 172.21.15.67:0/1768104912 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f557c05e050 con 0x562d1ba9e070 2024-07-31T14:55:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.478+0000 7f55925dc740 10 client.16749 fetch_fsmap learned FSMap version 39 2024-07-31T14:55:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.478+0000 7f55925dc740 1 -- 172.21.15.67:0/1768104912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562d1be61ea0 con 0x562d1ba9e070 2024-07-31T14:55:26.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.479+0000 7f556d7fa640 1 -- 172.21.15.67:0/1768104912 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f557c055070 con 0x562d1ba9e070 2024-07-31T14:55:26.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.479+0000 7f556d7fa640 1 -- 172.21.15.67:0/1768104912 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f557c04f070 con 0x562d1ba9e070 2024-07-31T14:55:26.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.479+0000 7f55925dc740 10 client.16749 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:55:26.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.479+0000 7f55925dc740 10 client.16749 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:55:26.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.479+0000 7f55925dc740 20 client.16749 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:26.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.479+0000 7f55925dc740 1 --2- 172.21.15.67:0/1768104912 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x562d1ba5c9f0 0x562d1be65200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.480+0000 7f55925dc740 4 client.16749 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:55:26.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.480+0000 7f55925dc740 1 -- 172.21.15.67:0/1768104912 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f557c036040 con 0x562d1ba5c9f0 2024-07-31T14:55:26.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.480+0000 7f5589e27640 1 --2- 172.21.15.67:0/1768104912 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x562d1ba5c9f0 0x562d1be65200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.481+0000 7f556d7fa640 10 client.16749 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:55:26.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.481+0000 7f5589e27640 1 --2- 172.21.15.67:0/1768104912 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x562d1ba5c9f0 0x562d1be65200 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=df9df53078242762 server_cookie=301b84cb7ce939f8 in_seq=0 out_seq=0 2024-07-31T14:55:26.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.482+0000 7f556d7fa640 1 -- 172.21.15.67:0/1768104912 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f557c036040 con 0x562d1ba5c9f0 2024-07-31T14:55:26.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.482+0000 7f556d7fa640 10 client.16749 handle_command_reply: tid=0 2024-07-31T14:55:26.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.483+0000 7f55925dc740 1 client.16749 shutdown 2024-07-31T14:55:26.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.483+0000 7f55925dc740 1 -- 172.21.15.67:0/1768104912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba9e070 msgr2=0x562d1be5f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:26.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.483+0000 7f55925dc740 1 --2- 172.21.15.67:0/1768104912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba9e070 0x562d1be5f200 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f557c005e30 tx=0x7f557c003c20 comp rx=0 tx=0).stop 2024-07-31T14:55:26.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.483+0000 7f55925dc740 1 -- 172.21.15.67:0/1768104912 shutdown_connections 2024-07-31T14:55:26.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.483+0000 7f55925dc740 1 --2- 172.21.15.67:0/1768104912 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x562d1ba5c9f0 0x562d1be65200 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.483+0000 7f55925dc740 1 --2- 172.21.15.67:0/1768104912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba9e070 0x562d1be5f200 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.483+0000 7f55925dc740 1 --2- 172.21.15.67:0/1768104912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 0x562d1be53a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.483+0000 7f556d7fa640 0 client.16749 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:55:26.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.483+0000 7f55925dc740 1 --2- 172.21.15.67:0/1768104912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.483+0000 7f55925dc740 1 -- 172.21.15.67:0/1768104912 >> 172.21.15.67:0/1768104912 conn(0x562d1ba92460 msgr2=0x562d1be3fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:26.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.483+0000 7f55925dc740 1 -- 172.21.15.67:0/1768104912 shutdown_connections 2024-07-31T14:55:26.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.483+0000 7f55925dc740 1 -- 172.21.15.67:0/1768104912 wait complete. 2024-07-31T14:55:26.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.483+0000 7f55925dc740 20 client.16749 trim_cache size 0 max 16384 2024-07-31T14:55:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.500+0000 7f55925dc740 1 Processor -- start 2024-07-31T14:55:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.500+0000 7f55925dc740 1 -- start start 2024-07-31T14:55:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.500+0000 7f55925dc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be41240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.500+0000 7f55925dc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 0x562d1be5eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.500+0000 7f55925dc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba9e070 0x562d1be5f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.500+0000 7f55925dc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f55841ae3f0 con 0x562d1ba9e070 2024-07-31T14:55:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.500+0000 7f55925dc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f55841b00b0 con 0x562d1ba751b0 2024-07-31T14:55:26.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.500+0000 7f55925dc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f55841aeac0 con 0x562d1ba7d390 2024-07-31T14:55:26.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.501+0000 7f558a628640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba9e070 0x562d1be5f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.501+0000 7f5589e27640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be41240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.501+0000 7f558a628640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba9e070 0x562d1be5f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43456/0 (socket says 172.21.15.67:43456) 2024-07-31T14:55:26.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.501+0000 7f558a628640 1 -- 172.21.15.67:0/370975521 learned_addr learned my addr 172.21.15.67:0/370975521 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:26.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.501+0000 7f5589e27640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be41240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45150/0 (socket says 172.21.15.67:45150) 2024-07-31T14:55:26.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.501+0000 7f5589626640 1 --2- 172.21.15.67:0/370975521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 0x562d1be5eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.501+0000 7f558a628640 1 -- 172.21.15.67:0/370975521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 msgr2=0x562d1be5eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.501+0000 7f558a628640 1 --2- 172.21.15.67:0/370975521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 0x562d1be5eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.501+0000 7f558a628640 1 -- 172.21.15.67:0/370975521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 msgr2=0x562d1be41240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.501+0000 7f558a628640 1 --2- 172.21.15.67:0/370975521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be41240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.501+0000 7f558a628640 1 -- 172.21.15.67:0/370975521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1be44360 con 0x562d1ba9e070 2024-07-31T14:55:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.501+0000 7f5589e27640 1 --2- 172.21.15.67:0/370975521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be41240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.501+0000 7f558a628640 1 --2- 172.21.15.67:0/370975521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba9e070 0x562d1be5f530 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f557c000f80 tx=0x7f557c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83becaf2b73a786 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.501+0000 7f556f7fe640 1 -- 172.21.15.67:0/370975521 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c032eb0 con 0x562d1ba9e070 2024-07-31T14:55:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.502+0000 7f556f7fe640 1 -- 172.21.15.67:0/370975521 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f557c036070 con 0x562d1ba9e070 2024-07-31T14:55:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.502+0000 7f556f7fe640 1 -- 172.21.15.67:0/370975521 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c004400 con 0x562d1ba9e070 2024-07-31T14:55:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.502+0000 7f55925dc740 1 -- 172.21.15.67:0/370975521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba9e070 msgr2=0x562d1be5f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.502+0000 7f55925dc740 1 --2- 172.21.15.67:0/370975521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba9e070 0x562d1be5f530 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f557c000f80 tx=0x7f557c009b20 comp rx=0 tx=0).stop 2024-07-31T14:55:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.503+0000 7f55925dc740 1 -- 172.21.15.67:0/370975521 shutdown_connections 2024-07-31T14:55:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.503+0000 7f55925dc740 1 --2- 172.21.15.67:0/370975521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba9e070 0x562d1be5f530 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.503+0000 7f55925dc740 1 --2- 172.21.15.67:0/370975521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba7d390 0x562d1be5eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.503+0000 7f55925dc740 1 --2- 172.21.15.67:0/370975521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be41240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.503+0000 7f55925dc740 1 -- 172.21.15.67:0/370975521 >> 172.21.15.67:0/370975521 conn(0x562d1ba92460 msgr2=0x562d1be40b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:26.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.503+0000 7f55925dc740 1 -- 172.21.15.67:0/370975521 shutdown_connections 2024-07-31T14:55:26.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.503+0000 7f55925dc740 1 -- 172.21.15.67:0/370975521 wait complete. 2024-07-31T14:55:26.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.503+0000 7f55925dc740 1 -- start start 2024-07-31T14:55:26.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.503+0000 7f55925dc740 1 Processor -- start 2024-07-31T14:55:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f55925dc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be5b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f55925dc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba7d390 0x562d1be56d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f55925dc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba9e070 0x562d1be572a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f55925dc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f55841af9e0 con 0x562d1ba7d390 2024-07-31T14:55:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f55925dc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f55841af250 con 0x562d1ba751b0 2024-07-31T14:55:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f55925dc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f55841ac3d0 con 0x562d1ba9e070 2024-07-31T14:55:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f55925dc740 10 client.0 _ll_register_callbacks cb 0x7f558000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f5589e27640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be5b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f5589e27640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be5b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45162/0 (socket says 172.21.15.67:45162) 2024-07-31T14:55:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f5589e27640 1 -- 172.21.15.67:0/1203588717 learned_addr learned my addr 172.21.15.67:0/1203588717 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f5589626640 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba7d390 0x562d1be56d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f558a628640 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba9e070 0x562d1be572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f5589e27640 1 -- 172.21.15.67:0/1203588717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba9e070 msgr2=0x562d1be572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f5589e27640 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba9e070 0x562d1be572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f5589e27640 1 -- 172.21.15.67:0/1203588717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba7d390 msgr2=0x562d1be56d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f5589e27640 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba7d390 0x562d1be56d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f5589e27640 1 -- 172.21.15.67:0/1203588717 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1be57a30 con 0x562d1ba751b0 2024-07-31T14:55:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.504+0000 7f558a628640 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba9e070 0x562d1be572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.505+0000 7f5589626640 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba7d390 0x562d1be56d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.505+0000 7f5589e27640 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be5b870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x562d1be65800 tx=0x7f55800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ae0c47dc985a3a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.505+0000 7f556d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:55:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.505+0000 7f556d7fa640 1 -- 172.21.15.67:0/1203588717 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558001e970 con 0x562d1ba751b0 2024-07-31T14:55:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.505+0000 7f55925dc740 1 -- 172.21.15.67:0/1203588717 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f557c036070 con 0x562d1ba751b0 2024-07-31T14:55:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.505+0000 7f556d7fa640 1 -- 172.21.15.67:0/1203588717 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f558001eb10 con 0x562d1ba751b0 2024-07-31T14:55:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.505+0000 7f556d7fa640 1 -- 172.21.15.67:0/1203588717 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f558003c050 con 0x562d1ba751b0 2024-07-31T14:55:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.505+0000 7f55925dc740 10 client.26351 fetch_fsmap learned FSMap version 39 2024-07-31T14:55:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.505+0000 7f55925dc740 1 -- 172.21.15.67:0/1203588717 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562d1becd840 con 0x562d1ba751b0 2024-07-31T14:55:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.506+0000 7f556d7fa640 1 -- 172.21.15.67:0/1203588717 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f557c036070 con 0x562d1ba751b0 2024-07-31T14:55:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.506+0000 7f556d7fa640 1 -- 172.21.15.67:0/1203588717 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f5580033480 con 0x562d1ba751b0 2024-07-31T14:55:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.506+0000 7f55925dc740 10 client.26351 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:55:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.506+0000 7f55925dc740 10 client.26351 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:55:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.506+0000 7f55925dc740 20 client.26351 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.506+0000 7f55925dc740 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x562d1ba31da0 0x562d1becffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.506+0000 7f55925dc740 4 client.26351 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:55:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.506+0000 7f55925dc740 1 -- 172.21.15.67:0/1203588717 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d1bdbbb00 con 0x562d1ba31da0 2024-07-31T14:55:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.506+0000 7f5589626640 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x562d1ba31da0 0x562d1becffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.507+0000 7f556d7fa640 10 client.26351 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:55:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.507+0000 7f5589626640 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x562d1ba31da0 0x562d1becffe0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a991aeb8908a782 server_cookie=4716233e6070396f in_seq=0 out_seq=0 2024-07-31T14:55:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.507+0000 7f556d7fa640 1 -- 172.21.15.67:0/1203588717 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d1bdbbb00 con 0x562d1ba31da0 2024-07-31T14:55:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.507+0000 7f556d7fa640 10 client.26351 handle_command_reply: tid=0 2024-07-31T14:55:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.507+0000 7f55925dc740 1 client.26351 shutdown 2024-07-31T14:55:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.507+0000 7f55925dc740 1 -- 172.21.15.67:0/1203588717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 msgr2=0x562d1be5b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.507+0000 7f55925dc740 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be5b870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x562d1be65800 tx=0x7f55800046c0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.507+0000 7f55925dc740 1 -- 172.21.15.67:0/1203588717 shutdown_connections 2024-07-31T14:55:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.507+0000 7f55925dc740 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x562d1ba31da0 0x562d1becffe0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.507+0000 7f55925dc740 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d1ba9e070 0x562d1be572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.508+0000 7f55925dc740 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562d1ba7d390 0x562d1be56d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.508+0000 7f556d7fa640 0 client.26351 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:55:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.508+0000 7f55925dc740 1 --2- 172.21.15.67:0/1203588717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d1ba751b0 0x562d1be5b870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.508+0000 7f55925dc740 1 -- 172.21.15.67:0/1203588717 >> 172.21.15.67:0/1203588717 conn(0x562d1ba92460 msgr2=0x562d1be40b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.508+0000 7f55925dc740 1 -- 172.21.15.67:0/1203588717 shutdown_connections 2024-07-31T14:55:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.508+0000 7f55925dc740 1 -- 172.21.15.67:0/1203588717 wait complete. 2024-07-31T14:55:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.508+0000 7f55925dc740 20 client.26351 trim_cache size 0 max 16384 2024-07-31T14:55:26.513 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:55:26.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.509+0000 7f558c08b640 1 -- 172.21.15.67:0/2944442919 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f55700b3d40 msgr2=0x7f55700b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:26.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.509+0000 7f558c08b640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f55700b3d40 0x7f55700b6250 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x562d1be421f0 tx=0x7f558000b390 comp rx=0 tx=0).stop 2024-07-31T14:55:26.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.509+0000 7f558c08b640 1 -- 172.21.15.67:0/2944442919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584108f00 msgr2=0x7f558419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:26.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.509+0000 7f558c08b640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584108f00 0x7f558419a410 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f557400e970 tx=0x7f557400ee30 comp rx=0 tx=0).stop 2024-07-31T14:55:26.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.509+0000 7f558c08b640 1 -- 172.21.15.67:0/2944442919 shutdown_connections 2024-07-31T14:55:26.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.509+0000 7f558c08b640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f55700b3d40 0x7f55700b6250 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.509+0000 7f558c08b640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584109850 0x7f558419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.509+0000 7f558c08b640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584108f00 0x7f558419a410 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.509+0000 7f558c08b640 1 --2- 172.21.15.67:0/2944442919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55841035c0 0x7f5584199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:26.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.509+0000 7f558c08b640 1 -- 172.21.15.67:0/2944442919 >> 172.21.15.67:0/2944442919 conn(0x7f55840ff010 msgr2=0x7f5584101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:26.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.509+0000 7f558c08b640 1 -- 172.21.15.67:0/2944442919 shutdown_connections 2024-07-31T14:55:26.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:26.509+0000 7f558c08b640 1 -- 172.21.15.67:0/2944442919 wait complete. 2024-07-31T14:55:27.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:26 smithi077 ceph-mon[37327]: pgmap v531: 129 pgs: 129 active+clean; 531 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 4.8 MiB/s wr, 792 op/s 2024-07-31T14:55:27.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:26 smithi067 ceph-mon[32750]: pgmap v531: 129 pgs: 129 active+clean; 531 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 4.8 MiB/s wr, 792 op/s 2024-07-31T14:55:27.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T14:55:27.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:26 smithi073 ceph-mon[36714]: pgmap v531: 129 pgs: 129 active+clean; 531 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 4.8 MiB/s wr, 792 op/s 2024-07-31T14:55:28.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.580+0000 7f0e9600e640 1 -- 172.21.15.67:0/2880479476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e90109820 msgr2=0x7f0e90110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:28.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.580+0000 7f0e9600e640 1 --2- 172.21.15.67:0/2880479476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e90109820 0x7f0e90110050 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009ca0 tx=0x7f0e8802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:55:28.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.580+0000 7f0e9600e640 1 -- 172.21.15.67:0/2880479476 shutdown_connections 2024-07-31T14:55:28.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.580+0000 7f0e9600e640 1 --2- 172.21.15.67:0/2880479476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e90109820 0x7f0e90110050 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:28.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.580+0000 7f0e9600e640 1 --2- 172.21.15.67:0/2880479476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e90108ed0 0x7f0e901092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:28.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.580+0000 7f0e9600e640 1 --2- 172.21.15.67:0/2880479476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e90103590 0x7f0e901039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:28.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.580+0000 7f0e9600e640 1 -- 172.21.15.67:0/2880479476 >> 172.21.15.67:0/2880479476 conn(0x7f0e9007b280 msgr2=0x7f0e9007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:28.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.580+0000 7f0e9600e640 1 -- 172.21.15.67:0/2880479476 shutdown_connections 2024-07-31T14:55:28.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.580+0000 7f0e9600e640 1 -- 172.21.15.67:0/2880479476 wait complete. 2024-07-31T14:55:28.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.581+0000 7f0e9600e640 1 Processor -- start 2024-07-31T14:55:28.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.581+0000 7f0e9600e640 1 -- start start 2024-07-31T14:55:28.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.581+0000 7f0e9600e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e90103590 0x7f0e90075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:28.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.581+0000 7f0e9600e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e90108ed0 0x7f0e90075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:28.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.581+0000 7f0e9600e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e90109820 0x7f0e90075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:28.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.581+0000 7f0e9600e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901134f0 con 0x7f0e90103590 2024-07-31T14:55:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.581+0000 7f0e9600e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90113370 con 0x7f0e90108ed0 2024-07-31T14:55:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.581+0000 7f0e9600e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90113670 con 0x7f0e90109820 2024-07-31T14:55:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.581+0000 7f0e8f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e90103590 0x7f0e90075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.581+0000 7f0e8f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e90103590 0x7f0e90075140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43496/0 (socket says 172.21.15.67:43496) 2024-07-31T14:55:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.581+0000 7f0e8f7fe640 1 -- 172.21.15.67:0/536706039 learned_addr learned my addr 172.21.15.67:0/536706039 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.581+0000 7f0e8ffff640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e90109820 0x7f0e90075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.581+0000 7f0e8effd640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e90108ed0 0x7f0e90075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.582+0000 7f0e8f7fe640 1 -- 172.21.15.67:0/536706039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e90109820 msgr2=0x7f0e90075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.582+0000 7f0e8f7fe640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e90109820 0x7f0e90075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.582+0000 7f0e8f7fe640 1 -- 172.21.15.67:0/536706039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e90108ed0 msgr2=0x7f0e90075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.582+0000 7f0e8f7fe640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e90108ed0 0x7f0e90075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:28.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.582+0000 7f0e8f7fe640 1 -- 172.21.15.67:0/536706039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e901a8180 con 0x7f0e90103590 2024-07-31T14:55:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.582+0000 7f0e8effd640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e90108ed0 0x7f0e90075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:55:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.582+0000 7f0e8f7fe640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e90103590 0x7f0e90075140 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000d950 tx=0x7f0e8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.582+0000 7f0e8cff9640 1 -- 172.21.15.67:0/536706039 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e80017030 con 0x7f0e90103590 2024-07-31T14:55:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.582+0000 7f0e8cff9640 1 -- 172.21.15.67:0/536706039 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e80004020 con 0x7f0e90103590 2024-07-31T14:55:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.582+0000 7f0e9600e640 1 -- 172.21.15.67:0/536706039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e901a8470 con 0x7f0e90103590 2024-07-31T14:55:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.582+0000 7f0e8cff9640 1 -- 172.21.15.67:0/536706039 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e800047a0 con 0x7f0e90103590 2024-07-31T14:55:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.582+0000 7f0e9600e640 1 -- 172.21.15.67:0/536706039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e901a8a00 con 0x7f0e90103590 2024-07-31T14:55:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.583+0000 7f0e9600e640 1 -- 172.21.15.67:0/536706039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e901039a0 con 0x7f0e90103590 2024-07-31T14:55:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.585+0000 7f0e8cff9640 1 -- 172.21.15.67:0/536706039 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0e80009a30 con 0x7f0e90103590 2024-07-31T14:55:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.585+0000 7f0e8cff9640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0e780b3dc0 0x7f0e780b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.586+0000 7f0e8effd640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0e780b3dc0 0x7f0e780b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:28.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.586+0000 7f0e8cff9640 1 -- 172.21.15.67:0/536706039 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f0e800e4400 con 0x7f0e90103590 2024-07-31T14:55:28.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.586+0000 7f0e8effd640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0e780b3dc0 0x7f0e780b62d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0e84005fd0 tx=0x7f0e8400b5d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:28.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.589+0000 7f0e8cff9640 1 -- 172.21.15.67:0/536706039 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f0e800ab480 con 0x7f0e90103590 2024-07-31T14:55:28.641 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:55:28.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.832+0000 7f0e9600e640 1 -- 172.21.15.67:0/536706039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e90076440 con 0x7f0e90103590 2024-07-31T14:55:28.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.832+0000 7f0e8cff9640 1 -- 172.21.15.67:0/536706039 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11980 (secure 0 0 0) 0x7f0e800aad80 con 0x7f0e90103590 2024-07-31T14:55:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:55:28.834 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":39,"btime":"2024-07-31T14:54:12:039205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:54:11.524751+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T14:55:28.835 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 39 2024-07-31T14:55:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.835+0000 7f0e9600e640 1 -- 172.21.15.67:0/536706039 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0e780b3dc0 msgr2=0x7f0e780b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.835+0000 7f0e9600e640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0e780b3dc0 0x7f0e780b62d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0e84005fd0 tx=0x7f0e8400b5d0 comp rx=0 tx=0).stop 2024-07-31T14:55:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.835+0000 7f0e9600e640 1 -- 172.21.15.67:0/536706039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e90103590 msgr2=0x7f0e90075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:28.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.835+0000 7f0e9600e640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e90103590 0x7f0e90075140 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000d950 tx=0x7f0e8000de10 comp rx=0 tx=0).stop 2024-07-31T14:55:28.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.835+0000 7f0e9600e640 1 -- 172.21.15.67:0/536706039 shutdown_connections 2024-07-31T14:55:28.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.835+0000 7f0e9600e640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0e780b3dc0 0x7f0e780b62d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:28.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.835+0000 7f0e9600e640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e90109820 0x7f0e90075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:28.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.835+0000 7f0e9600e640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e90108ed0 0x7f0e90075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:28.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.835+0000 7f0e9600e640 1 --2- 172.21.15.67:0/536706039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e90103590 0x7f0e90075140 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:28.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.835+0000 7f0e9600e640 1 -- 172.21.15.67:0/536706039 >> 172.21.15.67:0/536706039 conn(0x7f0e9007b280 msgr2=0x7f0e90106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:28.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.835+0000 7f0e9600e640 1 -- 172.21.15.67:0/536706039 shutdown_connections 2024-07-31T14:55:28.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:28.835+0000 7f0e9600e640 1 -- 172.21.15.67:0/536706039 wait complete. 2024-07-31T14:55:29.194 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:28 smithi073 ceph-mon[36714]: pgmap v532: 129 pgs: 129 active+clean; 572 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 9.8 MiB/s wr, 1.07k op/s 2024-07-31T14:55:29.195 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:28 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/536706039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:55:29.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:28 smithi077 ceph-mon[37327]: pgmap v532: 129 pgs: 129 active+clean; 572 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 9.8 MiB/s wr, 1.07k op/s 2024-07-31T14:55:29.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:28 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/536706039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:55:29.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:28 smithi067 ceph-mon[32750]: pgmap v532: 129 pgs: 129 active+clean; 572 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 9.8 MiB/s wr, 1.07k op/s 2024-07-31T14:55:29.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:28 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/536706039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:55:29.815 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 138.4 --await-for 45.0' 2024-07-31T14:55:29.816 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 138.4 --await-for 45.0 2024-07-31T14:55:31.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:30 smithi077 ceph-mon[37327]: pgmap v533: 129 pgs: 129 active+clean; 633 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 14 MiB/s wr, 1.13k op/s 2024-07-31T14:55:31.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:30 smithi067 ceph-mon[32750]: pgmap v533: 129 pgs: 129 active+clean; 633 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 14 MiB/s wr, 1.13k op/s 2024-07-31T14:55:31.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:30 smithi073 ceph-mon[36714]: pgmap v533: 129 pgs: 129 active+clean; 633 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 14 MiB/s wr, 1.13k op/s 2024-07-31T14:55:32.973 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:32 smithi067 ceph-mon[32750]: pgmap v534: 129 pgs: 129 active+clean; 658 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-07-31T14:55:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.972+0000 7f708eaf7640 1 -- 172.21.15.67:0/156936325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7088107df0 msgr2=0x7f7088108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.972+0000 7f708eaf7640 1 --2- 172.21.15.67:0/156936325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7088107df0 0x7f7088108200 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7074009860 tx=0x7f707402a340 comp rx=0 tx=0).stop 2024-07-31T14:55:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.972+0000 7f708eaf7640 1 -- 172.21.15.67:0/156936325 shutdown_connections 2024-07-31T14:55:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.972+0000 7f708eaf7640 1 --2- 172.21.15.67:0/156936325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7088108740 0x7f708810ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.972+0000 7f708eaf7640 1 --2- 172.21.15.67:0/156936325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7088107df0 0x7f7088108200 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.972+0000 7f708eaf7640 1 --2- 172.21.15.67:0/156936325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70881024b0 0x7f70881028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.972+0000 7f708eaf7640 1 -- 172.21.15.67:0/156936325 >> 172.21.15.67:0/156936325 conn(0x7f7088079170 msgr2=0x7f7088079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.972+0000 7f708eaf7640 1 -- 172.21.15.67:0/156936325 shutdown_connections 2024-07-31T14:55:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.972+0000 7f708eaf7640 1 -- 172.21.15.67:0/156936325 wait complete. 2024-07-31T14:55:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f708eaf7640 1 Processor -- start 2024-07-31T14:55:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f708eaf7640 1 -- start start 2024-07-31T14:55:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f708eaf7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70881024b0 0x7f7088199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f708eaf7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7088107df0 0x7f708819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f708eaf7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7088108740 0x7f708819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f708eaf7640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7088112370 con 0x7f7088107df0 2024-07-31T14:55:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f708eaf7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f70881121f0 con 0x7f70881024b0 2024-07-31T14:55:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f708eaf7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f70881124f0 con 0x7f7088108740 2024-07-31T14:55:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f7087fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7088107df0 0x7f708819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f708c893640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70881024b0 0x7f7088199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f7087fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7088107df0 0x7f708819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45684/0 (socket says 172.21.15.67:45684) 2024-07-31T14:55:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f7087fff640 1 -- 172.21.15.67:0/3274833560 learned_addr learned my addr 172.21.15.67:0/3274833560 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f708d094640 1 --2- 172.21.15.67:0/3274833560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7088108740 0x7f708819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f708c893640 1 -- 172.21.15.67:0/3274833560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7088108740 msgr2=0x7f708819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.973+0000 7f708c893640 1 --2- 172.21.15.67:0/3274833560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7088108740 0x7f708819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.974+0000 7f708c893640 1 -- 172.21.15.67:0/3274833560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7088107df0 msgr2=0x7f708819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:32.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.974+0000 7f708c893640 1 --2- 172.21.15.67:0/3274833560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7088107df0 0x7f708819a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.974+0000 7f708c893640 1 -- 172.21.15.67:0/3274833560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708819fdc0 con 0x7f70881024b0 2024-07-31T14:55:32.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.974+0000 7f7087fff640 1 --2- 172.21.15.67:0/3274833560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7088107df0 0x7f708819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:32.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.974+0000 7f708c893640 1 --2- 172.21.15.67:0/3274833560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70881024b0 0x7f7088199e50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f707c005fd0 tx=0x7f707c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:32.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.974+0000 7f7085ffb640 1 -- 172.21.15.67:0/3274833560 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c00cd00 con 0x7f70881024b0 2024-07-31T14:55:32.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.974+0000 7f7085ffb640 1 -- 172.21.15.67:0/3274833560 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f707c0035f0 con 0x7f70881024b0 2024-07-31T14:55:32.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.974+0000 7f7085ffb640 1 -- 172.21.15.67:0/3274833560 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c010690 con 0x7f70881024b0 2024-07-31T14:55:32.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.974+0000 7f708eaf7640 1 -- 172.21.15.67:0/3274833560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70881a00b0 con 0x7f70881024b0 2024-07-31T14:55:32.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.974+0000 7f708eaf7640 1 -- 172.21.15.67:0/3274833560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70881a6810 con 0x7f70881024b0 2024-07-31T14:55:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.977+0000 7f7085ffb640 1 -- 172.21.15.67:0/3274833560 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f707c003aa0 con 0x7f70881024b0 2024-07-31T14:55:32.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.977+0000 7f7095048740 1 Processor -- start 2024-07-31T14:55:32.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.977+0000 7f7095048740 1 -- start start 2024-07-31T14:55:32.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f7095048740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c64070 0x558e15009570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:32.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f7095048740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15009ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:32.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f7095048740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c43390 0x558e15009ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:32.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f7095048740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558e14f7fbb0 con 0x558e14c64070 2024-07-31T14:55:32.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f7095048740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558e14f6f0b0 con 0x558e14c3b1b0 2024-07-31T14:55:32.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f7095048740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558e14f730f0 con 0x558e14c43390 2024-07-31T14:55:32.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f708c893640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15009ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:32.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f708d094640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c64070 0x558e15009570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:32.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f708c893640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15009ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51122/0 (socket says 172.21.15.67:51122) 2024-07-31T14:55:32.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f708d094640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c64070 0x558e15009570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45694/0 (socket says 172.21.15.67:45694) 2024-07-31T14:55:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f708c893640 1 -- 172.21.15.67:0/3062411808 learned_addr learned my addr 172.21.15.67:0/3062411808 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f7087fff640 1 --2- 172.21.15.67:0/3062411808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c43390 0x558e15009ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f7085ffb640 1 --2- 172.21.15.67:0/3274833560 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f70780b3dc0 0x7f70780b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f708c893640 1 -- 172.21.15.67:0/3062411808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c43390 msgr2=0x558e15009ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f708c893640 1 --2- 172.21.15.67:0/3062411808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c43390 0x558e15009ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f708c893640 1 -- 172.21.15.67:0/3062411808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c64070 msgr2=0x558e15009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f708c893640 1 --2- 172.21.15.67:0/3062411808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c64070 0x558e15009570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f708c893640 1 -- 172.21.15.67:0/3062411808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e14f15b90 con 0x558e14c3b1b0 2024-07-31T14:55:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.978+0000 7f708d094640 1 --2- 172.21.15.67:0/3274833560 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f70780b3dc0 0x7f70780b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.979+0000 7f7085ffb640 1 -- 172.21.15.67:0/3274833560 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f707c0e4570 con 0x7f70881024b0 2024-07-31T14:55:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.979+0000 7f708c893640 1 --2- 172.21.15.67:0/3062411808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15009ab0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f707c0e1890 tx=0x7f707c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1dc2ed8b5a1ab6c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.979+0000 7f708d094640 1 --2- 172.21.15.67:0/3274833560 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f70780b3dc0 0x7f70780b62d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7080009fd0 tx=0x7f7080004ab0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:32.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.979+0000 7f7072ffd640 1 -- 172.21.15.67:0/3062411808 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c0aabe0 con 0x558e14c3b1b0 2024-07-31T14:55:32.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.979+0000 7f7072ffd640 1 -- 172.21.15.67:0/3062411808 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f707c0aa920 con 0x558e14c3b1b0 2024-07-31T14:55:32.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.979+0000 7f7072ffd640 1 -- 172.21.15.67:0/3062411808 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c0aa580 con 0x558e14c3b1b0 2024-07-31T14:55:32.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.980+0000 7f7095048740 1 -- 172.21.15.67:0/3062411808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 msgr2=0x558e15009ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:32.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.980+0000 7f7095048740 1 --2- 172.21.15.67:0/3062411808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15009ab0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f707c0e1890 tx=0x7f707c020000 comp rx=0 tx=0).stop 2024-07-31T14:55:32.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.980+0000 7f7095048740 1 -- 172.21.15.67:0/3062411808 shutdown_connections 2024-07-31T14:55:32.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.980+0000 7f7095048740 1 --2- 172.21.15.67:0/3062411808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c43390 0x558e15009ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.980+0000 7f7095048740 1 --2- 172.21.15.67:0/3062411808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15009ab0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.980+0000 7f7095048740 1 --2- 172.21.15.67:0/3062411808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c64070 0x558e15009570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.980+0000 7f7095048740 1 -- 172.21.15.67:0/3062411808 >> 172.21.15.67:0/3062411808 conn(0x558e14c58460 msgr2=0x558e15007460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:32.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.980+0000 7f7095048740 1 -- 172.21.15.67:0/3062411808 shutdown_connections 2024-07-31T14:55:32.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.980+0000 7f7095048740 1 -- 172.21.15.67:0/3062411808 wait complete. 2024-07-31T14:55:32.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.980+0000 7f7095048740 1 -- start start 2024-07-31T14:55:32.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.980+0000 7f7095048740 1 Processor -- start 2024-07-31T14:55:32.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f7095048740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15015190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:32.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f7095048740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c43390 0x558e150156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f7095048740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c64070 0x558e15025420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f7095048740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558e14dd2700 con 0x558e14c43390 2024-07-31T14:55:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f7095048740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558e14f119e0 con 0x558e14c3b1b0 2024-07-31T14:55:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f7095048740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558e1501c100 con 0x558e14c64070 2024-07-31T14:55:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f7095048740 10 client.0 _ll_register_callbacks cb 0x558e14f2c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f708c893640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15015190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f708c893640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15015190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51126/0 (socket says 172.21.15.67:51126) 2024-07-31T14:55:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f708c893640 1 -- 172.21.15.67:0/690761769 learned_addr learned my addr 172.21.15.67:0/690761769 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f7087fff640 1 --2- 172.21.15.67:0/690761769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c43390 0x558e150156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f708d094640 1 --2- 172.21.15.67:0/690761769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c64070 0x558e15025420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f708c893640 1 -- 172.21.15.67:0/690761769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c64070 msgr2=0x558e15025420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f708c893640 1 --2- 172.21.15.67:0/690761769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c64070 0x558e15025420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f708c893640 1 -- 172.21.15.67:0/690761769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c43390 msgr2=0x558e150156d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f708c893640 1 --2- 172.21.15.67:0/690761769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c43390 0x558e150156d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f708c893640 1 -- 172.21.15.67:0/690761769 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e14ccf650 con 0x558e14c3b1b0 2024-07-31T14:55:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.981+0000 7f708d094640 1 --2- 172.21.15.67:0/690761769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c64070 0x558e15025420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:55:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.982+0000 7f708c893640 1 --2- 172.21.15.67:0/690761769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15015190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f707c009230 tx=0x7f707c0af510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7190c7f56a2e2a33 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.982+0000 7f7067fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:55:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.982+0000 7f7067fff640 1 -- 172.21.15.67:0/690761769 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c0aa760 con 0x558e14c3b1b0 2024-07-31T14:55:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.982+0000 7f7067fff640 1 -- 172.21.15.67:0/690761769 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f707c0a6780 con 0x558e14c3b1b0 2024-07-31T14:55:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.982+0000 7f7095048740 1 -- 172.21.15.67:0/690761769 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f707c0aa920 con 0x558e14c3b1b0 2024-07-31T14:55:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.982+0000 7f7067fff640 1 -- 172.21.15.67:0/690761769 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f707c0aabe0 con 0x558e14c3b1b0 2024-07-31T14:55:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.982+0000 7f7095048740 10 client.26369 fetch_fsmap learned FSMap version 39 2024-07-31T14:55:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.982+0000 7f7095048740 1 -- 172.21.15.67:0/690761769 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558e150280c0 con 0x558e14c3b1b0 2024-07-31T14:55:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.982+0000 7f7067fff640 1 -- 172.21.15.67:0/690761769 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f707c0aa920 con 0x558e14c3b1b0 2024-07-31T14:55:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.982+0000 7f7067fff640 1 -- 172.21.15.67:0/690761769 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f707c0e1bd0 con 0x558e14c3b1b0 2024-07-31T14:55:32.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.982+0000 7f7095048740 10 client.26369 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:55:32.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.982+0000 7f7095048740 10 client.26369 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:55:32.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.982+0000 7f7095048740 20 client.26369 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:32.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.982+0000 7f7095048740 1 --2- 172.21.15.67:0/690761769 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x558e14c229f0 0x558e1502b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:32.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.983+0000 7f7095048740 4 client.26369 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:55:32.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.983+0000 7f7095048740 1 -- 172.21.15.67:0/690761769 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e14f81b00 con 0x558e14c229f0 2024-07-31T14:55:32.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.983+0000 7f7087fff640 1 --2- 172.21.15.67:0/690761769 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x558e14c229f0 0x558e1502b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:32.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.983+0000 7f7067fff640 10 client.26369 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:55:32.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.983+0000 7f7087fff640 1 --2- 172.21.15.67:0/690761769 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x558e14c229f0 0x558e1502b420 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4f44076e3978c7b3 server_cookie=1849ae51b980b445 in_seq=0 out_seq=0 2024-07-31T14:55:32.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.984+0000 7f7067fff640 1 -- 172.21.15.67:0/690761769 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e14f81b00 con 0x558e14c229f0 2024-07-31T14:55:32.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.984+0000 7f7067fff640 10 client.26369 handle_command_reply: tid=0 2024-07-31T14:55:32.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.984+0000 7f7095048740 1 client.26369 shutdown 2024-07-31T14:55:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.984+0000 7f7095048740 1 -- 172.21.15.67:0/690761769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 msgr2=0x558e15015190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.984+0000 7f7095048740 1 --2- 172.21.15.67:0/690761769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15015190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f707c009230 tx=0x7f707c0af510 comp rx=0 tx=0).stop 2024-07-31T14:55:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.984+0000 7f7095048740 1 -- 172.21.15.67:0/690761769 shutdown_connections 2024-07-31T14:55:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.984+0000 7f7095048740 1 --2- 172.21.15.67:0/690761769 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x558e14c229f0 0x558e1502b420 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.984+0000 7f7095048740 1 --2- 172.21.15.67:0/690761769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c64070 0x558e15025420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.984+0000 7f7095048740 1 --2- 172.21.15.67:0/690761769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c43390 0x558e150156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.984+0000 7f7067fff640 0 client.26369 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:55:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.984+0000 7f7095048740 1 --2- 172.21.15.67:0/690761769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15015190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.984+0000 7f7095048740 1 -- 172.21.15.67:0/690761769 >> 172.21.15.67:0/690761769 conn(0x558e14c58460 msgr2=0x558e15005c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.985+0000 7f7095048740 1 -- 172.21.15.67:0/690761769 shutdown_connections 2024-07-31T14:55:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.985+0000 7f7095048740 1 -- 172.21.15.67:0/690761769 wait complete. 2024-07-31T14:55:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.985+0000 7f7095048740 20 client.26369 trim_cache size 0 max 16384 2024-07-31T14:55:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.999+0000 7f7095048740 1 Processor -- start 2024-07-31T14:55:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.999+0000 7f7095048740 1 -- start start 2024-07-31T14:55:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.999+0000 7f7095048740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c229f0 0x558e15006350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.999+0000 7f7095048740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15025390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:32.999+0000 7f7095048740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c43390 0x558e150258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f7095048740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f70881ae440 con 0x558e14c229f0 2024-07-31T14:55:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f7095048740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f70881b0100 con 0x558e14c3b1b0 2024-07-31T14:55:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f7095048740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f70881aeb10 con 0x558e14c43390 2024-07-31T14:55:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f708c893640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c229f0 0x558e15006350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f7087fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15025390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f7087fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15025390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51128/0 (socket says 172.21.15.67:51128) 2024-07-31T14:55:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f708c893640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c229f0 0x558e15006350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45718/0 (socket says 172.21.15.67:45718) 2024-07-31T14:55:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f7087fff640 1 -- 172.21.15.67:0/2541481551 learned_addr learned my addr 172.21.15.67:0/2541481551 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f708d094640 1 --2- 172.21.15.67:0/2541481551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c43390 0x558e150258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f7087fff640 1 -- 172.21.15.67:0/2541481551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c43390 msgr2=0x558e150258d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f7087fff640 1 --2- 172.21.15.67:0/2541481551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c43390 0x558e150258d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f7087fff640 1 -- 172.21.15.67:0/2541481551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c229f0 msgr2=0x558e15006350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f7087fff640 1 --2- 172.21.15.67:0/2541481551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c229f0 0x558e15006350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f7087fff640 1 -- 172.21.15.67:0/2541481551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e150028c0 con 0x558e14c3b1b0 2024-07-31T14:55:33.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f7087fff640 1 --2- 172.21.15.67:0/2541481551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15025390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7074035040 tx=0x7f7074005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6c1dbfea7a02f6e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:33.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f70727fc640 1 -- 172.21.15.67:0/2541481551 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707404d070 con 0x558e14c3b1b0 2024-07-31T14:55:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f70727fc640 1 -- 172.21.15.67:0/2541481551 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7074030450 con 0x558e14c3b1b0 2024-07-31T14:55:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.000+0000 7f70727fc640 1 -- 172.21.15.67:0/2541481551 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074037040 con 0x558e14c3b1b0 2024-07-31T14:55:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.001+0000 7f7095048740 1 -- 172.21.15.67:0/2541481551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 msgr2=0x558e15025390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.001+0000 7f7095048740 1 --2- 172.21.15.67:0/2541481551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15025390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7074035040 tx=0x7f7074005e10 comp rx=0 tx=0).stop 2024-07-31T14:55:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.001+0000 7f7095048740 1 -- 172.21.15.67:0/2541481551 shutdown_connections 2024-07-31T14:55:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.001+0000 7f7095048740 1 --2- 172.21.15.67:0/2541481551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c43390 0x558e150258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.001+0000 7f7095048740 1 --2- 172.21.15.67:0/2541481551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c3b1b0 0x558e15025390 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.001+0000 7f7095048740 1 --2- 172.21.15.67:0/2541481551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c229f0 0x558e15006350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.001+0000 7f7095048740 1 -- 172.21.15.67:0/2541481551 >> 172.21.15.67:0/2541481551 conn(0x558e14c58460 msgr2=0x558e15005c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.001+0000 7f7095048740 1 -- 172.21.15.67:0/2541481551 shutdown_connections 2024-07-31T14:55:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.001+0000 7f7095048740 1 -- 172.21.15.67:0/2541481551 wait complete. 2024-07-31T14:55:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.002+0000 7f7095048740 1 -- start start 2024-07-31T14:55:33.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.002+0000 7f7095048740 1 Processor -- start 2024-07-31T14:55:33.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.002+0000 7f7095048740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c229f0 0x558e1502c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:33.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.002+0000 7f7095048740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c3b1b0 0x558e1502c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:33.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.002+0000 7f7095048740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c43390 0x558e1502ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:33.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.002+0000 7f7095048740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f70881afa30 con 0x558e14c43390 2024-07-31T14:55:33.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.002+0000 7f7095048740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f70881af2a0 con 0x558e14c229f0 2024-07-31T14:55:33.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.002+0000 7f7095048740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f70881ac420 con 0x558e14c3b1b0 2024-07-31T14:55:33.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.002+0000 7f7095048740 10 client.0 _ll_register_callbacks cb 0x7f7080009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:33.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.002+0000 7f708c893640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c229f0 0x558e1502c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:33.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f708d094640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c43390 0x558e1502ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:33.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f708d094640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c43390 0x558e1502ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45726/0 (socket says 172.21.15.67:45726) 2024-07-31T14:55:33.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f708d094640 1 -- 172.21.15.67:0/3570096737 learned_addr learned my addr 172.21.15.67:0/3570096737 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:33.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f708c893640 1 --2- 172.21.15.67:0/3570096737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c229f0 0x558e1502c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51144/0 (socket says 172.21.15.67:51144) 2024-07-31T14:55:33.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f7087fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c3b1b0 0x558e1502c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:33.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f708c893640 1 -- 172.21.15.67:0/3570096737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c3b1b0 msgr2=0x558e1502c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:33.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f708c893640 1 --2- 172.21.15.67:0/3570096737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c3b1b0 0x558e1502c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f708c893640 1 -- 172.21.15.67:0/3570096737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c43390 msgr2=0x558e1502ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:33.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f708c893640 1 --2- 172.21.15.67:0/3570096737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c43390 0x558e1502ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f708c893640 1 -- 172.21.15.67:0/3570096737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e15092a20 con 0x558e14c229f0 2024-07-31T14:55:33.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f708d094640 1 --2- 172.21.15.67:0/3570096737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c43390 0x558e1502ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:33.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f708c893640 1 --2- 172.21.15.67:0/3570096737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c229f0 0x558e1502c3f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f707c000c00 tx=0x7f707c0039b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b27e70c04c4ad72 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:33.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f7067fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:55:33.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f7067fff640 1 -- 172.21.15.67:0/3570096737 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c0aa760 con 0x558e14c229f0 2024-07-31T14:55:33.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f7095048740 1 -- 172.21.15.67:0/3570096737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7074030450 con 0x558e14c229f0 2024-07-31T14:55:33.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f7067fff640 1 -- 172.21.15.67:0/3570096737 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f707c0e7dc0 con 0x558e14c229f0 2024-07-31T14:55:33.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f7067fff640 1 -- 172.21.15.67:0/3570096737 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f707c0e1dd0 con 0x558e14c229f0 2024-07-31T14:55:33.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f7095048740 10 client.26375 fetch_fsmap learned FSMap version 39 2024-07-31T14:55:33.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.003+0000 7f7095048740 1 -- 172.21.15.67:0/3570096737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558e15093ca0 con 0x558e14c229f0 2024-07-31T14:55:33.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.004+0000 7f7067fff640 1 -- 172.21.15.67:0/3570096737 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7074030450 con 0x558e14c229f0 2024-07-31T14:55:33.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.004+0000 7f7067fff640 1 -- 172.21.15.67:0/3570096737 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7f707c0b3a30 con 0x558e14c229f0 2024-07-31T14:55:33.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.004+0000 7f7095048740 10 client.26375 fetch_fsmap finished waiting for FSMap version 39 2024-07-31T14:55:33.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.004+0000 7f7095048740 10 client.26375 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:55:33.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.004+0000 7f7095048740 20 client.26375 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:33.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.004+0000 7f7095048740 1 --2- 172.21.15.67:0/3570096737 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x558e14c64070 0x558e15096270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:33.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.004+0000 7f7095048740 4 client.26375 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:55:33.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.004+0000 7f7095048740 1 -- 172.21.15.67:0/3570096737 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e14f7d5a0 con 0x558e14c64070 2024-07-31T14:55:33.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.004+0000 7f7087fff640 1 --2- 172.21.15.67:0/3570096737 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x558e14c64070 0x558e15096270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:33.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7087fff640 1 --2- 172.21.15.67:0/3570096737 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x558e14c64070 0x558e15096270 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8823d11fae5fe8cb server_cookie=f6e7cec87dd62edb in_seq=0 out_seq=0 2024-07-31T14:55:33.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7067fff640 10 client.26375 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:55:33.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7067fff640 1 -- 172.21.15.67:0/3570096737 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e14f7d5a0 con 0x558e14c64070 2024-07-31T14:55:33.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7067fff640 10 client.26375 handle_command_reply: tid=0 2024-07-31T14:55:33.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7095048740 1 client.26375 shutdown 2024-07-31T14:55:33.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7095048740 1 -- 172.21.15.67:0/3570096737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c229f0 msgr2=0x558e1502c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:33.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7095048740 1 --2- 172.21.15.67:0/3570096737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c229f0 0x558e1502c3f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f707c000c00 tx=0x7f707c0039b0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7095048740 1 -- 172.21.15.67:0/3570096737 shutdown_connections 2024-07-31T14:55:33.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7095048740 1 --2- 172.21.15.67:0/3570096737 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x558e14c64070 0x558e15096270 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7095048740 1 --2- 172.21.15.67:0/3570096737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e14c43390 0x558e1502ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7095048740 1 --2- 172.21.15.67:0/3570096737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e14c3b1b0 0x558e1502c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7067fff640 0 client.26375 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:55:33.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7095048740 1 --2- 172.21.15.67:0/3570096737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e14c229f0 0x558e1502c3f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7095048740 1 -- 172.21.15.67:0/3570096737 >> 172.21.15.67:0/3570096737 conn(0x558e14c58460 msgr2=0x558e1500e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:33.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7095048740 1 -- 172.21.15.67:0/3570096737 shutdown_connections 2024-07-31T14:55:33.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7095048740 1 -- 172.21.15.67:0/3570096737 wait complete. 2024-07-31T14:55:33.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.005+0000 7f7095048740 20 client.26375 trim_cache size 0 max 16384 2024-07-31T14:55:33.013 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:55:33.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.006+0000 7f708eaf7640 1 -- 172.21.15.67:0/3274833560 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f70780b3dc0 msgr2=0x7f70780b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:33.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.006+0000 7f708eaf7640 1 --2- 172.21.15.67:0/3274833560 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f70780b3dc0 0x7f70780b62d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7080009fd0 tx=0x7f7080004ab0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.006+0000 7f708eaf7640 1 -- 172.21.15.67:0/3274833560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70881024b0 msgr2=0x7f7088199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:33.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.006+0000 7f708eaf7640 1 --2- 172.21.15.67:0/3274833560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70881024b0 0x7f7088199e50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f707c005fd0 tx=0x7f707c00ee50 comp rx=0 tx=0).stop 2024-07-31T14:55:33.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.006+0000 7f708eaf7640 1 -- 172.21.15.67:0/3274833560 shutdown_connections 2024-07-31T14:55:33.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.006+0000 7f708eaf7640 1 --2- 172.21.15.67:0/3274833560 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f70780b3dc0 0x7f70780b62d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.006+0000 7f708eaf7640 1 --2- 172.21.15.67:0/3274833560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7088108740 0x7f708819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.006+0000 7f708eaf7640 1 --2- 172.21.15.67:0/3274833560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7088107df0 0x7f708819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.006+0000 7f708eaf7640 1 --2- 172.21.15.67:0/3274833560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70881024b0 0x7f7088199e50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:33.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.006+0000 7f708eaf7640 1 -- 172.21.15.67:0/3274833560 >> 172.21.15.67:0/3274833560 conn(0x7f7088079170 msgr2=0x7f708806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:33.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.006+0000 7f708eaf7640 1 -- 172.21.15.67:0/3274833560 shutdown_connections 2024-07-31T14:55:33.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:33.006+0000 7f708eaf7640 1 -- 172.21.15.67:0/3274833560 wait complete. 2024-07-31T14:55:33.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:32 smithi077 ceph-mon[37327]: pgmap v534: 129 pgs: 129 active+clean; 658 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-07-31T14:55:33.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:32 smithi073 ceph-mon[36714]: pgmap v534: 129 pgs: 129 active+clean; 658 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-07-31T14:55:33.756 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:55:34.872 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T14:55:35.147 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:34 smithi067 ceph-mon[32750]: pgmap v535: 129 pgs: 129 active+clean; 660 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-07-31T14:55:35.148 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:34 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:55:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:34 smithi077 ceph-mon[37327]: pgmap v535: 129 pgs: 129 active+clean; 660 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-07-31T14:55:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:34 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:55:35.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:34 smithi073 ceph-mon[36714]: pgmap v535: 129 pgs: 129 active+clean; 660 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-07-31T14:55:35.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:34 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:55:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:55:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.781+0000 7fface8c9640 1 -- 172.21.15.67:0/3314454028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac810d7e0 msgr2=0x7ffac810fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.781+0000 7fface8c9640 1 --2- 172.21.15.67:0/3314454028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac810d7e0 0x7ffac810fcd0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009ca0 tx=0x7ffac002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.781+0000 7fface8c9640 1 -- 172.21.15.67:0/3314454028 shutdown_connections 2024-07-31T14:55:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.781+0000 7fface8c9640 1 --2- 172.21.15.67:0/3314454028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac810d7e0 0x7ffac810fcd0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.781+0000 7fface8c9640 1 --2- 172.21.15.67:0/3314454028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffac8104cd0 0x7ffac810d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.781+0000 7fface8c9640 1 --2- 172.21.15.67:0/3314454028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac8104380 0x7ffac8104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.781+0000 7fface8c9640 1 -- 172.21.15.67:0/3314454028 >> 172.21.15.67:0/3314454028 conn(0x7ffac807a190 msgr2=0x7ffac807a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.781+0000 7fface8c9640 1 -- 172.21.15.67:0/3314454028 shutdown_connections 2024-07-31T14:55:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.781+0000 7fface8c9640 1 -- 172.21.15.67:0/3314454028 wait complete. 2024-07-31T14:55:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.782+0000 7fface8c9640 1 Processor -- start 2024-07-31T14:55:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.782+0000 7fface8c9640 1 -- start start 2024-07-31T14:55:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.782+0000 7fface8c9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffac8104380 0x7ffac8199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.782+0000 7fface8c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac8104cd0 0x7ffac819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.782+0000 7fface8c9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac810d7e0 0x7ffac819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.782+0000 7fface8c9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81134c0 con 0x7ffac810d7e0 2024-07-31T14:55:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.782+0000 7fface8c9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8113340 con 0x7ffac8104380 2024-07-31T14:55:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.782+0000 7fface8c9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8113640 con 0x7ffac8104cd0 2024-07-31T14:55:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.782+0000 7ffacce66640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac810d7e0 0x7ffac819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.782+0000 7ffacce66640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac810d7e0 0x7ffac819f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45744/0 (socket says 172.21.15.67:45744) 2024-07-31T14:55:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.782+0000 7ffacce66640 1 -- 172.21.15.67:0/2563953211 learned_addr learned my addr 172.21.15.67:0/2563953211 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.783+0000 7ffac77fe640 1 --2- 172.21.15.67:0/2563953211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac8104cd0 0x7ffac819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.783+0000 7ffac7fff640 1 --2- 172.21.15.67:0/2563953211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffac8104380 0x7ffac8199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.783+0000 7ffacce66640 1 -- 172.21.15.67:0/2563953211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac8104cd0 msgr2=0x7ffac819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.783+0000 7ffacce66640 1 --2- 172.21.15.67:0/2563953211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac8104cd0 0x7ffac819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.783+0000 7ffacce66640 1 -- 172.21.15.67:0/2563953211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffac8104380 msgr2=0x7ffac8199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.783+0000 7ffacce66640 1 --2- 172.21.15.67:0/2563953211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffac8104380 0x7ffac8199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.783+0000 7ffacce66640 1 -- 172.21.15.67:0/2563953211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac819fdd0 con 0x7ffac810d7e0 2024-07-31T14:55:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.783+0000 7ffacce66640 1 --2- 172.21.15.67:0/2563953211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac810d7e0 0x7ffac819f6d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ffac00319f0 tx=0x7ffac0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.783+0000 7ffac57fa640 1 -- 172.21.15.67:0/2563953211 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0003bf0 con 0x7ffac810d7e0 2024-07-31T14:55:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.783+0000 7ffac57fa640 1 -- 172.21.15.67:0/2563953211 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac0003d90 con 0x7ffac810d7e0 2024-07-31T14:55:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.783+0000 7fface8c9640 1 -- 172.21.15.67:0/2563953211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac81a0060 con 0x7ffac810d7e0 2024-07-31T14:55:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.783+0000 7ffac57fa640 1 -- 172.21.15.67:0/2563953211 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac002fc00 con 0x7ffac810d7e0 2024-07-31T14:55:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.783+0000 7fface8c9640 1 -- 172.21.15.67:0/2563953211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac81a6850 con 0x7ffac810d7e0 2024-07-31T14:55:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.786+0000 7ffac57fa640 1 -- 172.21.15.67:0/2563953211 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ffac0004610 con 0x7ffac810d7e0 2024-07-31T14:55:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.786+0000 7ffad57c7740 1 Processor -- start 2024-07-31T14:55:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.786+0000 7ffad57c7740 1 -- start start 2024-07-31T14:55:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.786+0000 7ffad57c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 0x55ffd5700ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.786+0000 7ffad57c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd53352f0 0x55ffd5701220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.786+0000 7ffad57c7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd5701760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.786+0000 7ffad57c7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ffd56763b0 con 0x55ffd533d4d0 2024-07-31T14:55:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.786+0000 7ffad57c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ffd56658b0 con 0x55ffd535e1b0 2024-07-31T14:55:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.786+0000 7ffad57c7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ffd56698f0 con 0x55ffd53352f0 2024-07-31T14:55:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.786+0000 7ffac77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd5701760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.786+0000 7ffac77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd5701760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45750/0 (socket says 172.21.15.67:45750) 2024-07-31T14:55:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.786+0000 7ffac77fe640 1 -- 172.21.15.67:0/4044900036 learned_addr learned my addr 172.21.15.67:0/4044900036 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffac7fff640 1 --2- 172.21.15.67:0/4044900036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd53352f0 0x55ffd5701220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffac57fa640 1 --2- 172.21.15.67:0/2563953211 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffab00b3d40 0x7ffab00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffac77fe640 1 --2- 172.21.15.67:0/4044900036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 0x55ffd5700ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffacce66640 1 --2- 172.21.15.67:0/2563953211 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffab00b3d40 0x7ffab00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffac77fe640 1 -- 172.21.15.67:0/4044900036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd53352f0 msgr2=0x55ffd5701220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffac77fe640 1 --2- 172.21.15.67:0/4044900036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd53352f0 0x55ffd5701220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffac77fe640 1 -- 172.21.15.67:0/4044900036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 msgr2=0x55ffd5700ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffac77fe640 1 --2- 172.21.15.67:0/4044900036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 0x55ffd5700ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffac57fa640 1 -- 172.21.15.67:0/2563953211 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ffac0105910 con 0x7ffac810d7e0 2024-07-31T14:55:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffac77fe640 1 -- 172.21.15.67:0/4044900036 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd560c9a0 con 0x55ffd533d4d0 2024-07-31T14:55:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffac77fe640 1 --2- 172.21.15.67:0/4044900036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd5701760 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ffabc013940 tx=0x7ffabc00c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9830d44b7ac267fa server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffaae7fc640 1 -- 172.21.15.67:0/4044900036 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc0119e0 con 0x55ffd533d4d0 2024-07-31T14:55:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffacce66640 1 --2- 172.21.15.67:0/2563953211 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffab00b3d40 0x7ffab00b6250 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffac0103350 tx=0x7ffac002a8c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffaae7fc640 1 -- 172.21.15.67:0/4044900036 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffabc00f480 con 0x55ffd533d4d0 2024-07-31T14:55:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.787+0000 7ffaae7fc640 1 -- 172.21.15.67:0/4044900036 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc00e5e0 con 0x55ffd533d4d0 2024-07-31T14:55:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.788+0000 7ffad57c7740 1 -- 172.21.15.67:0/4044900036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 msgr2=0x55ffd5701760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.788+0000 7ffad57c7740 1 --2- 172.21.15.67:0/4044900036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd5701760 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ffabc013940 tx=0x7ffabc00c670 comp rx=0 tx=0).stop 2024-07-31T14:55:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.788+0000 7ffad57c7740 1 -- 172.21.15.67:0/4044900036 shutdown_connections 2024-07-31T14:55:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.788+0000 7ffad57c7740 1 --2- 172.21.15.67:0/4044900036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd5701760 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.788+0000 7ffad57c7740 1 --2- 172.21.15.67:0/4044900036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd53352f0 0x55ffd5701220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.788+0000 7ffad57c7740 1 --2- 172.21.15.67:0/4044900036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 0x55ffd5700ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.788+0000 7ffad57c7740 1 -- 172.21.15.67:0/4044900036 >> 172.21.15.67:0/4044900036 conn(0x55ffd5352300 msgr2=0x55ffd56febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.788+0000 7ffad57c7740 1 -- 172.21.15.67:0/4044900036 shutdown_connections 2024-07-31T14:55:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.788+0000 7ffad57c7740 1 -- 172.21.15.67:0/4044900036 wait complete. 2024-07-31T14:55:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.789+0000 7ffad57c7740 1 -- start start 2024-07-31T14:55:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.789+0000 7ffad57c7740 1 Processor -- start 2024-07-31T14:55:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.789+0000 7ffad57c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd53352f0 0x55ffd570c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.789+0000 7ffad57c7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd570cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.789+0000 7ffad57c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 0x55ffd57187b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.789+0000 7ffad57c7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ffd559fab0 con 0x55ffd533d4d0 2024-07-31T14:55:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.789+0000 7ffad57c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ffd55edcf0 con 0x55ffd535e1b0 2024-07-31T14:55:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.789+0000 7ffad57c7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ffd570f460 con 0x55ffd53352f0 2024-07-31T14:55:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.789+0000 7ffad57c7740 10 client.0 _ll_register_callbacks cb 0x55ffd565f9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.789+0000 7ffac7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd570cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.789+0000 7ffac7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd570cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45760/0 (socket says 172.21.15.67:45760) 2024-07-31T14:55:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.789+0000 7ffac7fff640 1 -- 172.21.15.67:0/4136182818 learned_addr learned my addr 172.21.15.67:0/4136182818 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffac77fe640 1 --2- 172.21.15.67:0/4136182818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 0x55ffd57187b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffac77fe640 1 --2- 172.21.15.67:0/4136182818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd53352f0 0x55ffd570c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffac7fff640 1 -- 172.21.15.67:0/4136182818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd53352f0 msgr2=0x55ffd570c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffac7fff640 1 --2- 172.21.15.67:0/4136182818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd53352f0 0x55ffd570c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffac7fff640 1 -- 172.21.15.67:0/4136182818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 msgr2=0x55ffd57187b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffac7fff640 1 --2- 172.21.15.67:0/4136182818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 0x55ffd57187b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffac7fff640 1 -- 172.21.15.67:0/4136182818 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd53c9790 con 0x55ffd533d4d0 2024-07-31T14:55:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffac77fe640 1 --2- 172.21.15.67:0/4136182818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 0x55ffd57187b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:55:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffac7fff640 1 --2- 172.21.15.67:0/4136182818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd570cd20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ffab80104d0 tx=0x7ffab8010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc441a2e451931d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffaa77fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:55:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffaa77fe640 1 -- 172.21.15.67:0/4136182818 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab801a020 con 0x55ffd533d4d0 2024-07-31T14:55:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffaa77fe640 1 -- 172.21.15.67:0/4136182818 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ffab8016690 con 0x55ffd533d4d0 2024-07-31T14:55:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffad57c7740 1 -- 172.21.15.67:0/4136182818 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffabc00f480 con 0x55ffd533d4d0 2024-07-31T14:55:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffaa77fe640 1 -- 172.21.15.67:0/4136182818 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ffab8025050 con 0x55ffd533d4d0 2024-07-31T14:55:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffad57c7740 10 client.16791 fetch_fsmap learned FSMap version 41 2024-07-31T14:55:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.790+0000 7ffad57c7740 1 -- 172.21.15.67:0/4136182818 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ffd5442620 con 0x55ffd533d4d0 2024-07-31T14:55:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.791+0000 7ffaa77fe640 1 -- 172.21.15.67:0/4136182818 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffabc00f480 con 0x55ffd533d4d0 2024-07-31T14:55:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.791+0000 7ffaa77fe640 1 -- 172.21.15.67:0/4136182818 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) ==== 8418+0+0 (secure 0 0 0) 0x7ffab80149c0 con 0x55ffd533d4d0 2024-07-31T14:55:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.791+0000 7ffad57c7740 10 client.16791 fetch_fsmap finished waiting for FSMap version 41 2024-07-31T14:55:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.791+0000 7ffad57c7740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.791+0000 7ffad57c7740 20 client.16791 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.791+0000 7ffad57c7740 1 --2- 172.21.15.67:0/4136182818 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ffd531cd80 0x55ffd571e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.791+0000 7ffad57c7740 4 client.16791 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:55:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.791+0000 7ffad57c7740 1 -- 172.21.15.67:0/4136182818 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffd5678300 con 0x55ffd531cd80 2024-07-31T14:55:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.791+0000 7ffac77fe640 1 --2- 172.21.15.67:0/4136182818 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ffd531cd80 0x55ffd571e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.792+0000 7ffaa77fe640 10 client.16791 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.792+0000 7ffac77fe640 1 --2- 172.21.15.67:0/4136182818 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ffd531cd80 0x55ffd571e910 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67810d82f539a348 server_cookie=1c246ed6c740af6 in_seq=0 out_seq=0 2024-07-31T14:55:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.792+0000 7ffaa77fe640 1 -- 172.21.15.67:0/4136182818 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffd5678300 con 0x55ffd531cd80 2024-07-31T14:55:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.792+0000 7ffaa77fe640 10 client.16791 handle_command_reply: tid=0 2024-07-31T14:55:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.792+0000 7ffad57c7740 1 client.16791 shutdown 2024-07-31T14:55:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.793+0000 7ffad57c7740 1 -- 172.21.15.67:0/4136182818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 msgr2=0x55ffd570cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.793+0000 7ffad57c7740 1 --2- 172.21.15.67:0/4136182818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd570cd20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ffab80104d0 tx=0x7ffab8010990 comp rx=0 tx=0).stop 2024-07-31T14:55:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.793+0000 7ffad57c7740 1 -- 172.21.15.67:0/4136182818 shutdown_connections 2024-07-31T14:55:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.793+0000 7ffad57c7740 1 --2- 172.21.15.67:0/4136182818 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ffd531cd80 0x55ffd571e910 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.793+0000 7ffad57c7740 1 --2- 172.21.15.67:0/4136182818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 0x55ffd57187b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.793+0000 7ffad57c7740 1 --2- 172.21.15.67:0/4136182818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd570cd20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.793+0000 7ffaa77fe640 0 client.16791 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.793+0000 7ffad57c7740 1 --2- 172.21.15.67:0/4136182818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd53352f0 0x55ffd570c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.793+0000 7ffad57c7740 1 -- 172.21.15.67:0/4136182818 >> 172.21.15.67:0/4136182818 conn(0x55ffd5352300 msgr2=0x55ffd5701d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.793+0000 7ffad57c7740 1 -- 172.21.15.67:0/4136182818 shutdown_connections 2024-07-31T14:55:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.793+0000 7ffad57c7740 1 -- 172.21.15.67:0/4136182818 wait complete. 2024-07-31T14:55:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.793+0000 7ffad57c7740 20 client.16791 trim_cache size 0 max 16384 2024-07-31T14:55:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.808+0000 7ffad57c7740 1 Processor -- start 2024-07-31T14:55:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.808+0000 7ffad57c7740 1 -- start start 2024-07-31T14:55:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.808+0000 7ffad57c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd53352f0 0x55ffd56fec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.808+0000 7ffad57c7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd5706db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.808+0000 7ffad57c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd535e1b0 0x55ffd570be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.808+0000 7ffad57c7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81b0710 con 0x55ffd533d4d0 2024-07-31T14:55:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.808+0000 7ffad57c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81b23d0 con 0x55ffd53352f0 2024-07-31T14:55:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.808+0000 7ffad57c7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81b0de0 con 0x55ffd535e1b0 2024-07-31T14:55:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.808+0000 7ffac7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd5706db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.808+0000 7ffac7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd5706db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45774/0 (socket says 172.21.15.67:45774) 2024-07-31T14:55:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.808+0000 7ffac7fff640 1 -- 172.21.15.67:0/3833659073 learned_addr learned my addr 172.21.15.67:0/3833659073 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.809+0000 7ffac77fe640 1 --2- 172.21.15.67:0/3833659073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd535e1b0 0x55ffd570be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.809+0000 7ffac7fff640 1 -- 172.21.15.67:0/3833659073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd535e1b0 msgr2=0x55ffd570be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.809+0000 7ffac7fff640 1 --2- 172.21.15.67:0/3833659073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd535e1b0 0x55ffd570be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.809+0000 7ffac7fff640 1 -- 172.21.15.67:0/3833659073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd53352f0 msgr2=0x55ffd56fec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.809+0000 7ffac7fff640 1 --2- 172.21.15.67:0/3833659073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd53352f0 0x55ffd56fec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.809+0000 7ffac7fff640 1 -- 172.21.15.67:0/3833659073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd5707d40 con 0x55ffd533d4d0 2024-07-31T14:55:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.809+0000 7ffac77fe640 1 --2- 172.21.15.67:0/3833659073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd535e1b0 0x55ffd570be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:55:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.809+0000 7ffac7fff640 1 --2- 172.21.15.67:0/3833659073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd5706db0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ffab800bad0 tx=0x7ffab8006b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f928b66ddd8faf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.809+0000 7ffaadffb640 1 -- 172.21.15.67:0/3833659073 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab801a070 con 0x55ffd533d4d0 2024-07-31T14:55:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.809+0000 7ffaadffb640 1 -- 172.21.15.67:0/3833659073 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffab800bd80 con 0x55ffd533d4d0 2024-07-31T14:55:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.809+0000 7ffaadffb640 1 -- 172.21.15.67:0/3833659073 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab8014780 con 0x55ffd533d4d0 2024-07-31T14:55:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.810+0000 7ffad57c7740 1 -- 172.21.15.67:0/3833659073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 msgr2=0x55ffd5706db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.810+0000 7ffad57c7740 1 --2- 172.21.15.67:0/3833659073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd5706db0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ffab800bad0 tx=0x7ffab8006b00 comp rx=0 tx=0).stop 2024-07-31T14:55:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.810+0000 7ffad57c7740 1 -- 172.21.15.67:0/3833659073 shutdown_connections 2024-07-31T14:55:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.810+0000 7ffad57c7740 1 --2- 172.21.15.67:0/3833659073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd535e1b0 0x55ffd570be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.810+0000 7ffad57c7740 1 --2- 172.21.15.67:0/3833659073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd533d4d0 0x55ffd5706db0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.810+0000 7ffad57c7740 1 --2- 172.21.15.67:0/3833659073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd53352f0 0x55ffd56fec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.810+0000 7ffad57c7740 1 -- 172.21.15.67:0/3833659073 >> 172.21.15.67:0/3833659073 conn(0x55ffd5352300 msgr2=0x55ffd56fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.810+0000 7ffad57c7740 1 -- 172.21.15.67:0/3833659073 shutdown_connections 2024-07-31T14:55:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.810+0000 7ffad57c7740 1 -- 172.21.15.67:0/3833659073 wait complete. 2024-07-31T14:55:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.810+0000 7ffad57c7740 1 -- start start 2024-07-31T14:55:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.810+0000 7ffad57c7740 1 Processor -- start 2024-07-31T14:55:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffad57c7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd53352f0 0x55ffd577e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffad57c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd533d4d0 0x55ffd577e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffad57c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 0x55ffd5710d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffad57c7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81b1d00 con 0x55ffd53352f0 2024-07-31T14:55:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffad57c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81b1570 con 0x55ffd535e1b0 2024-07-31T14:55:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffad57c7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81ae6f0 con 0x55ffd533d4d0 2024-07-31T14:55:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffad57c7740 10 client.0 _ll_register_callbacks cb 0x7ffab800b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffac77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd53352f0 0x55ffd577e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffac77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd53352f0 0x55ffd577e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45780/0 (socket says 172.21.15.67:45780) 2024-07-31T14:55:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffac77fe640 1 -- 172.21.15.67:0/3663601895 learned_addr learned my addr 172.21.15.67:0/3663601895 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffac7fff640 1 --2- 172.21.15.67:0/3663601895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd533d4d0 0x55ffd577e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffac77fe640 1 -- 172.21.15.67:0/3663601895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd533d4d0 msgr2=0x55ffd577e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffac77fe640 1 --2- 172.21.15.67:0/3663601895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd533d4d0 0x55ffd577e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffac77fe640 1 -- 172.21.15.67:0/3663601895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 msgr2=0x55ffd5710d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffac77fe640 1 --2- 172.21.15.67:0/3663601895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 0x55ffd5710d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffac77fe640 1 -- 172.21.15.67:0/3663601895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd57114f0 con 0x55ffd53352f0 2024-07-31T14:55:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffac7fff640 1 --2- 172.21.15.67:0/3663601895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd533d4d0 0x55ffd577e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:55:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffac77fe640 1 --2- 172.21.15.67:0/3663601895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd53352f0 0x55ffd577e3d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ffabc013910 tx=0x7ffabc005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442105351736eccd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffaa77fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:55:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.811+0000 7ffaa77fe640 1 -- 172.21.15.67:0/3663601895 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc031070 con 0x55ffd53352f0 2024-07-31T14:55:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.812+0000 7ffaa77fe640 1 -- 172.21.15.67:0/3663601895 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ffabc00f890 con 0x55ffd53352f0 2024-07-31T14:55:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.812+0000 7ffad57c7740 1 -- 172.21.15.67:0/3663601895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffab800bd80 con 0x55ffd53352f0 2024-07-31T14:55:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.812+0000 7ffaa77fe640 1 -- 172.21.15.67:0/3663601895 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ffabc004030 con 0x55ffd53352f0 2024-07-31T14:55:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.812+0000 7ffad57c7740 10 client.16797 fetch_fsmap learned FSMap version 41 2024-07-31T14:55:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.812+0000 7ffad57c7740 1 -- 172.21.15.67:0/3663601895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ffd5785b70 con 0x55ffd53352f0 2024-07-31T14:55:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.812+0000 7ffaa77fe640 1 -- 172.21.15.67:0/3663601895 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffab800bd80 con 0x55ffd53352f0 2024-07-31T14:55:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.812+0000 7ffaa77fe640 1 -- 172.21.15.67:0/3663601895 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) ==== 8418+0+0 (secure 0 0 0) 0x7ffabc034020 con 0x55ffd53352f0 2024-07-31T14:55:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.812+0000 7ffad57c7740 10 client.16797 fetch_fsmap finished waiting for FSMap version 41 2024-07-31T14:55:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.812+0000 7ffad57c7740 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.812+0000 7ffad57c7740 20 client.16797 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.812+0000 7ffad57c7740 1 --2- 172.21.15.67:0/3663601895 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ffd52f21c0 0x55ffd5788270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.812+0000 7ffad57c7740 4 client.16797 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:55:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.812+0000 7ffad57c7740 1 -- 172.21.15.67:0/3663601895 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ffd5673da0 con 0x55ffd52f21c0 2024-07-31T14:55:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.813+0000 7ffac7fff640 1 --2- 172.21.15.67:0/3663601895 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ffd52f21c0 0x55ffd5788270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.813+0000 7ffaa77fe640 10 client.16797 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:35.813+0000 7ffac7fff640 1 --2- 172.21.15.67:0/3663601895 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ffd52f21c0 0x55ffd5788270 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95b6fa67e90c4e1 server_cookie=4a706ca28c17e16d in_seq=0 out_seq=0 2024-07-31T14:55:36.029 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:35 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:55:36.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:35 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:55:36.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:35 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:55:37.176 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:36 smithi067 ceph-mon[32750]: pgmap v536: 129 pgs: 129 active+clean; 727 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 22 MiB/s wr, 1.00k op/s 2024-07-31T14:55:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:36 smithi077 ceph-mon[37327]: pgmap v536: 129 pgs: 129 active+clean; 727 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 22 MiB/s wr, 1.00k op/s 2024-07-31T14:55:37.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:36 smithi073 ceph-mon[36714]: pgmap v536: 129 pgs: 129 active+clean; 727 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 22 MiB/s wr, 1.00k op/s 2024-07-31T14:55:38.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.009+0000 7ff5be9db640 1 -- 172.21.15.67:0/27423130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5b810a740 msgr2=0x7ff5b811c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.009+0000 7ff5be9db640 1 --2- 172.21.15.67:0/27423130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5b810a740 0x7ff5b811c5d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009860 tx=0x7ff5ac02a360 comp rx=0 tx=0).stop 2024-07-31T14:55:38.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.013+0000 7ff5be9db640 1 -- 172.21.15.67:0/27423130 shutdown_connections 2024-07-31T14:55:38.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.013+0000 7ff5be9db640 1 --2- 172.21.15.67:0/27423130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b811cb10 0x7ff5b811ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.013+0000 7ff5be9db640 1 --2- 172.21.15.67:0/27423130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5b810a740 0x7ff5b811c5d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.013+0000 7ff5be9db640 1 --2- 172.21.15.67:0/27423130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5b8109df0 0x7ff5b810a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.013+0000 7ff5be9db640 1 -- 172.21.15.67:0/27423130 >> 172.21.15.67:0/27423130 conn(0x7ff5b80708b0 msgr2=0x7ff5b8070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:38.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.015+0000 7ff5be9db640 1 -- 172.21.15.67:0/27423130 shutdown_connections 2024-07-31T14:55:38.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.015+0000 7ff5be9db640 1 -- 172.21.15.67:0/27423130 wait complete. 2024-07-31T14:55:38.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.016+0000 7ff5be9db640 1 Processor -- start 2024-07-31T14:55:38.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.016+0000 7ff5be9db640 1 -- start start 2024-07-31T14:55:38.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.016+0000 7ff5be9db640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b8109df0 0x7ff5b81a8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.016+0000 7ff5be9db640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5b810a740 0x7ff5b81a9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.016+0000 7ff5be9db640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5b811cb10 0x7ff5b81ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.016+0000 7ff5be9db640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b8122380 con 0x7ff5b810a740 2024-07-31T14:55:38.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.016+0000 7ff5be9db640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b8122200 con 0x7ff5b8109df0 2024-07-31T14:55:38.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.016+0000 7ff5be9db640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b8122500 con 0x7ff5b811cb10 2024-07-31T14:55:38.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.018+0000 7ff5b77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5b810a740 0x7ff5b81a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.018+0000 7ff5b77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5b810a740 0x7ff5b81a9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45786/0 (socket says 172.21.15.67:45786) 2024-07-31T14:55:38.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.018+0000 7ff5b77fe640 1 -- 172.21.15.67:0/2483300900 learned_addr learned my addr 172.21.15.67:0/2483300900 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:38.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.018+0000 7ff5bcf78640 1 --2- 172.21.15.67:0/2483300900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5b811cb10 0x7ff5b81ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.020+0000 7ff5b77fe640 1 -- 172.21.15.67:0/2483300900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5b811cb10 msgr2=0x7ff5b81ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.020+0000 7ff5b77fe640 1 --2- 172.21.15.67:0/2483300900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5b811cb10 0x7ff5b81ae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.020+0000 7ff5b77fe640 1 -- 172.21.15.67:0/2483300900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b8109df0 msgr2=0x7ff5b81a8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.020+0000 7ff5b77fe640 1 --2- 172.21.15.67:0/2483300900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b8109df0 0x7ff5b81a8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.020+0000 7ff5b77fe640 1 -- 172.21.15.67:0/2483300900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b81aed20 con 0x7ff5b810a740 2024-07-31T14:55:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.020+0000 7ff5b77fe640 1 --2- 172.21.15.67:0/2483300900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5b810a740 0x7ff5b81a9350 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac035040 tx=0x7ff5ac032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.021+0000 7ff5b57fa640 1 -- 172.21.15.67:0/2483300900 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac040020 con 0x7ff5b810a740 2024-07-31T14:55:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.021+0000 7ff5b57fa640 1 -- 172.21.15.67:0/2483300900 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5ac038040 con 0x7ff5b810a740 2024-07-31T14:55:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.021+0000 7ff5b57fa640 1 -- 172.21.15.67:0/2483300900 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac037070 con 0x7ff5b810a740 2024-07-31T14:55:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.021+0000 7ff5be9db640 1 -- 172.21.15.67:0/2483300900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b81aefb0 con 0x7ff5b810a740 2024-07-31T14:55:38.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.022+0000 7ff5be9db640 1 -- 172.21.15.67:0/2483300900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b81b5770 con 0x7ff5b810a740 2024-07-31T14:55:38.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.028+0000 7ff5c4f2c740 1 Processor -- start 2024-07-31T14:55:38.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.029+0000 7ff5b57fa640 1 -- 172.21.15.67:0/2483300900 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff5ac043020 con 0x7ff5b810a740 2024-07-31T14:55:38.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.032+0000 7ff5c4f2c740 1 -- start start 2024-07-31T14:55:38.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.032+0000 7ff5b57fa640 1 --2- 172.21.15.67:0/2483300900 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5a00b3f50 0x7ff5a00b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.032+0000 7ff5bcf78640 1 --2- 172.21.15.67:0/2483300900 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5a00b3f50 0x7ff5a00b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.032+0000 7ff5b57fa640 1 -- 172.21.15.67:0/2483300900 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff5ac1040a0 con 0x7ff5b810a740 2024-07-31T14:55:38.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.032+0000 7ff5c4f2c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f2b840 0x55c9c92d1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.032+0000 7ff5c4f2c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f02980 0x55c9c92d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.032+0000 7ff5c4f2c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f0ab60 0x55c9c92d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.032+0000 7ff5c4f2c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c9c92467b0 con 0x55c9c8f2b840 2024-07-31T14:55:38.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.032+0000 7ff5c4f2c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c9c9235cb0 con 0x55c9c8f02980 2024-07-31T14:55:38.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.032+0000 7ff5c4f2c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c9c9239cf0 con 0x55c9c8f0ab60 2024-07-31T14:55:38.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.033+0000 7ff5bcf78640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f0ab60 0x55c9c92d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.033+0000 7ff5b77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f02980 0x55c9c92d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.033+0000 7ff5bcf78640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f0ab60 0x55c9c92d22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37924/0 (socket says 172.21.15.67:37924) 2024-07-31T14:55:38.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.033+0000 7ff5bcf78640 1 -- 172.21.15.67:0/1258650455 learned_addr learned my addr 172.21.15.67:0/1258650455 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:38.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.033+0000 7ff5b7fff640 1 --2- 172.21.15.67:0/1258650455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f2b840 0x55c9c92d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.034+0000 7ff5b77fe640 1 -- 172.21.15.67:0/1258650455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f0ab60 msgr2=0x55c9c92d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.034+0000 7ff5b77fe640 1 --2- 172.21.15.67:0/1258650455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f0ab60 0x55c9c92d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.034+0000 7ff5b77fe640 1 -- 172.21.15.67:0/1258650455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f2b840 msgr2=0x55c9c92d1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.034+0000 7ff5b77fe640 1 --2- 172.21.15.67:0/1258650455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f2b840 0x55c9c92d1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.035+0000 7ff5b77fe640 1 -- 172.21.15.67:0/1258650455 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9c91cb0b0 con 0x55c9c8f02980 2024-07-31T14:55:38.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.035+0000 7ff5b77fe640 1 --2- 172.21.15.67:0/1258650455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f02980 0x55c9c92d1db0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac101590 tx=0x7ff5ac004460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6083475693758dbf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:38.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.036+0000 7ff5bcf78640 1 --2- 172.21.15.67:0/2483300900 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5a00b3f50 0x7ff5a00b6460 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0005fd0 tx=0x7ff5b000f820 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:38.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.036+0000 7ff59e7fc640 1 -- 172.21.15.67:0/1258650455 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0c6040 con 0x55c9c8f02980 2024-07-31T14:55:38.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.036+0000 7ff59e7fc640 1 -- 172.21.15.67:0/1258650455 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5ac0328f0 con 0x55c9c8f02980 2024-07-31T14:55:38.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.037+0000 7ff59e7fc640 1 -- 172.21.15.67:0/1258650455 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0cf050 con 0x55c9c8f02980 2024-07-31T14:55:38.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.038+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/1258650455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f02980 msgr2=0x55c9c92d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.038+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/1258650455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f02980 0x55c9c92d1db0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac101590 tx=0x7ff5ac004460 comp rx=0 tx=0).stop 2024-07-31T14:55:38.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.038+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/1258650455 shutdown_connections 2024-07-31T14:55:38.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.038+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/1258650455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f0ab60 0x55c9c92d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.038+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/1258650455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f02980 0x55c9c92d1db0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.038+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/1258650455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f2b840 0x55c9c92d1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.038+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/1258650455 >> 172.21.15.67:0/1258650455 conn(0x55c9c8f1f990 msgr2=0x55c9c92cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:38.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.038+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/1258650455 shutdown_connections 2024-07-31T14:55:38.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.038+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/1258650455 wait complete. 2024-07-31T14:55:38.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.039+0000 7ff5c4f2c740 1 -- start start 2024-07-31T14:55:38.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.040+0000 7ff5c4f2c740 1 Processor -- start 2024-07-31T14:55:38.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.041+0000 7ff5c4f2c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f02980 0x55c9c92dade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.041+0000 7ff5c4f2c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f0ab60 0x55c9c92db320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.041+0000 7ff5b7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f02980 0x55c9c92dade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.041+0000 7ff5b7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f02980 0x55c9c92dade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37936/0 (socket says 172.21.15.67:37936) 2024-07-31T14:55:38.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.041+0000 7ff5b7fff640 1 -- 172.21.15.67:0/2678170540 learned_addr learned my addr 172.21.15.67:0/2678170540 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:38.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.041+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/2678170540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f2b840 0x55c9c92db860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.041+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/2678170540 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c9c90997a0 con 0x55c9c8f0ab60 2024-07-31T14:55:38.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.042+0000 7ff5bcf78640 1 --2- 172.21.15.67:0/2678170540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f2b840 0x55c9c92db860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.042+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/2678170540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c9c91d8320 con 0x55c9c8f2b840 2024-07-31T14:55:38.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.042+0000 7ff5b77fe640 1 --2- 172.21.15.67:0/2678170540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f0ab60 0x55c9c92db320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.042+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/2678170540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c9c9235cb0 con 0x55c9c8f02980 2024-07-31T14:55:38.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.042+0000 7ff5c4f2c740 10 client.0 _ll_register_callbacks cb 0x55c9c91e2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:38.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.042+0000 7ff5b7fff640 1 -- 172.21.15.67:0/2678170540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f2b840 msgr2=0x55c9c92db860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.042+0000 7ff5b7fff640 1 --2- 172.21.15.67:0/2678170540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f2b840 0x55c9c92db860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.042+0000 7ff5b7fff640 1 -- 172.21.15.67:0/2678170540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f0ab60 msgr2=0x55c9c92db320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.042+0000 7ff5b7fff640 1 --2- 172.21.15.67:0/2678170540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f0ab60 0x55c9c92db320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.042+0000 7ff5b7fff640 1 -- 172.21.15.67:0/2678170540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9c91dc4b0 con 0x55c9c8f02980 2024-07-31T14:55:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.043+0000 7ff5b7fff640 1 --2- 172.21.15.67:0/2678170540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f02980 0x55c9c92dade0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800d210 tx=0x7ff5a800bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17ad86b2a44132b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.043+0000 7ff5977fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:55:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.043+0000 7ff5977fe640 1 -- 172.21.15.67:0/2678170540 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8012630 con 0x55c9c8f02980 2024-07-31T14:55:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.043+0000 7ff5977fe640 1 -- 172.21.15.67:0/2678170540 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff5a80127d0 con 0x55c9c8f02980 2024-07-31T14:55:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.043+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/2678170540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5ac0328f0 con 0x55c9c8f02980 2024-07-31T14:55:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.043+0000 7ff5977fe640 1 -- 172.21.15.67:0/2678170540 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff5a8021050 con 0x55c9c8f02980 2024-07-31T14:55:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.043+0000 7ff5c4f2c740 10 client.26425 fetch_fsmap learned FSMap version 41 2024-07-31T14:55:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.043+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/2678170540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c9c8fa5b90 con 0x55c9c8f02980 2024-07-31T14:55:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.044+0000 7ff5977fe640 1 -- 172.21.15.67:0/2678170540 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5a80143f0 con 0x55c9c8f02980 2024-07-31T14:55:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.044+0000 7ff5977fe640 1 -- 172.21.15.67:0/2678170540 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) ==== 8418+0+0 (secure 0 0 0) 0x7ff5a801c030 con 0x55c9c8f02980 2024-07-31T14:55:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.044+0000 7ff5c4f2c740 10 client.26425 fetch_fsmap finished waiting for FSMap version 41 2024-07-31T14:55:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.044+0000 7ff5c4f2c740 10 client.26425 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:55:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.044+0000 7ff5c4f2c740 20 client.26425 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.044+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/2678170540 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c9c8eea1c0 0x55c9c92ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.044+0000 7ff5c4f2c740 4 client.26425 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:55:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.044+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/2678170540 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff5ac0328f0 con 0x55c9c8eea1c0 2024-07-31T14:55:38.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.044+0000 7ff5b77fe640 1 --2- 172.21.15.67:0/2678170540 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c9c8eea1c0 0x55c9c92ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.045+0000 7ff5b77fe640 1 --2- 172.21.15.67:0/2678170540 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c9c8eea1c0 0x55c9c92ffd40 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae896a3c45182990 server_cookie=f98f7eb89156dcf4 in_seq=0 out_seq=0 2024-07-31T14:55:38.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.045+0000 7ff5977fe640 10 client.26425 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:38.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.046+0000 7ff5977fe640 1 -- 172.21.15.67:0/2678170540 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff5ac0328f0 con 0x55c9c8eea1c0 2024-07-31T14:55:38.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.046+0000 7ff5977fe640 10 client.26425 handle_command_reply: tid=0 2024-07-31T14:55:38.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.046+0000 7ff5c4f2c740 1 client.26425 shutdown 2024-07-31T14:55:38.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.046+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/2678170540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f02980 msgr2=0x55c9c92dade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.046+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/2678170540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f02980 0x55c9c92dade0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800d210 tx=0x7ff5a800bf90 comp rx=0 tx=0).stop 2024-07-31T14:55:38.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.046+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/2678170540 shutdown_connections 2024-07-31T14:55:38.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.046+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/2678170540 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c9c8eea1c0 0x55c9c92ffd40 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.046+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/2678170540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f2b840 0x55c9c92db860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.046+0000 7ff5977fe640 0 client.26425 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.046+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/2678170540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f0ab60 0x55c9c92db320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.046+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/2678170540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f02980 0x55c9c92dade0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.046+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/2678170540 >> 172.21.15.67:0/2678170540 conn(0x55c9c8f1f990 msgr2=0x55c9c92d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.047+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/2678170540 shutdown_connections 2024-07-31T14:55:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.047+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/2678170540 wait complete. 2024-07-31T14:55:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.047+0000 7ff5c4f2c740 20 client.26425 trim_cache size 0 max 16384 2024-07-31T14:55:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.049+0000 7ffaa77fe640 1 -- 172.21.15.67:0/3663601895 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ffd5673da0 con 0x55ffd52f21c0 2024-07-31T14:55:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.049+0000 7ffaa77fe640 10 client.16797 handle_command_reply: tid=0 2024-07-31T14:55:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.049+0000 7ffad57c7740 1 client.16797 shutdown 2024-07-31T14:55:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.049+0000 7ffad57c7740 1 -- 172.21.15.67:0/3663601895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd53352f0 msgr2=0x55ffd577e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.049+0000 7ffad57c7740 1 --2- 172.21.15.67:0/3663601895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd53352f0 0x55ffd577e3d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ffabc013910 tx=0x7ffabc005c50 comp rx=0 tx=0).stop 2024-07-31T14:55:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.050+0000 7ffad57c7740 1 -- 172.21.15.67:0/3663601895 shutdown_connections 2024-07-31T14:55:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.050+0000 7ffad57c7740 1 --2- 172.21.15.67:0/3663601895 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ffd52f21c0 0x55ffd5788270 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.050+0000 7ffad57c7740 1 --2- 172.21.15.67:0/3663601895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffd535e1b0 0x55ffd5710d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.050+0000 7ffad57c7740 1 --2- 172.21.15.67:0/3663601895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd533d4d0 0x55ffd577e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.050+0000 7ffad57c7740 1 --2- 172.21.15.67:0/3663601895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffd53352f0 0x55ffd577e3d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.050+0000 7ffad57c7740 1 -- 172.21.15.67:0/3663601895 >> 172.21.15.67:0/3663601895 conn(0x55ffd5352300 msgr2=0x55ffd570e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.050+0000 7ffaa77fe640 0 client.16797 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.050+0000 7ffad57c7740 1 -- 172.21.15.67:0/3663601895 shutdown_connections 2024-07-31T14:55:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.050+0000 7ffad57c7740 1 -- 172.21.15.67:0/3663601895 wait complete. 2024-07-31T14:55:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.050+0000 7ffad57c7740 20 client.16797 trim_cache size 0 max 16384 2024-07-31T14:55:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:55:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:55:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:55:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:55:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.051+0000 7fface8c9640 1 -- 172.21.15.67:0/2563953211 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffab00b3d40 msgr2=0x7ffab00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.051+0000 7fface8c9640 1 --2- 172.21.15.67:0/2563953211 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffab00b3d40 0x7ffab00b6250 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffac0103350 tx=0x7ffac002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.051+0000 7fface8c9640 1 -- 172.21.15.67:0/2563953211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac810d7e0 msgr2=0x7ffac819f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.051+0000 7fface8c9640 1 --2- 172.21.15.67:0/2563953211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac810d7e0 0x7ffac819f6d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ffac00319f0 tx=0x7ffac0031a20 comp rx=0 tx=0).stop 2024-07-31T14:55:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.051+0000 7fface8c9640 1 -- 172.21.15.67:0/2563953211 shutdown_connections 2024-07-31T14:55:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.051+0000 7fface8c9640 1 --2- 172.21.15.67:0/2563953211 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffab00b3d40 0x7ffab00b6250 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.051+0000 7fface8c9640 1 --2- 172.21.15.67:0/2563953211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac810d7e0 0x7ffac819f6d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.051+0000 7fface8c9640 1 --2- 172.21.15.67:0/2563953211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac8104cd0 0x7ffac819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.051+0000 7fface8c9640 1 --2- 172.21.15.67:0/2563953211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffac8104380 0x7ffac8199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.051+0000 7fface8c9640 1 -- 172.21.15.67:0/2563953211 >> 172.21.15.67:0/2563953211 conn(0x7ffac807a190 msgr2=0x7ffac81051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.051+0000 7fface8c9640 1 -- 172.21.15.67:0/2563953211 shutdown_connections 2024-07-31T14:55:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:38.051+0000 7fface8c9640 1 -- 172.21.15.67:0/2563953211 wait complete. 2024-07-31T14:55:38.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.073+0000 7ff5c4f2c740 1 Processor -- start 2024-07-31T14:55:38.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.073+0000 7ff5c4f2c740 1 -- start start 2024-07-31T14:55:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.073+0000 7ff5c4f2c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8eea1c0 0x55c9c92ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.073+0000 7ff5c4f2c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f02980 0x55c9c92efc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.073+0000 7ff5c4f2c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f0ab60 0x55c9c92f9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.073+0000 7ff5c4f2c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81b6d70 con 0x55c9c8f0ab60 2024-07-31T14:55:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.073+0000 7ff5c4f2c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81b8a30 con 0x55c9c8eea1c0 2024-07-31T14:55:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.073+0000 7ff5c4f2c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81b7440 con 0x55c9c8f02980 2024-07-31T14:55:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.073+0000 7ff5b7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8eea1c0 0x55c9c92ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.073+0000 7ff5bcf78640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f0ab60 0x55c9c92f9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.073+0000 7ff5b7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8eea1c0 0x55c9c92ceb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51244/0 (socket says 172.21.15.67:51244) 2024-07-31T14:55:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.073+0000 7ff5b7fff640 1 -- 172.21.15.67:0/3407594996 learned_addr learned my addr 172.21.15.67:0/3407594996 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.074+0000 7ff5b77fe640 1 --2- 172.21.15.67:0/3407594996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f02980 0x55c9c92efc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.074+0000 7ff5b7fff640 1 -- 172.21.15.67:0/3407594996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f02980 msgr2=0x55c9c92efc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.074+0000 7ff5b7fff640 1 --2- 172.21.15.67:0/3407594996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f02980 0x55c9c92efc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.074+0000 7ff5b7fff640 1 -- 172.21.15.67:0/3407594996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f0ab60 msgr2=0x55c9c92f9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.074+0000 7ff5b7fff640 1 --2- 172.21.15.67:0/3407594996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f0ab60 0x55c9c92f9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.074+0000 7ff5b7fff640 1 -- 172.21.15.67:0/3407594996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9c92dbe80 con 0x55c9c8eea1c0 2024-07-31T14:55:38.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.074+0000 7ff5b7fff640 1 --2- 172.21.15.67:0/3407594996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8eea1c0 0x55c9c92ceb20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8006e00 tx=0x7ff5a8001ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=745c22f4c8877b2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:38.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.074+0000 7ff59dffb640 1 -- 172.21.15.67:0/3407594996 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8029020 con 0x55c9c8eea1c0 2024-07-31T14:55:38.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.074+0000 7ff59dffb640 1 -- 172.21.15.67:0/3407594996 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5a8012ae0 con 0x55c9c8eea1c0 2024-07-31T14:55:38.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.074+0000 7ff59dffb640 1 -- 172.21.15.67:0/3407594996 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8025c10 con 0x55c9c8eea1c0 2024-07-31T14:55:38.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.075+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/3407594996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8eea1c0 msgr2=0x55c9c92ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.075+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/3407594996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8eea1c0 0x55c9c92ceb20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8006e00 tx=0x7ff5a8001ef0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.075+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/3407594996 shutdown_connections 2024-07-31T14:55:38.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.075+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/3407594996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8f0ab60 0x55c9c92f9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.075+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/3407594996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f02980 0x55c9c92efc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.075+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/3407594996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8eea1c0 0x55c9c92ceb20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.075+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/3407594996 >> 172.21.15.67:0/3407594996 conn(0x55c9c8f1f990 msgr2=0x55c9c92dccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:38.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.075+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/3407594996 shutdown_connections 2024-07-31T14:55:38.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.075+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/3407594996 wait complete. 2024-07-31T14:55:38.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.076+0000 7ff5c4f2c740 1 -- start start 2024-07-31T14:55:38.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.076+0000 7ff5c4f2c740 1 Processor -- start 2024-07-31T14:55:38.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.076+0000 7ff5c4f2c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8eea1c0 0x55c9c92f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.076+0000 7ff5c4f2c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f02980 0x55c9c92f6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.076+0000 7ff5c4f2c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f0ab60 0x55c9c92f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.076+0000 7ff5c4f2c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81b8360 con 0x55c9c8eea1c0 2024-07-31T14:55:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.076+0000 7ff5c4f2c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81b7bd0 con 0x55c9c8f02980 2024-07-31T14:55:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.076+0000 7ff5c4f2c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81a9ee0 con 0x55c9c8f0ab60 2024-07-31T14:55:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.076+0000 7ff5c4f2c740 10 client.0 _ll_register_callbacks cb 0x7ff5a800bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.076+0000 7ff5b77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f02980 0x55c9c92f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.076+0000 7ff5b7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8eea1c0 0x55c9c92f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.076+0000 7ff5b77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f02980 0x55c9c92f6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51254/0 (socket says 172.21.15.67:51254) 2024-07-31T14:55:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.076+0000 7ff5b77fe640 1 -- 172.21.15.67:0/3379120735 learned_addr learned my addr 172.21.15.67:0/3379120735 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5bcf78640 1 --2- 172.21.15.67:0/3379120735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f0ab60 0x55c9c92f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5b77fe640 1 -- 172.21.15.67:0/3379120735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f0ab60 msgr2=0x55c9c92f0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5b77fe640 1 --2- 172.21.15.67:0/3379120735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f0ab60 0x55c9c92f0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5b77fe640 1 -- 172.21.15.67:0/3379120735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8eea1c0 msgr2=0x55c9c92f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5b77fe640 1 --2- 172.21.15.67:0/3379120735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8eea1c0 0x55c9c92f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5b77fe640 1 -- 172.21.15.67:0/3379120735 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9c92f6a80 con 0x55c9c8f02980 2024-07-31T14:55:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5b7fff640 1 --2- 172.21.15.67:0/3379120735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8eea1c0 0x55c9c92f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5b77fe640 1 --2- 172.21.15.67:0/3379120735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f02980 0x55c9c92f6440 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55c9c92d04f0 tx=0x7ff5ac030340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39c87b73cb20ca1a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5977fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:55:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5977fe640 1 -- 172.21.15.67:0/3379120735 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0c6040 con 0x55c9c8f02980 2024-07-31T14:55:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5977fe640 1 -- 172.21.15.67:0/3379120735 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff5ac0c7040 con 0x55c9c8f02980 2024-07-31T14:55:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/3379120735 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a8012ae0 con 0x55c9c8f02980 2024-07-31T14:55:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5977fe640 1 -- 172.21.15.67:0/3379120735 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff5ac0c61e0 con 0x55c9c8f02980 2024-07-31T14:55:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5c4f2c740 10 client.26399 fetch_fsmap learned FSMap version 41 2024-07-31T14:55:38.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.077+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/3379120735 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c9c9368990 con 0x55c9c8f02980 2024-07-31T14:55:38.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.078+0000 7ff5977fe640 1 -- 172.21.15.67:0/3379120735 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5a8012ae0 con 0x55c9c8f02980 2024-07-31T14:55:38.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.078+0000 7ff5977fe640 1 -- 172.21.15.67:0/3379120735 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 41) ==== 8418+0+0 (secure 0 0 0) 0x7ff5ac0c6ce0 con 0x55c9c8f02980 2024-07-31T14:55:38.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.078+0000 7ff5c4f2c740 10 client.26399 fetch_fsmap finished waiting for FSMap version 41 2024-07-31T14:55:38.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.078+0000 7ff5c4f2c740 10 client.26399 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:55:38.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.078+0000 7ff5c4f2c740 20 client.26399 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:38.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.078+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/3379120735 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c9c8f2b840 0x55c9c936b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.078+0000 7ff5c4f2c740 4 client.26399 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.4, "await_for": 45.0}] 2024-07-31T14:55:38.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.078+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/3379120735 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.4, "await_for": 45.0}) -- 0x55c9c9248700 con 0x55c9c8f2b840 2024-07-31T14:55:38.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.078+0000 7ff5b7fff640 1 --2- 172.21.15.67:0/3379120735 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c9c8f2b840 0x55c9c936b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.078+0000 7ff5977fe640 10 client.26399 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:38.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.078+0000 7ff5b7fff640 1 --2- 172.21.15.67:0/3379120735 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c9c8f2b840 0x55c9c936b050 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8be52706b33b0a4 server_cookie=f4214025957387e4 in_seq=0 out_seq=0 2024-07-31T14:55:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.421+0000 7f7cb134b640 1 -- 172.21.15.67:0/2819450597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cac109830 msgr2=0x7f7cac110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.421+0000 7f7cb134b640 1 --2- 172.21.15.67:0/2819450597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cac109830 0x7f7cac110060 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009be0 tx=0x7f7ca402a6c0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.421+0000 7f7cb134b640 1 -- 172.21.15.67:0/2819450597 shutdown_connections 2024-07-31T14:55:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.421+0000 7f7cb134b640 1 --2- 172.21.15.67:0/2819450597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cac109830 0x7f7cac110060 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.421+0000 7f7cb134b640 1 --2- 172.21.15.67:0/2819450597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cac108ee0 0x7f7cac1092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.421+0000 7f7cb134b640 1 --2- 172.21.15.67:0/2819450597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cac1035f0 0x7f7cac103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.421+0000 7f7cb134b640 1 -- 172.21.15.67:0/2819450597 >> 172.21.15.67:0/2819450597 conn(0x7f7cac07b340 msgr2=0x7f7cac07b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.421+0000 7f7cb134b640 1 -- 172.21.15.67:0/2819450597 shutdown_connections 2024-07-31T14:55:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.421+0000 7f7cb134b640 1 -- 172.21.15.67:0/2819450597 wait complete. 2024-07-31T14:55:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.422+0000 7f7cb134b640 1 Processor -- start 2024-07-31T14:55:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.422+0000 7f7cb134b640 1 -- start start 2024-07-31T14:55:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.423+0000 7f7cb134b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cac1035f0 0x7f7cac199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.423+0000 7f7cb134b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cac108ee0 0x7f7cac19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.423+0000 7f7caaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cac1035f0 0x7f7cac199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.423+0000 7f7caaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cac1035f0 0x7f7cac199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37968/0 (socket says 172.21.15.67:37968) 2024-07-31T14:55:38.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.424+0000 7f7caaffd640 1 -- 172.21.15.67:0/3204792538 learned_addr learned my addr 172.21.15.67:0/3204792538 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:38.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.424+0000 7f7cb134b640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cac109830 0x7f7cac19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.424+0000 7f7cb134b640 1 -- 172.21.15.67:0/3204792538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac1134e0 con 0x7f7cac108ee0 2024-07-31T14:55:38.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.424+0000 7f7cab7fe640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cac109830 0x7f7cac19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.424+0000 7f7ca3fff640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cac108ee0 0x7f7cac19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.425+0000 7f7cb134b640 1 -- 172.21.15.67:0/3204792538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac113360 con 0x7f7cac109830 2024-07-31T14:55:38.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.425+0000 7f7cb134b640 1 -- 172.21.15.67:0/3204792538 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac113660 con 0x7f7cac1035f0 2024-07-31T14:55:38.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.427+0000 7f7caaffd640 1 -- 172.21.15.67:0/3204792538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cac109830 msgr2=0x7f7cac19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.427+0000 7f7caaffd640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cac109830 0x7f7cac19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.427+0000 7f7caaffd640 1 -- 172.21.15.67:0/3204792538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cac108ee0 msgr2=0x7f7cac19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.427+0000 7f7caaffd640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cac108ee0 0x7f7cac19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.427+0000 7f7caaffd640 1 -- 172.21.15.67:0/3204792538 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cac19fde0 con 0x7f7cac1035f0 2024-07-31T14:55:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.428+0000 7f7caaffd640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cac1035f0 0x7f7cac199e70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e9f0 tx=0x7f7c9c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.428+0000 7f7ca8ff9640 1 -- 172.21.15.67:0/3204792538 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c00cd80 con 0x7f7cac1035f0 2024-07-31T14:55:38.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.429+0000 7f7ca8ff9640 1 -- 172.21.15.67:0/3204792538 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c9c0108e0 con 0x7f7cac1035f0 2024-07-31T14:55:38.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.429+0000 7f7cb134b640 1 -- 172.21.15.67:0/3204792538 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cac1a00d0 con 0x7f7cac1035f0 2024-07-31T14:55:38.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.429+0000 7f7cb134b640 1 -- 172.21.15.67:0/3204792538 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cac1a6830 con 0x7f7cac1035f0 2024-07-31T14:55:38.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.429+0000 7f7ca8ff9640 1 -- 172.21.15.67:0/3204792538 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c018980 con 0x7f7cac1035f0 2024-07-31T14:55:38.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.434+0000 7f7cb789c740 1 Processor -- start 2024-07-31T14:55:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.437+0000 7f7cb789c740 1 -- start start 2024-07-31T14:55:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.437+0000 7f7cb789c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10df070 0x5637a1484570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.437+0000 7f7cb789c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 0x5637a1484ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.438+0000 7f7cb789c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10be390 0x5637a1484ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.438+0000 7f7cb789c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5637a13fabb0 con 0x5637a10df070 2024-07-31T14:55:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.438+0000 7f7cb789c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5637a13ea0b0 con 0x5637a10be390 2024-07-31T14:55:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.438+0000 7f7cb789c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5637a13ee0f0 con 0x5637a10b61b0 2024-07-31T14:55:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.438+0000 7f7ca3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10be390 0x5637a1484ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.438+0000 7f7ca3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10be390 0x5637a1484ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51284/0 (socket says 172.21.15.67:51284) 2024-07-31T14:55:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.438+0000 7f7ca3fff640 1 -- 172.21.15.67:0/392771059 learned_addr learned my addr 172.21.15.67:0/392771059 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:38.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.439+0000 7f7cab7fe640 1 --2- 172.21.15.67:0/392771059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10df070 0x5637a1484570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.440+0000 7f7ca3fff640 1 -- 172.21.15.67:0/392771059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 msgr2=0x5637a1484ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:55:38.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.440+0000 7f7ca3fff640 1 --2- 172.21.15.67:0/392771059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 0x5637a1484ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.440+0000 7f7ca3fff640 1 -- 172.21.15.67:0/392771059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10df070 msgr2=0x5637a1484570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.440+0000 7f7ca3fff640 1 --2- 172.21.15.67:0/392771059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10df070 0x5637a1484570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.440+0000 7f7ca3fff640 1 -- 172.21.15.67:0/392771059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a1390b90 con 0x5637a10be390 2024-07-31T14:55:38.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.440+0000 7f7ca3fff640 1 --2- 172.21.15.67:0/392771059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10be390 0x5637a1484ff0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009c30 tx=0x7f7c94011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fb3ac2cfb0d2af4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:38.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.441+0000 7f7ca8ff9640 1 -- 172.21.15.67:0/3204792538 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7c9c010430 con 0x7f7cac1035f0 2024-07-31T14:55:38.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.442+0000 7f7ca8ff9640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c980b3f10 0x7f7c980b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.442+0000 7f7ca8ff9640 1 -- 172.21.15.67:0/3204792538 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f7c9c0e45a0 con 0x7f7cac1035f0 2024-07-31T14:55:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.443+0000 7f7ca17fa640 1 -- 172.21.15.67:0/392771059 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94004020 con 0x5637a10be390 2024-07-31T14:55:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.443+0000 7f7ca17fa640 1 -- 172.21.15.67:0/392771059 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c940041c0 con 0x5637a10be390 2024-07-31T14:55:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.443+0000 7f7cab7fe640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c980b3f10 0x7f7c980b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.443+0000 7f7ca17fa640 1 -- 172.21.15.67:0/392771059 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c940197a0 con 0x5637a10be390 2024-07-31T14:55:38.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.445+0000 7f7cb789c740 1 -- 172.21.15.67:0/392771059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10be390 msgr2=0x5637a1484ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.445+0000 7f7cb789c740 1 --2- 172.21.15.67:0/392771059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10be390 0x5637a1484ff0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009c30 tx=0x7f7c94011810 comp rx=0 tx=0).stop 2024-07-31T14:55:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.445+0000 7f7cb789c740 1 -- 172.21.15.67:0/392771059 shutdown_connections 2024-07-31T14:55:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.445+0000 7f7cb789c740 1 --2- 172.21.15.67:0/392771059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10be390 0x5637a1484ff0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.445+0000 7f7cb789c740 1 --2- 172.21.15.67:0/392771059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 0x5637a1484ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.445+0000 7f7cb789c740 1 --2- 172.21.15.67:0/392771059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10df070 0x5637a1484570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.445+0000 7f7cb789c740 1 -- 172.21.15.67:0/392771059 >> 172.21.15.67:0/392771059 conn(0x5637a10d3460 msgr2=0x5637a1482460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.445+0000 7f7cab7fe640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c980b3f10 0x7f7c980b6420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4004af0 tx=0x7f7ca4005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:38.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.446+0000 7f7cb789c740 1 -- 172.21.15.67:0/392771059 shutdown_connections 2024-07-31T14:55:38.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.446+0000 7f7cb789c740 1 -- 172.21.15.67:0/392771059 wait complete. 2024-07-31T14:55:38.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.447+0000 7f7cb789c740 1 -- start start 2024-07-31T14:55:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.447+0000 7f7cb789c740 1 Processor -- start 2024-07-31T14:55:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.448+0000 7f7cb789c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 0x5637a148c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.448+0000 7f7cb789c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a148c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.448+0000 7f7cb789c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10df070 0x5637a14a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.448+0000 7f7cb789c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5637a124d700 con 0x5637a10be390 2024-07-31T14:55:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.448+0000 7f7cb789c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5637a138c9e0 con 0x5637a10df070 2024-07-31T14:55:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.448+0000 7f7cb789c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5637a149b4e0 con 0x5637a10b61b0 2024-07-31T14:55:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.448+0000 7f7cb789c740 10 client.0 _ll_register_callbacks cb 0x5637a13a74f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.448+0000 7f7caaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 0x5637a148c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.448+0000 7f7caaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 0x5637a148c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37978/0 (socket says 172.21.15.67:37978) 2024-07-31T14:55:38.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.448+0000 7f7caaffd640 1 -- 172.21.15.67:0/3879542599 learned_addr learned my addr 172.21.15.67:0/3879542599 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:38.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.448+0000 7f7cab7fe640 1 --2- 172.21.15.67:0/3879542599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10df070 0x5637a14a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.449+0000 7f7caaffd640 1 -- 172.21.15.67:0/3879542599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10df070 msgr2=0x5637a14a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.449+0000 7f7caaffd640 1 --2- 172.21.15.67:0/3879542599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10df070 0x5637a14a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.449+0000 7f7caaffd640 1 -- 172.21.15.67:0/3879542599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 msgr2=0x5637a148c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.449+0000 7f7caaffd640 1 --2- 172.21.15.67:0/3879542599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a148c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.449+0000 7f7caaffd640 1 -- 172.21.15.67:0/3879542599 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a114a650 con 0x5637a10b61b0 2024-07-31T14:55:38.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.450+0000 7f7caaffd640 1 --2- 172.21.15.67:0/3879542599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 0x5637a148c120 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e9c0 tx=0x7f7c9c0aaa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b935be5f09f66fa1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:38.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.450+0000 7f7c927fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:55:38.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.450+0000 7f7c927fc640 1 -- 172.21.15.67:0/3879542599 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0ad030 con 0x5637a10b61b0 2024-07-31T14:55:38.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.450+0000 7f7c927fc640 1 -- 172.21.15.67:0/3879542599 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f7c9c0a74e0 con 0x5637a10b61b0 2024-07-31T14:55:38.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.450+0000 7f7cb789c740 1 -- 172.21.15.67:0/3879542599 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c940041c0 con 0x5637a10b61b0 2024-07-31T14:55:38.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.451+0000 7f7c927fc640 1 -- 172.21.15.67:0/3879542599 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7c9c0b3ef0 con 0x5637a10b61b0 2024-07-31T14:55:38.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.452+0000 7f7cb789c740 10 client.26440 fetch_fsmap learned FSMap version 41 2024-07-31T14:55:38.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.452+0000 7f7cb789c740 1 -- 172.21.15.67:0/3879542599 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5637a14914a0 con 0x5637a10b61b0 2024-07-31T14:55:38.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.453+0000 7f7c927fc640 1 -- 172.21.15.67:0/3879542599 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c940041c0 con 0x5637a10b61b0 2024-07-31T14:55:38.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.453+0000 7f7c927fc640 1 -- 172.21.15.67:0/3879542599 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) ==== 8418+0+0 (secure 0 0 0) 0x7f7c9c018cb0 con 0x5637a10b61b0 2024-07-31T14:55:38.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.454+0000 7f7cb789c740 10 client.26440 fetch_fsmap finished waiting for FSMap version 41 2024-07-31T14:55:38.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.454+0000 7f7cb789c740 10 client.26440 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:55:38.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.454+0000 7f7cb789c740 20 client.26440 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:38.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.454+0000 7f7cb789c740 1 --2- 172.21.15.67:0/3879542599 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5637a109d9f0 0x5637a1494f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.454+0000 7f7cb789c740 4 client.26440 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:55:38.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.454+0000 7f7cb789c740 1 -- 172.21.15.67:0/3879542599 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637a13fcb00 con 0x5637a109d9f0 2024-07-31T14:55:38.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.454+0000 7f7ca3fff640 1 --2- 172.21.15.67:0/3879542599 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5637a109d9f0 0x5637a1494f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.455+0000 7f7ca3fff640 1 --2- 172.21.15.67:0/3879542599 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5637a109d9f0 0x5637a1494f20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a3e012d90875eb1 server_cookie=2def8eabdd41f423 in_seq=0 out_seq=0 2024-07-31T14:55:38.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.455+0000 7f7c927fc640 10 client.26440 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:55:38.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.627+0000 7f7c927fc640 1 -- 172.21.15.67:0/3879542599 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637a13fcb00 con 0x5637a109d9f0 2024-07-31T14:55:38.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.627+0000 7f7c927fc640 10 client.26440 handle_command_reply: tid=0 2024-07-31T14:55:38.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.627+0000 7f7cb789c740 1 client.26440 shutdown 2024-07-31T14:55:38.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.627+0000 7f7cb789c740 1 -- 172.21.15.67:0/3879542599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 msgr2=0x5637a148c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.627+0000 7f7cb789c740 1 --2- 172.21.15.67:0/3879542599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 0x5637a148c120 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e9c0 tx=0x7f7c9c0aaa00 comp rx=0 tx=0).stop 2024-07-31T14:55:38.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.627+0000 7f7cb789c740 1 -- 172.21.15.67:0/3879542599 shutdown_connections 2024-07-31T14:55:38.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.627+0000 7f7cb789c740 1 --2- 172.21.15.67:0/3879542599 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5637a109d9f0 0x5637a1494f20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.627+0000 7f7cb789c740 1 --2- 172.21.15.67:0/3879542599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10df070 0x5637a14a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.627+0000 7f7cb789c740 1 --2- 172.21.15.67:0/3879542599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a148c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.627+0000 7f7cb789c740 1 --2- 172.21.15.67:0/3879542599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 0x5637a148c120 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.627+0000 7f7cb789c740 1 -- 172.21.15.67:0/3879542599 >> 172.21.15.67:0/3879542599 conn(0x5637a10d3460 msgr2=0x5637a1480c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:38.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.627+0000 7f7c927fc640 0 client.26440 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:55:38.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.628+0000 7f7cb789c740 1 -- 172.21.15.67:0/3879542599 shutdown_connections 2024-07-31T14:55:38.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.628+0000 7f7cb789c740 1 -- 172.21.15.67:0/3879542599 wait complete. 2024-07-31T14:55:38.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.628+0000 7f7cb789c740 20 client.26440 trim_cache size 0 max 16384 2024-07-31T14:55:38.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.648+0000 7f7cb789c740 1 Processor -- start 2024-07-31T14:55:38.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.648+0000 7f7cb789c740 1 -- start start 2024-07-31T14:55:38.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.649+0000 7f7cb789c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10b61b0 0x5637a1481350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.649+0000 7f7cb789c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a14a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.649+0000 7f7cb789c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10df070 0x5637a14bad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.649+0000 7f7cb789c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac1ae480 con 0x5637a10be390 2024-07-31T14:55:38.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.649+0000 7f7cb789c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac1b0140 con 0x5637a10b61b0 2024-07-31T14:55:38.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.649+0000 7f7cb789c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac1aeb50 con 0x5637a10df070 2024-07-31T14:55:38.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.649+0000 7f7ca3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a14a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.649+0000 7f7ca3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a14a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45880/0 (socket says 172.21.15.67:45880) 2024-07-31T14:55:38.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.649+0000 7f7ca3fff640 1 -- 172.21.15.67:0/2937287792 learned_addr learned my addr 172.21.15.67:0/2937287792 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:38.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.649+0000 7f7caaffd640 1 --2- 172.21.15.67:0/2937287792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10b61b0 0x5637a1481350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.649+0000 7f7cab7fe640 1 --2- 172.21.15.67:0/2937287792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10df070 0x5637a14bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.650+0000 7f7ca3fff640 1 -- 172.21.15.67:0/2937287792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10df070 msgr2=0x5637a14bad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.650+0000 7f7ca3fff640 1 --2- 172.21.15.67:0/2937287792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10df070 0x5637a14bad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.650+0000 7f7ca3fff640 1 -- 172.21.15.67:0/2937287792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10b61b0 msgr2=0x5637a1481350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.650+0000 7f7ca3fff640 1 --2- 172.21.15.67:0/2937287792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10b61b0 0x5637a1481350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.650+0000 7f7ca3fff640 1 -- 172.21.15.67:0/2937287792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a148d1e0 con 0x5637a10be390 2024-07-31T14:55:38.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.650+0000 7f7ca3fff640 1 --2- 172.21.15.67:0/2937287792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a14a4600 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7c9401b040 tx=0x7f7c94003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774a34077414a346 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:38.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.650+0000 7f7ca0ff9640 1 -- 172.21.15.67:0/2937287792 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94003ef0 con 0x5637a10be390 2024-07-31T14:55:38.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.651+0000 7f7ca0ff9640 1 -- 172.21.15.67:0/2937287792 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c940054f0 con 0x5637a10be390 2024-07-31T14:55:38.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.651+0000 7f7ca0ff9640 1 -- 172.21.15.67:0/2937287792 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c940039d0 con 0x5637a10be390 2024-07-31T14:55:38.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.651+0000 7f7cb789c740 1 -- 172.21.15.67:0/2937287792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 msgr2=0x5637a14a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.651+0000 7f7cb789c740 1 --2- 172.21.15.67:0/2937287792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a14a4600 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7c9401b040 tx=0x7f7c94003de0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.652+0000 7f7cb789c740 1 -- 172.21.15.67:0/2937287792 shutdown_connections 2024-07-31T14:55:38.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.652+0000 7f7cb789c740 1 --2- 172.21.15.67:0/2937287792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10df070 0x5637a14bad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.652+0000 7f7cb789c740 1 --2- 172.21.15.67:0/2937287792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a14a4600 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.652+0000 7f7cb789c740 1 --2- 172.21.15.67:0/2937287792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10b61b0 0x5637a1481350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.652+0000 7f7cb789c740 1 -- 172.21.15.67:0/2937287792 >> 172.21.15.67:0/2937287792 conn(0x5637a10d3460 msgr2=0x5637a1480c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:38.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.652+0000 7f7cb789c740 1 -- 172.21.15.67:0/2937287792 shutdown_connections 2024-07-31T14:55:38.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.652+0000 7f7cb789c740 1 -- 172.21.15.67:0/2937287792 wait complete. 2024-07-31T14:55:38.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.652+0000 7f7cb789c740 1 -- start start 2024-07-31T14:55:38.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.653+0000 7f7cb789c740 1 Processor -- start 2024-07-31T14:55:38.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.653+0000 7f7cb789c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 0x5637a14a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.654+0000 7f7cb789c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a149c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.654+0000 7f7cb789c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10df070 0x5637a149c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.654+0000 7f7caaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 0x5637a14a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.654+0000 7f7caaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 0x5637a14a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37994/0 (socket says 172.21.15.67:37994) 2024-07-31T14:55:38.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.654+0000 7f7cb789c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac1afa70 con 0x5637a10be390 2024-07-31T14:55:38.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.654+0000 7f7cb789c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac1af2e0 con 0x5637a10df070 2024-07-31T14:55:38.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.654+0000 7f7cb789c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac1ac460 con 0x5637a10b61b0 2024-07-31T14:55:38.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.654+0000 7f7cb789c740 10 client.0 _ll_register_callbacks cb 0x7f7c9c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:38.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.654+0000 7f7ca3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a149c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.654+0000 7f7ca3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a149c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45890/0 (socket says 172.21.15.67:45890) 2024-07-31T14:55:38.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.654+0000 7f7ca3fff640 1 -- 172.21.15.67:0/2548809204 learned_addr learned my addr 172.21.15.67:0/2548809204 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:38.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.654+0000 7f7cab7fe640 1 --2- 172.21.15.67:0/2548809204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10df070 0x5637a149c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.655+0000 7f7ca3fff640 1 -- 172.21.15.67:0/2548809204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 msgr2=0x5637a14a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.655+0000 7f7ca3fff640 1 --2- 172.21.15.67:0/2548809204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 0x5637a14a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.655+0000 7f7ca3fff640 1 -- 172.21.15.67:0/2548809204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10df070 msgr2=0x5637a149c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:38.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.655+0000 7f7ca3fff640 1 --2- 172.21.15.67:0/2548809204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10df070 0x5637a149c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.655+0000 7f7ca3fff640 1 -- 172.21.15.67:0/2548809204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a149ca80 con 0x5637a10be390 2024-07-31T14:55:38.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.655+0000 7f7cab7fe640 1 --2- 172.21.15.67:0/2548809204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10df070 0x5637a149c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:38.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.655+0000 7f7ca3fff640 1 --2- 172.21.15.67:0/2548809204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a149c000 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7c94000c00 tx=0x7f7c94005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71614af10f7ab935 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:38.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.655+0000 7f7c927fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:55:38.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.655+0000 7f7c927fc640 1 -- 172.21.15.67:0/2548809204 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94009510 con 0x5637a10be390 2024-07-31T14:55:38.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.655+0000 7f7c927fc640 1 -- 172.21.15.67:0/2548809204 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f7c94004100 con 0x5637a10be390 2024-07-31T14:55:38.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.655+0000 7f7cb789c740 1 -- 172.21.15.67:0/2548809204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c940054f0 con 0x5637a10be390 2024-07-31T14:55:38.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.656+0000 7f7c927fc640 1 -- 172.21.15.67:0/2548809204 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7c94009870 con 0x5637a10be390 2024-07-31T14:55:38.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.656+0000 7f7cb789c740 10 client.16830 fetch_fsmap learned FSMap version 41 2024-07-31T14:55:38.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.656+0000 7f7cb789c740 1 -- 172.21.15.67:0/2548809204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5637a151b990 con 0x5637a10be390 2024-07-31T14:55:38.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.656+0000 7f7c927fc640 1 -- 172.21.15.67:0/2548809204 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c940054f0 con 0x5637a10be390 2024-07-31T14:55:38.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.656+0000 7f7c927fc640 1 -- 172.21.15.67:0/2548809204 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) ==== 8418+0+0 (secure 0 0 0) 0x7f7c94004b50 con 0x5637a10be390 2024-07-31T14:55:38.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.656+0000 7f7cb789c740 10 client.16830 fetch_fsmap finished waiting for FSMap version 41 2024-07-31T14:55:38.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.656+0000 7f7cb789c740 10 client.16830 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:55:38.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.656+0000 7f7cb789c740 20 client.16830 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:38.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.656+0000 7f7cb789c740 1 --2- 172.21.15.67:0/2548809204 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5637a1072da0 0x5637a151dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:38.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.656+0000 7f7cb789c740 4 client.16830 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:55:38.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.656+0000 7f7cb789c740 1 -- 172.21.15.67:0/2548809204 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637a13f85a0 con 0x5637a1072da0 2024-07-31T14:55:38.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.657+0000 7f7caaffd640 1 --2- 172.21.15.67:0/2548809204 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5637a1072da0 0x5637a151dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:38.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.658+0000 7f7caaffd640 1 --2- 172.21.15.67:0/2548809204 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5637a1072da0 0x5637a151dfa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8349ec2b473a97a4 server_cookie=a44cc227c58d0ba3 in_seq=0 out_seq=0 2024-07-31T14:55:38.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.658+0000 7f7c927fc640 10 client.16830 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:55:38.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.659+0000 7f7c927fc640 1 -- 172.21.15.67:0/2548809204 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637a13f85a0 con 0x5637a1072da0 2024-07-31T14:55:38.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.659+0000 7f7c927fc640 10 client.16830 handle_command_reply: tid=0 2024-07-31T14:55:38.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.659+0000 7f7cb789c740 1 client.16830 shutdown 2024-07-31T14:55:38.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.659+0000 7f7cb789c740 1 -- 172.21.15.67:0/2548809204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 msgr2=0x5637a149c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.659+0000 7f7cb789c740 1 --2- 172.21.15.67:0/2548809204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a149c000 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7c94000c00 tx=0x7f7c94005340 comp rx=0 tx=0).stop 2024-07-31T14:55:38.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.659+0000 7f7cb789c740 1 -- 172.21.15.67:0/2548809204 shutdown_connections 2024-07-31T14:55:38.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.659+0000 7f7cb789c740 1 --2- 172.21.15.67:0/2548809204 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5637a1072da0 0x5637a151dfa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.659+0000 7f7cb789c740 1 --2- 172.21.15.67:0/2548809204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637a10df070 0x5637a149c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.659+0000 7f7c927fc640 0 client.16830 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:55:38.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.659+0000 7f7cb789c740 1 --2- 172.21.15.67:0/2548809204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5637a10be390 0x5637a149c000 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.659+0000 7f7cb789c740 1 --2- 172.21.15.67:0/2548809204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637a10b61b0 0x5637a14a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.660+0000 7f7cb789c740 1 -- 172.21.15.67:0/2548809204 >> 172.21.15.67:0/2548809204 conn(0x5637a10d3460 msgr2=0x5637a148d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:38.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.660+0000 7f7cb789c740 1 -- 172.21.15.67:0/2548809204 shutdown_connections 2024-07-31T14:55:38.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.660+0000 7f7cb789c740 1 -- 172.21.15.67:0/2548809204 wait complete. 2024-07-31T14:55:38.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.660+0000 7f7cb789c740 20 client.16830 trim_cache size 0 max 16384 2024-07-31T14:55:38.663 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:55:38.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.661+0000 7f7cb134b640 1 -- 172.21.15.67:0/3204792538 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c980b3f10 msgr2=0x7f7c980b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.661+0000 7f7cb134b640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c980b3f10 0x7f7c980b6420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4004af0 tx=0x7f7ca4005fd0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.661+0000 7f7cb134b640 1 -- 172.21.15.67:0/3204792538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cac1035f0 msgr2=0x7f7cac199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:38.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.661+0000 7f7cb134b640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cac1035f0 0x7f7cac199e70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e9f0 tx=0x7f7c9c00eeb0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.661+0000 7f7cb134b640 1 -- 172.21.15.67:0/3204792538 shutdown_connections 2024-07-31T14:55:38.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.661+0000 7f7cb134b640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c980b3f10 0x7f7c980b6420 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.661+0000 7f7cb134b640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cac109830 0x7f7cac19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.661+0000 7f7cb134b640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cac108ee0 0x7f7cac19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.661+0000 7f7cb134b640 1 --2- 172.21.15.67:0/3204792538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cac1035f0 0x7f7cac199e70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:38.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.661+0000 7f7cb134b640 1 -- 172.21.15.67:0/3204792538 >> 172.21.15.67:0/3204792538 conn(0x7f7cac07b340 msgr2=0x7f7cac106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:38.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.661+0000 7f7cb134b640 1 -- 172.21.15.67:0/3204792538 shutdown_connections 2024-07-31T14:55:38.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:38.661+0000 7f7cb134b640 1 -- 172.21.15.67:0/3204792538 wait complete. 2024-07-31T14:55:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:38 smithi077 ceph-mon[37327]: pgmap v537: 129 pgs: 129 active+clean; 798 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 24 MiB/s wr, 652 op/s 2024-07-31T14:55:39.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:38 smithi067 ceph-mon[32750]: pgmap v537: 129 pgs: 129 active+clean; 798 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 24 MiB/s wr, 652 op/s 2024-07-31T14:55:39.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:38 smithi073 ceph-mon[36714]: pgmap v537: 129 pgs: 129 active+clean; 798 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 24 MiB/s wr, 652 op/s 2024-07-31T14:55:39.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T14:55:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:55:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:39 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:55:40.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:39 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:55:40.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:39 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:55:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:40 smithi077 ceph-mon[37327]: pgmap v538: 129 pgs: 129 active+clean; 859 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 25 MiB/s wr, 728 op/s 2024-07-31T14:55:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:40 smithi067 ceph-mon[32750]: pgmap v538: 129 pgs: 129 active+clean; 859 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 25 MiB/s wr, 728 op/s 2024-07-31T14:55:41.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:40 smithi073 ceph-mon[36714]: pgmap v538: 129 pgs: 129 active+clean; 859 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 25 MiB/s wr, 728 op/s 2024-07-31T14:55:41.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.749+0000 7f4700597640 1 -- 172.21.15.67:0/2762696860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46f810d880 msgr2=0x7f46f810fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:41.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.749+0000 7f4700597640 1 --2- 172.21.15.67:0/2762696860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46f810d880 0x7f46f810fd50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009ca0 tx=0x7f46f002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.750+0000 7f4700597640 1 -- 172.21.15.67:0/2762696860 shutdown_connections 2024-07-31T14:55:41.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.750+0000 7f4700597640 1 --2- 172.21.15.67:0/2762696860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46f810d880 0x7f46f810fd50 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.750+0000 7f4700597640 1 --2- 172.21.15.67:0/2762696860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f8104d00 0x7f46f810d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.750+0000 7f4700597640 1 --2- 172.21.15.67:0/2762696860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f81042d0 0x7f46f81046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.750+0000 7f4700597640 1 -- 172.21.15.67:0/2762696860 >> 172.21.15.67:0/2762696860 conn(0x7f46f8079170 msgr2=0x7f46f8079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:41.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.750+0000 7f4700597640 1 -- 172.21.15.67:0/2762696860 shutdown_connections 2024-07-31T14:55:41.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.750+0000 7f4700597640 1 -- 172.21.15.67:0/2762696860 wait complete. 2024-07-31T14:55:41.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.751+0000 7f4700597640 1 Processor -- start 2024-07-31T14:55:41.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.751+0000 7f4700597640 1 -- start start 2024-07-31T14:55:41.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.751+0000 7f4700597640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f81042d0 0x7f46f819c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.751+0000 7f4700597640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46f8104d00 0x7f46f819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.751+0000 7f4700597640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f810d880 0x7f46f81a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.751+0000 7f4700597640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f46f8113510 con 0x7f46f8104d00 2024-07-31T14:55:41.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.751+0000 7f4700597640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46f8113390 con 0x7f46f81042d0 2024-07-31T14:55:41.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.751+0000 7f4700597640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46f8113690 con 0x7f46f810d880 2024-07-31T14:55:41.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.751+0000 7f46fe333640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f81042d0 0x7f46f819c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.751+0000 7f46fdb32640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46f8104d00 0x7f46f819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.751+0000 7f46fe333640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f81042d0 0x7f46f819c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40776/0 (socket says 172.21.15.67:40776) 2024-07-31T14:55:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.751+0000 7f46fe333640 1 -- 172.21.15.67:0/3636606302 learned_addr learned my addr 172.21.15.67:0/3636606302 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.751+0000 7f46feb34640 1 --2- 172.21.15.67:0/3636606302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f810d880 0x7f46f81a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.752+0000 7f46fe333640 1 -- 172.21.15.67:0/3636606302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f810d880 msgr2=0x7f46f81a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.752+0000 7f46fe333640 1 --2- 172.21.15.67:0/3636606302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f810d880 0x7f46f81a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.752+0000 7f46fe333640 1 -- 172.21.15.67:0/3636606302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46f8104d00 msgr2=0x7f46f819c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.752+0000 7f46fe333640 1 --2- 172.21.15.67:0/3636606302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46f8104d00 0x7f46f819c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.752+0000 7f46fe333640 1 -- 172.21.15.67:0/3636606302 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f81a1fb0 con 0x7f46f81042d0 2024-07-31T14:55:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.752+0000 7f46fdb32640 1 --2- 172.21.15.67:0/3636606302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46f8104d00 0x7f46f819c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.752+0000 7f46fe333640 1 --2- 172.21.15.67:0/3636606302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f81042d0 0x7f46f819c040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f46f400e9c0 tx=0x7f46f400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.752+0000 7f46ef7fe640 1 -- 172.21.15.67:0/3636606302 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f400cd80 con 0x7f46f81042d0 2024-07-31T14:55:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.752+0000 7f4700597640 1 -- 172.21.15.67:0/3636606302 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f81a22a0 con 0x7f46f81042d0 2024-07-31T14:55:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.752+0000 7f46ef7fe640 1 -- 172.21.15.67:0/3636606302 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f40035f0 con 0x7f46f81042d0 2024-07-31T14:55:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.752+0000 7f46ef7fe640 1 -- 172.21.15.67:0/3636606302 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4010690 con 0x7f46f81042d0 2024-07-31T14:55:41.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.752+0000 7f4700597640 1 -- 172.21.15.67:0/3636606302 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f81a8a00 con 0x7f46f81042d0 2024-07-31T14:55:41.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.755+0000 7f4706ae8740 1 Processor -- start 2024-07-31T14:55:41.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.756+0000 7f4706ae8740 1 -- start start 2024-07-31T14:55:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.756+0000 7f4706ae8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86eac070 0x556b87251570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.756+0000 7f4706ae8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e831b0 0x556b87251ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.756+0000 7f4706ae8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e8b390 0x556b87251ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.756+0000 7f4706ae8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556b871c7bb0 con 0x556b86e831b0 2024-07-31T14:55:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.756+0000 7f4706ae8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556b871b70b0 con 0x556b86eac070 2024-07-31T14:55:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.756+0000 7f4706ae8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556b871bb0f0 con 0x556b86e8b390 2024-07-31T14:55:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.756+0000 7f46feb34640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86eac070 0x556b87251570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.756+0000 7f46ef7fe640 1 -- 172.21.15.67:0/3636606302 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f46f4004030 con 0x7f46f81042d0 2024-07-31T14:55:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.756+0000 7f46feb34640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86eac070 0x556b87251570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40778/0 (socket says 172.21.15.67:40778) 2024-07-31T14:55:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.756+0000 7f46fdb32640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e8b390 0x556b87251ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.756+0000 7f46feb34640 1 -- 172.21.15.67:0/1987884949 learned_addr learned my addr 172.21.15.67:0/1987884949 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.756+0000 7f46fe333640 1 --2- 172.21.15.67:0/1987884949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e831b0 0x556b87251ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.757+0000 7f46feb34640 1 -- 172.21.15.67:0/1987884949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e8b390 msgr2=0x556b87251ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:41.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.757+0000 7f46feb34640 1 --2- 172.21.15.67:0/1987884949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e8b390 0x556b87251ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.757+0000 7f46feb34640 1 -- 172.21.15.67:0/1987884949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e831b0 msgr2=0x556b87251ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:41.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.757+0000 7f46feb34640 1 --2- 172.21.15.67:0/1987884949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e831b0 0x556b87251ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.757+0000 7f46feb34640 1 -- 172.21.15.67:0/1987884949 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8715db90 con 0x556b86eac070 2024-07-31T14:55:41.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.757+0000 7f46feb34640 1 --2- 172.21.15.67:0/1987884949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86eac070 0x556b87251570 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009ca0 tx=0x7f46f0009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7174400f48987ff server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:41.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.757+0000 7f46ecff9640 1 -- 172.21.15.67:0/1987884949 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0031cf0 con 0x556b86eac070 2024-07-31T14:55:41.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.757+0000 7f46ecff9640 1 -- 172.21.15.67:0/1987884949 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f0036030 con 0x556b86eac070 2024-07-31T14:55:41.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.757+0000 7f46ecff9640 1 -- 172.21.15.67:0/1987884949 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f00042a0 con 0x556b86eac070 2024-07-31T14:55:41.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.758+0000 7f46ef7fe640 1 --2- 172.21.15.67:0/3636606302 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46e40b3d40 0x7f46e40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.758+0000 7f46ef7fe640 1 -- 172.21.15.67:0/3636606302 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f46f40e4810 con 0x7f46f81042d0 2024-07-31T14:55:41.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.758+0000 7f46fe333640 1 --2- 172.21.15.67:0/3636606302 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46e40b3d40 0x7f46e40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.759+0000 7f4706ae8740 1 -- 172.21.15.67:0/1987884949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86eac070 msgr2=0x556b87251570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:41.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.759+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1987884949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86eac070 0x556b87251570 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009ca0 tx=0x7f46f0009430 comp rx=0 tx=0).stop 2024-07-31T14:55:41.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.759+0000 7f4706ae8740 1 -- 172.21.15.67:0/1987884949 shutdown_connections 2024-07-31T14:55:41.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.759+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1987884949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e8b390 0x556b87251ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.759+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1987884949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e831b0 0x556b87251ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.759+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1987884949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86eac070 0x556b87251570 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.759+0000 7f4706ae8740 1 -- 172.21.15.67:0/1987884949 >> 172.21.15.67:0/1987884949 conn(0x556b86ea0460 msgr2=0x556b8724f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:41.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.759+0000 7f46fe333640 1 --2- 172.21.15.67:0/3636606302 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46e40b3d40 0x7f46e40b6250 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f46f40e1d00 tx=0x7f46f4003e70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:41.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.760+0000 7f4706ae8740 1 -- 172.21.15.67:0/1987884949 shutdown_connections 2024-07-31T14:55:41.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.760+0000 7f4706ae8740 1 -- 172.21.15.67:0/1987884949 wait complete. 2024-07-31T14:55:41.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.760+0000 7f4706ae8740 1 -- start start 2024-07-31T14:55:41.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.761+0000 7f4706ae8740 1 Processor -- start 2024-07-31T14:55:41.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.761+0000 7f4706ae8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e831b0 0x556b8725d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.761+0000 7f4706ae8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e8b390 0x556b8725d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.761+0000 7f4706ae8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86eac070 0x556b87268e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.761+0000 7f46feb34640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e831b0 0x556b8725d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.761+0000 7f46feb34640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e831b0 0x556b8725d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40794/0 (socket says 172.21.15.67:40794) 2024-07-31T14:55:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.761+0000 7f46feb34640 1 -- 172.21.15.67:0/1712248165 learned_addr learned my addr 172.21.15.67:0/1712248165 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.761+0000 7f46fdb32640 1 --2- 172.21.15.67:0/1712248165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e8b390 0x556b8725d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.761+0000 7f4706ae8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556b8701a700 con 0x556b86eac070 2024-07-31T14:55:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.762+0000 7f4706ae8740 1 -- 172.21.15.67:0/1712248165 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556b871599e0 con 0x556b86e831b0 2024-07-31T14:55:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.762+0000 7f4706ae8740 1 -- 172.21.15.67:0/1712248165 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556b8725fb70 con 0x556b86e8b390 2024-07-31T14:55:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.762+0000 7f46feb34640 1 --2- 172.21.15.67:0/1712248165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86eac070 0x556b87268e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.762+0000 7f46fdb32640 1 -- 172.21.15.67:0/1712248165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e831b0 msgr2=0x556b8725d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.762+0000 7f46fdb32640 1 --2- 172.21.15.67:0/1712248165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e831b0 0x556b8725d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.762+0000 7f46fdb32640 1 -- 172.21.15.67:0/1712248165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86eac070 msgr2=0x556b87268e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.762+0000 7f46fdb32640 1 --2- 172.21.15.67:0/1712248165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86eac070 0x556b87268e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.762+0000 7f46fdb32640 1 -- 172.21.15.67:0/1712248165 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b86f17650 con 0x556b86e8b390 2024-07-31T14:55:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.762+0000 7f46fdb32640 1 --2- 172.21.15.67:0/1712248165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e8b390 0x556b8725d640 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f46f806b380 tx=0x7f46e800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d98bf9f503620f75 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.762+0000 7f46e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:55:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.762+0000 7f4706ae8740 10 client.0 _ll_register_callbacks cb 0x556b871744f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.762+0000 7f4706ae8740 1 -- 172.21.15.67:0/1712248165 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46f0036030 con 0x556b86e8b390 2024-07-31T14:55:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.762+0000 7f46e1ffb640 1 -- 172.21.15.67:0/1712248165 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8012110 con 0x556b86e8b390 2024-07-31T14:55:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.763+0000 7f46e1ffb640 1 -- 172.21.15.67:0/1712248165 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f46e8010430 con 0x556b86e8b390 2024-07-31T14:55:41.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.763+0000 7f46e1ffb640 1 -- 172.21.15.67:0/1712248165 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f46e80122b0 con 0x556b86e8b390 2024-07-31T14:55:41.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.763+0000 7f4706ae8740 10 client.26458 fetch_fsmap learned FSMap version 42 2024-07-31T14:55:41.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.763+0000 7f4706ae8740 1 -- 172.21.15.67:0/1712248165 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=42}) -- 0x556b8726b9f0 con 0x556b86e8b390 2024-07-31T14:55:41.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.763+0000 7f46e1ffb640 1 -- 172.21.15.67:0/1712248165 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f46e8010650 con 0x556b86e8b390 2024-07-31T14:55:41.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.764+0000 7f4706ae8740 10 client.26458 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:55:41.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.764+0000 7f4706ae8740 10 client.26458 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:55:41.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.764+0000 7f4706ae8740 20 client.26458 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:41.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.764+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1712248165 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556b86e6a9f0 0x556b8726efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.764+0000 7f4706ae8740 4 client.26458 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:55:41.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.764+0000 7f4706ae8740 1 -- 172.21.15.67:0/1712248165 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b871c9b00 con 0x556b86e6a9f0 2024-07-31T14:55:41.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.764+0000 7f46feb34640 1 --2- 172.21.15.67:0/1712248165 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556b86e6a9f0 0x556b8726efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.765+0000 7f46feb34640 1 --2- 172.21.15.67:0/1712248165 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556b86e6a9f0 0x556b8726efc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d600f33258e6e1 server_cookie=1c82ddd635594267 in_seq=0 out_seq=0 2024-07-31T14:55:41.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.765+0000 7f46e1ffb640 10 client.26458 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:55:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f46e1ffb640 1 -- 172.21.15.67:0/1712248165 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b871c9b00 con 0x556b86e6a9f0 2024-07-31T14:55:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f46e1ffb640 10 client.26458 handle_command_reply: tid=0 2024-07-31T14:55:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f4706ae8740 1 client.26458 shutdown 2024-07-31T14:55:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f4706ae8740 1 -- 172.21.15.67:0/1712248165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e8b390 msgr2=0x556b8725d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1712248165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e8b390 0x556b8725d640 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f46f806b380 tx=0x7f46e800f790 comp rx=0 tx=0).stop 2024-07-31T14:55:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f4706ae8740 1 -- 172.21.15.67:0/1712248165 shutdown_connections 2024-07-31T14:55:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1712248165 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556b86e6a9f0 0x556b8726efc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1712248165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86eac070 0x556b87268e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1712248165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e8b390 0x556b8725d640 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1712248165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e831b0 0x556b8725d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f46e1ffb640 0 client.26458 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:55:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f4706ae8740 1 -- 172.21.15.67:0/1712248165 >> 172.21.15.67:0/1712248165 conn(0x556b86ea0460 msgr2=0x556b8724dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:41.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f4706ae8740 1 -- 172.21.15.67:0/1712248165 shutdown_connections 2024-07-31T14:55:41.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f4706ae8740 1 -- 172.21.15.67:0/1712248165 wait complete. 2024-07-31T14:55:41.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.787+0000 7f4706ae8740 20 client.26458 trim_cache size 0 max 16384 2024-07-31T14:55:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.803+0000 7f4706ae8740 1 Processor -- start 2024-07-31T14:55:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.803+0000 7f4706ae8740 1 -- start start 2024-07-31T14:55:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f4706ae8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e6a9f0 0x556b8724e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f4706ae8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e831b0 0x556b87257690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f4706ae8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e8b390 0x556b87268c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f4706ae8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f46f81b2860 con 0x556b86e8b390 2024-07-31T14:55:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f4706ae8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46f81b4520 con 0x556b86e6a9f0 2024-07-31T14:55:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f4706ae8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46f81b2f30 con 0x556b86e831b0 2024-07-31T14:55:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f46feb34640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e6a9f0 0x556b8724e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f46fdb32640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e831b0 0x556b87257690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f46feb34640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e6a9f0 0x556b8724e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40810/0 (socket says 172.21.15.67:40810) 2024-07-31T14:55:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f46feb34640 1 -- 172.21.15.67:0/4040423421 learned_addr learned my addr 172.21.15.67:0/4040423421 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f46feb34640 1 --2- 172.21.15.67:0/4040423421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e8b390 0x556b87268c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f46feb34640 1 -- 172.21.15.67:0/4040423421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e831b0 msgr2=0x556b87257690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f46feb34640 1 --2- 172.21.15.67:0/4040423421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e831b0 0x556b87257690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f46feb34640 1 -- 172.21.15.67:0/4040423421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e8b390 msgr2=0x556b87268c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f46feb34640 1 --2- 172.21.15.67:0/4040423421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e8b390 0x556b87268c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.804+0000 7f46feb34640 1 -- 172.21.15.67:0/4040423421 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8725de30 con 0x556b86e6a9f0 2024-07-31T14:55:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.805+0000 7f46feb34640 1 --2- 172.21.15.67:0/4040423421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e6a9f0 0x556b8724e350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46f0003cb0 tx=0x7f46f0003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa4b18a6c7566826 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.805+0000 7f46e3fff640 1 -- 172.21.15.67:0/4040423421 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0031c60 con 0x556b86e6a9f0 2024-07-31T14:55:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.805+0000 7f46e3fff640 1 -- 172.21.15.67:0/4040423421 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f0031900 con 0x556b86e6a9f0 2024-07-31T14:55:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.805+0000 7f46e3fff640 1 -- 172.21.15.67:0/4040423421 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f002fd00 con 0x556b86e6a9f0 2024-07-31T14:55:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.806+0000 7f4706ae8740 1 -- 172.21.15.67:0/4040423421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e6a9f0 msgr2=0x556b8724e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.806+0000 7f4706ae8740 1 --2- 172.21.15.67:0/4040423421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e6a9f0 0x556b8724e350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46f0003cb0 tx=0x7f46f0003ce0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.806+0000 7f4706ae8740 1 -- 172.21.15.67:0/4040423421 shutdown_connections 2024-07-31T14:55:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.806+0000 7f4706ae8740 1 --2- 172.21.15.67:0/4040423421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e8b390 0x556b87268c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.806+0000 7f4706ae8740 1 --2- 172.21.15.67:0/4040423421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e831b0 0x556b87257690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.806+0000 7f4706ae8740 1 --2- 172.21.15.67:0/4040423421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e6a9f0 0x556b8724e350 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.806+0000 7f4706ae8740 1 -- 172.21.15.67:0/4040423421 >> 172.21.15.67:0/4040423421 conn(0x556b86ea0460 msgr2=0x556b8724dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.806+0000 7f4706ae8740 1 -- 172.21.15.67:0/4040423421 shutdown_connections 2024-07-31T14:55:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.806+0000 7f4706ae8740 1 -- 172.21.15.67:0/4040423421 wait complete. 2024-07-31T14:55:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.807+0000 7f4706ae8740 1 -- start start 2024-07-31T14:55:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.807+0000 7f4706ae8740 1 Processor -- start 2024-07-31T14:55:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.807+0000 7f4706ae8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e6a9f0 0x556b87265450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.807+0000 7f4706ae8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e831b0 0x556b87260900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.807+0000 7f4706ae8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e8b390 0x556b87260e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.807+0000 7f4706ae8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f46f81b3e50 con 0x556b86e8b390 2024-07-31T14:55:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.807+0000 7f4706ae8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46f81b36c0 con 0x556b86e831b0 2024-07-31T14:55:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.807+0000 7f4706ae8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46f81b0840 con 0x556b86e6a9f0 2024-07-31T14:55:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.807+0000 7f4706ae8740 10 client.0 _ll_register_callbacks cb 0x7f46e8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.807+0000 7f46feb34640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e6a9f0 0x556b87265450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.808+0000 7f46fdb32640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e831b0 0x556b87260900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.808+0000 7f46feb34640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e6a9f0 0x556b87265450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49656/0 (socket says 172.21.15.67:49656) 2024-07-31T14:55:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.808+0000 7f46feb34640 1 -- 172.21.15.67:0/1207417134 learned_addr learned my addr 172.21.15.67:0/1207417134 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.808+0000 7f46feb34640 1 --2- 172.21.15.67:0/1207417134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e8b390 0x556b87260e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.808+0000 7f46fdb32640 1 -- 172.21.15.67:0/1207417134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e6a9f0 msgr2=0x556b87265450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.808+0000 7f46fdb32640 1 --2- 172.21.15.67:0/1207417134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e6a9f0 0x556b87265450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.808+0000 7f46fdb32640 1 -- 172.21.15.67:0/1207417134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e8b390 msgr2=0x556b87260e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.808+0000 7f46fdb32640 1 --2- 172.21.15.67:0/1207417134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e8b390 0x556b87260e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.808+0000 7f46fdb32640 1 -- 172.21.15.67:0/1207417134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b87261580 con 0x556b86e831b0 2024-07-31T14:55:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.808+0000 7f46fdb32640 1 --2- 172.21.15.67:0/1207417134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e831b0 0x556b87260900 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f46e800f7d0 tx=0x7f46e80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e07a341cec015e21 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.808+0000 7f46e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:55:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.808+0000 7f46e1ffb640 1 -- 172.21.15.67:0/1207417134 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8012110 con 0x556b86e831b0 2024-07-31T14:55:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.808+0000 7f4706ae8740 1 -- 172.21.15.67:0/1207417134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556b871c55a0 con 0x556b86e831b0 2024-07-31T14:55:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.808+0000 7f46e1ffb640 1 -- 172.21.15.67:0/1207417134 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f46e8018040 con 0x556b86e831b0 2024-07-31T14:55:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.809+0000 7f46e1ffb640 1 -- 172.21.15.67:0/1207417134 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f46e803a050 con 0x556b86e831b0 2024-07-31T14:55:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.809+0000 7f4706ae8740 10 client.26435 fetch_fsmap learned FSMap version 42 2024-07-31T14:55:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.809+0000 7f4706ae8740 1 -- 172.21.15.67:0/1207417134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556b872d7640 con 0x556b86e831b0 2024-07-31T14:55:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.809+0000 7f46e1ffb640 1 -- 172.21.15.67:0/1207417134 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556b871c55a0 con 0x556b86e831b0 2024-07-31T14:55:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.809+0000 7f46e1ffb640 1 -- 172.21.15.67:0/1207417134 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f46e8005230 con 0x556b86e831b0 2024-07-31T14:55:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.809+0000 7f4706ae8740 10 client.26435 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:55:41.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.809+0000 7f4706ae8740 10 client.26435 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:55:41.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.809+0000 7f4706ae8740 20 client.26435 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:41.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.809+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1207417134 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556b86eac070 0x556b872d9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:41.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.809+0000 7f4706ae8740 4 client.26435 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:55:41.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.809+0000 7f4706ae8740 1 -- 172.21.15.67:0/1207417134 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b871c2ce0 con 0x556b86eac070 2024-07-31T14:55:41.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.809+0000 7f46feb34640 1 --2- 172.21.15.67:0/1207417134 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556b86eac070 0x556b872d9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:41.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.810+0000 7f46e1ffb640 10 client.26435 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:55:41.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.810+0000 7f46feb34640 1 --2- 172.21.15.67:0/1207417134 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556b86eac070 0x556b872d9dd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3f29c2f6a10de20 server_cookie=a2368f34cf0ff641 in_seq=0 out_seq=0 2024-07-31T14:55:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f46e1ffb640 1 -- 172.21.15.67:0/1207417134 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b871c2ce0 con 0x556b86eac070 2024-07-31T14:55:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f46e1ffb640 10 client.26435 handle_command_reply: tid=0 2024-07-31T14:55:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f4706ae8740 1 client.26435 shutdown 2024-07-31T14:55:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f4706ae8740 1 -- 172.21.15.67:0/1207417134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e831b0 msgr2=0x556b87260900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1207417134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e831b0 0x556b87260900 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f46e800f7d0 tx=0x7f46e80046c0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f4706ae8740 1 -- 172.21.15.67:0/1207417134 shutdown_connections 2024-07-31T14:55:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1207417134 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556b86eac070 0x556b872d9dd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1207417134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556b86e8b390 0x556b87260e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1207417134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b86e831b0 0x556b87260900 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f4706ae8740 1 --2- 172.21.15.67:0/1207417134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556b86e6a9f0 0x556b87265450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f4706ae8740 1 -- 172.21.15.67:0/1207417134 >> 172.21.15.67:0/1207417134 conn(0x556b86ea0460 msgr2=0x556b8725d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f46e1ffb640 0 client.26435 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:55:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f4706ae8740 1 -- 172.21.15.67:0/1207417134 shutdown_connections 2024-07-31T14:55:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f4706ae8740 1 -- 172.21.15.67:0/1207417134 wait complete. 2024-07-31T14:55:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.825+0000 7f4706ae8740 20 client.26435 trim_cache size 0 max 16384 2024-07-31T14:55:41.828 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:55:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.826+0000 7f4700597640 1 -- 172.21.15.67:0/3636606302 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46e40b3d40 msgr2=0x7f46e40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.826+0000 7f4700597640 1 --2- 172.21.15.67:0/3636606302 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46e40b3d40 0x7f46e40b6250 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f46f40e1d00 tx=0x7f46f4003e70 comp rx=0 tx=0).stop 2024-07-31T14:55:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.826+0000 7f4700597640 1 -- 172.21.15.67:0/3636606302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f81042d0 msgr2=0x7f46f819c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.826+0000 7f4700597640 1 --2- 172.21.15.67:0/3636606302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f81042d0 0x7f46f819c040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f46f400e9c0 tx=0x7f46f400ee80 comp rx=0 tx=0).stop 2024-07-31T14:55:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.826+0000 7f4700597640 1 -- 172.21.15.67:0/3636606302 shutdown_connections 2024-07-31T14:55:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.826+0000 7f4700597640 1 --2- 172.21.15.67:0/3636606302 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46e40b3d40 0x7f46e40b6250 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.826+0000 7f4700597640 1 --2- 172.21.15.67:0/3636606302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f810d880 0x7f46f81a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.826+0000 7f4700597640 1 --2- 172.21.15.67:0/3636606302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46f8104d00 0x7f46f819c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.826+0000 7f4700597640 1 --2- 172.21.15.67:0/3636606302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f81042d0 0x7f46f819c040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:41.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.826+0000 7f4700597640 1 -- 172.21.15.67:0/3636606302 >> 172.21.15.67:0/3636606302 conn(0x7f46f8079170 msgr2=0x7f46f810c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:41.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.827+0000 7f4700597640 1 -- 172.21.15.67:0/3636606302 shutdown_connections 2024-07-31T14:55:41.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:41.827+0000 7f4700597640 1 -- 172.21.15.67:0/3636606302 wait complete. 2024-07-31T14:55:42.661 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T14:55:42.945 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:42 smithi067 ceph-mon[32750]: pgmap v539: 129 pgs: 129 active+clean; 865 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 21 MiB/s wr, 930 op/s 2024-07-31T14:55:43.008 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:55:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:42 smithi077 ceph-mon[37327]: pgmap v539: 129 pgs: 129 active+clean; 865 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 21 MiB/s wr, 930 op/s 2024-07-31T14:55:43.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:42 smithi073 ceph-mon[36714]: pgmap v539: 129 pgs: 129 active+clean; 865 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 21 MiB/s wr, 930 op/s 2024-07-31T14:55:45.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:44 smithi077 ceph-mon[37327]: pgmap v540: 129 pgs: 129 active+clean; 907 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 23 MiB/s wr, 1.16k op/s 2024-07-31T14:55:45.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:44 smithi067 ceph-mon[32750]: pgmap v540: 129 pgs: 129 active+clean; 907 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 23 MiB/s wr, 1.16k op/s 2024-07-31T14:55:45.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:44 smithi073 ceph-mon[36714]: pgmap v540: 129 pgs: 129 active+clean; 907 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 23 MiB/s wr, 1.16k op/s 2024-07-31T14:55:46.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.258+0000 7f86eaed2640 1 -- 172.21.15.67:0/3717817971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e4108730 msgr2=0x7f86e410ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:46.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.258+0000 7f86eaed2640 1 --2- 172.21.15.67:0/3717817971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e4108730 0x7f86e410ef60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009ca0 tx=0x7f86dc02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.259+0000 7f86eaed2640 1 -- 172.21.15.67:0/3717817971 shutdown_connections 2024-07-31T14:55:46.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.259+0000 7f86eaed2640 1 --2- 172.21.15.67:0/3717817971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e4108730 0x7f86e410ef60 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.259+0000 7f86eaed2640 1 --2- 172.21.15.67:0/3717817971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e4107de0 0x7f86e41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.259+0000 7f86eaed2640 1 --2- 172.21.15.67:0/3717817971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e41024a0 0x7f86e41028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.259+0000 7f86eaed2640 1 -- 172.21.15.67:0/3717817971 >> 172.21.15.67:0/3717817971 conn(0x7f86e4079160 msgr2=0x7f86e4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:46.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.259+0000 7f86eaed2640 1 -- 172.21.15.67:0/3717817971 shutdown_connections 2024-07-31T14:55:46.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.259+0000 7f86eaed2640 1 -- 172.21.15.67:0/3717817971 wait complete. 2024-07-31T14:55:46.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.260+0000 7f86eaed2640 1 Processor -- start 2024-07-31T14:55:46.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.260+0000 7f86eaed2640 1 -- start start 2024-07-31T14:55:46.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86eaed2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e41024a0 0x7f86e4198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86eaed2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e4107de0 0x7f86e4199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86eaed2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e4108730 0x7f86e419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86eaed2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41123b0 con 0x7f86e4108730 2024-07-31T14:55:46.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86eaed2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4112230 con 0x7f86e4107de0 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86eaed2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4112530 con 0x7f86e41024a0 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86e3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e4107de0 0x7f86e4199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86e8c6e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e41024a0 0x7f86e4198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86e946f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e4108730 0x7f86e419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86e3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e4107de0 0x7f86e4199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40846/0 (socket says 172.21.15.67:40846) 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86e8c6e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e41024a0 0x7f86e4198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49678/0 (socket says 172.21.15.67:49678) 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86e3fff640 1 -- 172.21.15.67:0/3758643014 learned_addr learned my addr 172.21.15.67:0/3758643014 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86e3fff640 1 -- 172.21.15.67:0/3758643014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e41024a0 msgr2=0x7f86e4198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86e3fff640 1 --2- 172.21.15.67:0/3758643014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e41024a0 0x7f86e4198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86e3fff640 1 -- 172.21.15.67:0/3758643014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e4108730 msgr2=0x7f86e419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86e3fff640 1 --2- 172.21.15.67:0/3758643014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e4108730 0x7f86e419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86e3fff640 1 -- 172.21.15.67:0/3758643014 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e419edf0 con 0x7f86e4107de0 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.261+0000 7f86e3fff640 1 --2- 172.21.15.67:0/3758643014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e4107de0 0x7f86e4199420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f86d000ead0 tx=0x7f86d000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.262+0000 7f86e1ffb640 1 -- 172.21.15.67:0/3758643014 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d000bbd0 con 0x7f86e4107de0 2024-07-31T14:55:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.262+0000 7f86eaed2640 1 -- 172.21.15.67:0/3758643014 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e419f0e0 con 0x7f86e4107de0 2024-07-31T14:55:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.262+0000 7f86e1ffb640 1 -- 172.21.15.67:0/3758643014 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86d000bd70 con 0x7f86e4107de0 2024-07-31T14:55:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.262+0000 7f86eaed2640 1 -- 172.21.15.67:0/3758643014 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e406b510 con 0x7f86e4107de0 2024-07-31T14:55:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.262+0000 7f86e1ffb640 1 -- 172.21.15.67:0/3758643014 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0017410 con 0x7f86e4107de0 2024-07-31T14:55:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.265+0000 7f86e1ffb640 1 -- 172.21.15.67:0/3758643014 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f86d0004e90 con 0x7f86e4107de0 2024-07-31T14:55:46.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.266+0000 7f86f1423740 1 Processor -- start 2024-07-31T14:55:46.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.266+0000 7f86f1423740 1 -- start start 2024-07-31T14:55:46.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.266+0000 7f86e1ffb640 1 --2- 172.21.15.67:0/3758643014 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86d40b3d40 0x7f86d40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.266+0000 7f86f1423740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb404ad070 0x55bb40852570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.266+0000 7f86f1423740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404841b0 0x55bb40852ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.266+0000 7f86e946f640 1 --2- 172.21.15.67:0/3758643014 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86d40b3d40 0x7f86d40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.266+0000 7f86e1ffb640 1 -- 172.21.15.67:0/3758643014 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f86d00e5a40 con 0x7f86e4107de0 2024-07-31T14:55:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.266+0000 7f86f1423740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb4048c390 0x55bb40852ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.266+0000 7f86f1423740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bb407c8bb0 con 0x55bb404841b0 2024-07-31T14:55:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.266+0000 7f86f1423740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bb407b80b0 con 0x55bb4048c390 2024-07-31T14:55:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.267+0000 7f86f1423740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bb407bc0f0 con 0x55bb404ad070 2024-07-31T14:55:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.267+0000 7f86e3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404841b0 0x55bb40852ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.267+0000 7f86e3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404841b0 0x55bb40852ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44110/0 (socket says 172.21.15.67:44110) 2024-07-31T14:55:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.267+0000 7f86e3fff640 1 -- 172.21.15.67:0/3633690696 learned_addr learned my addr 172.21.15.67:0/3633690696 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.267+0000 7f86e8c6e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb404ad070 0x55bb40852570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.267+0000 7f86e946f640 1 --2- 172.21.15.67:0/3633690696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb4048c390 0x55bb40852ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.267+0000 7f86e3fff640 1 -- 172.21.15.67:0/3633690696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb404ad070 msgr2=0x55bb40852570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.267+0000 7f86e3fff640 1 --2- 172.21.15.67:0/3633690696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb404ad070 0x55bb40852570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.267+0000 7f86e3fff640 1 -- 172.21.15.67:0/3633690696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb4048c390 msgr2=0x55bb40852ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.267+0000 7f86e3fff640 1 --2- 172.21.15.67:0/3633690696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb4048c390 0x55bb40852ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.267+0000 7f86e3fff640 1 -- 172.21.15.67:0/3633690696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4075eb90 con 0x55bb404841b0 2024-07-31T14:55:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.267+0000 7f86e946f640 1 --2- 172.21.15.67:0/3758643014 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86d40b3d40 0x7f86d40b6250 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f86dc006010 tx=0x7f86dc031bd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.267+0000 7f86e8c6e640 1 --2- 172.21.15.67:0/3633690696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb404ad070 0x55bb40852570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.267+0000 7f86e3fff640 1 --2- 172.21.15.67:0/3633690696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404841b0 0x55bb40852ab0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f86d00a8470 tx=0x7f86d00ac4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b420f19ded2c77a1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.268+0000 7f86ceffd640 1 -- 172.21.15.67:0/3633690696 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d00b53f0 con 0x55bb404841b0 2024-07-31T14:55:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.268+0000 7f86ceffd640 1 -- 172.21.15.67:0/3633690696 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86d00055f0 con 0x55bb404841b0 2024-07-31T14:55:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.268+0000 7f86ceffd640 1 -- 172.21.15.67:0/3633690696 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d00053c0 con 0x55bb404841b0 2024-07-31T14:55:46.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.269+0000 7f86f1423740 1 -- 172.21.15.67:0/3633690696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404841b0 msgr2=0x55bb40852ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:46.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.269+0000 7f86f1423740 1 --2- 172.21.15.67:0/3633690696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404841b0 0x55bb40852ab0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f86d00a8470 tx=0x7f86d00ac4b0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.269+0000 7f86f1423740 1 -- 172.21.15.67:0/3633690696 shutdown_connections 2024-07-31T14:55:46.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.269+0000 7f86f1423740 1 --2- 172.21.15.67:0/3633690696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb4048c390 0x55bb40852ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.269+0000 7f86f1423740 1 --2- 172.21.15.67:0/3633690696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404841b0 0x55bb40852ab0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.269+0000 7f86f1423740 1 --2- 172.21.15.67:0/3633690696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb404ad070 0x55bb40852570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.269+0000 7f86f1423740 1 -- 172.21.15.67:0/3633690696 >> 172.21.15.67:0/3633690696 conn(0x55bb404a1460 msgr2=0x55bb40850460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:46.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.269+0000 7f86f1423740 1 -- 172.21.15.67:0/3633690696 shutdown_connections 2024-07-31T14:55:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.269+0000 7f86f1423740 1 -- 172.21.15.67:0/3633690696 wait complete. 2024-07-31T14:55:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.269+0000 7f86f1423740 1 -- start start 2024-07-31T14:55:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.269+0000 7f86f1423740 1 Processor -- start 2024-07-31T14:55:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.270+0000 7f86f1423740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404841b0 0x55bb4085e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.270+0000 7f86f1423740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4048c390 0x55bb4085e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.270+0000 7f86f1423740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404ad070 0x55bb4086a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.270+0000 7f86f1423740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bb4061b700 con 0x55bb404841b0 2024-07-31T14:55:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.270+0000 7f86f1423740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bb4075a9e0 con 0x55bb404ad070 2024-07-31T14:55:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.270+0000 7f86f1423740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bb40860d50 con 0x55bb4048c390 2024-07-31T14:55:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.270+0000 7f86f1423740 10 client.0 _ll_register_callbacks cb 0x55bb407754f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.270+0000 7f86e8c6e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404841b0 0x55bb4085e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.270+0000 7f86e3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4048c390 0x55bb4085e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.270+0000 7f86e8c6e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404841b0 0x55bb4085e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44118/0 (socket says 172.21.15.67:44118) 2024-07-31T14:55:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.270+0000 7f86e8c6e640 1 -- 172.21.15.67:0/2283487992 learned_addr learned my addr 172.21.15.67:0/2283487992 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.270+0000 7f86e946f640 1 --2- 172.21.15.67:0/2283487992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404ad070 0x55bb4086a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.271+0000 7f86e3fff640 1 -- 172.21.15.67:0/2283487992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404ad070 msgr2=0x55bb4086a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.271+0000 7f86e3fff640 1 --2- 172.21.15.67:0/2283487992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404ad070 0x55bb4086a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.271+0000 7f86e3fff640 1 -- 172.21.15.67:0/2283487992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404841b0 msgr2=0x55bb4085e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.271+0000 7f86e3fff640 1 --2- 172.21.15.67:0/2283487992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404841b0 0x55bb4085e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.271+0000 7f86e3fff640 1 -- 172.21.15.67:0/2283487992 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb40518650 con 0x55bb4048c390 2024-07-31T14:55:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.271+0000 7f86e8c6e640 1 --2- 172.21.15.67:0/2283487992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404841b0 0x55bb4085e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.271+0000 7f86e3fff640 1 --2- 172.21.15.67:0/2283487992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4048c390 0x55bb4085e620 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f86d00b0b00 tx=0x7f86d0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=739a042fbaab84b3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.271+0000 7f86c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:55:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.271+0000 7f86c3fff640 1 -- 172.21.15.67:0/2283487992 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d00ca030 con 0x55bb4048c390 2024-07-31T14:55:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.271+0000 7f86f1423740 1 -- 172.21.15.67:0/2283487992 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86d00055f0 con 0x55bb4048c390 2024-07-31T14:55:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.271+0000 7f86c3fff640 1 -- 172.21.15.67:0/2283487992 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f86d00a89e0 con 0x55bb4048c390 2024-07-31T14:55:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.271+0000 7f86c3fff640 1 -- 172.21.15.67:0/2283487992 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f86d0005390 con 0x55bb4048c390 2024-07-31T14:55:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.271+0000 7f86f1423740 10 client.26476 fetch_fsmap learned FSMap version 42 2024-07-31T14:55:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.271+0000 7f86f1423740 1 -- 172.21.15.67:0/2283487992 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bb4086cd10 con 0x55bb4048c390 2024-07-31T14:55:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.272+0000 7f86c3fff640 1 -- 172.21.15.67:0/2283487992 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86d00055f0 con 0x55bb4048c390 2024-07-31T14:55:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.272+0000 7f86c3fff640 1 -- 172.21.15.67:0/2283487992 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f86d00a7d60 con 0x55bb4048c390 2024-07-31T14:55:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.272+0000 7f86f1423740 10 client.26476 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:55:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.272+0000 7f86f1423740 10 client.26476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.272+0000 7f86f1423740 20 client.26476 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.272+0000 7f86f1423740 1 --2- 172.21.15.67:0/2283487992 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb4046b9f0 0x55bb40870070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.272+0000 7f86f1423740 4 client.26476 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:55:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.272+0000 7f86f1423740 1 -- 172.21.15.67:0/2283487992 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb407cab00 con 0x55bb4046b9f0 2024-07-31T14:55:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.272+0000 7f86e8c6e640 1 --2- 172.21.15.67:0/2283487992 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb4046b9f0 0x55bb40870070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.273+0000 7f86c3fff640 10 client.26476 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:46.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.273+0000 7f86e8c6e640 1 --2- 172.21.15.67:0/2283487992 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb4046b9f0 0x55bb40870070 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab10f8a1a2253fd8 server_cookie=ca2dd23e63d82171 in_seq=0 out_seq=0 2024-07-31T14:55:46.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.274+0000 7f86c3fff640 1 -- 172.21.15.67:0/2283487992 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb407cab00 con 0x55bb4046b9f0 2024-07-31T14:55:46.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.274+0000 7f86c3fff640 10 client.26476 handle_command_reply: tid=0 2024-07-31T14:55:46.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.274+0000 7f86f1423740 1 client.26476 shutdown 2024-07-31T14:55:46.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.274+0000 7f86f1423740 1 -- 172.21.15.67:0/2283487992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4048c390 msgr2=0x55bb4085e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:46.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.274+0000 7f86f1423740 1 --2- 172.21.15.67:0/2283487992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4048c390 0x55bb4085e620 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f86d00b0b00 tx=0x7f86d0009780 comp rx=0 tx=0).stop 2024-07-31T14:55:46.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.275+0000 7f86f1423740 1 -- 172.21.15.67:0/2283487992 shutdown_connections 2024-07-31T14:55:46.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.275+0000 7f86f1423740 1 --2- 172.21.15.67:0/2283487992 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb4046b9f0 0x55bb40870070 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.275+0000 7f86f1423740 1 --2- 172.21.15.67:0/2283487992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404ad070 0x55bb4086a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.275+0000 7f86f1423740 1 --2- 172.21.15.67:0/2283487992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4048c390 0x55bb4085e620 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.275+0000 7f86c3fff640 0 client.26476 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:46.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.275+0000 7f86f1423740 1 --2- 172.21.15.67:0/2283487992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404841b0 0x55bb4085e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.275+0000 7f86f1423740 1 -- 172.21.15.67:0/2283487992 >> 172.21.15.67:0/2283487992 conn(0x55bb404a1460 msgr2=0x55bb4084ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:46.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.275+0000 7f86f1423740 1 -- 172.21.15.67:0/2283487992 shutdown_connections 2024-07-31T14:55:46.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.275+0000 7f86f1423740 1 -- 172.21.15.67:0/2283487992 wait complete. 2024-07-31T14:55:46.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.275+0000 7f86f1423740 20 client.26476 trim_cache size 0 max 16384 2024-07-31T14:55:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.289+0000 7f86f1423740 1 Processor -- start 2024-07-31T14:55:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.289+0000 7f86f1423740 1 -- start start 2024-07-31T14:55:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.289+0000 7f86f1423740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404841b0 0x55bb40850560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.289+0000 7f86f1423740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4048c390 0x55bb40858690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.289+0000 7f86f1423740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404ad070 0x55bb40869fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.289+0000 7f86f1423740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41ae510 con 0x55bb404ad070 2024-07-31T14:55:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.289+0000 7f86f1423740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41b01d0 con 0x55bb404841b0 2024-07-31T14:55:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.289+0000 7f86f1423740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41aebe0 con 0x55bb4048c390 2024-07-31T14:55:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.290+0000 7f86e946f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404ad070 0x55bb40869fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.290+0000 7f86e3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4048c390 0x55bb40858690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.290+0000 7f86e946f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404ad070 0x55bb40869fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44132/0 (socket says 172.21.15.67:44132) 2024-07-31T14:55:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.290+0000 7f86e3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4048c390 0x55bb40858690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49706/0 (socket says 172.21.15.67:49706) 2024-07-31T14:55:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.290+0000 7f86e8c6e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404841b0 0x55bb40850560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.290+0000 7f86e946f640 1 -- 172.21.15.67:0/2354000913 learned_addr learned my addr 172.21.15.67:0/2354000913 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.290+0000 7f86e946f640 1 -- 172.21.15.67:0/2354000913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4048c390 msgr2=0x55bb40858690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.290+0000 7f86e946f640 1 --2- 172.21.15.67:0/2354000913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4048c390 0x55bb40858690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.290+0000 7f86e946f640 1 -- 172.21.15.67:0/2354000913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404841b0 msgr2=0x55bb40850560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.290+0000 7f86e946f640 1 --2- 172.21.15.67:0/2354000913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404841b0 0x55bb40850560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.290+0000 7f86e946f640 1 -- 172.21.15.67:0/2354000913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4085f1a0 con 0x55bb404ad070 2024-07-31T14:55:46.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.290+0000 7f86e946f640 1 --2- 172.21.15.67:0/2354000913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404ad070 0x55bb40869fb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009a60 tx=0x7f86dc049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1dc92af48d747c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:46.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.291+0000 7f86ce7fc640 1 -- 172.21.15.67:0/2354000913 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc045040 con 0x55bb404ad070 2024-07-31T14:55:46.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.291+0000 7f86ce7fc640 1 -- 172.21.15.67:0/2354000913 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86dc003c30 con 0x55bb404ad070 2024-07-31T14:55:46.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.291+0000 7f86ce7fc640 1 -- 172.21.15.67:0/2354000913 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc02f990 con 0x55bb404ad070 2024-07-31T14:55:46.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.292+0000 7f86f1423740 1 -- 172.21.15.67:0/2354000913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404ad070 msgr2=0x55bb40869fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:46.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.292+0000 7f86f1423740 1 --2- 172.21.15.67:0/2354000913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404ad070 0x55bb40869fb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009a60 tx=0x7f86dc049000 comp rx=0 tx=0).stop 2024-07-31T14:55:46.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.292+0000 7f86f1423740 1 -- 172.21.15.67:0/2354000913 shutdown_connections 2024-07-31T14:55:46.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.292+0000 7f86f1423740 1 --2- 172.21.15.67:0/2354000913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb404ad070 0x55bb40869fb0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.292+0000 7f86f1423740 1 --2- 172.21.15.67:0/2354000913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4048c390 0x55bb40858690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.292+0000 7f86f1423740 1 --2- 172.21.15.67:0/2354000913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404841b0 0x55bb40850560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.292+0000 7f86f1423740 1 -- 172.21.15.67:0/2354000913 >> 172.21.15.67:0/2354000913 conn(0x55bb404a1460 msgr2=0x55bb4084fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:46.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.292+0000 7f86f1423740 1 -- 172.21.15.67:0/2354000913 shutdown_connections 2024-07-31T14:55:46.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.292+0000 7f86f1423740 1 -- 172.21.15.67:0/2354000913 wait complete. 2024-07-31T14:55:46.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.293+0000 7f86f1423740 1 -- start start 2024-07-31T14:55:46.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.293+0000 7f86f1423740 1 Processor -- start 2024-07-31T14:55:46.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.293+0000 7f86f1423740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb404841b0 0x55bb40873980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.293+0000 7f86f1423740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb4048c390 0x55bb40870e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.293+0000 7f86f1423740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404ad070 0x55bb408715a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.294+0000 7f86f1423740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41afb00 con 0x55bb4048c390 2024-07-31T14:55:46.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.294+0000 7f86f1423740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41af370 con 0x55bb404ad070 2024-07-31T14:55:46.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.294+0000 7f86f1423740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41ac4f0 con 0x55bb404841b0 2024-07-31T14:55:46.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.294+0000 7f86f1423740 10 client.0 _ll_register_callbacks cb 0x7f86d8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:46.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.294+0000 7f86e8c6e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb404841b0 0x55bb40873980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.294+0000 7f86e8c6e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb404841b0 0x55bb40873980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49716/0 (socket says 172.21.15.67:49716) 2024-07-31T14:55:46.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.294+0000 7f86e946f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404ad070 0x55bb408715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.294+0000 7f86e3fff640 1 --2- 172.21.15.67:0/3873108610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb4048c390 0x55bb40870e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.294+0000 7f86e8c6e640 1 -- 172.21.15.67:0/3873108610 learned_addr learned my addr 172.21.15.67:0/3873108610 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:46.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.294+0000 7f86e946f640 1 -- 172.21.15.67:0/3873108610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb404841b0 msgr2=0x55bb40873980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:46.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.294+0000 7f86e946f640 1 --2- 172.21.15.67:0/3873108610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb404841b0 0x55bb40873980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.294+0000 7f86e946f640 1 -- 172.21.15.67:0/3873108610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb4048c390 msgr2=0x55bb40870e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:46.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.294+0000 7f86e946f640 1 --2- 172.21.15.67:0/3873108610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb4048c390 0x55bb40870e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.294+0000 7f86e946f640 1 -- 172.21.15.67:0/3873108610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb408dfdd0 con 0x55bb404ad070 2024-07-31T14:55:46.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.295+0000 7f86e946f640 1 --2- 172.21.15.67:0/3873108610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404ad070 0x55bb408715a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f86dc005e30 tx=0x7f86dc031a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8085162940d37dd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:46.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.295+0000 7f86c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:55:46.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.295+0000 7f86c3fff640 1 -- 172.21.15.67:0/3873108610 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc045040 con 0x55bb404ad070 2024-07-31T14:55:46.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.295+0000 7f86c3fff640 1 -- 172.21.15.67:0/3873108610 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f86dc02fae0 con 0x55bb404ad070 2024-07-31T14:55:46.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.296+0000 7f86f1423740 1 -- 172.21.15.67:0/3873108610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86dc003c30 con 0x55bb404ad070 2024-07-31T14:55:46.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.296+0000 7f86c3fff640 1 -- 172.21.15.67:0/3873108610 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f86dc02fce0 con 0x55bb404ad070 2024-07-31T14:55:46.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.296+0000 7f86f1423740 10 client.26453 fetch_fsmap learned FSMap version 42 2024-07-31T14:55:46.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.296+0000 7f86f1423740 1 -- 172.21.15.67:0/3873108610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bb408e0f20 con 0x55bb404ad070 2024-07-31T14:55:46.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.296+0000 7f86c3fff640 1 -- 172.21.15.67:0/3873108610 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86dc059070 con 0x55bb404ad070 2024-07-31T14:55:46.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.296+0000 7f86c3fff640 1 -- 172.21.15.67:0/3873108610 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f86dc048030 con 0x55bb404ad070 2024-07-31T14:55:46.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.297+0000 7f86f1423740 10 client.26453 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:55:46.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.297+0000 7f86f1423740 10 client.26453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:46.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.297+0000 7f86f1423740 20 client.26453 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:46.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.297+0000 7f86f1423740 1 --2- 172.21.15.67:0/3873108610 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb40440da0 0x55bb408e3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:46.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.297+0000 7f86f1423740 4 client.26453 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:55:46.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.297+0000 7f86f1423740 1 -- 172.21.15.67:0/3873108610 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f86dc003c30 con 0x55bb40440da0 2024-07-31T14:55:46.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.297+0000 7f86e8c6e640 1 --2- 172.21.15.67:0/3873108610 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb40440da0 0x55bb408e3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:46.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.298+0000 7f86e8c6e640 1 --2- 172.21.15.67:0/3873108610 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb40440da0 0x55bb408e3650 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35645e40cb3f3bdb server_cookie=6121916f412182f6 in_seq=0 out_seq=0 2024-07-31T14:55:46.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.298+0000 7f86c3fff640 10 client.26453 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:46.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.302+0000 7f86c3fff640 1 -- 172.21.15.67:0/3873108610 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f86dc003c30 con 0x55bb40440da0 2024-07-31T14:55:46.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.302+0000 7f86c3fff640 10 client.26453 handle_command_reply: tid=0 2024-07-31T14:55:46.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.302+0000 7f86f1423740 1 client.26453 shutdown 2024-07-31T14:55:46.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.303+0000 7f86f1423740 1 -- 172.21.15.67:0/3873108610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404ad070 msgr2=0x55bb408715a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:46.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.303+0000 7f86f1423740 1 --2- 172.21.15.67:0/3873108610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404ad070 0x55bb408715a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f86dc005e30 tx=0x7f86dc031a50 comp rx=0 tx=0).stop 2024-07-31T14:55:46.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.303+0000 7f86f1423740 1 -- 172.21.15.67:0/3873108610 shutdown_connections 2024-07-31T14:55:46.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.303+0000 7f86f1423740 1 --2- 172.21.15.67:0/3873108610 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb40440da0 0x55bb408e3650 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.303+0000 7f86f1423740 1 --2- 172.21.15.67:0/3873108610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb404ad070 0x55bb408715a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.303+0000 7f86f1423740 1 --2- 172.21.15.67:0/3873108610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb4048c390 0x55bb40870e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.303+0000 7f86f1423740 1 --2- 172.21.15.67:0/3873108610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb404841b0 0x55bb40873980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.303+0000 7f86f1423740 1 -- 172.21.15.67:0/3873108610 >> 172.21.15.67:0/3873108610 conn(0x55bb404a1460 msgr2=0x55bb4085f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.303+0000 7f86c3fff640 0 client.26453 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.304+0000 7f86f1423740 1 -- 172.21.15.67:0/3873108610 shutdown_connections 2024-07-31T14:55:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.304+0000 7f86f1423740 1 -- 172.21.15.67:0/3873108610 wait complete. 2024-07-31T14:55:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.304+0000 7f86f1423740 20 client.26453 trim_cache size 0 max 16384 2024-07-31T14:55:46.307 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:55:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.305+0000 7f86eaed2640 1 -- 172.21.15.67:0/3758643014 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86d40b3d40 msgr2=0x7f86d40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.305+0000 7f86eaed2640 1 --2- 172.21.15.67:0/3758643014 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86d40b3d40 0x7f86d40b6250 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f86dc006010 tx=0x7f86dc031bd0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.305+0000 7f86eaed2640 1 -- 172.21.15.67:0/3758643014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e4107de0 msgr2=0x7f86e4199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.305+0000 7f86eaed2640 1 --2- 172.21.15.67:0/3758643014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e4107de0 0x7f86e4199420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f86d000ead0 tx=0x7f86d000ef90 comp rx=0 tx=0).stop 2024-07-31T14:55:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.305+0000 7f86eaed2640 1 -- 172.21.15.67:0/3758643014 shutdown_connections 2024-07-31T14:55:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.305+0000 7f86eaed2640 1 --2- 172.21.15.67:0/3758643014 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f86d40b3d40 0x7f86d40b6250 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.305+0000 7f86eaed2640 1 --2- 172.21.15.67:0/3758643014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e4108730 0x7f86e419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.305+0000 7f86eaed2640 1 --2- 172.21.15.67:0/3758643014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e4107de0 0x7f86e4199420 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.305+0000 7f86eaed2640 1 --2- 172.21.15.67:0/3758643014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e41024a0 0x7f86e4198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.305+0000 7f86eaed2640 1 -- 172.21.15.67:0/3758643014 >> 172.21.15.67:0/3758643014 conn(0x7f86e4079160 msgr2=0x7f86e4103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.306+0000 7f86eaed2640 1 -- 172.21.15.67:0/3758643014 shutdown_connections 2024-07-31T14:55:46.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:46.306+0000 7f86eaed2640 1 -- 172.21.15.67:0/3758643014 wait complete. 2024-07-31T14:55:47.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:46 smithi077 ceph-mon[37327]: pgmap v541: 129 pgs: 129 active+clean; 917 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 23 MiB/s wr, 1.36k op/s 2024-07-31T14:55:47.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:46 smithi067 ceph-mon[32750]: pgmap v541: 129 pgs: 129 active+clean; 917 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 23 MiB/s wr, 1.36k op/s 2024-07-31T14:55:47.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:46 smithi073 ceph-mon[36714]: pgmap v541: 129 pgs: 129 active+clean; 917 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 23 MiB/s wr, 1.36k op/s 2024-07-31T14:55:48.406 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T14:55:48.407 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T14:55:48.780 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:55:49.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:48 smithi067 ceph-mon[32750]: pgmap v542: 129 pgs: 129 active+clean; 922 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 18 MiB/s wr, 1.47k op/s 2024-07-31T14:55:49.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:48 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:55:49.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:48 smithi077 ceph-mon[37327]: pgmap v542: 129 pgs: 129 active+clean; 922 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 18 MiB/s wr, 1.47k op/s 2024-07-31T14:55:49.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:48 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:55:49.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:48 smithi073 ceph-mon[36714]: pgmap v542: 129 pgs: 129 active+clean; 922 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 18 MiB/s wr, 1.47k op/s 2024-07-31T14:55:49.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:48 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:55:50.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:55:50.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:55:50.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:49 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:55:50.290 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.290 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.290 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.290 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.290 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.290 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.290 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.290 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.290 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:55:50.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:55:50.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:49 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:55:50.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:55:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:55:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:55:50.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:49 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:55:50.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.697+0000 7f9a29de9640 1 -- 172.21.15.67:0/3794812060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a24108720 msgr2=0x7f9a2410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:50.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.697+0000 7f9a29de9640 1 --2- 172.21.15.67:0/3794812060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a24108720 0x7f9a2410ef50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009900 tx=0x7f9a1c02a410 comp rx=0 tx=0).stop 2024-07-31T14:55:50.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.697+0000 7f9a29de9640 1 -- 172.21.15.67:0/3794812060 shutdown_connections 2024-07-31T14:55:50.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.697+0000 7f9a29de9640 1 --2- 172.21.15.67:0/3794812060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a24108720 0x7f9a2410ef50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.697+0000 7f9a29de9640 1 --2- 172.21.15.67:0/3794812060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a24107dd0 0x7f9a241081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.697+0000 7f9a29de9640 1 --2- 172.21.15.67:0/3794812060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a24102490 0x7f9a241028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.697+0000 7f9a29de9640 1 -- 172.21.15.67:0/3794812060 >> 172.21.15.67:0/3794812060 conn(0x7f9a24079160 msgr2=0x7f9a24079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.697+0000 7f9a29de9640 1 -- 172.21.15.67:0/3794812060 shutdown_connections 2024-07-31T14:55:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.697+0000 7f9a29de9640 1 -- 172.21.15.67:0/3794812060 wait complete. 2024-07-31T14:55:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.698+0000 7f9a29de9640 1 Processor -- start 2024-07-31T14:55:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.698+0000 7f9a29de9640 1 -- start start 2024-07-31T14:55:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.698+0000 7f9a29de9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a24102490 0x7f9a24199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a29de9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a24107dd0 0x7f9a2419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a29de9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a24108720 0x7f9a2419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a29de9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241134b0 con 0x7f9a24107dd0 2024-07-31T14:55:50.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a29de9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9a24113330 con 0x7f9a24102490 2024-07-31T14:55:50.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a29de9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a24113630 con 0x7f9a24108720 2024-07-31T14:55:50.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a22ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a24107dd0 0x7f9a2419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a237fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a24102490 0x7f9a24199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a22ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a24107dd0 0x7f9a2419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44170/0 (socket says 172.21.15.67:44170) 2024-07-31T14:55:50.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a22ffd640 1 -- 172.21.15.67:0/2001063768 learned_addr learned my addr 172.21.15.67:0/2001063768 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:50.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a23fff640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a24108720 0x7f9a2419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a237fe640 1 -- 172.21.15.67:0/2001063768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a24108720 msgr2=0x7f9a2419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:50.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a237fe640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a24108720 0x7f9a2419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a237fe640 1 -- 172.21.15.67:0/2001063768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a24107dd0 msgr2=0x7f9a2419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:50.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a237fe640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a24107dd0 0x7f9a2419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a237fe640 1 -- 172.21.15.67:0/2001063768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2419fdd0 con 0x7f9a24102490 2024-07-31T14:55:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a22ffd640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a24107dd0 0x7f9a2419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a23fff640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a24108720 0x7f9a2419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a237fe640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a24102490 0x7f9a24199e60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009fd0 tx=0x7f9a1400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.699+0000 7f9a20ff9640 1 -- 172.21.15.67:0/2001063768 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14004b20 con 0x7f9a24102490 2024-07-31T14:55:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.700+0000 7f9a29de9640 1 -- 172.21.15.67:0/2001063768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a241a00c0 con 0x7f9a24102490 2024-07-31T14:55:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.700+0000 7f9a20ff9640 1 -- 172.21.15.67:0/2001063768 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a14004cc0 con 0x7f9a24102490 2024-07-31T14:55:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.700+0000 7f9a20ff9640 1 -- 172.21.15.67:0/2001063768 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14010720 con 0x7f9a24102490 2024-07-31T14:55:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.700+0000 7f9a29de9640 1 -- 172.21.15.67:0/2001063768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a241a6820 con 0x7f9a24102490 2024-07-31T14:55:50.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.702+0000 7f9a20ff9640 1 -- 172.21.15.67:0/2001063768 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f9a14005020 con 0x7f9a24102490 2024-07-31T14:55:50.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.702+0000 7f9a3033a740 1 Processor -- start 2024-07-31T14:55:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.702+0000 7f9a3033a740 1 -- start start 2024-07-31T14:55:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a3033a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec76350 0x556d9f01b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a3033a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 0x556d9f01b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a3033a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec55670 0x556d9f01be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a3033a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556d9ef032f0 con 0x556d9ec55670 2024-07-31T14:55:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a3033a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556d9ef91a80 con 0x556d9ec4d490 2024-07-31T14:55:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a3033a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556d9ef80f80 con 0x556d9ec76350 2024-07-31T14:55:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a22ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec55670 0x556d9f01be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a22ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec55670 0x556d9f01be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44176/0 (socket says 172.21.15.67:44176) 2024-07-31T14:55:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a22ffd640 1 -- 172.21.15.67:0/545817655 learned_addr learned my addr 172.21.15.67:0/545817655 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a237fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 0x556d9f01b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a23fff640 1 --2- 172.21.15.67:0/545817655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec76350 0x556d9f01b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a22ffd640 1 -- 172.21.15.67:0/545817655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec76350 msgr2=0x556d9f01b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a22ffd640 1 --2- 172.21.15.67:0/545817655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec76350 0x556d9f01b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a22ffd640 1 -- 172.21.15.67:0/545817655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 msgr2=0x556d9f01b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a22ffd640 1 --2- 172.21.15.67:0/545817655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 0x556d9f01b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a22ffd640 1 -- 172.21.15.67:0/545817655 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d9ece1930 con 0x556d9ec55670 2024-07-31T14:55:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.703+0000 7f9a20ff9640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a100b3d40 0x7f9a100b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.704+0000 7f9a22ffd640 1 --2- 172.21.15.67:0/545817655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec55670 0x556d9f01be30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c011510 tx=0x7f9a0c0119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827546e0b92dafc1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.704+0000 7f9a23fff640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a100b3d40 0x7f9a100b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.704+0000 7f9a20ff9640 1 -- 172.21.15.67:0/2001063768 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f9a140e4aa0 con 0x7f9a24102490 2024-07-31T14:55:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.704+0000 7f9a19ffb640 1 -- 172.21.15.67:0/545817655 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c01e070 con 0x556d9ec55670 2024-07-31T14:55:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.704+0000 7f9a19ffb640 1 -- 172.21.15.67:0/545817655 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a0c00fa10 con 0x556d9ec55670 2024-07-31T14:55:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.704+0000 7f9a19ffb640 1 -- 172.21.15.67:0/545817655 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c019aa0 con 0x556d9ec55670 2024-07-31T14:55:50.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.705+0000 7f9a3033a740 1 -- 172.21.15.67:0/545817655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec55670 msgr2=0x556d9f01be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:50.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.705+0000 7f9a23fff640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a100b3d40 0x7f9a100b6250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c006010 tx=0x7f9a1c0058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.705+0000 7f9a3033a740 1 --2- 172.21.15.67:0/545817655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec55670 0x556d9f01be30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c011510 tx=0x7f9a0c0119d0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.705+0000 7f9a3033a740 1 -- 172.21.15.67:0/545817655 shutdown_connections 2024-07-31T14:55:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.705+0000 7f9a3033a740 1 --2- 172.21.15.67:0/545817655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec55670 0x556d9f01be30 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.705+0000 7f9a3033a740 1 --2- 172.21.15.67:0/545817655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 0x556d9f01b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.705+0000 7f9a3033a740 1 --2- 172.21.15.67:0/545817655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec76350 0x556d9f01b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.705+0000 7f9a3033a740 1 -- 172.21.15.67:0/545817655 >> 172.21.15.67:0/545817655 conn(0x556d9ec6a740 msgr2=0x556d9f0192a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.705+0000 7f9a3033a740 1 -- 172.21.15.67:0/545817655 shutdown_connections 2024-07-31T14:55:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.705+0000 7f9a3033a740 1 -- 172.21.15.67:0/545817655 wait complete. 2024-07-31T14:55:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.705+0000 7f9a3033a740 1 -- start start 2024-07-31T14:55:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.705+0000 7f9a3033a740 1 Processor -- start 2024-07-31T14:55:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a3033a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 0x556d9f02b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a3033a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec55670 0x556d9f02b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a3033a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec76350 0x556d9f0370a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a3033a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556d9ef234f0 con 0x556d9ec76350 2024-07-31T14:55:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a3033a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556d9ef84fc0 con 0x556d9ec4d490 2024-07-31T14:55:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a3033a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556d9ede4890 con 0x556d9ec55670 2024-07-31T14:55:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a3033a740 10 client.0 _ll_register_callbacks cb 0x556d9ef2d500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a237fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 0x556d9f02b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a237fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 0x556d9f02b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40906/0 (socket says 172.21.15.67:40906) 2024-07-31T14:55:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a237fe640 1 -- 172.21.15.67:0/522146271 learned_addr learned my addr 172.21.15.67:0/522146271 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a23fff640 1 --2- 172.21.15.67:0/522146271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec76350 0x556d9f0370a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a22ffd640 1 --2- 172.21.15.67:0/522146271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec55670 0x556d9f02b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a237fe640 1 -- 172.21.15.67:0/522146271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec55670 msgr2=0x556d9f02b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a237fe640 1 --2- 172.21.15.67:0/522146271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec55670 0x556d9f02b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a237fe640 1 -- 172.21.15.67:0/522146271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec76350 msgr2=0x556d9f0370a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a237fe640 1 --2- 172.21.15.67:0/522146271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec76350 0x556d9f0370a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a237fe640 1 -- 172.21.15.67:0/522146271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d9ef32c50 con 0x556d9ec4d490 2024-07-31T14:55:50.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.706+0000 7f9a22ffd640 1 --2- 172.21.15.67:0/522146271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec55670 0x556d9f02b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:55:50.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.707+0000 7f9a237fe640 1 --2- 172.21.15.67:0/522146271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 0x556d9f02b340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9a140e1f40 tx=0x7f9a140e81e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a73663852abe99f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:50.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.707+0000 7f9a02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:55:50.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.707+0000 7f9a02ffd640 1 -- 172.21.15.67:0/522146271 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a140ea030 con 0x556d9ec4d490 2024-07-31T14:55:50.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.707+0000 7f9a02ffd640 1 -- 172.21.15.67:0/522146271 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f9a140a7990 con 0x556d9ec4d490 2024-07-31T14:55:50.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.707+0000 7f9a3033a740 1 -- 172.21.15.67:0/522146271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a0c00fa10 con 0x556d9ec4d490 2024-07-31T14:55:50.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.707+0000 7f9a02ffd640 1 -- 172.21.15.67:0/522146271 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9a140ab170 con 0x556d9ec4d490 2024-07-31T14:55:50.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.707+0000 7f9a3033a740 10 client.26465 fetch_fsmap learned FSMap version 42 2024-07-31T14:55:50.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.707+0000 7f9a3033a740 1 -- 172.21.15.67:0/522146271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556d9f039d70 con 0x556d9ec4d490 2024-07-31T14:55:50.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.707+0000 7f9a02ffd640 1 -- 172.21.15.67:0/522146271 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a0c00fa10 con 0x556d9ec4d490 2024-07-31T14:55:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.707+0000 7f9a02ffd640 1 -- 172.21.15.67:0/522146271 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f9a14005310 con 0x556d9ec4d490 2024-07-31T14:55:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.707+0000 7f9a3033a740 10 client.26465 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:55:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.707+0000 7f9a3033a740 10 client.26465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.708+0000 7f9a3033a740 20 client.26465 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.708+0000 7f9a3033a740 1 --2- 172.21.15.67:0/522146271 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556d9ec34cd0 0x556d9f03d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.708+0000 7f9a3033a740 4 client.26465 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:55:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.708+0000 7f9a3033a740 1 -- 172.21.15.67:0/522146271 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d9ef939d0 con 0x556d9ec34cd0 2024-07-31T14:55:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.708+0000 7f9a22ffd640 1 --2- 172.21.15.67:0/522146271 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556d9ec34cd0 0x556d9f03d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.708+0000 7f9a22ffd640 1 --2- 172.21.15.67:0/522146271 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556d9ec34cd0 0x556d9f03d360 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc5ff5578e4c86cd server_cookie=34b39971a37028aa in_seq=0 out_seq=0 2024-07-31T14:55:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.708+0000 7f9a02ffd640 10 client.26465 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:50.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a02ffd640 1 -- 172.21.15.67:0/522146271 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d9ef939d0 con 0x556d9ec34cd0 2024-07-31T14:55:50.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a02ffd640 10 client.26465 handle_command_reply: tid=0 2024-07-31T14:55:50.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a3033a740 1 client.26465 shutdown 2024-07-31T14:55:50.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a3033a740 1 -- 172.21.15.67:0/522146271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 msgr2=0x556d9f02b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:50.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a3033a740 1 --2- 172.21.15.67:0/522146271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 0x556d9f02b340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9a140e1f40 tx=0x7f9a140e81e0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a3033a740 1 -- 172.21.15.67:0/522146271 shutdown_connections 2024-07-31T14:55:50.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a3033a740 1 --2- 172.21.15.67:0/522146271 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556d9ec34cd0 0x556d9f03d360 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a3033a740 1 --2- 172.21.15.67:0/522146271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec76350 0x556d9f0370a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a3033a740 1 --2- 172.21.15.67:0/522146271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec55670 0x556d9f02b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a3033a740 1 --2- 172.21.15.67:0/522146271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 0x556d9f02b340 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a02ffd640 0 client.26465 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:50.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a3033a740 1 -- 172.21.15.67:0/522146271 >> 172.21.15.67:0/522146271 conn(0x556d9ec6a740 msgr2=0x556d9f01cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:50.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a3033a740 1 -- 172.21.15.67:0/522146271 shutdown_connections 2024-07-31T14:55:50.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a3033a740 1 -- 172.21.15.67:0/522146271 wait complete. 2024-07-31T14:55:50.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.710+0000 7f9a3033a740 20 client.26465 trim_cache size 0 max 16384 2024-07-31T14:55:50.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.728+0000 7f9a3033a740 1 Processor -- start 2024-07-31T14:55:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a3033a740 1 -- start start 2024-07-31T14:55:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a3033a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 0x556d9f018280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a3033a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec55670 0x556d9f036e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a3033a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec76350 0x556d9f01caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a3033a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241ae3e0 con 0x556d9ec76350 2024-07-31T14:55:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a3033a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241b00a0 con 0x556d9ec4d490 2024-07-31T14:55:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a3033a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241aeab0 con 0x556d9ec55670 2024-07-31T14:55:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a23fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec76350 0x556d9f01caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a237fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 0x556d9f018280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a23fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec76350 0x556d9f01caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44202/0 (socket says 172.21.15.67:44202) 2024-07-31T14:55:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a23fff640 1 -- 172.21.15.67:0/3813580732 learned_addr learned my addr 172.21.15.67:0/3813580732 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a22ffd640 1 --2- 172.21.15.67:0/3813580732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec55670 0x556d9f036e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a23fff640 1 -- 172.21.15.67:0/3813580732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec55670 msgr2=0x556d9f036e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a23fff640 1 --2- 172.21.15.67:0/3813580732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec55670 0x556d9f036e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a23fff640 1 -- 172.21.15.67:0/3813580732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 msgr2=0x556d9f018280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a23fff640 1 --2- 172.21.15.67:0/3813580732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 0x556d9f018280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a23fff640 1 -- 172.21.15.67:0/3813580732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d9f014a90 con 0x556d9ec76350 2024-07-31T14:55:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.729+0000 7f9a23fff640 1 --2- 172.21.15.67:0/3813580732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec76350 0x556d9f01caf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c02a930 tx=0x7f9a1c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d34330a6736e935 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.730+0000 7f9a197fa640 1 -- 172.21.15.67:0/3813580732 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c035040 con 0x556d9ec76350 2024-07-31T14:55:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.730+0000 7f9a197fa640 1 -- 172.21.15.67:0/3813580732 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c0044c0 con 0x556d9ec76350 2024-07-31T14:55:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.730+0000 7f9a197fa640 1 -- 172.21.15.67:0/3813580732 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c03dac0 con 0x556d9ec76350 2024-07-31T14:55:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.730+0000 7f9a3033a740 1 -- 172.21.15.67:0/3813580732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec76350 msgr2=0x556d9f01caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.730+0000 7f9a3033a740 1 --2- 172.21.15.67:0/3813580732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec76350 0x556d9f01caf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c02a930 tx=0x7f9a1c03f000 comp rx=0 tx=0).stop 2024-07-31T14:55:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.730+0000 7f9a3033a740 1 -- 172.21.15.67:0/3813580732 shutdown_connections 2024-07-31T14:55:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.730+0000 7f9a3033a740 1 --2- 172.21.15.67:0/3813580732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec76350 0x556d9f01caf0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.730+0000 7f9a3033a740 1 --2- 172.21.15.67:0/3813580732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec55670 0x556d9f036e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.730+0000 7f9a3033a740 1 --2- 172.21.15.67:0/3813580732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec4d490 0x556d9f018280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.730+0000 7f9a3033a740 1 -- 172.21.15.67:0/3813580732 >> 172.21.15.67:0/3813580732 conn(0x556d9ec6a740 msgr2=0x556d9f01c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.731+0000 7f9a3033a740 1 -- 172.21.15.67:0/3813580732 shutdown_connections 2024-07-31T14:55:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.731+0000 7f9a3033a740 1 -- 172.21.15.67:0/3813580732 wait complete. 2024-07-31T14:55:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.731+0000 7f9a3033a740 1 -- start start 2024-07-31T14:55:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.731+0000 7f9a3033a740 1 Processor -- start 2024-07-31T14:55:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.731+0000 7f9a3033a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec4d490 0x556d9f02f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.731+0000 7f9a3033a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec55670 0x556d9f02f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.731+0000 7f9a3033a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec76350 0x556d9f0a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.731+0000 7f9a3033a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241af9d0 con 0x556d9ec4d490 2024-07-31T14:55:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.731+0000 7f9a3033a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241af240 con 0x556d9ec55670 2024-07-31T14:55:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.731+0000 7f9a3033a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241ac3c0 con 0x556d9ec76350 2024-07-31T14:55:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a3033a740 10 client.0 _ll_register_callbacks cb 0x7f9a140e82e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a237fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec4d490 0x556d9f02f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a237fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec4d490 0x556d9f02f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44218/0 (socket says 172.21.15.67:44218) 2024-07-31T14:55:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a22ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec55670 0x556d9f02f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a237fe640 1 -- 172.21.15.67:0/3119895802 learned_addr learned my addr 172.21.15.67:0/3119895802 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a23fff640 1 --2- 172.21.15.67:0/3119895802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec76350 0x556d9f0a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a23fff640 1 -- 172.21.15.67:0/3119895802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec55670 msgr2=0x556d9f02f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a23fff640 1 --2- 172.21.15.67:0/3119895802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec55670 0x556d9f02f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a23fff640 1 -- 172.21.15.67:0/3119895802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec4d490 msgr2=0x556d9f02f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a23fff640 1 --2- 172.21.15.67:0/3119895802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec4d490 0x556d9f02f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a23fff640 1 -- 172.21.15.67:0/3119895802 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d9f0a5070 con 0x556d9ec76350 2024-07-31T14:55:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a237fe640 1 --2- 172.21.15.67:0/3119895802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec4d490 0x556d9f02f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a23fff640 1 --2- 172.21.15.67:0/3119895802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec76350 0x556d9f0a4b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c005940 tx=0x7f9a1c03d4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed363755500084ac server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:55:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.732+0000 7f9a02ffd640 1 -- 172.21.15.67:0/3119895802 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c060020 con 0x556d9ec76350 2024-07-31T14:55:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.733+0000 7f9a3033a740 1 -- 172.21.15.67:0/3119895802 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d9ef8f470 con 0x556d9ec76350 2024-07-31T14:55:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.733+0000 7f9a02ffd640 1 -- 172.21.15.67:0/3119895802 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f9a1c03d800 con 0x556d9ec76350 2024-07-31T14:55:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.733+0000 7f9a02ffd640 1 -- 172.21.15.67:0/3119895802 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9a1c06b050 con 0x556d9ec76350 2024-07-31T14:55:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.733+0000 7f9a3033a740 10 client.26500 fetch_fsmap learned FSMap version 42 2024-07-31T14:55:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.733+0000 7f9a3033a740 1 -- 172.21.15.67:0/3119895802 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556d9f0a61d0 con 0x556d9ec76350 2024-07-31T14:55:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.733+0000 7f9a02ffd640 1 -- 172.21.15.67:0/3119895802 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556d9ef8f470 con 0x556d9ec76350 2024-07-31T14:55:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.733+0000 7f9a02ffd640 1 -- 172.21.15.67:0/3119895802 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f9a1c05b030 con 0x556d9ec76350 2024-07-31T14:55:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.733+0000 7f9a3033a740 10 client.26500 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:55:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.733+0000 7f9a3033a740 10 client.26500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.733+0000 7f9a3033a740 20 client.26500 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.733+0000 7f9a3033a740 1 --2- 172.21.15.67:0/3119895802 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556d9ec0a080 0x556d9f0a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.733+0000 7f9a3033a740 4 client.26500 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T14:55:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.733+0000 7f9a3033a740 1 -- 172.21.15.67:0/3119895802 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556d9ef8cbb0 con 0x556d9ec0a080 2024-07-31T14:55:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.733+0000 7f9a237fe640 1 --2- 172.21.15.67:0/3119895802 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556d9ec0a080 0x556d9f0a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.734+0000 7f9a02ffd640 10 client.26500 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.734+0000 7f9a237fe640 1 --2- 172.21.15.67:0/3119895802 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556d9ec0a080 0x556d9f0a8a50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44cfc5df36519ca4 server_cookie=2001d313af8aa244 in_seq=0 out_seq=0 2024-07-31T14:55:50.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.863+0000 7f9a02ffd640 1 -- 172.21.15.67:0/3119895802 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556d9ef8cbb0 con 0x556d9ec0a080 2024-07-31T14:55:50.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.863+0000 7f9a02ffd640 10 client.26500 handle_command_reply: tid=0 2024-07-31T14:55:50.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.863+0000 7f9a3033a740 1 client.26500 shutdown 2024-07-31T14:55:50.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.864+0000 7f9a3033a740 1 -- 172.21.15.67:0/3119895802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec76350 msgr2=0x556d9f0a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:50.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.864+0000 7f9a3033a740 1 --2- 172.21.15.67:0/3119895802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec76350 0x556d9f0a4b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c005940 tx=0x7f9a1c03d4e0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.864+0000 7f9a3033a740 1 -- 172.21.15.67:0/3119895802 shutdown_connections 2024-07-31T14:55:50.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.864+0000 7f9a3033a740 1 --2- 172.21.15.67:0/3119895802 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556d9ec0a080 0x556d9f0a8a50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.864+0000 7f9a3033a740 1 --2- 172.21.15.67:0/3119895802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9ec76350 0x556d9f0a4b30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.864+0000 7f9a3033a740 1 --2- 172.21.15.67:0/3119895802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d9ec55670 0x556d9f02f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.864+0000 7f9a3033a740 1 --2- 172.21.15.67:0/3119895802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d9ec4d490 0x556d9f02f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.864+0000 7f9a02ffd640 0 client.26500 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:50.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.864+0000 7f9a3033a740 1 -- 172.21.15.67:0/3119895802 >> 172.21.15.67:0/3119895802 conn(0x556d9ec6a740 msgr2=0x556d9f018050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:50.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.864+0000 7f9a3033a740 1 -- 172.21.15.67:0/3119895802 shutdown_connections 2024-07-31T14:55:50.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.864+0000 7f9a3033a740 1 -- 172.21.15.67:0/3119895802 wait complete. 2024-07-31T14:55:50.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.864+0000 7f9a3033a740 20 client.26500 trim_cache size 0 max 16384 2024-07-31T14:55:50.867 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:55:50.867 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T14:55:50.868 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "ce8320d6-b226-42dc-9adb-9aff7bde586b", 2024-07-31T14:55:50.868 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T14:55:50.868 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:55:50.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.865+0000 7f9a29de9640 1 -- 172.21.15.67:0/2001063768 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a100b3d40 msgr2=0x7f9a100b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:50.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.865+0000 7f9a29de9640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a100b3d40 0x7f9a100b6250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c006010 tx=0x7f9a1c0058d0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.865+0000 7f9a29de9640 1 -- 172.21.15.67:0/2001063768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a24102490 msgr2=0x7f9a24199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:50.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.865+0000 7f9a29de9640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a24102490 0x7f9a24199e60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009fd0 tx=0x7f9a1400ef90 comp rx=0 tx=0).stop 2024-07-31T14:55:50.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.865+0000 7f9a29de9640 1 -- 172.21.15.67:0/2001063768 shutdown_connections 2024-07-31T14:55:50.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.865+0000 7f9a29de9640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a100b3d40 0x7f9a100b6250 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.865+0000 7f9a29de9640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a24108720 0x7f9a2419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.865+0000 7f9a29de9640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a24107dd0 0x7f9a2419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.865+0000 7f9a29de9640 1 --2- 172.21.15.67:0/2001063768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a24102490 0x7f9a24199e60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:50.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.865+0000 7f9a29de9640 1 -- 172.21.15.67:0/2001063768 >> 172.21.15.67:0/2001063768 conn(0x7f9a24079160 msgr2=0x7f9a24103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:50.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.865+0000 7f9a29de9640 1 -- 172.21.15.67:0/2001063768 shutdown_connections 2024-07-31T14:55:50.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:50.865+0000 7f9a29de9640 1 -- 172.21.15.67:0/2001063768 wait complete. 2024-07-31T14:55:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.190+0000 7f0faaa68640 1 -- 172.21.15.67:0/2175178315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fa4108790 msgr2=0x7f0fa410efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.190+0000 7f0faaa68640 1 --2- 172.21.15.67:0/2175178315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fa4108790 0x7f0fa410efc0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009ca0 tx=0x7f0f9c02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.190+0000 7f0faaa68640 1 -- 172.21.15.67:0/2175178315 shutdown_connections 2024-07-31T14:55:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.190+0000 7f0faaa68640 1 --2- 172.21.15.67:0/2175178315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fa4108790 0x7f0fa410efc0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.190+0000 7f0faaa68640 1 --2- 172.21.15.67:0/2175178315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fa4107e40 0x7f0fa4108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.190+0000 7f0faaa68640 1 --2- 172.21.15.67:0/2175178315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fa4102b60 0x7f0fa4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.190+0000 7f0faaa68640 1 -- 172.21.15.67:0/2175178315 >> 172.21.15.67:0/2175178315 conn(0x7f0fa4079880 msgr2=0x7f0fa4079c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.190+0000 7f0faaa68640 1 -- 172.21.15.67:0/2175178315 shutdown_connections 2024-07-31T14:55:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.190+0000 7f0faaa68640 1 -- 172.21.15.67:0/2175178315 wait complete. 2024-07-31T14:55:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.191+0000 7f0faaa68640 1 Processor -- start 2024-07-31T14:55:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.191+0000 7f0faaa68640 1 -- start start 2024-07-31T14:55:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0faaa68640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fa4102b60 0x7f0fa419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0faaa68640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fa4107e40 0x7f0fa419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0faaa68640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fa4108790 0x7f0fa419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0faaa68640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa4112470 con 0x7f0fa4102b60 2024-07-31T14:55:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0faaa68640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41122f0 con 0x7f0fa4108790 2024-07-31T14:55:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0faaa68640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41125f0 con 0x7f0fa4107e40 2024-07-31T14:55:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0fa8804640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fa4102b60 0x7f0fa419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0fa3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fa4107e40 0x7f0fa419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0fa8804640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fa4102b60 0x7f0fa419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58418/0 (socket says 172.21.15.67:58418) 2024-07-31T14:55:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0fa9005640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fa4108790 0x7f0fa419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0fa8804640 1 -- 172.21.15.67:0/7714930 learned_addr learned my addr 172.21.15.67:0/7714930 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0fa3fff640 1 -- 172.21.15.67:0/7714930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fa4108790 msgr2=0x7f0fa419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0fa3fff640 1 --2- 172.21.15.67:0/7714930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fa4108790 0x7f0fa419b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0fa3fff640 1 -- 172.21.15.67:0/7714930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fa4102b60 msgr2=0x7f0fa419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0fa3fff640 1 --2- 172.21.15.67:0/7714930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fa4102b60 0x7f0fa419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0fa3fff640 1 -- 172.21.15.67:0/7714930 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa41a6050 con 0x7f0fa4107e40 2024-07-31T14:55:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.192+0000 7f0fa8804640 1 --2- 172.21.15.67:0/7714930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fa4102b60 0x7f0fa419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.193+0000 7f0fa3fff640 1 --2- 172.21.15.67:0/7714930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fa4107e40 0x7f0fa419b300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4075360 tx=0x7f0f9000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.193+0000 7f0fa1ffb640 1 -- 172.21.15.67:0/7714930 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9000cc40 con 0x7f0fa4107e40 2024-07-31T14:55:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.193+0000 7f0fa1ffb640 1 -- 172.21.15.67:0/7714930 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f9000cde0 con 0x7f0fa4107e40 2024-07-31T14:55:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.193+0000 7f0faaa68640 1 -- 172.21.15.67:0/7714930 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa41a6340 con 0x7f0fa4107e40 2024-07-31T14:55:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.193+0000 7f0fa1ffb640 1 -- 172.21.15.67:0/7714930 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90004a00 con 0x7f0fa4107e40 2024-07-31T14:55:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.193+0000 7f0faaa68640 1 -- 172.21.15.67:0/7714930 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa41a6890 con 0x7f0fa4107e40 2024-07-31T14:55:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.196+0000 7f0fb1a33740 1 Processor -- start 2024-07-31T14:55:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.196+0000 7f0fb1a33740 1 -- start start 2024-07-31T14:55:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.196+0000 7f0fb1a33740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 0x55ed20afcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.197+0000 7f0fb1a33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 0x55ed20afd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.197+0000 7f0fb1a33740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207394d0 0x55ed20afd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.197+0000 7f0fb1a33740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ed20a723b0 con 0x55ed207394d0 2024-07-31T14:55:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.197+0000 7f0fb1a33740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ed20a618b0 con 0x55ed2075a1b0 2024-07-31T14:55:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.197+0000 7f0fb1a33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ed20a658f0 con 0x55ed207312f0 2024-07-31T14:55:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.197+0000 7f0fa9005640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 0x55ed20afcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.197+0000 7f0fa9005640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 0x55ed20afcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:33556/0 (socket says 172.21.15.67:33556) 2024-07-31T14:55:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.197+0000 7f0fa9005640 1 -- 172.21.15.67:0/613446637 learned_addr learned my addr 172.21.15.67:0/613446637 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.197+0000 7f0fa9005640 1 -- 172.21.15.67:0/613446637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 msgr2=0x55ed20afd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.197+0000 7f0fa9005640 1 --2- 172.21.15.67:0/613446637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 0x55ed20afd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.197+0000 7f0fa9005640 1 -- 172.21.15.67:0/613446637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207394d0 msgr2=0x55ed20afd760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:55:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.197+0000 7f0fa9005640 1 --2- 172.21.15.67:0/613446637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207394d0 0x55ed20afd760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.197+0000 7f0fa9005640 1 -- 172.21.15.67:0/613446637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed20a089a0 con 0x55ed2075a1b0 2024-07-31T14:55:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.197+0000 7f0fa9005640 1 --2- 172.21.15.67:0/613446637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 0x55ed20afcce0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c02f440 tx=0x7f0f9c02f470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b4da1c3be387cba server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.198+0000 7f0fa1ffb640 1 -- 172.21.15.67:0/7714930 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0f90004ba0 con 0x7f0fa4107e40 2024-07-31T14:55:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.198+0000 7f0fa1ffb640 1 --2- 172.21.15.67:0/7714930 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f940b3dc0 0x7f0f940b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.199+0000 7f0f8effd640 1 -- 172.21.15.67:0/613446637 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c031cf0 con 0x55ed2075a1b0 2024-07-31T14:55:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.199+0000 7f0fa8804640 1 --2- 172.21.15.67:0/7714930 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f940b3dc0 0x7f0f940b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.199+0000 7f0fa1ffb640 1 -- 172.21.15.67:0/7714930 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f0f900e4140 con 0x7f0fa4107e40 2024-07-31T14:55:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.199+0000 7f0f8effd640 1 -- 172.21.15.67:0/613446637 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f9c036070 con 0x55ed2075a1b0 2024-07-31T14:55:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.199+0000 7f0f8effd640 1 -- 172.21.15.67:0/613446637 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0042a0 con 0x55ed2075a1b0 2024-07-31T14:55:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.200+0000 7f0fb1a33740 1 -- 172.21.15.67:0/613446637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 msgr2=0x55ed20afcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.200+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/613446637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 0x55ed20afcce0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c02f440 tx=0x7f0f9c02f470 comp rx=0 tx=0).stop 2024-07-31T14:55:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.200+0000 7f0fa8804640 1 --2- 172.21.15.67:0/7714930 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f940b3dc0 0x7f0f940b62d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009fd0 tx=0x7f0f98009780 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.200+0000 7f0fb1a33740 1 -- 172.21.15.67:0/613446637 shutdown_connections 2024-07-31T14:55:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.200+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/613446637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207394d0 0x55ed20afd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.200+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/613446637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 0x55ed20afd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.200+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/613446637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 0x55ed20afcce0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.200+0000 7f0fb1a33740 1 -- 172.21.15.67:0/613446637 >> 172.21.15.67:0/613446637 conn(0x55ed2074e300 msgr2=0x55ed20afabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.200+0000 7f0fb1a33740 1 -- 172.21.15.67:0/613446637 shutdown_connections 2024-07-31T14:55:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.200+0000 7f0fb1a33740 1 -- 172.21.15.67:0/613446637 wait complete. 2024-07-31T14:55:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.200+0000 7f0fb1a33740 1 -- start start 2024-07-31T14:55:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.200+0000 7f0fb1a33740 1 Processor -- start 2024-07-31T14:55:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.201+0000 7f0fb1a33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 0x55ed20b088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.201+0000 7f0fb1a33740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207394d0 0x55ed20b08e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.201+0000 7f0fb1a33740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 0x55ed20b145d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.201+0000 7f0fb1a33740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ed2099bab0 con 0x55ed207394d0 2024-07-31T14:55:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.201+0000 7f0fb1a33740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ed209e9cf0 con 0x55ed2075a1b0 2024-07-31T14:55:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.201+0000 7f0fb1a33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ed20b0b280 con 0x55ed207312f0 2024-07-31T14:55:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.201+0000 7f0fa3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207394d0 0x55ed20b08e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.201+0000 7f0fb1a33740 10 client.0 _ll_register_callbacks cb 0x55ed20a5b9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.201+0000 7f0fa3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207394d0 0x55ed20b08e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58422/0 (socket says 172.21.15.67:58422) 2024-07-31T14:55:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.201+0000 7f0fa3fff640 1 -- 172.21.15.67:0/2622872286 learned_addr learned my addr 172.21.15.67:0/2622872286 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.201+0000 7f0fa8804640 1 --2- 172.21.15.67:0/2622872286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 0x55ed20b088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.201+0000 7f0fa9005640 1 --2- 172.21.15.67:0/2622872286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 0x55ed20b145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.202+0000 7f0fa8804640 1 -- 172.21.15.67:0/2622872286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 msgr2=0x55ed20b145d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.202+0000 7f0fa8804640 1 --2- 172.21.15.67:0/2622872286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 0x55ed20b145d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.202+0000 7f0fa8804640 1 -- 172.21.15.67:0/2622872286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207394d0 msgr2=0x55ed20b08e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.202+0000 7f0fa8804640 1 --2- 172.21.15.67:0/2622872286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207394d0 0x55ed20b08e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.202+0000 7f0fa8804640 1 -- 172.21.15.67:0/2622872286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed207c5790 con 0x55ed207312f0 2024-07-31T14:55:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.202+0000 7f0fa3fff640 1 --2- 172.21.15.67:0/2622872286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207394d0 0x55ed20b08e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.202+0000 7f0fa8804640 1 --2- 172.21.15.67:0/2622872286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 0x55ed20b088d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55ed20afb960 tx=0x7f0f98005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72435fa0816a8358 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.202+0000 7f0f83fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:55:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.202+0000 7f0f83fff640 1 -- 172.21.15.67:0/2622872286 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800f7a0 con 0x55ed207312f0 2024-07-31T14:55:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.202+0000 7f0f83fff640 1 -- 172.21.15.67:0/2622872286 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f0f9800f940 con 0x55ed207312f0 2024-07-31T14:55:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.202+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2622872286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f9c036070 con 0x55ed207312f0 2024-07-31T14:55:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.203+0000 7f0f83fff640 1 -- 172.21.15.67:0/2622872286 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0f9800fb40 con 0x55ed207312f0 2024-07-31T14:55:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.203+0000 7f0fb1a33740 10 client.26509 fetch_fsmap learned FSMap version 42 2024-07-31T14:55:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.203+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2622872286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ed2083e620 con 0x55ed207312f0 2024-07-31T14:55:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.203+0000 7f0f83fff640 1 -- 172.21.15.67:0/2622872286 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f9800fce0 con 0x55ed207312f0 2024-07-31T14:55:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.203+0000 7f0f83fff640 1 -- 172.21.15.67:0/2622872286 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f0f9801d4b0 con 0x55ed207312f0 2024-07-31T14:55:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.203+0000 7f0fb1a33740 10 client.26509 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:55:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.203+0000 7f0fb1a33740 10 client.26509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.203+0000 7f0fb1a33740 20 client.26509 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.203+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/2622872286 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ed20718d80 0x55ed20b1eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.203+0000 7f0fb1a33740 4 client.26509 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:55:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.203+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2622872286 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0f9c036070 con 0x55ed20718d80 2024-07-31T14:55:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.204+0000 7f0fa3fff640 1 --2- 172.21.15.67:0/2622872286 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ed20718d80 0x55ed20b1eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.204+0000 7f0f83fff640 10 client.26509 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.204+0000 7f0fa3fff640 1 --2- 172.21.15.67:0/2622872286 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ed20718d80 0x55ed20b1eb90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c541ea21bfe0f20 server_cookie=7347e83f1ea6733 in_seq=0 out_seq=0 2024-07-31T14:55:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.207+0000 7f0f83fff640 1 -- 172.21.15.67:0/2622872286 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0f9c036070 con 0x55ed20718d80 2024-07-31T14:55:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.207+0000 7f0f83fff640 10 client.26509 handle_command_reply: tid=0 2024-07-31T14:55:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.207+0000 7f0fb1a33740 1 client.26509 shutdown 2024-07-31T14:55:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.208+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2622872286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 msgr2=0x55ed20b088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.208+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/2622872286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 0x55ed20b088d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55ed20afb960 tx=0x7f0f98005f90 comp rx=0 tx=0).stop 2024-07-31T14:55:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.208+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2622872286 shutdown_connections 2024-07-31T14:55:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.208+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/2622872286 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ed20718d80 0x55ed20b1eb90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.208+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/2622872286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 0x55ed20b145d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.208+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/2622872286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207394d0 0x55ed20b08e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.208+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/2622872286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 0x55ed20b088d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.208+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2622872286 >> 172.21.15.67:0/2622872286 conn(0x55ed2074e300 msgr2=0x55ed20af92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.208+0000 7f0f83fff640 0 client.26509 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.208+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2622872286 shutdown_connections 2024-07-31T14:55:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.208+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2622872286 wait complete. 2024-07-31T14:55:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.208+0000 7f0fb1a33740 20 client.26509 trim_cache size 0 max 16384 2024-07-31T14:55:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.223+0000 7f0fb1a33740 1 Processor -- start 2024-07-31T14:55:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.223+0000 7f0fb1a33740 1 -- start start 2024-07-31T14:55:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fb1a33740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207312f0 0x55ed20b07f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fb1a33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207394d0 0x55ed20b143c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fb1a33740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 0x55ed20af92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fb1a33740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41ae400 con 0x55ed207312f0 2024-07-31T14:55:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fb1a33740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41b00c0 con 0x55ed2075a1b0 2024-07-31T14:55:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fb1a33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41aead0 con 0x55ed207394d0 2024-07-31T14:55:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fa8804640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207312f0 0x55ed20b07f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fa8804640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207312f0 0x55ed20b07f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58424/0 (socket says 172.21.15.67:58424) 2024-07-31T14:55:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fa8804640 1 -- 172.21.15.67:0/918682327 learned_addr learned my addr 172.21.15.67:0/918682327 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fa9005640 1 --2- 172.21.15.67:0/918682327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 0x55ed20af92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fa3fff640 1 --2- 172.21.15.67:0/918682327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207394d0 0x55ed20b143c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fa8804640 1 -- 172.21.15.67:0/918682327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207394d0 msgr2=0x55ed20b143c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fa8804640 1 --2- 172.21.15.67:0/918682327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207394d0 0x55ed20b143c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fa8804640 1 -- 172.21.15.67:0/918682327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 msgr2=0x55ed20af92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fa8804640 1 --2- 172.21.15.67:0/918682327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 0x55ed20af92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fa8804640 1 -- 172.21.15.67:0/918682327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed20718fa0 con 0x55ed207312f0 2024-07-31T14:55:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.224+0000 7f0fa8804640 1 --2- 172.21.15.67:0/918682327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207312f0 0x55ed20b07f30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0f9801da90 tx=0x7f0f98003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea60b93f3ef16d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.225+0000 7f0f8e7fc640 1 -- 172.21.15.67:0/918682327 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800fce0 con 0x55ed207312f0 2024-07-31T14:55:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.225+0000 7f0f8e7fc640 1 -- 172.21.15.67:0/918682327 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f9800fe80 con 0x55ed207312f0 2024-07-31T14:55:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.225+0000 7f0f8e7fc640 1 -- 172.21.15.67:0/918682327 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f980306d0 con 0x55ed207312f0 2024-07-31T14:55:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.225+0000 7f0fb1a33740 1 -- 172.21.15.67:0/918682327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207312f0 msgr2=0x55ed20b07f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.225+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/918682327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207312f0 0x55ed20b07f30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0f9801da90 tx=0x7f0f98003e50 comp rx=0 tx=0).stop 2024-07-31T14:55:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.225+0000 7f0fb1a33740 1 -- 172.21.15.67:0/918682327 shutdown_connections 2024-07-31T14:55:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.225+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/918682327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed2075a1b0 0x55ed20af92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.225+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/918682327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207394d0 0x55ed20b143c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.225+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/918682327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed207312f0 0x55ed20b07f30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.226+0000 7f0fb1a33740 1 -- 172.21.15.67:0/918682327 >> 172.21.15.67:0/918682327 conn(0x55ed2074e300 msgr2=0x55ed20af59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.226+0000 7f0fb1a33740 1 -- 172.21.15.67:0/918682327 shutdown_connections 2024-07-31T14:55:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.226+0000 7f0fb1a33740 1 -- 172.21.15.67:0/918682327 wait complete. 2024-07-31T14:55:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.226+0000 7f0fb1a33740 1 -- start start 2024-07-31T14:55:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.226+0000 7f0fb1a33740 1 Processor -- start 2024-07-31T14:55:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.226+0000 7f0fb1a33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 0x55ed20b10530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.226+0000 7f0fb1a33740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed207394d0 0x55ed20b10a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.226+0000 7f0fb1a33740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed2075a1b0 0x55ed20b0d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fb1a33740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41af9f0 con 0x55ed2075a1b0 2024-07-31T14:55:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fb1a33740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41af260 con 0x55ed207394d0 2024-07-31T14:55:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fb1a33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41ac3e0 con 0x55ed207312f0 2024-07-31T14:55:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fb1a33740 10 client.0 _ll_register_callbacks cb 0x7f0f98009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fa9005640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed2075a1b0 0x55ed20b0d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fa9005640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed2075a1b0 0x55ed20b0d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58438/0 (socket says 172.21.15.67:58438) 2024-07-31T14:55:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fa9005640 1 -- 172.21.15.67:0/2526530046 learned_addr learned my addr 172.21.15.67:0/2526530046 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fa3fff640 1 --2- 172.21.15.67:0/2526530046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed207394d0 0x55ed20b10a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fa8804640 1 --2- 172.21.15.67:0/2526530046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 0x55ed20b10530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fa9005640 1 -- 172.21.15.67:0/2526530046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 msgr2=0x55ed20b10530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fa9005640 1 --2- 172.21.15.67:0/2526530046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 0x55ed20b10530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fa9005640 1 -- 172.21.15.67:0/2526530046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed207394d0 msgr2=0x55ed20b10a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fa9005640 1 --2- 172.21.15.67:0/2526530046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed207394d0 0x55ed20b10a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fa9005640 1 -- 172.21.15.67:0/2526530046 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed20b0da40 con 0x55ed2075a1b0 2024-07-31T14:55:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fa9005640 1 --2- 172.21.15.67:0/2526530046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed2075a1b0 0x55ed20b0d500 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c02f440 tx=0x7f0f9c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65129e33be9550f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0f83fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:55:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0f83fff640 1 -- 172.21.15.67:0/2526530046 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c056020 con 0x55ed2075a1b0 2024-07-31T14:55:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0f83fff640 1 -- 172.21.15.67:0/2526530046 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f0f9c02f660 con 0x55ed2075a1b0 2024-07-31T14:55:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.227+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2526530046 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ed20a74300 con 0x55ed2075a1b0 2024-07-31T14:55:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.228+0000 7f0f83fff640 1 -- 172.21.15.67:0/2526530046 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0f9c061050 con 0x55ed2075a1b0 2024-07-31T14:55:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.228+0000 7f0fb1a33740 10 client.16899 fetch_fsmap learned FSMap version 42 2024-07-31T14:55:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.228+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2526530046 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ed20b85930 con 0x55ed2075a1b0 2024-07-31T14:55:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.228+0000 7f0f83fff640 1 -- 172.21.15.67:0/2526530046 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ed20a74300 con 0x55ed2075a1b0 2024-07-31T14:55:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.228+0000 7f0f83fff640 1 -- 172.21.15.67:0/2526530046 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f0f9c043020 con 0x55ed2075a1b0 2024-07-31T14:55:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.228+0000 7f0fb1a33740 10 client.16899 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:55:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.228+0000 7f0fb1a33740 10 client.16899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.228+0000 7f0fb1a33740 20 client.16899 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.228+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/2526530046 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ed206ee1c0 0x55ed20b87eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.228+0000 7f0fb1a33740 4 client.16899 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:55:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.228+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2526530046 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed20a6fda0 con 0x55ed206ee1c0 2024-07-31T14:55:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.228+0000 7f0fa8804640 1 --2- 172.21.15.67:0/2526530046 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ed206ee1c0 0x55ed20b87eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.229+0000 7f0f83fff640 10 client.16899 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:51.229+0000 7f0fa8804640 1 --2- 172.21.15.67:0/2526530046 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ed206ee1c0 0x55ed20b87eb0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d884f0049d57e2f server_cookie=9c4c266725ac7c49 in_seq=0 out_seq=0 2024-07-31T14:55:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:50 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi073 to 176.2M 2024-07-31T14:55:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:50 smithi077 ceph-mon[37327]: Unable to set osd_memory_target on smithi073 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-07-31T14:55:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:50 smithi077 ceph-mon[37327]: pgmap v543: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 13 MiB/s wr, 1.31k op/s 2024-07-31T14:55:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:50 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi067 to 176.0M 2024-07-31T14:55:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:50 smithi077 ceph-mon[37327]: Unable to set osd_memory_target on smithi067 to 184601804: error parsing value: Value '184601804' is below minimum 939524096 2024-07-31T14:55:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:50 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T14:55:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:50 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:51.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:50 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:51.239 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:50 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi073 to 176.2M 2024-07-31T14:55:51.239 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:50 smithi067 ceph-mon[32750]: Unable to set osd_memory_target on smithi073 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-07-31T14:55:51.239 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:50 smithi067 ceph-mon[32750]: pgmap v543: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 13 MiB/s wr, 1.31k op/s 2024-07-31T14:55:51.239 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:50 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi067 to 176.0M 2024-07-31T14:55:51.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:50 smithi067 ceph-mon[32750]: Unable to set osd_memory_target on smithi067 to 184601804: error parsing value: Value '184601804' is below minimum 939524096 2024-07-31T14:55:51.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:50 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T14:55:51.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:50 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:51.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:50 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:51.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:50 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi073 to 176.2M 2024-07-31T14:55:51.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:50 smithi073 ceph-mon[36714]: Unable to set osd_memory_target on smithi073 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-07-31T14:55:51.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:50 smithi073 ceph-mon[36714]: pgmap v543: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 13 MiB/s wr, 1.31k op/s 2024-07-31T14:55:51.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:50 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi067 to 176.0M 2024-07-31T14:55:51.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:50 smithi073 ceph-mon[36714]: Unable to set osd_memory_target on smithi067 to 184601804: error parsing value: Value '184601804' is below minimum 939524096 2024-07-31T14:55:51.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:50 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T14:55:51.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:50 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:51.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:50 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:51.695 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:55:52.052 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:55:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:52 smithi077 ceph-mon[37327]: pgmap v544: 129 pgs: 129 active+clean; 939 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.5 MiB/s wr, 1.04k op/s 2024-07-31T14:55:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:53.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:52 smithi067 ceph-mon[32750]: pgmap v544: 129 pgs: 129 active+clean; 939 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.5 MiB/s wr, 1.04k op/s 2024-07-31T14:55:53.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:52 smithi073 ceph-mon[36714]: pgmap v544: 129 pgs: 129 active+clean; 939 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.5 MiB/s wr, 1.04k op/s 2024-07-31T14:55:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.757+0000 7f0f83fff640 1 -- 172.21.15.67:0/2526530046 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed20a6fda0 con 0x55ed206ee1c0 2024-07-31T14:55:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.757+0000 7f0f83fff640 10 client.16899 handle_command_reply: tid=0 2024-07-31T14:55:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.757+0000 7f0fb1a33740 1 client.16899 shutdown 2024-07-31T14:55:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.757+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2526530046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed2075a1b0 msgr2=0x55ed20b0d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.757+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/2526530046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed2075a1b0 0x55ed20b0d500 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c02f440 tx=0x7f0f9c0095c0 comp rx=0 tx=0).stop 2024-07-31T14:55:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.757+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2526530046 shutdown_connections 2024-07-31T14:55:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.757+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/2526530046 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ed206ee1c0 0x55ed20b87eb0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.757+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/2526530046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ed2075a1b0 0x55ed20b0d500 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.757+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/2526530046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed207394d0 0x55ed20b10a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.757+0000 7f0f83fff640 0 client.16899 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.757+0000 7f0fb1a33740 1 --2- 172.21.15.67:0/2526530046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed207312f0 0x55ed20b10530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.757+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2526530046 >> 172.21.15.67:0/2526530046 conn(0x55ed2074e300 msgr2=0x55ed20af59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.758+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2526530046 shutdown_connections 2024-07-31T14:55:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.758+0000 7f0fb1a33740 1 -- 172.21.15.67:0/2526530046 wait complete. 2024-07-31T14:55:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.758+0000 7f0fb1a33740 20 client.16899 trim_cache size 0 max 16384 2024-07-31T14:55:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:55:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:55:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:55:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:55:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.758+0000 7f0faaa68640 1 -- 172.21.15.67:0/7714930 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f940b3dc0 msgr2=0x7f0f940b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.758+0000 7f0faaa68640 1 --2- 172.21.15.67:0/7714930 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f940b3dc0 0x7f0f940b62d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009fd0 tx=0x7f0f98009780 comp rx=0 tx=0).stop 2024-07-31T14:55:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.758+0000 7f0faaa68640 1 -- 172.21.15.67:0/7714930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fa4107e40 msgr2=0x7f0fa419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.759+0000 7f0faaa68640 1 --2- 172.21.15.67:0/7714930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fa4107e40 0x7f0fa419b300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4075360 tx=0x7f0f9000ee50 comp rx=0 tx=0).stop 2024-07-31T14:55:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.759+0000 7f0faaa68640 1 -- 172.21.15.67:0/7714930 shutdown_connections 2024-07-31T14:55:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.759+0000 7f0faaa68640 1 --2- 172.21.15.67:0/7714930 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f940b3dc0 0x7f0f940b62d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.759+0000 7f0faaa68640 1 --2- 172.21.15.67:0/7714930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fa4108790 0x7f0fa419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.759+0000 7f0faaa68640 1 --2- 172.21.15.67:0/7714930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fa4107e40 0x7f0fa419b300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.759+0000 7f0faaa68640 1 --2- 172.21.15.67:0/7714930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fa4102b60 0x7f0fa419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.759+0000 7f0faaa68640 1 -- 172.21.15.67:0/7714930 >> 172.21.15.67:0/7714930 conn(0x7f0fa4079880 msgr2=0x7f0fa4105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.759+0000 7f0faaa68640 1 -- 172.21.15.67:0/7714930 shutdown_connections 2024-07-31T14:55:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:55:53.759+0000 7f0faaa68640 1 -- 172.21.15.67:0/7714930 wait complete. 2024-07-31T14:55:54.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:54.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:54.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:54.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:54.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:54.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:54.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:54.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:54.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:54.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:54.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:54.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:54.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:54.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:54.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:54.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:54.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:54.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:54.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:54.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:54.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: pgmap v545: 129 pgs: 129 active+clean; 955 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.4 MiB/s wr, 943 op/s 2024-07-31T14:55:55.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.242 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.242 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.242 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.242 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.242 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.242 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.242 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.242 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.242 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:55.242 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: pgmap v545: 129 pgs: 129 active+clean; 955 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.4 MiB/s wr, 943 op/s 2024-07-31T14:55:55.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:55.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: pgmap v545: 129 pgs: 129 active+clean; 955 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.4 MiB/s wr, 943 op/s 2024-07-31T14:55:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:56.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:56.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:56.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:56.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:56.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.071+0000 7f69eba01640 1 -- 172.21.15.67:0/2365803446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41024d0 msgr2=0x7f69e41028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.071+0000 7f69eba01640 1 --2- 172.21.15.67:0/2365803446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41024d0 0x7f69e41028e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009900 tx=0x7f69e002a410 comp rx=0 tx=0).stop 2024-07-31T14:55:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.071+0000 7f69eba01640 1 -- 172.21.15.67:0/2365803446 shutdown_connections 2024-07-31T14:55:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.071+0000 7f69eba01640 1 --2- 172.21.15.67:0/2365803446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69e4108760 0x7f69e410ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.071+0000 7f69eba01640 1 --2- 172.21.15.67:0/2365803446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69e4107e10 0x7f69e4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.071+0000 7f69eba01640 1 --2- 172.21.15.67:0/2365803446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41024d0 0x7f69e41028e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.071+0000 7f69eba01640 1 -- 172.21.15.67:0/2365803446 >> 172.21.15.67:0/2365803446 conn(0x7f69e40791e0 msgr2=0x7f69e40795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.071+0000 7f69eba01640 1 -- 172.21.15.67:0/2365803446 shutdown_connections 2024-07-31T14:55:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.071+0000 7f69eba01640 1 -- 172.21.15.67:0/2365803446 wait complete. 2024-07-31T14:55:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.072+0000 7f69eba01640 1 Processor -- start 2024-07-31T14:55:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.072+0000 7f69eba01640 1 -- start start 2024-07-31T14:55:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.072+0000 7f69eba01640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41024d0 0x7f69e419c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.072+0000 7f69eba01640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69e4107e10 0x7f69e419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.072+0000 7f69eba01640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69e4108760 0x7f69e41a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.072+0000 7f69eba01640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f69e4112410 con 0x7f69e41024d0 2024-07-31T14:55:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.072+0000 7f69eba01640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69e4112290 con 0x7f69e4108760 2024-07-31T14:55:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.072+0000 7f69eba01640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69e4112590 con 0x7f69e4107e10 2024-07-31T14:55:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.072+0000 7f69e979d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41024d0 0x7f69e419c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.072+0000 7f69e9f9e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69e4108760 0x7f69e41a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.072+0000 7f69e979d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41024d0 0x7f69e419c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58464/0 (socket says 172.21.15.67:58464) 2024-07-31T14:55:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.072+0000 7f69e979d640 1 -- 172.21.15.67:0/1189145987 learned_addr learned my addr 172.21.15.67:0/1189145987 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.072+0000 7f69e8f9c640 1 --2- 172.21.15.67:0/1189145987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69e4107e10 0x7f69e419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.073+0000 7f69e9f9e640 1 -- 172.21.15.67:0/1189145987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69e4107e10 msgr2=0x7f69e419c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.073+0000 7f69e9f9e640 1 --2- 172.21.15.67:0/1189145987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69e4107e10 0x7f69e419c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.073+0000 7f69e9f9e640 1 -- 172.21.15.67:0/1189145987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41024d0 msgr2=0x7f69e419c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.073+0000 7f69e9f9e640 1 --2- 172.21.15.67:0/1189145987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41024d0 0x7f69e419c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.073+0000 7f69e9f9e640 1 -- 172.21.15.67:0/1189145987 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e41a20b0 con 0x7f69e4108760 2024-07-31T14:55:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.073+0000 7f69e9f9e640 1 --2- 172.21.15.67:0/1189145987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69e4108760 0x7f69e41a1980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00e940 tx=0x7f69dc00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.073+0000 7f69da7fc640 1 -- 172.21.15.67:0/1189145987 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc004d20 con 0x7f69e4108760 2024-07-31T14:55:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.073+0000 7f69eba01640 1 -- 172.21.15.67:0/1189145987 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e41a23a0 con 0x7f69e4108760 2024-07-31T14:55:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.073+0000 7f69da7fc640 1 -- 172.21.15.67:0/1189145987 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69dc005020 con 0x7f69e4108760 2024-07-31T14:55:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.073+0000 7f69eba01640 1 -- 172.21.15.67:0/1189145987 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e41a8b00 con 0x7f69e4108760 2024-07-31T14:55:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.073+0000 7f69da7fc640 1 -- 172.21.15.67:0/1189145987 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0116d0 con 0x7f69e4108760 2024-07-31T14:55:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.076+0000 7f69f1f52740 1 Processor -- start 2024-07-31T14:55:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.077+0000 7f69da7fc640 1 -- 172.21.15.67:0/1189145987 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f69dc011870 con 0x7f69e4108760 2024-07-31T14:55:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.077+0000 7f69f1f52740 1 -- start start 2024-07-31T14:55:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.077+0000 7f69f1f52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 0x556bef7be550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.077+0000 7f69f1f52740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef3f01b0 0x556bef7bea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.077+0000 7f69f1f52740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f8390 0x556bef7befd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.077+0000 7f69f1f52740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556bef734b90 con 0x556bef3f01b0 2024-07-31T14:55:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.077+0000 7f69f1f52740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556bef724090 con 0x556bef3f8390 2024-07-31T14:55:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.077+0000 7f69f1f52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556bef7280d0 con 0x556bef419070 2024-07-31T14:55:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.077+0000 7f69e8f9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 0x556bef7be550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69e979d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef3f01b0 0x556bef7bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69e8f9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 0x556bef7be550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:55212/0 (socket says 172.21.15.67:55212) 2024-07-31T14:55:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69e8f9c640 1 -- 172.21.15.67:0/270429262 learned_addr learned my addr 172.21.15.67:0/270429262 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69e8f9c640 1 --2- 172.21.15.67:0/270429262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f8390 0x556bef7befd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69da7fc640 1 --2- 172.21.15.67:0/1189145987 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69d00b3dc0 0x7f69d00b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69e979d640 1 -- 172.21.15.67:0/270429262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 msgr2=0x556bef7be550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69e979d640 1 --2- 172.21.15.67:0/270429262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 0x556bef7be550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69e979d640 1 -- 172.21.15.67:0/270429262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f8390 msgr2=0x556bef7befd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69e979d640 1 --2- 172.21.15.67:0/270429262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f8390 0x556bef7befd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69e979d640 1 -- 172.21.15.67:0/270429262 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bef6cab70 con 0x556bef3f01b0 2024-07-31T14:55:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69e8f9c640 1 --2- 172.21.15.67:0/270429262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 0x556bef7be550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69e9f9e640 1 --2- 172.21.15.67:0/1189145987 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69d00b3dc0 0x7f69d00b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69e979d640 1 --2- 172.21.15.67:0/270429262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef3f01b0 0x556bef7bea90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f69e0004770 tx=0x7f69e0031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ab5890fddb7b65 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69cf7fe640 1 -- 172.21.15.67:0/270429262 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0040070 con 0x556bef3f01b0 2024-07-31T14:55:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.078+0000 7f69da7fc640 1 -- 172.21.15.67:0/1189145987 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f69dc0e40a0 con 0x7f69e4108760 2024-07-31T14:55:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.079+0000 7f69cf7fe640 1 -- 172.21.15.67:0/270429262 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69e0004030 con 0x556bef3f01b0 2024-07-31T14:55:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.079+0000 7f69cf7fe640 1 -- 172.21.15.67:0/270429262 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0035040 con 0x556bef3f01b0 2024-07-31T14:55:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.080+0000 7f69f1f52740 1 -- 172.21.15.67:0/270429262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef3f01b0 msgr2=0x556bef7bea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.080+0000 7f69e9f9e640 1 --2- 172.21.15.67:0/1189145987 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69d00b3dc0 0x7f69d00b62d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0e1ae0 tx=0x7f69dc09f2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.080+0000 7f69f1f52740 1 --2- 172.21.15.67:0/270429262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef3f01b0 0x556bef7bea90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f69e0004770 tx=0x7f69e0031b90 comp rx=0 tx=0).stop 2024-07-31T14:55:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.080+0000 7f69f1f52740 1 -- 172.21.15.67:0/270429262 shutdown_connections 2024-07-31T14:55:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.080+0000 7f69f1f52740 1 --2- 172.21.15.67:0/270429262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f8390 0x556bef7befd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.080+0000 7f69f1f52740 1 --2- 172.21.15.67:0/270429262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef3f01b0 0x556bef7bea90 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.080+0000 7f69f1f52740 1 --2- 172.21.15.67:0/270429262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 0x556bef7be550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.080+0000 7f69f1f52740 1 -- 172.21.15.67:0/270429262 >> 172.21.15.67:0/270429262 conn(0x556bef40d460 msgr2=0x556bef7bc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.080+0000 7f69f1f52740 1 -- 172.21.15.67:0/270429262 shutdown_connections 2024-07-31T14:55:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.080+0000 7f69f1f52740 1 -- 172.21.15.67:0/270429262 wait complete. 2024-07-31T14:55:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.080+0000 7f69f1f52740 1 -- start start 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.080+0000 7f69f1f52740 1 Processor -- start 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.081+0000 7f69f1f52740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f01b0 0x556bef7ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.081+0000 7f69f1f52740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef3f8390 0x556bef7ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.081+0000 7f69f1f52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 0x556bef7d6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.081+0000 7f69f1f52740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556bef587700 con 0x556bef3f8390 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.081+0000 7f69f1f52740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556bef6c69c0 con 0x556bef3f01b0 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.081+0000 7f69f1f52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556bef7cccd0 con 0x556bef419070 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.081+0000 7f69e8f9c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f01b0 0x556bef7ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.081+0000 7f69e8f9c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f01b0 0x556bef7ca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:33622/0 (socket says 172.21.15.67:33622) 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.081+0000 7f69e8f9c640 1 -- 172.21.15.67:0/1860683873 learned_addr learned my addr 172.21.15.67:0/1860683873 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.081+0000 7f69e979d640 1 --2- 172.21.15.67:0/1860683873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef3f8390 0x556bef7ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.081+0000 7f69f1f52740 10 client.0 _ll_register_callbacks cb 0x556bef6e14d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.081+0000 7f69e8f9c640 1 --2- 172.21.15.67:0/1860683873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 0x556bef7d6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.082+0000 7f69e8f9c640 1 -- 172.21.15.67:0/1860683873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 msgr2=0x556bef7d6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.082+0000 7f69e8f9c640 1 --2- 172.21.15.67:0/1860683873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 0x556bef7d6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.082+0000 7f69e8f9c640 1 -- 172.21.15.67:0/1860683873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef3f8390 msgr2=0x556bef7ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.082+0000 7f69e8f9c640 1 --2- 172.21.15.67:0/1860683873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef3f8390 0x556bef7ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.082+0000 7f69e8f9c640 1 -- 172.21.15.67:0/1860683873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bef484650 con 0x556bef3f01b0 2024-07-31T14:55:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.082+0000 7f69e8f9c640 1 --2- 172.21.15.67:0/1860683873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f01b0 0x556bef7ca0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f69d4004550 tx=0x7f69d4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f0a78f834ef375c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.082+0000 7f69ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:55:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.082+0000 7f69ccff9640 1 -- 172.21.15.67:0/1860683873 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4005220 con 0x556bef3f01b0 2024-07-31T14:55:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.083+0000 7f69ccff9640 1 -- 172.21.15.67:0/1860683873 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f69d40053c0 con 0x556bef3f01b0 2024-07-31T14:55:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.083+0000 7f69f1f52740 1 -- 172.21.15.67:0/1860683873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69e0004030 con 0x556bef3f01b0 2024-07-31T14:55:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.083+0000 7f69ccff9640 1 -- 172.21.15.67:0/1860683873 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f69d402a050 con 0x556bef3f01b0 2024-07-31T14:55:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.083+0000 7f69f1f52740 10 client.26501 fetch_fsmap learned FSMap version 42 2024-07-31T14:55:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.083+0000 7f69f1f52740 1 -- 172.21.15.67:0/1860683873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556bef7d8ce0 con 0x556bef3f01b0 2024-07-31T14:55:56.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.083+0000 7f69ccff9640 1 -- 172.21.15.67:0/1860683873 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69e0004030 con 0x556bef3f01b0 2024-07-31T14:55:56.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.083+0000 7f69ccff9640 1 -- 172.21.15.67:0/1860683873 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f69d4011900 con 0x556bef3f01b0 2024-07-31T14:55:56.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.084+0000 7f69f1f52740 10 client.26501 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:55:56.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.084+0000 7f69f1f52740 10 client.26501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:56.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.084+0000 7f69f1f52740 20 client.26501 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:56.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.084+0000 7f69f1f52740 1 --2- 172.21.15.67:0/1860683873 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bef3d79f0 0x556bef7dc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.084+0000 7f69f1f52740 4 client.26501 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:55:56.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.084+0000 7f69f1f52740 1 -- 172.21.15.67:0/1860683873 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bef736ae0 con 0x556bef3d79f0 2024-07-31T14:55:56.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.084+0000 7f69e979d640 1 --2- 172.21.15.67:0/1860683873 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bef3d79f0 0x556bef7dc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.084+0000 7f69ccff9640 10 client.26501 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:56.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.084+0000 7f69e979d640 1 --2- 172.21.15.67:0/1860683873 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bef3d79f0 0x556bef7dc6e0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76ba09ad84bc56ae server_cookie=32f2d970ff786539 in_seq=0 out_seq=0 2024-07-31T14:55:56.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.088+0000 7f69ccff9640 1 -- 172.21.15.67:0/1860683873 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556bef736ae0 con 0x556bef3d79f0 2024-07-31T14:55:56.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.088+0000 7f69ccff9640 10 client.26501 handle_command_reply: tid=0 2024-07-31T14:55:56.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.088+0000 7f69f1f52740 1 client.26501 shutdown 2024-07-31T14:55:56.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.088+0000 7f69f1f52740 1 -- 172.21.15.67:0/1860683873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f01b0 msgr2=0x556bef7ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:56.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.088+0000 7f69f1f52740 1 --2- 172.21.15.67:0/1860683873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f01b0 0x556bef7ca0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f69d4004550 tx=0x7f69d4005f90 comp rx=0 tx=0).stop 2024-07-31T14:55:56.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.088+0000 7f69f1f52740 1 -- 172.21.15.67:0/1860683873 shutdown_connections 2024-07-31T14:55:56.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.088+0000 7f69f1f52740 1 --2- 172.21.15.67:0/1860683873 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bef3d79f0 0x556bef7dc6e0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.088+0000 7f69f1f52740 1 --2- 172.21.15.67:0/1860683873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 0x556bef7d6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.088+0000 7f69f1f52740 1 --2- 172.21.15.67:0/1860683873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef3f8390 0x556bef7ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.088+0000 7f69f1f52740 1 --2- 172.21.15.67:0/1860683873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f01b0 0x556bef7ca0c0 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f69d4004550 tx=0x7f69d4005f90 comp rx=0 tx=0).stop 2024-07-31T14:55:56.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.088+0000 7f69f1f52740 1 -- 172.21.15.67:0/1860683873 >> 172.21.15.67:0/1860683873 conn(0x556bef40d460 msgr2=0x556bef7c16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:56.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.088+0000 7f69ccff9640 0 client.26501 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:56.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.088+0000 7f69f1f52740 1 -- 172.21.15.67:0/1860683873 shutdown_connections 2024-07-31T14:55:56.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.088+0000 7f69f1f52740 1 -- 172.21.15.67:0/1860683873 wait complete. 2024-07-31T14:55:56.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.089+0000 7f69f1f52740 20 client.26501 trim_cache size 0 max 16384 2024-07-31T14:55:56.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.109+0000 7f69f1f52740 1 Processor -- start 2024-07-31T14:55:56.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.109+0000 7f69f1f52740 1 -- start start 2024-07-31T14:55:56.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.109+0000 7f69f1f52740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef40d460 0x556bef7c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.109+0000 7f69f1f52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef3f8390 0x556bef7c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.109+0000 7f69f1f52740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef419070 0x556bef7c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.109+0000 7f69f1f52740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f69c800fe10 con 0x556bef40d460 2024-07-31T14:55:56.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.109+0000 7f69f1f52740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69c8011ad0 con 0x556bef419070 2024-07-31T14:55:56.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.109+0000 7f69f1f52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69c80104e0 con 0x556bef3f8390 2024-07-31T14:55:56.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.109+0000 7f69e8f9c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef40d460 0x556bef7c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.109+0000 7f69e8f9c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef40d460 0x556bef7c1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58492/0 (socket says 172.21.15.67:58492) 2024-07-31T14:55:56.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.109+0000 7f69e8f9c640 1 -- 172.21.15.67:0/2302166963 learned_addr learned my addr 172.21.15.67:0/2302166963 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:56.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.109+0000 7f69e979d640 1 --2- 172.21.15.67:0/2302166963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef3f8390 0x556bef7c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.110+0000 7f69e8f9c640 1 --2- 172.21.15.67:0/2302166963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef419070 0x556bef7c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.110+0000 7f69e8f9c640 1 -- 172.21.15.67:0/2302166963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef3f8390 msgr2=0x556bef7c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:56.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.110+0000 7f69e8f9c640 1 --2- 172.21.15.67:0/2302166963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef3f8390 0x556bef7c37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.110+0000 7f69e8f9c640 1 -- 172.21.15.67:0/2302166963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef419070 msgr2=0x556bef7c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:56.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.110+0000 7f69e8f9c640 1 --2- 172.21.15.67:0/2302166963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef419070 0x556bef7c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.110+0000 7f69e8f9c640 1 -- 172.21.15.67:0/2302166963 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bef7cb180 con 0x556bef40d460 2024-07-31T14:55:56.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.110+0000 7f69e979d640 1 --2- 172.21.15.67:0/2302166963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef3f8390 0x556bef7c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:55:56.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.110+0000 7f69e8f9c640 1 --2- 172.21.15.67:0/2302166963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef40d460 0x556bef7c1d60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f69d40097c0 tx=0x7f69d4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5ded58cf9da106 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:56.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.110+0000 7f69ceffd640 1 -- 172.21.15.67:0/2302166963 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4003e50 con 0x556bef40d460 2024-07-31T14:55:56.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.110+0000 7f69ceffd640 1 -- 172.21.15.67:0/2302166963 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69d4021450 con 0x556bef40d460 2024-07-31T14:55:56.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.110+0000 7f69ceffd640 1 -- 172.21.15.67:0/2302166963 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4004730 con 0x556bef40d460 2024-07-31T14:55:56.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.111+0000 7f69f1f52740 1 -- 172.21.15.67:0/2302166963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef40d460 msgr2=0x556bef7c1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:56.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.111+0000 7f69f1f52740 1 --2- 172.21.15.67:0/2302166963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef40d460 0x556bef7c1d60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f69d40097c0 tx=0x7f69d4005850 comp rx=0 tx=0).stop 2024-07-31T14:55:56.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.111+0000 7f69f1f52740 1 -- 172.21.15.67:0/2302166963 shutdown_connections 2024-07-31T14:55:56.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.111+0000 7f69f1f52740 1 --2- 172.21.15.67:0/2302166963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef419070 0x556bef7c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.111+0000 7f69f1f52740 1 --2- 172.21.15.67:0/2302166963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef3f8390 0x556bef7c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.111+0000 7f69f1f52740 1 --2- 172.21.15.67:0/2302166963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef40d460 0x556bef7c1d60 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.111+0000 7f69f1f52740 1 -- 172.21.15.67:0/2302166963 >> 172.21.15.67:0/2302166963 conn(0x556bef7ca780 msgr2=0x556bef7c1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.111+0000 7f69f1f52740 1 -- 172.21.15.67:0/2302166963 shutdown_connections 2024-07-31T14:55:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.111+0000 7f69f1f52740 1 -- 172.21.15.67:0/2302166963 wait complete. 2024-07-31T14:55:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.112+0000 7f69f1f52740 1 -- start start 2024-07-31T14:55:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.112+0000 7f69f1f52740 1 Processor -- start 2024-07-31T14:55:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.112+0000 7f69f1f52740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f8390 0x556bef7d2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.112+0000 7f69f1f52740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef40d460 0x556bef7d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.112+0000 7f69f1f52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 0x556bef7d5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.112+0000 7f69f1f52740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f69c8011400 con 0x556bef40d460 2024-07-31T14:55:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.112+0000 7f69f1f52740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69c8010c70 con 0x556bef3f8390 2024-07-31T14:55:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.112+0000 7f69f1f52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69c800ddf0 con 0x556bef419070 2024-07-31T14:55:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.112+0000 7f69f1f52740 10 client.0 _ll_register_callbacks cb 0x7f69d4004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:55:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.112+0000 7f69e979d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef40d460 0x556bef7d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.112+0000 7f69e979d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef40d460 0x556bef7d27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58508/0 (socket says 172.21.15.67:58508) 2024-07-31T14:55:56.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.112+0000 7f69e979d640 1 -- 172.21.15.67:0/2820001431 learned_addr learned my addr 172.21.15.67:0/2820001431 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:55:56.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.112+0000 7f69e8f9c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f8390 0x556bef7d2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.113+0000 7f69e8f9c640 1 --2- 172.21.15.67:0/2820001431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 0x556bef7d5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.113+0000 7f69e979d640 1 -- 172.21.15.67:0/2820001431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 msgr2=0x556bef7d5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:56.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.113+0000 7f69e979d640 1 --2- 172.21.15.67:0/2820001431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 0x556bef7d5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.113+0000 7f69e979d640 1 -- 172.21.15.67:0/2820001431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f8390 msgr2=0x556bef7d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:55:56.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.113+0000 7f69e979d640 1 --2- 172.21.15.67:0/2820001431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f8390 0x556bef7d2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.113+0000 7f69e979d640 1 -- 172.21.15.67:0/2820001431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bef7d5db0 con 0x556bef40d460 2024-07-31T14:55:56.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.113+0000 7f69e979d640 1 --2- 172.21.15.67:0/2820001431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef40d460 0x556bef7d27e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f69e0005090 tx=0x7f69e00050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2e6890aeefe048 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:55:56.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.113+0000 7f69ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:55:56.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.113+0000 7f69ccff9640 1 -- 172.21.15.67:0/2820001431 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0040040 con 0x556bef40d460 2024-07-31T14:55:56.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.113+0000 7f69ccff9640 1 -- 172.21.15.67:0/2820001431 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f69e0004450 con 0x556bef40d460 2024-07-31T14:55:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.113+0000 7f69f1f52740 1 -- 172.21.15.67:0/2820001431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69d4021450 con 0x556bef40d460 2024-07-31T14:55:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.113+0000 7f69ccff9640 1 -- 172.21.15.67:0/2820001431 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f69e0035040 con 0x556bef40d460 2024-07-31T14:55:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.113+0000 7f69f1f52740 10 client.16917 fetch_fsmap learned FSMap version 42 2024-07-31T14:55:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.113+0000 7f69f1f52740 1 -- 172.21.15.67:0/2820001431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556bef7d1a60 con 0x556bef40d460 2024-07-31T14:55:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.114+0000 7f69ccff9640 1 -- 172.21.15.67:0/2820001431 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69d4021450 con 0x556bef40d460 2024-07-31T14:55:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.114+0000 7f69ccff9640 1 -- 172.21.15.67:0/2820001431 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f69e00047a0 con 0x556bef40d460 2024-07-31T14:55:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.114+0000 7f69f1f52740 10 client.16917 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:55:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.114+0000 7f69f1f52740 10 client.16917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:55:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.114+0000 7f69f1f52740 20 client.16917 populate_metadata read hostname 'smithi067' 2024-07-31T14:55:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.114+0000 7f69f1f52740 1 --2- 172.21.15.67:0/2820001431 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bef3acda0 0x556bef846ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:55:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.114+0000 7f69f1f52740 4 client.16917 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:55:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.114+0000 7f69f1f52740 1 -- 172.21.15.67:0/2820001431 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556bef732580 con 0x556bef3acda0 2024-07-31T14:55:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.114+0000 7f69e8f9c640 1 --2- 172.21.15.67:0/2820001431 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bef3acda0 0x556bef846ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:55:56.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.114+0000 7f69ccff9640 10 client.16917 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:56.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.114+0000 7f69e8f9c640 1 --2- 172.21.15.67:0/2820001431 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bef3acda0 0x556bef846ec0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0f0371211a1ea7b server_cookie=f0f028b00a419c27 in_seq=0 out_seq=0 2024-07-31T14:55:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.117+0000 7f69ccff9640 1 -- 172.21.15.67:0/2820001431 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x556bef732580 con 0x556bef3acda0 2024-07-31T14:55:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.117+0000 7f69ccff9640 10 client.16917 handle_command_reply: tid=0 2024-07-31T14:55:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.117+0000 7f69f1f52740 1 client.16917 shutdown 2024-07-31T14:55:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69f1f52740 1 -- 172.21.15.67:0/2820001431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef40d460 msgr2=0x556bef7d27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69f1f52740 1 --2- 172.21.15.67:0/2820001431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef40d460 0x556bef7d27e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f69e0005090 tx=0x7f69e00050c0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69f1f52740 1 -- 172.21.15.67:0/2820001431 shutdown_connections 2024-07-31T14:55:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69f1f52740 1 --2- 172.21.15.67:0/2820001431 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556bef3acda0 0x556bef846ec0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69f1f52740 1 --2- 172.21.15.67:0/2820001431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bef419070 0x556bef7d5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69f1f52740 1 --2- 172.21.15.67:0/2820001431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556bef40d460 0x556bef7d27e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69f1f52740 1 --2- 172.21.15.67:0/2820001431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556bef3f8390 0x556bef7d2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69f1f52740 1 -- 172.21.15.67:0/2820001431 >> 172.21.15.67:0/2820001431 conn(0x556bef7ca780 msgr2=0x556bef7cb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:56.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69ccff9640 0 client.16917 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:56.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69f1f52740 1 -- 172.21.15.67:0/2820001431 shutdown_connections 2024-07-31T14:55:56.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69f1f52740 1 -- 172.21.15.67:0/2820001431 wait complete. 2024-07-31T14:55:56.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69f1f52740 20 client.16917 trim_cache size 0 max 16384 2024-07-31T14:55:56.123 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:55:56.123 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (2292 inodes in the stack)", 2024-07-31T14:55:56.123 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T14:55:56.123 INFO:teuthology.orchestra.run.smithi067.stdout: "ce8320d6-b226-42dc-9adb-9aff7bde586b": { 2024-07-31T14:55:56.124 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T14:55:56.124 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "ce8320d6-b226-42dc-9adb-9aff7bde586b", 2024-07-31T14:55:56.124 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T14:55:56.124 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:55:56.124 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:55:56.124 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:55:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69eba01640 1 -- 172.21.15.67:0/1189145987 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69d00b3dc0 msgr2=0x7f69d00b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69eba01640 1 --2- 172.21.15.67:0/1189145987 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69d00b3dc0 0x7f69d00b62d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0e1ae0 tx=0x7f69dc09f2d0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69eba01640 1 -- 172.21.15.67:0/1189145987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69e4108760 msgr2=0x7f69e41a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69eba01640 1 --2- 172.21.15.67:0/1189145987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69e4108760 0x7f69e41a1980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00e940 tx=0x7f69dc00ee00 comp rx=0 tx=0).stop 2024-07-31T14:55:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69eba01640 1 -- 172.21.15.67:0/1189145987 shutdown_connections 2024-07-31T14:55:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69eba01640 1 --2- 172.21.15.67:0/1189145987 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69d00b3dc0 0x7f69d00b62d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69eba01640 1 --2- 172.21.15.67:0/1189145987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69e4108760 0x7f69e41a1980 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69eba01640 1 --2- 172.21.15.67:0/1189145987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69e4107e10 0x7f69e419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69eba01640 1 --2- 172.21.15.67:0/1189145987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41024d0 0x7f69e419c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.118+0000 7f69eba01640 1 -- 172.21.15.67:0/1189145987 >> 172.21.15.67:0/1189145987 conn(0x7f69e40791e0 msgr2=0x7f69e406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.120+0000 7f69eba01640 1 -- 172.21.15.67:0/1189145987 shutdown_connections 2024-07-31T14:55:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:56.120+0000 7f69eba01640 1 -- 172.21.15.67:0/1189145987 wait complete. 2024-07-31T14:55:56.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:56.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:56.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:56.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:56.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:56.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:56.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:56.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:56.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:56.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:56.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:56.809 INFO:tasks.cephfs.filesystem:scrub status for tag:ce8320d6-b226-42dc-9adb-9aff7bde586b - {'path': '/', 'tag': 'ce8320d6-b226-42dc-9adb-9aff7bde586b', 'options': 'recursive,force'} 2024-07-31T14:55:57.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:57.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:57.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:56 smithi067 ceph-mon[32750]: pgmap v546: 129 pgs: 129 active+clean; 961 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 1.05k op/s 2024-07-31T14:55:57.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:57.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:57.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:57.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:56 smithi077 ceph-mon[37327]: pgmap v546: 129 pgs: 129 active+clean; 961 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 1.05k op/s 2024-07-31T14:55:57.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:57.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:56 smithi073 ceph-mon[36714]: pgmap v546: 129 pgs: 129 active+clean; 961 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 1.05k op/s 2024-07-31T14:55:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.065+0000 7ff5977fe640 1 -- 172.21.15.67:0/3379120735 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c9c9248700 con 0x55c9c8f2b840 2024-07-31T14:55:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.065+0000 7ff5977fe640 10 client.26399 handle_command_reply: tid=0 2024-07-31T14:55:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.065+0000 7ff5c4f2c740 1 client.26399 shutdown 2024-07-31T14:55:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.066+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/3379120735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f02980 msgr2=0x55c9c92f6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.066+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/3379120735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f02980 0x55c9c92f6440 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55c9c92d04f0 tx=0x7ff5ac030340 comp rx=0 tx=0).stop 2024-07-31T14:55:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.066+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/3379120735 shutdown_connections 2024-07-31T14:55:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.066+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/3379120735 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c9c8f2b840 0x55c9c936b050 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.066+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/3379120735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c9c8f0ab60 0x55c9c92f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.066+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/3379120735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c9c8f02980 0x55c9c92f6440 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.066+0000 7ff5977fe640 0 client.26399 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:55:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.066+0000 7ff5c4f2c740 1 --2- 172.21.15.67:0/3379120735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c9c8eea1c0 0x55c9c92f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:58.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.066+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/3379120735 >> 172.21.15.67:0/3379120735 conn(0x55c9c8f1f990 msgr2=0x55c9c92d6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:58.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.066+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/3379120735 shutdown_connections 2024-07-31T14:55:58.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.066+0000 7ff5c4f2c740 1 -- 172.21.15.67:0/3379120735 wait complete. 2024-07-31T14:55:58.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.066+0000 7ff5c4f2c740 20 client.26399 trim_cache size 0 max 16384 2024-07-31T14:55:58.069 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:55:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 42, 2024-07-31T14:55:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T14:55:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 10, 2024-07-31T14:55:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T14:55:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "c736f69": { 2024-07-31T14:55:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 10, 2024-07-31T14:55:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T14:55:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:55:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T14:55:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:55:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:55:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T14:55:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 138.4, 2024-07-31T14:55:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T14:55:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T14:55:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T14:55:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:55:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T14:55:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:55:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:55:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:55:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:55:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:55:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:55:58.071 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:55:58.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.067+0000 7ff5be9db640 1 -- 172.21.15.67:0/2483300900 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5a00b3f50 msgr2=0x7ff5a00b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:58.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.067+0000 7ff5be9db640 1 --2- 172.21.15.67:0/2483300900 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5a00b3f50 0x7ff5a00b6460 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0005fd0 tx=0x7ff5b000f820 comp rx=0 tx=0).stop 2024-07-31T14:55:58.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.067+0000 7ff5be9db640 1 -- 172.21.15.67:0/2483300900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5b810a740 msgr2=0x7ff5b81a9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:55:58.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.067+0000 7ff5be9db640 1 --2- 172.21.15.67:0/2483300900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5b810a740 0x7ff5b81a9350 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac035040 tx=0x7ff5ac032cd0 comp rx=0 tx=0).stop 2024-07-31T14:55:58.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.067+0000 7ff5be9db640 1 -- 172.21.15.67:0/2483300900 shutdown_connections 2024-07-31T14:55:58.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.067+0000 7ff5be9db640 1 --2- 172.21.15.67:0/2483300900 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff5a00b3f50 0x7ff5a00b6460 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:58.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.067+0000 7ff5be9db640 1 --2- 172.21.15.67:0/2483300900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5b811cb10 0x7ff5b81ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:58.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.067+0000 7ff5be9db640 1 --2- 172.21.15.67:0/2483300900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5b810a740 0x7ff5b81a9350 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:58.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.067+0000 7ff5be9db640 1 --2- 172.21.15.67:0/2483300900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b8109df0 0x7ff5b81a8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:55:58.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.067+0000 7ff5be9db640 1 -- 172.21.15.67:0/2483300900 >> 172.21.15.67:0/2483300900 conn(0x7ff5b80708b0 msgr2=0x7ff5b8071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:55:58.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.067+0000 7ff5be9db640 1 -- 172.21.15.67:0/2483300900 shutdown_connections 2024-07-31T14:55:58.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:55:58.067+0000 7ff5be9db640 1 -- 172.21.15.67:0/2483300900 wait complete. 2024-07-31T14:55:58.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:58.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:58.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:58.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:58.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:58.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:58.872 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c736f69', quiesce took 35.5 seconds. Will release after: -17.1 2024-07-31T14:55:58.872 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:55:59.200 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:55:59.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: pgmap v547: 129 pgs: 129 active+clean; 976 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 1.16k op/s 2024-07-31T14:55:59.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:55:59.298 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: pgmap v547: 129 pgs: 129 active+clean; 976 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 1.16k op/s 2024-07-31T14:55:59.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:55:59.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: pgmap v547: 129 pgs: 129 active+clean; 976 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 1.16k op/s 2024-07-31T14:55:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:55:59.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:55:59.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:00.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:55:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:00.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:55:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:00.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:00.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:55:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: pgmap v548: 129 pgs: 129 active+clean; 978 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 1.22k op/s 2024-07-31T14:56:01.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:56:01.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: pgmap v548: 129 pgs: 129 active+clean; 978 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 1.22k op/s 2024-07-31T14:56:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:56:01.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: pgmap v548: 129 pgs: 129 active+clean; 978 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 1.22k op/s 2024-07-31T14:56:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:56:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:56:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.818+0000 7f2950413640 1 -- 172.21.15.67:0/1073930778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2948108720 msgr2=0x7f294810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.818+0000 7f2950413640 1 --2- 172.21.15.67:0/1073930778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2948108720 0x7f294810ef50 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2940009900 tx=0x7f294002a410 comp rx=0 tx=0).stop 2024-07-31T14:56:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.819+0000 7f2950413640 1 -- 172.21.15.67:0/1073930778 shutdown_connections 2024-07-31T14:56:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.819+0000 7f2950413640 1 --2- 172.21.15.67:0/1073930778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2948108720 0x7f294810ef50 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.819+0000 7f2950413640 1 --2- 172.21.15.67:0/1073930778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2948107dd0 0x7f29481081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.819+0000 7f2950413640 1 --2- 172.21.15.67:0/1073930778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2948102490 0x7f29481028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.819+0000 7f2950413640 1 -- 172.21.15.67:0/1073930778 >> 172.21.15.67:0/1073930778 conn(0x7f2948079160 msgr2=0x7f2948079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.819+0000 7f2950413640 1 -- 172.21.15.67:0/1073930778 shutdown_connections 2024-07-31T14:56:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.819+0000 7f2950413640 1 -- 172.21.15.67:0/1073930778 wait complete. 2024-07-31T14:56:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.819+0000 7f2950413640 1 Processor -- start 2024-07-31T14:56:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.819+0000 7f2950413640 1 -- start start 2024-07-31T14:56:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f2950413640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2948102490 0x7f294819c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f2950413640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2948107dd0 0x7f294819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:01.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f2950413640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2948108720 0x7f29481a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:01.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f2950413640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f29481134b0 con 0x7f2948107dd0 2024-07-31T14:56:01.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f2950413640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2948113330 con 0x7f2948108720 2024-07-31T14:56:01.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f2950413640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2948113630 con 0x7f2948102490 2024-07-31T14:56:01.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f294d9ae640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2948107dd0 0x7f294819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:01.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f294d9ae640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2948107dd0 0x7f294819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37138/0 (socket says 172.21.15.67:37138) 2024-07-31T14:56:01.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f294e1af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2948102490 0x7f294819c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:01.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f294d9ae640 1 -- 172.21.15.67:0/362194356 learned_addr learned my addr 172.21.15.67:0/362194356 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:01.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f294e9b0640 1 --2- 172.21.15.67:0/362194356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2948108720 0x7f29481a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:01.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f294d9ae640 1 -- 172.21.15.67:0/362194356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2948102490 msgr2=0x7f294819c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:01.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f294d9ae640 1 --2- 172.21.15.67:0/362194356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2948102490 0x7f294819c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:01.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f294d9ae640 1 -- 172.21.15.67:0/362194356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2948108720 msgr2=0x7f29481a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:01.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f294d9ae640 1 --2- 172.21.15.67:0/362194356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2948108720 0x7f29481a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:01.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f294d9ae640 1 -- 172.21.15.67:0/362194356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29481a2050 con 0x7f2948107dd0 2024-07-31T14:56:01.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.820+0000 7f294e1af640 1 --2- 172.21.15.67:0/362194356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2948102490 0x7f294819c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:01.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.821+0000 7f294d9ae640 1 --2- 172.21.15.67:0/362194356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2948107dd0 0x7f294819c680 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f293c00e9f0 tx=0x7f293c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:01.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.821+0000 7f29477fe640 1 -- 172.21.15.67:0/362194356 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c00ccc0 con 0x7f2948107dd0 2024-07-31T14:56:01.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.821+0000 7f2950413640 1 -- 172.21.15.67:0/362194356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29481a2340 con 0x7f2948107dd0 2024-07-31T14:56:01.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.821+0000 7f29477fe640 1 -- 172.21.15.67:0/362194356 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f293c0054d0 con 0x7f2948107dd0 2024-07-31T14:56:01.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.821+0000 7f29477fe640 1 -- 172.21.15.67:0/362194356 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c010430 con 0x7f2948107dd0 2024-07-31T14:56:01.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.821+0000 7f2950413640 1 -- 172.21.15.67:0/362194356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29481a8aa0 con 0x7f2948107dd0 2024-07-31T14:56:01.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.822+0000 7f2950413640 1 -- 172.21.15.67:0/362194356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29481028a0 con 0x7f2948107dd0 2024-07-31T14:56:01.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.824+0000 7f29477fe640 1 -- 172.21.15.67:0/362194356 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f293c0046c0 con 0x7f2948107dd0 2024-07-31T14:56:01.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.825+0000 7f29477fe640 1 --2- 172.21.15.67:0/362194356 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f29300b3d40 0x7f29300b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:01.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.825+0000 7f294e1af640 1 --2- 172.21.15.67:0/362194356 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f29300b3d40 0x7f29300b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:01.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.825+0000 7f29477fe640 1 -- 172.21.15.67:0/362194356 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f293c0e47c0 con 0x7f2948107dd0 2024-07-31T14:56:01.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.826+0000 7f294e1af640 1 --2- 172.21.15.67:0/362194356 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f29300b3d40 0x7f29300b6250 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2938009fd0 tx=0x7f2938009760 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:01.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:01.828+0000 7f29477fe640 1 -- 172.21.15.67:0/362194356 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f293c0555e0 con 0x7f2948107dd0 2024-07-31T14:56:02.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:02.061+0000 7f2950413640 1 -- 172.21.15.67:0/362194356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f294819d210 con 0x7f2948107dd0 2024-07-31T14:56:02.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:02.061+0000 7f29477fe640 1 -- 172.21.15.67:0/362194356 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11981 (secure 0 0 0) 0x7f293c0ab7c0 con 0x7f2948107dd0 2024-07-31T14:56:02.063 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:56:02.063 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":42,"btime":"2024-07-31T14:55:38:924536+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:55:38.678172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:56:02.064 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 42 2024-07-31T14:56:02.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:02.064+0000 7f2950413640 1 -- 172.21.15.67:0/362194356 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f29300b3d40 msgr2=0x7f29300b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:02.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:02.064+0000 7f2950413640 1 --2- 172.21.15.67:0/362194356 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f29300b3d40 0x7f29300b6250 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2938009fd0 tx=0x7f2938009760 comp rx=0 tx=0).stop 2024-07-31T14:56:02.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:02.064+0000 7f2950413640 1 -- 172.21.15.67:0/362194356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2948107dd0 msgr2=0x7f294819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:02.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:02.064+0000 7f2950413640 1 --2- 172.21.15.67:0/362194356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2948107dd0 0x7f294819c680 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f293c00e9f0 tx=0x7f293c00eeb0 comp rx=0 tx=0).stop 2024-07-31T14:56:02.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:02.064+0000 7f2950413640 1 -- 172.21.15.67:0/362194356 shutdown_connections 2024-07-31T14:56:02.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:02.064+0000 7f2950413640 1 --2- 172.21.15.67:0/362194356 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f29300b3d40 0x7f29300b6250 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:02.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:02.064+0000 7f2950413640 1 --2- 172.21.15.67:0/362194356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2948108720 0x7f29481a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:02.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:02.064+0000 7f2950413640 1 --2- 172.21.15.67:0/362194356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2948107dd0 0x7f294819c680 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:02.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:02.064+0000 7f2950413640 1 --2- 172.21.15.67:0/362194356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2948102490 0x7f294819c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:02.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:02.064+0000 7f2950413640 1 -- 172.21.15.67:0/362194356 >> 172.21.15.67:0/362194356 conn(0x7f2948079160 msgr2=0x7f294810f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:02.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:02.064+0000 7f2950413640 1 -- 172.21.15.67:0/362194356 shutdown_connections 2024-07-31T14:56:02.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:02.064+0000 7f2950413640 1 -- 172.21.15.67:0/362194356 wait complete. 2024-07-31T14:56:02.808 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id c736f69 --release --await' 2024-07-31T14:56:02.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id c736f69 --release --await 2024-07-31T14:56:03.077 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:56:03.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:02 smithi067 ceph-mon[32750]: pgmap v549: 129 pgs: 129 active+clean; 988 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 1.52k op/s 2024-07-31T14:56:03.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:02 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/362194356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:56:03.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:02 smithi073 ceph-mon[36714]: pgmap v549: 129 pgs: 129 active+clean; 988 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 1.52k op/s 2024-07-31T14:56:03.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:02 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/362194356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:56:03.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:02 smithi077 ceph-mon[37327]: pgmap v549: 129 pgs: 129 active+clean; 988 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 1.52k op/s 2024-07-31T14:56:03.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:02 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/362194356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:56:05.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:05 smithi067 ceph-mon[32750]: pgmap v550: 129 pgs: 129 active+clean; 989 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 1.93k op/s 2024-07-31T14:56:05.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.358+0000 7f81f7947640 1 -- 172.21.15.67:0/2418160050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f01024b0 msgr2=0x7f81f01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:05.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.358+0000 7f81f7947640 1 --2- 172.21.15.67:0/2418160050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f01024b0 0x7f81f01028c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009900 tx=0x7f81ec02a410 comp rx=0 tx=0).stop 2024-07-31T14:56:05.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.359+0000 7f81f7947640 1 -- 172.21.15.67:0/2418160050 shutdown_connections 2024-07-31T14:56:05.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.359+0000 7f81f7947640 1 --2- 172.21.15.67:0/2418160050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f0108740 0x7f81f010ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.359+0000 7f81f7947640 1 --2- 172.21.15.67:0/2418160050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0107df0 0x7f81f0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.359+0000 7f81f7947640 1 --2- 172.21.15.67:0/2418160050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f01024b0 0x7f81f01028c0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.359+0000 7f81f7947640 1 -- 172.21.15.67:0/2418160050 >> 172.21.15.67:0/2418160050 conn(0x7f81f0079170 msgr2=0x7f81f0079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:05.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.359+0000 7f81f7947640 1 -- 172.21.15.67:0/2418160050 shutdown_connections 2024-07-31T14:56:05.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.359+0000 7f81f7947640 1 -- 172.21.15.67:0/2418160050 wait complete. 2024-07-31T14:56:05.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.359+0000 7f81f7947640 1 Processor -- start 2024-07-31T14:56:05.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.359+0000 7f81f7947640 1 -- start start 2024-07-31T14:56:05.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f7947640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f01024b0 0x7f81f006b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f7947640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0107df0 0x7f81f006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f7947640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f0108740 0x7f81f019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f7947640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01123a0 con 0x7f81f0108740 2024-07-31T14:56:05.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f7947640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f81f0112220 con 0x7f81f0107df0 2024-07-31T14:56:05.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f7947640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f81f0112520 con 0x7f81f01024b0 2024-07-31T14:56:05.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f5ee4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f0108740 0x7f81f019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f5ee4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f0108740 0x7f81f019c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37162/0 (socket says 172.21.15.67:37162) 2024-07-31T14:56:05.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f5ee4640 1 -- 172.21.15.67:0/3076467241 learned_addr learned my addr 172.21.15.67:0/3076467241 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:05.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f56e3640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f01024b0 0x7f81f006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f4ee2640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0107df0 0x7f81f006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f5ee4640 1 -- 172.21.15.67:0/3076467241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f01024b0 msgr2=0x7f81f006b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:05.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f5ee4640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f01024b0 0x7f81f006b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f5ee4640 1 -- 172.21.15.67:0/3076467241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0107df0 msgr2=0x7f81f006bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:05.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f5ee4640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0107df0 0x7f81f006bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f5ee4640 1 -- 172.21.15.67:0/3076467241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f019cbc0 con 0x7f81f0108740 2024-07-31T14:56:05.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f4ee2640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0107df0 0x7f81f006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:56:05.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.360+0000 7f81f5ee4640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f0108740 0x7f81f019c490 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f81e800cad0 tx=0x7f81e800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:05.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.361+0000 7f81e27fc640 1 -- 172.21.15.67:0/3076467241 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e800ae20 con 0x7f81f0108740 2024-07-31T14:56:05.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.361+0000 7f81e27fc640 1 -- 172.21.15.67:0/3076467241 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e8005020 con 0x7f81f0108740 2024-07-31T14:56:05.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.361+0000 7f81f7947640 1 -- 172.21.15.67:0/3076467241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f019ceb0 con 0x7f81f0108740 2024-07-31T14:56:05.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.361+0000 7f81e27fc640 1 -- 172.21.15.67:0/3076467241 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e800f6b0 con 0x7f81f0108740 2024-07-31T14:56:05.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.361+0000 7f81f7947640 1 -- 172.21.15.67:0/3076467241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f01a6850 con 0x7f81f0108740 2024-07-31T14:56:05.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.364+0000 7f81fde98740 1 Processor -- start 2024-07-31T14:56:05.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.364+0000 7f81e27fc640 1 -- 172.21.15.67:0/3076467241 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f81e80051d0 con 0x7f81f0108740 2024-07-31T14:56:05.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.364+0000 7f81fde98740 1 -- start start 2024-07-31T14:56:05.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.364+0000 7f81fde98740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6f1520 0x55c89da97070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.364+0000 7f81fde98740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6c8660 0x55c89da975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.364+0000 7f81fde98740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6d0840 0x55c89da97af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.364+0000 7f81fde98740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c89da0ce70 con 0x55c89d6d0840 2024-07-31T14:56:05.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.364+0000 7f81fde98740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c89d9fc370 con 0x55c89d6f1520 2024-07-31T14:56:05.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.364+0000 7f81fde98740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c89da003b0 con 0x55c89d6c8660 2024-07-31T14:56:05.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.364+0000 7f81f4ee2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6d0840 0x55c89da97af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.364+0000 7f81f4ee2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6d0840 0x55c89da97af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37164/0 (socket says 172.21.15.67:37164) 2024-07-31T14:56:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.364+0000 7f81f4ee2640 1 -- 172.21.15.67:0/3402684796 learned_addr learned my addr 172.21.15.67:0/3402684796 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.364+0000 7f81f56e3640 1 --2- 172.21.15.67:0/3402684796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6c8660 0x55c89da975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.364+0000 7f81f4ee2640 1 --2- 172.21.15.67:0/3402684796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6f1520 0x55c89da97070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.365+0000 7f81f4ee2640 1 -- 172.21.15.67:0/3402684796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6c8660 msgr2=0x55c89da975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.365+0000 7f81f4ee2640 1 --2- 172.21.15.67:0/3402684796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6c8660 0x55c89da975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.365+0000 7f81f4ee2640 1 -- 172.21.15.67:0/3402684796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6f1520 msgr2=0x55c89da97070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.365+0000 7f81f4ee2640 1 --2- 172.21.15.67:0/3402684796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6f1520 0x55c89da97070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.365+0000 7f81f4ee2640 1 -- 172.21.15.67:0/3402684796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c89d9c9040 con 0x55c89d6d0840 2024-07-31T14:56:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.365+0000 7f81e27fc640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81dc0b3d40 0x7f81dc0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.365+0000 7f81f4ee2640 1 --2- 172.21.15.67:0/3402684796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6d0840 0x55c89da97af0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f81e4014a50 tx=0x7f81e400c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7601355ebaaacbc0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.365+0000 7f81f56e3640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81dc0b3d40 0x7f81dc0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.365+0000 7f81e27fc640 1 -- 172.21.15.67:0/3076467241 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f81e80e4460 con 0x7f81f0108740 2024-07-31T14:56:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.365+0000 7f81db7fe640 1 -- 172.21.15.67:0/3402684796 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e40142d0 con 0x55c89d6d0840 2024-07-31T14:56:05.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.365+0000 7f81db7fe640 1 -- 172.21.15.67:0/3402684796 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e400f730 con 0x55c89d6d0840 2024-07-31T14:56:05.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.365+0000 7f81db7fe640 1 -- 172.21.15.67:0/3402684796 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e4003850 con 0x55c89d6d0840 2024-07-31T14:56:05.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.366+0000 7f81fde98740 1 -- 172.21.15.67:0/3402684796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6d0840 msgr2=0x55c89da97af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:05.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.366+0000 7f81fde98740 1 --2- 172.21.15.67:0/3402684796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6d0840 0x55c89da97af0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f81e4014a50 tx=0x7f81e400c540 comp rx=0 tx=0).stop 2024-07-31T14:56:05.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.366+0000 7f81f56e3640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81dc0b3d40 0x7f81dc0b6250 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f81ec031440 tx=0x7f81ec009640 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:05.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.366+0000 7f81fde98740 1 -- 172.21.15.67:0/3402684796 shutdown_connections 2024-07-31T14:56:05.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.366+0000 7f81fde98740 1 --2- 172.21.15.67:0/3402684796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6d0840 0x55c89da97af0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.366+0000 7f81fde98740 1 --2- 172.21.15.67:0/3402684796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6c8660 0x55c89da975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.366+0000 7f81fde98740 1 --2- 172.21.15.67:0/3402684796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6f1520 0x55c89da97070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.366+0000 7f81fde98740 1 -- 172.21.15.67:0/3402684796 >> 172.21.15.67:0/3402684796 conn(0x55c89d6e5910 msgr2=0x55c89da94f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:05.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.366+0000 7f81fde98740 1 -- 172.21.15.67:0/3402684796 shutdown_connections 2024-07-31T14:56:05.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.366+0000 7f81fde98740 1 -- 172.21.15.67:0/3402684796 wait complete. 2024-07-31T14:56:05.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.367+0000 7f81fde98740 1 -- start start 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.367+0000 7f81fde98740 1 Processor -- start 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.367+0000 7f81fde98740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6c8660 0x55c89daa6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.367+0000 7f81fde98740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6d0840 0x55c89daa7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.367+0000 7f81fde98740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6f1520 0x55c89dab2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.367+0000 7f81fde98740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c89d85fbb0 con 0x55c89d6d0840 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.367+0000 7f81fde98740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c89d99eac0 con 0x55c89d6c8660 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.367+0000 7f81fde98740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c89daa9b20 con 0x55c89d6f1520 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.367+0000 7f81fde98740 10 client.0 _ll_register_callbacks cb 0x55c89d9a73b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.368+0000 7f81f4ee2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6d0840 0x55c89daa7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.368+0000 7f81f4ee2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6d0840 0x55c89daa7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37170/0 (socket says 172.21.15.67:37170) 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.368+0000 7f81f4ee2640 1 -- 172.21.15.67:0/640084064 learned_addr learned my addr 172.21.15.67:0/640084064 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.368+0000 7f81f56e3640 1 --2- 172.21.15.67:0/640084064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6c8660 0x55c89daa6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.368+0000 7f81f5ee4640 1 --2- 172.21.15.67:0/640084064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6f1520 0x55c89dab2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.368+0000 7f81f56e3640 1 -- 172.21.15.67:0/640084064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6f1520 msgr2=0x55c89dab2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.368+0000 7f81f56e3640 1 --2- 172.21.15.67:0/640084064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6f1520 0x55c89dab2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.368+0000 7f81f56e3640 1 -- 172.21.15.67:0/640084064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6d0840 msgr2=0x55c89daa7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.368+0000 7f81f56e3640 1 --2- 172.21.15.67:0/640084064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6d0840 0x55c89daa7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.368+0000 7f81f56e3640 1 -- 172.21.15.67:0/640084064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c89d75cb00 con 0x55c89d6c8660 2024-07-31T14:56:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.368+0000 7f81f4ee2640 1 --2- 172.21.15.67:0/640084064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6d0840 0x55c89daa7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.368+0000 7f81f56e3640 1 --2- 172.21.15.67:0/640084064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6c8660 0x55c89daa6fd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009230 tx=0x7f81ec009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=857452d81a06cedf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.368+0000 7f81d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:56:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.369+0000 7f81d8ff9640 1 -- 172.21.15.67:0/640084064 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec04c030 con 0x55c89d6c8660 2024-07-31T14:56:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.369+0000 7f81fde98740 1 -- 172.21.15.67:0/640084064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81e400f730 con 0x55c89d6c8660 2024-07-31T14:56:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.369+0000 7f81d8ff9640 1 -- 172.21.15.67:0/640084064 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f81ec003800 con 0x55c89d6c8660 2024-07-31T14:56:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.369+0000 7f81d8ff9640 1 -- 172.21.15.67:0/640084064 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f81ec059050 con 0x55c89d6c8660 2024-07-31T14:56:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.369+0000 7f81fde98740 10 client.26525 fetch_fsmap learned FSMap version 42 2024-07-31T14:56:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.369+0000 7f81fde98740 1 -- 172.21.15.67:0/640084064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c89d7d3960 con 0x55c89d6c8660 2024-07-31T14:56:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.369+0000 7f81d8ff9640 1 -- 172.21.15.67:0/640084064 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e400f730 con 0x55c89d6c8660 2024-07-31T14:56:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.369+0000 7f81d8ff9640 1 -- 172.21.15.67:0/640084064 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f81ec0489e0 con 0x55c89d6c8660 2024-07-31T14:56:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.369+0000 7f81fde98740 10 client.26525 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:56:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.369+0000 7f81fde98740 10 client.26525 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:56:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.369+0000 7f81fde98740 20 client.26525 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.369+0000 7f81fde98740 1 --2- 172.21.15.67:0/640084064 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c89d6afea0 0x55c89dab8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.370+0000 7f81fde98740 4 client.26525 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:56:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.370+0000 7f81fde98740 1 -- 172.21.15.67:0/640084064 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c89da0edc0 con 0x55c89d6afea0 2024-07-31T14:56:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.370+0000 7f81f4ee2640 1 --2- 172.21.15.67:0/640084064 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c89d6afea0 0x55c89dab8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.370+0000 7f81f4ee2640 1 --2- 172.21.15.67:0/640084064 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c89d6afea0 0x55c89dab8f80 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc865094ba1258a4 server_cookie=7e8089c557a9ad98 in_seq=0 out_seq=0 2024-07-31T14:56:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.370+0000 7f81d8ff9640 10 client.26525 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.371+0000 7f81d8ff9640 1 -- 172.21.15.67:0/640084064 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c89da0edc0 con 0x55c89d6afea0 2024-07-31T14:56:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.371+0000 7f81d8ff9640 10 client.26525 handle_command_reply: tid=0 2024-07-31T14:56:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.371+0000 7f81fde98740 1 client.26525 shutdown 2024-07-31T14:56:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.371+0000 7f81fde98740 1 -- 172.21.15.67:0/640084064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6c8660 msgr2=0x55c89daa6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.371+0000 7f81fde98740 1 --2- 172.21.15.67:0/640084064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6c8660 0x55c89daa6fd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009230 tx=0x7f81ec009260 comp rx=0 tx=0).stop 2024-07-31T14:56:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.371+0000 7f81fde98740 1 -- 172.21.15.67:0/640084064 shutdown_connections 2024-07-31T14:56:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.371+0000 7f81fde98740 1 --2- 172.21.15.67:0/640084064 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c89d6afea0 0x55c89dab8f80 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.371+0000 7f81fde98740 1 --2- 172.21.15.67:0/640084064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6f1520 0x55c89dab2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.371+0000 7f81fde98740 1 --2- 172.21.15.67:0/640084064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6d0840 0x55c89daa7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.371+0000 7f81fde98740 1 --2- 172.21.15.67:0/640084064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6c8660 0x55c89daa6fd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.371+0000 7f81d8ff9640 0 client.26525 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.371+0000 7f81fde98740 1 -- 172.21.15.67:0/640084064 >> 172.21.15.67:0/640084064 conn(0x55c89d6e5910 msgr2=0x55c89da9a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.371+0000 7f81fde98740 1 -- 172.21.15.67:0/640084064 shutdown_connections 2024-07-31T14:56:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.372+0000 7f81fde98740 1 -- 172.21.15.67:0/640084064 wait complete. 2024-07-31T14:56:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.372+0000 7f81fde98740 20 client.26525 trim_cache size 0 max 16384 2024-07-31T14:56:05.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.390+0000 7f81fde98740 1 Processor -- start 2024-07-31T14:56:05.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.390+0000 7f81fde98740 1 -- start start 2024-07-31T14:56:05.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.390+0000 7f81fde98740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6c8660 0x55c89da94e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.390+0000 7f81fde98740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6d0840 0x55c89da9ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.390+0000 7f81fde98740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6f1520 0x55c89da987b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.390+0000 7f81fde98740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01ae480 con 0x55c89d6f1520 2024-07-31T14:56:05.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.390+0000 7f81fde98740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01b0140 con 0x55c89d6d0840 2024-07-31T14:56:05.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.390+0000 7f81fde98740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01aeb50 con 0x55c89d6c8660 2024-07-31T14:56:05.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.391+0000 7f81f5ee4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6f1520 0x55c89da987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.391+0000 7f81f5ee4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6f1520 0x55c89da987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37176/0 (socket says 172.21.15.67:37176) 2024-07-31T14:56:05.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.391+0000 7f81f5ee4640 1 -- 172.21.15.67:0/3982665294 learned_addr learned my addr 172.21.15.67:0/3982665294 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:05.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.391+0000 7f81f4ee2640 1 --2- 172.21.15.67:0/3982665294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6d0840 0x55c89da9ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.391+0000 7f81f56e3640 1 --2- 172.21.15.67:0/3982665294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6c8660 0x55c89da94e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.391+0000 7f81f5ee4640 1 -- 172.21.15.67:0/3982665294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6c8660 msgr2=0x55c89da94e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:05.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.391+0000 7f81f5ee4640 1 --2- 172.21.15.67:0/3982665294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6c8660 0x55c89da94e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.391+0000 7f81f5ee4640 1 -- 172.21.15.67:0/3982665294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6d0840 msgr2=0x55c89da9ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:05.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.391+0000 7f81f5ee4640 1 --2- 172.21.15.67:0/3982665294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6d0840 0x55c89da9ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.391+0000 7f81f5ee4640 1 -- 172.21.15.67:0/3982665294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c89dab5a10 con 0x55c89d6f1520 2024-07-31T14:56:05.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.391+0000 7f81f5ee4640 1 --2- 172.21.15.67:0/3982665294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6f1520 0x55c89da987b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f81e800a6e0 tx=0x7f81e80aad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0676cfd4aaeb073 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:05.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.391+0000 7f81daffd640 1 -- 172.21.15.67:0/3982665294 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e80c5070 con 0x55c89d6f1520 2024-07-31T14:56:05.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.391+0000 7f81daffd640 1 -- 172.21.15.67:0/3982665294 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e80b3e10 con 0x55c89d6f1520 2024-07-31T14:56:05.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.391+0000 7f81daffd640 1 -- 172.21.15.67:0/3982665294 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e801d040 con 0x55c89d6f1520 2024-07-31T14:56:05.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:05 smithi073 ceph-mon[36714]: pgmap v550: 129 pgs: 129 active+clean; 989 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 1.93k op/s 2024-07-31T14:56:05.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.392+0000 7f81fde98740 1 -- 172.21.15.67:0/3982665294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6f1520 msgr2=0x55c89da987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:05.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.392+0000 7f81fde98740 1 --2- 172.21.15.67:0/3982665294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6f1520 0x55c89da987b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f81e800a6e0 tx=0x7f81e80aad60 comp rx=0 tx=0).stop 2024-07-31T14:56:05.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.392+0000 7f81fde98740 1 -- 172.21.15.67:0/3982665294 shutdown_connections 2024-07-31T14:56:05.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.392+0000 7f81fde98740 1 --2- 172.21.15.67:0/3982665294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6f1520 0x55c89da987b0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.392+0000 7f81fde98740 1 --2- 172.21.15.67:0/3982665294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6d0840 0x55c89da9ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.392+0000 7f81fde98740 1 --2- 172.21.15.67:0/3982665294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6c8660 0x55c89da94e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.392+0000 7f81fde98740 1 -- 172.21.15.67:0/3982665294 >> 172.21.15.67:0/3982665294 conn(0x55c89d6e5910 msgr2=0x55c89da980a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:05.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.392+0000 7f81fde98740 1 -- 172.21.15.67:0/3982665294 shutdown_connections 2024-07-31T14:56:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.392+0000 7f81fde98740 1 -- 172.21.15.67:0/3982665294 wait complete. 2024-07-31T14:56:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.393+0000 7f81fde98740 1 -- start start 2024-07-31T14:56:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.393+0000 7f81fde98740 1 Processor -- start 2024-07-31T14:56:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.393+0000 7f81fde98740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6c8660 0x55c89daaaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.393+0000 7f81fde98740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6d0840 0x55c89daab410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.393+0000 7f81fde98740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6f1520 0x55c89daab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.393+0000 7f81fde98740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01afa70 con 0x55c89d6c8660 2024-07-31T14:56:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.393+0000 7f81fde98740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01af2e0 con 0x55c89d6d0840 2024-07-31T14:56:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.393+0000 7f81fde98740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01ac460 con 0x55c89d6f1520 2024-07-31T14:56:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.393+0000 7f81fde98740 10 client.0 _ll_register_callbacks cb 0x7f81ec005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.393+0000 7f81f56e3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6c8660 0x55c89daaaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.393+0000 7f81f56e3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6c8660 0x55c89daaaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37178/0 (socket says 172.21.15.67:37178) 2024-07-31T14:56:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.393+0000 7f81f56e3640 1 -- 172.21.15.67:0/1386040597 learned_addr learned my addr 172.21.15.67:0/1386040597 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.393+0000 7f81f4ee2640 1 --2- 172.21.15.67:0/1386040597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6d0840 0x55c89daab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.394+0000 7f81f5ee4640 1 --2- 172.21.15.67:0/1386040597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6f1520 0x55c89daab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.394+0000 7f81f56e3640 1 -- 172.21.15.67:0/1386040597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6f1520 msgr2=0x55c89daab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.394+0000 7f81f56e3640 1 --2- 172.21.15.67:0/1386040597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6f1520 0x55c89daab970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.394+0000 7f81f56e3640 1 -- 172.21.15.67:0/1386040597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6d0840 msgr2=0x55c89daab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.394+0000 7f81f56e3640 1 --2- 172.21.15.67:0/1386040597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6d0840 0x55c89daab410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.394+0000 7f81f56e3640 1 -- 172.21.15.67:0/1386040597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c89db20980 con 0x55c89d6c8660 2024-07-31T14:56:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.394+0000 7f81f56e3640 1 --2- 172.21.15.67:0/1386040597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6c8660 0x55c89daaaed0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f81ec006010 tx=0x7f81ec004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cd3e25cc23da7f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.394+0000 7f81d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:56:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.394+0000 7f81d8ff9640 1 -- 172.21.15.67:0/1386040597 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec062060 con 0x55c89d6c8660 2024-07-31T14:56:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.394+0000 7f81d8ff9640 1 -- 172.21.15.67:0/1386040597 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f81ec003800 con 0x55c89d6c8660 2024-07-31T14:56:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.394+0000 7f81fde98740 1 -- 172.21.15.67:0/1386040597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81e80b3e10 con 0x55c89d6c8660 2024-07-31T14:56:05.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.394+0000 7f81d8ff9640 1 -- 172.21.15.67:0/1386040597 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f81ec069050 con 0x55c89d6c8660 2024-07-31T14:56:05.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.394+0000 7f81fde98740 10 client.16941 fetch_fsmap learned FSMap version 42 2024-07-31T14:56:05.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.394+0000 7f81fde98740 1 -- 172.21.15.67:0/1386040597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c89db21c40 con 0x55c89d6c8660 2024-07-31T14:56:05.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.395+0000 7f81d8ff9640 1 -- 172.21.15.67:0/1386040597 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e80b3e10 con 0x55c89d6c8660 2024-07-31T14:56:05.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.395+0000 7f81d8ff9640 1 -- 172.21.15.67:0/1386040597 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f81ec0489e0 con 0x55c89d6c8660 2024-07-31T14:56:05.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.395+0000 7f81fde98740 10 client.16941 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:56:05.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.395+0000 7f81fde98740 10 client.16941 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:56:05.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.395+0000 7f81fde98740 20 client.16941 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:05.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.395+0000 7f81fde98740 1 --2- 172.21.15.67:0/1386040597 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c89d685250 0x55c89db24490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:05.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.395+0000 7f81fde98740 4 client.16941 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c736f69", "release": true, "await": true}] 2024-07-31T14:56:05.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.395+0000 7f81fde98740 1 -- 172.21.15.67:0/1386040597 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c736f69", "release": true, "await": true}) -- 0x55c89da0a860 con 0x55c89d685250 2024-07-31T14:56:05.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.395+0000 7f81f4ee2640 1 --2- 172.21.15.67:0/1386040597 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c89d685250 0x55c89db24490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:05.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.395+0000 7f81d8ff9640 10 client.16941 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:05.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:05.395+0000 7f81f4ee2640 1 --2- 172.21.15.67:0/1386040597 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c89d685250 0x55c89db24490 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f1edf93ab57bb3 server_cookie=2fbcb5655d60141b in_seq=0 out_seq=0 2024-07-31T14:56:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:05 smithi077 ceph-mon[37327]: pgmap v550: 129 pgs: 129 active+clean; 989 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 1.93k op/s 2024-07-31T14:56:06.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.173+0000 7f81d8ff9640 1 -- 172.21.15.67:0/1386040597 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c89da0a860 con 0x55c89d685250 2024-07-31T14:56:06.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.173+0000 7f81d8ff9640 10 client.16941 handle_command_reply: tid=0 2024-07-31T14:56:06.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.174+0000 7f81fde98740 1 client.16941 shutdown 2024-07-31T14:56:06.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.174+0000 7f81fde98740 1 -- 172.21.15.67:0/1386040597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6c8660 msgr2=0x55c89daaaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:06.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.174+0000 7f81fde98740 1 --2- 172.21.15.67:0/1386040597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6c8660 0x55c89daaaed0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f81ec006010 tx=0x7f81ec004530 comp rx=0 tx=0).stop 2024-07-31T14:56:06.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.174+0000 7f81fde98740 1 -- 172.21.15.67:0/1386040597 shutdown_connections 2024-07-31T14:56:06.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.174+0000 7f81fde98740 1 --2- 172.21.15.67:0/1386040597 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55c89d685250 0x55c89db24490 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.174+0000 7f81fde98740 1 --2- 172.21.15.67:0/1386040597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c89d6f1520 0x55c89daab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.174+0000 7f81d8ff9640 0 client.16941 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:06.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.174+0000 7f81fde98740 1 --2- 172.21.15.67:0/1386040597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c89d6d0840 0x55c89daab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.174+0000 7f81fde98740 1 --2- 172.21.15.67:0/1386040597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c89d6c8660 0x55c89daaaed0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.174+0000 7f81fde98740 1 -- 172.21.15.67:0/1386040597 >> 172.21.15.67:0/1386040597 conn(0x55c89d6e5910 msgr2=0x55c89da9a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:06.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.174+0000 7f81fde98740 1 -- 172.21.15.67:0/1386040597 shutdown_connections 2024-07-31T14:56:06.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.174+0000 7f81fde98740 1 -- 172.21.15.67:0/1386040597 wait complete. 2024-07-31T14:56:06.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.174+0000 7f81fde98740 20 client.16941 trim_cache size 0 max 16384 2024-07-31T14:56:06.177 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:56:06.177 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 42, 2024-07-31T14:56:06.177 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T14:56:06.178 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 12, 2024-07-31T14:56:06.178 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T14:56:06.178 INFO:teuthology.orchestra.run.smithi067.stdout: "c736f69": { 2024-07-31T14:56:06.178 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 12, 2024-07-31T14:56:06.178 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T14:56:06.178 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:56:06.178 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T14:56:06.178 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:56:06.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:56:06.178 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T14:56:06.178 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 138.4, 2024-07-31T14:56:06.178 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T14:56:06.178 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T14:56:06.178 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T14:56:06.179 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:56:06.179 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T14:56:06.179 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:56:06.179 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:56:06.179 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:56:06.179 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:56:06.179 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:56:06.179 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:56:06.179 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:56:06.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.175+0000 7f81f7947640 1 -- 172.21.15.67:0/3076467241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81dc0b3d40 msgr2=0x7f81dc0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:06.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.175+0000 7f81f7947640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81dc0b3d40 0x7f81dc0b6250 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f81ec031440 tx=0x7f81ec009640 comp rx=0 tx=0).stop 2024-07-31T14:56:06.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.175+0000 7f81f7947640 1 -- 172.21.15.67:0/3076467241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f0108740 msgr2=0x7f81f019c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:06.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.175+0000 7f81f7947640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f0108740 0x7f81f019c490 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f81e800cad0 tx=0x7f81e800cf90 comp rx=0 tx=0).stop 2024-07-31T14:56:06.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.175+0000 7f81f7947640 1 -- 172.21.15.67:0/3076467241 shutdown_connections 2024-07-31T14:56:06.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.175+0000 7f81f7947640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81dc0b3d40 0x7f81dc0b6250 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.175+0000 7f81f7947640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f0108740 0x7f81f019c490 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.175+0000 7f81f7947640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0107df0 0x7f81f006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.175+0000 7f81f7947640 1 --2- 172.21.15.67:0/3076467241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f01024b0 0x7f81f006b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.175+0000 7f81f7947640 1 -- 172.21.15.67:0/3076467241 >> 172.21.15.67:0/3076467241 conn(0x7f81f0079170 msgr2=0x7f81f0100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:06.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.175+0000 7f81f7947640 1 -- 172.21.15.67:0/3076467241 shutdown_connections 2024-07-31T14:56:06.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:06.175+0000 7f81f7947640 1 -- 172.21.15.67:0/3076467241 wait complete. 2024-07-31T14:56:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.865+0000 7f12a3b87640 1 -- 172.21.15.67:0/1832245432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f129c1023f0 msgr2=0x7f129c102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.865+0000 7f12a3b87640 1 --2- 172.21.15.67:0/1832245432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f129c1023f0 0x7f129c102800 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f128c009ca0 tx=0x7f128c02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.865+0000 7f12a3b87640 1 -- 172.21.15.67:0/1832245432 shutdown_connections 2024-07-31T14:56:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.865+0000 7f12a3b87640 1 --2- 172.21.15.67:0/1832245432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f129c108670 0x7f129c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.865+0000 7f12a3b87640 1 --2- 172.21.15.67:0/1832245432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f129c107d20 0x7f129c108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.866+0000 7f12a3b87640 1 --2- 172.21.15.67:0/1832245432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f129c1023f0 0x7f129c102800 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.866+0000 7f12a3b87640 1 -- 172.21.15.67:0/1832245432 >> 172.21.15.67:0/1832245432 conn(0x7f129c0fddf0 msgr2=0x7f129c100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.866+0000 7f12a3b87640 1 -- 172.21.15.67:0/1832245432 shutdown_connections 2024-07-31T14:56:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.866+0000 7f12a3b87640 1 -- 172.21.15.67:0/1832245432 wait complete. 2024-07-31T14:56:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.866+0000 7f12a3b87640 1 Processor -- start 2024-07-31T14:56:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.866+0000 7f12a3b87640 1 -- start start 2024-07-31T14:56:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a3b87640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f129c1023f0 0x7f129c199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a3b87640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f129c107d20 0x7f129c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a3b87640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f129c108670 0x7f129c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a3b87640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f129c112340 con 0x7f129c1023f0 2024-07-31T14:56:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a3b87640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1121c0 con 0x7f129c107d20 2024-07-31T14:56:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a3b87640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1124c0 con 0x7f129c108670 2024-07-31T14:56:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a1923640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f129c1023f0 0x7f129c199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a1923640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f129c1023f0 0x7f129c199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37202/0 (socket says 172.21.15.67:37202) 2024-07-31T14:56:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a1923640 1 -- 172.21.15.67:0/876738841 learned_addr learned my addr 172.21.15.67:0/876738841 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a1122640 1 --2- 172.21.15.67:0/876738841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f129c107d20 0x7f129c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a2124640 1 --2- 172.21.15.67:0/876738841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f129c108670 0x7f129c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a1923640 1 -- 172.21.15.67:0/876738841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f129c108670 msgr2=0x7f129c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a1923640 1 --2- 172.21.15.67:0/876738841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f129c108670 0x7f129c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a1923640 1 -- 172.21.15.67:0/876738841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f129c107d20 msgr2=0x7f129c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a1923640 1 --2- 172.21.15.67:0/876738841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f129c107d20 0x7f129c19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a1923640 1 -- 172.21.15.67:0/876738841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c19fd80 con 0x7f129c1023f0 2024-07-31T14:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.867+0000 7f12a1923640 1 --2- 172.21.15.67:0/876738841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f129c1023f0 0x7f129c199e50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f128c033040 tx=0x7f128c0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.868+0000 7f129affd640 1 -- 172.21.15.67:0/876738841 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c003b70 con 0x7f129c1023f0 2024-07-31T14:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.868+0000 7f129affd640 1 -- 172.21.15.67:0/876738841 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f128c003d10 con 0x7f129c1023f0 2024-07-31T14:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.868+0000 7f12a3b87640 1 -- 172.21.15.67:0/876738841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129c1a0010 con 0x7f129c1023f0 2024-07-31T14:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.868+0000 7f129affd640 1 -- 172.21.15.67:0/876738841 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c02f9e0 con 0x7f129c1023f0 2024-07-31T14:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.868+0000 7f12a3b87640 1 -- 172.21.15.67:0/876738841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129c1a6800 con 0x7f129c1023f0 2024-07-31T14:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.871+0000 7f12aaa83740 1 Processor -- start 2024-07-31T14:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.871+0000 7f12aaa83740 1 -- start start 2024-07-31T14:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12aaa83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2133f1b0 0x55cf216e1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12aaa83740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf213162f0 0x55cf216e2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12aaa83740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2131e4d0 0x55cf216e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12aaa83740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cf216573b0 con 0x55cf213162f0 2024-07-31T14:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12aaa83740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cf216468b0 con 0x55cf2131e4d0 2024-07-31T14:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12aaa83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cf2164a8f0 con 0x55cf2133f1b0 2024-07-31T14:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f129affd640 1 -- 172.21.15.67:0/876738841 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f128c02fb80 con 0x7f129c1023f0 2024-07-31T14:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12a1122640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2131e4d0 0x55cf216e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12a1122640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2131e4d0 0x55cf216e2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46436/0 (socket says 172.21.15.67:46436) 2024-07-31T14:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12a1122640 1 -- 172.21.15.67:0/4100149229 learned_addr learned my addr 172.21.15.67:0/4100149229 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12a2124640 1 --2- 172.21.15.67:0/4100149229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2133f1b0 0x55cf216e1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12a1923640 1 --2- 172.21.15.67:0/4100149229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf213162f0 0x55cf216e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12a1122640 1 -- 172.21.15.67:0/4100149229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2133f1b0 msgr2=0x55cf216e1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12a1122640 1 --2- 172.21.15.67:0/4100149229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2133f1b0 0x55cf216e1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12a1122640 1 -- 172.21.15.67:0/4100149229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf213162f0 msgr2=0x55cf216e2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12a1122640 1 --2- 172.21.15.67:0/4100149229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf213162f0 0x55cf216e2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12a1122640 1 -- 172.21.15.67:0/4100149229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf215ed9a0 con 0x55cf2131e4d0 2024-07-31T14:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12a1923640 1 --2- 172.21.15.67:0/4100149229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf213162f0 0x55cf216e2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:56:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.872+0000 7f12a1122640 1 --2- 172.21.15.67:0/4100149229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2131e4d0 0x55cf216e2760 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1290009c50 tx=0x7f1290011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8afe547052bc127d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.873+0000 7f1283fff640 1 -- 172.21.15.67:0/4100149229 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290004230 con 0x55cf2131e4d0 2024-07-31T14:56:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.873+0000 7f129affd640 1 --2- 172.21.15.67:0/876738841 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f12840b3dc0 0x7f12840b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.873+0000 7f1283fff640 1 -- 172.21.15.67:0/4100149229 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12900043d0 con 0x55cf2131e4d0 2024-07-31T14:56:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.873+0000 7f1283fff640 1 -- 172.21.15.67:0/4100149229 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290004d50 con 0x55cf2131e4d0 2024-07-31T14:56:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.873+0000 7f129affd640 1 -- 172.21.15.67:0/876738841 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f128c1057b0 con 0x7f129c1023f0 2024-07-31T14:56:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.873+0000 7f12a2124640 1 --2- 172.21.15.67:0/876738841 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f12840b3dc0 0x7f12840b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.874+0000 7f12aaa83740 1 -- 172.21.15.67:0/4100149229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2131e4d0 msgr2=0x55cf216e2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.874+0000 7f12a2124640 1 --2- 172.21.15.67:0/876738841 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f12840b3dc0 0x7f12840b62d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1294009da0 tx=0x7f1294005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.874+0000 7f12aaa83740 1 --2- 172.21.15.67:0/4100149229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2131e4d0 0x55cf216e2760 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1290009c50 tx=0x7f1290011810 comp rx=0 tx=0).stop 2024-07-31T14:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.874+0000 7f12aaa83740 1 -- 172.21.15.67:0/4100149229 shutdown_connections 2024-07-31T14:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.874+0000 7f12aaa83740 1 --2- 172.21.15.67:0/4100149229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2131e4d0 0x55cf216e2760 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.874+0000 7f12aaa83740 1 --2- 172.21.15.67:0/4100149229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf213162f0 0x55cf216e2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.874+0000 7f12aaa83740 1 --2- 172.21.15.67:0/4100149229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2133f1b0 0x55cf216e1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.874+0000 7f12aaa83740 1 -- 172.21.15.67:0/4100149229 >> 172.21.15.67:0/4100149229 conn(0x55cf21333300 msgr2=0x55cf216dfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.874+0000 7f12aaa83740 1 -- 172.21.15.67:0/4100149229 shutdown_connections 2024-07-31T14:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.874+0000 7f12aaa83740 1 -- 172.21.15.67:0/4100149229 wait complete. 2024-07-31T14:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.874+0000 7f12aaa83740 1 -- start start 2024-07-31T14:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.874+0000 7f12aaa83740 1 Processor -- start 2024-07-31T14:56:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12aaa83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf213162f0 0x55cf216ed7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12aaa83740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2131e4d0 0x55cf216edce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12aaa83740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf2133f1b0 0x55cf216f97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12aaa83740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cf21580ab0 con 0x55cf2133f1b0 2024-07-31T14:56:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12aaa83740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cf215cecf0 con 0x55cf2131e4d0 2024-07-31T14:56:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12aaa83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cf216f0460 con 0x55cf213162f0 2024-07-31T14:56:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12aaa83740 10 client.0 _ll_register_callbacks cb 0x55cf216409f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12a2124640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf2133f1b0 0x55cf216f97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12a1122640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2131e4d0 0x55cf216edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12a1122640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2131e4d0 0x55cf216edce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46446/0 (socket says 172.21.15.67:46446) 2024-07-31T14:56:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12a1122640 1 -- 172.21.15.67:0/1964096131 learned_addr learned my addr 172.21.15.67:0/1964096131 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12a1923640 1 --2- 172.21.15.67:0/1964096131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf213162f0 0x55cf216ed7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12a1122640 1 -- 172.21.15.67:0/1964096131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf213162f0 msgr2=0x55cf216ed7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12a1122640 1 --2- 172.21.15.67:0/1964096131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf213162f0 0x55cf216ed7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12a1122640 1 -- 172.21.15.67:0/1964096131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf2133f1b0 msgr2=0x55cf216f97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12a1122640 1 --2- 172.21.15.67:0/1964096131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf2133f1b0 0x55cf216f97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.875+0000 7f12a1122640 1 -- 172.21.15.67:0/1964096131 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf213aa790 con 0x55cf2131e4d0 2024-07-31T14:56:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.876+0000 7f12a1122640 1 --2- 172.21.15.67:0/1964096131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2131e4d0 0x55cf216edce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f12900049b0 tx=0x7f129001ab30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25c06932df73ea1e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.876+0000 7f12817fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:56:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.876+0000 7f12817fa640 1 -- 172.21.15.67:0/1964096131 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290028020 con 0x55cf2131e4d0 2024-07-31T14:56:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.876+0000 7f12aaa83740 1 -- 172.21.15.67:0/1964096131 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12900043d0 con 0x55cf2131e4d0 2024-07-31T14:56:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.876+0000 7f12817fa640 1 -- 172.21.15.67:0/1964096131 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f1290024620 con 0x55cf2131e4d0 2024-07-31T14:56:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.876+0000 7f12817fa640 1 -- 172.21.15.67:0/1964096131 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1290033050 con 0x55cf2131e4d0 2024-07-31T14:56:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.876+0000 7f12aaa83740 10 client.26543 fetch_fsmap learned FSMap version 42 2024-07-31T14:56:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.876+0000 7f12aaa83740 1 -- 172.21.15.67:0/1964096131 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cf21423620 con 0x55cf2131e4d0 2024-07-31T14:56:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.876+0000 7f12817fa640 1 -- 172.21.15.67:0/1964096131 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12900043d0 con 0x55cf2131e4d0 2024-07-31T14:56:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.876+0000 7f12817fa640 1 -- 172.21.15.67:0/1964096131 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f1290018070 con 0x55cf2131e4d0 2024-07-31T14:56:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.876+0000 7f12aaa83740 10 client.26543 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:56:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.876+0000 7f12aaa83740 10 client.26543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:56:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.876+0000 7f12aaa83740 20 client.26543 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.877+0000 7f12aaa83740 1 --2- 172.21.15.67:0/1964096131 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cf212fdd80 0x55cf216ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.877+0000 7f12aaa83740 4 client.26543 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:56:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.877+0000 7f12aaa83740 1 -- 172.21.15.67:0/1964096131 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf21659300 con 0x55cf212fdd80 2024-07-31T14:56:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.877+0000 7f12a1923640 1 --2- 172.21.15.67:0/1964096131 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cf212fdd80 0x55cf216ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.877+0000 7f12817fa640 10 client.26543 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.877+0000 7f12a1923640 1 --2- 172.21.15.67:0/1964096131 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cf212fdd80 0x55cf216ff5c0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01554c2b08dbfb6 server_cookie=12e2a3176ef8806 in_seq=0 out_seq=0 2024-07-31T14:56:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12817fa640 1 -- 172.21.15.67:0/1964096131 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf21659300 con 0x55cf212fdd80 2024-07-31T14:56:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12817fa640 10 client.26543 handle_command_reply: tid=0 2024-07-31T14:56:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12aaa83740 1 client.26543 shutdown 2024-07-31T14:56:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12aaa83740 1 -- 172.21.15.67:0/1964096131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2131e4d0 msgr2=0x55cf216edce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12aaa83740 1 --2- 172.21.15.67:0/1964096131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2131e4d0 0x55cf216edce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f12900049b0 tx=0x7f129001ab30 comp rx=0 tx=0).stop 2024-07-31T14:56:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12aaa83740 1 -- 172.21.15.67:0/1964096131 shutdown_connections 2024-07-31T14:56:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12aaa83740 1 --2- 172.21.15.67:0/1964096131 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cf212fdd80 0x55cf216ff5c0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12aaa83740 1 --2- 172.21.15.67:0/1964096131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf2133f1b0 0x55cf216f97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12817fa640 0 client.26543 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12aaa83740 1 --2- 172.21.15.67:0/1964096131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2131e4d0 0x55cf216edce0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12aaa83740 1 --2- 172.21.15.67:0/1964096131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf213162f0 0x55cf216ed7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12aaa83740 1 -- 172.21.15.67:0/1964096131 >> 172.21.15.67:0/1964096131 conn(0x55cf21333300 msgr2=0x55cf216de2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12aaa83740 1 -- 172.21.15.67:0/1964096131 shutdown_connections 2024-07-31T14:56:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12aaa83740 1 -- 172.21.15.67:0/1964096131 wait complete. 2024-07-31T14:56:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.879+0000 7f12aaa83740 20 client.26543 trim_cache size 0 max 16384 2024-07-31T14:56:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.894+0000 7f12aaa83740 1 Processor -- start 2024-07-31T14:56:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.894+0000 7f12aaa83740 1 -- start start 2024-07-31T14:56:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12aaa83740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf213162f0 0x55cf216dfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12aaa83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2131e4d0 0x55cf216e7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12aaa83740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf2133f1b0 0x55cf216de2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12aaa83740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1b0670 con 0x55cf2133f1b0 2024-07-31T14:56:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12aaa83740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1b2330 con 0x55cf213162f0 2024-07-31T14:56:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12aaa83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1b0d40 con 0x55cf2131e4d0 2024-07-31T14:56:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12a1923640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf213162f0 0x55cf216dfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12a2124640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf2133f1b0 0x55cf216de2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12a1923640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf213162f0 0x55cf216dfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46462/0 (socket says 172.21.15.67:46462) 2024-07-31T14:56:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12a1923640 1 -- 172.21.15.67:0/1282124917 learned_addr learned my addr 172.21.15.67:0/1282124917 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12a1122640 1 --2- 172.21.15.67:0/1282124917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2131e4d0 0x55cf216e7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12a1923640 1 -- 172.21.15.67:0/1282124917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2131e4d0 msgr2=0x55cf216e7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12a1923640 1 --2- 172.21.15.67:0/1282124917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2131e4d0 0x55cf216e7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12a1923640 1 -- 172.21.15.67:0/1282124917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf2133f1b0 msgr2=0x55cf216de2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12a1923640 1 --2- 172.21.15.67:0/1282124917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf2133f1b0 0x55cf216de2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12a1923640 1 -- 172.21.15.67:0/1282124917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf216e8de0 con 0x55cf213162f0 2024-07-31T14:56:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12a1122640 1 --2- 172.21.15.67:0/1282124917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2131e4d0 0x55cf216e7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:56:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12a2124640 1 --2- 172.21.15.67:0/1282124917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf2133f1b0 0x55cf216de2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.895+0000 7f12a1923640 1 --2- 172.21.15.67:0/1282124917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf213162f0 0x55cf216dfc20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f128c005e30 tx=0x7f128c107cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=889d36c9f07d8c11 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.896+0000 7f12837fe640 1 -- 172.21.15.67:0/1282124917 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c0cc8c0 con 0x55cf213162f0 2024-07-31T14:56:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.896+0000 7f12837fe640 1 -- 172.21.15.67:0/1282124917 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f128c10b030 con 0x55cf213162f0 2024-07-31T14:56:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.896+0000 7f12837fe640 1 -- 172.21.15.67:0/1282124917 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c105480 con 0x55cf213162f0 2024-07-31T14:56:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.896+0000 7f12aaa83740 1 -- 172.21.15.67:0/1282124917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf213162f0 msgr2=0x55cf216dfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.896+0000 7f12aaa83740 1 --2- 172.21.15.67:0/1282124917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf213162f0 0x55cf216dfc20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f128c005e30 tx=0x7f128c107cc0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.896+0000 7f12aaa83740 1 -- 172.21.15.67:0/1282124917 shutdown_connections 2024-07-31T14:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.896+0000 7f12aaa83740 1 --2- 172.21.15.67:0/1282124917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf2133f1b0 0x55cf216de2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.896+0000 7f12aaa83740 1 --2- 172.21.15.67:0/1282124917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2131e4d0 0x55cf216e7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.896+0000 7f12aaa83740 1 --2- 172.21.15.67:0/1282124917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf213162f0 0x55cf216dfc20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.896+0000 7f12aaa83740 1 -- 172.21.15.67:0/1282124917 >> 172.21.15.67:0/1282124917 conn(0x55cf21333300 msgr2=0x55cf216df510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.897+0000 7f12aaa83740 1 -- 172.21.15.67:0/1282124917 shutdown_connections 2024-07-31T14:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.897+0000 7f12aaa83740 1 -- 172.21.15.67:0/1282124917 wait complete. 2024-07-31T14:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.897+0000 7f12aaa83740 1 -- start start 2024-07-31T14:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.897+0000 7f12aaa83740 1 Processor -- start 2024-07-31T14:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12aaa83740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf213162f0 0x55cf216f5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12aaa83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2131e4d0 0x55cf216f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12aaa83740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2133f1b0 0x55cf216f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12aaa83740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1b1c60 con 0x55cf213162f0 2024-07-31T14:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12aaa83740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1b14d0 con 0x55cf2133f1b0 2024-07-31T14:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12aaa83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1ae650 con 0x55cf2131e4d0 2024-07-31T14:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12aaa83740 10 client.0 _ll_register_callbacks cb 0x7f1294009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12a1923640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf213162f0 0x55cf216f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12a1923640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf213162f0 0x55cf216f5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37232/0 (socket says 172.21.15.67:37232) 2024-07-31T14:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12a1923640 1 -- 172.21.15.67:0/2536599194 learned_addr learned my addr 172.21.15.67:0/2536599194 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12a1122640 1 --2- 172.21.15.67:0/2536599194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2131e4d0 0x55cf216f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12a1923640 1 -- 172.21.15.67:0/2536599194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2131e4d0 msgr2=0x55cf216f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12a1923640 1 --2- 172.21.15.67:0/2536599194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2131e4d0 0x55cf216f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12a1923640 1 -- 172.21.15.67:0/2536599194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2133f1b0 msgr2=0x55cf216f27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12a1923640 1 --2- 172.21.15.67:0/2536599194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2133f1b0 0x55cf216f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12a1923640 1 -- 172.21.15.67:0/2536599194 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf216f2f00 con 0x55cf213162f0 2024-07-31T14:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12a1923640 1 --2- 172.21.15.67:0/2536599194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf213162f0 0x55cf216f5920 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f128c1031f0 tx=0x7f128c1056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7b2e201838c779 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12817fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12817fa640 1 -- 172.21.15.67:0/2536599194 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c0e5070 con 0x55cf213162f0 2024-07-31T14:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12817fa640 1 -- 172.21.15.67:0/2536599194 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f128c0c87b0 con 0x55cf213162f0 2024-07-31T14:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.898+0000 7f12aaa83740 1 -- 172.21.15.67:0/2536599194 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f128c10b030 con 0x55cf213162f0 2024-07-31T14:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.899+0000 7f12817fa640 1 -- 172.21.15.67:0/2536599194 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f128c0cc1c0 con 0x55cf213162f0 2024-07-31T14:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.899+0000 7f12aaa83740 10 client.16959 fetch_fsmap learned FSMap version 42 2024-07-31T14:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.899+0000 7f12aaa83740 1 -- 172.21.15.67:0/2536599194 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cf217667a0 con 0x55cf213162f0 2024-07-31T14:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.899+0000 7f12817fa640 1 -- 172.21.15.67:0/2536599194 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f128c10b030 con 0x55cf213162f0 2024-07-31T14:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.899+0000 7f12817fa640 1 -- 172.21.15.67:0/2536599194 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7f128c0d4de0 con 0x55cf213162f0 2024-07-31T14:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.899+0000 7f12aaa83740 10 client.16959 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.899+0000 7f12aaa83740 10 client.16959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.899+0000 7f12aaa83740 20 client.16959 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.899+0000 7f12aaa83740 1 --2- 172.21.15.67:0/2536599194 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cf212d31c0 0x55cf21768fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.899+0000 7f12aaa83740 4 client.16959 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:56:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.899+0000 7f12aaa83740 1 -- 172.21.15.67:0/2536599194 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf21654da0 con 0x55cf212d31c0 2024-07-31T14:56:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.899+0000 7f12a1122640 1 --2- 172.21.15.67:0/2536599194 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cf212d31c0 0x55cf21768fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.900+0000 7f12817fa640 10 client.16959 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:06.900+0000 7f12a1122640 1 --2- 172.21.15.67:0/2536599194 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cf212d31c0 0x55cf21768fe0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b96127c54d725aef server_cookie=fec1d72b55274c87 in_seq=0 out_seq=0 2024-07-31T14:56:06.991 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c736f69', total seconds elapsed: 43.7 2024-07-31T14:56:07.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:07 smithi067 ceph-mon[32750]: pgmap v551: 129 pgs: 129 active+clean; 989 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 2.09k op/s 2024-07-31T14:56:07.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:07 smithi073 ceph-mon[36714]: pgmap v551: 129 pgs: 129 active+clean; 989 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 2.09k op/s 2024-07-31T14:56:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:07 smithi077 ceph-mon[37327]: pgmap v551: 129 pgs: 129 active+clean; 989 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 2.09k op/s 2024-07-31T14:56:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.375+0000 7f12817fa640 1 -- 172.21.15.67:0/2536599194 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf21654da0 con 0x55cf212d31c0 2024-07-31T14:56:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.375+0000 7f12817fa640 10 client.16959 handle_command_reply: tid=0 2024-07-31T14:56:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.375+0000 7f12aaa83740 1 client.16959 shutdown 2024-07-31T14:56:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.376+0000 7f12aaa83740 1 -- 172.21.15.67:0/2536599194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf213162f0 msgr2=0x55cf216f5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.376+0000 7f12aaa83740 1 --2- 172.21.15.67:0/2536599194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf213162f0 0x55cf216f5920 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f128c1031f0 tx=0x7f128c1056b0 comp rx=0 tx=0).stop 2024-07-31T14:56:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.376+0000 7f12aaa83740 1 -- 172.21.15.67:0/2536599194 shutdown_connections 2024-07-31T14:56:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.376+0000 7f12aaa83740 1 --2- 172.21.15.67:0/2536599194 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cf212d31c0 0x55cf21768fe0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.376+0000 7f12aaa83740 1 --2- 172.21.15.67:0/2536599194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf2133f1b0 0x55cf216f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.376+0000 7f12aaa83740 1 --2- 172.21.15.67:0/2536599194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf2131e4d0 0x55cf216f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.376+0000 7f12aaa83740 1 --2- 172.21.15.67:0/2536599194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cf213162f0 0x55cf216f5920 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.376+0000 7f12817fa640 0 client.16959 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.376+0000 7f12aaa83740 1 -- 172.21.15.67:0/2536599194 >> 172.21.15.67:0/2536599194 conn(0x55cf21333300 msgr2=0x55cf216eca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.376+0000 7f12aaa83740 1 -- 172.21.15.67:0/2536599194 shutdown_connections 2024-07-31T14:56:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.376+0000 7f12aaa83740 1 -- 172.21.15.67:0/2536599194 wait complete. 2024-07-31T14:56:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.376+0000 7f12aaa83740 20 client.16959 trim_cache size 0 max 16384 2024-07-31T14:56:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:56:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:56:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:56:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:56:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.377+0000 7f12a3b87640 1 -- 172.21.15.67:0/876738841 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f12840b3dc0 msgr2=0x7f12840b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.377+0000 7f12a3b87640 1 --2- 172.21.15.67:0/876738841 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f12840b3dc0 0x7f12840b62d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1294009da0 tx=0x7f1294005f90 comp rx=0 tx=0).stop 2024-07-31T14:56:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.377+0000 7f12a3b87640 1 -- 172.21.15.67:0/876738841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f129c1023f0 msgr2=0x7f129c199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.377+0000 7f12a3b87640 1 --2- 172.21.15.67:0/876738841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f129c1023f0 0x7f129c199e50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f128c033040 tx=0x7f128c0316f0 comp rx=0 tx=0).stop 2024-07-31T14:56:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.377+0000 7f12a3b87640 1 -- 172.21.15.67:0/876738841 shutdown_connections 2024-07-31T14:56:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.377+0000 7f12a3b87640 1 --2- 172.21.15.67:0/876738841 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f12840b3dc0 0x7f12840b62d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.377+0000 7f12a3b87640 1 --2- 172.21.15.67:0/876738841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f129c108670 0x7f129c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.377+0000 7f12a3b87640 1 --2- 172.21.15.67:0/876738841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f129c107d20 0x7f129c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.377+0000 7f12a3b87640 1 --2- 172.21.15.67:0/876738841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f129c1023f0 0x7f129c199e50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.377+0000 7f12a3b87640 1 -- 172.21.15.67:0/876738841 >> 172.21.15.67:0/876738841 conn(0x7f129c0fddf0 msgr2=0x7f129c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.377+0000 7f12a3b87640 1 -- 172.21.15.67:0/876738841 shutdown_connections 2024-07-31T14:56:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:08.377+0000 7f12a3b87640 1 -- 172.21.15.67:0/876738841 wait complete. 2024-07-31T14:56:09.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:09 smithi067 ceph-mon[32750]: pgmap v552: 129 pgs: 129 active+clean; 992 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 1.73k op/s 2024-07-31T14:56:09.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:09 smithi073 ceph-mon[36714]: pgmap v552: 129 pgs: 129 active+clean; 992 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 1.73k op/s 2024-07-31T14:56:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:09 smithi077 ceph-mon[37327]: pgmap v552: 129 pgs: 129 active+clean; 992 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 1.73k op/s 2024-07-31T14:56:11.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:11 smithi067 ceph-mon[32750]: pgmap v553: 129 pgs: 129 active+clean; 992 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 1.50k op/s 2024-07-31T14:56:11.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:11 smithi073 ceph-mon[36714]: pgmap v553: 129 pgs: 129 active+clean; 992 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 1.50k op/s 2024-07-31T14:56:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:11 smithi077 ceph-mon[37327]: pgmap v553: 129 pgs: 129 active+clean; 992 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 1.50k op/s 2024-07-31T14:56:12.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:12 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:56:12.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:12 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:56:12.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:12 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:56:12.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:12 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:56:12.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:12 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:56:12.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:12 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:56:12.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:12 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:56:12.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:12 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:56:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:12 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:56:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:12 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:56:12.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:12 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:56:12.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:12 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:56:12.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:12 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:56:12.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:12 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:56:12.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:12 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:56:13.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:13 smithi067 ceph-mon[32750]: pgmap v554: 129 pgs: 129 active+clean; 990 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 1.38k op/s 2024-07-31T14:56:13.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:13 smithi073 ceph-mon[36714]: pgmap v554: 129 pgs: 129 active+clean; 990 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 1.38k op/s 2024-07-31T14:56:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:13 smithi077 ceph-mon[37327]: pgmap v554: 129 pgs: 129 active+clean; 990 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 1.38k op/s 2024-07-31T14:56:15.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:15 smithi067 ceph-mon[32750]: pgmap v555: 129 pgs: 129 active+clean; 1000 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 1.22k op/s 2024-07-31T14:56:15.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:15 smithi073 ceph-mon[36714]: pgmap v555: 129 pgs: 129 active+clean; 1000 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 1.22k op/s 2024-07-31T14:56:15.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:15 smithi077 ceph-mon[37327]: pgmap v555: 129 pgs: 129 active+clean; 1000 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 1.22k op/s 2024-07-31T14:56:17.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:17 smithi067 ceph-mon[32750]: pgmap v556: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 893 op/s 2024-07-31T14:56:17.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:17 smithi073 ceph-mon[36714]: pgmap v556: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 893 op/s 2024-07-31T14:56:17.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:17 smithi077 ceph-mon[37327]: pgmap v556: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 893 op/s 2024-07-31T14:56:19.289 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:19 smithi077 ceph-mon[37327]: pgmap v557: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.9 MiB/s wr, 580 op/s 2024-07-31T14:56:19.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:19 smithi067 ceph-mon[32750]: pgmap v557: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.9 MiB/s wr, 580 op/s 2024-07-31T14:56:19.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:19 smithi073 ceph-mon[36714]: pgmap v557: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.9 MiB/s wr, 580 op/s 2024-07-31T14:56:21.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:21 smithi067 ceph-mon[32750]: pgmap v558: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 579 op/s 2024-07-31T14:56:21.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:21 smithi073 ceph-mon[36714]: pgmap v558: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 579 op/s 2024-07-31T14:56:21.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:21 smithi077 ceph-mon[37327]: pgmap v558: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 579 op/s 2024-07-31T14:56:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.488+0000 7fbede81c640 1 -- 172.21.15.67:0/2030932492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed8104cf0 msgr2=0x7fbed810d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.488+0000 7fbede81c640 1 --2- 172.21.15.67:0/2030932492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed8104cf0 0x7fbed810d230 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009900 tx=0x7fbecc02a410 comp rx=0 tx=0).stop 2024-07-31T14:56:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.488+0000 7fbede81c640 1 -- 172.21.15.67:0/2030932492 shutdown_connections 2024-07-31T14:56:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.488+0000 7fbede81c640 1 --2- 172.21.15.67:0/2030932492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbed810d7c0 0x7fbed810fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.488+0000 7fbede81c640 1 --2- 172.21.15.67:0/2030932492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed8104cf0 0x7fbed810d230 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.488+0000 7fbede81c640 1 --2- 172.21.15.67:0/2030932492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed81043a0 0x7fbed81047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.488+0000 7fbede81c640 1 -- 172.21.15.67:0/2030932492 >> 172.21.15.67:0/2030932492 conn(0x7fbed80fddf0 msgr2=0x7fbed8100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.488+0000 7fbede81c640 1 -- 172.21.15.67:0/2030932492 shutdown_connections 2024-07-31T14:56:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.488+0000 7fbede81c640 1 -- 172.21.15.67:0/2030932492 wait complete. 2024-07-31T14:56:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.489+0000 7fbede81c640 1 Processor -- start 2024-07-31T14:56:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.489+0000 7fbede81c640 1 -- start start 2024-07-31T14:56:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.489+0000 7fbede81c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbed81043a0 0x7fbed8199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.489+0000 7fbede81c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed8104cf0 0x7fbed819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.489+0000 7fbede81c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed810d7c0 0x7fbed819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.489+0000 7fbede81c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbed8113450 con 0x7fbed8104cf0 2024-07-31T14:56:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.489+0000 7fbede81c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81132d0 con 0x7fbed810d7c0 2024-07-31T14:56:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.489+0000 7fbede81c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81135d0 con 0x7fbed81043a0 2024-07-31T14:56:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.489+0000 7fbed77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed8104cf0 0x7fbed819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.489+0000 7fbed77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed8104cf0 0x7fbed819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48052/0 (socket says 172.21.15.67:48052) 2024-07-31T14:56:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.489+0000 7fbed77fe640 1 -- 172.21.15.67:0/2996914870 learned_addr learned my addr 172.21.15.67:0/2996914870 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.490+0000 7fbedcdb9640 1 --2- 172.21.15.67:0/2996914870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed810d7c0 0x7fbed819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.490+0000 7fbed7fff640 1 --2- 172.21.15.67:0/2996914870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbed81043a0 0x7fbed8199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.490+0000 7fbed77fe640 1 -- 172.21.15.67:0/2996914870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbed81043a0 msgr2=0x7fbed8199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.490+0000 7fbed77fe640 1 --2- 172.21.15.67:0/2996914870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbed81043a0 0x7fbed8199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.490+0000 7fbed77fe640 1 -- 172.21.15.67:0/2996914870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed810d7c0 msgr2=0x7fbed819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.490+0000 7fbed77fe640 1 --2- 172.21.15.67:0/2996914870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed810d7c0 0x7fbed819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.490+0000 7fbed77fe640 1 -- 172.21.15.67:0/2996914870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed819fe00 con 0x7fbed8104cf0 2024-07-31T14:56:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.490+0000 7fbed77fe640 1 --2- 172.21.15.67:0/2996914870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed8104cf0 0x7fbed819a470 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbecc006010 tx=0x7fbecc004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.490+0000 7fbed57fa640 1 -- 172.21.15.67:0/2996914870 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc03f030 con 0x7fbed8104cf0 2024-07-31T14:56:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.490+0000 7fbede81c640 1 -- 172.21.15.67:0/2996914870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed81a0090 con 0x7fbed8104cf0 2024-07-31T14:56:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.490+0000 7fbed57fa640 1 -- 172.21.15.67:0/2996914870 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbecc003aa0 con 0x7fbed8104cf0 2024-07-31T14:56:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.490+0000 7fbed57fa640 1 -- 172.21.15.67:0/2996914870 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc035070 con 0x7fbed8104cf0 2024-07-31T14:56:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.490+0000 7fbede81c640 1 -- 172.21.15.67:0/2996914870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed81a6880 con 0x7fbed8104cf0 2024-07-31T14:56:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.493+0000 7fbee57ea740 1 Processor -- start 2024-07-31T14:56:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.493+0000 7fbee57ea740 1 -- start start 2024-07-31T14:56:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.493+0000 7fbed57fa640 1 -- 172.21.15.67:0/2996914870 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbecc042030 con 0x7fbed8104cf0 2024-07-31T14:56:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.493+0000 7fbee57ea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f2f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.493+0000 7fbee57ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef292f0 0x55611f2f5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.493+0000 7fbee57ea740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef314d0 0x55611f2f5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.493+0000 7fbee57ea740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55611f26a3b0 con 0x55611ef521b0 2024-07-31T14:56:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.493+0000 7fbee57ea740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55611f2598b0 con 0x55611ef314d0 2024-07-31T14:56:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.493+0000 7fbee57ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55611f25d8f0 con 0x55611ef292f0 2024-07-31T14:56:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.493+0000 7fbedcdb9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f2f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.493+0000 7fbedcdb9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f2f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48060/0 (socket says 172.21.15.67:48060) 2024-07-31T14:56:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.493+0000 7fbedcdb9640 1 -- 172.21.15.67:0/3902294379 learned_addr learned my addr 172.21.15.67:0/3902294379 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.493+0000 7fbed77fe640 1 --2- 172.21.15.67:0/3902294379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef314d0 0x55611f2f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.494+0000 7fbed7fff640 1 --2- 172.21.15.67:0/3902294379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef292f0 0x55611f2f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.494+0000 7fbedcdb9640 1 -- 172.21.15.67:0/3902294379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef292f0 msgr2=0x55611f2f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.494+0000 7fbedcdb9640 1 --2- 172.21.15.67:0/3902294379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef292f0 0x55611f2f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.494+0000 7fbedcdb9640 1 -- 172.21.15.67:0/3902294379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef314d0 msgr2=0x55611f2f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.494+0000 7fbedcdb9640 1 --2- 172.21.15.67:0/3902294379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef314d0 0x55611f2f5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.494+0000 7fbedcdb9640 1 -- 172.21.15.67:0/3902294379 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55611f2009a0 con 0x55611ef521b0 2024-07-31T14:56:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.494+0000 7fbed77fe640 1 --2- 172.21.15.67:0/3902294379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef314d0 0x55611f2f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.494+0000 7fbed57fa640 1 --2- 172.21.15.67:0/2996914870 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbec00b3d40 0x7fbec00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.494+0000 7fbedcdb9640 1 --2- 172.21.15.67:0/3902294379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f2f4ce0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbed000a430 tx=0x7fbed0009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319d5f60df4ceb3b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.494+0000 7fbed7fff640 1 --2- 172.21.15.67:0/2996914870 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbec00b3d40 0x7fbec00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.494+0000 7fbed57fa640 1 -- 172.21.15.67:0/2996914870 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fbecc104820 con 0x7fbed8104cf0 2024-07-31T14:56:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.494+0000 7fbebe7fc640 1 -- 172.21.15.67:0/3902294379 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0013790 con 0x55611ef521b0 2024-07-31T14:56:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.494+0000 7fbebe7fc640 1 -- 172.21.15.67:0/3902294379 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed0019070 con 0x55611ef521b0 2024-07-31T14:56:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.494+0000 7fbebe7fc640 1 -- 172.21.15.67:0/3902294379 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0014ca0 con 0x55611ef521b0 2024-07-31T14:56:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.495+0000 7fbee57ea740 1 -- 172.21.15.67:0/3902294379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 msgr2=0x55611f2f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.495+0000 7fbed7fff640 1 --2- 172.21.15.67:0/2996914870 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbec00b3d40 0x7fbec00b6250 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55611f2f3960 tx=0x7fbec8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.495+0000 7fbee57ea740 1 --2- 172.21.15.67:0/3902294379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f2f4ce0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbed000a430 tx=0x7fbed0009650 comp rx=0 tx=0).stop 2024-07-31T14:56:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.495+0000 7fbee57ea740 1 -- 172.21.15.67:0/3902294379 shutdown_connections 2024-07-31T14:56:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.495+0000 7fbee57ea740 1 --2- 172.21.15.67:0/3902294379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef314d0 0x55611f2f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.495+0000 7fbee57ea740 1 --2- 172.21.15.67:0/3902294379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef292f0 0x55611f2f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.495+0000 7fbee57ea740 1 --2- 172.21.15.67:0/3902294379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f2f4ce0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.495+0000 7fbee57ea740 1 -- 172.21.15.67:0/3902294379 >> 172.21.15.67:0/3902294379 conn(0x55611ef46300 msgr2=0x55611f2f2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.495+0000 7fbee57ea740 1 -- 172.21.15.67:0/3902294379 shutdown_connections 2024-07-31T14:56:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.495+0000 7fbee57ea740 1 -- 172.21.15.67:0/3902294379 wait complete. 2024-07-31T14:56:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.496+0000 7fbee57ea740 1 -- start start 2024-07-31T14:56:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.496+0000 7fbee57ea740 1 Processor -- start 2024-07-31T14:56:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.496+0000 7fbee57ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef292f0 0x55611f304bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.496+0000 7fbee57ea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef314d0 0x55611f305130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.496+0000 7fbee57ea740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef521b0 0x55611f310ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.496+0000 7fbee57ea740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55611f193ab0 con 0x55611ef314d0 2024-07-31T14:56:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.496+0000 7fbee57ea740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55611f1e1cf0 con 0x55611ef521b0 2024-07-31T14:56:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.496+0000 7fbee57ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55611f307840 con 0x55611ef292f0 2024-07-31T14:56:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.496+0000 7fbee57ea740 10 client.0 _ll_register_callbacks cb 0x55611f2539f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbed77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef314d0 0x55611f305130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbed77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef314d0 0x55611f305130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48068/0 (socket says 172.21.15.67:48068) 2024-07-31T14:56:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbed77fe640 1 -- 172.21.15.67:0/2646781246 learned_addr learned my addr 172.21.15.67:0/2646781246 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbedcdb9640 1 --2- 172.21.15.67:0/2646781246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef521b0 0x55611f310ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbed7fff640 1 --2- 172.21.15.67:0/2646781246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef292f0 0x55611f304bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbedcdb9640 1 -- 172.21.15.67:0/2646781246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef292f0 msgr2=0x55611f304bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbedcdb9640 1 --2- 172.21.15.67:0/2646781246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef292f0 0x55611f304bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbedcdb9640 1 -- 172.21.15.67:0/2646781246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef314d0 msgr2=0x55611f305130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbedcdb9640 1 --2- 172.21.15.67:0/2646781246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef314d0 0x55611f305130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbedcdb9640 1 -- 172.21.15.67:0/2646781246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55611efbd790 con 0x55611ef521b0 2024-07-31T14:56:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbed77fe640 1 --2- 172.21.15.67:0/2646781246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef314d0 0x55611f305130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbedcdb9640 1 --2- 172.21.15.67:0/2646781246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef521b0 0x55611f310ba0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbed0004a80 tx=0x7fbed0009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd8f08879cf260a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbeb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:56:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbeb77fe640 1 -- 172.21.15.67:0/2646781246 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0013590 con 0x55611ef521b0 2024-07-31T14:56:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbee57ea740 1 -- 172.21.15.67:0/2646781246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed0019070 con 0x55611ef521b0 2024-07-31T14:56:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbeb77fe640 1 -- 172.21.15.67:0/2646781246 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fbed0013730 con 0x55611ef521b0 2024-07-31T14:56:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.497+0000 7fbeb77fe640 1 -- 172.21.15.67:0/2646781246 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbed0031050 con 0x55611ef521b0 2024-07-31T14:56:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.498+0000 7fbee57ea740 10 client.26558 fetch_fsmap learned FSMap version 42 2024-07-31T14:56:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.498+0000 7fbee57ea740 1 -- 172.21.15.67:0/2646781246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55611f036620 con 0x55611ef521b0 2024-07-31T14:56:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.498+0000 7fbeb77fe640 1 -- 172.21.15.67:0/2646781246 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed0019070 con 0x55611ef521b0 2024-07-31T14:56:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.498+0000 7fbeb77fe640 1 -- 172.21.15.67:0/2646781246 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7fbed0028820 con 0x55611ef521b0 2024-07-31T14:56:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.498+0000 7fbee57ea740 10 client.26558 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:56:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.498+0000 7fbee57ea740 10 client.26558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:56:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.498+0000 7fbee57ea740 20 client.26558 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.498+0000 7fbee57ea740 1 --2- 172.21.15.67:0/2646781246 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55611ef10d80 0x55611f316a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.498+0000 7fbee57ea740 4 client.26558 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:56:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.498+0000 7fbee57ea740 1 -- 172.21.15.67:0/2646781246 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55611f26c300 con 0x55611ef10d80 2024-07-31T14:56:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.498+0000 7fbed7fff640 1 --2- 172.21.15.67:0/2646781246 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55611ef10d80 0x55611f316a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.499+0000 7fbeb77fe640 10 client.26558 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.499+0000 7fbed7fff640 1 --2- 172.21.15.67:0/2646781246 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55611ef10d80 0x55611f316a60 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e8514658bfed85 server_cookie=636a47121f186f6c in_seq=0 out_seq=0 2024-07-31T14:56:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbeb77fe640 1 -- 172.21.15.67:0/2646781246 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55611f26c300 con 0x55611ef10d80 2024-07-31T14:56:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbeb77fe640 10 client.26558 handle_command_reply: tid=0 2024-07-31T14:56:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbee57ea740 1 client.26558 shutdown 2024-07-31T14:56:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbee57ea740 1 -- 172.21.15.67:0/2646781246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef521b0 msgr2=0x55611f310ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbee57ea740 1 --2- 172.21.15.67:0/2646781246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef521b0 0x55611f310ba0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbed0004a80 tx=0x7fbed0009c60 comp rx=0 tx=0).stop 2024-07-31T14:56:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbee57ea740 1 -- 172.21.15.67:0/2646781246 shutdown_connections 2024-07-31T14:56:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbee57ea740 1 --2- 172.21.15.67:0/2646781246 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55611ef10d80 0x55611f316a60 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbee57ea740 1 --2- 172.21.15.67:0/2646781246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef521b0 0x55611f310ba0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbee57ea740 1 --2- 172.21.15.67:0/2646781246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef314d0 0x55611f305130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbee57ea740 1 --2- 172.21.15.67:0/2646781246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef292f0 0x55611f304bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbeb77fe640 0 client.26558 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbee57ea740 1 -- 172.21.15.67:0/2646781246 >> 172.21.15.67:0/2646781246 conn(0x55611ef46300 msgr2=0x55611f2f66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbee57ea740 1 -- 172.21.15.67:0/2646781246 shutdown_connections 2024-07-31T14:56:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbee57ea740 1 -- 172.21.15.67:0/2646781246 wait complete. 2024-07-31T14:56:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.500+0000 7fbee57ea740 20 client.26558 trim_cache size 0 max 16384 2024-07-31T14:56:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.517+0000 7fbee57ea740 1 Processor -- start 2024-07-31T14:56:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.517+0000 7fbee57ea740 1 -- start start 2024-07-31T14:56:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.517+0000 7fbee57ea740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef292f0 0x55611f300150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.517+0000 7fbee57ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef314d0 0x55611f306610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.517+0000 7fbee57ea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f310b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.517+0000 7fbee57ea740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81ae530 con 0x55611ef521b0 2024-07-31T14:56:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.517+0000 7fbee57ea740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81b01f0 con 0x55611ef292f0 2024-07-31T14:56:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.517+0000 7fbee57ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81aec00 con 0x55611ef314d0 2024-07-31T14:56:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.517+0000 7fbedcdb9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f310b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.517+0000 7fbed7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef292f0 0x55611f300150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.517+0000 7fbedcdb9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f310b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48084/0 (socket says 172.21.15.67:48084) 2024-07-31T14:56:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.518+0000 7fbedcdb9640 1 -- 172.21.15.67:0/3500333005 learned_addr learned my addr 172.21.15.67:0/3500333005 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.518+0000 7fbed77fe640 1 --2- 172.21.15.67:0/3500333005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef314d0 0x55611f306610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.518+0000 7fbed7fff640 1 -- 172.21.15.67:0/3500333005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef314d0 msgr2=0x55611f306610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.518+0000 7fbed7fff640 1 --2- 172.21.15.67:0/3500333005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef314d0 0x55611f306610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.518+0000 7fbed7fff640 1 -- 172.21.15.67:0/3500333005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 msgr2=0x55611f310b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.518+0000 7fbed7fff640 1 --2- 172.21.15.67:0/3500333005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f310b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.518+0000 7fbed7fff640 1 -- 172.21.15.67:0/3500333005 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55611f2f7b40 con 0x55611ef292f0 2024-07-31T14:56:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.518+0000 7fbedcdb9640 1 --2- 172.21.15.67:0/3500333005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f310b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.518+0000 7fbed7fff640 1 --2- 172.21.15.67:0/3500333005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef292f0 0x55611f300150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55611f193b50 tx=0x7fbec8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b67f2320cfbcd2ed server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.518+0000 7fbebdffb640 1 -- 172.21.15.67:0/3500333005 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec802b420 con 0x55611ef292f0 2024-07-31T14:56:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.518+0000 7fbebdffb640 1 -- 172.21.15.67:0/3500333005 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbec802ba70 con 0x55611ef292f0 2024-07-31T14:56:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.518+0000 7fbebdffb640 1 -- 172.21.15.67:0/3500333005 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec801dc00 con 0x55611ef292f0 2024-07-31T14:56:21.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.519+0000 7fbee57ea740 1 -- 172.21.15.67:0/3500333005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef292f0 msgr2=0x55611f300150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:21.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.519+0000 7fbee57ea740 1 --2- 172.21.15.67:0/3500333005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef292f0 0x55611f300150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55611f193b50 tx=0x7fbec8017000 comp rx=0 tx=0).stop 2024-07-31T14:56:21.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.519+0000 7fbee57ea740 1 -- 172.21.15.67:0/3500333005 shutdown_connections 2024-07-31T14:56:21.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.519+0000 7fbee57ea740 1 --2- 172.21.15.67:0/3500333005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f310b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.519+0000 7fbee57ea740 1 --2- 172.21.15.67:0/3500333005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef314d0 0x55611f306610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.519+0000 7fbee57ea740 1 --2- 172.21.15.67:0/3500333005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef292f0 0x55611f300150 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.519+0000 7fbee57ea740 1 -- 172.21.15.67:0/3500333005 >> 172.21.15.67:0/3500333005 conn(0x55611ef46300 msgr2=0x55611f305f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:21.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.519+0000 7fbee57ea740 1 -- 172.21.15.67:0/3500333005 shutdown_connections 2024-07-31T14:56:21.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.519+0000 7fbee57ea740 1 -- 172.21.15.67:0/3500333005 wait complete. 2024-07-31T14:56:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.519+0000 7fbee57ea740 1 -- start start 2024-07-31T14:56:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbee57ea740 1 Processor -- start 2024-07-31T14:56:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbee57ea740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef292f0 0x55611f371160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbee57ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef314d0 0x55611f371700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbee57ea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f3731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbee57ea740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81afb20 con 0x55611ef521b0 2024-07-31T14:56:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbee57ea740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81af390 con 0x55611ef292f0 2024-07-31T14:56:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbee57ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81ac510 con 0x55611ef314d0 2024-07-31T14:56:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbee57ea740 10 client.0 _ll_register_callbacks cb 0x7fbecc10a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbedcdb9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f3731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbedcdb9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f3731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48088/0 (socket says 172.21.15.67:48088) 2024-07-31T14:56:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbedcdb9640 1 -- 172.21.15.67:0/3648261228 learned_addr learned my addr 172.21.15.67:0/3648261228 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbed7fff640 1 --2- 172.21.15.67:0/3648261228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef292f0 0x55611f371160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbed77fe640 1 --2- 172.21.15.67:0/3648261228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef314d0 0x55611f371700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbedcdb9640 1 -- 172.21.15.67:0/3648261228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef314d0 msgr2=0x55611f371700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbedcdb9640 1 --2- 172.21.15.67:0/3648261228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef314d0 0x55611f371700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.520+0000 7fbedcdb9640 1 -- 172.21.15.67:0/3648261228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef292f0 msgr2=0x55611f371160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.521+0000 7fbedcdb9640 1 --2- 172.21.15.67:0/3648261228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef292f0 0x55611f371160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.521+0000 7fbedcdb9640 1 -- 172.21.15.67:0/3648261228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55611f2f7210 con 0x55611ef521b0 2024-07-31T14:56:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.521+0000 7fbedcdb9640 1 --2- 172.21.15.67:0/3648261228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f3731e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbed00049c0 tx=0x7fbed00049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead1547ffec3b000 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.521+0000 7fbeb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:56:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.521+0000 7fbeb77fe640 1 -- 172.21.15.67:0/3648261228 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0028e60 con 0x55611ef521b0 2024-07-31T14:56:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.521+0000 7fbeb77fe640 1 -- 172.21.15.67:0/3648261228 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fbed00285e0 con 0x55611ef521b0 2024-07-31T14:56:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.521+0000 7fbee57ea740 1 -- 172.21.15.67:0/3648261228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbec802ba70 con 0x55611ef521b0 2024-07-31T14:56:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.521+0000 7fbeb77fe640 1 -- 172.21.15.67:0/3648261228 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbed00287e0 con 0x55611ef521b0 2024-07-31T14:56:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.521+0000 7fbee57ea740 10 client.16977 fetch_fsmap learned FSMap version 42 2024-07-31T14:56:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.521+0000 7fbee57ea740 1 -- 172.21.15.67:0/3648261228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55611f37dcd0 con 0x55611ef521b0 2024-07-31T14:56:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.521+0000 7fbeb77fe640 1 -- 172.21.15.67:0/3648261228 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbec802ba70 con 0x55611ef521b0 2024-07-31T14:56:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.522+0000 7fbeb77fe640 1 -- 172.21.15.67:0/3648261228 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 42) ==== 8422+0+0 (secure 0 0 0) 0x7fbed0013660 con 0x55611ef521b0 2024-07-31T14:56:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.522+0000 7fbee57ea740 10 client.16977 fetch_fsmap finished waiting for FSMap version 42 2024-07-31T14:56:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.522+0000 7fbee57ea740 10 client.16977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:56:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.522+0000 7fbee57ea740 20 client.16977 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.522+0000 7fbee57ea740 1 --2- 172.21.15.67:0/3648261228 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55611eee61c0 0x55611f380450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.522+0000 7fbee57ea740 4 client.16977 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:56:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.522+0000 7fbee57ea740 1 -- 172.21.15.67:0/3648261228 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55611f267da0 con 0x55611eee61c0 2024-07-31T14:56:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.522+0000 7fbed7fff640 1 --2- 172.21.15.67:0/3648261228 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55611eee61c0 0x55611f380450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.522+0000 7fbeb77fe640 10 client.16977 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:21.522+0000 7fbed7fff640 1 --2- 172.21.15.67:0/3648261228 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55611eee61c0 0x55611f380450 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=645e2504b029afff server_cookie=f79de3d9df403b25 in_seq=0 out_seq=0 2024-07-31T14:56:23.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:23 smithi067 ceph-mon[32750]: pgmap v559: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 495 op/s 2024-07-31T14:56:23.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:23 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:56:23.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:23 smithi073 ceph-mon[36714]: pgmap v559: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 495 op/s 2024-07-31T14:56:23.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:23 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:56:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:23 smithi077 ceph-mon[37327]: pgmap v559: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 495 op/s 2024-07-31T14:56:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:23 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:56:24.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.047+0000 7fbeb77fe640 1 -- 172.21.15.67:0/3648261228 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55611f267da0 con 0x55611eee61c0 2024-07-31T14:56:24.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.047+0000 7fbeb77fe640 10 client.16977 handle_command_reply: tid=0 2024-07-31T14:56:24.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.047+0000 7fbee57ea740 1 client.16977 shutdown 2024-07-31T14:56:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.048+0000 7fbee57ea740 1 -- 172.21.15.67:0/3648261228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 msgr2=0x55611f3731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.048+0000 7fbee57ea740 1 --2- 172.21.15.67:0/3648261228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f3731e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbed00049c0 tx=0x7fbed00049f0 comp rx=0 tx=0).stop 2024-07-31T14:56:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.048+0000 7fbee57ea740 1 -- 172.21.15.67:0/3648261228 shutdown_connections 2024-07-31T14:56:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.048+0000 7fbee57ea740 1 --2- 172.21.15.67:0/3648261228 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55611eee61c0 0x55611f380450 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.048+0000 7fbee57ea740 1 --2- 172.21.15.67:0/3648261228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55611ef521b0 0x55611f3731e0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.048+0000 7fbee57ea740 1 --2- 172.21.15.67:0/3648261228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55611ef314d0 0x55611f371700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.048+0000 7fbeb77fe640 0 client.16977 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.048+0000 7fbee57ea740 1 --2- 172.21.15.67:0/3648261228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55611ef292f0 0x55611f371160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.048+0000 7fbee57ea740 1 -- 172.21.15.67:0/3648261228 >> 172.21.15.67:0/3648261228 conn(0x55611ef46300 msgr2=0x55611f2f2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.048+0000 7fbee57ea740 1 -- 172.21.15.67:0/3648261228 shutdown_connections 2024-07-31T14:56:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.048+0000 7fbee57ea740 1 -- 172.21.15.67:0/3648261228 wait complete. 2024-07-31T14:56:24.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.048+0000 7fbee57ea740 20 client.16977 trim_cache size 0 max 16384 2024-07-31T14:56:24.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:56:24.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:56:24.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:56:24.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:56:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.049+0000 7fbede81c640 1 -- 172.21.15.67:0/2996914870 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbec00b3d40 msgr2=0x7fbec00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.049+0000 7fbede81c640 1 --2- 172.21.15.67:0/2996914870 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbec00b3d40 0x7fbec00b6250 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55611f2f3960 tx=0x7fbec8005f90 comp rx=0 tx=0).stop 2024-07-31T14:56:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.049+0000 7fbede81c640 1 -- 172.21.15.67:0/2996914870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed8104cf0 msgr2=0x7fbed819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.049+0000 7fbede81c640 1 --2- 172.21.15.67:0/2996914870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed8104cf0 0x7fbed819a470 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbecc006010 tx=0x7fbecc004060 comp rx=0 tx=0).stop 2024-07-31T14:56:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.049+0000 7fbede81c640 1 -- 172.21.15.67:0/2996914870 shutdown_connections 2024-07-31T14:56:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.049+0000 7fbede81c640 1 --2- 172.21.15.67:0/2996914870 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbec00b3d40 0x7fbec00b6250 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.049+0000 7fbede81c640 1 --2- 172.21.15.67:0/2996914870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbed810d7c0 0x7fbed819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.049+0000 7fbede81c640 1 --2- 172.21.15.67:0/2996914870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbed8104cf0 0x7fbed819a470 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.049+0000 7fbede81c640 1 --2- 172.21.15.67:0/2996914870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbed81043a0 0x7fbed8199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.049+0000 7fbede81c640 1 -- 172.21.15.67:0/2996914870 >> 172.21.15.67:0/2996914870 conn(0x7fbed80fddf0 msgr2=0x7fbed81072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.049+0000 7fbede81c640 1 -- 172.21.15.67:0/2996914870 shutdown_connections 2024-07-31T14:56:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:24.049+0000 7fbede81c640 1 -- 172.21.15.67:0/2996914870 wait complete. 2024-07-31T14:56:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:25 smithi067 ceph-mon[32750]: pgmap v560: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 440 op/s 2024-07-31T14:56:25.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:25 smithi073 ceph-mon[36714]: pgmap v560: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 440 op/s 2024-07-31T14:56:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:25 smithi077 ceph-mon[37327]: pgmap v560: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 440 op/s 2024-07-31T14:56:26.811 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:56:27.176 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:56:27.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:27 smithi067 ceph-mon[32750]: pgmap v561: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 630 op/s 2024-07-31T14:56:27.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:27 smithi073 ceph-mon[36714]: pgmap v561: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 630 op/s 2024-07-31T14:56:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:27 smithi077 ceph-mon[37327]: pgmap v561: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 630 op/s 2024-07-31T14:56:29.160 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:29 smithi067 ceph-mon[32750]: pgmap v562: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 840 op/s 2024-07-31T14:56:29.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.263+0000 7f8801848640 1 -- 172.21.15.67:0/2658806124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc108760 msgr2=0x7f87fc10ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:29.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.263+0000 7f8801848640 1 --2- 172.21.15.67:0/2658806124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc108760 0x7f87fc10ef90 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009ca0 tx=0x7f87f402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.263+0000 7f8801848640 1 -- 172.21.15.67:0/2658806124 shutdown_connections 2024-07-31T14:56:29.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.263+0000 7f8801848640 1 --2- 172.21.15.67:0/2658806124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc108760 0x7f87fc10ef90 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.263+0000 7f8801848640 1 --2- 172.21.15.67:0/2658806124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc107e10 0x7f87fc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.263+0000 7f8801848640 1 --2- 172.21.15.67:0/2658806124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87fc1024d0 0x7f87fc1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.263+0000 7f8801848640 1 -- 172.21.15.67:0/2658806124 >> 172.21.15.67:0/2658806124 conn(0x7f87fc0791e0 msgr2=0x7f87fc0795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:29.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.264+0000 7f8801848640 1 -- 172.21.15.67:0/2658806124 shutdown_connections 2024-07-31T14:56:29.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.264+0000 7f8801848640 1 -- 172.21.15.67:0/2658806124 wait complete. 2024-07-31T14:56:29.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.264+0000 7f8801848640 1 Processor -- start 2024-07-31T14:56:29.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f8801848640 1 -- start start 2024-07-31T14:56:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f8801848640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc1024d0 0x7f87fc199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f8801848640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc107e10 0x7f87fc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f8801848640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87fc108760 0x7f87fc19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f8801848640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc112430 con 0x7f87fc1024d0 2024-07-31T14:56:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f8801848640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1122b0 con 0x7f87fc108760 2024-07-31T14:56:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f8801848640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1125b0 con 0x7f87fc107e10 2024-07-31T14:56:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f87faffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc1024d0 0x7f87fc199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f87fb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87fc108760 0x7f87fc19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f87faffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc1024d0 0x7f87fc199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48112/0 (socket says 172.21.15.67:48112) 2024-07-31T14:56:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f87fb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87fc108760 0x7f87fc19f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:39608/0 (socket says 172.21.15.67:39608) 2024-07-31T14:56:29.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f87faffd640 1 -- 172.21.15.67:0/3677203898 learned_addr learned my addr 172.21.15.67:0/3677203898 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:29.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f87fa7fc640 1 --2- 172.21.15.67:0/3677203898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc107e10 0x7f87fc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f87fb7fe640 1 -- 172.21.15.67:0/3677203898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc107e10 msgr2=0x7f87fc19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:29.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f87fb7fe640 1 --2- 172.21.15.67:0/3677203898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc107e10 0x7f87fc19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f87fb7fe640 1 -- 172.21.15.67:0/3677203898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc1024d0 msgr2=0x7f87fc199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:29.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f87fb7fe640 1 --2- 172.21.15.67:0/3677203898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc1024d0 0x7f87fc199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.265+0000 7f87fb7fe640 1 -- 172.21.15.67:0/3677203898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87fc19fe60 con 0x7f87fc108760 2024-07-31T14:56:29.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.266+0000 7f87fb7fe640 1 --2- 172.21.15.67:0/3677203898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87fc108760 0x7f87fc19f730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f87f402ade0 tx=0x7f87f40316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:29.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.266+0000 7f8800846640 1 -- 172.21.15.67:0/3677203898 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4003b70 con 0x7f87fc108760 2024-07-31T14:56:29.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.266+0000 7f8801848640 1 -- 172.21.15.67:0/3677203898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87fc1a00f0 con 0x7f87fc108760 2024-07-31T14:56:29.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.266+0000 7f8800846640 1 -- 172.21.15.67:0/3677203898 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f4003d10 con 0x7f87fc108760 2024-07-31T14:56:29.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.266+0000 7f8800846640 1 -- 172.21.15.67:0/3677203898 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f402f9e0 con 0x7f87fc108760 2024-07-31T14:56:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.266+0000 7f8801848640 1 -- 172.21.15.67:0/3677203898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87fc1a68b0 con 0x7f87fc108760 2024-07-31T14:56:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.269+0000 7f8807d99740 1 Processor -- start 2024-07-31T14:56:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.269+0000 7f8800846640 1 -- 172.21.15.67:0/3677203898 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f87f402fb80 con 0x7f87fc108760 2024-07-31T14:56:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.269+0000 7f8807d99740 1 -- start start 2024-07-31T14:56:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.269+0000 7f8807d99740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 0x55d480187520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.269+0000 7f8807d99740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdb91b0 0x55d480187a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.269+0000 7f8807d99740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdc1390 0x55d480187fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.269+0000 7f8807d99740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d4800fdb60 con 0x55d47fdc1390 2024-07-31T14:56:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.269+0000 7f8807d99740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d4800ed060 con 0x55d47fdb91b0 2024-07-31T14:56:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.269+0000 7f8807d99740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d4800f10a0 con 0x55d47fde2070 2024-07-31T14:56:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.270+0000 7f87fa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdc1390 0x55d480187fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.270+0000 7f87faffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdb91b0 0x55d480187a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.270+0000 7f87faffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdb91b0 0x55d480187a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:39618/0 (socket says 172.21.15.67:39618) 2024-07-31T14:56:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.270+0000 7f87fa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdc1390 0x55d480187fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48122/0 (socket says 172.21.15.67:48122) 2024-07-31T14:56:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.270+0000 7f87faffd640 1 -- 172.21.15.67:0/1572342878 learned_addr learned my addr 172.21.15.67:0/1572342878 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.270+0000 7f8800846640 1 --2- 172.21.15.67:0/3677203898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f87e40b3d40 0x7f87e40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.270+0000 7f87fb7fe640 1 --2- 172.21.15.67:0/3677203898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f87e40b3d40 0x7f87e40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.270+0000 7f87faffd640 1 -- 172.21.15.67:0/1572342878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 msgr2=0x55d480187520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.270+0000 7f87faffd640 1 --2- 172.21.15.67:0/1572342878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 0x55d480187520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.270+0000 7f87faffd640 1 -- 172.21.15.67:0/1572342878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdc1390 msgr2=0x55d480187fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.270+0000 7f87faffd640 1 --2- 172.21.15.67:0/1572342878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdc1390 0x55d480187fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.270+0000 7f87faffd640 1 -- 172.21.15.67:0/1572342878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d480093b40 con 0x55d47fdb91b0 2024-07-31T14:56:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.270+0000 7f8800846640 1 -- 172.21.15.67:0/3677203898 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f87f4105510 con 0x7f87fc108760 2024-07-31T14:56:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.270+0000 7f87fa7fc640 1 --2- 172.21.15.67:0/1572342878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdc1390 0x55d480187fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.271+0000 7f87faffd640 1 --2- 172.21.15.67:0/1572342878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdb91b0 0x55d480187a60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009dc0 tx=0x7f87ec011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae63599fbf9746ac server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:29.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.271+0000 7f87fb7fe640 1 --2- 172.21.15.67:0/3677203898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f87e40b3d40 0x7f87e40b6250 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f87f40cbeb0 tx=0x7f87f40983d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:29.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.271+0000 7f87e1ffb640 1 -- 172.21.15.67:0/1572342878 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec005220 con 0x55d47fdb91b0 2024-07-31T14:56:29.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.271+0000 7f87e1ffb640 1 -- 172.21.15.67:0/1572342878 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87ec004c50 con 0x55d47fdb91b0 2024-07-31T14:56:29.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.271+0000 7f87e1ffb640 1 -- 172.21.15.67:0/1572342878 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec0197f0 con 0x55d47fdb91b0 2024-07-31T14:56:29.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.272+0000 7f8807d99740 1 -- 172.21.15.67:0/1572342878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdb91b0 msgr2=0x55d480187a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:29.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.272+0000 7f8807d99740 1 --2- 172.21.15.67:0/1572342878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdb91b0 0x55d480187a60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009dc0 tx=0x7f87ec011810 comp rx=0 tx=0).stop 2024-07-31T14:56:29.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.272+0000 7f8807d99740 1 -- 172.21.15.67:0/1572342878 shutdown_connections 2024-07-31T14:56:29.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.272+0000 7f8807d99740 1 --2- 172.21.15.67:0/1572342878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdc1390 0x55d480187fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.272+0000 7f8807d99740 1 --2- 172.21.15.67:0/1572342878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdb91b0 0x55d480187a60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.272+0000 7f8807d99740 1 --2- 172.21.15.67:0/1572342878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 0x55d480187520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.272+0000 7f8807d99740 1 -- 172.21.15.67:0/1572342878 >> 172.21.15.67:0/1572342878 conn(0x55d47fdd6460 msgr2=0x55d480185410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:29.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.272+0000 7f8807d99740 1 -- 172.21.15.67:0/1572342878 shutdown_connections 2024-07-31T14:56:29.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.272+0000 7f8807d99740 1 -- 172.21.15.67:0/1572342878 wait complete. 2024-07-31T14:56:29.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.272+0000 7f8807d99740 1 -- start start 2024-07-31T14:56:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.272+0000 7f8807d99740 1 Processor -- start 2024-07-31T14:56:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f8807d99740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdb91b0 0x55d4801930e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f8807d99740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdc1390 0x55d480193620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f8807d99740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 0x55d48019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f8807d99740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d47ff50700 con 0x55d47fdb91b0 2024-07-31T14:56:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f8807d99740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d48008f990 con 0x55d47fdc1390 2024-07-31T14:56:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f8807d99740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d480195d00 con 0x55d47fde2070 2024-07-31T14:56:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f8807d99740 10 client.0 _ll_register_callbacks cb 0x55d4800aa4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f87faffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdc1390 0x55d480193620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f87fa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdb91b0 0x55d4801930e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f87faffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdc1390 0x55d480193620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:39622/0 (socket says 172.21.15.67:39622) 2024-07-31T14:56:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f87faffd640 1 -- 172.21.15.67:0/2907223231 learned_addr learned my addr 172.21.15.67:0/2907223231 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f87fa7fc640 1 --2- 172.21.15.67:0/2907223231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 0x55d48019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f87faffd640 1 -- 172.21.15.67:0/2907223231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 msgr2=0x55d48019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f87faffd640 1 --2- 172.21.15.67:0/2907223231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 0x55d48019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f87faffd640 1 -- 172.21.15.67:0/2907223231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdb91b0 msgr2=0x55d4801930e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f87faffd640 1 --2- 172.21.15.67:0/2907223231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdb91b0 0x55d4801930e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.273+0000 7f87faffd640 1 -- 172.21.15.67:0/2907223231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d47fe4d650 con 0x55d47fdc1390 2024-07-31T14:56:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.274+0000 7f87faffd640 1 --2- 172.21.15.67:0/2907223231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdc1390 0x55d480193620 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f87ec005790 tx=0x7f87ec004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f755105206c707 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.274+0000 7f87daffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:56:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.274+0000 7f87daffd640 1 -- 172.21.15.67:0/2907223231 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec027070 con 0x55d47fdc1390 2024-07-31T14:56:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.274+0000 7f87daffd640 1 -- 172.21.15.67:0/2907223231 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f87ec005200 con 0x55d47fdc1390 2024-07-31T14:56:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.274+0000 7f8807d99740 1 -- 172.21.15.67:0/2907223231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87ec004c50 con 0x55d47fdc1390 2024-07-31T14:56:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.274+0000 7f87daffd640 1 -- 172.21.15.67:0/2907223231 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f87ec031050 con 0x55d47fdc1390 2024-07-31T14:56:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.274+0000 7f8807d99740 10 client.26576 fetch_fsmap learned FSMap version 43 2024-07-31T14:56:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.274+0000 7f8807d99740 1 -- 172.21.15.67:0/2907223231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d4801a1d20 con 0x55d47fdc1390 2024-07-31T14:56:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.274+0000 7f87daffd640 1 -- 172.21.15.67:0/2907223231 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87ec004c50 con 0x55d47fdc1390 2024-07-31T14:56:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.275+0000 7f87daffd640 1 -- 172.21.15.67:0/2907223231 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) ==== 8434+0+0 (secure 0 0 0) 0x7f87ec003be0 con 0x55d47fdc1390 2024-07-31T14:56:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.275+0000 7f8807d99740 10 client.26576 fetch_fsmap finished waiting for FSMap version 43 2024-07-31T14:56:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.275+0000 7f8807d99740 10 client.26576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:56:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.275+0000 7f8807d99740 20 client.26576 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.275+0000 7f8807d99740 1 --2- 172.21.15.67:0/2907223231 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d47fda09f0 0x55d4801a5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.275+0000 7f8807d99740 4 client.26576 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:56:29.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.275+0000 7f8807d99740 1 -- 172.21.15.67:0/2907223231 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4800ffab0 con 0x55d47fda09f0 2024-07-31T14:56:29.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.275+0000 7f87fa7fc640 1 --2- 172.21.15.67:0/2907223231 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d47fda09f0 0x55d4801a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.275+0000 7f87daffd640 10 client.26576 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:29.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.276+0000 7f87fa7fc640 1 --2- 172.21.15.67:0/2907223231 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d47fda09f0 0x55d4801a5190 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7767e034abf52f4 server_cookie=fda1e4baa8dba55b in_seq=0 out_seq=0 2024-07-31T14:56:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.276+0000 7f87daffd640 1 -- 172.21.15.67:0/2907223231 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4800ffab0 con 0x55d47fda09f0 2024-07-31T14:56:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.276+0000 7f87daffd640 10 client.26576 handle_command_reply: tid=0 2024-07-31T14:56:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.276+0000 7f8807d99740 1 client.26576 shutdown 2024-07-31T14:56:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.277+0000 7f8807d99740 1 -- 172.21.15.67:0/2907223231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdc1390 msgr2=0x55d480193620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.277+0000 7f8807d99740 1 --2- 172.21.15.67:0/2907223231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdc1390 0x55d480193620 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f87ec005790 tx=0x7f87ec004270 comp rx=0 tx=0).stop 2024-07-31T14:56:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.277+0000 7f8807d99740 1 -- 172.21.15.67:0/2907223231 shutdown_connections 2024-07-31T14:56:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.277+0000 7f8807d99740 1 --2- 172.21.15.67:0/2907223231 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d47fda09f0 0x55d4801a5190 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.277+0000 7f8807d99740 1 --2- 172.21.15.67:0/2907223231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 0x55d48019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.277+0000 7f8807d99740 1 --2- 172.21.15.67:0/2907223231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdc1390 0x55d480193620 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.277+0000 7f8807d99740 1 --2- 172.21.15.67:0/2907223231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdb91b0 0x55d4801930e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.277+0000 7f8807d99740 1 -- 172.21.15.67:0/2907223231 >> 172.21.15.67:0/2907223231 conn(0x55d47fdd6460 msgr2=0x55d480183bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.277+0000 7f87daffd640 0 client.26576 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.277+0000 7f8807d99740 1 -- 172.21.15.67:0/2907223231 shutdown_connections 2024-07-31T14:56:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.277+0000 7f8807d99740 1 -- 172.21.15.67:0/2907223231 wait complete. 2024-07-31T14:56:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.277+0000 7f8807d99740 20 client.26576 trim_cache size 0 max 16384 2024-07-31T14:56:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.295+0000 7f8807d99740 1 Processor -- start 2024-07-31T14:56:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.295+0000 7f8807d99740 1 -- start start 2024-07-31T14:56:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.296+0000 7f8807d99740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdb91b0 0x55d48019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.296+0000 7f8807d99740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdc1390 0x55d480183bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.296+0000 7f8807d99740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 0x55d480184130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.296+0000 7f8807d99740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1b0780 con 0x55d47fdb91b0 2024-07-31T14:56:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.296+0000 7f8807d99740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1b24d0 con 0x55d47fdc1390 2024-07-31T14:56:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.296+0000 7f8807d99740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1b0ee0 con 0x55d47fde2070 2024-07-31T14:56:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.296+0000 7f87fa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdb91b0 0x55d48019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.296+0000 7f87fa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdb91b0 0x55d48019ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48136/0 (socket says 172.21.15.67:48136) 2024-07-31T14:56:29.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.296+0000 7f87fa7fc640 1 -- 172.21.15.67:0/2427127299 learned_addr learned my addr 172.21.15.67:0/2427127299 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:29.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.296+0000 7f87fa7fc640 1 -- 172.21.15.67:0/2427127299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 msgr2=0x55d480184130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.296+0000 7f87fa7fc640 1 --2- 172.21.15.67:0/2427127299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 0x55d480184130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.296+0000 7f87fa7fc640 1 -- 172.21.15.67:0/2427127299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdc1390 msgr2=0x55d480183bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:56:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.296+0000 7f87fa7fc640 1 --2- 172.21.15.67:0/2427127299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdc1390 0x55d480183bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.296+0000 7f87fa7fc640 1 -- 172.21.15.67:0/2427127299 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4801941a0 con 0x55d47fdb91b0 2024-07-31T14:56:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.297+0000 7f87fa7fc640 1 --2- 172.21.15.67:0/2427127299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdb91b0 0x55d48019ede0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f87fc06a4d0 tx=0x7f87f0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc1e045fbd27a9c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.297+0000 7f87e17fa640 1 -- 172.21.15.67:0/2427127299 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0004f90 con 0x55d47fdb91b0 2024-07-31T14:56:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.297+0000 7f87e17fa640 1 -- 172.21.15.67:0/2427127299 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f00035f0 con 0x55d47fdb91b0 2024-07-31T14:56:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.297+0000 7f87e17fa640 1 -- 172.21.15.67:0/2427127299 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0020420 con 0x55d47fdb91b0 2024-07-31T14:56:29.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.298+0000 7f8807d99740 1 -- 172.21.15.67:0/2427127299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdb91b0 msgr2=0x55d48019ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:29.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.298+0000 7f8807d99740 1 --2- 172.21.15.67:0/2427127299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdb91b0 0x55d48019ede0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f87fc06a4d0 tx=0x7f87f0005c90 comp rx=0 tx=0).stop 2024-07-31T14:56:29.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.298+0000 7f8807d99740 1 -- 172.21.15.67:0/2427127299 shutdown_connections 2024-07-31T14:56:29.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.298+0000 7f8807d99740 1 --2- 172.21.15.67:0/2427127299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 0x55d480184130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.298+0000 7f8807d99740 1 --2- 172.21.15.67:0/2427127299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdc1390 0x55d480183bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.298+0000 7f8807d99740 1 --2- 172.21.15.67:0/2427127299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdb91b0 0x55d48019ede0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.298+0000 7f8807d99740 1 -- 172.21.15.67:0/2427127299 >> 172.21.15.67:0/2427127299 conn(0x55d47fdd6460 msgr2=0x55d480191760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.298+0000 7f8807d99740 1 -- 172.21.15.67:0/2427127299 shutdown_connections 2024-07-31T14:56:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.298+0000 7f8807d99740 1 -- 172.21.15.67:0/2427127299 wait complete. 2024-07-31T14:56:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.299+0000 7f8807d99740 1 -- start start 2024-07-31T14:56:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.299+0000 7f8807d99740 1 Processor -- start 2024-07-31T14:56:29.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.299+0000 7f8807d99740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdb91b0 0x55d48019b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.299+0000 7f8807d99740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdc1390 0x55d4801969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.299+0000 7f8807d99740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 0x55d480196f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.299+0000 7f8807d99740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1b1e00 con 0x55d47fdc1390 2024-07-31T14:56:29.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.299+0000 7f8807d99740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1b1670 con 0x55d47fdb91b0 2024-07-31T14:56:29.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.299+0000 7f8807d99740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1ae760 con 0x55d47fde2070 2024-07-31T14:56:29.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.299+0000 7f8807d99740 10 client.0 _ll_register_callbacks cb 0x7f87f000b7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:29.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.299+0000 7f87faffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdc1390 0x55d4801969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f87fa7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdb91b0 0x55d48019b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f87faffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdc1390 0x55d4801969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48150/0 (socket says 172.21.15.67:48150) 2024-07-31T14:56:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f87faffd640 1 -- 172.21.15.67:0/2701631297 learned_addr learned my addr 172.21.15.67:0/2701631297 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f87fa7fc640 1 --2- 172.21.15.67:0/2701631297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 0x55d480196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f87faffd640 1 -- 172.21.15.67:0/2701631297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 msgr2=0x55d480196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f87faffd640 1 --2- 172.21.15.67:0/2701631297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 0x55d480196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f87faffd640 1 -- 172.21.15.67:0/2701631297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdb91b0 msgr2=0x55d48019b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f87faffd640 1 --2- 172.21.15.67:0/2701631297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdb91b0 0x55d48019b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f87faffd640 1 -- 172.21.15.67:0/2701631297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4801975d0 con 0x55d47fdc1390 2024-07-31T14:56:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f87fa7fc640 1 --2- 172.21.15.67:0/2701631297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdb91b0 0x55d48019b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f87faffd640 1 --2- 172.21.15.67:0/2701631297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdc1390 0x55d4801969f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f87ec000c00 tx=0x7f87ec0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc591f2eacbe868 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f87daffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:56:29.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f87daffd640 1 -- 172.21.15.67:0/2701631297 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec03a070 con 0x55d47fdc1390 2024-07-31T14:56:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f87daffd640 1 -- 172.21.15.67:0/2701631297 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f87ec003a00 con 0x55d47fdc1390 2024-07-31T14:56:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.300+0000 7f8807d99740 1 -- 172.21.15.67:0/2701631297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f00035f0 con 0x55d47fdc1390 2024-07-31T14:56:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.301+0000 7f87daffd640 1 -- 172.21.15.67:0/2701631297 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f87ec046050 con 0x55d47fdc1390 2024-07-31T14:56:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.301+0000 7f8807d99740 10 client.16995 fetch_fsmap learned FSMap version 43 2024-07-31T14:56:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.301+0000 7f8807d99740 1 -- 172.21.15.67:0/2701631297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d48020d720 con 0x55d47fdc1390 2024-07-31T14:56:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.301+0000 7f87daffd640 1 -- 172.21.15.67:0/2701631297 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f00035f0 con 0x55d47fdc1390 2024-07-31T14:56:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.301+0000 7f87daffd640 1 -- 172.21.15.67:0/2701631297 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 43) ==== 8434+0+0 (secure 0 0 0) 0x7f87ec023910 con 0x55d47fdc1390 2024-07-31T14:56:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.301+0000 7f8807d99740 10 client.16995 fetch_fsmap finished waiting for FSMap version 43 2024-07-31T14:56:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.301+0000 7f8807d99740 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:56:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.301+0000 7f8807d99740 20 client.16995 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.301+0000 7f8807d99740 1 --2- 172.21.15.67:0/2701631297 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d47fd75da0 0x55d48020fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.301+0000 7f8807d99740 4 client.16995 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:56:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.301+0000 7f8807d99740 1 -- 172.21.15.67:0/2701631297 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d4800fb550 con 0x55d47fd75da0 2024-07-31T14:56:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.301+0000 7f87fa7fc640 1 --2- 172.21.15.67:0/2701631297 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d47fd75da0 0x55d48020fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:29.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.302+0000 7f87daffd640 10 client.16995 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:29.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.302+0000 7f87fa7fc640 1 --2- 172.21.15.67:0/2701631297 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d47fd75da0 0x55d48020fee0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c35c6a6cc32fa1be server_cookie=792d1caf1cb9ddc6 in_seq=0 out_seq=0 2024-07-31T14:56:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.302+0000 7f87daffd640 1 -- 172.21.15.67:0/2701631297 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d4800fb550 con 0x55d47fd75da0 2024-07-31T14:56:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.302+0000 7f87daffd640 10 client.16995 handle_command_reply: tid=0 2024-07-31T14:56:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.302+0000 7f8807d99740 1 client.16995 shutdown 2024-07-31T14:56:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.303+0000 7f8807d99740 1 -- 172.21.15.67:0/2701631297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdc1390 msgr2=0x55d4801969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.303+0000 7f8807d99740 1 --2- 172.21.15.67:0/2701631297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdc1390 0x55d4801969f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f87ec000c00 tx=0x7f87ec0042e0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.303+0000 7f8807d99740 1 -- 172.21.15.67:0/2701631297 shutdown_connections 2024-07-31T14:56:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.303+0000 7f8807d99740 1 --2- 172.21.15.67:0/2701631297 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d47fd75da0 0x55d48020fee0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.303+0000 7f8807d99740 1 --2- 172.21.15.67:0/2701631297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47fde2070 0x55d480196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.303+0000 7f87daffd640 0 client.16995 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.303+0000 7f8807d99740 1 --2- 172.21.15.67:0/2701631297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d47fdc1390 0x55d4801969f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.303+0000 7f8807d99740 1 --2- 172.21.15.67:0/2701631297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d47fdb91b0 0x55d48019b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.303+0000 7f8807d99740 1 -- 172.21.15.67:0/2701631297 >> 172.21.15.67:0/2701631297 conn(0x55d47fdd6460 msgr2=0x55d480183bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:29.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.303+0000 7f8807d99740 1 -- 172.21.15.67:0/2701631297 shutdown_connections 2024-07-31T14:56:29.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.303+0000 7f8807d99740 1 -- 172.21.15.67:0/2701631297 wait complete. 2024-07-31T14:56:29.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.303+0000 7f8807d99740 20 client.16995 trim_cache size 0 max 16384 2024-07-31T14:56:29.308 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:56:29.308 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T14:56:29.308 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T14:56:29.308 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:56:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.304+0000 7f8801848640 1 -- 172.21.15.67:0/3677203898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f87e40b3d40 msgr2=0x7f87e40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.304+0000 7f8801848640 1 --2- 172.21.15.67:0/3677203898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f87e40b3d40 0x7f87e40b6250 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f87f40cbeb0 tx=0x7f87f40983d0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.304+0000 7f8801848640 1 -- 172.21.15.67:0/3677203898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87fc108760 msgr2=0x7f87fc19f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.304+0000 7f8801848640 1 --2- 172.21.15.67:0/3677203898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87fc108760 0x7f87fc19f730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f87f402ade0 tx=0x7f87f40316f0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.304+0000 7f8801848640 1 -- 172.21.15.67:0/3677203898 shutdown_connections 2024-07-31T14:56:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.304+0000 7f8801848640 1 --2- 172.21.15.67:0/3677203898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f87e40b3d40 0x7f87e40b6250 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.304+0000 7f8801848640 1 --2- 172.21.15.67:0/3677203898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87fc108760 0x7f87fc19f730 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.304+0000 7f8801848640 1 --2- 172.21.15.67:0/3677203898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc107e10 0x7f87fc19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.304+0000 7f8801848640 1 --2- 172.21.15.67:0/3677203898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc1024d0 0x7f87fc199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.304+0000 7f8801848640 1 -- 172.21.15.67:0/3677203898 >> 172.21.15.67:0/3677203898 conn(0x7f87fc0791e0 msgr2=0x7f87fc1014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.304+0000 7f8801848640 1 -- 172.21.15.67:0/3677203898 shutdown_connections 2024-07-31T14:56:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:29.304+0000 7f8801848640 1 -- 172.21.15.67:0/3677203898 wait complete. 2024-07-31T14:56:29.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:29 smithi073 ceph-mon[36714]: pgmap v562: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 840 op/s 2024-07-31T14:56:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:29 smithi077 ceph-mon[37327]: pgmap v562: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 840 op/s 2024-07-31T14:56:30.147 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T14:56:30.148 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:56:30.487 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:56:31.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:31 smithi067 ceph-mon[32750]: pgmap v563: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 916 op/s 2024-07-31T14:56:31.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:31 smithi073 ceph-mon[36714]: pgmap v563: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 916 op/s 2024-07-31T14:56:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:31 smithi077 ceph-mon[37327]: pgmap v563: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 916 op/s 2024-07-31T14:56:33.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:33 smithi067 ceph-mon[32750]: pgmap v564: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 1.01k op/s 2024-07-31T14:56:33.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:33 smithi073 ceph-mon[36714]: pgmap v564: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 1.01k op/s 2024-07-31T14:56:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.392+0000 7fb5610f6640 1 -- 172.21.15.67:0/194497848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb55c105510 msgr2=0x7fb55c10ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.392+0000 7fb5610f6640 1 --2- 172.21.15.67:0/194497848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb55c105510 0x7fb55c10ef80 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009ca0 tx=0x7fb54c02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:56:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.392+0000 7fb5610f6640 1 -- 172.21.15.67:0/194497848 shutdown_connections 2024-07-31T14:56:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.392+0000 7fb5610f6640 1 --2- 172.21.15.67:0/194497848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb55c105510 0x7fb55c10ef80 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.392+0000 7fb5610f6640 1 --2- 172.21.15.67:0/194497848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb55c104bc0 0x7fb55c104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:33.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.392+0000 7fb5610f6640 1 --2- 172.21.15.67:0/194497848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb55c102bf0 0x7fb55c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.392+0000 7fb5610f6640 1 -- 172.21.15.67:0/194497848 >> 172.21.15.67:0/194497848 conn(0x7fb55c077690 msgr2=0x7fb55c077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.392+0000 7fb5610f6640 1 -- 172.21.15.67:0/194497848 shutdown_connections 2024-07-31T14:56:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.392+0000 7fb5610f6640 1 -- 172.21.15.67:0/194497848 wait complete. 2024-07-31T14:56:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.393+0000 7fb5610f6640 1 Processor -- start 2024-07-31T14:56:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb5610f6640 1 -- start start 2024-07-31T14:56:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb5610f6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb55c102bf0 0x7fb55c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb5610f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb55c104bc0 0x7fb55c19b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb5610f6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb55c105510 0x7fb55c19b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb5610f6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c112430 con 0x7fb55c102bf0 2024-07-31T14:56:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb5610f6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c1122b0 con 0x7fb55c105510 2024-07-31T14:56:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb5610f6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c1125b0 con 0x7fb55c104bc0 2024-07-31T14:56:33.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb55ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb55c102bf0 0x7fb55c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:33.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb55ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb55c102bf0 0x7fb55c19aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46144/0 (socket says 172.21.15.67:46144) 2024-07-31T14:56:33.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb55ad9d640 1 -- 172.21.15.67:0/604326411 learned_addr learned my addr 172.21.15.67:0/604326411 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:33.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb55b59e640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb55c105510 0x7fb55c19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:33.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb55a59c640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb55c104bc0 0x7fb55c19b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:33.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb55b59e640 1 -- 172.21.15.67:0/604326411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb55c104bc0 msgr2=0x7fb55c19b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:33.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb55b59e640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb55c104bc0 0x7fb55c19b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:33.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb55b59e640 1 -- 172.21.15.67:0/604326411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb55c102bf0 msgr2=0x7fb55c19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:33.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb55b59e640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb55c102bf0 0x7fb55c19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:33.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb55b59e640 1 -- 172.21.15.67:0/604326411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55c1a5f30 con 0x7fb55c105510 2024-07-31T14:56:33.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.394+0000 7fb55ad9d640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb55c102bf0 0x7fb55c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:33.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.395+0000 7fb55b59e640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb55c105510 0x7fb55c19b720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb54c005e30 tx=0x7fb54c0317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:33.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.395+0000 7fb553fff640 1 -- 172.21.15.67:0/604326411 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c003b70 con 0x7fb55c105510 2024-07-31T14:56:33.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.395+0000 7fb553fff640 1 -- 172.21.15.67:0/604326411 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb54c003d10 con 0x7fb55c105510 2024-07-31T14:56:33.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.395+0000 7fb5610f6640 1 -- 172.21.15.67:0/604326411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55c1a61c0 con 0x7fb55c105510 2024-07-31T14:56:33.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.395+0000 7fb553fff640 1 -- 172.21.15.67:0/604326411 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c02f9e0 con 0x7fb55c105510 2024-07-31T14:56:33.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.395+0000 7fb5610f6640 1 -- 172.21.15.67:0/604326411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb55c1a6750 con 0x7fb55c105510 2024-07-31T14:56:33.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.396+0000 7fb5610f6640 1 -- 172.21.15.67:0/604326411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb55c103000 con 0x7fb55c105510 2024-07-31T14:56:33.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.398+0000 7fb553fff640 1 -- 172.21.15.67:0/604326411 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb54c004b40 con 0x7fb55c105510 2024-07-31T14:56:33.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.398+0000 7fb553fff640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5480b3df0 0x7fb5480b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:33.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.398+0000 7fb55ad9d640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5480b3df0 0x7fb5480b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:33.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.399+0000 7fb553fff640 1 -- 172.21.15.67:0/604326411 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fb54c105570 con 0x7fb55c105510 2024-07-31T14:56:33.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.399+0000 7fb55ad9d640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5480b3df0 0x7fb5480b6300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb55c1023b0 tx=0x7fb55400b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:33.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.401+0000 7fb553fff640 1 -- 172.21.15.67:0/604326411 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb54c0cc680 con 0x7fb55c105510 2024-07-31T14:56:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:33 smithi077 ceph-mon[37327]: pgmap v564: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 1.01k op/s 2024-07-31T14:56:33.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.655+0000 7fb5610f6640 1 -- 172.21.15.67:0/604326411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb55c101c60 con 0x7fb55c105510 2024-07-31T14:56:33.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.656+0000 7fb553fff640 1 -- 172.21.15.67:0/604326411 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11986 (secure 0 0 0) 0x7fb54c0cbf80 con 0x7fb55c105510 2024-07-31T14:56:33.657 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:56:33.658 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":43,"btime":"2024-07-31T14:56:22:038870+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:56:21.264777+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T14:56:33.658 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 43 2024-07-31T14:56:33.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.658+0000 7fb5610f6640 1 -- 172.21.15.67:0/604326411 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5480b3df0 msgr2=0x7fb5480b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:33.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.658+0000 7fb5610f6640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5480b3df0 0x7fb5480b6300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb55c1023b0 tx=0x7fb55400b410 comp rx=0 tx=0).stop 2024-07-31T14:56:33.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.658+0000 7fb5610f6640 1 -- 172.21.15.67:0/604326411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb55c105510 msgr2=0x7fb55c19b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:33.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.658+0000 7fb5610f6640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb55c105510 0x7fb55c19b720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb54c005e30 tx=0x7fb54c0317c0 comp rx=0 tx=0).stop 2024-07-31T14:56:33.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.658+0000 7fb5610f6640 1 -- 172.21.15.67:0/604326411 shutdown_connections 2024-07-31T14:56:33.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.658+0000 7fb5610f6640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5480b3df0 0x7fb5480b6300 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:33.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.658+0000 7fb5610f6640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb55c105510 0x7fb55c19b720 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:33.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.658+0000 7fb5610f6640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb55c104bc0 0x7fb55c19b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:33.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.658+0000 7fb5610f6640 1 --2- 172.21.15.67:0/604326411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb55c102bf0 0x7fb55c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:33.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.658+0000 7fb5610f6640 1 -- 172.21.15.67:0/604326411 >> 172.21.15.67:0/604326411 conn(0x7fb55c077690 msgr2=0x7fb55c106d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:33.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.658+0000 7fb5610f6640 1 -- 172.21.15.67:0/604326411 shutdown_connections 2024-07-31T14:56:33.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:33.658+0000 7fb5610f6640 1 -- 172.21.15.67:0/604326411 wait complete. 2024-07-31T14:56:34.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:34 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/604326411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:56:34.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:34 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/604326411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:56:34.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:34 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/604326411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:56:34.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T14:56:35.026 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:56:35.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:35 smithi073 ceph-mon[36714]: pgmap v565: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 1.07k op/s 2024-07-31T14:56:35.425 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:35 smithi067 ceph-mon[32750]: pgmap v565: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 1.07k op/s 2024-07-31T14:56:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:35 smithi077 ceph-mon[37327]: pgmap v565: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 1.07k op/s 2024-07-31T14:56:36.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.813+0000 7fd640c9b640 1 -- 172.21.15.67:0/4117687945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd63c108720 msgr2=0x7fd63c10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:36.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.813+0000 7fd640c9b640 1 --2- 172.21.15.67:0/4117687945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd63c108720 0x7fd63c10ef50 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd634009ca0 tx=0x7fd63402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.813+0000 7fd640c9b640 1 -- 172.21.15.67:0/4117687945 shutdown_connections 2024-07-31T14:56:36.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.813+0000 7fd640c9b640 1 --2- 172.21.15.67:0/4117687945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd63c108720 0x7fd63c10ef50 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.813+0000 7fd640c9b640 1 --2- 172.21.15.67:0/4117687945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c107dd0 0x7fd63c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.813+0000 7fd640c9b640 1 --2- 172.21.15.67:0/4117687945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c102490 0x7fd63c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.813+0000 7fd640c9b640 1 -- 172.21.15.67:0/4117687945 >> 172.21.15.67:0/4117687945 conn(0x7fd63c079160 msgr2=0x7fd63c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:36.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.813+0000 7fd640c9b640 1 -- 172.21.15.67:0/4117687945 shutdown_connections 2024-07-31T14:56:36.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.813+0000 7fd640c9b640 1 -- 172.21.15.67:0/4117687945 wait complete. 2024-07-31T14:56:36.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.815+0000 7fd640c9b640 1 Processor -- start 2024-07-31T14:56:36.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.815+0000 7fd640c9b640 1 -- start start 2024-07-31T14:56:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.815+0000 7fd640c9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c102490 0x7fd63c19c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.815+0000 7fd640c9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd63c107dd0 0x7fd63c19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.815+0000 7fd640c9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c108720 0x7fd63c1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.815+0000 7fd640c9b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1123a0 con 0x7fd63c107dd0 2024-07-31T14:56:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.815+0000 7fd640c9b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c112220 con 0x7fd63c102490 2024-07-31T14:56:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.815+0000 7fd640c9b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c112520 con 0x7fd63c108720 2024-07-31T14:56:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.815+0000 7fd639d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd63c107dd0 0x7fd63c19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.815+0000 7fd63a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c102490 0x7fd63c19c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.815+0000 7fd63ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c108720 0x7fd63c1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.815+0000 7fd63a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c102490 0x7fd63c19c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51732/0 (socket says 172.21.15.67:51732) 2024-07-31T14:56:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.815+0000 7fd639d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd63c107dd0 0x7fd63c19c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46164/0 (socket says 172.21.15.67:46164) 2024-07-31T14:56:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.815+0000 7fd63a59c640 1 -- 172.21.15.67:0/2246803547 learned_addr learned my addr 172.21.15.67:0/2246803547 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.816+0000 7fd63ad9d640 1 -- 172.21.15.67:0/2246803547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c102490 msgr2=0x7fd63c19c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.816+0000 7fd63ad9d640 1 --2- 172.21.15.67:0/2246803547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c102490 0x7fd63c19c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.816+0000 7fd63ad9d640 1 -- 172.21.15.67:0/2246803547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd63c107dd0 msgr2=0x7fd63c19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.816+0000 7fd63ad9d640 1 --2- 172.21.15.67:0/2246803547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd63c107dd0 0x7fd63c19c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.816+0000 7fd63ad9d640 1 -- 172.21.15.67:0/2246803547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c1a2000 con 0x7fd63c108720 2024-07-31T14:56:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.816+0000 7fd63a59c640 1 --2- 172.21.15.67:0/2246803547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c102490 0x7fd63c19c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.816+0000 7fd63ad9d640 1 --2- 172.21.15.67:0/2246803547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c108720 0x7fd63c1a18d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd63402ade0 tx=0x7fd634031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.816+0000 7fd62b7fe640 1 -- 172.21.15.67:0/2246803547 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634003b70 con 0x7fd63c108720 2024-07-31T14:56:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.816+0000 7fd640c9b640 1 -- 172.21.15.67:0/2246803547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c1a2290 con 0x7fd63c108720 2024-07-31T14:56:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.816+0000 7fd62b7fe640 1 -- 172.21.15.67:0/2246803547 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd634031910 con 0x7fd63c108720 2024-07-31T14:56:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.816+0000 7fd62b7fe640 1 -- 172.21.15.67:0/2246803547 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63402f950 con 0x7fd63c108720 2024-07-31T14:56:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.816+0000 7fd640c9b640 1 -- 172.21.15.67:0/2246803547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c1a8a50 con 0x7fd63c108720 2024-07-31T14:56:36.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.819+0000 7fd6471ec740 1 Processor -- start 2024-07-31T14:56:36.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.819+0000 7fd62b7fe640 1 -- 172.21.15.67:0/2246803547 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd634004040 con 0x7fd63c108720 2024-07-31T14:56:36.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.820+0000 7fd6471ec740 1 -- start start 2024-07-31T14:56:36.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.820+0000 7fd6471ec740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb46be070 0x555eb4a63570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.820+0000 7fd6471ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a63ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.820+0000 7fd6471ec740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb469d390 0x555eb4a63ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.820+0000 7fd6471ec740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555eb49d9bb0 con 0x555eb469d390 2024-07-31T14:56:36.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.820+0000 7fd6471ec740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555eb49c90b0 con 0x555eb46be070 2024-07-31T14:56:36.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.820+0000 7fd6471ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555eb49cd0f0 con 0x555eb46951b0 2024-07-31T14:56:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.820+0000 7fd63a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a63ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.820+0000 7fd639d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb46be070 0x555eb4a63570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.820+0000 7fd63a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a63ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:51402/0 (socket says 172.21.15.67:51402) 2024-07-31T14:56:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.820+0000 7fd63a59c640 1 -- 172.21.15.67:0/318952054 learned_addr learned my addr 172.21.15.67:0/318952054 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.820+0000 7fd62b7fe640 1 --2- 172.21.15.67:0/2246803547 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd6240b3d40 0x7fd6240b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.820+0000 7fd639d9b640 1 --2- 172.21.15.67:0/318952054 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb469d390 0x555eb4a63ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.821+0000 7fd63ad9d640 1 --2- 172.21.15.67:0/2246803547 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd6240b3d40 0x7fd6240b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.821+0000 7fd63a59c640 1 -- 172.21.15.67:0/318952054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb46be070 msgr2=0x555eb4a63570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.821+0000 7fd63a59c640 1 --2- 172.21.15.67:0/318952054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb46be070 0x555eb4a63570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.821+0000 7fd63a59c640 1 -- 172.21.15.67:0/318952054 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb469d390 msgr2=0x555eb4a63ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.821+0000 7fd63a59c640 1 --2- 172.21.15.67:0/318952054 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb469d390 0x555eb4a63ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.821+0000 7fd63a59c640 1 -- 172.21.15.67:0/318952054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb496fb90 con 0x555eb46951b0 2024-07-31T14:56:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.821+0000 7fd639d9b640 1 --2- 172.21.15.67:0/318952054 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb469d390 0x555eb4a63ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:56:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.821+0000 7fd62b7fe640 1 -- 172.21.15.67:0/2246803547 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fd6341054d0 con 0x7fd63c108720 2024-07-31T14:56:36.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.821+0000 7fd63a59c640 1 --2- 172.21.15.67:0/318952054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a63ab0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009fd0 tx=0x7fd62c011ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14e601eebd7a332d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:36.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.821+0000 7fd628ff9640 1 -- 172.21.15.67:0/318952054 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c01e070 con 0x555eb46951b0 2024-07-31T14:56:36.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.821+0000 7fd63ad9d640 1 --2- 172.21.15.67:0/2246803547 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd6240b3d40 0x7fd6240b6250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd634102ad0 tx=0x7fd634099630 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:36.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.821+0000 7fd628ff9640 1 -- 172.21.15.67:0/318952054 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd62c00fba0 con 0x555eb46951b0 2024-07-31T14:56:36.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.821+0000 7fd628ff9640 1 -- 172.21.15.67:0/318952054 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c019c50 con 0x555eb46951b0 2024-07-31T14:56:36.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.822+0000 7fd6471ec740 1 -- 172.21.15.67:0/318952054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 msgr2=0x555eb4a63ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:36.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.822+0000 7fd6471ec740 1 --2- 172.21.15.67:0/318952054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a63ab0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009fd0 tx=0x7fd62c011ae0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.822+0000 7fd6471ec740 1 -- 172.21.15.67:0/318952054 shutdown_connections 2024-07-31T14:56:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.822+0000 7fd6471ec740 1 --2- 172.21.15.67:0/318952054 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb469d390 0x555eb4a63ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.822+0000 7fd6471ec740 1 --2- 172.21.15.67:0/318952054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a63ab0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.822+0000 7fd6471ec740 1 --2- 172.21.15.67:0/318952054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb46be070 0x555eb4a63570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.822+0000 7fd6471ec740 1 -- 172.21.15.67:0/318952054 >> 172.21.15.67:0/318952054 conn(0x555eb46b2460 msgr2=0x555eb4a61460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.822+0000 7fd6471ec740 1 -- 172.21.15.67:0/318952054 shutdown_connections 2024-07-31T14:56:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.822+0000 7fd6471ec740 1 -- 172.21.15.67:0/318952054 wait complete. 2024-07-31T14:56:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.823+0000 7fd6471ec740 1 -- start start 2024-07-31T14:56:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.823+0000 7fd6471ec740 1 Processor -- start 2024-07-31T14:56:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.823+0000 7fd6471ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a6f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.823+0000 7fd6471ec740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb469d390 0x555eb4a6f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.823+0000 7fd6471ec740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb46be070 0x555eb4a7b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.823+0000 7fd6471ec740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555eb482c700 con 0x555eb469d390 2024-07-31T14:56:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.823+0000 7fd6471ec740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555eb496b9e0 con 0x555eb46be070 2024-07-31T14:56:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.823+0000 7fd6471ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555eb4a71d50 con 0x555eb46951b0 2024-07-31T14:56:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.823+0000 7fd6471ec740 10 client.0 _ll_register_callbacks cb 0x555eb49864f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.823+0000 7fd63a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb469d390 0x555eb4a6f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.823+0000 7fd639d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb46be070 0x555eb4a7b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.823+0000 7fd63a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb469d390 0x555eb4a6f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46174/0 (socket says 172.21.15.67:46174) 2024-07-31T14:56:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.823+0000 7fd63a59c640 1 -- 172.21.15.67:0/2456183956 learned_addr learned my addr 172.21.15.67:0/2456183956 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.824+0000 7fd639d9b640 1 --2- 172.21.15.67:0/2456183956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a6f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.824+0000 7fd639d9b640 1 -- 172.21.15.67:0/2456183956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 msgr2=0x555eb4a6f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.824+0000 7fd639d9b640 1 --2- 172.21.15.67:0/2456183956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a6f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.824+0000 7fd639d9b640 1 -- 172.21.15.67:0/2456183956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb469d390 msgr2=0x555eb4a6f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.824+0000 7fd639d9b640 1 --2- 172.21.15.67:0/2456183956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb469d390 0x555eb4a6f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.824+0000 7fd639d9b640 1 -- 172.21.15.67:0/2456183956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb4729650 con 0x555eb46be070 2024-07-31T14:56:36.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.824+0000 7fd639d9b640 1 --2- 172.21.15.67:0/2456183956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb46be070 0x555eb4a7b040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x555eb4a717a0 tx=0x7fd630013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35a266721b5ead44 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:36.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.824+0000 7fd621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:56:36.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.824+0000 7fd621ffb640 1 -- 172.21.15.67:0/2456183956 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd630026060 con 0x555eb46be070 2024-07-31T14:56:36.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.824+0000 7fd6471ec740 1 -- 172.21.15.67:0/2456183956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd62c00fba0 con 0x555eb46be070 2024-07-31T14:56:36.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.824+0000 7fd621ffb640 1 -- 172.21.15.67:0/2456183956 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fd63000fba0 con 0x555eb46be070 2024-07-31T14:56:36.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.824+0000 7fd621ffb640 1 -- 172.21.15.67:0/2456183956 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd63000fda0 con 0x555eb46be070 2024-07-31T14:56:36.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.824+0000 7fd6471ec740 10 client.26597 fetch_fsmap learned FSMap version 43 2024-07-31T14:56:36.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.824+0000 7fd6471ec740 1 -- 172.21.15.67:0/2456183956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555eb4a7de40 con 0x555eb46be070 2024-07-31T14:56:36.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.825+0000 7fd621ffb640 1 -- 172.21.15.67:0/2456183956 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd62c00fba0 con 0x555eb46be070 2024-07-31T14:56:36.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.825+0000 7fd621ffb640 1 -- 172.21.15.67:0/2456183956 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) ==== 8434+0+0 (secure 0 0 0) 0x7fd630004b00 con 0x555eb46be070 2024-07-31T14:56:36.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.825+0000 7fd6471ec740 10 client.26597 fetch_fsmap finished waiting for FSMap version 43 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.825+0000 7fd6471ec740 10 client.26597 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.825+0000 7fd6471ec740 20 client.26597 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.825+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2456183956 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x555eb467c9f0 0x555eb4a81320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.825+0000 7fd6471ec740 4 client.26597 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.825+0000 7fd6471ec740 1 -- 172.21.15.67:0/2456183956 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eb49dbb00 con 0x555eb467c9f0 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.825+0000 7fd63a59c640 1 --2- 172.21.15.67:0/2456183956 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x555eb467c9f0 0x555eb4a81320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.826+0000 7fd621ffb640 10 client.26597 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.826+0000 7fd63a59c640 1 --2- 172.21.15.67:0/2456183956 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x555eb467c9f0 0x555eb4a81320 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7387e5056f7bc2d7 server_cookie=85639c1887b00a3 in_seq=0 out_seq=0 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.826+0000 7fd621ffb640 1 -- 172.21.15.67:0/2456183956 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555eb49dbb00 con 0x555eb467c9f0 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.826+0000 7fd621ffb640 10 client.26597 handle_command_reply: tid=0 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.826+0000 7fd6471ec740 1 client.26597 shutdown 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.827+0000 7fd6471ec740 1 -- 172.21.15.67:0/2456183956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb46be070 msgr2=0x555eb4a7b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.827+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2456183956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb46be070 0x555eb4a7b040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x555eb4a717a0 tx=0x7fd630013040 comp rx=0 tx=0).stop 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.827+0000 7fd6471ec740 1 -- 172.21.15.67:0/2456183956 shutdown_connections 2024-07-31T14:56:36.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.827+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2456183956 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x555eb467c9f0 0x555eb4a81320 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.827+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2456183956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb46be070 0x555eb4a7b040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.827+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2456183956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb469d390 0x555eb4a6f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.827+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2456183956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a6f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.827+0000 7fd621ffb640 0 client.26597 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:56:36.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.827+0000 7fd6471ec740 1 -- 172.21.15.67:0/2456183956 >> 172.21.15.67:0/2456183956 conn(0x555eb46b2460 msgr2=0x555eb4a5fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:36.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.827+0000 7fd6471ec740 1 -- 172.21.15.67:0/2456183956 shutdown_connections 2024-07-31T14:56:36.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.827+0000 7fd6471ec740 1 -- 172.21.15.67:0/2456183956 wait complete. 2024-07-31T14:56:36.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.827+0000 7fd6471ec740 20 client.26597 trim_cache size 0 max 16384 2024-07-31T14:56:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd6471ec740 1 Processor -- start 2024-07-31T14:56:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd6471ec740 1 -- start start 2024-07-31T14:56:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd6471ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a68d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd6471ec740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb469d390 0x555eb4a7ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd6471ec740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb46be070 0x555eb4a7b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd6471ec740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1b2900 con 0x555eb46be070 2024-07-31T14:56:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd6471ec740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1b4650 con 0x555eb469d390 2024-07-31T14:56:36.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd6471ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1b3060 con 0x555eb46951b0 2024-07-31T14:56:36.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd639d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb46be070 0x555eb4a7b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd639d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb46be070 0x555eb4a7b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46178/0 (socket says 172.21.15.67:46178) 2024-07-31T14:56:36.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd639d9b640 1 -- 172.21.15.67:0/2479085330 learned_addr learned my addr 172.21.15.67:0/2479085330 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:36.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd63a59c640 1 --2- 172.21.15.67:0/2479085330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb469d390 0x555eb4a7ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd639d9b640 1 --2- 172.21.15.67:0/2479085330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a68d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd639d9b640 1 -- 172.21.15.67:0/2479085330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 msgr2=0x555eb4a68d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:36.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd639d9b640 1 --2- 172.21.15.67:0/2479085330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a68d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd639d9b640 1 -- 172.21.15.67:0/2479085330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb469d390 msgr2=0x555eb4a7ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:36.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd639d9b640 1 --2- 172.21.15.67:0/2479085330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb469d390 0x555eb4a7ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.841+0000 7fd639d9b640 1 -- 172.21.15.67:0/2479085330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb4a7dc50 con 0x555eb46be070 2024-07-31T14:56:36.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.842+0000 7fd639d9b640 1 --2- 172.21.15.67:0/2479085330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb46be070 0x555eb4a7b370 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd630011740 tx=0x7fd630009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788e5d86b304e1ee server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:36.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.842+0000 7fd623fff640 1 -- 172.21.15.67:0/2479085330 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6300046c0 con 0x555eb46be070 2024-07-31T14:56:36.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.842+0000 7fd623fff640 1 -- 172.21.15.67:0/2479085330 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd630004fd0 con 0x555eb46be070 2024-07-31T14:56:36.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.842+0000 7fd623fff640 1 -- 172.21.15.67:0/2479085330 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6300216b0 con 0x555eb46be070 2024-07-31T14:56:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.843+0000 7fd6471ec740 1 -- 172.21.15.67:0/2479085330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb46be070 msgr2=0x555eb4a7b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.843+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2479085330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb46be070 0x555eb4a7b370 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd630011740 tx=0x7fd630009b40 comp rx=0 tx=0).stop 2024-07-31T14:56:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.843+0000 7fd6471ec740 1 -- 172.21.15.67:0/2479085330 shutdown_connections 2024-07-31T14:56:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.843+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2479085330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb46be070 0x555eb4a7b370 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.843+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2479085330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb469d390 0x555eb4a7ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.843+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2479085330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a68d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.843+0000 7fd6471ec740 1 -- 172.21.15.67:0/2479085330 >> 172.21.15.67:0/2479085330 conn(0x555eb46b2460 msgr2=0x555eb4a68680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.843+0000 7fd6471ec740 1 -- 172.21.15.67:0/2479085330 shutdown_connections 2024-07-31T14:56:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.843+0000 7fd6471ec740 1 -- 172.21.15.67:0/2479085330 wait complete. 2024-07-31T14:56:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.843+0000 7fd6471ec740 1 -- start start 2024-07-31T14:56:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.843+0000 7fd6471ec740 1 Processor -- start 2024-07-31T14:56:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd6471ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a73200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd6471ec740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb469d390 0x555eb4a73740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd6471ec740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb46be070 0x555eb4ae8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd6471ec740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1b3f80 con 0x555eb46be070 2024-07-31T14:56:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd6471ec740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1b37f0 con 0x555eb469d390 2024-07-31T14:56:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd6471ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1b08e0 con 0x555eb46951b0 2024-07-31T14:56:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd6471ec740 10 client.0 _ll_register_callbacks cb 0x7fd630009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd639d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb46be070 0x555eb4ae8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd63a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb469d390 0x555eb4a73740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd639d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb46be070 0x555eb4ae8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46194/0 (socket says 172.21.15.67:46194) 2024-07-31T14:56:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd639d9b640 1 -- 172.21.15.67:0/2029706552 learned_addr learned my addr 172.21.15.67:0/2029706552 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd639d9b640 1 --2- 172.21.15.67:0/2029706552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a73200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd63a59c640 1 -- 172.21.15.67:0/2029706552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 msgr2=0x555eb4a73200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd63a59c640 1 --2- 172.21.15.67:0/2029706552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a73200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd63a59c640 1 -- 172.21.15.67:0/2029706552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb46be070 msgr2=0x555eb4ae8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd63a59c640 1 --2- 172.21.15.67:0/2029706552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb46be070 0x555eb4ae8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd63a59c640 1 -- 172.21.15.67:0/2029706552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb4ae8ff0 con 0x555eb469d390 2024-07-31T14:56:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd639d9b640 1 --2- 172.21.15.67:0/2029706552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb46be070 0x555eb4ae8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd63a59c640 1 --2- 172.21.15.67:0/2029706552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb469d390 0x555eb4a73740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x555eb4a81a50 tx=0x7fd62c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14b0359ea65ff230 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.844+0000 7fd621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:56:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.845+0000 7fd621ffb640 1 -- 172.21.15.67:0/2029706552 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c01e040 con 0x555eb469d390 2024-07-31T14:56:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.845+0000 7fd621ffb640 1 -- 172.21.15.67:0/2029706552 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fd62c019490 con 0x555eb469d390 2024-07-31T14:56:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.845+0000 7fd6471ec740 1 -- 172.21.15.67:0/2029706552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd630004fd0 con 0x555eb469d390 2024-07-31T14:56:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.845+0000 7fd621ffb640 1 -- 172.21.15.67:0/2029706552 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd62c00f6f0 con 0x555eb469d390 2024-07-31T14:56:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.845+0000 7fd6471ec740 10 client.26603 fetch_fsmap learned FSMap version 43 2024-07-31T14:56:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.845+0000 7fd6471ec740 1 -- 172.21.15.67:0/2029706552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555eb4aea0c0 con 0x555eb469d390 2024-07-31T14:56:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.845+0000 7fd621ffb640 1 -- 172.21.15.67:0/2029706552 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd630004fd0 con 0x555eb469d390 2024-07-31T14:56:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.845+0000 7fd621ffb640 1 -- 172.21.15.67:0/2029706552 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) ==== 8434+0+0 (secure 0 0 0) 0x7fd62c005040 con 0x555eb469d390 2024-07-31T14:56:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.845+0000 7fd6471ec740 10 client.26603 fetch_fsmap finished waiting for FSMap version 43 2024-07-31T14:56:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.845+0000 7fd6471ec740 10 client.26603 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:56:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.845+0000 7fd6471ec740 20 client.26603 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.845+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2029706552 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x555eb4651da0 0x555eb4aec6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.845+0000 7fd6471ec740 4 client.26603 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:56:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.845+0000 7fd6471ec740 1 -- 172.21.15.67:0/2029706552 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555eb49d75a0 con 0x555eb4651da0 2024-07-31T14:56:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.846+0000 7fd639d9b640 1 --2- 172.21.15.67:0/2029706552 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x555eb4651da0 0x555eb4aec6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.846+0000 7fd639d9b640 1 --2- 172.21.15.67:0/2029706552 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x555eb4651da0 0x555eb4aec6c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7cd7eb1a93ba8f1c server_cookie=fdfef4bae2f2042 in_seq=0 out_seq=0 2024-07-31T14:56:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.846+0000 7fd621ffb640 10 client.26603 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:56:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.846+0000 7fd621ffb640 1 -- 172.21.15.67:0/2029706552 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555eb49d75a0 con 0x555eb4651da0 2024-07-31T14:56:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.846+0000 7fd621ffb640 10 client.26603 handle_command_reply: tid=0 2024-07-31T14:56:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.846+0000 7fd6471ec740 1 client.26603 shutdown 2024-07-31T14:56:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.846+0000 7fd6471ec740 1 -- 172.21.15.67:0/2029706552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb469d390 msgr2=0x555eb4a73740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.846+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2029706552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb469d390 0x555eb4a73740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x555eb4a81a50 tx=0x7fd62c009500 comp rx=0 tx=0).stop 2024-07-31T14:56:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd6471ec740 1 -- 172.21.15.67:0/2029706552 shutdown_connections 2024-07-31T14:56:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2029706552 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x555eb4651da0 0x555eb4aec6c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2029706552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555eb46be070 0x555eb4ae8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2029706552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eb469d390 0x555eb4a73740 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd621ffb640 0 client.26603 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:56:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd6471ec740 1 --2- 172.21.15.67:0/2029706552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555eb46951b0 0x555eb4a73200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd6471ec740 1 -- 172.21.15.67:0/2029706552 >> 172.21.15.67:0/2029706552 conn(0x555eb46b2460 msgr2=0x555eb4a60b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd6471ec740 1 -- 172.21.15.67:0/2029706552 shutdown_connections 2024-07-31T14:56:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd6471ec740 1 -- 172.21.15.67:0/2029706552 wait complete. 2024-07-31T14:56:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd6471ec740 20 client.26603 trim_cache size 0 max 16384 2024-07-31T14:56:36.854 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:56:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd640c9b640 1 -- 172.21.15.67:0/2246803547 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd6240b3d40 msgr2=0x7fd6240b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd640c9b640 1 --2- 172.21.15.67:0/2246803547 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd6240b3d40 0x7fd6240b6250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd634102ad0 tx=0x7fd634099630 comp rx=0 tx=0).stop 2024-07-31T14:56:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd640c9b640 1 -- 172.21.15.67:0/2246803547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c108720 msgr2=0x7fd63c1a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd640c9b640 1 --2- 172.21.15.67:0/2246803547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c108720 0x7fd63c1a18d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd63402ade0 tx=0x7fd634031660 comp rx=0 tx=0).stop 2024-07-31T14:56:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd640c9b640 1 -- 172.21.15.67:0/2246803547 shutdown_connections 2024-07-31T14:56:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd640c9b640 1 --2- 172.21.15.67:0/2246803547 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd6240b3d40 0x7fd6240b6250 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd640c9b640 1 --2- 172.21.15.67:0/2246803547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c108720 0x7fd63c1a18d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd640c9b640 1 --2- 172.21.15.67:0/2246803547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd63c107dd0 0x7fd63c19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd640c9b640 1 --2- 172.21.15.67:0/2246803547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c102490 0x7fd63c19c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd640c9b640 1 -- 172.21.15.67:0/2246803547 >> 172.21.15.67:0/2246803547 conn(0x7fd63c079160 msgr2=0x7fd63c104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd640c9b640 1 -- 172.21.15.67:0/2246803547 shutdown_connections 2024-07-31T14:56:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:36.847+0000 7fd640c9b640 1 -- 172.21.15.67:0/2246803547 wait complete. 2024-07-31T14:56:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.156+0000 7f1cc7ce8640 1 -- 172.21.15.67:0/3553182667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0107d20 msgr2=0x7f1cc0108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.156+0000 7f1cc7ce8640 1 --2- 172.21.15.67:0/3553182667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0107d20 0x7f1cc0108130 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009900 tx=0x7f1cb002a410 comp rx=0 tx=0).stop 2024-07-31T14:56:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.156+0000 7f1cc7ce8640 1 -- 172.21.15.67:0/3553182667 shutdown_connections 2024-07-31T14:56:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.156+0000 7f1cc7ce8640 1 --2- 172.21.15.67:0/3553182667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc0108670 0x7f1cc010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.156+0000 7f1cc7ce8640 1 --2- 172.21.15.67:0/3553182667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0107d20 0x7f1cc0108130 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.156+0000 7f1cc7ce8640 1 --2- 172.21.15.67:0/3553182667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc01023f0 0x7f1cc0102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.156+0000 7f1cc7ce8640 1 -- 172.21.15.67:0/3553182667 >> 172.21.15.67:0/3553182667 conn(0x7f1cc00fddf0 msgr2=0x7f1cc0100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.157+0000 7f1cc7ce8640 1 -- 172.21.15.67:0/3553182667 shutdown_connections 2024-07-31T14:56:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.157+0000 7f1cc7ce8640 1 -- 172.21.15.67:0/3553182667 wait complete. 2024-07-31T14:56:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.157+0000 7f1cc7ce8640 1 Processor -- start 2024-07-31T14:56:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.157+0000 7f1cc7ce8640 1 -- start start 2024-07-31T14:56:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.157+0000 7f1cc7ce8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc01023f0 0x7f1cc0199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.157+0000 7f1cc7ce8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0107d20 0x7f1cc019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.157+0000 7f1cc7ce8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc0108670 0x7f1cc019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc7ce8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01122e0 con 0x7f1cc0107d20 2024-07-31T14:56:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc7ce8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0112160 con 0x7f1cc0108670 2024-07-31T14:56:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc7ce8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0112460 con 0x7f1cc01023f0 2024-07-31T14:56:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc5283640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0107d20 0x7f1cc019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc5283640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0107d20 0x7f1cc019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46200/0 (socket says 172.21.15.67:46200) 2024-07-31T14:56:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc5283640 1 -- 172.21.15.67:0/3446032476 learned_addr learned my addr 172.21.15.67:0/3446032476 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc6285640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc0108670 0x7f1cc019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc5a84640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc01023f0 0x7f1cc0199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc5283640 1 -- 172.21.15.67:0/3446032476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc01023f0 msgr2=0x7f1cc0199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc5283640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc01023f0 0x7f1cc0199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc5283640 1 -- 172.21.15.67:0/3446032476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc0108670 msgr2=0x7f1cc019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc5283640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc0108670 0x7f1cc019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc5283640 1 -- 172.21.15.67:0/3446032476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc019fd40 con 0x7f1cc0107d20 2024-07-31T14:56:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc6285640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc0108670 0x7f1cc019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc5283640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0107d20 0x7f1cc019a3b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006010 tx=0x7f1cb0004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cb6ffd640 1 -- 172.21.15.67:0/3446032476 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb003f030 con 0x7f1cc0107d20 2024-07-31T14:56:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc7ce8640 1 -- 172.21.15.67:0/3446032476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc019ffd0 con 0x7f1cc0107d20 2024-07-31T14:56:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cb6ffd640 1 -- 172.21.15.67:0/3446032476 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb0003aa0 con 0x7f1cc0107d20 2024-07-31T14:56:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cb6ffd640 1 -- 172.21.15.67:0/3446032476 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0035070 con 0x7f1cc0107d20 2024-07-31T14:56:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.158+0000 7f1cc7ce8640 1 -- 172.21.15.67:0/3446032476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc01a67c0 con 0x7f1cc0107d20 2024-07-31T14:56:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.161+0000 7f1cb6ffd640 1 -- 172.21.15.67:0/3446032476 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1cb0042030 con 0x7f1cc0107d20 2024-07-31T14:56:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.162+0000 7f1ccebde740 1 Processor -- start 2024-07-31T14:56:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.162+0000 7f1ccebde740 1 -- start start 2024-07-31T14:56:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.162+0000 7f1cb6ffd640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1cac0b3d40 0x7f1cac0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.162+0000 7f1ccebde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e890851b0 0x557e89427ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.162+0000 7f1ccebde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e8905c2f0 0x557e89428220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.162+0000 7f1ccebde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 0x557e89428760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.162+0000 7f1ccebde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557e8939d3b0 con 0x557e8905c2f0 2024-07-31T14:56:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.162+0000 7f1ccebde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557e8938c8b0 con 0x557e890851b0 2024-07-31T14:56:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.162+0000 7f1ccebde740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557e893908f0 con 0x557e890644d0 2024-07-31T14:56:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cc6285640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1cac0b3d40 0x7f1cac0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cc5283640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e8905c2f0 0x557e89428220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cc5a84640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e890851b0 0x557e89427ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cc5a84640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e890851b0 0x557e89427ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51796/0 (socket says 172.21.15.67:51796) 2024-07-31T14:56:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cc5a84640 1 -- 172.21.15.67:0/1501813410 learned_addr learned my addr 172.21.15.67:0/1501813410 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cb6ffd640 1 -- 172.21.15.67:0/3446032476 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f1cb0104830 con 0x7f1cc0107d20 2024-07-31T14:56:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cc5a84640 1 -- 172.21.15.67:0/1501813410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 msgr2=0x557e89428760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cc6285640 1 --2- 172.21.15.67:0/1501813410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 0x557e89428760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cc5a84640 1 --2- 172.21.15.67:0/1501813410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 0x557e89428760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cc5a84640 1 -- 172.21.15.67:0/1501813410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e8905c2f0 msgr2=0x557e89428220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cc5a84640 1 --2- 172.21.15.67:0/1501813410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e8905c2f0 0x557e89428220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cc5a84640 1 -- 172.21.15.67:0/1501813410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e893339a0 con 0x557e890851b0 2024-07-31T14:56:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cc6285640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1cac0b3d40 0x7f1cac0b6250 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80098b0 tx=0x7f1cb800ebf0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cc5a84640 1 --2- 172.21.15.67:0/1501813410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e890851b0 0x557e89427ce0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009cb0 tx=0x7f1cbc0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53d6761d7a70999f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cabfff640 1 -- 172.21.15.67:0/1501813410 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc012450 con 0x557e890851b0 2024-07-31T14:56:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cabfff640 1 -- 172.21.15.67:0/1501813410 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cbc016070 con 0x557e890851b0 2024-07-31T14:56:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.163+0000 7f1cabfff640 1 -- 172.21.15.67:0/1501813410 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc01a8a0 con 0x557e890851b0 2024-07-31T14:56:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.164+0000 7f1ccebde740 1 -- 172.21.15.67:0/1501813410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e890851b0 msgr2=0x557e89427ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.164+0000 7f1ccebde740 1 --2- 172.21.15.67:0/1501813410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e890851b0 0x557e89427ce0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009cb0 tx=0x7f1cbc0117b0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.164+0000 7f1ccebde740 1 -- 172.21.15.67:0/1501813410 shutdown_connections 2024-07-31T14:56:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.164+0000 7f1ccebde740 1 --2- 172.21.15.67:0/1501813410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 0x557e89428760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.164+0000 7f1ccebde740 1 --2- 172.21.15.67:0/1501813410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e8905c2f0 0x557e89428220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.164+0000 7f1ccebde740 1 --2- 172.21.15.67:0/1501813410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e890851b0 0x557e89427ce0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.164+0000 7f1ccebde740 1 -- 172.21.15.67:0/1501813410 >> 172.21.15.67:0/1501813410 conn(0x557e89079300 msgr2=0x557e89425bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.164+0000 7f1ccebde740 1 -- 172.21.15.67:0/1501813410 shutdown_connections 2024-07-31T14:56:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.164+0000 7f1ccebde740 1 -- 172.21.15.67:0/1501813410 wait complete. 2024-07-31T14:56:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.165+0000 7f1ccebde740 1 -- start start 2024-07-31T14:56:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.165+0000 7f1ccebde740 1 Processor -- start 2024-07-31T14:56:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.165+0000 7f1ccebde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8905c2f0 0x557e894337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.165+0000 7f1ccebde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 0x557e89433d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.165+0000 7f1ccebde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e8943f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.165+0000 7f1ccebde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557e892c6ab0 con 0x557e890851b0 2024-07-31T14:56:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.165+0000 7f1ccebde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557e89314cf0 con 0x557e8905c2f0 2024-07-31T14:56:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.165+0000 7f1ccebde740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557e89436460 con 0x557e890644d0 2024-07-31T14:56:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.165+0000 7f1ccebde740 10 client.0 _ll_register_callbacks cb 0x557e893869f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.165+0000 7f1cc6285640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e8943f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1cc6285640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e8943f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46208/0 (socket says 172.21.15.67:46208) 2024-07-31T14:56:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1cc6285640 1 -- 172.21.15.67:0/2637567321 learned_addr learned my addr 172.21.15.67:0/2637567321 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1cc5a84640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8905c2f0 0x557e894337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1cc5283640 1 --2- 172.21.15.67:0/2637567321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 0x557e89433d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1cc6285640 1 -- 172.21.15.67:0/2637567321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 msgr2=0x557e89433d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1cc6285640 1 --2- 172.21.15.67:0/2637567321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 0x557e89433d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1cc6285640 1 -- 172.21.15.67:0/2637567321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8905c2f0 msgr2=0x557e894337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1cc6285640 1 --2- 172.21.15.67:0/2637567321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8905c2f0 0x557e894337d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1cc6285640 1 -- 172.21.15.67:0/2637567321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e890f0790 con 0x557e890851b0 2024-07-31T14:56:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1cc5a84640 1 --2- 172.21.15.67:0/2637567321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8905c2f0 0x557e894337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1cc5283640 1 --2- 172.21.15.67:0/2637567321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 0x557e89433d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:56:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1cc6285640 1 --2- 172.21.15.67:0/2637567321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e8943f7b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80125f0 tx=0x7f1cb8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd478bf99352d7bc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1ca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:56:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1ca97fa640 1 -- 172.21.15.67:0/2637567321 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8024030 con 0x557e890851b0 2024-07-31T14:56:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1ca97fa640 1 -- 172.21.15.67:0/2637567321 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f1cb801f700 con 0x557e890851b0 2024-07-31T14:56:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1ccebde740 1 -- 172.21.15.67:0/2637567321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cbc016070 con 0x557e890851b0 2024-07-31T14:56:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1ca97fa640 1 -- 172.21.15.67:0/2637567321 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1cb8031050 con 0x557e890851b0 2024-07-31T14:56:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.166+0000 7f1ccebde740 10 client.17031 fetch_fsmap learned FSMap version 43 2024-07-31T14:56:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.167+0000 7f1ccebde740 1 -- 172.21.15.67:0/2637567321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557e89169620 con 0x557e890851b0 2024-07-31T14:56:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.167+0000 7f1ca97fa640 1 -- 172.21.15.67:0/2637567321 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cbc016070 con 0x557e890851b0 2024-07-31T14:56:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.167+0000 7f1ca97fa640 1 -- 172.21.15.67:0/2637567321 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 43) ==== 8434+0+0 (secure 0 0 0) 0x7f1cb800f040 con 0x557e890851b0 2024-07-31T14:56:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.167+0000 7f1ccebde740 10 client.17031 fetch_fsmap finished waiting for FSMap version 43 2024-07-31T14:56:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.167+0000 7f1ccebde740 10 client.17031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:56:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.167+0000 7f1ccebde740 20 client.17031 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.167+0000 7f1ccebde740 1 --2- 172.21.15.67:0/2637567321 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e89043d80 0x557e894458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.167+0000 7f1ccebde740 4 client.17031 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:56:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.167+0000 7f1ccebde740 1 -- 172.21.15.67:0/2637567321 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e8939f300 con 0x557e89043d80 2024-07-31T14:56:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.167+0000 7f1cc5a84640 1 --2- 172.21.15.67:0/2637567321 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e89043d80 0x557e894458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.168+0000 7f1ca97fa640 10 client.17031 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.168+0000 7f1cc5a84640 1 --2- 172.21.15.67:0/2637567321 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e89043d80 0x557e894458f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293b82cb5f229eeb server_cookie=4f0cd89eec9152a5 in_seq=0 out_seq=0 2024-07-31T14:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.168+0000 7f1ca97fa640 1 -- 172.21.15.67:0/2637567321 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e8939f300 con 0x557e89043d80 2024-07-31T14:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.168+0000 7f1ca97fa640 10 client.17031 handle_command_reply: tid=0 2024-07-31T14:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.168+0000 7f1ccebde740 1 client.17031 shutdown 2024-07-31T14:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.168+0000 7f1ccebde740 1 -- 172.21.15.67:0/2637567321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 msgr2=0x557e8943f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.168+0000 7f1ccebde740 1 --2- 172.21.15.67:0/2637567321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e8943f7b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80125f0 tx=0x7f1cb8004ab0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.168+0000 7f1ccebde740 1 -- 172.21.15.67:0/2637567321 shutdown_connections 2024-07-31T14:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.168+0000 7f1ccebde740 1 --2- 172.21.15.67:0/2637567321 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e89043d80 0x557e894458f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.168+0000 7f1ccebde740 1 --2- 172.21.15.67:0/2637567321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e8943f7b0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.168+0000 7f1ccebde740 1 --2- 172.21.15.67:0/2637567321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 0x557e89433d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.168+0000 7f1ccebde740 1 --2- 172.21.15.67:0/2637567321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8905c2f0 0x557e894337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.168+0000 7f1ca97fa640 0 client.17031 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.168+0000 7f1ccebde740 1 -- 172.21.15.67:0/2637567321 >> 172.21.15.67:0/2637567321 conn(0x557e89079300 msgr2=0x557e89428d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.169+0000 7f1ccebde740 1 -- 172.21.15.67:0/2637567321 shutdown_connections 2024-07-31T14:56:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.169+0000 7f1ccebde740 1 -- 172.21.15.67:0/2637567321 wait complete. 2024-07-31T14:56:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.169+0000 7f1ccebde740 20 client.17031 trim_cache size 0 max 16384 2024-07-31T14:56:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.185+0000 7f1ccebde740 1 Processor -- start 2024-07-31T14:56:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1ccebde740 1 -- start start 2024-07-31T14:56:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1ccebde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e8905c2f0 0x557e894354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1ccebde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e890644d0 0x557e89432d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1ccebde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e894249c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1ccebde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01b0590 con 0x557e890851b0 2024-07-31T14:56:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1ccebde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01b22e0 con 0x557e890644d0 2024-07-31T14:56:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1ccebde740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01b0cf0 con 0x557e8905c2f0 2024-07-31T14:56:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1cc6285640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e894249c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1cc6285640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e894249c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46216/0 (socket says 172.21.15.67:46216) 2024-07-31T14:56:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1cc6285640 1 -- 172.21.15.67:0/3997892358 learned_addr learned my addr 172.21.15.67:0/3997892358 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1cc5283640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e890644d0 0x557e89432d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1cc5a84640 1 --2- 172.21.15.67:0/3997892358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e8905c2f0 0x557e894354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1cc5283640 1 -- 172.21.15.67:0/3997892358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e8905c2f0 msgr2=0x557e894354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1cc5283640 1 --2- 172.21.15.67:0/3997892358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e8905c2f0 0x557e894354a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1cc5283640 1 -- 172.21.15.67:0/3997892358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 msgr2=0x557e894249c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1cc5283640 1 --2- 172.21.15.67:0/3997892358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e894249c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1cc5283640 1 -- 172.21.15.67:0/3997892358 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e89431aa0 con 0x557e890644d0 2024-07-31T14:56:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.186+0000 7f1cc6285640 1 --2- 172.21.15.67:0/3997892358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e894249c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.187+0000 7f1cc5283640 1 --2- 172.21.15.67:0/3997892358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e890644d0 0x557e89432d00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0101c70 tx=0x7f1cb003b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc3f6cfbdf1e49b9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.187+0000 7f1cab7fe640 1 -- 172.21.15.67:0/3997892358 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0049040 con 0x557e890644d0 2024-07-31T14:56:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.187+0000 7f1cab7fe640 1 -- 172.21.15.67:0/3997892358 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb00cac40 con 0x557e890644d0 2024-07-31T14:56:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.187+0000 7f1cab7fe640 1 -- 172.21.15.67:0/3997892358 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00cf730 con 0x557e890644d0 2024-07-31T14:56:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.188+0000 7f1ccebde740 1 -- 172.21.15.67:0/3997892358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e890644d0 msgr2=0x557e89432d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.188+0000 7f1ccebde740 1 --2- 172.21.15.67:0/3997892358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e890644d0 0x557e89432d00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0101c70 tx=0x7f1cb003b5d0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.188+0000 7f1ccebde740 1 -- 172.21.15.67:0/3997892358 shutdown_connections 2024-07-31T14:56:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.188+0000 7f1ccebde740 1 --2- 172.21.15.67:0/3997892358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e894249c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.188+0000 7f1ccebde740 1 --2- 172.21.15.67:0/3997892358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e890644d0 0x557e89432d00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.188+0000 7f1ccebde740 1 --2- 172.21.15.67:0/3997892358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e8905c2f0 0x557e894354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.188+0000 7f1ccebde740 1 -- 172.21.15.67:0/3997892358 >> 172.21.15.67:0/3997892358 conn(0x557e89079300 msgr2=0x557e894242b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.188+0000 7f1ccebde740 1 -- 172.21.15.67:0/3997892358 shutdown_connections 2024-07-31T14:56:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.188+0000 7f1ccebde740 1 -- 172.21.15.67:0/3997892358 wait complete. 2024-07-31T14:56:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.188+0000 7f1ccebde740 1 -- start start 2024-07-31T14:56:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.188+0000 7f1ccebde740 1 Processor -- start 2024-07-31T14:56:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1ccebde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8905c2f0 0x557e8943ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1ccebde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 0x557e8943c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1ccebde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e89438870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1ccebde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01b1c10 con 0x557e890851b0 2024-07-31T14:56:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1ccebde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01b1480 con 0x557e8905c2f0 2024-07-31T14:56:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1ccebde740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01ae570 con 0x557e890644d0 2024-07-31T14:56:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1ccebde740 10 client.0 _ll_register_callbacks cb 0x7f1cb800ee40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1cc6285640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e89438870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1cc5a84640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8905c2f0 0x557e8943ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1cc6285640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e89438870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46220/0 (socket says 172.21.15.67:46220) 2024-07-31T14:56:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1cc6285640 1 -- 172.21.15.67:0/382179022 learned_addr learned my addr 172.21.15.67:0/382179022 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1cc5283640 1 --2- 172.21.15.67:0/382179022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 0x557e8943c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1cc6285640 1 -- 172.21.15.67:0/382179022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 msgr2=0x557e8943c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1cc6285640 1 --2- 172.21.15.67:0/382179022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 0x557e8943c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1cc6285640 1 -- 172.21.15.67:0/382179022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8905c2f0 msgr2=0x557e8943ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1cc6285640 1 --2- 172.21.15.67:0/382179022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8905c2f0 0x557e8943ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1cc6285640 1 -- 172.21.15.67:0/382179022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e89439000 con 0x557e890851b0 2024-07-31T14:56:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.189+0000 7f1cc5a84640 1 --2- 172.21.15.67:0/382179022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8905c2f0 0x557e8943ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1cc6285640 1 --2- 172.21.15.67:0/382179022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e89438870 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80209d0 tx=0x7f1cb8020e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fcaee54289e949a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:56:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ca97fa640 1 -- 172.21.15.67:0/382179022 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb803a060 con 0x557e890851b0 2024-07-31T14:56:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ccebde740 1 -- 172.21.15.67:0/382179022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb00cac40 con 0x557e890851b0 2024-07-31T14:56:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ca97fa640 1 -- 172.21.15.67:0/382179022 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f1cb801f900 con 0x557e890851b0 2024-07-31T14:56:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ca97fa640 1 -- 172.21.15.67:0/382179022 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1cb8041050 con 0x557e890851b0 2024-07-31T14:56:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ccebde740 10 client.17037 fetch_fsmap learned FSMap version 43 2024-07-31T14:56:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ccebde740 1 -- 172.21.15.67:0/382179022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557e894acc70 con 0x557e890851b0 2024-07-31T14:56:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ca97fa640 1 -- 172.21.15.67:0/382179022 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb00cac40 con 0x557e890851b0 2024-07-31T14:56:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ca97fa640 1 -- 172.21.15.67:0/382179022 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 43) ==== 8434+0+0 (secure 0 0 0) 0x7f1cb8020610 con 0x557e890851b0 2024-07-31T14:56:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ccebde740 10 client.17037 fetch_fsmap finished waiting for FSMap version 43 2024-07-31T14:56:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ccebde740 10 client.17037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:56:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ccebde740 20 client.17037 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ccebde740 1 --2- 172.21.15.67:0/382179022 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e890191c0 0x557e894af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ccebde740 4 client.17037 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:56:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.190+0000 7f1ccebde740 1 -- 172.21.15.67:0/382179022 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e8939ada0 con 0x557e890191c0 2024-07-31T14:56:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.191+0000 7f1cc5a84640 1 --2- 172.21.15.67:0/382179022 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e890191c0 0x557e894af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.191+0000 7f1ca97fa640 10 client.17037 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:37.191+0000 7f1cc5a84640 1 --2- 172.21.15.67:0/382179022 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e890191c0 0x557e894af3a0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eb699177121011a server_cookie=1a41de0e4babf847 in_seq=0 out_seq=0 2024-07-31T14:56:37.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:37 smithi073 ceph-mon[36714]: pgmap v566: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.4 MiB/s wr, 1.18k op/s 2024-07-31T14:56:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:37 smithi077 ceph-mon[37327]: pgmap v566: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.4 MiB/s wr, 1.18k op/s 2024-07-31T14:56:37.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:37 smithi067 ceph-mon[32750]: pgmap v566: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.4 MiB/s wr, 1.18k op/s 2024-07-31T14:56:37.964 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T14:56:38.234 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:56:39.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:39 smithi067 ceph-mon[32750]: pgmap v567: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 786 op/s 2024-07-31T14:56:39.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:39 smithi073 ceph-mon[36714]: pgmap v567: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 786 op/s 2024-07-31T14:56:39.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:39 smithi077 ceph-mon[37327]: pgmap v567: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 786 op/s 2024-07-31T14:56:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.932+0000 7f6a85fd7640 1 -- 172.21.15.67:0/456571029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a80113e60 msgr2=0x7f6a80116350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.932+0000 7f6a85fd7640 1 --2- 172.21.15.67:0/456571029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a80113e60 0x7f6a80116350 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009ca0 tx=0x7f6a7802a8f0 comp rx=0 tx=0).stop 2024-07-31T14:56:40.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.932+0000 7f6a85fd7640 1 -- 172.21.15.67:0/456571029 shutdown_connections 2024-07-31T14:56:40.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.933+0000 7f6a85fd7640 1 --2- 172.21.15.67:0/456571029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a80113e60 0x7f6a80116350 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:40.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.933+0000 7f6a85fd7640 1 --2- 172.21.15.67:0/456571029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a80075720 0x7f6a80075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:40.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.933+0000 7f6a85fd7640 1 --2- 172.21.15.67:0/456571029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a80076d60 0x7f6a800751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:40.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.933+0000 7f6a85fd7640 1 -- 172.21.15.67:0/456571029 >> 172.21.15.67:0/456571029 conn(0x7f6a8007b340 msgr2=0x7f6a8007b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:40.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.940+0000 7f6a85fd7640 1 -- 172.21.15.67:0/456571029 shutdown_connections 2024-07-31T14:56:40.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.941+0000 7f6a85fd7640 1 -- 172.21.15.67:0/456571029 wait complete. 2024-07-31T14:56:40.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.942+0000 7f6a85fd7640 1 Processor -- start 2024-07-31T14:56:40.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.943+0000 7f6a85fd7640 1 -- start start 2024-07-31T14:56:40.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.944+0000 7f6a85fd7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a80075720 0x7f6a800747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:40.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.944+0000 7f6a85fd7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a80076d60 0x7f6a8006da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:40.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.944+0000 7f6a85fd7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a80113e60 0x7f6a8006dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:40.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.944+0000 7f6a85fd7640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6a80119b10 con 0x7f6a80075720 2024-07-31T14:56:40.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.944+0000 7f6a85fd7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6a80119990 con 0x7f6a80113e60 2024-07-31T14:56:40.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.944+0000 7f6a85fd7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a80119c90 con 0x7f6a80076d60 2024-07-31T14:56:40.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.946+0000 7f6a7ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a80113e60 0x7f6a8006dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:40.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.946+0000 7f6a7ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a80113e60 0x7f6a8006dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:47372/0 (socket says 172.21.15.67:47372) 2024-07-31T14:56:40.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.946+0000 7f6a7ffff640 1 -- 172.21.15.67:0/3638774138 learned_addr learned my addr 172.21.15.67:0/3638774138 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.947+0000 7f6a7f7fe640 1 --2- 172.21.15.67:0/3638774138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a80075720 0x7f6a800747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.948+0000 7f6a7ffff640 1 -- 172.21.15.67:0/3638774138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a80076d60 msgr2=0x7f6a8006da70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T14:56:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.948+0000 7f6a7ffff640 1 --2- 172.21.15.67:0/3638774138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a80076d60 0x7f6a8006da70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.948+0000 7f6a7ffff640 1 -- 172.21.15.67:0/3638774138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a80075720 msgr2=0x7f6a800747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.948+0000 7f6a7ffff640 1 --2- 172.21.15.67:0/3638774138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a80075720 0x7f6a800747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.948+0000 7f6a7ffff640 1 -- 172.21.15.67:0/3638774138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8006e730 con 0x7f6a80113e60 2024-07-31T14:56:40.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.957+0000 7f6a7ffff640 1 --2- 172.21.15.67:0/3638774138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a80113e60 0x7f6a8006dfb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a78006010 tx=0x7f6a78003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:40.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.958+0000 7f6a7cff9640 1 -- 172.21.15.67:0/3638774138 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78004450 con 0x7f6a80113e60 2024-07-31T14:56:40.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.958+0000 7f6a85fd7640 1 -- 172.21.15.67:0/3638774138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a80073410 con 0x7f6a80113e60 2024-07-31T14:56:40.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.958+0000 7f6a85fd7640 1 -- 172.21.15.67:0/3638774138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a80073950 con 0x7f6a80113e60 2024-07-31T14:56:40.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.958+0000 7f6a7cff9640 1 -- 172.21.15.67:0/3638774138 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a78040070 con 0x7f6a80113e60 2024-07-31T14:56:40.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.959+0000 7f6a7cff9640 1 -- 172.21.15.67:0/3638774138 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78030850 con 0x7f6a80113e60 2024-07-31T14:56:40.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.982+0000 7f6a7cff9640 1 -- 172.21.15.67:0/3638774138 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6a78031b90 con 0x7f6a80113e60 2024-07-31T14:56:40.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.982+0000 7f6a8c528740 1 Processor -- start 2024-07-31T14:56:40.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.982+0000 7f6a8c528740 1 -- start start 2024-07-31T14:56:40.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.989+0000 7f6a8c528740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906be2070 0x560906f87690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:40.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.989+0000 7f6a8c528740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906bb91b0 0x560906f87bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:40.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.989+0000 7f6a8c528740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906bc1390 0x560906f88110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:40.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.989+0000 7f6a8c528740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560906efdbb0 con 0x560906be2070 2024-07-31T14:56:40.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.989+0000 7f6a7cff9640 1 --2- 172.21.15.67:0/3638774138 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6a680b3da0 0x7f6a680b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:40.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.989+0000 7f6a8c528740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560906eed0b0 con 0x560906bb91b0 2024-07-31T14:56:40.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.989+0000 7f6a8c528740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560906ef10f0 con 0x560906bc1390 2024-07-31T14:56:40.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.989+0000 7f6a7ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906bc1390 0x560906f88110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:40.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.989+0000 7f6a7ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906bc1390 0x560906f88110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34104/0 (socket says 172.21.15.67:34104) 2024-07-31T14:56:40.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.989+0000 7f6a7ffff640 1 -- 172.21.15.67:0/505874093 learned_addr learned my addr 172.21.15.67:0/505874093 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:40.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.990+0000 7f6a7ffff640 1 -- 172.21.15.67:0/505874093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906bb91b0 msgr2=0x560906f87bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:40.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.990+0000 7f6a7ffff640 1 --2- 172.21.15.67:0/505874093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906bb91b0 0x560906f87bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:40.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.990+0000 7f6a7ffff640 1 -- 172.21.15.67:0/505874093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906be2070 msgr2=0x560906f87690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:40.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.990+0000 7f6a7ffff640 1 --2- 172.21.15.67:0/505874093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906be2070 0x560906f87690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:40.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.990+0000 7f6a7ffff640 1 -- 172.21.15.67:0/505874093 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560906e93b90 con 0x560906bc1390 2024-07-31T14:56:40.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.990+0000 7f6a7cff9640 1 -- 172.21.15.67:0/3638774138 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f6a78105710 con 0x7f6a80113e60 2024-07-31T14:56:40.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.990+0000 7f6a7ffff640 1 --2- 172.21.15.67:0/505874093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906bc1390 0x560906f88110 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6a780c81a0 tx=0x7f6a7802a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca057cbdb46e66fd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:40.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.991+0000 7f6a7effd640 1 --2- 172.21.15.67:0/3638774138 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6a680b3da0 0x7f6a680b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:40.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.991+0000 7f6a65ffb640 1 -- 172.21.15.67:0/505874093 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a780098d0 con 0x560906bc1390 2024-07-31T14:56:40.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.995+0000 7f6a65ffb640 1 -- 172.21.15.67:0/505874093 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a78036030 con 0x560906bc1390 2024-07-31T14:56:40.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.995+0000 7f6a65ffb640 1 -- 172.21.15.67:0/505874093 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a780d51d0 con 0x560906bc1390 2024-07-31T14:56:40.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.997+0000 7f6a8c528740 1 -- 172.21.15.67:0/505874093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906bc1390 msgr2=0x560906f88110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:40.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:40.997+0000 7f6a8c528740 1 --2- 172.21.15.67:0/505874093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906bc1390 0x560906f88110 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6a780c81a0 tx=0x7f6a7802a8f0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.002+0000 7f6a8c528740 1 -- 172.21.15.67:0/505874093 shutdown_connections 2024-07-31T14:56:41.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.002+0000 7f6a8c528740 1 --2- 172.21.15.67:0/505874093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906bc1390 0x560906f88110 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.002+0000 7f6a8c528740 1 --2- 172.21.15.67:0/505874093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906bb91b0 0x560906f87bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.002+0000 7f6a8c528740 1 --2- 172.21.15.67:0/505874093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906be2070 0x560906f87690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.002+0000 7f6a8c528740 1 -- 172.21.15.67:0/505874093 >> 172.21.15.67:0/505874093 conn(0x560906bd6460 msgr2=0x560906f85580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:41.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.006+0000 7f6a8c528740 1 -- 172.21.15.67:0/505874093 shutdown_connections 2024-07-31T14:56:41.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.006+0000 7f6a7effd640 1 --2- 172.21.15.67:0/3638774138 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6a680b3da0 0x7f6a680b62b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x560906f86310 tx=0x7f6a70005cf0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:41.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.006+0000 7f6a8c528740 1 -- 172.21.15.67:0/505874093 wait complete. 2024-07-31T14:56:41.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.007+0000 7f6a8c528740 1 -- start start 2024-07-31T14:56:41.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.007+0000 7f6a8c528740 1 Processor -- start 2024-07-31T14:56:41.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.008+0000 7f6a8c528740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bb91b0 0x560906f932c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:41.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.008+0000 7f6a8c528740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906bc1390 0x560906f93800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:41.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.008+0000 7f6a8c528740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906be2070 0x560906f9eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:41.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.008+0000 7f6a8c528740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560906d50700 con 0x560906bb91b0 2024-07-31T14:56:41.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.009+0000 7f6a8c528740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560906e8f9e0 con 0x560906bc1390 2024-07-31T14:56:41.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.009+0000 7f6a8c528740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560906f95c30 con 0x560906be2070 2024-07-31T14:56:41.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.009+0000 7f6a8c528740 10 client.0 _ll_register_callbacks cb 0x560906eaa4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:41.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.009+0000 7f6a7f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bb91b0 0x560906f932c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:41.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.009+0000 7f6a7f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bb91b0 0x560906f932c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57962/0 (socket says 172.21.15.67:57962) 2024-07-31T14:56:41.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.009+0000 7f6a7f7fe640 1 -- 172.21.15.67:0/1068917017 learned_addr learned my addr 172.21.15.67:0/1068917017 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:41.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.010+0000 7f6a7ffff640 1 --2- 172.21.15.67:0/1068917017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906be2070 0x560906f9eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:41.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.012+0000 7f6a7ffff640 1 -- 172.21.15.67:0/1068917017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906bc1390 msgr2=0x560906f93800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:56:41.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.012+0000 7f6a7ffff640 1 --2- 172.21.15.67:0/1068917017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906bc1390 0x560906f93800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.012+0000 7f6a7ffff640 1 -- 172.21.15.67:0/1068917017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bb91b0 msgr2=0x560906f932c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:41.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.012+0000 7f6a7ffff640 1 --2- 172.21.15.67:0/1068917017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bb91b0 0x560906f932c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.012+0000 7f6a7ffff640 1 -- 172.21.15.67:0/1068917017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560906c4d650 con 0x560906be2070 2024-07-31T14:56:41.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.012+0000 7f6a7ffff640 1 --2- 172.21.15.67:0/1068917017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906be2070 0x560906f9eee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a780c81a0 tx=0x7f6a78009950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cf64ce4688b86e9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:41.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.012+0000 7f6a5effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:56:41.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.013+0000 7f6a5effd640 1 -- 172.21.15.67:0/1068917017 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a780e6060 con 0x560906be2070 2024-07-31T14:56:41.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.013+0000 7f6a5effd640 1 -- 172.21.15.67:0/1068917017 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f6a780d4d40 con 0x560906be2070 2024-07-31T14:56:41.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.015+0000 7f6a8c528740 1 -- 172.21.15.67:0/1068917017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a78036030 con 0x560906be2070 2024-07-31T14:56:41.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.018+0000 7f6a5effd640 1 -- 172.21.15.67:0/1068917017 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6a780cc820 con 0x560906be2070 2024-07-31T14:56:41.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.022+0000 7f6a8c528740 10 client.26641 fetch_fsmap learned FSMap version 43 2024-07-31T14:56:41.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.022+0000 7f6a8c528740 1 -- 172.21.15.67:0/1068917017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560906fa1c10 con 0x560906be2070 2024-07-31T14:56:41.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.026+0000 7f6a5effd640 1 -- 172.21.15.67:0/1068917017 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a7802d070 con 0x560906be2070 2024-07-31T14:56:41.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.026+0000 7f6a5effd640 1 -- 172.21.15.67:0/1068917017 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 43) ==== 8434+0+0 (secure 0 0 0) 0x7f6a7804c020 con 0x560906be2070 2024-07-31T14:56:41.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.027+0000 7f6a8c528740 10 client.26641 fetch_fsmap finished waiting for FSMap version 43 2024-07-31T14:56:41.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.027+0000 7f6a8c528740 10 client.26641 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:56:41.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.027+0000 7f6a8c528740 20 client.26641 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:41.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.027+0000 7f6a8c528740 1 --2- 172.21.15.67:0/1068917017 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560906ba09f0 0x560906fa4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:41.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.027+0000 7f6a8c528740 4 client.26641 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:56:41.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.027+0000 7f6a8c528740 1 -- 172.21.15.67:0/1068917017 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6a78036030 con 0x560906ba09f0 2024-07-31T14:56:41.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.029+0000 7f6a7f7fe640 1 --2- 172.21.15.67:0/1068917017 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560906ba09f0 0x560906fa4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:41.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.035+0000 7f6a5effd640 10 client.26641 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:56:41.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.035+0000 7f6a7f7fe640 1 --2- 172.21.15.67:0/1068917017 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560906ba09f0 0x560906fa4da0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=96c7564b16c9b8e2 server_cookie=30f84156b132dc08 in_seq=0 out_seq=0 2024-07-31T14:56:41.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.038+0000 7f6a5effd640 1 -- 172.21.15.67:0/1068917017 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6a78036030 con 0x560906ba09f0 2024-07-31T14:56:41.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.038+0000 7f6a5effd640 10 client.26641 handle_command_reply: tid=0 2024-07-31T14:56:41.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.039+0000 7f6a8c528740 1 client.26641 shutdown 2024-07-31T14:56:41.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.043+0000 7f6a8c528740 1 -- 172.21.15.67:0/1068917017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906be2070 msgr2=0x560906f9eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:41.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.043+0000 7f6a8c528740 1 --2- 172.21.15.67:0/1068917017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906be2070 0x560906f9eee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a780c81a0 tx=0x7f6a78009950 comp rx=0 tx=0).stop 2024-07-31T14:56:41.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.043+0000 7f6a8c528740 1 -- 172.21.15.67:0/1068917017 shutdown_connections 2024-07-31T14:56:41.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.043+0000 7f6a8c528740 1 --2- 172.21.15.67:0/1068917017 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560906ba09f0 0x560906fa4da0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.043+0000 7f6a8c528740 1 --2- 172.21.15.67:0/1068917017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906be2070 0x560906f9eee0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.043+0000 7f6a8c528740 1 --2- 172.21.15.67:0/1068917017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906bc1390 0x560906f93800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.043+0000 7f6a8c528740 1 --2- 172.21.15.67:0/1068917017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bb91b0 0x560906f932c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.043+0000 7f6a8c528740 1 -- 172.21.15.67:0/1068917017 >> 172.21.15.67:0/1068917017 conn(0x560906bd6460 msgr2=0x560906f83d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:41.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.043+0000 7f6a5effd640 0 client.26641 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:56:41.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.043+0000 7f6a8c528740 1 -- 172.21.15.67:0/1068917017 shutdown_connections 2024-07-31T14:56:41.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.043+0000 7f6a8c528740 1 -- 172.21.15.67:0/1068917017 wait complete. 2024-07-31T14:56:41.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.043+0000 7f6a8c528740 20 client.26641 trim_cache size 0 max 16384 2024-07-31T14:56:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.071+0000 7f6a8c528740 1 Processor -- start 2024-07-31T14:56:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.071+0000 7f6a8c528740 1 -- start start 2024-07-31T14:56:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.072+0000 7f6a8c528740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bb91b0 0x560906f9ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.072+0000 7f6a8c528740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906bc1390 0x560906f8c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.072+0000 7f6a8c528740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906be2070 0x560906f8cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.072+0000 7f6a8c528740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c00fe20 con 0x560906bb91b0 2024-07-31T14:56:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.072+0000 7f6a8c528740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c011b70 con 0x560906bc1390 2024-07-31T14:56:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.072+0000 7f6a8c528740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c010580 con 0x560906be2070 2024-07-31T14:56:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.073+0000 7f6a7ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906be2070 0x560906f8cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.073+0000 7f6a7ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906be2070 0x560906f8cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34122/0 (socket says 172.21.15.67:34122) 2024-07-31T14:56:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.073+0000 7f6a7ffff640 1 -- 172.21.15.67:0/1017876304 learned_addr learned my addr 172.21.15.67:0/1017876304 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.073+0000 7f6a7ffff640 1 -- 172.21.15.67:0/1017876304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906bc1390 msgr2=0x560906f8c7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:56:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.073+0000 7f6a7ffff640 1 --2- 172.21.15.67:0/1017876304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906bc1390 0x560906f8c7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.073+0000 7f6a7ffff640 1 -- 172.21.15.67:0/1017876304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bb91b0 msgr2=0x560906f9ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:41.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.073+0000 7f6a7ffff640 1 --2- 172.21.15.67:0/1017876304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bb91b0 0x560906f9ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.073+0000 7f6a7ffff640 1 -- 172.21.15.67:0/1017876304 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560906f89980 con 0x560906be2070 2024-07-31T14:56:41.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.075+0000 7f6a7ffff640 1 --2- 172.21.15.67:0/1017876304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906be2070 0x560906f8cce0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6a78005e30 tx=0x7f6a780cbd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd9c82c0e6295e79 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:41.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.076+0000 7f6a657fa640 1 -- 172.21.15.67:0/1017876304 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a780e6070 con 0x560906be2070 2024-07-31T14:56:41.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.076+0000 7f6a657fa640 1 -- 172.21.15.67:0/1017876304 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a78036070 con 0x560906be2070 2024-07-31T14:56:41.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.076+0000 7f6a657fa640 1 -- 172.21.15.67:0/1017876304 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a780d51d0 con 0x560906be2070 2024-07-31T14:56:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.078+0000 7f6a8c528740 1 -- 172.21.15.67:0/1017876304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906be2070 msgr2=0x560906f8cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.078+0000 7f6a8c528740 1 --2- 172.21.15.67:0/1017876304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906be2070 0x560906f8cce0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6a78005e30 tx=0x7f6a780cbd90 comp rx=0 tx=0).stop 2024-07-31T14:56:41.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.081+0000 7f6a8c528740 1 -- 172.21.15.67:0/1017876304 shutdown_connections 2024-07-31T14:56:41.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.081+0000 7f6a8c528740 1 --2- 172.21.15.67:0/1017876304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906be2070 0x560906f8cce0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.081+0000 7f6a8c528740 1 --2- 172.21.15.67:0/1017876304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906bc1390 0x560906f8c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.081+0000 7f6a8c528740 1 --2- 172.21.15.67:0/1017876304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bb91b0 0x560906f9ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.081+0000 7f6a8c528740 1 -- 172.21.15.67:0/1017876304 >> 172.21.15.67:0/1017876304 conn(0x560906bd6460 msgr2=0x560906f91940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:41.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.085+0000 7f6a8c528740 1 -- 172.21.15.67:0/1017876304 shutdown_connections 2024-07-31T14:56:41.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.089+0000 7f6a8c528740 1 -- 172.21.15.67:0/1017876304 wait complete. 2024-07-31T14:56:41.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.089+0000 7f6a8c528740 1 -- start start 2024-07-31T14:56:41.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.089+0000 7f6a8c528740 1 Processor -- start 2024-07-31T14:56:41.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.093+0000 7f6a8c528740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906bb91b0 0x560907005660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:41.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.093+0000 7f6a8c528740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bc1390 0x560907005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:41.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.093+0000 7f6a8c528740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906be2070 0x560906f9aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:41.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.093+0000 7f6a8c528740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c0114a0 con 0x560906bc1390 2024-07-31T14:56:41.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.093+0000 7f6a8c528740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c010d10 con 0x560906be2070 2024-07-31T14:56:41.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.093+0000 7f6a8c528740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c00de00 con 0x560906bb91b0 2024-07-31T14:56:41.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.093+0000 7f6a8c528740 10 client.0 _ll_register_callbacks cb 0x7f6a78009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.095+0000 7f6a7effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bc1390 0x560907005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.096+0000 7f6a7f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906bb91b0 0x560907005660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.096+0000 7f6a7ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906be2070 0x560906f9aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.096+0000 7f6a7ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906be2070 0x560906f9aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:47396/0 (socket says 172.21.15.67:47396) 2024-07-31T14:56:41.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.096+0000 7f6a7ffff640 1 -- 172.21.15.67:0/671702628 learned_addr learned my addr 172.21.15.67:0/671702628 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.098+0000 7f6a7f7fe640 1 -- 172.21.15.67:0/671702628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906be2070 msgr2=0x560906f9aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.098+0000 7f6a7f7fe640 1 --2- 172.21.15.67:0/671702628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906be2070 0x560906f9aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.098+0000 7f6a7f7fe640 1 -- 172.21.15.67:0/671702628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bc1390 msgr2=0x560907005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:41.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.098+0000 7f6a7f7fe640 1 --2- 172.21.15.67:0/671702628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bc1390 0x560907005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.098+0000 7f6a7f7fe640 1 -- 172.21.15.67:0/671702628 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560906f9b210 con 0x560906bb91b0 2024-07-31T14:56:41.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.098+0000 7f6a7f7fe640 1 --2- 172.21.15.67:0/671702628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906bb91b0 0x560907005660 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x560906f861c0 tx=0x7f6a74012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55649ec32c554d34 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:41.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.100+0000 7f6a5effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:56:41.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.106+0000 7f6a5effd640 1 -- 172.21.15.67:0/671702628 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74004440 con 0x560906bb91b0 2024-07-31T14:56:41.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.106+0000 7f6a5effd640 1 -- 172.21.15.67:0/671702628 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f6a740035f0 con 0x560906bb91b0 2024-07-31T14:56:41.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.110+0000 7f6a8c528740 1 -- 172.21.15.67:0/671702628 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a78036070 con 0x560906bb91b0 2024-07-31T14:56:41.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.115+0000 7f6a5effd640 1 -- 172.21.15.67:0/671702628 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6a740037f0 con 0x560906bb91b0 2024-07-31T14:56:41.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.117+0000 7f6a8c528740 10 client.26647 fetch_fsmap learned FSMap version 43 2024-07-31T14:56:41.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.117+0000 7f6a8c528740 1 -- 172.21.15.67:0/671702628 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56090700cf30 con 0x560906bb91b0 2024-07-31T14:56:41.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.119+0000 7f6a5effd640 1 -- 172.21.15.67:0/671702628 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a78036070 con 0x560906bb91b0 2024-07-31T14:56:41.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.119+0000 7f6a5effd640 1 -- 172.21.15.67:0/671702628 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 43) ==== 8434+0+0 (secure 0 0 0) 0x7f6a7401b950 con 0x560906bb91b0 2024-07-31T14:56:41.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.122+0000 7f6a8c528740 10 client.26647 fetch_fsmap finished waiting for FSMap version 43 2024-07-31T14:56:41.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.122+0000 7f6a8c528740 10 client.26647 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:56:41.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.122+0000 7f6a8c528740 20 client.26647 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:41.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.122+0000 7f6a8c528740 1 --2- 172.21.15.67:0/671702628 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560906b75da0 0x56090700f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:41.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.122+0000 7f6a8c528740 4 client.26647 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:56:41.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.122+0000 7f6a8c528740 1 -- 172.21.15.67:0/671702628 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560906effb00 con 0x560906b75da0 2024-07-31T14:56:41.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.124+0000 7f6a7effd640 1 --2- 172.21.15.67:0/671702628 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560906b75da0 0x56090700f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:41.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.131+0000 7f6a7effd640 1 --2- 172.21.15.67:0/671702628 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560906b75da0 0x56090700f720 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e8ac8f8e8f5b2b7e server_cookie=2e9d22e1e2466e71 in_seq=0 out_seq=0 2024-07-31T14:56:41.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.131+0000 7f6a5effd640 10 client.26647 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:56:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.132+0000 7f6a5effd640 1 -- 172.21.15.67:0/671702628 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560906effb00 con 0x560906b75da0 2024-07-31T14:56:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.132+0000 7f6a5effd640 10 client.26647 handle_command_reply: tid=0 2024-07-31T14:56:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.132+0000 7f6a8c528740 1 client.26647 shutdown 2024-07-31T14:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.134+0000 7f6a8c528740 1 -- 172.21.15.67:0/671702628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906bb91b0 msgr2=0x560907005660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.134+0000 7f6a8c528740 1 --2- 172.21.15.67:0/671702628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906bb91b0 0x560907005660 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x560906f861c0 tx=0x7f6a74012820 comp rx=0 tx=0).stop 2024-07-31T14:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.134+0000 7f6a8c528740 1 -- 172.21.15.67:0/671702628 shutdown_connections 2024-07-31T14:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.134+0000 7f6a8c528740 1 --2- 172.21.15.67:0/671702628 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560906b75da0 0x56090700f720 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.134+0000 7f6a8c528740 1 --2- 172.21.15.67:0/671702628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560906be2070 0x560906f9aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.134+0000 7f6a8c528740 1 --2- 172.21.15.67:0/671702628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560906bc1390 0x560907005c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.134+0000 7f6a5effd640 0 client.26647 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.134+0000 7f6a8c528740 1 --2- 172.21.15.67:0/671702628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560906bb91b0 0x560907005660 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.134+0000 7f6a8c528740 1 -- 172.21.15.67:0/671702628 >> 172.21.15.67:0/671702628 conn(0x560906bd6460 msgr2=0x560906f83d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:41.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.134+0000 7f6a8c528740 1 -- 172.21.15.67:0/671702628 shutdown_connections 2024-07-31T14:56:41.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.134+0000 7f6a8c528740 1 -- 172.21.15.67:0/671702628 wait complete. 2024-07-31T14:56:41.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.135+0000 7f6a8c528740 20 client.26647 trim_cache size 0 max 16384 2024-07-31T14:56:41.138 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:56:41.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.136+0000 7f6a85fd7640 1 -- 172.21.15.67:0/3638774138 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6a680b3da0 msgr2=0x7f6a680b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:41.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.136+0000 7f6a85fd7640 1 --2- 172.21.15.67:0/3638774138 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6a680b3da0 0x7f6a680b62b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x560906f86310 tx=0x7f6a70005cf0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.137+0000 7f6a85fd7640 1 -- 172.21.15.67:0/3638774138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a80113e60 msgr2=0x7f6a8006dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:41.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.137+0000 7f6a85fd7640 1 --2- 172.21.15.67:0/3638774138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a80113e60 0x7f6a8006dfb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a78006010 tx=0x7f6a78003f10 comp rx=0 tx=0).stop 2024-07-31T14:56:41.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.137+0000 7f6a85fd7640 1 -- 172.21.15.67:0/3638774138 shutdown_connections 2024-07-31T14:56:41.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.137+0000 7f6a85fd7640 1 --2- 172.21.15.67:0/3638774138 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6a680b3da0 0x7f6a680b62b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.137+0000 7f6a85fd7640 1 --2- 172.21.15.67:0/3638774138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a80113e60 0x7f6a8006dfb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.137+0000 7f6a85fd7640 1 --2- 172.21.15.67:0/3638774138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a80076d60 0x7f6a8006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.137+0000 7f6a85fd7640 1 --2- 172.21.15.67:0/3638774138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a80075720 0x7f6a800747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:41.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.137+0000 7f6a85fd7640 1 -- 172.21.15.67:0/3638774138 >> 172.21.15.67:0/3638774138 conn(0x7f6a8007b340 msgr2=0x7f6a80112ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:41.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.137+0000 7f6a85fd7640 1 -- 172.21.15.67:0/3638774138 shutdown_connections 2024-07-31T14:56:41.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:41.138+0000 7f6a85fd7640 1 -- 172.21.15.67:0/3638774138 wait complete. 2024-07-31T14:56:41.163 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-07-31T14:56:41.164 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:56:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:41 smithi067 ceph-mon[32750]: pgmap v568: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 556 op/s 2024-07-31T14:56:41.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:41 smithi073 ceph-mon[36714]: pgmap v568: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 556 op/s 2024-07-31T14:56:41.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:41 smithi077 ceph-mon[37327]: pgmap v568: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 556 op/s 2024-07-31T14:56:42.750 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T14:56:42.827 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:56:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:43 smithi067 ceph-mon[32750]: pgmap v569: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 673 op/s 2024-07-31T14:56:43.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:43 smithi073 ceph-mon[36714]: pgmap v569: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 673 op/s 2024-07-31T14:56:43.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:43 smithi077 ceph-mon[37327]: pgmap v569: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 673 op/s 2024-07-31T14:56:45.159 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:56:45.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:45 smithi073 ceph-mon[36714]: pgmap v570: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 1.24k op/s 2024-07-31T14:56:45.418 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:45 smithi067 ceph-mon[32750]: pgmap v570: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 1.24k op/s 2024-07-31T14:56:45.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:45 smithi077 ceph-mon[37327]: pgmap v570: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 1.24k op/s 2024-07-31T14:56:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.622+0000 7f1ca97fa640 1 -- 172.21.15.67:0/382179022 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e8939ada0 con 0x557e890191c0 2024-07-31T14:56:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.622+0000 7f1ca97fa640 10 client.17037 handle_command_reply: tid=0 2024-07-31T14:56:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.622+0000 7f1ccebde740 1 client.17037 shutdown 2024-07-31T14:56:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.622+0000 7f1ccebde740 1 -- 172.21.15.67:0/382179022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 msgr2=0x557e89438870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.622+0000 7f1ccebde740 1 --2- 172.21.15.67:0/382179022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e89438870 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80209d0 tx=0x7f1cb8020e70 comp rx=0 tx=0).stop 2024-07-31T14:56:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.622+0000 7f1ccebde740 1 -- 172.21.15.67:0/382179022 shutdown_connections 2024-07-31T14:56:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.622+0000 7f1ccebde740 1 --2- 172.21.15.67:0/382179022 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e890191c0 0x557e894af3a0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.622+0000 7f1ccebde740 1 --2- 172.21.15.67:0/382179022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e890851b0 0x557e89438870 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.622+0000 7f1ccebde740 1 --2- 172.21.15.67:0/382179022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e890644d0 0x557e8943c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.622+0000 7f1ca97fa640 0 client.17037 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.622+0000 7f1ccebde740 1 --2- 172.21.15.67:0/382179022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8905c2f0 0x557e8943ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.622+0000 7f1ccebde740 1 -- 172.21.15.67:0/382179022 >> 172.21.15.67:0/382179022 conn(0x557e89079300 msgr2=0x557e89435200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.623+0000 7f1ccebde740 1 -- 172.21.15.67:0/382179022 shutdown_connections 2024-07-31T14:56:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.623+0000 7f1ccebde740 1 -- 172.21.15.67:0/382179022 wait complete. 2024-07-31T14:56:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.623+0000 7f1ccebde740 20 client.17037 trim_cache size 0 max 16384 2024-07-31T14:56:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:56:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:56:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:56:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:56:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.624+0000 7f1cc7ce8640 1 -- 172.21.15.67:0/3446032476 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1cac0b3d40 msgr2=0x7f1cac0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.624+0000 7f1cc7ce8640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1cac0b3d40 0x7f1cac0b6250 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80098b0 tx=0x7f1cb800ebf0 comp rx=0 tx=0).stop 2024-07-31T14:56:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.624+0000 7f1cc7ce8640 1 -- 172.21.15.67:0/3446032476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0107d20 msgr2=0x7f1cc019a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.624+0000 7f1cc7ce8640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0107d20 0x7f1cc019a3b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006010 tx=0x7f1cb0004060 comp rx=0 tx=0).stop 2024-07-31T14:56:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.624+0000 7f1cc7ce8640 1 -- 172.21.15.67:0/3446032476 shutdown_connections 2024-07-31T14:56:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.624+0000 7f1cc7ce8640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1cac0b3d40 0x7f1cac0b6250 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.624+0000 7f1cc7ce8640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc0108670 0x7f1cc019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.624+0000 7f1cc7ce8640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cc0107d20 0x7f1cc019a3b0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.624+0000 7f1cc7ce8640 1 --2- 172.21.15.67:0/3446032476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc01023f0 0x7f1cc0199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.624+0000 7f1cc7ce8640 1 -- 172.21.15.67:0/3446032476 >> 172.21.15.67:0/3446032476 conn(0x7f1cc00fddf0 msgr2=0x7f1cc0106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.624+0000 7f1cc7ce8640 1 -- 172.21.15.67:0/3446032476 shutdown_connections 2024-07-31T14:56:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:45.624+0000 7f1cc7ce8640 1 -- 172.21.15.67:0/3446032476 wait complete. 2024-07-31T14:56:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:46 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:56:46.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:46 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:56:46.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:46 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:56:46.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.915+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/1205212170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d981022a0 msgr2=0x7f0d981026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:46.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.915+0000 7f0d9d4f1640 1 --2- 172.21.15.67:0/1205212170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d981022a0 0x7f0d981026b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009860 tx=0x7f0d8802a360 comp rx=0 tx=0).stop 2024-07-31T14:56:46.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.916+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/1205212170 shutdown_connections 2024-07-31T14:56:46.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.916+0000 7f0d9d4f1640 1 --2- 172.21.15.67:0/1205212170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d98108530 0x7f0d9810ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:46.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.916+0000 7f0d9d4f1640 1 --2- 172.21.15.67:0/1205212170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d98107be0 0x7f0d98107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:46.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.916+0000 7f0d9d4f1640 1 --2- 172.21.15.67:0/1205212170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d981022a0 0x7f0d981026b0 secure :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009860 tx=0x7f0d8802a360 comp rx=0 tx=0).stop 2024-07-31T14:56:46.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.916+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/1205212170 >> 172.21.15.67:0/1205212170 conn(0x7f0d980fdcf0 msgr2=0x7f0d98100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:46.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.918+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/1205212170 shutdown_connections 2024-07-31T14:56:46.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.919+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/1205212170 wait complete. 2024-07-31T14:56:46.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.920+0000 7f0d9d4f1640 1 Processor -- start 2024-07-31T14:56:46.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.920+0000 7f0d9d4f1640 1 -- start start 2024-07-31T14:56:46.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.921+0000 7f0d9d4f1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d98107be0 0x7f0d98197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:46.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.922+0000 7f0d9d4f1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d98108530 0x7f0d981981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:46.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.922+0000 7f0d9d4f1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9819d3c0 0x7f0d9819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:46.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.922+0000 7f0d9d4f1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0d981110f0 con 0x7f0d9819d3c0 2024-07-31T14:56:46.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.922+0000 7f0d9d4f1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0d98110f70 con 0x7f0d98108530 2024-07-31T14:56:46.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.922+0000 7f0d9d4f1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0d98111270 con 0x7f0d98107be0 2024-07-31T14:56:46.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.922+0000 7f0d97fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d98107be0 0x7f0d98197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:46.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.922+0000 7f0d977fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d98108530 0x7f0d981981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:46.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.922+0000 7f0d97fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d98107be0 0x7f0d98197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34148/0 (socket says 172.21.15.67:34148) 2024-07-31T14:56:46.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.922+0000 7f0d97fff640 1 -- 172.21.15.67:0/2584187279 learned_addr learned my addr 172.21.15.67:0/2584187279 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:46.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.922+0000 7f0d977fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d98108530 0x7f0d981981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:47428/0 (socket says 172.21.15.67:47428) 2024-07-31T14:56:46.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.923+0000 7f0d9ccf0640 1 --2- 172.21.15.67:0/2584187279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9819d3c0 0x7f0d9819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.924+0000 7f0d9ccf0640 1 -- 172.21.15.67:0/2584187279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d98107be0 msgr2=0x7f0d98197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.924+0000 7f0d9ccf0640 1 --2- 172.21.15.67:0/2584187279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d98107be0 0x7f0d98197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.924+0000 7f0d9ccf0640 1 -- 172.21.15.67:0/2584187279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d98108530 msgr2=0x7f0d981981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.924+0000 7f0d9ccf0640 1 --2- 172.21.15.67:0/2584187279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d98108530 0x7f0d981981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.924+0000 7f0d9ccf0640 1 -- 172.21.15.67:0/2584187279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9819dfa0 con 0x7f0d9819d3c0 2024-07-31T14:56:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.924+0000 7f0d9ccf0640 1 --2- 172.21.15.67:0/2584187279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9819d3c0 0x7f0d9819d7d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009fd0 tx=0x7f0d8c00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:46.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.927+0000 7f0d957fa640 1 -- 172.21.15.67:0/2584187279 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c004230 con 0x7f0d9819d3c0 2024-07-31T14:56:46.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.927+0000 7f0d957fa640 1 -- 172.21.15.67:0/2584187279 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d8c0043d0 con 0x7f0d9819d3c0 2024-07-31T14:56:46.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.927+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/2584187279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d981a4540 con 0x7f0d9819d3c0 2024-07-31T14:56:46.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.927+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/2584187279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d981a4940 con 0x7f0d9819d3c0 2024-07-31T14:56:46.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.927+0000 7f0d957fa640 1 -- 172.21.15.67:0/2584187279 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c0047a0 con 0x7f0d9819d3c0 2024-07-31T14:56:46.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.932+0000 7f0d957fa640 1 -- 172.21.15.67:0/2584187279 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0d8c004a60 con 0x7f0d9819d3c0 2024-07-31T14:56:46.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.933+0000 7f0d957fa640 1 --2- 172.21.15.67:0/2584187279 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0d800b82a0 0x7f0d800ba7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:46.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.934+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/2584187279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d981026b0 con 0x7f0d9819d3c0 2024-07-31T14:56:46.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.934+0000 7f0d957fa640 1 -- 172.21.15.67:0/2584187279 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f0d8c0a7300 con 0x7f0d9819d3c0 2024-07-31T14:56:46.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.936+0000 7f0d97fff640 1 --2- 172.21.15.67:0/2584187279 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0d800b82a0 0x7f0d800ba7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:46.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.938+0000 7f0d97fff640 1 --2- 172.21.15.67:0/2584187279 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0d800b82a0 0x7f0d800ba7b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0d88000c00 tx=0x7f0d88005d40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:46.943+0000 7f0d957fa640 1 -- 172.21.15.67:0/2584187279 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f0d8c0ab410 con 0x7f0d9819d3c0 2024-07-31T14:56:47.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:47 smithi067 ceph-mon[32750]: pgmap v571: 129 pgs: 129 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.9 MiB/s wr, 2.06k op/s 2024-07-31T14:56:47.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:47 smithi073 ceph-mon[36714]: pgmap v571: 129 pgs: 129 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.9 MiB/s wr, 2.06k op/s 2024-07-31T14:56:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.397+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/2584187279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d9810a300 con 0x7f0d9819d3c0 2024-07-31T14:56:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.400+0000 7f0d957fa640 1 -- 172.21.15.67:0/2584187279 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11984 (secure 0 0 0) 0x7f0d8c0aad10 con 0x7f0d9819d3c0 2024-07-31T14:56:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:56:47.402 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":44,"btime":"2024-07-31T14:56:45:102341+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:56:44.804223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:56:47.402 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 44 2024-07-31T14:56:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.405+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/2584187279 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0d800b82a0 msgr2=0x7f0d800ba7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.405+0000 7f0d9d4f1640 1 --2- 172.21.15.67:0/2584187279 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0d800b82a0 0x7f0d800ba7b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0d88000c00 tx=0x7f0d88005d40 comp rx=0 tx=0).stop 2024-07-31T14:56:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.406+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/2584187279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9819d3c0 msgr2=0x7f0d9819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.406+0000 7f0d9d4f1640 1 --2- 172.21.15.67:0/2584187279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9819d3c0 0x7f0d9819d7d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009fd0 tx=0x7f0d8c00eba0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.407+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/2584187279 shutdown_connections 2024-07-31T14:56:47.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.407+0000 7f0d9d4f1640 1 --2- 172.21.15.67:0/2584187279 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0d800b82a0 0x7f0d800ba7b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.407+0000 7f0d9d4f1640 1 --2- 172.21.15.67:0/2584187279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9819d3c0 0x7f0d9819d7d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.407+0000 7f0d9d4f1640 1 --2- 172.21.15.67:0/2584187279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d98108530 0x7f0d981981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.407+0000 7f0d9d4f1640 1 --2- 172.21.15.67:0/2584187279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d98107be0 0x7f0d98197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.407+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/2584187279 >> 172.21.15.67:0/2584187279 conn(0x7f0d980fdcf0 msgr2=0x7f0d98100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.407+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/2584187279 shutdown_connections 2024-07-31T14:56:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.407+0000 7f0d9d4f1640 1 -- 172.21.15.67:0/2584187279 wait complete. 2024-07-31T14:56:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.448+0000 7f39d4ee7640 1 -- 172.21.15.67:0/2893345803 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c40035f0 con 0x7f39d011c660 2024-07-31T14:56:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.448+0000 7f39d814d640 1 -- 172.21.15.67:0/2893345803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d011c660 msgr2=0x7f39d011eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.448+0000 7f39d814d640 1 --2- 172.21.15.67:0/2893345803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d011c660 0x7f39d011eb70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009900 tx=0x7f39c402a440 comp rx=0 tx=0).stop 2024-07-31T14:56:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.448+0000 7f39d814d640 1 -- 172.21.15.67:0/2893345803 shutdown_connections 2024-07-31T14:56:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.448+0000 7f39d814d640 1 --2- 172.21.15.67:0/2893345803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d011c660 0x7f39d011eb70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.448+0000 7f39d814d640 1 --2- 172.21.15.67:0/2893345803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39d010a410 0x7f39d010a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.448+0000 7f39d814d640 1 --2- 172.21.15.67:0/2893345803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d0111970 0x7f39d0109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.449+0000 7f39d814d640 1 -- 172.21.15.67:0/2893345803 >> 172.21.15.67:0/2893345803 conn(0x7f39d00708f0 msgr2=0x7f39d0070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.449+0000 7f39d814d640 1 -- 172.21.15.67:0/2893345803 shutdown_connections 2024-07-31T14:56:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.449+0000 7f39d814d640 1 -- 172.21.15.67:0/2893345803 wait complete. 2024-07-31T14:56:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.450+0000 7f39d814d640 1 Processor -- start 2024-07-31T14:56:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.450+0000 7f39d814d640 1 -- start start 2024-07-31T14:56:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.450+0000 7f39d814d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d010a410 0x7f39d01a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.450+0000 7f39d814d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39d0111970 0x7f39d01a91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.450+0000 7f39d814d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d011c660 0x7f39d01ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.450+0000 7f39d814d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f39d0122320 con 0x7f39d011c660 2024-07-31T14:56:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.450+0000 7f39d814d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01221a0 con 0x7f39d0111970 2024-07-31T14:56:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.450+0000 7f39d814d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01224a0 con 0x7f39d010a410 2024-07-31T14:56:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.451+0000 7f39d5ee9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d010a410 0x7f39d01a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.451+0000 7f39d5ee9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d010a410 0x7f39d01a8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34160/0 (socket says 172.21.15.67:34160) 2024-07-31T14:56:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.451+0000 7f39d5ee9640 1 -- 172.21.15.67:0/2975990565 learned_addr learned my addr 172.21.15.67:0/2975990565 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:47.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.451+0000 7f39d66ea640 1 --2- 172.21.15.67:0/2975990565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d011c660 0x7f39d01ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.452+0000 7f39d56e8640 1 --2- 172.21.15.67:0/2975990565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39d0111970 0x7f39d01a91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.452+0000 7f39d5ee9640 1 -- 172.21.15.67:0/2975990565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39d0111970 msgr2=0x7f39d01a91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.452+0000 7f39d5ee9640 1 --2- 172.21.15.67:0/2975990565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39d0111970 0x7f39d01a91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.452+0000 7f39d5ee9640 1 -- 172.21.15.67:0/2975990565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d011c660 msgr2=0x7f39d01ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:47.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.452+0000 7f39d5ee9640 1 --2- 172.21.15.67:0/2975990565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d011c660 0x7f39d01ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.452+0000 7f39d5ee9640 1 -- 172.21.15.67:0/2975990565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d01aebc0 con 0x7f39d010a410 2024-07-31T14:56:47.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.453+0000 7f39d5ee9640 1 --2- 172.21.15.67:0/2975990565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d010a410 0x7f39d01a8c50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f39c800d950 tx=0x7f39c800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:47.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.453+0000 7f39ceffd640 1 -- 172.21.15.67:0/2975990565 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8017070 con 0x7f39d010a410 2024-07-31T14:56:47.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.454+0000 7f39ceffd640 1 -- 172.21.15.67:0/2975990565 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c8009c30 con 0x7f39d010a410 2024-07-31T14:56:47.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.454+0000 7f39ceffd640 1 -- 172.21.15.67:0/2975990565 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c800f620 con 0x7f39d010a410 2024-07-31T14:56:47.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.454+0000 7f39d814d640 1 -- 172.21.15.67:0/2975990565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d01aeeb0 con 0x7f39d010a410 2024-07-31T14:56:47.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.454+0000 7f39d814d640 1 -- 172.21.15.67:0/2975990565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d007b050 con 0x7f39d010a410 2024-07-31T14:56:47.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.458+0000 7f39de69e740 1 Processor -- start 2024-07-31T14:56:47.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.458+0000 7f39de69e740 1 -- start start 2024-07-31T14:56:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.459+0000 7f39de69e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c50570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.459+0000 7f39de69e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607798821b0 0x560779c50ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.459+0000 7f39de69e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56077988a390 0x560779c50ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.459+0000 7f39d66ea640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c50570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.459+0000 7f39d66ea640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c50570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58024/0 (socket says 172.21.15.67:58024) 2024-07-31T14:56:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.459+0000 7f39d66ea640 1 -- 172.21.15.67:0/8750615 learned_addr learned my addr 172.21.15.67:0/8750615 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.459+0000 7f39de69e740 1 -- 172.21.15.67:0/8750615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560779bc6bb0 con 0x5607798ab070 2024-07-31T14:56:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.459+0000 7f39de69e740 1 -- 172.21.15.67:0/8750615 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560779bb60b0 con 0x5607798821b0 2024-07-31T14:56:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.459+0000 7f39de69e740 1 -- 172.21.15.67:0/8750615 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560779bba0f0 con 0x56077988a390 2024-07-31T14:56:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.459+0000 7f39d56e8640 1 --2- 172.21.15.67:0/8750615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56077988a390 0x560779c50ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.460+0000 7f39d56e8640 1 -- 172.21.15.67:0/8750615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607798821b0 msgr2=0x560779c50ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:56:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.460+0000 7f39d56e8640 1 --2- 172.21.15.67:0/8750615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607798821b0 0x560779c50ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.460+0000 7f39d56e8640 1 -- 172.21.15.67:0/8750615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 msgr2=0x560779c50570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.460+0000 7f39d56e8640 1 --2- 172.21.15.67:0/8750615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c50570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.460+0000 7f39d56e8640 1 -- 172.21.15.67:0/8750615 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560779b5cb90 con 0x56077988a390 2024-07-31T14:56:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.460+0000 7f39d56e8640 1 --2- 172.21.15.67:0/8750615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56077988a390 0x560779c50ff0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f39d006c3a0 tx=0x7f39c0010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f2aeea3daaf39d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.460+0000 7f39b7fff640 1 -- 172.21.15.67:0/8750615 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0016070 con 0x56077988a390 2024-07-31T14:56:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.461+0000 7f39b7fff640 1 -- 172.21.15.67:0/8750615 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c0003e20 con 0x56077988a390 2024-07-31T14:56:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.461+0000 7f39b7fff640 1 -- 172.21.15.67:0/8750615 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0005020 con 0x56077988a390 2024-07-31T14:56:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.462+0000 7f39de69e740 1 -- 172.21.15.67:0/8750615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56077988a390 msgr2=0x560779c50ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.462+0000 7f39de69e740 1 --2- 172.21.15.67:0/8750615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56077988a390 0x560779c50ff0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f39d006c3a0 tx=0x7f39c0010740 comp rx=0 tx=0).stop 2024-07-31T14:56:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.462+0000 7f39ceffd640 1 -- 172.21.15.67:0/2975990565 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f39c8010460 con 0x7f39d010a410 2024-07-31T14:56:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.463+0000 7f39ceffd640 1 --2- 172.21.15.67:0/2975990565 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f39b80b3ed0 0x7f39b80b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.464+0000 7f39ceffd640 1 -- 172.21.15.67:0/2975990565 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f39c80e3c50 con 0x7f39d010a410 2024-07-31T14:56:47.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.464+0000 7f39d66ea640 1 --2- 172.21.15.67:0/2975990565 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f39b80b3ed0 0x7f39b80b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:47.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.465+0000 7f39de69e740 1 -- 172.21.15.67:0/8750615 shutdown_connections 2024-07-31T14:56:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.465+0000 7f39de69e740 1 --2- 172.21.15.67:0/8750615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56077988a390 0x560779c50ff0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.465+0000 7f39de69e740 1 --2- 172.21.15.67:0/8750615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607798821b0 0x560779c50ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.465+0000 7f39de69e740 1 --2- 172.21.15.67:0/8750615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c50570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.465+0000 7f39de69e740 1 -- 172.21.15.67:0/8750615 >> 172.21.15.67:0/8750615 conn(0x56077989f460 msgr2=0x560779c4e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.466+0000 7f39d66ea640 1 --2- 172.21.15.67:0/2975990565 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f39b80b3ed0 0x7f39b80b63e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f39c4005ab0 tx=0x7f39c4005a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.470+0000 7f39de69e740 1 -- 172.21.15.67:0/8750615 shutdown_connections 2024-07-31T14:56:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.471+0000 7f39de69e740 1 -- 172.21.15.67:0/8750615 wait complete. 2024-07-31T14:56:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.472+0000 7f39de69e740 1 -- start start 2024-07-31T14:56:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.472+0000 7f39de69e740 1 Processor -- start 2024-07-31T14:56:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.473+0000 7f39de69e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607798821b0 0x560779c5c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.473+0000 7f39de69e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077988a390 0x560779c5c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.473+0000 7f39d5ee9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607798821b0 0x560779c5c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:47.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.473+0000 7f39d5ee9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607798821b0 0x560779c5c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34178/0 (socket says 172.21.15.67:34178) 2024-07-31T14:56:47.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.473+0000 7f39d5ee9640 1 -- 172.21.15.67:0/1962097860 learned_addr learned my addr 172.21.15.67:0/1962097860 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.473+0000 7f39de69e740 1 --2- 172.21.15.67:0/1962097860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c68040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.474+0000 7f39de69e740 1 -- 172.21.15.67:0/1962097860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560779a19700 con 0x5607798ab070 2024-07-31T14:56:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.474+0000 7f39de69e740 1 -- 172.21.15.67:0/1962097860 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560779b589e0 con 0x56077988a390 2024-07-31T14:56:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.474+0000 7f39de69e740 1 -- 172.21.15.67:0/1962097860 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560779c5ed50 con 0x5607798821b0 2024-07-31T14:56:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.474+0000 7f39d66ea640 1 --2- 172.21.15.67:0/1962097860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c68040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.474+0000 7f39d56e8640 1 --2- 172.21.15.67:0/1962097860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077988a390 0x560779c5c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.474+0000 7f39de69e740 10 client.0 _ll_register_callbacks cb 0x560779b734f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.475+0000 7f39d66ea640 1 -- 172.21.15.67:0/1962097860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607798821b0 msgr2=0x560779c5c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.475+0000 7f39d66ea640 1 --2- 172.21.15.67:0/1962097860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607798821b0 0x560779c5c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.475+0000 7f39d66ea640 1 -- 172.21.15.67:0/1962097860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077988a390 msgr2=0x560779c5c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.475+0000 7f39d66ea640 1 --2- 172.21.15.67:0/1962097860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077988a390 0x560779c5c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.475+0000 7f39d66ea640 1 -- 172.21.15.67:0/1962097860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560779916650 con 0x5607798ab070 2024-07-31T14:56:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.477+0000 7f39d66ea640 1 --2- 172.21.15.67:0/1962097860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c68040 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009230 tx=0x7f39c4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cdd10ba97a5a2c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.477+0000 7f39b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:56:47.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.478+0000 7f39b57fa640 1 -- 172.21.15.67:0/1962097860 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4047870 con 0x5607798ab070 2024-07-31T14:56:47.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.478+0000 7f39b57fa640 1 -- 172.21.15.67:0/1962097860 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f39c4047a10 con 0x5607798ab070 2024-07-31T14:56:47.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.478+0000 7f39de69e740 1 -- 172.21.15.67:0/1962097860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39c0003e20 con 0x5607798ab070 2024-07-31T14:56:47.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.483+0000 7f39b57fa640 1 -- 172.21.15.67:0/1962097860 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f39c4047c10 con 0x5607798ab070 2024-07-31T14:56:47.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.484+0000 7f39de69e740 10 client.17067 fetch_fsmap learned FSMap version 44 2024-07-31T14:56:47.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.484+0000 7f39de69e740 1 -- 172.21.15.67:0/1962097860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560779c6add0 con 0x5607798ab070 2024-07-31T14:56:47.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:47 smithi077 ceph-mon[37327]: pgmap v571: 129 pgs: 129 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.9 MiB/s wr, 2.06k op/s 2024-07-31T14:56:47.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.487+0000 7f39b57fa640 1 -- 172.21.15.67:0/1962097860 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c4047db0 con 0x5607798ab070 2024-07-31T14:56:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.488+0000 7f39b57fa640 1 -- 172.21.15.67:0/1962097860 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7f39c4050070 con 0x5607798ab070 2024-07-31T14:56:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.488+0000 7f39de69e740 10 client.17067 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:56:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.488+0000 7f39de69e740 10 client.17067 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:56:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.488+0000 7f39de69e740 20 client.17067 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.488+0000 7f39de69e740 1 --2- 172.21.15.67:0/1962097860 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5607798699f0 0x560779c6e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.488+0000 7f39de69e740 4 client.17067 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:56:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.488+0000 7f39de69e740 1 -- 172.21.15.67:0/1962097860 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f39c0003e20 con 0x5607798699f0 2024-07-31T14:56:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.490+0000 7f39d5ee9640 1 --2- 172.21.15.67:0/1962097860 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5607798699f0 0x560779c6e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:47.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.496+0000 7f39d5ee9640 1 --2- 172.21.15.67:0/1962097860 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5607798699f0 0x560779c6e2f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e1159a3b1e34650 server_cookie=f0c3a615368e8a2b in_seq=0 out_seq=0 2024-07-31T14:56:47.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.496+0000 7f39b57fa640 10 client.17067 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:56:47.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.498+0000 7f39b57fa640 1 -- 172.21.15.67:0/1962097860 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f39c0003e20 con 0x5607798699f0 2024-07-31T14:56:47.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.498+0000 7f39b57fa640 10 client.17067 handle_command_reply: tid=0 2024-07-31T14:56:47.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.498+0000 7f39de69e740 1 client.17067 shutdown 2024-07-31T14:56:47.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.499+0000 7f39de69e740 1 -- 172.21.15.67:0/1962097860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 msgr2=0x560779c68040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:47.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.499+0000 7f39de69e740 1 --2- 172.21.15.67:0/1962097860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c68040 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009230 tx=0x7f39c4009260 comp rx=0 tx=0).stop 2024-07-31T14:56:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.499+0000 7f39de69e740 1 -- 172.21.15.67:0/1962097860 shutdown_connections 2024-07-31T14:56:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.499+0000 7f39de69e740 1 --2- 172.21.15.67:0/1962097860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c68040 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.499+0000 7f39de69e740 1 --2- 172.21.15.67:0/1962097860 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5607798699f0 0x560779c6e2f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.499+0000 7f39de69e740 1 --2- 172.21.15.67:0/1962097860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077988a390 0x560779c5c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.499+0000 7f39de69e740 1 --2- 172.21.15.67:0/1962097860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607798821b0 0x560779c5c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.499+0000 7f39de69e740 1 -- 172.21.15.67:0/1962097860 >> 172.21.15.67:0/1962097860 conn(0x56077989f460 msgr2=0x560779c53750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.499+0000 7f39b57fa640 0 client.17067 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:56:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.500+0000 7f39de69e740 1 -- 172.21.15.67:0/1962097860 shutdown_connections 2024-07-31T14:56:47.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.501+0000 7f39de69e740 1 -- 172.21.15.67:0/1962097860 wait complete. 2024-07-31T14:56:47.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.502+0000 7f39de69e740 20 client.17067 trim_cache size 0 max 16384 2024-07-31T14:56:47.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.531+0000 7f39de69e740 1 Processor -- start 2024-07-31T14:56:47.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.531+0000 7f39de69e740 1 -- start start 2024-07-31T14:56:47.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.532+0000 7f39de69e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798821b0 0x560779c53d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.532+0000 7f39de69e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077988a390 0x560779c5e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.532+0000 7f39d5ee9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798821b0 0x560779c53d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.532+0000 7f39de69e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607798ab070 0x560779c55650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.532+0000 7f39de69e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f39d0085090 con 0x5607798821b0 2024-07-31T14:56:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.532+0000 7f39d5ee9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798821b0 0x560779c53d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58038/0 (socket says 172.21.15.67:58038) 2024-07-31T14:56:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.532+0000 7f39d5ee9640 1 -- 172.21.15.67:0/894790374 learned_addr learned my addr 172.21.15.67:0/894790374 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.532+0000 7f39d66ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607798ab070 0x560779c55650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.532+0000 7f39de69e740 1 -- 172.21.15.67:0/894790374 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01d23a0 con 0x56077988a390 2024-07-31T14:56:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.533+0000 7f39de69e740 1 -- 172.21.15.67:0/894790374 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f39d00857f0 con 0x5607798ab070 2024-07-31T14:56:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.533+0000 7f39d66ea640 1 -- 172.21.15.67:0/894790374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077988a390 msgr2=0x560779c5e280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:56:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.533+0000 7f39d66ea640 1 --2- 172.21.15.67:0/894790374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077988a390 0x560779c5e280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.533+0000 7f39d66ea640 1 -- 172.21.15.67:0/894790374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798821b0 msgr2=0x560779c53d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:47.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.533+0000 7f39d66ea640 1 --2- 172.21.15.67:0/894790374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798821b0 0x560779c53d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.533+0000 7f39d66ea640 1 -- 172.21.15.67:0/894790374 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560779c578c0 con 0x5607798ab070 2024-07-31T14:56:47.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.534+0000 7f39d66ea640 1 --2- 172.21.15.67:0/894790374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607798ab070 0x560779c55650 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f39c4033040 tx=0x7f39c4004aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a339c450be2aa563 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:47.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.535+0000 7f39b77fe640 1 -- 172.21.15.67:0/894790374 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c402ae60 con 0x5607798ab070 2024-07-31T14:56:47.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.535+0000 7f39b77fe640 1 -- 172.21.15.67:0/894790374 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c403e810 con 0x5607798ab070 2024-07-31T14:56:47.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.536+0000 7f39de69e740 1 -- 172.21.15.67:0/894790374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607798ab070 msgr2=0x560779c55650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:47.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.536+0000 7f39de69e740 1 --2- 172.21.15.67:0/894790374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607798ab070 0x560779c55650 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f39c4033040 tx=0x7f39c4004aa0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.537+0000 7f39de69e740 1 -- 172.21.15.67:0/894790374 shutdown_connections 2024-07-31T14:56:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.537+0000 7f39de69e740 1 --2- 172.21.15.67:0/894790374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607798ab070 0x560779c55650 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.537+0000 7f39de69e740 1 --2- 172.21.15.67:0/894790374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077988a390 0x560779c5e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.537+0000 7f39de69e740 1 --2- 172.21.15.67:0/894790374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798821b0 0x560779c53d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.537+0000 7f39de69e740 1 -- 172.21.15.67:0/894790374 >> 172.21.15.67:0/894790374 conn(0x56077989f460 msgr2=0x560779c53670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.538+0000 7f39de69e740 1 -- 172.21.15.67:0/894790374 shutdown_connections 2024-07-31T14:56:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.538+0000 7f39de69e740 1 -- 172.21.15.67:0/894790374 wait complete. 2024-07-31T14:56:47.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.538+0000 7f39de69e740 1 -- start start 2024-07-31T14:56:47.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.539+0000 7f39de69e740 1 Processor -- start 2024-07-31T14:56:47.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.540+0000 7f39de69e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607798821b0 0x560779c647b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.540+0000 7f39de69e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56077988a390 0x560779c67ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.540+0000 7f39de69e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c68180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.540+0000 7f39de69e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01d1cd0 con 0x5607798ab070 2024-07-31T14:56:47.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.540+0000 7f39de69e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01d15f0 con 0x5607798821b0 2024-07-31T14:56:47.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.540+0000 7f39de69e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f39d010a820 con 0x56077988a390 2024-07-31T14:56:47.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.540+0000 7f39de69e740 10 client.0 _ll_register_callbacks cb 0x7f39c4009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:47.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.540+0000 7f39d66ea640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c68180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:47.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.540+0000 7f39d66ea640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c68180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58048/0 (socket says 172.21.15.67:58048) 2024-07-31T14:56:47.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.540+0000 7f39d66ea640 1 -- 172.21.15.67:0/568674665 learned_addr learned my addr 172.21.15.67:0/568674665 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:47.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.541+0000 7f39d66ea640 1 -- 172.21.15.67:0/568674665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56077988a390 msgr2=0x560779c67ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:56:47.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.541+0000 7f39d66ea640 1 --2- 172.21.15.67:0/568674665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56077988a390 0x560779c67ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.541+0000 7f39d66ea640 1 -- 172.21.15.67:0/568674665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607798821b0 msgr2=0x560779c647b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:56:47.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.541+0000 7f39d66ea640 1 --2- 172.21.15.67:0/568674665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607798821b0 0x560779c647b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.541+0000 7f39d66ea640 1 -- 172.21.15.67:0/568674665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560779c688f0 con 0x5607798ab070 2024-07-31T14:56:47.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.542+0000 7f39d66ea640 1 --2- 172.21.15.67:0/568674665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c68180 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f39c403e790 tx=0x7f39c40478a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a43b706be469ee server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:47.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.542+0000 7f39b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:56:47.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.542+0000 7f39b57fa640 1 -- 172.21.15.67:0/568674665 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c403edd0 con 0x5607798ab070 2024-07-31T14:56:47.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.543+0000 7f39de69e740 1 -- 172.21.15.67:0/568674665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39c403e810 con 0x5607798ab070 2024-07-31T14:56:47.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.543+0000 7f39b57fa640 1 -- 172.21.15.67:0/568674665 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f39c4052900 con 0x5607798ab070 2024-07-31T14:56:47.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.544+0000 7f39b57fa640 1 -- 172.21.15.67:0/568674665 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f39c4052b20 con 0x5607798ab070 2024-07-31T14:56:47.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.544+0000 7f39de69e740 10 client.17073 fetch_fsmap learned FSMap version 44 2024-07-31T14:56:47.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.544+0000 7f39de69e740 1 -- 172.21.15.67:0/568674665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560779cd6b40 con 0x5607798ab070 2024-07-31T14:56:47.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.545+0000 7f39b57fa640 1 -- 172.21.15.67:0/568674665 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c403e810 con 0x5607798ab070 2024-07-31T14:56:47.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.546+0000 7f39b57fa640 1 -- 172.21.15.67:0/568674665 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7f39c4050040 con 0x5607798ab070 2024-07-31T14:56:47.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.546+0000 7f39de69e740 10 client.17073 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:56:47.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.546+0000 7f39de69e740 10 client.17073 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:56:47.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.546+0000 7f39de69e740 20 client.17073 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:47.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.546+0000 7f39de69e740 1 --2- 172.21.15.67:0/568674665 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56077983eda0 0x560779cd91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:47.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.546+0000 7f39de69e740 4 client.17073 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:56:47.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.546+0000 7f39de69e740 1 -- 172.21.15.67:0/568674665 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560779bc8b00 con 0x56077983eda0 2024-07-31T14:56:47.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.552+0000 7f39d5ee9640 1 --2- 172.21.15.67:0/568674665 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56077983eda0 0x560779cd91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:47.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.559+0000 7f39d5ee9640 1 --2- 172.21.15.67:0/568674665 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56077983eda0 0x560779cd91c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3ca508fdfb76d2d server_cookie=ac4f53dce6db12d2 in_seq=0 out_seq=0 2024-07-31T14:56:47.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.559+0000 7f39b57fa640 10 client.17073 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:56:47.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.559+0000 7f39b57fa640 1 -- 172.21.15.67:0/568674665 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560779bc8b00 con 0x56077983eda0 2024-07-31T14:56:47.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.560+0000 7f39b57fa640 10 client.17073 handle_command_reply: tid=0 2024-07-31T14:56:47.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.560+0000 7f39de69e740 1 client.17073 shutdown 2024-07-31T14:56:47.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.561+0000 7f39de69e740 1 -- 172.21.15.67:0/568674665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 msgr2=0x560779c68180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:47.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.561+0000 7f39de69e740 1 --2- 172.21.15.67:0/568674665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c68180 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f39c403e790 tx=0x7f39c40478a0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.563+0000 7f39de69e740 1 -- 172.21.15.67:0/568674665 shutdown_connections 2024-07-31T14:56:47.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.563+0000 7f39de69e740 1 --2- 172.21.15.67:0/568674665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607798ab070 0x560779c68180 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.563+0000 7f39de69e740 1 --2- 172.21.15.67:0/568674665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56077988a390 0x560779c67ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.563+0000 7f39de69e740 1 --2- 172.21.15.67:0/568674665 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56077983eda0 0x560779cd91c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.563+0000 7f39de69e740 1 --2- 172.21.15.67:0/568674665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607798821b0 0x560779c647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.563+0000 7f39de69e740 1 -- 172.21.15.67:0/568674665 >> 172.21.15.67:0/568674665 conn(0x56077989f460 msgr2=0x560779c4cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:47.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.563+0000 7f39b57fa640 0 client.17073 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:56:47.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.564+0000 7f39de69e740 1 -- 172.21.15.67:0/568674665 shutdown_connections 2024-07-31T14:56:47.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.564+0000 7f39de69e740 1 -- 172.21.15.67:0/568674665 wait complete. 2024-07-31T14:56:47.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.564+0000 7f39de69e740 20 client.17073 trim_cache size 0 max 16384 2024-07-31T14:56:47.566 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:56:47.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.568+0000 7f39d814d640 1 -- 172.21.15.67:0/2975990565 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f39b80b3ed0 msgr2=0x7f39b80b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:47.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.568+0000 7f39d814d640 1 --2- 172.21.15.67:0/2975990565 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f39b80b3ed0 0x7f39b80b63e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f39c4005ab0 tx=0x7f39c4005a40 comp rx=0 tx=0).stop 2024-07-31T14:56:47.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.568+0000 7f39d814d640 1 -- 172.21.15.67:0/2975990565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d010a410 msgr2=0x7f39d01a8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:47.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.568+0000 7f39d814d640 1 --2- 172.21.15.67:0/2975990565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d010a410 0x7f39d01a8c50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f39c800d950 tx=0x7f39c800de10 comp rx=0 tx=0).stop 2024-07-31T14:56:47.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.570+0000 7f39d814d640 1 -- 172.21.15.67:0/2975990565 shutdown_connections 2024-07-31T14:56:47.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.570+0000 7f39d814d640 1 --2- 172.21.15.67:0/2975990565 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f39b80b3ed0 0x7f39b80b63e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.570+0000 7f39d814d640 1 --2- 172.21.15.67:0/2975990565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d011c660 0x7f39d01ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.570+0000 7f39d814d640 1 --2- 172.21.15.67:0/2975990565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39d0111970 0x7f39d01a91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.570+0000 7f39d814d640 1 --2- 172.21.15.67:0/2975990565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d010a410 0x7f39d01a8c50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:47.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.570+0000 7f39d814d640 1 -- 172.21.15.67:0/2975990565 >> 172.21.15.67:0/2975990565 conn(0x7f39d00708f0 msgr2=0x7f39d0071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:47.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.570+0000 7f39d814d640 1 -- 172.21.15.67:0/2975990565 shutdown_connections 2024-07-31T14:56:47.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:47.570+0000 7f39d814d640 1 -- 172.21.15.67:0/2975990565 wait complete. 2024-07-31T14:56:48.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:48 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2584187279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:56:48.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:48 smithi067 ceph-mon[32750]: pgmap v572: 129 pgs: 129 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.6 MiB/s wr, 2.11k op/s 2024-07-31T14:56:48.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:48 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2584187279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:56:48.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:48 smithi073 ceph-mon[36714]: pgmap v572: 129 pgs: 129 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.6 MiB/s wr, 2.11k op/s 2024-07-31T14:56:48.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:48 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2584187279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:56:48.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:48 smithi077 ceph-mon[37327]: pgmap v572: 129 pgs: 129 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.6 MiB/s wr, 2.11k op/s 2024-07-31T14:56:48.669 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T14:56:49.147 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-07-31T14:56:49.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-07-31T14:56:49.290 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:56:50.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:50 smithi073 ceph-mon[36714]: pgmap v573: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 2.38k op/s 2024-07-31T14:56:50.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:56:50.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:56:50.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:56:50.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:56:50.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:56:50.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:56:50.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:56:50.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:56:50.909 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:56:50.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:50 smithi067 ceph-mon[32750]: pgmap v573: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 2.38k op/s 2024-07-31T14:56:50.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:56:50.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:56:50.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:56:50.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:56:50.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:56:50.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:56:50.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:56:50.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:56:50.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:50 smithi077 ceph-mon[37327]: pgmap v573: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 2.38k op/s 2024-07-31T14:56:50.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:56:50.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:56:50.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:56:50.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:56:50.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-31T14:56:50.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:56:50.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:56:50.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:56:51.881 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:51 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-07-31T14:56:51.882 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:51 smithi067 ceph-mon[32750]: Unable to set osd_memory_target on smithi077 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-07-31T14:56:51.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:51 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-07-31T14:56:51.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:51 smithi073 ceph-mon[36714]: Unable to set osd_memory_target on smithi077 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-07-31T14:56:51.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.898+0000 7fc95b7cf640 1 -- 172.21.15.67:0/1519546527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9541035c0 msgr2=0x7fc9541039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:51.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.898+0000 7fc95b7cf640 1 --2- 172.21.15.67:0/1519546527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9541035c0 0x7fc9541039d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc944009ca0 tx=0x7fc94402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.898+0000 7fc95b7cf640 1 -- 172.21.15.67:0/1519546527 shutdown_connections 2024-07-31T14:56:51.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.898+0000 7fc95b7cf640 1 --2- 172.21.15.67:0/1519546527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc954109850 0x7fc954110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.898+0000 7fc95b7cf640 1 --2- 172.21.15.67:0/1519546527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc954108f00 0x7fc954109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.898+0000 7fc95b7cf640 1 --2- 172.21.15.67:0/1519546527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9541035c0 0x7fc9541039d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.898+0000 7fc95b7cf640 1 -- 172.21.15.67:0/1519546527 >> 172.21.15.67:0/1519546527 conn(0x7fc9540ff030 msgr2=0x7fc954101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:51.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.899+0000 7fc95b7cf640 1 -- 172.21.15.67:0/1519546527 shutdown_connections 2024-07-31T14:56:51.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.899+0000 7fc95b7cf640 1 -- 172.21.15.67:0/1519546527 wait complete. 2024-07-31T14:56:51.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.899+0000 7fc95b7cf640 1 Processor -- start 2024-07-31T14:56:51.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.899+0000 7fc95b7cf640 1 -- start start 2024-07-31T14:56:51.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc95b7cf640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9541035c0 0x7fc954199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc95b7cf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc954108f00 0x7fc95419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc95b7cf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc954109850 0x7fc95419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc95b7cf640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc954113520 con 0x7fc9541035c0 2024-07-31T14:56:51.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc95b7cf640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541133a0 con 0x7fc954108f00 2024-07-31T14:56:51.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc95b7cf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541136a0 con 0x7fc954109850 2024-07-31T14:56:51.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc95956b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9541035c0 0x7fc954199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc958d6a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc954108f00 0x7fc95419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc95956b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9541035c0 0x7fc954199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46104/0 (socket says 172.21.15.67:46104) 2024-07-31T14:56:51.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc95956b640 1 -- 172.21.15.67:0/1220040297 learned_addr learned my addr 172.21.15.67:0/1220040297 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:51.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc959d6c640 1 --2- 172.21.15.67:0/1220040297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc954109850 0x7fc95419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc958d6a640 1 -- 172.21.15.67:0/1220040297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc954109850 msgr2=0x7fc95419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:51.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc958d6a640 1 --2- 172.21.15.67:0/1220040297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc954109850 0x7fc95419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc958d6a640 1 -- 172.21.15.67:0/1220040297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9541035c0 msgr2=0x7fc954199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:51.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc958d6a640 1 --2- 172.21.15.67:0/1220040297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9541035c0 0x7fc954199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc958d6a640 1 -- 172.21.15.67:0/1220040297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95419fe40 con 0x7fc954108f00 2024-07-31T14:56:51.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.900+0000 7fc959d6c640 1 --2- 172.21.15.67:0/1220040297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc954109850 0x7fc95419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:56:51.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.901+0000 7fc958d6a640 1 --2- 172.21.15.67:0/1220040297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc954108f00 0x7fc95419a470 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc95406b770 tx=0x7fc94800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:51.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.901+0000 7fc9527fc640 1 -- 172.21.15.67:0/1220040297 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94800cc40 con 0x7fc954108f00 2024-07-31T14:56:51.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.901+0000 7fc95b7cf640 1 -- 172.21.15.67:0/1220040297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9541a0130 con 0x7fc954108f00 2024-07-31T14:56:51.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.901+0000 7fc9527fc640 1 -- 172.21.15.67:0/1220040297 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc94800cde0 con 0x7fc954108f00 2024-07-31T14:56:51.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.901+0000 7fc9527fc640 1 -- 172.21.15.67:0/1220040297 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948018760 con 0x7fc954108f00 2024-07-31T14:56:51.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.901+0000 7fc95b7cf640 1 -- 172.21.15.67:0/1220040297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc954075020 con 0x7fc954108f00 2024-07-31T14:56:51.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.904+0000 7fc961d20740 1 Processor -- start 2024-07-31T14:56:51.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.904+0000 7fc961d20740 1 -- start start 2024-07-31T14:56:51.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.904+0000 7fc961d20740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26d8070 0x5632f2a7d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.904+0000 7fc961d20740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26af1b0 0x5632f2a7dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.904+0000 7fc961d20740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 0x5632f2a7dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.904+0000 7fc961d20740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5632f29f3bb0 con 0x5632f26b7390 2024-07-31T14:56:51.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.904+0000 7fc961d20740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5632f29e30b0 con 0x5632f26d8070 2024-07-31T14:56:51.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.904+0000 7fc961d20740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5632f29e70f0 con 0x5632f26af1b0 2024-07-31T14:56:51.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.904+0000 7fc959d6c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26d8070 0x5632f2a7d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.904+0000 7fc959d6c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26d8070 0x5632f2a7d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:42654/0 (socket says 172.21.15.67:42654) 2024-07-31T14:56:51.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.904+0000 7fc959d6c640 1 -- 172.21.15.67:0/3652944153 learned_addr learned my addr 172.21.15.67:0/3652944153 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:51.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.904+0000 7fc95956b640 1 --2- 172.21.15.67:0/3652944153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26af1b0 0x5632f2a7dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.905+0000 7fc959d6c640 1 -- 172.21.15.67:0/3652944153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26af1b0 msgr2=0x5632f2a7dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:51.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.905+0000 7fc959d6c640 1 --2- 172.21.15.67:0/3652944153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26af1b0 0x5632f2a7dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.905+0000 7fc959d6c640 1 -- 172.21.15.67:0/3652944153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 msgr2=0x5632f2a7dff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:56:51.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.905+0000 7fc959d6c640 1 --2- 172.21.15.67:0/3652944153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 0x5632f2a7dff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.905+0000 7fc959d6c640 1 -- 172.21.15.67:0/3652944153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632f2989b90 con 0x5632f26d8070 2024-07-31T14:56:51.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.905+0000 7fc9527fc640 1 -- 172.21.15.67:0/1220040297 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc948010a90 con 0x7fc954108f00 2024-07-31T14:56:51.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.905+0000 7fc95956b640 1 --2- 172.21.15.67:0/3652944153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26af1b0 0x5632f2a7dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:56:51.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.905+0000 7fc959d6c640 1 --2- 172.21.15.67:0/3652944153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26d8070 0x5632f2a7d570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009bb0 tx=0x7fc94c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6049b79b8e455076 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:51.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.905+0000 7fc93b7fe640 1 -- 172.21.15.67:0/3652944153 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c004020 con 0x5632f26d8070 2024-07-31T14:56:51.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.905+0000 7fc93b7fe640 1 -- 172.21.15.67:0/3652944153 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc94c0041c0 con 0x5632f26d8070 2024-07-31T14:56:51.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.905+0000 7fc93b7fe640 1 -- 172.21.15.67:0/3652944153 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0126b0 con 0x5632f26d8070 2024-07-31T14:56:51.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.906+0000 7fc9527fc640 1 --2- 172.21.15.67:0/1220040297 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc93c0b3d40 0x7fc93c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.906+0000 7fc95956b640 1 --2- 172.21.15.67:0/1220040297 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc93c0b3d40 0x7fc93c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.906+0000 7fc961d20740 1 -- 172.21.15.67:0/3652944153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26d8070 msgr2=0x5632f2a7d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:51.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.906+0000 7fc961d20740 1 --2- 172.21.15.67:0/3652944153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26d8070 0x5632f2a7d570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009bb0 tx=0x7fc94c010740 comp rx=0 tx=0).stop 2024-07-31T14:56:51.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.906+0000 7fc9527fc640 1 -- 172.21.15.67:0/1220040297 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fc9480e4670 con 0x7fc954108f00 2024-07-31T14:56:51.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.906+0000 7fc961d20740 1 -- 172.21.15.67:0/3652944153 shutdown_connections 2024-07-31T14:56:51.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.906+0000 7fc961d20740 1 --2- 172.21.15.67:0/3652944153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 0x5632f2a7dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.906+0000 7fc961d20740 1 --2- 172.21.15.67:0/3652944153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26af1b0 0x5632f2a7dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.906+0000 7fc961d20740 1 --2- 172.21.15.67:0/3652944153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26d8070 0x5632f2a7d570 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.906+0000 7fc961d20740 1 -- 172.21.15.67:0/3652944153 >> 172.21.15.67:0/3652944153 conn(0x5632f26cc460 msgr2=0x5632f2a7b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.906+0000 7fc961d20740 1 -- 172.21.15.67:0/3652944153 shutdown_connections 2024-07-31T14:56:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.906+0000 7fc961d20740 1 -- 172.21.15.67:0/3652944153 wait complete. 2024-07-31T14:56:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.907+0000 7fc95956b640 1 --2- 172.21.15.67:0/1220040297 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc93c0b3d40 0x7fc93c0b6250 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc94402f500 tx=0x7fc94402f310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.907+0000 7fc961d20740 1 -- start start 2024-07-31T14:56:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.907+0000 7fc961d20740 1 Processor -- start 2024-07-31T14:56:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.908+0000 7fc961d20740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26af1b0 0x5632f2a89160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.908+0000 7fc961d20740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 0x5632f2a896e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.908+0000 7fc961d20740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26d8070 0x5632f2a95070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.908+0000 7fc95956b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26af1b0 0x5632f2a89160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.908+0000 7fc95956b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26af1b0 0x5632f2a89160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:42656/0 (socket says 172.21.15.67:42656) 2024-07-31T14:56:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.908+0000 7fc95956b640 1 -- 172.21.15.67:0/1333080308 learned_addr learned my addr 172.21.15.67:0/1333080308 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.908+0000 7fc961d20740 1 -- 172.21.15.67:0/1333080308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5632f2846700 con 0x5632f26b7390 2024-07-31T14:56:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.908+0000 7fc961d20740 1 -- 172.21.15.67:0/1333080308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5632f29859e0 con 0x5632f26af1b0 2024-07-31T14:56:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.908+0000 7fc961d20740 1 -- 172.21.15.67:0/1333080308 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5632f2a8bd50 con 0x5632f26d8070 2024-07-31T14:56:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.908+0000 7fc959d6c640 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26d8070 0x5632f2a95070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.908+0000 7fc961d20740 10 client.0 _ll_register_callbacks cb 0x5632f29a04f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc95956b640 1 -- 172.21.15.67:0/1333080308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26d8070 msgr2=0x5632f2a95070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc95956b640 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26d8070 0x5632f2a95070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc95956b640 1 -- 172.21.15.67:0/1333080308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 msgr2=0x5632f2a896e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc958d6a640 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 0x5632f2a896e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc95956b640 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 0x5632f2a896e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc95956b640 1 -- 172.21.15.67:0/1333080308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632f2743650 con 0x5632f26af1b0 2024-07-31T14:56:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc961d20740 1 -- 172.21.15.67:0/1333080308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc94c0041c0 con 0x5632f26af1b0 2024-07-31T14:56:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc958d6a640 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 0x5632f2a896e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:56:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc959d6c640 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26d8070 0x5632f2a95070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc95956b640 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26af1b0 0x5632f2a89160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc944006010 tx=0x7fc94402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed6544adf16ea9f1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc938ff9640 10 client.26657 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:56:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc938ff9640 1 -- 172.21.15.67:0/1333080308 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944003b80 con 0x5632f26af1b0 2024-07-31T14:56:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc938ff9640 1 -- 172.21.15.67:0/1333080308 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fc944003d20 con 0x5632f26af1b0 2024-07-31T14:56:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc938ff9640 1 -- 172.21.15.67:0/1333080308 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc944004040 con 0x5632f26af1b0 2024-07-31T14:56:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc961d20740 10 client.26657 fetch_fsmap learned FSMap version 44 2024-07-31T14:56:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.909+0000 7fc961d20740 1 -- 172.21.15.67:0/1333080308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=44}) -- 0x5632f2a97d90 con 0x5632f26af1b0 2024-07-31T14:56:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.910+0000 7fc938ff9640 1 -- 172.21.15.67:0/1333080308 <== mon.2 v2:172.21.15.73:3300/0 4 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7fc94405e060 con 0x5632f26af1b0 2024-07-31T14:56:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.910+0000 7fc961d20740 10 client.26657 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:56:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.910+0000 7fc961d20740 10 client.26657 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:56:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.910+0000 7fc961d20740 20 client.26657 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.910+0000 7fc961d20740 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632f26969f0 0x5632f2a9b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.910+0000 7fc961d20740 4 client.26657 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:56:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.910+0000 7fc961d20740 1 -- 172.21.15.67:0/1333080308 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632f29f5b00 con 0x5632f26969f0 2024-07-31T14:56:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.910+0000 7fc958d6a640 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632f26969f0 0x5632f2a9b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.911+0000 7fc958d6a640 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632f26969f0 0x5632f2a9b290 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ec5dd70658357b5 server_cookie=f46e8d553a4b6814 in_seq=0 out_seq=0 2024-07-31T14:56:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.911+0000 7fc938ff9640 10 client.26657 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:56:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.911+0000 7fc938ff9640 1 -- 172.21.15.67:0/1333080308 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5632f29f5b00 con 0x5632f26969f0 2024-07-31T14:56:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.911+0000 7fc938ff9640 10 client.26657 handle_command_reply: tid=0 2024-07-31T14:56:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.911+0000 7fc961d20740 1 client.26657 shutdown 2024-07-31T14:56:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.911+0000 7fc961d20740 1 -- 172.21.15.67:0/1333080308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26af1b0 msgr2=0x5632f2a89160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.911+0000 7fc961d20740 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26af1b0 0x5632f2a89160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc944006010 tx=0x7fc94402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.911+0000 7fc961d20740 1 -- 172.21.15.67:0/1333080308 shutdown_connections 2024-07-31T14:56:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.911+0000 7fc961d20740 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632f26969f0 0x5632f2a9b290 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.911+0000 7fc961d20740 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26d8070 0x5632f2a95070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.911+0000 7fc938ff9640 0 client.26657 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:56:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.911+0000 7fc961d20740 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 0x5632f2a896e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.911+0000 7fc961d20740 1 --2- 172.21.15.67:0/1333080308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26af1b0 0x5632f2a89160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.911+0000 7fc961d20740 1 -- 172.21.15.67:0/1333080308 >> 172.21.15.67:0/1333080308 conn(0x5632f26cc460 msgr2=0x5632f2a79c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.912+0000 7fc961d20740 1 -- 172.21.15.67:0/1333080308 shutdown_connections 2024-07-31T14:56:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.912+0000 7fc961d20740 1 -- 172.21.15.67:0/1333080308 wait complete. 2024-07-31T14:56:51.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.912+0000 7fc961d20740 20 client.26657 trim_cache size 0 max 16384 2024-07-31T14:56:51.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.926+0000 7fc961d20740 1 Processor -- start 2024-07-31T14:56:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.926+0000 7fc961d20740 1 -- start start 2024-07-31T14:56:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.926+0000 7fc961d20740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26af1b0 0x5632f2a7a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.926+0000 7fc961d20740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26b7390 0x5632f2a7ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.926+0000 7fc961d20740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26d8070 0x5632f2a7b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.926+0000 7fc961d20740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541b2780 con 0x5632f26af1b0 2024-07-31T14:56:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.926+0000 7fc961d20740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541b44d0 con 0x5632f26d8070 2024-07-31T14:56:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.926+0000 7fc961d20740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541b2ee0 con 0x5632f26b7390 2024-07-31T14:56:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.926+0000 7fc959d6c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26d8070 0x5632f2a7b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.926+0000 7fc958d6a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26b7390 0x5632f2a7ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.926+0000 7fc95956b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26af1b0 0x5632f2a7a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.926+0000 7fc958d6a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26b7390 0x5632f2a7ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37440/0 (socket says 172.21.15.67:37440) 2024-07-31T14:56:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.926+0000 7fc959d6c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26d8070 0x5632f2a7b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:42660/0 (socket says 172.21.15.67:42660) 2024-07-31T14:56:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.926+0000 7fc958d6a640 1 -- 172.21.15.67:0/3898380638 learned_addr learned my addr 172.21.15.67:0/3898380638 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:51.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.927+0000 7fc958d6a640 1 -- 172.21.15.67:0/3898380638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26d8070 msgr2=0x5632f2a7b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:51.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.927+0000 7fc958d6a640 1 --2- 172.21.15.67:0/3898380638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26d8070 0x5632f2a7b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.927+0000 7fc958d6a640 1 -- 172.21.15.67:0/3898380638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26af1b0 msgr2=0x5632f2a7a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:51.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.927+0000 7fc958d6a640 1 --2- 172.21.15.67:0/3898380638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26af1b0 0x5632f2a7a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.927+0000 7fc958d6a640 1 -- 172.21.15.67:0/3898380638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632f2a8b7e0 con 0x5632f26b7390 2024-07-31T14:56:51.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.927+0000 7fc958d6a640 1 --2- 172.21.15.67:0/3898380638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26b7390 0x5632f2a7ae50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc9480a6ff0 tx=0x7fc9480a6f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2e141e4d9fda1d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:51.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.927+0000 7fc93affd640 1 -- 172.21.15.67:0/3898380638 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9480b4020 con 0x5632f26b7390 2024-07-31T14:56:51.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.927+0000 7fc93affd640 1 -- 172.21.15.67:0/3898380638 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9480e1d30 con 0x5632f26b7390 2024-07-31T14:56:51.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.927+0000 7fc93affd640 1 -- 172.21.15.67:0/3898380638 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9480a6b20 con 0x5632f26b7390 2024-07-31T14:56:51.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.928+0000 7fc961d20740 1 -- 172.21.15.67:0/3898380638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26b7390 msgr2=0x5632f2a7ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:51.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.928+0000 7fc961d20740 1 --2- 172.21.15.67:0/3898380638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26b7390 0x5632f2a7ae50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc9480a6ff0 tx=0x7fc9480a6f50 comp rx=0 tx=0).stop 2024-07-31T14:56:51.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.928+0000 7fc961d20740 1 -- 172.21.15.67:0/3898380638 shutdown_connections 2024-07-31T14:56:51.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.928+0000 7fc961d20740 1 --2- 172.21.15.67:0/3898380638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26d8070 0x5632f2a7b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.928+0000 7fc961d20740 1 --2- 172.21.15.67:0/3898380638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26b7390 0x5632f2a7ae50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.928+0000 7fc961d20740 1 --2- 172.21.15.67:0/3898380638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26af1b0 0x5632f2a7a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.928+0000 7fc961d20740 1 -- 172.21.15.67:0/3898380638 >> 172.21.15.67:0/3898380638 conn(0x5632f26cc460 msgr2=0x5632f2a79c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:51.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.928+0000 7fc961d20740 1 -- 172.21.15.67:0/3898380638 shutdown_connections 2024-07-31T14:56:51.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.928+0000 7fc961d20740 1 -- 172.21.15.67:0/3898380638 wait complete. 2024-07-31T14:56:51.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.929+0000 7fc961d20740 1 -- start start 2024-07-31T14:56:51.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.929+0000 7fc961d20740 1 Processor -- start 2024-07-31T14:56:51.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.929+0000 7fc961d20740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26af1b0 0x5632f2a91740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.929+0000 7fc961d20740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 0x5632f2a8cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.929+0000 7fc961d20740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26d8070 0x5632f2a8d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.929+0000 7fc961d20740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541b3e00 con 0x5632f26b7390 2024-07-31T14:56:51.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.929+0000 7fc961d20740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541b3670 con 0x5632f26af1b0 2024-07-31T14:56:51.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.929+0000 7fc961d20740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541b07b0 con 0x5632f26d8070 2024-07-31T14:56:51.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.929+0000 7fc961d20740 10 client.0 _ll_register_callbacks cb 0x7fc94c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:51.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.929+0000 7fc958d6a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 0x5632f2a8cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.929+0000 7fc95956b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26af1b0 0x5632f2a91740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.929+0000 7fc958d6a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 0x5632f2a8cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46126/0 (socket says 172.21.15.67:46126) 2024-07-31T14:56:51.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.929+0000 7fc958d6a640 1 -- 172.21.15.67:0/3657617027 learned_addr learned my addr 172.21.15.67:0/3657617027 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:51.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.929+0000 7fc959d6c640 1 --2- 172.21.15.67:0/3657617027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26d8070 0x5632f2a8d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc958d6a640 1 -- 172.21.15.67:0/3657617027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26d8070 msgr2=0x5632f2a8d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:51.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc958d6a640 1 --2- 172.21.15.67:0/3657617027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26d8070 0x5632f2a8d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc958d6a640 1 -- 172.21.15.67:0/3657617027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26af1b0 msgr2=0x5632f2a91740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:51.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc958d6a640 1 --2- 172.21.15.67:0/3657617027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26af1b0 0x5632f2a91740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc958d6a640 1 -- 172.21.15.67:0/3657617027 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632f2a8d9a0 con 0x5632f26b7390 2024-07-31T14:56:51.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc95956b640 1 --2- 172.21.15.67:0/3657617027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26af1b0 0x5632f2a91740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:51.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc959d6c640 1 --2- 172.21.15.67:0/3657617027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26d8070 0x5632f2a8d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:51.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc958d6a640 1 --2- 172.21.15.67:0/3657617027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 0x5632f2a8cc40 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc94800c6a0 tx=0x7fc9480a7410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ce1d4a3292f399 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:51.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc938ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:56:51.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc938ff9640 1 -- 172.21.15.67:0/3657617027 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9480c4020 con 0x5632f26b7390 2024-07-31T14:56:51.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc961d20740 1 -- 172.21.15.67:0/3657617027 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9480e1d30 con 0x5632f26b7390 2024-07-31T14:56:51.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc938ff9640 1 -- 172.21.15.67:0/3657617027 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fc9480aa1d0 con 0x5632f26b7390 2024-07-31T14:56:51.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc938ff9640 1 -- 172.21.15.67:0/3657617027 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc9480aa3d0 con 0x5632f26b7390 2024-07-31T14:56:51.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc961d20740 10 client.17085 fetch_fsmap learned FSMap version 44 2024-07-31T14:56:51.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.930+0000 7fc961d20740 1 -- 172.21.15.67:0/3657617027 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5632f2b03a40 con 0x5632f26b7390 2024-07-31T14:56:51.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.931+0000 7fc938ff9640 1 -- 172.21.15.67:0/3657617027 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9480e1d30 con 0x5632f26b7390 2024-07-31T14:56:51.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.931+0000 7fc938ff9640 1 -- 172.21.15.67:0/3657617027 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7fc9480a9e60 con 0x5632f26b7390 2024-07-31T14:56:51.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.931+0000 7fc961d20740 10 client.17085 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:56:51.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.931+0000 7fc961d20740 10 client.17085 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:56:51.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.931+0000 7fc961d20740 20 client.17085 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:51.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.931+0000 7fc961d20740 1 --2- 172.21.15.67:0/3657617027 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632f266bda0 0x5632f2b061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:51.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.931+0000 7fc961d20740 4 client.17085 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:56:51.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.931+0000 7fc961d20740 1 -- 172.21.15.67:0/3657617027 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632f29f15a0 con 0x5632f266bda0 2024-07-31T14:56:51.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.931+0000 7fc95956b640 1 --2- 172.21.15.67:0/3657617027 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632f266bda0 0x5632f2b061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:51.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.931+0000 7fc95956b640 1 --2- 172.21.15.67:0/3657617027 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632f266bda0 0x5632f2b061b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72e30b47e468d8e7 server_cookie=9d6da0449ab60ff9 in_seq=0 out_seq=0 2024-07-31T14:56:51.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.931+0000 7fc938ff9640 10 client.17085 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:56:51.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc938ff9640 1 -- 172.21.15.67:0/3657617027 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632f29f15a0 con 0x5632f266bda0 2024-07-31T14:56:51.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc938ff9640 10 client.17085 handle_command_reply: tid=0 2024-07-31T14:56:51.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc961d20740 1 client.17085 shutdown 2024-07-31T14:56:51.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc961d20740 1 -- 172.21.15.67:0/3657617027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 msgr2=0x5632f2a8cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:51.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc961d20740 1 --2- 172.21.15.67:0/3657617027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 0x5632f2a8cc40 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc94800c6a0 tx=0x7fc9480a7410 comp rx=0 tx=0).stop 2024-07-31T14:56:51.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc961d20740 1 -- 172.21.15.67:0/3657617027 shutdown_connections 2024-07-31T14:56:51.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc961d20740 1 --2- 172.21.15.67:0/3657617027 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632f266bda0 0x5632f2b061b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc961d20740 1 --2- 172.21.15.67:0/3657617027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632f26d8070 0x5632f2a8d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc938ff9640 0 client.17085 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:56:51.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc961d20740 1 --2- 172.21.15.67:0/3657617027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632f26b7390 0x5632f2a8cc40 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc961d20740 1 --2- 172.21.15.67:0/3657617027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f26af1b0 0x5632f2a91740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc961d20740 1 -- 172.21.15.67:0/3657617027 >> 172.21.15.67:0/3657617027 conn(0x5632f26cc460 msgr2=0x5632f2a7adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:51.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc961d20740 1 -- 172.21.15.67:0/3657617027 shutdown_connections 2024-07-31T14:56:51.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc961d20740 1 -- 172.21.15.67:0/3657617027 wait complete. 2024-07-31T14:56:51.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.932+0000 7fc961d20740 20 client.17085 trim_cache size 0 max 16384 2024-07-31T14:56:51.939 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:56:51.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.933+0000 7fc95b7cf640 1 -- 172.21.15.67:0/1220040297 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc93c0b3d40 msgr2=0x7fc93c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:51.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.933+0000 7fc95b7cf640 1 --2- 172.21.15.67:0/1220040297 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc93c0b3d40 0x7fc93c0b6250 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc94402f500 tx=0x7fc94402f310 comp rx=0 tx=0).stop 2024-07-31T14:56:51.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.933+0000 7fc95b7cf640 1 -- 172.21.15.67:0/1220040297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc954108f00 msgr2=0x7fc95419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:51.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.933+0000 7fc95b7cf640 1 --2- 172.21.15.67:0/1220040297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc954108f00 0x7fc95419a470 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc95406b770 tx=0x7fc94800ee50 comp rx=0 tx=0).stop 2024-07-31T14:56:51.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.933+0000 7fc95b7cf640 1 -- 172.21.15.67:0/1220040297 shutdown_connections 2024-07-31T14:56:51.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.933+0000 7fc95b7cf640 1 --2- 172.21.15.67:0/1220040297 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc93c0b3d40 0x7fc93c0b6250 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.933+0000 7fc95b7cf640 1 --2- 172.21.15.67:0/1220040297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc954109850 0x7fc95419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.933+0000 7fc95b7cf640 1 --2- 172.21.15.67:0/1220040297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc954108f00 0x7fc95419a470 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.933+0000 7fc95b7cf640 1 --2- 172.21.15.67:0/1220040297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9541035c0 0x7fc954199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:51.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.933+0000 7fc95b7cf640 1 -- 172.21.15.67:0/1220040297 >> 172.21.15.67:0/1220040297 conn(0x7fc9540ff030 msgr2=0x7fc954101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:51.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.933+0000 7fc95b7cf640 1 -- 172.21.15.67:0/1220040297 shutdown_connections 2024-07-31T14:56:51.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:51.933+0000 7fc95b7cf640 1 -- 172.21.15.67:0/1220040297 wait complete. 2024-07-31T14:56:51.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:51 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-07-31T14:56:51.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:51 smithi077 ceph-mon[37327]: Unable to set osd_memory_target on smithi077 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-07-31T14:56:52.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:52 smithi067 ceph-mon[32750]: pgmap v574: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 2.34k op/s 2024-07-31T14:56:52.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:52 smithi073 ceph-mon[36714]: pgmap v574: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 2.34k op/s 2024-07-31T14:56:52.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:52 smithi077 ceph-mon[37327]: pgmap v574: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 2.34k op/s 2024-07-31T14:56:53.456 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T14:56:53.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.727+0000 7f528380b640 1 -- 172.21.15.67:0/3722182671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c107df0 msgr2=0x7f527c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:53.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.727+0000 7f528380b640 1 --2- 172.21.15.67:0/3722182671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c107df0 0x7f527c108200 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f526c009900 tx=0x7f526c02a410 comp rx=0 tx=0).stop 2024-07-31T14:56:53.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.727+0000 7f528380b640 1 -- 172.21.15.67:0/3722182671 shutdown_connections 2024-07-31T14:56:53.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.727+0000 7f528380b640 1 --2- 172.21.15.67:0/3722182671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f527c108740 0x7f527c10ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.727+0000 7f528380b640 1 --2- 172.21.15.67:0/3722182671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c107df0 0x7f527c108200 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.727+0000 7f528380b640 1 --2- 172.21.15.67:0/3722182671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f527c1024b0 0x7f527c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.727+0000 7f528380b640 1 -- 172.21.15.67:0/3722182671 >> 172.21.15.67:0/3722182671 conn(0x7f527c079190 msgr2=0x7f527c0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:53.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.727+0000 7f528380b640 1 -- 172.21.15.67:0/3722182671 shutdown_connections 2024-07-31T14:56:53.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.727+0000 7f528380b640 1 -- 172.21.15.67:0/3722182671 wait complete. 2024-07-31T14:56:53.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.728+0000 7f528380b640 1 Processor -- start 2024-07-31T14:56:53.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.728+0000 7f528380b640 1 -- start start 2024-07-31T14:56:53.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.728+0000 7f528380b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f527c1024b0 0x7f527c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.728+0000 7f528380b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c107df0 0x7f527c19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.728+0000 7f528380b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f527c108740 0x7f527c19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.728+0000 7f528380b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f527c112390 con 0x7f527c107df0 2024-07-31T14:56:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.728+0000 7f528380b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f527c112210 con 0x7f527c1024b0 2024-07-31T14:56:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.728+0000 7f528380b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f527c112510 con 0x7f527c108740 2024-07-31T14:56:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.728+0000 7f52815a7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f527c1024b0 0x7f527c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.728+0000 7f5280da6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c107df0 0x7f527c19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.728+0000 7f52815a7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f527c1024b0 0x7f527c19ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:42686/0 (socket says 172.21.15.67:42686) 2024-07-31T14:56:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.728+0000 7f52815a7640 1 -- 172.21.15.67:0/740627995 learned_addr learned my addr 172.21.15.67:0/740627995 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.728+0000 7f5281da8640 1 --2- 172.21.15.67:0/740627995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f527c108740 0x7f527c19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.729+0000 7f52815a7640 1 -- 172.21.15.67:0/740627995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f527c108740 msgr2=0x7f527c19b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.729+0000 7f52815a7640 1 --2- 172.21.15.67:0/740627995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f527c108740 0x7f527c19b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.729+0000 7f52815a7640 1 -- 172.21.15.67:0/740627995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c107df0 msgr2=0x7f527c19b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.729+0000 7f52815a7640 1 --2- 172.21.15.67:0/740627995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c107df0 0x7f527c19b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.729+0000 7f52815a7640 1 -- 172.21.15.67:0/740627995 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527c1a6010 con 0x7f527c1024b0 2024-07-31T14:56:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.729+0000 7f52815a7640 1 --2- 172.21.15.67:0/740627995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f527c1024b0 0x7f527c19ad80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f527800d770 tx=0x7f527800db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:53.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.729+0000 7f52727fc640 1 -- 172.21.15.67:0/740627995 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278017070 con 0x7f527c1024b0 2024-07-31T14:56:53.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.729+0000 7f528380b640 1 -- 172.21.15.67:0/740627995 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527c1a6300 con 0x7f527c1024b0 2024-07-31T14:56:53.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.729+0000 7f528380b640 1 -- 172.21.15.67:0/740627995 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f527c1a6810 con 0x7f527c1024b0 2024-07-31T14:56:53.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.729+0000 7f52727fc640 1 -- 172.21.15.67:0/740627995 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5278009820 con 0x7f527c1024b0 2024-07-31T14:56:53.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.729+0000 7f52727fc640 1 -- 172.21.15.67:0/740627995 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278004c20 con 0x7f527c1024b0 2024-07-31T14:56:53.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.732+0000 7f5289d5c740 1 Processor -- start 2024-07-31T14:56:53.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.732+0000 7f5289d5c740 1 -- start start 2024-07-31T14:56:53.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.732+0000 7f52727fc640 1 -- 172.21.15.67:0/740627995 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5278010460 con 0x7f527c1024b0 2024-07-31T14:56:53.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5289d5c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedc0a840 0x556aedfb0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5289d5c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbe1980 0x556aedfb0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5289d5c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe9b60 0x556aedfb12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5289d5c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556aedf25780 con 0x556aedbe9b60 2024-07-31T14:56:53.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5289d5c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556aedf14c80 con 0x556aedbe1980 2024-07-31T14:56:53.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5289d5c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556aedf18cc0 con 0x556aedc0a840 2024-07-31T14:56:53.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5280da6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe9b60 0x556aedfb12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f52815a7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbe1980 0x556aedfb0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5281da8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedc0a840 0x556aedfb0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5280da6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe9b60 0x556aedfb12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46146/0 (socket says 172.21.15.67:46146) 2024-07-31T14:56:53.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5280da6640 1 -- 172.21.15.67:0/3683607654 learned_addr learned my addr 172.21.15.67:0/3683607654 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:53.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5281da8640 1 -- 172.21.15.67:0/3683607654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbe1980 msgr2=0x556aedfb0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:53.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5281da8640 1 --2- 172.21.15.67:0/3683607654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbe1980 0x556aedfb0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5281da8640 1 -- 172.21.15.67:0/3683607654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe9b60 msgr2=0x556aedfb12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:53.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5281da8640 1 --2- 172.21.15.67:0/3683607654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe9b60 0x556aedfb12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.733+0000 7f5281da8640 1 -- 172.21.15.67:0/3683607654 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aedeaa080 con 0x556aedc0a840 2024-07-31T14:56:53.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.734+0000 7f52727fc640 1 --2- 172.21.15.67:0/740627995 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f52680b3df0 0x7f52680b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.734+0000 7f5281da8640 1 --2- 172.21.15.67:0/3683607654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedc0a840 0x556aedfb0840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5274009f10 tx=0x7f5274011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c2a688097bdffe6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:53.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.734+0000 7f52815a7640 1 --2- 172.21.15.67:0/740627995 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f52680b3df0 0x7f52680b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.734+0000 7f52677fe640 1 -- 172.21.15.67:0/3683607654 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274004d20 con 0x556aedc0a840 2024-07-31T14:56:53.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.734+0000 7f52677fe640 1 -- 172.21.15.67:0/3683607654 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5274004030 con 0x556aedc0a840 2024-07-31T14:56:53.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.734+0000 7f52677fe640 1 -- 172.21.15.67:0/3683607654 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274005280 con 0x556aedc0a840 2024-07-31T14:56:53.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.734+0000 7f52727fc640 1 -- 172.21.15.67:0/740627995 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f52780e3a00 con 0x7f527c1024b0 2024-07-31T14:56:53.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.735+0000 7f5289d5c740 1 -- 172.21.15.67:0/3683607654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedc0a840 msgr2=0x556aedfb0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.735+0000 7f5289d5c740 1 --2- 172.21.15.67:0/3683607654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedc0a840 0x556aedfb0840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5274009f10 tx=0x7f5274011810 comp rx=0 tx=0).stop 2024-07-31T14:56:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.735+0000 7f52815a7640 1 --2- 172.21.15.67:0/740627995 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f52680b3df0 0x7f52680b6300 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x556aedfaf4c0 tx=0x7f52780e9510 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.735+0000 7f5289d5c740 1 -- 172.21.15.67:0/3683607654 shutdown_connections 2024-07-31T14:56:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.735+0000 7f5289d5c740 1 --2- 172.21.15.67:0/3683607654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe9b60 0x556aedfb12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.735+0000 7f5289d5c740 1 --2- 172.21.15.67:0/3683607654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbe1980 0x556aedfb0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.735+0000 7f5289d5c740 1 --2- 172.21.15.67:0/3683607654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedc0a840 0x556aedfb0840 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.735+0000 7f5289d5c740 1 -- 172.21.15.67:0/3683607654 >> 172.21.15.67:0/3683607654 conn(0x556aedbfe990 msgr2=0x556aedfae730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.735+0000 7f5289d5c740 1 -- 172.21.15.67:0/3683607654 shutdown_connections 2024-07-31T14:56:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.735+0000 7f5289d5c740 1 -- 172.21.15.67:0/3683607654 wait complete. 2024-07-31T14:56:53.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.736+0000 7f5289d5c740 1 -- start start 2024-07-31T14:56:53.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.736+0000 7f5289d5c740 1 Processor -- start 2024-07-31T14:56:53.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.736+0000 7f5289d5c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe1980 0x556aedfc0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.736+0000 7f5289d5c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbe9b60 0x556aedfc0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.736+0000 7f5289d5c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedc0a840 0x556aedfc0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.736+0000 7f5289d5c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556aedd787a0 con 0x556aedbe1980 2024-07-31T14:56:53.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.736+0000 7f5289d5c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556aedeb72f0 con 0x556aedbe9b60 2024-07-31T14:56:53.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.736+0000 7f5289d5c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556aedfc3380 con 0x556aedc0a840 2024-07-31T14:56:53.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.737+0000 7f5280da6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbe9b60 0x556aedfc0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.737+0000 7f5280da6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbe9b60 0x556aedfc0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:42704/0 (socket says 172.21.15.67:42704) 2024-07-31T14:56:53.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.737+0000 7f5280da6640 1 -- 172.21.15.67:0/4094565513 learned_addr learned my addr 172.21.15.67:0/4094565513 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:53.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.736+0000 7f5289d5c740 10 client.0 _ll_register_callbacks cb 0x556aedec1110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:53.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.737+0000 7f5281da8640 1 --2- 172.21.15.67:0/4094565513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe1980 0x556aedfc0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.737+0000 7f5280da6640 1 -- 172.21.15.67:0/4094565513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedc0a840 msgr2=0x556aedfc0ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:56:53.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.737+0000 7f5280da6640 1 --2- 172.21.15.67:0/4094565513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedc0a840 0x556aedfc0ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.737+0000 7f5280da6640 1 -- 172.21.15.67:0/4094565513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe1980 msgr2=0x556aedfc0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:53.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.737+0000 7f5280da6640 1 --2- 172.21.15.67:0/4094565513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe1980 0x556aedfc0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.737+0000 7f5280da6640 1 -- 172.21.15.67:0/4094565513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aedebb480 con 0x556aedbe9b60 2024-07-31T14:56:53.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.737+0000 7f5281da8640 1 --2- 172.21.15.67:0/4094565513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe1980 0x556aedfc0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:53.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.737+0000 7f5280da6640 1 --2- 172.21.15.67:0/4094565513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbe9b60 0x556aedfc0760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f526c032e70 tx=0x7f526c032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa370174d9a8069c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:53.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.737+0000 7f5264ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:56:53.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.738+0000 7f5264ff9640 1 -- 172.21.15.67:0/4094565513 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c035040 con 0x556aedbe9b60 2024-07-31T14:56:53.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.738+0000 7f5289d5c740 1 -- 172.21.15.67:0/4094565513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5274004030 con 0x556aedbe9b60 2024-07-31T14:56:53.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.738+0000 7f5264ff9640 1 -- 172.21.15.67:0/4094565513 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f526c004040 con 0x556aedbe9b60 2024-07-31T14:56:53.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.738+0000 7f5264ff9640 1 -- 172.21.15.67:0/4094565513 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f526c004240 con 0x556aedbe9b60 2024-07-31T14:56:53.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.738+0000 7f5289d5c740 10 client.26675 fetch_fsmap learned FSMap version 44 2024-07-31T14:56:53.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.738+0000 7f5289d5c740 1 -- 172.21.15.67:0/4094565513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x556aedc84b90 con 0x556aedbe9b60 2024-07-31T14:56:53.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.738+0000 7f5264ff9640 1 -- 172.21.15.67:0/4094565513 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5274004030 con 0x556aedbe9b60 2024-07-31T14:56:53.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.739+0000 7f5264ff9640 1 -- 172.21.15.67:0/4094565513 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7f526c004980 con 0x556aedbe9b60 2024-07-31T14:56:53.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.739+0000 7f5289d5c740 10 client.26675 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:56:53.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.739+0000 7f5289d5c740 10 client.26675 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:56:53.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.739+0000 7f5289d5c740 20 client.26675 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:53.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.739+0000 7f5289d5c740 1 --2- 172.21.15.67:0/4094565513 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556aedbc91c0 0x556aedfb92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.739+0000 7f5289d5c740 4 client.26675 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:56:53.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.739+0000 7f5289d5c740 1 -- 172.21.15.67:0/4094565513 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aedf276d0 con 0x556aedbc91c0 2024-07-31T14:56:53.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.740+0000 7f5281da8640 1 --2- 172.21.15.67:0/4094565513 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556aedbc91c0 0x556aedfb92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.740+0000 7f5264ff9640 10 client.26675 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:53.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.740+0000 7f5281da8640 1 --2- 172.21.15.67:0/4094565513 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556aedbc91c0 0x556aedfb92b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e668ecda4b9c6eb server_cookie=760bad6e99216ec5 in_seq=0 out_seq=0 2024-07-31T14:56:53.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5264ff9640 1 -- 172.21.15.67:0/4094565513 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556aedf276d0 con 0x556aedbc91c0 2024-07-31T14:56:53.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5264ff9640 10 client.26675 handle_command_reply: tid=0 2024-07-31T14:56:53.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5289d5c740 1 client.26675 shutdown 2024-07-31T14:56:53.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5289d5c740 1 -- 172.21.15.67:0/4094565513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbe9b60 msgr2=0x556aedfc0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:53.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5289d5c740 1 --2- 172.21.15.67:0/4094565513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbe9b60 0x556aedfc0760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f526c032e70 tx=0x7f526c032ea0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5289d5c740 1 -- 172.21.15.67:0/4094565513 shutdown_connections 2024-07-31T14:56:53.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5289d5c740 1 --2- 172.21.15.67:0/4094565513 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556aedbc91c0 0x556aedfb92b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5289d5c740 1 --2- 172.21.15.67:0/4094565513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedc0a840 0x556aedfc0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5289d5c740 1 --2- 172.21.15.67:0/4094565513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbe9b60 0x556aedfc0760 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f526c032e70 tx=0x7f526c032ea0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5264ff9640 0 client.26675 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:53.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5289d5c740 1 --2- 172.21.15.67:0/4094565513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe1980 0x556aedfc0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5289d5c740 1 -- 172.21.15.67:0/4094565513 >> 172.21.15.67:0/4094565513 conn(0x556aedbfe990 msgr2=0x556aedfb2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:53.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5289d5c740 1 -- 172.21.15.67:0/4094565513 shutdown_connections 2024-07-31T14:56:53.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5289d5c740 1 -- 172.21.15.67:0/4094565513 wait complete. 2024-07-31T14:56:53.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.742+0000 7f5289d5c740 20 client.26675 trim_cache size 0 max 16384 2024-07-31T14:56:53.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.763+0000 7f5289d5c740 1 Processor -- start 2024-07-31T14:56:53.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.763+0000 7f5289d5c740 1 -- start start 2024-07-31T14:56:53.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.763+0000 7f5289d5c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedbc91c0 0x556aedfc0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.763+0000 7f5289d5c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe1980 0x556aedfbd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.763+0000 7f5289d5c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedc0a840 0x556aedfbd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.763+0000 7f5289d5c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f526000fd50 con 0x556aedbe1980 2024-07-31T14:56:53.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.763+0000 7f5289d5c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5260011aa0 con 0x556aedc0a840 2024-07-31T14:56:53.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.763+0000 7f5289d5c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52600104b0 con 0x556aedbc91c0 2024-07-31T14:56:53.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.763+0000 7f5280da6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe1980 0x556aedfbd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.763+0000 7f5281da8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedc0a840 0x556aedfbd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.763+0000 7f5280da6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe1980 0x556aedfbd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46176/0 (socket says 172.21.15.67:46176) 2024-07-31T14:56:53.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.764+0000 7f5280da6640 1 -- 172.21.15.67:0/1940706416 learned_addr learned my addr 172.21.15.67:0/1940706416 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:53.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.764+0000 7f5281da8640 1 --2- 172.21.15.67:0/1940706416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedbc91c0 0x556aedfc0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.764+0000 7f5280da6640 1 -- 172.21.15.67:0/1940706416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedbc91c0 msgr2=0x556aedfc0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.764+0000 7f5280da6640 1 --2- 172.21.15.67:0/1940706416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedbc91c0 0x556aedfc0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.764+0000 7f5280da6640 1 -- 172.21.15.67:0/1940706416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedc0a840 msgr2=0x556aedfbd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.764+0000 7f5280da6640 1 --2- 172.21.15.67:0/1940706416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedc0a840 0x556aedfbd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.764+0000 7f5280da6640 1 -- 172.21.15.67:0/1940706416 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aedfc14e0 con 0x556aedbe1980 2024-07-31T14:56:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.764+0000 7f5281da8640 1 --2- 172.21.15.67:0/1940706416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedc0a840 0x556aedfbd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.764+0000 7f5280da6640 1 --2- 172.21.15.67:0/1940706416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe1980 0x556aedfbd410 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f526c009230 tx=0x7f526c046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c55565c9bdd4f12 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.764+0000 7f5266ffd640 1 -- 172.21.15.67:0/1940706416 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c061070 con 0x556aedbe1980 2024-07-31T14:56:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.764+0000 7f5266ffd640 1 -- 172.21.15.67:0/1940706416 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f526c0043e0 con 0x556aedbe1980 2024-07-31T14:56:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.764+0000 7f5266ffd640 1 -- 172.21.15.67:0/1940706416 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c035030 con 0x556aedbe1980 2024-07-31T14:56:53.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.765+0000 7f5289d5c740 1 -- 172.21.15.67:0/1940706416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe1980 msgr2=0x556aedfbd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.765+0000 7f5289d5c740 1 --2- 172.21.15.67:0/1940706416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe1980 0x556aedfbd410 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f526c009230 tx=0x7f526c046040 comp rx=0 tx=0).stop 2024-07-31T14:56:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.765+0000 7f5289d5c740 1 -- 172.21.15.67:0/1940706416 shutdown_connections 2024-07-31T14:56:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.765+0000 7f5289d5c740 1 --2- 172.21.15.67:0/1940706416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedc0a840 0x556aedfbd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.765+0000 7f5289d5c740 1 --2- 172.21.15.67:0/1940706416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedbe1980 0x556aedfbd410 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.765+0000 7f5289d5c740 1 --2- 172.21.15.67:0/1940706416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedbc91c0 0x556aedfc0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.766+0000 7f5289d5c740 1 -- 172.21.15.67:0/1940706416 >> 172.21.15.67:0/1940706416 conn(0x556aedbfe990 msgr2=0x556aedfad150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.766+0000 7f5289d5c740 1 -- 172.21.15.67:0/1940706416 shutdown_connections 2024-07-31T14:56:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.766+0000 7f5289d5c740 1 -- 172.21.15.67:0/1940706416 wait complete. 2024-07-31T14:56:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.766+0000 7f5289d5c740 1 -- start start 2024-07-31T14:56:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.766+0000 7f5289d5c740 1 Processor -- start 2024-07-31T14:56:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5289d5c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbc91c0 0x556aedfd1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5289d5c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedbe1980 0x556aedfd2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5289d5c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedc0a840 0x556aedfd26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5289d5c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f52600113d0 con 0x556aedc0a840 2024-07-31T14:56:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5289d5c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5260010c40 con 0x556aedbc91c0 2024-07-31T14:56:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5289d5c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f526000dd30 con 0x556aedbe1980 2024-07-31T14:56:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5289d5c740 10 client.0 _ll_register_callbacks cb 0x7f526c032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5281da8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedc0a840 0x556aedfd26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5280da6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedbe1980 0x556aedfd2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5281da8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbc91c0 0x556aedfd1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5281da8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedc0a840 0x556aedfd26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46184/0 (socket says 172.21.15.67:46184) 2024-07-31T14:56:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5281da8640 1 -- 172.21.15.67:0/2607080064 learned_addr learned my addr 172.21.15.67:0/2607080064 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5281da8640 1 -- 172.21.15.67:0/2607080064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedbe1980 msgr2=0x556aedfd2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5281da8640 1 --2- 172.21.15.67:0/2607080064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedbe1980 0x556aedfd2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5281da8640 1 -- 172.21.15.67:0/2607080064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedc0a840 msgr2=0x556aedfd26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5281da8640 1 --2- 172.21.15.67:0/2607080064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedc0a840 0x556aedfd26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5281da8640 1 -- 172.21.15.67:0/2607080064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aee035d30 con 0x556aedbc91c0 2024-07-31T14:56:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.767+0000 7f5280da6640 1 --2- 172.21.15.67:0/2607080064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedbe1980 0x556aedfd2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.768+0000 7f5281da8640 1 --2- 172.21.15.67:0/2607080064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbc91c0 0x556aedfd1c50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5274004860 tx=0x7f526c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aa2b76376e250d4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.768+0000 7f5264ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:56:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.768+0000 7f5264ff9640 1 -- 172.21.15.67:0/2607080064 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527401e040 con 0x556aedbc91c0 2024-07-31T14:56:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.768+0000 7f5264ff9640 1 -- 172.21.15.67:0/2607080064 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f5274003ed0 con 0x556aedbc91c0 2024-07-31T14:56:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.768+0000 7f5289d5c740 1 -- 172.21.15.67:0/2607080064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f526c0043e0 con 0x556aedbc91c0 2024-07-31T14:56:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.768+0000 7f5264ff9640 1 -- 172.21.15.67:0/2607080064 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5274042050 con 0x556aedbc91c0 2024-07-31T14:56:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.768+0000 7f5289d5c740 10 client.26681 fetch_fsmap learned FSMap version 44 2024-07-31T14:56:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.768+0000 7f5289d5c740 1 -- 172.21.15.67:0/2607080064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x556aee036f70 con 0x556aedbc91c0 2024-07-31T14:56:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.768+0000 7f5264ff9640 1 -- 172.21.15.67:0/2607080064 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f526c0043e0 con 0x556aedbc91c0 2024-07-31T14:56:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.769+0000 7f5264ff9640 1 -- 172.21.15.67:0/2607080064 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7f527402e040 con 0x556aedbc91c0 2024-07-31T14:56:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.769+0000 7f5289d5c740 10 client.26681 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:56:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.769+0000 7f5289d5c740 10 client.26681 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:56:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.769+0000 7f5289d5c740 20 client.26681 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.769+0000 7f5289d5c740 1 --2- 172.21.15.67:0/2607080064 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556aedb9e570 0x556aee0396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.769+0000 7f5289d5c740 4 client.26681 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-07-31T14:56:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.769+0000 7f5289d5c740 1 -- 172.21.15.67:0/2607080064 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x556aedf23170 con 0x556aedb9e570 2024-07-31T14:56:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.769+0000 7f5280da6640 1 --2- 172.21.15.67:0/2607080064 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556aedb9e570 0x556aee0396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.769+0000 7f5264ff9640 10 client.26681 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:53.769+0000 7f5280da6640 1 --2- 172.21.15.67:0/2607080064 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556aedb9e570 0x556aee0396d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc04aa8ebcc7b28 server_cookie=9d6716f3408da9e3 in_seq=0 out_seq=0 2024-07-31T14:56:53.801 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:56:54.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:54 smithi073 ceph-mon[36714]: pgmap v575: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 14 MiB/s wr, 2.16k op/s 2024-07-31T14:56:54.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:54 smithi077 ceph-mon[37327]: pgmap v575: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 14 MiB/s wr, 2.16k op/s 2024-07-31T14:56:55.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:54 smithi067 ceph-mon[32750]: pgmap v575: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 14 MiB/s wr, 2.16k op/s 2024-07-31T14:56:56.713 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:56 smithi067 ceph-mon[32750]: pgmap v576: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-07-31T14:56:56.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.808+0000 7fb78a222640 1 -- 172.21.15.67:0/4021511969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb784102490 msgr2=0x7fb7841028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:56.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.808+0000 7fb78a222640 1 --2- 172.21.15.67:0/4021511969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb784102490 0x7fb7841028a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb774009900 tx=0x7fb77402a410 comp rx=0 tx=0).stop 2024-07-31T14:56:56.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.808+0000 7fb78a222640 1 -- 172.21.15.67:0/4021511969 shutdown_connections 2024-07-31T14:56:56.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.808+0000 7fb78a222640 1 --2- 172.21.15.67:0/4021511969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb784108720 0x7fb78410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.808+0000 7fb78a222640 1 --2- 172.21.15.67:0/4021511969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb784107dd0 0x7fb7841081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.808+0000 7fb78a222640 1 --2- 172.21.15.67:0/4021511969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb784102490 0x7fb7841028a0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.808+0000 7fb78a222640 1 -- 172.21.15.67:0/4021511969 >> 172.21.15.67:0/4021511969 conn(0x7fb784079160 msgr2=0x7fb784079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:56.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.808+0000 7fb78a222640 1 -- 172.21.15.67:0/4021511969 shutdown_connections 2024-07-31T14:56:56.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.808+0000 7fb78a222640 1 -- 172.21.15.67:0/4021511969 wait complete. 2024-07-31T14:56:56.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.809+0000 7fb78a222640 1 Processor -- start 2024-07-31T14:56:56.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.809+0000 7fb78a222640 1 -- start start 2024-07-31T14:56:56.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.809+0000 7fb78a222640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb784102490 0x7fb78419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.809+0000 7fb78a222640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb784107dd0 0x7fb78419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.809+0000 7fb78a222640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb784108720 0x7fb78419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.809+0000 7fb78a222640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841123a0 con 0x7fb784107dd0 2024-07-31T14:56:56.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.809+0000 7fb78a222640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb784112220 con 0x7fb784102490 2024-07-31T14:56:56.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.809+0000 7fb78a222640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb784112520 con 0x7fb784108720 2024-07-31T14:56:56.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.809+0000 7fb782ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb784107dd0 0x7fb78419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.809+0000 7fb782ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb784107dd0 0x7fb78419b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46204/0 (socket says 172.21.15.67:46204) 2024-07-31T14:56:56.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.809+0000 7fb782ffd640 1 -- 172.21.15.67:0/3727535407 learned_addr learned my addr 172.21.15.67:0/3727535407 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.809+0000 7fb7837fe640 1 --2- 172.21.15.67:0/3727535407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb784102490 0x7fb78419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.810+0000 7fb783fff640 1 --2- 172.21.15.67:0/3727535407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb784108720 0x7fb78419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.810+0000 7fb782ffd640 1 -- 172.21.15.67:0/3727535407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb784108720 msgr2=0x7fb78419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.810+0000 7fb782ffd640 1 --2- 172.21.15.67:0/3727535407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb784108720 0x7fb78419b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.810+0000 7fb782ffd640 1 -- 172.21.15.67:0/3727535407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb784102490 msgr2=0x7fb78419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.810+0000 7fb782ffd640 1 --2- 172.21.15.67:0/3727535407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb784102490 0x7fb78419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.810+0000 7fb782ffd640 1 -- 172.21.15.67:0/3727535407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7841a5f60 con 0x7fb784107dd0 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.810+0000 7fb782ffd640 1 --2- 172.21.15.67:0/3727535407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb784107dd0 0x7fb78419b210 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb77800d770 tx=0x7fb77800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.810+0000 7fb780ff9640 1 -- 172.21.15.67:0/3727535407 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778017070 con 0x7fb784107dd0 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.810+0000 7fb780ff9640 1 -- 172.21.15.67:0/3727535407 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb778009c30 con 0x7fb784107dd0 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.810+0000 7fb78a222640 1 -- 172.21.15.67:0/3727535407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7841a6250 con 0x7fb784107dd0 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.810+0000 7fb780ff9640 1 -- 172.21.15.67:0/3727535407 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77800f620 con 0x7fb784107dd0 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.810+0000 7fb78a222640 1 -- 172.21.15.67:0/3727535407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7841a67e0 con 0x7fb784107dd0 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.813+0000 7fb790773740 1 Processor -- start 2024-07-31T14:56:56.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.813+0000 7fb790773740 1 -- start start 2024-07-31T14:56:56.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.813+0000 7fb780ff9640 1 -- 172.21.15.67:0/3727535407 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb77801a070 con 0x7fb784107dd0 2024-07-31T14:56:56.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.813+0000 7fb790773740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc70570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.813+0000 7fb790773740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babc70ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.813+0000 7fb790773740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8aa390 0x561babc70ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.813+0000 7fb790773740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561babbe6bb0 con 0x561bab8a21b0 2024-07-31T14:56:56.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.813+0000 7fb790773740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561babbd60b0 con 0x561bab8aa390 2024-07-31T14:56:56.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.813+0000 7fb790773740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561babbda0f0 con 0x561bab8cb070 2024-07-31T14:56:56.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.813+0000 7fb7837fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babc70ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.813+0000 7fb7837fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babc70ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46210/0 (socket says 172.21.15.67:46210) 2024-07-31T14:56:56.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.813+0000 7fb7837fe640 1 -- 172.21.15.67:0/2687664910 learned_addr learned my addr 172.21.15.67:0/2687664910 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:56.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb783fff640 1 --2- 172.21.15.67:0/2687664910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc70570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb782ffd640 1 --2- 172.21.15.67:0/2687664910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8aa390 0x561babc70ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb783fff640 1 -- 172.21.15.67:0/2687664910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8aa390 msgr2=0x561babc70ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:56.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb783fff640 1 --2- 172.21.15.67:0/2687664910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8aa390 0x561babc70ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb783fff640 1 -- 172.21.15.67:0/2687664910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 msgr2=0x561babc70ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:56.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb783fff640 1 --2- 172.21.15.67:0/2687664910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babc70ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb783fff640 1 -- 172.21.15.67:0/2687664910 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561babb7cb90 con 0x561bab8cb070 2024-07-31T14:56:56.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb7837fe640 1 --2- 172.21.15.67:0/2687664910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babc70ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:56.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb783fff640 1 --2- 172.21.15.67:0/2687664910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc70570 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009f00 tx=0x7fb77c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0529cf77e376c38 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:56.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb780ff9640 1 --2- 172.21.15.67:0/3727535407 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb76c0b3df0 0x7fb76c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb769ffb640 1 -- 172.21.15.67:0/2687664910 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c004d20 con 0x561bab8cb070 2024-07-31T14:56:56.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb769ffb640 1 -- 172.21.15.67:0/2687664910 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb77c004030 con 0x561bab8cb070 2024-07-31T14:56:56.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb769ffb640 1 -- 172.21.15.67:0/2687664910 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c005260 con 0x561bab8cb070 2024-07-31T14:56:56.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb7837fe640 1 --2- 172.21.15.67:0/3727535407 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb76c0b3df0 0x7fb76c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.814+0000 7fb780ff9640 1 -- 172.21.15.67:0/3727535407 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fb7780e3cf0 con 0x7fb784107dd0 2024-07-31T14:56:56.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.815+0000 7fb790773740 1 -- 172.21.15.67:0/2687664910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 msgr2=0x561babc70570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:56.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.815+0000 7fb7837fe640 1 --2- 172.21.15.67:0/3727535407 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb76c0b3df0 0x7fb76c0b6300 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb7740096c0 tx=0x7fb774031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:56.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.815+0000 7fb790773740 1 --2- 172.21.15.67:0/2687664910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc70570 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009f00 tx=0x7fb77c011810 comp rx=0 tx=0).stop 2024-07-31T14:56:56.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.815+0000 7fb790773740 1 -- 172.21.15.67:0/2687664910 shutdown_connections 2024-07-31T14:56:56.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.815+0000 7fb790773740 1 --2- 172.21.15.67:0/2687664910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8aa390 0x561babc70ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.815+0000 7fb790773740 1 --2- 172.21.15.67:0/2687664910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babc70ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.815+0000 7fb790773740 1 --2- 172.21.15.67:0/2687664910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc70570 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.815+0000 7fb790773740 1 -- 172.21.15.67:0/2687664910 >> 172.21.15.67:0/2687664910 conn(0x561bab8bf460 msgr2=0x561babc6e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:56.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.815+0000 7fb790773740 1 -- 172.21.15.67:0/2687664910 shutdown_connections 2024-07-31T14:56:56.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.815+0000 7fb790773740 1 -- 172.21.15.67:0/2687664910 wait complete. 2024-07-31T14:56:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.816+0000 7fb790773740 1 -- start start 2024-07-31T14:56:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.816+0000 7fb790773740 1 Processor -- start 2024-07-31T14:56:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.816+0000 7fb790773740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8a21b0 0x561babc7c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.816+0000 7fb790773740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8aa390 0x561babc7c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.816+0000 7fb790773740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc88070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.816+0000 7fb790773740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561baba39700 con 0x561bab8aa390 2024-07-31T14:56:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.816+0000 7fb790773740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561babb789e0 con 0x561bab8a21b0 2024-07-31T14:56:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.816+0000 7fb790773740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561babc7ed50 con 0x561bab8cb070 2024-07-31T14:56:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.816+0000 7fb790773740 10 client.0 _ll_register_callbacks cb 0x561babb934f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.816+0000 7fb782ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8aa390 0x561babc7c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.816+0000 7fb782ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8aa390 0x561babc7c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46222/0 (socket says 172.21.15.67:46222) 2024-07-31T14:56:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.816+0000 7fb782ffd640 1 -- 172.21.15.67:0/3734910107 learned_addr learned my addr 172.21.15.67:0/3734910107 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb783fff640 1 --2- 172.21.15.67:0/3734910107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc88070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb7837fe640 1 --2- 172.21.15.67:0/3734910107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8a21b0 0x561babc7c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb782ffd640 1 -- 172.21.15.67:0/3734910107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 msgr2=0x561babc88070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb782ffd640 1 --2- 172.21.15.67:0/3734910107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc88070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb782ffd640 1 -- 172.21.15.67:0/3734910107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8a21b0 msgr2=0x561babc7c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb782ffd640 1 --2- 172.21.15.67:0/3734910107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8a21b0 0x561babc7c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb782ffd640 1 -- 172.21.15.67:0/3734910107 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bab936650 con 0x561bab8aa390 2024-07-31T14:56:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb783fff640 1 --2- 172.21.15.67:0/3734910107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc88070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb782ffd640 1 --2- 172.21.15.67:0/3734910107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8aa390 0x561babc7c690 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb7780a7910 tx=0x7fb7780e7350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406fa1893fef154b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb762ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:56:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb762ffd640 1 -- 172.21.15.67:0/3734910107 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778021030 con 0x561bab8aa390 2024-07-31T14:56:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb762ffd640 1 -- 172.21.15.67:0/3734910107 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fb7780a9900 con 0x561bab8aa390 2024-07-31T14:56:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb790773740 1 -- 172.21.15.67:0/3734910107 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb77c004030 con 0x561bab8aa390 2024-07-31T14:56:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb762ffd640 1 -- 172.21.15.67:0/3734910107 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb7780a9ff0 con 0x561bab8aa390 2024-07-31T14:56:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb790773740 10 client.17115 fetch_fsmap learned FSMap version 44 2024-07-31T14:56:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.817+0000 7fb790773740 1 -- 172.21.15.67:0/3734910107 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561babc8ad60 con 0x561bab8aa390 2024-07-31T14:56:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.818+0000 7fb762ffd640 1 -- 172.21.15.67:0/3734910107 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb77c004030 con 0x561bab8aa390 2024-07-31T14:56:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.818+0000 7fb762ffd640 1 -- 172.21.15.67:0/3734910107 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7fb7780a6010 con 0x561bab8aa390 2024-07-31T14:56:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.818+0000 7fb790773740 10 client.17115 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:56:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.818+0000 7fb790773740 10 client.17115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:56:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.818+0000 7fb790773740 20 client.17115 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.818+0000 7fb790773740 1 --2- 172.21.15.67:0/3734910107 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561bab8899f0 0x561babc8e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.818+0000 7fb790773740 4 client.17115 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:56:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.818+0000 7fb790773740 1 -- 172.21.15.67:0/3734910107 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561babbe8b00 con 0x561bab8899f0 2024-07-31T14:56:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.818+0000 7fb7837fe640 1 --2- 172.21.15.67:0/3734910107 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561bab8899f0 0x561babc8e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.818+0000 7fb762ffd640 10 client.17115 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.818+0000 7fb7837fe640 1 --2- 172.21.15.67:0/3734910107 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561bab8899f0 0x561babc8e2b0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39bdc2ec4b9a1c64 server_cookie=882915a361f542f9 in_seq=0 out_seq=0 2024-07-31T14:56:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.819+0000 7fb762ffd640 1 -- 172.21.15.67:0/3734910107 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561babbe8b00 con 0x561bab8899f0 2024-07-31T14:56:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.819+0000 7fb762ffd640 10 client.17115 handle_command_reply: tid=0 2024-07-31T14:56:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.819+0000 7fb790773740 1 client.17115 shutdown 2024-07-31T14:56:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.819+0000 7fb790773740 1 -- 172.21.15.67:0/3734910107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8aa390 msgr2=0x561babc7c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.819+0000 7fb790773740 1 --2- 172.21.15.67:0/3734910107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8aa390 0x561babc7c690 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb7780a7910 tx=0x7fb7780e7350 comp rx=0 tx=0).stop 2024-07-31T14:56:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.819+0000 7fb790773740 1 -- 172.21.15.67:0/3734910107 shutdown_connections 2024-07-31T14:56:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.819+0000 7fb790773740 1 --2- 172.21.15.67:0/3734910107 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561bab8899f0 0x561babc8e2b0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.819+0000 7fb790773740 1 --2- 172.21.15.67:0/3734910107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc88070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.819+0000 7fb790773740 1 --2- 172.21.15.67:0/3734910107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8aa390 0x561babc7c690 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.819+0000 7fb790773740 1 --2- 172.21.15.67:0/3734910107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8a21b0 0x561babc7c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.819+0000 7fb790773740 1 -- 172.21.15.67:0/3734910107 >> 172.21.15.67:0/3734910107 conn(0x561bab8bf460 msgr2=0x561babc6cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.820+0000 7fb762ffd640 0 client.17115 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.820+0000 7fb790773740 1 -- 172.21.15.67:0/3734910107 shutdown_connections 2024-07-31T14:56:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.820+0000 7fb790773740 1 -- 172.21.15.67:0/3734910107 wait complete. 2024-07-31T14:56:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.820+0000 7fb790773740 20 client.17115 trim_cache size 0 max 16384 2024-07-31T14:56:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb790773740 1 Processor -- start 2024-07-31T14:56:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb790773740 1 -- start start 2024-07-31T14:56:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb790773740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babc6d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb790773740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8aa390 0x561babc775d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb790773740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8cb070 0x561babc87e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb790773740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841b06f0 con 0x561bab8a21b0 2024-07-31T14:56:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb790773740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841b2440 con 0x561bab8cb070 2024-07-31T14:56:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb790773740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841b0e50 con 0x561bab8aa390 2024-07-31T14:56:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb7837fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babc6d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb7837fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babc6d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46234/0 (socket says 172.21.15.67:46234) 2024-07-31T14:56:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb7837fe640 1 -- 172.21.15.67:0/195158247 learned_addr learned my addr 172.21.15.67:0/195158247 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb782ffd640 1 --2- 172.21.15.67:0/195158247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8aa390 0x561babc775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb783fff640 1 --2- 172.21.15.67:0/195158247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8cb070 0x561babc87e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb7837fe640 1 -- 172.21.15.67:0/195158247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8aa390 msgr2=0x561babc775d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb7837fe640 1 --2- 172.21.15.67:0/195158247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8aa390 0x561babc775d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb7837fe640 1 -- 172.21.15.67:0/195158247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8cb070 msgr2=0x561babc87e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb7837fe640 1 --2- 172.21.15.67:0/195158247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8cb070 0x561babc87e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb7837fe640 1 -- 172.21.15.67:0/195158247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561babc7aeb0 con 0x561bab8a21b0 2024-07-31T14:56:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.834+0000 7fb783fff640 1 --2- 172.21.15.67:0/195158247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8cb070 0x561babc87e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:56:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.835+0000 7fb7837fe640 1 --2- 172.21.15.67:0/195158247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babc6d350 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb774032fd0 tx=0x7fb77403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c81219431a59f8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.835+0000 7fb7697fa640 1 -- 172.21.15.67:0/195158247 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774035040 con 0x561bab8a21b0 2024-07-31T14:56:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.835+0000 7fb7697fa640 1 -- 172.21.15.67:0/195158247 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb774056070 con 0x561bab8a21b0 2024-07-31T14:56:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.835+0000 7fb7697fa640 1 -- 172.21.15.67:0/195158247 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7740049b0 con 0x561bab8a21b0 2024-07-31T14:56:56.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.836+0000 7fb790773740 1 -- 172.21.15.67:0/195158247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 msgr2=0x561babc6d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:56.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.836+0000 7fb790773740 1 --2- 172.21.15.67:0/195158247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babc6d350 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb774032fd0 tx=0x7fb77403f000 comp rx=0 tx=0).stop 2024-07-31T14:56:56.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.836+0000 7fb790773740 1 -- 172.21.15.67:0/195158247 shutdown_connections 2024-07-31T14:56:56.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.836+0000 7fb790773740 1 --2- 172.21.15.67:0/195158247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8cb070 0x561babc87e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.836+0000 7fb790773740 1 --2- 172.21.15.67:0/195158247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8aa390 0x561babc775d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.836+0000 7fb790773740 1 --2- 172.21.15.67:0/195158247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babc6d350 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.836+0000 7fb790773740 1 -- 172.21.15.67:0/195158247 >> 172.21.15.67:0/195158247 conn(0x561bab8bf460 msgr2=0x561babc6cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:56.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.836+0000 7fb790773740 1 -- 172.21.15.67:0/195158247 shutdown_connections 2024-07-31T14:56:56.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.836+0000 7fb790773740 1 -- 172.21.15.67:0/195158247 wait complete. 2024-07-31T14:56:56.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.836+0000 7fb790773740 1 -- start start 2024-07-31T14:56:56.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.836+0000 7fb790773740 1 Processor -- start 2024-07-31T14:56:56.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb790773740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babceed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb790773740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8aa390 0x561babcef330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb790773740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc7f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb790773740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841b1d70 con 0x561bab8a21b0 2024-07-31T14:56:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb790773740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841b15e0 con 0x561bab8aa390 2024-07-31T14:56:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb790773740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841ae6d0 con 0x561bab8cb070 2024-07-31T14:56:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb790773740 10 client.0 _ll_register_callbacks cb 0x7fb778010b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb783fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc7f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb7837fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babceed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb783fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc7f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37552/0 (socket says 172.21.15.67:37552) 2024-07-31T14:56:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb783fff640 1 -- 172.21.15.67:0/471645137 learned_addr learned my addr 172.21.15.67:0/471645137 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb782ffd640 1 --2- 172.21.15.67:0/471645137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8aa390 0x561babcef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb783fff640 1 -- 172.21.15.67:0/471645137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8aa390 msgr2=0x561babcef330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb783fff640 1 --2- 172.21.15.67:0/471645137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8aa390 0x561babcef330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb783fff640 1 -- 172.21.15.67:0/471645137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 msgr2=0x561babceed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb783fff640 1 --2- 172.21.15.67:0/471645137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babceed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb783fff640 1 -- 172.21.15.67:0/471645137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561babc7fe30 con 0x561bab8cb070 2024-07-31T14:56:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb782ffd640 1 --2- 172.21.15.67:0/471645137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8aa390 0x561babcef330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:56:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.837+0000 7fb7837fe640 1 --2- 172.21.15.67:0/471645137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babceed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:56:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.838+0000 7fb783fff640 1 --2- 172.21.15.67:0/471645137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc7f6c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009ed0 tx=0x7fb77c0046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e05a888aae0e0fcd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.838+0000 7fb762ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:56:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.838+0000 7fb762ffd640 1 -- 172.21.15.67:0/471645137 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c029070 con 0x561bab8cb070 2024-07-31T14:56:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.838+0000 7fb790773740 1 -- 172.21.15.67:0/471645137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb774056070 con 0x561bab8cb070 2024-07-31T14:56:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.838+0000 7fb762ffd640 1 -- 172.21.15.67:0/471645137 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fb77c004b90 con 0x561bab8cb070 2024-07-31T14:56:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.838+0000 7fb762ffd640 1 -- 172.21.15.67:0/471645137 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb77c035050 con 0x561bab8cb070 2024-07-31T14:56:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.838+0000 7fb790773740 10 client.26716 fetch_fsmap learned FSMap version 44 2024-07-31T14:56:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.838+0000 7fb790773740 1 -- 172.21.15.67:0/471645137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561babcf64c0 con 0x561bab8cb070 2024-07-31T14:56:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.838+0000 7fb762ffd640 1 -- 172.21.15.67:0/471645137 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb774056070 con 0x561bab8cb070 2024-07-31T14:56:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.838+0000 7fb762ffd640 1 -- 172.21.15.67:0/471645137 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7fb77c018070 con 0x561bab8cb070 2024-07-31T14:56:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.839+0000 7fb790773740 10 client.26716 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:56:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.839+0000 7fb790773740 10 client.26716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:56:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.839+0000 7fb790773740 20 client.26716 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.839+0000 7fb790773740 1 --2- 172.21.15.67:0/471645137 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561bab85eda0 0x561babcf8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.839+0000 7fb790773740 4 client.26716 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:56:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.839+0000 7fb790773740 1 -- 172.21.15.67:0/471645137 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561babbe45a0 con 0x561bab85eda0 2024-07-31T14:56:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.839+0000 7fb7837fe640 1 --2- 172.21.15.67:0/471645137 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561bab85eda0 0x561babcf8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.839+0000 7fb7837fe640 1 --2- 172.21.15.67:0/471645137 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561bab85eda0 0x561babcf8b60 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3784b8f0f70498c server_cookie=46a2c165b40426e4 in_seq=0 out_seq=0 2024-07-31T14:56:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.839+0000 7fb762ffd640 10 client.26716 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.839+0000 7fb762ffd640 1 -- 172.21.15.67:0/471645137 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561babbe45a0 con 0x561bab85eda0 2024-07-31T14:56:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.839+0000 7fb762ffd640 10 client.26716 handle_command_reply: tid=0 2024-07-31T14:56:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.839+0000 7fb790773740 1 client.26716 shutdown 2024-07-31T14:56:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.839+0000 7fb790773740 1 -- 172.21.15.67:0/471645137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 msgr2=0x561babc7f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.839+0000 7fb790773740 1 --2- 172.21.15.67:0/471645137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc7f6c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009ed0 tx=0x7fb77c0046e0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.840+0000 7fb790773740 1 -- 172.21.15.67:0/471645137 shutdown_connections 2024-07-31T14:56:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.840+0000 7fb790773740 1 --2- 172.21.15.67:0/471645137 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561bab85eda0 0x561babcf8b60 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.840+0000 7fb790773740 1 --2- 172.21.15.67:0/471645137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bab8cb070 0x561babc7f6c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.840+0000 7fb790773740 1 --2- 172.21.15.67:0/471645137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561bab8aa390 0x561babcef330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.840+0000 7fb762ffd640 0 client.26716 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.840+0000 7fb790773740 1 --2- 172.21.15.67:0/471645137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561bab8a21b0 0x561babceed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.840+0000 7fb790773740 1 -- 172.21.15.67:0/471645137 >> 172.21.15.67:0/471645137 conn(0x561bab8bf460 msgr2=0x561babc75680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.840+0000 7fb790773740 1 -- 172.21.15.67:0/471645137 shutdown_connections 2024-07-31T14:56:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.840+0000 7fb790773740 1 -- 172.21.15.67:0/471645137 wait complete. 2024-07-31T14:56:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.840+0000 7fb790773740 20 client.26716 trim_cache size 0 max 16384 2024-07-31T14:56:56.848 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:56:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.840+0000 7fb78a222640 1 -- 172.21.15.67:0/3727535407 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb76c0b3df0 msgr2=0x7fb76c0b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.840+0000 7fb78a222640 1 --2- 172.21.15.67:0/3727535407 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb76c0b3df0 0x7fb76c0b6300 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb7740096c0 tx=0x7fb774031400 comp rx=0 tx=0).stop 2024-07-31T14:56:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.840+0000 7fb78a222640 1 -- 172.21.15.67:0/3727535407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb784107dd0 msgr2=0x7fb78419b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.840+0000 7fb78a222640 1 --2- 172.21.15.67:0/3727535407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb784107dd0 0x7fb78419b210 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb77800d770 tx=0x7fb77800dc30 comp rx=0 tx=0).stop 2024-07-31T14:56:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.841+0000 7fb78a222640 1 -- 172.21.15.67:0/3727535407 shutdown_connections 2024-07-31T14:56:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.841+0000 7fb78a222640 1 --2- 172.21.15.67:0/3727535407 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb76c0b3df0 0x7fb76c0b6300 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.841+0000 7fb78a222640 1 --2- 172.21.15.67:0/3727535407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb784108720 0x7fb78419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.841+0000 7fb78a222640 1 --2- 172.21.15.67:0/3727535407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb784107dd0 0x7fb78419b210 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.841+0000 7fb78a222640 1 --2- 172.21.15.67:0/3727535407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb784102490 0x7fb78419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.841+0000 7fb78a222640 1 -- 172.21.15.67:0/3727535407 >> 172.21.15.67:0/3727535407 conn(0x7fb784079160 msgr2=0x7fb784100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.841+0000 7fb78a222640 1 -- 172.21.15.67:0/3727535407 shutdown_connections 2024-07-31T14:56:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:56:56.841+0000 7fb78a222640 1 -- 172.21.15.67:0/3727535407 wait complete. 2024-07-31T14:56:56.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:56 smithi073 ceph-mon[36714]: pgmap v576: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-07-31T14:56:56.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:56 smithi077 ceph-mon[37327]: pgmap v576: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-07-31T14:56:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.768+0000 7f4474387640 1 -- 172.21.15.67:0/3837541336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f446c107d20 msgr2=0x7f446c108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.768+0000 7f4474387640 1 --2- 172.21.15.67:0/3837541336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f446c107d20 0x7f446c108130 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f445c009900 tx=0x7f445c02a410 comp rx=0 tx=0).stop 2024-07-31T14:56:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.768+0000 7f4474387640 1 -- 172.21.15.67:0/3837541336 shutdown_connections 2024-07-31T14:56:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.768+0000 7f4474387640 1 --2- 172.21.15.67:0/3837541336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f446c108670 0x7f446c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.768+0000 7f4474387640 1 --2- 172.21.15.67:0/3837541336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f446c107d20 0x7f446c108130 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.768+0000 7f4474387640 1 --2- 172.21.15.67:0/3837541336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f446c1023f0 0x7f446c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.768+0000 7f4474387640 1 -- 172.21.15.67:0/3837541336 >> 172.21.15.67:0/3837541336 conn(0x7f446c0fddf0 msgr2=0x7f446c100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.768+0000 7f4474387640 1 -- 172.21.15.67:0/3837541336 shutdown_connections 2024-07-31T14:56:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.768+0000 7f4474387640 1 -- 172.21.15.67:0/3837541336 wait complete. 2024-07-31T14:56:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4474387640 1 Processor -- start 2024-07-31T14:56:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4474387640 1 -- start start 2024-07-31T14:56:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4474387640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f446c1023f0 0x7f446c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4474387640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f446c107d20 0x7f446c198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4474387640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f446c108670 0x7f446c199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4474387640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f446c112300 con 0x7f446c1023f0 2024-07-31T14:56:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4474387640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f446c112180 con 0x7f446c107d20 2024-07-31T14:56:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4474387640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f446c112480 con 0x7f446c108670 2024-07-31T14:56:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4472123640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f446c1023f0 0x7f446c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4472123640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f446c1023f0 0x7f446c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46260/0 (socket says 172.21.15.67:46260) 2024-07-31T14:56:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4472123640 1 -- 172.21.15.67:0/3393444406 learned_addr learned my addr 172.21.15.67:0/3393444406 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4471922640 1 --2- 172.21.15.67:0/3393444406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f446c107d20 0x7f446c198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4472924640 1 --2- 172.21.15.67:0/3393444406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f446c108670 0x7f446c199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4472123640 1 -- 172.21.15.67:0/3393444406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f446c108670 msgr2=0x7f446c199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4472123640 1 --2- 172.21.15.67:0/3393444406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f446c108670 0x7f446c199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4472123640 1 -- 172.21.15.67:0/3393444406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f446c107d20 msgr2=0x7f446c198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4472123640 1 --2- 172.21.15.67:0/3393444406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f446c107d20 0x7f446c198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.769+0000 7f4472123640 1 -- 172.21.15.67:0/3393444406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446c1a8150 con 0x7f446c1023f0 2024-07-31T14:56:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.770+0000 7f4472123640 1 --2- 172.21.15.67:0/3393444406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f446c1023f0 0x7f446c198aa0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f446800e9f0 tx=0x7f446800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.770+0000 7f44637fe640 1 -- 172.21.15.67:0/3393444406 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446800cd20 con 0x7f446c1023f0 2024-07-31T14:56:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.770+0000 7f4474387640 1 -- 172.21.15.67:0/3393444406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446c1a8440 con 0x7f446c1023f0 2024-07-31T14:56:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.770+0000 7f44637fe640 1 -- 172.21.15.67:0/3393444406 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44680054d0 con 0x7f446c1023f0 2024-07-31T14:56:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.770+0000 7f44637fe640 1 -- 172.21.15.67:0/3393444406 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468018630 con 0x7f446c1023f0 2024-07-31T14:56:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.770+0000 7f4474387640 1 -- 172.21.15.67:0/3393444406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446c1a8960 con 0x7f446c1023f0 2024-07-31T14:56:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.773+0000 7f44637fe640 1 -- 172.21.15.67:0/3393444406 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4468004e10 con 0x7f446c1023f0 2024-07-31T14:56:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f44637fe640 1 --2- 172.21.15.67:0/3393444406 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f44580b3d40 0x7f44580b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f447b280740 1 Processor -- start 2024-07-31T14:56:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f4471922640 1 --2- 172.21.15.67:0/3393444406 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f44580b3d40 0x7f44580b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f44637fe640 1 -- 172.21.15.67:0/3393444406 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f44680e47e0 con 0x7f446c1023f0 2024-07-31T14:56:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f447b280740 1 -- start start 2024-07-31T14:56:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f447b280740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4881b0 0x5614ba82ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f447b280740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba45f2f0 0x5614ba82b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f447b280740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba4674d0 0x5614ba82b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f447b280740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5614ba7a03b0 con 0x5614ba4881b0 2024-07-31T14:56:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f447b280740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5614ba78f8b0 con 0x5614ba45f2f0 2024-07-31T14:56:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f447b280740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5614ba7938f0 con 0x5614ba4674d0 2024-07-31T14:56:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f4471922640 1 --2- 172.21.15.67:0/3393444406 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f44580b3d40 0x7f44580b6250 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f445c009230 tx=0x7f445c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f4472123640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4881b0 0x5614ba82ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f4472123640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4881b0 0x5614ba82ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46268/0 (socket says 172.21.15.67:46268) 2024-07-31T14:56:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f4472924640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba4674d0 0x5614ba82b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.774+0000 7f4472123640 1 -- 172.21.15.67:0/3637543778 learned_addr learned my addr 172.21.15.67:0/3637543778 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.775+0000 7f4471922640 1 --2- 172.21.15.67:0/3637543778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba45f2f0 0x5614ba82b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.775+0000 7f4472123640 1 -- 172.21.15.67:0/3637543778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba4674d0 msgr2=0x5614ba82b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.775+0000 7f4472123640 1 --2- 172.21.15.67:0/3637543778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba4674d0 0x5614ba82b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.775+0000 7f4472123640 1 -- 172.21.15.67:0/3637543778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba45f2f0 msgr2=0x5614ba82b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.775+0000 7f4472123640 1 --2- 172.21.15.67:0/3637543778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba45f2f0 0x5614ba82b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.775+0000 7f4472123640 1 -- 172.21.15.67:0/3637543778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614ba7369a0 con 0x5614ba4881b0 2024-07-31T14:56:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.775+0000 7f4472123640 1 --2- 172.21.15.67:0/3637543778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4881b0 0x5614ba82ace0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f44680ab070 tx=0x7f44680aa910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745c72d6f17ef6c0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.775+0000 7f4460bfe640 1 -- 172.21.15.67:0/3637543778 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44680ab0e0 con 0x5614ba4881b0 2024-07-31T14:56:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.775+0000 7f4460bfe640 1 -- 172.21.15.67:0/3637543778 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4468092040 con 0x5614ba4881b0 2024-07-31T14:56:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.775+0000 7f4460bfe640 1 -- 172.21.15.67:0/3637543778 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44680b3d00 con 0x5614ba4881b0 2024-07-31T14:56:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.776+0000 7f447b280740 1 -- 172.21.15.67:0/3637543778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4881b0 msgr2=0x5614ba82ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.776+0000 7f447b280740 1 --2- 172.21.15.67:0/3637543778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4881b0 0x5614ba82ace0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f44680ab070 tx=0x7f44680aa910 comp rx=0 tx=0).stop 2024-07-31T14:56:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.776+0000 7f447b280740 1 -- 172.21.15.67:0/3637543778 shutdown_connections 2024-07-31T14:56:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.776+0000 7f447b280740 1 --2- 172.21.15.67:0/3637543778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba4674d0 0x5614ba82b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.776+0000 7f447b280740 1 --2- 172.21.15.67:0/3637543778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba45f2f0 0x5614ba82b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.776+0000 7f447b280740 1 --2- 172.21.15.67:0/3637543778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4881b0 0x5614ba82ace0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.776+0000 7f447b280740 1 -- 172.21.15.67:0/3637543778 >> 172.21.15.67:0/3637543778 conn(0x5614ba47c300 msgr2=0x5614ba828bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.776+0000 7f447b280740 1 -- 172.21.15.67:0/3637543778 shutdown_connections 2024-07-31T14:56:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.776+0000 7f447b280740 1 -- 172.21.15.67:0/3637543778 wait complete. 2024-07-31T14:56:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.777+0000 7f447b280740 1 -- start start 2024-07-31T14:56:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.777+0000 7f447b280740 1 Processor -- start 2024-07-31T14:56:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.777+0000 7f447b280740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 0x5614ba8367f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.777+0000 7f447b280740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4674d0 0x5614ba836d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.777+0000 7f447b280740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4881b0 0x5614ba842780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.777+0000 7f447b280740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5614ba6c9ab0 con 0x5614ba4674d0 2024-07-31T14:56:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.777+0000 7f447b280740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5614ba717cf0 con 0x5614ba4881b0 2024-07-31T14:56:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.777+0000 7f447b280740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5614ba839460 con 0x5614ba45f2f0 2024-07-31T14:56:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.777+0000 7f447b280740 10 client.0 _ll_register_callbacks cb 0x5614ba7899f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.777+0000 7f4471922640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4674d0 0x5614ba836d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.777+0000 7f4471922640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4674d0 0x5614ba836d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46274/0 (socket says 172.21.15.67:46274) 2024-07-31T14:56:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f4472123640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 0x5614ba8367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f4471922640 1 -- 172.21.15.67:0/2330000793 learned_addr learned my addr 172.21.15.67:0/2330000793 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f4472924640 1 --2- 172.21.15.67:0/2330000793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4881b0 0x5614ba842780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f4472123640 1 -- 172.21.15.67:0/2330000793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4881b0 msgr2=0x5614ba842780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f4472123640 1 --2- 172.21.15.67:0/2330000793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4881b0 0x5614ba842780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f4472123640 1 -- 172.21.15.67:0/2330000793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4674d0 msgr2=0x5614ba836d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f4472123640 1 --2- 172.21.15.67:0/2330000793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4674d0 0x5614ba836d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f4472123640 1 -- 172.21.15.67:0/2330000793 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614ba4f3790 con 0x5614ba45f2f0 2024-07-31T14:56:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f4472123640 1 --2- 172.21.15.67:0/2330000793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 0x5614ba8367f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4468004a00 tx=0x7f44680ab920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f69a653cca7349b9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f4455ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:56:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f4455ffb640 1 -- 172.21.15.67:0/2330000793 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44680a74e0 con 0x5614ba45f2f0 2024-07-31T14:56:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f447b280740 1 -- 172.21.15.67:0/2330000793 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4468092040 con 0x5614ba45f2f0 2024-07-31T14:56:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f4455ffb640 1 -- 172.21.15.67:0/2330000793 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f44680b3f10 con 0x5614ba45f2f0 2024-07-31T14:56:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f4455ffb640 1 -- 172.21.15.67:0/2330000793 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f44680ab0e0 con 0x5614ba45f2f0 2024-07-31T14:56:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f447b280740 10 client.26728 fetch_fsmap learned FSMap version 44 2024-07-31T14:56:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.778+0000 7f447b280740 1 -- 172.21.15.67:0/2330000793 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5614ba56c620 con 0x5614ba45f2f0 2024-07-31T14:56:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.779+0000 7f4455ffb640 1 -- 172.21.15.67:0/2330000793 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4468092040 con 0x5614ba45f2f0 2024-07-31T14:56:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.779+0000 7f4455ffb640 1 -- 172.21.15.67:0/2330000793 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7f44680a9db0 con 0x5614ba45f2f0 2024-07-31T14:56:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.779+0000 7f447b280740 10 client.26728 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:56:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.779+0000 7f447b280740 10 client.26728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:56:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.779+0000 7f447b280740 20 client.26728 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.779+0000 7f447b280740 1 --2- 172.21.15.67:0/2330000793 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5614ba446d80 0x5614ba848590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.779+0000 7f447b280740 4 client.26728 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:56:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.779+0000 7f447b280740 1 -- 172.21.15.67:0/2330000793 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614ba7a2300 con 0x5614ba446d80 2024-07-31T14:56:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.779+0000 7f4471922640 1 --2- 172.21.15.67:0/2330000793 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5614ba446d80 0x5614ba848590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.779+0000 7f4455ffb640 10 client.26728 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.779+0000 7f4471922640 1 --2- 172.21.15.67:0/2330000793 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5614ba446d80 0x5614ba848590 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b75a58873b6db6ef server_cookie=404367da11a1db88 in_seq=0 out_seq=0 2024-07-31T14:56:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.783+0000 7f4455ffb640 1 -- 172.21.15.67:0/2330000793 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614ba7a2300 con 0x5614ba446d80 2024-07-31T14:56:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.783+0000 7f4455ffb640 10 client.26728 handle_command_reply: tid=0 2024-07-31T14:56:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.784+0000 7f447b280740 1 client.26728 shutdown 2024-07-31T14:56:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.784+0000 7f447b280740 1 -- 172.21.15.67:0/2330000793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 msgr2=0x5614ba8367f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.784+0000 7f447b280740 1 --2- 172.21.15.67:0/2330000793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 0x5614ba8367f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4468004a00 tx=0x7f44680ab920 comp rx=0 tx=0).stop 2024-07-31T14:56:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.784+0000 7f447b280740 1 -- 172.21.15.67:0/2330000793 shutdown_connections 2024-07-31T14:56:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.784+0000 7f447b280740 1 --2- 172.21.15.67:0/2330000793 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5614ba446d80 0x5614ba848590 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.784+0000 7f447b280740 1 --2- 172.21.15.67:0/2330000793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4881b0 0x5614ba842780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.784+0000 7f447b280740 1 --2- 172.21.15.67:0/2330000793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4674d0 0x5614ba836d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.784+0000 7f447b280740 1 --2- 172.21.15.67:0/2330000793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 0x5614ba8367f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.784+0000 7f4455ffb640 0 client.26728 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.784+0000 7f447b280740 1 -- 172.21.15.67:0/2330000793 >> 172.21.15.67:0/2330000793 conn(0x5614ba47c300 msgr2=0x5614ba82bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.784+0000 7f447b280740 1 -- 172.21.15.67:0/2330000793 shutdown_connections 2024-07-31T14:56:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.784+0000 7f447b280740 1 -- 172.21.15.67:0/2330000793 wait complete. 2024-07-31T14:56:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.784+0000 7f447b280740 20 client.26728 trim_cache size 0 max 16384 2024-07-31T14:56:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.802+0000 7f447b280740 1 Processor -- start 2024-07-31T14:56:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.802+0000 7f447b280740 1 -- start start 2024-07-31T14:56:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.802+0000 7f447b280740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 0x5614ba8384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.802+0000 7f447b280740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4674d0 0x5614ba835e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.802+0000 7f447b280740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4881b0 0x5614ba84db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.802+0000 7f447b280740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1b2870 con 0x5614ba4674d0 2024-07-31T14:56:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.802+0000 7f447b280740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1b45c0 con 0x5614ba4881b0 2024-07-31T14:56:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.802+0000 7f447b280740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1b2fd0 con 0x5614ba45f2f0 2024-07-31T14:56:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.802+0000 7f4471922640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4674d0 0x5614ba835e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.802+0000 7f4471922640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4674d0 0x5614ba835e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46276/0 (socket says 172.21.15.67:46276) 2024-07-31T14:56:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.802+0000 7f4471922640 1 -- 172.21.15.67:0/710759994 learned_addr learned my addr 172.21.15.67:0/710759994 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.802+0000 7f4472123640 1 --2- 172.21.15.67:0/710759994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 0x5614ba8384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.803+0000 7f4472924640 1 --2- 172.21.15.67:0/710759994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4881b0 0x5614ba84db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.803+0000 7f4471922640 1 -- 172.21.15.67:0/710759994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 msgr2=0x5614ba8384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.803+0000 7f4471922640 1 --2- 172.21.15.67:0/710759994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 0x5614ba8384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.803+0000 7f4471922640 1 -- 172.21.15.67:0/710759994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4881b0 msgr2=0x5614ba84db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.803+0000 7f4471922640 1 --2- 172.21.15.67:0/710759994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4881b0 0x5614ba84db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.803+0000 7f4471922640 1 -- 172.21.15.67:0/710759994 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614ba848900 con 0x5614ba4674d0 2024-07-31T14:56:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.803+0000 7f4471922640 1 --2- 172.21.15.67:0/710759994 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4674d0 0x5614ba835e50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f445c0315f0 tx=0x7f445c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b668cf805c95e19 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.803+0000 7f4457fff640 1 -- 172.21.15.67:0/710759994 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c035040 con 0x5614ba4674d0 2024-07-31T14:56:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.803+0000 7f4457fff640 1 -- 172.21.15.67:0/710759994 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f445c058070 con 0x5614ba4674d0 2024-07-31T14:56:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.803+0000 7f4457fff640 1 -- 172.21.15.67:0/710759994 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c004a40 con 0x5614ba4674d0 2024-07-31T14:56:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.804+0000 7f447b280740 1 -- 172.21.15.67:0/710759994 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4674d0 msgr2=0x5614ba835e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:56:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.804+0000 7f447b280740 1 --2- 172.21.15.67:0/710759994 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4674d0 0x5614ba835e50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f445c0315f0 tx=0x7f445c040000 comp rx=0 tx=0).stop 2024-07-31T14:56:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.804+0000 7f447b280740 1 -- 172.21.15.67:0/710759994 shutdown_connections 2024-07-31T14:56:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.804+0000 7f447b280740 1 --2- 172.21.15.67:0/710759994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4881b0 0x5614ba84db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.804+0000 7f447b280740 1 --2- 172.21.15.67:0/710759994 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4674d0 0x5614ba835e50 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.804+0000 7f447b280740 1 --2- 172.21.15.67:0/710759994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 0x5614ba8384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.804+0000 7f447b280740 1 -- 172.21.15.67:0/710759994 >> 172.21.15.67:0/710759994 conn(0x5614ba47c300 msgr2=0x5614ba82fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:56:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.804+0000 7f447b280740 1 -- 172.21.15.67:0/710759994 shutdown_connections 2024-07-31T14:56:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.804+0000 7f447b280740 1 -- 172.21.15.67:0/710759994 wait complete. 2024-07-31T14:56:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.804+0000 7f447b280740 1 -- start start 2024-07-31T14:56:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.804+0000 7f447b280740 1 Processor -- start 2024-07-31T14:56:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f447b280740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 0x5614ba83ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f447b280740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4674d0 0x5614ba83b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f447b280740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4881b0 0x5614ba83b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f447b280740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1b3ef0 con 0x5614ba4881b0 2024-07-31T14:56:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f447b280740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1b3760 con 0x5614ba4674d0 2024-07-31T14:56:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f447b280740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1b0850 con 0x5614ba45f2f0 2024-07-31T14:56:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f447b280740 10 client.0 _ll_register_callbacks cb 0x7f445c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:56:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4472924640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4881b0 0x5614ba83b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4472123640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 0x5614ba83ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4471922640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4674d0 0x5614ba83b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4472123640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 0x5614ba83ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37596/0 (socket says 172.21.15.67:37596) 2024-07-31T14:56:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4472924640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4881b0 0x5614ba83b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46286/0 (socket says 172.21.15.67:46286) 2024-07-31T14:56:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4472123640 1 -- 172.21.15.67:0/3680049672 learned_addr learned my addr 172.21.15.67:0/3680049672 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:56:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4472924640 1 -- 172.21.15.67:0/3680049672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 msgr2=0x5614ba83ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4472924640 1 --2- 172.21.15.67:0/3680049672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 0x5614ba83ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4472924640 1 -- 172.21.15.67:0/3680049672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4674d0 msgr2=0x5614ba83b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:56:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4472924640 1 --2- 172.21.15.67:0/3680049672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4674d0 0x5614ba83b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:56:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4472924640 1 -- 172.21.15.67:0/3680049672 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614ba83bef0 con 0x5614ba4881b0 2024-07-31T14:56:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4472924640 1 --2- 172.21.15.67:0/3680049672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4881b0 0x5614ba83b780 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f44640042c0 tx=0x7f4464003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8daf782dbf60d9d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:56:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4455ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:56:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4455ffb640 1 -- 172.21.15.67:0/3680049672 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464022060 con 0x5614ba4881b0 2024-07-31T14:56:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f4455ffb640 1 -- 172.21.15.67:0/3680049672 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f4464012bb0 con 0x5614ba4881b0 2024-07-31T14:56:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.805+0000 7f447b280740 1 -- 172.21.15.67:0/3680049672 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f445c058070 con 0x5614ba4881b0 2024-07-31T14:56:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.806+0000 7f4455ffb640 1 -- 172.21.15.67:0/3680049672 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4464029050 con 0x5614ba4881b0 2024-07-31T14:56:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.806+0000 7f447b280740 10 client.17139 fetch_fsmap learned FSMap version 44 2024-07-31T14:56:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.806+0000 7f447b280740 1 -- 172.21.15.67:0/3680049672 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5614ba8afa90 con 0x5614ba4881b0 2024-07-31T14:56:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.806+0000 7f4455ffb640 1 -- 172.21.15.67:0/3680049672 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f445c058070 con 0x5614ba4881b0 2024-07-31T14:56:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.806+0000 7f4455ffb640 1 -- 172.21.15.67:0/3680049672 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7f4464010430 con 0x5614ba4881b0 2024-07-31T14:56:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.806+0000 7f447b280740 10 client.17139 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:56:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.806+0000 7f447b280740 10 client.17139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:56:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.806+0000 7f447b280740 20 client.17139 populate_metadata read hostname 'smithi067' 2024-07-31T14:56:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.806+0000 7f447b280740 1 --2- 172.21.15.67:0/3680049672 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5614ba41c1c0 0x5614ba8b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:56:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.806+0000 7f447b280740 4 client.17139 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:56:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.806+0000 7f447b280740 1 -- 172.21.15.67:0/3680049672 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614ba79dda0 con 0x5614ba41c1c0 2024-07-31T14:56:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.806+0000 7f4472123640 1 --2- 172.21.15.67:0/3680049672 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5614ba41c1c0 0x5614ba8b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:56:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.807+0000 7f4472123640 1 --2- 172.21.15.67:0/3680049672 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5614ba41c1c0 0x5614ba8b2370 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffa376138ab03624 server_cookie=b1bd9e81e8ef1b16 in_seq=0 out_seq=0 2024-07-31T14:56:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:56:58.807+0000 7f4455ffb640 10 client.17139 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:56:58.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:56:58 smithi073 ceph-mon[36714]: pgmap v577: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.1 MiB/s wr, 623 op/s 2024-07-31T14:56:58.901 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T14:56:58.901 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T14:56:58.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:56:58 smithi067 ceph-mon[32750]: pgmap v577: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.1 MiB/s wr, 623 op/s 2024-07-31T14:56:58.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:56:58 smithi077 ceph-mon[37327]: pgmap v577: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.1 MiB/s wr, 623 op/s 2024-07-31T14:56:59.201 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:57:00.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:00 smithi073 ceph-mon[36714]: pgmap v578: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.5 MiB/s wr, 519 op/s 2024-07-31T14:57:00.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:00 smithi077 ceph-mon[37327]: pgmap v578: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.5 MiB/s wr, 519 op/s 2024-07-31T14:57:01.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:00 smithi067 ceph-mon[32750]: pgmap v578: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.5 MiB/s wr, 519 op/s 2024-07-31T14:57:01.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.588+0000 7fa22cf4e640 1 -- 172.21.15.67:0/3917255044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa228107610 msgr2=0x7fa228107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:01.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.588+0000 7fa22cf4e640 1 --2- 172.21.15.67:0/3917255044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa228107610 0x7fa228107af0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa22001ace0 tx=0x7fa22003b990 comp rx=0 tx=0).stop 2024-07-31T14:57:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.592+0000 7fa22cf4e640 1 -- 172.21.15.67:0/3917255044 shutdown_connections 2024-07-31T14:57:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.592+0000 7fa22cf4e640 1 --2- 172.21.15.67:0/3917255044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa228107610 0x7fa228107af0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.592+0000 7fa22cf4e640 1 --2- 172.21.15.67:0/3917255044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa228102b20 0x7fa228107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.592+0000 7fa22cf4e640 1 --2- 172.21.15.67:0/3917255044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2281021d0 0x7fa2281025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.592+0000 7fa22cf4e640 1 -- 172.21.15.67:0/3917255044 >> 172.21.15.67:0/3917255044 conn(0x7fa228079160 msgr2=0x7fa228079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:01.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.596+0000 7fa22cf4e640 1 -- 172.21.15.67:0/3917255044 shutdown_connections 2024-07-31T14:57:01.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.598+0000 7fa22cf4e640 1 -- 172.21.15.67:0/3917255044 wait complete. 2024-07-31T14:57:01.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.599+0000 7fa22cf4e640 1 Processor -- start 2024-07-31T14:57:01.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.599+0000 7fa22cf4e640 1 -- start start 2024-07-31T14:57:01.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.599+0000 7fa22cf4e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2281021d0 0x7fa228116840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.599+0000 7fa22cf4e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa228102b20 0x7fa228112b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.599+0000 7fa22cf4e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa228107610 0x7fa2281131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.599+0000 7fa22cf4e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa22810af60 con 0x7fa2281021d0 2024-07-31T14:57:01.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.600+0000 7fa22cf4e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa22810ade0 con 0x7fa228107610 2024-07-31T14:57:01.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.600+0000 7fa22cf4e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa22810b0e0 con 0x7fa228102b20 2024-07-31T14:57:01.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.600+0000 7fa21ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa228102b20 0x7fa228112b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:01.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.600+0000 7fa21ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa228102b20 0x7fa228112b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58312/0 (socket says 172.21.15.67:58312) 2024-07-31T14:57:01.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.600+0000 7fa21ffff640 1 -- 172.21.15.67:0/2808176815 learned_addr learned my addr 172.21.15.67:0/2808176815 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:01.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.600+0000 7fa22659c640 1 --2- 172.21.15.67:0/2808176815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2281021d0 0x7fa228116840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:01.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.600+0000 7fa226d9d640 1 --2- 172.21.15.67:0/2808176815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa228107610 0x7fa2281131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:01.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.604+0000 7fa22659c640 1 -- 172.21.15.67:0/2808176815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa228102b20 msgr2=0x7fa228112b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:01.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.604+0000 7fa22659c640 1 --2- 172.21.15.67:0/2808176815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa228102b20 0x7fa228112b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.604+0000 7fa22659c640 1 -- 172.21.15.67:0/2808176815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa228107610 msgr2=0x7fa2281131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:01.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.604+0000 7fa22659c640 1 --2- 172.21.15.67:0/2808176815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa228107610 0x7fa2281131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.604+0000 7fa22659c640 1 -- 172.21.15.67:0/2808176815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2281c5fe0 con 0x7fa2281021d0 2024-07-31T14:57:01.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.606+0000 7fa22659c640 1 --2- 172.21.15.67:0/2808176815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2281021d0 0x7fa228116840 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa210009fd0 tx=0x7fa21000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:01.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.610+0000 7fa21f7fe640 1 -- 172.21.15.67:0/2808176815 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210015020 con 0x7fa2281021d0 2024-07-31T14:57:01.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.610+0000 7fa21f7fe640 1 -- 172.21.15.67:0/2808176815 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa210011ce0 con 0x7fa2281021d0 2024-07-31T14:57:01.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.610+0000 7fa21f7fe640 1 -- 172.21.15.67:0/2808176815 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210010e00 con 0x7fa2281021d0 2024-07-31T14:57:01.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.610+0000 7fa22cf4e640 1 -- 172.21.15.67:0/2808176815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2281c61b0 con 0x7fa2281021d0 2024-07-31T14:57:01.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.610+0000 7fa22cf4e640 1 -- 172.21.15.67:0/2808176815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2281c6720 con 0x7fa2281021d0 2024-07-31T14:57:01.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.614+0000 7fa23349f740 1 Processor -- start 2024-07-31T14:57:01.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.615+0000 7fa23349f740 1 -- start start 2024-07-31T14:57:01.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.615+0000 7fa23349f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d3b350 0x5605b60e03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.615+0000 7fa23349f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d12490 0x5605b60e08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.615+0000 7fa23349f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d1a670 0x5605b60e0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.616+0000 7fa23349f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5605b5fc82f0 con 0x5605b5d3b350 2024-07-31T14:57:01.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.616+0000 7fa23349f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5605b6056a80 con 0x5605b5d12490 2024-07-31T14:57:01.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.616+0000 7fa23349f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5605b6045f80 con 0x5605b5d1a670 2024-07-31T14:57:01.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.616+0000 7fa226d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d3b350 0x5605b60e03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:01.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.616+0000 7fa226d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d3b350 0x5605b60e03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53082/0 (socket says 172.21.15.67:53082) 2024-07-31T14:57:01.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.616+0000 7fa226d9d640 1 -- 172.21.15.67:0/3555799238 learned_addr learned my addr 172.21.15.67:0/3555799238 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:01.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.618+0000 7fa226d9d640 1 -- 172.21.15.67:0/3555799238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d1a670 msgr2=0x5605b60e0e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:57:01.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.618+0000 7fa226d9d640 1 --2- 172.21.15.67:0/3555799238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d1a670 0x5605b60e0e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.618+0000 7fa226d9d640 1 -- 172.21.15.67:0/3555799238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d12490 msgr2=0x5605b60e08f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:57:01.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.618+0000 7fa226d9d640 1 --2- 172.21.15.67:0/3555799238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d12490 0x5605b60e08f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.618+0000 7fa226d9d640 1 -- 172.21.15.67:0/3555799238 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b5da6930 con 0x5605b5d3b350 2024-07-31T14:57:01.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.619+0000 7fa21f7fe640 1 -- 172.21.15.67:0/2808176815 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fa210018020 con 0x7fa2281021d0 2024-07-31T14:57:01.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.620+0000 7fa226d9d640 1 --2- 172.21.15.67:0/3555799238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d3b350 0x5605b60e03b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa22001ae00 tx=0x7fa22001aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e18ada6b400c2ed server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.620+0000 7fa21f7fe640 1 --2- 172.21.15.67:0/2808176815 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2140b3df0 0x7fa2140b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.620+0000 7fa21cff9640 1 -- 172.21.15.67:0/3555799238 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220011070 con 0x5605b5d3b350 2024-07-31T14:57:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.620+0000 7fa21f7fe640 1 -- 172.21.15.67:0/2808176815 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fa2100e3890 con 0x7fa2281021d0 2024-07-31T14:57:01.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.620+0000 7fa21cff9640 1 -- 172.21.15.67:0/3555799238 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa220003ca0 con 0x5605b5d3b350 2024-07-31T14:57:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.622+0000 7fa23349f740 1 -- 172.21.15.67:0/3555799238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d3b350 msgr2=0x5605b60e03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.622+0000 7fa23349f740 1 --2- 172.21.15.67:0/3555799238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d3b350 0x5605b60e03b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa22001ae00 tx=0x7fa22001aa30 comp rx=0 tx=0).stop 2024-07-31T14:57:01.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.622+0000 7fa22659c640 1 --2- 172.21.15.67:0/2808176815 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2140b3df0 0x7fa2140b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:01.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.623+0000 7fa22659c640 1 --2- 172.21.15.67:0/2808176815 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2140b3df0 0x7fa2140b6300 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x5605b60df030 tx=0x7fa210010a00 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:01.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.623+0000 7fa23349f740 1 -- 172.21.15.67:0/3555799238 shutdown_connections 2024-07-31T14:57:01.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.623+0000 7fa23349f740 1 --2- 172.21.15.67:0/3555799238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d1a670 0x5605b60e0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.623+0000 7fa23349f740 1 --2- 172.21.15.67:0/3555799238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d12490 0x5605b60e08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.623+0000 7fa23349f740 1 --2- 172.21.15.67:0/3555799238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d3b350 0x5605b60e03b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.623+0000 7fa23349f740 1 -- 172.21.15.67:0/3555799238 >> 172.21.15.67:0/3555799238 conn(0x5605b5d2f740 msgr2=0x5605b60de2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:01.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.624+0000 7fa23349f740 1 -- 172.21.15.67:0/3555799238 shutdown_connections 2024-07-31T14:57:01.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.624+0000 7fa23349f740 1 -- 172.21.15.67:0/3555799238 wait complete. 2024-07-31T14:57:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.625+0000 7fa23349f740 1 -- start start 2024-07-31T14:57:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.625+0000 7fa23349f740 1 Processor -- start 2024-07-31T14:57:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.625+0000 7fa23349f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d12490 0x5605b60ebc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.625+0000 7fa23349f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60ec180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.626+0000 7fa23349f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d3b350 0x5605b60f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.626+0000 7fa23349f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5605b5fe84f0 con 0x5605b5d1a670 2024-07-31T14:57:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.626+0000 7fa23349f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5605b6049fc0 con 0x5605b5d12490 2024-07-31T14:57:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.626+0000 7fa23349f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5605b5ea9890 con 0x5605b5d3b350 2024-07-31T14:57:01.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.626+0000 7fa23349f740 10 client.0 _ll_register_callbacks cb 0x5605b5ff2500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:01.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.627+0000 7fa226d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d3b350 0x5605b60f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:01.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.627+0000 7fa226d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d3b350 0x5605b60f7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58330/0 (socket says 172.21.15.67:58330) 2024-07-31T14:57:01.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.627+0000 7fa226d9d640 1 -- 172.21.15.67:0/1289792396 learned_addr learned my addr 172.21.15.67:0/1289792396 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:01.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.627+0000 7fa226d9d640 1 --2- 172.21.15.67:0/1289792396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d12490 0x5605b60ebc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:01.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.627+0000 7fa21ffff640 1 --2- 172.21.15.67:0/1289792396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60ec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:01.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.627+0000 7fa226d9d640 1 -- 172.21.15.67:0/1289792396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d12490 msgr2=0x5605b60ebc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:01.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.627+0000 7fa226d9d640 1 --2- 172.21.15.67:0/1289792396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d12490 0x5605b60ebc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.627+0000 7fa226d9d640 1 -- 172.21.15.67:0/1289792396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 msgr2=0x5605b60ec180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:01.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.627+0000 7fa226d9d640 1 --2- 172.21.15.67:0/1289792396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60ec180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.627+0000 7fa226d9d640 1 -- 172.21.15.67:0/1289792396 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b5ff7c50 con 0x5605b5d3b350 2024-07-31T14:57:01.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.627+0000 7fa226d9d640 1 --2- 172.21.15.67:0/1289792396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d3b350 0x5605b60f7dd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x5605b60ee450 tx=0x7fa2200040e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e28aaf47f3cbfbad server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.632+0000 7fa20dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:57:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.632+0000 7fa20dffb640 1 -- 172.21.15.67:0/1289792396 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22004e020 con 0x5605b5d3b350 2024-07-31T14:57:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.632+0000 7fa20dffb640 1 -- 172.21.15.67:0/1289792396 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fa220040c70 con 0x5605b5d3b350 2024-07-31T14:57:01.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.633+0000 7fa23349f740 1 -- 172.21.15.67:0/1289792396 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa220003ca0 con 0x5605b5d3b350 2024-07-31T14:57:01.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.637+0000 7fa20dffb640 1 -- 172.21.15.67:0/1289792396 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa220011040 con 0x5605b5d3b350 2024-07-31T14:57:01.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.637+0000 7fa23349f740 10 client.26740 fetch_fsmap learned FSMap version 44 2024-07-31T14:57:01.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.637+0000 7fa23349f740 1 -- 172.21.15.67:0/1289792396 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5605b60fa940 con 0x5605b5d3b350 2024-07-31T14:57:01.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.640+0000 7fa20dffb640 1 -- 172.21.15.67:0/1289792396 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa220003ca0 con 0x5605b5d3b350 2024-07-31T14:57:01.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.640+0000 7fa20dffb640 1 -- 172.21.15.67:0/1289792396 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7fa220041440 con 0x5605b5d3b350 2024-07-31T14:57:01.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.640+0000 7fa23349f740 10 client.26740 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:57:01.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.640+0000 7fa23349f740 10 client.26740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:57:01.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.640+0000 7fa23349f740 20 client.26740 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:01.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.640+0000 7fa23349f740 1 --2- 172.21.15.67:0/1289792396 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5605b5cf9cd0 0x5605b60fe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.640+0000 7fa23349f740 4 client.26740 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:57:01.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.640+0000 7fa23349f740 1 -- 172.21.15.67:0/1289792396 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605b60589d0 con 0x5605b5cf9cd0 2024-07-31T14:57:01.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.642+0000 7fa21ffff640 1 --2- 172.21.15.67:0/1289792396 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5605b5cf9cd0 0x5605b60fe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:01.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.648+0000 7fa21ffff640 1 --2- 172.21.15.67:0/1289792396 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5605b5cf9cd0 0x5605b60fe030 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f5ba12113f23102 server_cookie=31509a880060f932 in_seq=0 out_seq=0 2024-07-31T14:57:01.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.649+0000 7fa20dffb640 10 client.26740 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:01.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.652+0000 7fa20dffb640 1 -- 172.21.15.67:0/1289792396 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605b60589d0 con 0x5605b5cf9cd0 2024-07-31T14:57:01.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.652+0000 7fa20dffb640 10 client.26740 handle_command_reply: tid=0 2024-07-31T14:57:01.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.652+0000 7fa23349f740 1 client.26740 shutdown 2024-07-31T14:57:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.653+0000 7fa23349f740 1 -- 172.21.15.67:0/1289792396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d3b350 msgr2=0x5605b60f7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.653+0000 7fa23349f740 1 --2- 172.21.15.67:0/1289792396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d3b350 0x5605b60f7dd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x5605b60ee450 tx=0x7fa2200040e0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.653+0000 7fa23349f740 1 -- 172.21.15.67:0/1289792396 shutdown_connections 2024-07-31T14:57:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.653+0000 7fa23349f740 1 --2- 172.21.15.67:0/1289792396 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5605b5cf9cd0 0x5605b60fe030 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.653+0000 7fa23349f740 1 --2- 172.21.15.67:0/1289792396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d3b350 0x5605b60f7dd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.653+0000 7fa23349f740 1 --2- 172.21.15.67:0/1289792396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60ec180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.653+0000 7fa23349f740 1 --2- 172.21.15.67:0/1289792396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d12490 0x5605b60ebc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.653+0000 7fa23349f740 1 -- 172.21.15.67:0/1289792396 >> 172.21.15.67:0/1289792396 conn(0x5605b5d2f740 msgr2=0x5605b60e3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.653+0000 7fa20dffb640 0 client.26740 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.653+0000 7fa23349f740 1 -- 172.21.15.67:0/1289792396 shutdown_connections 2024-07-31T14:57:01.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.653+0000 7fa23349f740 1 -- 172.21.15.67:0/1289792396 wait complete. 2024-07-31T14:57:01.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.653+0000 7fa23349f740 20 client.26740 trim_cache size 0 max 16384 2024-07-31T14:57:01.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.694+0000 7fa23349f740 1 Processor -- start 2024-07-31T14:57:01.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.694+0000 7fa23349f740 1 -- start start 2024-07-31T14:57:01.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.695+0000 7fa23349f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d12490 0x5605b60e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.695+0000 7fa23349f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60e1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.695+0000 7fa23349f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d3b350 0x5605b60f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.695+0000 7fa23349f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa20800fe60 con 0x5605b5d1a670 2024-07-31T14:57:01.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.695+0000 7fa23349f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa208011bb0 con 0x5605b5d3b350 2024-07-31T14:57:01.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.695+0000 7fa23349f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2080105c0 con 0x5605b5d12490 2024-07-31T14:57:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.695+0000 7fa226d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d12490 0x5605b60e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.695+0000 7fa226d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d12490 0x5605b60e64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58346/0 (socket says 172.21.15.67:58346) 2024-07-31T14:57:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.695+0000 7fa226d9d640 1 -- 172.21.15.67:0/2933053307 learned_addr learned my addr 172.21.15.67:0/2933053307 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.695+0000 7fa226d9d640 1 --2- 172.21.15.67:0/2933053307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d3b350 0x5605b60f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.695+0000 7fa21ffff640 1 --2- 172.21.15.67:0/2933053307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60e1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.695+0000 7fa226d9d640 1 -- 172.21.15.67:0/2933053307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d3b350 msgr2=0x5605b60f7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.696+0000 7fa226d9d640 1 --2- 172.21.15.67:0/2933053307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d3b350 0x5605b60f7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.696+0000 7fa226d9d640 1 -- 172.21.15.67:0/2933053307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 msgr2=0x5605b60e1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.696+0000 7fa226d9d640 1 --2- 172.21.15.67:0/2933053307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60e1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.696+0000 7fa226d9d640 1 -- 172.21.15.67:0/2933053307 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b60eea70 con 0x5605b5d12490 2024-07-31T14:57:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.696+0000 7fa21ffff640 1 --2- 172.21.15.67:0/2933053307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60e1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:57:01.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.699+0000 7fa226d9d640 1 --2- 172.21.15.67:0/2933053307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d12490 0x5605b60e64d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa2200049b0 tx=0x7fa22001aa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de7598d77b37e978 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:01.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.704+0000 7fa20ffff640 1 -- 172.21.15.67:0/2933053307 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22004e040 con 0x5605b5d12490 2024-07-31T14:57:01.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.704+0000 7fa20ffff640 1 -- 172.21.15.67:0/2933053307 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa220003b50 con 0x5605b5d12490 2024-07-31T14:57:01.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.704+0000 7fa20ffff640 1 -- 172.21.15.67:0/2933053307 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220040a70 con 0x5605b5d12490 2024-07-31T14:57:01.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.706+0000 7fa23349f740 1 -- 172.21.15.67:0/2933053307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d12490 msgr2=0x5605b60e64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:01.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.706+0000 7fa23349f740 1 --2- 172.21.15.67:0/2933053307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d12490 0x5605b60e64d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa2200049b0 tx=0x7fa22001aa30 comp rx=0 tx=0).stop 2024-07-31T14:57:01.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.709+0000 7fa23349f740 1 -- 172.21.15.67:0/2933053307 shutdown_connections 2024-07-31T14:57:01.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.709+0000 7fa23349f740 1 --2- 172.21.15.67:0/2933053307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d3b350 0x5605b60f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.709+0000 7fa23349f740 1 --2- 172.21.15.67:0/2933053307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60e1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.709+0000 7fa23349f740 1 --2- 172.21.15.67:0/2933053307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d12490 0x5605b60e64d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.709+0000 7fa23349f740 1 -- 172.21.15.67:0/2933053307 >> 172.21.15.67:0/2933053307 conn(0x5605b5d2f740 msgr2=0x5605b60dcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:01.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.711+0000 7fa23349f740 1 -- 172.21.15.67:0/2933053307 shutdown_connections 2024-07-31T14:57:01.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.712+0000 7fa23349f740 1 -- 172.21.15.67:0/2933053307 wait complete. 2024-07-31T14:57:01.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.713+0000 7fa23349f740 1 -- start start 2024-07-31T14:57:01.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.713+0000 7fa23349f740 1 Processor -- start 2024-07-31T14:57:01.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.714+0000 7fa23349f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d12490 0x5605b60ef560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.714+0000 7fa23349f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60efb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.715+0000 7fa23349f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d3b350 0x5605b60f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.715+0000 7fa23349f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa2080114e0 con 0x5605b5d1a670 2024-07-31T14:57:01.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.715+0000 7fa23349f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa208010d50 con 0x5605b5d12490 2024-07-31T14:57:01.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.715+0000 7fa23349f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa20800de40 con 0x5605b5d3b350 2024-07-31T14:57:01.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.715+0000 7fa21ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:01.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.715+0000 7fa21ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60efb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53094/0 (socket says 172.21.15.67:53094) 2024-07-31T14:57:01.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.715+0000 7fa21ffff640 1 -- 172.21.15.67:0/3210250736 learned_addr learned my addr 172.21.15.67:0/3210250736 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:01.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.715+0000 7fa23349f740 10 client.0 _ll_register_callbacks cb 0x7fa2200049e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:01.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.716+0000 7fa21ffff640 1 -- 172.21.15.67:0/3210250736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d3b350 msgr2=0x5605b60f3a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:57:01.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.716+0000 7fa21ffff640 1 --2- 172.21.15.67:0/3210250736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d3b350 0x5605b60f3a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.716+0000 7fa21ffff640 1 -- 172.21.15.67:0/3210250736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d12490 msgr2=0x5605b60ef560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:01.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.716+0000 7fa21ffff640 1 --2- 172.21.15.67:0/3210250736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d12490 0x5605b60ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:01.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.716+0000 7fa21ffff640 1 -- 172.21.15.67:0/3210250736 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b60f40a0 con 0x5605b5d1a670 2024-07-31T14:57:01.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.717+0000 7fa21ffff640 1 --2- 172.21.15.67:0/3210250736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60efb40 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa218004740 tx=0x7fa21800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4805209af12c0e47 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:01.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.719+0000 7fa20dffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:57:01.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.720+0000 7fa20dffb640 1 -- 172.21.15.67:0/3210250736 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218021020 con 0x5605b5d1a670 2024-07-31T14:57:01.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.720+0000 7fa23349f740 1 -- 172.21.15.67:0/3210250736 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa220003b50 con 0x5605b5d1a670 2024-07-31T14:57:01.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.720+0000 7fa20dffb640 1 -- 172.21.15.67:0/3210250736 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fa218017c40 con 0x5605b5d1a670 2024-07-31T14:57:01.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.721+0000 7fa20dffb640 1 -- 172.21.15.67:0/3210250736 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa21802c050 con 0x5605b5d1a670 2024-07-31T14:57:01.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.723+0000 7fa23349f740 10 client.17151 fetch_fsmap learned FSMap version 44 2024-07-31T14:57:01.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.723+0000 7fa23349f740 1 -- 172.21.15.67:0/3210250736 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5605b61661c0 con 0x5605b5d1a670 2024-07-31T14:57:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.725+0000 7fa20dffb640 1 -- 172.21.15.67:0/3210250736 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa218017e40 con 0x5605b5d1a670 2024-07-31T14:57:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.725+0000 7fa20dffb640 1 -- 172.21.15.67:0/3210250736 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7fa2180145a0 con 0x5605b5d1a670 2024-07-31T14:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.725+0000 7fa23349f740 10 client.17151 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.725+0000 7fa23349f740 10 client.17151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.725+0000 7fa23349f740 20 client.17151 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.725+0000 7fa23349f740 1 --2- 172.21.15.67:0/3210250736 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5605b5ccf080 0x5605b6168a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.725+0000 7fa23349f740 4 client.17151 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T14:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.725+0000 7fa23349f740 1 -- 172.21.15.67:0/3210250736 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fa220003b50 con 0x5605b5ccf080 2024-07-31T14:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.725+0000 7fa226d9d640 1 --2- 172.21.15.67:0/3210250736 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5605b5ccf080 0x5605b6168a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:01.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.729+0000 7fa226d9d640 1 --2- 172.21.15.67:0/3210250736 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5605b5ccf080 0x5605b6168a90 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b6a09adcfd266f server_cookie=3be56237b191cced in_seq=0 out_seq=0 2024-07-31T14:57:01.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:01.729+0000 7fa20dffb640 10 client.17151 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:02.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:02 smithi073 ceph-mon[36714]: pgmap v579: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 456 op/s 2024-07-31T14:57:02.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:02 smithi077 ceph-mon[37327]: pgmap v579: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 456 op/s 2024-07-31T14:57:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:02 smithi067 ceph-mon[32750]: pgmap v579: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 456 op/s 2024-07-31T14:57:03.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.114+0000 7fa20dffb640 1 -- 172.21.15.67:0/3210250736 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fa220003b50 con 0x5605b5ccf080 2024-07-31T14:57:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.114+0000 7fa20dffb640 10 client.17151 handle_command_reply: tid=0 2024-07-31T14:57:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.114+0000 7fa23349f740 1 client.17151 shutdown 2024-07-31T14:57:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.115+0000 7fa23349f740 1 -- 172.21.15.67:0/3210250736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 msgr2=0x5605b60efb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.115+0000 7fa23349f740 1 --2- 172.21.15.67:0/3210250736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60efb40 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa218004740 tx=0x7fa21800bd00 comp rx=0 tx=0).stop 2024-07-31T14:57:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.115+0000 7fa23349f740 1 -- 172.21.15.67:0/3210250736 shutdown_connections 2024-07-31T14:57:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.115+0000 7fa23349f740 1 --2- 172.21.15.67:0/3210250736 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5605b5ccf080 0x5605b6168a90 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.115+0000 7fa23349f740 1 --2- 172.21.15.67:0/3210250736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605b5d3b350 0x5605b60f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.115+0000 7fa23349f740 1 --2- 172.21.15.67:0/3210250736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5605b5d1a670 0x5605b60efb40 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.115+0000 7fa23349f740 1 --2- 172.21.15.67:0/3210250736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605b5d12490 0x5605b60ef560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.115+0000 7fa20dffb640 0 client.17151 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.115+0000 7fa23349f740 1 -- 172.21.15.67:0/3210250736 >> 172.21.15.67:0/3210250736 conn(0x5605b5d2f740 msgr2=0x5605b60ecec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:03.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.115+0000 7fa23349f740 1 -- 172.21.15.67:0/3210250736 shutdown_connections 2024-07-31T14:57:03.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.115+0000 7fa23349f740 1 -- 172.21.15.67:0/3210250736 wait complete. 2024-07-31T14:57:03.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.115+0000 7fa23349f740 20 client.17151 trim_cache size 0 max 16384 2024-07-31T14:57:03.118 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:57:03.118 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T14:57:03.118 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "e85ecfcf-f95a-43ba-9803-181acbdd5565", 2024-07-31T14:57:03.118 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T14:57:03.118 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:57:03.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.116+0000 7fa22cf4e640 1 -- 172.21.15.67:0/2808176815 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2140b3df0 msgr2=0x7fa2140b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:03.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.116+0000 7fa22cf4e640 1 --2- 172.21.15.67:0/2808176815 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2140b3df0 0x7fa2140b6300 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x5605b60df030 tx=0x7fa210010a00 comp rx=0 tx=0).stop 2024-07-31T14:57:03.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.116+0000 7fa22cf4e640 1 -- 172.21.15.67:0/2808176815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2281021d0 msgr2=0x7fa228116840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:03.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.116+0000 7fa22cf4e640 1 --2- 172.21.15.67:0/2808176815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2281021d0 0x7fa228116840 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa210009fd0 tx=0x7fa21000b870 comp rx=0 tx=0).stop 2024-07-31T14:57:03.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.116+0000 7fa22cf4e640 1 -- 172.21.15.67:0/2808176815 shutdown_connections 2024-07-31T14:57:03.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.116+0000 7fa22cf4e640 1 --2- 172.21.15.67:0/2808176815 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa2140b3df0 0x7fa2140b6300 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:03.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.116+0000 7fa22cf4e640 1 --2- 172.21.15.67:0/2808176815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa228107610 0x7fa2281131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:03.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.116+0000 7fa22cf4e640 1 --2- 172.21.15.67:0/2808176815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa228102b20 0x7fa228112b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:03.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.116+0000 7fa22cf4e640 1 --2- 172.21.15.67:0/2808176815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2281021d0 0x7fa228116840 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:03.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.116+0000 7fa22cf4e640 1 -- 172.21.15.67:0/2808176815 >> 172.21.15.67:0/2808176815 conn(0x7fa228079160 msgr2=0x7fa228105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:03.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.116+0000 7fa22cf4e640 1 -- 172.21.15.67:0/2808176815 shutdown_connections 2024-07-31T14:57:03.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:03.116+0000 7fa22cf4e640 1 -- 172.21.15.67:0/2808176815 wait complete. 2024-07-31T14:57:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.131+0000 7f4455ffb640 1 -- 172.21.15.67:0/3680049672 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5614ba79dda0 con 0x5614ba41c1c0 2024-07-31T14:57:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.132+0000 7f4455ffb640 10 client.17139 handle_command_reply: tid=0 2024-07-31T14:57:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.132+0000 7f447b280740 1 client.17139 shutdown 2024-07-31T14:57:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.132+0000 7f447b280740 1 -- 172.21.15.67:0/3680049672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4881b0 msgr2=0x5614ba83b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.132+0000 7f447b280740 1 --2- 172.21.15.67:0/3680049672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4881b0 0x5614ba83b780 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f44640042c0 tx=0x7f4464003f90 comp rx=0 tx=0).stop 2024-07-31T14:57:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.132+0000 7f447b280740 1 -- 172.21.15.67:0/3680049672 shutdown_connections 2024-07-31T14:57:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.132+0000 7f447b280740 1 --2- 172.21.15.67:0/3680049672 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5614ba41c1c0 0x5614ba8b2370 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.132+0000 7f447b280740 1 --2- 172.21.15.67:0/3680049672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5614ba4881b0 0x5614ba83b780 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.132+0000 7f447b280740 1 --2- 172.21.15.67:0/3680049672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ba4674d0 0x5614ba83b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.132+0000 7f447b280740 1 --2- 172.21.15.67:0/3680049672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5614ba45f2f0 0x5614ba83ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.132+0000 7f447b280740 1 -- 172.21.15.67:0/3680049672 >> 172.21.15.67:0/3680049672 conn(0x5614ba47c300 msgr2=0x5614ba838220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.132+0000 7f4455ffb640 0 client.17139 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.132+0000 7f447b280740 1 -- 172.21.15.67:0/3680049672 shutdown_connections 2024-07-31T14:57:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.132+0000 7f447b280740 1 -- 172.21.15.67:0/3680049672 wait complete. 2024-07-31T14:57:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.132+0000 7f447b280740 20 client.17139 trim_cache size 0 max 16384 2024-07-31T14:57:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:57:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:57:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:57:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:57:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.133+0000 7f4474387640 1 -- 172.21.15.67:0/3393444406 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f44580b3d40 msgr2=0x7f44580b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.133+0000 7f4474387640 1 --2- 172.21.15.67:0/3393444406 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f44580b3d40 0x7f44580b6250 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f445c009230 tx=0x7f445c031400 comp rx=0 tx=0).stop 2024-07-31T14:57:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.133+0000 7f4474387640 1 -- 172.21.15.67:0/3393444406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f446c1023f0 msgr2=0x7f446c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.133+0000 7f4474387640 1 --2- 172.21.15.67:0/3393444406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f446c1023f0 0x7f446c198aa0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f446800e9f0 tx=0x7f446800edf0 comp rx=0 tx=0).stop 2024-07-31T14:57:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.133+0000 7f4474387640 1 -- 172.21.15.67:0/3393444406 shutdown_connections 2024-07-31T14:57:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.133+0000 7f4474387640 1 --2- 172.21.15.67:0/3393444406 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f44580b3d40 0x7f44580b6250 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.133+0000 7f4474387640 1 --2- 172.21.15.67:0/3393444406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f446c108670 0x7f446c199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.133+0000 7f4474387640 1 --2- 172.21.15.67:0/3393444406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f446c107d20 0x7f446c198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.133+0000 7f4474387640 1 --2- 172.21.15.67:0/3393444406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f446c1023f0 0x7f446c198aa0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.133+0000 7f4474387640 1 -- 172.21.15.67:0/3393444406 >> 172.21.15.67:0/3393444406 conn(0x7f446c0fddf0 msgr2=0x7f446c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.134+0000 7f4474387640 1 -- 172.21.15.67:0/3393444406 shutdown_connections 2024-07-31T14:57:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:04.134+0000 7f4474387640 1 -- 172.21.15.67:0/3393444406 wait complete. 2024-07-31T14:57:04.632 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 14:57:04 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T14:57:04.287+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000162a [...7,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch/ auth{1=1,2=1,3=1,4=1} v2970 ap=3 RANDEPHEMERALPIN f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22) n(v19 rc2024-07-31T14:54:53.561226+0000 b106958628 17855=16920+935) old_inodes=3 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:19737 nref=3)) | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=0 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7c26ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000162a:[<0x10000000202/arch v378>,<0x10000000201/linux-6.5.11 v96>,<0x10000000006/t v68>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v57>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v60>,<0x10000000001/sv_1 v84>,<0x10000000000/qa v70>,<0x1/volumes v70>]//[]","memoryvalue":"(2)0x1000000162a:[<0x10000000202/arch v2970>,<0x10000000201/linux-6.5.11 v451>,<0x10000000006/t v337>,<0x10000000005/tmp v304>,<0x100000001fb/client.0 v301>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v279>,<0x10000000001/sv_1 v304>,<0x10000000000/qa v286>,<0x1/volumes v284>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:54:49.652308+0000 23=2+21)","ondisk_value.rstat":"n(v0 rc2024-07-31T14:54:53.561226+0000 b106958628 17855=16920+935)","memory_value.dirstat":"f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22)","memory_value.rstat":"n(v19 rc2024-07-31T14:54:53.561226+0000 b106958628 17855=16920+935)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T14:57:04.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:04 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T14:57:04.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:04.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:04.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:04 smithi067 ceph-mon[32750]: pgmap v580: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 808 op/s 2024-07-31T14:57:04.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:04.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:04 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T14:57:04.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:04 smithi073 ceph-mon[36714]: pgmap v580: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 808 op/s 2024-07-31T14:57:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:04.960 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:57:04.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:04 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T14:57:04.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:04.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:04.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:04 smithi077 ceph-mon[37327]: pgmap v580: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 808 op/s 2024-07-31T14:57:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.415 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:57:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T14:57:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:05.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T14:57:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:05.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:05.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T14:57:06.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:06.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:06.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:06.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:06.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:06.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:06.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: pgmap v581: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 1.38k op/s 2024-07-31T14:57:06.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:06.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:06.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:06.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:06.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:06.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:06.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:06.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: pgmap v581: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 1.38k op/s 2024-07-31T14:57:06.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:06.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:06.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:06.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:06.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: pgmap v581: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 1.38k op/s 2024-07-31T14:57:07.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:07.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:07.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:07.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:07.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:07.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:07.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:07.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:07.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:07.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:07.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:08.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:08.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:08.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:08.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:08.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:08.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:08.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:08.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.404+0000 7f558a554640 1 -- 172.21.15.67:0/3275226930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5584107550 msgr2=0x7f5584107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.404+0000 7f558a554640 1 --2- 172.21.15.67:0/3275226930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5584107550 0x7f5584107960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f556c009900 tx=0x7f556c02a410 comp rx=0 tx=0).stop 2024-07-31T14:57:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.404+0000 7f558a554640 1 -- 172.21.15.67:0/3275226930 shutdown_connections 2024-07-31T14:57:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.404+0000 7f558a554640 1 --2- 172.21.15.67:0/3275226930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584107ea0 0x7f558410e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.404+0000 7f558a554640 1 --2- 172.21.15.67:0/3275226930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5584107550 0x7f5584107960 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.404+0000 7f558a554640 1 --2- 172.21.15.67:0/3275226930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584101c60 0x7f5584102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.404+0000 7f558a554640 1 -- 172.21.15.67:0/3275226930 >> 172.21.15.67:0/3275226930 conn(0x7f5584077690 msgr2=0x7f5584077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.404+0000 7f558a554640 1 -- 172.21.15.67:0/3275226930 shutdown_connections 2024-07-31T14:57:08.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.404+0000 7f558a554640 1 -- 172.21.15.67:0/3275226930 wait complete. 2024-07-31T14:57:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.405+0000 7f558a554640 1 Processor -- start 2024-07-31T14:57:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.405+0000 7f558a554640 1 -- start start 2024-07-31T14:57:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.406+0000 7f558a554640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5584101c60 0x7f5584198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.406+0000 7f558a554640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584107550 0x7f55841993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.406+0000 7f558a554640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584107ea0 0x7f558419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.406+0000 7f558a554640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5584111b60 con 0x7f5584107ea0 2024-07-31T14:57:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.406+0000 7f558a554640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f55841119e0 con 0x7f5584107550 2024-07-31T14:57:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.406+0000 7f558a554640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5584111ce0 con 0x7f5584101c60 2024-07-31T14:57:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.406+0000 7f557bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584107550 0x7f55841993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.406+0000 7f557bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584107550 0x7f55841993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52176/0 (socket says 172.21.15.67:52176) 2024-07-31T14:57:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.406+0000 7f557bfff640 1 -- 172.21.15.67:0/2149273130 learned_addr learned my addr 172.21.15.67:0/2149273130 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.406+0000 7f5588af1640 1 --2- 172.21.15.67:0/2149273130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584107ea0 0x7f558419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.407+0000 7f557bfff640 1 -- 172.21.15.67:0/2149273130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5584101c60 msgr2=0x7f5584198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.407+0000 7f557bfff640 1 --2- 172.21.15.67:0/2149273130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5584101c60 0x7f5584198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.407+0000 7f557bfff640 1 -- 172.21.15.67:0/2149273130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584107ea0 msgr2=0x7f558419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.407+0000 7f557bfff640 1 --2- 172.21.15.67:0/2149273130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584107ea0 0x7f558419e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.407+0000 7f557bfff640 1 -- 172.21.15.67:0/2149273130 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558419edc0 con 0x7f5584107550 2024-07-31T14:57:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.407+0000 7f557bfff640 1 --2- 172.21.15.67:0/2149273130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584107550 0x7f55841993f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f556c02a930 tx=0x7f556c031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.407+0000 7f5581ffb640 1 -- 172.21.15.67:0/2149273130 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c03f030 con 0x7f5584107550 2024-07-31T14:57:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.407+0000 7f558a554640 1 -- 172.21.15.67:0/2149273130 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558419f050 con 0x7f5584107550 2024-07-31T14:57:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.407+0000 7f5581ffb640 1 -- 172.21.15.67:0/2149273130 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f556c004490 con 0x7f5584107550 2024-07-31T14:57:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.407+0000 7f5581ffb640 1 -- 172.21.15.67:0/2149273130 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c035070 con 0x7f5584107550 2024-07-31T14:57:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.407+0000 7f558a554640 1 -- 172.21.15.67:0/2149273130 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5584105360 con 0x7f5584107550 2024-07-31T14:57:08.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.411+0000 7f5590aa5740 1 Processor -- start 2024-07-31T14:57:08.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.411+0000 7f5590aa5740 1 -- start start 2024-07-31T14:57:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.411+0000 7f5590aa5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84ff070 0x5557a88a4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.411+0000 7f5590aa5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.411+0000 7f5590aa5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84de390 0x5557a88a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.412+0000 7f5590aa5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5557a881ab90 con 0x5557a84d61b0 2024-07-31T14:57:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.412+0000 7f5590aa5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5557a880a090 con 0x5557a84ff070 2024-07-31T14:57:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.412+0000 7f5590aa5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5557a880e0d0 con 0x5557a84de390 2024-07-31T14:57:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.412+0000 7f5583fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.412+0000 7f5588af1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84ff070 0x5557a88a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.412+0000 7f5583fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88a4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53118/0 (socket says 172.21.15.67:53118) 2024-07-31T14:57:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.412+0000 7f5588af1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84ff070 0x5557a88a4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52188/0 (socket says 172.21.15.67:52188) 2024-07-31T14:57:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.412+0000 7f5583fff640 1 -- 172.21.15.67:0/2939374878 learned_addr learned my addr 172.21.15.67:0/2939374878 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.412+0000 7f5583fff640 1 -- 172.21.15.67:0/2939374878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84de390 msgr2=0x5557a88a4fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:57:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.412+0000 7f5583fff640 1 --2- 172.21.15.67:0/2939374878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84de390 0x5557a88a4fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.412+0000 7f5583fff640 1 -- 172.21.15.67:0/2939374878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84ff070 msgr2=0x5557a88a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.412+0000 7f5583fff640 1 --2- 172.21.15.67:0/2939374878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84ff070 0x5557a88a4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.412+0000 7f5583fff640 1 -- 172.21.15.67:0/2939374878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a87b0b70 con 0x5557a84d61b0 2024-07-31T14:57:08.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.413+0000 7f5581ffb640 1 -- 172.21.15.67:0/2149273130 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f556c042030 con 0x7f5584107550 2024-07-31T14:57:08.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.413+0000 7f5583fff640 1 --2- 172.21.15.67:0/2939374878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88a4a90 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5574009c40 tx=0x7f557400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7287c98617619075 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:08.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.413+0000 7f557a7fc640 1 -- 172.21.15.67:0/2939374878 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574005020 con 0x5557a84d61b0 2024-07-31T14:57:08.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.413+0000 7f557a7fc640 1 -- 172.21.15.67:0/2939374878 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55740047a0 con 0x5557a84d61b0 2024-07-31T14:57:08.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.413+0000 7f557a7fc640 1 -- 172.21.15.67:0/2939374878 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574010690 con 0x5557a84d61b0 2024-07-31T14:57:08.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.414+0000 7f5581ffb640 1 --2- 172.21.15.67:0/2149273130 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f55700b3d40 0x7f55700b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.414+0000 7f5590aa5740 1 -- 172.21.15.67:0/2939374878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 msgr2=0x5557a88a4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:08.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.414+0000 7f5590aa5740 1 --2- 172.21.15.67:0/2939374878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88a4a90 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5574009c40 tx=0x7f557400e720 comp rx=0 tx=0).stop 2024-07-31T14:57:08.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.414+0000 7f5588af1640 1 --2- 172.21.15.67:0/2149273130 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f55700b3d40 0x7f55700b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.414+0000 7f5581ffb640 1 -- 172.21.15.67:0/2149273130 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f556c104550 con 0x7f5584107550 2024-07-31T14:57:08.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.414+0000 7f5590aa5740 1 -- 172.21.15.67:0/2939374878 shutdown_connections 2024-07-31T14:57:08.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.414+0000 7f5590aa5740 1 --2- 172.21.15.67:0/2939374878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84de390 0x5557a88a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.414+0000 7f5590aa5740 1 --2- 172.21.15.67:0/2939374878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88a4a90 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.414+0000 7f5590aa5740 1 --2- 172.21.15.67:0/2939374878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84ff070 0x5557a88a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.414+0000 7f5590aa5740 1 -- 172.21.15.67:0/2939374878 >> 172.21.15.67:0/2939374878 conn(0x5557a84f3460 msgr2=0x5557a88a2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:08.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.414+0000 7f5590aa5740 1 -- 172.21.15.67:0/2939374878 shutdown_connections 2024-07-31T14:57:08.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.414+0000 7f5590aa5740 1 -- 172.21.15.67:0/2939374878 wait complete. 2024-07-31T14:57:08.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.415+0000 7f5590aa5740 1 -- start start 2024-07-31T14:57:08.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.415+0000 7f5588af1640 1 --2- 172.21.15.67:0/2149273130 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f55700b3d40 0x7f55700b6250 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f557c005fd0 tx=0x7f557c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:08.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.415+0000 7f5590aa5740 1 Processor -- start 2024-07-31T14:57:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5590aa5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5590aa5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84de390 0x5557a88ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5590aa5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84ff070 0x5557a88c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5590aa5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5557a866d700 con 0x5557a84d61b0 2024-07-31T14:57:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5590aa5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5557a87ac9c0 con 0x5557a84de390 2024-07-31T14:57:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5590aa5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5557a88bb4c0 con 0x5557a84ff070 2024-07-31T14:57:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5590aa5740 10 client.0 _ll_register_callbacks cb 0x5557a87c74d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5583fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f557bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84de390 0x5557a88ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f557bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84de390 0x5557a88ac630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52200/0 (socket says 172.21.15.67:52200) 2024-07-31T14:57:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5588af1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84ff070 0x5557a88c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f557bfff640 1 -- 172.21.15.67:0/1310721646 learned_addr learned my addr 172.21.15.67:0/1310721646 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5583fff640 1 -- 172.21.15.67:0/1310721646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84ff070 msgr2=0x5557a88c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5583fff640 1 --2- 172.21.15.67:0/1310721646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84ff070 0x5557a88c47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5583fff640 1 -- 172.21.15.67:0/1310721646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84de390 msgr2=0x5557a88ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5583fff640 1 --2- 172.21.15.67:0/1310721646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84de390 0x5557a88ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5583fff640 1 -- 172.21.15.67:0/1310721646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a856a650 con 0x5557a84d61b0 2024-07-31T14:57:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.416+0000 7f5588af1640 1 --2- 172.21.15.67:0/1310721646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84ff070 0x5557a88c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:57:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.417+0000 7f5583fff640 1 --2- 172.21.15.67:0/1310721646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88ac0f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x5557a88a31d0 tx=0x7f5574004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfa85d8c8e91a14 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.417+0000 7f55637fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:57:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.417+0000 7f55637fe640 1 -- 172.21.15.67:0/1310721646 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557401d040 con 0x5557a84d61b0 2024-07-31T14:57:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.417+0000 7f55637fe640 1 -- 172.21.15.67:0/1310721646 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f5574009d30 con 0x5557a84d61b0 2024-07-31T14:57:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.417+0000 7f5590aa5740 1 -- 172.21.15.67:0/1310721646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55740047a0 con 0x5557a84d61b0 2024-07-31T14:57:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.417+0000 7f55637fe640 1 -- 172.21.15.67:0/1310721646 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f557402f050 con 0x5557a84d61b0 2024-07-31T14:57:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.417+0000 7f5590aa5740 10 client.17163 fetch_fsmap learned FSMap version 44 2024-07-31T14:57:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.417+0000 7f5590aa5740 1 -- 172.21.15.67:0/1310721646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5557a88c7520 con 0x5557a84d61b0 2024-07-31T14:57:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.418+0000 7f55637fe640 1 -- 172.21.15.67:0/1310721646 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55740047a0 con 0x5557a84d61b0 2024-07-31T14:57:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.418+0000 7f55637fe640 1 -- 172.21.15.67:0/1310721646 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7f55740035f0 con 0x5557a84d61b0 2024-07-31T14:57:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.418+0000 7f5590aa5740 10 client.17163 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:57:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.418+0000 7f5590aa5740 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:57:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.418+0000 7f5590aa5740 20 client.17163 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.418+0000 7f5590aa5740 1 --2- 172.21.15.67:0/1310721646 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a84bd9f0 0x5557a88cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.418+0000 7f5590aa5740 4 client.17163 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:57:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.418+0000 7f5590aa5740 1 -- 172.21.15.67:0/1310721646 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557a881cae0 con 0x5557a84bd9f0 2024-07-31T14:57:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.418+0000 7f557bfff640 1 --2- 172.21.15.67:0/1310721646 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a84bd9f0 0x5557a88cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.419+0000 7f55637fe640 10 client.17163 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:08.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.419+0000 7f557bfff640 1 --2- 172.21.15.67:0/1310721646 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a84bd9f0 0x5557a88cad90 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26d95525f165ba22 server_cookie=2854ab96fc488b51 in_seq=0 out_seq=0 2024-07-31T14:57:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.430+0000 7f55637fe640 1 -- 172.21.15.67:0/1310721646 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557a881cae0 con 0x5557a84bd9f0 2024-07-31T14:57:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.430+0000 7f55637fe640 10 client.17163 handle_command_reply: tid=0 2024-07-31T14:57:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.430+0000 7f5590aa5740 1 client.17163 shutdown 2024-07-31T14:57:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.430+0000 7f5590aa5740 1 -- 172.21.15.67:0/1310721646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 msgr2=0x5557a88ac0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.430+0000 7f5590aa5740 1 --2- 172.21.15.67:0/1310721646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88ac0f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x5557a88a31d0 tx=0x7f5574004f60 comp rx=0 tx=0).stop 2024-07-31T14:57:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.430+0000 7f5590aa5740 1 -- 172.21.15.67:0/1310721646 shutdown_connections 2024-07-31T14:57:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.430+0000 7f5590aa5740 1 --2- 172.21.15.67:0/1310721646 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a84bd9f0 0x5557a88cad90 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.430+0000 7f5590aa5740 1 --2- 172.21.15.67:0/1310721646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84ff070 0x5557a88c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.430+0000 7f5590aa5740 1 --2- 172.21.15.67:0/1310721646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84de390 0x5557a88ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.430+0000 7f5590aa5740 1 --2- 172.21.15.67:0/1310721646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88ac0f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.430+0000 7f55637fe640 0 client.17163 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.430+0000 7f5590aa5740 1 -- 172.21.15.67:0/1310721646 >> 172.21.15.67:0/1310721646 conn(0x5557a84f3460 msgr2=0x5557a88a0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.431+0000 7f5590aa5740 1 -- 172.21.15.67:0/1310721646 shutdown_connections 2024-07-31T14:57:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.431+0000 7f5590aa5740 1 -- 172.21.15.67:0/1310721646 wait complete. 2024-07-31T14:57:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.431+0000 7f5590aa5740 20 client.17163 trim_cache size 0 max 16384 2024-07-31T14:57:08.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.453+0000 7f5590aa5740 1 Processor -- start 2024-07-31T14:57:08.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.453+0000 7f5590aa5740 1 -- start start 2024-07-31T14:57:08.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.453+0000 7f5590aa5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84d61b0 0x5557a88c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f5590aa5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84de390 0x5557a88a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f5590aa5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84ff070 0x5557a88a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f5590aa5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f55841ae600 con 0x5557a84ff070 2024-07-31T14:57:08.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f5590aa5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f55841b0350 con 0x5557a84d61b0 2024-07-31T14:57:08.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f5590aa5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f55841aed60 con 0x5557a84de390 2024-07-31T14:57:08.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f5583fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84d61b0 0x5557a88c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f5583fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84d61b0 0x5557a88c45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52210/0 (socket says 172.21.15.67:52210) 2024-07-31T14:57:08.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f557bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84de390 0x5557a88a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f5583fff640 1 -- 172.21.15.67:0/1837832999 learned_addr learned my addr 172.21.15.67:0/1837832999 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:08.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f5588af1640 1 --2- 172.21.15.67:0/1837832999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84ff070 0x5557a88a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f5583fff640 1 -- 172.21.15.67:0/1837832999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84de390 msgr2=0x5557a88a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:08.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f5583fff640 1 --2- 172.21.15.67:0/1837832999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84de390 0x5557a88a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f5583fff640 1 -- 172.21.15.67:0/1837832999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84ff070 msgr2=0x5557a88a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:08.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f5583fff640 1 --2- 172.21.15.67:0/1837832999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84ff070 0x5557a88a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.454+0000 7f5583fff640 1 -- 172.21.15.67:0/1837832999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a88ba990 con 0x5557a84d61b0 2024-07-31T14:57:08.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.455+0000 7f5583fff640 1 --2- 172.21.15.67:0/1837832999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84d61b0 0x5557a88c45e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f55740039c0 tx=0x7f5574005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30d3559c41e68a1f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:08.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.455+0000 7f5579ffb640 1 -- 172.21.15.67:0/1837832999 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574004cd0 con 0x5557a84d61b0 2024-07-31T14:57:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.455+0000 7f5579ffb640 1 -- 172.21.15.67:0/1837832999 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5574038070 con 0x5557a84d61b0 2024-07-31T14:57:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.455+0000 7f5579ffb640 1 -- 172.21.15.67:0/1837832999 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574003c30 con 0x5557a84d61b0 2024-07-31T14:57:08.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.456+0000 7f5590aa5740 1 -- 172.21.15.67:0/1837832999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84d61b0 msgr2=0x5557a88c45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:08.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.456+0000 7f5590aa5740 1 --2- 172.21.15.67:0/1837832999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84d61b0 0x5557a88c45e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f55740039c0 tx=0x7f5574005f90 comp rx=0 tx=0).stop 2024-07-31T14:57:08.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.456+0000 7f5590aa5740 1 -- 172.21.15.67:0/1837832999 shutdown_connections 2024-07-31T14:57:08.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.456+0000 7f5590aa5740 1 --2- 172.21.15.67:0/1837832999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84ff070 0x5557a88a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.456+0000 7f5590aa5740 1 --2- 172.21.15.67:0/1837832999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84de390 0x5557a88a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.456+0000 7f5590aa5740 1 --2- 172.21.15.67:0/1837832999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84d61b0 0x5557a88c45e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.456+0000 7f5590aa5740 1 -- 172.21.15.67:0/1837832999 >> 172.21.15.67:0/1837832999 conn(0x5557a84f3460 msgr2=0x5557a88aa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:08.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.456+0000 7f5590aa5740 1 -- 172.21.15.67:0/1837832999 shutdown_connections 2024-07-31T14:57:08.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.456+0000 7f5590aa5740 1 -- 172.21.15.67:0/1837832999 wait complete. 2024-07-31T14:57:08.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.457+0000 7f5590aa5740 1 -- start start 2024-07-31T14:57:08.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.457+0000 7f5590aa5740 1 Processor -- start 2024-07-31T14:57:08.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f5590aa5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88bb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f5590aa5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84de390 0x5557a88bbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f5590aa5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84ff070 0x5557a8931510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f5590aa5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f55841afc80 con 0x5557a84d61b0 2024-07-31T14:57:08.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f5590aa5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f55841af4f0 con 0x5557a84de390 2024-07-31T14:57:08.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f5590aa5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f55841ac5e0 con 0x5557a84ff070 2024-07-31T14:57:08.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f5590aa5740 10 client.0 _ll_register_callbacks cb 0x7f557c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:08.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f557bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84de390 0x5557a88bbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f5583fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88bb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f557bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84de390 0x5557a88bbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52216/0 (socket says 172.21.15.67:52216) 2024-07-31T14:57:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f557bfff640 1 -- 172.21.15.67:0/34315397 learned_addr learned my addr 172.21.15.67:0/34315397 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f5588af1640 1 --2- 172.21.15.67:0/34315397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84ff070 0x5557a8931510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f557bfff640 1 -- 172.21.15.67:0/34315397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84ff070 msgr2=0x5557a8931510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f557bfff640 1 --2- 172.21.15.67:0/34315397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84ff070 0x5557a8931510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f557bfff640 1 -- 172.21.15.67:0/34315397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 msgr2=0x5557a88bb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f557bfff640 1 --2- 172.21.15.67:0/34315397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88bb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f557bfff640 1 -- 172.21.15.67:0/34315397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a88bc5d0 con 0x5557a84de390 2024-07-31T14:57:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f557bfff640 1 --2- 172.21.15.67:0/34315397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84de390 0x5557a88bbdc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f556c03b510 tx=0x7f556c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2d9c564a2f947ca server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.458+0000 7f55637fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:57:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.459+0000 7f55637fe640 1 -- 172.21.15.67:0/34315397 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c0cae50 con 0x5557a84de390 2024-07-31T14:57:08.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.459+0000 7f55637fe640 1 -- 172.21.15.67:0/34315397 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f556c0c70e0 con 0x5557a84de390 2024-07-31T14:57:08.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.459+0000 7f5590aa5740 1 -- 172.21.15.67:0/34315397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5574038070 con 0x5557a84de390 2024-07-31T14:57:08.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.459+0000 7f55637fe640 1 -- 172.21.15.67:0/34315397 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f556c0cf450 con 0x5557a84de390 2024-07-31T14:57:08.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.459+0000 7f5590aa5740 10 client.26741 fetch_fsmap learned FSMap version 44 2024-07-31T14:57:08.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.459+0000 7f5590aa5740 1 -- 172.21.15.67:0/34315397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5557a8932a80 con 0x5557a84de390 2024-07-31T14:57:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.459+0000 7f55637fe640 1 -- 172.21.15.67:0/34315397 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5574038070 con 0x5557a84de390 2024-07-31T14:57:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.459+0000 7f55637fe640 1 -- 172.21.15.67:0/34315397 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7f556c03ba30 con 0x5557a84de390 2024-07-31T14:57:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.459+0000 7f5590aa5740 10 client.26741 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:57:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.459+0000 7f5590aa5740 10 client.26741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:57:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.459+0000 7f5590aa5740 20 client.26741 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.460+0000 7f5590aa5740 1 --2- 172.21.15.67:0/34315397 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a8492da0 0x5557a89352b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.460+0000 7f5590aa5740 4 client.26741 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:57:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.460+0000 7f5590aa5740 1 -- 172.21.15.67:0/34315397 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5557a8818580 con 0x5557a8492da0 2024-07-31T14:57:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.460+0000 7f5583fff640 1 --2- 172.21.15.67:0/34315397 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a8492da0 0x5557a89352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.460+0000 7f55637fe640 10 client.26741 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:08.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.461+0000 7f5583fff640 1 --2- 172.21.15.67:0/34315397 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a8492da0 0x5557a89352b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e264da1b8cc6d8cf server_cookie=9b30f408e9099055 in_seq=0 out_seq=0 2024-07-31T14:57:08.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f55637fe640 1 -- 172.21.15.67:0/34315397 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5557a8818580 con 0x5557a8492da0 2024-07-31T14:57:08.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f55637fe640 10 client.26741 handle_command_reply: tid=0 2024-07-31T14:57:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f5590aa5740 1 client.26741 shutdown 2024-07-31T14:57:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f5590aa5740 1 -- 172.21.15.67:0/34315397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84de390 msgr2=0x5557a88bbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f5590aa5740 1 --2- 172.21.15.67:0/34315397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84de390 0x5557a88bbdc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f556c03b510 tx=0x7f556c009260 comp rx=0 tx=0).stop 2024-07-31T14:57:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f5590aa5740 1 -- 172.21.15.67:0/34315397 shutdown_connections 2024-07-31T14:57:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f5590aa5740 1 --2- 172.21.15.67:0/34315397 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5557a8492da0 0x5557a89352b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f55637fe640 0 client.26741 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f5590aa5740 1 --2- 172.21.15.67:0/34315397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a84ff070 0x5557a8931510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f5590aa5740 1 --2- 172.21.15.67:0/34315397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557a84de390 0x5557a88bbdc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f5590aa5740 1 --2- 172.21.15.67:0/34315397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5557a84d61b0 0x5557a88bb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f5590aa5740 1 -- 172.21.15.67:0/34315397 >> 172.21.15.67:0/34315397 conn(0x5557a84f3460 msgr2=0x5557a88a5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f5590aa5740 1 -- 172.21.15.67:0/34315397 shutdown_connections 2024-07-31T14:57:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f5590aa5740 1 -- 172.21.15.67:0/34315397 wait complete. 2024-07-31T14:57:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.477+0000 7f5590aa5740 20 client.26741 trim_cache size 0 max 16384 2024-07-31T14:57:08.480 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:57:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (7588 inodes in the stack)", 2024-07-31T14:57:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T14:57:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "e85ecfcf-f95a-43ba-9803-181acbdd5565": { 2024-07-31T14:57:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T14:57:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "e85ecfcf-f95a-43ba-9803-181acbdd5565", 2024-07-31T14:57:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T14:57:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:57:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:57:08.481 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:57:08.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.478+0000 7f558a554640 1 -- 172.21.15.67:0/2149273130 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f55700b3d40 msgr2=0x7f55700b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:08.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.478+0000 7f558a554640 1 --2- 172.21.15.67:0/2149273130 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f55700b3d40 0x7f55700b6250 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f557c005fd0 tx=0x7f557c009500 comp rx=0 tx=0).stop 2024-07-31T14:57:08.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.478+0000 7f558a554640 1 -- 172.21.15.67:0/2149273130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584107550 msgr2=0x7f55841993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:08.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.478+0000 7f558a554640 1 --2- 172.21.15.67:0/2149273130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584107550 0x7f55841993f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f556c02a930 tx=0x7f556c031a20 comp rx=0 tx=0).stop 2024-07-31T14:57:08.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.478+0000 7f558a554640 1 -- 172.21.15.67:0/2149273130 shutdown_connections 2024-07-31T14:57:08.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.478+0000 7f558a554640 1 --2- 172.21.15.67:0/2149273130 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f55700b3d40 0x7f55700b6250 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.478+0000 7f558a554640 1 --2- 172.21.15.67:0/2149273130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5584107ea0 0x7f558419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.478+0000 7f558a554640 1 --2- 172.21.15.67:0/2149273130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5584107550 0x7f55841993f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.478+0000 7f558a554640 1 --2- 172.21.15.67:0/2149273130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5584101c60 0x7f5584198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:08.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.478+0000 7f558a554640 1 -- 172.21.15.67:0/2149273130 >> 172.21.15.67:0/2149273130 conn(0x7f5584077690 msgr2=0x7f5584100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:08.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.479+0000 7f558a554640 1 -- 172.21.15.67:0/2149273130 shutdown_connections 2024-07-31T14:57:08.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:08.479+0000 7f558a554640 1 -- 172.21.15.67:0/2149273130 wait complete. 2024-07-31T14:57:08.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: pgmap v582: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 1.49k op/s 2024-07-31T14:57:08.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:08.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: pgmap v582: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 1.49k op/s 2024-07-31T14:57:08.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:08.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:08.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: pgmap v582: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 1.49k op/s 2024-07-31T14:57:09.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:09.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.306 INFO:tasks.cephfs.filesystem:scrub status for tag:e85ecfcf-f95a-43ba-9803-181acbdd5565 - {'path': '/', 'tag': 'e85ecfcf-f95a-43ba-9803-181acbdd5565', 'options': 'recursive,force'} 2024-07-31T14:57:09.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:09.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:09.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:09.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:09.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:09.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:09.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:09.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:09.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:09.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:09.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:09.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:09.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:10.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:10.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:10.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:10.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:10.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:10.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:10.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:10.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:10.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:10.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:10.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:10.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:10.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:10.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:10.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:10.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:10.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:10.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:10.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:10.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:10.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:10.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:10 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:10 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:10 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:10.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:10 smithi077 ceph-mon[37327]: pgmap v583: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 3.7 MiB/s wr, 1.62k op/s 2024-07-31T14:57:11.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:10 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:11.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:10 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:11.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:10 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:11.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:10 smithi067 ceph-mon[32750]: pgmap v583: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 3.7 MiB/s wr, 1.62k op/s 2024-07-31T14:57:11.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:10 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:11.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:10 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:11.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:10 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:11.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:10 smithi073 ceph-mon[36714]: pgmap v583: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 3.7 MiB/s wr, 1.62k op/s 2024-07-31T14:57:12.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:12 smithi077 ceph-mon[37327]: pgmap v584: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 3.5 MiB/s wr, 1.67k op/s 2024-07-31T14:57:13.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:12 smithi067 ceph-mon[32750]: pgmap v584: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 3.5 MiB/s wr, 1.67k op/s 2024-07-31T14:57:13.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:12 smithi073 ceph-mon[36714]: pgmap v584: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 3.5 MiB/s wr, 1.67k op/s 2024-07-31T14:57:14.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:14 smithi077 ceph-mon[37327]: pgmap v585: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 1.68k op/s 2024-07-31T14:57:15.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:14 smithi067 ceph-mon[32750]: pgmap v585: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 1.68k op/s 2024-07-31T14:57:15.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:14 smithi073 ceph-mon[36714]: pgmap v585: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 1.68k op/s 2024-07-31T14:57:15.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:15.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:16.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:16.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:16.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:16.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:16.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:16 smithi077 ceph-mon[37327]: pgmap v586: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 1.41k op/s 2024-07-31T14:57:16.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:16.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:17.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:16 smithi067 ceph-mon[32750]: pgmap v586: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 1.41k op/s 2024-07-31T14:57:17.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:17.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:17.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:16 smithi073 ceph-mon[36714]: pgmap v586: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 1.41k op/s 2024-07-31T14:57:17.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:17.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.249+0000 7f0b4597c640 1 -- 172.21.15.67:0/585777906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4010d7a0 msgr2=0x7f0b4010fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.249+0000 7f0b4597c640 1 --2- 172.21.15.67:0/585777906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4010d7a0 0x7f0b4010fc90 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009ca0 tx=0x7f0b3002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.249+0000 7f0b4597c640 1 -- 172.21.15.67:0/585777906 shutdown_connections 2024-07-31T14:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.249+0000 7f0b4597c640 1 --2- 172.21.15.67:0/585777906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4010d7a0 0x7f0b4010fc90 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.249+0000 7f0b4597c640 1 --2- 172.21.15.67:0/585777906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b40104c90 0x7f0b4010d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.249+0000 7f0b4597c640 1 --2- 172.21.15.67:0/585777906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b40104340 0x7f0b40104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.249+0000 7f0b4597c640 1 -- 172.21.15.67:0/585777906 >> 172.21.15.67:0/585777906 conn(0x7f0b400fddf0 msgr2=0x7f0b40100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.249+0000 7f0b4597c640 1 -- 172.21.15.67:0/585777906 shutdown_connections 2024-07-31T14:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.249+0000 7f0b4597c640 1 -- 172.21.15.67:0/585777906 wait complete. 2024-07-31T14:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.250+0000 7f0b4597c640 1 Processor -- start 2024-07-31T14:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.250+0000 7f0b4597c640 1 -- start start 2024-07-31T14:57:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.250+0000 7f0b4597c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b40104340 0x7f0b40199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.250+0000 7f0b4597c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b40104c90 0x7f0b4019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.250+0000 7f0b4597c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4010d7a0 0x7f0b4019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.250+0000 7f0b4597c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0b40113430 con 0x7f0b4010d7a0 2024-07-31T14:57:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b4597c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401132b0 con 0x7f0b40104340 2024-07-31T14:57:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b4597c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401135b0 con 0x7f0b40104c90 2024-07-31T14:57:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b3f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4010d7a0 0x7f0b4019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b3f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4010d7a0 0x7f0b4019f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52022/0 (socket says 172.21.15.67:52022) 2024-07-31T14:57:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b3f7fe640 1 -- 172.21.15.67:0/3382271068 learned_addr learned my addr 172.21.15.67:0/3382271068 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b3effd640 1 --2- 172.21.15.67:0/3382271068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b40104340 0x7f0b40199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b3e7fc640 1 --2- 172.21.15.67:0/3382271068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b40104c90 0x7f0b4019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b3f7fe640 1 -- 172.21.15.67:0/3382271068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b40104c90 msgr2=0x7f0b4019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b3f7fe640 1 --2- 172.21.15.67:0/3382271068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b40104c90 0x7f0b4019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b3f7fe640 1 -- 172.21.15.67:0/3382271068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b40104340 msgr2=0x7f0b40199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b3f7fe640 1 --2- 172.21.15.67:0/3382271068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b40104340 0x7f0b40199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b3f7fe640 1 -- 172.21.15.67:0/3382271068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4019fd40 con 0x7f0b4010d7a0 2024-07-31T14:57:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b3f7fe640 1 --2- 172.21.15.67:0/3382271068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4010d7a0 0x7f0b4019f640 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0b30005cc0 tx=0x7f0b30031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b4497a640 1 -- 172.21.15.67:0/3382271068 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30003b70 con 0x7f0b4010d7a0 2024-07-31T14:57:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b4597c640 1 -- 172.21.15.67:0/3382271068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4019ffd0 con 0x7f0b4010d7a0 2024-07-31T14:57:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b4497a640 1 -- 172.21.15.67:0/3382271068 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b300318f0 con 0x7f0b4010d7a0 2024-07-31T14:57:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.251+0000 7f0b4497a640 1 -- 172.21.15.67:0/3382271068 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3002f950 con 0x7f0b4010d7a0 2024-07-31T14:57:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.252+0000 7f0b4597c640 1 -- 172.21.15.67:0/3382271068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b401a67c0 con 0x7f0b4010d7a0 2024-07-31T14:57:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.254+0000 7f0b4c87b740 1 Processor -- start 2024-07-31T14:57:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.255+0000 7f0b4497a640 1 -- 172.21.15.67:0/3382271068 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0b3002fb30 con 0x7f0b4010d7a0 2024-07-31T14:57:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.255+0000 7f0b4c87b740 1 -- start start 2024-07-31T14:57:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.255+0000 7f0b4c87b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202c11b0 0x55cc20663ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.255+0000 7f0b4c87b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202982f0 0x55cc20664220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.255+0000 7f0b4c87b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202a04d0 0x55cc20664760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.255+0000 7f0b4c87b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cc205d93b0 con 0x55cc202c11b0 2024-07-31T14:57:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.255+0000 7f0b4c87b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cc205c88b0 con 0x55cc202982f0 2024-07-31T14:57:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.256+0000 7f0b4c87b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cc205cc8f0 con 0x55cc202a04d0 2024-07-31T14:57:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.256+0000 7f0b3effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202982f0 0x55cc20664220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.256+0000 7f0b3effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202982f0 0x55cc20664220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45764/0 (socket says 172.21.15.67:45764) 2024-07-31T14:57:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.256+0000 7f0b3effd640 1 -- 172.21.15.67:0/721448309 learned_addr learned my addr 172.21.15.67:0/721448309 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.256+0000 7f0b3e7fc640 1 --2- 172.21.15.67:0/721448309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202c11b0 0x55cc20663ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.256+0000 7f0b3e7fc640 1 --2- 172.21.15.67:0/721448309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202a04d0 0x55cc20664760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.256+0000 7f0b3effd640 1 -- 172.21.15.67:0/721448309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202a04d0 msgr2=0x55cc20664760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.256+0000 7f0b3effd640 1 --2- 172.21.15.67:0/721448309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202a04d0 0x55cc20664760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.256+0000 7f0b3effd640 1 -- 172.21.15.67:0/721448309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202c11b0 msgr2=0x55cc20663ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.256+0000 7f0b3effd640 1 --2- 172.21.15.67:0/721448309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202c11b0 0x55cc20663ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.256+0000 7f0b3effd640 1 -- 172.21.15.67:0/721448309 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc2056f9a0 con 0x55cc202982f0 2024-07-31T14:57:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.256+0000 7f0b3effd640 1 --2- 172.21.15.67:0/721448309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202982f0 0x55cc20664220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009cf0 tx=0x7f0b38011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fd75febea58e720 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.257+0000 7f0b4497a640 1 --2- 172.21.15.67:0/3382271068 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0b2c0b3d40 0x7f0b2c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.257+0000 7f0b35ffb640 1 -- 172.21.15.67:0/721448309 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38004230 con 0x55cc202982f0 2024-07-31T14:57:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.257+0000 7f0b35ffb640 1 -- 172.21.15.67:0/721448309 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b380043d0 con 0x55cc202982f0 2024-07-31T14:57:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.257+0000 7f0b3e7fc640 1 --2- 172.21.15.67:0/3382271068 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0b2c0b3d40 0x7f0b2c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.257+0000 7f0b35ffb640 1 -- 172.21.15.67:0/721448309 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b380197c0 con 0x55cc202982f0 2024-07-31T14:57:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.257+0000 7f0b4497a640 1 -- 172.21.15.67:0/3382271068 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f0b30105740 con 0x7f0b4010d7a0 2024-07-31T14:57:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.258+0000 7f0b4c87b740 1 -- 172.21.15.67:0/721448309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202982f0 msgr2=0x55cc20664220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.258+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/721448309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202982f0 0x55cc20664220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009cf0 tx=0x7f0b38011810 comp rx=0 tx=0).stop 2024-07-31T14:57:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.259+0000 7f0b3e7fc640 1 --2- 172.21.15.67:0/3382271068 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0b2c0b3d40 0x7f0b2c0b6250 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009710 tx=0x7f0b28005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.259+0000 7f0b4c87b740 1 -- 172.21.15.67:0/721448309 shutdown_connections 2024-07-31T14:57:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.259+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/721448309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202a04d0 0x55cc20664760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.259+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/721448309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202982f0 0x55cc20664220 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.259+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/721448309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202c11b0 0x55cc20663ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.259+0000 7f0b4c87b740 1 -- 172.21.15.67:0/721448309 >> 172.21.15.67:0/721448309 conn(0x55cc202b5300 msgr2=0x55cc20661bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.259+0000 7f0b4c87b740 1 -- 172.21.15.67:0/721448309 shutdown_connections 2024-07-31T14:57:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.259+0000 7f0b4c87b740 1 -- 172.21.15.67:0/721448309 wait complete. 2024-07-31T14:57:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.260+0000 7f0b4c87b740 1 -- start start 2024-07-31T14:57:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.260+0000 7f0b4c87b740 1 Processor -- start 2024-07-31T14:57:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.260+0000 7f0b4c87b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202982f0 0x55cc20673980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.260+0000 7f0b4c87b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202a04d0 0x55cc20673f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.261+0000 7f0b4c87b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202c11b0 0x55cc2067fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.261+0000 7f0b3effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202982f0 0x55cc20673980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.261+0000 7f0b3effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202982f0 0x55cc20673980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52040/0 (socket says 172.21.15.67:52040) 2024-07-31T14:57:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.261+0000 7f0b3effd640 1 -- 172.21.15.67:0/2461311978 learned_addr learned my addr 172.21.15.67:0/2461311978 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.261+0000 7f0b3e7fc640 1 --2- 172.21.15.67:0/2461311978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202a04d0 0x55cc20673f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.261+0000 7f0b3f7fe640 1 --2- 172.21.15.67:0/2461311978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202c11b0 0x55cc2067fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.261+0000 7f0b4c87b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cc20502ab0 con 0x55cc202982f0 2024-07-31T14:57:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.261+0000 7f0b4c87b740 1 -- 172.21.15.67:0/2461311978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cc20550cf0 con 0x55cc202a04d0 2024-07-31T14:57:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.261+0000 7f0b4c87b740 1 -- 172.21.15.67:0/2461311978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cc20676890 con 0x55cc202c11b0 2024-07-31T14:57:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.262+0000 7f0b4c87b740 10 client.0 _ll_register_callbacks cb 0x55cc205c29f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.262+0000 7f0b3effd640 1 -- 172.21.15.67:0/2461311978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202c11b0 msgr2=0x55cc2067fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.262+0000 7f0b3effd640 1 --2- 172.21.15.67:0/2461311978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202c11b0 0x55cc2067fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.262+0000 7f0b3effd640 1 -- 172.21.15.67:0/2461311978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202a04d0 msgr2=0x55cc20673f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.262+0000 7f0b3effd640 1 --2- 172.21.15.67:0/2461311978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202a04d0 0x55cc20673f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.262+0000 7f0b3effd640 1 -- 172.21.15.67:0/2461311978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc2032c790 con 0x55cc202982f0 2024-07-31T14:57:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.262+0000 7f0b3e7fc640 1 --2- 172.21.15.67:0/2461311978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202a04d0 0x55cc20673f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:57:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.262+0000 7f0b3effd640 1 --2- 172.21.15.67:0/2461311978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202982f0 0x55cc20673980 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0b38003f50 tx=0x7f0b380045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e483dcce081efa15 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.262+0000 7f0b1effd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:57:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.263+0000 7f0b1effd640 1 -- 172.21.15.67:0/2461311978 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38027070 con 0x55cc202982f0 2024-07-31T14:57:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.263+0000 7f0b4c87b740 1 -- 172.21.15.67:0/2461311978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b380043d0 con 0x55cc202982f0 2024-07-31T14:57:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.263+0000 7f0b1effd640 1 -- 172.21.15.67:0/2461311978 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f0b380037f0 con 0x55cc202982f0 2024-07-31T14:57:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.263+0000 7f0b1effd640 1 -- 172.21.15.67:0/2461311978 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0b38033050 con 0x55cc202982f0 2024-07-31T14:57:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.264+0000 7f0b4c87b740 10 client.17181 fetch_fsmap learned FSMap version 44 2024-07-31T14:57:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.264+0000 7f0b4c87b740 1 -- 172.21.15.67:0/2461311978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55cc203a5620 con 0x55cc202982f0 2024-07-31T14:57:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.264+0000 7f0b1effd640 1 -- 172.21.15.67:0/2461311978 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b380043d0 con 0x55cc202982f0 2024-07-31T14:57:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.265+0000 7f0b1effd640 1 -- 172.21.15.67:0/2461311978 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7f0b38004b70 con 0x55cc202982f0 2024-07-31T14:57:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.265+0000 7f0b4c87b740 10 client.17181 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:57:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.265+0000 7f0b4c87b740 10 client.17181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:57:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.265+0000 7f0b4c87b740 20 client.17181 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.265+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/2461311978 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc2027fd80 0x55cc20685a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.265+0000 7f0b4c87b740 4 client.17181 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:57:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.265+0000 7f0b4c87b740 1 -- 172.21.15.67:0/2461311978 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc205db300 con 0x55cc2027fd80 2024-07-31T14:57:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.266+0000 7f0b3e7fc640 1 --2- 172.21.15.67:0/2461311978 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc2027fd80 0x55cc20685a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.267+0000 7f0b3e7fc640 1 --2- 172.21.15.67:0/2461311978 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc2027fd80 0x55cc20685a60 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a5a9f40bdb10af2 server_cookie=4aaa2d2e2a50ec7b in_seq=0 out_seq=0 2024-07-31T14:57:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.267+0000 7f0b1effd640 10 client.17181 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.273+0000 7f0b1effd640 1 -- 172.21.15.67:0/2461311978 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc205db300 con 0x55cc2027fd80 2024-07-31T14:57:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.273+0000 7f0b1effd640 10 client.17181 handle_command_reply: tid=0 2024-07-31T14:57:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.273+0000 7f0b4c87b740 1 client.17181 shutdown 2024-07-31T14:57:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.273+0000 7f0b4c87b740 1 -- 172.21.15.67:0/2461311978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202982f0 msgr2=0x55cc20673980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.274+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/2461311978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202982f0 0x55cc20673980 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0b38003f50 tx=0x7f0b380045f0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.274+0000 7f0b4c87b740 1 -- 172.21.15.67:0/2461311978 shutdown_connections 2024-07-31T14:57:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.274+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/2461311978 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc2027fd80 0x55cc20685a60 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.274+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/2461311978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202c11b0 0x55cc2067fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.274+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/2461311978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202a04d0 0x55cc20673f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.274+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/2461311978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202982f0 0x55cc20673980 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.274+0000 7f0b4c87b740 1 -- 172.21.15.67:0/2461311978 >> 172.21.15.67:0/2461311978 conn(0x55cc202b5300 msgr2=0x55cc206656d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.274+0000 7f0b1effd640 0 client.17181 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.274+0000 7f0b4c87b740 1 -- 172.21.15.67:0/2461311978 shutdown_connections 2024-07-31T14:57:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.274+0000 7f0b4c87b740 1 -- 172.21.15.67:0/2461311978 wait complete. 2024-07-31T14:57:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.274+0000 7f0b4c87b740 20 client.17181 trim_cache size 0 max 16384 2024-07-31T14:57:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.301+0000 7f0b4c87b740 1 Processor -- start 2024-07-31T14:57:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.302+0000 7f0b4c87b740 1 -- start start 2024-07-31T14:57:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.302+0000 7f0b4c87b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202a04d0 0x55cc2067fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.302+0000 7f0b4c87b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202c11b0 0x55cc2067ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.302+0000 7f0b4c87b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202551c0 0x55cc206731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.302+0000 7f0b4c87b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401ae4b0 con 0x55cc202551c0 2024-07-31T14:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.302+0000 7f0b4c87b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401b0200 con 0x55cc202a04d0 2024-07-31T14:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.302+0000 7f0b4c87b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401aec10 con 0x55cc202c11b0 2024-07-31T14:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.302+0000 7f0b3f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202551c0 0x55cc206731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.302+0000 7f0b3effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202a04d0 0x55cc2067fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.302+0000 7f0b3f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202551c0 0x55cc206731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52048/0 (socket says 172.21.15.67:52048) 2024-07-31T14:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.302+0000 7f0b3f7fe640 1 -- 172.21.15.67:0/1868197568 learned_addr learned my addr 172.21.15.67:0/1868197568 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.302+0000 7f0b3e7fc640 1 --2- 172.21.15.67:0/1868197568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202c11b0 0x55cc2067ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.303+0000 7f0b3effd640 1 -- 172.21.15.67:0/1868197568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202c11b0 msgr2=0x55cc2067ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.303+0000 7f0b3effd640 1 --2- 172.21.15.67:0/1868197568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202c11b0 0x55cc2067ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.303+0000 7f0b3effd640 1 -- 172.21.15.67:0/1868197568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202551c0 msgr2=0x55cc206731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.303+0000 7f0b3effd640 1 --2- 172.21.15.67:0/1868197568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202551c0 0x55cc206731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.303+0000 7f0b3effd640 1 -- 172.21.15.67:0/1868197568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc20680490 con 0x55cc202a04d0 2024-07-31T14:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.303+0000 7f0b3f7fe640 1 --2- 172.21.15.67:0/1868197568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202551c0 0x55cc206731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.303+0000 7f0b3effd640 1 --2- 172.21.15.67:0/1868197568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202a04d0 0x55cc2067fa10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0b38013000 tx=0x7f0b38019420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeb87775667a000 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.303+0000 7f0b357fa640 1 -- 172.21.15.67:0/1868197568 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38030070 con 0x55cc202a04d0 2024-07-31T14:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.303+0000 7f0b357fa640 1 -- 172.21.15.67:0/1868197568 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b38009910 con 0x55cc202a04d0 2024-07-31T14:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.303+0000 7f0b357fa640 1 -- 172.21.15.67:0/1868197568 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38027040 con 0x55cc202a04d0 2024-07-31T14:57:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.304+0000 7f0b4c87b740 1 -- 172.21.15.67:0/1868197568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202a04d0 msgr2=0x55cc2067fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.304+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/1868197568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202a04d0 0x55cc2067fa10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0b38013000 tx=0x7f0b38019420 comp rx=0 tx=0).stop 2024-07-31T14:57:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.304+0000 7f0b4c87b740 1 -- 172.21.15.67:0/1868197568 shutdown_connections 2024-07-31T14:57:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.304+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/1868197568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202551c0 0x55cc206731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.304+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/1868197568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202c11b0 0x55cc2067ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.304+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/1868197568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202a04d0 0x55cc2067fa10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.304+0000 7f0b4c87b740 1 -- 172.21.15.67:0/1868197568 >> 172.21.15.67:0/1868197568 conn(0x55cc202b5300 msgr2=0x55cc2066ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.304+0000 7f0b4c87b740 1 -- 172.21.15.67:0/1868197568 shutdown_connections 2024-07-31T14:57:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.304+0000 7f0b4c87b740 1 -- 172.21.15.67:0/1868197568 wait complete. 2024-07-31T14:57:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.304+0000 7f0b4c87b740 1 -- start start 2024-07-31T14:57:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b4c87b740 1 Processor -- start 2024-07-31T14:57:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b4c87b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202551c0 0x55cc2067c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b4c87b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202a04d0 0x55cc20678b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b4c87b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202c11b0 0x55cc20679240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b4c87b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401afb30 con 0x55cc202c11b0 2024-07-31T14:57:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b4c87b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401af3a0 con 0x55cc202551c0 2024-07-31T14:57:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b4c87b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401ac490 con 0x55cc202a04d0 2024-07-31T14:57:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b4c87b740 10 client.0 _ll_register_callbacks cb 0x7f0b28009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b3effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202551c0 0x55cc2067c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b3effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202551c0 0x55cc2067c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45780/0 (socket says 172.21.15.67:45780) 2024-07-31T14:57:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b3effd640 1 -- 172.21.15.67:0/1008776811 learned_addr learned my addr 172.21.15.67:0/1008776811 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b3f7fe640 1 --2- 172.21.15.67:0/1008776811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202c11b0 0x55cc20679240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b3e7fc640 1 --2- 172.21.15.67:0/1008776811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202a04d0 0x55cc20678b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b3effd640 1 -- 172.21.15.67:0/1008776811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202a04d0 msgr2=0x55cc20678b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b3effd640 1 --2- 172.21.15.67:0/1008776811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202a04d0 0x55cc20678b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b3effd640 1 -- 172.21.15.67:0/1008776811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202c11b0 msgr2=0x55cc20679240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b3effd640 1 --2- 172.21.15.67:0/1008776811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202c11b0 0x55cc20679240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b3effd640 1 -- 172.21.15.67:0/1008776811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc206ec1e0 con 0x55cc202551c0 2024-07-31T14:57:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.305+0000 7f0b3f7fe640 1 --2- 172.21.15.67:0/1008776811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202c11b0 0x55cc20679240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:57:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b3effd640 1 --2- 172.21.15.67:0/1008776811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202551c0 0x55cc2067c3d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0b38000c00 tx=0x7f0b38019690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97257ecbe0bbb34 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b1effd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:57:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b1effd640 1 -- 172.21.15.67:0/1008776811 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38047060 con 0x55cc202551c0 2024-07-31T14:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b4c87b740 1 -- 172.21.15.67:0/1008776811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b38009910 con 0x55cc202551c0 2024-07-31T14:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b1effd640 1 -- 172.21.15.67:0/1008776811 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f0b38019b30 con 0x55cc202551c0 2024-07-31T14:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b1effd640 1 -- 172.21.15.67:0/1008776811 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0b3804e050 con 0x55cc202551c0 2024-07-31T14:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b4c87b740 10 client.26759 fetch_fsmap learned FSMap version 44 2024-07-31T14:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b4c87b740 1 -- 172.21.15.67:0/1008776811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55cc206ed2b0 con 0x55cc202551c0 2024-07-31T14:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b1effd640 1 -- 172.21.15.67:0/1008776811 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b38009910 con 0x55cc202551c0 2024-07-31T14:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b1effd640 1 -- 172.21.15.67:0/1008776811 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7f0b38004720 con 0x55cc202551c0 2024-07-31T14:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b4c87b740 10 client.26759 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b4c87b740 10 client.26759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:57:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b4c87b740 20 client.26759 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/1008776811 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc206ed9d0 0x55cc206efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b4c87b740 4 client.26759 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:57:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.306+0000 7f0b4c87b740 1 -- 172.21.15.67:0/1008776811 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc205d6da0 con 0x55cc206ed9d0 2024-07-31T14:57:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.307+0000 7f0b3e7fc640 1 --2- 172.21.15.67:0/1008776811 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc206ed9d0 0x55cc206efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.307+0000 7f0b1effd640 10 client.26759 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:17.307+0000 7f0b3e7fc640 1 --2- 172.21.15.67:0/1008776811 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc206ed9d0 0x55cc206efdf0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a83e8cd7a429b68f server_cookie=e65753ad4513756a in_seq=0 out_seq=0 2024-07-31T14:57:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.473+0000 7f0b1effd640 1 -- 172.21.15.67:0/1008776811 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc205d6da0 con 0x55cc206ed9d0 2024-07-31T14:57:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.473+0000 7f0b1effd640 10 client.26759 handle_command_reply: tid=0 2024-07-31T14:57:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.473+0000 7f0b4c87b740 1 client.26759 shutdown 2024-07-31T14:57:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.474+0000 7f0b4c87b740 1 -- 172.21.15.67:0/1008776811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202551c0 msgr2=0x55cc2067c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.474+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/1008776811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202551c0 0x55cc2067c3d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0b38000c00 tx=0x7f0b38019690 comp rx=0 tx=0).stop 2024-07-31T14:57:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.474+0000 7f0b4c87b740 1 -- 172.21.15.67:0/1008776811 shutdown_connections 2024-07-31T14:57:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.474+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/1008776811 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc206ed9d0 0x55cc206efdf0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.474+0000 7f0b1effd640 0 client.26759 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.474+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/1008776811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc202c11b0 0x55cc20679240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.474+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/1008776811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc202a04d0 0x55cc20678b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.474+0000 7f0b4c87b740 1 --2- 172.21.15.67:0/1008776811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc202551c0 0x55cc2067c3d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.474+0000 7f0b4c87b740 1 -- 172.21.15.67:0/1008776811 >> 172.21.15.67:0/1008776811 conn(0x55cc202b5300 msgr2=0x55cc2066ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.474+0000 7f0b4c87b740 1 -- 172.21.15.67:0/1008776811 shutdown_connections 2024-07-31T14:57:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.474+0000 7f0b4c87b740 1 -- 172.21.15.67:0/1008776811 wait complete. 2024-07-31T14:57:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.474+0000 7f0b4c87b740 20 client.26759 trim_cache size 0 max 16384 2024-07-31T14:57:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:57:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:57:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:57:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:57:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.475+0000 7f0b4597c640 1 -- 172.21.15.67:0/3382271068 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0b2c0b3d40 msgr2=0x7f0b2c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.475+0000 7f0b4597c640 1 --2- 172.21.15.67:0/3382271068 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0b2c0b3d40 0x7f0b2c0b6250 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009710 tx=0x7f0b28005850 comp rx=0 tx=0).stop 2024-07-31T14:57:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.475+0000 7f0b4597c640 1 -- 172.21.15.67:0/3382271068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4010d7a0 msgr2=0x7f0b4019f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.475+0000 7f0b4597c640 1 --2- 172.21.15.67:0/3382271068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4010d7a0 0x7f0b4019f640 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0b30005cc0 tx=0x7f0b30031700 comp rx=0 tx=0).stop 2024-07-31T14:57:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.475+0000 7f0b4597c640 1 -- 172.21.15.67:0/3382271068 shutdown_connections 2024-07-31T14:57:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.475+0000 7f0b4597c640 1 --2- 172.21.15.67:0/3382271068 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0b2c0b3d40 0x7f0b2c0b6250 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.475+0000 7f0b4597c640 1 --2- 172.21.15.67:0/3382271068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4010d7a0 0x7f0b4019f640 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.475+0000 7f0b4597c640 1 --2- 172.21.15.67:0/3382271068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b40104c90 0x7f0b4019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.475+0000 7f0b4597c640 1 --2- 172.21.15.67:0/3382271068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b40104340 0x7f0b40199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.475+0000 7f0b4597c640 1 -- 172.21.15.67:0/3382271068 >> 172.21.15.67:0/3382271068 conn(0x7f0b400fddf0 msgr2=0x7f0b40105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.475+0000 7f0b4597c640 1 -- 172.21.15.67:0/3382271068 shutdown_connections 2024-07-31T14:57:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:18.475+0000 7f0b4597c640 1 -- 172.21.15.67:0/3382271068 wait complete. 2024-07-31T14:57:18.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:18.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:18.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:18 smithi077 ceph-mon[37327]: pgmap v587: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 830 op/s 2024-07-31T14:57:18.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:18.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:19.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:19.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:19.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:18 smithi067 ceph-mon[32750]: pgmap v587: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 830 op/s 2024-07-31T14:57:19.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:19.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:19.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:19.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:19.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:18 smithi073 ceph-mon[36714]: pgmap v587: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 830 op/s 2024-07-31T14:57:19.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:19.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:19.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:19.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:19.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:19.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:19.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:19.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:19.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:19.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:19.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:19.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:19.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:19.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:19.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:19.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:19.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:19.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:19.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:19.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:20.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:20.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:20.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:20.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:20.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:20.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:20.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:20.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:20.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:20.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:20.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:20.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:20.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:20.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:20.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:20.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:20.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:20.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:20.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:20.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:20 smithi077 ceph-mon[37327]: pgmap v588: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 827 op/s 2024-07-31T14:57:20.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:20.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:20.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:20.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:21.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:21.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:20 smithi067 ceph-mon[32750]: pgmap v588: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 827 op/s 2024-07-31T14:57:21.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:21.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:21.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:21.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:21.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:20 smithi073 ceph-mon[36714]: pgmap v588: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 827 op/s 2024-07-31T14:57:21.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:21.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:21.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:21.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:21.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:21.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:21.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:21.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:21.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:21.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:21.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:22.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:22.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:22.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:22.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:22.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:22.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:22.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:22.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:22.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:22.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.869+0000 7f5264ff9640 1 -- 172.21.15.67:0/2607080064 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556aedf23170 con 0x556aedb9e570 2024-07-31T14:57:22.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.869+0000 7f5264ff9640 10 client.26681 handle_command_reply: tid=0 2024-07-31T14:57:22.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.869+0000 7f5289d5c740 1 client.26681 shutdown 2024-07-31T14:57:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.869+0000 7f5289d5c740 1 -- 172.21.15.67:0/2607080064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbc91c0 msgr2=0x556aedfd1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.869+0000 7f5289d5c740 1 --2- 172.21.15.67:0/2607080064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbc91c0 0x556aedfd1c50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5274004860 tx=0x7f526c005ef0 comp rx=0 tx=0).stop 2024-07-31T14:57:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.869+0000 7f5289d5c740 1 -- 172.21.15.67:0/2607080064 shutdown_connections 2024-07-31T14:57:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.869+0000 7f5289d5c740 1 --2- 172.21.15.67:0/2607080064 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556aedb9e570 0x556aee0396d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.870+0000 7f5289d5c740 1 --2- 172.21.15.67:0/2607080064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aedc0a840 0x556aedfd26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.870+0000 7f5264ff9640 0 client.26681 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.870+0000 7f5289d5c740 1 --2- 172.21.15.67:0/2607080064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aedbe1980 0x556aedfd2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.870+0000 7f5289d5c740 1 --2- 172.21.15.67:0/2607080064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556aedbc91c0 0x556aedfd1c50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.870+0000 7f5289d5c740 1 -- 172.21.15.67:0/2607080064 >> 172.21.15.67:0/2607080064 conn(0x556aedbfe990 msgr2=0x556aedfcb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:22.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.870+0000 7f5289d5c740 1 -- 172.21.15.67:0/2607080064 shutdown_connections 2024-07-31T14:57:22.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.870+0000 7f5289d5c740 1 -- 172.21.15.67:0/2607080064 wait complete. 2024-07-31T14:57:22.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.870+0000 7f5289d5c740 20 client.26681 trim_cache size 0 max 16384 2024-07-31T14:57:22.873 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:57:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 44, 2024-07-31T14:57:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T14:57:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 14, 2024-07-31T14:57:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T14:57:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "c8d56e19": { 2024-07-31T14:57:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 14, 2024-07-31T14:57:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T14:57:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:57:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T14:57:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:57:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:57:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T14:57:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 130.0, 2024-07-31T14:57:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T14:57:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T14:57:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T14:57:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:57:22.875 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T14:57:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:57:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:57:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:57:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:57:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:57:22.876 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:57:22.876 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:57:22.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.871+0000 7f528380b640 1 -- 172.21.15.67:0/740627995 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f52680b3df0 msgr2=0x7f52680b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:22.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.871+0000 7f528380b640 1 --2- 172.21.15.67:0/740627995 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f52680b3df0 0x7f52680b6300 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x556aedfaf4c0 tx=0x7f52780e9510 comp rx=0 tx=0).stop 2024-07-31T14:57:22.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.871+0000 7f528380b640 1 -- 172.21.15.67:0/740627995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f527c1024b0 msgr2=0x7f527c19ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:22.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.871+0000 7f528380b640 1 --2- 172.21.15.67:0/740627995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f527c1024b0 0x7f527c19ad80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f527800d770 tx=0x7f527800db70 comp rx=0 tx=0).stop 2024-07-31T14:57:22.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.871+0000 7f528380b640 1 -- 172.21.15.67:0/740627995 shutdown_connections 2024-07-31T14:57:22.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.871+0000 7f528380b640 1 --2- 172.21.15.67:0/740627995 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f52680b3df0 0x7f52680b6300 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:22.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.871+0000 7f528380b640 1 --2- 172.21.15.67:0/740627995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f527c108740 0x7f527c19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:22.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.871+0000 7f528380b640 1 --2- 172.21.15.67:0/740627995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c107df0 0x7f527c19b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:22.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.871+0000 7f528380b640 1 --2- 172.21.15.67:0/740627995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f527c1024b0 0x7f527c19ad80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:22.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.871+0000 7f528380b640 1 -- 172.21.15.67:0/740627995 >> 172.21.15.67:0/740627995 conn(0x7f527c079190 msgr2=0x7f527c103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:22.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.871+0000 7f528380b640 1 -- 172.21.15.67:0/740627995 shutdown_connections 2024-07-31T14:57:22.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:22.871+0000 7f528380b640 1 -- 172.21.15.67:0/740627995 wait complete. 2024-07-31T14:57:22.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:22 smithi077 ceph-mon[37327]: pgmap v589: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 854 op/s 2024-07-31T14:57:23.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:22 smithi067 ceph-mon[32750]: pgmap v589: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 854 op/s 2024-07-31T14:57:23.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:22 smithi073 ceph-mon[36714]: pgmap v589: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 854 op/s 2024-07-31T14:57:23.723 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c8d56e19', quiesce took 42.6 seconds. Will release after: -32.6 2024-07-31T14:57:23.724 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:57:23.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:23.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.027 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:57:24.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:24.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.965 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.965 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.965 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.965 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.965 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.965 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.965 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.965 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.966 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.966 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.966 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.966 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.966 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.966 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.966 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.966 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.966 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.966 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.966 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.966 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: pgmap v590: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 941 op/s 2024-07-31T14:57:24.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: pgmap v590: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 941 op/s 2024-07-31T14:57:24.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:24.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:24.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: pgmap v590: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 941 op/s 2024-07-31T14:57:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.148 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.148 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.148 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.148 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.148 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.148 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.148 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.148 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:25.148 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:25.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:25.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.147 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.148 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.148 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.148 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.148 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:26.148 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: pgmap v591: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.2 MiB/s wr, 1.61k op/s 2024-07-31T14:57:26.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:57:26.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: pgmap v591: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.2 MiB/s wr, 1.61k op/s 2024-07-31T14:57:26.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:57:26.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:26.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.118+0000 7f4694c04640 1 -- 172.21.15.67:0/1199300731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46901035b0 msgr2=0x7f46901039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.118+0000 7f4694c04640 1 --2- 172.21.15.67:0/1199300731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46901035b0 0x7f46901039c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4680009860 tx=0x7f468002a340 comp rx=0 tx=0).stop 2024-07-31T14:57:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.118+0000 7f4694c04640 1 -- 172.21.15.67:0/1199300731 shutdown_connections 2024-07-31T14:57:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.118+0000 7f4694c04640 1 --2- 172.21.15.67:0/1199300731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4690109840 0x7f4690110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.118+0000 7f4694c04640 1 --2- 172.21.15.67:0/1199300731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4690108ef0 0x7f4690109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.118+0000 7f4694c04640 1 --2- 172.21.15.67:0/1199300731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46901035b0 0x7f46901039c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.118+0000 7f4694c04640 1 -- 172.21.15.67:0/1199300731 >> 172.21.15.67:0/1199300731 conn(0x7f46900ff040 msgr2=0x7f4690101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.118+0000 7f4694c04640 1 -- 172.21.15.67:0/1199300731 shutdown_connections 2024-07-31T14:57:27.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.118+0000 7f4694c04640 1 -- 172.21.15.67:0/1199300731 wait complete. 2024-07-31T14:57:27.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.119+0000 7f4694c04640 1 Processor -- start 2024-07-31T14:57:27.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.119+0000 7f4694c04640 1 -- start start 2024-07-31T14:57:27.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.119+0000 7f4694c04640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46901035b0 0x7f4690199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:27.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f4694c04640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4690108ef0 0x7f469019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:27.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f4694c04640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4690109840 0x7f469019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:27.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f4694c04640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f46901134f0 con 0x7f4690108ef0 2024-07-31T14:57:27.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f4694c04640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4690113370 con 0x7f4690109840 2024-07-31T14:57:27.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f4694c04640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4690113670 con 0x7f46901035b0 2024-07-31T14:57:27.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f468dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4690108ef0 0x7f469019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:27.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f468e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46901035b0 0x7f4690199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:27.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f468dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4690108ef0 0x7f469019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46328/0 (socket says 172.21.15.67:46328) 2024-07-31T14:57:27.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f468dd9b640 1 -- 172.21.15.67:0/3173910539 learned_addr learned my addr 172.21.15.67:0/3173910539 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:27.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f468ed9d640 1 --2- 172.21.15.67:0/3173910539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4690109840 0x7f469019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:27.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f468e59c640 1 -- 172.21.15.67:0/3173910539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4690109840 msgr2=0x7f469019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:27.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f468e59c640 1 --2- 172.21.15.67:0/3173910539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4690109840 0x7f469019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:27.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f468e59c640 1 -- 172.21.15.67:0/3173910539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4690108ef0 msgr2=0x7f469019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:27.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f468e59c640 1 --2- 172.21.15.67:0/3173910539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4690108ef0 0x7f469019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:27.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f468e59c640 1 -- 172.21.15.67:0/3173910539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469019fe50 con 0x7f46901035b0 2024-07-31T14:57:27.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f468dd9b640 1 --2- 172.21.15.67:0/3173910539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4690108ef0 0x7f469019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:57:27.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f468e59c640 1 --2- 172.21.15.67:0/3173910539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46901035b0 0x7f4690199ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4680035040 tx=0x7f4680032dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:27.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.120+0000 7f467f7fe640 1 -- 172.21.15.67:0/3173910539 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680040020 con 0x7f46901035b0 2024-07-31T14:57:27.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.121+0000 7f467f7fe640 1 -- 172.21.15.67:0/3173910539 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4680030450 con 0x7f46901035b0 2024-07-31T14:57:27.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.121+0000 7f4694c04640 1 -- 172.21.15.67:0/3173910539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46901a00e0 con 0x7f46901035b0 2024-07-31T14:57:27.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.121+0000 7f467f7fe640 1 -- 172.21.15.67:0/3173910539 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680037040 con 0x7f46901035b0 2024-07-31T14:57:27.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.121+0000 7f4694c04640 1 -- 172.21.15.67:0/3173910539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46901a68a0 con 0x7f46901035b0 2024-07-31T14:57:27.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.122+0000 7f4694c04640 1 -- 172.21.15.67:0/3173910539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4690104830 con 0x7f46901035b0 2024-07-31T14:57:27.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.123+0000 7f467f7fe640 1 -- 172.21.15.67:0/3173910539 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4680043020 con 0x7f46901035b0 2024-07-31T14:57:27.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.124+0000 7f467f7fe640 1 --2- 172.21.15.67:0/3173910539 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46780b3dc0 0x7f46780b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:27.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.124+0000 7f468dd9b640 1 --2- 172.21.15.67:0/3173910539 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46780b3dc0 0x7f46780b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:27.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.124+0000 7f467f7fe640 1 -- 172.21.15.67:0/3173910539 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f4680104590 con 0x7f46901035b0 2024-07-31T14:57:27.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.125+0000 7f468dd9b640 1 --2- 172.21.15.67:0/3173910539 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46780b3dc0 0x7f46780b62d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4690108af0 tx=0x7f468400b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:27.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.127+0000 7f467f7fe640 1 -- 172.21.15.67:0/3173910539 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f468004a020 con 0x7f46901035b0 2024-07-31T14:57:27.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: pgmap v591: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.2 MiB/s wr, 1.61k op/s 2024-07-31T14:57:27.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:57:27.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:57:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.379+0000 7f4694c04640 1 -- 172.21.15.67:0/3173910539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f469010b5a0 con 0x7f46901035b0 2024-07-31T14:57:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.380+0000 7f467f7fe640 1 -- 172.21.15.67:0/3173910539 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11984 (secure 0 0 0) 0x7f46800cb740 con 0x7f46901035b0 2024-07-31T14:57:27.382 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:57:27.382 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":44,"btime":"2024-07-31T14:56:45:102341+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:56:44.804223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:57:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 44 2024-07-31T14:57:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.382+0000 7f4694c04640 1 -- 172.21.15.67:0/3173910539 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46780b3dc0 msgr2=0x7f46780b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.382+0000 7f4694c04640 1 --2- 172.21.15.67:0/3173910539 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46780b3dc0 0x7f46780b62d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4690108af0 tx=0x7f468400b410 comp rx=0 tx=0).stop 2024-07-31T14:57:27.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.382+0000 7f4694c04640 1 -- 172.21.15.67:0/3173910539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46901035b0 msgr2=0x7f4690199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:27.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.382+0000 7f4694c04640 1 --2- 172.21.15.67:0/3173910539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46901035b0 0x7f4690199ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4680035040 tx=0x7f4680032dc0 comp rx=0 tx=0).stop 2024-07-31T14:57:27.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.382+0000 7f4694c04640 1 -- 172.21.15.67:0/3173910539 shutdown_connections 2024-07-31T14:57:27.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.382+0000 7f4694c04640 1 --2- 172.21.15.67:0/3173910539 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46780b3dc0 0x7f46780b62d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:27.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.382+0000 7f4694c04640 1 --2- 172.21.15.67:0/3173910539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4690109840 0x7f469019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:27.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.382+0000 7f4694c04640 1 --2- 172.21.15.67:0/3173910539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4690108ef0 0x7f469019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:27.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.382+0000 7f4694c04640 1 --2- 172.21.15.67:0/3173910539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46901035b0 0x7f4690199ee0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:27.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.382+0000 7f4694c04640 1 -- 172.21.15.67:0/3173910539 >> 172.21.15.67:0/3173910539 conn(0x7f46900ff040 msgr2=0x7f4690101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:27.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.382+0000 7f4694c04640 1 -- 172.21.15.67:0/3173910539 shutdown_connections 2024-07-31T14:57:27.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:27.382+0000 7f4694c04640 1 -- 172.21.15.67:0/3173910539 wait complete. 2024-07-31T14:57:27.962 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:27 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3173910539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:57:27.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:27 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3173910539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:57:28.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:27 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3173910539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:57:28.475 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id c8d56e19 --release --await' 2024-07-31T14:57:28.475 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id c8d56e19 --release --await 2024-07-31T14:57:28.750 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:57:28.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:28 smithi077 ceph-mon[37327]: pgmap v592: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 1.91k op/s 2024-07-31T14:57:29.008 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:28 smithi067 ceph-mon[32750]: pgmap v592: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 1.91k op/s 2024-07-31T14:57:29.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:28 smithi073 ceph-mon[36714]: pgmap v592: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 1.91k op/s 2024-07-31T14:57:30.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.593+0000 7f9363d2c640 1 -- 172.21.15.67:0/2146133229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f935c108720 msgr2=0x7f935c10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:30.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.593+0000 7f9363d2c640 1 --2- 172.21.15.67:0/2146133229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f935c108720 0x7f935c10ef50 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9354009ca0 tx=0x7f935402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.593+0000 7f9363d2c640 1 -- 172.21.15.67:0/2146133229 shutdown_connections 2024-07-31T14:57:30.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.593+0000 7f9363d2c640 1 --2- 172.21.15.67:0/2146133229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f935c108720 0x7f935c10ef50 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.593+0000 7f9363d2c640 1 --2- 172.21.15.67:0/2146133229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935c107dd0 0x7f935c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.593+0000 7f9363d2c640 1 --2- 172.21.15.67:0/2146133229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f935c102490 0x7f935c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.593+0000 7f9363d2c640 1 -- 172.21.15.67:0/2146133229 >> 172.21.15.67:0/2146133229 conn(0x7f935c079160 msgr2=0x7f935c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:30.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.593+0000 7f9363d2c640 1 -- 172.21.15.67:0/2146133229 shutdown_connections 2024-07-31T14:57:30.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.593+0000 7f9363d2c640 1 -- 172.21.15.67:0/2146133229 wait complete. 2024-07-31T14:57:30.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.594+0000 7f9363d2c640 1 Processor -- start 2024-07-31T14:57:30.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.594+0000 7f9363d2c640 1 -- start start 2024-07-31T14:57:30.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.594+0000 7f9363d2c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f935c102490 0x7f935c198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.594+0000 7f9363d2c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f935c107dd0 0x7f935c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.594+0000 7f9363d2c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935c108720 0x7f935c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.594+0000 7f9363d2c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1123f0 con 0x7f935c107dd0 2024-07-31T14:57:30.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.594+0000 7f9363d2c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f935c112270 con 0x7f935c108720 2024-07-31T14:57:30.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.594+0000 7f9363d2c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f935c112570 con 0x7f935c102490 2024-07-31T14:57:30.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.594+0000 7f93612c7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f935c107dd0 0x7f935c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.594+0000 7f93612c7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f935c107dd0 0x7f935c199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46334/0 (socket says 172.21.15.67:46334) 2024-07-31T14:57:30.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f93612c7640 1 -- 172.21.15.67:0/1418098312 learned_addr learned my addr 172.21.15.67:0/1418098312 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:30.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f93622c9640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935c108720 0x7f935c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f9361ac8640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f935c102490 0x7f935c198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f93612c7640 1 -- 172.21.15.67:0/1418098312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f935c102490 msgr2=0x7f935c198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:30.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f93612c7640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f935c102490 0x7f935c198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f93612c7640 1 -- 172.21.15.67:0/1418098312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935c108720 msgr2=0x7f935c19e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:30.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f93612c7640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935c108720 0x7f935c19e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f93612c7640 1 -- 172.21.15.67:0/1418098312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c19ecd0 con 0x7f935c107dd0 2024-07-31T14:57:30.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f9361ac8640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f935c102490 0x7f935c198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:57:30.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f93612c7640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f935c107dd0 0x7f935c199300 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f934c00c970 tx=0x7f934c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:30.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f9352ffd640 1 -- 172.21.15.67:0/1418098312 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c007bb0 con 0x7f935c107dd0 2024-07-31T14:57:30.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f9352ffd640 1 -- 172.21.15.67:0/1418098312 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f934c007d50 con 0x7f935c107dd0 2024-07-31T14:57:30.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f9363d2c640 1 -- 172.21.15.67:0/1418098312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935c19efc0 con 0x7f935c107dd0 2024-07-31T14:57:30.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f9352ffd640 1 -- 172.21.15.67:0/1418098312 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c00f450 con 0x7f935c107dd0 2024-07-31T14:57:30.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.595+0000 7f9363d2c640 1 -- 172.21.15.67:0/1418098312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935c06a890 con 0x7f935c107dd0 2024-07-31T14:57:30.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.598+0000 7f936a27d740 1 Processor -- start 2024-07-31T14:57:30.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.598+0000 7f936a27d740 1 -- start start 2024-07-31T14:57:30.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f9352ffd640 1 -- 172.21.15.67:0/1418098312 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f934c004f20 con 0x7f935c107dd0 2024-07-31T14:57:30.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f936a27d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 0x561e4792f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f936a27d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47560660 0x561e4792f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f936a27d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47568840 0x561e4792faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f936a27d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561e478a4e70 con 0x561e47560660 2024-07-31T14:57:30.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f936a27d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561e47894370 con 0x561e47589520 2024-07-31T14:57:30.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f936a27d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561e478983b0 con 0x561e47568840 2024-07-31T14:57:30.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f9361ac8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47560660 0x561e4792f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f9361ac8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47560660 0x561e4792f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46346/0 (socket says 172.21.15.67:46346) 2024-07-31T14:57:30.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f9361ac8640 1 -- 172.21.15.67:0/493755665 learned_addr learned my addr 172.21.15.67:0/493755665 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:30.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f93622c9640 1 --2- 172.21.15.67:0/493755665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 0x561e4792f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f93612c7640 1 --2- 172.21.15.67:0/493755665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47568840 0x561e4792faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f9361ac8640 1 -- 172.21.15.67:0/493755665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47568840 msgr2=0x561e4792faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:30.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f9361ac8640 1 --2- 172.21.15.67:0/493755665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47568840 0x561e4792faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f9361ac8640 1 -- 172.21.15.67:0/493755665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 msgr2=0x561e4792f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:30.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f9361ac8640 1 --2- 172.21.15.67:0/493755665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 0x561e4792f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f9361ac8640 1 -- 172.21.15.67:0/493755665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e47861040 con 0x561e47560660 2024-07-31T14:57:30.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.599+0000 7f93612c7640 1 --2- 172.21.15.67:0/493755665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47568840 0x561e4792faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:57:30.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.600+0000 7f9361ac8640 1 --2- 172.21.15.67:0/493755665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47560660 0x561e4792f5b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f935800cca0 tx=0x7f935800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cd82c2375314ce server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:30.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.600+0000 7f9347fff640 1 -- 172.21.15.67:0/493755665 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358005220 con 0x561e47560660 2024-07-31T14:57:30.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.600+0000 7f9347fff640 1 -- 172.21.15.67:0/493755665 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9358004c50 con 0x561e47560660 2024-07-31T14:57:30.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.600+0000 7f9347fff640 1 -- 172.21.15.67:0/493755665 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358010850 con 0x561e47560660 2024-07-31T14:57:30.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.600+0000 7f9352ffd640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93480b3dc0 0x7f93480b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.600+0000 7f93622c9640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93480b3dc0 0x7f93480b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.600+0000 7f9352ffd640 1 -- 172.21.15.67:0/1418098312 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f934c0e4320 con 0x7f935c107dd0 2024-07-31T14:57:30.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.601+0000 7f936a27d740 1 -- 172.21.15.67:0/493755665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47560660 msgr2=0x561e4792f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:30.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.601+0000 7f93622c9640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93480b3dc0 0x7f93480b62d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9354009430 tx=0x7f9354009800 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:30.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.601+0000 7f936a27d740 1 --2- 172.21.15.67:0/493755665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47560660 0x561e4792f5b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f935800cca0 tx=0x7f935800e720 comp rx=0 tx=0).stop 2024-07-31T14:57:30.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.601+0000 7f936a27d740 1 -- 172.21.15.67:0/493755665 shutdown_connections 2024-07-31T14:57:30.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.601+0000 7f936a27d740 1 --2- 172.21.15.67:0/493755665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47568840 0x561e4792faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.601+0000 7f936a27d740 1 --2- 172.21.15.67:0/493755665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47560660 0x561e4792f5b0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.601+0000 7f936a27d740 1 --2- 172.21.15.67:0/493755665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 0x561e4792f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.601+0000 7f936a27d740 1 -- 172.21.15.67:0/493755665 >> 172.21.15.67:0/493755665 conn(0x561e4757d910 msgr2=0x561e4792cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:30.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.601+0000 7f936a27d740 1 -- 172.21.15.67:0/493755665 shutdown_connections 2024-07-31T14:57:30.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.601+0000 7f936a27d740 1 -- 172.21.15.67:0/493755665 wait complete. 2024-07-31T14:57:30.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.601+0000 7f936a27d740 1 -- start start 2024-07-31T14:57:30.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.602+0000 7f936a27d740 1 Processor -- start 2024-07-31T14:57:30.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.602+0000 7f936a27d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47560660 0x561e4793ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.602+0000 7f936a27d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4793b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.602+0000 7f936a27d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 0x561e47946b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.602+0000 7f936a27d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561e476f7bb0 con 0x561e47568840 2024-07-31T14:57:30.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.602+0000 7f936a27d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561e47836ac0 con 0x561e47589520 2024-07-31T14:57:30.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.602+0000 7f936a27d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561e4793d850 con 0x561e47560660 2024-07-31T14:57:30.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.602+0000 7f936a27d740 10 client.0 _ll_register_callbacks cb 0x561e4783f3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:30.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.602+0000 7f93612c7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4793b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.602+0000 7f93612c7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4793b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46358/0 (socket says 172.21.15.67:46358) 2024-07-31T14:57:30.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.602+0000 7f93612c7640 1 -- 172.21.15.67:0/1630956674 learned_addr learned my addr 172.21.15.67:0/1630956674 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:30.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.602+0000 7f93622c9640 1 --2- 172.21.15.67:0/1630956674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 0x561e47946b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.602+0000 7f9361ac8640 1 --2- 172.21.15.67:0/1630956674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47560660 0x561e4793ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.603+0000 7f93612c7640 1 -- 172.21.15.67:0/1630956674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47560660 msgr2=0x561e4793ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:30.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.603+0000 7f93612c7640 1 --2- 172.21.15.67:0/1630956674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47560660 0x561e4793ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.603+0000 7f93612c7640 1 -- 172.21.15.67:0/1630956674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 msgr2=0x561e47946b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:30.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.603+0000 7f93612c7640 1 --2- 172.21.15.67:0/1630956674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 0x561e47946b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.603+0000 7f93612c7640 1 -- 172.21.15.67:0/1630956674 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e475f4b00 con 0x561e47568840 2024-07-31T14:57:30.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.603+0000 7f93612c7640 1 --2- 172.21.15.67:0/1630956674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4793b190 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f934c00afd0 tx=0x7f934c0aa570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90d8addebfc892e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.603+0000 7f93457fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.603+0000 7f93457fa640 1 -- 172.21.15.67:0/1630956674 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c01d040 con 0x561e47568840 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.603+0000 7f93457fa640 1 -- 172.21.15.67:0/1630956674 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f934c0a6eb0 con 0x561e47568840 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.603+0000 7f936a27d740 1 -- 172.21.15.67:0/1630956674 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9358004c50 con 0x561e47568840 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.603+0000 7f93457fa640 1 -- 172.21.15.67:0/1630956674 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f934c0e4170 con 0x561e47568840 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.603+0000 7f936a27d740 10 client.17205 fetch_fsmap learned FSMap version 44 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.603+0000 7f936a27d740 1 -- 172.21.15.67:0/1630956674 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561e4766b960 con 0x561e47568840 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.603+0000 7f93457fa640 1 -- 172.21.15.67:0/1630956674 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9358004c50 con 0x561e47568840 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.604+0000 7f93457fa640 1 -- 172.21.15.67:0/1630956674 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7f934c005210 con 0x561e47568840 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.604+0000 7f936a27d740 10 client.17205 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.604+0000 7f936a27d740 10 client.17205 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.604+0000 7f936a27d740 20 client.17205 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.604+0000 7f936a27d740 1 --2- 172.21.15.67:0/1630956674 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e47547ea0 0x561e4794cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.604+0000 7f936a27d740 4 client.17205 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:57:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.604+0000 7f936a27d740 1 -- 172.21.15.67:0/1630956674 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e478a6dc0 con 0x561e47547ea0 2024-07-31T14:57:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.604+0000 7f9361ac8640 1 --2- 172.21.15.67:0/1630956674 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e47547ea0 0x561e4794cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.604+0000 7f93457fa640 10 client.17205 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.604+0000 7f9361ac8640 1 --2- 172.21.15.67:0/1630956674 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e47547ea0 0x561e4794cbf0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2e661f0fab956f3 server_cookie=676f8accfa46b89a in_seq=0 out_seq=0 2024-07-31T14:57:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.605+0000 7f93457fa640 1 -- 172.21.15.67:0/1630956674 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e478a6dc0 con 0x561e47547ea0 2024-07-31T14:57:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.605+0000 7f93457fa640 10 client.17205 handle_command_reply: tid=0 2024-07-31T14:57:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.605+0000 7f936a27d740 1 client.17205 shutdown 2024-07-31T14:57:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.605+0000 7f936a27d740 1 -- 172.21.15.67:0/1630956674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 msgr2=0x561e4793b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.605+0000 7f936a27d740 1 --2- 172.21.15.67:0/1630956674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4793b190 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f934c00afd0 tx=0x7f934c0aa570 comp rx=0 tx=0).stop 2024-07-31T14:57:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.606+0000 7f936a27d740 1 -- 172.21.15.67:0/1630956674 shutdown_connections 2024-07-31T14:57:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.606+0000 7f936a27d740 1 --2- 172.21.15.67:0/1630956674 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e47547ea0 0x561e4794cbf0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.606+0000 7f936a27d740 1 --2- 172.21.15.67:0/1630956674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 0x561e47946b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.606+0000 7f936a27d740 1 --2- 172.21.15.67:0/1630956674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4793b190 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.606+0000 7f93457fa640 0 client.17205 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.606+0000 7f936a27d740 1 --2- 172.21.15.67:0/1630956674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47560660 0x561e4793ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.606+0000 7f936a27d740 1 -- 172.21.15.67:0/1630956674 >> 172.21.15.67:0/1630956674 conn(0x561e4757d910 msgr2=0x561e47932200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:30.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.606+0000 7f936a27d740 1 -- 172.21.15.67:0/1630956674 shutdown_connections 2024-07-31T14:57:30.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.606+0000 7f936a27d740 1 -- 172.21.15.67:0/1630956674 wait complete. 2024-07-31T14:57:30.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.606+0000 7f936a27d740 20 client.17205 trim_cache size 0 max 16384 2024-07-31T14:57:30.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f936a27d740 1 Processor -- start 2024-07-31T14:57:30.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f936a27d740 1 -- start start 2024-07-31T14:57:30.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f936a27d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47560660 0x561e47946960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f936a27d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4792bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f936a27d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 0x561e479523a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f936a27d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1b28a0 con 0x561e47568840 2024-07-31T14:57:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f936a27d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1b45f0 con 0x561e47589520 2024-07-31T14:57:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f936a27d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1b3000 con 0x561e47560660 2024-07-31T14:57:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f93612c7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4792bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f93612c7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4792bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46370/0 (socket says 172.21.15.67:46370) 2024-07-31T14:57:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f93612c7640 1 -- 172.21.15.67:0/1262412336 learned_addr learned my addr 172.21.15.67:0/1262412336 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f9361ac8640 1 --2- 172.21.15.67:0/1262412336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47560660 0x561e47946960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f93622c9640 1 --2- 172.21.15.67:0/1262412336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 0x561e479523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f93612c7640 1 -- 172.21.15.67:0/1262412336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47560660 msgr2=0x561e47946960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f93612c7640 1 --2- 172.21.15.67:0/1262412336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47560660 0x561e47946960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f93612c7640 1 -- 172.21.15.67:0/1262412336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 msgr2=0x561e479523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f93612c7640 1 --2- 172.21.15.67:0/1262412336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 0x561e479523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f93612c7640 1 -- 172.21.15.67:0/1262412336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e4792d4a0 con 0x561e47568840 2024-07-31T14:57:30.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f93622c9640 1 --2- 172.21.15.67:0/1262412336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 0x561e479523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:57:30.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.624+0000 7f93612c7640 1 --2- 172.21.15.67:0/1262412336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4792bc80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f934c0aabb0 tx=0x7f934c0b3cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca9d3d8f0831d08 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:30.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.625+0000 7f93477fe640 1 -- 172.21.15.67:0/1262412336 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c0aa390 con 0x561e47568840 2024-07-31T14:57:30.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.625+0000 7f93477fe640 1 -- 172.21.15.67:0/1262412336 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f934c0e4170 con 0x561e47568840 2024-07-31T14:57:30.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.625+0000 7f93477fe640 1 -- 172.21.15.67:0/1262412336 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c0a6eb0 con 0x561e47568840 2024-07-31T14:57:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.625+0000 7f936a27d740 1 -- 172.21.15.67:0/1262412336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 msgr2=0x561e4792bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.625+0000 7f936a27d740 1 --2- 172.21.15.67:0/1262412336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4792bc80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f934c0aabb0 tx=0x7f934c0b3cd0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.625+0000 7f936a27d740 1 -- 172.21.15.67:0/1262412336 shutdown_connections 2024-07-31T14:57:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.625+0000 7f936a27d740 1 --2- 172.21.15.67:0/1262412336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47589520 0x561e479523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.625+0000 7f936a27d740 1 --2- 172.21.15.67:0/1262412336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4792bc80 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.625+0000 7f936a27d740 1 --2- 172.21.15.67:0/1262412336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47560660 0x561e47946960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.625+0000 7f936a27d740 1 -- 172.21.15.67:0/1262412336 >> 172.21.15.67:0/1262412336 conn(0x561e4757d910 msgr2=0x561e4792b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.626+0000 7f936a27d740 1 -- 172.21.15.67:0/1262412336 shutdown_connections 2024-07-31T14:57:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.626+0000 7f936a27d740 1 -- 172.21.15.67:0/1262412336 wait complete. 2024-07-31T14:57:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.626+0000 7f936a27d740 1 -- start start 2024-07-31T14:57:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.626+0000 7f936a27d740 1 Processor -- start 2024-07-31T14:57:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.626+0000 7f936a27d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47560660 0x561e4793ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.626+0000 7f936a27d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4793f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.626+0000 7f936a27d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47589520 0x561e479b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.626+0000 7f936a27d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1b3f20 con 0x561e47568840 2024-07-31T14:57:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.627+0000 7f936a27d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1b3790 con 0x561e47560660 2024-07-31T14:57:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.627+0000 7f936a27d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1b0880 con 0x561e47589520 2024-07-31T14:57:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.627+0000 7f936a27d740 10 client.0 _ll_register_callbacks cb 0x7f934c007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.627+0000 7f93612c7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4793f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.627+0000 7f93612c7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4793f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46386/0 (socket says 172.21.15.67:46386) 2024-07-31T14:57:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.627+0000 7f93612c7640 1 -- 172.21.15.67:0/2331773502 learned_addr learned my addr 172.21.15.67:0/2331773502 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.627+0000 7f9361ac8640 1 --2- 172.21.15.67:0/2331773502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47560660 0x561e4793ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.627+0000 7f93622c9640 1 --2- 172.21.15.67:0/2331773502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47589520 0x561e479b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.627+0000 7f93612c7640 1 -- 172.21.15.67:0/2331773502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47589520 msgr2=0x561e479b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.627+0000 7f93612c7640 1 --2- 172.21.15.67:0/2331773502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47589520 0x561e479b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.627+0000 7f93612c7640 1 -- 172.21.15.67:0/2331773502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47560660 msgr2=0x561e4793ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.627+0000 7f93612c7640 1 --2- 172.21.15.67:0/2331773502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47560660 0x561e4793ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.627+0000 7f93612c7640 1 -- 172.21.15.67:0/2331773502 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e4793f990 con 0x561e47568840 2024-07-31T14:57:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.628+0000 7f93612c7640 1 --2- 172.21.15.67:0/2331773502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4793f170 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f934c0058c0 tx=0x7f934c0048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1981bf2b4454e0e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.628+0000 7f93457fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:57:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.628+0000 7f93457fa640 1 -- 172.21.15.67:0/2331773502 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c084e60 con 0x561e47568840 2024-07-31T14:57:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.628+0000 7f93457fa640 1 -- 172.21.15.67:0/2331773502 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f934c0922c0 con 0x561e47568840 2024-07-31T14:57:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.628+0000 7f936a27d740 1 -- 172.21.15.67:0/2331773502 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f934c0e4170 con 0x561e47568840 2024-07-31T14:57:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.628+0000 7f93457fa640 1 -- 172.21.15.67:0/2331773502 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f934c0aa390 con 0x561e47568840 2024-07-31T14:57:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.628+0000 7f936a27d740 10 client.17211 fetch_fsmap learned FSMap version 44 2024-07-31T14:57:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.628+0000 7f936a27d740 1 -- 172.21.15.67:0/2331773502 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561e479b5830 con 0x561e47568840 2024-07-31T14:57:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.628+0000 7f93457fa640 1 -- 172.21.15.67:0/2331773502 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f934c0e4170 con 0x561e47568840 2024-07-31T14:57:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.628+0000 7f93457fa640 1 -- 172.21.15.67:0/2331773502 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 44) ==== 8430+0+0 (secure 0 0 0) 0x7f934c0a6590 con 0x561e47568840 2024-07-31T14:57:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.629+0000 7f936a27d740 10 client.17211 fetch_fsmap finished waiting for FSMap version 44 2024-07-31T14:57:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.629+0000 7f936a27d740 10 client.17211 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:57:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.629+0000 7f936a27d740 20 client.17211 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.629+0000 7f936a27d740 1 --2- 172.21.15.67:0/2331773502 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e4751d250 0x561e479b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.629+0000 7f936a27d740 4 client.17211 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c8d56e19", "release": true, "await": true}] 2024-07-31T14:57:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.629+0000 7f936a27d740 1 -- 172.21.15.67:0/2331773502 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c8d56e19", "release": true, "await": true}) -- 0x561e478a2860 con 0x561e4751d250 2024-07-31T14:57:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.629+0000 7f9361ac8640 1 --2- 172.21.15.67:0/2331773502 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e4751d250 0x561e479b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.629+0000 7f93457fa640 10 client.17211 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:30.629+0000 7f9361ac8640 1 --2- 172.21.15.67:0/2331773502 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e4751d250 0x561e479b7d10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ed0480e1d15b7f5 server_cookie=29b1c09a02f12895 in_seq=0 out_seq=0 2024-07-31T14:57:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:30 smithi067 ceph-mon[32750]: pgmap v593: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 2.35k op/s 2024-07-31T14:57:30.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:30 smithi077 ceph-mon[37327]: pgmap v593: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 2.35k op/s 2024-07-31T14:57:31.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:30 smithi073 ceph-mon[36714]: pgmap v593: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 2.35k op/s 2024-07-31T14:57:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.581+0000 7f40155d1640 1 -- 172.21.15.67:0/4212200463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4010108670 msgr2=0x7f401010eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.581+0000 7f40155d1640 1 --2- 172.21.15.67:0/4212200463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4010108670 0x7f401010eea0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4000009ca0 tx=0x7f400002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.581+0000 7f40155d1640 1 -- 172.21.15.67:0/4212200463 shutdown_connections 2024-07-31T14:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.581+0000 7f40155d1640 1 --2- 172.21.15.67:0/4212200463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4010108670 0x7f401010eea0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.581+0000 7f40155d1640 1 --2- 172.21.15.67:0/4212200463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4010107d20 0x7f4010108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.581+0000 7f40155d1640 1 --2- 172.21.15.67:0/4212200463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40101023f0 0x7f4010102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.581+0000 7f40155d1640 1 -- 172.21.15.67:0/4212200463 >> 172.21.15.67:0/4212200463 conn(0x7f40100fddf0 msgr2=0x7f4010100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.581+0000 7f40155d1640 1 -- 172.21.15.67:0/4212200463 shutdown_connections 2024-07-31T14:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.581+0000 7f40155d1640 1 -- 172.21.15.67:0/4212200463 wait complete. 2024-07-31T14:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.582+0000 7f40155d1640 1 Processor -- start 2024-07-31T14:57:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.582+0000 7f40155d1640 1 -- start start 2024-07-31T14:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.582+0000 7f40155d1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40101023f0 0x7f4010199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.582+0000 7f40155d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4010107d20 0x7f401019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.582+0000 7f40155d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4010108670 0x7f401019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.582+0000 7f40155d1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4010113450 con 0x7f4010107d20 2024-07-31T14:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.582+0000 7f40155d1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40101132d0 con 0x7f40101023f0 2024-07-31T14:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.582+0000 7f40155d1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40101135d0 con 0x7f4010108670 2024-07-31T14:57:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f400e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4010107d20 0x7f401019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f400effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40101023f0 0x7f4010199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f400e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4010107d20 0x7f401019a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60266/0 (socket says 172.21.15.67:60266) 2024-07-31T14:57:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f400e7fc640 1 -- 172.21.15.67:0/1944542113 learned_addr learned my addr 172.21.15.67:0/1944542113 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f400effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40101023f0 0x7f4010199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46062/0 (socket says 172.21.15.67:46062) 2024-07-31T14:57:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f400f7fe640 1 --2- 172.21.15.67:0/1944542113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4010108670 0x7f401019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f400effd640 1 -- 172.21.15.67:0/1944542113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4010108670 msgr2=0x7f401019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f400effd640 1 --2- 172.21.15.67:0/1944542113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4010108670 0x7f401019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f400effd640 1 -- 172.21.15.67:0/1944542113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4010107d20 msgr2=0x7f401019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f400effd640 1 --2- 172.21.15.67:0/1944542113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4010107d20 0x7f401019a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f400effd640 1 -- 172.21.15.67:0/1944542113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401019fd60 con 0x7f40101023f0 2024-07-31T14:57:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f400e7fc640 1 --2- 172.21.15.67:0/1944542113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4010107d20 0x7f401019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:57:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f400effd640 1 --2- 172.21.15.67:0/1944542113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40101023f0 0x7f4010199e30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f400800e9f0 tx=0x7f400800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f4007fff640 1 -- 172.21.15.67:0/1944542113 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400800ccc0 con 0x7f40101023f0 2024-07-31T14:57:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f40155d1640 1 -- 172.21.15.67:0/1944542113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40101a0050 con 0x7f40101023f0 2024-07-31T14:57:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f4007fff640 1 -- 172.21.15.67:0/1944542113 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4008005020 con 0x7f40101023f0 2024-07-31T14:57:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f4007fff640 1 -- 172.21.15.67:0/1944542113 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008004a00 con 0x7f40101023f0 2024-07-31T14:57:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.583+0000 7f40155d1640 1 -- 172.21.15.67:0/1944542113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40101a67e0 con 0x7f40101023f0 2024-07-31T14:57:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f401c5a5740 1 Processor -- start 2024-07-31T14:57:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f401c5a5740 1 -- start start 2024-07-31T14:57:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f401c5a5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722831b0 0x559572625ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f401c5a5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55957225a2f0 0x559572626220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f401c5a5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595722624d0 0x559572626760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f401c5a5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55957259b3b0 con 0x5595722624d0 2024-07-31T14:57:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f401c5a5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55957258a8b0 con 0x5595722831b0 2024-07-31T14:57:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f401c5a5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55957258e8f0 con 0x55957225a2f0 2024-07-31T14:57:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f400e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595722624d0 0x559572626760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f400f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722831b0 0x559572625ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f400e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595722624d0 0x559572626760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60272/0 (socket says 172.21.15.67:60272) 2024-07-31T14:57:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f400e7fc640 1 -- 172.21.15.67:0/1279169917 learned_addr learned my addr 172.21.15.67:0/1279169917 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f400e7fc640 1 -- 172.21.15.67:0/1279169917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55957225a2f0 msgr2=0x559572626220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:57:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f400e7fc640 1 --2- 172.21.15.67:0/1279169917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55957225a2f0 0x559572626220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f400e7fc640 1 -- 172.21.15.67:0/1279169917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722831b0 msgr2=0x559572625ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f400e7fc640 1 --2- 172.21.15.67:0/1279169917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722831b0 0x559572625ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f400e7fc640 1 -- 172.21.15.67:0/1279169917 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595725319a0 con 0x5595722624d0 2024-07-31T14:57:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.587+0000 7f4007fff640 1 -- 172.21.15.67:0/1944542113 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4008004ba0 con 0x7f40101023f0 2024-07-31T14:57:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.588+0000 7f400e7fc640 1 --2- 172.21.15.67:0/1279169917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595722624d0 0x559572626760 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8011540 tx=0x7f3ff8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccc063ffa2f5d59 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.588+0000 7f40057fa640 1 -- 172.21.15.67:0/1279169917 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff801e070 con 0x5595722624d0 2024-07-31T14:57:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.588+0000 7f40057fa640 1 -- 172.21.15.67:0/1279169917 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff800fa70 con 0x5595722624d0 2024-07-31T14:57:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.588+0000 7f40057fa640 1 -- 172.21.15.67:0/1279169917 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8019b20 con 0x5595722624d0 2024-07-31T14:57:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.588+0000 7f4007fff640 1 --2- 172.21.15.67:0/1944542113 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3ffc0b3d40 0x7f3ffc0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.589+0000 7f400f7fe640 1 --2- 172.21.15.67:0/1944542113 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3ffc0b3d40 0x7f3ffc0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.589+0000 7f401c5a5740 1 -- 172.21.15.67:0/1279169917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595722624d0 msgr2=0x559572626760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.589+0000 7f401c5a5740 1 --2- 172.21.15.67:0/1279169917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595722624d0 0x559572626760 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8011540 tx=0x7f3ff8011940 comp rx=0 tx=0).stop 2024-07-31T14:57:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.589+0000 7f4007fff640 1 -- 172.21.15.67:0/1944542113 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f40080e4300 con 0x7f40101023f0 2024-07-31T14:57:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.589+0000 7f401c5a5740 1 -- 172.21.15.67:0/1279169917 shutdown_connections 2024-07-31T14:57:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.589+0000 7f401c5a5740 1 --2- 172.21.15.67:0/1279169917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595722624d0 0x559572626760 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.589+0000 7f401c5a5740 1 --2- 172.21.15.67:0/1279169917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55957225a2f0 0x559572626220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.589+0000 7f401c5a5740 1 --2- 172.21.15.67:0/1279169917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722831b0 0x559572625ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.589+0000 7f401c5a5740 1 -- 172.21.15.67:0/1279169917 >> 172.21.15.67:0/1279169917 conn(0x559572277300 msgr2=0x559572623bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.589+0000 7f401c5a5740 1 -- 172.21.15.67:0/1279169917 shutdown_connections 2024-07-31T14:57:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.589+0000 7f401c5a5740 1 -- 172.21.15.67:0/1279169917 wait complete. 2024-07-31T14:57:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.589+0000 7f400f7fe640 1 --2- 172.21.15.67:0/1944542113 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3ffc0b3d40 0x7f3ffc0b6250 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f400002f500 tx=0x7f400002f310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.589+0000 7f401c5a5740 1 -- start start 2024-07-31T14:57:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.589+0000 7f401c5a5740 1 Processor -- start 2024-07-31T14:57:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f401c5a5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55957225a2f0 0x559572631860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f401c5a5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722624d0 0x559572631da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f401c5a5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722831b0 0x55957263d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f401c5a5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5595724c4ab0 con 0x55957225a2f0 2024-07-31T14:57:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f401c5a5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559572512cf0 con 0x5595722831b0 2024-07-31T14:57:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f401c5a5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559572634460 con 0x5595722624d0 2024-07-31T14:57:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f401c5a5740 10 client.0 _ll_register_callbacks cb 0x5595725849f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f400effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55957225a2f0 0x559572631860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f400effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55957225a2f0 0x559572631860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60288/0 (socket says 172.21.15.67:60288) 2024-07-31T14:57:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f400effd640 1 -- 172.21.15.67:0/1644474832 learned_addr learned my addr 172.21.15.67:0/1644474832 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f400f7fe640 1 --2- 172.21.15.67:0/1644474832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722831b0 0x55957263d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f400e7fc640 1 --2- 172.21.15.67:0/1644474832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722624d0 0x559572631da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f400effd640 1 -- 172.21.15.67:0/1644474832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722624d0 msgr2=0x559572631da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f400effd640 1 --2- 172.21.15.67:0/1644474832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722624d0 0x559572631da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f400effd640 1 -- 172.21.15.67:0/1644474832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722831b0 msgr2=0x55957263d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f400effd640 1 --2- 172.21.15.67:0/1644474832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722831b0 0x55957263d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f400effd640 1 -- 172.21.15.67:0/1644474832 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595722ee790 con 0x55957225a2f0 2024-07-31T14:57:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f400effd640 1 --2- 172.21.15.67:0/1644474832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55957225a2f0 0x559572631860 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f40080e1d40 tx=0x7f40080aa7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81f3b50f4650697 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f3ff67fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:57:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f3ff67fc640 1 -- 172.21.15.67:0/1644474832 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400801c040 con 0x55957225a2f0 2024-07-31T14:57:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f3ff67fc640 1 -- 172.21.15.67:0/1644474832 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f40080920d0 con 0x55957225a2f0 2024-07-31T14:57:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.590+0000 7f401c5a5740 1 -- 172.21.15.67:0/1644474832 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff800fa70 con 0x55957225a2f0 2024-07-31T14:57:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.591+0000 7f3ff67fc640 1 -- 172.21.15.67:0/1644474832 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f40080af200 con 0x55957225a2f0 2024-07-31T14:57:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.591+0000 7f401c5a5740 10 client.17223 fetch_fsmap learned FSMap version 45 2024-07-31T14:57:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.591+0000 7f401c5a5740 1 -- 172.21.15.67:0/1644474832 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559572367620 con 0x55957225a2f0 2024-07-31T14:57:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.591+0000 7f3ff67fc640 1 -- 172.21.15.67:0/1644474832 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff800fa70 con 0x55957225a2f0 2024-07-31T14:57:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.591+0000 7f3ff67fc640 1 -- 172.21.15.67:0/1644474832 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7f40080c2030 con 0x55957225a2f0 2024-07-31T14:57:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.591+0000 7f401c5a5740 10 client.17223 fetch_fsmap finished waiting for FSMap version 45 2024-07-31T14:57:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.591+0000 7f401c5a5740 10 client.17223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:57:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.591+0000 7f401c5a5740 20 client.17223 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.591+0000 7f401c5a5740 1 --2- 172.21.15.67:0/1644474832 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559572241d80 0x5595726437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.591+0000 7f401c5a5740 4 client.17223 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:57:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.591+0000 7f401c5a5740 1 -- 172.21.15.67:0/1644474832 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55957259d300 con 0x559572241d80 2024-07-31T14:57:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.591+0000 7f400e7fc640 1 --2- 172.21.15.67:0/1644474832 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559572241d80 0x5595726437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.592+0000 7f3ff67fc640 10 client.17223 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.592+0000 7f400e7fc640 1 --2- 172.21.15.67:0/1644474832 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559572241d80 0x5595726437f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266aeefa5333acf8 server_cookie=9ee766f65896554b in_seq=0 out_seq=0 2024-07-31T14:57:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f3ff67fc640 1 -- 172.21.15.67:0/1644474832 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55957259d300 con 0x559572241d80 2024-07-31T14:57:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f3ff67fc640 10 client.17223 handle_command_reply: tid=0 2024-07-31T14:57:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f401c5a5740 1 client.17223 shutdown 2024-07-31T14:57:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f401c5a5740 1 -- 172.21.15.67:0/1644474832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55957225a2f0 msgr2=0x559572631860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f401c5a5740 1 --2- 172.21.15.67:0/1644474832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55957225a2f0 0x559572631860 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f40080e1d40 tx=0x7f40080aa7d0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f401c5a5740 1 -- 172.21.15.67:0/1644474832 shutdown_connections 2024-07-31T14:57:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f401c5a5740 1 --2- 172.21.15.67:0/1644474832 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559572241d80 0x5595726437f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f401c5a5740 1 --2- 172.21.15.67:0/1644474832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722831b0 0x55957263d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f401c5a5740 1 --2- 172.21.15.67:0/1644474832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722624d0 0x559572631da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f3ff67fc640 0 client.17223 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f401c5a5740 1 --2- 172.21.15.67:0/1644474832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55957225a2f0 0x559572631860 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f401c5a5740 1 -- 172.21.15.67:0/1644474832 >> 172.21.15.67:0/1644474832 conn(0x559572277300 msgr2=0x559572626d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f401c5a5740 1 -- 172.21.15.67:0/1644474832 shutdown_connections 2024-07-31T14:57:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f401c5a5740 1 -- 172.21.15.67:0/1644474832 wait complete. 2024-07-31T14:57:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.950+0000 7f401c5a5740 20 client.17223 trim_cache size 0 max 16384 2024-07-31T14:57:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.966+0000 7f401c5a5740 1 Processor -- start 2024-07-31T14:57:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.966+0000 7f401c5a5740 1 -- start start 2024-07-31T14:57:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.966+0000 7f401c5a5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55957225a2f0 0x559572633530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.966+0000 7f401c5a5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722624d0 0x55957262bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.966+0000 7f401c5a5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722831b0 0x559572648ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.966+0000 7f401c5a5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f40101b0560 con 0x55957225a2f0 2024-07-31T14:57:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.966+0000 7f401c5a5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40101b22b0 con 0x5595722624d0 2024-07-31T14:57:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.966+0000 7f401c5a5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40101b0cc0 con 0x5595722831b0 2024-07-31T14:57:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.966+0000 7f400effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55957225a2f0 0x559572633530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.966+0000 7f400effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55957225a2f0 0x559572633530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60296/0 (socket says 172.21.15.67:60296) 2024-07-31T14:57:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.966+0000 7f400effd640 1 -- 172.21.15.67:0/3787672588 learned_addr learned my addr 172.21.15.67:0/3787672588 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.966+0000 7f400f7fe640 1 --2- 172.21.15.67:0/3787672588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722831b0 0x559572648ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.966+0000 7f400e7fc640 1 --2- 172.21.15.67:0/3787672588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722624d0 0x55957262bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.967+0000 7f400effd640 1 -- 172.21.15.67:0/3787672588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722831b0 msgr2=0x559572648ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.967+0000 7f400effd640 1 --2- 172.21.15.67:0/3787672588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722831b0 0x559572648ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.967+0000 7f400effd640 1 -- 172.21.15.67:0/3787672588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722624d0 msgr2=0x55957262bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.967+0000 7f400effd640 1 --2- 172.21.15.67:0/3787672588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722624d0 0x55957262bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.967+0000 7f400effd640 1 -- 172.21.15.67:0/3787672588 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559572627fe0 con 0x55957225a2f0 2024-07-31T14:57:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.967+0000 7f400effd640 1 --2- 172.21.15.67:0/3787672588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55957225a2f0 0x559572633530 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f40080e17f0 tx=0x7f4008005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b899ed60236932 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.967+0000 7f4004ff9640 1 -- 172.21.15.67:0/3787672588 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40080ca070 con 0x55957225a2f0 2024-07-31T14:57:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.967+0000 7f4004ff9640 1 -- 172.21.15.67:0/3787672588 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40080b3cb0 con 0x55957225a2f0 2024-07-31T14:57:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.967+0000 7f4004ff9640 1 -- 172.21.15.67:0/3787672588 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400801c070 con 0x55957225a2f0 2024-07-31T14:57:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.968+0000 7f401c5a5740 1 -- 172.21.15.67:0/3787672588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55957225a2f0 msgr2=0x559572633530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.968+0000 7f401c5a5740 1 --2- 172.21.15.67:0/3787672588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55957225a2f0 0x559572633530 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f40080e17f0 tx=0x7f4008005460 comp rx=0 tx=0).stop 2024-07-31T14:57:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.968+0000 7f401c5a5740 1 -- 172.21.15.67:0/3787672588 shutdown_connections 2024-07-31T14:57:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.968+0000 7f401c5a5740 1 --2- 172.21.15.67:0/3787672588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722831b0 0x559572648ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.968+0000 7f401c5a5740 1 --2- 172.21.15.67:0/3787672588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595722624d0 0x55957262bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.968+0000 7f401c5a5740 1 --2- 172.21.15.67:0/3787672588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55957225a2f0 0x559572633530 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.968+0000 7f401c5a5740 1 -- 172.21.15.67:0/3787672588 >> 172.21.15.67:0/3787672588 conn(0x559572277300 msgr2=0x55957262feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.968+0000 7f401c5a5740 1 -- 172.21.15.67:0/3787672588 shutdown_connections 2024-07-31T14:57:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.968+0000 7f401c5a5740 1 -- 172.21.15.67:0/3787672588 wait complete. 2024-07-31T14:57:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f401c5a5740 1 -- start start 2024-07-31T14:57:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f401c5a5740 1 Processor -- start 2024-07-31T14:57:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f401c5a5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55957225a2f0 0x559572639fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f401c5a5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722624d0 0x5595726364c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f401c5a5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595722831b0 0x559572636bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f401c5a5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f40101b1be0 con 0x5595722831b0 2024-07-31T14:57:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f401c5a5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40101b1450 con 0x55957225a2f0 2024-07-31T14:57:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f401c5a5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ae540 con 0x5595722624d0 2024-07-31T14:57:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f401c5a5740 10 client.0 _ll_register_callbacks cb 0x7f40080046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f400effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55957225a2f0 0x559572639fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f400f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595722831b0 0x559572636bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f400effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55957225a2f0 0x559572639fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46088/0 (socket says 172.21.15.67:46088) 2024-07-31T14:57:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f400e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722624d0 0x5595726364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f400effd640 1 -- 172.21.15.67:0/3680410525 learned_addr learned my addr 172.21.15.67:0/3680410525 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f400f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595722831b0 0x559572636bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60302/0 (socket says 172.21.15.67:60302) 2024-07-31T14:57:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f400f7fe640 1 -- 172.21.15.67:0/3680410525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722624d0 msgr2=0x5595726364c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f400f7fe640 1 --2- 172.21.15.67:0/3680410525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722624d0 0x5595726364c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f400f7fe640 1 -- 172.21.15.67:0/3680410525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55957225a2f0 msgr2=0x559572639fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f400f7fe640 1 --2- 172.21.15.67:0/3680410525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55957225a2f0 0x559572639fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f400f7fe640 1 -- 172.21.15.67:0/3680410525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595726a9fc0 con 0x5595722831b0 2024-07-31T14:57:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.969+0000 7f400effd640 1 --2- 172.21.15.67:0/3680410525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55957225a2f0 0x559572639fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:57:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.970+0000 7f400f7fe640 1 --2- 172.21.15.67:0/3680410525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595722831b0 0x559572636bd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4000006010 tx=0x7f40000095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45cd241226ecffc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.970+0000 7f3ff67fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:57:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.970+0000 7f3ff67fc640 1 -- 172.21.15.67:0/3680410525 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000058060 con 0x5595722831b0 2024-07-31T14:57:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.970+0000 7f3ff67fc640 1 -- 172.21.15.67:0/3680410525 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f4000004040 con 0x5595722831b0 2024-07-31T14:57:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.970+0000 7f401c5a5740 1 -- 172.21.15.67:0/3680410525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40080b3cb0 con 0x5595722831b0 2024-07-31T14:57:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.970+0000 7f3ff67fc640 1 -- 172.21.15.67:0/3680410525 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f400005f050 con 0x5595722831b0 2024-07-31T14:57:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.970+0000 7f401c5a5740 10 client.17229 fetch_fsmap learned FSMap version 45 2024-07-31T14:57:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.970+0000 7f401c5a5740 1 -- 172.21.15.67:0/3680410525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5595726ab120 con 0x5595722831b0 2024-07-31T14:57:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.970+0000 7f3ff67fc640 1 -- 172.21.15.67:0/3680410525 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40080b3cb0 con 0x5595722831b0 2024-07-31T14:57:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.971+0000 7f3ff67fc640 1 -- 172.21.15.67:0/3680410525 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7f400002fc60 con 0x5595722831b0 2024-07-31T14:57:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.971+0000 7f401c5a5740 10 client.17229 fetch_fsmap finished waiting for FSMap version 45 2024-07-31T14:57:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.971+0000 7f401c5a5740 10 client.17229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:57:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.971+0000 7f401c5a5740 20 client.17229 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.971+0000 7f401c5a5740 1 --2- 172.21.15.67:0/3680410525 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5595722171c0 0x5595726ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.971+0000 7f401c5a5740 4 client.17229 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:57:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.971+0000 7f401c5a5740 1 -- 172.21.15.67:0/3680410525 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559572598da0 con 0x5595722171c0 2024-07-31T14:57:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.971+0000 7f400effd640 1 --2- 172.21.15.67:0/3680410525 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5595722171c0 0x5595726ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.971+0000 7f400effd640 1 --2- 172.21.15.67:0/3680410525 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5595722171c0 0x5595726ad9d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b15bb62932c4177 server_cookie=401dd48dcc6521c4 in_seq=0 out_seq=0 2024-07-31T14:57:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:31.971+0000 7f3ff67fc640 10 client.17229 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:31.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:31 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:32.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:31 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f93457fa640 1 -- 172.21.15.67:0/2331773502 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561e478a2860 con 0x561e4751d250 2024-07-31T14:57:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f93457fa640 10 client.17211 handle_command_reply: tid=0 2024-07-31T14:57:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f936a27d740 1 client.17211 shutdown 2024-07-31T14:57:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f936a27d740 1 -- 172.21.15.67:0/2331773502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 msgr2=0x561e4793f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f936a27d740 1 --2- 172.21.15.67:0/2331773502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4793f170 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f934c0058c0 tx=0x7f934c0048d0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f936a27d740 1 -- 172.21.15.67:0/2331773502 shutdown_connections 2024-07-31T14:57:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f936a27d740 1 --2- 172.21.15.67:0/2331773502 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e4751d250 0x561e479b7d10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f936a27d740 1 --2- 172.21.15.67:0/2331773502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e47589520 0x561e479b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f936a27d740 1 --2- 172.21.15.67:0/2331773502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e47568840 0x561e4793f170 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f93457fa640 0 client.17211 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f936a27d740 1 --2- 172.21.15.67:0/2331773502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e47560660 0x561e4793ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f936a27d740 1 -- 172.21.15.67:0/2331773502 >> 172.21.15.67:0/2331773502 conn(0x561e4757d910 msgr2=0x561e47932170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f936a27d740 1 -- 172.21.15.67:0/2331773502 shutdown_connections 2024-07-31T14:57:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f936a27d740 1 -- 172.21.15.67:0/2331773502 wait complete. 2024-07-31T14:57:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.095+0000 7f936a27d740 20 client.17211 trim_cache size 0 max 16384 2024-07-31T14:57:32.100 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:57:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 45, 2024-07-31T14:57:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T14:57:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 16, 2024-07-31T14:57:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T14:57:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "c8d56e19": { 2024-07-31T14:57:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 16, 2024-07-31T14:57:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T14:57:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:57:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T14:57:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:57:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:57:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T14:57:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 130.0, 2024-07-31T14:57:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T14:57:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T14:57:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T14:57:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:57:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T14:57:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:57:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:57:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:57:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:57:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:57:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:57:32.103 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:57:32.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.096+0000 7f9363d2c640 1 -- 172.21.15.67:0/1418098312 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93480b3dc0 msgr2=0x7f93480b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:32.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.096+0000 7f9363d2c640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93480b3dc0 0x7f93480b62d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9354009430 tx=0x7f9354009800 comp rx=0 tx=0).stop 2024-07-31T14:57:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.096+0000 7f9363d2c640 1 -- 172.21.15.67:0/1418098312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f935c107dd0 msgr2=0x7f935c199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.096+0000 7f9363d2c640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f935c107dd0 0x7f935c199300 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f934c00c970 tx=0x7f934c00ce30 comp rx=0 tx=0).stop 2024-07-31T14:57:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.096+0000 7f9363d2c640 1 -- 172.21.15.67:0/1418098312 shutdown_connections 2024-07-31T14:57:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.096+0000 7f9363d2c640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93480b3dc0 0x7f93480b62d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.096+0000 7f9363d2c640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f935c108720 0x7f935c19e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.096+0000 7f9363d2c640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f935c107dd0 0x7f935c199300 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.096+0000 7f9363d2c640 1 --2- 172.21.15.67:0/1418098312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f935c102490 0x7f935c198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.096+0000 7f9363d2c640 1 -- 172.21.15.67:0/1418098312 >> 172.21.15.67:0/1418098312 conn(0x7f935c079160 msgr2=0x7f935c104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.096+0000 7f9363d2c640 1 -- 172.21.15.67:0/1418098312 shutdown_connections 2024-07-31T14:57:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:32.096+0000 7f9363d2c640 1 -- 172.21.15.67:0/1418098312 wait complete. 2024-07-31T14:57:32.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:31 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f3ff67fc640 1 -- 172.21.15.67:0/3680410525 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559572598da0 con 0x5595722171c0 2024-07-31T14:57:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f3ff67fc640 10 client.17229 handle_command_reply: tid=0 2024-07-31T14:57:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f401c5a5740 1 client.17229 shutdown 2024-07-31T14:57:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f401c5a5740 1 -- 172.21.15.67:0/3680410525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595722831b0 msgr2=0x559572636bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f401c5a5740 1 --2- 172.21.15.67:0/3680410525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595722831b0 0x559572636bd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4000006010 tx=0x7f40000095c0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f401c5a5740 1 -- 172.21.15.67:0/3680410525 shutdown_connections 2024-07-31T14:57:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f401c5a5740 1 --2- 172.21.15.67:0/3680410525 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5595722171c0 0x5595726ad9d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f401c5a5740 1 --2- 172.21.15.67:0/3680410525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5595722831b0 0x559572636bd0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f401c5a5740 1 --2- 172.21.15.67:0/3680410525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595722624d0 0x5595726364c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f401c5a5740 1 --2- 172.21.15.67:0/3680410525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55957225a2f0 0x559572639fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f3ff67fc640 0 client.17229 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f401c5a5740 1 -- 172.21.15.67:0/3680410525 >> 172.21.15.67:0/3680410525 conn(0x559572277300 msgr2=0x559572633290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f401c5a5740 1 -- 172.21.15.67:0/3680410525 shutdown_connections 2024-07-31T14:57:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f401c5a5740 1 -- 172.21.15.67:0/3680410525 wait complete. 2024-07-31T14:57:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.814+0000 7f401c5a5740 20 client.17229 trim_cache size 0 max 16384 2024-07-31T14:57:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:57:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:57:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:57:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:57:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.815+0000 7f40155d1640 1 -- 172.21.15.67:0/1944542113 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3ffc0b3d40 msgr2=0x7f3ffc0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.815+0000 7f40155d1640 1 --2- 172.21.15.67:0/1944542113 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3ffc0b3d40 0x7f3ffc0b6250 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f400002f500 tx=0x7f400002f310 comp rx=0 tx=0).stop 2024-07-31T14:57:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.815+0000 7f40155d1640 1 -- 172.21.15.67:0/1944542113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40101023f0 msgr2=0x7f4010199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.815+0000 7f40155d1640 1 --2- 172.21.15.67:0/1944542113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40101023f0 0x7f4010199e30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f400800e9f0 tx=0x7f400800eeb0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.815+0000 7f40155d1640 1 -- 172.21.15.67:0/1944542113 shutdown_connections 2024-07-31T14:57:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.815+0000 7f40155d1640 1 --2- 172.21.15.67:0/1944542113 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3ffc0b3d40 0x7f3ffc0b6250 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.815+0000 7f40155d1640 1 --2- 172.21.15.67:0/1944542113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4010108670 0x7f401019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.815+0000 7f40155d1640 1 --2- 172.21.15.67:0/1944542113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4010107d20 0x7f401019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.815+0000 7f40155d1640 1 --2- 172.21.15.67:0/1944542113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40101023f0 0x7f4010199e30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.815+0000 7f40155d1640 1 -- 172.21.15.67:0/1944542113 >> 172.21.15.67:0/1944542113 conn(0x7f40100fddf0 msgr2=0x7f4010107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.815+0000 7f40155d1640 1 -- 172.21.15.67:0/1944542113 shutdown_connections 2024-07-31T14:57:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:32.816+0000 7f40155d1640 1 -- 172.21.15.67:0/1944542113 wait complete. 2024-07-31T14:57:32.896 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c8d56e19', total seconds elapsed: 51.7 2024-07-31T14:57:32.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:32 smithi077 ceph-mon[37327]: pgmap v594: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 2.35k op/s 2024-07-31T14:57:33.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:32 smithi067 ceph-mon[32750]: pgmap v594: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 2.35k op/s 2024-07-31T14:57:33.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:32 smithi073 ceph-mon[36714]: pgmap v594: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 2.35k op/s 2024-07-31T14:57:35.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:34 smithi067 ceph-mon[32750]: pgmap v595: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 2.21k op/s 2024-07-31T14:57:35.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:34 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:35.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:34 smithi073 ceph-mon[36714]: pgmap v595: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 2.21k op/s 2024-07-31T14:57:35.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:34 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:34 smithi077 ceph-mon[37327]: pgmap v595: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 2.21k op/s 2024-07-31T14:57:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:34 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:37.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:36 smithi067 ceph-mon[32750]: pgmap v596: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 2.05k op/s 2024-07-31T14:57:37.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:36 smithi073 ceph-mon[36714]: pgmap v596: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 2.05k op/s 2024-07-31T14:57:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:36 smithi077 ceph-mon[37327]: pgmap v596: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 2.05k op/s 2024-07-31T14:57:38.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:37 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:57:38.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:37 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:57:38.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:37 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:57:38.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:37 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:57:38.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:37 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:57:38.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:37 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:57:38.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:37 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:57:38.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:37 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:57:38.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:37 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:57:39.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:38 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:57:39.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:38 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:57:39.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:38 smithi067 ceph-mon[32750]: pgmap v597: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 1.17k op/s 2024-07-31T14:57:39.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:38 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:39.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:38 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:57:39.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:38 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:57:39.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:38 smithi073 ceph-mon[36714]: pgmap v597: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 1.17k op/s 2024-07-31T14:57:39.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:38 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:38 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:57:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:38 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:57:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:38 smithi077 ceph-mon[37327]: pgmap v597: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 1.17k op/s 2024-07-31T14:57:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:38 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:39.308 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:57:39.582 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:57:41.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:40 smithi067 ceph-mon[32750]: pgmap v598: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 742 op/s 2024-07-31T14:57:41.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:40 smithi073 ceph-mon[36714]: pgmap v598: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 742 op/s 2024-07-31T14:57:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:40 smithi077 ceph-mon[37327]: pgmap v598: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 742 op/s 2024-07-31T14:57:41.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.436+0000 7ff032e16640 1 -- 172.21.15.67:0/4278172250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108720 msgr2=0x7ff02c10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.436+0000 7ff032e16640 1 --2- 172.21.15.67:0/4278172250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108720 0x7ff02c10ef50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff024009ca0 tx=0x7ff02402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.436+0000 7ff032e16640 1 -- 172.21.15.67:0/4278172250 shutdown_connections 2024-07-31T14:57:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.436+0000 7ff032e16640 1 --2- 172.21.15.67:0/4278172250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108720 0x7ff02c10ef50 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.436+0000 7ff032e16640 1 --2- 172.21.15.67:0/4278172250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff02c107dd0 0x7ff02c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.436+0000 7ff032e16640 1 --2- 172.21.15.67:0/4278172250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff02c102490 0x7ff02c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.436+0000 7ff032e16640 1 -- 172.21.15.67:0/4278172250 >> 172.21.15.67:0/4278172250 conn(0x7ff02c079190 msgr2=0x7ff02c0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.436+0000 7ff032e16640 1 -- 172.21.15.67:0/4278172250 shutdown_connections 2024-07-31T14:57:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.436+0000 7ff032e16640 1 -- 172.21.15.67:0/4278172250 wait complete. 2024-07-31T14:57:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff032e16640 1 Processor -- start 2024-07-31T14:57:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff032e16640 1 -- start start 2024-07-31T14:57:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff032e16640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff02c102490 0x7ff02c10fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff032e16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff02c107dd0 0x7ff02c10ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff032e16640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108720 0x7ff02c1104b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff032e16640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c113500 con 0x7ff02c108720 2024-07-31T14:57:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff032e16640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c113380 con 0x7ff02c107dd0 2024-07-31T14:57:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff032e16640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c113680 con 0x7ff02c102490 2024-07-31T14:57:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff0313b3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108720 0x7ff02c1104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff02c107dd0 0x7ff02c10ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff0313b3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108720 0x7ff02c1104b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33902/0 (socket says 172.21.15.67:33902) 2024-07-31T14:57:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff02c107dd0 0x7ff02c10ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:38474/0 (socket says 172.21.15.67:38474) 2024-07-31T14:57:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff0313b3640 1 -- 172.21.15.67:0/215358852 learned_addr learned my addr 172.21.15.67:0/215358852 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff030bb2640 1 --2- 172.21.15.67:0/215358852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff02c102490 0x7ff02c10fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff0313b3640 1 -- 172.21.15.67:0/215358852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff02c102490 msgr2=0x7ff02c10fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff0313b3640 1 --2- 172.21.15.67:0/215358852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff02c102490 0x7ff02c10fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff0313b3640 1 -- 172.21.15.67:0/215358852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff02c107dd0 msgr2=0x7ff02c10ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.437+0000 7ff0313b3640 1 --2- 172.21.15.67:0/215358852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff02c107dd0 0x7ff02c10ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.438+0000 7ff0313b3640 1 -- 172.21.15.67:0/215358852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c1a80f0 con 0x7ff02c108720 2024-07-31T14:57:41.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.438+0000 7ff02bfff640 1 --2- 172.21.15.67:0/215358852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff02c107dd0 0x7ff02c10ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:57:41.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.438+0000 7ff0313b3640 1 --2- 172.21.15.67:0/215358852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108720 0x7ff02c1104b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff0240098d0 tx=0x7ff0240317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:41.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.438+0000 7ff029ffb640 1 -- 172.21.15.67:0/215358852 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024003b70 con 0x7ff02c108720 2024-07-31T14:57:41.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.438+0000 7ff032e16640 1 -- 172.21.15.67:0/215358852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02c1a8380 con 0x7ff02c108720 2024-07-31T14:57:41.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.438+0000 7ff029ffb640 1 -- 172.21.15.67:0/215358852 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff024003d10 con 0x7ff02c108720 2024-07-31T14:57:41.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.438+0000 7ff029ffb640 1 -- 172.21.15.67:0/215358852 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02402fa00 con 0x7ff02c108720 2024-07-31T14:57:41.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.438+0000 7ff032e16640 1 -- 172.21.15.67:0/215358852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02c1a8910 con 0x7ff02c108720 2024-07-31T14:57:41.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.441+0000 7ff039367740 1 Processor -- start 2024-07-31T14:57:41.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.441+0000 7ff029ffb640 1 -- 172.21.15.67:0/215358852 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff02402fbc0 con 0x7ff02c108720 2024-07-31T14:57:41.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.441+0000 7ff039367740 1 -- start start 2024-07-31T14:57:41.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.441+0000 7ff039367740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787f4070 0x55b078b99520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.441+0000 7ff039367740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787cb1b0 0x55b078b99a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.441+0000 7ff039367740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787d3390 0x55b078b99fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.441+0000 7ff039367740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b078b0fb60 con 0x55b0787d3390 2024-07-31T14:57:41.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.441+0000 7ff039367740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b078aff060 con 0x55b0787cb1b0 2024-07-31T14:57:41.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.441+0000 7ff039367740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b078b030a0 con 0x55b0787f4070 2024-07-31T14:57:41.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff030bb2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787cb1b0 0x55b078b99a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787d3390 0x55b078b99fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff030bb2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787cb1b0 0x55b078b99a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:38484/0 (socket says 172.21.15.67:38484) 2024-07-31T14:57:41.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787d3390 0x55b078b99fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33912/0 (socket says 172.21.15.67:33912) 2024-07-31T14:57:41.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff030bb2640 1 -- 172.21.15.67:0/58130280 learned_addr learned my addr 172.21.15.67:0/58130280 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:41.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff02bfff640 1 --2- 172.21.15.67:0/58130280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787f4070 0x55b078b99520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff030bb2640 1 -- 172.21.15.67:0/58130280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787f4070 msgr2=0x55b078b99520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:41.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff030bb2640 1 --2- 172.21.15.67:0/58130280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787f4070 0x55b078b99520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff030bb2640 1 -- 172.21.15.67:0/58130280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787d3390 msgr2=0x55b078b99fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:41.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff030bb2640 1 --2- 172.21.15.67:0/58130280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787d3390 0x55b078b99fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff029ffb640 1 --2- 172.21.15.67:0/215358852 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff01c0b3d40 0x7ff01c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff030bb2640 1 -- 172.21.15.67:0/58130280 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b078aa5b40 con 0x55b0787cb1b0 2024-07-31T14:57:41.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff02bfff640 1 --2- 172.21.15.67:0/58130280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787f4070 0x55b078b99520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:57:41.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff02bfff640 1 --2- 172.21.15.67:0/215358852 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff01c0b3d40 0x7ff01c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff029ffb640 1 -- 172.21.15.67:0/215358852 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff024105850 con 0x7ff02c108720 2024-07-31T14:57:41.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.442+0000 7ff030bb2640 1 --2- 172.21.15.67:0/58130280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787cb1b0 0x55b078b99a60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff020009c90 tx=0x7ff020011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d9aea9914d51f5d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:41.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.443+0000 7ff016ffd640 1 -- 172.21.15.67:0/58130280 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020005220 con 0x55b0787cb1b0 2024-07-31T14:57:41.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.443+0000 7ff016ffd640 1 -- 172.21.15.67:0/58130280 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff020004c50 con 0x55b0787cb1b0 2024-07-31T14:57:41.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.443+0000 7ff016ffd640 1 -- 172.21.15.67:0/58130280 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0200197a0 con 0x55b0787cb1b0 2024-07-31T14:57:41.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.443+0000 7ff039367740 1 -- 172.21.15.67:0/58130280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787cb1b0 msgr2=0x55b078b99a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:41.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.443+0000 7ff02bfff640 1 --2- 172.21.15.67:0/215358852 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff01c0b3d40 0x7ff01c0b6250 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff02c1058e0 tx=0x7ff018013040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:41.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.443+0000 7ff039367740 1 --2- 172.21.15.67:0/58130280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787cb1b0 0x55b078b99a60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff020009c90 tx=0x7ff020011810 comp rx=0 tx=0).stop 2024-07-31T14:57:41.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.443+0000 7ff039367740 1 -- 172.21.15.67:0/58130280 shutdown_connections 2024-07-31T14:57:41.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.444+0000 7ff039367740 1 --2- 172.21.15.67:0/58130280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787d3390 0x55b078b99fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.444+0000 7ff039367740 1 --2- 172.21.15.67:0/58130280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787cb1b0 0x55b078b99a60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.444+0000 7ff039367740 1 --2- 172.21.15.67:0/58130280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787f4070 0x55b078b99520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.444+0000 7ff039367740 1 -- 172.21.15.67:0/58130280 >> 172.21.15.67:0/58130280 conn(0x55b0787e8460 msgr2=0x55b078b97410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:41.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.444+0000 7ff039367740 1 -- 172.21.15.67:0/58130280 shutdown_connections 2024-07-31T14:57:41.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.444+0000 7ff039367740 1 -- 172.21.15.67:0/58130280 wait complete. 2024-07-31T14:57:41.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.444+0000 7ff039367740 1 -- start start 2024-07-31T14:57:41.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.444+0000 7ff039367740 1 Processor -- start 2024-07-31T14:57:41.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff039367740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787cb1b0 0x55b078ba50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff039367740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787d3390 0x55b078ba5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff039367740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787f4070 0x55b078bb0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff039367740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b078962700 con 0x55b0787d3390 2024-07-31T14:57:41.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff039367740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b078aa1990 con 0x55b0787f4070 2024-07-31T14:57:41.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff039367740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b078ba7d00 con 0x55b0787cb1b0 2024-07-31T14:57:41.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff039367740 10 client.0 _ll_register_callbacks cb 0x55b078abc4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:41.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787d3390 0x55b078ba5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787d3390 0x55b078ba5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33922/0 (socket says 172.21.15.67:33922) 2024-07-31T14:57:41.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff02bfff640 1 -- 172.21.15.67:0/318342111 learned_addr learned my addr 172.21.15.67:0/318342111 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:41.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff0313b3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787f4070 0x55b078bb0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff030bb2640 1 --2- 172.21.15.67:0/318342111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787cb1b0 0x55b078ba50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff02bfff640 1 -- 172.21.15.67:0/318342111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787cb1b0 msgr2=0x55b078ba50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff02bfff640 1 --2- 172.21.15.67:0/318342111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787cb1b0 0x55b078ba50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff02bfff640 1 -- 172.21.15.67:0/318342111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787f4070 msgr2=0x55b078bb0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff02bfff640 1 --2- 172.21.15.67:0/318342111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787f4070 0x55b078bb0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff02bfff640 1 -- 172.21.15.67:0/318342111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b07885f650 con 0x55b0787d3390 2024-07-31T14:57:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff0313b3640 1 --2- 172.21.15.67:0/318342111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787f4070 0x55b078bb0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:57:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff02bfff640 1 --2- 172.21.15.67:0/318342111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787d3390 0x55b078ba5630 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff0180206c0 tx=0x7ff018004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc902a1a4777d3c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.445+0000 7ff00bfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:57:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.446+0000 7ff00bfff640 1 -- 172.21.15.67:0/318342111 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01801e840 con 0x55b0787d3390 2024-07-31T14:57:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.446+0000 7ff00bfff640 1 -- 172.21.15.67:0/318342111 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff01801e9e0 con 0x55b0787d3390 2024-07-31T14:57:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.446+0000 7ff039367740 1 -- 172.21.15.67:0/318342111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff020004c50 con 0x55b0787d3390 2024-07-31T14:57:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.446+0000 7ff00bfff640 1 -- 172.21.15.67:0/318342111 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff01801ebe0 con 0x55b0787d3390 2024-07-31T14:57:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.446+0000 7ff039367740 10 client.17241 fetch_fsmap learned FSMap version 47 2024-07-31T14:57:41.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.446+0000 7ff039367740 1 -- 172.21.15.67:0/318342111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b078bb3dd0 con 0x55b0787d3390 2024-07-31T14:57:41.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.446+0000 7ff00bfff640 1 -- 172.21.15.67:0/318342111 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff020004c50 con 0x55b0787d3390 2024-07-31T14:57:41.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.446+0000 7ff00bfff640 1 -- 172.21.15.67:0/318342111 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7ff01801d3b0 con 0x55b0787d3390 2024-07-31T14:57:41.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.446+0000 7ff039367740 10 client.17241 fetch_fsmap finished waiting for FSMap version 47 2024-07-31T14:57:41.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.446+0000 7ff039367740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:57:41.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.447+0000 7ff039367740 20 client.17241 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:41.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.447+0000 7ff039367740 1 --2- 172.21.15.67:0/318342111 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b0787b29f0 0x55b078bb73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.447+0000 7ff039367740 4 client.17241 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:57:41.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.447+0000 7ff039367740 1 -- 172.21.15.67:0/318342111 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b078b11ab0 con 0x55b0787b29f0 2024-07-31T14:57:41.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.447+0000 7ff030bb2640 1 --2- 172.21.15.67:0/318342111 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b0787b29f0 0x55b078bb73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.447+0000 7ff00bfff640 10 client.17241 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:41.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.447+0000 7ff030bb2640 1 --2- 172.21.15.67:0/318342111 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b0787b29f0 0x55b078bb73c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7feb09913fd80404 server_cookie=1cd43bd9effa3a3d in_seq=0 out_seq=0 2024-07-31T14:57:41.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff00bfff640 1 -- 172.21.15.67:0/318342111 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b078b11ab0 con 0x55b0787b29f0 2024-07-31T14:57:41.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff00bfff640 10 client.17241 handle_command_reply: tid=0 2024-07-31T14:57:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff039367740 1 client.17241 shutdown 2024-07-31T14:57:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff039367740 1 -- 172.21.15.67:0/318342111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787d3390 msgr2=0x55b078ba5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff039367740 1 --2- 172.21.15.67:0/318342111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787d3390 0x55b078ba5630 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff0180206c0 tx=0x7ff018004730 comp rx=0 tx=0).stop 2024-07-31T14:57:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff039367740 1 -- 172.21.15.67:0/318342111 shutdown_connections 2024-07-31T14:57:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff039367740 1 --2- 172.21.15.67:0/318342111 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b0787b29f0 0x55b078bb73c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff039367740 1 --2- 172.21.15.67:0/318342111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787f4070 0x55b078bb0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff039367740 1 --2- 172.21.15.67:0/318342111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787d3390 0x55b078ba5630 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff039367740 1 --2- 172.21.15.67:0/318342111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787cb1b0 0x55b078ba50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff039367740 1 -- 172.21.15.67:0/318342111 >> 172.21.15.67:0/318342111 conn(0x55b0787e8460 msgr2=0x55b078b9c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff00bfff640 0 client.17241 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff039367740 1 -- 172.21.15.67:0/318342111 shutdown_connections 2024-07-31T14:57:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff039367740 1 -- 172.21.15.67:0/318342111 wait complete. 2024-07-31T14:57:41.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.449+0000 7ff039367740 20 client.17241 trim_cache size 0 max 16384 2024-07-31T14:57:41.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.468+0000 7ff039367740 1 Processor -- start 2024-07-31T14:57:41.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.468+0000 7ff039367740 1 -- start start 2024-07-31T14:57:41.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.468+0000 7ff039367740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787cb1b0 0x55b078b97480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.468+0000 7ff039367740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787d3390 0x55b078b9e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.468+0000 7ff039367740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787f4070 0x55b078b9ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.468+0000 7ff039367740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b28f0 con 0x55b0787f4070 2024-07-31T14:57:41.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.468+0000 7ff039367740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b4610 con 0x55b0787cb1b0 2024-07-31T14:57:41.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.468+0000 7ff039367740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b3050 con 0x55b0787d3390 2024-07-31T14:57:41.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.468+0000 7ff0313b3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787f4070 0x55b078b9ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.468+0000 7ff030bb2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787cb1b0 0x55b078b97480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.468+0000 7ff0313b3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787f4070 0x55b078b9ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33928/0 (socket says 172.21.15.67:33928) 2024-07-31T14:57:41.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.468+0000 7ff0313b3640 1 -- 172.21.15.67:0/1802149789 learned_addr learned my addr 172.21.15.67:0/1802149789 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:41.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.468+0000 7ff02bfff640 1 --2- 172.21.15.67:0/1802149789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787d3390 0x55b078b9e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.469+0000 7ff0313b3640 1 -- 172.21.15.67:0/1802149789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787d3390 msgr2=0x55b078b9e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.469+0000 7ff0313b3640 1 --2- 172.21.15.67:0/1802149789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787d3390 0x55b078b9e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.469+0000 7ff0313b3640 1 -- 172.21.15.67:0/1802149789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787cb1b0 msgr2=0x55b078b97480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.469+0000 7ff0313b3640 1 --2- 172.21.15.67:0/1802149789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787cb1b0 0x55b078b97480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.469+0000 7ff0313b3640 1 -- 172.21.15.67:0/1802149789 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b078bb3be0 con 0x55b0787f4070 2024-07-31T14:57:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.469+0000 7ff030bb2640 1 --2- 172.21.15.67:0/1802149789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787cb1b0 0x55b078b97480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:57:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.469+0000 7ff0313b3640 1 --2- 172.21.15.67:0/1802149789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787f4070 0x55b078b9ece0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff024102e00 tx=0x7ff0240c81e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289783f14454377 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.469+0000 7ff0167fc640 1 -- 172.21.15.67:0/1802149789 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240cc960 con 0x55b0787f4070 2024-07-31T14:57:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.469+0000 7ff0167fc640 1 -- 172.21.15.67:0/1802149789 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff024036030 con 0x55b0787f4070 2024-07-31T14:57:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.469+0000 7ff0167fc640 1 -- 172.21.15.67:0/1802149789 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240d3be0 con 0x55b0787f4070 2024-07-31T14:57:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.470+0000 7ff039367740 1 -- 172.21.15.67:0/1802149789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787f4070 msgr2=0x55b078b9ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.470+0000 7ff039367740 1 --2- 172.21.15.67:0/1802149789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787f4070 0x55b078b9ece0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff024102e00 tx=0x7ff0240c81e0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.470+0000 7ff039367740 1 -- 172.21.15.67:0/1802149789 shutdown_connections 2024-07-31T14:57:41.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.470+0000 7ff039367740 1 --2- 172.21.15.67:0/1802149789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787f4070 0x55b078b9ece0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.470+0000 7ff039367740 1 --2- 172.21.15.67:0/1802149789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787d3390 0x55b078b9e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.470+0000 7ff039367740 1 --2- 172.21.15.67:0/1802149789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787cb1b0 0x55b078b97480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.470+0000 7ff039367740 1 -- 172.21.15.67:0/1802149789 >> 172.21.15.67:0/1802149789 conn(0x55b0787e8460 msgr2=0x55b078b96d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:41.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.470+0000 7ff039367740 1 -- 172.21.15.67:0/1802149789 shutdown_connections 2024-07-31T14:57:41.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.470+0000 7ff039367740 1 -- 172.21.15.67:0/1802149789 wait complete. 2024-07-31T14:57:41.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.470+0000 7ff039367740 1 -- start start 2024-07-31T14:57:41.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.470+0000 7ff039367740 1 Processor -- start 2024-07-31T14:57:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff039367740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787cb1b0 0x55b078bad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff039367740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787d3390 0x55b078bb0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff039367740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787f4070 0x55b078bb0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff039367740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b3f40 con 0x55b0787cb1b0 2024-07-31T14:57:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff039367740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b37b0 con 0x55b0787f4070 2024-07-31T14:57:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff039367740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b08d0 con 0x55b0787d3390 2024-07-31T14:57:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff039367740 10 client.0 _ll_register_callbacks cb 0x7ff01800ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff030bb2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787cb1b0 0x55b078bad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff0313b3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787f4070 0x55b078bb0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff030bb2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787cb1b0 0x55b078bad470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33942/0 (socket says 172.21.15.67:33942) 2024-07-31T14:57:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff030bb2640 1 -- 172.21.15.67:0/2492387234 learned_addr learned my addr 172.21.15.67:0/2492387234 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff0313b3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787f4070 0x55b078bb0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:38520/0 (socket says 172.21.15.67:38520) 2024-07-31T14:57:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff02bfff640 1 --2- 172.21.15.67:0/2492387234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787d3390 0x55b078bb0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff030bb2640 1 -- 172.21.15.67:0/2492387234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787d3390 msgr2=0x55b078bb0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:41.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff030bb2640 1 --2- 172.21.15.67:0/2492387234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787d3390 0x55b078bb0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff030bb2640 1 -- 172.21.15.67:0/2492387234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787f4070 msgr2=0x55b078bb0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:41.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff030bb2640 1 --2- 172.21.15.67:0/2492387234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787f4070 0x55b078bb0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff030bb2640 1 -- 172.21.15.67:0/2492387234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b078bb1230 con 0x55b0787cb1b0 2024-07-31T14:57:41.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.471+0000 7ff0313b3640 1 --2- 172.21.15.67:0/2492387234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787f4070 0x55b078bb0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:57:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.472+0000 7ff030bb2640 1 --2- 172.21.15.67:0/2492387234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787cb1b0 0x55b078bad470 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff020000c00 tx=0x7ff020009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8454c281a03355b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.472+0000 7ff00bfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:57:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.472+0000 7ff00bfff640 1 -- 172.21.15.67:0/2492387234 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020005c50 con 0x55b0787cb1b0 2024-07-31T14:57:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.472+0000 7ff039367740 1 -- 172.21.15.67:0/2492387234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff024036030 con 0x55b0787cb1b0 2024-07-31T14:57:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.472+0000 7ff00bfff640 1 -- 172.21.15.67:0/2492387234 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7ff0200051b0 con 0x55b0787cb1b0 2024-07-31T14:57:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.472+0000 7ff00bfff640 1 -- 172.21.15.67:0/2492387234 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff02003a050 con 0x55b0787cb1b0 2024-07-31T14:57:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.472+0000 7ff039367740 10 client.17247 fetch_fsmap learned FSMap version 47 2024-07-31T14:57:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.472+0000 7ff039367740 1 -- 172.21.15.67:0/2492387234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b078baad40 con 0x55b0787cb1b0 2024-07-31T14:57:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.472+0000 7ff00bfff640 1 -- 172.21.15.67:0/2492387234 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff024036030 con 0x55b0787cb1b0 2024-07-31T14:57:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.472+0000 7ff00bfff640 1 -- 172.21.15.67:0/2492387234 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7ff02002c430 con 0x55b0787cb1b0 2024-07-31T14:57:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.472+0000 7ff039367740 10 client.17247 fetch_fsmap finished waiting for FSMap version 47 2024-07-31T14:57:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.472+0000 7ff039367740 10 client.17247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:57:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.472+0000 7ff039367740 20 client.17247 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.473+0000 7ff039367740 1 --2- 172.21.15.67:0/2492387234 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b078787da0 0x55b078c22060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.473+0000 7ff039367740 4 client.17247 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:57:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.473+0000 7ff039367740 1 -- 172.21.15.67:0/2492387234 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b078b0d550 con 0x55b078787da0 2024-07-31T14:57:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.473+0000 7ff02bfff640 1 --2- 172.21.15.67:0/2492387234 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b078787da0 0x55b078c22060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.473+0000 7ff00bfff640 10 client.17247 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.473+0000 7ff02bfff640 1 --2- 172.21.15.67:0/2492387234 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b078787da0 0x55b078c22060 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a5b2a3584f559b7 server_cookie=d717956f3e4cadda in_seq=0 out_seq=0 2024-07-31T14:57:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.474+0000 7ff00bfff640 1 -- 172.21.15.67:0/2492387234 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b078b0d550 con 0x55b078787da0 2024-07-31T14:57:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.474+0000 7ff00bfff640 10 client.17247 handle_command_reply: tid=0 2024-07-31T14:57:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.474+0000 7ff039367740 1 client.17247 shutdown 2024-07-31T14:57:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.475+0000 7ff039367740 1 -- 172.21.15.67:0/2492387234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787cb1b0 msgr2=0x55b078bad470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.475+0000 7ff039367740 1 --2- 172.21.15.67:0/2492387234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787cb1b0 0x55b078bad470 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff020000c00 tx=0x7ff020009690 comp rx=0 tx=0).stop 2024-07-31T14:57:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.475+0000 7ff039367740 1 -- 172.21.15.67:0/2492387234 shutdown_connections 2024-07-31T14:57:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.475+0000 7ff039367740 1 --2- 172.21.15.67:0/2492387234 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b078787da0 0x55b078c22060 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.475+0000 7ff039367740 1 --2- 172.21.15.67:0/2492387234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0787f4070 0x55b078bb0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.475+0000 7ff039367740 1 --2- 172.21.15.67:0/2492387234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b0787d3390 0x55b078bb0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.475+0000 7ff039367740 1 --2- 172.21.15.67:0/2492387234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0787cb1b0 0x55b078bad470 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.475+0000 7ff039367740 1 -- 172.21.15.67:0/2492387234 >> 172.21.15.67:0/2492387234 conn(0x55b0787e8460 msgr2=0x55b078b9a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.475+0000 7ff00bfff640 0 client.17247 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.475+0000 7ff039367740 1 -- 172.21.15.67:0/2492387234 shutdown_connections 2024-07-31T14:57:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.475+0000 7ff039367740 1 -- 172.21.15.67:0/2492387234 wait complete. 2024-07-31T14:57:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.475+0000 7ff039367740 20 client.17247 trim_cache size 0 max 16384 2024-07-31T14:57:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:57:41.482 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T14:57:41.482 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T14:57:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:57:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.476+0000 7ff032e16640 1 -- 172.21.15.67:0/215358852 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff01c0b3d40 msgr2=0x7ff01c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.476+0000 7ff032e16640 1 --2- 172.21.15.67:0/215358852 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff01c0b3d40 0x7ff01c0b6250 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff02c1058e0 tx=0x7ff018013040 comp rx=0 tx=0).stop 2024-07-31T14:57:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.476+0000 7ff032e16640 1 -- 172.21.15.67:0/215358852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108720 msgr2=0x7ff02c1104b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.476+0000 7ff032e16640 1 --2- 172.21.15.67:0/215358852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108720 0x7ff02c1104b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff0240098d0 tx=0x7ff0240317b0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.476+0000 7ff032e16640 1 -- 172.21.15.67:0/215358852 shutdown_connections 2024-07-31T14:57:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.476+0000 7ff032e16640 1 --2- 172.21.15.67:0/215358852 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff01c0b3d40 0x7ff01c0b6250 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.476+0000 7ff032e16640 1 --2- 172.21.15.67:0/215358852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108720 0x7ff02c1104b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.476+0000 7ff032e16640 1 --2- 172.21.15.67:0/215358852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff02c107dd0 0x7ff02c10ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.476+0000 7ff032e16640 1 --2- 172.21.15.67:0/215358852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff02c102490 0x7ff02c10fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.476+0000 7ff032e16640 1 -- 172.21.15.67:0/215358852 >> 172.21.15.67:0/215358852 conn(0x7ff02c079190 msgr2=0x7ff02c103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.476+0000 7ff032e16640 1 -- 172.21.15.67:0/215358852 shutdown_connections 2024-07-31T14:57:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:41.476+0000 7ff032e16640 1 -- 172.21.15.67:0/215358852 wait complete. 2024-07-31T14:57:42.582 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T14:57:42.582 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:57:42.849 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:57:42.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:42 smithi067 ceph-mon[32750]: pgmap v599: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 348 op/s 2024-07-31T14:57:43.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:42 smithi073 ceph-mon[36714]: pgmap v599: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 348 op/s 2024-07-31T14:57:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:42 smithi077 ceph-mon[37327]: pgmap v599: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 348 op/s 2024-07-31T14:57:44.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:43 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:44.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:43 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:44.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:43 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:45.043 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:44 smithi067 ceph-mon[32750]: pgmap v600: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 306 op/s 2024-07-31T14:57:45.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.044+0000 7fe87c42c640 1 -- 172.21.15.67:0/1354241408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874109870 msgr2=0x7fe8741100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:45.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.044+0000 7fe87c42c640 1 --2- 172.21.15.67:0/1354241408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874109870 0x7fe8741100a0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009900 tx=0x7fe86c02a410 comp rx=0 tx=0).stop 2024-07-31T14:57:45.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.044+0000 7fe87c42c640 1 -- 172.21.15.67:0/1354241408 shutdown_connections 2024-07-31T14:57:45.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.044+0000 7fe87c42c640 1 --2- 172.21.15.67:0/1354241408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874109870 0x7fe8741100a0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.044+0000 7fe87c42c640 1 --2- 172.21.15.67:0/1354241408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe874108f20 0x7fe874109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.044+0000 7fe87c42c640 1 --2- 172.21.15.67:0/1354241408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8741035e0 0x7fe8741039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.044+0000 7fe87c42c640 1 -- 172.21.15.67:0/1354241408 >> 172.21.15.67:0/1354241408 conn(0x7fe8740ff030 msgr2=0x7fe874101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:45.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.044+0000 7fe87c42c640 1 -- 172.21.15.67:0/1354241408 shutdown_connections 2024-07-31T14:57:45.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.044+0000 7fe87c42c640 1 -- 172.21.15.67:0/1354241408 wait complete. 2024-07-31T14:57:45.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.045+0000 7fe87c42c640 1 Processor -- start 2024-07-31T14:57:45.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.045+0000 7fe87c42c640 1 -- start start 2024-07-31T14:57:45.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.045+0000 7fe87c42c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8741035e0 0x7fe874199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.045+0000 7fe87c42c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe874108f20 0x7fe87419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.045+0000 7fe87c42c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874109870 0x7fe87419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.045+0000 7fe87c42c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe874113540 con 0x7fe874109870 2024-07-31T14:57:45.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.045+0000 7fe87c42c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741133c0 con 0x7fe8741035e0 2024-07-31T14:57:45.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.045+0000 7fe87c42c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741136c0 con 0x7fe874108f20 2024-07-31T14:57:45.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe87a9c9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874109870 0x7fe87419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe87a1c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8741035e0 0x7fe874199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe87a1c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8741035e0 0x7fe874199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:38524/0 (socket says 172.21.15.67:38524) 2024-07-31T14:57:45.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe87a1c8640 1 -- 172.21.15.67:0/2187980096 learned_addr learned my addr 172.21.15.67:0/2187980096 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:45.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe8799c7640 1 --2- 172.21.15.67:0/2187980096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe874108f20 0x7fe87419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe87a1c8640 1 -- 172.21.15.67:0/2187980096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe874108f20 msgr2=0x7fe87419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:45.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe87a1c8640 1 --2- 172.21.15.67:0/2187980096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe874108f20 0x7fe87419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe87a1c8640 1 -- 172.21.15.67:0/2187980096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874109870 msgr2=0x7fe87419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:45.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe87a1c8640 1 --2- 172.21.15.67:0/2187980096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874109870 0x7fe87419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe87a1c8640 1 -- 172.21.15.67:0/2187980096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87419fe40 con 0x7fe8741035e0 2024-07-31T14:57:45.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe87a1c8640 1 --2- 172.21.15.67:0/2187980096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8741035e0 0x7fe874199ed0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe86400ea80 tx=0x7fe86400ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:45.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe8737fe640 1 -- 172.21.15.67:0/2187980096 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86400ce40 con 0x7fe8741035e0 2024-07-31T14:57:45.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe87c42c640 1 -- 172.21.15.67:0/2187980096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8741a0130 con 0x7fe8741035e0 2024-07-31T14:57:45.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe8737fe640 1 -- 172.21.15.67:0/2187980096 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe864005020 con 0x7fe8741035e0 2024-07-31T14:57:45.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe8737fe640 1 -- 172.21.15.67:0/2187980096 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8640116d0 con 0x7fe8741035e0 2024-07-31T14:57:45.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.046+0000 7fe87c42c640 1 -- 172.21.15.67:0/2187980096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8741a6890 con 0x7fe8741035e0 2024-07-31T14:57:45.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.047+0000 7fe87c42c640 1 -- 172.21.15.67:0/2187980096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8741039f0 con 0x7fe8741035e0 2024-07-31T14:57:45.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.049+0000 7fe8737fe640 1 -- 172.21.15.67:0/2187980096 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe8640054d0 con 0x7fe8741035e0 2024-07-31T14:57:45.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.050+0000 7fe8737fe640 1 --2- 172.21.15.67:0/2187980096 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe85c0b3d40 0x7fe85c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.050+0000 7fe8799c7640 1 --2- 172.21.15.67:0/2187980096 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe85c0b3d40 0x7fe85c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.050+0000 7fe8737fe640 1 -- 172.21.15.67:0/2187980096 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fe8640e48f0 con 0x7fe8741035e0 2024-07-31T14:57:45.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.051+0000 7fe8799c7640 1 --2- 172.21.15.67:0/2187980096 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe85c0b3d40 0x7fe85c0b6250 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe874108890 tx=0x7fe868005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:45.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.053+0000 7fe8737fe640 1 -- 172.21.15.67:0/2187980096 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe864016030 con 0x7fe8741035e0 2024-07-31T14:57:45.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:44 smithi073 ceph-mon[36714]: pgmap v600: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 306 op/s 2024-07-31T14:57:45.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:44 smithi077 ceph-mon[37327]: pgmap v600: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 306 op/s 2024-07-31T14:57:45.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.288+0000 7fe87c42c640 1 -- 172.21.15.67:0/2187980096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe87419b000 con 0x7fe8741035e0 2024-07-31T14:57:45.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.289+0000 7fe8737fe640 1 -- 172.21.15.67:0/2187980096 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11976 (secure 0 0 0) 0x7fe8640ab2a0 con 0x7fe8741035e0 2024-07-31T14:57:45.291 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:57:45.291 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":48,"btime":"2024-07-31T14:57:42:760027+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:57:41.861865+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:57:45.292 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 48 2024-07-31T14:57:45.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.292+0000 7fe87c42c640 1 -- 172.21.15.67:0/2187980096 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe85c0b3d40 msgr2=0x7fe85c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:45.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.292+0000 7fe87c42c640 1 --2- 172.21.15.67:0/2187980096 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe85c0b3d40 0x7fe85c0b6250 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe874108890 tx=0x7fe868005f90 comp rx=0 tx=0).stop 2024-07-31T14:57:45.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.292+0000 7fe87c42c640 1 -- 172.21.15.67:0/2187980096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8741035e0 msgr2=0x7fe874199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:45.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.292+0000 7fe87c42c640 1 --2- 172.21.15.67:0/2187980096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8741035e0 0x7fe874199ed0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe86400ea80 tx=0x7fe86400ef40 comp rx=0 tx=0).stop 2024-07-31T14:57:45.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.292+0000 7fe87c42c640 1 -- 172.21.15.67:0/2187980096 shutdown_connections 2024-07-31T14:57:45.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.292+0000 7fe87c42c640 1 --2- 172.21.15.67:0/2187980096 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe85c0b3d40 0x7fe85c0b6250 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.292+0000 7fe87c42c640 1 --2- 172.21.15.67:0/2187980096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874109870 0x7fe87419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.292+0000 7fe87c42c640 1 --2- 172.21.15.67:0/2187980096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe874108f20 0x7fe87419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.292+0000 7fe87c42c640 1 --2- 172.21.15.67:0/2187980096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8741035e0 0x7fe874199ed0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.292+0000 7fe87c42c640 1 -- 172.21.15.67:0/2187980096 >> 172.21.15.67:0/2187980096 conn(0x7fe8740ff030 msgr2=0x7fe874101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:45.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.292+0000 7fe87c42c640 1 -- 172.21.15.67:0/2187980096 shutdown_connections 2024-07-31T14:57:45.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:45.292+0000 7fe87c42c640 1 -- 172.21.15.67:0/2187980096 wait complete. 2024-07-31T14:57:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.918+0000 7f60ae77e640 1 -- 172.21.15.67:0/206676992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60a8107d20 msgr2=0x7f60a8108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.918+0000 7f60ae77e640 1 --2- 172.21.15.67:0/206676992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60a8107d20 0x7f60a8108130 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f609c009900 tx=0x7f609c02a410 comp rx=0 tx=0).stop 2024-07-31T14:57:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.919+0000 7f60ae77e640 1 -- 172.21.15.67:0/206676992 shutdown_connections 2024-07-31T14:57:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.919+0000 7f60ae77e640 1 --2- 172.21.15.67:0/206676992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60a8108670 0x7f60a810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.919+0000 7f60ae77e640 1 --2- 172.21.15.67:0/206676992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60a8107d20 0x7f60a8108130 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.919+0000 7f60ae77e640 1 --2- 172.21.15.67:0/206676992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a81023f0 0x7f60a8102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.919+0000 7f60ae77e640 1 -- 172.21.15.67:0/206676992 >> 172.21.15.67:0/206676992 conn(0x7f60a80fddf0 msgr2=0x7f60a8100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.919+0000 7f60ae77e640 1 -- 172.21.15.67:0/206676992 shutdown_connections 2024-07-31T14:57:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.919+0000 7f60ae77e640 1 -- 172.21.15.67:0/206676992 wait complete. 2024-07-31T14:57:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.919+0000 7f60ae77e640 1 Processor -- start 2024-07-31T14:57:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.919+0000 7f60ae77e640 1 -- start start 2024-07-31T14:57:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60ae77e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a81023f0 0x7f60a8198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60ae77e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60a8107d20 0x7f60a8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60ae77e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60a8108670 0x7f60a819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60ae77e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f60a81122e0 con 0x7f60a8107d20 2024-07-31T14:57:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60ae77e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f60a8112160 con 0x7f60a81023f0 2024-07-31T14:57:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60ae77e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f60a8112460 con 0x7f60a8108670 2024-07-31T14:57:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60a77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60a8107d20 0x7f60a8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60a77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60a8107d20 0x7f60a8199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33988/0 (socket says 172.21.15.67:33988) 2024-07-31T14:57:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60a77fe640 1 -- 172.21.15.67:0/2904329245 learned_addr learned my addr 172.21.15.67:0/2904329245 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60a7fff640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a81023f0 0x7f60a8198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60acd1b640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60a8108670 0x7f60a819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60a77fe640 1 -- 172.21.15.67:0/2904329245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60a8108670 msgr2=0x7f60a819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60a77fe640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60a8108670 0x7f60a819e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60a77fe640 1 -- 172.21.15.67:0/2904329245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a81023f0 msgr2=0x7f60a8198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60a77fe640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a81023f0 0x7f60a8198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60a77fe640 1 -- 172.21.15.67:0/2904329245 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a819ecb0 con 0x7f60a8107d20 2024-07-31T14:57:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60acd1b640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60a8108670 0x7f60a819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:57:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.920+0000 7f60a77fe640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60a8107d20 0x7f60a8199320 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f609c006010 tx=0x7f609c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.921+0000 7f60a57fa640 1 -- 172.21.15.67:0/2904329245 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c03f030 con 0x7f60a8107d20 2024-07-31T14:57:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.921+0000 7f60a57fa640 1 -- 172.21.15.67:0/2904329245 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f609c003aa0 con 0x7f60a8107d20 2024-07-31T14:57:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.921+0000 7f60ae77e640 1 -- 172.21.15.67:0/2904329245 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a819ef40 con 0x7f60a8107d20 2024-07-31T14:57:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.921+0000 7f60a57fa640 1 -- 172.21.15.67:0/2904329245 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c035070 con 0x7f60a8107d20 2024-07-31T14:57:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.921+0000 7f60ae77e640 1 -- 172.21.15.67:0/2904329245 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a81a5730 con 0x7f60a8107d20 2024-07-31T14:57:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.923+0000 7f60b5674740 1 Processor -- start 2024-07-31T14:57:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.923+0000 7f60b5674740 1 -- start start 2024-07-31T14:57:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60a57fa640 1 -- 172.21.15.67:0/2904329245 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f609c042030 con 0x7f60a8107d20 2024-07-31T14:57:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60b5674740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 0x561621980ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60b5674740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 0x561621981220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60b5674740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x561621981760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60b5674740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5616218f63b0 con 0x5616215bd4d0 2024-07-31T14:57:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60b5674740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5616218e58b0 con 0x5616215de1b0 2024-07-31T14:57:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60b5674740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5616218e98f0 con 0x5616215b52f0 2024-07-31T14:57:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60a77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x561621981760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60a77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x561621981760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34004/0 (socket says 172.21.15.67:34004) 2024-07-31T14:57:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60a77fe640 1 -- 172.21.15.67:0/15900578 learned_addr learned my addr 172.21.15.67:0/15900578 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60acd1b640 1 --2- 172.21.15.67:0/15900578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 0x561621980ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60a7fff640 1 --2- 172.21.15.67:0/15900578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 0x561621981220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60a77fe640 1 -- 172.21.15.67:0/15900578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 msgr2=0x561621981220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60a77fe640 1 --2- 172.21.15.67:0/15900578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 0x561621981220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60a77fe640 1 -- 172.21.15.67:0/15900578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 msgr2=0x561621980ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60a77fe640 1 --2- 172.21.15.67:0/15900578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 0x561621980ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60a77fe640 1 -- 172.21.15.67:0/15900578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162188c9a0 con 0x5616215bd4d0 2024-07-31T14:57:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.924+0000 7f60a7fff640 1 --2- 172.21.15.67:0/15900578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 0x561621981220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:57:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.925+0000 7f60a77fe640 1 --2- 172.21.15.67:0/15900578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x561621981760 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f609c101b90 tx=0x7f609c10acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cced2079142cb0ab server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.925+0000 7f60a57fa640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f60900b3dc0 0x7f60900b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.925+0000 7f60acd1b640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f60900b3dc0 0x7f60900b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.925+0000 7f608e7fc640 1 -- 172.21.15.67:0/15900578 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c0ca850 con 0x5616215bd4d0 2024-07-31T14:57:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.925+0000 7f608e7fc640 1 -- 172.21.15.67:0/15900578 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f609c114020 con 0x5616215bd4d0 2024-07-31T14:57:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.925+0000 7f608e7fc640 1 -- 172.21.15.67:0/15900578 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c0caa30 con 0x5616215bd4d0 2024-07-31T14:57:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.925+0000 7f60a57fa640 1 -- 172.21.15.67:0/2904329245 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f609c1047e0 con 0x7f60a8107d20 2024-07-31T14:57:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.926+0000 7f60b5674740 1 -- 172.21.15.67:0/15900578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 msgr2=0x561621981760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.926+0000 7f60b5674740 1 --2- 172.21.15.67:0/15900578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x561621981760 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f609c101b90 tx=0x7f609c10acd0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.926+0000 7f60acd1b640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f60900b3dc0 0x7f60900b62d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f60a0005fd0 tx=0x7f60a00098a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.926+0000 7f60b5674740 1 -- 172.21.15.67:0/15900578 shutdown_connections 2024-07-31T14:57:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.926+0000 7f60b5674740 1 --2- 172.21.15.67:0/15900578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x561621981760 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.926+0000 7f60b5674740 1 --2- 172.21.15.67:0/15900578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 0x561621981220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.926+0000 7f60b5674740 1 --2- 172.21.15.67:0/15900578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 0x561621980ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.926+0000 7f60b5674740 1 -- 172.21.15.67:0/15900578 >> 172.21.15.67:0/15900578 conn(0x5616215d2300 msgr2=0x56162197ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.926+0000 7f60b5674740 1 -- 172.21.15.67:0/15900578 shutdown_connections 2024-07-31T14:57:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.926+0000 7f60b5674740 1 -- 172.21.15.67:0/15900578 wait complete. 2024-07-31T14:57:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.926+0000 7f60b5674740 1 -- start start 2024-07-31T14:57:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60b5674740 1 Processor -- start 2024-07-31T14:57:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60b5674740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 0x56162198c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60b5674740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x56162198cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60b5674740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 0x5616219984d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60b5674740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56162181fab0 con 0x5616215bd4d0 2024-07-31T14:57:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60b5674740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56162186dcf0 con 0x5616215de1b0 2024-07-31T14:57:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60b5674740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56162198f220 con 0x5616215b52f0 2024-07-31T14:57:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60b5674740 10 client.0 _ll_register_callbacks cb 0x5616218df9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60a77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x56162198cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60a77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x56162198cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34016/0 (socket says 172.21.15.67:34016) 2024-07-31T14:57:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60a77fe640 1 -- 172.21.15.67:0/1255742460 learned_addr learned my addr 172.21.15.67:0/1255742460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60acd1b640 1 --2- 172.21.15.67:0/1255742460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 0x5616219984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60a7fff640 1 --2- 172.21.15.67:0/1255742460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 0x56162198c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60a77fe640 1 -- 172.21.15.67:0/1255742460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 msgr2=0x56162198c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60a77fe640 1 --2- 172.21.15.67:0/1255742460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 0x56162198c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60a77fe640 1 -- 172.21.15.67:0/1255742460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 msgr2=0x5616219984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60a77fe640 1 --2- 172.21.15.67:0/1255742460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 0x5616219984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.927+0000 7f60a77fe640 1 -- 172.21.15.67:0/1255742460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561621649790 con 0x5616215bd4d0 2024-07-31T14:57:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60a77fe640 1 --2- 172.21.15.67:0/1255742460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x56162198cb20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f609c032af0 tx=0x7f609c03b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febf86e5a56887ee server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60877fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:57:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60877fe640 1 -- 172.21.15.67:0/1255742460 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c113020 con 0x5616215bd4d0 2024-07-31T14:57:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60877fe640 1 -- 172.21.15.67:0/1255742460 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f609c0c7370 con 0x5616215bd4d0 2024-07-31T14:57:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60b5674740 1 -- 172.21.15.67:0/1255742460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f609c114020 con 0x5616215bd4d0 2024-07-31T14:57:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60877fe640 1 -- 172.21.15.67:0/1255742460 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f609c0cac20 con 0x5616215bd4d0 2024-07-31T14:57:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60b5674740 10 client.17265 fetch_fsmap learned FSMap version 48 2024-07-31T14:57:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60b5674740 1 -- 172.21.15.67:0/1255742460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5616216c2620 con 0x5616215bd4d0 2024-07-31T14:57:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60877fe640 1 -- 172.21.15.67:0/1255742460 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f609c114020 con 0x5616215bd4d0 2024-07-31T14:57:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60877fe640 1 -- 172.21.15.67:0/1255742460 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 48) ==== 8414+0+0 (secure 0 0 0) 0x7f609c104470 con 0x5616215bd4d0 2024-07-31T14:57:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60b5674740 10 client.17265 fetch_fsmap finished waiting for FSMap version 48 2024-07-31T14:57:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60b5674740 10 client.17265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:57:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60b5674740 20 client.17265 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60b5674740 1 --2- 172.21.15.67:0/1255742460 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56162159cd80 0x5616219a0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60b5674740 4 client.17265 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:57:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.928+0000 7f60b5674740 1 -- 172.21.15.67:0/1255742460 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616218f8300 con 0x56162159cd80 2024-07-31T14:57:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.929+0000 7f60a7fff640 1 --2- 172.21.15.67:0/1255742460 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56162159cd80 0x5616219a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.929+0000 7f60877fe640 10 client.17265 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.929+0000 7f60a7fff640 1 --2- 172.21.15.67:0/1255742460 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56162159cd80 0x5616219a0540 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfb989e45d349b67 server_cookie=d435cbfe5ef39d3b in_seq=0 out_seq=0 2024-07-31T14:57:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60877fe640 1 -- 172.21.15.67:0/1255742460 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616218f8300 con 0x56162159cd80 2024-07-31T14:57:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60877fe640 10 client.17265 handle_command_reply: tid=0 2024-07-31T14:57:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60b5674740 1 client.17265 shutdown 2024-07-31T14:57:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60b5674740 1 -- 172.21.15.67:0/1255742460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 msgr2=0x56162198cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60b5674740 1 --2- 172.21.15.67:0/1255742460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x56162198cb20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f609c032af0 tx=0x7f609c03b5d0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60b5674740 1 -- 172.21.15.67:0/1255742460 shutdown_connections 2024-07-31T14:57:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60b5674740 1 --2- 172.21.15.67:0/1255742460 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56162159cd80 0x5616219a0540 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60b5674740 1 --2- 172.21.15.67:0/1255742460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 0x5616219984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60b5674740 1 --2- 172.21.15.67:0/1255742460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x56162198cb20 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60b5674740 1 --2- 172.21.15.67:0/1255742460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 0x56162198c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60877fe640 0 client.17265 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60b5674740 1 -- 172.21.15.67:0/1255742460 >> 172.21.15.67:0/1255742460 conn(0x5616215d2300 msgr2=0x561621981d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60b5674740 1 -- 172.21.15.67:0/1255742460 shutdown_connections 2024-07-31T14:57:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60b5674740 1 -- 172.21.15.67:0/1255742460 wait complete. 2024-07-31T14:57:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.930+0000 7f60b5674740 20 client.17265 trim_cache size 0 max 16384 2024-07-31T14:57:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.946+0000 7f60b5674740 1 Processor -- start 2024-07-31T14:57:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.946+0000 7f60b5674740 1 -- start start 2024-07-31T14:57:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.946+0000 7f60b5674740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 0x56162198e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.946+0000 7f60b5674740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x56162197e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.946+0000 7f60b5674740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 0x56162197e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.946+0000 7f60b5674740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f60a81ae480 con 0x5616215bd4d0 2024-07-31T14:57:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.946+0000 7f60b5674740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f60a81b0170 con 0x5616215de1b0 2024-07-31T14:57:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.946+0000 7f60b5674740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f60a81aebe0 con 0x5616215b52f0 2024-07-31T14:57:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.946+0000 7f60a77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x56162197e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.946+0000 7f60a77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x56162197e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34020/0 (socket says 172.21.15.67:34020) 2024-07-31T14:57:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.946+0000 7f60a77fe640 1 -- 172.21.15.67:0/4268529706 learned_addr learned my addr 172.21.15.67:0/4268529706 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.947+0000 7f60acd1b640 1 --2- 172.21.15.67:0/4268529706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 0x56162197e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.947+0000 7f60a7fff640 1 --2- 172.21.15.67:0/4268529706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 0x56162198e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.947+0000 7f60a77fe640 1 -- 172.21.15.67:0/4268529706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 msgr2=0x56162198e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.947+0000 7f60a77fe640 1 --2- 172.21.15.67:0/4268529706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 0x56162198e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.947+0000 7f60a77fe640 1 -- 172.21.15.67:0/4268529706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 msgr2=0x56162197e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.947+0000 7f60a77fe640 1 --2- 172.21.15.67:0/4268529706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 0x56162197e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.947+0000 7f60a77fe640 1 -- 172.21.15.67:0/4268529706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162199aa30 con 0x5616215bd4d0 2024-07-31T14:57:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.947+0000 7f60a7fff640 1 --2- 172.21.15.67:0/4268529706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 0x56162198e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:57:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.947+0000 7f60a77fe640 1 --2- 172.21.15.67:0/4268529706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x56162197e2c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f609c0cf700 tx=0x7f609c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d27ce03cc2100d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.947+0000 7f608dffb640 1 -- 172.21.15.67:0/4268529706 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c113030 con 0x5616215bd4d0 2024-07-31T14:57:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.947+0000 7f608dffb640 1 -- 172.21.15.67:0/4268529706 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f609c0b2030 con 0x5616215bd4d0 2024-07-31T14:57:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.947+0000 7f608dffb640 1 -- 172.21.15.67:0/4268529706 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c049070 con 0x5616215bd4d0 2024-07-31T14:57:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.948+0000 7f60b5674740 1 -- 172.21.15.67:0/4268529706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 msgr2=0x56162197e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.948+0000 7f60b5674740 1 --2- 172.21.15.67:0/4268529706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x56162197e2c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f609c0cf700 tx=0x7f609c000bc0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.948+0000 7f60b5674740 1 -- 172.21.15.67:0/4268529706 shutdown_connections 2024-07-31T14:57:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.948+0000 7f60b5674740 1 --2- 172.21.15.67:0/4268529706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215de1b0 0x56162197e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.948+0000 7f60b5674740 1 --2- 172.21.15.67:0/4268529706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215bd4d0 0x56162197e2c0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.948+0000 7f60b5674740 1 --2- 172.21.15.67:0/4268529706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215b52f0 0x56162198e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.948+0000 7f60b5674740 1 -- 172.21.15.67:0/4268529706 >> 172.21.15.67:0/4268529706 conn(0x5616215d2300 msgr2=0x5616219972f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.948+0000 7f60b5674740 1 -- 172.21.15.67:0/4268529706 shutdown_connections 2024-07-31T14:57:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.948+0000 7f60b5674740 1 -- 172.21.15.67:0/4268529706 wait complete. 2024-07-31T14:57:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.948+0000 7f60b5674740 1 -- start start 2024-07-31T14:57:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60b5674740 1 Processor -- start 2024-07-31T14:57:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60b5674740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215b52f0 0x561621994900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60b5674740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215bd4d0 0x561621990db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60b5674740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215de1b0 0x5616219912f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60b5674740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f60a81afaa0 con 0x5616215b52f0 2024-07-31T14:57:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60b5674740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f60a81af310 con 0x5616215bd4d0 2024-07-31T14:57:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60b5674740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f60a81ac460 con 0x5616215de1b0 2024-07-31T14:57:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60b5674740 10 client.0 _ll_register_callbacks cb 0x7f60a0009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215b52f0 0x561621994900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215b52f0 0x561621994900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34026/0 (socket says 172.21.15.67:34026) 2024-07-31T14:57:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60a7fff640 1 -- 172.21.15.67:0/3819899188 learned_addr learned my addr 172.21.15.67:0/3819899188 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60acd1b640 1 --2- 172.21.15.67:0/3819899188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215de1b0 0x5616219912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60a77fe640 1 --2- 172.21.15.67:0/3819899188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215bd4d0 0x561621990db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60a7fff640 1 -- 172.21.15.67:0/3819899188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215de1b0 msgr2=0x5616219912f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60a7fff640 1 --2- 172.21.15.67:0/3819899188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215de1b0 0x5616219912f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60a7fff640 1 -- 172.21.15.67:0/3819899188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215bd4d0 msgr2=0x561621990db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60a7fff640 1 --2- 172.21.15.67:0/3819899188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215bd4d0 0x561621990db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60a7fff640 1 -- 172.21.15.67:0/3819899188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561621991830 con 0x5616215b52f0 2024-07-31T14:57:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60a7fff640 1 --2- 172.21.15.67:0/3819899188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215b52f0 0x561621994900 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f6098004700 tx=0x7f60980044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd1953449ef5cdd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.949+0000 7f60877fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:57:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.950+0000 7f60877fe640 1 -- 172.21.15.67:0/3819899188 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098027630 con 0x5616215b52f0 2024-07-31T14:57:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.950+0000 7f60877fe640 1 -- 172.21.15.67:0/3819899188 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f60980277d0 con 0x5616215b52f0 2024-07-31T14:57:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.950+0000 7f60b5674740 1 -- 172.21.15.67:0/3819899188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f609c0b2030 con 0x5616215b52f0 2024-07-31T14:57:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.950+0000 7f60877fe640 1 -- 172.21.15.67:0/3819899188 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6098030050 con 0x5616215b52f0 2024-07-31T14:57:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.950+0000 7f60b5674740 10 client.17271 fetch_fsmap learned FSMap version 48 2024-07-31T14:57:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.950+0000 7f60b5674740 1 -- 172.21.15.67:0/3819899188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561621a09e10 con 0x5616215b52f0 2024-07-31T14:57:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.950+0000 7f60877fe640 1 -- 172.21.15.67:0/3819899188 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f609c0b2030 con 0x5616215b52f0 2024-07-31T14:57:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.950+0000 7f60877fe640 1 -- 172.21.15.67:0/3819899188 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 48) ==== 8414+0+0 (secure 0 0 0) 0x7f6098010a40 con 0x5616215b52f0 2024-07-31T14:57:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.950+0000 7f60b5674740 10 client.17271 fetch_fsmap finished waiting for FSMap version 48 2024-07-31T14:57:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.950+0000 7f60b5674740 10 client.17271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:57:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.950+0000 7f60b5674740 20 client.17271 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.950+0000 7f60b5674740 1 --2- 172.21.15.67:0/3819899188 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5616215721c0 0x561621a0c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.950+0000 7f60b5674740 4 client.17271 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:57:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.950+0000 7f60b5674740 1 -- 172.21.15.67:0/3819899188 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5616218f3da0 con 0x5616215721c0 2024-07-31T14:57:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.951+0000 7f60a77fe640 1 --2- 172.21.15.67:0/3819899188 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5616215721c0 0x561621a0c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.951+0000 7f60a77fe640 1 --2- 172.21.15.67:0/3819899188 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5616215721c0 0x561621a0c670 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa935d0bd75f0bb server_cookie=4d85a5724c4cc0b7 in_seq=0 out_seq=0 2024-07-31T14:57:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:45.951+0000 7f60877fe640 10 client.17271 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:45.973 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:45 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2187980096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:57:46.001 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T14:57:46.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:45 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2187980096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:57:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:45 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2187980096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:57:46.276 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:57:47.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:46 smithi067 ceph-mon[32750]: pgmap v601: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 298 op/s 2024-07-31T14:57:47.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:46 smithi073 ceph-mon[36714]: pgmap v601: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 298 op/s 2024-07-31T14:57:47.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:46 smithi077 ceph-mon[37327]: pgmap v601: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 298 op/s 2024-07-31T14:57:48.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.498+0000 7f20b2e60640 1 -- 172.21.15.67:0/4237916743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ac075800 msgr2=0x7f20ac075c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:48.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.498+0000 7f20b2e60640 1 --2- 172.21.15.67:0/4237916743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ac075800 0x7f20ac075c10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2098009900 tx=0x7f209802a410 comp rx=0 tx=0).stop 2024-07-31T14:57:48.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.498+0000 7f20b2e60640 1 -- 172.21.15.67:0/4237916743 shutdown_connections 2024-07-31T14:57:48.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.498+0000 7f20b2e60640 1 --2- 172.21.15.67:0/4237916743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20ac113e90 0x7f20ac116360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.498+0000 7f20b2e60640 1 --2- 172.21.15.67:0/4237916743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ac075800 0x7f20ac075c10 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.498+0000 7f20b2e60640 1 --2- 172.21.15.67:0/4237916743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20ac076db0 0x7f20ac0751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.498+0000 7f20b2e60640 1 -- 172.21.15.67:0/4237916743 >> 172.21.15.67:0/4237916743 conn(0x7f20ac0ff030 msgr2=0x7f20ac101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:48.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.498+0000 7f20b2e60640 1 -- 172.21.15.67:0/4237916743 shutdown_connections 2024-07-31T14:57:48.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.498+0000 7f20b2e60640 1 -- 172.21.15.67:0/4237916743 wait complete. 2024-07-31T14:57:48.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b2e60640 1 Processor -- start 2024-07-31T14:57:48.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b2e60640 1 -- start start 2024-07-31T14:57:48.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b2e60640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ac075800 0x7f20ac1a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b2e60640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20ac076db0 0x7f20ac1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b2e60640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20ac113e90 0x7f20ac1a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b2e60640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac119a90 con 0x7f20ac075800 2024-07-31T14:57:48.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b2e60640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac119910 con 0x7f20ac113e90 2024-07-31T14:57:48.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b2e60640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac119c10 con 0x7f20ac076db0 2024-07-31T14:57:48.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b0bfc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ac075800 0x7f20ac1a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b0bfc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ac075800 0x7f20ac1a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34044/0 (socket says 172.21.15.67:34044) 2024-07-31T14:57:48.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b0bfc640 1 -- 172.21.15.67:0/4193146991 learned_addr learned my addr 172.21.15.67:0/4193146991 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:48.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20abfff640 1 --2- 172.21.15.67:0/4193146991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20ac076db0 0x7f20ac1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b13fd640 1 --2- 172.21.15.67:0/4193146991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20ac113e90 0x7f20ac1a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b0bfc640 1 -- 172.21.15.67:0/4193146991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20ac076db0 msgr2=0x7f20ac1a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:48.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b0bfc640 1 --2- 172.21.15.67:0/4193146991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20ac076db0 0x7f20ac1a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b0bfc640 1 -- 172.21.15.67:0/4193146991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20ac113e90 msgr2=0x7f20ac1a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:48.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b0bfc640 1 --2- 172.21.15.67:0/4193146991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20ac113e90 0x7f20ac1a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.499+0000 7f20b0bfc640 1 -- 172.21.15.67:0/4193146991 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ac1a6410 con 0x7f20ac075800 2024-07-31T14:57:48.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.500+0000 7f20b0bfc640 1 --2- 172.21.15.67:0/4193146991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ac075800 0x7f20ac1a04a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f20a000e9f0 tx=0x7f20a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.500+0000 7f20a9ffb640 1 -- 172.21.15.67:0/4193146991 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a000cd80 con 0x7f20ac075800 2024-07-31T14:57:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.500+0000 7f20a9ffb640 1 -- 172.21.15.67:0/4193146991 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20a00108e0 con 0x7f20ac075800 2024-07-31T14:57:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.500+0000 7f20a9ffb640 1 -- 172.21.15.67:0/4193146991 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0018980 con 0x7f20ac075800 2024-07-31T14:57:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.500+0000 7f20b2e60640 1 -- 172.21.15.67:0/4193146991 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ac1a6700 con 0x7f20ac075800 2024-07-31T14:57:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.500+0000 7f20b2e60640 1 -- 172.21.15.67:0/4193146991 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ac1ace60 con 0x7f20ac075800 2024-07-31T14:57:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.503+0000 7f20b93b1740 1 Processor -- start 2024-07-31T14:57:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.503+0000 7f20a9ffb640 1 -- 172.21.15.67:0/4193146991 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f20a0010430 con 0x7f20ac075800 2024-07-31T14:57:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.503+0000 7f20b93b1740 1 -- start start 2024-07-31T14:57:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.503+0000 7f20b93b1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf85070 0x5577fd32a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.503+0000 7f20b93b1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 0x5577fd32aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.503+0000 7f20b93b1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf64390 0x5577fd32aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.503+0000 7f20b93b1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5577fd2a0bb0 con 0x5577fcf64390 2024-07-31T14:57:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.503+0000 7f20b93b1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5577fd2900b0 con 0x5577fcf85070 2024-07-31T14:57:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.503+0000 7f20b93b1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5577fd2940f0 con 0x5577fcf5c1b0 2024-07-31T14:57:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.503+0000 7f20abfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf64390 0x5577fd32aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.503+0000 7f20abfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf64390 0x5577fd32aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34050/0 (socket says 172.21.15.67:34050) 2024-07-31T14:57:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.503+0000 7f20abfff640 1 -- 172.21.15.67:0/2631243447 learned_addr learned my addr 172.21.15.67:0/2631243447 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.503+0000 7f20b13fd640 1 --2- 172.21.15.67:0/2631243447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf85070 0x5577fd32a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.503+0000 7f20b0bfc640 1 --2- 172.21.15.67:0/2631243447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 0x5577fd32aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.504+0000 7f20abfff640 1 -- 172.21.15.67:0/2631243447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 msgr2=0x5577fd32aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.504+0000 7f20abfff640 1 --2- 172.21.15.67:0/2631243447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 0x5577fd32aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.504+0000 7f20abfff640 1 -- 172.21.15.67:0/2631243447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf85070 msgr2=0x5577fd32a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.504+0000 7f20abfff640 1 --2- 172.21.15.67:0/2631243447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf85070 0x5577fd32a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.504+0000 7f20abfff640 1 -- 172.21.15.67:0/2631243447 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577fd236b90 con 0x5577fcf64390 2024-07-31T14:57:48.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.504+0000 7f20b0bfc640 1 --2- 172.21.15.67:0/2631243447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 0x5577fd32aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:57:48.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.504+0000 7f20a9ffb640 1 --2- 172.21.15.67:0/4193146991 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f209c0b3dc0 0x7f209c0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.504+0000 7f20abfff640 1 --2- 172.21.15.67:0/2631243447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf64390 0x5577fd32aff0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2098009900 tx=0x7f2098032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88396040bd02333 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:48.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.504+0000 7f20b13fd640 1 --2- 172.21.15.67:0/4193146991 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f209c0b3dc0 0x7f209c0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.504+0000 7f20a9ffb640 1 -- 172.21.15.67:0/4193146991 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f20a00e45b0 con 0x7f20ac075800 2024-07-31T14:57:48.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.504+0000 7f2096ffd640 1 -- 172.21.15.67:0/2631243447 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098040070 con 0x5577fcf64390 2024-07-31T14:57:48.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.504+0000 7f2096ffd640 1 -- 172.21.15.67:0/2631243447 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2098004030 con 0x5577fcf64390 2024-07-31T14:57:48.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.504+0000 7f2096ffd640 1 -- 172.21.15.67:0/2631243447 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098035040 con 0x5577fcf64390 2024-07-31T14:57:48.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.505+0000 7f20b93b1740 1 -- 172.21.15.67:0/2631243447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf64390 msgr2=0x5577fd32aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:48.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.505+0000 7f20b93b1740 1 --2- 172.21.15.67:0/2631243447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf64390 0x5577fd32aff0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2098009900 tx=0x7f2098032b20 comp rx=0 tx=0).stop 2024-07-31T14:57:48.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.505+0000 7f20b13fd640 1 --2- 172.21.15.67:0/4193146991 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f209c0b3dc0 0x7f209c0b62d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f20a4005fd0 tx=0x7f20a4009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:48.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.505+0000 7f20b93b1740 1 -- 172.21.15.67:0/2631243447 shutdown_connections 2024-07-31T14:57:48.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.505+0000 7f20b93b1740 1 --2- 172.21.15.67:0/2631243447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf64390 0x5577fd32aff0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.505+0000 7f20b93b1740 1 --2- 172.21.15.67:0/2631243447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 0x5577fd32aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.505+0000 7f20b93b1740 1 --2- 172.21.15.67:0/2631243447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf85070 0x5577fd32a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.505+0000 7f20b93b1740 1 -- 172.21.15.67:0/2631243447 >> 172.21.15.67:0/2631243447 conn(0x5577fcf79460 msgr2=0x5577fd328460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:48.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.505+0000 7f20b93b1740 1 -- 172.21.15.67:0/2631243447 shutdown_connections 2024-07-31T14:57:48.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.505+0000 7f20b93b1740 1 -- 172.21.15.67:0/2631243447 wait complete. 2024-07-31T14:57:48.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.506+0000 7f20b93b1740 1 -- start start 2024-07-31T14:57:48.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.506+0000 7f20b93b1740 1 Processor -- start 2024-07-31T14:57:48.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.506+0000 7f20b93b1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 0x5577fd336100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.506+0000 7f20b93b1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf64390 0x5577fd336640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.506+0000 7f20b93b1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf85070 0x5577fd342010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.506+0000 7f20b93b1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5577fd0f3700 con 0x5577fcf85070 2024-07-31T14:57:48.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.506+0000 7f20b93b1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5577fd2329e0 con 0x5577fcf64390 2024-07-31T14:57:48.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.506+0000 7f20b93b1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5577fd338cf0 con 0x5577fcf5c1b0 2024-07-31T14:57:48.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.506+0000 7f20b93b1740 10 client.0 _ll_register_callbacks cb 0x5577fd24d4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:48.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.506+0000 7f20b13fd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf85070 0x5577fd342010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.506+0000 7f20b13fd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf85070 0x5577fd342010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34054/0 (socket says 172.21.15.67:34054) 2024-07-31T14:57:48.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.506+0000 7f20b13fd640 1 -- 172.21.15.67:0/2646443513 learned_addr learned my addr 172.21.15.67:0/2646443513 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:48.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.507+0000 7f20b0bfc640 1 --2- 172.21.15.67:0/2646443513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 0x5577fd336100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.507+0000 7f20b13fd640 1 -- 172.21.15.67:0/2646443513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 msgr2=0x5577fd336100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:48.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.507+0000 7f20b13fd640 1 --2- 172.21.15.67:0/2646443513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 0x5577fd336100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.507+0000 7f20b13fd640 1 -- 172.21.15.67:0/2646443513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf64390 msgr2=0x5577fd336640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:48.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.507+0000 7f20b13fd640 1 --2- 172.21.15.67:0/2646443513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf64390 0x5577fd336640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.507+0000 7f20b13fd640 1 -- 172.21.15.67:0/2646443513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577fcff0650 con 0x5577fcf85070 2024-07-31T14:57:48.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.507+0000 7f20b13fd640 1 --2- 172.21.15.67:0/2646443513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf85070 0x5577fd342010 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f20a4003fd0 tx=0x7f20a4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bdcee0bdbdb9b4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:48.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.507+0000 7f208bfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:57:48.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.507+0000 7f208bfff640 1 -- 172.21.15.67:0/2646443513 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a400f8b0 con 0x5577fcf85070 2024-07-31T14:57:48.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.507+0000 7f208bfff640 1 -- 172.21.15.67:0/2646443513 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f20a400fa50 con 0x5577fcf85070 2024-07-31T14:57:48.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.507+0000 7f20b93b1740 1 -- 172.21.15.67:0/2646443513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2098004030 con 0x5577fcf85070 2024-07-31T14:57:48.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.507+0000 7f208bfff640 1 -- 172.21.15.67:0/2646443513 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f20a400fc50 con 0x5577fcf85070 2024-07-31T14:57:48.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.507+0000 7f20b93b1740 10 client.17283 fetch_fsmap learned FSMap version 49 2024-07-31T14:57:48.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.507+0000 7f20b93b1740 1 -- 172.21.15.67:0/2646443513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5577fd344d60 con 0x5577fcf85070 2024-07-31T14:57:48.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.508+0000 7f208bfff640 1 -- 172.21.15.67:0/2646443513 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2098004030 con 0x5577fcf85070 2024-07-31T14:57:48.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.508+0000 7f208bfff640 1 -- 172.21.15.67:0/2646443513 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f20a4026070 con 0x5577fcf85070 2024-07-31T14:57:48.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.508+0000 7f20b93b1740 10 client.17283 fetch_fsmap finished waiting for FSMap version 49 2024-07-31T14:57:48.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.508+0000 7f20b93b1740 10 client.17283 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:57:48.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.508+0000 7f20b93b1740 20 client.17283 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:48.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.508+0000 7f20b93b1740 1 --2- 172.21.15.67:0/2646443513 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5577fcf439f0 0x5577fd348360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.508+0000 7f20b93b1740 4 client.17283 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:57:48.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.508+0000 7f20b93b1740 1 -- 172.21.15.67:0/2646443513 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577fd2a2b00 con 0x5577fcf439f0 2024-07-31T14:57:48.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.508+0000 7f20b0bfc640 1 --2- 172.21.15.67:0/2646443513 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5577fcf439f0 0x5577fd348360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.509+0000 7f20b0bfc640 1 --2- 172.21.15.67:0/2646443513 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5577fcf439f0 0x5577fd348360 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ae9f50138009e072 server_cookie=29c7fa32e8d719e6 in_seq=0 out_seq=0 2024-07-31T14:57:48.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.509+0000 7f208bfff640 10 client.17283 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:57:48.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f208bfff640 1 -- 172.21.15.67:0/2646443513 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577fd2a2b00 con 0x5577fcf439f0 2024-07-31T14:57:48.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f208bfff640 10 client.17283 handle_command_reply: tid=0 2024-07-31T14:57:48.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f20b93b1740 1 client.17283 shutdown 2024-07-31T14:57:48.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f20b93b1740 1 -- 172.21.15.67:0/2646443513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf85070 msgr2=0x5577fd342010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:48.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f20b93b1740 1 --2- 172.21.15.67:0/2646443513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf85070 0x5577fd342010 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f20a4003fd0 tx=0x7f20a4004ab0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f20b93b1740 1 -- 172.21.15.67:0/2646443513 shutdown_connections 2024-07-31T14:57:48.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f20b93b1740 1 --2- 172.21.15.67:0/2646443513 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5577fcf439f0 0x5577fd348360 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f20b93b1740 1 --2- 172.21.15.67:0/2646443513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf85070 0x5577fd342010 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f20b93b1740 1 --2- 172.21.15.67:0/2646443513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf64390 0x5577fd336640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f20b93b1740 1 --2- 172.21.15.67:0/2646443513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 0x5577fd336100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f208bfff640 0 client.17283 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:57:48.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f20b93b1740 1 -- 172.21.15.67:0/2646443513 >> 172.21.15.67:0/2646443513 conn(0x5577fcf79460 msgr2=0x5577fd326c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:48.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f20b93b1740 1 -- 172.21.15.67:0/2646443513 shutdown_connections 2024-07-31T14:57:48.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f20b93b1740 1 -- 172.21.15.67:0/2646443513 wait complete. 2024-07-31T14:57:48.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.510+0000 7f20b93b1740 20 client.17283 trim_cache size 0 max 16384 2024-07-31T14:57:48.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.527+0000 7f20b93b1740 1 Processor -- start 2024-07-31T14:57:48.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.531+0000 7f20b93b1740 1 -- start start 2024-07-31T14:57:48.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.531+0000 7f20b93b1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 0x5577fd32fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.531+0000 7f20b93b1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf64390 0x5577fd327e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.531+0000 7f20b93b1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf85070 0x5577fd328390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.531+0000 7f20b93b1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1b6ce0 con 0x5577fcf64390 2024-07-31T14:57:48.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.531+0000 7f20b93b1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1b8970 con 0x5577fcf85070 2024-07-31T14:57:48.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.531+0000 7f20b93b1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1b7440 con 0x5577fcf5c1b0 2024-07-31T14:57:48.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.532+0000 7f20abfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf64390 0x5577fd327e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.532+0000 7f20b13fd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf85070 0x5577fd328390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.532+0000 7f20b13fd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf85070 0x5577fd328390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:38624/0 (socket says 172.21.15.67:38624) 2024-07-31T14:57:48.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.532+0000 7f20b13fd640 1 -- 172.21.15.67:0/1670073720 learned_addr learned my addr 172.21.15.67:0/1670073720 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:48.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.532+0000 7f20b13fd640 1 -- 172.21.15.67:0/1670073720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 msgr2=0x5577fd32fd90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:57:48.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.532+0000 7f20b13fd640 1 --2- 172.21.15.67:0/1670073720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 0x5577fd32fd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.532+0000 7f20b13fd640 1 -- 172.21.15.67:0/1670073720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf64390 msgr2=0x5577fd327e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:48.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.532+0000 7f20b13fd640 1 --2- 172.21.15.67:0/1670073720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf64390 0x5577fd327e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.532+0000 7f20b13fd640 1 -- 172.21.15.67:0/1670073720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577fd32d3d0 con 0x5577fcf85070 2024-07-31T14:57:48.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.534+0000 7f20b13fd640 1 --2- 172.21.15.67:0/1670073720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf85070 0x5577fd328390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20a40284c0 tx=0x7f20a4009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c12011a93c6ea51 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:48.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.534+0000 7f20967fc640 1 -- 172.21.15.67:0/1670073720 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a400fdf0 con 0x5577fcf85070 2024-07-31T14:57:48.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.534+0000 7f20967fc640 1 -- 172.21.15.67:0/1670073720 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20a400f8b0 con 0x5577fcf85070 2024-07-31T14:57:48.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.534+0000 7f20967fc640 1 -- 172.21.15.67:0/1670073720 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a401d920 con 0x5577fcf85070 2024-07-31T14:57:48.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.536+0000 7f20b93b1740 1 -- 172.21.15.67:0/1670073720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf85070 msgr2=0x5577fd328390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:48.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.536+0000 7f20b93b1740 1 --2- 172.21.15.67:0/1670073720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf85070 0x5577fd328390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20a40284c0 tx=0x7f20a4009610 comp rx=0 tx=0).stop 2024-07-31T14:57:48.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.536+0000 7f20b93b1740 1 -- 172.21.15.67:0/1670073720 shutdown_connections 2024-07-31T14:57:48.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.536+0000 7f20b93b1740 1 --2- 172.21.15.67:0/1670073720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf85070 0x5577fd328390 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.536+0000 7f20b93b1740 1 --2- 172.21.15.67:0/1670073720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf64390 0x5577fd327e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.536+0000 7f20b93b1740 1 --2- 172.21.15.67:0/1670073720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf5c1b0 0x5577fd32fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.536+0000 7f20b93b1740 1 -- 172.21.15.67:0/1670073720 >> 172.21.15.67:0/1670073720 conn(0x5577fcf79460 msgr2=0x5577fd32f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:48.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.548+0000 7f20b93b1740 1 -- 172.21.15.67:0/1670073720 shutdown_connections 2024-07-31T14:57:48.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.548+0000 7f20b93b1740 1 -- 172.21.15.67:0/1670073720 wait complete. 2024-07-31T14:57:48.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.549+0000 7f20b93b1740 1 -- start start 2024-07-31T14:57:48.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.549+0000 7f20b93b1740 1 Processor -- start 2024-07-31T14:57:48.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.549+0000 7f20b93b1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf5c1b0 0x5577fd33a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.550+0000 7f20b93b1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf64390 0x5577fd33a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.550+0000 7f20b93b1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf85070 0x5577fd33ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.550+0000 7f20b93b1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1b82a0 con 0x5577fcf5c1b0 2024-07-31T14:57:48.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.550+0000 7f20b93b1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1b7b10 con 0x5577fcf64390 2024-07-31T14:57:48.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.550+0000 7f20b93b1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1b4cc0 con 0x5577fcf85070 2024-07-31T14:57:48.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.550+0000 7f20abfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf64390 0x5577fd33a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.550+0000 7f20abfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf64390 0x5577fd33a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:38636/0 (socket says 172.21.15.67:38636) 2024-07-31T14:57:48.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.550+0000 7f20abfff640 1 -- 172.21.15.67:0/4200430514 learned_addr learned my addr 172.21.15.67:0/4200430514 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:48.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.550+0000 7f20b93b1740 10 client.0 _ll_register_callbacks cb 0x7f20a4009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:48.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.551+0000 7f20b0bfc640 1 --2- 172.21.15.67:0/4200430514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf5c1b0 0x5577fd33a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.552+0000 7f20b0bfc640 1 -- 172.21.15.67:0/4200430514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf85070 msgr2=0x5577fd33ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:48.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.552+0000 7f20b0bfc640 1 --2- 172.21.15.67:0/4200430514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf85070 0x5577fd33ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.552+0000 7f20b0bfc640 1 -- 172.21.15.67:0/4200430514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf64390 msgr2=0x5577fd33a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:48.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.552+0000 7f20b0bfc640 1 --2- 172.21.15.67:0/4200430514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf64390 0x5577fd33a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.552+0000 7f20b0bfc640 1 -- 172.21.15.67:0/4200430514 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577fd3afd40 con 0x5577fcf5c1b0 2024-07-31T14:57:48.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.555+0000 7f20b0bfc640 1 --2- 172.21.15.67:0/4200430514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf5c1b0 0x5577fd33a070 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f20a00e19f0 tx=0x7f20a000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c734b43b8d9f9ae server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:48.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.557+0000 7f208bfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:57:48.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.557+0000 7f208bfff640 1 -- 172.21.15.67:0/4200430514 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a00af4b0 con 0x5577fcf5c1b0 2024-07-31T14:57:48.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.557+0000 7f20b93b1740 1 -- 172.21.15.67:0/4200430514 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20a400f8b0 con 0x5577fcf5c1b0 2024-07-31T14:57:48.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.557+0000 7f208bfff640 1 -- 172.21.15.67:0/4200430514 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f20a00e1c70 con 0x5577fcf5c1b0 2024-07-31T14:57:48.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.563+0000 7f208bfff640 1 -- 172.21.15.67:0/4200430514 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f20a00b3f60 con 0x5577fcf5c1b0 2024-07-31T14:57:48.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.563+0000 7f20b93b1740 10 client.17289 fetch_fsmap learned FSMap version 49 2024-07-31T14:57:48.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.563+0000 7f20b93b1740 1 -- 172.21.15.67:0/4200430514 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5577fd3b1000 con 0x5577fcf5c1b0 2024-07-31T14:57:48.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.565+0000 7f208bfff640 1 -- 172.21.15.67:0/4200430514 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20a400f8b0 con 0x5577fcf5c1b0 2024-07-31T14:57:48.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.565+0000 7f208bfff640 1 -- 172.21.15.67:0/4200430514 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f20a00c0080 con 0x5577fcf5c1b0 2024-07-31T14:57:48.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.565+0000 7f20b93b1740 10 client.17289 fetch_fsmap finished waiting for FSMap version 49 2024-07-31T14:57:48.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.565+0000 7f20b93b1740 10 client.17289 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:57:48.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.565+0000 7f20b93b1740 20 client.17289 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:48.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.565+0000 7f20b93b1740 1 --2- 172.21.15.67:0/4200430514 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5577fcf18da0 0x5577fd3b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:48.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.565+0000 7f20b93b1740 4 client.17289 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:57:48.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.565+0000 7f20b93b1740 1 -- 172.21.15.67:0/4200430514 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577fd29e5a0 con 0x5577fcf18da0 2024-07-31T14:57:48.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.566+0000 7f20abfff640 1 --2- 172.21.15.67:0/4200430514 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5577fcf18da0 0x5577fd3b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:48.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.567+0000 7f20abfff640 1 --2- 172.21.15.67:0/4200430514 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5577fcf18da0 0x5577fd3b38c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dc85f3aaca6c631f server_cookie=415bf60c737e8d1b in_seq=0 out_seq=0 2024-07-31T14:57:48.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.569+0000 7f208bfff640 10 client.17289 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:57:48.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.569+0000 7f208bfff640 1 -- 172.21.15.67:0/4200430514 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577fd29e5a0 con 0x5577fcf18da0 2024-07-31T14:57:48.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.569+0000 7f208bfff640 10 client.17289 handle_command_reply: tid=0 2024-07-31T14:57:48.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.569+0000 7f20b93b1740 1 client.17289 shutdown 2024-07-31T14:57:48.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.573+0000 7f20b93b1740 1 -- 172.21.15.67:0/4200430514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf5c1b0 msgr2=0x5577fd33a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:48.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.573+0000 7f20b93b1740 1 --2- 172.21.15.67:0/4200430514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf5c1b0 0x5577fd33a070 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f20a00e19f0 tx=0x7f20a000cf20 comp rx=0 tx=0).stop 2024-07-31T14:57:48.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.574+0000 7f20b93b1740 1 -- 172.21.15.67:0/4200430514 shutdown_connections 2024-07-31T14:57:48.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.574+0000 7f20b93b1740 1 --2- 172.21.15.67:0/4200430514 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5577fcf18da0 0x5577fd3b38c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.574+0000 7f20b93b1740 1 --2- 172.21.15.67:0/4200430514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577fcf85070 0x5577fd33ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.574+0000 7f20b93b1740 1 --2- 172.21.15.67:0/4200430514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577fcf64390 0x5577fd33a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.574+0000 7f20b93b1740 1 --2- 172.21.15.67:0/4200430514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577fcf5c1b0 0x5577fd33a070 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.574+0000 7f208bfff640 0 client.17289 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:57:48.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.574+0000 7f20b93b1740 1 -- 172.21.15.67:0/4200430514 >> 172.21.15.67:0/4200430514 conn(0x5577fcf79460 msgr2=0x5577fd327cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:48.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.575+0000 7f20b93b1740 1 -- 172.21.15.67:0/4200430514 shutdown_connections 2024-07-31T14:57:48.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.579+0000 7f20b93b1740 1 -- 172.21.15.67:0/4200430514 wait complete. 2024-07-31T14:57:48.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.583+0000 7f20b93b1740 20 client.17289 trim_cache size 0 max 16384 2024-07-31T14:57:48.712 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:57:48.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.589+0000 7f20b2e60640 1 -- 172.21.15.67:0/4193146991 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f209c0b3dc0 msgr2=0x7f209c0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:48.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.589+0000 7f20b2e60640 1 --2- 172.21.15.67:0/4193146991 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f209c0b3dc0 0x7f209c0b62d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f20a4005fd0 tx=0x7f20a4009500 comp rx=0 tx=0).stop 2024-07-31T14:57:48.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.589+0000 7f20b2e60640 1 -- 172.21.15.67:0/4193146991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ac075800 msgr2=0x7f20ac1a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:48.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.589+0000 7f20b2e60640 1 --2- 172.21.15.67:0/4193146991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ac075800 0x7f20ac1a04a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f20a000e9f0 tx=0x7f20a000eeb0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.594+0000 7f20b2e60640 1 -- 172.21.15.67:0/4193146991 shutdown_connections 2024-07-31T14:57:48.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.594+0000 7f20b2e60640 1 --2- 172.21.15.67:0/4193146991 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f209c0b3dc0 0x7f209c0b62d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.594+0000 7f20b2e60640 1 --2- 172.21.15.67:0/4193146991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20ac113e90 0x7f20ac1a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.594+0000 7f20b2e60640 1 --2- 172.21.15.67:0/4193146991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20ac076db0 0x7f20ac1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.594+0000 7f20b2e60640 1 --2- 172.21.15.67:0/4193146991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ac075800 0x7f20ac1a04a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:48.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.594+0000 7f20b2e60640 1 -- 172.21.15.67:0/4193146991 >> 172.21.15.67:0/4193146991 conn(0x7f20ac0ff030 msgr2=0x7f20ac10fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:48.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.594+0000 7f20b2e60640 1 -- 172.21.15.67:0/4193146991 shutdown_connections 2024-07-31T14:57:48.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:48.594+0000 7f20b2e60640 1 -- 172.21.15.67:0/4193146991 wait complete. 2024-07-31T14:57:48.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:48 smithi067 ceph-mon[32750]: pgmap v602: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 374 op/s 2024-07-31T14:57:48.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:48 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:49.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:48 smithi073 ceph-mon[36714]: pgmap v602: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 374 op/s 2024-07-31T14:57:49.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:48 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:49.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:48 smithi077 ceph-mon[37327]: pgmap v602: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 374 op/s 2024-07-31T14:57:49.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:48 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:49.518 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T14:57:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:57:50.138 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:49 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:50.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:49 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:50.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:49 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:57:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:57:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:57:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:57:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:57:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.789+0000 7f60877fe640 1 -- 172.21.15.67:0/3819899188 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5616218f3da0 con 0x5616215721c0 2024-07-31T14:57:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.789+0000 7f60877fe640 10 client.17271 handle_command_reply: tid=0 2024-07-31T14:57:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.789+0000 7f60b5674740 1 client.17271 shutdown 2024-07-31T14:57:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.790+0000 7f60b5674740 1 -- 172.21.15.67:0/3819899188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215b52f0 msgr2=0x561621994900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.790+0000 7f60b5674740 1 --2- 172.21.15.67:0/3819899188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215b52f0 0x561621994900 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f6098004700 tx=0x7f60980044a0 comp rx=0 tx=0).stop 2024-07-31T14:57:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.790+0000 7f60b5674740 1 -- 172.21.15.67:0/3819899188 shutdown_connections 2024-07-31T14:57:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.790+0000 7f60b5674740 1 --2- 172.21.15.67:0/3819899188 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5616215721c0 0x561621a0c670 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.790+0000 7f60b5674740 1 --2- 172.21.15.67:0/3819899188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616215de1b0 0x5616219912f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.790+0000 7f60b5674740 1 --2- 172.21.15.67:0/3819899188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616215bd4d0 0x561621990db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.790+0000 7f60b5674740 1 --2- 172.21.15.67:0/3819899188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616215b52f0 0x561621994900 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.790+0000 7f60b5674740 1 -- 172.21.15.67:0/3819899188 >> 172.21.15.67:0/3819899188 conn(0x5616215d2300 msgr2=0x561621999300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.790+0000 7f60877fe640 0 client.17271 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:57:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.790+0000 7f60b5674740 1 -- 172.21.15.67:0/3819899188 shutdown_connections 2024-07-31T14:57:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.790+0000 7f60b5674740 1 -- 172.21.15.67:0/3819899188 wait complete. 2024-07-31T14:57:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.790+0000 7f60b5674740 20 client.17271 trim_cache size 0 max 16384 2024-07-31T14:57:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.791+0000 7f60ae77e640 1 -- 172.21.15.67:0/2904329245 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f60900b3dc0 msgr2=0x7f60900b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.791+0000 7f60ae77e640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f60900b3dc0 0x7f60900b62d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f60a0005fd0 tx=0x7f60a00098a0 comp rx=0 tx=0).stop 2024-07-31T14:57:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.791+0000 7f60ae77e640 1 -- 172.21.15.67:0/2904329245 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60a8107d20 msgr2=0x7f60a8199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.791+0000 7f60ae77e640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60a8107d20 0x7f60a8199320 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f609c006010 tx=0x7f609c004060 comp rx=0 tx=0).stop 2024-07-31T14:57:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.791+0000 7f60ae77e640 1 -- 172.21.15.67:0/2904329245 shutdown_connections 2024-07-31T14:57:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.791+0000 7f60ae77e640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f60900b3dc0 0x7f60900b62d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.791+0000 7f60ae77e640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60a8108670 0x7f60a819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.791+0000 7f60ae77e640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60a8107d20 0x7f60a8199320 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.791+0000 7f60ae77e640 1 --2- 172.21.15.67:0/2904329245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a81023f0 0x7f60a8198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.791+0000 7f60ae77e640 1 -- 172.21.15.67:0/2904329245 >> 172.21.15.67:0/2904329245 conn(0x7f60a80fddf0 msgr2=0x7f60a8100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.791+0000 7f60ae77e640 1 -- 172.21.15.67:0/2904329245 shutdown_connections 2024-07-31T14:57:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:57:50.791+0000 7f60ae77e640 1 -- 172.21.15.67:0/2904329245 wait complete. 2024-07-31T14:57:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:50 smithi067 ceph-mon[32750]: pgmap v603: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 781 op/s 2024-07-31T14:57:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:57:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:57:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:57:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:57:51.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:50 smithi073 ceph-mon[36714]: pgmap v603: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 781 op/s 2024-07-31T14:57:51.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:57:51.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:57:51.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:57:51.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:57:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:50 smithi077 ceph-mon[37327]: pgmap v603: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 781 op/s 2024-07-31T14:57:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:57:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:57:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:57:51.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:57:51.469 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-31T14:57:51.469 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-07-31T14:57:51.469 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:57:52.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.276+0000 7f988d19d640 1 -- 172.21.15.67:0/1035501317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9888102760 msgr2=0x7f988810f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:52.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.276+0000 7f988d19d640 1 --2- 172.21.15.67:0/1035501317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9888102760 0x7f988810f480 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9870009900 tx=0x7f987002a480 comp rx=0 tx=0).stop 2024-07-31T14:57:52.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.276+0000 7f988d19d640 1 -- 172.21.15.67:0/1035501317 shutdown_connections 2024-07-31T14:57:52.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.276+0000 7f988d19d640 1 --2- 172.21.15.67:0/1035501317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f988810f9c0 0x7f9888111de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.276+0000 7f988d19d640 1 --2- 172.21.15.67:0/1035501317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9888102760 0x7f988810f480 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.276+0000 7f988d19d640 1 --2- 172.21.15.67:0/1035501317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9888101e10 0x7f9888102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.276+0000 7f988d19d640 1 -- 172.21.15.67:0/1035501317 >> 172.21.15.67:0/1035501317 conn(0x7f98880fdcf0 msgr2=0x7f9888100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:52.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.276+0000 7f988d19d640 1 -- 172.21.15.67:0/1035501317 shutdown_connections 2024-07-31T14:57:52.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.276+0000 7f988d19d640 1 -- 172.21.15.67:0/1035501317 wait complete. 2024-07-31T14:57:52.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.277+0000 7f988d19d640 1 Processor -- start 2024-07-31T14:57:52.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.277+0000 7f988d19d640 1 -- start start 2024-07-31T14:57:52.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.277+0000 7f988d19d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9888101e10 0x7f988819bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.277+0000 7f988d19d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9888102760 0x7f988819c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.277+0000 7f988d19d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f988810f9c0 0x7f98881a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.277+0000 7f988d19d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f98881154a0 con 0x7f9888102760 2024-07-31T14:57:52.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.277+0000 7f988d19d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9888115320 con 0x7f9888101e10 2024-07-31T14:57:52.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.277+0000 7f988d19d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9888115620 con 0x7f988810f9c0 2024-07-31T14:57:52.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.277+0000 7f987ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9888102760 0x7f988819c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.277+0000 7f9887fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9888101e10 0x7f988819bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.277+0000 7f987ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9888102760 0x7f988819c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55764/0 (socket says 172.21.15.67:55764) 2024-07-31T14:57:52.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.277+0000 7f9887fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9888101e10 0x7f988819bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60636/0 (socket says 172.21.15.67:60636) 2024-07-31T14:57:52.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.277+0000 7f987ffff640 1 -- 172.21.15.67:0/2143940896 learned_addr learned my addr 172.21.15.67:0/2143940896 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:52.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.278+0000 7f988c99c640 1 --2- 172.21.15.67:0/2143940896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f988810f9c0 0x7f98881a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.278+0000 7f9887fff640 1 -- 172.21.15.67:0/2143940896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f988810f9c0 msgr2=0x7f98881a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:52.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.278+0000 7f9887fff640 1 --2- 172.21.15.67:0/2143940896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f988810f9c0 0x7f98881a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.278+0000 7f9887fff640 1 -- 172.21.15.67:0/2143940896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9888102760 msgr2=0x7f988819c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:52.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.278+0000 7f9887fff640 1 --2- 172.21.15.67:0/2143940896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9888102760 0x7f988819c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.278+0000 7f9887fff640 1 -- 172.21.15.67:0/2143940896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98881a1e40 con 0x7f9888101e10 2024-07-31T14:57:52.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.278+0000 7f987ffff640 1 --2- 172.21.15.67:0/2143940896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9888102760 0x7f988819c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:57:52.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.278+0000 7f9887fff640 1 --2- 172.21.15.67:0/2143940896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9888101e10 0x7f988819bed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f987800e9f0 tx=0x7f987800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:52.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.278+0000 7f9885ffb640 1 -- 172.21.15.67:0/2143940896 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987800ccc0 con 0x7f9888101e10 2024-07-31T14:57:52.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.278+0000 7f988d19d640 1 -- 172.21.15.67:0/2143940896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98881a2130 con 0x7f9888101e10 2024-07-31T14:57:52.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.278+0000 7f988d19d640 1 -- 172.21.15.67:0/2143940896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98881a8890 con 0x7f9888101e10 2024-07-31T14:57:52.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.278+0000 7f9885ffb640 1 -- 172.21.15.67:0/2143940896 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98780108e0 con 0x7f9888101e10 2024-07-31T14:57:52.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.278+0000 7f9885ffb640 1 -- 172.21.15.67:0/2143940896 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878018880 con 0x7f9888101e10 2024-07-31T14:57:52.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.281+0000 7f9894950740 1 Processor -- start 2024-07-31T14:57:52.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.281+0000 7f9894950740 1 -- start start 2024-07-31T14:57:52.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.281+0000 7f9894950740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cafb070 0x55917cea0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.281+0000 7f9894950740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917cea0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.281+0000 7f9894950740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917cea1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.281+0000 7f9894950740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55917ce16bb0 con 0x55917cada390 2024-07-31T14:57:52.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.281+0000 7f9894950740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55917ce060b0 con 0x55917cad21b0 2024-07-31T14:57:52.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.281+0000 7f9894950740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55917ce0a0f0 con 0x55917cafb070 2024-07-31T14:57:52.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.281+0000 7f987ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917cea1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.281+0000 7f988c99c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cafb070 0x55917cea0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.281+0000 7f988c99c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cafb070 0x55917cea0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:52812/0 (socket says 172.21.15.67:52812) 2024-07-31T14:57:52.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.281+0000 7f988c99c640 1 -- 172.21.15.67:0/1233043165 learned_addr learned my addr 172.21.15.67:0/1233043165 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:52.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.281+0000 7f987ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917cea1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55778/0 (socket says 172.21.15.67:55778) 2024-07-31T14:57:52.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.282+0000 7f9885ffb640 1 -- 172.21.15.67:0/2143940896 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f9878010430 con 0x7f9888101e10 2024-07-31T14:57:52.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.282+0000 7f9887fff640 1 --2- 172.21.15.67:0/1233043165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917cea0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.282+0000 7f988c99c640 1 -- 172.21.15.67:0/1233043165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 msgr2=0x55917cea0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:52.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.282+0000 7f988c99c640 1 --2- 172.21.15.67:0/1233043165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917cea0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.282+0000 7f988c99c640 1 -- 172.21.15.67:0/1233043165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 msgr2=0x55917cea1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:52.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.282+0000 7f988c99c640 1 --2- 172.21.15.67:0/1233043165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917cea1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.282+0000 7f988c99c640 1 -- 172.21.15.67:0/1233043165 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917cdacb90 con 0x55917cafb070 2024-07-31T14:57:52.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.282+0000 7f988c99c640 1 --2- 172.21.15.67:0/1233043165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cafb070 0x55917cea0810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9880009fd0 tx=0x7f98800118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88ce567921e741d9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:52.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.282+0000 7f987e7fc640 1 -- 172.21.15.67:0/1233043165 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880004d20 con 0x55917cafb070 2024-07-31T14:57:52.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.282+0000 7f987e7fc640 1 -- 172.21.15.67:0/1233043165 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9880004050 con 0x55917cafb070 2024-07-31T14:57:52.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.282+0000 7f987e7fc640 1 -- 172.21.15.67:0/1233043165 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880005280 con 0x55917cafb070 2024-07-31T14:57:52.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.282+0000 7f9885ffb640 1 --2- 172.21.15.67:0/2143940896 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98740b3d40 0x7f98740b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.283+0000 7f9887fff640 1 --2- 172.21.15.67:0/2143940896 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98740b3d40 0x7f98740b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.283+0000 7f9885ffb640 1 -- 172.21.15.67:0/2143940896 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f98780e4680 con 0x7f9888101e10 2024-07-31T14:57:52.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.283+0000 7f9894950740 1 -- 172.21.15.67:0/1233043165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cafb070 msgr2=0x55917cea0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:52.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.283+0000 7f9887fff640 1 --2- 172.21.15.67:0/2143940896 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98740b3d40 0x7f98740b6250 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55917ce9f490 tx=0x7f987800ce60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:52.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.283+0000 7f9894950740 1 --2- 172.21.15.67:0/1233043165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cafb070 0x55917cea0810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9880009fd0 tx=0x7f98800118d0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.283+0000 7f9894950740 1 -- 172.21.15.67:0/1233043165 shutdown_connections 2024-07-31T14:57:52.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.283+0000 7f9894950740 1 --2- 172.21.15.67:0/1233043165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917cea1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.283+0000 7f9894950740 1 --2- 172.21.15.67:0/1233043165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917cea0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.283+0000 7f9894950740 1 --2- 172.21.15.67:0/1233043165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cafb070 0x55917cea0810 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.283+0000 7f9894950740 1 -- 172.21.15.67:0/1233043165 >> 172.21.15.67:0/1233043165 conn(0x55917caef460 msgr2=0x55917ce9e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:52.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.283+0000 7f9894950740 1 -- 172.21.15.67:0/1233043165 shutdown_connections 2024-07-31T14:57:52.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.283+0000 7f9894950740 1 -- 172.21.15.67:0/1233043165 wait complete. 2024-07-31T14:57:52.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.284+0000 7f9894950740 1 -- start start 2024-07-31T14:57:52.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.284+0000 7f9894950740 1 Processor -- start 2024-07-31T14:57:52.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.284+0000 7f9894950740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917ceac410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.284+0000 7f9894950740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917ceac990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.284+0000 7f9894950740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cafb070 0x55917ceb82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.284+0000 7f9894950740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55917cc69700 con 0x55917cada390 2024-07-31T14:57:52.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.284+0000 7f9894950740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55917cda89e0 con 0x55917cad21b0 2024-07-31T14:57:52.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.284+0000 7f9894950740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55917ceaeff0 con 0x55917cafb070 2024-07-31T14:57:52.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f9894950740 10 client.0 _ll_register_callbacks cb 0x55917cdc34f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:52.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f987ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917ceac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f987ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917ceac990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55782/0 (socket says 172.21.15.67:55782) 2024-07-31T14:57:52.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f987ffff640 1 -- 172.21.15.67:0/2988726023 learned_addr learned my addr 172.21.15.67:0/2988726023 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:52.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f988c99c640 1 --2- 172.21.15.67:0/2988726023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917ceac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f988c99c640 1 --2- 172.21.15.67:0/2988726023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cafb070 0x55917ceb82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f987ffff640 1 -- 172.21.15.67:0/2988726023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cafb070 msgr2=0x55917ceb82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:52.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f987ffff640 1 --2- 172.21.15.67:0/2988726023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cafb070 0x55917ceb82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f987ffff640 1 -- 172.21.15.67:0/2988726023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 msgr2=0x55917ceac410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:52.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f987ffff640 1 --2- 172.21.15.67:0/2988726023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917ceac410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f987ffff640 1 -- 172.21.15.67:0/2988726023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917cb66650 con 0x55917cada390 2024-07-31T14:57:52.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f988c99c640 1 --2- 172.21.15.67:0/2988726023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cafb070 0x55917ceb82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:57:52.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f987ffff640 1 --2- 172.21.15.67:0/2988726023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917ceac990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f9870000c00 tx=0x7f9870031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76031489b091155 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:52.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f98677fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:57:52.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f98677fe640 1 -- 172.21.15.67:0/2988726023 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870035040 con 0x55917cada390 2024-07-31T14:57:52.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f98677fe640 1 -- 172.21.15.67:0/2988726023 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f9870004450 con 0x55917cada390 2024-07-31T14:57:52.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.285+0000 7f9894950740 1 -- 172.21.15.67:0/2988726023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9880004050 con 0x55917cada390 2024-07-31T14:57:52.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.286+0000 7f98677fe640 1 -- 172.21.15.67:0/2988726023 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9870050050 con 0x55917cada390 2024-07-31T14:57:52.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.286+0000 7f9894950740 10 client.17301 fetch_fsmap learned FSMap version 50 2024-07-31T14:57:52.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.286+0000 7f9894950740 1 -- 172.21.15.67:0/2988726023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55917cebb050 con 0x55917cada390 2024-07-31T14:57:52.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.286+0000 7f98677fe640 1 -- 172.21.15.67:0/2988726023 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9880004050 con 0x55917cada390 2024-07-31T14:57:52.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.286+0000 7f98677fe640 1 -- 172.21.15.67:0/2988726023 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f9870004c40 con 0x55917cada390 2024-07-31T14:57:52.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.286+0000 7f9894950740 10 client.17301 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:57:52.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.286+0000 7f9894950740 10 client.17301 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:57:52.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.286+0000 7f9894950740 20 client.17301 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:52.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.286+0000 7f9894950740 1 --2- 172.21.15.67:0/2988726023 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55917cab99f0 0x55917cebe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.286+0000 7f9894950740 4 client.17301 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:57:52.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.286+0000 7f9894950740 1 -- 172.21.15.67:0/2988726023 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55917ce18b00 con 0x55917cab99f0 2024-07-31T14:57:52.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.286+0000 7f988c99c640 1 --2- 172.21.15.67:0/2988726023 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55917cab99f0 0x55917cebe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.287+0000 7f98677fe640 10 client.17301 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:57:52.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.287+0000 7f988c99c640 1 --2- 172.21.15.67:0/2988726023 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55917cab99f0 0x55917cebe5b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=99b90caba0fc16f2 server_cookie=8a20d42b3a11aea8 in_seq=0 out_seq=0 2024-07-31T14:57:52.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f98677fe640 1 -- 172.21.15.67:0/2988726023 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55917ce18b00 con 0x55917cab99f0 2024-07-31T14:57:52.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f98677fe640 10 client.17301 handle_command_reply: tid=0 2024-07-31T14:57:52.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f9894950740 1 client.17301 shutdown 2024-07-31T14:57:52.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f9894950740 1 -- 172.21.15.67:0/2988726023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 msgr2=0x55917ceac990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:52.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f9894950740 1 --2- 172.21.15.67:0/2988726023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917ceac990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f9870000c00 tx=0x7f9870031810 comp rx=0 tx=0).stop 2024-07-31T14:57:52.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f9894950740 1 -- 172.21.15.67:0/2988726023 shutdown_connections 2024-07-31T14:57:52.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f9894950740 1 --2- 172.21.15.67:0/2988726023 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55917cab99f0 0x55917cebe5b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f98677fe640 0 client.17301 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:57:52.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f9894950740 1 --2- 172.21.15.67:0/2988726023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cafb070 0x55917ceb82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f9894950740 1 --2- 172.21.15.67:0/2988726023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917ceac990 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f9894950740 1 --2- 172.21.15.67:0/2988726023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917ceac410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f9894950740 1 -- 172.21.15.67:0/2988726023 >> 172.21.15.67:0/2988726023 conn(0x55917caef460 msgr2=0x55917cea39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:52.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f9894950740 1 -- 172.21.15.67:0/2988726023 shutdown_connections 2024-07-31T14:57:52.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f9894950740 1 -- 172.21.15.67:0/2988726023 wait complete. 2024-07-31T14:57:52.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.288+0000 7f9894950740 20 client.17301 trim_cache size 0 max 16384 2024-07-31T14:57:52.303 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:57:52.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.303+0000 7f9894950740 1 Processor -- start 2024-07-31T14:57:52.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.303+0000 7f9894950740 1 -- start start 2024-07-31T14:57:52.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.303+0000 7f9894950740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cab99f0 0x55917cea1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.303+0000 7f9894950740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917cea5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.303+0000 7f9894950740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917cea5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.303+0000 7f9894950740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f98881b2730 con 0x55917cada390 2024-07-31T14:57:52.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.303+0000 7f9894950740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f98881b43c0 con 0x55917cad21b0 2024-07-31T14:57:52.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.303+0000 7f9894950740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f98881b2e90 con 0x55917cab99f0 2024-07-31T14:57:52.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.303+0000 7f988c99c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cab99f0 0x55917cea1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.303+0000 7f987ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917cea5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.303+0000 7f988c99c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cab99f0 0x55917cea1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:52842/0 (socket says 172.21.15.67:52842) 2024-07-31T14:57:52.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.303+0000 7f988c99c640 1 -- 172.21.15.67:0/108200198 learned_addr learned my addr 172.21.15.67:0/108200198 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:52.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.303+0000 7f988c99c640 1 --2- 172.21.15.67:0/108200198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917cea5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.304+0000 7f988c99c640 1 -- 172.21.15.67:0/108200198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 msgr2=0x55917cea5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:52.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.304+0000 7f988c99c640 1 --2- 172.21.15.67:0/108200198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917cea5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.304+0000 7f988c99c640 1 -- 172.21.15.67:0/108200198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 msgr2=0x55917cea5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:52.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.304+0000 7f988c99c640 1 --2- 172.21.15.67:0/108200198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917cea5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.304+0000 7f988c99c640 1 -- 172.21.15.67:0/108200198 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917cea3670 con 0x55917cab99f0 2024-07-31T14:57:52.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.304+0000 7f988c99c640 1 --2- 172.21.15.67:0/108200198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cab99f0 0x55917cea1f50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9880011dd0 tx=0x7f9880005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a95cc30d6cc8346 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:52.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.304+0000 7f987dffb640 1 -- 172.21.15.67:0/108200198 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880004450 con 0x55917cab99f0 2024-07-31T14:57:52.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.304+0000 7f987dffb640 1 -- 172.21.15.67:0/108200198 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98800045f0 con 0x55917cab99f0 2024-07-31T14:57:52.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.304+0000 7f987dffb640 1 -- 172.21.15.67:0/108200198 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98800296b0 con 0x55917cab99f0 2024-07-31T14:57:52.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.305+0000 7f9894950740 1 -- 172.21.15.67:0/108200198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cab99f0 msgr2=0x55917cea1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:52.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.305+0000 7f9894950740 1 --2- 172.21.15.67:0/108200198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cab99f0 0x55917cea1f50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9880011dd0 tx=0x7f9880005bd0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.305+0000 7f9894950740 1 -- 172.21.15.67:0/108200198 shutdown_connections 2024-07-31T14:57:52.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.305+0000 7f9894950740 1 --2- 172.21.15.67:0/108200198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cada390 0x55917cea5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.305+0000 7f9894950740 1 --2- 172.21.15.67:0/108200198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917cea5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.305+0000 7f9894950740 1 --2- 172.21.15.67:0/108200198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cab99f0 0x55917cea1f50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.305+0000 7f9894950740 1 -- 172.21.15.67:0/108200198 >> 172.21.15.67:0/108200198 conn(0x55917caef460 msgr2=0x55917cea1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:52.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.306+0000 7f9894950740 1 -- 172.21.15.67:0/108200198 shutdown_connections 2024-07-31T14:57:52.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.306+0000 7f9894950740 1 -- 172.21.15.67:0/108200198 wait complete. 2024-07-31T14:57:52.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.306+0000 7f9894950740 1 -- start start 2024-07-31T14:57:52.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.306+0000 7f9894950740 1 Processor -- start 2024-07-31T14:57:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.307+0000 7f9894950740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cab99f0 0x55917ceb7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.307+0000 7f9894950740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917ceb7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.307+0000 7f9894950740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cada390 0x55917ceb3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.307+0000 7f9894950740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f98881b3cf0 con 0x55917cab99f0 2024-07-31T14:57:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.307+0000 7f9894950740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f98881b3560 con 0x55917cad21b0 2024-07-31T14:57:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.307+0000 7f9894950740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f98881b0710 con 0x55917cada390 2024-07-31T14:57:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.307+0000 7f9894950740 10 client.0 _ll_register_callbacks cb 0x7f9870009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:57:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.307+0000 7f987ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917ceb7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.307+0000 7f988c99c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cab99f0 0x55917ceb7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.307+0000 7f987ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917ceb7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60660/0 (socket says 172.21.15.67:60660) 2024-07-31T14:57:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.307+0000 7f987ffff640 1 -- 172.21.15.67:0/3684547376 learned_addr learned my addr 172.21.15.67:0/3684547376 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:52.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f988c99c640 1 --2- 172.21.15.67:0/3684547376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cada390 0x55917ceb3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f987ffff640 1 -- 172.21.15.67:0/3684547376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cada390 msgr2=0x55917ceb3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:52.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f987ffff640 1 --2- 172.21.15.67:0/3684547376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cada390 0x55917ceb3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f987ffff640 1 -- 172.21.15.67:0/3684547376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cab99f0 msgr2=0x55917ceb7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:57:52.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f987ffff640 1 --2- 172.21.15.67:0/3684547376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cab99f0 0x55917ceb7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f987ffff640 1 -- 172.21.15.67:0/3684547376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917ceb38b0 con 0x55917cad21b0 2024-07-31T14:57:52.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f988c99c640 1 --2- 172.21.15.67:0/3684547376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cada390 0x55917ceb3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:57:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f987ffff640 1 --2- 172.21.15.67:0/3684547376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917ceb7ed0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f98700096c0 tx=0x7f9870003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e78a12091527913d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f98677fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:57:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f98677fe640 1 -- 172.21.15.67:0/3684547376 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870035070 con 0x55917cad21b0 2024-07-31T14:57:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f98677fe640 1 -- 172.21.15.67:0/3684547376 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f9870003c30 con 0x55917cad21b0 2024-07-31T14:57:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f9894950740 1 -- 172.21.15.67:0/3684547376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98800045f0 con 0x55917cad21b0 2024-07-31T14:57:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f98677fe640 1 -- 172.21.15.67:0/3684547376 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9870003e30 con 0x55917cad21b0 2024-07-31T14:57:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f9894950740 10 client.26870 fetch_fsmap learned FSMap version 50 2024-07-31T14:57:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.308+0000 7f9894950740 1 -- 172.21.15.67:0/3684547376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55917ceb4b40 con 0x55917cad21b0 2024-07-31T14:57:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.309+0000 7f98677fe640 1 -- 172.21.15.67:0/3684547376 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98800045f0 con 0x55917cad21b0 2024-07-31T14:57:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.309+0000 7f98677fe640 1 -- 172.21.15.67:0/3684547376 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f9870004770 con 0x55917cad21b0 2024-07-31T14:57:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.309+0000 7f9894950740 10 client.26870 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:57:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.309+0000 7f9894950740 10 client.26870 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:57:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.309+0000 7f9894950740 20 client.26870 populate_metadata read hostname 'smithi067' 2024-07-31T14:57:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.309+0000 7f9894950740 1 --2- 172.21.15.67:0/3684547376 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55917cafb070 0x55917cf290b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.309+0000 7f9894950740 4 client.26870 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:57:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.309+0000 7f9894950740 1 -- 172.21.15.67:0/3684547376 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55917ce145a0 con 0x55917cafb070 2024-07-31T14:57:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.309+0000 7f988c99c640 1 --2- 172.21.15.67:0/3684547376 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55917cafb070 0x55917cf290b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f98677fe640 10 client.26870 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:57:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f988c99c640 1 --2- 172.21.15.67:0/3684547376 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55917cafb070 0x55917cf290b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fe0049e611ea26cf server_cookie=69ff6e005921c29e in_seq=0 out_seq=0 2024-07-31T14:57:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f98677fe640 1 -- 172.21.15.67:0/3684547376 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55917ce145a0 con 0x55917cafb070 2024-07-31T14:57:52.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f98677fe640 10 client.26870 handle_command_reply: tid=0 2024-07-31T14:57:52.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f9894950740 1 client.26870 shutdown 2024-07-31T14:57:52.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f9894950740 1 -- 172.21.15.67:0/3684547376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 msgr2=0x55917ceb7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:52.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f9894950740 1 --2- 172.21.15.67:0/3684547376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917ceb7ed0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f98700096c0 tx=0x7f9870003820 comp rx=0 tx=0).stop 2024-07-31T14:57:52.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f9894950740 1 -- 172.21.15.67:0/3684547376 shutdown_connections 2024-07-31T14:57:52.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f9894950740 1 --2- 172.21.15.67:0/3684547376 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55917cafb070 0x55917cf290b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f9894950740 1 --2- 172.21.15.67:0/3684547376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55917cada390 0x55917ceb3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f9894950740 1 --2- 172.21.15.67:0/3684547376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917cad21b0 0x55917ceb7ed0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f9894950740 1 --2- 172.21.15.67:0/3684547376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55917cab99f0 0x55917ceb7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f9894950740 1 -- 172.21.15.67:0/3684547376 >> 172.21.15.67:0/3684547376 conn(0x55917caef460 msgr2=0x55917cea40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:52.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f98677fe640 0 client.26870 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:57:52.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f9894950740 1 -- 172.21.15.67:0/3684547376 shutdown_connections 2024-07-31T14:57:52.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.310+0000 7f9894950740 1 -- 172.21.15.67:0/3684547376 wait complete. 2024-07-31T14:57:52.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.311+0000 7f9894950740 20 client.26870 trim_cache size 0 max 16384 2024-07-31T14:57:52.318 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:57:52.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.312+0000 7f988d19d640 1 -- 172.21.15.67:0/2143940896 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98740b3d40 msgr2=0x7f98740b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:52.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.312+0000 7f988d19d640 1 --2- 172.21.15.67:0/2143940896 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98740b3d40 0x7f98740b6250 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55917ce9f490 tx=0x7f987800ce60 comp rx=0 tx=0).stop 2024-07-31T14:57:52.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.312+0000 7f988d19d640 1 -- 172.21.15.67:0/2143940896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9888101e10 msgr2=0x7f988819bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:52.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.312+0000 7f988d19d640 1 --2- 172.21.15.67:0/2143940896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9888101e10 0x7f988819bed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f987800e9f0 tx=0x7f987800eeb0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.312+0000 7f988d19d640 1 -- 172.21.15.67:0/2143940896 shutdown_connections 2024-07-31T14:57:52.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.312+0000 7f988d19d640 1 --2- 172.21.15.67:0/2143940896 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98740b3d40 0x7f98740b6250 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.312+0000 7f988d19d640 1 --2- 172.21.15.67:0/2143940896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f988810f9c0 0x7f98881a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.312+0000 7f988d19d640 1 --2- 172.21.15.67:0/2143940896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9888102760 0x7f988819c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.312+0000 7f988d19d640 1 --2- 172.21.15.67:0/2143940896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9888101e10 0x7f988819bed0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:52.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.312+0000 7f988d19d640 1 -- 172.21.15.67:0/2143940896 >> 172.21.15.67:0/2143940896 conn(0x7f98880fdcf0 msgr2=0x7f9888111b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:52.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.312+0000 7f988d19d640 1 -- 172.21.15.67:0/2143940896 shutdown_connections 2024-07-31T14:57:52.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:52.312+0000 7f988d19d640 1 -- 172.21.15.67:0/2143940896 wait complete. 2024-07-31T14:57:53.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:52 smithi073 ceph-mon[36714]: pgmap v604: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 1.09k op/s 2024-07-31T14:57:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:52 smithi077 ceph-mon[37327]: pgmap v604: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 1.09k op/s 2024-07-31T14:57:53.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:52 smithi067 ceph-mon[32750]: pgmap v604: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 1.09k op/s 2024-07-31T14:57:53.561 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T14:57:55.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:54 smithi067 ceph-mon[32750]: pgmap v605: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 1.39k op/s 2024-07-31T14:57:55.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:54 smithi073 ceph-mon[36714]: pgmap v605: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 1.39k op/s 2024-07-31T14:57:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:54 smithi077 ceph-mon[37327]: pgmap v605: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 1.39k op/s 2024-07-31T14:57:57.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.088+0000 7fc63e67f640 1 -- 172.21.15.67:0/459182038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc638107dd0 msgr2=0x7fc6381081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:57.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.088+0000 7fc63e67f640 1 --2- 172.21.15.67:0/459182038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc638107dd0 0x7fc6381081e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009900 tx=0x7fc62c02a410 comp rx=0 tx=0).stop 2024-07-31T14:57:57.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.088+0000 7fc63e67f640 1 -- 172.21.15.67:0/459182038 shutdown_connections 2024-07-31T14:57:57.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.088+0000 7fc63e67f640 1 --2- 172.21.15.67:0/459182038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc638108720 0x7fc63810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:57.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.088+0000 7fc63e67f640 1 --2- 172.21.15.67:0/459182038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc638107dd0 0x7fc6381081e0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:57.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.088+0000 7fc63e67f640 1 --2- 172.21.15.67:0/459182038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638102490 0x7fc6381028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:57.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.088+0000 7fc63e67f640 1 -- 172.21.15.67:0/459182038 >> 172.21.15.67:0/459182038 conn(0x7fc638079160 msgr2=0x7fc638079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:57.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.088+0000 7fc63e67f640 1 -- 172.21.15.67:0/459182038 shutdown_connections 2024-07-31T14:57:57.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.088+0000 7fc63e67f640 1 -- 172.21.15.67:0/459182038 wait complete. 2024-07-31T14:57:57.091 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:56 smithi067 ceph-mon[32750]: pgmap v606: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 1.58k op/s 2024-07-31T14:57:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.089+0000 7fc63e67f640 1 Processor -- start 2024-07-31T14:57:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.089+0000 7fc63e67f640 1 -- start start 2024-07-31T14:57:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.089+0000 7fc63e67f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc638102490 0x7fc638199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.089+0000 7fc63e67f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638107dd0 0x7fc63819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.089+0000 7fc63e67f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc638108720 0x7fc63819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.089+0000 7fc63e67f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc638112370 con 0x7fc638108720 2024-07-31T14:57:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.089+0000 7fc63e67f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc6381121f0 con 0x7fc638102490 2024-07-31T14:57:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.089+0000 7fc63e67f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc6381124f0 con 0x7fc638107dd0 2024-07-31T14:57:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.090+0000 7fc637fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc638102490 0x7fc638199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:57.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.090+0000 7fc63cc1c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc638108720 0x7fc63819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:57.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.090+0000 7fc637fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc638102490 0x7fc638199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60674/0 (socket says 172.21.15.67:60674) 2024-07-31T14:57:57.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.090+0000 7fc637fff640 1 -- 172.21.15.67:0/505796958 learned_addr learned my addr 172.21.15.67:0/505796958 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:57:57.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.090+0000 7fc6377fe640 1 --2- 172.21.15.67:0/505796958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638107dd0 0x7fc63819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:57.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.090+0000 7fc637fff640 1 -- 172.21.15.67:0/505796958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638107dd0 msgr2=0x7fc63819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:57.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.090+0000 7fc637fff640 1 --2- 172.21.15.67:0/505796958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638107dd0 0x7fc63819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:57.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.090+0000 7fc637fff640 1 -- 172.21.15.67:0/505796958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc638108720 msgr2=0x7fc63819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.090+0000 7fc637fff640 1 --2- 172.21.15.67:0/505796958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc638108720 0x7fc63819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.090+0000 7fc637fff640 1 -- 172.21.15.67:0/505796958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63819fe40 con 0x7fc638102490 2024-07-31T14:57:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.090+0000 7fc63cc1c640 1 --2- 172.21.15.67:0/505796958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc638108720 0x7fc63819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:57:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.090+0000 7fc637fff640 1 --2- 172.21.15.67:0/505796958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc638102490 0x7fc638199ed0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc62800e9f0 tx=0x7fc62800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.091+0000 7fc6357fa640 1 -- 172.21.15.67:0/505796958 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62800cd80 con 0x7fc638102490 2024-07-31T14:57:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.091+0000 7fc63e67f640 1 -- 172.21.15.67:0/505796958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6381a0130 con 0x7fc638102490 2024-07-31T14:57:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.091+0000 7fc6357fa640 1 -- 172.21.15.67:0/505796958 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6280054d0 con 0x7fc638102490 2024-07-31T14:57:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.091+0000 7fc6357fa640 1 -- 172.21.15.67:0/505796958 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628010690 con 0x7fc638102490 2024-07-31T14:57:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.091+0000 7fc63e67f640 1 -- 172.21.15.67:0/505796958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6380fef80 con 0x7fc638102490 2024-07-31T14:57:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.092+0000 7fc63e67f640 1 -- 172.21.15.67:0/505796958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6381028a0 con 0x7fc638102490 2024-07-31T14:57:57.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.094+0000 7fc6357fa640 1 -- 172.21.15.67:0/505796958 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc628004e90 con 0x7fc638102490 2024-07-31T14:57:57.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.094+0000 7fc6357fa640 1 --2- 172.21.15.67:0/505796958 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6200b3dc0 0x7fc6200b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:57:57.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.095+0000 7fc6377fe640 1 --2- 172.21.15.67:0/505796958 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6200b3dc0 0x7fc6200b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:57:57.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.095+0000 7fc6357fa640 1 -- 172.21.15.67:0/505796958 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fc6280e4750 con 0x7fc638102490 2024-07-31T14:57:57.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.095+0000 7fc6377fe640 1 --2- 172.21.15.67:0/505796958 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6200b3dc0 0x7fc6200b62d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc62c000c00 tx=0x7fc62c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:57:57.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.099+0000 7fc6357fa640 1 -- 172.21.15.67:0/505796958 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc6280ab750 con 0x7fc638102490 2024-07-31T14:57:57.110 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:57:57.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:56 smithi073 ceph-mon[36714]: pgmap v606: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 1.58k op/s 2024-07-31T14:57:57.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:56 smithi077 ceph-mon[37327]: pgmap v606: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 1.58k op/s 2024-07-31T14:57:57.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.340+0000 7fc63e67f640 1 -- 172.21.15.67:0/505796958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc63819b000 con 0x7fc638102490 2024-07-31T14:57:57.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.341+0000 7fc6357fa640 1 -- 172.21.15.67:0/505796958 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11972 (secure 0 0 0) 0x7fc63819b000 con 0x7fc638102490 2024-07-31T14:57:57.343 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:57:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":50,"btime":"2024-07-31T14:57:48:808322+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:57:48.683213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:57:57.343 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 50 2024-07-31T14:57:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.344+0000 7fc63e67f640 1 -- 172.21.15.67:0/505796958 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6200b3dc0 msgr2=0x7fc6200b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.344+0000 7fc63e67f640 1 --2- 172.21.15.67:0/505796958 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6200b3dc0 0x7fc6200b62d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc62c000c00 tx=0x7fc62c031400 comp rx=0 tx=0).stop 2024-07-31T14:57:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.344+0000 7fc63e67f640 1 -- 172.21.15.67:0/505796958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc638102490 msgr2=0x7fc638199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:57:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.344+0000 7fc63e67f640 1 --2- 172.21.15.67:0/505796958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc638102490 0x7fc638199ed0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc62800e9f0 tx=0x7fc62800edf0 comp rx=0 tx=0).stop 2024-07-31T14:57:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.344+0000 7fc63e67f640 1 -- 172.21.15.67:0/505796958 shutdown_connections 2024-07-31T14:57:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.344+0000 7fc63e67f640 1 --2- 172.21.15.67:0/505796958 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6200b3dc0 0x7fc6200b62d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:57.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.344+0000 7fc63e67f640 1 --2- 172.21.15.67:0/505796958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc638108720 0x7fc63819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:57.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.344+0000 7fc63e67f640 1 --2- 172.21.15.67:0/505796958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638107dd0 0x7fc63819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:57.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.344+0000 7fc63e67f640 1 --2- 172.21.15.67:0/505796958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc638102490 0x7fc638199ed0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:57:57.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.344+0000 7fc63e67f640 1 -- 172.21.15.67:0/505796958 >> 172.21.15.67:0/505796958 conn(0x7fc638079160 msgr2=0x7fc638101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:57:57.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.344+0000 7fc63e67f640 1 -- 172.21.15.67:0/505796958 shutdown_connections 2024-07-31T14:57:57.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:57:57.344+0000 7fc63e67f640 1 -- 172.21.15.67:0/505796958 wait complete. 2024-07-31T14:57:58.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:57 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/505796958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:57:58.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:57 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/505796958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:57:58.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:57 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/505796958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:57:59.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:57:58 smithi067 ceph-mon[32750]: pgmap v607: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 4.4 MiB/s wr, 1.64k op/s 2024-07-31T14:57:59.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:57:58 smithi073 ceph-mon[36714]: pgmap v607: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 4.4 MiB/s wr, 1.64k op/s 2024-07-31T14:57:59.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:57:58 smithi077 ceph-mon[37327]: pgmap v607: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 4.4 MiB/s wr, 1.64k op/s 2024-07-31T14:57:59.875 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-07-31T14:57:59.875 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-07-31T14:58:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.415+0000 7f85dcbdd640 1 -- 172.21.15.67:0/280979915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d8102490 msgr2=0x7f85d81028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.415+0000 7f85dcbdd640 1 --2- 172.21.15.67:0/280979915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d8102490 0x7f85d81028a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009ca0 tx=0x7f85d002a8c0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.415+0000 7f85dcbdd640 1 -- 172.21.15.67:0/280979915 shutdown_connections 2024-07-31T14:58:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.415+0000 7f85dcbdd640 1 --2- 172.21.15.67:0/280979915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d8108720 0x7f85d810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.415+0000 7f85dcbdd640 1 --2- 172.21.15.67:0/280979915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d8107dd0 0x7f85d81081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.415+0000 7f85dcbdd640 1 --2- 172.21.15.67:0/280979915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d8102490 0x7f85d81028a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.415+0000 7f85dcbdd640 1 -- 172.21.15.67:0/280979915 >> 172.21.15.67:0/280979915 conn(0x7f85d8079160 msgr2=0x7f85d8079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.415+0000 7f85dcbdd640 1 -- 172.21.15.67:0/280979915 shutdown_connections 2024-07-31T14:58:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.415+0000 7f85dcbdd640 1 -- 172.21.15.67:0/280979915 wait complete. 2024-07-31T14:58:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.416+0000 7f85dcbdd640 1 Processor -- start 2024-07-31T14:58:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.416+0000 7f85dcbdd640 1 -- start start 2024-07-31T14:58:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.416+0000 7f85dcbdd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d8102490 0x7f85d8103c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.416+0000 7f85dcbdd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d8107dd0 0x7f85d81041c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.416+0000 7f85dcbdd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d8108720 0x7f85d8104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.416+0000 7f85dcbdd640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8112380 con 0x7f85d8108720 2024-07-31T14:58:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.416+0000 7f85dcbdd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8112200 con 0x7f85d8107dd0 2024-07-31T14:58:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.416+0000 7f85dcbdd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8112500 con 0x7f85d8102490 2024-07-31T14:58:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.417+0000 7f85d5d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d8107dd0 0x7f85d81041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.417+0000 7f85d6d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d8108720 0x7f85d8104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.417+0000 7f85d5d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d8107dd0 0x7f85d81041c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60692/0 (socket says 172.21.15.67:60692) 2024-07-31T14:58:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.417+0000 7f85d5d9b640 1 -- 172.21.15.67:0/1161624417 learned_addr learned my addr 172.21.15.67:0/1161624417 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.417+0000 7f85d659c640 1 --2- 172.21.15.67:0/1161624417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d8102490 0x7f85d8103c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.417+0000 7f85d5d9b640 1 -- 172.21.15.67:0/1161624417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d8102490 msgr2=0x7f85d8103c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.417+0000 7f85d5d9b640 1 --2- 172.21.15.67:0/1161624417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d8102490 0x7f85d8103c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.417+0000 7f85d5d9b640 1 -- 172.21.15.67:0/1161624417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d8108720 msgr2=0x7f85d8104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.417+0000 7f85d5d9b640 1 --2- 172.21.15.67:0/1161624417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d8108720 0x7f85d8104700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.417+0000 7f85d5d9b640 1 -- 172.21.15.67:0/1161624417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d81a8160 con 0x7f85d8107dd0 2024-07-31T14:58:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.418+0000 7f85d5d9b640 1 --2- 172.21.15.67:0/1161624417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d8107dd0 0x7f85d81041c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f85c000d770 tx=0x7f85c000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.418+0000 7f85cf7fe640 1 -- 172.21.15.67:0/1161624417 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0017070 con 0x7f85d8107dd0 2024-07-31T14:58:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.418+0000 7f85dcbdd640 1 -- 172.21.15.67:0/1161624417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d81a8450 con 0x7f85d8107dd0 2024-07-31T14:58:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.418+0000 7f85cf7fe640 1 -- 172.21.15.67:0/1161624417 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85c0009c30 con 0x7f85d8107dd0 2024-07-31T14:58:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.418+0000 7f85cf7fe640 1 -- 172.21.15.67:0/1161624417 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c000f620 con 0x7f85d8107dd0 2024-07-31T14:58:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.418+0000 7f85dcbdd640 1 -- 172.21.15.67:0/1161624417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d81a8990 con 0x7f85d8107dd0 2024-07-31T14:58:00.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.421+0000 7f85e312e740 1 Processor -- start 2024-07-31T14:58:00.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.421+0000 7f85e312e740 1 -- start start 2024-07-31T14:58:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.422+0000 7f85e312e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 0x55ea41454570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.422+0000 7f85e312e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea410861b0 0x55ea41454ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.422+0000 7f85e312e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea4108e390 0x55ea41454ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.422+0000 7f85e312e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ea413cabb0 con 0x55ea4108e390 2024-07-31T14:58:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.422+0000 7f85e312e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ea413ba0b0 con 0x55ea410861b0 2024-07-31T14:58:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.422+0000 7f85e312e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ea413be0f0 con 0x55ea410af070 2024-07-31T14:58:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.422+0000 7f85d659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea410861b0 0x55ea41454ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.422+0000 7f85d6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 0x55ea41454570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.422+0000 7f85d659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea410861b0 0x55ea41454ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60708/0 (socket says 172.21.15.67:60708) 2024-07-31T14:58:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.422+0000 7f85d659c640 1 -- 172.21.15.67:0/3412239857 learned_addr learned my addr 172.21.15.67:0/3412239857 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.422+0000 7f85cf7fe640 1 -- 172.21.15.67:0/1161624417 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f85c001a070 con 0x7f85d8107dd0 2024-07-31T14:58:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.422+0000 7f85d659c640 1 -- 172.21.15.67:0/3412239857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 msgr2=0x55ea41454570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.422+0000 7f85d659c640 1 --2- 172.21.15.67:0/3412239857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 0x55ea41454570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.422+0000 7f85d659c640 1 -- 172.21.15.67:0/3412239857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea4108e390 msgr2=0x55ea41454ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:58:00.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.423+0000 7f85d659c640 1 --2- 172.21.15.67:0/3412239857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea4108e390 0x55ea41454ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.423+0000 7f85d659c640 1 -- 172.21.15.67:0/3412239857 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea41360b90 con 0x55ea410861b0 2024-07-31T14:58:00.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.423+0000 7f85d659c640 1 --2- 172.21.15.67:0/3412239857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea410861b0 0x55ea41454ab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009ca0 tx=0x7f85d0009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95513ab1ba867229 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:00.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.423+0000 7f85ccff9640 1 -- 172.21.15.67:0/3412239857 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0031db0 con 0x55ea410861b0 2024-07-31T14:58:00.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.424+0000 7f85ccff9640 1 -- 172.21.15.67:0/3412239857 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d0036030 con 0x55ea410861b0 2024-07-31T14:58:00.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.424+0000 7f85ccff9640 1 -- 172.21.15.67:0/3412239857 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0004330 con 0x55ea410861b0 2024-07-31T14:58:00.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.424+0000 7f85cf7fe640 1 --2- 172.21.15.67:0/1161624417 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f85c40b3d40 0x7f85c40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.424+0000 7f85d6d9d640 1 --2- 172.21.15.67:0/1161624417 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f85c40b3d40 0x7f85c40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.424+0000 7f85cf7fe640 1 -- 172.21.15.67:0/1161624417 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f85c00e3da0 con 0x7f85d8107dd0 2024-07-31T14:58:00.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.425+0000 7f85d6d9d640 1 --2- 172.21.15.67:0/1161624417 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f85c40b3d40 0x7f85c40b6250 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009da0 tx=0x7f85c8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:00.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.425+0000 7f85e312e740 1 -- 172.21.15.67:0/3412239857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea410861b0 msgr2=0x55ea41454ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:00.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.425+0000 7f85e312e740 1 --2- 172.21.15.67:0/3412239857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea410861b0 0x55ea41454ab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009ca0 tx=0x7f85d0009800 comp rx=0 tx=0).stop 2024-07-31T14:58:00.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.425+0000 7f85e312e740 1 -- 172.21.15.67:0/3412239857 shutdown_connections 2024-07-31T14:58:00.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.425+0000 7f85e312e740 1 --2- 172.21.15.67:0/3412239857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea4108e390 0x55ea41454ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.425+0000 7f85e312e740 1 --2- 172.21.15.67:0/3412239857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea410861b0 0x55ea41454ab0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.425+0000 7f85e312e740 1 --2- 172.21.15.67:0/3412239857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 0x55ea41454570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.425+0000 7f85e312e740 1 -- 172.21.15.67:0/3412239857 >> 172.21.15.67:0/3412239857 conn(0x55ea410a3460 msgr2=0x55ea41452460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.425+0000 7f85e312e740 1 -- 172.21.15.67:0/3412239857 shutdown_connections 2024-07-31T14:58:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.425+0000 7f85e312e740 1 -- 172.21.15.67:0/3412239857 wait complete. 2024-07-31T14:58:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.425+0000 7f85e312e740 1 -- start start 2024-07-31T14:58:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.426+0000 7f85e312e740 1 Processor -- start 2024-07-31T14:58:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.426+0000 7f85e312e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410861b0 0x55ea41460210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.426+0000 7f85e312e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea4108e390 0x55ea414607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.426+0000 7f85e312e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea410af070 0x55ea41465e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.426+0000 7f85e312e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ea4121d700 con 0x55ea410af070 2024-07-31T14:58:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.426+0000 7f85e312e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ea4135c9e0 con 0x55ea4108e390 2024-07-31T14:58:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.426+0000 7f85e312e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ea41462b70 con 0x55ea410861b0 2024-07-31T14:58:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.426+0000 7f85d5d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea4108e390 0x55ea414607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.426+0000 7f85d5d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea4108e390 0x55ea414607f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60714/0 (socket says 172.21.15.67:60714) 2024-07-31T14:58:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.426+0000 7f85d5d9b640 1 -- 172.21.15.67:0/2766824678 learned_addr learned my addr 172.21.15.67:0/2766824678 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:00.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.426+0000 7f85e312e740 10 client.0 _ll_register_callbacks cb 0x55ea413774f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:00.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.426+0000 7f85d6d9d640 1 --2- 172.21.15.67:0/2766824678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea410af070 0x55ea41465e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.427+0000 7f85d659c640 1 --2- 172.21.15.67:0/2766824678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410861b0 0x55ea41460210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.427+0000 7f85d5d9b640 1 -- 172.21.15.67:0/2766824678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410861b0 msgr2=0x55ea41460210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:00.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.427+0000 7f85d5d9b640 1 --2- 172.21.15.67:0/2766824678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410861b0 0x55ea41460210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.427+0000 7f85d5d9b640 1 -- 172.21.15.67:0/2766824678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea410af070 msgr2=0x55ea41465e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.427+0000 7f85d5d9b640 1 --2- 172.21.15.67:0/2766824678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea410af070 0x55ea41465e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.427+0000 7f85d5d9b640 1 -- 172.21.15.67:0/2766824678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea4111a650 con 0x55ea4108e390 2024-07-31T14:58:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.427+0000 7f85d6d9d640 1 --2- 172.21.15.67:0/2766824678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea410af070 0x55ea41465e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:58:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.428+0000 7f85d5d9b640 1 --2- 172.21.15.67:0/2766824678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea4108e390 0x55ea414607f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f85c00a67d0 tx=0x7f85c00aa650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5b64693acd19ee6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.428+0000 7f85bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:58:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.428+0000 7f85bdffb640 1 -- 172.21.15.67:0/2766824678 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0021070 con 0x55ea4108e390 2024-07-31T14:58:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.428+0000 7f85bdffb640 1 -- 172.21.15.67:0/2766824678 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f85c00cbba0 con 0x55ea4108e390 2024-07-31T14:58:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.428+0000 7f85e312e740 1 -- 172.21.15.67:0/2766824678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d0036030 con 0x55ea4108e390 2024-07-31T14:58:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.429+0000 7f85bdffb640 1 -- 172.21.15.67:0/2766824678 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f85c00aa260 con 0x55ea4108e390 2024-07-31T14:58:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.429+0000 7f85e312e740 10 client.26888 fetch_fsmap learned FSMap version 50 2024-07-31T14:58:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.429+0000 7f85e312e740 1 -- 172.21.15.67:0/2766824678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55ea4146ecb0 con 0x55ea4108e390 2024-07-31T14:58:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.429+0000 7f85bdffb640 1 -- 172.21.15.67:0/2766824678 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85c0009de0 con 0x55ea4108e390 2024-07-31T14:58:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.430+0000 7f85bdffb640 1 -- 172.21.15.67:0/2766824678 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f85c00a60c0 con 0x55ea4108e390 2024-07-31T14:58:00.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.430+0000 7f85e312e740 10 client.26888 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:58:00.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.430+0000 7f85e312e740 10 client.26888 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:58:00.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.430+0000 7f85e312e740 20 client.26888 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:00.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.430+0000 7f85e312e740 1 --2- 172.21.15.67:0/2766824678 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ea4106d9f0 0x55ea41472150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.430+0000 7f85e312e740 4 client.26888 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:58:00.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.430+0000 7f85e312e740 1 -- 172.21.15.67:0/2766824678 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f85d0036030 con 0x55ea4106d9f0 2024-07-31T14:58:00.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.430+0000 7f85d659c640 1 --2- 172.21.15.67:0/2766824678 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ea4106d9f0 0x55ea41472150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.431+0000 7f85d659c640 1 --2- 172.21.15.67:0/2766824678 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ea4106d9f0 0x55ea41472150 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8ac00ab6613c549 server_cookie=631c8135bc5d67df in_seq=0 out_seq=0 2024-07-31T14:58:00.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.431+0000 7f85bdffb640 10 client.26888 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:58:00.434 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:58:00.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85bdffb640 1 -- 172.21.15.67:0/2766824678 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f85d0036030 con 0x55ea4106d9f0 2024-07-31T14:58:00.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85bdffb640 10 client.26888 handle_command_reply: tid=0 2024-07-31T14:58:00.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85e312e740 1 client.26888 shutdown 2024-07-31T14:58:00.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85e312e740 1 -- 172.21.15.67:0/2766824678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea4108e390 msgr2=0x55ea414607f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:00.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85e312e740 1 --2- 172.21.15.67:0/2766824678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea4108e390 0x55ea414607f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f85c00a67d0 tx=0x7f85c00aa650 comp rx=0 tx=0).stop 2024-07-31T14:58:00.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85e312e740 1 -- 172.21.15.67:0/2766824678 shutdown_connections 2024-07-31T14:58:00.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85e312e740 1 --2- 172.21.15.67:0/2766824678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea410af070 0x55ea41465e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85e312e740 1 --2- 172.21.15.67:0/2766824678 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ea4106d9f0 0x55ea41472150 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85e312e740 1 --2- 172.21.15.67:0/2766824678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea4108e390 0x55ea414607f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85e312e740 1 --2- 172.21.15.67:0/2766824678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410861b0 0x55ea41460210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85bdffb640 0 client.26888 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:58:00.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85e312e740 1 -- 172.21.15.67:0/2766824678 >> 172.21.15.67:0/2766824678 conn(0x55ea410a3460 msgr2=0x55ea41450c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:00.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85e312e740 1 -- 172.21.15.67:0/2766824678 shutdown_connections 2024-07-31T14:58:00.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85e312e740 1 -- 172.21.15.67:0/2766824678 wait complete. 2024-07-31T14:58:00.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.433+0000 7f85e312e740 20 client.26888 trim_cache size 0 max 16384 2024-07-31T14:58:00.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.449+0000 7f85e312e740 1 Processor -- start 2024-07-31T14:58:00.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.449+0000 7f85e312e740 1 -- start start 2024-07-31T14:58:00.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.449+0000 7f85e312e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea410861b0 0x55ea41465da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.449+0000 7f85e312e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea4108e390 0x55ea41452490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.449+0000 7f85e312e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 0x55ea41459680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.449+0000 7f85e312e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81b27b0 con 0x55ea4108e390 2024-07-31T14:58:00.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.450+0000 7f85e312e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81b4440 con 0x55ea410861b0 2024-07-31T14:58:00.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.450+0000 7f85e312e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81b2f10 con 0x55ea410af070 2024-07-31T14:58:00.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.450+0000 7f85d659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea410861b0 0x55ea41465da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.450+0000 7f85d659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea410861b0 0x55ea41465da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60716/0 (socket says 172.21.15.67:60716) 2024-07-31T14:58:00.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.450+0000 7f85d659c640 1 -- 172.21.15.67:0/2121975432 learned_addr learned my addr 172.21.15.67:0/2121975432 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:00.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.450+0000 7f85d5d9b640 1 --2- 172.21.15.67:0/2121975432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea4108e390 0x55ea41452490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.450+0000 7f85d6d9d640 1 --2- 172.21.15.67:0/2121975432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 0x55ea41459680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.450+0000 7f85d659c640 1 -- 172.21.15.67:0/2121975432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 msgr2=0x55ea41459680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:00.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.450+0000 7f85d659c640 1 --2- 172.21.15.67:0/2121975432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 0x55ea41459680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.450+0000 7f85d659c640 1 -- 172.21.15.67:0/2121975432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea4108e390 msgr2=0x55ea41452490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:00.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.450+0000 7f85d659c640 1 --2- 172.21.15.67:0/2121975432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea4108e390 0x55ea41452490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.450+0000 7f85d659c640 1 -- 172.21.15.67:0/2121975432 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea4144db10 con 0x55ea410861b0 2024-07-31T14:58:00.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.450+0000 7f85d5d9b640 1 --2- 172.21.15.67:0/2121975432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea4108e390 0x55ea41452490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:58:00.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.451+0000 7f85d659c640 1 --2- 172.21.15.67:0/2121975432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea410861b0 0x55ea41465da0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009a60 tx=0x7f85d0003e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=959035840714691d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:00.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.451+0000 7f85bffff640 1 -- 172.21.15.67:0/2121975432 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d005f070 con 0x55ea410861b0 2024-07-31T14:58:00.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.451+0000 7f85bffff640 1 -- 172.21.15.67:0/2121975432 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d0036040 con 0x55ea410861b0 2024-07-31T14:58:00.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.451+0000 7f85bffff640 1 -- 172.21.15.67:0/2121975432 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d00318f0 con 0x55ea410861b0 2024-07-31T14:58:00.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.452+0000 7f85e312e740 1 -- 172.21.15.67:0/2121975432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea410861b0 msgr2=0x55ea41465da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:00.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.452+0000 7f85e312e740 1 --2- 172.21.15.67:0/2121975432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea410861b0 0x55ea41465da0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009a60 tx=0x7f85d0003e40 comp rx=0 tx=0).stop 2024-07-31T14:58:00.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.452+0000 7f85e312e740 1 -- 172.21.15.67:0/2121975432 shutdown_connections 2024-07-31T14:58:00.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.452+0000 7f85e312e740 1 --2- 172.21.15.67:0/2121975432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 0x55ea41459680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.452+0000 7f85e312e740 1 --2- 172.21.15.67:0/2121975432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea4108e390 0x55ea41452490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.452+0000 7f85e312e740 1 --2- 172.21.15.67:0/2121975432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea410861b0 0x55ea41465da0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.452+0000 7f85e312e740 1 -- 172.21.15.67:0/2121975432 >> 172.21.15.67:0/2121975432 conn(0x55ea410a3460 msgr2=0x55ea41451d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.452+0000 7f85e312e740 1 -- 172.21.15.67:0/2121975432 shutdown_connections 2024-07-31T14:58:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.452+0000 7f85e312e740 1 -- 172.21.15.67:0/2121975432 wait complete. 2024-07-31T14:58:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.452+0000 7f85e312e740 1 -- start start 2024-07-31T14:58:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85e312e740 1 Processor -- start 2024-07-31T14:58:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85e312e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea410861b0 0x55ea4146ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85e312e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea4108e390 0x55ea41467ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85e312e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 0x55ea41468490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85e312e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81b3d70 con 0x55ea410861b0 2024-07-31T14:58:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85e312e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81b35e0 con 0x55ea4108e390 2024-07-31T14:58:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85e312e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81b0790 con 0x55ea410af070 2024-07-31T14:58:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85e312e740 10 client.0 _ll_register_callbacks cb 0x7f85c0010b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85d5d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea4108e390 0x55ea41467ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85d659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea410861b0 0x55ea4146ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85d5d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea4108e390 0x55ea41467ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60726/0 (socket says 172.21.15.67:60726) 2024-07-31T14:58:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85d659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea410861b0 0x55ea4146ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55876/0 (socket says 172.21.15.67:55876) 2024-07-31T14:58:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85d5d9b640 1 -- 172.21.15.67:0/3596275504 learned_addr learned my addr 172.21.15.67:0/3596275504 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85d6d9d640 1 --2- 172.21.15.67:0/3596275504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 0x55ea41468490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85d5d9b640 1 -- 172.21.15.67:0/3596275504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 msgr2=0x55ea41468490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85d5d9b640 1 --2- 172.21.15.67:0/3596275504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 0x55ea41468490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85d5d9b640 1 -- 172.21.15.67:0/3596275504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea410861b0 msgr2=0x55ea4146ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.453+0000 7f85d5d9b640 1 --2- 172.21.15.67:0/3596275504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea410861b0 0x55ea4146ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.454+0000 7f85d5d9b640 1 -- 172.21.15.67:0/3596275504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea41468c20 con 0x55ea4108e390 2024-07-31T14:58:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.454+0000 7f85d5d9b640 1 --2- 172.21.15.67:0/3596275504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea4108e390 0x55ea41467ef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f85c00a79c0 tx=0x7f85c000df10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f1c57d91f2729f1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.454+0000 7f85bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:58:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.454+0000 7f85bdffb640 1 -- 172.21.15.67:0/3596275504 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0021070 con 0x55ea4108e390 2024-07-31T14:58:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.454+0000 7f85bdffb640 1 -- 172.21.15.67:0/3596275504 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f85c00cbdb0 con 0x55ea4108e390 2024-07-31T14:58:00.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.454+0000 7f85e312e740 1 -- 172.21.15.67:0/3596275504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d0036040 con 0x55ea4108e390 2024-07-31T14:58:00.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.454+0000 7f85bdffb640 1 -- 172.21.15.67:0/3596275504 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f85c00c4080 con 0x55ea4108e390 2024-07-31T14:58:00.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.454+0000 7f85e312e740 10 client.26894 fetch_fsmap learned FSMap version 50 2024-07-31T14:58:00.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.454+0000 7f85e312e740 1 -- 172.21.15.67:0/3596275504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55ea414da830 con 0x55ea4108e390 2024-07-31T14:58:00.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.454+0000 7f85bdffb640 1 -- 172.21.15.67:0/3596275504 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85c0009de0 con 0x55ea4108e390 2024-07-31T14:58:00.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.455+0000 7f85bdffb640 1 -- 172.21.15.67:0/3596275504 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f85c00afe30 con 0x55ea4108e390 2024-07-31T14:58:00.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.455+0000 7f85e312e740 10 client.26894 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:58:00.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.455+0000 7f85e312e740 10 client.26894 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:58:00.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.455+0000 7f85e312e740 20 client.26894 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:00.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.455+0000 7f85e312e740 1 --2- 172.21.15.67:0/3596275504 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ea41042da0 0x55ea414dd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:00.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.455+0000 7f85e312e740 4 client.26894 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:58:00.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.455+0000 7f85e312e740 1 -- 172.21.15.67:0/3596275504 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f85d0036040 con 0x55ea41042da0 2024-07-31T14:58:00.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.456+0000 7f85d659c640 1 --2- 172.21.15.67:0/3596275504 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ea41042da0 0x55ea414dd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:00.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.456+0000 7f85d659c640 1 --2- 172.21.15.67:0/3596275504 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ea41042da0 0x55ea414dd020 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a2bc2f4f56b1ce9 server_cookie=53e6c1f6d5f6d30f in_seq=0 out_seq=0 2024-07-31T14:58:00.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.456+0000 7f85bdffb640 10 client.26894 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:58:00.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.456+0000 7f85bdffb640 1 -- 172.21.15.67:0/3596275504 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f85d0036040 con 0x55ea41042da0 2024-07-31T14:58:00.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.456+0000 7f85bdffb640 10 client.26894 handle_command_reply: tid=0 2024-07-31T14:58:00.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.456+0000 7f85e312e740 1 client.26894 shutdown 2024-07-31T14:58:00.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.456+0000 7f85e312e740 1 -- 172.21.15.67:0/3596275504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea4108e390 msgr2=0x55ea41467ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:00.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.456+0000 7f85e312e740 1 --2- 172.21.15.67:0/3596275504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea4108e390 0x55ea41467ef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f85c00a79c0 tx=0x7f85c000df10 comp rx=0 tx=0).stop 2024-07-31T14:58:00.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.457+0000 7f85e312e740 1 -- 172.21.15.67:0/3596275504 shutdown_connections 2024-07-31T14:58:00.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.457+0000 7f85e312e740 1 --2- 172.21.15.67:0/3596275504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea410af070 0x55ea41468490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.457+0000 7f85e312e740 1 --2- 172.21.15.67:0/3596275504 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ea41042da0 0x55ea414dd020 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.457+0000 7f85e312e740 1 --2- 172.21.15.67:0/3596275504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea4108e390 0x55ea41467ef0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.457+0000 7f85e312e740 1 --2- 172.21.15.67:0/3596275504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ea410861b0 0x55ea4146ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.457+0000 7f85e312e740 1 -- 172.21.15.67:0/3596275504 >> 172.21.15.67:0/3596275504 conn(0x55ea410a3460 msgr2=0x55ea41459680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:00.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.457+0000 7f85bdffb640 0 client.26894 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:58:00.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.457+0000 7f85e312e740 1 -- 172.21.15.67:0/3596275504 shutdown_connections 2024-07-31T14:58:00.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.457+0000 7f85e312e740 1 -- 172.21.15.67:0/3596275504 wait complete. 2024-07-31T14:58:00.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.457+0000 7f85e312e740 20 client.26894 trim_cache size 0 max 16384 2024-07-31T14:58:00.464 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:58:00.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.458+0000 7f85dcbdd640 1 -- 172.21.15.67:0/1161624417 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f85c40b3d40 msgr2=0x7f85c40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:00.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.458+0000 7f85dcbdd640 1 --2- 172.21.15.67:0/1161624417 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f85c40b3d40 0x7f85c40b6250 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009da0 tx=0x7f85c8005f90 comp rx=0 tx=0).stop 2024-07-31T14:58:00.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.458+0000 7f85dcbdd640 1 -- 172.21.15.67:0/1161624417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d8107dd0 msgr2=0x7f85d81041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:00.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.458+0000 7f85dcbdd640 1 --2- 172.21.15.67:0/1161624417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d8107dd0 0x7f85d81041c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f85c000d770 tx=0x7f85c000dc30 comp rx=0 tx=0).stop 2024-07-31T14:58:00.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.458+0000 7f85dcbdd640 1 -- 172.21.15.67:0/1161624417 shutdown_connections 2024-07-31T14:58:00.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.458+0000 7f85dcbdd640 1 --2- 172.21.15.67:0/1161624417 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f85c40b3d40 0x7f85c40b6250 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.458+0000 7f85dcbdd640 1 --2- 172.21.15.67:0/1161624417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d8108720 0x7f85d8104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.458+0000 7f85dcbdd640 1 --2- 172.21.15.67:0/1161624417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85d8107dd0 0x7f85d81041c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.458+0000 7f85dcbdd640 1 --2- 172.21.15.67:0/1161624417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d8102490 0x7f85d8103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:00.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.458+0000 7f85dcbdd640 1 -- 172.21.15.67:0/1161624417 >> 172.21.15.67:0/1161624417 conn(0x7f85d8079160 msgr2=0x7f85d8101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:00.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.458+0000 7f85dcbdd640 1 -- 172.21.15.67:0/1161624417 shutdown_connections 2024-07-31T14:58:00.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:00.458+0000 7f85dcbdd640 1 -- 172.21.15.67:0/1161624417 wait complete. 2024-07-31T14:58:01.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:00 smithi073 ceph-mon[36714]: pgmap v608: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.0 MiB/s wr, 1.64k op/s 2024-07-31T14:58:01.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:00 smithi077 ceph-mon[37327]: pgmap v608: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.0 MiB/s wr, 1.64k op/s 2024-07-31T14:58:01.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:00 smithi067 ceph-mon[32750]: pgmap v608: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.0 MiB/s wr, 1.64k op/s 2024-07-31T14:58:02.190 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T14:58:03.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:02 smithi073 ceph-mon[36714]: pgmap v609: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 1.29k op/s 2024-07-31T14:58:03.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:02 smithi077 ceph-mon[37327]: pgmap v609: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 1.29k op/s 2024-07-31T14:58:03.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:02 smithi067 ceph-mon[32750]: pgmap v609: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 1.29k op/s 2024-07-31T14:58:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:58:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.959+0000 7f986e7eb640 1 -- 172.21.15.67:0/1959235773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98681118d0 msgr2=0x7f9868109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.959+0000 7f986e7eb640 1 --2- 172.21.15.67:0/1959235773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98681118d0 0x7f9868109df0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9858009900 tx=0x7f985802a4b0 comp rx=0 tx=0).stop 2024-07-31T14:58:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.961+0000 7f986e7eb640 1 -- 172.21.15.67:0/1959235773 shutdown_connections 2024-07-31T14:58:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.961+0000 7f986e7eb640 1 --2- 172.21.15.67:0/1959235773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f986811c5d0 0x7f986811eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.961+0000 7f986e7eb640 1 --2- 172.21.15.67:0/1959235773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f986810a330 0x7f986810a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.961+0000 7f986e7eb640 1 --2- 172.21.15.67:0/1959235773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98681118d0 0x7f9868109df0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.961+0000 7f986e7eb640 1 -- 172.21.15.67:0/1959235773 >> 172.21.15.67:0/1959235773 conn(0x7f98680707c0 msgr2=0x7f9868070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.962+0000 7f986e7eb640 1 -- 172.21.15.67:0/1959235773 shutdown_connections 2024-07-31T14:58:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.962+0000 7f986e7eb640 1 -- 172.21.15.67:0/1959235773 wait complete. 2024-07-31T14:58:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.964+0000 7f986e7eb640 1 Processor -- start 2024-07-31T14:58:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.964+0000 7f986e7eb640 1 -- start start 2024-07-31T14:58:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.965+0000 7f986e7eb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f986810a330 0x7f986811bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.965+0000 7f986e7eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98681118d0 0x7f9868116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.965+0000 7f986e7eb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f986811c5d0 0x7f98681168f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.965+0000 7f986e7eb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f98681126b0 con 0x7f986811c5d0 2024-07-31T14:58:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.965+0000 7f986e7eb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9868112530 con 0x7f986810a330 2024-07-31T14:58:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.965+0000 7f986e7eb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9868112830 con 0x7f98681118d0 2024-07-31T14:58:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.967+0000 7f9867fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f986810a330 0x7f986811bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.967+0000 7f98677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98681118d0 0x7f9868116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.967+0000 7f98677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98681118d0 0x7f9868116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:56874/0 (socket says 172.21.15.67:56874) 2024-07-31T14:58:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.967+0000 7f98677fe640 1 -- 172.21.15.67:0/4252163761 learned_addr learned my addr 172.21.15.67:0/4252163761 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.968+0000 7f986cd88640 1 --2- 172.21.15.67:0/4252163761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f986811c5d0 0x7f98681168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.969+0000 7f98677fe640 1 -- 172.21.15.67:0/4252163761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f986810a330 msgr2=0x7f986811bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.969+0000 7f98677fe640 1 --2- 172.21.15.67:0/4252163761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f986810a330 0x7f986811bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.969+0000 7f98677fe640 1 -- 172.21.15.67:0/4252163761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f986811c5d0 msgr2=0x7f98681168f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.969+0000 7f98677fe640 1 --2- 172.21.15.67:0/4252163761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f986811c5d0 0x7f98681168f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.969+0000 7f98677fe640 1 -- 172.21.15.67:0/4252163761 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98681bd6c0 con 0x7f98681118d0 2024-07-31T14:58:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.969+0000 7f98677fe640 1 --2- 172.21.15.67:0/4252163761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98681118d0 0x7f9868116240 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f985c00c970 tx=0x7f985c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.970+0000 7f98657fa640 1 -- 172.21.15.67:0/4252163761 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c007bb0 con 0x7f98681118d0 2024-07-31T14:58:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.970+0000 7f98657fa640 1 -- 172.21.15.67:0/4252163761 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f985c007d50 con 0x7f98681118d0 2024-07-31T14:58:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.970+0000 7f98657fa640 1 -- 172.21.15.67:0/4252163761 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c00f450 con 0x7f98681118d0 2024-07-31T14:58:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.971+0000 7f986e7eb640 1 -- 172.21.15.67:0/4252163761 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98681bd890 con 0x7f98681118d0 2024-07-31T14:58:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.972+0000 7f986e7eb640 1 -- 172.21.15.67:0/4252163761 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98681bdd60 con 0x7f98681118d0 2024-07-31T14:58:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.980+0000 7f98657fa640 1 -- 172.21.15.67:0/4252163761 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f985c0046c0 con 0x7f98681118d0 2024-07-31T14:58:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.982+0000 7f98657fa640 1 --2- 172.21.15.67:0/4252163761 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98500b3ed0 0x7f98500b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.982+0000 7f9867fff640 1 --2- 172.21.15.67:0/4252163761 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98500b3ed0 0x7f98500b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.982+0000 7f98757c0740 1 Processor -- start 2024-07-31T14:58:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.982+0000 7f98657fa640 1 -- 172.21.15.67:0/4252163761 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f985c0e43e0 con 0x7f98681118d0 2024-07-31T14:58:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.983+0000 7f98757c0740 1 -- start start 2024-07-31T14:58:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.983+0000 7f98757c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767561b0 0x564176af86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.983+0000 7f98757c0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 0x564176af8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.984+0000 7f98757c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767354d0 0x564176af9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.984+0000 7f98757c0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564176a6e3b0 con 0x56417672d2f0 2024-07-31T14:58:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.984+0000 7f98757c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564176a5d8b0 con 0x5641767354d0 2024-07-31T14:58:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.984+0000 7f986cd88640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767354d0 0x564176af9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.984+0000 7f986cd88640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767354d0 0x564176af9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46052/0 (socket says 172.21.15.67:46052) 2024-07-31T14:58:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.984+0000 7f986cd88640 1 -- 172.21.15.67:0/4005920394 learned_addr learned my addr 172.21.15.67:0/4005920394 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.984+0000 7f98757c0740 1 -- 172.21.15.67:0/4005920394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564176a618f0 con 0x5641767561b0 2024-07-31T14:58:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.984+0000 7f98677fe640 1 --2- 172.21.15.67:0/4005920394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 0x564176af8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.984+0000 7f9867fff640 1 --2- 172.21.15.67:0/4252163761 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98500b3ed0 0x7f98500b63e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f98580058c0 tx=0x7f9858005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.985+0000 7f986cd88640 1 -- 172.21.15.67:0/4005920394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767561b0 msgr2=0x564176af86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.985+0000 7f986cd88640 1 --2- 172.21.15.67:0/4005920394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767561b0 0x564176af86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.985+0000 7f986cd88640 1 -- 172.21.15.67:0/4005920394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 msgr2=0x564176af8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.985+0000 7f986cd88640 1 --2- 172.21.15.67:0/4005920394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 0x564176af8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.985+0000 7f986cd88640 1 -- 172.21.15.67:0/4005920394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564176a049a0 con 0x5641767354d0 2024-07-31T14:58:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.986+0000 7f986cd88640 1 --2- 172.21.15.67:0/4005920394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767354d0 0x564176af9130 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9860009e90 tx=0x7f9860010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fefb72c1606c081e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.988+0000 7f984e7fc640 1 -- 172.21.15.67:0/4005920394 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860016070 con 0x5641767354d0 2024-07-31T14:58:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.988+0000 7f984e7fc640 1 -- 172.21.15.67:0/4005920394 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9860004d20 con 0x5641767354d0 2024-07-31T14:58:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.988+0000 7f984e7fc640 1 -- 172.21.15.67:0/4005920394 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860012740 con 0x5641767354d0 2024-07-31T14:58:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.990+0000 7f98757c0740 1 -- 172.21.15.67:0/4005920394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767354d0 msgr2=0x564176af9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.990+0000 7f98757c0740 1 --2- 172.21.15.67:0/4005920394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767354d0 0x564176af9130 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9860009e90 tx=0x7f9860010740 comp rx=0 tx=0).stop 2024-07-31T14:58:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.992+0000 7f98757c0740 1 -- 172.21.15.67:0/4005920394 shutdown_connections 2024-07-31T14:58:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.992+0000 7f98757c0740 1 --2- 172.21.15.67:0/4005920394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767354d0 0x564176af9130 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.992+0000 7f98757c0740 1 --2- 172.21.15.67:0/4005920394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 0x564176af8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.992+0000 7f98757c0740 1 --2- 172.21.15.67:0/4005920394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767561b0 0x564176af86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.992+0000 7f98757c0740 1 -- 172.21.15.67:0/4005920394 >> 172.21.15.67:0/4005920394 conn(0x56417674a300 msgr2=0x564176af65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.997+0000 7f98757c0740 1 -- 172.21.15.67:0/4005920394 shutdown_connections 2024-07-31T14:58:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.997+0000 7f98757c0740 1 -- 172.21.15.67:0/4005920394 wait complete. 2024-07-31T14:58:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.998+0000 7f98757c0740 1 -- start start 2024-07-31T14:58:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:03.999+0000 7f98757c0740 1 Processor -- start 2024-07-31T14:58:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.000+0000 7f98757c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56417672d2f0 0x564176b042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.000+0000 7f98757c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767354d0 0x564176b04890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.000+0000 7f9867fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56417672d2f0 0x564176b042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.000+0000 7f9867fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56417672d2f0 0x564176b042f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:56888/0 (socket says 172.21.15.67:56888) 2024-07-31T14:58:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.000+0000 7f9867fff640 1 -- 172.21.15.67:0/2090796871 learned_addr learned my addr 172.21.15.67:0/2090796871 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.000+0000 7f98757c0740 1 --2- 172.21.15.67:0/2090796871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641767561b0 0x564176b10150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.001+0000 7f98757c0740 1 -- 172.21.15.67:0/2090796871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564176997ab0 con 0x5641767561b0 2024-07-31T14:58:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.001+0000 7f98757c0740 1 -- 172.21.15.67:0/2090796871 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5641769e5cf0 con 0x5641767354d0 2024-07-31T14:58:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.001+0000 7f98757c0740 1 -- 172.21.15.67:0/2090796871 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564176b06e30 con 0x56417672d2f0 2024-07-31T14:58:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.001+0000 7f986cd88640 1 --2- 172.21.15.67:0/2090796871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641767561b0 0x564176b10150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.001+0000 7f98757c0740 10 client.0 _ll_register_callbacks cb 0x564176a573e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.001+0000 7f98677fe640 1 --2- 172.21.15.67:0/2090796871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767354d0 0x564176b04890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.001+0000 7f9867fff640 1 -- 172.21.15.67:0/2090796871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767354d0 msgr2=0x564176b04890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.001+0000 7f9867fff640 1 --2- 172.21.15.67:0/2090796871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767354d0 0x564176b04890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.001+0000 7f9867fff640 1 -- 172.21.15.67:0/2090796871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641767561b0 msgr2=0x564176b10150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.001+0000 7f9867fff640 1 --2- 172.21.15.67:0/2090796871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641767561b0 0x564176b10150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.001+0000 7f9867fff640 1 -- 172.21.15.67:0/2090796871 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641767c1790 con 0x56417672d2f0 2024-07-31T14:58:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.002+0000 7f98757c0740 1 -- 172.21.15.67:0/2090796871 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9860004d20 con 0x56417672d2f0 2024-07-31T14:58:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.002+0000 7f9867fff640 1 --2- 172.21.15.67:0/2090796871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56417672d2f0 0x564176b042f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f98580096c0 tx=0x7f98580052c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a3b38e3633d4602 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.002+0000 7f98477fe640 10 client.26902 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:58:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.002+0000 7f98477fe640 1 -- 172.21.15.67:0/2090796871 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858035050 con 0x56417672d2f0 2024-07-31T14:58:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.003+0000 7f98477fe640 1 -- 172.21.15.67:0/2090796871 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f9858004cd0 con 0x56417672d2f0 2024-07-31T14:58:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.003+0000 7f98477fe640 1 -- 172.21.15.67:0/2090796871 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f98580035f0 con 0x56417672d2f0 2024-07-31T14:58:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.003+0000 7f98757c0740 10 client.26902 fetch_fsmap learned FSMap version 50 2024-07-31T14:58:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.003+0000 7f98757c0740 1 -- 172.21.15.67:0/2090796871 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=50}) -- 0x56417683a620 con 0x56417672d2f0 2024-07-31T14:58:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.006+0000 7f98477fe640 1 -- 172.21.15.67:0/2090796871 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f9858003790 con 0x56417672d2f0 2024-07-31T14:58:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.006+0000 7f98757c0740 10 client.26902 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:58:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.007+0000 7f98757c0740 10 client.26902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:58:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.007+0000 7f98757c0740 20 client.26902 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.007+0000 7f98757c0740 1 --2- 172.21.15.67:0/2090796871 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564176714d80 0x564176b16100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.007+0000 7f98757c0740 4 client.26902 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:58:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.007+0000 7f98757c0740 1 -- 172.21.15.67:0/2090796871 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564176a70300 con 0x564176714d80 2024-07-31T14:58:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.008+0000 7f98677fe640 1 --2- 172.21.15.67:0/2090796871 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564176714d80 0x564176b16100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.011+0000 7f98677fe640 1 --2- 172.21.15.67:0/2090796871 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564176714d80 0x564176b16100 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9296435d441b6cb server_cookie=b24e7f94b5fca283 in_seq=0 out_seq=0 2024-07-31T14:58:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.011+0000 7f98477fe640 10 client.26902 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.014+0000 7f98477fe640 1 -- 172.21.15.67:0/2090796871 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564176a70300 con 0x564176714d80 2024-07-31T14:58:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.014+0000 7f98477fe640 10 client.26902 handle_command_reply: tid=0 2024-07-31T14:58:04.017 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:03 smithi067 ceph-mon[32750]: osdmap e91: 12 total, 12 up, 12 in 2024-07-31T14:58:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.016+0000 7f98757c0740 1 client.26902 shutdown 2024-07-31T14:58:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.017+0000 7f98757c0740 1 -- 172.21.15.67:0/2090796871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56417672d2f0 msgr2=0x564176b042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.018+0000 7f98757c0740 1 --2- 172.21.15.67:0/2090796871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56417672d2f0 0x564176b042f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f98580096c0 tx=0x7f98580052c0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.018+0000 7f98757c0740 1 -- 172.21.15.67:0/2090796871 shutdown_connections 2024-07-31T14:58:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.018+0000 7f98757c0740 1 --2- 172.21.15.67:0/2090796871 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564176714d80 0x564176b16100 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.018+0000 7f98757c0740 1 --2- 172.21.15.67:0/2090796871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641767561b0 0x564176b10150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.018+0000 7f98757c0740 1 --2- 172.21.15.67:0/2090796871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767354d0 0x564176b04890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.018+0000 7f98757c0740 1 --2- 172.21.15.67:0/2090796871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56417672d2f0 0x564176b042f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.018+0000 7f98757c0740 1 -- 172.21.15.67:0/2090796871 >> 172.21.15.67:0/2090796871 conn(0x56417674a300 msgr2=0x564176af4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.019+0000 7f98477fe640 0 client.26902 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.021+0000 7f98757c0740 1 -- 172.21.15.67:0/2090796871 shutdown_connections 2024-07-31T14:58:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.021+0000 7f98757c0740 1 -- 172.21.15.67:0/2090796871 wait complete. 2024-07-31T14:58:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.023+0000 7f98757c0740 20 client.26902 trim_cache size 0 max 16384 2024-07-31T14:58:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.065+0000 7f98757c0740 1 Processor -- start 2024-07-31T14:58:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.065+0000 7f98757c0740 1 -- start start 2024-07-31T14:58:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.065+0000 7f98757c0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 0x564176b0ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.065+0000 7f98757c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767354d0 0x564176afd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.065+0000 7f98757c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767561b0 0x564176afdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.065+0000 7f98757c0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f98681c8020 con 0x56417672d2f0 2024-07-31T14:58:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.065+0000 7f98757c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f98681c9cb0 con 0x5641767561b0 2024-07-31T14:58:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.065+0000 7f98757c0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f98681c8780 con 0x5641767354d0 2024-07-31T14:58:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.066+0000 7f98677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767354d0 0x564176afd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.066+0000 7f98677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767354d0 0x564176afd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:56898/0 (socket says 172.21.15.67:56898) 2024-07-31T14:58:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.066+0000 7f98677fe640 1 -- 172.21.15.67:0/2339697090 learned_addr learned my addr 172.21.15.67:0/2339697090 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.067+0000 7f98677fe640 1 -- 172.21.15.67:0/2339697090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767561b0 msgr2=0x564176afdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.067+0000 7f98677fe640 1 --2- 172.21.15.67:0/2339697090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767561b0 0x564176afdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.067+0000 7f98677fe640 1 -- 172.21.15.67:0/2339697090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 msgr2=0x564176b0ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.067+0000 7f9867fff640 1 --2- 172.21.15.67:0/2339697090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 0x564176b0ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.067+0000 7f98677fe640 1 --2- 172.21.15.67:0/2339697090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 0x564176b0ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.067+0000 7f98677fe640 1 -- 172.21.15.67:0/2339697090 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564176707870 con 0x5641767354d0 2024-07-31T14:58:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.067+0000 7f9867fff640 1 --2- 172.21.15.67:0/2339697090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 0x564176b0ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:58:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.067+0000 7f98677fe640 1 --2- 172.21.15.67:0/2339697090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767354d0 0x564176afd770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f985c0057a0 tx=0x7f985c0049d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=931af261cfd498c4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.068+0000 7f984dffb640 1 -- 172.21.15.67:0/2339697090 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c0ab350 con 0x5641767354d0 2024-07-31T14:58:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.068+0000 7f984dffb640 1 -- 172.21.15.67:0/2339697090 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f985c0aac50 con 0x5641767354d0 2024-07-31T14:58:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.069+0000 7f984dffb640 1 -- 172.21.15.67:0/2339697090 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c0e1990 con 0x5641767354d0 2024-07-31T14:58:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.070+0000 7f98757c0740 1 -- 172.21.15.67:0/2339697090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767354d0 msgr2=0x564176afd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.070+0000 7f98757c0740 1 --2- 172.21.15.67:0/2339697090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767354d0 0x564176afd770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f985c0057a0 tx=0x7f985c0049d0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.073+0000 7f98757c0740 1 -- 172.21.15.67:0/2339697090 shutdown_connections 2024-07-31T14:58:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.073+0000 7f98757c0740 1 --2- 172.21.15.67:0/2339697090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767561b0 0x564176afdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.073+0000 7f98757c0740 1 --2- 172.21.15.67:0/2339697090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767354d0 0x564176afd770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.073+0000 7f98757c0740 1 --2- 172.21.15.67:0/2339697090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 0x564176b0ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.073+0000 7f98757c0740 1 -- 172.21.15.67:0/2339697090 >> 172.21.15.67:0/2339697090 conn(0x56417674a300 msgr2=0x564176af4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.078+0000 7f98757c0740 1 -- 172.21.15.67:0/2339697090 shutdown_connections 2024-07-31T14:58:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.078+0000 7f98757c0740 1 -- 172.21.15.67:0/2339697090 wait complete. 2024-07-31T14:58:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.078+0000 7f98757c0740 1 -- start start 2024-07-31T14:58:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.078+0000 7f98757c0740 1 Processor -- start 2024-07-31T14:58:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.079+0000 7f98757c0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 0x564176b0c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.079+0000 7f98757c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767354d0 0x564176b0c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.079+0000 7f98757c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767561b0 0x564176b091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.079+0000 7f98757c0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f98681c95e0 con 0x56417672d2f0 2024-07-31T14:58:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.079+0000 7f98757c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f98681c8e50 con 0x5641767561b0 2024-07-31T14:58:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.079+0000 7f98757c0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f98681c6020 con 0x5641767354d0 2024-07-31T14:58:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.079+0000 7f98757c0740 10 client.0 _ll_register_callbacks cb 0x7f9858009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.080+0000 7f98677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767354d0 0x564176b0c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.080+0000 7f98677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767354d0 0x564176b0c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:56912/0 (socket says 172.21.15.67:56912) 2024-07-31T14:58:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.080+0000 7f98677fe640 1 -- 172.21.15.67:0/2135886646 learned_addr learned my addr 172.21.15.67:0/2135886646 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.080+0000 7f98677fe640 1 -- 172.21.15.67:0/2135886646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767561b0 msgr2=0x564176b091e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.080+0000 7f98677fe640 1 --2- 172.21.15.67:0/2135886646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767561b0 0x564176b091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.080+0000 7f98677fe640 1 -- 172.21.15.67:0/2135886646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 msgr2=0x564176b0c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.080+0000 7f98677fe640 1 --2- 172.21.15.67:0/2135886646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 0x564176b0c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.080+0000 7f98677fe640 1 -- 172.21.15.67:0/2135886646 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564176b09950 con 0x5641767354d0 2024-07-31T14:58:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.081+0000 7f98677fe640 1 --2- 172.21.15.67:0/2135886646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767354d0 0x564176b0c930 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f985c0058c0 tx=0x7f985c0e16e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=880e940149f5a264 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.081+0000 7f98477fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:58:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.081+0000 7f98477fe640 1 -- 172.21.15.67:0/2135886646 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c0b8060 con 0x5641767354d0 2024-07-31T14:58:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.082+0000 7f98757c0740 1 -- 172.21.15.67:0/2135886646 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f985c0aac50 con 0x5641767354d0 2024-07-31T14:58:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.082+0000 7f98477fe640 1 -- 172.21.15.67:0/2135886646 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f985c0a9eb0 con 0x5641767354d0 2024-07-31T14:58:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.082+0000 7f98477fe640 1 -- 172.21.15.67:0/2135886646 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f985c0e1e60 con 0x5641767354d0 2024-07-31T14:58:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.083+0000 7f98757c0740 10 client.26908 fetch_fsmap learned FSMap version 50 2024-07-31T14:58:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.084+0000 7f98757c0740 1 -- 172.21.15.67:0/2135886646 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564176b7d380 con 0x5641767354d0 2024-07-31T14:58:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.084+0000 7f98477fe640 1 -- 172.21.15.67:0/2135886646 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f985c0aac50 con 0x5641767354d0 2024-07-31T14:58:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.085+0000 7f98477fe640 1 -- 172.21.15.67:0/2135886646 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f985c0c1040 con 0x5641767354d0 2024-07-31T14:58:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.086+0000 7f98757c0740 10 client.26908 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:58:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.086+0000 7f98757c0740 10 client.26908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:58:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.086+0000 7f98757c0740 20 client.26908 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.086+0000 7f98757c0740 1 --2- 172.21.15.67:0/2135886646 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5641766ea1c0 0x564176b7fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.086+0000 7f98757c0740 4 client.26908 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:58:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.086+0000 7f98757c0740 1 -- 172.21.15.67:0/2135886646 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564176a6bda0 con 0x5641766ea1c0 2024-07-31T14:58:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.087+0000 7f9867fff640 1 --2- 172.21.15.67:0/2135886646 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5641766ea1c0 0x564176b7fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.087+0000 7f9867fff640 1 --2- 172.21.15.67:0/2135886646 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5641766ea1c0 0x564176b7fbd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8192b67387f9e83 server_cookie=cf02daf34ce149b8 in_seq=0 out_seq=0 2024-07-31T14:58:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:04.087+0000 7f98477fe640 10 client.26908 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:04.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:03 smithi077 ceph-mon[37327]: osdmap e91: 12 total, 12 up, 12 in 2024-07-31T14:58:04.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:03 smithi073 ceph-mon[36714]: osdmap e91: 12 total, 12 up, 12 in 2024-07-31T14:58:04.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.401+0000 7f6902f21640 1 -- 172.21.15.67:0/1011783941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68f4095030 msgr2=0x7f68f4095440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:04.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.401+0000 7f6902f21640 1 --2- 172.21.15.67:0/1011783941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68f4095030 0x7f68f4095440 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00fb60 tx=0x7f68ec02e4c0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.402+0000 7f6902f21640 1 -- 172.21.15.67:0/1011783941 shutdown_connections 2024-07-31T14:58:04.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.402+0000 7f6902f21640 1 --2- 172.21.15.67:0/1011783941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68f409b2c0 0x7f68f40a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.402+0000 7f6902f21640 1 --2- 172.21.15.67:0/1011783941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68f409a970 0x7f68f409ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.402+0000 7f6902f21640 1 --2- 172.21.15.67:0/1011783941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68f4095030 0x7f68f4095440 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.402+0000 7f6902f21640 1 -- 172.21.15.67:0/1011783941 >> 172.21.15.67:0/1011783941 conn(0x7f68f400cd80 msgr2=0x7f68f400d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:04.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.403+0000 7f6902f21640 1 -- 172.21.15.67:0/1011783941 shutdown_connections 2024-07-31T14:58:04.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.403+0000 7f6902f21640 1 -- 172.21.15.67:0/1011783941 wait complete. 2024-07-31T14:58:04.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.404+0000 7f6902f21640 1 Processor -- start 2024-07-31T14:58:04.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.410+0000 7f6902f21640 1 -- start start 2024-07-31T14:58:04.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.410+0000 7f6902f21640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68f4095030 0x7f68f4144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.410+0000 7f6902f21640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68f409a970 0x7f68f41445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.410+0000 7f6902f21640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68f409b2c0 0x7f68f4149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.410+0000 7f6902f21640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f68f40ac670 con 0x7f68f409a970 2024-07-31T14:58:04.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.410+0000 7f6902f21640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f68f40ac4f0 con 0x7f68f4095030 2024-07-31T14:58:04.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.410+0000 7f6902f21640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68f40ac7f0 con 0x7f68f409b2c0 2024-07-31T14:58:04.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.411+0000 7f6901f1f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68f4095030 0x7f68f4144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.412+0000 7f6901f1f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68f4095030 0x7f68f4144010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46102/0 (socket says 172.21.15.67:46102) 2024-07-31T14:58:04.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.412+0000 7f6901f1f640 1 -- 172.21.15.67:0/996968272 learned_addr learned my addr 172.21.15.67:0/996968272 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:04.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.413+0000 7f6901f1f640 1 -- 172.21.15.67:0/996968272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68f409b2c0 msgr2=0x7f68f4149850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T14:58:04.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.413+0000 7f6901f1f640 1 --2- 172.21.15.67:0/996968272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68f409b2c0 0x7f68f4149850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.413+0000 7f6901f1f640 1 -- 172.21.15.67:0/996968272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68f409a970 msgr2=0x7f68f41445b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T14:58:04.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.413+0000 7f6901f1f640 1 --2- 172.21.15.67:0/996968272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68f409a970 0x7f68f41445b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.413+0000 7f6901f1f640 1 -- 172.21.15.67:0/996968272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f4149f50 con 0x7f68f4095030 2024-07-31T14:58:04.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.413+0000 7f6901f1f640 1 --2- 172.21.15.67:0/996968272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68f4095030 0x7f68f4144010 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00fc80 tx=0x7f68ec034cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:04.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.416+0000 7f68f2ffd640 1 -- 172.21.15.67:0/996968272 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec004240 con 0x7f68f4095030 2024-07-31T14:58:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.416+0000 7f68f2ffd640 1 -- 172.21.15.67:0/996968272 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68ec043070 con 0x7f68f4095030 2024-07-31T14:58:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.416+0000 7f6902f21640 1 -- 172.21.15.67:0/996968272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f414a1e0 con 0x7f68f4095030 2024-07-31T14:58:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.418+0000 7f68f2ffd640 1 -- 172.21.15.67:0/996968272 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec003a80 con 0x7f68f4095030 2024-07-31T14:58:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.418+0000 7f6902f21640 1 -- 172.21.15.67:0/996968272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f414a690 con 0x7f68f4095030 2024-07-31T14:58:04.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.422+0000 7f690a6d4740 1 Processor -- start 2024-07-31T14:58:04.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.423+0000 7f68f2ffd640 1 -- 172.21.15.67:0/996968272 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f68ec003c20 con 0x7f68f4095030 2024-07-31T14:58:04.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.426+0000 7f690a6d4740 1 -- start start 2024-07-31T14:58:04.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.426+0000 7f690a6d4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83bae7840 0x55b83be8da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.426+0000 7f690a6d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83babe980 0x55b83be8dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.426+0000 7f690a6d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83bac6b60 0x55b83be8e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.426+0000 7f690a6d4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b83be027b0 con 0x55b83bae7840 2024-07-31T14:58:04.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.426+0000 7f690a6d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b83bdf1cb0 con 0x55b83babe980 2024-07-31T14:58:04.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.426+0000 7f690a6d4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b83bdf5cf0 con 0x55b83bac6b60 2024-07-31T14:58:04.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.427+0000 7f68f2ffd640 1 --2- 172.21.15.67:0/996968272 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68e80b3ca0 0x7f68e80b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.427+0000 7f68f2ffd640 1 -- 172.21.15.67:0/996968272 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f68ec104f60 con 0x7f68f4095030 2024-07-31T14:58:04.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.428+0000 7f6902720640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83bae7840 0x55b83be8da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.429+0000 7f6902720640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83bae7840 0x55b83be8da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34674/0 (socket says 172.21.15.67:34674) 2024-07-31T14:58:04.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.429+0000 7f6902720640 1 -- 172.21.15.67:0/3632685876 learned_addr learned my addr 172.21.15.67:0/3632685876 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:04.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.429+0000 7f6902720640 1 --2- 172.21.15.67:0/996968272 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68e80b3ca0 0x7f68e80b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.429+0000 7f690171e640 1 --2- 172.21.15.67:0/3632685876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83bac6b60 0x55b83be8e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.429+0000 7f6902720640 1 -- 172.21.15.67:0/3632685876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83bac6b60 msgr2=0x55b83be8e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:04.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.429+0000 7f6902720640 1 --2- 172.21.15.67:0/3632685876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83bac6b60 0x55b83be8e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.429+0000 7f6902720640 1 -- 172.21.15.67:0/3632685876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83babe980 msgr2=0x55b83be8dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:04.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.429+0000 7f6902720640 1 --2- 172.21.15.67:0/3632685876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83babe980 0x55b83be8dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.429+0000 7f6902720640 1 -- 172.21.15.67:0/3632685876 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b83bd870b0 con 0x55b83bae7840 2024-07-31T14:58:04.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.430+0000 7f6902720640 1 --2- 172.21.15.67:0/996968272 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68e80b3ca0 0x7f68e80b61b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f68f800d520 tx=0x7f68f800d3e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:04.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.432+0000 7f6902720640 1 --2- 172.21.15.67:0/3632685876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83bae7840 0x55b83be8da60 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009eb0 tx=0x7f68f801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de9e8c2f7b9c3c3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:04.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.433+0000 7f68e7fff640 1 -- 172.21.15.67:0/3632685876 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8005020 con 0x55b83bae7840 2024-07-31T14:58:04.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.433+0000 7f68e7fff640 1 -- 172.21.15.67:0/3632685876 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68f801f070 con 0x55b83bae7840 2024-07-31T14:58:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.434+0000 7f690a6d4740 1 -- 172.21.15.67:0/3632685876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83bae7840 msgr2=0x55b83be8da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.434+0000 7f690a6d4740 1 --2- 172.21.15.67:0/3632685876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83bae7840 0x55b83be8da60 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009eb0 tx=0x7f68f801c000 comp rx=0 tx=0).stop 2024-07-31T14:58:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.434+0000 7f690a6d4740 1 -- 172.21.15.67:0/3632685876 shutdown_connections 2024-07-31T14:58:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.434+0000 7f690a6d4740 1 --2- 172.21.15.67:0/3632685876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83bac6b60 0x55b83be8e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.434+0000 7f690a6d4740 1 --2- 172.21.15.67:0/3632685876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83babe980 0x55b83be8dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.434+0000 7f690a6d4740 1 --2- 172.21.15.67:0/3632685876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83bae7840 0x55b83be8da60 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.434+0000 7f690a6d4740 1 -- 172.21.15.67:0/3632685876 >> 172.21.15.67:0/3632685876 conn(0x55b83badb990 msgr2=0x55b83be8b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.436+0000 7f690a6d4740 1 -- 172.21.15.67:0/3632685876 shutdown_connections 2024-07-31T14:58:04.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.437+0000 7f690a6d4740 1 -- 172.21.15.67:0/3632685876 wait complete. 2024-07-31T14:58:04.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.438+0000 7f690a6d4740 1 -- start start 2024-07-31T14:58:04.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.438+0000 7f690a6d4740 1 Processor -- start 2024-07-31T14:58:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.440+0000 7f690a6d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83babe980 0x55b83be9b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.440+0000 7f690a6d4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83bac6b60 0x55b83be98ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.440+0000 7f690a6d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83baa61c0 0x55b83be99340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.440+0000 7f690a6d4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b83bc557a0 con 0x55b83bac6b60 2024-07-31T14:58:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.440+0000 7f690a6d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b83bd94320 con 0x55b83baa61c0 2024-07-31T14:58:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.440+0000 7f690a6d4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b83be9c280 con 0x55b83babe980 2024-07-31T14:58:04.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.440+0000 7f690a6d4740 10 client.0 _ll_register_callbacks cb 0x55b83bd9e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:04.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.441+0000 7f690171e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83bac6b60 0x55b83be98ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.441+0000 7f690171e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83bac6b60 0x55b83be98ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34688/0 (socket says 172.21.15.67:34688) 2024-07-31T14:58:04.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.441+0000 7f690171e640 1 -- 172.21.15.67:0/504002796 learned_addr learned my addr 172.21.15.67:0/504002796 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:04.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.441+0000 7f6902720640 1 --2- 172.21.15.67:0/504002796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83baa61c0 0x55b83be99340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.441+0000 7f6901f1f640 1 --2- 172.21.15.67:0/504002796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83babe980 0x55b83be9b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.442+0000 7f6902720640 1 -- 172.21.15.67:0/504002796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83babe980 msgr2=0x55b83be9b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:04.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.442+0000 7f6902720640 1 --2- 172.21.15.67:0/504002796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83babe980 0x55b83be9b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.442+0000 7f6902720640 1 -- 172.21.15.67:0/504002796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83bac6b60 msgr2=0x55b83be98ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:04.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.442+0000 7f6902720640 1 --2- 172.21.15.67:0/504002796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83bac6b60 0x55b83be98ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.442+0000 7f6902720640 1 -- 172.21.15.67:0/504002796 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b83bd984b0 con 0x55b83baa61c0 2024-07-31T14:58:04.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.443+0000 7f6902720640 1 --2- 172.21.15.67:0/504002796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83baa61c0 0x55b83be99340 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009eb0 tx=0x7f68f8003840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5c326e66753f9c3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:04.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.443+0000 7f68e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:58:04.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.444+0000 7f68e57fa640 1 -- 172.21.15.67:0/504002796 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8003e40 con 0x55b83baa61c0 2024-07-31T14:58:04.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.444+0000 7f68e57fa640 1 -- 172.21.15.67:0/504002796 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f68f8003c40 con 0x55b83baa61c0 2024-07-31T14:58:04.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.444+0000 7f690a6d4740 1 -- 172.21.15.67:0/504002796 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b83be04700 con 0x55b83baa61c0 2024-07-31T14:58:04.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.445+0000 7f68e57fa640 1 -- 172.21.15.67:0/504002796 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f68f8003e40 con 0x55b83baa61c0 2024-07-31T14:58:04.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.446+0000 7f690a6d4740 10 client.26906 fetch_fsmap learned FSMap version 50 2024-07-31T14:58:04.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.446+0000 7f690a6d4740 1 -- 172.21.15.67:0/504002796 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55b83bb61b90 con 0x55b83baa61c0 2024-07-31T14:58:04.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.447+0000 7f68e57fa640 1 -- 172.21.15.67:0/504002796 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b83be04700 con 0x55b83baa61c0 2024-07-31T14:58:04.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.447+0000 7f68e57fa640 1 -- 172.21.15.67:0/504002796 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f68f802bb00 con 0x55b83baa61c0 2024-07-31T14:58:04.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.447+0000 7f690a6d4740 10 client.26906 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:58:04.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.447+0000 7f690a6d4740 10 client.26906 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:58:04.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.447+0000 7f690a6d4740 20 client.26906 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:04.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.447+0000 7f690a6d4740 1 --2- 172.21.15.67:0/504002796 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b83ba7b570 0x55b83beaaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.447+0000 7f690a6d4740 4 client.26906 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:58:04.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.447+0000 7f690a6d4740 1 -- 172.21.15.67:0/504002796 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b83be001a0 con 0x55b83ba7b570 2024-07-31T14:58:04.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.451+0000 7f6901f1f640 1 --2- 172.21.15.67:0/504002796 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b83ba7b570 0x55b83beaaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.458+0000 7f6901f1f640 1 --2- 172.21.15.67:0/504002796 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b83ba7b570 0x55b83beaaef0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a52405152b05eda7 server_cookie=572078474cf0bb23 in_seq=0 out_seq=0 2024-07-31T14:58:04.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.458+0000 7f68e57fa640 10 client.26906 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.720+0000 7f68e57fa640 1 -- 172.21.15.67:0/504002796 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b83be001a0 con 0x55b83ba7b570 2024-07-31T14:58:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.720+0000 7f68e57fa640 10 client.26906 handle_command_reply: tid=0 2024-07-31T14:58:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.721+0000 7f690a6d4740 1 client.26906 shutdown 2024-07-31T14:58:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.721+0000 7f690a6d4740 1 -- 172.21.15.67:0/504002796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83baa61c0 msgr2=0x55b83be99340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.721+0000 7f690a6d4740 1 --2- 172.21.15.67:0/504002796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83baa61c0 0x55b83be99340 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009eb0 tx=0x7f68f8003840 comp rx=0 tx=0).stop 2024-07-31T14:58:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.721+0000 7f690a6d4740 1 -- 172.21.15.67:0/504002796 shutdown_connections 2024-07-31T14:58:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.721+0000 7f690a6d4740 1 --2- 172.21.15.67:0/504002796 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b83ba7b570 0x55b83beaaef0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.721+0000 7f68e57fa640 0 client.26906 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.721+0000 7f690a6d4740 1 --2- 172.21.15.67:0/504002796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83baa61c0 0x55b83be99340 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.721+0000 7f690a6d4740 1 --2- 172.21.15.67:0/504002796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83bac6b60 0x55b83be98ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.721+0000 7f690a6d4740 1 --2- 172.21.15.67:0/504002796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83babe980 0x55b83be9b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.721+0000 7f690a6d4740 1 -- 172.21.15.67:0/504002796 >> 172.21.15.67:0/504002796 conn(0x55b83badb990 msgr2=0x55b83be90c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.722+0000 7f690a6d4740 1 -- 172.21.15.67:0/504002796 shutdown_connections 2024-07-31T14:58:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.722+0000 7f690a6d4740 1 -- 172.21.15.67:0/504002796 wait complete. 2024-07-31T14:58:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.722+0000 7f690a6d4740 20 client.26906 trim_cache size 0 max 16384 2024-07-31T14:58:04.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.759+0000 7f690a6d4740 1 Processor -- start 2024-07-31T14:58:04.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.760+0000 7f690a6d4740 1 -- start start 2024-07-31T14:58:04.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.761+0000 7f690a6d4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83baa61c0 0x55b83be8af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.761+0000 7f690a6d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83babe980 0x55b83be8b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.762+0000 7f6901f1f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83baa61c0 0x55b83be8af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.762+0000 7f6901f1f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83baa61c0 0x55b83be8af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34690/0 (socket says 172.21.15.67:34690) 2024-07-31T14:58:04.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.762+0000 7f6901f1f640 1 -- 172.21.15.67:0/2750391816 learned_addr learned my addr 172.21.15.67:0/2750391816 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:04.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.762+0000 7f690171e640 1 --2- 172.21.15.67:0/2750391816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83babe980 0x55b83be8b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.763+0000 7f690a6d4740 1 --2- 172.21.15.67:0/2750391816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83bac6b60 0x55b83be95e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.763+0000 7f690a6d4740 1 -- 172.21.15.67:0/2750391816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f68f41631c0 con 0x55b83baa61c0 2024-07-31T14:58:04.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.763+0000 7f690a6d4740 1 -- 172.21.15.67:0/2750391816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f68f4164e50 con 0x55b83bac6b60 2024-07-31T14:58:04.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.764+0000 7f690a6d4740 1 -- 172.21.15.67:0/2750391816 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68f4163920 con 0x55b83babe980 2024-07-31T14:58:04.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.765+0000 7f6902720640 1 --2- 172.21.15.67:0/2750391816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83bac6b60 0x55b83be95e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.769+0000 7f6901f1f640 1 -- 172.21.15.67:0/2750391816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83babe980 msgr2=0x55b83be8b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.769+0000 7f6901f1f640 1 --2- 172.21.15.67:0/2750391816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83babe980 0x55b83be8b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.769+0000 7f6901f1f640 1 -- 172.21.15.67:0/2750391816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83bac6b60 msgr2=0x55b83be95e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.769+0000 7f6901f1f640 1 --2- 172.21.15.67:0/2750391816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83bac6b60 0x55b83be95e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.769+0000 7f6901f1f640 1 -- 172.21.15.67:0/2750391816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b83be9c120 con 0x55b83baa61c0 2024-07-31T14:58:04.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.785+0000 7f6901f1f640 1 --2- 172.21.15.67:0/2750391816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83baa61c0 0x55b83be8af20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f68ec102350 tx=0x7f68ec00f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f39fe95d40c13b2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:04.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.790+0000 7f68e77fe640 1 -- 172.21.15.67:0/2750391816 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec0cb2c0 con 0x55b83baa61c0 2024-07-31T14:58:04.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.790+0000 7f68e77fe640 1 -- 172.21.15.67:0/2750391816 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b83be001a0 con 0x55b83baa61c0 2024-07-31T14:58:04.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.791+0000 7f690a6d4740 1 -- 172.21.15.67:0/2750391816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83baa61c0 msgr2=0x55b83be8af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:04.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.791+0000 7f690a6d4740 1 --2- 172.21.15.67:0/2750391816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83baa61c0 0x55b83be8af20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f68ec102350 tx=0x7f68ec00f360 comp rx=0 tx=0).stop 2024-07-31T14:58:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.795+0000 7f690a6d4740 1 -- 172.21.15.67:0/2750391816 shutdown_connections 2024-07-31T14:58:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.795+0000 7f690a6d4740 1 --2- 172.21.15.67:0/2750391816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83bac6b60 0x55b83be95e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.795+0000 7f690a6d4740 1 --2- 172.21.15.67:0/2750391816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83babe980 0x55b83be8b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.795+0000 7f690a6d4740 1 --2- 172.21.15.67:0/2750391816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83baa61c0 0x55b83be8af20 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.795+0000 7f690a6d4740 1 -- 172.21.15.67:0/2750391816 >> 172.21.15.67:0/2750391816 conn(0x55b83badb990 msgr2=0x55b83be9a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:04.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.801+0000 7f690a6d4740 1 -- 172.21.15.67:0/2750391816 shutdown_connections 2024-07-31T14:58:04.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.802+0000 7f690a6d4740 1 -- 172.21.15.67:0/2750391816 wait complete. 2024-07-31T14:58:04.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.802+0000 7f690a6d4740 1 -- start start 2024-07-31T14:58:04.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.802+0000 7f690a6d4740 1 Processor -- start 2024-07-31T14:58:04.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.803+0000 7f690a6d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83baa61c0 0x55b83beac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.803+0000 7f690a6d4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83babe980 0x55b83beace80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.803+0000 7f690a6d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83bac6b60 0x55b83bf0b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.803+0000 7f690a6d4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f68f4164780 con 0x55b83babe980 2024-07-31T14:58:04.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.803+0000 7f690a6d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f68f4163ff0 con 0x55b83bac6b60 2024-07-31T14:58:04.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.803+0000 7f690a6d4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68f4145140 con 0x55b83baa61c0 2024-07-31T14:58:04.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.803+0000 7f690a6d4740 10 client.0 _ll_register_callbacks cb 0x7f68f80054e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:04.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.806+0000 7f6901f1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83baa61c0 0x55b83beac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.806+0000 7f6902720640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83bac6b60 0x55b83bf0b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.806+0000 7f6902720640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83bac6b60 0x55b83bf0b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46136/0 (socket says 172.21.15.67:46136) 2024-07-31T14:58:04.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.806+0000 7f6902720640 1 -- 172.21.15.67:0/52498627 learned_addr learned my addr 172.21.15.67:0/52498627 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:04.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.810+0000 7f6902720640 1 -- 172.21.15.67:0/52498627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83baa61c0 msgr2=0x55b83beac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:04.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.810+0000 7f6902720640 1 --2- 172.21.15.67:0/52498627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83baa61c0 0x55b83beac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.810+0000 7f6902720640 1 -- 172.21.15.67:0/52498627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83babe980 msgr2=0x55b83beace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:04.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.810+0000 7f6902720640 1 --2- 172.21.15.67:0/52498627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83babe980 0x55b83beace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.810+0000 7f6902720640 1 -- 172.21.15.67:0/52498627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b83bf0ba70 con 0x55b83bac6b60 2024-07-31T14:58:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.810+0000 7f6901f1f640 1 --2- 172.21.15.67:0/52498627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83baa61c0 0x55b83beac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:58:04.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.816+0000 7f6902720640 1 --2- 172.21.15.67:0/52498627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83bac6b60 0x55b83bf0b400 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009eb0 tx=0x7f68f802bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19482e9dbeb8af42 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.817+0000 7f68e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:58:04.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.820+0000 7f68e57fa640 1 -- 172.21.15.67:0/52498627 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8003e40 con 0x55b83bac6b60 2024-07-31T14:58:04.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.820+0000 7f690a6d4740 1 -- 172.21.15.67:0/52498627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b83be001a0 con 0x55b83bac6b60 2024-07-31T14:58:04.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.820+0000 7f68e57fa640 1 -- 172.21.15.67:0/52498627 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f68f8003c40 con 0x55b83bac6b60 2024-07-31T14:58:04.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.823+0000 7f68e57fa640 1 -- 172.21.15.67:0/52498627 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f68f804b050 con 0x55b83bac6b60 2024-07-31T14:58:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.827+0000 7f690a6d4740 10 client.26912 fetch_fsmap learned FSMap version 50 2024-07-31T14:58:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.827+0000 7f690a6d4740 1 -- 172.21.15.67:0/52498627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55b83bf13f10 con 0x55b83bac6b60 2024-07-31T14:58:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.828+0000 7f68e57fa640 1 -- 172.21.15.67:0/52498627 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b83be001a0 con 0x55b83bac6b60 2024-07-31T14:58:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.829+0000 7f68e57fa640 1 -- 172.21.15.67:0/52498627 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f68f803d8f0 con 0x55b83bac6b60 2024-07-31T14:58:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.829+0000 7f690a6d4740 10 client.26912 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:58:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.829+0000 7f690a6d4740 10 client.26912 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:58:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.829+0000 7f690a6d4740 20 client.26912 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.829+0000 7f690a6d4740 1 --2- 172.21.15.67:0/52498627 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b83bf146a0 0x55b83bf16ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.830+0000 7f690a6d4740 4 client.26912 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-07-31T14:58:04.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.830+0000 7f690a6d4740 1 -- 172.21.15.67:0/52498627 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55b83bdfd8e0 con 0x55b83bf146a0 2024-07-31T14:58:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.833+0000 7f6901f1f640 1 --2- 172.21.15.67:0/52498627 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b83bf146a0 0x55b83bf16ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:04.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.841+0000 7f68e57fa640 10 client.26912 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:04.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:04.841+0000 7f6901f1f640 1 --2- 172.21.15.67:0/52498627 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b83bf146a0 0x55b83bf16ac0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c60a68a392914564 server_cookie=79b73894f0ec5f2a in_seq=0 out_seq=0 2024-07-31T14:58:05.010 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:04 smithi067 ceph-mon[32750]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.8 MiB/s wr, 700 op/s 2024-07-31T14:58:05.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:04 smithi077 ceph-mon[37327]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.8 MiB/s wr, 700 op/s 2024-07-31T14:58:05.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:04 smithi073 ceph-mon[36714]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.8 MiB/s wr, 700 op/s 2024-07-31T14:58:06.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.450+0000 7f958c31e640 1 -- 172.21.15.67:0/2167678918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9584111970 msgr2=0x7f9584109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:06.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.450+0000 7f958c31e640 1 --2- 172.21.15.67:0/2167678918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9584111970 0x7f9584109df0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9580009be0 tx=0x7f958002a710 comp rx=0 tx=0).stop 2024-07-31T14:58:06.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.451+0000 7f958c31e640 1 -- 172.21.15.67:0/2167678918 shutdown_connections 2024-07-31T14:58:06.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.451+0000 7f958c31e640 1 --2- 172.21.15.67:0/2167678918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f958411c660 0x7f958411eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:06.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.451+0000 7f958c31e640 1 --2- 172.21.15.67:0/2167678918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f958410a410 0x7f958410a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:06.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.451+0000 7f958c31e640 1 --2- 172.21.15.67:0/2167678918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9584111970 0x7f9584109df0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:06.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.451+0000 7f958c31e640 1 -- 172.21.15.67:0/2167678918 >> 172.21.15.67:0/2167678918 conn(0x7f95840708f0 msgr2=0x7f9584070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:06.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.458+0000 7f958c31e640 1 -- 172.21.15.67:0/2167678918 shutdown_connections 2024-07-31T14:58:06.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.459+0000 7f958c31e640 1 -- 172.21.15.67:0/2167678918 wait complete. 2024-07-31T14:58:06.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.460+0000 7f958c31e640 1 Processor -- start 2024-07-31T14:58:06.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.462+0000 7f958c31e640 1 -- start start 2024-07-31T14:58:06.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.462+0000 7f958c31e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f958410a410 0x7f95841a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:06.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.462+0000 7f958c31e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9584111970 0x7f95841a9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:06.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.462+0000 7f958c31e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f958411c660 0x7f95841ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:06.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.462+0000 7f958c31e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9584122390 con 0x7f9584111970 2024-07-31T14:58:06.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.462+0000 7f958c31e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9584122210 con 0x7f958410a410 2024-07-31T14:58:06.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.462+0000 7f958c31e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9584122510 con 0x7f958411c660 2024-07-31T14:58:06.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.463+0000 7f958a0ba640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f958410a410 0x7f95841a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:06.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.463+0000 7f958a0ba640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f958410a410 0x7f95841a8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46166/0 (socket says 172.21.15.67:46166) 2024-07-31T14:58:06.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.463+0000 7f958a0ba640 1 -- 172.21.15.67:0/3591229398 learned_addr learned my addr 172.21.15.67:0/3591229398 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:06.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.463+0000 7f95898b9640 1 --2- 172.21.15.67:0/3591229398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9584111970 0x7f95841a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:06.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.464+0000 7f958a8bb640 1 --2- 172.21.15.67:0/3591229398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f958411c660 0x7f95841ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:06.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.464+0000 7f958a0ba640 1 -- 172.21.15.67:0/3591229398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f958411c660 msgr2=0x7f95841ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:06.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.464+0000 7f958a0ba640 1 --2- 172.21.15.67:0/3591229398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f958411c660 0x7f95841ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:06.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.464+0000 7f958a0ba640 1 -- 172.21.15.67:0/3591229398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9584111970 msgr2=0x7f95841a9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:06.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.464+0000 7f958a0ba640 1 --2- 172.21.15.67:0/3591229398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9584111970 0x7f95841a9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:06.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.464+0000 7f958a0ba640 1 -- 172.21.15.67:0/3591229398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95841aed20 con 0x7f958410a410 2024-07-31T14:58:06.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.469+0000 7f958a0ba640 1 --2- 172.21.15.67:0/3591229398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f958410a410 0x7f95841a8de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9580034040 tx=0x7f9580003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:06.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.471+0000 7f957b7fe640 1 -- 172.21.15.67:0/3591229398 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580041020 con 0x7f958410a410 2024-07-31T14:58:06.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.471+0000 7f957b7fe640 1 -- 172.21.15.67:0/3591229398 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95800318f0 con 0x7f958410a410 2024-07-31T14:58:06.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.471+0000 7f958c31e640 1 -- 172.21.15.67:0/3591229398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95841aefb0 con 0x7f958410a410 2024-07-31T14:58:06.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.471+0000 7f958c31e640 1 -- 172.21.15.67:0/3591229398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9584111db0 con 0x7f958410a410 2024-07-31T14:58:06.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.471+0000 7f957b7fe640 1 -- 172.21.15.67:0/3591229398 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580036040 con 0x7f958410a410 2024-07-31T14:58:06.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.475+0000 7f959286f740 1 Processor -- start 2024-07-31T14:58:06.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.478+0000 7f959286f740 1 -- start start 2024-07-31T14:58:06.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.479+0000 7f959286f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cfe070 0x5639980a3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:06.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.479+0000 7f959286f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cd51b0 0x5639980a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:06.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.479+0000 7f959286f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cdd390 0x5639980a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:06.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.479+0000 7f959286f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563998019bb0 con 0x563997cd51b0 2024-07-31T14:58:06.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.479+0000 7f959286f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5639980090b0 con 0x563997cfe070 2024-07-31T14:58:06.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.479+0000 7f959286f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56399800d0f0 con 0x563997cdd390 2024-07-31T14:58:06.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.480+0000 7f957b7fe640 1 -- 172.21.15.67:0/3591229398 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f9580044020 con 0x7f958410a410 2024-07-31T14:58:06.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.480+0000 7f95898b9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cdd390 0x5639980a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:06.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.480+0000 7f95898b9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cdd390 0x5639980a3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:56988/0 (socket says 172.21.15.67:56988) 2024-07-31T14:58:06.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.480+0000 7f95898b9640 1 -- 172.21.15.67:0/1077763894 learned_addr learned my addr 172.21.15.67:0/1077763894 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:06.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.481+0000 7f957b7fe640 1 --2- 172.21.15.67:0/3591229398 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f95700b3f50 0x7f95700b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:06.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.482+0000 7f95898b9640 1 -- 172.21.15.67:0/1077763894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cfe070 msgr2=0x5639980a3570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:58:06.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.482+0000 7f95898b9640 1 --2- 172.21.15.67:0/1077763894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cfe070 0x5639980a3570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:06.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.482+0000 7f95898b9640 1 -- 172.21.15.67:0/1077763894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cd51b0 msgr2=0x5639980a3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:06.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.482+0000 7f95898b9640 1 --2- 172.21.15.67:0/1077763894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cd51b0 0x5639980a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:06.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.482+0000 7f95898b9640 1 -- 172.21.15.67:0/1077763894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563997fafb90 con 0x563997cdd390 2024-07-31T14:58:06.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.482+0000 7f957b7fe640 1 -- 172.21.15.67:0/3591229398 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f9580104b10 con 0x7f958410a410 2024-07-31T14:58:06.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.484+0000 7f95898b9640 1 --2- 172.21.15.67:0/1077763894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cdd390 0x5639980a3ff0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f957400b790 tx=0x7f957400bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f0559c6b488b43 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:06.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.484+0000 7f958a8bb640 1 --2- 172.21.15.67:0/3591229398 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f95700b3f50 0x7f95700b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:06.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.486+0000 7f9578d99640 1 -- 172.21.15.67:0/1077763894 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957400d950 con 0x563997cdd390 2024-07-31T14:58:06.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.486+0000 7f9578d99640 1 -- 172.21.15.67:0/1077763894 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9574018070 con 0x563997cdd390 2024-07-31T14:58:06.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.486+0000 7f9578d99640 1 -- 172.21.15.67:0/1077763894 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95740123e0 con 0x563997cdd390 2024-07-31T14:58:06.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.486+0000 7f958a8bb640 1 --2- 172.21.15.67:0/3591229398 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f95700b3f50 0x7f95700b6460 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f957c009530 tx=0x7f957c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:06.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.488+0000 7f959286f740 1 -- 172.21.15.67:0/1077763894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cdd390 msgr2=0x5639980a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:06.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.488+0000 7f959286f740 1 --2- 172.21.15.67:0/1077763894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cdd390 0x5639980a3ff0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f957400b790 tx=0x7f957400bc50 comp rx=0 tx=0).stop 2024-07-31T14:58:06.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.490+0000 7f959286f740 1 -- 172.21.15.67:0/1077763894 shutdown_connections 2024-07-31T14:58:06.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.490+0000 7f959286f740 1 --2- 172.21.15.67:0/1077763894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cdd390 0x5639980a3ff0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:06.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.490+0000 7f959286f740 1 --2- 172.21.15.67:0/1077763894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cd51b0 0x5639980a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:06.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.490+0000 7f959286f740 1 --2- 172.21.15.67:0/1077763894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cfe070 0x5639980a3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:06.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.490+0000 7f959286f740 1 -- 172.21.15.67:0/1077763894 >> 172.21.15.67:0/1077763894 conn(0x563997cf2460 msgr2=0x5639980a1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.494+0000 7f959286f740 1 -- 172.21.15.67:0/1077763894 shutdown_connections 2024-07-31T14:58:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.496+0000 7f959286f740 1 -- 172.21.15.67:0/1077763894 wait complete. 2024-07-31T14:58:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.497+0000 7f959286f740 1 -- start start 2024-07-31T14:58:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.497+0000 7f959286f740 1 Processor -- start 2024-07-31T14:58:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.497+0000 7f959286f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cd51b0 0x5639980af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.498+0000 7f959286f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cdd390 0x5639980af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.498+0000 7f958a0ba640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cd51b0 0x5639980af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.498+0000 7f958a0ba640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cd51b0 0x5639980af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34752/0 (socket says 172.21.15.67:34752) 2024-07-31T14:58:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.498+0000 7f958a0ba640 1 -- 172.21.15.67:0/186415293 learned_addr learned my addr 172.21.15.67:0/186415293 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.498+0000 7f959286f740 1 --2- 172.21.15.67:0/186415293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cfe070 0x5639980bb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.498+0000 7f95898b9640 1 --2- 172.21.15.67:0/186415293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cdd390 0x5639980af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.498+0000 7f959286f740 1 -- 172.21.15.67:0/186415293 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563997e6c700 con 0x563997cd51b0 2024-07-31T14:58:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.498+0000 7f959286f740 1 -- 172.21.15.67:0/186415293 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563997fab9e0 con 0x563997cdd390 2024-07-31T14:58:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.498+0000 7f959286f740 1 -- 172.21.15.67:0/186415293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5639980b1d50 con 0x563997cfe070 2024-07-31T14:58:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.498+0000 7f958a8bb640 1 --2- 172.21.15.67:0/186415293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cfe070 0x5639980bb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.498+0000 7f959286f740 10 client.0 _ll_register_callbacks cb 0x563997fc64f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.499+0000 7f958a8bb640 1 -- 172.21.15.67:0/186415293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cdd390 msgr2=0x5639980af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.499+0000 7f958a8bb640 1 --2- 172.21.15.67:0/186415293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cdd390 0x5639980af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.499+0000 7f958a8bb640 1 -- 172.21.15.67:0/186415293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cd51b0 msgr2=0x5639980af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.499+0000 7f958a8bb640 1 --2- 172.21.15.67:0/186415293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cd51b0 0x5639980af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.499+0000 7f958a8bb640 1 -- 172.21.15.67:0/186415293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563997d69650 con 0x563997cfe070 2024-07-31T14:58:06.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.499+0000 7f958a8bb640 1 --2- 172.21.15.67:0/186415293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cfe070 0x5639980bb0d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f957c012680 tx=0x7f957c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55a8874cfff4e1b8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:06.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.499+0000 7f956dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:58:06.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.500+0000 7f956dffb640 1 -- 172.21.15.67:0/186415293 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c005220 con 0x563997cfe070 2024-07-31T14:58:06.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.500+0000 7f959286f740 1 -- 172.21.15.67:0/186415293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9574018070 con 0x563997cfe070 2024-07-31T14:58:06.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.500+0000 7f956dffb640 1 -- 172.21.15.67:0/186415293 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f957c0053c0 con 0x563997cfe070 2024-07-31T14:58:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.500+0000 7f956dffb640 1 -- 172.21.15.67:0/186415293 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f957c02f050 con 0x563997cfe070 2024-07-31T14:58:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.500+0000 7f959286f740 10 client.26932 fetch_fsmap learned FSMap version 50 2024-07-31T14:58:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.500+0000 7f959286f740 1 -- 172.21.15.67:0/186415293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5639980bdf10 con 0x563997cfe070 2024-07-31T14:58:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.502+0000 7f956dffb640 1 -- 172.21.15.67:0/186415293 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9574018070 con 0x563997cfe070 2024-07-31T14:58:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.502+0000 7f956dffb640 1 -- 172.21.15.67:0/186415293 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f957c004b80 con 0x563997cfe070 2024-07-31T14:58:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.502+0000 7f959286f740 10 client.26932 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:58:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.502+0000 7f959286f740 10 client.26932 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:58:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.502+0000 7f959286f740 20 client.26932 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.502+0000 7f959286f740 1 --2- 172.21.15.67:0/186415293 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x563997cbc9f0 0x5639980c18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.502+0000 7f959286f740 4 client.26932 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:58:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.502+0000 7f959286f740 1 -- 172.21.15.67:0/186415293 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56399801bb00 con 0x563997cbc9f0 2024-07-31T14:58:06.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.505+0000 7f958a0ba640 1 --2- 172.21.15.67:0/186415293 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x563997cbc9f0 0x5639980c18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:06.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.508+0000 7f958a0ba640 1 --2- 172.21.15.67:0/186415293 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x563997cbc9f0 0x5639980c18d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce9590eac06747ad server_cookie=946532f8ae584917 in_seq=0 out_seq=0 2024-07-31T14:58:06.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:06.508+0000 7f956dffb640 10 client.26932 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:58:07.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:06 smithi077 ceph-mon[37327]: pgmap v612: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 525 op/s 2024-07-31T14:58:07.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:06 smithi067 ceph-mon[32750]: pgmap v612: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 525 op/s 2024-07-31T14:58:07.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:06 smithi073 ceph-mon[36714]: pgmap v612: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 525 op/s 2024-07-31T14:58:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.221+0000 7f956dffb640 1 -- 172.21.15.67:0/186415293 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56399801bb00 con 0x563997cbc9f0 2024-07-31T14:58:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.221+0000 7f956dffb640 10 client.26932 handle_command_reply: tid=0 2024-07-31T14:58:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.221+0000 7f959286f740 1 client.26932 shutdown 2024-07-31T14:58:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.222+0000 7f959286f740 1 -- 172.21.15.67:0/186415293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cfe070 msgr2=0x5639980bb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.222+0000 7f959286f740 1 --2- 172.21.15.67:0/186415293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cfe070 0x5639980bb0d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f957c012680 tx=0x7f957c004730 comp rx=0 tx=0).stop 2024-07-31T14:58:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.222+0000 7f959286f740 1 -- 172.21.15.67:0/186415293 shutdown_connections 2024-07-31T14:58:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.222+0000 7f959286f740 1 --2- 172.21.15.67:0/186415293 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x563997cbc9f0 0x5639980c18d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.222+0000 7f959286f740 1 --2- 172.21.15.67:0/186415293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cfe070 0x5639980bb0d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.222+0000 7f959286f740 1 --2- 172.21.15.67:0/186415293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cdd390 0x5639980af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.222+0000 7f959286f740 1 --2- 172.21.15.67:0/186415293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cd51b0 0x5639980af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.222+0000 7f959286f740 1 -- 172.21.15.67:0/186415293 >> 172.21.15.67:0/186415293 conn(0x563997cf2460 msgr2=0x56399809fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:08.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.223+0000 7f956dffb640 0 client.26932 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:58:08.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.223+0000 7f959286f740 1 -- 172.21.15.67:0/186415293 shutdown_connections 2024-07-31T14:58:08.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.223+0000 7f959286f740 1 -- 172.21.15.67:0/186415293 wait complete. 2024-07-31T14:58:08.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.223+0000 7f959286f740 20 client.26932 trim_cache size 0 max 16384 2024-07-31T14:58:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.250+0000 7f959286f740 1 Processor -- start 2024-07-31T14:58:08.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.251+0000 7f959286f740 1 -- start start 2024-07-31T14:58:08.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.251+0000 7f959286f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cd51b0 0x5639980a0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:08.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.251+0000 7f959286f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cdd390 0x5639980a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:08.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.252+0000 7f959286f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cfe070 0x5639980a8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:08.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.252+0000 7f958a0ba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cd51b0 0x5639980a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:08.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.252+0000 7f959286f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f95680100e0 con 0x563997cfe070 2024-07-31T14:58:08.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.252+0000 7f959286f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9568011d70 con 0x563997cdd390 2024-07-31T14:58:08.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.252+0000 7f95898b9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cdd390 0x5639980a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:08.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.252+0000 7f95898b9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cdd390 0x5639980a87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46202/0 (socket says 172.21.15.67:46202) 2024-07-31T14:58:08.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.252+0000 7f95898b9640 1 -- 172.21.15.67:0/3760930884 learned_addr learned my addr 172.21.15.67:0/3760930884 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:08.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.252+0000 7f959286f740 1 -- 172.21.15.67:0/3760930884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9568010840 con 0x563997cd51b0 2024-07-31T14:58:08.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.254+0000 7f958a8bb640 1 --2- 172.21.15.67:0/3760930884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cfe070 0x5639980a8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:08.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.255+0000 7f958a0ba640 1 -- 172.21.15.67:0/3760930884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cdd390 msgr2=0x5639980a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:08.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.255+0000 7f958a0ba640 1 --2- 172.21.15.67:0/3760930884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cdd390 0x5639980a87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.255+0000 7f958a0ba640 1 -- 172.21.15.67:0/3760930884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cfe070 msgr2=0x5639980a8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:08.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.255+0000 7f958a0ba640 1 --2- 172.21.15.67:0/3760930884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cfe070 0x5639980a8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.255+0000 7f958a0ba640 1 -- 172.21.15.67:0/3760930884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639980a5a90 con 0x563997cd51b0 2024-07-31T14:58:08.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.258+0000 7f958a0ba640 1 --2- 172.21.15.67:0/3760930884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cd51b0 0x5639980a0350 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9580000f80 tx=0x7f9580003f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=871df5e8e38164d8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:08.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.258+0000 7f956ffff640 1 -- 172.21.15.67:0/3760930884 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800e5070 con 0x563997cd51b0 2024-07-31T14:58:08.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.259+0000 7f956ffff640 1 -- 172.21.15.67:0/3760930884 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95800cba00 con 0x563997cd51b0 2024-07-31T14:58:08.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.259+0000 7f956ffff640 1 -- 172.21.15.67:0/3760930884 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800c7590 con 0x563997cd51b0 2024-07-31T14:58:08.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.260+0000 7f959286f740 1 -- 172.21.15.67:0/3760930884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cd51b0 msgr2=0x5639980a0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:08.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.260+0000 7f959286f740 1 --2- 172.21.15.67:0/3760930884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cd51b0 0x5639980a0350 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9580000f80 tx=0x7f9580003f40 comp rx=0 tx=0).stop 2024-07-31T14:58:08.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.260+0000 7f959286f740 1 -- 172.21.15.67:0/3760930884 shutdown_connections 2024-07-31T14:58:08.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.261+0000 7f959286f740 1 --2- 172.21.15.67:0/3760930884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cfe070 0x5639980a8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.261+0000 7f959286f740 1 --2- 172.21.15.67:0/3760930884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cdd390 0x5639980a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.261+0000 7f959286f740 1 --2- 172.21.15.67:0/3760930884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cd51b0 0x5639980a0350 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.261+0000 7f959286f740 1 -- 172.21.15.67:0/3760930884 >> 172.21.15.67:0/3760930884 conn(0x563997cf2460 msgr2=0x56399809fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:08.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.261+0000 7f959286f740 1 -- 172.21.15.67:0/3760930884 shutdown_connections 2024-07-31T14:58:08.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.261+0000 7f959286f740 1 -- 172.21.15.67:0/3760930884 wait complete. 2024-07-31T14:58:08.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.261+0000 7f959286f740 1 -- start start 2024-07-31T14:58:08.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.262+0000 7f959286f740 1 Processor -- start 2024-07-31T14:58:08.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.262+0000 7f959286f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cd51b0 0x5639980b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:08.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.262+0000 7f959286f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cdd390 0x5639980b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:08.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.262+0000 7f958a0ba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cd51b0 0x5639980b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:08.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.263+0000 7f958a0ba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cd51b0 0x5639980b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:57022/0 (socket says 172.21.15.67:57022) 2024-07-31T14:58:08.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.263+0000 7f958a0ba640 1 -- 172.21.15.67:0/1674171009 learned_addr learned my addr 172.21.15.67:0/1674171009 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:08.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.263+0000 7f959286f740 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cfe070 0x5639980ba750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:08.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.263+0000 7f959286f740 1 -- 172.21.15.67:0/1674171009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f95680116a0 con 0x563997cdd390 2024-07-31T14:58:08.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.263+0000 7f959286f740 1 -- 172.21.15.67:0/1674171009 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9568010f10 con 0x563997cfe070 2024-07-31T14:58:08.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.263+0000 7f958a8bb640 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cfe070 0x5639980ba750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:08.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.263+0000 7f95898b9640 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cdd390 0x5639980b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:08.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.263+0000 7f959286f740 1 -- 172.21.15.67:0/1674171009 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f956800e0e0 con 0x563997cd51b0 2024-07-31T14:58:08.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.265+0000 7f95898b9640 1 -- 172.21.15.67:0/1674171009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cd51b0 msgr2=0x5639980b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:08.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.265+0000 7f95898b9640 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cd51b0 0x5639980b72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.265+0000 7f95898b9640 1 -- 172.21.15.67:0/1674171009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cfe070 msgr2=0x5639980ba750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:08.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.265+0000 7f95898b9640 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cfe070 0x5639980ba750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.265+0000 7f95898b9640 1 -- 172.21.15.67:0/1674171009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639980baee0 con 0x563997cdd390 2024-07-31T14:58:08.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.265+0000 7f959286f740 10 client.0 _ll_register_callbacks cb 0x7f957c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:08.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.265+0000 7f959286f740 1 -- 172.21.15.67:0/1674171009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95800cba00 con 0x563997cdd390 2024-07-31T14:58:08.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.265+0000 7f95898b9640 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cdd390 0x5639980b7890 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x563997f91990 tx=0x7f957400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db82b20754ecb17 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:08.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.265+0000 7f956dffb640 10 client.17364 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:58:08.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.265+0000 7f956dffb640 1 -- 172.21.15.67:0/1674171009 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574013480 con 0x563997cdd390 2024-07-31T14:58:08.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.266+0000 7f956dffb640 1 -- 172.21.15.67:0/1674171009 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f9574013620 con 0x563997cdd390 2024-07-31T14:58:08.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.266+0000 7f958a0ba640 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cd51b0 0x5639980b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:58:08.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.266+0000 7f956dffb640 1 -- 172.21.15.67:0/1674171009 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f95740138a0 con 0x563997cdd390 2024-07-31T14:58:08.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.266+0000 7f959286f740 10 client.17364 fetch_fsmap learned FSMap version 50 2024-07-31T14:58:08.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.266+0000 7f959286f740 1 -- 172.21.15.67:0/1674171009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({fsmap=50}) -- 0x5639980b6a30 con 0x563997cdd390 2024-07-31T14:58:08.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.266+0000 7f956dffb640 1 -- 172.21.15.67:0/1674171009 <== mon.0 v2:172.21.15.67:3300/0 4 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f957402e420 con 0x563997cdd390 2024-07-31T14:58:08.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.266+0000 7f959286f740 10 client.17364 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:58:08.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.266+0000 7f959286f740 10 client.17364 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:58:08.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.266+0000 7f959286f740 20 client.17364 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:08.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.266+0000 7f959286f740 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x563997c91da0 0x56399812c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:08.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.266+0000 7f959286f740 4 client.17364 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:58:08.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.266+0000 7f959286f740 1 -- 172.21.15.67:0/1674171009 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5639980175a0 con 0x563997c91da0 2024-07-31T14:58:08.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.269+0000 7f958a0ba640 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x563997c91da0 0x56399812c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:08.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.280+0000 7f958a0ba640 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x563997c91da0 0x56399812c0b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b6c60c3e2daafa0 server_cookie=6a4e5ee9553a5fa3 in_seq=0 out_seq=0 2024-07-31T14:58:08.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.281+0000 7f956dffb640 10 client.17364 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:58:08.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.483+0000 7f956dffb640 1 -- 172.21.15.67:0/1674171009 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5639980175a0 con 0x563997c91da0 2024-07-31T14:58:08.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.483+0000 7f956dffb640 10 client.17364 handle_command_reply: tid=0 2024-07-31T14:58:08.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.483+0000 7f959286f740 1 client.17364 shutdown 2024-07-31T14:58:08.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.483+0000 7f959286f740 1 -- 172.21.15.67:0/1674171009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cdd390 msgr2=0x5639980b7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:08.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.483+0000 7f959286f740 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cdd390 0x5639980b7890 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x563997f91990 tx=0x7f957400dbf0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.484+0000 7f959286f740 1 -- 172.21.15.67:0/1674171009 shutdown_connections 2024-07-31T14:58:08.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.484+0000 7f959286f740 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x563997c91da0 0x56399812c0b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.484+0000 7f959286f740 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563997cfe070 0x5639980ba750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.484+0000 7f956dffb640 0 client.17364 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:58:08.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.484+0000 7f959286f740 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563997cdd390 0x5639980b7890 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.484+0000 7f959286f740 1 --2- 172.21.15.67:0/1674171009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563997cd51b0 0x5639980b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.484+0000 7f959286f740 1 -- 172.21.15.67:0/1674171009 >> 172.21.15.67:0/1674171009 conn(0x563997cf2460 msgr2=0x5639980a0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:08.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.484+0000 7f959286f740 1 -- 172.21.15.67:0/1674171009 shutdown_connections 2024-07-31T14:58:08.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.484+0000 7f959286f740 1 -- 172.21.15.67:0/1674171009 wait complete. 2024-07-31T14:58:08.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.484+0000 7f959286f740 20 client.17364 trim_cache size 0 max 16384 2024-07-31T14:58:08.487 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:58:08.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.485+0000 7f958c31e640 1 -- 172.21.15.67:0/3591229398 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f95700b3f50 msgr2=0x7f95700b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:08.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.485+0000 7f958c31e640 1 --2- 172.21.15.67:0/3591229398 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f95700b3f50 0x7f95700b6460 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f957c009530 tx=0x7f957c009260 comp rx=0 tx=0).stop 2024-07-31T14:58:08.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.485+0000 7f958c31e640 1 -- 172.21.15.67:0/3591229398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f958410a410 msgr2=0x7f95841a8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:08.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.485+0000 7f958c31e640 1 --2- 172.21.15.67:0/3591229398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f958410a410 0x7f95841a8de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9580034040 tx=0x7f9580003ba0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.487+0000 7f958c31e640 1 -- 172.21.15.67:0/3591229398 shutdown_connections 2024-07-31T14:58:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.487+0000 7f958c31e640 1 --2- 172.21.15.67:0/3591229398 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f95700b3f50 0x7f95700b6460 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.487+0000 7f958c31e640 1 --2- 172.21.15.67:0/3591229398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f958411c660 0x7f95841ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.487+0000 7f958c31e640 1 --2- 172.21.15.67:0/3591229398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9584111970 0x7f95841a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.487+0000 7f958c31e640 1 --2- 172.21.15.67:0/3591229398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f958410a410 0x7f95841a8de0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:08.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.487+0000 7f958c31e640 1 -- 172.21.15.67:0/3591229398 >> 172.21.15.67:0/3591229398 conn(0x7f95840708f0 msgr2=0x7f9584071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:08.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.487+0000 7f958c31e640 1 -- 172.21.15.67:0/3591229398 shutdown_connections 2024-07-31T14:58:08.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:08.487+0000 7f958c31e640 1 -- 172.21.15.67:0/3591229398 wait complete. 2024-07-31T14:58:09.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:08 smithi077 ceph-mon[37327]: pgmap v613: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 788 op/s 2024-07-31T14:58:09.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:08 smithi067 ceph-mon[32750]: pgmap v613: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 788 op/s 2024-07-31T14:58:09.355 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T14:58:09.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:08 smithi073 ceph-mon[36714]: pgmap v613: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 788 op/s 2024-07-31T14:58:09.899 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:58:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.308+0000 7f98477fe640 1 -- 172.21.15.67:0/2135886646 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564176a6bda0 con 0x5641766ea1c0 2024-07-31T14:58:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.308+0000 7f98477fe640 10 client.26908 handle_command_reply: tid=0 2024-07-31T14:58:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.308+0000 7f98757c0740 1 client.26908 shutdown 2024-07-31T14:58:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.309+0000 7f98757c0740 1 -- 172.21.15.67:0/2135886646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767354d0 msgr2=0x564176b0c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.309+0000 7f98757c0740 1 --2- 172.21.15.67:0/2135886646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767354d0 0x564176b0c930 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f985c0058c0 tx=0x7f985c0e16e0 comp rx=0 tx=0).stop 2024-07-31T14:58:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.309+0000 7f98757c0740 1 -- 172.21.15.67:0/2135886646 shutdown_connections 2024-07-31T14:58:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.309+0000 7f98757c0740 1 --2- 172.21.15.67:0/2135886646 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5641766ea1c0 0x564176b7fbd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.309+0000 7f98757c0740 1 --2- 172.21.15.67:0/2135886646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641767561b0 0x564176b091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.309+0000 7f98757c0740 1 --2- 172.21.15.67:0/2135886646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5641767354d0 0x564176b0c930 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.309+0000 7f98757c0740 1 --2- 172.21.15.67:0/2135886646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56417672d2f0 0x564176b0c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.309+0000 7f98757c0740 1 -- 172.21.15.67:0/2135886646 >> 172.21.15.67:0/2135886646 conn(0x56417674a300 msgr2=0x564176afd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.309+0000 7f98477fe640 0 client.26908 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.309+0000 7f98757c0740 1 -- 172.21.15.67:0/2135886646 shutdown_connections 2024-07-31T14:58:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.310+0000 7f98757c0740 1 -- 172.21.15.67:0/2135886646 wait complete. 2024-07-31T14:58:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.310+0000 7f98757c0740 20 client.26908 trim_cache size 0 max 16384 2024-07-31T14:58:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:58:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:58:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:58:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:58:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.311+0000 7f986e7eb640 1 -- 172.21.15.67:0/4252163761 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98500b3ed0 msgr2=0x7f98500b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.311+0000 7f986e7eb640 1 --2- 172.21.15.67:0/4252163761 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98500b3ed0 0x7f98500b63e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f98580058c0 tx=0x7f9858005850 comp rx=0 tx=0).stop 2024-07-31T14:58:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.311+0000 7f986e7eb640 1 -- 172.21.15.67:0/4252163761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98681118d0 msgr2=0x7f9868116240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.311+0000 7f986e7eb640 1 --2- 172.21.15.67:0/4252163761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98681118d0 0x7f9868116240 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f985c00c970 tx=0x7f985c00ce30 comp rx=0 tx=0).stop 2024-07-31T14:58:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.311+0000 7f986e7eb640 1 -- 172.21.15.67:0/4252163761 shutdown_connections 2024-07-31T14:58:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.311+0000 7f986e7eb640 1 --2- 172.21.15.67:0/4252163761 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98500b3ed0 0x7f98500b63e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.312+0000 7f986e7eb640 1 --2- 172.21.15.67:0/4252163761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f986811c5d0 0x7f98681168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.312+0000 7f986e7eb640 1 --2- 172.21.15.67:0/4252163761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98681118d0 0x7f9868116240 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.312+0000 7f986e7eb640 1 --2- 172.21.15.67:0/4252163761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f986810a330 0x7f986811bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.312+0000 7f986e7eb640 1 -- 172.21.15.67:0/4252163761 >> 172.21.15.67:0/4252163761 conn(0x7f98680707c0 msgr2=0x7f986811ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.313+0000 7f986e7eb640 1 -- 172.21.15.67:0/4252163761 shutdown_connections 2024-07-31T14:58:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:10.313+0000 7f986e7eb640 1 -- 172.21.15.67:0/4252163761 wait complete. 2024-07-31T14:58:11.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:10 smithi077 ceph-mon[37327]: pgmap v614: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 1.18k op/s 2024-07-31T14:58:11.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:10 smithi067 ceph-mon[32750]: pgmap v614: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 1.18k op/s 2024-07-31T14:58:11.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:10 smithi073 ceph-mon[36714]: pgmap v614: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 1.18k op/s 2024-07-31T14:58:12.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.109+0000 7f8063fff640 1 -- 172.21.15.67:0/1623547993 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80680318a0 con 0x7f806c107630 2024-07-31T14:58:12.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.110+0000 7f8073477640 1 -- 172.21.15.67:0/1623547993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f806c107630 msgr2=0x7f806c107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:12.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.110+0000 7f8073477640 1 --2- 172.21.15.67:0/1623547993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f806c107630 0x7f806c107b10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8068009930 tx=0x7f806802a420 comp rx=0 tx=0).stop 2024-07-31T14:58:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.110+0000 7f8073477640 1 -- 172.21.15.67:0/1623547993 shutdown_connections 2024-07-31T14:58:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.110+0000 7f8073477640 1 --2- 172.21.15.67:0/1623547993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f806c107630 0x7f806c107b10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.110+0000 7f8073477640 1 --2- 172.21.15.67:0/1623547993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f806c102b40 0x7f806c107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.110+0000 7f8073477640 1 --2- 172.21.15.67:0/1623547993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f806c1021f0 0x7f806c102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.110+0000 7f8073477640 1 -- 172.21.15.67:0/1623547993 >> 172.21.15.67:0/1623547993 conn(0x7f806c079170 msgr2=0x7f806c079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.111+0000 7f8073477640 1 -- 172.21.15.67:0/1623547993 shutdown_connections 2024-07-31T14:58:12.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.111+0000 7f8073477640 1 -- 172.21.15.67:0/1623547993 wait complete. 2024-07-31T14:58:12.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.112+0000 7f8073477640 1 Processor -- start 2024-07-31T14:58:12.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.112+0000 7f8073477640 1 -- start start 2024-07-31T14:58:12.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.112+0000 7f8073477640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f806c1021f0 0x7f806c0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.112+0000 7f8073477640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f806c102b40 0x7f806c074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.112+0000 7f8073477640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f806c107630 0x7f806c06e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.112+0000 7f8073477640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f806c06cce0 con 0x7f806c107630 2024-07-31T14:58:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.112+0000 7f8073477640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f806c06cb60 con 0x7f806c1021f0 2024-07-31T14:58:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.112+0000 7f8073477640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f806c06ce60 con 0x7f806c102b40 2024-07-31T14:58:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.112+0000 7f8070a12640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f806c102b40 0x7f806c074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.112+0000 7f8070a12640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f806c102b40 0x7f806c074a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58864/0 (socket says 172.21.15.67:58864) 2024-07-31T14:58:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.112+0000 7f8070a12640 1 -- 172.21.15.67:0/1131713931 learned_addr learned my addr 172.21.15.67:0/1131713931 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.113+0000 7f8071a14640 1 --2- 172.21.15.67:0/1131713931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f806c107630 0x7f806c06e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.113+0000 7f8070a12640 1 -- 172.21.15.67:0/1131713931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f806c1021f0 msgr2=0x7f806c0744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.113+0000 7f8070a12640 1 --2- 172.21.15.67:0/1131713931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f806c1021f0 0x7f806c0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.113+0000 7f8070a12640 1 -- 172.21.15.67:0/1131713931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f806c107630 msgr2=0x7f806c06e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.113+0000 7f8070a12640 1 --2- 172.21.15.67:0/1131713931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f806c107630 0x7f806c06e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.113+0000 7f8070a12640 1 -- 172.21.15.67:0/1131713931 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c06f1a0 con 0x7f806c102b40 2024-07-31T14:58:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.113+0000 7f8071a14640 1 --2- 172.21.15.67:0/1131713931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f806c107630 0x7f806c06e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:58:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.114+0000 7f8070a12640 1 --2- 172.21.15.67:0/1131713931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f806c102b40 0x7f806c074a50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f805c00d740 tx=0x7f805c00dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.114+0000 7f80627fc640 1 -- 172.21.15.67:0/1131713931 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c017070 con 0x7f806c102b40 2024-07-31T14:58:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.114+0000 7f8073477640 1 -- 172.21.15.67:0/1131713931 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806c06f490 con 0x7f806c102b40 2024-07-31T14:58:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.114+0000 7f8073477640 1 -- 172.21.15.67:0/1131713931 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f806c11f9d0 con 0x7f806c102b40 2024-07-31T14:58:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.118+0000 7f80799c8740 1 Processor -- start 2024-07-31T14:58:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.118+0000 7f80627fc640 1 -- 172.21.15.67:0/1131713931 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f805c00de50 con 0x7f806c102b40 2024-07-31T14:58:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.118+0000 7f80627fc640 1 -- 172.21.15.67:0/1131713931 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c00f620 con 0x7f806c102b40 2024-07-31T14:58:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.118+0000 7f80799c8740 1 -- start start 2024-07-31T14:58:12.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.120+0000 7f80799c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7eb070 0x56340fb90570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.120+0000 7f80799c8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 0x56340fb90ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.121+0000 7f80799c8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7ca390 0x56340fb90ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.121+0000 7f80799c8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56340fb06bb0 con 0x56340f7ca390 2024-07-31T14:58:12.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.121+0000 7f80799c8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56340faf60b0 con 0x56340f7c21b0 2024-07-31T14:58:12.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.121+0000 7f80799c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56340fafa0f0 con 0x56340f7eb070 2024-07-31T14:58:12.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.123+0000 7f8071213640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 0x56340fb90ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:12.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.123+0000 7f8071213640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 0x56340fb90ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48280/0 (socket says 172.21.15.67:48280) 2024-07-31T14:58:12.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.123+0000 7f8071213640 1 -- 172.21.15.67:0/2398668273 learned_addr learned my addr 172.21.15.67:0/2398668273 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.123+0000 7f80627fc640 1 -- 172.21.15.67:0/1131713931 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f805c01a070 con 0x7f806c102b40 2024-07-31T14:58:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.124+0000 7f8071213640 1 -- 172.21.15.67:0/2398668273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7eb070 msgr2=0x56340fb90570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.124+0000 7f8071a14640 1 --2- 172.21.15.67:0/2398668273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7eb070 0x56340fb90570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.124+0000 7f8071213640 1 --2- 172.21.15.67:0/2398668273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7eb070 0x56340fb90570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.124+0000 7f8071213640 1 -- 172.21.15.67:0/2398668273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7ca390 msgr2=0x56340fb90ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.124+0000 7f8071213640 1 --2- 172.21.15.67:0/2398668273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7ca390 0x56340fb90ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.124+0000 7f80627fc640 1 --2- 172.21.15.67:0/1131713931 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f80580b3e40 0x7f80580b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.124+0000 7f8071213640 1 -- 172.21.15.67:0/2398668273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56340fa9cb90 con 0x56340f7c21b0 2024-07-31T14:58:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.125+0000 7f8071a14640 1 --2- 172.21.15.67:0/2398668273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7eb070 0x56340fb90570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:58:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.125+0000 7f80627fc640 1 -- 172.21.15.67:0/1131713931 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f805c0e3c10 con 0x7f806c102b40 2024-07-31T14:58:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.125+0000 7f8071213640 1 --2- 172.21.15.67:0/2398668273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 0x56340fb90ab0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8064009d20 tx=0x7f806400b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751dae410ac1622a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.125+0000 7f80577fe640 1 -- 172.21.15.67:0/2398668273 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806400bd40 con 0x56340f7c21b0 2024-07-31T14:58:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.125+0000 7f80577fe640 1 -- 172.21.15.67:0/2398668273 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8064017070 con 0x56340f7c21b0 2024-07-31T14:58:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.126+0000 7f8071a14640 1 --2- 172.21.15.67:0/1131713931 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f80580b3e40 0x7f80580b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.127+0000 7f80799c8740 1 -- 172.21.15.67:0/2398668273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 msgr2=0x56340fb90ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.127+0000 7f80799c8740 1 --2- 172.21.15.67:0/2398668273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 0x56340fb90ab0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8064009d20 tx=0x7f806400b420 comp rx=0 tx=0).stop 2024-07-31T14:58:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.127+0000 7f80799c8740 1 -- 172.21.15.67:0/2398668273 shutdown_connections 2024-07-31T14:58:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.127+0000 7f80799c8740 1 --2- 172.21.15.67:0/2398668273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7ca390 0x56340fb90ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.127+0000 7f80799c8740 1 --2- 172.21.15.67:0/2398668273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 0x56340fb90ab0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.127+0000 7f80799c8740 1 --2- 172.21.15.67:0/2398668273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7eb070 0x56340fb90570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.127+0000 7f80799c8740 1 -- 172.21.15.67:0/2398668273 >> 172.21.15.67:0/2398668273 conn(0x56340f7df460 msgr2=0x56340fb8e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.127+0000 7f8071a14640 1 --2- 172.21.15.67:0/1131713931 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f80580b3e40 0x7f80580b6350 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8068005ab0 tx=0x7f8068031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.129+0000 7f80799c8740 1 -- 172.21.15.67:0/2398668273 shutdown_connections 2024-07-31T14:58:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.129+0000 7f80799c8740 1 -- 172.21.15.67:0/2398668273 wait complete. 2024-07-31T14:58:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.130+0000 7f80799c8740 1 -- start start 2024-07-31T14:58:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.130+0000 7f80799c8740 1 Processor -- start 2024-07-31T14:58:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.131+0000 7f80799c8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 0x56340fb9c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.131+0000 7f80799c8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7ca390 0x56340fb9c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.131+0000 7f80799c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7eb070 0x56340fba8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.131+0000 7f80799c8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56340f959700 con 0x56340f7ca390 2024-07-31T14:58:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.131+0000 7f80799c8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56340fa989e0 con 0x56340f7c21b0 2024-07-31T14:58:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.131+0000 7f80799c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56340fb9eda0 con 0x56340f7eb070 2024-07-31T14:58:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.131+0000 7f80799c8740 10 client.0 _ll_register_callbacks cb 0x56340fab34f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.131+0000 7f8071a14640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7eb070 0x56340fba8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.131+0000 7f8071a14640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7eb070 0x56340fba8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58882/0 (socket says 172.21.15.67:58882) 2024-07-31T14:58:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.131+0000 7f8071a14640 1 -- 172.21.15.67:0/3609263775 learned_addr learned my addr 172.21.15.67:0/3609263775 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.131+0000 7f8070a12640 1 --2- 172.21.15.67:0/3609263775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7ca390 0x56340fb9c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.131+0000 7f8071213640 1 --2- 172.21.15.67:0/3609263775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 0x56340fb9c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:12.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.132+0000 7f8071a14640 1 -- 172.21.15.67:0/3609263775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 msgr2=0x56340fb9c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:12.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.132+0000 7f8071a14640 1 --2- 172.21.15.67:0/3609263775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 0x56340fb9c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.132+0000 7f8071a14640 1 -- 172.21.15.67:0/3609263775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7ca390 msgr2=0x56340fb9c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:12.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.132+0000 7f8071a14640 1 --2- 172.21.15.67:0/3609263775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7ca390 0x56340fb9c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.132+0000 7f8071a14640 1 -- 172.21.15.67:0/3609263775 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56340f856650 con 0x56340f7eb070 2024-07-31T14:58:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.132+0000 7f8071a14640 1 --2- 172.21.15.67:0/3609263775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7eb070 0x56340fba8170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f806802a940 tx=0x7f806802a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7b12415beaf8091 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.134+0000 7f8054ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:58:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.134+0000 7f8054ff9640 1 -- 172.21.15.67:0/3609263775 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806804b020 con 0x56340f7eb070 2024-07-31T14:58:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.134+0000 7f8054ff9640 1 -- 172.21.15.67:0/3609263775 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f80680037f0 con 0x56340f7eb070 2024-07-31T14:58:12.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.134+0000 7f80799c8740 1 -- 172.21.15.67:0/3609263775 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8064017070 con 0x56340f7eb070 2024-07-31T14:58:12.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.141+0000 7f8054ff9640 1 -- 172.21.15.67:0/3609263775 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8068056050 con 0x56340f7eb070 2024-07-31T14:58:12.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.143+0000 7f80799c8740 10 client.26944 fetch_fsmap learned FSMap version 50 2024-07-31T14:58:12.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.143+0000 7f80799c8740 1 -- 172.21.15.67:0/3609263775 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56340fbaae70 con 0x56340f7eb070 2024-07-31T14:58:12.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.144+0000 7f8054ff9640 1 -- 172.21.15.67:0/3609263775 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8064017070 con 0x56340f7eb070 2024-07-31T14:58:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.145+0000 7f8054ff9640 1 -- 172.21.15.67:0/3609263775 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f806803bb10 con 0x56340f7eb070 2024-07-31T14:58:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.145+0000 7f80799c8740 10 client.26944 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:58:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.145+0000 7f80799c8740 10 client.26944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:58:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.145+0000 7f80799c8740 20 client.26944 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.145+0000 7f80799c8740 1 --2- 172.21.15.67:0/3609263775 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56340f7a99f0 0x56340fbae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.146+0000 7f80799c8740 4 client.26944 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:58:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.146+0000 7f80799c8740 1 -- 172.21.15.67:0/3609263775 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56340fb08b00 con 0x56340f7a99f0 2024-07-31T14:58:12.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.147+0000 7f8071213640 1 --2- 172.21.15.67:0/3609263775 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56340f7a99f0 0x56340fbae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:12.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.152+0000 7f8071213640 1 --2- 172.21.15.67:0/3609263775 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56340f7a99f0 0x56340fbae8e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1fce278db012c55 server_cookie=43d541e3bc5dc270 in_seq=0 out_seq=0 2024-07-31T14:58:12.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.153+0000 7f8054ff9640 10 client.26944 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:12.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.156+0000 7f8054ff9640 1 -- 172.21.15.67:0/3609263775 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56340fb08b00 con 0x56340f7a99f0 2024-07-31T14:58:12.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.156+0000 7f8054ff9640 10 client.26944 handle_command_reply: tid=0 2024-07-31T14:58:12.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.156+0000 7f80799c8740 1 client.26944 shutdown 2024-07-31T14:58:12.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.157+0000 7f80799c8740 1 -- 172.21.15.67:0/3609263775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7eb070 msgr2=0x56340fba8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:12.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.157+0000 7f80799c8740 1 --2- 172.21.15.67:0/3609263775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7eb070 0x56340fba8170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f806802a940 tx=0x7f806802a420 comp rx=0 tx=0).stop 2024-07-31T14:58:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.158+0000 7f80799c8740 1 -- 172.21.15.67:0/3609263775 shutdown_connections 2024-07-31T14:58:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.158+0000 7f80799c8740 1 --2- 172.21.15.67:0/3609263775 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56340f7a99f0 0x56340fbae8e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.158+0000 7f80799c8740 1 --2- 172.21.15.67:0/3609263775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7eb070 0x56340fba8170 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.158+0000 7f80799c8740 1 --2- 172.21.15.67:0/3609263775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7ca390 0x56340fb9c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.158+0000 7f80799c8740 1 --2- 172.21.15.67:0/3609263775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 0x56340fb9c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.158+0000 7f80799c8740 1 -- 172.21.15.67:0/3609263775 >> 172.21.15.67:0/3609263775 conn(0x56340f7df460 msgr2=0x56340fb8cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.158+0000 7f8054ff9640 0 client.26944 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.159+0000 7f80799c8740 1 -- 172.21.15.67:0/3609263775 shutdown_connections 2024-07-31T14:58:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.159+0000 7f80799c8740 1 -- 172.21.15.67:0/3609263775 wait complete. 2024-07-31T14:58:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.159+0000 7f80799c8740 20 client.26944 trim_cache size 0 max 16384 2024-07-31T14:58:12.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.186+0000 7f80799c8740 1 Processor -- start 2024-07-31T14:58:12.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.186+0000 7f80799c8740 1 -- start start 2024-07-31T14:58:12.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.187+0000 7f80799c8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7c21b0 0x56340fb8d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.187+0000 7f80799c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7ca390 0x56340fb95680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.187+0000 7f80799c8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7eb070 0x56340fb95bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.187+0000 7f80799c8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f805000ffc0 con 0x56340f7c21b0 2024-07-31T14:58:12.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.187+0000 7f80799c8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8050011c50 con 0x56340f7eb070 2024-07-31T14:58:12.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.187+0000 7f80799c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8050010720 con 0x56340f7ca390 2024-07-31T14:58:12.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.188+0000 7f8071213640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7c21b0 0x56340fb8d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:12.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.188+0000 7f8071213640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7c21b0 0x56340fb8d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43956/0 (socket says 172.21.15.67:43956) 2024-07-31T14:58:12.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.188+0000 7f8071213640 1 -- 172.21.15.67:0/561904301 learned_addr learned my addr 172.21.15.67:0/561904301 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:12.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.189+0000 7f8071213640 1 -- 172.21.15.67:0/561904301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7ca390 msgr2=0x56340fb95680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:58:12.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.189+0000 7f8071213640 1 --2- 172.21.15.67:0/561904301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7ca390 0x56340fb95680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.189+0000 7f8071213640 1 -- 172.21.15.67:0/561904301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7eb070 msgr2=0x56340fb95bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:12.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.189+0000 7f8071213640 1 --2- 172.21.15.67:0/561904301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7eb070 0x56340fb95bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.189+0000 7f8071213640 1 -- 172.21.15.67:0/561904301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56340fb9d190 con 0x56340f7c21b0 2024-07-31T14:58:12.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.190+0000 7f8071213640 1 --2- 172.21.15.67:0/561904301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7c21b0 0x56340fb8d350 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8064009d20 tx=0x7f8064012e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4e957cdb38fa74 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:12.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.191+0000 7f8056ffd640 1 -- 172.21.15.67:0/561904301 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80640118a0 con 0x56340f7c21b0 2024-07-31T14:58:12.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.191+0000 7f8056ffd640 1 -- 172.21.15.67:0/561904301 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8064017040 con 0x56340f7c21b0 2024-07-31T14:58:12.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.191+0000 7f8056ffd640 1 -- 172.21.15.67:0/561904301 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80640273b0 con 0x56340f7c21b0 2024-07-31T14:58:12.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.192+0000 7f80799c8740 1 -- 172.21.15.67:0/561904301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7c21b0 msgr2=0x56340fb8d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:12.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.192+0000 7f80799c8740 1 --2- 172.21.15.67:0/561904301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7c21b0 0x56340fb8d350 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8064009d20 tx=0x7f8064012e80 comp rx=0 tx=0).stop 2024-07-31T14:58:12.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.196+0000 7f80799c8740 1 -- 172.21.15.67:0/561904301 shutdown_connections 2024-07-31T14:58:12.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.196+0000 7f80799c8740 1 --2- 172.21.15.67:0/561904301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7eb070 0x56340fb95bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.196+0000 7f80799c8740 1 --2- 172.21.15.67:0/561904301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7ca390 0x56340fb95680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.196+0000 7f80799c8740 1 --2- 172.21.15.67:0/561904301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7c21b0 0x56340fb8d350 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.196+0000 7f80799c8740 1 -- 172.21.15.67:0/561904301 >> 172.21.15.67:0/561904301 conn(0x56340f7df460 msgr2=0x56340fb8cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:12.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.200+0000 7f80799c8740 1 -- 172.21.15.67:0/561904301 shutdown_connections 2024-07-31T14:58:12.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.204+0000 7f80799c8740 1 -- 172.21.15.67:0/561904301 wait complete. 2024-07-31T14:58:12.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.204+0000 7f80799c8740 1 -- start start 2024-07-31T14:58:12.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.205+0000 7f80799c8740 1 Processor -- start 2024-07-31T14:58:12.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.206+0000 7f80799c8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 0x56340fb9f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.206+0000 7f80799c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7ca390 0x56340fb9fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.206+0000 7f80799c8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7eb070 0x56340fc0f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.206+0000 7f80799c8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8050011580 con 0x56340f7eb070 2024-07-31T14:58:12.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.206+0000 7f80799c8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8050010df0 con 0x56340f7c21b0 2024-07-31T14:58:12.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.206+0000 7f80799c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f805000dfc0 con 0x56340f7ca390 2024-07-31T14:58:12.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.206+0000 7f80799c8740 10 client.0 _ll_register_callbacks cb 0x7f8068009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:12.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.206+0000 7f8070a12640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7ca390 0x56340fb9fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:12.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.207+0000 7f8070a12640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7ca390 0x56340fb9fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58894/0 (socket says 172.21.15.67:58894) 2024-07-31T14:58:12.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.207+0000 7f8070a12640 1 -- 172.21.15.67:0/285752690 learned_addr learned my addr 172.21.15.67:0/285752690 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:12.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.208+0000 7f8070a12640 1 -- 172.21.15.67:0/285752690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 msgr2=0x56340fb9f880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:58:12.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.208+0000 7f8070a12640 1 --2- 172.21.15.67:0/285752690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 0x56340fb9f880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.208+0000 7f8070a12640 1 -- 172.21.15.67:0/285752690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7eb070 msgr2=0x56340fc0f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:12.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.208+0000 7f8070a12640 1 --2- 172.21.15.67:0/285752690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7eb070 0x56340fc0f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.208+0000 7f8070a12640 1 -- 172.21.15.67:0/285752690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56340fc0f6c0 con 0x56340f7ca390 2024-07-31T14:58:12.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.208+0000 7f8070a12640 1 --2- 172.21.15.67:0/285752690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7ca390 0x56340fb9fe60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f805c0aa390 tx=0x7f805c0a9c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6b863340c13013c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:12.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.208+0000 7f8054ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:58:12.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.209+0000 7f8054ff9640 1 -- 172.21.15.67:0/285752690 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c021040 con 0x56340f7ca390 2024-07-31T14:58:12.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.209+0000 7f80799c8740 1 -- 172.21.15.67:0/285752690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8064017040 con 0x56340f7ca390 2024-07-31T14:58:12.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.209+0000 7f8054ff9640 1 -- 172.21.15.67:0/285752690 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f805c0b3230 con 0x56340f7ca390 2024-07-31T14:58:12.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.209+0000 7f8054ff9640 1 -- 172.21.15.67:0/285752690 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f805c0aa400 con 0x56340f7ca390 2024-07-31T14:58:12.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.210+0000 7f80799c8740 10 client.26947 fetch_fsmap learned FSMap version 50 2024-07-31T14:58:12.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.210+0000 7f80799c8740 1 -- 172.21.15.67:0/285752690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56340fc169e0 con 0x56340f7ca390 2024-07-31T14:58:12.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.213+0000 7f8054ff9640 1 -- 172.21.15.67:0/285752690 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8064017040 con 0x56340f7ca390 2024-07-31T14:58:12.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.213+0000 7f8054ff9640 1 -- 172.21.15.67:0/285752690 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f805c0a5d70 con 0x56340f7ca390 2024-07-31T14:58:12.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.213+0000 7f80799c8740 10 client.26947 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:58:12.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.213+0000 7f80799c8740 10 client.26947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:58:12.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.213+0000 7f80799c8740 20 client.26947 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:12.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.213+0000 7f80799c8740 1 --2- 172.21.15.67:0/285752690 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56340f77eda0 0x56340fc191c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:12.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.213+0000 7f80799c8740 4 client.26947 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:58:12.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.213+0000 7f80799c8740 1 -- 172.21.15.67:0/285752690 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56340fb045a0 con 0x56340f77eda0 2024-07-31T14:58:12.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.218+0000 7f8071213640 1 --2- 172.21.15.67:0/285752690 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56340f77eda0 0x56340fc191c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:12.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.219+0000 7f8071213640 1 --2- 172.21.15.67:0/285752690 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56340f77eda0 0x56340fc191c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ece1dc2e0cfc49e server_cookie=a65ad56e917bf545 in_seq=0 out_seq=0 2024-07-31T14:58:12.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.219+0000 7f8054ff9640 10 client.26947 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:12.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.220+0000 7f8054ff9640 1 -- 172.21.15.67:0/285752690 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56340fb045a0 con 0x56340f77eda0 2024-07-31T14:58:12.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.220+0000 7f8054ff9640 10 client.26947 handle_command_reply: tid=0 2024-07-31T14:58:12.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.220+0000 7f80799c8740 1 client.26947 shutdown 2024-07-31T14:58:12.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.222+0000 7f80799c8740 1 -- 172.21.15.67:0/285752690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7ca390 msgr2=0x56340fb9fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:12.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.222+0000 7f80799c8740 1 --2- 172.21.15.67:0/285752690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7ca390 0x56340fb9fe60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f805c0aa390 tx=0x7f805c0a9c30 comp rx=0 tx=0).stop 2024-07-31T14:58:12.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.222+0000 7f80799c8740 1 -- 172.21.15.67:0/285752690 shutdown_connections 2024-07-31T14:58:12.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.222+0000 7f80799c8740 1 --2- 172.21.15.67:0/285752690 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56340f77eda0 0x56340fc191c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.222+0000 7f80799c8740 1 --2- 172.21.15.67:0/285752690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56340f7eb070 0x56340fc0f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.222+0000 7f80799c8740 1 --2- 172.21.15.67:0/285752690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56340f7ca390 0x56340fb9fe60 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f805c0aa390 tx=0x7f805c0a9c30 comp rx=0 tx=0).stop 2024-07-31T14:58:12.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.222+0000 7f80799c8740 1 --2- 172.21.15.67:0/285752690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56340f7c21b0 0x56340fb9f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.222+0000 7f80799c8740 1 -- 172.21.15.67:0/285752690 >> 172.21.15.67:0/285752690 conn(0x56340f7df460 msgr2=0x56340fb95680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:12.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.222+0000 7f8054ff9640 0 client.26947 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:12.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.223+0000 7f80799c8740 1 -- 172.21.15.67:0/285752690 shutdown_connections 2024-07-31T14:58:12.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.223+0000 7f80799c8740 1 -- 172.21.15.67:0/285752690 wait complete. 2024-07-31T14:58:12.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.223+0000 7f80799c8740 20 client.26947 trim_cache size 0 max 16384 2024-07-31T14:58:12.225 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:58:12.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.224+0000 7f8073477640 1 -- 172.21.15.67:0/1131713931 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f80580b3e40 msgr2=0x7f80580b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:12.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.224+0000 7f8073477640 1 --2- 172.21.15.67:0/1131713931 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f80580b3e40 0x7f80580b6350 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8068005ab0 tx=0x7f8068031400 comp rx=0 tx=0).stop 2024-07-31T14:58:12.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.224+0000 7f8073477640 1 -- 172.21.15.67:0/1131713931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f806c102b40 msgr2=0x7f806c074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:12.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.224+0000 7f8073477640 1 --2- 172.21.15.67:0/1131713931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f806c102b40 0x7f806c074a50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f805c00d740 tx=0x7f805c00dc00 comp rx=0 tx=0).stop 2024-07-31T14:58:12.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.224+0000 7f8073477640 1 -- 172.21.15.67:0/1131713931 shutdown_connections 2024-07-31T14:58:12.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.224+0000 7f8073477640 1 --2- 172.21.15.67:0/1131713931 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f80580b3e40 0x7f80580b6350 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.224+0000 7f8073477640 1 --2- 172.21.15.67:0/1131713931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f806c107630 0x7f806c06e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.224+0000 7f8073477640 1 --2- 172.21.15.67:0/1131713931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f806c102b40 0x7f806c074a50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.224+0000 7f8073477640 1 --2- 172.21.15.67:0/1131713931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f806c1021f0 0x7f806c0744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:12.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.224+0000 7f8073477640 1 -- 172.21.15.67:0/1131713931 >> 172.21.15.67:0/1131713931 conn(0x7f806c079170 msgr2=0x7f806c105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:12.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.225+0000 7f8073477640 1 -- 172.21.15.67:0/1131713931 shutdown_connections 2024-07-31T14:58:12.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:12.225+0000 7f8073477640 1 -- 172.21.15.67:0/1131713931 wait complete. 2024-07-31T14:58:13.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:12 smithi077 ceph-mon[37327]: pgmap v615: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 1.30k op/s 2024-07-31T14:58:13.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:12 smithi067 ceph-mon[32750]: pgmap v615: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 1.30k op/s 2024-07-31T14:58:13.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:12 smithi073 ceph-mon[36714]: pgmap v615: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 1.30k op/s 2024-07-31T14:58:14.417 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T14:58:14.754 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T14:58:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:14 smithi077 ceph-mon[37327]: pgmap v616: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.9 MiB/s wr, 1.21k op/s 2024-07-31T14:58:15.301 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:58:15.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:14 smithi067 ceph-mon[32750]: pgmap v616: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.9 MiB/s wr, 1.21k op/s 2024-07-31T14:58:15.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:14 smithi073 ceph-mon[36714]: pgmap v616: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.9 MiB/s wr, 1.21k op/s 2024-07-31T14:58:17.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:16 smithi077 ceph-mon[37327]: pgmap v617: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 1.17k op/s 2024-07-31T14:58:17.245 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:16 smithi067 ceph-mon[32750]: pgmap v617: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 1.17k op/s 2024-07-31T14:58:17.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:16 smithi073 ceph-mon[36714]: pgmap v617: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 1.17k op/s 2024-07-31T14:58:18.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.013+0000 7fe4b8bda640 1 -- 172.21.15.67:0/3480802724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4b410a410 msgr2=0x7fe4b410a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.013+0000 7fe4b8bda640 1 --2- 172.21.15.67:0/3480802724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4b410a410 0x7fe4b410a820 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009ca0 tx=0x7fe4ac02a8f0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.014+0000 7fe4b8bda640 1 -- 172.21.15.67:0/3480802724 shutdown_connections 2024-07-31T14:58:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.014+0000 7fe4b8bda640 1 --2- 172.21.15.67:0/3480802724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4b411c660 0x7fe4b411eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.014+0000 7fe4b8bda640 1 --2- 172.21.15.67:0/3480802724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4b410a410 0x7fe4b410a820 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.014+0000 7fe4b8bda640 1 --2- 172.21.15.67:0/3480802724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4b4111970 0x7fe4b4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.014+0000 7fe4b8bda640 1 -- 172.21.15.67:0/3480802724 >> 172.21.15.67:0/3480802724 conn(0x7fe4b40708f0 msgr2=0x7fe4b4070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:18.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.016+0000 7fe4b8bda640 1 -- 172.21.15.67:0/3480802724 shutdown_connections 2024-07-31T14:58:18.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.017+0000 7fe4b8bda640 1 -- 172.21.15.67:0/3480802724 wait complete. 2024-07-31T14:58:18.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.018+0000 7fe4b8bda640 1 Processor -- start 2024-07-31T14:58:18.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.021+0000 7fe4b8bda640 1 -- start start 2024-07-31T14:58:18.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.021+0000 7fe4b8bda640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4b410a410 0x7fe4b41a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.022+0000 7fe4b8bda640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4b4111970 0x7fe4b41a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.022+0000 7fe4b8bda640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4b411c660 0x7fe4b41ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.022+0000 7fe4b8bda640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b4122370 con 0x7fe4b410a410 2024-07-31T14:58:18.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.022+0000 7fe4b8bda640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b41221f0 con 0x7fe4b411c660 2024-07-31T14:58:18.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.022+0000 7fe4b8bda640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b41224f0 con 0x7fe4b4111970 2024-07-31T14:58:18.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.024+0000 7fe4b1d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4b4111970 0x7fe4b41a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:18.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.024+0000 7fe4b1d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4b4111970 0x7fe4b41a92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58898/0 (socket says 172.21.15.67:58898) 2024-07-31T14:58:18.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.024+0000 7fe4b1d9b640 1 -- 172.21.15.67:0/3169969465 learned_addr learned my addr 172.21.15.67:0/3169969465 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:18.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.024+0000 7fe4b1d9b640 1 -- 172.21.15.67:0/3169969465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4b411c660 msgr2=0x7fe4b41ae560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T14:58:18.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.024+0000 7fe4b1d9b640 1 --2- 172.21.15.67:0/3169969465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4b411c660 0x7fe4b41ae560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.024+0000 7fe4b1d9b640 1 -- 172.21.15.67:0/3169969465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4b410a410 msgr2=0x7fe4b41a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:18.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.024+0000 7fe4b1d9b640 1 --2- 172.21.15.67:0/3169969465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4b410a410 0x7fe4b41a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.024+0000 7fe4b1d9b640 1 -- 172.21.15.67:0/3169969465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b41aec90 con 0x7fe4b4111970 2024-07-31T14:58:18.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.025+0000 7fe4b1d9b640 1 --2- 172.21.15.67:0/3169969465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4b4111970 0x7fe4b41a92c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009bf0 tx=0x7fe4ac0307c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:18.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.025+0000 7fe4a37fe640 1 -- 172.21.15.67:0/3169969465 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac030c30 con 0x7fe4b4111970 2024-07-31T14:58:18.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.026+0000 7fe4b8bda640 1 -- 172.21.15.67:0/3169969465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b41aef20 con 0x7fe4b4111970 2024-07-31T14:58:18.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.026+0000 7fe4b8bda640 1 -- 172.21.15.67:0/3169969465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b41b56e0 con 0x7fe4b4111970 2024-07-31T14:58:18.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.026+0000 7fe4a37fe640 1 -- 172.21.15.67:0/3169969465 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4ac02d070 con 0x7fe4b4111970 2024-07-31T14:58:18.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.026+0000 7fe4a37fe640 1 -- 172.21.15.67:0/3169969465 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac004570 con 0x7fe4b4111970 2024-07-31T14:58:18.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.031+0000 7fe4a37fe640 1 -- 172.21.15.67:0/3169969465 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe4ac004710 con 0x7fe4b4111970 2024-07-31T14:58:18.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.033+0000 7fe4a37fe640 1 --2- 172.21.15.67:0/3169969465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe49c0b3e70 0x7fe49c0b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.033+0000 7fe4a37fe640 1 -- 172.21.15.67:0/3169969465 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7fe4ac1069f0 con 0x7fe4b4111970 2024-07-31T14:58:18.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.034+0000 7fe4bf12b740 1 Processor -- start 2024-07-31T14:58:18.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.037+0000 7fe4bf12b740 1 -- start start 2024-07-31T14:58:18.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.038+0000 7fe4bf12b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e3c350 0x5616711e13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.038+0000 7fe4bf12b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e13490 0x5616711e18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.038+0000 7fe4bf12b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e1b670 0x5616711e1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.038+0000 7fe4bf12b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5616710c92f0 con 0x561670e13490 2024-07-31T14:58:18.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.038+0000 7fe4bf12b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561671157a80 con 0x561670e3c350 2024-07-31T14:58:18.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.038+0000 7fe4bf12b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561671146f80 con 0x561670e1b670 2024-07-31T14:58:18.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.039+0000 7fe4b2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e1b670 0x5616711e1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:18.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.039+0000 7fe4b2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e1b670 0x5616711e1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58902/0 (socket says 172.21.15.67:58902) 2024-07-31T14:58:18.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.039+0000 7fe4b2d9d640 1 -- 172.21.15.67:0/3337463640 learned_addr learned my addr 172.21.15.67:0/3337463640 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:18.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.040+0000 7fe4b259c640 1 --2- 172.21.15.67:0/3169969465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe49c0b3e70 0x7fe49c0b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:18.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.040+0000 7fe4b1d9b640 1 --2- 172.21.15.67:0/3337463640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e13490 0x5616711e18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:18.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.040+0000 7fe4b2d9d640 1 -- 172.21.15.67:0/3337463640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e3c350 msgr2=0x5616711e13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:18.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.040+0000 7fe4b2d9d640 1 --2- 172.21.15.67:0/3337463640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e3c350 0x5616711e13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.040+0000 7fe4b2d9d640 1 -- 172.21.15.67:0/3337463640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e13490 msgr2=0x5616711e18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:18.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.040+0000 7fe4b2d9d640 1 --2- 172.21.15.67:0/3337463640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e13490 0x5616711e18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.040+0000 7fe4b2d9d640 1 -- 172.21.15.67:0/3337463640 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561670ea7930 con 0x561670e1b670 2024-07-31T14:58:18.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.041+0000 7fe4b1d9b640 1 --2- 172.21.15.67:0/3337463640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e13490 0x5616711e18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:58:18.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.041+0000 7fe4b259c640 1 --2- 172.21.15.67:0/3169969465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe49c0b3e70 0x7fe49c0b6380 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800a960 tx=0x7fe4a800ad60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:18.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.044+0000 7fe4b2d9d640 1 --2- 172.21.15.67:0/3337463640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e1b670 0x5616711e1e30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009720 tx=0x7fe4a4009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6b1021d0342cfae server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:18.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.051+0000 7fe4a0ff9640 1 -- 172.21.15.67:0/3337463640 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4015020 con 0x561670e1b670 2024-07-31T14:58:18.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.051+0000 7fe4a0ff9640 1 -- 172.21.15.67:0/3337463640 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4a4009e30 con 0x561670e1b670 2024-07-31T14:58:18.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.051+0000 7fe4a0ff9640 1 -- 172.21.15.67:0/3337463640 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4010380 con 0x561670e1b670 2024-07-31T14:58:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.053+0000 7fe4bf12b740 1 -- 172.21.15.67:0/3337463640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e1b670 msgr2=0x5616711e1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.053+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/3337463640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e1b670 0x5616711e1e30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009720 tx=0x7fe4a4009be0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.057+0000 7fe4bf12b740 1 -- 172.21.15.67:0/3337463640 shutdown_connections 2024-07-31T14:58:18.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.057+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/3337463640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e1b670 0x5616711e1e30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.057+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/3337463640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e13490 0x5616711e18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.057+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/3337463640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e3c350 0x5616711e13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.057+0000 7fe4bf12b740 1 -- 172.21.15.67:0/3337463640 >> 172.21.15.67:0/3337463640 conn(0x561670e30740 msgr2=0x5616711df2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:18.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.064+0000 7fe4bf12b740 1 -- 172.21.15.67:0/3337463640 shutdown_connections 2024-07-31T14:58:18.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.067+0000 7fe4bf12b740 1 -- 172.21.15.67:0/3337463640 wait complete. 2024-07-31T14:58:18.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.067+0000 7fe4bf12b740 1 -- start start 2024-07-31T14:58:18.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.067+0000 7fe4bf12b740 1 Processor -- start 2024-07-31T14:58:18.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.068+0000 7fe4bf12b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e13490 0x5616711ecf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.068+0000 7fe4bf12b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e1b670 0x5616711ed470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.068+0000 7fe4bf12b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e3c350 0x5616711f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.068+0000 7fe4bf12b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5616710e94f0 con 0x561670e3c350 2024-07-31T14:58:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.068+0000 7fe4bf12b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56167114afc0 con 0x561670e1b670 2024-07-31T14:58:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.068+0000 7fe4bf12b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561670faa890 con 0x561670e13490 2024-07-31T14:58:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.068+0000 7fe4bf12b740 10 client.0 _ll_register_callbacks cb 0x5616710f3500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.069+0000 7fe4b2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e3c350 0x5616711f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.069+0000 7fe4b2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e3c350 0x5616711f8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44000/0 (socket says 172.21.15.67:44000) 2024-07-31T14:58:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.069+0000 7fe4b2d9d640 1 -- 172.21.15.67:0/3757221108 learned_addr learned my addr 172.21.15.67:0/3757221108 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:18.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.069+0000 7fe4b2d9d640 1 -- 172.21.15.67:0/3757221108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e13490 msgr2=0x5616711ecf30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:58:18.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.069+0000 7fe4b2d9d640 1 --2- 172.21.15.67:0/3757221108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e13490 0x5616711ecf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.069+0000 7fe4b2d9d640 1 -- 172.21.15.67:0/3757221108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e1b670 msgr2=0x5616711ed470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:58:18.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.069+0000 7fe4b2d9d640 1 --2- 172.21.15.67:0/3757221108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e1b670 0x5616711ed470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.069+0000 7fe4b2d9d640 1 -- 172.21.15.67:0/3757221108 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616710f8c50 con 0x561670e3c350 2024-07-31T14:58:18.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.073+0000 7fe4b2d9d640 1 --2- 172.21.15.67:0/3757221108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e3c350 0x5616711f8d60 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4011bc0 tx=0x7fe4a4011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2667faeecf773f22 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:18.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.073+0000 7fe499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:58:18.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.074+0000 7fe499ffb640 1 -- 172.21.15.67:0/3757221108 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4015030 con 0x561670e3c350 2024-07-31T14:58:18.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.074+0000 7fe4bf12b740 1 -- 172.21.15.67:0/3757221108 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a4009e30 con 0x561670e3c350 2024-07-31T14:58:18.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.075+0000 7fe499ffb640 1 -- 172.21.15.67:0/3757221108 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7fe4a400bce0 con 0x561670e3c350 2024-07-31T14:58:18.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.075+0000 7fe499ffb640 1 -- 172.21.15.67:0/3757221108 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe4a401a420 con 0x561670e3c350 2024-07-31T14:58:18.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.075+0000 7fe4bf12b740 10 client.17376 fetch_fsmap learned FSMap version 50 2024-07-31T14:58:18.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.075+0000 7fe4bf12b740 1 -- 172.21.15.67:0/3757221108 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5616711fb9e0 con 0x561670e3c350 2024-07-31T14:58:18.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.076+0000 7fe499ffb640 1 -- 172.21.15.67:0/3757221108 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4a4009e30 con 0x561670e3c350 2024-07-31T14:58:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.076+0000 7fe499ffb640 1 -- 172.21.15.67:0/3757221108 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7fe4a4022670 con 0x561670e3c350 2024-07-31T14:58:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.076+0000 7fe4bf12b740 10 client.17376 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:58:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.076+0000 7fe4bf12b740 10 client.17376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:58:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.076+0000 7fe4bf12b740 20 client.17376 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.076+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/3757221108 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561670dfacd0 0x5616711ff310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.076+0000 7fe4bf12b740 4 client.17376 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:58:18.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.076+0000 7fe4bf12b740 1 -- 172.21.15.67:0/3757221108 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616711599d0 con 0x561670dfacd0 2024-07-31T14:58:18.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.078+0000 7fe4b259c640 1 --2- 172.21.15.67:0/3757221108 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561670dfacd0 0x5616711ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:18.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.081+0000 7fe4b259c640 1 --2- 172.21.15.67:0/3757221108 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561670dfacd0 0x5616711ff310 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcdafd8aa70f40af server_cookie=8cf432a23481e4f4 in_seq=0 out_seq=0 2024-07-31T14:58:18.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.081+0000 7fe499ffb640 10 client.17376 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:18.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.105+0000 7fe499ffb640 1 -- 172.21.15.67:0/3757221108 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616711599d0 con 0x561670dfacd0 2024-07-31T14:58:18.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.105+0000 7fe499ffb640 10 client.17376 handle_command_reply: tid=0 2024-07-31T14:58:18.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.105+0000 7fe4bf12b740 1 client.17376 shutdown 2024-07-31T14:58:18.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.106+0000 7fe4bf12b740 1 -- 172.21.15.67:0/3757221108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e3c350 msgr2=0x5616711f8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:18.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.106+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/3757221108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e3c350 0x5616711f8d60 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4011bc0 tx=0x7fe4a4011650 comp rx=0 tx=0).stop 2024-07-31T14:58:18.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.106+0000 7fe4bf12b740 1 -- 172.21.15.67:0/3757221108 shutdown_connections 2024-07-31T14:58:18.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.106+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/3757221108 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561670dfacd0 0x5616711ff310 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.106+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/3757221108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e3c350 0x5616711f8d60 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.106+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/3757221108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e1b670 0x5616711ed470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.106+0000 7fe499ffb640 0 client.17376 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.106+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/3757221108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e13490 0x5616711ecf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.106+0000 7fe4bf12b740 1 -- 172.21.15.67:0/3757221108 >> 172.21.15.67:0/3757221108 conn(0x561670e30740 msgr2=0x5616711e4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.106+0000 7fe4bf12b740 1 -- 172.21.15.67:0/3757221108 shutdown_connections 2024-07-31T14:58:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.106+0000 7fe4bf12b740 1 -- 172.21.15.67:0/3757221108 wait complete. 2024-07-31T14:58:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.106+0000 7fe4bf12b740 20 client.17376 trim_cache size 0 max 16384 2024-07-31T14:58:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.142+0000 7fe4bf12b740 1 Processor -- start 2024-07-31T14:58:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.142+0000 7fe4bf12b740 1 -- start start 2024-07-31T14:58:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.142+0000 7fe4bf12b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e13490 0x5616711ee960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.143+0000 7fe4bf12b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e1b670 0x5616711f8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.143+0000 7fe4bf12b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e3c350 0x5616711f9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.143+0000 7fe4b1d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e1b670 0x5616711f8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:18.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.143+0000 7fe4b1d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e1b670 0x5616711f8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58910/0 (socket says 172.21.15.67:58910) 2024-07-31T14:58:18.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.143+0000 7fe4b2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e3c350 0x5616711f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:18.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.143+0000 7fe4b2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e3c350 0x5616711f9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44010/0 (socket says 172.21.15.67:44010) 2024-07-31T14:58:18.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.143+0000 7fe4b1d9b640 1 -- 172.21.15.67:0/766885933 learned_addr learned my addr 172.21.15.67:0/766885933 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:18.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.143+0000 7fe4bf12b740 1 -- 172.21.15.67:0/766885933 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe4940100b0 con 0x561670e3c350 2024-07-31T14:58:18.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.143+0000 7fe4bf12b740 1 -- 172.21.15.67:0/766885933 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe494011d40 con 0x561670e13490 2024-07-31T14:58:18.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.143+0000 7fe4bf12b740 1 -- 172.21.15.67:0/766885933 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe494010810 con 0x561670e1b670 2024-07-31T14:58:18.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.144+0000 7fe4b1d9b640 1 -- 172.21.15.67:0/766885933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e13490 msgr2=0x5616711ee960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:58:18.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.144+0000 7fe4b1d9b640 1 --2- 172.21.15.67:0/766885933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e13490 0x5616711ee960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.144+0000 7fe4b1d9b640 1 -- 172.21.15.67:0/766885933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e3c350 msgr2=0x5616711f9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:18.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.144+0000 7fe4b1d9b640 1 --2- 172.21.15.67:0/766885933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e3c350 0x5616711f9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.144+0000 7fe4b1d9b640 1 -- 172.21.15.67:0/766885933 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616711e36a0 con 0x561670e1b670 2024-07-31T14:58:18.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.144+0000 7fe4b1d9b640 1 --2- 172.21.15.67:0/766885933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e1b670 0x5616711f8b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0c9370 tx=0x7fe4ac02a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e901603d0ec3339 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.146+0000 7fe49bfff640 1 -- 172.21.15.67:0/766885933 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac0cd9f0 con 0x561670e1b670 2024-07-31T14:58:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.146+0000 7fe49bfff640 1 -- 172.21.15.67:0/766885933 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4ac036030 con 0x561670e1b670 2024-07-31T14:58:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.146+0000 7fe49bfff640 1 -- 172.21.15.67:0/766885933 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac103fa0 con 0x561670e1b670 2024-07-31T14:58:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.147+0000 7fe4bf12b740 1 -- 172.21.15.67:0/766885933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e1b670 msgr2=0x5616711f8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.147+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/766885933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e1b670 0x5616711f8b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0c9370 tx=0x7fe4ac02a8f0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.147+0000 7fe4bf12b740 1 -- 172.21.15.67:0/766885933 shutdown_connections 2024-07-31T14:58:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.147+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/766885933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e3c350 0x5616711f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.147+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/766885933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670e1b670 0x5616711f8b50 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0c9370 tx=0x7fe4ac02a8f0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.147+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/766885933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e13490 0x5616711ee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.147+0000 7fe4bf12b740 1 -- 172.21.15.67:0/766885933 >> 172.21.15.67:0/766885933 conn(0x561670e30740 msgr2=0x5616711ee250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.149+0000 7fe4bf12b740 1 -- 172.21.15.67:0/766885933 shutdown_connections 2024-07-31T14:58:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.149+0000 7fe4bf12b740 1 -- 172.21.15.67:0/766885933 wait complete. 2024-07-31T14:58:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.150+0000 7fe4bf12b740 1 -- start start 2024-07-31T14:58:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.150+0000 7fe4bf12b740 1 Processor -- start 2024-07-31T14:58:18.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.157+0000 7fe4bf12b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e13490 0x5616711f5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.157+0000 7fe4bf12b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e3c350 0x5616711f0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.157+0000 7fe4bf12b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670dd0080 0x5616711f1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.157+0000 7fe4bf12b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe494011670 con 0x561670e13490 2024-07-31T14:58:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.158+0000 7fe4b259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e13490 0x5616711f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.158+0000 7fe4b259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e13490 0x5616711f5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44014/0 (socket says 172.21.15.67:44014) 2024-07-31T14:58:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.158+0000 7fe4b259c640 1 -- 172.21.15.67:0/2629567314 learned_addr learned my addr 172.21.15.67:0/2629567314 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.158+0000 7fe4bf12b740 1 -- 172.21.15.67:0/2629567314 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe494010ee0 con 0x561670e3c350 2024-07-31T14:58:18.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.158+0000 7fe4bf12b740 1 -- 172.21.15.67:0/2629567314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe49400e0b0 con 0x561670dd0080 2024-07-31T14:58:18.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.158+0000 7fe4bf12b740 10 client.0 _ll_register_callbacks cb 0x7fe4a4009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:18.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.158+0000 7fe4b2d9d640 1 --2- 172.21.15.67:0/2629567314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670dd0080 0x5616711f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:18.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.159+0000 7fe4b2d9d640 1 -- 172.21.15.67:0/2629567314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e3c350 msgr2=0x5616711f0a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:58:18.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.159+0000 7fe4b2d9d640 1 --2- 172.21.15.67:0/2629567314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e3c350 0x5616711f0a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.159+0000 7fe4b2d9d640 1 -- 172.21.15.67:0/2629567314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e13490 msgr2=0x5616711f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:18.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.159+0000 7fe4b2d9d640 1 --2- 172.21.15.67:0/2629567314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e13490 0x5616711f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.159+0000 7fe4b2d9d640 1 -- 172.21.15.67:0/2629567314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561671277b90 con 0x561670dd0080 2024-07-31T14:58:18.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.160+0000 7fe4b259c640 1 --2- 172.21.15.67:0/2629567314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e13490 0x5616711f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:58:18.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.160+0000 7fe4b2d9d640 1 --2- 172.21.15.67:0/2629567314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670dd0080 0x5616711f1140 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400b4b0 tx=0x7fe4a40117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c18ac7e5f66b8fda server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:18.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.161+0000 7fe499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:58:18.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.161+0000 7fe499ffb640 1 -- 172.21.15.67:0/2629567314 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4015070 con 0x561670dd0080 2024-07-31T14:58:18.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.161+0000 7fe499ffb640 1 -- 172.21.15.67:0/2629567314 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7fe4a40223f0 con 0x561670dd0080 2024-07-31T14:58:18.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.162+0000 7fe4bf12b740 1 -- 172.21.15.67:0/2629567314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4ac036030 con 0x561670dd0080 2024-07-31T14:58:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.162+0000 7fe499ffb640 1 -- 172.21.15.67:0/2629567314 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe4a40225f0 con 0x561670dd0080 2024-07-31T14:58:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.162+0000 7fe4bf12b740 10 client.26959 fetch_fsmap learned FSMap version 50 2024-07-31T14:58:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.162+0000 7fe4bf12b740 1 -- 172.21.15.67:0/2629567314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x561671278cc0 con 0x561670dd0080 2024-07-31T14:58:18.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.166+0000 7fe499ffb640 1 -- 172.21.15.67:0/2629567314 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4ac036030 con 0x561670dd0080 2024-07-31T14:58:18.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.166+0000 7fe499ffb640 1 -- 172.21.15.67:0/2629567314 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7fe4a4037450 con 0x561670dd0080 2024-07-31T14:58:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.167+0000 7fe4bf12b740 10 client.26959 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:58:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.167+0000 7fe4bf12b740 10 client.26959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:58:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.167+0000 7fe4bf12b740 20 client.26959 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.167+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/2629567314 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561671279380 0x56167127b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.167+0000 7fe4bf12b740 4 client.26959 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T14:58:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.167+0000 7fe4bf12b740 1 -- 172.21.15.67:0/2629567314 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561671155470 con 0x561671279380 2024-07-31T14:58:18.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.172+0000 7fe4b259c640 1 --2- 172.21.15.67:0/2629567314 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561671279380 0x56167127b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.175+0000 7fe4b259c640 1 --2- 172.21.15.67:0/2629567314 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561671279380 0x56167127b7a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29589a32ca8517ce server_cookie=3bd609bc52db06aa in_seq=0 out_seq=0 2024-07-31T14:58:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.175+0000 7fe499ffb640 10 client.26959 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:18.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.220+0000 7fe499ffb640 1 -- 172.21.15.67:0/2629567314 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561671155470 con 0x561671279380 2024-07-31T14:58:18.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.220+0000 7fe499ffb640 10 client.26959 handle_command_reply: tid=0 2024-07-31T14:58:18.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.220+0000 7fe4bf12b740 1 client.26959 shutdown 2024-07-31T14:58:18.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.220+0000 7fe4bf12b740 1 -- 172.21.15.67:0/2629567314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670dd0080 msgr2=0x5616711f1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:18.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.220+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/2629567314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670dd0080 0x5616711f1140 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400b4b0 tx=0x7fe4a40117b0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.221+0000 7fe4bf12b740 1 -- 172.21.15.67:0/2629567314 shutdown_connections 2024-07-31T14:58:18.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.221+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/2629567314 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561671279380 0x56167127b7a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.221+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/2629567314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561670dd0080 0x5616711f1140 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.221+0000 7fe499ffb640 0 client.26959 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:18.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.221+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/2629567314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561670e3c350 0x5616711f0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.221+0000 7fe4bf12b740 1 --2- 172.21.15.67:0/2629567314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561670e13490 0x5616711f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.221+0000 7fe4bf12b740 1 -- 172.21.15.67:0/2629567314 >> 172.21.15.67:0/2629567314 conn(0x561670e30740 msgr2=0x5616711df2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:18.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.221+0000 7fe4bf12b740 1 -- 172.21.15.67:0/2629567314 shutdown_connections 2024-07-31T14:58:18.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.221+0000 7fe4bf12b740 1 -- 172.21.15.67:0/2629567314 wait complete. 2024-07-31T14:58:18.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.221+0000 7fe4bf12b740 20 client.26959 trim_cache size 0 max 16384 2024-07-31T14:58:18.227 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:58:18.227 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T14:58:18.227 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "d7bf5c48-7b0c-4bad-a7d0-07bd7c41685f", 2024-07-31T14:58:18.227 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T14:58:18.228 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:58:18.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.222+0000 7fe4b8bda640 1 -- 172.21.15.67:0/3169969465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe49c0b3e70 msgr2=0x7fe49c0b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:18.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.222+0000 7fe4b8bda640 1 --2- 172.21.15.67:0/3169969465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe49c0b3e70 0x7fe49c0b6380 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800a960 tx=0x7fe4a800ad60 comp rx=0 tx=0).stop 2024-07-31T14:58:18.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.222+0000 7fe4b8bda640 1 -- 172.21.15.67:0/3169969465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4b4111970 msgr2=0x7fe4b41a92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:18.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.222+0000 7fe4b8bda640 1 --2- 172.21.15.67:0/3169969465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4b4111970 0x7fe4b41a92c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009bf0 tx=0x7fe4ac0307c0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.222+0000 7fe4b8bda640 1 -- 172.21.15.67:0/3169969465 shutdown_connections 2024-07-31T14:58:18.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.222+0000 7fe4b8bda640 1 --2- 172.21.15.67:0/3169969465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe49c0b3e70 0x7fe49c0b6380 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.222+0000 7fe4b8bda640 1 --2- 172.21.15.67:0/3169969465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4b411c660 0x7fe4b41ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.223+0000 7fe4b8bda640 1 --2- 172.21.15.67:0/3169969465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4b4111970 0x7fe4b41a92c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.223+0000 7fe4b8bda640 1 --2- 172.21.15.67:0/3169969465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4b410a410 0x7fe4b41a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:18.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.223+0000 7fe4b8bda640 1 -- 172.21.15.67:0/3169969465 >> 172.21.15.67:0/3169969465 conn(0x7fe4b40708f0 msgr2=0x7fe4b4071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:18.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.223+0000 7fe4b8bda640 1 -- 172.21.15.67:0/3169969465 shutdown_connections 2024-07-31T14:58:18.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:18.223+0000 7fe4b8bda640 1 -- 172.21.15.67:0/3169969465 wait complete. 2024-07-31T14:58:19.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:18 smithi077 ceph-mon[37327]: pgmap v618: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.1 MiB/s wr, 1.05k op/s 2024-07-31T14:58:19.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:19.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:19.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:19.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:19.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:18 smithi067 ceph-mon[32750]: pgmap v618: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.1 MiB/s wr, 1.05k op/s 2024-07-31T14:58:19.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:19.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:19.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:19.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:19.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:18 smithi073 ceph-mon[36714]: pgmap v618: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.1 MiB/s wr, 1.05k op/s 2024-07-31T14:58:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:19.943 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 14:58:19 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T14:58:19.619+0000 7f4cfe9c3640 -1 calc_pg_upmaps abort due to max <= 0 2024-07-31T14:58:20.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:19 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T14:58:20.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:20.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:20.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:20.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:19 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T14:58:20.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:20.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:20.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:19 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T14:58:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:21.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:20 smithi077 ceph-mon[37327]: pgmap v619: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.2 MiB/s wr, 794 op/s 2024-07-31T14:58:21.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:20 smithi067 ceph-mon[32750]: pgmap v619: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.2 MiB/s wr, 794 op/s 2024-07-31T14:58:21.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:20 smithi073 ceph-mon[36714]: pgmap v619: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.2 MiB/s wr, 794 op/s 2024-07-31T14:58:22.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:22.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:22.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:23.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:22 smithi077 ceph-mon[37327]: pgmap v620: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.9 MiB/s wr, 521 op/s 2024-07-31T14:58:23.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:23.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:23.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:23.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:23.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:22 smithi067 ceph-mon[32750]: pgmap v620: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.9 MiB/s wr, 521 op/s 2024-07-31T14:58:23.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:23.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:23.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:23.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:23.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:22 smithi073 ceph-mon[36714]: pgmap v620: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.9 MiB/s wr, 521 op/s 2024-07-31T14:58:23.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:23.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:23.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:23.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.489+0000 7fd7c65a5640 1 -- 172.21.15.67:0/3774101411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7c0075800 msgr2=0x7fd7c0075c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.489+0000 7fd7c65a5640 1 --2- 172.21.15.67:0/3774101411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7c0075800 0x7fd7c0075c10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009900 tx=0x7fd7b402a3f0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.492+0000 7fd7c65a5640 1 -- 172.21.15.67:0/3774101411 shutdown_connections 2024-07-31T14:58:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.492+0000 7fd7c65a5640 1 --2- 172.21.15.67:0/3774101411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c0113dd0 0x7fd7c01162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.492+0000 7fd7c65a5640 1 --2- 172.21.15.67:0/3774101411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7c0075800 0x7fd7c0075c10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.492+0000 7fd7c65a5640 1 --2- 172.21.15.67:0/3774101411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7c0076dc0 0x7fd7c00751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.492+0000 7fd7c65a5640 1 -- 172.21.15.67:0/3774101411 >> 172.21.15.67:0/3774101411 conn(0x7fd7c00fef10 msgr2=0x7fd7c0101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.497+0000 7fd7c65a5640 1 -- 172.21.15.67:0/3774101411 shutdown_connections 2024-07-31T14:58:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.498+0000 7fd7c65a5640 1 -- 172.21.15.67:0/3774101411 wait complete. 2024-07-31T14:58:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.499+0000 7fd7c65a5640 1 Processor -- start 2024-07-31T14:58:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.499+0000 7fd7c65a5640 1 -- start start 2024-07-31T14:58:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.500+0000 7fd7c65a5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7c0075800 0x7fd7c0074a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.500+0000 7fd7c65a5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c0076dc0 0x7fd7c006dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.500+0000 7fd7c65a5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7c0113dd0 0x7fd7c006e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.500+0000 7fd7c65a5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c0119a60 con 0x7fd7c0075800 2024-07-31T14:58:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.500+0000 7fd7c65a5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01198e0 con 0x7fd7c0113dd0 2024-07-31T14:58:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.500+0000 7fd7c65a5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c0119be0 con 0x7fd7c0076dc0 2024-07-31T14:58:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.500+0000 7fd7bffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7c0075800 0x7fd7c0074a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.501+0000 7fd7bffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7c0075800 0x7fd7c0074a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51356/0 (socket says 172.21.15.67:51356) 2024-07-31T14:58:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.501+0000 7fd7bffff640 1 -- 172.21.15.67:0/261461695 learned_addr learned my addr 172.21.15.67:0/261461695 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.501+0000 7fd7c4b42640 1 --2- 172.21.15.67:0/261461695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7c0113dd0 0x7fd7c006e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.501+0000 7fd7bf7fe640 1 --2- 172.21.15.67:0/261461695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c0076dc0 0x7fd7c006dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.501+0000 7fd7bffff640 1 -- 172.21.15.67:0/261461695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c0076dc0 msgr2=0x7fd7c006dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.501+0000 7fd7bffff640 1 --2- 172.21.15.67:0/261461695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c0076dc0 0x7fd7c006dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.501+0000 7fd7bffff640 1 -- 172.21.15.67:0/261461695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7c0113dd0 msgr2=0x7fd7c006e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.501+0000 7fd7bffff640 1 --2- 172.21.15.67:0/261461695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7c0113dd0 0x7fd7c006e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.501+0000 7fd7bffff640 1 -- 172.21.15.67:0/261461695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c006eaa0 con 0x7fd7c0075800 2024-07-31T14:58:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.502+0000 7fd7bffff640 1 --2- 172.21.15.67:0/261461695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7c0075800 0x7fd7c0074a00 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009d50 tx=0x7fd7b800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.502+0000 7fd7bd7fa640 1 -- 172.21.15.67:0/261461695 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8003b70 con 0x7fd7c0075800 2024-07-31T14:58:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.502+0000 7fd7bd7fa640 1 -- 172.21.15.67:0/261461695 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7b8003d10 con 0x7fd7c0075800 2024-07-31T14:58:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.502+0000 7fd7bd7fa640 1 -- 172.21.15.67:0/261461695 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8003b70 con 0x7fd7c0075800 2024-07-31T14:58:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.502+0000 7fd7c65a5640 1 -- 172.21.15.67:0/261461695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c0073720 con 0x7fd7c0075800 2024-07-31T14:58:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.502+0000 7fd7c65a5640 1 -- 172.21.15.67:0/261461695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c0073bf0 con 0x7fd7c0075800 2024-07-31T14:58:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.506+0000 7fd7cd572740 1 Processor -- start 2024-07-31T14:58:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.508+0000 7fd7cd572740 1 -- start start 2024-07-31T14:58:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.508+0000 7fd7cd572740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4981b0 0x560a1c83ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.508+0000 7fd7cd572740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c46f2f0 0x560a1c83b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.508+0000 7fd7cd572740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c4774d0 0x560a1c83b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.508+0000 7fd7cd572740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560a1c7b03b0 con 0x560a1c46f2f0 2024-07-31T14:58:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.508+0000 7fd7cd572740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560a1c79f8b0 con 0x560a1c4774d0 2024-07-31T14:58:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.508+0000 7fd7cd572740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560a1c7a38f0 con 0x560a1c4981b0 2024-07-31T14:58:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.509+0000 7fd7c4b42640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4981b0 0x560a1c83ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.509+0000 7fd7c4b42640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4981b0 0x560a1c83ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:40074/0 (socket says 172.21.15.67:40074) 2024-07-31T14:58:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.509+0000 7fd7c4b42640 1 -- 172.21.15.67:0/1277207949 learned_addr learned my addr 172.21.15.67:0/1277207949 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.509+0000 7fd7bd7fa640 1 -- 172.21.15.67:0/261461695 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd7b8003ec0 con 0x7fd7c0075800 2024-07-31T14:58:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.510+0000 7fd7bd7fa640 1 --2- 172.21.15.67:0/261461695 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7a80b4140 0x7fd7a80b6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.510+0000 7fd7bf7fe640 1 --2- 172.21.15.67:0/1277207949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c4774d0 0x560a1c83b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.510+0000 7fd7c4b42640 1 -- 172.21.15.67:0/1277207949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c4774d0 msgr2=0x560a1c83b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.510+0000 7fd7c4b42640 1 --2- 172.21.15.67:0/1277207949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c4774d0 0x560a1c83b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.510+0000 7fd7c4b42640 1 -- 172.21.15.67:0/1277207949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c46f2f0 msgr2=0x560a1c83b220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:58:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.510+0000 7fd7c4b42640 1 --2- 172.21.15.67:0/1277207949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c46f2f0 0x560a1c83b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.510+0000 7fd7c4b42640 1 -- 172.21.15.67:0/1277207949 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a1c7469a0 con 0x560a1c4981b0 2024-07-31T14:58:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.510+0000 7fd7bd7fa640 1 -- 172.21.15.67:0/261461695 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7fd7b8003970 con 0x7fd7c0075800 2024-07-31T14:58:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.511+0000 7fd7c4b42640 1 --2- 172.21.15.67:0/1277207949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4981b0 0x560a1c83ace0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009cc0 tx=0x7fd7b0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=104b83e2da519afc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.516+0000 7fd7c4b42640 1 --2- 172.21.15.67:0/261461695 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7a80b4140 0x7fd7a80b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.516+0000 7fd7a67fc640 1 -- 172.21.15.67:0/1277207949 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b001e070 con 0x560a1c4981b0 2024-07-31T14:58:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.516+0000 7fd7a67fc640 1 -- 172.21.15.67:0/1277207949 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7b0019ae0 con 0x560a1c4981b0 2024-07-31T14:58:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.516+0000 7fd7a67fc640 1 -- 172.21.15.67:0/1277207949 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b000ecf0 con 0x560a1c4981b0 2024-07-31T14:58:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.518+0000 7fd7cd572740 1 -- 172.21.15.67:0/1277207949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4981b0 msgr2=0x560a1c83ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.518+0000 7fd7cd572740 1 --2- 172.21.15.67:0/1277207949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4981b0 0x560a1c83ace0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009cc0 tx=0x7fd7b0011810 comp rx=0 tx=0).stop 2024-07-31T14:58:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.521+0000 7fd7cd572740 1 -- 172.21.15.67:0/1277207949 shutdown_connections 2024-07-31T14:58:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.521+0000 7fd7cd572740 1 --2- 172.21.15.67:0/1277207949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c4774d0 0x560a1c83b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.521+0000 7fd7cd572740 1 --2- 172.21.15.67:0/1277207949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c46f2f0 0x560a1c83b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.521+0000 7fd7cd572740 1 --2- 172.21.15.67:0/1277207949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4981b0 0x560a1c83ace0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.521+0000 7fd7cd572740 1 -- 172.21.15.67:0/1277207949 >> 172.21.15.67:0/1277207949 conn(0x560a1c48c300 msgr2=0x560a1c838bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.523+0000 7fd7cd572740 1 -- 172.21.15.67:0/1277207949 shutdown_connections 2024-07-31T14:58:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.524+0000 7fd7c4b42640 1 --2- 172.21.15.67:0/261461695 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7a80b4140 0x7fd7a80b6650 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0013010 tx=0x7fd7b000ee90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.524+0000 7fd7cd572740 1 -- 172.21.15.67:0/1277207949 wait complete. 2024-07-31T14:58:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.525+0000 7fd7cd572740 1 -- start start 2024-07-31T14:58:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.525+0000 7fd7cd572740 1 Processor -- start 2024-07-31T14:58:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.525+0000 7fd7cd572740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c8468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.526+0000 7fd7cd572740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4774d0 0x560a1c846df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.526+0000 7fd7cd572740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4981b0 0x560a1c852750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.526+0000 7fd7bffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c8468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.526+0000 7fd7bffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c8468b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:39678/0 (socket says 172.21.15.67:39678) 2024-07-31T14:58:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.526+0000 7fd7bffff640 1 -- 172.21.15.67:0/1284375960 learned_addr learned my addr 172.21.15.67:0/1284375960 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.526+0000 7fd7cd572740 1 -- 172.21.15.67:0/1284375960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560a1c6d9ab0 con 0x560a1c4981b0 2024-07-31T14:58:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.526+0000 7fd7cd572740 1 -- 172.21.15.67:0/1284375960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560a1c727cf0 con 0x560a1c46f2f0 2024-07-31T14:58:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.526+0000 7fd7cd572740 1 -- 172.21.15.67:0/1284375960 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560a1c849460 con 0x560a1c4774d0 2024-07-31T14:58:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.526+0000 7fd7cd572740 10 client.0 _ll_register_callbacks cb 0x560a1c7999f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.526+0000 7fd7c4b42640 1 --2- 172.21.15.67:0/1284375960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4981b0 0x560a1c852750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.528+0000 7fd7c4b42640 1 -- 172.21.15.67:0/1284375960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4774d0 msgr2=0x560a1c846df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.528+0000 7fd7c4b42640 1 --2- 172.21.15.67:0/1284375960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4774d0 0x560a1c846df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.528+0000 7fd7c4b42640 1 -- 172.21.15.67:0/1284375960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 msgr2=0x560a1c8468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.528+0000 7fd7c4b42640 1 --2- 172.21.15.67:0/1284375960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c8468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.528+0000 7fd7c4b42640 1 -- 172.21.15.67:0/1284375960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a1c503790 con 0x560a1c4981b0 2024-07-31T14:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.529+0000 7fd7c4b42640 1 --2- 172.21.15.67:0/1284375960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4981b0 0x560a1c852750 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd7b001b000 tx=0x7fd7b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5214034ff77f53c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.529+0000 7fd79f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.529+0000 7fd79f7fe640 1 -- 172.21.15.67:0/1284375960 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0035060 con 0x560a1c4981b0 2024-07-31T14:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.529+0000 7fd79f7fe640 1 -- 172.21.15.67:0/1284375960 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7fd7b0018bb0 con 0x560a1c4981b0 2024-07-31T14:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.529+0000 7fd7bffff640 1 --2- 172.21.15.67:0/1284375960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c8468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.529+0000 7fd7cd572740 1 -- 172.21.15.67:0/1284375960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7b0019ae0 con 0x560a1c4981b0 2024-07-31T14:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.529+0000 7fd79f7fe640 1 -- 172.21.15.67:0/1284375960 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd7b003c050 con 0x560a1c4981b0 2024-07-31T14:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.529+0000 7fd7cd572740 10 client.17388 fetch_fsmap learned FSMap version 50 2024-07-31T14:58:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.530+0000 7fd7cd572740 1 -- 172.21.15.67:0/1284375960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x560a1c57c620 con 0x560a1c4981b0 2024-07-31T14:58:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.530+0000 7fd79f7fe640 1 -- 172.21.15.67:0/1284375960 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7b0019ae0 con 0x560a1c4981b0 2024-07-31T14:58:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.530+0000 7fd79f7fe640 1 -- 172.21.15.67:0/1284375960 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7fd7b0005030 con 0x560a1c4981b0 2024-07-31T14:58:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.530+0000 7fd7cd572740 10 client.17388 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:58:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.530+0000 7fd7cd572740 10 client.17388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:58:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.530+0000 7fd7cd572740 20 client.17388 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.530+0000 7fd7cd572740 1 --2- 172.21.15.67:0/1284375960 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a1c456d80 0x560a1c858cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.531+0000 7fd7cd572740 4 client.17388 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:58:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.531+0000 7fd7cd572740 1 -- 172.21.15.67:0/1284375960 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a1c7b2300 con 0x560a1c456d80 2024-07-31T14:58:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.531+0000 7fd7bffff640 1 --2- 172.21.15.67:0/1284375960 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a1c456d80 0x560a1c858cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.534+0000 7fd7bffff640 1 --2- 172.21.15.67:0/1284375960 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a1c456d80 0x560a1c858cd0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dd4a563901a6b92 server_cookie=7aa677bb236554c1 in_seq=0 out_seq=0 2024-07-31T14:58:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.534+0000 7fd79f7fe640 10 client.17388 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.570+0000 7fd79f7fe640 1 -- 172.21.15.67:0/1284375960 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a1c7b2300 con 0x560a1c456d80 2024-07-31T14:58:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.570+0000 7fd79f7fe640 10 client.17388 handle_command_reply: tid=0 2024-07-31T14:58:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.570+0000 7fd7cd572740 1 client.17388 shutdown 2024-07-31T14:58:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.571+0000 7fd7cd572740 1 -- 172.21.15.67:0/1284375960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4981b0 msgr2=0x560a1c852750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.571+0000 7fd7cd572740 1 --2- 172.21.15.67:0/1284375960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4981b0 0x560a1c852750 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd7b001b000 tx=0x7fd7b0019000 comp rx=0 tx=0).stop 2024-07-31T14:58:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.572+0000 7fd7cd572740 1 -- 172.21.15.67:0/1284375960 shutdown_connections 2024-07-31T14:58:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.572+0000 7fd7cd572740 1 --2- 172.21.15.67:0/1284375960 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a1c456d80 0x560a1c858cd0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.572+0000 7fd7cd572740 1 --2- 172.21.15.67:0/1284375960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4981b0 0x560a1c852750 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.572+0000 7fd7cd572740 1 --2- 172.21.15.67:0/1284375960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4774d0 0x560a1c846df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.572+0000 7fd7cd572740 1 --2- 172.21.15.67:0/1284375960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c8468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.572+0000 7fd7cd572740 1 -- 172.21.15.67:0/1284375960 >> 172.21.15.67:0/1284375960 conn(0x560a1c48c300 msgr2=0x560a1c83bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.572+0000 7fd79f7fe640 0 client.17388 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.572+0000 7fd7cd572740 1 -- 172.21.15.67:0/1284375960 shutdown_connections 2024-07-31T14:58:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.573+0000 7fd7cd572740 1 -- 172.21.15.67:0/1284375960 wait complete. 2024-07-31T14:58:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.574+0000 7fd7cd572740 20 client.17388 trim_cache size 0 max 16384 2024-07-31T14:58:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.604+0000 7fd7cd572740 1 Processor -- start 2024-07-31T14:58:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.606+0000 7fd7cd572740 1 -- start start 2024-07-31T14:58:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.606+0000 7fd7cd572740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c858e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.606+0000 7fd7cd572740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4774d0 0x560a1c83bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.607+0000 7fd7cd572740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4981b0 0x560a1c83c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.607+0000 7fd7bffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c858e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.607+0000 7fd7bffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c858e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:39694/0 (socket says 172.21.15.67:39694) 2024-07-31T14:58:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.607+0000 7fd7bffff640 1 -- 172.21.15.67:0/1067042972 learned_addr learned my addr 172.21.15.67:0/1067042972 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.607+0000 7fd7cd572740 1 -- 172.21.15.67:0/1067042972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01bf8e0 con 0x560a1c4774d0 2024-07-31T14:58:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.607+0000 7fd7cd572740 1 -- 172.21.15.67:0/1067042972 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01c1570 con 0x560a1c46f2f0 2024-07-31T14:58:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.607+0000 7fd7cd572740 1 -- 172.21.15.67:0/1067042972 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01c0040 con 0x560a1c4981b0 2024-07-31T14:58:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.607+0000 7fd7c4b42640 1 --2- 172.21.15.67:0/1067042972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4981b0 0x560a1c83c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.608+0000 7fd7bffff640 1 -- 172.21.15.67:0/1067042972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4981b0 msgr2=0x560a1c83c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.608+0000 7fd7bffff640 1 --2- 172.21.15.67:0/1067042972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4981b0 0x560a1c83c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.608+0000 7fd7bffff640 1 -- 172.21.15.67:0/1067042972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4774d0 msgr2=0x560a1c83bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.608+0000 7fd7bffff640 1 --2- 172.21.15.67:0/1067042972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4774d0 0x560a1c83bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.608+0000 7fd7bffff640 1 -- 172.21.15.67:0/1067042972 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a1c846420 con 0x560a1c46f2f0 2024-07-31T14:58:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.608+0000 7fd7bffff640 1 --2- 172.21.15.67:0/1067042972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c858e10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8000f80 tx=0x7fd7b80af520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f243bcf53149c875 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.609+0000 7fd7a5ffb640 1 -- 172.21.15.67:0/1067042972 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80a7520 con 0x560a1c46f2f0 2024-07-31T14:58:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.609+0000 7fd7a5ffb640 1 -- 172.21.15.67:0/1067042972 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7b8009820 con 0x560a1c46f2f0 2024-07-31T14:58:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.609+0000 7fd7a5ffb640 1 -- 172.21.15.67:0/1067042972 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80a76c0 con 0x560a1c46f2f0 2024-07-31T14:58:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.610+0000 7fd7cd572740 1 -- 172.21.15.67:0/1067042972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 msgr2=0x560a1c858e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.610+0000 7fd7cd572740 1 --2- 172.21.15.67:0/1067042972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c858e10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8000f80 tx=0x7fd7b80af520 comp rx=0 tx=0).stop 2024-07-31T14:58:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.612+0000 7fd7cd572740 1 -- 172.21.15.67:0/1067042972 shutdown_connections 2024-07-31T14:58:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.612+0000 7fd7cd572740 1 --2- 172.21.15.67:0/1067042972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4981b0 0x560a1c83c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.612+0000 7fd7cd572740 1 --2- 172.21.15.67:0/1067042972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4774d0 0x560a1c83bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.612+0000 7fd7cd572740 1 --2- 172.21.15.67:0/1067042972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c858e10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.612+0000 7fd7cd572740 1 -- 172.21.15.67:0/1067042972 >> 172.21.15.67:0/1067042972 conn(0x560a1c48c300 msgr2=0x560a1c844f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.615+0000 7fd7cd572740 1 -- 172.21.15.67:0/1067042972 shutdown_connections 2024-07-31T14:58:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.616+0000 7fd7cd572740 1 -- 172.21.15.67:0/1067042972 wait complete. 2024-07-31T14:58:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.617+0000 7fd7cd572740 1 -- start start 2024-07-31T14:58:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.617+0000 7fd7cd572740 1 Processor -- start 2024-07-31T14:58:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.618+0000 7fd7cd572740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c84a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.618+0000 7fd7cd572740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4774d0 0x560a1c84aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.618+0000 7fd7cd572740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4981b0 0x560a1c84afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.618+0000 7fd7cd572740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01c0ea0 con 0x560a1c4981b0 2024-07-31T14:58:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.618+0000 7fd7cd572740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01c0710 con 0x560a1c46f2f0 2024-07-31T14:58:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.618+0000 7fd7cd572740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01bd8e0 con 0x560a1c4774d0 2024-07-31T14:58:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.618+0000 7fd7cd572740 10 client.0 _ll_register_callbacks cb 0x7fd7b001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.619+0000 7fd7c4b42640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4981b0 0x560a1c84afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.619+0000 7fd7c4b42640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4981b0 0x560a1c84afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51390/0 (socket says 172.21.15.67:51390) 2024-07-31T14:58:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.619+0000 7fd7c4b42640 1 -- 172.21.15.67:0/2309685906 learned_addr learned my addr 172.21.15.67:0/2309685906 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.619+0000 7fd7bffff640 1 --2- 172.21.15.67:0/2309685906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c84a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.619+0000 7fd7c4b42640 1 -- 172.21.15.67:0/2309685906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4774d0 msgr2=0x560a1c84aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.619+0000 7fd7c4b42640 1 --2- 172.21.15.67:0/2309685906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4774d0 0x560a1c84aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.619+0000 7fd7c4b42640 1 -- 172.21.15.67:0/2309685906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 msgr2=0x560a1c84a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.619+0000 7fd7c4b42640 1 --2- 172.21.15.67:0/2309685906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c84a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.619+0000 7fd7c4b42640 1 -- 172.21.15.67:0/2309685906 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a1c8be490 con 0x560a1c4981b0 2024-07-31T14:58:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.620+0000 7fd7c4b42640 1 --2- 172.21.15.67:0/2309685906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4981b0 0x560a1c84afa0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0018fd0 tx=0x7fd7b00099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c236a2241c6a89 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.620+0000 7fd79f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:58:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.620+0000 7fd79f7fe640 1 -- 172.21.15.67:0/2309685906 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0035050 con 0x560a1c4981b0 2024-07-31T14:58:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.620+0000 7fd7cd572740 1 -- 172.21.15.67:0/2309685906 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7b8009820 con 0x560a1c4981b0 2024-07-31T14:58:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.620+0000 7fd79f7fe640 1 -- 172.21.15.67:0/2309685906 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7fd7b00055f0 con 0x560a1c4981b0 2024-07-31T14:58:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.621+0000 7fd79f7fe640 1 -- 172.21.15.67:0/2309685906 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd7b004e050 con 0x560a1c4981b0 2024-07-31T14:58:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.624+0000 7fd7cd572740 10 client.17391 fetch_fsmap learned FSMap version 50 2024-07-31T14:58:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.624+0000 7fd7cd572740 1 -- 172.21.15.67:0/2309685906 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x560a1c8bf6d0 con 0x560a1c4981b0 2024-07-31T14:58:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.625+0000 7fd79f7fe640 1 -- 172.21.15.67:0/2309685906 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7b8009820 con 0x560a1c4981b0 2024-07-31T14:58:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.625+0000 7fd79f7fe640 1 -- 172.21.15.67:0/2309685906 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7fd7b00035f0 con 0x560a1c4981b0 2024-07-31T14:58:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.625+0000 7fd7cd572740 10 client.17391 fetch_fsmap finished waiting for FSMap version 50 2024-07-31T14:58:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.625+0000 7fd7cd572740 10 client.17391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:58:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.625+0000 7fd7cd572740 20 client.17391 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.625+0000 7fd7cd572740 1 --2- 172.21.15.67:0/2309685906 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a1c42c1c0 0x560a1c8c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.625+0000 7fd7cd572740 4 client.17391 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:58:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.625+0000 7fd7cd572740 1 -- 172.21.15.67:0/2309685906 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a1c7adda0 con 0x560a1c42c1c0 2024-07-31T14:58:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.626+0000 7fd7bffff640 1 --2- 172.21.15.67:0/2309685906 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a1c42c1c0 0x560a1c8c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.627+0000 7fd7bffff640 1 --2- 172.21.15.67:0/2309685906 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a1c42c1c0 0x560a1c8c1e90 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcc3bf463b7a334a server_cookie=acd7b3a45749d989 in_seq=0 out_seq=0 2024-07-31T14:58:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:23.627+0000 7fd79f7fe640 10 client.17391 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:24.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:24.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:24.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:24.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:24.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:24.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:24.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:24.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:24.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:24.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:24.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:24.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:24.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:24.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:24.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:24.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:24.643 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 14:58:24 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T14:58:24.211+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000162a [...7,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch/ auth{1=1,2=1,3=1,4=1} v2970 ap=3 RANDEPHEMERALPIN f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22) n(v19 rc2024-07-31T14:54:53.561226+0000 b106958628 17855=16920+935) old_inodes=3 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:49075 nref=3)) | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=0 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7c26ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000162a:[<0x10000000202/arch v378>,<0x10000000201/linux-6.5.11 v96>,<0x10000000006/t v68>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v57>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v60>,<0x10000000001/sv_1 v84>,<0x10000000000/qa v70>,<0x1/volumes v70>]//[]","memoryvalue":"(2)0x1000000162a:[<0x10000000202/arch v2970>,<0x10000000201/linux-6.5.11 v621>,<0x10000000006/t v456>,<0x10000000005/tmp v424>,<0x100000001fb/client.0 v412>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v397>,<0x10000000001/sv_1 v417>,<0x10000000000/qa v403>,<0x1/volumes v394>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:54:49.652308+0000 23=2+21)","ondisk_value.rstat":"n(v0 rc2024-07-31T14:54:53.561226+0000 b106958628 17855=16920+935)","memory_value.dirstat":"f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22)","memory_value.rstat":"n(v19 rc2024-07-31T14:54:53.561226+0000 b106958628 17855=16920+935)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T14:58:25.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:24 smithi077 ceph-mon[37327]: pgmap v621: 129 pgs: 17 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 67 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 2.5 MiB/s wr, 417 op/s 2024-07-31T14:58:25.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:24 smithi067 ceph-mon[32750]: pgmap v621: 129 pgs: 17 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 67 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 2.5 MiB/s wr, 417 op/s 2024-07-31T14:58:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.344+0000 7fd79f7fe640 1 -- 172.21.15.67:0/2309685906 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a1c7adda0 con 0x560a1c42c1c0 2024-07-31T14:58:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.344+0000 7fd79f7fe640 10 client.17391 handle_command_reply: tid=0 2024-07-31T14:58:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.344+0000 7fd7cd572740 1 client.17391 shutdown 2024-07-31T14:58:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.345+0000 7fd7cd572740 1 -- 172.21.15.67:0/2309685906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4981b0 msgr2=0x560a1c84afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.345+0000 7fd7cd572740 1 --2- 172.21.15.67:0/2309685906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4981b0 0x560a1c84afa0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0018fd0 tx=0x7fd7b00099d0 comp rx=0 tx=0).stop 2024-07-31T14:58:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.345+0000 7fd7cd572740 1 -- 172.21.15.67:0/2309685906 shutdown_connections 2024-07-31T14:58:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.345+0000 7fd7cd572740 1 --2- 172.21.15.67:0/2309685906 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560a1c42c1c0 0x560a1c8c1e90 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.345+0000 7fd7cd572740 1 --2- 172.21.15.67:0/2309685906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a1c4981b0 0x560a1c84afa0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.345+0000 7fd7cd572740 1 --2- 172.21.15.67:0/2309685906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a1c4774d0 0x560a1c84aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.345+0000 7fd7cd572740 1 --2- 172.21.15.67:0/2309685906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a1c46f2f0 0x560a1c84a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.345+0000 7fd7cd572740 1 -- 172.21.15.67:0/2309685906 >> 172.21.15.67:0/2309685906 conn(0x560a1c48c300 msgr2=0x560a1c844e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.345+0000 7fd79f7fe640 0 client.17391 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.345+0000 7fd7cd572740 1 -- 172.21.15.67:0/2309685906 shutdown_connections 2024-07-31T14:58:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.345+0000 7fd7cd572740 1 -- 172.21.15.67:0/2309685906 wait complete. 2024-07-31T14:58:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.345+0000 7fd7cd572740 20 client.17391 trim_cache size 0 max 16384 2024-07-31T14:58:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:58:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:58:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:58:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:58:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.346+0000 7fd7c65a5640 1 -- 172.21.15.67:0/261461695 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7a80b4140 msgr2=0x7fd7a80b6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.346+0000 7fd7c65a5640 1 --2- 172.21.15.67:0/261461695 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7a80b4140 0x7fd7a80b6650 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0013010 tx=0x7fd7b000ee90 comp rx=0 tx=0).stop 2024-07-31T14:58:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.346+0000 7fd7c65a5640 1 -- 172.21.15.67:0/261461695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7c0075800 msgr2=0x7fd7c0074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.346+0000 7fd7c65a5640 1 --2- 172.21.15.67:0/261461695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7c0075800 0x7fd7c0074a00 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009d50 tx=0x7fd7b800c6a0 comp rx=0 tx=0).stop 2024-07-31T14:58:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.346+0000 7fd7c65a5640 1 -- 172.21.15.67:0/261461695 shutdown_connections 2024-07-31T14:58:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.346+0000 7fd7c65a5640 1 --2- 172.21.15.67:0/261461695 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7a80b4140 0x7fd7a80b6650 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.346+0000 7fd7c65a5640 1 --2- 172.21.15.67:0/261461695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7c0113dd0 0x7fd7c006e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.346+0000 7fd7c65a5640 1 --2- 172.21.15.67:0/261461695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c0076dc0 0x7fd7c006dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.346+0000 7fd7c65a5640 1 --2- 172.21.15.67:0/261461695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7c0075800 0x7fd7c0074a00 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.346+0000 7fd7c65a5640 1 -- 172.21.15.67:0/261461695 >> 172.21.15.67:0/261461695 conn(0x7fd7c00fef10 msgr2=0x7fd7c010fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.347+0000 7fd7c65a5640 1 -- 172.21.15.67:0/261461695 shutdown_connections 2024-07-31T14:58:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:25.347+0000 7fd7c65a5640 1 -- 172.21.15.67:0/261461695 wait complete. 2024-07-31T14:58:25.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:24 smithi073 ceph-mon[36714]: pgmap v621: 129 pgs: 17 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 67 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 2.5 MiB/s wr, 417 op/s 2024-07-31T14:58:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:25.727 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:58:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:58:26.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:26 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T14:58:26.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:26 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:58:26.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:26.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:26.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:26.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:26.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:26.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:26.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:26.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:26 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T14:58:26.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:26 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:58:26.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:26.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:26.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:26.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:26.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:26.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:26.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:26.503 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:26 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T14:58:26.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:26 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:58:26.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:26.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:26.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:26.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:26.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:26.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:26.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: pgmap v622: 129 pgs: 15 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 82 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 3.9 MiB/s wr, 582 op/s 2024-07-31T14:58:27.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:27.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:27.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:27.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:27.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:27.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:27.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:27.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:27.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: pgmap v622: 129 pgs: 15 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 82 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 3.9 MiB/s wr, 582 op/s 2024-07-31T14:58:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:27.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:27.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:27.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: pgmap v622: 129 pgs: 15 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 82 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 3.9 MiB/s wr, 582 op/s 2024-07-31T14:58:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:28.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.262+0000 7f5025663640 1 -- 172.21.15.67:0/980605652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502010a080 msgr2=0x7f502010a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:28.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.263+0000 7f5025663640 1 --2- 172.21.15.67:0/980605652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502010a080 0x7f502010a490 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f50100074e0 tx=0x7f501002b0a0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.269+0000 7f5025663640 1 -- 172.21.15.67:0/980605652 shutdown_connections 2024-07-31T14:58:28.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.269+0000 7f5025663640 1 --2- 172.21.15.67:0/980605652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f502010d050 0x7f5020116ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.269+0000 7f5025663640 1 --2- 172.21.15.67:0/980605652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f502010c700 0x7f502010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.269+0000 7f5025663640 1 --2- 172.21.15.67:0/980605652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502010a080 0x7f502010a490 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.269+0000 7f5025663640 1 -- 172.21.15.67:0/980605652 >> 172.21.15.67:0/980605652 conn(0x7f50200708f0 msgr2=0x7f5020070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:28.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.270+0000 7f5025663640 1 -- 172.21.15.67:0/980605652 shutdown_connections 2024-07-31T14:58:28.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.270+0000 7f5025663640 1 -- 172.21.15.67:0/980605652 wait complete. 2024-07-31T14:58:28.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.271+0000 7f5025663640 1 Processor -- start 2024-07-31T14:58:28.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.271+0000 7f5025663640 1 -- start start 2024-07-31T14:58:28.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.272+0000 7f5025663640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f502010a080 0x7f50201a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.272+0000 7f5025663640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502010c700 0x7f50201a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.272+0000 7f5025663640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f502010d050 0x7f50201a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.272+0000 7f5025663640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5020119c30 con 0x7f502010c700 2024-07-31T14:58:28.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.272+0000 7f501effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f502010a080 0x7f50201a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.272+0000 7f5025663640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5020119ab0 con 0x7f502010a080 2024-07-31T14:58:28.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.272+0000 7f501effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f502010a080 0x7f50201a04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:39728/0 (socket says 172.21.15.67:39728) 2024-07-31T14:58:28.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.272+0000 7f501effd640 1 -- 172.21.15.67:0/3172753188 learned_addr learned my addr 172.21.15.67:0/3172753188 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:28.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.273+0000 7f5025663640 1 -- 172.21.15.67:0/3172753188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5020119db0 con 0x7f502010d050 2024-07-31T14:58:28.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.273+0000 7f501f7fe640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f502010d050 0x7f50201a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.274+0000 7f5017fff640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502010c700 0x7f50201a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.274+0000 7f501effd640 1 -- 172.21.15.67:0/3172753188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f502010d050 msgr2=0x7f50201a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:28.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.274+0000 7f501effd640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f502010d050 0x7f50201a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.274+0000 7f501effd640 1 -- 172.21.15.67:0/3172753188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502010c700 msgr2=0x7f50201a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:28.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.274+0000 7f501effd640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502010c700 0x7f50201a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.274+0000 7f501effd640 1 -- 172.21.15.67:0/3172753188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50201a6430 con 0x7f502010a080 2024-07-31T14:58:28.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.274+0000 7f5017fff640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502010c700 0x7f50201a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:58:28.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.282+0000 7f501effd640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f502010a080 0x7f50201a04c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f50100074b0 tx=0x7f501000a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:28.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.286+0000 7f501cff9640 1 -- 172.21.15.67:0/3172753188 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010041020 con 0x7f502010a080 2024-07-31T14:58:28.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.286+0000 7f5025663640 1 -- 172.21.15.67:0/3172753188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50201a66c0 con 0x7f502010a080 2024-07-31T14:58:28.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.286+0000 7f5025663640 1 -- 172.21.15.67:0/3172753188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50201ace80 con 0x7f502010a080 2024-07-31T14:58:28.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.286+0000 7f501cff9640 1 -- 172.21.15.67:0/3172753188 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f501002be20 con 0x7f502010a080 2024-07-31T14:58:28.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.286+0000 7f501cff9640 1 -- 172.21.15.67:0/3172753188 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010004920 con 0x7f502010a080 2024-07-31T14:58:28.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.291+0000 7f502bbb4740 1 Processor -- start 2024-07-31T14:58:28.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.294+0000 7f502bbb4740 1 -- start start 2024-07-31T14:58:28.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.295+0000 7f502bbb4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b41355b070 0x55b413900640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.295+0000 7f502bbb4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4135321b0 0x55b413900b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.295+0000 7f502bbb4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41353a390 0x55b4139010c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.295+0000 7f502bbb4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b413876b60 con 0x55b41353a390 2024-07-31T14:58:28.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.295+0000 7f502bbb4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b413866060 con 0x55b4135321b0 2024-07-31T14:58:28.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.295+0000 7f502bbb4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b41386a0a0 con 0x55b41355b070 2024-07-31T14:58:28.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.295+0000 7f5017fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41353a390 0x55b4139010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.295+0000 7f5017fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41353a390 0x55b4139010c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51406/0 (socket says 172.21.15.67:51406) 2024-07-31T14:58:28.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.295+0000 7f5017fff640 1 -- 172.21.15.67:0/886248481 learned_addr learned my addr 172.21.15.67:0/886248481 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:28.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.295+0000 7f501effd640 1 --2- 172.21.15.67:0/886248481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4135321b0 0x55b413900b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.296+0000 7f5017fff640 1 -- 172.21.15.67:0/886248481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b41355b070 msgr2=0x55b413900640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:28.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.296+0000 7f5017fff640 1 --2- 172.21.15.67:0/886248481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b41355b070 0x55b413900640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.296+0000 7f5017fff640 1 -- 172.21.15.67:0/886248481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4135321b0 msgr2=0x55b413900b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:28.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.296+0000 7f5017fff640 1 --2- 172.21.15.67:0/886248481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4135321b0 0x55b413900b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.296+0000 7f5017fff640 1 -- 172.21.15.67:0/886248481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b41380cb40 con 0x55b41353a390 2024-07-31T14:58:28.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.297+0000 7f5017fff640 1 --2- 172.21.15.67:0/886248481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41353a390 0x55b4139010c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f500800a940 tx=0x7f500800ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba17dee201f6df67 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:28.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.299+0000 7f501cff9640 1 -- 172.21.15.67:0/3172753188 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f50100035f0 con 0x7f502010a080 2024-07-31T14:58:28.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.300+0000 7f501cff9640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f500c0b3f10 0x7f500c0b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.301+0000 7f501cff9640 1 -- 172.21.15.67:0/3172753188 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f50101041d0 con 0x7f502010a080 2024-07-31T14:58:28.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.301+0000 7f501f7fe640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f500c0b3f10 0x7f500c0b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.302+0000 7f50157fa640 1 -- 172.21.15.67:0/886248481 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500800cc00 con 0x55b41353a390 2024-07-31T14:58:28.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.302+0000 7f50157fa640 1 -- 172.21.15.67:0/886248481 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5008037070 con 0x55b41353a390 2024-07-31T14:58:28.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.303+0000 7f502bbb4740 1 -- 172.21.15.67:0/886248481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41353a390 msgr2=0x55b4139010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:28.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.303+0000 7f502bbb4740 1 --2- 172.21.15.67:0/886248481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41353a390 0x55b4139010c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f500800a940 tx=0x7f500800ae00 comp rx=0 tx=0).stop 2024-07-31T14:58:28.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.303+0000 7f502bbb4740 1 -- 172.21.15.67:0/886248481 shutdown_connections 2024-07-31T14:58:28.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.303+0000 7f502bbb4740 1 --2- 172.21.15.67:0/886248481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41353a390 0x55b4139010c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.303+0000 7f502bbb4740 1 --2- 172.21.15.67:0/886248481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4135321b0 0x55b413900b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.303+0000 7f502bbb4740 1 --2- 172.21.15.67:0/886248481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b41355b070 0x55b413900640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.303+0000 7f502bbb4740 1 -- 172.21.15.67:0/886248481 >> 172.21.15.67:0/886248481 conn(0x55b41354f460 msgr2=0x55b4138fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:28.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.304+0000 7f501f7fe640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f500c0b3f10 0x7f500c0b6420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f501800d250 tx=0x7f501800d690 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:28.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.304+0000 7f502bbb4740 1 -- 172.21.15.67:0/886248481 shutdown_connections 2024-07-31T14:58:28.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.304+0000 7f502bbb4740 1 -- 172.21.15.67:0/886248481 wait complete. 2024-07-31T14:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.304+0000 7f502bbb4740 1 -- start start 2024-07-31T14:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.304+0000 7f502bbb4740 1 Processor -- start 2024-07-31T14:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.305+0000 7f502bbb4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4135321b0 0x55b41390c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.305+0000 7f502bbb4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b41353a390 0x55b41390ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.305+0000 7f502bbb4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b41355b070 0x55b413911e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.305+0000 7f502bbb4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b4136c9700 con 0x55b4135321b0 2024-07-31T14:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.305+0000 7f502bbb4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b413808990 con 0x55b41353a390 2024-07-31T14:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.305+0000 7f502bbb4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b413906d40 con 0x55b41355b070 2024-07-31T14:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.305+0000 7f502bbb4740 10 client.0 _ll_register_callbacks cb 0x55b4138234a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.306+0000 7f501f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b41355b070 0x55b413911e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.306+0000 7f501effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4135321b0 0x55b41390c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.306+0000 7f501effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4135321b0 0x55b41390c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51414/0 (socket says 172.21.15.67:51414) 2024-07-31T14:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.306+0000 7f501effd640 1 -- 172.21.15.67:0/3903195128 learned_addr learned my addr 172.21.15.67:0/3903195128 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.306+0000 7f5017fff640 1 --2- 172.21.15.67:0/3903195128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b41353a390 0x55b41390ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.307+0000 7f501effd640 1 -- 172.21.15.67:0/3903195128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b41355b070 msgr2=0x55b413911e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.307+0000 7f501effd640 1 --2- 172.21.15.67:0/3903195128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b41355b070 0x55b413911e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.307+0000 7f501effd640 1 -- 172.21.15.67:0/3903195128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b41353a390 msgr2=0x55b41390ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.307+0000 7f501effd640 1 --2- 172.21.15.67:0/3903195128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b41353a390 0x55b41390ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.307+0000 7f501effd640 1 -- 172.21.15.67:0/3903195128 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4135c6650 con 0x55b4135321b0 2024-07-31T14:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.308+0000 7f501effd640 1 --2- 172.21.15.67:0/3903195128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4135321b0 0x55b41390c830 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f501002bfd0 tx=0x7f5010045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299e9187ac4f15a8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.308+0000 7f50067fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.309+0000 7f50067fc640 1 -- 172.21.15.67:0/3903195128 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50100c6a20 con 0x55b4135321b0 2024-07-31T14:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.309+0000 7f50067fc640 1 -- 172.21.15.67:0/3903195128 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f50100d15a0 con 0x55b4135321b0 2024-07-31T14:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.309+0000 7f502bbb4740 1 -- 172.21.15.67:0/3903195128 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5008037070 con 0x55b4135321b0 2024-07-31T14:58:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.310+0000 7f50067fc640 1 -- 172.21.15.67:0/3903195128 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f50100e1070 con 0x55b4135321b0 2024-07-31T14:58:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.311+0000 7f502bbb4740 10 client.17400 fetch_fsmap learned FSMap version 51 2024-07-31T14:58:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.311+0000 7f502bbb4740 1 -- 172.21.15.67:0/3903195128 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55b41391ad80 con 0x55b4135321b0 2024-07-31T14:58:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.319+0000 7f50067fc640 1 -- 172.21.15.67:0/3903195128 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5008037070 con 0x55b4135321b0 2024-07-31T14:58:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.319+0000 7f50067fc640 1 -- 172.21.15.67:0/3903195128 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7f50100d1070 con 0x55b4135321b0 2024-07-31T14:58:28.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.319+0000 7f502bbb4740 10 client.17400 fetch_fsmap finished waiting for FSMap version 51 2024-07-31T14:58:28.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.319+0000 7f502bbb4740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:58:28.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.319+0000 7f502bbb4740 20 client.17400 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:28.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.319+0000 7f502bbb4740 1 --2- 172.21.15.67:0/3903195128 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4135199f0 0x55b41391e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.319+0000 7f502bbb4740 4 client.17400 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:58:28.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.319+0000 7f502bbb4740 1 -- 172.21.15.67:0/3903195128 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b413878ab0 con 0x55b4135199f0 2024-07-31T14:58:28.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.322+0000 7f5017fff640 1 --2- 172.21.15.67:0/3903195128 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4135199f0 0x55b41391e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.333+0000 7f5017fff640 1 --2- 172.21.15.67:0/3903195128 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4135199f0 0x55b41391e280 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab05698ab4951e26 server_cookie=8f99cc593608fb91 in_seq=0 out_seq=0 2024-07-31T14:58:28.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.334+0000 7f50067fc640 10 client.17400 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.338+0000 7f50067fc640 1 -- 172.21.15.67:0/3903195128 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b413878ab0 con 0x55b4135199f0 2024-07-31T14:58:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.338+0000 7f50067fc640 10 client.17400 handle_command_reply: tid=0 2024-07-31T14:58:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.338+0000 7f502bbb4740 1 client.17400 shutdown 2024-07-31T14:58:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.339+0000 7f502bbb4740 1 -- 172.21.15.67:0/3903195128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4135321b0 msgr2=0x55b41390c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.339+0000 7f502bbb4740 1 --2- 172.21.15.67:0/3903195128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4135321b0 0x55b41390c830 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f501002bfd0 tx=0x7f5010045000 comp rx=0 tx=0).stop 2024-07-31T14:58:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.343+0000 7f502bbb4740 1 -- 172.21.15.67:0/3903195128 shutdown_connections 2024-07-31T14:58:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.343+0000 7f502bbb4740 1 --2- 172.21.15.67:0/3903195128 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4135199f0 0x55b41391e280 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.343+0000 7f502bbb4740 1 --2- 172.21.15.67:0/3903195128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b41355b070 0x55b413911e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.343+0000 7f502bbb4740 1 --2- 172.21.15.67:0/3903195128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b41353a390 0x55b41390ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.343+0000 7f502bbb4740 1 --2- 172.21.15.67:0/3903195128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4135321b0 0x55b41390c830 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.343+0000 7f502bbb4740 1 -- 172.21.15.67:0/3903195128 >> 172.21.15.67:0/3903195128 conn(0x55b41354f460 msgr2=0x55b4138fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.344+0000 7f50067fc640 0 client.17400 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:28.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.349+0000 7f502bbb4740 1 -- 172.21.15.67:0/3903195128 shutdown_connections 2024-07-31T14:58:28.351 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:28.351 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:28.351 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:28.351 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:28.351 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:28.352 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:28.352 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:28.352 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:28.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.351+0000 7f502bbb4740 1 -- 172.21.15.67:0/3903195128 wait complete. 2024-07-31T14:58:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.355+0000 7f502bbb4740 20 client.17400 trim_cache size 0 max 16384 2024-07-31T14:58:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.390+0000 7f502bbb4740 1 Processor -- start 2024-07-31T14:58:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.390+0000 7f502bbb4740 1 -- start start 2024-07-31T14:58:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.391+0000 7f502bbb4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4135321b0 0x55b41390db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.391+0000 7f502bbb4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41353a390 0x55b4138fdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.391+0000 7f502bbb4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b41355b070 0x55b4138fe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.391+0000 7f502bbb4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f500000fff0 con 0x55b41353a390 2024-07-31T14:58:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.391+0000 7f502bbb4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5000011c80 con 0x55b41355b070 2024-07-31T14:58:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.391+0000 7f502bbb4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5000010750 con 0x55b4135321b0 2024-07-31T14:58:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.392+0000 7f501f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b41355b070 0x55b4138fe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.392+0000 7f501f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b41355b070 0x55b4138fe200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:39758/0 (socket says 172.21.15.67:39758) 2024-07-31T14:58:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.392+0000 7f501f7fe640 1 -- 172.21.15.67:0/3803866387 learned_addr learned my addr 172.21.15.67:0/3803866387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.392+0000 7f5017fff640 1 --2- 172.21.15.67:0/3803866387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41353a390 0x55b4138fdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.393+0000 7f5017fff640 1 -- 172.21.15.67:0/3803866387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4135321b0 msgr2=0x55b41390db50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:58:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.393+0000 7f5017fff640 1 --2- 172.21.15.67:0/3803866387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4135321b0 0x55b41390db50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.393+0000 7f5017fff640 1 -- 172.21.15.67:0/3803866387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b41355b070 msgr2=0x55b4138fe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.393+0000 7f5017fff640 1 --2- 172.21.15.67:0/3803866387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b41355b070 0x55b4138fe200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.393+0000 7f5017fff640 1 -- 172.21.15.67:0/3803866387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b41390ea00 con 0x55b41353a390 2024-07-31T14:58:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.393+0000 7f5017fff640 1 --2- 172.21.15.67:0/3803866387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41353a390 0x55b4138fdcc0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5008011ce0 tx=0x7f5008006c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb26ae2cc47cf1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.395+0000 7f5014ff9640 1 -- 172.21.15.67:0/3803866387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500800ca80 con 0x55b41353a390 2024-07-31T14:58:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.395+0000 7f5014ff9640 1 -- 172.21.15.67:0/3803866387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5008037070 con 0x55b41353a390 2024-07-31T14:58:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.395+0000 7f5014ff9640 1 -- 172.21.15.67:0/3803866387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008011650 con 0x55b41353a390 2024-07-31T14:58:28.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.396+0000 7f502bbb4740 1 -- 172.21.15.67:0/3803866387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41353a390 msgr2=0x55b4138fdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:28.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.396+0000 7f502bbb4740 1 --2- 172.21.15.67:0/3803866387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41353a390 0x55b4138fdcc0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5008011ce0 tx=0x7f5008006c40 comp rx=0 tx=0).stop 2024-07-31T14:58:28.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.397+0000 7f502bbb4740 1 -- 172.21.15.67:0/3803866387 shutdown_connections 2024-07-31T14:58:28.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.397+0000 7f502bbb4740 1 --2- 172.21.15.67:0/3803866387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b41355b070 0x55b4138fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.397+0000 7f502bbb4740 1 --2- 172.21.15.67:0/3803866387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41353a390 0x55b4138fdcc0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.397+0000 7f502bbb4740 1 --2- 172.21.15.67:0/3803866387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4135321b0 0x55b41390db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.397+0000 7f502bbb4740 1 -- 172.21.15.67:0/3803866387 >> 172.21.15.67:0/3803866387 conn(0x55b41354f460 msgr2=0x55b413902030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:28.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.399+0000 7f502bbb4740 1 -- 172.21.15.67:0/3803866387 shutdown_connections 2024-07-31T14:58:28.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.399+0000 7f502bbb4740 1 -- 172.21.15.67:0/3803866387 wait complete. 2024-07-31T14:58:28.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.400+0000 7f502bbb4740 1 -- start start 2024-07-31T14:58:28.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.400+0000 7f502bbb4740 1 Processor -- start 2024-07-31T14:58:28.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.400+0000 7f502bbb4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4135321b0 0x55b4139182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.400+0000 7f502bbb4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b41353a390 0x55b413913790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.400+0000 7f502bbb4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41355b070 0x55b413913cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.400+0000 7f502bbb4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f50000115b0 con 0x55b41355b070 2024-07-31T14:58:28.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.400+0000 7f502bbb4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5000010e20 con 0x55b4135321b0 2024-07-31T14:58:28.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.400+0000 7f502bbb4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f500000dff0 con 0x55b41353a390 2024-07-31T14:58:28.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.400+0000 7f502bbb4740 10 client.0 _ll_register_callbacks cb 0x7f50101019b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:28.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.402+0000 7f501effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4135321b0 0x55b4139182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.402+0000 7f501effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4135321b0 0x55b4139182e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:39774/0 (socket says 172.21.15.67:39774) 2024-07-31T14:58:28.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.402+0000 7f501effd640 1 -- 172.21.15.67:0/1865579371 learned_addr learned my addr 172.21.15.67:0/1865579371 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:28.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.402+0000 7f501f7fe640 1 --2- 172.21.15.67:0/1865579371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41355b070 0x55b413913cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.402+0000 7f5017fff640 1 --2- 172.21.15.67:0/1865579371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b41353a390 0x55b413913790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.402+0000 7f501effd640 1 -- 172.21.15.67:0/1865579371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b41353a390 msgr2=0x55b413913790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:28.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.402+0000 7f501effd640 1 --2- 172.21.15.67:0/1865579371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b41353a390 0x55b413913790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.402+0000 7f501effd640 1 -- 172.21.15.67:0/1865579371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41355b070 msgr2=0x55b413913cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:28.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.402+0000 7f501effd640 1 --2- 172.21.15.67:0/1865579371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41355b070 0x55b413913cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.402+0000 7f501effd640 1 -- 172.21.15.67:0/1865579371 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b413914210 con 0x55b4135321b0 2024-07-31T14:58:28.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.403+0000 7f501effd640 1 --2- 172.21.15.67:0/1865579371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4135321b0 0x55b4139182e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5010008b30 tx=0x7f50101044e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec1fda048dc91724 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:28.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.411+0000 7f50067fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:58:28.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.411+0000 7f50067fc640 1 -- 172.21.15.67:0/1865579371 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50100038e0 con 0x55b4135321b0 2024-07-31T14:58:28.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.411+0000 7f50067fc640 1 -- 172.21.15.67:0/1865579371 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f5010107870 con 0x55b4135321b0 2024-07-31T14:58:28.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.413+0000 7f502bbb4740 1 -- 172.21.15.67:0/1865579371 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5008037070 con 0x55b4135321b0 2024-07-31T14:58:28.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.413+0000 7f50067fc640 1 -- 172.21.15.67:0/1865579371 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f50100ca2d0 con 0x55b4135321b0 2024-07-31T14:58:28.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.417+0000 7f502bbb4740 10 client.26969 fetch_fsmap learned FSMap version 51 2024-07-31T14:58:28.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.417+0000 7f502bbb4740 1 -- 172.21.15.67:0/1865579371 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55b413914210 con 0x55b4135321b0 2024-07-31T14:58:28.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.422+0000 7f50067fc640 1 -- 172.21.15.67:0/1865579371 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5008037070 con 0x55b4135321b0 2024-07-31T14:58:28.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.422+0000 7f50067fc640 1 -- 172.21.15.67:0/1865579371 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7f50100d1050 con 0x55b4135321b0 2024-07-31T14:58:28.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.423+0000 7f502bbb4740 10 client.26969 fetch_fsmap finished waiting for FSMap version 51 2024-07-31T14:58:28.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.423+0000 7f502bbb4740 10 client.26969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:58:28.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.423+0000 7f502bbb4740 20 client.26969 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:28.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.423+0000 7f502bbb4740 1 --2- 172.21.15.67:0/1865579371 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4134eeda0 0x55b413988b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:28.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.423+0000 7f502bbb4740 4 client.26969 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:58:28.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.423+0000 7f502bbb4740 1 -- 172.21.15.67:0/1865579371 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b413874550 con 0x55b4134eeda0 2024-07-31T14:58:28.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.423+0000 7f5017fff640 1 --2- 172.21.15.67:0/1865579371 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4134eeda0 0x55b413988b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:28.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.425+0000 7f5017fff640 1 --2- 172.21.15.67:0/1865579371 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4134eeda0 0x55b413988b90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1043150f072b15ab server_cookie=e8afaedc2946e068 in_seq=0 out_seq=0 2024-07-31T14:58:28.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.427+0000 7f50067fc640 10 client.26969 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:28.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.429+0000 7f50067fc640 1 -- 172.21.15.67:0/1865579371 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55b413874550 con 0x55b4134eeda0 2024-07-31T14:58:28.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.429+0000 7f50067fc640 10 client.26969 handle_command_reply: tid=0 2024-07-31T14:58:28.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.429+0000 7f502bbb4740 1 client.26969 shutdown 2024-07-31T14:58:28.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.433+0000 7f502bbb4740 1 -- 172.21.15.67:0/1865579371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4135321b0 msgr2=0x55b4139182e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:28.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.433+0000 7f502bbb4740 1 --2- 172.21.15.67:0/1865579371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4135321b0 0x55b4139182e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5010008b30 tx=0x7f50101044e0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.433+0000 7f502bbb4740 1 -- 172.21.15.67:0/1865579371 shutdown_connections 2024-07-31T14:58:28.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.433+0000 7f502bbb4740 1 --2- 172.21.15.67:0/1865579371 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4134eeda0 0x55b413988b90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.433+0000 7f502bbb4740 1 --2- 172.21.15.67:0/1865579371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b41355b070 0x55b413913cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.433+0000 7f502bbb4740 1 --2- 172.21.15.67:0/1865579371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b41353a390 0x55b413913790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.433+0000 7f502bbb4740 1 --2- 172.21.15.67:0/1865579371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4135321b0 0x55b4139182e0 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5010008b30 tx=0x7f50101044e0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.433+0000 7f50067fc640 0 client.26969 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:28.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.433+0000 7f502bbb4740 1 -- 172.21.15.67:0/1865579371 >> 172.21.15.67:0/1865579371 conn(0x55b41354f460 msgr2=0x55b413902030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:28.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.433+0000 7f502bbb4740 1 -- 172.21.15.67:0/1865579371 shutdown_connections 2024-07-31T14:58:28.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.433+0000 7f502bbb4740 1 -- 172.21.15.67:0/1865579371 wait complete. 2024-07-31T14:58:28.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.433+0000 7f502bbb4740 20 client.26969 trim_cache size 0 max 16384 2024-07-31T14:58:28.436 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:58:28.436 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (4906 inodes in the stack)", 2024-07-31T14:58:28.436 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T14:58:28.436 INFO:teuthology.orchestra.run.smithi067.stdout: "d7bf5c48-7b0c-4bad-a7d0-07bd7c41685f": { 2024-07-31T14:58:28.436 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T14:58:28.436 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "d7bf5c48-7b0c-4bad-a7d0-07bd7c41685f", 2024-07-31T14:58:28.436 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T14:58:28.436 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:58:28.436 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:58:28.436 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:58:28.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.437+0000 7f5025663640 1 -- 172.21.15.67:0/3172753188 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f500c0b3f10 msgr2=0x7f500c0b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:28.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.437+0000 7f5025663640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f500c0b3f10 0x7f500c0b6420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f501800d250 tx=0x7f501800d690 comp rx=0 tx=0).stop 2024-07-31T14:58:28.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.437+0000 7f5025663640 1 -- 172.21.15.67:0/3172753188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f502010a080 msgr2=0x7f50201a04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:28.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.437+0000 7f5025663640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f502010a080 0x7f50201a04c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f50100074b0 tx=0x7f501000a6f0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.437+0000 7f5025663640 1 -- 172.21.15.67:0/3172753188 shutdown_connections 2024-07-31T14:58:28.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.437+0000 7f5025663640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f500c0b3f10 0x7f500c0b6420 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.437+0000 7f5025663640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f502010d050 0x7f50201a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.437+0000 7f5025663640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502010c700 0x7f50201a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.437+0000 7f5025663640 1 --2- 172.21.15.67:0/3172753188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f502010a080 0x7f50201a04c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:28.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.437+0000 7f5025663640 1 -- 172.21.15.67:0/3172753188 >> 172.21.15.67:0/3172753188 conn(0x7f50200708f0 msgr2=0x7f5020071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:28.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.437+0000 7f5025663640 1 -- 172.21.15.67:0/3172753188 shutdown_connections 2024-07-31T14:58:28.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:28.437+0000 7f5025663640 1 -- 172.21.15.67:0/3172753188 wait complete. 2024-07-31T14:58:28.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:28.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:28.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:28.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:28.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:28.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:28.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:28.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:28.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:29.343 INFO:tasks.cephfs.filesystem:scrub status for tag:d7bf5c48-7b0c-4bad-a7d0-07bd7c41685f - {'path': '/', 'tag': 'd7bf5c48-7b0c-4bad-a7d0-07bd7c41685f', 'options': 'recursive,force'} 2024-07-31T14:58:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: pgmap v623: 129 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 93 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 3.5 MiB/s wr, 628 op/s 2024-07-31T14:58:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:29.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:29.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:29.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:29.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:29.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:29.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:29.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:29.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:29.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:29.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: pgmap v623: 129 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 93 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 3.5 MiB/s wr, 628 op/s 2024-07-31T14:58:29.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:29.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:29.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:29.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:29.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:29.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:29.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:29.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:29.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:29.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:29.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:29.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: pgmap v623: 129 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 93 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 3.5 MiB/s wr, 628 op/s 2024-07-31T14:58:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:30.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:30.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:30.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:30.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:30.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:30.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:30.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:30.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:30.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:30 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:58:30.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:30 smithi077 ceph-mon[37327]: pgmap v624: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 3.5 MiB/s wr, 669 op/s 2024-07-31T14:58:30.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:30.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:30.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:30.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:30.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:30.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:30.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:30.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:30.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:30 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:58:30.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:30 smithi073 ceph-mon[36714]: pgmap v624: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 3.5 MiB/s wr, 669 op/s 2024-07-31T14:58:30.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:30.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:30.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:30.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:30.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:30.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:30.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:30.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:30.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:30 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:58:30.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:30 smithi067 ceph-mon[32750]: pgmap v624: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 3.5 MiB/s wr, 669 op/s 2024-07-31T14:58:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:31.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:31.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:31.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:31.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:31.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:32.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:32.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:32.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:32.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:32 smithi077 ceph-mon[37327]: pgmap v625: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.3 MiB/s wr, 854 op/s 2024-07-31T14:58:32.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:32.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:32.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:32.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:32 smithi067 ceph-mon[32750]: pgmap v625: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.3 MiB/s wr, 854 op/s 2024-07-31T14:58:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:32 smithi073 ceph-mon[36714]: pgmap v625: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.3 MiB/s wr, 854 op/s 2024-07-31T14:58:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:33.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:33.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:33.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:33.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:33.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:33.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:33.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:33.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:33.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:33.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:33.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:33.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:34.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:34.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:34.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:34.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:34 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:58:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:34 smithi077 ceph-mon[37327]: pgmap v626: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 904 op/s 2024-07-31T14:58:34.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:34.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:34.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:34.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:34.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:34 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:58:34.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:34.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:34.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:34 smithi067 ceph-mon[32750]: pgmap v626: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 904 op/s 2024-07-31T14:58:34.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:34 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:58:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:34 smithi073 ceph-mon[36714]: pgmap v626: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 904 op/s 2024-07-31T14:58:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:35.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:35.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:35.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:35.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:35.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:35.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:36.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:36.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:36.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:36 smithi077 ceph-mon[37327]: pgmap v627: 129 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 103 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 1.03k op/s 2024-07-31T14:58:36.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:36.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:36.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:36 smithi067 ceph-mon[32750]: pgmap v627: 129 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 103 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 1.03k op/s 2024-07-31T14:58:36.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:36.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:36.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:36 smithi073 ceph-mon[36714]: pgmap v627: 129 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 103 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 1.03k op/s 2024-07-31T14:58:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:37.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:37.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:37.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:37.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:37.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:37.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:37.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:37.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:37.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:37.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:37.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:37.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.470+0000 7fae06cec640 1 -- 172.21.15.67:0/1834543436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae001097c0 msgr2=0x7fae0010fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.470+0000 7fae06cec640 1 --2- 172.21.15.67:0/1834543436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae001097c0 0x7fae0010fff0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009ca0 tx=0x7fadf802a8c0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.470+0000 7fae06cec640 1 -- 172.21.15.67:0/1834543436 shutdown_connections 2024-07-31T14:58:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.470+0000 7fae06cec640 1 --2- 172.21.15.67:0/1834543436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae001097c0 0x7fae0010fff0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.470+0000 7fae06cec640 1 --2- 172.21.15.67:0/1834543436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae00108e70 0x7fae00109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.470+0000 7fae06cec640 1 --2- 172.21.15.67:0/1834543436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae00103540 0x7fae00103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.470+0000 7fae06cec640 1 -- 172.21.15.67:0/1834543436 >> 172.21.15.67:0/1834543436 conn(0x7fae000fefa0 msgr2=0x7fae001013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.470+0000 7fae06cec640 1 -- 172.21.15.67:0/1834543436 shutdown_connections 2024-07-31T14:58:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.470+0000 7fae06cec640 1 -- 172.21.15.67:0/1834543436 wait complete. 2024-07-31T14:58:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.471+0000 7fae06cec640 1 Processor -- start 2024-07-31T14:58:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.471+0000 7fae06cec640 1 -- start start 2024-07-31T14:58:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.471+0000 7fae06cec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae00103540 0x7fae00199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.471+0000 7fae06cec640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae00108e70 0x7fae0019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.471+0000 7fae06cec640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae001097c0 0x7fae0019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.471+0000 7fae06cec640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fae00113470 con 0x7fae00108e70 2024-07-31T14:58:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.471+0000 7fae06cec640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fae001132f0 con 0x7fae001097c0 2024-07-31T14:58:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.471+0000 7fae06cec640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae001135f0 con 0x7fae00103540 2024-07-31T14:58:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.471+0000 7fadfffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae00108e70 0x7fae0019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.471+0000 7fadfffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae00108e70 0x7fae0019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60800/0 (socket says 172.21.15.67:60800) 2024-07-31T14:58:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.471+0000 7fadfffff640 1 -- 172.21.15.67:0/2709129465 learned_addr learned my addr 172.21.15.67:0/2709129465 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.472+0000 7fae05289640 1 --2- 172.21.15.67:0/2709129465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae001097c0 0x7fae0019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.472+0000 7fae04a88640 1 --2- 172.21.15.67:0/2709129465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae00103540 0x7fae00199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.472+0000 7fadfffff640 1 -- 172.21.15.67:0/2709129465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae00103540 msgr2=0x7fae00199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.472+0000 7fadfffff640 1 --2- 172.21.15.67:0/2709129465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae00103540 0x7fae00199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.472+0000 7fadfffff640 1 -- 172.21.15.67:0/2709129465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae001097c0 msgr2=0x7fae0019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.472+0000 7fadfffff640 1 --2- 172.21.15.67:0/2709129465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae001097c0 0x7fae0019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.472+0000 7fadfffff640 1 -- 172.21.15.67:0/2709129465 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0019fcf0 con 0x7fae00108e70 2024-07-31T14:58:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.472+0000 7fadfffff640 1 --2- 172.21.15.67:0/2709129465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae00108e70 0x7fae0019a360 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fadec00e9f0 tx=0x7fadec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.472+0000 7fadfdffb640 1 -- 172.21.15.67:0/2709129465 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec00ccc0 con 0x7fae00108e70 2024-07-31T14:58:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.472+0000 7fae06cec640 1 -- 172.21.15.67:0/2709129465 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0019ffe0 con 0x7fae00108e70 2024-07-31T14:58:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.472+0000 7fadfdffb640 1 -- 172.21.15.67:0/2709129465 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadec0108e0 con 0x7fae00108e70 2024-07-31T14:58:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.472+0000 7fadfdffb640 1 -- 172.21.15.67:0/2709129465 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec018860 con 0x7fae00108e70 2024-07-31T14:58:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.473+0000 7fae06cec640 1 -- 172.21.15.67:0/2709129465 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae001a6770 con 0x7fae00108e70 2024-07-31T14:58:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.476+0000 7fae0dbe3740 1 Processor -- start 2024-07-31T14:58:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.476+0000 7fae0dbe3740 1 -- start start 2024-07-31T14:58:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.476+0000 7fae0dbe3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc551b0 0x55a05dff7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.476+0000 7fae0dbe3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05dff8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.476+0000 7fae0dbe3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc344d0 0x55a05dff8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.476+0000 7fae0dbe3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a05df6d3b0 con 0x55a05dc2c2f0 2024-07-31T14:58:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.476+0000 7fae0dbe3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a05df5c8b0 con 0x55a05dc551b0 2024-07-31T14:58:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.476+0000 7fae0dbe3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a05df608f0 con 0x55a05dc344d0 2024-07-31T14:58:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.476+0000 7fadfdffb640 1 -- 172.21.15.67:0/2709129465 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fadec010430 con 0x7fae00108e70 2024-07-31T14:58:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.476+0000 7fae05289640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc551b0 0x55a05dff7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.476+0000 7fadfffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc344d0 0x55a05dff8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.476+0000 7fae04a88640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05dff8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.476+0000 7fae05289640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc551b0 0x55a05dff7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44228/0 (socket says 172.21.15.67:44228) 2024-07-31T14:58:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.476+0000 7fae05289640 1 -- 172.21.15.67:0/2605656175 learned_addr learned my addr 172.21.15.67:0/2605656175 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.477+0000 7fae05289640 1 -- 172.21.15.67:0/2605656175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc344d0 msgr2=0x55a05dff8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.477+0000 7fae05289640 1 --2- 172.21.15.67:0/2605656175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc344d0 0x55a05dff8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.477+0000 7fae05289640 1 -- 172.21.15.67:0/2605656175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 msgr2=0x55a05dff8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.477+0000 7fae05289640 1 --2- 172.21.15.67:0/2605656175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05dff8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.477+0000 7fae05289640 1 -- 172.21.15.67:0/2605656175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05df039a0 con 0x55a05dc551b0 2024-07-31T14:58:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.477+0000 7fadfffff640 1 --2- 172.21.15.67:0/2605656175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc344d0 0x55a05dff8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:58:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.477+0000 7fae04a88640 1 --2- 172.21.15.67:0/2605656175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05dff8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:58:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.477+0000 7fae05289640 1 --2- 172.21.15.67:0/2605656175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc551b0 0x55a05dff7ce0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fadf802f410 tx=0x7fadf802f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=379425f0603dc554 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.478+0000 7fadeaffd640 1 -- 172.21.15.67:0/2605656175 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8031cf0 con 0x55a05dc551b0 2024-07-31T14:58:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.478+0000 7fadfdffb640 1 --2- 172.21.15.67:0/2709129465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fadf00b3d40 0x7fadf00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.478+0000 7fadeaffd640 1 -- 172.21.15.67:0/2605656175 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadf8036030 con 0x55a05dc551b0 2024-07-31T14:58:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.478+0000 7fadeaffd640 1 -- 172.21.15.67:0/2605656175 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf80042a0 con 0x55a05dc551b0 2024-07-31T14:58:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.478+0000 7fae04a88640 1 --2- 172.21.15.67:0/2709129465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fadf00b3d40 0x7fadf00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.478+0000 7fadfdffb640 1 -- 172.21.15.67:0/2709129465 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7fadec0e4a50 con 0x7fae00108e70 2024-07-31T14:58:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.479+0000 7fae0dbe3740 1 -- 172.21.15.67:0/2605656175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc551b0 msgr2=0x55a05dff7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.479+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/2605656175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc551b0 0x55a05dff7ce0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fadf802f410 tx=0x7fadf802f440 comp rx=0 tx=0).stop 2024-07-31T14:58:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.479+0000 7fae04a88640 1 --2- 172.21.15.67:0/2709129465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fadf00b3d40 0x7fadf00b6250 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55a05dff6960 tx=0x7fadf4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.479+0000 7fae0dbe3740 1 -- 172.21.15.67:0/2605656175 shutdown_connections 2024-07-31T14:58:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.479+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/2605656175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc344d0 0x55a05dff8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.479+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/2605656175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05dff8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.479+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/2605656175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc551b0 0x55a05dff7ce0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.479+0000 7fae0dbe3740 1 -- 172.21.15.67:0/2605656175 >> 172.21.15.67:0/2605656175 conn(0x55a05dc49300 msgr2=0x55a05dff5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.479+0000 7fae0dbe3740 1 -- 172.21.15.67:0/2605656175 shutdown_connections 2024-07-31T14:58:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.479+0000 7fae0dbe3740 1 -- 172.21.15.67:0/2605656175 wait complete. 2024-07-31T14:58:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.480+0000 7fae0dbe3740 1 -- start start 2024-07-31T14:58:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.480+0000 7fae0dbe3740 1 Processor -- start 2024-07-31T14:58:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.480+0000 7fae0dbe3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc2c2f0 0x55a05e007bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.480+0000 7fae0dbe3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc344d0 0x55a05e0080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.480+0000 7fae0dbe3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc551b0 0x55a05e0139f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.480+0000 7fae0dbe3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a05de96ab0 con 0x55a05dc551b0 2024-07-31T14:58:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.480+0000 7fae0dbe3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a05dee4cf0 con 0x55a05dc344d0 2024-07-31T14:58:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.480+0000 7fae0dbe3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a05e00a660 con 0x55a05dc2c2f0 2024-07-31T14:58:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.480+0000 7fae0dbe3740 10 client.0 _ll_register_callbacks cb 0x55a05df569f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.480+0000 7fae04a88640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc2c2f0 0x55a05e007bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.481+0000 7fae04a88640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc2c2f0 0x55a05e007bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:52176/0 (socket says 172.21.15.67:52176) 2024-07-31T14:58:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.481+0000 7fae04a88640 1 -- 172.21.15.67:0/2424814503 learned_addr learned my addr 172.21.15.67:0/2424814503 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.481+0000 7fadfffff640 1 --2- 172.21.15.67:0/2424814503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc344d0 0x55a05e0080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.481+0000 7fae05289640 1 --2- 172.21.15.67:0/2424814503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc551b0 0x55a05e0139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.481+0000 7fae04a88640 1 -- 172.21.15.67:0/2424814503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc344d0 msgr2=0x55a05e0080f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.481+0000 7fae04a88640 1 --2- 172.21.15.67:0/2424814503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc344d0 0x55a05e0080f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.481+0000 7fae04a88640 1 -- 172.21.15.67:0/2424814503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc551b0 msgr2=0x55a05e0139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.481+0000 7fae04a88640 1 --2- 172.21.15.67:0/2424814503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc551b0 0x55a05e0139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.481+0000 7fae04a88640 1 -- 172.21.15.67:0/2424814503 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a05dcc0790 con 0x55a05dc2c2f0 2024-07-31T14:58:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.481+0000 7fadfffff640 1 --2- 172.21.15.67:0/2424814503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc344d0 0x55a05e0080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:58:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.481+0000 7fae04a88640 1 --2- 172.21.15.67:0/2424814503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc2c2f0 0x55a05e007bb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fadf4020a10 tx=0x7fadf4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1f9752f1915b055 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.481+0000 7faddffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:58:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.481+0000 7faddffff640 1 -- 172.21.15.67:0/2424814503 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf402d060 con 0x55a05dc2c2f0 2024-07-31T14:58:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.482+0000 7fae0dbe3740 1 -- 172.21.15.67:0/2424814503 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadf8036030 con 0x55a05dc2c2f0 2024-07-31T14:58:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.482+0000 7faddffff640 1 -- 172.21.15.67:0/2424814503 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7fadf4018030 con 0x55a05dc2c2f0 2024-07-31T14:58:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.482+0000 7faddffff640 1 -- 172.21.15.67:0/2424814503 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fadf4034050 con 0x55a05dc2c2f0 2024-07-31T14:58:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.482+0000 7fae0dbe3740 10 client.26992 fetch_fsmap learned FSMap version 53 2024-07-31T14:58:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.482+0000 7fae0dbe3740 1 -- 172.21.15.67:0/2424814503 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55a05dd39620 con 0x55a05dc2c2f0 2024-07-31T14:58:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.482+0000 7faddffff640 1 -- 172.21.15.67:0/2424814503 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadf8036030 con 0x55a05dc2c2f0 2024-07-31T14:58:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.482+0000 7faddffff640 1 -- 172.21.15.67:0/2424814503 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 53) ==== 8390+0+0 (secure 0 0 0) 0x7fadf401d920 con 0x55a05dc2c2f0 2024-07-31T14:58:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.482+0000 7fae0dbe3740 10 client.26992 fetch_fsmap finished waiting for FSMap version 53 2024-07-31T14:58:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.482+0000 7fae0dbe3740 10 client.26992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:58:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.482+0000 7fae0dbe3740 20 client.26992 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.483+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/2424814503 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a05dc13d80 0x55a05e019a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.483+0000 7fae0dbe3740 4 client.26992 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:58:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.483+0000 7fae0dbe3740 1 -- 172.21.15.67:0/2424814503 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a05df6f300 con 0x55a05dc13d80 2024-07-31T14:58:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.483+0000 7fadfffff640 1 --2- 172.21.15.67:0/2424814503 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a05dc13d80 0x55a05e019a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.483+0000 7faddffff640 10 client.26992 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.483+0000 7fadfffff640 1 --2- 172.21.15.67:0/2424814503 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a05dc13d80 0x55a05e019a90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8841dfef98a4a10a server_cookie=96ee7b1fa9f5ac8 in_seq=0 out_seq=0 2024-07-31T14:58:38.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:38.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:38.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:38.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:38.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:38.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:38.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:38 smithi077 ceph-mon[37327]: pgmap v628: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 935 op/s 2024-07-31T14:58:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.485+0000 7faddffff640 1 -- 172.21.15.67:0/2424814503 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a05df6f300 con 0x55a05dc13d80 2024-07-31T14:58:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.485+0000 7faddffff640 10 client.26992 handle_command_reply: tid=0 2024-07-31T14:58:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.485+0000 7fae0dbe3740 1 client.26992 shutdown 2024-07-31T14:58:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.485+0000 7fae0dbe3740 1 -- 172.21.15.67:0/2424814503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc2c2f0 msgr2=0x55a05e007bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.485+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/2424814503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc2c2f0 0x55a05e007bb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fadf4020a10 tx=0x7fadf4004730 comp rx=0 tx=0).stop 2024-07-31T14:58:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.485+0000 7fae0dbe3740 1 -- 172.21.15.67:0/2424814503 shutdown_connections 2024-07-31T14:58:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.485+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/2424814503 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a05dc13d80 0x55a05e019a90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.485+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/2424814503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc551b0 0x55a05e0139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.485+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/2424814503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc344d0 0x55a05e0080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.485+0000 7faddffff640 0 client.26992 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.485+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/2424814503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc2c2f0 0x55a05e007bb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.485+0000 7fae0dbe3740 1 -- 172.21.15.67:0/2424814503 >> 172.21.15.67:0/2424814503 conn(0x55a05dc49300 msgr2=0x55a05dff42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.485+0000 7fae0dbe3740 1 -- 172.21.15.67:0/2424814503 shutdown_connections 2024-07-31T14:58:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.485+0000 7fae0dbe3740 1 -- 172.21.15.67:0/2424814503 wait complete. 2024-07-31T14:58:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.486+0000 7fae0dbe3740 20 client.26992 trim_cache size 0 max 16384 2024-07-31T14:58:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.503+0000 7fae0dbe3740 1 Processor -- start 2024-07-31T14:58:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.503+0000 7fae0dbe3740 1 -- start start 2024-07-31T14:58:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.503+0000 7fae0dbe3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05e009880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.503+0000 7fae0dbe3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc344d0 0x55a05dff9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.503+0000 7fae0dbe3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc551b0 0x55a05e0137e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fae0dbe3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fae001ae5b0 con 0x55a05dc2c2f0 2024-07-31T14:58:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fae0dbe3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fae001b01e0 con 0x55a05dc344d0 2024-07-31T14:58:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fae0dbe3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae001aecb0 con 0x55a05dc551b0 2024-07-31T14:58:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fadfffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc344d0 0x55a05dff9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fae05289640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc551b0 0x55a05e0137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fae05289640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc551b0 0x55a05e0137e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:52188/0 (socket says 172.21.15.67:52188) 2024-07-31T14:58:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fadfffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc344d0 0x55a05dff9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44236/0 (socket says 172.21.15.67:44236) 2024-07-31T14:58:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fae05289640 1 -- 172.21.15.67:0/300288019 learned_addr learned my addr 172.21.15.67:0/300288019 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fae04a88640 1 --2- 172.21.15.67:0/300288019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05e009880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fae05289640 1 -- 172.21.15.67:0/300288019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc344d0 msgr2=0x55a05dff9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fae05289640 1 --2- 172.21.15.67:0/300288019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc344d0 0x55a05dff9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fae05289640 1 -- 172.21.15.67:0/300288019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 msgr2=0x55a05e009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fae05289640 1 --2- 172.21.15.67:0/300288019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05e009880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fae05289640 1 -- 172.21.15.67:0/300288019 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05dfefd60 con 0x55a05dc551b0 2024-07-31T14:58:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fadfffff640 1 --2- 172.21.15.67:0/300288019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc344d0 0x55a05dff9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:58:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.504+0000 7fae05289640 1 --2- 172.21.15.67:0/300288019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc551b0 0x55a05e0137e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fadf802f4d0 tx=0x7fadf8003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c59a542917b0aa08 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.505+0000 7fadea7fc640 1 -- 172.21.15.67:0/300288019 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8055020 con 0x55a05dc551b0 2024-07-31T14:58:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.505+0000 7fadea7fc640 1 -- 172.21.15.67:0/300288019 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadf8036040 con 0x55a05dc551b0 2024-07-31T14:58:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.505+0000 7fadea7fc640 1 -- 172.21.15.67:0/300288019 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf804a040 con 0x55a05dc551b0 2024-07-31T14:58:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.506+0000 7fae0dbe3740 1 -- 172.21.15.67:0/300288019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc551b0 msgr2=0x55a05e0137e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.506+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/300288019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc551b0 0x55a05e0137e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fadf802f4d0 tx=0x7fadf8003c20 comp rx=0 tx=0).stop 2024-07-31T14:58:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.506+0000 7fae0dbe3740 1 -- 172.21.15.67:0/300288019 shutdown_connections 2024-07-31T14:58:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.506+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/300288019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc551b0 0x55a05e0137e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.506+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/300288019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc344d0 0x55a05dff9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.506+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/300288019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05e009880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.506+0000 7fae0dbe3740 1 -- 172.21.15.67:0/300288019 >> 172.21.15.67:0/300288019 conn(0x55a05dc49300 msgr2=0x55a05dff8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.506+0000 7fae0dbe3740 1 -- 172.21.15.67:0/300288019 shutdown_connections 2024-07-31T14:58:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.506+0000 7fae0dbe3740 1 -- 172.21.15.67:0/300288019 wait complete. 2024-07-31T14:58:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.506+0000 7fae0dbe3740 1 -- start start 2024-07-31T14:58:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.506+0000 7fae0dbe3740 1 Processor -- start 2024-07-31T14:58:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fae0dbe3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05e00fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fae0dbe3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc344d0 0x55a05e010280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fae0dbe3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc551b0 0x55a05e00caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fae0dbe3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fae001afb10 con 0x55a05dc2c2f0 2024-07-31T14:58:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fae0dbe3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fae001af380 con 0x55a05dc551b0 2024-07-31T14:58:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fae0dbe3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae001ac5b0 con 0x55a05dc344d0 2024-07-31T14:58:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fae04a88640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05e00fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fadfffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc344d0 0x55a05e010280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fae04a88640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05e00fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60822/0 (socket says 172.21.15.67:60822) 2024-07-31T14:58:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fae04a88640 1 -- 172.21.15.67:0/3536404778 learned_addr learned my addr 172.21.15.67:0/3536404778 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fadfffff640 1 --2- 172.21.15.67:0/3536404778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc344d0 0x55a05e010280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:52202/0 (socket says 172.21.15.67:52202) 2024-07-31T14:58:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fae0dbe3740 10 client.0 _ll_register_callbacks cb 0x7fadf4005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fae05289640 1 --2- 172.21.15.67:0/3536404778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc551b0 0x55a05e00caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fadfffff640 1 -- 172.21.15.67:0/3536404778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc551b0 msgr2=0x55a05e00caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fadfffff640 1 --2- 172.21.15.67:0/3536404778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc551b0 0x55a05e00caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fadfffff640 1 -- 172.21.15.67:0/3536404778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 msgr2=0x55a05e00fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fadfffff640 1 --2- 172.21.15.67:0/3536404778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05e00fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fadfffff640 1 -- 172.21.15.67:0/3536404778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a05e00d280 con 0x55a05dc344d0 2024-07-31T14:58:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.507+0000 7fae04a88640 1 --2- 172.21.15.67:0/3536404778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05e00fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:58:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.508+0000 7fadfffff640 1 --2- 172.21.15.67:0/3536404778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc344d0 0x55a05e010280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fadec0e1d80 tx=0x7fadec00cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa3885be40caf75e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.508+0000 7faddffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:58:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.508+0000 7faddffff640 1 -- 172.21.15.67:0/3536404778 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec0ab240 con 0x55a05dc344d0 2024-07-31T14:58:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.508+0000 7faddffff640 1 -- 172.21.15.67:0/3536404778 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7fadec0e8030 con 0x55a05dc344d0 2024-07-31T14:58:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.508+0000 7fae0dbe3740 1 -- 172.21.15.67:0/3536404778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadf8036040 con 0x55a05dc344d0 2024-07-31T14:58:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.508+0000 7faddffff640 1 -- 172.21.15.67:0/3536404778 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fadec0af840 con 0x55a05dc344d0 2024-07-31T14:58:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.508+0000 7fae0dbe3740 10 client.26998 fetch_fsmap learned FSMap version 53 2024-07-31T14:58:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.508+0000 7fae0dbe3740 1 -- 172.21.15.67:0/3536404778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55a05e080e90 con 0x55a05dc344d0 2024-07-31T14:58:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.508+0000 7faddffff640 1 -- 172.21.15.67:0/3536404778 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadf8036040 con 0x55a05dc344d0 2024-07-31T14:58:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.509+0000 7faddffff640 1 -- 172.21.15.67:0/3536404778 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 53) ==== 8390+0+0 (secure 0 0 0) 0x7fadec0a6bb0 con 0x55a05dc344d0 2024-07-31T14:58:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.509+0000 7fae0dbe3740 10 client.26998 fetch_fsmap finished waiting for FSMap version 53 2024-07-31T14:58:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.509+0000 7fae0dbe3740 10 client.26998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:58:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.509+0000 7fae0dbe3740 20 client.26998 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.509+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/3536404778 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a05dbe91c0 0x55a05e083710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.509+0000 7fae0dbe3740 4 client.26998 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:58:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.509+0000 7fae0dbe3740 1 -- 172.21.15.67:0/3536404778 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a05df6ada0 con 0x55a05dbe91c0 2024-07-31T14:58:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.509+0000 7fae04a88640 1 --2- 172.21.15.67:0/3536404778 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a05dbe91c0 0x55a05e083710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.510+0000 7faddffff640 10 client.26998 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:38.510+0000 7fae04a88640 1 --2- 172.21.15.67:0/3536404778 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a05dbe91c0 0x55a05e083710 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1cc561cf52f443 server_cookie=9a688b5ba18ac686 in_seq=0 out_seq=0 2024-07-31T14:58:38.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:38.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:38.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:38 smithi067 ceph-mon[32750]: pgmap v628: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 935 op/s 2024-07-31T14:58:38.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:38 smithi073 ceph-mon[36714]: pgmap v628: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 935 op/s 2024-07-31T14:58:39.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:39.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:39.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:39.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:39.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:39.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.143+0000 7faddffff640 1 -- 172.21.15.67:0/3536404778 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a05df6ada0 con 0x55a05dbe91c0 2024-07-31T14:58:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.143+0000 7faddffff640 10 client.26998 handle_command_reply: tid=0 2024-07-31T14:58:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.143+0000 7fae0dbe3740 1 client.26998 shutdown 2024-07-31T14:58:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.143+0000 7fae0dbe3740 1 -- 172.21.15.67:0/3536404778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc344d0 msgr2=0x55a05e010280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.143+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/3536404778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc344d0 0x55a05e010280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fadec0e1d80 tx=0x7fadec00cef0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.143+0000 7fae0dbe3740 1 -- 172.21.15.67:0/3536404778 shutdown_connections 2024-07-31T14:58:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.143+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/3536404778 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a05dbe91c0 0x55a05e083710 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.143+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/3536404778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a05dc551b0 0x55a05e00caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.143+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/3536404778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a05dc344d0 0x55a05e010280 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.143+0000 7faddffff640 0 client.26998 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.143+0000 7fae0dbe3740 1 --2- 172.21.15.67:0/3536404778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a05dc2c2f0 0x55a05e00fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.143+0000 7fae0dbe3740 1 -- 172.21.15.67:0/3536404778 >> 172.21.15.67:0/3536404778 conn(0x55a05dc49300 msgr2=0x55a05dff94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.143+0000 7fae0dbe3740 1 -- 172.21.15.67:0/3536404778 shutdown_connections 2024-07-31T14:58:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.143+0000 7fae0dbe3740 1 -- 172.21.15.67:0/3536404778 wait complete. 2024-07-31T14:58:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.144+0000 7fae0dbe3740 20 client.26998 trim_cache size 0 max 16384 2024-07-31T14:58:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:58:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:58:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:58:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:58:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.144+0000 7fae06cec640 1 -- 172.21.15.67:0/2709129465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fadf00b3d40 msgr2=0x7fadf00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.144+0000 7fae06cec640 1 --2- 172.21.15.67:0/2709129465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fadf00b3d40 0x7fadf00b6250 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55a05dff6960 tx=0x7fadf4005f90 comp rx=0 tx=0).stop 2024-07-31T14:58:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.144+0000 7fae06cec640 1 -- 172.21.15.67:0/2709129465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae00108e70 msgr2=0x7fae0019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.144+0000 7fae06cec640 1 --2- 172.21.15.67:0/2709129465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae00108e70 0x7fae0019a360 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fadec00e9f0 tx=0x7fadec00eeb0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.144+0000 7fae06cec640 1 -- 172.21.15.67:0/2709129465 shutdown_connections 2024-07-31T14:58:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.144+0000 7fae06cec640 1 --2- 172.21.15.67:0/2709129465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fadf00b3d40 0x7fadf00b6250 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.144+0000 7fae06cec640 1 --2- 172.21.15.67:0/2709129465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae001097c0 0x7fae0019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.144+0000 7fae06cec640 1 --2- 172.21.15.67:0/2709129465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae00108e70 0x7fae0019a360 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.144+0000 7fae06cec640 1 --2- 172.21.15.67:0/2709129465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae00103540 0x7fae00199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.144+0000 7fae06cec640 1 -- 172.21.15.67:0/2709129465 >> 172.21.15.67:0/2709129465 conn(0x7fae000fefa0 msgr2=0x7fae00108460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.145+0000 7fae06cec640 1 -- 172.21.15.67:0/2709129465 shutdown_connections 2024-07-31T14:58:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:40.145+0000 7fae06cec640 1 -- 172.21.15.67:0/2709129465 wait complete. 2024-07-31T14:58:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:40 smithi067 ceph-mon[32750]: pgmap v629: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 887 op/s 2024-07-31T14:58:40.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:40.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:40 smithi073 ceph-mon[36714]: pgmap v629: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 887 op/s 2024-07-31T14:58:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.646+0000 7f68e57fa640 1 -- 172.21.15.67:0/52498627 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b83bdfd8e0 con 0x55b83bf146a0 2024-07-31T14:58:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.646+0000 7f68e57fa640 10 client.26912 handle_command_reply: tid=0 2024-07-31T14:58:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.646+0000 7f690a6d4740 1 client.26912 shutdown 2024-07-31T14:58:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.646+0000 7f690a6d4740 1 -- 172.21.15.67:0/52498627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83bac6b60 msgr2=0x55b83bf0b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.646+0000 7f690a6d4740 1 --2- 172.21.15.67:0/52498627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83bac6b60 0x55b83bf0b400 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009eb0 tx=0x7f68f802bc00 comp rx=0 tx=0).stop 2024-07-31T14:58:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.646+0000 7f690a6d4740 1 -- 172.21.15.67:0/52498627 shutdown_connections 2024-07-31T14:58:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.646+0000 7f690a6d4740 1 --2- 172.21.15.67:0/52498627 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b83bf146a0 0x55b83bf16ac0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.647+0000 7f690a6d4740 1 --2- 172.21.15.67:0/52498627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b83bac6b60 0x55b83bf0b400 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.647+0000 7f690a6d4740 1 --2- 172.21.15.67:0/52498627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b83babe980 0x55b83beace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.647+0000 7f68e57fa640 0 client.26912 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.647+0000 7f690a6d4740 1 --2- 172.21.15.67:0/52498627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b83baa61c0 0x55b83beac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.647+0000 7f690a6d4740 1 -- 172.21.15.67:0/52498627 >> 172.21.15.67:0/52498627 conn(0x55b83badb990 msgr2=0x55b83be90b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.647+0000 7f690a6d4740 1 -- 172.21.15.67:0/52498627 shutdown_connections 2024-07-31T14:58:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.647+0000 7f690a6d4740 1 -- 172.21.15.67:0/52498627 wait complete. 2024-07-31T14:58:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.647+0000 7f690a6d4740 20 client.26912 trim_cache size 0 max 16384 2024-07-31T14:58:40.649 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:58:40.650 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 53, 2024-07-31T14:58:40.650 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T14:58:40.650 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 18, 2024-07-31T14:58:40.650 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T14:58:40.650 INFO:teuthology.orchestra.run.smithi067.stdout: "88463319": { 2024-07-31T14:58:40.650 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 18, 2024-07-31T14:58:40.650 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T14:58:40.650 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:58:40.650 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T14:58:40.650 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:58:40.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:58:40.651 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T14:58:40.651 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-07-31T14:58:40.651 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T14:58:40.651 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T14:58:40.651 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T14:58:40.651 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:58:40.651 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T14:58:40.651 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:58:40.651 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:58:40.651 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:58:40.651 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:58:40.651 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:58:40.651 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:58:40.651 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:58:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.648+0000 7f6902f21640 1 -- 172.21.15.67:0/996968272 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68e80b3ca0 msgr2=0x7f68e80b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.648+0000 7f6902f21640 1 --2- 172.21.15.67:0/996968272 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68e80b3ca0 0x7f68e80b61b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f68f800d520 tx=0x7f68f800d3e0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.648+0000 7f6902f21640 1 -- 172.21.15.67:0/996968272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68f4095030 msgr2=0x7f68f4144010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.648+0000 7f6902f21640 1 --2- 172.21.15.67:0/996968272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68f4095030 0x7f68f4144010 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00fc80 tx=0x7f68ec034cf0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.648+0000 7f6902f21640 1 -- 172.21.15.67:0/996968272 shutdown_connections 2024-07-31T14:58:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.648+0000 7f6902f21640 1 --2- 172.21.15.67:0/996968272 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68e80b3ca0 0x7f68e80b61b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.648+0000 7f6902f21640 1 --2- 172.21.15.67:0/996968272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68f409b2c0 0x7f68f4149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.648+0000 7f6902f21640 1 --2- 172.21.15.67:0/996968272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68f409a970 0x7f68f41445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.648+0000 7f6902f21640 1 --2- 172.21.15.67:0/996968272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68f4095030 0x7f68f4144010 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.648+0000 7f6902f21640 1 -- 172.21.15.67:0/996968272 >> 172.21.15.67:0/996968272 conn(0x7f68f400cd80 msgr2=0x7f68f4098a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.648+0000 7f6902f21640 1 -- 172.21.15.67:0/996968272 shutdown_connections 2024-07-31T14:58:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:40.648+0000 7f6902f21640 1 -- 172.21.15.67:0/996968272 wait complete. 2024-07-31T14:58:40.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:40.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:40.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:40 smithi077 ceph-mon[37327]: pgmap v629: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 887 op/s 2024-07-31T14:58:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:41.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:41.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:41.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:41.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:41.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:41.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:41.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:41.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:41.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:41.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:41.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:41.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.171 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '88463319', quiesce took 50.7 seconds. Will release after: -20.700000000000003 2024-07-31T14:58:42.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:58:42.521 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:58:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: pgmap v630: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 965 op/s 2024-07-31T14:58:42.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:42.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: pgmap v630: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 965 op/s 2024-07-31T14:58:42.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:42.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: pgmap v630: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 965 op/s 2024-07-31T14:58:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:42.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:42.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:43.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:43.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:43.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:43.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.268+0000 7f2ea1bde640 1 -- 172.21.15.67:0/76045315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e9c107dd0 msgr2=0x7f2e9c1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:44.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.268+0000 7f2ea1bde640 1 --2- 172.21.15.67:0/76045315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e9c107dd0 0x7f2e9c1081e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009900 tx=0x7f2e9002a3f0 comp rx=0 tx=0).stop 2024-07-31T14:58:44.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.268+0000 7f2ea1bde640 1 -- 172.21.15.67:0/76045315 shutdown_connections 2024-07-31T14:58:44.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.268+0000 7f2ea1bde640 1 --2- 172.21.15.67:0/76045315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e9c108720 0x7f2e9c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:44.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.268+0000 7f2ea1bde640 1 --2- 172.21.15.67:0/76045315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e9c107dd0 0x7f2e9c1081e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:44.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.268+0000 7f2ea1bde640 1 --2- 172.21.15.67:0/76045315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e9c102490 0x7f2e9c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:44.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.268+0000 7f2ea1bde640 1 -- 172.21.15.67:0/76045315 >> 172.21.15.67:0/76045315 conn(0x7f2e9c079160 msgr2=0x7f2e9c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:44.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.268+0000 7f2ea1bde640 1 -- 172.21.15.67:0/76045315 shutdown_connections 2024-07-31T14:58:44.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.268+0000 7f2ea1bde640 1 -- 172.21.15.67:0/76045315 wait complete. 2024-07-31T14:58:44.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.269+0000 7f2ea1bde640 1 Processor -- start 2024-07-31T14:58:44.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.270+0000 7f2ea1bde640 1 -- start start 2024-07-31T14:58:44.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.270+0000 7f2ea1bde640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e9c102490 0x7f2e9c19c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:44.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.270+0000 7f2ea1bde640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e9c107dd0 0x7f2e9c19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:44.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.270+0000 7f2ea1bde640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e9c108720 0x7f2e9c1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:44.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.270+0000 7f2ea1bde640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9c1112d0 con 0x7f2e9c108720 2024-07-31T14:58:44.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.270+0000 7f2ea1bde640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9c111150 con 0x7f2e9c102490 2024-07-31T14:58:44.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.270+0000 7f2ea1bde640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9c111450 con 0x7f2e9c107dd0 2024-07-31T14:58:44.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.270+0000 7f2e9affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e9c107dd0 0x7f2e9c19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:44.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.270+0000 7f2e9bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e9c108720 0x7f2e9c1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:44.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.270+0000 7f2e9b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e9c102490 0x7f2e9c19c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:44.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.270+0000 7f2e9affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e9c107dd0 0x7f2e9c19c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:36784/0 (socket says 172.21.15.67:36784) 2024-07-31T14:58:44.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.270+0000 7f2e9affd640 1 -- 172.21.15.67:0/2594945938 learned_addr learned my addr 172.21.15.67:0/2594945938 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:44.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.271+0000 7f2e9bfff640 1 -- 172.21.15.67:0/2594945938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e9c107dd0 msgr2=0x7f2e9c19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:44.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.271+0000 7f2e9bfff640 1 --2- 172.21.15.67:0/2594945938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e9c107dd0 0x7f2e9c19c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:44.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.271+0000 7f2e9bfff640 1 -- 172.21.15.67:0/2594945938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e9c102490 msgr2=0x7f2e9c19c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:44.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.271+0000 7f2e9bfff640 1 --2- 172.21.15.67:0/2594945938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e9c102490 0x7f2e9c19c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:44.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.271+0000 7f2e9bfff640 1 -- 172.21.15.67:0/2594945938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9c1a2080 con 0x7f2e9c108720 2024-07-31T14:58:44.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.271+0000 7f2e9affd640 1 --2- 172.21.15.67:0/2594945938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e9c107dd0 0x7f2e9c19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:58:44.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.271+0000 7f2e9bfff640 1 --2- 172.21.15.67:0/2594945938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e9c108720 0x7f2e9c1a1950 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400ec50 tx=0x7f2e9400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:44.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.271+0000 7f2e98ff9640 1 -- 172.21.15.67:0/2594945938 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94003b70 con 0x7f2e9c108720 2024-07-31T14:58:44.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.271+0000 7f2e98ff9640 1 -- 172.21.15.67:0/2594945938 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e94003d10 con 0x7f2e9c108720 2024-07-31T14:58:44.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.271+0000 7f2ea1bde640 1 -- 172.21.15.67:0/2594945938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e9c1a2370 con 0x7f2e9c108720 2024-07-31T14:58:44.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.271+0000 7f2e98ff9640 1 -- 172.21.15.67:0/2594945938 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94010830 con 0x7f2e9c108720 2024-07-31T14:58:44.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.271+0000 7f2ea1bde640 1 -- 172.21.15.67:0/2594945938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e9c1a8ad0 con 0x7f2e9c108720 2024-07-31T14:58:44.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.273+0000 7f2ea1bde640 1 -- 172.21.15.67:0/2594945938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e9c1028a0 con 0x7f2e9c108720 2024-07-31T14:58:44.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.275+0000 7f2e98ff9640 1 -- 172.21.15.67:0/2594945938 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2e94004b40 con 0x7f2e9c108720 2024-07-31T14:58:44.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.276+0000 7f2e98ff9640 1 --2- 172.21.15.67:0/2594945938 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2e840b3d30 0x7f2e840b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:44.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.276+0000 7f2e98ff9640 1 -- 172.21.15.67:0/2594945938 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9483+0+0 (secure 0 0 0) 0x7f2e940e4940 con 0x7f2e9c108720 2024-07-31T14:58:44.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.276+0000 7f2e9b7fe640 1 --2- 172.21.15.67:0/2594945938 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2e840b3d30 0x7f2e840b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:44.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.277+0000 7f2e9b7fe640 1 --2- 172.21.15.67:0/2594945938 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2e840b3d30 0x7f2e840b6240 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00bbc0 tx=0x7f2e8c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:44.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.279+0000 7f2e98ff9640 1 -- 172.21.15.67:0/2594945938 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f2e940ab920 con 0x7f2e9c108720 2024-07-31T14:58:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: pgmap v631: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 1.31k op/s 2024-07-31T14:58:44.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:44.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.567+0000 7f2ea1bde640 1 -- 172.21.15.67:0/2594945938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e9c19d240 con 0x7f2e9c108720 2024-07-31T14:58:44.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.568+0000 7f2e98ff9640 1 -- 172.21.15.67:0/2594945938 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11967 (secure 0 0 0) 0x7f2e9c19d240 con 0x7f2e9c108720 2024-07-31T14:58:44.570 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:58:44.570 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":53,"btime":"2024-07-31T14:58:33:190058+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":53,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:58:33.148765+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T14:58:44.570 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 53 2024-07-31T14:58:44.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.570+0000 7f2ea1bde640 1 -- 172.21.15.67:0/2594945938 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2e840b3d30 msgr2=0x7f2e840b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:44.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.570+0000 7f2ea1bde640 1 --2- 172.21.15.67:0/2594945938 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2e840b3d30 0x7f2e840b6240 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00bbc0 tx=0x7f2e8c00b390 comp rx=0 tx=0).stop 2024-07-31T14:58:44.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.571+0000 7f2ea1bde640 1 -- 172.21.15.67:0/2594945938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e9c108720 msgr2=0x7f2e9c1a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:44.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.571+0000 7f2ea1bde640 1 --2- 172.21.15.67:0/2594945938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e9c108720 0x7f2e9c1a1950 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400ec50 tx=0x7f2e9400c6a0 comp rx=0 tx=0).stop 2024-07-31T14:58:44.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.571+0000 7f2ea1bde640 1 -- 172.21.15.67:0/2594945938 shutdown_connections 2024-07-31T14:58:44.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.571+0000 7f2ea1bde640 1 --2- 172.21.15.67:0/2594945938 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2e840b3d30 0x7f2e840b6240 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:44.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.571+0000 7f2ea1bde640 1 --2- 172.21.15.67:0/2594945938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e9c108720 0x7f2e9c1a1950 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:44.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.571+0000 7f2ea1bde640 1 --2- 172.21.15.67:0/2594945938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e9c107dd0 0x7f2e9c19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:44.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.571+0000 7f2ea1bde640 1 --2- 172.21.15.67:0/2594945938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e9c102490 0x7f2e9c19c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:44.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.571+0000 7f2ea1bde640 1 -- 172.21.15.67:0/2594945938 >> 172.21.15.67:0/2594945938 conn(0x7f2e9c079160 msgr2=0x7f2e9c103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:44.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.571+0000 7f2ea1bde640 1 -- 172.21.15.67:0/2594945938 shutdown_connections 2024-07-31T14:58:44.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:44.571+0000 7f2ea1bde640 1 -- 172.21.15.67:0/2594945938 wait complete. 2024-07-31T14:58:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: pgmap v631: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 1.31k op/s 2024-07-31T14:58:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:44.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: pgmap v631: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 1.31k op/s 2024-07-31T14:58:44.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:44.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.275 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 88463319 --release --await' 2024-07-31T14:58:45.275 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 88463319 --release --await 2024-07-31T14:58:45.541 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:58:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2594945938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:58:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:45 smithi067 ceph-mon[32750]: osdmap e92: 12 total, 12 up, 12 in 2024-07-31T14:58:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2594945938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:58:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:45 smithi073 ceph-mon[36714]: osdmap e92: 12 total, 12 up, 12 in 2024-07-31T14:58:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2594945938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:58:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:45 smithi077 ceph-mon[37327]: osdmap e92: 12 total, 12 up, 12 in 2024-07-31T14:58:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:58:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:46 smithi067 ceph-mon[32750]: pgmap v633: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.1 MiB/s wr, 2.90k op/s 2024-07-31T14:58:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:58:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:46 smithi073 ceph-mon[36714]: pgmap v633: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.1 MiB/s wr, 2.90k op/s 2024-07-31T14:58:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:58:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:58:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:46 smithi077 ceph-mon[37327]: pgmap v633: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.1 MiB/s wr, 2.90k op/s 2024-07-31T14:58:47.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.357+0000 7fe8048e9640 1 -- 172.21.15.67:0/786307914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe800101a20 msgr2=0x7fe800101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:47.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.357+0000 7fe8048e9640 1 --2- 172.21.15.67:0/786307914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe800101a20 0x7fe800101e30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009900 tx=0x7fe7f002a410 comp rx=0 tx=0).stop 2024-07-31T14:58:47.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.357+0000 7fe8048e9640 1 -- 172.21.15.67:0/786307914 shutdown_connections 2024-07-31T14:58:47.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.357+0000 7fe8048e9640 1 --2- 172.21.15.67:0/786307914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe800106650 0x7fe8001100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.357+0000 7fe8048e9640 1 --2- 172.21.15.67:0/786307914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe800105d00 0x7fe800106110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.357+0000 7fe8048e9640 1 --2- 172.21.15.67:0/786307914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe800101a20 0x7fe800101e30 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.357+0000 7fe8048e9640 1 -- 172.21.15.67:0/786307914 >> 172.21.15.67:0/786307914 conn(0x7fe800079860 msgr2=0x7fe800079c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.357+0000 7fe8048e9640 1 -- 172.21.15.67:0/786307914 shutdown_connections 2024-07-31T14:58:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.357+0000 7fe8048e9640 1 -- 172.21.15.67:0/786307914 wait complete. 2024-07-31T14:58:47.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.358+0000 7fe8048e9640 1 Processor -- start 2024-07-31T14:58:47.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.358+0000 7fe8048e9640 1 -- start start 2024-07-31T14:58:47.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.358+0000 7fe8048e9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe800101a20 0x7fe800199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.358+0000 7fe8048e9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe800105d00 0x7fe80019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.358+0000 7fe8048e9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe800106650 0x7fe80019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.358+0000 7fe8048e9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001134f0 con 0x7fe800101a20 2024-07-31T14:58:47.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.358+0000 7fe8048e9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe800113370 con 0x7fe800105d00 2024-07-31T14:58:47.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe8048e9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe800113670 con 0x7fe800106650 2024-07-31T14:58:47.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe800101a20 0x7fe800199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe800101a20 0x7fe800199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46926/0 (socket says 172.21.15.67:46926) 2024-07-31T14:58:47.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe7fe59c640 1 -- 172.21.15.67:0/4180869682 learned_addr learned my addr 172.21.15.67:0/4180869682 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:47.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe7fdd9b640 1 --2- 172.21.15.67:0/4180869682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe800105d00 0x7fe80019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe7fed9d640 1 --2- 172.21.15.67:0/4180869682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe800106650 0x7fe80019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe7fe59c640 1 -- 172.21.15.67:0/4180869682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe800106650 msgr2=0x7fe80019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:47.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe7fe59c640 1 --2- 172.21.15.67:0/4180869682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe800106650 0x7fe80019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe7fe59c640 1 -- 172.21.15.67:0/4180869682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe800105d00 msgr2=0x7fe80019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:47.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe7fe59c640 1 --2- 172.21.15.67:0/4180869682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe800105d00 0x7fe80019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe7fe59c640 1 -- 172.21.15.67:0/4180869682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80019fe10 con 0x7fe800101a20 2024-07-31T14:58:47.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe7fe59c640 1 --2- 172.21.15.67:0/4180869682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe800101a20 0x7fe800199ea0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0005940 tx=0x7fe7f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:47.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe7ef7fe640 1 -- 172.21.15.67:0/4180869682 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f003f030 con 0x7fe800101a20 2024-07-31T14:58:47.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe7ef7fe640 1 -- 172.21.15.67:0/4180869682 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7f0003cb0 con 0x7fe800101a20 2024-07-31T14:58:47.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.359+0000 7fe8048e9640 1 -- 172.21.15.67:0/4180869682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8001a00a0 con 0x7fe800101a20 2024-07-31T14:58:47.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.360+0000 7fe7ef7fe640 1 -- 172.21.15.67:0/4180869682 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0035070 con 0x7fe800101a20 2024-07-31T14:58:47.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.360+0000 7fe8048e9640 1 -- 172.21.15.67:0/4180869682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8001a6860 con 0x7fe800101a20 2024-07-31T14:58:47.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.362+0000 7fe80ae3a740 1 Processor -- start 2024-07-31T14:58:47.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.362+0000 7fe80ae3a740 1 -- start start 2024-07-31T14:58:47.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe7ef7fe640 1 -- 172.21.15.67:0/4180869682 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe7f0042030 con 0x7fe800101a20 2024-07-31T14:58:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe80ae3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 0x5571989ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe80ae3a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe80ae3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985e7840 0x5571989aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe80ae3a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557198923e70 con 0x5571985df660 2024-07-31T14:58:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe80ae3a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557198913370 con 0x557198608520 2024-07-31T14:58:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe80ae3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5571989173b0 con 0x5571985e7840 2024-07-31T14:58:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989ae5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46932/0 (socket says 172.21.15.67:46932) 2024-07-31T14:58:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe7fe59c640 1 -- 172.21.15.67:0/199711170 learned_addr learned my addr 172.21.15.67:0/199711170 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe7fed9d640 1 --2- 172.21.15.67:0/199711170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 0x5571989ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe7fdd9b640 1 --2- 172.21.15.67:0/199711170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985e7840 0x5571989aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe7fe59c640 1 -- 172.21.15.67:0/199711170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985e7840 msgr2=0x5571989aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe7fe59c640 1 --2- 172.21.15.67:0/199711170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985e7840 0x5571989aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe7fe59c640 1 -- 172.21.15.67:0/199711170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 msgr2=0x5571989ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe7fe59c640 1 --2- 172.21.15.67:0/199711170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 0x5571989ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.363+0000 7fe7fe59c640 1 -- 172.21.15.67:0/199711170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571988e0040 con 0x5571985df660 2024-07-31T14:58:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.364+0000 7fe7fed9d640 1 --2- 172.21.15.67:0/199711170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 0x5571989ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:58:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.364+0000 7fe7ef7fe640 1 --2- 172.21.15.67:0/4180869682 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7e80b3d40 0x7fe7e80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.364+0000 7fe7fe59c640 1 --2- 172.21.15.67:0/199711170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989ae5b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0101c60 tx=0x7fe7f010ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e930dc616c0afc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.364+0000 7fe7fed9d640 1 --2- 172.21.15.67:0/4180869682 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7e80b3d40 0x7fe7e80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.364+0000 7fe7ecff9640 1 -- 172.21.15.67:0/199711170 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f00c7070 con 0x5571985df660 2024-07-31T14:58:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.364+0000 7fe7ef7fe640 1 -- 172.21.15.67:0/4180869682 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fe7f0104910 con 0x7fe800101a20 2024-07-31T14:58:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.364+0000 7fe7ecff9640 1 -- 172.21.15.67:0/199711170 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7f00cacc0 con 0x5571985df660 2024-07-31T14:58:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.364+0000 7fe7ecff9640 1 -- 172.21.15.67:0/199711170 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f00ca920 con 0x5571985df660 2024-07-31T14:58:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.365+0000 7fe80ae3a740 1 -- 172.21.15.67:0/199711170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 msgr2=0x5571989ae5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.365+0000 7fe7fed9d640 1 --2- 172.21.15.67:0/4180869682 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7e80b3d40 0x7fe7e80b6250 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800be70 tx=0x7fe7f800b450 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.365+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/199711170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989ae5b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0101c60 tx=0x7fe7f010ac40 comp rx=0 tx=0).stop 2024-07-31T14:58:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.365+0000 7fe80ae3a740 1 -- 172.21.15.67:0/199711170 shutdown_connections 2024-07-31T14:58:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.365+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/199711170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985e7840 0x5571989aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.365+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/199711170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989ae5b0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.365+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/199711170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 0x5571989ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.365+0000 7fe80ae3a740 1 -- 172.21.15.67:0/199711170 >> 172.21.15.67:0/199711170 conn(0x5571985fc910 msgr2=0x5571989abf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.365+0000 7fe80ae3a740 1 -- 172.21.15.67:0/199711170 shutdown_connections 2024-07-31T14:58:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.365+0000 7fe80ae3a740 1 -- 172.21.15.67:0/199711170 wait complete. 2024-07-31T14:58:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe80ae3a740 1 -- start start 2024-07-31T14:58:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe80ae3a740 1 Processor -- start 2024-07-31T14:58:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe80ae3a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989b9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe80ae3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985e7840 0x5571989ba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe80ae3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 0x5571989c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe80ae3a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557198776bb0 con 0x5571985df660 2024-07-31T14:58:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe80ae3a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5571988b5ac0 con 0x557198608520 2024-07-31T14:58:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe80ae3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5571989bc610 con 0x5571985e7840 2024-07-31T14:58:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe80ae3a740 10 client.0 _ll_register_callbacks cb 0x5571988be3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989b9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989b9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46944/0 (socket says 172.21.15.67:46944) 2024-07-31T14:58:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe7fe59c640 1 -- 172.21.15.67:0/3999711419 learned_addr learned my addr 172.21.15.67:0/3999711419 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe7fdd9b640 1 --2- 172.21.15.67:0/3999711419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985e7840 0x5571989ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe7fed9d640 1 --2- 172.21.15.67:0/3999711419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 0x5571989c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe7fe59c640 1 -- 172.21.15.67:0/3999711419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985e7840 msgr2=0x5571989ba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe7fe59c640 1 --2- 172.21.15.67:0/3999711419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985e7840 0x5571989ba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe7fe59c640 1 -- 172.21.15.67:0/3999711419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 msgr2=0x5571989c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe7fe59c640 1 --2- 172.21.15.67:0/3999711419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 0x5571989c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe7fe59c640 1 -- 172.21.15.67:0/3999711419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557198673b00 con 0x5571985df660 2024-07-31T14:58:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.366+0000 7fe7fed9d640 1 --2- 172.21.15.67:0/3999711419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 0x5571989c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:58:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.367+0000 7fe7fe59c640 1 --2- 172.21.15.67:0/3999711419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989b9c00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0032af0 tx=0x7fe7f003b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd892ed07ca00ff8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.367+0000 7fe7e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:58:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.367+0000 7fe7e5ffb640 1 -- 172.21.15.67:0/3999711419 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0049070 con 0x5571985df660 2024-07-31T14:58:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.367+0000 7fe7e5ffb640 1 -- 172.21.15.67:0/3999711419 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fe7f0101f70 con 0x5571985df660 2024-07-31T14:58:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.367+0000 7fe80ae3a740 1 -- 172.21.15.67:0/3999711419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f00cacc0 con 0x5571985df660 2024-07-31T14:58:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.367+0000 7fe7e5ffb640 1 -- 172.21.15.67:0/3999711419 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fe7f00cab00 con 0x5571985df660 2024-07-31T14:58:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.367+0000 7fe80ae3a740 10 client.17442 fetch_fsmap learned FSMap version 53 2024-07-31T14:58:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.367+0000 7fe80ae3a740 1 -- 172.21.15.67:0/3999711419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5571986ea960 con 0x5571985df660 2024-07-31T14:58:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.367+0000 7fe7e5ffb640 1 -- 172.21.15.67:0/3999711419 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7f00cacc0 con 0x5571985df660 2024-07-31T14:58:47.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.367+0000 7fe7e5ffb640 1 -- 172.21.15.67:0/3999711419 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 53) ==== 8390+0+0 (secure 0 0 0) 0x7fe7f00330d0 con 0x5571985df660 2024-07-31T14:58:47.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.368+0000 7fe80ae3a740 10 client.17442 fetch_fsmap finished waiting for FSMap version 53 2024-07-31T14:58:47.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.368+0000 7fe80ae3a740 10 client.17442 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:58:47.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.368+0000 7fe80ae3a740 20 client.17442 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:47.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.368+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/3999711419 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5571985c6ea0 0x5571989cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.368+0000 7fe80ae3a740 4 client.17442 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:58:47.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.368+0000 7fe80ae3a740 1 -- 172.21.15.67:0/3999711419 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557198925dc0 con 0x5571985c6ea0 2024-07-31T14:58:47.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.368+0000 7fe7fdd9b640 1 --2- 172.21.15.67:0/3999711419 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5571985c6ea0 0x5571989cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.368+0000 7fe7fdd9b640 1 --2- 172.21.15.67:0/3999711419 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5571985c6ea0 0x5571989cb6f0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ef7def66abf8bf server_cookie=8632c871474218dc in_seq=0 out_seq=0 2024-07-31T14:58:47.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.368+0000 7fe7e5ffb640 10 client.17442 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:47.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.369+0000 7fe7e5ffb640 1 -- 172.21.15.67:0/3999711419 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557198925dc0 con 0x5571985c6ea0 2024-07-31T14:58:47.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.369+0000 7fe7e5ffb640 10 client.17442 handle_command_reply: tid=0 2024-07-31T14:58:47.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.369+0000 7fe80ae3a740 1 client.17442 shutdown 2024-07-31T14:58:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.369+0000 7fe80ae3a740 1 -- 172.21.15.67:0/3999711419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 msgr2=0x5571989b9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.369+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/3999711419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989b9c00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0032af0 tx=0x7fe7f003b6b0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.369+0000 7fe80ae3a740 1 -- 172.21.15.67:0/3999711419 shutdown_connections 2024-07-31T14:58:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.369+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/3999711419 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5571985c6ea0 0x5571989cb6f0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.369+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/3999711419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 0x5571989c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.369+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/3999711419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985e7840 0x5571989ba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.369+0000 7fe7e5ffb640 0 client.17442 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.369+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/3999711419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989b9c00 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.369+0000 7fe80ae3a740 1 -- 172.21.15.67:0/3999711419 >> 172.21.15.67:0/3999711419 conn(0x5571985fc910 msgr2=0x5571989b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.369+0000 7fe80ae3a740 1 -- 172.21.15.67:0/3999711419 shutdown_connections 2024-07-31T14:58:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.369+0000 7fe80ae3a740 1 -- 172.21.15.67:0/3999711419 wait complete. 2024-07-31T14:58:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.370+0000 7fe80ae3a740 20 client.17442 trim_cache size 0 max 16384 2024-07-31T14:58:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.387+0000 7fe80ae3a740 1 Processor -- start 2024-07-31T14:58:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.387+0000 7fe80ae3a740 1 -- start start 2024-07-31T14:58:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe80ae3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985df660 0x5571989aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe80ae3a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985e7840 0x5571989b8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe80ae3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 0x5571989c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe80ae3a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001b0740 con 0x5571985e7840 2024-07-31T14:58:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe80ae3a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001b2370 con 0x557198608520 2024-07-31T14:58:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe80ae3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001b0e40 con 0x5571985df660 2024-07-31T14:58:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe7fdd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985e7840 0x5571989b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe7fdd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985e7840 0x5571989b8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46952/0 (socket says 172.21.15.67:46952) 2024-07-31T14:58:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe7fdd9b640 1 -- 172.21.15.67:0/2949321150 learned_addr learned my addr 172.21.15.67:0/2949321150 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe7fe59c640 1 --2- 172.21.15.67:0/2949321150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985df660 0x5571989aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe7fdd9b640 1 -- 172.21.15.67:0/2949321150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985df660 msgr2=0x5571989aaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe7fdd9b640 1 --2- 172.21.15.67:0/2949321150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985df660 0x5571989aaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe7fdd9b640 1 -- 172.21.15.67:0/2949321150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 msgr2=0x5571989c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe7fdd9b640 1 --2- 172.21.15.67:0/2949321150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 0x5571989c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe7fdd9b640 1 -- 172.21.15.67:0/2949321150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571989b3860 con 0x5571985e7840 2024-07-31T14:58:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.388+0000 7fe7fdd9b640 1 --2- 172.21.15.67:0/2949321150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985e7840 0x5571989b8280 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x5571989cbcf0 tx=0x7fe7f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e299e1dfbf1463ad server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.389+0000 7fe7e7fff640 1 -- 172.21.15.67:0/2949321150 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4014c90 con 0x5571985e7840 2024-07-31T14:58:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.389+0000 7fe7e7fff640 1 -- 172.21.15.67:0/2949321150 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7f4014e30 con 0x5571985e7840 2024-07-31T14:58:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.389+0000 7fe7e7fff640 1 -- 172.21.15.67:0/2949321150 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4012cf0 con 0x5571985e7840 2024-07-31T14:58:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.389+0000 7fe80ae3a740 1 -- 172.21.15.67:0/2949321150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985e7840 msgr2=0x5571989b8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.389+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/2949321150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985e7840 0x5571989b8280 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x5571989cbcf0 tx=0x7fe7f4004a40 comp rx=0 tx=0).stop 2024-07-31T14:58:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.389+0000 7fe80ae3a740 1 -- 172.21.15.67:0/2949321150 shutdown_connections 2024-07-31T14:58:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.389+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/2949321150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557198608520 0x5571989c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.389+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/2949321150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985e7840 0x5571989b8280 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.389+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/2949321150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571985df660 0x5571989aaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.389+0000 7fe80ae3a740 1 -- 172.21.15.67:0/2949321150 >> 172.21.15.67:0/2949321150 conn(0x5571985fc910 msgr2=0x5571989ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.389+0000 7fe80ae3a740 1 -- 172.21.15.67:0/2949321150 shutdown_connections 2024-07-31T14:58:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.389+0000 7fe80ae3a740 1 -- 172.21.15.67:0/2949321150 wait complete. 2024-07-31T14:58:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.390+0000 7fe80ae3a740 1 -- start start 2024-07-31T14:58:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.390+0000 7fe80ae3a740 1 Processor -- start 2024-07-31T14:58:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.390+0000 7fe80ae3a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989be060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.390+0000 7fe80ae3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571985e7840 0x5571989be620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.390+0000 7fe80ae3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557198608520 0x557198a2cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.390+0000 7fe80ae3a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001b1ca0 con 0x5571985df660 2024-07-31T14:58:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.390+0000 7fe80ae3a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001b1510 con 0x5571985e7840 2024-07-31T14:58:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.390+0000 7fe80ae3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001ae740 con 0x557198608520 2024-07-31T14:58:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.390+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989be060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.390+0000 7fe80ae3a740 10 client.0 _ll_register_callbacks cb 0x7fe7f00d7070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.390+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989be060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46958/0 (socket says 172.21.15.67:46958) 2024-07-31T14:58:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.390+0000 7fe7fe59c640 1 -- 172.21.15.67:0/480824620 learned_addr learned my addr 172.21.15.67:0/480824620 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe7fdd9b640 1 --2- 172.21.15.67:0/480824620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571985e7840 0x5571989be620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe7fed9d640 1 --2- 172.21.15.67:0/480824620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557198608520 0x557198a2cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe7fe59c640 1 -- 172.21.15.67:0/480824620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557198608520 msgr2=0x557198a2cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe7fe59c640 1 --2- 172.21.15.67:0/480824620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557198608520 0x557198a2cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe7fe59c640 1 -- 172.21.15.67:0/480824620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571985e7840 msgr2=0x5571989be620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe7fe59c640 1 --2- 172.21.15.67:0/480824620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571985e7840 0x5571989be620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe7fe59c640 1 -- 172.21.15.67:0/480824620 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557198a2d3d0 con 0x5571985df660 2024-07-31T14:58:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe7fed9d640 1 --2- 172.21.15.67:0/480824620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557198608520 0x557198a2cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:58:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe7fe59c640 1 --2- 172.21.15.67:0/480824620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989be060 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0101da0 tx=0x7fe7f00cf930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9669328551ad49 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe7e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:58:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe7e5ffb640 1 -- 172.21.15.67:0/480824620 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0049070 con 0x5571985df660 2024-07-31T14:58:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe7e5ffb640 1 -- 172.21.15.67:0/480824620 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fe7f00c77a0 con 0x5571985df660 2024-07-31T14:58:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe80ae3a740 1 -- 172.21.15.67:0/480824620 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f4014e30 con 0x5571985df660 2024-07-31T14:58:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe7e5ffb640 1 -- 172.21.15.67:0/480824620 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fe7f00caf80 con 0x5571985df660 2024-07-31T14:58:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe80ae3a740 10 client.17448 fetch_fsmap learned FSMap version 53 2024-07-31T14:58:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.391+0000 7fe80ae3a740 1 -- 172.21.15.67:0/480824620 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x557198a2d3d0 con 0x5571985df660 2024-07-31T14:58:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.392+0000 7fe7e5ffb640 1 -- 172.21.15.67:0/480824620 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7f4014e30 con 0x5571985df660 2024-07-31T14:58:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.392+0000 7fe7e5ffb640 1 -- 172.21.15.67:0/480824620 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 53) ==== 8390+0+0 (secure 0 0 0) 0x7fe7f00cb410 con 0x5571985df660 2024-07-31T14:58:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.392+0000 7fe80ae3a740 10 client.17448 fetch_fsmap finished waiting for FSMap version 53 2024-07-31T14:58:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.392+0000 7fe80ae3a740 10 client.17448 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:58:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.392+0000 7fe80ae3a740 20 client.17448 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.392+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/480824620 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55719859c250 0x557198a3b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.392+0000 7fe80ae3a740 4 client.17448 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "88463319", "release": true, "await": true}] 2024-07-31T14:58:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.392+0000 7fe80ae3a740 1 -- 172.21.15.67:0/480824620 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "88463319", "release": true, "await": true}) -- 0x557198921860 con 0x55719859c250 2024-07-31T14:58:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.392+0000 7fe7fdd9b640 1 --2- 172.21.15.67:0/480824620 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55719859c250 0x557198a3b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.393+0000 7fe7fdd9b640 1 --2- 172.21.15.67:0/480824620 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55719859c250 0x557198a3b000 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=974d0af6a8e3f724 server_cookie=6858ea83a9f55b in_seq=0 out_seq=0 2024-07-31T14:58:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:47.393+0000 7fe7e5ffb640 10 client.17448 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:48.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:48 smithi077 ceph-mon[37327]: pgmap v634: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 856 KiB/s wr, 3.32k op/s 2024-07-31T14:58:49.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.038+0000 7fe7e5ffb640 1 -- 172.21.15.67:0/480824620 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557198921860 con 0x55719859c250 2024-07-31T14:58:49.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.038+0000 7fe7e5ffb640 10 client.17448 handle_command_reply: tid=0 2024-07-31T14:58:49.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.038+0000 7fe80ae3a740 1 client.17448 shutdown 2024-07-31T14:58:49.040 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:48 smithi067 ceph-mon[32750]: pgmap v634: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 856 KiB/s wr, 3.32k op/s 2024-07-31T14:58:49.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.039+0000 7fe80ae3a740 1 -- 172.21.15.67:0/480824620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 msgr2=0x5571989be060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:49.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.039+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/480824620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989be060 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0101da0 tx=0x7fe7f00cf930 comp rx=0 tx=0).stop 2024-07-31T14:58:49.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.039+0000 7fe80ae3a740 1 -- 172.21.15.67:0/480824620 shutdown_connections 2024-07-31T14:58:49.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.039+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/480824620 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55719859c250 0x557198a3b000 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:49.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.039+0000 7fe7e5ffb640 0 client.17448 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:49.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.039+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/480824620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557198608520 0x557198a2cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.039+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/480824620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571985e7840 0x5571989be620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.039+0000 7fe80ae3a740 1 --2- 172.21.15.67:0/480824620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5571985df660 0x5571989be060 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.039+0000 7fe80ae3a740 1 -- 172.21.15.67:0/480824620 >> 172.21.15.67:0/480824620 conn(0x5571985fc910 msgr2=0x5571989ab8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.039+0000 7fe80ae3a740 1 -- 172.21.15.67:0/480824620 shutdown_connections 2024-07-31T14:58:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.039+0000 7fe80ae3a740 1 -- 172.21.15.67:0/480824620 wait complete. 2024-07-31T14:58:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.040+0000 7fe80ae3a740 20 client.17448 trim_cache size 0 max 16384 2024-07-31T14:58:49.042 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:58:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 54, 2024-07-31T14:58:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T14:58:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 20, 2024-07-31T14:58:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T14:58:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "88463319": { 2024-07-31T14:58:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 20, 2024-07-31T14:58:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T14:58:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:58:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T14:58:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:58:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:58:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T14:58:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-07-31T14:58:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T14:58:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T14:58:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T14:58:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:58:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T14:58:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:58:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:58:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:58:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:58:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:58:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:58:49.044 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:58:49.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.041+0000 7fe8048e9640 1 -- 172.21.15.67:0/4180869682 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7e80b3d40 msgr2=0x7fe7e80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:49.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.041+0000 7fe8048e9640 1 --2- 172.21.15.67:0/4180869682 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7e80b3d40 0x7fe7e80b6250 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800be70 tx=0x7fe7f800b450 comp rx=0 tx=0).stop 2024-07-31T14:58:49.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.041+0000 7fe8048e9640 1 -- 172.21.15.67:0/4180869682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe800101a20 msgr2=0x7fe800199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:49.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.041+0000 7fe8048e9640 1 --2- 172.21.15.67:0/4180869682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe800101a20 0x7fe800199ea0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0005940 tx=0x7fe7f00042f0 comp rx=0 tx=0).stop 2024-07-31T14:58:49.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.041+0000 7fe8048e9640 1 -- 172.21.15.67:0/4180869682 shutdown_connections 2024-07-31T14:58:49.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.041+0000 7fe8048e9640 1 --2- 172.21.15.67:0/4180869682 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7e80b3d40 0x7fe7e80b6250 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:49.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.041+0000 7fe8048e9640 1 --2- 172.21.15.67:0/4180869682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe800106650 0x7fe80019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:49.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.041+0000 7fe8048e9640 1 --2- 172.21.15.67:0/4180869682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe800105d00 0x7fe80019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:49.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.041+0000 7fe8048e9640 1 --2- 172.21.15.67:0/4180869682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe800101a20 0x7fe800199ea0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:49.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.041+0000 7fe8048e9640 1 -- 172.21.15.67:0/4180869682 >> 172.21.15.67:0/4180869682 conn(0x7fe800079860 msgr2=0x7fe8001079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:49.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.041+0000 7fe8048e9640 1 -- 172.21.15.67:0/4180869682 shutdown_connections 2024-07-31T14:58:49.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:49.041+0000 7fe8048e9640 1 -- 172.21.15.67:0/4180869682 wait complete. 2024-07-31T14:58:49.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:48 smithi073 ceph-mon[36714]: pgmap v634: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 856 KiB/s wr, 3.32k op/s 2024-07-31T14:58:49.759 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '88463319', total seconds elapsed: 58.3 2024-07-31T14:58:49.759 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-07-31T14:58:49.759 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:58:49.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:49 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:58:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:58:50.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:49 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:58:50.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:49 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:58:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:50 smithi077 ceph-mon[37327]: pgmap v635: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 727 KiB/s wr, 3.50k op/s 2024-07-31T14:58:51.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:50 smithi077 ceph-mon[37327]: osdmap e93: 12 total, 12 up, 12 in 2024-07-31T14:58:51.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:50 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:58:51.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:50 smithi067 ceph-mon[32750]: pgmap v635: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 727 KiB/s wr, 3.50k op/s 2024-07-31T14:58:51.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:50 smithi067 ceph-mon[32750]: osdmap e93: 12 total, 12 up, 12 in 2024-07-31T14:58:51.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:50 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:58:51.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:50 smithi073 ceph-mon[36714]: pgmap v635: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 727 KiB/s wr, 3.50k op/s 2024-07-31T14:58:51.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:50 smithi073 ceph-mon[36714]: osdmap e93: 12 total, 12 up, 12 in 2024-07-31T14:58:51.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:50 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:58:51.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.793+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/2847306916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0c0108720 msgr2=0x7fd0c010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:51.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.793+0000 7fd0c6ae9640 1 --2- 172.21.15.67:0/2847306916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0c0108720 0x7fd0c010ef50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8009900 tx=0x7fd0b802a410 comp rx=0 tx=0).stop 2024-07-31T14:58:51.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.793+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/2847306916 shutdown_connections 2024-07-31T14:58:51.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.793+0000 7fd0c6ae9640 1 --2- 172.21.15.67:0/2847306916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0c0108720 0x7fd0c010ef50 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:51.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.793+0000 7fd0c6ae9640 1 --2- 172.21.15.67:0/2847306916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0c0107dd0 0x7fd0c01081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:51.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.793+0000 7fd0c6ae9640 1 --2- 172.21.15.67:0/2847306916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0c0102490 0x7fd0c01028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:51.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.793+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/2847306916 >> 172.21.15.67:0/2847306916 conn(0x7fd0c0079160 msgr2=0x7fd0c0079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:51.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.793+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/2847306916 shutdown_connections 2024-07-31T14:58:51.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.793+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/2847306916 wait complete. 2024-07-31T14:58:51.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.794+0000 7fd0c6ae9640 1 Processor -- start 2024-07-31T14:58:51.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.794+0000 7fd0c6ae9640 1 -- start start 2024-07-31T14:58:51.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.794+0000 7fd0c6ae9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0c0102490 0x7fd0c0104b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:51.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.794+0000 7fd0c6ae9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0c0107dd0 0x7fd0c0105050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:51.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.794+0000 7fd0c6ae9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0c0108720 0x7fd0c0105590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:51.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.794+0000 7fd0c6ae9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c0112380 con 0x7fd0c0108720 2024-07-31T14:58:51.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.794+0000 7fd0c6ae9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c0112200 con 0x7fd0c0102490 2024-07-31T14:58:51.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.794+0000 7fd0c6ae9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c0112500 con 0x7fd0c0107dd0 2024-07-31T14:58:51.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0c5086640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0c0108720 0x7fd0c0105590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:51.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0c5086640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0c0108720 0x7fd0c0105590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54364/0 (socket says 172.21.15.67:54364) 2024-07-31T14:58:51.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0c5086640 1 -- 172.21.15.67:0/827877385 learned_addr learned my addr 172.21.15.67:0/827877385 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:51.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0c4885640 1 --2- 172.21.15.67:0/827877385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0c0102490 0x7fd0c0104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:51.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0bffff640 1 --2- 172.21.15.67:0/827877385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0c0107dd0 0x7fd0c0105050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:51.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0c5086640 1 -- 172.21.15.67:0/827877385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0c0107dd0 msgr2=0x7fd0c0105050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:51.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0c5086640 1 --2- 172.21.15.67:0/827877385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0c0107dd0 0x7fd0c0105050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:51.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0c5086640 1 -- 172.21.15.67:0/827877385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0c0102490 msgr2=0x7fd0c0104b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:51.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0c5086640 1 --2- 172.21.15.67:0/827877385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0c0102490 0x7fd0c0104b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:51.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0c5086640 1 -- 172.21.15.67:0/827877385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c01a80d0 con 0x7fd0c0108720 2024-07-31T14:58:51.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0c5086640 1 --2- 172.21.15.67:0/827877385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0c0108720 0x7fd0c0105590 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8004770 tx=0x7fd0b8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:51.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0bdffb640 1 -- 172.21.15.67:0/827877385 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b803f030 con 0x7fd0c0108720 2024-07-31T14:58:51.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0bdffb640 1 -- 172.21.15.67:0/827877385 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b8004440 con 0x7fd0c0108720 2024-07-31T14:58:51.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/827877385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c01a8360 con 0x7fd0c0108720 2024-07-31T14:58:51.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.795+0000 7fd0bdffb640 1 -- 172.21.15.67:0/827877385 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8035070 con 0x7fd0c0108720 2024-07-31T14:58:51.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.796+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/827877385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c01a8840 con 0x7fd0c0108720 2024-07-31T14:58:51.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.797+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/827877385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c01028a0 con 0x7fd0c0108720 2024-07-31T14:58:51.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.798+0000 7fd0bdffb640 1 -- 172.21.15.67:0/827877385 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd0b8042030 con 0x7fd0c0108720 2024-07-31T14:58:51.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.799+0000 7fd0bdffb640 1 --2- 172.21.15.67:0/827877385 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd0ac0b3d40 0x7fd0ac0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:51.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.799+0000 7fd0c4885640 1 --2- 172.21.15.67:0/827877385 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd0ac0b3d40 0x7fd0ac0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:51.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.799+0000 7fd0bdffb640 1 -- 172.21.15.67:0/827877385 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7fd0b8104640 con 0x7fd0c0108720 2024-07-31T14:58:51.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.799+0000 7fd0c4885640 1 --2- 172.21.15.67:0/827877385 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd0ac0b3d40 0x7fd0ac0b6250 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0106560 tx=0x7fd0a8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:51.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:51.803+0000 7fd0bdffb640 1 -- 172.21.15.67:0/827877385 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd0b80cafc0 con 0x7fd0c0108720 2024-07-31T14:58:52.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:52.039+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/827877385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0c002e0c0 con 0x7fd0c0108720 2024-07-31T14:58:52.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:52.040+0000 7fd0bdffb640 1 -- 172.21.15.67:0/827877385 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11971 (secure 0 0 0) 0x7fd0b80cf510 con 0x7fd0c0108720 2024-07-31T14:58:52.044 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:58:52.045 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":54,"btime":"2024-07-31T14:58:48:655841+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:58:48.279201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:58:52.047 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 54 2024-07-31T14:58:52.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:52.042+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/827877385 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd0ac0b3d40 msgr2=0x7fd0ac0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:52.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:52.042+0000 7fd0c6ae9640 1 --2- 172.21.15.67:0/827877385 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd0ac0b3d40 0x7fd0ac0b6250 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0106560 tx=0x7fd0a8005f90 comp rx=0 tx=0).stop 2024-07-31T14:58:52.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:52.042+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/827877385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0c0108720 msgr2=0x7fd0c0105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:52.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:52.042+0000 7fd0c6ae9640 1 --2- 172.21.15.67:0/827877385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0c0108720 0x7fd0c0105590 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8004770 tx=0x7fd0b8004040 comp rx=0 tx=0).stop 2024-07-31T14:58:52.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:52.043+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/827877385 shutdown_connections 2024-07-31T14:58:52.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:52.043+0000 7fd0c6ae9640 1 --2- 172.21.15.67:0/827877385 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd0ac0b3d40 0x7fd0ac0b6250 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:52.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:52.043+0000 7fd0c6ae9640 1 --2- 172.21.15.67:0/827877385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0c0108720 0x7fd0c0105590 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:52.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:52.043+0000 7fd0c6ae9640 1 --2- 172.21.15.67:0/827877385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0c0107dd0 0x7fd0c0105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:52.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:52.043+0000 7fd0c6ae9640 1 --2- 172.21.15.67:0/827877385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0c0102490 0x7fd0c0104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:52.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:52.043+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/827877385 >> 172.21.15.67:0/827877385 conn(0x7fd0c0079160 msgr2=0x7fd0c006abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:52.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:52.043+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/827877385 shutdown_connections 2024-07-31T14:58:52.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:52.043+0000 7fd0c6ae9640 1 -- 172.21.15.67:0/827877385 wait complete. 2024-07-31T14:58:52.718 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-07-31T14:58:52.718 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-07-31T14:58:52.985 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:58:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:52 smithi077 ceph-mon[37327]: pgmap v637: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 46 KiB/s wr, 3.43k op/s 2024-07-31T14:58:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:52 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/827877385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:58:53.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:52 smithi067 ceph-mon[32750]: pgmap v637: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 46 KiB/s wr, 3.43k op/s 2024-07-31T14:58:53.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:52 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/827877385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:58:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.249+0000 7fe901570640 1 -- 172.21.15.67:0/1779959930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8fc108690 msgr2=0x7fe8fc10eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.249+0000 7fe901570640 1 --2- 172.21.15.67:0/1779959930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8fc108690 0x7fe8fc10eec0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009ca0 tx=0x7fe8f402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.249+0000 7fe901570640 1 -- 172.21.15.67:0/1779959930 shutdown_connections 2024-07-31T14:58:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.249+0000 7fe901570640 1 --2- 172.21.15.67:0/1779959930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8fc108690 0x7fe8fc10eec0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.249+0000 7fe901570640 1 --2- 172.21.15.67:0/1779959930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8fc107d40 0x7fe8fc108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.249+0000 7fe901570640 1 --2- 172.21.15.67:0/1779959930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8fc102410 0x7fe8fc102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.249+0000 7fe901570640 1 -- 172.21.15.67:0/1779959930 >> 172.21.15.67:0/1779959930 conn(0x7fe8fc07a140 msgr2=0x7fe8fc07a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.249+0000 7fe901570640 1 -- 172.21.15.67:0/1779959930 shutdown_connections 2024-07-31T14:58:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.249+0000 7fe901570640 1 -- 172.21.15.67:0/1779959930 wait complete. 2024-07-31T14:58:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.250+0000 7fe901570640 1 Processor -- start 2024-07-31T14:58:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.250+0000 7fe901570640 1 -- start start 2024-07-31T14:58:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.250+0000 7fe901570640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8fc102410 0x7fe8fc19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.250+0000 7fe901570640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8fc107d40 0x7fe8fc19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.250+0000 7fe901570640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8fc108690 0x7fe8fc19b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.250+0000 7fe901570640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc112300 con 0x7fe8fc107d40 2024-07-31T14:58:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.250+0000 7fe901570640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc112180 con 0x7fe8fc102410 2024-07-31T14:58:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.250+0000 7fe901570640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc112480 con 0x7fe8fc108690 2024-07-31T14:58:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.250+0000 7fe8f3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8fc107d40 0x7fe8fc19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.250+0000 7fe8f3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8fc107d40 0x7fe8fc19b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54378/0 (socket says 172.21.15.67:54378) 2024-07-31T14:58:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.250+0000 7fe8f3fff640 1 -- 172.21.15.67:0/1291605956 learned_addr learned my addr 172.21.15.67:0/1291605956 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.250+0000 7fe8faffd640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8fc102410 0x7fe8fc19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.250+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8fc108690 0x7fe8fc19b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.251+0000 7fe8faffd640 1 -- 172.21.15.67:0/1291605956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8fc108690 msgr2=0x7fe8fc19b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.251+0000 7fe8faffd640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8fc108690 0x7fe8fc19b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.251+0000 7fe8faffd640 1 -- 172.21.15.67:0/1291605956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8fc107d40 msgr2=0x7fe8fc19b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.251+0000 7fe8faffd640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8fc107d40 0x7fe8fc19b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.251+0000 7fe8faffd640 1 -- 172.21.15.67:0/1291605956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc1a5ee0 con 0x7fe8fc102410 2024-07-31T14:58:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.251+0000 7fe8f3fff640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8fc107d40 0x7fe8fc19b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:58:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.251+0000 7fe8faffd640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8fc102410 0x7fe8fc19ac50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009d20 tx=0x7fe8ec00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.251+0000 7fe8f8ff9640 1 -- 172.21.15.67:0/1291605956 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec01f070 con 0x7fe8fc102410 2024-07-31T14:58:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.251+0000 7fe901570640 1 -- 172.21.15.67:0/1291605956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc198aa0 con 0x7fe8fc102410 2024-07-31T14:58:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.252+0000 7fe8f8ff9640 1 -- 172.21.15.67:0/1291605956 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ec011750 con 0x7fe8fc102410 2024-07-31T14:58:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.252+0000 7fe8f8ff9640 1 -- 172.21.15.67:0/1291605956 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec01a690 con 0x7fe8fc102410 2024-07-31T14:58:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.252+0000 7fe901570640 1 -- 172.21.15.67:0/1291605956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc199040 con 0x7fe8fc102410 2024-07-31T14:58:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.255+0000 7fe908464740 1 Processor -- start 2024-07-31T14:58:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.255+0000 7fe908464740 1 -- start start 2024-07-31T14:58:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.255+0000 7fe908464740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080851b0 0x565108427ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.255+0000 7fe908464740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56510805c2f0 0x565108428220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.255+0000 7fe908464740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080644d0 0x565108428760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.255+0000 7fe908464740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56510839d3b0 con 0x5651080644d0 2024-07-31T14:58:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.255+0000 7fe908464740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56510838c8b0 con 0x5651080851b0 2024-07-31T14:58:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.255+0000 7fe908464740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5651083908f0 con 0x56510805c2f0 2024-07-31T14:58:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.256+0000 7fe8f8ff9640 1 -- 172.21.15.67:0/1291605956 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe8ec022070 con 0x7fe8fc102410 2024-07-31T14:58:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.256+0000 7fe8f3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080644d0 0x565108428760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.256+0000 7fe8f3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080644d0 0x565108428760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54382/0 (socket says 172.21.15.67:54382) 2024-07-31T14:58:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.256+0000 7fe8f3fff640 1 -- 172.21.15.67:0/2185627949 learned_addr learned my addr 172.21.15.67:0/2185627949 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.256+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/2185627949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080851b0 0x565108427ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.256+0000 7fe8faffd640 1 --2- 172.21.15.67:0/2185627949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56510805c2f0 0x565108428220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.256+0000 7fe8fb7fe640 1 -- 172.21.15.67:0/2185627949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56510805c2f0 msgr2=0x565108428220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.256+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/2185627949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56510805c2f0 0x565108428220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.256+0000 7fe8fb7fe640 1 -- 172.21.15.67:0/2185627949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080644d0 msgr2=0x565108428760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.256+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/2185627949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080644d0 0x565108428760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.256+0000 7fe8fb7fe640 1 -- 172.21.15.67:0/2185627949 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651083339a0 con 0x5651080851b0 2024-07-31T14:58:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.256+0000 7fe8faffd640 1 --2- 172.21.15.67:0/2185627949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56510805c2f0 0x565108428220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:58:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.256+0000 7fe8f3fff640 1 --2- 172.21.15.67:0/2185627949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080644d0 0x565108428760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:58:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.257+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/2185627949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080851b0 0x565108427ce0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe8f402ade0 tx=0x7fe8f402f4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a02b8d8b9607fe4f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.257+0000 7fe8f8ff9640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe8e80b3d40 0x7fe8e80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.257+0000 7fe8f17fa640 1 -- 172.21.15.67:0/2185627949 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4031cf0 con 0x5651080851b0 2024-07-31T14:58:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.257+0000 7fe8f17fa640 1 -- 172.21.15.67:0/2185627949 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8f4036070 con 0x5651080851b0 2024-07-31T14:58:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.257+0000 7fe8faffd640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe8e80b3d40 0x7fe8e80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.257+0000 7fe8f17fa640 1 -- 172.21.15.67:0/2185627949 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40042a0 con 0x5651080851b0 2024-07-31T14:58:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.257+0000 7fe8f8ff9640 1 -- 172.21.15.67:0/1291605956 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7fe8ec0e7f30 con 0x7fe8fc102410 2024-07-31T14:58:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.258+0000 7fe908464740 1 -- 172.21.15.67:0/2185627949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080851b0 msgr2=0x565108427ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.258+0000 7fe8faffd640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe8e80b3d40 0x7fe8e80b6250 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0e53f0 tx=0x7fe8ec009f60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.258+0000 7fe908464740 1 --2- 172.21.15.67:0/2185627949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080851b0 0x565108427ce0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe8f402ade0 tx=0x7fe8f402f4c0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.258+0000 7fe908464740 1 -- 172.21.15.67:0/2185627949 shutdown_connections 2024-07-31T14:58:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.258+0000 7fe908464740 1 --2- 172.21.15.67:0/2185627949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080644d0 0x565108428760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.258+0000 7fe908464740 1 --2- 172.21.15.67:0/2185627949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56510805c2f0 0x565108428220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.258+0000 7fe908464740 1 --2- 172.21.15.67:0/2185627949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080851b0 0x565108427ce0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.258+0000 7fe908464740 1 -- 172.21.15.67:0/2185627949 >> 172.21.15.67:0/2185627949 conn(0x565108079300 msgr2=0x565108425bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.258+0000 7fe908464740 1 -- 172.21.15.67:0/2185627949 shutdown_connections 2024-07-31T14:58:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.258+0000 7fe908464740 1 -- 172.21.15.67:0/2185627949 wait complete. 2024-07-31T14:58:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.259+0000 7fe908464740 1 -- start start 2024-07-31T14:58:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.259+0000 7fe908464740 1 Processor -- start 2024-07-31T14:58:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.259+0000 7fe908464740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56510805c2f0 0x565108433810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.259+0000 7fe908464740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080644d0 0x565108433d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.259+0000 7fe908464740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080851b0 0x56510843f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.259+0000 7fe908464740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5651082c6ab0 con 0x5651080851b0 2024-07-31T14:58:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.259+0000 7fe908464740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x565108314cf0 con 0x5651080644d0 2024-07-31T14:58:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.259+0000 7fe908464740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565108436280 con 0x56510805c2f0 2024-07-31T14:58:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.259+0000 7fe908464740 10 client.0 _ll_register_callbacks cb 0x5651083869f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.259+0000 7fe8fb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080851b0 0x56510843f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.259+0000 7fe8fb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080851b0 0x56510843f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54394/0 (socket says 172.21.15.67:54394) 2024-07-31T14:58:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.259+0000 7fe8fb7fe640 1 -- 172.21.15.67:0/4167866669 learned_addr learned my addr 172.21.15.67:0/4167866669 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.259+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/4167866669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56510805c2f0 0x565108433810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.259+0000 7fe8f3fff640 1 --2- 172.21.15.67:0/4167866669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080644d0 0x565108433d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.260+0000 7fe8f3fff640 1 -- 172.21.15.67:0/4167866669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56510805c2f0 msgr2=0x565108433810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.260+0000 7fe8f3fff640 1 --2- 172.21.15.67:0/4167866669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56510805c2f0 0x565108433810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.260+0000 7fe8f3fff640 1 -- 172.21.15.67:0/4167866669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080851b0 msgr2=0x56510843f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.260+0000 7fe8f3fff640 1 --2- 172.21.15.67:0/4167866669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080851b0 0x56510843f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.260+0000 7fe8f3fff640 1 -- 172.21.15.67:0/4167866669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651080f0790 con 0x5651080644d0 2024-07-31T14:58:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.260+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/4167866669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080851b0 0x56510843f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:58:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.260+0000 7fe8f3fff640 1 --2- 172.21.15.67:0/4167866669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080644d0 0x565108433d50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4011780 tx=0x7fe8e4011c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc31cec3e337e57b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.260+0000 7fe8e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:58:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.260+0000 7fe8e27fc640 1 -- 172.21.15.67:0/4167866669 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4010c60 con 0x5651080644d0 2024-07-31T14:58:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.260+0000 7fe8e27fc640 1 -- 172.21.15.67:0/4167866669 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7fe8e4010e00 con 0x5651080644d0 2024-07-31T14:58:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.260+0000 7fe908464740 1 -- 172.21.15.67:0/4167866669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8f4036070 con 0x5651080644d0 2024-07-31T14:58:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.260+0000 7fe8e27fc640 1 -- 172.21.15.67:0/4167866669 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe8e4025050 con 0x5651080644d0 2024-07-31T14:58:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.260+0000 7fe908464740 10 client.27023 fetch_fsmap learned FSMap version 54 2024-07-31T14:58:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.260+0000 7fe908464740 1 -- 172.21.15.67:0/4167866669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x565108169620 con 0x5651080644d0 2024-07-31T14:58:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.261+0000 7fe8e27fc640 1 -- 172.21.15.67:0/4167866669 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8f4036070 con 0x5651080644d0 2024-07-31T14:58:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.261+0000 7fe8e27fc640 1 -- 172.21.15.67:0/4167866669 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7fe8e401a400 con 0x5651080644d0 2024-07-31T14:58:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.261+0000 7fe908464740 10 client.27023 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:58:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.261+0000 7fe908464740 10 client.27023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:58:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.261+0000 7fe908464740 20 client.27023 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.261+0000 7fe908464740 1 --2- 172.21.15.67:0/4167866669 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x565108043d80 0x565108445720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.261+0000 7fe908464740 4 client.27023 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:58:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.261+0000 7fe908464740 1 -- 172.21.15.67:0/4167866669 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56510839f300 con 0x565108043d80 2024-07-31T14:58:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.261+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/4167866669 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x565108043d80 0x565108445720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.261+0000 7fe8e27fc640 10 client.27023 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.261+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/4167866669 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x565108043d80 0x565108445720 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=552b33d0ba024b60 server_cookie=247d1b9e114c7e66 in_seq=0 out_seq=0 2024-07-31T14:58:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.262+0000 7fe8e27fc640 1 -- 172.21.15.67:0/4167866669 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56510839f300 con 0x565108043d80 2024-07-31T14:58:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.262+0000 7fe8e27fc640 10 client.27023 handle_command_reply: tid=0 2024-07-31T14:58:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.262+0000 7fe908464740 1 client.27023 shutdown 2024-07-31T14:58:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.262+0000 7fe908464740 1 -- 172.21.15.67:0/4167866669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080644d0 msgr2=0x565108433d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.262+0000 7fe908464740 1 --2- 172.21.15.67:0/4167866669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080644d0 0x565108433d50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4011780 tx=0x7fe8e4011c40 comp rx=0 tx=0).stop 2024-07-31T14:58:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.262+0000 7fe908464740 1 -- 172.21.15.67:0/4167866669 shutdown_connections 2024-07-31T14:58:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.262+0000 7fe908464740 1 --2- 172.21.15.67:0/4167866669 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x565108043d80 0x565108445720 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.262+0000 7fe908464740 1 --2- 172.21.15.67:0/4167866669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080851b0 0x56510843f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.262+0000 7fe908464740 1 --2- 172.21.15.67:0/4167866669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080644d0 0x565108433d50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.262+0000 7fe908464740 1 --2- 172.21.15.67:0/4167866669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56510805c2f0 0x565108433810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.262+0000 7fe8e27fc640 0 client.27023 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.262+0000 7fe908464740 1 -- 172.21.15.67:0/4167866669 >> 172.21.15.67:0/4167866669 conn(0x565108079300 msgr2=0x5651084242b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.262+0000 7fe908464740 1 -- 172.21.15.67:0/4167866669 shutdown_connections 2024-07-31T14:58:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.262+0000 7fe908464740 1 -- 172.21.15.67:0/4167866669 wait complete. 2024-07-31T14:58:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.263+0000 7fe908464740 20 client.27023 trim_cache size 0 max 16384 2024-07-31T14:58:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.279+0000 7fe908464740 1 Processor -- start 2024-07-31T14:58:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.279+0000 7fe908464740 1 -- start start 2024-07-31T14:58:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.279+0000 7fe908464740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56510805c2f0 0x565108435230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.279+0000 7fe908464740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080644d0 0x565108432e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.279+0000 7fe908464740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651080851b0 0x56510843f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.279+0000 7fe908464740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc00fd60 con 0x56510805c2f0 2024-07-31T14:58:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.279+0000 7fe908464740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc0119f0 con 0x5651080644d0 2024-07-31T14:58:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.279+0000 7fe908464740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc010430 con 0x5651080851b0 2024-07-31T14:58:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8fb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56510805c2f0 0x565108435230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8fb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56510805c2f0 0x565108435230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54404/0 (socket says 172.21.15.67:54404) 2024-07-31T14:58:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8fb7fe640 1 -- 172.21.15.67:0/2734312810 learned_addr learned my addr 172.21.15.67:0/2734312810 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8f3fff640 1 --2- 172.21.15.67:0/2734312810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080644d0 0x565108432e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/2734312810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651080851b0 0x56510843f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8fb7fe640 1 -- 172.21.15.67:0/2734312810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651080851b0 msgr2=0x56510843f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/2734312810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651080851b0 0x56510843f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8fb7fe640 1 -- 172.21.15.67:0/2734312810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080644d0 msgr2=0x565108432e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/2734312810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080644d0 0x565108432e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8fb7fe640 1 -- 172.21.15.67:0/2734312810 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565108426570 con 0x56510805c2f0 2024-07-31T14:58:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8f3fff640 1 --2- 172.21.15.67:0/2734312810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080644d0 0x565108432e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:58:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/2734312810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56510805c2f0 0x565108435230 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009620 tx=0x7fe8f402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317a2dc5d2be4349 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8f0ff9640 1 -- 172.21.15.67:0/2734312810 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4009b20 con 0x56510805c2f0 2024-07-31T14:58:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8f0ff9640 1 -- 172.21.15.67:0/2734312810 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8f4036040 con 0x56510805c2f0 2024-07-31T14:58:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.280+0000 7fe8f0ff9640 1 -- 172.21.15.67:0/2734312810 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40047e0 con 0x56510805c2f0 2024-07-31T14:58:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.281+0000 7fe908464740 1 -- 172.21.15.67:0/2734312810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56510805c2f0 msgr2=0x565108435230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.281+0000 7fe908464740 1 --2- 172.21.15.67:0/2734312810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56510805c2f0 0x565108435230 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009620 tx=0x7fe8f402f3a0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.281+0000 7fe908464740 1 -- 172.21.15.67:0/2734312810 shutdown_connections 2024-07-31T14:58:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.281+0000 7fe908464740 1 --2- 172.21.15.67:0/2734312810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651080851b0 0x56510843f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.281+0000 7fe908464740 1 --2- 172.21.15.67:0/2734312810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651080644d0 0x565108432e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.281+0000 7fe908464740 1 --2- 172.21.15.67:0/2734312810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56510805c2f0 0x565108435230 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.281+0000 7fe908464740 1 -- 172.21.15.67:0/2734312810 >> 172.21.15.67:0/2734312810 conn(0x565108079300 msgr2=0x565108434b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.281+0000 7fe908464740 1 -- 172.21.15.67:0/2734312810 shutdown_connections 2024-07-31T14:58:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.281+0000 7fe908464740 1 -- 172.21.15.67:0/2734312810 wait complete. 2024-07-31T14:58:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.282+0000 7fe908464740 1 -- start start 2024-07-31T14:58:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.282+0000 7fe908464740 1 Processor -- start 2024-07-31T14:58:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.282+0000 7fe908464740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56510805c2f0 0x5651084386f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.282+0000 7fe908464740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651080644d0 0x565108438c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.282+0000 7fe908464740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080851b0 0x5651084abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.282+0000 7fe908464740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc011320 con 0x5651080851b0 2024-07-31T14:58:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.282+0000 7fe908464740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc010b90 con 0x56510805c2f0 2024-07-31T14:58:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.282+0000 7fe908464740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc00dd60 con 0x5651080644d0 2024-07-31T14:58:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.282+0000 7fe908464740 10 client.0 _ll_register_callbacks cb 0x7fe8e400be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.282+0000 7fe8fb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080851b0 0x5651084abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.282+0000 7fe8fb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080851b0 0x5651084abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54420/0 (socket says 172.21.15.67:54420) 2024-07-31T14:58:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.282+0000 7fe8fb7fe640 1 -- 172.21.15.67:0/3178554448 learned_addr learned my addr 172.21.15.67:0/3178554448 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.282+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/3178554448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56510805c2f0 0x5651084386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.282+0000 7fe8f3fff640 1 --2- 172.21.15.67:0/3178554448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651080644d0 0x565108438c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe8fb7fe640 1 -- 172.21.15.67:0/3178554448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651080644d0 msgr2=0x565108438c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/3178554448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651080644d0 0x565108438c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe8fb7fe640 1 -- 172.21.15.67:0/3178554448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56510805c2f0 msgr2=0x5651084386f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/3178554448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56510805c2f0 0x5651084386f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe8fb7fe640 1 -- 172.21.15.67:0/3178554448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565108442100 con 0x5651080851b0 2024-07-31T14:58:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe8f3fff640 1 --2- 172.21.15.67:0/3178554448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651080644d0 0x565108438c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:58:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe8fb7fe640 1 --2- 172.21.15.67:0/3178554448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080851b0 0x5651084abe90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4005e30 tx=0x7fe8f402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ccd7a7b5925a16 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe8e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:58:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe8e27fc640 1 -- 172.21.15.67:0/3178554448 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4049070 con 0x5651080851b0 2024-07-31T14:58:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe8e27fc640 1 -- 172.21.15.67:0/3178554448 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7fe8f4004660 con 0x5651080851b0 2024-07-31T14:58:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe908464740 1 -- 172.21.15.67:0/3178554448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8f4036040 con 0x5651080851b0 2024-07-31T14:58:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe8e27fc640 1 -- 172.21.15.67:0/3178554448 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe8f4004860 con 0x5651080851b0 2024-07-31T14:58:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe908464740 10 client.17472 fetch_fsmap learned FSMap version 54 2024-07-31T14:58:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe908464740 1 -- 172.21.15.67:0/3178554448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56510842aaf0 con 0x5651080851b0 2024-07-31T14:58:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.283+0000 7fe8e27fc640 1 -- 172.21.15.67:0/3178554448 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8f4036040 con 0x5651080851b0 2024-07-31T14:58:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.284+0000 7fe8e27fc640 1 -- 172.21.15.67:0/3178554448 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7fe8f4043020 con 0x5651080851b0 2024-07-31T14:58:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.284+0000 7fe908464740 10 client.17472 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:58:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.284+0000 7fe908464740 10 client.17472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:58:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.284+0000 7fe908464740 20 client.17472 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.284+0000 7fe908464740 1 --2- 172.21.15.67:0/3178554448 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5651080191c0 0x5651084af300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.284+0000 7fe908464740 4 client.17472 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:58:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.284+0000 7fe908464740 1 -- 172.21.15.67:0/3178554448 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56510839ada0 con 0x5651080191c0 2024-07-31T14:58:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.284+0000 7fe8f3fff640 1 --2- 172.21.15.67:0/3178554448 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5651080191c0 0x5651084af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.284+0000 7fe8f3fff640 1 --2- 172.21.15.67:0/3178554448 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5651080191c0 0x5651084af300 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4af28b0ac743e402 server_cookie=b4b3f3095164e3cb in_seq=0 out_seq=0 2024-07-31T14:58:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:53.284+0000 7fe8e27fc640 10 client.17472 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:52 smithi073 ceph-mon[36714]: pgmap v637: 129 pgs: 129 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 46 KiB/s wr, 3.43k op/s 2024-07-31T14:58:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:52 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/827877385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:58:55.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.047+0000 7f695691c640 1 -- 172.21.15.67:0/3289375977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6950108740 msgr2=0x7f695010ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:55.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.047+0000 7f695691c640 1 --2- 172.21.15.67:0/3289375977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6950108740 0x7f695010ef70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f694000fb10 tx=0x7f694002e5b0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.050+0000 7f695691c640 1 -- 172.21.15.67:0/3289375977 shutdown_connections 2024-07-31T14:58:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.050+0000 7f695691c640 1 --2- 172.21.15.67:0/3289375977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6950108740 0x7f695010ef70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.050+0000 7f695691c640 1 --2- 172.21.15.67:0/3289375977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6950107df0 0x7f6950108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.050+0000 7f695691c640 1 --2- 172.21.15.67:0/3289375977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69501024b0 0x7f69501028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.051+0000 7f695691c640 1 -- 172.21.15.67:0/3289375977 >> 172.21.15.67:0/3289375977 conn(0x7f69500792a0 msgr2=0x7f69500796b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:55.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.052+0000 7f695691c640 1 -- 172.21.15.67:0/3289375977 shutdown_connections 2024-07-31T14:58:55.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.052+0000 7f695691c640 1 -- 172.21.15.67:0/3289375977 wait complete. 2024-07-31T14:58:55.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.053+0000 7f695691c640 1 Processor -- start 2024-07-31T14:58:55.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.055+0000 7f695691c640 1 -- start start 2024-07-31T14:58:55.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.056+0000 7f695691c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69501024b0 0x7f695006cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.056+0000 7f695691c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6950107df0 0x7f695006d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.056+0000 7f695691c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6950108740 0x7f6950072340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.056+0000 7f695691c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6950111360 con 0x7f6950107df0 2024-07-31T14:58:55.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.056+0000 7f695691c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69501111e0 con 0x7f6950108740 2024-07-31T14:58:55.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.056+0000 7f695691c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69501114e0 con 0x7f69501024b0 2024-07-31T14:58:55.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.060+0000 7f6954eb9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6950108740 0x7f6950072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:55.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.060+0000 7f6954eb9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6950108740 0x7f6950072340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51448/0 (socket says 172.21.15.67:51448) 2024-07-31T14:58:55.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.060+0000 7f6954eb9640 1 -- 172.21.15.67:0/1860407684 learned_addr learned my addr 172.21.15.67:0/1860407684 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:55.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.060+0000 7f694f7fe640 1 --2- 172.21.15.67:0/1860407684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6950107df0 0x7f695006d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:55.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.061+0000 7f694f7fe640 1 -- 172.21.15.67:0/1860407684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69501024b0 msgr2=0x7f695006cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:55.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.061+0000 7f694f7fe640 1 --2- 172.21.15.67:0/1860407684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69501024b0 0x7f695006cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.061+0000 7f694f7fe640 1 -- 172.21.15.67:0/1860407684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6950108740 msgr2=0x7f6950072340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:55.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.061+0000 7f694f7fe640 1 --2- 172.21.15.67:0/1860407684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6950108740 0x7f6950072340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.061+0000 7f694f7fe640 1 -- 172.21.15.67:0/1860407684 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6950072a70 con 0x7f6950107df0 2024-07-31T14:58:55.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.066+0000 7f694f7fe640 1 --2- 172.21.15.67:0/1860407684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6950107df0 0x7f695006d080 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f695006a4f0 tx=0x7f6944007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.068+0000 7f694d7fa640 1 -- 172.21.15.67:0/1860407684 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944007e50 con 0x7f6950107df0 2024-07-31T14:58:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.068+0000 7f695691c640 1 -- 172.21.15.67:0/1860407684 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6950072d00 con 0x7f6950107df0 2024-07-31T14:58:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.068+0000 7f695691c640 1 -- 172.21.15.67:0/1860407684 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69500734b0 con 0x7f6950107df0 2024-07-31T14:58:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.069+0000 7f694d7fa640 1 -- 172.21.15.67:0/1860407684 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f694401a070 con 0x7f6950107df0 2024-07-31T14:58:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.069+0000 7f694d7fa640 1 -- 172.21.15.67:0/1860407684 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944015c30 con 0x7f6950107df0 2024-07-31T14:58:55.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.079+0000 7f695ce6d740 1 Processor -- start 2024-07-31T14:58:55.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.081+0000 7f695ce6d740 1 -- start start 2024-07-31T14:58:55.091 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:54 smithi067 ceph-mon[32750]: pgmap v638: 129 pgs: 129 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 218 KiB/s wr, 1.44k op/s 2024-07-31T14:58:55.092 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:54 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:58:55.092 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:54 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:58:55.092 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:54 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:58:55.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.090+0000 7f695ce6d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cd7840 0x562a1807d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.090+0000 7f695ce6d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cae980 0x562a1807dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.090+0000 7f695ce6d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cb6b60 0x562a1807e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.090+0000 7f695ce6d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562a17ff2780 con 0x562a17cae980 2024-07-31T14:58:55.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.090+0000 7f695ce6d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562a17fe1c80 con 0x562a17cb6b60 2024-07-31T14:58:55.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.090+0000 7f695ce6d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562a17fe5cc0 con 0x562a17cd7840 2024-07-31T14:58:55.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.090+0000 7f694ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cae980 0x562a1807dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:55.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.091+0000 7f694ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cae980 0x562a1807dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54436/0 (socket says 172.21.15.67:54436) 2024-07-31T14:58:55.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.091+0000 7f694ffff640 1 -- 172.21.15.67:0/1252534985 learned_addr learned my addr 172.21.15.67:0/1252534985 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:55.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.092+0000 7f694d7fa640 1 -- 172.21.15.67:0/1860407684 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6944015470 con 0x7f6950107df0 2024-07-31T14:58:55.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.095+0000 7f694d7fa640 1 --2- 172.21.15.67:0/1860407684 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69380bc900 0x7f69380bee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.095+0000 7f694d7fa640 1 -- 172.21.15.67:0/1860407684 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7f69440e4130 con 0x7f6950107df0 2024-07-31T14:58:55.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.095+0000 7f694ffff640 1 -- 172.21.15.67:0/1252534985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cd7840 msgr2=0x562a1807d840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:58:55.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.095+0000 7f694ffff640 1 --2- 172.21.15.67:0/1252534985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cd7840 0x562a1807d840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.095+0000 7f694ffff640 1 -- 172.21.15.67:0/1252534985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cb6b60 msgr2=0x562a1807e2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:58:55.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.095+0000 7f694ffff640 1 --2- 172.21.15.67:0/1252534985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cb6b60 0x562a1807e2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.096+0000 7f694ffff640 1 -- 172.21.15.67:0/1252534985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a17f77080 con 0x562a17cae980 2024-07-31T14:58:55.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.101+0000 7f6954eb9640 1 --2- 172.21.15.67:0/1860407684 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69380bc900 0x7f69380bee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:55.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.101+0000 7f694ffff640 1 --2- 172.21.15.67:0/1252534985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cae980 0x562a1807dd80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6948009d50 tx=0x7f694800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2695162ff173a4c0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:55.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.103+0000 7f69367fc640 1 -- 172.21.15.67:0/1252534985 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694800bd40 con 0x562a17cae980 2024-07-31T14:58:55.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.103+0000 7f69367fc640 1 -- 172.21.15.67:0/1252534985 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6948017070 con 0x562a17cae980 2024-07-31T14:58:55.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.103+0000 7f69367fc640 1 -- 172.21.15.67:0/1252534985 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480115f0 con 0x562a17cae980 2024-07-31T14:58:55.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.104+0000 7f695ce6d740 1 -- 172.21.15.67:0/1252534985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cae980 msgr2=0x562a1807dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:55.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.104+0000 7f695ce6d740 1 --2- 172.21.15.67:0/1252534985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cae980 0x562a1807dd80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6948009d50 tx=0x7f694800b420 comp rx=0 tx=0).stop 2024-07-31T14:58:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.107+0000 7f695ce6d740 1 -- 172.21.15.67:0/1252534985 shutdown_connections 2024-07-31T14:58:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.107+0000 7f695ce6d740 1 --2- 172.21.15.67:0/1252534985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cb6b60 0x562a1807e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.107+0000 7f695ce6d740 1 --2- 172.21.15.67:0/1252534985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cae980 0x562a1807dd80 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.107+0000 7f695ce6d740 1 --2- 172.21.15.67:0/1252534985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cd7840 0x562a1807d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.107+0000 7f695ce6d740 1 -- 172.21.15.67:0/1252534985 >> 172.21.15.67:0/1252534985 conn(0x562a17ccb990 msgr2=0x562a1807b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.115+0000 7f695ce6d740 1 -- 172.21.15.67:0/1252534985 shutdown_connections 2024-07-31T14:58:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.115+0000 7f6954eb9640 1 --2- 172.21.15.67:0/1860407684 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69380bc900 0x7f69380bee10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6940006010 tx=0x7f6940005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.116+0000 7f695ce6d740 1 -- 172.21.15.67:0/1252534985 wait complete. 2024-07-31T14:58:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.116+0000 7f695ce6d740 1 -- start start 2024-07-31T14:58:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.116+0000 7f695ce6d740 1 Processor -- start 2024-07-31T14:58:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.117+0000 7f695ce6d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cae980 0x562a18088ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.117+0000 7f695ce6d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cb6b60 0x562a18089410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.117+0000 7f695ce6d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cd7840 0x562a18089990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.117+0000 7f695ce6d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562a17e457a0 con 0x562a17cb6b60 2024-07-31T14:58:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.117+0000 7f695ce6d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562a17f842f0 con 0x562a17cd7840 2024-07-31T14:58:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.117+0000 7f695ce6d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562a1808c020 con 0x562a17cae980 2024-07-31T14:58:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.117+0000 7f694f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cb6b60 0x562a18089410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.117+0000 7f694f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cb6b60 0x562a18089410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54446/0 (socket says 172.21.15.67:54446) 2024-07-31T14:58:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.117+0000 7f694f7fe640 1 -- 172.21.15.67:0/265105782 learned_addr learned my addr 172.21.15.67:0/265105782 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.117+0000 7f6954eb9640 1 --2- 172.21.15.67:0/265105782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cd7840 0x562a18089990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.118+0000 7f695ce6d740 10 client.0 _ll_register_callbacks cb 0x562a17f8e110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.118+0000 7f694f7fe640 1 -- 172.21.15.67:0/265105782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cae980 msgr2=0x562a18088ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:58:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.118+0000 7f694f7fe640 1 --2- 172.21.15.67:0/265105782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cae980 0x562a18088ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.118+0000 7f694f7fe640 1 -- 172.21.15.67:0/265105782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cd7840 msgr2=0x562a18089990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.118+0000 7f694f7fe640 1 --2- 172.21.15.67:0/265105782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cd7840 0x562a18089990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.118+0000 7f694f7fe640 1 -- 172.21.15.67:0/265105782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a17f88480 con 0x562a17cb6b60 2024-07-31T14:58:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.119+0000 7f694f7fe640 1 --2- 172.21.15.67:0/265105782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cb6b60 0x562a18089410 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f694400b480 tx=0x7f69440aa4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ddd60f00aadca0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.119+0000 7f692f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:58:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.120+0000 7f692f7fe640 1 -- 172.21.15.67:0/265105782 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69440c5060 con 0x562a17cb6b60 2024-07-31T14:58:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.120+0000 7f695ce6d740 1 -- 172.21.15.67:0/265105782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6948017070 con 0x562a17cb6b60 2024-07-31T14:58:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.120+0000 7f692f7fe640 1 -- 172.21.15.67:0/265105782 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7f69440c5200 con 0x562a17cb6b60 2024-07-31T14:58:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.121+0000 7f692f7fe640 1 -- 172.21.15.67:0/265105782 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f69440aa9a0 con 0x562a17cb6b60 2024-07-31T14:58:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.121+0000 7f695ce6d740 10 client.17481 fetch_fsmap learned FSMap version 54 2024-07-31T14:58:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.121+0000 7f695ce6d740 1 -- 172.21.15.67:0/265105782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562a17d51b90 con 0x562a17cb6b60 2024-07-31T14:58:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.121+0000 7f692f7fe640 1 -- 172.21.15.67:0/265105782 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6948017070 con 0x562a17cb6b60 2024-07-31T14:58:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.121+0000 7f692f7fe640 1 -- 172.21.15.67:0/265105782 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f69440a9670 con 0x562a17cb6b60 2024-07-31T14:58:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.122+0000 7f695ce6d740 10 client.17481 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:58:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.122+0000 7f695ce6d740 10 client.17481 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:58:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.122+0000 7f695ce6d740 20 client.17481 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.122+0000 7f695ce6d740 1 --2- 172.21.15.67:0/265105782 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562a17c961c0 0x562a180ac620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.122+0000 7f695ce6d740 4 client.17481 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:58:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.122+0000 7f695ce6d740 1 -- 172.21.15.67:0/265105782 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a17ff46d0 con 0x562a17c961c0 2024-07-31T14:58:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.122+0000 7f694ffff640 1 --2- 172.21.15.67:0/265105782 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562a17c961c0 0x562a180ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:55.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.131+0000 7f692f7fe640 10 client.17481 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:55.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.131+0000 7f694ffff640 1 --2- 172.21.15.67:0/265105782 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562a17c961c0 0x562a180ac620 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2817c087be98bf6 server_cookie=e69955261ee8cda8 in_seq=0 out_seq=0 2024-07-31T14:58:55.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.135+0000 7f692f7fe640 1 -- 172.21.15.67:0/265105782 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a17ff46d0 con 0x562a17c961c0 2024-07-31T14:58:55.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.135+0000 7f692f7fe640 10 client.17481 handle_command_reply: tid=0 2024-07-31T14:58:55.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.135+0000 7f695ce6d740 1 client.17481 shutdown 2024-07-31T14:58:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.137+0000 7f695ce6d740 1 -- 172.21.15.67:0/265105782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cb6b60 msgr2=0x562a18089410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.137+0000 7f695ce6d740 1 --2- 172.21.15.67:0/265105782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cb6b60 0x562a18089410 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f694400b480 tx=0x7f69440aa4b0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.139+0000 7f695ce6d740 1 -- 172.21.15.67:0/265105782 shutdown_connections 2024-07-31T14:58:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.139+0000 7f695ce6d740 1 --2- 172.21.15.67:0/265105782 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562a17c961c0 0x562a180ac620 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.139+0000 7f695ce6d740 1 --2- 172.21.15.67:0/265105782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cd7840 0x562a18089990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.139+0000 7f695ce6d740 1 --2- 172.21.15.67:0/265105782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cb6b60 0x562a18089410 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.139+0000 7f695ce6d740 1 --2- 172.21.15.67:0/265105782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cae980 0x562a18088ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.139+0000 7f695ce6d740 1 -- 172.21.15.67:0/265105782 >> 172.21.15.67:0/265105782 conn(0x562a17ccb990 msgr2=0x562a180809d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.139+0000 7f692f7fe640 0 client.17481 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.143+0000 7f695ce6d740 1 -- 172.21.15.67:0/265105782 shutdown_connections 2024-07-31T14:58:55.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.144+0000 7f695ce6d740 1 -- 172.21.15.67:0/265105782 wait complete. 2024-07-31T14:58:55.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.146+0000 7f695ce6d740 20 client.17481 trim_cache size 0 max 16384 2024-07-31T14:58:55.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.184+0000 7f695ce6d740 1 Processor -- start 2024-07-31T14:58:55.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.185+0000 7f695ce6d740 1 -- start start 2024-07-31T14:58:55.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.185+0000 7f695ce6d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cae980 0x562a1808a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.185+0000 7f695ce6d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cb6b60 0x562a180828d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.186+0000 7f695ce6d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cd7840 0x562a18082e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.186+0000 7f694ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cae980 0x562a1808a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:55.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.186+0000 7f694f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cb6b60 0x562a180828d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:55.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.186+0000 7f694f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cb6b60 0x562a180828d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54458/0 (socket says 172.21.15.67:54458) 2024-07-31T14:58:55.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.186+0000 7f694f7fe640 1 -- 172.21.15.67:0/146610681 learned_addr learned my addr 172.21.15.67:0/146610681 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:55.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.186+0000 7f695ce6d740 1 -- 172.21.15.67:0/146610681 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f693c00fe30 con 0x562a17cb6b60 2024-07-31T14:58:55.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.186+0000 7f695ce6d740 1 -- 172.21.15.67:0/146610681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f693c011ac0 con 0x562a17cae980 2024-07-31T14:58:55.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.186+0000 7f695ce6d740 1 -- 172.21.15.67:0/146610681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f693c010500 con 0x562a17cd7840 2024-07-31T14:58:55.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.187+0000 7f694f7fe640 1 -- 172.21.15.67:0/146610681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cd7840 msgr2=0x562a18082e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:55.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.187+0000 7f694f7fe640 1 --2- 172.21.15.67:0/146610681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cd7840 0x562a18082e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.187+0000 7f694f7fe640 1 -- 172.21.15.67:0/146610681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cae980 msgr2=0x562a1808a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:55.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.187+0000 7f694f7fe640 1 --2- 172.21.15.67:0/146610681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cae980 0x562a1808a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.187+0000 7f694f7fe640 1 -- 172.21.15.67:0/146610681 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a1807c0a0 con 0x562a17cb6b60 2024-07-31T14:58:55.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.187+0000 7f694f7fe640 1 --2- 172.21.15.67:0/146610681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cb6b60 0x562a180828d0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f69440aa6a0 tx=0x7f69440aa1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff1d8133d7e48cf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:55.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.191+0000 7f6935ffb640 1 -- 172.21.15.67:0/146610681 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69440c5040 con 0x562a17cb6b60 2024-07-31T14:58:55.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.192+0000 7f6935ffb640 1 -- 172.21.15.67:0/146610681 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69440b04d0 con 0x562a17cb6b60 2024-07-31T14:58:55.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.192+0000 7f6935ffb640 1 -- 172.21.15.67:0/146610681 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69440a9e10 con 0x562a17cb6b60 2024-07-31T14:58:55.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.193+0000 7f695ce6d740 1 -- 172.21.15.67:0/146610681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cb6b60 msgr2=0x562a180828d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:55.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.193+0000 7f695ce6d740 1 --2- 172.21.15.67:0/146610681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cb6b60 0x562a180828d0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f69440aa6a0 tx=0x7f69440aa1e0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.193+0000 7f695ce6d740 1 -- 172.21.15.67:0/146610681 shutdown_connections 2024-07-31T14:58:55.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.193+0000 7f695ce6d740 1 --2- 172.21.15.67:0/146610681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cd7840 0x562a18082e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.193+0000 7f695ce6d740 1 --2- 172.21.15.67:0/146610681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17cb6b60 0x562a180828d0 secure :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f69440aa6a0 tx=0x7f69440aa1e0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.193+0000 7f695ce6d740 1 --2- 172.21.15.67:0/146610681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cae980 0x562a1808a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.193+0000 7f695ce6d740 1 -- 172.21.15.67:0/146610681 >> 172.21.15.67:0/146610681 conn(0x562a17ccb990 msgr2=0x562a1807f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:55.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.196+0000 7f695ce6d740 1 -- 172.21.15.67:0/146610681 shutdown_connections 2024-07-31T14:58:55.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.196+0000 7f695ce6d740 1 -- 172.21.15.67:0/146610681 wait complete. 2024-07-31T14:58:55.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.197+0000 7f695ce6d740 1 -- start start 2024-07-31T14:58:55.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.197+0000 7f695ce6d740 1 Processor -- start 2024-07-31T14:58:55.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.201+0000 7f695ce6d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cae980 0x562a1808d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.201+0000 7f695ce6d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cd7840 0x562a1808d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.201+0000 7f695ce6d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17c6b570 0x562a18113df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.201+0000 7f695ce6d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f693c0113f0 con 0x562a17c6b570 2024-07-31T14:58:55.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.201+0000 7f695ce6d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f693c010c60 con 0x562a17cd7840 2024-07-31T14:58:55.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.201+0000 7f695ce6d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f693c00de30 con 0x562a17cae980 2024-07-31T14:58:55.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.201+0000 7f695ce6d740 10 client.0 _ll_register_callbacks cb 0x7f69440aa110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:58:55.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.202+0000 7f694f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cd7840 0x562a1808d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:55.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.202+0000 7f694f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cd7840 0x562a1808d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51488/0 (socket says 172.21.15.67:51488) 2024-07-31T14:58:55.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.202+0000 7f694f7fe640 1 -- 172.21.15.67:0/4206318261 learned_addr learned my addr 172.21.15.67:0/4206318261 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:58:55.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.202+0000 7f694ffff640 1 --2- 172.21.15.67:0/4206318261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cae980 0x562a1808d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:55.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.205+0000 7f6954eb9640 1 --2- 172.21.15.67:0/4206318261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17c6b570 0x562a18113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:55.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.205+0000 7f694f7fe640 1 -- 172.21.15.67:0/4206318261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cae980 msgr2=0x562a1808d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:55.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.205+0000 7f694f7fe640 1 --2- 172.21.15.67:0/4206318261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cae980 0x562a1808d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.205+0000 7f694f7fe640 1 -- 172.21.15.67:0/4206318261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17c6b570 msgr2=0x562a18113df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:58:55.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.205+0000 7f694f7fe640 1 --2- 172.21.15.67:0/4206318261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17c6b570 0x562a18113df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:55.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.205+0000 7f694f7fe640 1 -- 172.21.15.67:0/4206318261 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a1808e150 con 0x562a17cd7840 2024-07-31T14:58:55.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.208+0000 7f694f7fe640 1 --2- 172.21.15.67:0/4206318261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cd7840 0x562a1808d960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f69440a9a70 tx=0x7f69440c5410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab96b996ba8c6633 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:58:55.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.208+0000 7f692f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:58:55.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.209+0000 7f692f7fe640 1 -- 172.21.15.67:0/4206318261 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69440c5040 con 0x562a17cd7840 2024-07-31T14:58:55.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.209+0000 7f692f7fe640 1 -- 172.21.15.67:0/4206318261 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7f69440a6310 con 0x562a17cd7840 2024-07-31T14:58:55.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.210+0000 7f695ce6d740 1 -- 172.21.15.67:0/4206318261 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69440b04d0 con 0x562a17cd7840 2024-07-31T14:58:55.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.211+0000 7f692f7fe640 1 -- 172.21.15.67:0/4206318261 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f69440b3a50 con 0x562a17cd7840 2024-07-31T14:58:55.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.212+0000 7f695ce6d740 10 client.27041 fetch_fsmap learned FSMap version 54 2024-07-31T14:58:55.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.212+0000 7f695ce6d740 1 -- 172.21.15.67:0/4206318261 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562a1808e150 con 0x562a17cd7840 2024-07-31T14:58:55.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.217+0000 7f692f7fe640 1 -- 172.21.15.67:0/4206318261 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69440b04d0 con 0x562a17cd7840 2024-07-31T14:58:55.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.217+0000 7f692f7fe640 1 -- 172.21.15.67:0/4206318261 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f69440ba070 con 0x562a17cd7840 2024-07-31T14:58:55.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.218+0000 7f695ce6d740 10 client.27041 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:58:55.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.218+0000 7f695ce6d740 10 client.27041 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:58:55.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.218+0000 7f695ce6d740 20 client.27041 populate_metadata read hostname 'smithi067' 2024-07-31T14:58:55.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.218+0000 7f695ce6d740 1 --2- 172.21.15.67:0/4206318261 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562a18115660 0x562a18117a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:58:55.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.218+0000 7f695ce6d740 4 client.27041 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-07-31T14:58:55.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.218+0000 7f695ce6d740 1 -- 172.21.15.67:0/4206318261 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x562a17ff0170 con 0x562a18115660 2024-07-31T14:58:55.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.223+0000 7f694ffff640 1 --2- 172.21.15.67:0/4206318261 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562a18115660 0x562a18117a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:58:55.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.231+0000 7f694ffff640 1 --2- 172.21.15.67:0/4206318261 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562a18115660 0x562a18117a80 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fb9f24072b8c2fe server_cookie=9d88d2590d330782 in_seq=0 out_seq=0 2024-07-31T14:58:55.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:58:55.231+0000 7f692f7fe640 10 client.27041 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:54 smithi077 ceph-mon[37327]: pgmap v638: 129 pgs: 129 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 218 KiB/s wr, 1.44k op/s 2024-07-31T14:58:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:54 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:58:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:54 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:58:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:54 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:58:55.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:54 smithi073 ceph-mon[36714]: pgmap v638: 129 pgs: 129 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 218 KiB/s wr, 1.44k op/s 2024-07-31T14:58:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:54 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T14:58:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:54 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T14:58:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:54 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:58:57.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:56 smithi077 ceph-mon[37327]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 1.19k op/s 2024-07-31T14:58:57.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:56 smithi067 ceph-mon[32750]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 1.19k op/s 2024-07-31T14:58:57.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:56 smithi073 ceph-mon[36714]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 1.19k op/s 2024-07-31T14:58:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.493+0000 7fe8e27fc640 1 -- 172.21.15.67:0/3178554448 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56510839ada0 con 0x5651080191c0 2024-07-31T14:58:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.493+0000 7fe8e27fc640 10 client.17472 handle_command_reply: tid=0 2024-07-31T14:58:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.493+0000 7fe908464740 1 client.17472 shutdown 2024-07-31T14:58:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.493+0000 7fe908464740 1 -- 172.21.15.67:0/3178554448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080851b0 msgr2=0x5651084abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.493+0000 7fe908464740 1 --2- 172.21.15.67:0/3178554448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080851b0 0x5651084abe90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4005e30 tx=0x7fe8f402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:58:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.493+0000 7fe908464740 1 -- 172.21.15.67:0/3178554448 shutdown_connections 2024-07-31T14:58:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.493+0000 7fe908464740 1 --2- 172.21.15.67:0/3178554448 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5651080191c0 0x5651084af300 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.493+0000 7fe908464740 1 --2- 172.21.15.67:0/3178554448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651080851b0 0x5651084abe90 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.493+0000 7fe908464740 1 --2- 172.21.15.67:0/3178554448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651080644d0 0x565108438c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.493+0000 7fe908464740 1 --2- 172.21.15.67:0/3178554448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56510805c2f0 0x5651084386f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.493+0000 7fe8e27fc640 0 client.17472 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:58:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.493+0000 7fe908464740 1 -- 172.21.15.67:0/3178554448 >> 172.21.15.67:0/3178554448 conn(0x565108079300 msgr2=0x565108434a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe908464740 1 -- 172.21.15.67:0/3178554448 shutdown_connections 2024-07-31T14:58:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe908464740 1 -- 172.21.15.67:0/3178554448 wait complete. 2024-07-31T14:58:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe908464740 20 client.17472 trim_cache size 0 max 16384 2024-07-31T14:58:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:58:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:58:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:58:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:58:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe901570640 1 -- 172.21.15.67:0/1291605956 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe8e80b3d40 msgr2=0x7fe8e80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe901570640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe8e80b3d40 0x7fe8e80b6250 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0e53f0 tx=0x7fe8ec009f60 comp rx=0 tx=0).stop 2024-07-31T14:58:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe901570640 1 -- 172.21.15.67:0/1291605956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8fc102410 msgr2=0x7fe8fc19ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:58:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe901570640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8fc102410 0x7fe8fc19ac50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009d20 tx=0x7fe8ec00c6a0 comp rx=0 tx=0).stop 2024-07-31T14:58:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe901570640 1 -- 172.21.15.67:0/1291605956 shutdown_connections 2024-07-31T14:58:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe901570640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe8e80b3d40 0x7fe8e80b6250 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe901570640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8fc108690 0x7fe8fc19b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe901570640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8fc107d40 0x7fe8fc19b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe901570640 1 --2- 172.21.15.67:0/1291605956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8fc102410 0x7fe8fc19ac50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:58:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe901570640 1 -- 172.21.15.67:0/1291605956 >> 172.21.15.67:0/1291605956 conn(0x7fe8fc07a140 msgr2=0x7fe8fc103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:58:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe901570640 1 -- 172.21.15.67:0/1291605956 shutdown_connections 2024-07-31T14:58:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:58:57.494+0000 7fe901570640 1 -- 172.21.15.67:0/1291605956 wait complete. 2024-07-31T14:58:59.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:58 smithi077 ceph-mon[37327]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.9 MiB/s wr, 821 op/s 2024-07-31T14:58:59.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:58 smithi067 ceph-mon[32750]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.9 MiB/s wr, 821 op/s 2024-07-31T14:58:59.344 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:58:59.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:58 smithi073 ceph-mon[36714]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.9 MiB/s wr, 821 op/s 2024-07-31T14:58:59.628 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:59:00.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:59 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:59:00.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:59 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:59:00.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:59 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:59:00.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:58:59 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:59:00.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:59 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:59:00.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:59 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:59:00.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:59 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:59:00.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:58:59 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:59:00.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:59 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:59:00.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:59 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:59:00.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:59 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:59:00.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:58:59 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:59:01.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:00 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T14:59:01.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:00 smithi067 ceph-mon[32750]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 2.7 MiB/s wr, 685 op/s 2024-07-31T14:59:01.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.371+0000 7f401162e640 1 -- 172.21.15.67:0/2064901991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c1087b0 msgr2=0x7f400c10efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:01.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.371+0000 7f401162e640 1 --2- 172.21.15.67:0/2064901991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c1087b0 0x7f400c10efe0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4004009ca0 tx=0x7f400402a8c0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.371+0000 7f401162e640 1 -- 172.21.15.67:0/2064901991 shutdown_connections 2024-07-31T14:59:01.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.371+0000 7f401162e640 1 --2- 172.21.15.67:0/2064901991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c1087b0 0x7f400c10efe0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.371+0000 7f401162e640 1 --2- 172.21.15.67:0/2064901991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f400c107e60 0x7f400c108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.371+0000 7f401162e640 1 --2- 172.21.15.67:0/2064901991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f400c102520 0x7f400c102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.371+0000 7f401162e640 1 -- 172.21.15.67:0/2064901991 >> 172.21.15.67:0/2064901991 conn(0x7f400c0791a0 msgr2=0x7f400c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:01.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.372+0000 7f401162e640 1 -- 172.21.15.67:0/2064901991 shutdown_connections 2024-07-31T14:59:01.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.372+0000 7f401162e640 1 -- 172.21.15.67:0/2064901991 wait complete. 2024-07-31T14:59:01.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.372+0000 7f401162e640 1 Processor -- start 2024-07-31T14:59:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f401162e640 1 -- start start 2024-07-31T14:59:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f401162e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c102520 0x7f400c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f401162e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f400c107e60 0x7f400c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f401162e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f400c1087b0 0x7f400c19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f401162e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f400c112480 con 0x7f400c102520 2024-07-31T14:59:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f401162e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f400c112300 con 0x7f400c107e60 2024-07-31T14:59:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f401162e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f400c112600 con 0x7f400c1087b0 2024-07-31T14:59:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f400affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c102520 0x7f400c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f400affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c102520 0x7f400c198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48516/0 (socket says 172.21.15.67:48516) 2024-07-31T14:59:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f400affd640 1 -- 172.21.15.67:0/4030055365 learned_addr learned my addr 172.21.15.67:0/4030055365 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f400b7fe640 1 --2- 172.21.15.67:0/4030055365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f400c1087b0 0x7f400c19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f400a7fc640 1 --2- 172.21.15.67:0/4030055365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f400c107e60 0x7f400c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f400affd640 1 -- 172.21.15.67:0/4030055365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f400c1087b0 msgr2=0x7f400c19e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f400affd640 1 --2- 172.21.15.67:0/4030055365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f400c1087b0 0x7f400c19e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f400affd640 1 -- 172.21.15.67:0/4030055365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f400c107e60 msgr2=0x7f400c199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.373+0000 7f400affd640 1 --2- 172.21.15.67:0/4030055365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f400c107e60 0x7f400c199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.374+0000 7f400affd640 1 -- 172.21.15.67:0/4030055365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f400c19ed20 con 0x7f400c102520 2024-07-31T14:59:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.374+0000 7f400affd640 1 --2- 172.21.15.67:0/4030055365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c102520 0x7f400c198db0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009900 tx=0x7f3ffc009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.374+0000 7f3ff3fff640 1 -- 172.21.15.67:0/4030055365 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc00d030 con 0x7f400c102520 2024-07-31T14:59:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.374+0000 7f3ff3fff640 1 -- 172.21.15.67:0/4030055365 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ffc00d1d0 con 0x7f400c102520 2024-07-31T14:59:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.374+0000 7f401162e640 1 -- 172.21.15.67:0/4030055365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f400c19f010 con 0x7f400c102520 2024-07-31T14:59:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.374+0000 7f3ff3fff640 1 -- 172.21.15.67:0/4030055365 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc005280 con 0x7f400c102520 2024-07-31T14:59:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.374+0000 7f401162e640 1 -- 172.21.15.67:0/4030055365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f400c06b590 con 0x7f400c102520 2024-07-31T14:59:01.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.376+0000 7f4017b7f740 1 Processor -- start 2024-07-31T14:59:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.377+0000 7f4017b7f740 1 -- start start 2024-07-31T14:59:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.377+0000 7f3ff3fff640 1 -- 172.21.15.67:0/4030055365 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3ffc005420 con 0x7f400c102520 2024-07-31T14:59:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.377+0000 7f4017b7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e7c070 0x557f39221520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.377+0000 7f4017b7f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e531b0 0x557f39221a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.377+0000 7f4017b7f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e5b390 0x557f39221fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.377+0000 7f4017b7f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557f39197b60 con 0x557f38e5b390 2024-07-31T14:59:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.377+0000 7f4017b7f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557f39187060 con 0x557f38e531b0 2024-07-31T14:59:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.377+0000 7f4017b7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557f3918b0a0 con 0x557f38e7c070 2024-07-31T14:59:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.377+0000 7f400a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e5b390 0x557f39221fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.377+0000 7f400b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e7c070 0x557f39221520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.377+0000 7f400a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e5b390 0x557f39221fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48520/0 (socket says 172.21.15.67:48520) 2024-07-31T14:59:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.377+0000 7f400b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e7c070 0x557f39221520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35888/0 (socket says 172.21.15.67:35888) 2024-07-31T14:59:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.377+0000 7f400b7fe640 1 -- 172.21.15.67:0/2725618280 learned_addr learned my addr 172.21.15.67:0/2725618280 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.377+0000 7f400affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e531b0 0x557f39221a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.378+0000 7f400b7fe640 1 -- 172.21.15.67:0/2725618280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e531b0 msgr2=0x557f39221a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.378+0000 7f400b7fe640 1 --2- 172.21.15.67:0/2725618280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e531b0 0x557f39221a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.378+0000 7f400b7fe640 1 -- 172.21.15.67:0/2725618280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e5b390 msgr2=0x557f39221fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.378+0000 7f400b7fe640 1 --2- 172.21.15.67:0/2725618280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e5b390 0x557f39221fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.378+0000 7f400b7fe640 1 -- 172.21.15.67:0/2725618280 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f3912db40 con 0x557f38e7c070 2024-07-31T14:59:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.378+0000 7f400a7fc640 1 --2- 172.21.15.67:0/2725618280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e5b390 0x557f39221fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:59:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.378+0000 7f3ff3fff640 1 --2- 172.21.15.67:0/4030055365 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3ff40b3d40 0x7f3ff40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.378+0000 7f400affd640 1 --2- 172.21.15.67:0/4030055365 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3ff40b3d40 0x7f3ff40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.378+0000 7f3ff3fff640 1 -- 172.21.15.67:0/4030055365 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7f3ffc0e56a0 con 0x7f400c102520 2024-07-31T14:59:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.378+0000 7f400b7fe640 1 --2- 172.21.15.67:0/2725618280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e7c070 0x557f39221520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4004009920 tx=0x7f4004009950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69f12a50885f21af server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:01.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.378+0000 7f3ff17fa640 1 -- 172.21.15.67:0/2725618280 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004031e30 con 0x557f38e7c070 2024-07-31T14:59:01.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.378+0000 7f3ff17fa640 1 -- 172.21.15.67:0/2725618280 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4004036030 con 0x557f38e7c070 2024-07-31T14:59:01.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.378+0000 7f3ff17fa640 1 -- 172.21.15.67:0/2725618280 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40040042a0 con 0x557f38e7c070 2024-07-31T14:59:01.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.379+0000 7f4017b7f740 1 -- 172.21.15.67:0/2725618280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e7c070 msgr2=0x557f39221520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:01.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.379+0000 7f400affd640 1 --2- 172.21.15.67:0/4030055365 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3ff40b3d40 0x7f3ff40b6250 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc0e2b60 tx=0x7f3ffc0eb550 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:01.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.379+0000 7f4017b7f740 1 --2- 172.21.15.67:0/2725618280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e7c070 0x557f39221520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4004009920 tx=0x7f4004009950 comp rx=0 tx=0).stop 2024-07-31T14:59:01.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.379+0000 7f4017b7f740 1 -- 172.21.15.67:0/2725618280 shutdown_connections 2024-07-31T14:59:01.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.379+0000 7f4017b7f740 1 --2- 172.21.15.67:0/2725618280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e5b390 0x557f39221fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.379+0000 7f4017b7f740 1 --2- 172.21.15.67:0/2725618280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e531b0 0x557f39221a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.379+0000 7f4017b7f740 1 --2- 172.21.15.67:0/2725618280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e7c070 0x557f39221520 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.379+0000 7f4017b7f740 1 -- 172.21.15.67:0/2725618280 >> 172.21.15.67:0/2725618280 conn(0x557f38e70460 msgr2=0x557f3921f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:01.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.379+0000 7f4017b7f740 1 -- 172.21.15.67:0/2725618280 shutdown_connections 2024-07-31T14:59:01.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.379+0000 7f4017b7f740 1 -- 172.21.15.67:0/2725618280 wait complete. 2024-07-31T14:59:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.380+0000 7f4017b7f740 1 -- start start 2024-07-31T14:59:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.380+0000 7f4017b7f740 1 Processor -- start 2024-07-31T14:59:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.380+0000 7f4017b7f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e531b0 0x557f3922d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.380+0000 7f4017b7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e5b390 0x557f3922d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.380+0000 7f4017b7f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e7c070 0x557f39238e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.380+0000 7f4017b7f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557f38fea700 con 0x557f38e7c070 2024-07-31T14:59:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.380+0000 7f4017b7f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557f39129990 con 0x557f38e531b0 2024-07-31T14:59:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.380+0000 7f4017b7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557f3922fb20 con 0x557f38e5b390 2024-07-31T14:59:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.380+0000 7f4017b7f740 10 client.0 _ll_register_callbacks cb 0x557f391444a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.380+0000 7f400b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e7c070 0x557f39238e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.380+0000 7f400b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e7c070 0x557f39238e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48530/0 (socket says 172.21.15.67:48530) 2024-07-31T14:59:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.380+0000 7f400b7fe640 1 -- 172.21.15.67:0/891782780 learned_addr learned my addr 172.21.15.67:0/891782780 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.380+0000 7f400a7fc640 1 --2- 172.21.15.67:0/891782780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e5b390 0x557f3922d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.380+0000 7f400b7fe640 1 --2- 172.21.15.67:0/891782780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e531b0 0x557f3922d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.381+0000 7f400b7fe640 1 -- 172.21.15.67:0/891782780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e5b390 msgr2=0x557f3922d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.381+0000 7f400b7fe640 1 --2- 172.21.15.67:0/891782780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e5b390 0x557f3922d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.381+0000 7f400b7fe640 1 -- 172.21.15.67:0/891782780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e531b0 msgr2=0x557f3922d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.381+0000 7f400b7fe640 1 --2- 172.21.15.67:0/891782780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e531b0 0x557f3922d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.381+0000 7f400b7fe640 1 -- 172.21.15.67:0/891782780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f38ee7650 con 0x557f38e7c070 2024-07-31T14:59:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.381+0000 7f400b7fe640 1 --2- 172.21.15.67:0/891782780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e7c070 0x557f39238e10 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f40040096c0 tx=0x7f4004003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0cbb001561161c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.381+0000 7f3feeff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:59:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.381+0000 7f3feeff5640 1 -- 172.21.15.67:0/891782780 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004059060 con 0x557f38e7c070 2024-07-31T14:59:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.381+0000 7f4017b7f740 1 -- 172.21.15.67:0/891782780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4004036030 con 0x557f38e7c070 2024-07-31T14:59:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.381+0000 7f3feeff5640 1 -- 172.21.15.67:0/891782780 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7f4004004700 con 0x557f38e7c070 2024-07-31T14:59:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.381+0000 7f3feeff5640 1 -- 172.21.15.67:0/891782780 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f4004004b40 con 0x557f38e7c070 2024-07-31T14:59:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.381+0000 7f4017b7f740 10 client.17499 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.381+0000 7f4017b7f740 1 -- 172.21.15.67:0/891782780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x557f3923bb10 con 0x557f38e7c070 2024-07-31T14:59:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.382+0000 7f3feeff5640 1 -- 172.21.15.67:0/891782780 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4004036030 con 0x557f38e7c070 2024-07-31T14:59:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.382+0000 7f3feeff5640 1 -- 172.21.15.67:0/891782780 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f4004043070 con 0x557f38e7c070 2024-07-31T14:59:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.382+0000 7f4017b7f740 10 client.17499 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.382+0000 7f4017b7f740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.382+0000 7f4017b7f740 20 client.17499 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.382+0000 7f4017b7f740 1 --2- 172.21.15.67:0/891782780 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f38e3a9f0 0x557f3923ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.382+0000 7f4017b7f740 4 client.17499 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:59:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.382+0000 7f4017b7f740 1 -- 172.21.15.67:0/891782780 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f39199ab0 con 0x557f38e3a9f0 2024-07-31T14:59:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.382+0000 7f400a7fc640 1 --2- 172.21.15.67:0/891782780 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f38e3a9f0 0x557f3923ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.382+0000 7f400a7fc640 1 --2- 172.21.15.67:0/891782780 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f38e3a9f0 0x557f3923ee70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cc848fd4f413e24 server_cookie=61fb5c6bf2f56652 in_seq=0 out_seq=0 2024-07-31T14:59:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.382+0000 7f3feeff5640 10 client.17499 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:01.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:00 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T14:59:01.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:00 smithi073 ceph-mon[36714]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 2.7 MiB/s wr, 685 op/s 2024-07-31T14:59:01.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:00 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T14:59:01.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:00 smithi077 ceph-mon[37327]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 2.7 MiB/s wr, 685 op/s 2024-07-31T14:59:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.699+0000 7f3feeff5640 1 -- 172.21.15.67:0/891782780 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f39199ab0 con 0x557f38e3a9f0 2024-07-31T14:59:01.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.699+0000 7f3feeff5640 10 client.17499 handle_command_reply: tid=0 2024-07-31T14:59:01.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.699+0000 7f4017b7f740 1 client.17499 shutdown 2024-07-31T14:59:01.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.700+0000 7f4017b7f740 1 -- 172.21.15.67:0/891782780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e7c070 msgr2=0x557f39238e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:01.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.700+0000 7f4017b7f740 1 --2- 172.21.15.67:0/891782780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e7c070 0x557f39238e10 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f40040096c0 tx=0x7f4004003e50 comp rx=0 tx=0).stop 2024-07-31T14:59:01.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.700+0000 7f4017b7f740 1 -- 172.21.15.67:0/891782780 shutdown_connections 2024-07-31T14:59:01.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.700+0000 7f4017b7f740 1 --2- 172.21.15.67:0/891782780 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f38e3a9f0 0x557f3923ee70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.700+0000 7f4017b7f740 1 --2- 172.21.15.67:0/891782780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e7c070 0x557f39238e10 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.700+0000 7f4017b7f740 1 --2- 172.21.15.67:0/891782780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e5b390 0x557f3922d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.700+0000 7f4017b7f740 1 --2- 172.21.15.67:0/891782780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e531b0 0x557f3922d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.700+0000 7f3feeff5640 0 client.17499 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:01.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.700+0000 7f4017b7f740 1 -- 172.21.15.67:0/891782780 >> 172.21.15.67:0/891782780 conn(0x557f38e70460 msgr2=0x557f39224700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:01.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.700+0000 7f4017b7f740 1 -- 172.21.15.67:0/891782780 shutdown_connections 2024-07-31T14:59:01.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.700+0000 7f4017b7f740 1 -- 172.21.15.67:0/891782780 wait complete. 2024-07-31T14:59:01.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.700+0000 7f4017b7f740 20 client.17499 trim_cache size 0 max 16384 2024-07-31T14:59:01.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.719+0000 7f4017b7f740 1 Processor -- start 2024-07-31T14:59:01.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.719+0000 7f4017b7f740 1 -- start start 2024-07-31T14:59:01.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.719+0000 7f4017b7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e3a9f0 0x557f39227640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.719+0000 7f4017b7f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e531b0 0x557f39238c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.719+0000 7f4017b7f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e5b390 0x557f3921e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.719+0000 7f4017b7f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1ae4e0 con 0x557f38e531b0 2024-07-31T14:59:01.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.719+0000 7f4017b7f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1b0170 con 0x557f38e5b390 2024-07-31T14:59:01.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.719+0000 7f4017b7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1aebb0 con 0x557f38e3a9f0 2024-07-31T14:59:01.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.719+0000 7f400a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e531b0 0x557f39238c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.720+0000 7f400a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e531b0 0x557f39238c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48544/0 (socket says 172.21.15.67:48544) 2024-07-31T14:59:01.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.720+0000 7f400a7fc640 1 -- 172.21.15.67:0/173593371 learned_addr learned my addr 172.21.15.67:0/173593371 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:01.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.720+0000 7f400b7fe640 1 --2- 172.21.15.67:0/173593371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e5b390 0x557f3921e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.720+0000 7f400b7fe640 1 --2- 172.21.15.67:0/173593371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e3a9f0 0x557f39227640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.720+0000 7f400a7fc640 1 -- 172.21.15.67:0/173593371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e3a9f0 msgr2=0x557f39227640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:01.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.720+0000 7f400a7fc640 1 --2- 172.21.15.67:0/173593371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e3a9f0 0x557f39227640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.720+0000 7f400a7fc640 1 -- 172.21.15.67:0/173593371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e5b390 msgr2=0x557f3921e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:01.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.720+0000 7f400a7fc640 1 --2- 172.21.15.67:0/173593371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e5b390 0x557f3921e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.720+0000 7f400a7fc640 1 -- 172.21.15.67:0/173593371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f39223810 con 0x557f38e531b0 2024-07-31T14:59:01.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.720+0000 7f400a7fc640 1 --2- 172.21.15.67:0/173593371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e531b0 0x557f39238c00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f400c06b080 tx=0x7f400000a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befe18183bca51af server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:01.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.720+0000 7f3ff0ff9640 1 -- 172.21.15.67:0/173593371 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000004220 con 0x557f38e531b0 2024-07-31T14:59:01.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.720+0000 7f3ff0ff9640 1 -- 172.21.15.67:0/173593371 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40000043c0 con 0x557f38e531b0 2024-07-31T14:59:01.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.720+0000 7f3ff0ff9640 1 -- 172.21.15.67:0/173593371 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40000035f0 con 0x557f38e531b0 2024-07-31T14:59:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.721+0000 7f4017b7f740 1 -- 172.21.15.67:0/173593371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e531b0 msgr2=0x557f39238c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.721+0000 7f4017b7f740 1 --2- 172.21.15.67:0/173593371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e531b0 0x557f39238c00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f400c06b080 tx=0x7f400000a250 comp rx=0 tx=0).stop 2024-07-31T14:59:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.721+0000 7f4017b7f740 1 -- 172.21.15.67:0/173593371 shutdown_connections 2024-07-31T14:59:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.721+0000 7f4017b7f740 1 --2- 172.21.15.67:0/173593371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e5b390 0x557f3921e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.721+0000 7f4017b7f740 1 --2- 172.21.15.67:0/173593371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e531b0 0x557f39238c00 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.721+0000 7f4017b7f740 1 --2- 172.21.15.67:0/173593371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e3a9f0 0x557f39227640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.721+0000 7f4017b7f740 1 -- 172.21.15.67:0/173593371 >> 172.21.15.67:0/173593371 conn(0x557f38e70460 msgr2=0x557f3921dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.721+0000 7f4017b7f740 1 -- 172.21.15.67:0/173593371 shutdown_connections 2024-07-31T14:59:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.721+0000 7f4017b7f740 1 -- 172.21.15.67:0/173593371 wait complete. 2024-07-31T14:59:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.722+0000 7f4017b7f740 1 -- start start 2024-07-31T14:59:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.722+0000 7f4017b7f740 1 Processor -- start 2024-07-31T14:59:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.722+0000 7f4017b7f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e3a9f0 0x557f39235190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.722+0000 7f4017b7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e531b0 0x557f39230640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.722+0000 7f4017b7f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e5b390 0x557f39230b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.722+0000 7f4017b7f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1afaa0 con 0x557f38e3a9f0 2024-07-31T14:59:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.722+0000 7f4017b7f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1af310 con 0x557f38e5b390 2024-07-31T14:59:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.722+0000 7f4017b7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1ac4e0 con 0x557f38e531b0 2024-07-31T14:59:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.722+0000 7f4017b7f740 10 client.0 _ll_register_callbacks cb 0x7f400000a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.722+0000 7f400b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e3a9f0 0x557f39235190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.722+0000 7f400b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e3a9f0 0x557f39235190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48546/0 (socket says 172.21.15.67:48546) 2024-07-31T14:59:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.722+0000 7f400b7fe640 1 -- 172.21.15.67:0/2726813794 learned_addr learned my addr 172.21.15.67:0/2726813794 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.722+0000 7f400a7fc640 1 --2- 172.21.15.67:0/2726813794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e531b0 0x557f39230640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.722+0000 7f400b7fe640 1 --2- 172.21.15.67:0/2726813794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e5b390 0x557f39230b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.723+0000 7f400b7fe640 1 -- 172.21.15.67:0/2726813794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e531b0 msgr2=0x557f39230640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.723+0000 7f400b7fe640 1 --2- 172.21.15.67:0/2726813794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e531b0 0x557f39230640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.723+0000 7f400b7fe640 1 -- 172.21.15.67:0/2726813794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e5b390 msgr2=0x557f39230b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.723+0000 7f400b7fe640 1 --2- 172.21.15.67:0/2726813794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e5b390 0x557f39230b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.723+0000 7f400b7fe640 1 -- 172.21.15.67:0/2726813794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f392310c0 con 0x557f38e3a9f0 2024-07-31T14:59:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.723+0000 7f400b7fe640 1 --2- 172.21.15.67:0/2726813794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e3a9f0 0x557f39235190 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f400402ade0 tx=0x7f4004003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0162a633e1cb205 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.723+0000 7f3feeff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:59:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.723+0000 7f3feeff5640 1 -- 172.21.15.67:0/2726813794 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004059040 con 0x557f38e3a9f0 2024-07-31T14:59:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.723+0000 7f3feeff5640 1 -- 172.21.15.67:0/2726813794 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7f4004004480 con 0x557f38e3a9f0 2024-07-31T14:59:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.723+0000 7f4017b7f740 1 -- 172.21.15.67:0/2726813794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40000043c0 con 0x557f38e3a9f0 2024-07-31T14:59:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.723+0000 7f3feeff5640 1 -- 172.21.15.67:0/2726813794 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f4004004680 con 0x557f38e3a9f0 2024-07-31T14:59:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.723+0000 7f4017b7f740 10 client.17505 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.723+0000 7f4017b7f740 1 -- 172.21.15.67:0/2726813794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x557f392a7140 con 0x557f38e3a9f0 2024-07-31T14:59:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.724+0000 7f3feeff5640 1 -- 172.21.15.67:0/2726813794 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40000043c0 con 0x557f38e3a9f0 2024-07-31T14:59:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.724+0000 7f3feeff5640 1 -- 172.21.15.67:0/2726813794 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f4004043070 con 0x557f38e3a9f0 2024-07-31T14:59:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.724+0000 7f4017b7f740 10 client.17505 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.724+0000 7f4017b7f740 10 client.17505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.724+0000 7f4017b7f740 20 client.17505 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.724+0000 7f4017b7f740 1 --2- 172.21.15.67:0/2726813794 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f38e7c070 0x557f392a9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.724+0000 7f4017b7f740 4 client.17505 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:59:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.724+0000 7f4017b7f740 1 -- 172.21.15.67:0/2726813794 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557f39195550 con 0x557f38e7c070 2024-07-31T14:59:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.724+0000 7f400a7fc640 1 --2- 172.21.15.67:0/2726813794 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f38e7c070 0x557f392a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.725+0000 7f400a7fc640 1 --2- 172.21.15.67:0/2726813794 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f38e7c070 0x557f392a9880 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=151be67a6101a637 server_cookie=8cd6603a69381c6b in_seq=0 out_seq=0 2024-07-31T14:59:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:01.725+0000 7f3feeff5640 10 client.17505 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:03.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.062+0000 7f3feeff5640 1 -- 172.21.15.67:0/2726813794 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557f39195550 con 0x557f38e7c070 2024-07-31T14:59:03.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.062+0000 7f3feeff5640 10 client.17505 handle_command_reply: tid=0 2024-07-31T14:59:03.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.062+0000 7f4017b7f740 1 client.17505 shutdown 2024-07-31T14:59:03.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.063+0000 7f4017b7f740 1 -- 172.21.15.67:0/2726813794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e3a9f0 msgr2=0x557f39235190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:03.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.063+0000 7f4017b7f740 1 --2- 172.21.15.67:0/2726813794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e3a9f0 0x557f39235190 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f400402ade0 tx=0x7f4004003db0 comp rx=0 tx=0).stop 2024-07-31T14:59:03.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.063+0000 7f4017b7f740 1 -- 172.21.15.67:0/2726813794 shutdown_connections 2024-07-31T14:59:03.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.063+0000 7f4017b7f740 1 --2- 172.21.15.67:0/2726813794 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f38e7c070 0x557f392a9880 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:03.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.063+0000 7f4017b7f740 1 --2- 172.21.15.67:0/2726813794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f38e5b390 0x557f39230b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:03.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.063+0000 7f4017b7f740 1 --2- 172.21.15.67:0/2726813794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f38e531b0 0x557f39230640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:03.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.063+0000 7f4017b7f740 1 --2- 172.21.15.67:0/2726813794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f38e3a9f0 0x557f39235190 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:03.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.063+0000 7f3feeff5640 0 client.17505 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:03.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.063+0000 7f4017b7f740 1 -- 172.21.15.67:0/2726813794 >> 172.21.15.67:0/2726813794 conn(0x557f38e70460 msgr2=0x557f3922e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:03.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.063+0000 7f4017b7f740 1 -- 172.21.15.67:0/2726813794 shutdown_connections 2024-07-31T14:59:03.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.063+0000 7f4017b7f740 1 -- 172.21.15.67:0/2726813794 wait complete. 2024-07-31T14:59:03.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.063+0000 7f4017b7f740 20 client.17505 trim_cache size 0 max 16384 2024-07-31T14:59:03.066 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:59:03.066 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T14:59:03.066 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T14:59:03.066 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:59:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.064+0000 7f401162e640 1 -- 172.21.15.67:0/4030055365 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3ff40b3d40 msgr2=0x7f3ff40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.064+0000 7f401162e640 1 --2- 172.21.15.67:0/4030055365 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3ff40b3d40 0x7f3ff40b6250 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc0e2b60 tx=0x7f3ffc0eb550 comp rx=0 tx=0).stop 2024-07-31T14:59:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.064+0000 7f401162e640 1 -- 172.21.15.67:0/4030055365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c102520 msgr2=0x7f400c198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.064+0000 7f401162e640 1 --2- 172.21.15.67:0/4030055365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c102520 0x7f400c198db0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009900 tx=0x7f3ffc009dc0 comp rx=0 tx=0).stop 2024-07-31T14:59:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.064+0000 7f401162e640 1 -- 172.21.15.67:0/4030055365 shutdown_connections 2024-07-31T14:59:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.064+0000 7f401162e640 1 --2- 172.21.15.67:0/4030055365 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3ff40b3d40 0x7f3ff40b6250 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.064+0000 7f401162e640 1 --2- 172.21.15.67:0/4030055365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f400c1087b0 0x7f400c19e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.064+0000 7f401162e640 1 --2- 172.21.15.67:0/4030055365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f400c107e60 0x7f400c199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.064+0000 7f401162e640 1 --2- 172.21.15.67:0/4030055365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c102520 0x7f400c198db0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.064+0000 7f401162e640 1 -- 172.21.15.67:0/4030055365 >> 172.21.15.67:0/4030055365 conn(0x7f400c0791a0 msgr2=0x7f400c105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.064+0000 7f401162e640 1 -- 172.21.15.67:0/4030055365 shutdown_connections 2024-07-31T14:59:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:03.064+0000 7f401162e640 1 -- 172.21.15.67:0/4030055365 wait complete. 2024-07-31T14:59:03.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:02 smithi067 ceph-mon[32750]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 3.5 MiB/s wr, 610 op/s 2024-07-31T14:59:03.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:02 smithi073 ceph-mon[36714]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 3.5 MiB/s wr, 610 op/s 2024-07-31T14:59:03.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:02 smithi077 ceph-mon[37327]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 3.5 MiB/s wr, 610 op/s 2024-07-31T14:59:05.067 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T14:59:05.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:59:05.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:04 smithi067 ceph-mon[32750]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 597 op/s 2024-07-31T14:59:05.345 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:59:05.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:04 smithi073 ceph-mon[36714]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 597 op/s 2024-07-31T14:59:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:04 smithi077 ceph-mon[37327]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 597 op/s 2024-07-31T14:59:07.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:06 smithi067 ceph-mon[32750]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 3.4 MiB/s wr, 595 op/s 2024-07-31T14:59:07.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:06 smithi073 ceph-mon[36714]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 3.4 MiB/s wr, 595 op/s 2024-07-31T14:59:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:06 smithi077 ceph-mon[37327]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 3.4 MiB/s wr, 595 op/s 2024-07-31T14:59:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.128+0000 7f46c80d6640 1 -- 172.21.15.67:0/1216370126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46c010fe70 msgr2=0x7f46c01166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.128+0000 7f46c80d6640 1 --2- 172.21.15.67:0/1216370126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46c010fe70 0x7f46c01166a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009be0 tx=0x7f46b802a6f0 comp rx=0 tx=0).stop 2024-07-31T14:59:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.130+0000 7f46c80d6640 1 -- 172.21.15.67:0/1216370126 shutdown_connections 2024-07-31T14:59:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.130+0000 7f46c80d6640 1 --2- 172.21.15.67:0/1216370126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46c010fe70 0x7f46c01166a0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.130+0000 7f46c80d6640 1 --2- 172.21.15.67:0/1216370126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c010f520 0x7f46c010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.130+0000 7f46c80d6640 1 --2- 172.21.15.67:0/1216370126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c0109be0 0x7f46c0109ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.130+0000 7f46c80d6640 1 -- 172.21.15.67:0/1216370126 >> 172.21.15.67:0/1216370126 conn(0x7f46c0077b10 msgr2=0x7f46c0107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.132+0000 7f46c80d6640 1 -- 172.21.15.67:0/1216370126 shutdown_connections 2024-07-31T14:59:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.133+0000 7f46c80d6640 1 -- 172.21.15.67:0/1216370126 wait complete. 2024-07-31T14:59:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.134+0000 7f46c80d6640 1 Processor -- start 2024-07-31T14:59:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.134+0000 7f46c80d6640 1 -- start start 2024-07-31T14:59:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.135+0000 7f46c80d6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c0109be0 0x7f46c01a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.135+0000 7f46c80d6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c010f520 0x7f46c01a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.135+0000 7f46c80d6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46c010fe70 0x7f46c01a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:08.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.135+0000 7f46c80d6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f46c0119b30 con 0x7f46c010fe70 2024-07-31T14:59:08.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.135+0000 7f46c80d6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46c01199b0 con 0x7f46c010f520 2024-07-31T14:59:08.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.135+0000 7f46c80d6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46c0119cb0 con 0x7f46c0109be0 2024-07-31T14:59:08.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.137+0000 7f46c5e72640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c0109be0 0x7f46c01a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:08.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.137+0000 7f46c5e72640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c0109be0 0x7f46c01a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35950/0 (socket says 172.21.15.67:35950) 2024-07-31T14:59:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.137+0000 7f46c5e72640 1 -- 172.21.15.67:0/3073073218 learned_addr learned my addr 172.21.15.67:0/3073073218 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.138+0000 7f46c5e72640 1 -- 172.21.15.67:0/3073073218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c010f520 msgr2=0x7f46c01a0b00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T14:59:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.138+0000 7f46c5e72640 1 --2- 172.21.15.67:0/3073073218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c010f520 0x7f46c01a0b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.138+0000 7f46c5e72640 1 -- 172.21.15.67:0/3073073218 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46c010fe70 msgr2=0x7f46c01a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.138+0000 7f46c5e72640 1 --2- 172.21.15.67:0/3073073218 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46c010fe70 0x7f46c01a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.138+0000 7f46c5e72640 1 -- 172.21.15.67:0/3073073218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c01a64d0 con 0x7f46c0109be0 2024-07-31T14:59:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.140+0000 7f46c5e72640 1 --2- 172.21.15.67:0/3073073218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c0109be0 0x7f46c01a0560 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00d950 tx=0x7f46bc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.141+0000 7f46b6ffd640 1 -- 172.21.15.67:0/3073073218 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc017070 con 0x7f46c0109be0 2024-07-31T14:59:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.141+0000 7f46c80d6640 1 -- 172.21.15.67:0/3073073218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c01a67c0 con 0x7f46c0109be0 2024-07-31T14:59:08.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.141+0000 7f46c80d6640 1 -- 172.21.15.67:0/3073073218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c01acf20 con 0x7f46c0109be0 2024-07-31T14:59:08.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.141+0000 7f46b6ffd640 1 -- 172.21.15.67:0/3073073218 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46bc009c30 con 0x7f46c0109be0 2024-07-31T14:59:08.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.142+0000 7f46b6ffd640 1 -- 172.21.15.67:0/3073073218 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc00f620 con 0x7f46c0109be0 2024-07-31T14:59:08.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.146+0000 7f46c80d6640 1 -- 172.21.15.67:0/3073073218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a40052c0 con 0x7f46c0109be0 2024-07-31T14:59:08.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.146+0000 7f46b6ffd640 1 -- 172.21.15.67:0/3073073218 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f46bc010460 con 0x7f46c0109be0 2024-07-31T14:59:08.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.147+0000 7f46b6ffd640 1 --2- 172.21.15.67:0/3073073218 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46ac0b3ed0 0x7f46ac0b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:08.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.148+0000 7f46b6ffd640 1 -- 172.21.15.67:0/3073073218 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7f46bc0e3db0 con 0x7f46c0109be0 2024-07-31T14:59:08.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.152+0000 7f46c5671640 1 --2- 172.21.15.67:0/3073073218 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46ac0b3ed0 0x7f46ac0b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:08.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.152+0000 7f46b6ffd640 1 -- 172.21.15.67:0/3073073218 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f46bc0aad80 con 0x7f46c0109be0 2024-07-31T14:59:08.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.161+0000 7f46c5671640 1 --2- 172.21.15.67:0/3073073218 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46ac0b3ed0 0x7f46ac0b63e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f46c006ab10 tx=0x7f46b0009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:08.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.612+0000 7f46c80d6640 1 -- 172.21.15.67:0/3073073218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46a4005880 con 0x7f46c0109be0 2024-07-31T14:59:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.617+0000 7f46b6ffd640 1 -- 172.21.15.67:0/3073073218 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11971 (secure 0 0 0) 0x7f46bc009de0 con 0x7f46c0109be0 2024-07-31T14:59:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:59:08.620 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":54,"btime":"2024-07-31T14:58:48:655841+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:58:48.279201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:59:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 54 2024-07-31T14:59:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.622+0000 7f46c80d6640 1 -- 172.21.15.67:0/3073073218 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46ac0b3ed0 msgr2=0x7f46ac0b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.622+0000 7f46c80d6640 1 --2- 172.21.15.67:0/3073073218 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46ac0b3ed0 0x7f46ac0b63e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f46c006ab10 tx=0x7f46b0009260 comp rx=0 tx=0).stop 2024-07-31T14:59:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.622+0000 7f46c80d6640 1 -- 172.21.15.67:0/3073073218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c0109be0 msgr2=0x7f46c01a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.622+0000 7f46c80d6640 1 --2- 172.21.15.67:0/3073073218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c0109be0 0x7f46c01a0560 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00d950 tx=0x7f46bc00de10 comp rx=0 tx=0).stop 2024-07-31T14:59:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.622+0000 7f46c80d6640 1 -- 172.21.15.67:0/3073073218 shutdown_connections 2024-07-31T14:59:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.622+0000 7f46c80d6640 1 --2- 172.21.15.67:0/3073073218 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46ac0b3ed0 0x7f46ac0b63e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.622+0000 7f46c80d6640 1 --2- 172.21.15.67:0/3073073218 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46c010fe70 0x7f46c01a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.622+0000 7f46c80d6640 1 --2- 172.21.15.67:0/3073073218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c010f520 0x7f46c01a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.622+0000 7f46c80d6640 1 --2- 172.21.15.67:0/3073073218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46c0109be0 0x7f46c01a0560 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.622+0000 7f46c80d6640 1 -- 172.21.15.67:0/3073073218 >> 172.21.15.67:0/3073073218 conn(0x7f46c0077b10 msgr2=0x7f46c0107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.624+0000 7f46c80d6640 1 -- 172.21.15.67:0/3073073218 shutdown_connections 2024-07-31T14:59:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:08.624+0000 7f46c80d6640 1 -- 172.21.15.67:0/3073073218 wait complete. 2024-07-31T14:59:09.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:09 smithi067 ceph-mon[32750]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 2.9 MiB/s wr, 408 op/s 2024-07-31T14:59:09.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:09 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3073073218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:59:09.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:09 smithi073 ceph-mon[36714]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 2.9 MiB/s wr, 408 op/s 2024-07-31T14:59:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:09 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3073073218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:59:09.474 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T14:59:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:09 smithi077 ceph-mon[37327]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 2.9 MiB/s wr, 408 op/s 2024-07-31T14:59:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:09 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3073073218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:59:09.747 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:59:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.599+0000 7f0c956e3640 1 -- 172.21.15.67:0/3056514270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c90107d40 msgr2=0x7f0c90108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.599+0000 7f0c956e3640 1 --2- 172.21.15.67:0/3056514270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c90107d40 0x7f0c90108150 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009900 tx=0x7f0c7802a410 comp rx=0 tx=0).stop 2024-07-31T14:59:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.599+0000 7f0c956e3640 1 -- 172.21.15.67:0/3056514270 shutdown_connections 2024-07-31T14:59:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.599+0000 7f0c956e3640 1 --2- 172.21.15.67:0/3056514270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c90108690 0x7f0c9010eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.599+0000 7f0c956e3640 1 --2- 172.21.15.67:0/3056514270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c90107d40 0x7f0c90108150 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.599+0000 7f0c956e3640 1 --2- 172.21.15.67:0/3056514270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c90102410 0x7f0c90102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.599+0000 7f0c956e3640 1 -- 172.21.15.67:0/3056514270 >> 172.21.15.67:0/3056514270 conn(0x7f0c9007a140 msgr2=0x7f0c9007a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.599+0000 7f0c956e3640 1 -- 172.21.15.67:0/3056514270 shutdown_connections 2024-07-31T14:59:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.599+0000 7f0c956e3640 1 -- 172.21.15.67:0/3056514270 wait complete. 2024-07-31T14:59:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.600+0000 7f0c956e3640 1 Processor -- start 2024-07-31T14:59:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.600+0000 7f0c956e3640 1 -- start start 2024-07-31T14:59:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.600+0000 7f0c956e3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c90102410 0x7f0c90199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.600+0000 7f0c956e3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c90107d40 0x7f0c9019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.600+0000 7f0c956e3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c90108690 0x7f0c9019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.600+0000 7f0c956e3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0c90113470 con 0x7f0c90107d40 2024-07-31T14:59:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.600+0000 7f0c956e3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901132f0 con 0x7f0c90108690 2024-07-31T14:59:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.600+0000 7f0c956e3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901135f0 con 0x7f0c90102410 2024-07-31T14:59:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.600+0000 7f0c87fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c90107d40 0x7f0c9019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.600+0000 7f0c87fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c90107d40 0x7f0c9019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48592/0 (socket says 172.21.15.67:48592) 2024-07-31T14:59:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.600+0000 7f0c87fff640 1 -- 172.21.15.67:0/1572439515 learned_addr learned my addr 172.21.15.67:0/1572439515 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.600+0000 7f0c8f7fe640 1 --2- 172.21.15.67:0/1572439515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c90108690 0x7f0c9019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.600+0000 7f0c8effd640 1 --2- 172.21.15.67:0/1572439515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c90102410 0x7f0c90199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.601+0000 7f0c87fff640 1 -- 172.21.15.67:0/1572439515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c90102410 msgr2=0x7f0c90199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.601+0000 7f0c87fff640 1 --2- 172.21.15.67:0/1572439515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c90102410 0x7f0c90199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.601+0000 7f0c87fff640 1 -- 172.21.15.67:0/1572439515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c90108690 msgr2=0x7f0c9019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.601+0000 7f0c87fff640 1 --2- 172.21.15.67:0/1572439515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c90108690 0x7f0c9019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.601+0000 7f0c87fff640 1 -- 172.21.15.67:0/1572439515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9019fdc0 con 0x7f0c90107d40 2024-07-31T14:59:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.601+0000 7f0c87fff640 1 --2- 172.21.15.67:0/1572439515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c90107d40 0x7f0c9019a430 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0c78006010 tx=0x7f0c78004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.601+0000 7f0c8cff9640 1 -- 172.21.15.67:0/1572439515 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7803f030 con 0x7f0c90107d40 2024-07-31T14:59:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.601+0000 7f0c8cff9640 1 -- 172.21.15.67:0/1572439515 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c78003aa0 con 0x7f0c90107d40 2024-07-31T14:59:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.601+0000 7f0c8cff9640 1 -- 172.21.15.67:0/1572439515 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78035070 con 0x7f0c90107d40 2024-07-31T14:59:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.601+0000 7f0c956e3640 1 -- 172.21.15.67:0/1572439515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c901a0050 con 0x7f0c90107d40 2024-07-31T14:59:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.601+0000 7f0c956e3640 1 -- 172.21.15.67:0/1572439515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c901a6840 con 0x7f0c90107d40 2024-07-31T14:59:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.605+0000 7f0c9c5d3740 1 Processor -- start 2024-07-31T14:59:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.605+0000 7f0c8cff9640 1 -- 172.21.15.67:0/1572439515 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0c78042030 con 0x7f0c90107d40 2024-07-31T14:59:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.605+0000 7f0c9c5d3740 1 -- start start 2024-07-31T14:59:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.605+0000 7f0c9c5d3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1129201b0 0x55e112cc2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.605+0000 7f0c9c5d3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1128f72f0 0x55e112cc3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.605+0000 7f0c9c5d3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112cc3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.605+0000 7f0c9c5d3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e112c383b0 con 0x55e1128ff4d0 2024-07-31T14:59:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.605+0000 7f0c9c5d3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e112c278b0 con 0x55e1129201b0 2024-07-31T14:59:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.605+0000 7f0c9c5d3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e112c2b8f0 con 0x55e1128f72f0 2024-07-31T14:59:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.605+0000 7f0c8f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1129201b0 0x55e112cc2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.605+0000 7f0c87fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112cc3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.605+0000 7f0c8f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1129201b0 0x55e112cc2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35284/0 (socket says 172.21.15.67:35284) 2024-07-31T14:59:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.605+0000 7f0c8f7fe640 1 -- 172.21.15.67:0/2497025667 learned_addr learned my addr 172.21.15.67:0/2497025667 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.606+0000 7f0c8effd640 1 --2- 172.21.15.67:0/2497025667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1128f72f0 0x55e112cc3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.606+0000 7f0c8f7fe640 1 -- 172.21.15.67:0/2497025667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1128f72f0 msgr2=0x55e112cc3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.606+0000 7f0c8f7fe640 1 --2- 172.21.15.67:0/2497025667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1128f72f0 0x55e112cc3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.606+0000 7f0c8f7fe640 1 -- 172.21.15.67:0/2497025667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 msgr2=0x55e112cc3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.606+0000 7f0c8f7fe640 1 --2- 172.21.15.67:0/2497025667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112cc3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.606+0000 7f0c8f7fe640 1 -- 172.21.15.67:0/2497025667 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e112bce9a0 con 0x55e1129201b0 2024-07-31T14:59:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.606+0000 7f0c8cff9640 1 --2- 172.21.15.67:0/1572439515 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0c7c0b3dc0 0x7f0c7c0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.606+0000 7f0c8f7fe640 1 --2- 172.21.15.67:0/2497025667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1129201b0 0x55e112cc2ce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009fa0 tx=0x7f0c88010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bb8938856dbd9d2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.606+0000 7f0c8effd640 1 --2- 172.21.15.67:0/1572439515 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0c7c0b3dc0 0x7f0c7c0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.606+0000 7f0c8cff9640 1 -- 172.21.15.67:0/1572439515 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7f0c78104690 con 0x7f0c90107d40 2024-07-31T14:59:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.606+0000 7f0c857fa640 1 -- 172.21.15.67:0/2497025667 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88016070 con 0x55e1129201b0 2024-07-31T14:59:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.606+0000 7f0c857fa640 1 -- 172.21.15.67:0/2497025667 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c88004d20 con 0x55e1129201b0 2024-07-31T14:59:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.606+0000 7f0c857fa640 1 -- 172.21.15.67:0/2497025667 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88012870 con 0x55e1129201b0 2024-07-31T14:59:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.607+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/2497025667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1129201b0 msgr2=0x55e112cc2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.607+0000 7f0c8effd640 1 --2- 172.21.15.67:0/1572439515 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0c7c0b3dc0 0x7f0c7c0b62d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55e112cc1960 tx=0x7f0c80005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.607+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/2497025667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1129201b0 0x55e112cc2ce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009fa0 tx=0x7f0c88010800 comp rx=0 tx=0).stop 2024-07-31T14:59:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.607+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/2497025667 shutdown_connections 2024-07-31T14:59:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.607+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/2497025667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112cc3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.607+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/2497025667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1128f72f0 0x55e112cc3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.607+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/2497025667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1129201b0 0x55e112cc2ce0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.607+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/2497025667 >> 172.21.15.67:0/2497025667 conn(0x55e112914300 msgr2=0x55e112cc0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.607+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/2497025667 shutdown_connections 2024-07-31T14:59:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.607+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/2497025667 wait complete. 2024-07-31T14:59:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.608+0000 7f0c9c5d3740 1 -- start start 2024-07-31T14:59:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.608+0000 7f0c9c5d3740 1 Processor -- start 2024-07-31T14:59:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.608+0000 7f0c9c5d3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1128f72f0 0x55e112cce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.608+0000 7f0c9c5d3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128ff4d0 0x55e112ccedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.608+0000 7f0c9c5d3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1129201b0 0x55e112cda780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.608+0000 7f0c9c5d3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e112b61ab0 con 0x55e1129201b0 2024-07-31T14:59:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.608+0000 7f0c9c5d3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e112bafcf0 con 0x55e1128ff4d0 2024-07-31T14:59:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.608+0000 7f0c9c5d3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e112cd1460 con 0x55e1128f72f0 2024-07-31T14:59:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.608+0000 7f0c9c5d3740 10 client.0 _ll_register_callbacks cb 0x55e112c219f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.608+0000 7f0c8f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1129201b0 0x55e112cda780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.608+0000 7f0c87fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128ff4d0 0x55e112ccedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.608+0000 7f0c8f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1129201b0 0x55e112cda780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48610/0 (socket says 172.21.15.67:48610) 2024-07-31T14:59:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.608+0000 7f0c8f7fe640 1 -- 172.21.15.67:0/282591904 learned_addr learned my addr 172.21.15.67:0/282591904 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.608+0000 7f0c8effd640 1 --2- 172.21.15.67:0/282591904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1128f72f0 0x55e112cce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.609+0000 7f0c8f7fe640 1 -- 172.21.15.67:0/282591904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1128f72f0 msgr2=0x55e112cce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.609+0000 7f0c8f7fe640 1 --2- 172.21.15.67:0/282591904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1128f72f0 0x55e112cce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.609+0000 7f0c8f7fe640 1 -- 172.21.15.67:0/282591904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128ff4d0 msgr2=0x55e112ccedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.609+0000 7f0c8f7fe640 1 --2- 172.21.15.67:0/282591904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128ff4d0 0x55e112ccedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.609+0000 7f0c8f7fe640 1 -- 172.21.15.67:0/282591904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e11298b790 con 0x55e1129201b0 2024-07-31T14:59:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.609+0000 7f0c8f7fe640 1 --2- 172.21.15.67:0/282591904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1129201b0 0x55e112cda780 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0c88012550 tx=0x7f0c88004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b8591904e457eb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.609+0000 7f0c767fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:59:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.609+0000 7f0c767fc640 1 -- 172.21.15.67:0/282591904 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88016030 con 0x55e1129201b0 2024-07-31T14:59:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.609+0000 7f0c767fc640 1 -- 172.21.15.67:0/282591904 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7f0c880051e0 con 0x55e1129201b0 2024-07-31T14:59:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.609+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/282591904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c88004d20 con 0x55e1129201b0 2024-07-31T14:59:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.609+0000 7f0c767fc640 1 -- 172.21.15.67:0/282591904 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0c880053e0 con 0x55e1129201b0 2024-07-31T14:59:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.609+0000 7f0c9c5d3740 10 client.17520 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.609+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/282591904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e112a04620 con 0x55e1129201b0 2024-07-31T14:59:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.610+0000 7f0c767fc640 1 -- 172.21.15.67:0/282591904 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c88004d20 con 0x55e1129201b0 2024-07-31T14:59:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.610+0000 7f0c767fc640 1 -- 172.21.15.67:0/282591904 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f0c8800f040 con 0x55e1129201b0 2024-07-31T14:59:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.610+0000 7f0c9c5d3740 10 client.17520 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.610+0000 7f0c9c5d3740 10 client.17520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.610+0000 7f0c9c5d3740 20 client.17520 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.610+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/282591904 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e1128ded80 0x55e112ce0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.610+0000 7f0c9c5d3740 4 client.17520 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:59:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.610+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/282591904 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e112c3a300 con 0x55e1128ded80 2024-07-31T14:59:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.611+0000 7f0c8effd640 1 --2- 172.21.15.67:0/282591904 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e1128ded80 0x55e112ce0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.611+0000 7f0c767fc640 10 client.17520 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.611+0000 7f0c8effd640 1 --2- 172.21.15.67:0/282591904 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e1128ded80 0x55e112ce0670 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb5b5f2e8a3a8ff server_cookie=611e1d667bc19efe in_seq=0 out_seq=0 2024-07-31T14:59:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.613+0000 7f0c767fc640 1 -- 172.21.15.67:0/282591904 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e112c3a300 con 0x55e1128ded80 2024-07-31T14:59:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.613+0000 7f0c767fc640 10 client.17520 handle_command_reply: tid=0 2024-07-31T14:59:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.613+0000 7f0c9c5d3740 1 client.17520 shutdown 2024-07-31T14:59:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.614+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/282591904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1129201b0 msgr2=0x55e112cda780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.614+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/282591904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1129201b0 0x55e112cda780 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0c88012550 tx=0x7f0c88004c70 comp rx=0 tx=0).stop 2024-07-31T14:59:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.614+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/282591904 shutdown_connections 2024-07-31T14:59:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.614+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/282591904 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e1128ded80 0x55e112ce0670 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.614+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/282591904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1129201b0 0x55e112cda780 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.614+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/282591904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128ff4d0 0x55e112ccedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.614+0000 7f0c767fc640 0 client.17520 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.614+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/282591904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1128f72f0 0x55e112cce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.614+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/282591904 >> 172.21.15.67:0/282591904 conn(0x55e112914300 msgr2=0x55e112cc3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.614+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/282591904 shutdown_connections 2024-07-31T14:59:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.614+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/282591904 wait complete. 2024-07-31T14:59:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.614+0000 7f0c9c5d3740 20 client.17520 trim_cache size 0 max 16384 2024-07-31T14:59:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.629+0000 7f0c9c5d3740 1 Processor -- start 2024-07-31T14:59:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.629+0000 7f0c9c5d3740 1 -- start start 2024-07-31T14:59:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c9c5d3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128f72f0 0x55e112cc4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c9c5d3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112cda6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c9c5d3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1129201b0 0x55e112cc8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c9c5d3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901b06e0 con 0x55e1128ff4d0 2024-07-31T14:59:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c9c5d3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901b2370 con 0x55e1128f72f0 2024-07-31T14:59:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c9c5d3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901b0db0 con 0x55e1129201b0 2024-07-31T14:59:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c8effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128f72f0 0x55e112cc4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c8effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128f72f0 0x55e112cc4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35294/0 (socket says 172.21.15.67:35294) 2024-07-31T14:59:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c8effd640 1 -- 172.21.15.67:0/3970179114 learned_addr learned my addr 172.21.15.67:0/3970179114 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c8f7fe640 1 --2- 172.21.15.67:0/3970179114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1129201b0 0x55e112cc8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c8f7fe640 1 -- 172.21.15.67:0/3970179114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128f72f0 msgr2=0x55e112cc4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c8f7fe640 1 --2- 172.21.15.67:0/3970179114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128f72f0 0x55e112cc4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c8f7fe640 1 -- 172.21.15.67:0/3970179114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 msgr2=0x55e112cda6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c87fff640 1 --2- 172.21.15.67:0/3970179114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112cda6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c8f7fe640 1 --2- 172.21.15.67:0/3970179114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112cda6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.630+0000 7f0c8f7fe640 1 -- 172.21.15.67:0/3970179114 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e112cc9e40 con 0x55e1129201b0 2024-07-31T14:59:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.631+0000 7f0c8effd640 1 --2- 172.21.15.67:0/3970179114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128f72f0 0x55e112cc4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:59:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.631+0000 7f0c87fff640 1 --2- 172.21.15.67:0/3970179114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112cda6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:59:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.631+0000 7f0c8f7fe640 1 --2- 172.21.15.67:0/3970179114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1129201b0 0x55e112cc8e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009640 tx=0x7f0c88009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f30f9dedc0583c41 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.631+0000 7f0c84ff9640 1 -- 172.21.15.67:0/3970179114 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8801e070 con 0x55e1129201b0 2024-07-31T14:59:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.631+0000 7f0c84ff9640 1 -- 172.21.15.67:0/3970179114 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c88010d00 con 0x55e1129201b0 2024-07-31T14:59:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.631+0000 7f0c84ff9640 1 -- 172.21.15.67:0/3970179114 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88016070 con 0x55e1129201b0 2024-07-31T14:59:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.632+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/3970179114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1129201b0 msgr2=0x55e112cc8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.632+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/3970179114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1129201b0 0x55e112cc8e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009640 tx=0x7f0c88009800 comp rx=0 tx=0).stop 2024-07-31T14:59:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.632+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/3970179114 shutdown_connections 2024-07-31T14:59:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.632+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/3970179114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1129201b0 0x55e112cc8e20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.632+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/3970179114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112cda6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.632+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/3970179114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128f72f0 0x55e112cc4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.632+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/3970179114 >> 172.21.15.67:0/3970179114 conn(0x55e112914300 msgr2=0x55e112cc3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.632+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/3970179114 shutdown_connections 2024-07-31T14:59:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.632+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/3970179114 wait complete. 2024-07-31T14:59:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.633+0000 7f0c9c5d3740 1 -- start start 2024-07-31T14:59:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.633+0000 7f0c9c5d3740 1 Processor -- start 2024-07-31T14:59:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.633+0000 7f0c9c5d3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128f72f0 0x55e112ce3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.633+0000 7f0c9c5d3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112ce1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.633+0000 7f0c9c5d3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1129201b0 0x55e112ce2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.633+0000 7f0c9c5d3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901b1ca0 con 0x55e1128ff4d0 2024-07-31T14:59:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.633+0000 7f0c9c5d3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901b1510 con 0x55e1128f72f0 2024-07-31T14:59:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.633+0000 7f0c9c5d3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0c901ae6e0 con 0x55e1129201b0 2024-07-31T14:59:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.633+0000 7f0c9c5d3740 10 client.0 _ll_register_callbacks cb 0x7f0c7810a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.633+0000 7f0c87fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112ce1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.633+0000 7f0c87fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112ce1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48618/0 (socket says 172.21.15.67:48618) 2024-07-31T14:59:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.633+0000 7f0c87fff640 1 -- 172.21.15.67:0/1108695123 learned_addr learned my addr 172.21.15.67:0/1108695123 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.633+0000 7f0c8f7fe640 1 --2- 172.21.15.67:0/1108695123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1129201b0 0x55e112ce2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.633+0000 7f0c8effd640 1 --2- 172.21.15.67:0/1108695123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128f72f0 0x55e112ce3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.634+0000 7f0c8effd640 1 -- 172.21.15.67:0/1108695123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1129201b0 msgr2=0x55e112ce2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.634+0000 7f0c8effd640 1 --2- 172.21.15.67:0/1108695123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1129201b0 0x55e112ce2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.634+0000 7f0c8effd640 1 -- 172.21.15.67:0/1108695123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 msgr2=0x55e112ce1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.634+0000 7f0c8effd640 1 --2- 172.21.15.67:0/1108695123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112ce1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.634+0000 7f0c8effd640 1 -- 172.21.15.67:0/1108695123 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e112ce2a40 con 0x55e1128f72f0 2024-07-31T14:59:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.634+0000 7f0c87fff640 1 --2- 172.21.15.67:0/1108695123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112ce1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:59:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.634+0000 7f0c8effd640 1 --2- 172.21.15.67:0/1108695123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128f72f0 0x55e112ce3af0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55e112b61c00 tx=0x7f0c80017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad7d11c92645483 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.634+0000 7f0c767fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:59:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.634+0000 7f0c767fc640 1 -- 172.21.15.67:0/1108695123 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80003b40 con 0x55e1128f72f0 2024-07-31T14:59:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.634+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/1108695123 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c88010d00 con 0x55e1128f72f0 2024-07-31T14:59:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.634+0000 7f0c767fc640 1 -- 172.21.15.67:0/1108695123 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7f0c80003ce0 con 0x55e1128f72f0 2024-07-31T14:59:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.635+0000 7f0c767fc640 1 -- 172.21.15.67:0/1108695123 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0c8003c050 con 0x55e1128f72f0 2024-07-31T14:59:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.635+0000 7f0c9c5d3740 10 client.27080 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.635+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/1108695123 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e112d47880 con 0x55e1128f72f0 2024-07-31T14:59:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.635+0000 7f0c767fc640 1 -- 172.21.15.67:0/1108695123 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c88010d00 con 0x55e1128f72f0 2024-07-31T14:59:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.635+0000 7f0c767fc640 1 -- 172.21.15.67:0/1108695123 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f0c80004b90 con 0x55e1128f72f0 2024-07-31T14:59:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.635+0000 7f0c9c5d3740 10 client.27080 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.635+0000 7f0c9c5d3740 10 client.27080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.635+0000 7f0c9c5d3740 20 client.27080 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.635+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/1108695123 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e1128b41c0 0x55e112d4a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.635+0000 7f0c9c5d3740 4 client.27080 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:59:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.635+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/1108695123 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e112c35da0 con 0x55e1128b41c0 2024-07-31T14:59:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.636+0000 7f0c87fff640 1 --2- 172.21.15.67:0/1108695123 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e1128b41c0 0x55e112d4a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.636+0000 7f0c767fc640 10 client.27080 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:10.636+0000 7f0c87fff640 1 --2- 172.21.15.67:0/1108695123 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e1128b41c0 0x55e112d4a0e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa5e29bc8ec26de server_cookie=fdba6c3bbd49d3be in_seq=0 out_seq=0 2024-07-31T14:59:11.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:11 smithi067 ceph-mon[32750]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-07-31T14:59:11.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:11 smithi073 ceph-mon[36714]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-07-31T14:59:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:11 smithi077 ceph-mon[37327]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-07-31T14:59:11.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.951+0000 7f334fa7d640 1 -- 172.21.15.67:0/62651993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3348111970 msgr2=0x7f3348109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:11.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.951+0000 7f334fa7d640 1 --2- 172.21.15.67:0/62651993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3348111970 0x7f3348109df0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3340009be0 tx=0x7f334002a710 comp rx=0 tx=0).stop 2024-07-31T14:59:11.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.953+0000 7f334fa7d640 1 -- 172.21.15.67:0/62651993 shutdown_connections 2024-07-31T14:59:11.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.953+0000 7f334fa7d640 1 --2- 172.21.15.67:0/62651993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f334811c660 0x7f334811eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:11.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.953+0000 7f334fa7d640 1 --2- 172.21.15.67:0/62651993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f334810a410 0x7f334810a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:11.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.953+0000 7f334fa7d640 1 --2- 172.21.15.67:0/62651993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3348111970 0x7f3348109df0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:11.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.953+0000 7f334fa7d640 1 -- 172.21.15.67:0/62651993 >> 172.21.15.67:0/62651993 conn(0x7f33480708f0 msgr2=0x7f3348070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:11.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.955+0000 7f334fa7d640 1 -- 172.21.15.67:0/62651993 shutdown_connections 2024-07-31T14:59:11.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.957+0000 7f334fa7d640 1 -- 172.21.15.67:0/62651993 wait complete. 2024-07-31T14:59:11.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.958+0000 7f334fa7d640 1 Processor -- start 2024-07-31T14:59:11.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.961+0000 7f334fa7d640 1 -- start start 2024-07-31T14:59:11.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.961+0000 7f334fa7d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f334810a410 0x7f3348119f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:11.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.961+0000 7f334fa7d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3348111970 0x7f3348113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:11.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.961+0000 7f334fa7d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f334811c660 0x7f3348113880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:11.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.961+0000 7f334fa7d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3348122320 con 0x7f334810a410 2024-07-31T14:59:11.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.961+0000 7f334fa7d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f33481221a0 con 0x7f334811c660 2024-07-31T14:59:11.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.961+0000 7f334fa7d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f33481224a0 con 0x7f3348111970 2024-07-31T14:59:11.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.967+0000 7f334d819640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f334810a410 0x7f3348119f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:11.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.967+0000 7f334d018640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3348111970 0x7f3348113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:11.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.967+0000 7f334d819640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f334810a410 0x7f3348119f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60654/0 (socket says 172.21.15.67:60654) 2024-07-31T14:59:11.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.967+0000 7f334d819640 1 -- 172.21.15.67:0/1719861124 learned_addr learned my addr 172.21.15.67:0/1719861124 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:11.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.968+0000 7f334e01a640 1 --2- 172.21.15.67:0/1719861124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f334811c660 0x7f3348113880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:11.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.974+0000 7f334d819640 1 -- 172.21.15.67:0/1719861124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3348111970 msgr2=0x7f3348113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:11.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.974+0000 7f334d819640 1 --2- 172.21.15.67:0/1719861124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3348111970 0x7f3348113200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:11.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.974+0000 7f334d819640 1 -- 172.21.15.67:0/1719861124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f334811c660 msgr2=0x7f3348113880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:11.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.974+0000 7f334d819640 1 --2- 172.21.15.67:0/1719861124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f334811c660 0x7f3348113880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:11.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.974+0000 7f334d819640 1 -- 172.21.15.67:0/1719861124 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3348118ba0 con 0x7f334810a410 2024-07-31T14:59:11.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.982+0000 7f334d819640 1 --2- 172.21.15.67:0/1719861124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f334810a410 0x7f3348119f30 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f334002ac80 tx=0x7f3340030bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.987+0000 7f333effd640 1 -- 172.21.15.67:0/1719861124 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3340040020 con 0x7f334810a410 2024-07-31T14:59:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.987+0000 7f333effd640 1 -- 172.21.15.67:0/1719861124 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33400318f0 con 0x7f334810a410 2024-07-31T14:59:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.987+0000 7f333effd640 1 -- 172.21.15.67:0/1719861124 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3340036040 con 0x7f334810a410 2024-07-31T14:59:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.989+0000 7f334fa7d640 1 -- 172.21.15.67:0/1719861124 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3348118e30 con 0x7f334810a410 2024-07-31T14:59:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.989+0000 7f334fa7d640 1 -- 172.21.15.67:0/1719861124 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33481192c0 con 0x7f334810a410 2024-07-31T14:59:11.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.993+0000 7f3355fce740 1 Processor -- start 2024-07-31T14:59:11.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.996+0000 7f3355fce740 1 -- start start 2024-07-31T14:59:11.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.996+0000 7f3355fce740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153735070 0x559153ada570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:11.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.996+0000 7f3355fce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 0x559153adaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:12.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.996+0000 7f3355fce740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559153714390 0x559153adaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:12.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.996+0000 7f3355fce740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559153a50bb0 con 0x559153714390 2024-07-31T14:59:12.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.996+0000 7f3355fce740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559153a400b0 con 0x559153735070 2024-07-31T14:59:12.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.996+0000 7f3355fce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559153a440f0 con 0x55915370c1b0 2024-07-31T14:59:12.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.996+0000 7f333effd640 1 -- 172.21.15.67:0/1719861124 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3340043020 con 0x7f334810a410 2024-07-31T14:59:12.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.997+0000 7f334e01a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153735070 0x559153ada570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:12.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.997+0000 7f334e01a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153735070 0x559153ada570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44386/0 (socket says 172.21.15.67:44386) 2024-07-31T14:59:12.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.997+0000 7f334e01a640 1 -- 172.21.15.67:0/2707220682 learned_addr learned my addr 172.21.15.67:0/2707220682 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:12.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.997+0000 7f334e01a640 1 -- 172.21.15.67:0/2707220682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 msgr2=0x559153adaab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:59:12.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.997+0000 7f334e01a640 1 --2- 172.21.15.67:0/2707220682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 0x559153adaab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.997+0000 7f334e01a640 1 -- 172.21.15.67:0/2707220682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559153714390 msgr2=0x559153adaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:12.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.997+0000 7f334e01a640 1 --2- 172.21.15.67:0/2707220682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559153714390 0x559153adaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.997+0000 7f334e01a640 1 -- 172.21.15.67:0/2707220682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591539e6b90 con 0x559153735070 2024-07-31T14:59:12.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.997+0000 7f334e01a640 1 --2- 172.21.15.67:0/2707220682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153735070 0x559153ada570 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3338009be0 tx=0x7f3338011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc648a0d27b842f6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:12.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.999+0000 7f333effd640 1 --2- 172.21.15.67:0/1719861124 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33340b3f50 0x7f33340b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:12.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:11.999+0000 7f333effd640 1 -- 172.21.15.67:0/1719861124 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7f3340104600 con 0x7f334810a410 2024-07-31T14:59:12.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.000+0000 7f3333fff640 1 -- 172.21.15.67:0/2707220682 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333801e070 con 0x559153735070 2024-07-31T14:59:12.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.000+0000 7f3333fff640 1 -- 172.21.15.67:0/2707220682 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3338009e50 con 0x559153735070 2024-07-31T14:59:12.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.001+0000 7f3355fce740 1 -- 172.21.15.67:0/2707220682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153735070 msgr2=0x559153ada570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:12.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.001+0000 7f3355fce740 1 --2- 172.21.15.67:0/2707220682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153735070 0x559153ada570 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3338009be0 tx=0x7f3338011810 comp rx=0 tx=0).stop 2024-07-31T14:59:12.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.006+0000 7f334d819640 1 --2- 172.21.15.67:0/1719861124 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33340b3f50 0x7f33340b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:12.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.006+0000 7f3355fce740 1 -- 172.21.15.67:0/2707220682 shutdown_connections 2024-07-31T14:59:12.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.006+0000 7f3355fce740 1 --2- 172.21.15.67:0/2707220682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559153714390 0x559153adaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.006+0000 7f3355fce740 1 --2- 172.21.15.67:0/2707220682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 0x559153adaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.006+0000 7f3355fce740 1 --2- 172.21.15.67:0/2707220682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153735070 0x559153ada570 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.006+0000 7f3355fce740 1 -- 172.21.15.67:0/2707220682 >> 172.21.15.67:0/2707220682 conn(0x559153729460 msgr2=0x559153ad8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.014+0000 7f3355fce740 1 -- 172.21.15.67:0/2707220682 shutdown_connections 2024-07-31T14:59:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.015+0000 7f3355fce740 1 -- 172.21.15.67:0/2707220682 wait complete. 2024-07-31T14:59:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.016+0000 7f3355fce740 1 -- start start 2024-07-31T14:59:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.016+0000 7f3355fce740 1 Processor -- start 2024-07-31T14:59:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.017+0000 7f3355fce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 0x559153ae61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.017+0000 7f3355fce740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153714390 0x559153ae6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.017+0000 7f334e01a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 0x559153ae61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.017+0000 7f334e01a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 0x559153ae61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44840/0 (socket says 172.21.15.67:44840) 2024-07-31T14:59:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.017+0000 7f334e01a640 1 -- 172.21.15.67:0/3502170049 learned_addr learned my addr 172.21.15.67:0/3502170049 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.017+0000 7f3355fce740 1 --2- 172.21.15.67:0/3502170049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559153735070 0x559153af20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.018+0000 7f3355fce740 1 -- 172.21.15.67:0/3502170049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5591538a3700 con 0x559153735070 2024-07-31T14:59:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.018+0000 7f3355fce740 1 -- 172.21.15.67:0/3502170049 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5591539e29e0 con 0x559153714390 2024-07-31T14:59:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.018+0000 7f3355fce740 1 -- 172.21.15.67:0/3502170049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559153ae8da0 con 0x55915370c1b0 2024-07-31T14:59:12.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.018+0000 7f334e01a640 1 --2- 172.21.15.67:0/3502170049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559153735070 0x559153af20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:12.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.018+0000 7f334e01a640 1 -- 172.21.15.67:0/3502170049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153714390 msgr2=0x559153ae6770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:59:12.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.018+0000 7f334e01a640 1 --2- 172.21.15.67:0/3502170049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153714390 0x559153ae6770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.018+0000 7f334e01a640 1 -- 172.21.15.67:0/3502170049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559153735070 msgr2=0x559153af20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:12.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.018+0000 7f334e01a640 1 --2- 172.21.15.67:0/3502170049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559153735070 0x559153af20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.018+0000 7f334e01a640 1 -- 172.21.15.67:0/3502170049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591537a0650 con 0x55915370c1b0 2024-07-31T14:59:12.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.019+0000 7f334e01a640 1 --2- 172.21.15.67:0/3502170049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 0x559153ae61b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f333800e590 tx=0x7f333800e690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fef4d59efc22ee5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:12.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.019+0000 7f3355fce740 10 client.0 _ll_register_callbacks cb 0x5591539fd4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:12.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.019+0000 7f3355fce740 1 -- 172.21.15.67:0/3502170049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3338009e50 con 0x55915370c1b0 2024-07-31T14:59:12.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.020+0000 7f33317fa640 10 client.27094 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:59:12.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.020+0000 7f33317fa640 1 -- 172.21.15.67:0/3502170049 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333801e070 con 0x55915370c1b0 2024-07-31T14:59:12.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.020+0000 7f33317fa640 1 -- 172.21.15.67:0/3502170049 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7f3338004220 con 0x55915370c1b0 2024-07-31T14:59:12.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.020+0000 7f33317fa640 1 -- 172.21.15.67:0/3502170049 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3338004420 con 0x55915370c1b0 2024-07-31T14:59:12.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.020+0000 7f3355fce740 10 client.27094 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:12.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.020+0000 7f3355fce740 1 -- 172.21.15.67:0/3502170049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=54}) -- 0x559153af4c50 con 0x55915370c1b0 2024-07-31T14:59:12.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.024+0000 7f33317fa640 1 -- 172.21.15.67:0/3502170049 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f3338004850 con 0x55915370c1b0 2024-07-31T14:59:12.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.024+0000 7f334d819640 1 --2- 172.21.15.67:0/1719861124 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33340b3f50 0x7f33340b6460 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x559153ad91f0 tx=0x7f33400cac90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:12.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.025+0000 7f3355fce740 10 client.27094 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:12.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.025+0000 7f3355fce740 10 client.27094 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:59:12.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.025+0000 7f3355fce740 20 client.27094 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:12.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.025+0000 7f3355fce740 1 --2- 172.21.15.67:0/3502170049 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5591536f39f0 0x559153af7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:12.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.025+0000 7f3355fce740 4 client.27094 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:59:12.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.025+0000 7f3355fce740 1 -- 172.21.15.67:0/3502170049 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3338009e50 con 0x5591536f39f0 2024-07-31T14:59:12.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.027+0000 7f334d018640 1 --2- 172.21.15.67:0/3502170049 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5591536f39f0 0x559153af7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:12.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.032+0000 7f334d018640 1 --2- 172.21.15.67:0/3502170049 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5591536f39f0 0x559153af7fb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9795ea9f8867d7a3 server_cookie=211732baefaabea6 in_seq=0 out_seq=0 2024-07-31T14:59:12.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.033+0000 7f33317fa640 10 client.27094 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:59:12.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.036+0000 7f33317fa640 1 -- 172.21.15.67:0/3502170049 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f3338009e50 con 0x5591536f39f0 2024-07-31T14:59:12.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.036+0000 7f33317fa640 10 client.27094 handle_command_reply: tid=0 2024-07-31T14:59:12.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.036+0000 7f3355fce740 1 client.27094 shutdown 2024-07-31T14:59:12.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.037+0000 7f3355fce740 1 -- 172.21.15.67:0/3502170049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 msgr2=0x559153ae61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:12.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.037+0000 7f3355fce740 1 --2- 172.21.15.67:0/3502170049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 0x559153ae61b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f333800e590 tx=0x7f333800e690 comp rx=0 tx=0).stop 2024-07-31T14:59:12.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.039+0000 7f3355fce740 1 -- 172.21.15.67:0/3502170049 shutdown_connections 2024-07-31T14:59:12.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.039+0000 7f3355fce740 1 --2- 172.21.15.67:0/3502170049 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5591536f39f0 0x559153af7fb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.039+0000 7f3355fce740 1 --2- 172.21.15.67:0/3502170049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559153735070 0x559153af20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.039+0000 7f3355fce740 1 --2- 172.21.15.67:0/3502170049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153714390 0x559153ae6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.039+0000 7f3355fce740 1 --2- 172.21.15.67:0/3502170049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 0x559153ae61b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.039+0000 7f3355fce740 1 -- 172.21.15.67:0/3502170049 >> 172.21.15.67:0/3502170049 conn(0x559153729460 msgr2=0x559153ad6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:12.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.039+0000 7f33317fa640 0 client.27094 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:59:12.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.043+0000 7f3355fce740 1 -- 172.21.15.67:0/3502170049 shutdown_connections 2024-07-31T14:59:12.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.043+0000 7f3355fce740 1 -- 172.21.15.67:0/3502170049 wait complete. 2024-07-31T14:59:12.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.044+0000 7f3355fce740 20 client.27094 trim_cache size 0 max 16384 2024-07-31T14:59:12.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.080+0000 7f3355fce740 1 Processor -- start 2024-07-31T14:59:12.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.081+0000 7f3355fce740 1 -- start start 2024-07-31T14:59:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.081+0000 7f3355fce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 0x559153ad84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.081+0000 7f3355fce740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153714390 0x559153af1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.081+0000 7f3355fce740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559153735070 0x559153afd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.081+0000 7f3355fce740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f33481c7fa0 con 0x559153735070 2024-07-31T14:59:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.081+0000 7f3355fce740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f33481c9c30 con 0x559153714390 2024-07-31T14:59:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.081+0000 7f3355fce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f33481c8670 con 0x55915370c1b0 2024-07-31T14:59:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.082+0000 7f334d018640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153714390 0x559153af1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.082+0000 7f334d018640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153714390 0x559153af1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44404/0 (socket says 172.21.15.67:44404) 2024-07-31T14:59:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.082+0000 7f334d018640 1 -- 172.21.15.67:0/4272685746 learned_addr learned my addr 172.21.15.67:0/4272685746 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:12.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.086+0000 7f334d018640 1 -- 172.21.15.67:0/4272685746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 msgr2=0x559153ad84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:12.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.086+0000 7f334d018640 1 --2- 172.21.15.67:0/4272685746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 0x559153ad84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.086+0000 7f334d018640 1 -- 172.21.15.67:0/4272685746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559153735070 msgr2=0x559153afd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.086+0000 7f334d018640 1 --2- 172.21.15.67:0/4272685746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559153735070 0x559153afd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.086+0000 7f334d018640 1 -- 172.21.15.67:0/4272685746 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559153ae86f0 con 0x559153714390 2024-07-31T14:59:12.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.090+0000 7f334d018640 1 --2- 172.21.15.67:0/4272685746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153714390 0x559153af1ed0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x559153af8520 tx=0x7f33440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbf106c4ca25e3fb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.092+0000 7f33337fe640 1 -- 172.21.15.67:0/4272685746 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334401e070 con 0x559153714390 2024-07-31T14:59:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.092+0000 7f33337fe640 1 -- 172.21.15.67:0/4272685746 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33440035f0 con 0x559153714390 2024-07-31T14:59:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.092+0000 7f33337fe640 1 -- 172.21.15.67:0/4272685746 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33440185d0 con 0x559153714390 2024-07-31T14:59:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.093+0000 7f3355fce740 1 -- 172.21.15.67:0/4272685746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153714390 msgr2=0x559153af1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.093+0000 7f3355fce740 1 --2- 172.21.15.67:0/4272685746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153714390 0x559153af1ed0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x559153af8520 tx=0x7f33440046c0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.095+0000 7f3355fce740 1 -- 172.21.15.67:0/4272685746 shutdown_connections 2024-07-31T14:59:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.095+0000 7f3355fce740 1 --2- 172.21.15.67:0/4272685746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559153735070 0x559153afd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.095+0000 7f3355fce740 1 --2- 172.21.15.67:0/4272685746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559153714390 0x559153af1ed0 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x559153af8520 tx=0x7f33440046c0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.095+0000 7f3355fce740 1 --2- 172.21.15.67:0/4272685746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55915370c1b0 0x559153ad84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.095+0000 7f3355fce740 1 -- 172.21.15.67:0/4272685746 >> 172.21.15.67:0/4272685746 conn(0x559153729460 msgr2=0x559153ad7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:12.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.097+0000 7f3355fce740 1 -- 172.21.15.67:0/4272685746 shutdown_connections 2024-07-31T14:59:12.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.097+0000 7f3355fce740 1 -- 172.21.15.67:0/4272685746 wait complete. 2024-07-31T14:59:12.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.098+0000 7f3355fce740 1 -- start start 2024-07-31T14:59:12.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.098+0000 7f3355fce740 1 Processor -- start 2024-07-31T14:59:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.099+0000 7f3355fce740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55915370c1b0 0x559153aee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.099+0000 7f3355fce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559153735070 0x559153ae9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.099+0000 7f3355fce740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591536c8da0 0x559153aea000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.099+0000 7f3355fce740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f33481c9560 con 0x55915370c1b0 2024-07-31T14:59:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.100+0000 7f3355fce740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f33481c8dd0 con 0x5591536c8da0 2024-07-31T14:59:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.100+0000 7f3355fce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f33481c5fa0 con 0x559153735070 2024-07-31T14:59:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.100+0000 7f3355fce740 10 client.0 _ll_register_callbacks cb 0x7f333801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.104+0000 7f334e01a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55915370c1b0 0x559153aee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.104+0000 7f334e01a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55915370c1b0 0x559153aee5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60698/0 (socket says 172.21.15.67:60698) 2024-07-31T14:59:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.104+0000 7f334e01a640 1 -- 172.21.15.67:0/3553593133 learned_addr learned my addr 172.21.15.67:0/3553593133 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:12.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.107+0000 7f334d018640 1 --2- 172.21.15.67:0/3553593133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559153735070 0x559153ae9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:12.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.109+0000 7f334e01a640 1 --2- 172.21.15.67:0/3553593133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591536c8da0 0x559153aea000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:12.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.110+0000 7f334e01a640 1 -- 172.21.15.67:0/3553593133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559153735070 msgr2=0x559153ae9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:12.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.110+0000 7f334e01a640 1 --2- 172.21.15.67:0/3553593133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559153735070 0x559153ae9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.110+0000 7f334e01a640 1 -- 172.21.15.67:0/3553593133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591536c8da0 msgr2=0x559153aea000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.110+0000 7f334e01a640 1 --2- 172.21.15.67:0/3553593133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591536c8da0 0x559153aea000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.110+0000 7f334e01a640 1 -- 172.21.15.67:0/3553593133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559153aea640 con 0x55915370c1b0 2024-07-31T14:59:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.112+0000 7f334e01a640 1 --2- 172.21.15.67:0/3553593133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55915370c1b0 0x559153aee5c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f3338001eb0 tx=0x7f3338011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a433b1c7eb5f4e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.113+0000 7f33317fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.119+0000 7f33317fa640 1 -- 172.21.15.67:0/3553593133 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338009e50 con 0x55915370c1b0 2024-07-31T14:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.119+0000 7f33317fa640 1 -- 172.21.15.67:0/3553593133 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9351+0+0 (secure 0 0 0) 0x7f3338004020 con 0x55915370c1b0 2024-07-31T14:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.119+0000 7f3355fce740 1 -- 172.21.15.67:0/3553593133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33440035f0 con 0x55915370c1b0 2024-07-31T14:59:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.126+0000 7f33317fa640 1 -- 172.21.15.67:0/3553593133 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3338042050 con 0x55915370c1b0 2024-07-31T14:59:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.126+0000 7f3355fce740 10 client.17535 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.126+0000 7f3355fce740 1 -- 172.21.15.67:0/3553593133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559153b60340 con 0x55915370c1b0 2024-07-31T14:59:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.131+0000 7f33317fa640 1 -- 172.21.15.67:0/3553593133 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33440035f0 con 0x55915370c1b0 2024-07-31T14:59:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.131+0000 7f33317fa640 1 -- 172.21.15.67:0/3553593133 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f3338003c20 con 0x55915370c1b0 2024-07-31T14:59:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.131+0000 7f3355fce740 10 client.17535 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.131+0000 7f3355fce740 10 client.17535 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T14:59:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.131+0000 7f3355fce740 20 client.17535 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.131+0000 7f3355fce740 1 --2- 172.21.15.67:0/3553593133 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x559153b609a0 0x559153b62dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.131+0000 7f3355fce740 4 client.17535 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:59:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.131+0000 7f3355fce740 1 -- 172.21.15.67:0/3553593133 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559153a52b00 con 0x559153b609a0 2024-07-31T14:59:12.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.135+0000 7f334d018640 1 --2- 172.21.15.67:0/3553593133 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x559153b609a0 0x559153b62dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:12.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.136+0000 7f334d018640 1 --2- 172.21.15.67:0/3553593133 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x559153b609a0 0x559153b62dc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5b783f12790a1745 server_cookie=4fc95cd39ab66813 in_seq=0 out_seq=0 2024-07-31T14:59:12.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.136+0000 7f33317fa640 10 client.17535 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T14:59:12.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.136+0000 7f33317fa640 1 -- 172.21.15.67:0/3553593133 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559153a52b00 con 0x559153b609a0 2024-07-31T14:59:12.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.136+0000 7f33317fa640 10 client.17535 handle_command_reply: tid=0 2024-07-31T14:59:12.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.136+0000 7f3355fce740 1 client.17535 shutdown 2024-07-31T14:59:12.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.142+0000 7f3355fce740 1 -- 172.21.15.67:0/3553593133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55915370c1b0 msgr2=0x559153aee5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:12.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.142+0000 7f3355fce740 1 --2- 172.21.15.67:0/3553593133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55915370c1b0 0x559153aee5c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f3338001eb0 tx=0x7f3338011810 comp rx=0 tx=0).stop 2024-07-31T14:59:12.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.142+0000 7f3355fce740 1 -- 172.21.15.67:0/3553593133 shutdown_connections 2024-07-31T14:59:12.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.142+0000 7f3355fce740 1 --2- 172.21.15.67:0/3553593133 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x559153b609a0 0x559153b62dc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.142+0000 7f3355fce740 1 --2- 172.21.15.67:0/3553593133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591536c8da0 0x559153aea000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.142+0000 7f3355fce740 1 --2- 172.21.15.67:0/3553593133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559153735070 0x559153ae9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.142+0000 7f3355fce740 1 --2- 172.21.15.67:0/3553593133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55915370c1b0 0x559153aee5c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.142+0000 7f3355fce740 1 -- 172.21.15.67:0/3553593133 >> 172.21.15.67:0/3553593133 conn(0x559153729460 msgr2=0x559153adf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:12.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.142+0000 7f33317fa640 0 client.17535 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T14:59:12.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.143+0000 7f3355fce740 1 -- 172.21.15.67:0/3553593133 shutdown_connections 2024-07-31T14:59:12.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.143+0000 7f3355fce740 1 -- 172.21.15.67:0/3553593133 wait complete. 2024-07-31T14:59:12.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.143+0000 7f3355fce740 20 client.17535 trim_cache size 0 max 16384 2024-07-31T14:59:12.145 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:59:12.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.144+0000 7f334fa7d640 1 -- 172.21.15.67:0/1719861124 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33340b3f50 msgr2=0x7f33340b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:12.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.144+0000 7f334fa7d640 1 --2- 172.21.15.67:0/1719861124 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33340b3f50 0x7f33340b6460 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x559153ad91f0 tx=0x7f33400cac90 comp rx=0 tx=0).stop 2024-07-31T14:59:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.144+0000 7f334fa7d640 1 -- 172.21.15.67:0/1719861124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f334810a410 msgr2=0x7f3348119f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.144+0000 7f334fa7d640 1 --2- 172.21.15.67:0/1719861124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f334810a410 0x7f3348119f30 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f334002ac80 tx=0x7f3340030bf0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.144+0000 7f334fa7d640 1 -- 172.21.15.67:0/1719861124 shutdown_connections 2024-07-31T14:59:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.144+0000 7f334fa7d640 1 --2- 172.21.15.67:0/1719861124 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33340b3f50 0x7f33340b6460 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.144+0000 7f334fa7d640 1 --2- 172.21.15.67:0/1719861124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f334811c660 0x7f3348113880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.144+0000 7f334fa7d640 1 --2- 172.21.15.67:0/1719861124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3348111970 0x7f3348113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.144+0000 7f334fa7d640 1 --2- 172.21.15.67:0/1719861124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f334810a410 0x7f3348119f30 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.144+0000 7f334fa7d640 1 -- 172.21.15.67:0/1719861124 >> 172.21.15.67:0/1719861124 conn(0x7f33480708f0 msgr2=0x7f334811deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.144+0000 7f334fa7d640 1 -- 172.21.15.67:0/1719861124 shutdown_connections 2024-07-31T14:59:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:12.144+0000 7f334fa7d640 1 -- 172.21.15.67:0/1719861124 wait complete. 2024-07-31T14:59:12.997 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T14:59:13.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:13 smithi067 ceph-mon[32750]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.1 MiB/s wr, 270 op/s 2024-07-31T14:59:13.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:13 smithi073 ceph-mon[36714]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.1 MiB/s wr, 270 op/s 2024-07-31T14:59:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:13 smithi077 ceph-mon[37327]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.1 MiB/s wr, 270 op/s 2024-07-31T14:59:13.532 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:59:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c767fc640 1 -- 172.21.15.67:0/1108695123 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e112c35da0 con 0x55e1128b41c0 2024-07-31T14:59:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c767fc640 10 client.27080 handle_command_reply: tid=0 2024-07-31T14:59:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c9c5d3740 1 client.27080 shutdown 2024-07-31T14:59:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/1108695123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128f72f0 msgr2=0x55e112ce3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/1108695123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128f72f0 0x55e112ce3af0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55e112b61c00 tx=0x7f0c80017000 comp rx=0 tx=0).stop 2024-07-31T14:59:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/1108695123 shutdown_connections 2024-07-31T14:59:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/1108695123 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e1128b41c0 0x55e112d4a0e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/1108695123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1129201b0 0x55e112ce2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/1108695123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1128ff4d0 0x55e112ce1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c9c5d3740 1 --2- 172.21.15.67:0/1108695123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e1128f72f0 0x55e112ce3af0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c767fc640 0 client.27080 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/1108695123 >> 172.21.15.67:0/1108695123 conn(0x55e112914300 msgr2=0x55e112ccca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/1108695123 shutdown_connections 2024-07-31T14:59:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c9c5d3740 1 -- 172.21.15.67:0/1108695123 wait complete. 2024-07-31T14:59:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.756+0000 7f0c9c5d3740 20 client.27080 trim_cache size 0 max 16384 2024-07-31T14:59:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:59:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:59:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:59:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:59:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.757+0000 7f0c956e3640 1 -- 172.21.15.67:0/1572439515 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0c7c0b3dc0 msgr2=0x7f0c7c0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.757+0000 7f0c956e3640 1 --2- 172.21.15.67:0/1572439515 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0c7c0b3dc0 0x7f0c7c0b62d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55e112cc1960 tx=0x7f0c80005f90 comp rx=0 tx=0).stop 2024-07-31T14:59:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.757+0000 7f0c956e3640 1 -- 172.21.15.67:0/1572439515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c90107d40 msgr2=0x7f0c9019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.757+0000 7f0c956e3640 1 --2- 172.21.15.67:0/1572439515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c90107d40 0x7f0c9019a430 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0c78006010 tx=0x7f0c78004060 comp rx=0 tx=0).stop 2024-07-31T14:59:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.757+0000 7f0c956e3640 1 -- 172.21.15.67:0/1572439515 shutdown_connections 2024-07-31T14:59:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.757+0000 7f0c956e3640 1 --2- 172.21.15.67:0/1572439515 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0c7c0b3dc0 0x7f0c7c0b62d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.757+0000 7f0c956e3640 1 --2- 172.21.15.67:0/1572439515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c90108690 0x7f0c9019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.757+0000 7f0c956e3640 1 --2- 172.21.15.67:0/1572439515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c90107d40 0x7f0c9019a430 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.757+0000 7f0c956e3640 1 --2- 172.21.15.67:0/1572439515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c90102410 0x7f0c90199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.757+0000 7f0c956e3640 1 -- 172.21.15.67:0/1572439515 >> 172.21.15.67:0/1572439515 conn(0x7f0c9007a140 msgr2=0x7f0c90103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.757+0000 7f0c956e3640 1 -- 172.21.15.67:0/1572439515 shutdown_connections 2024-07-31T14:59:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:14.757+0000 7f0c956e3640 1 -- 172.21.15.67:0/1572439515 wait complete. 2024-07-31T14:59:15.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:15 smithi067 ceph-mon[32750]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.1 MiB/s wr, 565 op/s 2024-07-31T14:59:15.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:15 smithi073 ceph-mon[36714]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.1 MiB/s wr, 565 op/s 2024-07-31T14:59:15.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.423+0000 7fd3cffff640 1 -- 172.21.15.67:0/1988850076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3c80a50f0 msgr2=0x7fd3c80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:15.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.423+0000 7fd3cffff640 1 --2- 172.21.15.67:0/1988850076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3c80a50f0 0x7fd3c80a5500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009900 tx=0x7fd3c002a420 comp rx=0 tx=0).stop 2024-07-31T14:59:15.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.429+0000 7fd3cffff640 1 -- 172.21.15.67:0/1988850076 shutdown_connections 2024-07-31T14:59:15.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.429+0000 7fd3cffff640 1 --2- 172.21.15.67:0/1988850076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3c80ab1b0 0x7fd3c80ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.429+0000 7fd3cffff640 1 --2- 172.21.15.67:0/1988850076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c80aa860 0x7fd3c80aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.429+0000 7fd3cffff640 1 --2- 172.21.15.67:0/1988850076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3c80a50f0 0x7fd3c80a5500 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.429+0000 7fd3cffff640 1 -- 172.21.15.67:0/1988850076 >> 172.21.15.67:0/1988850076 conn(0x7fd3c801cc70 msgr2=0x7fd3c801d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:15.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.433+0000 7fd3cffff640 1 -- 172.21.15.67:0/1988850076 shutdown_connections 2024-07-31T14:59:15.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.434+0000 7fd3cffff640 1 -- 172.21.15.67:0/1988850076 wait complete. 2024-07-31T14:59:15.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.436+0000 7fd3cffff640 1 Processor -- start 2024-07-31T14:59:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.440+0000 7fd3cffff640 1 -- start start 2024-07-31T14:59:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.441+0000 7fd3cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3c80a50f0 0x7fd3c8143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.441+0000 7fd3cffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c80aa860 0x7fd3c8144510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.441+0000 7fd3cffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3c80ab1b0 0x7fd3c81497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.441+0000 7fd3ceffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3c80a50f0 0x7fd3c8143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.441+0000 7fd3ceffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3c80a50f0 0x7fd3c8143f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44878/0 (socket says 172.21.15.67:44878) 2024-07-31T14:59:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.441+0000 7fd3ce7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c80aa860 0x7fd3c8144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.441+0000 7fd3ce7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c80aa860 0x7fd3c8144510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44426/0 (socket says 172.21.15.67:44426) 2024-07-31T14:59:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.442+0000 7fd3ce7fc640 1 -- 172.21.15.67:0/1970408811 learned_addr learned my addr 172.21.15.67:0/1970408811 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.442+0000 7fd3cffff640 1 -- 172.21.15.67:0/1970408811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80bd790 con 0x7fd3c80ab1b0 2024-07-31T14:59:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.442+0000 7fd3cffff640 1 -- 172.21.15.67:0/1970408811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80bd610 con 0x7fd3c80aa860 2024-07-31T14:59:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.442+0000 7fd3cffff640 1 -- 172.21.15.67:0/1970408811 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80bd910 con 0x7fd3c80a50f0 2024-07-31T14:59:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.443+0000 7fd3ce7fc640 1 -- 172.21.15.67:0/1970408811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3c80a50f0 msgr2=0x7fd3c8143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.443+0000 7fd3ce7fc640 1 --2- 172.21.15.67:0/1970408811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3c80a50f0 0x7fd3c8143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.443+0000 7fd3ce7fc640 1 -- 172.21.15.67:0/1970408811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3c80ab1b0 msgr2=0x7fd3c81497b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T14:59:15.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.443+0000 7fd3ce7fc640 1 --2- 172.21.15.67:0/1970408811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3c80ab1b0 0x7fd3c81497b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.443+0000 7fd3ce7fc640 1 -- 172.21.15.67:0/1970408811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c8149ee0 con 0x7fd3c80aa860 2024-07-31T14:59:15.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.444+0000 7fd3ce7fc640 1 --2- 172.21.15.67:0/1970408811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c80aa860 0x7fd3c8144510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00666a0 tx=0x7fd3d0072ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:15.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.444+0000 7fd3b7fff640 1 -- 172.21.15.67:0/1970408811 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d00709e0 con 0x7fd3c80aa860 2024-07-31T14:59:15.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.444+0000 7fd3cffff640 1 -- 172.21.15.67:0/1970408811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c814a1d0 con 0x7fd3c80aa860 2024-07-31T14:59:15.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.444+0000 7fd3cffff640 1 -- 172.21.15.67:0/1970408811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c8150930 con 0x7fd3c80aa860 2024-07-31T14:59:15.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.445+0000 7fd3b7fff640 1 -- 172.21.15.67:0/1970408811 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d0070b80 con 0x7fd3c80aa860 2024-07-31T14:59:15.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.445+0000 7fd3b7fff640 1 -- 172.21.15.67:0/1970408811 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d007d660 con 0x7fd3c80aa860 2024-07-31T14:59:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.449+0000 7fd3dba98740 1 Processor -- start 2024-07-31T14:59:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.451+0000 7fd3dba98740 1 -- start start 2024-07-31T14:59:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.451+0000 7fd3dba98740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 0x55c32de7e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.451+0000 7fd3dba98740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32dab01b0 0x55c32de7ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.451+0000 7fd3dba98740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 0x55c32de7f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.451+0000 7fd3dba98740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c32ddf4bb0 con 0x55c32dab01b0 2024-07-31T14:59:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.451+0000 7fd3dba98740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c32dde40b0 con 0x55c32dad9070 2024-07-31T14:59:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.451+0000 7fd3dba98740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c32dde80f0 con 0x55c32dab8390 2024-07-31T14:59:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.453+0000 7fd3cf7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 0x55c32de7e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.453+0000 7fd3cf7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 0x55c32de7e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44432/0 (socket says 172.21.15.67:44432) 2024-07-31T14:59:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.453+0000 7fd3cf7fe640 1 -- 172.21.15.67:0/2831284288 learned_addr learned my addr 172.21.15.67:0/2831284288 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.453+0000 7fd3b7fff640 1 -- 172.21.15.67:0/1970408811 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd3d0075430 con 0x7fd3c80aa860 2024-07-31T14:59:15.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.454+0000 7fd3ceffd640 1 --2- 172.21.15.67:0/2831284288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32dab01b0 0x55c32de7ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:15.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.454+0000 7fd3ce7fc640 1 --2- 172.21.15.67:0/2831284288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 0x55c32de7f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:15.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.455+0000 7fd3ceffd640 1 -- 172.21.15.67:0/2831284288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 msgr2=0x55c32de7f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:15.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.455+0000 7fd3ceffd640 1 --2- 172.21.15.67:0/2831284288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 0x55c32de7f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.455+0000 7fd3ceffd640 1 -- 172.21.15.67:0/2831284288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 msgr2=0x55c32de7e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:15.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.455+0000 7fd3ceffd640 1 --2- 172.21.15.67:0/2831284288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 0x55c32de7e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.455+0000 7fd3ceffd640 1 -- 172.21.15.67:0/2831284288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c32dd8ab90 con 0x55c32dab01b0 2024-07-31T14:59:15.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.455+0000 7fd3b7fff640 1 --2- 172.21.15.67:0/1970408811 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd3b80b3e40 0x7fd3b80b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.456+0000 7fd3b7fff640 1 -- 172.21.15.67:0/1970408811 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7fd3d01495d0 con 0x7fd3c80aa860 2024-07-31T14:59:15.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.456+0000 7fd3ceffd640 1 --2- 172.21.15.67:0/2831284288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32dab01b0 0x55c32de7ec50 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009850 tx=0x7fd3c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd3555bc3c6ec7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:15.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.458+0000 7fd3b57fa640 1 -- 172.21.15.67:0/2831284288 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0040070 con 0x55c32dab01b0 2024-07-31T14:59:15.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.458+0000 7fd3b57fa640 1 -- 172.21.15.67:0/2831284288 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3c00054d0 con 0x55c32dab01b0 2024-07-31T14:59:15.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.458+0000 7fd3b57fa640 1 -- 172.21.15.67:0/2831284288 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0035040 con 0x55c32dab01b0 2024-07-31T14:59:15.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.459+0000 7fd3cf7fe640 1 --2- 172.21.15.67:0/1970408811 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd3b80b3e40 0x7fd3b80b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:15.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.462+0000 7fd3dba98740 1 -- 172.21.15.67:0/2831284288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32dab01b0 msgr2=0x55c32de7ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:15.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.462+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2831284288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32dab01b0 0x55c32de7ec50 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009850 tx=0x7fd3c0000bc0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.462+0000 7fd3cf7fe640 1 --2- 172.21.15.67:0/1970408811 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd3b80b3e40 0x7fd3b80b6350 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009920 tx=0x7fd3c40090e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:15.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.462+0000 7fd3dba98740 1 -- 172.21.15.67:0/2831284288 shutdown_connections 2024-07-31T14:59:15.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.462+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2831284288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 0x55c32de7f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.462+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2831284288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32dab01b0 0x55c32de7ec50 secure :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009850 tx=0x7fd3c0000bc0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.462+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2831284288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 0x55c32de7e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.462+0000 7fd3dba98740 1 -- 172.21.15.67:0/2831284288 >> 172.21.15.67:0/2831284288 conn(0x55c32dacd460 msgr2=0x55c32de7c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:15.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.462+0000 7fd3dba98740 1 -- 172.21.15.67:0/2831284288 shutdown_connections 2024-07-31T14:59:15.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.463+0000 7fd3dba98740 1 -- 172.21.15.67:0/2831284288 wait complete. 2024-07-31T14:59:15.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.463+0000 7fd3dba98740 1 -- start start 2024-07-31T14:59:15.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.463+0000 7fd3dba98740 1 Processor -- start 2024-07-31T14:59:15.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.464+0000 7fd3dba98740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 0x55c32de9b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.464+0000 7fd3dba98740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 0x55c32de9b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.464+0000 7fd3dba98740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32dea7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.464+0000 7fd3dba98740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c32dc47700 con 0x55c32da979f0 2024-07-31T14:59:15.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.464+0000 7fd3dba98740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c32dd869e0 con 0x55c32dad9070 2024-07-31T14:59:15.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.464+0000 7fd3dba98740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c32de9ddc0 con 0x55c32dab8390 2024-07-31T14:59:15.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.464+0000 7fd3dba98740 10 client.0 _ll_register_callbacks cb 0x55c32dda14f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:15.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.465+0000 7fd3cf7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32dea7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:15.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.465+0000 7fd3cf7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32dea7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60720/0 (socket says 172.21.15.67:60720) 2024-07-31T14:59:15.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.465+0000 7fd3cf7fe640 1 -- 172.21.15.67:0/2895488083 learned_addr learned my addr 172.21.15.67:0/2895488083 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:15.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.466+0000 7fd3cf7fe640 1 -- 172.21.15.67:0/2895488083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 msgr2=0x55c32de9b360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:59:15.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.466+0000 7fd3cf7fe640 1 --2- 172.21.15.67:0/2895488083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 0x55c32de9b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.466+0000 7fd3cf7fe640 1 -- 172.21.15.67:0/2895488083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 msgr2=0x55c32de9b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.466+0000 7fd3cf7fe640 1 --2- 172.21.15.67:0/2895488083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 0x55c32de9b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.466+0000 7fd3cf7fe640 1 -- 172.21.15.67:0/2895488083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c32db44650 con 0x55c32da979f0 2024-07-31T14:59:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.466+0000 7fd3cf7fe640 1 --2- 172.21.15.67:0/2895488083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32dea7180 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4015730 tx=0x7fd3c40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f63132c859671a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.468+0000 7fd3b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:59:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.468+0000 7fd3b2ff5640 1 -- 172.21.15.67:0/2895488083 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4023020 con 0x55c32da979f0 2024-07-31T14:59:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.468+0000 7fd3b2ff5640 1 -- 172.21.15.67:0/2895488083 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7fd3c4013540 con 0x55c32da979f0 2024-07-31T14:59:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.468+0000 7fd3dba98740 1 -- 172.21.15.67:0/2895488083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3c00054d0 con 0x55c32da979f0 2024-07-31T14:59:15.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.469+0000 7fd3b2ff5640 1 -- 172.21.15.67:0/2895488083 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fd3c401b040 con 0x55c32da979f0 2024-07-31T14:59:15.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.469+0000 7fd3dba98740 10 client.17541 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:15.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.469+0000 7fd3dba98740 1 -- 172.21.15.67:0/2895488083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c32dea9fc0 con 0x55c32da979f0 2024-07-31T14:59:15.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.470+0000 7fd3b2ff5640 1 -- 172.21.15.67:0/2895488083 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3c00054d0 con 0x55c32da979f0 2024-07-31T14:59:15.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.470+0000 7fd3b2ff5640 1 -- 172.21.15.67:0/2895488083 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7fd3c401e030 con 0x55c32da979f0 2024-07-31T14:59:15.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.470+0000 7fd3dba98740 10 client.17541 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:15.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.470+0000 7fd3dba98740 10 client.17541 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:59:15.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.470+0000 7fd3dba98740 20 client.17541 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:15.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.470+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2895488083 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55c32da6cda0 0x55c32dead9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.470+0000 7fd3dba98740 4 client.17541 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:59:15.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.470+0000 7fd3dba98740 1 -- 172.21.15.67:0/2895488083 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c32ddf6b00 con 0x55c32da6cda0 2024-07-31T14:59:15.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.473+0000 7fd3ceffd640 1 --2- 172.21.15.67:0/2895488083 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55c32da6cda0 0x55c32dead9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:15.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.474+0000 7fd3ceffd640 1 --2- 172.21.15.67:0/2895488083 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55c32da6cda0 0x55c32dead9c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=563a92aa65fd03d5 server_cookie=8baaf15b58edf1c2 in_seq=0 out_seq=0 2024-07-31T14:59:15.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.474+0000 7fd3b2ff5640 10 client.17541 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:59:15.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.475+0000 7fd3b2ff5640 1 -- 172.21.15.67:0/2895488083 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c32ddf6b00 con 0x55c32da6cda0 2024-07-31T14:59:15.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.475+0000 7fd3b2ff5640 10 client.17541 handle_command_reply: tid=0 2024-07-31T14:59:15.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.475+0000 7fd3dba98740 1 client.17541 shutdown 2024-07-31T14:59:15.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.481+0000 7fd3dba98740 1 -- 172.21.15.67:0/2895488083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 msgr2=0x55c32dea7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:15.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.481+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2895488083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32dea7180 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4015730 tx=0x7fd3c40047d0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.481+0000 7fd3dba98740 1 -- 172.21.15.67:0/2895488083 shutdown_connections 2024-07-31T14:59:15.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.481+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2895488083 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55c32da6cda0 0x55c32dead9c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.481+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2895488083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32dea7180 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.481+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2895488083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 0x55c32de9b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.481+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2895488083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 0x55c32de9b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.481+0000 7fd3b2ff5640 0 client.17541 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:59:15.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.481+0000 7fd3dba98740 1 -- 172.21.15.67:0/2895488083 >> 172.21.15.67:0/2895488083 conn(0x55c32dacd460 msgr2=0x55c32de818a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:15.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.481+0000 7fd3dba98740 1 -- 172.21.15.67:0/2895488083 shutdown_connections 2024-07-31T14:59:15.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.482+0000 7fd3dba98740 1 -- 172.21.15.67:0/2895488083 wait complete. 2024-07-31T14:59:15.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.482+0000 7fd3dba98740 20 client.17541 trim_cache size 0 max 16384 2024-07-31T14:59:15.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:15 smithi077 ceph-mon[37327]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.1 MiB/s wr, 565 op/s 2024-07-31T14:59:15.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.511+0000 7fd3dba98740 1 Processor -- start 2024-07-31T14:59:15.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.512+0000 7fd3dba98740 1 -- start start 2024-07-31T14:59:15.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.512+0000 7fd3dba98740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32de9cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.512+0000 7fd3dba98740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dab8390 0x55c32de83a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.512+0000 7fd3dba98740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dad9070 0x55c32de83f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.512+0000 7fd3dba98740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc0100a0 con 0x55c32da979f0 2024-07-31T14:59:15.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.512+0000 7fd3dba98740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc011d30 con 0x55c32dab8390 2024-07-31T14:59:15.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.512+0000 7fd3dba98740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc010770 con 0x55c32dad9070 2024-07-31T14:59:15.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.512+0000 7fd3ce7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dab8390 0x55c32de83a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:15.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.513+0000 7fd3ce7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dab8390 0x55c32de83a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44442/0 (socket says 172.21.15.67:44442) 2024-07-31T14:59:15.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.513+0000 7fd3ce7fc640 1 -- 172.21.15.67:0/3340272451 learned_addr learned my addr 172.21.15.67:0/3340272451 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:15.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.513+0000 7fd3ceffd640 1 --2- 172.21.15.67:0/3340272451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32de9cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:15.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.513+0000 7fd3ce7fc640 1 -- 172.21.15.67:0/3340272451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dad9070 msgr2=0x55c32de83f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:59:15.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.513+0000 7fd3ce7fc640 1 --2- 172.21.15.67:0/3340272451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dad9070 0x55c32de83f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.513+0000 7fd3ce7fc640 1 -- 172.21.15.67:0/3340272451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 msgr2=0x55c32de9cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:15.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.513+0000 7fd3ce7fc640 1 --2- 172.21.15.67:0/3340272451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32de9cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.513+0000 7fd3ce7fc640 1 -- 172.21.15.67:0/3340272451 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c32de9c150 con 0x55c32dab8390 2024-07-31T14:59:15.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.513+0000 7fd3ce7fc640 1 --2- 172.21.15.67:0/3340272451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dab8390 0x55c32de83a00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd3d010fac0 tx=0x7fd3d010f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7b25d3a3f476090 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:15.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.514+0000 7fd3b4ff9640 1 -- 172.21.15.67:0/3340272451 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d00f5e60 con 0x55c32dab8390 2024-07-31T14:59:15.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.514+0000 7fd3b4ff9640 1 -- 172.21.15.67:0/3340272451 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d01104c0 con 0x55c32dab8390 2024-07-31T14:59:15.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.514+0000 7fd3b4ff9640 1 -- 172.21.15.67:0/3340272451 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d010c050 con 0x55c32dab8390 2024-07-31T14:59:15.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.515+0000 7fd3dba98740 1 -- 172.21.15.67:0/3340272451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dab8390 msgr2=0x55c32de83a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:15.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.516+0000 7fd3dba98740 1 --2- 172.21.15.67:0/3340272451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dab8390 0x55c32de83a00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd3d010fac0 tx=0x7fd3d010f710 comp rx=0 tx=0).stop 2024-07-31T14:59:15.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.516+0000 7fd3dba98740 1 -- 172.21.15.67:0/3340272451 shutdown_connections 2024-07-31T14:59:15.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.516+0000 7fd3dba98740 1 --2- 172.21.15.67:0/3340272451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dad9070 0x55c32de83f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.516+0000 7fd3dba98740 1 --2- 172.21.15.67:0/3340272451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dab8390 0x55c32de83a00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.516+0000 7fd3dba98740 1 --2- 172.21.15.67:0/3340272451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32de9cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.516+0000 7fd3dba98740 1 -- 172.21.15.67:0/3340272451 >> 172.21.15.67:0/3340272451 conn(0x55c32dacd460 msgr2=0x55c32de9c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:15.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.517+0000 7fd3dba98740 1 -- 172.21.15.67:0/3340272451 shutdown_connections 2024-07-31T14:59:15.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.517+0000 7fd3dba98740 1 -- 172.21.15.67:0/3340272451 wait complete. 2024-07-31T14:59:15.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.518+0000 7fd3dba98740 1 -- start start 2024-07-31T14:59:15.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.518+0000 7fd3dba98740 1 Processor -- start 2024-07-31T14:59:15.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.519+0000 7fd3dba98740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32dea3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.519+0000 7fd3dba98740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 0x55c32dea6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.520+0000 7fd3dba98740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 0x55c32dea70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.520+0000 7fd3ce7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 0x55c32dea6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:15.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.520+0000 7fd3ce7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 0x55c32dea6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44900/0 (socket says 172.21.15.67:44900) 2024-07-31T14:59:15.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.520+0000 7fd3ce7fc640 1 -- 172.21.15.67:0/2722603577 learned_addr learned my addr 172.21.15.67:0/2722603577 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:15.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.520+0000 7fd3cf7fe640 1 --2- 172.21.15.67:0/2722603577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 0x55c32dea70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:15.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.520+0000 7fd3dba98740 1 -- 172.21.15.67:0/2722603577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc011660 con 0x55c32da979f0 2024-07-31T14:59:15.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.520+0000 7fd3dba98740 1 -- 172.21.15.67:0/2722603577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc010ed0 con 0x55c32dad9070 2024-07-31T14:59:15.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.520+0000 7fd3dba98740 1 -- 172.21.15.67:0/2722603577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc00e0a0 con 0x55c32dab8390 2024-07-31T14:59:15.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.520+0000 7fd3ceffd640 1 --2- 172.21.15.67:0/2722603577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32dea3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:15.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.521+0000 7fd3ce7fc640 1 -- 172.21.15.67:0/2722603577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 msgr2=0x55c32dea70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:15.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.521+0000 7fd3ce7fc640 1 --2- 172.21.15.67:0/2722603577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 0x55c32dea70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.521+0000 7fd3ce7fc640 1 -- 172.21.15.67:0/2722603577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 msgr2=0x55c32dea3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.521+0000 7fd3ce7fc640 1 --2- 172.21.15.67:0/2722603577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32dea3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.521+0000 7fd3ce7fc640 1 -- 172.21.15.67:0/2722603577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c32dea7850 con 0x55c32dab8390 2024-07-31T14:59:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.521+0000 7fd3ce7fc640 1 --2- 172.21.15.67:0/2722603577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 0x55c32dea6b20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd3d010fa70 tx=0x7fd3d010c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baf9595ea4d23f71 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.521+0000 7fd3dba98740 10 client.0 _ll_register_callbacks cb 0x7fd3c40092d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.521+0000 7fd3dba98740 1 -- 172.21.15.67:0/2722603577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3d01104c0 con 0x55c32dab8390 2024-07-31T14:59:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.522+0000 7fd3ceffd640 1 --2- 172.21.15.67:0/2722603577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32dea3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:59:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.522+0000 7fd3b2ff5640 10 client.27106 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:59:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.522+0000 7fd3b2ff5640 1 -- 172.21.15.67:0/2722603577 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d00f5e60 con 0x55c32dab8390 2024-07-31T14:59:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.522+0000 7fd3b2ff5640 1 -- 172.21.15.67:0/2722603577 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7fd3d0146b80 con 0x55c32dab8390 2024-07-31T14:59:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.522+0000 7fd3b2ff5640 1 -- 172.21.15.67:0/2722603577 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fd3d0118e70 con 0x55c32dab8390 2024-07-31T14:59:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.522+0000 7fd3dba98740 10 client.27106 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:15.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.522+0000 7fd3dba98740 1 -- 172.21.15.67:0/2722603577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=54}) -- 0x55c32dea0090 con 0x55c32dab8390 2024-07-31T14:59:15.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.526+0000 7fd3b2ff5640 1 -- 172.21.15.67:0/2722603577 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7fd3d0085070 con 0x55c32dab8390 2024-07-31T14:59:15.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.527+0000 7fd3dba98740 10 client.27106 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:15.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.527+0000 7fd3dba98740 10 client.27106 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T14:59:15.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.527+0000 7fd3dba98740 20 client.27106 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:15.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.527+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2722603577 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55c32df165b0 0x55c32df189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:15.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.527+0000 7fd3dba98740 4 client.27106 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:59:15.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.527+0000 7fd3dba98740 1 -- 172.21.15.67:0/2722603577 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd3d01104c0 con 0x55c32df165b0 2024-07-31T14:59:15.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.528+0000 7fd3ceffd640 1 --2- 172.21.15.67:0/2722603577 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55c32df165b0 0x55c32df189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:15.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.537+0000 7fd3b2ff5640 10 client.27106 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T14:59:15.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.537+0000 7fd3ceffd640 1 --2- 172.21.15.67:0/2722603577 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55c32df165b0 0x55c32df189d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bca964633758bdf9 server_cookie=98f18b20d00682f7 in_seq=0 out_seq=0 2024-07-31T14:59:15.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.538+0000 7fd3b2ff5640 1 -- 172.21.15.67:0/2722603577 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd3d01104c0 con 0x55c32df165b0 2024-07-31T14:59:15.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.538+0000 7fd3b2ff5640 10 client.27106 handle_command_reply: tid=0 2024-07-31T14:59:15.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.538+0000 7fd3dba98740 1 client.27106 shutdown 2024-07-31T14:59:15.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.541+0000 7fd3dba98740 1 -- 172.21.15.67:0/2722603577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 msgr2=0x55c32dea6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:15.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.541+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2722603577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 0x55c32dea6b20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd3d010fa70 tx=0x7fd3d010c300 comp rx=0 tx=0).stop 2024-07-31T14:59:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.541+0000 7fd3dba98740 1 -- 172.21.15.67:0/2722603577 shutdown_connections 2024-07-31T14:59:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.541+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2722603577 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55c32df165b0 0x55c32df189d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.541+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2722603577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c32dad9070 0x55c32dea70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.541+0000 7fd3b2ff5640 0 client.27106 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T14:59:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.541+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2722603577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c32dab8390 0x55c32dea6b20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.541+0000 7fd3dba98740 1 --2- 172.21.15.67:0/2722603577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c32da979f0 0x55c32dea3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.541+0000 7fd3dba98740 1 -- 172.21.15.67:0/2722603577 >> 172.21.15.67:0/2722603577 conn(0x55c32dacd460 msgr2=0x55c32de7f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.543+0000 7fd3dba98740 1 -- 172.21.15.67:0/2722603577 shutdown_connections 2024-07-31T14:59:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.543+0000 7fd3dba98740 1 -- 172.21.15.67:0/2722603577 wait complete. 2024-07-31T14:59:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.543+0000 7fd3dba98740 20 client.27106 trim_cache size 0 max 16384 2024-07-31T14:59:15.545 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:59:15.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.548+0000 7fd3cffff640 1 -- 172.21.15.67:0/1970408811 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd3b80b3e40 msgr2=0x7fd3b80b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:15.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.548+0000 7fd3cffff640 1 --2- 172.21.15.67:0/1970408811 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd3b80b3e40 0x7fd3b80b6350 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009920 tx=0x7fd3c40090e0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.548+0000 7fd3cffff640 1 -- 172.21.15.67:0/1970408811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c80aa860 msgr2=0x7fd3c8144510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:15.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.548+0000 7fd3cffff640 1 --2- 172.21.15.67:0/1970408811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c80aa860 0x7fd3c8144510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00666a0 tx=0x7fd3d0072ca0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.548+0000 7fd3cffff640 1 -- 172.21.15.67:0/1970408811 shutdown_connections 2024-07-31T14:59:15.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.548+0000 7fd3cffff640 1 --2- 172.21.15.67:0/1970408811 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd3b80b3e40 0x7fd3b80b6350 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.548+0000 7fd3cffff640 1 --2- 172.21.15.67:0/1970408811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3c80ab1b0 0x7fd3c81497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.548+0000 7fd3cffff640 1 --2- 172.21.15.67:0/1970408811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c80aa860 0x7fd3c8144510 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.548+0000 7fd3cffff640 1 --2- 172.21.15.67:0/1970408811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3c80a50f0 0x7fd3c8143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:15.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.548+0000 7fd3cffff640 1 -- 172.21.15.67:0/1970408811 >> 172.21.15.67:0/1970408811 conn(0x7fd3c801cc70 msgr2=0x7fd3c80a39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:15.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.548+0000 7fd3cffff640 1 -- 172.21.15.67:0/1970408811 shutdown_connections 2024-07-31T14:59:15.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:15.548+0000 7fd3cffff640 1 -- 172.21.15.67:0/1970408811 wait complete. 2024-07-31T14:59:16.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:16 smithi067 ceph-mon[32750]: osdmap e94: 12 total, 12 up, 12 in 2024-07-31T14:59:16.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:16 smithi073 ceph-mon[36714]: osdmap e94: 12 total, 12 up, 12 in 2024-07-31T14:59:16.402 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T14:59:16.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:16 smithi077 ceph-mon[37327]: osdmap e94: 12 total, 12 up, 12 in 2024-07-31T14:59:16.965 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:59:17.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:17 smithi067 ceph-mon[32750]: pgmap v650: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 3.6 MiB/s wr, 1.09k op/s 2024-07-31T14:59:17.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:17 smithi073 ceph-mon[36714]: pgmap v650: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 3.6 MiB/s wr, 1.09k op/s 2024-07-31T14:59:17.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:17 smithi077 ceph-mon[37327]: pgmap v650: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 3.6 MiB/s wr, 1.09k op/s 2024-07-31T14:59:18.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.119+0000 7f692f7fe640 1 -- 172.21.15.67:0/4206318261 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562a17ff0170 con 0x562a18115660 2024-07-31T14:59:18.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.119+0000 7f692f7fe640 10 client.27041 handle_command_reply: tid=0 2024-07-31T14:59:18.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.119+0000 7f695ce6d740 1 client.27041 shutdown 2024-07-31T14:59:18.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.120+0000 7f695ce6d740 1 -- 172.21.15.67:0/4206318261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cd7840 msgr2=0x562a1808d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:18.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.120+0000 7f695ce6d740 1 --2- 172.21.15.67:0/4206318261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cd7840 0x562a1808d960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f69440a9a70 tx=0x7f69440c5410 comp rx=0 tx=0).stop 2024-07-31T14:59:18.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.120+0000 7f695ce6d740 1 -- 172.21.15.67:0/4206318261 shutdown_connections 2024-07-31T14:59:18.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.120+0000 7f695ce6d740 1 --2- 172.21.15.67:0/4206318261 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562a18115660 0x562a18117a80 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:18.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.120+0000 7f695ce6d740 1 --2- 172.21.15.67:0/4206318261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a17c6b570 0x562a18113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:18.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.120+0000 7f695ce6d740 1 --2- 172.21.15.67:0/4206318261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a17cd7840 0x562a1808d960 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:18.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.120+0000 7f695ce6d740 1 --2- 172.21.15.67:0/4206318261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562a17cae980 0x562a1808d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:18.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.120+0000 7f695ce6d740 1 -- 172.21.15.67:0/4206318261 >> 172.21.15.67:0/4206318261 conn(0x562a17ccb990 msgr2=0x562a1807f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:18.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.120+0000 7f692f7fe640 0 client.27041 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:18.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.120+0000 7f695ce6d740 1 -- 172.21.15.67:0/4206318261 shutdown_connections 2024-07-31T14:59:18.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.120+0000 7f695ce6d740 1 -- 172.21.15.67:0/4206318261 wait complete. 2024-07-31T14:59:18.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.120+0000 7f695ce6d740 20 client.27041 trim_cache size 0 max 16384 2024-07-31T14:59:18.123 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:59:18.123 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 54, 2024-07-31T14:59:18.123 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T14:59:18.123 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 22, 2024-07-31T14:59:18.123 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T14:59:18.123 INFO:teuthology.orchestra.run.smithi067.stdout: "578e3b69": { 2024-07-31T14:59:18.123 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 22, 2024-07-31T14:59:18.123 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T14:59:18.123 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:59:18.124 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T14:59:18.124 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:59:18.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:59:18.124 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T14:59:18.124 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-07-31T14:59:18.124 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T14:59:18.124 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T14:59:18.124 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T14:59:18.124 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:59:18.124 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T14:59:18.124 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:59:18.124 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:59:18.124 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:59:18.125 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:59:18.125 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:59:18.125 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:59:18.125 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:59:18.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.121+0000 7f695691c640 1 -- 172.21.15.67:0/1860407684 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69380bc900 msgr2=0x7f69380bee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:18.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.121+0000 7f695691c640 1 --2- 172.21.15.67:0/1860407684 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69380bc900 0x7f69380bee10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6940006010 tx=0x7f6940005850 comp rx=0 tx=0).stop 2024-07-31T14:59:18.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.121+0000 7f695691c640 1 -- 172.21.15.67:0/1860407684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6950107df0 msgr2=0x7f695006d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:18.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.121+0000 7f695691c640 1 --2- 172.21.15.67:0/1860407684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6950107df0 0x7f695006d080 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f695006a4f0 tx=0x7f6944007c00 comp rx=0 tx=0).stop 2024-07-31T14:59:18.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.122+0000 7f695691c640 1 -- 172.21.15.67:0/1860407684 shutdown_connections 2024-07-31T14:59:18.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.122+0000 7f695691c640 1 --2- 172.21.15.67:0/1860407684 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69380bc900 0x7f69380bee10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:18.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.122+0000 7f695691c640 1 --2- 172.21.15.67:0/1860407684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6950108740 0x7f6950072340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:18.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.122+0000 7f695691c640 1 --2- 172.21.15.67:0/1860407684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6950107df0 0x7f695006d080 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:18.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.122+0000 7f695691c640 1 --2- 172.21.15.67:0/1860407684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69501024b0 0x7f695006cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:18.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.122+0000 7f695691c640 1 -- 172.21.15.67:0/1860407684 >> 172.21.15.67:0/1860407684 conn(0x7f69500792a0 msgr2=0x7f6950105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:18.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.122+0000 7f695691c640 1 -- 172.21.15.67:0/1860407684 shutdown_connections 2024-07-31T14:59:18.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:18.122+0000 7f695691c640 1 -- 172.21.15.67:0/1860407684 wait complete. 2024-07-31T14:59:19.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:19 smithi067 ceph-mon[32750]: pgmap v651: 129 pgs: 7 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 102 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 3.3 MiB/s wr, 1.08k op/s 2024-07-31T14:59:19.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:19 smithi073 ceph-mon[36714]: pgmap v651: 129 pgs: 7 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 102 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 3.3 MiB/s wr, 1.08k op/s 2024-07-31T14:59:19.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.421+0000 7f6577fff640 1 -- 172.21.15.67:0/294300248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f657810f330 msgr2=0x7f657810f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:19.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.422+0000 7f6577fff640 1 --2- 172.21.15.67:0/294300248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f657810f330 0x7f657810f740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f656c009900 tx=0x7f656c02a410 comp rx=0 tx=0).stop 2024-07-31T14:59:19.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.428+0000 7f6577fff640 1 -- 172.21.15.67:0/294300248 shutdown_connections 2024-07-31T14:59:19.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.428+0000 7f6577fff640 1 --2- 172.21.15.67:0/294300248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f657810fc80 0x7f65781164b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.428+0000 7f6577fff640 1 --2- 172.21.15.67:0/294300248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f657810f330 0x7f657810f740 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.428+0000 7f6577fff640 1 --2- 172.21.15.67:0/294300248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65781099f0 0x7f6578109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.428+0000 7f6577fff640 1 -- 172.21.15.67:0/294300248 >> 172.21.15.67:0/294300248 conn(0x7f6578078ff0 msgr2=0x7f6578107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:19.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.432+0000 7f6577fff640 1 -- 172.21.15.67:0/294300248 shutdown_connections 2024-07-31T14:59:19.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.433+0000 7f6577fff640 1 -- 172.21.15.67:0/294300248 wait complete. 2024-07-31T14:59:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.434+0000 7f6577fff640 1 Processor -- start 2024-07-31T14:59:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.434+0000 7f6577fff640 1 -- start start 2024-07-31T14:59:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.434+0000 7f6577fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65781099f0 0x7f657807b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.434+0000 7f6577fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f657810f330 0x7f657807beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.434+0000 7f6577fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f657810fc80 0x7f6578081170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.434+0000 7f6577fff640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f65781199d0 con 0x7f657810f330 2024-07-31T14:59:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.434+0000 7f6577fff640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6578119850 con 0x7f657810fc80 2024-07-31T14:59:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.434+0000 7f6577fff640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6578119b50 con 0x7f65781099f0 2024-07-31T14:59:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.435+0000 7f65777fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f657810fc80 0x7f6578081170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.435+0000 7f65777fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f657810fc80 0x7f6578081170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44474/0 (socket says 172.21.15.67:44474) 2024-07-31T14:59:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.435+0000 7f65777fe640 1 -- 172.21.15.67:0/2005946382 learned_addr learned my addr 172.21.15.67:0/2005946382 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.436+0000 7f65767fc640 1 --2- 172.21.15.67:0/2005946382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f657810f330 0x7f657807beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.436+0000 7f65777fe640 1 -- 172.21.15.67:0/2005946382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65781099f0 msgr2=0x7f657807b930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T14:59:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.436+0000 7f65777fe640 1 --2- 172.21.15.67:0/2005946382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65781099f0 0x7f657807b930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.436+0000 7f65777fe640 1 -- 172.21.15.67:0/2005946382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f657810f330 msgr2=0x7f657807beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.436+0000 7f65777fe640 1 --2- 172.21.15.67:0/2005946382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f657810f330 0x7f657807beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.436+0000 7f65777fe640 1 -- 172.21.15.67:0/2005946382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65780818a0 con 0x7f657810fc80 2024-07-31T14:59:19.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.442+0000 7f65777fe640 1 --2- 172.21.15.67:0/2005946382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f657810fc80 0x7f6578081170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f656802d9e0 tx=0x7f656802dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:19.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.452+0000 7f655ffff640 1 -- 172.21.15.67:0/2005946382 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656803b070 con 0x7f657810fc80 2024-07-31T14:59:19.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.452+0000 7f655ffff640 1 -- 172.21.15.67:0/2005946382 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f656802eba0 con 0x7f657810fc80 2024-07-31T14:59:19.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.452+0000 7f6577fff640 1 -- 172.21.15.67:0/2005946382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6578081b30 con 0x7f657810fc80 2024-07-31T14:59:19.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.452+0000 7f655ffff640 1 -- 172.21.15.67:0/2005946382 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65680295e0 con 0x7f657810fc80 2024-07-31T14:59:19.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.452+0000 7f6577fff640 1 -- 172.21.15.67:0/2005946382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65780822e0 con 0x7f657810fc80 2024-07-31T14:59:19.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.458+0000 7f6583ad8740 1 Processor -- start 2024-07-31T14:59:19.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.459+0000 7f6583ad8740 1 -- start start 2024-07-31T14:59:19.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.460+0000 7f6583ad8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013877070 0x561013c1c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.460+0000 7f6583ad8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56101384e1b0 0x561013c1cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.460+0000 7f6583ad8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013856390 0x561013c1d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.460+0000 7f6583ad8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561013b92bb0 con 0x56101384e1b0 2024-07-31T14:59:19.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.460+0000 7f6583ad8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561013b820b0 con 0x561013856390 2024-07-31T14:59:19.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.460+0000 7f6583ad8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561013b860f0 con 0x561013877070 2024-07-31T14:59:19.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.464+0000 7f65767fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013877070 0x561013c1c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.464+0000 7f65767fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013856390 0x561013c1d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.464+0000 7f65767fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013877070 0x561013c1c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44914/0 (socket says 172.21.15.67:44914) 2024-07-31T14:59:19.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.464+0000 7f65767fc640 1 -- 172.21.15.67:0/1919435751 learned_addr learned my addr 172.21.15.67:0/1919435751 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:19.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.465+0000 7f6576ffd640 1 --2- 172.21.15.67:0/1919435751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56101384e1b0 0x561013c1cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.467+0000 7f65767fc640 1 -- 172.21.15.67:0/1919435751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013856390 msgr2=0x561013c1d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:19.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.467+0000 7f65767fc640 1 --2- 172.21.15.67:0/1919435751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013856390 0x561013c1d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.467+0000 7f65767fc640 1 -- 172.21.15.67:0/1919435751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56101384e1b0 msgr2=0x561013c1cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:19.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.467+0000 7f65767fc640 1 --2- 172.21.15.67:0/1919435751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56101384e1b0 0x561013c1cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.467+0000 7f65767fc640 1 -- 172.21.15.67:0/1919435751 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561013b28b90 con 0x561013877070 2024-07-31T14:59:19.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.468+0000 7f65767fc640 1 --2- 172.21.15.67:0/1919435751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013877070 0x561013c1c6f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f656c006010 tx=0x7f656c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a1d3b4f21ce46ed server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:19.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.468+0000 7f655d7fa640 1 -- 172.21.15.67:0/1919435751 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c035050 con 0x561013877070 2024-07-31T14:59:19.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.468+0000 7f655d7fa640 1 -- 172.21.15.67:0/1919435751 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f656c005560 con 0x561013877070 2024-07-31T14:59:19.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.468+0000 7f655d7fa640 1 -- 172.21.15.67:0/1919435751 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c0038e0 con 0x561013877070 2024-07-31T14:59:19.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.469+0000 7f6583ad8740 1 -- 172.21.15.67:0/1919435751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013877070 msgr2=0x561013c1c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:19.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.469+0000 7f6583ad8740 1 --2- 172.21.15.67:0/1919435751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013877070 0x561013c1c6f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f656c006010 tx=0x7f656c000bc0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.472+0000 7f6583ad8740 1 -- 172.21.15.67:0/1919435751 shutdown_connections 2024-07-31T14:59:19.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.472+0000 7f6583ad8740 1 --2- 172.21.15.67:0/1919435751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013856390 0x561013c1d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.472+0000 7f6583ad8740 1 --2- 172.21.15.67:0/1919435751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56101384e1b0 0x561013c1cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.472+0000 7f6583ad8740 1 --2- 172.21.15.67:0/1919435751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013877070 0x561013c1c6f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.472+0000 7f6583ad8740 1 -- 172.21.15.67:0/1919435751 >> 172.21.15.67:0/1919435751 conn(0x56101386b460 msgr2=0x561013c1a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:19.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.472+0000 7f655ffff640 1 -- 172.21.15.67:0/2005946382 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6568036440 con 0x7f657810fc80 2024-07-31T14:59:19.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.474+0000 7f655ffff640 1 --2- 172.21.15.67:0/2005946382 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65600b3f50 0x7f65600b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.474+0000 7f655ffff640 1 -- 172.21.15.67:0/2005946382 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f6568105860 con 0x7f657810fc80 2024-07-31T14:59:19.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.476+0000 7f6583ad8740 1 -- 172.21.15.67:0/1919435751 shutdown_connections 2024-07-31T14:59:19.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.477+0000 7f6576ffd640 1 --2- 172.21.15.67:0/2005946382 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65600b3f50 0x7f65600b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.477+0000 7f6583ad8740 1 -- 172.21.15.67:0/1919435751 wait complete. 2024-07-31T14:59:19.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.478+0000 7f6583ad8740 1 -- start start 2024-07-31T14:59:19.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.478+0000 7f6583ad8740 1 Processor -- start 2024-07-31T14:59:19.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.479+0000 7f6576ffd640 1 --2- 172.21.15.67:0/2005946382 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65600b3f50 0x7f65600b6460 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6570005fd0 tx=0x7f6570005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:19.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.479+0000 7f6583ad8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56101384e1b0 0x561013c282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.479+0000 7f6583ad8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561013856390 0x561013c287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.479+0000 7f6583ad8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013877070 0x561013c341e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.479+0000 7f6583ad8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5610139e5700 con 0x561013856390 2024-07-31T14:59:19.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.479+0000 7f6583ad8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561013b249e0 con 0x561013877070 2024-07-31T14:59:19.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.479+0000 7f6583ad8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561013c2aec0 con 0x56101384e1b0 2024-07-31T14:59:19.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.479+0000 7f6583ad8740 10 client.0 _ll_register_callbacks cb 0x561013b3f4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:19.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.479+0000 7f6576ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56101384e1b0 0x561013c282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.479+0000 7f6576ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56101384e1b0 0x561013c282b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44928/0 (socket says 172.21.15.67:44928) 2024-07-31T14:59:19.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.479+0000 7f6576ffd640 1 -- 172.21.15.67:0/2671794751 learned_addr learned my addr 172.21.15.67:0/2671794751 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:19.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.480+0000 7f6576ffd640 1 -- 172.21.15.67:0/2671794751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013877070 msgr2=0x561013c341e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:59:19.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.480+0000 7f6576ffd640 1 --2- 172.21.15.67:0/2671794751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013877070 0x561013c341e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.480+0000 7f6576ffd640 1 -- 172.21.15.67:0/2671794751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561013856390 msgr2=0x561013c287f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:19.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.480+0000 7f6576ffd640 1 --2- 172.21.15.67:0/2671794751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561013856390 0x561013c287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.480+0000 7f6576ffd640 1 -- 172.21.15.67:0/2671794751 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610138e2650 con 0x56101384e1b0 2024-07-31T14:59:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.481+0000 7f6576ffd640 1 --2- 172.21.15.67:0/2671794751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56101384e1b0 0x561013c282b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f657000bfd0 tx=0x7f65700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60fd077d83fb0e9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.481+0000 7f655aff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:59:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.481+0000 7f655aff5640 1 -- 172.21.15.67:0/2671794751 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570022070 con 0x56101384e1b0 2024-07-31T14:59:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.482+0000 7f6583ad8740 1 -- 172.21.15.67:0/2671794751 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f656c005560 con 0x56101384e1b0 2024-07-31T14:59:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.482+0000 7f655aff5640 1 -- 172.21.15.67:0/2671794751 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f657001e650 con 0x56101384e1b0 2024-07-31T14:59:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.482+0000 7f655aff5640 1 -- 172.21.15.67:0/2671794751 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f657002e050 con 0x56101384e1b0 2024-07-31T14:59:19.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.482+0000 7f6583ad8740 10 client.27112 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:19.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.482+0000 7f6583ad8740 1 -- 172.21.15.67:0/2671794751 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x561013c36f00 con 0x56101384e1b0 2024-07-31T14:59:19.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:19 smithi077 ceph-mon[37327]: pgmap v651: 129 pgs: 7 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 102 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 3.3 MiB/s wr, 1.08k op/s 2024-07-31T14:59:19.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.489+0000 7f655aff5640 1 -- 172.21.15.67:0/2671794751 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f656c005560 con 0x56101384e1b0 2024-07-31T14:59:19.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.489+0000 7f655aff5640 1 -- 172.21.15.67:0/2671794751 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f6570010aa0 con 0x56101384e1b0 2024-07-31T14:59:19.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.490+0000 7f6583ad8740 10 client.27112 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:19.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.490+0000 7f6583ad8740 10 client.27112 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:59:19.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.490+0000 7f6583ad8740 20 client.27112 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:19.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.490+0000 7f6583ad8740 1 --2- 172.21.15.67:0/2671794751 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5610138359f0 0x561013c3a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.490+0000 7f6583ad8740 4 client.27112 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:59:19.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.490+0000 7f6583ad8740 1 -- 172.21.15.67:0/2671794751 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561013b94b00 con 0x5610138359f0 2024-07-31T14:59:19.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.497+0000 7f65767fc640 1 --2- 172.21.15.67:0/2671794751 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5610138359f0 0x561013c3a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.499+0000 7f65767fc640 1 --2- 172.21.15.67:0/2671794751 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5610138359f0 0x561013c3a4b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e12bba84771d8b5 server_cookie=31522ec04faebb09 in_seq=0 out_seq=0 2024-07-31T14:59:19.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.499+0000 7f655aff5640 10 client.27112 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:59:19.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.502+0000 7f655aff5640 1 -- 172.21.15.67:0/2671794751 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561013b94b00 con 0x5610138359f0 2024-07-31T14:59:19.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.502+0000 7f655aff5640 10 client.27112 handle_command_reply: tid=0 2024-07-31T14:59:19.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.502+0000 7f6583ad8740 1 client.27112 shutdown 2024-07-31T14:59:19.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.506+0000 7f6583ad8740 1 -- 172.21.15.67:0/2671794751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56101384e1b0 msgr2=0x561013c282b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:19.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.506+0000 7f6583ad8740 1 --2- 172.21.15.67:0/2671794751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56101384e1b0 0x561013c282b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f657000bfd0 tx=0x7f65700046c0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.508+0000 7f6583ad8740 1 -- 172.21.15.67:0/2671794751 shutdown_connections 2024-07-31T14:59:19.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.508+0000 7f6583ad8740 1 --2- 172.21.15.67:0/2671794751 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5610138359f0 0x561013c3a4b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.508+0000 7f6583ad8740 1 --2- 172.21.15.67:0/2671794751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013877070 0x561013c341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.508+0000 7f6583ad8740 1 --2- 172.21.15.67:0/2671794751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561013856390 0x561013c287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.508+0000 7f6583ad8740 1 --2- 172.21.15.67:0/2671794751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56101384e1b0 0x561013c282b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.508+0000 7f6583ad8740 1 -- 172.21.15.67:0/2671794751 >> 172.21.15.67:0/2671794751 conn(0x56101386b460 msgr2=0x561013c20240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:19.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.508+0000 7f655aff5640 0 client.27112 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:59:19.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.509+0000 7f6583ad8740 1 -- 172.21.15.67:0/2671794751 shutdown_connections 2024-07-31T14:59:19.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.509+0000 7f6583ad8740 1 -- 172.21.15.67:0/2671794751 wait complete. 2024-07-31T14:59:19.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.509+0000 7f6583ad8740 20 client.27112 trim_cache size 0 max 16384 2024-07-31T14:59:19.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.536+0000 7f6583ad8740 1 Processor -- start 2024-07-31T14:59:19.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.536+0000 7f6583ad8740 1 -- start start 2024-07-31T14:59:19.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.536+0000 7f6583ad8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56101384e1b0 0x561013c21f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.536+0000 7f6583ad8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561013856390 0x561013c33fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.536+0000 7f6583ad8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013877070 0x561013c34510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.536+0000 7f6583ad8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f65781a85d0 con 0x561013856390 2024-07-31T14:59:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.536+0000 7f6583ad8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f65781aa260 con 0x56101384e1b0 2024-07-31T14:59:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.536+0000 7f6583ad8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f65781a8ca0 con 0x561013877070 2024-07-31T14:59:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.536+0000 7f65767fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561013856390 0x561013c33fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.536+0000 7f65767fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561013856390 0x561013c33fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60780/0 (socket says 172.21.15.67:60780) 2024-07-31T14:59:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.536+0000 7f65767fc640 1 -- 172.21.15.67:0/3009779100 learned_addr learned my addr 172.21.15.67:0/3009779100 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.536+0000 7f6576ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56101384e1b0 0x561013c21f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.536+0000 7f65777fe640 1 --2- 172.21.15.67:0/3009779100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013877070 0x561013c34510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.537+0000 7f65767fc640 1 -- 172.21.15.67:0/3009779100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013877070 msgr2=0x561013c34510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:19.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.537+0000 7f65767fc640 1 --2- 172.21.15.67:0/3009779100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013877070 0x561013c34510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.537+0000 7f65767fc640 1 -- 172.21.15.67:0/3009779100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56101384e1b0 msgr2=0x561013c21f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:19.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.537+0000 7f65767fc640 1 --2- 172.21.15.67:0/3009779100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56101384e1b0 0x561013c21f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.537+0000 7f65767fc640 1 -- 172.21.15.67:0/3009779100 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561013c27010 con 0x561013856390 2024-07-31T14:59:19.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.537+0000 7f65767fc640 1 --2- 172.21.15.67:0/3009779100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561013856390 0x561013c33fd0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f656c044000 tx=0x7f656c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afc3b5bdf4999ed server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:19.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.537+0000 7f655cff9640 1 -- 172.21.15.67:0/3009779100 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c035040 con 0x561013856390 2024-07-31T14:59:19.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.537+0000 7f655cff9640 1 -- 172.21.15.67:0/3009779100 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f656c005560 con 0x561013856390 2024-07-31T14:59:19.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.537+0000 7f655cff9640 1 -- 172.21.15.67:0/3009779100 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c003830 con 0x561013856390 2024-07-31T14:59:19.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.538+0000 7f6583ad8740 1 -- 172.21.15.67:0/3009779100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561013856390 msgr2=0x561013c33fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:19.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.538+0000 7f6583ad8740 1 --2- 172.21.15.67:0/3009779100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561013856390 0x561013c33fd0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f656c044000 tx=0x7f656c031400 comp rx=0 tx=0).stop 2024-07-31T14:59:19.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.538+0000 7f6583ad8740 1 -- 172.21.15.67:0/3009779100 shutdown_connections 2024-07-31T14:59:19.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.538+0000 7f6583ad8740 1 --2- 172.21.15.67:0/3009779100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013877070 0x561013c34510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.538+0000 7f6583ad8740 1 --2- 172.21.15.67:0/3009779100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561013856390 0x561013c33fd0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.538+0000 7f6583ad8740 1 --2- 172.21.15.67:0/3009779100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56101384e1b0 0x561013c21f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.538+0000 7f6583ad8740 1 -- 172.21.15.67:0/3009779100 >> 172.21.15.67:0/3009779100 conn(0x56101386b460 msgr2=0x561013c21800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.538+0000 7f6583ad8740 1 -- 172.21.15.67:0/3009779100 shutdown_connections 2024-07-31T14:59:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.538+0000 7f6583ad8740 1 -- 172.21.15.67:0/3009779100 wait complete. 2024-07-31T14:59:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.538+0000 7f6583ad8740 1 -- start start 2024-07-31T14:59:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6583ad8740 1 Processor -- start 2024-07-31T14:59:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6583ad8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56101384e1b0 0x561013c2b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6583ad8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013856390 0x561013c2ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6583ad8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013877070 0x561013ca0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6583ad8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f65781a9b90 con 0x56101384e1b0 2024-07-31T14:59:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6583ad8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f65781a9400 con 0x561013877070 2024-07-31T14:59:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6583ad8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f65781a65d0 con 0x561013856390 2024-07-31T14:59:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6583ad8740 10 client.0 _ll_register_callbacks cb 0x7f6570015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6576ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56101384e1b0 0x561013c2b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6576ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56101384e1b0 0x561013c2b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60790/0 (socket says 172.21.15.67:60790) 2024-07-31T14:59:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6576ffd640 1 -- 172.21.15.67:0/281925051 learned_addr learned my addr 172.21.15.67:0/281925051 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f65767fc640 1 --2- 172.21.15.67:0/281925051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013856390 0x561013c2ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f65777fe640 1 --2- 172.21.15.67:0/281925051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013877070 0x561013ca0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6576ffd640 1 -- 172.21.15.67:0/281925051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013856390 msgr2=0x561013c2ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6576ffd640 1 --2- 172.21.15.67:0/281925051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013856390 0x561013c2ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6576ffd640 1 -- 172.21.15.67:0/281925051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013877070 msgr2=0x561013ca0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6576ffd640 1 --2- 172.21.15.67:0/281925051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013877070 0x561013ca0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f6576ffd640 1 -- 172.21.15.67:0/281925051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561013c2c320 con 0x56101384e1b0 2024-07-31T14:59:19.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.539+0000 7f65777fe640 1 --2- 172.21.15.67:0/281925051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013877070 0x561013ca0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:59:19.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f6576ffd640 1 --2- 172.21.15.67:0/281925051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56101384e1b0 0x561013c2b510 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6570010e40 tx=0x7f65700267d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36fbbb60b5d29db server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:19.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f655aff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:59:19.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f655aff5640 1 -- 172.21.15.67:0/281925051 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570037020 con 0x56101384e1b0 2024-07-31T14:59:19.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f655aff5640 1 -- 172.21.15.67:0/281925051 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f65700103d0 con 0x56101384e1b0 2024-07-31T14:59:19.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f6583ad8740 1 -- 172.21.15.67:0/281925051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f656c005560 con 0x56101384e1b0 2024-07-31T14:59:19.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f655aff5640 1 -- 172.21.15.67:0/281925051 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f6570042050 con 0x56101384e1b0 2024-07-31T14:59:19.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f6583ad8740 10 client.17562 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:19.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f6583ad8740 1 -- 172.21.15.67:0/281925051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x561013ca2220 con 0x56101384e1b0 2024-07-31T14:59:19.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f655aff5640 1 -- 172.21.15.67:0/281925051 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f656c005560 con 0x56101384e1b0 2024-07-31T14:59:19.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f655aff5640 1 -- 172.21.15.67:0/281925051 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f657001e810 con 0x56101384e1b0 2024-07-31T14:59:19.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f6583ad8740 10 client.17562 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:19.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f6583ad8740 10 client.17562 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T14:59:19.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f6583ad8740 20 client.17562 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:19.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f6583ad8740 1 --2- 172.21.15.67:0/281925051 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56101380ada0 0x561013ca4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:19.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f6583ad8740 4 client.17562 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:59:19.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.540+0000 7f6583ad8740 1 -- 172.21.15.67:0/281925051 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561013b905a0 con 0x56101380ada0 2024-07-31T14:59:19.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.541+0000 7f65767fc640 1 --2- 172.21.15.67:0/281925051 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56101380ada0 0x561013ca4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:19.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.541+0000 7f65767fc640 1 --2- 172.21.15.67:0/281925051 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56101380ada0 0x561013ca4a60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aec6601188837a06 server_cookie=27aa8d2ebd8c46ed in_seq=0 out_seq=0 2024-07-31T14:59:19.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.541+0000 7f655aff5640 10 client.17562 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T14:59:19.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.541+0000 7f655aff5640 1 -- 172.21.15.67:0/281925051 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561013b905a0 con 0x56101380ada0 2024-07-31T14:59:19.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.541+0000 7f655aff5640 10 client.17562 handle_command_reply: tid=0 2024-07-31T14:59:19.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.541+0000 7f6583ad8740 1 client.17562 shutdown 2024-07-31T14:59:19.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6583ad8740 1 -- 172.21.15.67:0/281925051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56101384e1b0 msgr2=0x561013c2b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:19.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6583ad8740 1 --2- 172.21.15.67:0/281925051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56101384e1b0 0x561013c2b510 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6570010e40 tx=0x7f65700267d0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6583ad8740 1 -- 172.21.15.67:0/281925051 shutdown_connections 2024-07-31T14:59:19.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6583ad8740 1 --2- 172.21.15.67:0/281925051 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x56101380ada0 0x561013ca4a60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6583ad8740 1 --2- 172.21.15.67:0/281925051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561013877070 0x561013ca0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6583ad8740 1 --2- 172.21.15.67:0/281925051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561013856390 0x561013c2ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6583ad8740 1 --2- 172.21.15.67:0/281925051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56101384e1b0 0x561013c2b510 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f655aff5640 0 client.17562 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T14:59:19.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6583ad8740 1 -- 172.21.15.67:0/281925051 >> 172.21.15.67:0/281925051 conn(0x56101386b460 msgr2=0x561013c29560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:19.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6583ad8740 1 -- 172.21.15.67:0/281925051 shutdown_connections 2024-07-31T14:59:19.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6583ad8740 1 -- 172.21.15.67:0/281925051 wait complete. 2024-07-31T14:59:19.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6583ad8740 20 client.17562 trim_cache size 0 max 16384 2024-07-31T14:59:19.550 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:59:19.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6577fff640 1 -- 172.21.15.67:0/2005946382 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65600b3f50 msgr2=0x7f65600b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:19.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6577fff640 1 --2- 172.21.15.67:0/2005946382 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65600b3f50 0x7f65600b6460 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6570005fd0 tx=0x7f6570005850 comp rx=0 tx=0).stop 2024-07-31T14:59:19.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6577fff640 1 -- 172.21.15.67:0/2005946382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f657810fc80 msgr2=0x7f6578081170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:19.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6577fff640 1 --2- 172.21.15.67:0/2005946382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f657810fc80 0x7f6578081170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f656802d9e0 tx=0x7f656802dea0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6577fff640 1 -- 172.21.15.67:0/2005946382 shutdown_connections 2024-07-31T14:59:19.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6577fff640 1 --2- 172.21.15.67:0/2005946382 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65600b3f50 0x7f65600b6460 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6577fff640 1 --2- 172.21.15.67:0/2005946382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f657810fc80 0x7f6578081170 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6577fff640 1 --2- 172.21.15.67:0/2005946382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f657810f330 0x7f657807beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6577fff640 1 --2- 172.21.15.67:0/2005946382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65781099f0 0x7f657807b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:19.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.542+0000 7f6577fff640 1 -- 172.21.15.67:0/2005946382 >> 172.21.15.67:0/2005946382 conn(0x7f6578078ff0 msgr2=0x7f6578107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:19.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.543+0000 7f6577fff640 1 -- 172.21.15.67:0/2005946382 shutdown_connections 2024-07-31T14:59:19.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:19.543+0000 7f6577fff640 1 -- 172.21.15.67:0/2005946382 wait complete. 2024-07-31T14:59:20.541 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '578e3b69', quiesce took 30.8 seconds. Will release after: -0.8000000000000007 2024-07-31T14:59:20.541 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T14:59:21.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:21 smithi067 ceph-mon[32750]: pgmap v652: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 1.04k op/s 2024-07-31T14:59:21.371 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T14:59:21.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:21 smithi073 ceph-mon[36714]: pgmap v652: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 1.04k op/s 2024-07-31T14:59:21.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:21 smithi077 ceph-mon[37327]: pgmap v652: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 1.04k op/s 2024-07-31T14:59:21.567 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:59:23.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:23 smithi067 ceph-mon[32750]: pgmap v653: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 991 op/s 2024-07-31T14:59:23.329 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:59:23.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:23 smithi073 ceph-mon[36714]: pgmap v653: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 991 op/s 2024-07-31T14:59:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:23 smithi077 ceph-mon[37327]: pgmap v653: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 991 op/s 2024-07-31T14:59:24.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.660+0000 7f0fa0981640 1 -- 172.21.15.67:0/3404561779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c10a330 msgr2=0x7f0f9c10a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:24.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.660+0000 7f0fa0981640 1 --2- 172.21.15.67:0/3404561779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c10a330 0x7f0f9c10a740 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009be0 tx=0x7f0f9402a790 comp rx=0 tx=0).stop 2024-07-31T14:59:24.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.663+0000 7f0fa0981640 1 -- 172.21.15.67:0/3404561779 shutdown_connections 2024-07-31T14:59:24.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.663+0000 7f0fa0981640 1 --2- 172.21.15.67:0/3404561779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c11c5d0 0x7f0f9c11e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:24.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.663+0000 7f0fa0981640 1 --2- 172.21.15.67:0/3404561779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c10a330 0x7f0f9c10a740 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:24.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.663+0000 7f0fa0981640 1 --2- 172.21.15.67:0/3404561779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f9c111790 0x7f0f9c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:24.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.663+0000 7f0fa0981640 1 -- 172.21.15.67:0/3404561779 >> 172.21.15.67:0/3404561779 conn(0x7f0f9c06e420 msgr2=0x7f0f9c06e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:24.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.664+0000 7f0fa0981640 1 -- 172.21.15.67:0/3404561779 shutdown_connections 2024-07-31T14:59:24.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.664+0000 7f0fa0981640 1 -- 172.21.15.67:0/3404561779 wait complete. 2024-07-31T14:59:24.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.665+0000 7f0fa0981640 1 Processor -- start 2024-07-31T14:59:24.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.665+0000 7f0fa0981640 1 -- start start 2024-07-31T14:59:24.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.665+0000 7f0fa0981640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c10a330 0x7f0f9c11be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:24.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.665+0000 7f0fa0981640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f9c111790 0x7f0f9c118150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:24.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.665+0000 7f0fa0981640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c11c5d0 0x7f0f9c1186f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:24.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.665+0000 7f0fa0981640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c114740 con 0x7f0f9c11c5d0 2024-07-31T14:59:24.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.666+0000 7f0fa0981640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1145c0 con 0x7f0f9c10a330 2024-07-31T14:59:24.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.666+0000 7f0fa0981640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1148c0 con 0x7f0f9c111790 2024-07-31T14:59:24.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.667+0000 7f0f9affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f9c111790 0x7f0f9c118150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:24.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.667+0000 7f0f9affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f9c111790 0x7f0f9c118150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60028/0 (socket says 172.21.15.67:60028) 2024-07-31T14:59:24.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.667+0000 7f0f9affd640 1 -- 172.21.15.67:0/3728999240 learned_addr learned my addr 172.21.15.67:0/3728999240 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:24.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.668+0000 7f0f9affd640 1 -- 172.21.15.67:0/3728999240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c10a330 msgr2=0x7f0f9c11be70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T14:59:24.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.668+0000 7f0f9affd640 1 --2- 172.21.15.67:0/3728999240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c10a330 0x7f0f9c11be70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:24.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.668+0000 7f0f9affd640 1 -- 172.21.15.67:0/3728999240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c11c5d0 msgr2=0x7f0f9c1186f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:24.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.668+0000 7f0f9affd640 1 --2- 172.21.15.67:0/3728999240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c11c5d0 0x7f0f9c1186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:24.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.668+0000 7f0f9affd640 1 -- 172.21.15.67:0/3728999240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c118f10 con 0x7f0f9c111790 2024-07-31T14:59:24.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.669+0000 7f0f9affd640 1 --2- 172.21.15.67:0/3728999240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f9c111790 0x7f0f9c118150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f94034040 tx=0x7f0f94003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:24.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.670+0000 7f0f98ff9640 1 -- 172.21.15.67:0/3728999240 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94041020 con 0x7f0f9c111790 2024-07-31T14:59:24.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.670+0000 7f0fa0981640 1 -- 172.21.15.67:0/3728999240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9c07d0d0 con 0x7f0f9c111790 2024-07-31T14:59:24.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.670+0000 7f0fa0981640 1 -- 172.21.15.67:0/3728999240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f9c07d5b0 con 0x7f0f9c111790 2024-07-31T14:59:24.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.671+0000 7f0f98ff9640 1 -- 172.21.15.67:0/3728999240 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f940318f0 con 0x7f0f9c111790 2024-07-31T14:59:24.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.671+0000 7f0f98ff9640 1 -- 172.21.15.67:0/3728999240 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94036040 con 0x7f0f9c111790 2024-07-31T14:59:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.673+0000 7f0fa0981640 1 -- 172.21.15.67:0/3728999240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f9c10a740 con 0x7f0f9c111790 2024-07-31T14:59:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.677+0000 7f0f98ff9640 1 -- 172.21.15.67:0/3728999240 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0f94044020 con 0x7f0f9c111790 2024-07-31T14:59:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.678+0000 7f0f98ff9640 1 --2- 172.21.15.67:0/3728999240 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f840b3f50 0x7f0f840b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.679+0000 7f0f98ff9640 1 -- 172.21.15.67:0/3728999240 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f0f94104a20 con 0x7f0f9c111790 2024-07-31T14:59:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.679+0000 7f0f9b7fe640 1 --2- 172.21.15.67:0/3728999240 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f840b3f50 0x7f0f840b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:24.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.682+0000 7f0f9b7fe640 1 --2- 172.21.15.67:0/3728999240 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f840b3f50 0x7f0f840b6460 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c10a250 tx=0x7f0f8c01a270 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:24.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:24.687+0000 7f0f98ff9640 1 -- 172.21.15.67:0/3728999240 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f0f9404b070 con 0x7f0f9c111790 2024-07-31T14:59:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T14:59:25.098 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":54,"btime":"2024-07-31T14:58:48:655841+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T14:58:48.279201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T14:59:25.099 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 54 2024-07-31T14:59:25.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:25.092+0000 7f0fa0981640 1 -- 172.21.15.67:0/3728999240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f9c07d9e0 con 0x7f0f9c111790 2024-07-31T14:59:25.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:25.095+0000 7f0f98ff9640 1 -- 172.21.15.67:0/3728999240 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11971 (secure 0 0 0) 0x7f0f940cb910 con 0x7f0f9c111790 2024-07-31T14:59:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:25.103+0000 7f0fa0981640 1 -- 172.21.15.67:0/3728999240 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f840b3f50 msgr2=0x7f0f840b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:25.103+0000 7f0fa0981640 1 --2- 172.21.15.67:0/3728999240 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f840b3f50 0x7f0f840b6460 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c10a250 tx=0x7f0f8c01a270 comp rx=0 tx=0).stop 2024-07-31T14:59:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:25.103+0000 7f0fa0981640 1 -- 172.21.15.67:0/3728999240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f9c111790 msgr2=0x7f0f9c118150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:25.103+0000 7f0fa0981640 1 --2- 172.21.15.67:0/3728999240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f9c111790 0x7f0f9c118150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f94034040 tx=0x7f0f94003c30 comp rx=0 tx=0).stop 2024-07-31T14:59:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:25.104+0000 7f0fa0981640 1 -- 172.21.15.67:0/3728999240 shutdown_connections 2024-07-31T14:59:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:25.104+0000 7f0fa0981640 1 --2- 172.21.15.67:0/3728999240 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0f840b3f50 0x7f0f840b6460 secure :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c10a250 tx=0x7f0f8c01a270 comp rx=0 tx=0).stop 2024-07-31T14:59:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:25.104+0000 7f0fa0981640 1 --2- 172.21.15.67:0/3728999240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9c11c5d0 0x7f0f9c1186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:25.104+0000 7f0fa0981640 1 --2- 172.21.15.67:0/3728999240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f9c111790 0x7f0f9c118150 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:25.104+0000 7f0fa0981640 1 --2- 172.21.15.67:0/3728999240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c10a330 0x7f0f9c11be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:25.104+0000 7f0fa0981640 1 -- 172.21.15.67:0/3728999240 >> 172.21.15.67:0/3728999240 conn(0x7f0f9c06e420 msgr2=0x7f0f9c073680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:25.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:25.106+0000 7f0fa0981640 1 -- 172.21.15.67:0/3728999240 shutdown_connections 2024-07-31T14:59:25.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:25.106+0000 7f0fa0981640 1 -- 172.21.15.67:0/3728999240 wait complete. 2024-07-31T14:59:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:25 smithi067 ceph-mon[32750]: pgmap v654: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1005 KiB/s wr, 544 op/s 2024-07-31T14:59:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:25 smithi073 ceph-mon[36714]: pgmap v654: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1005 KiB/s wr, 544 op/s 2024-07-31T14:59:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:25 smithi077 ceph-mon[37327]: pgmap v654: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1005 KiB/s wr, 544 op/s 2024-07-31T14:59:26.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:26 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3728999240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:59:26.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:26 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3728999240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:59:26.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:26 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3728999240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T14:59:27.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:27 smithi073 ceph-mon[36714]: pgmap v655: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 875 KiB/s wr, 323 op/s 2024-07-31T14:59:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:27 smithi077 ceph-mon[37327]: pgmap v655: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 875 KiB/s wr, 323 op/s 2024-07-31T14:59:27.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:27 smithi067 ceph-mon[32750]: pgmap v655: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 875 KiB/s wr, 323 op/s 2024-07-31T14:59:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.884+0000 7f4201387640 1 -- 172.21.15.67:0/2725456078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41fc102b60 msgr2=0x7f41fc102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.884+0000 7f4201387640 1 --2- 172.21.15.67:0/2725456078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41fc102b60 0x7f41fc102f70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009ca0 tx=0x7f41ec02a8c0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.884+0000 7f4201387640 1 -- 172.21.15.67:0/2725456078 shutdown_connections 2024-07-31T14:59:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.884+0000 7f4201387640 1 --2- 172.21.15.67:0/2725456078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41fc108790 0x7f41fc10efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.884+0000 7f4201387640 1 --2- 172.21.15.67:0/2725456078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41fc107e40 0x7f41fc108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.884+0000 7f4201387640 1 --2- 172.21.15.67:0/2725456078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41fc102b60 0x7f41fc102f70 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.884+0000 7f4201387640 1 -- 172.21.15.67:0/2725456078 >> 172.21.15.67:0/2725456078 conn(0x7f41fc079880 msgr2=0x7f41fc079c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.884+0000 7f4201387640 1 -- 172.21.15.67:0/2725456078 shutdown_connections 2024-07-31T14:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.884+0000 7f4201387640 1 -- 172.21.15.67:0/2725456078 wait complete. 2024-07-31T14:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.885+0000 7f4201387640 1 Processor -- start 2024-07-31T14:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.885+0000 7f4201387640 1 -- start start 2024-07-31T14:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.885+0000 7f4201387640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41fc102b60 0x7f41fc19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.885+0000 7f4201387640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41fc107e40 0x7f41fc19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f4201387640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41fc108790 0x7f41fc19b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f4201387640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc112440 con 0x7f41fc108790 2024-07-31T14:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f4201387640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1122c0 con 0x7f41fc107e40 2024-07-31T14:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f4201387640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1125c0 con 0x7f41fc102b60 2024-07-31T14:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f41fa7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41fc107e40 0x7f41fc19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f41fb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41fc108790 0x7f41fc19b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f41fa7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41fc107e40 0x7f41fc19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48930/0 (socket says 172.21.15.67:48930) 2024-07-31T14:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f41faffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41fc102b60 0x7f41fc19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f41fa7fc640 1 -- 172.21.15.67:0/4092082224 learned_addr learned my addr 172.21.15.67:0/4092082224 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f41fa7fc640 1 -- 172.21.15.67:0/4092082224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41fc102b60 msgr2=0x7f41fc19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f41fa7fc640 1 --2- 172.21.15.67:0/4092082224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41fc102b60 0x7f41fc19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f41fa7fc640 1 -- 172.21.15.67:0/4092082224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41fc108790 msgr2=0x7f41fc19b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f41fa7fc640 1 --2- 172.21.15.67:0/4092082224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41fc108790 0x7f41fc19b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f41fa7fc640 1 -- 172.21.15.67:0/4092082224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc1a6040 con 0x7f41fc107e40 2024-07-31T14:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f41fa7fc640 1 --2- 172.21.15.67:0/4092082224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41fc107e40 0x7f41fc19b2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f41e400e9f0 tx=0x7f41e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f41f3fff640 1 -- 172.21.15.67:0/4092082224 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e400ccc0 con 0x7f41fc107e40 2024-07-31T14:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f41f3fff640 1 -- 172.21.15.67:0/4092082224 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41e40108e0 con 0x7f41fc107e40 2024-07-31T14:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f4201387640 1 -- 172.21.15.67:0/4092082224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41fc1a6330 con 0x7f41fc107e40 2024-07-31T14:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f41f3fff640 1 -- 172.21.15.67:0/4092082224 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4018860 con 0x7f41fc107e40 2024-07-31T14:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.886+0000 7f4201387640 1 -- 172.21.15.67:0/4092082224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41fc1a6880 con 0x7f41fc107e40 2024-07-31T14:59:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.889+0000 7f4208284740 1 Processor -- start 2024-07-31T14:59:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.889+0000 7f41f3fff640 1 -- 172.21.15.67:0/4092082224 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f41e4010430 con 0x7f41fc107e40 2024-07-31T14:59:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.889+0000 7f4208284740 1 -- start start 2024-07-31T14:59:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.889+0000 7f4208284740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d483ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.889+0000 7f4208284740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 0x557a5d484220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.889+0000 7f4208284740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 0x557a5d484760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.889+0000 7f4208284740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557a5d3f93b0 con 0x557a5d0b82f0 2024-07-31T14:59:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f4208284740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a5d3e88b0 con 0x557a5d0e11b0 2024-07-31T14:59:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f4208284740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a5d3ec8f0 con 0x557a5d0c04d0 2024-07-31T14:59:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41fb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d483ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41faffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 0x557a5d484220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41fb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d483ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48946/0 (socket says 172.21.15.67:48946) 2024-07-31T14:59:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41fb7fe640 1 -- 172.21.15.67:0/1831378024 learned_addr learned my addr 172.21.15.67:0/1831378024 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41fa7fc640 1 --2- 172.21.15.67:0/1831378024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 0x557a5d484760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41faffd640 1 -- 172.21.15.67:0/1831378024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 msgr2=0x557a5d484760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41faffd640 1 --2- 172.21.15.67:0/1831378024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 0x557a5d484760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41faffd640 1 -- 172.21.15.67:0/1831378024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 msgr2=0x557a5d483ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41faffd640 1 --2- 172.21.15.67:0/1831378024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d483ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41faffd640 1 -- 172.21.15.67:0/1831378024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a5d38f9a0 con 0x557a5d0b82f0 2024-07-31T14:59:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41fb7fe640 1 --2- 172.21.15.67:0/1831378024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d483ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:59:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41f3fff640 1 --2- 172.21.15.67:0/4092082224 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41e80b3d40 0x7f41e80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41faffd640 1 --2- 172.21.15.67:0/1831378024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 0x557a5d484220 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f41ec006010 tx=0x7f41ec009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1973af178e09ff0d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41fb7fe640 1 --2- 172.21.15.67:0/4092082224 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41e80b3d40 0x7f41e80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41f3fff640 1 -- 172.21.15.67:0/4092082224 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f41e40e4650 con 0x7f41fc107e40 2024-07-31T14:59:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.890+0000 7f41f17fa640 1 -- 172.21.15.67:0/1831378024 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec031cf0 con 0x557a5d0b82f0 2024-07-31T14:59:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.891+0000 7f41f17fa640 1 -- 172.21.15.67:0/1831378024 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41ec036030 con 0x557a5d0b82f0 2024-07-31T14:59:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.891+0000 7f41f17fa640 1 -- 172.21.15.67:0/1831378024 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0042a0 con 0x557a5d0b82f0 2024-07-31T14:59:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.891+0000 7f41fb7fe640 1 --2- 172.21.15.67:0/4092082224 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41e80b3d40 0x7f41e80b6250 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f41f4012560 tx=0x7f41f4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.891+0000 7f4208284740 1 -- 172.21.15.67:0/1831378024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 msgr2=0x557a5d484220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.891+0000 7f4208284740 1 --2- 172.21.15.67:0/1831378024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 0x557a5d484220 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f41ec006010 tx=0x7f41ec009410 comp rx=0 tx=0).stop 2024-07-31T14:59:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.891+0000 7f4208284740 1 -- 172.21.15.67:0/1831378024 shutdown_connections 2024-07-31T14:59:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.891+0000 7f4208284740 1 --2- 172.21.15.67:0/1831378024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 0x557a5d484760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.891+0000 7f4208284740 1 --2- 172.21.15.67:0/1831378024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 0x557a5d484220 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.891+0000 7f4208284740 1 --2- 172.21.15.67:0/1831378024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d483ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.891+0000 7f4208284740 1 -- 172.21.15.67:0/1831378024 >> 172.21.15.67:0/1831378024 conn(0x557a5d0d5300 msgr2=0x557a5d481bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.892+0000 7f4208284740 1 -- 172.21.15.67:0/1831378024 shutdown_connections 2024-07-31T14:59:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.892+0000 7f4208284740 1 -- 172.21.15.67:0/1831378024 wait complete. 2024-07-31T14:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.893+0000 7f4208284740 1 -- start start 2024-07-31T14:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.893+0000 7f4208284740 1 Processor -- start 2024-07-31T14:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.894+0000 7f4208284740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 0x557a5d48f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.894+0000 7f4208284740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 0x557a5d48fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.894+0000 7f4208284740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d495570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.894+0000 7f4208284740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557a5d322ab0 con 0x557a5d0b82f0 2024-07-31T14:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.894+0000 7f4208284740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a5d370cf0 con 0x557a5d0e11b0 2024-07-31T14:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.894+0000 7f4208284740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a5d492280 con 0x557a5d0c04d0 2024-07-31T14:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.894+0000 7f4208284740 10 client.0 _ll_register_callbacks cb 0x557a5d3e29f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.894+0000 7f41fb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d495570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.894+0000 7f41fb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d495570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48952/0 (socket says 172.21.15.67:48952) 2024-07-31T14:59:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.894+0000 7f41fb7fe640 1 -- 172.21.15.67:0/3427783430 learned_addr learned my addr 172.21.15.67:0/3427783430 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.898+0000 7f41faffd640 1 --2- 172.21.15.67:0/3427783430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 0x557a5d48f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.900+0000 7f41fb7fe640 1 -- 172.21.15.67:0/3427783430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 msgr2=0x557a5d48fcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:59:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.900+0000 7f41fb7fe640 1 --2- 172.21.15.67:0/3427783430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 0x557a5d48fcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.900+0000 7f41fb7fe640 1 -- 172.21.15.67:0/3427783430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 msgr2=0x557a5d48f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.900+0000 7f41fb7fe640 1 --2- 172.21.15.67:0/3427783430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 0x557a5d48f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.900+0000 7f41fb7fe640 1 -- 172.21.15.67:0/3427783430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a5d14c790 con 0x557a5d0e11b0 2024-07-31T14:59:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.901+0000 7f41fb7fe640 1 --2- 172.21.15.67:0/3427783430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d495570 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f41f400fa30 tx=0x7f41f4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a28fc0668d286a4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.903+0000 7f41e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:59:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.903+0000 7f41e27fc640 1 -- 172.21.15.67:0/3427783430 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f402d060 con 0x557a5d0e11b0 2024-07-31T14:59:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.903+0000 7f41e27fc640 1 -- 172.21.15.67:0/3427783430 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f41f4018030 con 0x557a5d0e11b0 2024-07-31T14:59:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.904+0000 7f4208284740 1 -- 172.21.15.67:0/3427783430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41ec036030 con 0x557a5d0e11b0 2024-07-31T14:59:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.912+0000 7f41e27fc640 1 -- 172.21.15.67:0/3427783430 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f41f4004020 con 0x557a5d0e11b0 2024-07-31T14:59:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.917+0000 7f4208284740 10 client.27137 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.917+0000 7f4208284740 1 -- 172.21.15.67:0/3427783430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x557a5d1c5620 con 0x557a5d0e11b0 2024-07-31T14:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.919+0000 7f41e27fc640 1 -- 172.21.15.67:0/3427783430 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41ec036030 con 0x557a5d0e11b0 2024-07-31T14:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.921+0000 7f41e27fc640 1 -- 172.21.15.67:0/3427783430 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f41f40035f0 con 0x557a5d0e11b0 2024-07-31T14:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.921+0000 7f4208284740 10 client.27137 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.921+0000 7f4208284740 10 client.27137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.921+0000 7f4208284740 20 client.27137 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.921+0000 7f4208284740 1 --2- 172.21.15.67:0/3427783430 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a5d09fd80 0x557a5d49b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.921+0000 7f4208284740 4 client.27137 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:59:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.921+0000 7f4208284740 1 -- 172.21.15.67:0/3427783430 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a5d3fb300 con 0x557a5d09fd80 2024-07-31T14:59:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.924+0000 7f41faffd640 1 --2- 172.21.15.67:0/3427783430 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a5d09fd80 0x557a5d49b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.925+0000 7f41faffd640 1 --2- 172.21.15.67:0/3427783430 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a5d09fd80 0x557a5d49b6c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=659de5f72c538ee0 server_cookie=464c2ddd4d7ea796 in_seq=0 out_seq=0 2024-07-31T14:59:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.925+0000 7f41e27fc640 10 client.27137 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.928+0000 7f41e27fc640 1 -- 172.21.15.67:0/3427783430 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a5d3fb300 con 0x557a5d09fd80 2024-07-31T14:59:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.928+0000 7f41e27fc640 10 client.27137 handle_command_reply: tid=0 2024-07-31T14:59:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.928+0000 7f4208284740 1 client.27137 shutdown 2024-07-31T14:59:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.930+0000 7f4208284740 1 -- 172.21.15.67:0/3427783430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 msgr2=0x557a5d495570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.930+0000 7f4208284740 1 --2- 172.21.15.67:0/3427783430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d495570 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f41f400fa30 tx=0x7f41f4004730 comp rx=0 tx=0).stop 2024-07-31T14:59:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.934+0000 7f4208284740 1 -- 172.21.15.67:0/3427783430 shutdown_connections 2024-07-31T14:59:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.934+0000 7f4208284740 1 --2- 172.21.15.67:0/3427783430 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a5d09fd80 0x557a5d49b6c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.934+0000 7f4208284740 1 --2- 172.21.15.67:0/3427783430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d495570 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.934+0000 7f4208284740 1 --2- 172.21.15.67:0/3427783430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 0x557a5d48fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.934+0000 7f4208284740 1 --2- 172.21.15.67:0/3427783430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 0x557a5d48f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.934+0000 7f4208284740 1 -- 172.21.15.67:0/3427783430 >> 172.21.15.67:0/3427783430 conn(0x557a5d0d5300 msgr2=0x557a5d484d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.934+0000 7f41e27fc640 0 client.27137 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.934+0000 7f4208284740 1 -- 172.21.15.67:0/3427783430 shutdown_connections 2024-07-31T14:59:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.934+0000 7f4208284740 1 -- 172.21.15.67:0/3427783430 wait complete. 2024-07-31T14:59:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.935+0000 7f4208284740 20 client.27137 trim_cache size 0 max 16384 2024-07-31T14:59:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.969+0000 7f4208284740 1 Processor -- start 2024-07-31T14:59:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.969+0000 7f4208284740 1 -- start start 2024-07-31T14:59:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.970+0000 7f4208284740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0b82f0 0x557a5d485420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.970+0000 7f4208284740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0c04d0 0x557a5d489db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.970+0000 7f41faffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0b82f0 0x557a5d485420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.970+0000 7f41faffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0b82f0 0x557a5d485420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48958/0 (socket says 172.21.15.67:48958) 2024-07-31T14:59:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.970+0000 7f41faffd640 1 -- 172.21.15.67:0/3138486405 learned_addr learned my addr 172.21.15.67:0/3138486405 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.970+0000 7f4208284740 1 --2- 172.21.15.67:0/3138486405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0e11b0 0x557a5d4954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.970+0000 7f4208284740 1 -- 172.21.15.67:0/3138486405 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1ae5d0 con 0x557a5d0c04d0 2024-07-31T14:59:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.970+0000 7f4208284740 1 -- 172.21.15.67:0/3138486405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b0260 con 0x557a5d0b82f0 2024-07-31T14:59:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.971+0000 7f41fb7fe640 1 --2- 172.21.15.67:0/3138486405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0e11b0 0x557a5d4954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.971+0000 7f4208284740 1 -- 172.21.15.67:0/3138486405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1aeca0 con 0x557a5d0e11b0 2024-07-31T14:59:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.975+0000 7f41faffd640 1 -- 172.21.15.67:0/3138486405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0e11b0 msgr2=0x557a5d4954e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.975+0000 7f41faffd640 1 --2- 172.21.15.67:0/3138486405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0e11b0 0x557a5d4954e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.975+0000 7f41faffd640 1 -- 172.21.15.67:0/3138486405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0c04d0 msgr2=0x557a5d489db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.975+0000 7f41faffd640 1 --2- 172.21.15.67:0/3138486405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0c04d0 0x557a5d489db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.975+0000 7f41faffd640 1 -- 172.21.15.67:0/3138486405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a5d486210 con 0x557a5d0b82f0 2024-07-31T14:59:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.977+0000 7f41faffd640 1 --2- 172.21.15.67:0/3138486405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0b82f0 0x557a5d485420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f41ec02ade0 tx=0x7f41ec003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edeb5df63721d14a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.978+0000 7f41f0ff9640 1 -- 172.21.15.67:0/3138486405 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec060070 con 0x557a5d0b82f0 2024-07-31T14:59:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.979+0000 7f41f0ff9640 1 -- 172.21.15.67:0/3138486405 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41ec036070 con 0x557a5d0b82f0 2024-07-31T14:59:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.980+0000 7f4208284740 1 -- 172.21.15.67:0/3138486405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0b82f0 msgr2=0x557a5d485420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.980+0000 7f4208284740 1 --2- 172.21.15.67:0/3138486405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0b82f0 0x557a5d485420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f41ec02ade0 tx=0x7f41ec003d20 comp rx=0 tx=0).stop 2024-07-31T14:59:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.987+0000 7f4208284740 1 -- 172.21.15.67:0/3138486405 shutdown_connections 2024-07-31T14:59:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.987+0000 7f4208284740 1 --2- 172.21.15.67:0/3138486405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0e11b0 0x557a5d4954e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.987+0000 7f4208284740 1 --2- 172.21.15.67:0/3138486405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0c04d0 0x557a5d489db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.987+0000 7f4208284740 1 --2- 172.21.15.67:0/3138486405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0b82f0 0x557a5d485420 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.987+0000 7f4208284740 1 -- 172.21.15.67:0/3138486405 >> 172.21.15.67:0/3138486405 conn(0x557a5d0d5300 msgr2=0x557a5d484d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.992+0000 7f4208284740 1 -- 172.21.15.67:0/3138486405 shutdown_connections 2024-07-31T14:59:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.997+0000 7f4208284740 1 -- 172.21.15.67:0/3138486405 wait complete. 2024-07-31T14:59:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.997+0000 7f4208284740 1 -- start start 2024-07-31T14:59:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:27.997+0000 7f4208284740 1 Processor -- start 2024-07-31T14:59:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.002+0000 7f4208284740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 0x557a5d49ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.002+0000 7f4208284740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 0x557a5d49d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.003+0000 7f4208284740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d49d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.003+0000 7f4208284740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1afb90 con 0x557a5d0b82f0 2024-07-31T14:59:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.003+0000 7f4208284740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1af400 con 0x557a5d0e11b0 2024-07-31T14:59:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.003+0000 7f4208284740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1ac660 con 0x557a5d0c04d0 2024-07-31T14:59:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.003+0000 7f4208284740 10 client.0 _ll_register_callbacks cb 0x7f41f4009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.004+0000 7f41fa7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 0x557a5d49d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.004+0000 7f41fa7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 0x557a5d49d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60092/0 (socket says 172.21.15.67:60092) 2024-07-31T14:59:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.004+0000 7f41fa7fc640 1 -- 172.21.15.67:0/3599788257 learned_addr learned my addr 172.21.15.67:0/3599788257 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.004+0000 7f41faffd640 1 --2- 172.21.15.67:0/3599788257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 0x557a5d49ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.004+0000 7f41fa7fc640 1 -- 172.21.15.67:0/3599788257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 msgr2=0x557a5d49d880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:59:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.004+0000 7f41fa7fc640 1 --2- 172.21.15.67:0/3599788257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d49d880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.004+0000 7f41fa7fc640 1 -- 172.21.15.67:0/3599788257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 msgr2=0x557a5d49ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.004+0000 7f41fa7fc640 1 --2- 172.21.15.67:0/3599788257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 0x557a5d49ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.004+0000 7f41fa7fc640 1 -- 172.21.15.67:0/3599788257 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a5d48f2f0 con 0x557a5d0c04d0 2024-07-31T14:59:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.009+0000 7f41fa7fc640 1 --2- 172.21.15.67:0/3599788257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 0x557a5d49d340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f41e400c6a0 tx=0x7f41e40096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ef705397b14a298 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.009+0000 7f41e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:59:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.009+0000 7f41e27fc640 1 -- 172.21.15.67:0/3599788257 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e40aa6c0 con 0x557a5d0c04d0 2024-07-31T14:59:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.010+0000 7f4208284740 1 -- 172.21.15.67:0/3599788257 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41ec036070 con 0x557a5d0c04d0 2024-07-31T14:59:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.011+0000 7f41e27fc640 1 -- 172.21.15.67:0/3599788257 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f41e40a7540 con 0x557a5d0c04d0 2024-07-31T14:59:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.011+0000 7f41e27fc640 1 -- 172.21.15.67:0/3599788257 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f41e4010720 con 0x557a5d0c04d0 2024-07-31T14:59:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.013+0000 7f4208284740 10 client.27139 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.013+0000 7f4208284740 1 -- 172.21.15.67:0/3599788257 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x557a5d51a340 con 0x557a5d0c04d0 2024-07-31T14:59:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.018+0000 7f41e27fc640 1 -- 172.21.15.67:0/3599788257 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41ec036070 con 0x557a5d0c04d0 2024-07-31T14:59:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.018+0000 7f41e27fc640 1 -- 172.21.15.67:0/3599788257 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f41e40e4230 con 0x557a5d0c04d0 2024-07-31T14:59:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.019+0000 7f4208284740 10 client.27139 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.019+0000 7f4208284740 10 client.27139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.019+0000 7f4208284740 20 client.27139 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.019+0000 7f4208284740 1 --2- 172.21.15.67:0/3599788257 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a5d0751c0 0x557a5d51cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.019+0000 7f4208284740 4 client.27139 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:59:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.019+0000 7f4208284740 1 -- 172.21.15.67:0/3599788257 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a5d3f6da0 con 0x557a5d0751c0 2024-07-31T14:59:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.020+0000 7f41faffd640 1 --2- 172.21.15.67:0/3599788257 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a5d0751c0 0x557a5d51cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:28.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.026+0000 7f41faffd640 1 --2- 172.21.15.67:0/3599788257 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a5d0751c0 0x557a5d51cb20 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db9975510167ce28 server_cookie=24ebb844cb6efa78 in_seq=0 out_seq=0 2024-07-31T14:59:28.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.026+0000 7f41e27fc640 10 client.27139 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.439+0000 7f41e27fc640 1 -- 172.21.15.67:0/3599788257 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a5d3f6da0 con 0x557a5d0751c0 2024-07-31T14:59:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.439+0000 7f41e27fc640 10 client.27139 handle_command_reply: tid=0 2024-07-31T14:59:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.439+0000 7f4208284740 1 client.27139 shutdown 2024-07-31T14:59:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.439+0000 7f4208284740 1 -- 172.21.15.67:0/3599788257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 msgr2=0x557a5d49d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.439+0000 7f4208284740 1 --2- 172.21.15.67:0/3599788257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 0x557a5d49d340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f41e400c6a0 tx=0x7f41e40096a0 comp rx=0 tx=0).stop 2024-07-31T14:59:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.440+0000 7f4208284740 1 -- 172.21.15.67:0/3599788257 shutdown_connections 2024-07-31T14:59:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.440+0000 7f4208284740 1 --2- 172.21.15.67:0/3599788257 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a5d0751c0 0x557a5d51cb20 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.440+0000 7f4208284740 1 --2- 172.21.15.67:0/3599788257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a5d0e11b0 0x557a5d49d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.440+0000 7f4208284740 1 --2- 172.21.15.67:0/3599788257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a5d0c04d0 0x557a5d49d340 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.440+0000 7f4208284740 1 --2- 172.21.15.67:0/3599788257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a5d0b82f0 0x557a5d49ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.440+0000 7f4208284740 1 -- 172.21.15.67:0/3599788257 >> 172.21.15.67:0/3599788257 conn(0x557a5d0d5300 msgr2=0x557a5d490920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.440+0000 7f41e27fc640 0 client.27139 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.440+0000 7f4208284740 1 -- 172.21.15.67:0/3599788257 shutdown_connections 2024-07-31T14:59:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.440+0000 7f4208284740 1 -- 172.21.15.67:0/3599788257 wait complete. 2024-07-31T14:59:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.440+0000 7f4208284740 20 client.27139 trim_cache size 0 max 16384 2024-07-31T14:59:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:59:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:59:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:59:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:59:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.441+0000 7f4201387640 1 -- 172.21.15.67:0/4092082224 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41e80b3d40 msgr2=0x7f41e80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.441+0000 7f4201387640 1 --2- 172.21.15.67:0/4092082224 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41e80b3d40 0x7f41e80b6250 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f41f4012560 tx=0x7f41f4005f90 comp rx=0 tx=0).stop 2024-07-31T14:59:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.441+0000 7f4201387640 1 -- 172.21.15.67:0/4092082224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41fc107e40 msgr2=0x7f41fc19b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.441+0000 7f4201387640 1 --2- 172.21.15.67:0/4092082224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41fc107e40 0x7f41fc19b2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f41e400e9f0 tx=0x7f41e400eeb0 comp rx=0 tx=0).stop 2024-07-31T14:59:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.441+0000 7f4201387640 1 -- 172.21.15.67:0/4092082224 shutdown_connections 2024-07-31T14:59:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.441+0000 7f4201387640 1 --2- 172.21.15.67:0/4092082224 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f41e80b3d40 0x7f41e80b6250 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.441+0000 7f4201387640 1 --2- 172.21.15.67:0/4092082224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41fc108790 0x7f41fc19b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.441+0000 7f4201387640 1 --2- 172.21.15.67:0/4092082224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41fc107e40 0x7f41fc19b2f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.441+0000 7f4201387640 1 --2- 172.21.15.67:0/4092082224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41fc102b60 0x7f41fc19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.441+0000 7f4201387640 1 -- 172.21.15.67:0/4092082224 >> 172.21.15.67:0/4092082224 conn(0x7f41fc079880 msgr2=0x7f41fc105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.442+0000 7f4201387640 1 -- 172.21.15.67:0/4092082224 shutdown_connections 2024-07-31T14:59:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:28.442+0000 7f4201387640 1 -- 172.21.15.67:0/4092082224 wait complete. 2024-07-31T14:59:29.096 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 578e3b69 --release --await' 2024-07-31T14:59:29.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 578e3b69 --release --await 2024-07-31T14:59:29.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:29 smithi067 ceph-mon[32750]: pgmap v656: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 723 KiB/s wr, 105 op/s 2024-07-31T14:59:29.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:29 smithi073 ceph-mon[36714]: pgmap v656: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 723 KiB/s wr, 105 op/s 2024-07-31T14:59:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:29 smithi077 ceph-mon[37327]: pgmap v656: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 723 KiB/s wr, 105 op/s 2024-07-31T14:59:29.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.701+0000 7fe4702d5640 1 -- 172.21.15.67:0/3022792820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe46810a080 msgr2=0x7fe46810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.701+0000 7fe4702d5640 1 --2- 172.21.15.67:0/3022792820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe46810a080 0x7fe46810a490 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe464009900 tx=0x7fe46402a440 comp rx=0 tx=0).stop 2024-07-31T14:59:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.704+0000 7fe4702d5640 1 -- 172.21.15.67:0/3022792820 shutdown_connections 2024-07-31T14:59:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.704+0000 7fe4702d5640 1 --2- 172.21.15.67:0/3022792820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe46810d050 0x7fe468116ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.704+0000 7fe4702d5640 1 --2- 172.21.15.67:0/3022792820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe46810c700 0x7fe46810cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.704+0000 7fe4702d5640 1 --2- 172.21.15.67:0/3022792820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe46810a080 0x7fe46810a490 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.704+0000 7fe4702d5640 1 -- 172.21.15.67:0/3022792820 >> 172.21.15.67:0/3022792820 conn(0x7fe468070d20 msgr2=0x7fe468073880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.713+0000 7fe4702d5640 1 -- 172.21.15.67:0/3022792820 shutdown_connections 2024-07-31T14:59:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.718+0000 7fe4702d5640 1 -- 172.21.15.67:0/3022792820 wait complete. 2024-07-31T14:59:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.718+0000 7fe4702d5640 1 Processor -- start 2024-07-31T14:59:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.719+0000 7fe4702d5640 1 -- start start 2024-07-31T14:59:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.719+0000 7fe4702d5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe46810a080 0x7fe4681a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.719+0000 7fe4702d5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe46810c700 0x7fe4681a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.719+0000 7fe4702d5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe46810d050 0x7fe4681a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.719+0000 7fe4702d5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe468119bd0 con 0x7fe46810d050 2024-07-31T14:59:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.719+0000 7fe4702d5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe468119a50 con 0x7fe46810a080 2024-07-31T14:59:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.719+0000 7fe4702d5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe468119d50 con 0x7fe46810c700 2024-07-31T14:59:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.720+0000 7fe46e071640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe46810a080 0x7fe4681a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.722+0000 7fe46d870640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe46810c700 0x7fe4681a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:29.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.723+0000 7fe46e071640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe46810a080 0x7fe4681a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48972/0 (socket says 172.21.15.67:48972) 2024-07-31T14:59:29.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.723+0000 7fe46e071640 1 -- 172.21.15.67:0/3127734138 learned_addr learned my addr 172.21.15.67:0/3127734138 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:29.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.725+0000 7fe46e071640 1 -- 172.21.15.67:0/3127734138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe46810c700 msgr2=0x7fe4681a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:29.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.725+0000 7fe46e071640 1 --2- 172.21.15.67:0/3127734138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe46810c700 0x7fe4681a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.725+0000 7fe46e071640 1 -- 172.21.15.67:0/3127734138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe46810d050 msgr2=0x7fe4681a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:29.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.725+0000 7fe46e071640 1 --2- 172.21.15.67:0/3127734138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe46810d050 0x7fe4681a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.725+0000 7fe46e071640 1 -- 172.21.15.67:0/3127734138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4681a6510 con 0x7fe46810a080 2024-07-31T14:59:29.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.736+0000 7fe46e071640 1 --2- 172.21.15.67:0/3127734138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe46810a080 0x7fe4681a05a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe46402a9b0 tx=0x7fe464003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:29.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.738+0000 7fe45f7fe640 1 -- 172.21.15.67:0/3127734138 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46403f070 con 0x7fe46810a080 2024-07-31T14:59:29.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.738+0000 7fe45f7fe640 1 -- 172.21.15.67:0/3127734138 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe464031b50 con 0x7fe46810a080 2024-07-31T14:59:29.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.738+0000 7fe45f7fe640 1 -- 172.21.15.67:0/3127734138 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464035030 con 0x7fe46810a080 2024-07-31T14:59:29.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.739+0000 7fe4702d5640 1 -- 172.21.15.67:0/3127734138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4681a67a0 con 0x7fe46810a080 2024-07-31T14:59:29.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.739+0000 7fe4702d5640 1 -- 172.21.15.67:0/3127734138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4681acf60 con 0x7fe46810a080 2024-07-31T14:59:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.743+0000 7fe476826740 1 Processor -- start 2024-07-31T14:59:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.743+0000 7fe476826740 1 -- start start 2024-07-31T14:59:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.744+0000 7fe476826740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 0x5567170f3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.744+0000 7fe476826740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d1c1b0 0x5567170f3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.744+0000 7fe476826740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d24390 0x5567170f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.744+0000 7fe476826740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556717060bb0 con 0x556716d1c1b0 2024-07-31T14:59:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.744+0000 7fe476826740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5567170500b0 con 0x556716d45070 2024-07-31T14:59:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.744+0000 7fe476826740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5567170540f0 con 0x556716d24390 2024-07-31T14:59:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.746+0000 7fe46d870640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d24390 0x5567170f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.746+0000 7fe46d870640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d24390 0x5567170f3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60120/0 (socket says 172.21.15.67:60120) 2024-07-31T14:59:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.746+0000 7fe46d870640 1 -- 172.21.15.67:0/363918174 learned_addr learned my addr 172.21.15.67:0/363918174 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.746+0000 7fe46e872640 1 --2- 172.21.15.67:0/363918174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 0x5567170f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.746+0000 7fe46d870640 1 -- 172.21.15.67:0/363918174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 msgr2=0x5567170f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.746+0000 7fe46d870640 1 --2- 172.21.15.67:0/363918174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 0x5567170f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.746+0000 7fe46d870640 1 -- 172.21.15.67:0/363918174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d1c1b0 msgr2=0x5567170f3570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:59:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.746+0000 7fe46d870640 1 --2- 172.21.15.67:0/363918174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d1c1b0 0x5567170f3570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.747+0000 7fe46d870640 1 -- 172.21.15.67:0/363918174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556716ff6b90 con 0x556716d24390 2024-07-31T14:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.748+0000 7fe45f7fe640 1 -- 172.21.15.67:0/3127734138 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe464042070 con 0x7fe46810a080 2024-07-31T14:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.748+0000 7fe46d870640 1 --2- 172.21.15.67:0/363918174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d24390 0x5567170f3ab0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe458009c20 tx=0x7fe458011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1257893e18b9b6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.750+0000 7fe45f7fe640 1 --2- 172.21.15.67:0/3127734138 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe4540b3f40 0x7fe4540b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.750+0000 7fe45f7fe640 1 -- 172.21.15.67:0/3127734138 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7fe464104db0 con 0x7fe46810a080 2024-07-31T14:59:29.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.751+0000 7fe453fff640 1 -- 172.21.15.67:0/363918174 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458004050 con 0x556716d24390 2024-07-31T14:59:29.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.751+0000 7fe453fff640 1 -- 172.21.15.67:0/363918174 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4580054d0 con 0x556716d24390 2024-07-31T14:59:29.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.751+0000 7fe453fff640 1 -- 172.21.15.67:0/363918174 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458004d20 con 0x556716d24390 2024-07-31T14:59:29.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.752+0000 7fe46e872640 1 --2- 172.21.15.67:0/3127734138 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe4540b3f40 0x7fe4540b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:29.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.753+0000 7fe46e872640 1 --2- 172.21.15.67:0/3127734138 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe4540b3f40 0x7fe4540b6450 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe460009f20 tx=0x7fe46000d040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.757+0000 7fe476826740 1 -- 172.21.15.67:0/363918174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d24390 msgr2=0x5567170f3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.757+0000 7fe476826740 1 --2- 172.21.15.67:0/363918174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d24390 0x5567170f3ab0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe458009c20 tx=0x7fe458011810 comp rx=0 tx=0).stop 2024-07-31T14:59:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.757+0000 7fe476826740 1 -- 172.21.15.67:0/363918174 shutdown_connections 2024-07-31T14:59:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.757+0000 7fe476826740 1 --2- 172.21.15.67:0/363918174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d24390 0x5567170f3ab0 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe458009c20 tx=0x7fe458011810 comp rx=0 tx=0).stop 2024-07-31T14:59:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.757+0000 7fe476826740 1 --2- 172.21.15.67:0/363918174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d1c1b0 0x5567170f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.757+0000 7fe476826740 1 --2- 172.21.15.67:0/363918174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 0x5567170f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.757+0000 7fe476826740 1 -- 172.21.15.67:0/363918174 >> 172.21.15.67:0/363918174 conn(0x556716d39460 msgr2=0x5567170f0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:29.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.758+0000 7fe476826740 1 -- 172.21.15.67:0/363918174 shutdown_connections 2024-07-31T14:59:29.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.758+0000 7fe476826740 1 -- 172.21.15.67:0/363918174 wait complete. 2024-07-31T14:59:29.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.759+0000 7fe476826740 1 -- start start 2024-07-31T14:59:29.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.759+0000 7fe476826740 1 Processor -- start 2024-07-31T14:59:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.762+0000 7fe476826740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d1c1b0 0x5567170ed320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.762+0000 7fe476826740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 0x5567170ed860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.762+0000 7fe476826740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d039f0 0x5567171133d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.762+0000 7fe476826740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556716eb3700 con 0x556716d039f0 2024-07-31T14:59:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.762+0000 7fe476826740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556716ff29e0 con 0x556716d45070 2024-07-31T14:59:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.762+0000 7fe476826740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5567170f02d0 con 0x556716d1c1b0 2024-07-31T14:59:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.762+0000 7fe476826740 10 client.0 _ll_register_callbacks cb 0x55671700d4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.763+0000 7fe46d870640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 0x5567170ed860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.763+0000 7fe46d870640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 0x5567170ed860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49004/0 (socket says 172.21.15.67:49004) 2024-07-31T14:59:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.763+0000 7fe46d870640 1 -- 172.21.15.67:0/3354036483 learned_addr learned my addr 172.21.15.67:0/3354036483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.765+0000 7fe46d870640 1 -- 172.21.15.67:0/3354036483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d1c1b0 msgr2=0x5567170ed320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:59:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.765+0000 7fe46d870640 1 --2- 172.21.15.67:0/3354036483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d1c1b0 0x5567170ed320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.765+0000 7fe46d870640 1 -- 172.21.15.67:0/3354036483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d039f0 msgr2=0x5567171133d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:59:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.765+0000 7fe46d870640 1 --2- 172.21.15.67:0/3354036483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d039f0 0x5567171133d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.765+0000 7fe46d870640 1 -- 172.21.15.67:0/3354036483 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556716db0650 con 0x556716d45070 2024-07-31T14:59:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.766+0000 7fe46d870640 1 --2- 172.21.15.67:0/3354036483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 0x5567170ed860 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe458004980 tx=0x7fe4580045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f654c2d29451893 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.768+0000 7fe451ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:59:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.768+0000 7fe451ffb640 1 -- 172.21.15.67:0/3354036483 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458029070 con 0x556716d45070 2024-07-31T14:59:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.768+0000 7fe476826740 1 -- 172.21.15.67:0/3354036483 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4580054d0 con 0x556716d45070 2024-07-31T14:59:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.768+0000 7fe451ffb640 1 -- 172.21.15.67:0/3354036483 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7fe458003800 con 0x556716d45070 2024-07-31T14:59:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.769+0000 7fe451ffb640 1 -- 172.21.15.67:0/3354036483 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe458035050 con 0x556716d45070 2024-07-31T14:59:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.769+0000 7fe476826740 10 client.27149 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.769+0000 7fe476826740 1 -- 172.21.15.67:0/3354036483 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x556717115df0 con 0x556716d45070 2024-07-31T14:59:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.769+0000 7fe451ffb640 1 -- 172.21.15.67:0/3354036483 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4580054d0 con 0x556716d45070 2024-07-31T14:59:29.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.772+0000 7fe451ffb640 1 -- 172.21.15.67:0/3354036483 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7fe458004fd0 con 0x556716d45070 2024-07-31T14:59:29.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.773+0000 7fe476826740 10 client.27149 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.773+0000 7fe476826740 10 client.27149 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:59:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.773+0000 7fe476826740 20 client.27149 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.773+0000 7fe476826740 1 --2- 172.21.15.67:0/3354036483 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556716cd8da0 0x556717119650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.773+0000 7fe476826740 4 client.27149 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:59:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.773+0000 7fe476826740 1 -- 172.21.15.67:0/3354036483 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556717062b00 con 0x556716cd8da0 2024-07-31T14:59:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.773+0000 7fe46e071640 1 --2- 172.21.15.67:0/3354036483 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556716cd8da0 0x556717119650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:29.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.778+0000 7fe46e071640 1 --2- 172.21.15.67:0/3354036483 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556716cd8da0 0x556717119650 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6465ec37ad585dc5 server_cookie=335cf8eb6c412e8f in_seq=0 out_seq=0 2024-07-31T14:59:29.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.778+0000 7fe451ffb640 10 client.27149 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:59:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.780+0000 7fe451ffb640 1 -- 172.21.15.67:0/3354036483 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556717062b00 con 0x556716cd8da0 2024-07-31T14:59:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.780+0000 7fe451ffb640 10 client.27149 handle_command_reply: tid=0 2024-07-31T14:59:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.780+0000 7fe476826740 1 client.27149 shutdown 2024-07-31T14:59:29.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.780+0000 7fe476826740 1 -- 172.21.15.67:0/3354036483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 msgr2=0x5567170ed860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:29.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.780+0000 7fe476826740 1 --2- 172.21.15.67:0/3354036483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 0x5567170ed860 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe458004980 tx=0x7fe4580045e0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.781+0000 7fe476826740 1 -- 172.21.15.67:0/3354036483 shutdown_connections 2024-07-31T14:59:29.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.781+0000 7fe476826740 1 --2- 172.21.15.67:0/3354036483 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556716cd8da0 0x556717119650 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.781+0000 7fe476826740 1 --2- 172.21.15.67:0/3354036483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d039f0 0x5567171133d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.781+0000 7fe476826740 1 --2- 172.21.15.67:0/3354036483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 0x5567170ed860 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.781+0000 7fe476826740 1 --2- 172.21.15.67:0/3354036483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d1c1b0 0x5567170ed320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.781+0000 7fe476826740 1 -- 172.21.15.67:0/3354036483 >> 172.21.15.67:0/3354036483 conn(0x556716d39460 msgr2=0x5567170f4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:29.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.781+0000 7fe451ffb640 0 client.27149 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:59:29.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.781+0000 7fe476826740 1 -- 172.21.15.67:0/3354036483 shutdown_connections 2024-07-31T14:59:29.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.781+0000 7fe476826740 1 -- 172.21.15.67:0/3354036483 wait complete. 2024-07-31T14:59:29.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.782+0000 7fe476826740 20 client.27149 trim_cache size 0 max 16384 2024-07-31T14:59:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.809+0000 7fe476826740 1 Processor -- start 2024-07-31T14:59:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.809+0000 7fe476826740 1 -- start start 2024-07-31T14:59:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.810+0000 7fe476826740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d039f0 0x556717113240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.810+0000 7fe476826740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d1c1b0 0x5567170ebba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.810+0000 7fe476826740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d45070 0x5567170ec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.811+0000 7fe46e071640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d039f0 0x556717113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.811+0000 7fe46e071640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d039f0 0x556717113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60132/0 (socket says 172.21.15.67:60132) 2024-07-31T14:59:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.811+0000 7fe46e872640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d45070 0x5567170ec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.811+0000 7fe46e872640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d45070 0x5567170ec0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58896/0 (socket says 172.21.15.67:58896) 2024-07-31T14:59:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.811+0000 7fe46e071640 1 -- 172.21.15.67:0/170623035 learned_addr learned my addr 172.21.15.67:0/170623035 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.811+0000 7fe476826740 1 -- 172.21.15.67:0/170623035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681b70a0 con 0x556716d45070 2024-07-31T14:59:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.811+0000 7fe476826740 1 -- 172.21.15.67:0/170623035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681b8d30 con 0x556716d1c1b0 2024-07-31T14:59:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.811+0000 7fe476826740 1 -- 172.21.15.67:0/170623035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681b7770 con 0x556716d039f0 2024-07-31T14:59:29.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.812+0000 7fe46e872640 1 -- 172.21.15.67:0/170623035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d039f0 msgr2=0x556717113240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:29.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.812+0000 7fe46e872640 1 --2- 172.21.15.67:0/170623035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d039f0 0x556717113240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.812+0000 7fe46e872640 1 -- 172.21.15.67:0/170623035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d1c1b0 msgr2=0x5567170ebba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:59:29.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.812+0000 7fe46e872640 1 --2- 172.21.15.67:0/170623035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d1c1b0 0x5567170ebba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.812+0000 7fe46e872640 1 -- 172.21.15.67:0/170623035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567170af330 con 0x556716d45070 2024-07-31T14:59:29.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.812+0000 7fe46e872640 1 --2- 172.21.15.67:0/170623035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d45070 0x5567170ec0e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe46000ac60 tx=0x7fe460011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d46f240e4b5bba server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:29.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.813+0000 7fe4537fe640 1 -- 172.21.15.67:0/170623035 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460025070 con 0x556716d45070 2024-07-31T14:59:29.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.813+0000 7fe4537fe640 1 -- 172.21.15.67:0/170623035 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe460015db0 con 0x556716d45070 2024-07-31T14:59:29.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.814+0000 7fe4537fe640 1 -- 172.21.15.67:0/170623035 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460020d70 con 0x556716d45070 2024-07-31T14:59:29.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.814+0000 7fe476826740 1 -- 172.21.15.67:0/170623035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d45070 msgr2=0x5567170ec0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:29.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.814+0000 7fe476826740 1 --2- 172.21.15.67:0/170623035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d45070 0x5567170ec0e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe46000ac60 tx=0x7fe460011000 comp rx=0 tx=0).stop 2024-07-31T14:59:29.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.815+0000 7fe476826740 1 -- 172.21.15.67:0/170623035 shutdown_connections 2024-07-31T14:59:29.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.815+0000 7fe476826740 1 --2- 172.21.15.67:0/170623035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d45070 0x5567170ec0e0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.815+0000 7fe476826740 1 --2- 172.21.15.67:0/170623035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d1c1b0 0x5567170ebba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.815+0000 7fe476826740 1 --2- 172.21.15.67:0/170623035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d039f0 0x556717113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.815+0000 7fe476826740 1 -- 172.21.15.67:0/170623035 >> 172.21.15.67:0/170623035 conn(0x556716d39460 msgr2=0x5567170e86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:29.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.821+0000 7fe476826740 1 -- 172.21.15.67:0/170623035 shutdown_connections 2024-07-31T14:59:29.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.822+0000 7fe476826740 1 -- 172.21.15.67:0/170623035 wait complete. 2024-07-31T14:59:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.822+0000 7fe476826740 1 -- start start 2024-07-31T14:59:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.822+0000 7fe476826740 1 Processor -- start 2024-07-31T14:59:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.823+0000 7fe476826740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d039f0 0x55671710a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.823+0000 7fe476826740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d1c1b0 0x55671710a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.823+0000 7fe476826740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 0x55671710adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.823+0000 7fe476826740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681b8660 con 0x556716d039f0 2024-07-31T14:59:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.823+0000 7fe476826740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681b7ed0 con 0x556716d45070 2024-07-31T14:59:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.823+0000 7fe476826740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681b50a0 con 0x556716d1c1b0 2024-07-31T14:59:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.823+0000 7fe476826740 10 client.0 _ll_register_callbacks cb 0x7fe460009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.824+0000 7fe46e071640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d039f0 0x55671710a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.824+0000 7fe46e071640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d039f0 0x55671710a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58900/0 (socket says 172.21.15.67:58900) 2024-07-31T14:59:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.824+0000 7fe46e071640 1 -- 172.21.15.67:0/3916706922 learned_addr learned my addr 172.21.15.67:0/3916706922 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:29.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.824+0000 7fe46e071640 1 -- 172.21.15.67:0/3916706922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d1c1b0 msgr2=0x55671710a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:29.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.824+0000 7fe46e071640 1 --2- 172.21.15.67:0/3916706922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d1c1b0 0x55671710a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.824+0000 7fe46e071640 1 -- 172.21.15.67:0/3916706922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 msgr2=0x55671710adf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:59:29.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.824+0000 7fe46e071640 1 --2- 172.21.15.67:0/3916706922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 0x55671710adf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.824+0000 7fe46e071640 1 -- 172.21.15.67:0/3916706922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55671717feb0 con 0x556716d039f0 2024-07-31T14:59:29.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.825+0000 7fe46e071640 1 --2- 172.21.15.67:0/3916706922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d039f0 0x55671710a370 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe4640cb530 tx=0x7fe46403b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384cb274857df039 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:29.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.825+0000 7fe4517fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:59:29.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.825+0000 7fe4517fa640 1 -- 172.21.15.67:0/3916706922 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464049040 con 0x556716d039f0 2024-07-31T14:59:29.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.825+0000 7fe476826740 1 -- 172.21.15.67:0/3916706922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe460015db0 con 0x556716d039f0 2024-07-31T14:59:29.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.826+0000 7fe4517fa640 1 -- 172.21.15.67:0/3916706922 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7fe4640c7830 con 0x556716d039f0 2024-07-31T14:59:29.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.827+0000 7fe4517fa640 1 -- 172.21.15.67:0/3916706922 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe4640cb5a0 con 0x556716d039f0 2024-07-31T14:59:29.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.827+0000 7fe476826740 10 client.17589 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.827+0000 7fe476826740 1 -- 172.21.15.67:0/3916706922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5567171811b0 con 0x556716d039f0 2024-07-31T14:59:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.828+0000 7fe4517fa640 1 -- 172.21.15.67:0/3916706922 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe460015db0 con 0x556716d039f0 2024-07-31T14:59:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.828+0000 7fe4517fa640 1 -- 172.21.15.67:0/3916706922 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7fe4640cb800 con 0x556716d039f0 2024-07-31T14:59:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.828+0000 7fe476826740 10 client.17589 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.828+0000 7fe476826740 10 client.17589 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T14:59:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.828+0000 7fe476826740 20 client.17589 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.828+0000 7fe476826740 1 --2- 172.21.15.67:0/3916706922 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556717181ab0 0x556717183f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.828+0000 7fe476826740 4 client.17589 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:59:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.828+0000 7fe476826740 1 -- 172.21.15.67:0/3916706922 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567170ecb80 con 0x556717181ab0 2024-07-31T14:59:29.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.830+0000 7fe46d870640 1 --2- 172.21.15.67:0/3916706922 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556717181ab0 0x556717183f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:29.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.835+0000 7fe4517fa640 10 client.17589 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T14:59:29.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.835+0000 7fe46d870640 1 --2- 172.21.15.67:0/3916706922 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556717181ab0 0x556717183f60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2a09caab29b3cca server_cookie=48ecd56fc5af1221 in_seq=0 out_seq=0 2024-07-31T14:59:29.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.835+0000 7fe4517fa640 1 -- 172.21.15.67:0/3916706922 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567170ecb80 con 0x556717181ab0 2024-07-31T14:59:29.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.835+0000 7fe4517fa640 10 client.17589 handle_command_reply: tid=0 2024-07-31T14:59:29.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.835+0000 7fe476826740 1 client.17589 shutdown 2024-07-31T14:59:29.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.837+0000 7fe476826740 1 -- 172.21.15.67:0/3916706922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d039f0 msgr2=0x55671710a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:29.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.837+0000 7fe476826740 1 --2- 172.21.15.67:0/3916706922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d039f0 0x55671710a370 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe4640cb530 tx=0x7fe46403b720 comp rx=0 tx=0).stop 2024-07-31T14:59:29.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.841+0000 7fe476826740 1 -- 172.21.15.67:0/3916706922 shutdown_connections 2024-07-31T14:59:29.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.841+0000 7fe476826740 1 --2- 172.21.15.67:0/3916706922 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x556717181ab0 0x556717183f60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.841+0000 7fe476826740 1 --2- 172.21.15.67:0/3916706922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556716d45070 0x55671710adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.841+0000 7fe476826740 1 --2- 172.21.15.67:0/3916706922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556716d1c1b0 0x55671710a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.841+0000 7fe476826740 1 --2- 172.21.15.67:0/3916706922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556716d039f0 0x55671710a370 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.841+0000 7fe476826740 1 -- 172.21.15.67:0/3916706922 >> 172.21.15.67:0/3916706922 conn(0x556716d39460 msgr2=0x5567170e6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:29.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.841+0000 7fe4517fa640 0 client.17589 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T14:59:29.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.841+0000 7fe476826740 1 -- 172.21.15.67:0/3916706922 shutdown_connections 2024-07-31T14:59:29.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.841+0000 7fe476826740 1 -- 172.21.15.67:0/3916706922 wait complete. 2024-07-31T14:59:29.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.841+0000 7fe476826740 20 client.17589 trim_cache size 0 max 16384 2024-07-31T14:59:29.845 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:59:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.844+0000 7fe45d59a640 1 -- 172.21.15.67:0/3127734138 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe4540b3f40 msgr2=0x7fe4540b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.844+0000 7fe45d59a640 1 --2- 172.21.15.67:0/3127734138 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe4540b3f40 0x7fe4540b6450 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe460009f20 tx=0x7fe46000d040 comp rx=0 tx=0).stop 2024-07-31T14:59:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.844+0000 7fe45d59a640 1 -- 172.21.15.67:0/3127734138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe46810a080 msgr2=0x7fe4681a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.844+0000 7fe45d59a640 1 --2- 172.21.15.67:0/3127734138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe46810a080 0x7fe4681a05a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe46402a9b0 tx=0x7fe464003b90 comp rx=0 tx=0).stop 2024-07-31T14:59:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.844+0000 7fe45d59a640 1 -- 172.21.15.67:0/3127734138 shutdown_connections 2024-07-31T14:59:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.844+0000 7fe45d59a640 1 --2- 172.21.15.67:0/3127734138 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe4540b3f40 0x7fe4540b6450 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.844+0000 7fe45d59a640 1 --2- 172.21.15.67:0/3127734138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe46810d050 0x7fe4681a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.844+0000 7fe45d59a640 1 --2- 172.21.15.67:0/3127734138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe46810c700 0x7fe4681a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.844+0000 7fe45d59a640 1 --2- 172.21.15.67:0/3127734138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe46810a080 0x7fe4681a05a0 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe46402a9b0 tx=0x7fe464003b90 comp rx=0 tx=0).stop 2024-07-31T14:59:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.844+0000 7fe45d59a640 1 -- 172.21.15.67:0/3127734138 >> 172.21.15.67:0/3127734138 conn(0x7fe468070d20 msgr2=0x7fe468074280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:29.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.844+0000 7fe45d59a640 1 -- 172.21.15.67:0/3127734138 shutdown_connections 2024-07-31T14:59:29.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:29.844+0000 7fe45d59a640 1 -- 172.21.15.67:0/3127734138 wait complete. 2024-07-31T14:59:30.204 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:59:30.838 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T14:59:31.337 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:31 smithi067 ceph-mon[32750]: pgmap v657: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 778 KiB/s wr, 115 op/s 2024-07-31T14:59:31.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:31 smithi073 ceph-mon[36714]: pgmap v657: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 778 KiB/s wr, 115 op/s 2024-07-31T14:59:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:31 smithi077 ceph-mon[37327]: pgmap v657: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 778 KiB/s wr, 115 op/s 2024-07-31T14:59:32.322 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:59:33.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.017+0000 7f3da522c640 1 -- 172.21.15.67:0/78752135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3da0101c60 msgr2=0x7f3da0102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:33.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.017+0000 7f3da522c640 1 --2- 172.21.15.67:0/78752135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3da0101c60 0x7f3da0102070 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009900 tx=0x7f3d8802a420 comp rx=0 tx=0).stop 2024-07-31T14:59:33.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.018+0000 7f3da522c640 1 -- 172.21.15.67:0/78752135 shutdown_connections 2024-07-31T14:59:33.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.018+0000 7f3da522c640 1 --2- 172.21.15.67:0/78752135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3da0107ef0 0x7f3da010e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.018+0000 7f3da522c640 1 --2- 172.21.15.67:0/78752135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3da01075a0 0x7f3da01079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.018+0000 7f3da522c640 1 --2- 172.21.15.67:0/78752135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3da0101c60 0x7f3da0102070 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.018+0000 7f3da522c640 1 -- 172.21.15.67:0/78752135 >> 172.21.15.67:0/78752135 conn(0x7f3da00776f0 msgr2=0x7f3da0077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:33.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.021+0000 7f3da522c640 1 -- 172.21.15.67:0/78752135 shutdown_connections 2024-07-31T14:59:33.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.023+0000 7f3da522c640 1 -- 172.21.15.67:0/78752135 wait complete. 2024-07-31T14:59:33.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.024+0000 7f3da522c640 1 Processor -- start 2024-07-31T14:59:33.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.025+0000 7f3da522c640 1 -- start start 2024-07-31T14:59:33.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.025+0000 7f3da522c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3da0101c60 0x7f3da0113160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.025+0000 7f3da522c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3da01075a0 0x7f3da01136e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.025+0000 7f3da522c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3da0107ef0 0x7f3da01189a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.025+0000 7f3da522c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01113e0 con 0x7f3da0101c60 2024-07-31T14:59:33.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.025+0000 7f3da522c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0111260 con 0x7f3da0107ef0 2024-07-31T14:59:33.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.025+0000 7f3da522c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0111560 con 0x7f3da01075a0 2024-07-31T14:59:33.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.027+0000 7f3d9ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3da0101c60 0x7f3da0113160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.027+0000 7f3d97fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3da01075a0 0x7f3da01136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.028+0000 7f3d97fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3da01075a0 0x7f3da01136e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38724/0 (socket says 172.21.15.67:38724) 2024-07-31T14:59:33.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.028+0000 7f3d97fff640 1 -- 172.21.15.67:0/1683989104 learned_addr learned my addr 172.21.15.67:0/1683989104 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:33.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.028+0000 7f3d97fff640 1 -- 172.21.15.67:0/1683989104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3da0107ef0 msgr2=0x7f3da01189a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T14:59:33.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.028+0000 7f3d97fff640 1 --2- 172.21.15.67:0/1683989104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3da0107ef0 0x7f3da01189a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.028+0000 7f3d97fff640 1 -- 172.21.15.67:0/1683989104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3da0101c60 msgr2=0x7f3da0113160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:33.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.028+0000 7f3d97fff640 1 --2- 172.21.15.67:0/1683989104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3da0101c60 0x7f3da0113160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.028+0000 7f3d97fff640 1 -- 172.21.15.67:0/1683989104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da0119070 con 0x7f3da01075a0 2024-07-31T14:59:33.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.029+0000 7f3d97fff640 1 --2- 172.21.15.67:0/1683989104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3da01075a0 0x7f3da01136e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3d90006fd0 tx=0x7f3d9000b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:33.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.029+0000 7f3d9cd99640 1 -- 172.21.15.67:0/1683989104 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90011830 con 0x7f3da01075a0 2024-07-31T14:59:33.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.029+0000 7f3da522c640 1 -- 172.21.15.67:0/1683989104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da0119360 con 0x7f3da01075a0 2024-07-31T14:59:33.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.029+0000 7f3da522c640 1 -- 172.21.15.67:0/1683989104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da011db30 con 0x7f3da01075a0 2024-07-31T14:59:33.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.030+0000 7f3d9cd99640 1 -- 172.21.15.67:0/1683989104 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d90016070 con 0x7f3da01075a0 2024-07-31T14:59:33.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.030+0000 7f3d9cd99640 1 -- 172.21.15.67:0/1683989104 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90010e20 con 0x7f3da01075a0 2024-07-31T14:59:33.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.034+0000 7f3dab77d740 1 Processor -- start 2024-07-31T14:59:33.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.036+0000 7f3d9cd99640 1 -- 172.21.15.67:0/1683989104 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3d9001b4a0 con 0x7f3da01075a0 2024-07-31T14:59:33.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.037+0000 7f3d9cd99640 1 --2- 172.21.15.67:0/1683989104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3d8c0b3f10 0x7f3d8c0b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.038+0000 7f3d9cd99640 1 -- 172.21.15.67:0/1683989104 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f3d900e37b0 con 0x7f3da01075a0 2024-07-31T14:59:33.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.038+0000 7f3dab77d740 1 -- start start 2024-07-31T14:59:33.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.039+0000 7f3dab77d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6875c520 0x55bc68b02070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.039+0000 7f3dab77d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b025b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.039+0000 7f3dab77d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc6873b840 0x55bc68b02af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.039+0000 7f3dab77d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bc68a77e70 con 0x55bc68733660 2024-07-31T14:59:33.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.039+0000 7f3dab77d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bc68a67370 con 0x55bc6873b840 2024-07-31T14:59:33.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.039+0000 7f3dab77d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bc68a6b3b0 con 0x55bc6875c520 2024-07-31T14:59:33.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.040+0000 7f3d97fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.041+0000 7f3d9ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6875c520 0x55bc68b02070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.041+0000 7f3d9ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6875c520 0x55bc68b02070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38734/0 (socket says 172.21.15.67:38734) 2024-07-31T14:59:33.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.041+0000 7f3d9ed9d640 1 -- 172.21.15.67:0/2650719884 learned_addr learned my addr 172.21.15.67:0/2650719884 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:33.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.042+0000 7f3d97fff640 1 -- 172.21.15.67:0/2650719884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6875c520 msgr2=0x55bc68b02070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:33.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.042+0000 7f3d97fff640 1 --2- 172.21.15.67:0/2650719884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6875c520 0x55bc68b02070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.042+0000 7f3d97fff640 1 -- 172.21.15.67:0/2650719884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc6873b840 msgr2=0x55bc68b02af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:59:33.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.042+0000 7f3d97fff640 1 --2- 172.21.15.67:0/2650719884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc6873b840 0x55bc68b02af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.042+0000 7f3d97fff640 1 -- 172.21.15.67:0/2650719884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc68a34040 con 0x55bc68733660 2024-07-31T14:59:33.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.042+0000 7f3d97fff640 1 --2- 172.21.15.67:0/2650719884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b025b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3d900e0bf0 tx=0x7f3d9000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48845dd4b26fed7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:33.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.043+0000 7f3d957fa640 1 -- 172.21.15.67:0/2650719884 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d900ae6b0 con 0x55bc68733660 2024-07-31T14:59:33.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.043+0000 7f3d957fa640 1 -- 172.21.15.67:0/2650719884 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d900b3160 con 0x55bc68733660 2024-07-31T14:59:33.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.044+0000 7f3dab77d740 1 -- 172.21.15.67:0/2650719884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 msgr2=0x55bc68b025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:33.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.044+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2650719884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b025b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3d900e0bf0 tx=0x7f3d9000bf90 comp rx=0 tx=0).stop 2024-07-31T14:59:33.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.044+0000 7f3d957fa640 1 -- 172.21.15.67:0/2650719884 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d900a5c80 con 0x55bc68733660 2024-07-31T14:59:33.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.045+0000 7f3d9f59e640 1 --2- 172.21.15.67:0/1683989104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3d8c0b3f10 0x7f3d8c0b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.047+0000 7f3dab77d740 1 -- 172.21.15.67:0/2650719884 shutdown_connections 2024-07-31T14:59:33.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.047+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2650719884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc6873b840 0x55bc68b02af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.048+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2650719884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b025b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.048+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2650719884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6875c520 0x55bc68b02070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.048+0000 7f3dab77d740 1 -- 172.21.15.67:0/2650719884 >> 172.21.15.67:0/2650719884 conn(0x55bc68750910 msgr2=0x55bc68afff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:33.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.051+0000 7f3d9f59e640 1 --2- 172.21.15.67:0/1683989104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3d8c0b3f10 0x7f3d8c0b6420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800aaf0 tx=0x7f3d98009290 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:33.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.055+0000 7f3dab77d740 1 -- 172.21.15.67:0/2650719884 shutdown_connections 2024-07-31T14:59:33.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.057+0000 7f3dab77d740 1 -- 172.21.15.67:0/2650719884 wait complete. 2024-07-31T14:59:33.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.058+0000 7f3dab77d740 1 -- start start 2024-07-31T14:59:33.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.058+0000 7f3dab77d740 1 Processor -- start 2024-07-31T14:59:33.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.060+0000 7f3dab77d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b0dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.060+0000 7f3dab77d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc6873b840 0x55bc68b0e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.060+0000 7f3dab77d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6875c520 0x55bc68b19be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.060+0000 7f3dab77d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bc688cabb0 con 0x55bc68733660 2024-07-31T14:59:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.060+0000 7f3dab77d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bc68a09ac0 con 0x55bc6873b840 2024-07-31T14:59:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.060+0000 7f3dab77d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bc68b10840 con 0x55bc6875c520 2024-07-31T14:59:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.060+0000 7f3dab77d740 10 client.0 _ll_register_callbacks cb 0x55bc68a123b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.061+0000 7f3d9f59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6875c520 0x55bc68b19be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.061+0000 7f3d9f59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6875c520 0x55bc68b19be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38744/0 (socket says 172.21.15.67:38744) 2024-07-31T14:59:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.061+0000 7f3d9f59e640 1 -- 172.21.15.67:0/2777650568 learned_addr learned my addr 172.21.15.67:0/2777650568 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.061+0000 7f3d97fff640 1 --2- 172.21.15.67:0/2777650568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc6873b840 0x55bc68b0e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.066+0000 7f3d9ed9d640 1 --2- 172.21.15.67:0/2777650568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b0dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.066+0000 7f3d9f59e640 1 -- 172.21.15.67:0/2777650568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc6873b840 msgr2=0x55bc68b0e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:33.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.066+0000 7f3d9f59e640 1 --2- 172.21.15.67:0/2777650568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc6873b840 0x55bc68b0e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.066+0000 7f3d9f59e640 1 -- 172.21.15.67:0/2777650568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 msgr2=0x55bc68b0dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:33.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.066+0000 7f3d9f59e640 1 --2- 172.21.15.67:0/2777650568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b0dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.066+0000 7f3d9f59e640 1 -- 172.21.15.67:0/2777650568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc687c7b00 con 0x55bc6875c520 2024-07-31T14:59:33.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.066+0000 7f3d9ed9d640 1 --2- 172.21.15.67:0/2777650568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b0dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T14:59:33.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.067+0000 7f3d9f59e640 1 --2- 172.21.15.67:0/2777650568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6875c520 0x55bc68b19be0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009d30 tx=0x7f3d98004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff436af0c93a14a3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:33.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.067+0000 7f3d867fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:59:33.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.069+0000 7f3d867fc640 1 -- 172.21.15.67:0/2777650568 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9800ce30 con 0x55bc6875c520 2024-07-31T14:59:33.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.069+0000 7f3dab77d740 1 -- 172.21.15.67:0/2777650568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d900b3160 con 0x55bc6875c520 2024-07-31T14:59:33.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.069+0000 7f3d867fc640 1 -- 172.21.15.67:0/2777650568 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f3d98019420 con 0x55bc6875c520 2024-07-31T14:59:33.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.069+0000 7f3d867fc640 1 -- 172.21.15.67:0/2777650568 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3d98019620 con 0x55bc6875c520 2024-07-31T14:59:33.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.069+0000 7f3dab77d740 10 client.27160 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:33.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.069+0000 7f3dab77d740 1 -- 172.21.15.67:0/2777650568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55bc6883e960 con 0x55bc6875c520 2024-07-31T14:59:33.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.070+0000 7f3d867fc640 1 -- 172.21.15.67:0/2777650568 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d900b3160 con 0x55bc6875c520 2024-07-31T14:59:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.071+0000 7f3d867fc640 1 -- 172.21.15.67:0/2777650568 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f3d980229a0 con 0x55bc6875c520 2024-07-31T14:59:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.071+0000 7f3dab77d740 10 client.27160 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.071+0000 7f3dab77d740 10 client.27160 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:59:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.072+0000 7f3dab77d740 20 client.27160 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.072+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2777650568 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc6871aea0 0x55bc68b1fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.072+0000 7f3dab77d740 4 client.27160 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:59:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.072+0000 7f3dab77d740 1 -- 172.21.15.67:0/2777650568 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc68a79dc0 con 0x55bc6871aea0 2024-07-31T14:59:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.073+0000 7f3d9ed9d640 1 --2- 172.21.15.67:0/2777650568 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc6871aea0 0x55bc68b1fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.081+0000 7f3d9ed9d640 1 --2- 172.21.15.67:0/2777650568 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc6871aea0 0x55bc68b1fd10 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af47e1e7f363c60c server_cookie=513229f823fb6392 in_seq=0 out_seq=0 2024-07-31T14:59:33.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.082+0000 7f3d867fc640 10 client.27160 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:33.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.083+0000 7f3d867fc640 1 -- 172.21.15.67:0/2777650568 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc68a79dc0 con 0x55bc6871aea0 2024-07-31T14:59:33.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.083+0000 7f3d867fc640 10 client.27160 handle_command_reply: tid=0 2024-07-31T14:59:33.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.084+0000 7f3dab77d740 1 client.27160 shutdown 2024-07-31T14:59:33.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.085+0000 7f3dab77d740 1 -- 172.21.15.67:0/2777650568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6875c520 msgr2=0x55bc68b19be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:33.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.085+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2777650568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6875c520 0x55bc68b19be0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009d30 tx=0x7f3d98004ba0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.086+0000 7f3dab77d740 1 -- 172.21.15.67:0/2777650568 shutdown_connections 2024-07-31T14:59:33.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.086+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2777650568 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc6871aea0 0x55bc68b1fd10 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.086+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2777650568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6875c520 0x55bc68b19be0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.086+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2777650568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc6873b840 0x55bc68b0e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.086+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2777650568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b0dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.086+0000 7f3dab77d740 1 -- 172.21.15.67:0/2777650568 >> 172.21.15.67:0/2777650568 conn(0x55bc68750910 msgr2=0x55bc68b05200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:33.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.086+0000 7f3d867fc640 0 client.27160 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.087+0000 7f3dab77d740 1 -- 172.21.15.67:0/2777650568 shutdown_connections 2024-07-31T14:59:33.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.092+0000 7f3dab77d740 1 -- 172.21.15.67:0/2777650568 wait complete. 2024-07-31T14:59:33.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.096+0000 7f3dab77d740 20 client.27160 trim_cache size 0 max 16384 2024-07-31T14:59:33.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.130+0000 7f3dab77d740 1 Processor -- start 2024-07-31T14:59:33.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.130+0000 7f3dab77d740 1 -- start start 2024-07-31T14:59:33.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.131+0000 7f3dab77d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.131+0000 7f3dab77d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6873b840 0x55bc68b199d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.131+0000 7f3dab77d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc6875c520 0x55bc68b19f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.131+0000 7f3dab77d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8000ff50 con 0x55bc68733660 2024-07-31T14:59:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.131+0000 7f3dab77d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80011be0 con 0x55bc6875c520 2024-07-31T14:59:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.131+0000 7f3dab77d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80010620 con 0x55bc6873b840 2024-07-31T14:59:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.131+0000 7f3d9ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.131+0000 7f3d9ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b037b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40732/0 (socket says 172.21.15.67:40732) 2024-07-31T14:59:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.131+0000 7f3d9ed9d640 1 -- 172.21.15.67:0/2243641999 learned_addr learned my addr 172.21.15.67:0/2243641999 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.131+0000 7f3d97fff640 1 --2- 172.21.15.67:0/2243641999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6873b840 0x55bc68b199d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.132+0000 7f3d9f59e640 1 --2- 172.21.15.67:0/2243641999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc6875c520 0x55bc68b19f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.132+0000 7f3d97fff640 1 -- 172.21.15.67:0/2243641999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc6875c520 msgr2=0x55bc68b19f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.132+0000 7f3d97fff640 1 --2- 172.21.15.67:0/2243641999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc6875c520 0x55bc68b19f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.132+0000 7f3d97fff640 1 -- 172.21.15.67:0/2243641999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 msgr2=0x55bc68b037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.132+0000 7f3d97fff640 1 --2- 172.21.15.67:0/2243641999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.132+0000 7f3d97fff640 1 -- 172.21.15.67:0/2243641999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc68b0ea90 con 0x55bc6873b840 2024-07-31T14:59:33.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.137+0000 7f3d97fff640 1 --2- 172.21.15.67:0/2243641999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6873b840 0x55bc68b199d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55bc68b00cf0 tx=0x7f3d900aa230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ed5bf486aad1e64 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.141+0000 7f3d94ff9640 1 -- 172.21.15.67:0/2243641999 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d900c9070 con 0x55bc6873b840 2024-07-31T14:59:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.141+0000 7f3d94ff9640 1 -- 172.21.15.67:0/2243641999 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d900ae6b0 con 0x55bc6873b840 2024-07-31T14:59:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.141+0000 7f3d94ff9640 1 -- 172.21.15.67:0/2243641999 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d900a5c80 con 0x55bc6873b840 2024-07-31T14:59:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.142+0000 7f3dab77d740 1 -- 172.21.15.67:0/2243641999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6873b840 msgr2=0x55bc68b199d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.142+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2243641999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6873b840 0x55bc68b199d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55bc68b00cf0 tx=0x7f3d900aa230 comp rx=0 tx=0).stop 2024-07-31T14:59:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.144+0000 7f3dab77d740 1 -- 172.21.15.67:0/2243641999 shutdown_connections 2024-07-31T14:59:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.144+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2243641999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc6875c520 0x55bc68b19f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.144+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2243641999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6873b840 0x55bc68b199d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.144+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2243641999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc68733660 0x55bc68b037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.144+0000 7f3dab77d740 1 -- 172.21.15.67:0/2243641999 >> 172.21.15.67:0/2243641999 conn(0x55bc68750910 msgr2=0x55bc68b030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:33.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.150+0000 7f3dab77d740 1 -- 172.21.15.67:0/2243641999 shutdown_connections 2024-07-31T14:59:33.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.150+0000 7f3dab77d740 1 -- 172.21.15.67:0/2243641999 wait complete. 2024-07-31T14:59:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.151+0000 7f3dab77d740 1 -- start start 2024-07-31T14:59:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.151+0000 7f3dab77d740 1 Processor -- start 2024-07-31T14:59:33.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.152+0000 7f3dab77d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc68733660 0x55bc68b163c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.152+0000 7f3dab77d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6873b840 0x55bc68b11870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.152+0000 7f3dab77d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc6875c520 0x55bc68b11f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.152+0000 7f3d9ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc68733660 0x55bc68b163c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.152+0000 7f3d9ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc68733660 0x55bc68b163c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:33026/0 (socket says 172.21.15.67:33026) 2024-07-31T14:59:33.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.152+0000 7f3d9ed9d640 1 -- 172.21.15.67:0/2623096664 learned_addr learned my addr 172.21.15.67:0/2623096664 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:33.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.152+0000 7f3dab77d740 1 -- 172.21.15.67:0/2623096664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80011510 con 0x55bc6875c520 2024-07-31T14:59:33.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.152+0000 7f3dab77d740 1 -- 172.21.15.67:0/2623096664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80010d80 con 0x55bc68733660 2024-07-31T14:59:33.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.153+0000 7f3dab77d740 1 -- 172.21.15.67:0/2623096664 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8000dfe0 con 0x55bc6873b840 2024-07-31T14:59:33.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.153+0000 7f3d9f59e640 1 --2- 172.21.15.67:0/2623096664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc6875c520 0x55bc68b11f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.153+0000 7f3dab77d740 10 client.0 _ll_register_callbacks cb 0x7f3d9800cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:33.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.158+0000 7f3d9ed9d640 1 -- 172.21.15.67:0/2623096664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6873b840 msgr2=0x55bc68b11870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:59:33.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.158+0000 7f3d9ed9d640 1 --2- 172.21.15.67:0/2623096664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6873b840 0x55bc68b11870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.158+0000 7f3d9ed9d640 1 -- 172.21.15.67:0/2623096664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc6875c520 msgr2=0x55bc68b11f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:33.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.158+0000 7f3d9ed9d640 1 --2- 172.21.15.67:0/2623096664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc6875c520 0x55bc68b11f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:33.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.158+0000 7f3d9ed9d640 1 -- 172.21.15.67:0/2623096664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc68b874a0 con 0x55bc68733660 2024-07-31T14:59:33.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.159+0000 7f3d9ed9d640 1 --2- 172.21.15.67:0/2623096664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc68733660 0x55bc68b163c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3d88033000 tx=0x7f3d88004330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5951fc1ec5ecf3d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:33.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.160+0000 7f3d867fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:59:33.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.160+0000 7f3d867fc640 1 -- 172.21.15.67:0/2623096664 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88035040 con 0x55bc68733660 2024-07-31T14:59:33.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.160+0000 7f3d867fc640 1 -- 172.21.15.67:0/2623096664 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f3d880316a0 con 0x55bc68733660 2024-07-31T14:59:33.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.160+0000 7f3dab77d740 1 -- 172.21.15.67:0/2623096664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d900ae6b0 con 0x55bc68733660 2024-07-31T14:59:33.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.160+0000 7f3d867fc640 1 -- 172.21.15.67:0/2623096664 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3d880318a0 con 0x55bc68733660 2024-07-31T14:59:33.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.168+0000 7f3dab77d740 10 client.27155 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:33.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.168+0000 7f3dab77d740 1 -- 172.21.15.67:0/2623096664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55bc68b88610 con 0x55bc68733660 2024-07-31T14:59:33.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.168+0000 7f3d867fc640 1 -- 172.21.15.67:0/2623096664 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d900ae6b0 con 0x55bc68733660 2024-07-31T14:59:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.170+0000 7f3d867fc640 1 -- 172.21.15.67:0/2623096664 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f3d880035f0 con 0x55bc68733660 2024-07-31T14:59:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.170+0000 7f3dab77d740 10 client.27155 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:33.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.170+0000 7f3dab77d740 10 client.27155 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T14:59:33.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.170+0000 7f3dab77d740 20 client.27155 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:33.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.170+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2623096664 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc686f0250 0x55bc68b8aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:33.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.170+0000 7f3dab77d740 4 client.27155 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "578e3b69", "release": true, "await": true}] 2024-07-31T14:59:33.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.170+0000 7f3dab77d740 1 -- 172.21.15.67:0/2623096664 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "578e3b69", "release": true, "await": true}) -- 0x55bc68a75860 con 0x55bc686f0250 2024-07-31T14:59:33.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.173+0000 7f3d97fff640 1 --2- 172.21.15.67:0/2623096664 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc686f0250 0x55bc68b8aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:33.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.182+0000 7f3d97fff640 1 --2- 172.21.15.67:0/2623096664 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc686f0250 0x55bc68b8aec0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c211e5484183765e server_cookie=46c9afc0ab3d9e5a in_seq=0 out_seq=0 2024-07-31T14:59:33.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:33.182+0000 7f3d867fc640 10 client.27155 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:33.270 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:33 smithi067 ceph-mon[32750]: pgmap v658: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 455 KiB/s wr, 88 op/s 2024-07-31T14:59:33.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:33 smithi073 ceph-mon[36714]: pgmap v658: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 455 KiB/s wr, 88 op/s 2024-07-31T14:59:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:33 smithi077 ceph-mon[37327]: pgmap v658: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 455 KiB/s wr, 88 op/s 2024-07-31T14:59:34.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.812+0000 7f561e28d640 1 -- 172.21.15.67:0/2810698015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5618102970 msgr2=0x7f5618106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:34.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.812+0000 7f561e28d640 1 --2- 172.21.15.67:0/2810698015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5618102970 0x7f5618106ea0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f561401a940 tx=0x7f561403b470 comp rx=0 tx=0).stop 2024-07-31T14:59:34.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.834+0000 7f561e28d640 1 -- 172.21.15.67:0/2810698015 shutdown_connections 2024-07-31T14:59:34.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.834+0000 7f561e28d640 1 --2- 172.21.15.67:0/2810698015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5618107460 0x7f5618107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:34.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.834+0000 7f561e28d640 1 --2- 172.21.15.67:0/2810698015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5618102970 0x7f5618106ea0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:34.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.834+0000 7f561e28d640 1 --2- 172.21.15.67:0/2810698015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5618102020 0x7f5618102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:34.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.834+0000 7f561e28d640 1 -- 172.21.15.67:0/2810698015 >> 172.21.15.67:0/2810698015 conn(0x7f56180fdd30 msgr2=0x7f5618100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:34.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.835+0000 7f561e28d640 1 -- 172.21.15.67:0/2810698015 shutdown_connections 2024-07-31T14:59:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.837+0000 7f561e28d640 1 -- 172.21.15.67:0/2810698015 wait complete. 2024-07-31T14:59:34.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.838+0000 7f561e28d640 1 Processor -- start 2024-07-31T14:59:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.839+0000 7f561e28d640 1 -- start start 2024-07-31T14:59:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.839+0000 7f561e28d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5618102020 0x7f5618116700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.839+0000 7f561e28d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5618102970 0x7f56181129e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.839+0000 7f561d28b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5618102020 0x7f5618116700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.840+0000 7f561d28b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5618102020 0x7f5618116700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40756/0 (socket says 172.21.15.67:40756) 2024-07-31T14:59:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.840+0000 7f561d28b640 1 -- 172.21.15.67:0/273439479 learned_addr learned my addr 172.21.15.67:0/273439479 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:34.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.840+0000 7f561e28d640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5618107460 0x7f5618113130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:34.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.840+0000 7f561e28d640 1 -- 172.21.15.67:0/273439479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f561810ad60 con 0x7f5618102020 2024-07-31T14:59:34.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.840+0000 7f561e28d640 1 -- 172.21.15.67:0/273439479 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f561810abe0 con 0x7f5618107460 2024-07-31T14:59:34.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.840+0000 7f561e28d640 1 -- 172.21.15.67:0/273439479 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f561810aee0 con 0x7f5618102970 2024-07-31T14:59:34.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.841+0000 7f561ca8a640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5618102970 0x7f56181129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:34.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.843+0000 7f561da8c640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5618107460 0x7f5618113130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:34.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.843+0000 7f561d28b640 1 -- 172.21.15.67:0/273439479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5618102970 msgr2=0x7f56181129e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:34.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.843+0000 7f561d28b640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5618102970 0x7f56181129e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:34.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.843+0000 7f561d28b640 1 -- 172.21.15.67:0/273439479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5618107460 msgr2=0x7f5618113130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:34.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.843+0000 7f561d28b640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5618107460 0x7f5618113130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:34.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.843+0000 7f561d28b640 1 -- 172.21.15.67:0/273439479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56181c5e30 con 0x7f5618102020 2024-07-31T14:59:34.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.844+0000 7f561ca8a640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5618102970 0x7f56181129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T14:59:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.850+0000 7f561d28b640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5618102020 0x7f5618116700 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f561000ebc0 tx=0x7f561000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.854+0000 7f560e7fc640 1 -- 172.21.15.67:0/273439479 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610010630 con 0x7f5618102020 2024-07-31T14:59:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.854+0000 7f560e7fc640 1 -- 172.21.15.67:0/273439479 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5610010c80 con 0x7f5618102020 2024-07-31T14:59:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.855+0000 7f560e7fc640 1 -- 172.21.15.67:0/273439479 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610018b60 con 0x7f5618102020 2024-07-31T14:59:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.858+0000 7f561e28d640 1 -- 172.21.15.67:0/273439479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56181c60c0 con 0x7f5618102020 2024-07-31T14:59:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.858+0000 7f561e28d640 1 -- 172.21.15.67:0/273439479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56181c6580 con 0x7f5618102020 2024-07-31T14:59:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.862+0000 7f5625a40740 1 Processor -- start 2024-07-31T14:59:34.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.863+0000 7f5625a40740 1 -- start start 2024-07-31T14:59:34.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.863+0000 7f5625a40740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2beaf070 0x557a2c2573c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:34.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.863+0000 7f5625a40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be861b0 0x557a2c2549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:34.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.863+0000 7f5625a40740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2be8e390 0x557a2c254f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:34.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.863+0000 7f5625a40740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557a2c1cabb0 con 0x557a2be8e390 2024-07-31T14:59:34.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.863+0000 7f5625a40740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a2c1ba0b0 con 0x557a2beaf070 2024-07-31T14:59:34.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.863+0000 7f5625a40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a2c1be0f0 con 0x557a2be861b0 2024-07-31T14:59:34.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.865+0000 7f561da8c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2beaf070 0x557a2c2573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:34.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.865+0000 7f561da8c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2beaf070 0x557a2c2573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:33042/0 (socket says 172.21.15.67:33042) 2024-07-31T14:59:34.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.865+0000 7f561da8c640 1 -- 172.21.15.67:0/1278789490 learned_addr learned my addr 172.21.15.67:0/1278789490 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:34.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.865+0000 7f561ca8a640 1 --2- 172.21.15.67:0/1278789490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2be8e390 0x557a2c254f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:34.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.866+0000 7f561da8c640 1 -- 172.21.15.67:0/1278789490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be861b0 msgr2=0x557a2c2549d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:59:34.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.866+0000 7f561da8c640 1 --2- 172.21.15.67:0/1278789490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be861b0 0x557a2c2549d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:34.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.866+0000 7f561da8c640 1 -- 172.21.15.67:0/1278789490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2be8e390 msgr2=0x557a2c254f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:34.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.866+0000 7f561da8c640 1 --2- 172.21.15.67:0/1278789490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2be8e390 0x557a2c254f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:34.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.866+0000 7f561da8c640 1 -- 172.21.15.67:0/1278789490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a2c160b90 con 0x557a2beaf070 2024-07-31T14:59:34.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.866+0000 7f561da8c640 1 --2- 172.21.15.67:0/1278789490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2beaf070 0x557a2c2573c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f560800a840 tx=0x7f560800ad00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=293da9f379cb687d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:34.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.866+0000 7f561e28d640 1 -- 172.21.15.67:0/1278789490 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608015070 con 0x557a2beaf070 2024-07-31T14:59:34.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.867+0000 7f561e28d640 1 -- 172.21.15.67:0/1278789490 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56080113f0 con 0x557a2beaf070 2024-07-31T14:59:34.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.867+0000 7f561e28d640 1 -- 172.21.15.67:0/1278789490 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56080185c0 con 0x557a2beaf070 2024-07-31T14:59:34.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.869+0000 7f5625a40740 1 -- 172.21.15.67:0/1278789490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2beaf070 msgr2=0x557a2c2573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:34.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.869+0000 7f5625a40740 1 --2- 172.21.15.67:0/1278789490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2beaf070 0x557a2c2573c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f560800a840 tx=0x7f560800ad00 comp rx=0 tx=0).stop 2024-07-31T14:59:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.871+0000 7f5625a40740 1 -- 172.21.15.67:0/1278789490 shutdown_connections 2024-07-31T14:59:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.871+0000 7f5625a40740 1 --2- 172.21.15.67:0/1278789490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2be8e390 0x557a2c254f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.871+0000 7f5625a40740 1 --2- 172.21.15.67:0/1278789490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be861b0 0x557a2c2549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.871+0000 7f5625a40740 1 --2- 172.21.15.67:0/1278789490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2beaf070 0x557a2c2573c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.871+0000 7f5625a40740 1 -- 172.21.15.67:0/1278789490 >> 172.21.15.67:0/1278789490 conn(0x557a2bea3460 msgr2=0x557a2c2528c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:34.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.873+0000 7f560e7fc640 1 -- 172.21.15.67:0/273439479 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f561001f020 con 0x7f5618102020 2024-07-31T14:59:34.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.874+0000 7f560e7fc640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56040b3f10 0x7f56040b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:34.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.874+0000 7f560e7fc640 1 -- 172.21.15.67:0/273439479 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f56100e49c0 con 0x7f5618102020 2024-07-31T14:59:34.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.875+0000 7f5625a40740 1 -- 172.21.15.67:0/1278789490 shutdown_connections 2024-07-31T14:59:34.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.875+0000 7f5625a40740 1 -- 172.21.15.67:0/1278789490 wait complete. 2024-07-31T14:59:34.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.875+0000 7f5625a40740 1 -- start start 2024-07-31T14:59:34.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.875+0000 7f5625a40740 1 Processor -- start 2024-07-31T14:59:34.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.877+0000 7f561ca8a640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56040b3f10 0x7f56040b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:34.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.878+0000 7f5625a40740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be8e390 0x557a2c260610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:34.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.878+0000 7f5625a40740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c260bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:34.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.878+0000 7f5625a40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be6d9f0 0x557a2c26c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:34.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.878+0000 7f5625a40740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557a2c01d700 con 0x557a2beaf070 2024-07-31T14:59:34.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.878+0000 7f5625a40740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a2c15c9e0 con 0x557a2be8e390 2024-07-31T14:59:34.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.878+0000 7f5625a40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a2c262e70 con 0x557a2be6d9f0 2024-07-31T14:59:34.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.878+0000 7f5625a40740 10 client.0 _ll_register_callbacks cb 0x557a2c1774f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:34.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.879+0000 7f561ca8a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c260bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:34.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.879+0000 7f561ca8a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c260bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40774/0 (socket says 172.21.15.67:40774) 2024-07-31T14:59:34.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.879+0000 7f561ca8a640 1 -- 172.21.15.67:0/1055154127 learned_addr learned my addr 172.21.15.67:0/1055154127 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:34.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.879+0000 7f561da8c640 1 --2- 172.21.15.67:0/1055154127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be6d9f0 0x557a2c26c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:34.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.879+0000 7f561ca8a640 1 -- 172.21.15.67:0/1055154127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be6d9f0 msgr2=0x557a2c26c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:34.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.879+0000 7f561ca8a640 1 --2- 172.21.15.67:0/1055154127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be6d9f0 0x557a2c26c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:34.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.879+0000 7f561ca8a640 1 -- 172.21.15.67:0/1055154127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be8e390 msgr2=0x557a2c260610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:59:34.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.879+0000 7f561ca8a640 1 --2- 172.21.15.67:0/1055154127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be8e390 0x557a2c260610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:34.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.879+0000 7f561ca8a640 1 -- 172.21.15.67:0/1055154127 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a2bf1a650 con 0x557a2beaf070 2024-07-31T14:59:34.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.879+0000 7f561ca8a640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56040b3f10 0x7f56040b6420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5614000c00 tx=0x7f5614004870 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:34.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.880+0000 7f561ca8a640 1 --2- 172.21.15.67:0/1055154127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c260bd0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f561401a910 tx=0x7f5614019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96f07f36b2db8e1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:34.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.880+0000 7f5600ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:59:34.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.881+0000 7f5600ff9640 1 -- 172.21.15.67:0/1055154127 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614046050 con 0x557a2beaf070 2024-07-31T14:59:34.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.881+0000 7f5600ff9640 1 -- 172.21.15.67:0/1055154127 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f5614040bb0 con 0x557a2beaf070 2024-07-31T14:59:34.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.884+0000 7f5625a40740 1 -- 172.21.15.67:0/1055154127 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56080113f0 con 0x557a2beaf070 2024-07-31T14:59:34.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.886+0000 7f5600ff9640 1 -- 172.21.15.67:0/1055154127 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5614040db0 con 0x557a2beaf070 2024-07-31T14:59:34.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.887+0000 7f5625a40740 10 client.17616 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:34.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.887+0000 7f5625a40740 1 -- 172.21.15.67:0/1055154127 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x557a2c26ef80 con 0x557a2beaf070 2024-07-31T14:59:34.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.888+0000 7f5600ff9640 1 -- 172.21.15.67:0/1055154127 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56080113f0 con 0x557a2beaf070 2024-07-31T14:59:34.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.888+0000 7f5600ff9640 1 -- 172.21.15.67:0/1055154127 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f5614011430 con 0x557a2beaf070 2024-07-31T14:59:34.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.891+0000 7f5625a40740 10 client.17616 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:34.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.891+0000 7f5625a40740 10 client.17616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:34.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.891+0000 7f5625a40740 20 client.17616 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:34.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.891+0000 7f5625a40740 1 --2- 172.21.15.67:0/1055154127 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a2be42da0 0x557a2c2726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:34.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.891+0000 7f5625a40740 4 client.17616 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:59:34.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.891+0000 7f5625a40740 1 -- 172.21.15.67:0/1055154127 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a2c1ccb00 con 0x557a2be42da0 2024-07-31T14:59:34.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.893+0000 7f561d28b640 1 --2- 172.21.15.67:0/1055154127 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a2be42da0 0x557a2c2726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:34.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.907+0000 7f561d28b640 1 --2- 172.21.15.67:0/1055154127 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a2be42da0 0x557a2c2726e0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eced90b48dbbf6d7 server_cookie=2dd478f1f94f7e43 in_seq=0 out_seq=0 2024-07-31T14:59:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:34.907+0000 7f5600ff9640 10 client.17616 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:35.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:35 smithi067 ceph-mon[32750]: pgmap v659: 129 pgs: 17 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 223 KiB/s wr, 84 op/s 2024-07-31T14:59:35.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:35 smithi073 ceph-mon[36714]: pgmap v659: 129 pgs: 17 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 223 KiB/s wr, 84 op/s 2024-07-31T14:59:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:35 smithi077 ceph-mon[37327]: pgmap v659: 129 pgs: 17 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 223 KiB/s wr, 84 op/s 2024-07-31T14:59:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.378+0000 7f5600ff9640 1 -- 172.21.15.67:0/1055154127 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a2c1ccb00 con 0x557a2be42da0 2024-07-31T14:59:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.378+0000 7f5600ff9640 10 client.17616 handle_command_reply: tid=0 2024-07-31T14:59:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.378+0000 7f5625a40740 1 client.17616 shutdown 2024-07-31T14:59:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.379+0000 7f5625a40740 1 -- 172.21.15.67:0/1055154127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 msgr2=0x557a2c260bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.379+0000 7f5625a40740 1 --2- 172.21.15.67:0/1055154127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c260bd0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f561401a910 tx=0x7f5614019000 comp rx=0 tx=0).stop 2024-07-31T14:59:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.379+0000 7f5625a40740 1 -- 172.21.15.67:0/1055154127 shutdown_connections 2024-07-31T14:59:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.379+0000 7f5625a40740 1 --2- 172.21.15.67:0/1055154127 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a2be42da0 0x557a2c2726e0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.379+0000 7f5625a40740 1 --2- 172.21.15.67:0/1055154127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be6d9f0 0x557a2c26c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.379+0000 7f5625a40740 1 --2- 172.21.15.67:0/1055154127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c260bd0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.379+0000 7f5625a40740 1 --2- 172.21.15.67:0/1055154127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be8e390 0x557a2c260610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.379+0000 7f5625a40740 1 -- 172.21.15.67:0/1055154127 >> 172.21.15.67:0/1055154127 conn(0x557a2bea3460 msgr2=0x557a2c2563c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.379+0000 7f5600ff9640 0 client.17616 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.379+0000 7f5625a40740 1 -- 172.21.15.67:0/1055154127 shutdown_connections 2024-07-31T14:59:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.379+0000 7f5625a40740 1 -- 172.21.15.67:0/1055154127 wait complete. 2024-07-31T14:59:36.387 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:59:36.387 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 54, 2024-07-31T14:59:36.387 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T14:59:36.387 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 24, 2024-07-31T14:59:36.387 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T14:59:36.387 INFO:teuthology.orchestra.run.smithi067.stdout: "578e3b69": { 2024-07-31T14:59:36.387 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 24, 2024-07-31T14:59:36.387 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T14:59:36.387 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:59:36.388 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T14:59:36.388 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:59:36.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T14:59:36.388 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T14:59:36.388 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-07-31T14:59:36.388 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T14:59:36.388 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T14:59:36.388 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T14:59:36.388 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T14:59:36.388 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T14:59:36.388 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T14:59:36.388 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:59:36.388 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:59:36.389 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:59:36.389 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:59:36.389 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:59:36.389 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:59:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.384+0000 7f3d867fc640 1 -- 172.21.15.67:0/2623096664 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bc68a75860 con 0x55bc686f0250 2024-07-31T14:59:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.384+0000 7f3d867fc640 10 client.27155 handle_command_reply: tid=0 2024-07-31T14:59:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.384+0000 7f3dab77d740 1 client.27155 shutdown 2024-07-31T14:59:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.384+0000 7f3dab77d740 1 -- 172.21.15.67:0/2623096664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc68733660 msgr2=0x55bc68b163c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.384+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2623096664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc68733660 0x55bc68b163c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3d88033000 tx=0x7f3d88004330 comp rx=0 tx=0).stop 2024-07-31T14:59:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.384+0000 7f3dab77d740 1 -- 172.21.15.67:0/2623096664 shutdown_connections 2024-07-31T14:59:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.384+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2623096664 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc686f0250 0x55bc68b8aec0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.384+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2623096664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc6875c520 0x55bc68b11f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.384+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2623096664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc6873b840 0x55bc68b11870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.384+0000 7f3dab77d740 1 --2- 172.21.15.67:0/2623096664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc68733660 0x55bc68b163c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.384+0000 7f3dab77d740 1 -- 172.21.15.67:0/2623096664 >> 172.21.15.67:0/2623096664 conn(0x55bc68750910 msgr2=0x55bc68b02f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.384+0000 7f3d867fc640 0 client.27155 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.385+0000 7f3dab77d740 1 -- 172.21.15.67:0/2623096664 shutdown_connections 2024-07-31T14:59:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.385+0000 7f3dab77d740 1 -- 172.21.15.67:0/2623096664 wait complete. 2024-07-31T14:59:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.385+0000 7f3dab77d740 20 client.27155 trim_cache size 0 max 16384 2024-07-31T14:59:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.379+0000 7f5625a40740 20 client.17616 trim_cache size 0 max 16384 2024-07-31T14:59:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.388+0000 7f3da522c640 1 -- 172.21.15.67:0/1683989104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3d8c0b3f10 msgr2=0x7f3d8c0b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.388+0000 7f3da522c640 1 --2- 172.21.15.67:0/1683989104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3d8c0b3f10 0x7f3d8c0b6420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800aaf0 tx=0x7f3d98009290 comp rx=0 tx=0).stop 2024-07-31T14:59:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.388+0000 7f3da522c640 1 -- 172.21.15.67:0/1683989104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3da01075a0 msgr2=0x7f3da01136e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.388+0000 7f3da522c640 1 --2- 172.21.15.67:0/1683989104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3da01075a0 0x7f3da01136e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3d90006fd0 tx=0x7f3d9000b8e0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.388+0000 7f3da522c640 1 -- 172.21.15.67:0/1683989104 shutdown_connections 2024-07-31T14:59:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.388+0000 7f3da522c640 1 --2- 172.21.15.67:0/1683989104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3d8c0b3f10 0x7f3d8c0b6420 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.388+0000 7f3da522c640 1 --2- 172.21.15.67:0/1683989104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3da0107ef0 0x7f3da01189a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.388+0000 7f3da522c640 1 --2- 172.21.15.67:0/1683989104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3da01075a0 0x7f3da01136e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.388+0000 7f3da522c640 1 --2- 172.21.15.67:0/1683989104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3da0101c60 0x7f3da0113160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.388+0000 7f3da522c640 1 -- 172.21.15.67:0/1683989104 >> 172.21.15.67:0/1683989104 conn(0x7f3da00776f0 msgr2=0x7f3da01056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.389+0000 7f3da522c640 1 -- 172.21.15.67:0/1683989104 shutdown_connections 2024-07-31T14:59:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.389+0000 7f3da522c640 1 -- 172.21.15.67:0/1683989104 wait complete. 2024-07-31T14:59:36.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.408+0000 7f5625a40740 1 Processor -- start 2024-07-31T14:59:36.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.408+0000 7f5625a40740 1 -- start start 2024-07-31T14:59:36.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.409+0000 7f5625a40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be6d9f0 0x557a2c251c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.409+0000 7f5625a40740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be8e390 0x557a2c26bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.409+0000 7f5625a40740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c25aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.409+0000 7f561d28b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be6d9f0 0x557a2c251c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.409+0000 7f561d28b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be6d9f0 0x557a2c251c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38790/0 (socket says 172.21.15.67:38790) 2024-07-31T14:59:36.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.409+0000 7f561ca8a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be8e390 0x557a2c26bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:36.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.410+0000 7f561ca8a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be8e390 0x557a2c26bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:33056/0 (socket says 172.21.15.67:33056) 2024-07-31T14:59:36.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.410+0000 7f5625a40740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f56181d0780 con 0x557a2beaf070 2024-07-31T14:59:36.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.410+0000 7f561da8c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c25aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:36.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.410+0000 7f561da8c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c25aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40780/0 (socket says 172.21.15.67:40780) 2024-07-31T14:59:36.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.410+0000 7f561da8c640 1 -- 172.21.15.67:0/2736720576 learned_addr learned my addr 172.21.15.67:0/2736720576 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:36.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.411+0000 7f5625a40740 1 -- 172.21.15.67:0/2736720576 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f56181d2410 con 0x557a2be8e390 2024-07-31T14:59:36.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.411+0000 7f5625a40740 1 -- 172.21.15.67:0/2736720576 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f56181d0e50 con 0x557a2be6d9f0 2024-07-31T14:59:36.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.414+0000 7f561ca8a640 1 -- 172.21.15.67:0/2736720576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be6d9f0 msgr2=0x557a2c251c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:36.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.414+0000 7f561ca8a640 1 --2- 172.21.15.67:0/2736720576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be6d9f0 0x557a2c251c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.414+0000 7f561ca8a640 1 -- 172.21.15.67:0/2736720576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 msgr2=0x557a2c25aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:36.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.414+0000 7f561ca8a640 1 --2- 172.21.15.67:0/2736720576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c25aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.414+0000 7f561ca8a640 1 -- 172.21.15.67:0/2736720576 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a2c253250 con 0x557a2be8e390 2024-07-31T14:59:36.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.417+0000 7f561ca8a640 1 --2- 172.21.15.67:0/2736720576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be8e390 0x557a2c26bfc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f561401a940 tx=0x7f5614004e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54839784c49438bc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:36.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.419+0000 7f5602ffd640 1 -- 172.21.15.67:0/2736720576 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614063070 con 0x557a2be8e390 2024-07-31T14:59:36.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.419+0000 7f5602ffd640 1 -- 172.21.15.67:0/2736720576 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5614011ae0 con 0x557a2be8e390 2024-07-31T14:59:36.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.421+0000 7f5625a40740 1 -- 172.21.15.67:0/2736720576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be8e390 msgr2=0x557a2c26bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:36.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.421+0000 7f5625a40740 1 --2- 172.21.15.67:0/2736720576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be8e390 0x557a2c26bfc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f561401a940 tx=0x7f5614004e80 comp rx=0 tx=0).stop 2024-07-31T14:59:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.426+0000 7f5625a40740 1 -- 172.21.15.67:0/2736720576 shutdown_connections 2024-07-31T14:59:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.426+0000 7f5625a40740 1 --2- 172.21.15.67:0/2736720576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c25aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.426+0000 7f5625a40740 1 --2- 172.21.15.67:0/2736720576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be8e390 0x557a2c26bfc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.426+0000 7f5625a40740 1 --2- 172.21.15.67:0/2736720576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be6d9f0 0x557a2c251c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.426+0000 7f5625a40740 1 -- 172.21.15.67:0/2736720576 >> 172.21.15.67:0/2736720576 conn(0x557a2bea3460 msgr2=0x557a2c2545c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:36.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.428+0000 7f5625a40740 1 -- 172.21.15.67:0/2736720576 shutdown_connections 2024-07-31T14:59:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.428+0000 7f5625a40740 1 -- 172.21.15.67:0/2736720576 wait complete. 2024-07-31T14:59:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.428+0000 7f5625a40740 1 -- start start 2024-07-31T14:59:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.429+0000 7f5625a40740 1 Processor -- start 2024-07-31T14:59:36.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.429+0000 7f5625a40740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be6d9f0 0x557a2c2770e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:36.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.429+0000 7f5625a40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be8e390 0x557a2c277620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:36.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.429+0000 7f5625a40740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c267130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:36.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.429+0000 7f5625a40740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f56181d1d40 con 0x557a2beaf070 2024-07-31T14:59:36.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.429+0000 7f5625a40740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f56181d15b0 con 0x557a2be6d9f0 2024-07-31T14:59:36.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.429+0000 7f5625a40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f56181ce780 con 0x557a2be8e390 2024-07-31T14:59:36.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.429+0000 7f5625a40740 10 client.0 _ll_register_callbacks cb 0x7f56100093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:36.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.431+0000 7f561d28b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be6d9f0 0x557a2c2770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:36.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.431+0000 7f561d28b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be6d9f0 0x557a2c2770e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:33058/0 (socket says 172.21.15.67:33058) 2024-07-31T14:59:36.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.431+0000 7f561d28b640 1 -- 172.21.15.67:0/541546172 learned_addr learned my addr 172.21.15.67:0/541546172 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:36.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.431+0000 7f561da8c640 1 --2- 172.21.15.67:0/541546172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c267130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:36.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.432+0000 7f561ca8a640 1 --2- 172.21.15.67:0/541546172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be8e390 0x557a2c277620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:36.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.434+0000 7f561da8c640 1 -- 172.21.15.67:0/541546172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be8e390 msgr2=0x557a2c277620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:36.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.434+0000 7f561da8c640 1 --2- 172.21.15.67:0/541546172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be8e390 0x557a2c277620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.434+0000 7f561da8c640 1 -- 172.21.15.67:0/541546172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be6d9f0 msgr2=0x557a2c2770e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:36.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.434+0000 7f561da8c640 1 --2- 172.21.15.67:0/541546172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be6d9f0 0x557a2c2770e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.434+0000 7f561da8c640 1 -- 172.21.15.67:0/541546172 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a2c277e70 con 0x557a2beaf070 2024-07-31T14:59:36.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.434+0000 7f561da8c640 1 --2- 172.21.15.67:0/541546172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c267130 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f560800c520 tx=0x7f5608004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af70a6f35c019e09 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:36.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.437+0000 7f5600ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:59:36.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.437+0000 7f5600ff9640 1 -- 172.21.15.67:0/541546172 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608015040 con 0x557a2beaf070 2024-07-31T14:59:36.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.437+0000 7f5600ff9640 1 -- 172.21.15.67:0/541546172 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f56080038b0 con 0x557a2beaf070 2024-07-31T14:59:36.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.437+0000 7f5625a40740 1 -- 172.21.15.67:0/541546172 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5614011ae0 con 0x557a2beaf070 2024-07-31T14:59:36.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.438+0000 7f5600ff9640 1 -- 172.21.15.67:0/541546172 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f560800a430 con 0x557a2beaf070 2024-07-31T14:59:36.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.440+0000 7f5625a40740 10 client.17622 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:36.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.440+0000 7f5625a40740 1 -- 172.21.15.67:0/541546172 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x557a2c268680 con 0x557a2beaf070 2024-07-31T14:59:36.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.443+0000 7f5600ff9640 1 -- 172.21.15.67:0/541546172 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5614011ae0 con 0x557a2beaf070 2024-07-31T14:59:36.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.443+0000 7f5600ff9640 1 -- 172.21.15.67:0/541546172 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f560802d030 con 0x557a2beaf070 2024-07-31T14:59:36.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.444+0000 7f5625a40740 10 client.17622 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:36.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.444+0000 7f5625a40740 10 client.17622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:36.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.444+0000 7f5625a40740 20 client.17622 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:36.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.444+0000 7f5625a40740 1 --2- 172.21.15.67:0/541546172 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a2c268b20 0x557a2c2dcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:36.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.444+0000 7f5625a40740 4 client.17622 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T14:59:36.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.444+0000 7f5625a40740 1 -- 172.21.15.67:0/541546172 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a2c1c85a0 con 0x557a2c268b20 2024-07-31T14:59:36.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.445+0000 7f561d28b640 1 --2- 172.21.15.67:0/541546172 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a2c268b20 0x557a2c2dcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:36.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.451+0000 7f561d28b640 1 --2- 172.21.15.67:0/541546172 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a2c268b20 0x557a2c2dcf20 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c105b347b666bfa6 server_cookie=13006e553695fa5c in_seq=0 out_seq=0 2024-07-31T14:59:36.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.451+0000 7f5600ff9640 10 client.17622 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:36.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.453+0000 7f5600ff9640 1 -- 172.21.15.67:0/541546172 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a2c1c85a0 con 0x557a2c268b20 2024-07-31T14:59:36.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.453+0000 7f5600ff9640 10 client.17622 handle_command_reply: tid=0 2024-07-31T14:59:36.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.453+0000 7f5625a40740 1 client.17622 shutdown 2024-07-31T14:59:36.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.454+0000 7f5625a40740 1 -- 172.21.15.67:0/541546172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 msgr2=0x557a2c267130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:36.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.454+0000 7f5625a40740 1 --2- 172.21.15.67:0/541546172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c267130 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f560800c520 tx=0x7f5608004270 comp rx=0 tx=0).stop 2024-07-31T14:59:36.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.454+0000 7f5625a40740 1 -- 172.21.15.67:0/541546172 shutdown_connections 2024-07-31T14:59:36.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.454+0000 7f5625a40740 1 --2- 172.21.15.67:0/541546172 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a2c268b20 0x557a2c2dcf20 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.454+0000 7f5625a40740 1 --2- 172.21.15.67:0/541546172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a2beaf070 0x557a2c267130 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.454+0000 7f5625a40740 1 --2- 172.21.15.67:0/541546172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a2be8e390 0x557a2c277620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.454+0000 7f5625a40740 1 --2- 172.21.15.67:0/541546172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2be6d9f0 0x557a2c2770e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.454+0000 7f5625a40740 1 -- 172.21.15.67:0/541546172 >> 172.21.15.67:0/541546172 conn(0x557a2bea3460 msgr2=0x557a2c2618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.454+0000 7f5600ff9640 0 client.17622 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.454+0000 7f5625a40740 1 -- 172.21.15.67:0/541546172 shutdown_connections 2024-07-31T14:59:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.455+0000 7f5625a40740 1 -- 172.21.15.67:0/541546172 wait complete. 2024-07-31T14:59:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.455+0000 7f5625a40740 20 client.17622 trim_cache size 0 max 16384 2024-07-31T14:59:36.457 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T14:59:36.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.458+0000 7f5603fff640 1 -- 172.21.15.67:0/273439479 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56040b3f10 msgr2=0x7f56040b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:36.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.458+0000 7f5603fff640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56040b3f10 0x7f56040b6420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5614000c00 tx=0x7f5614004870 comp rx=0 tx=0).stop 2024-07-31T14:59:36.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.458+0000 7f5603fff640 1 -- 172.21.15.67:0/273439479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5618102020 msgr2=0x7f5618116700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:36.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.459+0000 7f5603fff640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5618102020 0x7f5618116700 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f561000ebc0 tx=0x7f561000c6a0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.459+0000 7f5603fff640 1 -- 172.21.15.67:0/273439479 shutdown_connections 2024-07-31T14:59:36.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.459+0000 7f5603fff640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f56040b3f10 0x7f56040b6420 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.459+0000 7f5603fff640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5618107460 0x7f5618113130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.459+0000 7f5603fff640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5618102970 0x7f56181129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.459+0000 7f5603fff640 1 --2- 172.21.15.67:0/273439479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5618102020 0x7f5618116700 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:36.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.459+0000 7f5603fff640 1 -- 172.21.15.67:0/273439479 >> 172.21.15.67:0/273439479 conn(0x7f56180fdd30 msgr2=0x7f5618104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:36.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.459+0000 7f5603fff640 1 -- 172.21.15.67:0/273439479 shutdown_connections 2024-07-31T14:59:36.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:36.459+0000 7f5603fff640 1 -- 172.21.15.67:0/273439479 wait complete. 2024-07-31T14:59:37.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:37 smithi067 ceph-mon[32750]: pgmap v660: 129 pgs: 16 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 223 KiB/s wr, 82 op/s 2024-07-31T14:59:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:37 smithi077 ceph-mon[37327]: pgmap v660: 129 pgs: 16 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 223 KiB/s wr, 82 op/s 2024-07-31T14:59:37.561 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '578e3b69', total seconds elapsed: 47.8 2024-07-31T14:59:37.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:37 smithi073 ceph-mon[36714]: pgmap v660: 129 pgs: 16 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 223 KiB/s wr, 82 op/s 2024-07-31T14:59:39.054 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T14:59:39.054 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T14:59:39.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:39 smithi077 ceph-mon[37327]: pgmap v661: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 342 KiB/s wr, 84 op/s 2024-07-31T14:59:39.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:39 smithi067 ceph-mon[32750]: pgmap v661: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 342 KiB/s wr, 84 op/s 2024-07-31T14:59:39.555 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:59:39.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:39 smithi073 ceph-mon[36714]: pgmap v661: 129 pgs: 15 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 66 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 342 KiB/s wr, 84 op/s 2024-07-31T14:59:41.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:41 smithi067 ceph-mon[32750]: pgmap v662: 129 pgs: 12 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 84 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 365 KiB/s wr, 86 op/s 2024-07-31T14:59:41.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:41 smithi077 ceph-mon[37327]: pgmap v662: 129 pgs: 12 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 84 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 365 KiB/s wr, 86 op/s 2024-07-31T14:59:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.619+0000 7fdafcf0e640 1 -- 172.21.15.67:0/618261743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaf810b970 msgr2=0x7fdaf810bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.619+0000 7fdafcf0e640 1 --2- 172.21.15.67:0/618261743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaf810b970 0x7fdaf810bd80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009ca0 tx=0x7fdaf002a8f0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.619+0000 7fdafcf0e640 1 -- 172.21.15.67:0/618261743 shutdown_connections 2024-07-31T14:59:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.619+0000 7fdafcf0e640 1 --2- 172.21.15.67:0/618261743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf8114ea0 0x7fdaf81173b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.619+0000 7fdafcf0e640 1 --2- 172.21.15.67:0/618261743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf810c2c0 0x7fdaf8114800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.619+0000 7fdafcf0e640 1 --2- 172.21.15.67:0/618261743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaf810b970 0x7fdaf810bd80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.619+0000 7fdafcf0e640 1 -- 172.21.15.67:0/618261743 >> 172.21.15.67:0/618261743 conn(0x7fdaf80707c0 msgr2=0x7fdaf8070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.622+0000 7fdafcf0e640 1 -- 172.21.15.67:0/618261743 shutdown_connections 2024-07-31T14:59:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.622+0000 7fdafcf0e640 1 -- 172.21.15.67:0/618261743 wait complete. 2024-07-31T14:59:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.623+0000 7fdafcf0e640 1 Processor -- start 2024-07-31T14:59:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.623+0000 7fdafcf0e640 1 -- start start 2024-07-31T14:59:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.623+0000 7fdafcf0e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf810b970 0x7fdaf811c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.623+0000 7fdafcf0e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf810c2c0 0x7fdaf811c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.623+0000 7fdafcf0e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaf8114ea0 0x7fdaf8127440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.623+0000 7fdafcf0e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8119b00 con 0x7fdaf810c2c0 2024-07-31T14:59:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.623+0000 7fdafcf0e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8119980 con 0x7fdaf810b970 2024-07-31T14:59:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.623+0000 7fdafcf0e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8119c80 con 0x7fdaf8114ea0 2024-07-31T14:59:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.624+0000 7fdaf659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf810b970 0x7fdaf811c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.624+0000 7fdaf659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf810b970 0x7fdaf811c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51694/0 (socket says 172.21.15.67:51694) 2024-07-31T14:59:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.624+0000 7fdaf659c640 1 -- 172.21.15.67:0/2692003606 learned_addr learned my addr 172.21.15.67:0/2692003606 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.624+0000 7fdaf6d9d640 1 --2- 172.21.15.67:0/2692003606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaf8114ea0 0x7fdaf8127440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.625+0000 7fdaf5d9b640 1 --2- 172.21.15.67:0/2692003606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf810c2c0 0x7fdaf811c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.625+0000 7fdaf659c640 1 -- 172.21.15.67:0/2692003606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaf8114ea0 msgr2=0x7fdaf8127440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.625+0000 7fdaf659c640 1 --2- 172.21.15.67:0/2692003606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaf8114ea0 0x7fdaf8127440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.625+0000 7fdaf659c640 1 -- 172.21.15.67:0/2692003606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf810c2c0 msgr2=0x7fdaf811c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.625+0000 7fdaf659c640 1 --2- 172.21.15.67:0/2692003606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf810c2c0 0x7fdaf811c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.625+0000 7fdaf659c640 1 -- 172.21.15.67:0/2692003606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf8127980 con 0x7fdaf810b970 2024-07-31T14:59:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.630+0000 7fdaf659c640 1 --2- 172.21.15.67:0/2692003606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf810b970 0x7fdaf811c3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0005e30 tx=0x7fdaf0031720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.634+0000 7fdae77fe640 1 -- 172.21.15.67:0/2692003606 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0003b70 con 0x7fdaf810b970 2024-07-31T14:59:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.634+0000 7fdae77fe640 1 -- 172.21.15.67:0/2692003606 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf002d070 con 0x7fdaf810b970 2024-07-31T14:59:41.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.636+0000 7fdae77fe640 1 -- 172.21.15.67:0/2692003606 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf002f8a0 con 0x7fdaf810b970 2024-07-31T14:59:41.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.638+0000 7fdafcf0e640 1 -- 172.21.15.67:0/2692003606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf8127c10 con 0x7fdaf810b970 2024-07-31T14:59:41.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.638+0000 7fdafcf0e640 1 -- 172.21.15.67:0/2692003606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf81ace40 con 0x7fdaf810b970 2024-07-31T14:59:41.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:41 smithi073 ceph-mon[36714]: pgmap v662: 129 pgs: 12 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 84 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 365 KiB/s wr, 86 op/s 2024-07-31T14:59:41.645 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.644+0000 7fdae77fe640 1 -- 172.21.15.67:0/2692003606 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fdaf0004040 con 0x7fdaf810b970 2024-07-31T14:59:41.645 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.644+0000 7fdb03e01740 1 Processor -- start 2024-07-31T14:59:41.645 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.644+0000 7fdb03e01740 1 -- start start 2024-07-31T14:59:41.646 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.645+0000 7fdb03e01740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0254b71b0 0x55e025859ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.646 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.645+0000 7fdb03e01740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02548e2f0 0x55e02585a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.645+0000 7fdb03e01740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0254964d0 0x55e02585a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.645+0000 7fdb03e01740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e0257cf3b0 con 0x55e0254964d0 2024-07-31T14:59:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.645+0000 7fdb03e01740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e0257be8b0 con 0x55e02548e2f0 2024-07-31T14:59:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.645+0000 7fdb03e01740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e0257c28f0 con 0x55e0254b71b0 2024-07-31T14:59:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.645+0000 7fdaf6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0254b71b0 0x55e025859ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.645+0000 7fdaf659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02548e2f0 0x55e02585a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.645+0000 7fdaf659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02548e2f0 0x55e02585a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51706/0 (socket says 172.21.15.67:51706) 2024-07-31T14:59:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.645+0000 7fdaf659c640 1 -- 172.21.15.67:0/428372920 learned_addr learned my addr 172.21.15.67:0/428372920 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.646+0000 7fdaf6d9d640 1 -- 172.21.15.67:0/428372920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02548e2f0 msgr2=0x55e02585a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.646+0000 7fdaf6d9d640 1 --2- 172.21.15.67:0/428372920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02548e2f0 0x55e02585a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.646+0000 7fdaf6d9d640 1 -- 172.21.15.67:0/428372920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0254964d0 msgr2=0x55e02585a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.646+0000 7fdaf6d9d640 1 --2- 172.21.15.67:0/428372920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0254964d0 0x55e02585a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.646+0000 7fdaf6d9d640 1 -- 172.21.15.67:0/428372920 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0257659a0 con 0x55e0254b71b0 2024-07-31T14:59:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.646+0000 7fdaf6d9d640 1 --2- 172.21.15.67:0/428372920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0254b71b0 0x55e025859ce0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009bb0 tx=0x7fdaec011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391244fc5895e454 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:41.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.647+0000 7fdae77fe640 1 --2- 172.21.15.67:0/2692003606 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdae00b3d20 0x7fdae00b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.648+0000 7fdae4ff9640 1 -- 172.21.15.67:0/428372920 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec004230 con 0x55e0254b71b0 2024-07-31T14:59:41.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.648+0000 7fdae4ff9640 1 -- 172.21.15.67:0/428372920 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaec009e20 con 0x55e0254b71b0 2024-07-31T14:59:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.649+0000 7fdae77fe640 1 -- 172.21.15.67:0/2692003606 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7fdaf0105730 con 0x7fdaf810b970 2024-07-31T14:59:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.649+0000 7fdb03e01740 1 -- 172.21.15.67:0/428372920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0254b71b0 msgr2=0x55e025859ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.649+0000 7fdb03e01740 1 --2- 172.21.15.67:0/428372920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0254b71b0 0x55e025859ce0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009bb0 tx=0x7fdaec011810 comp rx=0 tx=0).stop 2024-07-31T14:59:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.649+0000 7fdaf659c640 1 --2- 172.21.15.67:0/2692003606 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdae00b3d20 0x7fdae00b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.650+0000 7fdb03e01740 1 -- 172.21.15.67:0/428372920 shutdown_connections 2024-07-31T14:59:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.650+0000 7fdb03e01740 1 --2- 172.21.15.67:0/428372920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0254964d0 0x55e02585a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.650+0000 7fdb03e01740 1 --2- 172.21.15.67:0/428372920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02548e2f0 0x55e02585a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.650+0000 7fdb03e01740 1 --2- 172.21.15.67:0/428372920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0254b71b0 0x55e025859ce0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.650+0000 7fdb03e01740 1 -- 172.21.15.67:0/428372920 >> 172.21.15.67:0/428372920 conn(0x55e0254ab300 msgr2=0x55e025857bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:41.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.650+0000 7fdaf659c640 1 --2- 172.21.15.67:0/2692003606 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdae00b3d20 0x7fdae00b6230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00cbd30 tx=0x7fdaf010b530 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.652+0000 7fdb03e01740 1 -- 172.21.15.67:0/428372920 shutdown_connections 2024-07-31T14:59:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.653+0000 7fdb03e01740 1 -- 172.21.15.67:0/428372920 wait complete. 2024-07-31T14:59:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.653+0000 7fdb03e01740 1 -- start start 2024-07-31T14:59:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.653+0000 7fdb03e01740 1 Processor -- start 2024-07-31T14:59:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.654+0000 7fdb03e01740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e025865520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.654+0000 7fdb03e01740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0254964d0 0x55e025865ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.654+0000 7fdb03e01740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0254b71b0 0x55e025871850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.654+0000 7fdb03e01740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e0256f8ab0 con 0x55e0254b71b0 2024-07-31T14:59:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.654+0000 7fdb03e01740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e025746cf0 con 0x55e0254964d0 2024-07-31T14:59:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.654+0000 7fdb03e01740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e0258684b0 con 0x55e02548e2f0 2024-07-31T14:59:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.654+0000 7fdb03e01740 10 client.0 _ll_register_callbacks cb 0x55e0257b89f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.654+0000 7fdaf5d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0254964d0 0x55e025865ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.654+0000 7fdaf5d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0254964d0 0x55e025865ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51712/0 (socket says 172.21.15.67:51712) 2024-07-31T14:59:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.654+0000 7fdaf5d9b640 1 -- 172.21.15.67:0/1508811837 learned_addr learned my addr 172.21.15.67:0/1508811837 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.654+0000 7fdaf6d9d640 1 --2- 172.21.15.67:0/1508811837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e025865520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.655+0000 7fdaf5d9b640 1 -- 172.21.15.67:0/1508811837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 msgr2=0x55e025865520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.655+0000 7fdaf5d9b640 1 --2- 172.21.15.67:0/1508811837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e025865520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.655+0000 7fdaf5d9b640 1 -- 172.21.15.67:0/1508811837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0254b71b0 msgr2=0x55e025871850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.655+0000 7fdaf5d9b640 1 --2- 172.21.15.67:0/1508811837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0254b71b0 0x55e025871850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.655+0000 7fdaf5d9b640 1 -- 172.21.15.67:0/1508811837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e025522790 con 0x55e0254964d0 2024-07-31T14:59:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.655+0000 7fdaf5d9b640 1 --2- 172.21.15.67:0/1508811837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0254964d0 0x55e025865ae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdae800aef0 tx=0x7fdae800c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a00a0debe80a39ec server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.657+0000 7fdaddffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:59:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.657+0000 7fdaddffb640 1 -- 172.21.15.67:0/1508811837 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8015070 con 0x55e0254964d0 2024-07-31T14:59:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.657+0000 7fdb03e01740 1 -- 172.21.15.67:0/1508811837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaec009e20 con 0x55e0254964d0 2024-07-31T14:59:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.657+0000 7fdaddffb640 1 -- 172.21.15.67:0/1508811837 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7fdae80115f0 con 0x55e0254964d0 2024-07-31T14:59:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.657+0000 7fdaddffb640 1 -- 172.21.15.67:0/1508811837 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fdae8021050 con 0x55e0254964d0 2024-07-31T14:59:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.658+0000 7fdb03e01740 10 client.27176 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.658+0000 7fdb03e01740 1 -- 172.21.15.67:0/1508811837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e02559b620 con 0x55e0254964d0 2024-07-31T14:59:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.660+0000 7fdaddffb640 1 -- 172.21.15.67:0/1508811837 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaec009e20 con 0x55e0254964d0 2024-07-31T14:59:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.662+0000 7fdaddffb640 1 -- 172.21.15.67:0/1508811837 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7fdae8019a20 con 0x55e0254964d0 2024-07-31T14:59:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.662+0000 7fdb03e01740 10 client.27176 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.662+0000 7fdb03e01740 10 client.27176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.662+0000 7fdb03e01740 20 client.27176 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.662+0000 7fdb03e01740 1 --2- 172.21.15.67:0/1508811837 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e025475d80 0x55e025877d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.662+0000 7fdb03e01740 4 client.27176 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:59:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.662+0000 7fdb03e01740 1 -- 172.21.15.67:0/1508811837 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0257d1300 con 0x55e025475d80 2024-07-31T14:59:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.662+0000 7fdaf6d9d640 1 --2- 172.21.15.67:0/1508811837 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e025475d80 0x55e025877d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.663+0000 7fdaf6d9d640 1 --2- 172.21.15.67:0/1508811837 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e025475d80 0x55e025877d10 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0f428101728fe1 server_cookie=9354e5e88832d160 in_seq=0 out_seq=0 2024-07-31T14:59:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.663+0000 7fdaddffb640 10 client.27176 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.666+0000 7fdaddffb640 1 -- 172.21.15.67:0/1508811837 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0257d1300 con 0x55e025475d80 2024-07-31T14:59:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.666+0000 7fdaddffb640 10 client.27176 handle_command_reply: tid=0 2024-07-31T14:59:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.666+0000 7fdb03e01740 1 client.27176 shutdown 2024-07-31T14:59:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.669+0000 7fdb03e01740 1 -- 172.21.15.67:0/1508811837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0254964d0 msgr2=0x55e025865ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.669+0000 7fdb03e01740 1 --2- 172.21.15.67:0/1508811837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0254964d0 0x55e025865ae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdae800aef0 tx=0x7fdae800c520 comp rx=0 tx=0).stop 2024-07-31T14:59:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.669+0000 7fdb03e01740 1 -- 172.21.15.67:0/1508811837 shutdown_connections 2024-07-31T14:59:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.669+0000 7fdb03e01740 1 --2- 172.21.15.67:0/1508811837 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e025475d80 0x55e025877d10 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.669+0000 7fdb03e01740 1 --2- 172.21.15.67:0/1508811837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0254b71b0 0x55e025871850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.669+0000 7fdb03e01740 1 --2- 172.21.15.67:0/1508811837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0254964d0 0x55e025865ae0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.669+0000 7fdb03e01740 1 --2- 172.21.15.67:0/1508811837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e025865520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.669+0000 7fdb03e01740 1 -- 172.21.15.67:0/1508811837 >> 172.21.15.67:0/1508811837 conn(0x55e0254ab300 msgr2=0x55e0258562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.669+0000 7fdaddffb640 0 client.27176 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.669+0000 7fdb03e01740 1 -- 172.21.15.67:0/1508811837 shutdown_connections 2024-07-31T14:59:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.670+0000 7fdb03e01740 1 -- 172.21.15.67:0/1508811837 wait complete. 2024-07-31T14:59:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.671+0000 7fdb03e01740 20 client.27176 trim_cache size 0 max 16384 2024-07-31T14:59:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.707+0000 7fdb03e01740 1 Processor -- start 2024-07-31T14:59:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.707+0000 7fdb03e01740 1 -- start start 2024-07-31T14:59:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.708+0000 7fdb03e01740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e025865b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.708+0000 7fdb03e01740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0254b71b0 0x55e025871640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.708+0000 7fdaf6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e025865b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.708+0000 7fdaf6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e025865b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:40780/0 (socket says 172.21.15.67:40780) 2024-07-31T14:59:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.708+0000 7fdaf6d9d640 1 -- 172.21.15.67:0/3638767222 learned_addr learned my addr 172.21.15.67:0/3638767222 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.708+0000 7fdaf5d9b640 1 --2- 172.21.15.67:0/3638767222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0254b71b0 0x55e025871640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.712+0000 7fdb03e01740 1 --2- 172.21.15.67:0/3638767222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02544b1c0 0x55e025871b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.712+0000 7fdb03e01740 1 -- 172.21.15.67:0/3638767222 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81b6c00 con 0x55e0254b71b0 2024-07-31T14:59:41.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.711+0000 7f4719061640 1 -- 172.21.15.67:0/372261261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f471410ae50 msgr2=0x7f471410dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:41.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.711+0000 7f4719061640 1 --2- 172.21.15.67:0/372261261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f471410ae50 0x7f471410dbb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4708013150 tx=0x7f47080327a0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.711+0000 7f4719061640 1 -- 172.21.15.67:0/372261261 shutdown_connections 2024-07-31T14:59:41.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.711+0000 7f4719061640 1 --2- 172.21.15.67:0/372261261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f471410b260 0x7f471410e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.711+0000 7f4719061640 1 --2- 172.21.15.67:0/372261261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f471410ae50 0x7f471410dbb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.711+0000 7f4719061640 1 --2- 172.21.15.67:0/372261261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4714109df0 0x7f471410a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.711+0000 7f4719061640 1 -- 172.21.15.67:0/372261261 >> 172.21.15.67:0/372261261 conn(0x7f4714070d20 msgr2=0x7f4714073880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:41.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.711+0000 7f4719061640 1 -- 172.21.15.67:0/372261261 shutdown_connections 2024-07-31T14:59:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.712+0000 7fdb03e01740 1 -- 172.21.15.67:0/3638767222 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81b8890 con 0x55e02544b1c0 2024-07-31T14:59:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.712+0000 7fdb03e01740 1 -- 172.21.15.67:0/3638767222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81b72d0 con 0x55e02548e2f0 2024-07-31T14:59:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.716+0000 7fdaf6d9d640 1 --2- 172.21.15.67:0/3638767222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02544b1c0 0x55e025871b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.711+0000 7f4719061640 1 -- 172.21.15.67:0/372261261 wait complete. 2024-07-31T14:59:41.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.712+0000 7f4719061640 1 Processor -- start 2024-07-31T14:59:41.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.713+0000 7f4719061640 1 -- start start 2024-07-31T14:59:41.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.713+0000 7f4719061640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4714109df0 0x7f47141b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.713+0000 7f4719061640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f471410ae50 0x7f47141b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.713+0000 7f4719061640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f471410b260 0x7f47141b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.713+0000 7f4719061640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4714113b20 con 0x7f471410ae50 2024-07-31T14:59:41.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.713+0000 7f4719061640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47141139a0 con 0x7f471410b260 2024-07-31T14:59:41.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.713+0000 7f4719061640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4714113ca0 con 0x7f4714109df0 2024-07-31T14:59:41.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.714+0000 7f471359e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f471410b260 0x7f47141b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.714+0000 7f471359e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f471410b260 0x7f47141b6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51728/0 (socket says 172.21.15.67:51728) 2024-07-31T14:59:41.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.714+0000 7f471359e640 1 -- 172.21.15.67:0/2331680308 learned_addr learned my addr 172.21.15.67:0/2331680308 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:41.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.714+0000 7f471359e640 1 -- 172.21.15.67:0/2331680308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4714109df0 msgr2=0x7f47141b1570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T14:59:41.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.714+0000 7f471359e640 1 --2- 172.21.15.67:0/2331680308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4714109df0 0x7f47141b1570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.714+0000 7f471359e640 1 -- 172.21.15.67:0/2331680308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f471410ae50 msgr2=0x7f47141b1b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T14:59:41.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.714+0000 7f471359e640 1 --2- 172.21.15.67:0/2331680308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f471410ae50 0x7f47141b1b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.714+0000 7f471359e640 1 -- 172.21.15.67:0/2331680308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47141b74e0 con 0x7f471410b260 2024-07-31T14:59:41.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.714+0000 7f471359e640 1 --2- 172.21.15.67:0/2331680308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f471410b260 0x7f47141b6db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f470400e970 tx=0x7f470400ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:41.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.715+0000 7f46fbfff640 1 -- 172.21.15.67:0/2331680308 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470400ccc0 con 0x7f471410b260 2024-07-31T14:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.716+0000 7fdaf6d9d640 1 -- 172.21.15.67:0/3638767222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02544b1c0 msgr2=0x55e025871b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.716+0000 7fdaf6d9d640 1 --2- 172.21.15.67:0/3638767222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02544b1c0 0x55e025871b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.716+0000 7fdaf6d9d640 1 -- 172.21.15.67:0/3638767222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0254b71b0 msgr2=0x55e025871640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.716+0000 7fdaf6d9d640 1 --2- 172.21.15.67:0/3638767222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0254b71b0 0x55e025871640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.716+0000 7fdaf6d9d640 1 -- 172.21.15.67:0/3638767222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e025858110 con 0x55e02548e2f0 2024-07-31T14:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.717+0000 7fdaf6d9d640 1 --2- 172.21.15.67:0/3638767222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e025865b90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdaec011d10 tx=0x7fdaec009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f69e28cbb89bfe13 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.717+0000 7fdadffff640 1 -- 172.21.15.67:0/3638767222 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec009800 con 0x55e02548e2f0 2024-07-31T14:59:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.718+0000 7fdadffff640 1 -- 172.21.15.67:0/3638767222 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaec005420 con 0x55e02548e2f0 2024-07-31T14:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.718+0000 7fdadffff640 1 -- 172.21.15.67:0/3638767222 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec031c70 con 0x55e02548e2f0 2024-07-31T14:59:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.719+0000 7fdb03e01740 1 -- 172.21.15.67:0/3638767222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 msgr2=0x55e025865b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.719+0000 7fdb03e01740 1 --2- 172.21.15.67:0/3638767222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e025865b90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdaec011d10 tx=0x7fdaec009490 comp rx=0 tx=0).stop 2024-07-31T14:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.719+0000 7fdb03e01740 1 -- 172.21.15.67:0/3638767222 shutdown_connections 2024-07-31T14:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.719+0000 7fdb03e01740 1 --2- 172.21.15.67:0/3638767222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02544b1c0 0x55e025871b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.719+0000 7fdb03e01740 1 --2- 172.21.15.67:0/3638767222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0254b71b0 0x55e025871640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.719+0000 7fdb03e01740 1 --2- 172.21.15.67:0/3638767222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e025865b90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.719+0000 7fdb03e01740 1 -- 172.21.15.67:0/3638767222 >> 172.21.15.67:0/3638767222 conn(0x55e0254ab300 msgr2=0x55e025863d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.720+0000 7fdb03e01740 1 -- 172.21.15.67:0/3638767222 shutdown_connections 2024-07-31T14:59:41.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.715+0000 7f4719061640 1 -- 172.21.15.67:0/2331680308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47141b77d0 con 0x7f471410b260 2024-07-31T14:59:41.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.715+0000 7f4719061640 1 -- 172.21.15.67:0/2331680308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47141bdf30 con 0x7f471410b260 2024-07-31T14:59:41.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.719+0000 7f471f5b2740 1 Processor -- start 2024-07-31T14:59:41.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.719+0000 7f471f5b2740 1 -- start start 2024-07-31T14:59:41.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.719+0000 7f471f5b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a258350 0x55ff4a5fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.719+0000 7f471f5b2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a5fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.719+0000 7f471f5b2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a237670 0x55ff4a5fde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.719+0000 7f471f5b2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ff4a4e52f0 con 0x55ff4a237670 2024-07-31T14:59:41.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.719+0000 7f471f5b2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ff4a573a80 con 0x55ff4a22f490 2024-07-31T14:59:41.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.719+0000 7f471f5b2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ff4a562f80 con 0x55ff4a258350 2024-07-31T14:59:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.721+0000 7fdb03e01740 1 -- 172.21.15.67:0/3638767222 wait complete. 2024-07-31T14:59:41.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.719+0000 7f46fbfff640 1 -- 172.21.15.67:0/2331680308 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f470400ce60 con 0x7f471410b260 2024-07-31T14:59:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.722+0000 7fdb03e01740 1 -- start start 2024-07-31T14:59:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.722+0000 7fdb03e01740 1 Processor -- start 2024-07-31T14:59:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.722+0000 7fdb03e01740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e02544b1c0 0x55e02586dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.723+0000 7fdb03e01740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e02586a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.723+0000 7fdb03e01740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0254b71b0 0x55e02586acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.723+0000 7fdaf5d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e02586a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.723+0000 7fdaf5d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e02586a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:40794/0 (socket says 172.21.15.67:40794) 2024-07-31T14:59:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.719+0000 7f46fbfff640 1 -- 172.21.15.67:0/2331680308 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4704010430 con 0x7f471410b260 2024-07-31T14:59:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.723+0000 7fdaf6d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0254b71b0 0x55e02586acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.723+0000 7fdaf6d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0254b71b0 0x55e02586acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51762/0 (socket says 172.21.15.67:51762) 2024-07-31T14:59:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.723+0000 7fdaf5d9b640 1 -- 172.21.15.67:0/1799497173 learned_addr learned my addr 172.21.15.67:0/1799497173 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.723+0000 7fdb03e01740 1 -- 172.21.15.67:0/1799497173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81b81c0 con 0x55e02544b1c0 2024-07-31T14:59:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.723+0000 7fdb03e01740 1 -- 172.21.15.67:0/1799497173 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81b7a30 con 0x55e0254b71b0 2024-07-31T14:59:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.724+0000 7fdb03e01740 1 -- 172.21.15.67:0/1799497173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8109df0 con 0x55e02548e2f0 2024-07-31T14:59:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.724+0000 7fdb03e01740 10 client.0 _ll_register_callbacks cb 0x7fdae8006e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.724+0000 7fdaf5d9b640 1 -- 172.21.15.67:0/1799497173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0254b71b0 msgr2=0x55e02586acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.724+0000 7fdaf5d9b640 1 --2- 172.21.15.67:0/1799497173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0254b71b0 0x55e02586acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.724+0000 7fdaf5d9b640 1 -- 172.21.15.67:0/1799497173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e02544b1c0 msgr2=0x55e02586dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.724+0000 7fdaf5d9b640 1 --2- 172.21.15.67:0/1799497173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e02544b1c0 0x55e02586dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.724+0000 7fdaf5d9b640 1 -- 172.21.15.67:0/1799497173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0258720c0 con 0x55e02548e2f0 2024-07-31T14:59:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.726+0000 7fdaf5d9b640 1 --2- 172.21.15.67:0/1799497173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e02586a780 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdae800ca20 tx=0x7fdae80185c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85b3940a0aac3d94 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.726+0000 7fdaddffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:59:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.726+0000 7fdaddffb640 1 -- 172.21.15.67:0/1799497173 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8029020 con 0x55e02548e2f0 2024-07-31T14:59:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.726+0000 7fdb03e01740 1 -- 172.21.15.67:0/1799497173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaec005420 con 0x55e02548e2f0 2024-07-31T14:59:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.727+0000 7fdaddffb640 1 -- 172.21.15.67:0/1799497173 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7fdae8004450 con 0x55e02548e2f0 2024-07-31T14:59:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.727+0000 7fdaddffb640 1 -- 172.21.15.67:0/1799497173 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fdae8034050 con 0x55e02548e2f0 2024-07-31T14:59:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.727+0000 7fdb03e01740 10 client.27199 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.727+0000 7fdb03e01740 1 -- 172.21.15.67:0/1799497173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e0258df020 con 0x55e02548e2f0 2024-07-31T14:59:41.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.721+0000 7f4712d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a5fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.721+0000 7f4712d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a5fd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51760/0 (socket says 172.21.15.67:51760) 2024-07-31T14:59:41.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.721+0000 7f4712d9d640 1 -- 172.21.15.67:0/3920033226 learned_addr learned my addr 172.21.15.67:0/3920033226 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:41.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.721+0000 7f471259c640 1 --2- 172.21.15.67:0/3920033226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a258350 0x55ff4a5fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.721+0000 7f471259c640 1 --2- 172.21.15.67:0/3920033226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a237670 0x55ff4a5fde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.725+0000 7f4712d9d640 1 -- 172.21.15.67:0/3920033226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a258350 msgr2=0x55ff4a5fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.725+0000 7f4712d9d640 1 --2- 172.21.15.67:0/3920033226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a258350 0x55ff4a5fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.725+0000 7f4712d9d640 1 -- 172.21.15.67:0/3920033226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a237670 msgr2=0x55ff4a5fde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.725+0000 7f4712d9d640 1 --2- 172.21.15.67:0/3920033226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a237670 0x55ff4a5fde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.725+0000 7f4712d9d640 1 -- 172.21.15.67:0/3920033226 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff4a2c3930 con 0x55ff4a22f490 2024-07-31T14:59:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.726+0000 7f4712d9d640 1 --2- 172.21.15.67:0/3920033226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a5fd8f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f470c009d80 tx=0x7f470c00b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6c4fbbaa1c502ab server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.726+0000 7f46f97fa640 1 -- 172.21.15.67:0/3920033226 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c00bd40 con 0x55ff4a22f490 2024-07-31T14:59:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.726+0000 7f46f97fa640 1 -- 172.21.15.67:0/3920033226 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f470c017070 con 0x55ff4a22f490 2024-07-31T14:59:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.726+0000 7f46f97fa640 1 -- 172.21.15.67:0/3920033226 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c0115d0 con 0x55ff4a22f490 2024-07-31T14:59:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.727+0000 7fdaddffb640 1 -- 172.21.15.67:0/1799497173 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae800a1f0 con 0x55e02548e2f0 2024-07-31T14:59:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.728+0000 7fdaddffb640 1 -- 172.21.15.67:0/1799497173 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7fdae8003950 con 0x55e02548e2f0 2024-07-31T14:59:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.728+0000 7fdb03e01740 10 client.27199 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.728+0000 7fdb03e01740 10 client.27199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.728+0000 7fdb03e01740 20 client.27199 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.728+0000 7fdb03e01740 1 --2- 172.21.15.67:0/1799497173 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e0258df8e0 0x55e0258e1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.728+0000 7fdb03e01740 4 client.27199 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:59:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.728+0000 7fdb03e01740 1 -- 172.21.15.67:0/1799497173 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fdaec005420 con 0x55e0258df8e0 2024-07-31T14:59:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.729+0000 7fdaf6d9d640 1 --2- 172.21.15.67:0/1799497173 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e0258df8e0 0x55e0258e1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.730+0000 7fdaf6d9d640 1 --2- 172.21.15.67:0/1799497173 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e0258df8e0 0x55e0258e1d00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47985d56a9cf8e8a server_cookie=c222c9b0eb46f624 in_seq=0 out_seq=0 2024-07-31T14:59:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:41.730+0000 7fdaddffb640 10 client.27199 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.728+0000 7f471f5b2740 1 -- 172.21.15.67:0/3920033226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 msgr2=0x55ff4a5fd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.728+0000 7f471f5b2740 1 --2- 172.21.15.67:0/3920033226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a5fd8f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f470c009d80 tx=0x7f470c00b420 comp rx=0 tx=0).stop 2024-07-31T14:59:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.726+0000 7f46fbfff640 1 -- 172.21.15.67:0/2331680308 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4704010630 con 0x7f471410b260 2024-07-31T14:59:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.729+0000 7f46fbfff640 1 --2- 172.21.15.67:0/2331680308 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46fc0b3e70 0x7f46fc0b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.730+0000 7f46fbfff640 1 -- 172.21.15.67:0/2331680308 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f47040e4660 con 0x7f471410b260 2024-07-31T14:59:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.730+0000 7f471259c640 1 --2- 172.21.15.67:0/2331680308 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46fc0b3e70 0x7f46fc0b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.730+0000 7f471f5b2740 1 -- 172.21.15.67:0/3920033226 shutdown_connections 2024-07-31T14:59:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.730+0000 7f471f5b2740 1 --2- 172.21.15.67:0/3920033226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a237670 0x55ff4a5fde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.730+0000 7f471f5b2740 1 --2- 172.21.15.67:0/3920033226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a5fd8f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.730+0000 7f471f5b2740 1 --2- 172.21.15.67:0/3920033226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a258350 0x55ff4a5fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.730+0000 7f471f5b2740 1 -- 172.21.15.67:0/3920033226 >> 172.21.15.67:0/3920033226 conn(0x55ff4a24c740 msgr2=0x55ff4a5fb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.731+0000 7f471259c640 1 --2- 172.21.15.67:0/2331680308 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46fc0b3e70 0x7f46fc0b6380 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4708005ab0 tx=0x7f4708011a70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.737+0000 7f471f5b2740 1 -- 172.21.15.67:0/3920033226 shutdown_connections 2024-07-31T14:59:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.737+0000 7f471f5b2740 1 -- 172.21.15.67:0/3920033226 wait complete. 2024-07-31T14:59:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.737+0000 7f471f5b2740 1 -- start start 2024-07-31T14:59:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.738+0000 7f471f5b2740 1 Processor -- start 2024-07-31T14:59:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.738+0000 7f471f5b2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a608fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.738+0000 7f471f5b2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a237670 0x55ff4a6094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.738+0000 7f4712d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a608fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.739+0000 7f4712d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a608fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51776/0 (socket says 172.21.15.67:51776) 2024-07-31T14:59:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.739+0000 7f4712d9d640 1 -- 172.21.15.67:0/361831659 learned_addr learned my addr 172.21.15.67:0/361831659 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.739+0000 7f471259c640 1 --2- 172.21.15.67:0/361831659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a237670 0x55ff4a6094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.739+0000 7f471f5b2740 1 --2- 172.21.15.67:0/361831659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a258350 0x55ff4a636da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.739+0000 7f471f5b2740 1 -- 172.21.15.67:0/361831659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ff4a5054f0 con 0x55ff4a237670 2024-07-31T14:59:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.739+0000 7f471f5b2740 1 -- 172.21.15.67:0/361831659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ff4a566fc0 con 0x55ff4a22f490 2024-07-31T14:59:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.739+0000 7f471f5b2740 1 -- 172.21.15.67:0/361831659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ff4a3c6890 con 0x55ff4a258350 2024-07-31T14:59:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.739+0000 7f471f5b2740 10 client.0 _ll_register_callbacks cb 0x55ff4a50f500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.739+0000 7f4712d9d640 1 -- 172.21.15.67:0/361831659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a258350 msgr2=0x55ff4a636da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.739+0000 7f4712d9d640 1 --2- 172.21.15.67:0/361831659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a258350 0x55ff4a636da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.739+0000 7f4712d9d640 1 -- 172.21.15.67:0/361831659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a237670 msgr2=0x55ff4a6094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.739+0000 7f4712d9d640 1 --2- 172.21.15.67:0/361831659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a237670 0x55ff4a6094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.740+0000 7f4712d9d640 1 -- 172.21.15.67:0/361831659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff4a514c50 con 0x55ff4a22f490 2024-07-31T14:59:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.740+0000 7f4712d9d640 1 --2- 172.21.15.67:0/361831659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a608fa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f470c012d90 tx=0x7f470c012c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64ce7c144d2fc96d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.740+0000 7f46f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:59:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.740+0000 7f471f5b2740 1 -- 172.21.15.67:0/361831659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f470c017070 con 0x55ff4a22f490 2024-07-31T14:59:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.740+0000 7f46f6ff5640 1 -- 172.21.15.67:0/361831659 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c012940 con 0x55ff4a22f490 2024-07-31T14:59:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.741+0000 7f46f6ff5640 1 -- 172.21.15.67:0/361831659 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f470c01b3f0 con 0x55ff4a22f490 2024-07-31T14:59:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.741+0000 7f46f6ff5640 1 -- 172.21.15.67:0/361831659 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f470c01b5f0 con 0x55ff4a22f490 2024-07-31T14:59:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.745+0000 7f471f5b2740 10 client.27194 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.745+0000 7f471f5b2740 1 -- 172.21.15.67:0/361831659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=54}) -- 0x55ff4a6399a0 con 0x55ff4a22f490 2024-07-31T14:59:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.746+0000 7f46f6ff5640 1 -- 172.21.15.67:0/361831659 <== mon.2 v2:172.21.15.73:3300/0 4 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f470c011070 con 0x55ff4a22f490 2024-07-31T14:59:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.746+0000 7f471f5b2740 10 client.27194 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.746+0000 7f471f5b2740 10 client.27194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.746+0000 7f471f5b2740 20 client.27194 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.746+0000 7f471f5b2740 1 --2- 172.21.15.67:0/361831659 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ff4a216cd0 0x55ff4a63cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.746+0000 7f471f5b2740 4 client.27194 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:59:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.747+0000 7f471f5b2740 1 -- 172.21.15.67:0/361831659 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f470c017070 con 0x55ff4a216cd0 2024-07-31T14:59:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.747+0000 7f471259c640 1 --2- 172.21.15.67:0/361831659 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ff4a216cd0 0x55ff4a63cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.747+0000 7f471259c640 1 --2- 172.21.15.67:0/361831659 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ff4a216cd0 0x55ff4a63cd00 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=539d73153d68a115 server_cookie=52b3e856f7e70994 in_seq=0 out_seq=0 2024-07-31T14:59:41.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.747+0000 7f46f6ff5640 10 client.27194 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:41.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.749+0000 7f46f6ff5640 1 -- 172.21.15.67:0/361831659 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f470c017070 con 0x55ff4a216cd0 2024-07-31T14:59:41.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.749+0000 7f46f6ff5640 10 client.27194 handle_command_reply: tid=0 2024-07-31T14:59:41.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.750+0000 7f471f5b2740 1 client.27194 shutdown 2024-07-31T14:59:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.752+0000 7f471f5b2740 1 -- 172.21.15.67:0/361831659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 msgr2=0x55ff4a608fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.752+0000 7f471f5b2740 1 --2- 172.21.15.67:0/361831659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a608fa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f470c012d90 tx=0x7f470c012c20 comp rx=0 tx=0).stop 2024-07-31T14:59:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.753+0000 7f471f5b2740 1 -- 172.21.15.67:0/361831659 shutdown_connections 2024-07-31T14:59:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.753+0000 7f471f5b2740 1 --2- 172.21.15.67:0/361831659 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ff4a216cd0 0x55ff4a63cd00 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.753+0000 7f46f6ff5640 0 client.27194 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:41.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.753+0000 7f471f5b2740 1 --2- 172.21.15.67:0/361831659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a258350 0x55ff4a636da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.753+0000 7f471f5b2740 1 --2- 172.21.15.67:0/361831659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a237670 0x55ff4a6094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.753+0000 7f471f5b2740 1 --2- 172.21.15.67:0/361831659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a608fa0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.753+0000 7f471f5b2740 1 -- 172.21.15.67:0/361831659 >> 172.21.15.67:0/361831659 conn(0x55ff4a24c740 msgr2=0x55ff4a600540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:41.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.753+0000 7f471f5b2740 1 -- 172.21.15.67:0/361831659 shutdown_connections 2024-07-31T14:59:41.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.753+0000 7f471f5b2740 1 -- 172.21.15.67:0/361831659 wait complete. 2024-07-31T14:59:41.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.754+0000 7f471f5b2740 20 client.27194 trim_cache size 0 max 16384 2024-07-31T14:59:41.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.795+0000 7f471f5b2740 1 Processor -- start 2024-07-31T14:59:41.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.795+0000 7f471f5b2740 1 -- start start 2024-07-31T14:59:41.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.795+0000 7f471f5b2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a216cd0 0x55ff4a5fb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.796+0000 7f471f5b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a22f490 0x55ff4a636b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.796+0000 7f4712d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a216cd0 0x55ff4a5fb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.796+0000 7f4712d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a216cd0 0x55ff4a5fb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51788/0 (socket says 172.21.15.67:51788) 2024-07-31T14:59:41.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.796+0000 7f4712d9d640 1 -- 172.21.15.67:0/3175291864 learned_addr learned my addr 172.21.15.67:0/3175291864 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:41.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.796+0000 7f471f5b2740 1 --2- 172.21.15.67:0/3175291864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a237670 0x55ff4a6370d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.796+0000 7f471259c640 1 --2- 172.21.15.67:0/3175291864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a22f490 0x55ff4a636b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.796+0000 7f471f5b2740 1 -- 172.21.15.67:0/3175291864 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f470000fef0 con 0x55ff4a237670 2024-07-31T14:59:41.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.796+0000 7f471f5b2740 1 -- 172.21.15.67:0/3175291864 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4700011b80 con 0x55ff4a216cd0 2024-07-31T14:59:41.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.796+0000 7f471f5b2740 1 -- 172.21.15.67:0/3175291864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47000105c0 con 0x55ff4a22f490 2024-07-31T14:59:41.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.797+0000 7f471259c640 1 -- 172.21.15.67:0/3175291864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a216cd0 msgr2=0x55ff4a5fb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.797+0000 7f471259c640 1 --2- 172.21.15.67:0/3175291864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a216cd0 0x55ff4a5fb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.797+0000 7f471259c640 1 -- 172.21.15.67:0/3175291864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a237670 msgr2=0x55ff4a6370d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.797+0000 7f471259c640 1 --2- 172.21.15.67:0/3175291864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a237670 0x55ff4a6370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.797+0000 7f471259c640 1 -- 172.21.15.67:0/3175291864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff4a600210 con 0x55ff4a22f490 2024-07-31T14:59:41.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.798+0000 7f471259c640 1 --2- 172.21.15.67:0/3175291864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a22f490 0x55ff4a636b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4708013120 tx=0x7f470800b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d10cec558aaf7823 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:41.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.801+0000 7f46f8ff9640 1 -- 172.21.15.67:0/3175291864 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470803e040 con 0x55ff4a22f490 2024-07-31T14:59:41.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.801+0000 7f46f8ff9640 1 -- 172.21.15.67:0/3175291864 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f470800f680 con 0x55ff4a22f490 2024-07-31T14:59:41.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.802+0000 7f471f5b2740 1 -- 172.21.15.67:0/3175291864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a22f490 msgr2=0x55ff4a636b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:41.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.802+0000 7f471f5b2740 1 --2- 172.21.15.67:0/3175291864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a22f490 0x55ff4a636b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4708013120 tx=0x7f470800b000 comp rx=0 tx=0).stop 2024-07-31T14:59:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.803+0000 7f471f5b2740 1 -- 172.21.15.67:0/3175291864 shutdown_connections 2024-07-31T14:59:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.803+0000 7f471f5b2740 1 --2- 172.21.15.67:0/3175291864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a237670 0x55ff4a6370d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.803+0000 7f471f5b2740 1 --2- 172.21.15.67:0/3175291864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a22f490 0x55ff4a636b90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.803+0000 7f471f5b2740 1 --2- 172.21.15.67:0/3175291864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a216cd0 0x55ff4a5fb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.803+0000 7f471f5b2740 1 -- 172.21.15.67:0/3175291864 >> 172.21.15.67:0/3175291864 conn(0x55ff4a24c740 msgr2=0x55ff4a5fabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.804+0000 7f471f5b2740 1 -- 172.21.15.67:0/3175291864 shutdown_connections 2024-07-31T14:59:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.805+0000 7f471f5b2740 1 -- 172.21.15.67:0/3175291864 wait complete. 2024-07-31T14:59:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.805+0000 7f471f5b2740 1 -- start start 2024-07-31T14:59:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.805+0000 7f471f5b2740 1 Processor -- start 2024-07-31T14:59:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.806+0000 7f471f5b2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a216cd0 0x55ff4a62f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.806+0000 7f471f5b2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a62f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.806+0000 7f471f5b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a237670 0x55ff4a6a4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.806+0000 7f471f5b2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f47000114b0 con 0x55ff4a216cd0 2024-07-31T14:59:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.806+0000 7f471f5b2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4700010d20 con 0x55ff4a22f490 2024-07-31T14:59:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.806+0000 7f471f5b2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f470000def0 con 0x55ff4a237670 2024-07-31T14:59:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.806+0000 7f471f5b2740 10 client.0 _ll_register_callbacks cb 0x7f470c007170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.806+0000 7f471259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a62f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.806+0000 7f471259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a62f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51792/0 (socket says 172.21.15.67:51792) 2024-07-31T14:59:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.806+0000 7f471259c640 1 -- 172.21.15.67:0/1355591930 learned_addr learned my addr 172.21.15.67:0/1355591930 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.808+0000 7f471359e640 1 --2- 172.21.15.67:0/1355591930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a237670 0x55ff4a6a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.810+0000 7f4712d9d640 1 --2- 172.21.15.67:0/1355591930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a216cd0 0x55ff4a62f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.810+0000 7f471359e640 1 -- 172.21.15.67:0/1355591930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 msgr2=0x55ff4a62f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.810+0000 7f471359e640 1 --2- 172.21.15.67:0/1355591930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a62f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.810+0000 7f471359e640 1 -- 172.21.15.67:0/1355591930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a216cd0 msgr2=0x55ff4a62f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.810+0000 7f471359e640 1 --2- 172.21.15.67:0/1355591930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a216cd0 0x55ff4a62f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.810+0000 7f471359e640 1 -- 172.21.15.67:0/1355591930 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff4a5f66d0 con 0x55ff4a237670 2024-07-31T14:59:41.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.814+0000 7f471259c640 1 --2- 172.21.15.67:0/1355591930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a62f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T14:59:41.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.815+0000 7f471359e640 1 --2- 172.21.15.67:0/1355591930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a237670 0x55ff4a6a4480 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f47040e1990 tx=0x7f47040aaa20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9aabe86aadd45ba server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:41.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.815+0000 7f46f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:59:41.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.816+0000 7f46f6ff5640 1 -- 172.21.15.67:0/1355591930 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47040af210 con 0x55ff4a237670 2024-07-31T14:59:41.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.816+0000 7f46f6ff5640 1 -- 172.21.15.67:0/1355591930 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f47040e4400 con 0x55ff4a237670 2024-07-31T14:59:41.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.818+0000 7f471f5b2740 1 -- 172.21.15.67:0/1355591930 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f470800f680 con 0x55ff4a237670 2024-07-31T14:59:41.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.819+0000 7f46f6ff5640 1 -- 172.21.15.67:0/1355591930 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f47040af450 con 0x55ff4a237670 2024-07-31T14:59:41.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.819+0000 7f471f5b2740 10 client.27205 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.819+0000 7f471f5b2740 1 -- 172.21.15.67:0/1355591930 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ff4a6c7bf0 con 0x55ff4a237670 2024-07-31T14:59:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.820+0000 7f46f6ff5640 1 -- 172.21.15.67:0/1355591930 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f470800f680 con 0x55ff4a237670 2024-07-31T14:59:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.820+0000 7f46f6ff5640 1 -- 172.21.15.67:0/1355591930 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f47040a69a0 con 0x55ff4a237670 2024-07-31T14:59:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.820+0000 7f471f5b2740 10 client.27205 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.820+0000 7f471f5b2740 10 client.27205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.820+0000 7f471f5b2740 20 client.27205 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.820+0000 7f471f5b2740 1 --2- 172.21.15.67:0/1355591930 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ff4a258350 0x55ff4a6ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.820+0000 7f471f5b2740 4 client.27205 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T14:59:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.820+0000 7f471f5b2740 1 -- 172.21.15.67:0/1355591930 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ff4a5759d0 con 0x55ff4a258350 2024-07-31T14:59:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.822+0000 7f4712d9d640 1 --2- 172.21.15.67:0/1355591930 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ff4a258350 0x55ff4a6ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:41.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.830+0000 7f4712d9d640 1 --2- 172.21.15.67:0/1355591930 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ff4a258350 0x55ff4a6ca4e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5acb85786422a210 server_cookie=638c4f0e9304f40c in_seq=0 out_seq=0 2024-07-31T14:59:41.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.831+0000 7f46f6ff5640 10 client.27205 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:41.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.884+0000 7f46f6ff5640 1 -- 172.21.15.67:0/1355591930 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ff4a5759d0 con 0x55ff4a258350 2024-07-31T14:59:41.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.884+0000 7f46f6ff5640 10 client.27205 handle_command_reply: tid=0 2024-07-31T14:59:41.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.884+0000 7f471f5b2740 1 client.27205 shutdown 2024-07-31T14:59:41.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.885+0000 7f471f5b2740 1 -- 172.21.15.67:0/1355591930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a237670 msgr2=0x55ff4a6a4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:41.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.885+0000 7f471f5b2740 1 --2- 172.21.15.67:0/1355591930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a237670 0x55ff4a6a4480 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f47040e1990 tx=0x7f47040aaa20 comp rx=0 tx=0).stop 2024-07-31T14:59:41.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.885+0000 7f471f5b2740 1 -- 172.21.15.67:0/1355591930 shutdown_connections 2024-07-31T14:59:41.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.885+0000 7f471f5b2740 1 --2- 172.21.15.67:0/1355591930 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ff4a258350 0x55ff4a6ca4e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.885+0000 7f471f5b2740 1 --2- 172.21.15.67:0/1355591930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff4a237670 0x55ff4a6a4480 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.885+0000 7f471f5b2740 1 --2- 172.21.15.67:0/1355591930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff4a22f490 0x55ff4a62f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.885+0000 7f471f5b2740 1 --2- 172.21.15.67:0/1355591930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff4a216cd0 0x55ff4a62f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.885+0000 7f471f5b2740 1 -- 172.21.15.67:0/1355591930 >> 172.21.15.67:0/1355591930 conn(0x55ff4a24c740 msgr2=0x55ff4a5fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:41.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.885+0000 7f46f6ff5640 0 client.27205 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:41.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.885+0000 7f471f5b2740 1 -- 172.21.15.67:0/1355591930 shutdown_connections 2024-07-31T14:59:41.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.885+0000 7f471f5b2740 1 -- 172.21.15.67:0/1355591930 wait complete. 2024-07-31T14:59:41.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.885+0000 7f471f5b2740 20 client.27205 trim_cache size 0 max 16384 2024-07-31T14:59:41.889 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:59:41.889 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T14:59:41.890 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "c791f779-17a0-47ec-8943-a777d1eb6ea8", 2024-07-31T14:59:41.890 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T14:59:41.890 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:59:41.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.886+0000 7f4719061640 1 -- 172.21.15.67:0/2331680308 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46fc0b3e70 msgr2=0x7f46fc0b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:41.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.886+0000 7f4719061640 1 --2- 172.21.15.67:0/2331680308 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46fc0b3e70 0x7f46fc0b6380 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4708005ab0 tx=0x7f4708011a70 comp rx=0 tx=0).stop 2024-07-31T14:59:41.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.886+0000 7f4719061640 1 -- 172.21.15.67:0/2331680308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f471410b260 msgr2=0x7f47141b6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:41.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.886+0000 7f4719061640 1 --2- 172.21.15.67:0/2331680308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f471410b260 0x7f47141b6db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f470400e970 tx=0x7f470400ed70 comp rx=0 tx=0).stop 2024-07-31T14:59:41.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.886+0000 7f4719061640 1 -- 172.21.15.67:0/2331680308 shutdown_connections 2024-07-31T14:59:41.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.886+0000 7f4719061640 1 --2- 172.21.15.67:0/2331680308 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46fc0b3e70 0x7f46fc0b6380 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.886+0000 7f4719061640 1 --2- 172.21.15.67:0/2331680308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f471410b260 0x7f47141b6db0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.886+0000 7f4719061640 1 --2- 172.21.15.67:0/2331680308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f471410ae50 0x7f47141b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.886+0000 7f4719061640 1 --2- 172.21.15.67:0/2331680308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4714109df0 0x7f47141b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:41.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.887+0000 7f4719061640 1 -- 172.21.15.67:0/2331680308 >> 172.21.15.67:0/2331680308 conn(0x7f4714070d20 msgr2=0x7f471410c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:41.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.887+0000 7f4719061640 1 -- 172.21.15.67:0/2331680308 shutdown_connections 2024-07-31T14:59:41.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:41.887+0000 7f4719061640 1 -- 172.21.15.67:0/2331680308 wait complete. 2024-07-31T14:59:42.475 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:42 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T14:59:42.475 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:42.475 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:42.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:42 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T14:59:42.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:42.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:42 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T14:59:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.888+0000 7fdaddffb640 1 -- 172.21.15.67:0/1799497173 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fdaec005420 con 0x55e0258df8e0 2024-07-31T14:59:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.888+0000 7fdaddffb640 10 client.27199 handle_command_reply: tid=0 2024-07-31T14:59:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.888+0000 7fdb03e01740 1 client.27199 shutdown 2024-07-31T14:59:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.889+0000 7fdb03e01740 1 -- 172.21.15.67:0/1799497173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 msgr2=0x55e02586a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.889+0000 7fdb03e01740 1 --2- 172.21.15.67:0/1799497173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e02586a780 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdae800ca20 tx=0x7fdae80185c0 comp rx=0 tx=0).stop 2024-07-31T14:59:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.889+0000 7fdb03e01740 1 -- 172.21.15.67:0/1799497173 shutdown_connections 2024-07-31T14:59:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.889+0000 7fdb03e01740 1 --2- 172.21.15.67:0/1799497173 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e0258df8e0 0x55e0258e1d00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.889+0000 7fdb03e01740 1 --2- 172.21.15.67:0/1799497173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0254b71b0 0x55e02586acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.889+0000 7fdb03e01740 1 --2- 172.21.15.67:0/1799497173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e02548e2f0 0x55e02586a780 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.889+0000 7fdb03e01740 1 --2- 172.21.15.67:0/1799497173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e02544b1c0 0x55e02586dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.889+0000 7fdb03e01740 1 -- 172.21.15.67:0/1799497173 >> 172.21.15.67:0/1799497173 conn(0x55e0254ab300 msgr2=0x55e02585ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.890+0000 7fdaddffb640 0 client.27199 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:42.897 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 14:59:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T14:59:42.473+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000162a [...7,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch/ auth{1=1,2=1,3=1,4=1} v2970 ap=2 RANDEPHEMERALPIN f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22) n(v19 rc2024-07-31T14:54:53.561226+0000 b106958628 17855=16920+935) old_inodes=3 (inest sync->mix g=1,2,3,4) | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=0 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7c26ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000162a:[<0x10000000202/arch v378>,<0x10000000201/linux-6.5.11 v96>,<0x10000000006/t v68>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v57>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v60>,<0x10000000001/sv_1 v84>,<0x10000000000/qa v70>,<0x1/volumes v70>]//[]","memoryvalue":"(2)0x1000000162a:[<0x10000000202/arch v2970>,<0x10000000201/linux-6.5.11 v798>,<0x10000000006/t v565>,<0x10000000005/tmp v532>,<0x100000001fb/client.0 v517>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v497>,<0x10000000001/sv_1 v505>,<0x10000000000/qa v501>,<0x1/volumes v483>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:54:49.652308+0000 23=2+21)","ondisk_value.rstat":"n(v0 rc2024-07-31T14:54:53.561226+0000 b106958628 17855=16920+935)","memory_value.dirstat":"f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22)","memory_value.rstat":"n(v19 rc2024-07-31T14:54:53.561226+0000 b106958628 17855=16920+935)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T14:59:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.890+0000 7fdb03e01740 1 -- 172.21.15.67:0/1799497173 shutdown_connections 2024-07-31T14:59:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.890+0000 7fdb03e01740 1 -- 172.21.15.67:0/1799497173 wait complete. 2024-07-31T14:59:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.890+0000 7fdb03e01740 20 client.27199 trim_cache size 0 max 16384 2024-07-31T14:59:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T14:59:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T14:59:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T14:59:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T14:59:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.891+0000 7fdafcf0e640 1 -- 172.21.15.67:0/2692003606 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdae00b3d20 msgr2=0x7fdae00b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.891+0000 7fdafcf0e640 1 --2- 172.21.15.67:0/2692003606 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdae00b3d20 0x7fdae00b6230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00cbd30 tx=0x7fdaf010b530 comp rx=0 tx=0).stop 2024-07-31T14:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.891+0000 7fdafcf0e640 1 -- 172.21.15.67:0/2692003606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf810b970 msgr2=0x7fdaf811c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.891+0000 7fdafcf0e640 1 --2- 172.21.15.67:0/2692003606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf810b970 0x7fdaf811c3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0005e30 tx=0x7fdaf0031720 comp rx=0 tx=0).stop 2024-07-31T14:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.891+0000 7fdafcf0e640 1 -- 172.21.15.67:0/2692003606 shutdown_connections 2024-07-31T14:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.891+0000 7fdafcf0e640 1 --2- 172.21.15.67:0/2692003606 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdae00b3d20 0x7fdae00b6230 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.891+0000 7fdafcf0e640 1 --2- 172.21.15.67:0/2692003606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaf8114ea0 0x7fdaf8127440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.891+0000 7fdafcf0e640 1 --2- 172.21.15.67:0/2692003606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf810c2c0 0x7fdaf811c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.891+0000 7fdafcf0e640 1 --2- 172.21.15.67:0/2692003606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf810b970 0x7fdaf811c3c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.891+0000 7fdafcf0e640 1 -- 172.21.15.67:0/2692003606 >> 172.21.15.67:0/2692003606 conn(0x7fdaf80707c0 msgr2=0x7fdaf811aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.891+0000 7fdafcf0e640 1 -- 172.21.15.67:0/2692003606 shutdown_connections 2024-07-31T14:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:42.891+0000 7fdafcf0e640 1 -- 172.21.15.67:0/2692003606 wait complete. 2024-07-31T14:59:43.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T14:59:43.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:43 smithi077 ceph-mon[37327]: pgmap v663: 129 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 93 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 348 KiB/s wr, 61 op/s 2024-07-31T14:59:43.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:43 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T14:59:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:43.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:43 smithi067 ceph-mon[32750]: pgmap v663: 129 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 93 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 348 KiB/s wr, 61 op/s 2024-07-31T14:59:43.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:43.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:43 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T14:59:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:43.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:43 smithi073 ceph-mon[36714]: pgmap v663: 129 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 93 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 348 KiB/s wr, 61 op/s 2024-07-31T14:59:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:43 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T14:59:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:43.729 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T14:59:44.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:44 smithi077 ceph-mon[37327]: pgmap v664: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 910 KiB/s wr, 147 op/s 2024-07-31T14:59:44.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:44.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:44.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:44 smithi067 ceph-mon[32750]: pgmap v664: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 910 KiB/s wr, 147 op/s 2024-07-31T14:59:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:44 smithi073 ceph-mon[36714]: pgmap v664: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 910 KiB/s wr, 147 op/s 2024-07-31T14:59:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:46.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.812+0000 7f1bc11d7640 1 -- 172.21.15.67:0/2819497899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bbc10a5b0 msgr2=0x7f1bbc10a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.812+0000 7f1bc11d7640 1 --2- 172.21.15.67:0/2819497899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bbc10a5b0 0x7f1bbc10a9c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009900 tx=0x7f1bac02a440 comp rx=0 tx=0).stop 2024-07-31T14:59:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.813+0000 7f1bc11d7640 1 -- 172.21.15.67:0/2819497899 shutdown_connections 2024-07-31T14:59:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.813+0000 7f1bc11d7640 1 --2- 172.21.15.67:0/2819497899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bbc11c5d0 0x7f1bbc11e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.813+0000 7f1bc11d7640 1 --2- 172.21.15.67:0/2819497899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bbc10a5b0 0x7f1bbc10a9c0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.813+0000 7f1bc11d7640 1 --2- 172.21.15.67:0/2819497899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc111770 0x7f1bbc10a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.813+0000 7f1bc11d7640 1 -- 172.21.15.67:0/2819497899 >> 172.21.15.67:0/2819497899 conn(0x7f1bbc06d380 msgr2=0x7f1bbc06d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:46.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.815+0000 7f1bc11d7640 1 -- 172.21.15.67:0/2819497899 shutdown_connections 2024-07-31T14:59:46.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.815+0000 7f1bc11d7640 1 -- 172.21.15.67:0/2819497899 wait complete. 2024-07-31T14:59:46.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.816+0000 7f1bc11d7640 1 Processor -- start 2024-07-31T14:59:46.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.816+0000 7f1bc11d7640 1 -- start start 2024-07-31T14:59:46.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.816+0000 7f1bc11d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc10a5b0 0x7f1bbc11be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.816+0000 7f1bc11d7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bbc111770 0x7f1bbc115150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.816+0000 7f1bc11d7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bbc11c5d0 0x7f1bbc115710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.816+0000 7f1bc11d7640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1221c0 con 0x7f1bbc11c5d0 2024-07-31T14:59:46.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.817+0000 7f1bc11d7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc122040 con 0x7f1bbc111770 2024-07-31T14:59:46.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.817+0000 7f1bc11d7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc122340 con 0x7f1bbc10a5b0 2024-07-31T14:59:46.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.818+0000 7f1bbb59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bbc11c5d0 0x7f1bbc115710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:46.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.818+0000 7f1bbb59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bbc11c5d0 0x7f1bbc115710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36020/0 (socket says 172.21.15.67:36020) 2024-07-31T14:59:46.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.818+0000 7f1bbb59e640 1 -- 172.21.15.67:0/146102817 learned_addr learned my addr 172.21.15.67:0/146102817 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.818+0000 7f1bbad9d640 1 --2- 172.21.15.67:0/146102817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc10a5b0 0x7f1bbc11be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.819+0000 7f1bba59c640 1 --2- 172.21.15.67:0/146102817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bbc111770 0x7f1bbc115150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.819+0000 7f1bbb59e640 1 -- 172.21.15.67:0/146102817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc10a5b0 msgr2=0x7f1bbc11be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.819+0000 7f1bbb59e640 1 --2- 172.21.15.67:0/146102817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc10a5b0 0x7f1bbc11be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.819+0000 7f1bbb59e640 1 -- 172.21.15.67:0/146102817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bbc111770 msgr2=0x7f1bbc115150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.819+0000 7f1bbb59e640 1 --2- 172.21.15.67:0/146102817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bbc111770 0x7f1bbc115150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.819+0000 7f1bbb59e640 1 -- 172.21.15.67:0/146102817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc115f30 con 0x7f1bbc11c5d0 2024-07-31T14:59:46.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.825+0000 7f1bbb59e640 1 --2- 172.21.15.67:0/146102817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bbc11c5d0 0x7f1bbc115710 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400cad0 tx=0x7f1bb400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.827+0000 7f1ba3fff640 1 -- 172.21.15.67:0/146102817 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb400ae20 con 0x7f1bbc11c5d0 2024-07-31T14:59:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.827+0000 7f1ba3fff640 1 -- 172.21.15.67:0/146102817 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bb4005020 con 0x7f1bbc11c5d0 2024-07-31T14:59:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.827+0000 7f1bc11d7640 1 -- 172.21.15.67:0/146102817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bbc11abb0 con 0x7f1bbc11c5d0 2024-07-31T14:59:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.827+0000 7f1bc11d7640 1 -- 172.21.15.67:0/146102817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bbc11b0c0 con 0x7f1bbc11c5d0 2024-07-31T14:59:46.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.828+0000 7f1ba3fff640 1 -- 172.21.15.67:0/146102817 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb400f6b0 con 0x7f1bbc11c5d0 2024-07-31T14:59:46.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.833+0000 7f1bc7728740 1 Processor -- start 2024-07-31T14:59:46.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.835+0000 7f1ba3fff640 1 -- 172.21.15.67:0/146102817 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1bb4004a00 con 0x7f1bbc11c5d0 2024-07-31T14:59:46.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.835+0000 7f1bc7728740 1 -- start start 2024-07-31T14:59:46.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.835+0000 7f1bc7728740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26ab070 0x559de2a506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.835+0000 7f1bc7728740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de26821b0 0x559de2a50c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.835+0000 7f1bc7728740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de268a390 0x559de2a51170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.835+0000 7f1bc7728740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559de29c6b90 con 0x559de26ab070 2024-07-31T14:59:46.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.835+0000 7f1bc7728740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559de29b6090 con 0x559de268a390 2024-07-31T14:59:46.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.835+0000 7f1bba59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26ab070 0x559de2a506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:46.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.836+0000 7f1bc7728740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559de29ba0d0 con 0x559de26821b0 2024-07-31T14:59:46.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.836+0000 7f1bba59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26ab070 0x559de2a506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36024/0 (socket says 172.21.15.67:36024) 2024-07-31T14:59:46.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.836+0000 7f1bba59c640 1 -- 172.21.15.67:0/1858839911 learned_addr learned my addr 172.21.15.67:0/1858839911 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:46.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.836+0000 7f1bbad9d640 1 --2- 172.21.15.67:0/1858839911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de26821b0 0x559de2a50c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:46.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.836+0000 7f1bba59c640 1 --2- 172.21.15.67:0/1858839911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de268a390 0x559de2a51170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:46.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.840+0000 7f1bba59c640 1 -- 172.21.15.67:0/1858839911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de26821b0 msgr2=0x559de2a50c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:46.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.840+0000 7f1bba59c640 1 --2- 172.21.15.67:0/1858839911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de26821b0 0x559de2a50c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.841+0000 7f1bba59c640 1 -- 172.21.15.67:0/1858839911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de268a390 msgr2=0x559de2a51170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:46.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.841+0000 7f1bba59c640 1 --2- 172.21.15.67:0/1858839911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de268a390 0x559de2a51170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.841+0000 7f1bba59c640 1 -- 172.21.15.67:0/1858839911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559de295cb70 con 0x559de26ab070 2024-07-31T14:59:46.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.842+0000 7f1bba59c640 1 --2- 172.21.15.67:0/1858839911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26ab070 0x559de2a506f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1bac000c00 tx=0x7f1bac009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b355e492cace10a3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:46.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.842+0000 7f1ba3fff640 1 --2- 172.21.15.67:0/146102817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ba40b3d70 0x7f1ba40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.843+0000 7f1ba3fff640 1 -- 172.21.15.67:0/146102817 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f1bb40e4280 con 0x7f1bbc11c5d0 2024-07-31T14:59:46.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.843+0000 7f1bbad9d640 1 --2- 172.21.15.67:0/146102817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ba40b3d70 0x7f1ba40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.844+0000 7f1ba17fa640 1 -- 172.21.15.67:0/1858839911 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac044020 con 0x559de26ab070 2024-07-31T14:59:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.844+0000 7f1bbad9d640 1 --2- 172.21.15.67:0/146102817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ba40b3d70 0x7f1ba40b6280 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x559de2a4f370 tx=0x7f1bb00090e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.844+0000 7f1ba17fa640 1 -- 172.21.15.67:0/1858839911 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bac003e20 con 0x559de26ab070 2024-07-31T14:59:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.845+0000 7f1bc7728740 1 -- 172.21.15.67:0/1858839911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26ab070 msgr2=0x559de2a506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.845+0000 7f1bc7728740 1 --2- 172.21.15.67:0/1858839911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26ab070 0x559de2a506f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1bac000c00 tx=0x7f1bac009460 comp rx=0 tx=0).stop 2024-07-31T14:59:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.846+0000 7f1bc7728740 1 -- 172.21.15.67:0/1858839911 shutdown_connections 2024-07-31T14:59:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.846+0000 7f1bc7728740 1 --2- 172.21.15.67:0/1858839911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de268a390 0x559de2a51170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.846+0000 7f1bc7728740 1 --2- 172.21.15.67:0/1858839911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de26821b0 0x559de2a50c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.846+0000 7f1bc7728740 1 --2- 172.21.15.67:0/1858839911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26ab070 0x559de2a506f0 secure :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1bac000c00 tx=0x7f1bac009460 comp rx=0 tx=0).stop 2024-07-31T14:59:46.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.846+0000 7f1bc7728740 1 -- 172.21.15.67:0/1858839911 >> 172.21.15.67:0/1858839911 conn(0x559de269f460 msgr2=0x559de2a4e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:46.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.847+0000 7f1bc7728740 1 -- 172.21.15.67:0/1858839911 shutdown_connections 2024-07-31T14:59:46.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.847+0000 7f1bc7728740 1 -- 172.21.15.67:0/1858839911 wait complete. 2024-07-31T14:59:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.847+0000 7f1bc7728740 1 -- start start 2024-07-31T14:59:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.848+0000 7f1bc7728740 1 Processor -- start 2024-07-31T14:59:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.848+0000 7f1bc7728740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de26821b0 0x559de2a5c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.848+0000 7f1bc7728740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de268a390 0x559de2a5c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.849+0000 7f1bc7728740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26699f0 0x559de2a68290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.849+0000 7f1bc7728740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559de2819700 con 0x559de26699f0 2024-07-31T14:59:46.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.849+0000 7f1bbb59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26699f0 0x559de2a68290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:46.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.849+0000 7f1bc7728740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559de29589c0 con 0x559de26821b0 2024-07-31T14:59:46.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.849+0000 7f1bc7728740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559de2a5eec0 con 0x559de268a390 2024-07-31T14:59:46.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.849+0000 7f1bbb59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26699f0 0x559de2a68290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36032/0 (socket says 172.21.15.67:36032) 2024-07-31T14:59:46.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.849+0000 7f1bbb59e640 1 -- 172.21.15.67:0/3526566018 learned_addr learned my addr 172.21.15.67:0/3526566018 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:46.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.849+0000 7f1bc7728740 10 client.0 _ll_register_callbacks cb 0x559de29734d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:46.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.851+0000 7f1bbb59e640 1 -- 172.21.15.67:0/3526566018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de268a390 msgr2=0x559de2a5c810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:59:46.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.851+0000 7f1bbb59e640 1 --2- 172.21.15.67:0/3526566018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de268a390 0x559de2a5c810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.851+0000 7f1bbb59e640 1 -- 172.21.15.67:0/3526566018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de26821b0 msgr2=0x559de2a5c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:46.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.851+0000 7f1bbb59e640 1 --2- 172.21.15.67:0/3526566018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de26821b0 0x559de2a5c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.851+0000 7f1bbb59e640 1 -- 172.21.15.67:0/3526566018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559de2716650 con 0x559de26699f0 2024-07-31T14:59:46.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.851+0000 7f1bbb59e640 1 --2- 172.21.15.67:0/3526566018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26699f0 0x559de2a68290 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40aaa60 tx=0x7f1bb40ab290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c720dd9060e4185e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:46.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.851+0000 7f1b9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:59:46.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.852+0000 7f1b9eff5640 1 -- 172.21.15.67:0/3526566018 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb40c1020 con 0x559de26699f0 2024-07-31T14:59:46.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.852+0000 7f1b9eff5640 1 -- 172.21.15.67:0/3526566018 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f1bb40a70c0 con 0x559de26699f0 2024-07-31T14:59:46.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.852+0000 7f1bc7728740 1 -- 172.21.15.67:0/3526566018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bac003e20 con 0x559de26699f0 2024-07-31T14:59:46.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.853+0000 7f1b9eff5640 1 -- 172.21.15.67:0/3526566018 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f1bb401d040 con 0x559de26699f0 2024-07-31T14:59:46.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.853+0000 7f1bc7728740 10 client.17646 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:46.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.853+0000 7f1bc7728740 1 -- 172.21.15.67:0/3526566018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559de2a6afe0 con 0x559de26699f0 2024-07-31T14:59:46.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.854+0000 7f1b9eff5640 1 -- 172.21.15.67:0/3526566018 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bac003e20 con 0x559de26699f0 2024-07-31T14:59:46.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.854+0000 7f1b9eff5640 1 -- 172.21.15.67:0/3526566018 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f1bb40a65d0 con 0x559de26699f0 2024-07-31T14:59:46.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.855+0000 7f1bc7728740 10 client.17646 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:46.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.855+0000 7f1bc7728740 10 client.17646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:46.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.855+0000 7f1bc7728740 20 client.17646 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:46.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.855+0000 7f1bc7728740 1 --2- 172.21.15.67:0/3526566018 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559de263eda0 0x559de2a6eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.855+0000 7f1bc7728740 4 client.17646 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:59:46.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.855+0000 7f1bc7728740 1 -- 172.21.15.67:0/3526566018 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559de29c8ae0 con 0x559de263eda0 2024-07-31T14:59:46.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.856+0000 7f1bbad9d640 1 --2- 172.21.15.67:0/3526566018 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559de263eda0 0x559de2a6eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:46.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.857+0000 7f1bbad9d640 1 --2- 172.21.15.67:0/3526566018 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559de263eda0 0x559de2a6eb10 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c83a2443a17ee9 server_cookie=6d068f74df500ee6 in_seq=0 out_seq=0 2024-07-31T14:59:46.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.859+0000 7f1b9eff5640 10 client.17646 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:46.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.862+0000 7f1b9eff5640 1 -- 172.21.15.67:0/3526566018 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559de29c8ae0 con 0x559de263eda0 2024-07-31T14:59:46.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.862+0000 7f1b9eff5640 10 client.17646 handle_command_reply: tid=0 2024-07-31T14:59:46.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.862+0000 7f1bc7728740 1 client.17646 shutdown 2024-07-31T14:59:46.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.863+0000 7f1bc7728740 1 -- 172.21.15.67:0/3526566018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26699f0 msgr2=0x559de2a68290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:46.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.863+0000 7f1bc7728740 1 --2- 172.21.15.67:0/3526566018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26699f0 0x559de2a68290 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40aaa60 tx=0x7f1bb40ab290 comp rx=0 tx=0).stop 2024-07-31T14:59:46.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.865+0000 7f1bc7728740 1 -- 172.21.15.67:0/3526566018 shutdown_connections 2024-07-31T14:59:46.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.865+0000 7f1bc7728740 1 --2- 172.21.15.67:0/3526566018 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559de263eda0 0x559de2a6eb10 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.865+0000 7f1bc7728740 1 --2- 172.21.15.67:0/3526566018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26699f0 0x559de2a68290 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.865+0000 7f1bc7728740 1 --2- 172.21.15.67:0/3526566018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de268a390 0x559de2a5c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.865+0000 7f1bc7728740 1 --2- 172.21.15.67:0/3526566018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de26821b0 0x559de2a5c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.865+0000 7f1b9eff5640 0 client.17646 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:46.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.865+0000 7f1bc7728740 1 -- 172.21.15.67:0/3526566018 >> 172.21.15.67:0/3526566018 conn(0x559de269f460 msgr2=0x559de2a538d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:46.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.866+0000 7f1bc7728740 1 -- 172.21.15.67:0/3526566018 shutdown_connections 2024-07-31T14:59:46.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.866+0000 7f1bc7728740 1 -- 172.21.15.67:0/3526566018 wait complete. 2024-07-31T14:59:46.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.866+0000 7f1bc7728740 20 client.17646 trim_cache size 0 max 16384 2024-07-31T14:59:46.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.903+0000 7f1bc7728740 1 Processor -- start 2024-07-31T14:59:46.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.903+0000 7f1bc7728740 1 -- start start 2024-07-31T14:59:46.905 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:46.905 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:46 smithi067 ceph-mon[32750]: pgmap v665: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.3 MiB/s wr, 358 op/s 2024-07-31T14:59:46.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.904+0000 7f1bc7728740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de26699f0 0x559de2a56810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.904+0000 7f1bc7728740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26821b0 0x559de2a5dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.904+0000 7f1bc7728740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de268a390 0x559de2a68080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.904+0000 7f1bc7728740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8010090 con 0x559de26821b0 2024-07-31T14:59:46.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.904+0000 7f1bc7728740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8011d20 con 0x559de26699f0 2024-07-31T14:59:46.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.904+0000 7f1bc7728740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8010760 con 0x559de268a390 2024-07-31T14:59:46.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.904+0000 7f1bba59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26821b0 0x559de2a5dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:46.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.904+0000 7f1bba59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26821b0 0x559de2a5dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36044/0 (socket says 172.21.15.67:36044) 2024-07-31T14:59:46.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.904+0000 7f1bba59c640 1 -- 172.21.15.67:0/2589181279 learned_addr learned my addr 172.21.15.67:0/2589181279 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:46.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.905+0000 7f1bba59c640 1 -- 172.21.15.67:0/2589181279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de268a390 msgr2=0x559de2a68080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:59:46.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.905+0000 7f1bba59c640 1 --2- 172.21.15.67:0/2589181279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de268a390 0x559de2a68080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.905+0000 7f1bba59c640 1 -- 172.21.15.67:0/2589181279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de26699f0 msgr2=0x559de2a56810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T14:59:46.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.905+0000 7f1bba59c640 1 --2- 172.21.15.67:0/2589181279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de26699f0 0x559de2a56810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.905+0000 7f1bba59c640 1 -- 172.21.15.67:0/2589181279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559de2a5a5d0 con 0x559de26821b0 2024-07-31T14:59:46.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.906+0000 7f1bba59c640 1 --2- 172.21.15.67:0/2589181279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26821b0 0x559de2a5dd00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009900 tx=0x7f1bac0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c3c3ff4fe4c76b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:46.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.908+0000 7f1ba0ff9640 1 -- 172.21.15.67:0/2589181279 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac044070 con 0x559de26821b0 2024-07-31T14:59:46.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.908+0000 7f1ba0ff9640 1 -- 172.21.15.67:0/2589181279 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bac003770 con 0x559de26821b0 2024-07-31T14:59:46.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.908+0000 7f1ba0ff9640 1 -- 172.21.15.67:0/2589181279 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac04a690 con 0x559de26821b0 2024-07-31T14:59:46.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.910+0000 7f1bc7728740 1 -- 172.21.15.67:0/2589181279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26821b0 msgr2=0x559de2a5dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:46.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.910+0000 7f1bc7728740 1 --2- 172.21.15.67:0/2589181279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26821b0 0x559de2a5dd00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009900 tx=0x7f1bac0043b0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.912+0000 7f1bc7728740 1 -- 172.21.15.67:0/2589181279 shutdown_connections 2024-07-31T14:59:46.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.912+0000 7f1bc7728740 1 --2- 172.21.15.67:0/2589181279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de268a390 0x559de2a68080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.912+0000 7f1bc7728740 1 --2- 172.21.15.67:0/2589181279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26821b0 0x559de2a5dd00 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.912+0000 7f1bc7728740 1 --2- 172.21.15.67:0/2589181279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de26699f0 0x559de2a56810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.912+0000 7f1bc7728740 1 -- 172.21.15.67:0/2589181279 >> 172.21.15.67:0/2589181279 conn(0x559de269f460 msgr2=0x559de2a5d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:46.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.918+0000 7f1bc7728740 1 -- 172.21.15.67:0/2589181279 shutdown_connections 2024-07-31T14:59:46.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.920+0000 7f1bc7728740 1 -- 172.21.15.67:0/2589181279 wait complete. 2024-07-31T14:59:46.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.921+0000 7f1bc7728740 1 -- start start 2024-07-31T14:59:46.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.921+0000 7f1bc7728740 1 Processor -- start 2024-07-31T14:59:46.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.921+0000 7f1bc7728740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de26699f0 0x559de2a5f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.921+0000 7f1bc7728740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26821b0 0x559de2a5f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.921+0000 7f1bc7728740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de268a390 0x559de2a5fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.921+0000 7f1bc7728740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8011650 con 0x559de26821b0 2024-07-31T14:59:46.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.921+0000 7f1bc7728740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8010ec0 con 0x559de268a390 2024-07-31T14:59:46.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.921+0000 7f1bc7728740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba800e090 con 0x559de26699f0 2024-07-31T14:59:46.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.921+0000 7f1bc7728740 10 client.0 _ll_register_callbacks cb 0x7f1bb40aa8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:46.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.922+0000 7f1bbb59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de268a390 0x559de2a5fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:46.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.922+0000 7f1bba59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26821b0 0x559de2a5f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:46.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.922+0000 7f1bba59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26821b0 0x559de2a5f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36058/0 (socket says 172.21.15.67:36058) 2024-07-31T14:59:46.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.922+0000 7f1bba59c640 1 -- 172.21.15.67:0/442517756 learned_addr learned my addr 172.21.15.67:0/442517756 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:46.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.923+0000 7f1bba59c640 1 -- 172.21.15.67:0/442517756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de26699f0 msgr2=0x559de2a5f240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:59:46.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.923+0000 7f1bba59c640 1 --2- 172.21.15.67:0/442517756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de26699f0 0x559de2a5f240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.923+0000 7f1bba59c640 1 -- 172.21.15.67:0/442517756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de268a390 msgr2=0x559de2a5fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.923+0000 7f1bba59c640 1 --2- 172.21.15.67:0/442517756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de268a390 0x559de2a5fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.923+0000 7f1bba59c640 1 -- 172.21.15.67:0/442517756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559de2ad54f0 con 0x559de26821b0 2024-07-31T14:59:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.923+0000 7f1bba59c640 1 --2- 172.21.15.67:0/442517756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26821b0 0x559de2a5f780 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009850 tx=0x7f1bac003b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d85187fa41d5ec server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.923+0000 7f1b9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T14:59:46.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.926+0000 7f1b9eff5640 1 -- 172.21.15.67:0/442517756 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac058060 con 0x559de26821b0 2024-07-31T14:59:46.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.926+0000 7f1b9eff5640 1 -- 172.21.15.67:0/442517756 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7f1bac031630 con 0x559de26821b0 2024-07-31T14:59:46.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.926+0000 7f1bc7728740 1 -- 172.21.15.67:0/442517756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bac003770 con 0x559de26821b0 2024-07-31T14:59:46.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.926+0000 7f1b9eff5640 1 -- 172.21.15.67:0/442517756 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f1bac05f050 con 0x559de26821b0 2024-07-31T14:59:46.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.927+0000 7f1bc7728740 10 client.17652 fetch_fsmap learned FSMap version 54 2024-07-31T14:59:46.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.927+0000 7f1bc7728740 1 -- 172.21.15.67:0/442517756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559de2ad6780 con 0x559de26821b0 2024-07-31T14:59:46.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.928+0000 7f1b9eff5640 1 -- 172.21.15.67:0/442517756 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bac003770 con 0x559de26821b0 2024-07-31T14:59:46.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.929+0000 7f1b9eff5640 1 -- 172.21.15.67:0/442517756 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 54) ==== 8398+0+0 (secure 0 0 0) 0x7f1bac04ab80 con 0x559de26821b0 2024-07-31T14:59:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.929+0000 7f1bc7728740 10 client.17652 fetch_fsmap finished waiting for FSMap version 54 2024-07-31T14:59:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.929+0000 7f1bc7728740 10 client.17652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.929+0000 7f1bc7728740 20 client.17652 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.929+0000 7f1bc7728740 1 --2- 172.21.15.67:0/442517756 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559de2ad6950 0x559de2ad8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.929+0000 7f1bc7728740 4 client.17652 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T14:59:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.929+0000 7f1bc7728740 1 -- 172.21.15.67:0/442517756 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559de29c4580 con 0x559de2ad6950 2024-07-31T14:59:46.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.933+0000 7f1bbad9d640 1 --2- 172.21.15.67:0/442517756 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559de2ad6950 0x559de2ad8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:46.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.939+0000 7f1bbad9d640 1 --2- 172.21.15.67:0/442517756 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559de2ad6950 0x559de2ad8d70 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4fb788bdf62f13e server_cookie=43c1c47e3d909285 in_seq=0 out_seq=0 2024-07-31T14:59:46.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.939+0000 7f1b9eff5640 10 client.17652 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:46.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.952+0000 7f1b9eff5640 1 -- 172.21.15.67:0/442517756 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x559de29c4580 con 0x559de2ad6950 2024-07-31T14:59:46.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.952+0000 7f1b9eff5640 10 client.17652 handle_command_reply: tid=0 2024-07-31T14:59:46.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.952+0000 7f1bc7728740 1 client.17652 shutdown 2024-07-31T14:59:46.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.954+0000 7f1bc7728740 1 -- 172.21.15.67:0/442517756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26821b0 msgr2=0x559de2a5f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:46.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.954+0000 7f1bc7728740 1 --2- 172.21.15.67:0/442517756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26821b0 0x559de2a5f780 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009850 tx=0x7f1bac003b50 comp rx=0 tx=0).stop 2024-07-31T14:59:46.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.955+0000 7f1bc7728740 1 -- 172.21.15.67:0/442517756 shutdown_connections 2024-07-31T14:59:46.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.955+0000 7f1bc7728740 1 --2- 172.21.15.67:0/442517756 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559de2ad6950 0x559de2ad8d70 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.955+0000 7f1bc7728740 1 --2- 172.21.15.67:0/442517756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559de268a390 0x559de2a5fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.955+0000 7f1bc7728740 1 --2- 172.21.15.67:0/442517756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559de26821b0 0x559de2a5f780 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.955+0000 7f1bc7728740 1 --2- 172.21.15.67:0/442517756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559de26699f0 0x559de2a5f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.955+0000 7f1bc7728740 1 -- 172.21.15.67:0/442517756 >> 172.21.15.67:0/442517756 conn(0x559de269f460 msgr2=0x559de2a51720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:46.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.955+0000 7f1b9eff5640 0 client.17652 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:46.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.956+0000 7f1bc7728740 1 -- 172.21.15.67:0/442517756 shutdown_connections 2024-07-31T14:59:46.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.956+0000 7f1bc7728740 1 -- 172.21.15.67:0/442517756 wait complete. 2024-07-31T14:59:46.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.956+0000 7f1bc7728740 20 client.17652 trim_cache size 0 max 16384 2024-07-31T14:59:46.958 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T14:59:46.958 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (13242 inodes in the stack)", 2024-07-31T14:59:46.958 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T14:59:46.958 INFO:teuthology.orchestra.run.smithi067.stdout: "c791f779-17a0-47ec-8943-a777d1eb6ea8": { 2024-07-31T14:59:46.958 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T14:59:46.958 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "c791f779-17a0-47ec-8943-a777d1eb6ea8", 2024-07-31T14:59:46.958 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T14:59:46.958 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:59:46.958 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T14:59:46.959 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T14:59:46.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.957+0000 7f1bc11d7640 1 -- 172.21.15.67:0/146102817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ba40b3d70 msgr2=0x7f1ba40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:46.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.957+0000 7f1bc11d7640 1 --2- 172.21.15.67:0/146102817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ba40b3d70 0x7f1ba40b6280 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x559de2a4f370 tx=0x7f1bb00090e0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.957+0000 7f1bc11d7640 1 -- 172.21.15.67:0/146102817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bbc11c5d0 msgr2=0x7f1bbc115710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.957+0000 7f1bc11d7640 1 --2- 172.21.15.67:0/146102817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bbc11c5d0 0x7f1bbc115710 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400cad0 tx=0x7f1bb400cf90 comp rx=0 tx=0).stop 2024-07-31T14:59:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.957+0000 7f1bc11d7640 1 -- 172.21.15.67:0/146102817 shutdown_connections 2024-07-31T14:59:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.957+0000 7f1bc11d7640 1 --2- 172.21.15.67:0/146102817 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1ba40b3d70 0x7f1ba40b6280 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.957+0000 7f1bc11d7640 1 --2- 172.21.15.67:0/146102817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bbc11c5d0 0x7f1bbc115710 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.957+0000 7f1bc11d7640 1 --2- 172.21.15.67:0/146102817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bbc111770 0x7f1bbc115150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.957+0000 7f1bc11d7640 1 --2- 172.21.15.67:0/146102817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc10a5b0 0x7f1bbc11be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.957+0000 7f1bc11d7640 1 -- 172.21.15.67:0/146102817 >> 172.21.15.67:0/146102817 conn(0x7f1bbc06d380 msgr2=0x7f1bbc110400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.957+0000 7f1bc11d7640 1 -- 172.21.15.67:0/146102817 shutdown_connections 2024-07-31T14:59:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T14:59:46.957+0000 7f1bc11d7640 1 -- 172.21.15.67:0/146102817 wait complete. 2024-07-31T14:59:46.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:46.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:46 smithi077 ceph-mon[37327]: pgmap v665: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.3 MiB/s wr, 358 op/s 2024-07-31T14:59:47.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:47.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:46 smithi073 ceph-mon[36714]: pgmap v665: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.3 MiB/s wr, 358 op/s 2024-07-31T14:59:47.701 INFO:tasks.cephfs.filesystem:scrub status for tag:c791f779-17a0-47ec-8943-a777d1eb6ea8 - {'path': '/', 'tag': 'c791f779-17a0-47ec-8943-a777d1eb6ea8', 'options': 'recursive,force'} 2024-07-31T14:59:47.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:47.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:47.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:47.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:47.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:47.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:47.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:47.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:48.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:48.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:48.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:48.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:48.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:48.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:48.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:48.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:48.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:48.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:48.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:48.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:48.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:48.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:48.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:48.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:48.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:48.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:48.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:48.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:48.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:48.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:48.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:48.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:48.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:48.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:48 smithi077 ceph-mon[37327]: pgmap v666: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.4 MiB/s wr, 544 op/s 2024-07-31T14:59:48.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:48.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:48.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:48.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:49.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:48 smithi067 ceph-mon[32750]: pgmap v666: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.4 MiB/s wr, 544 op/s 2024-07-31T14:59:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:49.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:49.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:49.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:49.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:49.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:48 smithi073 ceph-mon[36714]: pgmap v666: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.4 MiB/s wr, 544 op/s 2024-07-31T14:59:49.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:49.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:49.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:49.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:49.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:50.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:50.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:50.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:50.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:50.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:50 smithi077 ceph-mon[37327]: pgmap v667: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 717 op/s 2024-07-31T14:59:50.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:50 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:50.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:50 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:50.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:50 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:51.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:50 smithi067 ceph-mon[32750]: pgmap v667: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 717 op/s 2024-07-31T14:59:51.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:50 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:51.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:50 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:51.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:50 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:51.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:50 smithi073 ceph-mon[36714]: pgmap v667: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 717 op/s 2024-07-31T14:59:51.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:50 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:51.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:50 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:51.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:50 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:51.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:51.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:52.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:52.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:52.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:52.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:52.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:52.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:52.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:52.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:52.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:52.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:52.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:52 smithi077 ceph-mon[37327]: pgmap v668: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 909 op/s 2024-07-31T14:59:53.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:53.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:53.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:53.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:53.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:53.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:53.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:53.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:53.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:52 smithi067 ceph-mon[32750]: pgmap v668: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 909 op/s 2024-07-31T14:59:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:53.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:52 smithi073 ceph-mon[36714]: pgmap v668: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 909 op/s 2024-07-31T14:59:54.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:54.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:54.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:54.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:54.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:54.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:54.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:54.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:54.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:54.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:54.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:54.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:54.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:54.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:54.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T14:59:55.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:54 smithi067 ceph-mon[32750]: pgmap v669: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 963 op/s 2024-07-31T14:59:55.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:54 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:59:55.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:54 smithi073 ceph-mon[36714]: pgmap v669: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 963 op/s 2024-07-31T14:59:55.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:54 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:59:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:54 smithi077 ceph-mon[37327]: pgmap v669: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 963 op/s 2024-07-31T14:59:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:54 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T14:59:56.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:55 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:59:56.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:55 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:56.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:55 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.083+0000 7fe59ce4a640 1 -- 172.21.15.67:0/16388788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5981118d0 msgr2=0x7fe598109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.083+0000 7fe59ce4a640 1 --2- 172.21.15.67:0/16388788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5981118d0 0x7fe598109df0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe590009ca0 tx=0x7fe59002a970 comp rx=0 tx=0).stop 2024-07-31T14:59:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.086+0000 7fe59ce4a640 1 -- 172.21.15.67:0/16388788 shutdown_connections 2024-07-31T14:59:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.086+0000 7fe59ce4a640 1 --2- 172.21.15.67:0/16388788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59811c5d0 0x7fe59811eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.086+0000 7fe59ce4a640 1 --2- 172.21.15.67:0/16388788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe59810a330 0x7fe59810a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.086+0000 7fe59ce4a640 1 --2- 172.21.15.67:0/16388788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5981118d0 0x7fe598109df0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.086+0000 7fe59ce4a640 1 -- 172.21.15.67:0/16388788 >> 172.21.15.67:0/16388788 conn(0x7fe5980707c0 msgr2=0x7fe598070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.090+0000 7fe59ce4a640 1 -- 172.21.15.67:0/16388788 shutdown_connections 2024-07-31T14:59:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.094+0000 7fe59ce4a640 1 -- 172.21.15.67:0/16388788 wait complete. 2024-07-31T14:59:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.095+0000 7fe59ce4a640 1 Processor -- start 2024-07-31T14:59:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.098+0000 7fe59ce4a640 1 -- start start 2024-07-31T14:59:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.098+0000 7fe59ce4a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5981118d0 0x7fe59811b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.098+0000 7fe59ce4a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe59811c5d0 0x7fe598114300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.098+0000 7fe59ce4a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe598114960 0x7fe598119ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.098+0000 7fe59ce4a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981222e0 con 0x7fe59811c5d0 2024-07-31T14:59:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.098+0000 7fe59ce4a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe598122160 con 0x7fe5981118d0 2024-07-31T14:59:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.098+0000 7fe59ce4a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe598122460 con 0x7fe598114960 2024-07-31T14:59:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.102+0000 7fe59659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5981118d0 0x7fe59811b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.102+0000 7fe59659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5981118d0 0x7fe59811b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:32960/0 (socket says 172.21.15.67:32960) 2024-07-31T14:59:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.102+0000 7fe59659c640 1 -- 172.21.15.67:0/1415405369 learned_addr learned my addr 172.21.15.67:0/1415405369 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.102+0000 7fe596d9d640 1 --2- 172.21.15.67:0/1415405369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe598114960 0x7fe598119ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.102+0000 7fe59659c640 1 -- 172.21.15.67:0/1415405369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe598114960 msgr2=0x7fe598119ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.102+0000 7fe59659c640 1 --2- 172.21.15.67:0/1415405369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe598114960 0x7fe598119ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.102+0000 7fe59659c640 1 -- 172.21.15.67:0/1415405369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe59811c5d0 msgr2=0x7fe598114300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.102+0000 7fe59659c640 1 --2- 172.21.15.67:0/1415405369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe59811c5d0 0x7fe598114300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.103+0000 7fe59659c640 1 -- 172.21.15.67:0/1415405369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe598115100 con 0x7fe5981118d0 2024-07-31T14:59:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.103+0000 7fe59659c640 1 --2- 172.21.15.67:0/1415405369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5981118d0 0x7fe59811b040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe59002af60 tx=0x7fe5900308a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.105+0000 7fe5877fe640 1 -- 172.21.15.67:0/1415405369 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590034050 con 0x7fe5981118d0 2024-07-31T14:59:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.105+0000 7fe5877fe640 1 -- 172.21.15.67:0/1415405369 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe590004040 con 0x7fe5981118d0 2024-07-31T14:59:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.105+0000 7fe59ce4a640 1 -- 172.21.15.67:0/1415405369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59811a2a0 con 0x7fe5981118d0 2024-07-31T14:59:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.105+0000 7fe59ce4a640 1 -- 172.21.15.67:0/1415405369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe59811a790 con 0x7fe5981118d0 2024-07-31T14:59:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.108+0000 7fe5877fe640 1 -- 172.21.15.67:0/1415405369 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59002e7e0 con 0x7fe5981118d0 2024-07-31T14:59:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.111+0000 7fe5a3e17740 1 Processor -- start 2024-07-31T14:59:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.115+0000 7fe5a3e17740 1 -- start start 2024-07-31T14:59:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.115+0000 7fe5877fe640 1 -- 172.21.15.67:0/1415405369 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe590037020 con 0x7fe5981118d0 2024-07-31T14:59:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.120+0000 7fe5a3e17740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 0x55df32ba4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.120+0000 7fe5a3e17740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327d92f0 0x55df32ba5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.121+0000 7fe5a3e17740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327e14d0 0x55df32ba5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.121+0000 7fe5a3e17740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55df32b1a3b0 con 0x55df328021b0 2024-07-31T14:59:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.121+0000 7fe5a3e17740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55df32b098b0 con 0x55df327d92f0 2024-07-31T14:59:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.121+0000 7fe5a3e17740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55df32b0d8f0 con 0x55df327e14d0 2024-07-31T14:59:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.122+0000 7fe59659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327d92f0 0x55df32ba5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.122+0000 7fe59659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327d92f0 0x55df32ba5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:32966/0 (socket says 172.21.15.67:32966) 2024-07-31T14:59:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.122+0000 7fe59659c640 1 -- 172.21.15.67:0/369564808 learned_addr learned my addr 172.21.15.67:0/369564808 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.124+0000 7fe596d9d640 1 --2- 172.21.15.67:0/369564808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 0x55df32ba4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.124+0000 7fe59659c640 1 -- 172.21.15.67:0/369564808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327e14d0 msgr2=0x55df32ba5760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T14:59:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.124+0000 7fe59659c640 1 --2- 172.21.15.67:0/369564808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327e14d0 0x55df32ba5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.124+0000 7fe59659c640 1 -- 172.21.15.67:0/369564808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 msgr2=0x55df32ba4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.124+0000 7fe59659c640 1 --2- 172.21.15.67:0/369564808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 0x55df32ba4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.124+0000 7fe59659c640 1 -- 172.21.15.67:0/369564808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df32ab09a0 con 0x55df327d92f0 2024-07-31T14:59:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.124+0000 7fe5877fe640 1 --2- 172.21.15.67:0/1415405369 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe5800b3da0 0x7fe5800b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.125+0000 7fe5877fe640 1 -- 172.21.15.67:0/1415405369 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7fe590105620 con 0x7fe5981118d0 2024-07-31T14:59:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.125+0000 7fe59659c640 1 --2- 172.21.15.67:0/369564808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327d92f0 0x55df32ba5220 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe590102890 tx=0x7fe59010b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97e3b011eaec828c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.126+0000 7fe596d9d640 1 --2- 172.21.15.67:0/1415405369 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe5800b3da0 0x7fe5800b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.128+0000 7fe596d9d640 1 --2- 172.21.15.67:0/1415405369 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe5800b3da0 0x7fe5800b62b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe58800b710 tx=0x7fe58800b2e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.129+0000 7fe584ff9640 1 -- 172.21.15.67:0/369564808 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5900c7cd0 con 0x55df327d92f0 2024-07-31T14:59:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.129+0000 7fe584ff9640 1 -- 172.21.15.67:0/369564808 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe590112070 con 0x55df327d92f0 2024-07-31T14:59:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.129+0000 7fe584ff9640 1 -- 172.21.15.67:0/369564808 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59002e320 con 0x55df327d92f0 2024-07-31T14:59:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.131+0000 7fe5a3e17740 1 -- 172.21.15.67:0/369564808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327d92f0 msgr2=0x55df32ba5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.131+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/369564808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327d92f0 0x55df32ba5220 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe590102890 tx=0x7fe59010b9f0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.134+0000 7fe5a3e17740 1 -- 172.21.15.67:0/369564808 shutdown_connections 2024-07-31T14:59:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.134+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/369564808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327e14d0 0x55df32ba5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.134+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/369564808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327d92f0 0x55df32ba5220 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.134+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/369564808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 0x55df32ba4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.134+0000 7fe5a3e17740 1 -- 172.21.15.67:0/369564808 >> 172.21.15.67:0/369564808 conn(0x55df327f6300 msgr2=0x55df32ba2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.136+0000 7fe5a3e17740 1 -- 172.21.15.67:0/369564808 shutdown_connections 2024-07-31T14:59:56.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:55 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:59:56.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:55 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:56.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:55 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.141+0000 7fe5a3e17740 1 -- 172.21.15.67:0/369564808 wait complete. 2024-07-31T14:59:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.141+0000 7fe5a3e17740 1 -- start start 2024-07-31T14:59:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.141+0000 7fe5a3e17740 1 Processor -- start 2024-07-31T14:59:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.142+0000 7fe5a3e17740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327d92f0 0x55df32bb0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.142+0000 7fe5a3e17740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327e14d0 0x55df32bb0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.142+0000 7fe5a3e17740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 0x55df32bbc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.142+0000 7fe5a3e17740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55df32a43ab0 con 0x55df328021b0 2024-07-31T14:59:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.142+0000 7fe5a3e17740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55df32a91cf0 con 0x55df327d92f0 2024-07-31T14:59:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.142+0000 7fe5a3e17740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55df32b098b0 con 0x55df327e14d0 2024-07-31T14:59:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.142+0000 7fe5a3e17740 10 client.0 _ll_register_callbacks cb 0x55df32b039f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.142+0000 7fe59659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327d92f0 0x55df32bb0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.142+0000 7fe596d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 0x55df32bbc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.142+0000 7fe59659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327d92f0 0x55df32bb0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:32976/0 (socket says 172.21.15.67:32976) 2024-07-31T14:59:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.142+0000 7fe59659c640 1 -- 172.21.15.67:0/2667842740 learned_addr learned my addr 172.21.15.67:0/2667842740 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.146+0000 7fe595d9b640 1 --2- 172.21.15.67:0/2667842740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327e14d0 0x55df32bb0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.146+0000 7fe59659c640 1 -- 172.21.15.67:0/2667842740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327e14d0 msgr2=0x55df32bb0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.146+0000 7fe59659c640 1 --2- 172.21.15.67:0/2667842740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327e14d0 0x55df32bb0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.146+0000 7fe59659c640 1 -- 172.21.15.67:0/2667842740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 msgr2=0x55df32bbc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.146+0000 7fe59659c640 1 --2- 172.21.15.67:0/2667842740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 0x55df32bbc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.146+0000 7fe59659c640 1 -- 172.21.15.67:0/2667842740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3286d790 con 0x55df327d92f0 2024-07-31T14:59:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.149+0000 7fe59659c640 1 --2- 172.21.15.67:0/2667842740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327d92f0 0x55df32bb0820 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe59003b040 tx=0x7fe590102e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a996d09532574537 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.149+0000 7fe57dffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T14:59:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.150+0000 7fe57dffb640 1 -- 172.21.15.67:0/2667842740 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5900cb580 con 0x55df327d92f0 2024-07-31T14:59:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.150+0000 7fe57dffb640 1 -- 172.21.15.67:0/2667842740 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7fe5901028c0 con 0x55df327d92f0 2024-07-31T14:59:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.151+0000 7fe5a3e17740 1 -- 172.21.15.67:0/2667842740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe590112070 con 0x55df327d92f0 2024-07-31T14:59:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.153+0000 7fe57dffb640 1 -- 172.21.15.67:0/2667842740 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe5900cb760 con 0x55df327d92f0 2024-07-31T14:59:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.154+0000 7fe5a3e17740 10 client.27218 fetch_fsmap learned FSMap version 55 2024-07-31T14:59:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.154+0000 7fe5a3e17740 1 -- 172.21.15.67:0/2667842740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55df328e6620 con 0x55df327d92f0 2024-07-31T14:59:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.157+0000 7fe57dffb640 1 -- 172.21.15.67:0/2667842740 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe590112070 con 0x55df327d92f0 2024-07-31T14:59:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.157+0000 7fe57dffb640 1 -- 172.21.15.67:0/2667842740 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7fe590041070 con 0x55df327d92f0 2024-07-31T14:59:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.158+0000 7fe5a3e17740 10 client.27218 fetch_fsmap finished waiting for FSMap version 55 2024-07-31T14:59:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.158+0000 7fe5a3e17740 10 client.27218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.158+0000 7fe5a3e17740 20 client.27218 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.158+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/2667842740 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55df327c0d80 0x55df32be4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.158+0000 7fe5a3e17740 4 client.27218 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T14:59:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.158+0000 7fe5a3e17740 1 -- 172.21.15.67:0/2667842740 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df32b1c300 con 0x55df327c0d80 2024-07-31T14:59:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.162+0000 7fe595d9b640 1 --2- 172.21.15.67:0/2667842740 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55df327c0d80 0x55df32be4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.182+0000 7fe595d9b640 1 --2- 172.21.15.67:0/2667842740 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55df327c0d80 0x55df32be4240 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d387cfcda4afe5a server_cookie=2c768ab4033ee4f0 in_seq=0 out_seq=0 2024-07-31T14:59:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.184+0000 7fe57dffb640 10 client.27218 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.188+0000 7fe57dffb640 1 -- 172.21.15.67:0/2667842740 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df32b1c300 con 0x55df327c0d80 2024-07-31T14:59:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.188+0000 7fe57dffb640 10 client.27218 handle_command_reply: tid=0 2024-07-31T14:59:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.193+0000 7fe5a3e17740 1 client.27218 shutdown 2024-07-31T14:59:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.208+0000 7fe5a3e17740 1 -- 172.21.15.67:0/2667842740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327d92f0 msgr2=0x55df32bb0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.208+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/2667842740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327d92f0 0x55df32bb0820 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe59003b040 tx=0x7fe590102e00 comp rx=0 tx=0).stop 2024-07-31T14:59:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.213+0000 7fe5a3e17740 1 -- 172.21.15.67:0/2667842740 shutdown_connections 2024-07-31T14:59:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.213+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/2667842740 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55df327c0d80 0x55df32be4240 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.213+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/2667842740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 0x55df32bbc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.213+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/2667842740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327e14d0 0x55df32bb0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.213+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/2667842740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327d92f0 0x55df32bb0820 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.213+0000 7fe5a3e17740 1 -- 172.21.15.67:0/2667842740 >> 172.21.15.67:0/2667842740 conn(0x55df327f6300 msgr2=0x55df32ba5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.213+0000 7fe57dffb640 0 client.27218 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.225+0000 7fe5a3e17740 1 -- 172.21.15.67:0/2667842740 shutdown_connections 2024-07-31T14:59:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.231+0000 7fe5a3e17740 1 -- 172.21.15.67:0/2667842740 wait complete. 2024-07-31T14:59:56.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:55 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T14:59:56.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:55 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:56.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:55 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.235+0000 7fe5a3e17740 20 client.27218 trim_cache size 0 max 16384 2024-07-31T14:59:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.283+0000 7fe5a3e17740 1 Processor -- start 2024-07-31T14:59:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.285+0000 7fe5a3e17740 1 -- start start 2024-07-31T14:59:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.285+0000 7fe5a3e17740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327d92f0 0x55df32ba6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.285+0000 7fe5a3e17740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327e14d0 0x55df32baae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.285+0000 7fe5a3e17740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 0x55df32bb0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.286+0000 7fe5a3e17740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe57800ffe0 con 0x55df328021b0 2024-07-31T14:59:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.286+0000 7fe5a3e17740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe578011c70 con 0x55df327e14d0 2024-07-31T14:59:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.286+0000 7fe5a3e17740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe5780106b0 con 0x55df327d92f0 2024-07-31T14:59:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.287+0000 7fe596d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 0x55df32bb0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.287+0000 7fe596d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 0x55df32bb0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49268/0 (socket says 172.21.15.67:49268) 2024-07-31T14:59:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.287+0000 7fe596d9d640 1 -- 172.21.15.67:0/1266474888 learned_addr learned my addr 172.21.15.67:0/1266474888 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.290+0000 7fe59659c640 1 --2- 172.21.15.67:0/1266474888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327d92f0 0x55df32ba6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.291+0000 7fe595d9b640 1 --2- 172.21.15.67:0/1266474888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327e14d0 0x55df32baae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.292+0000 7fe596d9d640 1 -- 172.21.15.67:0/1266474888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327d92f0 msgr2=0x55df32ba6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.292+0000 7fe596d9d640 1 --2- 172.21.15.67:0/1266474888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327d92f0 0x55df32ba6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.292+0000 7fe596d9d640 1 -- 172.21.15.67:0/1266474888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327e14d0 msgr2=0x55df32baae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.292+0000 7fe596d9d640 1 --2- 172.21.15.67:0/1266474888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327e14d0 0x55df32baae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.292+0000 7fe596d9d640 1 -- 172.21.15.67:0/1266474888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df32bb1600 con 0x55df328021b0 2024-07-31T14:59:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.293+0000 7fe596d9d640 1 --2- 172.21.15.67:0/1266474888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 0x55df32bb0580 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe588003d40 tx=0x7fe588010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51de5775f10804af server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.296+0000 7fe57ffff640 1 -- 172.21.15.67:0/1266474888 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58802b070 con 0x55df328021b0 2024-07-31T14:59:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.296+0000 7fe57ffff640 1 -- 172.21.15.67:0/1266474888 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe58801b910 con 0x55df328021b0 2024-07-31T14:59:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.297+0000 7fe5a3e17740 1 -- 172.21.15.67:0/1266474888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 msgr2=0x55df32bb0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T14:59:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.297+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/1266474888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 0x55df32bb0580 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe588003d40 tx=0x7fe588010000 comp rx=0 tx=0).stop 2024-07-31T14:59:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.298+0000 7fe5a3e17740 1 -- 172.21.15.67:0/1266474888 shutdown_connections 2024-07-31T14:59:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.298+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/1266474888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df328021b0 0x55df32bb0580 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.298+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/1266474888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327e14d0 0x55df32baae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.298+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/1266474888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327d92f0 0x55df32ba6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.298+0000 7fe5a3e17740 1 -- 172.21.15.67:0/1266474888 >> 172.21.15.67:0/1266474888 conn(0x55df327f6300 msgr2=0x55df32ba5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T14:59:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.301+0000 7fe5a3e17740 1 -- 172.21.15.67:0/1266474888 shutdown_connections 2024-07-31T14:59:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.306+0000 7fe5a3e17740 1 -- 172.21.15.67:0/1266474888 wait complete. 2024-07-31T14:59:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.306+0000 7fe5a3e17740 1 -- start start 2024-07-31T14:59:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.306+0000 7fe5a3e17740 1 Processor -- start 2024-07-31T14:59:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.307+0000 7fe5a3e17740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327d92f0 0x55df32bb52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.307+0000 7fe5a3e17740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df327e14d0 0x55df32bb5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.307+0000 7fe5a3e17740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327961c0 0x55df32bb5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.307+0000 7fe5a3e17740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe5780115a0 con 0x55df327e14d0 2024-07-31T14:59:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.307+0000 7fe5a3e17740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe578010e10 con 0x55df327961c0 2024-07-31T14:59:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.307+0000 7fe5a3e17740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe57800dfe0 con 0x55df327d92f0 2024-07-31T14:59:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.307+0000 7fe5a3e17740 10 client.0 _ll_register_callbacks cb 0x7fe59010b3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T14:59:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.310+0000 7fe59659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327d92f0 0x55df32bb52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.310+0000 7fe59659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327d92f0 0x55df32bb52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:46918/0 (socket says 172.21.15.67:46918) 2024-07-31T14:59:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.310+0000 7fe59659c640 1 -- 172.21.15.67:0/571471061 learned_addr learned my addr 172.21.15.67:0/571471061 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T14:59:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.311+0000 7fe596d9d640 1 --2- 172.21.15.67:0/571471061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327961c0 0x55df32bb5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.313+0000 7fe595d9b640 1 --2- 172.21.15.67:0/571471061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df327e14d0 0x55df32bb5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.313+0000 7fe59659c640 1 -- 172.21.15.67:0/571471061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327961c0 msgr2=0x55df32bb5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.313+0000 7fe59659c640 1 --2- 172.21.15.67:0/571471061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327961c0 0x55df32bb5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.313+0000 7fe59659c640 1 -- 172.21.15.67:0/571471061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df327e14d0 msgr2=0x55df32bb5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T14:59:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.313+0000 7fe59659c640 1 --2- 172.21.15.67:0/571471061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df327e14d0 0x55df32bb5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T14:59:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.313+0000 7fe59659c640 1 -- 172.21.15.67:0/571471061 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df32c4aa50 con 0x55df327d92f0 2024-07-31T14:59:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.322+0000 7fe59659c640 1 --2- 172.21.15.67:0/571471061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327d92f0 0x55df32bb52c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe5900cbb10 tx=0x7fe59010d140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d33361fc5807544 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T14:59:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.323+0000 7fe57dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T14:59:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.326+0000 7fe57dffb640 1 -- 172.21.15.67:0/571471061 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5900cbbe0 con 0x55df327d92f0 2024-07-31T14:59:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.326+0000 7fe5a3e17740 1 -- 172.21.15.67:0/571471061 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55df32b17da0 con 0x55df327d92f0 2024-07-31T14:59:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.327+0000 7fe57dffb640 1 -- 172.21.15.67:0/571471061 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9483+0+0 (secure 0 0 0) 0x7fe5900d4c40 con 0x55df327d92f0 2024-07-31T14:59:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.330+0000 7fe57dffb640 1 -- 172.21.15.67:0/571471061 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe5900ea030 con 0x55df327d92f0 2024-07-31T14:59:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.331+0000 7fe5a3e17740 10 client.27220 fetch_fsmap learned FSMap version 55 2024-07-31T14:59:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.331+0000 7fe5a3e17740 1 -- 172.21.15.67:0/571471061 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55df32c4aa50 con 0x55df327d92f0 2024-07-31T14:59:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.340+0000 7fe57dffb640 1 -- 172.21.15.67:0/571471061 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55df32b17da0 con 0x55df327d92f0 2024-07-31T14:59:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.340+0000 7fe57dffb640 1 -- 172.21.15.67:0/571471061 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7fe590041040 con 0x55df327d92f0 2024-07-31T14:59:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.341+0000 7fe5a3e17740 10 client.27220 fetch_fsmap finished waiting for FSMap version 55 2024-07-31T14:59:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.341+0000 7fe5a3e17740 10 client.27220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T14:59:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.341+0000 7fe5a3e17740 20 client.27220 populate_metadata read hostname 'smithi067' 2024-07-31T14:59:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.341+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/571471061 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55df32c4c1a0 0x55df32c4e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T14:59:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.341+0000 7fe5a3e17740 4 client.27220 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T14:59:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.341+0000 7fe5a3e17740 1 -- 172.21.15.67:0/571471061 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df32b154e0 con 0x55df32c4c1a0 2024-07-31T14:59:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.343+0000 7fe595d9b640 1 --2- 172.21.15.67:0/571471061 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55df32c4c1a0 0x55df32c4e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T14:59:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.348+0000 7fe595d9b640 1 --2- 172.21.15.67:0/571471061 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55df32c4c1a0 0x55df32c4e5c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39966a899e002843 server_cookie=3ea8e4f809fe1ce8 in_seq=0 out_seq=0 2024-07-31T14:59:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T14:59:56.351+0000 7fe57dffb640 10 client.27220 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T14:59:56.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:56 smithi073 ceph-mon[36714]: pgmap v670: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 1.12k op/s 2024-07-31T14:59:57.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:56 smithi067 ceph-mon[32750]: pgmap v670: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 1.12k op/s 2024-07-31T14:59:57.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:56 smithi077 ceph-mon[37327]: pgmap v670: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 1.12k op/s 2024-07-31T14:59:58.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:58.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:58.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T14:59:58.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:58 smithi077 ceph-mon[37327]: pgmap v671: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 1.04k op/s 2024-07-31T14:59:58.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:58.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 14:59:58 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:58.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:58.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:58.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T14:59:58.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:58 smithi067 ceph-mon[32750]: pgmap v671: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 1.04k op/s 2024-07-31T14:59:58.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:58.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 14:59:58 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:58.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:58.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:58.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T14:59:58.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:58 smithi073 ceph-mon[36714]: pgmap v671: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 1.04k op/s 2024-07-31T14:59:58.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T14:59:58.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 14:59:58 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:00:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:00:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:00:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:00 smithi077 ceph-mon[37327]: pgmap v672: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 1.04k op/s 2024-07-31T15:00:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:00 smithi077 ceph-mon[37327]: osdmap e95: 12 total, 12 up, 12 in 2024-07-31T15:00:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:00 smithi077 ceph-mon[37327]: overall HEALTH_OK 2024-07-31T15:00:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:00:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:00:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:00 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:00:00.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:00:00.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:00:00.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:00.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:00 smithi067 ceph-mon[32750]: pgmap v672: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 1.04k op/s 2024-07-31T15:00:00.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:00 smithi067 ceph-mon[32750]: osdmap e95: 12 total, 12 up, 12 in 2024-07-31T15:00:00.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:00 smithi067 ceph-mon[32750]: overall HEALTH_OK 2024-07-31T15:00:00.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:00:00.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:00:00.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:00 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:00 smithi073 ceph-mon[36714]: pgmap v672: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 1.04k op/s 2024-07-31T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:00 smithi073 ceph-mon[36714]: osdmap e95: 12 total, 12 up, 12 in 2024-07-31T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:00 smithi073 ceph-mon[36714]: overall HEALTH_OK 2024-07-31T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:00 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:00:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe57dffb640 1 -- 172.21.15.67:0/571471061 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df32b154e0 con 0x55df32c4c1a0 2024-07-31T15:00:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe57dffb640 10 client.27220 handle_command_reply: tid=0 2024-07-31T15:00:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe5a3e17740 1 client.27220 shutdown 2024-07-31T15:00:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe5a3e17740 1 -- 172.21.15.67:0/571471061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327d92f0 msgr2=0x55df32bb52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/571471061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327d92f0 0x55df32bb52c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe5900cbb10 tx=0x7fe59010d140 comp rx=0 tx=0).stop 2024-07-31T15:00:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe5a3e17740 1 -- 172.21.15.67:0/571471061 shutdown_connections 2024-07-31T15:00:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/571471061 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55df32c4c1a0 0x55df32c4e5c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/571471061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df327961c0 0x55df32bb5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe57dffb640 0 client.27220 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/571471061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55df327e14d0 0x55df32bb5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe5a3e17740 1 --2- 172.21.15.67:0/571471061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df327d92f0 0x55df32bb52c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe5a3e17740 1 -- 172.21.15.67:0/571471061 >> 172.21.15.67:0/571471061 conn(0x55df327f6300 msgr2=0x55df32bb0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe5a3e17740 1 -- 172.21.15.67:0/571471061 shutdown_connections 2024-07-31T15:00:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe5a3e17740 1 -- 172.21.15.67:0/571471061 wait complete. 2024-07-31T15:00:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.689+0000 7fe5a3e17740 20 client.27220 trim_cache size 0 max 16384 2024-07-31T15:00:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:00:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:00:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:00:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:00:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.690+0000 7fe59ce4a640 1 -- 172.21.15.67:0/1415405369 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe5800b3da0 msgr2=0x7fe5800b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.690+0000 7fe59ce4a640 1 --2- 172.21.15.67:0/1415405369 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe5800b3da0 0x7fe5800b62b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe58800b710 tx=0x7fe58800b2e0 comp rx=0 tx=0).stop 2024-07-31T15:00:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.690+0000 7fe59ce4a640 1 -- 172.21.15.67:0/1415405369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5981118d0 msgr2=0x7fe59811b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.690+0000 7fe59ce4a640 1 --2- 172.21.15.67:0/1415405369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5981118d0 0x7fe59811b040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe59002af60 tx=0x7fe5900308a0 comp rx=0 tx=0).stop 2024-07-31T15:00:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.690+0000 7fe59ce4a640 1 -- 172.21.15.67:0/1415405369 shutdown_connections 2024-07-31T15:00:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.690+0000 7fe59ce4a640 1 --2- 172.21.15.67:0/1415405369 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe5800b3da0 0x7fe5800b62b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.690+0000 7fe59ce4a640 1 --2- 172.21.15.67:0/1415405369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe598114960 0x7fe598119ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.690+0000 7fe59ce4a640 1 --2- 172.21.15.67:0/1415405369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe59811c5d0 0x7fe598114300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.690+0000 7fe59ce4a640 1 --2- 172.21.15.67:0/1415405369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5981118d0 0x7fe59811b040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.690+0000 7fe59ce4a640 1 -- 172.21.15.67:0/1415405369 >> 172.21.15.67:0/1415405369 conn(0x7fe5980707c0 msgr2=0x7fe598071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.690+0000 7fe59ce4a640 1 -- 172.21.15.67:0/1415405369 shutdown_connections 2024-07-31T15:00:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:01.690+0000 7fe59ce4a640 1 -- 172.21.15.67:0/1415405369 wait complete. 2024-07-31T15:00:02.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:02 smithi077 ceph-mon[37327]: pgmap v674: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.7 MiB/s wr, 1.30k op/s 2024-07-31T15:00:02.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:02.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:02 smithi067 ceph-mon[32750]: pgmap v674: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.7 MiB/s wr, 1.30k op/s 2024-07-31T15:00:03.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:03.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:03.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:03.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:03.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:03.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:02 smithi073 ceph-mon[36714]: pgmap v674: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.7 MiB/s wr, 1.30k op/s 2024-07-31T15:00:03.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:03.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:03.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:03.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:03.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:03.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:03.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:04.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:04.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:04.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:04 smithi077 ceph-mon[37327]: pgmap v675: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.8 MiB/s wr, 1.45k op/s 2024-07-31T15:00:05.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:05.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:05.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:05.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:05.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:05.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:05.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:05.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:05.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:05.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:05.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:05.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:04 smithi067 ceph-mon[32750]: pgmap v675: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.8 MiB/s wr, 1.45k op/s 2024-07-31T15:00:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:05.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:05.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:05.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:05.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:05.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:05.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:05.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:05.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:04 smithi073 ceph-mon[36714]: pgmap v675: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.8 MiB/s wr, 1.45k op/s 2024-07-31T15:00:05.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:05.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:06.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:06.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:06.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:06.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:06.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:06.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:06.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:06.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:06.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:06 smithi077 ceph-mon[37327]: pgmap v676: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.3 MiB/s wr, 1.68k op/s 2024-07-31T15:00:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:06.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:06 smithi067 ceph-mon[32750]: pgmap v676: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.3 MiB/s wr, 1.68k op/s 2024-07-31T15:00:07.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:07.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:07.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:07.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:06 smithi073 ceph-mon[36714]: pgmap v676: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.3 MiB/s wr, 1.68k op/s 2024-07-31T15:00:07.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:07.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:07.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:07.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:07.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:07.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:07.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:07.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:07 smithi077 ceph-mon[37327]: osdmap e96: 12 total, 12 up, 12 in 2024-07-31T15:00:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:08.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:08.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:08.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:08.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:08.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:07 smithi067 ceph-mon[32750]: osdmap e96: 12 total, 12 up, 12 in 2024-07-31T15:00:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:07 smithi073 ceph-mon[36714]: osdmap e96: 12 total, 12 up, 12 in 2024-07-31T15:00:09.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:08 smithi067 ceph-mon[32750]: pgmap v678: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 1.82k op/s 2024-07-31T15:00:09.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:09.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:09.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:08 smithi073 ceph-mon[36714]: pgmap v678: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 1.82k op/s 2024-07-31T15:00:09.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:09.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:09.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:08 smithi077 ceph-mon[37327]: pgmap v678: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 1.82k op/s 2024-07-31T15:00:09.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:09.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:11.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:10 smithi067 ceph-mon[32750]: pgmap v679: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 1.63k op/s 2024-07-31T15:00:11.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:10 smithi073 ceph-mon[36714]: pgmap v679: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 1.63k op/s 2024-07-31T15:00:11.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:10 smithi077 ceph-mon[37327]: pgmap v679: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 1.63k op/s 2024-07-31T15:00:12.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:12.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:12.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:12.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:12.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:12.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:13.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:12 smithi067 ceph-mon[32750]: pgmap v680: 129 pgs: 19 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 54 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 1.22k op/s 2024-07-31T15:00:13.083 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd linux-6.5.11 2024-07-31T15:00:13.092 INFO:tasks.workunit.client.0.smithi067.stderr:+ make defconfig 2024-07-31T15:00:13.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:12 smithi073 ceph-mon[36714]: pgmap v680: 129 pgs: 19 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 54 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 1.22k op/s 2024-07-31T15:00:13.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:12 smithi077 ceph-mon[37327]: pgmap v680: 129 pgs: 19 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 54 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 1.22k op/s 2024-07-31T15:00:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.883+0000 7f81fdaa4640 1 -- 172.21.15.67:0/1412309067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f810a330 msgr2=0x7f81f810a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.883+0000 7f81fdaa4640 1 --2- 172.21.15.67:0/1412309067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f810a330 0x7f81f810a740 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009ca0 tx=0x7f81f002a980 comp rx=0 tx=0).stop 2024-07-31T15:00:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.887+0000 7f81fdaa4640 1 -- 172.21.15.67:0/1412309067 shutdown_connections 2024-07-31T15:00:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.887+0000 7f81fdaa4640 1 --2- 172.21.15.67:0/1412309067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f811c5d0 0x7f81f811eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.887+0000 7f81fdaa4640 1 --2- 172.21.15.67:0/1412309067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f810a330 0x7f81f810a740 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.887+0000 7f81fdaa4640 1 --2- 172.21.15.67:0/1412309067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f81118d0 0x7f81f8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.887+0000 7f81fdaa4640 1 -- 172.21.15.67:0/1412309067 >> 172.21.15.67:0/1412309067 conn(0x7f81f80707c0 msgr2=0x7f81f8070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.893+0000 7f81fdaa4640 1 -- 172.21.15.67:0/1412309067 shutdown_connections 2024-07-31T15:00:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.899+0000 7f81fdaa4640 1 -- 172.21.15.67:0/1412309067 wait complete. 2024-07-31T15:00:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.900+0000 7f81fdaa4640 1 Processor -- start 2024-07-31T15:00:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.907+0000 7f81fdaa4640 1 -- start start 2024-07-31T15:00:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.909+0000 7f81fdaa4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f810a330 0x7f81f811aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.909+0000 7f81fdaa4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f81118d0 0x7f81f8114160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.909+0000 7f81fdaa4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f811c5d0 0x7f81f8114750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.909+0000 7f81fdaa4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f81f8122360 con 0x7f81f810a330 2024-07-31T15:00:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.909+0000 7f81fdaa4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f81f81221e0 con 0x7f81f811c5d0 2024-07-31T15:00:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.909+0000 7f81fdaa4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f81f81224e0 con 0x7f81f81118d0 2024-07-31T15:00:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.911+0000 7f81f7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f811c5d0 0x7f81f8114750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.912+0000 7f81f7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f811c5d0 0x7f81f8114750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46422/0 (socket says 172.21.15.67:46422) 2024-07-31T15:00:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.912+0000 7f81f7fff640 1 -- 172.21.15.67:0/211078586 learned_addr learned my addr 172.21.15.67:0/211078586 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.912+0000 7f81f77fe640 1 --2- 172.21.15.67:0/211078586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f810a330 0x7f81f811aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.914+0000 7f81f7fff640 1 -- 172.21.15.67:0/211078586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f81118d0 msgr2=0x7f81f8114160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:00:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.914+0000 7f81f7fff640 1 --2- 172.21.15.67:0/211078586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f81118d0 0x7f81f8114160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.914+0000 7f81f7fff640 1 -- 172.21.15.67:0/211078586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f810a330 msgr2=0x7f81f811aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.914+0000 7f81f7fff640 1 --2- 172.21.15.67:0/211078586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f810a330 0x7f81f811aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.914+0000 7f81f7fff640 1 -- 172.21.15.67:0/211078586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f8114f70 con 0x7f81f811c5d0 2024-07-31T15:00:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.917+0000 7f81f7fff640 1 --2- 172.21.15.67:0/211078586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f811c5d0 0x7f81f8114750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009fd0 tx=0x7f81e800b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.919+0000 7f81f4ff9640 1 -- 172.21.15.67:0/211078586 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8011830 con 0x7f81f811c5d0 2024-07-31T15:00:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.919+0000 7f81f4ff9640 1 -- 172.21.15.67:0/211078586 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e8016070 con 0x7f81f811c5d0 2024-07-31T15:00:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.919+0000 7f81f4ff9640 1 -- 172.21.15.67:0/211078586 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8010de0 con 0x7f81f811c5d0 2024-07-31T15:00:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.920+0000 7f81fdaa4640 1 -- 172.21.15.67:0/211078586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f8119c20 con 0x7f81f811c5d0 2024-07-31T15:00:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.920+0000 7f81fdaa4640 1 -- 172.21.15.67:0/211078586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f811a170 con 0x7f81f811c5d0 2024-07-31T15:00:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.924+0000 7f8204999740 1 Processor -- start 2024-07-31T15:00:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.933+0000 7f8204999740 1 -- start start 2024-07-31T15:00:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.933+0000 7f81f4ff9640 1 -- 172.21.15.67:0/211078586 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f81e80119d0 con 0x7f81f811c5d0 2024-07-31T15:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.933+0000 7f8204999740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeaf53ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.933+0000 7f8204999740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf54220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.933+0000 7f8204999740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf54760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.933+0000 7f8204999740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558aeaec93b0 con 0x558aeabb11b0 2024-07-31T15:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.933+0000 7f8204999740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558aeaeb88b0 con 0x558aeab882f0 2024-07-31T15:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.933+0000 7f8204999740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558aeaebc8f0 con 0x558aeab904d0 2024-07-31T15:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.937+0000 7f81f77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf54220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.937+0000 7f81f6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf54760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.937+0000 7f81f6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeaf53ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.937+0000 7f81f77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf54220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46436/0 (socket says 172.21.15.67:46436) 2024-07-31T15:00:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.937+0000 7f81f77fe640 1 -- 172.21.15.67:0/637833427 learned_addr learned my addr 172.21.15.67:0/637833427 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.942+0000 7f81f4ff9640 1 --2- 172.21.15.67:0/211078586 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81e00b3f50 0x7f81e00b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.942+0000 7f81f4ff9640 1 -- 172.21.15.67:0/211078586 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9515+0+0 (secure 0 0 0) 0x7f81e80e3800 con 0x7f81f811c5d0 2024-07-31T15:00:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.947+0000 7f81f7fff640 1 --2- 172.21.15.67:0/211078586 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81e00b3f50 0x7f81e00b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.948+0000 7f81f6ffd640 1 -- 172.21.15.67:0/637833427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 msgr2=0x558aeaf54760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.948+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/637833427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf54760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.948+0000 7f81f6ffd640 1 -- 172.21.15.67:0/637833427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 msgr2=0x558aeaf54220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.948+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/637833427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf54220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.948+0000 7f81f6ffd640 1 -- 172.21.15.67:0/637833427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aeae5f9a0 con 0x558aeabb11b0 2024-07-31T15:00:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.949+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/637833427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeaf53ce0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f81f002af70 tx=0x7f81f0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72cc3ba5b190ac56 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.950+0000 7f81ddffb640 1 -- 172.21.15.67:0/637833427 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0034040 con 0x558aeabb11b0 2024-07-31T15:00:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.950+0000 7f81ddffb640 1 -- 172.21.15.67:0/637833427 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f0003e20 con 0x558aeabb11b0 2024-07-31T15:00:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.950+0000 7f81ddffb640 1 -- 172.21.15.67:0/637833427 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f00308e0 con 0x558aeabb11b0 2024-07-31T15:00:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.952+0000 7f8204999740 1 -- 172.21.15.67:0/637833427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 msgr2=0x558aeaf53ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.952+0000 7f8204999740 1 --2- 172.21.15.67:0/637833427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeaf53ce0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f81f002af70 tx=0x7f81f0005fd0 comp rx=0 tx=0).stop 2024-07-31T15:00:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.954+0000 7f8204999740 1 -- 172.21.15.67:0/637833427 shutdown_connections 2024-07-31T15:00:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.954+0000 7f8204999740 1 --2- 172.21.15.67:0/637833427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf54760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.954+0000 7f8204999740 1 --2- 172.21.15.67:0/637833427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf54220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.954+0000 7f8204999740 1 --2- 172.21.15.67:0/637833427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeaf53ce0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.954+0000 7f8204999740 1 -- 172.21.15.67:0/637833427 >> 172.21.15.67:0/637833427 conn(0x558aeaba5300 msgr2=0x558aeaf51bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.958+0000 7f8204999740 1 -- 172.21.15.67:0/637833427 shutdown_connections 2024-07-31T15:00:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.959+0000 7f81f7fff640 1 --2- 172.21.15.67:0/211078586 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81e00b3f50 0x7f81e00b6460 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f81e80a9de0 tx=0x7f81e80047d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.962+0000 7f8204999740 1 -- 172.21.15.67:0/637833427 wait complete. 2024-07-31T15:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.963+0000 7f8204999740 1 -- start start 2024-07-31T15:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.963+0000 7f8204999740 1 Processor -- start 2024-07-31T15:00:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.964+0000 7f8204999740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab882f0 0x558aeaf5f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.964+0000 7f8204999740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab904d0 0x558aeaf5fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.964+0000 7f8204999740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeaf6b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.964+0000 7f8204999740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558aeadf2ab0 con 0x558aeabb11b0 2024-07-31T15:00:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.964+0000 7f8204999740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558aeae40cf0 con 0x558aeab904d0 2024-07-31T15:00:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.964+0000 7f8204999740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558aeaf623a0 con 0x558aeab882f0 2024-07-31T15:00:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.964+0000 7f8204999740 10 client.0 _ll_register_callbacks cb 0x558aeaeb29f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:00:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.966+0000 7f81f6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab882f0 0x558aeaf5f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.966+0000 7f81f6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab882f0 0x558aeaf5f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58516/0 (socket says 172.21.15.67:58516) 2024-07-31T15:00:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.966+0000 7f81f6ffd640 1 -- 172.21.15.67:0/2919486600 learned_addr learned my addr 172.21.15.67:0/2919486600 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.966+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/2919486600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeaf6b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.967+0000 7f81f6ffd640 1 -- 172.21.15.67:0/2919486600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab904d0 msgr2=0x558aeaf5fd60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:00:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.967+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/2919486600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab904d0 0x558aeaf5fd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.967+0000 7f81f6ffd640 1 -- 172.21.15.67:0/2919486600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 msgr2=0x558aeaf6b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.967+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/2919486600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeaf6b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.967+0000 7f81f6ffd640 1 -- 172.21.15.67:0/2919486600 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aeac1c790 con 0x558aeab882f0 2024-07-31T15:00:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.967+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/2919486600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab882f0 0x558aeaf5f820 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f81f002f910 tx=0x7f81f0041040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8dac722c327f875 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.975+0000 7f81d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:00:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.975+0000 7f81d6ffd640 1 -- 172.21.15.67:0/2919486600 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0003c70 con 0x558aeab882f0 2024-07-31T15:00:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.975+0000 7f8204999740 1 -- 172.21.15.67:0/2919486600 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81f0003e20 con 0x558aeab882f0 2024-07-31T15:00:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.975+0000 7f81d6ffd640 1 -- 172.21.15.67:0/2919486600 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9515+0+0 (secure 0 0 0) 0x7f81f0004b40 con 0x558aeab882f0 2024-07-31T15:00:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.983+0000 7f81d6ffd640 1 -- 172.21.15.67:0/2919486600 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f81f0004d40 con 0x558aeab882f0 2024-07-31T15:00:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.988+0000 7f8204999740 10 client.27226 fetch_fsmap learned FSMap version 55 2024-07-31T15:00:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.988+0000 7f8204999740 1 -- 172.21.15.67:0/2919486600 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558aeac95620 con 0x558aeab882f0 2024-07-31T15:00:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.993+0000 7f81d6ffd640 1 -- 172.21.15.67:0/2919486600 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f0003e20 con 0x558aeab882f0 2024-07-31T15:00:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.993+0000 7f81d6ffd640 1 -- 172.21.15.67:0/2919486600 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7f81f00043a0 con 0x558aeab882f0 2024-07-31T15:00:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.993+0000 7f8204999740 10 client.27226 fetch_fsmap finished waiting for FSMap version 55 2024-07-31T15:00:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.993+0000 7f8204999740 10 client.27226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:00:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.993+0000 7f8204999740 20 client.27226 populate_metadata read hostname 'smithi067' 2024-07-31T15:00:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.993+0000 7f8204999740 1 --2- 172.21.15.67:0/2919486600 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558aeab6fd80 0x558aeaf71750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.993+0000 7f8204999740 4 client.27226 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:00:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.993+0000 7f8204999740 1 -- 172.21.15.67:0/2919486600 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558aeaecb300 con 0x558aeab6fd80 2024-07-31T15:00:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:14.996+0000 7f81f77fe640 1 --2- 172.21.15.67:0/2919486600 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558aeab6fd80 0x558aeaf71750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.014+0000 7f81f77fe640 1 --2- 172.21.15.67:0/2919486600 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558aeab6fd80 0x558aeaf71750 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccd0a40878432689 server_cookie=4b4cf4d625899bc in_seq=0 out_seq=0 2024-07-31T15:00:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.015+0000 7f81d6ffd640 10 client.27226 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.025+0000 7f81d6ffd640 1 -- 172.21.15.67:0/2919486600 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558aeaecb300 con 0x558aeab6fd80 2024-07-31T15:00:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.025+0000 7f81d6ffd640 10 client.27226 handle_command_reply: tid=0 2024-07-31T15:00:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.025+0000 7f8204999740 1 client.27226 shutdown 2024-07-31T15:00:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.030+0000 7f8204999740 1 -- 172.21.15.67:0/2919486600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab882f0 msgr2=0x558aeaf5f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.030+0000 7f8204999740 1 --2- 172.21.15.67:0/2919486600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab882f0 0x558aeaf5f820 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f81f002f910 tx=0x7f81f0041040 comp rx=0 tx=0).stop 2024-07-31T15:00:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.034+0000 7f8204999740 1 -- 172.21.15.67:0/2919486600 shutdown_connections 2024-07-31T15:00:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.034+0000 7f8204999740 1 --2- 172.21.15.67:0/2919486600 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558aeab6fd80 0x558aeaf71750 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.034+0000 7f8204999740 1 --2- 172.21.15.67:0/2919486600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeaf6b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.034+0000 7f8204999740 1 --2- 172.21.15.67:0/2919486600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab904d0 0x558aeaf5fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.034+0000 7f8204999740 1 --2- 172.21.15.67:0/2919486600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab882f0 0x558aeaf5f820 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.034+0000 7f8204999740 1 -- 172.21.15.67:0/2919486600 >> 172.21.15.67:0/2919486600 conn(0x558aeaba5300 msgr2=0x558aeaf54d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.035+0000 7f81d6ffd640 0 client.27226 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.037+0000 7f8204999740 1 -- 172.21.15.67:0/2919486600 shutdown_connections 2024-07-31T15:00:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.041+0000 7f8204999740 1 -- 172.21.15.67:0/2919486600 wait complete. 2024-07-31T15:00:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.044+0000 7f8204999740 20 client.27226 trim_cache size 0 max 16384 2024-07-31T15:00:15.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:14 smithi067 ceph-mon[32750]: pgmap v681: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 1.05k op/s 2024-07-31T15:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.078+0000 7f8204999740 1 Processor -- start 2024-07-31T15:00:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.081+0000 7f8204999740 1 -- start start 2024-07-31T15:00:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.081+0000 7f8204999740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf5fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.081+0000 7f8204999740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf6b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.081+0000 7f8204999740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeaf6ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.081+0000 7f8204999740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f81e4010020 con 0x558aeabb11b0 2024-07-31T15:00:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.081+0000 7f8204999740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f81e4011cb0 con 0x558aeab882f0 2024-07-31T15:00:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.081+0000 7f8204999740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f81e40106f0 con 0x558aeab904d0 2024-07-31T15:00:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.082+0000 7f81f77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf6b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.082+0000 7f81f77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf6b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58532/0 (socket says 172.21.15.67:58532) 2024-07-31T15:00:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.082+0000 7f81f77fe640 1 -- 172.21.15.67:0/4230152438 learned_addr learned my addr 172.21.15.67:0/4230152438 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:15.088 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.086+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/4230152438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf5fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.088+0000 7f81f77fe640 1 -- 172.21.15.67:0/4230152438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 msgr2=0x558aeaf5fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.088+0000 7f81f77fe640 1 --2- 172.21.15.67:0/4230152438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf5fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.088+0000 7f81f77fe640 1 -- 172.21.15.67:0/4230152438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 msgr2=0x558aeaf6ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.088+0000 7f81f77fe640 1 --2- 172.21.15.67:0/4230152438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeaf6ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.088+0000 7f81f77fe640 1 -- 172.21.15.67:0/4230152438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aeaf5f390 con 0x558aeab904d0 2024-07-31T15:00:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.090+0000 7f81f77fe640 1 --2- 172.21.15.67:0/4230152438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf6b510 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x558aeaf71ec0 tx=0x7f81ec0099c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ddd4546fc6b1476 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.095+0000 7f81dd7fa640 1 -- 172.21.15.67:0/4230152438 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec0165b0 con 0x558aeab904d0 2024-07-31T15:00:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.095+0000 7f81dd7fa640 1 -- 172.21.15.67:0/4230152438 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ec022070 con 0x558aeab904d0 2024-07-31T15:00:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.095+0000 7f81dd7fa640 1 -- 172.21.15.67:0/4230152438 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec014b40 con 0x558aeab904d0 2024-07-31T15:00:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.096+0000 7f8204999740 1 -- 172.21.15.67:0/4230152438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 msgr2=0x558aeaf6b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.096+0000 7f8204999740 1 --2- 172.21.15.67:0/4230152438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf6b510 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x558aeaf71ec0 tx=0x7f81ec0099c0 comp rx=0 tx=0).stop 2024-07-31T15:00:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.099+0000 7f8204999740 1 -- 172.21.15.67:0/4230152438 shutdown_connections 2024-07-31T15:00:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.099+0000 7f8204999740 1 --2- 172.21.15.67:0/4230152438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeaf6ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.099+0000 7f8204999740 1 --2- 172.21.15.67:0/4230152438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf6b510 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.099+0000 7f8204999740 1 --2- 172.21.15.67:0/4230152438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf5fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.099+0000 7f8204999740 1 -- 172.21.15.67:0/4230152438 >> 172.21.15.67:0/4230152438 conn(0x558aeaba5300 msgr2=0x558aeaf5f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.106+0000 7f8204999740 1 -- 172.21.15.67:0/4230152438 shutdown_connections 2024-07-31T15:00:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.111+0000 7f8204999740 1 -- 172.21.15.67:0/4230152438 wait complete. 2024-07-31T15:00:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.111+0000 7f8204999740 1 -- start start 2024-07-31T15:00:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.111+0000 7f8204999740 1 Processor -- start 2024-07-31T15:00:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.112+0000 7f8204999740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf63f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.112+0000 7f8204999740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.112+0000 7f8204999740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeafd0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.112+0000 7f8204999740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f81e40115e0 con 0x558aeabb11b0 2024-07-31T15:00:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.112+0000 7f8204999740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f81e4010e50 con 0x558aeab882f0 2024-07-31T15:00:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.112+0000 7f8204999740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f81e400e020 con 0x558aeab904d0 2024-07-31T15:00:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.112+0000 7f8204999740 10 client.0 _ll_register_callbacks cb 0x7f81f002fac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:00:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.113+0000 7f81f6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf63f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.113+0000 7f81f6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf63f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46470/0 (socket says 172.21.15.67:46470) 2024-07-31T15:00:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.113+0000 7f81f6ffd640 1 -- 172.21.15.67:0/3874603265 learned_addr learned my addr 172.21.15.67:0/3874603265 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.113+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/3874603265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeafd0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.113+0000 7f81f77fe640 1 --2- 172.21.15.67:0/3874603265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.118+0000 7f81f6ffd640 1 -- 172.21.15.67:0/3874603265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 msgr2=0x558aeaf644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.118+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/3874603265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.118+0000 7f81f6ffd640 1 -- 172.21.15.67:0/3874603265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 msgr2=0x558aeafd0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.118+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/3874603265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeafd0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.118+0000 7f81f6ffd640 1 -- 172.21.15.67:0/3874603265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aeafd1550 con 0x558aeab882f0 2024-07-31T15:00:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.120+0000 7f81f77fe640 1 --2- 172.21.15.67:0/3874603265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:00:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.122+0000 7f81f6ffd640 1 --2- 172.21.15.67:0/3874603265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf63f00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f81f0002230 tx=0x7f81f0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4f7da9124812e53 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.122+0000 7f81d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:00:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.123+0000 7f81d6ffd640 1 -- 172.21.15.67:0/3874603265 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0045040 con 0x558aeab882f0 2024-07-31T15:00:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.123+0000 7f81d6ffd640 1 -- 172.21.15.67:0/3874603265 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9515+0+0 (secure 0 0 0) 0x7f81f0004b40 con 0x558aeab882f0 2024-07-31T15:00:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.124+0000 7f8204999740 1 -- 172.21.15.67:0/3874603265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81ec022070 con 0x558aeab882f0 2024-07-31T15:00:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.125+0000 7f81d6ffd640 1 -- 172.21.15.67:0/3874603265 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f81f0034040 con 0x558aeab882f0 2024-07-31T15:00:15.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.129+0000 7f8204999740 10 client.27239 fetch_fsmap learned FSMap version 55 2024-07-31T15:00:15.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.129+0000 7f8204999740 1 -- 172.21.15.67:0/3874603265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558aeafd1550 con 0x558aeab882f0 2024-07-31T15:00:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.133+0000 7f81d6ffd640 1 -- 172.21.15.67:0/3874603265 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ec022070 con 0x558aeab882f0 2024-07-31T15:00:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.133+0000 7f81d6ffd640 1 -- 172.21.15.67:0/3874603265 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7f81f0003c50 con 0x558aeab882f0 2024-07-31T15:00:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.133+0000 7f8204999740 10 client.27239 fetch_fsmap finished waiting for FSMap version 55 2024-07-31T15:00:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.133+0000 7f8204999740 10 client.27239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:00:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.133+0000 7f8204999740 20 client.27239 populate_metadata read hostname 'smithi067' 2024-07-31T15:00:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.133+0000 7f8204999740 1 --2- 172.21.15.67:0/3874603265 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558aeab451c0 0x558aeaffcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.133+0000 7f8204999740 4 client.27239 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:00:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.133+0000 7f8204999740 1 -- 172.21.15.67:0/3874603265 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558aeaec6da0 con 0x558aeab451c0 2024-07-31T15:00:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.137+0000 7f81f77fe640 1 --2- 172.21.15.67:0/3874603265 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558aeab451c0 0x558aeaffcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:15.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:14 smithi073 ceph-mon[36714]: pgmap v681: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 1.05k op/s 2024-07-31T15:00:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.152+0000 7f81f77fe640 1 --2- 172.21.15.67:0/3874603265 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558aeab451c0 0x558aeaffcc50 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f4f156ac4d2b62b server_cookie=3c80a70552a8d973 in_seq=0 out_seq=0 2024-07-31T15:00:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:15.153+0000 7f81d6ffd640 10 client.27239 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:14 smithi077 ceph-mon[37327]: pgmap v681: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 1.05k op/s 2024-07-31T15:00:17.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:16 smithi067 ceph-mon[32750]: pgmap v682: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 929 KiB/s wr, 586 op/s 2024-07-31T15:00:17.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:16 smithi073 ceph-mon[36714]: pgmap v682: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 929 KiB/s wr, 586 op/s 2024-07-31T15:00:17.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:16 smithi077 ceph-mon[37327]: pgmap v682: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 929 KiB/s wr, 586 op/s 2024-07-31T15:00:17.702 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:00:18.363 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:00:18.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:18 smithi067 ceph-mon[32750]: pgmap v683: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 749 op/s 2024-07-31T15:00:19.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:18 smithi073 ceph-mon[36714]: pgmap v683: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 749 op/s 2024-07-31T15:00:19.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:18 smithi077 ceph-mon[37327]: pgmap v683: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 749 op/s 2024-07-31T15:00:20.053 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 15:00:19 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T15:00:19.677+0000 7f4cfe9c3640 -1 calc_pg_upmaps abort due to max <= 0 2024-07-31T15:00:21.302 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:21 smithi073 ceph-mon[36714]: pgmap v684: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 647 op/s 2024-07-31T15:00:21.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:21 smithi067 ceph-mon[32750]: pgmap v684: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 647 op/s 2024-07-31T15:00:21.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:21 smithi077 ceph-mon[37327]: pgmap v684: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 647 op/s 2024-07-31T15:00:21.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.501+0000 7f8a52ea8640 1 -- 172.21.15.67:0/3939791399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a4c10f500 msgr2=0x7f8a4c10f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:21.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.501+0000 7f8a52ea8640 1 --2- 172.21.15.67:0/3939791399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a4c10f500 0x7f8a4c10f910 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009900 tx=0x7f8a3c02a440 comp rx=0 tx=0).stop 2024-07-31T15:00:21.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.503+0000 7f8a52ea8640 1 -- 172.21.15.67:0/3939791399 shutdown_connections 2024-07-31T15:00:21.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.503+0000 7f8a52ea8640 1 --2- 172.21.15.67:0/3939791399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a4c10fe50 0x7f8a4c116680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.503+0000 7f8a52ea8640 1 --2- 172.21.15.67:0/3939791399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a4c10f500 0x7f8a4c10f910 secure :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009900 tx=0x7f8a3c02a440 comp rx=0 tx=0).stop 2024-07-31T15:00:21.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.503+0000 7f8a52ea8640 1 --2- 172.21.15.67:0/3939791399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a4c109bc0 0x7f8a4c109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.503+0000 7f8a52ea8640 1 -- 172.21.15.67:0/3939791399 >> 172.21.15.67:0/3939791399 conn(0x7f8a4c077b20 msgr2=0x7f8a4c107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:21.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.513+0000 7f8a52ea8640 1 -- 172.21.15.67:0/3939791399 shutdown_connections 2024-07-31T15:00:21.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.513+0000 7f8a52ea8640 1 -- 172.21.15.67:0/3939791399 wait complete. 2024-07-31T15:00:21.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.515+0000 7f8a52ea8640 1 Processor -- start 2024-07-31T15:00:21.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.516+0000 7f8a52ea8640 1 -- start start 2024-07-31T15:00:21.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.516+0000 7f8a52ea8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a4c109bc0 0x7f8a4c1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.516+0000 7f8a52ea8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a4c10fe50 0x7f8a4c1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.516+0000 7f8a52ea8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a4c1a5f70 0x7f8a4c1a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.516+0000 7f8a52ea8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c119c40 con 0x7f8a4c109bc0 2024-07-31T15:00:21.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.516+0000 7f8a52ea8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c119ac0 con 0x7f8a4c1a5f70 2024-07-31T15:00:21.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.516+0000 7f8a52ea8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c119dc0 con 0x7f8a4c10fe50 2024-07-31T15:00:21.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.518+0000 7f8a51445640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a4c1a5f70 0x7f8a4c1a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:21.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.519+0000 7f8a4bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a4c10fe50 0x7f8a4c1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:21.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.520+0000 7f8a51445640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a4c1a5f70 0x7f8a4c1a6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49808/0 (socket says 172.21.15.67:49808) 2024-07-31T15:00:21.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.520+0000 7f8a51445640 1 -- 172.21.15.67:0/763112284 learned_addr learned my addr 172.21.15.67:0/763112284 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:21.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.524+0000 7f8a51445640 1 -- 172.21.15.67:0/763112284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a4c10fe50 msgr2=0x7f8a4c1a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:21.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.524+0000 7f8a51445640 1 --2- 172.21.15.67:0/763112284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a4c10fe50 0x7f8a4c1a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.524+0000 7f8a51445640 1 -- 172.21.15.67:0/763112284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a4c109bc0 msgr2=0x7f8a4c1a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:21.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.524+0000 7f8a51445640 1 --2- 172.21.15.67:0/763112284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a4c109bc0 0x7f8a4c1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.524+0000 7f8a51445640 1 -- 172.21.15.67:0/763112284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4c1a6b50 con 0x7f8a4c1a5f70 2024-07-31T15:00:21.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.529+0000 7f8a51445640 1 --2- 172.21.15.67:0/763112284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a4c1a5f70 0x7f8a4c1a6380 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000b3f0 tx=0x7f8a4000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:21.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.531+0000 7f8a49ffb640 1 -- 172.21.15.67:0/763112284 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40015020 con 0x7f8a4c1a5f70 2024-07-31T15:00:21.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.531+0000 7f8a49ffb640 1 -- 172.21.15.67:0/763112284 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a40011ce0 con 0x7f8a4c1a5f70 2024-07-31T15:00:21.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.531+0000 7f8a49ffb640 1 -- 172.21.15.67:0/763112284 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40010da0 con 0x7f8a4c1a5f70 2024-07-31T15:00:21.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.532+0000 7f8a52ea8640 1 -- 172.21.15.67:0/763112284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4c06c810 con 0x7f8a4c1a5f70 2024-07-31T15:00:21.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.532+0000 7f8a52ea8640 1 -- 172.21.15.67:0/763112284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a4c06ccf0 con 0x7f8a4c1a5f70 2024-07-31T15:00:21.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.543+0000 7f8a49ffb640 1 -- 172.21.15.67:0/763112284 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8a40018020 con 0x7f8a4c1a5f70 2024-07-31T15:00:21.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.544+0000 7f8a49ffb640 1 --2- 172.21.15.67:0/763112284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a340b3f50 0x7f8a340b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.544+0000 7f8a49ffb640 1 -- 172.21.15.67:0/763112284 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9515+0+0 (secure 0 0 0) 0x7f8a400e3be0 con 0x7f8a4c1a5f70 2024-07-31T15:00:21.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.544+0000 7f8a593f9740 1 Processor -- start 2024-07-31T15:00:21.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.548+0000 7f8a50c44640 1 --2- 172.21.15.67:0/763112284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a340b3f50 0x7f8a340b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:21.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.554+0000 7f8a593f9740 1 -- start start 2024-07-31T15:00:21.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.554+0000 7f8a593f9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d50070 0x5650570f5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.554+0000 7f8a593f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d271b0 0x5650570f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.554+0000 7f8a593f9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d2f390 0x5650570f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.554+0000 7f8a593f9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56505706bb60 con 0x565056d50070 2024-07-31T15:00:21.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.554+0000 7f8a593f9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56505705b060 con 0x565056d2f390 2024-07-31T15:00:21.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.554+0000 7f8a593f9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56505705f0a0 con 0x565056d271b0 2024-07-31T15:00:21.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.555+0000 7f8a51445640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d2f390 0x5650570f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:21.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.555+0000 7f8a51445640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d2f390 0x5650570f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49810/0 (socket says 172.21.15.67:49810) 2024-07-31T15:00:21.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.555+0000 7f8a51445640 1 -- 172.21.15.67:0/1769428716 learned_addr learned my addr 172.21.15.67:0/1769428716 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:21.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.556+0000 7f8a51445640 1 -- 172.21.15.67:0/1769428716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d271b0 msgr2=0x5650570f5b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:00:21.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.556+0000 7f8a51445640 1 --2- 172.21.15.67:0/1769428716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d271b0 0x5650570f5b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.556+0000 7f8a51445640 1 -- 172.21.15.67:0/1769428716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d50070 msgr2=0x5650570f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:21.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.556+0000 7f8a51445640 1 --2- 172.21.15.67:0/1769428716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d50070 0x5650570f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.556+0000 7f8a51445640 1 -- 172.21.15.67:0/1769428716 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565057001b40 con 0x565056d2f390 2024-07-31T15:00:21.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.556+0000 7f8a51445640 1 --2- 172.21.15.67:0/1769428716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d2f390 0x5650570f60c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8a400a6540 tx=0x7f8a400a9d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=414101a820a4bc18 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:21.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.557+0000 7f8a50c44640 1 --2- 172.21.15.67:0/763112284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a340b3f50 0x7f8a340b6460 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400ada0 tx=0x7f8a4400a690 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:21.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.559+0000 7f8a32ffd640 1 -- 172.21.15.67:0/1769428716 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a400aa4c0 con 0x565056d2f390 2024-07-31T15:00:21.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.559+0000 7f8a32ffd640 1 -- 172.21.15.67:0/1769428716 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a400aeac0 con 0x565056d2f390 2024-07-31T15:00:21.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.559+0000 7f8a32ffd640 1 -- 172.21.15.67:0/1769428716 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a400b3340 con 0x565056d2f390 2024-07-31T15:00:21.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.560+0000 7f8a593f9740 1 -- 172.21.15.67:0/1769428716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d2f390 msgr2=0x5650570f60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:21.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.560+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1769428716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d2f390 0x5650570f60c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8a400a6540 tx=0x7f8a400a9d60 comp rx=0 tx=0).stop 2024-07-31T15:00:21.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.563+0000 7f8a593f9740 1 -- 172.21.15.67:0/1769428716 shutdown_connections 2024-07-31T15:00:21.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.563+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1769428716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d2f390 0x5650570f60c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.563+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1769428716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d271b0 0x5650570f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.563+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1769428716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d50070 0x5650570f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.563+0000 7f8a593f9740 1 -- 172.21.15.67:0/1769428716 >> 172.21.15.67:0/1769428716 conn(0x565056d44460 msgr2=0x5650570f3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:21.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.570+0000 7f8a593f9740 1 -- 172.21.15.67:0/1769428716 shutdown_connections 2024-07-31T15:00:21.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.572+0000 7f8a593f9740 1 -- 172.21.15.67:0/1769428716 wait complete. 2024-07-31T15:00:21.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.572+0000 7f8a593f9740 1 -- start start 2024-07-31T15:00:21.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.572+0000 7f8a593f9740 1 Processor -- start 2024-07-31T15:00:21.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.579+0000 7f8a593f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d2f390 0x5650571011f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.579+0000 7f8a593f9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d50070 0x565057101730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.579+0000 7f8a593f9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d0e9f0 0x56505710d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.579+0000 7f8a593f9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x565056ebe700 con 0x565056d0e9f0 2024-07-31T15:00:21.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.579+0000 7f8a593f9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x565056ffd990 con 0x565056d50070 2024-07-31T15:00:21.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.579+0000 7f8a593f9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565057103e70 con 0x565056d2f390 2024-07-31T15:00:21.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.579+0000 7f8a593f9740 10 client.0 _ll_register_callbacks cb 0x5650570184a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:00:21.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.580+0000 7f8a4bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d50070 0x565057101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:21.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.580+0000 7f8a4bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d50070 0x565057101730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49816/0 (socket says 172.21.15.67:49816) 2024-07-31T15:00:21.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.580+0000 7f8a4bfff640 1 -- 172.21.15.67:0/439131071 learned_addr learned my addr 172.21.15.67:0/439131071 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:21.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.584+0000 7f8a4bfff640 1 -- 172.21.15.67:0/439131071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d2f390 msgr2=0x5650571011f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:00:21.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.584+0000 7f8a4bfff640 1 --2- 172.21.15.67:0/439131071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d2f390 0x5650571011f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.584+0000 7f8a4bfff640 1 -- 172.21.15.67:0/439131071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d0e9f0 msgr2=0x56505710d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:21.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.584+0000 7f8a4bfff640 1 --2- 172.21.15.67:0/439131071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d0e9f0 0x56505710d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.584+0000 7f8a4bfff640 1 -- 172.21.15.67:0/439131071 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565056dbb650 con 0x565056d50070 2024-07-31T15:00:21.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.588+0000 7f8a4bfff640 1 --2- 172.21.15.67:0/439131071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d50070 0x565057101730 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c02a9b0 tx=0x7f8a3c004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4f6e31e2599ce14 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:21.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.588+0000 7f8a2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:00:21.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.591+0000 7f8a2bfff640 1 -- 172.21.15.67:0/439131071 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c03f070 con 0x565056d50070 2024-07-31T15:00:21.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.591+0000 7f8a593f9740 1 -- 172.21.15.67:0/439131071 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a400aeac0 con 0x565056d50070 2024-07-31T15:00:21.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.591+0000 7f8a2bfff640 1 -- 172.21.15.67:0/439131071 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9515+0+0 (secure 0 0 0) 0x7f8a3c003d30 con 0x565056d50070 2024-07-31T15:00:21.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.598+0000 7f8a2bfff640 1 -- 172.21.15.67:0/439131071 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8a3c035070 con 0x565056d50070 2024-07-31T15:00:21.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.599+0000 7f8a593f9740 10 client.27248 fetch_fsmap learned FSMap version 55 2024-07-31T15:00:21.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.599+0000 7f8a593f9740 1 -- 172.21.15.67:0/439131071 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56505710fed0 con 0x565056d50070 2024-07-31T15:00:21.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.603+0000 7f8a2bfff640 1 -- 172.21.15.67:0/439131071 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a400aeac0 con 0x565056d50070 2024-07-31T15:00:21.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.603+0000 7f8a2bfff640 1 -- 172.21.15.67:0/439131071 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7f8a3c03b4d0 con 0x565056d50070 2024-07-31T15:00:21.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.603+0000 7f8a593f9740 10 client.27248 fetch_fsmap finished waiting for FSMap version 55 2024-07-31T15:00:21.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.603+0000 7f8a593f9740 10 client.27248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:00:21.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.603+0000 7f8a593f9740 20 client.27248 populate_metadata read hostname 'smithi067' 2024-07-31T15:00:21.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.603+0000 7f8a593f9740 1 --2- 172.21.15.67:0/439131071 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x565056ce3da0 0x565057113990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.603+0000 7f8a593f9740 4 client.27248 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:00:21.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.603+0000 7f8a593f9740 1 -- 172.21.15.67:0/439131071 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56505706dab0 con 0x565056ce3da0 2024-07-31T15:00:21.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.607+0000 7f8a50c44640 1 --2- 172.21.15.67:0/439131071 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x565056ce3da0 0x565057113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:21.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.623+0000 7f8a50c44640 1 --2- 172.21.15.67:0/439131071 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x565056ce3da0 0x565057113990 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac86756cad2d50e server_cookie=750f820dc60a825e in_seq=0 out_seq=0 2024-07-31T15:00:21.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.624+0000 7f8a2bfff640 10 client.27248 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:21.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.637+0000 7f8a2bfff640 1 -- 172.21.15.67:0/439131071 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56505706dab0 con 0x565056ce3da0 2024-07-31T15:00:21.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.637+0000 7f8a2bfff640 10 client.27248 handle_command_reply: tid=0 2024-07-31T15:00:21.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.637+0000 7f8a593f9740 1 client.27248 shutdown 2024-07-31T15:00:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.638+0000 7f8a593f9740 1 -- 172.21.15.67:0/439131071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d50070 msgr2=0x565057101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.638+0000 7f8a593f9740 1 --2- 172.21.15.67:0/439131071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d50070 0x565057101730 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c02a9b0 tx=0x7f8a3c004610 comp rx=0 tx=0).stop 2024-07-31T15:00:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.638+0000 7f8a593f9740 1 -- 172.21.15.67:0/439131071 shutdown_connections 2024-07-31T15:00:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.638+0000 7f8a593f9740 1 --2- 172.21.15.67:0/439131071 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x565056ce3da0 0x565057113990 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.638+0000 7f8a593f9740 1 --2- 172.21.15.67:0/439131071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d0e9f0 0x56505710d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.638+0000 7f8a593f9740 1 --2- 172.21.15.67:0/439131071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d50070 0x565057101730 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.638+0000 7f8a593f9740 1 --2- 172.21.15.67:0/439131071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d2f390 0x5650571011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.638+0000 7f8a593f9740 1 -- 172.21.15.67:0/439131071 >> 172.21.15.67:0/439131071 conn(0x565056d44460 msgr2=0x5650570f1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.638+0000 7f8a2bfff640 0 client.27248 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:21.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.638+0000 7f8a593f9740 1 -- 172.21.15.67:0/439131071 shutdown_connections 2024-07-31T15:00:21.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.638+0000 7f8a593f9740 1 -- 172.21.15.67:0/439131071 wait complete. 2024-07-31T15:00:21.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.638+0000 7f8a593f9740 20 client.27248 trim_cache size 0 max 16384 2024-07-31T15:00:21.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.678+0000 7f8a593f9740 1 Processor -- start 2024-07-31T15:00:21.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.679+0000 7f8a593f9740 1 -- start start 2024-07-31T15:00:21.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.679+0000 7f8a593f9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d0e9f0 0x5650570fae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.679+0000 7f8a593f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d2f390 0x56505710cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.680+0000 7f8a593f9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d50070 0x5650570f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.680+0000 7f8a4bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d2f390 0x56505710cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.680+0000 7f8a4bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d2f390 0x56505710cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38684/0 (socket says 172.21.15.67:38684) 2024-07-31T15:00:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.680+0000 7f8a4bfff640 1 -- 172.21.15.67:0/1359785985 learned_addr learned my addr 172.21.15.67:0/1359785985 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.680+0000 7f8a593f9740 1 -- 172.21.15.67:0/1359785985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1bfee0 con 0x565056d0e9f0 2024-07-31T15:00:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.680+0000 7f8a593f9740 1 -- 172.21.15.67:0/1359785985 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1c1b70 con 0x565056d50070 2024-07-31T15:00:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.680+0000 7f8a51445640 1 --2- 172.21.15.67:0/1359785985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d50070 0x5650570f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.680+0000 7f8a50c44640 1 --2- 172.21.15.67:0/1359785985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d0e9f0 0x5650570fae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.680+0000 7f8a593f9740 1 -- 172.21.15.67:0/1359785985 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1c05b0 con 0x565056d2f390 2024-07-31T15:00:21.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.681+0000 7f8a4bfff640 1 -- 172.21.15.67:0/1359785985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d50070 msgr2=0x5650570f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:21.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.681+0000 7f8a4bfff640 1 --2- 172.21.15.67:0/1359785985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d50070 0x5650570f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.681+0000 7f8a4bfff640 1 -- 172.21.15.67:0/1359785985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d0e9f0 msgr2=0x5650570fae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:21.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.681+0000 7f8a4bfff640 1 --2- 172.21.15.67:0/1359785985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d0e9f0 0x5650570fae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.681+0000 7f8a4bfff640 1 -- 172.21.15.67:0/1359785985 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650570eef50 con 0x565056d2f390 2024-07-31T15:00:21.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.688+0000 7f8a4bfff640 1 --2- 172.21.15.67:0/1359785985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d2f390 0x56505710cfa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009850 tx=0x7f8a3c004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c5ceb1c1f2de75 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:21.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.691+0000 7f8a327fc640 1 -- 172.21.15.67:0/1359785985 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c03f070 con 0x565056d2f390 2024-07-31T15:00:21.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.691+0000 7f8a327fc640 1 -- 172.21.15.67:0/1359785985 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a3c03bac0 con 0x565056d2f390 2024-07-31T15:00:21.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.691+0000 7f8a327fc640 1 -- 172.21.15.67:0/1359785985 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0054e0 con 0x565056d2f390 2024-07-31T15:00:21.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.693+0000 7f8a593f9740 1 -- 172.21.15.67:0/1359785985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d2f390 msgr2=0x56505710cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:21.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.693+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1359785985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d2f390 0x56505710cfa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009850 tx=0x7f8a3c004510 comp rx=0 tx=0).stop 2024-07-31T15:00:21.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.694+0000 7f8a593f9740 1 -- 172.21.15.67:0/1359785985 shutdown_connections 2024-07-31T15:00:21.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.694+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1359785985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d50070 0x5650570f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.694+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1359785985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565056d2f390 0x56505710cfa0 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009850 tx=0x7f8a3c004510 comp rx=0 tx=0).stop 2024-07-31T15:00:21.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.694+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1359785985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d0e9f0 0x5650570fae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.694+0000 7f8a593f9740 1 -- 172.21.15.67:0/1359785985 >> 172.21.15.67:0/1359785985 conn(0x565056d44460 msgr2=0x5650570fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:21.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.696+0000 7f8a593f9740 1 -- 172.21.15.67:0/1359785985 shutdown_connections 2024-07-31T15:00:21.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.696+0000 7f8a593f9740 1 -- 172.21.15.67:0/1359785985 wait complete. 2024-07-31T15:00:21.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.697+0000 7f8a593f9740 1 -- start start 2024-07-31T15:00:21.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.697+0000 7f8a593f9740 1 Processor -- start 2024-07-31T15:00:21.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.698+0000 7f8a593f9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d0e9f0 0x565057109a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.698+0000 7f8a593f9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d50070 0x565057104f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.698+0000 7f8a50c44640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d0e9f0 0x565057109a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.698+0000 7f8a50c44640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d0e9f0 0x565057109a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49842/0 (socket says 172.21.15.67:49842) 2024-07-31T15:00:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.698+0000 7f8a50c44640 1 -- 172.21.15.67:0/1671995747 learned_addr learned my addr 172.21.15.67:0/1671995747 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.698+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1671995747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650571055b0 0x56505717b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.699+0000 7f8a593f9740 1 -- 172.21.15.67:0/1671995747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1c14a0 con 0x565056d50070 2024-07-31T15:00:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.699+0000 7f8a593f9740 1 -- 172.21.15.67:0/1671995747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1c0d10 con 0x565056d0e9f0 2024-07-31T15:00:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.699+0000 7f8a51445640 1 --2- 172.21.15.67:0/1671995747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650571055b0 0x56505717b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.699+0000 7f8a593f9740 1 -- 172.21.15.67:0/1671995747 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c074df0 con 0x5650571055b0 2024-07-31T15:00:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.699+0000 7f8a4bfff640 1 --2- 172.21.15.67:0/1671995747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d50070 0x565057104f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.699+0000 7f8a593f9740 10 client.0 _ll_register_callbacks cb 0x7f8a3c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:00:21.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.703+0000 7f8a50c44640 1 -- 172.21.15.67:0/1671995747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650571055b0 msgr2=0x56505717b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:21.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.703+0000 7f8a50c44640 1 --2- 172.21.15.67:0/1671995747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650571055b0 0x56505717b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.703+0000 7f8a50c44640 1 -- 172.21.15.67:0/1671995747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d50070 msgr2=0x565057104f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:21.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.703+0000 7f8a50c44640 1 --2- 172.21.15.67:0/1671995747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d50070 0x565057104f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.703+0000 7f8a50c44640 1 -- 172.21.15.67:0/1671995747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565057105cd0 con 0x565056d0e9f0 2024-07-31T15:00:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.705+0000 7f8a50c44640 1 --2- 172.21.15.67:0/1671995747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d0e9f0 0x565057109a30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400ecb0 tx=0x7f8a44004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=665f3fc055295ec0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.705+0000 7f8a2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:00:21.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.710+0000 7f8a2bfff640 1 -- 172.21.15.67:0/1671995747 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4402a070 con 0x565056d0e9f0 2024-07-31T15:00:21.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.710+0000 7f8a593f9740 1 -- 172.21.15.67:0/1671995747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a3c03bac0 con 0x565056d0e9f0 2024-07-31T15:00:21.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.710+0000 7f8a2bfff640 1 -- 172.21.15.67:0/1671995747 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9515+0+0 (secure 0 0 0) 0x7f8a4400b830 con 0x565056d0e9f0 2024-07-31T15:00:21.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.710+0000 7f8a2bfff640 1 -- 172.21.15.67:0/1671995747 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8a44036050 con 0x565056d0e9f0 2024-07-31T15:00:21.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.714+0000 7f8a593f9740 10 client.27254 fetch_fsmap learned FSMap version 55 2024-07-31T15:00:21.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.714+0000 7f8a593f9740 1 -- 172.21.15.67:0/1671995747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56505717c800 con 0x565056d0e9f0 2024-07-31T15:00:21.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.718+0000 7f8a2bfff640 1 -- 172.21.15.67:0/1671995747 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a3c03bac0 con 0x565056d0e9f0 2024-07-31T15:00:21.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.718+0000 7f8a2bfff640 1 -- 172.21.15.67:0/1671995747 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7f8a44025070 con 0x565056d0e9f0 2024-07-31T15:00:21.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.718+0000 7f8a593f9740 10 client.27254 fetch_fsmap finished waiting for FSMap version 55 2024-07-31T15:00:21.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.718+0000 7f8a593f9740 10 client.27254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:00:21.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.718+0000 7f8a593f9740 20 client.27254 populate_metadata read hostname 'smithi067' 2024-07-31T15:00:21.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.718+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1671995747 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56505717d0a0 0x56505717f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:21.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.718+0000 7f8a593f9740 4 client.27254 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:00:21.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.718+0000 7f8a593f9740 1 -- 172.21.15.67:0/1671995747 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565057069550 con 0x56505717d0a0 2024-07-31T15:00:21.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.726+0000 7f8a4bfff640 1 --2- 172.21.15.67:0/1671995747 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56505717d0a0 0x56505717f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.729+0000 7f8a4bfff640 1 --2- 172.21.15.67:0/1671995747 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56505717d0a0 0x56505717f4c0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb5f00bc46ef03e server_cookie=27c771c78ee0a645 in_seq=0 out_seq=0 2024-07-31T15:00:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.729+0000 7f8a2bfff640 10 client.27254 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:21.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.732+0000 7f8a2bfff640 1 -- 172.21.15.67:0/1671995747 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x565057069550 con 0x56505717d0a0 2024-07-31T15:00:21.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.732+0000 7f8a2bfff640 10 client.27254 handle_command_reply: tid=0 2024-07-31T15:00:21.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.732+0000 7f8a593f9740 1 client.27254 shutdown 2024-07-31T15:00:21.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.732+0000 7f8a593f9740 1 -- 172.21.15.67:0/1671995747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d0e9f0 msgr2=0x565057109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:21.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.732+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1671995747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d0e9f0 0x565057109a30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400ecb0 tx=0x7f8a44004b40 comp rx=0 tx=0).stop 2024-07-31T15:00:21.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.732+0000 7f8a593f9740 1 -- 172.21.15.67:0/1671995747 shutdown_connections 2024-07-31T15:00:21.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.732+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1671995747 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56505717d0a0 0x56505717f4c0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.732+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1671995747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650571055b0 0x56505717b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.732+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1671995747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565056d50070 0x565057104f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.732+0000 7f8a593f9740 1 --2- 172.21.15.67:0/1671995747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565056d0e9f0 0x565057109a30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.732+0000 7f8a593f9740 1 -- 172.21.15.67:0/1671995747 >> 172.21.15.67:0/1671995747 conn(0x565056d44460 msgr2=0x5650570f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:21.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.732+0000 7f8a2bfff640 0 client.27254 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.732+0000 7f8a593f9740 1 -- 172.21.15.67:0/1671995747 shutdown_connections 2024-07-31T15:00:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.733+0000 7f8a593f9740 1 -- 172.21.15.67:0/1671995747 wait complete. 2024-07-31T15:00:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.733+0000 7f8a593f9740 20 client.27254 trim_cache size 0 max 16384 2024-07-31T15:00:21.736 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:00:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (9928 inodes in the stack)", 2024-07-31T15:00:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T15:00:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "c791f779-17a0-47ec-8943-a777d1eb6ea8": { 2024-07-31T15:00:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T15:00:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "c791f779-17a0-47ec-8943-a777d1eb6ea8", 2024-07-31T15:00:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T15:00:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:00:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:00:21.737 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:00:21.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.734+0000 7f8a52ea8640 1 -- 172.21.15.67:0/763112284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a340b3f50 msgr2=0x7f8a340b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:21.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.734+0000 7f8a52ea8640 1 --2- 172.21.15.67:0/763112284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a340b3f50 0x7f8a340b6460 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400ada0 tx=0x7f8a4400a690 comp rx=0 tx=0).stop 2024-07-31T15:00:21.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.734+0000 7f8a52ea8640 1 -- 172.21.15.67:0/763112284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a4c1a5f70 msgr2=0x7f8a4c1a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:21.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.734+0000 7f8a52ea8640 1 --2- 172.21.15.67:0/763112284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a4c1a5f70 0x7f8a4c1a6380 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000b3f0 tx=0x7f8a4000b8b0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.734+0000 7f8a52ea8640 1 -- 172.21.15.67:0/763112284 shutdown_connections 2024-07-31T15:00:21.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.734+0000 7f8a52ea8640 1 --2- 172.21.15.67:0/763112284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a340b3f50 0x7f8a340b6460 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.735+0000 7f8a52ea8640 1 --2- 172.21.15.67:0/763112284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a4c1a5f70 0x7f8a4c1a6380 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.735+0000 7f8a52ea8640 1 --2- 172.21.15.67:0/763112284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a4c10fe50 0x7f8a4c1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.735+0000 7f8a52ea8640 1 --2- 172.21.15.67:0/763112284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a4c109bc0 0x7f8a4c1a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:21.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.735+0000 7f8a52ea8640 1 -- 172.21.15.67:0/763112284 >> 172.21.15.67:0/763112284 conn(0x7f8a4c077b20 msgr2=0x7f8a4c107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:21.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.739+0000 7f8a52ea8640 1 -- 172.21.15.67:0/763112284 shutdown_connections 2024-07-31T15:00:21.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:21.739+0000 7f8a52ea8640 1 -- 172.21.15.67:0/763112284 wait complete. 2024-07-31T15:00:22.081 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/basic/fixdep 2024-07-31T15:00:23.108 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/conf.o 2024-07-31T15:00:23.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:23 smithi067 ceph-mon[32750]: pgmap v685: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 1.07k op/s 2024-07-31T15:00:23.309 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:23 smithi077 ceph-mon[37327]: pgmap v685: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 1.07k op/s 2024-07-31T15:00:23.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:23 smithi073 ceph-mon[36714]: pgmap v685: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 1.07k op/s 2024-07-31T15:00:23.499 INFO:tasks.cephfs.filesystem:scrub status for tag:c791f779-17a0-47ec-8943-a777d1eb6ea8 - {'path': '/', 'tag': 'c791f779-17a0-47ec-8943-a777d1eb6ea8', 'options': 'recursive,force'} 2024-07-31T15:00:25.237 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/confdata.o 2024-07-31T15:00:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:25 smithi067 ceph-mon[32750]: pgmap v686: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 3.0 MiB/s wr, 1.24k op/s 2024-07-31T15:00:25.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:25 smithi073 ceph-mon[36714]: pgmap v686: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 3.0 MiB/s wr, 1.24k op/s 2024-07-31T15:00:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:25 smithi077 ceph-mon[37327]: pgmap v686: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 3.0 MiB/s wr, 1.24k op/s 2024-07-31T15:00:25.705 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/expr.o 2024-07-31T15:00:26.155 INFO:tasks.workunit.client.0.smithi067.stdout: LEX scripts/kconfig/lexer.lex.c 2024-07-31T15:00:26.241 INFO:tasks.workunit.client.0.smithi067.stdout: YACC scripts/kconfig/parser.tab.[ch] 2024-07-31T15:00:26.412 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/lexer.lex.o 2024-07-31T15:00:26.916 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/menu.o 2024-07-31T15:00:27.269 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/parser.tab.o 2024-07-31T15:00:27.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:27 smithi067 ceph-mon[32750]: pgmap v687: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.1 MiB/s wr, 1.31k op/s 2024-07-31T15:00:27.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:27 smithi073 ceph-mon[36714]: pgmap v687: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.1 MiB/s wr, 1.31k op/s 2024-07-31T15:00:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:27 smithi077 ceph-mon[37327]: pgmap v687: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.1 MiB/s wr, 1.31k op/s 2024-07-31T15:00:27.690 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/preprocess.o 2024-07-31T15:00:27.921 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/symbol.o 2024-07-31T15:00:28.444 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/util.o 2024-07-31T15:00:28.596 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTLD scripts/kconfig/conf 2024-07-31T15:00:28.678 INFO:tasks.workunit.client.0.smithi067.stdout:*** Default configuration is based on 'x86_64_defconfig' 2024-07-31T15:00:29.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:29 smithi067 ceph-mon[32750]: pgmap v688: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 1.39k op/s 2024-07-31T15:00:29.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:29 smithi073 ceph-mon[36714]: pgmap v688: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 1.39k op/s 2024-07-31T15:00:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:29 smithi077 ceph-mon[37327]: pgmap v688: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 1.39k op/s 2024-07-31T15:00:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.787+0000 7f81d6ffd640 1 -- 172.21.15.67:0/3874603265 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558aeaec6da0 con 0x558aeab451c0 2024-07-31T15:00:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.787+0000 7f81d6ffd640 10 client.27239 handle_command_reply: tid=0 2024-07-31T15:00:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.787+0000 7f8204999740 1 client.27239 shutdown 2024-07-31T15:00:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.787+0000 7f8204999740 1 -- 172.21.15.67:0/3874603265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 msgr2=0x558aeaf63f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.787+0000 7f8204999740 1 --2- 172.21.15.67:0/3874603265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf63f00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f81f0002230 tx=0x7f81f0005fd0 comp rx=0 tx=0).stop 2024-07-31T15:00:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.787+0000 7f8204999740 1 -- 172.21.15.67:0/3874603265 shutdown_connections 2024-07-31T15:00:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.787+0000 7f8204999740 1 --2- 172.21.15.67:0/3874603265 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558aeab451c0 0x558aeaffcc50 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.787+0000 7f8204999740 1 --2- 172.21.15.67:0/3874603265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558aeabb11b0 0x558aeafd0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.787+0000 7f8204999740 1 --2- 172.21.15.67:0/3874603265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aeab904d0 0x558aeaf644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.787+0000 7f8204999740 1 --2- 172.21.15.67:0/3874603265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558aeab882f0 0x558aeaf63f00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.787+0000 7f8204999740 1 -- 172.21.15.67:0/3874603265 >> 172.21.15.67:0/3874603265 conn(0x558aeaba5300 msgr2=0x558aeaf608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.787+0000 7f81d6ffd640 0 client.27239 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.788+0000 7f8204999740 1 -- 172.21.15.67:0/3874603265 shutdown_connections 2024-07-31T15:00:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.788+0000 7f8204999740 1 -- 172.21.15.67:0/3874603265 wait complete. 2024-07-31T15:00:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.788+0000 7f8204999740 20 client.27239 trim_cache size 0 max 16384 2024-07-31T15:00:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:00:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:00:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:00:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:00:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.791+0000 7f81fdaa4640 1 -- 172.21.15.67:0/211078586 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81e00b3f50 msgr2=0x7f81e00b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.791+0000 7f81fdaa4640 1 --2- 172.21.15.67:0/211078586 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81e00b3f50 0x7f81e00b6460 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f81e80a9de0 tx=0x7f81e80047d0 comp rx=0 tx=0).stop 2024-07-31T15:00:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.791+0000 7f81fdaa4640 1 -- 172.21.15.67:0/211078586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f811c5d0 msgr2=0x7f81f8114750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.791+0000 7f81fdaa4640 1 --2- 172.21.15.67:0/211078586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f811c5d0 0x7f81f8114750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009fd0 tx=0x7f81e800b8e0 comp rx=0 tx=0).stop 2024-07-31T15:00:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.792+0000 7f81fdaa4640 1 -- 172.21.15.67:0/211078586 shutdown_connections 2024-07-31T15:00:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.792+0000 7f81fdaa4640 1 --2- 172.21.15.67:0/211078586 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f81e00b3f50 0x7f81e00b6460 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.792+0000 7f81fdaa4640 1 --2- 172.21.15.67:0/211078586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f811c5d0 0x7f81f8114750 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.792+0000 7f81fdaa4640 1 --2- 172.21.15.67:0/211078586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f81118d0 0x7f81f8114160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.792+0000 7f81fdaa4640 1 --2- 172.21.15.67:0/211078586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81f810a330 0x7f81f811aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.792+0000 7f81fdaa4640 1 -- 172.21.15.67:0/211078586 >> 172.21.15.67:0/211078586 conn(0x7f81f80707c0 msgr2=0x7f81f8071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.793+0000 7f81fdaa4640 1 -- 172.21.15.67:0/211078586 shutdown_connections 2024-07-31T15:00:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:29.793+0000 7f81fdaa4640 1 -- 172.21.15.67:0/211078586 wait complete. 2024-07-31T15:00:31.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:31 smithi067 ceph-mon[32750]: pgmap v689: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.1 MiB/s wr, 1.18k op/s 2024-07-31T15:00:31.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:31.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:31.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:31 smithi073 ceph-mon[36714]: pgmap v689: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.1 MiB/s wr, 1.18k op/s 2024-07-31T15:00:31.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:31.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:31 smithi077 ceph-mon[37327]: pgmap v689: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.1 MiB/s wr, 1.18k op/s 2024-07-31T15:00:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:33.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:33 smithi067 ceph-mon[32750]: pgmap v690: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 1.18k op/s 2024-07-31T15:00:33.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:33.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:33.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:33 smithi073 ceph-mon[36714]: pgmap v690: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 1.18k op/s 2024-07-31T15:00:33.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:33.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:33 smithi077 ceph-mon[37327]: pgmap v690: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 1.18k op/s 2024-07-31T15:00:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:34.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:34.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:34.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:34.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:34.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:34.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:34.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:34.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:34.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:35.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:35 smithi073 ceph-mon[36714]: pgmap v691: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.2 MiB/s wr, 735 op/s 2024-07-31T15:00:35.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:35.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:35.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:35.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:35 smithi077 ceph-mon[37327]: pgmap v691: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.2 MiB/s wr, 735 op/s 2024-07-31T15:00:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:35.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:35 smithi067 ceph-mon[32750]: pgmap v691: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.2 MiB/s wr, 735 op/s 2024-07-31T15:00:35.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:35.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:35.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:35.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:36.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:36.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:36.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:36.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:36.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:36.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:36.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:36.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:36.547 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:37 smithi073 ceph-mon[36714]: pgmap v692: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 605 KiB/s wr, 521 op/s 2024-07-31T15:00:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:37 smithi077 ceph-mon[37327]: pgmap v692: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 605 KiB/s wr, 521 op/s 2024-07-31T15:00:37.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:37.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:37.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:37.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:37.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:37.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:37 smithi067 ceph-mon[32750]: pgmap v692: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 605 KiB/s wr, 521 op/s 2024-07-31T15:00:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:39.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:39 smithi073 ceph-mon[36714]: pgmap v693: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 454 KiB/s wr, 594 op/s 2024-07-31T15:00:39.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:39 smithi077 ceph-mon[37327]: pgmap v693: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 454 KiB/s wr, 594 op/s 2024-07-31T15:00:39.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:39 smithi067 ceph-mon[32750]: pgmap v693: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 454 KiB/s wr, 594 op/s 2024-07-31T15:00:40.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:40 smithi073 ceph-mon[36714]: osdmap e97: 12 total, 12 up, 12 in 2024-07-31T15:00:40.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:40.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:40.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:40.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:40.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:40 smithi077 ceph-mon[37327]: osdmap e97: 12 total, 12 up, 12 in 2024-07-31T15:00:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:40 smithi067 ceph-mon[32750]: osdmap e97: 12 total, 12 up, 12 in 2024-07-31T15:00:41.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:41 smithi073 ceph-mon[36714]: pgmap v694: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 250 KiB/s wr, 463 op/s 2024-07-31T15:00:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:41.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:41.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:41.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:41.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:41 smithi077 ceph-mon[37327]: pgmap v694: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 250 KiB/s wr, 463 op/s 2024-07-31T15:00:41.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:41.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:41.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:41.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:41.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:41.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:41.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:41.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:41.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:41 smithi067 ceph-mon[32750]: pgmap v694: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 250 KiB/s wr, 463 op/s 2024-07-31T15:00:41.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:41.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:41.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:41.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:41.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:41.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:41.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:42.110 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:42.110 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:42.111 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:42.111 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:42.111 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:42.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:42.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:42.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:42.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:42.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:42.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:42.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:42.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:42.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:42.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.967+0000 7ff3b6244640 1 -- 172.21.15.67:0/546133280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b010a330 msgr2=0x7ff3b010a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.967+0000 7ff3b6244640 1 --2- 172.21.15.67:0/546133280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b010a330 0x7ff3b010a740 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff3a801acb0 tx=0x7ff3a803b9b0 comp rx=0 tx=0).stop 2024-07-31T15:00:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.968+0000 7ff3b6244640 1 -- 172.21.15.67:0/546133280 shutdown_connections 2024-07-31T15:00:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.968+0000 7ff3b6244640 1 --2- 172.21.15.67:0/546133280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3b011c5d0 0x7ff3b011eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.968+0000 7ff3b6244640 1 --2- 172.21.15.67:0/546133280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b010a330 0x7ff3b010a740 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.968+0000 7ff3b6244640 1 --2- 172.21.15.67:0/546133280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b01118d0 0x7ff3b0109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.968+0000 7ff3b6244640 1 -- 172.21.15.67:0/546133280 >> 172.21.15.67:0/546133280 conn(0x7ff3b00707c0 msgr2=0x7ff3b0070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.973+0000 7ff3b6244640 1 -- 172.21.15.67:0/546133280 shutdown_connections 2024-07-31T15:00:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.973+0000 7ff3b6244640 1 -- 172.21.15.67:0/546133280 wait complete. 2024-07-31T15:00:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.974+0000 7ff3b6244640 1 Processor -- start 2024-07-31T15:00:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.974+0000 7ff3b6244640 1 -- start start 2024-07-31T15:00:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.974+0000 7ff3b6244640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b010a330 0x7ff3b011bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.975+0000 7ff3b6244640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3b01118d0 0x7ff3b01181f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.975+0000 7ff3b6244640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b011c5d0 0x7ff3b0118870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.975+0000 7ff3b6244640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b0114720 con 0x7ff3b01118d0 2024-07-31T15:00:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.975+0000 7ff3b6244640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01145a0 con 0x7ff3b010a330 2024-07-31T15:00:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.975+0000 7ff3b6244640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01148a0 con 0x7ff3b011c5d0 2024-07-31T15:00:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.975+0000 7ff3affff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b011c5d0 0x7ff3b0118870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.975+0000 7ff3affff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b011c5d0 0x7ff3b0118870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:40102/0 (socket says 172.21.15.67:40102) 2024-07-31T15:00:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.975+0000 7ff3affff640 1 -- 172.21.15.67:0/2975779621 learned_addr learned my addr 172.21.15.67:0/2975779621 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.975+0000 7ff3aeffd640 1 --2- 172.21.15.67:0/2975779621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3b01118d0 0x7ff3b01181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.975+0000 7ff3affff640 1 -- 172.21.15.67:0/2975779621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b010a330 msgr2=0x7ff3b011bf20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:00:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.976+0000 7ff3affff640 1 --2- 172.21.15.67:0/2975779621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b010a330 0x7ff3b011bf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.976+0000 7ff3affff640 1 -- 172.21.15.67:0/2975779621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3b01118d0 msgr2=0x7ff3b01181f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.976+0000 7ff3affff640 1 --2- 172.21.15.67:0/2975779621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3b01118d0 0x7ff3b01181f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.976+0000 7ff3affff640 1 -- 172.21.15.67:0/2975779621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b007b050 con 0x7ff3b011c5d0 2024-07-31T15:00:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.976+0000 7ff3affff640 1 --2- 172.21.15.67:0/2975779621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b011c5d0 0x7ff3b0118870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0009fd0 tx=0x7ff3a000b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.976+0000 7ff3acff9640 1 -- 172.21.15.67:0/2975779621 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0011830 con 0x7ff3b011c5d0 2024-07-31T15:00:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.977+0000 7ff3b6244640 1 -- 172.21.15.67:0/2975779621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b007b340 con 0x7ff3b011c5d0 2024-07-31T15:00:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.977+0000 7ff3b6244640 1 -- 172.21.15.67:0/2975779621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b007b860 con 0x7ff3b011c5d0 2024-07-31T15:00:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.977+0000 7ff3acff9640 1 -- 172.21.15.67:0/2975779621 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3a0016070 con 0x7ff3b011c5d0 2024-07-31T15:00:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.977+0000 7ff3acff9640 1 -- 172.21.15.67:0/2975779621 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0010de0 con 0x7ff3b011c5d0 2024-07-31T15:00:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.981+0000 7ff3acff9640 1 -- 172.21.15.67:0/2975779621 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff3a00119d0 con 0x7ff3b011c5d0 2024-07-31T15:00:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.982+0000 7ff3acff9640 1 --2- 172.21.15.67:0/2975779621 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff39c0b3f50 0x7ff39c0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.983+0000 7ff3acff9640 1 -- 172.21.15.67:0/2975779621 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9455+0+0 (secure 0 0 0) 0x7ff3a00e36e0 con 0x7ff3b011c5d0 2024-07-31T15:00:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.986+0000 7ff3bd21e740 1 Processor -- start 2024-07-31T15:00:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.984+0000 7ff3af7fe640 1 --2- 172.21.15.67:0/2975779621 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff39c0b3f50 0x7ff39c0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.989+0000 7ff3bd21e740 1 -- start start 2024-07-31T15:00:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.989+0000 7ff3bd21e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a540446ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.989+0000 7ff3bd21e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a54007b2f0 0x55a540447220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.989+0000 7ff3bd21e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5400834d0 0x55a540447760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.989+0000 7ff3bd21e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a5403bc3b0 con 0x55a5400a41b0 2024-07-31T15:00:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.989+0000 7ff3bd21e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a5403ab8b0 con 0x55a5400834d0 2024-07-31T15:00:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.989+0000 7ff3bd21e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a5403af8f0 con 0x55a54007b2f0 2024-07-31T15:00:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.990+0000 7ff3af7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a540446ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.990+0000 7ff3af7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a540446ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57180/0 (socket says 172.21.15.67:57180) 2024-07-31T15:00:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.990+0000 7ff3af7fe640 1 -- 172.21.15.67:0/1043824897 learned_addr learned my addr 172.21.15.67:0/1043824897 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.990+0000 7ff3af7fe640 1 --2- 172.21.15.67:0/2975779621 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff39c0b3f50 0x7ff39c0b6460 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff3b011bc60 tx=0x7ff398006bd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.990+0000 7ff3af7fe640 1 -- 172.21.15.67:0/1043824897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a54007b2f0 msgr2=0x55a540447220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:00:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.990+0000 7ff3af7fe640 1 --2- 172.21.15.67:0/1043824897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a54007b2f0 0x55a540447220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.990+0000 7ff3af7fe640 1 -- 172.21.15.67:0/1043824897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5400834d0 msgr2=0x55a540447760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:00:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.990+0000 7ff3af7fe640 1 --2- 172.21.15.67:0/1043824897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5400834d0 0x55a540447760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.990+0000 7ff3af7fe640 1 -- 172.21.15.67:0/1043824897 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5403529a0 con 0x55a5400a41b0 2024-07-31T15:00:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.991+0000 7ff3af7fe640 1 --2- 172.21.15.67:0/1043824897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a540446ce0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff3980138c0 tx=0x7ff39800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cb7fe6a548504c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.991+0000 7ff3a5ffb640 1 -- 172.21.15.67:0/1043824897 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff398012ca0 con 0x55a5400a41b0 2024-07-31T15:00:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.991+0000 7ff3a5ffb640 1 -- 172.21.15.67:0/1043824897 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff398022070 con 0x55a5400a41b0 2024-07-31T15:00:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.993+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1043824897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 msgr2=0x55a540446ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.993+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/1043824897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a540446ce0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff3980138c0 tx=0x7ff39800e000 comp rx=0 tx=0).stop 2024-07-31T15:00:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.995+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1043824897 shutdown_connections 2024-07-31T15:00:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.995+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/1043824897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5400834d0 0x55a540447760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.995+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/1043824897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a54007b2f0 0x55a540447220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.995+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/1043824897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a540446ce0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.995+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1043824897 >> 172.21.15.67:0/1043824897 conn(0x55a540098300 msgr2=0x55a540444bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.995+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1043824897 shutdown_connections 2024-07-31T15:00:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.995+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1043824897 wait complete. 2024-07-31T15:00:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.996+0000 7ff3bd21e740 1 -- start start 2024-07-31T15:00:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.996+0000 7ff3bd21e740 1 Processor -- start 2024-07-31T15:00:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.997+0000 7ff3bd21e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a54007b2f0 0x55a540452860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.997+0000 7ff3bd21e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5400834d0 0x55a540452da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.997+0000 7ff3bd21e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5400a41b0 0x55a540457230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.997+0000 7ff3bd21e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a5402e5ab0 con 0x55a54007b2f0 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.997+0000 7ff3bd21e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a540333cf0 con 0x55a5400a41b0 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.997+0000 7ff3bd21e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a540477470 con 0x55a5400834d0 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.997+0000 7ff3bd21e740 10 client.0 _ll_register_callbacks cb 0x55a5403a59f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.998+0000 7ff3aeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5400834d0 0x55a540452da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.998+0000 7ff3aeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5400834d0 0x55a540452da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:40118/0 (socket says 172.21.15.67:40118) 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.998+0000 7ff3aeffd640 1 -- 172.21.15.67:0/2746207556 learned_addr learned my addr 172.21.15.67:0/2746207556 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.998+0000 7ff3affff640 1 --2- 172.21.15.67:0/2746207556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5400a41b0 0x55a540457230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.998+0000 7ff3af7fe640 1 --2- 172.21.15.67:0/2746207556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a54007b2f0 0x55a540452860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.998+0000 7ff3affff640 1 -- 172.21.15.67:0/2746207556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5400834d0 msgr2=0x55a540452da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.998+0000 7ff3affff640 1 --2- 172.21.15.67:0/2746207556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5400834d0 0x55a540452da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.998+0000 7ff3affff640 1 -- 172.21.15.67:0/2746207556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a54007b2f0 msgr2=0x55a540452860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.998+0000 7ff3affff640 1 --2- 172.21.15.67:0/2746207556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a54007b2f0 0x55a540452860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.998+0000 7ff3affff640 1 -- 172.21.15.67:0/2746207556 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a54010f790 con 0x55a5400a41b0 2024-07-31T15:00:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.999+0000 7ff3affff640 1 --2- 172.21.15.67:0/2746207556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5400a41b0 0x55a540457230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00e0b40 tx=0x7ff3a00105b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c79074251b33dee3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.999+0000 7ff38effd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:00:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.999+0000 7ff38effd640 1 -- 172.21.15.67:0/2746207556 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00ae600 con 0x55a5400a41b0 2024-07-31T15:00:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.999+0000 7ff3bd21e740 1 -- 172.21.15.67:0/2746207556 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a5403be300 con 0x55a5400a41b0 2024-07-31T15:00:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.999+0000 7ff38effd640 1 -- 172.21.15.67:0/2746207556 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9455+0+0 (secure 0 0 0) 0x7ff3a00a6290 con 0x55a5400a41b0 2024-07-31T15:00:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:42.999+0000 7ff38effd640 1 -- 172.21.15.67:0/2746207556 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff3a00b30b0 con 0x55a5400a41b0 2024-07-31T15:00:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.002+0000 7ff3bd21e740 10 client.27260 fetch_fsmap learned FSMap version 55 2024-07-31T15:00:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.002+0000 7ff3bd21e740 1 -- 172.21.15.67:0/2746207556 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a540188620 con 0x55a5400a41b0 2024-07-31T15:00:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.004+0000 7ff38effd640 1 -- 172.21.15.67:0/2746207556 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a5403be300 con 0x55a5400a41b0 2024-07-31T15:00:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.004+0000 7ff38effd640 1 -- 172.21.15.67:0/2746207556 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7ff3a00a9260 con 0x55a5400a41b0 2024-07-31T15:00:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.004+0000 7ff3bd21e740 10 client.27260 fetch_fsmap finished waiting for FSMap version 55 2024-07-31T15:00:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.004+0000 7ff3bd21e740 10 client.27260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:00:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.004+0000 7ff3bd21e740 20 client.27260 populate_metadata read hostname 'smithi067' 2024-07-31T15:00:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.004+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/2746207556 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a540062d80 0x55a54045d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.004+0000 7ff3bd21e740 4 client.27260 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:00:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.004+0000 7ff3bd21e740 1 -- 172.21.15.67:0/2746207556 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5403b9da0 con 0x55a540062d80 2024-07-31T15:00:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.004+0000 7ff3af7fe640 1 --2- 172.21.15.67:0/2746207556 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a540062d80 0x55a54045d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.006+0000 7ff3af7fe640 1 --2- 172.21.15.67:0/2746207556 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a540062d80 0x55a54045d3f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6bd1dc718eb3fc0 server_cookie=fca2c16250052340 in_seq=0 out_seq=0 2024-07-31T15:00:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.006+0000 7ff38effd640 10 client.27260 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.011+0000 7ff38effd640 1 -- 172.21.15.67:0/2746207556 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5403b9da0 con 0x55a540062d80 2024-07-31T15:00:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.011+0000 7ff38effd640 10 client.27260 handle_command_reply: tid=0 2024-07-31T15:00:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.011+0000 7ff3bd21e740 1 client.27260 shutdown 2024-07-31T15:00:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.014+0000 7ff3bd21e740 1 -- 172.21.15.67:0/2746207556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5400a41b0 msgr2=0x55a540457230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.014+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/2746207556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5400a41b0 0x55a540457230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00e0b40 tx=0x7ff3a00105b0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.017+0000 7ff3bd21e740 1 -- 172.21.15.67:0/2746207556 shutdown_connections 2024-07-31T15:00:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.017+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/2746207556 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a540062d80 0x55a54045d3f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.017+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/2746207556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5400a41b0 0x55a540457230 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.017+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/2746207556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5400834d0 0x55a540452da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.017+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/2746207556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a54007b2f0 0x55a540452860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.017+0000 7ff3bd21e740 1 -- 172.21.15.67:0/2746207556 >> 172.21.15.67:0/2746207556 conn(0x55a540098300 msgr2=0x55a540447d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.018+0000 7ff38effd640 0 client.27260 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.023+0000 7ff3bd21e740 1 -- 172.21.15.67:0/2746207556 shutdown_connections 2024-07-31T15:00:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.023+0000 7ff3bd21e740 1 -- 172.21.15.67:0/2746207556 wait complete. 2024-07-31T15:00:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.023+0000 7ff3bd21e740 20 client.27260 trim_cache size 0 max 16384 2024-07-31T15:00:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.058+0000 7ff3bd21e740 1 Processor -- start 2024-07-31T15:00:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.059+0000 7ff3bd21e740 1 -- start start 2024-07-31T15:00:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.059+0000 7ff3bd21e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a54007b2f0 0x55a540476910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.059+0000 7ff3bd21e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5400834d0 0x55a540451ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.059+0000 7ff3bd21e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a5404444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.059+0000 7ff3bd21e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff39000ff00 con 0x55a5400a41b0 2024-07-31T15:00:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.059+0000 7ff3bd21e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff390011b90 con 0x55a5400834d0 2024-07-31T15:00:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.059+0000 7ff3bd21e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff3900105d0 con 0x55a54007b2f0 2024-07-31T15:00:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.059+0000 7ff3affff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a5404444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.060+0000 7ff3affff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a5404444f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57194/0 (socket says 172.21.15.67:57194) 2024-07-31T15:00:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.060+0000 7ff3affff640 1 -- 172.21.15.67:0/1786137153 learned_addr learned my addr 172.21.15.67:0/1786137153 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.060+0000 7ff3affff640 1 -- 172.21.15.67:0/1786137153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a54007b2f0 msgr2=0x55a540476910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:00:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.060+0000 7ff3affff640 1 --2- 172.21.15.67:0/1786137153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a54007b2f0 0x55a540476910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.060+0000 7ff3affff640 1 -- 172.21.15.67:0/1786137153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5400834d0 msgr2=0x55a540451ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.060+0000 7ff3affff640 1 --2- 172.21.15.67:0/1786137153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5400834d0 0x55a540451ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.060+0000 7ff3affff640 1 -- 172.21.15.67:0/1786137153 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5404487f0 con 0x55a5400a41b0 2024-07-31T15:00:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.063+0000 7ff3affff640 1 --2- 172.21.15.67:0/1786137153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a5404444f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00a9f40 tx=0x7ff3a00aa180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e874ce11773a4623 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.064+0000 7ff3a57fa640 1 -- 172.21.15.67:0/1786137153 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00a9770 con 0x55a5400a41b0 2024-07-31T15:00:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.064+0000 7ff3a57fa640 1 -- 172.21.15.67:0/1786137153 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3a00e2070 con 0x55a5400a41b0 2024-07-31T15:00:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.066+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1786137153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 msgr2=0x55a5404444f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.066+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/1786137153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a5404444f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00a9f40 tx=0x7ff3a00aa180 comp rx=0 tx=0).stop 2024-07-31T15:00:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.070+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1786137153 shutdown_connections 2024-07-31T15:00:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.070+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/1786137153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a5404444f0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.070+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/1786137153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5400834d0 0x55a540451ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.070+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/1786137153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a54007b2f0 0x55a540476910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.070+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1786137153 >> 172.21.15.67:0/1786137153 conn(0x55a540098300 msgr2=0x55a54044bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.078+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1786137153 shutdown_connections 2024-07-31T15:00:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.083+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1786137153 wait complete. 2024-07-31T15:00:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.083+0000 7ff3bd21e740 1 -- start start 2024-07-31T15:00:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.083+0000 7ff3bd21e740 1 Processor -- start 2024-07-31T15:00:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.084+0000 7ff3bd21e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a54007b2f0 0x55a54047dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.084+0000 7ff3bd21e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5400834d0 0x55a540478480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.084+0000 7ff3bd21e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a5404789c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.084+0000 7ff3bd21e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff3900114c0 con 0x55a5400a41b0 2024-07-31T15:00:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.084+0000 7ff3bd21e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff390010d30 con 0x55a54007b2f0 2024-07-31T15:00:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.084+0000 7ff3bd21e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff39000dee0 con 0x55a5400834d0 2024-07-31T15:00:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.084+0000 7ff3bd21e740 10 client.0 _ll_register_callbacks cb 0x7ff3a0009280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:00:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.085+0000 7ff3aeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5400834d0 0x55a540478480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.085+0000 7ff3affff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a5404789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.085+0000 7ff3af7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a54007b2f0 0x55a54047dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.085+0000 7ff3affff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a5404789c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57206/0 (socket says 172.21.15.67:57206) 2024-07-31T15:00:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.085+0000 7ff3affff640 1 -- 172.21.15.67:0/1428991057 learned_addr learned my addr 172.21.15.67:0/1428991057 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.088+0000 7ff3affff640 1 -- 172.21.15.67:0/1428991057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5400834d0 msgr2=0x55a540478480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.088+0000 7ff3affff640 1 --2- 172.21.15.67:0/1428991057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5400834d0 0x55a540478480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.088+0000 7ff3affff640 1 -- 172.21.15.67:0/1428991057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a54007b2f0 msgr2=0x55a54047dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.088+0000 7ff3affff640 1 --2- 172.21.15.67:0/1428991057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a54007b2f0 0x55a54047dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.088+0000 7ff3affff640 1 -- 172.21.15.67:0/1428991057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a540479100 con 0x55a5400a41b0 2024-07-31T15:00:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.088+0000 7ff3aeffd640 1 --2- 172.21.15.67:0/1428991057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5400834d0 0x55a540478480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:00:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.090+0000 7ff3affff640 1 --2- 172.21.15.67:0/1428991057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a5404789c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00a6080 tx=0x7ff3a00108a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3097d617c00eb8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.092+0000 7ff38effd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:00:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.096+0000 7ff38effd640 1 -- 172.21.15.67:0/1428991057 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00a9770 con 0x55a5400a41b0 2024-07-31T15:00:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.096+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1428991057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3a00e2070 con 0x55a5400a41b0 2024-07-31T15:00:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.096+0000 7ff38effd640 1 -- 172.21.15.67:0/1428991057 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9455+0+0 (secure 0 0 0) 0x7ff3a00a65f0 con 0x55a5400a41b0 2024-07-31T15:00:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.097+0000 7ff38effd640 1 -- 172.21.15.67:0/1428991057 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff3a00ae600 con 0x55a5400a41b0 2024-07-31T15:00:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.103+0000 7ff3bd21e740 10 client.17703 fetch_fsmap learned FSMap version 55 2024-07-31T15:00:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.103+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1428991057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a54050f940 con 0x55a5400a41b0 2024-07-31T15:00:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.108+0000 7ff38effd640 1 -- 172.21.15.67:0/1428991057 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3a00e2070 con 0x55a5400a41b0 2024-07-31T15:00:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.108+0000 7ff38effd640 1 -- 172.21.15.67:0/1428991057 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 55) ==== 8406+0+0 (secure 0 0 0) 0x7ff3a00a8cd0 con 0x55a5400a41b0 2024-07-31T15:00:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.110+0000 7ff3bd21e740 10 client.17703 fetch_fsmap finished waiting for FSMap version 55 2024-07-31T15:00:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.110+0000 7ff3bd21e740 10 client.17703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:00:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.110+0000 7ff3bd21e740 20 client.17703 populate_metadata read hostname 'smithi067' 2024-07-31T15:00:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.110+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/1428991057 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a5400381c0 0x55a540511da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.110+0000 7ff3bd21e740 4 client.17703 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:00:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.110+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1428991057 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a5403b74e0 con 0x55a5400381c0 2024-07-31T15:00:43.113 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.110+0000 7ff3af7fe640 1 --2- 172.21.15.67:0/1428991057 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a5400381c0 0x55a540511da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.115+0000 7ff3af7fe640 1 --2- 172.21.15.67:0/1428991057 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a5400381c0 0x55a540511da0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edf30e3d288f1ad8 server_cookie=24680eabc5c17d in_seq=0 out_seq=0 2024-07-31T15:00:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.116+0000 7ff38effd640 10 client.17703 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:43 smithi073 ceph-mon[36714]: pgmap v696: 129 pgs: 15 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 67 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 211 KiB/s wr, 680 op/s 2024-07-31T15:00:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:43.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:43.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:43.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:43 smithi077 ceph-mon[37327]: pgmap v696: 129 pgs: 15 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 67 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 211 KiB/s wr, 680 op/s 2024-07-31T15:00:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:43.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:43.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.544+0000 7ff38effd640 1 -- 172.21.15.67:0/1428991057 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a5403b74e0 con 0x55a5400381c0 2024-07-31T15:00:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.544+0000 7ff38effd640 10 client.17703 handle_command_reply: tid=0 2024-07-31T15:00:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.544+0000 7ff3bd21e740 1 client.17703 shutdown 2024-07-31T15:00:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.545+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1428991057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 msgr2=0x55a5404789c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.545+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/1428991057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a5404789c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00a6080 tx=0x7ff3a00108a0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.545+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1428991057 shutdown_connections 2024-07-31T15:00:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.545+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/1428991057 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a5400381c0 0x55a540511da0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.545+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/1428991057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a5400a41b0 0x55a5404789c0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.545+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/1428991057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5400834d0 0x55a540478480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.545+0000 7ff3bd21e740 1 --2- 172.21.15.67:0/1428991057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a54007b2f0 0x55a54047dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.545+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1428991057 >> 172.21.15.67:0/1428991057 conn(0x55a540098300 msgr2=0x55a5404477e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.545+0000 7ff38effd640 0 client.17703 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.545+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1428991057 shutdown_connections 2024-07-31T15:00:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.545+0000 7ff3bd21e740 1 -- 172.21.15.67:0/1428991057 wait complete. 2024-07-31T15:00:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.545+0000 7ff3bd21e740 20 client.17703 trim_cache size 0 max 16384 2024-07-31T15:00:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:00:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:00:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:00:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:00:43.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.546+0000 7ff3b6244640 1 -- 172.21.15.67:0/2975779621 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff39c0b3f50 msgr2=0x7ff39c0b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.546+0000 7ff3b6244640 1 --2- 172.21.15.67:0/2975779621 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff39c0b3f50 0x7ff39c0b6460 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff3b011bc60 tx=0x7ff398006bd0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.547+0000 7ff3b6244640 1 -- 172.21.15.67:0/2975779621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b011c5d0 msgr2=0x7ff3b0118870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.547+0000 7ff3b6244640 1 --2- 172.21.15.67:0/2975779621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b011c5d0 0x7ff3b0118870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0009fd0 tx=0x7ff3a000b8e0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.548+0000 7ff3b6244640 1 -- 172.21.15.67:0/2975779621 shutdown_connections 2024-07-31T15:00:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.548+0000 7ff3b6244640 1 --2- 172.21.15.67:0/2975779621 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff39c0b3f50 0x7ff39c0b6460 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.548+0000 7ff3b6244640 1 --2- 172.21.15.67:0/2975779621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b011c5d0 0x7ff3b0118870 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.548+0000 7ff3b6244640 1 --2- 172.21.15.67:0/2975779621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3b01118d0 0x7ff3b01181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.548+0000 7ff3b6244640 1 --2- 172.21.15.67:0/2975779621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b010a330 0x7ff3b011bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.548+0000 7ff3b6244640 1 -- 172.21.15.67:0/2975779621 >> 172.21.15.67:0/2975779621 conn(0x7ff3b00707c0 msgr2=0x7ff3b0073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.549+0000 7ff3b6244640 1 -- 172.21.15.67:0/2975779621 shutdown_connections 2024-07-31T15:00:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:43.549+0000 7ff3b6244640 1 -- 172.21.15.67:0/2975779621 wait complete. 2024-07-31T15:00:43.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:43 smithi067 ceph-mon[32750]: pgmap v696: 129 pgs: 15 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 67 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 211 KiB/s wr, 680 op/s 2024-07-31T15:00:43.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:43.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:44.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:44.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:44.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:44.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:44.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:44.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: pgmap v697: 129 pgs: 12 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 84 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 278 KiB/s wr, 774 op/s 2024-07-31T15:00:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:45.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: pgmap v697: 129 pgs: 12 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 84 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 278 KiB/s wr, 774 op/s 2024-07-31T15:00:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:45.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:45.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: pgmap v697: 129 pgs: 12 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 84 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 278 KiB/s wr, 774 op/s 2024-07-31T15:00:45.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:45.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:45.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:46.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:46.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:46.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:46.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:46.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:46.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:46.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:46.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:47.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:47 smithi073 ceph-mon[36714]: pgmap v698: 129 pgs: 12 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 84 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 327 KiB/s wr, 853 op/s 2024-07-31T15:00:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:47.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:47 smithi077 ceph-mon[37327]: pgmap v698: 129 pgs: 12 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 84 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 327 KiB/s wr, 853 op/s 2024-07-31T15:00:47.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:47.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:47.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:47.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:47.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:47.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:47.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:47 smithi067 ceph-mon[32750]: pgmap v698: 129 pgs: 12 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 84 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 327 KiB/s wr, 853 op/s 2024-07-31T15:00:47.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:47.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:47.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:47.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:47.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:47.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:48.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:48.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:48.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:48.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:48.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:48.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:48.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:48.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:48.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: pgmap v699: 129 pgs: 12 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 84 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 590 KiB/s wr, 1.08k op/s 2024-07-31T15:00:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: pgmap v699: 129 pgs: 12 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 84 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 590 KiB/s wr, 1.08k op/s 2024-07-31T15:00:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: pgmap v699: 129 pgs: 12 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 84 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 590 KiB/s wr, 1.08k op/s 2024-07-31T15:00:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:50.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:50 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:50.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:50 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:50.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:50 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:50.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:50 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:50.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:50 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:50.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:50 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:51.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:51 smithi077 ceph-mon[37327]: pgmap v700: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 590 KiB/s wr, 1.10k op/s 2024-07-31T15:00:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:51 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:00:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:51.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:51 smithi067 ceph-mon[32750]: pgmap v700: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 590 KiB/s wr, 1.10k op/s 2024-07-31T15:00:51.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:51 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:00:51.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:51.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:51.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:51.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:51.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:51.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:51 smithi073 ceph-mon[36714]: pgmap v700: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 590 KiB/s wr, 1.10k op/s 2024-07-31T15:00:51.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:51 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:00:51.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:51.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:51.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:51.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:51.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:52.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: pgmap v701: 129 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 497 KiB/s wr, 1.25k op/s 2024-07-31T15:00:53.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:53.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:53.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:53.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:53.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:53.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.500 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:00:53.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: pgmap v701: 129 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 497 KiB/s wr, 1.25k op/s 2024-07-31T15:00:53.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:53.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:53.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:53.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:53.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:53.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:53.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:53.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:53.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:53.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:53.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:53.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:53.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: pgmap v701: 129 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 497 KiB/s wr, 1.25k op/s 2024-07-31T15:00:53.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:53.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:53.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:53.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:53.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:53.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:53.921 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:00:54.386 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:00:54.387 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:54.479 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:00:54.479 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:54.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:00:54.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:00:55.392 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:55 smithi067 ceph-mon[32750]: pgmap v702: 129 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 566 KiB/s wr, 1.36k op/s 2024-07-31T15:00:55.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:55 smithi077 ceph-mon[37327]: pgmap v702: 129 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 566 KiB/s wr, 1.36k op/s 2024-07-31T15:00:55.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:55 smithi073 ceph-mon[36714]: pgmap v702: 129 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 110 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 566 KiB/s wr, 1.36k op/s 2024-07-31T15:00:56.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.270+0000 7f4281979640 1 -- 172.21.15.67:0/3486416925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427c113e80 msgr2=0x7f427c116350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.270+0000 7f4281979640 1 --2- 172.21.15.67:0/3486416925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427c113e80 0x7f427c116350 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4274009ca0 tx=0x7f427402a8b0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.270+0000 7f4281979640 1 -- 172.21.15.67:0/3486416925 shutdown_connections 2024-07-31T15:00:56.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.270+0000 7f4281979640 1 --2- 172.21.15.67:0/3486416925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427c113e80 0x7f427c116350 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.270+0000 7f4281979640 1 --2- 172.21.15.67:0/3486416925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f427c075800 0x7f427c075c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.270+0000 7f4281979640 1 --2- 172.21.15.67:0/3486416925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f427c076db0 0x7f427c0751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.270+0000 7f4281979640 1 -- 172.21.15.67:0/3486416925 >> 172.21.15.67:0/3486416925 conn(0x7f427c0fefe0 msgr2=0x7f427c101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:56.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.271+0000 7f4281979640 1 -- 172.21.15.67:0/3486416925 shutdown_connections 2024-07-31T15:00:56.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.271+0000 7f4281979640 1 -- 172.21.15.67:0/3486416925 wait complete. 2024-07-31T15:00:56.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.271+0000 7f4281979640 1 Processor -- start 2024-07-31T15:00:56.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.272+0000 7f4281979640 1 -- start start 2024-07-31T15:00:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.272+0000 7f4281979640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427c075800 0x7f427c1a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.272+0000 7f4281979640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f427c076db0 0x7f427c1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.272+0000 7f4281979640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f427c113e80 0x7f427c1a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.272+0000 7f4281979640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f427c119b70 con 0x7f427c075800 2024-07-31T15:00:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.272+0000 7f4281979640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1199f0 con 0x7f427c113e80 2024-07-31T15:00:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.272+0000 7f4281979640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f427c119cf0 con 0x7f427c076db0 2024-07-31T15:00:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.272+0000 7f427affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427c075800 0x7f427c1a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.272+0000 7f427affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427c075800 0x7f427c1a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47792/0 (socket says 172.21.15.67:47792) 2024-07-31T15:00:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.272+0000 7f427affd640 1 -- 172.21.15.67:0/1784895726 learned_addr learned my addr 172.21.15.67:0/1784895726 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.272+0000 7f427a7fc640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f427c076db0 0x7f427c1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.272+0000 7f427b7fe640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f427c113e80 0x7f427c1a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.273+0000 7f427affd640 1 -- 172.21.15.67:0/1784895726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f427c076db0 msgr2=0x7f427c1a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.273+0000 7f427affd640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f427c076db0 0x7f427c1a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.273+0000 7f427affd640 1 -- 172.21.15.67:0/1784895726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f427c113e80 msgr2=0x7f427c1a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.273+0000 7f427affd640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f427c113e80 0x7f427c1a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.273+0000 7f427affd640 1 -- 172.21.15.67:0/1784895726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c1a6410 con 0x7f427c075800 2024-07-31T15:00:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.273+0000 7f427a7fc640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f427c076db0 0x7f427c1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:00:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.273+0000 7f427affd640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427c075800 0x7f427c1a04a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f426c00c890 tx=0x7f426c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.273+0000 7f4280977640 1 -- 172.21.15.67:0/1784895726 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c004220 con 0x7f427c075800 2024-07-31T15:00:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.273+0000 7f4280977640 1 -- 172.21.15.67:0/1784895726 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f426c0043c0 con 0x7f427c075800 2024-07-31T15:00:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.274+0000 7f4280977640 1 -- 172.21.15.67:0/1784895726 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c004a00 con 0x7f427c075800 2024-07-31T15:00:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.274+0000 7f4281979640 1 -- 172.21.15.67:0/1784895726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427c1a6700 con 0x7f427c075800 2024-07-31T15:00:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.274+0000 7f4281979640 1 -- 172.21.15.67:0/1784895726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427c1acee0 con 0x7f427c075800 2024-07-31T15:00:56.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.277+0000 7f4287eca740 1 Processor -- start 2024-07-31T15:00:56.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.277+0000 7f4287eca740 1 -- start start 2024-07-31T15:00:56.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.277+0000 7f4287eca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f25070 0x5641322ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.277+0000 7f4287eca740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131efc1b0 0x5641322cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.277+0000 7f4287eca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131f04390 0x5641322cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.277+0000 7f4287eca740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564132240b60 con 0x564131efc1b0 2024-07-31T15:00:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.277+0000 7f4287eca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564132230060 con 0x564131f04390 2024-07-31T15:00:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.277+0000 7f4287eca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5641322340a0 con 0x564131f25070 2024-07-31T15:00:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.277+0000 7f427b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f25070 0x5641322ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.278+0000 7f427b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f25070 0x5641322ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44012/0 (socket says 172.21.15.67:44012) 2024-07-31T15:00:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.278+0000 7f427b7fe640 1 -- 172.21.15.67:0/3807168431 learned_addr learned my addr 172.21.15.67:0/3807168431 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.278+0000 7f427a7fc640 1 --2- 172.21.15.67:0/3807168431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131f04390 0x5641322cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.278+0000 7f427b7fe640 1 -- 172.21.15.67:0/3807168431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131f04390 msgr2=0x5641322cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.278+0000 7f427b7fe640 1 --2- 172.21.15.67:0/3807168431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131f04390 0x5641322cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.278+0000 7f427b7fe640 1 -- 172.21.15.67:0/3807168431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131efc1b0 msgr2=0x5641322cab80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:00:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.278+0000 7f427b7fe640 1 --2- 172.21.15.67:0/3807168431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131efc1b0 0x5641322cab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.278+0000 7f427b7fe640 1 -- 172.21.15.67:0/3807168431 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641321d6b40 con 0x564131f25070 2024-07-31T15:00:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.278+0000 7f427b7fe640 1 --2- 172.21.15.67:0/3807168431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f25070 0x5641322ca640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4274009ca0 tx=0x7f4274030450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e8d24ddaeb6c11 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.279+0000 7f4261ffb640 1 -- 172.21.15.67:0/3807168431 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274030a00 con 0x564131f25070 2024-07-31T15:00:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.279+0000 7f4261ffb640 1 -- 172.21.15.67:0/3807168431 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4274036030 con 0x564131f25070 2024-07-31T15:00:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.279+0000 7f4261ffb640 1 -- 172.21.15.67:0/3807168431 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274004640 con 0x564131f25070 2024-07-31T15:00:56.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.280+0000 7f4280977640 1 -- 172.21.15.67:0/1784895726 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f426c0079b0 con 0x7f427c075800 2024-07-31T15:00:56.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.280+0000 7f4287eca740 1 -- 172.21.15.67:0/3807168431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f25070 msgr2=0x5641322ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.280+0000 7f4287eca740 1 --2- 172.21.15.67:0/3807168431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f25070 0x5641322ca640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4274009ca0 tx=0x7f4274030450 comp rx=0 tx=0).stop 2024-07-31T15:00:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.280+0000 7f4287eca740 1 -- 172.21.15.67:0/3807168431 shutdown_connections 2024-07-31T15:00:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.280+0000 7f4287eca740 1 --2- 172.21.15.67:0/3807168431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131f04390 0x5641322cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.280+0000 7f4287eca740 1 --2- 172.21.15.67:0/3807168431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131efc1b0 0x5641322cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.280+0000 7f4287eca740 1 --2- 172.21.15.67:0/3807168431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f25070 0x5641322ca640 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.280+0000 7f4287eca740 1 -- 172.21.15.67:0/3807168431 >> 172.21.15.67:0/3807168431 conn(0x564131f19460 msgr2=0x5641322c8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.281+0000 7f4287eca740 1 -- 172.21.15.67:0/3807168431 shutdown_connections 2024-07-31T15:00:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.281+0000 7f4280977640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42640b3cd0 0x7f42640b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.281+0000 7f4287eca740 1 -- 172.21.15.67:0/3807168431 wait complete. 2024-07-31T15:00:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.281+0000 7f4280977640 1 -- 172.21.15.67:0/1784895726 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9455+0+0 (secure 0 0 0) 0x7f426c0e3fd0 con 0x7f427c075800 2024-07-31T15:00:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.281+0000 7f427a7fc640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42640b3cd0 0x7f42640b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.281+0000 7f4287eca740 1 -- start start 2024-07-31T15:00:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f4287eca740 1 Processor -- start 2024-07-31T15:00:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f427a7fc640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42640b3cd0 0x7f42640b61e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f427c115bd0 tx=0x7f4270009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f4287eca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131efc1b0 0x5641322d61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f4287eca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f04390 0x5641322d66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f4287eca740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f25070 0x5641322e2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f4287eca740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564132093700 con 0x564131f25070 2024-07-31T15:00:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f4287eca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5641321d2990 con 0x564131efc1b0 2024-07-31T15:00:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f4287eca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5641322d8dc0 con 0x564131f04390 2024-07-31T15:00:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f4287eca740 10 client.0 _ll_register_callbacks cb 0x5641321ed4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:00:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f427affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131efc1b0 0x5641322d61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f427affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131efc1b0 0x5641322d61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:50056/0 (socket says 172.21.15.67:50056) 2024-07-31T15:00:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f427affd640 1 -- 172.21.15.67:0/1957799760 learned_addr learned my addr 172.21.15.67:0/1957799760 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f427a7fc640 1 --2- 172.21.15.67:0/1957799760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f04390 0x5641322d66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f427b7fe640 1 --2- 172.21.15.67:0/1957799760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f25070 0x5641322e2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f427affd640 1 -- 172.21.15.67:0/1957799760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f04390 msgr2=0x5641322d66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.282+0000 7f427affd640 1 --2- 172.21.15.67:0/1957799760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f04390 0x5641322d66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.283+0000 7f427affd640 1 -- 172.21.15.67:0/1957799760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f25070 msgr2=0x5641322e2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.283+0000 7f427affd640 1 --2- 172.21.15.67:0/1957799760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f25070 0x5641322e2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.283+0000 7f427affd640 1 -- 172.21.15.67:0/1957799760 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564131f90650 con 0x564131efc1b0 2024-07-31T15:00:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.283+0000 7f427b7fe640 1 --2- 172.21.15.67:0/1957799760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f25070 0x5641322e2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:00:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.283+0000 7f427affd640 1 --2- 172.21.15.67:0/1957799760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131efc1b0 0x5641322d61a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f426c000c00 tx=0x7f426c0b31d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0de0dcca14254d0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.283+0000 7f425affd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:00:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.283+0000 7f425affd640 1 -- 172.21.15.67:0/1957799760 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c01d040 con 0x564131efc1b0 2024-07-31T15:00:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.283+0000 7f425affd640 1 -- 172.21.15.67:0/1957799760 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9455+0+0 (secure 0 0 0) 0x7f426c0cd110 con 0x564131efc1b0 2024-07-31T15:00:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.283+0000 7f4287eca740 1 -- 172.21.15.67:0/1957799760 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4274036030 con 0x564131efc1b0 2024-07-31T15:00:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.283+0000 7f425affd640 1 -- 172.21.15.67:0/1957799760 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f426c0aa7e0 con 0x564131efc1b0 2024-07-31T15:00:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.283+0000 7f4287eca740 10 client.27272 fetch_fsmap learned FSMap version 57 2024-07-31T15:00:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.283+0000 7f4287eca740 1 -- 172.21.15.67:0/1957799760 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5641322e4d80 con 0x564131efc1b0 2024-07-31T15:00:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.284+0000 7f425affd640 1 -- 172.21.15.67:0/1957799760 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4274036030 con 0x564131efc1b0 2024-07-31T15:00:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.284+0000 7f425affd640 1 -- 172.21.15.67:0/1957799760 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 57) ==== 8398+0+0 (secure 0 0 0) 0x7f426c0cbce0 con 0x564131efc1b0 2024-07-31T15:00:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.284+0000 7f4287eca740 10 client.27272 fetch_fsmap finished waiting for FSMap version 57 2024-07-31T15:00:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.284+0000 7f4287eca740 10 client.27272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:00:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.284+0000 7f4287eca740 20 client.27272 populate_metadata read hostname 'smithi067' 2024-07-31T15:00:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.284+0000 7f4287eca740 1 --2- 172.21.15.67:0/1957799760 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564131ee39f0 0x5641322e8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.284+0000 7f4287eca740 4 client.27272 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:00:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.284+0000 7f4287eca740 1 -- 172.21.15.67:0/1957799760 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564132242ab0 con 0x564131ee39f0 2024-07-31T15:00:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.284+0000 7f427a7fc640 1 --2- 172.21.15.67:0/1957799760 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564131ee39f0 0x5641322e8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.284+0000 7f425affd640 10 client.27272 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.285+0000 7f427a7fc640 1 --2- 172.21.15.67:0/1957799760 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564131ee39f0 0x5641322e8330 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5363638a76ac01ab server_cookie=579acafd889fc63c in_seq=0 out_seq=0 2024-07-31T15:00:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.288+0000 7f425affd640 1 -- 172.21.15.67:0/1957799760 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564132242ab0 con 0x564131ee39f0 2024-07-31T15:00:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.288+0000 7f425affd640 10 client.27272 handle_command_reply: tid=0 2024-07-31T15:00:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.288+0000 7f4287eca740 1 client.27272 shutdown 2024-07-31T15:00:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.289+0000 7f4287eca740 1 -- 172.21.15.67:0/1957799760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131efc1b0 msgr2=0x5641322d61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.289+0000 7f4287eca740 1 --2- 172.21.15.67:0/1957799760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131efc1b0 0x5641322d61a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f426c000c00 tx=0x7f426c0b31d0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.289+0000 7f4287eca740 1 -- 172.21.15.67:0/1957799760 shutdown_connections 2024-07-31T15:00:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.289+0000 7f4287eca740 1 --2- 172.21.15.67:0/1957799760 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564131ee39f0 0x5641322e8330 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.289+0000 7f4287eca740 1 --2- 172.21.15.67:0/1957799760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f25070 0x5641322e2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.289+0000 7f4287eca740 1 --2- 172.21.15.67:0/1957799760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f04390 0x5641322d66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.289+0000 7f4287eca740 1 --2- 172.21.15.67:0/1957799760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131efc1b0 0x5641322d61a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.289+0000 7f4287eca740 1 -- 172.21.15.67:0/1957799760 >> 172.21.15.67:0/1957799760 conn(0x564131f19460 msgr2=0x5641322c6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.290+0000 7f425affd640 0 client.27272 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.290+0000 7f4287eca740 1 -- 172.21.15.67:0/1957799760 shutdown_connections 2024-07-31T15:00:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.290+0000 7f4287eca740 1 -- 172.21.15.67:0/1957799760 wait complete. 2024-07-31T15:00:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.290+0000 7f4287eca740 20 client.27272 trim_cache size 0 max 16384 2024-07-31T15:00:56.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.325+0000 7f4287eca740 1 Processor -- start 2024-07-31T15:00:56.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.325+0000 7f4287eca740 1 -- start start 2024-07-31T15:00:56.327 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:56 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:00:56.327 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:56 smithi067 ceph-mon[32750]: pgmap v703: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 600 KiB/s wr, 1.35k op/s 2024-07-31T15:00:56.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f4287eca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131efc1b0 0x5641322c8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f4287eca740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f04390 0x5641322d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f4287eca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f25070 0x5641322e1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f4287eca740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1b6e20 con 0x564131f04390 2024-07-31T15:00:56.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f4287eca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1b8a50 con 0x564131efc1b0 2024-07-31T15:00:56.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f4287eca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1b74f0 con 0x564131f25070 2024-07-31T15:00:56.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f427b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f25070 0x5641322e1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f427b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f25070 0x5641322e1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44028/0 (socket says 172.21.15.67:44028) 2024-07-31T15:00:56.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f427b7fe640 1 -- 172.21.15.67:0/2392466213 learned_addr learned my addr 172.21.15.67:0/2392466213 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:56.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f427affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131efc1b0 0x5641322c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f427a7fc640 1 --2- 172.21.15.67:0/2392466213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f04390 0x5641322d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f427b7fe640 1 -- 172.21.15.67:0/2392466213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131efc1b0 msgr2=0x5641322c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f427b7fe640 1 --2- 172.21.15.67:0/2392466213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131efc1b0 0x5641322c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f427b7fe640 1 -- 172.21.15.67:0/2392466213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f04390 msgr2=0x5641322d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f427b7fe640 1 --2- 172.21.15.67:0/2392466213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f04390 0x5641322d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.326+0000 7f427b7fe640 1 -- 172.21.15.67:0/2392466213 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641322d7270 con 0x564131f25070 2024-07-31T15:00:56.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.327+0000 7f427b7fe640 1 --2- 172.21.15.67:0/2392466213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f25070 0x5641322e1f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4274045000 tx=0x7f4274000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5fb3ba20a25da46 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:56.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.327+0000 7f42617fa640 1 -- 172.21.15.67:0/2392466213 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274030800 con 0x564131f25070 2024-07-31T15:00:56.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.327+0000 7f42617fa640 1 -- 172.21.15.67:0/2392466213 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4274036040 con 0x564131f25070 2024-07-31T15:00:56.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.327+0000 7f42617fa640 1 -- 172.21.15.67:0/2392466213 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42740043c0 con 0x564131f25070 2024-07-31T15:00:56.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.328+0000 7f4287eca740 1 -- 172.21.15.67:0/2392466213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f25070 msgr2=0x5641322e1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.328+0000 7f4287eca740 1 --2- 172.21.15.67:0/2392466213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f25070 0x5641322e1f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4274045000 tx=0x7f4274000f40 comp rx=0 tx=0).stop 2024-07-31T15:00:56.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.329+0000 7f4287eca740 1 -- 172.21.15.67:0/2392466213 shutdown_connections 2024-07-31T15:00:56.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.329+0000 7f4287eca740 1 --2- 172.21.15.67:0/2392466213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131f25070 0x5641322e1f00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.329+0000 7f4287eca740 1 --2- 172.21.15.67:0/2392466213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f04390 0x5641322d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.329+0000 7f4287eca740 1 --2- 172.21.15.67:0/2392466213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131efc1b0 0x5641322c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.329+0000 7f4287eca740 1 -- 172.21.15.67:0/2392466213 >> 172.21.15.67:0/2392466213 conn(0x564131f19460 msgr2=0x5641322c7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:56.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.329+0000 7f4287eca740 1 -- 172.21.15.67:0/2392466213 shutdown_connections 2024-07-31T15:00:56.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.329+0000 7f4287eca740 1 -- 172.21.15.67:0/2392466213 wait complete. 2024-07-31T15:00:56.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.330+0000 7f4287eca740 1 -- start start 2024-07-31T15:00:56.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.330+0000 7f4287eca740 1 Processor -- start 2024-07-31T15:00:56.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.330+0000 7f4287eca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131efc1b0 0x5641322d9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.331+0000 7f4287eca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131f04390 0x5641322da3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.331+0000 7f427affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131efc1b0 0x5641322d9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.331+0000 7f427affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131efc1b0 0x5641322d9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44038/0 (socket says 172.21.15.67:44038) 2024-07-31T15:00:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.331+0000 7f427affd640 1 -- 172.21.15.67:0/340468017 learned_addr learned my addr 172.21.15.67:0/340468017 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.331+0000 7f4287eca740 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f25070 0x5641322da8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.331+0000 7f427a7fc640 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131f04390 0x5641322da3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.331+0000 7f4287eca740 1 -- 172.21.15.67:0/340468017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1b8380 con 0x564131f25070 2024-07-31T15:00:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.331+0000 7f4287eca740 1 -- 172.21.15.67:0/340468017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1b7bf0 con 0x564131f04390 2024-07-31T15:00:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.331+0000 7f4287eca740 1 -- 172.21.15.67:0/340468017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1b4e00 con 0x564131efc1b0 2024-07-31T15:00:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.332+0000 7f427b7fe640 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f25070 0x5641322da8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.332+0000 7f427affd640 1 -- 172.21.15.67:0/340468017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131f04390 msgr2=0x5641322da3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.332+0000 7f4287eca740 10 client.0 _ll_register_callbacks cb 0x7f426c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:00:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.332+0000 7f427affd640 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131f04390 0x5641322da3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.332+0000 7f427affd640 1 -- 172.21.15.67:0/340468017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f25070 msgr2=0x5641322da8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.332+0000 7f427affd640 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f25070 0x5641322da8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.332+0000 7f427affd640 1 -- 172.21.15.67:0/340468017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56413234fb00 con 0x564131efc1b0 2024-07-31T15:00:56.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.332+0000 7f427b7fe640 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f25070 0x5641322da8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:00:56.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.332+0000 7f4287eca740 1 -- 172.21.15.67:0/340468017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4274036040 con 0x564131efc1b0 2024-07-31T15:00:56.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.332+0000 7f427affd640 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131efc1b0 0x5641322d9e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f426c0a6860 tx=0x7f426c0cd340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64697bb800ce9a6e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.333+0000 7f425affd640 10 client.27265 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:00:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.333+0000 7f425affd640 1 -- 172.21.15.67:0/340468017 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c01d040 con 0x564131efc1b0 2024-07-31T15:00:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.333+0000 7f425affd640 1 -- 172.21.15.67:0/340468017 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9455+0+0 (secure 0 0 0) 0x7f426c0a9c50 con 0x564131efc1b0 2024-07-31T15:00:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.333+0000 7f425affd640 1 -- 172.21.15.67:0/340468017 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f426c0aede0 con 0x564131efc1b0 2024-07-31T15:00:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.333+0000 7f4287eca740 10 client.27265 fetch_fsmap learned FSMap version 57 2024-07-31T15:00:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.333+0000 7f4287eca740 1 -- 172.21.15.67:0/340468017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=57}) -- 0x564132350b90 con 0x564131efc1b0 2024-07-31T15:00:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.334+0000 7f425affd640 1 -- 172.21.15.67:0/340468017 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 57) ==== 8398+0+0 (secure 0 0 0) 0x7f426c0cbce0 con 0x564131efc1b0 2024-07-31T15:00:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.334+0000 7f4287eca740 10 client.27265 fetch_fsmap finished waiting for FSMap version 57 2024-07-31T15:00:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.334+0000 7f4287eca740 10 client.27265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:00:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.334+0000 7f4287eca740 20 client.27265 populate_metadata read hostname 'smithi067' 2024-07-31T15:00:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.334+0000 7f4287eca740 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564131eb8da0 0x5641323530b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.334+0000 7f4287eca740 4 client.27265 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:00:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.334+0000 7f4287eca740 1 -- 172.21.15.67:0/340468017 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56413223e550 con 0x564131eb8da0 2024-07-31T15:00:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.334+0000 7f427a7fc640 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564131eb8da0 0x5641323530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.335+0000 7f425affd640 10 client.27265 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.335+0000 7f427a7fc640 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564131eb8da0 0x5641323530b0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdf89fb4c39aeb70 server_cookie=cb4b7cea64c17c4e in_seq=0 out_seq=0 2024-07-31T15:00:56.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.337+0000 7f425affd640 1 -- 172.21.15.67:0/340468017 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56413223e550 con 0x564131eb8da0 2024-07-31T15:00:56.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.337+0000 7f425affd640 10 client.27265 handle_command_reply: tid=0 2024-07-31T15:00:56.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.337+0000 7f4287eca740 1 client.27265 shutdown 2024-07-31T15:00:56.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.338+0000 7f4287eca740 1 -- 172.21.15.67:0/340468017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131efc1b0 msgr2=0x5641322d9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.338+0000 7f4287eca740 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131efc1b0 0x5641322d9e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f426c0a6860 tx=0x7f426c0cd340 comp rx=0 tx=0).stop 2024-07-31T15:00:56.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.338+0000 7f4287eca740 1 -- 172.21.15.67:0/340468017 shutdown_connections 2024-07-31T15:00:56.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.338+0000 7f4287eca740 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564131eb8da0 0x5641323530b0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.338+0000 7f425affd640 0 client.27265 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:56.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.338+0000 7f4287eca740 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564131f25070 0x5641322da8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.338+0000 7f4287eca740 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564131f04390 0x5641322da3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.338+0000 7f4287eca740 1 --2- 172.21.15.67:0/340468017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564131efc1b0 0x5641322d9e60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.338+0000 7f4287eca740 1 -- 172.21.15.67:0/340468017 >> 172.21.15.67:0/340468017 conn(0x564131f19460 msgr2=0x5641322cb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:56.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.339+0000 7f4287eca740 1 -- 172.21.15.67:0/340468017 shutdown_connections 2024-07-31T15:00:56.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.339+0000 7f4287eca740 1 -- 172.21.15.67:0/340468017 wait complete. 2024-07-31T15:00:56.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.339+0000 7f4287eca740 20 client.27265 trim_cache size 0 max 16384 2024-07-31T15:00:56.342 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:00:56.342 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-31T15:00:56.342 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T15:00:56.342 INFO:teuthology.orchestra.run.smithi067.stdout: "c791f779-17a0-47ec-8943-a777d1eb6ea8": { 2024-07-31T15:00:56.342 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T15:00:56.342 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "c791f779-17a0-47ec-8943-a777d1eb6ea8", 2024-07-31T15:00:56.342 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T15:00:56.342 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:00:56.343 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:00:56.343 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:00:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.340+0000 7f4281979640 1 -- 172.21.15.67:0/1784895726 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42640b3cd0 msgr2=0x7f42640b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.340+0000 7f4281979640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42640b3cd0 0x7f42640b61e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f427c115bd0 tx=0x7f4270009110 comp rx=0 tx=0).stop 2024-07-31T15:00:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.340+0000 7f4281979640 1 -- 172.21.15.67:0/1784895726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427c075800 msgr2=0x7f427c1a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.340+0000 7f4281979640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427c075800 0x7f427c1a04a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f426c00c890 tx=0x7f426c00cc90 comp rx=0 tx=0).stop 2024-07-31T15:00:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.340+0000 7f4281979640 1 -- 172.21.15.67:0/1784895726 shutdown_connections 2024-07-31T15:00:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.340+0000 7f4281979640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42640b3cd0 0x7f42640b61e0 secure :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f427c115bd0 tx=0x7f4270009110 comp rx=0 tx=0).stop 2024-07-31T15:00:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.340+0000 7f4281979640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f427c113e80 0x7f427c1a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.340+0000 7f4281979640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f427c076db0 0x7f427c1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.340+0000 7f4281979640 1 --2- 172.21.15.67:0/1784895726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427c075800 0x7f427c1a04a0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.340+0000 7f4281979640 1 -- 172.21.15.67:0/1784895726 >> 172.21.15.67:0/1784895726 conn(0x7f427c0fefe0 msgr2=0x7f427c10fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:56.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.341+0000 7f4281979640 1 -- 172.21.15.67:0/1784895726 shutdown_connections 2024-07-31T15:00:56.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:00:56.341+0000 7f4281979640 1 -- 172.21.15.67:0/1784895726 wait complete. 2024-07-31T15:00:56.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:56 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:00:56.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:56 smithi077 ceph-mon[37327]: pgmap v703: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 600 KiB/s wr, 1.35k op/s 2024-07-31T15:00:56.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:56 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:00:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:56 smithi073 ceph-mon[36714]: pgmap v703: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 600 KiB/s wr, 1.35k op/s 2024-07-31T15:00:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.701+0000 7fcbd6ffd640 1 -- 172.21.15.67:0/645501264 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80318a0 con 0x7fcbd8107680 2024-07-31T15:00:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.702+0000 7fcbde3c8640 1 -- 172.21.15.67:0/645501264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd8107680 msgr2=0x7fcbd8107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.702+0000 7fcbde3c8640 1 --2- 172.21.15.67:0/645501264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd8107680 0x7fcbd8107ea0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009900 tx=0x7fcbc802a440 comp rx=0 tx=0).stop 2024-07-31T15:00:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.702+0000 7fcbde3c8640 1 -- 172.21.15.67:0/645501264 shutdown_connections 2024-07-31T15:00:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.702+0000 7fcbde3c8640 1 --2- 172.21.15.67:0/645501264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd8107680 0x7fcbd8107ea0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.702+0000 7fcbde3c8640 1 --2- 172.21.15.67:0/645501264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbd8106d30 0x7fcbd8107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.702+0000 7fcbde3c8640 1 --2- 172.21.15.67:0/645501264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd8101a50 0x7fcbd8101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.702+0000 7fcbde3c8640 1 -- 172.21.15.67:0/645501264 >> 172.21.15.67:0/645501264 conn(0x7fcbd8079880 msgr2=0x7fcbd8079c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.702+0000 7fcbde3c8640 1 -- 172.21.15.67:0/645501264 shutdown_connections 2024-07-31T15:00:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.702+0000 7fcbde3c8640 1 -- 172.21.15.67:0/645501264 wait complete. 2024-07-31T15:00:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.703+0000 7fcbde3c8640 1 Processor -- start 2024-07-31T15:00:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.705+0000 7fcbde3c8640 1 -- start start 2024-07-31T15:00:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.706+0000 7fcbde3c8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd8101a50 0x7fcbd81a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.706+0000 7fcbde3c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbd8106d30 0x7fcbd81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.706+0000 7fcbde3c8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd8107680 0x7fcbd81a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.706+0000 7fcbde3c8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd8119c10 con 0x7fcbd8107680 2024-07-31T15:00:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.706+0000 7fcbde3c8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd8119a90 con 0x7fcbd8106d30 2024-07-31T15:00:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.706+0000 7fcbde3c8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd8119d90 con 0x7fcbd8101a50 2024-07-31T15:00:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.706+0000 7fcbdc965640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd8107680 0x7fcbd81a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.706+0000 7fcbdc965640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd8107680 0x7fcbd81a5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47828/0 (socket says 172.21.15.67:47828) 2024-07-31T15:00:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.706+0000 7fcbdc965640 1 -- 172.21.15.67:0/3782949872 learned_addr learned my addr 172.21.15.67:0/3782949872 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.706+0000 7fcbd77fe640 1 --2- 172.21.15.67:0/3782949872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbd8106d30 0x7fcbd81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.707+0000 7fcbd77fe640 1 -- 172.21.15.67:0/3782949872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd8101a50 msgr2=0x7fcbd81a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.707+0000 7fcbd77fe640 1 --2- 172.21.15.67:0/3782949872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd8101a50 0x7fcbd81a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.707+0000 7fcbd77fe640 1 -- 172.21.15.67:0/3782949872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd8107680 msgr2=0x7fcbd81a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.707+0000 7fcbd77fe640 1 --2- 172.21.15.67:0/3782949872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd8107680 0x7fcbd81a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.707+0000 7fcbd77fe640 1 -- 172.21.15.67:0/3782949872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd81a6490 con 0x7fcbd8106d30 2024-07-31T15:00:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.709+0000 7fcbd77fe640 1 --2- 172.21.15.67:0/3782949872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbd8106d30 0x7fcbd81a0b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8075360 tx=0x7fcbcc00cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.709+0000 7fcbd57fa640 1 -- 172.21.15.67:0/3782949872 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc007bb0 con 0x7fcbd8106d30 2024-07-31T15:00:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.709+0000 7fcbd57fa640 1 -- 172.21.15.67:0/3782949872 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbcc007d50 con 0x7fcbd8106d30 2024-07-31T15:00:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.709+0000 7fcbde3c8640 1 -- 172.21.15.67:0/3782949872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd81a6780 con 0x7fcbd8106d30 2024-07-31T15:00:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.709+0000 7fcbde3c8640 1 -- 172.21.15.67:0/3782949872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd81acfa0 con 0x7fcbd8106d30 2024-07-31T15:00:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.710+0000 7fcbd57fa640 1 -- 172.21.15.67:0/3782949872 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00e6e0 con 0x7fcbd8106d30 2024-07-31T15:00:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.713+0000 7fcbd57fa640 1 -- 172.21.15.67:0/3782949872 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fcbcc00e880 con 0x7fcbd8106d30 2024-07-31T15:00:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.713+0000 7fcbe5397740 1 Processor -- start 2024-07-31T15:00:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.713+0000 7fcbe5397740 1 -- start start 2024-07-31T15:00:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.713+0000 7fcbe5397740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffc71b0 0x55acc0369ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.713+0000 7fcbe5397740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff9e2f0 0x55acc036a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.713+0000 7fcbe5397740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbffa64d0 0x55acc036a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.713+0000 7fcbe5397740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55acc02df3b0 con 0x55acbff9e2f0 2024-07-31T15:00:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.713+0000 7fcbe5397740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55acc02ce8b0 con 0x55acbffc71b0 2024-07-31T15:00:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.713+0000 7fcbe5397740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55acc02d28f0 con 0x55acbffa64d0 2024-07-31T15:00:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.714+0000 7fcbd77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbffa64d0 0x55acc036a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.714+0000 7fcbd7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff9e2f0 0x55acc036a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.714+0000 7fcbd77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbffa64d0 0x55acc036a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44068/0 (socket says 172.21.15.67:44068) 2024-07-31T15:00:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.714+0000 7fcbd77fe640 1 -- 172.21.15.67:0/3262201213 learned_addr learned my addr 172.21.15.67:0/3262201213 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.714+0000 7fcbdc965640 1 --2- 172.21.15.67:0/3262201213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffc71b0 0x55acc0369ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.714+0000 7fcbd77fe640 1 -- 172.21.15.67:0/3262201213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffc71b0 msgr2=0x55acc0369ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.714+0000 7fcbd77fe640 1 --2- 172.21.15.67:0/3262201213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffc71b0 0x55acc0369ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.714+0000 7fcbd77fe640 1 -- 172.21.15.67:0/3262201213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff9e2f0 msgr2=0x55acc036a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.714+0000 7fcbd77fe640 1 --2- 172.21.15.67:0/3262201213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff9e2f0 0x55acc036a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.714+0000 7fcbd77fe640 1 -- 172.21.15.67:0/3262201213 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acc02759a0 con 0x55acbffa64d0 2024-07-31T15:00:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.714+0000 7fcbd77fe640 1 --2- 172.21.15.67:0/3262201213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbffa64d0 0x55acc036a760 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0ec2f0 tx=0x7fcbcc0e7f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97aed3c9ceebfd86 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.715+0000 7fcbd57fa640 1 --2- 172.21.15.67:0/3782949872 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcbc00b3f50 0x7fcbc00b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.715+0000 7fcbbe7fc640 1 -- 172.21.15.67:0/3262201213 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc0f7070 con 0x55acbffa64d0 2024-07-31T15:00:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.715+0000 7fcbdc965640 1 --2- 172.21.15.67:0/3782949872 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcbc00b3f50 0x7fcbc00b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.715+0000 7fcbd57fa640 1 -- 172.21.15.67:0/3782949872 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9455+0+0 (secure 0 0 0) 0x7fcbcc0e42d0 con 0x7fcbd8106d30 2024-07-31T15:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.715+0000 7fcbbe7fc640 1 -- 172.21.15.67:0/3262201213 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbcc0e64e0 con 0x55acbffa64d0 2024-07-31T15:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.715+0000 7fcbbe7fc640 1 -- 172.21.15.67:0/3262201213 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc0d5a30 con 0x55acbffa64d0 2024-07-31T15:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.716+0000 7fcbe5397740 1 -- 172.21.15.67:0/3262201213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbffa64d0 msgr2=0x55acc036a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.716+0000 7fcbe5397740 1 --2- 172.21.15.67:0/3262201213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbffa64d0 0x55acc036a760 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0ec2f0 tx=0x7fcbcc0e7f90 comp rx=0 tx=0).stop 2024-07-31T15:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.717+0000 7fcbdc965640 1 --2- 172.21.15.67:0/3782949872 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcbc00b3f50 0x7fcbc00b6460 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8005dc0 tx=0x7fcbc8005d10 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.717+0000 7fcbe5397740 1 -- 172.21.15.67:0/3262201213 shutdown_connections 2024-07-31T15:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.717+0000 7fcbe5397740 1 --2- 172.21.15.67:0/3262201213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbffa64d0 0x55acc036a760 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.717+0000 7fcbe5397740 1 --2- 172.21.15.67:0/3262201213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff9e2f0 0x55acc036a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.717+0000 7fcbe5397740 1 --2- 172.21.15.67:0/3262201213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffc71b0 0x55acc0369ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.717+0000 7fcbe5397740 1 -- 172.21.15.67:0/3262201213 >> 172.21.15.67:0/3262201213 conn(0x55acbffbb300 msgr2=0x55acc0367bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.717+0000 7fcbe5397740 1 -- 172.21.15.67:0/3262201213 shutdown_connections 2024-07-31T15:00:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.717+0000 7fcbe5397740 1 -- 172.21.15.67:0/3262201213 wait complete. 2024-07-31T15:00:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.718+0000 7fcbe5397740 1 -- start start 2024-07-31T15:00:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.718+0000 7fcbe5397740 1 Processor -- start 2024-07-31T15:00:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.719+0000 7fcbe5397740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 0x55acc0375780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.719+0000 7fcbe5397740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 0x55acc0375cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.719+0000 7fcbe5397740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbffc71b0 0x55acc03817e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.719+0000 7fcbe5397740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55acc0208ab0 con 0x55acbffc71b0 2024-07-31T15:00:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.719+0000 7fcbdc965640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbffc71b0 0x55acc03817e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.720+0000 7fcbdc965640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbffc71b0 0x55acc03817e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47842/0 (socket says 172.21.15.67:47842) 2024-07-31T15:00:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.720+0000 7fcbd77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 0x55acc0375cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.720+0000 7fcbdc965640 1 -- 172.21.15.67:0/172359799 learned_addr learned my addr 172.21.15.67:0/172359799 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.720+0000 7fcbe5397740 1 -- 172.21.15.67:0/172359799 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55acc0256cf0 con 0x55acbffa64d0 2024-07-31T15:00:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.720+0000 7fcbe5397740 1 -- 172.21.15.67:0/172359799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55acc0378460 con 0x55acbff9e2f0 2024-07-31T15:00:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.720+0000 7fcbe5397740 10 client.0 _ll_register_callbacks cb 0x55acc02c89f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:00:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.720+0000 7fcbdc965640 1 -- 172.21.15.67:0/172359799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 msgr2=0x55acc0375780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.720+0000 7fcbdc965640 1 --2- 172.21.15.67:0/172359799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 0x55acc0375780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.720+0000 7fcbdc965640 1 -- 172.21.15.67:0/172359799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 msgr2=0x55acc0375cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.720+0000 7fcbdc965640 1 --2- 172.21.15.67:0/172359799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 0x55acc0375cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.720+0000 7fcbdc965640 1 -- 172.21.15.67:0/172359799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acc0032790 con 0x55acbffc71b0 2024-07-31T15:00:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.721+0000 7fcbdc965640 1 --2- 172.21.15.67:0/172359799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbffc71b0 0x55acc03817e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8031440 tx=0x7fcbc8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de047f78fa8b3833 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.721+0000 7fcbb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:00:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.721+0000 7fcbb77fe640 1 -- 172.21.15.67:0/172359799 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8035070 con 0x55acbffc71b0 2024-07-31T15:00:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.721+0000 7fcbb77fe640 1 -- 172.21.15.67:0/172359799 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9455+0+0 (secure 0 0 0) 0x7fcbc8031e00 con 0x55acbffc71b0 2024-07-31T15:00:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.722+0000 7fcbe5397740 1 -- 172.21.15.67:0/172359799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55acc02e1300 con 0x55acbffc71b0 2024-07-31T15:00:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.722+0000 7fcbb77fe640 1 -- 172.21.15.67:0/172359799 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fcbc8050470 con 0x55acbffc71b0 2024-07-31T15:00:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.722+0000 7fcbe5397740 10 client.17727 fetch_fsmap learned FSMap version 57 2024-07-31T15:00:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.722+0000 7fcbe5397740 1 -- 172.21.15.67:0/172359799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55acc00ab620 con 0x55acbffc71b0 2024-07-31T15:00:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.722+0000 7fcbb77fe640 1 -- 172.21.15.67:0/172359799 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbc804b070 con 0x55acbffc71b0 2024-07-31T15:00:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.723+0000 7fcbb77fe640 1 -- 172.21.15.67:0/172359799 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 57) ==== 8398+0+0 (secure 0 0 0) 0x7fcbc8043030 con 0x55acbffc71b0 2024-07-31T15:00:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.723+0000 7fcbe5397740 10 client.17727 fetch_fsmap finished waiting for FSMap version 57 2024-07-31T15:00:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.723+0000 7fcbe5397740 10 client.17727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:00:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.723+0000 7fcbe5397740 20 client.17727 populate_metadata read hostname 'smithi067' 2024-07-31T15:00:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.723+0000 7fcbe5397740 1 --2- 172.21.15.67:0/172359799 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55acbff85d80 0x55acc03878f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.723+0000 7fcbe5397740 4 client.17727 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:00:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.723+0000 7fcbe5397740 1 -- 172.21.15.67:0/172359799 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acc02e1300 con 0x55acbff85d80 2024-07-31T15:00:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.724+0000 7fcbd7fff640 1 --2- 172.21.15.67:0/172359799 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55acbff85d80 0x55acc03878f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.725+0000 7fcbd7fff640 1 --2- 172.21.15.67:0/172359799 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55acbff85d80 0x55acc03878f0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e2aa44b2a909fb server_cookie=a6cb36d433cf0d8f in_seq=0 out_seq=0 2024-07-31T15:00:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.725+0000 7fcbb77fe640 10 client.17727 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.727+0000 7fcbb77fe640 1 -- 172.21.15.67:0/172359799 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55acc02e1300 con 0x55acbff85d80 2024-07-31T15:00:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.727+0000 7fcbb77fe640 10 client.17727 handle_command_reply: tid=0 2024-07-31T15:00:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.727+0000 7fcbe5397740 1 client.17727 shutdown 2024-07-31T15:00:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.728+0000 7fcbe5397740 1 -- 172.21.15.67:0/172359799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbffc71b0 msgr2=0x55acc03817e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.728+0000 7fcbe5397740 1 --2- 172.21.15.67:0/172359799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbffc71b0 0x55acc03817e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8031440 tx=0x7fcbc8009260 comp rx=0 tx=0).stop 2024-07-31T15:00:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.728+0000 7fcbe5397740 1 -- 172.21.15.67:0/172359799 shutdown_connections 2024-07-31T15:00:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.728+0000 7fcbe5397740 1 --2- 172.21.15.67:0/172359799 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55acbff85d80 0x55acc03878f0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.728+0000 7fcbe5397740 1 --2- 172.21.15.67:0/172359799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbffc71b0 0x55acc03817e0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.728+0000 7fcbe5397740 1 --2- 172.21.15.67:0/172359799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 0x55acc0375cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.728+0000 7fcbb77fe640 0 client.17727 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.728+0000 7fcbe5397740 1 --2- 172.21.15.67:0/172359799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 0x55acc0375780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.728+0000 7fcbe5397740 1 -- 172.21.15.67:0/172359799 >> 172.21.15.67:0/172359799 conn(0x55acbffbb300 msgr2=0x55acc036ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.728+0000 7fcbe5397740 1 -- 172.21.15.67:0/172359799 shutdown_connections 2024-07-31T15:00:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.728+0000 7fcbe5397740 1 -- 172.21.15.67:0/172359799 wait complete. 2024-07-31T15:00:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.728+0000 7fcbe5397740 20 client.17727 trim_cache size 0 max 16384 2024-07-31T15:00:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.765+0000 7fcbe5397740 1 Processor -- start 2024-07-31T15:00:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.765+0000 7fcbe5397740 1 -- start start 2024-07-31T15:00:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.765+0000 7fcbe5397740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff85d80 0x55acc03815d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.765+0000 7fcbe5397740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 0x55acc0381b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.765+0000 7fcbe5397740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 0x55acc036ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.765+0000 7fcbe5397740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc400ffa0 con 0x55acbff85d80 2024-07-31T15:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.765+0000 7fcbe5397740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4011bd0 con 0x55acbffa64d0 2024-07-31T15:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.765+0000 7fcbe5397740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4010670 con 0x55acbff9e2f0 2024-07-31T15:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.765+0000 7fcbd7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff85d80 0x55acc03815d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.766+0000 7fcbd7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff85d80 0x55acc03815d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47858/0 (socket says 172.21.15.67:47858) 2024-07-31T15:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.766+0000 7fcbd7fff640 1 -- 172.21.15.67:0/3774295602 learned_addr learned my addr 172.21.15.67:0/3774295602 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.766+0000 7fcbd77fe640 1 --2- 172.21.15.67:0/3774295602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 0x55acc0381b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.766+0000 7fcbdc965640 1 --2- 172.21.15.67:0/3774295602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 0x55acc036ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.766+0000 7fcbd77fe640 1 -- 172.21.15.67:0/3774295602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 msgr2=0x55acc036ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.766+0000 7fcbd77fe640 1 --2- 172.21.15.67:0/3774295602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 0x55acc036ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.766+0000 7fcbd77fe640 1 -- 172.21.15.67:0/3774295602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff85d80 msgr2=0x55acc03815d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.766+0000 7fcbd77fe640 1 --2- 172.21.15.67:0/3774295602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff85d80 0x55acc03815d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.766+0000 7fcbd77fe640 1 -- 172.21.15.67:0/3774295602 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acc036aac0 con 0x55acbff9e2f0 2024-07-31T15:00:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.766+0000 7fcbd77fe640 1 --2- 172.21.15.67:0/3774295602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 0x55acc0381b10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0ec2f0 tx=0x7fcbcc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7be4d7e9eaa6ba9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.766+0000 7fcbbdffb640 1 -- 172.21.15.67:0/3774295602 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc0dc990 con 0x55acbff9e2f0 2024-07-31T15:00:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.767+0000 7fcbbdffb640 1 -- 172.21.15.67:0/3774295602 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbcc0aa5f0 con 0x55acbff9e2f0 2024-07-31T15:00:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.767+0000 7fcbbdffb640 1 -- 172.21.15.67:0/3774295602 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc0d9920 con 0x55acbff9e2f0 2024-07-31T15:00:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.768+0000 7fcbe5397740 1 -- 172.21.15.67:0/3774295602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 msgr2=0x55acc0381b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.768+0000 7fcbe5397740 1 --2- 172.21.15.67:0/3774295602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 0x55acc0381b10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0ec2f0 tx=0x7fcbcc000bc0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.768+0000 7fcbe5397740 1 -- 172.21.15.67:0/3774295602 shutdown_connections 2024-07-31T15:00:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.768+0000 7fcbe5397740 1 --2- 172.21.15.67:0/3774295602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 0x55acc036ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.768+0000 7fcbe5397740 1 --2- 172.21.15.67:0/3774295602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 0x55acc0381b10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.768+0000 7fcbe5397740 1 --2- 172.21.15.67:0/3774295602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff85d80 0x55acc03815d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.768+0000 7fcbe5397740 1 -- 172.21.15.67:0/3774295602 >> 172.21.15.67:0/3774295602 conn(0x55acbffbb300 msgr2=0x55acc0373dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.768+0000 7fcbe5397740 1 -- 172.21.15.67:0/3774295602 shutdown_connections 2024-07-31T15:00:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.768+0000 7fcbe5397740 1 -- 172.21.15.67:0/3774295602 wait complete. 2024-07-31T15:00:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.769+0000 7fcbe5397740 1 -- start start 2024-07-31T15:00:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.769+0000 7fcbe5397740 1 Processor -- start 2024-07-31T15:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.769+0000 7fcbe5397740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff85d80 0x55acc037df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.769+0000 7fcbe5397740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 0x55acc037a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbe5397740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 0x55acc037ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbe5397740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4011500 con 0x55acbff85d80 2024-07-31T15:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbe5397740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4010d70 con 0x55acbffa64d0 2024-07-31T15:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbe5397740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc400df80 con 0x55acbff9e2f0 2024-07-31T15:00:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbd77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 0x55acc037a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbd77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 0x55acc037a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44100/0 (socket says 172.21.15.67:44100) 2024-07-31T15:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbd77fe640 1 -- 172.21.15.67:0/159830906 learned_addr learned my addr 172.21.15.67:0/159830906 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbdc965640 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 0x55acc037ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbd7fff640 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff85d80 0x55acc037df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbd77fe640 1 -- 172.21.15.67:0/159830906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 msgr2=0x55acc037ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbd77fe640 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 0x55acc037ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbd77fe640 1 -- 172.21.15.67:0/159830906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff85d80 msgr2=0x55acc037df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbd77fe640 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff85d80 0x55acc037df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbd77fe640 1 -- 172.21.15.67:0/159830906 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acc037b4f0 con 0x55acbff9e2f0 2024-07-31T15:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbdc965640 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 0x55acc037ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbd7fff640 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff85d80 0x55acc037df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:00:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbe5397740 10 client.0 _ll_register_callbacks cb 0x7fcbc8004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbe5397740 1 -- 172.21.15.67:0/159830906 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbcc0aa5f0 con 0x55acbff9e2f0 2024-07-31T15:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbd77fe640 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 0x55acc037a710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0aaa70 tx=0x7fcbcc00a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5156bc50e5713c72 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.770+0000 7fcbb77fe640 10 client.27277 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.771+0000 7fcbb77fe640 1 -- 172.21.15.67:0/159830906 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc0d9af0 con 0x55acbff9e2f0 2024-07-31T15:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.771+0000 7fcbb77fe640 1 -- 172.21.15.67:0/159830906 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9455+0+0 (secure 0 0 0) 0x7fcbcc0a70d0 con 0x55acbff9e2f0 2024-07-31T15:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.771+0000 7fcbb77fe640 1 -- 172.21.15.67:0/159830906 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fcbcc0d85e0 con 0x55acbff9e2f0 2024-07-31T15:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.771+0000 7fcbe5397740 10 client.27277 fetch_fsmap learned FSMap version 57 2024-07-31T15:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.771+0000 7fcbe5397740 1 -- 172.21.15.67:0/159830906 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=57}) -- 0x55acc03eef00 con 0x55acbff9e2f0 2024-07-31T15:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.772+0000 7fcbb77fe640 1 -- 172.21.15.67:0/159830906 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 57) ==== 8398+0+0 (secure 0 0 0) 0x7fcbcc0de6e0 con 0x55acbff9e2f0 2024-07-31T15:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.772+0000 7fcbe5397740 10 client.27277 fetch_fsmap finished waiting for FSMap version 57 2024-07-31T15:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.772+0000 7fcbe5397740 10 client.27277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:00:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.772+0000 7fcbe5397740 20 client.27277 populate_metadata read hostname 'smithi067' 2024-07-31T15:00:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.772+0000 7fcbe5397740 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55acbffc71b0 0x55acc03f1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:00:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.772+0000 7fcbe5397740 4 client.27277 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:00:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.772+0000 7fcbe5397740 1 -- 172.21.15.67:0/159830906 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fcbcc0aa5f0 con 0x55acbffc71b0 2024-07-31T15:00:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.772+0000 7fcbd7fff640 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55acbffc71b0 0x55acc03f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:00:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.774+0000 7fcbb77fe640 10 client.27277 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:56.774+0000 7fcbd7fff640 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55acbffc71b0 0x55acc03f1130 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89fc4c877e19cec0 server_cookie=ef9affb713ac57cf in_seq=0 out_seq=0 2024-07-31T15:00:56.884 INFO:tasks.workunit.client.0.smithi067.stdout:# 2024-07-31T15:00:56.884 INFO:tasks.workunit.client.0.smithi067.stdout:# configuration written to .config 2024-07-31T15:00:56.884 INFO:tasks.workunit.client.0.smithi067.stdout:# 2024-07-31T15:00:56.887 INFO:tasks.workunit.client.0.smithi067.stderr:++ grep -c processor /proc/cpuinfo 2024-07-31T15:00:56.888 INFO:tasks.workunit.client.0.smithi067.stderr:+ make -j8 2024-07-31T15:00:57.199 INFO:tasks.cephfs.filesystem:scrub status for tag:c791f779-17a0-47ec-8943-a777d1eb6ea8 - {'path': '/', 'tag': 'c791f779-17a0-47ec-8943-a777d1eb6ea8', 'options': 'recursive,force'} 2024-07-31T15:00:58.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:00:58 smithi077 ceph-mon[37327]: pgmap v704: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 557 KiB/s wr, 1.34k op/s 2024-07-31T15:00:59.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:00:58 smithi067 ceph-mon[32750]: pgmap v704: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 557 KiB/s wr, 1.34k op/s 2024-07-31T15:00:59.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:00:58 smithi073 ceph-mon[36714]: pgmap v704: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 557 KiB/s wr, 1.34k op/s 2024-07-31T15:00:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.434+0000 7fcbb77fe640 1 -- 172.21.15.67:0/159830906 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fcbcc0aa5f0 con 0x55acbffc71b0 2024-07-31T15:00:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.434+0000 7fcbb77fe640 10 client.27277 handle_command_reply: tid=0 2024-07-31T15:00:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.434+0000 7fcbe5397740 1 client.27277 shutdown 2024-07-31T15:00:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.435+0000 7fcbe5397740 1 -- 172.21.15.67:0/159830906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 msgr2=0x55acc037a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.435+0000 7fcbe5397740 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 0x55acc037a710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0aaa70 tx=0x7fcbcc00a530 comp rx=0 tx=0).stop 2024-07-31T15:00:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.435+0000 7fcbe5397740 1 -- 172.21.15.67:0/159830906 shutdown_connections 2024-07-31T15:00:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.435+0000 7fcbe5397740 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55acbffc71b0 0x55acc03f1130 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.435+0000 7fcbe5397740 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acbffa64d0 0x55acc037ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.435+0000 7fcbe5397740 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55acbff9e2f0 0x55acc037a710 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.435+0000 7fcbe5397740 1 --2- 172.21.15.67:0/159830906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55acbff85d80 0x55acc037df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.435+0000 7fcbb77fe640 0 client.27277 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:00:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.435+0000 7fcbe5397740 1 -- 172.21.15.67:0/159830906 >> 172.21.15.67:0/159830906 conn(0x55acbffbb300 msgr2=0x55acc036aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.435+0000 7fcbe5397740 1 -- 172.21.15.67:0/159830906 shutdown_connections 2024-07-31T15:00:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.435+0000 7fcbe5397740 1 -- 172.21.15.67:0/159830906 wait complete. 2024-07-31T15:00:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.435+0000 7fcbe5397740 20 client.27277 trim_cache size 0 max 16384 2024-07-31T15:00:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:00:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:00:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:00:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:00:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.436+0000 7fcbde3c8640 1 -- 172.21.15.67:0/3782949872 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcbc00b3f50 msgr2=0x7fcbc00b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.436+0000 7fcbde3c8640 1 --2- 172.21.15.67:0/3782949872 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcbc00b3f50 0x7fcbc00b6460 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8005dc0 tx=0x7fcbc8005d10 comp rx=0 tx=0).stop 2024-07-31T15:00:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.436+0000 7fcbde3c8640 1 -- 172.21.15.67:0/3782949872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbd8106d30 msgr2=0x7fcbd81a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:00:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.436+0000 7fcbde3c8640 1 --2- 172.21.15.67:0/3782949872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbd8106d30 0x7fcbd81a0b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8075360 tx=0x7fcbcc00cdd0 comp rx=0 tx=0).stop 2024-07-31T15:00:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.436+0000 7fcbde3c8640 1 -- 172.21.15.67:0/3782949872 shutdown_connections 2024-07-31T15:00:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.436+0000 7fcbde3c8640 1 --2- 172.21.15.67:0/3782949872 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcbc00b3f50 0x7fcbc00b6460 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.436+0000 7fcbde3c8640 1 --2- 172.21.15.67:0/3782949872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd8107680 0x7fcbd81a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.436+0000 7fcbde3c8640 1 --2- 172.21.15.67:0/3782949872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbd8106d30 0x7fcbd81a0b90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.436+0000 7fcbde3c8640 1 --2- 172.21.15.67:0/3782949872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbd8101a50 0x7fcbd81a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:00:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.436+0000 7fcbde3c8640 1 -- 172.21.15.67:0/3782949872 >> 172.21.15.67:0/3782949872 conn(0x7fcbd8079880 msgr2=0x7fcbd81005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:00:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.436+0000 7fcbde3c8640 1 -- 172.21.15.67:0/3782949872 shutdown_connections 2024-07-31T15:00:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:00:59.436+0000 7fcbde3c8640 1 -- 172.21.15.67:0/3782949872 wait complete. 2024-07-31T15:01:01.296 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:01 smithi077 ceph-mon[37327]: pgmap v705: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 391 KiB/s wr, 1.07k op/s 2024-07-31T15:01:01.296 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:01 smithi077 ceph-mon[37327]: osdmap e98: 12 total, 12 up, 12 in 2024-07-31T15:01:01.296 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:01 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:01:01.296 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:01 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:01.296 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:01 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:01.296 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:01 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:01.296 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:01 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:01.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:01 smithi067 ceph-mon[32750]: pgmap v705: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 391 KiB/s wr, 1.07k op/s 2024-07-31T15:01:01.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:01 smithi067 ceph-mon[32750]: osdmap e98: 12 total, 12 up, 12 in 2024-07-31T15:01:01.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:01 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:01:01.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:01 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:01.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:01 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:01.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:01 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:01.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:01 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:01.353 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:01 smithi073 ceph-mon[36714]: pgmap v705: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 391 KiB/s wr, 1.07k op/s 2024-07-31T15:01:01.353 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:01 smithi073 ceph-mon[36714]: osdmap e98: 12 total, 12 up, 12 in 2024-07-31T15:01:01.353 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:01 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:01:01.353 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:01 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:01.353 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:01 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:01.353 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:01 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:01.353 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:01 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:02.174 INFO:tasks.workunit.client.0.smithi067.stdout: GEN arch/x86/include/generated/asm/orc_hash.h 2024-07-31T15:01:02.189 INFO:tasks.workunit.client.0.smithi067.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_32.h 2024-07-31T15:01:02.189 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC arch/x86/tools/relocs_32.o 2024-07-31T15:01:02.195 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC arch/x86/tools/relocs_64.o 2024-07-31T15:01:02.200 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/generated/uapi/linux/version.h 2024-07-31T15:01:02.202 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC arch/x86/tools/relocs_common.o 2024-07-31T15:01:02.220 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/config/kernel.release 2024-07-31T15:01:02.227 INFO:tasks.workunit.client.0.smithi067.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_64.h 2024-07-31T15:01:02.228 INFO:tasks.workunit.client.0.smithi067.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_x32.h 2024-07-31T15:01:02.231 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/generated/compile.h 2024-07-31T15:01:02.238 INFO:tasks.workunit.client.0.smithi067.stdout: SYSTBL arch/x86/include/generated/asm/syscalls_32.h 2024-07-31T15:01:02.266 INFO:tasks.workunit.client.0.smithi067.stdout: SYSHDR arch/x86/include/generated/asm/unistd_32_ia32.h 2024-07-31T15:01:02.272 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/bpf_perf_event.h 2024-07-31T15:01:02.275 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/errno.h 2024-07-31T15:01:02.277 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/fcntl.h 2024-07-31T15:01:02.278 INFO:tasks.workunit.client.0.smithi067.stdout: SYSHDR arch/x86/include/generated/asm/unistd_64_x32.h 2024-07-31T15:01:02.279 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/ioctl.h 2024-07-31T15:01:02.283 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/ioctls.h 2024-07-31T15:01:02.286 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/ipcbuf.h 2024-07-31T15:01:02.290 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/param.h 2024-07-31T15:01:02.293 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/poll.h 2024-07-31T15:01:02.294 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/resource.h 2024-07-31T15:01:02.296 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/socket.h 2024-07-31T15:01:02.298 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/sockios.h 2024-07-31T15:01:02.299 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/termbits.h 2024-07-31T15:01:02.302 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/termios.h 2024-07-31T15:01:02.302 INFO:tasks.workunit.client.0.smithi067.stdout: SYSTBL arch/x86/include/generated/asm/syscalls_64.h 2024-07-31T15:01:02.303 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/types.h 2024-07-31T15:01:02.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:01:02.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:01:02.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:02.335 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/generated/utsrelease.h 2024-07-31T15:01:02.335 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kallsyms 2024-07-31T15:01:02.349 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/selinux/genheaders/genheaders 2024-07-31T15:01:02.355 INFO:tasks.workunit.client.0.smithi067.stdout: DESCEND objtool 2024-07-31T15:01:02.360 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/selinux/mdp/mdp 2024-07-31T15:01:02.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:01:02.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:01:02.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:02.418 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/early_ioremap.h 2024-07-31T15:01:02.418 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/export.h 2024-07-31T15:01:02.421 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/mcs_spinlock.h 2024-07-31T15:01:02.425 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/irq_regs.h 2024-07-31T15:01:02.429 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/kmap_size.h 2024-07-31T15:01:02.434 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/local64.h 2024-07-31T15:01:02.439 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/mmiowb.h 2024-07-31T15:01:02.442 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/module.lds.h 2024-07-31T15:01:02.447 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/rwonce.h 2024-07-31T15:01:02.453 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/unaligned.h 2024-07-31T15:01:02.468 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/sorttable 2024-07-31T15:01:02.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:01:02.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:01:02.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:01:02.546 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/asn1_compiler 2024-07-31T15:01:02.555 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTLD arch/x86/tools/relocs 2024-07-31T15:01:02.575 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep.o 2024-07-31T15:01:02.638 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTLD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep-in.o 2024-07-31T15:01:02.649 INFO:tasks.workunit.client.0.smithi067.stdout: LINK /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep 2024-07-31T15:01:02.705 INFO:tasks.workunit.client.0.smithi067.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/exec-cmd.h 2024-07-31T15:01:02.705 INFO:tasks.workunit.client.0.smithi067.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/help.h 2024-07-31T15:01:02.705 INFO:tasks.workunit.client.0.smithi067.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/pager.h 2024-07-31T15:01:02.706 INFO:tasks.workunit.client.0.smithi067.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/parse-options.h 2024-07-31T15:01:02.710 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/exec-cmd.o 2024-07-31T15:01:02.719 INFO:tasks.workunit.client.0.smithi067.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/run-command.h 2024-07-31T15:01:02.720 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/help.o 2024-07-31T15:01:02.722 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/pager.o 2024-07-31T15:01:02.724 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/parse-options.o 2024-07-31T15:01:02.730 INFO:tasks.workunit.client.0.smithi067.stdout: INSTALL libsubcmd_headers 2024-07-31T15:01:02.734 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/run-command.o 2024-07-31T15:01:03.010 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/sigchain.o 2024-07-31T15:01:03.041 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/subcmd-config.o 2024-07-31T15:01:03.266 INFO:tasks.workunit.client.0.smithi067.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/libsubcmd-in.o 2024-07-31T15:01:03.282 INFO:tasks.workunit.client.0.smithi067.stdout: AR /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/libsubcmd.a 2024-07-31T15:01:03.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:03 smithi073 ceph-mon[36714]: pgmap v707: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 840 KiB/s wr, 1.13k op/s 2024-07-31T15:01:03.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:03 smithi067 ceph-mon[32750]: pgmap v707: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 840 KiB/s wr, 1.13k op/s 2024-07-31T15:01:03.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:03 smithi077 ceph-mon[37327]: pgmap v707: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 840 KiB/s wr, 1.13k op/s 2024-07-31T15:01:04.224 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/weak.o 2024-07-31T15:01:04.224 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/check.o 2024-07-31T15:01:04.224 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/special.o 2024-07-31T15:01:04.225 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/builtin-check.o 2024-07-31T15:01:04.226 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/elf.o 2024-07-31T15:01:04.226 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool.o 2024-07-31T15:01:04.235 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/special.o 2024-07-31T15:01:04.361 INFO:tasks.workunit.client.0.smithi067.stdout: MKDIR /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/lib/ 2024-07-31T15:01:04.363 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/orc_gen.o 2024-07-31T15:01:04.366 INFO:tasks.workunit.client.0.smithi067.stdout: GEN /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/lib/inat-tables.c 2024-07-31T15:01:04.379 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/orc_dump.o 2024-07-31T15:01:04.398 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/decode.o 2024-07-31T15:01:04.402 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libstring.o 2024-07-31T15:01:04.438 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libctype.o 2024-07-31T15:01:04.480 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/str_error_r.o 2024-07-31T15:01:04.518 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/librbtree.o 2024-07-31T15:01:04.962 INFO:tasks.workunit.client.0.smithi067.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/objtool-in.o 2024-07-31T15:01:05.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:05 smithi067 ceph-mon[32750]: pgmap v708: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 724 KiB/s wr, 920 op/s 2024-07-31T15:01:05.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:05 smithi073 ceph-mon[36714]: pgmap v708: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 724 KiB/s wr, 920 op/s 2024-07-31T15:01:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:05 smithi077 ceph-mon[37327]: pgmap v708: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 724 KiB/s wr, 920 op/s 2024-07-31T15:01:06.242 INFO:tasks.workunit.client.0.smithi067.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool-in.o 2024-07-31T15:01:06.261 INFO:tasks.workunit.client.0.smithi067.stdout: LINK /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool 2024-07-31T15:01:07.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:07 smithi067 ceph-mon[32750]: pgmap v709: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1015 KiB/s wr, 786 op/s 2024-07-31T15:01:07.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:07 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:01:07.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:07 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:01:07.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:07 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:01:07.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:07 smithi073 ceph-mon[36714]: pgmap v709: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1015 KiB/s wr, 786 op/s 2024-07-31T15:01:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:07 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:01:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:07 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:01:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:07 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:01:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:07 smithi077 ceph-mon[37327]: pgmap v709: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1015 KiB/s wr, 786 op/s 2024-07-31T15:01:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:07 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:01:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:07 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:01:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:07 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:01:08.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:08 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:01:08.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:08 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:01:08.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:08 smithi067 ceph-mon[32750]: osdmap e99: 12 total, 12 up, 12 in 2024-07-31T15:01:08.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:08 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:01:08.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:08 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:01:08.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:08 smithi073 ceph-mon[36714]: osdmap e99: 12 total, 12 up, 12 in 2024-07-31T15:01:08.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:08 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:01:08.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:08 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:01:08.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:08 smithi077 ceph-mon[37327]: osdmap e99: 12 total, 12 up, 12 in 2024-07-31T15:01:09.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:09 smithi073 ceph-mon[36714]: pgmap v711: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.1 MiB/s wr, 619 op/s 2024-07-31T15:01:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:09 smithi077 ceph-mon[37327]: pgmap v711: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.1 MiB/s wr, 619 op/s 2024-07-31T15:01:09.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:09 smithi067 ceph-mon[32750]: pgmap v711: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.1 MiB/s wr, 619 op/s 2024-07-31T15:01:11.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:11 smithi067 ceph-mon[32750]: pgmap v712: 129 pgs: 27 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 92 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 939 KiB/s wr, 380 op/s 2024-07-31T15:01:11.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:11 smithi067 ceph-mon[32750]: osdmap e100: 12 total, 12 up, 12 in 2024-07-31T15:01:11.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:11 smithi073 ceph-mon[36714]: pgmap v712: 129 pgs: 27 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 92 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 939 KiB/s wr, 380 op/s 2024-07-31T15:01:11.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:11 smithi073 ceph-mon[36714]: osdmap e100: 12 total, 12 up, 12 in 2024-07-31T15:01:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:11 smithi077 ceph-mon[37327]: pgmap v712: 129 pgs: 27 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 92 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 939 KiB/s wr, 380 op/s 2024-07-31T15:01:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:11 smithi077 ceph-mon[37327]: osdmap e100: 12 total, 12 up, 12 in 2024-07-31T15:01:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.617+0000 7f0575fab640 1 -- 172.21.15.67:0/2183873858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05680a53c0 msgr2=0x7f05680a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.617+0000 7f0575fab640 1 --2- 172.21.15.67:0/2183873858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05680a53c0 0x7f05680a57d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f055c009860 tx=0x7f055c02a340 comp rx=0 tx=0).stop 2024-07-31T15:01:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.623+0000 7f0575fab640 1 -- 172.21.15.67:0/2183873858 shutdown_connections 2024-07-31T15:01:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.623+0000 7f0575fab640 1 --2- 172.21.15.67:0/2183873858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05680ab640 0x7f05680b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.623+0000 7f0575fab640 1 --2- 172.21.15.67:0/2183873858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05680aacf0 0x7f05680ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.623+0000 7f0575fab640 1 --2- 172.21.15.67:0/2183873858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05680a53c0 0x7f05680a57d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.623+0000 7f0575fab640 1 -- 172.21.15.67:0/2183873858 >> 172.21.15.67:0/2183873858 conn(0x7f056801d0c0 msgr2=0x7f05680a39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.627+0000 7f0575fab640 1 -- 172.21.15.67:0/2183873858 shutdown_connections 2024-07-31T15:01:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.629+0000 7f0575fab640 1 -- 172.21.15.67:0/2183873858 wait complete. 2024-07-31T15:01:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.630+0000 7f0575fab640 1 Processor -- start 2024-07-31T15:01:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.630+0000 7f0575fab640 1 -- start start 2024-07-31T15:01:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.630+0000 7f0575fab640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05680a53c0 0x7f056813ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.631+0000 7f0575fab640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05680aacf0 0x7f056813bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.631+0000 7f0575fab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05680ab640 0x7f0568141250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.631+0000 7f0575fab640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f05680b4ee0 con 0x7f05680ab640 2024-07-31T15:01:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.631+0000 7f0575fab640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05680b4d60 con 0x7f05680a53c0 2024-07-31T15:01:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.631+0000 7f0575fab640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05680b5060 con 0x7f05680aacf0 2024-07-31T15:01:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.631+0000 7f05757aa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05680ab640 0x7f0568141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.631+0000 7f05757aa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05680ab640 0x7f0568141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41048/0 (socket says 172.21.15.67:41048) 2024-07-31T15:01:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.631+0000 7f05757aa640 1 -- 172.21.15.67:0/1660692939 learned_addr learned my addr 172.21.15.67:0/1660692939 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.632+0000 7f05757aa640 1 -- 172.21.15.67:0/1660692939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05680aacf0 msgr2=0x7f056813bfc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:01:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.632+0000 7f05757aa640 1 --2- 172.21.15.67:0/1660692939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05680aacf0 0x7f056813bfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.632+0000 7f05757aa640 1 -- 172.21.15.67:0/1660692939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05680a53c0 msgr2=0x7f056813ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.632+0000 7f05757aa640 1 --2- 172.21.15.67:0/1660692939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05680a53c0 0x7f056813ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.632+0000 7f05757aa640 1 -- 172.21.15.67:0/1660692939 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0568141950 con 0x7f05680ab640 2024-07-31T15:01:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.632+0000 7f05757aa640 1 --2- 172.21.15.67:0/1660692939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05680ab640 0x7f0568141250 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f056400ea10 tx=0x7f056400eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.634+0000 7f056dffb640 1 -- 172.21.15.67:0/1660692939 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056400ccc0 con 0x7f05680ab640 2024-07-31T15:01:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.634+0000 7f056dffb640 1 -- 172.21.15.67:0/1660692939 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05640054d0 con 0x7f05680ab640 2024-07-31T15:01:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.634+0000 7f0575fab640 1 -- 172.21.15.67:0/1660692939 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0568141c40 con 0x7f05680ab640 2024-07-31T15:01:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.634+0000 7f0575fab640 1 -- 172.21.15.67:0/1660692939 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f056800dbe0 con 0x7f05680ab640 2024-07-31T15:01:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.634+0000 7f056dffb640 1 -- 172.21.15.67:0/1660692939 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564017600 con 0x7f05680ab640 2024-07-31T15:01:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.642+0000 7f057e1d8740 1 Processor -- start 2024-07-31T15:01:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.645+0000 7f056dffb640 1 -- 172.21.15.67:0/1660692939 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f05640046c0 con 0x7f05680ab640 2024-07-31T15:01:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.651+0000 7f056dffb640 1 --2- 172.21.15.67:0/1660692939 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f05600b3f50 0x7f05600b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.651+0000 7f056dffb640 1 -- 172.21.15.67:0/1660692939 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f05640e41b0 con 0x7f05680ab640 2024-07-31T15:01:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.652+0000 7f057e1d8740 1 -- start start 2024-07-31T15:01:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.652+0000 7f057e1d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953b1b1b0 0x55b953ebb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.652+0000 7f057e1d8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953af22f0 0x55b953ebbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.652+0000 7f057e1d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953afa4d0 0x55b953ece300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.652+0000 7f057e1d8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b953e333b0 con 0x55b953af22f0 2024-07-31T15:01:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.652+0000 7f057e1d8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b953e228b0 con 0x55b953b1b1b0 2024-07-31T15:01:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.652+0000 7f057e1d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b953e268f0 con 0x55b953afa4d0 2024-07-31T15:01:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.653+0000 7f056ffff640 1 --2- 172.21.15.67:0/1660692939 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f05600b3f50 0x7f05600b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.653+0000 7f056ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953af22f0 0x55b953ebbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.653+0000 7f056ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953af22f0 0x55b953ebbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41058/0 (socket says 172.21.15.67:41058) 2024-07-31T15:01:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.653+0000 7f056ffff640 1 -- 172.21.15.67:0/1241694624 learned_addr learned my addr 172.21.15.67:0/1241694624 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.653+0000 7f0574fa9640 1 --2- 172.21.15.67:0/1241694624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953b1b1b0 0x55b953ebb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.655+0000 7f05757aa640 1 --2- 172.21.15.67:0/1241694624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953afa4d0 0x55b953ece300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.655+0000 7f056ffff640 1 -- 172.21.15.67:0/1241694624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953afa4d0 msgr2=0x55b953ece300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.655+0000 7f056ffff640 1 --2- 172.21.15.67:0/1241694624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953afa4d0 0x55b953ece300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.655+0000 7f056ffff640 1 -- 172.21.15.67:0/1241694624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953b1b1b0 msgr2=0x55b953ebb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.655+0000 7f056ffff640 1 --2- 172.21.15.67:0/1241694624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953b1b1b0 0x55b953ebb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.655+0000 7f056ffff640 1 -- 172.21.15.67:0/1241694624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b953dc99a0 con 0x55b953af22f0 2024-07-31T15:01:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.656+0000 7f056ffff640 1 --2- 172.21.15.67:0/1660692939 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f05600b3f50 0x7f05600b6460 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0570068ca0 tx=0x7f0570068810 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.656+0000 7f056ffff640 1 --2- 172.21.15.67:0/1241694624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953af22f0 0x55b953ebbe90 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f05700643b0 tx=0x7f0570069040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75df6e3267fb7340 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.658+0000 7f055affd640 1 -- 172.21.15.67:0/1241694624 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057007a830 con 0x55b953af22f0 2024-07-31T15:01:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.659+0000 7f055affd640 1 -- 172.21.15.67:0/1241694624 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0570075040 con 0x55b953af22f0 2024-07-31T15:01:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.659+0000 7f055affd640 1 -- 172.21.15.67:0/1241694624 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057007cd90 con 0x55b953af22f0 2024-07-31T15:01:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.660+0000 7f057e1d8740 1 -- 172.21.15.67:0/1241694624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953af22f0 msgr2=0x55b953ebbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.660+0000 7f057e1d8740 1 --2- 172.21.15.67:0/1241694624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953af22f0 0x55b953ebbe90 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f05700643b0 tx=0x7f0570069040 comp rx=0 tx=0).stop 2024-07-31T15:01:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.665+0000 7f057e1d8740 1 -- 172.21.15.67:0/1241694624 shutdown_connections 2024-07-31T15:01:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.665+0000 7f057e1d8740 1 --2- 172.21.15.67:0/1241694624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953afa4d0 0x55b953ece300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.665+0000 7f057e1d8740 1 --2- 172.21.15.67:0/1241694624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953af22f0 0x55b953ebbe90 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.665+0000 7f057e1d8740 1 --2- 172.21.15.67:0/1241694624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953b1b1b0 0x55b953ebb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.665+0000 7f057e1d8740 1 -- 172.21.15.67:0/1241694624 >> 172.21.15.67:0/1241694624 conn(0x55b953b0f300 msgr2=0x55b953ebde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.670+0000 7f057e1d8740 1 -- 172.21.15.67:0/1241694624 shutdown_connections 2024-07-31T15:01:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.673+0000 7f057e1d8740 1 -- 172.21.15.67:0/1241694624 wait complete. 2024-07-31T15:01:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.673+0000 7f057e1d8740 1 -- start start 2024-07-31T15:01:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.673+0000 7f057e1d8740 1 Processor -- start 2024-07-31T15:01:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.677+0000 7f057e1d8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953af22f0 0x55b953ed2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.677+0000 7f057e1d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953afa4d0 0x55b953ed2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.677+0000 7f057e1d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953b1b1b0 0x55b953ed7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.677+0000 7f057e1d8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b953d5cab0 con 0x55b953af22f0 2024-07-31T15:01:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.677+0000 7f057e1d8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b953daacf0 con 0x55b953afa4d0 2024-07-31T15:01:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.677+0000 7f057e1d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b953eced80 con 0x55b953b1b1b0 2024-07-31T15:01:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.677+0000 7f057e1d8740 10 client.0 _ll_register_callbacks cb 0x55b953e1c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.682+0000 7f056ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953afa4d0 0x55b953ed2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.682+0000 7f056ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953afa4d0 0x55b953ed2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:43670/0 (socket says 172.21.15.67:43670) 2024-07-31T15:01:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.682+0000 7f056ffff640 1 -- 172.21.15.67:0/2509430586 learned_addr learned my addr 172.21.15.67:0/2509430586 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.682+0000 7f05757aa640 1 --2- 172.21.15.67:0/2509430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953b1b1b0 0x55b953ed7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.683+0000 7f0574fa9640 1 --2- 172.21.15.67:0/2509430586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953af22f0 0x55b953ed2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.685+0000 7f056ffff640 1 -- 172.21.15.67:0/2509430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953b1b1b0 msgr2=0x55b953ed7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.685+0000 7f056ffff640 1 --2- 172.21.15.67:0/2509430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953b1b1b0 0x55b953ed7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.685+0000 7f056ffff640 1 -- 172.21.15.67:0/2509430586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953af22f0 msgr2=0x55b953ed2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.685+0000 7f056ffff640 1 --2- 172.21.15.67:0/2509430586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953af22f0 0x55b953ed2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.685+0000 7f056ffff640 1 -- 172.21.15.67:0/2509430586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b953b86790 con 0x55b953afa4d0 2024-07-31T15:01:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.687+0000 7f056ffff640 1 --2- 172.21.15.67:0/2509430586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953afa4d0 0x55b953ed2f50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f05700687e0 tx=0x7f057007c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41fd3e8eaa384afa server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.688+0000 7f054ffff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:01:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.689+0000 7f054ffff640 1 -- 172.21.15.67:0/2509430586 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570092020 con 0x55b953afa4d0 2024-07-31T15:01:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.689+0000 7f054ffff640 1 -- 172.21.15.67:0/2509430586 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f057007a830 con 0x55b953afa4d0 2024-07-31T15:01:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.691+0000 7f057e1d8740 1 -- 172.21.15.67:0/2509430586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0570075040 con 0x55b953afa4d0 2024-07-31T15:01:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.697+0000 7f054ffff640 1 -- 172.21.15.67:0/2509430586 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f0570089040 con 0x55b953afa4d0 2024-07-31T15:01:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.698+0000 7f057e1d8740 10 client.27302 fetch_fsmap learned FSMap version 57 2024-07-31T15:01:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.698+0000 7f057e1d8740 1 -- 172.21.15.67:0/2509430586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55b953bff620 con 0x55b953afa4d0 2024-07-31T15:01:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.704+0000 7f054ffff640 1 -- 172.21.15.67:0/2509430586 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0570075040 con 0x55b953afa4d0 2024-07-31T15:01:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.705+0000 7f054ffff640 1 -- 172.21.15.67:0/2509430586 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 57) ==== 8398+0+0 (secure 0 0 0) 0x7f0570083070 con 0x55b953afa4d0 2024-07-31T15:01:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.707+0000 7f057e1d8740 10 client.27302 fetch_fsmap finished waiting for FSMap version 57 2024-07-31T15:01:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.707+0000 7f057e1d8740 10 client.27302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:01:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.707+0000 7f057e1d8740 20 client.27302 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.707+0000 7f057e1d8740 1 --2- 172.21.15.67:0/2509430586 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b953ad9d80 0x55b953ee40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.707+0000 7f057e1d8740 4 client.27302 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:01:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.707+0000 7f057e1d8740 1 -- 172.21.15.67:0/2509430586 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b953e35300 con 0x55b953ad9d80 2024-07-31T15:01:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.709+0000 7f0574fa9640 1 --2- 172.21.15.67:0/2509430586 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b953ad9d80 0x55b953ee40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.710+0000 7f0574fa9640 1 --2- 172.21.15.67:0/2509430586 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b953ad9d80 0x55b953ee40e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7cffa190f1bdff6 server_cookie=b300ffb9c128de92 in_seq=0 out_seq=0 2024-07-31T15:01:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.713+0000 7f054ffff640 10 client.27302 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.713+0000 7f054ffff640 1 -- 172.21.15.67:0/2509430586 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b953e35300 con 0x55b953ad9d80 2024-07-31T15:01:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.713+0000 7f054ffff640 10 client.27302 handle_command_reply: tid=0 2024-07-31T15:01:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.714+0000 7f057e1d8740 1 client.27302 shutdown 2024-07-31T15:01:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.721+0000 7f057e1d8740 1 -- 172.21.15.67:0/2509430586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953afa4d0 msgr2=0x55b953ed2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.721+0000 7f057e1d8740 1 --2- 172.21.15.67:0/2509430586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953afa4d0 0x55b953ed2f50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f05700687e0 tx=0x7f057007c640 comp rx=0 tx=0).stop 2024-07-31T15:01:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.724+0000 7f057e1d8740 1 -- 172.21.15.67:0/2509430586 shutdown_connections 2024-07-31T15:01:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.724+0000 7f057e1d8740 1 --2- 172.21.15.67:0/2509430586 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b953ad9d80 0x55b953ee40e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.724+0000 7f057e1d8740 1 --2- 172.21.15.67:0/2509430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953b1b1b0 0x55b953ed7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.724+0000 7f057e1d8740 1 --2- 172.21.15.67:0/2509430586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953afa4d0 0x55b953ed2f50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.724+0000 7f057e1d8740 1 --2- 172.21.15.67:0/2509430586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953af22f0 0x55b953ed2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.724+0000 7f057e1d8740 1 -- 172.21.15.67:0/2509430586 >> 172.21.15.67:0/2509430586 conn(0x55b953b0f300 msgr2=0x55b953ebac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.725+0000 7f054ffff640 0 client.27302 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.731+0000 7f057e1d8740 1 -- 172.21.15.67:0/2509430586 shutdown_connections 2024-07-31T15:01:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.732+0000 7f057e1d8740 1 -- 172.21.15.67:0/2509430586 wait complete. 2024-07-31T15:01:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.733+0000 7f057e1d8740 20 client.27302 trim_cache size 0 max 16384 2024-07-31T15:01:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.768+0000 7f057e1d8740 1 Processor -- start 2024-07-31T15:01:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.768+0000 7f057e1d8740 1 -- start start 2024-07-31T15:01:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.768+0000 7f057e1d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953af22f0 0x55b953ed0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.768+0000 7f057e1d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953afa4d0 0x55b953ebb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.768+0000 7f057e1d8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953b1b1b0 0x55b953ebbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.768+0000 7f057e1d8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f055400ff10 con 0x55b953b1b1b0 2024-07-31T15:01:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.768+0000 7f057e1d8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0554011b40 con 0x55b953afa4d0 2024-07-31T15:01:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.768+0000 7f057e1d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05540105e0 con 0x55b953af22f0 2024-07-31T15:01:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.770+0000 7f05757aa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953b1b1b0 0x55b953ebbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.770+0000 7f05757aa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953b1b1b0 0x55b953ebbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41086/0 (socket says 172.21.15.67:41086) 2024-07-31T15:01:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.770+0000 7f05757aa640 1 -- 172.21.15.67:0/1894352305 learned_addr learned my addr 172.21.15.67:0/1894352305 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.770+0000 7f056ffff640 1 --2- 172.21.15.67:0/1894352305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953afa4d0 0x55b953ebb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.771+0000 7f0574fa9640 1 --2- 172.21.15.67:0/1894352305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953af22f0 0x55b953ed0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.773+0000 7f05757aa640 1 -- 172.21.15.67:0/1894352305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953af22f0 msgr2=0x55b953ed0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.773+0000 7f05757aa640 1 --2- 172.21.15.67:0/1894352305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953af22f0 0x55b953ed0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.773+0000 7f05757aa640 1 -- 172.21.15.67:0/1894352305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953afa4d0 msgr2=0x55b953ebb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.773+0000 7f05757aa640 1 --2- 172.21.15.67:0/1894352305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953afa4d0 0x55b953ebb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.773+0000 7f05757aa640 1 -- 172.21.15.67:0/1894352305 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b953ebe760 con 0x55b953b1b1b0 2024-07-31T15:01:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.773+0000 7f0574fa9640 1 --2- 172.21.15.67:0/1894352305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953af22f0 0x55b953ed0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:01:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.785+0000 7f05757aa640 1 --2- 172.21.15.67:0/1894352305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953b1b1b0 0x55b953ebbb00 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f0564004be0 tx=0x7f05640a61e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf8c41ea246555f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.788+0000 7f055a7fc640 1 -- 172.21.15.67:0/1894352305 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05640c3030 con 0x55b953b1b1b0 2024-07-31T15:01:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.789+0000 7f055a7fc640 1 -- 172.21.15.67:0/1894352305 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05640c52c0 con 0x55b953b1b1b0 2024-07-31T15:01:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.790+0000 7f057e1d8740 1 -- 172.21.15.67:0/1894352305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953b1b1b0 msgr2=0x55b953ebbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.790+0000 7f057e1d8740 1 --2- 172.21.15.67:0/1894352305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953b1b1b0 0x55b953ebbb00 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f0564004be0 tx=0x7f05640a61e0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.795+0000 7f057e1d8740 1 -- 172.21.15.67:0/1894352305 shutdown_connections 2024-07-31T15:01:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.795+0000 7f057e1d8740 1 --2- 172.21.15.67:0/1894352305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953b1b1b0 0x55b953ebbb00 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.795+0000 7f057e1d8740 1 --2- 172.21.15.67:0/1894352305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953afa4d0 0x55b953ebb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.795+0000 7f057e1d8740 1 --2- 172.21.15.67:0/1894352305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953af22f0 0x55b953ed0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.795+0000 7f057e1d8740 1 -- 172.21.15.67:0/1894352305 >> 172.21.15.67:0/1894352305 conn(0x55b953b0f300 msgr2=0x55b953ecdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.797+0000 7f057e1d8740 1 -- 172.21.15.67:0/1894352305 shutdown_connections 2024-07-31T15:01:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.797+0000 7f057e1d8740 1 -- 172.21.15.67:0/1894352305 wait complete. 2024-07-31T15:01:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.798+0000 7f057e1d8740 1 -- start start 2024-07-31T15:01:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.798+0000 7f057e1d8740 1 Processor -- start 2024-07-31T15:01:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.799+0000 7f057e1d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953af22f0 0x55b953ee9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.799+0000 7f057e1d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953afa4d0 0x55b953ee8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.799+0000 7f057e1d8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953b1b1b0 0x55b953ee8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.799+0000 7f057e1d8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0554011470 con 0x55b953b1b1b0 2024-07-31T15:01:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.799+0000 7f057e1d8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0554010ce0 con 0x55b953af22f0 2024-07-31T15:01:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.799+0000 7f057e1d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f055400dfa0 con 0x55b953afa4d0 2024-07-31T15:01:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.799+0000 7f057e1d8740 10 client.0 _ll_register_callbacks cb 0x7f055c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.800+0000 7f05757aa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953b1b1b0 0x55b953ee8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.800+0000 7f05757aa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953b1b1b0 0x55b953ee8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41098/0 (socket says 172.21.15.67:41098) 2024-07-31T15:01:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.800+0000 7f05757aa640 1 -- 172.21.15.67:0/3782757396 learned_addr learned my addr 172.21.15.67:0/3782757396 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.802+0000 7f0574fa9640 1 --2- 172.21.15.67:0/3782757396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953af22f0 0x55b953ee9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.802+0000 7f056ffff640 1 --2- 172.21.15.67:0/3782757396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953afa4d0 0x55b953ee8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.803+0000 7f05757aa640 1 -- 172.21.15.67:0/3782757396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953afa4d0 msgr2=0x55b953ee8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.803+0000 7f05757aa640 1 --2- 172.21.15.67:0/3782757396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953afa4d0 0x55b953ee8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.803+0000 7f05757aa640 1 -- 172.21.15.67:0/3782757396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953af22f0 msgr2=0x55b953ee9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.803+0000 7f05757aa640 1 --2- 172.21.15.67:0/3782757396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953af22f0 0x55b953ee9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.803+0000 7f05757aa640 1 -- 172.21.15.67:0/3782757396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b953ee9360 con 0x55b953b1b1b0 2024-07-31T15:01:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.804+0000 7f05757aa640 1 --2- 172.21.15.67:0/3782757396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953b1b1b0 0x55b953ee8dd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f05640c1040 tx=0x7f056401e4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b9a54c254255d1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.807+0000 7f054ffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:01:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.807+0000 7f054ffff640 1 -- 172.21.15.67:0/3782757396 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05640b7020 con 0x55b953b1b1b0 2024-07-31T15:01:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.807+0000 7f054ffff640 1 -- 172.21.15.67:0/3782757396 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f05640b35b0 con 0x55b953b1b1b0 2024-07-31T15:01:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.809+0000 7f057e1d8740 1 -- 172.21.15.67:0/3782757396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05640c52c0 con 0x55b953b1b1b0 2024-07-31T15:01:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.815+0000 7f054ffff640 1 -- 172.21.15.67:0/3782757396 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f0564005680 con 0x55b953b1b1b0 2024-07-31T15:01:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.816+0000 7f057e1d8740 10 client.17751 fetch_fsmap learned FSMap version 57 2024-07-31T15:01:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.816+0000 7f057e1d8740 1 -- 172.21.15.67:0/3782757396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55b953ec2700 con 0x55b953b1b1b0 2024-07-31T15:01:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.817+0000 7f054ffff640 1 -- 172.21.15.67:0/3782757396 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05640c52c0 con 0x55b953b1b1b0 2024-07-31T15:01:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.817+0000 7f054ffff640 1 -- 172.21.15.67:0/3782757396 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 57) ==== 8398+0+0 (secure 0 0 0) 0x7f05640a9660 con 0x55b953b1b1b0 2024-07-31T15:01:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.821+0000 7f057e1d8740 10 client.17751 fetch_fsmap finished waiting for FSMap version 57 2024-07-31T15:01:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.821+0000 7f057e1d8740 10 client.17751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:01:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.821+0000 7f057e1d8740 20 client.17751 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.821+0000 7f057e1d8740 1 --2- 172.21.15.67:0/3782757396 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b953aaf1c0 0x55b953ec4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.821+0000 7f057e1d8740 4 client.17751 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:01:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.821+0000 7f057e1d8740 1 -- 172.21.15.67:0/3782757396 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b953edf1b0 con 0x55b953aaf1c0 2024-07-31T15:01:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.824+0000 7f0574fa9640 1 --2- 172.21.15.67:0/3782757396 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b953aaf1c0 0x55b953ec4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.838+0000 7f0574fa9640 1 --2- 172.21.15.67:0/3782757396 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b953aaf1c0 0x55b953ec4f10 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f53db6457a486e server_cookie=680ebf59a02fcf5a in_seq=0 out_seq=0 2024-07-31T15:01:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:12.839+0000 7f054ffff640 10 client.17751 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:13.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:13 smithi067 ceph-mon[32750]: pgmap v714: 129 pgs: 53 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 57 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.5 MiB/s wr, 81 op/s 2024-07-31T15:01:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.306+0000 7f054ffff640 1 -- 172.21.15.67:0/3782757396 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b953edf1b0 con 0x55b953aaf1c0 2024-07-31T15:01:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.306+0000 7f054ffff640 10 client.17751 handle_command_reply: tid=0 2024-07-31T15:01:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.306+0000 7f057e1d8740 1 client.17751 shutdown 2024-07-31T15:01:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.306+0000 7f057e1d8740 1 -- 172.21.15.67:0/3782757396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953b1b1b0 msgr2=0x55b953ee8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.306+0000 7f057e1d8740 1 --2- 172.21.15.67:0/3782757396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953b1b1b0 0x55b953ee8dd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f05640c1040 tx=0x7f056401e4e0 comp rx=0 tx=0).stop 2024-07-31T15:01:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.307+0000 7f057e1d8740 1 -- 172.21.15.67:0/3782757396 shutdown_connections 2024-07-31T15:01:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.307+0000 7f057e1d8740 1 --2- 172.21.15.67:0/3782757396 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b953aaf1c0 0x55b953ec4f10 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.307+0000 7f057e1d8740 1 --2- 172.21.15.67:0/3782757396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b953b1b1b0 0x55b953ee8dd0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.307+0000 7f057e1d8740 1 --2- 172.21.15.67:0/3782757396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b953afa4d0 0x55b953ee8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.307+0000 7f057e1d8740 1 --2- 172.21.15.67:0/3782757396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b953af22f0 0x55b953ee9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.307+0000 7f057e1d8740 1 -- 172.21.15.67:0/3782757396 >> 172.21.15.67:0/3782757396 conn(0x55b953b0f300 msgr2=0x55b953ecdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.307+0000 7f054ffff640 0 client.17751 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.307+0000 7f057e1d8740 1 -- 172.21.15.67:0/3782757396 shutdown_connections 2024-07-31T15:01:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.307+0000 7f057e1d8740 1 -- 172.21.15.67:0/3782757396 wait complete. 2024-07-31T15:01:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.307+0000 7f057e1d8740 20 client.17751 trim_cache size 0 max 16384 2024-07-31T15:01:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:01:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:01:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:01:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:01:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.308+0000 7f0575fab640 1 -- 172.21.15.67:0/1660692939 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f05600b3f50 msgr2=0x7f05600b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.308+0000 7f0575fab640 1 --2- 172.21.15.67:0/1660692939 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f05600b3f50 0x7f05600b6460 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0570068ca0 tx=0x7f0570068810 comp rx=0 tx=0).stop 2024-07-31T15:01:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.308+0000 7f0575fab640 1 -- 172.21.15.67:0/1660692939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05680ab640 msgr2=0x7f0568141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.308+0000 7f0575fab640 1 --2- 172.21.15.67:0/1660692939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05680ab640 0x7f0568141250 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f056400ea10 tx=0x7f056400eed0 comp rx=0 tx=0).stop 2024-07-31T15:01:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.308+0000 7f0575fab640 1 -- 172.21.15.67:0/1660692939 shutdown_connections 2024-07-31T15:01:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.308+0000 7f0575fab640 1 --2- 172.21.15.67:0/1660692939 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f05600b3f50 0x7f05600b6460 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.308+0000 7f0575fab640 1 --2- 172.21.15.67:0/1660692939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05680ab640 0x7f0568141250 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.308+0000 7f0575fab640 1 --2- 172.21.15.67:0/1660692939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05680aacf0 0x7f056813bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.308+0000 7f0575fab640 1 --2- 172.21.15.67:0/1660692939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05680a53c0 0x7f056813ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.308+0000 7f0575fab640 1 -- 172.21.15.67:0/1660692939 >> 172.21.15.67:0/1660692939 conn(0x7f056801d0c0 msgr2=0x7f05680aa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.308+0000 7f0575fab640 1 -- 172.21.15.67:0/1660692939 shutdown_connections 2024-07-31T15:01:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:13.308+0000 7f0575fab640 1 -- 172.21.15.67:0/1660692939 wait complete. 2024-07-31T15:01:13.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:13 smithi073 ceph-mon[36714]: pgmap v714: 129 pgs: 53 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 57 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.5 MiB/s wr, 81 op/s 2024-07-31T15:01:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:13 smithi077 ceph-mon[37327]: pgmap v714: 129 pgs: 53 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 57 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.5 MiB/s wr, 81 op/s 2024-07-31T15:01:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:15 smithi073 ceph-mon[36714]: pgmap v715: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.3 MiB/s wr, 83 op/s 2024-07-31T15:01:15.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:15 smithi077 ceph-mon[37327]: pgmap v715: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.3 MiB/s wr, 83 op/s 2024-07-31T15:01:15.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:15 smithi067 ceph-mon[32750]: pgmap v715: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.3 MiB/s wr, 83 op/s 2024-07-31T15:01:16.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:16 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:01:16.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:16 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:01:16.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:16 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:01:17.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:17 smithi073 ceph-mon[36714]: pgmap v716: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-07-31T15:01:17.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:17 smithi077 ceph-mon[37327]: pgmap v716: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-07-31T15:01:17.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:17 smithi067 ceph-mon[32750]: pgmap v716: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-07-31T15:01:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:19 smithi073 ceph-mon[36714]: pgmap v717: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-07-31T15:01:19.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:19 smithi077 ceph-mon[37327]: pgmap v717: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-07-31T15:01:19.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:19 smithi067 ceph-mon[32750]: pgmap v717: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-07-31T15:01:20.053 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 15:01:19 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T15:01:19.706+0000 7f4cfe9c3640 -1 calc_pg_upmaps abort due to max <= 0 2024-07-31T15:01:21.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:21 smithi073 ceph-mon[36714]: pgmap v718: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 957 KiB/s wr, 173 op/s 2024-07-31T15:01:21.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:21 smithi077 ceph-mon[37327]: pgmap v718: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 957 KiB/s wr, 173 op/s 2024-07-31T15:01:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:21 smithi067 ceph-mon[32750]: pgmap v718: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 957 KiB/s wr, 173 op/s 2024-07-31T15:01:23.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:23 smithi073 ceph-mon[36714]: pgmap v719: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 730 KiB/s wr, 145 op/s 2024-07-31T15:01:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:23 smithi077 ceph-mon[37327]: pgmap v719: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 730 KiB/s wr, 145 op/s 2024-07-31T15:01:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:23 smithi067 ceph-mon[32750]: pgmap v719: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 730 KiB/s wr, 145 op/s 2024-07-31T15:01:24.193 INFO:tasks.workunit.client.0.smithi067.stdout: CC scripts/mod/empty.o 2024-07-31T15:01:24.197 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/mod/mk_elfconfig 2024-07-31T15:01:24.199 INFO:tasks.workunit.client.0.smithi067.stdout: CC scripts/mod/devicetable-offsets.s 2024-07-31T15:01:24.409 INFO:tasks.workunit.client.0.smithi067.stdout: MKELF scripts/mod/elfconfig.h 2024-07-31T15:01:24.416 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/mod/modpost.o 2024-07-31T15:01:24.419 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/mod/sumversion.o 2024-07-31T15:01:25.177 INFO:tasks.workunit.client.0.smithi067.stdout: UPD scripts/mod/devicetable-offsets.h 2024-07-31T15:01:25.212 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/mod/file2alias.o 2024-07-31T15:01:25.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:25 smithi073 ceph-mon[36714]: pgmap v720: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 542 KiB/s wr, 163 op/s 2024-07-31T15:01:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:25 smithi077 ceph-mon[37327]: pgmap v720: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 542 KiB/s wr, 163 op/s 2024-07-31T15:01:25.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:25 smithi067 ceph-mon[32750]: pgmap v720: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 542 KiB/s wr, 163 op/s 2024-07-31T15:01:25.971 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTLD scripts/mod/modpost 2024-07-31T15:01:26.096 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/bounds.s 2024-07-31T15:01:26.098 INFO:tasks.workunit.client.0.smithi067.stdout: CHKSHA1 include/linux/atomic/atomic-arch-fallback.h 2024-07-31T15:01:26.098 INFO:tasks.workunit.client.0.smithi067.stdout: CHKSHA1 include/linux/atomic/atomic-instrumented.h 2024-07-31T15:01:26.098 INFO:tasks.workunit.client.0.smithi067.stdout: CHKSHA1 include/linux/atomic/atomic-long.h 2024-07-31T15:01:26.416 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/generated/timeconst.h 2024-07-31T15:01:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.480+0000 7f1b26f70640 1 -- 172.21.15.67:0/2141817572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b20106d30 msgr2=0x7f1b20107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.480+0000 7f1b26f70640 1 --2- 172.21.15.67:0/2141817572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b20106d30 0x7f1b20107140 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1b1001acb0 tx=0x7f1b1003b920 comp rx=0 tx=0).stop 2024-07-31T15:01:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.483+0000 7f1b26f70640 1 -- 172.21.15.67:0/2141817572 shutdown_connections 2024-07-31T15:01:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.483+0000 7f1b26f70640 1 --2- 172.21.15.67:0/2141817572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b20107680 0x7f1b20107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.483+0000 7f1b26f70640 1 --2- 172.21.15.67:0/2141817572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b20106d30 0x7f1b20107140 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.483+0000 7f1b26f70640 1 --2- 172.21.15.67:0/2141817572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b20101a50 0x7f1b20101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.483+0000 7f1b26f70640 1 -- 172.21.15.67:0/2141817572 >> 172.21.15.67:0/2141817572 conn(0x7f1b20079880 msgr2=0x7f1b20079c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.490+0000 7f1b26f70640 1 -- 172.21.15.67:0/2141817572 shutdown_connections 2024-07-31T15:01:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.494+0000 7f1b26f70640 1 -- 172.21.15.67:0/2141817572 wait complete. 2024-07-31T15:01:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.494+0000 7f1b26f70640 1 Processor -- start 2024-07-31T15:01:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.499+0000 7f1b26f70640 1 -- start start 2024-07-31T15:01:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.499+0000 7f1b26f70640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b20101a50 0x7f1b20073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.499+0000 7f1b26f70640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b20106d30 0x7f1b20074330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.499+0000 7f1b26f70640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b20107680 0x7f1b2010f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.499+0000 7f1b26f70640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2010b430 con 0x7f1b20106d30 2024-07-31T15:01:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.499+0000 7f1b26f70640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2010b2b0 con 0x7f1b20107680 2024-07-31T15:01:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.499+0000 7f1b26f70640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2010b5b0 con 0x7f1b20101a50 2024-07-31T15:01:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.502+0000 7f1b1ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b20106d30 0x7f1b20074330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.502+0000 7f1b1ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b20106d30 0x7f1b20074330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44590/0 (socket says 172.21.15.67:44590) 2024-07-31T15:01:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.502+0000 7f1b1ffff640 1 -- 172.21.15.67:0/2589299094 learned_addr learned my addr 172.21.15.67:0/2589299094 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.503+0000 7f1b24d0c640 1 --2- 172.21.15.67:0/2589299094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b20101a50 0x7f1b20073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.503+0000 7f1b2550d640 1 --2- 172.21.15.67:0/2589299094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b20107680 0x7f1b2010f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.509+0000 7f1b1ffff640 1 -- 172.21.15.67:0/2589299094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b20101a50 msgr2=0x7f1b20073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.509+0000 7f1b1ffff640 1 --2- 172.21.15.67:0/2589299094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b20101a50 0x7f1b20073db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.509+0000 7f1b1ffff640 1 -- 172.21.15.67:0/2589299094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b20107680 msgr2=0x7f1b2010f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.509+0000 7f1b1ffff640 1 --2- 172.21.15.67:0/2589299094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b20107680 0x7f1b2010f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.509+0000 7f1b1ffff640 1 -- 172.21.15.67:0/2589299094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b201174f0 con 0x7f1b20106d30 2024-07-31T15:01:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.512+0000 7f1b1ffff640 1 --2- 172.21.15.67:0/2589299094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b20106d30 0x7f1b20074330 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1b10041040 tx=0x7f1b10040610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.513+0000 7f1b1dffb640 1 -- 172.21.15.67:0/2589299094 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10045070 con 0x7f1b20106d30 2024-07-31T15:01:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.513+0000 7f1b1dffb640 1 -- 172.21.15.67:0/2589299094 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b10003e20 con 0x7f1b20106d30 2024-07-31T15:01:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.513+0000 7f1b26f70640 1 -- 172.21.15.67:0/2589299094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2010f660 con 0x7f1b20106d30 2024-07-31T15:01:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.514+0000 7f1b26f70640 1 -- 172.21.15.67:0/2589299094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2010fb50 con 0x7f1b20106d30 2024-07-31T15:01:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.515+0000 7f1b1dffb640 1 -- 172.21.15.67:0/2589299094 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1004b690 con 0x7f1b20106d30 2024-07-31T15:01:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.519+0000 7f1b2de69740 1 Processor -- start 2024-07-31T15:01:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.523+0000 7f1b1dffb640 1 -- 172.21.15.67:0/2589299094 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1b10048020 con 0x7f1b20106d30 2024-07-31T15:01:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.525+0000 7f1b1dffb640 1 --2- 172.21.15.67:0/2589299094 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1b080b3f50 0x7f1b080b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.525+0000 7f1b1dffb640 1 -- 172.21.15.67:0/2589299094 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f1b10115ea0 con 0x7f1b20106d30 2024-07-31T15:01:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.527+0000 7f1b2de69740 1 -- start start 2024-07-31T15:01:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.527+0000 7f1b2de69740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56348162e1b0 0x5634819d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.527+0000 7f1b2de69740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634816052f0 0x5634819d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.527+0000 7f1b2de69740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348160d4d0 0x5634819d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.527+0000 7f1b2de69740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5634819463b0 con 0x56348162e1b0 2024-07-31T15:01:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.527+0000 7f1b2de69740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5634819358b0 con 0x5634816052f0 2024-07-31T15:01:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.527+0000 7f1b2de69740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5634819398f0 con 0x56348160d4d0 2024-07-31T15:01:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.527+0000 7f1b2550d640 1 --2- 172.21.15.67:0/2589299094 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1b080b3f50 0x7f1b080b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.528+0000 7f1b2550d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348160d4d0 0x5634819d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.528+0000 7f1b2550d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348160d4d0 0x5634819d1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37286/0 (socket says 172.21.15.67:37286) 2024-07-31T15:01:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.528+0000 7f1b2550d640 1 -- 172.21.15.67:0/737416692 learned_addr learned my addr 172.21.15.67:0/737416692 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.528+0000 7f1b2550d640 1 -- 172.21.15.67:0/737416692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634816052f0 msgr2=0x5634819d1220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:01:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.528+0000 7f1b2550d640 1 --2- 172.21.15.67:0/737416692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634816052f0 0x5634819d1220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.528+0000 7f1b2550d640 1 -- 172.21.15.67:0/737416692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56348162e1b0 msgr2=0x5634819d0ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:01:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.528+0000 7f1b2550d640 1 --2- 172.21.15.67:0/737416692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56348162e1b0 0x5634819d0ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.528+0000 7f1b2550d640 1 -- 172.21.15.67:0/737416692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634818dc9a0 con 0x56348160d4d0 2024-07-31T15:01:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.528+0000 7f1b2550d640 1 --2- 172.21.15.67:0/737416692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348160d4d0 0x5634819d1760 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1b18017970 tx=0x7f1b18017e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caa84c3eb3ad65ee server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.529+0000 7f1b2550d640 1 --2- 172.21.15.67:0/2589299094 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1b080b3f50 0x7f1b080b6460 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1b18015790 tx=0x7f1b1800b000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.529+0000 7f1b06ffd640 1 -- 172.21.15.67:0/737416692 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18027070 con 0x56348160d4d0 2024-07-31T15:01:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.529+0000 7f1b06ffd640 1 -- 172.21.15.67:0/737416692 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b1801c070 con 0x56348160d4d0 2024-07-31T15:01:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.531+0000 7f1b2de69740 1 -- 172.21.15.67:0/737416692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348160d4d0 msgr2=0x5634819d1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.531+0000 7f1b2de69740 1 --2- 172.21.15.67:0/737416692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348160d4d0 0x5634819d1760 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1b18017970 tx=0x7f1b18017e30 comp rx=0 tx=0).stop 2024-07-31T15:01:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.533+0000 7f1b2de69740 1 -- 172.21.15.67:0/737416692 shutdown_connections 2024-07-31T15:01:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.533+0000 7f1b2de69740 1 --2- 172.21.15.67:0/737416692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348160d4d0 0x5634819d1760 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.533+0000 7f1b2de69740 1 --2- 172.21.15.67:0/737416692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634816052f0 0x5634819d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.533+0000 7f1b2de69740 1 --2- 172.21.15.67:0/737416692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56348162e1b0 0x5634819d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.533+0000 7f1b2de69740 1 -- 172.21.15.67:0/737416692 >> 172.21.15.67:0/737416692 conn(0x563481622300 msgr2=0x5634819cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.541+0000 7f1b2de69740 1 -- 172.21.15.67:0/737416692 shutdown_connections 2024-07-31T15:01:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.541+0000 7f1b2de69740 1 -- 172.21.15.67:0/737416692 wait complete. 2024-07-31T15:01:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.541+0000 7f1b2de69740 1 -- start start 2024-07-31T15:01:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.542+0000 7f1b2de69740 1 Processor -- start 2024-07-31T15:01:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.542+0000 7f1b2de69740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634816052f0 0x5634819dc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.542+0000 7f1b2de69740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56348160d4d0 0x5634819dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.542+0000 7f1b2de69740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348162e1b0 0x5634819e8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.542+0000 7f1b2de69740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56348186fab0 con 0x5634816052f0 2024-07-31T15:01:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.542+0000 7f1b2de69740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5634818bdcf0 con 0x56348160d4d0 2024-07-31T15:01:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.542+0000 7f1b2de69740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5634819df450 con 0x56348162e1b0 2024-07-31T15:01:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.542+0000 7f1b2de69740 10 client.0 _ll_register_callbacks cb 0x56348192f9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.543+0000 7f1b1ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56348160d4d0 0x5634819dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.543+0000 7f1b1ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56348160d4d0 0x5634819dcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46464/0 (socket says 172.21.15.67:46464) 2024-07-31T15:01:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.543+0000 7f1b1ffff640 1 -- 172.21.15.67:0/1053959419 learned_addr learned my addr 172.21.15.67:0/1053959419 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.543+0000 7f1b2550d640 1 --2- 172.21.15.67:0/1053959419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348162e1b0 0x5634819e8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.543+0000 7f1b1ffff640 1 -- 172.21.15.67:0/1053959419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348162e1b0 msgr2=0x5634819e8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.543+0000 7f1b1ffff640 1 --2- 172.21.15.67:0/1053959419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348162e1b0 0x5634819e8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.543+0000 7f1b1ffff640 1 -- 172.21.15.67:0/1053959419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634816052f0 msgr2=0x5634819dc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.543+0000 7f1b1ffff640 1 --2- 172.21.15.67:0/1053959419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634816052f0 0x5634819dc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.543+0000 7f1b1ffff640 1 -- 172.21.15.67:0/1053959419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563481699790 con 0x56348160d4d0 2024-07-31T15:01:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.544+0000 7f1b1ffff640 1 --2- 172.21.15.67:0/1053959419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56348160d4d0 0x5634819dcd20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1b10043000 tx=0x7f1b100dbf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f458b21384c740e9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.545+0000 7f1afffff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:01:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.545+0000 7f1afffff640 1 -- 172.21.15.67:0/1053959419 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b100f6060 con 0x56348160d4d0 2024-07-31T15:01:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.545+0000 7f1b2de69740 1 -- 172.21.15.67:0/1053959419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b1801c070 con 0x56348160d4d0 2024-07-31T15:01:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.545+0000 7f1afffff640 1 -- 172.21.15.67:0/1053959419 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f1b10040a70 con 0x56348160d4d0 2024-07-31T15:01:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.545+0000 7f1afffff640 1 -- 172.21.15.67:0/1053959419 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1b10113650 con 0x56348160d4d0 2024-07-31T15:01:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.545+0000 7f1b2de69740 10 client.27311 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.545+0000 7f1b2de69740 1 -- 172.21.15.67:0/1053959419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563481712620 con 0x56348160d4d0 2024-07-31T15:01:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.547+0000 7f1afffff640 1 -- 172.21.15.67:0/1053959419 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b1801c070 con 0x56348160d4d0 2024-07-31T15:01:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.548+0000 7f1afffff640 1 -- 172.21.15.67:0/1053959419 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7f1b100ff0e0 con 0x56348160d4d0 2024-07-31T15:01:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.548+0000 7f1b2de69740 10 client.27311 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.548+0000 7f1b2de69740 10 client.27311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:01:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.548+0000 7f1b2de69740 20 client.27311 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.548+0000 7f1b2de69740 1 --2- 172.21.15.67:0/1053959419 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5634815ecd80 0x5634819eea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.548+0000 7f1b2de69740 4 client.27311 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:01:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.548+0000 7f1b2de69740 1 -- 172.21.15.67:0/1053959419 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563481948300 con 0x5634815ecd80 2024-07-31T15:01:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.556+0000 7f1b24d0c640 1 --2- 172.21.15.67:0/1053959419 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5634815ecd80 0x5634819eea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.565+0000 7f1b24d0c640 1 --2- 172.21.15.67:0/1053959419 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5634815ecd80 0x5634819eea50 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a30dd88a37d1ea55 server_cookie=76766617d4798cb2 in_seq=0 out_seq=0 2024-07-31T15:01:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.565+0000 7f1afffff640 10 client.27311 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.567+0000 7f1afffff640 1 -- 172.21.15.67:0/1053959419 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563481948300 con 0x5634815ecd80 2024-07-31T15:01:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.567+0000 7f1afffff640 10 client.27311 handle_command_reply: tid=0 2024-07-31T15:01:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.567+0000 7f1b2de69740 1 client.27311 shutdown 2024-07-31T15:01:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.573+0000 7f1b2de69740 1 -- 172.21.15.67:0/1053959419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56348160d4d0 msgr2=0x5634819dcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.573+0000 7f1b2de69740 1 --2- 172.21.15.67:0/1053959419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56348160d4d0 0x5634819dcd20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1b10043000 tx=0x7f1b100dbf00 comp rx=0 tx=0).stop 2024-07-31T15:01:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.573+0000 7f1b2de69740 1 -- 172.21.15.67:0/1053959419 shutdown_connections 2024-07-31T15:01:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.573+0000 7f1b2de69740 1 --2- 172.21.15.67:0/1053959419 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5634815ecd80 0x5634819eea50 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.573+0000 7f1b2de69740 1 --2- 172.21.15.67:0/1053959419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348162e1b0 0x5634819e8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.573+0000 7f1b2de69740 1 --2- 172.21.15.67:0/1053959419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56348160d4d0 0x5634819dcd20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.573+0000 7f1b2de69740 1 --2- 172.21.15.67:0/1053959419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634816052f0 0x5634819dc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.573+0000 7f1b2de69740 1 -- 172.21.15.67:0/1053959419 >> 172.21.15.67:0/1053959419 conn(0x563481622300 msgr2=0x5634819cd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.574+0000 7f1afffff640 0 client.27311 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.581+0000 7f1b2de69740 1 -- 172.21.15.67:0/1053959419 shutdown_connections 2024-07-31T15:01:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.581+0000 7f1b2de69740 1 -- 172.21.15.67:0/1053959419 wait complete. 2024-07-31T15:01:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.581+0000 7f1b2de69740 20 client.27311 trim_cache size 0 max 16384 2024-07-31T15:01:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.608+0000 7f1b2de69740 1 Processor -- start 2024-07-31T15:01:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.609+0000 7f1b2de69740 1 -- start start 2024-07-31T15:01:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.609+0000 7f1b2de69740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634816052f0 0x5634819d6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.609+0000 7f1b2de69740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348160d4d0 0x5634819dbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.609+0000 7f1b24d0c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634816052f0 0x5634819d6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.609+0000 7f1b24d0c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634816052f0 0x5634819d6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46468/0 (socket says 172.21.15.67:46468) 2024-07-31T15:01:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.609+0000 7f1b1ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348160d4d0 0x5634819dbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.609+0000 7f1b1ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348160d4d0 0x5634819dbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37300/0 (socket says 172.21.15.67:37300) 2024-07-31T15:01:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.609+0000 7f1b24d0c640 1 -- 172.21.15.67:0/85695749 learned_addr learned my addr 172.21.15.67:0/85695749 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.609+0000 7f1b2de69740 1 --2- 172.21.15.67:0/85695749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56348162e1b0 0x5634819ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.610+0000 7f1b2de69740 1 -- 172.21.15.67:0/85695749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c00fed0 con 0x56348162e1b0 2024-07-31T15:01:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.610+0000 7f1b2de69740 1 -- 172.21.15.67:0/85695749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c011ad0 con 0x5634816052f0 2024-07-31T15:01:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.610+0000 7f1b2de69740 1 -- 172.21.15.67:0/85695749 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c0105a0 con 0x56348160d4d0 2024-07-31T15:01:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.610+0000 7f1b2550d640 1 --2- 172.21.15.67:0/85695749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56348162e1b0 0x5634819ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.610+0000 7f1b1ffff640 1 -- 172.21.15.67:0/85695749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634816052f0 msgr2=0x5634819d6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.610+0000 7f1b1ffff640 1 --2- 172.21.15.67:0/85695749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634816052f0 0x5634819d6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.610+0000 7f1b1ffff640 1 -- 172.21.15.67:0/85695749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56348162e1b0 msgr2=0x5634819ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.610+0000 7f1b1ffff640 1 --2- 172.21.15.67:0/85695749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56348162e1b0 0x5634819ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.610+0000 7f1b1ffff640 1 -- 172.21.15.67:0/85695749 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634819dd890 con 0x56348160d4d0 2024-07-31T15:01:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.611+0000 7f1b1ffff640 1 --2- 172.21.15.67:0/85695749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348160d4d0 0x5634819dbe40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1b100d9a00 tx=0x7f1b1001a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cdd84f073114758 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.611+0000 7f1b067fc640 1 -- 172.21.15.67:0/85695749 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b100dc0e0 con 0x56348160d4d0 2024-07-31T15:01:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.611+0000 7f1b067fc640 1 -- 172.21.15.67:0/85695749 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b100dc790 con 0x56348160d4d0 2024-07-31T15:01:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.611+0000 7f1b067fc640 1 -- 172.21.15.67:0/85695749 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b100ff110 con 0x56348160d4d0 2024-07-31T15:01:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.612+0000 7f1b2de69740 1 -- 172.21.15.67:0/85695749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348160d4d0 msgr2=0x5634819dbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.612+0000 7f1b2de69740 1 --2- 172.21.15.67:0/85695749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348160d4d0 0x5634819dbe40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1b100d9a00 tx=0x7f1b1001a5d0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.613+0000 7f1b2de69740 1 -- 172.21.15.67:0/85695749 shutdown_connections 2024-07-31T15:01:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.613+0000 7f1b2de69740 1 --2- 172.21.15.67:0/85695749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56348162e1b0 0x5634819ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.613+0000 7f1b2de69740 1 --2- 172.21.15.67:0/85695749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348160d4d0 0x5634819dbe40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.613+0000 7f1b2de69740 1 --2- 172.21.15.67:0/85695749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634816052f0 0x5634819d6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.613+0000 7f1b2de69740 1 -- 172.21.15.67:0/85695749 >> 172.21.15.67:0/85695749 conn(0x563481622300 msgr2=0x5634819ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.613+0000 7f1b2de69740 1 -- 172.21.15.67:0/85695749 shutdown_connections 2024-07-31T15:01:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.613+0000 7f1b2de69740 1 -- 172.21.15.67:0/85695749 wait complete. 2024-07-31T15:01:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.614+0000 7f1b2de69740 1 -- start start 2024-07-31T15:01:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.614+0000 7f1b2de69740 1 Processor -- start 2024-07-31T15:01:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.615+0000 7f1b2de69740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634816052f0 0x5634819e3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.615+0000 7f1b2de69740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56348160d4d0 0x5634819e3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.615+0000 7f1b2de69740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348162e1b0 0x5634819e4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.615+0000 7f1b2de69740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c011400 con 0x5634816052f0 2024-07-31T15:01:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.615+0000 7f1b2de69740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c010c70 con 0x56348160d4d0 2024-07-31T15:01:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.615+0000 7f1b2de69740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c00df60 con 0x56348162e1b0 2024-07-31T15:01:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.615+0000 7f1b2de69740 10 client.0 _ll_register_callbacks cb 0x7f1b1001ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.615+0000 7f1b24d0c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634816052f0 0x5634819e3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.615+0000 7f1b24d0c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634816052f0 0x5634819e3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44624/0 (socket says 172.21.15.67:44624) 2024-07-31T15:01:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.615+0000 7f1b24d0c640 1 -- 172.21.15.67:0/3269928376 learned_addr learned my addr 172.21.15.67:0/3269928376 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.616+0000 7f1b24d0c640 1 -- 172.21.15.67:0/3269928376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348162e1b0 msgr2=0x5634819e4640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:01:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.616+0000 7f1b24d0c640 1 --2- 172.21.15.67:0/3269928376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348162e1b0 0x5634819e4640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.616+0000 7f1b24d0c640 1 -- 172.21.15.67:0/3269928376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56348160d4d0 msgr2=0x5634819e3f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:01:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.616+0000 7f1b24d0c640 1 --2- 172.21.15.67:0/3269928376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56348160d4d0 0x5634819e3f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.616+0000 7f1b24d0c640 1 -- 172.21.15.67:0/3269928376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634819e4dd0 con 0x5634816052f0 2024-07-31T15:01:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.616+0000 7f1b24d0c640 1 --2- 172.21.15.67:0/3269928376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634816052f0 0x5634819e3990 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009e30 tx=0x7f1b14009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5188f9f7a9834246 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.616+0000 7f1afffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:01:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.617+0000 7f1afffff640 1 -- 172.21.15.67:0/3269928376 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14016c90 con 0x5634816052f0 2024-07-31T15:01:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.617+0000 7f1afffff640 1 -- 172.21.15.67:0/3269928376 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f1b14005970 con 0x5634816052f0 2024-07-31T15:01:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.617+0000 7f1b2de69740 1 -- 172.21.15.67:0/3269928376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b100dc790 con 0x5634816052f0 2024-07-31T15:01:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.621+0000 7f1afffff640 1 -- 172.21.15.67:0/3269928376 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1b1402c050 con 0x5634816052f0 2024-07-31T15:01:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.621+0000 7f1b2de69740 10 client.17763 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.621+0000 7f1b2de69740 1 -- 172.21.15.67:0/3269928376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563481a77cf0 con 0x5634816052f0 2024-07-31T15:01:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.625+0000 7f1afffff640 1 -- 172.21.15.67:0/3269928376 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b14023070 con 0x5634816052f0 2024-07-31T15:01:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.625+0000 7f1afffff640 1 -- 172.21.15.67:0/3269928376 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7f1b14015410 con 0x5634816052f0 2024-07-31T15:01:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.626+0000 7f1b2de69740 10 client.17763 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.626+0000 7f1b2de69740 10 client.17763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:01:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.626+0000 7f1b2de69740 20 client.17763 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.626+0000 7f1b2de69740 1 --2- 172.21.15.67:0/3269928376 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5634815c21c0 0x563481a7a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.626+0000 7f1b2de69740 4 client.17763 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:01:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.626+0000 7f1b2de69740 1 -- 172.21.15.67:0/3269928376 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1b100dc790 con 0x5634815c21c0 2024-07-31T15:01:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.628+0000 7f1b1ffff640 1 --2- 172.21.15.67:0/3269928376 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5634815c21c0 0x563481a7a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.632+0000 7f1b1ffff640 1 --2- 172.21.15.67:0/3269928376 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5634815c21c0 0x563481a7a5a0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52746fdf6b1466e server_cookie=fd6aa44152e4d583 in_seq=0 out_seq=0 2024-07-31T15:01:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:26.632+0000 7f1afffff640 10 client.17763 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.102+0000 7f1afffff640 1 -- 172.21.15.67:0/3269928376 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1b100dc790 con 0x5634815c21c0 2024-07-31T15:01:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.102+0000 7f1afffff640 10 client.17763 handle_command_reply: tid=0 2024-07-31T15:01:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.102+0000 7f1b2de69740 1 client.17763 shutdown 2024-07-31T15:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.103+0000 7f1b2de69740 1 -- 172.21.15.67:0/3269928376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634816052f0 msgr2=0x5634819e3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.103+0000 7f1b2de69740 1 --2- 172.21.15.67:0/3269928376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634816052f0 0x5634819e3990 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009e30 tx=0x7f1b14009f20 comp rx=0 tx=0).stop 2024-07-31T15:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.103+0000 7f1b2de69740 1 -- 172.21.15.67:0/3269928376 shutdown_connections 2024-07-31T15:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.103+0000 7f1b2de69740 1 --2- 172.21.15.67:0/3269928376 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5634815c21c0 0x563481a7a5a0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.103+0000 7f1afffff640 0 client.17763 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.103+0000 7f1b2de69740 1 --2- 172.21.15.67:0/3269928376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56348162e1b0 0x5634819e4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.103+0000 7f1b2de69740 1 --2- 172.21.15.67:0/3269928376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56348160d4d0 0x5634819e3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.103+0000 7f1b2de69740 1 --2- 172.21.15.67:0/3269928376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634816052f0 0x5634819e3990 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.103+0000 7f1b2de69740 1 -- 172.21.15.67:0/3269928376 >> 172.21.15.67:0/3269928376 conn(0x563481622300 msgr2=0x5634819d1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.103+0000 7f1b2de69740 1 -- 172.21.15.67:0/3269928376 shutdown_connections 2024-07-31T15:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.104+0000 7f1b2de69740 1 -- 172.21.15.67:0/3269928376 wait complete. 2024-07-31T15:01:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.104+0000 7f1b2de69740 20 client.17763 trim_cache size 0 max 16384 2024-07-31T15:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:01:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.105+0000 7f1b26f70640 1 -- 172.21.15.67:0/2589299094 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1b080b3f50 msgr2=0x7f1b080b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.105+0000 7f1b26f70640 1 --2- 172.21.15.67:0/2589299094 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1b080b3f50 0x7f1b080b6460 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1b18015790 tx=0x7f1b1800b000 comp rx=0 tx=0).stop 2024-07-31T15:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.105+0000 7f1b26f70640 1 -- 172.21.15.67:0/2589299094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b20106d30 msgr2=0x7f1b20074330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.105+0000 7f1b26f70640 1 --2- 172.21.15.67:0/2589299094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b20106d30 0x7f1b20074330 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1b10041040 tx=0x7f1b10040610 comp rx=0 tx=0).stop 2024-07-31T15:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.105+0000 7f1b26f70640 1 -- 172.21.15.67:0/2589299094 shutdown_connections 2024-07-31T15:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.105+0000 7f1b26f70640 1 --2- 172.21.15.67:0/2589299094 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1b080b3f50 0x7f1b080b6460 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.105+0000 7f1b26f70640 1 --2- 172.21.15.67:0/2589299094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b20107680 0x7f1b2010f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.105+0000 7f1b26f70640 1 --2- 172.21.15.67:0/2589299094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b20106d30 0x7f1b20074330 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.105+0000 7f1b26f70640 1 --2- 172.21.15.67:0/2589299094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b20101a50 0x7f1b20073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.105+0000 7f1b26f70640 1 -- 172.21.15.67:0/2589299094 >> 172.21.15.67:0/2589299094 conn(0x7f1b20079880 msgr2=0x7f1b201005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.105+0000 7f1b26f70640 1 -- 172.21.15.67:0/2589299094 shutdown_connections 2024-07-31T15:01:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:27.105+0000 7f1b26f70640 1 -- 172.21.15.67:0/2589299094 wait complete. 2024-07-31T15:01:27.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:01:27.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:27 smithi073 ceph-mon[36714]: pgmap v721: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 385 KiB/s wr, 143 op/s 2024-07-31T15:01:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:27 smithi077 ceph-mon[37327]: pgmap v721: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 385 KiB/s wr, 143 op/s 2024-07-31T15:01:27.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:27 smithi067 ceph-mon[32750]: pgmap v721: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 385 KiB/s wr, 143 op/s 2024-07-31T15:01:27.653 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/generated/bounds.h 2024-07-31T15:01:27.670 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/asm-offsets.s 2024-07-31T15:01:27.811 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:01:28.991 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-07-31T15:01:28.991 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:01:29.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:29 smithi067 ceph-mon[32750]: pgmap v722: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 359 KiB/s wr, 118 op/s 2024-07-31T15:01:29.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:29 smithi073 ceph-mon[36714]: pgmap v722: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 359 KiB/s wr, 118 op/s 2024-07-31T15:01:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:29 smithi077 ceph-mon[37327]: pgmap v722: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 359 KiB/s wr, 118 op/s 2024-07-31T15:01:30.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.001+0000 7efc8d454640 1 -- 172.21.15.67:0/147584252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc88107dd0 msgr2=0x7efc881081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:30.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.001+0000 7efc8d454640 1 --2- 172.21.15.67:0/147584252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc88107dd0 0x7efc881081e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7efc7800d910 tx=0x7efc7802e5c0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.001+0000 7efc8d454640 1 -- 172.21.15.67:0/147584252 shutdown_connections 2024-07-31T15:01:30.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.001+0000 7efc8d454640 1 --2- 172.21.15.67:0/147584252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc88108720 0x7efc8810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.001+0000 7efc8d454640 1 --2- 172.21.15.67:0/147584252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc88107dd0 0x7efc881081e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.001+0000 7efc8d454640 1 --2- 172.21.15.67:0/147584252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc88102490 0x7efc881028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.001+0000 7efc8d454640 1 -- 172.21.15.67:0/147584252 >> 172.21.15.67:0/147584252 conn(0x7efc88079160 msgr2=0x7efc88079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.005+0000 7efc8d454640 1 -- 172.21.15.67:0/147584252 shutdown_connections 2024-07-31T15:01:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.005+0000 7efc8d454640 1 -- 172.21.15.67:0/147584252 wait complete. 2024-07-31T15:01:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.007+0000 7efc8d454640 1 Processor -- start 2024-07-31T15:01:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.007+0000 7efc8d454640 1 -- start start 2024-07-31T15:01:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.007+0000 7efc8d454640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc88102490 0x7efc880749c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.007+0000 7efc8d454640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc88107dd0 0x7efc8806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.007+0000 7efc8d454640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc88108720 0x7efc8806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.007+0000 7efc8d454640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efc88119be0 con 0x7efc88107dd0 2024-07-31T15:01:30.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.007+0000 7efc8d454640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc88119a60 con 0x7efc88108720 2024-07-31T15:01:30.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.007+0000 7efc8d454640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efc88119d60 con 0x7efc88102490 2024-07-31T15:01:30.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.008+0000 7efc877fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc88108720 0x7efc8806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.008+0000 7efc877fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc88108720 0x7efc8806e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46500/0 (socket says 172.21.15.67:46500) 2024-07-31T15:01:30.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.008+0000 7efc877fe640 1 -- 172.21.15.67:0/3757540284 learned_addr learned my addr 172.21.15.67:0/3757540284 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:30.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.008+0000 7efc86ffd640 1 --2- 172.21.15.67:0/3757540284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc88102490 0x7efc880749c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.009+0000 7efc86ffd640 1 -- 172.21.15.67:0/3757540284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc88108720 msgr2=0x7efc8806e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.009+0000 7efc86ffd640 1 --2- 172.21.15.67:0/3757540284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc88108720 0x7efc8806e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.009+0000 7efc86ffd640 1 -- 172.21.15.67:0/3757540284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc88107dd0 msgr2=0x7efc8806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.009+0000 7efc86ffd640 1 --2- 172.21.15.67:0/3757540284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc88107dd0 0x7efc8806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.009+0000 7efc86ffd640 1 -- 172.21.15.67:0/3757540284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc88073630 con 0x7efc88102490 2024-07-31T15:01:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.010+0000 7efc86ffd640 1 --2- 172.21.15.67:0/3757540284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc88102490 0x7efc880749c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009fd0 tx=0x7efc7c00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:30.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.019+0000 7efc6ffff640 1 -- 172.21.15.67:0/3757540284 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c015020 con 0x7efc88102490 2024-07-31T15:01:30.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.019+0000 7efc6ffff640 1 -- 172.21.15.67:0/3757540284 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc7c011ce0 con 0x7efc88102490 2024-07-31T15:01:30.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.019+0000 7efc6ffff640 1 -- 172.21.15.67:0/3757540284 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c010e10 con 0x7efc88102490 2024-07-31T15:01:30.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.019+0000 7efc8d454640 1 -- 172.21.15.67:0/3757540284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc88073920 con 0x7efc88102490 2024-07-31T15:01:30.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.019+0000 7efc8d454640 1 -- 172.21.15.67:0/3757540284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc88073e10 con 0x7efc88102490 2024-07-31T15:01:30.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.025+0000 7efc6ffff640 1 -- 172.21.15.67:0/3757540284 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7efc7c018020 con 0x7efc88102490 2024-07-31T15:01:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.026+0000 7efc939a5740 1 Processor -- start 2024-07-31T15:01:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.026+0000 7efc939a5740 1 -- start start 2024-07-31T15:01:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.027+0000 7efc939a5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f557070 0x56148f8fc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.027+0000 7efc939a5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f52e1b0 0x56148f8fca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.027+0000 7efc939a5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f536390 0x56148f8fcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.027+0000 7efc939a5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56148f872b90 con 0x56148f52e1b0 2024-07-31T15:01:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.027+0000 7efc939a5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56148f862090 con 0x56148f557070 2024-07-31T15:01:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.027+0000 7efc939a5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56148f8660d0 con 0x56148f536390 2024-07-31T15:01:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.027+0000 7efc877fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f557070 0x56148f8fc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.027+0000 7efc877fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f557070 0x56148f8fc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46508/0 (socket says 172.21.15.67:46508) 2024-07-31T15:01:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.027+0000 7efc877fe640 1 -- 172.21.15.67:0/3617925849 learned_addr learned my addr 172.21.15.67:0/3617925849 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.027+0000 7efc867fc640 1 --2- 172.21.15.67:0/3617925849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f536390 0x56148f8fcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.027+0000 7efc86ffd640 1 --2- 172.21.15.67:0/3617925849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f52e1b0 0x56148f8fca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.028+0000 7efc877fe640 1 -- 172.21.15.67:0/3617925849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f536390 msgr2=0x56148f8fcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.028+0000 7efc877fe640 1 --2- 172.21.15.67:0/3617925849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f536390 0x56148f8fcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.028+0000 7efc877fe640 1 -- 172.21.15.67:0/3617925849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f52e1b0 msgr2=0x56148f8fca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.028+0000 7efc877fe640 1 --2- 172.21.15.67:0/3617925849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f52e1b0 0x56148f8fca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.028+0000 7efc877fe640 1 -- 172.21.15.67:0/3617925849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56148f808b70 con 0x56148f557070 2024-07-31T15:01:30.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.028+0000 7efc6ffff640 1 --2- 172.21.15.67:0/3757540284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efc700b3f40 0x7efc700b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.028+0000 7efc877fe640 1 --2- 172.21.15.67:0/3617925849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f557070 0x56148f8fc550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc800073a0 tx=0x7efc80007860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96a2b4da4d967cb8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:30.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.029+0000 7efc6ffff640 1 -- 172.21.15.67:0/3757540284 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7efc7c0e3b30 con 0x7efc88102490 2024-07-31T15:01:30.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.029+0000 7efc6d7fa640 1 -- 172.21.15.67:0/3617925849 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80019070 con 0x56148f557070 2024-07-31T15:01:30.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.029+0000 7efc6d7fa640 1 -- 172.21.15.67:0/3617925849 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc80012bc0 con 0x56148f557070 2024-07-31T15:01:30.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.030+0000 7efc939a5740 1 -- 172.21.15.67:0/3617925849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f557070 msgr2=0x56148f8fc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:30.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.030+0000 7efc939a5740 1 --2- 172.21.15.67:0/3617925849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f557070 0x56148f8fc550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc800073a0 tx=0x7efc80007860 comp rx=0 tx=0).stop 2024-07-31T15:01:30.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.030+0000 7efc939a5740 1 -- 172.21.15.67:0/3617925849 shutdown_connections 2024-07-31T15:01:30.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.030+0000 7efc939a5740 1 --2- 172.21.15.67:0/3617925849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f536390 0x56148f8fcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.030+0000 7efc939a5740 1 --2- 172.21.15.67:0/3617925849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f52e1b0 0x56148f8fca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.030+0000 7efc939a5740 1 --2- 172.21.15.67:0/3617925849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f557070 0x56148f8fc550 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.030+0000 7efc939a5740 1 -- 172.21.15.67:0/3617925849 >> 172.21.15.67:0/3617925849 conn(0x56148f54b460 msgr2=0x56148f8fa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:30.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.036+0000 7efc939a5740 1 -- 172.21.15.67:0/3617925849 shutdown_connections 2024-07-31T15:01:30.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.036+0000 7efc86ffd640 1 --2- 172.21.15.67:0/3757540284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efc700b3f40 0x7efc700b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.036+0000 7efc939a5740 1 -- 172.21.15.67:0/3617925849 wait complete. 2024-07-31T15:01:30.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.037+0000 7efc939a5740 1 -- start start 2024-07-31T15:01:30.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.037+0000 7efc939a5740 1 Processor -- start 2024-07-31T15:01:30.040 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:01:30.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.038+0000 7efc939a5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f52e1b0 0x56148f908130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.038+0000 7efc939a5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f536390 0x56148f908670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.038+0000 7efc939a5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f557070 0x56148f9250d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.038+0000 7efc939a5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56148f6c5700 con 0x56148f557070 2024-07-31T15:01:30.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.038+0000 7efc939a5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56148f8049c0 con 0x56148f52e1b0 2024-07-31T15:01:30.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.038+0000 7efc939a5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56148f91bd90 con 0x56148f536390 2024-07-31T15:01:30.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.038+0000 7efc939a5740 10 client.0 _ll_register_callbacks cb 0x56148f81f4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:30.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.038+0000 7efc877fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f52e1b0 0x56148f908130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.038+0000 7efc877fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f52e1b0 0x56148f908130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46516/0 (socket says 172.21.15.67:46516) 2024-07-31T15:01:30.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.038+0000 7efc877fe640 1 -- 172.21.15.67:0/548318967 learned_addr learned my addr 172.21.15.67:0/548318967 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:30.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.038+0000 7efc877fe640 1 --2- 172.21.15.67:0/548318967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f557070 0x56148f9250d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.041+0000 7efc877fe640 1 -- 172.21.15.67:0/548318967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f536390 msgr2=0x56148f908670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:01:30.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.041+0000 7efc877fe640 1 --2- 172.21.15.67:0/548318967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f536390 0x56148f908670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.041+0000 7efc877fe640 1 -- 172.21.15.67:0/548318967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f557070 msgr2=0x56148f9250d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:30.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.041+0000 7efc877fe640 1 --2- 172.21.15.67:0/548318967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f557070 0x56148f9250d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.041+0000 7efc877fe640 1 -- 172.21.15.67:0/548318967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56148f5c2650 con 0x56148f52e1b0 2024-07-31T15:01:30.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.043+0000 7efc877fe640 1 --2- 172.21.15.67:0/548318967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f52e1b0 0x56148f908130 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efc80009da0 tx=0x7efc800147d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f84cb3ab0626273 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:30.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.043+0000 7efc6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:01:30.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.043+0000 7efc6aff5640 1 -- 172.21.15.67:0/548318967 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80028060 con 0x56148f52e1b0 2024-07-31T15:01:30.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.044+0000 7efc939a5740 1 -- 172.21.15.67:0/548318967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc80012bc0 con 0x56148f52e1b0 2024-07-31T15:01:30.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.044+0000 7efc6aff5640 1 -- 172.21.15.67:0/548318967 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7efc80003a10 con 0x56148f52e1b0 2024-07-31T15:01:30.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.044+0000 7efc6aff5640 1 -- 172.21.15.67:0/548318967 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efc80019070 con 0x56148f52e1b0 2024-07-31T15:01:30.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.044+0000 7efc939a5740 10 client.27323 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:30.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.044+0000 7efc939a5740 1 -- 172.21.15.67:0/548318967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56148f927dc0 con 0x56148f52e1b0 2024-07-31T15:01:30.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.045+0000 7efc6aff5640 1 -- 172.21.15.67:0/548318967 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc80012bc0 con 0x56148f52e1b0 2024-07-31T15:01:30.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.047+0000 7efc6aff5640 1 -- 172.21.15.67:0/548318967 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7efc80004680 con 0x56148f52e1b0 2024-07-31T15:01:30.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.047+0000 7efc939a5740 10 client.27323 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:30.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.047+0000 7efc939a5740 10 client.27323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:01:30.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.047+0000 7efc939a5740 20 client.27323 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:30.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.047+0000 7efc939a5740 1 --2- 172.21.15.67:0/548318967 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56148f5159f0 0x56148f92b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.047+0000 7efc939a5740 4 client.27323 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:01:30.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.047+0000 7efc939a5740 1 -- 172.21.15.67:0/548318967 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56148f874ae0 con 0x56148f5159f0 2024-07-31T15:01:30.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.053+0000 7efc867fc640 1 --2- 172.21.15.67:0/548318967 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56148f5159f0 0x56148f92b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.057+0000 7efc86ffd640 1 --2- 172.21.15.67:0/3757540284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efc700b3f40 0x7efc700b6450 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x56148f8fb1d0 tx=0x7efc7c010870 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:30.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.058+0000 7efc867fc640 1 --2- 172.21.15.67:0/548318967 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56148f5159f0 0x56148f92b710 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7e221fd3ec50387 server_cookie=8401940cf739c5b5 in_seq=0 out_seq=0 2024-07-31T15:01:30.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.058+0000 7efc6aff5640 10 client.27323 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:30.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.059+0000 7efc6aff5640 1 -- 172.21.15.67:0/548318967 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56148f874ae0 con 0x56148f5159f0 2024-07-31T15:01:30.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.059+0000 7efc6aff5640 10 client.27323 handle_command_reply: tid=0 2024-07-31T15:01:30.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.063+0000 7efc939a5740 1 client.27323 shutdown 2024-07-31T15:01:30.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.069+0000 7efc939a5740 1 -- 172.21.15.67:0/548318967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f52e1b0 msgr2=0x56148f908130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:30.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.069+0000 7efc939a5740 1 --2- 172.21.15.67:0/548318967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f52e1b0 0x56148f908130 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efc80009da0 tx=0x7efc800147d0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.070+0000 7efc939a5740 1 -- 172.21.15.67:0/548318967 shutdown_connections 2024-07-31T15:01:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.070+0000 7efc939a5740 1 --2- 172.21.15.67:0/548318967 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56148f5159f0 0x56148f92b710 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.071+0000 7efc939a5740 1 --2- 172.21.15.67:0/548318967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f557070 0x56148f9250d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.071+0000 7efc939a5740 1 --2- 172.21.15.67:0/548318967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f536390 0x56148f908670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.071+0000 7efc939a5740 1 --2- 172.21.15.67:0/548318967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f52e1b0 0x56148f908130 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.071+0000 7efc939a5740 1 -- 172.21.15.67:0/548318967 >> 172.21.15.67:0/548318967 conn(0x56148f54b460 msgr2=0x56148f8f8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:30.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.071+0000 7efc6aff5640 0 client.27323 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:30.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.071+0000 7efc939a5740 1 -- 172.21.15.67:0/548318967 shutdown_connections 2024-07-31T15:01:30.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.071+0000 7efc939a5740 1 -- 172.21.15.67:0/548318967 wait complete. 2024-07-31T15:01:30.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.071+0000 7efc939a5740 20 client.27323 trim_cache size 0 max 16384 2024-07-31T15:01:30.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.105+0000 7efc939a5740 1 Processor -- start 2024-07-31T15:01:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.108+0000 7efc939a5740 1 -- start start 2024-07-31T15:01:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.108+0000 7efc939a5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f52e1b0 0x56148f901d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.109+0000 7efc939a5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f536390 0x56148f924ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.109+0000 7efc877fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f52e1b0 0x56148f901d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.109+0000 7efc877fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f52e1b0 0x56148f901d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46528/0 (socket says 172.21.15.67:46528) 2024-07-31T15:01:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.109+0000 7efc877fe640 1 -- 172.21.15.67:0/4086261100 learned_addr learned my addr 172.21.15.67:0/4086261100 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.109+0000 7efc939a5740 1 --2- 172.21.15.67:0/4086261100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f557070 0x56148f930f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.109+0000 7efc867fc640 1 --2- 172.21.15.67:0/4086261100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f536390 0x56148f924ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.109+0000 7efc939a5740 1 -- 172.21.15.67:0/4086261100 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efc881a6340 con 0x56148f536390 2024-07-31T15:01:30.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.109+0000 7efc939a5740 1 -- 172.21.15.67:0/4086261100 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc881a7f40 con 0x56148f52e1b0 2024-07-31T15:01:30.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.109+0000 7efc939a5740 1 -- 172.21.15.67:0/4086261100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efc881a6a10 con 0x56148f557070 2024-07-31T15:01:30.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.110+0000 7efc877fe640 1 --2- 172.21.15.67:0/4086261100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f557070 0x56148f930f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.110+0000 7efc877fe640 1 -- 172.21.15.67:0/4086261100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f557070 msgr2=0x56148f930f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:30.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.110+0000 7efc877fe640 1 --2- 172.21.15.67:0/4086261100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f557070 0x56148f930f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.110+0000 7efc877fe640 1 -- 172.21.15.67:0/4086261100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f536390 msgr2=0x56148f924ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:30.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.110+0000 7efc877fe640 1 --2- 172.21.15.67:0/4086261100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f536390 0x56148f924ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.110+0000 7efc877fe640 1 -- 172.21.15.67:0/4086261100 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56148f9260e0 con 0x56148f52e1b0 2024-07-31T15:01:30.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.110+0000 7efc877fe640 1 --2- 172.21.15.67:0/4086261100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f52e1b0 0x56148f901d70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc800147a0 tx=0x7efc80009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba627c58c61c491a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:30.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.118+0000 7efc6cff9640 1 -- 172.21.15.67:0/4086261100 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80028040 con 0x56148f52e1b0 2024-07-31T15:01:30.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.119+0000 7efc6cff9640 1 -- 172.21.15.67:0/4086261100 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc8001a040 con 0x56148f52e1b0 2024-07-31T15:01:30.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.120+0000 7efc939a5740 1 -- 172.21.15.67:0/4086261100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f52e1b0 msgr2=0x56148f901d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:30.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.120+0000 7efc939a5740 1 --2- 172.21.15.67:0/4086261100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f52e1b0 0x56148f901d70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc800147a0 tx=0x7efc80009b20 comp rx=0 tx=0).stop 2024-07-31T15:01:30.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.120+0000 7efc939a5740 1 -- 172.21.15.67:0/4086261100 shutdown_connections 2024-07-31T15:01:30.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.120+0000 7efc939a5740 1 --2- 172.21.15.67:0/4086261100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f557070 0x56148f930f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.120+0000 7efc939a5740 1 --2- 172.21.15.67:0/4086261100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f536390 0x56148f924ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.120+0000 7efc939a5740 1 --2- 172.21.15.67:0/4086261100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f52e1b0 0x56148f901d70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.120+0000 7efc939a5740 1 -- 172.21.15.67:0/4086261100 >> 172.21.15.67:0/4086261100 conn(0x56148f54b460 msgr2=0x56148f901660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:30.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.121+0000 7efc939a5740 1 -- 172.21.15.67:0/4086261100 shutdown_connections 2024-07-31T15:01:30.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.121+0000 7efc939a5740 1 -- 172.21.15.67:0/4086261100 wait complete. 2024-07-31T15:01:30.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.121+0000 7efc939a5740 1 -- start start 2024-07-31T15:01:30.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.122+0000 7efc939a5740 1 Processor -- start 2024-07-31T15:01:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.122+0000 7efc939a5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f52e1b0 0x56148f921670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.123+0000 7efc939a5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f536390 0x56148f91cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.123+0000 7efc939a5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f557070 0x56148f91d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.123+0000 7efc867fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f536390 0x56148f91cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.123+0000 7efc867fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f536390 0x56148f91cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46540/0 (socket says 172.21.15.67:46540) 2024-07-31T15:01:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.123+0000 7efc867fc640 1 -- 172.21.15.67:0/2182078766 learned_addr learned my addr 172.21.15.67:0/2182078766 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.123+0000 7efc939a5740 1 -- 172.21.15.67:0/2182078766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efc881a7870 con 0x56148f52e1b0 2024-07-31T15:01:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.123+0000 7efc877fe640 1 --2- 172.21.15.67:0/2182078766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f557070 0x56148f91d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.123+0000 7efc939a5740 1 -- 172.21.15.67:0/2182078766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc881a70e0 con 0x56148f536390 2024-07-31T15:01:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.123+0000 7efc939a5740 1 -- 172.21.15.67:0/2182078766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efc88103710 con 0x56148f557070 2024-07-31T15:01:30.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.124+0000 7efc867fc640 1 -- 172.21.15.67:0/2182078766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f557070 msgr2=0x56148f91d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:30.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.124+0000 7efc867fc640 1 --2- 172.21.15.67:0/2182078766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f557070 0x56148f91d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.124+0000 7efc867fc640 1 -- 172.21.15.67:0/2182078766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f52e1b0 msgr2=0x56148f921670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:30.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.124+0000 7efc867fc640 1 --2- 172.21.15.67:0/2182078766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f52e1b0 0x56148f921670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.124+0000 7efc867fc640 1 -- 172.21.15.67:0/2182078766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56148f91d750 con 0x56148f536390 2024-07-31T15:01:30.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.124+0000 7efc867fc640 1 --2- 172.21.15.67:0/2182078766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f536390 0x56148f91cb20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc78037040 tx=0x7efc7800d270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=667dfc50188c62b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:30.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.125+0000 7efc939a5740 10 client.0 _ll_register_callbacks cb 0x7efc80017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:30.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.125+0000 7efc939a5740 1 -- 172.21.15.67:0/2182078766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc8001a040 con 0x56148f536390 2024-07-31T15:01:30.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.125+0000 7efc6aff5640 10 client.27329 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:01:30.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.125+0000 7efc6aff5640 1 -- 172.21.15.67:0/2182078766 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78004040 con 0x56148f536390 2024-07-31T15:01:30.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.125+0000 7efc6aff5640 1 -- 172.21.15.67:0/2182078766 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7efc780041e0 con 0x56148f536390 2024-07-31T15:01:30.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.125+0000 7efc6aff5640 1 -- 172.21.15.67:0/2182078766 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efc78055050 con 0x56148f536390 2024-07-31T15:01:30.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.125+0000 7efc939a5740 10 client.27329 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:30.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.125+0000 7efc939a5740 1 -- 172.21.15.67:0/2182078766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=58}) -- 0x56148f993cb0 con 0x56148f536390 2024-07-31T15:01:30.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.126+0000 7efc6aff5640 1 -- 172.21.15.67:0/2182078766 <== mon.2 v2:172.21.15.73:3300/0 4 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7efc780046c0 con 0x56148f536390 2024-07-31T15:01:30.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.131+0000 7efc939a5740 10 client.27329 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:30.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.131+0000 7efc939a5740 10 client.27329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:01:30.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.131+0000 7efc939a5740 20 client.27329 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:30.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.131+0000 7efc939a5740 1 --2- 172.21.15.67:0/2182078766 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56148f4eada0 0x56148f996380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:30.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.131+0000 7efc939a5740 4 client.27329 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:01:30.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.131+0000 7efc939a5740 1 -- 172.21.15.67:0/2182078766 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7efc8001a040 con 0x56148f4eada0 2024-07-31T15:01:30.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.132+0000 7efc877fe640 1 --2- 172.21.15.67:0/2182078766 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56148f4eada0 0x56148f996380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.139+0000 7efc6aff5640 10 client.27329 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.139+0000 7efc877fe640 1 --2- 172.21.15.67:0/2182078766 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56148f4eada0 0x56148f996380 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9d5e105a9fcf024 server_cookie=205bb38c47632ef5 in_seq=0 out_seq=0 2024-07-31T15:01:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.140+0000 7efc6aff5640 1 -- 172.21.15.67:0/2182078766 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7efc8001a040 con 0x56148f4eada0 2024-07-31T15:01:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.140+0000 7efc6aff5640 10 client.27329 handle_command_reply: tid=0 2024-07-31T15:01:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.140+0000 7efc939a5740 1 client.27329 shutdown 2024-07-31T15:01:30.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.140+0000 7efc939a5740 1 -- 172.21.15.67:0/2182078766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f536390 msgr2=0x56148f91cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:30.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.140+0000 7efc939a5740 1 --2- 172.21.15.67:0/2182078766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f536390 0x56148f91cb20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc78037040 tx=0x7efc7800d270 comp rx=0 tx=0).stop 2024-07-31T15:01:30.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.143+0000 7efc939a5740 1 -- 172.21.15.67:0/2182078766 shutdown_connections 2024-07-31T15:01:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.143+0000 7efc939a5740 1 --2- 172.21.15.67:0/2182078766 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56148f4eada0 0x56148f996380 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.143+0000 7efc939a5740 1 --2- 172.21.15.67:0/2182078766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148f557070 0x56148f91d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.143+0000 7efc939a5740 1 --2- 172.21.15.67:0/2182078766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56148f536390 0x56148f91cb20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.143+0000 7efc939a5740 1 --2- 172.21.15.67:0/2182078766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56148f52e1b0 0x56148f921670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.143+0000 7efc939a5740 1 -- 172.21.15.67:0/2182078766 >> 172.21.15.67:0/2182078766 conn(0x56148f54b460 msgr2=0x56148f8f9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.143+0000 7efc6aff5640 0 client.27329 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.144+0000 7efc939a5740 1 -- 172.21.15.67:0/2182078766 shutdown_connections 2024-07-31T15:01:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.144+0000 7efc939a5740 1 -- 172.21.15.67:0/2182078766 wait complete. 2024-07-31T15:01:30.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.145+0000 7efc939a5740 20 client.27329 trim_cache size 0 max 16384 2024-07-31T15:01:30.151 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:01:30.151 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T15:01:30.151 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T15:01:30.151 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:01:30.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.150+0000 7efc8d454640 1 -- 172.21.15.67:0/3757540284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efc700b3f40 msgr2=0x7efc700b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:30.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.150+0000 7efc8d454640 1 --2- 172.21.15.67:0/3757540284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efc700b3f40 0x7efc700b6450 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x56148f8fb1d0 tx=0x7efc7c010870 comp rx=0 tx=0).stop 2024-07-31T15:01:30.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.150+0000 7efc8d454640 1 -- 172.21.15.67:0/3757540284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc88102490 msgr2=0x7efc880749c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:30.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.150+0000 7efc8d454640 1 --2- 172.21.15.67:0/3757540284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc88102490 0x7efc880749c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009fd0 tx=0x7efc7c00b870 comp rx=0 tx=0).stop 2024-07-31T15:01:30.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.150+0000 7efc8d454640 1 -- 172.21.15.67:0/3757540284 shutdown_connections 2024-07-31T15:01:30.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.150+0000 7efc8d454640 1 --2- 172.21.15.67:0/3757540284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efc700b3f40 0x7efc700b6450 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.150+0000 7efc8d454640 1 --2- 172.21.15.67:0/3757540284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc88108720 0x7efc8806e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.150+0000 7efc8d454640 1 --2- 172.21.15.67:0/3757540284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc88107dd0 0x7efc8806dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.150+0000 7efc8d454640 1 --2- 172.21.15.67:0/3757540284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc88102490 0x7efc880749c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:30.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.150+0000 7efc8d454640 1 -- 172.21.15.67:0/3757540284 >> 172.21.15.67:0/3757540284 conn(0x7efc88079160 msgr2=0x7efc88103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:30.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.151+0000 7efc8d454640 1 -- 172.21.15.67:0/3757540284 shutdown_connections 2024-07-31T15:01:30.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:30.151+0000 7efc8d454640 1 -- 172.21.15.67:0/3757540284 wait complete. 2024-07-31T15:01:31.219 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T15:01:31.219 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:01:31.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:31 smithi073 ceph-mon[36714]: pgmap v723: 129 pgs: 51 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 60 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 400 KiB/s wr, 125 op/s 2024-07-31T15:01:31.406 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:31 smithi067 ceph-mon[32750]: pgmap v723: 129 pgs: 51 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 60 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 400 KiB/s wr, 125 op/s 2024-07-31T15:01:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:31 smithi077 ceph-mon[37327]: pgmap v723: 129 pgs: 51 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 60 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 400 KiB/s wr, 125 op/s 2024-07-31T15:01:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:01:33.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:33 smithi067 ceph-mon[32750]: pgmap v724: 129 pgs: 41 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 71 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 360 KiB/s wr, 123 op/s 2024-07-31T15:01:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:33 smithi077 ceph-mon[37327]: pgmap v724: 129 pgs: 41 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 71 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 360 KiB/s wr, 123 op/s 2024-07-31T15:01:33.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:33 smithi073 ceph-mon[36714]: pgmap v724: 129 pgs: 41 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 71 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 360 KiB/s wr, 123 op/s 2024-07-31T15:01:33.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.635+0000 7fbc982b3640 1 -- 172.21.15.67:0/4156081233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc90111970 msgr2=0x7fbc90109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:33.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.635+0000 7fbc982b3640 1 --2- 172.21.15.67:0/4156081233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc90111970 0x7fbc90109df0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009900 tx=0x7fbc8002a440 comp rx=0 tx=0).stop 2024-07-31T15:01:33.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.638+0000 7fbc982b3640 1 -- 172.21.15.67:0/4156081233 shutdown_connections 2024-07-31T15:01:33.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.638+0000 7fbc982b3640 1 --2- 172.21.15.67:0/4156081233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc9011c660 0x7fbc9011eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.638+0000 7fbc982b3640 1 --2- 172.21.15.67:0/4156081233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc9010a410 0x7fbc9010a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.638+0000 7fbc982b3640 1 --2- 172.21.15.67:0/4156081233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc90111970 0x7fbc90109df0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.638+0000 7fbc982b3640 1 -- 172.21.15.67:0/4156081233 >> 172.21.15.67:0/4156081233 conn(0x7fbc900708f0 msgr2=0x7fbc90070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.640+0000 7fbc982b3640 1 -- 172.21.15.67:0/4156081233 shutdown_connections 2024-07-31T15:01:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.640+0000 7fbc982b3640 1 -- 172.21.15.67:0/4156081233 wait complete. 2024-07-31T15:01:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.641+0000 7fbc982b3640 1 Processor -- start 2024-07-31T15:01:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.641+0000 7fbc982b3640 1 -- start start 2024-07-31T15:01:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.642+0000 7fbc982b3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc9010a410 0x7fbc90119e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.642+0000 7fbc982b3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc90111970 0x7fbc901130e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.642+0000 7fbc982b3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc9011c660 0x7fbc90113640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.642+0000 7fbc982b3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbc90122430 con 0x7fbc9011c660 2024-07-31T15:01:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.642+0000 7fbc982b3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901222b0 con 0x7fbc90111970 2024-07-31T15:01:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.642+0000 7fbc982b3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901225b0 con 0x7fbc9010a410 2024-07-31T15:01:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.642+0000 7fbc96850640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc9011c660 0x7fbc90113640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.642+0000 7fbc96850640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc9011c660 0x7fbc90113640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37866/0 (socket says 172.21.15.67:37866) 2024-07-31T15:01:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.643+0000 7fbc96850640 1 -- 172.21.15.67:0/2233406205 learned_addr learned my addr 172.21.15.67:0/2233406205 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.643+0000 7fbc9604f640 1 --2- 172.21.15.67:0/2233406205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc9010a410 0x7fbc90119e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.643+0000 7fbc96850640 1 -- 172.21.15.67:0/2233406205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc9010a410 msgr2=0x7fbc90119e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.643+0000 7fbc96850640 1 --2- 172.21.15.67:0/2233406205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc9010a410 0x7fbc90119e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.643+0000 7fbc96850640 1 -- 172.21.15.67:0/2233406205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc90111970 msgr2=0x7fbc901130e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.643+0000 7fbc96850640 1 --2- 172.21.15.67:0/2233406205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc90111970 0x7fbc901130e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.643+0000 7fbc96850640 1 -- 172.21.15.67:0/2233406205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc90113e60 con 0x7fbc9011c660 2024-07-31T15:01:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.644+0000 7fbc9604f640 1 --2- 172.21.15.67:0/2233406205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc9010a410 0x7fbc90119e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:01:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.644+0000 7fbc96850640 1 --2- 172.21.15.67:0/2233406205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc9011c660 0x7fbc90113640 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00b3f0 tx=0x7fbc8c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.644+0000 7fbc877fe640 1 -- 172.21.15.67:0/2233406205 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c015020 con 0x7fbc9011c660 2024-07-31T15:01:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.645+0000 7fbc982b3640 1 -- 172.21.15.67:0/2233406205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc90118ae0 con 0x7fbc9011c660 2024-07-31T15:01:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.645+0000 7fbc982b3640 1 -- 172.21.15.67:0/2233406205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc90119020 con 0x7fbc9011c660 2024-07-31T15:01:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.645+0000 7fbc877fe640 1 -- 172.21.15.67:0/2233406205 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc8c011ce0 con 0x7fbc9011c660 2024-07-31T15:01:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.645+0000 7fbc877fe640 1 -- 172.21.15.67:0/2233406205 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c010da0 con 0x7fbc9011c660 2024-07-31T15:01:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.651+0000 7fbc877fe640 1 -- 172.21.15.67:0/2233406205 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbc8c018020 con 0x7fbc9011c660 2024-07-31T15:01:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.651+0000 7fbc982b3640 1 -- 172.21.15.67:0/2233406205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc740052c0 con 0x7fbc9011c660 2024-07-31T15:01:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.666+0000 7fbc877fe640 1 --2- 172.21.15.67:0/2233406205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbc7c0b3f50 0x7fbc7c0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.666+0000 7fbc877fe640 1 -- 172.21.15.67:0/2233406205 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7fbc8c0e3a20 con 0x7fbc9011c660 2024-07-31T15:01:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.668+0000 7fbc9604f640 1 --2- 172.21.15.67:0/2233406205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbc7c0b3f50 0x7fbc7c0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.669+0000 7fbc877fe640 1 -- 172.21.15.67:0/2233406205 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fbc8c01f020 con 0x7fbc9011c660 2024-07-31T15:01:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:33.685+0000 7fbc9604f640 1 --2- 172.21.15.67:0/2233406205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbc7c0b3f50 0x7fbc7c0b6460 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbc80005f70 tx=0x7fbc80031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:34.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:34.130+0000 7fbc982b3640 1 -- 172.21.15.67:0/2233406205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc74005880 con 0x7fbc9011c660 2024-07-31T15:01:34.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:34.132+0000 7fbc877fe640 1 -- 172.21.15.67:0/2233406205 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11969 (secure 0 0 0) 0x7fbc8c0b3430 con 0x7fbc9011c660 2024-07-31T15:01:34.135 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:01:34.135 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":58,"btime":"2024-07-31T15:01:15:058622+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:01:14.559063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:01:34.136 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 58 2024-07-31T15:01:34.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:34.136+0000 7fbc982b3640 1 -- 172.21.15.67:0/2233406205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbc7c0b3f50 msgr2=0x7fbc7c0b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:34.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:34.136+0000 7fbc982b3640 1 --2- 172.21.15.67:0/2233406205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbc7c0b3f50 0x7fbc7c0b6460 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbc80005f70 tx=0x7fbc80031400 comp rx=0 tx=0).stop 2024-07-31T15:01:34.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:34.136+0000 7fbc982b3640 1 -- 172.21.15.67:0/2233406205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc9011c660 msgr2=0x7fbc90113640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:34.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:34.136+0000 7fbc982b3640 1 --2- 172.21.15.67:0/2233406205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc9011c660 0x7fbc90113640 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00b3f0 tx=0x7fbc8c00b8b0 comp rx=0 tx=0).stop 2024-07-31T15:01:34.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:34.137+0000 7fbc982b3640 1 -- 172.21.15.67:0/2233406205 shutdown_connections 2024-07-31T15:01:34.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:34.137+0000 7fbc982b3640 1 --2- 172.21.15.67:0/2233406205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbc7c0b3f50 0x7fbc7c0b6460 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:34.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:34.137+0000 7fbc982b3640 1 --2- 172.21.15.67:0/2233406205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc9011c660 0x7fbc90113640 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:34.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:34.137+0000 7fbc982b3640 1 --2- 172.21.15.67:0/2233406205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc90111970 0x7fbc901130e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:34.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:34.137+0000 7fbc982b3640 1 --2- 172.21.15.67:0/2233406205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc9010a410 0x7fbc90119e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:34.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:34.137+0000 7fbc982b3640 1 -- 172.21.15.67:0/2233406205 >> 172.21.15.67:0/2233406205 conn(0x7fbc900708f0 msgr2=0x7fbc90071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:34.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:34.141+0000 7fbc982b3640 1 -- 172.21.15.67:0/2233406205 shutdown_connections 2024-07-31T15:01:34.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:34.143+0000 7fbc982b3640 1 -- 172.21.15.67:0/2233406205 wait complete. 2024-07-31T15:01:34.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:34 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2233406205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:01:34.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:34 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2233406205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:01:34.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:34 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2233406205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:01:35.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:35 smithi067 ceph-mon[32750]: pgmap v725: 129 pgs: 41 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 72 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 363 KiB/s wr, 130 op/s 2024-07-31T15:01:35.454 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-07-31T15:01:35.455 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-07-31T15:01:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:35 smithi077 ceph-mon[37327]: pgmap v725: 129 pgs: 41 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 72 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 363 KiB/s wr, 130 op/s 2024-07-31T15:01:35.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:35 smithi073 ceph-mon[36714]: pgmap v725: 129 pgs: 41 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 72 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 363 KiB/s wr, 130 op/s 2024-07-31T15:01:36.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.588+0000 7fd40759e640 1 -- 172.21.15.67:0/777331167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4000956b0 msgr2=0x7fd400099be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:36.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.588+0000 7fd40759e640 1 --2- 172.21.15.67:0/777331167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4000956b0 0x7fd400099be0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd3f801a910 tx=0x7fd3f803b450 comp rx=0 tx=0).stop 2024-07-31T15:01:36.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.594+0000 7fd40759e640 1 -- 172.21.15.67:0/777331167 shutdown_connections 2024-07-31T15:01:36.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.594+0000 7fd40759e640 1 --2- 172.21.15.67:0/777331167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd40009a1a0 0x7fd40009a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:36.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.594+0000 7fd40759e640 1 --2- 172.21.15.67:0/777331167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4000956b0 0x7fd400099be0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:36.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.594+0000 7fd40759e640 1 --2- 172.21.15.67:0/777331167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd400094d60 0x7fd400095170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:36.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.594+0000 7fd40759e640 1 -- 172.21.15.67:0/777331167 >> 172.21.15.67:0/777331167 conn(0x7fd40000cd70 msgr2=0x7fd40000d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:36.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.597+0000 7fd40759e640 1 -- 172.21.15.67:0/777331167 shutdown_connections 2024-07-31T15:01:36.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.597+0000 7fd40759e640 1 -- 172.21.15.67:0/777331167 wait complete. 2024-07-31T15:01:36.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.601+0000 7fd40759e640 1 Processor -- start 2024-07-31T15:01:36.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.601+0000 7fd40759e640 1 -- start start 2024-07-31T15:01:36.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.602+0000 7fd40759e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd400094d60 0x7fd4000a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:36.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.602+0000 7fd40759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4000956b0 0x7fd4000a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:36.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.602+0000 7fd40759e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd40009a1a0 0x7fd4000a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:36.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.602+0000 7fd40759e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd40009daa0 con 0x7fd40009a1a0 2024-07-31T15:01:36.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.602+0000 7fd40759e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd40009d920 con 0x7fd400094d60 2024-07-31T15:01:36.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.602+0000 7fd40759e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd40009dc20 con 0x7fd4000956b0 2024-07-31T15:01:36.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.602+0000 7fd406d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd40009a1a0 0x7fd4000a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:36.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.602+0000 7fd405d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4000956b0 0x7fd4000a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:36.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.602+0000 7fd405d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4000956b0 0x7fd4000a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:40718/0 (socket says 172.21.15.67:40718) 2024-07-31T15:01:36.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.602+0000 7fd405d9b640 1 -- 172.21.15.67:0/3444647363 learned_addr learned my addr 172.21.15.67:0/3444647363 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:36.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.603+0000 7fd40659c640 1 --2- 172.21.15.67:0/3444647363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd400094d60 0x7fd4000a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:36.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.603+0000 7fd405d9b640 1 -- 172.21.15.67:0/3444647363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd400094d60 msgr2=0x7fd4000a92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:36.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.603+0000 7fd405d9b640 1 --2- 172.21.15.67:0/3444647363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd400094d60 0x7fd4000a92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:36.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.603+0000 7fd405d9b640 1 -- 172.21.15.67:0/3444647363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd40009a1a0 msgr2=0x7fd4000a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:36.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.603+0000 7fd405d9b640 1 --2- 172.21.15.67:0/3444647363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd40009a1a0 0x7fd4000a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:36.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.603+0000 7fd405d9b640 1 -- 172.21.15.67:0/3444647363 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4000a6380 con 0x7fd4000956b0 2024-07-31T15:01:36.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.604+0000 7fd405d9b640 1 --2- 172.21.15.67:0/3444647363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4000956b0 0x7fd4000a55a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd3f803ba40 tx=0x7fd3f80048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:36.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.604+0000 7fd3ff7fe640 1 -- 172.21.15.67:0/3444647363 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f804f070 con 0x7fd4000956b0 2024-07-31T15:01:36.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.604+0000 7fd40759e640 1 -- 172.21.15.67:0/3444647363 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd400147bd0 con 0x7fd4000956b0 2024-07-31T15:01:36.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.604+0000 7fd40759e640 1 -- 172.21.15.67:0/3444647363 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd400148100 con 0x7fd4000956b0 2024-07-31T15:01:36.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.605+0000 7fd3ff7fe640 1 -- 172.21.15.67:0/3444647363 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3f80054d0 con 0x7fd4000956b0 2024-07-31T15:01:36.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.605+0000 7fd3ff7fe640 1 -- 172.21.15.67:0/3444647363 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8046030 con 0x7fd4000956b0 2024-07-31T15:01:36.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.607+0000 7fd40759e640 1 -- 172.21.15.67:0/3444647363 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd400099be0 con 0x7fd4000956b0 2024-07-31T15:01:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.614+0000 7fd3ff7fe640 1 -- 172.21.15.67:0/3444647363 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd3f8052070 con 0x7fd4000956b0 2024-07-31T15:01:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.615+0000 7fd3ff7fe640 1 --2- 172.21.15.67:0/3444647363 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd3f40b3e70 0x7fd3f40b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.616+0000 7fd3ff7fe640 1 -- 172.21.15.67:0/3444647363 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7fd3f8116030 con 0x7fd4000956b0 2024-07-31T15:01:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.617+0000 7fd40659c640 1 --2- 172.21.15.67:0/3444647363 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd3f40b3e70 0x7fd3f40b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:36.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.618+0000 7fd40659c640 1 --2- 172.21.15.67:0/3444647363 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd3f40b3e70 0x7fd3f40b6380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd4000a9010 tx=0x7fd3f0008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:36.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:36.620+0000 7fd3ff7fe640 1 -- 172.21.15.67:0/3444647363 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd3f80dcf10 con 0x7fd4000956b0 2024-07-31T15:01:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:01:37.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:37.086+0000 7fd40759e640 1 -- 172.21.15.67:0/3444647363 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4000a68a0 con 0x7fd4000956b0 2024-07-31T15:01:37.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:37.087+0000 7fd3ff7fe640 1 -- 172.21.15.67:0/3444647363 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11969 (secure 0 0 0) 0x7fd4000a68a0 con 0x7fd4000956b0 2024-07-31T15:01:37.094 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:01:37.094 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":58,"btime":"2024-07-31T15:01:15:058622+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:01:14.559063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:01:37.094 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 58 2024-07-31T15:01:37.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:37.092+0000 7fd40759e640 1 -- 172.21.15.67:0/3444647363 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd3f40b3e70 msgr2=0x7fd3f40b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:37.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:37.092+0000 7fd40759e640 1 --2- 172.21.15.67:0/3444647363 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd3f40b3e70 0x7fd3f40b6380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd4000a9010 tx=0x7fd3f0008040 comp rx=0 tx=0).stop 2024-07-31T15:01:37.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:37.092+0000 7fd40759e640 1 -- 172.21.15.67:0/3444647363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4000956b0 msgr2=0x7fd4000a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:37.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:37.092+0000 7fd40759e640 1 --2- 172.21.15.67:0/3444647363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4000956b0 0x7fd4000a55a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd3f803ba40 tx=0x7fd3f80048f0 comp rx=0 tx=0).stop 2024-07-31T15:01:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:37.095+0000 7fd40759e640 1 -- 172.21.15.67:0/3444647363 shutdown_connections 2024-07-31T15:01:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:37.095+0000 7fd40759e640 1 --2- 172.21.15.67:0/3444647363 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd3f40b3e70 0x7fd3f40b6380 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:37.095+0000 7fd40759e640 1 --2- 172.21.15.67:0/3444647363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd40009a1a0 0x7fd4000a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:37.095+0000 7fd40759e640 1 --2- 172.21.15.67:0/3444647363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4000956b0 0x7fd4000a55a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:37.095+0000 7fd40759e640 1 --2- 172.21.15.67:0/3444647363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd400094d60 0x7fd4000a92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:37.095+0000 7fd40759e640 1 -- 172.21.15.67:0/3444647363 >> 172.21.15.67:0/3444647363 conn(0x7fd40000cd70 msgr2=0x7fd400097bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:37.095+0000 7fd40759e640 1 -- 172.21.15.67:0/3444647363 shutdown_connections 2024-07-31T15:01:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:37.095+0000 7fd40759e640 1 -- 172.21.15.67:0/3444647363 wait complete. 2024-07-31T15:01:37.103 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/generated/asm-offsets.h 2024-07-31T15:01:37.112 INFO:tasks.workunit.client.0.smithi067.stdout: CALL scripts/checksyscalls.sh 2024-07-31T15:01:37.263 INFO:tasks.workunit.client.0.smithi067.stdout: LDS scripts/module.lds 2024-07-31T15:01:37.443 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC usr/gen_init_cpio 2024-07-31T15:01:37.444 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/main.o 2024-07-31T15:01:37.452 INFO:tasks.workunit.client.0.smithi067.stdout: CC certs/system_keyring.o 2024-07-31T15:01:37.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:37 smithi067 ceph-mon[32750]: pgmap v726: 129 pgs: 28 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 88 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 267 KiB/s wr, 114 op/s 2024-07-31T15:01:37.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:37 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3444647363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:01:37.486 INFO:tasks.workunit.client.0.smithi067.stdout: UPD init/utsversion-tmp.h 2024-07-31T15:01:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:37 smithi077 ceph-mon[37327]: pgmap v726: 129 pgs: 28 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 88 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 267 KiB/s wr, 114 op/s 2024-07-31T15:01:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:37 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3444647363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:01:37.488 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/filemap.o 2024-07-31T15:01:37.570 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/dnotify/dnotify.o 2024-07-31T15:01:37.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:37 smithi073 ceph-mon[36714]: pgmap v726: 129 pgs: 28 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 88 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 267 KiB/s wr, 114 op/s 2024-07-31T15:01:37.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:37 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3444647363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:01:37.645 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/amd/core.o 2024-07-31T15:01:37.764 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vma.o 2024-07-31T15:01:37.955 INFO:tasks.workunit.client.0.smithi067.stdout: GEN usr/initramfs_data.cpio 2024-07-31T15:01:38.066 INFO:tasks.workunit.client.0.smithi067.stdout: COPY usr/initramfs_inc_data 2024-07-31T15:01:38.082 INFO:tasks.workunit.client.0.smithi067.stdout: AS usr/initramfs_data.o 2024-07-31T15:01:38.178 INFO:tasks.workunit.client.0.smithi067.stdout: AR usr/built-in.a 2024-07-31T15:01:38.179 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/sched/core.o 2024-07-31T15:01:38.340 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/amd/lbr.o 2024-07-31T15:01:38.377 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/notify/dnotify/built-in.a 2024-07-31T15:01:38.399 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC certs/extract-cert 2024-07-31T15:01:38.557 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/inotify/inotify_fsnotify.o 2024-07-31T15:01:38.673 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T15:01:38.755 INFO:tasks.workunit.client.0.smithi067.stdout: CERT certs/x509_certificate_list 2024-07-31T15:01:38.775 INFO:tasks.workunit.client.0.smithi067.stdout: CERT certs/signing_key.x509 2024-07-31T15:01:38.798 INFO:tasks.workunit.client.0.smithi067.stdout: AS certs/system_certificates.o 2024-07-31T15:01:38.887 INFO:tasks.workunit.client.0.smithi067.stdout: AR certs/built-in.a 2024-07-31T15:01:39.076 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/inotify/inotify_user.o 2024-07-31T15:01:39.210 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/notify/fanotify/built-in.a 2024-07-31T15:01:39.287 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/extable.o 2024-07-31T15:01:39.323 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/amd/ibs.o 2024-07-31T15:01:39.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:39 smithi077 ceph-mon[37327]: pgmap v727: 129 pgs: 28 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 89 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 269 KiB/s wr, 119 op/s 2024-07-31T15:01:39.492 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vdso32-setup.o 2024-07-31T15:01:39.501 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/amd/uncore.o 2024-07-31T15:01:39.509 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:01:39.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:39 smithi067 ceph-mon[32750]: pgmap v727: 129 pgs: 28 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 89 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 269 KiB/s wr, 119 op/s 2024-07-31T15:01:39.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:39 smithi073 ceph-mon[36714]: pgmap v727: 129 pgs: 28 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 89 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 269 KiB/s wr, 119 op/s 2024-07-31T15:01:39.846 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/compat.o 2024-07-31T15:01:40.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.326+0000 7fbbdf899640 1 -- 172.21.15.67:0/2258863860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbd8107df0 msgr2=0x7fbbd8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:40.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.326+0000 7fbbdf899640 1 --2- 172.21.15.67:0/2258863860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbd8107df0 0x7fbbd8108200 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009ca0 tx=0x7fbbd002e9c0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.326+0000 7fbbdf899640 1 -- 172.21.15.67:0/2258863860 shutdown_connections 2024-07-31T15:01:40.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.326+0000 7fbbdf899640 1 --2- 172.21.15.67:0/2258863860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd8108740 0x7fbbd810ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.326+0000 7fbbdf899640 1 --2- 172.21.15.67:0/2258863860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbd8107df0 0x7fbbd8108200 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.326+0000 7fbbdf899640 1 --2- 172.21.15.67:0/2258863860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbd81024b0 0x7fbbd81028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.326+0000 7fbbdf899640 1 -- 172.21.15.67:0/2258863860 >> 172.21.15.67:0/2258863860 conn(0x7fbbd8079190 msgr2=0x7fbbd80795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:40.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.327+0000 7fbbdf899640 1 -- 172.21.15.67:0/2258863860 shutdown_connections 2024-07-31T15:01:40.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.327+0000 7fbbdf899640 1 -- 172.21.15.67:0/2258863860 wait complete. 2024-07-31T15:01:40.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.328+0000 7fbbdf899640 1 Processor -- start 2024-07-31T15:01:40.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.328+0000 7fbbdf899640 1 -- start start 2024-07-31T15:01:40.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.328+0000 7fbbdf899640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd81024b0 0x7fbbd8074440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.329+0000 7fbbdf899640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbd8107df0 0x7fbbd80749c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.329+0000 7fbbdf899640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbd8108740 0x7fbbd806e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.329+0000 7fbbdf899640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd806cce0 con 0x7fbbd81024b0 2024-07-31T15:01:40.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.329+0000 7fbbdf899640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd806cb60 con 0x7fbbd8108740 2024-07-31T15:01:40.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.329+0000 7fbbdf899640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd806ce60 con 0x7fbbd8107df0 2024-07-31T15:01:40.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.330+0000 7fbbdde36640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbd8108740 0x7fbbd806e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.330+0000 7fbbdde36640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbd8108740 0x7fbbd806e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:34596/0 (socket says 172.21.15.67:34596) 2024-07-31T15:01:40.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.330+0000 7fbbdde36640 1 -- 172.21.15.67:0/421732493 learned_addr learned my addr 172.21.15.67:0/421732493 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:40.334 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/notify/inotify/built-in.a 2024-07-31T15:01:40.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.330+0000 7fbbdd635640 1 --2- 172.21.15.67:0/421732493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd81024b0 0x7fbbd8074440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.331+0000 7fbbdd635640 1 -- 172.21.15.67:0/421732493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbd8107df0 msgr2=0x7fbbd80749c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:40.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.331+0000 7fbbdd635640 1 --2- 172.21.15.67:0/421732493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbd8107df0 0x7fbbd80749c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.331+0000 7fbbdd635640 1 -- 172.21.15.67:0/421732493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbd8108740 msgr2=0x7fbbd806e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:40.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.331+0000 7fbbdd635640 1 --2- 172.21.15.67:0/421732493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbd8108740 0x7fbbd806e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.331+0000 7fbbdd635640 1 -- 172.21.15.67:0/421732493 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd806f030 con 0x7fbbd81024b0 2024-07-31T15:01:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.331+0000 7fbbdd635640 1 --2- 172.21.15.67:0/421732493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd81024b0 0x7fbbd8074440 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400c970 tx=0x7fbbd400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.332+0000 7fbbce7fc640 1 -- 172.21.15.67:0/421732493 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4007bb0 con 0x7fbbd81024b0 2024-07-31T15:01:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.332+0000 7fbbce7fc640 1 -- 172.21.15.67:0/421732493 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd4007d50 con 0x7fbbd81024b0 2024-07-31T15:01:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.332+0000 7fbbce7fc640 1 -- 172.21.15.67:0/421732493 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400f450 con 0x7fbbd81024b0 2024-07-31T15:01:40.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.336+0000 7fbbdf899640 1 -- 172.21.15.67:0/421732493 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd806f320 con 0x7fbbd81024b0 2024-07-31T15:01:40.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.336+0000 7fbbdf899640 1 -- 172.21.15.67:0/421732493 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd81acd80 con 0x7fbbd81024b0 2024-07-31T15:01:40.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.340+0000 7fbbe5dea740 1 Processor -- start 2024-07-31T15:01:40.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.340+0000 7fbbe5dea740 1 -- start start 2024-07-31T15:01:40.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.340+0000 7fbbe5dea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39e1840 0x55e4b3d87840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.340+0000 7fbbe5dea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39b8980 0x55e4b3d87d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.340+0000 7fbbe5dea740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39c0b60 0x55e4b3d882c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.341+0000 7fbbe5dea740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e4b3cfc780 con 0x55e4b39e1840 2024-07-31T15:01:40.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.341+0000 7fbbe5dea740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e4b3cebc80 con 0x55e4b39c0b60 2024-07-31T15:01:40.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.341+0000 7fbbe5dea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e4b3cefcc0 con 0x55e4b39b8980 2024-07-31T15:01:40.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.341+0000 7fbbdd635640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39b8980 0x55e4b3d87d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.341+0000 7fbbdd635640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39b8980 0x55e4b3d87d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:40738/0 (socket says 172.21.15.67:40738) 2024-07-31T15:01:40.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.341+0000 7fbbdd635640 1 -- 172.21.15.67:0/2396673156 learned_addr learned my addr 172.21.15.67:0/2396673156 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:40.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.341+0000 7fbbdce34640 1 --2- 172.21.15.67:0/2396673156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39c0b60 0x55e4b3d882c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.342+0000 7fbbdde36640 1 --2- 172.21.15.67:0/2396673156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39e1840 0x55e4b3d87840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.343+0000 7fbbdd635640 1 -- 172.21.15.67:0/2396673156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39c0b60 msgr2=0x55e4b3d882c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:40.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.343+0000 7fbbdd635640 1 --2- 172.21.15.67:0/2396673156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39c0b60 0x55e4b3d882c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.343+0000 7fbbdd635640 1 -- 172.21.15.67:0/2396673156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39e1840 msgr2=0x55e4b3d87840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:40.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.343+0000 7fbbdd635640 1 --2- 172.21.15.67:0/2396673156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39e1840 0x55e4b3d87840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.343+0000 7fbbdd635640 1 -- 172.21.15.67:0/2396673156 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4b3c81080 con 0x55e4b39b8980 2024-07-31T15:01:40.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.343+0000 7fbbdd635640 1 --2- 172.21.15.67:0/2396673156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39b8980 0x55e4b3d87d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4005d60 tx=0x7fbbd4005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c11a877465047e6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.347+0000 7fbbc37fe640 1 -- 172.21.15.67:0/2396673156 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4032070 con 0x55e4b39b8980 2024-07-31T15:01:40.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.348+0000 7fbbce7fc640 1 -- 172.21.15.67:0/421732493 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbbd402da40 con 0x7fbbd81024b0 2024-07-31T15:01:40.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.348+0000 7fbbc37fe640 1 -- 172.21.15.67:0/2396673156 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd4005610 con 0x55e4b39b8980 2024-07-31T15:01:40.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.349+0000 7fbbe5dea740 1 -- 172.21.15.67:0/2396673156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39b8980 msgr2=0x55e4b3d87d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:40.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.349+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/2396673156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39b8980 0x55e4b3d87d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4005d60 tx=0x7fbbd4005250 comp rx=0 tx=0).stop 2024-07-31T15:01:40.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.349+0000 7fbbce7fc640 1 --2- 172.21.15.67:0/421732493 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbbc40b3f50 0x7fbbc40b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.349+0000 7fbbe5dea740 1 -- 172.21.15.67:0/2396673156 shutdown_connections 2024-07-31T15:01:40.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.349+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/2396673156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39c0b60 0x55e4b3d882c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.349+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/2396673156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39b8980 0x55e4b3d87d80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.349+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/2396673156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39e1840 0x55e4b3d87840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.349+0000 7fbbe5dea740 1 -- 172.21.15.67:0/2396673156 >> 172.21.15.67:0/2396673156 conn(0x55e4b39d5990 msgr2=0x55e4b3d85730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:40.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.349+0000 7fbbce7fc640 1 -- 172.21.15.67:0/421732493 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7fbbd40e8550 con 0x7fbbd81024b0 2024-07-31T15:01:40.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.350+0000 7fbbe5dea740 1 -- 172.21.15.67:0/2396673156 shutdown_connections 2024-07-31T15:01:40.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.350+0000 7fbbe5dea740 1 -- 172.21.15.67:0/2396673156 wait complete. 2024-07-31T15:01:40.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.350+0000 7fbbe5dea740 1 -- start start 2024-07-31T15:01:40.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.350+0000 7fbbe5dea740 1 Processor -- start 2024-07-31T15:01:40.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.351+0000 7fbbe5dea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39c0b60 0x55e4b3d95450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.352+0000 7fbbe5dea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39e1840 0x55e4b3d92920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.352+0000 7fbbe5dea740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39a01c0 0x55e4b3d92e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.352+0000 7fbbe5dea740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e4b3b4f7a0 con 0x55e4b39c0b60 2024-07-31T15:01:40.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.352+0000 7fbbe5dea740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e4b3c8e2f0 con 0x55e4b39a01c0 2024-07-31T15:01:40.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.352+0000 7fbbe5dea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e4b3d95fc0 con 0x55e4b39e1840 2024-07-31T15:01:40.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.352+0000 7fbbe5dea740 10 client.0 _ll_register_callbacks cb 0x55e4b3c98110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.353+0000 7fbbdd635640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39c0b60 0x55e4b3d95450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.353+0000 7fbbdd635640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39c0b60 0x55e4b3d95450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37936/0 (socket says 172.21.15.67:37936) 2024-07-31T15:01:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.353+0000 7fbbdd635640 1 -- 172.21.15.67:0/409577867 learned_addr learned my addr 172.21.15.67:0/409577867 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.354+0000 7fbbdd635640 1 -- 172.21.15.67:0/409577867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39e1840 msgr2=0x55e4b3d92920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:01:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.354+0000 7fbbdd635640 1 --2- 172.21.15.67:0/409577867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39e1840 0x55e4b3d92920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.354+0000 7fbbdd635640 1 -- 172.21.15.67:0/409577867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39a01c0 msgr2=0x55e4b3d92e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:01:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.354+0000 7fbbdd635640 1 --2- 172.21.15.67:0/409577867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39a01c0 0x55e4b3d92e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.354+0000 7fbbdd635640 1 -- 172.21.15.67:0/409577867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4b3c92480 con 0x55e4b39c0b60 2024-07-31T15:01:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.354+0000 7fbbdde36640 1 --2- 172.21.15.67:0/421732493 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbbc40b3f50 0x7fbbc40b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.354+0000 7fbbdd635640 1 --2- 172.21.15.67:0/409577867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39c0b60 0x55e4b3d95450 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40ac0e0 tx=0x7fbbd40aed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b4174555f90fac server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.355+0000 7fbbc0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:01:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.355+0000 7fbbc0ff9640 1 -- 172.21.15.67:0/409577867 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40ae980 con 0x55e4b39c0b60 2024-07-31T15:01:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.355+0000 7fbbc0ff9640 1 -- 172.21.15.67:0/409577867 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7fbbd40ab460 con 0x55e4b39c0b60 2024-07-31T15:01:40.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.355+0000 7fbbe5dea740 1 -- 172.21.15.67:0/409577867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e4b3cfe6d0 con 0x55e4b39c0b60 2024-07-31T15:01:40.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.356+0000 7fbbc0ff9640 1 -- 172.21.15.67:0/409577867 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fbbd40aee70 con 0x55e4b39c0b60 2024-07-31T15:01:40.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.357+0000 7fbbdde36640 1 --2- 172.21.15.67:0/421732493 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbbc40b3f50 0x7fbbc40b6460 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55e4b3d95bc0 tx=0x7fbbc8008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.362+0000 7fbbe5dea740 10 client.17790 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.362+0000 7fbbe5dea740 1 -- 172.21.15.67:0/409577867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55e4b3a5bb90 con 0x55e4b39c0b60 2024-07-31T15:01:40.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.364+0000 7fbbc0ff9640 1 -- 172.21.15.67:0/409577867 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd40b2030 con 0x55e4b39c0b60 2024-07-31T15:01:40.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.365+0000 7fbbc0ff9640 1 -- 172.21.15.67:0/409577867 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7fbbd40af0d0 con 0x55e4b39c0b60 2024-07-31T15:01:40.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.365+0000 7fbbe5dea740 10 client.17790 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:40.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.365+0000 7fbbe5dea740 10 client.17790 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:01:40.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.365+0000 7fbbe5dea740 20 client.17790 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:40.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.365+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/409577867 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e4b3975570 0x55e4b3da4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.365+0000 7fbbe5dea740 4 client.17790 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:01:40.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.365+0000 7fbbe5dea740 1 -- 172.21.15.67:0/409577867 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4b3cfe6d0 con 0x55e4b3975570 2024-07-31T15:01:40.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.366+0000 7fbbdce34640 1 --2- 172.21.15.67:0/409577867 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e4b3975570 0x55e4b3da4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.368+0000 7fbbdce34640 1 --2- 172.21.15.67:0/409577867 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e4b3975570 0x55e4b3da4920 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d1351c68a85cb9 server_cookie=dc9c23f4ab8c369a in_seq=0 out_seq=0 2024-07-31T15:01:40.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.368+0000 7fbbc0ff9640 10 client.17790 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:40.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.370+0000 7fbbc0ff9640 1 -- 172.21.15.67:0/409577867 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4b3cfe6d0 con 0x55e4b3975570 2024-07-31T15:01:40.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.370+0000 7fbbc0ff9640 10 client.17790 handle_command_reply: tid=0 2024-07-31T15:01:40.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.370+0000 7fbbe5dea740 1 client.17790 shutdown 2024-07-31T15:01:40.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.373+0000 7fbbe5dea740 1 -- 172.21.15.67:0/409577867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39c0b60 msgr2=0x55e4b3d95450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:40.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.373+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/409577867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39c0b60 0x55e4b3d95450 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40ac0e0 tx=0x7fbbd40aed30 comp rx=0 tx=0).stop 2024-07-31T15:01:40.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.375+0000 7fbbe5dea740 1 -- 172.21.15.67:0/409577867 shutdown_connections 2024-07-31T15:01:40.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.375+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/409577867 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e4b3975570 0x55e4b3da4920 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.375+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/409577867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39a01c0 0x55e4b3d92e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.375+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/409577867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39e1840 0x55e4b3d92920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.375+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/409577867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39c0b60 0x55e4b3d95450 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.375+0000 7fbbe5dea740 1 -- 172.21.15.67:0/409577867 >> 172.21.15.67:0/409577867 conn(0x55e4b39d5990 msgr2=0x55e4b3d8aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:40.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.375+0000 7fbbc0ff9640 0 client.17790 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:40.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.379+0000 7fbbe5dea740 1 -- 172.21.15.67:0/409577867 shutdown_connections 2024-07-31T15:01:40.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.380+0000 7fbbe5dea740 1 -- 172.21.15.67:0/409577867 wait complete. 2024-07-31T15:01:40.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.380+0000 7fbbe5dea740 20 client.17790 trim_cache size 0 max 16384 2024-07-31T15:01:40.394 INFO:tasks.workunit.client.0.smithi067.stdout: LDS arch/x86/entry/vdso/vdso.lds 2024-07-31T15:01:40.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.429+0000 7fbbe5dea740 1 Processor -- start 2024-07-31T15:01:40.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.439+0000 7fbbe5dea740 1 -- start start 2024-07-31T15:01:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.439+0000 7fc414de2640 1 -- 172.21.15.67:0/826998216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc410111650 msgr2=0x7fc410109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.439+0000 7fc414de2640 1 --2- 172.21.15.67:0/826998216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc410111650 0x7fc410109df0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc400009900 tx=0x7fc40002a440 comp rx=0 tx=0).stop 2024-07-31T15:01:40.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.439+0000 7fbbe5dea740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39a01c0 0x55e4b3d94800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.440+0000 7fbbe5dea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39c0b60 0x55e4b3d9e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.440+0000 7fbbdd635640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39a01c0 0x55e4b3d94800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.440+0000 7fbbdd635640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39a01c0 0x55e4b3d94800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:34632/0 (socket says 172.21.15.67:34632) 2024-07-31T15:01:40.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.440+0000 7fbbdd635640 1 -- 172.21.15.67:0/2964521411 learned_addr learned my addr 172.21.15.67:0/2964521411 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:40.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.440+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/2964521411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39e1840 0x55e4b3d9ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.440+0000 7fbbe5dea740 1 -- 172.21.15.67:0/2964521411 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd81b6f30 con 0x55e4b39c0b60 2024-07-31T15:01:40.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.440+0000 7fbbe5dea740 1 -- 172.21.15.67:0/2964521411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd81b8b30 con 0x55e4b39a01c0 2024-07-31T15:01:40.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.440+0000 7fbbdde36640 1 --2- 172.21.15.67:0/2964521411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39e1840 0x55e4b3d9ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.440+0000 7fbbe5dea740 1 -- 172.21.15.67:0/2964521411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd81b7600 con 0x55e4b39e1840 2024-07-31T15:01:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.441+0000 7fbbdd635640 1 -- 172.21.15.67:0/2964521411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39e1840 msgr2=0x55e4b3d9ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.441+0000 7fbbdd635640 1 --2- 172.21.15.67:0/2964521411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39e1840 0x55e4b3d9ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.441+0000 7fbbdd635640 1 -- 172.21.15.67:0/2964521411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39c0b60 msgr2=0x55e4b3d9e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.441+0000 7fbbdd635640 1 --2- 172.21.15.67:0/2964521411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39c0b60 0x55e4b3d9e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.441+0000 7fbbdd635640 1 -- 172.21.15.67:0/2964521411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4b3d95f00 con 0x55e4b39a01c0 2024-07-31T15:01:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.446+0000 7fc414de2640 1 -- 172.21.15.67:0/826998216 shutdown_connections 2024-07-31T15:01:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.446+0000 7fc414de2640 1 --2- 172.21.15.67:0/826998216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc41011c5d0 0x7fc41011e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.446+0000 7fc414de2640 1 --2- 172.21.15.67:0/826998216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc41010a330 0x7fc41010a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.446+0000 7fc414de2640 1 --2- 172.21.15.67:0/826998216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc410111650 0x7fc410109df0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.446+0000 7fc414de2640 1 -- 172.21.15.67:0/826998216 >> 172.21.15.67:0/826998216 conn(0x7fc41006e320 msgr2=0x7fc41006e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.447+0000 7fc414de2640 1 -- 172.21.15.67:0/826998216 shutdown_connections 2024-07-31T15:01:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.447+0000 7fc414de2640 1 -- 172.21.15.67:0/826998216 wait complete. 2024-07-31T15:01:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.448+0000 7fc414de2640 1 Processor -- start 2024-07-31T15:01:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.448+0000 7fc414de2640 1 -- start start 2024-07-31T15:01:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.449+0000 7fc414de2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc41010a330 0x7fc4101a8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.449+0000 7fc414de2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc410111650 0x7fc4101a9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.449+0000 7fc414de2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc41011c5d0 0x7fc4101ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.449+0000 7fc414de2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc410122110 con 0x7fc41011c5d0 2024-07-31T15:01:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.449+0000 7fc414de2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc410121f90 con 0x7fc410111650 2024-07-31T15:01:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.449+0000 7fc414de2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc410122290 con 0x7fc41010a330 2024-07-31T15:01:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.449+0000 7fc40f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc41010a330 0x7fc4101a8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.449+0000 7fc40f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc41010a330 0x7fc4101a8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:40768/0 (socket says 172.21.15.67:40768) 2024-07-31T15:01:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.450+0000 7fc40f7fe640 1 -- 172.21.15.67:0/2408011756 learned_addr learned my addr 172.21.15.67:0/2408011756 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.450+0000 7fc40effd640 1 --2- 172.21.15.67:0/2408011756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc410111650 0x7fc4101a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.451+0000 7fc40ffff640 1 --2- 172.21.15.67:0/2408011756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc41011c5d0 0x7fc4101ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.454+0000 7fc40f7fe640 1 -- 172.21.15.67:0/2408011756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc410111650 msgr2=0x7fc4101a9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.454+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/2408011756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc410111650 0x7fc4101a9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.454+0000 7fc40f7fe640 1 -- 172.21.15.67:0/2408011756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc41011c5d0 msgr2=0x7fc4101ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.454+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/2408011756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc41011c5d0 0x7fc4101ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.454+0000 7fc40f7fe640 1 -- 172.21.15.67:0/2408011756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4101aeaa0 con 0x7fc41010a330 2024-07-31T15:01:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.454+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/2408011756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc41010a330 0x7fc4101a8b70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc4000058c0 tx=0x7fc4000054f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.457+0000 7fc40cff9640 1 -- 172.21.15.67:0/2408011756 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400041070 con 0x7fc41010a330 2024-07-31T15:01:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.457+0000 7fc40cff9640 1 -- 172.21.15.67:0/2408011756 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4000041b0 con 0x7fc41010a330 2024-07-31T15:01:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.457+0000 7fc414de2640 1 -- 172.21.15.67:0/2408011756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4101aed30 con 0x7fc41010a330 2024-07-31T15:01:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.457+0000 7fc414de2640 1 -- 172.21.15.67:0/2408011756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc410111db0 con 0x7fc41010a330 2024-07-31T15:01:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.458+0000 7fc40cff9640 1 -- 172.21.15.67:0/2408011756 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40003dda0 con 0x7fc41010a330 2024-07-31T15:01:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.448+0000 7fbbdd635640 1 --2- 172.21.15.67:0/2964521411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39a01c0 0x55e4b3d94800 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40aaef0 tx=0x7fbbd4004910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6db69c8a290f636 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.451+0000 7fbbc2ffd640 1 -- 172.21.15.67:0/2964521411 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40b7bb0 con 0x55e4b39a01c0 2024-07-31T15:01:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.451+0000 7fbbc2ffd640 1 -- 172.21.15.67:0/2964521411 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd40b2070 con 0x55e4b39a01c0 2024-07-31T15:01:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.451+0000 7fbbc2ffd640 1 -- 172.21.15.67:0/2964521411 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40af0d0 con 0x55e4b39a01c0 2024-07-31T15:01:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.453+0000 7fbbe5dea740 1 -- 172.21.15.67:0/2964521411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39a01c0 msgr2=0x55e4b3d94800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.453+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/2964521411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39a01c0 0x55e4b3d94800 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40aaef0 tx=0x7fbbd4004910 comp rx=0 tx=0).stop 2024-07-31T15:01:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.456+0000 7fbbe5dea740 1 -- 172.21.15.67:0/2964521411 shutdown_connections 2024-07-31T15:01:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.456+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/2964521411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39e1840 0x55e4b3d9ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.456+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/2964521411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39c0b60 0x55e4b3d9e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.456+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/2964521411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39a01c0 0x55e4b3d94800 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.456+0000 7fbbe5dea740 1 -- 172.21.15.67:0/2964521411 >> 172.21.15.67:0/2964521411 conn(0x55e4b39d5990 msgr2=0x55e4b3d9e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.456+0000 7fbbe5dea740 1 -- 172.21.15.67:0/2964521411 shutdown_connections 2024-07-31T15:01:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.456+0000 7fbbe5dea740 1 -- 172.21.15.67:0/2964521411 wait complete. 2024-07-31T15:01:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.457+0000 7fbbe5dea740 1 -- start start 2024-07-31T15:01:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.457+0000 7fbbe5dea740 1 Processor -- start 2024-07-31T15:01:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.458+0000 7fbbe5dea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39a01c0 0x55e4b3da7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.458+0000 7fbbe5dea740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39c0b60 0x55e4b3da7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.459+0000 7fbbdd635640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39a01c0 0x55e4b3da7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.459+0000 7fbbdd635640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39a01c0 0x55e4b3da7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:40780/0 (socket says 172.21.15.67:40780) 2024-07-31T15:01:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.459+0000 7fbbdd635640 1 -- 172.21.15.67:0/691836377 learned_addr learned my addr 172.21.15.67:0/691836377 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.459+0000 7fbbdce34640 1 --2- 172.21.15.67:0/691836377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39c0b60 0x55e4b3da7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.459+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/691836377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39e1840 0x55e4b3e053a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.459+0000 7fbbe5dea740 1 -- 172.21.15.67:0/691836377 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd81b8460 con 0x55e4b39e1840 2024-07-31T15:01:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.459+0000 7fbbe5dea740 1 -- 172.21.15.67:0/691836377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd81b7cd0 con 0x55e4b39c0b60 2024-07-31T15:01:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.459+0000 7fbbe5dea740 1 -- 172.21.15.67:0/691836377 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd81b4fc0 con 0x55e4b39a01c0 2024-07-31T15:01:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.460+0000 7fbbe5dea740 10 client.0 _ll_register_callbacks cb 0x7fbbd40aada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.460+0000 7fbbdd635640 1 -- 172.21.15.67:0/691836377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39c0b60 msgr2=0x55e4b3da7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.460+0000 7fbbdd635640 1 --2- 172.21.15.67:0/691836377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39c0b60 0x55e4b3da7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.460+0000 7fbbdd635640 1 -- 172.21.15.67:0/691836377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39e1840 msgr2=0x55e4b3e053a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.460+0000 7fbbdd635640 1 --2- 172.21.15.67:0/691836377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39e1840 0x55e4b3e053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.460+0000 7fbbdd635640 1 -- 172.21.15.67:0/691836377 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4b3e05a10 con 0x55e4b39a01c0 2024-07-31T15:01:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.460+0000 7fbbdd635640 1 --2- 172.21.15.67:0/691836377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39a01c0 0x55e4b3da7790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4005840 tx=0x7fbbd40e5d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f45796b5f3747f61 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.460+0000 7fbbc0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:01:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.460+0000 7fbbc0ff9640 1 -- 172.21.15.67:0/691836377 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40b7bb0 con 0x55e4b39a01c0 2024-07-31T15:01:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.461+0000 7fc41d00e740 1 Processor -- start 2024-07-31T15:01:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.461+0000 7fc41d00e740 1 -- start start 2024-07-31T15:01:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.462+0000 7fc41d00e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fd401b0 0x564c600e2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.462+0000 7fc41d00e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd172f0 0x564c600e3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.462+0000 7fc41d00e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 0x564c600e3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.462+0000 7fc41d00e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564c600583b0 con 0x564c5fd401b0 2024-07-31T15:01:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.462+0000 7fc41d00e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564c600478b0 con 0x564c5fd1f4d0 2024-07-31T15:01:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.462+0000 7fc41d00e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564c6004b8f0 con 0x564c5fd172f0 2024-07-31T15:01:40.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.461+0000 7fbbc0ff9640 1 -- 172.21.15.67:0/691836377 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7fbbd402a040 con 0x55e4b39a01c0 2024-07-31T15:01:40.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.461+0000 7fbbe5dea740 1 -- 172.21.15.67:0/691836377 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd40b2070 con 0x55e4b39a01c0 2024-07-31T15:01:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.464+0000 7fc40ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fd401b0 0x564c600e2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.464+0000 7fc40ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fd401b0 0x564c600e2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37976/0 (socket says 172.21.15.67:37976) 2024-07-31T15:01:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.464+0000 7fc40ffff640 1 -- 172.21.15.67:0/767961472 learned_addr learned my addr 172.21.15.67:0/767961472 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.464+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/767961472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd172f0 0x564c600e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.465+0000 7fc40cff9640 1 -- 172.21.15.67:0/2408011756 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc400044070 con 0x7fc41010a330 2024-07-31T15:01:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.466+0000 7fc40f7fe640 1 -- 172.21.15.67:0/767961472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 msgr2=0x564c600e3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.466+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/767961472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 0x564c600e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.466+0000 7fc40f7fe640 1 -- 172.21.15.67:0/767961472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fd401b0 msgr2=0x564c600e2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.466+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/767961472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fd401b0 0x564c600e2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.466+0000 7fc40f7fe640 1 -- 172.21.15.67:0/767961472 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c5ffee9a0 con 0x564c5fd172f0 2024-07-31T15:01:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.467+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/767961472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd172f0 0x564c600e3420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc400102380 tx=0x7fc40010d2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1428fe12ce1733c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.467+0000 7fc40cff9640 1 --2- 172.21.15.67:0/2408011756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3f80b3f50 0x7fc3f80b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.467+0000 7fc40cff9640 1 -- 172.21.15.67:0/2408011756 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7fc400105090 con 0x7fc41010a330 2024-07-31T15:01:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.468+0000 7fc3f5ffb640 1 -- 172.21.15.67:0/767961472 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400117470 con 0x564c5fd172f0 2024-07-31T15:01:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.468+0000 7fc40ffff640 1 --2- 172.21.15.67:0/2408011756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3f80b3f50 0x7fc3f80b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.468+0000 7fc3f5ffb640 1 -- 172.21.15.67:0/767961472 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc400112070 con 0x564c5fd172f0 2024-07-31T15:01:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.468+0000 7fc3f5ffb640 1 -- 172.21.15.67:0/767961472 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4001155d0 con 0x564c5fd172f0 2024-07-31T15:01:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.469+0000 7fc41d00e740 1 -- 172.21.15.67:0/767961472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd172f0 msgr2=0x564c600e3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.469+0000 7fc40ffff640 1 --2- 172.21.15.67:0/2408011756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3f80b3f50 0x7fc3f80b6460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc40400ba90 tx=0x7fc404008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.469+0000 7fc41d00e740 1 --2- 172.21.15.67:0/767961472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd172f0 0x564c600e3420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc400102380 tx=0x7fc40010d2c0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.469+0000 7fc41d00e740 1 -- 172.21.15.67:0/767961472 shutdown_connections 2024-07-31T15:01:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.469+0000 7fc41d00e740 1 --2- 172.21.15.67:0/767961472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 0x564c600e3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.469+0000 7fc41d00e740 1 --2- 172.21.15.67:0/767961472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd172f0 0x564c600e3420 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc400102380 tx=0x7fc40010d2c0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.469+0000 7fc41d00e740 1 --2- 172.21.15.67:0/767961472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fd401b0 0x564c600e2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.469+0000 7fc41d00e740 1 -- 172.21.15.67:0/767961472 >> 172.21.15.67:0/767961472 conn(0x564c5fd34300 msgr2=0x564c600e0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.462+0000 7fbbc0ff9640 1 -- 172.21.15.67:0/691836377 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fbbd40b33c0 con 0x55e4b39a01c0 2024-07-31T15:01:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.462+0000 7fbbe5dea740 10 client.27331 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.462+0000 7fbbe5dea740 1 -- 172.21.15.67:0/691836377 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=58}) -- 0x55e4b3e05a10 con 0x55e4b39a01c0 2024-07-31T15:01:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.470+0000 7fbbc0ff9640 1 -- 172.21.15.67:0/691836377 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7fbbd40aa7e0 con 0x55e4b39a01c0 2024-07-31T15:01:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.470+0000 7fbbe5dea740 10 client.27331 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.470+0000 7fbbe5dea740 10 client.27331 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:01:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.470+0000 7fbbe5dea740 20 client.27331 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.470+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/691836377 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e4b3e0cfa0 0x55e4b3e0f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.470+0000 7fbbe5dea740 4 client.27331 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-07-31T15:01:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.470+0000 7fbbe5dea740 1 -- 172.21.15.67:0/691836377 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x7fbbd40b2070 con 0x55e4b3e0cfa0 2024-07-31T15:01:40.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.472+0000 7fbbdce34640 1 --2- 172.21.15.67:0/691836377 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e4b3e0cfa0 0x55e4b3e0f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.473+0000 7fc41d00e740 1 -- 172.21.15.67:0/767961472 shutdown_connections 2024-07-31T15:01:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.473+0000 7fc41d00e740 1 -- 172.21.15.67:0/767961472 wait complete. 2024-07-31T15:01:40.475 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/vdso/vdso-note.o 2024-07-31T15:01:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.474+0000 7fc41d00e740 1 -- start start 2024-07-31T15:01:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.474+0000 7fc41d00e740 1 Processor -- start 2024-07-31T15:01:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.475+0000 7fc41d00e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 0x564c600eeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.475+0000 7fc41d00e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd401b0 0x564c600ef160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.475+0000 7fc41d00e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fcfed80 0x564c600f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.475+0000 7fc41d00e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564c5ff81ab0 con 0x564c5fcfed80 2024-07-31T15:01:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.475+0000 7fc41d00e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564c5ffcfcf0 con 0x564c5fd1f4d0 2024-07-31T15:01:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.475+0000 7fc41d00e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564c600f1420 con 0x564c5fd401b0 2024-07-31T15:01:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.475+0000 7fc41d00e740 10 client.0 _ll_register_callbacks cb 0x564c600419f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.475+0000 7fc40f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 0x564c600eeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.475+0000 7fc40f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 0x564c600eeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:34672/0 (socket says 172.21.15.67:34672) 2024-07-31T15:01:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.475+0000 7fc40f7fe640 1 -- 172.21.15.67:0/321805062 learned_addr learned my addr 172.21.15.67:0/321805062 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.475+0000 7fc40effd640 1 --2- 172.21.15.67:0/321805062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd401b0 0x564c600ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.475+0000 7fc40f7fe640 1 -- 172.21.15.67:0/321805062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd401b0 msgr2=0x564c600ef160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.475+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/321805062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd401b0 0x564c600ef160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.476+0000 7fc40f7fe640 1 -- 172.21.15.67:0/321805062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fcfed80 msgr2=0x564c600f46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.476+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/321805062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fcfed80 0x564c600f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.476+0000 7fc40f7fe640 1 -- 172.21.15.67:0/321805062 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c5fdab790 con 0x564c5fd1f4d0 2024-07-31T15:01:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.476+0000 7fc40effd640 1 --2- 172.21.15.67:0/321805062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd401b0 0x564c600ef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:01:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.478+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/321805062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 0x564c600eeba0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc4000c78f0 tx=0x7fc400108bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32fb68f6e36373fe server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.478+0000 7fc3f37f6640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:01:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.478+0000 7fc3f37f6640 1 -- 172.21.15.67:0/321805062 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40004b040 con 0x564c5fd1f4d0 2024-07-31T15:01:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.478+0000 7fc3f37f6640 1 -- 172.21.15.67:0/321805062 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7fc4000c7b00 con 0x564c5fd1f4d0 2024-07-31T15:01:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.478+0000 7fc41d00e740 1 -- 172.21.15.67:0/321805062 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc400112070 con 0x564c5fd1f4d0 2024-07-31T15:01:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.479+0000 7fc3f37f6640 1 -- 172.21.15.67:0/321805062 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fc4000c7730 con 0x564c5fd1f4d0 2024-07-31T15:01:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.481+0000 7fc41d00e740 10 client.27359 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.481+0000 7fc41d00e740 1 -- 172.21.15.67:0/321805062 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x564c5fe24620 con 0x564c5fd1f4d0 2024-07-31T15:01:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.483+0000 7fc3f37f6640 1 -- 172.21.15.67:0/321805062 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc400112070 con 0x564c5fd1f4d0 2024-07-31T15:01:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.483+0000 7fc3f37f6640 1 -- 172.21.15.67:0/321805062 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7fc40010dc70 con 0x564c5fd1f4d0 2024-07-31T15:01:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.483+0000 7fc41d00e740 10 client.27359 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.483+0000 7fc41d00e740 10 client.27359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:01:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.483+0000 7fc41d00e740 20 client.27359 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.483+0000 7fc41d00e740 1 --2- 172.21.15.67:0/321805062 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564c5fcd41c0 0x564c601006f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.483+0000 7fc41d00e740 4 client.27359 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:01:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.483+0000 7fc41d00e740 1 -- 172.21.15.67:0/321805062 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c6005a300 con 0x564c5fcd41c0 2024-07-31T15:01:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.484+0000 7fc40effd640 1 --2- 172.21.15.67:0/321805062 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564c5fcd41c0 0x564c601006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.484+0000 7fbbc0ff9640 10 client.27331 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:40.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:40.484+0000 7fbbdce34640 1 --2- 172.21.15.67:0/691836377 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e4b3e0cfa0 0x55e4b3e0f3c0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b7c7286c0949e48 server_cookie=86ee2305dcc44609 in_seq=0 out_seq=0 2024-07-31T15:01:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.485+0000 7fc40effd640 1 --2- 172.21.15.67:0/321805062 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564c5fcd41c0 0x564c601006f0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ec1011ee75efa39 server_cookie=5ce45324308d1bef in_seq=0 out_seq=0 2024-07-31T15:01:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.486+0000 7fc3f37f6640 10 client.27359 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.490+0000 7fc3f37f6640 1 -- 172.21.15.67:0/321805062 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c6005a300 con 0x564c5fcd41c0 2024-07-31T15:01:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.490+0000 7fc3f37f6640 10 client.27359 handle_command_reply: tid=0 2024-07-31T15:01:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.490+0000 7fc41d00e740 1 client.27359 shutdown 2024-07-31T15:01:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.493+0000 7fc41d00e740 1 -- 172.21.15.67:0/321805062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 msgr2=0x564c600eeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.493+0000 7fc41d00e740 1 --2- 172.21.15.67:0/321805062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 0x564c600eeba0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc4000c78f0 tx=0x7fc400108bd0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.493+0000 7fc41d00e740 1 -- 172.21.15.67:0/321805062 shutdown_connections 2024-07-31T15:01:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.493+0000 7fc41d00e740 1 --2- 172.21.15.67:0/321805062 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564c5fcd41c0 0x564c601006f0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.493+0000 7fc41d00e740 1 --2- 172.21.15.67:0/321805062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fcfed80 0x564c600f46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.493+0000 7fc41d00e740 1 --2- 172.21.15.67:0/321805062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd401b0 0x564c600ef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.494+0000 7fc41d00e740 1 --2- 172.21.15.67:0/321805062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 0x564c600eeba0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.494+0000 7fc41d00e740 1 -- 172.21.15.67:0/321805062 >> 172.21.15.67:0/321805062 conn(0x564c5fd34300 msgr2=0x564c600e3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.497+0000 7fc3f37f6640 0 client.27359 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.497+0000 7fc41d00e740 1 -- 172.21.15.67:0/321805062 shutdown_connections 2024-07-31T15:01:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.497+0000 7fc41d00e740 1 -- 172.21.15.67:0/321805062 wait complete. 2024-07-31T15:01:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.497+0000 7fc41d00e740 20 client.27359 trim_cache size 0 max 16384 2024-07-31T15:01:40.512 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/fsnotify.o 2024-07-31T15:01:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.583+0000 7fc41d00e740 1 Processor -- start 2024-07-31T15:01:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.586+0000 7fc41d00e740 1 -- start start 2024-07-31T15:01:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.587+0000 7fc41d00e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fcfed80 0x564c600e4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.587+0000 7fc41d00e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 0x564c600e7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.587+0000 7fc41d00e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd401b0 0x564c600e84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.587+0000 7fc40f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fcfed80 0x564c600e4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.587+0000 7fc40f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fcfed80 0x564c600e4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37988/0 (socket says 172.21.15.67:37988) 2024-07-31T15:01:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.588+0000 7fc40f7fe640 1 -- 172.21.15.67:0/1623615205 learned_addr learned my addr 172.21.15.67:0/1623615205 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.588+0000 7fc41d00e740 1 -- 172.21.15.67:0/1623615205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc010030 con 0x564c5fcfed80 2024-07-31T15:01:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.588+0000 7fc41d00e740 1 -- 172.21.15.67:0/1623615205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc011c30 con 0x564c5fd1f4d0 2024-07-31T15:01:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.588+0000 7fc41d00e740 1 -- 172.21.15.67:0/1623615205 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc010700 con 0x564c5fd401b0 2024-07-31T15:01:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.588+0000 7fc40ffff640 1 --2- 172.21.15.67:0/1623615205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd401b0 0x564c600e84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.590+0000 7fc40f7fe640 1 -- 172.21.15.67:0/1623615205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd401b0 msgr2=0x564c600e84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.590+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/1623615205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd401b0 0x564c600e84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.590+0000 7fc40f7fe640 1 -- 172.21.15.67:0/1623615205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 msgr2=0x564c600e7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.590+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/1623615205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 0x564c600e7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.590+0000 7fc40f7fe640 1 -- 172.21.15.67:0/1623615205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c600daf50 con 0x564c5fcfed80 2024-07-31T15:01:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.591+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/1623615205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fcfed80 0x564c600e4620 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc400006010 tx=0x7fc400108ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4497025919db85ae server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.592+0000 7fc3f57fa640 1 -- 172.21.15.67:0/1623615205 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40004b070 con 0x564c5fcfed80 2024-07-31T15:01:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.593+0000 7fc3f57fa640 1 -- 172.21.15.67:0/1623615205 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc400112070 con 0x564c5fcfed80 2024-07-31T15:01:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.593+0000 7fc3f57fa640 1 -- 172.21.15.67:0/1623615205 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4000035f0 con 0x564c5fcfed80 2024-07-31T15:01:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.595+0000 7fc41d00e740 1 -- 172.21.15.67:0/1623615205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fcfed80 msgr2=0x564c600e4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.595+0000 7fc41d00e740 1 --2- 172.21.15.67:0/1623615205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fcfed80 0x564c600e4620 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc400006010 tx=0x7fc400108ae0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.595+0000 7fc41d00e740 1 -- 172.21.15.67:0/1623615205 shutdown_connections 2024-07-31T15:01:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.595+0000 7fc41d00e740 1 --2- 172.21.15.67:0/1623615205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd401b0 0x564c600e84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.595+0000 7fc41d00e740 1 --2- 172.21.15.67:0/1623615205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c5fd1f4d0 0x564c600e7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.595+0000 7fc41d00e740 1 --2- 172.21.15.67:0/1623615205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fcfed80 0x564c600e4620 secure :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc400006010 tx=0x7fc400108ae0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.595+0000 7fc41d00e740 1 -- 172.21.15.67:0/1623615205 >> 172.21.15.67:0/1623615205 conn(0x564c5fd34300 msgr2=0x564c600e3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.596+0000 7fc41d00e740 1 -- 172.21.15.67:0/1623615205 shutdown_connections 2024-07-31T15:01:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.596+0000 7fc41d00e740 1 -- 172.21.15.67:0/1623615205 wait complete. 2024-07-31T15:01:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.597+0000 7fc41d00e740 1 -- start start 2024-07-31T15:01:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.597+0000 7fc41d00e740 1 Processor -- start 2024-07-31T15:01:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.598+0000 7fc41d00e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd1f4d0 0x564c600fb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.598+0000 7fc41d00e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fd401b0 0x564c600f9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.598+0000 7fc40effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fd401b0 0x564c600f9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.598+0000 7fc40effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fd401b0 0x564c600f9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37992/0 (socket says 172.21.15.67:37992) 2024-07-31T15:01:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.598+0000 7fc40effd640 1 -- 172.21.15.67:0/1205056146 learned_addr learned my addr 172.21.15.67:0/1205056146 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.598+0000 7fc41d00e740 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c600f9b90 0x564c600f9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.599+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd1f4d0 0x564c600fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.599+0000 7fc40ffff640 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c600f9b90 0x564c600f9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.599+0000 7fc41d00e740 1 -- 172.21.15.67:0/1205056146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc011560 con 0x564c5fd401b0 2024-07-31T15:01:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.599+0000 7fc41d00e740 1 -- 172.21.15.67:0/1205056146 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc010dd0 con 0x564c600f9b90 2024-07-31T15:01:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.599+0000 7fc41d00e740 1 -- 172.21.15.67:0/1205056146 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc00e030 con 0x564c5fd1f4d0 2024-07-31T15:01:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.600+0000 7fc41d00e740 10 client.0 _ll_register_callbacks cb 0x7fc400108f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.600+0000 7fc40ffff640 1 -- 172.21.15.67:0/1205056146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd1f4d0 msgr2=0x564c600fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.600+0000 7fc40ffff640 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd1f4d0 0x564c600fb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.601+0000 7fc40ffff640 1 -- 172.21.15.67:0/1205056146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fd401b0 msgr2=0x564c600f9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.601+0000 7fc40ffff640 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fd401b0 0x564c600f9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.601+0000 7fc40ffff640 1 -- 172.21.15.67:0/1205056146 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c60166e50 con 0x564c600f9b90 2024-07-31T15:01:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.601+0000 7fc40effd640 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fd401b0 0x564c600f9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:01:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.601+0000 7fc41d00e740 1 -- 172.21.15.67:0/1205056146 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc400112070 con 0x564c600f9b90 2024-07-31T15:01:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.602+0000 7fc40ffff640 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c600f9b90 0x564c600f9fa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc40400f5b0 tx=0x7fc404014a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdd2c75d3af65006 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.603+0000 7fc3f37f6640 10 client.27362 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:01:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.603+0000 7fc3f37f6640 1 -- 172.21.15.67:0/1205056146 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404024020 con 0x564c600f9b90 2024-07-31T15:01:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.604+0000 7fc3f37f6640 1 -- 172.21.15.67:0/1205056146 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7fc404020610 con 0x564c600f9b90 2024-07-31T15:01:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.604+0000 7fc3f37f6640 1 -- 172.21.15.67:0/1205056146 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fc40401b040 con 0x564c600f9b90 2024-07-31T15:01:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.604+0000 7fc41d00e740 10 client.27362 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.604+0000 7fc41d00e740 1 -- 172.21.15.67:0/1205056146 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=58}) -- 0x564c60167f60 con 0x564c600f9b90 2024-07-31T15:01:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.607+0000 7fc3f37f6640 1 -- 172.21.15.67:0/1205056146 <== mon.2 v2:172.21.15.73:3300/0 4 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7fc404031020 con 0x564c600f9b90 2024-07-31T15:01:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.609+0000 7fc41d00e740 10 client.27362 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.609+0000 7fc41d00e740 10 client.27362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:01:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.609+0000 7fc41d00e740 20 client.27362 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.609+0000 7fc41d00e740 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564c60168670 0x564c6016aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.609+0000 7fc41d00e740 4 client.27362 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:01:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.609+0000 7fc41d00e740 1 -- 172.21.15.67:0/1205056146 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c60055da0 con 0x564c60168670 2024-07-31T15:01:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.611+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564c60168670 0x564c6016aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.619+0000 7fc40f7fe640 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564c60168670 0x564c6016aac0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a916faa122e7902c server_cookie=8c1e63da8de407ee in_seq=0 out_seq=0 2024-07-31T15:01:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:40.619+0000 7fc3f37f6640 10 client.27362 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.198+0000 7fc3f37f6640 1 -- 172.21.15.67:0/1205056146 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564c60055da0 con 0x564c60168670 2024-07-31T15:01:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.198+0000 7fc3f37f6640 10 client.27362 handle_command_reply: tid=0 2024-07-31T15:01:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.198+0000 7fc41d00e740 1 client.27362 shutdown 2024-07-31T15:01:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.198+0000 7fc41d00e740 1 -- 172.21.15.67:0/1205056146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c600f9b90 msgr2=0x564c600f9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.198+0000 7fc41d00e740 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c600f9b90 0x564c600f9fa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc40400f5b0 tx=0x7fc404014a60 comp rx=0 tx=0).stop 2024-07-31T15:01:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.199+0000 7fc41d00e740 1 -- 172.21.15.67:0/1205056146 shutdown_connections 2024-07-31T15:01:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.199+0000 7fc41d00e740 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564c60168670 0x564c6016aac0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.199+0000 7fc41d00e740 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c600f9b90 0x564c600f9fa0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.199+0000 7fc41d00e740 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564c5fd401b0 0x564c600f9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.199+0000 7fc41d00e740 1 --2- 172.21.15.67:0/1205056146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c5fd1f4d0 0x564c600fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.199+0000 7fc41d00e740 1 -- 172.21.15.67:0/1205056146 >> 172.21.15.67:0/1205056146 conn(0x564c5fd34300 msgr2=0x564c600e7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.199+0000 7fc3f37f6640 0 client.27362 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.199+0000 7fc41d00e740 1 -- 172.21.15.67:0/1205056146 shutdown_connections 2024-07-31T15:01:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.199+0000 7fc41d00e740 1 -- 172.21.15.67:0/1205056146 wait complete. 2024-07-31T15:01:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.199+0000 7fc41d00e740 20 client.27362 trim_cache size 0 max 16384 2024-07-31T15:01:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:01:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:01:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:01:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:01:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.201+0000 7fc414de2640 1 -- 172.21.15.67:0/2408011756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3f80b3f50 msgr2=0x7fc3f80b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.201+0000 7fc414de2640 1 --2- 172.21.15.67:0/2408011756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3f80b3f50 0x7fc3f80b6460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc40400ba90 tx=0x7fc404008040 comp rx=0 tx=0).stop 2024-07-31T15:01:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.201+0000 7fc414de2640 1 -- 172.21.15.67:0/2408011756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc41010a330 msgr2=0x7fc4101a8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.202+0000 7fc414de2640 1 --2- 172.21.15.67:0/2408011756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc41010a330 0x7fc4101a8b70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc4000058c0 tx=0x7fc4000054f0 comp rx=0 tx=0).stop 2024-07-31T15:01:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.203+0000 7fc414de2640 1 -- 172.21.15.67:0/2408011756 shutdown_connections 2024-07-31T15:01:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.203+0000 7fc414de2640 1 --2- 172.21.15.67:0/2408011756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3f80b3f50 0x7fc3f80b6460 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.203+0000 7fc414de2640 1 --2- 172.21.15.67:0/2408011756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc41011c5d0 0x7fc4101ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.203+0000 7fc414de2640 1 --2- 172.21.15.67:0/2408011756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc410111650 0x7fc4101a9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.203+0000 7fc414de2640 1 --2- 172.21.15.67:0/2408011756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc41010a330 0x7fc4101a8b70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.203+0000 7fc414de2640 1 -- 172.21.15.67:0/2408011756 >> 172.21.15.67:0/2408011756 conn(0x7fc41006e320 msgr2=0x7fc41011a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.207+0000 7fc414de2640 1 -- 172.21.15.67:0/2408011756 shutdown_connections 2024-07-31T15:01:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:41.207+0000 7fc414de2640 1 -- 172.21.15.67:0/2408011756 wait complete. 2024-07-31T15:01:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:41 smithi067 ceph-mon[32750]: pgmap v728: 129 pgs: 22 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 96 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 272 KiB/s wr, 126 op/s 2024-07-31T15:01:41.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:41 smithi077 ceph-mon[37327]: pgmap v728: 129 pgs: 22 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 96 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 272 KiB/s wr, 126 op/s 2024-07-31T15:01:41.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:41 smithi073 ceph-mon[36714]: pgmap v728: 129 pgs: 22 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 96 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 272 KiB/s wr, 126 op/s 2024-07-31T15:01:42.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.178+0000 7f7dcf59e640 1 -- 172.21.15.67:0/2808814682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd01022a0 msgr2=0x7f7dd01026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:42.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.178+0000 7f7dcf59e640 1 --2- 172.21.15.67:0/2808814682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd01022a0 0x7f7dd01026b0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800b880 tx=0x7f7dc8005f90 comp rx=0 tx=0).stop 2024-07-31T15:01:42.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.180+0000 7f7dcf59e640 1 -- 172.21.15.67:0/2808814682 shutdown_connections 2024-07-31T15:01:42.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.180+0000 7f7dcf59e640 1 --2- 172.21.15.67:0/2808814682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dd0108530 0x7f7dd010ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.180+0000 7f7dcf59e640 1 --2- 172.21.15.67:0/2808814682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dd0107be0 0x7f7dd0107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.180+0000 7f7dcf59e640 1 --2- 172.21.15.67:0/2808814682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd01022a0 0x7f7dd01026b0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.180+0000 7f7dcf59e640 1 -- 172.21.15.67:0/2808814682 >> 172.21.15.67:0/2808814682 conn(0x7f7dd00fdcf0 msgr2=0x7f7dd0100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:42.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.180+0000 7f7dcf59e640 1 -- 172.21.15.67:0/2808814682 shutdown_connections 2024-07-31T15:01:42.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.180+0000 7f7dcf59e640 1 -- 172.21.15.67:0/2808814682 wait complete. 2024-07-31T15:01:42.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.181+0000 7f7dcf59e640 1 Processor -- start 2024-07-31T15:01:42.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.183+0000 7f7dcf59e640 1 -- start start 2024-07-31T15:01:42.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.184+0000 7f7dcf59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd01022a0 0x7f7dd0074960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.184+0000 7f7dcf59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dd0107be0 0x7f7dd006ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.184+0000 7f7dcf59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dd0108530 0x7f7dd006f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.184+0000 7f7dcf59e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd006d110 con 0x7f7dd01022a0 2024-07-31T15:01:42.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.184+0000 7f7dcf59e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd006cf90 con 0x7f7dd0108530 2024-07-31T15:01:42.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.184+0000 7f7dcf59e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd006d290 con 0x7f7dd0107be0 2024-07-31T15:01:42.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.184+0000 7f7dced9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dd0108530 0x7f7dd006f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:42.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.184+0000 7f7dced9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dd0108530 0x7f7dd006f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45652/0 (socket says 172.21.15.67:45652) 2024-07-31T15:01:42.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.184+0000 7f7dced9d640 1 -- 172.21.15.67:0/954921484 learned_addr learned my addr 172.21.15.67:0/954921484 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:42.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.184+0000 7f7dcdd9b640 1 --2- 172.21.15.67:0/954921484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dd0107be0 0x7f7dd006ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:42.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.184+0000 7f7dce59c640 1 --2- 172.21.15.67:0/954921484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd01022a0 0x7f7dd0074960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:42.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.185+0000 7f7dced9d640 1 -- 172.21.15.67:0/954921484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dd0107be0 msgr2=0x7f7dd006ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:42.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.185+0000 7f7dced9d640 1 --2- 172.21.15.67:0/954921484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dd0107be0 0x7f7dd006ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.185+0000 7f7dced9d640 1 -- 172.21.15.67:0/954921484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd01022a0 msgr2=0x7f7dd0074960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:42.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.185+0000 7f7dced9d640 1 --2- 172.21.15.67:0/954921484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd01022a0 0x7f7dd0074960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.185+0000 7f7dced9d640 1 -- 172.21.15.67:0/954921484 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd01bd5f0 con 0x7f7dd0108530 2024-07-31T15:01:42.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.186+0000 7f7dced9d640 1 --2- 172.21.15.67:0/954921484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dd0108530 0x7f7dd006f250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000b970 tx=0x7f7dc000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:42.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.188+0000 7f7dbf7fe640 1 -- 172.21.15.67:0/954921484 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0019070 con 0x7f7dd0108530 2024-07-31T15:01:42.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.188+0000 7f7dcf59e640 1 -- 172.21.15.67:0/954921484 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd01bd7c0 con 0x7f7dd0108530 2024-07-31T15:01:42.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.188+0000 7f7dcf59e640 1 -- 172.21.15.67:0/954921484 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd01bdcf0 con 0x7f7dd0108530 2024-07-31T15:01:42.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.188+0000 7f7dbf7fe640 1 -- 172.21.15.67:0/954921484 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dc0014440 con 0x7f7dd0108530 2024-07-31T15:01:42.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.188+0000 7f7dbf7fe640 1 -- 172.21.15.67:0/954921484 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00075d0 con 0x7f7dd0108530 2024-07-31T15:01:42.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.197+0000 7f7ddb4ef740 1 Processor -- start 2024-07-31T15:01:42.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.197+0000 7f7dbf7fe640 1 -- 172.21.15.67:0/954921484 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7dc001c070 con 0x7f7dd0108530 2024-07-31T15:01:42.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.198+0000 7f7dbf7fe640 1 --2- 172.21.15.67:0/954921484 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7db80b3ed0 0x7f7db80b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.198+0000 7f7dbf7fe640 1 -- 172.21.15.67:0/954921484 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f7dc00a6560 con 0x7f7dd0108530 2024-07-31T15:01:42.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.198+0000 7f7dcdd9b640 1 --2- 172.21.15.67:0/954921484 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7db80b3ed0 0x7f7db80b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:42.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.199+0000 7f7ddb4ef740 1 -- start start 2024-07-31T15:01:42.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.199+0000 7f7ddb4ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d1a8070 0x55de6d54d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.199+0000 7f7ddb4ef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d17f1b0 0x55de6d54dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.199+0000 7f7ddb4ef740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 0x55de6d54e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.199+0000 7f7ddb4ef740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55de6d4c3bb0 con 0x55de6d187390 2024-07-31T15:01:42.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.199+0000 7f7ddb4ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55de6d4b30b0 con 0x55de6d1a8070 2024-07-31T15:01:42.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.199+0000 7f7ddb4ef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55de6d4b70f0 con 0x55de6d17f1b0 2024-07-31T15:01:42.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.200+0000 7f7dced9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 0x55de6d54e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:42.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.200+0000 7f7dced9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 0x55de6d54e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58532/0 (socket says 172.21.15.67:58532) 2024-07-31T15:01:42.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.200+0000 7f7dced9d640 1 -- 172.21.15.67:0/2359918068 learned_addr learned my addr 172.21.15.67:0/2359918068 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:42.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.201+0000 7f7dced9d640 1 -- 172.21.15.67:0/2359918068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d17f1b0 msgr2=0x55de6d54dc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:01:42.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.201+0000 7f7dced9d640 1 --2- 172.21.15.67:0/2359918068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d17f1b0 0x55de6d54dc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.201+0000 7f7dced9d640 1 -- 172.21.15.67:0/2359918068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d1a8070 msgr2=0x55de6d54d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:42.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.201+0000 7f7dced9d640 1 --2- 172.21.15.67:0/2359918068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d1a8070 0x55de6d54d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.201+0000 7f7dced9d640 1 -- 172.21.15.67:0/2359918068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de6d459b90 con 0x55de6d187390 2024-07-31T15:01:42.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.202+0000 7f7dcdd9b640 1 --2- 172.21.15.67:0/954921484 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7db80b3ed0 0x7f7db80b63e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7dd01076d0 tx=0x7f7dc4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:42.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.203+0000 7f7dced9d640 1 --2- 172.21.15.67:0/2359918068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 0x55de6d54e170 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000b940 tx=0x7f7dc00ae760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5d858c0c61ff8a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:42.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.203+0000 7f7dbcff9640 1 -- 172.21.15.67:0/2359918068 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00a6760 con 0x55de6d187390 2024-07-31T15:01:42.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.204+0000 7f7dbcff9640 1 -- 172.21.15.67:0/2359918068 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dc00aa670 con 0x55de6d187390 2024-07-31T15:01:42.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.204+0000 7f7dbcff9640 1 -- 172.21.15.67:0/2359918068 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00a6900 con 0x55de6d187390 2024-07-31T15:01:42.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.205+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/2359918068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 msgr2=0x55de6d54e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:42.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.205+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/2359918068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 0x55de6d54e170 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000b940 tx=0x7f7dc00ae760 comp rx=0 tx=0).stop 2024-07-31T15:01:42.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.205+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/2359918068 shutdown_connections 2024-07-31T15:01:42.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.205+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/2359918068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 0x55de6d54e170 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.205+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/2359918068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d17f1b0 0x55de6d54dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.205+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/2359918068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d1a8070 0x55de6d54d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.205+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/2359918068 >> 172.21.15.67:0/2359918068 conn(0x55de6d19c460 msgr2=0x55de6d54b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:42.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.205+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/2359918068 shutdown_connections 2024-07-31T15:01:42.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.205+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/2359918068 wait complete. 2024-07-31T15:01:42.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.206+0000 7f7ddb4ef740 1 -- start start 2024-07-31T15:01:42.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.207+0000 7f7ddb4ef740 1 Processor -- start 2024-07-31T15:01:42.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.207+0000 7f7ddb4ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d17f1b0 0x55de6d559300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.207+0000 7f7ddb4ef740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 0x55de6d559860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.207+0000 7f7ddb4ef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d1a8070 0x55de6d5650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.207+0000 7f7ddb4ef740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55de6d316700 con 0x55de6d187390 2024-07-31T15:01:42.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.207+0000 7f7ddb4ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55de6d4559e0 con 0x55de6d17f1b0 2024-07-31T15:01:42.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.207+0000 7f7ddb4ef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55de6d55bd80 con 0x55de6d1a8070 2024-07-31T15:01:42.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.207+0000 7f7ddb4ef740 10 client.0 _ll_register_callbacks cb 0x55de6d4704f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.208+0000 7f7dce59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d17f1b0 0x55de6d559300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.208+0000 7f7dce59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d17f1b0 0x55de6d559300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45670/0 (socket says 172.21.15.67:45670) 2024-07-31T15:01:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.208+0000 7f7dce59c640 1 -- 172.21.15.67:0/3321181872 learned_addr learned my addr 172.21.15.67:0/3321181872 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.208+0000 7f7dced9d640 1 --2- 172.21.15.67:0/3321181872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d1a8070 0x55de6d5650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.208+0000 7f7dce59c640 1 -- 172.21.15.67:0/3321181872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d1a8070 msgr2=0x55de6d5650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.209+0000 7f7dce59c640 1 --2- 172.21.15.67:0/3321181872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d1a8070 0x55de6d5650d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.209+0000 7f7dce59c640 1 -- 172.21.15.67:0/3321181872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 msgr2=0x55de6d559860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.209+0000 7f7dce59c640 1 --2- 172.21.15.67:0/3321181872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 0x55de6d559860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.209+0000 7f7dce59c640 1 -- 172.21.15.67:0/3321181872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de6d213650 con 0x55de6d17f1b0 2024-07-31T15:01:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.209+0000 7f7dce59c640 1 --2- 172.21.15.67:0/3321181872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d17f1b0 0x55de6d559300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8000c00 tx=0x7f7dc800b510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceef581f4b4afc5c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:42.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.209+0000 7f7db5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:01:42.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.209+0000 7f7db5ffb640 1 -- 172.21.15.67:0/3321181872 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc803e070 con 0x55de6d17f1b0 2024-07-31T15:01:42.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.210+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/3321181872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dc00aa670 con 0x55de6d17f1b0 2024-07-31T15:01:42.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.210+0000 7f7db5ffb640 1 -- 172.21.15.67:0/3321181872 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f7dc80042c0 con 0x55de6d17f1b0 2024-07-31T15:01:42.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.210+0000 7f7db5ffb640 1 -- 172.21.15.67:0/3321181872 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7dc80156f0 con 0x55de6d17f1b0 2024-07-31T15:01:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.210+0000 7f7ddb4ef740 10 client.27371 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.210+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/3321181872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55de6d567ea0 con 0x55de6d17f1b0 2024-07-31T15:01:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.210+0000 7f7db5ffb640 1 -- 172.21.15.67:0/3321181872 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dc00aa670 con 0x55de6d17f1b0 2024-07-31T15:01:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.211+0000 7f7db5ffb640 1 -- 172.21.15.67:0/3321181872 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7f7dc8009b00 con 0x55de6d17f1b0 2024-07-31T15:01:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.211+0000 7f7ddb4ef740 10 client.27371 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.211+0000 7f7ddb4ef740 10 client.27371 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:01:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.211+0000 7f7ddb4ef740 20 client.27371 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.211+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/3321181872 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de6d1669f0 0x55de6d56b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.211+0000 7f7ddb4ef740 4 client.27371 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:01:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.211+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/3321181872 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de6d4c5b00 con 0x55de6d1669f0 2024-07-31T15:01:42.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.215+0000 7f7dcdd9b640 1 --2- 172.21.15.67:0/3321181872 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de6d1669f0 0x55de6d56b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:42.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.217+0000 7f7dcdd9b640 1 --2- 172.21.15.67:0/3321181872 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de6d1669f0 0x55de6d56b9f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=edf77435ea02ffd7 server_cookie=dafd39ae61ba690f in_seq=0 out_seq=0 2024-07-31T15:01:42.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.217+0000 7f7db5ffb640 10 client.27371 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:01:42.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.235+0000 7f7db5ffb640 1 -- 172.21.15.67:0/3321181872 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de6d4c5b00 con 0x55de6d1669f0 2024-07-31T15:01:42.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.235+0000 7f7db5ffb640 10 client.27371 handle_command_reply: tid=0 2024-07-31T15:01:42.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.235+0000 7f7ddb4ef740 1 client.27371 shutdown 2024-07-31T15:01:42.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.239+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/3321181872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d17f1b0 msgr2=0x55de6d559300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:42.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.239+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/3321181872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d17f1b0 0x55de6d559300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8000c00 tx=0x7f7dc800b510 comp rx=0 tx=0).stop 2024-07-31T15:01:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.241+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/3321181872 shutdown_connections 2024-07-31T15:01:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.241+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/3321181872 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de6d1669f0 0x55de6d56b9f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.241+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/3321181872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d1a8070 0x55de6d5650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.241+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/3321181872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 0x55de6d559860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.241+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/3321181872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d17f1b0 0x55de6d559300 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.241+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/3321181872 >> 172.21.15.67:0/3321181872 conn(0x55de6d19c460 msgr2=0x55de6d550880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.241+0000 7f7db5ffb640 0 client.27371 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:01:42.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.244+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/3321181872 shutdown_connections 2024-07-31T15:01:42.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.244+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/3321181872 wait complete. 2024-07-31T15:01:42.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.247+0000 7f7ddb4ef740 20 client.27371 trim_cache size 0 max 16384 2024-07-31T15:01:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.274+0000 7f7ddb4ef740 1 Processor -- start 2024-07-31T15:01:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.275+0000 7f7ddb4ef740 1 -- start start 2024-07-31T15:01:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.275+0000 7f7ddb4ef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d17f1b0 0x55de6d553810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.275+0000 7f7ddb4ef740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 0x55de6d54ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.275+0000 7f7ddb4ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d1a8070 0x55de6d564ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.275+0000 7f7ddb4ef740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7db000ff50 con 0x55de6d187390 2024-07-31T15:01:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.275+0000 7f7ddb4ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0011b50 con 0x55de6d1a8070 2024-07-31T15:01:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.275+0000 7f7ddb4ef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0010620 con 0x55de6d17f1b0 2024-07-31T15:01:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.276+0000 7f7dce59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d17f1b0 0x55de6d553810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.276+0000 7f7dced9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d1a8070 0x55de6d564ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.276+0000 7f7dced9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d1a8070 0x55de6d564ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45686/0 (socket says 172.21.15.67:45686) 2024-07-31T15:01:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.276+0000 7f7dced9d640 1 -- 172.21.15.67:0/4090549151 learned_addr learned my addr 172.21.15.67:0/4090549151 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.276+0000 7f7dced9d640 1 -- 172.21.15.67:0/4090549151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d17f1b0 msgr2=0x55de6d553810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.276+0000 7f7dced9d640 1 --2- 172.21.15.67:0/4090549151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d17f1b0 0x55de6d553810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.276+0000 7f7dced9d640 1 -- 172.21.15.67:0/4090549151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 msgr2=0x55de6d54ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.276+0000 7f7dced9d640 1 --2- 172.21.15.67:0/4090549151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 0x55de6d54ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.276+0000 7f7dced9d640 1 -- 172.21.15.67:0/4090549151 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de6d55a3c0 con 0x55de6d1a8070 2024-07-31T15:01:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.277+0000 7f7dced9d640 1 --2- 172.21.15.67:0/4090549151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d1a8070 0x55de6d564ec0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0014e80 tx=0x7f7dc00a96e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c92d1d403d1dae5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.277+0000 7f7db7fff640 1 -- 172.21.15.67:0/4090549151 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00a6ad0 con 0x55de6d1a8070 2024-07-31T15:01:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.277+0000 7f7db7fff640 1 -- 172.21.15.67:0/4090549151 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dc00a5c60 con 0x55de6d1a8070 2024-07-31T15:01:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.279+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/4090549151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d1a8070 msgr2=0x55de6d564ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.279+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/4090549151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d1a8070 0x55de6d564ec0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0014e80 tx=0x7f7dc00a96e0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.282+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/4090549151 shutdown_connections 2024-07-31T15:01:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.282+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/4090549151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d1a8070 0x55de6d564ec0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.282+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/4090549151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d187390 0x55de6d54ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.282+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/4090549151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d17f1b0 0x55de6d553810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.282+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/4090549151 >> 172.21.15.67:0/4090549151 conn(0x55de6d19c460 msgr2=0x55de6d54e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.282+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/4090549151 shutdown_connections 2024-07-31T15:01:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.282+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/4090549151 wait complete. 2024-07-31T15:01:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.282+0000 7f7ddb4ef740 1 -- start start 2024-07-31T15:01:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.283+0000 7f7ddb4ef740 1 Processor -- start 2024-07-31T15:01:42.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.283+0000 7f7ddb4ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d17f1b0 0x55de6d55bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.283+0000 7f7ddb4ef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d187390 0x55de6d55c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.283+0000 7f7ddb4ef740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d1a8070 0x55de6d55ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.283+0000 7f7ddb4ef740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0011480 con 0x55de6d1a8070 2024-07-31T15:01:42.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.283+0000 7f7ddb4ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0010cf0 con 0x55de6d17f1b0 2024-07-31T15:01:42.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.283+0000 7f7ddb4ef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7db000df50 con 0x55de6d187390 2024-07-31T15:01:42.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.283+0000 7f7ddb4ef740 10 client.0 _ll_register_callbacks cb 0x7f7dc803c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:42.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.284+0000 7f7dcdd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d187390 0x55de6d55c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:42.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.284+0000 7f7dcdd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d187390 0x55de6d55c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49120/0 (socket says 172.21.15.67:49120) 2024-07-31T15:01:42.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.284+0000 7f7dcdd9b640 1 -- 172.21.15.67:0/2472037251 learned_addr learned my addr 172.21.15.67:0/2472037251 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:42.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.284+0000 7f7dce59c640 1 --2- 172.21.15.67:0/2472037251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d17f1b0 0x55de6d55bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:42.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.287+0000 7f7dced9d640 1 --2- 172.21.15.67:0/2472037251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d1a8070 0x55de6d55ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:42.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.288+0000 7f7dced9d640 1 -- 172.21.15.67:0/2472037251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d187390 msgr2=0x55de6d55c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:42.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.288+0000 7f7dced9d640 1 --2- 172.21.15.67:0/2472037251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d187390 0x55de6d55c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.288+0000 7f7dced9d640 1 -- 172.21.15.67:0/2472037251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d17f1b0 msgr2=0x55de6d55bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:42.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.288+0000 7f7dced9d640 1 --2- 172.21.15.67:0/2472037251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d17f1b0 0x55de6d55bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.288+0000 7f7dced9d640 1 -- 172.21.15.67:0/2472037251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de6d5d21f0 con 0x55de6d1a8070 2024-07-31T15:01:42.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.289+0000 7f7dced9d640 1 --2- 172.21.15.67:0/2472037251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d1a8070 0x55de6d55ca70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00a5ff0 tx=0x7f7dc00a5e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a2296d8a16cb1e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:42.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.289+0000 7f7db5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:01:42.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.289+0000 7f7db5ffb640 1 -- 172.21.15.67:0/2472037251 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00a6e40 con 0x55de6d1a8070 2024-07-31T15:01:42.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.289+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/2472037251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dc00a5c60 con 0x55de6d1a8070 2024-07-31T15:01:42.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.289+0000 7f7db5ffb640 1 -- 172.21.15.67:0/2472037251 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f7dc000cd40 con 0x55de6d1a8070 2024-07-31T15:01:42.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.290+0000 7f7db5ffb640 1 -- 172.21.15.67:0/2472037251 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7dc00a7010 con 0x55de6d1a8070 2024-07-31T15:01:42.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.290+0000 7f7ddb4ef740 10 client.17814 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:42.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.290+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/2472037251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55de6d5d3470 con 0x55de6d1a8070 2024-07-31T15:01:42.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.291+0000 7f7db5ffb640 1 -- 172.21.15.67:0/2472037251 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dc00a5c60 con 0x55de6d1a8070 2024-07-31T15:01:42.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.292+0000 7f7db5ffb640 1 -- 172.21.15.67:0/2472037251 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7f7dc00c4cc0 con 0x55de6d1a8070 2024-07-31T15:01:42.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.292+0000 7f7ddb4ef740 10 client.17814 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:42.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.292+0000 7f7ddb4ef740 10 client.17814 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:01:42.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.292+0000 7f7ddb4ef740 20 client.17814 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:42.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.292+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/2472037251 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de6d13bda0 0x55de6d5d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:42.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.292+0000 7f7ddb4ef740 4 client.17814 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:01:42.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.292+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/2472037251 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de6d4c15a0 con 0x55de6d13bda0 2024-07-31T15:01:42.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.296+0000 7f7dce59c640 1 --2- 172.21.15.67:0/2472037251 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de6d13bda0 0x55de6d5d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:42.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.301+0000 7f7dce59c640 1 --2- 172.21.15.67:0/2472037251 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de6d13bda0 0x55de6d5d5a80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f83661b013b91d9d server_cookie=78332c2bee670c42 in_seq=0 out_seq=0 2024-07-31T15:01:42.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.301+0000 7f7db5ffb640 10 client.17814 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:01:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.331+0000 7f7db5ffb640 1 -- 172.21.15.67:0/2472037251 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de6d4c15a0 con 0x55de6d13bda0 2024-07-31T15:01:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.331+0000 7f7db5ffb640 10 client.17814 handle_command_reply: tid=0 2024-07-31T15:01:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.331+0000 7f7ddb4ef740 1 client.17814 shutdown 2024-07-31T15:01:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.331+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/2472037251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d1a8070 msgr2=0x55de6d55ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:42.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.331+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/2472037251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d1a8070 0x55de6d55ca70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00a5ff0 tx=0x7f7dc00a5e80 comp rx=0 tx=0).stop 2024-07-31T15:01:42.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.331+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/2472037251 shutdown_connections 2024-07-31T15:01:42.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.331+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/2472037251 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de6d13bda0 0x55de6d5d5a80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.331+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/2472037251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de6d1a8070 0x55de6d55ca70 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.331+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/2472037251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de6d187390 0x55de6d55c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.331+0000 7f7ddb4ef740 1 --2- 172.21.15.67:0/2472037251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de6d17f1b0 0x55de6d55bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.331+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/2472037251 >> 172.21.15.67:0/2472037251 conn(0x55de6d19c460 msgr2=0x55de6d54e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.331+0000 7f7db5ffb640 0 client.17814 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:01:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.332+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/2472037251 shutdown_connections 2024-07-31T15:01:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.332+0000 7f7ddb4ef740 1 -- 172.21.15.67:0/2472037251 wait complete. 2024-07-31T15:01:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.332+0000 7f7ddb4ef740 20 client.17814 trim_cache size 0 max 16384 2024-07-31T15:01:42.334 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:01:42.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.332+0000 7f7dcf59e640 1 -- 172.21.15.67:0/954921484 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7db80b3ed0 msgr2=0x7f7db80b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:42.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.333+0000 7f7dcf59e640 1 --2- 172.21.15.67:0/954921484 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7db80b3ed0 0x7f7db80b63e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7dd01076d0 tx=0x7f7dc4009260 comp rx=0 tx=0).stop 2024-07-31T15:01:42.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.333+0000 7f7dcf59e640 1 -- 172.21.15.67:0/954921484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dd0108530 msgr2=0x7f7dd006f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:42.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.333+0000 7f7dcf59e640 1 --2- 172.21.15.67:0/954921484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dd0108530 0x7f7dd006f250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000b970 tx=0x7f7dc000be30 comp rx=0 tx=0).stop 2024-07-31T15:01:42.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.333+0000 7f7dcf59e640 1 -- 172.21.15.67:0/954921484 shutdown_connections 2024-07-31T15:01:42.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.333+0000 7f7dcf59e640 1 --2- 172.21.15.67:0/954921484 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7db80b3ed0 0x7f7db80b63e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.333+0000 7f7dcf59e640 1 --2- 172.21.15.67:0/954921484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dd0108530 0x7f7dd006f250 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.333+0000 7f7dcf59e640 1 --2- 172.21.15.67:0/954921484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dd0107be0 0x7f7dd006ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.333+0000 7f7dcf59e640 1 --2- 172.21.15.67:0/954921484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd01022a0 0x7f7dd0074960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:42.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.333+0000 7f7dcf59e640 1 -- 172.21.15.67:0/954921484 >> 172.21.15.67:0/954921484 conn(0x7f7dd00fdcf0 msgr2=0x7f7dd0100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:42.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.333+0000 7f7dcf59e640 1 -- 172.21.15.67:0/954921484 shutdown_connections 2024-07-31T15:01:42.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:42.333+0000 7f7dcf59e640 1 -- 172.21.15.67:0/954921484 wait complete. 2024-07-31T15:01:43.039 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T15:01:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:43 smithi067 ceph-mon[32750]: pgmap v729: 129 pgs: 22 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 96 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 267 KiB/s wr, 127 op/s 2024-07-31T15:01:43.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:43 smithi077 ceph-mon[37327]: pgmap v729: 129 pgs: 22 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 96 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 267 KiB/s wr, 127 op/s 2024-07-31T15:01:43.511 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:01:43.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:43 smithi073 ceph-mon[36714]: pgmap v729: 129 pgs: 22 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 96 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 267 KiB/s wr, 127 op/s 2024-07-31T15:01:44.510 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:44 smithi067 ceph-mon[32750]: pgmap v730: 129 pgs: 22 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 96 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 147 KiB/s wr, 115 op/s 2024-07-31T15:01:44.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:44 smithi073 ceph-mon[36714]: pgmap v730: 129 pgs: 22 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 96 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 147 KiB/s wr, 115 op/s 2024-07-31T15:01:44.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:44 smithi077 ceph-mon[37327]: pgmap v730: 129 pgs: 22 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 96 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 147 KiB/s wr, 115 op/s 2024-07-31T15:01:46.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.215+0000 7f7070166640 1 -- 172.21.15.67:0/2620906084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7068111690 msgr2=0x7f7068113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:46.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.215+0000 7f7070166640 1 --2- 172.21.15.67:0/2620906084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7068111690 0x7f7068113ab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7058009860 tx=0x7f705802a340 comp rx=0 tx=0).stop 2024-07-31T15:01:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.216+0000 7f7070166640 1 -- 172.21.15.67:0/2620906084 shutdown_connections 2024-07-31T15:01:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.216+0000 7f7070166640 1 --2- 172.21.15.67:0/2620906084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7068113ff0 0x7f7068116410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.216+0000 7f7070166640 1 --2- 172.21.15.67:0/2620906084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7068111690 0x7f7068113ab0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.216+0000 7f7070166640 1 --2- 172.21.15.67:0/2620906084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70681019e0 0x7f7068101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.216+0000 7f7070166640 1 -- 172.21.15.67:0/2620906084 >> 172.21.15.67:0/2620906084 conn(0x7f7068077690 msgr2=0x7f7068077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.216+0000 7f7070166640 1 -- 172.21.15.67:0/2620906084 shutdown_connections 2024-07-31T15:01:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.216+0000 7f7070166640 1 -- 172.21.15.67:0/2620906084 wait complete. 2024-07-31T15:01:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.218+0000 7f7070166640 1 Processor -- start 2024-07-31T15:01:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.218+0000 7f7070166640 1 -- start start 2024-07-31T15:01:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.218+0000 7f7070166640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70681019e0 0x7f70681a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.219+0000 7f7070166640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7068111690 0x7f70681a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.219+0000 7f7070166640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7068113ff0 0x7f70681a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.219+0000 7f7070166640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7068119ae0 con 0x7f7068113ff0 2024-07-31T15:01:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.219+0000 7f706e703640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7068113ff0 0x7f70681a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.219+0000 7f706d701640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7068111690 0x7f70681a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.219+0000 7f706d701640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7068111690 0x7f70681a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49144/0 (socket says 172.21.15.67:49144) 2024-07-31T15:01:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.219+0000 7f706d701640 1 -- 172.21.15.67:0/1095454802 learned_addr learned my addr 172.21.15.67:0/1095454802 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.219+0000 7f706df02640 1 --2- 172.21.15.67:0/1095454802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70681019e0 0x7f70681a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.220+0000 7f7070166640 1 -- 172.21.15.67:0/1095454802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7068119960 con 0x7f70681019e0 2024-07-31T15:01:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.220+0000 7f7070166640 1 -- 172.21.15.67:0/1095454802 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7068119c60 con 0x7f7068111690 2024-07-31T15:01:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.221+0000 7f706e703640 1 -- 172.21.15.67:0/1095454802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7068111690 msgr2=0x7f70681a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.221+0000 7f706e703640 1 --2- 172.21.15.67:0/1095454802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7068111690 0x7f70681a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.221+0000 7f706e703640 1 -- 172.21.15.67:0/1095454802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70681019e0 msgr2=0x7f70681a02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.221+0000 7f706e703640 1 --2- 172.21.15.67:0/1095454802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70681019e0 0x7f70681a02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.221+0000 7f706e703640 1 -- 172.21.15.67:0/1095454802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70681a6240 con 0x7f7068113ff0 2024-07-31T15:01:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.221+0000 7f706e703640 1 --2- 172.21.15.67:0/1095454802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7068113ff0 0x7f70681a5b10 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7060009fd0 tx=0x7f706000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.222+0000 7f705effd640 1 -- 172.21.15.67:0/1095454802 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706000ce20 con 0x7f7068113ff0 2024-07-31T15:01:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.222+0000 7f705effd640 1 -- 172.21.15.67:0/1095454802 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7060005020 con 0x7f7068113ff0 2024-07-31T15:01:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.222+0000 7f705effd640 1 -- 172.21.15.67:0/1095454802 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060010690 con 0x7f7068113ff0 2024-07-31T15:01:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.222+0000 7f7070166640 1 -- 172.21.15.67:0/1095454802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70681a6530 con 0x7f7068113ff0 2024-07-31T15:01:46.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.224+0000 7f7070166640 1 -- 172.21.15.67:0/1095454802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70681acc90 con 0x7f7068113ff0 2024-07-31T15:01:46.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.228+0000 7f705effd640 1 -- 172.21.15.67:0/1095454802 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f70600054d0 con 0x7f7068113ff0 2024-07-31T15:01:46.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.229+0000 7f70766b7740 1 Processor -- start 2024-07-31T15:01:46.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.230+0000 7f70766b7740 1 -- start start 2024-07-31T15:01:46.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.230+0000 7f70766b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514194d070 0x565141cf2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.230+0000 7f70766b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651419241b0 0x565141cf2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f70766b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514192c390 0x565141cf2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f70766b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x565141c68bb0 con 0x56514194d070 2024-07-31T15:01:46.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f70766b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x565141c580b0 con 0x56514192c390 2024-07-31T15:01:46.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f70766b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565141c5c0f0 con 0x5651419241b0 2024-07-31T15:01:46.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f705effd640 1 --2- 172.21.15.67:0/1095454802 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f70540b3dc0 0x7f70540b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f706d701640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514194d070 0x565141cf2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f706d701640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514194d070 0x565141cf2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58582/0 (socket says 172.21.15.67:58582) 2024-07-31T15:01:46.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f706d701640 1 -- 172.21.15.67:0/1413081845 learned_addr learned my addr 172.21.15.67:0/1413081845 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:46.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f706df02640 1 --2- 172.21.15.67:0/1413081845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651419241b0 0x565141cf2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f706e703640 1 --2- 172.21.15.67:0/1095454802 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f70540b3dc0 0x7f70540b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f706d701640 1 --2- 172.21.15.67:0/1413081845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514192c390 0x565141cf2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f706d701640 1 -- 172.21.15.67:0/1413081845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651419241b0 msgr2=0x565141cf2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:46.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f706d701640 1 --2- 172.21.15.67:0/1413081845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651419241b0 0x565141cf2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f706d701640 1 -- 172.21.15.67:0/1413081845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514192c390 msgr2=0x565141cf2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:46.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f706d701640 1 --2- 172.21.15.67:0/1413081845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514192c390 0x565141cf2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.231+0000 7f706d701640 1 -- 172.21.15.67:0/1413081845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565141bfeb90 con 0x56514194d070 2024-07-31T15:01:46.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.232+0000 7f706d701640 1 --2- 172.21.15.67:0/1413081845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514194d070 0x565141cf2570 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f705802a860 tx=0x7f70580056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f41e760c28e509 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:46.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.232+0000 7f705effd640 1 -- 172.21.15.67:0/1095454802 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f70600e7170 con 0x7f7068113ff0 2024-07-31T15:01:46.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.232+0000 7f706e703640 1 --2- 172.21.15.67:0/1095454802 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f70540b3dc0 0x7f70540b62d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f70681a6110 tx=0x7f70600a6580 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:46.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.232+0000 7f7053fff640 1 -- 172.21.15.67:0/1413081845 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058048070 con 0x56514194d070 2024-07-31T15:01:46.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.232+0000 7f7053fff640 1 -- 172.21.15.67:0/1413081845 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7058005d30 con 0x56514194d070 2024-07-31T15:01:46.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.232+0000 7f7053fff640 1 -- 172.21.15.67:0/1413081845 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058037040 con 0x56514194d070 2024-07-31T15:01:46.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.234+0000 7f70766b7740 1 -- 172.21.15.67:0/1413081845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514194d070 msgr2=0x565141cf2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:46.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.234+0000 7f70766b7740 1 --2- 172.21.15.67:0/1413081845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514194d070 0x565141cf2570 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f705802a860 tx=0x7f70580056d0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.234+0000 7f70766b7740 1 -- 172.21.15.67:0/1413081845 shutdown_connections 2024-07-31T15:01:46.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.234+0000 7f70766b7740 1 --2- 172.21.15.67:0/1413081845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514192c390 0x565141cf2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.234+0000 7f70766b7740 1 --2- 172.21.15.67:0/1413081845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651419241b0 0x565141cf2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.234+0000 7f70766b7740 1 --2- 172.21.15.67:0/1413081845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514194d070 0x565141cf2570 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.234+0000 7f70766b7740 1 -- 172.21.15.67:0/1413081845 >> 172.21.15.67:0/1413081845 conn(0x565141941460 msgr2=0x565141cf0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:46.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.234+0000 7f70766b7740 1 -- 172.21.15.67:0/1413081845 shutdown_connections 2024-07-31T15:01:46.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.234+0000 7f70766b7740 1 -- 172.21.15.67:0/1413081845 wait complete. 2024-07-31T15:01:46.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.234+0000 7f70766b7740 1 -- start start 2024-07-31T15:01:46.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.235+0000 7f70766b7740 1 Processor -- start 2024-07-31T15:01:46.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.235+0000 7f70766b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651419241b0 0x565141cfe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.236+0000 7f70766b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56514192c390 0x565141cfe740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.236+0000 7f70766b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514194d070 0x565141d0a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.236+0000 7f70766b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x565141abb700 con 0x5651419241b0 2024-07-31T15:01:46.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.236+0000 7f706d701640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651419241b0 0x565141cfe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.236+0000 7f706d701640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651419241b0 0x565141cfe1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58590/0 (socket says 172.21.15.67:58590) 2024-07-31T15:01:46.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.236+0000 7f70766b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x565141bfa9e0 con 0x56514194d070 2024-07-31T15:01:46.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.236+0000 7f706d701640 1 -- 172.21.15.67:0/2875909651 learned_addr learned my addr 172.21.15.67:0/2875909651 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:46.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.236+0000 7f70766b7740 1 -- 172.21.15.67:0/2875909651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565141d00cf0 con 0x56514192c390 2024-07-31T15:01:46.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.236+0000 7f70766b7740 10 client.0 _ll_register_callbacks cb 0x565141c154f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:46.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.236+0000 7f706d701640 1 --2- 172.21.15.67:0/2875909651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514194d070 0x565141d0a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.237+0000 7f706d701640 1 -- 172.21.15.67:0/2875909651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56514192c390 msgr2=0x565141cfe740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:46.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.237+0000 7f706d701640 1 --2- 172.21.15.67:0/2875909651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56514192c390 0x565141cfe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.237+0000 7f706d701640 1 -- 172.21.15.67:0/2875909651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514194d070 msgr2=0x565141d0a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:46.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.237+0000 7f706d701640 1 --2- 172.21.15.67:0/2875909651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514194d070 0x565141d0a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.237+0000 7f706d701640 1 -- 172.21.15.67:0/2875909651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651419b8650 con 0x5651419241b0 2024-07-31T15:01:46.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.237+0000 7f706d701640 1 --2- 172.21.15.67:0/2875909651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651419241b0 0x565141cfe1c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7058000c00 tx=0x7f7058030b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b07e546bac3666 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:46.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.237+0000 7f70517fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:01:46.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.237+0000 7f70517fa640 1 -- 172.21.15.67:0/2875909651 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058037070 con 0x5651419241b0 2024-07-31T15:01:46.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.237+0000 7f70517fa640 1 -- 172.21.15.67:0/2875909651 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f70580038d0 con 0x5651419241b0 2024-07-31T15:01:46.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.238+0000 7f70766b7740 1 -- 172.21.15.67:0/2875909651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7058005d30 con 0x5651419241b0 2024-07-31T15:01:46.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.238+0000 7f70517fa640 1 -- 172.21.15.67:0/2875909651 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7058048040 con 0x5651419241b0 2024-07-31T15:01:46.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.238+0000 7f70766b7740 10 client.17826 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:46.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.238+0000 7f70766b7740 1 -- 172.21.15.67:0/2875909651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x565141d0cd80 con 0x5651419241b0 2024-07-31T15:01:46.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.238+0000 7f70517fa640 1 -- 172.21.15.67:0/2875909651 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7058005d30 con 0x5651419241b0 2024-07-31T15:01:46.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.238+0000 7f70517fa640 1 -- 172.21.15.67:0/2875909651 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7f705803d070 con 0x5651419241b0 2024-07-31T15:01:46.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.238+0000 7f70766b7740 10 client.17826 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:46.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.238+0000 7f70766b7740 10 client.17826 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:01:46.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.238+0000 7f70766b7740 20 client.17826 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:46.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.238+0000 7f70766b7740 1 --2- 172.21.15.67:0/2875909651 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x56514190b9f0 0x565141d100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.238+0000 7f70766b7740 4 client.17826 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:01:46.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.238+0000 7f70766b7740 1 -- 172.21.15.67:0/2875909651 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565141c6ab00 con 0x56514190b9f0 2024-07-31T15:01:46.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.243+0000 7f706df02640 1 --2- 172.21.15.67:0/2875909651 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x56514190b9f0 0x565141d100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.244+0000 7f706df02640 1 --2- 172.21.15.67:0/2875909651 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x56514190b9f0 0x565141d100e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a52175c45af20d3a server_cookie=88e3982508a741a7 in_seq=0 out_seq=0 2024-07-31T15:01:46.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.244+0000 7f70517fa640 10 client.17826 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:01:46.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.253+0000 7f70517fa640 1 -- 172.21.15.67:0/2875909651 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565141c6ab00 con 0x56514190b9f0 2024-07-31T15:01:46.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.253+0000 7f70517fa640 10 client.17826 handle_command_reply: tid=0 2024-07-31T15:01:46.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.253+0000 7f70766b7740 1 client.17826 shutdown 2024-07-31T15:01:46.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.254+0000 7f70766b7740 1 -- 172.21.15.67:0/2875909651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651419241b0 msgr2=0x565141cfe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:46.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.254+0000 7f70766b7740 1 --2- 172.21.15.67:0/2875909651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651419241b0 0x565141cfe1c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7058000c00 tx=0x7f7058030b50 comp rx=0 tx=0).stop 2024-07-31T15:01:46.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.254+0000 7f70766b7740 1 -- 172.21.15.67:0/2875909651 shutdown_connections 2024-07-31T15:01:46.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.254+0000 7f70766b7740 1 --2- 172.21.15.67:0/2875909651 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x56514190b9f0 0x565141d100e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.254+0000 7f70766b7740 1 --2- 172.21.15.67:0/2875909651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514194d070 0x565141d0a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.254+0000 7f70766b7740 1 --2- 172.21.15.67:0/2875909651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56514192c390 0x565141cfe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.254+0000 7f70766b7740 1 --2- 172.21.15.67:0/2875909651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651419241b0 0x565141cfe1c0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.254+0000 7f70766b7740 1 -- 172.21.15.67:0/2875909651 >> 172.21.15.67:0/2875909651 conn(0x565141941460 msgr2=0x565141ceec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:46.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.254+0000 7f70517fa640 0 client.17826 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:01:46.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.254+0000 7f70766b7740 1 -- 172.21.15.67:0/2875909651 shutdown_connections 2024-07-31T15:01:46.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.255+0000 7f70766b7740 1 -- 172.21.15.67:0/2875909651 wait complete. 2024-07-31T15:01:46.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.255+0000 7f70766b7740 20 client.17826 trim_cache size 0 max 16384 2024-07-31T15:01:46.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.279+0000 7f70766b7740 1 Processor -- start 2024-07-31T15:01:46.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.279+0000 7f70766b7740 1 -- start start 2024-07-31T15:01:46.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.279+0000 7f70766b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651419241b0 0x565141cfd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.279+0000 7f70766b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514192c390 0x565141cffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.279+0000 7f70766b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56514194d070 0x565141cf77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.279+0000 7f70766b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f70681b7090 con 0x56514192c390 2024-07-31T15:01:46.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.279+0000 7f70766b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f70681b8c90 con 0x5651419241b0 2024-07-31T15:01:46.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.279+0000 7f70766b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f70681b7760 con 0x56514194d070 2024-07-31T15:01:46.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.280+0000 7f706df02640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514192c390 0x565141cffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.280+0000 7f706df02640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514192c390 0x565141cffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58596/0 (socket says 172.21.15.67:58596) 2024-07-31T15:01:46.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.280+0000 7f706df02640 1 -- 172.21.15.67:0/243912761 learned_addr learned my addr 172.21.15.67:0/243912761 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:46.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.280+0000 7f706d701640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56514194d070 0x565141cf77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.280+0000 7f706d701640 1 --2- 172.21.15.67:0/243912761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651419241b0 0x565141cfd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.280+0000 7f706d701640 1 -- 172.21.15.67:0/243912761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56514194d070 msgr2=0x565141cf77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:46.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.280+0000 7f706d701640 1 --2- 172.21.15.67:0/243912761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56514194d070 0x565141cf77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.280+0000 7f706d701640 1 -- 172.21.15.67:0/243912761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514192c390 msgr2=0x565141cffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:46.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.280+0000 7f706d701640 1 --2- 172.21.15.67:0/243912761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514192c390 0x565141cffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.280+0000 7f706d701640 1 -- 172.21.15.67:0/243912761 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565141cfeab0 con 0x5651419241b0 2024-07-31T15:01:46.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.280+0000 7f706d701640 1 --2- 172.21.15.67:0/243912761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651419241b0 0x565141cfd4d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f705802d020 tx=0x7f7058009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=183930d9c4880784 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:46.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.281+0000 7f70537fe640 1 -- 172.21.15.67:0/243912761 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058037040 con 0x5651419241b0 2024-07-31T15:01:46.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.281+0000 7f70537fe640 1 -- 172.21.15.67:0/243912761 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7058049040 con 0x5651419241b0 2024-07-31T15:01:46.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.281+0000 7f70537fe640 1 -- 172.21.15.67:0/243912761 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058005320 con 0x5651419241b0 2024-07-31T15:01:46.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.282+0000 7f70766b7740 1 -- 172.21.15.67:0/243912761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651419241b0 msgr2=0x565141cfd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:46.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.282+0000 7f70766b7740 1 --2- 172.21.15.67:0/243912761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651419241b0 0x565141cfd4d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f705802d020 tx=0x7f7058009570 comp rx=0 tx=0).stop 2024-07-31T15:01:46.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.282+0000 7f70766b7740 1 -- 172.21.15.67:0/243912761 shutdown_connections 2024-07-31T15:01:46.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.282+0000 7f70766b7740 1 --2- 172.21.15.67:0/243912761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56514194d070 0x565141cf77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.282+0000 7f70766b7740 1 --2- 172.21.15.67:0/243912761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514192c390 0x565141cffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.282+0000 7f70766b7740 1 --2- 172.21.15.67:0/243912761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5651419241b0 0x565141cfd4d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.282+0000 7f70766b7740 1 -- 172.21.15.67:0/243912761 >> 172.21.15.67:0/243912761 conn(0x565141941460 msgr2=0x565141cff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:46.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.282+0000 7f70766b7740 1 -- 172.21.15.67:0/243912761 shutdown_connections 2024-07-31T15:01:46.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.282+0000 7f70766b7740 1 -- 172.21.15.67:0/243912761 wait complete. 2024-07-31T15:01:46.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.283+0000 7f70766b7740 1 -- start start 2024-07-31T15:01:46.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.283+0000 7f70766b7740 1 Processor -- start 2024-07-31T15:01:46.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.284+0000 7f70766b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651419241b0 0x565141d0e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.284+0000 7f70766b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514192c390 0x565141d0e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.284+0000 7f70766b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56514194d070 0x565141d76610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.284+0000 7f70766b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f70681b85c0 con 0x5651419241b0 2024-07-31T15:01:46.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.284+0000 7f70766b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f70681b7e30 con 0x56514192c390 2024-07-31T15:01:46.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.284+0000 7f70766b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f70681b5090 con 0x56514194d070 2024-07-31T15:01:46.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.284+0000 7f70766b7740 10 client.0 _ll_register_callbacks cb 0x7f7058046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:46.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.284+0000 7f706d701640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651419241b0 0x565141d0e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.284+0000 7f706d701640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651419241b0 0x565141d0e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58606/0 (socket says 172.21.15.67:58606) 2024-07-31T15:01:46.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.284+0000 7f706d701640 1 -- 172.21.15.67:0/3976440703 learned_addr learned my addr 172.21.15.67:0/3976440703 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:46.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.284+0000 7f706df02640 1 --2- 172.21.15.67:0/3976440703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514192c390 0x565141d0e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.284+0000 7f706d701640 1 --2- 172.21.15.67:0/3976440703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56514194d070 0x565141d76610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.285+0000 7f706d701640 1 -- 172.21.15.67:0/3976440703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56514194d070 msgr2=0x565141d76610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:46.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.285+0000 7f706d701640 1 --2- 172.21.15.67:0/3976440703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56514194d070 0x565141d76610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.285+0000 7f706d701640 1 -- 172.21.15.67:0/3976440703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514192c390 msgr2=0x565141d0e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:46.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.285+0000 7f706d701640 1 --2- 172.21.15.67:0/3976440703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514192c390 0x565141d0e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.285+0000 7f706d701640 1 -- 172.21.15.67:0/3976440703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565141d0f060 con 0x5651419241b0 2024-07-31T15:01:46.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.285+0000 7f706df02640 1 --2- 172.21.15.67:0/3976440703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514192c390 0x565141d0e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:01:46.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.285+0000 7f706d701640 1 --2- 172.21.15.67:0/3976440703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651419241b0 0x565141d0e2a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f705802a860 tx=0x7f7058032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a587e91fe5973896 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:46.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.285+0000 7f70517fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:01:46.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.285+0000 7f70517fa640 1 -- 172.21.15.67:0/3976440703 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058037040 con 0x5651419241b0 2024-07-31T15:01:46.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.285+0000 7f70517fa640 1 -- 172.21.15.67:0/3976440703 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f70580050a0 con 0x5651419241b0 2024-07-31T15:01:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.285+0000 7f70766b7740 1 -- 172.21.15.67:0/3976440703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7058049040 con 0x5651419241b0 2024-07-31T15:01:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.286+0000 7f70517fa640 1 -- 172.21.15.67:0/3976440703 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f70580052a0 con 0x5651419241b0 2024-07-31T15:01:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.286+0000 7f70766b7740 10 client.17832 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.286+0000 7f70766b7740 1 -- 172.21.15.67:0/3976440703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x565141d77b60 con 0x5651419241b0 2024-07-31T15:01:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.286+0000 7f70517fa640 1 -- 172.21.15.67:0/3976440703 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7058049040 con 0x5651419241b0 2024-07-31T15:01:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.286+0000 7f70517fa640 1 -- 172.21.15.67:0/3976440703 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7f705803d070 con 0x5651419241b0 2024-07-31T15:01:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.286+0000 7f70766b7740 10 client.17832 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.286+0000 7f70766b7740 10 client.17832 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:01:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.286+0000 7f70766b7740 20 client.17832 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.286+0000 7f70766b7740 1 --2- 172.21.15.67:0/3976440703 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5651418e0da0 0x565141d7a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.286+0000 7f70766b7740 4 client.17832 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:01:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.286+0000 7f70766b7740 1 -- 172.21.15.67:0/3976440703 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565141c665a0 con 0x5651418e0da0 2024-07-31T15:01:46.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.287+0000 7f706df02640 1 --2- 172.21.15.67:0/3976440703 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5651418e0da0 0x565141d7a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.289+0000 7f706df02640 1 --2- 172.21.15.67:0/3976440703 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5651418e0da0 0x565141d7a120 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=965c0f949297e44 server_cookie=a006cc42bc7fc40f in_seq=0 out_seq=0 2024-07-31T15:01:46.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.289+0000 7f70517fa640 10 client.17832 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:01:46.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.303+0000 7f70517fa640 1 -- 172.21.15.67:0/3976440703 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565141c665a0 con 0x5651418e0da0 2024-07-31T15:01:46.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.303+0000 7f70517fa640 10 client.17832 handle_command_reply: tid=0 2024-07-31T15:01:46.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.303+0000 7f70766b7740 1 client.17832 shutdown 2024-07-31T15:01:46.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.304+0000 7f70766b7740 1 -- 172.21.15.67:0/3976440703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651419241b0 msgr2=0x565141d0e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:46.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.304+0000 7f70766b7740 1 --2- 172.21.15.67:0/3976440703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651419241b0 0x565141d0e2a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f705802a860 tx=0x7f7058032dc0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.304+0000 7f70766b7740 1 -- 172.21.15.67:0/3976440703 shutdown_connections 2024-07-31T15:01:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.304+0000 7f70766b7740 1 --2- 172.21.15.67:0/3976440703 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5651418e0da0 0x565141d7a120 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.304+0000 7f70766b7740 1 --2- 172.21.15.67:0/3976440703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56514194d070 0x565141d76610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.304+0000 7f70766b7740 1 --2- 172.21.15.67:0/3976440703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514192c390 0x565141d0e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.304+0000 7f70766b7740 1 --2- 172.21.15.67:0/3976440703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5651419241b0 0x565141d0e2a0 secure :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f705802a860 tx=0x7f7058032dc0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.304+0000 7f70766b7740 1 -- 172.21.15.67:0/3976440703 >> 172.21.15.67:0/3976440703 conn(0x565141941460 msgr2=0x565141cf5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:46.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.304+0000 7f70517fa640 0 client.17832 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:01:46.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.305+0000 7f70766b7740 1 -- 172.21.15.67:0/3976440703 shutdown_connections 2024-07-31T15:01:46.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.305+0000 7f70766b7740 1 -- 172.21.15.67:0/3976440703 wait complete. 2024-07-31T15:01:46.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.305+0000 7f70766b7740 20 client.17832 trim_cache size 0 max 16384 2024-07-31T15:01:46.307 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:01:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.307+0000 7f7070166640 1 -- 172.21.15.67:0/1095454802 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f70540b3dc0 msgr2=0x7f70540b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.307+0000 7f7070166640 1 --2- 172.21.15.67:0/1095454802 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f70540b3dc0 0x7f70540b62d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f70681a6110 tx=0x7f70600a6580 comp rx=0 tx=0).stop 2024-07-31T15:01:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.307+0000 7f7070166640 1 -- 172.21.15.67:0/1095454802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7068113ff0 msgr2=0x7f70681a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.307+0000 7f7070166640 1 --2- 172.21.15.67:0/1095454802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7068113ff0 0x7f70681a5b10 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7060009fd0 tx=0x7f706000ef10 comp rx=0 tx=0).stop 2024-07-31T15:01:46.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.307+0000 7f7070166640 1 -- 172.21.15.67:0/1095454802 shutdown_connections 2024-07-31T15:01:46.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.307+0000 7f7070166640 1 --2- 172.21.15.67:0/1095454802 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f70540b3dc0 0x7f70540b62d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.307+0000 7f7070166640 1 --2- 172.21.15.67:0/1095454802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7068113ff0 0x7f70681a5b10 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.307+0000 7f7070166640 1 --2- 172.21.15.67:0/1095454802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7068111690 0x7f70681a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.307+0000 7f7070166640 1 --2- 172.21.15.67:0/1095454802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70681019e0 0x7f70681a02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:46.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.307+0000 7f7070166640 1 -- 172.21.15.67:0/1095454802 >> 172.21.15.67:0/1095454802 conn(0x7f7068077690 msgr2=0x7f7068115480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:46.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.307+0000 7f7070166640 1 -- 172.21.15.67:0/1095454802 shutdown_connections 2024-07-31T15:01:46.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:46.307+0000 7f7070166640 1 -- 172.21.15.67:0/1095454802 wait complete. 2024-07-31T15:01:47.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:46 smithi067 ceph-mon[32750]: pgmap v731: 129 pgs: 16 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 103 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 325 KiB/s wr, 147 op/s 2024-07-31T15:01:47.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T15:01:47.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:46 smithi073 ceph-mon[36714]: pgmap v731: 129 pgs: 16 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 103 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 325 KiB/s wr, 147 op/s 2024-07-31T15:01:47.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:46 smithi077 ceph-mon[37327]: pgmap v731: 129 pgs: 16 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 103 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 325 KiB/s wr, 147 op/s 2024-07-31T15:01:47.430 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:01:49.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:48 smithi067 ceph-mon[32750]: pgmap v732: 129 pgs: 16 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 103 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 396 KiB/s wr, 152 op/s 2024-07-31T15:01:49.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:48 smithi073 ceph-mon[36714]: pgmap v732: 129 pgs: 16 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 103 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 396 KiB/s wr, 152 op/s 2024-07-31T15:01:49.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:48 smithi077 ceph-mon[37327]: pgmap v732: 129 pgs: 16 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 103 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 396 KiB/s wr, 152 op/s 2024-07-31T15:01:50.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.547+0000 7efc09865640 1 -- 172.21.15.67:0/253675509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc0410cc40 msgr2=0x7efc041166d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:50.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.547+0000 7efc09865640 1 --2- 172.21.15.67:0/253675509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc0410cc40 0x7efc041166d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009900 tx=0x7efbfc02a440 comp rx=0 tx=0).stop 2024-07-31T15:01:50.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.548+0000 7efc09865640 1 -- 172.21.15.67:0/253675509 shutdown_connections 2024-07-31T15:01:50.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.548+0000 7efc09865640 1 --2- 172.21.15.67:0/253675509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc0410cc40 0x7efc041166d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.548+0000 7efc09865640 1 --2- 172.21.15.67:0/253675509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc0410c2f0 0x7efc0410c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.548+0000 7efc09865640 1 --2- 172.21.15.67:0/253675509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc04109ec0 0x7efc0410a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.548+0000 7efc09865640 1 -- 172.21.15.67:0/253675509 >> 172.21.15.67:0/253675509 conn(0x7efc04079100 msgr2=0x7efc0407b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:50.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.548+0000 7efc09865640 1 -- 172.21.15.67:0/253675509 shutdown_connections 2024-07-31T15:01:50.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.548+0000 7efc09865640 1 -- 172.21.15.67:0/253675509 wait complete. 2024-07-31T15:01:50.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.549+0000 7efc09865640 1 Processor -- start 2024-07-31T15:01:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.550+0000 7efc09865640 1 -- start start 2024-07-31T15:01:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.550+0000 7efc09865640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc04109ec0 0x7efc04085620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.550+0000 7efc09865640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc0410c2f0 0x7efc0407e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.550+0000 7efc09865640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc0410cc40 0x7efc0407ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.550+0000 7efc09865640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efc04119c00 con 0x7efc0410c2f0 2024-07-31T15:01:50.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.550+0000 7efc09865640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc04119a80 con 0x7efc04109ec0 2024-07-31T15:01:50.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.550+0000 7efc09865640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efc04119d80 con 0x7efc0410cc40 2024-07-31T15:01:50.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.551+0000 7efc02ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc04109ec0 0x7efc04085620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.551+0000 7efbfa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc0410c2f0 0x7efc0407e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.551+0000 7efbfa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc0410c2f0 0x7efc0407e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58624/0 (socket says 172.21.15.67:58624) 2024-07-31T15:01:50.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.551+0000 7efbfa7fc640 1 -- 172.21.15.67:0/2283894818 learned_addr learned my addr 172.21.15.67:0/2283894818 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:50.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.551+0000 7efc037fe640 1 --2- 172.21.15.67:0/2283894818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc0410cc40 0x7efc0407ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.552+0000 7efc02ffd640 1 -- 172.21.15.67:0/2283894818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc0410cc40 msgr2=0x7efc0407ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:50.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.552+0000 7efc02ffd640 1 --2- 172.21.15.67:0/2283894818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc0410cc40 0x7efc0407ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.552+0000 7efc02ffd640 1 -- 172.21.15.67:0/2283894818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc0410c2f0 msgr2=0x7efc0407e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:50.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.552+0000 7efc02ffd640 1 --2- 172.21.15.67:0/2283894818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc0410c2f0 0x7efc0407e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.552+0000 7efc02ffd640 1 -- 172.21.15.67:0/2283894818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0407f5d0 con 0x7efc04109ec0 2024-07-31T15:01:50.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.553+0000 7efc02ffd640 1 --2- 172.21.15.67:0/2283894818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc04109ec0 0x7efc04085620 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efbf400bdf0 tx=0x7efbf4009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:50.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.554+0000 7efc08863640 1 -- 172.21.15.67:0/2283894818 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4019070 con 0x7efc04109ec0 2024-07-31T15:01:50.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.554+0000 7efc09865640 1 -- 172.21.15.67:0/2283894818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc04084290 con 0x7efc04109ec0 2024-07-31T15:01:50.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.554+0000 7efc09865640 1 -- 172.21.15.67:0/2283894818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc040848a0 con 0x7efc04109ec0 2024-07-31T15:01:50.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.555+0000 7efc08863640 1 -- 172.21.15.67:0/2283894818 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf40148f0 con 0x7efc04109ec0 2024-07-31T15:01:50.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.556+0000 7efc08863640 1 -- 172.21.15.67:0/2283894818 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4012830 con 0x7efc04109ec0 2024-07-31T15:01:50.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.558+0000 7efc0fdb6740 1 Processor -- start 2024-07-31T15:01:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.561+0000 7efc0fdb6740 1 -- start start 2024-07-31T15:01:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.561+0000 7efc0fdb6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d8f070 0x558c44134690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.561+0000 7efc0fdb6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d661b0 0x558c44134bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.561+0000 7efc0fdb6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d6e390 0x558c44135110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.561+0000 7efc0fdb6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558c440aabb0 con 0x558c43d661b0 2024-07-31T15:01:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.561+0000 7efc0fdb6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558c4409a0b0 con 0x558c43d8f070 2024-07-31T15:01:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.561+0000 7efc0fdb6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558c4409e0f0 con 0x558c43d6e390 2024-07-31T15:01:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.562+0000 7efc08863640 1 -- 172.21.15.67:0/2283894818 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7efbf401c070 con 0x7efc04109ec0 2024-07-31T15:01:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.562+0000 7efbfa7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d6e390 0x558c44135110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.562+0000 7efbfa7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d6e390 0x558c44135110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49216/0 (socket says 172.21.15.67:49216) 2024-07-31T15:01:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.562+0000 7efbfa7fc640 1 -- 172.21.15.67:0/1702249990 learned_addr learned my addr 172.21.15.67:0/1702249990 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.562+0000 7efc02ffd640 1 --2- 172.21.15.67:0/1702249990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d661b0 0x558c44134bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.562+0000 7efbfa7fc640 1 -- 172.21.15.67:0/1702249990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d8f070 msgr2=0x558c44134690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:01:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.562+0000 7efbfa7fc640 1 --2- 172.21.15.67:0/1702249990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d8f070 0x558c44134690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.562+0000 7efbfa7fc640 1 -- 172.21.15.67:0/1702249990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d661b0 msgr2=0x558c44134bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.562+0000 7efbfa7fc640 1 --2- 172.21.15.67:0/1702249990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d661b0 0x558c44134bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.562+0000 7efbfa7fc640 1 -- 172.21.15.67:0/1702249990 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c44040b90 con 0x558c43d6e390 2024-07-31T15:01:50.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.564+0000 7efbfa7fc640 1 --2- 172.21.15.67:0/1702249990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d6e390 0x558c44135110 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efbec00bd10 tx=0x7efbec0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c8fbdaecf9ec721 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:50.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.566+0000 7efbf9ffb640 1 -- 172.21.15.67:0/1702249990 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec01c680 con 0x558c43d6e390 2024-07-31T15:01:50.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.566+0000 7efc08863640 1 --2- 172.21.15.67:0/2283894818 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efbf00b3e40 0x7efbf00b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.566+0000 7efbf9ffb640 1 -- 172.21.15.67:0/1702249990 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbec01ccd0 con 0x558c43d6e390 2024-07-31T15:01:50.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.567+0000 7efc037fe640 1 --2- 172.21.15.67:0/2283894818 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efbf00b3e40 0x7efbf00b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.567+0000 7efc08863640 1 -- 172.21.15.67:0/2283894818 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7efbf40e39b0 con 0x7efc04109ec0 2024-07-31T15:01:50.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.568+0000 7efbf9ffb640 1 -- 172.21.15.67:0/1702249990 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec011c40 con 0x558c43d6e390 2024-07-31T15:01:50.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.569+0000 7efc037fe640 1 --2- 172.21.15.67:0/2283894818 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efbf00b3e40 0x7efbf00b6350 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efbfc005ab0 tx=0x7efbfc005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:50.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.569+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1702249990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d6e390 msgr2=0x558c44135110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:50.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.569+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1702249990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d6e390 0x558c44135110 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efbec00bd10 tx=0x7efbec0107a0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.569+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1702249990 shutdown_connections 2024-07-31T15:01:50.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.569+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1702249990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d6e390 0x558c44135110 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.569+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1702249990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d661b0 0x558c44134bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.569+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1702249990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d8f070 0x558c44134690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.569+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1702249990 >> 172.21.15.67:0/1702249990 conn(0x558c43d83460 msgr2=0x558c44132580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:50.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.570+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1702249990 shutdown_connections 2024-07-31T15:01:50.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.570+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1702249990 wait complete. 2024-07-31T15:01:50.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.570+0000 7efc0fdb6740 1 -- start start 2024-07-31T15:01:50.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.571+0000 7efc0fdb6740 1 Processor -- start 2024-07-31T15:01:50.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.571+0000 7efc0fdb6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d661b0 0x558c44140270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.571+0000 7efc0fdb6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d6e390 0x558c441407b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.571+0000 7efc02ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d661b0 0x558c44140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.571+0000 7efc02ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d661b0 0x558c44140270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45754/0 (socket says 172.21.15.67:45754) 2024-07-31T15:01:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.572+0000 7efbfa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d6e390 0x558c441407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.572+0000 7efbfa7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d6e390 0x558c441407b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58652/0 (socket says 172.21.15.67:58652) 2024-07-31T15:01:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.572+0000 7efc02ffd640 1 -- 172.21.15.67:0/1055827962 learned_addr learned my addr 172.21.15.67:0/1055827962 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.572+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1055827962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 0x558c4414c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.572+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1055827962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558c43efd700 con 0x558c43d6e390 2024-07-31T15:01:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.572+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1055827962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558c4403c9e0 con 0x558c43d661b0 2024-07-31T15:01:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.572+0000 7efc037fe640 1 --2- 172.21.15.67:0/1055827962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 0x558c4414c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.572+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1055827962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558c44142ec0 con 0x558c43d8f070 2024-07-31T15:01:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.573+0000 7efc037fe640 1 -- 172.21.15.67:0/1055827962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d661b0 msgr2=0x558c44140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.573+0000 7efc037fe640 1 --2- 172.21.15.67:0/1055827962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d661b0 0x558c44140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.573+0000 7efc037fe640 1 -- 172.21.15.67:0/1055827962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d6e390 msgr2=0x558c441407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.573+0000 7efc037fe640 1 --2- 172.21.15.67:0/1055827962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d6e390 0x558c441407b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.573+0000 7efc037fe640 1 -- 172.21.15.67:0/1055827962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c43dfa650 con 0x558c43d8f070 2024-07-31T15:01:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.573+0000 7efc02ffd640 1 --2- 172.21.15.67:0/1055827962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d661b0 0x558c44140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:01:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.573+0000 7efc0fdb6740 10 client.0 _ll_register_callbacks cb 0x558c440574f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.573+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1055827962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbec01ccd0 con 0x558c43d8f070 2024-07-31T15:01:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.573+0000 7efc037fe640 1 --2- 172.21.15.67:0/1055827962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 0x558c4414c290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efbfc02a960 tx=0x7efbfc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a3c0b73cf3a1548 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.574+0000 7efbe2ffd640 10 client.27376 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:01:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.574+0000 7efbe2ffd640 1 -- 172.21.15.67:0/1055827962 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc057050 con 0x558c43d8f070 2024-07-31T15:01:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.574+0000 7efbe2ffd640 1 -- 172.21.15.67:0/1055827962 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7efbfc004ad0 con 0x558c43d8f070 2024-07-31T15:01:50.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.575+0000 7efbe2ffd640 1 -- 172.21.15.67:0/1055827962 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efbfc04d070 con 0x558c43d8f070 2024-07-31T15:01:50.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.575+0000 7efc0fdb6740 10 client.27376 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:50.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.575+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1055827962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=58}) -- 0x558c4414eda0 con 0x558c43d8f070 2024-07-31T15:01:50.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.576+0000 7efbe2ffd640 1 -- 172.21.15.67:0/1055827962 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7efbfc05a020 con 0x558c43d8f070 2024-07-31T15:01:50.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.576+0000 7efc0fdb6740 10 client.27376 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:50.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.576+0000 7efc0fdb6740 10 client.27376 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:01:50.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.576+0000 7efc0fdb6740 20 client.27376 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:50.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.576+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1055827962 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558c43d4d9f0 0x558c441522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.576+0000 7efc0fdb6740 4 client.27376 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:01:50.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.576+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1055827962 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c440acb00 con 0x558c43d4d9f0 2024-07-31T15:01:50.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.577+0000 7efc02ffd640 1 --2- 172.21.15.67:0/1055827962 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558c43d4d9f0 0x558c441522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.577+0000 7efc02ffd640 1 --2- 172.21.15.67:0/1055827962 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558c43d4d9f0 0x558c441522d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e30e6a9202b73f3e server_cookie=c8c229c1e381b67f in_seq=0 out_seq=0 2024-07-31T15:01:50.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.578+0000 7efbe2ffd640 10 client.27376 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:01:50.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.584+0000 7efbe2ffd640 1 -- 172.21.15.67:0/1055827962 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c440acb00 con 0x558c43d4d9f0 2024-07-31T15:01:50.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.585+0000 7efbe2ffd640 10 client.27376 handle_command_reply: tid=0 2024-07-31T15:01:50.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.585+0000 7efc0fdb6740 1 client.27376 shutdown 2024-07-31T15:01:50.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.586+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1055827962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 msgr2=0x558c4414c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:50.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.586+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1055827962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 0x558c4414c290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efbfc02a960 tx=0x7efbfc009260 comp rx=0 tx=0).stop 2024-07-31T15:01:50.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.586+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1055827962 shutdown_connections 2024-07-31T15:01:50.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.586+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1055827962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 0x558c4414c290 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.586+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1055827962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d6e390 0x558c441407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.586+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1055827962 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558c43d4d9f0 0x558c441522d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.586+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1055827962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d661b0 0x558c44140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.586+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1055827962 >> 172.21.15.67:0/1055827962 conn(0x558c43d83460 msgr2=0x558c44130d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:50.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.586+0000 7efbe2ffd640 0 client.27376 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:01:50.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.586+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1055827962 shutdown_connections 2024-07-31T15:01:50.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.586+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1055827962 wait complete. 2024-07-31T15:01:50.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.586+0000 7efc0fdb6740 20 client.27376 trim_cache size 0 max 16384 2024-07-31T15:01:50.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.616+0000 7efc0fdb6740 1 Processor -- start 2024-07-31T15:01:50.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.616+0000 7efc0fdb6740 1 -- start start 2024-07-31T15:01:50.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.616+0000 7efc0fdb6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d661b0 0x558c44131410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.616+0000 7efc0fdb6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d6e390 0x558c4414c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.616+0000 7efc0fdb6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 0x558c4414c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.617+0000 7efc0fdb6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efbe400fec0 con 0x558c43d661b0 2024-07-31T15:01:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.617+0000 7efc02ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d661b0 0x558c44131410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.617+0000 7efc0fdb6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efbe4011ac0 con 0x558c43d6e390 2024-07-31T15:01:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.617+0000 7efc0fdb6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efbe4010590 con 0x558c43d8f070 2024-07-31T15:01:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.617+0000 7efc02ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d661b0 0x558c44131410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58668/0 (socket says 172.21.15.67:58668) 2024-07-31T15:01:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.617+0000 7efc02ffd640 1 -- 172.21.15.67:0/2453971082 learned_addr learned my addr 172.21.15.67:0/2453971082 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.618+0000 7efc037fe640 1 --2- 172.21.15.67:0/2453971082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 0x558c4414c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.619+0000 7efbfa7fc640 1 --2- 172.21.15.67:0/2453971082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d6e390 0x558c4414c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.620+0000 7efc02ffd640 1 -- 172.21.15.67:0/2453971082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 msgr2=0x558c4414c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:50.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.620+0000 7efc02ffd640 1 --2- 172.21.15.67:0/2453971082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 0x558c4414c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.620+0000 7efc02ffd640 1 -- 172.21.15.67:0/2453971082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d6e390 msgr2=0x558c4414c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:50.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.620+0000 7efc02ffd640 1 --2- 172.21.15.67:0/2453971082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d6e390 0x558c4414c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.620+0000 7efc02ffd640 1 -- 172.21.15.67:0/2453971082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c44140e60 con 0x558c43d661b0 2024-07-31T15:01:50.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.621+0000 7efc02ffd640 1 --2- 172.21.15.67:0/2453971082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d661b0 0x558c44131410 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7efbf40e5010 tx=0x7efbf40e5e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab16b63c13d41e07 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:50.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.621+0000 7efbf97fa640 1 -- 172.21.15.67:0/2453971082 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf40aa350 con 0x558c43d661b0 2024-07-31T15:01:50.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.621+0000 7efbf97fa640 1 -- 172.21.15.67:0/2453971082 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf40ae8a0 con 0x558c43d661b0 2024-07-31T15:01:50.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.622+0000 7efbf97fa640 1 -- 172.21.15.67:0/2453971082 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf40b3120 con 0x558c43d661b0 2024-07-31T15:01:50.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.623+0000 7efc0fdb6740 1 -- 172.21.15.67:0/2453971082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d661b0 msgr2=0x558c44131410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:50.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.623+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/2453971082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d661b0 0x558c44131410 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7efbf40e5010 tx=0x7efbf40e5e40 comp rx=0 tx=0).stop 2024-07-31T15:01:50.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.623+0000 7efc0fdb6740 1 -- 172.21.15.67:0/2453971082 shutdown_connections 2024-07-31T15:01:50.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.623+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/2453971082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 0x558c4414c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.623+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/2453971082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d6e390 0x558c4414c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.623+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/2453971082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d661b0 0x558c44131410 secure :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7efbf40e5010 tx=0x7efbf40e5e40 comp rx=0 tx=0).stop 2024-07-31T15:01:50.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.623+0000 7efc0fdb6740 1 -- 172.21.15.67:0/2453971082 >> 172.21.15.67:0/2453971082 conn(0x558c43d83460 msgr2=0x558c44130d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:50.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.625+0000 7efc0fdb6740 1 -- 172.21.15.67:0/2453971082 shutdown_connections 2024-07-31T15:01:50.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.626+0000 7efc0fdb6740 1 -- 172.21.15.67:0/2453971082 wait complete. 2024-07-31T15:01:50.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.627+0000 7efc0fdb6740 1 -- start start 2024-07-31T15:01:50.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.627+0000 7efc0fdb6740 1 Processor -- start 2024-07-31T15:01:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.627+0000 7efc0fdb6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d6e390 0x558c441435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.627+0000 7efc0fdb6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 0x558c44143b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.627+0000 7efc0fdb6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d22da0 0x558c441b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.627+0000 7efc0fdb6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efbe40113f0 con 0x558c43d22da0 2024-07-31T15:01:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.627+0000 7efc0fdb6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efbe4010c60 con 0x558c43d6e390 2024-07-31T15:01:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.627+0000 7efc0fdb6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efbe400dec0 con 0x558c43d8f070 2024-07-31T15:01:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.628+0000 7efc0fdb6740 10 client.0 _ll_register_callbacks cb 0x7efbfc040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.628+0000 7efc037fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d22da0 0x558c441b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.628+0000 7efc037fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d22da0 0x558c441b8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58670/0 (socket says 172.21.15.67:58670) 2024-07-31T15:01:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.628+0000 7efc037fe640 1 -- 172.21.15.67:0/1728914052 learned_addr learned my addr 172.21.15.67:0/1728914052 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.628+0000 7efc02ffd640 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d6e390 0x558c441435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.628+0000 7efbfa7fc640 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 0x558c44143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.628+0000 7efc037fe640 1 -- 172.21.15.67:0/1728914052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 msgr2=0x558c44143b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.628+0000 7efc037fe640 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 0x558c44143b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.628+0000 7efc037fe640 1 -- 172.21.15.67:0/1728914052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d6e390 msgr2=0x558c441435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.628+0000 7efc037fe640 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d6e390 0x558c441435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.628+0000 7efc037fe640 1 -- 172.21.15.67:0/1728914052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c441443f0 con 0x558c43d22da0 2024-07-31T15:01:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.629+0000 7efc02ffd640 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d6e390 0x558c441435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:01:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.629+0000 7efbfa7fc640 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 0x558c44143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:01:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.629+0000 7efc037fe640 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d22da0 0x558c441b8a90 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009420 tx=0x7efbfc03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1160f98825b90cca server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.629+0000 7efbe2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:01:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.629+0000 7efbe2ffd640 1 -- 172.21.15.67:0/1728914052 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc057040 con 0x558c43d22da0 2024-07-31T15:01:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.629+0000 7efbe2ffd640 1 -- 172.21.15.67:0/1728914052 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7efbfc003a00 con 0x558c43d22da0 2024-07-31T15:01:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.629+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1728914052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf40ae8a0 con 0x558c43d22da0 2024-07-31T15:01:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.630+0000 7efbe2ffd640 1 -- 172.21.15.67:0/1728914052 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efbfc06c050 con 0x558c43d22da0 2024-07-31T15:01:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.630+0000 7efc0fdb6740 10 client.17850 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.630+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1728914052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x558c441ba080 con 0x558c43d22da0 2024-07-31T15:01:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.630+0000 7efbe2ffd640 1 -- 172.21.15.67:0/1728914052 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf40ae8a0 con 0x558c43d22da0 2024-07-31T15:01:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.630+0000 7efbe2ffd640 1 -- 172.21.15.67:0/1728914052 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7efbfc05a040 con 0x558c43d22da0 2024-07-31T15:01:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.630+0000 7efc0fdb6740 10 client.17850 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.630+0000 7efc0fdb6740 10 client.17850 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:01:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.630+0000 7efc0fdb6740 20 client.17850 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.630+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558c441ba880 0x558c441bcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.630+0000 7efc0fdb6740 4 client.17850 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:01:50.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.630+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1728914052 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c440a85a0 con 0x558c441ba880 2024-07-31T15:01:50.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.632+0000 7efc02ffd640 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558c441ba880 0x558c441bcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:50.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.633+0000 7efc02ffd640 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558c441ba880 0x558c441bcca0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91ba066d5112452d server_cookie=f599d1aeb4c6f7a6 in_seq=0 out_seq=0 2024-07-31T15:01:50.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.633+0000 7efbe2ffd640 10 client.17850 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:01:50.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.633+0000 7efbe2ffd640 1 -- 172.21.15.67:0/1728914052 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c440a85a0 con 0x558c441ba880 2024-07-31T15:01:50.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.633+0000 7efbe2ffd640 10 client.17850 handle_command_reply: tid=0 2024-07-31T15:01:50.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.634+0000 7efc0fdb6740 1 client.17850 shutdown 2024-07-31T15:01:50.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.634+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1728914052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d22da0 msgr2=0x558c441b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:50.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.634+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d22da0 0x558c441b8a90 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009420 tx=0x7efbfc03f000 comp rx=0 tx=0).stop 2024-07-31T15:01:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.634+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1728914052 shutdown_connections 2024-07-31T15:01:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.634+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c43d22da0 0x558c441b8a90 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.634+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c43d8f070 0x558c44143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.634+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558c441ba880 0x558c441bcca0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.634+0000 7efc0fdb6740 1 --2- 172.21.15.67:0/1728914052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c43d6e390 0x558c441435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.634+0000 7efbe2ffd640 0 client.17850 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:01:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.634+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1728914052 >> 172.21.15.67:0/1728914052 conn(0x558c43d83460 msgr2=0x558c44137790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.635+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1728914052 shutdown_connections 2024-07-31T15:01:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.635+0000 7efc0fdb6740 1 -- 172.21.15.67:0/1728914052 wait complete. 2024-07-31T15:01:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.636+0000 7efc0fdb6740 20 client.17850 trim_cache size 0 max 16384 2024-07-31T15:01:50.639 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:01:50.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.639+0000 7efc09865640 1 -- 172.21.15.67:0/2283894818 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efbf00b3e40 msgr2=0x7efbf00b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:50.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.639+0000 7efc09865640 1 --2- 172.21.15.67:0/2283894818 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efbf00b3e40 0x7efbf00b6350 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efbfc005ab0 tx=0x7efbfc005a20 comp rx=0 tx=0).stop 2024-07-31T15:01:50.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.639+0000 7efc09865640 1 -- 172.21.15.67:0/2283894818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc04109ec0 msgr2=0x7efc04085620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:50.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.639+0000 7efc09865640 1 --2- 172.21.15.67:0/2283894818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc04109ec0 0x7efc04085620 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efbf400bdf0 tx=0x7efbf4009f70 comp rx=0 tx=0).stop 2024-07-31T15:01:50.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.639+0000 7efc09865640 1 -- 172.21.15.67:0/2283894818 shutdown_connections 2024-07-31T15:01:50.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.639+0000 7efc09865640 1 --2- 172.21.15.67:0/2283894818 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efbf00b3e40 0x7efbf00b6350 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.639+0000 7efc09865640 1 --2- 172.21.15.67:0/2283894818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc0410cc40 0x7efc0407ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.639+0000 7efc09865640 1 --2- 172.21.15.67:0/2283894818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc0410c2f0 0x7efc0407e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.639+0000 7efc09865640 1 --2- 172.21.15.67:0/2283894818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc04109ec0 0x7efc04085620 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:50.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.639+0000 7efc09865640 1 -- 172.21.15.67:0/2283894818 >> 172.21.15.67:0/2283894818 conn(0x7efc04079100 msgr2=0x7efc0410da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:50.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.639+0000 7efc09865640 1 -- 172.21.15.67:0/2283894818 shutdown_connections 2024-07-31T15:01:50.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:50.639+0000 7efc09865640 1 -- 172.21.15.67:0/2283894818 wait complete. 2024-07-31T15:01:51.039 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:50 smithi077 ceph-mon[37327]: pgmap v733: 129 pgs: 16 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 105 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 566 KiB/s wr, 185 op/s 2024-07-31T15:01:51.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:50 smithi067 ceph-mon[32750]: pgmap v733: 129 pgs: 16 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 105 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 566 KiB/s wr, 185 op/s 2024-07-31T15:01:51.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:50 smithi073 ceph-mon[36714]: pgmap v733: 129 pgs: 16 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 105 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 566 KiB/s wr, 185 op/s 2024-07-31T15:01:51.593 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T15:01:51.965 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:01:53.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:52 smithi067 ceph-mon[32750]: pgmap v734: 129 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 111 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 586 KiB/s wr, 178 op/s 2024-07-31T15:01:53.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:52 smithi073 ceph-mon[36714]: pgmap v734: 129 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 111 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 586 KiB/s wr, 178 op/s 2024-07-31T15:01:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:52 smithi077 ceph-mon[37327]: pgmap v734: 129 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 111 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 586 KiB/s wr, 178 op/s 2024-07-31T15:01:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.339+0000 7ff7effe8640 1 -- 172.21.15.67:0/2265849424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7e8108670 msgr2=0x7ff7e810eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.339+0000 7ff7effe8640 1 --2- 172.21.15.67:0/2265849424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7e8108670 0x7ff7e810eea0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009ca0 tx=0x7ff7e002a8c0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.339+0000 7ff7effe8640 1 -- 172.21.15.67:0/2265849424 shutdown_connections 2024-07-31T15:01:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.339+0000 7ff7effe8640 1 --2- 172.21.15.67:0/2265849424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7e8108670 0x7ff7e810eea0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.339+0000 7ff7effe8640 1 --2- 172.21.15.67:0/2265849424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7e8107d20 0x7ff7e8108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.339+0000 7ff7effe8640 1 --2- 172.21.15.67:0/2265849424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e81023f0 0x7ff7e8102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.339+0000 7ff7effe8640 1 -- 172.21.15.67:0/2265849424 >> 172.21.15.67:0/2265849424 conn(0x7ff7e80fddf0 msgr2=0x7ff7e8100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.339+0000 7ff7effe8640 1 -- 172.21.15.67:0/2265849424 shutdown_connections 2024-07-31T15:01:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.339+0000 7ff7effe8640 1 -- 172.21.15.67:0/2265849424 wait complete. 2024-07-31T15:01:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.340+0000 7ff7effe8640 1 Processor -- start 2024-07-31T15:01:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.340+0000 7ff7effe8640 1 -- start start 2024-07-31T15:01:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7effe8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e81023f0 0x7ff7e819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7effe8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7e8107d20 0x7ff7e819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7effe8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7e8108670 0x7ff7e819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7effe8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81123b0 con 0x7ff7e8107d20 2024-07-31T15:01:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7effe8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8112230 con 0x7ff7e81023f0 2024-07-31T15:01:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7effe8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8112530 con 0x7ff7e8108670 2024-07-31T15:01:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7ed583640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7e8107d20 0x7ff7e819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7ee585640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7e8108670 0x7ff7e819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7edd84640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e81023f0 0x7ff7e819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7ed583640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7e8107d20 0x7ff7e819b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34748/0 (socket says 172.21.15.67:34748) 2024-07-31T15:01:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7ed583640 1 -- 172.21.15.67:0/1718275728 learned_addr learned my addr 172.21.15.67:0/1718275728 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7ed583640 1 -- 172.21.15.67:0/1718275728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7e8108670 msgr2=0x7ff7e819b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7ed583640 1 --2- 172.21.15.67:0/1718275728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7e8108670 0x7ff7e819b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7ed583640 1 -- 172.21.15.67:0/1718275728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e81023f0 msgr2=0x7ff7e819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7ed583640 1 --2- 172.21.15.67:0/1718275728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e81023f0 0x7ff7e819acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.341+0000 7ff7ed583640 1 -- 172.21.15.67:0/1718275728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e81a5f50 con 0x7ff7e8107d20 2024-07-31T15:01:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.342+0000 7ff7ed583640 1 --2- 172.21.15.67:0/1718275728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7e8107d20 0x7ff7e819b200 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800e9c0 tx=0x7ff7d800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.342+0000 7ff7deffd640 1 -- 172.21.15.67:0/1718275728 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d800ccc0 con 0x7ff7e8107d20 2024-07-31T15:01:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.342+0000 7ff7deffd640 1 -- 172.21.15.67:0/1718275728 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d80054d0 con 0x7ff7e8107d20 2024-07-31T15:01:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.343+0000 7ff7deffd640 1 -- 172.21.15.67:0/1718275728 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8010430 con 0x7ff7e8107d20 2024-07-31T15:01:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.343+0000 7ff7effe8640 1 -- 172.21.15.67:0/1718275728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e8198b10 con 0x7ff7e8107d20 2024-07-31T15:01:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.343+0000 7ff7effe8640 1 -- 172.21.15.67:0/1718275728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e8199080 con 0x7ff7e8107d20 2024-07-31T15:01:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.347+0000 7ff7f6fb9740 1 Processor -- start 2024-07-31T15:01:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.347+0000 7ff7f6fb9740 1 -- start start 2024-07-31T15:01:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.347+0000 7ff7f6fb9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd31c1b0 0x5624cd6c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.347+0000 7ff7f6fb9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2f32f0 0x5624cd6c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.347+0000 7ff7f6fb9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd2fb4d0 0x5624cd6c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.347+0000 7ff7f6fb9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5624cd6343b0 con 0x5624cd31c1b0 2024-07-31T15:01:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.347+0000 7ff7f6fb9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5624cd6238b0 con 0x5624cd2fb4d0 2024-07-31T15:01:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.347+0000 7ff7f6fb9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5624cd6278f0 con 0x5624cd2f32f0 2024-07-31T15:01:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.348+0000 7ff7ee585640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd31c1b0 0x5624cd6c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.348+0000 7ff7ee585640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd31c1b0 0x5624cd6c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34750/0 (socket says 172.21.15.67:34750) 2024-07-31T15:01:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.348+0000 7ff7ee585640 1 -- 172.21.15.67:0/2812992152 learned_addr learned my addr 172.21.15.67:0/2812992152 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.348+0000 7ff7deffd640 1 -- 172.21.15.67:0/1718275728 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff7d8004f40 con 0x7ff7e8107d20 2024-07-31T15:01:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.348+0000 7ff7edd84640 1 --2- 172.21.15.67:0/2812992152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2f32f0 0x5624cd6c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.348+0000 7ff7ed583640 1 --2- 172.21.15.67:0/2812992152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd2fb4d0 0x5624cd6c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.349+0000 7ff7ee585640 1 -- 172.21.15.67:0/2812992152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2f32f0 msgr2=0x5624cd6c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.349+0000 7ff7ee585640 1 --2- 172.21.15.67:0/2812992152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2f32f0 0x5624cd6c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.349+0000 7ff7ee585640 1 -- 172.21.15.67:0/2812992152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd2fb4d0 msgr2=0x5624cd6c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.349+0000 7ff7ee585640 1 --2- 172.21.15.67:0/2812992152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd2fb4d0 0x5624cd6c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.349+0000 7ff7ee585640 1 -- 172.21.15.67:0/2812992152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624cd5ca9a0 con 0x5624cd31c1b0 2024-07-31T15:01:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.349+0000 7ff7ee585640 1 --2- 172.21.15.67:0/2812992152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd31c1b0 0x5624cd6c74f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00098b0 tx=0x7ff7e00098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6bfd77b7edc9b3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.349+0000 7ff7d3fff640 1 -- 172.21.15.67:0/2812992152 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0031e30 con 0x5624cd31c1b0 2024-07-31T15:01:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.350+0000 7ff7d3fff640 1 -- 172.21.15.67:0/2812992152 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e0036030 con 0x5624cd31c1b0 2024-07-31T15:01:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.350+0000 7ff7d3fff640 1 -- 172.21.15.67:0/2812992152 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0004280 con 0x5624cd31c1b0 2024-07-31T15:01:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.350+0000 7ff7deffd640 1 --2- 172.21.15.67:0/1718275728 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff7d40b3dc0 0x7ff7d40b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.350+0000 7ff7deffd640 1 -- 172.21.15.67:0/1718275728 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7ff7d80e47a0 con 0x7ff7e8107d20 2024-07-31T15:01:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.350+0000 7ff7edd84640 1 --2- 172.21.15.67:0/1718275728 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff7d40b3dc0 0x7ff7d40b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.351+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/2812992152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd31c1b0 msgr2=0x5624cd6c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.351+0000 7ff7edd84640 1 --2- 172.21.15.67:0/1718275728 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff7d40b3dc0 0x7ff7d40b62d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5624cd6c6170 tx=0x7ff7e4009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.351+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/2812992152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd31c1b0 0x5624cd6c74f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00098b0 tx=0x7ff7e00098e0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.351+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/2812992152 shutdown_connections 2024-07-31T15:01:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.351+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/2812992152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd2fb4d0 0x5624cd6c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.351+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/2812992152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2f32f0 0x5624cd6c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.351+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/2812992152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd31c1b0 0x5624cd6c74f0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.351+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/2812992152 >> 172.21.15.67:0/2812992152 conn(0x5624cd310300 msgr2=0x5624cd6c53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.351+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/2812992152 shutdown_connections 2024-07-31T15:01:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.351+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/2812992152 wait complete. 2024-07-31T15:01:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.352+0000 7ff7f6fb9740 1 -- start start 2024-07-31T15:01:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.352+0000 7ff7f6fb9740 1 Processor -- start 2024-07-31T15:01:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.353+0000 7ff7f6fb9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2f32f0 0x5624cd6d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.353+0000 7ff7f6fb9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2fb4d0 0x5624cd6d3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.353+0000 7ff7f6fb9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd31c1b0 0x5624cd6dede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.353+0000 7ff7f6fb9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5624cd55dab0 con 0x5624cd2f32f0 2024-07-31T15:01:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.353+0000 7ff7f6fb9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5624cd5abcf0 con 0x5624cd31c1b0 2024-07-31T15:01:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.353+0000 7ff7f6fb9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5624cd6d5a90 con 0x5624cd2fb4d0 2024-07-31T15:01:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.353+0000 7ff7f6fb9740 10 client.0 _ll_register_callbacks cb 0x5624cd61d9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.353+0000 7ff7edd84640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2f32f0 0x5624cd6d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.353+0000 7ff7edd84640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2f32f0 0x5624cd6d3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34758/0 (socket says 172.21.15.67:34758) 2024-07-31T15:01:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.353+0000 7ff7edd84640 1 -- 172.21.15.67:0/21431996 learned_addr learned my addr 172.21.15.67:0/21431996 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.353+0000 7ff7ed583640 1 --2- 172.21.15.67:0/21431996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2fb4d0 0x5624cd6d3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.353+0000 7ff7ee585640 1 --2- 172.21.15.67:0/21431996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd31c1b0 0x5624cd6dede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.354+0000 7ff7edd84640 1 -- 172.21.15.67:0/21431996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2fb4d0 msgr2=0x5624cd6d3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.354+0000 7ff7edd84640 1 --2- 172.21.15.67:0/21431996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2fb4d0 0x5624cd6d3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.354+0000 7ff7edd84640 1 -- 172.21.15.67:0/21431996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd31c1b0 msgr2=0x5624cd6dede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.354+0000 7ff7edd84640 1 --2- 172.21.15.67:0/21431996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd31c1b0 0x5624cd6dede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.354+0000 7ff7edd84640 1 -- 172.21.15.67:0/21431996 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624cd387790 con 0x5624cd2f32f0 2024-07-31T15:01:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.354+0000 7ff7edd84640 1 --2- 172.21.15.67:0/21431996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2f32f0 0x5624cd6d3010 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40149c0 tx=0x7ff7e40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4905fe492e4efe server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.354+0000 7ff7d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:01:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.354+0000 7ff7d17fa640 1 -- 172.21.15.67:0/21431996 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4025070 con 0x5624cd2f32f0 2024-07-31T15:01:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.354+0000 7ff7d17fa640 1 -- 172.21.15.67:0/21431996 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7ff7e4021600 con 0x5624cd2f32f0 2024-07-31T15:01:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.354+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/21431996 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e0036030 con 0x5624cd2f32f0 2024-07-31T15:01:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.354+0000 7ff7d17fa640 1 -- 172.21.15.67:0/21431996 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff7e4031050 con 0x5624cd2f32f0 2024-07-31T15:01:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.355+0000 7ff7f6fb9740 10 client.17862 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.355+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/21431996 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5624cd400620 con 0x5624cd2f32f0 2024-07-31T15:01:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.355+0000 7ff7d17fa640 1 -- 172.21.15.67:0/21431996 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e0036030 con 0x5624cd2f32f0 2024-07-31T15:01:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.355+0000 7ff7d17fa640 1 -- 172.21.15.67:0/21431996 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7ff7e4020070 con 0x5624cd2f32f0 2024-07-31T15:01:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.355+0000 7ff7f6fb9740 10 client.17862 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.355+0000 7ff7f6fb9740 10 client.17862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:01:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.355+0000 7ff7f6fb9740 20 client.17862 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.355+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/21431996 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5624cd2dad80 0x5624cd6e4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.355+0000 7ff7f6fb9740 4 client.17862 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:01:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.355+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/21431996 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624cd636300 con 0x5624cd2dad80 2024-07-31T15:01:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.355+0000 7ff7ed583640 1 --2- 172.21.15.67:0/21431996 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5624cd2dad80 0x5624cd6e4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.356+0000 7ff7d17fa640 10 client.17862 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.356+0000 7ff7ed583640 1 --2- 172.21.15.67:0/21431996 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5624cd2dad80 0x5624cd6e4f90 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f9157893fd07558 server_cookie=40f6fa4c297a8734 in_seq=0 out_seq=0 2024-07-31T15:01:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7d17fa640 1 -- 172.21.15.67:0/21431996 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624cd636300 con 0x5624cd2dad80 2024-07-31T15:01:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7d17fa640 10 client.17862 handle_command_reply: tid=0 2024-07-31T15:01:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7f6fb9740 1 client.17862 shutdown 2024-07-31T15:01:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/21431996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2f32f0 msgr2=0x5624cd6d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/21431996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2f32f0 0x5624cd6d3010 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40149c0 tx=0x7ff7e40047d0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/21431996 shutdown_connections 2024-07-31T15:01:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/21431996 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5624cd2dad80 0x5624cd6e4f90 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/21431996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd31c1b0 0x5624cd6dede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/21431996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2fb4d0 0x5624cd6d3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7d17fa640 0 client.17862 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/21431996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2f32f0 0x5624cd6d3010 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/21431996 >> 172.21.15.67:0/21431996 conn(0x5624cd310300 msgr2=0x5624cd6c8520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/21431996 shutdown_connections 2024-07-31T15:01:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/21431996 wait complete. 2024-07-31T15:01:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.358+0000 7ff7f6fb9740 20 client.17862 trim_cache size 0 max 16384 2024-07-31T15:01:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.376+0000 7ff7f6fb9740 1 Processor -- start 2024-07-31T15:01:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.376+0000 7ff7f6fb9740 1 -- start start 2024-07-31T15:01:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.376+0000 7ff7f6fb9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2f32f0 0x5624cd6d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.376+0000 7ff7f6fb9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2fb4d0 0x5624cd6bb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.376+0000 7ff7f6fb9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd31c1b0 0x5624cd6debd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.376+0000 7ff7f6fb9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81ae630 con 0x5624cd2fb4d0 2024-07-31T15:01:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.376+0000 7ff7f6fb9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81b0230 con 0x5624cd31c1b0 2024-07-31T15:01:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.376+0000 7ff7f6fb9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81aed00 con 0x5624cd2f32f0 2024-07-31T15:01:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.376+0000 7ff7ed583640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2fb4d0 0x5624cd6bb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.376+0000 7ff7ed583640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2fb4d0 0x5624cd6bb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34760/0 (socket says 172.21.15.67:34760) 2024-07-31T15:01:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.376+0000 7ff7ee585640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd31c1b0 0x5624cd6debd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.376+0000 7ff7edd84640 1 --2- 172.21.15.67:0/3145711474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2f32f0 0x5624cd6d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.376+0000 7ff7ed583640 1 -- 172.21.15.67:0/3145711474 learned_addr learned my addr 172.21.15.67:0/3145711474 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.377+0000 7ff7edd84640 1 -- 172.21.15.67:0/3145711474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd31c1b0 msgr2=0x5624cd6debd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.377+0000 7ff7edd84640 1 --2- 172.21.15.67:0/3145711474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd31c1b0 0x5624cd6debd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.377+0000 7ff7edd84640 1 -- 172.21.15.67:0/3145711474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2fb4d0 msgr2=0x5624cd6bb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.377+0000 7ff7edd84640 1 --2- 172.21.15.67:0/3145711474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2fb4d0 0x5624cd6bb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.377+0000 7ff7edd84640 1 -- 172.21.15.67:0/3145711474 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624cd6ca300 con 0x5624cd2f32f0 2024-07-31T15:01:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.377+0000 7ff7edd84640 1 --2- 172.21.15.67:0/3145711474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2f32f0 0x5624cd6d32e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4000c00 tx=0x7ff7e40116e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e388559d893c138e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.377+0000 7ff7d37fe640 1 -- 172.21.15.67:0/3145711474 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4025040 con 0x5624cd2f32f0 2024-07-31T15:01:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.377+0000 7ff7d37fe640 1 -- 172.21.15.67:0/3145711474 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e401b070 con 0x5624cd2f32f0 2024-07-31T15:01:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.377+0000 7ff7d37fe640 1 -- 172.21.15.67:0/3145711474 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4021d50 con 0x5624cd2f32f0 2024-07-31T15:01:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.378+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/3145711474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2f32f0 msgr2=0x5624cd6d32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.378+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/3145711474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2f32f0 0x5624cd6d32e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4000c00 tx=0x7ff7e40116e0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.378+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/3145711474 shutdown_connections 2024-07-31T15:01:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.378+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/3145711474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd31c1b0 0x5624cd6debd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.378+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/3145711474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2fb4d0 0x5624cd6bb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.378+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/3145711474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd2f32f0 0x5624cd6d32e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.378+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/3145711474 >> 172.21.15.67:0/3145711474 conn(0x5624cd310300 msgr2=0x5624cd6bb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.378+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/3145711474 shutdown_connections 2024-07-31T15:01:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.378+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/3145711474 wait complete. 2024-07-31T15:01:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7f6fb9740 1 -- start start 2024-07-31T15:01:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7f6fb9740 1 Processor -- start 2024-07-31T15:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7f6fb9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd2f32f0 0x5624cd6d7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7f6fb9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2fb4d0 0x5624cd6d8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7f6fb9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd31c1b0 0x5624cd6d8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7f6fb9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81afb60 con 0x5624cd2fb4d0 2024-07-31T15:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7f6fb9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81af3d0 con 0x5624cd2f32f0 2024-07-31T15:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7f6fb9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81ac630 con 0x5624cd31c1b0 2024-07-31T15:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7f6fb9740 10 client.0 _ll_register_callbacks cb 0x7ff7e4014670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7ed583640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2fb4d0 0x5624cd6d8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7edd84640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd2f32f0 0x5624cd6d7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7ee585640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd31c1b0 0x5624cd6d8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7edd84640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd2f32f0 0x5624cd6d7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49716/0 (socket says 172.21.15.67:49716) 2024-07-31T15:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7ed583640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2fb4d0 0x5624cd6d8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34770/0 (socket says 172.21.15.67:34770) 2024-07-31T15:01:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.379+0000 7ff7edd84640 1 -- 172.21.15.67:0/667332596 learned_addr learned my addr 172.21.15.67:0/667332596 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.380+0000 7ff7ee585640 1 -- 172.21.15.67:0/667332596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd2f32f0 msgr2=0x5624cd6d7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.380+0000 7ff7ee585640 1 --2- 172.21.15.67:0/667332596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd2f32f0 0x5624cd6d7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.380+0000 7ff7ee585640 1 -- 172.21.15.67:0/667332596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2fb4d0 msgr2=0x5624cd6d8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.380+0000 7ff7ee585640 1 --2- 172.21.15.67:0/667332596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2fb4d0 0x5624cd6d8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.380+0000 7ff7ee585640 1 -- 172.21.15.67:0/667332596 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624cd74b760 con 0x5624cd31c1b0 2024-07-31T15:01:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.380+0000 7ff7ee585640 1 --2- 172.21.15.67:0/667332596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd31c1b0 0x5624cd6d8790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0005e30 tx=0x7ff7e002b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7656e0beaa2075d8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.380+0000 7ff7d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:01:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.380+0000 7ff7d17fa640 1 -- 172.21.15.67:0/667332596 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e004a040 con 0x5624cd31c1b0 2024-07-31T15:01:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.380+0000 7ff7d17fa640 1 -- 172.21.15.67:0/667332596 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7ff7e0031450 con 0x5624cd31c1b0 2024-07-31T15:01:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.380+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/667332596 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e401b070 con 0x5624cd31c1b0 2024-07-31T15:01:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.380+0000 7ff7d17fa640 1 -- 172.21.15.67:0/667332596 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff7e0009b20 con 0x5624cd31c1b0 2024-07-31T15:01:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.380+0000 7ff7f6fb9740 10 client.27400 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.380+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/667332596 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5624cd74c9c0 con 0x5624cd31c1b0 2024-07-31T15:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.381+0000 7ff7d17fa640 1 -- 172.21.15.67:0/667332596 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e401b070 con 0x5624cd31c1b0 2024-07-31T15:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.381+0000 7ff7d17fa640 1 -- 172.21.15.67:0/667332596 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7ff7e0043070 con 0x5624cd31c1b0 2024-07-31T15:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.381+0000 7ff7f6fb9740 10 client.27400 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.381+0000 7ff7f6fb9740 10 client.27400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.381+0000 7ff7f6fb9740 20 client.27400 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.381+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/667332596 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5624cd2b01c0 0x5624cd74ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.381+0000 7ff7f6fb9740 4 client.27400 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.381+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/667332596 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624cd631da0 con 0x5624cd2b01c0 2024-07-31T15:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.381+0000 7ff7edd84640 1 --2- 172.21.15.67:0/667332596 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5624cd2b01c0 0x5624cd74ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.381+0000 7ff7d17fa640 10 client.27400 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.382+0000 7ff7edd84640 1 --2- 172.21.15.67:0/667332596 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5624cd2b01c0 0x5624cd74ef40 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=838d6fece72ab258 server_cookie=205f08abfce8e893 in_seq=0 out_seq=0 2024-07-31T15:01:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7d17fa640 1 -- 172.21.15.67:0/667332596 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5624cd631da0 con 0x5624cd2b01c0 2024-07-31T15:01:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7d17fa640 10 client.27400 handle_command_reply: tid=0 2024-07-31T15:01:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7f6fb9740 1 client.27400 shutdown 2024-07-31T15:01:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/667332596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd31c1b0 msgr2=0x5624cd6d8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/667332596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd31c1b0 0x5624cd6d8790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0005e30 tx=0x7ff7e002b040 comp rx=0 tx=0).stop 2024-07-31T15:01:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/667332596 shutdown_connections 2024-07-31T15:01:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/667332596 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5624cd2b01c0 0x5624cd74ef40 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/667332596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cd31c1b0 0x5624cd6d8790 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/667332596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624cd2fb4d0 0x5624cd6d8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7d17fa640 0 client.27400 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7f6fb9740 1 --2- 172.21.15.67:0/667332596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cd2f32f0 0x5624cd6d7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/667332596 >> 172.21.15.67:0/667332596 conn(0x5624cd310300 msgr2=0x5624cd6d40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/667332596 shutdown_connections 2024-07-31T15:01:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7f6fb9740 1 -- 172.21.15.67:0/667332596 wait complete. 2024-07-31T15:01:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.536+0000 7ff7f6fb9740 20 client.27400 trim_cache size 0 max 16384 2024-07-31T15:01:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:01:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:01:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:01:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:01:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.537+0000 7ff7effe8640 1 -- 172.21.15.67:0/1718275728 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff7d40b3dc0 msgr2=0x7ff7d40b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.537+0000 7ff7effe8640 1 --2- 172.21.15.67:0/1718275728 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff7d40b3dc0 0x7ff7d40b62d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5624cd6c6170 tx=0x7ff7e4009110 comp rx=0 tx=0).stop 2024-07-31T15:01:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.537+0000 7ff7effe8640 1 -- 172.21.15.67:0/1718275728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7e8107d20 msgr2=0x7ff7e819b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.537+0000 7ff7effe8640 1 --2- 172.21.15.67:0/1718275728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7e8107d20 0x7ff7e819b200 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800e9c0 tx=0x7ff7d800edc0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.537+0000 7ff7effe8640 1 -- 172.21.15.67:0/1718275728 shutdown_connections 2024-07-31T15:01:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.537+0000 7ff7effe8640 1 --2- 172.21.15.67:0/1718275728 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff7d40b3dc0 0x7ff7d40b62d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.537+0000 7ff7effe8640 1 --2- 172.21.15.67:0/1718275728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7e8108670 0x7ff7e819b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.537+0000 7ff7effe8640 1 --2- 172.21.15.67:0/1718275728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7e8107d20 0x7ff7e819b200 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.537+0000 7ff7effe8640 1 --2- 172.21.15.67:0/1718275728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e81023f0 0x7ff7e819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.537+0000 7ff7effe8640 1 -- 172.21.15.67:0/1718275728 >> 172.21.15.67:0/1718275728 conn(0x7ff7e80fddf0 msgr2=0x7ff7e8107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.537+0000 7ff7effe8640 1 -- 172.21.15.67:0/1718275728 shutdown_connections 2024-07-31T15:01:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:01:54.538+0000 7ff7effe8640 1 -- 172.21.15.67:0/1718275728 wait complete. 2024-07-31T15:01:55.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:54 smithi067 ceph-mon[32750]: pgmap v735: 129 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 111 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 600 KiB/s wr, 167 op/s 2024-07-31T15:01:55.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.124+0000 7f430998a640 1 -- 172.21.15.67:0/126752444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4304102490 msgr2=0x7f43041028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.124+0000 7f430998a640 1 --2- 172.21.15.67:0/126752444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4304102490 0x7f43041028a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009860 tx=0x7f42f402a340 comp rx=0 tx=0).stop 2024-07-31T15:01:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.124+0000 7f430998a640 1 -- 172.21.15.67:0/126752444 shutdown_connections 2024-07-31T15:01:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.124+0000 7f430998a640 1 --2- 172.21.15.67:0/126752444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4304108720 0x7f430410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.124+0000 7f430998a640 1 --2- 172.21.15.67:0/126752444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4304107dd0 0x7f43041081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.124+0000 7f430998a640 1 --2- 172.21.15.67:0/126752444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4304102490 0x7f43041028a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.124+0000 7f430998a640 1 -- 172.21.15.67:0/126752444 >> 172.21.15.67:0/126752444 conn(0x7f4304079160 msgr2=0x7f4304079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.125+0000 7f430998a640 1 -- 172.21.15.67:0/126752444 shutdown_connections 2024-07-31T15:01:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.125+0000 7f430998a640 1 -- 172.21.15.67:0/126752444 wait complete. 2024-07-31T15:01:55.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.126+0000 7f430998a640 1 Processor -- start 2024-07-31T15:01:55.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.126+0000 7f430998a640 1 -- start start 2024-07-31T15:01:55.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.126+0000 7f430998a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4304102490 0x7f430419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.126+0000 7f430998a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4304107dd0 0x7f430419d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.126+0000 7f430998a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4304108720 0x7f43041a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.126+0000 7f430998a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f430406b690 con 0x7f4304108720 2024-07-31T15:01:55.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.126+0000 7f430998a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f430406b510 con 0x7f4304107dd0 2024-07-31T15:01:55.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.126+0000 7f430998a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f43041a2090 con 0x7f4304102490 2024-07-31T15:01:55.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.127+0000 7f43027fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4304107dd0 0x7f430419d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.127+0000 7f43027fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4304107dd0 0x7f430419d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49738/0 (socket says 172.21.15.67:49738) 2024-07-31T15:01:55.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.127+0000 7f43027fc640 1 -- 172.21.15.67:0/1094360453 learned_addr learned my addr 172.21.15.67:0/1094360453 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:55.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.127+0000 7f43027fc640 1 -- 172.21.15.67:0/1094360453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4304102490 msgr2=0x7f430419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:55.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.127+0000 7f43027fc640 1 --2- 172.21.15.67:0/1094360453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4304102490 0x7f430419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.127+0000 7f43027fc640 1 -- 172.21.15.67:0/1094360453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4304108720 msgr2=0x7f43041a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:55.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.127+0000 7f43027fc640 1 --2- 172.21.15.67:0/1094360453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4304108720 0x7f43041a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.127+0000 7f43027fc640 1 -- 172.21.15.67:0/1094360453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43041a2290 con 0x7f4304107dd0 2024-07-31T15:01:55.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.127+0000 7f43027fc640 1 --2- 172.21.15.67:0/1094360453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4304107dd0 0x7f430419d020 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f42f800e9a0 tx=0x7f42f800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:55.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.128+0000 7f4308988640 1 -- 172.21.15.67:0/1094360453 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f800cd80 con 0x7f4304107dd0 2024-07-31T15:01:55.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.128+0000 7f430998a640 1 -- 172.21.15.67:0/1094360453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43041a2580 con 0x7f4304107dd0 2024-07-31T15:01:55.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.128+0000 7f4308988640 1 -- 172.21.15.67:0/1094360453 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f80035f0 con 0x7f4304107dd0 2024-07-31T15:01:55.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.128+0000 7f4308988640 1 -- 172.21.15.67:0/1094360453 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8010690 con 0x7f4304107dd0 2024-07-31T15:01:55.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.128+0000 7f430998a640 1 -- 172.21.15.67:0/1094360453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43041a8ca0 con 0x7f4304107dd0 2024-07-31T15:01:55.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.131+0000 7f430fedb740 1 Processor -- start 2024-07-31T15:01:55.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.131+0000 7f430fedb740 1 -- start start 2024-07-31T15:01:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.132+0000 7f430fedb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a619c070 0x5634a6541570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.132+0000 7f430fedb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a61731b0 0x5634a6541ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.132+0000 7f43037fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a619c070 0x5634a6541570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.132+0000 7f43037fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a619c070 0x5634a6541570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34810/0 (socket says 172.21.15.67:34810) 2024-07-31T15:01:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.132+0000 7f4308988640 1 -- 172.21.15.67:0/1094360453 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f42f8010830 con 0x7f4304107dd0 2024-07-31T15:01:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.132+0000 7f43037fe640 1 -- 172.21.15.67:0/1809691227 learned_addr learned my addr 172.21.15.67:0/1809691227 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.132+0000 7f4302ffd640 1 --2- 172.21.15.67:0/1809691227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a61731b0 0x5634a6541ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.132+0000 7f430fedb740 1 --2- 172.21.15.67:0/1809691227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a617b390 0x5634a6541ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.132+0000 7f430fedb740 1 -- 172.21.15.67:0/1809691227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5634a64b7bb0 con 0x5634a619c070 2024-07-31T15:01:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.132+0000 7f430fedb740 1 -- 172.21.15.67:0/1809691227 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5634a64a70b0 con 0x5634a61731b0 2024-07-31T15:01:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.132+0000 7f430fedb740 1 -- 172.21.15.67:0/1809691227 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5634a64ab0f0 con 0x5634a617b390 2024-07-31T15:01:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.133+0000 7f43027fc640 1 --2- 172.21.15.67:0/1809691227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a617b390 0x5634a6541ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.133+0000 7f43037fe640 1 -- 172.21.15.67:0/1809691227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a617b390 msgr2=0x5634a6541ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:55.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.133+0000 7f43037fe640 1 --2- 172.21.15.67:0/1809691227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a617b390 0x5634a6541ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.133+0000 7f43037fe640 1 -- 172.21.15.67:0/1809691227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a61731b0 msgr2=0x5634a6541ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:55.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.133+0000 7f43037fe640 1 --2- 172.21.15.67:0/1809691227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a61731b0 0x5634a6541ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.133+0000 7f43037fe640 1 -- 172.21.15.67:0/1809691227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a644db90 con 0x5634a619c070 2024-07-31T15:01:55.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.134+0000 7f43037fe640 1 --2- 172.21.15.67:0/1809691227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a619c070 0x5634a6541570 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009e20 tx=0x7f42fc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8e35cc24a6044b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:55.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.134+0000 7f4308988640 1 --2- 172.21.15.67:0/1094360453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42ec0b3dc0 0x7f42ec0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.134+0000 7f4302ffd640 1 --2- 172.21.15.67:0/1094360453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42ec0b3dc0 0x7f42ec0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.134+0000 7f4308988640 1 -- 172.21.15.67:0/1094360453 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f42f80e47b0 con 0x7f4304107dd0 2024-07-31T15:01:55.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.134+0000 7f42e9ffb640 1 -- 172.21.15.67:0/1809691227 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc004d20 con 0x5634a619c070 2024-07-31T15:01:55.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.134+0000 7f42e9ffb640 1 -- 172.21.15.67:0/1809691227 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42fc00ee70 con 0x5634a619c070 2024-07-31T15:01:55.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.134+0000 7f42e9ffb640 1 -- 172.21.15.67:0/1809691227 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc018890 con 0x5634a619c070 2024-07-31T15:01:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.135+0000 7f430fedb740 1 -- 172.21.15.67:0/1809691227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a619c070 msgr2=0x5634a6541570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.135+0000 7f4302ffd640 1 --2- 172.21.15.67:0/1094360453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42ec0b3dc0 0x7f42ec0b62d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009830 tx=0x7f42f4030340 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.135+0000 7f430fedb740 1 --2- 172.21.15.67:0/1809691227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a619c070 0x5634a6541570 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009e20 tx=0x7f42fc00e720 comp rx=0 tx=0).stop 2024-07-31T15:01:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.135+0000 7f430fedb740 1 -- 172.21.15.67:0/1809691227 shutdown_connections 2024-07-31T15:01:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.135+0000 7f430fedb740 1 --2- 172.21.15.67:0/1809691227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a617b390 0x5634a6541ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.136+0000 7f430fedb740 1 --2- 172.21.15.67:0/1809691227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a61731b0 0x5634a6541ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.136+0000 7f430fedb740 1 --2- 172.21.15.67:0/1809691227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a619c070 0x5634a6541570 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.136+0000 7f430fedb740 1 -- 172.21.15.67:0/1809691227 >> 172.21.15.67:0/1809691227 conn(0x5634a6190460 msgr2=0x5634a653f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.136+0000 7f430fedb740 1 -- 172.21.15.67:0/1809691227 shutdown_connections 2024-07-31T15:01:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.136+0000 7f430fedb740 1 -- 172.21.15.67:0/1809691227 wait complete. 2024-07-31T15:01:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.136+0000 7f430fedb740 1 -- start start 2024-07-31T15:01:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.137+0000 7f430fedb740 1 Processor -- start 2024-07-31T15:01:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.137+0000 7f430fedb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a61731b0 0x5634a654d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.137+0000 7f430fedb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a617b390 0x5634a654d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.137+0000 7f430fedb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a619c070 0x5634a6559040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.137+0000 7f430fedb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5634a630a700 con 0x5634a61731b0 2024-07-31T15:01:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.137+0000 7f430fedb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5634a64499e0 con 0x5634a617b390 2024-07-31T15:01:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.137+0000 7f430fedb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5634a654fd50 con 0x5634a619c070 2024-07-31T15:01:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.137+0000 7f430fedb740 10 client.0 _ll_register_callbacks cb 0x5634a64644f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.138+0000 7f43037fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a619c070 0x5634a6559040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.138+0000 7f43037fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a619c070 0x5634a6559040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:33766/0 (socket says 172.21.15.67:33766) 2024-07-31T15:01:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.138+0000 7f43037fe640 1 -- 172.21.15.67:0/824728323 learned_addr learned my addr 172.21.15.67:0/824728323 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.138+0000 7f43027fc640 1 --2- 172.21.15.67:0/824728323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a617b390 0x5634a654d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.138+0000 7f4302ffd640 1 --2- 172.21.15.67:0/824728323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a61731b0 0x5634a654d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.138+0000 7f43037fe640 1 -- 172.21.15.67:0/824728323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a617b390 msgr2=0x5634a654d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.138+0000 7f43037fe640 1 --2- 172.21.15.67:0/824728323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a617b390 0x5634a654d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.138+0000 7f43037fe640 1 -- 172.21.15.67:0/824728323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a61731b0 msgr2=0x5634a654d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.138+0000 7f43037fe640 1 --2- 172.21.15.67:0/824728323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a61731b0 0x5634a654d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.138+0000 7f43037fe640 1 -- 172.21.15.67:0/824728323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a6207650 con 0x5634a619c070 2024-07-31T15:01:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.139+0000 7f43037fe640 1 --2- 172.21.15.67:0/824728323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a619c070 0x5634a6559040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f42fc005250 tx=0x7f42fc005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bc1f5bc86d13e95 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.139+0000 7f42e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:01:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.139+0000 7f42e2ffd640 1 -- 172.21.15.67:0/824728323 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc027060 con 0x5634a619c070 2024-07-31T15:01:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.139+0000 7f430fedb740 1 -- 172.21.15.67:0/824728323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42fc00ee70 con 0x5634a619c070 2024-07-31T15:01:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.139+0000 7f42e2ffd640 1 -- 172.21.15.67:0/824728323 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f42fc003970 con 0x5634a619c070 2024-07-31T15:01:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.140+0000 7f42e2ffd640 1 -- 172.21.15.67:0/824728323 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f42fc003bd0 con 0x5634a619c070 2024-07-31T15:01:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.140+0000 7f430fedb740 10 client.27409 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.140+0000 7f430fedb740 1 -- 172.21.15.67:0/824728323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5634a655be10 con 0x5634a619c070 2024-07-31T15:01:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.140+0000 7f42e2ffd640 1 -- 172.21.15.67:0/824728323 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42fc00ee70 con 0x5634a619c070 2024-07-31T15:01:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.140+0000 7f42e2ffd640 1 -- 172.21.15.67:0/824728323 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7f42fc0045b0 con 0x5634a619c070 2024-07-31T15:01:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.141+0000 7f430fedb740 10 client.27409 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.141+0000 7f430fedb740 10 client.27409 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:01:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.141+0000 7f430fedb740 20 client.27409 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.141+0000 7f430fedb740 1 --2- 172.21.15.67:0/824728323 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5634a615a9f0 0x5634a655f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.141+0000 7f430fedb740 4 client.27409 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:01:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.141+0000 7f430fedb740 1 -- 172.21.15.67:0/824728323 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634a64b9b00 con 0x5634a615a9f0 2024-07-31T15:01:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.141+0000 7f4302ffd640 1 --2- 172.21.15.67:0/824728323 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5634a615a9f0 0x5634a655f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:54 smithi073 ceph-mon[36714]: pgmap v735: 129 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 111 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 600 KiB/s wr, 167 op/s 2024-07-31T15:01:55.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.142+0000 7f42e2ffd640 10 client.27409 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:01:55.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.142+0000 7f4302ffd640 1 --2- 172.21.15.67:0/824728323 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5634a615a9f0 0x5634a655f170 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f05236ed89e331cc server_cookie=8b65f4597784c8e7 in_seq=0 out_seq=0 2024-07-31T15:01:55.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.144+0000 7f42e2ffd640 1 -- 172.21.15.67:0/824728323 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634a64b9b00 con 0x5634a615a9f0 2024-07-31T15:01:55.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.144+0000 7f42e2ffd640 10 client.27409 handle_command_reply: tid=0 2024-07-31T15:01:55.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.144+0000 7f430fedb740 1 client.27409 shutdown 2024-07-31T15:01:55.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.145+0000 7f430fedb740 1 -- 172.21.15.67:0/824728323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a619c070 msgr2=0x5634a6559040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:55.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.145+0000 7f430fedb740 1 --2- 172.21.15.67:0/824728323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a619c070 0x5634a6559040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f42fc005250 tx=0x7f42fc005280 comp rx=0 tx=0).stop 2024-07-31T15:01:55.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.145+0000 7f430fedb740 1 -- 172.21.15.67:0/824728323 shutdown_connections 2024-07-31T15:01:55.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.145+0000 7f430fedb740 1 --2- 172.21.15.67:0/824728323 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5634a615a9f0 0x5634a655f170 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.145+0000 7f430fedb740 1 --2- 172.21.15.67:0/824728323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a619c070 0x5634a6559040 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.145+0000 7f42e2ffd640 0 client.27409 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:01:55.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.145+0000 7f430fedb740 1 --2- 172.21.15.67:0/824728323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a617b390 0x5634a654d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.145+0000 7f430fedb740 1 --2- 172.21.15.67:0/824728323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a61731b0 0x5634a654d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.145+0000 7f430fedb740 1 -- 172.21.15.67:0/824728323 >> 172.21.15.67:0/824728323 conn(0x5634a6190460 msgr2=0x5634a653dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:55.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.145+0000 7f430fedb740 1 -- 172.21.15.67:0/824728323 shutdown_connections 2024-07-31T15:01:55.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.145+0000 7f430fedb740 1 -- 172.21.15.67:0/824728323 wait complete. 2024-07-31T15:01:55.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.145+0000 7f430fedb740 20 client.27409 trim_cache size 0 max 16384 2024-07-31T15:01:55.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.166+0000 7f430fedb740 1 Processor -- start 2024-07-31T15:01:55.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.166+0000 7f430fedb740 1 -- start start 2024-07-31T15:01:55.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.167+0000 7f430fedb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a615a9f0 0x5634a653f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.167+0000 7f430fedb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a61731b0 0x5634a6558fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.167+0000 7f430fedb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a617b390 0x5634a65594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.167+0000 7f430fedb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f43041b2c50 con 0x5634a617b390 2024-07-31T15:01:55.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.167+0000 7f430fedb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f43041b4850 con 0x5634a615a9f0 2024-07-31T15:01:55.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.167+0000 7f430fedb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f43041b3320 con 0x5634a61731b0 2024-07-31T15:01:55.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.167+0000 7f43037fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a617b390 0x5634a65594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.167+0000 7f4302ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a615a9f0 0x5634a653f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.167+0000 7f43027fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a61731b0 0x5634a6558fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.167+0000 7f4302ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a615a9f0 0x5634a653f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49770/0 (socket says 172.21.15.67:49770) 2024-07-31T15:01:55.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.167+0000 7f4302ffd640 1 -- 172.21.15.67:0/4106259091 learned_addr learned my addr 172.21.15.67:0/4106259091 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:55.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.168+0000 7f4302ffd640 1 -- 172.21.15.67:0/4106259091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a61731b0 msgr2=0x5634a6558fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:55.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.168+0000 7f4302ffd640 1 --2- 172.21.15.67:0/4106259091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a61731b0 0x5634a6558fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.168+0000 7f4302ffd640 1 -- 172.21.15.67:0/4106259091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a617b390 msgr2=0x5634a65594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:55.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.168+0000 7f4302ffd640 1 --2- 172.21.15.67:0/4106259091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a617b390 0x5634a65594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.168+0000 7f4302ffd640 1 -- 172.21.15.67:0/4106259091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a654f7e0 con 0x5634a615a9f0 2024-07-31T15:01:55.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.168+0000 7f43027fc640 1 --2- 172.21.15.67:0/4106259091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a61731b0 0x5634a6558fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:01:55.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.168+0000 7f43037fe640 1 --2- 172.21.15.67:0/4106259091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a617b390 0x5634a65594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:01:55.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.168+0000 7f4302ffd640 1 --2- 172.21.15.67:0/4106259091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a615a9f0 0x5634a653f4b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f42f402a930 tx=0x7f42f403c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9556b6770a720820 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:55.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.169+0000 7f42e97fa640 1 -- 172.21.15.67:0/4106259091 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4058070 con 0x5634a615a9f0 2024-07-31T15:01:55.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.169+0000 7f42e97fa640 1 -- 172.21.15.67:0/4106259091 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f40035f0 con 0x5634a615a9f0 2024-07-31T15:01:55.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.169+0000 7f42e97fa640 1 -- 172.21.15.67:0/4106259091 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4037040 con 0x5634a615a9f0 2024-07-31T15:01:55.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.170+0000 7f430fedb740 1 -- 172.21.15.67:0/4106259091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a615a9f0 msgr2=0x5634a653f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:55.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.170+0000 7f430fedb740 1 --2- 172.21.15.67:0/4106259091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a615a9f0 0x5634a653f4b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f42f402a930 tx=0x7f42f403c000 comp rx=0 tx=0).stop 2024-07-31T15:01:55.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.170+0000 7f430fedb740 1 -- 172.21.15.67:0/4106259091 shutdown_connections 2024-07-31T15:01:55.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.170+0000 7f430fedb740 1 --2- 172.21.15.67:0/4106259091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a617b390 0x5634a65594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.170+0000 7f430fedb740 1 --2- 172.21.15.67:0/4106259091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a61731b0 0x5634a6558fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.170+0000 7f430fedb740 1 --2- 172.21.15.67:0/4106259091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a615a9f0 0x5634a653f4b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.170+0000 7f430fedb740 1 -- 172.21.15.67:0/4106259091 >> 172.21.15.67:0/4106259091 conn(0x5634a6190460 msgr2=0x5634a653eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:55.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.170+0000 7f430fedb740 1 -- 172.21.15.67:0/4106259091 shutdown_connections 2024-07-31T15:01:55.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.171+0000 7f430fedb740 1 -- 172.21.15.67:0/4106259091 wait complete. 2024-07-31T15:01:55.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.171+0000 7f430fedb740 1 -- start start 2024-07-31T15:01:55.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.171+0000 7f430fedb740 1 Processor -- start 2024-07-31T15:01:55.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.172+0000 7f430fedb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a615a9f0 0x5634a6562840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.172+0000 7f430fedb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a61731b0 0x5634a655fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.172+0000 7f430fedb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a617b390 0x5634a6560250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.172+0000 7f430fedb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f43041b4180 con 0x5634a617b390 2024-07-31T15:01:55.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.172+0000 7f430fedb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f43041b39f0 con 0x5634a61731b0 2024-07-31T15:01:55.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.172+0000 7f430fedb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f43041b0c50 con 0x5634a615a9f0 2024-07-31T15:01:55.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.172+0000 7f43027fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a61731b0 0x5634a655fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.172+0000 7f43027fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a61731b0 0x5634a655fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49780/0 (socket says 172.21.15.67:49780) 2024-07-31T15:01:55.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.172+0000 7f43027fc640 1 -- 172.21.15.67:0/2442246046 learned_addr learned my addr 172.21.15.67:0/2442246046 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:55.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.172+0000 7f430fedb740 10 client.0 _ll_register_callbacks cb 0x7f42f4035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:55.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.172+0000 7f43037fe640 1 --2- 172.21.15.67:0/2442246046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a617b390 0x5634a6560250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.173+0000 7f4302ffd640 1 --2- 172.21.15.67:0/2442246046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a615a9f0 0x5634a6562840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.173+0000 7f43037fe640 1 -- 172.21.15.67:0/2442246046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a615a9f0 msgr2=0x5634a6562840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:55.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.173+0000 7f43037fe640 1 --2- 172.21.15.67:0/2442246046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a615a9f0 0x5634a6562840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.173+0000 7f43037fe640 1 -- 172.21.15.67:0/2442246046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a61731b0 msgr2=0x5634a655fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:55.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.173+0000 7f43037fe640 1 --2- 172.21.15.67:0/2442246046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a61731b0 0x5634a655fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.173+0000 7f43037fe640 1 -- 172.21.15.67:0/2442246046 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a65609c0 con 0x5634a617b390 2024-07-31T15:01:55.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.173+0000 7f43037fe640 1 --2- 172.21.15.67:0/2442246046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a617b390 0x5634a6560250 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00ec20 tx=0x7f42fc0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd949b4a588b8b5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:55.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.173+0000 7f42e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:01:55.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.174+0000 7f42e2ffd640 1 -- 172.21.15.67:0/2442246046 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc031080 con 0x5634a617b390 2024-07-31T15:01:55.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.174+0000 7f42e2ffd640 1 -- 172.21.15.67:0/2442246046 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7f42fc022870 con 0x5634a617b390 2024-07-31T15:01:55.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.174+0000 7f430fedb740 1 -- 172.21.15.67:0/2442246046 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f40035f0 con 0x5634a617b390 2024-07-31T15:01:55.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.174+0000 7f42e2ffd640 1 -- 172.21.15.67:0/2442246046 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f42fc044050 con 0x5634a617b390 2024-07-31T15:01:55.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.174+0000 7f430fedb740 10 client.17883 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:55.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.174+0000 7f430fedb740 1 -- 172.21.15.67:0/2442246046 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5634a65c75c0 con 0x5634a617b390 2024-07-31T15:01:55.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.175+0000 7f42e2ffd640 1 -- 172.21.15.67:0/2442246046 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42fc0052f0 con 0x5634a617b390 2024-07-31T15:01:55.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.175+0000 7f42e2ffd640 1 -- 172.21.15.67:0/2442246046 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7f42fc040510 con 0x5634a617b390 2024-07-31T15:01:55.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.175+0000 7f430fedb740 10 client.17883 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:55.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.175+0000 7f430fedb740 10 client.17883 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:01:55.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.175+0000 7f430fedb740 20 client.17883 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:55.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.175+0000 7f430fedb740 1 --2- 172.21.15.67:0/2442246046 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5634a619c070 0x5634a65c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:55.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.175+0000 7f430fedb740 4 client.17883 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:01:55.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.175+0000 7f430fedb740 1 -- 172.21.15.67:0/2442246046 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f42f40035f0 con 0x5634a619c070 2024-07-31T15:01:55.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.176+0000 7f4302ffd640 1 --2- 172.21.15.67:0/2442246046 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5634a619c070 0x5634a65c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:55.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.177+0000 7f42e2ffd640 10 client.17883 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:01:55.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.177+0000 7f4302ffd640 1 --2- 172.21.15.67:0/2442246046 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5634a619c070 0x5634a65c9ce0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50b4d7eddec68e2e server_cookie=44845e8a0aa53ec5 in_seq=0 out_seq=0 2024-07-31T15:01:55.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.178+0000 7f42e2ffd640 1 -- 172.21.15.67:0/2442246046 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f42f40035f0 con 0x5634a619c070 2024-07-31T15:01:55.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.178+0000 7f42e2ffd640 10 client.17883 handle_command_reply: tid=0 2024-07-31T15:01:55.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.178+0000 7f430fedb740 1 client.17883 shutdown 2024-07-31T15:01:55.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.179+0000 7f430fedb740 1 -- 172.21.15.67:0/2442246046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a617b390 msgr2=0x5634a6560250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:55.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.179+0000 7f430fedb740 1 --2- 172.21.15.67:0/2442246046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a617b390 0x5634a6560250 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00ec20 tx=0x7f42fc0058e0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.179+0000 7f430fedb740 1 -- 172.21.15.67:0/2442246046 shutdown_connections 2024-07-31T15:01:55.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.179+0000 7f430fedb740 1 --2- 172.21.15.67:0/2442246046 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5634a619c070 0x5634a65c9ce0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.179+0000 7f430fedb740 1 --2- 172.21.15.67:0/2442246046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5634a617b390 0x5634a6560250 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.179+0000 7f430fedb740 1 --2- 172.21.15.67:0/2442246046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634a61731b0 0x5634a655fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.179+0000 7f430fedb740 1 --2- 172.21.15.67:0/2442246046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5634a615a9f0 0x5634a6562840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.179+0000 7f430fedb740 1 -- 172.21.15.67:0/2442246046 >> 172.21.15.67:0/2442246046 conn(0x5634a6190460 msgr2=0x5634a6546680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:55.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.179+0000 7f42e2ffd640 0 client.17883 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:01:55.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.179+0000 7f430fedb740 1 -- 172.21.15.67:0/2442246046 shutdown_connections 2024-07-31T15:01:55.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.179+0000 7f430fedb740 1 -- 172.21.15.67:0/2442246046 wait complete. 2024-07-31T15:01:55.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.180+0000 7f430fedb740 20 client.17883 trim_cache size 0 max 16384 2024-07-31T15:01:55.182 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:01:55.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.180+0000 7f430998a640 1 -- 172.21.15.67:0/1094360453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42ec0b3dc0 msgr2=0x7f42ec0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:55.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.180+0000 7f430998a640 1 --2- 172.21.15.67:0/1094360453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42ec0b3dc0 0x7f42ec0b62d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009830 tx=0x7f42f4030340 comp rx=0 tx=0).stop 2024-07-31T15:01:55.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.180+0000 7f430998a640 1 -- 172.21.15.67:0/1094360453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4304107dd0 msgr2=0x7f430419d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:55.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.180+0000 7f430998a640 1 --2- 172.21.15.67:0/1094360453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4304107dd0 0x7f430419d020 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f42f800e9a0 tx=0x7f42f800ee60 comp rx=0 tx=0).stop 2024-07-31T15:01:55.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.181+0000 7f430998a640 1 -- 172.21.15.67:0/1094360453 shutdown_connections 2024-07-31T15:01:55.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.181+0000 7f430998a640 1 --2- 172.21.15.67:0/1094360453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42ec0b3dc0 0x7f42ec0b62d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.181+0000 7f430998a640 1 --2- 172.21.15.67:0/1094360453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4304108720 0x7f43041a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.181+0000 7f430998a640 1 --2- 172.21.15.67:0/1094360453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4304107dd0 0x7f430419d020 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.181+0000 7f430998a640 1 --2- 172.21.15.67:0/1094360453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4304102490 0x7f430419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:55.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.181+0000 7f430998a640 1 -- 172.21.15.67:0/1094360453 >> 172.21.15.67:0/1094360453 conn(0x7f4304079160 msgr2=0x7f4304101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:55.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.181+0000 7f430998a640 1 -- 172.21.15.67:0/1094360453 shutdown_connections 2024-07-31T15:01:55.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:55.181+0000 7f430998a640 1 -- 172.21.15.67:0/1094360453 wait complete. 2024-07-31T15:01:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:54 smithi077 ceph-mon[37327]: pgmap v735: 129 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 111 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 600 KiB/s wr, 167 op/s 2024-07-31T15:01:56.118 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T15:01:56.464 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:01:56.848 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:56 smithi067 ceph-mon[32750]: pgmap v736: 129 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 113 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 608 KiB/s wr, 157 op/s 2024-07-31T15:01:57.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:56 smithi073 ceph-mon[36714]: pgmap v736: 129 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 113 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 608 KiB/s wr, 157 op/s 2024-07-31T15:01:57.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:56 smithi077 ceph-mon[37327]: pgmap v736: 129 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 113 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 608 KiB/s wr, 157 op/s 2024-07-31T15:01:58.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.723+0000 7fb6dee8f640 1 -- 172.21.15.67:0/3515354831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d810d890 msgr2=0x7fb6d810fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:58.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.723+0000 7fb6dee8f640 1 --2- 172.21.15.67:0/3515354831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d810d890 0x7fb6d810fd80 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009ca0 tx=0x7fb6d002a8c0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.723+0000 7fb6dee8f640 1 -- 172.21.15.67:0/3515354831 shutdown_connections 2024-07-31T15:01:58.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.723+0000 7fb6dee8f640 1 --2- 172.21.15.67:0/3515354831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d810d890 0x7fb6d810fd80 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.723+0000 7fb6dee8f640 1 --2- 172.21.15.67:0/3515354831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d8104db0 0x7fb6d810d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.723+0000 7fb6dee8f640 1 --2- 172.21.15.67:0/3515354831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6d8104460 0x7fb6d8104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.723+0000 7fb6dee8f640 1 -- 172.21.15.67:0/3515354831 >> 172.21.15.67:0/3515354831 conn(0x7fb6d8079210 msgr2=0x7fb6d8079620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:58.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.723+0000 7fb6dee8f640 1 -- 172.21.15.67:0/3515354831 shutdown_connections 2024-07-31T15:01:58.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.723+0000 7fb6dee8f640 1 -- 172.21.15.67:0/3515354831 wait complete. 2024-07-31T15:01:58.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.724+0000 7fb6dee8f640 1 Processor -- start 2024-07-31T15:01:58.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.724+0000 7fb6dee8f640 1 -- start start 2024-07-31T15:01:58.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.724+0000 7fb6dee8f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6d8104460 0x7fb6d81021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.724+0000 7fb6dee8f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d8104db0 0x7fb6d81026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.724+0000 7fb6dee8f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d810d890 0x7fb6d8102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.724+0000 7fb6dee8f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d81134f0 con 0x7fb6d810d890 2024-07-31T15:01:58.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.724+0000 7fb6dee8f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d8113370 con 0x7fb6d8104db0 2024-07-31T15:01:58.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.724+0000 7fb6dee8f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d8113670 con 0x7fb6d8104460 2024-07-31T15:01:58.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.724+0000 7fb6dd42c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d810d890 0x7fb6d8102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.724+0000 7fb6dd42c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d810d890 0x7fb6d8102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34856/0 (socket says 172.21.15.67:34856) 2024-07-31T15:01:58.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.724+0000 7fb6dd42c640 1 -- 172.21.15.67:0/2569272920 learned_addr learned my addr 172.21.15.67:0/2569272920 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:58.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.724+0000 7fb6dcc2b640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6d8104460 0x7fb6d81021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.724+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d8104db0 0x7fb6d81026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.725+0000 7fb6dd42c640 1 -- 172.21.15.67:0/2569272920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6d8104460 msgr2=0x7fb6d81021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:58.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.725+0000 7fb6dd42c640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6d8104460 0x7fb6d81021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.725+0000 7fb6dd42c640 1 -- 172.21.15.67:0/2569272920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d8104db0 msgr2=0x7fb6d81026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:58.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.725+0000 7fb6dd42c640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d8104db0 0x7fb6d81026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.725+0000 7fb6dd42c640 1 -- 172.21.15.67:0/2569272920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d81a8160 con 0x7fb6d810d890 2024-07-31T15:01:58.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.725+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d8104db0 0x7fb6d81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:01:58.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.725+0000 7fb6dcc2b640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6d8104460 0x7fb6d81021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:01:58.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.725+0000 7fb6dd42c640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d810d890 0x7fb6d8102c20 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005cc0 tx=0x7fb6d0031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:58.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.725+0000 7fb6d5ffb640 1 -- 172.21.15.67:0/2569272920 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0003b70 con 0x7fb6d810d890 2024-07-31T15:01:58.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.725+0000 7fb6dee8f640 1 -- 172.21.15.67:0/2569272920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d81a83f0 con 0x7fb6d810d890 2024-07-31T15:01:58.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.725+0000 7fb6d5ffb640 1 -- 172.21.15.67:0/2569272920 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6d00318f0 con 0x7fb6d810d890 2024-07-31T15:01:58.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.725+0000 7fb6d5ffb640 1 -- 172.21.15.67:0/2569272920 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d002f950 con 0x7fb6d810d890 2024-07-31T15:01:58.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.725+0000 7fb6dee8f640 1 -- 172.21.15.67:0/2569272920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d81a8980 con 0x7fb6d810d890 2024-07-31T15:01:58.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.729+0000 7fb6e53e0740 1 Processor -- start 2024-07-31T15:01:58.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.729+0000 7fb6d5ffb640 1 -- 172.21.15.67:0/2569272920 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb6d002fb50 con 0x7fb6d810d890 2024-07-31T15:01:58.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.729+0000 7fb6e53e0740 1 -- start start 2024-07-31T15:01:58.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.729+0000 7fb6e53e0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa513070 0x562cfa8b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.729+0000 7fb6e53e0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4ea1b0 0x562cfa8b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.729+0000 7fb6e53e0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4f2390 0x562cfa8b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.729+0000 7fb6e53e0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562cfa82ebb0 con 0x562cfa513070 2024-07-31T15:01:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.729+0000 7fb6e53e0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562cfa81e0b0 con 0x562cfa4ea1b0 2024-07-31T15:01:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.729+0000 7fb6e53e0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562cfa8220f0 con 0x562cfa4f2390 2024-07-31T15:01:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.730+0000 7fb6d7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa513070 0x562cfa8b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.730+0000 7fb6d7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa513070 0x562cfa8b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34868/0 (socket says 172.21.15.67:34868) 2024-07-31T15:01:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.730+0000 7fb6d7fff640 1 -- 172.21.15.67:0/3900356126 learned_addr learned my addr 172.21.15.67:0/3900356126 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.730+0000 7fb6dcc2b640 1 --2- 172.21.15.67:0/3900356126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4ea1b0 0x562cfa8b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.730+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/3900356126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4f2390 0x562cfa8b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.730+0000 7fb6d5ffb640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6c80b3d40 0x7fb6c80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.730+0000 7fb6d7fff640 1 -- 172.21.15.67:0/3900356126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4f2390 msgr2=0x562cfa8b8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.730+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/3900356126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4f2390 0x562cfa8b8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.730+0000 7fb6d7fff640 1 -- 172.21.15.67:0/3900356126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4ea1b0 msgr2=0x562cfa8b8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.730+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/3900356126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4ea1b0 0x562cfa8b8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.730+0000 7fb6d7fff640 1 -- 172.21.15.67:0/3900356126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cfa7c4b90 con 0x562cfa513070 2024-07-31T15:01:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.731+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/3900356126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa513070 0x562cfa8b8570 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4013780 tx=0x7fb6c4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19278ec283b71bc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.731+0000 7fb6d5ffb640 1 -- 172.21.15.67:0/2569272920 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7fb6d0105700 con 0x7fb6d810d890 2024-07-31T15:01:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.731+0000 7fb6c2ffd640 1 -- 172.21.15.67:0/3900356126 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4013080 con 0x562cfa513070 2024-07-31T15:01:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.731+0000 7fb6dd42c640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6c80b3d40 0x7fb6c80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.731+0000 7fb6c2ffd640 1 -- 172.21.15.67:0/3900356126 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6c4013220 con 0x562cfa513070 2024-07-31T15:01:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.731+0000 7fb6c2ffd640 1 -- 172.21.15.67:0/3900356126 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40035f0 con 0x562cfa513070 2024-07-31T15:01:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.732+0000 7fb6e53e0740 1 -- 172.21.15.67:0/3900356126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa513070 msgr2=0x562cfa8b8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.732+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/3900356126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa513070 0x562cfa8b8570 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4013780 tx=0x7fb6c4009750 comp rx=0 tx=0).stop 2024-07-31T15:01:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.732+0000 7fb6dd42c640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6c80b3d40 0x7fb6c80b6250 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0102bf0 tx=0x7fb6d002a8c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:58.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.732+0000 7fb6e53e0740 1 -- 172.21.15.67:0/3900356126 shutdown_connections 2024-07-31T15:01:58.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.732+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/3900356126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4f2390 0x562cfa8b8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.732+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/3900356126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4ea1b0 0x562cfa8b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.732+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/3900356126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa513070 0x562cfa8b8570 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.732+0000 7fb6e53e0740 1 -- 172.21.15.67:0/3900356126 >> 172.21.15.67:0/3900356126 conn(0x562cfa507460 msgr2=0x562cfa8b6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:58.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.732+0000 7fb6e53e0740 1 -- 172.21.15.67:0/3900356126 shutdown_connections 2024-07-31T15:01:58.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.732+0000 7fb6e53e0740 1 -- 172.21.15.67:0/3900356126 wait complete. 2024-07-31T15:01:58.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.732+0000 7fb6e53e0740 1 -- start start 2024-07-31T15:01:58.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.732+0000 7fb6e53e0740 1 Processor -- start 2024-07-31T15:01:58.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.733+0000 7fb6e53e0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4ea1b0 0x562cfa8c4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.733+0000 7fb6e53e0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4f2390 0x562cfa8c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.733+0000 7fb6e53e0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa513070 0x562cfa8d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.733+0000 7fb6e53e0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562cfa681700 con 0x562cfa4f2390 2024-07-31T15:01:58.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.733+0000 7fb6e53e0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562cfa7c09e0 con 0x562cfa513070 2024-07-31T15:01:58.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.733+0000 7fb6e53e0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562cfa8c6d50 con 0x562cfa4ea1b0 2024-07-31T15:01:58.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.733+0000 7fb6d7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4ea1b0 0x562cfa8c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.733+0000 7fb6e53e0740 10 client.0 _ll_register_callbacks cb 0x562cfa7db4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.733+0000 7fb6d7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4ea1b0 0x562cfa8c4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:33804/0 (socket says 172.21.15.67:33804) 2024-07-31T15:01:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.733+0000 7fb6d7fff640 1 -- 172.21.15.67:0/4235620219 learned_addr learned my addr 172.21.15.67:0/4235620219 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.733+0000 7fb6dcc2b640 1 --2- 172.21.15.67:0/4235620219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4f2390 0x562cfa8c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.733+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/4235620219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa513070 0x562cfa8d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.733+0000 7fb6d7fff640 1 -- 172.21.15.67:0/4235620219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa513070 msgr2=0x562cfa8d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.734+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/4235620219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa513070 0x562cfa8d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.734+0000 7fb6d7fff640 1 -- 172.21.15.67:0/4235620219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4f2390 msgr2=0x562cfa8c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.734+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/4235620219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4f2390 0x562cfa8c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.734+0000 7fb6d7fff640 1 -- 172.21.15.67:0/4235620219 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cfa57e650 con 0x562cfa4ea1b0 2024-07-31T15:01:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.734+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/4235620219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4ea1b0 0x562cfa8c4210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009e90 tx=0x7fb6c400fba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b501764e11f8e367 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.734+0000 7fb6b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:01:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.734+0000 7fb6b7fff640 1 -- 172.21.15.67:0/4235620219 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c400fea0 con 0x562cfa4ea1b0 2024-07-31T15:01:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.734+0000 7fb6e53e0740 1 -- 172.21.15.67:0/4235620219 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c4013220 con 0x562cfa4ea1b0 2024-07-31T15:01:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.734+0000 7fb6b7fff640 1 -- 172.21.15.67:0/4235620219 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7fb6c4005230 con 0x562cfa4ea1b0 2024-07-31T15:01:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.734+0000 7fb6b7fff640 1 -- 172.21.15.67:0/4235620219 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fb6c4030050 con 0x562cfa4ea1b0 2024-07-31T15:01:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.734+0000 7fb6e53e0740 10 client.27427 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.734+0000 7fb6e53e0740 1 -- 172.21.15.67:0/4235620219 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562cfa8d2e20 con 0x562cfa4ea1b0 2024-07-31T15:01:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.734+0000 7fb6b7fff640 1 -- 172.21.15.67:0/4235620219 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6c4013220 con 0x562cfa4ea1b0 2024-07-31T15:01:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.735+0000 7fb6b7fff640 1 -- 172.21.15.67:0/4235620219 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7fb6c4004850 con 0x562cfa4ea1b0 2024-07-31T15:01:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.735+0000 7fb6e53e0740 10 client.27427 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.735+0000 7fb6e53e0740 10 client.27427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:01:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.735+0000 7fb6e53e0740 20 client.27427 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.735+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/4235620219 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562cfa4d19f0 0x562cfa8d6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.735+0000 7fb6e53e0740 4 client.27427 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:01:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.735+0000 7fb6e53e0740 1 -- 172.21.15.67:0/4235620219 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cfa830b00 con 0x562cfa4d19f0 2024-07-31T15:01:58.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.735+0000 7fb6dcc2b640 1 --2- 172.21.15.67:0/4235620219 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562cfa4d19f0 0x562cfa8d6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.736+0000 7fb6b7fff640 10 client.27427 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:58.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.736+0000 7fb6dcc2b640 1 --2- 172.21.15.67:0/4235620219 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562cfa4d19f0 0x562cfa8d6180 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0212329ded2f5a server_cookie=46277a4c7affce2c in_seq=0 out_seq=0 2024-07-31T15:01:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.738+0000 7fb6b7fff640 1 -- 172.21.15.67:0/4235620219 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cfa830b00 con 0x562cfa4d19f0 2024-07-31T15:01:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.738+0000 7fb6b7fff640 10 client.27427 handle_command_reply: tid=0 2024-07-31T15:01:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.738+0000 7fb6e53e0740 1 client.27427 shutdown 2024-07-31T15:01:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.738+0000 7fb6e53e0740 1 -- 172.21.15.67:0/4235620219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4ea1b0 msgr2=0x562cfa8c4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.738+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/4235620219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4ea1b0 0x562cfa8c4210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009e90 tx=0x7fb6c400fba0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.738+0000 7fb6e53e0740 1 -- 172.21.15.67:0/4235620219 shutdown_connections 2024-07-31T15:01:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.738+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/4235620219 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562cfa4d19f0 0x562cfa8d6180 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.738+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/4235620219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa513070 0x562cfa8d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.738+0000 7fb6b7fff640 0 client.27427 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.738+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/4235620219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4f2390 0x562cfa8c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.738+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/4235620219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4ea1b0 0x562cfa8c4210 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.738+0000 7fb6e53e0740 1 -- 172.21.15.67:0/4235620219 >> 172.21.15.67:0/4235620219 conn(0x562cfa507460 msgr2=0x562cfa8bb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.738+0000 7fb6e53e0740 1 -- 172.21.15.67:0/4235620219 shutdown_connections 2024-07-31T15:01:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.738+0000 7fb6e53e0740 1 -- 172.21.15.67:0/4235620219 wait complete. 2024-07-31T15:01:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.739+0000 7fb6e53e0740 20 client.27427 trim_cache size 0 max 16384 2024-07-31T15:01:58.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.758+0000 7fb6e53e0740 1 Processor -- start 2024-07-31T15:01:58.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.758+0000 7fb6e53e0740 1 -- start start 2024-07-31T15:01:58.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.758+0000 7fb6e53e0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4d19f0 0x562cfa8bbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.758+0000 7fb6e53e0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4ea1b0 0x562cfa8cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.758+0000 7fb6e53e0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4f2390 0x562cfa8bd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.758+0000 7fb6e53e0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d81ae720 con 0x562cfa4f2390 2024-07-31T15:01:58.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.758+0000 7fb6e53e0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d81b0320 con 0x562cfa4ea1b0 2024-07-31T15:01:58.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.758+0000 7fb6e53e0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d81aedf0 con 0x562cfa4d19f0 2024-07-31T15:01:58.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.759+0000 7fb6d7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4d19f0 0x562cfa8bbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.759+0000 7fb6dcc2b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4ea1b0 0x562cfa8cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.759+0000 7fb6dcc2b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4ea1b0 0x562cfa8cfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49826/0 (socket says 172.21.15.67:49826) 2024-07-31T15:01:58.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.759+0000 7fb6dcc2b640 1 -- 172.21.15.67:0/2589926132 learned_addr learned my addr 172.21.15.67:0/2589926132 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:58.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.759+0000 7fb6d7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4d19f0 0x562cfa8bbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:33810/0 (socket says 172.21.15.67:33810) 2024-07-31T15:01:58.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.759+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/2589926132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4f2390 0x562cfa8bd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.759+0000 7fb6d7fff640 1 -- 172.21.15.67:0/2589926132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4ea1b0 msgr2=0x562cfa8cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:58.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.759+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/2589926132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4ea1b0 0x562cfa8cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.759+0000 7fb6d7fff640 1 -- 172.21.15.67:0/2589926132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4f2390 msgr2=0x562cfa8bd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:58.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.759+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/2589926132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4f2390 0x562cfa8bd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.759+0000 7fb6d7fff640 1 -- 172.21.15.67:0/2589926132 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cfa8bf8a0 con 0x562cfa4d19f0 2024-07-31T15:01:58.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.759+0000 7fb6dcc2b640 1 --2- 172.21.15.67:0/2589926132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4ea1b0 0x562cfa8cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:01:58.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.759+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/2589926132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4d19f0 0x562cfa8bbd80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009e60 tx=0x7fb6c400fba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ae5e4902a6ac2fc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:58.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.760+0000 7fb6c27fc640 1 -- 172.21.15.67:0/2589926132 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c400fe70 con 0x562cfa4d19f0 2024-07-31T15:01:58.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.760+0000 7fb6c27fc640 1 -- 172.21.15.67:0/2589926132 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6c4021460 con 0x562cfa4d19f0 2024-07-31T15:01:58.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.760+0000 7fb6c27fc640 1 -- 172.21.15.67:0/2589926132 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4011ad0 con 0x562cfa4d19f0 2024-07-31T15:01:58.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.761+0000 7fb6e53e0740 1 -- 172.21.15.67:0/2589926132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4d19f0 msgr2=0x562cfa8bbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:58.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.761+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/2589926132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4d19f0 0x562cfa8bbd80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009e60 tx=0x7fb6c400fba0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.761+0000 7fb6e53e0740 1 -- 172.21.15.67:0/2589926132 shutdown_connections 2024-07-31T15:01:58.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.761+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/2589926132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4f2390 0x562cfa8bd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.761+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/2589926132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4ea1b0 0x562cfa8cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.761+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/2589926132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4d19f0 0x562cfa8bbd80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.761+0000 7fb6e53e0740 1 -- 172.21.15.67:0/2589926132 >> 172.21.15.67:0/2589926132 conn(0x562cfa507460 msgr2=0x562cfa8bb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:58.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.761+0000 7fb6e53e0740 1 -- 172.21.15.67:0/2589926132 shutdown_connections 2024-07-31T15:01:58.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.761+0000 7fb6e53e0740 1 -- 172.21.15.67:0/2589926132 wait complete. 2024-07-31T15:01:58.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.761+0000 7fb6e53e0740 1 -- start start 2024-07-31T15:01:58.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.761+0000 7fb6e53e0740 1 Processor -- start 2024-07-31T15:01:58.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6e53e0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4d19f0 0x562cfa8cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6e53e0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4ea1b0 0x562cfa8daf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6e53e0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4f2390 0x562cfa8db460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6e53e0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d81afc50 con 0x562cfa4ea1b0 2024-07-31T15:01:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6e53e0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d81af4c0 con 0x562cfa4f2390 2024-07-31T15:01:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6e53e0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d81ac780 con 0x562cfa4d19f0 2024-07-31T15:01:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6e53e0740 10 client.0 _ll_register_callbacks cb 0x7fb6cc009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:01:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6dcc2b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4ea1b0 0x562cfa8daf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6d7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4d19f0 0x562cfa8cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6d7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4d19f0 0x562cfa8cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:33818/0 (socket says 172.21.15.67:33818) 2024-07-31T15:01:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6d7fff640 1 -- 172.21.15.67:0/1260112296 learned_addr learned my addr 172.21.15.67:0/1260112296 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:01:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6dcc2b640 1 --2- 172.21.15.67:0/1260112296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4ea1b0 0x562cfa8daf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34908/0 (socket says 172.21.15.67:34908) 2024-07-31T15:01:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/1260112296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4f2390 0x562cfa8db460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6d7fff640 1 -- 172.21.15.67:0/1260112296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4f2390 msgr2=0x562cfa8db460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/1260112296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4f2390 0x562cfa8db460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6d7fff640 1 -- 172.21.15.67:0/1260112296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4ea1b0 msgr2=0x562cfa8daf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:01:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/1260112296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4ea1b0 0x562cfa8daf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.762+0000 7fb6d7fff640 1 -- 172.21.15.67:0/1260112296 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cfa8db9a0 con 0x562cfa4d19f0 2024-07-31T15:01:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.763+0000 7fb6dcc2b640 1 --2- 172.21.15.67:0/1260112296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4ea1b0 0x562cfa8daf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:01:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.763+0000 7fb6d7fff640 1 --2- 172.21.15.67:0/1260112296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4d19f0 0x562cfa8cc3f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400f730 tx=0x7fb6c4013300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ccd819b5cbb5ed4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:01:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.763+0000 7fb6b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:01:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.763+0000 7fb6b7fff640 1 -- 172.21.15.67:0/1260112296 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4011a70 con 0x562cfa4d19f0 2024-07-31T15:01:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.763+0000 7fb6b7fff640 1 -- 172.21.15.67:0/1260112296 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9499+0+0 (secure 0 0 0) 0x7fb6c4004680 con 0x562cfa4d19f0 2024-07-31T15:01:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.763+0000 7fb6e53e0740 1 -- 172.21.15.67:0/1260112296 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c4021460 con 0x562cfa4d19f0 2024-07-31T15:01:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.763+0000 7fb6b7fff640 1 -- 172.21.15.67:0/1260112296 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fb6c4004880 con 0x562cfa4d19f0 2024-07-31T15:01:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.763+0000 7fb6e53e0740 10 client.27433 fetch_fsmap learned FSMap version 58 2024-07-31T15:01:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.763+0000 7fb6e53e0740 1 -- 172.21.15.67:0/1260112296 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562cfa8cabf0 con 0x562cfa4d19f0 2024-07-31T15:01:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.763+0000 7fb6b7fff640 1 -- 172.21.15.67:0/1260112296 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6c4021460 con 0x562cfa4d19f0 2024-07-31T15:01:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.764+0000 7fb6b7fff640 1 -- 172.21.15.67:0/1260112296 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7fb6c4004f90 con 0x562cfa4d19f0 2024-07-31T15:01:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.764+0000 7fb6e53e0740 10 client.27433 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:01:58.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.764+0000 7fb6e53e0740 10 client.27433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:01:58.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.764+0000 7fb6e53e0740 20 client.27433 populate_metadata read hostname 'smithi067' 2024-07-31T15:01:58.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.764+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/1260112296 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562cfa513070 0x562cfa940bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:01:58.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.764+0000 7fb6e53e0740 4 client.27433 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:01:58.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.764+0000 7fb6e53e0740 1 -- 172.21.15.67:0/1260112296 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562cfa82c5a0 con 0x562cfa513070 2024-07-31T15:01:58.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.764+0000 7fb6dcc2b640 1 --2- 172.21.15.67:0/1260112296 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562cfa513070 0x562cfa940bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:01:58.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.765+0000 7fb6b7fff640 10 client.27433 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:58.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.765+0000 7fb6dcc2b640 1 --2- 172.21.15.67:0/1260112296 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562cfa513070 0x562cfa940bf0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85aa7819debe1885 server_cookie=d0c3316c3ab9f3d8 in_seq=0 out_seq=0 2024-07-31T15:01:58.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.766+0000 7fb6b7fff640 1 -- 172.21.15.67:0/1260112296 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562cfa82c5a0 con 0x562cfa513070 2024-07-31T15:01:58.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.766+0000 7fb6b7fff640 10 client.27433 handle_command_reply: tid=0 2024-07-31T15:01:58.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.766+0000 7fb6e53e0740 1 client.27433 shutdown 2024-07-31T15:01:58.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.766+0000 7fb6e53e0740 1 -- 172.21.15.67:0/1260112296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4d19f0 msgr2=0x562cfa8cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:58.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.766+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/1260112296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4d19f0 0x562cfa8cc3f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400f730 tx=0x7fb6c4013300 comp rx=0 tx=0).stop 2024-07-31T15:01:58.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.766+0000 7fb6e53e0740 1 -- 172.21.15.67:0/1260112296 shutdown_connections 2024-07-31T15:01:58.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.766+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/1260112296 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562cfa513070 0x562cfa940bf0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.766+0000 7fb6b7fff640 0 client.27433 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:01:58.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.767+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/1260112296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfa4f2390 0x562cfa8db460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.767+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/1260112296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562cfa4ea1b0 0x562cfa8daf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.767+0000 7fb6e53e0740 1 --2- 172.21.15.67:0/1260112296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cfa4d19f0 0x562cfa8cc3f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.767+0000 7fb6e53e0740 1 -- 172.21.15.67:0/1260112296 >> 172.21.15.67:0/1260112296 conn(0x562cfa507460 msgr2=0x562cfa8b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:58.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.767+0000 7fb6e53e0740 1 -- 172.21.15.67:0/1260112296 shutdown_connections 2024-07-31T15:01:58.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.767+0000 7fb6e53e0740 1 -- 172.21.15.67:0/1260112296 wait complete. 2024-07-31T15:01:58.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.767+0000 7fb6e53e0740 20 client.27433 trim_cache size 0 max 16384 2024-07-31T15:01:58.772 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:01:58.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.767+0000 7fb6dee8f640 1 -- 172.21.15.67:0/2569272920 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6c80b3d40 msgr2=0x7fb6c80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:58.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.767+0000 7fb6dee8f640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6c80b3d40 0x7fb6c80b6250 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0102bf0 tx=0x7fb6d002a8c0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.768+0000 7fb6dee8f640 1 -- 172.21.15.67:0/2569272920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d810d890 msgr2=0x7fb6d8102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:01:58.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.768+0000 7fb6dee8f640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d810d890 0x7fb6d8102c20 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005cc0 tx=0x7fb6d0031700 comp rx=0 tx=0).stop 2024-07-31T15:01:58.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.768+0000 7fb6dee8f640 1 -- 172.21.15.67:0/2569272920 shutdown_connections 2024-07-31T15:01:58.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.768+0000 7fb6dee8f640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb6c80b3d40 0x7fb6c80b6250 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.768+0000 7fb6dee8f640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6d810d890 0x7fb6d8102c20 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.768+0000 7fb6dee8f640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d8104db0 0x7fb6d81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.768+0000 7fb6dee8f640 1 --2- 172.21.15.67:0/2569272920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6d8104460 0x7fb6d81021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:01:58.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.768+0000 7fb6dee8f640 1 -- 172.21.15.67:0/2569272920 >> 172.21.15.67:0/2569272920 conn(0x7fb6d8079210 msgr2=0x7fb6d81073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:01:58.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.768+0000 7fb6dee8f640 1 -- 172.21.15.67:0/2569272920 shutdown_connections 2024-07-31T15:01:58.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:01:58.768+0000 7fb6dee8f640 1 -- 172.21.15.67:0/2569272920 wait complete. 2024-07-31T15:01:59.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:01:58 smithi067 ceph-mon[32750]: pgmap v737: 129 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 113 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 446 KiB/s wr, 128 op/s 2024-07-31T15:01:59.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:01:58 smithi073 ceph-mon[36714]: pgmap v737: 129 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 113 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 446 KiB/s wr, 128 op/s 2024-07-31T15:01:59.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:01:58 smithi077 ceph-mon[37327]: pgmap v737: 129 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 113 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 446 KiB/s wr, 128 op/s 2024-07-31T15:02:00.524 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T15:02:00.525 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T15:02:00.813 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:02:01.204 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:01 smithi067 ceph-mon[32750]: pgmap v738: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 599 KiB/s wr, 166 op/s 2024-07-31T15:02:01.204 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:01 smithi067 ceph-mon[32750]: osdmap e101: 12 total, 12 up, 12 in 2024-07-31T15:02:01.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:01 smithi073 ceph-mon[36714]: pgmap v738: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 599 KiB/s wr, 166 op/s 2024-07-31T15:02:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:01 smithi073 ceph-mon[36714]: osdmap e101: 12 total, 12 up, 12 in 2024-07-31T15:02:01.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:01 smithi077 ceph-mon[37327]: pgmap v738: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 599 KiB/s wr, 166 op/s 2024-07-31T15:02:01.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:01 smithi077 ceph-mon[37327]: osdmap e101: 12 total, 12 up, 12 in 2024-07-31T15:02:02.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:02 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:02:02.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:02 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:02:02.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:02 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:02:02.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.619+0000 7f5eec1dd640 1 -- 172.21.15.67:0/2960198538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ee4102490 msgr2=0x7f5ee41028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.619+0000 7f5eec1dd640 1 --2- 172.21.15.67:0/2960198538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ee4102490 0x7f5ee41028a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009900 tx=0x7f5ee002a410 comp rx=0 tx=0).stop 2024-07-31T15:02:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.619+0000 7f5eec1dd640 1 -- 172.21.15.67:0/2960198538 shutdown_connections 2024-07-31T15:02:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.619+0000 7f5eec1dd640 1 --2- 172.21.15.67:0/2960198538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ee4108720 0x7f5ee410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.619+0000 7f5eec1dd640 1 --2- 172.21.15.67:0/2960198538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee4107dd0 0x7f5ee41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.619+0000 7f5eec1dd640 1 --2- 172.21.15.67:0/2960198538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ee4102490 0x7f5ee41028a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.619+0000 7f5eec1dd640 1 -- 172.21.15.67:0/2960198538 >> 172.21.15.67:0/2960198538 conn(0x7f5ee4079160 msgr2=0x7f5ee4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:02.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.619+0000 7f5eec1dd640 1 -- 172.21.15.67:0/2960198538 shutdown_connections 2024-07-31T15:02:02.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.619+0000 7f5eec1dd640 1 -- 172.21.15.67:0/2960198538 wait complete. 2024-07-31T15:02:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.620+0000 7f5eec1dd640 1 Processor -- start 2024-07-31T15:02:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.620+0000 7f5eec1dd640 1 -- start start 2024-07-31T15:02:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.620+0000 7f5eec1dd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ee4102490 0x7f5ee419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.620+0000 7f5eec1dd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee4107dd0 0x7f5ee419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.620+0000 7f5eec1dd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ee4108720 0x7f5ee419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.620+0000 7f5eec1dd640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee4112380 con 0x7f5ee4107dd0 2024-07-31T15:02:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.620+0000 7f5eec1dd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee4112200 con 0x7f5ee4102490 2024-07-31T15:02:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.620+0000 7f5eec1dd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee4112500 con 0x7f5ee4108720 2024-07-31T15:02:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.620+0000 7f5ee9778640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee4107dd0 0x7f5ee419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.621+0000 7f5ee9778640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee4107dd0 0x7f5ee419b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53338/0 (socket says 172.21.15.67:53338) 2024-07-31T15:02:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.621+0000 7f5ee9778640 1 -- 172.21.15.67:0/3865342460 learned_addr learned my addr 172.21.15.67:0/3865342460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:02.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.621+0000 7f5eea77a640 1 --2- 172.21.15.67:0/3865342460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ee4108720 0x7f5ee419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.621+0000 7f5ee9778640 1 -- 172.21.15.67:0/3865342460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ee4108720 msgr2=0x7f5ee419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.621+0000 7f5ee9778640 1 --2- 172.21.15.67:0/3865342460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ee4108720 0x7f5ee419b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.621+0000 7f5ee9778640 1 -- 172.21.15.67:0/3865342460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ee4102490 msgr2=0x7f5ee419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.621+0000 7f5ee9778640 1 --2- 172.21.15.67:0/3865342460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ee4102490 0x7f5ee419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.621+0000 7f5ee9778640 1 -- 172.21.15.67:0/3865342460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee41a5f60 con 0x7f5ee4107dd0 2024-07-31T15:02:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.621+0000 7f5eea77a640 1 --2- 172.21.15.67:0/3865342460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ee4108720 0x7f5ee419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:02:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.621+0000 7f5ee9778640 1 --2- 172.21.15.67:0/3865342460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee4107dd0 0x7f5ee419b210 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400e9a0 tx=0x7f5ed400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.621+0000 7f5edaffd640 1 -- 172.21.15.67:0/3865342460 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed400cd80 con 0x7f5ee4107dd0 2024-07-31T15:02:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.622+0000 7f5edaffd640 1 -- 172.21.15.67:0/3865342460 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ed40137d0 con 0x7f5ee4107dd0 2024-07-31T15:02:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.622+0000 7f5eec1dd640 1 -- 172.21.15.67:0/3865342460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee4198b20 con 0x7f5ee4107dd0 2024-07-31T15:02:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.622+0000 7f5edaffd640 1 -- 172.21.15.67:0/3865342460 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40038e0 con 0x7f5ee4107dd0 2024-07-31T15:02:02.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.622+0000 7f5eec1dd640 1 -- 172.21.15.67:0/3865342460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee4199060 con 0x7f5ee4107dd0 2024-07-31T15:02:02.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.625+0000 7f5edaffd640 1 -- 172.21.15.67:0/3865342460 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5ed4013c80 con 0x7f5ee4107dd0 2024-07-31T15:02:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.626+0000 7f5edaffd640 1 --2- 172.21.15.67:0/3865342460 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5ed00b3d70 0x7f5ed00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.626+0000 7f5ef272e740 1 Processor -- start 2024-07-31T15:02:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.626+0000 7f5ee9f79640 1 --2- 172.21.15.67:0/3865342460 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5ed00b3d70 0x7f5ed00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.626+0000 7f5edaffd640 1 -- 172.21.15.67:0/3865342460 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9407+0+0 (secure 0 0 0) 0x7f5ed40e85e0 con 0x7f5ee4107dd0 2024-07-31T15:02:02.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.626+0000 7f5ef272e740 1 -- start start 2024-07-31T15:02:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.626+0000 7f5ef272e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f327b350 0x55a8f36203b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.626+0000 7f5ef272e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f3252490 0x55a8f36208f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ef272e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f325a670 0x55a8f3620e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ef272e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a8f35082f0 con 0x55a8f325a670 2024-07-31T15:02:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ef272e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a8f3596a80 con 0x55a8f327b350 2024-07-31T15:02:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ef272e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a8f3585f80 con 0x55a8f3252490 2024-07-31T15:02:02.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ee9f79640 1 --2- 172.21.15.67:0/3865342460 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5ed00b3d70 0x7f5ed00b6280 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5ee002a930 tx=0x7f5ee0031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ee9778640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f3252490 0x55a8f36208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5eea77a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f325a670 0x55a8f3620e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ee9778640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f3252490 0x55a8f36208f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:42910/0 (socket says 172.21.15.67:42910) 2024-07-31T15:02:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ee9778640 1 -- 172.21.15.67:0/4169120425 learned_addr learned my addr 172.21.15.67:0/4169120425 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ee9f79640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f327b350 0x55a8f36203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ee9778640 1 -- 172.21.15.67:0/4169120425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f327b350 msgr2=0x55a8f36203b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ee9778640 1 --2- 172.21.15.67:0/4169120425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f327b350 0x55a8f36203b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ee9778640 1 -- 172.21.15.67:0/4169120425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f325a670 msgr2=0x55a8f3620e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ee9778640 1 --2- 172.21.15.67:0/4169120425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f325a670 0x55a8f3620e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ee9778640 1 -- 172.21.15.67:0/4169120425 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8f32e6930 con 0x55a8f3252490 2024-07-31T15:02:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5eea77a640 1 --2- 172.21.15.67:0/4169120425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f325a670 0x55a8f3620e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:02:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.627+0000 7f5ee9778640 1 --2- 172.21.15.67:0/4169120425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f3252490 0x55a8f36208f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40ab010 tx=0x7f5ed40ae700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7b18935d28ac32a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.628+0000 7f5ecffff640 1 -- 172.21.15.67:0/4169120425 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40aef90 con 0x55a8f3252490 2024-07-31T15:02:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.628+0000 7f5ecffff640 1 -- 172.21.15.67:0/4169120425 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ed40e8040 con 0x55a8f3252490 2024-07-31T15:02:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.628+0000 7f5ecffff640 1 -- 172.21.15.67:0/4169120425 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40e81f0 con 0x55a8f3252490 2024-07-31T15:02:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.629+0000 7f5ef272e740 1 -- 172.21.15.67:0/4169120425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f3252490 msgr2=0x55a8f36208f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.629+0000 7f5ef272e740 1 --2- 172.21.15.67:0/4169120425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f3252490 0x55a8f36208f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40ab010 tx=0x7f5ed40ae700 comp rx=0 tx=0).stop 2024-07-31T15:02:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.629+0000 7f5ef272e740 1 -- 172.21.15.67:0/4169120425 shutdown_connections 2024-07-31T15:02:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.629+0000 7f5ef272e740 1 --2- 172.21.15.67:0/4169120425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f325a670 0x55a8f3620e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.629+0000 7f5ef272e740 1 --2- 172.21.15.67:0/4169120425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f3252490 0x55a8f36208f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.629+0000 7f5ef272e740 1 --2- 172.21.15.67:0/4169120425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f327b350 0x55a8f36203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.629+0000 7f5ef272e740 1 -- 172.21.15.67:0/4169120425 >> 172.21.15.67:0/4169120425 conn(0x55a8f326f740 msgr2=0x55a8f361e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.629+0000 7f5ef272e740 1 -- 172.21.15.67:0/4169120425 shutdown_connections 2024-07-31T15:02:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.629+0000 7f5ef272e740 1 -- 172.21.15.67:0/4169120425 wait complete. 2024-07-31T15:02:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.629+0000 7f5ef272e740 1 -- start start 2024-07-31T15:02:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.629+0000 7f5ef272e740 1 Processor -- start 2024-07-31T15:02:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.630+0000 7f5ef272e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f362c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.630+0000 7f5ef272e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f325a670 0x55a8f362c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.630+0000 7f5ef272e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f327b350 0x55a8f3637d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.630+0000 7f5ef272e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a8f35284f0 con 0x55a8f3252490 2024-07-31T15:02:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.630+0000 7f5ef272e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a8f3589fc0 con 0x55a8f325a670 2024-07-31T15:02:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.630+0000 7f5ef272e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a8f33e9890 con 0x55a8f327b350 2024-07-31T15:02:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.630+0000 7f5ef272e740 10 client.0 _ll_register_callbacks cb 0x55a8f3532500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.630+0000 7f5eea77a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f327b350 0x55a8f3637d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.630+0000 7f5ee9f79640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f362c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.630+0000 7f5ee9778640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f325a670 0x55a8f362c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.630+0000 7f5eea77a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f327b350 0x55a8f3637d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:42918/0 (socket says 172.21.15.67:42918) 2024-07-31T15:02:02.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.630+0000 7f5eea77a640 1 -- 172.21.15.67:0/2055520426 learned_addr learned my addr 172.21.15.67:0/2055520426 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:02.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5eea77a640 1 -- 172.21.15.67:0/2055520426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f325a670 msgr2=0x55a8f362c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:02.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5eea77a640 1 --2- 172.21.15.67:0/2055520426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f325a670 0x55a8f362c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5eea77a640 1 -- 172.21.15.67:0/2055520426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 msgr2=0x55a8f362c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:02.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5eea77a640 1 --2- 172.21.15.67:0/2055520426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f362c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5eea77a640 1 -- 172.21.15.67:0/2055520426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8f3537c50 con 0x55a8f327b350 2024-07-31T15:02:02.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5ee9778640 1 --2- 172.21.15.67:0/2055520426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f325a670 0x55a8f362c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:02:02.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5eea77a640 1 --2- 172.21.15.67:0/2055520426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f327b350 0x55a8f3637d00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00fe80 tx=0x7f5edc012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efd822f46e610055 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:02.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5ecd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:02:02.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5ecd7fa640 1 -- 172.21.15.67:0/2055520426 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc010430 con 0x55a8f327b350 2024-07-31T15:02:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5ecd7fa640 1 -- 172.21.15.67:0/2055520426 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9407+0+0 (secure 0 0 0) 0x7f5edc0105d0 con 0x55a8f327b350 2024-07-31T15:02:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5ef272e740 1 -- 172.21.15.67:0/2055520426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ed40e8040 con 0x55a8f327b350 2024-07-31T15:02:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5ecd7fa640 1 -- 172.21.15.67:0/2055520426 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f5edc0107d0 con 0x55a8f327b350 2024-07-31T15:02:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5ef272e740 10 client.27445 fetch_fsmap learned FSMap version 58 2024-07-31T15:02:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5ef272e740 1 -- 172.21.15.67:0/2055520426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55a8f363ab30 con 0x55a8f327b350 2024-07-31T15:02:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.631+0000 7f5ecd7fa640 1 -- 172.21.15.67:0/2055520426 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ed40e8040 con 0x55a8f327b350 2024-07-31T15:02:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.632+0000 7f5ecd7fa640 1 -- 172.21.15.67:0/2055520426 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7f5edc01c9c0 con 0x55a8f327b350 2024-07-31T15:02:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.632+0000 7f5ef272e740 10 client.27445 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:02:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.632+0000 7f5ef272e740 10 client.27445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:02:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.632+0000 7f5ef272e740 20 client.27445 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.632+0000 7f5ef272e740 1 --2- 172.21.15.67:0/2055520426 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a8f3239cd0 0x55a8f363e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.632+0000 7f5ef272e740 4 client.27445 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:02:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.632+0000 7f5ef272e740 1 -- 172.21.15.67:0/2055520426 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8f35989d0 con 0x55a8f3239cd0 2024-07-31T15:02:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.632+0000 7f5ee9f79640 1 --2- 172.21.15.67:0/2055520426 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a8f3239cd0 0x55a8f363e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.632+0000 7f5ee9f79640 1 --2- 172.21.15.67:0/2055520426 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a8f3239cd0 0x55a8f363e0f0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29c764059bd2683e server_cookie=3031cad669311a3d in_seq=0 out_seq=0 2024-07-31T15:02:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.632+0000 7f5ecd7fa640 10 client.27445 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:02.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.635+0000 7f5ecd7fa640 1 -- 172.21.15.67:0/2055520426 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8f35989d0 con 0x55a8f3239cd0 2024-07-31T15:02:02.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.635+0000 7f5ecd7fa640 10 client.27445 handle_command_reply: tid=0 2024-07-31T15:02:02.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.635+0000 7f5ef272e740 1 client.27445 shutdown 2024-07-31T15:02:02.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.635+0000 7f5ef272e740 1 -- 172.21.15.67:0/2055520426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f327b350 msgr2=0x55a8f3637d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:02.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.635+0000 7f5ef272e740 1 --2- 172.21.15.67:0/2055520426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f327b350 0x55a8f3637d00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00fe80 tx=0x7f5edc012820 comp rx=0 tx=0).stop 2024-07-31T15:02:02.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.635+0000 7f5ef272e740 1 -- 172.21.15.67:0/2055520426 shutdown_connections 2024-07-31T15:02:02.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.635+0000 7f5ef272e740 1 --2- 172.21.15.67:0/2055520426 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a8f3239cd0 0x55a8f363e0f0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.635+0000 7f5ef272e740 1 --2- 172.21.15.67:0/2055520426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f327b350 0x55a8f3637d00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.635+0000 7f5ecd7fa640 0 client.27445 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:02.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.635+0000 7f5ef272e740 1 --2- 172.21.15.67:0/2055520426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f325a670 0x55a8f362c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.635+0000 7f5ef272e740 1 --2- 172.21.15.67:0/2055520426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f362c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.635+0000 7f5ef272e740 1 -- 172.21.15.67:0/2055520426 >> 172.21.15.67:0/2055520426 conn(0x55a8f326f740 msgr2=0x55a8f3623590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:02.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.636+0000 7f5ef272e740 1 -- 172.21.15.67:0/2055520426 shutdown_connections 2024-07-31T15:02:02.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.636+0000 7f5ef272e740 1 -- 172.21.15.67:0/2055520426 wait complete. 2024-07-31T15:02:02.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.636+0000 7f5ef272e740 20 client.27445 trim_cache size 0 max 16384 2024-07-31T15:02:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.660+0000 7f5ef272e740 1 Processor -- start 2024-07-31T15:02:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.660+0000 7f5ef272e740 1 -- start start 2024-07-31T15:02:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.660+0000 7f5ef272e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f3239cd0 0x55a8f3621da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.660+0000 7f5ef272e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f362da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.660+0000 7f5ef272e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f325a670 0x55a8f36438d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.660+0000 7f5ef272e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee41ae6f0 con 0x55a8f3252490 2024-07-31T15:02:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.660+0000 7f5ef272e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee41b02f0 con 0x55a8f3239cd0 2024-07-31T15:02:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.660+0000 7f5ef272e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee41aedc0 con 0x55a8f325a670 2024-07-31T15:02:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.660+0000 7f5ee9778640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f362da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.660+0000 7f5ee9f79640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f3239cd0 0x55a8f3621da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.660+0000 7f5ee9778640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f362da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53364/0 (socket says 172.21.15.67:53364) 2024-07-31T15:02:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.660+0000 7f5ee9778640 1 -- 172.21.15.67:0/3171044878 learned_addr learned my addr 172.21.15.67:0/3171044878 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.660+0000 7f5eea77a640 1 --2- 172.21.15.67:0/3171044878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f325a670 0x55a8f36438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.661+0000 7f5ee9f79640 1 -- 172.21.15.67:0/3171044878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f325a670 msgr2=0x55a8f36438d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.661+0000 7f5ee9f79640 1 --2- 172.21.15.67:0/3171044878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f325a670 0x55a8f36438d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.661+0000 7f5ee9f79640 1 -- 172.21.15.67:0/3171044878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 msgr2=0x55a8f362da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.661+0000 7f5ee9f79640 1 --2- 172.21.15.67:0/3171044878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f362da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.661+0000 7f5ee9f79640 1 -- 172.21.15.67:0/3171044878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8f362d0e0 con 0x55a8f3239cd0 2024-07-31T15:02:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.661+0000 7f5ee9778640 1 --2- 172.21.15.67:0/3171044878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f362da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:02:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.661+0000 7f5eea77a640 1 --2- 172.21.15.67:0/3171044878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f325a670 0x55a8f36438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:02:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.661+0000 7f5ee9f79640 1 --2- 172.21.15.67:0/3171044878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f3239cd0 0x55a8f3621da0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55a8f363e870 tx=0x7f5ee003a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f4017d0f4b7892e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.661+0000 7f5ecf7fe640 1 -- 172.21.15.67:0/3171044878 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0055070 con 0x55a8f3239cd0 2024-07-31T15:02:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.661+0000 7f5ecf7fe640 1 -- 172.21.15.67:0/3171044878 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee0041040 con 0x55a8f3239cd0 2024-07-31T15:02:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.661+0000 7f5ecf7fe640 1 -- 172.21.15.67:0/3171044878 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0035040 con 0x55a8f3239cd0 2024-07-31T15:02:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.662+0000 7f5ef272e740 1 -- 172.21.15.67:0/3171044878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f3239cd0 msgr2=0x55a8f3621da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.662+0000 7f5ef272e740 1 --2- 172.21.15.67:0/3171044878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f3239cd0 0x55a8f3621da0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55a8f363e870 tx=0x7f5ee003a000 comp rx=0 tx=0).stop 2024-07-31T15:02:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.662+0000 7f5ef272e740 1 -- 172.21.15.67:0/3171044878 shutdown_connections 2024-07-31T15:02:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.662+0000 7f5ef272e740 1 --2- 172.21.15.67:0/3171044878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f325a670 0x55a8f36438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.662+0000 7f5ef272e740 1 --2- 172.21.15.67:0/3171044878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f362da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.662+0000 7f5ef272e740 1 --2- 172.21.15.67:0/3171044878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f3239cd0 0x55a8f3621da0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.662+0000 7f5ef272e740 1 -- 172.21.15.67:0/3171044878 >> 172.21.15.67:0/3171044878 conn(0x55a8f326f740 msgr2=0x55a8f362d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.662+0000 7f5ef272e740 1 -- 172.21.15.67:0/3171044878 shutdown_connections 2024-07-31T15:02:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.663+0000 7f5ef272e740 1 -- 172.21.15.67:0/3171044878 wait complete. 2024-07-31T15:02:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.663+0000 7f5ef272e740 1 -- start start 2024-07-31T15:02:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.663+0000 7f5ef272e740 1 Processor -- start 2024-07-31T15:02:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.663+0000 7f5ef272e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f3239cd0 0x55a8f362fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.663+0000 7f5ef272e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f36303d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.663+0000 7f5ef272e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f325a670 0x55a8f36a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.663+0000 7f5ef272e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee41afc20 con 0x55a8f3252490 2024-07-31T15:02:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.663+0000 7f5ef272e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee41af490 con 0x55a8f325a670 2024-07-31T15:02:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.663+0000 7f5ef272e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee41ac6d0 con 0x55a8f3239cd0 2024-07-31T15:02:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.664+0000 7f5ef272e740 10 client.0 _ll_register_callbacks cb 0x7f5edc00fa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.664+0000 7f5eea77a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f325a670 0x55a8f36a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.664+0000 7f5ee9778640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f36303d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.664+0000 7f5ee9f79640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f3239cd0 0x55a8f362fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.664+0000 7f5eea77a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f325a670 0x55a8f36a58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:42366/0 (socket says 172.21.15.67:42366) 2024-07-31T15:02:02.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.664+0000 7f5eea77a640 1 -- 172.21.15.67:0/636803436 learned_addr learned my addr 172.21.15.67:0/636803436 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:02.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.664+0000 7f5ee9778640 1 -- 172.21.15.67:0/636803436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f3239cd0 msgr2=0x55a8f362fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:02.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.664+0000 7f5ee9778640 1 --2- 172.21.15.67:0/636803436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f3239cd0 0x55a8f362fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.664+0000 7f5ee9778640 1 -- 172.21.15.67:0/636803436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f325a670 msgr2=0x55a8f36a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:02.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.664+0000 7f5ee9778640 1 --2- 172.21.15.67:0/636803436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f325a670 0x55a8f36a58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.664+0000 7f5ee9778640 1 -- 172.21.15.67:0/636803436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8f3630cc0 con 0x55a8f3252490 2024-07-31T15:02:02.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.664+0000 7f5ee9f79640 1 --2- 172.21.15.67:0/636803436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f3239cd0 0x55a8f362fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:02:02.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.664+0000 7f5ee9778640 1 --2- 172.21.15.67:0/636803436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f36303d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40e5fc0 tx=0x7f5ed40e8420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef17b2013cf3beac server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:02.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.664+0000 7f5ecd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:02:02.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.665+0000 7f5ecd7fa640 1 -- 172.21.15.67:0/636803436 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40b8040 con 0x55a8f3252490 2024-07-31T15:02:02.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.665+0000 7f5ecd7fa640 1 -- 172.21.15.67:0/636803436 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9407+0+0 (secure 0 0 0) 0x7f5ed40b7dc0 con 0x55a8f3252490 2024-07-31T15:02:02.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.665+0000 7f5ef272e740 1 -- 172.21.15.67:0/636803436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ee0041040 con 0x55a8f3252490 2024-07-31T15:02:02.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.665+0000 7f5ecd7fa640 1 -- 172.21.15.67:0/636803436 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f5ed4074180 con 0x55a8f3252490 2024-07-31T15:02:02.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.665+0000 7f5ef272e740 10 client.17919 fetch_fsmap learned FSMap version 58 2024-07-31T15:02:02.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.665+0000 7f5ef272e740 1 -- 172.21.15.67:0/636803436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55a8f36a6ed0 con 0x55a8f3252490 2024-07-31T15:02:02.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.665+0000 7f5ecd7fa640 1 -- 172.21.15.67:0/636803436 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee0041040 con 0x55a8f3252490 2024-07-31T15:02:02.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.665+0000 7f5ecd7fa640 1 -- 172.21.15.67:0/636803436 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7f5ed40add80 con 0x55a8f3252490 2024-07-31T15:02:02.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.665+0000 7f5ef272e740 10 client.17919 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:02:02.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.665+0000 7f5ef272e740 10 client.17919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:02:02.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.665+0000 7f5ef272e740 20 client.17919 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:02.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.665+0000 7f5ef272e740 1 --2- 172.21.15.67:0/636803436 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a8f327b350 0x55a8f36a9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:02.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.665+0000 7f5ef272e740 4 client.17919 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T15:02:02.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.665+0000 7f5ef272e740 1 -- 172.21.15.67:0/636803436 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a8f3594470 con 0x55a8f327b350 2024-07-31T15:02:02.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.666+0000 7f5ee9f79640 1 --2- 172.21.15.67:0/636803436 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a8f327b350 0x55a8f36a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:02.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.666+0000 7f5ecd7fa640 10 client.17919 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:02.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.666+0000 7f5ee9f79640 1 --2- 172.21.15.67:0/636803436 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a8f327b350 0x55a8f36a9220 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d5ba0b55fbabb53 server_cookie=7c35b9d5e5a20183 in_seq=0 out_seq=0 2024-07-31T15:02:02.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.668+0000 7f5ecd7fa640 1 -- 172.21.15.67:0/636803436 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a8f3594470 con 0x55a8f327b350 2024-07-31T15:02:02.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.668+0000 7f5ecd7fa640 10 client.17919 handle_command_reply: tid=0 2024-07-31T15:02:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.668+0000 7f5ef272e740 1 client.17919 shutdown 2024-07-31T15:02:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.668+0000 7f5ef272e740 1 -- 172.21.15.67:0/636803436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 msgr2=0x55a8f36303d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.668+0000 7f5ef272e740 1 --2- 172.21.15.67:0/636803436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f36303d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40e5fc0 tx=0x7f5ed40e8420 comp rx=0 tx=0).stop 2024-07-31T15:02:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.668+0000 7f5ef272e740 1 -- 172.21.15.67:0/636803436 shutdown_connections 2024-07-31T15:02:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.668+0000 7f5ef272e740 1 --2- 172.21.15.67:0/636803436 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a8f327b350 0x55a8f36a9220 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.668+0000 7f5ef272e740 1 --2- 172.21.15.67:0/636803436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8f325a670 0x55a8f36a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.668+0000 7f5ef272e740 1 --2- 172.21.15.67:0/636803436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a8f3252490 0x55a8f36303d0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.668+0000 7f5ecd7fa640 0 client.17919 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.668+0000 7f5ef272e740 1 --2- 172.21.15.67:0/636803436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8f3239cd0 0x55a8f362fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.668+0000 7f5ef272e740 1 -- 172.21.15.67:0/636803436 >> 172.21.15.67:0/636803436 conn(0x55a8f326f740 msgr2=0x55a8f362dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.668+0000 7f5ef272e740 1 -- 172.21.15.67:0/636803436 shutdown_connections 2024-07-31T15:02:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.668+0000 7f5ef272e740 1 -- 172.21.15.67:0/636803436 wait complete. 2024-07-31T15:02:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.669+0000 7f5ef272e740 20 client.17919 trim_cache size 0 max 16384 2024-07-31T15:02:02.673 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:02:02.673 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T15:02:02.673 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "acb195c0-e291-456a-ac4b-1332fba0bea8", 2024-07-31T15:02:02.673 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T15:02:02.673 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:02:02.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.669+0000 7f5eec1dd640 1 -- 172.21.15.67:0/3865342460 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5ed00b3d70 msgr2=0x7f5ed00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.669+0000 7f5eec1dd640 1 --2- 172.21.15.67:0/3865342460 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5ed00b3d70 0x7f5ed00b6280 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5ee002a930 tx=0x7f5ee0031400 comp rx=0 tx=0).stop 2024-07-31T15:02:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.669+0000 7f5eec1dd640 1 -- 172.21.15.67:0/3865342460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee4107dd0 msgr2=0x7f5ee419b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.669+0000 7f5eec1dd640 1 --2- 172.21.15.67:0/3865342460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee4107dd0 0x7f5ee419b210 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400e9a0 tx=0x7f5ed400ee60 comp rx=0 tx=0).stop 2024-07-31T15:02:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.669+0000 7f5eec1dd640 1 -- 172.21.15.67:0/3865342460 shutdown_connections 2024-07-31T15:02:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.669+0000 7f5eec1dd640 1 --2- 172.21.15.67:0/3865342460 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5ed00b3d70 0x7f5ed00b6280 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.669+0000 7f5eec1dd640 1 --2- 172.21.15.67:0/3865342460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ee4108720 0x7f5ee419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.670+0000 7f5eec1dd640 1 --2- 172.21.15.67:0/3865342460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee4107dd0 0x7f5ee419b210 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.670+0000 7f5eec1dd640 1 --2- 172.21.15.67:0/3865342460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ee4102490 0x7f5ee419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.670+0000 7f5eec1dd640 1 -- 172.21.15.67:0/3865342460 >> 172.21.15.67:0/3865342460 conn(0x7f5ee4079160 msgr2=0x7f5ee4100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.670+0000 7f5eec1dd640 1 -- 172.21.15.67:0/3865342460 shutdown_connections 2024-07-31T15:02:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:02.670+0000 7f5eec1dd640 1 -- 172.21.15.67:0/3865342460 wait complete. 2024-07-31T15:02:03.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:03 smithi067 ceph-mon[32750]: pgmap v740: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 783 KiB/s wr, 244 op/s 2024-07-31T15:02:03.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:03 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:02:03.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:03 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:02:03.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:03 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:02:03.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:03.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:03.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:03.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:03.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:03.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:03.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:03 smithi073 ceph-mon[36714]: pgmap v740: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 783 KiB/s wr, 244 op/s 2024-07-31T15:02:03.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:03 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:02:03.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:03 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:02:03.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:03 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:02:03.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:03.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:03.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:03.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:03.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:03.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:03.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:03 smithi077 ceph-mon[37327]: pgmap v740: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 783 KiB/s wr, 244 op/s 2024-07-31T15:02:03.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:03 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:02:03.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:03 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:02:03.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:03 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:02:03.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:03.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:03.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:03.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:03.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:03.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:03.547 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:02:03.848 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:02:04.245 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T15:02:04.245 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.245 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.245 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.245 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.245 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.245 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.245 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:04.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T15:02:04.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:04.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T15:02:04.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:04.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:04.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:05.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: pgmap v741: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 721 KiB/s wr, 253 op/s 2024-07-31T15:02:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:05.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:05.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:05.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:05.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:05.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:05 smithi067 ceph-mon[32750]: osdmap e102: 12 total, 12 up, 12 in 2024-07-31T15:02:05.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:05.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:05.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: pgmap v741: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 721 KiB/s wr, 253 op/s 2024-07-31T15:02:05.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:05.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:05.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:05.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:05.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:05.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:05.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:05.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:05.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:05.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:05 smithi073 ceph-mon[36714]: osdmap e102: 12 total, 12 up, 12 in 2024-07-31T15:02:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: pgmap v741: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 721 KiB/s wr, 253 op/s 2024-07-31T15:02:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:05 smithi077 ceph-mon[37327]: osdmap e102: 12 total, 12 up, 12 in 2024-07-31T15:02:05.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.630+0000 7fea1c9cf640 1 -- 172.21.15.67:0/3760776132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea18106620 msgr2=0x7fea18110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:05.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.630+0000 7fea1c9cf640 1 --2- 172.21.15.67:0/3760776132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea18106620 0x7fea18110070 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fea10009ca0 tx=0x7fea1002a8c0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.631+0000 7fea1c9cf640 1 -- 172.21.15.67:0/3760776132 shutdown_connections 2024-07-31T15:02:05.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.631+0000 7fea1c9cf640 1 --2- 172.21.15.67:0/3760776132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea18106620 0x7fea18110070 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.631+0000 7fea1c9cf640 1 --2- 172.21.15.67:0/3760776132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea18105cd0 0x7fea181060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.631+0000 7fea1c9cf640 1 --2- 172.21.15.67:0/3760776132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea181019f0 0x7fea18101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.631+0000 7fea1c9cf640 1 -- 172.21.15.67:0/3760776132 >> 172.21.15.67:0/3760776132 conn(0x7fea18077720 msgr2=0x7fea18077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:05.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.631+0000 7fea1c9cf640 1 -- 172.21.15.67:0/3760776132 shutdown_connections 2024-07-31T15:02:05.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.631+0000 7fea1c9cf640 1 -- 172.21.15.67:0/3760776132 wait complete. 2024-07-31T15:02:05.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.632+0000 7fea1c9cf640 1 Processor -- start 2024-07-31T15:02:05.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.632+0000 7fea1c9cf640 1 -- start start 2024-07-31T15:02:05.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.632+0000 7fea1c9cf640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea181019f0 0x7fea18102990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.632+0000 7fea1c9cf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea18105cd0 0x7fea18102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.632+0000 7fea1c9cf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea18106620 0x7fea18103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.632+0000 7fea1c9cf640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fea18113500 con 0x7fea181019f0 2024-07-31T15:02:05.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.632+0000 7fea1c9cf640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea18113380 con 0x7fea18106620 2024-07-31T15:02:05.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.632+0000 7fea1c9cf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fea18113680 con 0x7fea18105cd0 2024-07-31T15:02:05.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.632+0000 7fea1659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea181019f0 0x7fea18102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.632+0000 7fea15d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea18105cd0 0x7fea18102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.632+0000 7fea1659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea181019f0 0x7fea18102990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53392/0 (socket says 172.21.15.67:53392) 2024-07-31T15:02:05.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.632+0000 7fea1659c640 1 -- 172.21.15.67:0/3937552267 learned_addr learned my addr 172.21.15.67:0/3937552267 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:05.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.632+0000 7fea16d9d640 1 --2- 172.21.15.67:0/3937552267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea18106620 0x7fea18103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.633+0000 7fea1659c640 1 -- 172.21.15.67:0/3937552267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea18105cd0 msgr2=0x7fea18102ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:05.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.633+0000 7fea1659c640 1 --2- 172.21.15.67:0/3937552267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea18105cd0 0x7fea18102ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.633+0000 7fea1659c640 1 -- 172.21.15.67:0/3937552267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea18106620 msgr2=0x7fea18103410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:05.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.633+0000 7fea1659c640 1 --2- 172.21.15.67:0/3937552267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea18106620 0x7fea18103410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.633+0000 7fea1659c640 1 -- 172.21.15.67:0/3937552267 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea181a8210 con 0x7fea181019f0 2024-07-31T15:02:05.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.633+0000 7fea15d9b640 1 --2- 172.21.15.67:0/3937552267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea18105cd0 0x7fea18102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:02:05.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.633+0000 7fea1659c640 1 --2- 172.21.15.67:0/3937552267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea181019f0 0x7fea18102990 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fea0800ea60 tx=0x7fea0800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:05.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.633+0000 7fea077fe640 1 -- 172.21.15.67:0/3937552267 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0800cde0 con 0x7fea181019f0 2024-07-31T15:02:05.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.634+0000 7fea077fe640 1 -- 172.21.15.67:0/3937552267 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea080035f0 con 0x7fea181019f0 2024-07-31T15:02:05.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.634+0000 7fea1c9cf640 1 -- 172.21.15.67:0/3937552267 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea181a8500 con 0x7fea181019f0 2024-07-31T15:02:05.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.634+0000 7fea077fe640 1 -- 172.21.15.67:0/3937552267 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08010690 con 0x7fea181019f0 2024-07-31T15:02:05.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.634+0000 7fea1c9cf640 1 -- 172.21.15.67:0/3937552267 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea181a8a90 con 0x7fea181019f0 2024-07-31T15:02:05.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.637+0000 7fea077fe640 1 -- 172.21.15.67:0/3937552267 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fea080037a0 con 0x7fea181019f0 2024-07-31T15:02:05.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.637+0000 7fea22f20740 1 Processor -- start 2024-07-31T15:02:05.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.637+0000 7fea22f20740 1 -- start start 2024-07-31T15:02:05.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.638+0000 7fea22f20740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f733070 0x564b4fad8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.638+0000 7fea22f20740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f70a1b0 0x564b4fad8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.638+0000 7fea22f20740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f712390 0x564b4fad90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.638+0000 7fea22f20740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564b4fa4eb60 con 0x564b4f70a1b0 2024-07-31T15:02:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.638+0000 7fea22f20740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564b4fa3e060 con 0x564b4f712390 2024-07-31T15:02:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.638+0000 7fea22f20740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b4fa420a0 con 0x564b4f733070 2024-07-31T15:02:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.638+0000 7fea1659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f70a1b0 0x564b4fad8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.638+0000 7fea16d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f733070 0x564b4fad8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.638+0000 7fea15d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f712390 0x564b4fad90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.638+0000 7fea15d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f712390 0x564b4fad90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:42394/0 (socket says 172.21.15.67:42394) 2024-07-31T15:02:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.638+0000 7fea16d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f733070 0x564b4fad8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:42954/0 (socket says 172.21.15.67:42954) 2024-07-31T15:02:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.638+0000 7fea15d9b640 1 -- 172.21.15.67:0/1665684375 learned_addr learned my addr 172.21.15.67:0/1665684375 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.638+0000 7fea1659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f70a1b0 0x564b4fad8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53400/0 (socket says 172.21.15.67:53400) 2024-07-31T15:02:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.639+0000 7fea1659c640 1 -- 172.21.15.67:0/1665684375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f733070 msgr2=0x564b4fad8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.639+0000 7fea1659c640 1 --2- 172.21.15.67:0/1665684375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f733070 0x564b4fad8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.639+0000 7fea1659c640 1 -- 172.21.15.67:0/1665684375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f712390 msgr2=0x564b4fad90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.639+0000 7fea1659c640 1 --2- 172.21.15.67:0/1665684375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f712390 0x564b4fad90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.639+0000 7fea1659c640 1 -- 172.21.15.67:0/1665684375 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b4f9e4b40 con 0x564b4f70a1b0 2024-07-31T15:02:05.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.639+0000 7fea077fe640 1 --2- 172.21.15.67:0/3937552267 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fea000b3e40 0x7fea000b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.639+0000 7fea1659c640 1 --2- 172.21.15.67:0/1665684375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f70a1b0 0x564b4fad8b80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fea080e1980 tx=0x7fea080ea9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8e886146fe6323 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:05.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.639+0000 7fea16d9d640 1 --2- 172.21.15.67:0/3937552267 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fea000b3e40 0x7fea000b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.639+0000 7fea077fe640 1 -- 172.21.15.67:0/3937552267 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9379+0+0 (secure 0 0 0) 0x7fea080e45f0 con 0x7fea181019f0 2024-07-31T15:02:05.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.639+0000 7fea04ff9640 1 -- 172.21.15.67:0/1665684375 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea080ad070 con 0x564b4f70a1b0 2024-07-31T15:02:05.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.639+0000 7fea04ff9640 1 -- 172.21.15.67:0/1665684375 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea080aa950 con 0x564b4f70a1b0 2024-07-31T15:02:05.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.639+0000 7fea04ff9640 1 -- 172.21.15.67:0/1665684375 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea080a6d00 con 0x564b4f70a1b0 2024-07-31T15:02:05.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.641+0000 7fea22f20740 1 -- 172.21.15.67:0/1665684375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f70a1b0 msgr2=0x564b4fad8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:05.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.641+0000 7fea22f20740 1 --2- 172.21.15.67:0/1665684375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f70a1b0 0x564b4fad8b80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fea080e1980 tx=0x7fea080ea9b0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.641+0000 7fea16d9d640 1 --2- 172.21.15.67:0/3937552267 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fea000b3e40 0x7fea000b6350 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea1002f500 tx=0x7fea1002f310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:05.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.641+0000 7fea22f20740 1 -- 172.21.15.67:0/1665684375 shutdown_connections 2024-07-31T15:02:05.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.641+0000 7fea22f20740 1 --2- 172.21.15.67:0/1665684375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f712390 0x564b4fad90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.641+0000 7fea22f20740 1 --2- 172.21.15.67:0/1665684375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f70a1b0 0x564b4fad8b80 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.641+0000 7fea22f20740 1 --2- 172.21.15.67:0/1665684375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f733070 0x564b4fad8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.641+0000 7fea22f20740 1 -- 172.21.15.67:0/1665684375 >> 172.21.15.67:0/1665684375 conn(0x564b4f727460 msgr2=0x564b4fad6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:05.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.641+0000 7fea22f20740 1 -- 172.21.15.67:0/1665684375 shutdown_connections 2024-07-31T15:02:05.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.641+0000 7fea22f20740 1 -- 172.21.15.67:0/1665684375 wait complete. 2024-07-31T15:02:05.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.642+0000 7fea22f20740 1 -- start start 2024-07-31T15:02:05.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.642+0000 7fea22f20740 1 Processor -- start 2024-07-31T15:02:05.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea22f20740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4fae4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea22f20740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 0x564b4fae46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea22f20740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f733070 0x564b4faf0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea22f20740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564b4f8a1700 con 0x564b4f712390 2024-07-31T15:02:05.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea22f20740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564b4f9e0990 con 0x564b4f733070 2024-07-31T15:02:05.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea22f20740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b4fae6e20 con 0x564b4f70a1b0 2024-07-31T15:02:05.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea1659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4fae4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea15d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 0x564b4fae46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea1659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4fae4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:42966/0 (socket says 172.21.15.67:42966) 2024-07-31T15:02:05.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea1659c640 1 -- 172.21.15.67:0/2772518063 learned_addr learned my addr 172.21.15.67:0/2772518063 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:05.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea22f20740 10 client.0 _ll_register_callbacks cb 0x564b4f9fb4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:05.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea16d9d640 1 --2- 172.21.15.67:0/2772518063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f733070 0x564b4faf0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea1659c640 1 -- 172.21.15.67:0/2772518063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f733070 msgr2=0x564b4faf0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:05.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea1659c640 1 --2- 172.21.15.67:0/2772518063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f733070 0x564b4faf0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea1659c640 1 -- 172.21.15.67:0/2772518063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 msgr2=0x564b4fae46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:05.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea1659c640 1 --2- 172.21.15.67:0/2772518063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 0x564b4fae46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.643+0000 7fea1659c640 1 -- 172.21.15.67:0/2772518063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b4f79e650 con 0x564b4f70a1b0 2024-07-31T15:02:05.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.644+0000 7fea1659c640 1 --2- 172.21.15.67:0/2772518063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4fae4170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea08003e40 tx=0x7fea080045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e5c14fd255d93a3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:05.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.644+0000 7fe9fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:02:05.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.644+0000 7fe9fdffb640 1 -- 172.21.15.67:0/2772518063 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea080ad020 con 0x564b4f70a1b0 2024-07-31T15:02:05.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.644+0000 7fe9fdffb640 1 -- 172.21.15.67:0/2772518063 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9379+0+0 (secure 0 0 0) 0x7fea080e4210 con 0x564b4f70a1b0 2024-07-31T15:02:05.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.644+0000 7fea22f20740 1 -- 172.21.15.67:0/2772518063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea080aa950 con 0x564b4f70a1b0 2024-07-31T15:02:05.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.644+0000 7fe9fdffb640 1 -- 172.21.15.67:0/2772518063 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fea080e4070 con 0x564b4f70a1b0 2024-07-31T15:02:05.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.644+0000 7fea22f20740 10 client.27463 fetch_fsmap learned FSMap version 58 2024-07-31T15:02:05.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.644+0000 7fea22f20740 1 -- 172.21.15.67:0/2772518063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x564b4faf2da0 con 0x564b4f70a1b0 2024-07-31T15:02:05.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.644+0000 7fe9fdffb640 1 -- 172.21.15.67:0/2772518063 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea080aa950 con 0x564b4f70a1b0 2024-07-31T15:02:05.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.645+0000 7fe9fdffb640 1 -- 172.21.15.67:0/2772518063 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7fea080ab0a0 con 0x564b4f70a1b0 2024-07-31T15:02:05.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.645+0000 7fea22f20740 10 client.27463 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:02:05.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.645+0000 7fea22f20740 10 client.27463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:02:05.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.645+0000 7fea22f20740 20 client.27463 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:05.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.645+0000 7fea22f20740 1 --2- 172.21.15.67:0/2772518063 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b4f6f19f0 0x564b4faf6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.645+0000 7fea22f20740 4 client.27463 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:02:05.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.645+0000 7fea22f20740 1 -- 172.21.15.67:0/2772518063 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b4fa50ab0 con 0x564b4f6f19f0 2024-07-31T15:02:05.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.645+0000 7fea15d9b640 1 --2- 172.21.15.67:0/2772518063 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b4f6f19f0 0x564b4faf6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.646+0000 7fe9fdffb640 10 client.27463 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:05.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.646+0000 7fea15d9b640 1 --2- 172.21.15.67:0/2772518063 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b4f6f19f0 0x564b4faf6100 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc4853842fa8643f server_cookie=4a06066f7b0a3d42 in_seq=0 out_seq=0 2024-07-31T15:02:05.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.648+0000 7fe9fdffb640 1 -- 172.21.15.67:0/2772518063 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b4fa50ab0 con 0x564b4f6f19f0 2024-07-31T15:02:05.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.648+0000 7fe9fdffb640 10 client.27463 handle_command_reply: tid=0 2024-07-31T15:02:05.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.648+0000 7fea22f20740 1 client.27463 shutdown 2024-07-31T15:02:05.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.648+0000 7fea22f20740 1 -- 172.21.15.67:0/2772518063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 msgr2=0x564b4fae4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:05.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.648+0000 7fea22f20740 1 --2- 172.21.15.67:0/2772518063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4fae4170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea08003e40 tx=0x7fea080045d0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.648+0000 7fea22f20740 1 -- 172.21.15.67:0/2772518063 shutdown_connections 2024-07-31T15:02:05.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.648+0000 7fea22f20740 1 --2- 172.21.15.67:0/2772518063 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b4f6f19f0 0x564b4faf6100 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.648+0000 7fea22f20740 1 --2- 172.21.15.67:0/2772518063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f733070 0x564b4faf0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.648+0000 7fea22f20740 1 --2- 172.21.15.67:0/2772518063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 0x564b4fae46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.648+0000 7fe9fdffb640 0 client.27463 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:05.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.648+0000 7fea22f20740 1 --2- 172.21.15.67:0/2772518063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4fae4170 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.648+0000 7fea22f20740 1 -- 172.21.15.67:0/2772518063 >> 172.21.15.67:0/2772518063 conn(0x564b4f727460 msgr2=0x564b4fad4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:05.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.649+0000 7fea22f20740 1 -- 172.21.15.67:0/2772518063 shutdown_connections 2024-07-31T15:02:05.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.649+0000 7fea22f20740 1 -- 172.21.15.67:0/2772518063 wait complete. 2024-07-31T15:02:05.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.649+0000 7fea22f20740 20 client.27463 trim_cache size 0 max 16384 2024-07-31T15:02:05.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.675+0000 7fea22f20740 1 Processor -- start 2024-07-31T15:02:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.675+0000 7fea22f20740 1 -- start start 2024-07-31T15:02:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.675+0000 7fea22f20740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f6f19f0 0x564b4fade760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.675+0000 7fea22f20740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4fad53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.675+0000 7fea22f20740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 0x564b4faf00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.675+0000 7fea22f20740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fea181b2a50 con 0x564b4f712390 2024-07-31T15:02:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.675+0000 7fea22f20740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea181b4650 con 0x564b4f6f19f0 2024-07-31T15:02:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.675+0000 7fea22f20740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fea181b3120 con 0x564b4f70a1b0 2024-07-31T15:02:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.676+0000 7fea15d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4fad53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.676+0000 7fea1659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f6f19f0 0x564b4fade760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.676+0000 7fea15d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4fad53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:42970/0 (socket says 172.21.15.67:42970) 2024-07-31T15:02:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.676+0000 7fea15d9b640 1 -- 172.21.15.67:0/1798969905 learned_addr learned my addr 172.21.15.67:0/1798969905 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.676+0000 7fea16d9d640 1 --2- 172.21.15.67:0/1798969905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 0x564b4faf00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.676+0000 7fea15d9b640 1 -- 172.21.15.67:0/1798969905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f6f19f0 msgr2=0x564b4fade760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.676+0000 7fea15d9b640 1 --2- 172.21.15.67:0/1798969905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f6f19f0 0x564b4fade760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.676+0000 7fea15d9b640 1 -- 172.21.15.67:0/1798969905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 msgr2=0x564b4faf00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.676+0000 7fea15d9b640 1 --2- 172.21.15.67:0/1798969905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 0x564b4faf00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.676+0000 7fea15d9b640 1 -- 172.21.15.67:0/1798969905 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b4fae2ed0 con 0x564b4f70a1b0 2024-07-31T15:02:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.676+0000 7fea16d9d640 1 --2- 172.21.15.67:0/1798969905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 0x564b4faf00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:02:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.676+0000 7fea15d9b640 1 --2- 172.21.15.67:0/1798969905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4fad53c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea0c000c00 tx=0x7fea0c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f18f898c46e88c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.677+0000 7fe9fffff640 1 -- 172.21.15.67:0/1798969905 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c022070 con 0x564b4f70a1b0 2024-07-31T15:02:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.677+0000 7fe9fffff640 1 -- 172.21.15.67:0/1798969905 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea0c0054d0 con 0x564b4f70a1b0 2024-07-31T15:02:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.677+0000 7fe9fffff640 1 -- 172.21.15.67:0/1798969905 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c004e50 con 0x564b4f70a1b0 2024-07-31T15:02:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.678+0000 7fea22f20740 1 -- 172.21.15.67:0/1798969905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 msgr2=0x564b4fad53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.678+0000 7fea22f20740 1 --2- 172.21.15.67:0/1798969905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4fad53c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea0c000c00 tx=0x7fea0c005f90 comp rx=0 tx=0).stop 2024-07-31T15:02:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.678+0000 7fea22f20740 1 -- 172.21.15.67:0/1798969905 shutdown_connections 2024-07-31T15:02:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.679+0000 7fea22f20740 1 --2- 172.21.15.67:0/1798969905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 0x564b4faf00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.679+0000 7fea22f20740 1 --2- 172.21.15.67:0/1798969905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4fad53c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.679+0000 7fea22f20740 1 --2- 172.21.15.67:0/1798969905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f6f19f0 0x564b4fade760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.679+0000 7fea22f20740 1 -- 172.21.15.67:0/1798969905 >> 172.21.15.67:0/1798969905 conn(0x564b4f727460 msgr2=0x564b4fad4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.679+0000 7fea22f20740 1 -- 172.21.15.67:0/1798969905 shutdown_connections 2024-07-31T15:02:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.679+0000 7fea22f20740 1 -- 172.21.15.67:0/1798969905 wait complete. 2024-07-31T15:02:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.679+0000 7fea22f20740 1 -- start start 2024-07-31T15:02:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.680+0000 7fea22f20740 1 Processor -- start 2024-07-31T15:02:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.680+0000 7fea22f20740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f6f19f0 0x564b4faf9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.680+0000 7fea22f20740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4faf6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.680+0000 7fea22f20740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 0x564b4faf6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.680+0000 7fea22f20740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fea181b3f80 con 0x564b4f712390 2024-07-31T15:02:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.680+0000 7fea22f20740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea181b37f0 con 0x564b4f6f19f0 2024-07-31T15:02:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.680+0000 7fea22f20740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fea181b0a30 con 0x564b4f70a1b0 2024-07-31T15:02:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.680+0000 7fea15d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4faf6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.680+0000 7fea15d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4faf6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:42980/0 (socket says 172.21.15.67:42980) 2024-07-31T15:02:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.680+0000 7fea15d9b640 1 -- 172.21.15.67:0/3481074681 learned_addr learned my addr 172.21.15.67:0/3481074681 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.680+0000 7fea1659c640 1 --2- 172.21.15.67:0/3481074681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f6f19f0 0x564b4faf9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.680+0000 7fea16d9d640 1 --2- 172.21.15.67:0/3481074681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 0x564b4faf6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.680+0000 7fea22f20740 10 client.0 _ll_register_callbacks cb 0x7fea0c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.681+0000 7fea15d9b640 1 -- 172.21.15.67:0/3481074681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f6f19f0 msgr2=0x564b4faf9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.681+0000 7fea15d9b640 1 --2- 172.21.15.67:0/3481074681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f6f19f0 0x564b4faf9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.681+0000 7fea15d9b640 1 -- 172.21.15.67:0/3481074681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 msgr2=0x564b4faf6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.681+0000 7fea15d9b640 1 --2- 172.21.15.67:0/3481074681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 0x564b4faf6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.681+0000 7fea15d9b640 1 -- 172.21.15.67:0/3481074681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b4faf7520 con 0x564b4f70a1b0 2024-07-31T15:02:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.681+0000 7fea15d9b640 1 --2- 172.21.15.67:0/3481074681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4faf6aa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea0c005020 tx=0x7fea0c005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=739d0f9cca659938 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.681+0000 7fe9fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:02:05.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.682+0000 7fe9fdffb640 1 -- 172.21.15.67:0/3481074681 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c022050 con 0x564b4f70a1b0 2024-07-31T15:02:05.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.682+0000 7fe9fdffb640 1 -- 172.21.15.67:0/3481074681 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9379+0+0 (secure 0 0 0) 0x7fea0c004ad0 con 0x564b4f70a1b0 2024-07-31T15:02:05.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.682+0000 7fea22f20740 1 -- 172.21.15.67:0/3481074681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea0c0054d0 con 0x564b4f70a1b0 2024-07-31T15:02:05.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.682+0000 7fe9fdffb640 1 -- 172.21.15.67:0/3481074681 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fea0c039050 con 0x564b4f70a1b0 2024-07-31T15:02:05.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.682+0000 7fea22f20740 10 client.27469 fetch_fsmap learned FSMap version 58 2024-07-31T15:02:05.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.682+0000 7fea22f20740 1 -- 172.21.15.67:0/3481074681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x564b4fb66a70 con 0x564b4f70a1b0 2024-07-31T15:02:05.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.683+0000 7fe9fdffb640 1 -- 172.21.15.67:0/3481074681 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea0c0054d0 con 0x564b4f70a1b0 2024-07-31T15:02:05.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.683+0000 7fe9fdffb640 1 -- 172.21.15.67:0/3481074681 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7fea0c003bc0 con 0x564b4f70a1b0 2024-07-31T15:02:05.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.683+0000 7fea22f20740 10 client.27469 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:02:05.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.683+0000 7fea22f20740 10 client.27469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:02:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.683+0000 7fea22f20740 20 client.27469 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.683+0000 7fea22f20740 1 --2- 172.21.15.67:0/3481074681 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b4f733070 0x564b4fb691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.683+0000 7fea22f20740 4 client.27469 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:02:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.683+0000 7fea22f20740 1 -- 172.21.15.67:0/3481074681 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564b4fa4c550 con 0x564b4f733070 2024-07-31T15:02:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.684+0000 7fea1659c640 1 --2- 172.21.15.67:0/3481074681 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b4f733070 0x564b4fb691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.684+0000 7fea1659c640 1 --2- 172.21.15.67:0/3481074681 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b4f733070 0x564b4fb691c0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b19a067c27da1735 server_cookie=6e80f59a6ef6c7c1 in_seq=0 out_seq=0 2024-07-31T15:02:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.684+0000 7fe9fdffb640 10 client.27469 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fe9fdffb640 1 -- 172.21.15.67:0/3481074681 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x564b4fa4c550 con 0x564b4f733070 2024-07-31T15:02:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fe9fdffb640 10 client.27469 handle_command_reply: tid=0 2024-07-31T15:02:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fea22f20740 1 client.27469 shutdown 2024-07-31T15:02:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fea22f20740 1 -- 172.21.15.67:0/3481074681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 msgr2=0x564b4faf6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fea22f20740 1 --2- 172.21.15.67:0/3481074681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4faf6aa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea0c005020 tx=0x7fea0c005050 comp rx=0 tx=0).stop 2024-07-31T15:02:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fea22f20740 1 -- 172.21.15.67:0/3481074681 shutdown_connections 2024-07-31T15:02:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fea22f20740 1 --2- 172.21.15.67:0/3481074681 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b4f733070 0x564b4fb691c0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fea22f20740 1 --2- 172.21.15.67:0/3481074681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b4f712390 0x564b4faf6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fea22f20740 1 --2- 172.21.15.67:0/3481074681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b4f70a1b0 0x564b4faf6aa0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fea22f20740 1 --2- 172.21.15.67:0/3481074681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b4f6f19f0 0x564b4faf9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fea22f20740 1 -- 172.21.15.67:0/3481074681 >> 172.21.15.67:0/3481074681 conn(0x564b4f727460 msgr2=0x564b4fad9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fe9fdffb640 0 client.27469 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fea22f20740 1 -- 172.21.15.67:0/3481074681 shutdown_connections 2024-07-31T15:02:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fea22f20740 1 -- 172.21.15.67:0/3481074681 wait complete. 2024-07-31T15:02:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.686+0000 7fea22f20740 20 client.27469 trim_cache size 0 max 16384 2024-07-31T15:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (14556 inodes in the stack)", 2024-07-31T15:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T15:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: "acb195c0-e291-456a-ac4b-1332fba0bea8": { 2024-07-31T15:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T15:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "acb195c0-e291-456a-ac4b-1332fba0bea8", 2024-07-31T15:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T15:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:02:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.687+0000 7fea1c9cf640 1 -- 172.21.15.67:0/3937552267 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fea000b3e40 msgr2=0x7fea000b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.687+0000 7fea1c9cf640 1 --2- 172.21.15.67:0/3937552267 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fea000b3e40 0x7fea000b6350 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea1002f500 tx=0x7fea1002f310 comp rx=0 tx=0).stop 2024-07-31T15:02:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.687+0000 7fea1c9cf640 1 -- 172.21.15.67:0/3937552267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea181019f0 msgr2=0x7fea18102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.687+0000 7fea1c9cf640 1 --2- 172.21.15.67:0/3937552267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea181019f0 0x7fea18102990 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fea0800ea60 tx=0x7fea0800ef20 comp rx=0 tx=0).stop 2024-07-31T15:02:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.687+0000 7fea1c9cf640 1 -- 172.21.15.67:0/3937552267 shutdown_connections 2024-07-31T15:02:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.687+0000 7fea1c9cf640 1 --2- 172.21.15.67:0/3937552267 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fea000b3e40 0x7fea000b6350 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.687+0000 7fea1c9cf640 1 --2- 172.21.15.67:0/3937552267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea18106620 0x7fea18103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.687+0000 7fea1c9cf640 1 --2- 172.21.15.67:0/3937552267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea18105cd0 0x7fea18102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.687+0000 7fea1c9cf640 1 --2- 172.21.15.67:0/3937552267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea181019f0 0x7fea18102990 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.687+0000 7fea1c9cf640 1 -- 172.21.15.67:0/3937552267 >> 172.21.15.67:0/3937552267 conn(0x7fea18077720 msgr2=0x7fea18106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.687+0000 7fea1c9cf640 1 -- 172.21.15.67:0/3937552267 shutdown_connections 2024-07-31T15:02:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:05.687+0000 7fea1c9cf640 1 -- 172.21.15.67:0/3937552267 wait complete. 2024-07-31T15:02:06.393 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:02:06 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:02:06.235+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000cde2 [...10,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders/ auth{4=1} v112 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0) n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:137623 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@5} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8dc52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v29>,<0x1000000cbec/selinux v646>,<0x10000000202/scripts v8820>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v112>,<0x1000000cbec/selinux v749>,<0x10000000202/scripts v11639>,<0x10000000201/linux-6.5.11 v1447>,<0x10000000006/t v777>,<0x10000000005/tmp v733>,<0x100000001fb/client.0 v717>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v700>,<0x10000000001/sv_1 v700>,<0x10000000000/qa v698>,<0x1/volumes v669>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:58.999116+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:02:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:06.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:06 smithi073 ceph-mon[36714]: osdmap e103: 12 total, 12 up, 12 in 2024-07-31T15:02:06.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:06 smithi077 ceph-mon[37327]: osdmap e103: 12 total, 12 up, 12 in 2024-07-31T15:02:06.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:06.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:06 smithi067 ceph-mon[32750]: osdmap e103: 12 total, 12 up, 12 in 2024-07-31T15:02:06.638 INFO:tasks.cephfs.filesystem:scrub status for tag:acb195c0-e291-456a-ac4b-1332fba0bea8 - {'path': '/', 'tag': 'acb195c0-e291-456a-ac4b-1332fba0bea8', 'options': 'recursive,force'} 2024-07-31T15:02:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: pgmap v743: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 871 KiB/s wr, 500 op/s 2024-07-31T15:02:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:07.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:07.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:07.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:07.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: pgmap v743: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 871 KiB/s wr, 500 op/s 2024-07-31T15:02:07.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:07.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:07.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:07.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:07.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:07.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:07.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:07.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: pgmap v743: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 871 KiB/s wr, 500 op/s 2024-07-31T15:02:07.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:07.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:07.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:07.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:07.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:07.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:07.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.711+0000 7fa7c164b640 1 -- 172.21.15.67:0/806546194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7bc1118d0 msgr2=0x7fa7bc109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.711+0000 7fa7c164b640 1 --2- 172.21.15.67:0/806546194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7bc1118d0 0x7fa7bc109df0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009900 tx=0x7fa7a402a4b0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.715+0000 7fa7c164b640 1 -- 172.21.15.67:0/806546194 shutdown_connections 2024-07-31T15:02:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.715+0000 7fa7c164b640 1 --2- 172.21.15.67:0/806546194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7bc11c5d0 0x7fa7bc11eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.715+0000 7fa7c164b640 1 --2- 172.21.15.67:0/806546194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7bc10a330 0x7fa7bc10a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.715+0000 7fa7c164b640 1 --2- 172.21.15.67:0/806546194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7bc1118d0 0x7fa7bc109df0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.715+0000 7fa7c164b640 1 -- 172.21.15.67:0/806546194 >> 172.21.15.67:0/806546194 conn(0x7fa7bc0707c0 msgr2=0x7fa7bc070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.718+0000 7fa7c164b640 1 -- 172.21.15.67:0/806546194 shutdown_connections 2024-07-31T15:02:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.720+0000 7fa7c164b640 1 -- 172.21.15.67:0/806546194 wait complete. 2024-07-31T15:02:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.721+0000 7fa7c164b640 1 Processor -- start 2024-07-31T15:02:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.725+0000 7fa7c164b640 1 -- start start 2024-07-31T15:02:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.725+0000 7fa7c164b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7bc10a330 0x7fa7bc1a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.725+0000 7fa7c164b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7bc1118d0 0x7fa7bc1a9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.726+0000 7fa7baffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7bc10a330 0x7fa7bc1a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.726+0000 7fa7baffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7bc10a330 0x7fa7bc1a8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43000/0 (socket says 172.21.15.67:43000) 2024-07-31T15:02:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.726+0000 7fa7c164b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7bc11c5d0 0x7fa7bc1ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.726+0000 7fa7baffd640 1 -- 172.21.15.67:0/2633877006 learned_addr learned my addr 172.21.15.67:0/2633877006 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.726+0000 7fa7c164b640 1 -- 172.21.15.67:0/2633877006 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc1222e0 con 0x7fa7bc11c5d0 2024-07-31T15:02:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.726+0000 7fa7c164b640 1 -- 172.21.15.67:0/2633877006 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc122160 con 0x7fa7bc1118d0 2024-07-31T15:02:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.726+0000 7fa7c164b640 1 -- 172.21.15.67:0/2633877006 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc122460 con 0x7fa7bc10a330 2024-07-31T15:02:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.730+0000 7fa7ba7fc640 1 --2- 172.21.15.67:0/2633877006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7bc1118d0 0x7fa7bc1a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.730+0000 7fa7baffd640 1 -- 172.21.15.67:0/2633877006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7bc1118d0 msgr2=0x7fa7bc1a9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.730+0000 7fa7baffd640 1 --2- 172.21.15.67:0/2633877006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7bc1118d0 0x7fa7bc1a9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.730+0000 7fa7baffd640 1 -- 172.21.15.67:0/2633877006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7bc11c5d0 msgr2=0x7fa7bc1ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.730+0000 7fa7baffd640 1 --2- 172.21.15.67:0/2633877006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7bc11c5d0 0x7fa7bc1ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.730+0000 7fa7baffd640 1 -- 172.21.15.67:0/2633877006 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7bc1aea20 con 0x7fa7bc10a330 2024-07-31T15:02:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.735+0000 7fa7baffd640 1 --2- 172.21.15.67:0/2633877006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7bc10a330 0x7fa7bc1a8af0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa7a402aaa0 tx=0x7fa7a4030730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.738+0000 7fa7b3fff640 1 -- 172.21.15.67:0/2633877006 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4050020 con 0x7fa7bc10a330 2024-07-31T15:02:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.738+0000 7fa7b3fff640 1 -- 172.21.15.67:0/2633877006 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7a4030950 con 0x7fa7bc10a330 2024-07-31T15:02:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.738+0000 7fa7c164b640 1 -- 172.21.15.67:0/2633877006 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7bc1aecb0 con 0x7fa7bc10a330 2024-07-31T15:02:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.738+0000 7fa7c164b640 1 -- 172.21.15.67:0/2633877006 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7bc1af160 con 0x7fa7bc10a330 2024-07-31T15:02:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.738+0000 7fa7b3fff640 1 -- 172.21.15.67:0/2633877006 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4035070 con 0x7fa7bc10a330 2024-07-31T15:02:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.746+0000 7fa7c8624740 1 Processor -- start 2024-07-31T15:02:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.748+0000 7fa7b3fff640 1 -- 172.21.15.67:0/2633877006 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fa7a4053020 con 0x7fa7bc10a330 2024-07-31T15:02:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.749+0000 7fa7b3fff640 1 --2- 172.21.15.67:0/2633877006 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa7a80b3f50 0x7fa7a80b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.750+0000 7fa7c8624740 1 -- start start 2024-07-31T15:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.750+0000 7fa7b3fff640 1 -- 172.21.15.67:0/2633877006 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9499+0+0 (secure 0 0 0) 0x7fa7a41160e0 con 0x7fa7bc10a330 2024-07-31T15:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.750+0000 7fa7c8624740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f6761b0 0x55a86fa18ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.750+0000 7fa7c8624740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f64d2f0 0x55a86fa19220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.750+0000 7fa7c8624740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa19760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.750+0000 7fa7c8624740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a86f98e3b0 con 0x55a86f6554d0 2024-07-31T15:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.750+0000 7fa7c8624740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a86f97d8b0 con 0x55a86f6761b0 2024-07-31T15:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.750+0000 7fa7c8624740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a86f9818f0 con 0x55a86f64d2f0 2024-07-31T15:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.751+0000 7fa7bb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa19760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.751+0000 7fa7bb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa19760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53446/0 (socket says 172.21.15.67:53446) 2024-07-31T15:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.751+0000 7fa7bb7fe640 1 -- 172.21.15.67:0/3128679208 learned_addr learned my addr 172.21.15.67:0/3128679208 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.751+0000 7fa7bb7fe640 1 --2- 172.21.15.67:0/2633877006 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa7a80b3f50 0x7fa7a80b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.752+0000 7fa7bb7fe640 1 -- 172.21.15.67:0/3128679208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f64d2f0 msgr2=0x55a86fa19220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.752+0000 7fa7bb7fe640 1 --2- 172.21.15.67:0/3128679208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f64d2f0 0x55a86fa19220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.752+0000 7fa7bb7fe640 1 -- 172.21.15.67:0/3128679208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f6761b0 msgr2=0x55a86fa18ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:02:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.752+0000 7fa7bb7fe640 1 --2- 172.21.15.67:0/3128679208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f6761b0 0x55a86fa18ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.752+0000 7fa7bb7fe640 1 -- 172.21.15.67:0/3128679208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86f9249a0 con 0x55a86f6554d0 2024-07-31T15:02:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.752+0000 7fa7bb7fe640 1 --2- 172.21.15.67:0/3128679208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa19760 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0134d0 tx=0x7fa7ac013990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915d869a60ec58d0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.753+0000 7fa7b17fa640 1 -- 172.21.15.67:0/3128679208 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac020070 con 0x55a86f6554d0 2024-07-31T15:02:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.753+0000 7fa7b17fa640 1 -- 172.21.15.67:0/3128679208 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7ac011a30 con 0x55a86f6554d0 2024-07-31T15:02:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.754+0000 7fa7c8624740 1 -- 172.21.15.67:0/3128679208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 msgr2=0x55a86fa19760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.754+0000 7fa7c8624740 1 --2- 172.21.15.67:0/3128679208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa19760 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0134d0 tx=0x7fa7ac013990 comp rx=0 tx=0).stop 2024-07-31T15:02:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.755+0000 7fa7bb7fe640 1 --2- 172.21.15.67:0/2633877006 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa7a80b3f50 0x7fa7a80b6460 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac01d000 tx=0x7fa7ac01bdd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.759+0000 7fa7c8624740 1 -- 172.21.15.67:0/3128679208 shutdown_connections 2024-07-31T15:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.759+0000 7fa7c8624740 1 --2- 172.21.15.67:0/3128679208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa19760 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.759+0000 7fa7c8624740 1 --2- 172.21.15.67:0/3128679208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f64d2f0 0x55a86fa19220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.759+0000 7fa7c8624740 1 --2- 172.21.15.67:0/3128679208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f6761b0 0x55a86fa18ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.759+0000 7fa7c8624740 1 -- 172.21.15.67:0/3128679208 >> 172.21.15.67:0/3128679208 conn(0x55a86f66a300 msgr2=0x55a86fa16bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.765+0000 7fa7c8624740 1 -- 172.21.15.67:0/3128679208 shutdown_connections 2024-07-31T15:02:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.767+0000 7fa7c8624740 1 -- 172.21.15.67:0/3128679208 wait complete. 2024-07-31T15:02:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.767+0000 7fa7c8624740 1 -- start start 2024-07-31T15:02:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.768+0000 7fa7c8624740 1 Processor -- start 2024-07-31T15:02:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.769+0000 7fa7c8624740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f64d2f0 0x55a86fa244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.769+0000 7fa7c8624740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.769+0000 7fa7c8624740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f6761b0 0x55a86fa30880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.769+0000 7fa7c8624740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a86f8b7ab0 con 0x55a86f6554d0 2024-07-31T15:02:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.769+0000 7fa7c8624740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a86f905cf0 con 0x55a86f64d2f0 2024-07-31T15:02:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.769+0000 7fa7c8624740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a86fa274b0 con 0x55a86f6761b0 2024-07-31T15:02:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.769+0000 7fa7c8624740 10 client.0 _ll_register_callbacks cb 0x55a86f9779f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.770+0000 7fa7bb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f6761b0 0x55a86fa30880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.770+0000 7fa7ba7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.770+0000 7fa7ba7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa249e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53452/0 (socket says 172.21.15.67:53452) 2024-07-31T15:02:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.770+0000 7fa7ba7fc640 1 -- 172.21.15.67:0/1538680899 learned_addr learned my addr 172.21.15.67:0/1538680899 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.771+0000 7fa7ba7fc640 1 -- 172.21.15.67:0/1538680899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f6761b0 msgr2=0x55a86fa30880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.771+0000 7fa7ba7fc640 1 --2- 172.21.15.67:0/1538680899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f6761b0 0x55a86fa30880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.771+0000 7fa7ba7fc640 1 -- 172.21.15.67:0/1538680899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f64d2f0 msgr2=0x55a86fa244a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:02:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.771+0000 7fa7ba7fc640 1 --2- 172.21.15.67:0/1538680899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f64d2f0 0x55a86fa244a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.771+0000 7fa7ba7fc640 1 -- 172.21.15.67:0/1538680899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86f6e1790 con 0x55a86f6554d0 2024-07-31T15:02:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.772+0000 7fa7ba7fc640 1 --2- 172.21.15.67:0/1538680899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa249e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400c9c0 tx=0x7fa7b400ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6e37c608693052 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.772+0000 7fa7a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:02:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.773+0000 7fa7a27fc640 1 -- 172.21.15.67:0/1538680899 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b403c060 con 0x55a86f6554d0 2024-07-31T15:02:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.773+0000 7fa7c8624740 1 -- 172.21.15.67:0/1538680899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7ac011a30 con 0x55a86f6554d0 2024-07-31T15:02:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.773+0000 7fa7a27fc640 1 -- 172.21.15.67:0/1538680899 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9499+0+0 (secure 0 0 0) 0x7fa7b40079b0 con 0x55a86f6554d0 2024-07-31T15:02:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.773+0000 7fa7a27fc640 1 -- 172.21.15.67:0/1538680899 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa7b4043050 con 0x55a86f6554d0 2024-07-31T15:02:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.773+0000 7fa7c8624740 10 client.17940 fetch_fsmap learned FSMap version 58 2024-07-31T15:02:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.773+0000 7fa7c8624740 1 -- 172.21.15.67:0/1538680899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55a86f75a620 con 0x55a86f6554d0 2024-07-31T15:02:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.774+0000 7fa7a27fc640 1 -- 172.21.15.67:0/1538680899 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7ac011a30 con 0x55a86f6554d0 2024-07-31T15:02:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.774+0000 7fa7a27fc640 1 -- 172.21.15.67:0/1538680899 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7fa7b4037ca0 con 0x55a86f6554d0 2024-07-31T15:02:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.774+0000 7fa7c8624740 10 client.17940 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:02:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.774+0000 7fa7c8624740 10 client.17940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:02:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.774+0000 7fa7c8624740 20 client.17940 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.774+0000 7fa7c8624740 1 --2- 172.21.15.67:0/1538680899 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a86f634d80 0x55a86fa36dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.774+0000 7fa7c8624740 4 client.17940 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:02:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.774+0000 7fa7c8624740 1 -- 172.21.15.67:0/1538680899 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a86f990300 con 0x55a86f634d80 2024-07-31T15:02:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.775+0000 7fa7baffd640 1 --2- 172.21.15.67:0/1538680899 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a86f634d80 0x55a86fa36dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.788+0000 7fa7baffd640 1 --2- 172.21.15.67:0/1538680899 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a86f634d80 0x55a86fa36dd0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d5fb1c375a1d363 server_cookie=d70ff7dc5676e854 in_seq=0 out_seq=0 2024-07-31T15:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.788+0000 7fa7a27fc640 10 client.17940 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.792+0000 7fa7a27fc640 1 -- 172.21.15.67:0/1538680899 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a86f990300 con 0x55a86f634d80 2024-07-31T15:02:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.792+0000 7fa7a27fc640 10 client.17940 handle_command_reply: tid=0 2024-07-31T15:02:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.792+0000 7fa7c8624740 1 client.17940 shutdown 2024-07-31T15:02:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.792+0000 7fa7c8624740 1 -- 172.21.15.67:0/1538680899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 msgr2=0x55a86fa249e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.792+0000 7fa7c8624740 1 --2- 172.21.15.67:0/1538680899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa249e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400c9c0 tx=0x7fa7b400ce80 comp rx=0 tx=0).stop 2024-07-31T15:02:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.794+0000 7fa7c8624740 1 -- 172.21.15.67:0/1538680899 shutdown_connections 2024-07-31T15:02:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.794+0000 7fa7c8624740 1 --2- 172.21.15.67:0/1538680899 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a86f634d80 0x55a86fa36dd0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.794+0000 7fa7c8624740 1 --2- 172.21.15.67:0/1538680899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f6761b0 0x55a86fa30880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.794+0000 7fa7c8624740 1 --2- 172.21.15.67:0/1538680899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa249e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.794+0000 7fa7c8624740 1 --2- 172.21.15.67:0/1538680899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f64d2f0 0x55a86fa244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.794+0000 7fa7c8624740 1 -- 172.21.15.67:0/1538680899 >> 172.21.15.67:0/1538680899 conn(0x55a86f66a300 msgr2=0x55a86fa19d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.794+0000 7fa7a27fc640 0 client.17940 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.795+0000 7fa7c8624740 1 -- 172.21.15.67:0/1538680899 shutdown_connections 2024-07-31T15:02:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.799+0000 7fa7c8624740 1 -- 172.21.15.67:0/1538680899 wait complete. 2024-07-31T15:02:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.803+0000 7fa7c8624740 20 client.17940 trim_cache size 0 max 16384 2024-07-31T15:02:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.839+0000 7fa7c8624740 1 Processor -- start 2024-07-31T15:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.840+0000 7fa7c8624740 1 -- start start 2024-07-31T15:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.841+0000 7fa7c8624740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f64d2f0 0x55a86fa1a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.841+0000 7fa7c8624740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f6554d0 0x55a86fa260d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.841+0000 7fa7c8624740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6761b0 0x55a86fa30670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.842+0000 7fa7c8624740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c0100c0 con 0x55a86f6761b0 2024-07-31T15:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.842+0000 7fa7c8624740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c011cc0 con 0x55a86f6554d0 2024-07-31T15:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.842+0000 7fa7c8624740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c010790 con 0x55a86f64d2f0 2024-07-31T15:02:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.842+0000 7fa7ba7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f6554d0 0x55a86fa260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.842+0000 7fa7ba7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f6554d0 0x55a86fa260d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:42442/0 (socket says 172.21.15.67:42442) 2024-07-31T15:02:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.842+0000 7fa7ba7fc640 1 -- 172.21.15.67:0/284713030 learned_addr learned my addr 172.21.15.67:0/284713030 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.842+0000 7fa7baffd640 1 --2- 172.21.15.67:0/284713030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f64d2f0 0x55a86fa1a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.842+0000 7fa7ba7fc640 1 -- 172.21.15.67:0/284713030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f64d2f0 msgr2=0x55a86fa1a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.842+0000 7fa7ba7fc640 1 --2- 172.21.15.67:0/284713030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f64d2f0 0x55a86fa1a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.842+0000 7fa7ba7fc640 1 -- 172.21.15.67:0/284713030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6761b0 msgr2=0x55a86fa30670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:02:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.842+0000 7fa7ba7fc640 1 --2- 172.21.15.67:0/284713030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6761b0 0x55a86fa30670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.842+0000 7fa7ba7fc640 1 -- 172.21.15.67:0/284713030 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86fa17570 con 0x55a86f6554d0 2024-07-31T15:02:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.843+0000 7fa7ba7fc640 1 --2- 172.21.15.67:0/284713030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f6554d0 0x55a86fa260d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009a10 tx=0x7fa7b4009a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6652ff9a8ccba35d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.843+0000 7fa7b0ff9640 1 -- 172.21.15.67:0/284713030 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b404b030 con 0x55a86f6554d0 2024-07-31T15:02:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.843+0000 7fa7b0ff9640 1 -- 172.21.15.67:0/284713030 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7b4007e60 con 0x55a86f6554d0 2024-07-31T15:02:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.843+0000 7fa7b0ff9640 1 -- 172.21.15.67:0/284713030 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400ae20 con 0x55a86f6554d0 2024-07-31T15:02:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.844+0000 7fa7c8624740 1 -- 172.21.15.67:0/284713030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f6554d0 msgr2=0x55a86fa260d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.845+0000 7fa7c8624740 1 --2- 172.21.15.67:0/284713030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f6554d0 0x55a86fa260d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009a10 tx=0x7fa7b4009a40 comp rx=0 tx=0).stop 2024-07-31T15:02:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.849+0000 7fa7c8624740 1 -- 172.21.15.67:0/284713030 shutdown_connections 2024-07-31T15:02:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.849+0000 7fa7c8624740 1 --2- 172.21.15.67:0/284713030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6761b0 0x55a86fa30670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.849+0000 7fa7c8624740 1 --2- 172.21.15.67:0/284713030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f6554d0 0x55a86fa260d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.849+0000 7fa7c8624740 1 --2- 172.21.15.67:0/284713030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f64d2f0 0x55a86fa1a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.849+0000 7fa7c8624740 1 -- 172.21.15.67:0/284713030 >> 172.21.15.67:0/284713030 conn(0x55a86f66a300 msgr2=0x55a86fa259c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.851+0000 7fa7c8624740 1 -- 172.21.15.67:0/284713030 shutdown_connections 2024-07-31T15:02:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.854+0000 7fa7c8624740 1 -- 172.21.15.67:0/284713030 wait complete. 2024-07-31T15:02:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.855+0000 7fa7c8624740 1 -- start start 2024-07-31T15:02:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.855+0000 7fa7c8624740 1 Processor -- start 2024-07-31T15:02:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.859+0000 7fa7c8624740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f64d2f0 0x55a86fa2c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.859+0000 7fa7c8624740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa2cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.859+0000 7fa7c8624740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f6761b0 0x55a86fa297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.859+0000 7fa7c8624740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c0115f0 con 0x55a86f6554d0 2024-07-31T15:02:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.859+0000 7fa7c8624740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c010e60 con 0x55a86f64d2f0 2024-07-31T15:02:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.859+0000 7fa7c8624740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c00e0c0 con 0x55a86f6761b0 2024-07-31T15:02:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.859+0000 7fa7c8624740 10 client.0 _ll_register_callbacks cb 0x7fa7b40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.861+0000 7fa7ba7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa2cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.861+0000 7fa7ba7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa2cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53460/0 (socket says 172.21.15.67:53460) 2024-07-31T15:02:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.861+0000 7fa7ba7fc640 1 -- 172.21.15.67:0/547744737 learned_addr learned my addr 172.21.15.67:0/547744737 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.863+0000 7fa7baffd640 1 --2- 172.21.15.67:0/547744737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f64d2f0 0x55a86fa2c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.864+0000 7fa7bb7fe640 1 --2- 172.21.15.67:0/547744737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f6761b0 0x55a86fa297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.864+0000 7fa7baffd640 1 -- 172.21.15.67:0/547744737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f6761b0 msgr2=0x55a86fa297b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.864+0000 7fa7baffd640 1 --2- 172.21.15.67:0/547744737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f6761b0 0x55a86fa297b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.864+0000 7fa7baffd640 1 -- 172.21.15.67:0/547744737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 msgr2=0x55a86fa2cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.864+0000 7fa7baffd640 1 --2- 172.21.15.67:0/547744737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa2cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.864+0000 7fa7baffd640 1 -- 172.21.15.67:0/547744737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86fa1b210 con 0x55a86f64d2f0 2024-07-31T15:02:07.869 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.867+0000 7fa7baffd640 1 --2- 172.21.15.67:0/547744737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f64d2f0 0x55a86fa2c7e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4113400 tx=0x7fa7a4003730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=938004452532ac3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:07.873 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.869+0000 7fa7a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:02:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.873+0000 7fa7a27fc640 1 -- 172.21.15.67:0/547744737 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a40f6020 con 0x55a86f64d2f0 2024-07-31T15:02:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.873+0000 7fa7c8624740 1 -- 172.21.15.67:0/547744737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7b4007e60 con 0x55a86f64d2f0 2024-07-31T15:02:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.873+0000 7fa7a27fc640 1 -- 172.21.15.67:0/547744737 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9499+0+0 (secure 0 0 0) 0x7fa7a40d9380 con 0x55a86f64d2f0 2024-07-31T15:02:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.876+0000 7fa7a27fc640 1 -- 172.21.15.67:0/547744737 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa7a40dc8a0 con 0x55a86f64d2f0 2024-07-31T15:02:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.881+0000 7fa7c8624740 10 client.27500 fetch_fsmap learned FSMap version 58 2024-07-31T15:02:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.881+0000 7fa7c8624740 1 -- 172.21.15.67:0/547744737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55a86fa9dc30 con 0x55a86f64d2f0 2024-07-31T15:02:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.887+0000 7fa7a27fc640 1 -- 172.21.15.67:0/547744737 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7b4007e60 con 0x55a86f64d2f0 2024-07-31T15:02:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.887+0000 7fa7a27fc640 1 -- 172.21.15.67:0/547744737 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7fa7a404a040 con 0x55a86f64d2f0 2024-07-31T15:02:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.887+0000 7fa7c8624740 10 client.27500 fetch_fsmap finished waiting for FSMap version 58 2024-07-31T15:02:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.887+0000 7fa7c8624740 10 client.27500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:02:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.887+0000 7fa7c8624740 20 client.27500 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.887+0000 7fa7c8624740 1 --2- 172.21.15.67:0/547744737 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a86f60a1c0 0x55a86faa0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.887+0000 7fa7c8624740 4 client.27500 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:02:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.887+0000 7fa7c8624740 1 -- 172.21.15.67:0/547744737 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a86f98bda0 con 0x55a86f60a1c0 2024-07-31T15:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.893+0000 7fa7ba7fc640 1 --2- 172.21.15.67:0/547744737 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a86f60a1c0 0x55a86faa0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.903+0000 7fa7ba7fc640 1 --2- 172.21.15.67:0/547744737 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a86f60a1c0 0x55a86faa0490 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e990756a7001260 server_cookie=2b0b97225fb106ed in_seq=0 out_seq=0 2024-07-31T15:02:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:07.905+0000 7fa7a27fc640 10 client.27500 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.272+0000 7fa7a27fc640 1 -- 172.21.15.67:0/547744737 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a86f98bda0 con 0x55a86f60a1c0 2024-07-31T15:02:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.272+0000 7fa7a27fc640 10 client.27500 handle_command_reply: tid=0 2024-07-31T15:02:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.272+0000 7fa7c8624740 1 client.27500 shutdown 2024-07-31T15:02:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.272+0000 7fa7c8624740 1 -- 172.21.15.67:0/547744737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f64d2f0 msgr2=0x55a86fa2c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.272+0000 7fa7c8624740 1 --2- 172.21.15.67:0/547744737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f64d2f0 0x55a86fa2c7e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4113400 tx=0x7fa7a4003730 comp rx=0 tx=0).stop 2024-07-31T15:02:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.272+0000 7fa7c8624740 1 -- 172.21.15.67:0/547744737 shutdown_connections 2024-07-31T15:02:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.272+0000 7fa7c8624740 1 --2- 172.21.15.67:0/547744737 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a86f60a1c0 0x55a86faa0490 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.272+0000 7fa7c8624740 1 --2- 172.21.15.67:0/547744737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86f6761b0 0x55a86fa297b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.272+0000 7fa7c8624740 1 --2- 172.21.15.67:0/547744737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a86f6554d0 0x55a86fa2cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.272+0000 7fa7c8624740 1 --2- 172.21.15.67:0/547744737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a86f64d2f0 0x55a86fa2c7e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.272+0000 7fa7c8624740 1 -- 172.21.15.67:0/547744737 >> 172.21.15.67:0/547744737 conn(0x55a86f66a300 msgr2=0x55a86fa1a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.272+0000 7fa7a27fc640 0 client.27500 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.272+0000 7fa7c8624740 1 -- 172.21.15.67:0/547744737 shutdown_connections 2024-07-31T15:02:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.273+0000 7fa7c8624740 1 -- 172.21.15.67:0/547744737 wait complete. 2024-07-31T15:02:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.273+0000 7fa7c8624740 20 client.27500 trim_cache size 0 max 16384 2024-07-31T15:02:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:02:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:02:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:02:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:02:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.274+0000 7fa7c164b640 1 -- 172.21.15.67:0/2633877006 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa7a80b3f50 msgr2=0x7fa7a80b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.274+0000 7fa7c164b640 1 --2- 172.21.15.67:0/2633877006 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa7a80b3f50 0x7fa7a80b6460 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac01d000 tx=0x7fa7ac01bdd0 comp rx=0 tx=0).stop 2024-07-31T15:02:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.274+0000 7fa7c164b640 1 -- 172.21.15.67:0/2633877006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7bc10a330 msgr2=0x7fa7bc1a8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.274+0000 7fa7c164b640 1 --2- 172.21.15.67:0/2633877006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7bc10a330 0x7fa7bc1a8af0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa7a402aaa0 tx=0x7fa7a4030730 comp rx=0 tx=0).stop 2024-07-31T15:02:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.274+0000 7fa7c164b640 1 -- 172.21.15.67:0/2633877006 shutdown_connections 2024-07-31T15:02:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.274+0000 7fa7c164b640 1 --2- 172.21.15.67:0/2633877006 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa7a80b3f50 0x7fa7a80b6460 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.274+0000 7fa7c164b640 1 --2- 172.21.15.67:0/2633877006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7bc11c5d0 0x7fa7bc1ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.274+0000 7fa7c164b640 1 --2- 172.21.15.67:0/2633877006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7bc1118d0 0x7fa7bc1a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.274+0000 7fa7c164b640 1 --2- 172.21.15.67:0/2633877006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7bc10a330 0x7fa7bc1a8af0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.275+0000 7fa7c164b640 1 -- 172.21.15.67:0/2633877006 >> 172.21.15.67:0/2633877006 conn(0x7fa7bc0707c0 msgr2=0x7fa7bc11b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.275+0000 7fa7c164b640 1 -- 172.21.15.67:0/2633877006 shutdown_connections 2024-07-31T15:02:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:08.275+0000 7fa7c164b640 1 -- 172.21.15.67:0/2633877006 wait complete. 2024-07-31T15:02:08.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:08 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:02:08.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:08 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:02:08.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:08 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:02:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:09 smithi073 ceph-mon[36714]: pgmap v745: 129 pgs: 13 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 111 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 522 KiB/s wr, 731 op/s 2024-07-31T15:02:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:09 smithi077 ceph-mon[37327]: pgmap v745: 129 pgs: 13 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 111 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 522 KiB/s wr, 731 op/s 2024-07-31T15:02:09.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:09.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:09.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:09 smithi067 ceph-mon[32750]: pgmap v745: 129 pgs: 13 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 111 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 522 KiB/s wr, 731 op/s 2024-07-31T15:02:09.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:09.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:11 smithi073 ceph-mon[36714]: pgmap v746: 129 pgs: 41 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 74 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 141 KiB/s wr, 933 op/s 2024-07-31T15:02:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:11 smithi073 ceph-mon[36714]: osdmap e104: 12 total, 12 up, 12 in 2024-07-31T15:02:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:11 smithi077 ceph-mon[37327]: pgmap v746: 129 pgs: 41 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 74 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 141 KiB/s wr, 933 op/s 2024-07-31T15:02:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:11 smithi077 ceph-mon[37327]: osdmap e104: 12 total, 12 up, 12 in 2024-07-31T15:02:11.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:11 smithi067 ceph-mon[32750]: pgmap v746: 129 pgs: 41 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 74 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 141 KiB/s wr, 933 op/s 2024-07-31T15:02:11.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:11 smithi067 ceph-mon[32750]: osdmap e104: 12 total, 12 up, 12 in 2024-07-31T15:02:13.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:13 smithi073 ceph-mon[36714]: pgmap v748: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 302 KiB/s wr, 1.05k op/s 2024-07-31T15:02:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:13 smithi077 ceph-mon[37327]: pgmap v748: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 302 KiB/s wr, 1.05k op/s 2024-07-31T15:02:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:13.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:13 smithi067 ceph-mon[32750]: pgmap v748: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 302 KiB/s wr, 1.05k op/s 2024-07-31T15:02:13.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:13.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:14.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:14.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:14.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:14.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:14.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:14.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:14.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:14.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:14.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:14.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:14.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:14.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:15 smithi073 ceph-mon[36714]: pgmap v749: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 261 KiB/s wr, 924 op/s 2024-07-31T15:02:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:15.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:15 smithi077 ceph-mon[37327]: pgmap v749: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 261 KiB/s wr, 924 op/s 2024-07-31T15:02:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:15.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:15.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:15.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:15 smithi067 ceph-mon[32750]: pgmap v749: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 261 KiB/s wr, 924 op/s 2024-07-31T15:02:15.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:15.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:15.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:17 smithi073 ceph-mon[36714]: pgmap v750: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 245 KiB/s wr, 847 op/s 2024-07-31T15:02:17.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:17 smithi077 ceph-mon[37327]: pgmap v750: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 245 KiB/s wr, 847 op/s 2024-07-31T15:02:17.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:17 smithi067 ceph-mon[32750]: pgmap v750: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 245 KiB/s wr, 847 op/s 2024-07-31T15:02:18.110 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:18.110 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:18.110 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:18.110 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:18.110 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:18.110 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:18.110 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:18.110 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:18.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:18.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:18.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:18.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:18.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:18.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:18.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:18.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:19 smithi073 ceph-mon[36714]: pgmap v751: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 236 KiB/s wr, 634 op/s 2024-07-31T15:02:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:19.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:19 smithi077 ceph-mon[37327]: pgmap v751: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 236 KiB/s wr, 634 op/s 2024-07-31T15:02:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:19.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:19 smithi067 ceph-mon[32750]: pgmap v751: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 236 KiB/s wr, 634 op/s 2024-07-31T15:02:19.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:19.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:19.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:19.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:20.053 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 15:02:19 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T15:02:19.728+0000 7f4cfe9c3640 -1 calc_pg_upmaps abort due to max <= 0 2024-07-31T15:02:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:20 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:02:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:20.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:20.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:20 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:02:20.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:20.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:20.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:20 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:02:20.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:20.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:21.388 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:21 smithi077 ceph-mon[37327]: pgmap v752: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 226 KiB/s wr, 398 op/s 2024-07-31T15:02:21.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:21 smithi073 ceph-mon[36714]: pgmap v752: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 226 KiB/s wr, 398 op/s 2024-07-31T15:02:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.438+0000 7f2b8ea1c640 1 -- 172.21.15.67:0/4029346960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b88107d20 msgr2=0x7f2b88108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.438+0000 7f2b8ea1c640 1 --2- 172.21.15.67:0/4029346960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b88107d20 0x7f2b88108130 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f2b7801a910 tx=0x7f2b7803b420 comp rx=0 tx=0).stop 2024-07-31T15:02:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.439+0000 7f2b8ea1c640 1 -- 172.21.15.67:0/4029346960 shutdown_connections 2024-07-31T15:02:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.439+0000 7f2b8ea1c640 1 --2- 172.21.15.67:0/4029346960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b88108670 0x7f2b8810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.439+0000 7f2b8ea1c640 1 --2- 172.21.15.67:0/4029346960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b88107d20 0x7f2b88108130 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.439+0000 7f2b8ea1c640 1 --2- 172.21.15.67:0/4029346960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b881023f0 0x7f2b88102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.439+0000 7f2b8ea1c640 1 -- 172.21.15.67:0/4029346960 >> 172.21.15.67:0/4029346960 conn(0x7f2b880fddf0 msgr2=0x7f2b88100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.441+0000 7f2b8ea1c640 1 -- 172.21.15.67:0/4029346960 shutdown_connections 2024-07-31T15:02:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.441+0000 7f2b8ea1c640 1 -- 172.21.15.67:0/4029346960 wait complete. 2024-07-31T15:02:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.442+0000 7f2b8ea1c640 1 Processor -- start 2024-07-31T15:02:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.442+0000 7f2b8ea1c640 1 -- start start 2024-07-31T15:02:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b8ea1c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b881023f0 0x7f2b88074740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b8ea1c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b88107d20 0x7f2b8806fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b8ea1c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b88108670 0x7f2b8806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b8ea1c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8806df90 con 0x7f2b88107d20 2024-07-31T15:02:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b8ea1c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8806de10 con 0x7f2b881023f0 2024-07-31T15:02:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b8ea1c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8806e110 con 0x7f2b88108670 2024-07-31T15:02:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b8cfb9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b88108670 0x7f2b8806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b8cfb9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b88108670 0x7f2b8806ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34032/0 (socket says 172.21.15.67:34032) 2024-07-31T15:02:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b8cfb9640 1 -- 172.21.15.67:0/3510532579 learned_addr learned my addr 172.21.15.67:0/3510532579 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b877fe640 1 --2- 172.21.15.67:0/3510532579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b88107d20 0x7f2b8806fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b8cfb9640 1 -- 172.21.15.67:0/3510532579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b881023f0 msgr2=0x7f2b88074740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:02:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b8cfb9640 1 --2- 172.21.15.67:0/3510532579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b881023f0 0x7f2b88074740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b8cfb9640 1 -- 172.21.15.67:0/3510532579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b88107d20 msgr2=0x7f2b8806fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b8cfb9640 1 --2- 172.21.15.67:0/3510532579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b88107d20 0x7f2b8806fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.443+0000 7f2b8cfb9640 1 -- 172.21.15.67:0/3510532579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b880705b0 con 0x7f2b88108670 2024-07-31T15:02:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.444+0000 7f2b877fe640 1 --2- 172.21.15.67:0/3510532579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b88107d20 0x7f2b8806fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:02:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.444+0000 7f2b8cfb9640 1 --2- 172.21.15.67:0/3510532579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b88108670 0x7f2b8806ff40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000bfa0 tx=0x7f2b80009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.446+0000 7f2b857fa640 1 -- 172.21.15.67:0/3510532579 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8001a070 con 0x7f2b88108670 2024-07-31T15:02:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.446+0000 7f2b857fa640 1 -- 172.21.15.67:0/3510532579 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b80015560 con 0x7f2b88108670 2024-07-31T15:02:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.446+0000 7f2b8ea1c640 1 -- 172.21.15.67:0/3510532579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b881bd850 con 0x7f2b88108670 2024-07-31T15:02:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.446+0000 7f2b8ea1c640 1 -- 172.21.15.67:0/3510532579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b881bdc50 con 0x7f2b88108670 2024-07-31T15:02:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.450+0000 7f2b959ec740 1 Processor -- start 2024-07-31T15:02:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.451+0000 7f2b857fa640 1 -- 172.21.15.67:0/3510532579 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80005280 con 0x7f2b88108670 2024-07-31T15:02:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.453+0000 7f2b959ec740 1 -- start start 2024-07-31T15:02:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.453+0000 7f2b857fa640 1 -- 172.21.15.67:0/3510532579 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2b800054a0 con 0x7f2b88108670 2024-07-31T15:02:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.454+0000 7f2b857fa640 1 --2- 172.21.15.67:0/3510532579 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2b700b3f80 0x7f2b700b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.454+0000 7f2b857fa640 1 -- 172.21.15.67:0/3510532579 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9439+0+0 (secure 0 0 0) 0x7f2b800e49c0 con 0x7f2b88108670 2024-07-31T15:02:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.455+0000 7f2b877fe640 1 --2- 172.21.15.67:0/3510532579 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2b700b3f80 0x7f2b700b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.456+0000 7f2b959ec740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503fe1b0 0x55fa507a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.456+0000 7f2b959ec740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503d52f0 0x55fa507a9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.456+0000 7f2b959ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503dd4d0 0x55fa507a9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.456+0000 7f2b959ec740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fa507163b0 con 0x55fa503fe1b0 2024-07-31T15:02:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.456+0000 7f2b959ec740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fa507058b0 con 0x55fa503d52f0 2024-07-31T15:02:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.456+0000 7f2b959ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fa507098f0 con 0x55fa503dd4d0 2024-07-31T15:02:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.456+0000 7f2b8cfb9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503dd4d0 0x55fa507a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.456+0000 7f2b8cfb9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503dd4d0 0x55fa507a9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34040/0 (socket says 172.21.15.67:34040) 2024-07-31T15:02:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.457+0000 7f2b8cfb9640 1 -- 172.21.15.67:0/968695141 learned_addr learned my addr 172.21.15.67:0/968695141 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.457+0000 7f2b877fe640 1 --2- 172.21.15.67:0/3510532579 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2b700b3f80 0x7f2b700b6490 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2b88101640 tx=0x7f2b78005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.457+0000 7f2b8cfb9640 1 -- 172.21.15.67:0/968695141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503d52f0 msgr2=0x55fa507a9400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:02:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.457+0000 7f2b8cfb9640 1 --2- 172.21.15.67:0/968695141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503d52f0 0x55fa507a9400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.457+0000 7f2b8cfb9640 1 -- 172.21.15.67:0/968695141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503fe1b0 msgr2=0x55fa507a8ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:02:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.457+0000 7f2b8cfb9640 1 --2- 172.21.15.67:0/968695141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503fe1b0 0x55fa507a8ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.457+0000 7f2b8cfb9640 1 -- 172.21.15.67:0/968695141 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa506ac9a0 con 0x55fa503dd4d0 2024-07-31T15:02:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.457+0000 7f2b8cfb9640 1 --2- 172.21.15.67:0/968695141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503dd4d0 0x55fa507a9940 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2b800a8560 tx=0x7f2b800abb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f419b2f4b604e658 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.458+0000 7f2b6e7fc640 1 -- 172.21.15.67:0/968695141 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80023030 con 0x55fa503dd4d0 2024-07-31T15:02:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.458+0000 7f2b6e7fc640 1 -- 172.21.15.67:0/968695141 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b800b43a0 con 0x55fa503dd4d0 2024-07-31T15:02:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.458+0000 7f2b6e7fc640 1 -- 172.21.15.67:0/968695141 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b800a6ec0 con 0x55fa503dd4d0 2024-07-31T15:02:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.459+0000 7f2b959ec740 1 -- 172.21.15.67:0/968695141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503dd4d0 msgr2=0x55fa507a9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.459+0000 7f2b959ec740 1 --2- 172.21.15.67:0/968695141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503dd4d0 0x55fa507a9940 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2b800a8560 tx=0x7f2b800abb30 comp rx=0 tx=0).stop 2024-07-31T15:02:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.461+0000 7f2b959ec740 1 -- 172.21.15.67:0/968695141 shutdown_connections 2024-07-31T15:02:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.461+0000 7f2b959ec740 1 --2- 172.21.15.67:0/968695141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503dd4d0 0x55fa507a9940 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.461+0000 7f2b959ec740 1 --2- 172.21.15.67:0/968695141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503d52f0 0x55fa507a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.461+0000 7f2b959ec740 1 --2- 172.21.15.67:0/968695141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503fe1b0 0x55fa507a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.461+0000 7f2b959ec740 1 -- 172.21.15.67:0/968695141 >> 172.21.15.67:0/968695141 conn(0x55fa503f2300 msgr2=0x55fa507a6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.463+0000 7f2b959ec740 1 -- 172.21.15.67:0/968695141 shutdown_connections 2024-07-31T15:02:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.467+0000 7f2b959ec740 1 -- 172.21.15.67:0/968695141 wait complete. 2024-07-31T15:02:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.467+0000 7f2b959ec740 1 -- start start 2024-07-31T15:02:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.468+0000 7f2b959ec740 1 Processor -- start 2024-07-31T15:02:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.468+0000 7f2b959ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503d52f0 0x55fa507a3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.468+0000 7f2b959ec740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503dd4d0 0x55fa507a3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.468+0000 7f2b959ec740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503fe1b0 0x55fa507c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.468+0000 7f2b959ec740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fa5063fab0 con 0x55fa503dd4d0 2024-07-31T15:02:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.468+0000 7f2b959ec740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fa5068dcf0 con 0x55fa503fe1b0 2024-07-31T15:02:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.468+0000 7f2b959ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fa507a6240 con 0x55fa503d52f0 2024-07-31T15:02:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.468+0000 7f2b87fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503d52f0 0x55fa507a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.469+0000 7f2b87fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503d52f0 0x55fa507a3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34042/0 (socket says 172.21.15.67:34042) 2024-07-31T15:02:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.469+0000 7f2b87fff640 1 -- 172.21.15.67:0/3415435760 learned_addr learned my addr 172.21.15.67:0/3415435760 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.469+0000 7f2b8cfb9640 1 --2- 172.21.15.67:0/3415435760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503fe1b0 0x55fa507c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.469+0000 7f2b959ec740 10 client.0 _ll_register_callbacks cb 0x55fa506ff3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.469+0000 7f2b87fff640 1 -- 172.21.15.67:0/3415435760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503fe1b0 msgr2=0x55fa507c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.469+0000 7f2b87fff640 1 --2- 172.21.15.67:0/3415435760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503fe1b0 0x55fa507c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.469+0000 7f2b87fff640 1 -- 172.21.15.67:0/3415435760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503dd4d0 msgr2=0x55fa507a3790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:02:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.469+0000 7f2b87fff640 1 --2- 172.21.15.67:0/3415435760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503dd4d0 0x55fa507a3790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.469+0000 7f2b87fff640 1 -- 172.21.15.67:0/3415435760 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa50469790 con 0x55fa503d52f0 2024-07-31T15:02:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.469+0000 7f2b87fff640 1 --2- 172.21.15.67:0/3415435760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503d52f0 0x55fa507a3250 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00c350 tx=0x7f2b7c00c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b15654ca705baa2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.470+0000 7f2b959ec740 1 -- 172.21.15.67:0/3415435760 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b800b43a0 con 0x55fa503d52f0 2024-07-31T15:02:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.470+0000 7f2b677fe640 10 client.27496 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:02:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.470+0000 7f2b677fe640 1 -- 172.21.15.67:0/3415435760 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c005220 con 0x55fa503d52f0 2024-07-31T15:02:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.470+0000 7f2b677fe640 1 -- 172.21.15.67:0/3415435760 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9439+0+0 (secure 0 0 0) 0x7f2b7c0053c0 con 0x55fa503d52f0 2024-07-31T15:02:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.470+0000 7f2b677fe640 1 -- 172.21.15.67:0/3415435760 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2b7c005600 con 0x55fa503d52f0 2024-07-31T15:02:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.472+0000 7f2b959ec740 10 client.27496 fetch_fsmap learned FSMap version 59 2024-07-31T15:02:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.472+0000 7f2b959ec740 1 -- 172.21.15.67:0/3415435760 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=59}) -- 0x55fa504e2620 con 0x55fa503d52f0 2024-07-31T15:02:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.473+0000 7f2b677fe640 1 -- 172.21.15.67:0/3415435760 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7f2b7c0047a0 con 0x55fa503d52f0 2024-07-31T15:02:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.473+0000 7f2b959ec740 10 client.27496 fetch_fsmap finished waiting for FSMap version 59 2024-07-31T15:02:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.473+0000 7f2b959ec740 10 client.27496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:02:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.473+0000 7f2b959ec740 20 client.27496 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.473+0000 7f2b959ec740 1 --2- 172.21.15.67:0/3415435760 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa503bcd80 0x55fa507cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.473+0000 7f2b959ec740 4 client.27496 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:02:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.473+0000 7f2b959ec740 1 -- 172.21.15.67:0/3415435760 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2b800b43a0 con 0x55fa503bcd80 2024-07-31T15:02:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.476+0000 7f2b877fe640 1 --2- 172.21.15.67:0/3415435760 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa503bcd80 0x55fa507cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.488+0000 7f2b877fe640 1 --2- 172.21.15.67:0/3415435760 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa503bcd80 0x55fa507cf590 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c71f3d328b5be4d6 server_cookie=57a780f58d6a47f9 in_seq=0 out_seq=0 2024-07-31T15:02:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.488+0000 7f2b677fe640 10 client.27496 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.493+0000 7f2b677fe640 1 -- 172.21.15.67:0/3415435760 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2b800b43a0 con 0x55fa503bcd80 2024-07-31T15:02:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.494+0000 7f2b677fe640 10 client.27496 handle_command_reply: tid=0 2024-07-31T15:02:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.494+0000 7f2b959ec740 1 client.27496 shutdown 2024-07-31T15:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.501+0000 7f2b959ec740 1 -- 172.21.15.67:0/3415435760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503d52f0 msgr2=0x55fa507a3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.501+0000 7f2b959ec740 1 --2- 172.21.15.67:0/3415435760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503d52f0 0x55fa507a3250 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00c350 tx=0x7f2b7c00c810 comp rx=0 tx=0).stop 2024-07-31T15:02:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.507+0000 7f2b959ec740 1 -- 172.21.15.67:0/3415435760 shutdown_connections 2024-07-31T15:02:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.507+0000 7f2b959ec740 1 --2- 172.21.15.67:0/3415435760 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa503bcd80 0x55fa507cf590 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.507+0000 7f2b959ec740 1 --2- 172.21.15.67:0/3415435760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503fe1b0 0x55fa507c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.507+0000 7f2b959ec740 1 --2- 172.21.15.67:0/3415435760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503dd4d0 0x55fa507a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.507+0000 7f2b959ec740 1 --2- 172.21.15.67:0/3415435760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503d52f0 0x55fa507a3250 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.507+0000 7f2b959ec740 1 -- 172.21.15.67:0/3415435760 >> 172.21.15.67:0/3415435760 conn(0x55fa503f2300 msgr2=0x55fa507a9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.507+0000 7f2b677fe640 0 client.27496 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.512+0000 7f2b959ec740 1 -- 172.21.15.67:0/3415435760 shutdown_connections 2024-07-31T15:02:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.515+0000 7f2b959ec740 1 -- 172.21.15.67:0/3415435760 wait complete. 2024-07-31T15:02:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.516+0000 7f2b959ec740 20 client.27496 trim_cache size 0 max 16384 2024-07-31T15:02:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:21 smithi067 ceph-mon[32750]: pgmap v752: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 226 KiB/s wr, 398 op/s 2024-07-31T15:02:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.554+0000 7f2b959ec740 1 Processor -- start 2024-07-31T15:02:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.554+0000 7f2b959ec740 1 -- start start 2024-07-31T15:02:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.555+0000 7f2b959ec740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503d52f0 0x55fa507a3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.555+0000 7f2b959ec740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503dd4d0 0x55fa507a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.555+0000 7f2b959ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503fe1b0 0x55fa507c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.555+0000 7f2b877fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503dd4d0 0x55fa507a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.555+0000 7f2b877fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503dd4d0 0x55fa507a1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40504/0 (socket says 172.21.15.67:40504) 2024-07-31T15:02:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.556+0000 7f2b87fff640 1 --2- 172.21.15.67:0/1201688121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503d52f0 0x55fa507a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.556+0000 7f2b8cfb9640 1 --2- 172.21.15.67:0/1201688121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503fe1b0 0x55fa507c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.555+0000 7f2b877fe640 1 -- 172.21.15.67:0/1201688121 learned_addr learned my addr 172.21.15.67:0/1201688121 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.556+0000 7f2b959ec740 1 -- 172.21.15.67:0/1201688121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7400ff70 con 0x55fa503d52f0 2024-07-31T15:02:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.556+0000 7f2b959ec740 1 -- 172.21.15.67:0/1201688121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74011b70 con 0x55fa503dd4d0 2024-07-31T15:02:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.556+0000 7f2b959ec740 1 -- 172.21.15.67:0/1201688121 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74010640 con 0x55fa503fe1b0 2024-07-31T15:02:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.557+0000 7f2b8cfb9640 1 -- 172.21.15.67:0/1201688121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503dd4d0 msgr2=0x55fa507a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.557+0000 7f2b8cfb9640 1 --2- 172.21.15.67:0/1201688121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503dd4d0 0x55fa507a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.557+0000 7f2b8cfb9640 1 -- 172.21.15.67:0/1201688121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503d52f0 msgr2=0x55fa507a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.557+0000 7f2b8cfb9640 1 --2- 172.21.15.67:0/1201688121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503d52f0 0x55fa507a3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.557+0000 7f2b8cfb9640 1 -- 172.21.15.67:0/1201688121 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa507cfb10 con 0x55fa503fe1b0 2024-07-31T15:02:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.557+0000 7f2b8cfb9640 1 --2- 172.21.15.67:0/1201688121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503fe1b0 0x55fa507c90d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2b800a8560 tx=0x7f2b8000b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3ced643b1712f3a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.558+0000 7f2b6dffb640 1 -- 172.21.15.67:0/1201688121 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80023070 con 0x55fa503fe1b0 2024-07-31T15:02:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.558+0000 7f2b6dffb640 1 -- 172.21.15.67:0/1201688121 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b80015350 con 0x55fa503fe1b0 2024-07-31T15:02:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.558+0000 7f2b6dffb640 1 -- 172.21.15.67:0/1201688121 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b800a6ec0 con 0x55fa503fe1b0 2024-07-31T15:02:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.559+0000 7f2b959ec740 1 -- 172.21.15.67:0/1201688121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503fe1b0 msgr2=0x55fa507c90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.559+0000 7f2b959ec740 1 --2- 172.21.15.67:0/1201688121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503fe1b0 0x55fa507c90d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2b800a8560 tx=0x7f2b8000b920 comp rx=0 tx=0).stop 2024-07-31T15:02:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.560+0000 7f2b959ec740 1 -- 172.21.15.67:0/1201688121 shutdown_connections 2024-07-31T15:02:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.560+0000 7f2b959ec740 1 --2- 172.21.15.67:0/1201688121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503fe1b0 0x55fa507c90d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.560+0000 7f2b959ec740 1 --2- 172.21.15.67:0/1201688121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503dd4d0 0x55fa507a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.560+0000 7f2b959ec740 1 --2- 172.21.15.67:0/1201688121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503d52f0 0x55fa507a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.560+0000 7f2b959ec740 1 -- 172.21.15.67:0/1201688121 >> 172.21.15.67:0/1201688121 conn(0x55fa503f2300 msgr2=0x55fa507ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.560+0000 7f2b959ec740 1 -- 172.21.15.67:0/1201688121 shutdown_connections 2024-07-31T15:02:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.560+0000 7f2b959ec740 1 -- 172.21.15.67:0/1201688121 wait complete. 2024-07-31T15:02:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.560+0000 7f2b959ec740 1 -- start start 2024-07-31T15:02:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.561+0000 7f2b959ec740 1 Processor -- start 2024-07-31T15:02:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.562+0000 7f2b959ec740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503d52f0 0x55fa507c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.562+0000 7f2b959ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503dd4d0 0x55fa507c2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.562+0000 7f2b87fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503d52f0 0x55fa507c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.562+0000 7f2b87fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503d52f0 0x55fa507c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40518/0 (socket says 172.21.15.67:40518) 2024-07-31T15:02:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.562+0000 7f2b87fff640 1 -- 172.21.15.67:0/3708317446 learned_addr learned my addr 172.21.15.67:0/3708317446 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.562+0000 7f2b877fe640 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503dd4d0 0x55fa507c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.562+0000 7f2b959ec740 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503fe1b0 0x55fa507c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.562+0000 7f2b959ec740 1 -- 172.21.15.67:0/3708317446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2b740114a0 con 0x55fa503fe1b0 2024-07-31T15:02:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.562+0000 7f2b959ec740 1 -- 172.21.15.67:0/3708317446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74010d10 con 0x55fa503d52f0 2024-07-31T15:02:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.562+0000 7f2b959ec740 1 -- 172.21.15.67:0/3708317446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7400df70 con 0x55fa503dd4d0 2024-07-31T15:02:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.562+0000 7f2b959ec740 10 client.0 _ll_register_callbacks cb 0x7f2b7c00ac10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.562+0000 7f2b8cfb9640 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503fe1b0 0x55fa507c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.563+0000 7f2b87fff640 1 -- 172.21.15.67:0/3708317446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503dd4d0 msgr2=0x55fa507c2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.563+0000 7f2b87fff640 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503dd4d0 0x55fa507c2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.563+0000 7f2b87fff640 1 -- 172.21.15.67:0/3708317446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503fe1b0 msgr2=0x55fa507c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.563+0000 7f2b87fff640 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503fe1b0 0x55fa507c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.563+0000 7f2b87fff640 1 -- 172.21.15.67:0/3708317446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa507c2c90 con 0x55fa503d52f0 2024-07-31T15:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.563+0000 7f2b877fe640 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503dd4d0 0x55fa507c2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.563+0000 7f2b87fff640 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503d52f0 0x55fa507c5690 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00c350 tx=0x7f2b7c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21d114ebc425348f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.563+0000 7f2b677fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.563+0000 7f2b677fe640 1 -- 172.21.15.67:0/3708317446 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c032060 con 0x55fa503d52f0 2024-07-31T15:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.563+0000 7f2b959ec740 1 -- 172.21.15.67:0/3708317446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b80015350 con 0x55fa503d52f0 2024-07-31T15:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.563+0000 7f2b677fe640 1 -- 172.21.15.67:0/3708317446 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9439+0+0 (secure 0 0 0) 0x7f2b7c00e680 con 0x55fa503d52f0 2024-07-31T15:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.563+0000 7f2b677fe640 1 -- 172.21.15.67:0/3708317446 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2b7c039050 con 0x55fa503d52f0 2024-07-31T15:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.563+0000 7f2b959ec740 10 client.27506 fetch_fsmap learned FSMap version 59 2024-07-31T15:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.563+0000 7f2b959ec740 1 -- 172.21.15.67:0/3708317446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55fa50836a30 con 0x55fa503d52f0 2024-07-31T15:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.564+0000 7f2b677fe640 1 -- 172.21.15.67:0/3708317446 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b80015350 con 0x55fa503d52f0 2024-07-31T15:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.564+0000 7f2b677fe640 1 -- 172.21.15.67:0/3708317446 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7f2b7c00f590 con 0x55fa503d52f0 2024-07-31T15:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.564+0000 7f2b959ec740 10 client.27506 fetch_fsmap finished waiting for FSMap version 59 2024-07-31T15:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.564+0000 7f2b959ec740 10 client.27506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:02:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.564+0000 7f2b959ec740 20 client.27506 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.564+0000 7f2b959ec740 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa503921c0 0x55fa508392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.564+0000 7f2b959ec740 4 client.27506 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:02:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.564+0000 7f2b959ec740 1 -- 172.21.15.67:0/3708317446 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa50718300 con 0x55fa503921c0 2024-07-31T15:02:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.564+0000 7f2b877fe640 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa503921c0 0x55fa508392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.565+0000 7f2b677fe640 10 client.27506 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.565+0000 7f2b877fe640 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa503921c0 0x55fa508392e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=576f958cdfc9c20b server_cookie=1aca845ce0463704 in_seq=0 out_seq=0 2024-07-31T15:02:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.825+0000 7f2b677fe640 1 -- 172.21.15.67:0/3708317446 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa50718300 con 0x55fa503921c0 2024-07-31T15:02:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.825+0000 7f2b677fe640 10 client.27506 handle_command_reply: tid=0 2024-07-31T15:02:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.825+0000 7f2b959ec740 1 client.27506 shutdown 2024-07-31T15:02:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.825+0000 7f2b959ec740 1 -- 172.21.15.67:0/3708317446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503d52f0 msgr2=0x55fa507c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.825+0000 7f2b959ec740 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503d52f0 0x55fa507c5690 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00c350 tx=0x7f2b7c0046c0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.826+0000 7f2b959ec740 1 -- 172.21.15.67:0/3708317446 shutdown_connections 2024-07-31T15:02:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.826+0000 7f2b959ec740 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa503921c0 0x55fa508392e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.826+0000 7f2b677fe640 0 client.27506 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.826+0000 7f2b959ec740 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa503fe1b0 0x55fa507c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.826+0000 7f2b959ec740 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa503dd4d0 0x55fa507c2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.826+0000 7f2b959ec740 1 --2- 172.21.15.67:0/3708317446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa503d52f0 0x55fa507c5690 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.826+0000 7f2b959ec740 1 -- 172.21.15.67:0/3708317446 >> 172.21.15.67:0/3708317446 conn(0x55fa503f2300 msgr2=0x55fa507a19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.826+0000 7f2b959ec740 1 -- 172.21.15.67:0/3708317446 shutdown_connections 2024-07-31T15:02:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.826+0000 7f2b959ec740 1 -- 172.21.15.67:0/3708317446 wait complete. 2024-07-31T15:02:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.826+0000 7f2b959ec740 20 client.27506 trim_cache size 0 max 16384 2024-07-31T15:02:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:02:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:02:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:02:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:02:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.827+0000 7f2b8ea1c640 1 -- 172.21.15.67:0/3510532579 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2b700b3f80 msgr2=0x7f2b700b6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.827+0000 7f2b8ea1c640 1 --2- 172.21.15.67:0/3510532579 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2b700b3f80 0x7f2b700b6490 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2b88101640 tx=0x7f2b78005fd0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.827+0000 7f2b8ea1c640 1 -- 172.21.15.67:0/3510532579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b88108670 msgr2=0x7f2b8806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.827+0000 7f2b8ea1c640 1 --2- 172.21.15.67:0/3510532579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b88108670 0x7f2b8806ff40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000bfa0 tx=0x7f2b80009f90 comp rx=0 tx=0).stop 2024-07-31T15:02:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.828+0000 7f2b8ea1c640 1 -- 172.21.15.67:0/3510532579 shutdown_connections 2024-07-31T15:02:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.828+0000 7f2b8ea1c640 1 --2- 172.21.15.67:0/3510532579 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2b700b3f80 0x7f2b700b6490 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.828+0000 7f2b8ea1c640 1 --2- 172.21.15.67:0/3510532579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b88108670 0x7f2b8806ff40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.828+0000 7f2b8ea1c640 1 --2- 172.21.15.67:0/3510532579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b88107d20 0x7f2b8806fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.828+0000 7f2b8ea1c640 1 --2- 172.21.15.67:0/3510532579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b881023f0 0x7f2b88074740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.828+0000 7f2b8ea1c640 1 -- 172.21.15.67:0/3510532579 >> 172.21.15.67:0/3510532579 conn(0x7f2b880fddf0 msgr2=0x7f2b881062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.828+0000 7f2b8ea1c640 1 -- 172.21.15.67:0/3510532579 shutdown_connections 2024-07-31T15:02:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:21.828+0000 7f2b8ea1c640 1 -- 172.21.15.67:0/3510532579 wait complete. 2024-07-31T15:02:22.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:22.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:22.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:22.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:22.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:22.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:22.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:22.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:22.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:22.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:22.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:22.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:23 smithi077 ceph-mon[37327]: pgmap v753: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 191 KiB/s wr, 363 op/s 2024-07-31T15:02:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:23 smithi067 ceph-mon[32750]: pgmap v753: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 191 KiB/s wr, 363 op/s 2024-07-31T15:02:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:23 smithi073 ceph-mon[36714]: pgmap v753: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 191 KiB/s wr, 363 op/s 2024-07-31T15:02:24.738 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:02:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 60, 2024-07-31T15:02:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:02:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 26, 2024-07-31T15:02:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:02:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "f864f25e": { 2024-07-31T15:02:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 26, 2024-07-31T15:02:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:02:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:02:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:02:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:02:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:02:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:02:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 130.0, 2024-07-31T15:02:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:02:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:02:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:02:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:02:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:02:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:02:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:02:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:02:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:02:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:02:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:02:24.740 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:02:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.734+0000 7fbbc0ff9640 1 -- 172.21.15.67:0/691836377 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fbbd40b2070 con 0x55e4b3e0cfa0 2024-07-31T15:02:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.734+0000 7fbbc0ff9640 10 client.27331 handle_command_reply: tid=0 2024-07-31T15:02:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.734+0000 7fbbe5dea740 1 client.27331 shutdown 2024-07-31T15:02:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.735+0000 7fbbe5dea740 1 -- 172.21.15.67:0/691836377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39a01c0 msgr2=0x55e4b3da7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.735+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/691836377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39a01c0 0x55e4b3da7790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4005840 tx=0x7fbbd40e5d20 comp rx=0 tx=0).stop 2024-07-31T15:02:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.735+0000 7fbbe5dea740 1 -- 172.21.15.67:0/691836377 shutdown_connections 2024-07-31T15:02:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.735+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/691836377 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e4b3e0cfa0 0x55e4b3e0f3c0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.735+0000 7fbbc0ff9640 0 client.27331 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.735+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/691836377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4b39e1840 0x55e4b3e053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.735+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/691836377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4b39c0b60 0x55e4b3da7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.735+0000 7fbbe5dea740 1 --2- 172.21.15.67:0/691836377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e4b39a01c0 0x55e4b3da7790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.735+0000 7fbbe5dea740 1 -- 172.21.15.67:0/691836377 >> 172.21.15.67:0/691836377 conn(0x55e4b39d5990 msgr2=0x55e4b3d94480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.735+0000 7fbbe5dea740 1 -- 172.21.15.67:0/691836377 shutdown_connections 2024-07-31T15:02:24.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.735+0000 7fbbe5dea740 1 -- 172.21.15.67:0/691836377 wait complete. 2024-07-31T15:02:24.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.736+0000 7fbbe5dea740 20 client.27331 trim_cache size 0 max 16384 2024-07-31T15:02:24.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.739+0000 7fbbdf899640 1 -- 172.21.15.67:0/421732493 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbbc40b3f50 msgr2=0x7fbbc40b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:24.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.739+0000 7fbbdf899640 1 --2- 172.21.15.67:0/421732493 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbbc40b3f50 0x7fbbc40b6460 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55e4b3d95bc0 tx=0x7fbbc8008040 comp rx=0 tx=0).stop 2024-07-31T15:02:24.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.739+0000 7fbbdf899640 1 -- 172.21.15.67:0/421732493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd81024b0 msgr2=0x7fbbd8074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:24.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.739+0000 7fbbdf899640 1 --2- 172.21.15.67:0/421732493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd81024b0 0x7fbbd8074440 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400c970 tx=0x7fbbd400ce30 comp rx=0 tx=0).stop 2024-07-31T15:02:24.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.742+0000 7fbbdf899640 1 -- 172.21.15.67:0/421732493 shutdown_connections 2024-07-31T15:02:24.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.742+0000 7fbbdf899640 1 --2- 172.21.15.67:0/421732493 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbbc40b3f50 0x7fbbc40b6460 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:24.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.742+0000 7fbbdf899640 1 --2- 172.21.15.67:0/421732493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbd8108740 0x7fbbd806e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:24.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.742+0000 7fbbdf899640 1 --2- 172.21.15.67:0/421732493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbd8107df0 0x7fbbd80749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:24.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.742+0000 7fbbdf899640 1 --2- 172.21.15.67:0/421732493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbd81024b0 0x7fbbd8074440 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:24.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.742+0000 7fbbdf899640 1 -- 172.21.15.67:0/421732493 >> 172.21.15.67:0/421732493 conn(0x7fbbd8079190 msgr2=0x7fbbd8103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:24.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.743+0000 7fbbdf899640 1 -- 172.21.15.67:0/421732493 shutdown_connections 2024-07-31T15:02:24.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:24.743+0000 7fbbdf899640 1 -- 172.21.15.67:0/421732493 wait complete. 2024-07-31T15:02:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:25 smithi077 ceph-mon[37327]: pgmap v754: 129 pgs: 53 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 58 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 78 KiB/s wr, 381 op/s 2024-07-31T15:02:25.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:25.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:25.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:25 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:02:25.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:25.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:25.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:25.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:25.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:25.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:25.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:25.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:25.525 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:25 smithi067 ceph-mon[32750]: pgmap v754: 129 pgs: 53 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 58 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 78 KiB/s wr, 381 op/s 2024-07-31T15:02:25.525 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:25.525 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:25.525 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:25 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:02:25.525 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:25.525 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:25.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:25.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:25.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:25.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:25.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:25.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:25.578 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f864f25e', quiesce took 56.6 seconds. Will release after: -46.6 2024-07-31T15:02:25.579 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:02:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:25 smithi073 ceph-mon[36714]: pgmap v754: 129 pgs: 53 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 58 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 78 KiB/s wr, 381 op/s 2024-07-31T15:02:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:25 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:02:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.097 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:02:26.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:26.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:26.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:26.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:26.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:26.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:26.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:27 smithi077 ceph-mon[37327]: pgmap v755: 129 pgs: 46 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 66 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 141 KiB/s wr, 361 op/s 2024-07-31T15:02:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:27.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:27.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:27 smithi067 ceph-mon[32750]: pgmap v755: 129 pgs: 46 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 66 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 141 KiB/s wr, 361 op/s 2024-07-31T15:02:27.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:27.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:27.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:27.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:27.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:27.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:27.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:27 smithi073 ceph-mon[36714]: pgmap v755: 129 pgs: 46 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 66 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 141 KiB/s wr, 361 op/s 2024-07-31T15:02:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.319+0000 7f354d06c640 1 -- 172.21.15.67:0/2554608604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f354810f330 msgr2=0x7f354810f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.319+0000 7f354d06c640 1 --2- 172.21.15.67:0/2554608604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f354810f330 0x7f354810f740 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3540009ca0 tx=0x7f354002a8f0 comp rx=0 tx=0).stop 2024-07-31T15:02:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.319+0000 7f354d06c640 1 -- 172.21.15.67:0/2554608604 shutdown_connections 2024-07-31T15:02:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.319+0000 7f354d06c640 1 --2- 172.21.15.67:0/2554608604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354810fc80 0x7f35481164b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.319+0000 7f354d06c640 1 --2- 172.21.15.67:0/2554608604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f354810f330 0x7f354810f740 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.319+0000 7f354d06c640 1 --2- 172.21.15.67:0/2554608604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35481099f0 0x7f3548109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.319+0000 7f354d06c640 1 -- 172.21.15.67:0/2554608604 >> 172.21.15.67:0/2554608604 conn(0x7f3548078ff0 msgr2=0x7f3548107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:28.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.327+0000 7f354d06c640 1 -- 172.21.15.67:0/2554608604 shutdown_connections 2024-07-31T15:02:28.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.327+0000 7f354d06c640 1 -- 172.21.15.67:0/2554608604 wait complete. 2024-07-31T15:02:28.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.328+0000 7f354d06c640 1 Processor -- start 2024-07-31T15:02:28.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.328+0000 7f354d06c640 1 -- start start 2024-07-31T15:02:28.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.329+0000 7f354d06c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35481099f0 0x7f3548074920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:28.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.329+0000 7f354d06c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f354810f330 0x7f354806dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:28.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.329+0000 7f354d06c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354810fc80 0x7f354806e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:28.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.329+0000 7f354d06c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f35481199b0 con 0x7f35481099f0 2024-07-31T15:02:28.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.329+0000 7f354d06c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3548119830 con 0x7f354810f330 2024-07-31T15:02:28.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.329+0000 7f354d06c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3548119b30 con 0x7f354810fc80 2024-07-31T15:02:28.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.330+0000 7f35477fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f354810f330 0x7f354806dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:28.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.330+0000 7f35477fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f354810f330 0x7f354806dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40528/0 (socket says 172.21.15.67:40528) 2024-07-31T15:02:28.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.330+0000 7f35477fe640 1 -- 172.21.15.67:0/2138819643 learned_addr learned my addr 172.21.15.67:0/2138819643 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.332+0000 7f354c86b640 1 --2- 172.21.15.67:0/2138819643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354810fc80 0x7f354806e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.332+0000 7f35477fe640 1 -- 172.21.15.67:0/2138819643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354810fc80 msgr2=0x7f354806e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.332+0000 7f35477fe640 1 --2- 172.21.15.67:0/2138819643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354810fc80 0x7f354806e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.332+0000 7f35477fe640 1 -- 172.21.15.67:0/2138819643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35481099f0 msgr2=0x7f3548074920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.332+0000 7f35477fe640 1 --2- 172.21.15.67:0/2138819643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35481099f0 0x7f3548074920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.332+0000 7f35477fe640 1 -- 172.21.15.67:0/2138819643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354806ea20 con 0x7f354810f330 2024-07-31T15:02:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.333+0000 7f35477fe640 1 --2- 172.21.15.67:0/2138819643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f354810f330 0x7f354806dbf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3540006010 tx=0x7f354002f3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.334+0000 7f35457fa640 1 -- 172.21.15.67:0/2138819643 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540003b70 con 0x7f354810f330 2024-07-31T15:02:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.335+0000 7f35457fa640 1 -- 172.21.15.67:0/2138819643 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f354002d070 con 0x7f354810f330 2024-07-31T15:02:28.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.336+0000 7f35457fa640 1 -- 172.21.15.67:0/2138819643 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354002f930 con 0x7f354810f330 2024-07-31T15:02:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.336+0000 7f354d06c640 1 -- 172.21.15.67:0/2138819643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3548073650 con 0x7f354810f330 2024-07-31T15:02:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.336+0000 7f354d06c640 1 -- 172.21.15.67:0/2138819643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3548073b60 con 0x7f354810f330 2024-07-31T15:02:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.340+0000 7f352effd640 1 -- 172.21.15.67:0/2138819643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3548109e00 con 0x7f354810f330 2024-07-31T15:02:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.343+0000 7f35457fa640 1 -- 172.21.15.67:0/2138819643 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3540003d10 con 0x7f354810f330 2024-07-31T15:02:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.344+0000 7f35457fa640 1 --2- 172.21.15.67:0/2138819643 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f35300b3d20 0x7f35300b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.345+0000 7f35457fa640 1 -- 172.21.15.67:0/2138819643 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9439+0+0 (secure 0 0 0) 0x7f3540105a10 con 0x7f354810f330 2024-07-31T15:02:28.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.348+0000 7f3547fff640 1 --2- 172.21.15.67:0/2138819643 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f35300b3d20 0x7f35300b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:28.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.349+0000 7f3547fff640 1 --2- 172.21.15.67:0/2138819643 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f35300b3d20 0x7f35300b6230 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3538009710 tx=0x7f3538006c90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:28.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.351+0000 7f35457fa640 1 -- 172.21.15.67:0/2138819643 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f354806ea20 con 0x7f354810f330 2024-07-31T15:02:28.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:28.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:28.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:28.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:28.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:28.547 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:28.547 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:28.547 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:28.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:28.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.832+0000 7f352effd640 1 -- 172.21.15.67:0/2138819643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f354806eef0 con 0x7f354810f330 2024-07-31T15:02:28.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.834+0000 7f35457fa640 1 -- 172.21.15.67:0/2138819643 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+11966 (secure 0 0 0) 0x7f35400ccac0 con 0x7f354810f330 2024-07-31T15:02:28.836 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:02:28.836 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":60,"btime":"2024-07-31T15:02:24:150368+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":60,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:02:23.699045+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:02:28.837 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 60 2024-07-31T15:02:28.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.838+0000 7f352effd640 1 -- 172.21.15.67:0/2138819643 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f35300b3d20 msgr2=0x7f35300b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:28.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.838+0000 7f352effd640 1 --2- 172.21.15.67:0/2138819643 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f35300b3d20 0x7f35300b6230 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3538009710 tx=0x7f3538006c90 comp rx=0 tx=0).stop 2024-07-31T15:02:28.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.838+0000 7f352effd640 1 -- 172.21.15.67:0/2138819643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f354810f330 msgr2=0x7f354806dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:28.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.838+0000 7f352effd640 1 --2- 172.21.15.67:0/2138819643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f354810f330 0x7f354806dbf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3540006010 tx=0x7f354002f3b0 comp rx=0 tx=0).stop 2024-07-31T15:02:28.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.838+0000 7f352effd640 1 -- 172.21.15.67:0/2138819643 shutdown_connections 2024-07-31T15:02:28.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.838+0000 7f352effd640 1 --2- 172.21.15.67:0/2138819643 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f35300b3d20 0x7f35300b6230 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:28.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.838+0000 7f352effd640 1 --2- 172.21.15.67:0/2138819643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354810fc80 0x7f354806e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:28.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.839+0000 7f352effd640 1 --2- 172.21.15.67:0/2138819643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f354810f330 0x7f354806dbf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:28.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.839+0000 7f352effd640 1 --2- 172.21.15.67:0/2138819643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35481099f0 0x7f3548074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:28.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.839+0000 7f352effd640 1 -- 172.21.15.67:0/2138819643 >> 172.21.15.67:0/2138819643 conn(0x7f3548078ff0 msgr2=0x7f35480797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:28.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.839+0000 7f352effd640 1 -- 172.21.15.67:0/2138819643 shutdown_connections 2024-07-31T15:02:28.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:28.844+0000 7f352effd640 1 -- 172.21.15.67:0/2138819643 wait complete. 2024-07-31T15:02:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:29 smithi077 ceph-mon[37327]: pgmap v756: 129 pgs: 40 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 73 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 119 KiB/s wr, 402 op/s 2024-07-31T15:02:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:29.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:29 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2138819643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:02:29.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:29.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:29.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:29 smithi067 ceph-mon[32750]: pgmap v756: 129 pgs: 40 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 73 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 119 KiB/s wr, 402 op/s 2024-07-31T15:02:29.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:29.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:29.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:29.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:29 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2138819643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:02:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:29 smithi073 ceph-mon[36714]: pgmap v756: 129 pgs: 40 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 73 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 119 KiB/s wr, 402 op/s 2024-07-31T15:02:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:29 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2138819643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:02:29.721 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id f864f25e --release --await' 2024-07-31T15:02:29.721 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id f864f25e --release --await 2024-07-31T15:02:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:02:30.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:30.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:30.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:30.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:30.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:30.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:30.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:30.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:30.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:30.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:30.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:30.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:30.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:30.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:30.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:30.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:30.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:31 smithi077 ceph-mon[37327]: pgmap v757: 129 pgs: 31 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 84 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 144 KiB/s wr, 507 op/s 2024-07-31T15:02:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.493 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:31 smithi067 ceph-mon[32750]: pgmap v757: 129 pgs: 31 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 84 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 144 KiB/s wr, 507 op/s 2024-07-31T15:02:31.493 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:31.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:31.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:31.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:31.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:31.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:31 smithi073 ceph-mon[36714]: pgmap v757: 129 pgs: 31 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 84 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 144 KiB/s wr, 507 op/s 2024-07-31T15:02:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.399+0000 7f135f235640 1 -- 172.21.15.67:0/1819647065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f135810a410 msgr2=0x7f135810a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:32.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.399+0000 7f135f235640 1 --2- 172.21.15.67:0/1819647065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f135810a410 0x7f135810a820 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1348009860 tx=0x7f134802a340 comp rx=0 tx=0).stop 2024-07-31T15:02:32.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.400+0000 7f135f235640 1 -- 172.21.15.67:0/1819647065 shutdown_connections 2024-07-31T15:02:32.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.400+0000 7f135f235640 1 --2- 172.21.15.67:0/1819647065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f135811c660 0x7f135811eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.400+0000 7f135f235640 1 --2- 172.21.15.67:0/1819647065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f135810a410 0x7f135810a820 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.400+0000 7f135f235640 1 --2- 172.21.15.67:0/1819647065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1358111970 0x7f1358109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.400+0000 7f135f235640 1 -- 172.21.15.67:0/1819647065 >> 172.21.15.67:0/1819647065 conn(0x7f13580708f0 msgr2=0x7f1358070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:32.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.405+0000 7f135f235640 1 -- 172.21.15.67:0/1819647065 shutdown_connections 2024-07-31T15:02:32.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.408+0000 7f135f235640 1 -- 172.21.15.67:0/1819647065 wait complete. 2024-07-31T15:02:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.409+0000 7f135f235640 1 Processor -- start 2024-07-31T15:02:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.409+0000 7f135f235640 1 -- start start 2024-07-31T15:02:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.410+0000 7f135f235640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f135810a410 0x7f13581a8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.410+0000 7f135f235640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1358111970 0x7f13581a9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.410+0000 7f135f235640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f135811c660 0x7f13581ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.410+0000 7f135f235640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f13581222f0 con 0x7f1358111970 2024-07-31T15:02:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.410+0000 7f135f235640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1358122170 con 0x7f135811c660 2024-07-31T15:02:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.410+0000 7f135f235640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1358122470 con 0x7f135810a410 2024-07-31T15:02:32.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.411+0000 7f135cfd1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f135810a410 0x7f13581a8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.411+0000 7f135cfd1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f135810a410 0x7f13581a8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35442/0 (socket says 172.21.15.67:35442) 2024-07-31T15:02:32.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.411+0000 7f135cfd1640 1 -- 172.21.15.67:0/1114357241 learned_addr learned my addr 172.21.15.67:0/1114357241 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:32.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.411+0000 7f134ffff640 1 --2- 172.21.15.67:0/1114357241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1358111970 0x7f13581a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.412+0000 7f134ffff640 1 -- 172.21.15.67:0/1114357241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f135810a410 msgr2=0x7f13581a8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:32.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.412+0000 7f134ffff640 1 --2- 172.21.15.67:0/1114357241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f135810a410 0x7f13581a8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.412+0000 7f134ffff640 1 -- 172.21.15.67:0/1114357241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f135811c660 msgr2=0x7f13581ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:32.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.412+0000 7f134ffff640 1 --2- 172.21.15.67:0/1114357241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f135811c660 0x7f13581ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.412+0000 7f134ffff640 1 -- 172.21.15.67:0/1114357241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13581aed30 con 0x7f1358111970 2024-07-31T15:02:32.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.413+0000 7f134ffff640 1 --2- 172.21.15.67:0/1114357241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1358111970 0x7f13581a9360 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f1348009780 tx=0x7f1348030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:32.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.413+0000 7f134dffb640 1 -- 172.21.15.67:0/1114357241 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348042070 con 0x7f1358111970 2024-07-31T15:02:32.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.413+0000 7f135f235640 1 -- 172.21.15.67:0/1114357241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13581aefc0 con 0x7f1358111970 2024-07-31T15:02:32.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.414+0000 7f135f235640 1 -- 172.21.15.67:0/1114357241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1358111db0 con 0x7f1358111970 2024-07-31T15:02:32.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.414+0000 7f134dffb640 1 -- 172.21.15.67:0/1114357241 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1348030900 con 0x7f1358111970 2024-07-31T15:02:32.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.414+0000 7f134dffb640 1 -- 172.21.15.67:0/1114357241 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348037040 con 0x7f1358111970 2024-07-31T15:02:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.423+0000 7f1365786740 1 Processor -- start 2024-07-31T15:02:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.424+0000 7f134dffb640 1 -- 172.21.15.67:0/1114357241 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f13480054f0 con 0x7f1358111970 2024-07-31T15:02:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.425+0000 7f1365786740 1 -- start start 2024-07-31T15:02:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.425+0000 7f134dffb640 1 --2- 172.21.15.67:0/1114357241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f13400c4f60 0x7f13400c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.425+0000 7f134dffb640 1 -- 172.21.15.67:0/1114357241 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9439+0+0 (secure 0 0 0) 0x7f13481038d0 con 0x7f1358111970 2024-07-31T15:02:32.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.425+0000 7f1365786740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446fc0520 0x55b447366070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.425+0000 7f1365786740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f97660 0x55b4473665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.425+0000 7f1365786740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f9f840 0x55b447366af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.425+0000 7f1365786740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b4472dbe70 con 0x55b446f97660 2024-07-31T15:02:32.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.425+0000 7f1365786740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b4472cb370 con 0x55b446fc0520 2024-07-31T15:02:32.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.425+0000 7f1365786740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b4472cf3b0 con 0x55b446f9f840 2024-07-31T15:02:32.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.427+0000 7f135cfd1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446fc0520 0x55b447366070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.427+0000 7f135cfd1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446fc0520 0x55b447366070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:50640/0 (socket says 172.21.15.67:50640) 2024-07-31T15:02:32.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.427+0000 7f135cfd1640 1 -- 172.21.15.67:0/1898595315 learned_addr learned my addr 172.21.15.67:0/1898595315 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:32.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.427+0000 7f134ffff640 1 --2- 172.21.15.67:0/1898595315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f97660 0x55b4473665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.427+0000 7f135cfd1640 1 -- 172.21.15.67:0/1898595315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f9f840 msgr2=0x55b447366af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:02:32.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.427+0000 7f135cfd1640 1 --2- 172.21.15.67:0/1898595315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f9f840 0x55b447366af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.427+0000 7f135cfd1640 1 -- 172.21.15.67:0/1898595315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f97660 msgr2=0x55b4473665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:32.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.427+0000 7f135cfd1640 1 --2- 172.21.15.67:0/1898595315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f97660 0x55b4473665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.427+0000 7f135cfd1640 1 -- 172.21.15.67:0/1898595315 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b447298040 con 0x55b446fc0520 2024-07-31T15:02:32.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.428+0000 7f135cfd1640 1 --2- 172.21.15.67:0/1898595315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446fc0520 0x55b447366070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1354009d80 tx=0x7f1354011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=443c633e203eb683 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.429+0000 7f133effd640 1 -- 172.21.15.67:0/1898595315 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135401e070 con 0x55b446fc0520 2024-07-31T15:02:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.429+0000 7f133effd640 1 -- 172.21.15.67:0/1898595315 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1354019ae0 con 0x55b446fc0520 2024-07-31T15:02:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.430+0000 7f135d7d2640 1 --2- 172.21.15.67:0/1114357241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f13400c4f60 0x7f13400c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.431+0000 7f1365786740 1 -- 172.21.15.67:0/1898595315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446fc0520 msgr2=0x55b447366070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.431+0000 7f1365786740 1 --2- 172.21.15.67:0/1898595315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446fc0520 0x55b447366070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1354009d80 tx=0x7f1354011810 comp rx=0 tx=0).stop 2024-07-31T15:02:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.432+0000 7f1365786740 1 -- 172.21.15.67:0/1898595315 shutdown_connections 2024-07-31T15:02:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.432+0000 7f1365786740 1 --2- 172.21.15.67:0/1898595315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f9f840 0x55b447366af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.432+0000 7f1365786740 1 --2- 172.21.15.67:0/1898595315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f97660 0x55b4473665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.432+0000 7f1365786740 1 --2- 172.21.15.67:0/1898595315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446fc0520 0x55b447366070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.432+0000 7f1365786740 1 -- 172.21.15.67:0/1898595315 >> 172.21.15.67:0/1898595315 conn(0x55b446fb4910 msgr2=0x55b447363f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:32.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.434+0000 7f1365786740 1 -- 172.21.15.67:0/1898595315 shutdown_connections 2024-07-31T15:02:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.434+0000 7f1365786740 1 -- 172.21.15.67:0/1898595315 wait complete. 2024-07-31T15:02:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.435+0000 7f1365786740 1 -- start start 2024-07-31T15:02:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.435+0000 7f1365786740 1 Processor -- start 2024-07-31T15:02:32.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.436+0000 7f1365786740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f97660 0x55b447371c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.436+0000 7f1365786740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f9f840 0x55b447372150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.436+0000 7f1365786740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446fc0520 0x55b44737dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.436+0000 7f1365786740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b44712ebb0 con 0x55b446f9f840 2024-07-31T15:02:32.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.436+0000 7f1365786740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b44726dac0 con 0x55b446fc0520 2024-07-31T15:02:32.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.436+0000 7f1365786740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b4473748a0 con 0x55b446f97660 2024-07-31T15:02:32.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.436+0000 7f1365786740 10 client.0 _ll_register_callbacks cb 0x55b4472763b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:32.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.437+0000 7f134ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f9f840 0x55b447372150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.437+0000 7f134ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f9f840 0x55b447372150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45240/0 (socket says 172.21.15.67:45240) 2024-07-31T15:02:32.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.437+0000 7f134ffff640 1 -- 172.21.15.67:0/2951664244 learned_addr learned my addr 172.21.15.67:0/2951664244 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:32.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.438+0000 7f135d7d2640 1 --2- 172.21.15.67:0/2951664244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446fc0520 0x55b44737dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.438+0000 7f135d7d2640 1 --2- 172.21.15.67:0/1114357241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f13400c4f60 0x7f13400c7470 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1350005fd0 tx=0x7f1350005f60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:32.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.438+0000 7f135cfd1640 1 --2- 172.21.15.67:0/2951664244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f97660 0x55b447371c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.442+0000 7f134ffff640 1 -- 172.21.15.67:0/2951664244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f97660 msgr2=0x55b447371c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:32.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.442+0000 7f134ffff640 1 --2- 172.21.15.67:0/2951664244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f97660 0x55b447371c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.442+0000 7f134ffff640 1 -- 172.21.15.67:0/2951664244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446fc0520 msgr2=0x55b44737dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:32.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.442+0000 7f134ffff640 1 --2- 172.21.15.67:0/2951664244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446fc0520 0x55b44737dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.442+0000 7f134ffff640 1 -- 172.21.15.67:0/2951664244 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b44702bb00 con 0x55b446f9f840 2024-07-31T15:02:32.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.442+0000 7f134ffff640 1 --2- 172.21.15.67:0/2951664244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f9f840 0x55b447372150 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f13480c7930 tx=0x7f1348004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd245d4f886fe78 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:32.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.443+0000 7f1337fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:02:32.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.443+0000 7f1337fff640 1 -- 172.21.15.67:0/2951664244 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480e4060 con 0x55b446f9f840 2024-07-31T15:02:32.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.443+0000 7f1365786740 1 -- 172.21.15.67:0/2951664244 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1354019ae0 con 0x55b446f9f840 2024-07-31T15:02:32.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.443+0000 7f1337fff640 1 -- 172.21.15.67:0/2951664244 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9439+0+0 (secure 0 0 0) 0x7f13480328a0 con 0x55b446f9f840 2024-07-31T15:02:32.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.444+0000 7f1337fff640 1 -- 172.21.15.67:0/2951664244 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f13480cadc0 con 0x55b446f9f840 2024-07-31T15:02:32.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.445+0000 7f1365786740 10 client.17964 fetch_fsmap learned FSMap version 60 2024-07-31T15:02:32.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.445+0000 7f1365786740 1 -- 172.21.15.67:0/2951664244 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55b4470a2960 con 0x55b446f9f840 2024-07-31T15:02:32.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.449+0000 7f1337fff640 1 -- 172.21.15.67:0/2951664244 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1354019ae0 con 0x55b446f9f840 2024-07-31T15:02:32.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.450+0000 7f1337fff640 1 -- 172.21.15.67:0/2951664244 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 60) ==== 8386+0+0 (secure 0 0 0) 0x7f13480c9920 con 0x55b446f9f840 2024-07-31T15:02:32.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.450+0000 7f1365786740 10 client.17964 fetch_fsmap finished waiting for FSMap version 60 2024-07-31T15:02:32.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.450+0000 7f1365786740 10 client.17964 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:02:32.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.450+0000 7f1365786740 20 client.17964 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:32.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.450+0000 7f1365786740 1 --2- 172.21.15.67:0/2951664244 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b446f7eea0 0x55b447383ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.451+0000 7f1365786740 4 client.17964 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:02:32.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.451+0000 7f1365786740 1 -- 172.21.15.67:0/2951664244 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4472dddc0 con 0x55b446f7eea0 2024-07-31T15:02:32.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.451+0000 7f135cfd1640 1 --2- 172.21.15.67:0/2951664244 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b446f7eea0 0x55b447383ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.452+0000 7f135cfd1640 1 --2- 172.21.15.67:0/2951664244 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b446f7eea0 0x55b447383ea0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d02e5624637cfa15 server_cookie=b0c8f78df0c7fb93 in_seq=0 out_seq=0 2024-07-31T15:02:32.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.452+0000 7f1337fff640 10 client.17964 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:32.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.454+0000 7f1337fff640 1 -- 172.21.15.67:0/2951664244 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4472dddc0 con 0x55b446f7eea0 2024-07-31T15:02:32.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.454+0000 7f1337fff640 10 client.17964 handle_command_reply: tid=0 2024-07-31T15:02:32.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.454+0000 7f1365786740 1 client.17964 shutdown 2024-07-31T15:02:32.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:32.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:32.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:32.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:32.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:32.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.456+0000 7f1365786740 1 -- 172.21.15.67:0/2951664244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f9f840 msgr2=0x55b447372150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:32.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.456+0000 7f1365786740 1 --2- 172.21.15.67:0/2951664244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f9f840 0x55b447372150 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f13480c7930 tx=0x7f1348004610 comp rx=0 tx=0).stop 2024-07-31T15:02:32.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.456+0000 7f1365786740 1 -- 172.21.15.67:0/2951664244 shutdown_connections 2024-07-31T15:02:32.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.456+0000 7f1365786740 1 --2- 172.21.15.67:0/2951664244 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b446f7eea0 0x55b447383ea0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.456+0000 7f1365786740 1 --2- 172.21.15.67:0/2951664244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446fc0520 0x55b44737dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.456+0000 7f1365786740 1 --2- 172.21.15.67:0/2951664244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f9f840 0x55b447372150 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.456+0000 7f1365786740 1 --2- 172.21.15.67:0/2951664244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f97660 0x55b447371c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.456+0000 7f1365786740 1 -- 172.21.15.67:0/2951664244 >> 172.21.15.67:0/2951664244 conn(0x55b446fb4910 msgr2=0x55b447369200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:32.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.456+0000 7f1337fff640 0 client.17964 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:32.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.457+0000 7f1365786740 1 -- 172.21.15.67:0/2951664244 shutdown_connections 2024-07-31T15:02:32.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.457+0000 7f1365786740 1 -- 172.21.15.67:0/2951664244 wait complete. 2024-07-31T15:02:32.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.457+0000 7f1365786740 20 client.17964 trim_cache size 0 max 16384 2024-07-31T15:02:32.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:32.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.497+0000 7f1365786740 1 Processor -- start 2024-07-31T15:02:32.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.497+0000 7f1365786740 1 -- start start 2024-07-31T15:02:32.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.497+0000 7f1365786740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446f97660 0x55b4473677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.498+0000 7f1365786740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446fc0520 0x55b44737da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.498+0000 7f1365786740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f54250 0x55b44737df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.498+0000 7f135cfd1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446f97660 0x55b4473677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.498+0000 7f135cfd1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446f97660 0x55b4473677b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:50656/0 (socket says 172.21.15.67:50656) 2024-07-31T15:02:32.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.498+0000 7f135cfd1640 1 -- 172.21.15.67:0/2264170604 learned_addr learned my addr 172.21.15.67:0/2264170604 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:32.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.498+0000 7f135d7d2640 1 --2- 172.21.15.67:0/2264170604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f54250 0x55b44737df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.498+0000 7f1365786740 1 -- 172.21.15.67:0/2264170604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f134400fe50 con 0x55b446f54250 2024-07-31T15:02:32.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.498+0000 7f1365786740 1 -- 172.21.15.67:0/2264170604 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1344011a50 con 0x55b446f97660 2024-07-31T15:02:32.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.498+0000 7f1365786740 1 -- 172.21.15.67:0/2264170604 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1344010520 con 0x55b446fc0520 2024-07-31T15:02:32.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.499+0000 7f135cfd1640 1 -- 172.21.15.67:0/2264170604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446fc0520 msgr2=0x55b44737da30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:02:32.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.499+0000 7f135cfd1640 1 --2- 172.21.15.67:0/2264170604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446fc0520 0x55b44737da30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.499+0000 7f135cfd1640 1 -- 172.21.15.67:0/2264170604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f54250 msgr2=0x55b44737df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:32.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.499+0000 7f135cfd1640 1 --2- 172.21.15.67:0/2264170604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f54250 0x55b44737df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.499+0000 7f135cfd1640 1 -- 172.21.15.67:0/2264170604 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b447371250 con 0x55b446f97660 2024-07-31T15:02:32.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.501+0000 7f135cfd1640 1 --2- 172.21.15.67:0/2264170604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446f97660 0x55b4473677b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1354000c00 tx=0x7f135400e810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7417b809b2454e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:32.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.502+0000 7f133e7fc640 1 -- 172.21.15.67:0/2264170604 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135401e040 con 0x55b446f97660 2024-07-31T15:02:32.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.502+0000 7f133e7fc640 1 -- 172.21.15.67:0/2264170604 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13540099b0 con 0x55b446f97660 2024-07-31T15:02:32.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.502+0000 7f133e7fc640 1 -- 172.21.15.67:0/2264170604 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354019a30 con 0x55b446f97660 2024-07-31T15:02:32.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.503+0000 7f1365786740 1 -- 172.21.15.67:0/2264170604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446f97660 msgr2=0x55b4473677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:32.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.504+0000 7f1365786740 1 --2- 172.21.15.67:0/2264170604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446f97660 0x55b4473677b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1354000c00 tx=0x7f135400e810 comp rx=0 tx=0).stop 2024-07-31T15:02:32.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.511+0000 7f1365786740 1 -- 172.21.15.67:0/2264170604 shutdown_connections 2024-07-31T15:02:32.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.511+0000 7f1365786740 1 --2- 172.21.15.67:0/2264170604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446f54250 0x55b44737df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.511+0000 7f1365786740 1 --2- 172.21.15.67:0/2264170604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446fc0520 0x55b44737da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.511+0000 7f1365786740 1 --2- 172.21.15.67:0/2264170604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446f97660 0x55b4473677b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.511+0000 7f1365786740 1 -- 172.21.15.67:0/2264170604 >> 172.21.15.67:0/2264170604 conn(0x55b446fb4910 msgr2=0x55b4473670a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:32.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.512+0000 7f1365786740 1 -- 172.21.15.67:0/2264170604 shutdown_connections 2024-07-31T15:02:32.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.512+0000 7f1365786740 1 -- 172.21.15.67:0/2264170604 wait complete. 2024-07-31T15:02:32.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.513+0000 7f1365786740 1 -- start start 2024-07-31T15:02:32.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.513+0000 7f1365786740 1 Processor -- start 2024-07-31T15:02:32.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.513+0000 7f1365786740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f54250 0x55b447375be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.513+0000 7f1365786740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446f97660 0x55b447376120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.513+0000 7f1365786740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446fc0520 0x55b447376660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.514+0000 7f1365786740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1344011380 con 0x55b446fc0520 2024-07-31T15:02:32.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.514+0000 7f1365786740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1344010bf0 con 0x55b446f97660 2024-07-31T15:02:32.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.514+0000 7f1365786740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f134400dee0 con 0x55b446f54250 2024-07-31T15:02:32.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.514+0000 7f1365786740 10 client.0 _ll_register_callbacks cb 0x7f13481010e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:32.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.515+0000 7f135cfd1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f54250 0x55b447375be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.515+0000 7f135cfd1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f54250 0x55b447375be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35472/0 (socket says 172.21.15.67:35472) 2024-07-31T15:02:32.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.515+0000 7f135cfd1640 1 -- 172.21.15.67:0/3067217500 learned_addr learned my addr 172.21.15.67:0/3067217500 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:32.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.515+0000 7f135d7d2640 1 --2- 172.21.15.67:0/3067217500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446fc0520 0x55b447376660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.515+0000 7f134ffff640 1 --2- 172.21.15.67:0/3067217500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446f97660 0x55b447376120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.521+0000 7f134ffff640 1 -- 172.21.15.67:0/3067217500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f54250 msgr2=0x55b447375be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:32.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.521+0000 7f134ffff640 1 --2- 172.21.15.67:0/3067217500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f54250 0x55b447375be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.521+0000 7f134ffff640 1 -- 172.21.15.67:0/3067217500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446fc0520 msgr2=0x55b447376660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:32.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.521+0000 7f134ffff640 1 --2- 172.21.15.67:0/3067217500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446fc0520 0x55b447376660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:32.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.521+0000 7f134ffff640 1 -- 172.21.15.67:0/3067217500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4473eb7b0 con 0x55b446f97660 2024-07-31T15:02:32.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.521+0000 7f134ffff640 1 --2- 172.21.15.67:0/3067217500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446f97660 0x55b447376120 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55b44726b1b0 tx=0x7f13480d3130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a46785187f0ab869 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:32.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.527+0000 7f1337fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:02:32.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.527+0000 7f1337fff640 1 -- 172.21.15.67:0/3067217500 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480c6580 con 0x55b446f97660 2024-07-31T15:02:32.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.527+0000 7f1365786740 1 -- 172.21.15.67:0/3067217500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b4472d9860 con 0x55b446f97660 2024-07-31T15:02:32.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.527+0000 7f1337fff640 1 -- 172.21.15.67:0/3067217500 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9439+0+0 (secure 0 0 0) 0x7f13480c6cb0 con 0x55b446f97660 2024-07-31T15:02:32.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.530+0000 7f1337fff640 1 -- 172.21.15.67:0/3067217500 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f13480cecc0 con 0x55b446f97660 2024-07-31T15:02:32.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.530+0000 7f1365786740 10 client.27527 fetch_fsmap learned FSMap version 60 2024-07-31T15:02:32.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.530+0000 7f1365786740 1 -- 172.21.15.67:0/3067217500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55b4473eca70 con 0x55b446f97660 2024-07-31T15:02:32.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.536+0000 7f1337fff640 1 -- 172.21.15.67:0/3067217500 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b4472d9860 con 0x55b446f97660 2024-07-31T15:02:32.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.536+0000 7f1337fff640 1 -- 172.21.15.67:0/3067217500 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 60) ==== 8386+0+0 (secure 0 0 0) 0x7f13480c61d0 con 0x55b446f97660 2024-07-31T15:02:32.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.536+0000 7f1365786740 10 client.27527 fetch_fsmap finished waiting for FSMap version 60 2024-07-31T15:02:32.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.536+0000 7f1365786740 10 client.27527 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:02:32.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.536+0000 7f1365786740 20 client.27527 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:32.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.536+0000 7f1365786740 1 --2- 172.21.15.67:0/3067217500 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4473ed000 0x55b4473ef420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:32.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.536+0000 7f1365786740 4 client.27527 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f864f25e", "release": true, "await": true}] 2024-07-31T15:02:32.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.536+0000 7f1365786740 1 -- 172.21.15.67:0/3067217500 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f864f25e", "release": true, "await": true}) -- 0x55b4472d6fa0 con 0x55b4473ed000 2024-07-31T15:02:32.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.538+0000 7f135cfd1640 1 --2- 172.21.15.67:0/3067217500 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4473ed000 0x55b4473ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:32.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.541+0000 7f135cfd1640 1 --2- 172.21.15.67:0/3067217500 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4473ed000 0x55b4473ef420 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3352108a54305f7a server_cookie=9fa25a77393bbc55 in_seq=0 out_seq=0 2024-07-31T15:02:32.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:32.543+0000 7f1337fff640 10 client.27527 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:33 smithi077 ceph-mon[37327]: pgmap v758: 129 pgs: 24 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 92 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 133 KiB/s wr, 550 op/s 2024-07-31T15:02:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:33.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:33.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:33.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:33 smithi073 ceph-mon[36714]: pgmap v758: 129 pgs: 24 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 92 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 133 KiB/s wr, 550 op/s 2024-07-31T15:02:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:33.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:33.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:33.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:33.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:33 smithi067 ceph-mon[32750]: pgmap v758: 129 pgs: 24 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 92 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 133 KiB/s wr, 550 op/s 2024-07-31T15:02:33.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:33.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:33.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:33.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:33.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:33.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:33.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:33.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:33.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:34.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:34 smithi077 ceph-mon[37327]: pgmap v759: 129 pgs: 23 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 95 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 138 KiB/s wr, 755 op/s 2024-07-31T15:02:34.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:34.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:34 smithi067 ceph-mon[32750]: pgmap v759: 129 pgs: 23 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 95 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 138 KiB/s wr, 755 op/s 2024-07-31T15:02:34.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:34 smithi073 ceph-mon[36714]: pgmap v759: 129 pgs: 23 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 95 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 138 KiB/s wr, 755 op/s 2024-07-31T15:02:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.989+0000 7fda076e3640 1 -- 172.21.15.67:0/38260509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda001097a0 msgr2=0x7fda0010ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.989+0000 7fda076e3640 1 --2- 172.21.15.67:0/38260509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda001097a0 0x7fda0010ffd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009ca0 tx=0x7fd9f802a8f0 comp rx=0 tx=0).stop 2024-07-31T15:02:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.989+0000 7fda076e3640 1 -- 172.21.15.67:0/38260509 shutdown_connections 2024-07-31T15:02:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.989+0000 7fda076e3640 1 --2- 172.21.15.67:0/38260509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda001097a0 0x7fda0010ffd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.989+0000 7fda076e3640 1 --2- 172.21.15.67:0/38260509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda00108e50 0x7fda00109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.989+0000 7fda076e3640 1 --2- 172.21.15.67:0/38260509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda00103520 0x7fda00103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.989+0000 7fda076e3640 1 -- 172.21.15.67:0/38260509 >> 172.21.15.67:0/38260509 conn(0x7fda000fefa0 msgr2=0x7fda001013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.991+0000 7fda076e3640 1 -- 172.21.15.67:0/38260509 shutdown_connections 2024-07-31T15:02:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.991+0000 7fda076e3640 1 -- 172.21.15.67:0/38260509 wait complete. 2024-07-31T15:02:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.992+0000 7fda076e3640 1 Processor -- start 2024-07-31T15:02:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.992+0000 7fda076e3640 1 -- start start 2024-07-31T15:02:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.992+0000 7fda076e3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda00103520 0x7fda0010d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.992+0000 7fda076e3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda00108e50 0x7fda0010da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.992+0000 7fda076e3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda001097a0 0x7fda0010dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.992+0000 7fda076e3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fda00076bd0 con 0x7fda00108e50 2024-07-31T15:02:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.992+0000 7fda076e3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fda00076a50 con 0x7fda001097a0 2024-07-31T15:02:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.992+0000 7fda076e3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda00076d50 con 0x7fda00103520 2024-07-31T15:02:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.993+0000 7fda05c80640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda001097a0 0x7fda0010dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.993+0000 7fda05c80640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda001097a0 0x7fda0010dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:50672/0 (socket says 172.21.15.67:50672) 2024-07-31T15:02:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.993+0000 7fda05c80640 1 -- 172.21.15.67:0/923080974 learned_addr learned my addr 172.21.15.67:0/923080974 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.993+0000 7fda04c7e640 1 --2- 172.21.15.67:0/923080974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda00108e50 0x7fda0010da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.993+0000 7fda05c80640 1 -- 172.21.15.67:0/923080974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda00103520 msgr2=0x7fda0010d550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:02:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.994+0000 7fda05c80640 1 --2- 172.21.15.67:0/923080974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda00103520 0x7fda0010d550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.994+0000 7fda05c80640 1 -- 172.21.15.67:0/923080974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda00108e50 msgr2=0x7fda0010da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.994+0000 7fda05c80640 1 --2- 172.21.15.67:0/923080974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda00108e50 0x7fda0010da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.994+0000 7fda05c80640 1 -- 172.21.15.67:0/923080974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda001ac510 con 0x7fda001097a0 2024-07-31T15:02:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.994+0000 7fda05c80640 1 --2- 172.21.15.67:0/923080974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda001097a0 0x7fda0010dfd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8031a10 tx=0x7fd9f8003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.994+0000 7fd9f67fc640 1 -- 172.21.15.67:0/923080974 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80307f0 con 0x7fda001097a0 2024-07-31T15:02:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.995+0000 7fda076e3640 1 -- 172.21.15.67:0/923080974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda001ac7a0 con 0x7fda001097a0 2024-07-31T15:02:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.995+0000 7fda076e3640 1 -- 172.21.15.67:0/923080974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda001acdc0 con 0x7fda001097a0 2024-07-31T15:02:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.995+0000 7fd9f67fc640 1 -- 172.21.15.67:0/923080974 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9f802d070 con 0x7fda001097a0 2024-07-31T15:02:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:34.995+0000 7fd9f67fc640 1 -- 172.21.15.67:0/923080974 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80307f0 con 0x7fda001097a0 2024-07-31T15:02:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.001+0000 7fda0e5d4740 1 Processor -- start 2024-07-31T15:02:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.002+0000 7fd9f67fc640 1 -- 172.21.15.67:0/923080974 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd9f8030990 con 0x7fda001097a0 2024-07-31T15:02:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.003+0000 7fd9f67fc640 1 --2- 172.21.15.67:0/923080974 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9ec0b4140 0x7fd9ec0b6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.003+0000 7fd9f67fc640 1 -- 172.21.15.67:0/923080974 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9439+0+0 (secure 0 0 0) 0x7fd9f80305f0 con 0x7fda001097a0 2024-07-31T15:02:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.004+0000 7fda0e5d4740 1 -- start start 2024-07-31T15:02:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.004+0000 7fda0e5d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6fd1b0 0x55b82ba9fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.004+0000 7fda0e5d4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6d42f0 0x55b82baa0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.004+0000 7fda0e5d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6dc4d0 0x55b82baa0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.005+0000 7fda0e5d4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b82ba153b0 con 0x55b82b6d42f0 2024-07-31T15:02:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.005+0000 7fda0e5d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b82ba048b0 con 0x55b82b6dc4d0 2024-07-31T15:02:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.005+0000 7fda0547f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6fd1b0 0x55b82ba9fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.005+0000 7fda0547f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6fd1b0 0x55b82ba9fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35488/0 (socket says 172.21.15.67:35488) 2024-07-31T15:02:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.005+0000 7fda0547f640 1 -- 172.21.15.67:0/2031017473 learned_addr learned my addr 172.21.15.67:0/2031017473 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.005+0000 7fda05c80640 1 --2- 172.21.15.67:0/2031017473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6dc4d0 0x55b82baa0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.005+0000 7fda0e5d4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b82ba088f0 con 0x55b82b6fd1b0 2024-07-31T15:02:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.005+0000 7fda04c7e640 1 --2- 172.21.15.67:0/923080974 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9ec0b4140 0x7fd9ec0b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.005+0000 7fda0547f640 1 -- 172.21.15.67:0/2031017473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6dc4d0 msgr2=0x55b82baa0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.005+0000 7fda0547f640 1 --2- 172.21.15.67:0/2031017473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6dc4d0 0x55b82baa0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.005+0000 7fda0547f640 1 -- 172.21.15.67:0/2031017473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6d42f0 msgr2=0x55b82baa0220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:02:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.005+0000 7fda0547f640 1 --2- 172.21.15.67:0/2031017473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6d42f0 0x55b82baa0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.005+0000 7fda0547f640 1 -- 172.21.15.67:0/2031017473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82b9ab9a0 con 0x55b82b6fd1b0 2024-07-31T15:02:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.006+0000 7fda0547f640 1 --2- 172.21.15.67:0/2031017473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6fd1b0 0x55b82ba9fce0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009ba0 tx=0x7fd9fc00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeebd9974088f1db server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.006+0000 7fd9eb7fe640 1 -- 172.21.15.67:0/2031017473 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc005020 con 0x55b82b6fd1b0 2024-07-31T15:02:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.006+0000 7fda04c7e640 1 --2- 172.21.15.67:0/923080974 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9ec0b4140 0x7fd9ec0b6650 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x55b82ba9e960 tx=0x7fd9f0009330 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.006+0000 7fd9eb7fe640 1 -- 172.21.15.67:0/2031017473 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9fc0047a0 con 0x55b82b6fd1b0 2024-07-31T15:02:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.007+0000 7fda0e5d4740 1 -- 172.21.15.67:0/2031017473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6fd1b0 msgr2=0x55b82ba9fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.007+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/2031017473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6fd1b0 0x55b82ba9fce0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009ba0 tx=0x7fd9fc00e720 comp rx=0 tx=0).stop 2024-07-31T15:02:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.008+0000 7fda0e5d4740 1 -- 172.21.15.67:0/2031017473 shutdown_connections 2024-07-31T15:02:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.008+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/2031017473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6dc4d0 0x55b82baa0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.008+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/2031017473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6d42f0 0x55b82baa0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.008+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/2031017473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6fd1b0 0x55b82ba9fce0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.008+0000 7fda0e5d4740 1 -- 172.21.15.67:0/2031017473 >> 172.21.15.67:0/2031017473 conn(0x55b82b6f1300 msgr2=0x55b82ba9dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.008+0000 7fda0e5d4740 1 -- 172.21.15.67:0/2031017473 shutdown_connections 2024-07-31T15:02:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.008+0000 7fda0e5d4740 1 -- 172.21.15.67:0/2031017473 wait complete. 2024-07-31T15:02:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.009+0000 7fda0e5d4740 1 -- start start 2024-07-31T15:02:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.009+0000 7fda0e5d4740 1 Processor -- start 2024-07-31T15:02:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.010+0000 7fda0e5d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6d42f0 0x55b82baab400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.010+0000 7fda0e5d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6dc4d0 0x55b82baab940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.010+0000 7fda0e5d4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6fd1b0 0x55b82bab78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.010+0000 7fda0547f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6d42f0 0x55b82baab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.010+0000 7fda0547f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6d42f0 0x55b82baab400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:50692/0 (socket says 172.21.15.67:50692) 2024-07-31T15:02:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.010+0000 7fda0547f640 1 -- 172.21.15.67:0/1348089913 learned_addr learned my addr 172.21.15.67:0/1348089913 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.010+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1348089913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b82b93eab0 con 0x55b82b6fd1b0 2024-07-31T15:02:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.010+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1348089913 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b82b98ccf0 con 0x55b82b6d42f0 2024-07-31T15:02:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.010+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1348089913 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b82baae500 con 0x55b82b6dc4d0 2024-07-31T15:02:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.010+0000 7fda04c7e640 1 --2- 172.21.15.67:0/1348089913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6dc4d0 0x55b82baab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.010+0000 7fda0e5d4740 10 client.0 _ll_register_callbacks cb 0x55b82b9fe9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.010+0000 7fda05c80640 1 --2- 172.21.15.67:0/1348089913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6fd1b0 0x55b82bab78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.011+0000 7fda0547f640 1 -- 172.21.15.67:0/1348089913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6dc4d0 msgr2=0x55b82baab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.011+0000 7fda0547f640 1 --2- 172.21.15.67:0/1348089913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6dc4d0 0x55b82baab940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.011+0000 7fda0547f640 1 -- 172.21.15.67:0/1348089913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6fd1b0 msgr2=0x55b82bab78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.011+0000 7fda0547f640 1 --2- 172.21.15.67:0/1348089913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6fd1b0 0x55b82bab78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.011+0000 7fda0547f640 1 -- 172.21.15.67:0/1348089913 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82b768790 con 0x55b82b6d42f0 2024-07-31T15:02:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.011+0000 7fda05c80640 1 --2- 172.21.15.67:0/1348089913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6fd1b0 0x55b82bab78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:02:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.011+0000 7fda0547f640 1 --2- 172.21.15.67:0/1348089913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6d42f0 0x55b82baab400 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc005de0 tx=0x7fd9fc0056e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab8caa4691ce0909 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.011+0000 7fd9e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:02:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.012+0000 7fd9e8ff9640 1 -- 172.21.15.67:0/1348089913 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc01d040 con 0x55b82b6d42f0 2024-07-31T15:02:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.012+0000 7fd9e8ff9640 1 -- 172.21.15.67:0/1348089913 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9439+0+0 (secure 0 0 0) 0x7fd9fc005200 con 0x55b82b6d42f0 2024-07-31T15:02:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.012+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1348089913 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9fc0047a0 con 0x55b82b6d42f0 2024-07-31T15:02:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.012+0000 7fd9e8ff9640 1 -- 172.21.15.67:0/1348089913 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fd9fc030050 con 0x55b82b6d42f0 2024-07-31T15:02:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.012+0000 7fda0e5d4740 10 client.27539 fetch_fsmap learned FSMap version 60 2024-07-31T15:02:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.012+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1348089913 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55b82b7e1620 con 0x55b82b6d42f0 2024-07-31T15:02:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.013+0000 7fd9e8ff9640 1 -- 172.21.15.67:0/1348089913 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9fc0047a0 con 0x55b82b6d42f0 2024-07-31T15:02:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.013+0000 7fd9e8ff9640 1 -- 172.21.15.67:0/1348089913 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 60) ==== 8386+0+0 (secure 0 0 0) 0x7fd9fc0185f0 con 0x55b82b6d42f0 2024-07-31T15:02:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.014+0000 7fda0e5d4740 10 client.27539 fetch_fsmap finished waiting for FSMap version 60 2024-07-31T15:02:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.014+0000 7fda0e5d4740 10 client.27539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:02:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.014+0000 7fda0e5d4740 20 client.27539 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.014+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/1348089913 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b82b6bbd80 0x55b82babd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.014+0000 7fda0e5d4740 4 client.27539 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:02:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.014+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1348089913 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b82ba17300 con 0x55b82b6bbd80 2024-07-31T15:02:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.014+0000 7fda04c7e640 1 --2- 172.21.15.67:0/1348089913 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b82b6bbd80 0x55b82babd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.015+0000 7fda04c7e640 1 --2- 172.21.15.67:0/1348089913 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b82b6bbd80 0x55b82babd5b0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258dd5eb6f780265 server_cookie=89adda3a42ef1b88 in_seq=0 out_seq=0 2024-07-31T15:02:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.015+0000 7fd9e8ff9640 10 client.27539 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fd9e8ff9640 1 -- 172.21.15.67:0/1348089913 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b82ba17300 con 0x55b82b6bbd80 2024-07-31T15:02:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fd9e8ff9640 10 client.27539 handle_command_reply: tid=0 2024-07-31T15:02:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fda0e5d4740 1 client.27539 shutdown 2024-07-31T15:02:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1348089913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6d42f0 msgr2=0x55b82baab400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/1348089913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6d42f0 0x55b82baab400 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc005de0 tx=0x7fd9fc0056e0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1348089913 shutdown_connections 2024-07-31T15:02:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/1348089913 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b82b6bbd80 0x55b82babd5b0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/1348089913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6fd1b0 0x55b82bab78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/1348089913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6dc4d0 0x55b82baab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fd9e8ff9640 0 client.27539 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/1348089913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6d42f0 0x55b82baab400 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1348089913 >> 172.21.15.67:0/1348089913 conn(0x55b82b6f1300 msgr2=0x55b82baa0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1348089913 shutdown_connections 2024-07-31T15:02:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1348089913 wait complete. 2024-07-31T15:02:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.925+0000 7fda0e5d4740 20 client.27539 trim_cache size 0 max 16384 2024-07-31T15:02:35.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.930+0000 7f1337fff640 1 -- 172.21.15.67:0/3067217500 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b4472d6fa0 con 0x55b4473ed000 2024-07-31T15:02:35.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.930+0000 7f1337fff640 10 client.27527 handle_command_reply: tid=0 2024-07-31T15:02:35.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.930+0000 7f1365786740 1 client.27527 shutdown 2024-07-31T15:02:35.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.931+0000 7f1365786740 1 -- 172.21.15.67:0/3067217500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446f97660 msgr2=0x55b447376120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:35.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.931+0000 7f1365786740 1 --2- 172.21.15.67:0/3067217500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446f97660 0x55b447376120 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55b44726b1b0 tx=0x7f13480d3130 comp rx=0 tx=0).stop 2024-07-31T15:02:35.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.931+0000 7f1365786740 1 -- 172.21.15.67:0/3067217500 shutdown_connections 2024-07-31T15:02:35.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.931+0000 7f1365786740 1 --2- 172.21.15.67:0/3067217500 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4473ed000 0x55b4473ef420 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.931+0000 7f1365786740 1 --2- 172.21.15.67:0/3067217500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b446fc0520 0x55b447376660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.931+0000 7f1365786740 1 --2- 172.21.15.67:0/3067217500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b446f97660 0x55b447376120 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.931+0000 7f1365786740 1 --2- 172.21.15.67:0/3067217500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b446f54250 0x55b447375be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.931+0000 7f1365786740 1 -- 172.21.15.67:0/3067217500 >> 172.21.15.67:0/3067217500 conn(0x55b446fb4910 msgr2=0x55b447363f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:35.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.931+0000 7f1337fff640 0 client.27527 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:35.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.931+0000 7f1365786740 1 -- 172.21.15.67:0/3067217500 shutdown_connections 2024-07-31T15:02:35.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.931+0000 7f1365786740 1 -- 172.21.15.67:0/3067217500 wait complete. 2024-07-31T15:02:35.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.931+0000 7f1365786740 20 client.27527 trim_cache size 0 max 16384 2024-07-31T15:02:35.934 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:02:35.934 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 60, 2024-07-31T15:02:35.934 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:02:35.934 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 28, 2024-07-31T15:02:35.935 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:02:35.935 INFO:teuthology.orchestra.run.smithi067.stdout: "f864f25e": { 2024-07-31T15:02:35.935 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 28, 2024-07-31T15:02:35.935 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:02:35.935 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:02:35.935 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:02:35.935 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:02:35.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:02:35.935 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:02:35.935 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 130.0, 2024-07-31T15:02:35.935 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:02:35.935 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:02:35.935 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:02:35.936 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:02:35.936 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:02:35.936 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:02:35.936 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:02:35.936 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:02:35.936 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:02:35.936 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:02:35.936 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:02:35.936 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:02:35.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.932+0000 7f135f235640 1 -- 172.21.15.67:0/1114357241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f13400c4f60 msgr2=0x7f13400c7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.932+0000 7f135f235640 1 --2- 172.21.15.67:0/1114357241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f13400c4f60 0x7f13400c7470 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1350005fd0 tx=0x7f1350005f60 comp rx=0 tx=0).stop 2024-07-31T15:02:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.933+0000 7f135f235640 1 -- 172.21.15.67:0/1114357241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1358111970 msgr2=0x7f13581a9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.933+0000 7f135f235640 1 --2- 172.21.15.67:0/1114357241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1358111970 0x7f13581a9360 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f1348009780 tx=0x7f1348030d80 comp rx=0 tx=0).stop 2024-07-31T15:02:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.933+0000 7f135f235640 1 -- 172.21.15.67:0/1114357241 shutdown_connections 2024-07-31T15:02:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.933+0000 7f135f235640 1 --2- 172.21.15.67:0/1114357241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f13400c4f60 0x7f13400c7470 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.933+0000 7f135f235640 1 --2- 172.21.15.67:0/1114357241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f135811c660 0x7f13581ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.933+0000 7f135f235640 1 --2- 172.21.15.67:0/1114357241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1358111970 0x7f13581a9360 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.933+0000 7f135f235640 1 --2- 172.21.15.67:0/1114357241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f135810a410 0x7f13581a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.933+0000 7f135f235640 1 -- 172.21.15.67:0/1114357241 >> 172.21.15.67:0/1114357241 conn(0x7f13580708f0 msgr2=0x7f1358071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.933+0000 7f135f235640 1 -- 172.21.15.67:0/1114357241 shutdown_connections 2024-07-31T15:02:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:35.933+0000 7f135f235640 1 -- 172.21.15.67:0/1114357241 wait complete. 2024-07-31T15:02:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.954+0000 7fda0e5d4740 1 Processor -- start 2024-07-31T15:02:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.954+0000 7fda0e5d4740 1 -- start start 2024-07-31T15:02:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.955+0000 7fda0e5d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6d42f0 0x55b82baad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.955+0000 7fda0e5d4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6dc4d0 0x55b82baad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.955+0000 7fda0547f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6d42f0 0x55b82baad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.955+0000 7fda0547f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6d42f0 0x55b82baad030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:50704/0 (socket says 172.21.15.67:50704) 2024-07-31T15:02:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.955+0000 7fda04c7e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6dc4d0 0x55b82baad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.955+0000 7fda0547f640 1 -- 172.21.15.67:0/3706899270 learned_addr learned my addr 172.21.15.67:0/3706899270 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.955+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/3706899270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6fd1b0 0x55b82bab76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.955+0000 7fda0e5d4740 1 -- 172.21.15.67:0/3706899270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e4010010 con 0x55b82b6dc4d0 2024-07-31T15:02:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.955+0000 7fda0e5d4740 1 -- 172.21.15.67:0/3706899270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e4011c10 con 0x55b82b6d42f0 2024-07-31T15:02:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.955+0000 7fda0e5d4740 1 -- 172.21.15.67:0/3706899270 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e40106e0 con 0x55b82b6fd1b0 2024-07-31T15:02:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.956+0000 7fda0547f640 1 -- 172.21.15.67:0/3706899270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6fd1b0 msgr2=0x55b82bab76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.956+0000 7fda0547f640 1 --2- 172.21.15.67:0/3706899270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6fd1b0 0x55b82bab76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.956+0000 7fda0547f640 1 -- 172.21.15.67:0/3706899270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6dc4d0 msgr2=0x55b82baad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.956+0000 7fda0547f640 1 --2- 172.21.15.67:0/3706899270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6dc4d0 0x55b82baad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.956+0000 7fda0547f640 1 -- 172.21.15.67:0/3706899270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82baaa340 con 0x55b82b6d42f0 2024-07-31T15:02:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.956+0000 7fda04c7e640 1 --2- 172.21.15.67:0/3706899270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6dc4d0 0x55b82baad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:02:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.956+0000 7fda0547f640 1 --2- 172.21.15.67:0/3706899270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6d42f0 0x55b82baad030 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0189c0 tx=0x7fd9fc018e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7a78ddd540a241a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.957+0000 7fd9eaffd640 1 -- 172.21.15.67:0/3706899270 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc038020 con 0x55b82b6d42f0 2024-07-31T15:02:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.957+0000 7fd9eaffd640 1 -- 172.21.15.67:0/3706899270 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9fc027070 con 0x55b82b6d42f0 2024-07-31T15:02:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.957+0000 7fd9eaffd640 1 -- 172.21.15.67:0/3706899270 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc038020 con 0x55b82b6d42f0 2024-07-31T15:02:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.959+0000 7fda0e5d4740 1 -- 172.21.15.67:0/3706899270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6d42f0 msgr2=0x55b82baad030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.959+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/3706899270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6d42f0 0x55b82baad030 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0189c0 tx=0x7fd9fc018e40 comp rx=0 tx=0).stop 2024-07-31T15:02:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.959+0000 7fda0e5d4740 1 -- 172.21.15.67:0/3706899270 shutdown_connections 2024-07-31T15:02:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.959+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/3706899270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6fd1b0 0x55b82bab76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.959+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/3706899270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6dc4d0 0x55b82baad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.959+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/3706899270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6d42f0 0x55b82baad030 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.959+0000 7fda0e5d4740 1 -- 172.21.15.67:0/3706899270 >> 172.21.15.67:0/3706899270 conn(0x55b82b6f1300 msgr2=0x55b82baac920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.960+0000 7fda0e5d4740 1 -- 172.21.15.67:0/3706899270 shutdown_connections 2024-07-31T15:02:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.960+0000 7fda0e5d4740 1 -- 172.21.15.67:0/3706899270 wait complete. 2024-07-31T15:02:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.960+0000 7fda0e5d4740 1 -- start start 2024-07-31T15:02:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.960+0000 7fda0e5d4740 1 Processor -- start 2024-07-31T15:02:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.961+0000 7fda0e5d4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6d42f0 0x55b82bab0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.961+0000 7fda0e5d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6dc4d0 0x55b82bab0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.961+0000 7fda0e5d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6fd1b0 0x55b82bb23bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda04c7e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6dc4d0 0x55b82bab0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda04c7e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6dc4d0 0x55b82bab0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35510/0 (socket says 172.21.15.67:35510) 2024-07-31T15:02:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda04c7e640 1 -- 172.21.15.67:0/1973131545 learned_addr learned my addr 172.21.15.67:0/1973131545 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1973131545 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e4011540 con 0x55b82b6d42f0 2024-07-31T15:02:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1973131545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e4010db0 con 0x55b82b6fd1b0 2024-07-31T15:02:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1973131545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e400e010 con 0x55b82b6dc4d0 2024-07-31T15:02:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda05c80640 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6fd1b0 0x55b82bb23bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda0e5d4740 10 client.0 _ll_register_callbacks cb 0x7fd9f8030fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda04c7e640 1 -- 172.21.15.67:0/1973131545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6fd1b0 msgr2=0x55b82bb23bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda04c7e640 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6fd1b0 0x55b82bb23bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda0547f640 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6d42f0 0x55b82bab0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda04c7e640 1 -- 172.21.15.67:0/1973131545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6d42f0 msgr2=0x55b82bab0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda04c7e640 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6d42f0 0x55b82bab0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda04c7e640 1 -- 172.21.15.67:0/1973131545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82bab16f0 con 0x55b82b6dc4d0 2024-07-31T15:02:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda05c80640 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6fd1b0 0x55b82bb23bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:02:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.962+0000 7fda0547f640 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6d42f0 0x55b82bab0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:02:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.963+0000 7fda04c7e640 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6dc4d0 0x55b82bab0ed0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00097b0 tx=0x7fd9f0011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa658d3ad8a3d709 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.963+0000 7fd9e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:02:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.963+0000 7fd9e8ff9640 1 -- 172.21.15.67:0/1973131545 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0025810 con 0x55b82b6dc4d0 2024-07-31T15:02:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.963+0000 7fd9e8ff9640 1 -- 172.21.15.67:0/1973131545 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9439+0+0 (secure 0 0 0) 0x7fd9f00259b0 con 0x55b82b6dc4d0 2024-07-31T15:02:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.963+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1973131545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9fc027070 con 0x55b82b6dc4d0 2024-07-31T15:02:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.964+0000 7fd9e8ff9640 1 -- 172.21.15.67:0/1973131545 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fd9f0025bb0 con 0x55b82b6dc4d0 2024-07-31T15:02:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.964+0000 7fda0e5d4740 10 client.27520 fetch_fsmap learned FSMap version 60 2024-07-31T15:02:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.964+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1973131545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55b82bb25140 con 0x55b82b6dc4d0 2024-07-31T15:02:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.964+0000 7fd9e8ff9640 1 -- 172.21.15.67:0/1973131545 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9fc027070 con 0x55b82b6dc4d0 2024-07-31T15:02:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.964+0000 7fd9e8ff9640 1 -- 172.21.15.67:0/1973131545 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 60) ==== 8386+0+0 (secure 0 0 0) 0x7fd9f0013070 con 0x55b82b6dc4d0 2024-07-31T15:02:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.964+0000 7fda0e5d4740 10 client.27520 fetch_fsmap finished waiting for FSMap version 60 2024-07-31T15:02:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.964+0000 7fda0e5d4740 10 client.27520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:02:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.964+0000 7fda0e5d4740 20 client.27520 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.964+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b82b6911c0 0x55b82bb279f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.964+0000 7fda0e5d4740 4 client.27520 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:02:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.964+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1973131545 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b82ba12da0 con 0x55b82b6911c0 2024-07-31T15:02:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.965+0000 7fda0547f640 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b82b6911c0 0x55b82bb279f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.965+0000 7fda0547f640 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b82b6911c0 0x55b82bb279f0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87527a18d2542aee server_cookie=f1b7208394967a96 in_seq=0 out_seq=0 2024-07-31T15:02:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:35.966+0000 7fd9e8ff9640 10 client.27520 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:36.065 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vclock_gettime.o 2024-07-31T15:02:36.333 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/notification.o 2024-07-31T15:02:36.374 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/util.o 2024-07-31T15:02:36.411 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/amd/iommu.o 2024-07-31T15:02:36.622 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/msgutil.o 2024-07-31T15:02:36.638 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:02:36.686 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/sched/fair.o 2024-07-31T15:02:36.853 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f864f25e', total seconds elapsed: 67.9 2024-07-31T15:02:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.906+0000 7fd9e8ff9640 1 -- 172.21.15.67:0/1973131545 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b82ba12da0 con 0x55b82b6911c0 2024-07-31T15:02:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.906+0000 7fd9e8ff9640 10 client.27520 handle_command_reply: tid=0 2024-07-31T15:02:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.906+0000 7fda0e5d4740 1 client.27520 shutdown 2024-07-31T15:02:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.906+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1973131545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6dc4d0 msgr2=0x55b82bab0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.906+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6dc4d0 0x55b82bab0ed0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00097b0 tx=0x7fd9f0011000 comp rx=0 tx=0).stop 2024-07-31T15:02:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.906+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1973131545 shutdown_connections 2024-07-31T15:02:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.906+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b82b6911c0 0x55b82bb279f0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.906+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b82b6fd1b0 0x55b82bb23bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.906+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82b6dc4d0 0x55b82bab0ed0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.906+0000 7fd9e8ff9640 0 client.27520 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.906+0000 7fda0e5d4740 1 --2- 172.21.15.67:0/1973131545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b82b6d42f0 0x55b82bab0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.906+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1973131545 >> 172.21.15.67:0/1973131545 conn(0x55b82b6f1300 msgr2=0x55b82baad270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.906+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1973131545 shutdown_connections 2024-07-31T15:02:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.906+0000 7fda0e5d4740 1 -- 172.21.15.67:0/1973131545 wait complete. 2024-07-31T15:02:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.907+0000 7fda0e5d4740 20 client.27520 trim_cache size 0 max 16384 2024-07-31T15:02:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:02:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:02:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:02:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:02:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.914+0000 7fda076e3640 1 -- 172.21.15.67:0/923080974 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9ec0b4140 msgr2=0x7fd9ec0b6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.914+0000 7fda076e3640 1 --2- 172.21.15.67:0/923080974 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9ec0b4140 0x7fd9ec0b6650 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x55b82ba9e960 tx=0x7fd9f0009330 comp rx=0 tx=0).stop 2024-07-31T15:02:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.914+0000 7fda076e3640 1 -- 172.21.15.67:0/923080974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda001097a0 msgr2=0x7fda0010dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.914+0000 7fda076e3640 1 --2- 172.21.15.67:0/923080974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda001097a0 0x7fda0010dfd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8031a10 tx=0x7fd9f8003ba0 comp rx=0 tx=0).stop 2024-07-31T15:02:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.918+0000 7fda076e3640 1 -- 172.21.15.67:0/923080974 shutdown_connections 2024-07-31T15:02:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.918+0000 7fda076e3640 1 --2- 172.21.15.67:0/923080974 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd9ec0b4140 0x7fd9ec0b6650 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.918+0000 7fda076e3640 1 --2- 172.21.15.67:0/923080974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda001097a0 0x7fda0010dfd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.918+0000 7fda076e3640 1 --2- 172.21.15.67:0/923080974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda00108e50 0x7fda0010da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.918+0000 7fda076e3640 1 --2- 172.21.15.67:0/923080974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda00103520 0x7fda0010d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.918+0000 7fda076e3640 1 -- 172.21.15.67:0/923080974 >> 172.21.15.67:0/923080974 conn(0x7fda000fefa0 msgr2=0x7fda00108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.918+0000 7fda076e3640 1 -- 172.21.15.67:0/923080974 shutdown_connections 2024-07-31T15:02:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:36.919+0000 7fda076e3640 1 -- 172.21.15.67:0/923080974 wait complete. 2024-07-31T15:02:37.025 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/group.o 2024-07-31T15:02:37.028 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mempool.o 2024-07-31T15:02:37.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:36 smithi067 ceph-mon[32750]: pgmap v760: 129 pgs: 17 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 102 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 161 KiB/s wr, 745 op/s 2024-07-31T15:02:37.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:37.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:37.072 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vgetcpu.o 2024-07-31T15:02:37.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:36 smithi073 ceph-mon[36714]: pgmap v760: 129 pgs: 17 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 102 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 161 KiB/s wr, 745 op/s 2024-07-31T15:02:37.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:37.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:37.169 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/do_mounts.o 2024-07-31T15:02:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:36 smithi077 ceph-mon[37327]: pgmap v760: 129 pgs: 17 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 102 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 161 KiB/s wr, 745 op/s 2024-07-31T15:02:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:37.258 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC arch/x86/entry/vdso/vdso2c 2024-07-31T15:02:37.432 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/events/amd/built-in.a 2024-07-31T15:02:37.490 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/msg.o 2024-07-31T15:02:37.530 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:02:37.607 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/core.o 2024-07-31T15:02:37.727 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/sched/build_policy.o 2024-07-31T15:02:37.974 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/mark.o 2024-07-31T15:02:38.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:38.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:38.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:38.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:38.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:38.058 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.059 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.059 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.059 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:38.059 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.097 INFO:tasks.workunit.client.0.smithi067.stdout: LDS arch/x86/entry/vdso/vdso32/vdso32.lds 2024-07-31T15:02:38.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:38.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:38.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:38.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:38.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:38.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:38.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.192 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/vdso/vdso32/note.o 2024-07-31T15:02:38.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:38.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:38.310 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/vdso/vdso32/system_call.o 2024-07-31T15:02:38.338 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/oom_kill.o 2024-07-31T15:02:38.556 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/vdso/vdso32/sigreturn.o 2024-07-31T15:02:38.703 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vdso32/vclock_gettime.o 2024-07-31T15:02:38.890 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/sem.o 2024-07-31T15:02:38.955 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/fdinfo.o 2024-07-31T15:02:39.068 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:39.068 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:39.068 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:39.068 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:39.068 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:38 smithi067 ceph-mon[32750]: pgmap v761: 129 pgs: 17 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 102 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 67 KiB/s wr, 765 op/s 2024-07-31T15:02:39.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:39.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:39.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:39.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:39.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:38 smithi073 ceph-mon[36714]: pgmap v761: 129 pgs: 17 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 102 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 67 KiB/s wr, 765 op/s 2024-07-31T15:02:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:38 smithi077 ceph-mon[37327]: pgmap v761: 129 pgs: 17 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 102 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 67 KiB/s wr, 765 op/s 2024-07-31T15:02:39.964 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vdso32/vgetcpu.o 2024-07-31T15:02:39.995 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/fadvise.o 2024-07-31T15:02:40.016 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/do_mounts_initrd.o 2024-07-31T15:02:40.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:40.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:40.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:40.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:40.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:40.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:40.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:40.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:40.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:40.268 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/notify/built-in.a 2024-07-31T15:02:40.277 INFO:tasks.workunit.client.0.smithi067.stdout: VDSO arch/x86/entry/vdso/vdso64.so.dbg 2024-07-31T15:02:40.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:40.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:40.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:40.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:40.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:40.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:40.379 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs_common/nfsacl.o 2024-07-31T15:02:40.505 INFO:tasks.workunit.client.0.smithi067.stdout: VDSO arch/x86/entry/vdso/vdso32.so.dbg 2024-07-31T15:02:40.571 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/shm.o 2024-07-31T15:02:40.614 INFO:tasks.workunit.client.0.smithi067.stdout: OBJCOPY arch/x86/entry/vdso/vdso64.so 2024-07-31T15:02:40.658 INFO:tasks.workunit.client.0.smithi067.stdout: OBJCOPY arch/x86/entry/vdso/vdso32.so 2024-07-31T15:02:40.710 INFO:tasks.workunit.client.0.smithi067.stdout: VDSO2C arch/x86/entry/vdso/vdso-image-64.c 2024-07-31T15:02:40.728 INFO:tasks.workunit.client.0.smithi067.stdout: VDSO2C arch/x86/entry/vdso/vdso-image-32.c 2024-07-31T15:02:40.744 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vdso-image-64.o 2024-07-31T15:02:40.801 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/initramfs.o 2024-07-31T15:02:40.822 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/bts.o 2024-07-31T15:02:40.823 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/syscall.o 2024-07-31T15:02:40.857 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/maccess.o 2024-07-31T15:02:41.019 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vdso-image-32.o 2024-07-31T15:02:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:40 smithi073 ceph-mon[36714]: pgmap v762: 129 pgs: 6 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 116 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 182 KiB/s wr, 825 op/s 2024-07-31T15:02:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:40 smithi073 ceph-mon[36714]: osdmap e105: 12 total, 12 up, 12 in 2024-07-31T15:02:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:41.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:41.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:41.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:41.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:40 smithi067 ceph-mon[32750]: pgmap v762: 129 pgs: 6 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 116 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 182 KiB/s wr, 825 op/s 2024-07-31T15:02:41.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:41.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:40 smithi067 ceph-mon[32750]: osdmap e105: 12 total, 12 up, 12 in 2024-07-31T15:02:41.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:41.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:41.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:41.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:41.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:40 smithi077 ceph-mon[37327]: pgmap v762: 129 pgs: 6 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 116 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 182 KiB/s wr, 825 op/s 2024-07-31T15:02:41.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:41.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:40 smithi077 ceph-mon[37327]: osdmap e105: 12 total, 12 up, 12 in 2024-07-31T15:02:41.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:41.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:41.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:41.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:41.244 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs_common/grace.o 2024-07-31T15:02:41.377 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/entry/vdso/built-in.a 2024-07-31T15:02:41.576 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vsyscall/vsyscall_64.o 2024-07-31T15:02:41.824 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/page-writeback.o 2024-07-31T15:02:41.871 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/ds.o 2024-07-31T15:02:41.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.911+0000 7f3d482e4640 1 -- 172.21.15.67:0/4162806979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d4010c700 msgr2=0x7f3d4010cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:41.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.911+0000 7f3d482e4640 1 --2- 172.21.15.67:0/4162806979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d4010c700 0x7f3d4010cb10 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009be0 tx=0x7f3d3802a710 comp rx=0 tx=0).stop 2024-07-31T15:02:41.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.913+0000 7f3d482e4640 1 -- 172.21.15.67:0/4162806979 shutdown_connections 2024-07-31T15:02:41.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.913+0000 7f3d482e4640 1 --2- 172.21.15.67:0/4162806979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d4010d050 0x7f3d40116ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:41.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.913+0000 7f3d482e4640 1 --2- 172.21.15.67:0/4162806979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d4010c700 0x7f3d4010cb10 secure :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009be0 tx=0x7f3d3802a710 comp rx=0 tx=0).stop 2024-07-31T15:02:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.913+0000 7f3d482e4640 1 --2- 172.21.15.67:0/4162806979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d4010a080 0x7f3d4010a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.913+0000 7f3d482e4640 1 -- 172.21.15.67:0/4162806979 >> 172.21.15.67:0/4162806979 conn(0x7f3d40070d20 msgr2=0x7f3d40073880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.914+0000 7f3d482e4640 1 -- 172.21.15.67:0/4162806979 shutdown_connections 2024-07-31T15:02:41.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.914+0000 7f3d482e4640 1 -- 172.21.15.67:0/4162806979 wait complete. 2024-07-31T15:02:41.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.915+0000 7f3d482e4640 1 Processor -- start 2024-07-31T15:02:41.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.915+0000 7f3d482e4640 1 -- start start 2024-07-31T15:02:41.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.915+0000 7f3d482e4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d4010a080 0x7f3d4011b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:41.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.915+0000 7f3d482e4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d4010d050 0x7f3d4011bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:41.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.915+0000 7f3d482e4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d401262b0 0x7f3d401266c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:41.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.915+0000 7f3d482e4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401197e0 con 0x7f3d4010a080 2024-07-31T15:02:41.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.915+0000 7f3d482e4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3d40119660 con 0x7f3d401262b0 2024-07-31T15:02:41.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.915+0000 7f3d482e4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3d40119960 con 0x7f3d4010d050 2024-07-31T15:02:41.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.916+0000 7f3d4587f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d4010d050 0x7f3d4011bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:41.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.916+0000 7f3d4587f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d4010d050 0x7f3d4011bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:51808/0 (socket says 172.21.15.67:51808) 2024-07-31T15:02:41.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.916+0000 7f3d4587f640 1 -- 172.21.15.67:0/2821343978 learned_addr learned my addr 172.21.15.67:0/2821343978 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:41.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.919+0000 7f3d46881640 1 --2- 172.21.15.67:0/2821343978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d401262b0 0x7f3d401266c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:41.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.921+0000 7f3d4587f640 1 -- 172.21.15.67:0/2821343978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d401262b0 msgr2=0x7f3d401266c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:41.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.921+0000 7f3d4587f640 1 --2- 172.21.15.67:0/2821343978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d401262b0 0x7f3d401266c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:41.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.921+0000 7f3d4587f640 1 -- 172.21.15.67:0/2821343978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d4010a080 msgr2=0x7f3d4011b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:41.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.921+0000 7f3d4587f640 1 --2- 172.21.15.67:0/2821343978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d4010a080 0x7f3d4011b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:41.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.921+0000 7f3d4587f640 1 -- 172.21.15.67:0/2821343978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d401211a0 con 0x7f3d4010d050 2024-07-31T15:02:41.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.926+0000 7f3d4587f640 1 --2- 172.21.15.67:0/2821343978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d4010d050 0x7f3d4011bf20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3d38034040 tx=0x7f3d38030a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:41.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.927+0000 7f3d377fe640 1 -- 172.21.15.67:0/2821343978 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38041070 con 0x7f3d4010d050 2024-07-31T15:02:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.927+0000 7f3d377fe640 1 -- 172.21.15.67:0/2821343978 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d3803c440 con 0x7f3d4010d050 2024-07-31T15:02:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.928+0000 7f3d377fe640 1 -- 172.21.15.67:0/2821343978 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38036040 con 0x7f3d4010d050 2024-07-31T15:02:41.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.929+0000 7f3d482e4640 1 -- 172.21.15.67:0/2821343978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d40121430 con 0x7f3d4010d050 2024-07-31T15:02:41.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.929+0000 7f3d482e4640 1 -- 172.21.15.67:0/2821343978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d40121890 con 0x7f3d4010d050 2024-07-31T15:02:41.931 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/sched/build_utility.o 2024-07-31T15:02:41.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.934+0000 7f3d4e835740 1 Processor -- start 2024-07-31T15:02:41.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.937+0000 7f3d4e835740 1 -- start start 2024-07-31T15:02:41.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.937+0000 7f3d4e835740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 0x55eb27c6d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:41.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.937+0000 7f3d4e835740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2789f1b0 0x55eb27c6da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:41.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.937+0000 7f3d4e835740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb278a7390 0x55eb27c6dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:41.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.937+0000 7f3d4e835740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55eb27be3b60 con 0x55eb2789f1b0 2024-07-31T15:02:41.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.937+0000 7f3d4e835740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55eb27bd3060 con 0x55eb278a7390 2024-07-31T15:02:41.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.937+0000 7f3d4e835740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55eb27bd70a0 con 0x55eb278c8070 2024-07-31T15:02:41.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.938+0000 7f3d377fe640 1 -- 172.21.15.67:0/2821343978 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3d38044070 con 0x7f3d4010d050 2024-07-31T15:02:41.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.941+0000 7f3d46080640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2789f1b0 0x55eb27c6da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:41.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.941+0000 7f3d46080640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2789f1b0 0x55eb27c6da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42228/0 (socket says 172.21.15.67:42228) 2024-07-31T15:02:41.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.941+0000 7f3d46080640 1 -- 172.21.15.67:0/1367666056 learned_addr learned my addr 172.21.15.67:0/1367666056 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:41.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.941+0000 7f3d46881640 1 --2- 172.21.15.67:0/1367666056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 0x55eb27c6d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:41.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.946+0000 7f3d377fe640 1 --2- 172.21.15.67:0/2821343978 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3d2c0b3f50 0x7f3d2c0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:41.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.949+0000 7f3d4587f640 1 --2- 172.21.15.67:0/1367666056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb278a7390 0x55eb27c6dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:41.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.950+0000 7f3d377fe640 1 -- 172.21.15.67:0/2821343978 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9407+0+0 (secure 0 0 0) 0x7f3d38104600 con 0x7f3d4010d050 2024-07-31T15:02:41.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.950+0000 7f3d46080640 1 -- 172.21.15.67:0/1367666056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 msgr2=0x55eb27c6d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:41.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.950+0000 7f3d46080640 1 --2- 172.21.15.67:0/1367666056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 0x55eb27c6d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:41.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.950+0000 7f3d46080640 1 -- 172.21.15.67:0/1367666056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb278a7390 msgr2=0x55eb27c6dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:41.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.950+0000 7f3d46080640 1 --2- 172.21.15.67:0/1367666056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb278a7390 0x55eb27c6dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:41.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.950+0000 7f3d46080640 1 -- 172.21.15.67:0/1367666056 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb27b79b40 con 0x55eb2789f1b0 2024-07-31T15:02:41.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.950+0000 7f3d46080640 1 --2- 172.21.15.67:0/1367666056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2789f1b0 0x55eb27c6da60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00a9c0 tx=0x7f3d3c00ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7409b52e851d0b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:41.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.953+0000 7f3d46881640 1 --2- 172.21.15.67:0/1367666056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 0x55eb27c6d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:02:41.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.954+0000 7f3d46881640 1 --2- 172.21.15.67:0/2821343978 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3d2c0b3f50 0x7f3d2c0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:41.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.955+0000 7f3d46881640 1 --2- 172.21.15.67:0/2821343978 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3d2c0b3f50 0x7f3d2c0b6460 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3d30005fd0 tx=0x7f3d300046c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:41.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.957+0000 7f3d34d99640 1 -- 172.21.15.67:0/1367666056 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c00cde0 con 0x55eb2789f1b0 2024-07-31T15:02:41.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.957+0000 7f3d34d99640 1 -- 172.21.15.67:0/1367666056 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d3c00f900 con 0x55eb2789f1b0 2024-07-31T15:02:41.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.957+0000 7f3d34d99640 1 -- 172.21.15.67:0/1367666056 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c00e990 con 0x55eb2789f1b0 2024-07-31T15:02:41.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.958+0000 7f3d4e835740 1 -- 172.21.15.67:0/1367666056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2789f1b0 msgr2=0x55eb27c6da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:41.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.958+0000 7f3d4e835740 1 --2- 172.21.15.67:0/1367666056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2789f1b0 0x55eb27c6da60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00a9c0 tx=0x7f3d3c00ae80 comp rx=0 tx=0).stop 2024-07-31T15:02:41.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.962+0000 7f3d4e835740 1 -- 172.21.15.67:0/1367666056 shutdown_connections 2024-07-31T15:02:41.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.962+0000 7f3d4e835740 1 --2- 172.21.15.67:0/1367666056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb278a7390 0x55eb27c6dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:41.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.962+0000 7f3d4e835740 1 --2- 172.21.15.67:0/1367666056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2789f1b0 0x55eb27c6da60 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:41.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.962+0000 7f3d4e835740 1 --2- 172.21.15.67:0/1367666056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 0x55eb27c6d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:41.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.962+0000 7f3d4e835740 1 -- 172.21.15.67:0/1367666056 >> 172.21.15.67:0/1367666056 conn(0x55eb278bc460 msgr2=0x55eb27c6b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:41.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.970+0000 7f3d4e835740 1 -- 172.21.15.67:0/1367666056 shutdown_connections 2024-07-31T15:02:41.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.971+0000 7f3d4e835740 1 -- 172.21.15.67:0/1367666056 wait complete. 2024-07-31T15:02:41.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.972+0000 7f3d4e835740 1 -- start start 2024-07-31T15:02:41.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.972+0000 7f3d4e835740 1 Processor -- start 2024-07-31T15:02:41.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.979+0000 7f3d4e835740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2789f1b0 0x55eb27c791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:41.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.979+0000 7f3d4e835740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278a7390 0x55eb27c797a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:41.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.979+0000 7f3d4e835740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb278c8070 0x55eb27c7efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:41.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.979+0000 7f3d4e835740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55eb27a36700 con 0x55eb2789f1b0 2024-07-31T15:02:41.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.979+0000 7f3d4e835740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55eb27b75990 con 0x55eb278c8070 2024-07-31T15:02:41.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.979+0000 7f3d4e835740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55eb27c7bd00 con 0x55eb278a7390 2024-07-31T15:02:41.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.979+0000 7f3d4e835740 10 client.0 _ll_register_callbacks cb 0x55eb27b904a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:41.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.980+0000 7f3d46080640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2789f1b0 0x55eb27c791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:41.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.980+0000 7f3d46080640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2789f1b0 0x55eb27c791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42234/0 (socket says 172.21.15.67:42234) 2024-07-31T15:02:41.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.980+0000 7f3d46080640 1 -- 172.21.15.67:0/259982416 learned_addr learned my addr 172.21.15.67:0/259982416 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:41.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.980+0000 7f3d46881640 1 --2- 172.21.15.67:0/259982416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb278c8070 0x55eb27c7efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:41.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.981+0000 7f3d46080640 1 -- 172.21.15.67:0/259982416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278a7390 msgr2=0x55eb27c797a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:41.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.981+0000 7f3d46080640 1 --2- 172.21.15.67:0/259982416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278a7390 0x55eb27c797a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:41.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.981+0000 7f3d46080640 1 -- 172.21.15.67:0/259982416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb278c8070 msgr2=0x55eb27c7efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:41.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.981+0000 7f3d46080640 1 --2- 172.21.15.67:0/259982416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb278c8070 0x55eb27c7efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:41.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.981+0000 7f3d46080640 1 -- 172.21.15.67:0/259982416 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb27933650 con 0x55eb2789f1b0 2024-07-31T15:02:41.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.982+0000 7f3d46080640 1 --2- 172.21.15.67:0/259982416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2789f1b0 0x55eb27c791c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c004700 tx=0x7f3d3c00f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f87a50a66ac734e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:41.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.985+0000 7f3d29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:02:41.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.985+0000 7f3d29ffb640 1 -- 172.21.15.67:0/259982416 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c00e6d0 con 0x55eb2789f1b0 2024-07-31T15:02:41.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.985+0000 7f3d4e835740 1 -- 172.21.15.67:0/259982416 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d3c00f900 con 0x55eb2789f1b0 2024-07-31T15:02:41.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.985+0000 7f3d29ffb640 1 -- 172.21.15.67:0/259982416 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9407+0+0 (secure 0 0 0) 0x7f3d3c00e870 con 0x55eb2789f1b0 2024-07-31T15:02:41.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.985+0000 7f3d29ffb640 1 -- 172.21.15.67:0/259982416 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f3d3c00ea70 con 0x55eb2789f1b0 2024-07-31T15:02:41.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.985+0000 7f3d4e835740 10 client.17991 fetch_fsmap learned FSMap version 60 2024-07-31T15:02:41.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.985+0000 7f3d4e835740 1 -- 172.21.15.67:0/259982416 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55eb27c87d60 con 0x55eb2789f1b0 2024-07-31T15:02:41.991 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/vsyscall/vsyscall_emu_64.o 2024-07-31T15:02:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.992+0000 7f3d29ffb640 1 -- 172.21.15.67:0/259982416 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d3c00f900 con 0x55eb2789f1b0 2024-07-31T15:02:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.992+0000 7f3d29ffb640 1 -- 172.21.15.67:0/259982416 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 60) ==== 8386+0+0 (secure 0 0 0) 0x7f3d3c022cb0 con 0x55eb2789f1b0 2024-07-31T15:02:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.992+0000 7f3d4e835740 10 client.17991 fetch_fsmap finished waiting for FSMap version 60 2024-07-31T15:02:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.992+0000 7f3d4e835740 10 client.17991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:02:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.993+0000 7f3d4e835740 20 client.17991 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.993+0000 7f3d4e835740 1 --2- 172.21.15.67:0/259982416 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55eb278869f0 0x55eb27c8b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.993+0000 7f3d4e835740 4 client.17991 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:02:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.993+0000 7f3d4e835740 1 -- 172.21.15.67:0/259982416 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb27be5ab0 con 0x55eb278869f0 2024-07-31T15:02:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:41.998+0000 7f3d4587f640 1 --2- 172.21.15.67:0/259982416 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55eb278869f0 0x55eb27c8b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:42.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.013+0000 7f3d4587f640 1 --2- 172.21.15.67:0/259982416 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55eb278869f0 0x55eb27c8b600 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e943145a40d1b16b server_cookie=c6a9d2bb53eff752 in_seq=0 out_seq=0 2024-07-31T15:02:42.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.016+0000 7f3d29ffb640 10 client.17991 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:42.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.016+0000 7f3d29ffb640 1 -- 172.21.15.67:0/259982416 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb27be5ab0 con 0x55eb278869f0 2024-07-31T15:02:42.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.016+0000 7f3d29ffb640 10 client.17991 handle_command_reply: tid=0 2024-07-31T15:02:42.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.016+0000 7f3d4e835740 1 client.17991 shutdown 2024-07-31T15:02:42.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.020+0000 7f3d4e835740 1 -- 172.21.15.67:0/259982416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2789f1b0 msgr2=0x55eb27c791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:42.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.020+0000 7f3d4e835740 1 --2- 172.21.15.67:0/259982416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2789f1b0 0x55eb27c791c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c004700 tx=0x7f3d3c00f680 comp rx=0 tx=0).stop 2024-07-31T15:02:42.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.025+0000 7f3d4e835740 1 -- 172.21.15.67:0/259982416 shutdown_connections 2024-07-31T15:02:42.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.025+0000 7f3d4e835740 1 --2- 172.21.15.67:0/259982416 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55eb278869f0 0x55eb27c8b600 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.025+0000 7f3d4e835740 1 --2- 172.21.15.67:0/259982416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb278c8070 0x55eb27c7efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.025+0000 7f3d4e835740 1 --2- 172.21.15.67:0/259982416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278a7390 0x55eb27c797a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.025+0000 7f3d4e835740 1 --2- 172.21.15.67:0/259982416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2789f1b0 0x55eb27c791c0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.025+0000 7f3d4e835740 1 -- 172.21.15.67:0/259982416 >> 172.21.15.67:0/259982416 conn(0x55eb278bc460 msgr2=0x55eb27c69bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:42.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.025+0000 7f3d29ffb640 0 client.17991 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:42.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.029+0000 7f3d4e835740 1 -- 172.21.15.67:0/259982416 shutdown_connections 2024-07-31T15:02:42.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.031+0000 7f3d4e835740 1 -- 172.21.15.67:0/259982416 wait complete. 2024-07-31T15:02:42.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.031+0000 7f3d4e835740 20 client.17991 trim_cache size 0 max 16384 2024-07-31T15:02:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.069+0000 7f3d4e835740 1 Processor -- start 2024-07-31T15:02:42.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.074+0000 7f3d4e835740 1 -- start start 2024-07-31T15:02:42.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.074+0000 7f3d4e835740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb2789f1b0 0x55eb27c7ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.074+0000 7f3d4e835740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb278a7390 0x55eb27c7f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.074+0000 7f3d4e835740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 0x55eb27c72630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.074+0000 7f3d46080640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb2789f1b0 0x55eb27c7ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.074+0000 7f3d4e835740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401c85c0 con 0x55eb278a7390 2024-07-31T15:02:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.074+0000 7f3d4e835740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401ca1c0 con 0x55eb2789f1b0 2024-07-31T15:02:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.074+0000 7f3d4e835740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401c8c90 con 0x55eb278c8070 2024-07-31T15:02:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.074+0000 7f3d46080640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb2789f1b0 0x55eb27c7ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:34736/0 (socket says 172.21.15.67:34736) 2024-07-31T15:02:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.074+0000 7f3d46080640 1 -- 172.21.15.67:0/1493601763 learned_addr learned my addr 172.21.15.67:0/1493601763 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.075+0000 7f3d4587f640 1 --2- 172.21.15.67:0/1493601763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb278a7390 0x55eb27c7f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.075+0000 7f3d46881640 1 --2- 172.21.15.67:0/1493601763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 0x55eb27c72630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.075+0000 7f3d46080640 1 -- 172.21.15.67:0/1493601763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 msgr2=0x55eb27c72630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.075+0000 7f3d46080640 1 --2- 172.21.15.67:0/1493601763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 0x55eb27c72630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.075+0000 7f3d46080640 1 -- 172.21.15.67:0/1493601763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb278a7390 msgr2=0x55eb27c7f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:42.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.075+0000 7f3d46080640 1 --2- 172.21.15.67:0/1493601763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb278a7390 0x55eb27c7f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.075+0000 7f3d46080640 1 -- 172.21.15.67:0/1493601763 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb27c74850 con 0x55eb2789f1b0 2024-07-31T15:02:42.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.078+0000 7f3d46080640 1 --2- 172.21.15.67:0/1493601763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb2789f1b0 0x55eb27c7ef30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00a8e0 tx=0x7f3d3c00ae80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e85938a4d1ca4d84 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:42.084 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/calibrate.o 2024-07-31T15:02:42.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.084+0000 7f3d2bfff640 1 -- 172.21.15.67:0/1493601763 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c005dc0 con 0x55eb2789f1b0 2024-07-31T15:02:42.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.084+0000 7f3d2bfff640 1 -- 172.21.15.67:0/1493601763 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d3c027070 con 0x55eb2789f1b0 2024-07-31T15:02:42.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.084+0000 7f3d2bfff640 1 -- 172.21.15.67:0/1493601763 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c02bae0 con 0x55eb2789f1b0 2024-07-31T15:02:42.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.088+0000 7f3d4e835740 1 -- 172.21.15.67:0/1493601763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb2789f1b0 msgr2=0x55eb27c7ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:42.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.088+0000 7f3d4e835740 1 --2- 172.21.15.67:0/1493601763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb2789f1b0 0x55eb27c7ef30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00a8e0 tx=0x7f3d3c00ae80 comp rx=0 tx=0).stop 2024-07-31T15:02:42.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.089+0000 7f3d4e835740 1 -- 172.21.15.67:0/1493601763 shutdown_connections 2024-07-31T15:02:42.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.089+0000 7f3d4e835740 1 --2- 172.21.15.67:0/1493601763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 0x55eb27c72630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.089+0000 7f3d4e835740 1 --2- 172.21.15.67:0/1493601763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb278a7390 0x55eb27c7f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.089+0000 7f3d4e835740 1 --2- 172.21.15.67:0/1493601763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb2789f1b0 0x55eb27c7ef30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.089+0000 7f3d4e835740 1 -- 172.21.15.67:0/1493601763 >> 172.21.15.67:0/1493601763 conn(0x55eb278bc460 msgr2=0x55eb27c77840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:42.099 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.102 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.102 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.102 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.102 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.102 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:42.102 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.097+0000 7f3d4e835740 1 -- 172.21.15.67:0/1493601763 shutdown_connections 2024-07-31T15:02:42.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.097+0000 7f3d4e835740 1 -- 172.21.15.67:0/1493601763 wait complete. 2024-07-31T15:02:42.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.097+0000 7f3d4e835740 1 -- start start 2024-07-31T15:02:42.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.097+0000 7f3d4e835740 1 Processor -- start 2024-07-31T15:02:42.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.098+0000 7f3d4e835740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb2789f1b0 0x55eb27c804e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:42.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.098+0000 7f3d4e835740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb278a7390 0x55eb27c80a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:42.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.098+0000 7f3d4e835740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 0x55eb27cf1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:42.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.098+0000 7f3d4e835740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401c9af0 con 0x55eb278a7390 2024-07-31T15:02:42.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.098+0000 7f3d4e835740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401c9360 con 0x55eb2789f1b0 2024-07-31T15:02:42.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.098+0000 7f3d4e835740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401c65a0 con 0x55eb278c8070 2024-07-31T15:02:42.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.098+0000 7f3d4e835740 10 client.0 _ll_register_callbacks cb 0x7f3d3c00a810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:42.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.099+0000 7f3d4587f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb278a7390 0x55eb27c80a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:42.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.099+0000 7f3d4587f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb278a7390 0x55eb27c80a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42256/0 (socket says 172.21.15.67:42256) 2024-07-31T15:02:42.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.099+0000 7f3d4587f640 1 -- 172.21.15.67:0/4017624254 learned_addr learned my addr 172.21.15.67:0/4017624254 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:42.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.099+0000 7f3d4587f640 1 -- 172.21.15.67:0/4017624254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 msgr2=0x55eb27cf1dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:02:42.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.099+0000 7f3d4587f640 1 --2- 172.21.15.67:0/4017624254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 0x55eb27cf1dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.099+0000 7f3d4587f640 1 -- 172.21.15.67:0/4017624254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb2789f1b0 msgr2=0x55eb27c804e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:02:42.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.099+0000 7f3d4587f640 1 --2- 172.21.15.67:0/4017624254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb2789f1b0 0x55eb27c804e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.099+0000 7f3d4587f640 1 -- 172.21.15.67:0/4017624254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb27c812a0 con 0x55eb278a7390 2024-07-31T15:02:42.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.100+0000 7f3d4587f640 1 --2- 172.21.15.67:0/4017624254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb278a7390 0x55eb27c80a20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3d380cabc0 tx=0x7f3d380cad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3516a81bf093a5d9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:42.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.100+0000 7f3d29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:02:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.100+0000 7f3d29ffb640 1 -- 172.21.15.67:0/4017624254 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d380ca630 con 0x55eb278a7390 2024-07-31T15:02:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.100+0000 7f3d4e835740 1 -- 172.21.15.67:0/4017624254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d3c027070 con 0x55eb278a7390 2024-07-31T15:02:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.101+0000 7f3d29ffb640 1 -- 172.21.15.67:0/4017624254 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9407+0+0 (secure 0 0 0) 0x7f3d380c74b0 con 0x55eb278a7390 2024-07-31T15:02:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.101+0000 7f3d29ffb640 1 -- 172.21.15.67:0/4017624254 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f3d380ca9d0 con 0x55eb278a7390 2024-07-31T15:02:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.101+0000 7f3d4e835740 10 client.17997 fetch_fsmap learned FSMap version 60 2024-07-31T15:02:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.101+0000 7f3d4e835740 1 -- 172.21.15.67:0/4017624254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55eb27cf33b0 con 0x55eb278a7390 2024-07-31T15:02:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.102+0000 7f3d29ffb640 1 -- 172.21.15.67:0/4017624254 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d380cb5c0 con 0x55eb278a7390 2024-07-31T15:02:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.102+0000 7f3d29ffb640 1 -- 172.21.15.67:0/4017624254 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 60) ==== 8386+0+0 (secure 0 0 0) 0x7f3d380c7150 con 0x55eb278a7390 2024-07-31T15:02:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.102+0000 7f3d4e835740 10 client.17997 fetch_fsmap finished waiting for FSMap version 60 2024-07-31T15:02:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.102+0000 7f3d4e835740 10 client.17997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:02:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.102+0000 7f3d4e835740 20 client.17997 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.102+0000 7f3d4e835740 1 --2- 172.21.15.67:0/4017624254 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55eb2785bda0 0x55eb27cf5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.102+0000 7f3d4e835740 4 client.17997 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:02:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.102+0000 7f3d4e835740 1 -- 172.21.15.67:0/4017624254 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3d3c027070 con 0x55eb2785bda0 2024-07-31T15:02:42.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.102+0000 7f3d46080640 1 --2- 172.21.15.67:0/4017624254 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55eb2785bda0 0x55eb27cf5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:42.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.104+0000 7f3d46080640 1 --2- 172.21.15.67:0/4017624254 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55eb2785bda0 0x55eb27cf5ca0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56c993e8bcf65bf9 server_cookie=3ce915dfe192b487 in_seq=0 out_seq=0 2024-07-31T15:02:42.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.110+0000 7f3d29ffb640 10 client.17997 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:42.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.110+0000 7f3d29ffb640 1 -- 172.21.15.67:0/4017624254 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x7f3d3c027070 con 0x55eb2785bda0 2024-07-31T15:02:42.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.110+0000 7f3d29ffb640 10 client.17997 handle_command_reply: tid=0 2024-07-31T15:02:42.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.110+0000 7f3d4e835740 1 client.17997 shutdown 2024-07-31T15:02:42.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.123+0000 7f3d4e835740 1 -- 172.21.15.67:0/4017624254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb278a7390 msgr2=0x55eb27c80a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:42.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.123+0000 7f3d4e835740 1 --2- 172.21.15.67:0/4017624254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb278a7390 0x55eb27c80a20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3d380cabc0 tx=0x7f3d380cad50 comp rx=0 tx=0).stop 2024-07-31T15:02:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.126+0000 7f3d4e835740 1 -- 172.21.15.67:0/4017624254 shutdown_connections 2024-07-31T15:02:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.126+0000 7f3d4e835740 1 --2- 172.21.15.67:0/4017624254 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55eb2785bda0 0x55eb27cf5ca0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.126+0000 7f3d4e835740 1 --2- 172.21.15.67:0/4017624254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb278c8070 0x55eb27cf1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.126+0000 7f3d4e835740 1 --2- 172.21.15.67:0/4017624254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb278a7390 0x55eb27c80a20 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.126+0000 7f3d29ffb640 0 client.17997 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.126+0000 7f3d4e835740 1 --2- 172.21.15.67:0/4017624254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb2789f1b0 0x55eb27c804e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.126+0000 7f3d4e835740 1 -- 172.21.15.67:0/4017624254 >> 172.21.15.67:0/4017624254 conn(0x55eb278bc460 msgr2=0x55eb27c72630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.126+0000 7f3d4e835740 1 -- 172.21.15.67:0/4017624254 shutdown_connections 2024-07-31T15:02:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.126+0000 7f3d4e835740 1 -- 172.21.15.67:0/4017624254 wait complete. 2024-07-31T15:02:42.129 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/ipc_sysctl.o 2024-07-31T15:02:42.130 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:02:42.130 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (7009 inodes in the stack)", 2024-07-31T15:02:42.130 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T15:02:42.130 INFO:teuthology.orchestra.run.smithi067.stdout: "acb195c0-e291-456a-ac4b-1332fba0bea8": { 2024-07-31T15:02:42.130 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T15:02:42.130 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "acb195c0-e291-456a-ac4b-1332fba0bea8", 2024-07-31T15:02:42.130 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T15:02:42.130 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:02:42.131 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:02:42.131 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:02:42.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.127+0000 7f3d4e835740 20 client.17997 trim_cache size 0 max 16384 2024-07-31T15:02:42.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.129+0000 7f3d482e4640 1 -- 172.21.15.67:0/2821343978 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3d2c0b3f50 msgr2=0x7f3d2c0b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:42.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.129+0000 7f3d482e4640 1 --2- 172.21.15.67:0/2821343978 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3d2c0b3f50 0x7f3d2c0b6460 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3d30005fd0 tx=0x7f3d300046c0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.129+0000 7f3d482e4640 1 -- 172.21.15.67:0/2821343978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d4010d050 msgr2=0x7f3d4011bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:42.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.129+0000 7f3d482e4640 1 --2- 172.21.15.67:0/2821343978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d4010d050 0x7f3d4011bf20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3d38034040 tx=0x7f3d38030a90 comp rx=0 tx=0).stop 2024-07-31T15:02:42.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.130+0000 7f3d482e4640 1 -- 172.21.15.67:0/2821343978 shutdown_connections 2024-07-31T15:02:42.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.130+0000 7f3d482e4640 1 --2- 172.21.15.67:0/2821343978 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3d2c0b3f50 0x7f3d2c0b6460 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.130+0000 7f3d482e4640 1 --2- 172.21.15.67:0/2821343978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d401262b0 0x7f3d401266c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.130+0000 7f3d482e4640 1 --2- 172.21.15.67:0/2821343978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d4010d050 0x7f3d4011bf20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.130+0000 7f3d482e4640 1 --2- 172.21.15.67:0/2821343978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d4010a080 0x7f3d4011b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:42.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.130+0000 7f3d482e4640 1 -- 172.21.15.67:0/2821343978 >> 172.21.15.67:0/2821343978 conn(0x7f3d40070d20 msgr2=0x7f3d4010f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:42.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.130+0000 7f3d482e4640 1 -- 172.21.15.67:0/2821343978 shutdown_connections 2024-07-31T15:02:42.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:42.130+0000 7f3d482e4640 1 -- 172.21.15.67:0/2821343978 wait complete. 2024-07-31T15:02:42.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:42.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.157 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/iomap/trace.o 2024-07-31T15:02:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:42.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:42.286 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/nfs_common/built-in.a 2024-07-31T15:02:42.385 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/mqueue.o 2024-07-31T15:02:42.541 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/init_task.o 2024-07-31T15:02:42.693 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/entry/vsyscall/built-in.a 2024-07-31T15:02:42.782 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/entry.o 2024-07-31T15:02:42.882 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/entry_64.o 2024-07-31T15:02:42.931 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/zhaoxin/core.o 2024-07-31T15:02:43.060 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/syscall_64.o 2024-07-31T15:02:43.104 INFO:tasks.cephfs.filesystem:scrub status for tag:acb195c0-e291-456a-ac4b-1332fba0bea8 - {'path': '/', 'tag': 'acb195c0-e291-456a-ac4b-1332fba0bea8', 'options': 'recursive,force'} 2024-07-31T15:02:43.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:43.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:43.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: pgmap v764: 129 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 119 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 237 KiB/s wr, 937 op/s 2024-07-31T15:02:43.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:43.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:43.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:43.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:43.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:43.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:43.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:43.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:43.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:43.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:43.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:43.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:43.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:43.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:43.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: pgmap v764: 129 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 119 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 237 KiB/s wr, 937 op/s 2024-07-31T15:02:43.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:43.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:43.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:43.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:43.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:43.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:43.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:43.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:43.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:43.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:43.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:43.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:43.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: pgmap v764: 129 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 119 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 237 KiB/s wr, 937 op/s 2024-07-31T15:02:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:43.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:43.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:43.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:43.375 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/version.o 2024-07-31T15:02:43.414 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/iomap/iter.o 2024-07-31T15:02:43.740 INFO:tasks.workunit.client.0.smithi067.stdout: AR init/built-in.a 2024-07-31T15:02:43.787 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/knc.o 2024-07-31T15:02:43.808 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/folio-compat.o 2024-07-31T15:02:43.811 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/namespace.o 2024-07-31T15:02:43.889 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/common.o 2024-07-31T15:02:43.960 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/events/zhaoxin/built-in.a 2024-07-31T15:02:44.066 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/readahead.o 2024-07-31T15:02:44.171 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/mq_sysctl.o 2024-07-31T15:02:44.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:44.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:44.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:44.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:44.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:44.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:44.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:44.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:44.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:44.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:44.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:44.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:44.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:44.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:44.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:44.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:44.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:44.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:44.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:44.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:44.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:44.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:44.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:44.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:44.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:44.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:44.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:44.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:44.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:44.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:44.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:44.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:44.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:44.444 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/swap.o 2024-07-31T15:02:44.455 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/iomap/buffered-io.o 2024-07-31T15:02:44.610 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/lbr.o 2024-07-31T15:02:44.699 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/thunk_64.o 2024-07-31T15:02:44.754 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/p4.o 2024-07-31T15:02:44.873 INFO:tasks.workunit.client.0.smithi067.stdout: AR ipc/built-in.a 2024-07-31T15:02:44.951 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/entry_64_compat.o 2024-07-31T15:02:44.961 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/p6.o 2024-07-31T15:02:45.088 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/syscall_32.o 2024-07-31T15:02:45.199 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/truncate.o 2024-07-31T15:02:45.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: pgmap v765: 129 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 120 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 416 KiB/s wr, 1.21k op/s 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:45.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: pgmap v765: 129 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 120 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 416 KiB/s wr, 1.21k op/s 2024-07-31T15:02:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:45.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: pgmap v765: 129 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 120 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 416 KiB/s wr, 1.21k op/s 2024-07-31T15:02:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:45.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:45.848 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/pt.o 2024-07-31T15:02:45.855 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/entry/built-in.a 2024-07-31T15:02:45.869 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/quota/dquot.o 2024-07-31T15:02:46.016 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/vmscan.o 2024-07-31T15:02:46.059 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/uncore.o 2024-07-31T15:02:46.165 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/sched/built-in.a 2024-07-31T15:02:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:46.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:46.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.271 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/mutex.o 2024-07-31T15:02:46.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:46.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:46.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.356 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/qos.o 2024-07-31T15:02:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:46.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:46.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:46.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:46.403 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/task_mmu.o 2024-07-31T15:02:46.416 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/iomap/direct-io.o 2024-07-31T15:02:47.219 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/inode.o 2024-07-31T15:02:47.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:47.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:47.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: pgmap v766: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 487 KiB/s wr, 1.74k op/s 2024-07-31T15:02:47.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:47.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:47.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:47.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:47.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:47.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:47.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:47.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: pgmap v766: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 487 KiB/s wr, 1.74k op/s 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:47.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:47.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: pgmap v766: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 487 KiB/s wr, 1.74k op/s 2024-07-31T15:02:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:47.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:47.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:47.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:47.430 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/main.o 2024-07-31T15:02:47.590 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/iomap/fiemap.o 2024-07-31T15:02:47.602 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/semaphore.o 2024-07-31T15:02:47.708 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/uncore_nhmex.o 2024-07-31T15:02:47.808 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/quota/quota_v2.o 2024-07-31T15:02:47.817 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/quota/quota_tree.o 2024-07-31T15:02:48.180 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/root.o 2024-07-31T15:02:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:48.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:48.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:48.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:48.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:48.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:48.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:48.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:48.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:48.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:48.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:48.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:48.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:48.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:48.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:48.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:48.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:48.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.312 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/iomap/seek.o 2024-07-31T15:02:48.316 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/rwsem.o 2024-07-31T15:02:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:48.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:48.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:48.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:48.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:48.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:48.476 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/iomap/swapfile.o 2024-07-31T15:02:48.580 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/console.o 2024-07-31T15:02:48.750 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/quota/quota.o 2024-07-31T15:02:48.784 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/uncore_snb.o 2024-07-31T15:02:48.789 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/base.o 2024-07-31T15:02:48.967 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/generic.o 2024-07-31T15:02:48.969 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/shmem.o 2024-07-31T15:02:49.116 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/percpu-rwsem.o 2024-07-31T15:02:49.227 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/iomap/built-in.a 2024-07-31T15:02:49.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:49.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:49.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:49.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:49.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:49.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:49.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:49.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: pgmap v767: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 514 KiB/s wr, 2.27k op/s 2024-07-31T15:02:49.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:49.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:02:49.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: pgmap v767: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 514 KiB/s wr, 2.27k op/s 2024-07-31T15:02:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:02:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.332 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/process.o 2024-07-31T15:02:49.339 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/array.o 2024-07-31T15:02:49.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:49.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:49.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:49.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:49.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:49.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:49.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:49.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: pgmap v767: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 514 KiB/s wr, 2.27k op/s 2024-07-31T15:02:49.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:49.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:02:49.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:49.671 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/uncore_snbep.o 2024-07-31T15:02:49.738 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/spinlock.o 2024-07-31T15:02:49.813 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/osq_lock.o 2024-07-31T15:02:49.882 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/quota/kqid.o 2024-07-31T15:02:50.183 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/quota/netlink.o 2024-07-31T15:02:50.201 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/qspinlock.o 2024-07-31T15:02:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.215+0000 7f8fc9147640 1 -- 172.21.15.67:0/3135898163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc411c5d0 msgr2=0x7f8fc411e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.215+0000 7f8fc9147640 1 --2- 172.21.15.67:0/3135898163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc411c5d0 0x7f8fc411e9f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009f00 tx=0x7f8fbc02ab60 comp rx=0 tx=0).stop 2024-07-31T15:02:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.216+0000 7f8fc9147640 1 -- 172.21.15.67:0/3135898163 shutdown_connections 2024-07-31T15:02:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.216+0000 7f8fc9147640 1 --2- 172.21.15.67:0/3135898163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc411c5d0 0x7f8fc411e9f0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.216+0000 7f8fc9147640 1 --2- 172.21.15.67:0/3135898163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fc410a330 0x7f8fc410a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.216+0000 7f8fc9147640 1 --2- 172.21.15.67:0/3135898163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc41116d0 0x7f8fc4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.216+0000 7f8fc9147640 1 -- 172.21.15.67:0/3135898163 >> 172.21.15.67:0/3135898163 conn(0x7f8fc406e350 msgr2=0x7f8fc406e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.216+0000 7f8fc9147640 1 -- 172.21.15.67:0/3135898163 shutdown_connections 2024-07-31T15:02:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.216+0000 7f8fc9147640 1 -- 172.21.15.67:0/3135898163 wait complete. 2024-07-31T15:02:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.217+0000 7f8fc9147640 1 Processor -- start 2024-07-31T15:02:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.217+0000 7f8fc9147640 1 -- start start 2024-07-31T15:02:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.218+0000 7f8fc9147640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fc410a330 0x7f8fc411bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.218+0000 7f8fc9147640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc41116d0 0x7f8fc4115020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.218+0000 7f8fc9147640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc411c5d0 0x7f8fc4115560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.218+0000 7f8fc9147640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41221c0 con 0x7f8fc41116d0 2024-07-31T15:02:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.218+0000 7f8fc9147640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4122040 con 0x7f8fc411c5d0 2024-07-31T15:02:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.218+0000 7f8fc9147640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4122340 con 0x7f8fc410a330 2024-07-31T15:02:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.218+0000 7f8fc259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc41116d0 0x7f8fc4115020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.218+0000 7f8fc259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc41116d0 0x7f8fc4115020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42278/0 (socket says 172.21.15.67:42278) 2024-07-31T15:02:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.218+0000 7f8fc259c640 1 -- 172.21.15.67:0/2820112434 learned_addr learned my addr 172.21.15.67:0/2820112434 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.218+0000 7f8fc2d9d640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fc410a330 0x7f8fc411bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.219+0000 7f8fc359e640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc411c5d0 0x7f8fc4115560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.219+0000 7f8fc2d9d640 1 -- 172.21.15.67:0/2820112434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc411c5d0 msgr2=0x7f8fc4115560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.219+0000 7f8fc2d9d640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc411c5d0 0x7f8fc4115560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.219+0000 7f8fc2d9d640 1 -- 172.21.15.67:0/2820112434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc41116d0 msgr2=0x7f8fc4115020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.219+0000 7f8fc2d9d640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc41116d0 0x7f8fc4115020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.219+0000 7f8fc2d9d640 1 -- 172.21.15.67:0/2820112434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc4115ce0 con 0x7f8fc410a330 2024-07-31T15:02:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.219+0000 7f8fc259c640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc41116d0 0x7f8fc4115020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:02:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.219+0000 7f8fc2d9d640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fc410a330 0x7f8fc411bd60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009fd0 tx=0x7f8fb800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.221+0000 7f8fabfff640 1 -- 172.21.15.67:0/2820112434 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8015020 con 0x7f8fc410a330 2024-07-31T15:02:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.221+0000 7f8fc9147640 1 -- 172.21.15.67:0/2820112434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc411a9c0 con 0x7f8fc410a330 2024-07-31T15:02:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.221+0000 7f8fc9147640 1 -- 172.21.15.67:0/2820112434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc411af30 con 0x7f8fc410a330 2024-07-31T15:02:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.221+0000 7f8fabfff640 1 -- 172.21.15.67:0/2820112434 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb8011ce0 con 0x7f8fc410a330 2024-07-31T15:02:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.222+0000 7f8fabfff640 1 -- 172.21.15.67:0/2820112434 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8010e00 con 0x7f8fc410a330 2024-07-31T15:02:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.228+0000 7f8fabfff640 1 -- 172.21.15.67:0/2820112434 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8fb8018020 con 0x7f8fc410a330 2024-07-31T15:02:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.228+0000 7f8fd0044740 1 Processor -- start 2024-07-31T15:02:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.234+0000 7f8fd0044740 1 -- start start 2024-07-31T15:02:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.234+0000 7f8fd0044740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9cb1b0 0x564b3ad6dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.234+0000 7f8fd0044740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9a22f0 0x564b3ad6e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.234+0000 7f8fd0044740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9aa4d0 0x564b3ad6e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.234+0000 7f8fd0044740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564b3ace33b0 con 0x564b3a9aa4d0 2024-07-31T15:02:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.234+0000 7f8fd0044740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564b3acd28b0 con 0x564b3a9a22f0 2024-07-31T15:02:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.234+0000 7f8fd0044740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b3acd68f0 con 0x564b3a9cb1b0 2024-07-31T15:02:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.234+0000 7f8fc359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9cb1b0 0x564b3ad6dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.234+0000 7f8fc359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9cb1b0 0x564b3ad6dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:51866/0 (socket says 172.21.15.67:51866) 2024-07-31T15:02:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.234+0000 7f8fc359e640 1 -- 172.21.15.67:0/1910900485 learned_addr learned my addr 172.21.15.67:0/1910900485 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.234+0000 7f8fc2d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9a22f0 0x564b3ad6e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.235+0000 7f8fc259c640 1 --2- 172.21.15.67:0/1910900485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9aa4d0 0x564b3ad6e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.243+0000 7f8fc359e640 1 -- 172.21.15.67:0/1910900485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9a22f0 msgr2=0x564b3ad6e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.243+0000 7f8fc359e640 1 --2- 172.21.15.67:0/1910900485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9a22f0 0x564b3ad6e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.243+0000 7f8fc359e640 1 -- 172.21.15.67:0/1910900485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9aa4d0 msgr2=0x564b3ad6e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.243+0000 7f8fc359e640 1 --2- 172.21.15.67:0/1910900485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9aa4d0 0x564b3ad6e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.243+0000 7f8fc359e640 1 -- 172.21.15.67:0/1910900485 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b3ac799a0 con 0x564b3a9cb1b0 2024-07-31T15:02:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.244+0000 7f8fc359e640 1 --2- 172.21.15.67:0/1910900485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9cb1b0 0x564b3ad6dea0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc035000 tx=0x7f8fbc02b510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57fb95812c3bdcc8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.244+0000 7f8fa97fa640 1 -- 172.21.15.67:0/1910900485 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc034420 con 0x564b3a9cb1b0 2024-07-31T15:02:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.244+0000 7f8fabfff640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8fac0b3f50 0x7f8fac0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.244+0000 7f8fa97fa640 1 -- 172.21.15.67:0/1910900485 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fbc02b640 con 0x564b3a9cb1b0 2024-07-31T15:02:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.244+0000 7f8fa97fa640 1 -- 172.21.15.67:0/1910900485 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc02c6f0 con 0x564b3a9cb1b0 2024-07-31T15:02:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.244+0000 7f8fabfff640 1 -- 172.21.15.67:0/2820112434 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9407+0+0 (secure 0 0 0) 0x7f8fb80e48e0 con 0x7f8fc410a330 2024-07-31T15:02:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.245+0000 7f8fc2d9d640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8fac0b3f50 0x7f8fac0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.245+0000 7f8fd0044740 1 -- 172.21.15.67:0/1910900485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9cb1b0 msgr2=0x564b3ad6dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.245+0000 7f8fd0044740 1 --2- 172.21.15.67:0/1910900485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9cb1b0 0x564b3ad6dea0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc035000 tx=0x7f8fbc02b510 comp rx=0 tx=0).stop 2024-07-31T15:02:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.245+0000 7f8fd0044740 1 -- 172.21.15.67:0/1910900485 shutdown_connections 2024-07-31T15:02:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.245+0000 7f8fd0044740 1 --2- 172.21.15.67:0/1910900485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9aa4d0 0x564b3ad6e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.245+0000 7f8fd0044740 1 --2- 172.21.15.67:0/1910900485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9a22f0 0x564b3ad6e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.245+0000 7f8fd0044740 1 --2- 172.21.15.67:0/1910900485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9cb1b0 0x564b3ad6dea0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.245+0000 7f8fd0044740 1 -- 172.21.15.67:0/1910900485 >> 172.21.15.67:0/1910900485 conn(0x564b3a9bf300 msgr2=0x564b3ad6bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.246+0000 7f8fc2d9d640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8fac0b3f50 0x7f8fac0b6460 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8010610 tx=0x7f8fb8010000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.246+0000 7f8fd0044740 1 -- 172.21.15.67:0/1910900485 shutdown_connections 2024-07-31T15:02:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.246+0000 7f8fd0044740 1 -- 172.21.15.67:0/1910900485 wait complete. 2024-07-31T15:02:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.247+0000 7f8fd0044740 1 -- start start 2024-07-31T15:02:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.247+0000 7f8fd0044740 1 Processor -- start 2024-07-31T15:02:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.247+0000 7f8fd0044740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9a22f0 0x564b3ad79a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.247+0000 7f8fd0044740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9aa4d0 0x564b3ad79f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.247+0000 7f8fd0044740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9cb1b0 0x564b3ad85910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.247+0000 7f8fd0044740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564b3ac0cab0 con 0x564b3a9cb1b0 2024-07-31T15:02:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.247+0000 7f8fd0044740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564b3ac5acf0 con 0x564b3a9aa4d0 2024-07-31T15:02:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.247+0000 7f8fd0044740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b3ad7c5c0 con 0x564b3a9a22f0 2024-07-31T15:02:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.248+0000 7f8fd0044740 10 client.0 _ll_register_callbacks cb 0x564b3accc9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.248+0000 7f8fc359e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9cb1b0 0x564b3ad85910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.248+0000 7f8fc259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9aa4d0 0x564b3ad79f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.248+0000 7f8fc259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9aa4d0 0x564b3ad79f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:34772/0 (socket says 172.21.15.67:34772) 2024-07-31T15:02:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.248+0000 7f8fc259c640 1 -- 172.21.15.67:0/300346240 learned_addr learned my addr 172.21.15.67:0/300346240 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.248+0000 7f8fc259c640 1 -- 172.21.15.67:0/300346240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9a22f0 msgr2=0x564b3ad79a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:02:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.248+0000 7f8fc259c640 1 --2- 172.21.15.67:0/300346240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9a22f0 0x564b3ad79a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.248+0000 7f8fc259c640 1 -- 172.21.15.67:0/300346240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9cb1b0 msgr2=0x564b3ad85910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.249+0000 7f8fc259c640 1 --2- 172.21.15.67:0/300346240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9cb1b0 0x564b3ad85910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.249+0000 7f8fc259c640 1 -- 172.21.15.67:0/300346240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b3aa36790 con 0x564b3a9aa4d0 2024-07-31T15:02:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.249+0000 7f8fc359e640 1 --2- 172.21.15.67:0/300346240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9cb1b0 0x564b3ad85910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:02:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.249+0000 7f8fc259c640 1 --2- 172.21.15.67:0/300346240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9aa4d0 0x564b3ad79f50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4011980 tx=0x7f8fb4011e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=835afbba071f440c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:50.252 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/suspend.o 2024-07-31T15:02:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.256+0000 7f8fa6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:02:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.256+0000 7f8fa6ff5640 1 -- 172.21.15.67:0/300346240 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400faf0 con 0x564b3a9aa4d0 2024-07-31T15:02:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.256+0000 7f8fd0044740 1 -- 172.21.15.67:0/300346240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fbc02b640 con 0x564b3a9aa4d0 2024-07-31T15:02:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.256+0000 7f8fa6ff5640 1 -- 172.21.15.67:0/300346240 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9407+0+0 (secure 0 0 0) 0x7f8fb400fc90 con 0x564b3a9aa4d0 2024-07-31T15:02:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.260+0000 7f8fa6ff5640 1 -- 172.21.15.67:0/300346240 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8fb400fe90 con 0x564b3a9aa4d0 2024-07-31T15:02:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.260+0000 7f8fd0044740 10 client.27566 fetch_fsmap learned FSMap version 60 2024-07-31T15:02:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.260+0000 7f8fd0044740 1 -- 172.21.15.67:0/300346240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x564b3aaaf620 con 0x564b3a9aa4d0 2024-07-31T15:02:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.260+0000 7f8fa6ff5640 1 -- 172.21.15.67:0/300346240 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fbc02b640 con 0x564b3a9aa4d0 2024-07-31T15:02:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.260+0000 7f8fa6ff5640 1 -- 172.21.15.67:0/300346240 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 60) ==== 8386+0+0 (secure 0 0 0) 0x7f8fb401d5e0 con 0x564b3a9aa4d0 2024-07-31T15:02:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.260+0000 7f8fd0044740 10 client.27566 fetch_fsmap finished waiting for FSMap version 60 2024-07-31T15:02:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.260+0000 7f8fd0044740 10 client.27566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:02:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.260+0000 7f8fd0044740 20 client.27566 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.260+0000 7f8fd0044740 1 --2- 172.21.15.67:0/300346240 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b3a989d80 0x564b3ad8bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.260+0000 7f8fd0044740 4 client.27566 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:02:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.260+0000 7f8fd0044740 1 -- 172.21.15.67:0/300346240 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b3ace5300 con 0x564b3a989d80 2024-07-31T15:02:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.261+0000 7f8fc359e640 1 --2- 172.21.15.67:0/300346240 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b3a989d80 0x564b3ad8bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.261+0000 7f8fa6ff5640 10 client.27566 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.261+0000 7f8fc359e640 1 --2- 172.21.15.67:0/300346240 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b3a989d80 0x564b3ad8bf70 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3037889c5af6477f server_cookie=2e23ef9eba0211f1 in_seq=0 out_seq=0 2024-07-31T15:02:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.262+0000 7f8fa6ff5640 1 -- 172.21.15.67:0/300346240 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b3ace5300 con 0x564b3a989d80 2024-07-31T15:02:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.262+0000 7f8fa6ff5640 10 client.27566 handle_command_reply: tid=0 2024-07-31T15:02:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.262+0000 7f8fd0044740 1 client.27566 shutdown 2024-07-31T15:02:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.263+0000 7f8fd0044740 1 -- 172.21.15.67:0/300346240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9aa4d0 msgr2=0x564b3ad79f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.263+0000 7f8fd0044740 1 --2- 172.21.15.67:0/300346240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9aa4d0 0x564b3ad79f50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4011980 tx=0x7f8fb4011e40 comp rx=0 tx=0).stop 2024-07-31T15:02:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.263+0000 7f8fd0044740 1 -- 172.21.15.67:0/300346240 shutdown_connections 2024-07-31T15:02:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.263+0000 7f8fd0044740 1 --2- 172.21.15.67:0/300346240 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b3a989d80 0x564b3ad8bf70 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.263+0000 7f8fd0044740 1 --2- 172.21.15.67:0/300346240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9cb1b0 0x564b3ad85910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.263+0000 7f8fd0044740 1 --2- 172.21.15.67:0/300346240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9aa4d0 0x564b3ad79f50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.263+0000 7f8fa6ff5640 0 client.27566 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.263+0000 7f8fd0044740 1 --2- 172.21.15.67:0/300346240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9a22f0 0x564b3ad79a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.263+0000 7f8fd0044740 1 -- 172.21.15.67:0/300346240 >> 172.21.15.67:0/300346240 conn(0x564b3a9bf300 msgr2=0x564b3ad6eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.263+0000 7f8fd0044740 1 -- 172.21.15.67:0/300346240 shutdown_connections 2024-07-31T15:02:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.263+0000 7f8fd0044740 1 -- 172.21.15.67:0/300346240 wait complete. 2024-07-31T15:02:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.263+0000 7f8fd0044740 20 client.27566 trim_cache size 0 max 16384 2024-07-31T15:02:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.296+0000 7f8fd0044740 1 Processor -- start 2024-07-31T15:02:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.296+0000 7f8fd0044740 1 -- start start 2024-07-31T15:02:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.296+0000 7f8fd0044740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9a22f0 0x564b3ad73fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.296+0000 7f8fd0044740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9aa4d0 0x564b3ad6bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.296+0000 7f8fd0044740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9cb1b0 0x564b3ad85700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.296+0000 7f8fd0044740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb000ffa0 con 0x564b3a9cb1b0 2024-07-31T15:02:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.296+0000 7f8fd0044740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0011ba0 con 0x564b3a9aa4d0 2024-07-31T15:02:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.296+0000 7f8fd0044740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0010670 con 0x564b3a9a22f0 2024-07-31T15:02:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.297+0000 7f8fc259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9aa4d0 0x564b3ad6bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.297+0000 7f8fc359e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9cb1b0 0x564b3ad85700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.297+0000 7f8fc259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9aa4d0 0x564b3ad6bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:34782/0 (socket says 172.21.15.67:34782) 2024-07-31T15:02:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.297+0000 7f8fc259c640 1 -- 172.21.15.67:0/4105204581 learned_addr learned my addr 172.21.15.67:0/4105204581 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.297+0000 7f8fc359e640 1 --2- 172.21.15.67:0/4105204581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9a22f0 0x564b3ad73fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.297+0000 7f8fc259c640 1 -- 172.21.15.67:0/4105204581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9a22f0 msgr2=0x564b3ad73fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.297+0000 7f8fc259c640 1 --2- 172.21.15.67:0/4105204581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9a22f0 0x564b3ad73fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.297+0000 7f8fc259c640 1 -- 172.21.15.67:0/4105204581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9cb1b0 msgr2=0x564b3ad85700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.297+0000 7f8fc259c640 1 --2- 172.21.15.67:0/4105204581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9cb1b0 0x564b3ad85700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.297+0000 7f8fc259c640 1 -- 172.21.15.67:0/4105204581 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b3ad70cb0 con 0x564b3a9aa4d0 2024-07-31T15:02:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.297+0000 7f8fc359e640 1 --2- 172.21.15.67:0/4105204581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9cb1b0 0x564b3ad85700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:02:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.298+0000 7f8fc259c640 1 --2- 172.21.15.67:0/4105204581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9aa4d0 0x564b3ad6bcb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4006fd0 tx=0x7f8fb400f6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16a7fae501f20e16 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.298+0000 7f8fa8ff9640 1 -- 172.21.15.67:0/4105204581 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb402f070 con 0x564b3a9aa4d0 2024-07-31T15:02:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.298+0000 7f8fa8ff9640 1 -- 172.21.15.67:0/4105204581 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb401d5e0 con 0x564b3a9aa4d0 2024-07-31T15:02:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.299+0000 7f8fd0044740 1 -- 172.21.15.67:0/4105204581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9aa4d0 msgr2=0x564b3ad6bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.299+0000 7f8fd0044740 1 --2- 172.21.15.67:0/4105204581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9aa4d0 0x564b3ad6bcb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4006fd0 tx=0x7f8fb400f6c0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.304+0000 7f8fd0044740 1 -- 172.21.15.67:0/4105204581 shutdown_connections 2024-07-31T15:02:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.304+0000 7f8fd0044740 1 --2- 172.21.15.67:0/4105204581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9cb1b0 0x564b3ad85700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.304+0000 7f8fd0044740 1 --2- 172.21.15.67:0/4105204581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9aa4d0 0x564b3ad6bcb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.304+0000 7f8fd0044740 1 --2- 172.21.15.67:0/4105204581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9a22f0 0x564b3ad73fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.304+0000 7f8fd0044740 1 -- 172.21.15.67:0/4105204581 >> 172.21.15.67:0/4105204581 conn(0x564b3a9bf300 msgr2=0x564b3ad6b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.308+0000 7f8fd0044740 1 -- 172.21.15.67:0/4105204581 shutdown_connections 2024-07-31T15:02:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.308+0000 7f8fd0044740 1 -- 172.21.15.67:0/4105204581 wait complete. 2024-07-31T15:02:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.308+0000 7f8fd0044740 1 -- start start 2024-07-31T15:02:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.308+0000 7f8fd0044740 1 Processor -- start 2024-07-31T15:02:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.309+0000 7f8fd0044740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9a22f0 0x564b3ad81e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.309+0000 7f8fd0044740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9aa4d0 0x564b3ad7e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.309+0000 7f8fd0044740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9cb1b0 0x564b3ad7ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.309+0000 7f8fd0044740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb00114d0 con 0x564b3a9aa4d0 2024-07-31T15:02:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.309+0000 7f8fd0044740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0010d40 con 0x564b3a9cb1b0 2024-07-31T15:02:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.309+0000 7f8fd0044740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb000df80 con 0x564b3a9a22f0 2024-07-31T15:02:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.309+0000 7f8fd0044740 10 client.0 _ll_register_callbacks cb 0x7f8fb4009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:02:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.310+0000 7f8fc359e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9cb1b0 0x564b3ad7ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.310+0000 7f8fc359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9a22f0 0x564b3ad81e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.310+0000 7f8fc359e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9cb1b0 0x564b3ad7ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:34794/0 (socket says 172.21.15.67:34794) 2024-07-31T15:02:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.310+0000 7f8fc359e640 1 -- 172.21.15.67:0/2176144456 learned_addr learned my addr 172.21.15.67:0/2176144456 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.310+0000 7f8fc259c640 1 --2- 172.21.15.67:0/2176144456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9aa4d0 0x564b3ad7e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.310+0000 7f8fc259c640 1 -- 172.21.15.67:0/2176144456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9a22f0 msgr2=0x564b3ad81e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.310+0000 7f8fc259c640 1 --2- 172.21.15.67:0/2176144456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9a22f0 0x564b3ad81e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.311+0000 7f8fc259c640 1 -- 172.21.15.67:0/2176144456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9cb1b0 msgr2=0x564b3ad7ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:02:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.311+0000 7f8fc259c640 1 --2- 172.21.15.67:0/2176144456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9cb1b0 0x564b3ad7ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.311+0000 7f8fc259c640 1 -- 172.21.15.67:0/2176144456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b3ad7f440 con 0x564b3a9aa4d0 2024-07-31T15:02:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.311+0000 7f8fc259c640 1 --2- 172.21.15.67:0/2176144456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9aa4d0 0x564b3ad7e840 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4011340 tx=0x7f8fb40116b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a29e80f41dec889 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.312+0000 7f8fa6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:02:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.312+0000 7f8fa6ff5640 1 -- 172.21.15.67:0/2176144456 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4038060 con 0x564b3a9aa4d0 2024-07-31T15:02:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.312+0000 7f8fd0044740 1 -- 172.21.15.67:0/2176144456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb401d5e0 con 0x564b3a9aa4d0 2024-07-31T15:02:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.313+0000 7f8fa6ff5640 1 -- 172.21.15.67:0/2176144456 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9407+0+0 (secure 0 0 0) 0x7f8fb4014640 con 0x564b3a9aa4d0 2024-07-31T15:02:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.313+0000 7f8fa6ff5640 1 -- 172.21.15.67:0/2176144456 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8fb403f050 con 0x564b3a9aa4d0 2024-07-31T15:02:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.313+0000 7f8fd0044740 10 client.18015 fetch_fsmap learned FSMap version 60 2024-07-31T15:02:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.313+0000 7f8fd0044740 1 -- 172.21.15.67:0/2176144456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x564b3adf3580 con 0x564b3a9aa4d0 2024-07-31T15:02:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.314+0000 7f8fa6ff5640 1 -- 172.21.15.67:0/2176144456 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb401d5e0 con 0x564b3a9aa4d0 2024-07-31T15:02:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.314+0000 7f8fa6ff5640 1 -- 172.21.15.67:0/2176144456 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 60) ==== 8386+0+0 (secure 0 0 0) 0x7f8fb4033710 con 0x564b3a9aa4d0 2024-07-31T15:02:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.314+0000 7f8fd0044740 10 client.18015 fetch_fsmap finished waiting for FSMap version 60 2024-07-31T15:02:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.314+0000 7f8fd0044740 10 client.18015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:02:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.314+0000 7f8fd0044740 20 client.18015 populate_metadata read hostname 'smithi067' 2024-07-31T15:02:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.314+0000 7f8fd0044740 1 --2- 172.21.15.67:0/2176144456 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b3a95f1c0 0x564b3adf5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.314+0000 7f8fd0044740 4 client.18015 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:02:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.314+0000 7f8fd0044740 1 -- 172.21.15.67:0/2176144456 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b3ace0da0 con 0x564b3a95f1c0 2024-07-31T15:02:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.314+0000 7f8fc359e640 1 --2- 172.21.15.67:0/2176144456 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b3a95f1c0 0x564b3adf5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.315+0000 7f8fa6ff5640 10 client.18015 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:50.315+0000 7f8fc359e640 1 --2- 172.21.15.67:0/2176144456 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b3a95f1c0 0x564b3adf5c40 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2df894f17e1354c3 server_cookie=bf56f194ccc3e587 in_seq=0 out_seq=0 2024-07-31T15:02:50.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:02:50.469 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/uncore_discovery.o 2024-07-31T15:02:50.508 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/hibernate.o 2024-07-31T15:02:50.688 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/fd.o 2024-07-31T15:02:51.181 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/util.o 2024-07-31T15:02:51.189 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/quota/built-in.a 2024-07-31T15:02:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:50 smithi077 ceph-mon[37327]: pgmap v768: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 452 KiB/s wr, 2.71k op/s 2024-07-31T15:02:51.312 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:50 smithi067 ceph-mon[32750]: pgmap v768: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 452 KiB/s wr, 2.71k op/s 2024-07-31T15:02:51.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:50 smithi073 ceph-mon[36714]: pgmap v768: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 452 KiB/s wr, 2.71k op/s 2024-07-31T15:02:51.413 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/rtmutex_api.o 2024-07-31T15:02:51.480 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/kernfs/mount.o 2024-07-31T15:02:51.735 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/qrwlock.o 2024-07-31T15:02:51.906 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/kernfs/inode.o 2024-07-31T15:02:52.122 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/proc_tty.o 2024-07-31T15:02:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fa6ff5640 1 -- 172.21.15.67:0/2176144456 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b3ace0da0 con 0x564b3a95f1c0 2024-07-31T15:02:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fa6ff5640 10 client.18015 handle_command_reply: tid=0 2024-07-31T15:02:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fd0044740 1 client.18015 shutdown 2024-07-31T15:02:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fd0044740 1 -- 172.21.15.67:0/2176144456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9aa4d0 msgr2=0x564b3ad7e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fd0044740 1 --2- 172.21.15.67:0/2176144456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9aa4d0 0x564b3ad7e840 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4011340 tx=0x7f8fb40116b0 comp rx=0 tx=0).stop 2024-07-31T15:02:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fd0044740 1 -- 172.21.15.67:0/2176144456 shutdown_connections 2024-07-31T15:02:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fd0044740 1 --2- 172.21.15.67:0/2176144456 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564b3a95f1c0 0x564b3adf5c40 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fd0044740 1 --2- 172.21.15.67:0/2176144456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564b3a9cb1b0 0x564b3ad7ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fd0044740 1 --2- 172.21.15.67:0/2176144456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b3a9aa4d0 0x564b3ad7e840 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fd0044740 1 --2- 172.21.15.67:0/2176144456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a9a22f0 0x564b3ad81e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fd0044740 1 -- 172.21.15.67:0/2176144456 >> 172.21.15.67:0/2176144456 conn(0x564b3a9bf300 msgr2=0x564b3ad6a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fa6ff5640 0 client.18015 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:02:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fd0044740 1 -- 172.21.15.67:0/2176144456 shutdown_connections 2024-07-31T15:02:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fd0044740 1 -- 172.21.15.67:0/2176144456 wait complete. 2024-07-31T15:02:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.217+0000 7f8fd0044740 20 client.18015 trim_cache size 0 max 16384 2024-07-31T15:02:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:02:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:02:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:02:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:02:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.229+0000 7f8fc9147640 1 -- 172.21.15.67:0/2820112434 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8fac0b3f50 msgr2=0x7f8fac0b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.229+0000 7f8fc9147640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8fac0b3f50 0x7f8fac0b6460 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8010610 tx=0x7f8fb8010000 comp rx=0 tx=0).stop 2024-07-31T15:02:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.229+0000 7f8fc9147640 1 -- 172.21.15.67:0/2820112434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fc410a330 msgr2=0x7f8fc411bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.229+0000 7f8fc9147640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fc410a330 0x7f8fc411bd60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009fd0 tx=0x7f8fb800b870 comp rx=0 tx=0).stop 2024-07-31T15:02:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.229+0000 7f8fc9147640 1 -- 172.21.15.67:0/2820112434 shutdown_connections 2024-07-31T15:02:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.229+0000 7f8fc9147640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8fac0b3f50 0x7f8fac0b6460 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.229+0000 7f8fc9147640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fc411c5d0 0x7f8fc4115560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.229+0000 7f8fc9147640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc41116d0 0x7f8fc4115020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.229+0000 7f8fc9147640 1 --2- 172.21.15.67:0/2820112434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fc410a330 0x7f8fc411bd60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.229+0000 7f8fc9147640 1 -- 172.21.15.67:0/2820112434 >> 172.21.15.67:0/2820112434 conn(0x7f8fc406e350 msgr2=0x7f8fc4110350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.230+0000 7f8fc9147640 1 -- 172.21.15.67:0/2820112434 shutdown_connections 2024-07-31T15:02:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:02:52.230+0000 7f8fc9147640 1 -- 172.21.15.67:0/2820112434 wait complete. 2024-07-31T15:02:52.236 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/cstate.o 2024-07-31T15:02:52.362 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mmzone.o 2024-07-31T15:02:52.463 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/sysfs/file.o 2024-07-31T15:02:52.478 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/snapshot.o 2024-07-31T15:02:52.534 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/sysfs/dir.o 2024-07-31T15:02:52.677 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/kernfs/dir.o 2024-07-31T15:02:52.735 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/locking/built-in.a 2024-07-31T15:02:52.790 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/cmdline.o 2024-07-31T15:02:52.819 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/consoles.o 2024-07-31T15:02:52.940 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/vmstat.o 2024-07-31T15:02:53.004 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/events/intel/built-in.a 2024-07-31T15:02:53.113 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/core.o 2024-07-31T15:02:53.152 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/backing-dev.o 2024-07-31T15:02:53.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:53 smithi067 ceph-mon[32750]: pgmap v769: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 485 KiB/s wr, 2.49k op/s 2024-07-31T15:02:53.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:53 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:02:53.317 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/sysfs/symlink.o 2024-07-31T15:02:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:53 smithi073 ceph-mon[36714]: pgmap v769: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 485 KiB/s wr, 2.49k op/s 2024-07-31T15:02:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:53 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:02:53.428 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/probe.o 2024-07-31T15:02:53.455 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/cpuinfo.o 2024-07-31T15:02:53.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:53 smithi077 ceph-mon[37327]: pgmap v769: 129 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 121 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 485 KiB/s wr, 2.49k op/s 2024-07-31T15:02:53.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:53 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:02:53.849 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/realmode/init.o 2024-07-31T15:02:53.876 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/kernfs/file.o 2024-07-31T15:02:53.950 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/sysfs/mount.o 2024-07-31T15:02:54.094 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/devices.o 2024-07-31T15:02:54.140 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/swap.o 2024-07-31T15:02:54.147 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/sysfs/group.o 2024-07-31T15:02:54.378 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mm_init.o 2024-07-31T15:02:54.469 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rm/header.o 2024-07-31T15:02:54.508 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rm/trampoline_64.o 2024-07-31T15:02:54.556 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rm/stack.o 2024-07-31T15:02:54.591 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rm/reboot.o 2024-07-31T15:02:54.633 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rm/wakeup_asm.o 2024-07-31T15:02:54.663 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/interrupts.o 2024-07-31T15:02:54.672 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/realmode/rm/wakemain.o 2024-07-31T15:02:54.679 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/kernfs/symlink.o 2024-07-31T15:02:54.708 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/gc.o 2024-07-31T15:02:54.772 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/realmode/rm/video-mode.o 2024-07-31T15:02:54.777 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/sysfs/built-in.a 2024-07-31T15:02:54.833 INFO:tasks.workunit.client.0.smithi067.stdout: GEN security/selinux/flask.h security/selinux/av_permissions.h 2024-07-31T15:02:54.847 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/avc.o 2024-07-31T15:02:54.868 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/utils.o 2024-07-31T15:02:54.885 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rm/copy.o 2024-07-31T15:02:54.919 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rm/bioscall.o 2024-07-31T15:02:54.951 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/realmode/rm/regs.o 2024-07-31T15:02:55.010 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/realmode/rm/video-vga.o 2024-07-31T15:02:55.127 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/realmode/rm/video-vesa.o 2024-07-31T15:02:55.138 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/loadavg.o 2024-07-31T15:02:55.212 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/realmode/rm/video-bios.o 2024-07-31T15:02:55.235 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/kernfs/built-in.a 2024-07-31T15:02:55.253 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/user.o 2024-07-31T15:02:55.276 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/hooks.o 2024-07-31T15:02:55.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:55 smithi067 ceph-mon[32750]: pgmap v770: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 930 KiB/s wr, 2.42k op/s 2024-07-31T15:02:55.310 INFO:tasks.workunit.client.0.smithi067.stdout: PASYMS arch/x86/realmode/rm/pasyms.h 2024-07-31T15:02:55.319 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/key.o 2024-07-31T15:02:55.334 INFO:tasks.workunit.client.0.smithi067.stdout: LDS arch/x86/realmode/rm/realmode.lds 2024-07-31T15:02:55.359 INFO:tasks.workunit.client.0.smithi067.stdout: LD arch/x86/realmode/rm/realmode.elf 2024-07-31T15:02:55.371 INFO:tasks.workunit.client.0.smithi067.stdout: RELOCS arch/x86/realmode/rm/realmode.relocs 2024-07-31T15:02:55.381 INFO:tasks.workunit.client.0.smithi067.stdout: OBJCOPY arch/x86/realmode/rm/realmode.bin 2024-07-31T15:02:55.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:55 smithi073 ceph-mon[36714]: pgmap v770: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 930 KiB/s wr, 2.42k op/s 2024-07-31T15:02:55.394 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rmpiggy.o 2024-07-31T15:02:55.426 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.4 2024-07-31T15:02:55.426 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:02:55.427 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/realmode/built-in.a 2024-07-31T15:02:55.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:55 smithi077 ceph-mon[37327]: pgmap v770: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 930 KiB/s wr, 2.42k op/s 2024-07-31T15:02:55.539 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/keyring.o 2024-07-31T15:02:55.544 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/percpu.o 2024-07-31T15:02:55.547 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/rapl.o 2024-07-31T15:02:55.739 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/meminfo.o 2024-07-31T15:02:56.104 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/poweroff.o 2024-07-31T15:02:56.129 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:02:56.242 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:56 smithi067 ceph-mon[32750]: osdmap e106: 12 total, 12 up, 12 in 2024-07-31T15:02:56.306 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/stat.o 2024-07-31T15:02:56.393 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/power/built-in.a 2024-07-31T15:02:56.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:56 smithi073 ceph-mon[36714]: osdmap e106: 12 total, 12 up, 12 in 2024-07-31T15:02:56.405 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/msr.o 2024-07-31T15:02:56.463 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/printk/printk.o 2024-07-31T15:02:56.474 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/printk/printk_safe.o 2024-07-31T15:02:56.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:56 smithi077 ceph-mon[37327]: osdmap e106: 12 total, 12 up, 12 in 2024-07-31T15:02:56.543 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/printk/printk_ringbuffer.o 2024-07-31T15:02:56.706 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/keyctl.o 2024-07-31T15:02:56.974 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/uptime.o 2024-07-31T15:02:57.072 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/permission.o 2024-07-31T15:02:57.139 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/selinuxfs.o 2024-07-31T15:02:57.232 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/events/built-in.a 2024-07-31T15:02:57.376 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:57 smithi067 ceph-mon[32750]: pgmap v772: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 1.76k op/s 2024-07-31T15:02:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:57 smithi073 ceph-mon[36714]: pgmap v772: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 1.76k op/s 2024-07-31T15:02:57.467 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/fpu/init.o 2024-07-31T15:02:57.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:57 smithi077 ceph-mon[37327]: pgmap v772: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 1.76k op/s 2024-07-31T15:02:57.492 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/util.o 2024-07-31T15:02:57.680 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/fpu/bugs.o 2024-07-31T15:02:57.698 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/slab_common.o 2024-07-31T15:02:57.855 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/process_keys.o 2024-07-31T15:02:57.866 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/fpu/core.o 2024-07-31T15:02:57.904 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mce/core.o 2024-07-31T15:02:58.011 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/version.o 2024-07-31T15:02:58.184 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/printk/sysctl.o 2024-07-31T15:02:58.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:58 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:02:58.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:58 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:02:58.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:58 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:02:58.496 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/softirqs.o 2024-07-31T15:02:58.498 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/printk/built-in.a 2024-07-31T15:02:58.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.556+0000 7efc6e765640 1 -- 172.21.15.67:0/2056545044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc68111770 msgr2=0x7efc6810a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:58.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.556+0000 7efc6e765640 1 --2- 172.21.15.67:0/2056545044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc68111770 0x7efc6810a070 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7efc58009900 tx=0x7efc5802a410 comp rx=0 tx=0).stop 2024-07-31T15:02:58.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.559+0000 7efc6e765640 1 -- 172.21.15.67:0/2056545044 shutdown_connections 2024-07-31T15:02:58.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.559+0000 7efc6e765640 1 --2- 172.21.15.67:0/2056545044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc6811c5d0 0x7efc6811e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:58.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.559+0000 7efc6e765640 1 --2- 172.21.15.67:0/2056545044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc6810a5b0 0x7efc6810a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:58.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.559+0000 7efc6e765640 1 --2- 172.21.15.67:0/2056545044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc68111770 0x7efc6810a070 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:58.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.559+0000 7efc6e765640 1 -- 172.21.15.67:0/2056545044 >> 172.21.15.67:0/2056545044 conn(0x7efc6806d380 msgr2=0x7efc6806d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:58.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.561+0000 7efc6e765640 1 -- 172.21.15.67:0/2056545044 shutdown_connections 2024-07-31T15:02:58.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.561+0000 7efc6e765640 1 -- 172.21.15.67:0/2056545044 wait complete. 2024-07-31T15:02:58.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.562+0000 7efc6e765640 1 Processor -- start 2024-07-31T15:02:58.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.563+0000 7efc6e765640 1 -- start start 2024-07-31T15:02:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.563+0000 7efc6e765640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc6810a5b0 0x7efc6811bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.563+0000 7efc6e765640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc68111770 0x7efc68115100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.563+0000 7efc6e765640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc6811c5d0 0x7efc681156c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.563+0000 7efc6e765640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efc68122150 con 0x7efc6810a5b0 2024-07-31T15:02:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.563+0000 7efc6e765640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc68121fd0 con 0x7efc68111770 2024-07-31T15:02:58.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.563+0000 7efc6e765640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efc681222d0 con 0x7efc6811c5d0 2024-07-31T15:02:58.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.565+0000 7efc677fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc68111770 0x7efc68115100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:58.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.565+0000 7efc677fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc68111770 0x7efc68115100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:53424/0 (socket says 172.21.15.67:53424) 2024-07-31T15:02:58.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.565+0000 7efc677fe640 1 -- 172.21.15.67:0/4280946750 learned_addr learned my addr 172.21.15.67:0/4280946750 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:02:58.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.566+0000 7efc67fff640 1 --2- 172.21.15.67:0/4280946750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc6810a5b0 0x7efc6811bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.566+0000 7efc677fe640 1 -- 172.21.15.67:0/4280946750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc6811c5d0 msgr2=0x7efc681156c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.566+0000 7efc677fe640 1 --2- 172.21.15.67:0/4280946750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc6811c5d0 0x7efc681156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.566+0000 7efc677fe640 1 -- 172.21.15.67:0/4280946750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc6810a5b0 msgr2=0x7efc6811bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.566+0000 7efc677fe640 1 --2- 172.21.15.67:0/4280946750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc6810a5b0 0x7efc6811bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.567+0000 7efc677fe640 1 -- 172.21.15.67:0/4280946750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc68115ee0 con 0x7efc68111770 2024-07-31T15:02:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.567+0000 7efc67fff640 1 --2- 172.21.15.67:0/4280946750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc6810a5b0 0x7efc6811bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:02:58.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.567+0000 7efc677fe640 1 --2- 172.21.15.67:0/4280946750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc68111770 0x7efc68115100 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efc6000eb90 tx=0x7efc6000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:58.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.571+0000 7efc657fa640 1 -- 172.21.15.67:0/4280946750 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60010630 con 0x7efc68111770 2024-07-31T15:02:58.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.571+0000 7efc657fa640 1 -- 172.21.15.67:0/4280946750 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc6000ee90 con 0x7efc68111770 2024-07-31T15:02:58.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.571+0000 7efc657fa640 1 -- 172.21.15.67:0/4280946750 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60018b60 con 0x7efc68111770 2024-07-31T15:02:58.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.571+0000 7efc6e765640 1 -- 172.21.15.67:0/4280946750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6811ab10 con 0x7efc68111770 2024-07-31T15:02:58.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.571+0000 7efc6e765640 1 -- 172.21.15.67:0/4280946750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6811b000 con 0x7efc68111770 2024-07-31T15:02:58.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.573+0000 7efc6e765640 1 -- 172.21.15.67:0/4280946750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6810a9c0 con 0x7efc68111770 2024-07-31T15:02:58.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.576+0000 7efc657fa640 1 -- 172.21.15.67:0/4280946750 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7efc60018d00 con 0x7efc68111770 2024-07-31T15:02:58.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.577+0000 7efc657fa640 1 --2- 172.21.15.67:0/4280946750 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efc500b3f50 0x7efc500b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:02:58.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.577+0000 7efc657fa640 1 -- 172.21.15.67:0/4280946750 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9379+0+0 (secure 0 0 0) 0x7efc600e4b00 con 0x7efc68111770 2024-07-31T15:02:58.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.578+0000 7efc67fff640 1 --2- 172.21.15.67:0/4280946750 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efc500b3f50 0x7efc500b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:02:58.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.579+0000 7efc67fff640 1 --2- 172.21.15.67:0/4280946750 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efc500b3f50 0x7efc500b6460 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efc58000c00 tx=0x7efc5802a910 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:02:58.587 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/irqdesc.o 2024-07-31T15:02:58.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:58.584+0000 7efc657fa640 1 -- 172.21.15.67:0/4280946750 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7efc600aba50 con 0x7efc68111770 2024-07-31T15:02:58.707 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/handle.o 2024-07-31T15:02:58.766 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/request_key.o 2024-07-31T15:02:58.908 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/netlink.o 2024-07-31T15:02:58.932 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/fpu/regset.o 2024-07-31T15:02:59.043 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/namespaces.o 2024-07-31T15:02:59.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:59.092+0000 7efc6e765640 1 -- 172.21.15.67:0/4280946750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc681163b0 con 0x7efc68111770 2024-07-31T15:02:59.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:59.094+0000 7efc657fa640 1 -- 172.21.15.67:0/4280946750 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+11960 (secure 0 0 0) 0x7efc600af950 con 0x7efc68111770 2024-07-31T15:02:59.099 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:02:59.099 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":62,"btime":"2024-07-31T15:02:57:093671+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":62,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:02:56.671224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:02:59.099 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 62 2024-07-31T15:02:59.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:59.104+0000 7efc4effd640 1 -- 172.21.15.67:0/4280946750 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efc500b3f50 msgr2=0x7efc500b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:59.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:59.104+0000 7efc4effd640 1 --2- 172.21.15.67:0/4280946750 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efc500b3f50 0x7efc500b6460 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efc58000c00 tx=0x7efc5802a910 comp rx=0 tx=0).stop 2024-07-31T15:02:59.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:59.104+0000 7efc4effd640 1 -- 172.21.15.67:0/4280946750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc68111770 msgr2=0x7efc68115100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:02:59.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:59.104+0000 7efc4effd640 1 --2- 172.21.15.67:0/4280946750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc68111770 0x7efc68115100 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efc6000eb90 tx=0x7efc6000c6a0 comp rx=0 tx=0).stop 2024-07-31T15:02:59.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:59.104+0000 7efc4effd640 1 -- 172.21.15.67:0/4280946750 shutdown_connections 2024-07-31T15:02:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:59.104+0000 7efc4effd640 1 --2- 172.21.15.67:0/4280946750 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efc500b3f50 0x7efc500b6460 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:59.104+0000 7efc4effd640 1 --2- 172.21.15.67:0/4280946750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc6811c5d0 0x7efc681156c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:59.104+0000 7efc4effd640 1 --2- 172.21.15.67:0/4280946750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc68111770 0x7efc68115100 secure :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efc6000eb90 tx=0x7efc6000c6a0 comp rx=0 tx=0).stop 2024-07-31T15:02:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:59.104+0000 7efc4effd640 1 --2- 172.21.15.67:0/4280946750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc6810a5b0 0x7efc6811bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:02:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:59.104+0000 7efc4effd640 1 -- 172.21.15.67:0/4280946750 >> 172.21.15.67:0/4280946750 conn(0x7efc6806d380 msgr2=0x7efc6810f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:02:59.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:59.109+0000 7efc4effd640 1 -- 172.21.15.67:0/4280946750 shutdown_connections 2024-07-31T15:02:59.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:02:59.113+0000 7efc4effd640 1 -- 172.21.15.67:0/4280946750 wait complete. 2024-07-31T15:02:59.240 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/compaction.o 2024-07-31T15:02:59.242 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/show_mem.o 2024-07-31T15:02:59.312 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/manage.o 2024-07-31T15:02:59.363 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mce/severity.o 2024-07-31T15:02:59.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:59 smithi073 ceph-mon[36714]: pgmap v773: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 1.14k op/s 2024-07-31T15:02:59.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:02:59 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/4280946750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:02:59.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:59 smithi067 ceph-mon[32750]: pgmap v773: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 1.14k op/s 2024-07-31T15:02:59.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:02:59 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/4280946750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:02:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:59 smithi077 ceph-mon[37327]: pgmap v773: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 1.14k op/s 2024-07-31T15:02:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:02:59 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/4280946750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:02:59.591 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/nlmsgtab.o 2024-07-31T15:02:59.615 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/request_key_auth.o 2024-07-31T15:02:59.638 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/fpu/signal.o 2024-07-31T15:02:59.754 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/self.o 2024-07-31T15:03:00.062 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 142.4 --await-for 45.0' 2024-07-31T15:03:00.063 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 142.4 --await-for 45.0 2024-07-31T15:03:00.067 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mce/genpool.o 2024-07-31T15:03:00.070 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mce/intel.o 2024-07-31T15:03:00.254 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/user_defined.o 2024-07-31T15:03:00.276 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/thread_self.o 2024-07-31T15:03:00.447 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/spurious.o 2024-07-31T15:03:00.450 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/netif.o 2024-07-31T15:03:00.454 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/fpu/xstate.o 2024-07-31T15:03:00.720 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:03:00.765 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mce/amd.o 2024-07-31T15:03:00.778 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mtrr/mtrr.o 2024-07-31T15:03:00.820 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/compat.o 2024-07-31T15:03:00.956 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/proc_sysctl.o 2024-07-31T15:03:01.012 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/resend.o 2024-07-31T15:03:01.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:01 smithi067 ceph-mon[32750]: pgmap v774: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 529 op/s 2024-07-31T15:03:01.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:01 smithi067 ceph-mon[32750]: osdmap e107: 12 total, 12 up, 12 in 2024-07-31T15:03:01.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:01 smithi073 ceph-mon[36714]: pgmap v774: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 529 op/s 2024-07-31T15:03:01.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:01 smithi073 ceph-mon[36714]: osdmap e107: 12 total, 12 up, 12 in 2024-07-31T15:03:01.435 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/netnode.o 2024-07-31T15:03:01.437 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/interval_tree.o 2024-07-31T15:03:01.456 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/proc.o 2024-07-31T15:03:01.481 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/chip.o 2024-07-31T15:03:01.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:01 smithi077 ceph-mon[37327]: pgmap v774: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 529 op/s 2024-07-31T15:03:01.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:01 smithi077 ceph-mon[37327]: osdmap e107: 12 total, 12 up, 12 in 2024-07-31T15:03:01.564 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mtrr/if.o 2024-07-31T15:03:01.616 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/kernel/fpu/built-in.a 2024-07-31T15:03:01.662 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/acpi/boot.o 2024-07-31T15:03:01.987 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mce/threshold.o 2024-07-31T15:03:02.039 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/sysctl.o 2024-07-31T15:03:02.143 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mtrr/generic.o 2024-07-31T15:03:02.168 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/list_lru.o 2024-07-31T15:03:02.300 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/proc_net.o 2024-07-31T15:03:02.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:02 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:03:02.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:02 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:03:02.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:02 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:03:02.317 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/dummychip.o 2024-07-31T15:03:02.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:02 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:03:02.404 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:02 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:03:02.404 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:02 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:03:02.458 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/netport.o 2024-07-31T15:03:02.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:02 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:03:02.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:02 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:03:02.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:02 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:03:02.567 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/keyctl_pkey.o 2024-07-31T15:03:02.778 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/acpi/sleep.o 2024-07-31T15:03:02.784 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/devres.o 2024-07-31T15:03:02.791 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/kernel/cpu/mce/built-in.a 2024-07-31T15:03:02.872 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/workingset.o 2024-07-31T15:03:02.886 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/microcode/core.o 2024-07-31T15:03:03.067 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/kcore.o 2024-07-31T15:03:03.121 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mtrr/cleanup.o 2024-07-31T15:03:03.190 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:03 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:03:03.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:03 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:03:03.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:03 smithi067 ceph-mon[32750]: pgmap v776: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 659 KiB/s wr, 152 op/s 2024-07-31T15:03:03.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:03 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:03:03.241 INFO:tasks.workunit.client.0.smithi067.stdout: AR security/keys/built-in.a 2024-07-31T15:03:03.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.255+0000 7f1211617640 1 -- 172.21.15.67:0/2487723987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f120c11c5d0 msgr2=0x7f120c11e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:03.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.255+0000 7f1211617640 1 --2- 172.21.15.67:0/2487723987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f120c11c5d0 0x7f120c11e9f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f1204009be0 tx=0x7f120402a6d0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.256+0000 7f1211617640 1 -- 172.21.15.67:0/2487723987 shutdown_connections 2024-07-31T15:03:03.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.256+0000 7f1211617640 1 --2- 172.21.15.67:0/2487723987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f120c11c5d0 0x7f120c11e9f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.256+0000 7f1211617640 1 --2- 172.21.15.67:0/2487723987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c10a330 0x7f120c10a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.256+0000 7f1211617640 1 --2- 172.21.15.67:0/2487723987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120c111730 0x7f120c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.256+0000 7f1211617640 1 -- 172.21.15.67:0/2487723987 >> 172.21.15.67:0/2487723987 conn(0x7f120c06e440 msgr2=0x7f120c06e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:03.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.258+0000 7f1211617640 1 -- 172.21.15.67:0/2487723987 shutdown_connections 2024-07-31T15:03:03.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.260+0000 7f1211617640 1 -- 172.21.15.67:0/2487723987 wait complete. 2024-07-31T15:03:03.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.261+0000 7f1211617640 1 Processor -- start 2024-07-31T15:03:03.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.261+0000 7f1211617640 1 -- start start 2024-07-31T15:03:03.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.262+0000 7f1211617640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120c10a330 0x7f120c11ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.262+0000 7f1211617640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c111730 0x7f120c114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.262+0000 7f1211617640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f120c11c5d0 0x7f120c114550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.262+0000 7f1211617640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f120c122170 con 0x7f120c11c5d0 2024-07-31T15:03:03.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.262+0000 7f1211617640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f120c121ff0 con 0x7f120c10a330 2024-07-31T15:03:03.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.262+0000 7f1211617640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f120c1222f0 con 0x7f120c111730 2024-07-31T15:03:03.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.262+0000 7f120bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120c10a330 0x7f120c11ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.262+0000 7f120bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120c10a330 0x7f120c11ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59822/0 (socket says 172.21.15.67:59822) 2024-07-31T15:03:03.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.262+0000 7f120bfff640 1 -- 172.21.15.67:0/3508625786 learned_addr learned my addr 172.21.15.67:0/3508625786 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:03.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.263+0000 7f120bfff640 1 -- 172.21.15.67:0/3508625786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c111730 msgr2=0x7f120c114010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:03:03.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.263+0000 7f120bfff640 1 --2- 172.21.15.67:0/3508625786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c111730 0x7f120c114010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.263+0000 7f120bfff640 1 -- 172.21.15.67:0/3508625786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f120c11c5d0 msgr2=0x7f120c114550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:03:03.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.263+0000 7f1210e16640 1 --2- 172.21.15.67:0/3508625786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f120c11c5d0 0x7f120c114550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.263+0000 7f120bfff640 1 --2- 172.21.15.67:0/3508625786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f120c11c5d0 0x7f120c114550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.263+0000 7f120bfff640 1 -- 172.21.15.67:0/3508625786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c114c40 con 0x7f120c10a330 2024-07-31T15:03:03.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.263+0000 7f120bfff640 1 --2- 172.21.15.67:0/3508625786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120c10a330 0x7f120c11ad40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00e9c0 tx=0x7f11fc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:03.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.264+0000 7f12097fa640 1 -- 172.21.15.67:0/3508625786 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc00ccc0 con 0x7f120c10a330 2024-07-31T15:03:03.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.264+0000 7f1211617640 1 -- 172.21.15.67:0/3508625786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120c1199b0 con 0x7f120c10a330 2024-07-31T15:03:03.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.264+0000 7f1211617640 1 -- 172.21.15.67:0/3508625786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120c119ef0 con 0x7f120c10a330 2024-07-31T15:03:03.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.264+0000 7f12097fa640 1 -- 172.21.15.67:0/3508625786 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11fc0054d0 con 0x7f120c10a330 2024-07-31T15:03:03.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.264+0000 7f12097fa640 1 -- 172.21.15.67:0/3508625786 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc010430 con 0x7f120c10a330 2024-07-31T15:03:03.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.275+0000 7f1218dca740 1 Processor -- start 2024-07-31T15:03:03.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.277+0000 7f1218dca740 1 -- start start 2024-07-31T15:03:03.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.278+0000 7f1218dca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f8d840 0x55fc7a333a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.278+0000 7f1218dca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 0x55fc7a333fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.279+0000 7f1210e16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f8d840 0x55fc7a333a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.279+0000 7f1210e16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f8d840 0x55fc7a333a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59830/0 (socket says 172.21.15.67:59830) 2024-07-31T15:03:03.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.279+0000 7f1210e16640 1 -- 172.21.15.67:0/3508169283 learned_addr learned my addr 172.21.15.67:0/3508169283 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:03.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.280+0000 7f1218dca740 1 --2- 172.21.15.67:0/3508169283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 0x55fc7a3344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.280+0000 7f120bfff640 1 --2- 172.21.15.67:0/3508169283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 0x55fc7a333fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.280+0000 7f120b7fe640 1 --2- 172.21.15.67:0/3508169283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 0x55fc7a3344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.281+0000 7f1218dca740 1 -- 172.21.15.67:0/3508169283 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fc7a2a87b0 con 0x55fc79f6cb60 2024-07-31T15:03:03.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.281+0000 7f1218dca740 1 -- 172.21.15.67:0/3508169283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fc7a297cb0 con 0x55fc79f8d840 2024-07-31T15:03:03.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.281+0000 7f1218dca740 1 -- 172.21.15.67:0/3508169283 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fc7a29bcf0 con 0x55fc79f64980 2024-07-31T15:03:03.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.282+0000 7f12097fa640 1 -- 172.21.15.67:0/3508625786 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f11fc0046c0 con 0x7f120c10a330 2024-07-31T15:03:03.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.282+0000 7f1210e16640 1 -- 172.21.15.67:0/3508169283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 msgr2=0x55fc7a333fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:03.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.282+0000 7f1210e16640 1 --2- 172.21.15.67:0/3508169283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 0x55fc7a333fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.282+0000 7f1210e16640 1 -- 172.21.15.67:0/3508169283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 msgr2=0x55fc7a3344e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:03.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.282+0000 7f1210e16640 1 --2- 172.21.15.67:0/3508169283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 0x55fc7a3344e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.282+0000 7f1210e16640 1 -- 172.21.15.67:0/3508169283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc7a22d0b0 con 0x55fc79f8d840 2024-07-31T15:03:03.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.283+0000 7f1210e16640 1 --2- 172.21.15.67:0/3508169283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f8d840 0x55fc7a333a60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1204034040 tx=0x7f1204009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deedf599b57a41ad server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:03.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.285+0000 7f120bfff640 1 --2- 172.21.15.67:0/3508169283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 0x55fc7a333fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:03:03.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.285+0000 7f11f27fc640 1 -- 172.21.15.67:0/3508169283 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204041020 con 0x55fc79f8d840 2024-07-31T15:03:03.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.285+0000 7f11f27fc640 1 -- 172.21.15.67:0/3508169283 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1204037040 con 0x55fc79f8d840 2024-07-31T15:03:03.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.285+0000 7f120b7fe640 1 --2- 172.21.15.67:0/3508169283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 0x55fc7a3344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:03:03.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.285+0000 7f12097fa640 1 --2- 172.21.15.67:0/3508625786 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11f40b3f50 0x7f11f40b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.286+0000 7f12097fa640 1 -- 172.21.15.67:0/3508625786 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9351+0+0 (secure 0 0 0) 0x7f11fc0e44b0 con 0x7f120c10a330 2024-07-31T15:03:03.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.286+0000 7f1218dca740 1 -- 172.21.15.67:0/3508169283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f8d840 msgr2=0x55fc7a333a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:03.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.286+0000 7f1218dca740 1 --2- 172.21.15.67:0/3508169283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f8d840 0x55fc7a333a60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1204034040 tx=0x7f1204009380 comp rx=0 tx=0).stop 2024-07-31T15:03:03.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.286+0000 7f1218dca740 1 -- 172.21.15.67:0/3508169283 shutdown_connections 2024-07-31T15:03:03.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.286+0000 7f1218dca740 1 --2- 172.21.15.67:0/3508169283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 0x55fc7a3344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.286+0000 7f1218dca740 1 --2- 172.21.15.67:0/3508169283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 0x55fc7a333fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.286+0000 7f1218dca740 1 --2- 172.21.15.67:0/3508169283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f8d840 0x55fc7a333a60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.286+0000 7f1218dca740 1 -- 172.21.15.67:0/3508169283 >> 172.21.15.67:0/3508169283 conn(0x55fc79f81990 msgr2=0x55fc7a331950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:03.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.288+0000 7f120bfff640 1 --2- 172.21.15.67:0/3508625786 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11f40b3f50 0x7f11f40b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.290+0000 7f1218dca740 1 -- 172.21.15.67:0/3508169283 shutdown_connections 2024-07-31T15:03:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.291+0000 7f120bfff640 1 --2- 172.21.15.67:0/3508625786 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11f40b3f50 0x7f11f40b6460 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55fc7a3326e0 tx=0x7f11fc009f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:03.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.292+0000 7f1218dca740 1 -- 172.21.15.67:0/3508169283 wait complete. 2024-07-31T15:03:03.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.292+0000 7f1218dca740 1 -- start start 2024-07-31T15:03:03.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.293+0000 7f1218dca740 1 Processor -- start 2024-07-31T15:03:03.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.294+0000 7f1218dca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f64980 0x55fc7a3416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.294+0000 7f1218dca740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 0x55fc7a33eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.295+0000 7f1210e16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f64980 0x55fc7a3416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.295+0000 7f1210e16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f64980 0x55fc7a3416c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59840/0 (socket says 172.21.15.67:59840) 2024-07-31T15:03:03.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.295+0000 7f1210e16640 1 -- 172.21.15.67:0/229128890 learned_addr learned my addr 172.21.15.67:0/229128890 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:03.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.295+0000 7f120b7fe640 1 --2- 172.21.15.67:0/229128890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 0x55fc7a33eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.296+0000 7f1218dca740 1 --2- 172.21.15.67:0/229128890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f4c1c0 0x55fc7a33f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.296+0000 7f1218dca740 1 -- 172.21.15.67:0/229128890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fc7a0fb7a0 con 0x55fc79f6cb60 2024-07-31T15:03:03.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.296+0000 7f1218dca740 1 -- 172.21.15.67:0/229128890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fc7a23a320 con 0x55fc79f64980 2024-07-31T15:03:03.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.296+0000 7f1218dca740 1 -- 172.21.15.67:0/229128890 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fc7a342230 con 0x55fc79f4c1c0 2024-07-31T15:03:03.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.296+0000 7f1218dca740 10 client.0 _ll_register_callbacks cb 0x55fc7a244140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:03.300 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/autoprobe.o 2024-07-31T15:03:03.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.300+0000 7f1210e16640 1 --2- 172.21.15.67:0/229128890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f4c1c0 0x55fc7a33f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.300+0000 7f120b7fe640 1 -- 172.21.15.67:0/229128890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f4c1c0 msgr2=0x55fc7a33f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:03.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.300+0000 7f120b7fe640 1 --2- 172.21.15.67:0/229128890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f4c1c0 0x55fc7a33f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.300+0000 7f120b7fe640 1 -- 172.21.15.67:0/229128890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f64980 msgr2=0x55fc7a3416c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:03.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.300+0000 7f120b7fe640 1 --2- 172.21.15.67:0/229128890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f64980 0x55fc7a3416c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.300+0000 7f120b7fe640 1 -- 172.21.15.67:0/229128890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc7a23e4b0 con 0x55fc79f6cb60 2024-07-31T15:03:03.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.301+0000 7f120b7fe640 1 --2- 172.21.15.67:0/229128890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 0x55fc7a33eb90 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f120c06c0b0 tx=0x7f120000d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5315bf4a265a12 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:03.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.301+0000 7f11eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:03.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.301+0000 7f11eb7fe640 1 -- 172.21.15.67:0/229128890 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200005220 con 0x55fc79f6cb60 2024-07-31T15:03:03.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.301+0000 7f11eb7fe640 1 -- 172.21.15.67:0/229128890 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9351+0+0 (secure 0 0 0) 0x7f12000053c0 con 0x55fc79f6cb60 2024-07-31T15:03:03.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.302+0000 7f1218dca740 1 -- 172.21.15.67:0/229128890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fc7a2aa700 con 0x55fc79f6cb60 2024-07-31T15:03:03.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.302+0000 7f11eb7fe640 1 -- 172.21.15.67:0/229128890 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f12000055e0 con 0x55fc79f6cb60 2024-07-31T15:03:03.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.302+0000 7f1218dca740 10 client.18027 fetch_fsmap learned FSMap version 62 2024-07-31T15:03:03.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.302+0000 7f1218dca740 1 -- 172.21.15.67:0/229128890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55fc7a007b90 con 0x55fc79f6cb60 2024-07-31T15:03:03.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.303+0000 7f11eb7fe640 1 -- 172.21.15.67:0/229128890 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1200004c50 con 0x55fc79f6cb60 2024-07-31T15:03:03.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.303+0000 7f11eb7fe640 1 -- 172.21.15.67:0/229128890 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 62) ==== 8374+0+0 (secure 0 0 0) 0x7f1200021470 con 0x55fc79f6cb60 2024-07-31T15:03:03.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.303+0000 7f1218dca740 10 client.18027 fetch_fsmap finished waiting for FSMap version 62 2024-07-31T15:03:03.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.303+0000 7f1218dca740 10 client.18027 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:03:03.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.303+0000 7f1218dca740 20 client.18027 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:03.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.303+0000 7f1218dca740 1 --2- 172.21.15.67:0/229128890 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fc79f21570 0x55fc7a350d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.303+0000 7f1218dca740 4 client.18027 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:03:03.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.303+0000 7f1218dca740 1 -- 172.21.15.67:0/229128890 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc7a2aa700 con 0x55fc79f21570 2024-07-31T15:03:03.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.304+0000 7f1210e16640 1 --2- 172.21.15.67:0/229128890 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fc79f21570 0x55fc7a350d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.307+0000 7f1210e16640 1 --2- 172.21.15.67:0/229128890 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fc79f21570 0x55fc7a350d40 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=239b482340cd6af0 server_cookie=fff85c8b952a3d2b in_seq=0 out_seq=0 2024-07-31T15:03:03.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.307+0000 7f11eb7fe640 10 client.18027 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:03.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.309+0000 7f11eb7fe640 1 -- 172.21.15.67:0/229128890 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc7a2aa700 con 0x55fc79f21570 2024-07-31T15:03:03.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.309+0000 7f11eb7fe640 10 client.18027 handle_command_reply: tid=0 2024-07-31T15:03:03.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.309+0000 7f1218dca740 1 client.18027 shutdown 2024-07-31T15:03:03.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.312+0000 7f1218dca740 1 -- 172.21.15.67:0/229128890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 msgr2=0x55fc7a33eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:03.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.312+0000 7f1218dca740 1 --2- 172.21.15.67:0/229128890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 0x55fc7a33eb90 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f120c06c0b0 tx=0x7f120000d920 comp rx=0 tx=0).stop 2024-07-31T15:03:03.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.316+0000 7f1218dca740 1 -- 172.21.15.67:0/229128890 shutdown_connections 2024-07-31T15:03:03.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.316+0000 7f1218dca740 1 --2- 172.21.15.67:0/229128890 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fc79f21570 0x55fc7a350d40 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.316+0000 7f1218dca740 1 --2- 172.21.15.67:0/229128890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f4c1c0 0x55fc7a33f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.316+0000 7f1218dca740 1 --2- 172.21.15.67:0/229128890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 0x55fc7a33eb90 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.316+0000 7f1218dca740 1 --2- 172.21.15.67:0/229128890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f64980 0x55fc7a3416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.316+0000 7f1218dca740 1 -- 172.21.15.67:0/229128890 >> 172.21.15.67:0/229128890 conn(0x55fc79f81990 msgr2=0x55fc7a336bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:03.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.316+0000 7f11eb7fe640 0 client.18027 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:03.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.318+0000 7f1218dca740 1 -- 172.21.15.67:0/229128890 shutdown_connections 2024-07-31T15:03:03.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.319+0000 7f1218dca740 1 -- 172.21.15.67:0/229128890 wait complete. 2024-07-31T15:03:03.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.319+0000 7f1218dca740 20 client.18027 trim_cache size 0 max 16384 2024-07-31T15:03:03.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.358+0000 7f1218dca740 1 Processor -- start 2024-07-31T15:03:03.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.359+0000 7f1218dca740 1 -- start start 2024-07-31T15:03:03.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.359+0000 7f1218dca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f4c1c0 0x55fc7a34b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.359+0000 7f1218dca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 0x55fc7a335450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.360+0000 7f1218dca740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 0x55fc7a3565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.360+0000 7f1218dca740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f11f800ff40 con 0x55fc79f6cb60 2024-07-31T15:03:03.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.360+0000 7f1218dca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11f8011ab0 con 0x55fc79f4c1c0 2024-07-31T15:03:03.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.360+0000 7f1218dca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11f8010610 con 0x55fc79f64980 2024-07-31T15:03:03.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.360+0000 7f1210e16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f4c1c0 0x55fc7a34b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.360+0000 7f1210e16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f4c1c0 0x55fc7a34b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59848/0 (socket says 172.21.15.67:59848) 2024-07-31T15:03:03.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.360+0000 7f1210e16640 1 -- 172.21.15.67:0/374018252 learned_addr learned my addr 172.21.15.67:0/374018252 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:03.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.361+0000 7f120b7fe640 1 --2- 172.21.15.67:0/374018252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 0x55fc7a335450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.361+0000 7f1210e16640 1 --2- 172.21.15.67:0/374018252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 0x55fc7a3565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.361+0000 7f1210e16640 1 -- 172.21.15.67:0/374018252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 msgr2=0x55fc7a335450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:03.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.361+0000 7f1210e16640 1 --2- 172.21.15.67:0/374018252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 0x55fc7a335450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.361+0000 7f1210e16640 1 -- 172.21.15.67:0/374018252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 msgr2=0x55fc7a3565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:03.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.361+0000 7f1210e16640 1 --2- 172.21.15.67:0/374018252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 0x55fc7a3565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.361+0000 7f1210e16640 1 -- 172.21.15.67:0/374018252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc7a336190 con 0x55fc79f4c1c0 2024-07-31T15:03:03.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.364+0000 7f1210e16640 1 --2- 172.21.15.67:0/374018252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f4c1c0 0x55fc7a34b3f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1204006010 tx=0x7f1204009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e3e77d37560a0c8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:03.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.367+0000 7f11f1ffb640 1 -- 172.21.15.67:0/374018252 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204041040 con 0x55fc79f4c1c0 2024-07-31T15:03:03.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.367+0000 7f11f1ffb640 1 -- 172.21.15.67:0/374018252 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1204003e20 con 0x55fc79f4c1c0 2024-07-31T15:03:03.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.367+0000 7f11f1ffb640 1 -- 172.21.15.67:0/374018252 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120403d6e0 con 0x55fc79f4c1c0 2024-07-31T15:03:03.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.368+0000 7f1218dca740 1 -- 172.21.15.67:0/374018252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f4c1c0 msgr2=0x55fc7a34b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:03.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.368+0000 7f1218dca740 1 --2- 172.21.15.67:0/374018252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f4c1c0 0x55fc7a34b3f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1204006010 tx=0x7f1204009840 comp rx=0 tx=0).stop 2024-07-31T15:03:03.370 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/devpts/inode.o 2024-07-31T15:03:03.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.373+0000 7f1218dca740 1 -- 172.21.15.67:0/374018252 shutdown_connections 2024-07-31T15:03:03.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.373+0000 7f1218dca740 1 --2- 172.21.15.67:0/374018252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f6cb60 0x55fc7a3565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.373+0000 7f1218dca740 1 --2- 172.21.15.67:0/374018252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 0x55fc7a335450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.373+0000 7f1218dca740 1 --2- 172.21.15.67:0/374018252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f4c1c0 0x55fc7a34b3f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.373+0000 7f1218dca740 1 -- 172.21.15.67:0/374018252 >> 172.21.15.67:0/374018252 conn(0x55fc79f81990 msgr2=0x55fc7a336b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:03.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.374+0000 7f1218dca740 1 -- 172.21.15.67:0/374018252 shutdown_connections 2024-07-31T15:03:03.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.375+0000 7f1218dca740 1 -- 172.21.15.67:0/374018252 wait complete. 2024-07-31T15:03:03.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.375+0000 7f1218dca740 1 -- start start 2024-07-31T15:03:03.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.376+0000 7f1218dca740 1 Processor -- start 2024-07-31T15:03:03.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.377+0000 7f1218dca740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f4c1c0 0x55fc7a343c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.377+0000 7f1218dca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 0x55fc7a344140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.377+0000 7f1210e16640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f4c1c0 0x55fc7a343c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.377+0000 7f1210e16640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f4c1c0 0x55fc7a343c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48476/0 (socket says 172.21.15.67:48476) 2024-07-31T15:03:03.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.377+0000 7f1210e16640 1 -- 172.21.15.67:0/543807090 learned_addr learned my addr 172.21.15.67:0/543807090 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:03.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.377+0000 7f1218dca740 1 --2- 172.21.15.67:0/543807090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f6cb60 0x55fc7a3b8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.377+0000 7f1218dca740 1 -- 172.21.15.67:0/543807090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f11f80113e0 con 0x55fc79f4c1c0 2024-07-31T15:03:03.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.378+0000 7f1218dca740 1 -- 172.21.15.67:0/543807090 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11f8010ce0 con 0x55fc79f6cb60 2024-07-31T15:03:03.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.378+0000 7f120b7fe640 1 --2- 172.21.15.67:0/543807090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 0x55fc7a344140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.378+0000 7f1218dca740 1 -- 172.21.15.67:0/543807090 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11f800df40 con 0x55fc79f64980 2024-07-31T15:03:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.378+0000 7f1210e16640 1 -- 172.21.15.67:0/543807090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 msgr2=0x55fc7a344140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.378+0000 7f1210e16640 1 --2- 172.21.15.67:0/543807090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 0x55fc7a344140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.378+0000 7f1210e16640 1 -- 172.21.15.67:0/543807090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f6cb60 msgr2=0x55fc7a3b8590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:03:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.378+0000 7f1210e16640 1 --2- 172.21.15.67:0/543807090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f6cb60 0x55fc7a3b8590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.378+0000 7f1210e16640 1 -- 172.21.15.67:0/543807090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc7a3b8b80 con 0x55fc79f4c1c0 2024-07-31T15:03:03.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.379+0000 7f1210e16640 1 --2- 172.21.15.67:0/543807090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f4c1c0 0x55fc7a343c00 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1204030b80 tx=0x7f1204000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32893676d2895a89 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:03.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.380+0000 7f11eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:03.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.380+0000 7f11eb7fe640 1 -- 172.21.15.67:0/543807090 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204041070 con 0x55fc79f4c1c0 2024-07-31T15:03:03.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.380+0000 7f1218dca740 10 client.0 _ll_register_callbacks cb 0x7f120000bc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:03.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.380+0000 7f11eb7fe640 1 -- 172.21.15.67:0/543807090 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9351+0+0 (secure 0 0 0) 0x7f12040044b0 con 0x55fc79f4c1c0 2024-07-31T15:03:03.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.380+0000 7f1218dca740 1 -- 172.21.15.67:0/543807090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1204003e20 con 0x55fc79f4c1c0 2024-07-31T15:03:03.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.381+0000 7f11eb7fe640 1 -- 172.21.15.67:0/543807090 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f120405f050 con 0x55fc79f4c1c0 2024-07-31T15:03:03.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.382+0000 7f1218dca740 10 client.18033 fetch_fsmap learned FSMap version 62 2024-07-31T15:03:03.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.382+0000 7f1218dca740 1 -- 172.21.15.67:0/543807090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({fsmap=62}) -- 0x55fc7a3b9a30 con 0x55fc79f4c1c0 2024-07-31T15:03:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.382+0000 7f11eb7fe640 1 -- 172.21.15.67:0/543807090 <== mon.0 v2:172.21.15.67:3300/0 4 ==== fsmap(e 62) ==== 8374+0+0 (secure 0 0 0) 0x7f1204003970 con 0x55fc79f4c1c0 2024-07-31T15:03:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.382+0000 7f1218dca740 10 client.18033 fetch_fsmap finished waiting for FSMap version 62 2024-07-31T15:03:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.382+0000 7f1218dca740 10 client.18033 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:03:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.383+0000 7f1218dca740 20 client.18033 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.383+0000 7f1218dca740 1 --2- 172.21.15.67:0/543807090 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fc7a3b9c90 0x55fc7a3bc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.383+0000 7f1218dca740 4 client.18033 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.4, "await_for": 45.0}] 2024-07-31T15:03:03.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.383+0000 7f1218dca740 1 -- 172.21.15.67:0/543807090 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.4, "await_for": 45.0}) -- 0x7f1204003e20 con 0x55fc7a3b9c90 2024-07-31T15:03:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.383+0000 7f120b7fe640 1 --2- 172.21.15.67:0/543807090 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fc7a3b9c90 0x55fc7a3bc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:03.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:03 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:03:03.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:03 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:03:03.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:03 smithi073 ceph-mon[36714]: pgmap v776: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 659 KiB/s wr, 152 op/s 2024-07-31T15:03:03.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:03 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:03:03.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.402+0000 7f120b7fe640 1 --2- 172.21.15.67:0/543807090 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fc7a3b9c90 0x55fc7a3bc0b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bab89cfa71b45ca1 server_cookie=55244d7d3026fcb6 in_seq=0 out_seq=0 2024-07-31T15:03:03.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:03.403+0000 7f11eb7fe640 10 client.18033 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:03.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:03 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:03:03.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:03 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:03:03.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:03 smithi077 ceph-mon[37327]: pgmap v776: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 659 KiB/s wr, 152 op/s 2024-07-31T15:03:03.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:03 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:03:04.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:03:04.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:03:04.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:03:04.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:03:04.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:03:04.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:03:04.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:03:04.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:03:04.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:03:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.356+0000 7ff2e569b640 1 -- 172.21.15.67:0/1509871300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2e0104cd0 msgr2=0x7ff2e010d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.356+0000 7ff2e569b640 1 --2- 172.21.15.67:0/1509871300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2e0104cd0 0x7ff2e010d210 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009900 tx=0x7ff2d402a410 comp rx=0 tx=0).stop 2024-07-31T15:03:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.356+0000 7ff2e569b640 1 -- 172.21.15.67:0/1509871300 shutdown_connections 2024-07-31T15:03:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.356+0000 7ff2e569b640 1 --2- 172.21.15.67:0/1509871300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2e010d7e0 0x7ff2e010fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.356+0000 7ff2e569b640 1 --2- 172.21.15.67:0/1509871300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2e0104cd0 0x7ff2e010d210 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.356+0000 7ff2e569b640 1 --2- 172.21.15.67:0/1509871300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2e0104380 0x7ff2e0104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.356+0000 7ff2e569b640 1 -- 172.21.15.67:0/1509871300 >> 172.21.15.67:0/1509871300 conn(0x7ff2e00fde30 msgr2=0x7ff2e0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.356+0000 7ff2e569b640 1 -- 172.21.15.67:0/1509871300 shutdown_connections 2024-07-31T15:03:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.356+0000 7ff2e569b640 1 -- 172.21.15.67:0/1509871300 wait complete. 2024-07-31T15:03:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.357+0000 7ff2e569b640 1 Processor -- start 2024-07-31T15:03:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.357+0000 7ff2e569b640 1 -- start start 2024-07-31T15:03:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.357+0000 7ff2e569b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2e0104380 0x7ff2e01020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.357+0000 7ff2e569b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2e0104cd0 0x7ff2e01025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.357+0000 7ff2e569b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2e010d7e0 0x7ff2e0102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.357+0000 7ff2e569b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e0113420 con 0x7ff2e0104380 2024-07-31T15:03:05.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.357+0000 7ff2e569b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01132a0 con 0x7ff2e010d7e0 2024-07-31T15:03:05.362 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.357+0000 7ff2e569b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01135a0 con 0x7ff2e0104cd0 2024-07-31T15:03:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2deffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2e0104380 0x7ff2e01020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2deffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2e0104380 0x7ff2e01020b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48490/0 (socket says 172.21.15.67:48490) 2024-07-31T15:03:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2deffd640 1 -- 172.21.15.67:0/3860907441 learned_addr learned my addr 172.21.15.67:0/3860907441 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2df7fe640 1 --2- 172.21.15.67:0/3860907441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2e010d7e0 0x7ff2e0102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2de7fc640 1 --2- 172.21.15.67:0/3860907441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2e0104cd0 0x7ff2e01025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2deffd640 1 -- 172.21.15.67:0/3860907441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2e0104cd0 msgr2=0x7ff2e01025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2deffd640 1 --2- 172.21.15.67:0/3860907441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2e0104cd0 0x7ff2e01025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2deffd640 1 -- 172.21.15.67:0/3860907441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2e010d7e0 msgr2=0x7ff2e0102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2deffd640 1 --2- 172.21.15.67:0/3860907441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2e010d7e0 0x7ff2e0102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2deffd640 1 -- 172.21.15.67:0/3860907441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e01a8150 con 0x7ff2e0104380 2024-07-31T15:03:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2deffd640 1 --2- 172.21.15.67:0/3860907441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2e0104380 0x7ff2e01020b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000e9f0 tx=0x7ff2d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2c7fff640 1 -- 172.21.15.67:0/3860907441 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d000cd80 con 0x7ff2e0104380 2024-07-31T15:03:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2c7fff640 1 -- 172.21.15.67:0/3860907441 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d00054d0 con 0x7ff2e0104380 2024-07-31T15:03:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2e569b640 1 -- 172.21.15.67:0/3860907441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e01a8440 con 0x7ff2e0104380 2024-07-31T15:03:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2c7fff640 1 -- 172.21.15.67:0/3860907441 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0010690 con 0x7ff2e0104380 2024-07-31T15:03:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.358+0000 7ff2e569b640 1 -- 172.21.15.67:0/3860907441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e01a8910 con 0x7ff2e0104380 2024-07-31T15:03:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.361+0000 7ff2ec593740 1 Processor -- start 2024-07-31T15:03:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.361+0000 7ff2ec593740 1 -- start start 2024-07-31T15:03:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.361+0000 7ff2c7fff640 1 -- 172.21.15.67:0/3860907441 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff2d0004e50 con 0x7ff2e0104380 2024-07-31T15:03:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.361+0000 7ff2ec593740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dd061b0 0x557a9e0a8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.361+0000 7ff2ec593740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0a9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.361+0000 7ff2ec593740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 0x557a9e0a9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.361+0000 7ff2ec593740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557a9e01e3b0 con 0x557a9dcdd2f0 2024-07-31T15:03:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.361+0000 7ff2ec593740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a9e00d8b0 con 0x557a9dd061b0 2024-07-31T15:03:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.361+0000 7ff2ec593740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a9e0118f0 con 0x557a9dce54d0 2024-07-31T15:03:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2deffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2df7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dd061b0 0x557a9e0a8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2deffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0a9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48496/0 (socket says 172.21.15.67:48496) 2024-07-31T15:03:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2df7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dd061b0 0x557a9e0a8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59878/0 (socket says 172.21.15.67:59878) 2024-07-31T15:03:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2deffd640 1 -- 172.21.15.67:0/625459118 learned_addr learned my addr 172.21.15.67:0/625459118 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2de7fc640 1 --2- 172.21.15.67:0/625459118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 0x557a9e0a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2de7fc640 1 -- 172.21.15.67:0/625459118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dd061b0 msgr2=0x557a9e0a8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2de7fc640 1 --2- 172.21.15.67:0/625459118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dd061b0 0x557a9e0a8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2de7fc640 1 -- 172.21.15.67:0/625459118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 msgr2=0x557a9e0a9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2de7fc640 1 --2- 172.21.15.67:0/625459118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0a9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2de7fc640 1 -- 172.21.15.67:0/625459118 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9dfb49a0 con 0x557a9dce54d0 2024-07-31T15:03:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2c7fff640 1 --2- 172.21.15.67:0/3860907441 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff2c80b3d70 0x7ff2c80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2de7fc640 1 --2- 172.21.15.67:0/625459118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 0x557a9e0a9760 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff2e006ad40 tx=0x7ff2d4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3986a731887b671d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2df7fe640 1 --2- 172.21.15.67:0/3860907441 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff2c80b3d70 0x7ff2c80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2c7fff640 1 -- 172.21.15.67:0/3860907441 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9351+0+0 (secure 0 0 0) 0x7ff2d00e4700 con 0x7ff2e0104380 2024-07-31T15:03:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.362+0000 7ff2c57fa640 1 -- 172.21.15.67:0/625459118 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4035040 con 0x557a9dce54d0 2024-07-31T15:03:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.363+0000 7ff2c57fa640 1 -- 172.21.15.67:0/625459118 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d4004030 con 0x557a9dce54d0 2024-07-31T15:03:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.363+0000 7ff2c57fa640 1 -- 172.21.15.67:0/625459118 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40053d0 con 0x557a9dce54d0 2024-07-31T15:03:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.363+0000 7ff2ec593740 1 -- 172.21.15.67:0/625459118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 msgr2=0x557a9e0a9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.363+0000 7ff2df7fe640 1 --2- 172.21.15.67:0/3860907441 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff2c80b3d70 0x7ff2c80b6280 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8005fd0 tx=0x7ff2d8009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.363+0000 7ff2ec593740 1 --2- 172.21.15.67:0/625459118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 0x557a9e0a9760 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff2e006ad40 tx=0x7ff2d4000bc0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.363+0000 7ff2ec593740 1 -- 172.21.15.67:0/625459118 shutdown_connections 2024-07-31T15:03:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.363+0000 7ff2ec593740 1 --2- 172.21.15.67:0/625459118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 0x557a9e0a9760 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.363+0000 7ff2ec593740 1 --2- 172.21.15.67:0/625459118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0a9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.363+0000 7ff2ec593740 1 --2- 172.21.15.67:0/625459118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dd061b0 0x557a9e0a8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.363+0000 7ff2ec593740 1 -- 172.21.15.67:0/625459118 >> 172.21.15.67:0/625459118 conn(0x557a9dcfa300 msgr2=0x557a9e0a6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.363+0000 7ff2ec593740 1 -- 172.21.15.67:0/625459118 shutdown_connections 2024-07-31T15:03:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.363+0000 7ff2ec593740 1 -- 172.21.15.67:0/625459118 wait complete. 2024-07-31T15:03:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.364+0000 7ff2ec593740 1 -- start start 2024-07-31T15:03:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.364+0000 7ff2ec593740 1 Processor -- start 2024-07-31T15:03:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.364+0000 7ff2ec593740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.364+0000 7ff2ec593740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dce54d0 0x557a9e0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.364+0000 7ff2ec593740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dd061b0 0x557a9e0c06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.364+0000 7ff2ec593740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557a9df47ab0 con 0x557a9dcdd2f0 2024-07-31T15:03:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.364+0000 7ff2ec593740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a9df95cf0 con 0x557a9dce54d0 2024-07-31T15:03:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.364+0000 7ff2ec593740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a9e0b7400 con 0x557a9dd061b0 2024-07-31T15:03:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.364+0000 7ff2ec593740 10 client.0 _ll_register_callbacks cb 0x557a9e0079f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.364+0000 7ff2deffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.364+0000 7ff2deffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48498/0 (socket says 172.21.15.67:48498) 2024-07-31T15:03:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.364+0000 7ff2deffd640 1 -- 172.21.15.67:0/2889605667 learned_addr learned my addr 172.21.15.67:0/2889605667 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.364+0000 7ff2df7fe640 1 --2- 172.21.15.67:0/2889605667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dd061b0 0x557a9e0c06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2de7fc640 1 --2- 172.21.15.67:0/2889605667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dce54d0 0x557a9e0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2deffd640 1 -- 172.21.15.67:0/2889605667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dd061b0 msgr2=0x557a9e0c06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2deffd640 1 --2- 172.21.15.67:0/2889605667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dd061b0 0x557a9e0c06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2deffd640 1 -- 172.21.15.67:0/2889605667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dce54d0 msgr2=0x557a9e0b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2deffd640 1 --2- 172.21.15.67:0/2889605667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dce54d0 0x557a9e0b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2deffd640 1 -- 172.21.15.67:0/2889605667 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a9dd71790 con 0x557a9dcdd2f0 2024-07-31T15:03:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2df7fe640 1 --2- 172.21.15.67:0/2889605667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dd061b0 0x557a9e0c06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:03:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2deffd640 1 --2- 172.21.15.67:0/2889605667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0b4810 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009250 tx=0x7ff2d0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca955aa37c913e0d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2c2ff5640 1 -- 172.21.15.67:0/2889605667 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00aa920 con 0x557a9dcdd2f0 2024-07-31T15:03:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2c2ff5640 1 -- 172.21.15.67:0/2889605667 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9351+0+0 (secure 0 0 0) 0x7ff2d00e4310 con 0x557a9dcdd2f0 2024-07-31T15:03:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2ec593740 1 -- 172.21.15.67:0/2889605667 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2d4004030 con 0x557a9dcdd2f0 2024-07-31T15:03:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2c2ff5640 1 -- 172.21.15.67:0/2889605667 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7ff2d00aa740 con 0x557a9dcdd2f0 2024-07-31T15:03:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2ec593740 10 client.18045 fetch_fsmap learned FSMap version 62 2024-07-31T15:03:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.365+0000 7ff2ec593740 1 -- 172.21.15.67:0/2889605667 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x557a9ddea620 con 0x557a9dcdd2f0 2024-07-31T15:03:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.366+0000 7ff2c2ff5640 1 -- 172.21.15.67:0/2889605667 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d4004030 con 0x557a9dcdd2f0 2024-07-31T15:03:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.366+0000 7ff2c2ff5640 1 -- 172.21.15.67:0/2889605667 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 62) ==== 8374+0+0 (secure 0 0 0) 0x7ff2d00b3bf0 con 0x557a9dcdd2f0 2024-07-31T15:03:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.366+0000 7ff2ec593740 10 client.18045 fetch_fsmap finished waiting for FSMap version 62 2024-07-31T15:03:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.366+0000 7ff2ec593740 10 client.18045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.366+0000 7ff2ec593740 20 client.18045 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.366+0000 7ff2ec593740 1 --2- 172.21.15.67:0/2889605667 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a9dcc4d80 0x557a9e0c6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.366+0000 7ff2ec593740 4 client.18045 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:03:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.366+0000 7ff2ec593740 1 -- 172.21.15.67:0/2889605667 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a9e020300 con 0x557a9dcc4d80 2024-07-31T15:03:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.366+0000 7ff2de7fc640 1 --2- 172.21.15.67:0/2889605667 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a9dcc4d80 0x557a9e0c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.366+0000 7ff2c2ff5640 10 client.18045 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.366+0000 7ff2de7fc640 1 --2- 172.21.15.67:0/2889605667 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a9dcc4d80 0x557a9e0c6730 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=195b29a97aac2328 server_cookie=bcfdc443fccc56b4 in_seq=0 out_seq=0 2024-07-31T15:03:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2c2ff5640 1 -- 172.21.15.67:0/2889605667 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a9e020300 con 0x557a9dcc4d80 2024-07-31T15:03:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2c2ff5640 10 client.18045 handle_command_reply: tid=0 2024-07-31T15:03:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2ec593740 1 client.18045 shutdown 2024-07-31T15:03:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2ec593740 1 -- 172.21.15.67:0/2889605667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 msgr2=0x557a9e0b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2ec593740 1 --2- 172.21.15.67:0/2889605667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0b4810 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009250 tx=0x7ff2d0009310 comp rx=0 tx=0).stop 2024-07-31T15:03:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2ec593740 1 -- 172.21.15.67:0/2889605667 shutdown_connections 2024-07-31T15:03:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2ec593740 1 --2- 172.21.15.67:0/2889605667 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a9dcc4d80 0x557a9e0c6730 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2ec593740 1 --2- 172.21.15.67:0/2889605667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dd061b0 0x557a9e0c06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2ec593740 1 --2- 172.21.15.67:0/2889605667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dce54d0 0x557a9e0b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2c2ff5640 0 client.18045 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2ec593740 1 --2- 172.21.15.67:0/2889605667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0b4810 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2ec593740 1 -- 172.21.15.67:0/2889605667 >> 172.21.15.67:0/2889605667 conn(0x557a9dcfa300 msgr2=0x557a9e0a52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2ec593740 1 -- 172.21.15.67:0/2889605667 shutdown_connections 2024-07-31T15:03:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2ec593740 1 -- 172.21.15.67:0/2889605667 wait complete. 2024-07-31T15:03:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.367+0000 7ff2ec593740 20 client.18045 trim_cache size 0 max 16384 2024-07-31T15:03:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.383+0000 7ff2ec593740 1 Processor -- start 2024-07-31T15:03:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.383+0000 7ff2ec593740 1 -- start start 2024-07-31T15:03:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.383+0000 7ff2ec593740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dcdd2f0 0x557a9e0aee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.383+0000 7ff2ec593740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 0x557a9e0aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.383+0000 7ff2ec593740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dd061b0 0x557a9e0b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.383+0000 7ff2ec593740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b28e0 con 0x557a9dd061b0 2024-07-31T15:03:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.383+0000 7ff2ec593740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b4450 con 0x557a9dcdd2f0 2024-07-31T15:03:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.383+0000 7ff2ec593740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b2fb0 con 0x557a9dce54d0 2024-07-31T15:03:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.383+0000 7ff2df7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dd061b0 0x557a9e0b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.383+0000 7ff2deffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dcdd2f0 0x557a9e0aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.383+0000 7ff2df7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dd061b0 0x557a9e0b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48510/0 (socket says 172.21.15.67:48510) 2024-07-31T15:03:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.383+0000 7ff2de7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 0x557a9e0aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.383+0000 7ff2df7fe640 1 -- 172.21.15.67:0/916181030 learned_addr learned my addr 172.21.15.67:0/916181030 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.384+0000 7ff2df7fe640 1 -- 172.21.15.67:0/916181030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 msgr2=0x557a9e0aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.384+0000 7ff2df7fe640 1 --2- 172.21.15.67:0/916181030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 0x557a9e0aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.384+0000 7ff2df7fe640 1 -- 172.21.15.67:0/916181030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dcdd2f0 msgr2=0x557a9e0aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.384+0000 7ff2df7fe640 1 --2- 172.21.15.67:0/916181030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dcdd2f0 0x557a9e0aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.384+0000 7ff2df7fe640 1 -- 172.21.15.67:0/916181030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9e0c3240 con 0x557a9dd061b0 2024-07-31T15:03:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.384+0000 7ff2df7fe640 1 --2- 172.21.15.67:0/916181030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dd061b0 0x557a9e0b4ae0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80042f0 tx=0x7ff2d801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ef4d66c1c9d4f7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.384+0000 7ff2c4ff9640 1 -- 172.21.15.67:0/916181030 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8004b20 con 0x557a9dd061b0 2024-07-31T15:03:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.384+0000 7ff2c4ff9640 1 -- 172.21.15.67:0/916181030 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d8004cc0 con 0x557a9dd061b0 2024-07-31T15:03:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.384+0000 7ff2c4ff9640 1 -- 172.21.15.67:0/916181030 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8005890 con 0x557a9dd061b0 2024-07-31T15:03:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.385+0000 7ff2ec593740 1 -- 172.21.15.67:0/916181030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dd061b0 msgr2=0x557a9e0b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.385+0000 7ff2ec593740 1 --2- 172.21.15.67:0/916181030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dd061b0 0x557a9e0b4ae0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80042f0 tx=0x7ff2d801a000 comp rx=0 tx=0).stop 2024-07-31T15:03:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.385+0000 7ff2ec593740 1 -- 172.21.15.67:0/916181030 shutdown_connections 2024-07-31T15:03:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.385+0000 7ff2ec593740 1 --2- 172.21.15.67:0/916181030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dd061b0 0x557a9e0b4ae0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.385+0000 7ff2ec593740 1 --2- 172.21.15.67:0/916181030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 0x557a9e0aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.385+0000 7ff2ec593740 1 --2- 172.21.15.67:0/916181030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dcdd2f0 0x557a9e0aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.385+0000 7ff2ec593740 1 -- 172.21.15.67:0/916181030 >> 172.21.15.67:0/916181030 conn(0x557a9dcfa300 msgr2=0x557a9e0b5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.385+0000 7ff2ec593740 1 -- 172.21.15.67:0/916181030 shutdown_connections 2024-07-31T15:03:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.385+0000 7ff2ec593740 1 -- 172.21.15.67:0/916181030 wait complete. 2024-07-31T15:03:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.385+0000 7ff2ec593740 1 -- start start 2024-07-31T15:03:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.385+0000 7ff2ec593740 1 Processor -- start 2024-07-31T15:03:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2ec593740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2ec593740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 0x557a9e0b91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2ec593740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dd061b0 0x557a9e0b96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2ec593740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b3d80 con 0x557a9dcdd2f0 2024-07-31T15:03:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2ec593740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b3680 con 0x557a9dd061b0 2024-07-31T15:03:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2ec593740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b08e0 con 0x557a9dce54d0 2024-07-31T15:03:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2ec593740 10 client.0 _ll_register_callbacks cb 0x7ff2d0009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2deffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2de7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 0x557a9e0b91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2deffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0bcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48514/0 (socket says 172.21.15.67:48514) 2024-07-31T15:03:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2deffd640 1 -- 172.21.15.67:0/1731031136 learned_addr learned my addr 172.21.15.67:0/1731031136 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2df7fe640 1 --2- 172.21.15.67:0/1731031136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dd061b0 0x557a9e0b96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2deffd640 1 -- 172.21.15.67:0/1731031136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 msgr2=0x557a9e0b91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2deffd640 1 --2- 172.21.15.67:0/1731031136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 0x557a9e0b91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2deffd640 1 -- 172.21.15.67:0/1731031136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dd061b0 msgr2=0x557a9e0b96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2deffd640 1 --2- 172.21.15.67:0/1731031136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dd061b0 0x557a9e0b96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2deffd640 1 -- 172.21.15.67:0/1731031136 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a9e0b9e60 con 0x557a9dcdd2f0 2024-07-31T15:03:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2de7fc640 1 --2- 172.21.15.67:0/1731031136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 0x557a9e0b91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:03:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2deffd640 1 --2- 172.21.15.67:0/1731031136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0bcd00 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000c6a0 tx=0x7ff2d00e45c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2dcfe7d4a53611 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2c2ff5640 1 -- 172.21.15.67:0/1731031136 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00aa920 con 0x557a9dcdd2f0 2024-07-31T15:03:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2c2ff5640 1 -- 172.21.15.67:0/1731031136 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9351+0+0 (secure 0 0 0) 0x7ff2d00aa440 con 0x557a9dcdd2f0 2024-07-31T15:03:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.386+0000 7ff2ec593740 1 -- 172.21.15.67:0/1731031136 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2d8004cc0 con 0x557a9dcdd2f0 2024-07-31T15:03:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.387+0000 7ff2c2ff5640 1 -- 172.21.15.67:0/1731031136 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7ff2d00a6e90 con 0x557a9dcdd2f0 2024-07-31T15:03:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.387+0000 7ff2ec593740 10 client.18051 fetch_fsmap learned FSMap version 62 2024-07-31T15:03:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.387+0000 7ff2ec593740 1 -- 172.21.15.67:0/1731031136 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x557a9e12de60 con 0x557a9dcdd2f0 2024-07-31T15:03:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.387+0000 7ff2c2ff5640 1 -- 172.21.15.67:0/1731031136 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d8004cc0 con 0x557a9dcdd2f0 2024-07-31T15:03:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.387+0000 7ff2c2ff5640 1 -- 172.21.15.67:0/1731031136 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 62) ==== 8374+0+0 (secure 0 0 0) 0x7ff2d00af060 con 0x557a9dcdd2f0 2024-07-31T15:03:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.387+0000 7ff2ec593740 10 client.18051 fetch_fsmap finished waiting for FSMap version 62 2024-07-31T15:03:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.387+0000 7ff2ec593740 10 client.18051 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.387+0000 7ff2ec593740 20 client.18051 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.387+0000 7ff2ec593740 1 --2- 172.21.15.67:0/1731031136 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a9dc9a1c0 0x557a9e1304c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.387+0000 7ff2ec593740 4 client.18051 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:03:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.387+0000 7ff2ec593740 1 -- 172.21.15.67:0/1731031136 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a9e01bda0 con 0x557a9dc9a1c0 2024-07-31T15:03:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.387+0000 7ff2de7fc640 1 --2- 172.21.15.67:0/1731031136 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a9dc9a1c0 0x557a9e1304c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.388+0000 7ff2c2ff5640 10 client.18051 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:05.388+0000 7ff2de7fc640 1 --2- 172.21.15.67:0/1731031136 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a9dc9a1c0 0x557a9e1304c0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=629cab5eacfc9236 server_cookie=5141464063d2aa17 in_seq=0 out_seq=0 2024-07-31T15:03:05.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:05 smithi073 ceph-mon[36714]: pgmap v777: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 389 KiB/s wr, 101 op/s 2024-07-31T15:03:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:05 smithi077 ceph-mon[37327]: pgmap v777: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 389 KiB/s wr, 101 op/s 2024-07-31T15:03:05.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:05 smithi067 ceph-mon[32750]: pgmap v777: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 389 KiB/s wr, 101 op/s 2024-07-31T15:03:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.616+0000 7ff2c2ff5640 1 -- 172.21.15.67:0/1731031136 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a9e01bda0 con 0x557a9dc9a1c0 2024-07-31T15:03:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.616+0000 7ff2c2ff5640 10 client.18051 handle_command_reply: tid=0 2024-07-31T15:03:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.616+0000 7ff2ec593740 1 client.18051 shutdown 2024-07-31T15:03:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.617+0000 7ff2ec593740 1 -- 172.21.15.67:0/1731031136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 msgr2=0x557a9e0bcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.617+0000 7ff2ec593740 1 --2- 172.21.15.67:0/1731031136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0bcd00 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000c6a0 tx=0x7ff2d00e45c0 comp rx=0 tx=0).stop 2024-07-31T15:03:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.617+0000 7ff2ec593740 1 -- 172.21.15.67:0/1731031136 shutdown_connections 2024-07-31T15:03:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.617+0000 7ff2ec593740 1 --2- 172.21.15.67:0/1731031136 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557a9dc9a1c0 0x557a9e1304c0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.617+0000 7ff2ec593740 1 --2- 172.21.15.67:0/1731031136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9dd061b0 0x557a9e0b96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.617+0000 7ff2ec593740 1 --2- 172.21.15.67:0/1731031136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a9dce54d0 0x557a9e0b91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.617+0000 7ff2c2ff5640 0 client.18051 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.617+0000 7ff2ec593740 1 --2- 172.21.15.67:0/1731031136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a9dcdd2f0 0x557a9e0bcd00 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.617+0000 7ff2ec593740 1 -- 172.21.15.67:0/1731031136 >> 172.21.15.67:0/1731031136 conn(0x557a9dcfa300 msgr2=0x557a9e0aa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.617+0000 7ff2ec593740 1 -- 172.21.15.67:0/1731031136 shutdown_connections 2024-07-31T15:03:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.617+0000 7ff2ec593740 1 -- 172.21.15.67:0/1731031136 wait complete. 2024-07-31T15:03:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.617+0000 7ff2ec593740 20 client.18051 trim_cache size 0 max 16384 2024-07-31T15:03:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:03:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:03:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:03:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:03:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.618+0000 7ff2e569b640 1 -- 172.21.15.67:0/3860907441 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff2c80b3d70 msgr2=0x7ff2c80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.618+0000 7ff2e569b640 1 --2- 172.21.15.67:0/3860907441 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff2c80b3d70 0x7ff2c80b6280 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8005fd0 tx=0x7ff2d8009500 comp rx=0 tx=0).stop 2024-07-31T15:03:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.618+0000 7ff2e569b640 1 -- 172.21.15.67:0/3860907441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2e0104380 msgr2=0x7ff2e01020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.618+0000 7ff2e569b640 1 --2- 172.21.15.67:0/3860907441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2e0104380 0x7ff2e01020b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000e9f0 tx=0x7ff2d000edf0 comp rx=0 tx=0).stop 2024-07-31T15:03:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.618+0000 7ff2e569b640 1 -- 172.21.15.67:0/3860907441 shutdown_connections 2024-07-31T15:03:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.618+0000 7ff2e569b640 1 --2- 172.21.15.67:0/3860907441 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff2c80b3d70 0x7ff2c80b6280 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.618+0000 7ff2e569b640 1 --2- 172.21.15.67:0/3860907441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2e010d7e0 0x7ff2e0102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.618+0000 7ff2e569b640 1 --2- 172.21.15.67:0/3860907441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2e0104cd0 0x7ff2e01025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.618+0000 7ff2e569b640 1 --2- 172.21.15.67:0/3860907441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2e0104380 0x7ff2e01020b0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.618+0000 7ff2e569b640 1 -- 172.21.15.67:0/3860907441 >> 172.21.15.67:0/3860907441 conn(0x7ff2e00fde30 msgr2=0x7ff2e01072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.618+0000 7ff2e569b640 1 -- 172.21.15.67:0/3860907441 shutdown_connections 2024-07-31T15:03:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:06.618+0000 7ff2e569b640 1 -- 172.21.15.67:0/3860907441 wait complete. 2024-07-31T15:03:07.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:07 smithi073 ceph-mon[36714]: pgmap v778: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 308 KiB/s wr, 94 op/s 2024-07-31T15:03:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:07 smithi077 ceph-mon[37327]: pgmap v778: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 308 KiB/s wr, 94 op/s 2024-07-31T15:03:07.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:07 smithi067 ceph-mon[32750]: pgmap v778: 129 pgs: 129 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 308 KiB/s wr, 94 op/s 2024-07-31T15:03:08.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:08 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:03:08.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:08 smithi073 ceph-mon[36714]: osdmap e108: 12 total, 12 up, 12 in 2024-07-31T15:03:08.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:08 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:03:08.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:08 smithi077 ceph-mon[37327]: osdmap e108: 12 total, 12 up, 12 in 2024-07-31T15:03:08.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:08 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:03:08.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:08 smithi067 ceph-mon[32750]: osdmap e108: 12 total, 12 up, 12 in 2024-07-31T15:03:09.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:09 smithi073 ceph-mon[36714]: pgmap v780: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 995 KiB/s wr, 149 op/s 2024-07-31T15:03:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:09 smithi077 ceph-mon[37327]: pgmap v780: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 995 KiB/s wr, 149 op/s 2024-07-31T15:03:09.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:09 smithi067 ceph-mon[32750]: pgmap v780: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 995 KiB/s wr, 149 op/s 2024-07-31T15:03:11.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:11 smithi073 ceph-mon[36714]: pgmap v781: 129 pgs: 12 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 80 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-07-31T15:03:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:11 smithi077 ceph-mon[37327]: pgmap v781: 129 pgs: 12 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 80 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-07-31T15:03:11.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:11 smithi067 ceph-mon[32750]: pgmap v781: 129 pgs: 12 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 80 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-07-31T15:03:13.106 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:03:13.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:13 smithi073 ceph-mon[36714]: pgmap v782: 129 pgs: 17 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 64 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-07-31T15:03:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:13 smithi077 ceph-mon[37327]: pgmap v782: 129 pgs: 17 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 64 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-07-31T15:03:13.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:13 smithi067 ceph-mon[32750]: pgmap v782: 129 pgs: 17 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 64 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-07-31T15:03:13.651 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:03:15.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:15 smithi073 ceph-mon[36714]: pgmap v783: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.6 MiB/s wr, 232 op/s 2024-07-31T15:03:15.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:15 smithi067 ceph-mon[32750]: pgmap v783: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.6 MiB/s wr, 232 op/s 2024-07-31T15:03:15.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:15 smithi077 ceph-mon[37327]: pgmap v783: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.6 MiB/s wr, 232 op/s 2024-07-31T15:03:15.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.748+0000 7fa0f9b0d640 1 -- 172.21.15.67:0/476570887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f4102b20 msgr2=0x7fa0f4107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:15.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.748+0000 7fa0f9b0d640 1 --2- 172.21.15.67:0/476570887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f4102b20 0x7fa0f4107050 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009890 tx=0x7fa0dc02a380 comp rx=0 tx=0).stop 2024-07-31T15:03:15.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.749+0000 7fa0f9b0d640 1 -- 172.21.15.67:0/476570887 shutdown_connections 2024-07-31T15:03:15.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.749+0000 7fa0f9b0d640 1 --2- 172.21.15.67:0/476570887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0f4107610 0x7fa0f4107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.749+0000 7fa0f9b0d640 1 --2- 172.21.15.67:0/476570887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f4102b20 0x7fa0f4107050 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.749+0000 7fa0f9b0d640 1 --2- 172.21.15.67:0/476570887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0f41021d0 0x7fa0f41025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.749+0000 7fa0f9b0d640 1 -- 172.21.15.67:0/476570887 >> 172.21.15.67:0/476570887 conn(0x7fa0f4079160 msgr2=0x7fa0f4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.750+0000 7fa0f9b0d640 1 -- 172.21.15.67:0/476570887 shutdown_connections 2024-07-31T15:03:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.750+0000 7fa0f9b0d640 1 -- 172.21.15.67:0/476570887 wait complete. 2024-07-31T15:03:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.751+0000 7fa0f9b0d640 1 Processor -- start 2024-07-31T15:03:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.751+0000 7fa0f9b0d640 1 -- start start 2024-07-31T15:03:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.752+0000 7fa0f9b0d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f41021d0 0x7fa0f4073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.752+0000 7fa0f9b0d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0f4102b20 0x7fa0f4074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.752+0000 7fa0f9b0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0f4107610 0x7fa0f410fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.752+0000 7fa0f9b0d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f410aec0 con 0x7fa0f41021d0 2024-07-31T15:03:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.752+0000 7fa0f9b0d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f410ad40 con 0x7fa0f4102b20 2024-07-31T15:03:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.752+0000 7fa0f9b0d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f410b040 con 0x7fa0f4107610 2024-07-31T15:03:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.752+0000 7fa0f37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f41021d0 0x7fa0f4073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.752+0000 7fa0f37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f41021d0 0x7fa0f4073e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45994/0 (socket says 172.21.15.67:45994) 2024-07-31T15:03:15.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.752+0000 7fa0f37fe640 1 -- 172.21.15.67:0/943423387 learned_addr learned my addr 172.21.15.67:0/943423387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:15.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.752+0000 7fa0f37fe640 1 -- 172.21.15.67:0/943423387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0f4107610 msgr2=0x7fa0f410fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:15.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.752+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/943423387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0f4107610 0x7fa0f410fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.752+0000 7fa0f37fe640 1 -- 172.21.15.67:0/943423387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0f4102b20 msgr2=0x7fa0f4074390 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:03:15.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.752+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/943423387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0f4102b20 0x7fa0f4074390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.752+0000 7fa0f37fe640 1 -- 172.21.15.67:0/943423387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f4110490 con 0x7fa0f41021d0 2024-07-31T15:03:15.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.757+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/943423387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f41021d0 0x7fa0f4073e10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400b470 tx=0x7fa0e400b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:15.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.758+0000 7fa0f17fa640 1 -- 172.21.15.67:0/943423387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4015020 con 0x7fa0f41021d0 2024-07-31T15:03:15.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.758+0000 7fa0f17fa640 1 -- 172.21.15.67:0/943423387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e4011d40 con 0x7fa0f41021d0 2024-07-31T15:03:15.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.758+0000 7fa0f9b0d640 1 -- 172.21.15.67:0/943423387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f4110720 con 0x7fa0f41021d0 2024-07-31T15:03:15.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.758+0000 7fa0f9b0d640 1 -- 172.21.15.67:0/943423387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f41acf90 con 0x7fa0f41021d0 2024-07-31T15:03:15.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.760+0000 7fa0f17fa640 1 -- 172.21.15.67:0/943423387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4015020 con 0x7fa0f41021d0 2024-07-31T15:03:15.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.767+0000 7fa10005e740 1 Processor -- start 2024-07-31T15:03:15.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.767+0000 7fa10005e740 1 -- start start 2024-07-31T15:03:15.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.768+0000 7fa10005e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 0x559f7214b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.768+0000 7fa10005e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d7d1b0 0x559f7214ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.768+0000 7fa10005e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d85390 0x559f7214bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.768+0000 7fa10005e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559f720c1b60 con 0x559f71d7d1b0 2024-07-31T15:03:15.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.768+0000 7fa10005e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559f720b1060 con 0x559f71d85390 2024-07-31T15:03:15.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.768+0000 7fa10005e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559f720b50a0 con 0x559f71da6070 2024-07-31T15:03:15.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.769+0000 7fa0f3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 0x559f7214b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:15.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.769+0000 7fa0f37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d7d1b0 0x559f7214ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:15.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.769+0000 7fa0f37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d7d1b0 0x559f7214ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45998/0 (socket says 172.21.15.67:45998) 2024-07-31T15:03:15.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.769+0000 7fa0f37fe640 1 -- 172.21.15.67:0/825568167 learned_addr learned my addr 172.21.15.67:0/825568167 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:15.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.769+0000 7fa0f17fa640 1 -- 172.21.15.67:0/943423387 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fa0e4018020 con 0x7fa0f41021d0 2024-07-31T15:03:15.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.769+0000 7fa0ebfff640 1 --2- 172.21.15.67:0/825568167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d85390 0x559f7214bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:15.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.774+0000 7fa0f17fa640 1 --2- 172.21.15.67:0/943423387 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa0e00b41c0 0x7fa0e00b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.775+0000 7fa0f17fa640 1 -- 172.21.15.67:0/943423387 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9483+0+0 (secure 0 0 0) 0x7fa0e4011690 con 0x7fa0f41021d0 2024-07-31T15:03:15.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.775+0000 7fa0f3fff640 1 --2- 172.21.15.67:0/943423387 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa0e00b41c0 0x7fa0e00b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:15.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.775+0000 7fa0f37fe640 1 -- 172.21.15.67:0/825568167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 msgr2=0x559f7214b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:15.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.775+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/825568167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 0x559f7214b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.775+0000 7fa0f37fe640 1 -- 172.21.15.67:0/825568167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d85390 msgr2=0x559f7214bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:15.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.775+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/825568167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d85390 0x559f7214bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.775+0000 7fa0f37fe640 1 -- 172.21.15.67:0/825568167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f72057b40 con 0x559f71d7d1b0 2024-07-31T15:03:15.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.776+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/825568167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d7d1b0 0x559f7214ba60 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40a9760 tx=0x7fa0e40a61d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd03dcdd1d270632 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:15.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.776+0000 7fa0f3fff640 1 --2- 172.21.15.67:0/943423387 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa0e00b41c0 0x7fa0e00b66d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec01afd0 tx=0x7fa0ec01a5d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:15.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.776+0000 7fa0e9ffb640 1 -- 172.21.15.67:0/825568167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4015070 con 0x559f71d7d1b0 2024-07-31T15:03:15.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.776+0000 7fa0e9ffb640 1 -- 172.21.15.67:0/825568167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e40a5070 con 0x559f71d7d1b0 2024-07-31T15:03:15.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.776+0000 7fa0e9ffb640 1 -- 172.21.15.67:0/825568167 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4015070 con 0x559f71d7d1b0 2024-07-31T15:03:15.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.778+0000 7fa10005e740 1 -- 172.21.15.67:0/825568167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d7d1b0 msgr2=0x559f7214ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:15.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.778+0000 7fa10005e740 1 --2- 172.21.15.67:0/825568167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d7d1b0 0x559f7214ba60 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40a9760 tx=0x7fa0e40a61d0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.778+0000 7fa10005e740 1 -- 172.21.15.67:0/825568167 shutdown_connections 2024-07-31T15:03:15.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.778+0000 7fa10005e740 1 --2- 172.21.15.67:0/825568167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d85390 0x559f7214bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.778+0000 7fa10005e740 1 --2- 172.21.15.67:0/825568167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d7d1b0 0x559f7214ba60 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.778+0000 7fa10005e740 1 --2- 172.21.15.67:0/825568167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 0x559f7214b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.778+0000 7fa10005e740 1 -- 172.21.15.67:0/825568167 >> 172.21.15.67:0/825568167 conn(0x559f71d9a460 msgr2=0x559f72149410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:15.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.778+0000 7fa10005e740 1 -- 172.21.15.67:0/825568167 shutdown_connections 2024-07-31T15:03:15.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.778+0000 7fa10005e740 1 -- 172.21.15.67:0/825568167 wait complete. 2024-07-31T15:03:15.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.779+0000 7fa10005e740 1 -- start start 2024-07-31T15:03:15.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.779+0000 7fa10005e740 1 Processor -- start 2024-07-31T15:03:15.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.780+0000 7fa10005e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d7d1b0 0x559f72157110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.780+0000 7fa10005e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d85390 0x559f72157690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.780+0000 7fa10005e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 0x559f72162ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.780+0000 7fa10005e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559f71f14700 con 0x559f71d7d1b0 2024-07-31T15:03:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.780+0000 7fa10005e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559f72053990 con 0x559f71d85390 2024-07-31T15:03:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.780+0000 7fa10005e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559f72159d00 con 0x559f71da6070 2024-07-31T15:03:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.780+0000 7fa0f3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 0x559f72162ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.780+0000 7fa0f3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 0x559f72162ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:52470/0 (socket says 172.21.15.67:52470) 2024-07-31T15:03:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.780+0000 7fa0f3fff640 1 -- 172.21.15.67:0/2714300748 learned_addr learned my addr 172.21.15.67:0/2714300748 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.781+0000 7fa10005e740 10 client.0 _ll_register_callbacks cb 0x559f7206e4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.781+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/2714300748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d7d1b0 0x559f72157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:15.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.782+0000 7fa0f37fe640 1 -- 172.21.15.67:0/2714300748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 msgr2=0x559f72162ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:15.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.782+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/2714300748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 0x559f72162ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.782+0000 7fa0f37fe640 1 -- 172.21.15.67:0/2714300748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d85390 msgr2=0x559f72157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:15.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.782+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/2714300748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d85390 0x559f72157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.782+0000 7fa0f37fe640 1 -- 172.21.15.67:0/2714300748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f71e11650 con 0x559f71d7d1b0 2024-07-31T15:03:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.782+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/2714300748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d7d1b0 0x559f72157110 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40a9760 tx=0x7fa0e40e0930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd6444ae4e3d797 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.785+0000 7fa0d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.787+0000 7fa0d2ffd640 1 -- 172.21.15.67:0/2714300748 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40c2020 con 0x559f71d7d1b0 2024-07-31T15:03:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.788+0000 7fa0d2ffd640 1 -- 172.21.15.67:0/2714300748 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9483+0+0 (secure 0 0 0) 0x7fa0e40a9ed0 con 0x559f71d7d1b0 2024-07-31T15:03:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.788+0000 7fa10005e740 1 -- 172.21.15.67:0/2714300748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e40a5070 con 0x559f71d7d1b0 2024-07-31T15:03:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.793+0000 7fa0d2ffd640 1 -- 172.21.15.67:0/2714300748 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fa0e4015070 con 0x559f71d7d1b0 2024-07-31T15:03:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.793+0000 7fa10005e740 10 client.18063 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.793+0000 7fa10005e740 1 -- 172.21.15.67:0/2714300748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x559f72165d60 con 0x559f71d7d1b0 2024-07-31T15:03:15.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.793+0000 7fa0d2ffd640 1 -- 172.21.15.67:0/2714300748 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e40a5070 con 0x559f71d7d1b0 2024-07-31T15:03:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.794+0000 7fa0d2ffd640 1 -- 172.21.15.67:0/2714300748 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fa0e40b25f0 con 0x559f71d7d1b0 2024-07-31T15:03:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.794+0000 7fa10005e740 10 client.18063 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.794+0000 7fa10005e740 10 client.18063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.794+0000 7fa10005e740 20 client.18063 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.794+0000 7fa10005e740 1 --2- 172.21.15.67:0/2714300748 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559f71d649f0 0x559f721690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.794+0000 7fa10005e740 4 client.18063 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:03:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.794+0000 7fa10005e740 1 -- 172.21.15.67:0/2714300748 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f720c3ab0 con 0x559f71d649f0 2024-07-31T15:03:15.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.798+0000 7fa0ebfff640 1 --2- 172.21.15.67:0/2714300748 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559f71d649f0 0x559f721690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:15.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.802+0000 7fa0d2ffd640 10 client.18063 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:15.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.802+0000 7fa0ebfff640 1 --2- 172.21.15.67:0/2714300748 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559f71d649f0 0x559f721690c0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=702f021efbae3547 server_cookie=3463f411caca7c3e in_seq=0 out_seq=0 2024-07-31T15:03:15.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.816+0000 7fa0d2ffd640 1 -- 172.21.15.67:0/2714300748 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f720c3ab0 con 0x559f71d649f0 2024-07-31T15:03:15.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.816+0000 7fa0d2ffd640 10 client.18063 handle_command_reply: tid=0 2024-07-31T15:03:15.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.816+0000 7fa10005e740 1 client.18063 shutdown 2024-07-31T15:03:15.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.817+0000 7fa10005e740 1 -- 172.21.15.67:0/2714300748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d7d1b0 msgr2=0x559f72157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:15.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.817+0000 7fa10005e740 1 --2- 172.21.15.67:0/2714300748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d7d1b0 0x559f72157110 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40a9760 tx=0x7fa0e40e0930 comp rx=0 tx=0).stop 2024-07-31T15:03:15.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.820+0000 7fa10005e740 1 -- 172.21.15.67:0/2714300748 shutdown_connections 2024-07-31T15:03:15.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.820+0000 7fa10005e740 1 --2- 172.21.15.67:0/2714300748 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559f71d649f0 0x559f721690c0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.820+0000 7fa10005e740 1 --2- 172.21.15.67:0/2714300748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 0x559f72162ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.820+0000 7fa10005e740 1 --2- 172.21.15.67:0/2714300748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d85390 0x559f72157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.820+0000 7fa10005e740 1 --2- 172.21.15.67:0/2714300748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d7d1b0 0x559f72157110 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.820+0000 7fa10005e740 1 -- 172.21.15.67:0/2714300748 >> 172.21.15.67:0/2714300748 conn(0x559f71d9a460 msgr2=0x559f7214e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:15.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.820+0000 7fa0d2ffd640 0 client.18063 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:15.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.822+0000 7fa10005e740 1 -- 172.21.15.67:0/2714300748 shutdown_connections 2024-07-31T15:03:15.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.822+0000 7fa10005e740 1 -- 172.21.15.67:0/2714300748 wait complete. 2024-07-31T15:03:15.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.822+0000 7fa10005e740 20 client.18063 trim_cache size 0 max 16384 2024-07-31T15:03:15.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.855+0000 7fa10005e740 1 Processor -- start 2024-07-31T15:03:15.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.856+0000 7fa10005e740 1 -- start start 2024-07-31T15:03:15.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.856+0000 7fa10005e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d7d1b0 0x559f72162f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.856+0000 7fa10005e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71d85390 0x559f72158b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.856+0000 7fa10005e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71da6070 0x559f721505b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.856+0000 7fa0f37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d7d1b0 0x559f72162f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:15.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.856+0000 7fa0f37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d7d1b0 0x559f72162f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:36822/0 (socket says 172.21.15.67:36822) 2024-07-31T15:03:15.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.856+0000 7fa0f37fe640 1 -- 172.21.15.67:0/4112648402 learned_addr learned my addr 172.21.15.67:0/4112648402 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:15.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.857+0000 7fa10005e740 1 -- 172.21.15.67:0/4112648402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f41b7070 con 0x559f71da6070 2024-07-31T15:03:15.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.857+0000 7fa10005e740 1 -- 172.21.15.67:0/4112648402 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f41b8bb0 con 0x559f71d7d1b0 2024-07-31T15:03:15.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.857+0000 7fa10005e740 1 -- 172.21.15.67:0/4112648402 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f41b7740 con 0x559f71d85390 2024-07-31T15:03:15.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.857+0000 7fa0f37fe640 1 -- 172.21.15.67:0/4112648402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71d85390 msgr2=0x559f72158b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:03:15.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.857+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/4112648402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71d85390 0x559f72158b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.857+0000 7fa0f37fe640 1 -- 172.21.15.67:0/4112648402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71da6070 msgr2=0x559f721505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:15.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.857+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/4112648402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71da6070 0x559f721505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.857+0000 7fa0f37fe640 1 -- 172.21.15.67:0/4112648402 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f72155410 con 0x559f71d7d1b0 2024-07-31T15:03:15.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.858+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/4112648402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d7d1b0 0x559f72162f60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40a6cd0 tx=0x7fa0e400bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=716d1c419f016c06 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:15.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.858+0000 7fa0e97fa640 1 -- 172.21.15.67:0/4112648402 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40c2070 con 0x559f71d7d1b0 2024-07-31T15:03:15.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.858+0000 7fa0e97fa640 1 -- 172.21.15.67:0/4112648402 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e40a5040 con 0x559f71d7d1b0 2024-07-31T15:03:15.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.858+0000 7fa0e97fa640 1 -- 172.21.15.67:0/4112648402 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40a9a60 con 0x559f71d7d1b0 2024-07-31T15:03:15.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.860+0000 7fa10005e740 1 -- 172.21.15.67:0/4112648402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d7d1b0 msgr2=0x559f72162f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:15.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.860+0000 7fa10005e740 1 --2- 172.21.15.67:0/4112648402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d7d1b0 0x559f72162f60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40a6cd0 tx=0x7fa0e400bf60 comp rx=0 tx=0).stop 2024-07-31T15:03:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.860+0000 7fa10005e740 1 -- 172.21.15.67:0/4112648402 shutdown_connections 2024-07-31T15:03:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.860+0000 7fa10005e740 1 --2- 172.21.15.67:0/4112648402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71da6070 0x559f721505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.860+0000 7fa10005e740 1 --2- 172.21.15.67:0/4112648402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71d85390 0x559f72158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.860+0000 7fa10005e740 1 --2- 172.21.15.67:0/4112648402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d7d1b0 0x559f72162f60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.860+0000 7fa10005e740 1 -- 172.21.15.67:0/4112648402 >> 172.21.15.67:0/4112648402 conn(0x559f71d9a460 msgr2=0x559f72158430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.860+0000 7fa10005e740 1 -- 172.21.15.67:0/4112648402 shutdown_connections 2024-07-31T15:03:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.861+0000 7fa10005e740 1 -- 172.21.15.67:0/4112648402 wait complete. 2024-07-31T15:03:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.861+0000 7fa10005e740 1 -- start start 2024-07-31T15:03:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.862+0000 7fa10005e740 1 Processor -- start 2024-07-31T15:03:15.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.862+0000 7fa10005e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d7d1b0 0x559f72167a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.862+0000 7fa10005e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d85390 0x559f72168000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.862+0000 7fa10005e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 0x559f7216be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.862+0000 7fa10005e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f41b84e0 con 0x559f71d85390 2024-07-31T15:03:15.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.862+0000 7fa10005e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f41b7e10 con 0x559f71d7d1b0 2024-07-31T15:03:15.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.862+0000 7fa10005e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f41b5070 con 0x559f71da6070 2024-07-31T15:03:15.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.862+0000 7fa10005e740 10 client.0 _ll_register_callbacks cb 0x7fa0ec01aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:15.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.862+0000 7fa0f37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d7d1b0 0x559f72167a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.863+0000 7fa0f37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d7d1b0 0x559f72167a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:36828/0 (socket says 172.21.15.67:36828) 2024-07-31T15:03:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.863+0000 7fa0f37fe640 1 -- 172.21.15.67:0/3915412354 learned_addr learned my addr 172.21.15.67:0/3915412354 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.863+0000 7fa0f3fff640 1 --2- 172.21.15.67:0/3915412354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 0x559f7216be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.863+0000 7fa0ebfff640 1 --2- 172.21.15.67:0/3915412354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d85390 0x559f72168000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.863+0000 7fa0f37fe640 1 -- 172.21.15.67:0/3915412354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 msgr2=0x559f7216be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.863+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/3915412354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 0x559f7216be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.863+0000 7fa0f37fe640 1 -- 172.21.15.67:0/3915412354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d85390 msgr2=0x559f72168000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.863+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/3915412354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d85390 0x559f72168000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.863+0000 7fa0f37fe640 1 -- 172.21.15.67:0/3915412354 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7216c4e0 con 0x559f71d7d1b0 2024-07-31T15:03:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.863+0000 7fa0f37fe640 1 --2- 172.21.15.67:0/3915412354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d7d1b0 0x559f72167a60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400bfd0 tx=0x7fa0e4005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79bdee47e1d77def server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.863+0000 7fa0d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:03:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.864+0000 7fa0d2ffd640 1 -- 172.21.15.67:0/3915412354 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4088060 con 0x559f71d7d1b0 2024-07-31T15:03:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.864+0000 7fa10005e740 1 -- 172.21.15.67:0/3915412354 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e40a5040 con 0x559f71d7d1b0 2024-07-31T15:03:15.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.864+0000 7fa0d2ffd640 1 -- 172.21.15.67:0/3915412354 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9483+0+0 (secure 0 0 0) 0x7fa0e4010bf0 con 0x559f71d7d1b0 2024-07-31T15:03:15.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.864+0000 7fa0d2ffd640 1 -- 172.21.15.67:0/3915412354 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fa0e40c2070 con 0x559f71d7d1b0 2024-07-31T15:03:15.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.865+0000 7fa10005e740 10 client.27620 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:15.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.865+0000 7fa10005e740 1 -- 172.21.15.67:0/3915412354 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x559f7216c4e0 con 0x559f71d7d1b0 2024-07-31T15:03:15.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.865+0000 7fa0d2ffd640 1 -- 172.21.15.67:0/3915412354 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e4011d40 con 0x559f71d7d1b0 2024-07-31T15:03:15.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.866+0000 7fa0d2ffd640 1 -- 172.21.15.67:0/3915412354 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fa0e4011890 con 0x559f71d7d1b0 2024-07-31T15:03:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.866+0000 7fa10005e740 10 client.27620 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.866+0000 7fa10005e740 10 client.27620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.866+0000 7fa10005e740 20 client.27620 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.866+0000 7fa10005e740 1 --2- 172.21.15.67:0/3915412354 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559f71d39da0 0x559f721d32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.866+0000 7fa10005e740 4 client.27620 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:03:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.866+0000 7fa10005e740 1 -- 172.21.15.67:0/3915412354 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa0e40a5040 con 0x559f71d39da0 2024-07-31T15:03:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.867+0000 7fa0ebfff640 1 --2- 172.21.15.67:0/3915412354 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559f71d39da0 0x559f721d32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:15.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.878+0000 7fa0ebfff640 1 --2- 172.21.15.67:0/3915412354 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559f71d39da0 0x559f721d32f0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7fe0caaefee9879 server_cookie=9e8e6fd2b867ba2d in_seq=0 out_seq=0 2024-07-31T15:03:15.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.878+0000 7fa0d2ffd640 10 client.27620 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:15.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.952+0000 7fa0d2ffd640 1 -- 172.21.15.67:0/3915412354 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fa0e40a5040 con 0x559f71d39da0 2024-07-31T15:03:15.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.952+0000 7fa0d2ffd640 10 client.27620 handle_command_reply: tid=0 2024-07-31T15:03:15.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.952+0000 7fa10005e740 1 client.27620 shutdown 2024-07-31T15:03:15.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.952+0000 7fa10005e740 1 -- 172.21.15.67:0/3915412354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d7d1b0 msgr2=0x559f72167a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:15.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.952+0000 7fa10005e740 1 --2- 172.21.15.67:0/3915412354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d7d1b0 0x559f72167a60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400bfd0 tx=0x7fa0e4005bc0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.952+0000 7fa10005e740 1 -- 172.21.15.67:0/3915412354 shutdown_connections 2024-07-31T15:03:15.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.952+0000 7fa10005e740 1 --2- 172.21.15.67:0/3915412354 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559f71d39da0 0x559f721d32f0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.952+0000 7fa10005e740 1 --2- 172.21.15.67:0/3915412354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f71da6070 0x559f7216be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.952+0000 7fa10005e740 1 --2- 172.21.15.67:0/3915412354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f71d85390 0x559f72168000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.952+0000 7fa10005e740 1 --2- 172.21.15.67:0/3915412354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f71d7d1b0 0x559f72167a60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.952+0000 7fa0d2ffd640 0 client.27620 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:15.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.952+0000 7fa10005e740 1 -- 172.21.15.67:0/3915412354 >> 172.21.15.67:0/3915412354 conn(0x559f71d9a460 msgr2=0x559f7214cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:15.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.953+0000 7fa10005e740 1 -- 172.21.15.67:0/3915412354 shutdown_connections 2024-07-31T15:03:15.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.953+0000 7fa10005e740 1 -- 172.21.15.67:0/3915412354 wait complete. 2024-07-31T15:03:15.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.953+0000 7fa10005e740 20 client.27620 trim_cache size 0 max 16384 2024-07-31T15:03:15.956 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:03:15.956 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T15:03:15.956 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T15:03:15.956 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:03:15.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.954+0000 7fa0f9b0d640 1 -- 172.21.15.67:0/943423387 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa0e00b41c0 msgr2=0x7fa0e00b66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:15.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.954+0000 7fa0f9b0d640 1 --2- 172.21.15.67:0/943423387 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa0e00b41c0 0x7fa0e00b66d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec01afd0 tx=0x7fa0ec01a5d0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.954+0000 7fa0f9b0d640 1 -- 172.21.15.67:0/943423387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f41021d0 msgr2=0x7fa0f4073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:15.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.954+0000 7fa0f9b0d640 1 --2- 172.21.15.67:0/943423387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f41021d0 0x7fa0f4073e10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400b470 tx=0x7fa0e400b930 comp rx=0 tx=0).stop 2024-07-31T15:03:15.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.954+0000 7fa0f9b0d640 1 -- 172.21.15.67:0/943423387 shutdown_connections 2024-07-31T15:03:15.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.954+0000 7fa0f9b0d640 1 --2- 172.21.15.67:0/943423387 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa0e00b41c0 0x7fa0e00b66d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.954+0000 7fa0f9b0d640 1 --2- 172.21.15.67:0/943423387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0f4107610 0x7fa0f410fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.954+0000 7fa0f9b0d640 1 --2- 172.21.15.67:0/943423387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0f4102b20 0x7fa0f4074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.954+0000 7fa0f9b0d640 1 --2- 172.21.15.67:0/943423387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f41021d0 0x7fa0f4073e10 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.954+0000 7fa0f9b0d640 1 -- 172.21.15.67:0/943423387 >> 172.21.15.67:0/943423387 conn(0x7fa0f4079160 msgr2=0x7fa0f4105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.954+0000 7fa0f9b0d640 1 -- 172.21.15.67:0/943423387 shutdown_connections 2024-07-31T15:03:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:15.954+0000 7fa0f9b0d640 1 -- 172.21.15.67:0/943423387 wait complete. 2024-07-31T15:03:17.088 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T15:03:17.088 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:03:17.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:17 smithi067 ceph-mon[32750]: pgmap v784: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-07-31T15:03:17.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:17 smithi073 ceph-mon[36714]: pgmap v784: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-07-31T15:03:17.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:17 smithi077 ceph-mon[37327]: pgmap v784: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-07-31T15:03:17.530 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:03:19.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:19 smithi077 ceph-mon[37327]: pgmap v785: 129 pgs: 18 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 70 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.0 MiB/s wr, 148 op/s 2024-07-31T15:03:19.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:19 smithi067 ceph-mon[32750]: pgmap v785: 129 pgs: 18 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 70 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.0 MiB/s wr, 148 op/s 2024-07-31T15:03:19.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:19 smithi073 ceph-mon[36714]: pgmap v785: 129 pgs: 18 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 70 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.0 MiB/s wr, 148 op/s 2024-07-31T15:03:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.782+0000 7f655477c640 1 -- 172.21.15.67:0/2509651219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f654c10a330 msgr2=0x7f654c10a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.782+0000 7f655477c640 1 --2- 172.21.15.67:0/2509651219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f654c10a330 0x7f654c10a740 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6548009900 tx=0x7f654802a480 comp rx=0 tx=0).stop 2024-07-31T15:03:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.782+0000 7f655477c640 1 -- 172.21.15.67:0/2509651219 shutdown_connections 2024-07-31T15:03:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.782+0000 7f655477c640 1 --2- 172.21.15.67:0/2509651219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f654c11c5d0 0x7f654c11eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.782+0000 7f655477c640 1 --2- 172.21.15.67:0/2509651219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f654c10a330 0x7f654c10a740 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.782+0000 7f655477c640 1 --2- 172.21.15.67:0/2509651219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f654c1118d0 0x7f654c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.782+0000 7f655477c640 1 -- 172.21.15.67:0/2509651219 >> 172.21.15.67:0/2509651219 conn(0x7f654c0707c0 msgr2=0x7f654c070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.787+0000 7f655477c640 1 -- 172.21.15.67:0/2509651219 shutdown_connections 2024-07-31T15:03:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.788+0000 7f655477c640 1 -- 172.21.15.67:0/2509651219 wait complete. 2024-07-31T15:03:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.788+0000 7f655477c640 1 Processor -- start 2024-07-31T15:03:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.788+0000 7f655477c640 1 -- start start 2024-07-31T15:03:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.788+0000 7f655477c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f654c10a330 0x7f654c11beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.788+0000 7f655477c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f654c1118d0 0x7f654c116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.788+0000 7f655477c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f654c11c5d0 0x7f654c116730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.788+0000 7f655477c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f654c1126b0 con 0x7f654c11c5d0 2024-07-31T15:03:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.788+0000 7f655477c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f654c112530 con 0x7f654c1118d0 2024-07-31T15:03:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.788+0000 7f655477c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f654c112830 con 0x7f654c10a330 2024-07-31T15:03:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.788+0000 7f6551d17640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f654c1118d0 0x7f654c116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.788+0000 7f6551d17640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f654c1118d0 0x7f654c116160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:36846/0 (socket says 172.21.15.67:36846) 2024-07-31T15:03:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.788+0000 7f6551d17640 1 -- 172.21.15.67:0/898597890 learned_addr learned my addr 172.21.15.67:0/898597890 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.788+0000 7f6552d19640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f654c11c5d0 0x7f654c116730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.788+0000 7f6552518640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f654c10a330 0x7f654c11beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.794+0000 7f6552d19640 1 -- 172.21.15.67:0/898597890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f654c10a330 msgr2=0x7f654c11beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.794+0000 7f6552d19640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f654c10a330 0x7f654c11beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.794+0000 7f6552d19640 1 -- 172.21.15.67:0/898597890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f654c1118d0 msgr2=0x7f654c116160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.794+0000 7f6552d19640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f654c1118d0 0x7f654c116160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.794+0000 7f6552d19640 1 -- 172.21.15.67:0/898597890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654c116f50 con 0x7f654c11c5d0 2024-07-31T15:03:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.794+0000 7f6552518640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f654c10a330 0x7f654c11beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:03:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.794+0000 7f6552d19640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f654c11c5d0 0x7f654c116730 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f653c00e970 tx=0x7f653c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.794+0000 7f65437fe640 1 -- 172.21.15.67:0/898597890 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c00ccc0 con 0x7f654c11c5d0 2024-07-31T15:03:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.794+0000 7f655477c640 1 -- 172.21.15.67:0/898597890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654c1bd7c0 con 0x7f654c11c5d0 2024-07-31T15:03:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.794+0000 7f655477c640 1 -- 172.21.15.67:0/898597890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f654c1bdce0 con 0x7f654c11c5d0 2024-07-31T15:03:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.794+0000 7f65437fe640 1 -- 172.21.15.67:0/898597890 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f653c005020 con 0x7f654c11c5d0 2024-07-31T15:03:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.794+0000 7f65437fe640 1 -- 172.21.15.67:0/898597890 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c004a00 con 0x7f654c11c5d0 2024-07-31T15:03:19.802 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.801+0000 7f65437fe640 1 -- 172.21.15.67:0/898597890 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f653c004ba0 con 0x7f654c11c5d0 2024-07-31T15:03:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.801+0000 7f65437fe640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65380b3f50 0x7f65380b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.801+0000 7f65437fe640 1 -- 172.21.15.67:0/898597890 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9483+0+0 (secure 0 0 0) 0x7f653c0e4100 con 0x7f654c11c5d0 2024-07-31T15:03:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.801+0000 7f655b67b740 1 Processor -- start 2024-07-31T15:03:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.801+0000 7f655b67b740 1 -- start start 2024-07-31T15:03:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.801+0000 7f655b67b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1c61b0 0x5588ae568ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f655b67b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae19d2f0 0x5588ae569220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f655b67b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae1a54d0 0x5588ae569760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f655b67b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5588ae4de3b0 con 0x5588ae1c61b0 2024-07-31T15:03:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f655b67b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5588ae4cd8b0 con 0x5588ae19d2f0 2024-07-31T15:03:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f655b67b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5588ae4d18f0 con 0x5588ae1a54d0 2024-07-31T15:03:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f6552518640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1c61b0 0x5588ae568ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f6552518640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65380b3f50 0x7f65380b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f6552d19640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae1a54d0 0x5588ae569760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f6552d19640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae1a54d0 0x5588ae569760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:52520/0 (socket says 172.21.15.67:52520) 2024-07-31T15:03:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f6552d19640 1 -- 172.21.15.67:0/3072692114 learned_addr learned my addr 172.21.15.67:0/3072692114 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f6551d17640 1 --2- 172.21.15.67:0/3072692114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae19d2f0 0x5588ae569220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f6551d17640 1 -- 172.21.15.67:0/3072692114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae1a54d0 msgr2=0x5588ae569760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f6551d17640 1 --2- 172.21.15.67:0/3072692114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae1a54d0 0x5588ae569760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f6551d17640 1 -- 172.21.15.67:0/3072692114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1c61b0 msgr2=0x5588ae568ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f6551d17640 1 --2- 172.21.15.67:0/3072692114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1c61b0 0x5588ae568ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f6551d17640 1 -- 172.21.15.67:0/3072692114 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ae4749a0 con 0x5588ae19d2f0 2024-07-31T15:03:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f6551d17640 1 --2- 172.21.15.67:0/3072692114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae19d2f0 0x5588ae569220 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6548005ab0 tx=0x7f6548004380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bbd020d7cb14792 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.807+0000 7f6552518640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65380b3f50 0x7f65380b6460 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f654c11bbf0 tx=0x7f65440097d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.812+0000 7f6540ff9640 1 -- 172.21.15.67:0/3072692114 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548040070 con 0x5588ae19d2f0 2024-07-31T15:03:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.812+0000 7f6540ff9640 1 -- 172.21.15.67:0/3072692114 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65480054e0 con 0x5588ae19d2f0 2024-07-31T15:03:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.812+0000 7f6540ff9640 1 -- 172.21.15.67:0/3072692114 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548035040 con 0x5588ae19d2f0 2024-07-31T15:03:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.812+0000 7f655b67b740 1 -- 172.21.15.67:0/3072692114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae19d2f0 msgr2=0x5588ae569220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.812+0000 7f655b67b740 1 --2- 172.21.15.67:0/3072692114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae19d2f0 0x5588ae569220 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6548005ab0 tx=0x7f6548004380 comp rx=0 tx=0).stop 2024-07-31T15:03:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.812+0000 7f655b67b740 1 -- 172.21.15.67:0/3072692114 shutdown_connections 2024-07-31T15:03:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.812+0000 7f655b67b740 1 --2- 172.21.15.67:0/3072692114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae1a54d0 0x5588ae569760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.812+0000 7f655b67b740 1 --2- 172.21.15.67:0/3072692114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae19d2f0 0x5588ae569220 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.812+0000 7f655b67b740 1 --2- 172.21.15.67:0/3072692114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1c61b0 0x5588ae568ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.812+0000 7f655b67b740 1 -- 172.21.15.67:0/3072692114 >> 172.21.15.67:0/3072692114 conn(0x5588ae1ba300 msgr2=0x5588ae566bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.812+0000 7f655b67b740 1 -- 172.21.15.67:0/3072692114 shutdown_connections 2024-07-31T15:03:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.812+0000 7f655b67b740 1 -- 172.21.15.67:0/3072692114 wait complete. 2024-07-31T15:03:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f655b67b740 1 -- start start 2024-07-31T15:03:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f655b67b740 1 Processor -- start 2024-07-31T15:03:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f655b67b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae19d2f0 0x5588ae5748b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f655b67b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae1a54d0 0x5588ae574df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f655b67b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1c61b0 0x5588ae580750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f655b67b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5588ae407ab0 con 0x5588ae1c61b0 2024-07-31T15:03:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f655b67b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5588ae455cf0 con 0x5588ae1a54d0 2024-07-31T15:03:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f6552518640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae19d2f0 0x5588ae5748b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f6552518640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae19d2f0 0x5588ae5748b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:52528/0 (socket says 172.21.15.67:52528) 2024-07-31T15:03:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f6552518640 1 -- 172.21.15.67:0/3859746896 learned_addr learned my addr 172.21.15.67:0/3859746896 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f655b67b740 1 -- 172.21.15.67:0/3859746896 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5588ae577400 con 0x5588ae19d2f0 2024-07-31T15:03:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f6551d17640 1 --2- 172.21.15.67:0/3859746896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae1a54d0 0x5588ae574df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f655b67b740 10 client.0 _ll_register_callbacks cb 0x5588ae4c79f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f6552518640 1 -- 172.21.15.67:0/3859746896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae1a54d0 msgr2=0x5588ae574df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f6552518640 1 --2- 172.21.15.67:0/3859746896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae1a54d0 0x5588ae574df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f6552518640 1 -- 172.21.15.67:0/3859746896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1c61b0 msgr2=0x5588ae580750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f6552518640 1 --2- 172.21.15.67:0/3859746896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1c61b0 0x5588ae580750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f6552518640 1 -- 172.21.15.67:0/3859746896 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ae231790 con 0x5588ae19d2f0 2024-07-31T15:03:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f6552518640 1 --2- 172.21.15.67:0/3859746896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae19d2f0 0x5588ae5748b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6544009e20 tx=0x7f654401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6526cd821896ac4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f6535ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:03:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f6535ffb640 1 -- 172.21.15.67:0/3859746896 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544024070 con 0x5588ae19d2f0 2024-07-31T15:03:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.817+0000 7f655b67b740 1 -- 172.21.15.67:0/3859746896 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65480054e0 con 0x5588ae19d2f0 2024-07-31T15:03:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.820+0000 7f6535ffb640 1 -- 172.21.15.67:0/3859746896 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9483+0+0 (secure 0 0 0) 0x7f65440044c0 con 0x5588ae19d2f0 2024-07-31T15:03:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.821+0000 7f6535ffb640 1 -- 172.21.15.67:0/3859746896 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f6544004700 con 0x5588ae19d2f0 2024-07-31T15:03:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.821+0000 7f655b67b740 10 client.27589 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.821+0000 7f655b67b740 1 -- 172.21.15.67:0/3859746896 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5588ae2aa620 con 0x5588ae19d2f0 2024-07-31T15:03:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.821+0000 7f6535ffb640 1 -- 172.21.15.67:0/3859746896 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65480054e0 con 0x5588ae19d2f0 2024-07-31T15:03:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.821+0000 7f6535ffb640 1 -- 172.21.15.67:0/3859746896 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f654401f050 con 0x5588ae19d2f0 2024-07-31T15:03:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.821+0000 7f655b67b740 10 client.27589 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.821+0000 7f655b67b740 10 client.27589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.821+0000 7f655b67b740 20 client.27589 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.821+0000 7f655b67b740 1 --2- 172.21.15.67:0/3859746896 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5588ae184d80 0x5588ae586a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.822+0000 7f655b67b740 4 client.27589 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:03:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.822+0000 7f655b67b740 1 -- 172.21.15.67:0/3859746896 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588ae4e0300 con 0x5588ae184d80 2024-07-31T15:03:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.822+0000 7f6551d17640 1 --2- 172.21.15.67:0/3859746896 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5588ae184d80 0x5588ae586a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.822+0000 7f6535ffb640 10 client.27589 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:19.823+0000 7f6551d17640 1 --2- 172.21.15.67:0/3859746896 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5588ae184d80 0x5588ae586a20 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e3354cc1f4dcec server_cookie=3404c472f9e85ae6 in_seq=0 out_seq=0 2024-07-31T15:03:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.043+0000 7f6535ffb640 1 -- 172.21.15.67:0/3859746896 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588ae4e0300 con 0x5588ae184d80 2024-07-31T15:03:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.043+0000 7f6535ffb640 10 client.27589 handle_command_reply: tid=0 2024-07-31T15:03:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.043+0000 7f655b67b740 1 client.27589 shutdown 2024-07-31T15:03:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.043+0000 7f655b67b740 1 -- 172.21.15.67:0/3859746896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae19d2f0 msgr2=0x5588ae5748b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.043+0000 7f655b67b740 1 --2- 172.21.15.67:0/3859746896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae19d2f0 0x5588ae5748b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6544009e20 tx=0x7f654401c000 comp rx=0 tx=0).stop 2024-07-31T15:03:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.043+0000 7f655b67b740 1 -- 172.21.15.67:0/3859746896 shutdown_connections 2024-07-31T15:03:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.043+0000 7f655b67b740 1 --2- 172.21.15.67:0/3859746896 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5588ae184d80 0x5588ae586a20 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.043+0000 7f655b67b740 1 --2- 172.21.15.67:0/3859746896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1c61b0 0x5588ae580750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.043+0000 7f6535ffb640 0 client.27589 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.044+0000 7f655b67b740 1 --2- 172.21.15.67:0/3859746896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae1a54d0 0x5588ae574df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.044+0000 7f655b67b740 1 --2- 172.21.15.67:0/3859746896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae19d2f0 0x5588ae5748b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.044+0000 7f655b67b740 1 -- 172.21.15.67:0/3859746896 >> 172.21.15.67:0/3859746896 conn(0x5588ae1ba300 msgr2=0x5588ae569d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.044+0000 7f655b67b740 1 -- 172.21.15.67:0/3859746896 shutdown_connections 2024-07-31T15:03:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.044+0000 7f655b67b740 1 -- 172.21.15.67:0/3859746896 wait complete. 2024-07-31T15:03:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.044+0000 7f655b67b740 20 client.27589 trim_cache size 0 max 16384 2024-07-31T15:03:20.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.060+0000 7fdf5d8a5640 1 -- 172.21.15.67:0/2520412755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5811c5d0 msgr2=0x7fdf5811e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:20.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.060+0000 7fdf5d8a5640 1 --2- 172.21.15.67:0/2520412755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5811c5d0 0x7fdf5811e9f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009ca0 tx=0x7fdf5002a980 comp rx=0 tx=0).stop 2024-07-31T15:03:20.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.061+0000 7fdf5d8a5640 1 -- 172.21.15.67:0/2520412755 shutdown_connections 2024-07-31T15:03:20.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.061+0000 7fdf5d8a5640 1 --2- 172.21.15.67:0/2520412755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5811c5d0 0x7fdf5811e9f0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.061+0000 7fdf5d8a5640 1 --2- 172.21.15.67:0/2520412755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf5810a330 0x7fdf5810a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.061+0000 7fdf5d8a5640 1 --2- 172.21.15.67:0/2520412755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf58111790 0x7fdf58109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.061+0000 7fdf5d8a5640 1 -- 172.21.15.67:0/2520412755 >> 172.21.15.67:0/2520412755 conn(0x7fdf5806e420 msgr2=0x7fdf5806e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:20.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.064+0000 7fdf5d8a5640 1 -- 172.21.15.67:0/2520412755 shutdown_connections 2024-07-31T15:03:20.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.065+0000 7fdf5d8a5640 1 -- 172.21.15.67:0/2520412755 wait complete. 2024-07-31T15:03:20.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.066+0000 7fdf5d8a5640 1 Processor -- start 2024-07-31T15:03:20.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.067+0000 7fdf5d8a5640 1 -- start start 2024-07-31T15:03:20.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.067+0000 7fdf5d8a5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf5810a330 0x7fdf58119f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:20.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.067+0000 7fdf5d8a5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf58111790 0x7fdf581131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:20.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.068+0000 7fdf5d8a5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5811c5d0 0x7fdf58113870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:20.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.068+0000 7fdf57fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf58111790 0x7fdf581131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:20.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.068+0000 7fdf57fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf58111790 0x7fdf581131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:52550/0 (socket says 172.21.15.67:52550) 2024-07-31T15:03:20.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.068+0000 7fdf57fff640 1 -- 172.21.15.67:0/2684769291 learned_addr learned my addr 172.21.15.67:0/2684769291 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:20.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.068+0000 7fdf5d8a5640 1 -- 172.21.15.67:0/2684769291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdf58122210 con 0x7fdf5811c5d0 2024-07-31T15:03:20.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.068+0000 7fdf5c8a3640 1 --2- 172.21.15.67:0/2684769291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf5810a330 0x7fdf58119f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:20.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.068+0000 7fdf5d8a5640 1 -- 172.21.15.67:0/2684769291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdf58122090 con 0x7fdf5810a330 2024-07-31T15:03:20.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.068+0000 7fdf5d8a5640 1 -- 172.21.15.67:0/2684769291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdf58122390 con 0x7fdf58111790 2024-07-31T15:03:20.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.070+0000 7fdf5d0a4640 1 --2- 172.21.15.67:0/2684769291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5811c5d0 0x7fdf58113870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:20.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.070+0000 7fdf57fff640 1 -- 172.21.15.67:0/2684769291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf5810a330 msgr2=0x7fdf58119f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:20.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.070+0000 7fdf57fff640 1 --2- 172.21.15.67:0/2684769291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf5810a330 0x7fdf58119f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.070+0000 7fdf57fff640 1 -- 172.21.15.67:0/2684769291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5811c5d0 msgr2=0x7fdf58113870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:20.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.070+0000 7fdf57fff640 1 --2- 172.21.15.67:0/2684769291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5811c5d0 0x7fdf58113870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.070+0000 7fdf57fff640 1 -- 172.21.15.67:0/2684769291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf58118b90 con 0x7fdf58111790 2024-07-31T15:03:20.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.072+0000 7fdf57fff640 1 --2- 172.21.15.67:0/2684769291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf58111790 0x7fdf581131f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800b450 tx=0x7fdf4800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:20.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.074+0000 7fdf55ffb640 1 -- 172.21.15.67:0/2684769291 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf48015020 con 0x7fdf58111790 2024-07-31T15:03:20.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.074+0000 7fdf55ffb640 1 -- 172.21.15.67:0/2684769291 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf48011d40 con 0x7fdf58111790 2024-07-31T15:03:20.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.074+0000 7fdf55ffb640 1 -- 172.21.15.67:0/2684769291 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4800f340 con 0x7fdf58111790 2024-07-31T15:03:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.076+0000 7f655b67b740 1 Processor -- start 2024-07-31T15:03:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.076+0000 7f655b67b740 1 -- start start 2024-07-31T15:03:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.076+0000 7f655b67b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae184d80 0x5588ae56a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.076+0000 7f655b67b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae19d2f0 0x5588ae56edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.076+0000 7f655b67b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae1a54d0 0x5588ae5664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.076+0000 7f655b67b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f653000ff80 con 0x5588ae19d2f0 2024-07-31T15:03:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.076+0000 7f655b67b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6530011ac0 con 0x5588ae184d80 2024-07-31T15:03:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.076+0000 7f655b67b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6530010650 con 0x5588ae1a54d0 2024-07-31T15:03:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.076+0000 7f6552d19640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae1a54d0 0x5588ae5664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.076+0000 7f6552d19640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae1a54d0 0x5588ae5664d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:52564/0 (socket says 172.21.15.67:52564) 2024-07-31T15:03:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.076+0000 7f6552d19640 1 -- 172.21.15.67:0/4125183195 learned_addr learned my addr 172.21.15.67:0/4125183195 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.077+0000 7f6552518640 1 --2- 172.21.15.67:0/4125183195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae184d80 0x5588ae56a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:20.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.075+0000 7fdf5d8a5640 1 -- 172.21.15.67:0/2684769291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf58118e80 con 0x7fdf58111790 2024-07-31T15:03:20.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.075+0000 7fdf5d8a5640 1 -- 172.21.15.67:0/2684769291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf58119340 con 0x7fdf58111790 2024-07-31T15:03:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.077+0000 7f6552d19640 1 -- 172.21.15.67:0/4125183195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae184d80 msgr2=0x5588ae56a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.077+0000 7f6552d19640 1 --2- 172.21.15.67:0/4125183195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae184d80 0x5588ae56a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.077+0000 7fdf3f7fe640 1 -- 172.21.15.67:0/2684769291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5810ae00 con 0x7fdf58111790 2024-07-31T15:03:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.077+0000 7f6552d19640 1 -- 172.21.15.67:0/4125183195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae19d2f0 msgr2=0x5588ae56edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.077+0000 7f6552d19640 1 --2- 172.21.15.67:0/4125183195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae19d2f0 0x5588ae56edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.077+0000 7f6552d19640 1 -- 172.21.15.67:0/4125183195 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ae56fe80 con 0x5588ae1a54d0 2024-07-31T15:03:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.078+0000 7f6552d19640 1 --2- 172.21.15.67:0/4125183195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae1a54d0 0x5588ae5664d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f653c00cfd0 tx=0x7f653c0aa060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d000866ee4f50eef server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.078+0000 7f6537fff640 1 -- 172.21.15.67:0/4125183195 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c0c4070 con 0x5588ae1a54d0 2024-07-31T15:03:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.078+0000 7f6537fff640 1 -- 172.21.15.67:0/4125183195 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f653c0a6050 con 0x5588ae1a54d0 2024-07-31T15:03:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.079+0000 7f655b67b740 1 -- 172.21.15.67:0/4125183195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae1a54d0 msgr2=0x5588ae5664d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.079+0000 7f655b67b740 1 --2- 172.21.15.67:0/4125183195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae1a54d0 0x5588ae5664d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f653c00cfd0 tx=0x7f653c0aa060 comp rx=0 tx=0).stop 2024-07-31T15:03:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.080+0000 7f655b67b740 1 -- 172.21.15.67:0/4125183195 shutdown_connections 2024-07-31T15:03:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.080+0000 7f655b67b740 1 --2- 172.21.15.67:0/4125183195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae1a54d0 0x5588ae5664d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.080+0000 7f655b67b740 1 --2- 172.21.15.67:0/4125183195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae19d2f0 0x5588ae56edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.080+0000 7f655b67b740 1 --2- 172.21.15.67:0/4125183195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae184d80 0x5588ae56a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.080+0000 7f655b67b740 1 -- 172.21.15.67:0/4125183195 >> 172.21.15.67:0/4125183195 conn(0x5588ae1ba300 msgr2=0x5588ae569d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.081+0000 7f655b67b740 1 -- 172.21.15.67:0/4125183195 shutdown_connections 2024-07-31T15:03:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.081+0000 7f655b67b740 1 -- 172.21.15.67:0/4125183195 wait complete. 2024-07-31T15:03:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.082+0000 7f655b67b740 1 -- start start 2024-07-31T15:03:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.082+0000 7f655b67b740 1 Processor -- start 2024-07-31T15:03:20.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.081+0000 7fdf55ffb640 1 -- 172.21.15.67:0/2684769291 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fdf48018020 con 0x7fdf58111790 2024-07-31T15:03:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.082+0000 7f655b67b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae184d80 0x5588ae57c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.082+0000 7f655b67b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae19d2f0 0x5588ae57ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.082+0000 7f655b67b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1a54d0 0x5588ae5796c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.082+0000 7f655b67b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f65300113f0 con 0x5588ae1a54d0 2024-07-31T15:03:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.082+0000 7f655b67b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6530010d20 con 0x5588ae184d80 2024-07-31T15:03:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.082+0000 7f655b67b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f653000df80 con 0x5588ae19d2f0 2024-07-31T15:03:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.082+0000 7f655b67b740 10 client.0 _ll_register_callbacks cb 0x7f654400fee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.083+0000 7f6552d19640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1a54d0 0x5588ae5796c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.083+0000 7f6552518640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae184d80 0x5588ae57c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.083+0000 7f6552d19640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1a54d0 0x5588ae5796c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46084/0 (socket says 172.21.15.67:46084) 2024-07-31T15:03:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.083+0000 7f6552518640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae184d80 0x5588ae57c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:36910/0 (socket says 172.21.15.67:36910) 2024-07-31T15:03:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.083+0000 7f6552d19640 1 -- 172.21.15.67:0/2323768082 learned_addr learned my addr 172.21.15.67:0/2323768082 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.083+0000 7f6551d17640 1 --2- 172.21.15.67:0/2323768082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae19d2f0 0x5588ae57ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:20.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.083+0000 7fdf55ffb640 1 --2- 172.21.15.67:0/2684769291 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdf400b3f70 0x7fdf400b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.083+0000 7f6552d19640 1 -- 172.21.15.67:0/2323768082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae19d2f0 msgr2=0x5588ae57ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.083+0000 7f6552d19640 1 --2- 172.21.15.67:0/2323768082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae19d2f0 0x5588ae57ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.083+0000 7f6552d19640 1 -- 172.21.15.67:0/2323768082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae184d80 msgr2=0x5588ae57c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.083+0000 7f6552d19640 1 --2- 172.21.15.67:0/2323768082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae184d80 0x5588ae57c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.083+0000 7f6552d19640 1 -- 172.21.15.67:0/2323768082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ae579e50 con 0x5588ae1a54d0 2024-07-31T15:03:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.083+0000 7f6552518640 1 --2- 172.21.15.67:0/2323768082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae184d80 0x5588ae57c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:03:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.083+0000 7f6551d17640 1 --2- 172.21.15.67:0/2323768082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae19d2f0 0x5588ae57ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:03:20.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.083+0000 7fdf55ffb640 1 -- 172.21.15.67:0/2684769291 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9483+0+0 (secure 0 0 0) 0x7fdf480e3cb0 con 0x7fdf58111790 2024-07-31T15:03:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.084+0000 7f6552d19640 1 --2- 172.21.15.67:0/2323768082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1a54d0 0x5588ae5796c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f653c005940 tx=0x7f653c0aa190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03806951b93ee9c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.084+0000 7f6535ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.084+0000 7f6535ffb640 1 -- 172.21.15.67:0/2323768082 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c0ca060 con 0x5588ae1a54d0 2024-07-31T15:03:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.084+0000 7f655b67b740 1 -- 172.21.15.67:0/2323768082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f653c0a6050 con 0x5588ae1a54d0 2024-07-31T15:03:20.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.084+0000 7fdf5c8a3640 1 --2- 172.21.15.67:0/2684769291 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdf400b3f70 0x7fdf400b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.084+0000 7f6535ffb640 1 -- 172.21.15.67:0/2323768082 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9483+0+0 (secure 0 0 0) 0x7f653c0b3740 con 0x5588ae1a54d0 2024-07-31T15:03:20.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.084+0000 7fdf5c8a3640 1 --2- 172.21.15.67:0/2684769291 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdf400b3f70 0x7fdf400b6480 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c006fd0 tx=0x7fdf4c008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.085+0000 7f6535ffb640 1 -- 172.21.15.67:0/2323768082 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f653c0b8070 con 0x5588ae1a54d0 2024-07-31T15:03:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.085+0000 7f655b67b740 10 client.18081 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.085+0000 7f655b67b740 1 -- 172.21.15.67:0/2323768082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5588ae5edc00 con 0x5588ae1a54d0 2024-07-31T15:03:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.085+0000 7f6535ffb640 1 -- 172.21.15.67:0/2323768082 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f653c0a6050 con 0x5588ae1a54d0 2024-07-31T15:03:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.085+0000 7f6535ffb640 1 -- 172.21.15.67:0/2323768082 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f653c0e1b80 con 0x5588ae1a54d0 2024-07-31T15:03:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.085+0000 7f655b67b740 10 client.18081 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.085+0000 7f655b67b740 10 client.18081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.085+0000 7f655b67b740 20 client.18081 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.085+0000 7f655b67b740 1 --2- 172.21.15.67:0/2323768082 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5588ae1c61b0 0x5588ae5f0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.085+0000 7f655b67b740 4 client.18081 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:03:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.085+0000 7f655b67b740 1 -- 172.21.15.67:0/2323768082 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588ae4dbda0 con 0x5588ae1c61b0 2024-07-31T15:03:20.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.086+0000 7fdf55ffb640 1 -- 172.21.15.67:0/2684769291 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fdf4800d040 con 0x7fdf58111790 2024-07-31T15:03:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.094+0000 7f6552518640 1 --2- 172.21.15.67:0/2323768082 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5588ae1c61b0 0x5588ae5f0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.096+0000 7f6552518640 1 --2- 172.21.15.67:0/2323768082 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5588ae1c61b0 0x5588ae5f0240 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bc33e8f0f73f286 server_cookie=6242aeb786b01494 in_seq=0 out_seq=0 2024-07-31T15:03:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.096+0000 7f6535ffb640 10 client.18081 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.400+0000 7f6535ffb640 1 -- 172.21.15.67:0/2323768082 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588ae4dbda0 con 0x5588ae1c61b0 2024-07-31T15:03:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.400+0000 7f6535ffb640 10 client.18081 handle_command_reply: tid=0 2024-07-31T15:03:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.400+0000 7f655b67b740 1 client.18081 shutdown 2024-07-31T15:03:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.401+0000 7f655b67b740 1 -- 172.21.15.67:0/2323768082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1a54d0 msgr2=0x5588ae5796c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.401+0000 7f655b67b740 1 --2- 172.21.15.67:0/2323768082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1a54d0 0x5588ae5796c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f653c005940 tx=0x7f653c0aa190 comp rx=0 tx=0).stop 2024-07-31T15:03:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.401+0000 7f655b67b740 1 -- 172.21.15.67:0/2323768082 shutdown_connections 2024-07-31T15:03:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.401+0000 7f655b67b740 1 --2- 172.21.15.67:0/2323768082 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5588ae1c61b0 0x5588ae5f0240 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.401+0000 7f655b67b740 1 --2- 172.21.15.67:0/2323768082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588ae1a54d0 0x5588ae5796c0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.401+0000 7f655b67b740 1 --2- 172.21.15.67:0/2323768082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588ae19d2f0 0x5588ae57ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.401+0000 7f655b67b740 1 --2- 172.21.15.67:0/2323768082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588ae184d80 0x5588ae57c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.401+0000 7f655b67b740 1 -- 172.21.15.67:0/2323768082 >> 172.21.15.67:0/2323768082 conn(0x5588ae1ba300 msgr2=0x5588ae566460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.401+0000 7f6535ffb640 0 client.18081 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.401+0000 7f655b67b740 1 -- 172.21.15.67:0/2323768082 shutdown_connections 2024-07-31T15:03:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.401+0000 7f655b67b740 1 -- 172.21.15.67:0/2323768082 wait complete. 2024-07-31T15:03:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.401+0000 7f655b67b740 20 client.18081 trim_cache size 0 max 16384 2024-07-31T15:03:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:03:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:03:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:03:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:03:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.402+0000 7f655477c640 1 -- 172.21.15.67:0/898597890 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65380b3f50 msgr2=0x7f65380b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.402+0000 7f655477c640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65380b3f50 0x7f65380b6460 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f654c11bbf0 tx=0x7f65440097d0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.402+0000 7f655477c640 1 -- 172.21.15.67:0/898597890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f654c11c5d0 msgr2=0x7f654c116730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.402+0000 7f655477c640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f654c11c5d0 0x7f654c116730 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f653c00e970 tx=0x7f653c00ee30 comp rx=0 tx=0).stop 2024-07-31T15:03:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.402+0000 7f655477c640 1 -- 172.21.15.67:0/898597890 shutdown_connections 2024-07-31T15:03:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.402+0000 7f655477c640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65380b3f50 0x7f65380b6460 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.402+0000 7f655477c640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f654c11c5d0 0x7f654c116730 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.402+0000 7f655477c640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f654c1118d0 0x7f654c116160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.402+0000 7f655477c640 1 --2- 172.21.15.67:0/898597890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f654c10a330 0x7f654c11beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.402+0000 7f655477c640 1 -- 172.21.15.67:0/898597890 >> 172.21.15.67:0/898597890 conn(0x7f654c0707c0 msgr2=0x7f654c11db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.403+0000 7f655477c640 1 -- 172.21.15.67:0/898597890 shutdown_connections 2024-07-31T15:03:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:20.403+0000 7f655477c640 1 -- 172.21.15.67:0/898597890 wait complete. 2024-07-31T15:03:20.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.537+0000 7fdf3f7fe640 1 -- 172.21.15.67:0/2684769291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf5810a7c0 con 0x7fdf58111790 2024-07-31T15:03:20.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.538+0000 7fdf55ffb640 1 -- 172.21.15.67:0/2684769291 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+11959 (secure 0 0 0) 0x7fdf480aad30 con 0x7fdf58111790 2024-07-31T15:03:20.540 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:03:20.540 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":63,"btime":"2024-07-31T15:03:07:099477+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:03:06.671820+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:03:20.540 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 63 2024-07-31T15:03:20.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.541+0000 7fdf3f7fe640 1 -- 172.21.15.67:0/2684769291 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdf400b3f70 msgr2=0x7fdf400b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:20.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.541+0000 7fdf3f7fe640 1 --2- 172.21.15.67:0/2684769291 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdf400b3f70 0x7fdf400b6480 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c006fd0 tx=0x7fdf4c008040 comp rx=0 tx=0).stop 2024-07-31T15:03:20.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.541+0000 7fdf3f7fe640 1 -- 172.21.15.67:0/2684769291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf58111790 msgr2=0x7fdf581131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:20.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.541+0000 7fdf3f7fe640 1 --2- 172.21.15.67:0/2684769291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf58111790 0x7fdf581131f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800b450 tx=0x7fdf4800b910 comp rx=0 tx=0).stop 2024-07-31T15:03:20.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.543+0000 7fdf3f7fe640 1 -- 172.21.15.67:0/2684769291 shutdown_connections 2024-07-31T15:03:20.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.543+0000 7fdf3f7fe640 1 --2- 172.21.15.67:0/2684769291 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdf400b3f70 0x7fdf400b6480 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.543+0000 7fdf3f7fe640 1 --2- 172.21.15.67:0/2684769291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5811c5d0 0x7fdf58113870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.543+0000 7fdf3f7fe640 1 --2- 172.21.15.67:0/2684769291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf58111790 0x7fdf581131f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.543+0000 7fdf3f7fe640 1 --2- 172.21.15.67:0/2684769291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf5810a330 0x7fdf58119f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:20.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.543+0000 7fdf3f7fe640 1 -- 172.21.15.67:0/2684769291 >> 172.21.15.67:0/2684769291 conn(0x7fdf5806e420 msgr2=0x7fdf58071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:20.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.550+0000 7fdf3f7fe640 1 -- 172.21.15.67:0/2684769291 shutdown_connections 2024-07-31T15:03:20.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:20.550+0000 7fdf3f7fe640 1 -- 172.21.15.67:0/2684769291 wait complete. 2024-07-31T15:03:21.252 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T15:03:21.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:21 smithi077 ceph-mon[37327]: pgmap v786: 129 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 875 KiB/s wr, 125 op/s 2024-07-31T15:03:21.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:21 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2684769291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:03:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:21 smithi067 ceph-mon[32750]: pgmap v786: 129 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 875 KiB/s wr, 125 op/s 2024-07-31T15:03:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:21 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2684769291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:03:21.586 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:03:21.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:21 smithi073 ceph-mon[36714]: pgmap v786: 129 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 875 KiB/s wr, 125 op/s 2024-07-31T15:03:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:21 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2684769291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:03:23.474 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:23 smithi067 ceph-mon[32750]: pgmap v787: 129 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 107 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 372 KiB/s wr, 52 op/s 2024-07-31T15:03:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:23 smithi077 ceph-mon[37327]: pgmap v787: 129 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 107 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 372 KiB/s wr, 52 op/s 2024-07-31T15:03:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:23 smithi073 ceph-mon[36714]: pgmap v787: 129 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 107 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 372 KiB/s wr, 52 op/s 2024-07-31T15:03:23.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.702+0000 7fc4c88d5640 1 -- 172.21.15.67:0/2388986717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c4103590 msgr2=0x7fc4c41039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:23.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.702+0000 7fc4c88d5640 1 --2- 172.21.15.67:0/2388986717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c4103590 0x7fc4c41039a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009ca0 tx=0x7fc4b402a8c0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.702+0000 7fc4c88d5640 1 -- 172.21.15.67:0/2388986717 shutdown_connections 2024-07-31T15:03:23.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.702+0000 7fc4c88d5640 1 --2- 172.21.15.67:0/2388986717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4109820 0x7fc4c4110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.702+0000 7fc4c88d5640 1 --2- 172.21.15.67:0/2388986717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c4108ed0 0x7fc4c41092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.702+0000 7fc4c88d5640 1 --2- 172.21.15.67:0/2388986717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c4103590 0x7fc4c41039a0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.702+0000 7fc4c88d5640 1 -- 172.21.15.67:0/2388986717 >> 172.21.15.67:0/2388986717 conn(0x7fc4c407b280 msgr2=0x7fc4c407b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:23.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.702+0000 7fc4c88d5640 1 -- 172.21.15.67:0/2388986717 shutdown_connections 2024-07-31T15:03:23.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.702+0000 7fc4c88d5640 1 -- 172.21.15.67:0/2388986717 wait complete. 2024-07-31T15:03:23.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c88d5640 1 Processor -- start 2024-07-31T15:03:23.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c88d5640 1 -- start start 2024-07-31T15:03:23.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c88d5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4103590 0x7fc4c4199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c88d5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c4108ed0 0x7fc4c419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c88d5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c4109820 0x7fc4c419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c88d5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41134d0 con 0x7fc4c4108ed0 2024-07-31T15:03:23.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c88d5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c4113350 con 0x7fc4c4103590 2024-07-31T15:03:23.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c88d5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c4113650 con 0x7fc4c4109820 2024-07-31T15:03:23.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4103590 0x7fc4c4199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4103590 0x7fc4c4199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:43714/0 (socket says 172.21.15.67:43714) 2024-07-31T15:03:23.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c259c640 1 -- 172.21.15.67:0/944817976 learned_addr learned my addr 172.21.15.67:0/944817976 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:23.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4bbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c4108ed0 0x7fc4c419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c2d9d640 1 --2- 172.21.15.67:0/944817976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c4109820 0x7fc4c419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c259c640 1 -- 172.21.15.67:0/944817976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c4109820 msgr2=0x7fc4c419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:23.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c259c640 1 --2- 172.21.15.67:0/944817976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c4109820 0x7fc4c419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c259c640 1 -- 172.21.15.67:0/944817976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c4108ed0 msgr2=0x7fc4c419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:23.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.703+0000 7fc4c259c640 1 --2- 172.21.15.67:0/944817976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c4108ed0 0x7fc4c419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.704+0000 7fc4c259c640 1 -- 172.21.15.67:0/944817976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c419fe50 con 0x7fc4c4103590 2024-07-31T15:03:23.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.704+0000 7fc4bbfff640 1 --2- 172.21.15.67:0/944817976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c4108ed0 0x7fc4c419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:03:23.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.704+0000 7fc4c259c640 1 --2- 172.21.15.67:0/944817976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4103590 0x7fc4c4199ee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc4b402ade0 tx=0x7fc4b4031770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:23.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.704+0000 7fc4bb7fe640 1 -- 172.21.15.67:0/944817976 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4003b70 con 0x7fc4c4103590 2024-07-31T15:03:23.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.704+0000 7fc4c88d5640 1 -- 172.21.15.67:0/944817976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c41a00e0 con 0x7fc4c4103590 2024-07-31T15:03:23.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.704+0000 7fc4bb7fe640 1 -- 172.21.15.67:0/944817976 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b4003d10 con 0x7fc4c4103590 2024-07-31T15:03:23.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.704+0000 7fc4bb7fe640 1 -- 172.21.15.67:0/944817976 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b402fa50 con 0x7fc4c4103590 2024-07-31T15:03:23.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.704+0000 7fc4c88d5640 1 -- 172.21.15.67:0/944817976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c41a68a0 con 0x7fc4c4103590 2024-07-31T15:03:23.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.707+0000 7fc4cee26740 1 Processor -- start 2024-07-31T15:03:23.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.707+0000 7fc4cee26740 1 -- start start 2024-07-31T15:03:23.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4bb7fe640 1 -- 172.21.15.67:0/944817976 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc4b4004580 con 0x7fc4c4103590 2024-07-31T15:03:23.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4cee26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e596070 0x55de5e93b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4cee26740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e56d1b0 0x55de5e93bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4cee26740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e575390 0x55de5e93bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4cee26740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55de5e8b1bb0 con 0x55de5e56d1b0 2024-07-31T15:03:23.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4cee26740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55de5e8a10b0 con 0x55de5e575390 2024-07-31T15:03:23.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4cee26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55de5e8a50f0 con 0x55de5e596070 2024-07-31T15:03:23.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4c259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e56d1b0 0x55de5e93bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4c259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e56d1b0 0x55de5e93bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53022/0 (socket says 172.21.15.67:53022) 2024-07-31T15:03:23.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4c259c640 1 -- 172.21.15.67:0/1315956311 learned_addr learned my addr 172.21.15.67:0/1315956311 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:23.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4bbfff640 1 --2- 172.21.15.67:0/1315956311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e575390 0x55de5e93bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4c2d9d640 1 --2- 172.21.15.67:0/1315956311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e596070 0x55de5e93b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4c259c640 1 -- 172.21.15.67:0/1315956311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e596070 msgr2=0x55de5e93b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:23.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4c259c640 1 --2- 172.21.15.67:0/1315956311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e596070 0x55de5e93b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4c259c640 1 -- 172.21.15.67:0/1315956311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e575390 msgr2=0x55de5e93bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:23.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4c259c640 1 --2- 172.21.15.67:0/1315956311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e575390 0x55de5e93bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.708+0000 7fc4c259c640 1 -- 172.21.15.67:0/1315956311 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de5e847b90 con 0x55de5e56d1b0 2024-07-31T15:03:23.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.709+0000 7fc4c259c640 1 --2- 172.21.15.67:0/1315956311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e56d1b0 0x55de5e93bab0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4102b10 tx=0x7fc4b410bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3785443cbe71ace2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.709+0000 7fc4bb7fe640 1 --2- 172.21.15.67:0/944817976 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc4b00b3d40 0x7fc4b00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.709+0000 7fc4b8ff9640 1 -- 172.21.15.67:0/1315956311 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b40cb9b0 con 0x55de5e56d1b0 2024-07-31T15:03:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.709+0000 7fc4c2d9d640 1 --2- 172.21.15.67:0/944817976 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc4b00b3d40 0x7fc4b00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.709+0000 7fc4bb7fe640 1 -- 172.21.15.67:0/944817976 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9483+0+0 (secure 0 0 0) 0x7fc4b4105700 con 0x7fc4c4103590 2024-07-31T15:03:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.709+0000 7fc4b8ff9640 1 -- 172.21.15.67:0/1315956311 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b4115020 con 0x55de5e56d1b0 2024-07-31T15:03:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.709+0000 7fc4b8ff9640 1 -- 172.21.15.67:0/1315956311 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b40cbba0 con 0x55de5e56d1b0 2024-07-31T15:03:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.710+0000 7fc4cee26740 1 -- 172.21.15.67:0/1315956311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e56d1b0 msgr2=0x55de5e93bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.710+0000 7fc4c2d9d640 1 --2- 172.21.15.67:0/944817976 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc4b00b3d40 0x7fc4b00b6250 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009d20 tx=0x7fc4bc005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.710+0000 7fc4cee26740 1 --2- 172.21.15.67:0/1315956311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e56d1b0 0x55de5e93bab0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4102b10 tx=0x7fc4b410bb40 comp rx=0 tx=0).stop 2024-07-31T15:03:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.710+0000 7fc4cee26740 1 -- 172.21.15.67:0/1315956311 shutdown_connections 2024-07-31T15:03:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.710+0000 7fc4cee26740 1 --2- 172.21.15.67:0/1315956311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e575390 0x55de5e93bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.710+0000 7fc4cee26740 1 --2- 172.21.15.67:0/1315956311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e56d1b0 0x55de5e93bab0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.710+0000 7fc4cee26740 1 --2- 172.21.15.67:0/1315956311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e596070 0x55de5e93b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.710+0000 7fc4cee26740 1 -- 172.21.15.67:0/1315956311 >> 172.21.15.67:0/1315956311 conn(0x55de5e58a460 msgr2=0x55de5e939460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.710+0000 7fc4cee26740 1 -- 172.21.15.67:0/1315956311 shutdown_connections 2024-07-31T15:03:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.710+0000 7fc4cee26740 1 -- 172.21.15.67:0/1315956311 wait complete. 2024-07-31T15:03:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.711+0000 7fc4cee26740 1 -- start start 2024-07-31T15:03:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.711+0000 7fc4cee26740 1 Processor -- start 2024-07-31T15:03:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.711+0000 7fc4cee26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e56d1b0 0x55de5e9470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.711+0000 7fc4cee26740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e575390 0x55de5e9475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.711+0000 7fc4cee26740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e952bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.711+0000 7fc4cee26740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55de5e704700 con 0x55de5e575390 2024-07-31T15:03:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.711+0000 7fc4cee26740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55de5e8439e0 con 0x55de5e596070 2024-07-31T15:03:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.711+0000 7fc4cee26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55de5e8b1bb0 con 0x55de5e56d1b0 2024-07-31T15:03:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.711+0000 7fc4cee26740 10 client.0 _ll_register_callbacks cb 0x55de5e85e4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.711+0000 7fc4c2d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e952bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.711+0000 7fc4bbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e575390 0x55de5e9475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.711+0000 7fc4bbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e575390 0x55de5e9475f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53028/0 (socket says 172.21.15.67:53028) 2024-07-31T15:03:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.711+0000 7fc4c259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e56d1b0 0x55de5e9470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.711+0000 7fc4bbfff640 1 -- 172.21.15.67:0/1949567420 learned_addr learned my addr 172.21.15.67:0/1949567420 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.712+0000 7fc4c2d9d640 1 -- 172.21.15.67:0/1949567420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e56d1b0 msgr2=0x55de5e9470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.712+0000 7fc4c2d9d640 1 --2- 172.21.15.67:0/1949567420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e56d1b0 0x55de5e9470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.712+0000 7fc4c2d9d640 1 -- 172.21.15.67:0/1949567420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e575390 msgr2=0x55de5e9475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.712+0000 7fc4c2d9d640 1 --2- 172.21.15.67:0/1949567420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e575390 0x55de5e9475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.712+0000 7fc4c2d9d640 1 -- 172.21.15.67:0/1949567420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de5e601650 con 0x55de5e596070 2024-07-31T15:03:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.712+0000 7fc4c2d9d640 1 --2- 172.21.15.67:0/1949567420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e952bf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc004550 tx=0x7fc4bc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcd8390ebbb700d4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.712+0000 7fc4a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:03:23.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.712+0000 7fc4a9ffb640 1 -- 172.21.15.67:0/1949567420 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc009e10 con 0x55de5e596070 2024-07-31T15:03:23.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.712+0000 7fc4cee26740 1 -- 172.21.15.67:0/1949567420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b4115020 con 0x55de5e596070 2024-07-31T15:03:23.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.712+0000 7fc4a9ffb640 1 -- 172.21.15.67:0/1949567420 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9483+0+0 (secure 0 0 0) 0x7fc4bc005020 con 0x55de5e596070 2024-07-31T15:03:23.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.712+0000 7fc4a9ffb640 1 -- 172.21.15.67:0/1949567420 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fc4bc005220 con 0x55de5e596070 2024-07-31T15:03:23.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.712+0000 7fc4cee26740 10 client.27653 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:23.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.712+0000 7fc4cee26740 1 -- 172.21.15.67:0/1949567420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55de5e9558f0 con 0x55de5e596070 2024-07-31T15:03:23.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.713+0000 7fc4a9ffb640 1 -- 172.21.15.67:0/1949567420 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b4115020 con 0x55de5e596070 2024-07-31T15:03:23.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.713+0000 7fc4a9ffb640 1 -- 172.21.15.67:0/1949567420 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fc4bc027070 con 0x55de5e596070 2024-07-31T15:03:23.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.713+0000 7fc4cee26740 10 client.27653 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:23.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.713+0000 7fc4cee26740 10 client.27653 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:03:23.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.713+0000 7fc4cee26740 20 client.27653 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:23.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.713+0000 7fc4cee26740 1 --2- 172.21.15.67:0/1949567420 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de5e5549f0 0x55de5e958e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.713+0000 7fc4cee26740 4 client.27653 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:03:23.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.713+0000 7fc4cee26740 1 -- 172.21.15.67:0/1949567420 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de5e8b3b00 con 0x55de5e5549f0 2024-07-31T15:03:23.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.713+0000 7fc4c259c640 1 --2- 172.21.15.67:0/1949567420 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de5e5549f0 0x55de5e958e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.714+0000 7fc4c259c640 1 --2- 172.21.15.67:0/1949567420 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de5e5549f0 0x55de5e958e90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89d63c7df8e44f5a server_cookie=e5200e00a2720ace in_seq=0 out_seq=0 2024-07-31T15:03:23.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.714+0000 7fc4a9ffb640 10 client.27653 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:03:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.714+0000 7fc4a9ffb640 1 -- 172.21.15.67:0/1949567420 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de5e8b3b00 con 0x55de5e5549f0 2024-07-31T15:03:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.714+0000 7fc4a9ffb640 10 client.27653 handle_command_reply: tid=0 2024-07-31T15:03:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.714+0000 7fc4cee26740 1 client.27653 shutdown 2024-07-31T15:03:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.714+0000 7fc4cee26740 1 -- 172.21.15.67:0/1949567420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 msgr2=0x55de5e952bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.715+0000 7fc4cee26740 1 --2- 172.21.15.67:0/1949567420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e952bf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc004550 tx=0x7fc4bc004730 comp rx=0 tx=0).stop 2024-07-31T15:03:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.715+0000 7fc4cee26740 1 -- 172.21.15.67:0/1949567420 shutdown_connections 2024-07-31T15:03:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.715+0000 7fc4cee26740 1 --2- 172.21.15.67:0/1949567420 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de5e5549f0 0x55de5e958e90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.715+0000 7fc4cee26740 1 --2- 172.21.15.67:0/1949567420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e952bf0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.715+0000 7fc4cee26740 1 --2- 172.21.15.67:0/1949567420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e575390 0x55de5e9475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.715+0000 7fc4a9ffb640 0 client.27653 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:03:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.715+0000 7fc4cee26740 1 --2- 172.21.15.67:0/1949567420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e56d1b0 0x55de5e9470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.715+0000 7fc4cee26740 1 -- 172.21.15.67:0/1949567420 >> 172.21.15.67:0/1949567420 conn(0x55de5e58a460 msgr2=0x55de5e937c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.715+0000 7fc4cee26740 1 -- 172.21.15.67:0/1949567420 shutdown_connections 2024-07-31T15:03:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.715+0000 7fc4cee26740 1 -- 172.21.15.67:0/1949567420 wait complete. 2024-07-31T15:03:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.715+0000 7fc4cee26740 20 client.27653 trim_cache size 0 max 16384 2024-07-31T15:03:23.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4cee26740 1 Processor -- start 2024-07-31T15:03:23.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4cee26740 1 -- start start 2024-07-31T15:03:23.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4cee26740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e56d1b0 0x55de5e939490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4cee26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e575390 0x55de5e941690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4cee26740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e9529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4cee26740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41b08d0 con 0x55de5e56d1b0 2024-07-31T15:03:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4cee26740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41b2410 con 0x55de5e596070 2024-07-31T15:03:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4cee26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41b0fa0 con 0x55de5e575390 2024-07-31T15:03:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4c259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e56d1b0 0x55de5e939490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4c259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e56d1b0 0x55de5e939490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53032/0 (socket says 172.21.15.67:53032) 2024-07-31T15:03:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4c259c640 1 -- 172.21.15.67:0/374731289 learned_addr learned my addr 172.21.15.67:0/374731289 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4c2d9d640 1 --2- 172.21.15.67:0/374731289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e9529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4bbfff640 1 --2- 172.21.15.67:0/374731289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e575390 0x55de5e941690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4c259c640 1 -- 172.21.15.67:0/374731289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e575390 msgr2=0x55de5e941690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4c259c640 1 --2- 172.21.15.67:0/374731289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e575390 0x55de5e941690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4c259c640 1 -- 172.21.15.67:0/374731289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 msgr2=0x55de5e9529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4c259c640 1 --2- 172.21.15.67:0/374731289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e9529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.729+0000 7fc4c259c640 1 -- 172.21.15.67:0/374731289 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de5e93d860 con 0x55de5e56d1b0 2024-07-31T15:03:23.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.730+0000 7fc4c259c640 1 --2- 172.21.15.67:0/374731289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e56d1b0 0x55de5e939490 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4103090 tx=0x7fc4b4009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21ccb04f839a675 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:23.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.730+0000 7fc4abfff640 1 -- 172.21.15.67:0/374731289 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4114070 con 0x55de5e56d1b0 2024-07-31T15:03:23.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.730+0000 7fc4abfff640 1 -- 172.21.15.67:0/374731289 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b4036070 con 0x55de5e56d1b0 2024-07-31T15:03:23.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.730+0000 7fc4abfff640 1 -- 172.21.15.67:0/374731289 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b40cbe30 con 0x55de5e56d1b0 2024-07-31T15:03:23.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.731+0000 7fc4cee26740 1 -- 172.21.15.67:0/374731289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e56d1b0 msgr2=0x55de5e939490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:23.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.731+0000 7fc4cee26740 1 --2- 172.21.15.67:0/374731289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e56d1b0 0x55de5e939490 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4103090 tx=0x7fc4b4009460 comp rx=0 tx=0).stop 2024-07-31T15:03:23.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.731+0000 7fc4cee26740 1 -- 172.21.15.67:0/374731289 shutdown_connections 2024-07-31T15:03:23.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.731+0000 7fc4cee26740 1 --2- 172.21.15.67:0/374731289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e9529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.731+0000 7fc4cee26740 1 --2- 172.21.15.67:0/374731289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e575390 0x55de5e941690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.731+0000 7fc4cee26740 1 --2- 172.21.15.67:0/374731289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e56d1b0 0x55de5e939490 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.731+0000 7fc4cee26740 1 -- 172.21.15.67:0/374731289 >> 172.21.15.67:0/374731289 conn(0x55de5e58a460 msgr2=0x55de5e938d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:23.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.731+0000 7fc4cee26740 1 -- 172.21.15.67:0/374731289 shutdown_connections 2024-07-31T15:03:23.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.731+0000 7fc4cee26740 1 -- 172.21.15.67:0/374731289 wait complete. 2024-07-31T15:03:23.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.731+0000 7fc4cee26740 1 -- start start 2024-07-31T15:03:23.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.731+0000 7fc4cee26740 1 Processor -- start 2024-07-31T15:03:23.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4cee26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e575390 0x55de5e94ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4cee26740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e94b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4cee26740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e529da0 0x55de5e94b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4cee26740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41b1d40 con 0x55de5e529da0 2024-07-31T15:03:23.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4cee26740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41b1670 con 0x55de5e596070 2024-07-31T15:03:23.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4cee26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41ae8d0 con 0x55de5e575390 2024-07-31T15:03:23.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4cee26740 10 client.0 _ll_register_callbacks cb 0x7fc4bc009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:23.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4bbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e94b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4bbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e94b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:43764/0 (socket says 172.21.15.67:43764) 2024-07-31T15:03:23.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4bbfff640 1 -- 172.21.15.67:0/3876963662 learned_addr learned my addr 172.21.15.67:0/3876963662 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:23.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4c2d9d640 1 --2- 172.21.15.67:0/3876963662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e529da0 0x55de5e94b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4c259c640 1 --2- 172.21.15.67:0/3876963662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e575390 0x55de5e94ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4bbfff640 1 -- 172.21.15.67:0/3876963662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e575390 msgr2=0x55de5e94ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:23.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4bbfff640 1 --2- 172.21.15.67:0/3876963662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e575390 0x55de5e94ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4bbfff640 1 -- 172.21.15.67:0/3876963662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e529da0 msgr2=0x55de5e94b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:23.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4bbfff640 1 --2- 172.21.15.67:0/3876963662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e529da0 0x55de5e94b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4bbfff640 1 -- 172.21.15.67:0/3876963662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de5e9c4b40 con 0x55de5e596070 2024-07-31T15:03:23.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4bbfff640 1 --2- 172.21.15.67:0/3876963662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e94b080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac012a00 tx=0x7fc4ac012ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f46d034270b512 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:23.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.732+0000 7fc4a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:03:23.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.733+0000 7fc4a9ffb640 1 -- 172.21.15.67:0/3876963662 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac022020 con 0x55de5e596070 2024-07-31T15:03:23.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.733+0000 7fc4cee26740 1 -- 172.21.15.67:0/3876963662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b4036070 con 0x55de5e596070 2024-07-31T15:03:23.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.733+0000 7fc4a9ffb640 1 -- 172.21.15.67:0/3876963662 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9483+0+0 (secure 0 0 0) 0x7fc4ac01d440 con 0x55de5e596070 2024-07-31T15:03:23.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.733+0000 7fc4a9ffb640 1 -- 172.21.15.67:0/3876963662 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fc4ac02d050 con 0x55de5e596070 2024-07-31T15:03:23.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.733+0000 7fc4cee26740 10 client.27659 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:23.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.733+0000 7fc4cee26740 1 -- 172.21.15.67:0/3876963662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55de5e9c5e70 con 0x55de5e596070 2024-07-31T15:03:23.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.733+0000 7fc4a9ffb640 1 -- 172.21.15.67:0/3876963662 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b4036070 con 0x55de5e596070 2024-07-31T15:03:23.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.733+0000 7fc4a9ffb640 1 -- 172.21.15.67:0/3876963662 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fc4ac01e700 con 0x55de5e596070 2024-07-31T15:03:23.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.733+0000 7fc4cee26740 10 client.27659 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:23.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.733+0000 7fc4cee26740 10 client.27659 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:03:23.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.733+0000 7fc4cee26740 20 client.27659 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:23.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.733+0000 7fc4cee26740 1 --2- 172.21.15.67:0/3876963662 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de5e9c6650 0x55de5e9c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:23.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.734+0000 7fc4cee26740 4 client.27659 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:03:23.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.734+0000 7fc4cee26740 1 -- 172.21.15.67:0/3876963662 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de5e8af5a0 con 0x55de5e9c6650 2024-07-31T15:03:23.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.734+0000 7fc4c259c640 1 --2- 172.21.15.67:0/3876963662 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de5e9c6650 0x55de5e9c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:23.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.734+0000 7fc4c259c640 1 --2- 172.21.15.67:0/3876963662 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de5e9c6650 0x55de5e9c8a70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=35bbeca6326c560c server_cookie=f502becfba736595 in_seq=0 out_seq=0 2024-07-31T15:03:23.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.734+0000 7fc4a9ffb640 10 client.27659 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:03:23.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.734+0000 7fc4a9ffb640 1 -- 172.21.15.67:0/3876963662 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de5e8af5a0 con 0x55de5e9c6650 2024-07-31T15:03:23.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.734+0000 7fc4a9ffb640 10 client.27659 handle_command_reply: tid=0 2024-07-31T15:03:23.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.735+0000 7fc4cee26740 1 client.27659 shutdown 2024-07-31T15:03:23.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.735+0000 7fc4cee26740 1 -- 172.21.15.67:0/3876963662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 msgr2=0x55de5e94b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:23.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.735+0000 7fc4cee26740 1 --2- 172.21.15.67:0/3876963662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e94b080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac012a00 tx=0x7fc4ac012ec0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.735+0000 7fc4cee26740 1 -- 172.21.15.67:0/3876963662 shutdown_connections 2024-07-31T15:03:23.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.735+0000 7fc4cee26740 1 --2- 172.21.15.67:0/3876963662 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55de5e9c6650 0x55de5e9c8a70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.735+0000 7fc4cee26740 1 --2- 172.21.15.67:0/3876963662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de5e529da0 0x55de5e94b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.735+0000 7fc4cee26740 1 --2- 172.21.15.67:0/3876963662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de5e596070 0x55de5e94b080 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.735+0000 7fc4a9ffb640 0 client.27659 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:03:23.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.735+0000 7fc4cee26740 1 --2- 172.21.15.67:0/3876963662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de5e575390 0x55de5e94ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.735+0000 7fc4cee26740 1 -- 172.21.15.67:0/3876963662 >> 172.21.15.67:0/3876963662 conn(0x55de5e58a460 msgr2=0x55de5e93c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:23.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.735+0000 7fc4cee26740 1 -- 172.21.15.67:0/3876963662 shutdown_connections 2024-07-31T15:03:23.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.735+0000 7fc4cee26740 1 -- 172.21.15.67:0/3876963662 wait complete. 2024-07-31T15:03:23.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.735+0000 7fc4cee26740 20 client.27659 trim_cache size 0 max 16384 2024-07-31T15:03:23.742 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:03:23.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.736+0000 7fc4c88d5640 1 -- 172.21.15.67:0/944817976 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc4b00b3d40 msgr2=0x7fc4b00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.736+0000 7fc4c88d5640 1 --2- 172.21.15.67:0/944817976 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc4b00b3d40 0x7fc4b00b6250 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009d20 tx=0x7fc4bc005f90 comp rx=0 tx=0).stop 2024-07-31T15:03:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.736+0000 7fc4c88d5640 1 -- 172.21.15.67:0/944817976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4103590 msgr2=0x7fc4c4199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.736+0000 7fc4c88d5640 1 --2- 172.21.15.67:0/944817976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4103590 0x7fc4c4199ee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc4b402ade0 tx=0x7fc4b4031770 comp rx=0 tx=0).stop 2024-07-31T15:03:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.736+0000 7fc4c88d5640 1 -- 172.21.15.67:0/944817976 shutdown_connections 2024-07-31T15:03:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.736+0000 7fc4c88d5640 1 --2- 172.21.15.67:0/944817976 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc4b00b3d40 0x7fc4b00b6250 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.736+0000 7fc4c88d5640 1 --2- 172.21.15.67:0/944817976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c4109820 0x7fc4c419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.736+0000 7fc4c88d5640 1 --2- 172.21.15.67:0/944817976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c4108ed0 0x7fc4c419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.736+0000 7fc4c88d5640 1 --2- 172.21.15.67:0/944817976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c4103590 0x7fc4c4199ee0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.736+0000 7fc4c88d5640 1 -- 172.21.15.67:0/944817976 >> 172.21.15.67:0/944817976 conn(0x7fc4c407b280 msgr2=0x7fc4c4106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.736+0000 7fc4c88d5640 1 -- 172.21.15.67:0/944817976 shutdown_connections 2024-07-31T15:03:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:23.736+0000 7fc4c88d5640 1 -- 172.21.15.67:0/944817976 wait complete. 2024-07-31T15:03:24.433 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T15:03:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:03:24.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.865+0000 7f11eb7fe640 1 -- 172.21.15.67:0/543807090 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f1204003e20 con 0x55fc7a3b9c90 2024-07-31T15:03:24.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.865+0000 7f11eb7fe640 10 client.18033 handle_command_reply: tid=0 2024-07-31T15:03:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.865+0000 7f1218dca740 1 client.18033 shutdown 2024-07-31T15:03:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.865+0000 7f1218dca740 1 -- 172.21.15.67:0/543807090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f4c1c0 msgr2=0x55fc7a343c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.865+0000 7f1218dca740 1 --2- 172.21.15.67:0/543807090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f4c1c0 0x55fc7a343c00 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1204030b80 tx=0x7f1204000f40 comp rx=0 tx=0).stop 2024-07-31T15:03:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.865+0000 7f1218dca740 1 -- 172.21.15.67:0/543807090 shutdown_connections 2024-07-31T15:03:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.865+0000 7f1218dca740 1 --2- 172.21.15.67:0/543807090 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fc7a3b9c90 0x55fc7a3bc0b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.865+0000 7f1218dca740 1 --2- 172.21.15.67:0/543807090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc79f6cb60 0x55fc7a3b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.865+0000 7f1218dca740 1 --2- 172.21.15.67:0/543807090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc79f64980 0x55fc7a344140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.865+0000 7f11eb7fe640 0 client.18033 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.865+0000 7f1218dca740 1 --2- 172.21.15.67:0/543807090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc79f4c1c0 0x55fc7a343c00 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.865+0000 7f1218dca740 1 -- 172.21.15.67:0/543807090 >> 172.21.15.67:0/543807090 conn(0x55fc79f81990 msgr2=0x55fc7a32ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.866+0000 7f1218dca740 1 -- 172.21.15.67:0/543807090 shutdown_connections 2024-07-31T15:03:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.866+0000 7f1218dca740 1 -- 172.21.15.67:0/543807090 wait complete. 2024-07-31T15:03:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.866+0000 7f1218dca740 20 client.18033 trim_cache size 0 max 16384 2024-07-31T15:03:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:03:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 63, 2024-07-31T15:03:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:03:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 30, 2024-07-31T15:03:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:03:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "54db3a03": { 2024-07-31T15:03:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 30, 2024-07-31T15:03:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:03:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:03:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:03:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:03:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:03:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:03:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 142.4, 2024-07-31T15:03:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:03:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:03:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:03:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:03:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:03:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:03:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:03:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:03:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:03:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:03:24.871 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:03:24.871 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:03:24.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.866+0000 7f1211617640 1 -- 172.21.15.67:0/3508625786 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11f40b3f50 msgr2=0x7f11f40b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:24.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.866+0000 7f1211617640 1 --2- 172.21.15.67:0/3508625786 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11f40b3f50 0x7f11f40b6460 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55fc7a3326e0 tx=0x7f11fc009f90 comp rx=0 tx=0).stop 2024-07-31T15:03:24.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.866+0000 7f1211617640 1 -- 172.21.15.67:0/3508625786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120c10a330 msgr2=0x7f120c11ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:24.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.866+0000 7f1211617640 1 --2- 172.21.15.67:0/3508625786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120c10a330 0x7f120c11ad40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00e9c0 tx=0x7f11fc00ee80 comp rx=0 tx=0).stop 2024-07-31T15:03:24.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.866+0000 7f1211617640 1 -- 172.21.15.67:0/3508625786 shutdown_connections 2024-07-31T15:03:24.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.866+0000 7f1211617640 1 --2- 172.21.15.67:0/3508625786 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11f40b3f50 0x7f11f40b6460 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:24.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.866+0000 7f1211617640 1 --2- 172.21.15.67:0/3508625786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f120c11c5d0 0x7f120c114550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:24.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.866+0000 7f1211617640 1 --2- 172.21.15.67:0/3508625786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c111730 0x7f120c114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:24.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.866+0000 7f1211617640 1 --2- 172.21.15.67:0/3508625786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120c10a330 0x7f120c11ad40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:24.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.866+0000 7f1211617640 1 -- 172.21.15.67:0/3508625786 >> 172.21.15.67:0/3508625786 conn(0x7f120c06e440 msgr2=0x7f120c071650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:24.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.867+0000 7f1211617640 1 -- 172.21.15.67:0/3508625786 shutdown_connections 2024-07-31T15:03:24.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:24.867+0000 7f1211617640 1 -- 172.21.15.67:0/3508625786 wait complete. 2024-07-31T15:03:25.445 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:25 smithi067 ceph-mon[32750]: pgmap v788: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 147 KiB/s wr, 41 op/s 2024-07-31T15:03:25.445 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:25 smithi067 ceph-mon[32750]: osdmap e109: 12 total, 12 up, 12 in 2024-07-31T15:03:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:25 smithi077 ceph-mon[37327]: pgmap v788: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 147 KiB/s wr, 41 op/s 2024-07-31T15:03:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:25 smithi077 ceph-mon[37327]: osdmap e109: 12 total, 12 up, 12 in 2024-07-31T15:03:25.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:25 smithi073 ceph-mon[36714]: pgmap v788: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 147 KiB/s wr, 41 op/s 2024-07-31T15:03:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:25 smithi073 ceph-mon[36714]: osdmap e109: 12 total, 12 up, 12 in 2024-07-31T15:03:26.699 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '54db3a03', quiesce took 31.3 seconds. Will release after: -8.900000000000002 2024-07-31T15:03:26.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:03:27.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:27 smithi067 ceph-mon[32750]: pgmap v790: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 188 KiB/s wr, 37 op/s 2024-07-31T15:03:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:27 smithi077 ceph-mon[37327]: pgmap v790: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 188 KiB/s wr, 37 op/s 2024-07-31T15:03:27.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.480+0000 7fe30a355640 1 -- 172.21.15.67:0/692581736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe304105ce0 msgr2=0x7fe3041060f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:27.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.480+0000 7fe30a355640 1 --2- 172.21.15.67:0/692581736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe304105ce0 0x7fe3041060f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009900 tx=0x7fe2f802a410 comp rx=0 tx=0).stop 2024-07-31T15:03:27.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.480+0000 7fe30a355640 1 -- 172.21.15.67:0/692581736 shutdown_connections 2024-07-31T15:03:27.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.480+0000 7fe30a355640 1 --2- 172.21.15.67:0/692581736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe304106630 0x7fe3041100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.480+0000 7fe30a355640 1 --2- 172.21.15.67:0/692581736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe304105ce0 0x7fe3041060f0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.480+0000 7fe30a355640 1 --2- 172.21.15.67:0/692581736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe304101a00 0x7fe304101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.480+0000 7fe30a355640 1 -- 172.21.15.67:0/692581736 >> 172.21.15.67:0/692581736 conn(0x7fe3040776f0 msgr2=0x7fe304077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:27.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.480+0000 7fe30a355640 1 -- 172.21.15.67:0/692581736 shutdown_connections 2024-07-31T15:03:27.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.480+0000 7fe30a355640 1 -- 172.21.15.67:0/692581736 wait complete. 2024-07-31T15:03:27.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.481+0000 7fe30a355640 1 Processor -- start 2024-07-31T15:03:27.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.481+0000 7fe30a355640 1 -- start start 2024-07-31T15:03:27.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.481+0000 7fe30a355640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe304101a00 0x7fe304199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.481+0000 7fe30a355640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe304105ce0 0x7fe30419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.481+0000 7fe30a355640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe304106630 0x7fe30419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.481+0000 7fe30a355640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe3041134e0 con 0x7fe304106630 2024-07-31T15:03:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.481+0000 7fe30a355640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe304113360 con 0x7fe304101a00 2024-07-31T15:03:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.481+0000 7fe30a355640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe304113660 con 0x7fe304105ce0 2024-07-31T15:03:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.481+0000 7fe3088f2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe304106630 0x7fe30419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.481+0000 7fe3088f2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe304106630 0x7fe30419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53066/0 (socket says 172.21.15.67:53066) 2024-07-31T15:03:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.482+0000 7fe3088f2640 1 -- 172.21.15.67:0/2096432691 learned_addr learned my addr 172.21.15.67:0/2096432691 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.482+0000 7fe303fff640 1 --2- 172.21.15.67:0/2096432691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe304101a00 0x7fe304199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.482+0000 7fe3037fe640 1 --2- 172.21.15.67:0/2096432691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe304105ce0 0x7fe30419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.482+0000 7fe3088f2640 1 -- 172.21.15.67:0/2096432691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe304105ce0 msgr2=0x7fe30419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.482+0000 7fe3088f2640 1 --2- 172.21.15.67:0/2096432691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe304105ce0 0x7fe30419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.482+0000 7fe3088f2640 1 -- 172.21.15.67:0/2096432691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe304101a00 msgr2=0x7fe304199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.482+0000 7fe3088f2640 1 --2- 172.21.15.67:0/2096432691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe304101a00 0x7fe304199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.482+0000 7fe3088f2640 1 -- 172.21.15.67:0/2096432691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30419fe60 con 0x7fe304106630 2024-07-31T15:03:27.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.482+0000 7fe3088f2640 1 --2- 172.21.15.67:0/2096432691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe304106630 0x7fe30419f730 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00eb10 tx=0x7fe2fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:27.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.482+0000 7fe3017fa640 1 -- 172.21.15.67:0/2096432691 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc004b20 con 0x7fe304106630 2024-07-31T15:03:27.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.482+0000 7fe3017fa640 1 -- 172.21.15.67:0/2096432691 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2fc004cc0 con 0x7fe304106630 2024-07-31T15:03:27.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.482+0000 7fe3017fa640 1 -- 172.21.15.67:0/2096432691 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc010670 con 0x7fe304106630 2024-07-31T15:03:27.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.482+0000 7fe30a355640 1 -- 172.21.15.67:0/2096432691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3041a0150 con 0x7fe304106630 2024-07-31T15:03:27.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.482+0000 7fe30a355640 1 -- 172.21.15.67:0/2096432691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3041a68b0 con 0x7fe304106630 2024-07-31T15:03:27.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.485+0000 7fe3017fa640 1 -- 172.21.15.67:0/2096432691 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe2fc005020 con 0x7fe304106630 2024-07-31T15:03:27.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.486+0000 7fe3017fa640 1 --2- 172.21.15.67:0/2096432691 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2ec0b3dc0 0x7fe2ec0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.486+0000 7fe3108a6740 1 Processor -- start 2024-07-31T15:03:27.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.486+0000 7fe3108a6740 1 -- start start 2024-07-31T15:03:27.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.486+0000 7fe303fff640 1 --2- 172.21.15.67:0/2096432691 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2ec0b3dc0 0x7fe2ec0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.486+0000 7fe3017fa640 1 -- 172.21.15.67:0/2096432691 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9395+0+0 (secure 0 0 0) 0x7fe2fc0e4840 con 0x7fe304106630 2024-07-31T15:03:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe3108a6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f73151a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe3108a6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f73151aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe3108a6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 0x55f73151aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe3108a6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f731490bb0 con 0x55f731175070 2024-07-31T15:03:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe3108a6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f7314800b0 con 0x55f73114c1b0 2024-07-31T15:03:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe3108a6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f7314840f0 con 0x55f731154390 2024-07-31T15:03:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe303fff640 1 --2- 172.21.15.67:0/2096432691 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2ec0b3dc0 0x7fe2ec0b62d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4005d00 tx=0x7fe2f400b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe303fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f73151a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe3037fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f73151aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe303fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f73151a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53070/0 (socket says 172.21.15.67:53070) 2024-07-31T15:03:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe303fff640 1 -- 172.21.15.67:0/164530519 learned_addr learned my addr 172.21.15.67:0/164530519 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe3037fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f73151aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:43792/0 (socket says 172.21.15.67:43792) 2024-07-31T15:03:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe3088f2640 1 --2- 172.21.15.67:0/164530519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 0x55f73151aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe303fff640 1 -- 172.21.15.67:0/164530519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 msgr2=0x55f73151aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe303fff640 1 --2- 172.21.15.67:0/164530519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 0x55f73151aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe303fff640 1 -- 172.21.15.67:0/164530519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 msgr2=0x55f73151aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe303fff640 1 --2- 172.21.15.67:0/164530519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f73151aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe303fff640 1 -- 172.21.15.67:0/164530519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f731426b90 con 0x55f731175070 2024-07-31T15:03:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.487+0000 7fe3037fe640 1 --2- 172.21.15.67:0/164530519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f73151aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:03:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.488+0000 7fe303fff640 1 --2- 172.21.15.67:0/164530519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f73151a570 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40114a0 tx=0x7fe2f4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd673e5514c19b48 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.488+0000 7fe2ea7fc640 1 -- 172.21.15.67:0/164530519 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f401e620 con 0x55f731175070 2024-07-31T15:03:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.488+0000 7fe2ea7fc640 1 -- 172.21.15.67:0/164530519 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2f401ec70 con 0x55f731175070 2024-07-31T15:03:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.488+0000 7fe2ea7fc640 1 -- 172.21.15.67:0/164530519 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f401dd20 con 0x55f731175070 2024-07-31T15:03:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.489+0000 7fe3108a6740 1 -- 172.21.15.67:0/164530519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 msgr2=0x55f73151a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.489+0000 7fe3108a6740 1 --2- 172.21.15.67:0/164530519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f73151a570 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40114a0 tx=0x7fe2f4013000 comp rx=0 tx=0).stop 2024-07-31T15:03:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.489+0000 7fe3108a6740 1 -- 172.21.15.67:0/164530519 shutdown_connections 2024-07-31T15:03:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.489+0000 7fe3108a6740 1 --2- 172.21.15.67:0/164530519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 0x55f73151aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.489+0000 7fe3108a6740 1 --2- 172.21.15.67:0/164530519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f73151aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.489+0000 7fe3108a6740 1 --2- 172.21.15.67:0/164530519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f73151a570 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.489+0000 7fe3108a6740 1 -- 172.21.15.67:0/164530519 >> 172.21.15.67:0/164530519 conn(0x55f731169460 msgr2=0x55f731518460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.489+0000 7fe3108a6740 1 -- 172.21.15.67:0/164530519 shutdown_connections 2024-07-31T15:03:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.489+0000 7fe3108a6740 1 -- 172.21.15.67:0/164530519 wait complete. 2024-07-31T15:03:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.489+0000 7fe3108a6740 1 -- start start 2024-07-31T15:03:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.490+0000 7fe3108a6740 1 Processor -- start 2024-07-31T15:03:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.490+0000 7fe3108a6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f73152a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.490+0000 7fe3108a6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 0x55f73152a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.490+0000 7fe3108a6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f731536410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.490+0000 7fe3108a6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f7312e3700 con 0x55f731175070 2024-07-31T15:03:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.490+0000 7fe3108a6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f7314229e0 con 0x55f73114c1b0 2024-07-31T15:03:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.490+0000 7fe3108a6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f73152d080 con 0x55f731154390 2024-07-31T15:03:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.490+0000 7fe3108a6740 10 client.0 _ll_register_callbacks cb 0x55f73143d4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.490+0000 7fe3088f2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f731536410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.491+0000 7fe3088f2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f731536410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53080/0 (socket says 172.21.15.67:53080) 2024-07-31T15:03:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.491+0000 7fe3088f2640 1 -- 172.21.15.67:0/138095471 learned_addr learned my addr 172.21.15.67:0/138095471 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.491+0000 7fe3037fe640 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 0x55f73152a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.491+0000 7fe3088f2640 1 -- 172.21.15.67:0/138095471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 msgr2=0x55f73152a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.491+0000 7fe3088f2640 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 0x55f73152a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.491+0000 7fe303fff640 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f73152a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.491+0000 7fe3088f2640 1 -- 172.21.15.67:0/138095471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 msgr2=0x55f73152a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.491+0000 7fe3088f2640 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f73152a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.491+0000 7fe3088f2640 1 -- 172.21.15.67:0/138095471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7311e0650 con 0x55f731175070 2024-07-31T15:03:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.491+0000 7fe3037fe640 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 0x55f73152a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:03:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.491+0000 7fe303fff640 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f73152a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:03:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.491+0000 7fe3088f2640 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f731536410 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc0aade0 tx=0x7fe2fc0055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a1105213ec7fe5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.491+0000 7fe2e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.491+0000 7fe2e37fe640 1 -- 172.21.15.67:0/138095471 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc092af0 con 0x55f731175070 2024-07-31T15:03:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.492+0000 7fe2e37fe640 1 -- 172.21.15.67:0/138095471 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9395+0+0 (secure 0 0 0) 0x7fe2fc0b3d00 con 0x55f731175070 2024-07-31T15:03:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.492+0000 7fe3108a6740 1 -- 172.21.15.67:0/138095471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2f401ec70 con 0x55f731175070 2024-07-31T15:03:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.492+0000 7fe2e37fe640 1 -- 172.21.15.67:0/138095471 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fe2fc0aabf0 con 0x55f731175070 2024-07-31T15:03:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.492+0000 7fe3108a6740 10 client.18111 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.492+0000 7fe3108a6740 1 -- 172.21.15.67:0/138095471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f7315390e0 con 0x55f731175070 2024-07-31T15:03:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.492+0000 7fe2e37fe640 1 -- 172.21.15.67:0/138095471 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2f401ec70 con 0x55f731175070 2024-07-31T15:03:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.492+0000 7fe2e37fe640 1 -- 172.21.15.67:0/138095471 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fe2fc0af170 con 0x55f731175070 2024-07-31T15:03:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.492+0000 7fe3108a6740 10 client.18111 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.492+0000 7fe3108a6740 10 client.18111 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:03:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.492+0000 7fe3108a6740 20 client.18111 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.492+0000 7fe3108a6740 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f7311339f0 0x55f73153c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.492+0000 7fe3108a6740 4 client.18111 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:03:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.492+0000 7fe3108a6740 1 -- 172.21.15.67:0/138095471 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f731492b00 con 0x55f7311339f0 2024-07-31T15:03:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.493+0000 7fe303fff640 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f7311339f0 0x55f73153c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.493+0000 7fe303fff640 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f7311339f0 0x55f73153c630 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9cef0daf311d2fc0 server_cookie=e652c0c16c927ad3 in_seq=0 out_seq=0 2024-07-31T15:03:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.493+0000 7fe2e37fe640 10 client.18111 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:03:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.494+0000 7fe2e37fe640 1 -- 172.21.15.67:0/138095471 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f731492b00 con 0x55f7311339f0 2024-07-31T15:03:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.494+0000 7fe2e37fe640 10 client.18111 handle_command_reply: tid=0 2024-07-31T15:03:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.494+0000 7fe3108a6740 1 client.18111 shutdown 2024-07-31T15:03:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.494+0000 7fe3108a6740 1 -- 172.21.15.67:0/138095471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 msgr2=0x55f731536410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.494+0000 7fe3108a6740 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f731536410 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc0aade0 tx=0x7fe2fc0055c0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.494+0000 7fe3108a6740 1 -- 172.21.15.67:0/138095471 shutdown_connections 2024-07-31T15:03:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.494+0000 7fe3108a6740 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f7311339f0 0x55f73153c630 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.494+0000 7fe3108a6740 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f731536410 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.494+0000 7fe3108a6740 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 0x55f73152a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.494+0000 7fe3108a6740 1 --2- 172.21.15.67:0/138095471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f73152a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.494+0000 7fe2e37fe640 0 client.18111 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:03:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.494+0000 7fe3108a6740 1 -- 172.21.15.67:0/138095471 >> 172.21.15.67:0/138095471 conn(0x55f731169460 msgr2=0x55f731516c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:27.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.494+0000 7fe3108a6740 1 -- 172.21.15.67:0/138095471 shutdown_connections 2024-07-31T15:03:27.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.494+0000 7fe3108a6740 1 -- 172.21.15.67:0/138095471 wait complete. 2024-07-31T15:03:27.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.495+0000 7fe3108a6740 20 client.18111 trim_cache size 0 max 16384 2024-07-31T15:03:27.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.510+0000 7fe3108a6740 1 Processor -- start 2024-07-31T15:03:27.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.511+0000 7fe3108a6740 1 -- start start 2024-07-31T15:03:27.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.511+0000 7fe3108a6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f73114c1b0 0x55f73152c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.511+0000 7fe3108a6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f731154390 0x55f731536200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.511+0000 7fe3108a6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731175070 0x55f731536740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.511+0000 7fe3108a6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe3041ae510 con 0x55f73114c1b0 2024-07-31T15:03:27.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.511+0000 7fe3108a6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe3041b0050 con 0x55f731154390 2024-07-31T15:03:27.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.511+0000 7fe3108a6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe3041aebe0 con 0x55f731175070 2024-07-31T15:03:27.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.511+0000 7fe303fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f73114c1b0 0x55f73152c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.511+0000 7fe303fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f73114c1b0 0x55f73152c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53084/0 (socket says 172.21.15.67:53084) 2024-07-31T15:03:27.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.511+0000 7fe303fff640 1 -- 172.21.15.67:0/3260455397 learned_addr learned my addr 172.21.15.67:0/3260455397 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:27.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.511+0000 7fe3037fe640 1 --2- 172.21.15.67:0/3260455397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f731154390 0x55f731536200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.511+0000 7fe3088f2640 1 --2- 172.21.15.67:0/3260455397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731175070 0x55f731536740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.512+0000 7fe3037fe640 1 -- 172.21.15.67:0/3260455397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731175070 msgr2=0x55f731536740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:27.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.512+0000 7fe3037fe640 1 --2- 172.21.15.67:0/3260455397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731175070 0x55f731536740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.512+0000 7fe3037fe640 1 -- 172.21.15.67:0/3260455397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f73114c1b0 msgr2=0x55f73152c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:27.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.512+0000 7fe3037fe640 1 --2- 172.21.15.67:0/3260455397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f73114c1b0 0x55f73152c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.512+0000 7fe3037fe640 1 -- 172.21.15.67:0/3260455397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73152b540 con 0x55f731154390 2024-07-31T15:03:27.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.512+0000 7fe303fff640 1 --2- 172.21.15.67:0/3260455397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f73114c1b0 0x55f73152c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:03:27.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.512+0000 7fe3037fe640 1 --2- 172.21.15.67:0/3260455397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f731154390 0x55f731536200 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe2f802a930 tx=0x7fe2f8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=434598360f02ed36 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:27.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.512+0000 7fe2e9ffb640 1 -- 172.21.15.67:0/3260455397 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8035040 con 0x55f731154390 2024-07-31T15:03:27.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.512+0000 7fe2e9ffb640 1 -- 172.21.15.67:0/3260455397 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2f80316a0 con 0x55f731154390 2024-07-31T15:03:27.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.512+0000 7fe2e9ffb640 1 -- 172.21.15.67:0/3260455397 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8046df0 con 0x55f731154390 2024-07-31T15:03:27.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.513+0000 7fe3108a6740 1 -- 172.21.15.67:0/3260455397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f731154390 msgr2=0x55f731536200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:27.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.513+0000 7fe3108a6740 1 --2- 172.21.15.67:0/3260455397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f731154390 0x55f731536200 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe2f802a930 tx=0x7fe2f8000bc0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.513+0000 7fe3108a6740 1 -- 172.21.15.67:0/3260455397 shutdown_connections 2024-07-31T15:03:27.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.513+0000 7fe3108a6740 1 --2- 172.21.15.67:0/3260455397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731175070 0x55f731536740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.513+0000 7fe3108a6740 1 --2- 172.21.15.67:0/3260455397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f731154390 0x55f731536200 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.513+0000 7fe3108a6740 1 --2- 172.21.15.67:0/3260455397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f73114c1b0 0x55f73152c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.513+0000 7fe3108a6740 1 -- 172.21.15.67:0/3260455397 >> 172.21.15.67:0/3260455397 conn(0x55f731169460 msgr2=0x55f73151bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.513+0000 7fe3108a6740 1 -- 172.21.15.67:0/3260455397 shutdown_connections 2024-07-31T15:03:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.514+0000 7fe3108a6740 1 -- 172.21.15.67:0/3260455397 wait complete. 2024-07-31T15:03:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.514+0000 7fe3108a6740 1 -- start start 2024-07-31T15:03:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.514+0000 7fe3108a6740 1 Processor -- start 2024-07-31T15:03:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.515+0000 7fe3108a6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f731520e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.515+0000 7fe3108a6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 0x55f731521390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.515+0000 7fe3108a6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f7315218d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.515+0000 7fe3108a6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe3041af980 con 0x55f731175070 2024-07-31T15:03:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.515+0000 7fe3108a6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe3041af2b0 con 0x55f73114c1b0 2024-07-31T15:03:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.515+0000 7fe3108a6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe3041ac510 con 0x55f731154390 2024-07-31T15:03:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.515+0000 7fe3088f2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f7315218d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.515+0000 7fe3088f2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f7315218d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53086/0 (socket says 172.21.15.67:53086) 2024-07-31T15:03:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.515+0000 7fe3088f2640 1 -- 172.21.15.67:0/3142805217 learned_addr learned my addr 172.21.15.67:0/3142805217 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.515+0000 7fe303fff640 1 --2- 172.21.15.67:0/3142805217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f731520e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.515+0000 7fe3108a6740 10 client.0 _ll_register_callbacks cb 0x7fe2fc004ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.515+0000 7fe3037fe640 1 --2- 172.21.15.67:0/3142805217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 0x55f731521390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.516+0000 7fe3088f2640 1 -- 172.21.15.67:0/3142805217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 msgr2=0x55f731521390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.516+0000 7fe3088f2640 1 --2- 172.21.15.67:0/3142805217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 0x55f731521390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.516+0000 7fe3088f2640 1 -- 172.21.15.67:0/3142805217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 msgr2=0x55f731520e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.516+0000 7fe3088f2640 1 --2- 172.21.15.67:0/3142805217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f731520e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.516+0000 7fe3088f2640 1 -- 172.21.15.67:0/3142805217 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f731531360 con 0x55f731175070 2024-07-31T15:03:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.516+0000 7fe3037fe640 1 --2- 172.21.15.67:0/3142805217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 0x55f731521390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:03:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.516+0000 7fe3088f2640 1 --2- 172.21.15.67:0/3142805217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f7315218d0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc0af510 tx=0x7fe2fc00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c85251ec087f3f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.516+0000 7fe2e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.516+0000 7fe2e37fe640 1 -- 172.21.15.67:0/3142805217 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc08c060 con 0x55f731175070 2024-07-31T15:03:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.516+0000 7fe2e37fe640 1 -- 172.21.15.67:0/3142805217 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9395+0+0 (secure 0 0 0) 0x7fe2fc0b3d00 con 0x55f731175070 2024-07-31T15:03:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.516+0000 7fe3108a6740 1 -- 172.21.15.67:0/3142805217 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2f80316a0 con 0x55f731175070 2024-07-31T15:03:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.516+0000 7fe2e37fe640 1 -- 172.21.15.67:0/3142805217 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fe2fc0ab7e0 con 0x55f731175070 2024-07-31T15:03:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.516+0000 7fe3108a6740 10 client.18117 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.516+0000 7fe3108a6740 1 -- 172.21.15.67:0/3142805217 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f731532530 con 0x55f731175070 2024-07-31T15:03:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.517+0000 7fe2e37fe640 1 -- 172.21.15.67:0/3142805217 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2f80316a0 con 0x55f731175070 2024-07-31T15:03:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.517+0000 7fe2e37fe640 1 -- 172.21.15.67:0/3142805217 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fe2fc0a6a50 con 0x55f731175070 2024-07-31T15:03:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.517+0000 7fe3108a6740 10 client.18117 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.517+0000 7fe3108a6740 10 client.18117 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:03:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.517+0000 7fe3108a6740 20 client.18117 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.517+0000 7fe3108a6740 1 --2- 172.21.15.67:0/3142805217 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f731108da0 0x55f7315a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.517+0000 7fe3108a6740 4 client.18117 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:03:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.518+0000 7fe3108a6740 1 -- 172.21.15.67:0/3142805217 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f73148e5a0 con 0x55f731108da0 2024-07-31T15:03:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.518+0000 7fe303fff640 1 --2- 172.21.15.67:0/3142805217 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f731108da0 0x55f7315a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.518+0000 7fe2e37fe640 10 client.18117 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:03:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.518+0000 7fe303fff640 1 --2- 172.21.15.67:0/3142805217 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f731108da0 0x55f7315a2db0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c39837cdee91a305 server_cookie=31f4ae99a613bb69 in_seq=0 out_seq=0 2024-07-31T15:03:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.519+0000 7fe2e37fe640 1 -- 172.21.15.67:0/3142805217 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f73148e5a0 con 0x55f731108da0 2024-07-31T15:03:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.519+0000 7fe2e37fe640 10 client.18117 handle_command_reply: tid=0 2024-07-31T15:03:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.519+0000 7fe3108a6740 1 client.18117 shutdown 2024-07-31T15:03:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.519+0000 7fe3108a6740 1 -- 172.21.15.67:0/3142805217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 msgr2=0x55f7315218d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.519+0000 7fe3108a6740 1 --2- 172.21.15.67:0/3142805217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f7315218d0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc0af510 tx=0x7fe2fc00cee0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.519+0000 7fe3108a6740 1 -- 172.21.15.67:0/3142805217 shutdown_connections 2024-07-31T15:03:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.519+0000 7fe3108a6740 1 --2- 172.21.15.67:0/3142805217 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f731108da0 0x55f7315a2db0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.519+0000 7fe3108a6740 1 --2- 172.21.15.67:0/3142805217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f731175070 0x55f7315218d0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.519+0000 7fe2e37fe640 0 client.18117 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:03:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.519+0000 7fe3108a6740 1 --2- 172.21.15.67:0/3142805217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f731154390 0x55f731521390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.519+0000 7fe3108a6740 1 --2- 172.21.15.67:0/3142805217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f73114c1b0 0x55f731520e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.519+0000 7fe3108a6740 1 -- 172.21.15.67:0/3142805217 >> 172.21.15.67:0/3142805217 conn(0x55f731169460 msgr2=0x55f73151b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:27.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.520+0000 7fe3108a6740 1 -- 172.21.15.67:0/3142805217 shutdown_connections 2024-07-31T15:03:27.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.520+0000 7fe3108a6740 1 -- 172.21.15.67:0/3142805217 wait complete. 2024-07-31T15:03:27.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.520+0000 7fe3108a6740 20 client.18117 trim_cache size 0 max 16384 2024-07-31T15:03:27.527 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:03:27.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.521+0000 7fe30a355640 1 -- 172.21.15.67:0/2096432691 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2ec0b3dc0 msgr2=0x7fe2ec0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:27.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.521+0000 7fe30a355640 1 --2- 172.21.15.67:0/2096432691 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2ec0b3dc0 0x7fe2ec0b62d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4005d00 tx=0x7fe2f400b410 comp rx=0 tx=0).stop 2024-07-31T15:03:27.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.521+0000 7fe30a355640 1 -- 172.21.15.67:0/2096432691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe304106630 msgr2=0x7fe30419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:27.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.521+0000 7fe30a355640 1 --2- 172.21.15.67:0/2096432691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe304106630 0x7fe30419f730 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00eb10 tx=0x7fe2fc00c2d0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.521+0000 7fe30a355640 1 -- 172.21.15.67:0/2096432691 shutdown_connections 2024-07-31T15:03:27.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.521+0000 7fe30a355640 1 --2- 172.21.15.67:0/2096432691 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2ec0b3dc0 0x7fe2ec0b62d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.521+0000 7fe30a355640 1 --2- 172.21.15.67:0/2096432691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe304106630 0x7fe30419f730 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.521+0000 7fe30a355640 1 --2- 172.21.15.67:0/2096432691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe304105ce0 0x7fe30419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.521+0000 7fe30a355640 1 --2- 172.21.15.67:0/2096432691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe304101a00 0x7fe304199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:27.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.521+0000 7fe30a355640 1 -- 172.21.15.67:0/2096432691 >> 172.21.15.67:0/2096432691 conn(0x7fe3040776f0 msgr2=0x7fe304107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:27.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.521+0000 7fe30a355640 1 -- 172.21.15.67:0/2096432691 shutdown_connections 2024-07-31T15:03:27.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:27.521+0000 7fe30a355640 1 -- 172.21.15.67:0/2096432691 wait complete. 2024-07-31T15:03:27.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:27 smithi073 ceph-mon[36714]: pgmap v790: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 188 KiB/s wr, 37 op/s 2024-07-31T15:03:28.969 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T15:03:28.974 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:03:29.238 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:29 smithi067 ceph-mon[32750]: pgmap v791: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 190 KiB/s wr, 39 op/s 2024-07-31T15:03:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:29 smithi077 ceph-mon[37327]: pgmap v791: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 190 KiB/s wr, 39 op/s 2024-07-31T15:03:29.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:29 smithi073 ceph-mon[36714]: pgmap v791: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 190 KiB/s wr, 39 op/s 2024-07-31T15:03:30.168 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:03:31.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.313+0000 7feb16949640 1 -- 172.21.15.67:0/3338825543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb10108720 msgr2=0x7feb1010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.313+0000 7feb16949640 1 --2- 172.21.15.67:0/3338825543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb10108720 0x7feb1010ef50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7feb08009ca0 tx=0x7feb0802a8c0 comp rx=0 tx=0).stop 2024-07-31T15:03:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.313+0000 7feb16949640 1 -- 172.21.15.67:0/3338825543 shutdown_connections 2024-07-31T15:03:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.313+0000 7feb16949640 1 --2- 172.21.15.67:0/3338825543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb10108720 0x7feb1010ef50 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.313+0000 7feb16949640 1 --2- 172.21.15.67:0/3338825543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb10107dd0 0x7feb101081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.313+0000 7feb16949640 1 --2- 172.21.15.67:0/3338825543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb10102490 0x7feb101028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.313+0000 7feb16949640 1 -- 172.21.15.67:0/3338825543 >> 172.21.15.67:0/3338825543 conn(0x7feb10079160 msgr2=0x7feb10079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.313+0000 7feb16949640 1 -- 172.21.15.67:0/3338825543 shutdown_connections 2024-07-31T15:03:31.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.313+0000 7feb16949640 1 -- 172.21.15.67:0/3338825543 wait complete. 2024-07-31T15:03:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.314+0000 7feb16949640 1 Processor -- start 2024-07-31T15:03:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.314+0000 7feb16949640 1 -- start start 2024-07-31T15:03:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.314+0000 7feb16949640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb10102490 0x7feb1019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.314+0000 7feb16949640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb10107dd0 0x7feb1019b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.314+0000 7feb16949640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb10108720 0x7feb1019b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.314+0000 7feb16949640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feb101123a0 con 0x7feb10108720 2024-07-31T15:03:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.314+0000 7feb16949640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feb10112220 con 0x7feb10102490 2024-07-31T15:03:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.314+0000 7feb16949640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feb10112520 con 0x7feb10107dd0 2024-07-31T15:03:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.314+0000 7feb14ee6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb10108720 0x7feb1019b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.314+0000 7feb14ee6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb10108720 0x7feb1019b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39434/0 (socket says 172.21.15.67:39434) 2024-07-31T15:03:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.314+0000 7feb14ee6640 1 -- 172.21.15.67:0/1273559290 learned_addr learned my addr 172.21.15.67:0/1273559290 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.314+0000 7feb0f7fe640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb10107dd0 0x7feb1019b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.315+0000 7feb0ffff640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb10102490 0x7feb1019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.315+0000 7feb14ee6640 1 -- 172.21.15.67:0/1273559290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb10107dd0 msgr2=0x7feb1019b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.315+0000 7feb14ee6640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb10107dd0 0x7feb1019b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.315+0000 7feb14ee6640 1 -- 172.21.15.67:0/1273559290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb10102490 msgr2=0x7feb1019acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.315+0000 7feb14ee6640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb10102490 0x7feb1019acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.315+0000 7feb14ee6640 1 -- 172.21.15.67:0/1273559290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb101a5f40 con 0x7feb10108720 2024-07-31T15:03:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.315+0000 7feb0ffff640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb10102490 0x7feb1019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:03:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.315+0000 7feb14ee6640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb10108720 0x7feb1019b730 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7feb08005cc0 tx=0x7feb0802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.315+0000 7feb0d7fa640 1 -- 172.21.15.67:0/1273559290 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08003bf0 con 0x7feb10108720 2024-07-31T15:03:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.315+0000 7feb16949640 1 -- 172.21.15.67:0/1273559290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb101a61d0 con 0x7feb10108720 2024-07-31T15:03:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.315+0000 7feb0d7fa640 1 -- 172.21.15.67:0/1273559290 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb08003d90 con 0x7feb10108720 2024-07-31T15:03:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.315+0000 7feb0d7fa640 1 -- 172.21.15.67:0/1273559290 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0802fbc0 con 0x7feb10108720 2024-07-31T15:03:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.315+0000 7feb16949640 1 -- 172.21.15.67:0/1273559290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb101a6760 con 0x7feb10108720 2024-07-31T15:03:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.316+0000 7feb16949640 1 -- 172.21.15.67:0/1273559290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb101028a0 con 0x7feb10108720 2024-07-31T15:03:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.319+0000 7feb0d7fa640 1 -- 172.21.15.67:0/1273559290 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7feb0802fd60 con 0x7feb10108720 2024-07-31T15:03:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.319+0000 7feb0d7fa640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feaf80b3d70 0x7feaf80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.320+0000 7feb0ffff640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feaf80b3d70 0x7feaf80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.320+0000 7feb0d7fa640 1 -- 172.21.15.67:0/1273559290 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7feb08105920 con 0x7feb10108720 2024-07-31T15:03:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.320+0000 7feb0ffff640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feaf80b3d70 0x7feaf80b6280 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7feb00005d00 tx=0x7feb00005c90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.323+0000 7feb0d7fa640 1 -- 172.21.15.67:0/1273559290 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7feb080cca00 con 0x7feb10108720 2024-07-31T15:03:31.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:31 smithi073 ceph-mon[36714]: pgmap v792: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 219 KiB/s wr, 47 op/s 2024-07-31T15:03:31.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:31 smithi073 ceph-mon[36714]: osdmap e110: 12 total, 12 up, 12 in 2024-07-31T15:03:31.461 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:31 smithi067 ceph-mon[32750]: pgmap v792: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 219 KiB/s wr, 47 op/s 2024-07-31T15:03:31.461 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:31 smithi067 ceph-mon[32750]: osdmap e110: 12 total, 12 up, 12 in 2024-07-31T15:03:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:31 smithi077 ceph-mon[37327]: pgmap v792: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 219 KiB/s wr, 47 op/s 2024-07-31T15:03:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:31 smithi077 ceph-mon[37327]: osdmap e110: 12 total, 12 up, 12 in 2024-07-31T15:03:31.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.558+0000 7feb16949640 1 -- 172.21.15.67:0/1273559290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb1006aca0 con 0x7feb10108720 2024-07-31T15:03:31.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.559+0000 7feb0d7fa640 1 -- 172.21.15.67:0/1273559290 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+11959 (secure 0 0 0) 0x7feb1006aca0 con 0x7feb10108720 2024-07-31T15:03:31.561 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:03:31.561 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":63,"btime":"2024-07-31T15:03:07:099477+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:03:06.671820+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:03:31.561 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 63 2024-07-31T15:03:31.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.561+0000 7feb16949640 1 -- 172.21.15.67:0/1273559290 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feaf80b3d70 msgr2=0x7feaf80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:31.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.561+0000 7feb16949640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feaf80b3d70 0x7feaf80b6280 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7feb00005d00 tx=0x7feb00005c90 comp rx=0 tx=0).stop 2024-07-31T15:03:31.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.561+0000 7feb16949640 1 -- 172.21.15.67:0/1273559290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb10108720 msgr2=0x7feb1019b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:31.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.561+0000 7feb16949640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb10108720 0x7feb1019b730 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7feb08005cc0 tx=0x7feb0802f380 comp rx=0 tx=0).stop 2024-07-31T15:03:31.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.561+0000 7feb16949640 1 -- 172.21.15.67:0/1273559290 shutdown_connections 2024-07-31T15:03:31.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.562+0000 7feb16949640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feaf80b3d70 0x7feaf80b6280 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:31.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.562+0000 7feb16949640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb10108720 0x7feb1019b730 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:31.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.562+0000 7feb16949640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb10107dd0 0x7feb1019b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:31.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.562+0000 7feb16949640 1 --2- 172.21.15.67:0/1273559290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb10102490 0x7feb1019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:31.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.562+0000 7feb16949640 1 -- 172.21.15.67:0/1273559290 >> 172.21.15.67:0/1273559290 conn(0x7feb10079160 msgr2=0x7feb10104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:31.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.562+0000 7feb16949640 1 -- 172.21.15.67:0/1273559290 shutdown_connections 2024-07-31T15:03:31.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:31.562+0000 7feb16949640 1 -- 172.21.15.67:0/1273559290 wait complete. 2024-07-31T15:03:32.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:32 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1273559290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:03:32.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:32 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1273559290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:03:32.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:32 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1273559290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:03:32.655 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 54db3a03 --release --await' 2024-07-31T15:03:32.656 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 54db3a03 --release --await 2024-07-31T15:03:33.202 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:33 smithi067 ceph-mon[32750]: pgmap v794: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 245 KiB/s wr, 42 op/s 2024-07-31T15:03:33.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.200+0000 7febee4e8640 1 -- 172.21.15.67:0/1966452208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febe8109850 msgr2=0x7febe8110080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.200+0000 7febee4e8640 1 --2- 172.21.15.67:0/1966452208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febe8109850 0x7febe8110080 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7febe0009ca0 tx=0x7febe002a8c0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.200+0000 7febee4e8640 1 -- 172.21.15.67:0/1966452208 shutdown_connections 2024-07-31T15:03:33.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.200+0000 7febee4e8640 1 --2- 172.21.15.67:0/1966452208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febe8109850 0x7febe8110080 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.200+0000 7febee4e8640 1 --2- 172.21.15.67:0/1966452208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febe8108f00 0x7febe8109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.200+0000 7febee4e8640 1 --2- 172.21.15.67:0/1966452208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febe81035c0 0x7febe81039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.200+0000 7febee4e8640 1 -- 172.21.15.67:0/1966452208 >> 172.21.15.67:0/1966452208 conn(0x7febe80ff030 msgr2=0x7febe8101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.200+0000 7febee4e8640 1 -- 172.21.15.67:0/1966452208 shutdown_connections 2024-07-31T15:03:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.200+0000 7febee4e8640 1 -- 172.21.15.67:0/1966452208 wait complete. 2024-07-31T15:03:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.201+0000 7febee4e8640 1 Processor -- start 2024-07-31T15:03:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.201+0000 7febee4e8640 1 -- start start 2024-07-31T15:03:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febee4e8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febe81035c0 0x7febe819c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febee4e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febe8108f00 0x7febe819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febee4e8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febe8109850 0x7febe81a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febee4e8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7febe8113550 con 0x7febe8109850 2024-07-31T15:03:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febee4e8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7febe81133d0 con 0x7febe81035c0 2024-07-31T15:03:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febee4e8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7febe81136d0 con 0x7febe8108f00 2024-07-31T15:03:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febeca85640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febe8109850 0x7febe81a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febe7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febe81035c0 0x7febe819c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febeca85640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febe8109850 0x7febe81a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39462/0 (socket says 172.21.15.67:39462) 2024-07-31T15:03:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febeca85640 1 -- 172.21.15.67:0/873009221 learned_addr learned my addr 172.21.15.67:0/873009221 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febe77fe640 1 --2- 172.21.15.67:0/873009221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febe8108f00 0x7febe819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febe7fff640 1 -- 172.21.15.67:0/873009221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febe8108f00 msgr2=0x7febe819c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febe7fff640 1 --2- 172.21.15.67:0/873009221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febe8108f00 0x7febe819c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febe7fff640 1 -- 172.21.15.67:0/873009221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febe8109850 msgr2=0x7febe81a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febe7fff640 1 --2- 172.21.15.67:0/873009221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febe8109850 0x7febe81a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febe7fff640 1 -- 172.21.15.67:0/873009221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe81a1ef0 con 0x7febe81035c0 2024-07-31T15:03:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.202+0000 7febeca85640 1 --2- 172.21.15.67:0/873009221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febe8109850 0x7febe81a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:03:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.203+0000 7febe7fff640 1 --2- 172.21.15.67:0/873009221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febe81035c0 0x7febe819c040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7febd800e9f0 tx=0x7febd800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.203+0000 7febe57fa640 1 -- 172.21.15.67:0/873009221 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd800ccc0 con 0x7febe81035c0 2024-07-31T15:03:33.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.203+0000 7febee4e8640 1 -- 172.21.15.67:0/873009221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe81a21e0 con 0x7febe81035c0 2024-07-31T15:03:33.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.203+0000 7febe57fa640 1 -- 172.21.15.67:0/873009221 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd80108e0 con 0x7febe81035c0 2024-07-31T15:03:33.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.203+0000 7febe57fa640 1 -- 172.21.15.67:0/873009221 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8018840 con 0x7febe81035c0 2024-07-31T15:03:33.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.203+0000 7febee4e8640 1 -- 172.21.15.67:0/873009221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe81a8a00 con 0x7febe81035c0 2024-07-31T15:03:33.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.206+0000 7febe57fa640 1 -- 172.21.15.67:0/873009221 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7febd80189e0 con 0x7febe81035c0 2024-07-31T15:03:33.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.206+0000 7febf4a39740 1 Processor -- start 2024-07-31T15:03:33.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.206+0000 7febf4a39740 1 -- start start 2024-07-31T15:03:33.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.206+0000 7febf4a39740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23d0b070 0x564e240b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.206+0000 7febf4a39740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23ce21b0 0x564e240b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.206+0000 7febf4a39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23cea390 0x564e240b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.206+0000 7febf4a39740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564e24026bb0 con 0x564e23d0b070 2024-07-31T15:03:33.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.206+0000 7febf4a39740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564e240160b0 con 0x564e23ce21b0 2024-07-31T15:03:33.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.206+0000 7febf4a39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564e2401a0f0 con 0x564e23cea390 2024-07-31T15:03:33.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.206+0000 7febeca85640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23d0b070 0x564e240b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.206+0000 7febeca85640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23d0b070 0x564e240b0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39478/0 (socket says 172.21.15.67:39478) 2024-07-31T15:03:33.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.206+0000 7febeca85640 1 -- 172.21.15.67:0/3582600790 learned_addr learned my addr 172.21.15.67:0/3582600790 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:33.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.206+0000 7febe7fff640 1 --2- 172.21.15.67:0/3582600790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23ce21b0 0x564e240b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.206+0000 7febe77fe640 1 --2- 172.21.15.67:0/3582600790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23cea390 0x564e240b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.207+0000 7febe77fe640 1 -- 172.21.15.67:0/3582600790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23ce21b0 msgr2=0x564e240b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.207+0000 7febe77fe640 1 --2- 172.21.15.67:0/3582600790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23ce21b0 0x564e240b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.207+0000 7febe77fe640 1 -- 172.21.15.67:0/3582600790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23d0b070 msgr2=0x564e240b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.207+0000 7febe77fe640 1 --2- 172.21.15.67:0/3582600790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23d0b070 0x564e240b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.207+0000 7febe77fe640 1 -- 172.21.15.67:0/3582600790 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e23fbcb90 con 0x564e23cea390 2024-07-31T15:03:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.207+0000 7febe57fa640 1 --2- 172.21.15.67:0/873009221 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7febd00b3d70 0x7febd00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.207+0000 7febeca85640 1 --2- 172.21.15.67:0/3582600790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23d0b070 0x564e240b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:03:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.207+0000 7febeca85640 1 --2- 172.21.15.67:0/873009221 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7febd00b3d70 0x7febd00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.207+0000 7febe57fa640 1 -- 172.21.15.67:0/873009221 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7febd80e45b0 con 0x7febe81035c0 2024-07-31T15:03:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.207+0000 7febe77fe640 1 --2- 172.21.15.67:0/3582600790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23cea390 0x564e240b0ff0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7febe806b4e0 tx=0x7febdc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94539b8128c059fa server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.208+0000 7febce7fc640 1 -- 172.21.15.67:0/3582600790 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc005220 con 0x564e23cea390 2024-07-31T15:03:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.208+0000 7febeca85640 1 --2- 172.21.15.67:0/873009221 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7febd00b3d70 0x7febd00b6280 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7febe00094e0 tx=0x7febe0005d80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.208+0000 7febce7fc640 1 -- 172.21.15.67:0/3582600790 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febdc0047a0 con 0x564e23cea390 2024-07-31T15:03:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.208+0000 7febce7fc640 1 -- 172.21.15.67:0/3582600790 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc0186a0 con 0x564e23cea390 2024-07-31T15:03:33.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.209+0000 7febf4a39740 1 -- 172.21.15.67:0/3582600790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23cea390 msgr2=0x564e240b0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.209+0000 7febf4a39740 1 --2- 172.21.15.67:0/3582600790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23cea390 0x564e240b0ff0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7febe806b4e0 tx=0x7febdc00f790 comp rx=0 tx=0).stop 2024-07-31T15:03:33.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.209+0000 7febf4a39740 1 -- 172.21.15.67:0/3582600790 shutdown_connections 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.209+0000 7febf4a39740 1 --2- 172.21.15.67:0/3582600790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23cea390 0x564e240b0ff0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.209+0000 7febf4a39740 1 --2- 172.21.15.67:0/3582600790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23ce21b0 0x564e240b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.209+0000 7febf4a39740 1 --2- 172.21.15.67:0/3582600790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23d0b070 0x564e240b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.209+0000 7febf4a39740 1 -- 172.21.15.67:0/3582600790 >> 172.21.15.67:0/3582600790 conn(0x564e23cff460 msgr2=0x564e240ae460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.209+0000 7febf4a39740 1 -- 172.21.15.67:0/3582600790 shutdown_connections 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.209+0000 7febf4a39740 1 -- 172.21.15.67:0/3582600790 wait complete. 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.210+0000 7febf4a39740 1 -- start start 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.210+0000 7febf4a39740 1 Processor -- start 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.210+0000 7febf4a39740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23ce21b0 0x564e240bc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.210+0000 7febf4a39740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23cea390 0x564e240bc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.210+0000 7febf4a39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23d0b070 0x564e240c8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.210+0000 7febf4a39740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564e23e79700 con 0x564e23ce21b0 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.210+0000 7febf4a39740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564e23fb89e0 con 0x564e23cea390 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.210+0000 7febf4a39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564e240bed50 con 0x564e23d0b070 2024-07-31T15:03:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.210+0000 7febf4a39740 10 client.0 _ll_register_callbacks cb 0x564e23fd34f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.210+0000 7febe7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23ce21b0 0x564e240bc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.210+0000 7febe7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23ce21b0 0x564e240bc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39482/0 (socket says 172.21.15.67:39482) 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.210+0000 7febe7fff640 1 -- 172.21.15.67:0/4203301122 learned_addr learned my addr 172.21.15.67:0/4203301122 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.210+0000 7febeca85640 1 --2- 172.21.15.67:0/4203301122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23d0b070 0x564e240c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.211+0000 7febe77fe640 1 --2- 172.21.15.67:0/4203301122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23cea390 0x564e240bc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.211+0000 7febe7fff640 1 -- 172.21.15.67:0/4203301122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23d0b070 msgr2=0x564e240c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.211+0000 7febe7fff640 1 --2- 172.21.15.67:0/4203301122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23d0b070 0x564e240c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.211+0000 7febe7fff640 1 -- 172.21.15.67:0/4203301122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23cea390 msgr2=0x564e240bc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.211+0000 7febe7fff640 1 --2- 172.21.15.67:0/4203301122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23cea390 0x564e240bc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.211+0000 7febe7fff640 1 -- 172.21.15.67:0/4203301122 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e23d76650 con 0x564e23ce21b0 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.211+0000 7febe7fff640 1 --2- 172.21.15.67:0/4203301122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23ce21b0 0x564e240bc100 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7febd800c6a0 tx=0x7febd80aac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fab93aff4416be server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.211+0000 7febc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.211+0000 7febc77fe640 1 -- 172.21.15.67:0/4203301122 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd80aa5f0 con 0x564e23ce21b0 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.211+0000 7febc77fe640 1 -- 172.21.15.67:0/4203301122 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7febd80e6ba0 con 0x564e23ce21b0 2024-07-31T15:03:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.211+0000 7febf4a39740 1 -- 172.21.15.67:0/4203301122 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febdc0047a0 con 0x564e23ce21b0 2024-07-31T15:03:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.211+0000 7febc77fe640 1 -- 172.21.15.67:0/4203301122 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7febd80aa990 con 0x564e23ce21b0 2024-07-31T15:03:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.211+0000 7febf4a39740 10 client.18135 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.211+0000 7febf4a39740 1 -- 172.21.15.67:0/4203301122 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x564e240cad80 con 0x564e23ce21b0 2024-07-31T15:03:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.212+0000 7febc77fe640 1 -- 172.21.15.67:0/4203301122 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febdc0047a0 con 0x564e23ce21b0 2024-07-31T15:03:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.212+0000 7febc77fe640 1 -- 172.21.15.67:0/4203301122 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7febd80e41f0 con 0x564e23ce21b0 2024-07-31T15:03:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.212+0000 7febf4a39740 10 client.18135 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.212+0000 7febf4a39740 10 client.18135 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:03:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.212+0000 7febf4a39740 20 client.18135 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.212+0000 7febf4a39740 1 --2- 172.21.15.67:0/4203301122 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564e23cc99f0 0x564e240ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.212+0000 7febf4a39740 4 client.18135 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:03:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.212+0000 7febf4a39740 1 -- 172.21.15.67:0/4203301122 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e24028b00 con 0x564e23cc99f0 2024-07-31T15:03:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.212+0000 7febe77fe640 1 --2- 172.21.15.67:0/4203301122 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564e23cc99f0 0x564e240ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.213+0000 7febe77fe640 1 --2- 172.21.15.67:0/4203301122 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564e23cc99f0 0x564e240ce280 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97558ff60768e5e2 server_cookie=19e352d94f1b9b94 in_seq=0 out_seq=0 2024-07-31T15:03:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.213+0000 7febc77fe640 10 client.18135 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:03:33.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.213+0000 7febc77fe640 1 -- 172.21.15.67:0/4203301122 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e24028b00 con 0x564e23cc99f0 2024-07-31T15:03:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.213+0000 7febc77fe640 10 client.18135 handle_command_reply: tid=0 2024-07-31T15:03:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.213+0000 7febf4a39740 1 client.18135 shutdown 2024-07-31T15:03:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.214+0000 7febf4a39740 1 -- 172.21.15.67:0/4203301122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23ce21b0 msgr2=0x564e240bc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.214+0000 7febf4a39740 1 --2- 172.21.15.67:0/4203301122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23ce21b0 0x564e240bc100 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7febd800c6a0 tx=0x7febd80aac50 comp rx=0 tx=0).stop 2024-07-31T15:03:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.214+0000 7febf4a39740 1 -- 172.21.15.67:0/4203301122 shutdown_connections 2024-07-31T15:03:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.214+0000 7febf4a39740 1 --2- 172.21.15.67:0/4203301122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23d0b070 0x564e240c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.214+0000 7febf4a39740 1 --2- 172.21.15.67:0/4203301122 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564e23cc99f0 0x564e240ce280 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.214+0000 7febf4a39740 1 --2- 172.21.15.67:0/4203301122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23cea390 0x564e240bc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.214+0000 7febf4a39740 1 --2- 172.21.15.67:0/4203301122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23ce21b0 0x564e240bc100 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.214+0000 7febf4a39740 1 -- 172.21.15.67:0/4203301122 >> 172.21.15.67:0/4203301122 conn(0x564e23cff460 msgr2=0x564e240b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.214+0000 7febc77fe640 0 client.18135 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:03:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.214+0000 7febf4a39740 1 -- 172.21.15.67:0/4203301122 shutdown_connections 2024-07-31T15:03:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.214+0000 7febf4a39740 1 -- 172.21.15.67:0/4203301122 wait complete. 2024-07-31T15:03:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.214+0000 7febf4a39740 20 client.18135 trim_cache size 0 max 16384 2024-07-31T15:03:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.242+0000 7febf4a39740 1 Processor -- start 2024-07-31T15:03:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.242+0000 7febf4a39740 1 -- start start 2024-07-31T15:03:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.242+0000 7febf4a39740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23ce21b0 0x564e240ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.242+0000 7febf4a39740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23cea390 0x564e240c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.242+0000 7febf4a39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23d0b070 0x564e240b5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.242+0000 7febf4a39740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7febe81b2910 con 0x564e23ce21b0 2024-07-31T15:03:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.242+0000 7febf4a39740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7febe81b4450 con 0x564e23cea390 2024-07-31T15:03:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.242+0000 7febf4a39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7febe81b2fe0 con 0x564e23d0b070 2024-07-31T15:03:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.242+0000 7febe7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23ce21b0 0x564e240ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.242+0000 7febe77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23cea390 0x564e240c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.242+0000 7febe77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23cea390 0x564e240c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:47774/0 (socket says 172.21.15.67:47774) 2024-07-31T15:03:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.242+0000 7febe7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23ce21b0 0x564e240ae470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39490/0 (socket says 172.21.15.67:39490) 2024-07-31T15:03:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.242+0000 7febe7fff640 1 -- 172.21.15.67:0/558122679 learned_addr learned my addr 172.21.15.67:0/558122679 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.242+0000 7febeca85640 1 --2- 172.21.15.67:0/558122679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23d0b070 0x564e240b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.243+0000 7febe7fff640 1 -- 172.21.15.67:0/558122679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23d0b070 msgr2=0x564e240b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.243+0000 7febe7fff640 1 --2- 172.21.15.67:0/558122679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23d0b070 0x564e240b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.243+0000 7febe7fff640 1 -- 172.21.15.67:0/558122679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23cea390 msgr2=0x564e240c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.243+0000 7febe7fff640 1 --2- 172.21.15.67:0/558122679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23cea390 0x564e240c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.243+0000 7febe7fff640 1 -- 172.21.15.67:0/558122679 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e240bd1c0 con 0x564e23ce21b0 2024-07-31T15:03:33.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.243+0000 7febe7fff640 1 --2- 172.21.15.67:0/558122679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23ce21b0 0x564e240ae470 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7febd80a6eb0 tx=0x7febd80aad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ffe60610b14ab7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:33.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.243+0000 7febcdffb640 1 -- 172.21.15.67:0/558122679 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd80ab580 con 0x564e23ce21b0 2024-07-31T15:03:33.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.243+0000 7febcdffb640 1 -- 172.21.15.67:0/558122679 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd80aae80 con 0x564e23ce21b0 2024-07-31T15:03:33.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.243+0000 7febcdffb640 1 -- 172.21.15.67:0/558122679 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd80a67f0 con 0x564e23ce21b0 2024-07-31T15:03:33.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.244+0000 7febf4a39740 1 -- 172.21.15.67:0/558122679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23ce21b0 msgr2=0x564e240ae470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.244+0000 7febf4a39740 1 --2- 172.21.15.67:0/558122679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23ce21b0 0x564e240ae470 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7febd80a6eb0 tx=0x7febd80aad80 comp rx=0 tx=0).stop 2024-07-31T15:03:33.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.244+0000 7febf4a39740 1 -- 172.21.15.67:0/558122679 shutdown_connections 2024-07-31T15:03:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.244+0000 7febf4a39740 1 --2- 172.21.15.67:0/558122679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23d0b070 0x564e240b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.244+0000 7febf4a39740 1 --2- 172.21.15.67:0/558122679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23cea390 0x564e240c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.244+0000 7febf4a39740 1 --2- 172.21.15.67:0/558122679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23ce21b0 0x564e240ae470 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.244+0000 7febf4a39740 1 -- 172.21.15.67:0/558122679 >> 172.21.15.67:0/558122679 conn(0x564e23cff460 msgr2=0x564e240add60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.244+0000 7febf4a39740 1 -- 172.21.15.67:0/558122679 shutdown_connections 2024-07-31T15:03:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.244+0000 7febf4a39740 1 -- 172.21.15.67:0/558122679 wait complete. 2024-07-31T15:03:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.245+0000 7febf4a39740 1 -- start start 2024-07-31T15:03:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.245+0000 7febf4a39740 1 Processor -- start 2024-07-31T15:03:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.246+0000 7febf4a39740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23ce21b0 0x564e240c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.246+0000 7febf4a39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23cea390 0x564e240d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.246+0000 7febf4a39740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23d0b070 0x564e240d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.246+0000 7febe7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23ce21b0 0x564e240c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.246+0000 7febe7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23ce21b0 0x564e240c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:47776/0 (socket says 172.21.15.67:47776) 2024-07-31T15:03:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.246+0000 7febe77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23cea390 0x564e240d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.246+0000 7febe77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23cea390 0x564e240d3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50132/0 (socket says 172.21.15.67:50132) 2024-07-31T15:03:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.246+0000 7febeca85640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23d0b070 0x564e240d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.246+0000 7febeca85640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23d0b070 0x564e240d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39502/0 (socket says 172.21.15.67:39502) 2024-07-31T15:03:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.247+0000 7febf4a39740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7febe81b3d80 con 0x564e23d0b070 2024-07-31T15:03:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.247+0000 7febf4a39740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7febe81b36b0 con 0x564e23ce21b0 2024-07-31T15:03:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.247+0000 7febf4a39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7febe81b0910 con 0x564e23cea390 2024-07-31T15:03:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.247+0000 7febf4a39740 10 client.0 _ll_register_callbacks cb 0x7febd80ea240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.247+0000 7febe7fff640 1 -- 172.21.15.67:0/2483603873 learned_addr learned my addr 172.21.15.67:0/2483603873 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.247+0000 7febeca85640 1 -- 172.21.15.67:0/2483603873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23cea390 msgr2=0x564e240d3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.247+0000 7febeca85640 1 --2- 172.21.15.67:0/2483603873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23cea390 0x564e240d3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.247+0000 7febeca85640 1 -- 172.21.15.67:0/2483603873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23ce21b0 msgr2=0x564e240c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.247+0000 7febeca85640 1 --2- 172.21.15.67:0/2483603873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23ce21b0 0x564e240c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.247+0000 7febeca85640 1 -- 172.21.15.67:0/2483603873 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e240d3cd0 con 0x564e23d0b070 2024-07-31T15:03:33.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.247+0000 7febeca85640 1 --2- 172.21.15.67:0/2483603873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23d0b070 0x564e240d3760 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7febe0004af0 tx=0x7febe002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a217d6570d9eec4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:33.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.247+0000 7febc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:33.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.248+0000 7febc77fe640 1 -- 172.21.15.67:0/2483603873 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0004250 con 0x564e23d0b070 2024-07-31T15:03:33.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.248+0000 7febf4a39740 1 -- 172.21.15.67:0/2483603873 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd80aae80 con 0x564e23d0b070 2024-07-31T15:03:33.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.248+0000 7febc77fe640 1 -- 172.21.15.67:0/2483603873 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7febe0043030 con 0x564e23d0b070 2024-07-31T15:03:33.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.248+0000 7febc77fe640 1 -- 172.21.15.67:0/2483603873 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7febe00043f0 con 0x564e23d0b070 2024-07-31T15:03:33.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.248+0000 7febf4a39740 10 client.18141 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:33.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.248+0000 7febf4a39740 1 -- 172.21.15.67:0/2483603873 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x564e240c2d30 con 0x564e23d0b070 2024-07-31T15:03:33.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.248+0000 7febc77fe640 1 -- 172.21.15.67:0/2483603873 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd80aae80 con 0x564e23d0b070 2024-07-31T15:03:33.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.248+0000 7febc77fe640 1 -- 172.21.15.67:0/2483603873 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7febe00305c0 con 0x564e23d0b070 2024-07-31T15:03:33.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.248+0000 7febf4a39740 10 client.18141 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:33.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.248+0000 7febf4a39740 10 client.18141 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:03:33.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.248+0000 7febf4a39740 20 client.18141 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:33.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.248+0000 7febf4a39740 1 --2- 172.21.15.67:0/2483603873 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564e23c9eda0 0x564e24138f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.248+0000 7febf4a39740 4 client.18141 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:03:33.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.248+0000 7febf4a39740 1 -- 172.21.15.67:0/2483603873 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e240245a0 con 0x564e23c9eda0 2024-07-31T15:03:33.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.249+0000 7febe7fff640 1 --2- 172.21.15.67:0/2483603873 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564e23c9eda0 0x564e24138f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.249+0000 7febc77fe640 10 client.18141 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:03:33.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.249+0000 7febe7fff640 1 --2- 172.21.15.67:0/2483603873 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564e23c9eda0 0x564e24138f80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19caa64b27ad106f server_cookie=3ac156e1eb5982e5 in_seq=0 out_seq=0 2024-07-31T15:03:33.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febc77fe640 1 -- 172.21.15.67:0/2483603873 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e240245a0 con 0x564e23c9eda0 2024-07-31T15:03:33.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febc77fe640 10 client.18141 handle_command_reply: tid=0 2024-07-31T15:03:33.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febf4a39740 1 client.18141 shutdown 2024-07-31T15:03:33.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febf4a39740 1 -- 172.21.15.67:0/2483603873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23d0b070 msgr2=0x564e240d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febf4a39740 1 --2- 172.21.15.67:0/2483603873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23d0b070 0x564e240d3760 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7febe0004af0 tx=0x7febe002a8c0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febf4a39740 1 -- 172.21.15.67:0/2483603873 shutdown_connections 2024-07-31T15:03:33.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febf4a39740 1 --2- 172.21.15.67:0/2483603873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564e23d0b070 0x564e240d3760 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febf4a39740 1 --2- 172.21.15.67:0/2483603873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564e23cea390 0x564e240d3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febf4a39740 1 --2- 172.21.15.67:0/2483603873 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564e23c9eda0 0x564e24138f80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febf4a39740 1 --2- 172.21.15.67:0/2483603873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e23ce21b0 0x564e240c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febf4a39740 1 -- 172.21.15.67:0/2483603873 >> 172.21.15.67:0/2483603873 conn(0x564e23cff460 msgr2=0x564e240acc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:33.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febc77fe640 0 client.18141 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:03:33.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febf4a39740 1 -- 172.21.15.67:0/2483603873 shutdown_connections 2024-07-31T15:03:33.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febf4a39740 1 -- 172.21.15.67:0/2483603873 wait complete. 2024-07-31T15:03:33.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.250+0000 7febf4a39740 20 client.18141 trim_cache size 0 max 16384 2024-07-31T15:03:33.257 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:03:33.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.251+0000 7febee4e8640 1 -- 172.21.15.67:0/873009221 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7febd00b3d70 msgr2=0x7febd00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.251+0000 7febee4e8640 1 --2- 172.21.15.67:0/873009221 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7febd00b3d70 0x7febd00b6280 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7febe00094e0 tx=0x7febe0005d80 comp rx=0 tx=0).stop 2024-07-31T15:03:33.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.251+0000 7febee4e8640 1 -- 172.21.15.67:0/873009221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febe81035c0 msgr2=0x7febe819c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.251+0000 7febee4e8640 1 --2- 172.21.15.67:0/873009221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febe81035c0 0x7febe819c040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7febd800e9f0 tx=0x7febd800edf0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.251+0000 7febee4e8640 1 -- 172.21.15.67:0/873009221 shutdown_connections 2024-07-31T15:03:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.251+0000 7febee4e8640 1 --2- 172.21.15.67:0/873009221 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7febd00b3d70 0x7febd00b6280 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.251+0000 7febee4e8640 1 --2- 172.21.15.67:0/873009221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febe8109850 0x7febe81a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.251+0000 7febee4e8640 1 --2- 172.21.15.67:0/873009221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febe8108f00 0x7febe819c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.251+0000 7febee4e8640 1 --2- 172.21.15.67:0/873009221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febe81035c0 0x7febe819c040 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.251+0000 7febee4e8640 1 -- 172.21.15.67:0/873009221 >> 172.21.15.67:0/873009221 conn(0x7febe80ff030 msgr2=0x7febe8107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.251+0000 7febee4e8640 1 -- 172.21.15.67:0/873009221 shutdown_connections 2024-07-31T15:03:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:33.251+0000 7febee4e8640 1 -- 172.21.15.67:0/873009221 wait complete. 2024-07-31T15:03:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:03:33.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:33 smithi073 ceph-mon[36714]: pgmap v794: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 245 KiB/s wr, 42 op/s 2024-07-31T15:03:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:33 smithi077 ceph-mon[37327]: pgmap v794: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 245 KiB/s wr, 42 op/s 2024-07-31T15:03:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.509+0000 7efd3f039640 1 -- 172.21.15.67:0/783624573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd38102420 msgr2=0x7efd38102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.509+0000 7efd3f039640 1 --2- 172.21.15.67:0/783624573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd38102420 0x7efd38102830 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7efd20009900 tx=0x7efd2002a410 comp rx=0 tx=0).stop 2024-07-31T15:03:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.509+0000 7efd3f039640 1 -- 172.21.15.67:0/783624573 shutdown_connections 2024-07-31T15:03:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.509+0000 7efd3f039640 1 --2- 172.21.15.67:0/783624573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd381086a0 0x7efd3810eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.509+0000 7efd3f039640 1 --2- 172.21.15.67:0/783624573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd38107d50 0x7efd38108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.509+0000 7efd3f039640 1 --2- 172.21.15.67:0/783624573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd38102420 0x7efd38102830 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.509+0000 7efd3f039640 1 -- 172.21.15.67:0/783624573 >> 172.21.15.67:0/783624573 conn(0x7efd380fdde0 msgr2=0x7efd38100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.509+0000 7efd3f039640 1 -- 172.21.15.67:0/783624573 shutdown_connections 2024-07-31T15:03:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.509+0000 7efd3f039640 1 -- 172.21.15.67:0/783624573 wait complete. 2024-07-31T15:03:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.510+0000 7efd3f039640 1 Processor -- start 2024-07-31T15:03:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.510+0000 7efd3f039640 1 -- start start 2024-07-31T15:03:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.510+0000 7efd3f039640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd38102420 0x7efd38199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.510+0000 7efd3f039640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd38107d50 0x7efd3819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.510+0000 7efd3f039640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd381086a0 0x7efd3819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.510+0000 7efd3f039640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efd381134b0 con 0x7efd38107d50 2024-07-31T15:03:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.510+0000 7efd3f039640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efd38113330 con 0x7efd381086a0 2024-07-31T15:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.510+0000 7efd3f039640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd38113630 con 0x7efd38102420 2024-07-31T15:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.510+0000 7efd37fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd38107d50 0x7efd3819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.511+0000 7efd37fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd38107d50 0x7efd3819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39530/0 (socket says 172.21.15.67:39530) 2024-07-31T15:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.511+0000 7efd37fff640 1 -- 172.21.15.67:0/2541726704 learned_addr learned my addr 172.21.15.67:0/2541726704 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.511+0000 7efd3cdd5640 1 --2- 172.21.15.67:0/2541726704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd38102420 0x7efd38199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.511+0000 7efd3d5d6640 1 --2- 172.21.15.67:0/2541726704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd381086a0 0x7efd3819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.511+0000 7efd37fff640 1 -- 172.21.15.67:0/2541726704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd38102420 msgr2=0x7efd38199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.511+0000 7efd37fff640 1 --2- 172.21.15.67:0/2541726704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd38102420 0x7efd38199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.511+0000 7efd37fff640 1 -- 172.21.15.67:0/2541726704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd381086a0 msgr2=0x7efd3819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.511+0000 7efd37fff640 1 --2- 172.21.15.67:0/2541726704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd381086a0 0x7efd3819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.511+0000 7efd37fff640 1 -- 172.21.15.67:0/2541726704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3819fde0 con 0x7efd38107d50 2024-07-31T15:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.511+0000 7efd37fff640 1 --2- 172.21.15.67:0/2541726704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd38107d50 0x7efd3819a450 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7efd2800ebc0 tx=0x7efd2800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.511+0000 7efd35ffb640 1 -- 172.21.15.67:0/2541726704 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28010630 con 0x7efd38107d50 2024-07-31T15:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.511+0000 7efd3f039640 1 -- 172.21.15.67:0/2541726704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd381a00d0 con 0x7efd38107d50 2024-07-31T15:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.511+0000 7efd35ffb640 1 -- 172.21.15.67:0/2541726704 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd28010c80 con 0x7efd38107d50 2024-07-31T15:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.511+0000 7efd35ffb640 1 -- 172.21.15.67:0/2541726704 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28017ba0 con 0x7efd38107d50 2024-07-31T15:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.512+0000 7efd3f039640 1 -- 172.21.15.67:0/2541726704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd381a6860 con 0x7efd38107d50 2024-07-31T15:03:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.515+0000 7efd46005740 1 Processor -- start 2024-07-31T15:03:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.515+0000 7efd46005740 1 -- start start 2024-07-31T15:03:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.515+0000 7efd46005740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f3b1b0 0x5568a92ddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.515+0000 7efd46005740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f122f0 0x5568a92de220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.515+0000 7efd46005740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f1a4d0 0x5568a92de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.515+0000 7efd46005740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5568a92533b0 con 0x5568a8f1a4d0 2024-07-31T15:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.515+0000 7efd46005740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5568a92428b0 con 0x5568a8f122f0 2024-07-31T15:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.515+0000 7efd46005740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5568a92468f0 con 0x5568a8f3b1b0 2024-07-31T15:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.515+0000 7efd35ffb640 1 -- 172.21.15.67:0/2541726704 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7efd280107d0 con 0x7efd38107d50 2024-07-31T15:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.515+0000 7efd37fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f1a4d0 0x5568a92de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.515+0000 7efd3d5d6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f3b1b0 0x5568a92ddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.515+0000 7efd37fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f1a4d0 0x5568a92de760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39544/0 (socket says 172.21.15.67:39544) 2024-07-31T15:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.515+0000 7efd37fff640 1 -- 172.21.15.67:0/2060710623 learned_addr learned my addr 172.21.15.67:0/2060710623 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.516+0000 7efd3cdd5640 1 --2- 172.21.15.67:0/2060710623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f122f0 0x5568a92de220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.516+0000 7efd3d5d6640 1 -- 172.21.15.67:0/2060710623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f122f0 msgr2=0x5568a92de220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.516+0000 7efd3d5d6640 1 --2- 172.21.15.67:0/2060710623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f122f0 0x5568a92de220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.516+0000 7efd3d5d6640 1 -- 172.21.15.67:0/2060710623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f1a4d0 msgr2=0x5568a92de760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.516+0000 7efd3d5d6640 1 --2- 172.21.15.67:0/2060710623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f1a4d0 0x5568a92de760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.516+0000 7efd3d5d6640 1 -- 172.21.15.67:0/2060710623 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a91e99a0 con 0x5568a8f3b1b0 2024-07-31T15:03:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.516+0000 7efd37fff640 1 --2- 172.21.15.67:0/2060710623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f1a4d0 0x5568a92de760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:03:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.516+0000 7efd3d5d6640 1 --2- 172.21.15.67:0/2060710623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f3b1b0 0x5568a92ddce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efd30009b20 tx=0x7efd300113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cbba885757cbede server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.516+0000 7efd35ffb640 1 --2- 172.21.15.67:0/2541726704 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efd240b3d40 0x7efd240b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.516+0000 7efd2effd640 1 -- 172.21.15.67:0/2060710623 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30011e10 con 0x5568a8f3b1b0 2024-07-31T15:03:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.516+0000 7efd2effd640 1 -- 172.21.15.67:0/2060710623 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd30012900 con 0x5568a8f3b1b0 2024-07-31T15:03:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.516+0000 7efd3cdd5640 1 --2- 172.21.15.67:0/2541726704 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efd240b3d40 0x7efd240b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.516+0000 7efd2effd640 1 -- 172.21.15.67:0/2060710623 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3000f980 con 0x5568a8f3b1b0 2024-07-31T15:03:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.516+0000 7efd35ffb640 1 -- 172.21.15.67:0/2541726704 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7efd280e4cd0 con 0x7efd38107d50 2024-07-31T15:03:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.517+0000 7efd46005740 1 -- 172.21.15.67:0/2060710623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f3b1b0 msgr2=0x5568a92ddce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.517+0000 7efd46005740 1 --2- 172.21.15.67:0/2060710623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f3b1b0 0x5568a92ddce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efd30009b20 tx=0x7efd300113e0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.517+0000 7efd3cdd5640 1 --2- 172.21.15.67:0/2541726704 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efd240b3d40 0x7efd240b6250 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efd20005ab0 tx=0x7efd20005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.517+0000 7efd46005740 1 -- 172.21.15.67:0/2060710623 shutdown_connections 2024-07-31T15:03:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.517+0000 7efd46005740 1 --2- 172.21.15.67:0/2060710623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f1a4d0 0x5568a92de760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.517+0000 7efd46005740 1 --2- 172.21.15.67:0/2060710623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f122f0 0x5568a92de220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.517+0000 7efd46005740 1 --2- 172.21.15.67:0/2060710623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f3b1b0 0x5568a92ddce0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.517+0000 7efd46005740 1 -- 172.21.15.67:0/2060710623 >> 172.21.15.67:0/2060710623 conn(0x5568a8f2f300 msgr2=0x5568a92dbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.518+0000 7efd46005740 1 -- 172.21.15.67:0/2060710623 shutdown_connections 2024-07-31T15:03:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.518+0000 7efd46005740 1 -- 172.21.15.67:0/2060710623 wait complete. 2024-07-31T15:03:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.518+0000 7efd46005740 1 -- start start 2024-07-31T15:03:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.518+0000 7efd46005740 1 Processor -- start 2024-07-31T15:03:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd46005740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f122f0 0x5568a92e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd46005740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f1a4d0 0x5568a92e9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd46005740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f3b1b0 0x5568a92f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd46005740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5568a917cab0 con 0x5568a8f122f0 2024-07-31T15:03:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd46005740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5568a91cacf0 con 0x5568a8f3b1b0 2024-07-31T15:03:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd46005740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5568a92ec460 con 0x5568a8f1a4d0 2024-07-31T15:03:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd46005740 10 client.0 _ll_register_callbacks cb 0x5568a923c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd3cdd5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f122f0 0x5568a92e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd3cdd5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f122f0 0x5568a92e97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39554/0 (socket says 172.21.15.67:39554) 2024-07-31T15:03:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd3cdd5640 1 -- 172.21.15.67:0/1522588925 learned_addr learned my addr 172.21.15.67:0/1522588925 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd37fff640 1 --2- 172.21.15.67:0/1522588925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f1a4d0 0x5568a92e9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd3d5d6640 1 --2- 172.21.15.67:0/1522588925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f3b1b0 0x5568a92f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd3cdd5640 1 -- 172.21.15.67:0/1522588925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f1a4d0 msgr2=0x5568a92e9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd3cdd5640 1 --2- 172.21.15.67:0/1522588925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f1a4d0 0x5568a92e9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd3cdd5640 1 -- 172.21.15.67:0/1522588925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f3b1b0 msgr2=0x5568a92f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd3cdd5640 1 --2- 172.21.15.67:0/1522588925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f3b1b0 0x5568a92f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.519+0000 7efd3cdd5640 1 -- 172.21.15.67:0/1522588925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a8fa6790 con 0x5568a8f122f0 2024-07-31T15:03:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.520+0000 7efd3cdd5640 1 --2- 172.21.15.67:0/1522588925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f122f0 0x5568a92e97d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7efd20033000 tx=0x7efd20009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2605d2027826a686 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.520+0000 7efd17fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.520+0000 7efd17fff640 1 -- 172.21.15.67:0/1522588925 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2004e070 con 0x5568a8f122f0 2024-07-31T15:03:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.520+0000 7efd17fff640 1 -- 172.21.15.67:0/1522588925 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7efd20004970 con 0x5568a8f122f0 2024-07-31T15:03:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.520+0000 7efd46005740 1 -- 172.21.15.67:0/1522588925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd30012900 con 0x5568a8f122f0 2024-07-31T15:03:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.520+0000 7efd17fff640 1 -- 172.21.15.67:0/1522588925 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7efd2005a050 con 0x5568a8f122f0 2024-07-31T15:03:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.520+0000 7efd46005740 10 client.18153 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.520+0000 7efd46005740 1 -- 172.21.15.67:0/1522588925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5568a901f620 con 0x5568a8f122f0 2024-07-31T15:03:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.520+0000 7efd17fff640 1 -- 172.21.15.67:0/1522588925 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd30012900 con 0x5568a8f122f0 2024-07-31T15:03:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.520+0000 7efd17fff640 1 -- 172.21.15.67:0/1522588925 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7efd200324b0 con 0x5568a8f122f0 2024-07-31T15:03:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.521+0000 7efd46005740 10 client.18153 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.521+0000 7efd46005740 10 client.18153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.521+0000 7efd46005740 20 client.18153 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.521+0000 7efd46005740 1 --2- 172.21.15.67:0/1522588925 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5568a8ef9d80 0x5568a92fb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.521+0000 7efd46005740 4 client.18153 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:03:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.521+0000 7efd46005740 1 -- 172.21.15.67:0/1522588925 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568a9255300 con 0x5568a8ef9d80 2024-07-31T15:03:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.521+0000 7efd37fff640 1 --2- 172.21.15.67:0/1522588925 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5568a8ef9d80 0x5568a92fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.522+0000 7efd37fff640 1 --2- 172.21.15.67:0/1522588925 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5568a8ef9d80 0x5568a92fb800 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6975c77e4122d8d7 server_cookie=c651420ce84d1aac in_seq=0 out_seq=0 2024-07-31T15:03:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.522+0000 7efd17fff640 10 client.18153 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.522+0000 7efd17fff640 1 -- 172.21.15.67:0/1522588925 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5568a9255300 con 0x5568a8ef9d80 2024-07-31T15:03:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.522+0000 7efd17fff640 10 client.18153 handle_command_reply: tid=0 2024-07-31T15:03:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.522+0000 7efd46005740 1 client.18153 shutdown 2024-07-31T15:03:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.523+0000 7efd46005740 1 -- 172.21.15.67:0/1522588925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f122f0 msgr2=0x5568a92e97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.523+0000 7efd46005740 1 --2- 172.21.15.67:0/1522588925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f122f0 0x5568a92e97d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7efd20033000 tx=0x7efd20009260 comp rx=0 tx=0).stop 2024-07-31T15:03:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.523+0000 7efd46005740 1 -- 172.21.15.67:0/1522588925 shutdown_connections 2024-07-31T15:03:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.523+0000 7efd46005740 1 --2- 172.21.15.67:0/1522588925 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5568a8ef9d80 0x5568a92fb800 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.523+0000 7efd46005740 1 --2- 172.21.15.67:0/1522588925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f3b1b0 0x5568a92f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.523+0000 7efd17fff640 0 client.18153 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.523+0000 7efd46005740 1 --2- 172.21.15.67:0/1522588925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f1a4d0 0x5568a92e9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.523+0000 7efd46005740 1 --2- 172.21.15.67:0/1522588925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f122f0 0x5568a92e97d0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.523+0000 7efd46005740 1 -- 172.21.15.67:0/1522588925 >> 172.21.15.67:0/1522588925 conn(0x5568a8f2f300 msgr2=0x5568a92da2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.523+0000 7efd46005740 1 -- 172.21.15.67:0/1522588925 shutdown_connections 2024-07-31T15:03:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.523+0000 7efd46005740 1 -- 172.21.15.67:0/1522588925 wait complete. 2024-07-31T15:03:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.523+0000 7efd46005740 20 client.18153 trim_cache size 0 max 16384 2024-07-31T15:03:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd46005740 1 Processor -- start 2024-07-31T15:03:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd46005740 1 -- start start 2024-07-31T15:03:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd46005740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8ef9d80 0x5568a92e3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd46005740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f122f0 0x5568a92e8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd46005740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f1a4d0 0x5568a92df420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd46005740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efd381ae510 con 0x5568a8f122f0 2024-07-31T15:03:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd46005740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efd381b0050 con 0x5568a8f1a4d0 2024-07-31T15:03:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd46005740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd381aebe0 con 0x5568a8ef9d80 2024-07-31T15:03:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd37fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f122f0 0x5568a92e8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd37fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f122f0 0x5568a92e8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39568/0 (socket says 172.21.15.67:39568) 2024-07-31T15:03:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd37fff640 1 -- 172.21.15.67:0/2052945046 learned_addr learned my addr 172.21.15.67:0/2052945046 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd3cdd5640 1 --2- 172.21.15.67:0/2052945046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8ef9d80 0x5568a92e3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd3d5d6640 1 --2- 172.21.15.67:0/2052945046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f1a4d0 0x5568a92df420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd37fff640 1 -- 172.21.15.67:0/2052945046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8ef9d80 msgr2=0x5568a92e3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd37fff640 1 --2- 172.21.15.67:0/2052945046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8ef9d80 0x5568a92e3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd37fff640 1 -- 172.21.15.67:0/2052945046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f1a4d0 msgr2=0x5568a92df420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd37fff640 1 --2- 172.21.15.67:0/2052945046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f1a4d0 0x5568a92df420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.539+0000 7efd37fff640 1 -- 172.21.15.67:0/2052945046 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a92dffe0 con 0x5568a8f122f0 2024-07-31T15:03:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.540+0000 7efd3d5d6640 1 --2- 172.21.15.67:0/2052945046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f1a4d0 0x5568a92df420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:03:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.540+0000 7efd37fff640 1 --2- 172.21.15.67:0/2052945046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f122f0 0x5568a92e8e30 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7efd2800cf90 tx=0x7efd280b4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201b2cef2fd9ee6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.540+0000 7efd2e7fc640 1 -- 172.21.15.67:0/2052945046 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd280c5070 con 0x5568a8f122f0 2024-07-31T15:03:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.540+0000 7efd2e7fc640 1 -- 172.21.15.67:0/2052945046 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd280e6040 con 0x5568a8f122f0 2024-07-31T15:03:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.540+0000 7efd2e7fc640 1 -- 172.21.15.67:0/2052945046 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28010ac0 con 0x5568a8f122f0 2024-07-31T15:03:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.541+0000 7efd46005740 1 -- 172.21.15.67:0/2052945046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f122f0 msgr2=0x5568a92e8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.541+0000 7efd46005740 1 --2- 172.21.15.67:0/2052945046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f122f0 0x5568a92e8e30 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7efd2800cf90 tx=0x7efd280b4040 comp rx=0 tx=0).stop 2024-07-31T15:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.541+0000 7efd46005740 1 -- 172.21.15.67:0/2052945046 shutdown_connections 2024-07-31T15:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.541+0000 7efd46005740 1 --2- 172.21.15.67:0/2052945046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8f1a4d0 0x5568a92df420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.541+0000 7efd46005740 1 --2- 172.21.15.67:0/2052945046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f122f0 0x5568a92e8e30 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.541+0000 7efd46005740 1 --2- 172.21.15.67:0/2052945046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8ef9d80 0x5568a92e3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.541+0000 7efd46005740 1 -- 172.21.15.67:0/2052945046 >> 172.21.15.67:0/2052945046 conn(0x5568a8f2f300 msgr2=0x5568a92ded10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.541+0000 7efd46005740 1 -- 172.21.15.67:0/2052945046 shutdown_connections 2024-07-31T15:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.541+0000 7efd46005740 1 -- 172.21.15.67:0/2052945046 wait complete. 2024-07-31T15:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.541+0000 7efd46005740 1 -- start start 2024-07-31T15:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.541+0000 7efd46005740 1 Processor -- start 2024-07-31T15:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd46005740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8ef9d80 0x5568a92f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd46005740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f122f0 0x5568a92ee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd46005740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f1a4d0 0x5568a92eec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd46005740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efd381af980 con 0x5568a8f1a4d0 2024-07-31T15:03:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd46005740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efd381af2b0 con 0x5568a8ef9d80 2024-07-31T15:03:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd46005740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd381ac510 con 0x5568a8f122f0 2024-07-31T15:03:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd46005740 10 client.0 _ll_register_callbacks cb 0x7efd20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd3d5d6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f1a4d0 0x5568a92eec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd3d5d6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f1a4d0 0x5568a92eec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39570/0 (socket says 172.21.15.67:39570) 2024-07-31T15:03:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd3d5d6640 1 -- 172.21.15.67:0/1283008699 learned_addr learned my addr 172.21.15.67:0/1283008699 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd3cdd5640 1 --2- 172.21.15.67:0/1283008699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8ef9d80 0x5568a92f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd37fff640 1 --2- 172.21.15.67:0/1283008699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f122f0 0x5568a92ee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd3d5d6640 1 -- 172.21.15.67:0/1283008699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f122f0 msgr2=0x5568a92ee6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd3d5d6640 1 --2- 172.21.15.67:0/1283008699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f122f0 0x5568a92ee6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd3d5d6640 1 -- 172.21.15.67:0/1283008699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8ef9d80 msgr2=0x5568a92f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd3d5d6640 1 --2- 172.21.15.67:0/1283008699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8ef9d80 0x5568a92f1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd3d5d6640 1 -- 172.21.15.67:0/1283008699 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a92ef160 con 0x5568a8f1a4d0 2024-07-31T15:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd3d5d6640 1 --2- 172.21.15.67:0/1283008699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f1a4d0 0x5568a92eec20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7efd30009b20 tx=0x7efd30005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c320a570224bd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.542+0000 7efd17fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.543+0000 7efd17fff640 1 -- 172.21.15.67:0/1283008699 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30030060 con 0x5568a8f1a4d0 2024-07-31T15:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.543+0000 7efd17fff640 1 -- 172.21.15.67:0/1283008699 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7efd300035f0 con 0x5568a8f1a4d0 2024-07-31T15:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.543+0000 7efd46005740 1 -- 172.21.15.67:0/1283008699 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd280e6040 con 0x5568a8f1a4d0 2024-07-31T15:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.543+0000 7efd17fff640 1 -- 172.21.15.67:0/1283008699 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7efd30037050 con 0x5568a8f1a4d0 2024-07-31T15:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.543+0000 7efd46005740 10 client.18159 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.543+0000 7efd46005740 1 -- 172.21.15.67:0/1283008699 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5568a9362c50 con 0x5568a8f1a4d0 2024-07-31T15:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.543+0000 7efd17fff640 1 -- 172.21.15.67:0/1283008699 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd280e6040 con 0x5568a8f1a4d0 2024-07-31T15:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.543+0000 7efd17fff640 1 -- 172.21.15.67:0/1283008699 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7efd3000f870 con 0x5568a8f1a4d0 2024-07-31T15:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.543+0000 7efd46005740 10 client.18159 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.543+0000 7efd46005740 10 client.18159 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.543+0000 7efd46005740 20 client.18159 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.543+0000 7efd46005740 1 --2- 172.21.15.67:0/1283008699 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5568a8f3b1b0 0x5568a9365100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.543+0000 7efd46005740 4 client.18159 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:03:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.543+0000 7efd46005740 1 -- 172.21.15.67:0/1283008699 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568a9250da0 con 0x5568a8f3b1b0 2024-07-31T15:03:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.544+0000 7efd3cdd5640 1 --2- 172.21.15.67:0/1283008699 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5568a8f3b1b0 0x5568a9365100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.544+0000 7efd17fff640 10 client.18159 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.544+0000 7efd3cdd5640 1 --2- 172.21.15.67:0/1283008699 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5568a8f3b1b0 0x5568a9365100 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=653b419895b3afaa server_cookie=43fc9ec86c478a25 in_seq=0 out_seq=0 2024-07-31T15:03:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.591+0000 7efd17fff640 1 -- 172.21.15.67:0/1283008699 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568a9250da0 con 0x5568a8f3b1b0 2024-07-31T15:03:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.591+0000 7efd17fff640 10 client.18159 handle_command_reply: tid=0 2024-07-31T15:03:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.592+0000 7efd46005740 1 client.18159 shutdown 2024-07-31T15:03:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.592+0000 7efd46005740 1 -- 172.21.15.67:0/1283008699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f1a4d0 msgr2=0x5568a92eec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.592+0000 7efd46005740 1 --2- 172.21.15.67:0/1283008699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f1a4d0 0x5568a92eec20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7efd30009b20 tx=0x7efd30005270 comp rx=0 tx=0).stop 2024-07-31T15:03:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.592+0000 7efd46005740 1 -- 172.21.15.67:0/1283008699 shutdown_connections 2024-07-31T15:03:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.592+0000 7efd46005740 1 --2- 172.21.15.67:0/1283008699 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5568a8f3b1b0 0x5568a9365100 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.592+0000 7efd46005740 1 --2- 172.21.15.67:0/1283008699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568a8f1a4d0 0x5568a92eec20 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.592+0000 7efd46005740 1 --2- 172.21.15.67:0/1283008699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568a8f122f0 0x5568a92ee6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.592+0000 7efd46005740 1 --2- 172.21.15.67:0/1283008699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568a8ef9d80 0x5568a92f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.592+0000 7efd17fff640 0 client.18159 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.592+0000 7efd46005740 1 -- 172.21.15.67:0/1283008699 >> 172.21.15.67:0/1283008699 conn(0x5568a8f2f300 msgr2=0x5568a92ded10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.592+0000 7efd46005740 1 -- 172.21.15.67:0/1283008699 shutdown_connections 2024-07-31T15:03:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.592+0000 7efd46005740 1 -- 172.21.15.67:0/1283008699 wait complete. 2024-07-31T15:03:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.592+0000 7efd46005740 20 client.18159 trim_cache size 0 max 16384 2024-07-31T15:03:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:03:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:03:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:03:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:03:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.593+0000 7efd3f039640 1 -- 172.21.15.67:0/2541726704 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efd240b3d40 msgr2=0x7efd240b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.593+0000 7efd3f039640 1 --2- 172.21.15.67:0/2541726704 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efd240b3d40 0x7efd240b6250 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efd20005ab0 tx=0x7efd20005a20 comp rx=0 tx=0).stop 2024-07-31T15:03:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.593+0000 7efd3f039640 1 -- 172.21.15.67:0/2541726704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd38107d50 msgr2=0x7efd3819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.593+0000 7efd3f039640 1 --2- 172.21.15.67:0/2541726704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd38107d50 0x7efd3819a450 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7efd2800ebc0 tx=0x7efd2800c6a0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.593+0000 7efd3f039640 1 -- 172.21.15.67:0/2541726704 shutdown_connections 2024-07-31T15:03:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.593+0000 7efd3f039640 1 --2- 172.21.15.67:0/2541726704 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efd240b3d40 0x7efd240b6250 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.593+0000 7efd3f039640 1 --2- 172.21.15.67:0/2541726704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd381086a0 0x7efd3819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.593+0000 7efd3f039640 1 --2- 172.21.15.67:0/2541726704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd38107d50 0x7efd3819a450 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.593+0000 7efd3f039640 1 --2- 172.21.15.67:0/2541726704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd38102420 0x7efd38199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.593+0000 7efd3f039640 1 -- 172.21.15.67:0/2541726704 >> 172.21.15.67:0/2541726704 conn(0x7efd380fdde0 msgr2=0x7efd38101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.593+0000 7efd3f039640 1 -- 172.21.15.67:0/2541726704 shutdown_connections 2024-07-31T15:03:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:33.593+0000 7efd3f039640 1 -- 172.21.15.67:0/2541726704 wait complete. 2024-07-31T15:03:34.682 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T15:03:35.190 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:03:35.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:35 smithi073 ceph-mon[36714]: pgmap v795: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 64 KiB/s wr, 28 op/s 2024-07-31T15:03:35.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:35 smithi067 ceph-mon[32750]: pgmap v795: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 64 KiB/s wr, 28 op/s 2024-07-31T15:03:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:35 smithi077 ceph-mon[37327]: pgmap v795: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 64 KiB/s wr, 28 op/s 2024-07-31T15:03:35.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.903+0000 7ff901bfb640 1 -- 172.21.15.67:0/3840083459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc108f30 msgr2=0x7ff8fc109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:35.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.903+0000 7ff901bfb640 1 --2- 172.21.15.67:0/3840083459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc108f30 0x7ff8fc109340 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009900 tx=0x7ff8f002a410 comp rx=0 tx=0).stop 2024-07-31T15:03:35.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.903+0000 7ff901bfb640 1 -- 172.21.15.67:0/3840083459 shutdown_connections 2024-07-31T15:03:35.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.903+0000 7ff901bfb640 1 --2- 172.21.15.67:0/3840083459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8fc109880 0x7ff8fc1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.903+0000 7ff901bfb640 1 --2- 172.21.15.67:0/3840083459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc108f30 0x7ff8fc109340 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.903+0000 7ff901bfb640 1 --2- 172.21.15.67:0/3840083459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8fc1019f0 0x7ff8fc101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.903+0000 7ff901bfb640 1 -- 172.21.15.67:0/3840083459 >> 172.21.15.67:0/3840083459 conn(0x7ff8fc0799c0 msgr2=0x7ff8fc079dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:35.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.903+0000 7ff901bfb640 1 -- 172.21.15.67:0/3840083459 shutdown_connections 2024-07-31T15:03:35.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.903+0000 7ff901bfb640 1 -- 172.21.15.67:0/3840083459 wait complete. 2024-07-31T15:03:35.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.904+0000 7ff901bfb640 1 Processor -- start 2024-07-31T15:03:35.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.904+0000 7ff901bfb640 1 -- start start 2024-07-31T15:03:35.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.904+0000 7ff901bfb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc1019f0 0x7ff8fc199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.904+0000 7ff901bfb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8fc108f30 0x7ff8fc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.904+0000 7ff901bfb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8fc109880 0x7ff8fc19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.904+0000 7ff901bfb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc113510 con 0x7ff8fc1019f0 2024-07-31T15:03:35.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.904+0000 7ff901bfb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc113390 con 0x7ff8fc108f30 2024-07-31T15:03:35.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.904+0000 7ff901bfb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc113690 con 0x7ff8fc109880 2024-07-31T15:03:35.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.904+0000 7ff8fb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc1019f0 0x7ff8fc199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.904+0000 7ff8fb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc1019f0 0x7ff8fc199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39582/0 (socket says 172.21.15.67:39582) 2024-07-31T15:03:35.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.904+0000 7ff8fb7fe640 1 -- 172.21.15.67:0/2729647957 learned_addr learned my addr 172.21.15.67:0/2729647957 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:35.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.904+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/2729647957 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8fc109880 0x7ff8fc19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.904+0000 7ff8faffd640 1 --2- 172.21.15.67:0/2729647957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8fc108f30 0x7ff8fc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.905+0000 7ff8fb7fe640 1 -- 172.21.15.67:0/2729647957 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8fc109880 msgr2=0x7ff8fc19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:35.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.905+0000 7ff8fb7fe640 1 --2- 172.21.15.67:0/2729647957 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8fc109880 0x7ff8fc19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.905+0000 7ff8fb7fe640 1 -- 172.21.15.67:0/2729647957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8fc108f30 msgr2=0x7ff8fc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:35.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.905+0000 7ff8fb7fe640 1 --2- 172.21.15.67:0/2729647957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8fc108f30 0x7ff8fc19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.905+0000 7ff8fb7fe640 1 -- 172.21.15.67:0/2729647957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc19fe90 con 0x7ff8fc1019f0 2024-07-31T15:03:35.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.905+0000 7ff8fb7fe640 1 --2- 172.21.15.67:0/2729647957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc1019f0 0x7ff8fc199f20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00d770 tx=0x7ff8ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:35.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.905+0000 7ff8f8ff9640 1 -- 172.21.15.67:0/2729647957 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec017070 con 0x7ff8fc1019f0 2024-07-31T15:03:35.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.905+0000 7ff8f8ff9640 1 -- 172.21.15.67:0/2729647957 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8ec0054d0 con 0x7ff8fc1019f0 2024-07-31T15:03:35.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.905+0000 7ff901bfb640 1 -- 172.21.15.67:0/2729647957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8fc1a0180 con 0x7ff8fc1019f0 2024-07-31T15:03:35.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.905+0000 7ff8f8ff9640 1 -- 172.21.15.67:0/2729647957 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec004c30 con 0x7ff8fc1019f0 2024-07-31T15:03:35.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.905+0000 7ff901bfb640 1 -- 172.21.15.67:0/2729647957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8fc1a68e0 con 0x7ff8fc1019f0 2024-07-31T15:03:35.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.908+0000 7ff90814c740 1 Processor -- start 2024-07-31T15:03:35.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.908+0000 7ff8f8ff9640 1 -- 172.21.15.67:0/2729647957 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff8ec004e10 con 0x7ff8fc1019f0 2024-07-31T15:03:35.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.908+0000 7ff90814c740 1 -- start start 2024-07-31T15:03:35.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.908+0000 7ff90814c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ec9520 0x55a7a226f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.908+0000 7ff90814c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea0660 0x55a7a226f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.908+0000 7ff90814c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea8840 0x55a7a226faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.908+0000 7ff90814c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a7a21e4e70 con 0x55a7a1ec9520 2024-07-31T15:03:35.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.908+0000 7ff90814c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a7a21d4370 con 0x55a7a1ea8840 2024-07-31T15:03:35.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.908+0000 7ff90814c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a7a21d83b0 con 0x55a7a1ea0660 2024-07-31T15:03:35.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.908+0000 7ff8fbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ec9520 0x55a7a226f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.908+0000 7ff8fbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ec9520 0x55a7a226f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39596/0 (socket says 172.21.15.67:39596) 2024-07-31T15:03:35.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.908+0000 7ff8fbfff640 1 -- 172.21.15.67:0/2684494119 learned_addr learned my addr 172.21.15.67:0/2684494119 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:35.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.908+0000 7ff8fb7fe640 1 --2- 172.21.15.67:0/2684494119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea0660 0x55a7a226f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.908+0000 7ff8faffd640 1 --2- 172.21.15.67:0/2684494119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea8840 0x55a7a226faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.909+0000 7ff8fbfff640 1 -- 172.21.15.67:0/2684494119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea0660 msgr2=0x55a7a226f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.909+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/2684494119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea0660 0x55a7a226f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.909+0000 7ff8fbfff640 1 -- 172.21.15.67:0/2684494119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea8840 msgr2=0x55a7a226faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.909+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/2684494119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea8840 0x55a7a226faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.909+0000 7ff8fbfff640 1 -- 172.21.15.67:0/2684494119 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7a21a1040 con 0x55a7a1ec9520 2024-07-31T15:03:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.909+0000 7ff8faffd640 1 --2- 172.21.15.67:0/2684494119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea8840 0x55a7a226faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:03:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.909+0000 7ff8f8ff9640 1 --2- 172.21.15.67:0/2729647957 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e40b3dc0 0x7ff8e40b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.909+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/2684494119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ec9520 0x55a7a226f070 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4010370 tx=0x7ff8f4010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3061b4527b850c46 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:35.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.909+0000 7ff8fb7fe640 1 --2- 172.21.15.67:0/2729647957 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e40b3dc0 0x7ff8e40b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.909+0000 7ff8f8ff9640 1 -- 172.21.15.67:0/2729647957 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7ff8ec0e3ce0 con 0x7ff8fc1019f0 2024-07-31T15:03:35.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.909+0000 7ff8e1ffb640 1 -- 172.21.15.67:0/2684494119 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4016070 con 0x55a7a1ec9520 2024-07-31T15:03:35.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.909+0000 7ff8e1ffb640 1 -- 172.21.15.67:0/2684494119 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8f4004d20 con 0x55a7a1ec9520 2024-07-31T15:03:35.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.909+0000 7ff8e1ffb640 1 -- 172.21.15.67:0/2684494119 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4005420 con 0x55a7a1ec9520 2024-07-31T15:03:35.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.910+0000 7ff90814c740 1 -- 172.21.15.67:0/2684494119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ec9520 msgr2=0x55a7a226f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:35.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.910+0000 7ff8fb7fe640 1 --2- 172.21.15.67:0/2729647957 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e40b3dc0 0x7ff8e40b62d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec0e11a0 tx=0x7ff8ec0b3040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:35.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.910+0000 7ff90814c740 1 --2- 172.21.15.67:0/2684494119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ec9520 0x55a7a226f070 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4010370 tx=0x7ff8f4010830 comp rx=0 tx=0).stop 2024-07-31T15:03:35.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.910+0000 7ff90814c740 1 -- 172.21.15.67:0/2684494119 shutdown_connections 2024-07-31T15:03:35.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.910+0000 7ff90814c740 1 --2- 172.21.15.67:0/2684494119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea8840 0x55a7a226faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.910+0000 7ff90814c740 1 --2- 172.21.15.67:0/2684494119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea0660 0x55a7a226f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.910+0000 7ff90814c740 1 --2- 172.21.15.67:0/2684494119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ec9520 0x55a7a226f070 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.910+0000 7ff90814c740 1 -- 172.21.15.67:0/2684494119 >> 172.21.15.67:0/2684494119 conn(0x55a7a1ebd910 msgr2=0x55a7a226cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:35.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.910+0000 7ff90814c740 1 -- 172.21.15.67:0/2684494119 shutdown_connections 2024-07-31T15:03:35.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.910+0000 7ff90814c740 1 -- 172.21.15.67:0/2684494119 wait complete. 2024-07-31T15:03:35.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.910+0000 7ff90814c740 1 -- start start 2024-07-31T15:03:35.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.911+0000 7ff90814c740 1 Processor -- start 2024-07-31T15:03:35.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.911+0000 7ff90814c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ea0660 0x55a7a227abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.911+0000 7ff90814c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea8840 0x55a7a227b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.911+0000 7ff90814c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ec9520 0x55a7a2286ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.911+0000 7ff90814c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a7a2037bb0 con 0x55a7a1ea0660 2024-07-31T15:03:35.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.911+0000 7ff90814c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a7a2176ac0 con 0x55a7a1ec9520 2024-07-31T15:03:35.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.911+0000 7ff90814c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a7a227d850 con 0x55a7a1ea8840 2024-07-31T15:03:35.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.911+0000 7ff90814c740 10 client.0 _ll_register_callbacks cb 0x55a7a217f3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:35.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.911+0000 7ff8fbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ea0660 0x55a7a227abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.911+0000 7ff8fbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ea0660 0x55a7a227abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39600/0 (socket says 172.21.15.67:39600) 2024-07-31T15:03:35.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.911+0000 7ff8fbfff640 1 -- 172.21.15.67:0/871896253 learned_addr learned my addr 172.21.15.67:0/871896253 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:35.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.911+0000 7ff8faffd640 1 --2- 172.21.15.67:0/871896253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea8840 0x55a7a227b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.911+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/871896253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ec9520 0x55a7a2286ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff8fbfff640 1 -- 172.21.15.67:0/871896253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea8840 msgr2=0x55a7a227b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/871896253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea8840 0x55a7a227b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff8fbfff640 1 -- 172.21.15.67:0/871896253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ec9520 msgr2=0x55a7a2286ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/871896253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ec9520 0x55a7a2286ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff8fbfff640 1 -- 172.21.15.67:0/871896253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7a1f34b00 con 0x55a7a1ea0660 2024-07-31T15:03:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff8faffd640 1 --2- 172.21.15.67:0/871896253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea8840 0x55a7a227b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:03:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/871896253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ea0660 0x55a7a227abb0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009db0 tx=0x7ff8f4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd19213b254e568 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff8daffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff8daffd640 1 -- 172.21.15.67:0/871896253 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4016070 con 0x55a7a1ea0660 2024-07-31T15:03:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff8daffd640 1 -- 172.21.15.67:0/871896253 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7ff8f4022040 con 0x55a7a1ea0660 2024-07-31T15:03:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff90814c740 1 -- 172.21.15.67:0/871896253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8f4004d20 con 0x55a7a1ea0660 2024-07-31T15:03:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff8daffd640 1 -- 172.21.15.67:0/871896253 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7ff8f4003de0 con 0x55a7a1ea0660 2024-07-31T15:03:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff90814c740 10 client.18171 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff90814c740 1 -- 172.21.15.67:0/871896253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55a7a1fab960 con 0x55a7a1ea0660 2024-07-31T15:03:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.912+0000 7ff8daffd640 1 -- 172.21.15.67:0/871896253 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8f4004d20 con 0x55a7a1ea0660 2024-07-31T15:03:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.913+0000 7ff8daffd640 1 -- 172.21.15.67:0/871896253 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7ff8f400f040 con 0x55a7a1ea0660 2024-07-31T15:03:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.913+0000 7ff90814c740 10 client.18171 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.913+0000 7ff90814c740 10 client.18171 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:03:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.913+0000 7ff90814c740 20 client.18171 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.913+0000 7ff90814c740 1 --2- 172.21.15.67:0/871896253 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a7a1e87ea0 0x55a7a228c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.913+0000 7ff90814c740 4 client.18171 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:03:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.913+0000 7ff90814c740 1 -- 172.21.15.67:0/871896253 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7a21e6dc0 con 0x55a7a1e87ea0 2024-07-31T15:03:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.913+0000 7ff8faffd640 1 --2- 172.21.15.67:0/871896253 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a7a1e87ea0 0x55a7a228c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.913+0000 7ff8faffd640 1 --2- 172.21.15.67:0/871896253 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a7a1e87ea0 0x55a7a228c8f0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=270ea67b496dbbea server_cookie=91975747076af068 in_seq=0 out_seq=0 2024-07-31T15:03:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.913+0000 7ff8daffd640 10 client.18171 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.914+0000 7ff8daffd640 1 -- 172.21.15.67:0/871896253 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7a21e6dc0 con 0x55a7a1e87ea0 2024-07-31T15:03:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.914+0000 7ff8daffd640 10 client.18171 handle_command_reply: tid=0 2024-07-31T15:03:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.914+0000 7ff90814c740 1 client.18171 shutdown 2024-07-31T15:03:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.914+0000 7ff90814c740 1 -- 172.21.15.67:0/871896253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ea0660 msgr2=0x55a7a227abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.914+0000 7ff90814c740 1 --2- 172.21.15.67:0/871896253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ea0660 0x55a7a227abb0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009db0 tx=0x7ff8f4005c50 comp rx=0 tx=0).stop 2024-07-31T15:03:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.914+0000 7ff90814c740 1 -- 172.21.15.67:0/871896253 shutdown_connections 2024-07-31T15:03:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.914+0000 7ff90814c740 1 --2- 172.21.15.67:0/871896253 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a7a1e87ea0 0x55a7a228c8f0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.914+0000 7ff90814c740 1 --2- 172.21.15.67:0/871896253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ec9520 0x55a7a2286ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.915+0000 7ff90814c740 1 --2- 172.21.15.67:0/871896253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea8840 0x55a7a227b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.915+0000 7ff90814c740 1 --2- 172.21.15.67:0/871896253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ea0660 0x55a7a227abb0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.915+0000 7ff90814c740 1 -- 172.21.15.67:0/871896253 >> 172.21.15.67:0/871896253 conn(0x55a7a1ebd910 msgr2=0x55a7a2272200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.915+0000 7ff8daffd640 0 client.18171 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.915+0000 7ff90814c740 1 -- 172.21.15.67:0/871896253 shutdown_connections 2024-07-31T15:03:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.915+0000 7ff90814c740 1 -- 172.21.15.67:0/871896253 wait complete. 2024-07-31T15:03:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.915+0000 7ff90814c740 20 client.18171 trim_cache size 0 max 16384 2024-07-31T15:03:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff90814c740 1 Processor -- start 2024-07-31T15:03:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff90814c740 1 -- start start 2024-07-31T15:03:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff90814c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea0660 0x55a7a2272b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff90814c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea8840 0x55a7a2286990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff90814c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ec9520 0x55a7a2286ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff90814c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b0830 con 0x55a7a1ec9520 2024-07-31T15:03:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff90814c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b2370 con 0x55a7a1ea8840 2024-07-31T15:03:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff90814c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b0f00 con 0x55a7a1ea0660 2024-07-31T15:03:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff8fbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ec9520 0x55a7a2286ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff8fbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ec9520 0x55a7a2286ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39604/0 (socket says 172.21.15.67:39604) 2024-07-31T15:03:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff8fbfff640 1 -- 172.21.15.67:0/1267515074 learned_addr learned my addr 172.21.15.67:0/1267515074 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/1267515074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea0660 0x55a7a2272b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff8faffd640 1 --2- 172.21.15.67:0/1267515074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea8840 0x55a7a2286990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff8fbfff640 1 -- 172.21.15.67:0/1267515074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea0660 msgr2=0x55a7a2272b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/1267515074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea0660 0x55a7a2272b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff8fbfff640 1 -- 172.21.15.67:0/1267515074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea8840 msgr2=0x55a7a2286990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/1267515074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea8840 0x55a7a2286990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff8fbfff640 1 -- 172.21.15.67:0/1267515074 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7a2271590 con 0x55a7a1ec9520 2024-07-31T15:03:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.933+0000 7ff8faffd640 1 --2- 172.21.15.67:0/1267515074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea8840 0x55a7a2286990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:03:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.934+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/1267515074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ec9520 0x55a7a2286ed0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x55a7a2275e30 tx=0x7ff8f4010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6ad363150e7d1f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.934+0000 7ff8e17fa640 1 -- 172.21.15.67:0/1267515074 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4016070 con 0x55a7a1ec9520 2024-07-31T15:03:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.934+0000 7ff8e17fa640 1 -- 172.21.15.67:0/1267515074 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8f4003970 con 0x55a7a1ec9520 2024-07-31T15:03:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.934+0000 7ff8e17fa640 1 -- 172.21.15.67:0/1267515074 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40045c0 con 0x55a7a1ec9520 2024-07-31T15:03:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.935+0000 7ff90814c740 1 -- 172.21.15.67:0/1267515074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ec9520 msgr2=0x55a7a2286ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.935+0000 7ff90814c740 1 --2- 172.21.15.67:0/1267515074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ec9520 0x55a7a2286ed0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x55a7a2275e30 tx=0x7ff8f4010830 comp rx=0 tx=0).stop 2024-07-31T15:03:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.935+0000 7ff90814c740 1 -- 172.21.15.67:0/1267515074 shutdown_connections 2024-07-31T15:03:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.935+0000 7ff90814c740 1 --2- 172.21.15.67:0/1267515074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ec9520 0x55a7a2286ed0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.935+0000 7ff90814c740 1 --2- 172.21.15.67:0/1267515074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea8840 0x55a7a2286990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.935+0000 7ff90814c740 1 --2- 172.21.15.67:0/1267515074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ea0660 0x55a7a2272b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.935+0000 7ff90814c740 1 -- 172.21.15.67:0/1267515074 >> 172.21.15.67:0/1267515074 conn(0x55a7a1ebd910 msgr2=0x55a7a226ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.935+0000 7ff90814c740 1 -- 172.21.15.67:0/1267515074 shutdown_connections 2024-07-31T15:03:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.935+0000 7ff90814c740 1 -- 172.21.15.67:0/1267515074 wait complete. 2024-07-31T15:03:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.935+0000 7ff90814c740 1 -- start start 2024-07-31T15:03:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff90814c740 1 Processor -- start 2024-07-31T15:03:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff90814c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea0660 0x55a7a2283040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff90814c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ea8840 0x55a7a227e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff90814c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ec9520 0x55a7a227ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff90814c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b1ca0 con 0x55a7a1ea8840 2024-07-31T15:03:35.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff90814c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b15d0 con 0x55a7a1ea0660 2024-07-31T15:03:35.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff90814c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1ae830 con 0x55a7a1ec9520 2024-07-31T15:03:35.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff90814c740 10 client.0 _ll_register_callbacks cb 0x7ff8f4009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:35.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff8faffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ea8840 0x55a7a227e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff8faffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ea8840 0x55a7a227e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39614/0 (socket says 172.21.15.67:39614) 2024-07-31T15:03:35.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff8faffd640 1 -- 172.21.15.67:0/3065075833 learned_addr learned my addr 172.21.15.67:0/3065075833 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:35.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/3065075833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea0660 0x55a7a2283040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/3065075833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ec9520 0x55a7a227ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff8faffd640 1 -- 172.21.15.67:0/3065075833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ec9520 msgr2=0x55a7a227ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff8faffd640 1 --2- 172.21.15.67:0/3065075833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ec9520 0x55a7a227ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff8faffd640 1 -- 172.21.15.67:0/3065075833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea0660 msgr2=0x55a7a2283040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff8faffd640 1 --2- 172.21.15.67:0/3065075833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea0660 0x55a7a2283040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff8faffd640 1 -- 172.21.15.67:0/3065075833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7a227f020 con 0x55a7a1ea8840 2024-07-31T15:03:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.936+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/3065075833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ec9520 0x55a7a227ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:03:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff8faffd640 1 --2- 172.21.15.67:0/3065075833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ea8840 0x55a7a227e4f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009900 tx=0x7ff8f0032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcacf4b58b3b7248 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff8daffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff8daffd640 1 -- 172.21.15.67:0/3065075833 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0035040 con 0x55a7a1ea8840 2024-07-31T15:03:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff8daffd640 1 -- 172.21.15.67:0/3065075833 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7ff8f0004310 con 0x55a7a1ea8840 2024-07-31T15:03:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff90814c740 1 -- 172.21.15.67:0/3065075833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8f4003970 con 0x55a7a1ea8840 2024-07-31T15:03:35.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff8daffd640 1 -- 172.21.15.67:0/3065075833 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7ff8f0051050 con 0x55a7a1ea8840 2024-07-31T15:03:35.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff90814c740 10 client.18177 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:35.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff90814c740 1 -- 172.21.15.67:0/3065075833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55a7a22f4be0 con 0x55a7a1ea8840 2024-07-31T15:03:35.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff8daffd640 1 -- 172.21.15.67:0/3065075833 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8f4003970 con 0x55a7a1ea8840 2024-07-31T15:03:35.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff8daffd640 1 -- 172.21.15.67:0/3065075833 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7ff8f00315c0 con 0x55a7a1ea8840 2024-07-31T15:03:35.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff90814c740 10 client.18177 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:35.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff90814c740 10 client.18177 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:03:35.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff90814c740 20 client.18177 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:35.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff90814c740 1 --2- 172.21.15.67:0/3065075833 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a7a1e5d250 0x55a7a22f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:35.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff90814c740 4 client.18177 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "54db3a03", "release": true, "await": true}] 2024-07-31T15:03:35.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.937+0000 7ff90814c740 1 -- 172.21.15.67:0/3065075833 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "54db3a03", "release": true, "await": true}) -- 0x55a7a21e2860 con 0x55a7a1e5d250 2024-07-31T15:03:35.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.938+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/3065075833 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a7a1e5d250 0x55a7a22f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:35.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.938+0000 7ff8fbfff640 1 --2- 172.21.15.67:0/3065075833 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a7a1e5d250 0x55a7a22f73c0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f92736a11653e53 server_cookie=1c32f8973bf1c0df in_seq=0 out_seq=0 2024-07-31T15:03:35.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:35.938+0000 7ff8daffd640 10 client.18177 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:37.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:37 smithi073 ceph-mon[36714]: pgmap v796: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 167 KiB/s wr, 31 op/s 2024-07-31T15:03:37.429 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:37 smithi067 ceph-mon[32750]: pgmap v796: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 167 KiB/s wr, 31 op/s 2024-07-31T15:03:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:37 smithi077 ceph-mon[37327]: pgmap v796: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 167 KiB/s wr, 31 op/s 2024-07-31T15:03:37.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.660+0000 7f1aaa07f640 1 -- 172.21.15.67:0/490134952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa410d840 msgr2=0x7f1aa410fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:37.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.660+0000 7f1aaa07f640 1 --2- 172.21.15.67:0/490134952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa410d840 0x7f1aa410fd30 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009ca0 tx=0x7f1a9c02a8c0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.660+0000 7f1aaa07f640 1 -- 172.21.15.67:0/490134952 shutdown_connections 2024-07-31T15:03:37.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.660+0000 7f1aaa07f640 1 --2- 172.21.15.67:0/490134952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa410d840 0x7f1aa410fd30 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.660+0000 7f1aaa07f640 1 --2- 172.21.15.67:0/490134952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa4104d30 0x7f1aa410d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.660+0000 7f1aaa07f640 1 --2- 172.21.15.67:0/490134952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1aa41043e0 0x7f1aa41047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.660+0000 7f1aaa07f640 1 -- 172.21.15.67:0/490134952 >> 172.21.15.67:0/490134952 conn(0x7f1aa4079160 msgr2=0x7f1aa4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:37.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.660+0000 7f1aaa07f640 1 -- 172.21.15.67:0/490134952 shutdown_connections 2024-07-31T15:03:37.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.660+0000 7f1aaa07f640 1 -- 172.21.15.67:0/490134952 wait complete. 2024-07-31T15:03:37.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.661+0000 7f1aaa07f640 1 Processor -- start 2024-07-31T15:03:37.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.661+0000 7f1aaa07f640 1 -- start start 2024-07-31T15:03:37.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.661+0000 7f1aaa07f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa41043e0 0x7f1aa4199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.661+0000 7f1aaa07f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa4104d30 0x7f1aa419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.661+0000 7f1aaa07f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1aa410d840 0x7f1aa419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.661+0000 7f1aaa07f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa41134a0 con 0x7f1aa41043e0 2024-07-31T15:03:37.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.661+0000 7f1aaa07f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4113320 con 0x7f1aa410d840 2024-07-31T15:03:37.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.661+0000 7f1aaa07f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4113620 con 0x7f1aa4104d30 2024-07-31T15:03:37.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.661+0000 7f1aa37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa41043e0 0x7f1aa4199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.662+0000 7f1aa37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa41043e0 0x7f1aa4199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39632/0 (socket says 172.21.15.67:39632) 2024-07-31T15:03:37.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.662+0000 7f1aa37fe640 1 -- 172.21.15.67:0/3750650284 learned_addr learned my addr 172.21.15.67:0/3750650284 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:37.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.662+0000 7f1aa3fff640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1aa410d840 0x7f1aa419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.662+0000 7f1aa2ffd640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa4104d30 0x7f1aa419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.662+0000 7f1aa3fff640 1 -- 172.21.15.67:0/3750650284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa4104d30 msgr2=0x7f1aa419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:37.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.662+0000 7f1aa3fff640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa4104d30 0x7f1aa419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.662+0000 7f1aa3fff640 1 -- 172.21.15.67:0/3750650284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa41043e0 msgr2=0x7f1aa4199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:37.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.662+0000 7f1aa3fff640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa41043e0 0x7f1aa4199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.662+0000 7f1aa3fff640 1 -- 172.21.15.67:0/3750650284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa419fe80 con 0x7f1aa410d840 2024-07-31T15:03:37.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.662+0000 7f1aa37fe640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa41043e0 0x7f1aa4199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:03:37.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.662+0000 7f1aa2ffd640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa4104d30 0x7f1aa419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:03:37.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.662+0000 7f1aa3fff640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1aa410d840 0x7f1aa419f750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c02ade0 tx=0x7f1a9c031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:37.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.662+0000 7f1aa0ff9640 1 -- 172.21.15.67:0/3750650284 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c003b70 con 0x7f1aa410d840 2024-07-31T15:03:37.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.663+0000 7f1aa0ff9640 1 -- 172.21.15.67:0/3750650284 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9c031910 con 0x7f1aa410d840 2024-07-31T15:03:37.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.663+0000 7f1aaa07f640 1 -- 172.21.15.67:0/3750650284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa41a0110 con 0x7f1aa410d840 2024-07-31T15:03:37.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.663+0000 7f1aa0ff9640 1 -- 172.21.15.67:0/3750650284 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c02f950 con 0x7f1aa410d840 2024-07-31T15:03:37.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.663+0000 7f1aaa07f640 1 -- 172.21.15.67:0/3750650284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa41a68d0 con 0x7f1aa410d840 2024-07-31T15:03:37.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.666+0000 7f1ab05d0740 1 Processor -- start 2024-07-31T15:03:37.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.666+0000 7f1aa0ff9640 1 -- 172.21.15.67:0/3750650284 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1a9c02faf0 con 0x7f1aa410d840 2024-07-31T15:03:37.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.666+0000 7f1ab05d0740 1 -- start start 2024-07-31T15:03:37.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.666+0000 7f1ab05d0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c159070 0x55ec4c4fe570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.666+0000 7f1ab05d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c1301b0 0x55ec4c4feab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.666+0000 7f1ab05d0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 0x55ec4c4feff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.666+0000 7f1ab05d0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ec4c474bb0 con 0x55ec4c138390 2024-07-31T15:03:37.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.666+0000 7f1ab05d0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ec4c4640b0 con 0x55ec4c159070 2024-07-31T15:03:37.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.666+0000 7f1ab05d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ec4c4680f0 con 0x55ec4c1301b0 2024-07-31T15:03:37.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1aa2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 0x55ec4c4feff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1aa2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c159070 0x55ec4c4fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1aa2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c159070 0x55ec4c4fe570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:47918/0 (socket says 172.21.15.67:47918) 2024-07-31T15:03:37.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1aa2ffd640 1 -- 172.21.15.67:0/4098738858 learned_addr learned my addr 172.21.15.67:0/4098738858 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:37.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1aa37fe640 1 --2- 172.21.15.67:0/4098738858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c1301b0 0x55ec4c4feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1aa0ff9640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1a8c0b3d40 0x7f1a8c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1aa2ffd640 1 -- 172.21.15.67:0/4098738858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c1301b0 msgr2=0x55ec4c4feab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:37.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1aa2ffd640 1 --2- 172.21.15.67:0/4098738858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c1301b0 0x55ec4c4feab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1aa2ffd640 1 -- 172.21.15.67:0/4098738858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 msgr2=0x55ec4c4feff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:37.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1aa2ffd640 1 --2- 172.21.15.67:0/4098738858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 0x55ec4c4feff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1aa2ffd640 1 -- 172.21.15.67:0/4098738858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4c40ab90 con 0x55ec4c159070 2024-07-31T15:03:37.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1aa3fff640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1a8c0b3d40 0x7f1a8c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1aa0ff9640 1 -- 172.21.15.67:0/3750650284 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f1a9c105580 con 0x7f1aa410d840 2024-07-31T15:03:37.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1aa2ffd640 1 --2- 172.21.15.67:0/4098738858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c159070 0x55ec4c4fe570 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1a98012750 tx=0x7f1a98005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=486a3052c56d8039 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:37.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.667+0000 7f1a89ffb640 1 -- 172.21.15.67:0/4098738858 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a980120d0 con 0x55ec4c159070 2024-07-31T15:03:37.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.668+0000 7f1a89ffb640 1 -- 172.21.15.67:0/4098738858 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a98012270 con 0x55ec4c159070 2024-07-31T15:03:37.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.668+0000 7f1a89ffb640 1 -- 172.21.15.67:0/4098738858 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a980035f0 con 0x55ec4c159070 2024-07-31T15:03:37.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.668+0000 7f1ab05d0740 1 -- 172.21.15.67:0/4098738858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c159070 msgr2=0x55ec4c4fe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:37.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.668+0000 7f1aa3fff640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1a8c0b3d40 0x7f1a8c0b6250 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c102b50 tx=0x7f1a9c02a8c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:37.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.668+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/4098738858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c159070 0x55ec4c4fe570 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1a98012750 tx=0x7f1a98005f90 comp rx=0 tx=0).stop 2024-07-31T15:03:37.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.668+0000 7f1ab05d0740 1 -- 172.21.15.67:0/4098738858 shutdown_connections 2024-07-31T15:03:37.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.668+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/4098738858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 0x55ec4c4feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.668+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/4098738858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c1301b0 0x55ec4c4feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.668+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/4098738858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c159070 0x55ec4c4fe570 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.668+0000 7f1ab05d0740 1 -- 172.21.15.67:0/4098738858 >> 172.21.15.67:0/4098738858 conn(0x55ec4c14d460 msgr2=0x55ec4c4fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:37.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.669+0000 7f1ab05d0740 1 -- 172.21.15.67:0/4098738858 shutdown_connections 2024-07-31T15:03:37.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.669+0000 7f1ab05d0740 1 -- 172.21.15.67:0/4098738858 wait complete. 2024-07-31T15:03:37.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.669+0000 7f1ab05d0740 1 -- start start 2024-07-31T15:03:37.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.669+0000 7f1ab05d0740 1 Processor -- start 2024-07-31T15:03:37.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1ab05d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c1301b0 0x55ec4c50e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1ab05d0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c138390 0x55ec4c50ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1ab05d0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c159070 0x55ec4c51a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1ab05d0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ec4c2c7700 con 0x55ec4c159070 2024-07-31T15:03:37.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1ab05d0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ec4c4069e0 con 0x55ec4c138390 2024-07-31T15:03:37.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1ab05d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ec4c510ff0 con 0x55ec4c1301b0 2024-07-31T15:03:37.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1ab05d0740 10 client.0 _ll_register_callbacks cb 0x55ec4c4214f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:37.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1aa37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c138390 0x55ec4c50ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1aa2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c159070 0x55ec4c51a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1aa37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c138390 0x55ec4c50ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:47924/0 (socket says 172.21.15.67:47924) 2024-07-31T15:03:37.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1aa37fe640 1 -- 172.21.15.67:0/650059179 learned_addr learned my addr 172.21.15.67:0/650059179 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:37.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1aa2ffd640 1 --2- 172.21.15.67:0/650059179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c1301b0 0x55ec4c50e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1aa37fe640 1 -- 172.21.15.67:0/650059179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c1301b0 msgr2=0x55ec4c50e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:37.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1aa37fe640 1 --2- 172.21.15.67:0/650059179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c1301b0 0x55ec4c50e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1aa37fe640 1 -- 172.21.15.67:0/650059179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c159070 msgr2=0x55ec4c51a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1aa37fe640 1 --2- 172.21.15.67:0/650059179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c159070 0x55ec4c51a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1aa37fe640 1 -- 172.21.15.67:0/650059179 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4c1c4650 con 0x55ec4c138390 2024-07-31T15:03:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1aa37fe640 1 --2- 172.21.15.67:0/650059179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c138390 0x55ec4c50ea70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1a94012d20 tx=0x7f1a9400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d77169a94fe4a18 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.670+0000 7f1a82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:03:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.671+0000 7f1a82ffd640 1 -- 172.21.15.67:0/650059179 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a940120d0 con 0x55ec4c138390 2024-07-31T15:03:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.671+0000 7f1a82ffd640 1 -- 172.21.15.67:0/650059179 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f1a94010430 con 0x55ec4c138390 2024-07-31T15:03:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.671+0000 7f1ab05d0740 1 -- 172.21.15.67:0/650059179 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a98012270 con 0x55ec4c138390 2024-07-31T15:03:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.671+0000 7f1a82ffd640 1 -- 172.21.15.67:0/650059179 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f1a94012270 con 0x55ec4c138390 2024-07-31T15:03:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.671+0000 7f1ab05d0740 10 client.27740 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.671+0000 7f1ab05d0740 1 -- 172.21.15.67:0/650059179 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55ec4c51d0d0 con 0x55ec4c138390 2024-07-31T15:03:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.671+0000 7f1a82ffd640 1 -- 172.21.15.67:0/650059179 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a98012270 con 0x55ec4c138390 2024-07-31T15:03:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.671+0000 7f1a82ffd640 1 -- 172.21.15.67:0/650059179 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f1a9401c6c0 con 0x55ec4c138390 2024-07-31T15:03:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.671+0000 7f1ab05d0740 10 client.27740 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.671+0000 7f1ab05d0740 10 client.27740 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:03:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.671+0000 7f1ab05d0740 20 client.27740 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.671+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/650059179 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ec4c1179f0 0x55ec4c520690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.671+0000 7f1ab05d0740 4 client.27740 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:03:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.671+0000 7f1ab05d0740 1 -- 172.21.15.67:0/650059179 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec4c476b00 con 0x55ec4c1179f0 2024-07-31T15:03:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.672+0000 7f1aa2ffd640 1 --2- 172.21.15.67:0/650059179 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ec4c1179f0 0x55ec4c520690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.672+0000 7f1aa2ffd640 1 --2- 172.21.15.67:0/650059179 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ec4c1179f0 0x55ec4c520690 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a710966647c4640a server_cookie=4d93cd580eceb99c in_seq=0 out_seq=0 2024-07-31T15:03:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.672+0000 7f1a82ffd640 10 client.27740 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:03:37.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.673+0000 7f1a82ffd640 1 -- 172.21.15.67:0/650059179 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec4c476b00 con 0x55ec4c1179f0 2024-07-31T15:03:37.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.673+0000 7f1a82ffd640 10 client.27740 handle_command_reply: tid=0 2024-07-31T15:03:37.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.673+0000 7f1ab05d0740 1 client.27740 shutdown 2024-07-31T15:03:37.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.673+0000 7f1ab05d0740 1 -- 172.21.15.67:0/650059179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c138390 msgr2=0x55ec4c50ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:37.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.673+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/650059179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c138390 0x55ec4c50ea70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1a94012d20 tx=0x7f1a9400f790 comp rx=0 tx=0).stop 2024-07-31T15:03:37.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.673+0000 7f1ab05d0740 1 -- 172.21.15.67:0/650059179 shutdown_connections 2024-07-31T15:03:37.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.673+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/650059179 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ec4c1179f0 0x55ec4c520690 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.673+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/650059179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c159070 0x55ec4c51a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.673+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/650059179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c138390 0x55ec4c50ea70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.673+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/650059179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c1301b0 0x55ec4c50e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.673+0000 7f1ab05d0740 1 -- 172.21.15.67:0/650059179 >> 172.21.15.67:0/650059179 conn(0x55ec4c14d460 msgr2=0x55ec4c4fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:37.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.673+0000 7f1a82ffd640 0 client.27740 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:03:37.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.673+0000 7f1ab05d0740 1 -- 172.21.15.67:0/650059179 shutdown_connections 2024-07-31T15:03:37.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.673+0000 7f1ab05d0740 1 -- 172.21.15.67:0/650059179 wait complete. 2024-07-31T15:03:37.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.674+0000 7f1ab05d0740 20 client.27740 trim_cache size 0 max 16384 2024-07-31T15:03:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.688+0000 7f1ab05d0740 1 Processor -- start 2024-07-31T15:03:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.688+0000 7f1ab05d0740 1 -- start start 2024-07-31T15:03:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.688+0000 7f1ab05d0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c1301b0 0x55ec4c4ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.688+0000 7f1ab05d0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 0x55ec4c51a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.688+0000 7f1ab05d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c159070 0x55ec4c508710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.688+0000 7f1ab05d0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa41ae5b0 con 0x55ec4c138390 2024-07-31T15:03:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.688+0000 7f1ab05d0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa41b00f0 con 0x55ec4c1301b0 2024-07-31T15:03:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.688+0000 7f1ab05d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa41aec80 con 0x55ec4c159070 2024-07-31T15:03:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.688+0000 7f1aa37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 0x55ec4c51a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.688+0000 7f1aa2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c1301b0 0x55ec4c4ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.688+0000 7f1aa2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c1301b0 0x55ec4c4ffcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:47940/0 (socket says 172.21.15.67:47940) 2024-07-31T15:03:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.688+0000 7f1aa2ffd640 1 -- 172.21.15.67:0/1055512888 learned_addr learned my addr 172.21.15.67:0/1055512888 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.688+0000 7f1aa2ffd640 1 --2- 172.21.15.67:0/1055512888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c159070 0x55ec4c508710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.689+0000 7f1aa37fe640 1 -- 172.21.15.67:0/1055512888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c159070 msgr2=0x55ec4c508710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.689+0000 7f1aa37fe640 1 --2- 172.21.15.67:0/1055512888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c159070 0x55ec4c508710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.689+0000 7f1aa37fe640 1 -- 172.21.15.67:0/1055512888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c1301b0 msgr2=0x55ec4c4ffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.689+0000 7f1aa37fe640 1 --2- 172.21.15.67:0/1055512888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c1301b0 0x55ec4c4ffcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.689+0000 7f1aa37fe640 1 -- 172.21.15.67:0/1055512888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4c510bb0 con 0x55ec4c138390 2024-07-31T15:03:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.689+0000 7f1aa37fe640 1 --2- 172.21.15.67:0/1055512888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 0x55ec4c51a170 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1a9401cb30 tx=0x7f1a9401b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e390b27ed68278 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.689+0000 7f1a897fa640 1 -- 172.21.15.67:0/1055512888 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9401c420 con 0x55ec4c138390 2024-07-31T15:03:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.689+0000 7f1a897fa640 1 -- 172.21.15.67:0/1055512888 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a940050b0 con 0x55ec4c138390 2024-07-31T15:03:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.689+0000 7f1a897fa640 1 -- 172.21.15.67:0/1055512888 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a940039e0 con 0x55ec4c138390 2024-07-31T15:03:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.690+0000 7f1ab05d0740 1 -- 172.21.15.67:0/1055512888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 msgr2=0x55ec4c51a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.690+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/1055512888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 0x55ec4c51a170 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1a9401cb30 tx=0x7f1a9401b420 comp rx=0 tx=0).stop 2024-07-31T15:03:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.690+0000 7f1ab05d0740 1 -- 172.21.15.67:0/1055512888 shutdown_connections 2024-07-31T15:03:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.690+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/1055512888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c159070 0x55ec4c508710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.690+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/1055512888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 0x55ec4c51a170 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.690+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/1055512888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c1301b0 0x55ec4c4ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.690+0000 7f1ab05d0740 1 -- 172.21.15.67:0/1055512888 >> 172.21.15.67:0/1055512888 conn(0x55ec4c14d460 msgr2=0x55ec4c4ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.690+0000 7f1ab05d0740 1 -- 172.21.15.67:0/1055512888 shutdown_connections 2024-07-31T15:03:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.690+0000 7f1ab05d0740 1 -- 172.21.15.67:0/1055512888 wait complete. 2024-07-31T15:03:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1ab05d0740 1 -- start start 2024-07-31T15:03:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1ab05d0740 1 Processor -- start 2024-07-31T15:03:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1ab05d0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c1301b0 0x55ec4c516630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1ab05d0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 0x55ec4c516bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1ab05d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c159070 0x55ec4c505a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1ab05d0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa41afa20 con 0x55ec4c138390 2024-07-31T15:03:37.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1ab05d0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa41af350 con 0x55ec4c1301b0 2024-07-31T15:03:37.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1ab05d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa41ac5b0 con 0x55ec4c159070 2024-07-31T15:03:37.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1aa2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c1301b0 0x55ec4c516630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1aa37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 0x55ec4c516bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1aa37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 0x55ec4c516bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39666/0 (socket says 172.21.15.67:39666) 2024-07-31T15:03:37.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1aa2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c1301b0 0x55ec4c516630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:47942/0 (socket says 172.21.15.67:47942) 2024-07-31T15:03:37.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1aa37fe640 1 -- 172.21.15.67:0/4285047719 learned_addr learned my addr 172.21.15.67:0/4285047719 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:37.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1aa2ffd640 1 --2- 172.21.15.67:0/4285047719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c159070 0x55ec4c505a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.691+0000 7f1ab05d0740 10 client.0 _ll_register_callbacks cb 0x7f1a94009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:37.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.692+0000 7f1aa2ffd640 1 -- 172.21.15.67:0/4285047719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c159070 msgr2=0x55ec4c505a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:37.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.692+0000 7f1aa2ffd640 1 --2- 172.21.15.67:0/4285047719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c159070 0x55ec4c505a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.692+0000 7f1aa2ffd640 1 -- 172.21.15.67:0/4285047719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 msgr2=0x55ec4c516bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.692+0000 7f1aa2ffd640 1 --2- 172.21.15.67:0/4285047719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 0x55ec4c516bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.692+0000 7f1aa2ffd640 1 -- 172.21.15.67:0/4285047719 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4c506200 con 0x55ec4c1301b0 2024-07-31T15:03:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.692+0000 7f1aa2ffd640 1 --2- 172.21.15.67:0/4285047719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c1301b0 0x55ec4c516630 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009f40 tx=0x7f1a98005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e62b39f6272b6157 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.692+0000 7f1a82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:03:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.692+0000 7f1a82ffd640 1 -- 172.21.15.67:0/4285047719 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9800fe60 con 0x55ec4c1301b0 2024-07-31T15:03:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.692+0000 7f1a82ffd640 1 -- 172.21.15.67:0/4285047719 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f1a980050b0 con 0x55ec4c1301b0 2024-07-31T15:03:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.692+0000 7f1ab05d0740 1 -- 172.21.15.67:0/4285047719 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a940050b0 con 0x55ec4c1301b0 2024-07-31T15:03:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.692+0000 7f1a82ffd640 1 -- 172.21.15.67:0/4285047719 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f1a980052b0 con 0x55ec4c1301b0 2024-07-31T15:03:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.692+0000 7f1ab05d0740 10 client.27746 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.692+0000 7f1ab05d0740 1 -- 172.21.15.67:0/4285047719 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55ec4c514240 con 0x55ec4c1301b0 2024-07-31T15:03:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.693+0000 7f1a82ffd640 1 -- 172.21.15.67:0/4285047719 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a940050b0 con 0x55ec4c1301b0 2024-07-31T15:03:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.693+0000 7f1a82ffd640 1 -- 172.21.15.67:0/4285047719 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f1a9802b350 con 0x55ec4c1301b0 2024-07-31T15:03:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.693+0000 7f1ab05d0740 10 client.27746 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.693+0000 7f1ab05d0740 10 client.27746 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:03:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.693+0000 7f1ab05d0740 20 client.27746 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.693+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/4285047719 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ec4c0ecda0 0x55ec4c58f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:37.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.693+0000 7f1ab05d0740 4 client.27746 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:03:37.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.693+0000 7f1ab05d0740 1 -- 172.21.15.67:0/4285047719 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec4c4725a0 con 0x55ec4c0ecda0 2024-07-31T15:03:37.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.693+0000 7f1aa37fe640 1 --2- 172.21.15.67:0/4285047719 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ec4c0ecda0 0x55ec4c58f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:37.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1a82ffd640 10 client.27746 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:03:37.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1aa37fe640 1 --2- 172.21.15.67:0/4285047719 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ec4c0ecda0 0x55ec4c58f840 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56fd67cfd160d1f4 server_cookie=5635b55ce75662e9 in_seq=0 out_seq=0 2024-07-31T15:03:37.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1a82ffd640 1 -- 172.21.15.67:0/4285047719 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec4c4725a0 con 0x55ec4c0ecda0 2024-07-31T15:03:37.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1a82ffd640 10 client.27746 handle_command_reply: tid=0 2024-07-31T15:03:37.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1ab05d0740 1 client.27746 shutdown 2024-07-31T15:03:37.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1ab05d0740 1 -- 172.21.15.67:0/4285047719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c1301b0 msgr2=0x55ec4c516630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:37.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/4285047719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c1301b0 0x55ec4c516630 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009f40 tx=0x7f1a98005f90 comp rx=0 tx=0).stop 2024-07-31T15:03:37.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1ab05d0740 1 -- 172.21.15.67:0/4285047719 shutdown_connections 2024-07-31T15:03:37.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/4285047719 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ec4c0ecda0 0x55ec4c58f840 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/4285047719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec4c159070 0x55ec4c505a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/4285047719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec4c138390 0x55ec4c516bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1ab05d0740 1 --2- 172.21.15.67:0/4285047719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec4c1301b0 0x55ec4c516630 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1a82ffd640 0 client.27746 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:03:37.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1ab05d0740 1 -- 172.21.15.67:0/4285047719 >> 172.21.15.67:0/4285047719 conn(0x55ec4c14d460 msgr2=0x55ec4c4ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:37.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1ab05d0740 1 -- 172.21.15.67:0/4285047719 shutdown_connections 2024-07-31T15:03:37.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1ab05d0740 1 -- 172.21.15.67:0/4285047719 wait complete. 2024-07-31T15:03:37.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.694+0000 7f1ab05d0740 20 client.27746 trim_cache size 0 max 16384 2024-07-31T15:03:37.705 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:03:37.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.695+0000 7f1aaa07f640 1 -- 172.21.15.67:0/3750650284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1a8c0b3d40 msgr2=0x7f1a8c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:37.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.695+0000 7f1aaa07f640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1a8c0b3d40 0x7f1a8c0b6250 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c102b50 tx=0x7f1a9c02a8c0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.695+0000 7f1aaa07f640 1 -- 172.21.15.67:0/3750650284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1aa410d840 msgr2=0x7f1aa419f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:37.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.695+0000 7f1aaa07f640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1aa410d840 0x7f1aa419f750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c02ade0 tx=0x7f1a9c031660 comp rx=0 tx=0).stop 2024-07-31T15:03:37.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.695+0000 7f1aaa07f640 1 -- 172.21.15.67:0/3750650284 shutdown_connections 2024-07-31T15:03:37.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.695+0000 7f1aaa07f640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1a8c0b3d40 0x7f1a8c0b6250 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.695+0000 7f1aaa07f640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1aa410d840 0x7f1aa419f750 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.695+0000 7f1aaa07f640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aa4104d30 0x7f1aa419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.695+0000 7f1aaa07f640 1 --2- 172.21.15.67:0/3750650284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa41043e0 0x7f1aa4199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.695+0000 7f1aaa07f640 1 -- 172.21.15.67:0/3750650284 >> 172.21.15.67:0/3750650284 conn(0x7f1aa4079160 msgr2=0x7f1aa40ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.695+0000 7f1aaa07f640 1 -- 172.21.15.67:0/3750650284 shutdown_connections 2024-07-31T15:03:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:37.695+0000 7f1aaa07f640 1 -- 172.21.15.67:0/3750650284 wait complete. 2024-07-31T15:03:38.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.187+0000 7ff8daffd640 1 -- 172.21.15.67:0/3065075833 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a7a21e2860 con 0x55a7a1e5d250 2024-07-31T15:03:38.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.187+0000 7ff8daffd640 10 client.18177 handle_command_reply: tid=0 2024-07-31T15:03:38.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.187+0000 7ff90814c740 1 client.18177 shutdown 2024-07-31T15:03:38.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.187+0000 7ff90814c740 1 -- 172.21.15.67:0/3065075833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ea8840 msgr2=0x55a7a227e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:38.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.187+0000 7ff90814c740 1 --2- 172.21.15.67:0/3065075833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ea8840 0x55a7a227e4f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009900 tx=0x7ff8f0032e70 comp rx=0 tx=0).stop 2024-07-31T15:03:38.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.187+0000 7ff90814c740 1 -- 172.21.15.67:0/3065075833 shutdown_connections 2024-07-31T15:03:38.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.187+0000 7ff90814c740 1 --2- 172.21.15.67:0/3065075833 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a7a1e5d250 0x55a7a22f73c0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:38.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.187+0000 7ff8daffd640 0 client.18177 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:38.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.187+0000 7ff90814c740 1 --2- 172.21.15.67:0/3065075833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7a1ec9520 0x55a7a227ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:38.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.187+0000 7ff90814c740 1 --2- 172.21.15.67:0/3065075833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7a1ea8840 0x55a7a227e4f0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:38.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.187+0000 7ff90814c740 1 --2- 172.21.15.67:0/3065075833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7a1ea0660 0x55a7a2283040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:38.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.187+0000 7ff90814c740 1 -- 172.21.15.67:0/3065075833 >> 172.21.15.67:0/3065075833 conn(0x55a7a1ebd910 msgr2=0x55a7a226ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:38.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.187+0000 7ff90814c740 1 -- 172.21.15.67:0/3065075833 shutdown_connections 2024-07-31T15:03:38.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.188+0000 7ff90814c740 1 -- 172.21.15.67:0/3065075833 wait complete. 2024-07-31T15:03:38.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.188+0000 7ff90814c740 20 client.18177 trim_cache size 0 max 16384 2024-07-31T15:03:38.191 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:03:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 63, 2024-07-31T15:03:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:03:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 32, 2024-07-31T15:03:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:03:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "54db3a03": { 2024-07-31T15:03:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 32, 2024-07-31T15:03:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:03:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:03:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:03:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:03:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:03:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:03:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 142.4, 2024-07-31T15:03:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:03:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:03:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:03:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:03:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:03:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:03:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:03:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:03:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:03:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:03:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:03:38.193 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:03:38.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.188+0000 7ff901bfb640 1 -- 172.21.15.67:0/2729647957 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e40b3dc0 msgr2=0x7ff8e40b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:38.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.188+0000 7ff901bfb640 1 --2- 172.21.15.67:0/2729647957 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e40b3dc0 0x7ff8e40b62d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec0e11a0 tx=0x7ff8ec0b3040 comp rx=0 tx=0).stop 2024-07-31T15:03:38.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.188+0000 7ff901bfb640 1 -- 172.21.15.67:0/2729647957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc1019f0 msgr2=0x7ff8fc199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:38.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.188+0000 7ff901bfb640 1 --2- 172.21.15.67:0/2729647957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc1019f0 0x7ff8fc199f20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00d770 tx=0x7ff8ec00dc30 comp rx=0 tx=0).stop 2024-07-31T15:03:38.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.188+0000 7ff901bfb640 1 -- 172.21.15.67:0/2729647957 shutdown_connections 2024-07-31T15:03:38.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.189+0000 7ff901bfb640 1 --2- 172.21.15.67:0/2729647957 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8e40b3dc0 0x7ff8e40b62d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:38.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.189+0000 7ff901bfb640 1 --2- 172.21.15.67:0/2729647957 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8fc109880 0x7ff8fc19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:38.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.189+0000 7ff901bfb640 1 --2- 172.21.15.67:0/2729647957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8fc108f30 0x7ff8fc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:38.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.189+0000 7ff901bfb640 1 --2- 172.21.15.67:0/2729647957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc1019f0 0x7ff8fc199f20 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:38.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.189+0000 7ff901bfb640 1 -- 172.21.15.67:0/2729647957 >> 172.21.15.67:0/2729647957 conn(0x7ff8fc0799c0 msgr2=0x7ff8fc106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:38.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.189+0000 7ff901bfb640 1 -- 172.21.15.67:0/2729647957 shutdown_connections 2024-07-31T15:03:38.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:38.189+0000 7ff901bfb640 1 -- 172.21.15.67:0/2729647957 wait complete. 2024-07-31T15:03:38.342 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/vmcore.o 2024-07-31T15:03:38.342 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/irqdomain.o 2024-07-31T15:03:38.355 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/devpts/built-in.a 2024-07-31T15:03:38.368 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/kernel/cpu/mtrr/built-in.a 2024-07-31T15:03:38.378 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/microcode/intel.o 2024-07-31T15:03:38.406 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T15:03:38.407 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/cacheinfo.o 2024-07-31T15:03:38.413 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/microcode/amd.o 2024-07-31T15:03:38.439 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/debug.o 2024-07-31T15:03:38.446 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/kernel/acpi/wakeup_64.o 2024-07-31T15:03:38.498 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/acpi/cppc.o 2024-07-31T15:03:38.617 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/status.o 2024-07-31T15:03:39.037 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/acpi/cstate.o 2024-07-31T15:03:39.152 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/ebitmap.o 2024-07-31T15:03:39.176 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '54db3a03', total seconds elapsed: 43.8 2024-07-31T15:03:39.221 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/kernel/cpu/microcode/built-in.a 2024-07-31T15:03:39.264 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/gup.o 2024-07-31T15:03:39.290 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/kmsg.o 2024-07-31T15:03:39.298 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/netfs/buffered_read.o 2024-07-31T15:03:39.338 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:03:39.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:39 smithi073 ceph-mon[36714]: pgmap v797: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 168 KiB/s wr, 35 op/s 2024-07-31T15:03:39.417 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/scattered.o 2024-07-31T15:03:39.431 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/proc.o 2024-07-31T15:03:39.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:39 smithi067 ceph-mon[32750]: pgmap v797: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 168 KiB/s wr, 35 op/s 2024-07-31T15:03:39.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:39 smithi077 ceph-mon[37327]: pgmap v797: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 168 KiB/s wr, 35 op/s 2024-07-31T15:03:39.553 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/kernel/acpi/built-in.a 2024-07-31T15:03:39.588 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mmap_lock.o 2024-07-31T15:03:39.621 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/hashtab.o 2024-07-31T15:03:39.768 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/page.o 2024-07-31T15:03:39.811 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/topology.o 2024-07-31T15:03:40.042 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/symtab.o 2024-07-31T15:03:40.080 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/migration.o 2024-07-31T15:03:40.107 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/netfs/io.o 2024-07-31T15:03:40.226 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/common.o 2024-07-31T15:03:40.274 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/apic/apic.o 2024-07-31T15:03:40.330 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/sidtab.o 2024-07-31T15:03:40.359 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/apic/apic_common.o 2024-07-31T15:03:40.491 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/cpuhotplug.o 2024-07-31T15:03:40.567 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/proc/built-in.a 2024-07-31T15:03:40.649 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/balloc.o 2024-07-31T15:03:40.739 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/bitmap.o 2024-07-31T15:03:40.977 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/netfs/iterator.o 2024-07-31T15:03:41.021 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/pm.o 2024-07-31T15:03:41.082 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/highmem.o 2024-07-31T15:03:41.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:41 smithi067 ceph-mon[32750]: pgmap v798: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 171 KiB/s wr, 39 op/s 2024-07-31T15:03:41.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:41 smithi073 ceph-mon[36714]: pgmap v798: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 171 KiB/s wr, 39 op/s 2024-07-31T15:03:41.402 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/avtab.o 2024-07-31T15:03:41.459 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/kprobes/core.o 2024-07-31T15:03:41.473 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/rdrand.o 2024-07-31T15:03:41.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:41 smithi077 ceph-mon[37327]: pgmap v798: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 171 KiB/s wr, 39 op/s 2024-07-31T15:03:41.651 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/match.o 2024-07-31T15:03:41.661 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/netfs/main.o 2024-07-31T15:03:41.706 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/memory.o 2024-07-31T15:03:41.722 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/msi.o 2024-07-31T15:03:41.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.737+0000 7f8526256640 1 -- 172.21.15.67:0/3164433517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85201114b0 msgr2=0x7f85201138d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:41.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.737+0000 7f8526256640 1 --2- 172.21.15.67:0/3164433517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85201114b0 0x7f85201138d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8514009900 tx=0x7f851402a420 comp rx=0 tx=0).stop 2024-07-31T15:03:41.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.739+0000 7f8526256640 1 -- 172.21.15.67:0/3164433517 shutdown_connections 2024-07-31T15:03:41.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.739+0000 7f8526256640 1 --2- 172.21.15.67:0/3164433517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520113e10 0x7f8520116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.739+0000 7f8526256640 1 --2- 172.21.15.67:0/3164433517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85201114b0 0x7f85201138d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.739+0000 7f8526256640 1 --2- 172.21.15.67:0/3164433517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8520101800 0x7f8520101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.739+0000 7f8526256640 1 -- 172.21.15.67:0/3164433517 >> 172.21.15.67:0/3164433517 conn(0x7f85200774d0 msgr2=0x7f8520077910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.740+0000 7f8526256640 1 -- 172.21.15.67:0/3164433517 shutdown_connections 2024-07-31T15:03:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.741+0000 7f8526256640 1 -- 172.21.15.67:0/3164433517 wait complete. 2024-07-31T15:03:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.742+0000 7f8526256640 1 Processor -- start 2024-07-31T15:03:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.742+0000 7f8526256640 1 -- start start 2024-07-31T15:03:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.742+0000 7f8526256640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8520101800 0x7f8520110c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.742+0000 7f8526256640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85201114b0 0x7f852010af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.742+0000 7f8526256640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520113e10 0x7f852010b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.742+0000 7f8526256640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8520109560 con 0x7f8520113e10 2024-07-31T15:03:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.742+0000 7f8526256640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f85201093e0 con 0x7f8520101800 2024-07-31T15:03:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.742+0000 7f8526256640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85201096e0 con 0x7f85201114b0 2024-07-31T15:03:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.743+0000 7f8525a55640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520113e10 0x7f852010b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.743+0000 7f8525a55640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520113e10 0x7f852010b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36712/0 (socket says 172.21.15.67:36712) 2024-07-31T15:03:41.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.743+0000 7f8525a55640 1 -- 172.21.15.67:0/3482131708 learned_addr learned my addr 172.21.15.67:0/3482131708 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:41.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.743+0000 7f8524a53640 1 --2- 172.21.15.67:0/3482131708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85201114b0 0x7f852010af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.743+0000 7f8524a53640 1 -- 172.21.15.67:0/3482131708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8520101800 msgr2=0x7f8520110c60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:03:41.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.743+0000 7f8524a53640 1 --2- 172.21.15.67:0/3482131708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8520101800 0x7f8520110c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.743+0000 7f8524a53640 1 -- 172.21.15.67:0/3482131708 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520113e10 msgr2=0x7f852010b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:41.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.743+0000 7f8524a53640 1 --2- 172.21.15.67:0/3482131708 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520113e10 0x7f852010b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.744+0000 7f8524a53640 1 -- 172.21.15.67:0/3482131708 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852010bbe0 con 0x7f85201114b0 2024-07-31T15:03:41.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.744+0000 7f8524a53640 1 --2- 172.21.15.67:0/3482131708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85201114b0 0x7f852010af20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8514009850 tx=0x7f85140305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:41.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.745+0000 7f851e7fc640 1 -- 172.21.15.67:0/3482131708 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851403f070 con 0x7f85201114b0 2024-07-31T15:03:41.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.746+0000 7f851e7fc640 1 -- 172.21.15.67:0/3482131708 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8514004440 con 0x7f85201114b0 2024-07-31T15:03:41.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.746+0000 7f851e7fc640 1 -- 172.21.15.67:0/3482131708 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514035070 con 0x7f85201114b0 2024-07-31T15:03:41.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.746+0000 7f8526256640 1 -- 172.21.15.67:0/3482131708 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8520072840 con 0x7f85201114b0 2024-07-31T15:03:41.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.746+0000 7f8526256640 1 -- 172.21.15.67:0/3482131708 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8520072be0 con 0x7f85201114b0 2024-07-31T15:03:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.753+0000 7f851e7fc640 1 -- 172.21.15.67:0/3482131708 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8514042070 con 0x7f85201114b0 2024-07-31T15:03:41.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.755+0000 7f852da09740 1 Processor -- start 2024-07-31T15:03:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.755+0000 7f852da09740 1 -- start start 2024-07-31T15:03:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.755+0000 7f852da09740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe8d070 0x561db0232710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.755+0000 7f852da09740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe641b0 0x561db0232c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.755+0000 7f852da09740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe6c390 0x561db0233190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.755+0000 7f852da09740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561db01a8bb0 con 0x561dafe641b0 2024-07-31T15:03:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.755+0000 7f852da09740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561db01980b0 con 0x561dafe8d070 2024-07-31T15:03:41.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.755+0000 7f852da09740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561db019c0f0 con 0x561dafe6c390 2024-07-31T15:03:41.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.755+0000 7f8525a55640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe8d070 0x561db0232710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.756+0000 7f8525a55640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe8d070 0x561db0232710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46102/0 (socket says 172.21.15.67:46102) 2024-07-31T15:03:41.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.756+0000 7f8525a55640 1 -- 172.21.15.67:0/1273003672 learned_addr learned my addr 172.21.15.67:0/1273003672 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:41.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.756+0000 7f8524a53640 1 --2- 172.21.15.67:0/1273003672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe6c390 0x561db0233190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.761+0000 7f8525254640 1 --2- 172.21.15.67:0/1273003672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe641b0 0x561db0232c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.765+0000 7f8525a55640 1 -- 172.21.15.67:0/1273003672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe6c390 msgr2=0x561db0233190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:41.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.765+0000 7f851e7fc640 1 --2- 172.21.15.67:0/3482131708 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f85080b3f50 0x7f85080b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.768+0000 7f851e7fc640 1 -- 172.21.15.67:0/3482131708 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f8514105be0 con 0x7f85201114b0 2024-07-31T15:03:41.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.768+0000 7f8525a55640 1 --2- 172.21.15.67:0/1273003672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe6c390 0x561db0233190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.768+0000 7f8525a55640 1 -- 172.21.15.67:0/1273003672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe641b0 msgr2=0x561db0232c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:41.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.768+0000 7f8525a55640 1 --2- 172.21.15.67:0/1273003672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe641b0 0x561db0232c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.768+0000 7f8525a55640 1 -- 172.21.15.67:0/1273003672 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561db013eb90 con 0x561dafe8d070 2024-07-31T15:03:41.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.768+0000 7f8525a55640 1 --2- 172.21.15.67:0/3482131708 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f85080b3f50 0x7f85080b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.769+0000 7f8525a55640 1 --2- 172.21.15.67:0/1273003672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe8d070 0x561db0232710 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8518009c60 tx=0x7f85180117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e7cf20afffb622b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:41.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.769+0000 7f85077fe640 1 -- 172.21.15.67:0/1273003672 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851801c700 con 0x561dafe8d070 2024-07-31T15:03:41.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.769+0000 7f85077fe640 1 -- 172.21.15.67:0/1273003672 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f851801cd50 con 0x561dafe8d070 2024-07-31T15:03:41.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.771+0000 7f852da09740 1 -- 172.21.15.67:0/1273003672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe8d070 msgr2=0x561db0232710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:41.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.771+0000 7f852da09740 1 --2- 172.21.15.67:0/1273003672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe8d070 0x561db0232710 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8518009c60 tx=0x7f85180117b0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.776 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/apic/apic_noop.o 2024-07-31T15:03:41.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.771+0000 7f8525a55640 1 --2- 172.21.15.67:0/3482131708 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f85080b3f50 0x7f85080b6460 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8518022000 tx=0x7f8518023000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:41.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.775+0000 7f8524a53640 1 --2- 172.21.15.67:0/1273003672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe6c390 0x561db0233190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:03:41.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.777+0000 7f852da09740 1 -- 172.21.15.67:0/1273003672 shutdown_connections 2024-07-31T15:03:41.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.777+0000 7f852da09740 1 --2- 172.21.15.67:0/1273003672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe6c390 0x561db0233190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.777+0000 7f852da09740 1 --2- 172.21.15.67:0/1273003672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe641b0 0x561db0232c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.777+0000 7f852da09740 1 --2- 172.21.15.67:0/1273003672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe8d070 0x561db0232710 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.777+0000 7f852da09740 1 -- 172.21.15.67:0/1273003672 >> 172.21.15.67:0/1273003672 conn(0x561dafe81460 msgr2=0x561db0230600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:41.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.783+0000 7f852da09740 1 -- 172.21.15.67:0/1273003672 shutdown_connections 2024-07-31T15:03:41.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.784+0000 7f852da09740 1 -- 172.21.15.67:0/1273003672 wait complete. 2024-07-31T15:03:41.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.784+0000 7f852da09740 1 -- start start 2024-07-31T15:03:41.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.784+0000 7f852da09740 1 Processor -- start 2024-07-31T15:03:41.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.786+0000 7f852da09740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe641b0 0x561db023e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.786+0000 7f852da09740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe6c390 0x561db023e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.786+0000 7f852da09740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe8d070 0x561db024a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.786+0000 7f852da09740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561dafffb700 con 0x561dafe641b0 2024-07-31T15:03:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.786+0000 7f852da09740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561db013a9e0 con 0x561dafe6c390 2024-07-31T15:03:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.786+0000 7f852da09740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561db0240f40 con 0x561dafe8d070 2024-07-31T15:03:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.786+0000 7f8525a55640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe8d070 0x561db024a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.786+0000 7f8525a55640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe8d070 0x561db024a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:36344/0 (socket says 172.21.15.67:36344) 2024-07-31T15:03:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.786+0000 7f8525a55640 1 -- 172.21.15.67:0/4110608428 learned_addr learned my addr 172.21.15.67:0/4110608428 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.787+0000 7f852da09740 10 client.0 _ll_register_callbacks cb 0x561db01554f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:41.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.787+0000 7f8524a53640 1 --2- 172.21.15.67:0/4110608428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe6c390 0x561db023e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.788+0000 7f8525254640 1 --2- 172.21.15.67:0/4110608428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe641b0 0x561db023e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.789+0000 7f8525a55640 1 -- 172.21.15.67:0/4110608428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe6c390 msgr2=0x561db023e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:41.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.789+0000 7f8525a55640 1 --2- 172.21.15.67:0/4110608428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe6c390 0x561db023e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.789+0000 7f8525a55640 1 -- 172.21.15.67:0/4110608428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe641b0 msgr2=0x561db023e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:41.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.789+0000 7f8525a55640 1 --2- 172.21.15.67:0/4110608428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe641b0 0x561db023e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.789+0000 7f8525a55640 1 -- 172.21.15.67:0/4110608428 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dafef8650 con 0x561dafe8d070 2024-07-31T15:03:41.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.794+0000 7f8525a55640 1 --2- 172.21.15.67:0/4110608428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe8d070 0x561db024a310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8518011cb0 tx=0x7f851804b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=496c6aed340f816b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:41.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.794+0000 7f8504ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:03:41.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.797+0000 7f8504ff9640 1 -- 172.21.15.67:0/4110608428 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518009920 con 0x561dafe8d070 2024-07-31T15:03:41.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.797+0000 7f8504ff9640 1 -- 172.21.15.67:0/4110608428 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f8518005260 con 0x561dafe8d070 2024-07-31T15:03:41.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.801+0000 7f852da09740 1 -- 172.21.15.67:0/4110608428 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f851801cd50 con 0x561dafe8d070 2024-07-31T15:03:41.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.806+0000 7f8504ff9640 1 -- 172.21.15.67:0/4110608428 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f851800f410 con 0x561dafe8d070 2024-07-31T15:03:41.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.809+0000 7f852da09740 10 client.27712 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:41.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.809+0000 7f852da09740 1 -- 172.21.15.67:0/4110608428 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x561db024ce90 con 0x561dafe8d070 2024-07-31T15:03:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.814+0000 7f8504ff9640 1 -- 172.21.15.67:0/4110608428 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f851801cd50 con 0x561dafe8d070 2024-07-31T15:03:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.814+0000 7f8504ff9640 1 -- 172.21.15.67:0/4110608428 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f8518004680 con 0x561dafe8d070 2024-07-31T15:03:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.815+0000 7f852da09740 10 client.27712 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.815+0000 7f852da09740 10 client.27712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.815+0000 7f852da09740 20 client.27712 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:41.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.815+0000 7f852da09740 1 --2- 172.21.15.67:0/4110608428 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561dafe4b9f0 0x561db02501f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.815+0000 7f852da09740 4 client.27712 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:03:41.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.815+0000 7f852da09740 1 -- 172.21.15.67:0/4110608428 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561db01aab00 con 0x561dafe4b9f0 2024-07-31T15:03:41.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.818+0000 7f8525254640 1 --2- 172.21.15.67:0/4110608428 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561dafe4b9f0 0x561db02501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.831+0000 7f8525254640 1 --2- 172.21.15.67:0/4110608428 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561dafe4b9f0 0x561db02501f0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9a5f2c9b538aa13 server_cookie=f0318a36480f9fc5 in_seq=0 out_seq=0 2024-07-31T15:03:41.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.832+0000 7f8504ff9640 10 client.27712 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:41.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.839+0000 7f8504ff9640 1 -- 172.21.15.67:0/4110608428 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561db01aab00 con 0x561dafe4b9f0 2024-07-31T15:03:41.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.839+0000 7f8504ff9640 10 client.27712 handle_command_reply: tid=0 2024-07-31T15:03:41.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.839+0000 7f852da09740 1 client.27712 shutdown 2024-07-31T15:03:41.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.846+0000 7f852da09740 1 -- 172.21.15.67:0/4110608428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe8d070 msgr2=0x561db024a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:41.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.846+0000 7f852da09740 1 --2- 172.21.15.67:0/4110608428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe8d070 0x561db024a310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8518011cb0 tx=0x7f851804b6b0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.850+0000 7f852da09740 1 -- 172.21.15.67:0/4110608428 shutdown_connections 2024-07-31T15:03:41.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.850+0000 7f852da09740 1 --2- 172.21.15.67:0/4110608428 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561dafe4b9f0 0x561db02501f0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.850+0000 7f852da09740 1 --2- 172.21.15.67:0/4110608428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe8d070 0x561db024a310 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.850+0000 7f852da09740 1 --2- 172.21.15.67:0/4110608428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe6c390 0x561db023e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.850+0000 7f852da09740 1 --2- 172.21.15.67:0/4110608428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe641b0 0x561db023e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.850+0000 7f8504ff9640 0 client.27712 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:41.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.850+0000 7f852da09740 1 -- 172.21.15.67:0/4110608428 >> 172.21.15.67:0/4110608428 conn(0x561dafe81460 msgr2=0x561db02358a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:41.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.850+0000 7f852da09740 1 -- 172.21.15.67:0/4110608428 shutdown_connections 2024-07-31T15:03:41.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.851+0000 7f852da09740 1 -- 172.21.15.67:0/4110608428 wait complete. 2024-07-31T15:03:41.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.851+0000 7f852da09740 20 client.27712 trim_cache size 0 max 16384 2024-07-31T15:03:41.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.892+0000 7f852da09740 1 Processor -- start 2024-07-31T15:03:41.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.893+0000 7f852da09740 1 -- start start 2024-07-31T15:03:41.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.893+0000 7f852da09740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe641b0 0x561db0238830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.893+0000 7f852da09740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe6c390 0x561db0233e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.893+0000 7f852da09740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe8d070 0x561db024a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.893+0000 7f852da09740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f850c00ff00 con 0x561dafe8d070 2024-07-31T15:03:41.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.893+0000 7f852da09740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f850c011a40 con 0x561dafe6c390 2024-07-31T15:03:41.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.893+0000 7f852da09740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f850c0105d0 con 0x561dafe641b0 2024-07-31T15:03:41.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.893+0000 7f8525a55640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe8d070 0x561db024a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.895+0000 7f8525254640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe641b0 0x561db0238830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.895+0000 7f8525254640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe641b0 0x561db0238830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:36360/0 (socket says 172.21.15.67:36360) 2024-07-31T15:03:41.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.895+0000 7f8525254640 1 -- 172.21.15.67:0/2936301191 learned_addr learned my addr 172.21.15.67:0/2936301191 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:41.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.897+0000 7f8524a53640 1 --2- 172.21.15.67:0/2936301191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe6c390 0x561db0233e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.902+0000 7f8525254640 1 -- 172.21.15.67:0/2936301191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe6c390 msgr2=0x561db0233e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:41.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.902+0000 7f8525254640 1 --2- 172.21.15.67:0/2936301191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe6c390 0x561db0233e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.902+0000 7f8525254640 1 -- 172.21.15.67:0/2936301191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe8d070 msgr2=0x561db024a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:41.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.902+0000 7f8525254640 1 --2- 172.21.15.67:0/2936301191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe8d070 0x561db024a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.902+0000 7f8525254640 1 -- 172.21.15.67:0/2936301191 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561db023f2f0 con 0x561dafe641b0 2024-07-31T15:03:41.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.910+0000 7f8525254640 1 --2- 172.21.15.67:0/2936301191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe641b0 0x561db0238830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8510009da0 tx=0x7f8510005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d67a3e1e8d254bb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.914+0000 7f8506ffd640 1 -- 172.21.15.67:0/2936301191 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851001b5b0 con 0x561dafe641b0 2024-07-31T15:03:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.914+0000 7f8506ffd640 1 -- 172.21.15.67:0/2936301191 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f851001f030 con 0x561dafe641b0 2024-07-31T15:03:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.914+0000 7f8506ffd640 1 -- 172.21.15.67:0/2936301191 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85100233d0 con 0x561dafe641b0 2024-07-31T15:03:41.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.916+0000 7f852da09740 1 -- 172.21.15.67:0/2936301191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe641b0 msgr2=0x561db0238830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:41.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.916+0000 7f852da09740 1 --2- 172.21.15.67:0/2936301191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe641b0 0x561db0238830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8510009da0 tx=0x7f8510005bd0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.917+0000 7f852da09740 1 -- 172.21.15.67:0/2936301191 shutdown_connections 2024-07-31T15:03:41.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.917+0000 7f852da09740 1 --2- 172.21.15.67:0/2936301191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe8d070 0x561db024a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.917+0000 7f852da09740 1 --2- 172.21.15.67:0/2936301191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe6c390 0x561db0233e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.917+0000 7f852da09740 1 --2- 172.21.15.67:0/2936301191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe641b0 0x561db0238830 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.917+0000 7f852da09740 1 -- 172.21.15.67:0/2936301191 >> 172.21.15.67:0/2936301191 conn(0x561dafe81460 msgr2=0x561db0233740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:41.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.925+0000 7f852da09740 1 -- 172.21.15.67:0/2936301191 shutdown_connections 2024-07-31T15:03:41.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.929+0000 7f852da09740 1 -- 172.21.15.67:0/2936301191 wait complete. 2024-07-31T15:03:41.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.929+0000 7f852da09740 1 -- start start 2024-07-31T15:03:41.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.929+0000 7f852da09740 1 Processor -- start 2024-07-31T15:03:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.930+0000 7f852da09740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe641b0 0x561db0250b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.930+0000 7f852da09740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe6c390 0x561db0251150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.930+0000 7f852da09740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe8d070 0x561db02b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.930+0000 7f852da09740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f850c011370 con 0x561dafe8d070 2024-07-31T15:03:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.930+0000 7f852da09740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f850c010ca0 con 0x561dafe641b0 2024-07-31T15:03:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.930+0000 7f852da09740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f850c00df00 con 0x561dafe6c390 2024-07-31T15:03:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.930+0000 7f8525a55640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe8d070 0x561db02b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.930+0000 7f8525a55640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe8d070 0x561db02b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36750/0 (socket says 172.21.15.67:36750) 2024-07-31T15:03:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.930+0000 7f8525a55640 1 -- 172.21.15.67:0/4034461713 learned_addr learned my addr 172.21.15.67:0/4034461713 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.931+0000 7f8525a55640 1 -- 172.21.15.67:0/4034461713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe6c390 msgr2=0x561db0251150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:03:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.931+0000 7f8525a55640 1 --2- 172.21.15.67:0/4034461713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe6c390 0x561db0251150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.931+0000 7f8525a55640 1 -- 172.21.15.67:0/4034461713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe641b0 msgr2=0x561db0250b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:03:41.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.931+0000 7f8525a55640 1 --2- 172.21.15.67:0/4034461713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe641b0 0x561db0250b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.931+0000 7f8525a55640 1 -- 172.21.15.67:0/4034461713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561db02b0de0 con 0x561dafe8d070 2024-07-31T15:03:41.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.931+0000 7f8525a55640 1 --2- 172.21.15.67:0/4034461713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe8d070 0x561db02b0770 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8518005020 tx=0x7f851800ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbf34525d0cbc0e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:41.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.932+0000 7f852da09740 10 client.0 _ll_register_callbacks cb 0x7f8518009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:41.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.932+0000 7f852da09740 1 -- 172.21.15.67:0/4034461713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f851001f030 con 0x561dafe8d070 2024-07-31T15:03:41.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.932+0000 7f8504ff9640 10 client.18213 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:41.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.932+0000 7f8504ff9640 1 -- 172.21.15.67:0/4034461713 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518065070 con 0x561dafe8d070 2024-07-31T15:03:41.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.932+0000 7f8504ff9640 1 -- 172.21.15.67:0/4034461713 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f8518005b90 con 0x561dafe8d070 2024-07-31T15:03:41.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.932+0000 7f8504ff9640 1 -- 172.21.15.67:0/4034461713 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8518009920 con 0x561dafe8d070 2024-07-31T15:03:41.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.934+0000 7f852da09740 10 client.18213 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:41.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.934+0000 7f852da09740 1 -- 172.21.15.67:0/4034461713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({fsmap=63}) -- 0x561db02da110 con 0x561dafe8d070 2024-07-31T15:03:41.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.936+0000 7f8504ff9640 1 -- 172.21.15.67:0/4034461713 <== mon.0 v2:172.21.15.67:3300/0 4 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f851800f5b0 con 0x561dafe8d070 2024-07-31T15:03:41.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.938+0000 7f852da09740 10 client.18213 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:41.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.938+0000 7f852da09740 10 client.18213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.938+0000 7f852da09740 20 client.18213 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.938+0000 7f852da09740 1 --2- 172.21.15.67:0/4034461713 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561dafe20da0 0x561db02dc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.938+0000 7f852da09740 4 client.18213 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:03:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.938+0000 7f852da09740 1 -- 172.21.15.67:0/4034461713 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f851001f030 con 0x561dafe20da0 2024-07-31T15:03:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.943+0000 7f8525254640 1 --2- 172.21.15.67:0/4034461713 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561dafe20da0 0x561db02dc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:41.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.958+0000 7f8525254640 1 --2- 172.21.15.67:0/4034461713 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561dafe20da0 0x561db02dc800 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2000902d89f199c7 server_cookie=29404fb6d5621bdd in_seq=0 out_seq=0 2024-07-31T15:03:41.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.958+0000 7f8504ff9640 10 client.18213 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:41.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.959+0000 7f8504ff9640 1 -- 172.21.15.67:0/4034461713 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f851001f030 con 0x561dafe20da0 2024-07-31T15:03:41.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.959+0000 7f8504ff9640 10 client.18213 handle_command_reply: tid=0 2024-07-31T15:03:41.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.959+0000 7f852da09740 1 client.18213 shutdown 2024-07-31T15:03:41.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.959+0000 7f852da09740 1 -- 172.21.15.67:0/4034461713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe8d070 msgr2=0x561db02b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:41.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.959+0000 7f852da09740 1 --2- 172.21.15.67:0/4034461713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe8d070 0x561db02b0770 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8518005020 tx=0x7f851800ff80 comp rx=0 tx=0).stop 2024-07-31T15:03:41.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.964+0000 7f852da09740 1 -- 172.21.15.67:0/4034461713 shutdown_connections 2024-07-31T15:03:41.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.964+0000 7f852da09740 1 --2- 172.21.15.67:0/4034461713 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561dafe20da0 0x561db02dc800 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.964+0000 7f852da09740 1 --2- 172.21.15.67:0/4034461713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561dafe8d070 0x561db02b0770 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.964+0000 7f852da09740 1 --2- 172.21.15.67:0/4034461713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dafe6c390 0x561db0251150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.964+0000 7f852da09740 1 --2- 172.21.15.67:0/4034461713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561dafe641b0 0x561db0250b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.964+0000 7f852da09740 1 -- 172.21.15.67:0/4034461713 >> 172.21.15.67:0/4034461713 conn(0x561dafe81460 msgr2=0x561db0233740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:41.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.964+0000 7f8504ff9640 0 client.18213 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:41.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.972+0000 7f852da09740 1 -- 172.21.15.67:0/4034461713 shutdown_connections 2024-07-31T15:03:41.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.972+0000 7f852da09740 1 -- 172.21.15.67:0/4034461713 wait complete. 2024-07-31T15:03:41.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.972+0000 7f852da09740 20 client.18213 trim_cache size 0 max 16384 2024-07-31T15:03:41.974 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:03:41.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.973+0000 7f8526256640 1 -- 172.21.15.67:0/3482131708 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f85080b3f50 msgr2=0x7f85080b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:41.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.973+0000 7f8526256640 1 --2- 172.21.15.67:0/3482131708 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f85080b3f50 0x7f85080b6460 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8518022000 tx=0x7f8518023000 comp rx=0 tx=0).stop 2024-07-31T15:03:41.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.973+0000 7f8526256640 1 -- 172.21.15.67:0/3482131708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85201114b0 msgr2=0x7f852010af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:41.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.973+0000 7f8526256640 1 --2- 172.21.15.67:0/3482131708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85201114b0 0x7f852010af20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8514009850 tx=0x7f85140305d0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.973+0000 7f8526256640 1 -- 172.21.15.67:0/3482131708 shutdown_connections 2024-07-31T15:03:41.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.973+0000 7f8526256640 1 --2- 172.21.15.67:0/3482131708 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f85080b3f50 0x7f85080b6460 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.973+0000 7f8526256640 1 --2- 172.21.15.67:0/3482131708 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520113e10 0x7f852010b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.973+0000 7f8526256640 1 --2- 172.21.15.67:0/3482131708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85201114b0 0x7f852010af20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.973+0000 7f8526256640 1 --2- 172.21.15.67:0/3482131708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8520101800 0x7f8520110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:41.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.973+0000 7f8526256640 1 -- 172.21.15.67:0/3482131708 >> 172.21.15.67:0/3482131708 conn(0x7f85200774d0 msgr2=0x7f8520114220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:41.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.977+0000 7f8526256640 1 -- 172.21.15.67:0/3482131708 shutdown_connections 2024-07-31T15:03:41.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:41.978+0000 7f8526256640 1 -- 172.21.15.67:0/3482131708 wait complete. 2024-07-31T15:03:42.095 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/apic/ipi.o 2024-07-31T15:03:42.124 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/bugs.o 2024-07-31T15:03:42.525 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/block_validity.o 2024-07-31T15:03:42.583 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/kprobes/opt.o 2024-07-31T15:03:42.640 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/policydb.o 2024-07-31T15:03:42.656 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/apic/vector.o 2024-07-31T15:03:42.706 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/netfs/objects.o 2024-07-31T15:03:42.868 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/affinity.o 2024-07-31T15:03:43.356 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/matrix.o 2024-07-31T15:03:43.380 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/kernel/kprobes/built-in.a 2024-07-31T15:03:43.384 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/dir.o 2024-07-31T15:03:43.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:43 smithi073 ceph-mon[36714]: pgmap v799: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 151 KiB/s wr, 47 op/s 2024-07-31T15:03:43.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:43 smithi067 ceph-mon[32750]: pgmap v799: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 151 KiB/s wr, 47 op/s 2024-07-31T15:03:43.407 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/netfs/built-in.a 2024-07-31T15:03:43.434 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/ext4_jbd2.o 2024-07-31T15:03:43.443 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/services.o 2024-07-31T15:03:43.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:43 smithi077 ceph-mon[37327]: pgmap v799: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 151 KiB/s wr, 47 op/s 2024-07-31T15:03:43.690 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/aperfmperf.o 2024-07-31T15:03:43.867 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T15:03:43.867 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T15:03:43.923 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/apic/hw_nmi.o 2024-07-31T15:03:44.287 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/cpuid-deps.o 2024-07-31T15:03:44.365 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/apic/io_apic.o 2024-07-31T15:03:44.375 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/irq/built-in.a 2024-07-31T15:03:44.479 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/rcu/update.o 2024-07-31T15:03:44.521 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:03:44.547 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/rcu/sync.o 2024-07-31T15:03:44.647 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/extents.o 2024-07-31T15:03:44.680 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/umwait.o 2024-07-31T15:03:44.714 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/apic/msi.o 2024-07-31T15:03:44.824 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/rcu/srcutree.o 2024-07-31T15:03:45.021 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mincore.o 2024-07-31T15:03:45.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:45 smithi067 ceph-mon[32750]: pgmap v800: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 237 KiB/s wr, 73 op/s 2024-07-31T15:03:45.306 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/proc.o 2024-07-31T15:03:45.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:45 smithi073 ceph-mon[36714]: pgmap v800: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 237 KiB/s wr, 73 op/s 2024-07-31T15:03:45.446 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/apic/apic_flat_64.o 2024-07-31T15:03:45.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:45 smithi077 ceph-mon[37327]: pgmap v800: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 237 KiB/s wr, 73 op/s 2024-07-31T15:03:45.606 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/conditional.o 2024-07-31T15:03:45.764 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/rcu/tree.o 2024-07-31T15:03:45.777 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/apic/probe_64.o 2024-07-31T15:03:45.795 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mlock.o 2024-07-31T15:03:45.881 INFO:tasks.workunit.client.0.smithi067.stdout: MKCAP arch/x86/kernel/cpu/capflags.c 2024-07-31T15:03:45.929 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mmap.o 2024-07-31T15:03:46.064 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/kernel/apic/built-in.a 2024-07-31T15:03:46.105 INFO:tasks.workunit.client.0.smithi067.stdout: LDS arch/x86/kernel/vmlinux.lds 2024-07-31T15:03:46.167 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/kernel/head_64.o 2024-07-31T15:03:46.189 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/mls.o 2024-07-31T15:03:46.235 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/context.o 2024-07-31T15:03:46.665 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/head64.o 2024-07-31T15:03:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.783+0000 7f6aad186640 1 -- 172.21.15.67:0/2233684533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa810a330 msgr2=0x7f6aa810a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.783+0000 7f6aad186640 1 --2- 172.21.15.67:0/2233684533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa810a330 0x7f6aa810a740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009ca0 tx=0x7f6aa002a8e0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.784+0000 7f6aad186640 1 -- 172.21.15.67:0/2233684533 shutdown_connections 2024-07-31T15:03:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.784+0000 7f6aad186640 1 --2- 172.21.15.67:0/2233684533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa811c5d0 0x7f6aa811e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.784+0000 7f6aad186640 1 --2- 172.21.15.67:0/2233684533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa810a330 0x7f6aa810a740 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.784+0000 7f6aad186640 1 --2- 172.21.15.67:0/2233684533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6aa8111650 0x7f6aa8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.784+0000 7f6aad186640 1 -- 172.21.15.67:0/2233684533 >> 172.21.15.67:0/2233684533 conn(0x7f6aa806e320 msgr2=0x7f6aa806e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.784+0000 7f6aad186640 1 -- 172.21.15.67:0/2233684533 shutdown_connections 2024-07-31T15:03:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.784+0000 7f6aad186640 1 -- 172.21.15.67:0/2233684533 wait complete. 2024-07-31T15:03:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.785+0000 7f6aad186640 1 Processor -- start 2024-07-31T15:03:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.785+0000 7f6aad186640 1 -- start start 2024-07-31T15:03:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.786+0000 7f6aad186640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa810a330 0x7f6aa81a8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.786+0000 7f6aad186640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa8111650 0x7f6aa81a9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.786+0000 7f6aad186640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6aa811c5d0 0x7f6aa81ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.786+0000 7f6aad186640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa8122060 con 0x7f6aa810a330 2024-07-31T15:03:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.786+0000 7f6aad186640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa8121ee0 con 0x7f6aa8111650 2024-07-31T15:03:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.786+0000 7f6aad186640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81221e0 con 0x7f6aa811c5d0 2024-07-31T15:03:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.786+0000 7f6aa77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa8111650 0x7f6aa81a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.786+0000 7f6aa7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa810a330 0x7f6aa81a8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.786+0000 7f6aa77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa8111650 0x7f6aa81a9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46132/0 (socket says 172.21.15.67:46132) 2024-07-31T15:03:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.786+0000 7f6aa77fe640 1 -- 172.21.15.67:0/3615233119 learned_addr learned my addr 172.21.15.67:0/3615233119 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.787+0000 7f6aa77fe640 1 -- 172.21.15.67:0/3615233119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6aa811c5d0 msgr2=0x7f6aa81ae2d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:03:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.787+0000 7f6aa77fe640 1 --2- 172.21.15.67:0/3615233119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6aa811c5d0 0x7f6aa81ae2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.787+0000 7f6aa77fe640 1 -- 172.21.15.67:0/3615233119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa810a330 msgr2=0x7f6aa81a8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.787+0000 7f6aa77fe640 1 --2- 172.21.15.67:0/3615233119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa810a330 0x7f6aa81a8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.787+0000 7f6aa77fe640 1 -- 172.21.15.67:0/3615233119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa81ae9d0 con 0x7f6aa8111650 2024-07-31T15:03:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.787+0000 7f6aa7fff640 1 --2- 172.21.15.67:0/3615233119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa810a330 0x7f6aa81a8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:03:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.787+0000 7f6aa77fe640 1 --2- 172.21.15.67:0/3615233119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa8111650 0x7f6aa81a9040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0000f80 tx=0x7f6aa0003ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.787+0000 7f6aa57fa640 1 -- 172.21.15.67:0/3615233119 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa003f070 con 0x7f6aa8111650 2024-07-31T15:03:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.787+0000 7f6aad186640 1 -- 172.21.15.67:0/3615233119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa81aec60 con 0x7f6aa8111650 2024-07-31T15:03:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.788+0000 7f6aad186640 1 -- 172.21.15.67:0/3615233119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa81b5450 con 0x7f6aa8111650 2024-07-31T15:03:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.788+0000 7f6aa57fa640 1 -- 172.21.15.67:0/3615233119 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa0036040 con 0x7f6aa8111650 2024-07-31T15:03:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.789+0000 7f6aa57fa640 1 -- 172.21.15.67:0/3615233119 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0030eb0 con 0x7f6aa8111650 2024-07-31T15:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.793+0000 7f6ab53b9740 1 Processor -- start 2024-07-31T15:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.794+0000 7f6aa57fa640 1 -- 172.21.15.67:0/3615233119 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6aa0004450 con 0x7f6aa8111650 2024-07-31T15:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.795+0000 7f6aa57fa640 1 --2- 172.21.15.67:0/3615233119 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6a900b3e80 0x7f6a900b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.795+0000 7f6aa57fa640 1 -- 172.21.15.67:0/3615233119 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f6aa0105300 con 0x7f6aa8111650 2024-07-31T15:03:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.796+0000 7f6ab53b9740 1 -- start start 2024-07-31T15:03:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.796+0000 7f6ab53b9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b74ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.796+0000 7f6ab53b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 0x557905b753e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.796+0000 7f6ab53b9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 0x557905b75920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.796+0000 7f6ab53b9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557905aea3b0 con 0x5579057b14d0 2024-07-31T15:03:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.796+0000 7f6ab53b9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557905ad98b0 con 0x5579057d21b0 2024-07-31T15:03:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.796+0000 7f6ab53b9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557905add8f0 con 0x5579057a92f0 2024-07-31T15:03:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.796+0000 7f6aac985640 1 --2- 172.21.15.67:0/3615233119 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6a900b3e80 0x7f6a900b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.797+0000 7f6aa7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b74ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.797+0000 7f6aa7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b74ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46146/0 (socket says 172.21.15.67:46146) 2024-07-31T15:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.797+0000 7f6aa7fff640 1 -- 172.21.15.67:0/3512791429 learned_addr learned my addr 172.21.15.67:0/3512791429 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.797+0000 7f6aa7fff640 1 -- 172.21.15.67:0/3512791429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 msgr2=0x557905b753e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.797+0000 7f6aa7fff640 1 --2- 172.21.15.67:0/3512791429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 0x557905b753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.797+0000 7f6aa7fff640 1 -- 172.21.15.67:0/3512791429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 msgr2=0x557905b75920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:03:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.797+0000 7f6aa7fff640 1 --2- 172.21.15.67:0/3512791429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 0x557905b75920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.797+0000 7f6aa7fff640 1 -- 172.21.15.67:0/3512791429 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557905a809a0 con 0x5579057d21b0 2024-07-31T15:03:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.798+0000 7f6aa7fff640 1 --2- 172.21.15.67:0/3512791429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b74ea0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6a980114f0 tx=0x7f6a980119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dece58b57f0df9ff server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.798+0000 7f6aac985640 1 --2- 172.21.15.67:0/3615233119 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6a900b3e80 0x7f6a900b6390 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00a8d0 tx=0x7f6a9c005be0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.798+0000 7f6a8e7fc640 1 -- 172.21.15.67:0/3512791429 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98004020 con 0x5579057d21b0 2024-07-31T15:03:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.799+0000 7f6a8e7fc640 1 -- 172.21.15.67:0/3512791429 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a980054d0 con 0x5579057d21b0 2024-07-31T15:03:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.799+0000 7f6a8e7fc640 1 -- 172.21.15.67:0/3512791429 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98004d10 con 0x5579057d21b0 2024-07-31T15:03:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.800+0000 7f6ab53b9740 1 -- 172.21.15.67:0/3512791429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 msgr2=0x557905b74ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.800+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/3512791429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b74ea0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6a980114f0 tx=0x7f6a980119b0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.800+0000 7f6ab53b9740 1 -- 172.21.15.67:0/3512791429 shutdown_connections 2024-07-31T15:03:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.800+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/3512791429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 0x557905b75920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.800+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/3512791429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 0x557905b753e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.800+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/3512791429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b74ea0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.800+0000 7f6ab53b9740 1 -- 172.21.15.67:0/3512791429 >> 172.21.15.67:0/3512791429 conn(0x5579057c6300 msgr2=0x557905b72d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.800+0000 7f6ab53b9740 1 -- 172.21.15.67:0/3512791429 shutdown_connections 2024-07-31T15:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.800+0000 7f6ab53b9740 1 -- 172.21.15.67:0/3512791429 wait complete. 2024-07-31T15:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.801+0000 7f6ab53b9740 1 -- start start 2024-07-31T15:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.801+0000 7f6ab53b9740 1 Processor -- start 2024-07-31T15:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.802+0000 7f6ab53b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 0x557905b80ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.802+0000 7f6ab53b9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 0x557905b80ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.802+0000 7f6ab53b9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b8c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.802+0000 7f6ab53b9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557905a13ab0 con 0x5579057b14d0 2024-07-31T15:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.802+0000 7f6ab53b9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557905a61cf0 con 0x5579057d21b0 2024-07-31T15:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.802+0000 7f6ab53b9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557905b83670 con 0x5579057a92f0 2024-07-31T15:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.802+0000 7f6ab53b9740 10 client.0 _ll_register_callbacks cb 0x557905ad39f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.802+0000 7f6aa77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 0x557905b80ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.802+0000 7f6aa77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 0x557905b80ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36782/0 (socket says 172.21.15.67:36782) 2024-07-31T15:03:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.802+0000 7f6aa77fe640 1 -- 172.21.15.67:0/1054575327 learned_addr learned my addr 172.21.15.67:0/1054575327 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.803+0000 7f6aa7fff640 1 --2- 172.21.15.67:0/1054575327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 0x557905b80ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.803+0000 7f6aac985640 1 --2- 172.21.15.67:0/1054575327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b8c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.803+0000 7f6aac985640 1 -- 172.21.15.67:0/1054575327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 msgr2=0x557905b80ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.803+0000 7f6aac985640 1 --2- 172.21.15.67:0/1054575327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 0x557905b80ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.803+0000 7f6aac985640 1 -- 172.21.15.67:0/1054575327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 msgr2=0x557905b80ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.803+0000 7f6aac985640 1 --2- 172.21.15.67:0/1054575327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 0x557905b80ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.803+0000 7f6aac985640 1 -- 172.21.15.67:0/1054575327 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55790583d790 con 0x5579057d21b0 2024-07-31T15:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.803+0000 7f6aa77fe640 1 --2- 172.21.15.67:0/1054575327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 0x557905b80ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.804+0000 7f6aac985640 1 --2- 172.21.15.67:0/1054575327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b8c9f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00c6a0 tx=0x7f6a9c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad1396cd8dd77ea0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.804+0000 7f6a877fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.804+0000 7f6a877fe640 1 -- 172.21.15.67:0/1054575327 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c020070 con 0x5579057d21b0 2024-07-31T15:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.804+0000 7f6ab53b9740 1 -- 172.21.15.67:0/1054575327 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a980054d0 con 0x5579057d21b0 2024-07-31T15:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.804+0000 7f6a877fe640 1 -- 172.21.15.67:0/1054575327 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f6a9c005220 con 0x5579057d21b0 2024-07-31T15:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.804+0000 7f6a877fe640 1 -- 172.21.15.67:0/1054575327 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f6a9c02b050 con 0x5579057d21b0 2024-07-31T15:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.805+0000 7f6ab53b9740 10 client.27770 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.805+0000 7f6ab53b9740 1 -- 172.21.15.67:0/1054575327 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5579058b6620 con 0x5579057d21b0 2024-07-31T15:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.805+0000 7f6a877fe640 1 -- 172.21.15.67:0/1054575327 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a980054d0 con 0x5579057d21b0 2024-07-31T15:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.805+0000 7f6a877fe640 1 -- 172.21.15.67:0/1054575327 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f6a9c01b070 con 0x5579057d21b0 2024-07-31T15:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.806+0000 7f6ab53b9740 10 client.27770 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.806+0000 7f6ab53b9740 10 client.27770 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.806+0000 7f6ab53b9740 20 client.27770 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.806+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/1054575327 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557905790d80 0x557905b92f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.806+0000 7f6ab53b9740 4 client.27770 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:03:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.806+0000 7f6ab53b9740 1 -- 172.21.15.67:0/1054575327 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557905aec300 con 0x557905790d80 2024-07-31T15:03:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.806+0000 7f6aa7fff640 1 --2- 172.21.15.67:0/1054575327 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557905790d80 0x557905b92f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.807+0000 7f6a877fe640 10 client.27770 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.808+0000 7f6aa7fff640 1 --2- 172.21.15.67:0/1054575327 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557905790d80 0x557905b92f80 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6327e7f2156e980 server_cookie=186ac9fab43a2e41 in_seq=0 out_seq=0 2024-07-31T15:03:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.809+0000 7f6a877fe640 1 -- 172.21.15.67:0/1054575327 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557905aec300 con 0x557905790d80 2024-07-31T15:03:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.809+0000 7f6a877fe640 10 client.27770 handle_command_reply: tid=0 2024-07-31T15:03:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.809+0000 7f6ab53b9740 1 client.27770 shutdown 2024-07-31T15:03:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.811+0000 7f6ab53b9740 1 -- 172.21.15.67:0/1054575327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 msgr2=0x557905b8c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.811+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/1054575327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b8c9f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00c6a0 tx=0x7f6a9c004730 comp rx=0 tx=0).stop 2024-07-31T15:03:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.812+0000 7f6ab53b9740 1 -- 172.21.15.67:0/1054575327 shutdown_connections 2024-07-31T15:03:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.812+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/1054575327 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557905790d80 0x557905b92f80 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.812+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/1054575327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b8c9f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.812+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/1054575327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 0x557905b80ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.812+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/1054575327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 0x557905b80ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.812+0000 7f6ab53b9740 1 -- 172.21.15.67:0/1054575327 >> 172.21.15.67:0/1054575327 conn(0x5579057c6300 msgr2=0x557905b780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.812+0000 7f6a877fe640 0 client.27770 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.812+0000 7f6ab53b9740 1 -- 172.21.15.67:0/1054575327 shutdown_connections 2024-07-31T15:03:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.812+0000 7f6ab53b9740 1 -- 172.21.15.67:0/1054575327 wait complete. 2024-07-31T15:03:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.812+0000 7f6ab53b9740 20 client.27770 trim_cache size 0 max 16384 2024-07-31T15:03:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.860+0000 7f6ab53b9740 1 Processor -- start 2024-07-31T15:03:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.860+0000 7f6ab53b9740 1 -- start start 2024-07-31T15:03:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.861+0000 7f6ab53b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 0x557905b80f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.861+0000 7f6ab53b9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 0x557905b8c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.862+0000 7f6aa77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 0x557905b8c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.862+0000 7f6aa7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 0x557905b80f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.862+0000 7f6aa7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 0x557905b80f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:36412/0 (socket says 172.21.15.67:36412) 2024-07-31T15:03:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.862+0000 7f6aa7fff640 1 -- 172.21.15.67:0/322302223 learned_addr learned my addr 172.21.15.67:0/322302223 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.862+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/322302223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b8cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.862+0000 7f6ab53b9740 1 -- 172.21.15.67:0/322302223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9400fdb0 con 0x5579057b14d0 2024-07-31T15:03:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.862+0000 7f6ab53b9740 1 -- 172.21.15.67:0/322302223 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6a940118f0 con 0x5579057d21b0 2024-07-31T15:03:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.862+0000 7f6ab53b9740 1 -- 172.21.15.67:0/322302223 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94010480 con 0x5579057a92f0 2024-07-31T15:03:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.862+0000 7f6aac985640 1 --2- 172.21.15.67:0/322302223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b8cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.863+0000 7f6aa7fff640 1 -- 172.21.15.67:0/322302223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 msgr2=0x557905b8cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.863+0000 7f6aa7fff640 1 --2- 172.21.15.67:0/322302223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b8cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.863+0000 7f6aa7fff640 1 -- 172.21.15.67:0/322302223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 msgr2=0x557905b8c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.863+0000 7f6aa7fff640 1 --2- 172.21.15.67:0/322302223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 0x557905b8c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.863+0000 7f6aa7fff640 1 -- 172.21.15.67:0/322302223 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557905b6cfe0 con 0x5579057a92f0 2024-07-31T15:03:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.864+0000 7f6aa7fff640 1 --2- 172.21.15.67:0/322302223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 0x557905b80f20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6a98014040 tx=0x7f6a980041d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ace7e786aa413f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.872+0000 7f6a8dffb640 1 -- 172.21.15.67:0/322302223 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9801d040 con 0x5579057a92f0 2024-07-31T15:03:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.872+0000 7f6a8dffb640 1 -- 172.21.15.67:0/322302223 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a98029030 con 0x5579057a92f0 2024-07-31T15:03:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.873+0000 7f6a8dffb640 1 -- 172.21.15.67:0/322302223 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98003ac0 con 0x5579057a92f0 2024-07-31T15:03:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.875+0000 7f6ab53b9740 1 -- 172.21.15.67:0/322302223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 msgr2=0x557905b80f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.875+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/322302223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 0x557905b80f20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6a98014040 tx=0x7f6a980041d0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.875+0000 7f6ab53b9740 1 -- 172.21.15.67:0/322302223 shutdown_connections 2024-07-31T15:03:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.875+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/322302223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b8cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.875+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/322302223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057b14d0 0x557905b8c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.875+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/322302223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057a92f0 0x557905b80f20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.875+0000 7f6ab53b9740 1 -- 172.21.15.67:0/322302223 >> 172.21.15.67:0/322302223 conn(0x5579057c6300 msgr2=0x557905b80810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.876+0000 7f6ab53b9740 1 -- 172.21.15.67:0/322302223 shutdown_connections 2024-07-31T15:03:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.876+0000 7f6ab53b9740 1 -- 172.21.15.67:0/322302223 wait complete. 2024-07-31T15:03:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.876+0000 7f6ab53b9740 1 -- start start 2024-07-31T15:03:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.877+0000 7f6ab53b9740 1 Processor -- start 2024-07-31T15:03:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.877+0000 7f6ab53b9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057a92f0 0x557905b890c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.877+0000 7f6ab53b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057b14d0 0x557905b85920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.877+0000 7f6ab53b9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b85f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.877+0000 7f6ab53b9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94011220 con 0x5579057a92f0 2024-07-31T15:03:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.877+0000 7f6ab53b9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94010b50 con 0x5579057d21b0 2024-07-31T15:03:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.877+0000 7f6ab53b9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9400ddb0 con 0x5579057b14d0 2024-07-31T15:03:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.877+0000 7f6ab53b9740 10 client.0 _ll_register_callbacks cb 0x7f6a9c00aef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.878+0000 7f6aa77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057b14d0 0x557905b85920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.878+0000 7f6aa77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057b14d0 0x557905b85920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:36442/0 (socket says 172.21.15.67:36442) 2024-07-31T15:03:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.878+0000 7f6aa77fe640 1 -- 172.21.15.67:0/634956637 learned_addr learned my addr 172.21.15.67:0/634956637 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.878+0000 7f6aac985640 1 --2- 172.21.15.67:0/634956637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b85f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.878+0000 7f6aac985640 1 -- 172.21.15.67:0/634956637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057b14d0 msgr2=0x557905b85920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.878+0000 7f6aac985640 1 --2- 172.21.15.67:0/634956637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057b14d0 0x557905b85920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.878+0000 7f6aac985640 1 -- 172.21.15.67:0/634956637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057a92f0 msgr2=0x557905b890c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.878+0000 7f6aac985640 1 --2- 172.21.15.67:0/634956637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057a92f0 0x557905b890c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.878+0000 7f6aac985640 1 -- 172.21.15.67:0/634956637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557905b86680 con 0x5579057d21b0 2024-07-31T15:03:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.879+0000 7f6aac985640 1 --2- 172.21.15.67:0/634956637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b85f20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c000c00 tx=0x7f6a9c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e43f6bbb6e799150 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.879+0000 7f6a877fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:03:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.879+0000 7f6a877fe640 1 -- 172.21.15.67:0/634956637 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c033070 con 0x5579057d21b0 2024-07-31T15:03:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.879+0000 7f6a877fe640 1 -- 172.21.15.67:0/634956637 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f6a9c005450 con 0x5579057d21b0 2024-07-31T15:03:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.879+0000 7f6ab53b9740 1 -- 172.21.15.67:0/634956637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a98029030 con 0x5579057d21b0 2024-07-31T15:03:46.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.883+0000 7f1b59c6b640 1 -- 172.21.15.67:0/1506521527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b5411c5d0 msgr2=0x7f1b5411e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:46.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.883+0000 7f1b59c6b640 1 --2- 172.21.15.67:0/1506521527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b5411c5d0 0x7f1b5411e9f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009ca0 tx=0x7f1b4c02a980 comp rx=0 tx=0).stop 2024-07-31T15:03:46.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.883+0000 7f1b59c6b640 1 -- 172.21.15.67:0/1506521527 shutdown_connections 2024-07-31T15:03:46.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.884+0000 7f1b59c6b640 1 --2- 172.21.15.67:0/1506521527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b5411c5d0 0x7f1b5411e9f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.884+0000 7f1b59c6b640 1 --2- 172.21.15.67:0/1506521527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b5410a330 0x7f1b5410a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.884+0000 7f1b59c6b640 1 --2- 172.21.15.67:0/1506521527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b54111790 0x7f1b54109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.884+0000 7f1b59c6b640 1 -- 172.21.15.67:0/1506521527 >> 172.21.15.67:0/1506521527 conn(0x7f1b5406e420 msgr2=0x7f1b5406e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:46.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.884+0000 7f1b59c6b640 1 -- 172.21.15.67:0/1506521527 shutdown_connections 2024-07-31T15:03:46.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.884+0000 7f1b59c6b640 1 -- 172.21.15.67:0/1506521527 wait complete. 2024-07-31T15:03:46.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.885+0000 7f1b59c6b640 1 Processor -- start 2024-07-31T15:03:46.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.885+0000 7f1b59c6b640 1 -- start start 2024-07-31T15:03:46.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.885+0000 7f1b59c6b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b5410a330 0x7f1b541a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.886+0000 7f1b59c6b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b54111790 0x7f1b541a9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.886+0000 7f1b59c6b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b5411c5d0 0x7f1b541ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.886+0000 7f1b59c6b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1b54122110 con 0x7f1b5411c5d0 2024-07-31T15:03:46.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.886+0000 7f1b59c6b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1b54121f90 con 0x7f1b5410a330 2024-07-31T15:03:46.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.886+0000 7f1b59c6b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b54122290 con 0x7f1b54111790 2024-07-31T15:03:46.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.886+0000 7f1b58c69640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b5410a330 0x7f1b541a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.886+0000 7f1b58c69640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b5410a330 0x7f1b541a8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46176/0 (socket says 172.21.15.67:46176) 2024-07-31T15:03:46.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.886+0000 7f1b58c69640 1 -- 172.21.15.67:0/4596797 learned_addr learned my addr 172.21.15.67:0/4596797 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:46.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.886+0000 7f1b5946a640 1 --2- 172.21.15.67:0/4596797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b5411c5d0 0x7f1b541ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.888+0000 7f1b5946a640 1 -- 172.21.15.67:0/4596797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b54111790 msgr2=0x7f1b541a9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:46.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.888+0000 7f1b5946a640 1 --2- 172.21.15.67:0/4596797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b54111790 0x7f1b541a9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.888+0000 7f1b5946a640 1 -- 172.21.15.67:0/4596797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b5410a330 msgr2=0x7f1b541a8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:46.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.888+0000 7f1b5946a640 1 --2- 172.21.15.67:0/4596797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b5410a330 0x7f1b541a8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.888+0000 7f1b5946a640 1 -- 172.21.15.67:0/4596797 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b541aea00 con 0x7f1b5411c5d0 2024-07-31T15:03:46.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.888+0000 7f1b5946a640 1 --2- 172.21.15.67:0/4596797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b5411c5d0 0x7f1b541ae300 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009bf0 tx=0x7f1b4c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:46.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.890+0000 7f1b51ffb640 1 -- 172.21.15.67:0/4596797 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c047070 con 0x7f1b5411c5d0 2024-07-31T15:03:46.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.890+0000 7f1b59c6b640 1 -- 172.21.15.67:0/4596797 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b541aec90 con 0x7f1b5411c5d0 2024-07-31T15:03:46.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.890+0000 7f1b59c6b640 1 -- 172.21.15.67:0/4596797 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b54111db0 con 0x7f1b5411c5d0 2024-07-31T15:03:46.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.891+0000 7f1b51ffb640 1 -- 172.21.15.67:0/4596797 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b4c0098b0 con 0x7f1b5411c5d0 2024-07-31T15:03:46.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.891+0000 7f1b51ffb640 1 -- 172.21.15.67:0/4596797 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c034070 con 0x7f1b5411c5d0 2024-07-31T15:03:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.893+0000 7f6a877fe640 1 -- 172.21.15.67:0/634956637 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f6a9c03d050 con 0x5579057d21b0 2024-07-31T15:03:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.894+0000 7f6ab53b9740 10 client.27779 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.894+0000 7f6ab53b9740 1 -- 172.21.15.67:0/634956637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x557905bfa6d0 con 0x5579057d21b0 2024-07-31T15:03:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.897+0000 7f6a877fe640 1 -- 172.21.15.67:0/634956637 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a98029030 con 0x5579057d21b0 2024-07-31T15:03:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.897+0000 7f6a877fe640 1 -- 172.21.15.67:0/634956637 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f6a9c01b040 con 0x5579057d21b0 2024-07-31T15:03:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.897+0000 7f6ab53b9740 10 client.27779 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.897+0000 7f6ab53b9740 10 client.27779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.897+0000 7f6ab53b9740 20 client.27779 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:46.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.894+0000 7f1b6141e740 1 Processor -- start 2024-07-31T15:03:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.897+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/634956637 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5579057661c0 0x557905bfcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.897+0000 7f6ab53b9740 4 client.27779 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:03:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.897+0000 7f6ab53b9740 1 -- 172.21.15.67:0/634956637 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557905ae7da0 con 0x5579057661c0 2024-07-31T15:03:46.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.898+0000 7f1b6141e740 1 -- start start 2024-07-31T15:03:46.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.898+0000 7f1b6141e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e50a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.898+0000 7f1b6141e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e50aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.898+0000 7f1b6141e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 0x55f15e50b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.898+0000 7f1b6141e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f15e3f22f0 con 0x55f15e165350 2024-07-31T15:03:46.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.898+0000 7f1b6141e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f15e480a80 con 0x55f15e13c490 2024-07-31T15:03:46.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.898+0000 7f1b6141e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f15e46ff80 con 0x55f15e144670 2024-07-31T15:03:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.899+0000 7f6aa7fff640 1 --2- 172.21.15.67:0/634956637 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5579057661c0 0x557905bfcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.899+0000 7f1b51ffb640 1 -- 172.21.15.67:0/4596797 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1b4c037020 con 0x7f1b5411c5d0 2024-07-31T15:03:46.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.899+0000 7f1b58c69640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e50aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.899+0000 7f1b58c69640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e50aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46178/0 (socket says 172.21.15.67:46178) 2024-07-31T15:03:46.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.899+0000 7f1b58c69640 1 -- 172.21.15.67:0/3266273963 learned_addr learned my addr 172.21.15.67:0/3266273963 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:46.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.900+0000 7f1b58c69640 1 -- 172.21.15.67:0/3266273963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 msgr2=0x55f15e50b000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:03:46.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.900+0000 7f1b58c69640 1 --2- 172.21.15.67:0/3266273963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 0x55f15e50b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.900+0000 7f1b58c69640 1 -- 172.21.15.67:0/3266273963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 msgr2=0x55f15e50a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:46.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.900+0000 7f1b58c69640 1 --2- 172.21.15.67:0/3266273963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e50a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.900+0000 7f1b58c69640 1 -- 172.21.15.67:0/3266273963 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f15e1d0930 con 0x55f15e13c490 2024-07-31T15:03:46.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.900+0000 7f1b58c69640 1 --2- 172.21.15.67:0/3266273963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e50aac0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400bd90 tx=0x7f1b4400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d214c156d8f2d021 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:46.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.901+0000 7f1b51ffb640 1 --2- 172.21.15.67:0/4596797 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1b3c0b3e70 0x7f1b3c0b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.902+0000 7f1b51ffb640 1 -- 172.21.15.67:0/4596797 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f1b4c104840 con 0x7f1b5411c5d0 2024-07-31T15:03:46.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.903+0000 7f1b3affd640 1 -- 172.21.15.67:0/3266273963 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44015070 con 0x55f15e13c490 2024-07-31T15:03:46.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.903+0000 7f1b3affd640 1 -- 172.21.15.67:0/3266273963 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b440035f0 con 0x55f15e13c490 2024-07-31T15:03:46.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.903+0000 7f1b3affd640 1 -- 172.21.15.67:0/3266273963 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b440116a0 con 0x55f15e13c490 2024-07-31T15:03:46.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.904+0000 7f1b6141e740 1 -- 172.21.15.67:0/3266273963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 msgr2=0x55f15e50aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:46.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.904+0000 7f1b6141e740 1 --2- 172.21.15.67:0/3266273963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e50aac0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400bd90 tx=0x7f1b4400f730 comp rx=0 tx=0).stop 2024-07-31T15:03:46.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.904+0000 7f1b6141e740 1 -- 172.21.15.67:0/3266273963 shutdown_connections 2024-07-31T15:03:46.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.904+0000 7f1b6141e740 1 --2- 172.21.15.67:0/3266273963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 0x55f15e50b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.904+0000 7f1b6141e740 1 --2- 172.21.15.67:0/3266273963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e50aac0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.904+0000 7f1b6141e740 1 --2- 172.21.15.67:0/3266273963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e50a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.904+0000 7f1b6141e740 1 -- 172.21.15.67:0/3266273963 >> 172.21.15.67:0/3266273963 conn(0x55f15e159740 msgr2=0x55f15e508470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:46.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.904+0000 7f1b6141e740 1 -- 172.21.15.67:0/3266273963 shutdown_connections 2024-07-31T15:03:46.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.905+0000 7f1b6141e740 1 -- 172.21.15.67:0/3266273963 wait complete. 2024-07-31T15:03:46.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.905+0000 7f1b53fff640 1 --2- 172.21.15.67:0/4596797 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1b3c0b3e70 0x7f1b3c0b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.905+0000 7f1b6141e740 1 -- start start 2024-07-31T15:03:46.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.905+0000 7f1b6141e740 1 Processor -- start 2024-07-31T15:03:46.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.906+0000 7f1b53fff640 1 --2- 172.21.15.67:0/4596797 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1b3c0b3e70 0x7f1b3c0b6380 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1b5411a910 tx=0x7f1b48006cf0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:46.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.906+0000 7f1b6141e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e13c490 0x55f15e516190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.906+0000 7f1b6141e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e144670 0x55f15e5166d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.906+0000 7f1b6141e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e521f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.906+0000 7f1b6141e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f15e4124f0 con 0x55f15e165350 2024-07-31T15:03:46.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.906+0000 7f1b6141e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f15e473fc0 con 0x55f15e144670 2024-07-31T15:03:46.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.906+0000 7f1b6141e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f15e2d3890 con 0x55f15e13c490 2024-07-31T15:03:46.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.906+0000 7f1b6141e740 10 client.0 _ll_register_callbacks cb 0x55f15e41c500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:46.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.907+0000 7f1b5946a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e521f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.907+0000 7f1b5946a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e521f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36832/0 (socket says 172.21.15.67:36832) 2024-07-31T15:03:46.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.907+0000 7f1b5946a640 1 -- 172.21.15.67:0/2410763365 learned_addr learned my addr 172.21.15.67:0/2410763365 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:46.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.907+0000 7f1b53fff640 1 --2- 172.21.15.67:0/2410763365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e144670 0x55f15e5166d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.907+0000 7f1b58c69640 1 --2- 172.21.15.67:0/2410763365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e13c490 0x55f15e516190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.907+0000 7f1b5946a640 1 -- 172.21.15.67:0/2410763365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e13c490 msgr2=0x55f15e516190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:46.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.907+0000 7f1b5946a640 1 --2- 172.21.15.67:0/2410763365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e13c490 0x55f15e516190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.907+0000 7f1b5946a640 1 -- 172.21.15.67:0/2410763365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e144670 msgr2=0x55f15e5166d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:46.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.907+0000 7f1b5946a640 1 --2- 172.21.15.67:0/2410763365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e144670 0x55f15e5166d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.907+0000 7f1b5946a640 1 -- 172.21.15.67:0/2410763365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f15e421c50 con 0x55f15e165350 2024-07-31T15:03:46.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.908+0000 7f1b5946a640 1 --2- 172.21.15.67:0/2410763365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e521f60 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c02af70 tx=0x7f1b4c0ca9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfacfa518c1b7592 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:46.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.908+0000 7f1b33fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:46.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.909+0000 7f1b33fff640 1 -- 172.21.15.67:0/2410763365 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c0cf770 con 0x55f15e165350 2024-07-31T15:03:46.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.909+0000 7f1b33fff640 1 -- 172.21.15.67:0/2410763365 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f1b4c101e70 con 0x55f15e165350 2024-07-31T15:03:46.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.909+0000 7f1b6141e740 1 -- 172.21.15.67:0/2410763365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b440035f0 con 0x55f15e165350 2024-07-31T15:03:46.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.910+0000 7f1b33fff640 1 -- 172.21.15.67:0/2410763365 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f1b4c102070 con 0x55f15e165350 2024-07-31T15:03:46.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.910+0000 7f1b6141e740 10 client.18228 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:46.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.910+0000 7f1b6141e740 1 -- 172.21.15.67:0/2410763365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f15e524c80 con 0x55f15e165350 2024-07-31T15:03:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.911+0000 7f6aa7fff640 1 --2- 172.21.15.67:0/634956637 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5579057661c0 0x557905bfcea0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1873cfc354cb73bf server_cookie=3775d150522b6ef5 in_seq=0 out_seq=0 2024-07-31T15:03:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:46.912+0000 7f6a877fe640 10 client.27779 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:46.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.911+0000 7f1b33fff640 1 -- 172.21.15.67:0/2410763365 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b440035f0 con 0x55f15e165350 2024-07-31T15:03:46.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.912+0000 7f1b33fff640 1 -- 172.21.15.67:0/2410763365 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f1b4c041030 con 0x55f15e165350 2024-07-31T15:03:46.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.912+0000 7f1b6141e740 10 client.18228 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:46.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.912+0000 7f1b6141e740 10 client.18228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:46.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.912+0000 7f1b6141e740 20 client.18228 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:46.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.912+0000 7f1b6141e740 1 --2- 172.21.15.67:0/2410763365 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f15e123cd0 0x55f15e528200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.913+0000 7f1b6141e740 4 client.18228 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:03:46.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.913+0000 7f1b6141e740 1 -- 172.21.15.67:0/2410763365 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f15e4829d0 con 0x55f15e123cd0 2024-07-31T15:03:46.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.914+0000 7f1b58c69640 1 --2- 172.21.15.67:0/2410763365 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f15e123cd0 0x55f15e528200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.917+0000 7f1b33fff640 10 client.18228 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:46.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.917+0000 7f1b58c69640 1 --2- 172.21.15.67:0/2410763365 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f15e123cd0 0x55f15e528200 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a74b948d88e01232 server_cookie=492016b98ee8cfba in_seq=0 out_seq=0 2024-07-31T15:03:46.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.926+0000 7f1b33fff640 1 -- 172.21.15.67:0/2410763365 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f15e4829d0 con 0x55f15e123cd0 2024-07-31T15:03:46.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.926+0000 7f1b33fff640 10 client.18228 handle_command_reply: tid=0 2024-07-31T15:03:46.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.926+0000 7f1b6141e740 1 client.18228 shutdown 2024-07-31T15:03:46.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.926+0000 7f1b6141e740 1 -- 172.21.15.67:0/2410763365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 msgr2=0x55f15e521f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:46.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.926+0000 7f1b6141e740 1 --2- 172.21.15.67:0/2410763365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e521f60 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c02af70 tx=0x7f1b4c0ca9d0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.927+0000 7f1b6141e740 1 -- 172.21.15.67:0/2410763365 shutdown_connections 2024-07-31T15:03:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.927+0000 7f1b6141e740 1 --2- 172.21.15.67:0/2410763365 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f15e123cd0 0x55f15e528200 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.927+0000 7f1b6141e740 1 --2- 172.21.15.67:0/2410763365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e521f60 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.927+0000 7f1b6141e740 1 --2- 172.21.15.67:0/2410763365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e144670 0x55f15e5166d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.927+0000 7f1b6141e740 1 --2- 172.21.15.67:0/2410763365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e13c490 0x55f15e516190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.927+0000 7f1b6141e740 1 -- 172.21.15.67:0/2410763365 >> 172.21.15.67:0/2410763365 conn(0x55f15e159740 msgr2=0x55f15e506b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.927+0000 7f1b33fff640 0 client.18228 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.927+0000 7f1b6141e740 1 -- 172.21.15.67:0/2410763365 shutdown_connections 2024-07-31T15:03:46.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.927+0000 7f1b6141e740 1 -- 172.21.15.67:0/2410763365 wait complete. 2024-07-31T15:03:46.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.927+0000 7f1b6141e740 20 client.18228 trim_cache size 0 max 16384 2024-07-31T15:03:46.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.963+0000 7f1b6141e740 1 Processor -- start 2024-07-31T15:03:46.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.972+0000 7f1b6141e740 1 -- start start 2024-07-31T15:03:46.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.972+0000 7f1b6141e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e521d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.972+0000 7f1b6141e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 0x55f15e508300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.972+0000 7f1b6141e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e52da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.973+0000 7f1b6141e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4000fe60 con 0x55f15e165350 2024-07-31T15:03:46.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.973+0000 7f1b6141e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1b400119a0 con 0x55f15e13c490 2024-07-31T15:03:46.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.973+0000 7f1b6141e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40010530 con 0x55f15e144670 2024-07-31T15:03:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.974+0000 7f1b58c69640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e521d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.976+0000 7f1b5946a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e52da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.977+0000 7f1b58c69640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e521d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46182/0 (socket says 172.21.15.67:46182) 2024-07-31T15:03:46.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.977+0000 7f1b58c69640 1 -- 172.21.15.67:0/4209026056 learned_addr learned my addr 172.21.15.67:0/4209026056 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:46.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.977+0000 7f1b53fff640 1 --2- 172.21.15.67:0/4209026056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 0x55f15e508300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.977+0000 7f1b5946a640 1 -- 172.21.15.67:0/4209026056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 msgr2=0x55f15e508300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:46.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.977+0000 7f1b5946a640 1 --2- 172.21.15.67:0/4209026056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 0x55f15e508300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.977+0000 7f1b5946a640 1 -- 172.21.15.67:0/4209026056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 msgr2=0x55f15e521d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:46.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.977+0000 7f1b5946a640 1 --2- 172.21.15.67:0/4209026056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e521d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.977+0000 7f1b5946a640 1 -- 172.21.15.67:0/4209026056 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f15e517250 con 0x55f15e165350 2024-07-31T15:03:46.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.978+0000 7f1b5946a640 1 --2- 172.21.15.67:0/4209026056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e52da00 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0c71f0 tx=0x7f1b4c0ca8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc9c02ef9ac5a0a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:46.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.978+0000 7f1b3a7fc640 1 -- 172.21.15.67:0/4209026056 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c0d8030 con 0x55f15e165350 2024-07-31T15:03:46.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.978+0000 7f1b3a7fc640 1 -- 172.21.15.67:0/4209026056 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b4c0ec370 con 0x55f15e165350 2024-07-31T15:03:46.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.980+0000 7f1b6141e740 1 -- 172.21.15.67:0/4209026056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 msgr2=0x55f15e52da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:46.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.980+0000 7f1b6141e740 1 --2- 172.21.15.67:0/4209026056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e52da00 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0c71f0 tx=0x7f1b4c0ca8e0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.983+0000 7f1b6141e740 1 -- 172.21.15.67:0/4209026056 shutdown_connections 2024-07-31T15:03:46.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.983+0000 7f1b6141e740 1 --2- 172.21.15.67:0/4209026056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e52da00 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.983+0000 7f1b6141e740 1 --2- 172.21.15.67:0/4209026056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 0x55f15e508300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.983+0000 7f1b6141e740 1 --2- 172.21.15.67:0/4209026056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e521d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.983+0000 7f1b6141e740 1 -- 172.21.15.67:0/4209026056 >> 172.21.15.67:0/4209026056 conn(0x55f15e159740 msgr2=0x55f15e507bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.987+0000 7f1b6141e740 1 -- 172.21.15.67:0/4209026056 shutdown_connections 2024-07-31T15:03:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.988+0000 7f1b6141e740 1 -- 172.21.15.67:0/4209026056 wait complete. 2024-07-31T15:03:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.988+0000 7f1b6141e740 1 -- start start 2024-07-31T15:03:46.992 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/ebda.o 2024-07-31T15:03:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.989+0000 7f1b6141e740 1 Processor -- start 2024-07-31T15:03:46.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.992+0000 7f1b6141e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e51e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.992+0000 7f1b6141e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 0x55f15e5198c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.992+0000 7f1b6141e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e519e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.992+0000 7f1b6141e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1b400112d0 con 0x55f15e165350 2024-07-31T15:03:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.992+0000 7f1b6141e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40010c00 con 0x55f15e13c490 2024-07-31T15:03:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.992+0000 7f1b6141e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4000de60 con 0x55f15e144670 2024-07-31T15:03:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.992+0000 7f1b6141e740 10 client.0 _ll_register_callbacks cb 0x7f1b4c0cab90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.992+0000 7f1b58c69640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e51e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.993+0000 7f1b58c69640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e51e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46198/0 (socket says 172.21.15.67:46198) 2024-07-31T15:03:46.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.993+0000 7f1b58c69640 1 -- 172.21.15.67:0/1845786735 learned_addr learned my addr 172.21.15.67:0/1845786735 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:46.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.993+0000 7f1b53fff640 1 --2- 172.21.15.67:0/1845786735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 0x55f15e5198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.995+0000 7f1b53fff640 1 -- 172.21.15.67:0/1845786735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 msgr2=0x55f15e51e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.995+0000 7f1b53fff640 1 --2- 172.21.15.67:0/1845786735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e51e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.995+0000 7f1b53fff640 1 -- 172.21.15.67:0/1845786735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 msgr2=0x55f15e519e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.995+0000 7f1b53fff640 1 --2- 172.21.15.67:0/1845786735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e519e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.995+0000 7f1b53fff640 1 -- 172.21.15.67:0/1845786735 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f15e51a440 con 0x55f15e144670 2024-07-31T15:03:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.995+0000 7f1b53fff640 1 --2- 172.21.15.67:0/1845786735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 0x55f15e5198c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009fd0 tx=0x7f1b480047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85fbbad98b63a459 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.995+0000 7f1b33fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:03:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.996+0000 7f1b33fff640 1 -- 172.21.15.67:0/1845786735 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4801f420 con 0x55f15e144670 2024-07-31T15:03:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.996+0000 7f1b6141e740 1 -- 172.21.15.67:0/1845786735 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f15e47e470 con 0x55f15e144670 2024-07-31T15:03:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.996+0000 7f1b33fff640 1 -- 172.21.15.67:0/1845786735 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f1b4801f5c0 con 0x55f15e144670 2024-07-31T15:03:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:46.996+0000 7f1b33fff640 1 -- 172.21.15.67:0/1845786735 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f1b4802d050 con 0x55f15e144670 2024-07-31T15:03:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.004+0000 7f1b6141e740 10 client.27739 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.004+0000 7f1b6141e740 1 -- 172.21.15.67:0/1845786735 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f15e590820 con 0x55f15e144670 2024-07-31T15:03:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.013+0000 7f1b33fff640 1 -- 172.21.15.67:0/1845786735 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f15e47e470 con 0x55f15e144670 2024-07-31T15:03:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.013+0000 7f1b33fff640 1 -- 172.21.15.67:0/1845786735 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f1b48028920 con 0x55f15e144670 2024-07-31T15:03:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.013+0000 7f1b6141e740 10 client.27739 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.013+0000 7f1b6141e740 10 client.27739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.013+0000 7f1b6141e740 20 client.27739 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.013+0000 7f1b6141e740 1 --2- 172.21.15.67:0/1845786735 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f15e0f9080 0x55f15e593040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.013+0000 7f1b6141e740 4 client.27739 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T15:03:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.013+0000 7f1b6141e740 1 -- 172.21.15.67:0/1845786735 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f15e47bbb0 con 0x55f15e0f9080 2024-07-31T15:03:47.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.018+0000 7f1b58c69640 1 --2- 172.21.15.67:0/1845786735 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f15e0f9080 0x55f15e593040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:47.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.021+0000 7f1b33fff640 10 client.27739 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:47.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.021+0000 7f1b58c69640 1 --2- 172.21.15.67:0/1845786735 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f15e0f9080 0x55f15e593040 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d34011ab59934741 server_cookie=92c2be9793d50060 in_seq=0 out_seq=0 2024-07-31T15:03:47.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.022+0000 7f1b33fff640 1 -- 172.21.15.67:0/1845786735 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f15e47bbb0 con 0x55f15e0f9080 2024-07-31T15:03:47.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.022+0000 7f1b33fff640 10 client.27739 handle_command_reply: tid=0 2024-07-31T15:03:47.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.023+0000 7f1b6141e740 1 client.27739 shutdown 2024-07-31T15:03:47.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.029+0000 7f1b6141e740 1 -- 172.21.15.67:0/1845786735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 msgr2=0x55f15e5198c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:47.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.029+0000 7f1b6141e740 1 --2- 172.21.15.67:0/1845786735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 0x55f15e5198c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009fd0 tx=0x7f1b480047d0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.030+0000 7f1b6141e740 1 -- 172.21.15.67:0/1845786735 shutdown_connections 2024-07-31T15:03:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.030+0000 7f1b6141e740 1 --2- 172.21.15.67:0/1845786735 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f15e0f9080 0x55f15e593040 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.030+0000 7f1b6141e740 1 --2- 172.21.15.67:0/1845786735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f15e165350 0x55f15e519e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.030+0000 7f1b6141e740 1 --2- 172.21.15.67:0/1845786735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f15e144670 0x55f15e5198c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.030+0000 7f1b6141e740 1 --2- 172.21.15.67:0/1845786735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f15e13c490 0x55f15e51e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.030+0000 7f1b33fff640 0 client.27739 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.030+0000 7f1b6141e740 1 -- 172.21.15.67:0/1845786735 >> 172.21.15.67:0/1845786735 conn(0x55f15e159740 msgr2=0x55f15e50b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:47.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.033+0000 7f1b6141e740 1 -- 172.21.15.67:0/1845786735 shutdown_connections 2024-07-31T15:03:47.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.033+0000 7f1b6141e740 1 -- 172.21.15.67:0/1845786735 wait complete. 2024-07-31T15:03:47.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.039+0000 7f1b6141e740 20 client.27739 trim_cache size 0 max 16384 2024-07-31T15:03:47.042 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:03:47.042 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T15:03:47.043 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "edd2c496-b8f0-45e2-9f9a-ec736f011bee", 2024-07-31T15:03:47.043 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T15:03:47.043 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:03:47.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.041+0000 7f1b59c6b640 1 -- 172.21.15.67:0/4596797 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1b3c0b3e70 msgr2=0x7f1b3c0b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:47.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.041+0000 7f1b59c6b640 1 --2- 172.21.15.67:0/4596797 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1b3c0b3e70 0x7f1b3c0b6380 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1b5411a910 tx=0x7f1b48006cf0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.041+0000 7f1b59c6b640 1 -- 172.21.15.67:0/4596797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b5411c5d0 msgr2=0x7f1b541ae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.041+0000 7f1b59c6b640 1 --2- 172.21.15.67:0/4596797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b5411c5d0 0x7f1b541ae300 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009bf0 tx=0x7f1b4c003f10 comp rx=0 tx=0).stop 2024-07-31T15:03:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.041+0000 7f1b59c6b640 1 -- 172.21.15.67:0/4596797 shutdown_connections 2024-07-31T15:03:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.041+0000 7f1b59c6b640 1 --2- 172.21.15.67:0/4596797 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1b3c0b3e70 0x7f1b3c0b6380 secure :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1b5411a910 tx=0x7f1b48006cf0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.041+0000 7f1b59c6b640 1 --2- 172.21.15.67:0/4596797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b5411c5d0 0x7f1b541ae300 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.041+0000 7f1b59c6b640 1 --2- 172.21.15.67:0/4596797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b54111790 0x7f1b541a9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.041+0000 7f1b59c6b640 1 --2- 172.21.15.67:0/4596797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b5410a330 0x7f1b541a8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.042+0000 7f1b59c6b640 1 -- 172.21.15.67:0/4596797 >> 172.21.15.67:0/4596797 conn(0x7f1b5406e420 msgr2=0x7f1b54071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.042+0000 7f1b59c6b640 1 -- 172.21.15.67:0/4596797 shutdown_connections 2024-07-31T15:03:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:47.042+0000 7f1b59c6b640 1 -- 172.21.15.67:0/4596797 wait complete. 2024-07-31T15:03:47.175 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/netlabel.o 2024-07-31T15:03:47.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:47 smithi067 ceph-mon[32750]: pgmap v801: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 435 KiB/s wr, 86 op/s 2024-07-31T15:03:47.341 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mmu_gather.o 2024-07-31T15:03:47.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:47 smithi073 ceph-mon[36714]: pgmap v801: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 435 KiB/s wr, 86 op/s 2024-07-31T15:03:47.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:47 smithi077 ceph-mon[37327]: pgmap v801: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 435 KiB/s wr, 86 op/s 2024-07-31T15:03:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.832+0000 7f6a877fe640 1 -- 172.21.15.67:0/634956637 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557905ae7da0 con 0x5579057661c0 2024-07-31T15:03:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.832+0000 7f6a877fe640 10 client.27779 handle_command_reply: tid=0 2024-07-31T15:03:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.832+0000 7f6ab53b9740 1 client.27779 shutdown 2024-07-31T15:03:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.836+0000 7f6ab53b9740 1 -- 172.21.15.67:0/634956637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 msgr2=0x557905b85f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.836+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/634956637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b85f20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c000c00 tx=0x7f6a9c00ee40 comp rx=0 tx=0).stop 2024-07-31T15:03:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.836+0000 7f6ab53b9740 1 -- 172.21.15.67:0/634956637 shutdown_connections 2024-07-31T15:03:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.836+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/634956637 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5579057661c0 0x557905bfcea0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.836+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/634956637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5579057d21b0 0x557905b85f20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.836+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/634956637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579057b14d0 0x557905b85920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.836+0000 7f6a877fe640 0 client.27779 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.836+0000 7f6ab53b9740 1 --2- 172.21.15.67:0/634956637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5579057a92f0 0x557905b890c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.836+0000 7f6ab53b9740 1 -- 172.21.15.67:0/634956637 >> 172.21.15.67:0/634956637 conn(0x5579057c6300 msgr2=0x557905b77fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:03:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:03:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:03:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:03:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.841+0000 7f6ab53b9740 1 -- 172.21.15.67:0/634956637 shutdown_connections 2024-07-31T15:03:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.841+0000 7f6ab53b9740 1 -- 172.21.15.67:0/634956637 wait complete. 2024-07-31T15:03:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.841+0000 7f6ab53b9740 20 client.27779 trim_cache size 0 max 16384 2024-07-31T15:03:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.855+0000 7f6aad186640 1 -- 172.21.15.67:0/3615233119 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6a900b3e80 msgr2=0x7f6a900b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.855+0000 7f6aad186640 1 --2- 172.21.15.67:0/3615233119 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6a900b3e80 0x7f6a900b6390 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00a8d0 tx=0x7f6a9c005be0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.855+0000 7f6aad186640 1 -- 172.21.15.67:0/3615233119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa8111650 msgr2=0x7f6aa81a9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.855+0000 7f6aad186640 1 --2- 172.21.15.67:0/3615233119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa8111650 0x7f6aa81a9040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0000f80 tx=0x7f6aa0003ef0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.855+0000 7f6aad186640 1 -- 172.21.15.67:0/3615233119 shutdown_connections 2024-07-31T15:03:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.855+0000 7f6aad186640 1 --2- 172.21.15.67:0/3615233119 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6a900b3e80 0x7f6a900b6390 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.855+0000 7f6aad186640 1 --2- 172.21.15.67:0/3615233119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6aa811c5d0 0x7f6aa81ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.855+0000 7f6aad186640 1 --2- 172.21.15.67:0/3615233119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa8111650 0x7f6aa81a9040 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.855+0000 7f6aad186640 1 --2- 172.21.15.67:0/3615233119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa810a330 0x7f6aa81a8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.855+0000 7f6aad186640 1 -- 172.21.15.67:0/3615233119 >> 172.21.15.67:0/3615233119 conn(0x7f6aa806e320 msgr2=0x7f6aa811a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.857+0000 7f6aad186640 1 -- 172.21.15.67:0/3615233119 shutdown_connections 2024-07-31T15:03:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:03:47.857+0000 7f6aad186640 1 -- 172.21.15.67:0/3615233119 wait complete. 2024-07-31T15:03:48.026 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:03:48.064 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/platform-quirks.o 2024-07-31T15:03:48.141 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/pat/set_memory.o 2024-07-31T15:03:48.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:48 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T15:03:48.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:48.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:48.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:48.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:48.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:48.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:48.320 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/jbd2/transaction.o 2024-07-31T15:03:48.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:48 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T15:03:48.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:48.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:48.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:48.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:48.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:48.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:48.597 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/jbd2/commit.o 2024-07-31T15:03:48.633 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mprotect.o 2024-07-31T15:03:48.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:48 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T15:03:48.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:48.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:48.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:48.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:48.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:48.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:48.657 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/extents_status.o 2024-07-31T15:03:48.782 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:03:48.814 INFO:tasks.workunit.client.0.smithi067.stdout: AR security/selinux/built-in.a 2024-07-31T15:03:48.948 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/integrity/iint.o 2024-07-31T15:03:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: pgmap v802: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 501 KiB/s wr, 96 op/s 2024-07-31T15:03:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: pgmap v802: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 501 KiB/s wr, 96 op/s 2024-07-31T15:03:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:49.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:49.643 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:03:49.352+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000cde2 [...10,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders/ auth{4=1} v112 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0) n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1) old_inodes=2 (inest sync->mix g=4) caps={15489=pAsLsXsFs/pAsLsXsFs@8} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8dc52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v29>,<0x1000000cbec/selinux v646>,<0x10000000202/scripts v8820>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v112>,<0x1000000cbec/selinux v749>,<0x10000000202/scripts v11639>,<0x10000000201/linux-6.5.11 v1654>,<0x10000000006/t v908>,<0x10000000005/tmp v863>,<0x100000001fb/client.0 v843>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v826>,<0x10000000001/sv_1 v828>,<0x10000000000/qa v824>,<0x1/volumes v790>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:58.999116+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:03:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: pgmap v802: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 501 KiB/s wr, 96 op/s 2024-07-31T15:03:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:49.786 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/integrity/integrity_audit.o 2024-07-31T15:03:49.916 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/rcu/rcu_segcblist.o 2024-07-31T15:03:49.993 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/jbd2/recovery.o 2024-07-31T15:03:50.029 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/jbd2/checkpoint.o 2024-07-31T15:03:50.127 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/pat/memtype.o 2024-07-31T15:03:50.191 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mremap.o 2024-07-31T15:03:50.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:50 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:50.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:50 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:50.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:50 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:50.547 INFO:tasks.workunit.client.0.smithi067.stdout: AR security/integrity/built-in.a 2024-07-31T15:03:50.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:50 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:50.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:50 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:50.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:50 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:50.602 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/commoncap.o 2024-07-31T15:03:50.624 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/rcu/built-in.a 2024-07-31T15:03:50.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:50 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:50.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:50 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:50.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:50 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:50.688 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/livepatch/built-in.a 2024-07-31T15:03:50.746 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/dma/mapping.o 2024-07-31T15:03:50.796 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/file.o 2024-07-31T15:03:50.985 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/jbd2/revoke.o 2024-07-31T15:03:51.080 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/msync.o 2024-07-31T15:03:51.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.158+0000 7f41745b7640 1 -- 172.21.15.67:0/2199209279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f416c111970 msgr2=0x7f416c109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:51.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.158+0000 7f41745b7640 1 --2- 172.21.15.67:0/2199209279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f416c111970 0x7f416c109df0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f416401a910 tx=0x7f416403b470 comp rx=0 tx=0).stop 2024-07-31T15:03:51.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.160+0000 7f41745b7640 1 -- 172.21.15.67:0/2199209279 shutdown_connections 2024-07-31T15:03:51.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.160+0000 7f41745b7640 1 --2- 172.21.15.67:0/2199209279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f416c11c660 0x7f416c11eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.160+0000 7f41745b7640 1 --2- 172.21.15.67:0/2199209279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f416c10a410 0x7f416c10a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.160+0000 7f41745b7640 1 --2- 172.21.15.67:0/2199209279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f416c111970 0x7f416c109df0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.160+0000 7f41745b7640 1 -- 172.21.15.67:0/2199209279 >> 172.21.15.67:0/2199209279 conn(0x7f416c0708f0 msgr2=0x7f416c070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:51.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.160+0000 7f41745b7640 1 -- 172.21.15.67:0/2199209279 shutdown_connections 2024-07-31T15:03:51.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.161+0000 7f41745b7640 1 -- 172.21.15.67:0/2199209279 wait complete. 2024-07-31T15:03:51.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.162+0000 7f41745b7640 1 Processor -- start 2024-07-31T15:03:51.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.162+0000 7f41745b7640 1 -- start start 2024-07-31T15:03:51.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.162+0000 7f41745b7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f416c10a410 0x7f416c11bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.162+0000 7f41745b7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f416c111970 0x7f416c118290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.162+0000 7f41745b7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f416c11c660 0x7f416c118960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.162+0000 7f41745b7640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f416c114740 con 0x7f416c111970 2024-07-31T15:03:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.162+0000 7f41745b7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f416c1145c0 con 0x7f416c11c660 2024-07-31T15:03:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.162+0000 7f41745b7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f416c1148c0 con 0x7f416c10a410 2024-07-31T15:03:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.163+0000 7f4172353640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f416c10a410 0x7f416c11bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.163+0000 7f4172353640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f416c10a410 0x7f416c11bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:41974/0 (socket says 172.21.15.67:41974) 2024-07-31T15:03:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.163+0000 7f4172353640 1 -- 172.21.15.67:0/1862802648 learned_addr learned my addr 172.21.15.67:0/1862802648 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.166+0000 7f4172b54640 1 --2- 172.21.15.67:0/1862802648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f416c11c660 0x7f416c118960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.167+0000 7f4172353640 1 -- 172.21.15.67:0/1862802648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f416c11c660 msgr2=0x7f416c118960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.167+0000 7f4172353640 1 --2- 172.21.15.67:0/1862802648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f416c11c660 0x7f416c118960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.167+0000 7f4172353640 1 -- 172.21.15.67:0/1862802648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f416c111970 msgr2=0x7f416c118290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.167+0000 7f4172353640 1 --2- 172.21.15.67:0/1862802648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f416c111970 0x7f416c118290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.167+0000 7f4172353640 1 -- 172.21.15.67:0/1862802648 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c1ac800 con 0x7f416c10a410 2024-07-31T15:03:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.167+0000 7f4172353640 1 --2- 172.21.15.67:0/1862802648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f416c10a410 0x7f416c11bfb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f416403b9e0 tx=0x7f4164004ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:51.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.171+0000 7f416b7fe640 1 -- 172.21.15.67:0/1862802648 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416404f020 con 0x7f416c10a410 2024-07-31T15:03:51.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.171+0000 7f416b7fe640 1 -- 172.21.15.67:0/1862802648 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4164004260 con 0x7f416c10a410 2024-07-31T15:03:51.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.171+0000 7f41745b7640 1 -- 172.21.15.67:0/1862802648 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c1ac9d0 con 0x7f416c10a410 2024-07-31T15:03:51.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.171+0000 7f41745b7640 1 -- 172.21.15.67:0/1862802648 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c1acf00 con 0x7f416c10a410 2024-07-31T15:03:51.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.171+0000 7f416b7fe640 1 -- 172.21.15.67:0/1862802648 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164046070 con 0x7f416c10a410 2024-07-31T15:03:51.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.184+0000 7f416b7fe640 1 -- 172.21.15.67:0/1862802648 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4164052020 con 0x7f416c10a410 2024-07-31T15:03:51.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.184+0000 7f417ab08740 1 Processor -- start 2024-07-31T15:03:51.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.187+0000 7f416b7fe640 1 --2- 172.21.15.67:0/1862802648 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f415c0b3f50 0x7f415c0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.188+0000 7f416b7fe640 1 -- 172.21.15.67:0/1862802648 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f4164115d90 con 0x7f416c10a410 2024-07-31T15:03:51.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.190+0000 7f417ab08740 1 -- start start 2024-07-31T15:03:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.190+0000 7f4172b54640 1 --2- 172.21.15.67:0/1862802648 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f415c0b3f50 0x7f415c0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.197+0000 7f417ab08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c0960b070 0x562c099b0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.197+0000 7f417ab08740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095e21b0 0x562c099b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.197+0000 7f417ab08740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.197+0000 7f417ab08740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562c09926b60 con 0x562c095ea390 2024-07-31T15:03:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.197+0000 7f417ab08740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562c09916060 con 0x562c095e21b0 2024-07-31T15:03:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.197+0000 7f417ab08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562c0991a0a0 con 0x562c0960b070 2024-07-31T15:03:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.198+0000 7f4172b54640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.198+0000 7f4172b54640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099b10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49196/0 (socket says 172.21.15.67:49196) 2024-07-31T15:03:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.198+0000 7f4172b54640 1 -- 172.21.15.67:0/4115499062 learned_addr learned my addr 172.21.15.67:0/4115499062 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.199+0000 7f4172b54640 1 -- 172.21.15.67:0/4115499062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c0960b070 msgr2=0x562c099b0640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:03:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.199+0000 7f4172b54640 1 --2- 172.21.15.67:0/4115499062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c0960b070 0x562c099b0640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.199+0000 7f4172b54640 1 -- 172.21.15.67:0/4115499062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095e21b0 msgr2=0x562c099b0b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:03:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.199+0000 7f4172b54640 1 --2- 172.21.15.67:0/4115499062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095e21b0 0x562c099b0b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.199+0000 7f4172b54640 1 -- 172.21.15.67:0/4115499062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c098bcb40 con 0x562c095ea390 2024-07-31T15:03:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.199+0000 7f4172b54640 1 --2- 172.21.15.67:0/1862802648 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f415c0b3f50 0x7f415c0b6460 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f416000c870 tx=0x7f416000cc70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.200+0000 7f4172b54640 1 --2- 172.21.15.67:0/4115499062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099b10c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4160007680 tx=0x7f416001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d16dc5e3b98a7c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.203+0000 7f4168ff9640 1 -- 172.21.15.67:0/4115499062 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416001cd00 con 0x562c095ea390 2024-07-31T15:03:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.203+0000 7f4168ff9640 1 -- 172.21.15.67:0/4115499062 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4160020070 con 0x562c095ea390 2024-07-31T15:03:51.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.204+0000 7f417ab08740 1 -- 172.21.15.67:0/4115499062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 msgr2=0x562c099b10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:51.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.204+0000 7f417ab08740 1 --2- 172.21.15.67:0/4115499062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099b10c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4160007680 tx=0x7f416001d000 comp rx=0 tx=0).stop 2024-07-31T15:03:51.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.204+0000 7f417ab08740 1 -- 172.21.15.67:0/4115499062 shutdown_connections 2024-07-31T15:03:51.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.204+0000 7f417ab08740 1 --2- 172.21.15.67:0/4115499062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099b10c0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.204+0000 7f417ab08740 1 --2- 172.21.15.67:0/4115499062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095e21b0 0x562c099b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.204+0000 7f417ab08740 1 --2- 172.21.15.67:0/4115499062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c0960b070 0x562c099b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.204+0000 7f417ab08740 1 -- 172.21.15.67:0/4115499062 >> 172.21.15.67:0/4115499062 conn(0x562c095ff460 msgr2=0x562c099ae530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:51.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.212+0000 7f417ab08740 1 -- 172.21.15.67:0/4115499062 shutdown_connections 2024-07-31T15:03:51.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.212+0000 7f417ab08740 1 -- 172.21.15.67:0/4115499062 wait complete. 2024-07-31T15:03:51.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.212+0000 7f417ab08740 1 -- start start 2024-07-31T15:03:51.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.212+0000 7f417ab08740 1 Processor -- start 2024-07-31T15:03:51.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.213+0000 7f417ab08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c095e21b0 0x562c099bc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.213+0000 7f417ab08740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099bc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.213+0000 7f417ab08740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c0960b070 0x562c099c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.213+0000 7f417ab08740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562c09779700 con 0x562c095ea390 2024-07-31T15:03:51.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.213+0000 7f417ab08740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562c098b8990 con 0x562c0960b070 2024-07-31T15:03:51.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.213+0000 7f417ab08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562c099bee10 con 0x562c095e21b0 2024-07-31T15:03:51.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.213+0000 7f417ab08740 10 client.0 _ll_register_callbacks cb 0x562c098d34a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:51.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.215+0000 7f4172353640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c095e21b0 0x562c099bc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:51.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.216+0000 7f4172353640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c095e21b0 0x562c099bc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:41976/0 (socket says 172.21.15.67:41976) 2024-07-31T15:03:51.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.216+0000 7f4172353640 1 -- 172.21.15.67:0/1397066135 learned_addr learned my addr 172.21.15.67:0/1397066135 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:51.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.217+0000 7f4172353640 1 -- 172.21.15.67:0/1397066135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c0960b070 msgr2=0x562c099c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:51.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.217+0000 7f4172353640 1 --2- 172.21.15.67:0/1397066135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c0960b070 0x562c099c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.217+0000 7f4172353640 1 -- 172.21.15.67:0/1397066135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 msgr2=0x562c099bc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:51.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.217+0000 7f4172353640 1 --2- 172.21.15.67:0/1397066135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099bc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.217+0000 7f4172353640 1 -- 172.21.15.67:0/1397066135 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c09676650 con 0x562c095e21b0 2024-07-31T15:03:51.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.217+0000 7f4172353640 1 --2- 172.21.15.67:0/1397066135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c095e21b0 0x562c099bc2e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f41640d8680 tx=0x7f41640dbd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f4c79de8e796dc1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:51.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.217+0000 7f4155ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:03:51.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.219+0000 7f4155ffb640 1 -- 172.21.15.67:0/1397066135 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640f6060 con 0x562c095e21b0 2024-07-31T15:03:51.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.219+0000 7f4155ffb640 1 -- 172.21.15.67:0/1397066135 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f41641133c0 con 0x562c095e21b0 2024-07-31T15:03:51.223 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:03:51.223 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.223 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: pgmap v803: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 823 KiB/s wr, 295 op/s 2024-07-31T15:03:51.223 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:51.223 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.223 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:51.223 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:51.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:51.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:51.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:51.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:51.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:51.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:51.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:51.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:51.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:51.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:51.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.222+0000 7f417ab08740 1 -- 172.21.15.67:0/1397066135 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562c09928ab0 con 0x562c095e21b0 2024-07-31T15:03:51.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.226+0000 7f4155ffb640 1 -- 172.21.15.67:0/1397066135 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f41641135c0 con 0x562c095e21b0 2024-07-31T15:03:51.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.226+0000 7f417ab08740 10 client.27745 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:51.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.226+0000 7f417ab08740 1 -- 172.21.15.67:0/1397066135 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x562c099ed080 con 0x562c095e21b0 2024-07-31T15:03:51.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.235+0000 7f4155ffb640 1 -- 172.21.15.67:0/1397066135 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41640dcd00 con 0x562c095e21b0 2024-07-31T15:03:51.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.235+0000 7f4155ffb640 1 -- 172.21.15.67:0/1397066135 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f416405cbd0 con 0x562c095e21b0 2024-07-31T15:03:51.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.235+0000 7f417ab08740 10 client.27745 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:51.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.235+0000 7f417ab08740 10 client.27745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:51.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.235+0000 7f417ab08740 20 client.27745 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:51.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.235+0000 7f417ab08740 1 --2- 172.21.15.67:0/1397066135 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562c095c99f0 0x562c099f04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.235+0000 7f417ab08740 4 client.27745 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:03:51.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.235+0000 7f417ab08740 1 -- 172.21.15.67:0/1397066135 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c09928ab0 con 0x562c095c99f0 2024-07-31T15:03:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.242+0000 7f4171b52640 1 --2- 172.21.15.67:0/1397066135 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562c095c99f0 0x562c099f04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:51.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.250+0000 7f4171b52640 1 --2- 172.21.15.67:0/1397066135 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562c095c99f0 0x562c099f04d0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c059909f72e93c server_cookie=6aba26841b8728e in_seq=0 out_seq=0 2024-07-31T15:03:51.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.250+0000 7f4155ffb640 10 client.27745 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:51.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.251+0000 7f4155ffb640 1 -- 172.21.15.67:0/1397066135 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c09928ab0 con 0x562c095c99f0 2024-07-31T15:03:51.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.251+0000 7f4155ffb640 10 client.27745 handle_command_reply: tid=0 2024-07-31T15:03:51.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.251+0000 7f417ab08740 1 client.27745 shutdown 2024-07-31T15:03:51.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.252+0000 7f417ab08740 1 -- 172.21.15.67:0/1397066135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c095e21b0 msgr2=0x562c099bc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:51.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.252+0000 7f417ab08740 1 --2- 172.21.15.67:0/1397066135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c095e21b0 0x562c099bc2e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f41640d8680 tx=0x7f41640dbd70 comp rx=0 tx=0).stop 2024-07-31T15:03:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.253+0000 7f417ab08740 1 -- 172.21.15.67:0/1397066135 shutdown_connections 2024-07-31T15:03:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.253+0000 7f417ab08740 1 --2- 172.21.15.67:0/1397066135 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562c095c99f0 0x562c099f04d0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.253+0000 7f417ab08740 1 --2- 172.21.15.67:0/1397066135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c0960b070 0x562c099c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.253+0000 7f417ab08740 1 --2- 172.21.15.67:0/1397066135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099bc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.253+0000 7f417ab08740 1 --2- 172.21.15.67:0/1397066135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c095e21b0 0x562c099bc2e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.253+0000 7f417ab08740 1 -- 172.21.15.67:0/1397066135 >> 172.21.15.67:0/1397066135 conn(0x562c095ff460 msgr2=0x562c099accb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:51.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.253+0000 7f4155ffb640 0 client.27745 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.256+0000 7f417ab08740 1 -- 172.21.15.67:0/1397066135 shutdown_connections 2024-07-31T15:03:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.257+0000 7f417ab08740 1 -- 172.21.15.67:0/1397066135 wait complete. 2024-07-31T15:03:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.257+0000 7f417ab08740 20 client.27745 trim_cache size 0 max 16384 2024-07-31T15:03:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.299+0000 7f417ab08740 1 Processor -- start 2024-07-31T15:03:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.299+0000 7f417ab08740 1 -- start start 2024-07-31T15:03:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.300+0000 7f417ab08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c095e21b0 0x562c099c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.300+0000 7f417ab08740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095ea390 0x562c099ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.300+0000 7f417ab08740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c0960b070 0x562c099b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.300+0000 7f4171b52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095ea390 0x562c099ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.300+0000 7f4171b52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095ea390 0x562c099ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:53216/0 (socket says 172.21.15.67:53216) 2024-07-31T15:03:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.300+0000 7f4171b52640 1 -- 172.21.15.67:0/663227714 learned_addr learned my addr 172.21.15.67:0/663227714 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.300+0000 7f417ab08740 1 -- 172.21.15.67:0/663227714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f415000fef0 con 0x562c0960b070 2024-07-31T15:03:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.300+0000 7f417ab08740 1 -- 172.21.15.67:0/663227714 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4150011a30 con 0x562c095ea390 2024-07-31T15:03:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.300+0000 7f417ab08740 1 -- 172.21.15.67:0/663227714 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41500105c0 con 0x562c095e21b0 2024-07-31T15:03:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.301+0000 7f4172353640 1 --2- 172.21.15.67:0/663227714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c095e21b0 0x562c099c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.301+0000 7f4171b52640 1 -- 172.21.15.67:0/663227714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c095e21b0 msgr2=0x562c099c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.301+0000 7f4171b52640 1 --2- 172.21.15.67:0/663227714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c095e21b0 0x562c099c2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.301+0000 7f4171b52640 1 -- 172.21.15.67:0/663227714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c0960b070 msgr2=0x562c099b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.301+0000 7f4171b52640 1 --2- 172.21.15.67:0/663227714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c0960b070 0x562c099b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.301+0000 7f4171b52640 1 -- 172.21.15.67:0/663227714 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c099b2930 con 0x562c095ea390 2024-07-31T15:03:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.301+0000 7f4171b52640 1 --2- 172.21.15.67:0/663227714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095ea390 0x562c099ae410 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f416c06c3a0 tx=0x7f4158009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e39c266032c9f35b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.301+0000 7f4157fff640 1 -- 172.21.15.67:0/663227714 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158044070 con 0x562c095ea390 2024-07-31T15:03:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.301+0000 7f4157fff640 1 -- 172.21.15.67:0/663227714 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4158012ca0 con 0x562c095ea390 2024-07-31T15:03:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.302+0000 7f4157fff640 1 -- 172.21.15.67:0/663227714 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415803b070 con 0x562c095ea390 2024-07-31T15:03:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.303+0000 7f417ab08740 1 -- 172.21.15.67:0/663227714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095ea390 msgr2=0x562c099ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.303+0000 7f417ab08740 1 --2- 172.21.15.67:0/663227714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095ea390 0x562c099ae410 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f416c06c3a0 tx=0x7f4158009cd0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.306+0000 7f417ab08740 1 -- 172.21.15.67:0/663227714 shutdown_connections 2024-07-31T15:03:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.306+0000 7f417ab08740 1 --2- 172.21.15.67:0/663227714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c0960b070 0x562c099b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.306+0000 7f417ab08740 1 --2- 172.21.15.67:0/663227714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095ea390 0x562c099ae410 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.306+0000 7f417ab08740 1 --2- 172.21.15.67:0/663227714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c095e21b0 0x562c099c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.306+0000 7f417ab08740 1 -- 172.21.15.67:0/663227714 >> 172.21.15.67:0/663227714 conn(0x562c095ff460 msgr2=0x562c099add00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.306+0000 7f417ab08740 1 -- 172.21.15.67:0/663227714 shutdown_connections 2024-07-31T15:03:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.306+0000 7f417ab08740 1 -- 172.21.15.67:0/663227714 wait complete. 2024-07-31T15:03:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.306+0000 7f417ab08740 1 -- start start 2024-07-31T15:03:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.306+0000 7f417ab08740 1 Processor -- start 2024-07-31T15:03:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.307+0000 7f417ab08740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095e21b0 0x562c099f6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.307+0000 7f417ab08740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.307+0000 7f417ab08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c0960b070 0x562c099c3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.307+0000 7f417ab08740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4150011360 con 0x562c095ea390 2024-07-31T15:03:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.307+0000 7f417ab08740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4150010c90 con 0x562c095e21b0 2024-07-31T15:03:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.307+0000 7f417ab08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f415000def0 con 0x562c0960b070 2024-07-31T15:03:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.307+0000 7f417ab08740 10 client.0 _ll_register_callbacks cb 0x7f41640dc3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:03:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.307+0000 7f4171b52640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.308+0000 7f4171b52640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49218/0 (socket says 172.21.15.67:49218) 2024-07-31T15:03:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.308+0000 7f4171b52640 1 -- 172.21.15.67:0/4181517538 learned_addr learned my addr 172.21.15.67:0/4181517538 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:03:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.308+0000 7f4171b52640 1 -- 172.21.15.67:0/4181517538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c0960b070 msgr2=0x562c099c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.308+0000 7f4171b52640 1 --2- 172.21.15.67:0/4181517538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c0960b070 0x562c099c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.308+0000 7f4171b52640 1 -- 172.21.15.67:0/4181517538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095e21b0 msgr2=0x562c099f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:03:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.308+0000 7f4171b52640 1 --2- 172.21.15.67:0/4181517538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095e21b0 0x562c099f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.308+0000 7f4171b52640 1 -- 172.21.15.67:0/4181517538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c099c3990 con 0x562c095ea390 2024-07-31T15:03:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.308+0000 7f4171b52640 1 --2- 172.21.15.67:0/4181517538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099c2e10 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f415803f950 tx=0x7f415803f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63bf8073d450552 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:03:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.309+0000 7f4155ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:03:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.309+0000 7f4155ffb640 1 -- 172.21.15.67:0/4181517538 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158044040 con 0x562c095ea390 2024-07-31T15:03:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.309+0000 7f417ab08740 1 -- 172.21.15.67:0/4181517538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4158012ca0 con 0x562c095ea390 2024-07-31T15:03:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.309+0000 7f4155ffb640 1 -- 172.21.15.67:0/4181517538 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f41580127f0 con 0x562c095ea390 2024-07-31T15:03:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.310+0000 7f4155ffb640 1 -- 172.21.15.67:0/4181517538 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f4158055050 con 0x562c095ea390 2024-07-31T15:03:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.310+0000 7f417ab08740 10 client.18240 fetch_fsmap learned FSMap version 63 2024-07-31T15:03:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.310+0000 7f417ab08740 1 -- 172.21.15.67:0/4181517538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x562c099c8c60 con 0x562c095ea390 2024-07-31T15:03:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.310+0000 7f4155ffb640 1 -- 172.21.15.67:0/4181517538 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4158012ca0 con 0x562c095ea390 2024-07-31T15:03:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.310+0000 7f4155ffb640 1 -- 172.21.15.67:0/4181517538 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f41580489b0 con 0x562c095ea390 2024-07-31T15:03:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.311+0000 7f417ab08740 10 client.18240 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:03:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.311+0000 7f417ab08740 10 client.18240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:03:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.311+0000 7f417ab08740 20 client.18240 populate_metadata read hostname 'smithi067' 2024-07-31T15:03:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.311+0000 7f417ab08740 1 --2- 172.21.15.67:0/4181517538 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562c0959eda0 0x562c09a5b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:03:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.311+0000 7f417ab08740 4 client.18240 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:03:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.311+0000 7f417ab08740 1 -- 172.21.15.67:0/4181517538 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562c09924550 con 0x562c0959eda0 2024-07-31T15:03:51.316 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/pat/memtype_interval.o 2024-07-31T15:03:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.311+0000 7f4172353640 1 --2- 172.21.15.67:0/4181517538 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562c0959eda0 0x562c09a5b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:03:51.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.319+0000 7f4172353640 1 --2- 172.21.15.67:0/4181517538 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562c0959eda0 0x562c09a5b210 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e51ccb1976447f05 server_cookie=efcf7215795bc2d6 in_seq=0 out_seq=0 2024-07-31T15:03:51.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.319+0000 7f4155ffb640 10 client.18240 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:51.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.323+0000 7f4155ffb640 1 -- 172.21.15.67:0/4181517538 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x562c09924550 con 0x562c0959eda0 2024-07-31T15:03:51.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.323+0000 7f4155ffb640 10 client.18240 handle_command_reply: tid=0 2024-07-31T15:03:51.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.323+0000 7f417ab08740 1 client.18240 shutdown 2024-07-31T15:03:51.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.329+0000 7f417ab08740 1 -- 172.21.15.67:0/4181517538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 msgr2=0x562c099c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:51.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.329+0000 7f417ab08740 1 --2- 172.21.15.67:0/4181517538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099c2e10 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f415803f950 tx=0x7f415803f680 comp rx=0 tx=0).stop 2024-07-31T15:03:51.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.330+0000 7f417ab08740 1 -- 172.21.15.67:0/4181517538 shutdown_connections 2024-07-31T15:03:51.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.330+0000 7f417ab08740 1 --2- 172.21.15.67:0/4181517538 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562c0959eda0 0x562c09a5b210 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.330+0000 7f417ab08740 1 --2- 172.21.15.67:0/4181517538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c0960b070 0x562c099c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.330+0000 7f417ab08740 1 --2- 172.21.15.67:0/4181517538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c095ea390 0x562c099c2e10 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.330+0000 7f417ab08740 1 --2- 172.21.15.67:0/4181517538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c095e21b0 0x562c099f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.330+0000 7f417ab08740 1 -- 172.21.15.67:0/4181517538 >> 172.21.15.67:0/4181517538 conn(0x562c095ff460 msgr2=0x562c099b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:51.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.330+0000 7f4155ffb640 0 client.18240 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:03:51.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.333+0000 7f417ab08740 1 -- 172.21.15.67:0/4181517538 shutdown_connections 2024-07-31T15:03:51.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.337+0000 7f417ab08740 1 -- 172.21.15.67:0/4181517538 wait complete. 2024-07-31T15:03:51.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.338+0000 7f417ab08740 20 client.18240 trim_cache size 0 max 16384 2024-07-31T15:03:51.340 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:03:51.340 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (13798 inodes in the stack)", 2024-07-31T15:03:51.340 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T15:03:51.340 INFO:teuthology.orchestra.run.smithi067.stdout: "edd2c496-b8f0-45e2-9f9a-ec736f011bee": { 2024-07-31T15:03:51.340 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T15:03:51.340 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "edd2c496-b8f0-45e2-9f9a-ec736f011bee", 2024-07-31T15:03:51.340 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T15:03:51.340 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:03:51.340 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:03:51.340 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:03:51.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.340+0000 7f41745b7640 1 -- 172.21.15.67:0/1862802648 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f415c0b3f50 msgr2=0x7f415c0b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:51.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.340+0000 7f41745b7640 1 --2- 172.21.15.67:0/1862802648 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f415c0b3f50 0x7f415c0b6460 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f416000c870 tx=0x7f416000cc70 comp rx=0 tx=0).stop 2024-07-31T15:03:51.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.340+0000 7f41745b7640 1 -- 172.21.15.67:0/1862802648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f416c10a410 msgr2=0x7f416c11bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:03:51.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.340+0000 7f41745b7640 1 --2- 172.21.15.67:0/1862802648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f416c10a410 0x7f416c11bfb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f416403b9e0 tx=0x7f4164004ac0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.340+0000 7f41745b7640 1 -- 172.21.15.67:0/1862802648 shutdown_connections 2024-07-31T15:03:51.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.340+0000 7f41745b7640 1 --2- 172.21.15.67:0/1862802648 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f415c0b3f50 0x7f415c0b6460 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.340+0000 7f41745b7640 1 --2- 172.21.15.67:0/1862802648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f416c11c660 0x7f416c118960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.340+0000 7f41745b7640 1 --2- 172.21.15.67:0/1862802648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f416c111970 0x7f416c118290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.340+0000 7f41745b7640 1 --2- 172.21.15.67:0/1862802648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f416c10a410 0x7f416c11bfb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:03:51.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.340+0000 7f41745b7640 1 -- 172.21.15.67:0/1862802648 >> 172.21.15.67:0/1862802648 conn(0x7f416c0708f0 msgr2=0x7f416c10b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:03:51.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.344+0000 7f41745b7640 1 -- 172.21.15.67:0/1862802648 shutdown_connections 2024-07-31T15:03:51.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:03:51.344+0000 7f41745b7640 1 -- 172.21.15.67:0/1862802648 wait complete. 2024-07-31T15:03:51.393 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/jbd2/journal.o 2024-07-31T15:03:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:03:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: pgmap v803: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 823 KiB/s wr, 295 op/s 2024-07-31T15:03:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:51.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:51.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:51.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:51.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:51.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:51.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:51.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:51.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:51.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:03:51.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: pgmap v803: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 823 KiB/s wr, 295 op/s 2024-07-31T15:03:51.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:51.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:51.808 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/min_addr.o 2024-07-31T15:03:51.817 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/page_vma_mapped.o 2024-07-31T15:03:51.888 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/dma/direct.o 2024-07-31T15:03:51.953 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/fsmap.o 2024-07-31T15:03:52.002 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/entry/common.o 2024-07-31T15:03:52.026 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/mm/pat/built-in.a 2024-07-31T15:03:52.080 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/init.o 2024-07-31T15:03:52.287 INFO:tasks.cephfs.filesystem:scrub status for tag:edd2c496-b8f0-45e2-9f9a-ec736f011bee - {'path': '/', 'tag': 'edd2c496-b8f0-45e2-9f9a-ec736f011bee', 'options': 'recursive,force'} 2024-07-31T15:03:52.475 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/security.o 2024-07-31T15:03:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:52.617 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/pagewalk.o 2024-07-31T15:03:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:52.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:52.769 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/dma/ops_helpers.o 2024-07-31T15:03:53.103 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/init_64.o 2024-07-31T15:03:53.122 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/entry/syscall_user_dispatch.o 2024-07-31T15:03:53.431 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/dma/dummy.o 2024-07-31T15:03:53.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: pgmap v804: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 869 KiB/s wr, 604 op/s 2024-07-31T15:03:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:53.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:53 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.519 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/pgtable-generic.o 2024-07-31T15:03:53.520 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/fsync.o 2024-07-31T15:03:53.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: pgmap v804: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 869 KiB/s wr, 604 op/s 2024-07-31T15:03:53.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:53.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:53 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: pgmap v804: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 869 KiB/s wr, 604 op/s 2024-07-31T15:03:53.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.656 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.656 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.656 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.656 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.656 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.656 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.656 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.656 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.656 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.656 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.656 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.656 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.657 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.657 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.657 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.657 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:53.657 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:53 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:53.766 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/entry/built-in.a 2024-07-31T15:03:53.823 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/dma/swiotlb.o 2024-07-31T15:03:53.873 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/jbd2/built-in.a 2024-07-31T15:03:53.915 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/lsm_audit.o 2024-07-31T15:03:54.041 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/fault.o 2024-07-31T15:03:54.295 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/rmap.o 2024-07-31T15:03:54.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:54.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.501 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/ioremap.o 2024-07-31T15:03:54.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:54.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:54.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:54.725 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/hash.o 2024-07-31T15:03:54.929 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/ialloc.o 2024-07-31T15:03:54.933 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/dma/remap.o 2024-07-31T15:03:55.114 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/device_cgroup.o 2024-07-31T15:03:55.342 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/extable.o 2024-07-31T15:03:55.462 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: pgmap v805: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 925 KiB/s wr, 1.33k op/s 2024-07-31T15:03:55.462 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.462 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.462 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.463 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.463 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.463 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.463 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.463 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.463 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.463 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.463 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.463 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.463 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.463 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.463 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.466 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.466 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:55.466 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.483 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/mmap.o 2024-07-31T15:03:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: pgmap v805: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 925 KiB/s wr, 1.33k op/s 2024-07-31T15:03:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.602 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/pgtable.o 2024-07-31T15:03:55.609 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/dma/built-in.a 2024-07-31T15:03:55.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: pgmap v805: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 925 KiB/s wr, 1.33k op/s 2024-07-31T15:03:55.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:55.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:55.685 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/module/main.o 2024-07-31T15:03:56.112 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/vmalloc.o 2024-07-31T15:03:56.149 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/physaddr.o 2024-07-31T15:03:56.165 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/tlb.o 2024-07-31T15:03:56.241 INFO:tasks.workunit.client.0.smithi067.stdout: AR security/built-in.a 2024-07-31T15:03:56.457 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/cpu_entry_area.o 2024-07-31T15:03:56.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:56.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:56.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.609 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/asymmetric_type.o 2024-07-31T15:03:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:56.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:56.816 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/restrict.o 2024-07-31T15:03:57.261 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.262 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.262 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.262 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.262 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: pgmap v806: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 898 KiB/s wr, 2.02k op/s 2024-07-31T15:03:57.262 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.262 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.262 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.263 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.263 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.263 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.263 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.263 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.263 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.263 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.263 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.264 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.264 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.264 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.264 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.264 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.264 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.264 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.265 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.265 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.265 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.265 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.265 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.265 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.265 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.265 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.265 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.266 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.266 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.266 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.266 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.266 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:57.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.294 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/maccess.o 2024-07-31T15:03:57.312 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/pgprot.o 2024-07-31T15:03:57.351 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/indirect.o 2024-07-31T15:03:57.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: pgmap v806: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 898 KiB/s wr, 2.02k op/s 2024-07-31T15:03:57.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.496 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.496 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:57.496 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: pgmap v806: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 898 KiB/s wr, 2.02k op/s 2024-07-31T15:03:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:57.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:57.818 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/time.o 2024-07-31T15:03:57.929 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/module/strict_rwx.o 2024-07-31T15:03:57.942 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/hugetlbpage.o 2024-07-31T15:03:58.113 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/module/kmod.o 2024-07-31T15:03:58.178 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/signature.o 2024-07-31T15:03:58.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:58.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:58.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:58.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:58.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:58.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:58.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:58.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:58.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:58.748 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/public_key.o 2024-07-31T15:03:58.852 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/partitions/core.o 2024-07-31T15:03:58.869 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/dump_pagetables.o 2024-07-31T15:03:58.970 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/timer.o 2024-07-31T15:03:58.987 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/process_vm_access.o 2024-07-31T15:03:59.077 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/module/tree_lookup.o 2024-07-31T15:03:59.187 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/inline.o 2024-07-31T15:03:59.239 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:59 smithi067 ceph-mon[32750]: pgmap v807: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 745 KiB/s wr, 3.17k op/s 2024-07-31T15:03:59.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:59.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:59.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:59.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:59.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:03:59.240 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:03:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.504 INFO:tasks.workunit.client.0.smithi067.stdout: ASN.1 crypto/asymmetric_keys/x509.asn1.[ch] 2024-07-31T15:03:59.516 INFO:tasks.workunit.client.0.smithi067.stdout: ASN.1 crypto/asymmetric_keys/x509_akid.asn1.[ch] 2024-07-31T15:03:59.571 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/x509_loader.o 2024-07-31T15:03:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:59 smithi073 ceph-mon[36714]: pgmap v807: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 745 KiB/s wr, 3.17k op/s 2024-07-31T15:03:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:03:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:03:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.721 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/module/kallsyms.o 2024-07-31T15:03:59.723 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/numa.o 2024-07-31T15:03:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:59 smithi077 ceph-mon[37327]: pgmap v807: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 745 KiB/s wr, 3.17k op/s 2024-07-31T15:03:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:03:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:03:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:03:59.782 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/page_alloc.o 2024-07-31T15:03:59.792 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/partitions/msdos.o 2024-07-31T15:03:59.820 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/x509_public_key.o 2024-07-31T15:03:59.875 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/powerflags.o 2024-07-31T15:04:00.084 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/feat_ctl.o 2024-07-31T15:04:00.467 INFO:tasks.workunit.client.0.smithi067.stdout: ASN.1 crypto/asymmetric_keys/pkcs7.asn1.[ch] 2024-07-31T15:04:00.480 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/pkcs7_trust.o 2024-07-31T15:04:00.506 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/module/procfs.o 2024-07-31T15:04:00.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.593 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/hrtimer.o 2024-07-31T15:04:00.593 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/intel.o 2024-07-31T15:04:00.636 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/partitions/efi.o 2024-07-31T15:04:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:00.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.701 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/inode.o 2024-07-31T15:04:00.704 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/numa_64.o 2024-07-31T15:04:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:00.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:00.806 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/pkcs7_verify.o 2024-07-31T15:04:01.083 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/module/sysfs.o 2024-07-31T15:04:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.189+0000 7f3643500640 1 -- 172.21.15.67:0/913487132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c10f880 msgr2=0x7f363c10fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.189+0000 7f3643500640 1 --2- 172.21.15.67:0/913487132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c10f880 0x7f363c10fc90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3638009900 tx=0x7f363802b550 comp rx=0 tx=0).stop 2024-07-31T15:04:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.190+0000 7f3643500640 1 -- 172.21.15.67:0/913487132 shutdown_connections 2024-07-31T15:04:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.190+0000 7f3643500640 1 --2- 172.21.15.67:0/913487132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f363c1101d0 0x7f363c116a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.190+0000 7f3643500640 1 --2- 172.21.15.67:0/913487132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c10f880 0x7f363c10fc90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.190+0000 7f3643500640 1 --2- 172.21.15.67:0/913487132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f363c10a090 0x7f363c10a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.190+0000 7f3643500640 1 -- 172.21.15.67:0/913487132 >> 172.21.15.67:0/913487132 conn(0x7f363c0707c0 msgr2=0x7f363c070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.190+0000 7f3643500640 1 -- 172.21.15.67:0/913487132 shutdown_connections 2024-07-31T15:04:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.190+0000 7f3643500640 1 -- 172.21.15.67:0/913487132 wait complete. 2024-07-31T15:04:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.192+0000 7f3643500640 1 Processor -- start 2024-07-31T15:04:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.192+0000 7f3643500640 1 -- start start 2024-07-31T15:04:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.194+0000 7f3643500640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f363c10a090 0x7f363c1a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.194+0000 7f3643500640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f363c10f880 0x7f363c1a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.194+0000 7f3643500640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c1101d0 0x7f363c1a7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.194+0000 7f364129c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f363c10a090 0x7f363c1a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.194+0000 7f364129c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f363c10a090 0x7f363c1a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49484/0 (socket says 172.21.15.67:49484) 2024-07-31T15:04:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.194+0000 7f364129c640 1 -- 172.21.15.67:0/4022253333 learned_addr learned my addr 172.21.15.67:0/4022253333 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.194+0000 7f3643500640 1 -- 172.21.15.67:0/4022253333 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f363c119ab0 con 0x7f363c10f880 2024-07-31T15:04:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.194+0000 7f3640a9b640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f363c10f880 0x7f363c1a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.194+0000 7f3643500640 1 -- 172.21.15.67:0/4022253333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f363c119930 con 0x7f363c1101d0 2024-07-31T15:04:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.194+0000 7f3643500640 1 -- 172.21.15.67:0/4022253333 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f363c119c30 con 0x7f363c10a090 2024-07-31T15:04:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.195+0000 7f3641a9d640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c1101d0 0x7f363c1a7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.195+0000 7f364129c640 1 -- 172.21.15.67:0/4022253333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c1101d0 msgr2=0x7f363c1a7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.195+0000 7f364129c640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c1101d0 0x7f363c1a7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.195+0000 7f364129c640 1 -- 172.21.15.67:0/4022253333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f363c10f880 msgr2=0x7f363c1a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.195+0000 7f364129c640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f363c10f880 0x7f363c1a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.195+0000 7f364129c640 1 -- 172.21.15.67:0/4022253333 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c1a8420 con 0x7f363c10a090 2024-07-31T15:04:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.195+0000 7f3641a9d640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c1101d0 0x7f363c1a7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.195+0000 7f3640a9b640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f363c10f880 0x7f363c1a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.196+0000 7f364129c640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f363c10a090 0x7f363c1a2580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f362c00c920 tx=0x7f362c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.197+0000 7f36327fc640 1 -- 172.21.15.67:0/4022253333 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c007c70 con 0x7f363c10a090 2024-07-31T15:04:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.197+0000 7f36327fc640 1 -- 172.21.15.67:0/4022253333 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f362c007e10 con 0x7f363c10a090 2024-07-31T15:04:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.197+0000 7f36327fc640 1 -- 172.21.15.67:0/4022253333 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c018820 con 0x7f363c10a090 2024-07-31T15:04:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.197+0000 7f3643500640 1 -- 172.21.15.67:0/4022253333 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363c1a8710 con 0x7f363c10a090 2024-07-31T15:04:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.197+0000 7f3643500640 1 -- 172.21.15.67:0/4022253333 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363c1a8b80 con 0x7f363c10a090 2024-07-31T15:04:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.201+0000 7f364a3f6740 1 Processor -- start 2024-07-31T15:04:01.206 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/x509.asn1.o 2024-07-31T15:04:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.203+0000 7f36327fc640 1 -- 172.21.15.67:0/4022253333 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f362c00f450 con 0x7f363c10a090 2024-07-31T15:04:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.205+0000 7f36327fc640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f36280b3e70 0x7f36280b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.205+0000 7f36327fc640 1 -- 172.21.15.67:0/4022253333 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f362c0e43e0 con 0x7f363c10a090 2024-07-31T15:04:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.205+0000 7f3641a9d640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f36280b3e70 0x7f36280b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.205+0000 7f364a3f6740 1 -- start start 2024-07-31T15:04:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.206+0000 7f3641a9d640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f36280b3e70 0x7f36280b6380 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f363400a7c0 tx=0x7f363400d040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.206+0000 7f364a3f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677891b0 0x562767b2bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.206+0000 7f364a3f6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677602f0 0x562767b2c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.207+0000 7f364a3f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677684d0 0x562767b2c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.207+0000 7f364a3f6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562767aa13b0 con 0x5627677602f0 2024-07-31T15:04:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.207+0000 7f364129c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677891b0 0x562767b2bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.207+0000 7f364129c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677891b0 0x562767b2bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49498/0 (socket says 172.21.15.67:49498) 2024-07-31T15:04:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.207+0000 7f364129c640 1 -- 172.21.15.67:0/1032885589 learned_addr learned my addr 172.21.15.67:0/1032885589 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.207+0000 7f3640a9b640 1 --2- 172.21.15.67:0/1032885589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677602f0 0x562767b2c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.207+0000 7f364a3f6740 1 -- 172.21.15.67:0/1032885589 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562767a908b0 con 0x5627677684d0 2024-07-31T15:04:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.207+0000 7f3641a9d640 1 --2- 172.21.15.67:0/1032885589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677684d0 0x562767b2c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.207+0000 7f364a3f6740 1 -- 172.21.15.67:0/1032885589 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562767a948f0 con 0x5627677891b0 2024-07-31T15:04:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.208+0000 7f364129c640 1 -- 172.21.15.67:0/1032885589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677684d0 msgr2=0x562767b2c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.208+0000 7f364129c640 1 --2- 172.21.15.67:0/1032885589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677684d0 0x562767b2c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.208+0000 7f364129c640 1 -- 172.21.15.67:0/1032885589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677602f0 msgr2=0x562767b2c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.208+0000 7f364129c640 1 --2- 172.21.15.67:0/1032885589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677602f0 0x562767b2c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.208+0000 7f364129c640 1 -- 172.21.15.67:0/1032885589 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562767a379a0 con 0x5627677891b0 2024-07-31T15:04:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.208+0000 7f3640a9b640 1 --2- 172.21.15.67:0/1032885589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677602f0 0x562767b2c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.208+0000 7f364129c640 1 --2- 172.21.15.67:0/1032885589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677891b0 0x562767b2bce0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f362c0a7ec0 tx=0x7f362c0aa6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4ca010400ed13f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.208+0000 7f36277fe640 1 -- 172.21.15.67:0/1032885589 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0af250 con 0x5627677891b0 2024-07-31T15:04:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.208+0000 7f36277fe640 1 -- 172.21.15.67:0/1032885589 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f362c0b3d00 con 0x5627677891b0 2024-07-31T15:04:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.208+0000 7f36277fe640 1 -- 172.21.15.67:0/1032885589 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0e1a10 con 0x5627677891b0 2024-07-31T15:04:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.210+0000 7f364a3f6740 1 -- 172.21.15.67:0/1032885589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677891b0 msgr2=0x562767b2bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.210+0000 7f364a3f6740 1 --2- 172.21.15.67:0/1032885589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677891b0 0x562767b2bce0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f362c0a7ec0 tx=0x7f362c0aa6d0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.210+0000 7f364a3f6740 1 -- 172.21.15.67:0/1032885589 shutdown_connections 2024-07-31T15:04:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.210+0000 7f364a3f6740 1 --2- 172.21.15.67:0/1032885589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677684d0 0x562767b2c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.210+0000 7f364a3f6740 1 --2- 172.21.15.67:0/1032885589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677602f0 0x562767b2c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.210+0000 7f364a3f6740 1 --2- 172.21.15.67:0/1032885589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677891b0 0x562767b2bce0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.210+0000 7f364a3f6740 1 -- 172.21.15.67:0/1032885589 >> 172.21.15.67:0/1032885589 conn(0x56276777d300 msgr2=0x562767b29bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.210+0000 7f364a3f6740 1 -- 172.21.15.67:0/1032885589 shutdown_connections 2024-07-31T15:04:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.210+0000 7f364a3f6740 1 -- 172.21.15.67:0/1032885589 wait complete. 2024-07-31T15:04:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.211+0000 7f364a3f6740 1 -- start start 2024-07-31T15:04:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.211+0000 7f364a3f6740 1 Processor -- start 2024-07-31T15:04:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.212+0000 7f364a3f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677602f0 0x562767b40610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.212+0000 7f364a3f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677684d0 0x562767b40bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.212+0000 7f364a3f6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677891b0 0x562767b45c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.212+0000 7f364a3f6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5627679caab0 con 0x5627677891b0 2024-07-31T15:04:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.212+0000 7f364a3f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562767a18cf0 con 0x5627677684d0 2024-07-31T15:04:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.212+0000 7f364a3f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562767b3a460 con 0x5627677602f0 2024-07-31T15:04:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.212+0000 7f364a3f6740 10 client.0 _ll_register_callbacks cb 0x562767a8a9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.212+0000 7f3640a9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677684d0 0x562767b40bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.212+0000 7f3640a9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677684d0 0x562767b40bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35522/0 (socket says 172.21.15.67:35522) 2024-07-31T15:04:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.212+0000 7f3640a9b640 1 -- 172.21.15.67:0/3625977093 learned_addr learned my addr 172.21.15.67:0/3625977093 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.212+0000 7f364129c640 1 --2- 172.21.15.67:0/3625977093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677602f0 0x562767b40610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.212+0000 7f3641a9d640 1 --2- 172.21.15.67:0/3625977093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677891b0 0x562767b45c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.215 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/amdtopology.o 2024-07-31T15:04:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.213+0000 7f364129c640 1 -- 172.21.15.67:0/3625977093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677684d0 msgr2=0x562767b40bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.213+0000 7f364129c640 1 --2- 172.21.15.67:0/3625977093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677684d0 0x562767b40bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.213+0000 7f364129c640 1 -- 172.21.15.67:0/3625977093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677891b0 msgr2=0x562767b45c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.213+0000 7f364129c640 1 --2- 172.21.15.67:0/3625977093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677891b0 0x562767b45c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.213+0000 7f364129c640 1 -- 172.21.15.67:0/3625977093 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627677f4790 con 0x5627677602f0 2024-07-31T15:04:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.213+0000 7f3641a9d640 1 --2- 172.21.15.67:0/3625977093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677891b0 0x562767b45c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:04:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.213+0000 7f364129c640 1 --2- 172.21.15.67:0/3625977093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677602f0 0x562767b40610 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f362c0aa950 tx=0x7f362c00fca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9446f8de2ddfedcf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.216+0000 7f3624ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:04:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.216+0000 7f3624ff9640 1 -- 172.21.15.67:0/3625977093 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0af250 con 0x5627677602f0 2024-07-31T15:04:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.216+0000 7f3624ff9640 1 -- 172.21.15.67:0/3625977093 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f362c0a7240 con 0x5627677602f0 2024-07-31T15:04:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.216+0000 7f364a3f6740 1 -- 172.21.15.67:0/3625977093 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f362c0b3d00 con 0x5627677602f0 2024-07-31T15:04:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.217+0000 7f3624ff9640 1 -- 172.21.15.67:0/3625977093 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f362c0e1a10 con 0x5627677602f0 2024-07-31T15:04:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.217+0000 7f364a3f6740 10 client.27754 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.217+0000 7f364a3f6740 1 -- 172.21.15.67:0/3625977093 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x56276786d620 con 0x5627677602f0 2024-07-31T15:04:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.217+0000 7f3624ff9640 1 -- 172.21.15.67:0/3625977093 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f362c0b3d00 con 0x5627677602f0 2024-07-31T15:04:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.219+0000 7f3624ff9640 1 -- 172.21.15.67:0/3625977093 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f362c0c1050 con 0x5627677602f0 2024-07-31T15:04:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.219+0000 7f364a3f6740 10 client.27754 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.219+0000 7f364a3f6740 10 client.27754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.219+0000 7f364a3f6740 20 client.27754 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.219+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3625977093 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562767747d80 0x562767b51e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.219+0000 7f364a3f6740 4 client.27754 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:04:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.219+0000 7f364a3f6740 1 -- 172.21.15.67:0/3625977093 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562767aa3300 con 0x562767747d80 2024-07-31T15:04:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.219+0000 7f3640a9b640 1 --2- 172.21.15.67:0/3625977093 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562767747d80 0x562767b51e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.220+0000 7f3640a9b640 1 --2- 172.21.15.67:0/3625977093 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562767747d80 0x562767b51e70 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba9d5125f202b398 server_cookie=14d7988c59625525 in_seq=0 out_seq=0 2024-07-31T15:04:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.220+0000 7f3624ff9640 10 client.27754 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.220+0000 7f3624ff9640 1 -- 172.21.15.67:0/3625977093 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562767aa3300 con 0x562767747d80 2024-07-31T15:04:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.221+0000 7f3624ff9640 10 client.27754 handle_command_reply: tid=0 2024-07-31T15:04:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.221+0000 7f364a3f6740 1 client.27754 shutdown 2024-07-31T15:04:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.221+0000 7f364a3f6740 1 -- 172.21.15.67:0/3625977093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677602f0 msgr2=0x562767b40610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.221+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3625977093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677602f0 0x562767b40610 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f362c0aa950 tx=0x7f362c00fca0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.221+0000 7f364a3f6740 1 -- 172.21.15.67:0/3625977093 shutdown_connections 2024-07-31T15:04:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.221+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3625977093 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562767747d80 0x562767b51e70 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.221+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3625977093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677891b0 0x562767b45c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.221+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3625977093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677684d0 0x562767b40bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.221+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3625977093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677602f0 0x562767b40610 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.221+0000 7f364a3f6740 1 -- 172.21.15.67:0/3625977093 >> 172.21.15.67:0/3625977093 conn(0x56276777d300 msgr2=0x562767b2cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.221+0000 7f3624ff9640 0 client.27754 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.222+0000 7f364a3f6740 1 -- 172.21.15.67:0/3625977093 shutdown_connections 2024-07-31T15:04:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.222+0000 7f364a3f6740 1 -- 172.21.15.67:0/3625977093 wait complete. 2024-07-31T15:04:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.222+0000 7f364a3f6740 20 client.27754 trim_cache size 0 max 16384 2024-07-31T15:04:01.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.255+0000 7f364a3f6740 1 Processor -- start 2024-07-31T15:04:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.255+0000 7f364a3f6740 1 -- start start 2024-07-31T15:04:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.256+0000 7f364a3f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677602f0 0x562767b397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.256+0000 7f364a3f6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677684d0 0x562767b4ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.256+0000 7f364a3f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677891b0 0x562767b3a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.256+0000 7f364a3f6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1b6bb0 con 0x5627677684d0 2024-07-31T15:04:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.256+0000 7f364a3f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1b86f0 con 0x5627677602f0 2024-07-31T15:04:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.256+0000 7f364a3f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1b7280 con 0x5627677891b0 2024-07-31T15:04:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.256+0000 7f3641a9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677891b0 0x562767b3a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.256+0000 7f3641a9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677891b0 0x562767b3a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49514/0 (socket says 172.21.15.67:49514) 2024-07-31T15:04:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.256+0000 7f3641a9d640 1 -- 172.21.15.67:0/3070355772 learned_addr learned my addr 172.21.15.67:0/3070355772 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.256+0000 7f364129c640 1 --2- 172.21.15.67:0/3070355772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677602f0 0x562767b397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.256+0000 7f3640a9b640 1 --2- 172.21.15.67:0/3070355772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677684d0 0x562767b4ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.257+0000 7f3641a9d640 1 -- 172.21.15.67:0/3070355772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677602f0 msgr2=0x562767b397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.257+0000 7f3641a9d640 1 --2- 172.21.15.67:0/3070355772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677602f0 0x562767b397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.257+0000 7f3641a9d640 1 -- 172.21.15.67:0/3070355772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677684d0 msgr2=0x562767b4ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.257+0000 7f3641a9d640 1 --2- 172.21.15.67:0/3070355772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677684d0 0x562767b4ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.257+0000 7f3641a9d640 1 -- 172.21.15.67:0/3070355772 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562767b2cb10 con 0x5627677891b0 2024-07-31T15:04:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.257+0000 7f3641a9d640 1 --2- 172.21.15.67:0/3070355772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677891b0 0x562767b3a6a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f363400eb00 tx=0x7f3634015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5130a4b3dc5911e9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.259+0000 7f3626ffd640 1 -- 172.21.15.67:0/3070355772 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363402a070 con 0x5627677891b0 2024-07-31T15:04:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.260+0000 7f3626ffd640 1 -- 172.21.15.67:0/3070355772 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3634016de0 con 0x5627677891b0 2024-07-31T15:04:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.264+0000 7f364a3f6740 1 -- 172.21.15.67:0/3070355772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677891b0 msgr2=0x562767b3a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.264+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3070355772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677891b0 0x562767b3a6a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f363400eb00 tx=0x7f3634015000 comp rx=0 tx=0).stop 2024-07-31T15:04:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.264+0000 7f364a3f6740 1 -- 172.21.15.67:0/3070355772 shutdown_connections 2024-07-31T15:04:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.264+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3070355772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677891b0 0x562767b3a6a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.264+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3070355772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627677684d0 0x562767b4ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.264+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3070355772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677602f0 0x562767b397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.264+0000 7f364a3f6740 1 -- 172.21.15.67:0/3070355772 >> 172.21.15.67:0/3070355772 conn(0x56276777d300 msgr2=0x562767b40370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.264+0000 7f364a3f6740 1 -- 172.21.15.67:0/3070355772 shutdown_connections 2024-07-31T15:04:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.264+0000 7f364a3f6740 1 -- 172.21.15.67:0/3070355772 wait complete. 2024-07-31T15:04:01.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.265+0000 7f364a3f6740 1 -- start start 2024-07-31T15:04:01.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.265+0000 7f364a3f6740 1 Processor -- start 2024-07-31T15:04:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.266+0000 7f364a3f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677602f0 0x562767b282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.266+0000 7f364a3f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677684d0 0x562767b287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.266+0000 7f364a3f6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56276771d1c0 0x562767b28d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.266+0000 7f364a3f6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1b8020 con 0x56276771d1c0 2024-07-31T15:04:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.266+0000 7f364a3f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1b7950 con 0x5627677602f0 2024-07-31T15:04:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.266+0000 7f364a3f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1b4c40 con 0x5627677684d0 2024-07-31T15:04:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.266+0000 7f364a3f6740 10 client.0 _ll_register_callbacks cb 0x7f362c0a63e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.266+0000 7f3640a9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677684d0 0x562767b287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.266+0000 7f364129c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677602f0 0x562767b282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.267+0000 7f364129c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677602f0 0x562767b282b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35528/0 (socket says 172.21.15.67:35528) 2024-07-31T15:04:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.267+0000 7f364129c640 1 -- 172.21.15.67:0/3912303001 learned_addr learned my addr 172.21.15.67:0/3912303001 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.267+0000 7f3641a9d640 1 --2- 172.21.15.67:0/3912303001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56276771d1c0 0x562767b28d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.267+0000 7f364129c640 1 -- 172.21.15.67:0/3912303001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677684d0 msgr2=0x562767b287f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.267+0000 7f364129c640 1 --2- 172.21.15.67:0/3912303001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677684d0 0x562767b287f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.267+0000 7f364129c640 1 -- 172.21.15.67:0/3912303001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56276771d1c0 msgr2=0x562767b28d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.267+0000 7f364129c640 1 --2- 172.21.15.67:0/3912303001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56276771d1c0 0x562767b28d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.267+0000 7f364129c640 1 -- 172.21.15.67:0/3912303001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562767b4b930 con 0x5627677602f0 2024-07-31T15:04:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.267+0000 7f3641a9d640 1 --2- 172.21.15.67:0/3912303001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56276771d1c0 0x562767b28d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:04:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.267+0000 7f364129c640 1 --2- 172.21.15.67:0/3912303001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677602f0 0x562767b282b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f362c0aa900 tx=0x7f362c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba0a6e331d6b0169 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.268+0000 7f3624ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:04:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.268+0000 7f3624ff9640 1 -- 172.21.15.67:0/3912303001 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0af250 con 0x5627677602f0 2024-07-31T15:04:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.268+0000 7f364a3f6740 1 -- 172.21.15.67:0/3912303001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562767a9eda0 con 0x5627677602f0 2024-07-31T15:04:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.268+0000 7f3624ff9640 1 -- 172.21.15.67:0/3912303001 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9351+0+0 (secure 0 0 0) 0x7f362c0e4180 con 0x5627677602f0 2024-07-31T15:04:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.268+0000 7f3624ff9640 1 -- 172.21.15.67:0/3912303001 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f362c0b3b00 con 0x5627677602f0 2024-07-31T15:04:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.268+0000 7f364a3f6740 10 client.27815 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.268+0000 7f364a3f6740 1 -- 172.21.15.67:0/3912303001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x562767bb8980 con 0x5627677602f0 2024-07-31T15:04:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.269+0000 7f3624ff9640 1 -- 172.21.15.67:0/3912303001 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562767a9eda0 con 0x5627677602f0 2024-07-31T15:04:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.270+0000 7f3624ff9640 1 -- 172.21.15.67:0/3912303001 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f362c0c1070 con 0x5627677602f0 2024-07-31T15:04:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.270+0000 7f364a3f6740 10 client.27815 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.270+0000 7f364a3f6740 10 client.27815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.270+0000 7f364a3f6740 20 client.27815 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.270+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3912303001 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562767bb8e40 0x562767bbb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.270+0000 7f364a3f6740 4 client.27815 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:04:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.270+0000 7f364a3f6740 1 -- 172.21.15.67:0/3912303001 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562767a9c4e0 con 0x562767bb8e40 2024-07-31T15:04:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.271+0000 7f3640a9b640 1 --2- 172.21.15.67:0/3912303001 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562767bb8e40 0x562767bbb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.271+0000 7f3640a9b640 1 --2- 172.21.15.67:0/3912303001 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562767bb8e40 0x562767bbb260 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69dd94e57d402fad server_cookie=41c9a64db46504fb in_seq=0 out_seq=0 2024-07-31T15:04:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:01.271+0000 7f3624ff9640 10 client.27815 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:01.281 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/x509_akid.asn1.o 2024-07-31T15:04:01.321 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/x509_cert_parser.o 2024-07-31T15:04:01.416 INFO:tasks.workunit.client.0.smithi067.stdout: AR block/partitions/built-in.a 2024-07-31T15:04:01.491 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/bdev.o 2024-07-31T15:04:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:01.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:01 smithi067 ceph-mon[32750]: pgmap v808: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 856 KiB/s wr, 3.97k op/s 2024-07-31T15:04:01.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:01.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:01.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:01.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:01.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:01 smithi073 ceph-mon[36714]: pgmap v808: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 856 KiB/s wr, 3.97k op/s 2024-07-31T15:04:01.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:01.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:01.702 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/intel_pconfig.o 2024-07-31T15:04:01.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:01.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:01.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:01 smithi077 ceph-mon[37327]: pgmap v808: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 856 KiB/s wr, 3.97k op/s 2024-07-31T15:04:01.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:01.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:01.984 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/pkcs7.asn1.o 2024-07-31T15:04:02.027 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/module/built-in.a 2024-07-31T15:04:02.046 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/pkcs7_parser.o 2024-07-31T15:04:02.055 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/tsx.o 2024-07-31T15:04:02.068 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/timekeeping.o 2024-07-31T15:04:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.082+0000 7f3624ff9640 1 -- 172.21.15.67:0/3912303001 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562767a9c4e0 con 0x562767bb8e40 2024-07-31T15:04:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.082+0000 7f3624ff9640 10 client.27815 handle_command_reply: tid=0 2024-07-31T15:04:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.082+0000 7f364a3f6740 1 client.27815 shutdown 2024-07-31T15:04:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.082+0000 7f364a3f6740 1 -- 172.21.15.67:0/3912303001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677602f0 msgr2=0x562767b282b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.082+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3912303001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677602f0 0x562767b282b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f362c0aa900 tx=0x7f362c000bc0 comp rx=0 tx=0).stop 2024-07-31T15:04:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.082+0000 7f364a3f6740 1 -- 172.21.15.67:0/3912303001 shutdown_connections 2024-07-31T15:04:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.082+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3912303001 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562767bb8e40 0x562767bbb260 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.082+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3912303001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56276771d1c0 0x562767b28d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.082+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3912303001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627677684d0 0x562767b287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.082+0000 7f364a3f6740 1 --2- 172.21.15.67:0/3912303001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627677602f0 0x562767b282b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.082+0000 7f364a3f6740 1 -- 172.21.15.67:0/3912303001 >> 172.21.15.67:0/3912303001 conn(0x56276777d300 msgr2=0x562767b40370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.082+0000 7f3624ff9640 0 client.27815 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.083+0000 7f364a3f6740 1 -- 172.21.15.67:0/3912303001 shutdown_connections 2024-07-31T15:04:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.083+0000 7f364a3f6740 1 -- 172.21.15.67:0/3912303001 wait complete. 2024-07-31T15:04:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.083+0000 7f364a3f6740 20 client.27815 trim_cache size 0 max 16384 2024-07-31T15:04:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:04:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:04:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:04:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:04:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.088+0000 7f3643500640 1 -- 172.21.15.67:0/4022253333 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f36280b3e70 msgr2=0x7f36280b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.088+0000 7f3643500640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f36280b3e70 0x7f36280b6380 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f363400a7c0 tx=0x7f363400d040 comp rx=0 tx=0).stop 2024-07-31T15:04:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.088+0000 7f3643500640 1 -- 172.21.15.67:0/4022253333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f363c10a090 msgr2=0x7f363c1a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.088+0000 7f3643500640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f363c10a090 0x7f363c1a2580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f362c00c920 tx=0x7f362c00cde0 comp rx=0 tx=0).stop 2024-07-31T15:04:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.088+0000 7f3643500640 1 -- 172.21.15.67:0/4022253333 shutdown_connections 2024-07-31T15:04:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.088+0000 7f3643500640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f36280b3e70 0x7f36280b6380 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.088+0000 7f3643500640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c1101d0 0x7f363c1a7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.088+0000 7f3643500640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f363c10f880 0x7f363c1a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.088+0000 7f3643500640 1 --2- 172.21.15.67:0/4022253333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f363c10a090 0x7f363c1a2580 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.088+0000 7f3643500640 1 -- 172.21.15.67:0/4022253333 >> 172.21.15.67:0/4022253333 conn(0x7f363c0707c0 msgr2=0x7f363c1108d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.088+0000 7f3643500640 1 -- 172.21.15.67:0/4022253333 shutdown_connections 2024-07-31T15:04:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:02.089+0000 7f3643500640 1 -- 172.21.15.67:0/4022253333 wait complete. 2024-07-31T15:04:02.105 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/ntp.o 2024-07-31T15:04:02.115 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/srat.o 2024-07-31T15:04:02.500 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/intel_epb.o 2024-07-31T15:04:02.552 INFO:tasks.workunit.client.0.smithi067.stdout: AR crypto/asymmetric_keys/built-in.a 2024-07-31T15:04:02.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:02 smithi067 ceph-mon[32750]: pgmap v809: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 672 KiB/s wr, 4.01k op/s 2024-07-31T15:04:02.594 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/api.o 2024-07-31T15:04:02.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:02 smithi073 ceph-mon[36714]: pgmap v809: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 672 KiB/s wr, 4.01k op/s 2024-07-31T15:04:02.692 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/pkeys.o 2024-07-31T15:04:02.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:02 smithi077 ceph-mon[37327]: pgmap v809: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 672 KiB/s wr, 4.01k op/s 2024-07-31T15:04:02.818 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/clocksource.o 2024-07-31T15:04:02.823 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/init-mm.o 2024-07-31T15:04:02.844 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/fops.o 2024-07-31T15:04:02.908 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/amd.o 2024-07-31T15:04:03.159 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/jiffies.o 2024-07-31T15:04:03.275 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/kaslr.o 2024-07-31T15:04:03.425 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/cipher.o 2024-07-31T15:04:03.653 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/timer_list.o 2024-07-31T15:04:03.676 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/memblock.o 2024-07-31T15:04:03.893 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/hygon.o 2024-07-31T15:04:03.967 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/centaur.o 2024-07-31T15:04:03.988 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/bio.o 2024-07-31T15:04:04.283 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/pti.o 2024-07-31T15:04:04.333 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/compress.o 2024-07-31T15:04:04.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:04 smithi067 ceph-mon[32750]: osdmap e111: 12 total, 12 up, 12 in 2024-07-31T15:04:04.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:04 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:04:04.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:04 smithi067 ceph-mon[32750]: pgmap v811: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.0 MiB/s wr, 3.87k op/s 2024-07-31T15:04:04.628 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/timeconv.o 2024-07-31T15:04:04.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:04 smithi073 ceph-mon[36714]: osdmap e111: 12 total, 12 up, 12 in 2024-07-31T15:04:04.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:04 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:04:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:04 smithi073 ceph-mon[36714]: pgmap v811: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.0 MiB/s wr, 3.87k op/s 2024-07-31T15:04:04.693 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/algapi.o 2024-07-31T15:04:04.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:04 smithi077 ceph-mon[37327]: osdmap e111: 12 total, 12 up, 12 in 2024-07-31T15:04:04.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:04 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:04:04.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:04 smithi077 ceph-mon[37327]: pgmap v811: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.0 MiB/s wr, 3.87k op/s 2024-07-31T15:04:04.741 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/ioctl.o 2024-07-31T15:04:04.887 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/zhaoxin.o 2024-07-31T15:04:05.368 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/timecounter.o 2024-07-31T15:04:05.423 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/scatterwalk.o 2024-07-31T15:04:05.617 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/alarmtimer.o 2024-07-31T15:04:05.699 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/mm/built-in.a 2024-07-31T15:04:05.854 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/perfctr-watchdog.o 2024-07-31T15:04:05.904 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/madvise.o 2024-07-31T15:04:05.925 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/crypto/built-in.a 2024-07-31T15:04:06.076 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/ia32/audit.o 2024-07-31T15:04:06.222 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/elevator.o 2024-07-31T15:04:06.512 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/proc.o 2024-07-31T15:04:06.681 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/ia32/built-in.a 2024-07-31T15:04:06.711 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/aead.o 2024-07-31T15:04:06.775 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/page_io.o 2024-07-31T15:04:06.835 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/vmware.o 2024-07-31T15:04:07.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:06 smithi073 ceph-mon[36714]: pgmap v812: 129 pgs: 8 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 98 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 3.15k op/s 2024-07-31T15:04:07.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:04:07.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:04:07.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:06 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:04:07.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:06 smithi077 ceph-mon[37327]: pgmap v812: 129 pgs: 8 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 98 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 3.15k op/s 2024-07-31T15:04:07.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:04:07.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:04:07.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:06 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:04:07.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:06 smithi067 ceph-mon[32750]: pgmap v812: 129 pgs: 8 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 98 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 3.15k op/s 2024-07-31T15:04:07.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:04:07.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:04:07.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:06 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:04:07.368 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/posix-timers.o 2024-07-31T15:04:07.443 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/geniv.o 2024-07-31T15:04:07.909 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/mballoc.o 2024-07-31T15:04:08.002 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-core.o 2024-07-31T15:04:08.079 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-sysfs.o 2024-07-31T15:04:08.310 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/hypervisor.o 2024-07-31T15:04:08.605 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-flush.o 2024-07-31T15:04:08.677 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/posix-cpu-timers.o 2024-07-31T15:04:08.855 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/swap_state.o 2024-07-31T15:04:09.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:08 smithi073 ceph-mon[36714]: pgmap v813: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 1.77k op/s 2024-07-31T15:04:09.145 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mshyperv.o 2024-07-31T15:04:09.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:08 smithi077 ceph-mon[37327]: pgmap v813: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 1.77k op/s 2024-07-31T15:04:09.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:08 smithi067 ceph-mon[32750]: pgmap v813: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 1.77k op/s 2024-07-31T15:04:09.444 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/skcipher.o 2024-07-31T15:04:09.669 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-settings.o 2024-07-31T15:04:09.702 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/swapfile.o 2024-07-31T15:04:10.467 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/posix-clock.o 2024-07-31T15:04:10.700 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/capflags.o 2024-07-31T15:04:10.745 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-ioc.o 2024-07-31T15:04:10.830 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/kernel/cpu/built-in.a 2024-07-31T15:04:10.872 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/platform/atom/built-in.a 2024-07-31T15:04:10.972 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/platform/ce4100/built-in.a 2024-07-31T15:04:10.987 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/process_64.o 2024-07-31T15:04:11.094 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/platform/efi/memmap.o 2024-07-31T15:04:11.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:10 smithi073 ceph-mon[36714]: pgmap v814: 129 pgs: 18 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 62 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 822 op/s 2024-07-31T15:04:11.180 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/itimer.o 2024-07-31T15:04:11.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:10 smithi077 ceph-mon[37327]: pgmap v814: 129 pgs: 18 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 62 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 822 op/s 2024-07-31T15:04:11.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:10 smithi067 ceph-mon[32750]: pgmap v814: 129 pgs: 18 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 62 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 822 op/s 2024-07-31T15:04:11.376 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/seqiv.o 2024-07-31T15:04:11.621 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/signal.o 2024-07-31T15:04:11.832 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-map.o 2024-07-31T15:04:12.305 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/swap_slots.o 2024-07-31T15:04:12.477 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/echainiv.o 2024-07-31T15:04:12.479 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/clockevents.o 2024-07-31T15:04:12.498 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/platform/efi/quirks.o 2024-07-31T15:04:12.658 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/platform/efi/efi.o 2024-07-31T15:04:12.934 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/signal_64.o 2024-07-31T15:04:13.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:12 smithi073 ceph-mon[36714]: pgmap v815: 129 pgs: 18 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 62 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.1 MiB/s wr, 537 op/s 2024-07-31T15:04:13.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:12 smithi077 ceph-mon[37327]: pgmap v815: 129 pgs: 18 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 62 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.1 MiB/s wr, 537 op/s 2024-07-31T15:04:13.304 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-merge.o 2024-07-31T15:04:13.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:12 smithi067 ceph-mon[32750]: pgmap v815: 129 pgs: 18 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 62 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.1 MiB/s wr, 537 op/s 2024-07-31T15:04:13.356 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/tick-common.o 2024-07-31T15:04:13.380 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/dmapool.o 2024-07-31T15:04:13.683 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/ahash.o 2024-07-31T15:04:13.871 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/migrate.o 2024-07-31T15:04:13.971 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/platform/efi/efi_64.o 2024-07-31T15:04:14.212 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/mmp.o 2024-07-31T15:04:14.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:13 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:04:14.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:13 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:04:14.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:13 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:04:14.265 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/traps.o 2024-07-31T15:04:14.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:13 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:04:14.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:13 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:04:14.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:13 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:04:14.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:13 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:04:14.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:13 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:04:14.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:13 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:04:14.506 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/hugetlb.o 2024-07-31T15:04:14.631 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/tick-broadcast.o 2024-07-31T15:04:14.970 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/shash.o 2024-07-31T15:04:15.139 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/move_extent.o 2024-07-31T15:04:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:14 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:04:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:14 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:04:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:14 smithi077 ceph-mon[37327]: pgmap v816: 129 pgs: 15 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 71 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 850 KiB/s wr, 207 op/s 2024-07-31T15:04:15.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:14 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:04:15.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:14 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:04:15.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:14 smithi067 ceph-mon[32750]: pgmap v816: 129 pgs: 15 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 71 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 850 KiB/s wr, 207 op/s 2024-07-31T15:04:15.311 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-timeout.o 2024-07-31T15:04:15.357 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/namei.o 2024-07-31T15:04:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.357+0000 7f072c9ea640 1 -- 172.21.15.67:0/2906123742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0728111650 msgr2=0x7f0728109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.357+0000 7f072c9ea640 1 --2- 172.21.15.67:0/2906123742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0728111650 0x7f0728109df0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f071c01a910 tx=0x7f071c03b470 comp rx=0 tx=0).stop 2024-07-31T15:04:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.369+0000 7f072c9ea640 1 -- 172.21.15.67:0/2906123742 shutdown_connections 2024-07-31T15:04:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.369+0000 7f072c9ea640 1 --2- 172.21.15.67:0/2906123742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f072811c5d0 0x7f072811e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.369+0000 7f072c9ea640 1 --2- 172.21.15.67:0/2906123742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f072810a330 0x7f072810a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.369+0000 7f072c9ea640 1 --2- 172.21.15.67:0/2906123742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0728111650 0x7f0728109df0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.369+0000 7f072c9ea640 1 -- 172.21.15.67:0/2906123742 >> 172.21.15.67:0/2906123742 conn(0x7f072806e320 msgr2=0x7f072806e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.378+0000 7f072c9ea640 1 -- 172.21.15.67:0/2906123742 shutdown_connections 2024-07-31T15:04:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.378+0000 7f072c9ea640 1 -- 172.21.15.67:0/2906123742 wait complete. 2024-07-31T15:04:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.379+0000 7f072c9ea640 1 Processor -- start 2024-07-31T15:04:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.380+0000 7f072c9ea640 1 -- start start 2024-07-31T15:04:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.380+0000 7f072c9ea640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f072810a330 0x7f072811c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.380+0000 7f072c9ea640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0728111650 0x7f07281183a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.380+0000 7f072c9ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f072811c5d0 0x7f0728118b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.380+0000 7f072c9ea640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0728114740 con 0x7f072810a330 2024-07-31T15:04:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.380+0000 7f072c9ea640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f07281145c0 con 0x7f0728111650 2024-07-31T15:04:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.380+0000 7f072c9ea640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07281148c0 con 0x7f072811c5d0 2024-07-31T15:04:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.381+0000 7f072759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f072811c5d0 0x7f0728118b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.381+0000 7f072759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f072811c5d0 0x7f0728118b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:47034/0 (socket says 172.21.15.67:47034) 2024-07-31T15:04:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.381+0000 7f072759e640 1 -- 172.21.15.67:0/789636271 learned_addr learned my addr 172.21.15.67:0/789636271 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.381+0000 7f072759e640 1 -- 172.21.15.67:0/789636271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0728111650 msgr2=0x7f07281183a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:04:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.381+0000 7f072759e640 1 --2- 172.21.15.67:0/789636271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0728111650 0x7f07281183a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.381+0000 7f072759e640 1 -- 172.21.15.67:0/789636271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f072810a330 msgr2=0x7f072811c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.381+0000 7f072759e640 1 --2- 172.21.15.67:0/789636271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f072810a330 0x7f072811c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.381+0000 7f072759e640 1 -- 172.21.15.67:0/789636271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07281ce570 con 0x7f072811c5d0 2024-07-31T15:04:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.381+0000 7f072759e640 1 --2- 172.21.15.67:0/789636271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f072811c5d0 0x7f0728118b10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f071800b3f0 tx=0x7f071800b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:15.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:14 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:04:15.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:14 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:04:15.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:14 smithi073 ceph-mon[36714]: pgmap v816: 129 pgs: 15 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 71 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 850 KiB/s wr, 207 op/s 2024-07-31T15:04:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.396+0000 7f0717fff640 1 -- 172.21.15.67:0/789636271 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718015020 con 0x7f072811c5d0 2024-07-31T15:04:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.396+0000 7f0717fff640 1 -- 172.21.15.67:0/789636271 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0718011ce0 con 0x7f072811c5d0 2024-07-31T15:04:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.396+0000 7f0717fff640 1 -- 172.21.15.67:0/789636271 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718010dc0 con 0x7f072811c5d0 2024-07-31T15:04:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.396+0000 7f072c9ea640 1 -- 172.21.15.67:0/789636271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07281ce800 con 0x7f072811c5d0 2024-07-31T15:04:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.396+0000 7f072c9ea640 1 -- 172.21.15.67:0/789636271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07281ced80 con 0x7f072811c5d0 2024-07-31T15:04:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.400+0000 7f07341bf740 1 Processor -- start 2024-07-31T15:04:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.402+0000 7f0717fff640 1 -- 172.21.15.67:0/789636271 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0718018020 con 0x7f072811c5d0 2024-07-31T15:04:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.404+0000 7f0717fff640 1 --2- 172.21.15.67:0/789636271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07100b3ec0 0x7f07100b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.404+0000 7f0717fff640 1 -- 172.21.15.67:0/789636271 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f07180e3e70 con 0x7f072811c5d0 2024-07-31T15:04:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.404+0000 7f07341bf740 1 -- start start 2024-07-31T15:04:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.404+0000 7f07341bf740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4601b0 0x56055a802ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.404+0000 7f07341bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4372f0 0x56055a8033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.405+0000 7f0726d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4601b0 0x56055a802ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.405+0000 7f0726d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4601b0 0x56055a802ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36292/0 (socket says 172.21.15.67:36292) 2024-07-31T15:04:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.405+0000 7f07341bf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a803920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.405+0000 7f07341bf740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56055a7783b0 con 0x56055a4601b0 2024-07-31T15:04:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.405+0000 7f07341bf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56055a7678b0 con 0x56055a43f4d0 2024-07-31T15:04:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.405+0000 7f07341bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56055a76b8f0 con 0x56055a4372f0 2024-07-31T15:04:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.405+0000 7f0726d9d640 1 -- 172.21.15.67:0/885538136 learned_addr learned my addr 172.21.15.67:0/885538136 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.405+0000 7f072659c640 1 --2- 172.21.15.67:0/885538136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4372f0 0x56055a8033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.406+0000 7f072659c640 1 -- 172.21.15.67:0/885538136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 msgr2=0x56055a803920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:04:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.406+0000 7f072659c640 1 --2- 172.21.15.67:0/885538136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a803920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.406+0000 7f072659c640 1 -- 172.21.15.67:0/885538136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4601b0 msgr2=0x56055a802ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.406+0000 7f072659c640 1 --2- 172.21.15.67:0/885538136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4601b0 0x56055a802ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.406+0000 7f072659c640 1 -- 172.21.15.67:0/885538136 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055a70e9a0 con 0x56055a4372f0 2024-07-31T15:04:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.406+0000 7f072659c640 1 --2- 172.21.15.67:0/885538136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4372f0 0x56055a8033e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f072806c140 tx=0x7f072000d700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5721bc33f3106301 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.408+0000 7f072659c640 1 --2- 172.21.15.67:0/789636271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07100b3ec0 0x7f07100b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.417+0000 7f07157fa640 1 -- 172.21.15.67:0/885538136 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720004d60 con 0x56055a4372f0 2024-07-31T15:04:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.417+0000 7f072659c640 1 --2- 172.21.15.67:0/789636271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07100b3ec0 0x7f07100b63d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0720016000 tx=0x7f072000fc70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.417+0000 7f07157fa640 1 -- 172.21.15.67:0/885538136 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0720006580 con 0x56055a4372f0 2024-07-31T15:04:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.417+0000 7f07157fa640 1 -- 172.21.15.67:0/885538136 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072000f840 con 0x56055a4372f0 2024-07-31T15:04:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.419+0000 7f07341bf740 1 -- 172.21.15.67:0/885538136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4372f0 msgr2=0x56055a8033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.419+0000 7f07341bf740 1 --2- 172.21.15.67:0/885538136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4372f0 0x56055a8033e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f072806c140 tx=0x7f072000d700 comp rx=0 tx=0).stop 2024-07-31T15:04:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.419+0000 7f07341bf740 1 -- 172.21.15.67:0/885538136 shutdown_connections 2024-07-31T15:04:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.419+0000 7f07341bf740 1 --2- 172.21.15.67:0/885538136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a803920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.419+0000 7f07341bf740 1 --2- 172.21.15.67:0/885538136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4372f0 0x56055a8033e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.419+0000 7f07341bf740 1 --2- 172.21.15.67:0/885538136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4601b0 0x56055a802ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.419+0000 7f07341bf740 1 -- 172.21.15.67:0/885538136 >> 172.21.15.67:0/885538136 conn(0x56055a454300 msgr2=0x56055a800d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.423+0000 7f07341bf740 1 -- 172.21.15.67:0/885538136 shutdown_connections 2024-07-31T15:04:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.423+0000 7f07341bf740 1 -- 172.21.15.67:0/885538136 wait complete. 2024-07-31T15:04:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.423+0000 7f07341bf740 1 -- start start 2024-07-31T15:04:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.423+0000 7f07341bf740 1 Processor -- start 2024-07-31T15:04:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.424+0000 7f07341bf740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4372f0 0x56055a80e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.424+0000 7f07341bf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a80ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.424+0000 7f07341bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4601b0 0x56055a81a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.424+0000 7f07341bf740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56055a6a1ab0 con 0x56055a4372f0 2024-07-31T15:04:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.424+0000 7f07341bf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56055a6efcf0 con 0x56055a43f4d0 2024-07-31T15:04:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.424+0000 7f07341bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56055a811620 con 0x56055a4601b0 2024-07-31T15:04:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.424+0000 7f07341bf740 10 client.0 _ll_register_callbacks cb 0x56055a7619f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.424+0000 7f072659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a80ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.424+0000 7f072659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a80ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:58410/0 (socket says 172.21.15.67:58410) 2024-07-31T15:04:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.424+0000 7f072659c640 1 -- 172.21.15.67:0/2746700085 learned_addr learned my addr 172.21.15.67:0/2746700085 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.424+0000 7f072759e640 1 --2- 172.21.15.67:0/2746700085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4601b0 0x56055a81a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.425+0000 7f072659c640 1 -- 172.21.15.67:0/2746700085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4601b0 msgr2=0x56055a81a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.425+0000 7f072659c640 1 --2- 172.21.15.67:0/2746700085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4601b0 0x56055a81a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.425+0000 7f072659c640 1 -- 172.21.15.67:0/2746700085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4372f0 msgr2=0x56055a80e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.425+0000 7f072659c640 1 --2- 172.21.15.67:0/2746700085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4372f0 0x56055a80e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.425+0000 7f072659c640 1 -- 172.21.15.67:0/2746700085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055a4cb790 con 0x56055a43f4d0 2024-07-31T15:04:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.425+0000 7f072659c640 1 --2- 172.21.15.67:0/2746700085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a80ef20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f072000df70 tx=0x7f072000faa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0f911d71c23f676 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.425+0000 7f070e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:04:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.439+0000 7f070e7fc640 1 -- 172.21.15.67:0/2746700085 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720030060 con 0x56055a43f4d0 2024-07-31T15:04:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.439+0000 7f070e7fc640 1 -- 172.21.15.67:0/2746700085 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f0720004b60 con 0x56055a43f4d0 2024-07-31T15:04:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.443+0000 7f07341bf740 1 -- 172.21.15.67:0/2746700085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0720006580 con 0x56055a43f4d0 2024-07-31T15:04:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.452+0000 7f070e7fc640 1 -- 172.21.15.67:0/2746700085 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f0720004d60 con 0x56055a43f4d0 2024-07-31T15:04:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.452+0000 7f07341bf740 10 client.27818 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.452+0000 7f07341bf740 1 -- 172.21.15.67:0/2746700085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x56055a544620 con 0x56055a43f4d0 2024-07-31T15:04:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.462+0000 7f070e7fc640 1 -- 172.21.15.67:0/2746700085 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0720006580 con 0x56055a43f4d0 2024-07-31T15:04:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.462+0000 7f070e7fc640 1 -- 172.21.15.67:0/2746700085 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f072002a030 con 0x56055a43f4d0 2024-07-31T15:04:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.462+0000 7f07341bf740 10 client.27818 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.462+0000 7f07341bf740 10 client.27818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.462+0000 7f07341bf740 20 client.27818 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.462+0000 7f07341bf740 1 --2- 172.21.15.67:0/2746700085 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56055a41ed80 0x56055a8208a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.462+0000 7f07341bf740 4 client.27818 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:04:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.462+0000 7f07341bf740 1 -- 172.21.15.67:0/2746700085 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56055a77a300 con 0x56055a41ed80 2024-07-31T15:04:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.464+0000 7f0726d9d640 1 --2- 172.21.15.67:0/2746700085 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56055a41ed80 0x56055a8208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.464+0000 7f0726d9d640 1 --2- 172.21.15.67:0/2746700085 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56055a41ed80 0x56055a8208a0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbf08b23aabf1680 server_cookie=a435873cdbafc29d in_seq=0 out_seq=0 2024-07-31T15:04:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.464+0000 7f070e7fc640 10 client.27818 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.465+0000 7f070e7fc640 1 -- 172.21.15.67:0/2746700085 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56055a77a300 con 0x56055a41ed80 2024-07-31T15:04:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.465+0000 7f070e7fc640 10 client.27818 handle_command_reply: tid=0 2024-07-31T15:04:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.467+0000 7f07341bf740 1 client.27818 shutdown 2024-07-31T15:04:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.473+0000 7f07341bf740 1 -- 172.21.15.67:0/2746700085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 msgr2=0x56055a80ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.473+0000 7f07341bf740 1 --2- 172.21.15.67:0/2746700085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a80ef20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f072000df70 tx=0x7f072000faa0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.474+0000 7f07341bf740 1 -- 172.21.15.67:0/2746700085 shutdown_connections 2024-07-31T15:04:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.474+0000 7f07341bf740 1 --2- 172.21.15.67:0/2746700085 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56055a41ed80 0x56055a8208a0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.475+0000 7f07341bf740 1 --2- 172.21.15.67:0/2746700085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4601b0 0x56055a81a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.475+0000 7f07341bf740 1 --2- 172.21.15.67:0/2746700085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a80ef20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.475+0000 7f07341bf740 1 --2- 172.21.15.67:0/2746700085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4372f0 0x56055a80e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.475+0000 7f07341bf740 1 -- 172.21.15.67:0/2746700085 >> 172.21.15.67:0/2746700085 conn(0x56055a454300 msgr2=0x56055a803ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.475+0000 7f070e7fc640 0 client.27818 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.482+0000 7f07341bf740 1 -- 172.21.15.67:0/2746700085 shutdown_connections 2024-07-31T15:04:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.482+0000 7f07341bf740 1 -- 172.21.15.67:0/2746700085 wait complete. 2024-07-31T15:04:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.486+0000 7f07341bf740 20 client.27818 trim_cache size 0 max 16384 2024-07-31T15:04:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.538+0000 7f07341bf740 1 Processor -- start 2024-07-31T15:04:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.542+0000 7f07341bf740 1 -- start start 2024-07-31T15:04:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.542+0000 7f07341bf740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4372f0 0x56055a7ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.543+0000 7f07341bf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a80ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.543+0000 7f07341bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4601b0 0x56055a8045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.543+0000 7f07341bf740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f070800ffb0 con 0x56055a4372f0 2024-07-31T15:04:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.543+0000 7f07341bf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0708011af0 con 0x56055a43f4d0 2024-07-31T15:04:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.543+0000 7f07341bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0708010680 con 0x56055a4601b0 2024-07-31T15:04:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.543+0000 7f072659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a80ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.543+0000 7f072659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a80ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:58416/0 (socket says 172.21.15.67:58416) 2024-07-31T15:04:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.543+0000 7f072659c640 1 -- 172.21.15.67:0/2551939674 learned_addr learned my addr 172.21.15.67:0/2551939674 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.544+0000 7f072759e640 1 --2- 172.21.15.67:0/2551939674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4601b0 0x56055a8045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.544+0000 7f072659c640 1 -- 172.21.15.67:0/2551939674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4601b0 msgr2=0x56055a8045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.544+0000 7f072659c640 1 --2- 172.21.15.67:0/2551939674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4601b0 0x56055a8045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.544+0000 7f072659c640 1 -- 172.21.15.67:0/2551939674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4372f0 msgr2=0x56055a7ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.544+0000 7f072659c640 1 --2- 172.21.15.67:0/2551939674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4372f0 0x56055a7ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.544+0000 7f072659c640 1 -- 172.21.15.67:0/2551939674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055a80fa90 con 0x56055a43f4d0 2024-07-31T15:04:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.544+0000 7f072759e640 1 --2- 172.21.15.67:0/2551939674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4601b0 0x56055a8045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:04:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.544+0000 7f072659c640 1 --2- 172.21.15.67:0/2551939674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a80ecb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f072000d740 tx=0x7f0720006bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9ba1cb32c7f9636 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.553+0000 7f0714ff9640 1 -- 172.21.15.67:0/2551939674 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720030050 con 0x56055a43f4d0 2024-07-31T15:04:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.553+0000 7f0714ff9640 1 -- 172.21.15.67:0/2551939674 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0720006560 con 0x56055a43f4d0 2024-07-31T15:04:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.553+0000 7f0714ff9640 1 -- 172.21.15.67:0/2551939674 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072003a080 con 0x56055a43f4d0 2024-07-31T15:04:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.555+0000 7f07341bf740 1 -- 172.21.15.67:0/2551939674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 msgr2=0x56055a80ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.555+0000 7f07341bf740 1 --2- 172.21.15.67:0/2551939674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a80ecb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f072000d740 tx=0x7f0720006bd0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.566+0000 7f07341bf740 1 -- 172.21.15.67:0/2551939674 shutdown_connections 2024-07-31T15:04:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.566+0000 7f07341bf740 1 --2- 172.21.15.67:0/2551939674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4601b0 0x56055a8045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.566+0000 7f07341bf740 1 --2- 172.21.15.67:0/2551939674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a80ecb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.566+0000 7f07341bf740 1 --2- 172.21.15.67:0/2551939674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4372f0 0x56055a7ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.566+0000 7f07341bf740 1 -- 172.21.15.67:0/2551939674 >> 172.21.15.67:0/2551939674 conn(0x56055a454300 msgr2=0x56055a803ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.572+0000 7f07341bf740 1 -- 172.21.15.67:0/2551939674 shutdown_connections 2024-07-31T15:04:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.572+0000 7f07341bf740 1 -- 172.21.15.67:0/2551939674 wait complete. 2024-07-31T15:04:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.572+0000 7f07341bf740 1 -- start start 2024-07-31T15:04:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.573+0000 7f07341bf740 1 Processor -- start 2024-07-31T15:04:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.573+0000 7f07341bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4372f0 0x56055a816900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.574+0000 7f07341bf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a816e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.574+0000 7f07341bf740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4601b0 0x56055a8138d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.574+0000 7f07341bf740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0708011420 con 0x56055a4601b0 2024-07-31T15:04:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.574+0000 7f07341bf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0708010d50 con 0x56055a43f4d0 2024-07-31T15:04:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.574+0000 7f07341bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f070800dfb0 con 0x56055a4372f0 2024-07-31T15:04:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.574+0000 7f07341bf740 10 client.0 _ll_register_callbacks cb 0x7f072000bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.579+0000 7f072659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a816e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.579+0000 7f0726d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4372f0 0x56055a816900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.579+0000 7f072659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a816e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:58418/0 (socket says 172.21.15.67:58418) 2024-07-31T15:04:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.579+0000 7f072659c640 1 -- 172.21.15.67:0/2707930121 learned_addr learned my addr 172.21.15.67:0/2707930121 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.579+0000 7f0726d9d640 1 -- 172.21.15.67:0/2707930121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 msgr2=0x56055a816e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.579+0000 7f0726d9d640 1 --2- 172.21.15.67:0/2707930121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a816e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.579+0000 7f0726d9d640 1 -- 172.21.15.67:0/2707930121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4601b0 msgr2=0x56055a8138d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.579+0000 7f0726d9d640 1 --2- 172.21.15.67:0/2707930121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4601b0 0x56055a8138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.579+0000 7f0726d9d640 1 -- 172.21.15.67:0/2707930121 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055a813e10 con 0x56055a4372f0 2024-07-31T15:04:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.580+0000 7f0726d9d640 1 --2- 172.21.15.67:0/2707930121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4372f0 0x56055a816900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f071c01a910 tx=0x7f071c01a1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=998c07a88a67d940 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.582+0000 7f070e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:04:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.582+0000 7f070e7fc640 1 -- 172.21.15.67:0/2707930121 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c046070 con 0x56055a4372f0 2024-07-31T15:04:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.582+0000 7f07341bf740 1 -- 172.21.15.67:0/2707930121 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0720006560 con 0x56055a4372f0 2024-07-31T15:04:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.583+0000 7f070e7fc640 1 -- 172.21.15.67:0/2707930121 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f071c004e20 con 0x56055a4372f0 2024-07-31T15:04:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.591+0000 7f070e7fc640 1 -- 172.21.15.67:0/2707930121 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f071c053050 con 0x56055a4372f0 2024-07-31T15:04:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.594+0000 7f07341bf740 10 client.27772 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.594+0000 7f07341bf740 1 -- 172.21.15.67:0/2707930121 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x56055a887750 con 0x56055a4372f0 2024-07-31T15:04:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.594+0000 7f070e7fc640 1 -- 172.21.15.67:0/2707930121 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f071c0054d0 con 0x56055a4372f0 2024-07-31T15:04:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.594+0000 7f070e7fc640 1 -- 172.21.15.67:0/2707930121 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f071c016070 con 0x56055a4372f0 2024-07-31T15:04:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.599+0000 7f07341bf740 10 client.27772 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.599+0000 7f07341bf740 10 client.27772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.599+0000 7f07341bf740 20 client.27772 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.599+0000 7f07341bf740 1 --2- 172.21.15.67:0/2707930121 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56055a3f41c0 0x56055a889f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.599+0000 7f07341bf740 4 client.27772 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:04:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.599+0000 7f07341bf740 1 -- 172.21.15.67:0/2707930121 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0720006560 con 0x56055a3f41c0 2024-07-31T15:04:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.606+0000 7f072659c640 1 --2- 172.21.15.67:0/2707930121 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56055a3f41c0 0x56055a889f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.606+0000 7f072659c640 1 --2- 172.21.15.67:0/2707930121 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56055a3f41c0 0x56055a889f10 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3675d2070889062f server_cookie=836e435f6a5b68d1 in_seq=0 out_seq=0 2024-07-31T15:04:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:15.607+0000 7f070e7fc640 10 client.27772 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:15.649 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/tick-broadcast-hrtimer.o 2024-07-31T15:04:16.276 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-lib.o 2024-07-31T15:04:16.293 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/platform/efi/efi_stub_64.o 2024-07-31T15:04:16.295 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/idt.o 2024-07-31T15:04:16.765 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/platform/efi/efi_thunk_64.o 2024-07-31T15:04:16.768 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/tick-oneshot.o 2024-07-31T15:04:16.770 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq.o 2024-07-31T15:04:16.775 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/akcipher.o 2024-07-31T15:04:16.846 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/platform/efi/runtime-map.o 2024-07-31T15:04:17.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:16 smithi077 ceph-mon[37327]: pgmap v817: 129 pgs: 13 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 83 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 771 KiB/s wr, 189 op/s 2024-07-31T15:04:17.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:16 smithi067 ceph-mon[32750]: pgmap v817: 129 pgs: 13 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 83 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 771 KiB/s wr, 189 op/s 2024-07-31T15:04:17.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:16 smithi073 ceph-mon[36714]: pgmap v817: 129 pgs: 13 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 83 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 771 KiB/s wr, 189 op/s 2024-07-31T15:04:17.430 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/tick-sched.o 2024-07-31T15:04:17.440 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/irq.o 2024-07-31T15:04:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.711+0000 7f070e7fc640 1 -- 172.21.15.67:0/2707930121 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f0720006560 con 0x56055a3f41c0 2024-07-31T15:04:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.711+0000 7f070e7fc640 10 client.27772 handle_command_reply: tid=0 2024-07-31T15:04:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.711+0000 7f07341bf740 1 client.27772 shutdown 2024-07-31T15:04:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.712+0000 7f07341bf740 1 -- 172.21.15.67:0/2707930121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4372f0 msgr2=0x56055a816900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.712+0000 7f07341bf740 1 --2- 172.21.15.67:0/2707930121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4372f0 0x56055a816900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f071c01a910 tx=0x7f071c01a1b0 comp rx=0 tx=0).stop 2024-07-31T15:04:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.712+0000 7f07341bf740 1 -- 172.21.15.67:0/2707930121 shutdown_connections 2024-07-31T15:04:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.712+0000 7f07341bf740 1 --2- 172.21.15.67:0/2707930121 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56055a3f41c0 0x56055a889f10 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.712+0000 7f07341bf740 1 --2- 172.21.15.67:0/2707930121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56055a4601b0 0x56055a8138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.712+0000 7f07341bf740 1 --2- 172.21.15.67:0/2707930121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055a43f4d0 0x56055a816e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.712+0000 7f07341bf740 1 --2- 172.21.15.67:0/2707930121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56055a4372f0 0x56055a816900 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.712+0000 7f07341bf740 1 -- 172.21.15.67:0/2707930121 >> 172.21.15.67:0/2707930121 conn(0x56055a454300 msgr2=0x56055a804840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.712+0000 7f070e7fc640 0 client.27772 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.713+0000 7f07341bf740 1 -- 172.21.15.67:0/2707930121 shutdown_connections 2024-07-31T15:04:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.713+0000 7f07341bf740 1 -- 172.21.15.67:0/2707930121 wait complete. 2024-07-31T15:04:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.713+0000 7f07341bf740 20 client.27772 trim_cache size 0 max 16384 2024-07-31T15:04:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:04:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:04:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:04:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:04:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.732+0000 7f072c9ea640 1 -- 172.21.15.67:0/789636271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07100b3ec0 msgr2=0x7f07100b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.732+0000 7f072c9ea640 1 --2- 172.21.15.67:0/789636271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07100b3ec0 0x7f07100b63d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0720016000 tx=0x7f072000fc70 comp rx=0 tx=0).stop 2024-07-31T15:04:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.732+0000 7f072c9ea640 1 -- 172.21.15.67:0/789636271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f072811c5d0 msgr2=0x7f0728118b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.732+0000 7f072c9ea640 1 --2- 172.21.15.67:0/789636271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f072811c5d0 0x7f0728118b10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f071800b3f0 tx=0x7f071800b8b0 comp rx=0 tx=0).stop 2024-07-31T15:04:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.732+0000 7f072c9ea640 1 -- 172.21.15.67:0/789636271 shutdown_connections 2024-07-31T15:04:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.732+0000 7f072c9ea640 1 --2- 172.21.15.67:0/789636271 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07100b3ec0 0x7f07100b63d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.732+0000 7f072c9ea640 1 --2- 172.21.15.67:0/789636271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f072811c5d0 0x7f0728118b10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.732+0000 7f072c9ea640 1 --2- 172.21.15.67:0/789636271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0728111650 0x7f07281183a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.732+0000 7f072c9ea640 1 --2- 172.21.15.67:0/789636271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f072810a330 0x7f072811c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.732+0000 7f072c9ea640 1 -- 172.21.15.67:0/789636271 >> 172.21.15.67:0/789636271 conn(0x7f072806e320 msgr2=0x7f072811dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.733+0000 7f072c9ea640 1 -- 172.21.15.67:0/789636271 shutdown_connections 2024-07-31T15:04:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:17.734+0000 7f072c9ea640 1 -- 172.21.15.67:0/789636271 wait complete. 2024-07-31T15:04:17.787 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/sig.o 2024-07-31T15:04:17.793 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/platform/efi/built-in.a 2024-07-31T15:04:17.849 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/irq_64.o 2024-07-31T15:04:17.892 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/platform/geode/built-in.a 2024-07-31T15:04:17.979 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/platform/iris/built-in.a 2024-07-31T15:04:18.064 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/platform/intel/iosf_mbi.o 2024-07-31T15:04:18.670 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/vsyscall.o 2024-07-31T15:04:18.674 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/timekeeping_debug.o 2024-07-31T15:04:18.675 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/kpp.o 2024-07-31T15:04:18.685 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/page-io.o 2024-07-31T15:04:18.693 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/hugetlb_vmemmap.o 2024-07-31T15:04:19.103 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/platform/intel/built-in.a 2024-07-31T15:04:19.187 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/namespace.o 2024-07-31T15:04:19.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:18 smithi077 ceph-mon[37327]: pgmap v818: 129 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 799 KiB/s wr, 101 op/s 2024-07-31T15:04:19.244 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/platform/intel-mid/built-in.a 2024-07-31T15:04:19.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:18 smithi067 ceph-mon[32750]: pgmap v818: 129 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 799 KiB/s wr, 101 op/s 2024-07-31T15:04:19.323 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/platform/intel-quark/built-in.a 2024-07-31T15:04:19.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:18 smithi073 ceph-mon[36714]: pgmap v818: 129 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 799 KiB/s wr, 101 op/s 2024-07-31T15:04:19.417 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/platform/olpc/built-in.a 2024-07-31T15:04:19.457 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/platform/scx200/built-in.a 2024-07-31T15:04:19.469 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/platform/ts5500/built-in.a 2024-07-31T15:04:19.503 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/dumpstack_64.o 2024-07-31T15:04:19.514 INFO:tasks.workunit.client.0.smithi067.stdout: ASN.1 crypto/rsapubkey.asn1.[ch] 2024-07-31T15:04:19.518 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/platform/uv/built-in.a 2024-07-31T15:04:19.526 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/time.o 2024-07-31T15:04:19.530 INFO:tasks.workunit.client.0.smithi067.stdout: ASN.1 crypto/rsaprivkey.asn1.[ch] 2024-07-31T15:04:19.554 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/platform/built-in.a 2024-07-31T15:04:19.557 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/rsa.o 2024-07-31T15:04:19.714 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mempolicy.o 2024-07-31T15:04:19.719 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/readpage.o 2024-07-31T15:04:19.781 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/io_uring.o 2024-07-31T15:04:20.015 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/time/built-in.a 2024-07-31T15:04:20.052 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq-tag.o 2024-07-31T15:04:20.053 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-stat.o 2024-07-31T15:04:20.121 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/futex/core.o 2024-07-31T15:04:20.182 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/ioport.o 2024-07-31T15:04:20.273 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/rsa_helper.o 2024-07-31T15:04:20.373 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/rsa-pkcs1pad.o 2024-07-31T15:04:20.651 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/resize.o 2024-07-31T15:04:20.728 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/futex/syscalls.o 2024-07-31T15:04:20.777 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.3 2024-07-31T15:04:20.777 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:04:20.840 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/dumpstack.o 2024-07-31T15:04:20.953 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq-sysfs.o 2024-07-31T15:04:21.026 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/cgroup.o 2024-07-31T15:04:21.157 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/acompress.o 2024-07-31T15:04:21.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:20 smithi077 ceph-mon[37327]: pgmap v819: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-07-31T15:04:21.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:20 smithi067 ceph-mon[32750]: pgmap v819: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-07-31T15:04:21.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:20 smithi073 ceph-mon[36714]: pgmap v819: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-07-31T15:04:21.495 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:04:21.531 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/sparse.o 2024-07-31T15:04:21.563 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/futex/pi.o 2024-07-31T15:04:21.576 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/nmi.o 2024-07-31T15:04:21.659 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq-cpumap.o 2024-07-31T15:04:21.913 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/scompress.o 2024-07-31T15:04:22.142 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/super.o 2024-07-31T15:04:22.148 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/futex/requeue.o 2024-07-31T15:04:22.277 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq-sched.o 2024-07-31T15:04:22.288 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:04:22.334 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/sparse-vmemmap.o 2024-07-31T15:04:22.411 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/ldt.o 2024-07-31T15:04:22.659 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/futex/waitwake.o 2024-07-31T15:04:22.663 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/algboss.o 2024-07-31T15:04:23.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:22 smithi077 ceph-mon[37327]: pgmap v820: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 914 KiB/s wr, 125 op/s 2024-07-31T15:04:23.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:22 smithi067 ceph-mon[32750]: pgmap v820: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 914 KiB/s wr, 125 op/s 2024-07-31T15:04:23.324 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/ioctl.o 2024-07-31T15:04:23.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:22 smithi073 ceph-mon[36714]: pgmap v820: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 914 KiB/s wr, 125 op/s 2024-07-31T15:04:23.448 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mmu_notifier.o 2024-07-31T15:04:23.495 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/setup.o 2024-07-31T15:04:23.506 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/futex/built-in.a 2024-07-31T15:04:23.535 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/xattr.o 2024-07-31T15:04:23.538 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/testmgr.o 2024-07-31T15:04:23.855 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_clock.o 2024-07-31T15:04:23.911 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:04:24.081 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/cmac.o 2024-07-31T15:04:24.302 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/slub.o 2024-07-31T15:04:24.314 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/genhd.o 2024-07-31T15:04:24.319 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/ring_buffer.o 2024-07-31T15:04:24.347 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/nop.o 2024-07-31T15:04:24.597 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/hmac.o 2024-07-31T15:04:24.610 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/x86_init.o 2024-07-31T15:04:24.872 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/rstat.o 2024-07-31T15:04:24.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.856+0000 7fb1e8c70640 1 -- 172.21.15.67:0/917425819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1e410a410 msgr2=0x7fb1e410a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:24.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.856+0000 7fb1e8c70640 1 --2- 172.21.15.67:0/917425819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1e410a410 0x7fb1e410a820 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009860 tx=0x7fb1d402a380 comp rx=0 tx=0).stop 2024-07-31T15:04:24.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.857+0000 7fb1e8c70640 1 -- 172.21.15.67:0/917425819 shutdown_connections 2024-07-31T15:04:24.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.857+0000 7fb1e8c70640 1 --2- 172.21.15.67:0/917425819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1e411c660 0x7fb1e411eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:24.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.857+0000 7fb1e8c70640 1 --2- 172.21.15.67:0/917425819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1e410a410 0x7fb1e410a820 secure :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009860 tx=0x7fb1d402a380 comp rx=0 tx=0).stop 2024-07-31T15:04:24.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.857+0000 7fb1e8c70640 1 --2- 172.21.15.67:0/917425819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1e4111970 0x7fb1e4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:24.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.857+0000 7fb1e8c70640 1 -- 172.21.15.67:0/917425819 >> 172.21.15.67:0/917425819 conn(0x7fb1e40708f0 msgr2=0x7fb1e4070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:24.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.865+0000 7fb1e8c70640 1 -- 172.21.15.67:0/917425819 shutdown_connections 2024-07-31T15:04:24.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.869+0000 7fb1e8c70640 1 -- 172.21.15.67:0/917425819 wait complete. 2024-07-31T15:04:24.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.870+0000 7fb1e8c70640 1 Processor -- start 2024-07-31T15:04:24.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.870+0000 7fb1e8c70640 1 -- start start 2024-07-31T15:04:24.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.870+0000 7fb1e8c70640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1e4111970 0x7fb1e41182d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:24.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.870+0000 7fb1e8c70640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1e411c660 0x7fb1e4118810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:24.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.870+0000 7fb1e8c70640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1e4118d50 0x7fb1e41ce810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:24.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.870+0000 7fb1e8c70640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41146a0 con 0x7fb1e4111970 2024-07-31T15:04:24.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.870+0000 7fb1e8c70640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4114520 con 0x7fb1e4118d50 2024-07-31T15:04:24.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.870+0000 7fb1e8c70640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4114820 con 0x7fb1e411c660 2024-07-31T15:04:24.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.871+0000 7fb1e2d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1e4118d50 0x7fb1e41ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:24.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.871+0000 7fb1e2d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1e4118d50 0x7fb1e41ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52224/0 (socket says 172.21.15.67:52224) 2024-07-31T15:04:24.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.871+0000 7fb1e2d9d640 1 -- 172.21.15.67:0/523725513 learned_addr learned my addr 172.21.15.67:0/523725513 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:24.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.871+0000 7fb1e2d9d640 1 -- 172.21.15.67:0/523725513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1e411c660 msgr2=0x7fb1e4118810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:04:24.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.871+0000 7fb1e2d9d640 1 --2- 172.21.15.67:0/523725513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1e411c660 0x7fb1e4118810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:24.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.871+0000 7fb1e2d9d640 1 -- 172.21.15.67:0/523725513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1e4111970 msgr2=0x7fb1e41182d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:04:24.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.871+0000 7fb1e2d9d640 1 --2- 172.21.15.67:0/523725513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1e4111970 0x7fb1e41182d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:24.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.871+0000 7fb1e2d9d640 1 -- 172.21.15.67:0/523725513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e41ced50 con 0x7fb1e4118d50 2024-07-31T15:04:24.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.872+0000 7fb1e2d9d640 1 --2- 172.21.15.67:0/523725513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1e4118d50 0x7fb1e41ce810 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800c970 tx=0x7fb1d800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:24.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.872+0000 7fb1d37fe640 1 -- 172.21.15.67:0/523725513 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8007c70 con 0x7fb1e4118d50 2024-07-31T15:04:24.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.872+0000 7fb1e8c70640 1 -- 172.21.15.67:0/523725513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e41cef80 con 0x7fb1e4118d50 2024-07-31T15:04:24.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.872+0000 7fb1e8c70640 1 -- 172.21.15.67:0/523725513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e41cf440 con 0x7fb1e4118d50 2024-07-31T15:04:24.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.872+0000 7fb1d37fe640 1 -- 172.21.15.67:0/523725513 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1d8007e10 con 0x7fb1e4118d50 2024-07-31T15:04:24.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.872+0000 7fb1d37fe640 1 -- 172.21.15.67:0/523725513 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d800f450 con 0x7fb1e4118d50 2024-07-31T15:04:24.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.875+0000 7fb1e8c70640 1 -- 172.21.15.67:0/523725513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c40052c0 con 0x7fb1e4118d50 2024-07-31T15:04:24.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.878+0000 7fb1d37fe640 1 -- 172.21.15.67:0/523725513 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb1d80046c0 con 0x7fb1e4118d50 2024-07-31T15:04:24.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.879+0000 7fb1d37fe640 1 --2- 172.21.15.67:0/523725513 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb1cc0b3f50 0x7fb1cc0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:24.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.884+0000 7fb1e259c640 1 --2- 172.21.15.67:0/523725513 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb1cc0b3f50 0x7fb1cc0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:24.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.885+0000 7fb1d37fe640 1 -- 172.21.15.67:0/523725513 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7fb1d80e42e0 con 0x7fb1e4118d50 2024-07-31T15:04:24.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.885+0000 7fb1e259c640 1 --2- 172.21.15.67:0/523725513 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb1cc0b3f50 0x7fb1cc0b6460 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb1e411bd40 tx=0x7fb1dc009250 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:24.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:24.885+0000 7fb1d37fe640 1 -- 172.21.15.67:0/523725513 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb1d8014030 con 0x7fb1e4118d50 2024-07-31T15:04:25.040 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:24 smithi067 ceph-mon[32750]: pgmap v821: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 794 KiB/s wr, 122 op/s 2024-07-31T15:04:25.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:24 smithi073 ceph-mon[36714]: pgmap v821: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 794 KiB/s wr, 122 op/s 2024-07-31T15:04:25.404 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:04:25.404 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":63,"btime":"2024-07-31T15:03:07:099477+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:03:06.671820+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:04:25.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:25.394+0000 7fb1e8c70640 1 -- 172.21.15.67:0/523725513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1c40055b0 con 0x7fb1e4118d50 2024-07-31T15:04:25.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:25.397+0000 7fb1d37fe640 1 -- 172.21.15.67:0/523725513 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+11959 (secure 0 0 0) 0x7fb1d80ab2d0 con 0x7fb1e4118d50 2024-07-31T15:04:25.405 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 63 2024-07-31T15:04:25.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:25.402+0000 7fb1e8c70640 1 -- 172.21.15.67:0/523725513 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb1cc0b3f50 msgr2=0x7fb1cc0b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:25.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:25.402+0000 7fb1e8c70640 1 --2- 172.21.15.67:0/523725513 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb1cc0b3f50 0x7fb1cc0b6460 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb1e411bd40 tx=0x7fb1dc009250 comp rx=0 tx=0).stop 2024-07-31T15:04:25.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:25.402+0000 7fb1e8c70640 1 -- 172.21.15.67:0/523725513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1e4118d50 msgr2=0x7fb1e41ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:25.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:25.402+0000 7fb1e8c70640 1 --2- 172.21.15.67:0/523725513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1e4118d50 0x7fb1e41ce810 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800c970 tx=0x7fb1d800ce30 comp rx=0 tx=0).stop 2024-07-31T15:04:25.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:25.402+0000 7fb1e8c70640 1 -- 172.21.15.67:0/523725513 shutdown_connections 2024-07-31T15:04:25.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:25.402+0000 7fb1e8c70640 1 --2- 172.21.15.67:0/523725513 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb1cc0b3f50 0x7fb1cc0b6460 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:25.402+0000 7fb1e8c70640 1 --2- 172.21.15.67:0/523725513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1e4118d50 0x7fb1e41ce810 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:25.402+0000 7fb1e8c70640 1 --2- 172.21.15.67:0/523725513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1e411c660 0x7fb1e4118810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:25.402+0000 7fb1e8c70640 1 --2- 172.21.15.67:0/523725513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1e4111970 0x7fb1e41182d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:25.402+0000 7fb1e8c70640 1 -- 172.21.15.67:0/523725513 >> 172.21.15.67:0/523725513 conn(0x7fb1e40708f0 msgr2=0x7fb1e4073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:25.403+0000 7fb1e8c70640 1 -- 172.21.15.67:0/523725513 shutdown_connections 2024-07-31T15:04:25.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:25.407+0000 7fb1e8c70640 1 -- 172.21.15.67:0/523725513 wait complete. 2024-07-31T15:04:25.447 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/fs.o 2024-07-31T15:04:25.449 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/ioprio.o 2024-07-31T15:04:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:24 smithi077 ceph-mon[37327]: pgmap v821: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 794 KiB/s wr, 122 op/s 2024-07-31T15:04:25.651 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/crypto_null.o 2024-07-31T15:04:25.678 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/i8259.o 2024-07-31T15:04:25.943 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/namespace.o 2024-07-31T15:04:26.264 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/md5.o 2024-07-31T15:04:26.280 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/splice.o 2024-07-31T15:04:26.377 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/badblocks.o 2024-07-31T15:04:26.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:26 smithi073 ceph-mon[36714]: osdmap e112: 12 total, 12 up, 12 in 2024-07-31T15:04:26.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:26 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/523725513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:04:26.445 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/irqinit.o 2024-07-31T15:04:26.472 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:26 smithi067 ceph-mon[32750]: osdmap e112: 12 total, 12 up, 12 in 2024-07-31T15:04:26.473 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:26 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/523725513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:04:26.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:26 smithi077 ceph-mon[37327]: osdmap e112: 12 total, 12 up, 12 in 2024-07-31T15:04:26.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:26 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/523725513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:04:26.594 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace.o 2024-07-31T15:04:26.875 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/cgroup-v1.o 2024-07-31T15:04:26.881 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/sha256_generic.o 2024-07-31T15:04:27.205 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/sync.o 2024-07-31T15:04:27.219 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-rq-qos.o 2024-07-31T15:04:27.258 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:27 smithi067 ceph-mon[32750]: pgmap v823: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1009 KiB/s wr, 136 op/s 2024-07-31T15:04:27.376 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/jump_label.o 2024-07-31T15:04:27.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:27 smithi073 ceph-mon[36714]: pgmap v823: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1009 KiB/s wr, 136 op/s 2024-07-31T15:04:27.398 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 144.3 --await-for 45.0' 2024-07-31T15:04:27.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 144.3 --await-for 45.0 2024-07-31T15:04:27.480 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/sha512_generic.o 2024-07-31T15:04:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:27 smithi077 ceph-mon[37327]: pgmap v823: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1009 KiB/s wr, 136 op/s 2024-07-31T15:04:27.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.139+0000 7f2ad99f5640 1 -- 172.21.15.67:0/2803655966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ad410c700 msgr2=0x7f2ad410cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:27.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.139+0000 7f2ad99f5640 1 --2- 172.21.15.67:0/2803655966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ad410c700 0x7f2ad410cb10 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009be0 tx=0x7f2acc02a6f0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.146+0000 7f2ad99f5640 1 -- 172.21.15.67:0/2803655966 shutdown_connections 2024-07-31T15:04:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.146+0000 7f2ad99f5640 1 --2- 172.21.15.67:0/2803655966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad410d050 0x7f2ad4116ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.146+0000 7f2ad99f5640 1 --2- 172.21.15.67:0/2803655966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ad410c700 0x7f2ad410cb10 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.146+0000 7f2ad99f5640 1 --2- 172.21.15.67:0/2803655966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad410a080 0x7f2ad410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.146+0000 7f2ad99f5640 1 -- 172.21.15.67:0/2803655966 >> 172.21.15.67:0/2803655966 conn(0x7f2ad4070d20 msgr2=0x7f2ad4073880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.149+0000 7f2ad99f5640 1 -- 172.21.15.67:0/2803655966 shutdown_connections 2024-07-31T15:04:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.150+0000 7f2ad99f5640 1 -- 172.21.15.67:0/2803655966 wait complete. 2024-07-31T15:04:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.151+0000 7f2ad99f5640 1 Processor -- start 2024-07-31T15:04:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.151+0000 7f2ad99f5640 1 -- start start 2024-07-31T15:04:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.152+0000 7f2ad99f5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad410a080 0x7f2ad4120130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.152+0000 7f2ad99f5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ad410c700 0x7f2ad411a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.152+0000 7f2ad99f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad410d050 0x7f2ad411aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.152+0000 7f2ad99f5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad4118840 con 0x7f2ad410c700 2024-07-31T15:04:27.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.152+0000 7f2ad99f5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad41186c0 con 0x7f2ad410a080 2024-07-31T15:04:27.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.152+0000 7f2ad99f5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad41189c0 con 0x7f2ad410d050 2024-07-31T15:04:27.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.153+0000 7f2ad2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad410a080 0x7f2ad4120130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:27.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.153+0000 7f2ad37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad410d050 0x7f2ad411aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:27.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.153+0000 7f2ad2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad410a080 0x7f2ad4120130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52238/0 (socket says 172.21.15.67:52238) 2024-07-31T15:04:27.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.153+0000 7f2ad2ffd640 1 -- 172.21.15.67:0/2532693450 learned_addr learned my addr 172.21.15.67:0/2532693450 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:27.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.153+0000 7f2ad37fe640 1 -- 172.21.15.67:0/2532693450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad410a080 msgr2=0x7f2ad4120130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:27.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.153+0000 7f2ad37fe640 1 --2- 172.21.15.67:0/2532693450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad410a080 0x7f2ad4120130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.153+0000 7f2ad37fe640 1 -- 172.21.15.67:0/2532693450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ad410c700 msgr2=0x7f2ad411a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:27.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.153+0000 7f2ad37fe640 1 --2- 172.21.15.67:0/2532693450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ad410c700 0x7f2ad411a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.154+0000 7f2ad37fe640 1 -- 172.21.15.67:0/2532693450 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad41bd7f0 con 0x7f2ad410d050 2024-07-31T15:04:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.160+0000 7f2ad37fe640 1 --2- 172.21.15.67:0/2532693450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad410d050 0x7f2ad411aa80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400ab50 tx=0x7f2ac4009a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.161+0000 7f2ad89f3640 1 -- 172.21.15.67:0/2532693450 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4018030 con 0x7f2ad410d050 2024-07-31T15:04:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.161+0000 7f2ad89f3640 1 -- 172.21.15.67:0/2532693450 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ac40054d0 con 0x7f2ad410d050 2024-07-31T15:04:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.161+0000 7f2ad89f3640 1 -- 172.21.15.67:0/2532693450 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4014880 con 0x7f2ad410d050 2024-07-31T15:04:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.163+0000 7f2ad99f5640 1 -- 172.21.15.67:0/2532693450 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad41bd9c0 con 0x7f2ad410d050 2024-07-31T15:04:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.163+0000 7f2ad99f5640 1 -- 172.21.15.67:0/2532693450 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad41bde80 con 0x7f2ad410d050 2024-07-31T15:04:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.170+0000 7f2ad89f3640 1 -- 172.21.15.67:0/2532693450 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2ac401b070 con 0x7f2ad410d050 2024-07-31T15:04:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.172+0000 7f2ad89f3640 1 --2- 172.21.15.67:0/2532693450 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2abc0b3ed0 0x7f2abc0b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.172+0000 7f2ad89f3640 1 -- 172.21.15.67:0/2532693450 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f2ac40e4950 con 0x7f2ad410d050 2024-07-31T15:04:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.173+0000 7f2adff46740 1 Processor -- start 2024-07-31T15:04:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.173+0000 7f2ad2ffd640 1 --2- 172.21.15.67:0/2532693450 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2abc0b3ed0 0x7f2abc0b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.174+0000 7f2adff46740 1 -- start start 2024-07-31T15:04:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.174+0000 7f2adff46740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f78209e070 0x55f782443520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.174+0000 7f2ad2ffd640 1 --2- 172.21.15.67:0/2532693450 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2abc0b3ed0 0x7f2abc0b63e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8005fd0 tx=0x7f2ac800b480 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.174+0000 7f2adff46740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7820751b0 0x55f782443a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.175+0000 7f2adff46740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78207d390 0x55f782443fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.175+0000 7f2ad37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78207d390 0x55f782443fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.175+0000 7f2ad37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78207d390 0x55f782443fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59482/0 (socket says 172.21.15.67:59482) 2024-07-31T15:04:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.175+0000 7f2ad37fe640 1 -- 172.21.15.67:0/1263599007 learned_addr learned my addr 172.21.15.67:0/1263599007 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.175+0000 7f2adff46740 1 -- 172.21.15.67:0/1263599007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f7823b9b60 con 0x55f78207d390 2024-07-31T15:04:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.175+0000 7f2adff46740 1 -- 172.21.15.67:0/1263599007 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f7823a9060 con 0x55f78209e070 2024-07-31T15:04:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.175+0000 7f2adff46740 1 -- 172.21.15.67:0/1263599007 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f7823ad0a0 con 0x55f7820751b0 2024-07-31T15:04:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.176+0000 7f2ad37fe640 1 -- 172.21.15.67:0/1263599007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7820751b0 msgr2=0x55f782443a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:04:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.176+0000 7f2ad37fe640 1 --2- 172.21.15.67:0/1263599007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7820751b0 0x55f782443a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.176+0000 7f2ad37fe640 1 -- 172.21.15.67:0/1263599007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f78209e070 msgr2=0x55f782443520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:04:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.176+0000 7f2ad37fe640 1 --2- 172.21.15.67:0/1263599007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f78209e070 0x55f782443520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.176+0000 7f2ad37fe640 1 -- 172.21.15.67:0/1263599007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f78234fb40 con 0x55f78207d390 2024-07-31T15:04:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.177+0000 7f2ad37fe640 1 --2- 172.21.15.67:0/1263599007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78207d390 0x55f782443fa0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009f30 tx=0x7f2ac40ab1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61721fc99142f8f2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.177+0000 7f2ab9ffb640 1 -- 172.21.15.67:0/1263599007 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac40c4020 con 0x55f78207d390 2024-07-31T15:04:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.177+0000 7f2ab9ffb640 1 -- 172.21.15.67:0/1263599007 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ac4005680 con 0x55f78207d390 2024-07-31T15:04:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.179+0000 7f2ab9ffb640 1 -- 172.21.15.67:0/1263599007 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac40a6da0 con 0x55f78207d390 2024-07-31T15:04:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.179+0000 7f2adff46740 1 -- 172.21.15.67:0/1263599007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78207d390 msgr2=0x55f782443fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.179+0000 7f2adff46740 1 --2- 172.21.15.67:0/1263599007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78207d390 0x55f782443fa0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009f30 tx=0x7f2ac40ab1f0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.179+0000 7f2adff46740 1 -- 172.21.15.67:0/1263599007 shutdown_connections 2024-07-31T15:04:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.179+0000 7f2adff46740 1 --2- 172.21.15.67:0/1263599007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78207d390 0x55f782443fa0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.179+0000 7f2adff46740 1 --2- 172.21.15.67:0/1263599007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7820751b0 0x55f782443a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.179+0000 7f2adff46740 1 --2- 172.21.15.67:0/1263599007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f78209e070 0x55f782443520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.179+0000 7f2adff46740 1 -- 172.21.15.67:0/1263599007 >> 172.21.15.67:0/1263599007 conn(0x55f782092460 msgr2=0x55f782441410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.180+0000 7f2adff46740 1 -- 172.21.15.67:0/1263599007 shutdown_connections 2024-07-31T15:04:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.180+0000 7f2adff46740 1 -- 172.21.15.67:0/1263599007 wait complete. 2024-07-31T15:04:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.180+0000 7f2adff46740 1 -- start start 2024-07-31T15:04:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.181+0000 7f2adff46740 1 Processor -- start 2024-07-31T15:04:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.182+0000 7f2adff46740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7820751b0 0x55f78244edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.182+0000 7f2adff46740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78207d390 0x55f78244f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.182+0000 7f2adff46740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f78209e070 0x55f78245b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.182+0000 7f2adff46740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f78220c700 con 0x55f78207d390 2024-07-31T15:04:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.182+0000 7f2adff46740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f78234b990 con 0x55f78209e070 2024-07-31T15:04:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.182+0000 7f2adff46740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f782451d50 con 0x55f7820751b0 2024-07-31T15:04:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.182+0000 7f2adff46740 10 client.0 _ll_register_callbacks cb 0x55f7823664a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.183+0000 7f2ad27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78207d390 0x55f78244f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.183+0000 7f2ad27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78207d390 0x55f78244f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59484/0 (socket says 172.21.15.67:59484) 2024-07-31T15:04:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.183+0000 7f2ad27fc640 1 -- 172.21.15.67:0/2740032621 learned_addr learned my addr 172.21.15.67:0/2740032621 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.184+0000 7f2ad27fc640 1 -- 172.21.15.67:0/2740032621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7820751b0 msgr2=0x55f78244edf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:04:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.184+0000 7f2ad27fc640 1 --2- 172.21.15.67:0/2740032621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7820751b0 0x55f78244edf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.184+0000 7f2ad27fc640 1 -- 172.21.15.67:0/2740032621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f78209e070 msgr2=0x55f78245b0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:04:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.184+0000 7f2ad27fc640 1 --2- 172.21.15.67:0/2740032621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f78209e070 0x55f78245b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.184+0000 7f2ad27fc640 1 -- 172.21.15.67:0/2740032621 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f782109650 con 0x55f78207d390 2024-07-31T15:04:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.185+0000 7f2ad27fc640 1 --2- 172.21.15.67:0/2740032621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78207d390 0x55f78244f3d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009b30 tx=0x7f2acc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22cafd1dee71340 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.189+0000 7f2ab2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:04:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.189+0000 7f2ab2ffd640 1 -- 172.21.15.67:0/2740032621 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc040020 con 0x55f78207d390 2024-07-31T15:04:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.189+0000 7f2adff46740 1 -- 172.21.15.67:0/2740032621 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ac4005680 con 0x55f78207d390 2024-07-31T15:04:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.189+0000 7f2ab2ffd640 1 -- 172.21.15.67:0/2740032621 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f2acc030970 con 0x55f78207d390 2024-07-31T15:04:27.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.190+0000 7f2ab2ffd640 1 -- 172.21.15.67:0/2740032621 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f2acc04b050 con 0x55f78207d390 2024-07-31T15:04:27.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.191+0000 7f2adff46740 10 client.18273 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:27.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.191+0000 7f2adff46740 1 -- 172.21.15.67:0/2740032621 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f78245dcf0 con 0x55f78207d390 2024-07-31T15:04:27.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.191+0000 7f2ab2ffd640 1 -- 172.21.15.67:0/2740032621 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ac4005680 con 0x55f78207d390 2024-07-31T15:04:27.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.191+0000 7f2ab2ffd640 1 -- 172.21.15.67:0/2740032621 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f2acc03c470 con 0x55f78207d390 2024-07-31T15:04:27.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.192+0000 7f2adff46740 10 client.18273 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:27.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.192+0000 7f2adff46740 10 client.18273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:27.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.192+0000 7f2adff46740 20 client.18273 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:27.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.192+0000 7f2adff46740 1 --2- 172.21.15.67:0/2740032621 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f78205c9f0 0x55f7824616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.192+0000 7f2adff46740 4 client.18273 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:04:27.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.192+0000 7f2adff46740 1 -- 172.21.15.67:0/2740032621 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7823bbab0 con 0x55f78205c9f0 2024-07-31T15:04:27.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.192+0000 7f2ad2ffd640 1 --2- 172.21.15.67:0/2740032621 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f78205c9f0 0x55f7824616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:27.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.199+0000 7f2ad2ffd640 1 --2- 172.21.15.67:0/2740032621 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f78205c9f0 0x55f7824616f0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b5eb1b47e88597 server_cookie=3c761826e87fb197 in_seq=0 out_seq=0 2024-07-31T15:04:27.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.199+0000 7f2ab2ffd640 10 client.18273 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:27.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.203+0000 7f2ab2ffd640 1 -- 172.21.15.67:0/2740032621 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7823bbab0 con 0x55f78205c9f0 2024-07-31T15:04:27.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.203+0000 7f2ab2ffd640 10 client.18273 handle_command_reply: tid=0 2024-07-31T15:04:27.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.203+0000 7f2adff46740 1 client.18273 shutdown 2024-07-31T15:04:27.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.203+0000 7f2adff46740 1 -- 172.21.15.67:0/2740032621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78207d390 msgr2=0x55f78244f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:27.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.203+0000 7f2adff46740 1 --2- 172.21.15.67:0/2740032621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78207d390 0x55f78244f3d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009b30 tx=0x7f2acc003ba0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.204+0000 7f2adff46740 1 -- 172.21.15.67:0/2740032621 shutdown_connections 2024-07-31T15:04:27.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.204+0000 7f2adff46740 1 --2- 172.21.15.67:0/2740032621 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f78205c9f0 0x55f7824616f0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.204+0000 7f2adff46740 1 --2- 172.21.15.67:0/2740032621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f78209e070 0x55f78245b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.204+0000 7f2adff46740 1 --2- 172.21.15.67:0/2740032621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78207d390 0x55f78244f3d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.204+0000 7f2adff46740 1 --2- 172.21.15.67:0/2740032621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f7820751b0 0x55f78244edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.204+0000 7f2adff46740 1 -- 172.21.15.67:0/2740032621 >> 172.21.15.67:0/2740032621 conn(0x55f782092460 msgr2=0x55f7824466b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:27.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.204+0000 7f2ab2ffd640 0 client.18273 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:27.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.204+0000 7f2adff46740 1 -- 172.21.15.67:0/2740032621 shutdown_connections 2024-07-31T15:04:27.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.204+0000 7f2adff46740 1 -- 172.21.15.67:0/2740032621 wait complete. 2024-07-31T15:04:27.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.209+0000 7f2adff46740 20 client.18273 trim_cache size 0 max 16384 2024-07-31T15:04:27.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.244+0000 7f2adff46740 1 Processor -- start 2024-07-31T15:04:27.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.244+0000 7f2adff46740 1 -- start start 2024-07-31T15:04:27.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2adff46740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7820751b0 0x55f782444c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2adff46740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f78207d390 0x55f78245aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2adff46740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78209e070 0x55f78245b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2adff46740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac000ff90 con 0x55f78209e070 2024-07-31T15:04:27.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2adff46740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac0011ad0 con 0x55f7820751b0 2024-07-31T15:04:27.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2adff46740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac0010660 con 0x55f78207d390 2024-07-31T15:04:27.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2ad37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78209e070 0x55f78245b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:27.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2ad37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78209e070 0x55f78245b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59486/0 (socket says 172.21.15.67:59486) 2024-07-31T15:04:27.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2ad27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f78207d390 0x55f78245aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:27.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2ad37fe640 1 -- 172.21.15.67:0/223226250 learned_addr learned my addr 172.21.15.67:0/223226250 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:27.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2ad2ffd640 1 --2- 172.21.15.67:0/223226250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7820751b0 0x55f782444c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:27.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2ad27fc640 1 -- 172.21.15.67:0/223226250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7820751b0 msgr2=0x55f782444c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:27.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2ad27fc640 1 --2- 172.21.15.67:0/223226250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7820751b0 0x55f782444c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2ad27fc640 1 -- 172.21.15.67:0/223226250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78209e070 msgr2=0x55f78245b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:27.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2ad27fc640 1 --2- 172.21.15.67:0/223226250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78209e070 0x55f78245b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.245+0000 7f2ad27fc640 1 -- 172.21.15.67:0/223226250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7824407e0 con 0x55f78207d390 2024-07-31T15:04:27.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.246+0000 7f2ad27fc640 1 --2- 172.21.15.67:0/223226250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f78207d390 0x55f78245aee0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2acc02ace0 tx=0x7f2acc005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66baa1d732c57ec4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:27.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.247+0000 7f2ab97fa640 1 -- 172.21.15.67:0/223226250 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc040070 con 0x55f78207d390 2024-07-31T15:04:27.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.247+0000 7f2ab97fa640 1 -- 172.21.15.67:0/223226250 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2acc030e20 con 0x55f78207d390 2024-07-31T15:04:27.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.249+0000 7f2adff46740 1 -- 172.21.15.67:0/223226250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f78207d390 msgr2=0x55f78245aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:27.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.249+0000 7f2adff46740 1 --2- 172.21.15.67:0/223226250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f78207d390 0x55f78245aee0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2acc02ace0 tx=0x7f2acc005fd0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.253+0000 7f2adff46740 1 -- 172.21.15.67:0/223226250 shutdown_connections 2024-07-31T15:04:27.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.253+0000 7f2adff46740 1 --2- 172.21.15.67:0/223226250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78209e070 0x55f78245b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.253+0000 7f2adff46740 1 --2- 172.21.15.67:0/223226250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f78207d390 0x55f78245aee0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.253+0000 7f2adff46740 1 --2- 172.21.15.67:0/223226250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7820751b0 0x55f782444c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.253+0000 7f2adff46740 1 -- 172.21.15.67:0/223226250 >> 172.21.15.67:0/223226250 conn(0x55f782092460 msgr2=0x55f782444550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:27.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.255+0000 7f2adff46740 1 -- 172.21.15.67:0/223226250 shutdown_connections 2024-07-31T15:04:27.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.259+0000 7f2adff46740 1 -- 172.21.15.67:0/223226250 wait complete. 2024-07-31T15:04:27.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.259+0000 7f2adff46740 1 -- start start 2024-07-31T15:04:27.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.259+0000 7f2adff46740 1 Processor -- start 2024-07-31T15:04:27.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.260+0000 7f2adff46740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7820751b0 0x55f782452e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.260+0000 7f2adff46740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f78207d390 0x55f7824533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.260+0000 7f2adff46740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78209e070 0x55f782453910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.260+0000 7f2adff46740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac0011400 con 0x55f78209e070 2024-07-31T15:04:27.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.260+0000 7f2adff46740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac0010d30 con 0x55f7820751b0 2024-07-31T15:04:27.551 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:04:27.551 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T15:04:27.551 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T15:04:27.551 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:04:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.260+0000 7f2adff46740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac000df90 con 0x55f78207d390 2024-07-31T15:04:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.260+0000 7f2adff46740 10 client.0 _ll_register_callbacks cb 0x7f2acc009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.261+0000 7f2ad37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78209e070 0x55f782453910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.261+0000 7f2ad37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78209e070 0x55f782453910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59488/0 (socket says 172.21.15.67:59488) 2024-07-31T15:04:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.261+0000 7f2ad37fe640 1 -- 172.21.15.67:0/3861248005 learned_addr learned my addr 172.21.15.67:0/3861248005 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.261+0000 7f2ad27fc640 1 --2- 172.21.15.67:0/3861248005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f78207d390 0x55f7824533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.261+0000 7f2ad2ffd640 1 --2- 172.21.15.67:0/3861248005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7820751b0 0x55f782452e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.262+0000 7f2ad37fe640 1 -- 172.21.15.67:0/3861248005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f78207d390 msgr2=0x55f7824533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.262+0000 7f2ad37fe640 1 --2- 172.21.15.67:0/3861248005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f78207d390 0x55f7824533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.262+0000 7f2ad37fe640 1 -- 172.21.15.67:0/3861248005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7820751b0 msgr2=0x55f782452e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:27.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.262+0000 7f2ad37fe640 1 --2- 172.21.15.67:0/3861248005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7820751b0 0x55f782452e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.262+0000 7f2ad37fe640 1 -- 172.21.15.67:0/3861248005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7824c8ec0 con 0x55f78209e070 2024-07-31T15:04:27.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.262+0000 7f2ad37fe640 1 --2- 172.21.15.67:0/3861248005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78209e070 0x55f782453910 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009f30 tx=0x7f2ac40a76b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5cb9e1e5f0366 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:27.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.264+0000 7f2ab2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:04:27.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.264+0000 7f2ab2ffd640 1 -- 172.21.15.67:0/3861248005 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac40aa660 con 0x55f78209e070 2024-07-31T15:04:27.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.264+0000 7f2ab2ffd640 1 -- 172.21.15.67:0/3861248005 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f2ac40cc390 con 0x55f78209e070 2024-07-31T15:04:27.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.266+0000 7f2adff46740 1 -- 172.21.15.67:0/3861248005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2acc030e20 con 0x55f78209e070 2024-07-31T15:04:27.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.267+0000 7f2ab2ffd640 1 -- 172.21.15.67:0/3861248005 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f2ac40a6b60 con 0x55f78209e070 2024-07-31T15:04:27.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.271+0000 7f2adff46740 10 client.18279 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:27.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.271+0000 7f2adff46740 1 -- 172.21.15.67:0/3861248005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f7824ca070 con 0x55f78209e070 2024-07-31T15:04:27.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.271+0000 7f2ab2ffd640 1 -- 172.21.15.67:0/3861248005 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2acc030e20 con 0x55f78209e070 2024-07-31T15:04:27.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.272+0000 7f2ab2ffd640 1 -- 172.21.15.67:0/3861248005 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f2ac4025060 con 0x55f78209e070 2024-07-31T15:04:27.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.274+0000 7f2adff46740 10 client.18279 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:27.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.274+0000 7f2adff46740 10 client.18279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:27.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.274+0000 7f2adff46740 20 client.18279 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:27.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.274+0000 7f2adff46740 1 --2- 172.21.15.67:0/3861248005 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f782031da0 0x55f7824cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:27.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.274+0000 7f2adff46740 4 client.18279 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:04:27.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.274+0000 7f2adff46740 1 -- 172.21.15.67:0/3861248005 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f782447240 con 0x55f782031da0 2024-07-31T15:04:27.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.275+0000 7f2ad2ffd640 1 --2- 172.21.15.67:0/3861248005 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f782031da0 0x55f7824cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:27.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.284+0000 7f2ad2ffd640 1 --2- 172.21.15.67:0/3861248005 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f782031da0 0x55f7824cc300 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ba084adc04cd94b server_cookie=8a9cae7dbe970280 in_seq=0 out_seq=0 2024-07-31T15:04:27.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.284+0000 7f2ab2ffd640 10 client.18279 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:27.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.285+0000 7f2ab2ffd640 1 -- 172.21.15.67:0/3861248005 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f782447240 con 0x55f782031da0 2024-07-31T15:04:27.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.285+0000 7f2ab2ffd640 10 client.18279 handle_command_reply: tid=0 2024-07-31T15:04:27.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.286+0000 7f2adff46740 1 client.18279 shutdown 2024-07-31T15:04:27.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.287+0000 7f2adff46740 1 -- 172.21.15.67:0/3861248005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78209e070 msgr2=0x55f782453910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:27.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.287+0000 7f2adff46740 1 --2- 172.21.15.67:0/3861248005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78209e070 0x55f782453910 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009f30 tx=0x7f2ac40a76b0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.288+0000 7f2adff46740 1 -- 172.21.15.67:0/3861248005 shutdown_connections 2024-07-31T15:04:27.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.288+0000 7f2adff46740 1 --2- 172.21.15.67:0/3861248005 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f782031da0 0x55f7824cc300 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.288+0000 7f2adff46740 1 --2- 172.21.15.67:0/3861248005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f78209e070 0x55f782453910 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.288+0000 7f2adff46740 1 --2- 172.21.15.67:0/3861248005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f78207d390 0x55f7824533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.288+0000 7f2adff46740 1 --2- 172.21.15.67:0/3861248005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7820751b0 0x55f782452e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.288+0000 7f2adff46740 1 -- 172.21.15.67:0/3861248005 >> 172.21.15.67:0/3861248005 conn(0x55f782092460 msgr2=0x55f782444ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.288+0000 7f2ab2ffd640 0 client.18279 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.290+0000 7f2adff46740 1 -- 172.21.15.67:0/3861248005 shutdown_connections 2024-07-31T15:04:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.290+0000 7f2adff46740 1 -- 172.21.15.67:0/3861248005 wait complete. 2024-07-31T15:04:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.292+0000 7f2adff46740 20 client.18279 trim_cache size 0 max 16384 2024-07-31T15:04:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.293+0000 7f2ad99f5640 1 -- 172.21.15.67:0/2532693450 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2abc0b3ed0 msgr2=0x7f2abc0b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.293+0000 7f2ad99f5640 1 --2- 172.21.15.67:0/2532693450 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2abc0b3ed0 0x7f2abc0b63e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8005fd0 tx=0x7f2ac800b480 comp rx=0 tx=0).stop 2024-07-31T15:04:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.293+0000 7f2ad99f5640 1 -- 172.21.15.67:0/2532693450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad410d050 msgr2=0x7f2ad411aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.293+0000 7f2ad99f5640 1 --2- 172.21.15.67:0/2532693450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad410d050 0x7f2ad411aa80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400ab50 tx=0x7f2ac4009a30 comp rx=0 tx=0).stop 2024-07-31T15:04:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.293+0000 7f2ad99f5640 1 -- 172.21.15.67:0/2532693450 shutdown_connections 2024-07-31T15:04:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.293+0000 7f2ad99f5640 1 --2- 172.21.15.67:0/2532693450 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2abc0b3ed0 0x7f2abc0b63e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.293+0000 7f2ad99f5640 1 --2- 172.21.15.67:0/2532693450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad410d050 0x7f2ad411aa80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.293+0000 7f2ad99f5640 1 --2- 172.21.15.67:0/2532693450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ad410c700 0x7f2ad411a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.293+0000 7f2ad99f5640 1 --2- 172.21.15.67:0/2532693450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad410a080 0x7f2ad4120130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.293+0000 7f2ad99f5640 1 -- 172.21.15.67:0/2532693450 >> 172.21.15.67:0/2532693450 conn(0x7f2ad4070d20 msgr2=0x7f2ad410f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.293+0000 7f2ad99f5640 1 -- 172.21.15.67:0/2532693450 shutdown_connections 2024-07-31T15:04:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:27.293+0000 7f2ad99f5640 1 -- 172.21.15.67:0/2532693450 wait complete. 2024-07-31T15:04:27.602 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/migrate.o 2024-07-31T15:04:27.942 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/disk-events.o 2024-07-31T15:04:27.961 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/irq_work.o 2024-07-31T15:04:28.027 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/advise.o 2024-07-31T15:04:28.083 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/freezer.o 2024-07-31T15:04:28.156 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/sha3_generic.o 2024-07-31T15:04:28.340 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T15:04:28.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:04:28.592 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:04:28.693 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-ia-ranges.o 2024-07-31T15:04:28.714 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/filetable.o 2024-07-31T15:04:28.731 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/cbc.o 2024-07-31T15:04:28.773 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/probe_roms.o 2024-07-31T15:04:28.917 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/legacy_freezer.o 2024-07-31T15:04:29.198 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/ctr.o 2024-07-31T15:04:29.251 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/memory-tiers.o 2024-07-31T15:04:29.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:29 smithi067 ceph-mon[32750]: pgmap v824: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-07-31T15:04:29.347 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/symlink.o 2024-07-31T15:04:29.374 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/early-lookup.o 2024-07-31T15:04:29.467 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/sys_ia32.o 2024-07-31T15:04:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:29 smithi077 ceph-mon[37327]: pgmap v824: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-07-31T15:04:29.626 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/pids.o 2024-07-31T15:04:29.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:29 smithi073 ceph-mon[36714]: pgmap v824: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-07-31T15:04:29.647 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/openclose.o 2024-07-31T15:04:29.651 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/gcm.o 2024-07-31T15:04:29.971 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_output.o 2024-07-31T15:04:30.010 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/bsg.o 2024-07-31T15:04:30.011 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/page_counter.o 2024-07-31T15:04:30.055 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/sysfs.o 2024-07-31T15:04:30.185 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/rdma.o 2024-07-31T15:04:30.287 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/signal_32.o 2024-07-31T15:04:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/hugetlb_cgroup.o 2024-07-31T15:04:30.327 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:04:30.555 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/ccm.o 2024-07-31T15:04:30.674 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/uring_cmd.o 2024-07-31T15:04:30.722 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-cgroup.o 2024-07-31T15:04:30.840 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/xattr.o 2024-07-31T15:04:30.848 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/cpuset.o 2024-07-31T15:04:31.092 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/sys_x86_64.o 2024-07-31T15:04:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.161+0000 7f8da4606640 1 -- 172.21.15.67:0/3536137263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d9c10fcc0 msgr2=0x7f8d9c1100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.161+0000 7f8da4606640 1 --2- 172.21.15.67:0/3536137263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d9c10fcc0 0x7f8d9c1100d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009860 tx=0x7f8d8c02a340 comp rx=0 tx=0).stop 2024-07-31T15:04:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.161+0000 7f8da4606640 1 -- 172.21.15.67:0/3536137263 shutdown_connections 2024-07-31T15:04:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.161+0000 7f8da4606640 1 --2- 172.21.15.67:0/3536137263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c110610 0x7f8d9c116ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.161+0000 7f8da4606640 1 --2- 172.21.15.67:0/3536137263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d9c10fcc0 0x7f8d9c1100d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.161+0000 7f8da4606640 1 --2- 172.21.15.67:0/3536137263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d9c10a920 0x7f8d9c10ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.161+0000 7f8da4606640 1 -- 172.21.15.67:0/3536137263 >> 172.21.15.67:0/3536137263 conn(0x7f8d9c0707c0 msgr2=0x7f8d9c070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.161+0000 7f8da4606640 1 -- 172.21.15.67:0/3536137263 shutdown_connections 2024-07-31T15:04:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.161+0000 7f8da4606640 1 -- 172.21.15.67:0/3536137263 wait complete. 2024-07-31T15:04:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.162+0000 7f8da4606640 1 Processor -- start 2024-07-31T15:04:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.163+0000 7f8da4606640 1 -- start start 2024-07-31T15:04:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.163+0000 7f8da4606640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c10a920 0x7f8d9c1a2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.163+0000 7f8da4606640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d9c10fcc0 0x7f8d9c1a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.163+0000 7f8da4606640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d9c110610 0x7f8d9c1a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.163+0000 7f8da4606640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c11bc70 con 0x7f8d9c110610 2024-07-31T15:04:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.163+0000 7f8da4606640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c11baf0 con 0x7f8d9c10a920 2024-07-31T15:04:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.163+0000 7f8da4606640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c11bdf0 con 0x7f8d9c10fcc0 2024-07-31T15:04:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.163+0000 7f8da23a2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c10a920 0x7f8d9c1a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.163+0000 7f8da1ba1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d9c10fcc0 0x7f8d9c1a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.163+0000 7f8da23a2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c10a920 0x7f8d9c1a2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56690/0 (socket says 172.21.15.67:56690) 2024-07-31T15:04:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.163+0000 7f8da1ba1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d9c10fcc0 0x7f8d9c1a2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50814/0 (socket says 172.21.15.67:50814) 2024-07-31T15:04:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.163+0000 7f8da23a2640 1 -- 172.21.15.67:0/3486051881 learned_addr learned my addr 172.21.15.67:0/3486051881 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.163+0000 7f8da2ba3640 1 --2- 172.21.15.67:0/3486051881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d9c110610 0x7f8d9c1a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.164+0000 7f8da23a2640 1 -- 172.21.15.67:0/3486051881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d9c10fcc0 msgr2=0x7f8d9c1a2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.164+0000 7f8da23a2640 1 --2- 172.21.15.67:0/3486051881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d9c10fcc0 0x7f8d9c1a2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.164+0000 7f8da23a2640 1 -- 172.21.15.67:0/3486051881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d9c110610 msgr2=0x7f8d9c1a7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.164+0000 7f8da23a2640 1 --2- 172.21.15.67:0/3486051881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d9c110610 0x7f8d9c1a7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.164+0000 7f8da23a2640 1 -- 172.21.15.67:0/3486051881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9c1a8640 con 0x7f8d9c10a920 2024-07-31T15:04:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.164+0000 7f8da1ba1640 1 --2- 172.21.15.67:0/3486051881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d9c10fcc0 0x7f8d9c1a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.164+0000 7f8da23a2640 1 --2- 172.21.15.67:0/3486051881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c10a920 0x7f8d9c1a2710 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400c420 tx=0x7f8d94009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.166+0000 7f8d937fe640 1 -- 172.21.15.67:0/3486051881 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94003970 con 0x7f8d9c10a920 2024-07-31T15:04:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.166+0000 7f8d937fe640 1 -- 172.21.15.67:0/3486051881 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d94004020 con 0x7f8d9c10a920 2024-07-31T15:04:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.166+0000 7f8da4606640 1 -- 172.21.15.67:0/3486051881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9c1a8930 con 0x7f8d9c10a920 2024-07-31T15:04:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.166+0000 7f8da4606640 1 -- 172.21.15.67:0/3486051881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d9c118520 con 0x7f8d9c10a920 2024-07-31T15:04:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.168+0000 7f8d937fe640 1 -- 172.21.15.67:0/3486051881 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400b980 con 0x7f8d9c10a920 2024-07-31T15:04:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.171+0000 7f8d937fe640 1 -- 172.21.15.67:0/3486051881 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8d9401d020 con 0x7f8d9c10a920 2024-07-31T15:04:31.172 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_seq.o 2024-07-31T15:04:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.172+0000 7f8d937fe640 1 --2- 172.21.15.67:0/3486051881 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8d880b3ed0 0x7f8d880b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.173+0000 7f8d937fe640 1 -- 172.21.15.67:0/3486051881 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f8d940e5010 con 0x7f8d9c10a920 2024-07-31T15:04:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.173+0000 7f8da1ba1640 1 --2- 172.21.15.67:0/3486051881 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8d880b3ed0 0x7f8d880b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.178+0000 7f8da1ba1640 1 --2- 172.21.15.67:0/3486051881 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8d880b3ed0 0x7f8d880b63e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c02a8b0 tx=0x7f8d8c02a840 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.180+0000 7f8dab5db740 1 Processor -- start 2024-07-31T15:04:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.180+0000 7f8dab5db740 1 -- start start 2024-07-31T15:04:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.180+0000 7f8dab5db740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f468a1b0 0x5581f4a2c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.180+0000 7f8dab5db740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f46612f0 0x5581f4a2cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.180+0000 7f8dab5db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46694d0 0x5581f4a2d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.180+0000 7f8dab5db740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5581f49a23b0 con 0x5581f46612f0 2024-07-31T15:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.180+0000 7f8dab5db740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5581f49918b0 con 0x5581f468a1b0 2024-07-31T15:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.180+0000 7f8dab5db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5581f49958f0 con 0x5581f46694d0 2024-07-31T15:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.181+0000 7f8da23a2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f468a1b0 0x5581f4a2c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.181+0000 7f8da23a2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f468a1b0 0x5581f4a2c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56702/0 (socket says 172.21.15.67:56702) 2024-07-31T15:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.181+0000 7f8da23a2640 1 -- 172.21.15.67:0/2870458752 learned_addr learned my addr 172.21.15.67:0/2870458752 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.181+0000 7f8da1ba1640 1 --2- 172.21.15.67:0/2870458752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f46612f0 0x5581f4a2cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.181+0000 7f8da2ba3640 1 --2- 172.21.15.67:0/2870458752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46694d0 0x5581f4a2d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.181+0000 7f8da23a2640 1 -- 172.21.15.67:0/2870458752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46694d0 msgr2=0x5581f4a2d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.181+0000 7f8da23a2640 1 --2- 172.21.15.67:0/2870458752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46694d0 0x5581f4a2d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.181+0000 7f8da23a2640 1 -- 172.21.15.67:0/2870458752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f46612f0 msgr2=0x5581f4a2cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.181+0000 7f8da23a2640 1 --2- 172.21.15.67:0/2870458752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f46612f0 0x5581f4a2cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.181+0000 7f8da23a2640 1 -- 172.21.15.67:0/2870458752 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581f49389a0 con 0x5581f468a1b0 2024-07-31T15:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.181+0000 7f8da2ba3640 1 --2- 172.21.15.67:0/2870458752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46694d0 0x5581f4a2d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:04:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.181+0000 7f8da1ba1640 1 --2- 172.21.15.67:0/2870458752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f46612f0 0x5581f4a2cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:04:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.181+0000 7f8da23a2640 1 --2- 172.21.15.67:0/2870458752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f468a1b0 0x5581f4a2c6b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8d940a5fd0 tx=0x7f8d940ab500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7626e2330cff3d2e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.180+0000 7fdb9a758640 1 -- 172.21.15.67:0/3777486556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9410c100 msgr2=0x7fdb9407b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.180+0000 7fdb9a758640 1 --2- 172.21.15.67:0/3777486556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9410c100 0x7fdb9407b950 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c005fd0 tx=0x7fdb8c02eae0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.185+0000 7fdb9a758640 1 -- 172.21.15.67:0/3777486556 shutdown_connections 2024-07-31T15:04:31.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.185+0000 7fdb9a758640 1 --2- 172.21.15.67:0/3777486556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9410c100 0x7fdb9407b950 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.185+0000 7fdb9a758640 1 --2- 172.21.15.67:0/3777486556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb9410c550 0x7fdb9410c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.185+0000 7fdb9a758640 1 --2- 172.21.15.67:0/3777486556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb9410a080 0x7fdb9410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.185+0000 7fdb9a758640 1 -- 172.21.15.67:0/3777486556 >> 172.21.15.67:0/3777486556 conn(0x7fdb9406e440 msgr2=0x7fdb9406e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.193+0000 7f8d90bfe640 1 -- 172.21.15.67:0/2870458752 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d940b4960 con 0x5581f468a1b0 2024-07-31T15:04:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.193+0000 7f8d90bfe640 1 -- 172.21.15.67:0/2870458752 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d940e2670 con 0x5581f468a1b0 2024-07-31T15:04:31.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.193+0000 7fdb9a758640 1 -- 172.21.15.67:0/3777486556 shutdown_connections 2024-07-31T15:04:31.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.194+0000 7fdb9a758640 1 -- 172.21.15.67:0/3777486556 wait complete. 2024-07-31T15:04:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.195+0000 7f8dab5db740 1 -- 172.21.15.67:0/2870458752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f468a1b0 msgr2=0x5581f4a2c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.195+0000 7f8dab5db740 1 --2- 172.21.15.67:0/2870458752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f468a1b0 0x5581f4a2c6b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8d940a5fd0 tx=0x7f8d940ab500 comp rx=0 tx=0).stop 2024-07-31T15:04:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.195+0000 7f8dab5db740 1 -- 172.21.15.67:0/2870458752 shutdown_connections 2024-07-31T15:04:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.195+0000 7f8dab5db740 1 --2- 172.21.15.67:0/2870458752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46694d0 0x5581f4a2d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.195+0000 7f8dab5db740 1 --2- 172.21.15.67:0/2870458752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f46612f0 0x5581f4a2cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.195+0000 7f8dab5db740 1 --2- 172.21.15.67:0/2870458752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f468a1b0 0x5581f4a2c6b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.195+0000 7f8dab5db740 1 -- 172.21.15.67:0/2870458752 >> 172.21.15.67:0/2870458752 conn(0x5581f467e300 msgr2=0x5581f4a2a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.195+0000 7fdb9a758640 1 Processor -- start 2024-07-31T15:04:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.195+0000 7fdb9a758640 1 -- start start 2024-07-31T15:04:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.201+0000 7f8dab5db740 1 -- 172.21.15.67:0/2870458752 shutdown_connections 2024-07-31T15:04:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.201+0000 7f8dab5db740 1 -- 172.21.15.67:0/2870458752 wait complete. 2024-07-31T15:04:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.202+0000 7f8dab5db740 1 -- start start 2024-07-31T15:04:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.202+0000 7f8dab5db740 1 Processor -- start 2024-07-31T15:04:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.202+0000 7f8dab5db740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46612f0 0x5581f4a38ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.202+0000 7f8dab5db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46694d0 0x5581f4a390a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.202+0000 7f8dab5db740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f4648d80 0x5581f4a3e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.202+0000 7f8dab5db740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5581f48cbab0 con 0x5581f4648d80 2024-07-31T15:04:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.202+0000 7f8dab5db740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5581f4919cf0 con 0x5581f46612f0 2024-07-31T15:04:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.202+0000 7f8dab5db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5581f4a32de0 con 0x5581f46694d0 2024-07-31T15:04:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.202+0000 7f8dab5db740 10 client.0 _ll_register_callbacks cb 0x5581f498b3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.203+0000 7f8da2ba3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f4648d80 0x5581f4a3e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.203+0000 7f8da23a2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46612f0 0x5581f4a38ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.203+0000 7f8da1ba1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46694d0 0x5581f4a390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.203+0000 7f8da2ba3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f4648d80 0x5581f4a3e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39000/0 (socket says 172.21.15.67:39000) 2024-07-31T15:04:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.203+0000 7f8da2ba3640 1 -- 172.21.15.67:0/1280887578 learned_addr learned my addr 172.21.15.67:0/1280887578 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.204+0000 7f8da23a2640 1 -- 172.21.15.67:0/1280887578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46694d0 msgr2=0x5581f4a390a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.204+0000 7f8da23a2640 1 --2- 172.21.15.67:0/1280887578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46694d0 0x5581f4a390a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.204+0000 7f8da23a2640 1 -- 172.21.15.67:0/1280887578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f4648d80 msgr2=0x5581f4a3e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.204+0000 7f8da23a2640 1 --2- 172.21.15.67:0/1280887578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f4648d80 0x5581f4a3e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.204+0000 7f8da23a2640 1 -- 172.21.15.67:0/1280887578 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581f46f5790 con 0x5581f46612f0 2024-07-31T15:04:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.204+0000 7f8da2ba3640 1 --2- 172.21.15.67:0/1280887578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f4648d80 0x5581f4a3e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.205+0000 7f8da23a2640 1 --2- 172.21.15.67:0/1280887578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46612f0 0x5581f4a38ae0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8d940e2970 tx=0x7f8d940e28a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c221e0c373d8409 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:31.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.205+0000 7fdb9a758640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb9410a080 0x7fdb94080080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.209+0000 7fdb9a758640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9410c100 0x7fdb940805c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.209+0000 7fdb9a758640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb9410c550 0x7fdb94080b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.209+0000 7fdb9a758640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9407e530 con 0x7fdb9410a080 2024-07-31T15:04:31.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.209+0000 7fdb9a758640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9407e3b0 con 0x7fdb9410c100 2024-07-31T15:04:31.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.209+0000 7fdb9a758640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9407e6b0 con 0x7fdb9410c550 2024-07-31T15:04:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.213+0000 7f8d85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:04:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.213+0000 7f8d85ffb640 1 -- 172.21.15.67:0/1280887578 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d940b4960 con 0x5581f46612f0 2024-07-31T15:04:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.213+0000 7f8d85ffb640 1 -- 172.21.15.67:0/1280887578 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f8d940b44d0 con 0x5581f46612f0 2024-07-31T15:04:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.213+0000 7f8dab5db740 1 -- 172.21.15.67:0/1280887578 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5581f49a4300 con 0x5581f46612f0 2024-07-31T15:04:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.213+0000 7f8d85ffb640 1 -- 172.21.15.67:0/1280887578 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8d9400be50 con 0x5581f46612f0 2024-07-31T15:04:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.213+0000 7f8dab5db740 10 client.27857 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.213+0000 7f8dab5db740 1 -- 172.21.15.67:0/1280887578 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5581f476e620 con 0x5581f46612f0 2024-07-31T15:04:31.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.209+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb9410a080 0x7fdb94080080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.212+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb9410a080 0x7fdb94080080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39002/0 (socket says 172.21.15.67:39002) 2024-07-31T15:04:31.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.212+0000 7fdb93fff640 1 -- 172.21.15.67:0/309455128 learned_addr learned my addr 172.21.15.67:0/309455128 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:31.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.212+0000 7fdb937fe640 1 --2- 172.21.15.67:0/309455128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9410c100 0x7fdb940805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.213+0000 7fdb98cf5640 1 --2- 172.21.15.67:0/309455128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb9410c550 0x7fdb94080b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.215+0000 7f8d85ffb640 1 -- 172.21.15.67:0/1280887578 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5581f49a4300 con 0x5581f46612f0 2024-07-31T15:04:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.215+0000 7f8d85ffb640 1 -- 172.21.15.67:0/1280887578 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f8d9400b040 con 0x5581f46612f0 2024-07-31T15:04:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.215+0000 7f8dab5db740 10 client.27857 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.215+0000 7f8dab5db740 10 client.27857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.215+0000 7f8dab5db740 20 client.27857 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.215+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1280887578 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5581f461e1c0 0x5581f4a5b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.215+0000 7f8dab5db740 4 client.27857 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:04:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.215+0000 7f8dab5db740 1 -- 172.21.15.67:0/1280887578 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581f499fda0 con 0x5581f461e1c0 2024-07-31T15:04:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.215+0000 7f8da1ba1640 1 --2- 172.21.15.67:0/1280887578 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5581f461e1c0 0x5581f4a5b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.216+0000 7f8d85ffb640 10 client.27857 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.217+0000 7f8da1ba1640 1 --2- 172.21.15.67:0/1280887578 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5581f461e1c0 0x5581f4a5b400 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d50c69acefe6f0 server_cookie=d8d8e7e3ad6022d4 in_seq=0 out_seq=0 2024-07-31T15:04:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.218+0000 7f8d85ffb640 1 -- 172.21.15.67:0/1280887578 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581f499fda0 con 0x5581f461e1c0 2024-07-31T15:04:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.218+0000 7f8d85ffb640 10 client.27857 handle_command_reply: tid=0 2024-07-31T15:04:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.218+0000 7f8dab5db740 1 client.27857 shutdown 2024-07-31T15:04:31.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.215+0000 7fdb93fff640 1 -- 172.21.15.67:0/309455128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb9410c550 msgr2=0x7fdb94080b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:31.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.215+0000 7fdb93fff640 1 --2- 172.21.15.67:0/309455128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb9410c550 0x7fdb94080b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.215+0000 7fdb93fff640 1 -- 172.21.15.67:0/309455128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9410c100 msgr2=0x7fdb940805c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:31.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.215+0000 7fdb93fff640 1 --2- 172.21.15.67:0/309455128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9410c100 0x7fdb940805c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.215+0000 7fdb93fff640 1 -- 172.21.15.67:0/309455128 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb941bd420 con 0x7fdb9410a080 2024-07-31T15:04:31.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.215+0000 7fdb93fff640 1 --2- 172.21.15.67:0/309455128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb9410a080 0x7fdb94080080 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400b450 tx=0x7fdb8400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.218+0000 7f8dab5db740 1 -- 172.21.15.67:0/1280887578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46612f0 msgr2=0x5581f4a38ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.218+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1280887578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46612f0 0x5581f4a38ae0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8d940e2970 tx=0x7f8d940e28a0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.218+0000 7f8dab5db740 1 -- 172.21.15.67:0/1280887578 shutdown_connections 2024-07-31T15:04:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.218+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1280887578 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5581f461e1c0 0x5581f4a5b400 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.218+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1280887578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f4648d80 0x5581f4a3e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.218+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1280887578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46694d0 0x5581f4a390a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.218+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1280887578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46612f0 0x5581f4a38ae0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.218+0000 7f8dab5db740 1 -- 172.21.15.67:0/1280887578 >> 172.21.15.67:0/1280887578 conn(0x5581f467e300 msgr2=0x5581f4a2f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.218+0000 7f8d85ffb640 0 client.27857 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.219+0000 7f8dab5db740 1 -- 172.21.15.67:0/1280887578 shutdown_connections 2024-07-31T15:04:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.219+0000 7f8dab5db740 1 -- 172.21.15.67:0/1280887578 wait complete. 2024-07-31T15:04:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.219+0000 7f8dab5db740 20 client.27857 trim_cache size 0 max 16384 2024-07-31T15:04:31.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.220+0000 7fdb917fa640 1 -- 172.21.15.67:0/309455128 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84015020 con 0x7fdb9410a080 2024-07-31T15:04:31.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.220+0000 7fdb917fa640 1 -- 172.21.15.67:0/309455128 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb84011d40 con 0x7fdb9410a080 2024-07-31T15:04:31.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.220+0000 7fdb917fa640 1 -- 172.21.15.67:0/309455128 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8400f340 con 0x7fdb9410a080 2024-07-31T15:04:31.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.222+0000 7fdb9a758640 1 -- 172.21.15.67:0/309455128 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb941bd710 con 0x7fdb9410a080 2024-07-31T15:04:31.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.222+0000 7fdb9a758640 1 -- 172.21.15.67:0/309455128 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb941bdc80 con 0x7fdb9410a080 2024-07-31T15:04:31.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.226+0000 7fdba0ca9740 1 Processor -- start 2024-07-31T15:04:31.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.228+0000 7fdba0ca9740 1 -- start start 2024-07-31T15:04:31.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.229+0000 7fdb917fa640 1 -- 172.21.15.67:0/309455128 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fdb84018020 con 0x7fdb9410a080 2024-07-31T15:04:31.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.234+0000 7fdba0ca9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a17e840 0x55941a524a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.234+0000 7fdba0ca9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a155980 0x55941a524f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.234+0000 7fdba0ca9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a15db60 0x55941a5254b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.234+0000 7fdba0ca9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55941a499780 con 0x55941a17e840 2024-07-31T15:04:31.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.234+0000 7fdba0ca9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55941a488c80 con 0x55941a15db60 2024-07-31T15:04:31.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.234+0000 7fdba0ca9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55941a48ccc0 con 0x55941a155980 2024-07-31T15:04:31.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.237+0000 7fdb98cf5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a17e840 0x55941a524a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.237+0000 7fdb98cf5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a17e840 0x55941a524a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39006/0 (socket says 172.21.15.67:39006) 2024-07-31T15:04:31.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.238+0000 7fdb98cf5640 1 -- 172.21.15.67:0/826309529 learned_addr learned my addr 172.21.15.67:0/826309529 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:31.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.238+0000 7fdb93fff640 1 --2- 172.21.15.67:0/826309529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a155980 0x55941a524f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.238+0000 7fdb98cf5640 1 -- 172.21.15.67:0/826309529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a155980 msgr2=0x55941a524f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:31.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.238+0000 7fdb98cf5640 1 --2- 172.21.15.67:0/826309529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a155980 0x55941a524f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.238+0000 7fdb98cf5640 1 -- 172.21.15.67:0/826309529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a15db60 msgr2=0x55941a5254b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:31.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.238+0000 7fdb98cf5640 1 --2- 172.21.15.67:0/826309529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a15db60 0x55941a5254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.238+0000 7fdb98cf5640 1 -- 172.21.15.67:0/826309529 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55941a41e080 con 0x55941a17e840 2024-07-31T15:04:31.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.238+0000 7fdb98cf5640 1 --2- 172.21.15.67:0/826309529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a17e840 0x55941a524a30 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0121a0 tx=0x7fdb8c0121d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f241bf842c05b2c9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:31.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.238+0000 7fdb917fa640 1 --2- 172.21.15.67:0/309455128 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdb7c0b3f40 0x7fdb7c0b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.239+0000 7fdb93fff640 1 --2- 172.21.15.67:0/309455128 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdb7c0b3f40 0x7fdb7c0b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.240+0000 7fdb93fff640 1 --2- 172.21.15.67:0/309455128 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdb7c0b3f40 0x7fdb7c0b6450 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55941a5236b0 tx=0x7fdb840e7290 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:31.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.240+0000 7fdb7a7fc640 1 -- 172.21.15.67:0/826309529 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c003970 con 0x55941a17e840 2024-07-31T15:04:31.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.240+0000 7fdb7a7fc640 1 -- 172.21.15.67:0/826309529 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb8c004020 con 0x55941a17e840 2024-07-31T15:04:31.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.240+0000 7fdb917fa640 1 -- 172.21.15.67:0/309455128 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fdb840e3d30 con 0x7fdb9410a080 2024-07-31T15:04:31.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.241+0000 7fdba0ca9740 1 -- 172.21.15.67:0/826309529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a17e840 msgr2=0x55941a524a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:31.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.241+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/826309529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a17e840 0x55941a524a30 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0121a0 tx=0x7fdb8c0121d0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.243+0000 7fdba0ca9740 1 -- 172.21.15.67:0/826309529 shutdown_connections 2024-07-31T15:04:31.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.243+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/826309529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a15db60 0x55941a5254b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.243+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/826309529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a155980 0x55941a524f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.243+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/826309529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a17e840 0x55941a524a30 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.243+0000 7fdba0ca9740 1 -- 172.21.15.67:0/826309529 >> 172.21.15.67:0/826309529 conn(0x55941a172990 msgr2=0x55941a522920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:31.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.245+0000 7fdba0ca9740 1 -- 172.21.15.67:0/826309529 shutdown_connections 2024-07-31T15:04:31.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.250+0000 7fdba0ca9740 1 -- 172.21.15.67:0/826309529 wait complete. 2024-07-31T15:04:31.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.250+0000 7fdba0ca9740 1 -- start start 2024-07-31T15:04:31.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.250+0000 7fdba0ca9740 1 Processor -- start 2024-07-31T15:04:31.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.252+0000 7fdba0ca9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a155980 0x55941a541020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.252+0000 7fdba0ca9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a15db60 0x55941a541560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.252+0000 7fdba0ca9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a17e840 0x55941a541aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.252+0000 7fdba0ca9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55941a2ec7a0 con 0x55941a155980 2024-07-31T15:04:31.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.252+0000 7fdba0ca9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55941a42b2f0 con 0x55941a15db60 2024-07-31T15:04:31.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.252+0000 7fdba0ca9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55941a544100 con 0x55941a17e840 2024-07-31T15:04:31.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.252+0000 7fdba0ca9740 10 client.0 _ll_register_callbacks cb 0x55941a435110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:31.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.256+0000 7fdb98cf5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a155980 0x55941a541020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.256+0000 7fdb98cf5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a155980 0x55941a541020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39012/0 (socket says 172.21.15.67:39012) 2024-07-31T15:04:31.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.256+0000 7fdb98cf5640 1 -- 172.21.15.67:0/1228550836 learned_addr learned my addr 172.21.15.67:0/1228550836 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:31.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.256+0000 7fdb98cf5640 1 --2- 172.21.15.67:0/1228550836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a17e840 0x55941a541aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.256+0000 7fdb937fe640 1 --2- 172.21.15.67:0/1228550836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a15db60 0x55941a541560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.256+0000 7fdb98cf5640 1 -- 172.21.15.67:0/1228550836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a15db60 msgr2=0x55941a541560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:31.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.256+0000 7fdb98cf5640 1 --2- 172.21.15.67:0/1228550836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a15db60 0x55941a541560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.256+0000 7fdb98cf5640 1 -- 172.21.15.67:0/1228550836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a155980 msgr2=0x55941a541020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:31.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.256+0000 7fdb98cf5640 1 --2- 172.21.15.67:0/1228550836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a155980 0x55941a541020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.256+0000 7fdb98cf5640 1 -- 172.21.15.67:0/1228550836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55941a42f480 con 0x55941a17e840 2024-07-31T15:04:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.262+0000 7fdb98cf5640 1 --2- 172.21.15.67:0/1228550836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a17e840 0x55941a541aa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c031fd0 tx=0x7fdb8c00f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=986d0923db34784d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.262+0000 7fdb737fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:04:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.263+0000 7fdb737fe640 1 -- 172.21.15.67:0/1228550836 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c004230 con 0x55941a17e840 2024-07-31T15:04:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.263+0000 7fdb737fe640 1 -- 172.21.15.67:0/1228550836 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fdb8c0043d0 con 0x55941a17e840 2024-07-31T15:04:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.263+0000 7fdba0ca9740 1 -- 172.21.15.67:0/1228550836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55941a49b6d0 con 0x55941a17e840 2024-07-31T15:04:31.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.268+0000 7fdb737fe640 1 -- 172.21.15.67:0/1228550836 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fdb8c0045d0 con 0x55941a17e840 2024-07-31T15:04:31.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.273+0000 7fdba0ca9740 10 client.27802 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:31.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.273+0000 7fdba0ca9740 1 -- 172.21.15.67:0/1228550836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55941a1f8b90 con 0x55941a17e840 2024-07-31T15:04:31.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.282+0000 7fdb737fe640 1 -- 172.21.15.67:0/1228550836 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55941a49b6d0 con 0x55941a17e840 2024-07-31T15:04:31.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.282+0000 7fdb737fe640 1 -- 172.21.15.67:0/1228550836 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fdb8c059450 con 0x55941a17e840 2024-07-31T15:04:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.283+0000 7f8dab5db740 1 Processor -- start 2024-07-31T15:04:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.284+0000 7f8dab5db740 1 -- start start 2024-07-31T15:04:31.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.284+0000 7fdba0ca9740 10 client.27802 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:31.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.284+0000 7fdba0ca9740 10 client.27802 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:04:31.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.284+0000 7fdba0ca9740 20 client.27802 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:31.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.284+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/1228550836 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55941a13d1c0 0x55941a552fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.284+0000 7fdba0ca9740 4 client.27802 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:04:31.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.284+0000 7fdba0ca9740 1 -- 172.21.15.67:0/1228550836 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55941a497170 con 0x55941a13d1c0 2024-07-31T15:04:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.284+0000 7f8dab5db740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f4648d80 0x5581f4a38140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.288+0000 7f8dab5db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46612f0 0x5581f4a319a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.288+0000 7f8dab5db740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46694d0 0x5581f4a31ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.288+0000 7fdb937fe640 1 --2- 172.21.15.67:0/1228550836 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55941a13d1c0 0x55941a552fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.288+0000 7f8da23a2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f4648d80 0x5581f4a38140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.288+0000 7f8da1ba1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46612f0 0x5581f4a319a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.288+0000 7f8dab5db740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8000ffb0 con 0x5581f4648d80 2024-07-31T15:04:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.288+0000 7f8da1ba1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46612f0 0x5581f4a319a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50862/0 (socket says 172.21.15.67:50862) 2024-07-31T15:04:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.288+0000 7f8da1ba1640 1 -- 172.21.15.67:0/1152498263 learned_addr learned my addr 172.21.15.67:0/1152498263 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.289+0000 7f8da2ba3640 1 --2- 172.21.15.67:0/1152498263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46694d0 0x5581f4a31ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.292+0000 7f8dab5db740 1 -- 172.21.15.67:0/1152498263 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8d80011af0 con 0x5581f46694d0 2024-07-31T15:04:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.292+0000 7f8dab5db740 1 -- 172.21.15.67:0/1152498263 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8d80010680 con 0x5581f46612f0 2024-07-31T15:04:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.292+0000 7f8da1ba1640 1 -- 172.21.15.67:0/1152498263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46694d0 msgr2=0x5581f4a31ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.292+0000 7f8da1ba1640 1 --2- 172.21.15.67:0/1152498263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46694d0 0x5581f4a31ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.292+0000 7f8da1ba1640 1 -- 172.21.15.67:0/1152498263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f4648d80 msgr2=0x5581f4a38140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.292+0000 7f8da1ba1640 1 --2- 172.21.15.67:0/1152498263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f4648d80 0x5581f4a38140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.292+0000 7f8da1ba1640 1 -- 172.21.15.67:0/1152498263 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581f4a32810 con 0x5581f46612f0 2024-07-31T15:04:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.292+0000 7f8da23a2640 1 --2- 172.21.15.67:0/1152498263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f4648d80 0x5581f4a38140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.293+0000 7f8da1ba1640 1 --2- 172.21.15.67:0/1152498263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46612f0 0x5581f4a319a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c02a930 tx=0x7f8d8c03f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c16785f8617df88f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.293+0000 7f8d87fff640 1 -- 172.21.15.67:0/1152498263 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c037040 con 0x5581f46612f0 2024-07-31T15:04:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.293+0000 7f8d87fff640 1 -- 172.21.15.67:0/1152498263 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d8c042040 con 0x5581f46612f0 2024-07-31T15:04:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.295+0000 7f8dab5db740 1 -- 172.21.15.67:0/1152498263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46612f0 msgr2=0x5581f4a319a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.295+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1152498263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46612f0 0x5581f4a319a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c02a930 tx=0x7f8d8c03f000 comp rx=0 tx=0).stop 2024-07-31T15:04:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.295+0000 7f8dab5db740 1 -- 172.21.15.67:0/1152498263 shutdown_connections 2024-07-31T15:04:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.295+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1152498263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46694d0 0x5581f4a31ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.295+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1152498263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f46612f0 0x5581f4a319a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.295+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1152498263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f4648d80 0x5581f4a38140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.295+0000 7f8dab5db740 1 -- 172.21.15.67:0/1152498263 >> 172.21.15.67:0/1152498263 conn(0x5581f467e300 msgr2=0x5581f4a253a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.295+0000 7f8dab5db740 1 -- 172.21.15.67:0/1152498263 shutdown_connections 2024-07-31T15:04:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.295+0000 7f8dab5db740 1 -- 172.21.15.67:0/1152498263 wait complete. 2024-07-31T15:04:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.295+0000 7f8dab5db740 1 -- start start 2024-07-31T15:04:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.296+0000 7f8dab5db740 1 Processor -- start 2024-07-31T15:04:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.296+0000 7f8dab5db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f4648d80 0x5581f4a6c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.296+0000 7f8dab5db740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f46612f0 0x5581f4a6cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.297+0000 7f8dab5db740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46694d0 0x5581f4a5fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.297+0000 7f8dab5db740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8d80011420 con 0x5581f46612f0 2024-07-31T15:04:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.297+0000 7f8da23a2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f4648d80 0x5581f4a6c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.297+0000 7f8dab5db740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8d80010d50 con 0x5581f46694d0 2024-07-31T15:04:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.297+0000 7f8da23a2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f4648d80 0x5581f4a6c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50868/0 (socket says 172.21.15.67:50868) 2024-07-31T15:04:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.297+0000 7f8dab5db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8000dfb0 con 0x5581f4648d80 2024-07-31T15:04:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.297+0000 7f8da23a2640 1 -- 172.21.15.67:0/1165832898 learned_addr learned my addr 172.21.15.67:0/1165832898 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.297+0000 7f8da1ba1640 1 --2- 172.21.15.67:0/1165832898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f46612f0 0x5581f4a6cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.297+0000 7f8dab5db740 10 client.0 _ll_register_callbacks cb 0x7f8d940c0f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.297+0000 7f8da2ba3640 1 --2- 172.21.15.67:0/1165832898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46694d0 0x5581f4a5fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.297+0000 7f8da2ba3640 1 -- 172.21.15.67:0/1165832898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f4648d80 msgr2=0x5581f4a6c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.297+0000 7f8da2ba3640 1 --2- 172.21.15.67:0/1165832898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f4648d80 0x5581f4a6c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.297+0000 7f8da2ba3640 1 -- 172.21.15.67:0/1165832898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f46612f0 msgr2=0x5581f4a6cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.298+0000 7f8da2ba3640 1 --2- 172.21.15.67:0/1165832898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f46612f0 0x5581f4a6cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.298+0000 7f8da2ba3640 1 -- 172.21.15.67:0/1165832898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581f4a60080 con 0x5581f46694d0 2024-07-31T15:04:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.298+0000 7f8da1ba1640 1 --2- 172.21.15.67:0/1165832898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f46612f0 0x5581f4a6cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.298+0000 7f8da2ba3640 1 --2- 172.21.15.67:0/1165832898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46694d0 0x5581f4a5fac0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8d980099b0 tx=0x7f8d98011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7410d7d0ab4fd311 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:31.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.297+0000 7fdb937fe640 1 --2- 172.21.15.67:0/1228550836 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55941a13d1c0 0x55941a552fb0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d70e61aa88cf44 server_cookie=5715f95ceaf60bcf in_seq=0 out_seq=0 2024-07-31T15:04:31.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.298+0000 7fdb737fe640 10 client.27802 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:31.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.299+0000 7fdb737fe640 1 -- 172.21.15.67:0/1228550836 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55941a497170 con 0x55941a13d1c0 2024-07-31T15:04:31.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.299+0000 7fdb737fe640 10 client.27802 handle_command_reply: tid=0 2024-07-31T15:04:31.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.299+0000 7fdba0ca9740 1 client.27802 shutdown 2024-07-31T15:04:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.299+0000 7f8d85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:04:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.299+0000 7f8d85ffb640 1 -- 172.21.15.67:0/1165832898 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98005020 con 0x5581f46694d0 2024-07-31T15:04:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.299+0000 7f8d85ffb640 1 -- 172.21.15.67:0/1165832898 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f8d980051c0 con 0x5581f46694d0 2024-07-31T15:04:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.299+0000 7f8dab5db740 1 -- 172.21.15.67:0/1165832898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d8c042040 con 0x5581f46694d0 2024-07-31T15:04:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.300+0000 7f8d85ffb640 1 -- 172.21.15.67:0/1165832898 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8d9802a050 con 0x5581f46694d0 2024-07-31T15:04:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.300+0000 7f8dab5db740 10 client.27869 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.300+0000 7f8dab5db740 1 -- 172.21.15.67:0/1165832898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5581f4a61370 con 0x5581f46694d0 2024-07-31T15:04:31.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.300+0000 7fdba0ca9740 1 -- 172.21.15.67:0/1228550836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a17e840 msgr2=0x55941a541aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:31.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.300+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/1228550836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a17e840 0x55941a541aa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c031fd0 tx=0x7fdb8c00f040 comp rx=0 tx=0).stop 2024-07-31T15:04:31.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.300+0000 7fdba0ca9740 1 -- 172.21.15.67:0/1228550836 shutdown_connections 2024-07-31T15:04:31.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.300+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/1228550836 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55941a13d1c0 0x55941a552fb0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.300+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/1228550836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a17e840 0x55941a541aa0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.300+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/1228550836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a15db60 0x55941a541560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.300+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/1228550836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a155980 0x55941a541020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.300+0000 7fdba0ca9740 1 -- 172.21.15.67:0/1228550836 >> 172.21.15.67:0/1228550836 conn(0x55941a172990 msgr2=0x55941a520ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:31.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.300+0000 7fdb737fe640 0 client.27802 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:31.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.300+0000 7fdba0ca9740 1 -- 172.21.15.67:0/1228550836 shutdown_connections 2024-07-31T15:04:31.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.300+0000 7fdba0ca9740 1 -- 172.21.15.67:0/1228550836 wait complete. 2024-07-31T15:04:31.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.301+0000 7fdba0ca9740 20 client.27802 trim_cache size 0 max 16384 2024-07-31T15:04:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.302+0000 7f8d85ffb640 1 -- 172.21.15.67:0/1165832898 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d8c042040 con 0x5581f46694d0 2024-07-31T15:04:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.302+0000 7f8d85ffb640 1 -- 172.21.15.67:0/1165832898 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f8d9801d3f0 con 0x5581f46694d0 2024-07-31T15:04:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.302+0000 7f8dab5db740 10 client.27869 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.302+0000 7f8dab5db740 10 client.27869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.302+0000 7f8dab5db740 20 client.27869 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.302+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1165832898 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5581f4ac2b80 0x5581f4ac4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.302+0000 7f8dab5db740 4 client.27869 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:04:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.302+0000 7f8dab5db740 1 -- 172.21.15.67:0/1165832898 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581f499d4e0 con 0x5581f4ac2b80 2024-07-31T15:04:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.303+0000 7f8da23a2640 1 --2- 172.21.15.67:0/1165832898 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5581f4ac2b80 0x5581f4ac4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.303+0000 7f8d85ffb640 10 client.27869 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:31.303+0000 7f8da23a2640 1 --2- 172.21.15.67:0/1165832898 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5581f4ac2b80 0x5581f4ac4fd0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=192469b2669d408a server_cookie=f9e2a78ab74461a3 in_seq=0 out_seq=0 2024-07-31T15:04:31.339 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:31 smithi067 ceph-mon[32750]: pgmap v825: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 928 KiB/s wr, 92 op/s 2024-07-31T15:04:31.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.347+0000 7fdba0ca9740 1 Processor -- start 2024-07-31T15:04:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.347+0000 7fdba0ca9740 1 -- start start 2024-07-31T15:04:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.347+0000 7fdba0ca9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a155980 0x55941a526120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.348+0000 7fdba0ca9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a15db60 0x55941a555fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.348+0000 7fdba0ca9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a17e840 0x55941a556510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.348+0000 7fdba0ca9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941c7b90 con 0x55941a17e840 2024-07-31T15:04:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.348+0000 7fdba0ca9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941c96d0 con 0x55941a15db60 2024-07-31T15:04:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.348+0000 7fdba0ca9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941c8260 con 0x55941a155980 2024-07-31T15:04:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.349+0000 7fdb98cf5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a155980 0x55941a526120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.349+0000 7fdb98cf5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a155980 0x55941a526120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50884/0 (socket says 172.21.15.67:50884) 2024-07-31T15:04:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.349+0000 7fdb98cf5640 1 -- 172.21.15.67:0/1578669986 learned_addr learned my addr 172.21.15.67:0/1578669986 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.350+0000 7fdb98cf5640 1 --2- 172.21.15.67:0/1578669986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a17e840 0x55941a556510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.350+0000 7fdb937fe640 1 --2- 172.21.15.67:0/1578669986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a15db60 0x55941a555fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.350+0000 7fdb98cf5640 1 -- 172.21.15.67:0/1578669986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a155980 msgr2=0x55941a526120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:31.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.350+0000 7fdb98cf5640 1 --2- 172.21.15.67:0/1578669986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a155980 0x55941a526120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.350+0000 7fdb98cf5640 1 -- 172.21.15.67:0/1578669986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a15db60 msgr2=0x55941a555fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:31.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.350+0000 7fdb98cf5640 1 --2- 172.21.15.67:0/1578669986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a15db60 0x55941a555fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.350+0000 7fdb98cf5640 1 -- 172.21.15.67:0/1578669986 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55941a525820 con 0x55941a17e840 2024-07-31T15:04:31.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.351+0000 7fdb98cf5640 1 --2- 172.21.15.67:0/1578669986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a17e840 0x55941a556510 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00aa80 tx=0x7fdb8c00a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d947cd7a8cea8446 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:31.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.352+0000 7fdb79ffb640 1 -- 172.21.15.67:0/1578669986 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c061070 con 0x55941a17e840 2024-07-31T15:04:31.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.353+0000 7fdb79ffb640 1 -- 172.21.15.67:0/1578669986 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb8c054070 con 0x55941a17e840 2024-07-31T15:04:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.354+0000 7fdba0ca9740 1 -- 172.21.15.67:0/1578669986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a17e840 msgr2=0x55941a556510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:31.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.354+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/1578669986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a17e840 0x55941a556510 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00aa80 tx=0x7fdb8c00a5e0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.355+0000 7fdba0ca9740 1 -- 172.21.15.67:0/1578669986 shutdown_connections 2024-07-31T15:04:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.355+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/1578669986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a17e840 0x55941a556510 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.355+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/1578669986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a15db60 0x55941a555fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.355+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/1578669986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a155980 0x55941a526120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.355+0000 7fdba0ca9740 1 -- 172.21.15.67:0/1578669986 >> 172.21.15.67:0/1578669986 conn(0x55941a172990 msgr2=0x55941a525a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.356+0000 7fdba0ca9740 1 -- 172.21.15.67:0/1578669986 shutdown_connections 2024-07-31T15:04:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.356+0000 7fdba0ca9740 1 -- 172.21.15.67:0/1578669986 wait complete. 2024-07-31T15:04:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.356+0000 7fdba0ca9740 1 -- start start 2024-07-31T15:04:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.357+0000 7fdba0ca9740 1 Processor -- start 2024-07-31T15:04:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.358+0000 7fdba0ca9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a155980 0x55941a553750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.358+0000 7fdba0ca9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a15db60 0x55941a553d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.358+0000 7fdba0ca9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a112570 0x55941a5b94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.358+0000 7fdba0ca9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941c9000 con 0x55941a15db60 2024-07-31T15:04:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.358+0000 7fdba0ca9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941c8930 con 0x55941a155980 2024-07-31T15:04:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.358+0000 7fdba0ca9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941c5b90 con 0x55941a112570 2024-07-31T15:04:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.358+0000 7fdba0ca9740 10 client.0 _ll_register_callbacks cb 0x7fdb8c003ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.358+0000 7fdb937fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a15db60 0x55941a553d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.359+0000 7fdb937fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a15db60 0x55941a553d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39048/0 (socket says 172.21.15.67:39048) 2024-07-31T15:04:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.359+0000 7fdb937fe640 1 -- 172.21.15.67:0/2361158393 learned_addr learned my addr 172.21.15.67:0/2361158393 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.360+0000 7fdb937fe640 1 -- 172.21.15.67:0/2361158393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a112570 msgr2=0x55941a5b94f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:04:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.360+0000 7fdb937fe640 1 --2- 172.21.15.67:0/2361158393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a112570 0x55941a5b94f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.360+0000 7fdb937fe640 1 -- 172.21.15.67:0/2361158393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a155980 msgr2=0x55941a553750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:04:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.360+0000 7fdb937fe640 1 --2- 172.21.15.67:0/2361158393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a155980 0x55941a553750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.360+0000 7fdb937fe640 1 -- 172.21.15.67:0/2361158393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55941a5b9b60 con 0x55941a15db60 2024-07-31T15:04:31.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.361+0000 7fdb937fe640 1 --2- 172.21.15.67:0/2361158393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a15db60 0x55941a553d10 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fdb9406ec50 tx=0x7fdb88004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e475b8a447ac36f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:31.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.362+0000 7fdb737fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:04:31.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.362+0000 7fdb737fe640 1 -- 172.21.15.67:0/2361158393 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88022640 con 0x55941a15db60 2024-07-31T15:04:31.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.362+0000 7fdba0ca9740 1 -- 172.21.15.67:0/2361158393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55941a4948b0 con 0x55941a15db60 2024-07-31T15:04:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.362+0000 7fdb737fe640 1 -- 172.21.15.67:0/2361158393 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fdb88010430 con 0x55941a15db60 2024-07-31T15:04:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.363+0000 7fdb737fe640 1 -- 172.21.15.67:0/2361158393 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fdb880227e0 con 0x55941a15db60 2024-07-31T15:04:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.363+0000 7fdba0ca9740 10 client.18312 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.363+0000 7fdba0ca9740 1 -- 172.21.15.67:0/2361158393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55941a5baec0 con 0x55941a15db60 2024-07-31T15:04:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.364+0000 7fdb737fe640 1 -- 172.21.15.67:0/2361158393 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55941a4948b0 con 0x55941a15db60 2024-07-31T15:04:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.364+0000 7fdb737fe640 1 -- 172.21.15.67:0/2361158393 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fdb88020070 con 0x55941a15db60 2024-07-31T15:04:31.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.366+0000 7fdba0ca9740 10 client.18312 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:31.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.366+0000 7fdba0ca9740 10 client.18312 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:04:31.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.366+0000 7fdba0ca9740 20 client.18312 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:31.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.366+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/2361158393 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55941a5bb700 0x55941a5bdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:31.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.366+0000 7fdba0ca9740 4 client.18312 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.3, "await_for": 45.0}] 2024-07-31T15:04:31.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.366+0000 7fdba0ca9740 1 -- 172.21.15.67:0/2361158393 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.3, "await_for": 45.0}) -- 0x55941a40c3d0 con 0x55941a5bb700 2024-07-31T15:04:31.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.367+0000 7fdb98cf5640 1 --2- 172.21.15.67:0/2361158393 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55941a5bb700 0x55941a5bdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:31.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.367+0000 7fdb98cf5640 1 --2- 172.21.15.67:0/2361158393 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55941a5bb700 0x55941a5bdb20 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c640ff3e916a1f6a server_cookie=8ed55cee783d467f in_seq=0 out_seq=0 2024-07-31T15:04:31.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:31.367+0000 7fdb737fe640 10 client.18312 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:31.382 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/early_ioremap.o 2024-07-31T15:04:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:31 smithi077 ceph-mon[37327]: pgmap v825: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 928 KiB/s wr, 92 op/s 2024-07-31T15:04:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:31 smithi073 ceph-mon[36714]: pgmap v825: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 928 KiB/s wr, 92 op/s 2024-07-31T15:04:32.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.872+0000 7f3b4a818640 1 -- 172.21.15.67:0/3721770702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b44109820 msgr2=0x7f3b44110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:32.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.872+0000 7f3b4a818640 1 --2- 172.21.15.67:0/3721770702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b44109820 0x7f3b44110050 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009ca0 tx=0x7f3b3c02a8c0 comp rx=0 tx=0).stop 2024-07-31T15:04:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.872+0000 7f3b4a818640 1 -- 172.21.15.67:0/3721770702 shutdown_connections 2024-07-31T15:04:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.872+0000 7f3b4a818640 1 --2- 172.21.15.67:0/3721770702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b44109820 0x7f3b44110050 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.872+0000 7f3b4a818640 1 --2- 172.21.15.67:0/3721770702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b44108ed0 0x7f3b441092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.872+0000 7f3b4a818640 1 --2- 172.21.15.67:0/3721770702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b44103590 0x7f3b441039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.872+0000 7f3b4a818640 1 -- 172.21.15.67:0/3721770702 >> 172.21.15.67:0/3721770702 conn(0x7f3b4407b280 msgr2=0x7f3b4407b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.872+0000 7f3b4a818640 1 -- 172.21.15.67:0/3721770702 shutdown_connections 2024-07-31T15:04:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.872+0000 7f3b4a818640 1 -- 172.21.15.67:0/3721770702 wait complete. 2024-07-31T15:04:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.873+0000 7f3b4a818640 1 Processor -- start 2024-07-31T15:04:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.873+0000 7f3b4a818640 1 -- start start 2024-07-31T15:04:32.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b4a818640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b44103590 0x7f3b44075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:32.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b4a818640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b44108ed0 0x7f3b44075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:32.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b4a818640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b44109820 0x7f3b44075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:32.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b4a818640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441134f0 con 0x7f3b44109820 2024-07-31T15:04:32.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b4a818640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3b44113370 con 0x7f3b44103590 2024-07-31T15:04:32.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b4a818640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3b44113670 con 0x7f3b44108ed0 2024-07-31T15:04:32.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b43fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b44103590 0x7f3b44075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:32.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b43fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b44103590 0x7f3b44075140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56786/0 (socket says 172.21.15.67:56786) 2024-07-31T15:04:32.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b43fff640 1 -- 172.21.15.67:0/1856911453 learned_addr learned my addr 172.21.15.67:0/1856911453 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:32.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b437fe640 1 --2- 172.21.15.67:0/1856911453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b44108ed0 0x7f3b44075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:32.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b48db5640 1 --2- 172.21.15.67:0/1856911453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b44109820 0x7f3b44075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:32.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b43fff640 1 -- 172.21.15.67:0/1856911453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b44108ed0 msgr2=0x7f3b44075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:32.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b43fff640 1 --2- 172.21.15.67:0/1856911453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b44108ed0 0x7f3b44075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:32.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b43fff640 1 -- 172.21.15.67:0/1856911453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b44109820 msgr2=0x7f3b44075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b43fff640 1 --2- 172.21.15.67:0/1856911453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b44109820 0x7f3b44075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b43fff640 1 -- 172.21.15.67:0/1856911453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b441a8180 con 0x7f3b44103590 2024-07-31T15:04:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.874+0000 7f3b43fff640 1 --2- 172.21.15.67:0/1856911453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b44103590 0x7f3b44075140 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400d950 tx=0x7f3b3400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.875+0000 7f3b417fa640 1 -- 172.21.15.67:0/1856911453 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34017030 con 0x7f3b44103590 2024-07-31T15:04:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.875+0000 7f3b417fa640 1 -- 172.21.15.67:0/1856911453 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b34004020 con 0x7f3b44103590 2024-07-31T15:04:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.875+0000 7f3b4a818640 1 -- 172.21.15.67:0/1856911453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b441a8470 con 0x7f3b44103590 2024-07-31T15:04:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.875+0000 7f3b417fa640 1 -- 172.21.15.67:0/1856911453 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340047a0 con 0x7f3b44103590 2024-07-31T15:04:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.875+0000 7f3b4a818640 1 -- 172.21.15.67:0/1856911453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b441a89b0 con 0x7f3b44103590 2024-07-31T15:04:32.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.876+0000 7f3b4a818640 1 -- 172.21.15.67:0/1856911453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b441039a0 con 0x7f3b44103590 2024-07-31T15:04:32.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.877+0000 7f3b417fa640 1 -- 172.21.15.67:0/1856911453 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3b34009a30 con 0x7f3b44103590 2024-07-31T15:04:32.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.878+0000 7f3b417fa640 1 --2- 172.21.15.67:0/1856911453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3b2c0b3d40 0x7f3b2c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:32.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.878+0000 7f3b437fe640 1 --2- 172.21.15.67:0/1856911453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3b2c0b3d40 0x7f3b2c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:32.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.878+0000 7f3b417fa640 1 -- 172.21.15.67:0/1856911453 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f3b340e4430 con 0x7f3b44103590 2024-07-31T15:04:32.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.879+0000 7f3b437fe640 1 --2- 172.21.15.67:0/1856911453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3b2c0b3d40 0x7f3b2c0b6250 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3b44108b20 tx=0x7f3b38005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:32.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:32.881+0000 7f3b417fa640 1 -- 172.21.15.67:0/1856911453 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f3b340ab3d0 con 0x7f3b44103590 2024-07-31T15:04:33.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:33.114+0000 7f3b4a818640 1 -- 172.21.15.67:0/1856911453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b44076440 con 0x7f3b44103590 2024-07-31T15:04:33.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:33.115+0000 7f3b417fa640 1 -- 172.21.15.67:0/1856911453 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+11959 (secure 0 0 0) 0x7f3b340aacd0 con 0x7f3b44103590 2024-07-31T15:04:33.116 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:04:33.116 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":63,"btime":"2024-07-31T15:03:07:099477+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:03:06.671820+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:04:33.117 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 63 2024-07-31T15:04:33.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:33.117+0000 7f3b4a818640 1 -- 172.21.15.67:0/1856911453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3b2c0b3d40 msgr2=0x7f3b2c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:33.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:33.117+0000 7f3b4a818640 1 --2- 172.21.15.67:0/1856911453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3b2c0b3d40 0x7f3b2c0b6250 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3b44108b20 tx=0x7f3b38005f90 comp rx=0 tx=0).stop 2024-07-31T15:04:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:33.117+0000 7f3b4a818640 1 -- 172.21.15.67:0/1856911453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b44103590 msgr2=0x7f3b44075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:33.117+0000 7f3b4a818640 1 --2- 172.21.15.67:0/1856911453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b44103590 0x7f3b44075140 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400d950 tx=0x7f3b3400de10 comp rx=0 tx=0).stop 2024-07-31T15:04:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:33.117+0000 7f3b4a818640 1 -- 172.21.15.67:0/1856911453 shutdown_connections 2024-07-31T15:04:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:33.117+0000 7f3b4a818640 1 --2- 172.21.15.67:0/1856911453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3b2c0b3d40 0x7f3b2c0b6250 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:33.117+0000 7f3b4a818640 1 --2- 172.21.15.67:0/1856911453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b44109820 0x7f3b44075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:33.117+0000 7f3b4a818640 1 --2- 172.21.15.67:0/1856911453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b44108ed0 0x7f3b44075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:33.117+0000 7f3b4a818640 1 --2- 172.21.15.67:0/1856911453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b44103590 0x7f3b44075140 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:33.117+0000 7f3b4a818640 1 -- 172.21.15.67:0/1856911453 >> 172.21.15.67:0/1856911453 conn(0x7f3b4407b280 msgr2=0x7f3b44106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:33.117+0000 7f3b4a818640 1 -- 172.21.15.67:0/1856911453 shutdown_connections 2024-07-31T15:04:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:33.117+0000 7f3b4a818640 1 -- 172.21.15.67:0/1856911453 wait complete. 2024-07-31T15:04:33.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:33 smithi067 ceph-mon[32750]: pgmap v826: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 833 KiB/s wr, 78 op/s 2024-07-31T15:04:33.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:33 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1856911453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:04:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:33 smithi077 ceph-mon[37327]: pgmap v826: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 833 KiB/s wr, 78 op/s 2024-07-31T15:04:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:33 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1856911453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:04:33.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:33 smithi073 ceph-mon[36714]: pgmap v826: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 833 KiB/s wr, 78 op/s 2024-07-31T15:04:33.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:33 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1856911453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:04:33.899 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T15:04:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.899+0000 7f8d85ffb640 1 -- 172.21.15.67:0/1165832898 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581f499d4e0 con 0x5581f4ac2b80 2024-07-31T15:04:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.899+0000 7f8d85ffb640 10 client.27869 handle_command_reply: tid=0 2024-07-31T15:04:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.899+0000 7f8dab5db740 1 client.27869 shutdown 2024-07-31T15:04:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.900+0000 7f8dab5db740 1 -- 172.21.15.67:0/1165832898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46694d0 msgr2=0x5581f4a5fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.900+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1165832898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46694d0 0x5581f4a5fac0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8d980099b0 tx=0x7f8d98011810 comp rx=0 tx=0).stop 2024-07-31T15:04:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.900+0000 7f8dab5db740 1 -- 172.21.15.67:0/1165832898 shutdown_connections 2024-07-31T15:04:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.900+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1165832898 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5581f4ac2b80 0x5581f4ac4fd0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.900+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1165832898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5581f46694d0 0x5581f4a5fac0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.900+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1165832898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5581f46612f0 0x5581f4a6cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.900+0000 7f8dab5db740 1 --2- 172.21.15.67:0/1165832898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581f4648d80 0x5581f4a6c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.900+0000 7f8d85ffb640 0 client.27869 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.900+0000 7f8dab5db740 1 -- 172.21.15.67:0/1165832898 >> 172.21.15.67:0/1165832898 conn(0x5581f467e300 msgr2=0x5581f4a29ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:04:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:04:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:04:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:04:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.900+0000 7f8dab5db740 1 -- 172.21.15.67:0/1165832898 shutdown_connections 2024-07-31T15:04:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.900+0000 7f8dab5db740 1 -- 172.21.15.67:0/1165832898 wait complete. 2024-07-31T15:04:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.900+0000 7f8dab5db740 20 client.27869 trim_cache size 0 max 16384 2024-07-31T15:04:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.901+0000 7f8da4606640 1 -- 172.21.15.67:0/3486051881 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8d880b3ed0 msgr2=0x7f8d880b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.901+0000 7f8da4606640 1 --2- 172.21.15.67:0/3486051881 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8d880b3ed0 0x7f8d880b63e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c02a8b0 tx=0x7f8d8c02a840 comp rx=0 tx=0).stop 2024-07-31T15:04:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.901+0000 7f8da4606640 1 -- 172.21.15.67:0/3486051881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c10a920 msgr2=0x7f8d9c1a2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.901+0000 7f8da4606640 1 --2- 172.21.15.67:0/3486051881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c10a920 0x7f8d9c1a2710 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400c420 tx=0x7f8d94009f90 comp rx=0 tx=0).stop 2024-07-31T15:04:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.901+0000 7f8da4606640 1 -- 172.21.15.67:0/3486051881 shutdown_connections 2024-07-31T15:04:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.901+0000 7f8da4606640 1 --2- 172.21.15.67:0/3486051881 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8d880b3ed0 0x7f8d880b63e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.901+0000 7f8da4606640 1 --2- 172.21.15.67:0/3486051881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d9c110610 0x7f8d9c1a7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.901+0000 7f8da4606640 1 --2- 172.21.15.67:0/3486051881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d9c10fcc0 0x7f8d9c1a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.901+0000 7f8da4606640 1 --2- 172.21.15.67:0/3486051881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9c10a920 0x7f8d9c1a2710 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.901+0000 7f8da4606640 1 -- 172.21.15.67:0/3486051881 >> 172.21.15.67:0/3486051881 conn(0x7f8d9c0707c0 msgr2=0x7f8d9c10ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.901+0000 7f8da4606640 1 -- 172.21.15.67:0/3486051881 shutdown_connections 2024-07-31T15:04:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:33.901+0000 7f8da4606640 1 -- 172.21.15.67:0/3486051881 wait complete. 2024-07-31T15:04:34.181 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:04:35.445 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:35 smithi073 ceph-mon[36714]: pgmap v827: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 819 KiB/s wr, 77 op/s 2024-07-31T15:04:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:35 smithi077 ceph-mon[37327]: pgmap v827: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 819 KiB/s wr, 77 op/s 2024-07-31T15:04:35.511 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:35 smithi067 ceph-mon[32750]: pgmap v827: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 819 KiB/s wr, 77 op/s 2024-07-31T15:04:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.948+0000 7f925ce20640 1 -- 172.21.15.67:0/1386380148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f925810d880 msgr2=0x7f925810fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.948+0000 7f925ce20640 1 --2- 172.21.15.67:0/1386380148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f925810d880 0x7f925810fd20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9248009ca0 tx=0x7f924802a8c0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.948+0000 7f925ce20640 1 -- 172.21.15.67:0/1386380148 shutdown_connections 2024-07-31T15:04:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.948+0000 7f925ce20640 1 --2- 172.21.15.67:0/1386380148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f925810d880 0x7f925810fd20 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.948+0000 7f925ce20640 1 --2- 172.21.15.67:0/1386380148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9258104d00 0x7f925810d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.948+0000 7f925ce20640 1 --2- 172.21.15.67:0/1386380148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92581042d0 0x7f92581046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.948+0000 7f925ce20640 1 -- 172.21.15.67:0/1386380148 >> 172.21.15.67:0/1386380148 conn(0x7f9258079160 msgr2=0x7f9258079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.948+0000 7f925ce20640 1 -- 172.21.15.67:0/1386380148 shutdown_connections 2024-07-31T15:04:35.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.948+0000 7f925ce20640 1 -- 172.21.15.67:0/1386380148 wait complete. 2024-07-31T15:04:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.949+0000 7f925ce20640 1 Processor -- start 2024-07-31T15:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.949+0000 7f925ce20640 1 -- start start 2024-07-31T15:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.949+0000 7f925ce20640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92581042d0 0x7f92581021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.949+0000 7f925ce20640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9258104d00 0x7f9258102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.949+0000 7f925ce20640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f925810d880 0x7f9258102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.949+0000 7f925ce20640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9258113490 con 0x7f9258104d00 2024-07-31T15:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.949+0000 7f925ce20640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9258113310 con 0x7f92581042d0 2024-07-31T15:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.949+0000 7f925ce20640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9258113610 con 0x7f925810d880 2024-07-31T15:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f925659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92581042d0 0x7f92581021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f9255d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9258104d00 0x7f9258102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f9256d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f925810d880 0x7f9258102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f925659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92581042d0 0x7f92581021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56810/0 (socket says 172.21.15.67:56810) 2024-07-31T15:04:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f9255d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9258104d00 0x7f9258102720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39080/0 (socket says 172.21.15.67:39080) 2024-07-31T15:04:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f9256d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f925810d880 0x7f9258102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50932/0 (socket says 172.21.15.67:50932) 2024-07-31T15:04:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f925659c640 1 -- 172.21.15.67:0/2698164768 learned_addr learned my addr 172.21.15.67:0/2698164768 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f925659c640 1 -- 172.21.15.67:0/2698164768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f925810d880 msgr2=0x7f9258102c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f925659c640 1 --2- 172.21.15.67:0/2698164768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f925810d880 0x7f9258102c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f925659c640 1 -- 172.21.15.67:0/2698164768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9258104d00 msgr2=0x7f9258102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f925659c640 1 --2- 172.21.15.67:0/2698164768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9258104d00 0x7f9258102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f925659c640 1 -- 172.21.15.67:0/2698164768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92581a8200 con 0x7f92581042d0 2024-07-31T15:04:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f9255d9b640 1 --2- 172.21.15.67:0/2698164768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9258104d00 0x7f9258102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f925659c640 1 --2- 172.21.15.67:0/2698164768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92581042d0 0x7f92581021e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f925000e9c0 tx=0x7f925000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f924f7fe640 1 -- 172.21.15.67:0/2698164768 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925000cd80 con 0x7f92581042d0 2024-07-31T15:04:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f925ce20640 1 -- 172.21.15.67:0/2698164768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92581a84f0 con 0x7f92581042d0 2024-07-31T15:04:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f924f7fe640 1 -- 172.21.15.67:0/2698164768 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92500035f0 con 0x7f92581042d0 2024-07-31T15:04:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f924f7fe640 1 -- 172.21.15.67:0/2698164768 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250010690 con 0x7f92581042d0 2024-07-31T15:04:35.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.950+0000 7f925ce20640 1 -- 172.21.15.67:0/2698164768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92581a8a80 con 0x7f92581042d0 2024-07-31T15:04:35.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.953+0000 7f9263371740 1 Processor -- start 2024-07-31T15:04:35.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.953+0000 7f9263371740 1 -- start start 2024-07-31T15:04:35.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.953+0000 7f9263371740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc50070 0x55702fff5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.953+0000 7f9263371740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc271b0 0x55702fff5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.953+0000 7f9263371740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x55702fff5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.953+0000 7f9263371740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55702ff6bbb0 con 0x55702fc50070 2024-07-31T15:04:35.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.953+0000 7f9263371740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55702ff5b0b0 con 0x55702fc2f390 2024-07-31T15:04:35.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.953+0000 7f9263371740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55702ff5f0f0 con 0x55702fc271b0 2024-07-31T15:04:35.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f924f7fe640 1 -- 172.21.15.67:0/2698164768 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f92500037a0 con 0x7f92581042d0 2024-07-31T15:04:35.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f9255d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x55702fff5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f9256d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc50070 0x55702fff5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f9255d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x55702fff5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56822/0 (socket says 172.21.15.67:56822) 2024-07-31T15:04:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f9256d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc50070 0x55702fff5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39096/0 (socket says 172.21.15.67:39096) 2024-07-31T15:04:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f9255d9b640 1 -- 172.21.15.67:0/4199989626 learned_addr learned my addr 172.21.15.67:0/4199989626 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f925659c640 1 --2- 172.21.15.67:0/4199989626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc271b0 0x55702fff5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f9255d9b640 1 -- 172.21.15.67:0/4199989626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc271b0 msgr2=0x55702fff5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f9255d9b640 1 --2- 172.21.15.67:0/4199989626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc271b0 0x55702fff5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f9255d9b640 1 -- 172.21.15.67:0/4199989626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc50070 msgr2=0x55702fff5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f9255d9b640 1 --2- 172.21.15.67:0/4199989626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc50070 0x55702fff5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f9255d9b640 1 -- 172.21.15.67:0/4199989626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55702ff01b90 con 0x55702fc2f390 2024-07-31T15:04:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f9255d9b640 1 --2- 172.21.15.67:0/4199989626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x55702fff5ff0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9240009fd0 tx=0x7f92400118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c92a9584f0fa93b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f924cff9640 1 -- 172.21.15.67:0/4199989626 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240004310 con 0x55702fc2f390 2024-07-31T15:04:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f924cff9640 1 -- 172.21.15.67:0/4199989626 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92400044b0 con 0x55702fc2f390 2024-07-31T15:04:35.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.954+0000 7f924cff9640 1 -- 172.21.15.67:0/4199989626 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92400035f0 con 0x55702fc2f390 2024-07-31T15:04:35.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.955+0000 7f924f7fe640 1 --2- 172.21.15.67:0/2698164768 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f92440b3d40 0x7f92440b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.955+0000 7f9256d9d640 1 --2- 172.21.15.67:0/2698164768 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f92440b3d40 0x7f92440b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.955+0000 7f924f7fe640 1 -- 172.21.15.67:0/2698164768 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f92500e4710 con 0x7f92581042d0 2024-07-31T15:04:35.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.955+0000 7f9263371740 1 -- 172.21.15.67:0/4199989626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 msgr2=0x55702fff5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:35.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.955+0000 7f9256d9d640 1 --2- 172.21.15.67:0/2698164768 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f92440b3d40 0x7f92440b6250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f924802f500 tx=0x7f924802f310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:35.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.955+0000 7f9263371740 1 --2- 172.21.15.67:0/4199989626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x55702fff5ff0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9240009fd0 tx=0x7f92400118d0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.955+0000 7f9263371740 1 -- 172.21.15.67:0/4199989626 shutdown_connections 2024-07-31T15:04:35.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.955+0000 7f9263371740 1 --2- 172.21.15.67:0/4199989626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x55702fff5ff0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.955+0000 7f9263371740 1 --2- 172.21.15.67:0/4199989626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc271b0 0x55702fff5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.955+0000 7f9263371740 1 --2- 172.21.15.67:0/4199989626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc50070 0x55702fff5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.955+0000 7f9263371740 1 -- 172.21.15.67:0/4199989626 >> 172.21.15.67:0/4199989626 conn(0x55702fc44460 msgr2=0x55702fff3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:35.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.956+0000 7f9263371740 1 -- 172.21.15.67:0/4199989626 shutdown_connections 2024-07-31T15:04:35.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.956+0000 7f9263371740 1 -- 172.21.15.67:0/4199989626 wait complete. 2024-07-31T15:04:35.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.956+0000 7f9263371740 1 -- start start 2024-07-31T15:04:35.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.956+0000 7f9263371740 1 Processor -- start 2024-07-31T15:04:35.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.956+0000 7f9263371740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc271b0 0x5570300010e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.956+0000 7f9263371740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x557030001620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.956+0000 7f9263371740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc50070 0x55703000d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.956+0000 7f9263371740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55702fdbe700 con 0x55702fc50070 2024-07-31T15:04:35.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.956+0000 7f9263371740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55702fefd9e0 con 0x55702fc2f390 2024-07-31T15:04:35.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.956+0000 7f9263371740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557030003d50 con 0x55702fc271b0 2024-07-31T15:04:35.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.956+0000 7f9263371740 10 client.0 _ll_register_callbacks cb 0x55702ff184f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:35.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f9255d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x557030001620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f9255d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x557030001620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56830/0 (socket says 172.21.15.67:56830) 2024-07-31T15:04:35.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f925659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc271b0 0x5570300010e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f9255d9b640 1 -- 172.21.15.67:0/2701252163 learned_addr learned my addr 172.21.15.67:0/2701252163 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:35.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f9256d9d640 1 --2- 172.21.15.67:0/2701252163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc50070 0x55703000d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f9255d9b640 1 -- 172.21.15.67:0/2701252163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc271b0 msgr2=0x5570300010e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:35.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f9255d9b640 1 --2- 172.21.15.67:0/2701252163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc271b0 0x5570300010e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f9255d9b640 1 -- 172.21.15.67:0/2701252163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc50070 msgr2=0x55703000d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:35.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f9255d9b640 1 --2- 172.21.15.67:0/2701252163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc50070 0x55703000d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f9255d9b640 1 -- 172.21.15.67:0/2701252163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55702fcbb650 con 0x55702fc2f390 2024-07-31T15:04:35.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f9255d9b640 1 --2- 172.21.15.67:0/2701252163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x557030001620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f924001b040 tx=0x7f92400049e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=213d1f69e2beb663 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f923dffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:04:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f923dffb640 1 -- 172.21.15.67:0/2701252163 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240029070 con 0x55702fc2f390 2024-07-31T15:04:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f9263371740 1 -- 172.21.15.67:0/2701252163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92400044b0 con 0x55702fc2f390 2024-07-31T15:04:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.957+0000 7f923dffb640 1 -- 172.21.15.67:0/2701252163 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f9240003b00 con 0x55702fc2f390 2024-07-31T15:04:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.958+0000 7f923dffb640 1 -- 172.21.15.67:0/2701252163 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f9240035050 con 0x55702fc2f390 2024-07-31T15:04:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.958+0000 7f9263371740 10 client.27890 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.958+0000 7f9263371740 1 -- 172.21.15.67:0/2701252163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55703000fd10 con 0x55702fc2f390 2024-07-31T15:04:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.958+0000 7f923dffb640 1 -- 172.21.15.67:0/2701252163 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92400044b0 con 0x55702fc2f390 2024-07-31T15:04:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.958+0000 7f923dffb640 1 -- 172.21.15.67:0/2701252163 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f9240004c70 con 0x55702fc2f390 2024-07-31T15:04:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.958+0000 7f9263371740 10 client.27890 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.958+0000 7f9263371740 10 client.27890 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:04:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.958+0000 7f9263371740 20 client.27890 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:35.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.958+0000 7f9263371740 1 --2- 172.21.15.67:0/2701252163 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55702fc0e9f0 0x5570300130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.958+0000 7f9263371740 4 client.27890 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:04:35.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.958+0000 7f9263371740 1 -- 172.21.15.67:0/2701252163 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55702ff6db00 con 0x55702fc0e9f0 2024-07-31T15:04:35.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.958+0000 7f925659c640 1 --2- 172.21.15.67:0/2701252163 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55702fc0e9f0 0x5570300130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.959+0000 7f925659c640 1 --2- 172.21.15.67:0/2701252163 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55702fc0e9f0 0x5570300130d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=26596faada01805 server_cookie=3b4b58e13243d5ce in_seq=0 out_seq=0 2024-07-31T15:04:35.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.959+0000 7f923dffb640 10 client.27890 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:04:35.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.959+0000 7f923dffb640 1 -- 172.21.15.67:0/2701252163 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55702ff6db00 con 0x55702fc0e9f0 2024-07-31T15:04:35.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.959+0000 7f923dffb640 10 client.27890 handle_command_reply: tid=0 2024-07-31T15:04:35.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.959+0000 7f9263371740 1 client.27890 shutdown 2024-07-31T15:04:35.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.959+0000 7f9263371740 1 -- 172.21.15.67:0/2701252163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 msgr2=0x557030001620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:35.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.959+0000 7f9263371740 1 --2- 172.21.15.67:0/2701252163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x557030001620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f924001b040 tx=0x7f92400049e0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.960+0000 7f9263371740 1 -- 172.21.15.67:0/2701252163 shutdown_connections 2024-07-31T15:04:35.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.960+0000 7f9263371740 1 --2- 172.21.15.67:0/2701252163 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55702fc0e9f0 0x5570300130d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.960+0000 7f9263371740 1 --2- 172.21.15.67:0/2701252163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc50070 0x55703000d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.960+0000 7f9263371740 1 --2- 172.21.15.67:0/2701252163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x557030001620 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.960+0000 7f923dffb640 0 client.27890 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:04:35.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.960+0000 7f9263371740 1 --2- 172.21.15.67:0/2701252163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc271b0 0x5570300010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.960+0000 7f9263371740 1 -- 172.21.15.67:0/2701252163 >> 172.21.15.67:0/2701252163 conn(0x55702fc44460 msgr2=0x55702fff1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:35.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.960+0000 7f9263371740 1 -- 172.21.15.67:0/2701252163 shutdown_connections 2024-07-31T15:04:35.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.960+0000 7f9263371740 1 -- 172.21.15.67:0/2701252163 wait complete. 2024-07-31T15:04:35.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.960+0000 7f9263371740 20 client.27890 trim_cache size 0 max 16384 2024-07-31T15:04:35.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f9263371740 1 Processor -- start 2024-07-31T15:04:35.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f9263371740 1 -- start start 2024-07-31T15:04:35.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f9263371740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc0e9f0 0x55702fff3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f9263371740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc271b0 0x55702fffb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f9263371740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x55702fff6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f9263371740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f92581b29e0 con 0x55702fc271b0 2024-07-31T15:04:35.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f9263371740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f92581b4520 con 0x55702fc2f390 2024-07-31T15:04:35.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f9263371740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92581b30b0 con 0x55702fc0e9f0 2024-07-31T15:04:35.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f9255d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc271b0 0x55702fffb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f925659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc0e9f0 0x55702fff3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f9255d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc271b0 0x55702fffb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39116/0 (socket says 172.21.15.67:39116) 2024-07-31T15:04:35.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f9255d9b640 1 -- 172.21.15.67:0/1474955009 learned_addr learned my addr 172.21.15.67:0/1474955009 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:35.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f9256d9d640 1 --2- 172.21.15.67:0/1474955009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x55702fff6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f9255d9b640 1 -- 172.21.15.67:0/1474955009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc0e9f0 msgr2=0x55702fff3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:35.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.974+0000 7f9255d9b640 1 --2- 172.21.15.67:0/1474955009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc0e9f0 0x55702fff3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.975+0000 7f9255d9b640 1 -- 172.21.15.67:0/1474955009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 msgr2=0x55702fff6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:35.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.975+0000 7f9255d9b640 1 --2- 172.21.15.67:0/1474955009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x55702fff6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.975+0000 7f9255d9b640 1 -- 172.21.15.67:0/1474955009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55702fff7860 con 0x55702fc271b0 2024-07-31T15:04:35.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.975+0000 7f925659c640 1 --2- 172.21.15.67:0/1474955009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc0e9f0 0x55702fff3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:35.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.975+0000 7f9256d9d640 1 --2- 172.21.15.67:0/1474955009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x55702fff6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:35.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.975+0000 7f9255d9b640 1 --2- 172.21.15.67:0/1474955009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc271b0 0x55702fffb690 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f924000a010 tx=0x7f9240009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4762b9682b3cac server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:35.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.975+0000 7f923ffff640 1 -- 172.21.15.67:0/1474955009 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240029070 con 0x55702fc271b0 2024-07-31T15:04:35.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.975+0000 7f923ffff640 1 -- 172.21.15.67:0/1474955009 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9240004a50 con 0x55702fc271b0 2024-07-31T15:04:35.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.975+0000 7f923ffff640 1 -- 172.21.15.67:0/1474955009 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240003cc0 con 0x55702fc271b0 2024-07-31T15:04:35.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.976+0000 7f9263371740 1 -- 172.21.15.67:0/1474955009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc271b0 msgr2=0x55702fffb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:35.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.976+0000 7f9263371740 1 --2- 172.21.15.67:0/1474955009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc271b0 0x55702fffb690 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f924000a010 tx=0x7f9240009ca0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.976+0000 7f9263371740 1 -- 172.21.15.67:0/1474955009 shutdown_connections 2024-07-31T15:04:35.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.976+0000 7f9263371740 1 --2- 172.21.15.67:0/1474955009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc2f390 0x55702fff6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.976+0000 7f9263371740 1 --2- 172.21.15.67:0/1474955009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc271b0 0x55702fffb690 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.976+0000 7f9263371740 1 --2- 172.21.15.67:0/1474955009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc0e9f0 0x55702fff3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.976+0000 7f9263371740 1 -- 172.21.15.67:0/1474955009 >> 172.21.15.67:0/1474955009 conn(0x55702fc44460 msgr2=0x55702fff65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:35.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.976+0000 7f9263371740 1 -- 172.21.15.67:0/1474955009 shutdown_connections 2024-07-31T15:04:35.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.976+0000 7f9263371740 1 -- 172.21.15.67:0/1474955009 wait complete. 2024-07-31T15:04:35.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.976+0000 7f9263371740 1 -- start start 2024-07-31T15:04:35.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.976+0000 7f9263371740 1 Processor -- start 2024-07-31T15:04:35.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f9263371740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc0e9f0 0x5570300040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f9263371740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc271b0 0x557030004630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f9263371740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc2f390 0x557030079790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f9263371740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f92581b3e50 con 0x55702fc0e9f0 2024-07-31T15:04:35.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f9263371740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f92581b3780 con 0x55702fc271b0 2024-07-31T15:04:35.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f9263371740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92581b09e0 con 0x55702fc2f390 2024-07-31T15:04:35.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f9263371740 10 client.0 _ll_register_callbacks cb 0x7f9240005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:35.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f925659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc0e9f0 0x5570300040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f925659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc0e9f0 0x5570300040f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39122/0 (socket says 172.21.15.67:39122) 2024-07-31T15:04:35.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f9255d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc271b0 0x557030004630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f925659c640 1 -- 172.21.15.67:0/2191714349 learned_addr learned my addr 172.21.15.67:0/2191714349 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:35.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f9256d9d640 1 --2- 172.21.15.67:0/2191714349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc2f390 0x557030079790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f925659c640 1 -- 172.21.15.67:0/2191714349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc2f390 msgr2=0x557030079790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:35.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f925659c640 1 --2- 172.21.15.67:0/2191714349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc2f390 0x557030079790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f925659c640 1 -- 172.21.15.67:0/2191714349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc271b0 msgr2=0x557030004630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:35.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f925659c640 1 --2- 172.21.15.67:0/2191714349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc271b0 0x557030004630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f925659c640 1 -- 172.21.15.67:0/2191714349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557030004e50 con 0x55702fc0e9f0 2024-07-31T15:04:35.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f9255d9b640 1 --2- 172.21.15.67:0/2191714349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc271b0 0x557030004630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:35.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f925659c640 1 --2- 172.21.15.67:0/2191714349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc0e9f0 0x5570300040f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f92500058c0 tx=0x7f92500aa9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3067ddc12614b8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:35.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.977+0000 7f923dffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:04:35.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f923dffb640 1 -- 172.21.15.67:0/2191714349 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92500aafb0 con 0x55702fc0e9f0 2024-07-31T15:04:35.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f923dffb640 1 -- 172.21.15.67:0/2191714349 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f92500e1d70 con 0x55702fc0e9f0 2024-07-31T15:04:35.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f9263371740 1 -- 172.21.15.67:0/2191714349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9240004a50 con 0x55702fc0e9f0 2024-07-31T15:04:35.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f923dffb640 1 -- 172.21.15.67:0/2191714349 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f92500af5b0 con 0x55702fc0e9f0 2024-07-31T15:04:35.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f9263371740 10 client.18336 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:35.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f9263371740 1 -- 172.21.15.67:0/2191714349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55703007aca0 con 0x55702fc0e9f0 2024-07-31T15:04:35.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f923dffb640 1 -- 172.21.15.67:0/2191714349 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f925008f070 con 0x55702fc0e9f0 2024-07-31T15:04:35.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f923dffb640 1 -- 172.21.15.67:0/2191714349 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f92500b3bd0 con 0x55702fc0e9f0 2024-07-31T15:04:35.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f9263371740 10 client.18336 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:35.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f9263371740 10 client.18336 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:04:35.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f9263371740 20 client.18336 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:35.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f9263371740 1 --2- 172.21.15.67:0/2191714349 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55702fc50070 0x55703007d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:35.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f9263371740 4 client.18336 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:04:35.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f9263371740 1 -- 172.21.15.67:0/2191714349 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55702ff695a0 con 0x55702fc50070 2024-07-31T15:04:35.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.978+0000 7f9255d9b640 1 --2- 172.21.15.67:0/2191714349 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55702fc50070 0x55703007d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:35.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.979+0000 7f923dffb640 10 client.18336 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:04:35.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.979+0000 7f9255d9b640 1 --2- 172.21.15.67:0/2191714349 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55702fc50070 0x55703007d460 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da822b69c5db3b44 server_cookie=894d9e6ee884f178 in_seq=0 out_seq=0 2024-07-31T15:04:35.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.979+0000 7f923dffb640 1 -- 172.21.15.67:0/2191714349 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55702ff695a0 con 0x55702fc50070 2024-07-31T15:04:35.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.979+0000 7f923dffb640 10 client.18336 handle_command_reply: tid=0 2024-07-31T15:04:35.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.979+0000 7f9263371740 1 client.18336 shutdown 2024-07-31T15:04:35.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.979+0000 7f9263371740 1 -- 172.21.15.67:0/2191714349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc0e9f0 msgr2=0x5570300040f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:35.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.979+0000 7f9263371740 1 --2- 172.21.15.67:0/2191714349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc0e9f0 0x5570300040f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f92500058c0 tx=0x7f92500aa9c0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.979+0000 7f9263371740 1 -- 172.21.15.67:0/2191714349 shutdown_connections 2024-07-31T15:04:35.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.979+0000 7f9263371740 1 --2- 172.21.15.67:0/2191714349 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55702fc50070 0x55703007d460 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.979+0000 7f9263371740 1 --2- 172.21.15.67:0/2191714349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55702fc2f390 0x557030079790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.979+0000 7f9263371740 1 --2- 172.21.15.67:0/2191714349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55702fc271b0 0x557030004630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.979+0000 7f9263371740 1 --2- 172.21.15.67:0/2191714349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55702fc0e9f0 0x5570300040f0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.979+0000 7f9263371740 1 -- 172.21.15.67:0/2191714349 >> 172.21.15.67:0/2191714349 conn(0x55702fc44460 msgr2=0x557030002390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:35.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.979+0000 7f923dffb640 0 client.18336 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:04:35.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f9263371740 1 -- 172.21.15.67:0/2191714349 shutdown_connections 2024-07-31T15:04:35.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f9263371740 1 -- 172.21.15.67:0/2191714349 wait complete. 2024-07-31T15:04:35.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f9263371740 20 client.18336 trim_cache size 0 max 16384 2024-07-31T15:04:35.989 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:04:35.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f925ce20640 1 -- 172.21.15.67:0/2698164768 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f92440b3d40 msgr2=0x7f92440b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:35.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f925ce20640 1 --2- 172.21.15.67:0/2698164768 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f92440b3d40 0x7f92440b6250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f924802f500 tx=0x7f924802f310 comp rx=0 tx=0).stop 2024-07-31T15:04:35.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f925ce20640 1 -- 172.21.15.67:0/2698164768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92581042d0 msgr2=0x7f92581021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:35.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f925ce20640 1 --2- 172.21.15.67:0/2698164768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92581042d0 0x7f92581021e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f925000e9c0 tx=0x7f925000ee80 comp rx=0 tx=0).stop 2024-07-31T15:04:35.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f925ce20640 1 -- 172.21.15.67:0/2698164768 shutdown_connections 2024-07-31T15:04:35.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f925ce20640 1 --2- 172.21.15.67:0/2698164768 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f92440b3d40 0x7f92440b6250 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f925ce20640 1 --2- 172.21.15.67:0/2698164768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f925810d880 0x7f9258102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f925ce20640 1 --2- 172.21.15.67:0/2698164768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9258104d00 0x7f9258102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f925ce20640 1 --2- 172.21.15.67:0/2698164768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92581042d0 0x7f92581021e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:35.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f925ce20640 1 -- 172.21.15.67:0/2698164768 >> 172.21.15.67:0/2698164768 conn(0x7f9258079160 msgr2=0x7f9258106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:35.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f925ce20640 1 -- 172.21.15.67:0/2698164768 shutdown_connections 2024-07-31T15:04:35.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:35.980+0000 7f925ce20640 1 -- 172.21.15.67:0/2698164768 wait complete. 2024-07-31T15:04:36.809 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T15:04:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:04:37.342 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:37 smithi067 ceph-mon[32750]: pgmap v828: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 628 KiB/s wr, 96 op/s 2024-07-31T15:04:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:37 smithi077 ceph-mon[37327]: pgmap v828: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 628 KiB/s wr, 96 op/s 2024-07-31T15:04:37.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:37 smithi073 ceph-mon[36714]: pgmap v828: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 628 KiB/s wr, 96 op/s 2024-07-31T15:04:38.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.797+0000 7f051c8cc640 1 -- 172.21.15.67:0/94762427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0518104d00 msgr2=0x7f051810d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:38.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.797+0000 7f051c8cc640 1 --2- 172.21.15.67:0/94762427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0518104d00 0x7f051810d340 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f050c009900 tx=0x7f050c02a410 comp rx=0 tx=0).stop 2024-07-31T15:04:38.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.797+0000 7f051c8cc640 1 -- 172.21.15.67:0/94762427 shutdown_connections 2024-07-31T15:04:38.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.797+0000 7f051c8cc640 1 --2- 172.21.15.67:0/94762427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f051810d880 0x7f051810fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.797+0000 7f051c8cc640 1 --2- 172.21.15.67:0/94762427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0518104d00 0x7f051810d340 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.797+0000 7f051c8cc640 1 --2- 172.21.15.67:0/94762427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05181042d0 0x7f05181046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.797+0000 7f051c8cc640 1 -- 172.21.15.67:0/94762427 >> 172.21.15.67:0/94762427 conn(0x7f0518079160 msgr2=0x7f0518079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:38.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.798+0000 7f051c8cc640 1 -- 172.21.15.67:0/94762427 shutdown_connections 2024-07-31T15:04:38.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.798+0000 7f051c8cc640 1 -- 172.21.15.67:0/94762427 wait complete. 2024-07-31T15:04:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.798+0000 7f051c8cc640 1 Processor -- start 2024-07-31T15:04:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051c8cc640 1 -- start start 2024-07-31T15:04:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051c8cc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05181042d0 0x7f0518199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051c8cc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0518104d00 0x7f051819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051c8cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f051810d880 0x7f051819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051c8cc640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0518113470 con 0x7f05181042d0 2024-07-31T15:04:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051c8cc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05181132f0 con 0x7f0518104d00 2024-07-31T15:04:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051c8cc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05181135f0 con 0x7f051810d880 2024-07-31T15:04:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05181042d0 0x7f0518199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05181042d0 0x7f0518199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39130/0 (socket says 172.21.15.67:39130) 2024-07-31T15:04:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051659c640 1 -- 172.21.15.67:0/3070285286 learned_addr learned my addr 172.21.15.67:0/3070285286 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f0515d9b640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0518104d00 0x7f051819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f0516d9d640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f051810d880 0x7f051819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051659c640 1 -- 172.21.15.67:0/3070285286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f051810d880 msgr2=0x7f051819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051659c640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f051810d880 0x7f051819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051659c640 1 -- 172.21.15.67:0/3070285286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0518104d00 msgr2=0x7f051819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051659c640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0518104d00 0x7f051819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f051659c640 1 -- 172.21.15.67:0/3070285286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051819fe10 con 0x7f05181042d0 2024-07-31T15:04:38.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.799+0000 7f0515d9b640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0518104d00 0x7f051819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:38.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.800+0000 7f051659c640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05181042d0 0x7f0518199ea0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f050800e9c0 tx=0x7f050800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:38.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.800+0000 7f05077fe640 1 -- 172.21.15.67:0/3070285286 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050800cde0 con 0x7f05181042d0 2024-07-31T15:04:38.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.800+0000 7f051c8cc640 1 -- 172.21.15.67:0/3070285286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05181a0100 con 0x7f05181042d0 2024-07-31T15:04:38.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.800+0000 7f05077fe640 1 -- 172.21.15.67:0/3070285286 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05080108e0 con 0x7f05181042d0 2024-07-31T15:04:38.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.800+0000 7f05077fe640 1 -- 172.21.15.67:0/3070285286 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0508018960 con 0x7f05181042d0 2024-07-31T15:04:38.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.800+0000 7f051c8cc640 1 -- 172.21.15.67:0/3070285286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05181a6860 con 0x7f05181042d0 2024-07-31T15:04:38.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.803+0000 7f0522e1d740 1 Processor -- start 2024-07-31T15:04:38.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.803+0000 7f0522e1d740 1 -- start start 2024-07-31T15:04:38.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.803+0000 7f05077fe640 1 -- 172.21.15.67:0/3070285286 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f050801f020 con 0x7f05181042d0 2024-07-31T15:04:38.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.803+0000 7f0522e1d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3549c070 0x557a35841570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.803+0000 7f0522e1d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a354731b0 0x557a35841ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.803+0000 7f0522e1d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3547b390 0x557a35841ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.803+0000 7f0522e1d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557a357b7bb0 con 0x557a354731b0 2024-07-31T15:04:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.803+0000 7f0522e1d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a357a70b0 con 0x557a3549c070 2024-07-31T15:04:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.803+0000 7f0522e1d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a357ab0f0 con 0x557a3547b390 2024-07-31T15:04:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.803+0000 7f051659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a354731b0 0x557a35841ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.803+0000 7f0516d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3549c070 0x557a35841570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.803+0000 7f051659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a354731b0 0x557a35841ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39146/0 (socket says 172.21.15.67:39146) 2024-07-31T15:04:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.803+0000 7f0515d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3547b390 0x557a35841ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.803+0000 7f051659c640 1 -- 172.21.15.67:0/1511340802 learned_addr learned my addr 172.21.15.67:0/1511340802 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.804+0000 7f0516d9d640 1 -- 172.21.15.67:0/1511340802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3547b390 msgr2=0x557a35841ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.804+0000 7f0516d9d640 1 --2- 172.21.15.67:0/1511340802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3547b390 0x557a35841ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.804+0000 7f0516d9d640 1 -- 172.21.15.67:0/1511340802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a354731b0 msgr2=0x557a35841ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.804+0000 7f0516d9d640 1 --2- 172.21.15.67:0/1511340802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a354731b0 0x557a35841ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.804+0000 7f0516d9d640 1 -- 172.21.15.67:0/1511340802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a3574db90 con 0x557a3549c070 2024-07-31T15:04:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.804+0000 7f0516d9d640 1 --2- 172.21.15.67:0/1511340802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3549c070 0x557a35841570 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0510009f00 tx=0x7f0510011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a06693c8db49032d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.804+0000 7f05077fe640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f05000b3d40 0x7f05000b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.804+0000 7f0504ff9640 1 -- 172.21.15.67:0/1511340802 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510004d20 con 0x557a3549c070 2024-07-31T15:04:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.804+0000 7f051659c640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f05000b3d40 0x7f05000b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.804+0000 7f0504ff9640 1 -- 172.21.15.67:0/1511340802 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0510004030 con 0x557a3549c070 2024-07-31T15:04:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.804+0000 7f0504ff9640 1 -- 172.21.15.67:0/1511340802 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510005260 con 0x557a3549c070 2024-07-31T15:04:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.805+0000 7f05077fe640 1 -- 172.21.15.67:0/3070285286 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f05080e45d0 con 0x7f05181042d0 2024-07-31T15:04:38.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.805+0000 7f0522e1d740 1 -- 172.21.15.67:0/1511340802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3549c070 msgr2=0x557a35841570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:38.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.805+0000 7f051659c640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f05000b3d40 0x7f05000b6250 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0508000c00 tx=0x7f0508092040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:38.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.805+0000 7f0522e1d740 1 --2- 172.21.15.67:0/1511340802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3549c070 0x557a35841570 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0510009f00 tx=0x7f0510011810 comp rx=0 tx=0).stop 2024-07-31T15:04:38.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.805+0000 7f0522e1d740 1 -- 172.21.15.67:0/1511340802 shutdown_connections 2024-07-31T15:04:38.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.805+0000 7f0522e1d740 1 --2- 172.21.15.67:0/1511340802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3547b390 0x557a35841ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.805+0000 7f0522e1d740 1 --2- 172.21.15.67:0/1511340802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a354731b0 0x557a35841ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.805+0000 7f0522e1d740 1 --2- 172.21.15.67:0/1511340802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3549c070 0x557a35841570 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.805+0000 7f0522e1d740 1 -- 172.21.15.67:0/1511340802 >> 172.21.15.67:0/1511340802 conn(0x557a35490460 msgr2=0x557a3583f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.806+0000 7f0522e1d740 1 -- 172.21.15.67:0/1511340802 shutdown_connections 2024-07-31T15:04:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.806+0000 7f0522e1d740 1 -- 172.21.15.67:0/1511340802 wait complete. 2024-07-31T15:04:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.806+0000 7f0522e1d740 1 -- start start 2024-07-31T15:04:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.806+0000 7f0522e1d740 1 Processor -- start 2024-07-31T15:04:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.806+0000 7f0522e1d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a3584d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.806+0000 7f0522e1d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a3584d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.806+0000 7f0522e1d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 0x557a35859070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.806+0000 7f0522e1d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557a3560a700 con 0x557a3547b390 2024-07-31T15:04:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.806+0000 7f0522e1d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a357499e0 con 0x557a354731b0 2024-07-31T15:04:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.806+0000 7f0522e1d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a3584fd50 con 0x557a3549c070 2024-07-31T15:04:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0516d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a3584d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0515d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a3584d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0522e1d740 10 client.0 _ll_register_callbacks cb 0x557a357644f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0516d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a3584d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56888/0 (socket says 172.21.15.67:56888) 2024-07-31T15:04:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0515d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a3584d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39156/0 (socket says 172.21.15.67:39156) 2024-07-31T15:04:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0516d9d640 1 -- 172.21.15.67:0/232095719 learned_addr learned my addr 172.21.15.67:0/232095719 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0516d9d640 1 --2- 172.21.15.67:0/232095719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 0x557a35859070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0516d9d640 1 -- 172.21.15.67:0/232095719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 msgr2=0x557a35859070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0516d9d640 1 --2- 172.21.15.67:0/232095719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 0x557a35859070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0516d9d640 1 -- 172.21.15.67:0/232095719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 msgr2=0x557a3584d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0516d9d640 1 --2- 172.21.15.67:0/232095719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a3584d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0516d9d640 1 -- 172.21.15.67:0/232095719 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a35507650 con 0x557a354731b0 2024-07-31T15:04:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0515d9b640 1 --2- 172.21.15.67:0/232095719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a3584d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0516d9d640 1 --2- 172.21.15.67:0/232095719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a3584d1c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f05100046d0 tx=0x7f05100047f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa3fc2d2f7241cd2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f04fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:04:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f04fdffb640 1 -- 172.21.15.67:0/232095719 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051002d060 con 0x557a354731b0 2024-07-31T15:04:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f0522e1d740 1 -- 172.21.15.67:0/232095719 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0510004030 con 0x557a354731b0 2024-07-31T15:04:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.807+0000 7f04fdffb640 1 -- 172.21.15.67:0/232095719 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f0510004a40 con 0x557a354731b0 2024-07-31T15:04:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.808+0000 7f04fdffb640 1 -- 172.21.15.67:0/232095719 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f0510034050 con 0x557a354731b0 2024-07-31T15:04:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.808+0000 7f0522e1d740 10 client.27908 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.808+0000 7f0522e1d740 1 -- 172.21.15.67:0/232095719 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x557a3585bd40 con 0x557a354731b0 2024-07-31T15:04:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.808+0000 7f04fdffb640 1 -- 172.21.15.67:0/232095719 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0510004030 con 0x557a354731b0 2024-07-31T15:04:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.808+0000 7f04fdffb640 1 -- 172.21.15.67:0/232095719 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f0510024070 con 0x557a354731b0 2024-07-31T15:04:38.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.808+0000 7f0522e1d740 10 client.27908 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:38.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.808+0000 7f0522e1d740 10 client.27908 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:04:38.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.808+0000 7f0522e1d740 20 client.27908 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:38.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.808+0000 7f0522e1d740 1 --2- 172.21.15.67:0/232095719 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x557a3545a9f0 0x557a3585f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.808+0000 7f0522e1d740 4 client.27908 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:04:38.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.808+0000 7f0522e1d740 1 -- 172.21.15.67:0/232095719 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a357b9b00 con 0x557a3545a9f0 2024-07-31T15:04:38.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.808+0000 7f0515d9b640 1 --2- 172.21.15.67:0/232095719 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x557a3545a9f0 0x557a3585f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.809+0000 7f04fdffb640 10 client.27908 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:04:38.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.809+0000 7f0515d9b640 1 --2- 172.21.15.67:0/232095719 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x557a3545a9f0 0x557a3585f1b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f5751c06e1357cf0 server_cookie=a3e8812cdb51d1ae in_seq=0 out_seq=0 2024-07-31T15:04:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f04fdffb640 1 -- 172.21.15.67:0/232095719 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a357b9b00 con 0x557a3545a9f0 2024-07-31T15:04:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f04fdffb640 10 client.27908 handle_command_reply: tid=0 2024-07-31T15:04:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f0522e1d740 1 client.27908 shutdown 2024-07-31T15:04:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f0522e1d740 1 -- 172.21.15.67:0/232095719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 msgr2=0x557a3584d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f0522e1d740 1 --2- 172.21.15.67:0/232095719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a3584d1c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f05100046d0 tx=0x7f05100047f0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f0522e1d740 1 -- 172.21.15.67:0/232095719 shutdown_connections 2024-07-31T15:04:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f0522e1d740 1 --2- 172.21.15.67:0/232095719 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x557a3545a9f0 0x557a3585f1b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f0522e1d740 1 --2- 172.21.15.67:0/232095719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 0x557a35859070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f0522e1d740 1 --2- 172.21.15.67:0/232095719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a3584d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f04fdffb640 0 client.27908 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:04:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f0522e1d740 1 --2- 172.21.15.67:0/232095719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a3584d1c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f0522e1d740 1 -- 172.21.15.67:0/232095719 >> 172.21.15.67:0/232095719 conn(0x557a35490460 msgr2=0x557a35844700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f0522e1d740 1 -- 172.21.15.67:0/232095719 shutdown_connections 2024-07-31T15:04:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f0522e1d740 1 -- 172.21.15.67:0/232095719 wait complete. 2024-07-31T15:04:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.810+0000 7f0522e1d740 20 client.27908 trim_cache size 0 max 16384 2024-07-31T15:04:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.824+0000 7f0522e1d740 1 Processor -- start 2024-07-31T15:04:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.824+0000 7f0522e1d740 1 -- start start 2024-07-31T15:04:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.824+0000 7f0522e1d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a3583f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.824+0000 7f0522e1d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a35858e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.824+0000 7f0522e1d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 0x557a35846600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.824+0000 7f0522e1d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f05181b07b0 con 0x557a3547b390 2024-07-31T15:04:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.824+0000 7f0522e1d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05181b22f0 con 0x557a354731b0 2024-07-31T15:04:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.824+0000 7f0522e1d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05181b0e80 con 0x557a3549c070 2024-07-31T15:04:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.824+0000 7f0515d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a35858e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.824+0000 7f0515d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a35858e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39164/0 (socket says 172.21.15.67:39164) 2024-07-31T15:04:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.824+0000 7f0515d9b640 1 -- 172.21.15.67:0/3367425720 learned_addr learned my addr 172.21.15.67:0/3367425720 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.824+0000 7f0516d9d640 1 --2- 172.21.15.67:0/3367425720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a3583f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.824+0000 7f0516d9d640 1 --2- 172.21.15.67:0/3367425720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 0x557a35846600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.825+0000 7f0515d9b640 1 -- 172.21.15.67:0/3367425720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 msgr2=0x557a35846600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.825+0000 7f0515d9b640 1 --2- 172.21.15.67:0/3367425720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 0x557a35846600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.825+0000 7f0515d9b640 1 -- 172.21.15.67:0/3367425720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 msgr2=0x557a3583f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:38.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.825+0000 7f0515d9b640 1 --2- 172.21.15.67:0/3367425720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a3583f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.825+0000 7f0515d9b640 1 -- 172.21.15.67:0/3367425720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a35843860 con 0x557a3547b390 2024-07-31T15:04:38.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.825+0000 7f0515d9b640 1 --2- 172.21.15.67:0/3367425720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a35858e60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f050c032fd0 tx=0x7f050c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8feaa678ad3a77d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:38.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.825+0000 7f04fffff640 1 -- 172.21.15.67:0/3367425720 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c035040 con 0x557a3547b390 2024-07-31T15:04:38.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.825+0000 7f04fffff640 1 -- 172.21.15.67:0/3367425720 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f050c005040 con 0x557a3547b390 2024-07-31T15:04:38.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.825+0000 7f04fffff640 1 -- 172.21.15.67:0/3367425720 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c004970 con 0x557a3547b390 2024-07-31T15:04:38.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.826+0000 7f0522e1d740 1 -- 172.21.15.67:0/3367425720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 msgr2=0x557a35858e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:38.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.826+0000 7f0522e1d740 1 --2- 172.21.15.67:0/3367425720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a35858e60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f050c032fd0 tx=0x7f050c031400 comp rx=0 tx=0).stop 2024-07-31T15:04:38.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.826+0000 7f0522e1d740 1 -- 172.21.15.67:0/3367425720 shutdown_connections 2024-07-31T15:04:38.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.826+0000 7f0522e1d740 1 --2- 172.21.15.67:0/3367425720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 0x557a35846600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.826+0000 7f0522e1d740 1 --2- 172.21.15.67:0/3367425720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a35858e60 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.826+0000 7f0522e1d740 1 --2- 172.21.15.67:0/3367425720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a3583f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.826+0000 7f0522e1d740 1 -- 172.21.15.67:0/3367425720 >> 172.21.15.67:0/3367425720 conn(0x557a35490460 msgr2=0x557a3583ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.826+0000 7f0522e1d740 1 -- 172.21.15.67:0/3367425720 shutdown_connections 2024-07-31T15:04:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.826+0000 7f0522e1d740 1 -- 172.21.15.67:0/3367425720 wait complete. 2024-07-31T15:04:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.826+0000 7f0522e1d740 1 -- start start 2024-07-31T15:04:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.826+0000 7f0522e1d740 1 Processor -- start 2024-07-31T15:04:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0522e1d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a35855550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0522e1d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a3583e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0522e1d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 0x557a3583e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0522e1d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f05181b1c20 con 0x557a3547b390 2024-07-31T15:04:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0522e1d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05181b1550 con 0x557a354731b0 2024-07-31T15:04:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0522e1d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05181ae7b0 con 0x557a3549c070 2024-07-31T15:04:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0522e1d740 10 client.0 _ll_register_callbacks cb 0x7f05080ea420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0515d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a3583e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0516d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a35855550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0515d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a3583e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39166/0 (socket says 172.21.15.67:39166) 2024-07-31T15:04:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0515d9b640 1 -- 172.21.15.67:0/2278862149 learned_addr learned my addr 172.21.15.67:0/2278862149 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0516d9d640 1 --2- 172.21.15.67:0/2278862149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 0x557a3583e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0515d9b640 1 -- 172.21.15.67:0/2278862149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 msgr2=0x557a3583e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0515d9b640 1 --2- 172.21.15.67:0/2278862149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 0x557a3583e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0515d9b640 1 -- 172.21.15.67:0/2278862149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 msgr2=0x557a35855550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0515d9b640 1 --2- 172.21.15.67:0/2278862149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a35855550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0515d9b640 1 -- 172.21.15.67:0/2278862149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a3583efd0 con 0x557a3547b390 2024-07-31T15:04:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0516d9d640 1 --2- 172.21.15.67:0/2278862149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a35855550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:38.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f0515d9b640 1 --2- 172.21.15.67:0/2278862149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a3583e4a0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f050c02a930 tx=0x7f050c032900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b023dddb44e835d1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:38.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.827+0000 7f04fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:04:38.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f04fdffb640 1 -- 172.21.15.67:0/2278862149 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c053030 con 0x557a3547b390 2024-07-31T15:04:38.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f04fdffb640 1 -- 172.21.15.67:0/2278862149 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f050c0054c0 con 0x557a3547b390 2024-07-31T15:04:38.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f0522e1d740 1 -- 172.21.15.67:0/2278862149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f050c005040 con 0x557a3547b390 2024-07-31T15:04:38.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f04fdffb640 1 -- 172.21.15.67:0/2278862149 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f050c060050 con 0x557a3547b390 2024-07-31T15:04:38.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f0522e1d740 10 client.18354 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:38.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f0522e1d740 1 -- 172.21.15.67:0/2278862149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x557a35852d80 con 0x557a3547b390 2024-07-31T15:04:38.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f04fdffb640 1 -- 172.21.15.67:0/2278862149 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f050c005040 con 0x557a3547b390 2024-07-31T15:04:38.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f04fdffb640 1 -- 172.21.15.67:0/2278862149 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f050c0035f0 con 0x557a3547b390 2024-07-31T15:04:38.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f0522e1d740 10 client.18354 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:38.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f0522e1d740 10 client.18354 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:04:38.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f0522e1d740 20 client.18354 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:38.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f0522e1d740 1 --2- 172.21.15.67:0/2278862149 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x557a3542fda0 0x557a358c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:38.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f0522e1d740 4 client.18354 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:04:38.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f0522e1d740 1 -- 172.21.15.67:0/2278862149 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a357b55a0 con 0x557a3542fda0 2024-07-31T15:04:38.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.828+0000 7f0516d9d640 1 --2- 172.21.15.67:0/2278862149 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x557a3542fda0 0x557a358c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:38.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f04fdffb640 10 client.18354 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:04:38.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f0516d9d640 1 --2- 172.21.15.67:0/2278862149 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x557a3542fda0 0x557a358c9ad0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4cc51084886be925 server_cookie=6c672cbda1b99eb7 in_seq=0 out_seq=0 2024-07-31T15:04:38.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f04fdffb640 1 -- 172.21.15.67:0/2278862149 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a357b55a0 con 0x557a3542fda0 2024-07-31T15:04:38.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f04fdffb640 10 client.18354 handle_command_reply: tid=0 2024-07-31T15:04:38.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f0522e1d740 1 client.18354 shutdown 2024-07-31T15:04:38.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f0522e1d740 1 -- 172.21.15.67:0/2278862149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 msgr2=0x557a3583e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:38.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f0522e1d740 1 --2- 172.21.15.67:0/2278862149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a3583e4a0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f050c02a930 tx=0x7f050c032900 comp rx=0 tx=0).stop 2024-07-31T15:04:38.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f0522e1d740 1 -- 172.21.15.67:0/2278862149 shutdown_connections 2024-07-31T15:04:38.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f0522e1d740 1 --2- 172.21.15.67:0/2278862149 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x557a3542fda0 0x557a358c9ad0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f0522e1d740 1 --2- 172.21.15.67:0/2278862149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3549c070 0x557a3583e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f0522e1d740 1 --2- 172.21.15.67:0/2278862149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3547b390 0x557a3583e4a0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f0522e1d740 1 --2- 172.21.15.67:0/2278862149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a354731b0 0x557a35855550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f0522e1d740 1 -- 172.21.15.67:0/2278862149 >> 172.21.15.67:0/2278862149 conn(0x557a35490460 msgr2=0x557a35842500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:38.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f04fdffb640 0 client.18354 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:04:38.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f0522e1d740 1 -- 172.21.15.67:0/2278862149 shutdown_connections 2024-07-31T15:04:38.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f0522e1d740 1 -- 172.21.15.67:0/2278862149 wait complete. 2024-07-31T15:04:38.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.829+0000 7f0522e1d740 20 client.18354 trim_cache size 0 max 16384 2024-07-31T15:04:38.840 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:04:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.830+0000 7f051c8cc640 1 -- 172.21.15.67:0/3070285286 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f05000b3d40 msgr2=0x7f05000b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.830+0000 7f051c8cc640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f05000b3d40 0x7f05000b6250 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0508000c00 tx=0x7f0508092040 comp rx=0 tx=0).stop 2024-07-31T15:04:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.830+0000 7f051c8cc640 1 -- 172.21.15.67:0/3070285286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05181042d0 msgr2=0x7f0518199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.830+0000 7f051c8cc640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05181042d0 0x7f0518199ea0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f050800e9c0 tx=0x7f050800ee80 comp rx=0 tx=0).stop 2024-07-31T15:04:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.830+0000 7f051c8cc640 1 -- 172.21.15.67:0/3070285286 shutdown_connections 2024-07-31T15:04:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.830+0000 7f051c8cc640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f05000b3d40 0x7f05000b6250 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.830+0000 7f051c8cc640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f051810d880 0x7f051819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.830+0000 7f051c8cc640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0518104d00 0x7f051819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.830+0000 7f051c8cc640 1 --2- 172.21.15.67:0/3070285286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05181042d0 0x7f0518199ea0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.830+0000 7f051c8cc640 1 -- 172.21.15.67:0/3070285286 >> 172.21.15.67:0/3070285286 conn(0x7f0518079160 msgr2=0x7f0518100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.830+0000 7f051c8cc640 1 -- 172.21.15.67:0/3070285286 shutdown_connections 2024-07-31T15:04:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:38.830+0000 7f051c8cc640 1 -- 172.21.15.67:0/3070285286 wait complete. 2024-07-31T15:04:39.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:39 smithi077 ceph-mon[37327]: pgmap v829: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 547 KiB/s wr, 110 op/s 2024-07-31T15:04:39.508 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:39 smithi067 ceph-mon[32750]: pgmap v829: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 547 KiB/s wr, 110 op/s 2024-07-31T15:04:39.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T15:04:39.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:39 smithi073 ceph-mon[36714]: pgmap v829: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 547 KiB/s wr, 110 op/s 2024-07-31T15:04:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:04:41.362 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:41 smithi067 ceph-mon[32750]: pgmap v830: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 409 KiB/s wr, 118 op/s 2024-07-31T15:04:41.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:41 smithi077 ceph-mon[37327]: pgmap v830: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 409 KiB/s wr, 118 op/s 2024-07-31T15:04:41.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:41 smithi073 ceph-mon[36714]: pgmap v830: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 409 KiB/s wr, 118 op/s 2024-07-31T15:04:41.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.643+0000 7fbd528f6640 1 -- 172.21.15.67:0/731628591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4c108ed0 msgr2=0x7fbd4c1092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:41.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.643+0000 7fbd528f6640 1 --2- 172.21.15.67:0/731628591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4c108ed0 0x7fbd4c1092e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009900 tx=0x7fbd4002a410 comp rx=0 tx=0).stop 2024-07-31T15:04:41.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.643+0000 7fbd528f6640 1 -- 172.21.15.67:0/731628591 shutdown_connections 2024-07-31T15:04:41.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.643+0000 7fbd528f6640 1 --2- 172.21.15.67:0/731628591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd4c109820 0x7fbd4c110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.643+0000 7fbd528f6640 1 --2- 172.21.15.67:0/731628591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4c108ed0 0x7fbd4c1092e0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.643+0000 7fbd528f6640 1 --2- 172.21.15.67:0/731628591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd4c103590 0x7fbd4c1039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.643+0000 7fbd528f6640 1 -- 172.21.15.67:0/731628591 >> 172.21.15.67:0/731628591 conn(0x7fbd4c07b280 msgr2=0x7fbd4c07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:41.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.643+0000 7fbd528f6640 1 -- 172.21.15.67:0/731628591 shutdown_connections 2024-07-31T15:04:41.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.643+0000 7fbd528f6640 1 -- 172.21.15.67:0/731628591 wait complete. 2024-07-31T15:04:41.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.644+0000 7fbd528f6640 1 Processor -- start 2024-07-31T15:04:41.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.644+0000 7fbd528f6640 1 -- start start 2024-07-31T15:04:41.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd528f6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd4c103590 0x7fbd4c19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd528f6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4c108ed0 0x7fbd4c19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd528f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd4c109820 0x7fbd4c1a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd528f6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1134b0 con 0x7fbd4c108ed0 2024-07-31T15:04:41.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd528f6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c113330 con 0x7fbd4c103590 2024-07-31T15:04:41.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd528f6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c113630 con 0x7fbd4c109820 2024-07-31T15:04:41.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd4c103590 0x7fbd4c19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd4b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4c108ed0 0x7fbd4c19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd4c103590 0x7fbd4c19c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60116/0 (socket says 172.21.15.67:60116) 2024-07-31T15:04:41.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd4bfff640 1 -- 172.21.15.67:0/3642177919 learned_addr learned my addr 172.21.15.67:0/3642177919 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:41.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd50e93640 1 --2- 172.21.15.67:0/3642177919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd4c109820 0x7fbd4c1a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd4bfff640 1 -- 172.21.15.67:0/3642177919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd4c109820 msgr2=0x7fbd4c1a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:41.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd4bfff640 1 --2- 172.21.15.67:0/3642177919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd4c109820 0x7fbd4c1a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd4bfff640 1 -- 172.21.15.67:0/3642177919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4c108ed0 msgr2=0x7fbd4c19c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:41.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd4bfff640 1 --2- 172.21.15.67:0/3642177919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4c108ed0 0x7fbd4c19c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd4bfff640 1 -- 172.21.15.67:0/3642177919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c1a2130 con 0x7fbd4c103590 2024-07-31T15:04:41.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd4bfff640 1 --2- 172.21.15.67:0/3642177919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd4c103590 0x7fbd4c19c1c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00d770 tx=0x7fbd3c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:41.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd497fa640 1 -- 172.21.15.67:0/3642177919 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c017070 con 0x7fbd4c103590 2024-07-31T15:04:41.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd528f6640 1 -- 172.21.15.67:0/3642177919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c1a2420 con 0x7fbd4c103590 2024-07-31T15:04:41.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.645+0000 7fbd497fa640 1 -- 172.21.15.67:0/3642177919 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd3c0054d0 con 0x7fbd4c103590 2024-07-31T15:04:41.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.646+0000 7fbd497fa640 1 -- 172.21.15.67:0/3642177919 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c004c30 con 0x7fbd4c103590 2024-07-31T15:04:41.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.646+0000 7fbd528f6640 1 -- 172.21.15.67:0/3642177919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c1a8b80 con 0x7fbd4c103590 2024-07-31T15:04:41.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd58e47740 1 Processor -- start 2024-07-31T15:04:41.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd58e47740 1 -- start start 2024-07-31T15:04:41.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd58e47740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb8d6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd58e47740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8d6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd58e47740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8d6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd58e47740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564ccb84cbb0 con 0x564ccb5081b0 2024-07-31T15:04:41.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd58e47740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564ccb83c0b0 con 0x564ccb510390 2024-07-31T15:04:41.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd58e47740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564ccb8400f0 con 0x564ccb531070 2024-07-31T15:04:41.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd497fa640 1 -- 172.21.15.67:0/3642177919 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbd3c004e10 con 0x7fbd4c103590 2024-07-31T15:04:41.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd4b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8d6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd50e93640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb8d6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd4b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8d6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60132/0 (socket says 172.21.15.67:60132) 2024-07-31T15:04:41.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd4b7fe640 1 -- 172.21.15.67:0/1523748768 learned_addr learned my addr 172.21.15.67:0/1523748768 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:41.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd4bfff640 1 --2- 172.21.15.67:0/1523748768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8d6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd4b7fe640 1 -- 172.21.15.67:0/1523748768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 msgr2=0x564ccb8d6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:41.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd4b7fe640 1 --2- 172.21.15.67:0/1523748768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb8d6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd4b7fe640 1 -- 172.21.15.67:0/1523748768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 msgr2=0x564ccb8d6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.649+0000 7fbd4b7fe640 1 --2- 172.21.15.67:0/1523748768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8d6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.650+0000 7fbd4b7fe640 1 -- 172.21.15.67:0/1523748768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ccb7e2b90 con 0x564ccb510390 2024-07-31T15:04:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.650+0000 7fbd4b7fe640 1 --2- 172.21.15.67:0/1523748768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8d6ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009900 tx=0x7fbd40000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c89cdf1d4387b3a6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.650+0000 7fbd327fc640 1 -- 172.21.15.67:0/1523748768 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40040070 con 0x564ccb510390 2024-07-31T15:04:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.650+0000 7fbd327fc640 1 -- 172.21.15.67:0/1523748768 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd40004030 con 0x564ccb510390 2024-07-31T15:04:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.650+0000 7fbd327fc640 1 -- 172.21.15.67:0/1523748768 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40035040 con 0x564ccb510390 2024-07-31T15:04:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.650+0000 7fbd497fa640 1 --2- 172.21.15.67:0/3642177919 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd340b3d40 0x7fbd340b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.651+0000 7fbd50e93640 1 --2- 172.21.15.67:0/3642177919 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd340b3d40 0x7fbd340b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.651+0000 7fbd497fa640 1 -- 172.21.15.67:0/3642177919 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fbd3c0e3eb0 con 0x7fbd4c103590 2024-07-31T15:04:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.651+0000 7fbd58e47740 1 -- 172.21.15.67:0/1523748768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 msgr2=0x564ccb8d6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.651+0000 7fbd50e93640 1 --2- 172.21.15.67:0/3642177919 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd340b3d40 0x7fbd340b6250 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbd44005fd0 tx=0x7fbd44009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.651+0000 7fbd58e47740 1 --2- 172.21.15.67:0/1523748768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8d6ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009900 tx=0x7fbd40000bc0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.651+0000 7fbd58e47740 1 -- 172.21.15.67:0/1523748768 shutdown_connections 2024-07-31T15:04:41.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.651+0000 7fbd58e47740 1 --2- 172.21.15.67:0/1523748768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8d6ff0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.651+0000 7fbd58e47740 1 --2- 172.21.15.67:0/1523748768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8d6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.651+0000 7fbd58e47740 1 --2- 172.21.15.67:0/1523748768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb8d6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.651+0000 7fbd58e47740 1 -- 172.21.15.67:0/1523748768 >> 172.21.15.67:0/1523748768 conn(0x564ccb525460 msgr2=0x564ccb8d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:41.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.651+0000 7fbd58e47740 1 -- 172.21.15.67:0/1523748768 shutdown_connections 2024-07-31T15:04:41.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.651+0000 7fbd58e47740 1 -- 172.21.15.67:0/1523748768 wait complete. 2024-07-31T15:04:41.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.652+0000 7fbd58e47740 1 -- start start 2024-07-31T15:04:41.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.652+0000 7fbd58e47740 1 Processor -- start 2024-07-31T15:04:41.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.652+0000 7fbd58e47740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8e21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.652+0000 7fbd58e47740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8e2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.652+0000 7fbd58e47740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb8ee010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.652+0000 7fbd58e47740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564ccb69f700 con 0x564ccb5081b0 2024-07-31T15:04:41.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.652+0000 7fbd58e47740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564ccb7de9e0 con 0x564ccb510390 2024-07-31T15:04:41.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.652+0000 7fbd58e47740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564ccb8e4cf0 con 0x564ccb531070 2024-07-31T15:04:41.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.652+0000 7fbd58e47740 10 client.0 _ll_register_callbacks cb 0x564ccb7f94f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:41.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.652+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8e21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.652+0000 7fbd4b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8e2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.652+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8e21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60792/0 (socket says 172.21.15.67:60792) 2024-07-31T15:04:41.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.652+0000 7fbd4b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8e2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60138/0 (socket says 172.21.15.67:60138) 2024-07-31T15:04:41.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.652+0000 7fbd4bfff640 1 -- 172.21.15.67:0/3957063381 learned_addr learned my addr 172.21.15.67:0/3957063381 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:41.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.653+0000 7fbd50e93640 1 --2- 172.21.15.67:0/3957063381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb8ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.653+0000 7fbd4b7fe640 1 -- 172.21.15.67:0/3957063381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 msgr2=0x564ccb8ee010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:41.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.653+0000 7fbd4b7fe640 1 --2- 172.21.15.67:0/3957063381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb8ee010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.653+0000 7fbd4b7fe640 1 -- 172.21.15.67:0/3957063381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 msgr2=0x564ccb8e21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:41.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.653+0000 7fbd4b7fe640 1 --2- 172.21.15.67:0/3957063381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8e21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.653+0000 7fbd4b7fe640 1 -- 172.21.15.67:0/3957063381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ccb59c650 con 0x564ccb510390 2024-07-31T15:04:41.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.653+0000 7fbd4bfff640 1 --2- 172.21.15.67:0/3957063381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8e21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:41.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.653+0000 7fbd4b7fe640 1 --2- 172.21.15.67:0/3957063381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8e2710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbd40043000 tx=0x7fbd4002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef4a5597b54d0be1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:41.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.653+0000 7fbd2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:04:41.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.653+0000 7fbd2b7fe640 1 -- 172.21.15.67:0/3957063381 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40040070 con 0x564ccb510390 2024-07-31T15:04:41.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.653+0000 7fbd2b7fe640 1 -- 172.21.15.67:0/3957063381 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fbd40005460 con 0x564ccb510390 2024-07-31T15:04:41.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.653+0000 7fbd58e47740 1 -- 172.21.15.67:0/3957063381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd40004030 con 0x564ccb510390 2024-07-31T15:04:41.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.654+0000 7fbd2b7fe640 1 -- 172.21.15.67:0/3957063381 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fbd40056050 con 0x564ccb510390 2024-07-31T15:04:41.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.654+0000 7fbd58e47740 10 client.27926 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:41.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.654+0000 7fbd58e47740 1 -- 172.21.15.67:0/3957063381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x564ccb8f0d60 con 0x564ccb510390 2024-07-31T15:04:41.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.654+0000 7fbd2b7fe640 1 -- 172.21.15.67:0/3957063381 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd40004030 con 0x564ccb510390 2024-07-31T15:04:41.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.654+0000 7fbd2b7fe640 1 -- 172.21.15.67:0/3957063381 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fbd40004d20 con 0x564ccb510390 2024-07-31T15:04:41.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.654+0000 7fbd58e47740 10 client.27926 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:41.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.654+0000 7fbd58e47740 10 client.27926 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:04:41.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.654+0000 7fbd58e47740 20 client.27926 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:41.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.654+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3957063381 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564ccb4ef9f0 0x564ccb8f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.654+0000 7fbd58e47740 4 client.27926 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:04:41.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.654+0000 7fbd58e47740 1 -- 172.21.15.67:0/3957063381 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ccb84eb00 con 0x564ccb4ef9f0 2024-07-31T15:04:41.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.654+0000 7fbd4bfff640 1 --2- 172.21.15.67:0/3957063381 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564ccb4ef9f0 0x564ccb8f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.655+0000 7fbd2b7fe640 10 client.27926 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:04:41.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.655+0000 7fbd4bfff640 1 --2- 172.21.15.67:0/3957063381 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564ccb4ef9f0 0x564ccb8f40c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f70ddf480348325d server_cookie=d80aeca78a458d03 in_seq=0 out_seq=0 2024-07-31T15:04:41.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd2b7fe640 1 -- 172.21.15.67:0/3957063381 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ccb84eb00 con 0x564ccb4ef9f0 2024-07-31T15:04:41.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd2b7fe640 10 client.27926 handle_command_reply: tid=0 2024-07-31T15:04:41.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd58e47740 1 client.27926 shutdown 2024-07-31T15:04:41.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd58e47740 1 -- 172.21.15.67:0/3957063381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 msgr2=0x564ccb8e2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:41.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3957063381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8e2710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbd40043000 tx=0x7fbd4002a410 comp rx=0 tx=0).stop 2024-07-31T15:04:41.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd58e47740 1 -- 172.21.15.67:0/3957063381 shutdown_connections 2024-07-31T15:04:41.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3957063381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb8ee010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3957063381 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564ccb4ef9f0 0x564ccb8f40c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3957063381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8e2710 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3957063381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8e21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd58e47740 1 -- 172.21.15.67:0/3957063381 >> 172.21.15.67:0/3957063381 conn(0x564ccb525460 msgr2=0x564ccb8d9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:41.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd2b7fe640 0 client.27926 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:04:41.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd58e47740 1 -- 172.21.15.67:0/3957063381 shutdown_connections 2024-07-31T15:04:41.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd58e47740 1 -- 172.21.15.67:0/3957063381 wait complete. 2024-07-31T15:04:41.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.656+0000 7fbd58e47740 20 client.27926 trim_cache size 0 max 16384 2024-07-31T15:04:41.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.670+0000 7fbd58e47740 1 Processor -- start 2024-07-31T15:04:41.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.670+0000 7fbd58e47740 1 -- start start 2024-07-31T15:04:41.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.670+0000 7fbd58e47740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.670+0000 7fbd58e47740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8edf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.670+0000 7fbd58e47740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb8ee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd58e47740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1b29d0 con 0x564ccb5081b0 2024-07-31T15:04:41.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd58e47740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1b4510 con 0x564ccb510390 2024-07-31T15:04:41.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd58e47740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1b30a0 con 0x564ccb531070 2024-07-31T15:04:41.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd4b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8edf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8d4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60796/0 (socket says 172.21.15.67:60796) 2024-07-31T15:04:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd4bfff640 1 -- 172.21.15.67:0/3987850012 learned_addr learned my addr 172.21.15.67:0/3987850012 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd50e93640 1 --2- 172.21.15.67:0/3987850012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb8ee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd4bfff640 1 -- 172.21.15.67:0/3987850012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 msgr2=0x564ccb8ee4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd4bfff640 1 --2- 172.21.15.67:0/3987850012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb8ee4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd4bfff640 1 -- 172.21.15.67:0/3987850012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 msgr2=0x564ccb8edf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd4bfff640 1 --2- 172.21.15.67:0/3987850012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8edf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd4bfff640 1 -- 172.21.15.67:0/3987850012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ccb8eea00 con 0x564ccb5081b0 2024-07-31T15:04:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd4b7fe640 1 --2- 172.21.15.67:0/3987850012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8edf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd4bfff640 1 --2- 172.21.15.67:0/3987850012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8d4560 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0aa680 tx=0x7fbd3c0cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da82c84816708a6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd31ffb640 1 -- 172.21.15.67:0/3987850012 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c020040 con 0x564ccb5081b0 2024-07-31T15:04:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd31ffb640 1 -- 172.21.15.67:0/3987850012 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd3c0aadf0 con 0x564ccb5081b0 2024-07-31T15:04:41.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.671+0000 7fbd31ffb640 1 -- 172.21.15.67:0/3987850012 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0cd0b0 con 0x564ccb5081b0 2024-07-31T15:04:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.672+0000 7fbd58e47740 1 -- 172.21.15.67:0/3987850012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 msgr2=0x564ccb8d4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.672+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3987850012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8d4560 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0aa680 tx=0x7fbd3c0cd040 comp rx=0 tx=0).stop 2024-07-31T15:04:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.672+0000 7fbd58e47740 1 -- 172.21.15.67:0/3987850012 shutdown_connections 2024-07-31T15:04:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.672+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3987850012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb8ee4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.672+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3987850012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8edf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.672+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3987850012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8d4560 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.672+0000 7fbd58e47740 1 -- 172.21.15.67:0/3987850012 >> 172.21.15.67:0/3987850012 conn(0x564ccb525460 msgr2=0x564ccb8d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.672+0000 7fbd58e47740 1 -- 172.21.15.67:0/3987850012 shutdown_connections 2024-07-31T15:04:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.672+0000 7fbd58e47740 1 -- 172.21.15.67:0/3987850012 wait complete. 2024-07-31T15:04:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.673+0000 7fbd58e47740 1 -- start start 2024-07-31T15:04:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.673+0000 7fbd58e47740 1 Processor -- start 2024-07-31T15:04:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.673+0000 7fbd58e47740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.673+0000 7fbd58e47740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.673+0000 7fbd58e47740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb95a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.673+0000 7fbd58e47740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1b3e40 con 0x564ccb5081b0 2024-07-31T15:04:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.673+0000 7fbd58e47740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1b3770 con 0x564ccb510390 2024-07-31T15:04:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.673+0000 7fbd58e47740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1b09d0 con 0x564ccb531070 2024-07-31T15:04:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.673+0000 7fbd58e47740 10 client.0 _ll_register_callbacks cb 0x7fbd440099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.673+0000 7fbd4b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.674+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.674+0000 7fbd4b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8f2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60150/0 (socket says 172.21.15.67:60150) 2024-07-31T15:04:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.674+0000 7fbd4b7fe640 1 -- 172.21.15.67:0/3541404952 learned_addr learned my addr 172.21.15.67:0/3541404952 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.674+0000 7fbd50e93640 1 --2- 172.21.15.67:0/3541404952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb95a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.674+0000 7fbd4b7fe640 1 -- 172.21.15.67:0/3541404952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 msgr2=0x564ccb95a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.674+0000 7fbd4b7fe640 1 --2- 172.21.15.67:0/3541404952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb95a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.674+0000 7fbd4b7fe640 1 -- 172.21.15.67:0/3541404952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 msgr2=0x564ccb8f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.674+0000 7fbd4b7fe640 1 --2- 172.21.15.67:0/3541404952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.674+0000 7fbd4b7fe640 1 -- 172.21.15.67:0/3541404952 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ccb95ab30 con 0x564ccb510390 2024-07-31T15:04:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.674+0000 7fbd4b7fe640 1 --2- 172.21.15.67:0/3541404952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8f2aa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009800 tx=0x7fbd40003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b3ac0062f409c07 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.674+0000 7fbd2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:04:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.674+0000 7fbd2b7fe640 1 -- 172.21.15.67:0/3541404952 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40040040 con 0x564ccb510390 2024-07-31T15:04:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.674+0000 7fbd58e47740 1 -- 172.21.15.67:0/3541404952 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd3c0aadf0 con 0x564ccb510390 2024-07-31T15:04:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.674+0000 7fbd2b7fe640 1 -- 172.21.15.67:0/3541404952 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fbd40003c80 con 0x564ccb510390 2024-07-31T15:04:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.675+0000 7fbd2b7fe640 1 -- 172.21.15.67:0/3541404952 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fbd4006a050 con 0x564ccb510390 2024-07-31T15:04:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.675+0000 7fbd58e47740 10 client.27932 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.675+0000 7fbd58e47740 1 -- 172.21.15.67:0/3541404952 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x564ccb95bce0 con 0x564ccb510390 2024-07-31T15:04:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.675+0000 7fbd2b7fe640 1 -- 172.21.15.67:0/3541404952 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd4005e020 con 0x564ccb510390 2024-07-31T15:04:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.675+0000 7fbd2b7fe640 1 -- 172.21.15.67:0/3541404952 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fbd40005040 con 0x564ccb510390 2024-07-31T15:04:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.675+0000 7fbd58e47740 10 client.27932 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.675+0000 7fbd58e47740 10 client.27932 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:04:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.675+0000 7fbd58e47740 20 client.27932 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.675+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3541404952 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564ccb4c4da0 0x564ccb95e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.675+0000 7fbd58e47740 4 client.27932 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:04:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.675+0000 7fbd58e47740 1 -- 172.21.15.67:0/3541404952 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ccb84a5a0 con 0x564ccb4c4da0 2024-07-31T15:04:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.675+0000 7fbd4bfff640 1 --2- 172.21.15.67:0/3541404952 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564ccb4c4da0 0x564ccb95e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd4bfff640 1 --2- 172.21.15.67:0/3541404952 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564ccb4c4da0 0x564ccb95e490 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b82f9fc99ca2f03 server_cookie=7ad439129a8efebb in_seq=0 out_seq=0 2024-07-31T15:04:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd2b7fe640 10 client.27932 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:04:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd2b7fe640 1 -- 172.21.15.67:0/3541404952 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ccb84a5a0 con 0x564ccb4c4da0 2024-07-31T15:04:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd2b7fe640 10 client.27932 handle_command_reply: tid=0 2024-07-31T15:04:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd58e47740 1 client.27932 shutdown 2024-07-31T15:04:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd58e47740 1 -- 172.21.15.67:0/3541404952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 msgr2=0x564ccb8f2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3541404952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8f2aa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009800 tx=0x7fbd40003a10 comp rx=0 tx=0).stop 2024-07-31T15:04:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd58e47740 1 -- 172.21.15.67:0/3541404952 shutdown_connections 2024-07-31T15:04:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3541404952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564ccb531070 0x564ccb95a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3541404952 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x564ccb4c4da0 0x564ccb95e490 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3541404952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ccb510390 0x564ccb8f2aa0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd58e47740 1 --2- 172.21.15.67:0/3541404952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ccb5081b0 0x564ccb8f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd2b7fe640 0 client.27932 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:04:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd58e47740 1 -- 172.21.15.67:0/3541404952 >> 172.21.15.67:0/3541404952 conn(0x564ccb525460 msgr2=0x564ccb8d9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd58e47740 1 -- 172.21.15.67:0/3541404952 shutdown_connections 2024-07-31T15:04:41.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd58e47740 1 -- 172.21.15.67:0/3541404952 wait complete. 2024-07-31T15:04:41.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.676+0000 7fbd58e47740 20 client.27932 trim_cache size 0 max 16384 2024-07-31T15:04:41.683 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:04:41.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.677+0000 7fbd528f6640 1 -- 172.21.15.67:0/3642177919 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd340b3d40 msgr2=0x7fbd340b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:41.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.677+0000 7fbd528f6640 1 --2- 172.21.15.67:0/3642177919 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd340b3d40 0x7fbd340b6250 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbd44005fd0 tx=0x7fbd44009500 comp rx=0 tx=0).stop 2024-07-31T15:04:41.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.677+0000 7fbd528f6640 1 -- 172.21.15.67:0/3642177919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd4c103590 msgr2=0x7fbd4c19c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:41.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.677+0000 7fbd528f6640 1 --2- 172.21.15.67:0/3642177919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd4c103590 0x7fbd4c19c1c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00d770 tx=0x7fbd3c00dc30 comp rx=0 tx=0).stop 2024-07-31T15:04:41.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.677+0000 7fbd528f6640 1 -- 172.21.15.67:0/3642177919 shutdown_connections 2024-07-31T15:04:41.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.677+0000 7fbd528f6640 1 --2- 172.21.15.67:0/3642177919 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd340b3d40 0x7fbd340b6250 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.677+0000 7fbd528f6640 1 --2- 172.21.15.67:0/3642177919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd4c109820 0x7fbd4c1a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.677+0000 7fbd528f6640 1 --2- 172.21.15.67:0/3642177919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4c108ed0 0x7fbd4c19c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.677+0000 7fbd528f6640 1 --2- 172.21.15.67:0/3642177919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd4c103590 0x7fbd4c19c1c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:41.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.677+0000 7fbd528f6640 1 -- 172.21.15.67:0/3642177919 >> 172.21.15.67:0/3642177919 conn(0x7fbd4c07b280 msgr2=0x7fbd4c106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:41.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.677+0000 7fbd528f6640 1 -- 172.21.15.67:0/3642177919 shutdown_connections 2024-07-31T15:04:41.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:41.677+0000 7fbd528f6640 1 -- 172.21.15.67:0/3642177919 wait complete. 2024-07-31T15:04:42.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T15:04:42.787 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:04:43.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:43 smithi077 ceph-mon[37327]: pgmap v831: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 328 KiB/s wr, 123 op/s 2024-07-31T15:04:43.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:43 smithi067 ceph-mon[32750]: pgmap v831: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 328 KiB/s wr, 123 op/s 2024-07-31T15:04:43.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:43 smithi073 ceph-mon[36714]: pgmap v831: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 328 KiB/s wr, 123 op/s 2024-07-31T15:04:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.480+0000 7fd192561640 1 -- 172.21.15.67:0/792655990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c108760 msgr2=0x7fd18c10ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.480+0000 7fd192561640 1 --2- 172.21.15.67:0/792655990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c108760 0x7fd18c10ef90 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd184009ca0 tx=0x7fd18402a8c0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.481+0000 7fd192561640 1 -- 172.21.15.67:0/792655990 shutdown_connections 2024-07-31T15:04:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.481+0000 7fd192561640 1 --2- 172.21.15.67:0/792655990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c108760 0x7fd18c10ef90 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.481+0000 7fd192561640 1 --2- 172.21.15.67:0/792655990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd18c107e10 0x7fd18c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.481+0000 7fd192561640 1 --2- 172.21.15.67:0/792655990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd18c1024d0 0x7fd18c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.481+0000 7fd192561640 1 -- 172.21.15.67:0/792655990 >> 172.21.15.67:0/792655990 conn(0x7fd18c079190 msgr2=0x7fd18c0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.481+0000 7fd192561640 1 -- 172.21.15.67:0/792655990 shutdown_connections 2024-07-31T15:04:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.481+0000 7fd192561640 1 -- 172.21.15.67:0/792655990 wait complete. 2024-07-31T15:04:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.481+0000 7fd192561640 1 Processor -- start 2024-07-31T15:04:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.481+0000 7fd192561640 1 -- start start 2024-07-31T15:04:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd192561640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd18c1024d0 0x7fd18c19c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd192561640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c107e10 0x7fd18c19c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd192561640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd18c108760 0x7fd18c1a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd192561640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c1123e0 con 0x7fd18c107e10 2024-07-31T15:04:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd192561640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c112260 con 0x7fd18c1024d0 2024-07-31T15:04:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd192561640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c112560 con 0x7fd18c108760 2024-07-31T15:04:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd183fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c107e10 0x7fd18c19c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd18bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd18c1024d0 0x7fd18c19c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd18bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd18c1024d0 0x7fd18c19c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60166/0 (socket says 172.21.15.67:60166) 2024-07-31T15:04:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd183fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c107e10 0x7fd18c19c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60820/0 (socket says 172.21.15.67:60820) 2024-07-31T15:04:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd18bfff640 1 -- 172.21.15.67:0/4205133154 learned_addr learned my addr 172.21.15.67:0/4205133154 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd190afe640 1 --2- 172.21.15.67:0/4205133154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd18c108760 0x7fd18c1a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd18bfff640 1 -- 172.21.15.67:0/4205133154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd18c108760 msgr2=0x7fd18c1a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd18bfff640 1 --2- 172.21.15.67:0/4205133154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd18c108760 0x7fd18c1a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd18bfff640 1 -- 172.21.15.67:0/4205133154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c107e10 msgr2=0x7fd18c19c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd18bfff640 1 --2- 172.21.15.67:0/4205133154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c107e10 0x7fd18c19c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd18bfff640 1 -- 172.21.15.67:0/4205133154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c1a1fb0 con 0x7fd18c1024d0 2024-07-31T15:04:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd190afe640 1 --2- 172.21.15.67:0/4205133154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd18c108760 0x7fd18c1a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:04:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd18bfff640 1 --2- 172.21.15.67:0/4205133154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd18c1024d0 0x7fd18c19c040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00d770 tx=0x7fd17c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.482+0000 7fd189ffb640 1 -- 172.21.15.67:0/4205133154 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c017070 con 0x7fd18c1024d0 2024-07-31T15:04:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.483+0000 7fd192561640 1 -- 172.21.15.67:0/4205133154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18c1a22a0 con 0x7fd18c1024d0 2024-07-31T15:04:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.483+0000 7fd192561640 1 -- 172.21.15.67:0/4205133154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18c1a8a00 con 0x7fd18c1024d0 2024-07-31T15:04:44.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.483+0000 7fd189ffb640 1 -- 172.21.15.67:0/4205133154 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd17c0054d0 con 0x7fd18c1024d0 2024-07-31T15:04:44.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.483+0000 7fd189ffb640 1 -- 172.21.15.67:0/4205133154 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c020060 con 0x7fd18c1024d0 2024-07-31T15:04:44.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.485+0000 7fd198ab2740 1 Processor -- start 2024-07-31T15:04:44.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.485+0000 7fd189ffb640 1 -- 172.21.15.67:0/4205133154 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd17c004c00 con 0x7fd18c1024d0 2024-07-31T15:04:44.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.485+0000 7fd198ab2740 1 -- start start 2024-07-31T15:04:44.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd198ab2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfc28070 0x5616cffcd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd198ab2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfbff1b0 0x5616cffcdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd198ab2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffcdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd198ab2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5616cff43bb0 con 0x5616cfc07390 2024-07-31T15:04:44.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd198ab2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5616cff330b0 con 0x5616cfc28070 2024-07-31T15:04:44.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd198ab2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5616cff370f0 con 0x5616cfbff1b0 2024-07-31T15:04:44.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd190afe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfc28070 0x5616cffcd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd183fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffcdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd190afe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfc28070 0x5616cffcd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60180/0 (socket says 172.21.15.67:60180) 2024-07-31T15:04:44.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd18bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfbff1b0 0x5616cffcdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd190afe640 1 -- 172.21.15.67:0/1100584071 learned_addr learned my addr 172.21.15.67:0/1100584071 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:44.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd190afe640 1 -- 172.21.15.67:0/1100584071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfbff1b0 msgr2=0x5616cffcdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:44.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd190afe640 1 --2- 172.21.15.67:0/1100584071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfbff1b0 0x5616cffcdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd190afe640 1 -- 172.21.15.67:0/1100584071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 msgr2=0x5616cffcdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:44.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd190afe640 1 --2- 172.21.15.67:0/1100584071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffcdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd190afe640 1 -- 172.21.15.67:0/1100584071 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616cfed9b90 con 0x5616cfc28070 2024-07-31T15:04:44.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd183fff640 1 --2- 172.21.15.67:0/1100584071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffcdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:44.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.486+0000 7fd189ffb640 1 --2- 172.21.15.67:0/4205133154 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd1780b3dc0 0x7fd1780b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.487+0000 7fd190afe640 1 --2- 172.21.15.67:0/1100584071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfc28070 0x5616cffcd570 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd184006010 tx=0x7fd184009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b6470a650ae2161 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:44.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.487+0000 7fd18bfff640 1 --2- 172.21.15.67:0/4205133154 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd1780b3dc0 0x7fd1780b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.487+0000 7fd189ffb640 1 -- 172.21.15.67:0/4205133154 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fd17c0e3d00 con 0x7fd18c1024d0 2024-07-31T15:04:44.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.487+0000 7fd1827fc640 1 -- 172.21.15.67:0/1100584071 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184031cf0 con 0x5616cfc28070 2024-07-31T15:04:44.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.487+0000 7fd1827fc640 1 -- 172.21.15.67:0/1100584071 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd184036070 con 0x5616cfc28070 2024-07-31T15:04:44.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.487+0000 7fd1827fc640 1 -- 172.21.15.67:0/1100584071 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1840042a0 con 0x5616cfc28070 2024-07-31T15:04:44.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.488+0000 7fd198ab2740 1 -- 172.21.15.67:0/1100584071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfc28070 msgr2=0x5616cffcd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:44.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.488+0000 7fd18bfff640 1 --2- 172.21.15.67:0/4205133154 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd1780b3dc0 0x7fd1780b62d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0e1190 tx=0x7fd17c0e7850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:44.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.488+0000 7fd198ab2740 1 --2- 172.21.15.67:0/1100584071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfc28070 0x5616cffcd570 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd184006010 tx=0x7fd184009800 comp rx=0 tx=0).stop 2024-07-31T15:04:44.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.488+0000 7fd198ab2740 1 -- 172.21.15.67:0/1100584071 shutdown_connections 2024-07-31T15:04:44.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.488+0000 7fd198ab2740 1 --2- 172.21.15.67:0/1100584071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffcdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.488+0000 7fd198ab2740 1 --2- 172.21.15.67:0/1100584071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfbff1b0 0x5616cffcdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.488+0000 7fd198ab2740 1 --2- 172.21.15.67:0/1100584071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfc28070 0x5616cffcd570 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.488+0000 7fd198ab2740 1 -- 172.21.15.67:0/1100584071 >> 172.21.15.67:0/1100584071 conn(0x5616cfc1c460 msgr2=0x5616cffcb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:44.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.488+0000 7fd198ab2740 1 -- 172.21.15.67:0/1100584071 shutdown_connections 2024-07-31T15:04:44.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.488+0000 7fd198ab2740 1 -- 172.21.15.67:0/1100584071 wait complete. 2024-07-31T15:04:44.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.488+0000 7fd198ab2740 1 -- start start 2024-07-31T15:04:44.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.488+0000 7fd198ab2740 1 Processor -- start 2024-07-31T15:04:44.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd198ab2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfbff1b0 0x5616cffd9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd198ab2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc07390 0x5616cffd9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd198ab2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfc28070 0x5616cffe5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd198ab2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5616cfd96700 con 0x5616cfbff1b0 2024-07-31T15:04:44.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd198ab2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5616cfed59e0 con 0x5616cfc28070 2024-07-31T15:04:44.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd198ab2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5616cffdbcf0 con 0x5616cfc07390 2024-07-31T15:04:44.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd198ab2740 10 client.0 _ll_register_callbacks cb 0x5616cfef04f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:44.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd190afe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfbff1b0 0x5616cffd9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd190afe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfbff1b0 0x5616cffd9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60836/0 (socket says 172.21.15.67:60836) 2024-07-31T15:04:44.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd190afe640 1 -- 172.21.15.67:0/962517202 learned_addr learned my addr 172.21.15.67:0/962517202 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:44.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd183fff640 1 --2- 172.21.15.67:0/962517202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc07390 0x5616cffd9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd190afe640 1 --2- 172.21.15.67:0/962517202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfc28070 0x5616cffe5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd183fff640 1 -- 172.21.15.67:0/962517202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfc28070 msgr2=0x5616cffe5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:44.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd183fff640 1 --2- 172.21.15.67:0/962517202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfc28070 0x5616cffe5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd183fff640 1 -- 172.21.15.67:0/962517202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfbff1b0 msgr2=0x5616cffd9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:44.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd183fff640 1 --2- 172.21.15.67:0/962517202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfbff1b0 0x5616cffd9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd183fff640 1 -- 172.21.15.67:0/962517202 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616cfc93650 con 0x5616cfc07390 2024-07-31T15:04:44.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.489+0000 7fd190afe640 1 --2- 172.21.15.67:0/962517202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfbff1b0 0x5616cffd9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:44.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.490+0000 7fd183fff640 1 --2- 172.21.15.67:0/962517202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc07390 0x5616cffd9680 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd18c06b0b0 tx=0x7fd174012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6608345eb12d2fb8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:44.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.490+0000 7fd16b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:04:44.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.490+0000 7fd16b7fe640 1 -- 172.21.15.67:0/962517202 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174004030 con 0x5616cfc07390 2024-07-31T15:04:44.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.490+0000 7fd198ab2740 1 -- 172.21.15.67:0/962517202 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd184036070 con 0x5616cfc07390 2024-07-31T15:04:44.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.490+0000 7fd16b7fe640 1 -- 172.21.15.67:0/962517202 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fd1740041d0 con 0x5616cfc07390 2024-07-31T15:04:44.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.490+0000 7fd16b7fe640 1 -- 172.21.15.67:0/962517202 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fd174025050 con 0x5616cfc07390 2024-07-31T15:04:44.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.490+0000 7fd198ab2740 10 client.27880 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:44.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.490+0000 7fd198ab2740 1 -- 172.21.15.67:0/962517202 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5616cffe7df0 con 0x5616cfc07390 2024-07-31T15:04:44.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.490+0000 7fd16b7fe640 1 -- 172.21.15.67:0/962517202 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd184036070 con 0x5616cfc07390 2024-07-31T15:04:44.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.491+0000 7fd16b7fe640 1 -- 172.21.15.67:0/962517202 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fd174003950 con 0x5616cfc07390 2024-07-31T15:04:44.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.491+0000 7fd198ab2740 10 client.27880 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:44.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.491+0000 7fd198ab2740 10 client.27880 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:04:44.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.491+0000 7fd198ab2740 20 client.27880 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:44.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.491+0000 7fd198ab2740 1 --2- 172.21.15.67:0/962517202 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5616cfbe69f0 0x5616cffeb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.491+0000 7fd198ab2740 4 client.27880 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:04:44.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.491+0000 7fd198ab2740 1 -- 172.21.15.67:0/962517202 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616cff45b00 con 0x5616cfbe69f0 2024-07-31T15:04:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.491+0000 7fd190afe640 1 --2- 172.21.15.67:0/962517202 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5616cfbe69f0 0x5616cffeb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.491+0000 7fd16b7fe640 10 client.27880 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:04:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.491+0000 7fd190afe640 1 --2- 172.21.15.67:0/962517202 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5616cfbe69f0 0x5616cffeb280 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f718ab290cbb4472 server_cookie=f16f8ebae04cbcd8 in_seq=0 out_seq=0 2024-07-31T15:04:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd16b7fe640 1 -- 172.21.15.67:0/962517202 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616cff45b00 con 0x5616cfbe69f0 2024-07-31T15:04:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd16b7fe640 10 client.27880 handle_command_reply: tid=0 2024-07-31T15:04:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd198ab2740 1 client.27880 shutdown 2024-07-31T15:04:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd198ab2740 1 -- 172.21.15.67:0/962517202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc07390 msgr2=0x5616cffd9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd198ab2740 1 --2- 172.21.15.67:0/962517202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc07390 0x5616cffd9680 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd18c06b0b0 tx=0x7fd174012b60 comp rx=0 tx=0).stop 2024-07-31T15:04:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd198ab2740 1 -- 172.21.15.67:0/962517202 shutdown_connections 2024-07-31T15:04:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd198ab2740 1 --2- 172.21.15.67:0/962517202 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5616cfbe69f0 0x5616cffeb280 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd198ab2740 1 --2- 172.21.15.67:0/962517202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfc28070 0x5616cffe5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd198ab2740 1 --2- 172.21.15.67:0/962517202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc07390 0x5616cffd9680 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd16b7fe640 0 client.27880 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:04:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd198ab2740 1 --2- 172.21.15.67:0/962517202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfbff1b0 0x5616cffd9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd198ab2740 1 -- 172.21.15.67:0/962517202 >> 172.21.15.67:0/962517202 conn(0x5616cfc1c460 msgr2=0x5616cffd0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:44.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd198ab2740 1 -- 172.21.15.67:0/962517202 shutdown_connections 2024-07-31T15:04:44.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd198ab2740 1 -- 172.21.15.67:0/962517202 wait complete. 2024-07-31T15:04:44.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.492+0000 7fd198ab2740 20 client.27880 trim_cache size 0 max 16384 2024-07-31T15:04:44.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.506+0000 7fd198ab2740 1 Processor -- start 2024-07-31T15:04:44.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.506+0000 7fd198ab2740 1 -- start start 2024-07-31T15:04:44.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.506+0000 7fd198ab2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfbff1b0 0x5616cffdab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.506+0000 7fd198ab2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffe4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.506+0000 7fd198ab2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc28070 0x5616cffe5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.506+0000 7fd198ab2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c00fe70 con 0x5616cfc07390 2024-07-31T15:04:44.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.506+0000 7fd198ab2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c0119b0 con 0x5616cfbff1b0 2024-07-31T15:04:44.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.506+0000 7fd198ab2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c010540 con 0x5616cfc28070 2024-07-31T15:04:44.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.507+0000 7fd183fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffe4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.507+0000 7fd190afe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfbff1b0 0x5616cffdab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.507+0000 7fd183fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffe4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60842/0 (socket says 172.21.15.67:60842) 2024-07-31T15:04:44.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.507+0000 7fd183fff640 1 -- 172.21.15.67:0/3747910351 learned_addr learned my addr 172.21.15.67:0/3747910351 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:44.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.507+0000 7fd190afe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfbff1b0 0x5616cffdab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60194/0 (socket says 172.21.15.67:60194) 2024-07-31T15:04:44.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.507+0000 7fd190afe640 1 --2- 172.21.15.67:0/3747910351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc28070 0x5616cffe5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.507+0000 7fd183fff640 1 -- 172.21.15.67:0/3747910351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc28070 msgr2=0x5616cffe5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.507+0000 7fd183fff640 1 --2- 172.21.15.67:0/3747910351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc28070 0x5616cffe5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.507+0000 7fd183fff640 1 -- 172.21.15.67:0/3747910351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfbff1b0 msgr2=0x5616cffdab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.507+0000 7fd183fff640 1 --2- 172.21.15.67:0/3747910351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfbff1b0 0x5616cffdab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.507+0000 7fd183fff640 1 -- 172.21.15.67:0/3747910351 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616cffd3c50 con 0x5616cfc07390 2024-07-31T15:04:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.507+0000 7fd190afe640 1 --2- 172.21.15.67:0/3747910351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfbff1b0 0x5616cffdab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.507+0000 7fd183fff640 1 --2- 172.21.15.67:0/3747910351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffe4e30 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd174009e00 tx=0x7fd1740104b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa0c7fe6ad480f1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.507+0000 7fd181ffb640 1 -- 172.21.15.67:0/3747910351 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17402e070 con 0x5616cfc07390 2024-07-31T15:04:44.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.508+0000 7fd181ffb640 1 -- 172.21.15.67:0/3747910351 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd174003aa0 con 0x5616cfc07390 2024-07-31T15:04:44.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.508+0000 7fd181ffb640 1 -- 172.21.15.67:0/3747910351 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174022040 con 0x5616cfc07390 2024-07-31T15:04:44.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.508+0000 7fd198ab2740 1 -- 172.21.15.67:0/3747910351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 msgr2=0x5616cffe4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:44.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.508+0000 7fd198ab2740 1 --2- 172.21.15.67:0/3747910351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffe4e30 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd174009e00 tx=0x7fd1740104b0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.508+0000 7fd198ab2740 1 -- 172.21.15.67:0/3747910351 shutdown_connections 2024-07-31T15:04:44.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.509+0000 7fd198ab2740 1 --2- 172.21.15.67:0/3747910351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc28070 0x5616cffe5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.509+0000 7fd198ab2740 1 --2- 172.21.15.67:0/3747910351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffe4e30 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.509+0000 7fd198ab2740 1 --2- 172.21.15.67:0/3747910351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfbff1b0 0x5616cffdab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.509+0000 7fd198ab2740 1 -- 172.21.15.67:0/3747910351 >> 172.21.15.67:0/3747910351 conn(0x5616cfc1c460 msgr2=0x5616cffda460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:44.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.509+0000 7fd198ab2740 1 -- 172.21.15.67:0/3747910351 shutdown_connections 2024-07-31T15:04:44.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.509+0000 7fd198ab2740 1 -- 172.21.15.67:0/3747910351 wait complete. 2024-07-31T15:04:44.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.509+0000 7fd198ab2740 1 -- start start 2024-07-31T15:04:44.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.509+0000 7fd198ab2740 1 Processor -- start 2024-07-31T15:04:44.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd198ab2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfbff1b0 0x5616cffe16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd198ab2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd198ab2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc28070 0x5616cffcace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd198ab2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c0112e0 con 0x5616cfc07390 2024-07-31T15:04:44.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd198ab2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c010c10 con 0x5616cfbff1b0 2024-07-31T15:04:44.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd198ab2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c00de70 con 0x5616cfc28070 2024-07-31T15:04:44.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd198ab2740 10 client.0 _ll_register_callbacks cb 0x7fd174009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:44.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd190afe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfbff1b0 0x5616cffe16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd183fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd190afe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfbff1b0 0x5616cffe16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60196/0 (socket says 172.21.15.67:60196) 2024-07-31T15:04:44.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd190afe640 1 -- 172.21.15.67:0/2128962119 learned_addr learned my addr 172.21.15.67:0/2128962119 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:44.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd190afe640 1 --2- 172.21.15.67:0/2128962119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc28070 0x5616cffcace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd190afe640 1 -- 172.21.15.67:0/2128962119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc28070 msgr2=0x5616cffcace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:44.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd190afe640 1 --2- 172.21.15.67:0/2128962119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc28070 0x5616cffcace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd190afe640 1 -- 172.21.15.67:0/2128962119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 msgr2=0x5616cffca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:44.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd190afe640 1 --2- 172.21.15.67:0/2128962119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd190afe640 1 -- 172.21.15.67:0/2128962119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616cffcb470 con 0x5616cfbff1b0 2024-07-31T15:04:44.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd183fff640 1 --2- 172.21.15.67:0/2128962119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:44.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd190afe640 1 --2- 172.21.15.67:0/2128962119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfbff1b0 0x5616cffe16c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd184009870 tx=0x7fd184009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=978b4ff0182f525d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:44.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.510+0000 7fd16b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:04:44.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.511+0000 7fd16b7fe640 1 -- 172.21.15.67:0/2128962119 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184031790 con 0x5616cfbff1b0 2024-07-31T15:04:44.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.511+0000 7fd198ab2740 1 -- 172.21.15.67:0/2128962119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd174003aa0 con 0x5616cfbff1b0 2024-07-31T15:04:44.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.511+0000 7fd16b7fe640 1 -- 172.21.15.67:0/2128962119 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fd18402f660 con 0x5616cfbff1b0 2024-07-31T15:04:44.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.511+0000 7fd16b7fe640 1 -- 172.21.15.67:0/2128962119 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fd18402f860 con 0x5616cfbff1b0 2024-07-31T15:04:44.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.511+0000 7fd198ab2740 10 client.27950 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:44.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.511+0000 7fd198ab2740 1 -- 172.21.15.67:0/2128962119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5616d0053a50 con 0x5616cfbff1b0 2024-07-31T15:04:44.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.511+0000 7fd16b7fe640 1 -- 172.21.15.67:0/2128962119 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd174003aa0 con 0x5616cfbff1b0 2024-07-31T15:04:44.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.511+0000 7fd16b7fe640 1 -- 172.21.15.67:0/2128962119 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fd184030a70 con 0x5616cfbff1b0 2024-07-31T15:04:44.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.511+0000 7fd198ab2740 10 client.27950 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:44.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.511+0000 7fd198ab2740 10 client.27950 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:04:44.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.511+0000 7fd198ab2740 20 client.27950 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:44.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.511+0000 7fd198ab2740 1 --2- 172.21.15.67:0/2128962119 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5616cfbbbda0 0x5616d0055df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:44.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.511+0000 7fd198ab2740 4 client.27950 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:04:44.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.511+0000 7fd198ab2740 1 -- 172.21.15.67:0/2128962119 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616cff415a0 con 0x5616cfbbbda0 2024-07-31T15:04:44.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.512+0000 7fd183fff640 1 --2- 172.21.15.67:0/2128962119 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5616cfbbbda0 0x5616d0055df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:44.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.512+0000 7fd183fff640 1 --2- 172.21.15.67:0/2128962119 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5616cfbbbda0 0x5616d0055df0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8804071f60fc165 server_cookie=4d82f5c0eb9e22e5 in_seq=0 out_seq=0 2024-07-31T15:04:44.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.512+0000 7fd16b7fe640 10 client.27950 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:04:44.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.512+0000 7fd16b7fe640 1 -- 172.21.15.67:0/2128962119 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616cff415a0 con 0x5616cfbbbda0 2024-07-31T15:04:44.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.512+0000 7fd16b7fe640 10 client.27950 handle_command_reply: tid=0 2024-07-31T15:04:44.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.512+0000 7fd198ab2740 1 client.27950 shutdown 2024-07-31T15:04:44.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd198ab2740 1 -- 172.21.15.67:0/2128962119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfbff1b0 msgr2=0x5616cffe16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:44.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd198ab2740 1 --2- 172.21.15.67:0/2128962119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfbff1b0 0x5616cffe16c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd184009870 tx=0x7fd184009800 comp rx=0 tx=0).stop 2024-07-31T15:04:44.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd198ab2740 1 -- 172.21.15.67:0/2128962119 shutdown_connections 2024-07-31T15:04:44.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd198ab2740 1 --2- 172.21.15.67:0/2128962119 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5616cfbbbda0 0x5616d0055df0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd198ab2740 1 --2- 172.21.15.67:0/2128962119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616cfc28070 0x5616cffcace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd198ab2740 1 --2- 172.21.15.67:0/2128962119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5616cfc07390 0x5616cffca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd16b7fe640 0 client.27950 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:04:44.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd198ab2740 1 --2- 172.21.15.67:0/2128962119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616cfbff1b0 0x5616cffe16c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd198ab2740 1 -- 172.21.15.67:0/2128962119 >> 172.21.15.67:0/2128962119 conn(0x5616cfc1c460 msgr2=0x5616cffcef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:44.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd198ab2740 1 -- 172.21.15.67:0/2128962119 shutdown_connections 2024-07-31T15:04:44.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd198ab2740 1 -- 172.21.15.67:0/2128962119 wait complete. 2024-07-31T15:04:44.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd198ab2740 20 client.27950 trim_cache size 0 max 16384 2024-07-31T15:04:44.520 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:04:44.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd192561640 1 -- 172.21.15.67:0/4205133154 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd1780b3dc0 msgr2=0x7fd1780b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:44.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd192561640 1 --2- 172.21.15.67:0/4205133154 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd1780b3dc0 0x7fd1780b62d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0e1190 tx=0x7fd17c0e7850 comp rx=0 tx=0).stop 2024-07-31T15:04:44.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd192561640 1 -- 172.21.15.67:0/4205133154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd18c1024d0 msgr2=0x7fd18c19c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:44.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd192561640 1 --2- 172.21.15.67:0/4205133154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd18c1024d0 0x7fd18c19c040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00d770 tx=0x7fd17c00dc30 comp rx=0 tx=0).stop 2024-07-31T15:04:44.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd192561640 1 -- 172.21.15.67:0/4205133154 shutdown_connections 2024-07-31T15:04:44.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd192561640 1 --2- 172.21.15.67:0/4205133154 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd1780b3dc0 0x7fd1780b62d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd192561640 1 --2- 172.21.15.67:0/4205133154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd18c108760 0x7fd18c1a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd192561640 1 --2- 172.21.15.67:0/4205133154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c107e10 0x7fd18c19c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.513+0000 7fd192561640 1 --2- 172.21.15.67:0/4205133154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd18c1024d0 0x7fd18c19c040 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:44.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.514+0000 7fd192561640 1 -- 172.21.15.67:0/4205133154 >> 172.21.15.67:0/4205133154 conn(0x7fd18c079190 msgr2=0x7fd18c06b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:44.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.514+0000 7fd192561640 1 -- 172.21.15.67:0/4205133154 shutdown_connections 2024-07-31T15:04:44.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:44.514+0000 7fd192561640 1 -- 172.21.15.67:0/4205133154 wait complete. 2024-07-31T15:04:45.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:45 smithi067 ceph-mon[32750]: pgmap v832: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 334 KiB/s wr, 120 op/s 2024-07-31T15:04:45.217 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T15:04:45.487 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:04:45.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:45 smithi077 ceph-mon[37327]: pgmap v832: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 334 KiB/s wr, 120 op/s 2024-07-31T15:04:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:45 smithi073 ceph-mon[36714]: pgmap v832: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 334 KiB/s wr, 120 op/s 2024-07-31T15:04:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.006+0000 7f6119ea9640 1 -- 172.21.15.67:0/2809501714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6114102bb0 msgr2=0x7f6114102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.006+0000 7f6119ea9640 1 --2- 172.21.15.67:0/2809501714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6114102bb0 0x7f6114102fc0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f6104009ca0 tx=0x7f610402a8c0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.007+0000 7f6119ea9640 1 -- 172.21.15.67:0/2809501714 shutdown_connections 2024-07-31T15:04:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.007+0000 7f6119ea9640 1 --2- 172.21.15.67:0/2809501714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6114105460 0x7f611410eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.007+0000 7f6119ea9640 1 --2- 172.21.15.67:0/2809501714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6114104b10 0x7f6114104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.007+0000 7f6119ea9640 1 --2- 172.21.15.67:0/2809501714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6114102bb0 0x7f6114102fc0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.007+0000 7f6119ea9640 1 -- 172.21.15.67:0/2809501714 >> 172.21.15.67:0/2809501714 conn(0x7f61140798d0 msgr2=0x7f6114079ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.007+0000 7f6119ea9640 1 -- 172.21.15.67:0/2809501714 shutdown_connections 2024-07-31T15:04:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.007+0000 7f6119ea9640 1 -- 172.21.15.67:0/2809501714 wait complete. 2024-07-31T15:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.008+0000 7f6119ea9640 1 Processor -- start 2024-07-31T15:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.008+0000 7f6119ea9640 1 -- start start 2024-07-31T15:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6119ea9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6114102bb0 0x7f6114199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6119ea9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6114104b10 0x7f611419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6119ea9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6114105460 0x7f611419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6119ea9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6114113480 con 0x7f6114105460 2024-07-31T15:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6119ea9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6114113300 con 0x7f6114102bb0 2024-07-31T15:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6119ea9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6114113600 con 0x7f6114104b10 2024-07-31T15:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6113fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6114105460 0x7f611419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6113fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6114105460 0x7f611419f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60872/0 (socket says 172.21.15.67:60872) 2024-07-31T15:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6113fff640 1 -- 172.21.15.67:0/459117028 learned_addr learned my addr 172.21.15.67:0/459117028 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f61137fe640 1 --2- 172.21.15.67:0/459117028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6114102bb0 0x7f6114199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6112ffd640 1 --2- 172.21.15.67:0/459117028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6114104b10 0x7f611419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6113fff640 1 -- 172.21.15.67:0/459117028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6114104b10 msgr2=0x7f611419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6113fff640 1 --2- 172.21.15.67:0/459117028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6114104b10 0x7f611419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6113fff640 1 -- 172.21.15.67:0/459117028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6114102bb0 msgr2=0x7f6114199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6113fff640 1 --2- 172.21.15.67:0/459117028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6114102bb0 0x7f6114199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6113fff640 1 -- 172.21.15.67:0/459117028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611419fd70 con 0x7f6114105460 2024-07-31T15:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.009+0000 7f6113fff640 1 --2- 172.21.15.67:0/459117028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6114105460 0x7f611419f670 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f610c00d750 tx=0x7f610c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.010+0000 7f6110ff9640 1 -- 172.21.15.67:0/459117028 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c017070 con 0x7f6114105460 2024-07-31T15:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.010+0000 7f6110ff9640 1 -- 172.21.15.67:0/459117028 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f610c0054d0 con 0x7f6114105460 2024-07-31T15:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.010+0000 7f6110ff9640 1 -- 172.21.15.67:0/459117028 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c004c60 con 0x7f6114105460 2024-07-31T15:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.010+0000 7f6119ea9640 1 -- 172.21.15.67:0/459117028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61141a0060 con 0x7f6114105460 2024-07-31T15:04:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.010+0000 7f6119ea9640 1 -- 172.21.15.67:0/459117028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61141a67f0 con 0x7f6114105460 2024-07-31T15:04:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.013+0000 7f6110ff9640 1 -- 172.21.15.67:0/459117028 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f610c009bd0 con 0x7f6114105460 2024-07-31T15:04:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.013+0000 7f6120d9f740 1 Processor -- start 2024-07-31T15:04:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.013+0000 7f6120d9f740 1 -- start start 2024-07-31T15:04:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.013+0000 7f6120d9f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd5ebce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.013+0000 7f6120d9f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2202f0 0x55bffd5ec220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.013+0000 7f6120d9f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2284d0 0x55bffd5ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.013+0000 7f6120d9f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bffd5613b0 con 0x55bffd2491b0 2024-07-31T15:04:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.013+0000 7f6120d9f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bffd5508b0 con 0x55bffd2284d0 2024-07-31T15:04:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.013+0000 7f6120d9f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bffd5548f0 con 0x55bffd2202f0 2024-07-31T15:04:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.013+0000 7f6112ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd5ebce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.013+0000 7f6112ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd5ebce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60882/0 (socket says 172.21.15.67:60882) 2024-07-31T15:04:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.013+0000 7f6112ffd640 1 -- 172.21.15.67:0/3855831483 learned_addr learned my addr 172.21.15.67:0/3855831483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.013+0000 7f6112ffd640 1 --2- 172.21.15.67:0/3855831483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2284d0 0x55bffd5ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.014+0000 7f61137fe640 1 --2- 172.21.15.67:0/3855831483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2202f0 0x55bffd5ec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.014+0000 7f6112ffd640 1 -- 172.21.15.67:0/3855831483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2202f0 msgr2=0x55bffd5ec220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.014+0000 7f6112ffd640 1 --2- 172.21.15.67:0/3855831483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2202f0 0x55bffd5ec220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.014+0000 7f6112ffd640 1 -- 172.21.15.67:0/3855831483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2284d0 msgr2=0x55bffd5ec760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.014+0000 7f6112ffd640 1 --2- 172.21.15.67:0/3855831483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2284d0 0x55bffd5ec760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.014+0000 7f6112ffd640 1 -- 172.21.15.67:0/3855831483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bffd4f79a0 con 0x55bffd2491b0 2024-07-31T15:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.014+0000 7f6110ff9640 1 --2- 172.21.15.67:0/459117028 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f61000b3dc0 0x7f61000b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.014+0000 7f6112ffd640 1 --2- 172.21.15.67:0/3855831483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd5ebce0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f60fc013750 tx=0x7f60fc009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b755411f1efa7934 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.014+0000 7f61137fe640 1 --2- 172.21.15.67:0/459117028 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f61000b3dc0 0x7f61000b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.014+0000 7f6109ffb640 1 -- 172.21.15.67:0/3855831483 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc0110d0 con 0x55bffd2491b0 2024-07-31T15:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.014+0000 7f6110ff9640 1 -- 172.21.15.67:0/459117028 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f610c0e3c70 con 0x7f6114105460 2024-07-31T15:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.014+0000 7f6109ffb640 1 -- 172.21.15.67:0/3855831483 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60fc013cd0 con 0x55bffd2491b0 2024-07-31T15:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.014+0000 7f6109ffb640 1 -- 172.21.15.67:0/3855831483 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc011e20 con 0x55bffd2491b0 2024-07-31T15:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.015+0000 7f6120d9f740 1 -- 172.21.15.67:0/3855831483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 msgr2=0x55bffd5ebce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.015+0000 7f61137fe640 1 --2- 172.21.15.67:0/459117028 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f61000b3dc0 0x7f61000b62d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x55bffd5ea960 tx=0x7f6104009360 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.015+0000 7f6120d9f740 1 --2- 172.21.15.67:0/3855831483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd5ebce0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f60fc013750 tx=0x7f60fc009680 comp rx=0 tx=0).stop 2024-07-31T15:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.015+0000 7f6120d9f740 1 -- 172.21.15.67:0/3855831483 shutdown_connections 2024-07-31T15:04:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.015+0000 7f6120d9f740 1 --2- 172.21.15.67:0/3855831483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2284d0 0x55bffd5ec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.015+0000 7f6120d9f740 1 --2- 172.21.15.67:0/3855831483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2202f0 0x55bffd5ec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.015+0000 7f6120d9f740 1 --2- 172.21.15.67:0/3855831483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd5ebce0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.015+0000 7f6120d9f740 1 -- 172.21.15.67:0/3855831483 >> 172.21.15.67:0/3855831483 conn(0x55bffd23d300 msgr2=0x55bffd5e9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.015+0000 7f6120d9f740 1 -- 172.21.15.67:0/3855831483 shutdown_connections 2024-07-31T15:04:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.015+0000 7f6120d9f740 1 -- 172.21.15.67:0/3855831483 wait complete. 2024-07-31T15:04:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.015+0000 7f6120d9f740 1 -- start start 2024-07-31T15:04:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6120d9f740 1 Processor -- start 2024-07-31T15:04:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6120d9f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2202f0 0x55bffd5f7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6120d9f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2284d0 0x55bffd5f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6120d9f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd603750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6120d9f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bffd48aab0 con 0x55bffd2491b0 2024-07-31T15:04:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6120d9f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bffd4d8cf0 con 0x55bffd2284d0 2024-07-31T15:04:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6120d9f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bffd5fa400 con 0x55bffd2202f0 2024-07-31T15:04:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6120d9f740 10 client.0 _ll_register_callbacks cb 0x55bffd54a9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6113fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd603750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6113fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd603750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60894/0 (socket says 172.21.15.67:60894) 2024-07-31T15:04:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6113fff640 1 -- 172.21.15.67:0/4240644381 learned_addr learned my addr 172.21.15.67:0/4240644381 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6112ffd640 1 --2- 172.21.15.67:0/4240644381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2284d0 0x55bffd5f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f61137fe640 1 --2- 172.21.15.67:0/4240644381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2202f0 0x55bffd5f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6113fff640 1 -- 172.21.15.67:0/4240644381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2202f0 msgr2=0x55bffd5f7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6113fff640 1 --2- 172.21.15.67:0/4240644381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2202f0 0x55bffd5f7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6113fff640 1 -- 172.21.15.67:0/4240644381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2284d0 msgr2=0x55bffd5f7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6113fff640 1 --2- 172.21.15.67:0/4240644381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2284d0 0x55bffd5f7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.016+0000 7f6113fff640 1 -- 172.21.15.67:0/4240644381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bffd2b4790 con 0x55bffd2491b0 2024-07-31T15:04:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f6113fff640 1 --2- 172.21.15.67:0/4240644381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd603750 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f610c0a6570 tx=0x7f610c0a6a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd0005e84523f16 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f60f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:04:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f60f2ffd640 1 -- 172.21.15.67:0/4240644381 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c020050 con 0x55bffd2491b0 2024-07-31T15:04:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f60f2ffd640 1 -- 172.21.15.67:0/4240644381 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f610c0a9960 con 0x55bffd2491b0 2024-07-31T15:04:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f6120d9f740 1 -- 172.21.15.67:0/4240644381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60fc013cd0 con 0x55bffd2491b0 2024-07-31T15:04:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f60f2ffd640 1 -- 172.21.15.67:0/4240644381 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f610c0aabf0 con 0x55bffd2491b0 2024-07-31T15:04:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f6120d9f740 10 client.18402 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f6120d9f740 1 -- 172.21.15.67:0/4240644381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55bffd32d620 con 0x55bffd2491b0 2024-07-31T15:04:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f60f2ffd640 1 -- 172.21.15.67:0/4240644381 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60fc013cd0 con 0x55bffd2491b0 2024-07-31T15:04:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f60f2ffd640 1 -- 172.21.15.67:0/4240644381 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f610c0a92e0 con 0x55bffd2491b0 2024-07-31T15:04:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f6120d9f740 10 client.18402 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f6120d9f740 10 client.18402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f6120d9f740 20 client.18402 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f6120d9f740 1 --2- 172.21.15.67:0/4240644381 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bffd207d80 0x55bffd609700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f6120d9f740 4 client.18402 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:04:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.017+0000 7f6120d9f740 1 -- 172.21.15.67:0/4240644381 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bffd563300 con 0x55bffd207d80 2024-07-31T15:04:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.018+0000 7f61137fe640 1 --2- 172.21.15.67:0/4240644381 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bffd207d80 0x55bffd609700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.018+0000 7f60f2ffd640 10 client.18402 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.018+0000 7f61137fe640 1 --2- 172.21.15.67:0/4240644381 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bffd207d80 0x55bffd609700 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2b48eb38d39dee2 server_cookie=4ed130a3545d1135 in_seq=0 out_seq=0 2024-07-31T15:04:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.020+0000 7f60f2ffd640 1 -- 172.21.15.67:0/4240644381 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bffd563300 con 0x55bffd207d80 2024-07-31T15:04:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.021+0000 7f60f2ffd640 10 client.18402 handle_command_reply: tid=0 2024-07-31T15:04:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.021+0000 7f6120d9f740 1 client.18402 shutdown 2024-07-31T15:04:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.021+0000 7f6120d9f740 1 -- 172.21.15.67:0/4240644381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 msgr2=0x55bffd603750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.021+0000 7f6120d9f740 1 --2- 172.21.15.67:0/4240644381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd603750 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f610c0a6570 tx=0x7f610c0a6a30 comp rx=0 tx=0).stop 2024-07-31T15:04:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.021+0000 7f6120d9f740 1 -- 172.21.15.67:0/4240644381 shutdown_connections 2024-07-31T15:04:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.021+0000 7f6120d9f740 1 --2- 172.21.15.67:0/4240644381 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bffd207d80 0x55bffd609700 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.021+0000 7f6120d9f740 1 --2- 172.21.15.67:0/4240644381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd603750 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.021+0000 7f6120d9f740 1 --2- 172.21.15.67:0/4240644381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2284d0 0x55bffd5f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.021+0000 7f6120d9f740 1 --2- 172.21.15.67:0/4240644381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2202f0 0x55bffd5f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.021+0000 7f60f2ffd640 0 client.18402 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.021+0000 7f6120d9f740 1 -- 172.21.15.67:0/4240644381 >> 172.21.15.67:0/4240644381 conn(0x55bffd23d300 msgr2=0x55bffd5ecd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.021+0000 7f6120d9f740 1 -- 172.21.15.67:0/4240644381 shutdown_connections 2024-07-31T15:04:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.021+0000 7f6120d9f740 1 -- 172.21.15.67:0/4240644381 wait complete. 2024-07-31T15:04:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.021+0000 7f6120d9f740 20 client.18402 trim_cache size 0 max 16384 2024-07-31T15:04:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.036+0000 7f6120d9f740 1 Processor -- start 2024-07-31T15:04:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.036+0000 7f6120d9f740 1 -- start start 2024-07-31T15:04:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f6120d9f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2202f0 0x55bffd6043d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f6120d9f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2284d0 0x55bffd603540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f6120d9f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2491b0 0x55bffd5f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f6120d9f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f61141b07c0 con 0x55bffd2202f0 2024-07-31T15:04:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f6120d9f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f61141b2300 con 0x55bffd2491b0 2024-07-31T15:04:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f6120d9f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61141b0e90 con 0x55bffd2284d0 2024-07-31T15:04:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f61137fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2202f0 0x55bffd6043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f61137fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2202f0 0x55bffd6043d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60906/0 (socket says 172.21.15.67:60906) 2024-07-31T15:04:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f61137fe640 1 -- 172.21.15.67:0/3239272276 learned_addr learned my addr 172.21.15.67:0/3239272276 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f6113fff640 1 --2- 172.21.15.67:0/3239272276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2491b0 0x55bffd5f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f6112ffd640 1 --2- 172.21.15.67:0/3239272276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2284d0 0x55bffd603540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f61137fe640 1 -- 172.21.15.67:0/3239272276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2284d0 msgr2=0x55bffd603540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f61137fe640 1 --2- 172.21.15.67:0/3239272276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2284d0 0x55bffd603540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f61137fe640 1 -- 172.21.15.67:0/3239272276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2491b0 msgr2=0x55bffd5f0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f61137fe640 1 --2- 172.21.15.67:0/3239272276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2491b0 0x55bffd5f0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f61137fe640 1 -- 172.21.15.67:0/3239272276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bffd5ee890 con 0x55bffd2202f0 2024-07-31T15:04:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f6112ffd640 1 --2- 172.21.15.67:0/3239272276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2284d0 0x55bffd603540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:04:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f61137fe640 1 --2- 172.21.15.67:0/3239272276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2202f0 0x55bffd6043d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f6104009ca0 tx=0x7f6104041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3005d37c5ce88c0c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f61097fa640 1 -- 172.21.15.67:0/3239272276 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104003970 con 0x55bffd2202f0 2024-07-31T15:04:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f61097fa640 1 -- 172.21.15.67:0/3239272276 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6104036050 con 0x55bffd2202f0 2024-07-31T15:04:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.037+0000 7f61097fa640 1 -- 172.21.15.67:0/3239272276 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610402f8c0 con 0x55bffd2202f0 2024-07-31T15:04:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.038+0000 7f6120d9f740 1 -- 172.21.15.67:0/3239272276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2202f0 msgr2=0x55bffd6043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.038+0000 7f6120d9f740 1 --2- 172.21.15.67:0/3239272276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2202f0 0x55bffd6043d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f6104009ca0 tx=0x7f6104041000 comp rx=0 tx=0).stop 2024-07-31T15:04:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.038+0000 7f6120d9f740 1 -- 172.21.15.67:0/3239272276 shutdown_connections 2024-07-31T15:04:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.038+0000 7f6120d9f740 1 --2- 172.21.15.67:0/3239272276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2491b0 0x55bffd5f0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.038+0000 7f6120d9f740 1 --2- 172.21.15.67:0/3239272276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2284d0 0x55bffd603540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.038+0000 7f6120d9f740 1 --2- 172.21.15.67:0/3239272276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2202f0 0x55bffd6043d0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.038+0000 7f6120d9f740 1 -- 172.21.15.67:0/3239272276 >> 172.21.15.67:0/3239272276 conn(0x55bffd23d300 msgr2=0x55bffd603cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.038+0000 7f6120d9f740 1 -- 172.21.15.67:0/3239272276 shutdown_connections 2024-07-31T15:04:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.038+0000 7f6120d9f740 1 -- 172.21.15.67:0/3239272276 wait complete. 2024-07-31T15:04:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.039+0000 7f6120d9f740 1 -- start start 2024-07-31T15:04:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.039+0000 7f6120d9f740 1 Processor -- start 2024-07-31T15:04:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.039+0000 7f6120d9f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2202f0 0x55bffd5fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.039+0000 7f6120d9f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2284d0 0x55bffd5fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.039+0000 7f6120d9f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd66fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.039+0000 7f6120d9f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f61141b1c30 con 0x55bffd2491b0 2024-07-31T15:04:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.039+0000 7f6120d9f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f61141b1560 con 0x55bffd2202f0 2024-07-31T15:04:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.039+0000 7f6120d9f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61141ae7c0 con 0x55bffd2284d0 2024-07-31T15:04:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.039+0000 7f6120d9f740 10 client.0 _ll_register_callbacks cb 0x7f610c0a63b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.039+0000 7f6113fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd66fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.039+0000 7f6113fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd66fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60910/0 (socket says 172.21.15.67:60910) 2024-07-31T15:04:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.039+0000 7f6113fff640 1 -- 172.21.15.67:0/3783860419 learned_addr learned my addr 172.21.15.67:0/3783860419 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.039+0000 7f61137fe640 1 --2- 172.21.15.67:0/3783860419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2202f0 0x55bffd5fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f6112ffd640 1 --2- 172.21.15.67:0/3783860419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2284d0 0x55bffd5fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f6113fff640 1 -- 172.21.15.67:0/3783860419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2284d0 msgr2=0x55bffd5fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f6113fff640 1 --2- 172.21.15.67:0/3783860419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2284d0 0x55bffd5fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f6113fff640 1 -- 172.21.15.67:0/3783860419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2202f0 msgr2=0x55bffd5fc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f6113fff640 1 --2- 172.21.15.67:0/3783860419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2202f0 0x55bffd5fc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f6113fff640 1 -- 172.21.15.67:0/3783860419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bffd5fd510 con 0x55bffd2491b0 2024-07-31T15:04:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f6113fff640 1 --2- 172.21.15.67:0/3783860419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd66fe90 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f610c0a7760 tx=0x7f610c0a9b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6d14ec094042cf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f60f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:04:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f60f2ffd640 1 -- 172.21.15.67:0/3783860419 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c020070 con 0x55bffd2491b0 2024-07-31T15:04:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f60f2ffd640 1 -- 172.21.15.67:0/3783860419 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f610c0a5e60 con 0x55bffd2491b0 2024-07-31T15:04:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f6120d9f740 1 -- 172.21.15.67:0/3783860419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6104036050 con 0x55bffd2491b0 2024-07-31T15:04:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f60f2ffd640 1 -- 172.21.15.67:0/3783860419 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f610c0aeaf0 con 0x55bffd2491b0 2024-07-31T15:04:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f6120d9f740 10 client.18408 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f6120d9f740 1 -- 172.21.15.67:0/3783860419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55bffd671300 con 0x55bffd2491b0 2024-07-31T15:04:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f60f2ffd640 1 -- 172.21.15.67:0/3783860419 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6104036050 con 0x55bffd2491b0 2024-07-31T15:04:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f60f2ffd640 1 -- 172.21.15.67:0/3783860419 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f610c0a6060 con 0x55bffd2491b0 2024-07-31T15:04:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f6120d9f740 10 client.18408 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f6120d9f740 10 client.18408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.040+0000 7f6120d9f740 20 client.18408 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.041+0000 7f6120d9f740 1 --2- 172.21.15.67:0/3783860419 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bffd1dd1c0 0x55bffd673af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.041+0000 7f6120d9f740 4 client.18408 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:04:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.041+0000 7f6120d9f740 1 -- 172.21.15.67:0/3783860419 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bffd55eda0 con 0x55bffd1dd1c0 2024-07-31T15:04:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.041+0000 7f61137fe640 1 --2- 172.21.15.67:0/3783860419 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bffd1dd1c0 0x55bffd673af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.041+0000 7f60f2ffd640 10 client.18408 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.041+0000 7f61137fe640 1 --2- 172.21.15.67:0/3783860419 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bffd1dd1c0 0x55bffd673af0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=531498470de1b557 server_cookie=adddec42d20f53e7 in_seq=0 out_seq=0 2024-07-31T15:04:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.171+0000 7f60f2ffd640 1 -- 172.21.15.67:0/3783860419 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bffd55eda0 con 0x55bffd1dd1c0 2024-07-31T15:04:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.171+0000 7f60f2ffd640 10 client.18408 handle_command_reply: tid=0 2024-07-31T15:04:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.171+0000 7f6120d9f740 1 client.18408 shutdown 2024-07-31T15:04:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.172+0000 7f6120d9f740 1 -- 172.21.15.67:0/3783860419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 msgr2=0x55bffd66fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.172+0000 7f6120d9f740 1 --2- 172.21.15.67:0/3783860419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd66fe90 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f610c0a7760 tx=0x7f610c0a9b70 comp rx=0 tx=0).stop 2024-07-31T15:04:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.172+0000 7f6120d9f740 1 -- 172.21.15.67:0/3783860419 shutdown_connections 2024-07-31T15:04:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.172+0000 7f6120d9f740 1 --2- 172.21.15.67:0/3783860419 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bffd1dd1c0 0x55bffd673af0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.172+0000 7f6120d9f740 1 --2- 172.21.15.67:0/3783860419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bffd2491b0 0x55bffd66fe90 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.172+0000 7f6120d9f740 1 --2- 172.21.15.67:0/3783860419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bffd2284d0 0x55bffd5fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.172+0000 7f60f2ffd640 0 client.18408 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.172+0000 7f6120d9f740 1 --2- 172.21.15.67:0/3783860419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bffd2202f0 0x55bffd5fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.172+0000 7f6120d9f740 1 -- 172.21.15.67:0/3783860419 >> 172.21.15.67:0/3783860419 conn(0x55bffd23d300 msgr2=0x55bffd5f0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.172+0000 7f6120d9f740 1 -- 172.21.15.67:0/3783860419 shutdown_connections 2024-07-31T15:04:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.172+0000 7f6120d9f740 1 -- 172.21.15.67:0/3783860419 wait complete. 2024-07-31T15:04:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.172+0000 7f6120d9f740 20 client.18408 trim_cache size 0 max 16384 2024-07-31T15:04:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:04:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:04:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:04:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:04:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.173+0000 7f6119ea9640 1 -- 172.21.15.67:0/459117028 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f61000b3dc0 msgr2=0x7f61000b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.173+0000 7f6119ea9640 1 --2- 172.21.15.67:0/459117028 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f61000b3dc0 0x7f61000b62d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x55bffd5ea960 tx=0x7f6104009360 comp rx=0 tx=0).stop 2024-07-31T15:04:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.173+0000 7f6119ea9640 1 -- 172.21.15.67:0/459117028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6114105460 msgr2=0x7f611419f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.173+0000 7f6119ea9640 1 --2- 172.21.15.67:0/459117028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6114105460 0x7f611419f670 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f610c00d750 tx=0x7f610c00dc10 comp rx=0 tx=0).stop 2024-07-31T15:04:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.173+0000 7f6119ea9640 1 -- 172.21.15.67:0/459117028 shutdown_connections 2024-07-31T15:04:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.173+0000 7f6119ea9640 1 --2- 172.21.15.67:0/459117028 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f61000b3dc0 0x7f61000b62d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.173+0000 7f6119ea9640 1 --2- 172.21.15.67:0/459117028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6114105460 0x7f611419f670 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.173+0000 7f6119ea9640 1 --2- 172.21.15.67:0/459117028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6114104b10 0x7f611419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.173+0000 7f6119ea9640 1 --2- 172.21.15.67:0/459117028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6114102bb0 0x7f6114199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.173+0000 7f6119ea9640 1 -- 172.21.15.67:0/459117028 >> 172.21.15.67:0/459117028 conn(0x7f61140798d0 msgr2=0x7f6114106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.173+0000 7f6119ea9640 1 -- 172.21.15.67:0/459117028 shutdown_connections 2024-07-31T15:04:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:04:47.173+0000 7f6119ea9640 1 -- 172.21.15.67:0/459117028 wait complete. 2024-07-31T15:04:47.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.352+0000 7fdb737fe640 1 -- 172.21.15.67:0/2361158393 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55941a40c3d0 con 0x55941a5bb700 2024-07-31T15:04:47.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.352+0000 7fdb737fe640 10 client.18312 handle_command_reply: tid=0 2024-07-31T15:04:47.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.352+0000 7fdba0ca9740 1 client.18312 shutdown 2024-07-31T15:04:47.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.352+0000 7fdba0ca9740 1 -- 172.21.15.67:0/2361158393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a15db60 msgr2=0x55941a553d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:47.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.352+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/2361158393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a15db60 0x55941a553d10 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fdb9406ec50 tx=0x7fdb88004a90 comp rx=0 tx=0).stop 2024-07-31T15:04:47.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.352+0000 7fdba0ca9740 1 -- 172.21.15.67:0/2361158393 shutdown_connections 2024-07-31T15:04:47.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.352+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/2361158393 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55941a5bb700 0x55941a5bdb20 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.352+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/2361158393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55941a112570 0x55941a5b94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.352+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/2361158393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55941a15db60 0x55941a553d10 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.352+0000 7fdb737fe640 0 client.18312 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:47.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.352+0000 7fdba0ca9740 1 --2- 172.21.15.67:0/2361158393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55941a155980 0x55941a553750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.352+0000 7fdba0ca9740 1 -- 172.21.15.67:0/2361158393 >> 172.21.15.67:0/2361158393 conn(0x55941a172990 msgr2=0x55941a5259f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:47.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.352+0000 7fdba0ca9740 1 -- 172.21.15.67:0/2361158393 shutdown_connections 2024-07-31T15:04:47.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.352+0000 7fdba0ca9740 1 -- 172.21.15.67:0/2361158393 wait complete. 2024-07-31T15:04:47.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.353+0000 7fdba0ca9740 20 client.18312 trim_cache size 0 max 16384 2024-07-31T15:04:47.357 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:04:47.357 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 63, 2024-07-31T15:04:47.357 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:04:47.357 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 34, 2024-07-31T15:04:47.357 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:04:47.357 INFO:teuthology.orchestra.run.smithi067.stdout: "4d8b91f9": { 2024-07-31T15:04:47.357 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 34, 2024-07-31T15:04:47.357 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:04:47.357 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:04:47.357 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:04:47.357 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:04:47.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:04:47.358 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:04:47.358 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 144.3, 2024-07-31T15:04:47.358 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:04:47.358 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:04:47.358 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:04:47.358 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:04:47.358 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:04:47.358 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:04:47.358 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:04:47.358 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:04:47.358 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:04:47.358 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:04:47.359 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:04:47.359 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:04:47.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.353+0000 7fdb9a758640 1 -- 172.21.15.67:0/309455128 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdb7c0b3f40 msgr2=0x7fdb7c0b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:47.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.353+0000 7fdb9a758640 1 --2- 172.21.15.67:0/309455128 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdb7c0b3f40 0x7fdb7c0b6450 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55941a5236b0 tx=0x7fdb840e7290 comp rx=0 tx=0).stop 2024-07-31T15:04:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.353+0000 7fdb9a758640 1 -- 172.21.15.67:0/309455128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb9410a080 msgr2=0x7fdb94080080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.353+0000 7fdb9a758640 1 --2- 172.21.15.67:0/309455128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb9410a080 0x7fdb94080080 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400b450 tx=0x7fdb8400b910 comp rx=0 tx=0).stop 2024-07-31T15:04:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.353+0000 7fdb9a758640 1 -- 172.21.15.67:0/309455128 shutdown_connections 2024-07-31T15:04:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.353+0000 7fdb9a758640 1 --2- 172.21.15.67:0/309455128 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdb7c0b3f40 0x7fdb7c0b6450 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.353+0000 7fdb9a758640 1 --2- 172.21.15.67:0/309455128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb9410c550 0x7fdb94080b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.353+0000 7fdb9a758640 1 --2- 172.21.15.67:0/309455128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9410c100 0x7fdb940805c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.353+0000 7fdb9a758640 1 --2- 172.21.15.67:0/309455128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb9410a080 0x7fdb94080080 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.353+0000 7fdb9a758640 1 -- 172.21.15.67:0/309455128 >> 172.21.15.67:0/309455128 conn(0x7fdb9406e440 msgr2=0x7fdb9410d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.353+0000 7fdb9a758640 1 -- 172.21.15.67:0/309455128 shutdown_connections 2024-07-31T15:04:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:47.353+0000 7fdb9a758640 1 -- 172.21.15.67:0/309455128 wait complete. 2024-07-31T15:04:47.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:47 smithi067 ceph-mon[32750]: pgmap v833: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 319 KiB/s wr, 113 op/s 2024-07-31T15:04:47.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:47 smithi077 ceph-mon[37327]: pgmap v833: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 319 KiB/s wr, 113 op/s 2024-07-31T15:04:47.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:47 smithi073 ceph-mon[36714]: pgmap v833: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 319 KiB/s wr, 113 op/s 2024-07-31T15:04:48.187 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4d8b91f9', quiesce took 27.4 seconds. Will release after: -3.099999999999998 2024-07-31T15:04:48.187 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:04:48.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.460+0000 7fcd53d70640 1 -- 172.21.15.67:0/1642684228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c108720 msgr2=0x7fcd4c10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.460+0000 7fcd53d70640 1 --2- 172.21.15.67:0/1642684228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c108720 0x7fcd4c10ef50 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009ca0 tx=0x7fcd4402a8c0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.460+0000 7fcd53d70640 1 -- 172.21.15.67:0/1642684228 shutdown_connections 2024-07-31T15:04:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.460+0000 7fcd53d70640 1 --2- 172.21.15.67:0/1642684228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c108720 0x7fcd4c10ef50 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.460+0000 7fcd53d70640 1 --2- 172.21.15.67:0/1642684228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd4c107dd0 0x7fcd4c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.460+0000 7fcd53d70640 1 --2- 172.21.15.67:0/1642684228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd4c102490 0x7fcd4c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.460+0000 7fcd53d70640 1 -- 172.21.15.67:0/1642684228 >> 172.21.15.67:0/1642684228 conn(0x7fcd4c079160 msgr2=0x7fcd4c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.460+0000 7fcd53d70640 1 -- 172.21.15.67:0/1642684228 shutdown_connections 2024-07-31T15:04:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.460+0000 7fcd53d70640 1 -- 172.21.15.67:0/1642684228 wait complete. 2024-07-31T15:04:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.461+0000 7fcd53d70640 1 Processor -- start 2024-07-31T15:04:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.461+0000 7fcd53d70640 1 -- start start 2024-07-31T15:04:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.461+0000 7fcd53d70640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c102490 0x7fcd4c199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.461+0000 7fcd53d70640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd4c107dd0 0x7fcd4c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.461+0000 7fcd53d70640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd4c108720 0x7fcd4c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.461+0000 7fcd53d70640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c1123a0 con 0x7fcd4c102490 2024-07-31T15:04:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.461+0000 7fcd53d70640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c112220 con 0x7fcd4c107dd0 2024-07-31T15:04:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.461+0000 7fcd53d70640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c112520 con 0x7fcd4c108720 2024-07-31T15:04:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd51b0c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c102490 0x7fcd4c199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd51b0c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c102490 0x7fcd4c199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60922/0 (socket says 172.21.15.67:60922) 2024-07-31T15:04:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd51b0c640 1 -- 172.21.15.67:0/205779309 learned_addr learned my addr 172.21.15.67:0/205779309 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd5130b640 1 --2- 172.21.15.67:0/205779309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd4c107dd0 0x7fcd4c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd5230d640 1 --2- 172.21.15.67:0/205779309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd4c108720 0x7fcd4c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd51b0c640 1 -- 172.21.15.67:0/205779309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd4c108720 msgr2=0x7fcd4c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd51b0c640 1 --2- 172.21.15.67:0/205779309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd4c108720 0x7fcd4c19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd51b0c640 1 -- 172.21.15.67:0/205779309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd4c107dd0 msgr2=0x7fcd4c19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd51b0c640 1 --2- 172.21.15.67:0/205779309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd4c107dd0 0x7fcd4c19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd51b0c640 1 -- 172.21.15.67:0/205779309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4c19fdb0 con 0x7fcd4c102490 2024-07-31T15:04:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd51b0c640 1 --2- 172.21.15.67:0/205779309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c102490 0x7fcd4c199e40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000e9f0 tx=0x7fcd4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd3effd640 1 -- 172.21.15.67:0/205779309 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4000cd80 con 0x7fcd4c102490 2024-07-31T15:04:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd3effd640 1 -- 172.21.15.67:0/205779309 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd400108e0 con 0x7fcd4c102490 2024-07-31T15:04:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd53d70640 1 -- 172.21.15.67:0/205779309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4c1a00a0 con 0x7fcd4c102490 2024-07-31T15:04:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd3effd640 1 -- 172.21.15.67:0/205779309 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40018980 con 0x7fcd4c102490 2024-07-31T15:04:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.462+0000 7fcd53d70640 1 -- 172.21.15.67:0/205779309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd4c1a6800 con 0x7fcd4c102490 2024-07-31T15:04:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.465+0000 7fcd5a2c1740 1 Processor -- start 2024-07-31T15:04:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.466+0000 7fcd5a2c1740 1 -- start start 2024-07-31T15:04:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.466+0000 7fcd5a2c1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440859070 0x557440bfe570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.466+0000 7fcd5a2c1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574408301b0 0x557440bfeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.466+0000 7fcd5a2c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557440838390 0x557440bfeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.466+0000 7fcd5a2c1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557440b74bb0 con 0x5574408301b0 2024-07-31T15:04:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.466+0000 7fcd5a2c1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557440b640b0 con 0x557440859070 2024-07-31T15:04:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.466+0000 7fcd5a2c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557440b680f0 con 0x557440838390 2024-07-31T15:04:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.466+0000 7fcd5130b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557440838390 0x557440bfeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.466+0000 7fcd5130b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557440838390 0x557440bfeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50620/0 (socket says 172.21.15.67:50620) 2024-07-31T15:04:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.466+0000 7fcd5130b640 1 -- 172.21.15.67:0/3350227298 learned_addr learned my addr 172.21.15.67:0/3350227298 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.466+0000 7fcd5230d640 1 --2- 172.21.15.67:0/3350227298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440859070 0x557440bfe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.466+0000 7fcd3effd640 1 -- 172.21.15.67:0/205779309 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fcd40010430 con 0x7fcd4c102490 2024-07-31T15:04:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.467+0000 7fcd5130b640 1 -- 172.21.15.67:0/3350227298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440859070 msgr2=0x557440bfe570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.467+0000 7fcd5130b640 1 --2- 172.21.15.67:0/3350227298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440859070 0x557440bfe570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.467+0000 7fcd5130b640 1 -- 172.21.15.67:0/3350227298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574408301b0 msgr2=0x557440bfeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.467+0000 7fcd5130b640 1 --2- 172.21.15.67:0/3350227298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574408301b0 0x557440bfeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.467+0000 7fcd5130b640 1 -- 172.21.15.67:0/3350227298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557440b0ab90 con 0x557440838390 2024-07-31T15:04:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.467+0000 7fcd5130b640 1 --2- 172.21.15.67:0/3350227298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557440838390 0x557440bfeff0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009c60 tx=0x7fcd48011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64a9610ec5a22930 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.467+0000 7fcd37fff640 1 -- 172.21.15.67:0/3350227298 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48004230 con 0x557440838390 2024-07-31T15:04:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.467+0000 7fcd37fff640 1 -- 172.21.15.67:0/3350227298 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd480043d0 con 0x557440838390 2024-07-31T15:04:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.467+0000 7fcd37fff640 1 -- 172.21.15.67:0/3350227298 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48019710 con 0x557440838390 2024-07-31T15:04:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.467+0000 7fcd3effd640 1 --2- 172.21.15.67:0/205779309 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcd380b3dc0 0x7fcd380b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.468+0000 7fcd5230d640 1 --2- 172.21.15.67:0/205779309 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcd380b3dc0 0x7fcd380b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.468+0000 7fcd3effd640 1 -- 172.21.15.67:0/205779309 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fcd400e46c0 con 0x7fcd4c102490 2024-07-31T15:04:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.468+0000 7fcd5230d640 1 --2- 172.21.15.67:0/205779309 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcd380b3dc0 0x7fcd380b62d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcd44006010 tx=0x7fcd44009440 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.468+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/3350227298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557440838390 msgr2=0x557440bfeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.468+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/3350227298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557440838390 0x557440bfeff0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009c60 tx=0x7fcd48011810 comp rx=0 tx=0).stop 2024-07-31T15:04:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.468+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/3350227298 shutdown_connections 2024-07-31T15:04:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.468+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/3350227298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557440838390 0x557440bfeff0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.468+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/3350227298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574408301b0 0x557440bfeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.468+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/3350227298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440859070 0x557440bfe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.468+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/3350227298 >> 172.21.15.67:0/3350227298 conn(0x55744084d460 msgr2=0x557440bfc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.468+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/3350227298 shutdown_connections 2024-07-31T15:04:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.468+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/3350227298 wait complete. 2024-07-31T15:04:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.469+0000 7fcd5a2c1740 1 -- start start 2024-07-31T15:04:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.469+0000 7fcd5a2c1740 1 Processor -- start 2024-07-31T15:04:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.469+0000 7fcd5a2c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574408301b0 0x557440c0a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5a2c1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440838390 0x557440c0a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5a2c1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440859070 0x557440c16070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5a2c1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5574409c7700 con 0x557440859070 2024-07-31T15:04:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5a2c1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557440b069e0 con 0x557440838390 2024-07-31T15:04:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5a2c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557440c0cd50 con 0x5574408301b0 2024-07-31T15:04:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5a2c1740 10 client.0 _ll_register_callbacks cb 0x557440b214f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5230d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440859070 0x557440c16070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5130b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440838390 0x557440c0a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5230d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440859070 0x557440c16070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60942/0 (socket says 172.21.15.67:60942) 2024-07-31T15:04:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5230d640 1 -- 172.21.15.67:0/3431261767 learned_addr learned my addr 172.21.15.67:0/3431261767 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd51b0c640 1 --2- 172.21.15.67:0/3431261767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574408301b0 0x557440c0a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5130b640 1 -- 172.21.15.67:0/3431261767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574408301b0 msgr2=0x557440c0a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5130b640 1 --2- 172.21.15.67:0/3431261767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574408301b0 0x557440c0a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5130b640 1 -- 172.21.15.67:0/3431261767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440859070 msgr2=0x557440c16070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5130b640 1 --2- 172.21.15.67:0/3431261767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440859070 0x557440c16070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5130b640 1 -- 172.21.15.67:0/3431261767 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574408c4650 con 0x557440838390 2024-07-31T15:04:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd5130b640 1 --2- 172.21.15.67:0/3431261767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440838390 0x557440c0a6e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcd48011d10 tx=0x7fcd480052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=196cc0817ff245aa server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd357fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:04:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.470+0000 7fcd357fa640 1 -- 172.21.15.67:0/3431261767 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48028020 con 0x557440838390 2024-07-31T15:04:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.471+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/3431261767 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd480043d0 con 0x557440838390 2024-07-31T15:04:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.471+0000 7fcd357fa640 1 -- 172.21.15.67:0/3431261767 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fcd480035f0 con 0x557440838390 2024-07-31T15:04:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.471+0000 7fcd357fa640 1 -- 172.21.15.67:0/3431261767 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fcd48033050 con 0x557440838390 2024-07-31T15:04:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.471+0000 7fcd5a2c1740 10 client.27980 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.471+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/3431261767 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x557440c18da0 con 0x557440838390 2024-07-31T15:04:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.471+0000 7fcd357fa640 1 -- 172.21.15.67:0/3431261767 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd480043d0 con 0x557440838390 2024-07-31T15:04:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.471+0000 7fcd357fa640 1 -- 172.21.15.67:0/3431261767 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fcd48018070 con 0x557440838390 2024-07-31T15:04:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.471+0000 7fcd5a2c1740 10 client.27980 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.471+0000 7fcd5a2c1740 10 client.27980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.471+0000 7fcd5a2c1740 20 client.27980 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.471+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/3431261767 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574408179f0 0x557440c1c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.471+0000 7fcd5a2c1740 4 client.27980 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:04:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.471+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/3431261767 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557440b76b00 con 0x5574408179f0 2024-07-31T15:04:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.471+0000 7fcd51b0c640 1 --2- 172.21.15.67:0/3431261767 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574408179f0 0x557440c1c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.472+0000 7fcd51b0c640 1 --2- 172.21.15.67:0/3431261767 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574408179f0 0x557440c1c160 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abd748cb567deac8 server_cookie=a0d12ac7855d2651 in_seq=0 out_seq=0 2024-07-31T15:04:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.472+0000 7fcd357fa640 10 client.27980 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.472+0000 7fcd357fa640 1 -- 172.21.15.67:0/3431261767 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557440b76b00 con 0x5574408179f0 2024-07-31T15:04:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.472+0000 7fcd357fa640 10 client.27980 handle_command_reply: tid=0 2024-07-31T15:04:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.472+0000 7fcd5a2c1740 1 client.27980 shutdown 2024-07-31T15:04:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.473+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/3431261767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440838390 msgr2=0x557440c0a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.473+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/3431261767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440838390 0x557440c0a6e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcd48011d10 tx=0x7fcd480052f0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.473+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/3431261767 shutdown_connections 2024-07-31T15:04:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.473+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/3431261767 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574408179f0 0x557440c1c160 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.473+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/3431261767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440859070 0x557440c16070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.473+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/3431261767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440838390 0x557440c0a6e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.473+0000 7fcd357fa640 0 client.27980 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.473+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/3431261767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574408301b0 0x557440c0a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.473+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/3431261767 >> 172.21.15.67:0/3431261767 conn(0x55744084d460 msgr2=0x557440c01700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.473+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/3431261767 shutdown_connections 2024-07-31T15:04:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.473+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/3431261767 wait complete. 2024-07-31T15:04:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.473+0000 7fcd5a2c1740 20 client.27980 trim_cache size 0 max 16384 2024-07-31T15:04:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.487+0000 7fcd5a2c1740 1 Processor -- start 2024-07-31T15:04:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.487+0000 7fcd5a2c1740 1 -- start start 2024-07-31T15:04:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.487+0000 7fcd5a2c1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574408301b0 0x557440c01d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.487+0000 7fcd5a2c1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440838390 0x557440c15e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.487+0000 7fcd5a2c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557440859070 0x557440c03600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.487+0000 7fcd5a2c1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c1b07c0 con 0x557440838390 2024-07-31T15:04:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.487+0000 7fcd5a2c1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c1b2300 con 0x5574408301b0 2024-07-31T15:04:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.487+0000 7fcd5a2c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c1b0e90 con 0x557440859070 2024-07-31T15:04:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.487+0000 7fcd5130b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440838390 0x557440c15e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.487+0000 7fcd5130b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440838390 0x557440c15e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60946/0 (socket says 172.21.15.67:60946) 2024-07-31T15:04:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.487+0000 7fcd5130b640 1 -- 172.21.15.67:0/1932410758 learned_addr learned my addr 172.21.15.67:0/1932410758 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.487+0000 7fcd5230d640 1 --2- 172.21.15.67:0/1932410758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557440859070 0x557440c03600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.487+0000 7fcd51b0c640 1 --2- 172.21.15.67:0/1932410758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574408301b0 0x557440c01d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.488+0000 7fcd5130b640 1 -- 172.21.15.67:0/1932410758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557440859070 msgr2=0x557440c03600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.488+0000 7fcd5130b640 1 --2- 172.21.15.67:0/1932410758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557440859070 0x557440c03600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.488+0000 7fcd5130b640 1 -- 172.21.15.67:0/1932410758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574408301b0 msgr2=0x557440c01d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.488+0000 7fcd5130b640 1 --2- 172.21.15.67:0/1932410758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574408301b0 0x557440c01d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.488+0000 7fcd5130b640 1 -- 172.21.15.67:0/1932410758 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557440c0b260 con 0x557440838390 2024-07-31T15:04:48.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.488+0000 7fcd5130b640 1 --2- 172.21.15.67:0/1932410758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440838390 0x557440c15e60 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fcd48004c00 tx=0x7fcd480096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79824036c68479c8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:48.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.488+0000 7fcd377fe640 1 -- 172.21.15.67:0/1932410758 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48028040 con 0x557440838390 2024-07-31T15:04:48.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.488+0000 7fcd377fe640 1 -- 172.21.15.67:0/1932410758 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd480043d0 con 0x557440838390 2024-07-31T15:04:48.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.488+0000 7fcd377fe640 1 -- 172.21.15.67:0/1932410758 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48000e60 con 0x557440838390 2024-07-31T15:04:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.489+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/1932410758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440838390 msgr2=0x557440c15e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.489+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/1932410758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440838390 0x557440c15e60 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fcd48004c00 tx=0x7fcd480096a0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.489+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/1932410758 shutdown_connections 2024-07-31T15:04:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.489+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/1932410758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557440859070 0x557440c03600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.489+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/1932410758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440838390 0x557440c15e60 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.489+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/1932410758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574408301b0 0x557440c01d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.489+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/1932410758 >> 172.21.15.67:0/1932410758 conn(0x55744084d460 msgr2=0x557440c01670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.489+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/1932410758 shutdown_connections 2024-07-31T15:04:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.489+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/1932410758 wait complete. 2024-07-31T15:04:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.489+0000 7fcd5a2c1740 1 -- start start 2024-07-31T15:04:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd5a2c1740 1 Processor -- start 2024-07-31T15:04:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd5a2c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574408301b0 0x557440c81810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd5a2c1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440838390 0x557440c81d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd5a2c1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440859070 0x557440c11fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd5a2c1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c1b1c30 con 0x557440838390 2024-07-31T15:04:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd5a2c1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c1b1560 con 0x557440859070 2024-07-31T15:04:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd5a2c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c1ae7c0 con 0x5574408301b0 2024-07-31T15:04:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd5a2c1740 10 client.0 _ll_register_callbacks cb 0x7fcd40005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd5130b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440838390 0x557440c81d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd51b0c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574408301b0 0x557440c81810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd5130b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440838390 0x557440c81d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60954/0 (socket says 172.21.15.67:60954) 2024-07-31T15:04:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd5130b640 1 -- 172.21.15.67:0/166770475 learned_addr learned my addr 172.21.15.67:0/166770475 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd5230d640 1 --2- 172.21.15.67:0/166770475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440859070 0x557440c11fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd5130b640 1 -- 172.21.15.67:0/166770475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574408301b0 msgr2=0x557440c81810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.490+0000 7fcd5130b640 1 --2- 172.21.15.67:0/166770475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574408301b0 0x557440c81810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.491+0000 7fcd5130b640 1 -- 172.21.15.67:0/166770475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440859070 msgr2=0x557440c11fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.491+0000 7fcd5130b640 1 --2- 172.21.15.67:0/166770475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440859070 0x557440c11fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.491+0000 7fcd5130b640 1 -- 172.21.15.67:0/166770475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557440c12500 con 0x557440838390 2024-07-31T15:04:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.491+0000 7fcd5230d640 1 --2- 172.21.15.67:0/166770475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440859070 0x557440c11fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:04:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.491+0000 7fcd5130b640 1 --2- 172.21.15.67:0/166770475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440838390 0x557440c81d50 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcd48005890 tx=0x7fcd48009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6a2e31b857bcdc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.491+0000 7fcd357fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:04:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.491+0000 7fcd357fa640 1 -- 172.21.15.67:0/166770475 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48028040 con 0x557440838390 2024-07-31T15:04:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.491+0000 7fcd357fa640 1 -- 172.21.15.67:0/166770475 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fcd480052f0 con 0x557440838390 2024-07-31T15:04:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.491+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/166770475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd480043d0 con 0x557440838390 2024-07-31T15:04:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.491+0000 7fcd357fa640 1 -- 172.21.15.67:0/166770475 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fcd4801f040 con 0x557440838390 2024-07-31T15:04:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.491+0000 7fcd5a2c1740 10 client.18423 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.491+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/166770475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x557440bfb830 con 0x557440838390 2024-07-31T15:04:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.491+0000 7fcd357fa640 1 -- 172.21.15.67:0/166770475 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd480043d0 con 0x557440838390 2024-07-31T15:04:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.492+0000 7fcd357fa640 1 -- 172.21.15.67:0/166770475 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fcd48018070 con 0x557440838390 2024-07-31T15:04:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.492+0000 7fcd5a2c1740 10 client.18423 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.492+0000 7fcd5a2c1740 10 client.18423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.492+0000 7fcd5a2c1740 20 client.18423 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.492+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/166770475 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574407ecda0 0x557440c8ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.492+0000 7fcd5a2c1740 4 client.18423 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:04:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.492+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/166770475 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557440b725a0 con 0x5574407ecda0 2024-07-31T15:04:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.492+0000 7fcd51b0c640 1 --2- 172.21.15.67:0/166770475 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574407ecda0 0x557440c8ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.492+0000 7fcd357fa640 10 client.18423 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.492+0000 7fcd51b0c640 1 --2- 172.21.15.67:0/166770475 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574407ecda0 0x557440c8ac30 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f07bb690d26f06 server_cookie=bb0a24c841c40881 in_seq=0 out_seq=0 2024-07-31T15:04:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd357fa640 1 -- 172.21.15.67:0/166770475 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557440b725a0 con 0x5574407ecda0 2024-07-31T15:04:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd357fa640 10 client.18423 handle_command_reply: tid=0 2024-07-31T15:04:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd5a2c1740 1 client.18423 shutdown 2024-07-31T15:04:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/166770475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440838390 msgr2=0x557440c81d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/166770475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440838390 0x557440c81d50 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcd48005890 tx=0x7fcd48009ab0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/166770475 shutdown_connections 2024-07-31T15:04:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/166770475 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574407ecda0 0x557440c8ac30 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/166770475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557440859070 0x557440c11fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/166770475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557440838390 0x557440c81d50 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd357fa640 0 client.18423 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd5a2c1740 1 --2- 172.21.15.67:0/166770475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574408301b0 0x557440c81810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/166770475 >> 172.21.15.67:0/166770475 conn(0x55744084d460 msgr2=0x557440c0b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/166770475 shutdown_connections 2024-07-31T15:04:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd5a2c1740 1 -- 172.21.15.67:0/166770475 wait complete. 2024-07-31T15:04:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.493+0000 7fcd5a2c1740 20 client.18423 trim_cache size 0 max 16384 2024-07-31T15:04:48.501 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:04:48.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.494+0000 7fcd53d70640 1 -- 172.21.15.67:0/205779309 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcd380b3dc0 msgr2=0x7fcd380b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:48.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.494+0000 7fcd53d70640 1 --2- 172.21.15.67:0/205779309 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcd380b3dc0 0x7fcd380b62d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcd44006010 tx=0x7fcd44009440 comp rx=0 tx=0).stop 2024-07-31T15:04:48.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.494+0000 7fcd53d70640 1 -- 172.21.15.67:0/205779309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c102490 msgr2=0x7fcd4c199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:48.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.494+0000 7fcd53d70640 1 --2- 172.21.15.67:0/205779309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c102490 0x7fcd4c199e40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000e9f0 tx=0x7fcd4000eeb0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.494+0000 7fcd53d70640 1 -- 172.21.15.67:0/205779309 shutdown_connections 2024-07-31T15:04:48.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.494+0000 7fcd53d70640 1 --2- 172.21.15.67:0/205779309 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcd380b3dc0 0x7fcd380b62d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.494+0000 7fcd53d70640 1 --2- 172.21.15.67:0/205779309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd4c108720 0x7fcd4c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.494+0000 7fcd53d70640 1 --2- 172.21.15.67:0/205779309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd4c107dd0 0x7fcd4c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.494+0000 7fcd53d70640 1 --2- 172.21.15.67:0/205779309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c102490 0x7fcd4c199e40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:48.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.494+0000 7fcd53d70640 1 -- 172.21.15.67:0/205779309 >> 172.21.15.67:0/205779309 conn(0x7fcd4c079160 msgr2=0x7fcd4c105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:48.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.494+0000 7fcd53d70640 1 -- 172.21.15.67:0/205779309 shutdown_connections 2024-07-31T15:04:48.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:48.494+0000 7fcd53d70640 1 -- 172.21.15.67:0/205779309 wait complete. 2024-07-31T15:04:49.297 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:04:49.420 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:49 smithi067 ceph-mon[32750]: pgmap v834: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 414 KiB/s wr, 105 op/s 2024-07-31T15:04:49.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:49 smithi077 ceph-mon[37327]: pgmap v834: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 414 KiB/s wr, 105 op/s 2024-07-31T15:04:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:49 smithi073 ceph-mon[36714]: pgmap v834: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 414 KiB/s wr, 105 op/s 2024-07-31T15:04:50.185 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T15:04:50.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T15:04:51.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.074+0000 7f503903a640 1 -- 172.21.15.67:0/1183569782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5034104ce0 msgr2=0x7f503410d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:51.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.074+0000 7f503903a640 1 --2- 172.21.15.67:0/1183569782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5034104ce0 0x7f503410d320 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5024009900 tx=0x7f502402a410 comp rx=0 tx=0).stop 2024-07-31T15:04:51.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.074+0000 7f503903a640 1 -- 172.21.15.67:0/1183569782 shutdown_connections 2024-07-31T15:04:51.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.074+0000 7f503903a640 1 --2- 172.21.15.67:0/1183569782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f503410d860 0x7f503410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:51.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.074+0000 7f503903a640 1 --2- 172.21.15.67:0/1183569782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5034104ce0 0x7f503410d320 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:51.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.074+0000 7f503903a640 1 --2- 172.21.15.67:0/1183569782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50341042b0 0x7f50341046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:51.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.074+0000 7f503903a640 1 -- 172.21.15.67:0/1183569782 >> 172.21.15.67:0/1183569782 conn(0x7f5034079160 msgr2=0x7f5034079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:51.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.074+0000 7f503903a640 1 -- 172.21.15.67:0/1183569782 shutdown_connections 2024-07-31T15:04:51.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.074+0000 7f503903a640 1 -- 172.21.15.67:0/1183569782 wait complete. 2024-07-31T15:04:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.075+0000 7f503903a640 1 Processor -- start 2024-07-31T15:04:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.075+0000 7f503903a640 1 -- start start 2024-07-31T15:04:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f503903a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50341042b0 0x7f50341021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f503903a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5034104ce0 0x7f5034102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f503903a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f503410d860 0x7f5034102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f503903a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5034113460 con 0x7f50341042b0 2024-07-31T15:04:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f503903a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f50341132e0 con 0x7f503410d860 2024-07-31T15:04:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f503903a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f50341135e0 con 0x7f5034104ce0 2024-07-31T15:04:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f5032d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50341042b0 0x7f50341021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f5032d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50341042b0 0x7f50341021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45866/0 (socket says 172.21.15.67:45866) 2024-07-31T15:04:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f5032d9d640 1 -- 172.21.15.67:0/384781798 learned_addr learned my addr 172.21.15.67:0/384781798 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f503259c640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5034104ce0 0x7f5034102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f503359e640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f503410d860 0x7f5034102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f5032d9d640 1 -- 172.21.15.67:0/384781798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5034104ce0 msgr2=0x7f5034102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f5032d9d640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5034104ce0 0x7f5034102700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f5032d9d640 1 -- 172.21.15.67:0/384781798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f503410d860 msgr2=0x7f5034102c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f5032d9d640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f503410d860 0x7f5034102c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f5032d9d640 1 -- 172.21.15.67:0/384781798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50341a81b0 con 0x7f50341042b0 2024-07-31T15:04:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.076+0000 7f503359e640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f503410d860 0x7f5034102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:04:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.077+0000 7f5032d9d640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50341042b0 0x7f50341021c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f501c00d770 tx=0x7f501c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.077+0000 7f502bfff640 1 -- 172.21.15.67:0/384781798 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c017070 con 0x7f50341042b0 2024-07-31T15:04:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.077+0000 7f503903a640 1 -- 172.21.15.67:0/384781798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50341a84a0 con 0x7f50341042b0 2024-07-31T15:04:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.077+0000 7f502bfff640 1 -- 172.21.15.67:0/384781798 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f501c009c30 con 0x7f50341042b0 2024-07-31T15:04:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.077+0000 7f502bfff640 1 -- 172.21.15.67:0/384781798 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c003cc0 con 0x7f50341042b0 2024-07-31T15:04:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.077+0000 7f503903a640 1 -- 172.21.15.67:0/384781798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50341a89b0 con 0x7f50341042b0 2024-07-31T15:04:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.078+0000 7f503903a640 1 -- 172.21.15.67:0/384781798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f503406aef0 con 0x7f50341042b0 2024-07-31T15:04:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.080+0000 7f502bfff640 1 -- 172.21.15.67:0/384781798 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f501c003e60 con 0x7f50341042b0 2024-07-31T15:04:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.081+0000 7f502bfff640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f50200b3d40 0x7f50200b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.081+0000 7f503259c640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f50200b3d40 0x7f50200b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.081+0000 7f502bfff640 1 -- 172.21.15.67:0/384781798 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f501c0e41b0 con 0x7f50341042b0 2024-07-31T15:04:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.081+0000 7f503259c640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f50200b3d40 0x7f50200b6250 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5024000c00 tx=0x7f5024031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.084+0000 7f502bfff640 1 -- 172.21.15.67:0/384781798 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f501c009de0 con 0x7f50341042b0 2024-07-31T15:04:51.097 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:04:51.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.321+0000 7f503903a640 1 -- 172.21.15.67:0/384781798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50341046c0 con 0x7f50341042b0 2024-07-31T15:04:51.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.322+0000 7f502bfff640 1 -- 172.21.15.67:0/384781798 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+11959 (secure 0 0 0) 0x7f501c0ab150 con 0x7f50341042b0 2024-07-31T15:04:51.324 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:04:51.324 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":63,"btime":"2024-07-31T15:03:07:099477+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:03:06.671820+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:04:51.324 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 63 2024-07-31T15:04:51.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.324+0000 7f503903a640 1 -- 172.21.15.67:0/384781798 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f50200b3d40 msgr2=0x7f50200b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:51.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.324+0000 7f503903a640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f50200b3d40 0x7f50200b6250 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5024000c00 tx=0x7f5024031400 comp rx=0 tx=0).stop 2024-07-31T15:04:51.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.324+0000 7f503903a640 1 -- 172.21.15.67:0/384781798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50341042b0 msgr2=0x7f50341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:51.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.324+0000 7f503903a640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50341042b0 0x7f50341021c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f501c00d770 tx=0x7f501c00dc30 comp rx=0 tx=0).stop 2024-07-31T15:04:51.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.324+0000 7f503903a640 1 -- 172.21.15.67:0/384781798 shutdown_connections 2024-07-31T15:04:51.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.324+0000 7f503903a640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f50200b3d40 0x7f50200b6250 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:51.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.324+0000 7f503903a640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f503410d860 0x7f5034102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:51.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.324+0000 7f503903a640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5034104ce0 0x7f5034102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:51.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.324+0000 7f503903a640 1 --2- 172.21.15.67:0/384781798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50341042b0 0x7f50341021c0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:51.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.324+0000 7f503903a640 1 -- 172.21.15.67:0/384781798 >> 172.21.15.67:0/384781798 conn(0x7f5034079160 msgr2=0x7f50341003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:51.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.325+0000 7f503903a640 1 -- 172.21.15.67:0/384781798 shutdown_connections 2024-07-31T15:04:51.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:51.325+0000 7f503903a640 1 -- 172.21.15.67:0/384781798 wait complete. 2024-07-31T15:04:51.336 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:51 smithi067 ceph-mon[32750]: pgmap v835: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 499 KiB/s wr, 110 op/s 2024-07-31T15:04:51.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:51 smithi077 ceph-mon[37327]: pgmap v835: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 499 KiB/s wr, 110 op/s 2024-07-31T15:04:51.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:51 smithi073 ceph-mon[36714]: pgmap v835: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 499 KiB/s wr, 110 op/s 2024-07-31T15:04:52.021 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 4d8b91f9 --release --await' 2024-07-31T15:04:52.021 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 4d8b91f9 --release --await 2024-07-31T15:04:52.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:52 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/384781798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:04:52.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:52 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/384781798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:04:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:52 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/384781798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:04:52.964 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:04:53.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:53 smithi067 ceph-mon[32750]: pgmap v836: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 552 KiB/s wr, 108 op/s 2024-07-31T15:04:53.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:53 smithi077 ceph-mon[37327]: pgmap v836: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 552 KiB/s wr, 108 op/s 2024-07-31T15:04:53.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:53 smithi073 ceph-mon[36714]: pgmap v836: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 552 KiB/s wr, 108 op/s 2024-07-31T15:04:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.105+0000 7f78304f5640 1 -- 172.21.15.67:0/858329289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7828104d70 msgr2=0x7f782810d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.105+0000 7f78304f5640 1 --2- 172.21.15.67:0/858329289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7828104d70 0x7f782810d3b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f781c009900 tx=0x7f781c02a410 comp rx=0 tx=0).stop 2024-07-31T15:04:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.105+0000 7f78304f5640 1 -- 172.21.15.67:0/858329289 shutdown_connections 2024-07-31T15:04:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.105+0000 7f78304f5640 1 --2- 172.21.15.67:0/858329289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f782810d8f0 0x7f782810fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.105+0000 7f78304f5640 1 --2- 172.21.15.67:0/858329289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7828104d70 0x7f782810d3b0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.105+0000 7f78304f5640 1 --2- 172.21.15.67:0/858329289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7828104340 0x7f7828104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.105+0000 7f78304f5640 1 -- 172.21.15.67:0/858329289 >> 172.21.15.67:0/858329289 conn(0x7f78280791a0 msgr2=0x7f78280795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.105+0000 7f78304f5640 1 -- 172.21.15.67:0/858329289 shutdown_connections 2024-07-31T15:04:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.105+0000 7f78304f5640 1 -- 172.21.15.67:0/858329289 wait complete. 2024-07-31T15:04:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.106+0000 7f78304f5640 1 Processor -- start 2024-07-31T15:04:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.106+0000 7f78304f5640 1 -- start start 2024-07-31T15:04:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.106+0000 7f78304f5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7828104340 0x7f7828199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.106+0000 7f78304f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7828104d70 0x7f782819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.106+0000 7f78304f5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f782810d8f0 0x7f782819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.106+0000 7f78304f5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7828113520 con 0x7f7828104340 2024-07-31T15:04:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.106+0000 7f78304f5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f78281133a0 con 0x7f782810d8f0 2024-07-31T15:04:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.106+0000 7f78304f5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78281136a0 con 0x7f7828104d70 2024-07-31T15:04:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.106+0000 7f782e291640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7828104340 0x7f7828199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.106+0000 7f782e291640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7828104340 0x7f7828199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45890/0 (socket says 172.21.15.67:45890) 2024-07-31T15:04:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.106+0000 7f782e291640 1 -- 172.21.15.67:0/1638119925 learned_addr learned my addr 172.21.15.67:0/1638119925 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.107+0000 7f782da90640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7828104d70 0x7f782819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.107+0000 7f782e291640 1 -- 172.21.15.67:0/1638119925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7828104d70 msgr2=0x7f782819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.107+0000 7f782ea92640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f782810d8f0 0x7f782819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.107+0000 7f782e291640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7828104d70 0x7f782819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.107+0000 7f782e291640 1 -- 172.21.15.67:0/1638119925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f782810d8f0 msgr2=0x7f782819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.107+0000 7f782e291640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f782810d8f0 0x7f782819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.107+0000 7f782e291640 1 -- 172.21.15.67:0/1638119925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782819fea0 con 0x7f7828104340 2024-07-31T15:04:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.107+0000 7f782da90640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7828104d70 0x7f782819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:04:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.107+0000 7f782e291640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7828104340 0x7f7828199f30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f781800db80 tx=0x7f781800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.107+0000 7f78277fe640 1 -- 172.21.15.67:0/1638119925 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818017070 con 0x7f7828104340 2024-07-31T15:04:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.107+0000 7f78277fe640 1 -- 172.21.15.67:0/1638119925 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7818003b70 con 0x7f7828104340 2024-07-31T15:04:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.107+0000 7f78304f5640 1 -- 172.21.15.67:0/1638119925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78281a0190 con 0x7f7828104340 2024-07-31T15:04:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.107+0000 7f78277fe640 1 -- 172.21.15.67:0/1638119925 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78180106c0 con 0x7f7828104340 2024-07-31T15:04:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.107+0000 7f78304f5640 1 -- 172.21.15.67:0/1638119925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78281a68f0 con 0x7f7828104340 2024-07-31T15:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.110+0000 7f7836a46740 1 Processor -- start 2024-07-31T15:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.110+0000 7f7836a46740 1 -- start start 2024-07-31T15:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.110+0000 7f7836a46740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29de350 0x563cb2d833b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.110+0000 7f7836a46740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29b5490 0x563cb2d838f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.110+0000 7f7836a46740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29bd670 0x563cb2d83e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.110+0000 7f7836a46740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563cb2c6b2f0 con 0x563cb29de350 2024-07-31T15:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.110+0000 7f7836a46740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563cb2cf9a80 con 0x563cb29b5490 2024-07-31T15:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.110+0000 7f7836a46740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563cb2ce8f80 con 0x563cb29bd670 2024-07-31T15:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.110+0000 7f78277fe640 1 -- 172.21.15.67:0/1638119925 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7818004b20 con 0x7f7828104340 2024-07-31T15:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.111+0000 7f782ea92640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29de350 0x563cb2d833b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.111+0000 7f782ea92640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29de350 0x563cb2d833b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45898/0 (socket says 172.21.15.67:45898) 2024-07-31T15:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.111+0000 7f782ea92640 1 -- 172.21.15.67:0/2699127440 learned_addr learned my addr 172.21.15.67:0/2699127440 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.111+0000 7f782da90640 1 --2- 172.21.15.67:0/2699127440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29bd670 0x563cb2d83e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.111+0000 7f782e291640 1 --2- 172.21.15.67:0/2699127440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29b5490 0x563cb2d838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.111+0000 7f782ea92640 1 -- 172.21.15.67:0/2699127440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29bd670 msgr2=0x563cb2d83e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.111+0000 7f782ea92640 1 --2- 172.21.15.67:0/2699127440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29bd670 0x563cb2d83e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.111+0000 7f782ea92640 1 -- 172.21.15.67:0/2699127440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29b5490 msgr2=0x563cb2d838f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.111+0000 7f782ea92640 1 --2- 172.21.15.67:0/2699127440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29b5490 0x563cb2d838f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.111+0000 7f782ea92640 1 -- 172.21.15.67:0/2699127440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cb2a49930 con 0x563cb29de350 2024-07-31T15:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.111+0000 7f782ea92640 1 --2- 172.21.15.67:0/2699127440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29de350 0x563cb2d833b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f7820009ab0 tx=0x7f7820009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2a787b3def6e6a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.111+0000 7f7824ff9640 1 -- 172.21.15.67:0/2699127440 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820016070 con 0x563cb29de350 2024-07-31T15:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.111+0000 7f7824ff9640 1 -- 172.21.15.67:0/2699127440 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f782000ecb0 con 0x563cb29de350 2024-07-31T15:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.111+0000 7f7824ff9640 1 -- 172.21.15.67:0/2699127440 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820010650 con 0x563cb29de350 2024-07-31T15:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.112+0000 7f78277fe640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78100b3d40 0x7f78100b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.112+0000 7f782e291640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78100b3d40 0x7f78100b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.112+0000 7f78277fe640 1 -- 172.21.15.67:0/1638119925 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f78180e45a0 con 0x7f7828104340 2024-07-31T15:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.112+0000 7f7836a46740 1 -- 172.21.15.67:0/2699127440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29de350 msgr2=0x563cb2d833b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.112+0000 7f782e291640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78100b3d40 0x7f78100b6250 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x563cb2d82030 tx=0x7f781800bce0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.112+0000 7f7836a46740 1 --2- 172.21.15.67:0/2699127440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29de350 0x563cb2d833b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f7820009ab0 tx=0x7f7820009f70 comp rx=0 tx=0).stop 2024-07-31T15:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.112+0000 7f7836a46740 1 -- 172.21.15.67:0/2699127440 shutdown_connections 2024-07-31T15:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.112+0000 7f7836a46740 1 --2- 172.21.15.67:0/2699127440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29bd670 0x563cb2d83e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.112+0000 7f7836a46740 1 --2- 172.21.15.67:0/2699127440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29b5490 0x563cb2d838f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.113+0000 7f7836a46740 1 --2- 172.21.15.67:0/2699127440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29de350 0x563cb2d833b0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.113+0000 7f7836a46740 1 -- 172.21.15.67:0/2699127440 >> 172.21.15.67:0/2699127440 conn(0x563cb29d2740 msgr2=0x563cb2d812a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.113+0000 7f7836a46740 1 -- 172.21.15.67:0/2699127440 shutdown_connections 2024-07-31T15:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.113+0000 7f7836a46740 1 -- 172.21.15.67:0/2699127440 wait complete. 2024-07-31T15:04:54.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.113+0000 7f7836a46740 1 -- start start 2024-07-31T15:04:54.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.113+0000 7f7836a46740 1 Processor -- start 2024-07-31T15:04:54.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.113+0000 7f7836a46740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29b5490 0x563cb2d8efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.113+0000 7f7836a46740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29bd670 0x563cb2d8f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.113+0000 7f7836a46740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29de350 0x563cb2d9ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.113+0000 7f7836a46740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563cb2c8b4f0 con 0x563cb29b5490 2024-07-31T15:04:54.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.113+0000 7f7836a46740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563cb2cecfc0 con 0x563cb29bd670 2024-07-31T15:04:54.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.113+0000 7f7836a46740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563cb2b4c890 con 0x563cb29de350 2024-07-31T15:04:54.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f7836a46740 10 client.0 _ll_register_callbacks cb 0x563cb2c95500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:54.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f782ea92640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29b5490 0x563cb2d8efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f782ea92640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29b5490 0x563cb2d8efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45912/0 (socket says 172.21.15.67:45912) 2024-07-31T15:04:54.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f782ea92640 1 -- 172.21.15.67:0/790662056 learned_addr learned my addr 172.21.15.67:0/790662056 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f782da90640 1 --2- 172.21.15.67:0/790662056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29bd670 0x563cb2d8f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f782ea92640 1 -- 172.21.15.67:0/790662056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29de350 msgr2=0x563cb2d9ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f782ea92640 1 --2- 172.21.15.67:0/790662056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29de350 0x563cb2d9ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f782ea92640 1 -- 172.21.15.67:0/790662056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29bd670 msgr2=0x563cb2d8f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f782ea92640 1 --2- 172.21.15.67:0/790662056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29bd670 0x563cb2d8f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f782ea92640 1 -- 172.21.15.67:0/790662056 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cb2c9ac50 con 0x563cb29b5490 2024-07-31T15:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f782ea92640 1 --2- 172.21.15.67:0/790662056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29b5490 0x563cb2d8efd0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f782000e780 tx=0x7f7820005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2590640d3ef03c96 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f780dffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f780dffb640 1 -- 172.21.15.67:0/790662056 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820016050 con 0x563cb29b5490 2024-07-31T15:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f780dffb640 1 -- 172.21.15.67:0/790662056 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f7820005570 con 0x563cb29b5490 2024-07-31T15:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.114+0000 7f7836a46740 1 -- 172.21.15.67:0/790662056 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f782000ecb0 con 0x563cb29b5490 2024-07-31T15:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.115+0000 7f780dffb640 1 -- 172.21.15.67:0/790662056 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f7820030050 con 0x563cb29b5490 2024-07-31T15:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.115+0000 7f7836a46740 10 client.18441 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.115+0000 7f7836a46740 1 -- 172.21.15.67:0/790662056 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x563cb2d9da90 con 0x563cb29b5490 2024-07-31T15:04:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.115+0000 7f780dffb640 1 -- 172.21.15.67:0/790662056 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f782000ecb0 con 0x563cb29b5490 2024-07-31T15:04:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.115+0000 7f780dffb640 1 -- 172.21.15.67:0/790662056 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f782002b030 con 0x563cb29b5490 2024-07-31T15:04:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.115+0000 7f7836a46740 10 client.18441 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.115+0000 7f7836a46740 10 client.18441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.115+0000 7f7836a46740 20 client.18441 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.115+0000 7f7836a46740 1 --2- 172.21.15.67:0/790662056 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563cb299ccd0 0x563cb2da0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.115+0000 7f7836a46740 4 client.18441 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:04:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.115+0000 7f7836a46740 1 -- 172.21.15.67:0/790662056 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cb2cfb9d0 con 0x563cb299ccd0 2024-07-31T15:04:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.115+0000 7f782da90640 1 --2- 172.21.15.67:0/790662056 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563cb299ccd0 0x563cb2da0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.116+0000 7f780dffb640 10 client.18441 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.116+0000 7f782da90640 1 --2- 172.21.15.67:0/790662056 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563cb299ccd0 0x563cb2da0f00 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c27435de2d2b841f server_cookie=27d61805122e2c9b in_seq=0 out_seq=0 2024-07-31T15:04:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f780dffb640 1 -- 172.21.15.67:0/790662056 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563cb2cfb9d0 con 0x563cb299ccd0 2024-07-31T15:04:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f780dffb640 10 client.18441 handle_command_reply: tid=0 2024-07-31T15:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f7836a46740 1 client.18441 shutdown 2024-07-31T15:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f7836a46740 1 -- 172.21.15.67:0/790662056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29b5490 msgr2=0x563cb2d8efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f7836a46740 1 --2- 172.21.15.67:0/790662056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29b5490 0x563cb2d8efd0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f782000e780 tx=0x7f7820005270 comp rx=0 tx=0).stop 2024-07-31T15:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f7836a46740 1 -- 172.21.15.67:0/790662056 shutdown_connections 2024-07-31T15:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f7836a46740 1 --2- 172.21.15.67:0/790662056 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563cb299ccd0 0x563cb2da0f00 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f7836a46740 1 --2- 172.21.15.67:0/790662056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29de350 0x563cb2d9ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f780dffb640 0 client.18441 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f7836a46740 1 --2- 172.21.15.67:0/790662056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29bd670 0x563cb2d8f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f7836a46740 1 --2- 172.21.15.67:0/790662056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29b5490 0x563cb2d8efd0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f7836a46740 1 -- 172.21.15.67:0/790662056 >> 172.21.15.67:0/790662056 conn(0x563cb29d2740 msgr2=0x563cb2d86590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f7836a46740 1 -- 172.21.15.67:0/790662056 shutdown_connections 2024-07-31T15:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f7836a46740 1 -- 172.21.15.67:0/790662056 wait complete. 2024-07-31T15:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.117+0000 7f7836a46740 20 client.18441 trim_cache size 0 max 16384 2024-07-31T15:04:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.136+0000 7f7836a46740 1 Processor -- start 2024-07-31T15:04:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.136+0000 7f7836a46740 1 -- start start 2024-07-31T15:04:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f7836a46740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29d2740 0x563cb2d90cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f7836a46740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29b5490 0x563cb2d9acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f7836a46740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29bd670 0x563cb2d9b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f7836a46740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f78281b0810 con 0x563cb29bd670 2024-07-31T15:04:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f7836a46740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f78281b2350 con 0x563cb29b5490 2024-07-31T15:04:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f7836a46740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78281b0ee0 con 0x563cb29d2740 2024-07-31T15:04:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f782ea92640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29bd670 0x563cb2d9b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f782ea92640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29bd670 0x563cb2d9b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45914/0 (socket says 172.21.15.67:45914) 2024-07-31T15:04:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f782ea92640 1 -- 172.21.15.67:0/2837366877 learned_addr learned my addr 172.21.15.67:0/2837366877 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f782da90640 1 --2- 172.21.15.67:0/2837366877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29b5490 0x563cb2d9acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f782ea92640 1 --2- 172.21.15.67:0/2837366877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29d2740 0x563cb2d90cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f782ea92640 1 -- 172.21.15.67:0/2837366877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29d2740 msgr2=0x563cb2d90cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f782ea92640 1 --2- 172.21.15.67:0/2837366877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29d2740 0x563cb2d90cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f782ea92640 1 -- 172.21.15.67:0/2837366877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29b5490 msgr2=0x563cb2d9acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f782ea92640 1 --2- 172.21.15.67:0/2837366877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29b5490 0x563cb2d9acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f782ea92640 1 -- 172.21.15.67:0/2837366877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cb2d81e60 con 0x563cb29bd670 2024-07-31T15:04:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.137+0000 7f782da90640 1 --2- 172.21.15.67:0/2837366877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29b5490 0x563cb2d9acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.138+0000 7f782ea92640 1 --2- 172.21.15.67:0/2837366877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29bd670 0x563cb2d9b210 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f78200059a0 tx=0x7f7820009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cdfe5c20b64c4a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.138+0000 7f780ffff640 1 -- 172.21.15.67:0/2837366877 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820016070 con 0x563cb29bd670 2024-07-31T15:04:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.138+0000 7f780ffff640 1 -- 172.21.15.67:0/2837366877 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7820005570 con 0x563cb29bd670 2024-07-31T15:04:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.138+0000 7f780ffff640 1 -- 172.21.15.67:0/2837366877 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782001bc50 con 0x563cb29bd670 2024-07-31T15:04:54.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.139+0000 7f7836a46740 1 -- 172.21.15.67:0/2837366877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29bd670 msgr2=0x563cb2d9b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:54.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.139+0000 7f7836a46740 1 --2- 172.21.15.67:0/2837366877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29bd670 0x563cb2d9b210 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f78200059a0 tx=0x7f7820009f70 comp rx=0 tx=0).stop 2024-07-31T15:04:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.139+0000 7f7836a46740 1 -- 172.21.15.67:0/2837366877 shutdown_connections 2024-07-31T15:04:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.139+0000 7f7836a46740 1 --2- 172.21.15.67:0/2837366877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29bd670 0x563cb2d9b210 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.139+0000 7f7836a46740 1 --2- 172.21.15.67:0/2837366877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29b5490 0x563cb2d9acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.139+0000 7f7836a46740 1 --2- 172.21.15.67:0/2837366877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29d2740 0x563cb2d90cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.139+0000 7f7836a46740 1 -- 172.21.15.67:0/2837366877 >> 172.21.15.67:0/2837366877 conn(0x563cb2d897c0 msgr2=0x563cb2d7ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.139+0000 7f7836a46740 1 -- 172.21.15.67:0/2837366877 shutdown_connections 2024-07-31T15:04:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.139+0000 7f7836a46740 1 -- 172.21.15.67:0/2837366877 wait complete. 2024-07-31T15:04:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.139+0000 7f7836a46740 1 -- start start 2024-07-31T15:04:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.139+0000 7f7836a46740 1 Processor -- start 2024-07-31T15:04:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f7836a46740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29b5490 0x563cb2da46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f7836a46740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29bd670 0x563cb2da2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f7836a46740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29d2740 0x563cb2da3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f7836a46740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f78281b1c80 con 0x563cb29b5490 2024-07-31T15:04:54.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f7836a46740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f78281b15b0 con 0x563cb29d2740 2024-07-31T15:04:54.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f7836a46740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78281ae810 con 0x563cb29bd670 2024-07-31T15:04:54.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f7836a46740 10 client.0 _ll_register_callbacks cb 0x7f7820011960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:54.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f782ea92640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29b5490 0x563cb2da46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f782ea92640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29b5490 0x563cb2da46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45916/0 (socket says 172.21.15.67:45916) 2024-07-31T15:04:54.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f782ea92640 1 -- 172.21.15.67:0/3752659557 learned_addr learned my addr 172.21.15.67:0/3752659557 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:54.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f782ea92640 1 --2- 172.21.15.67:0/3752659557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29d2740 0x563cb2da3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f782da90640 1 --2- 172.21.15.67:0/3752659557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29bd670 0x563cb2da2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f782ea92640 1 -- 172.21.15.67:0/3752659557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29bd670 msgr2=0x563cb2da2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:54.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f782ea92640 1 --2- 172.21.15.67:0/3752659557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29bd670 0x563cb2da2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f782ea92640 1 -- 172.21.15.67:0/3752659557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29d2740 msgr2=0x563cb2da3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:54.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f782ea92640 1 --2- 172.21.15.67:0/3752659557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29d2740 0x563cb2da3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f782ea92640 1 -- 172.21.15.67:0/3752659557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cb2d9b750 con 0x563cb29b5490 2024-07-31T15:04:54.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f782ea92640 1 --2- 172.21.15.67:0/3752659557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29b5490 0x563cb2da46e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f7820014040 tx=0x7f7820009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6103a444e8561c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:54.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f780dffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:04:54.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f780dffb640 1 -- 172.21.15.67:0/3752659557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820016070 con 0x563cb29b5490 2024-07-31T15:04:54.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f7836a46740 1 -- 172.21.15.67:0/3752659557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7820005570 con 0x563cb29b5490 2024-07-31T15:04:54.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.140+0000 7f780dffb640 1 -- 172.21.15.67:0/3752659557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f78200092a0 con 0x563cb29b5490 2024-07-31T15:04:54.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.141+0000 7f780dffb640 1 -- 172.21.15.67:0/3752659557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f7820049050 con 0x563cb29b5490 2024-07-31T15:04:54.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.141+0000 7f7836a46740 10 client.18447 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:54.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.141+0000 7f7836a46740 1 -- 172.21.15.67:0/3752659557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x563cb2e0e6f0 con 0x563cb29b5490 2024-07-31T15:04:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.141+0000 7f780dffb640 1 -- 172.21.15.67:0/3752659557 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7820005570 con 0x563cb29b5490 2024-07-31T15:04:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.141+0000 7f780dffb640 1 -- 172.21.15.67:0/3752659557 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f7820040070 con 0x563cb29b5490 2024-07-31T15:04:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.141+0000 7f7836a46740 10 client.18447 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.141+0000 7f7836a46740 10 client.18447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.141+0000 7f7836a46740 20 client.18447 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.141+0000 7f7836a46740 1 --2- 172.21.15.67:0/3752659557 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563cb29de350 0x563cb2e10c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.141+0000 7f7836a46740 4 client.18447 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T15:04:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.141+0000 7f7836a46740 1 -- 172.21.15.67:0/3752659557 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563cb2cf7470 con 0x563cb29de350 2024-07-31T15:04:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.142+0000 7f782da90640 1 --2- 172.21.15.67:0/3752659557 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563cb29de350 0x563cb2e10c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.142+0000 7f780dffb640 10 client.18447 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.142+0000 7f782da90640 1 --2- 172.21.15.67:0/3752659557 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563cb29de350 0x563cb2e10c60 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdf8f6c870435339 server_cookie=fbf55cfa6384e47e in_seq=0 out_seq=0 2024-07-31T15:04:54.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.143+0000 7f780dffb640 1 -- 172.21.15.67:0/3752659557 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563cb2cf7470 con 0x563cb29de350 2024-07-31T15:04:54.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.143+0000 7f780dffb640 10 client.18447 handle_command_reply: tid=0 2024-07-31T15:04:54.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.143+0000 7f7836a46740 1 client.18447 shutdown 2024-07-31T15:04:54.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.143+0000 7f7836a46740 1 -- 172.21.15.67:0/3752659557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29b5490 msgr2=0x563cb2da46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:54.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.143+0000 7f7836a46740 1 --2- 172.21.15.67:0/3752659557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29b5490 0x563cb2da46e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f7820014040 tx=0x7f7820009830 comp rx=0 tx=0).stop 2024-07-31T15:04:54.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.143+0000 7f7836a46740 1 -- 172.21.15.67:0/3752659557 shutdown_connections 2024-07-31T15:04:54.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.143+0000 7f7836a46740 1 --2- 172.21.15.67:0/3752659557 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563cb29de350 0x563cb2e10c60 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.143+0000 7f7836a46740 1 --2- 172.21.15.67:0/3752659557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563cb29d2740 0x563cb2da3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.143+0000 7f7836a46740 1 --2- 172.21.15.67:0/3752659557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cb29bd670 0x563cb2da2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.143+0000 7f7836a46740 1 --2- 172.21.15.67:0/3752659557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563cb29b5490 0x563cb2da46e0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.143+0000 7f7836a46740 1 -- 172.21.15.67:0/3752659557 >> 172.21.15.67:0/3752659557 conn(0x563cb2d897c0 msgr2=0x563cb2d90a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:54.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.144+0000 7f780dffb640 0 client.18447 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:54.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.144+0000 7f7836a46740 1 -- 172.21.15.67:0/3752659557 shutdown_connections 2024-07-31T15:04:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.144+0000 7f7836a46740 1 -- 172.21.15.67:0/3752659557 wait complete. 2024-07-31T15:04:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.144+0000 7f7836a46740 20 client.18447 trim_cache size 0 max 16384 2024-07-31T15:04:54.150 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:04:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T15:04:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "664fb307-5b5c-42e3-967e-742a749d1599", 2024-07-31T15:04:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T15:04:54.150 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:04:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.144+0000 7f78304f5640 1 -- 172.21.15.67:0/1638119925 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78100b3d40 msgr2=0x7f78100b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.144+0000 7f78304f5640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78100b3d40 0x7f78100b6250 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x563cb2d82030 tx=0x7f781800bce0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.144+0000 7f78304f5640 1 -- 172.21.15.67:0/1638119925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7828104340 msgr2=0x7f7828199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.144+0000 7f78304f5640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7828104340 0x7f7828199f30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f781800db80 tx=0x7f781800b630 comp rx=0 tx=0).stop 2024-07-31T15:04:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.144+0000 7f78304f5640 1 -- 172.21.15.67:0/1638119925 shutdown_connections 2024-07-31T15:04:54.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.144+0000 7f78304f5640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78100b3d40 0x7f78100b6250 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.144+0000 7f78304f5640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f782810d8f0 0x7f782819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.144+0000 7f78304f5640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7828104d70 0x7f782819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:54.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.144+0000 7f78304f5640 1 --2- 172.21.15.67:0/1638119925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7828104340 0x7f7828199f30 secure :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f781800db80 tx=0x7f781800b630 comp rx=0 tx=0).stop 2024-07-31T15:04:54.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.144+0000 7f78304f5640 1 -- 172.21.15.67:0/1638119925 >> 172.21.15.67:0/1638119925 conn(0x7f78280791a0 msgr2=0x7f7828100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:54.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.145+0000 7f78304f5640 1 -- 172.21.15.67:0/1638119925 shutdown_connections 2024-07-31T15:04:54.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:54.145+0000 7f78304f5640 1 -- 172.21.15.67:0/1638119925 wait complete. 2024-07-31T15:04:54.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:54 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T15:04:54.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:54.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:54 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:54.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:54 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T15:04:54.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:54.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:54 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:54.556 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:54 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T15:04:54.556 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:54.556 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:54 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:54.892 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:04:54 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:04:54.554+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000ce48 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity/ auth{1=1,2=1} v1695 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3) n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4) old_inodes=3 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:244274 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@9} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8aaac580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000ce48:[<0x1000000ce07/integrity v345>,<0x10000000202/security v8766>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000ce48:[<0x1000000ce07/integrity v1695>,<0x10000000202/security v15110>,<0x10000000201/linux-6.5.11 v1841>,<0x10000000006/t v1016>,<0x10000000005/tmp v969>,<0x100000001fb/client.0 v949>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v932>,<0x10000000001/sv_1 v933>,<0x10000000000/qa v929>,<0x1/volumes v893>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:59.470106+0000 10=7+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","memory_value.dirstat":"f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3)","memory_value.rstat":"n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:04:55.296 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: pgmap v837: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 614 KiB/s wr, 108 op/s 2024-07-31T15:04:55.335 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.335 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.336 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.336 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.336 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.336 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.336 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.336 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.336 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.336 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.337 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.337 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.337 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.337 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.337 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.337 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.337 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.337 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.337 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.337 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.337 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.338 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.338 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.338 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.338 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.338 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:04:55.338 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.338 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.338 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.339 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.339 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.339 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.339 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.339 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.339 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.339 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.339 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.339 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.340 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:55.340 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:55 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:04:55.342 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:04:55.393 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:04:54 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:04:54.994+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000be32 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module/ auth{2=1} v2029 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0) n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:253519 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@5} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce826b7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000be32:[<0x1000000bd0a/module v1134>,<0x10000000202/kernel v5648>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000be32:[<0x1000000bd0a/module v2029>,<0x10000000202/kernel v15111>,<0x10000000201/linux-6.5.11 v1841>,<0x10000000006/t v1016>,<0x10000000005/tmp v969>,<0x100000001fb/client.0 v949>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v932>,<0x10000000001/sv_1 v933>,<0x10000000000/qa v929>,<0x1/volumes v893>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:07.168188+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:04:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: pgmap v837: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 614 KiB/s wr, 108 op/s 2024-07-31T15:04:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:04:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.397 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:55.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:04:55.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: pgmap v837: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 614 KiB/s wr, 108 op/s 2024-07-31T15:04:55.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:04:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:55.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:55 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:04:55.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:04:55 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:04:55.472+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000cde2 [...10,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders/ auth{4=1} v112 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0) n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:269273 nref=3)) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8dc52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v29>,<0x1000000cbec/selinux v646>,<0x10000000202/scripts v8820>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v112>,<0x1000000cbec/selinux v749>,<0x10000000202/scripts v11639>,<0x10000000201/linux-6.5.11 v1841>,<0x10000000006/t v1016>,<0x10000000005/tmp v969>,<0x100000001fb/client.0 v949>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v932>,<0x10000000001/sv_1 v933>,<0x10000000000/qa v929>,<0x1/volumes v893>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:58.999116+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:04:56.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.996+0000 7f721db84640 1 -- 172.21.15.67:0/2793705797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f721810fc40 msgr2=0x7f7218112060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:56.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.996+0000 7f721db84640 1 --2- 172.21.15.67:0/2793705797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f721810fc40 0x7f7218112060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7210009be0 tx=0x7f721002a750 comp rx=0 tx=0).stop 2024-07-31T15:04:56.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.996+0000 7f721db84640 1 -- 172.21.15.67:0/2793705797 shutdown_connections 2024-07-31T15:04:56.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.996+0000 7f721db84640 1 --2- 172.21.15.67:0/2793705797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f721810fc40 0x7f7218112060 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.996+0000 7f721db84640 1 --2- 172.21.15.67:0/2793705797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72181029e0 0x7f721810f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.996+0000 7f721db84640 1 --2- 172.21.15.67:0/2793705797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7218102090 0x7f72181024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.996+0000 7f721db84640 1 -- 172.21.15.67:0/2793705797 >> 172.21.15.67:0/2793705797 conn(0x7f7218079160 msgr2=0x7f7218079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:56.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.996+0000 7f721db84640 1 -- 172.21.15.67:0/2793705797 shutdown_connections 2024-07-31T15:04:56.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.996+0000 7f721db84640 1 -- 172.21.15.67:0/2793705797 wait complete. 2024-07-31T15:04:56.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.997+0000 7f721db84640 1 Processor -- start 2024-07-31T15:04:56.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.998+0000 7f721db84640 1 -- start start 2024-07-31T15:04:56.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.998+0000 7f721db84640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7218102090 0x7f721819c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.998+0000 7f721db84640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72181029e0 0x7f721819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.998+0000 7f721db84640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f721810fc40 0x7f72181a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.998+0000 7f721db84640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7218115670 con 0x7f72181029e0 2024-07-31T15:04:56.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.998+0000 7f721db84640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f72181154f0 con 0x7f721810fc40 2024-07-31T15:04:56.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.998+0000 7f721db84640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f72181157f0 con 0x7f7218102090 2024-07-31T15:04:56.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.998+0000 7f72177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7218102090 0x7f721819c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.998+0000 7f7216ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72181029e0 0x7f721819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.998+0000 7f72177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7218102090 0x7f721819c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58750/0 (socket says 172.21.15.67:58750) 2024-07-31T15:04:56.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.998+0000 7f72177fe640 1 -- 172.21.15.67:0/3898701846 learned_addr learned my addr 172.21.15.67:0/3898701846 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:56.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.998+0000 7f7217fff640 1 --2- 172.21.15.67:0/3898701846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f721810fc40 0x7f72181a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.999+0000 7f72177fe640 1 -- 172.21.15.67:0/3898701846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f721810fc40 msgr2=0x7f72181a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:56.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.999+0000 7f72177fe640 1 --2- 172.21.15.67:0/3898701846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f721810fc40 0x7f72181a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.999+0000 7f72177fe640 1 -- 172.21.15.67:0/3898701846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72181029e0 msgr2=0x7f721819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:56.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.999+0000 7f72177fe640 1 --2- 172.21.15.67:0/3898701846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72181029e0 0x7f721819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.999+0000 7f72177fe640 1 -- 172.21.15.67:0/3898701846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72181a1f90 con 0x7f7218102090 2024-07-31T15:04:56.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.999+0000 7f72177fe640 1 --2- 172.21.15.67:0/3898701846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7218102090 0x7f721819c020 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f720800e9f0 tx=0x7f720800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:56.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:55.999+0000 7f7214ff9640 1 -- 172.21.15.67:0/3898701846 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720800cd80 con 0x7f7218102090 2024-07-31T15:04:56.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.000+0000 7f7214ff9640 1 -- 172.21.15.67:0/3898701846 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72080054d0 con 0x7f7218102090 2024-07-31T15:04:56.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.000+0000 7f721db84640 1 -- 172.21.15.67:0/3898701846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72181a2280 con 0x7f7218102090 2024-07-31T15:04:56.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.000+0000 7f7214ff9640 1 -- 172.21.15.67:0/3898701846 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7208010690 con 0x7f7218102090 2024-07-31T15:04:56.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.000+0000 7f721db84640 1 -- 172.21.15.67:0/3898701846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72181a89e0 con 0x7f7218102090 2024-07-31T15:04:56.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.004+0000 7f72240d5740 1 Processor -- start 2024-07-31T15:04:56.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.004+0000 7f72240d5740 1 -- start start 2024-07-31T15:04:56.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.004+0000 7f7214ff9640 1 -- 172.21.15.67:0/3898701846 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7208010830 con 0x7f7218102090 2024-07-31T15:04:56.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.004+0000 7f72240d5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d193520 0x55558d541880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.004+0000 7f72240d5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d541dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.004+0000 7f72240d5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d172840 0x55558d542300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.004+0000 7f72240d5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55558d4aee70 con 0x55558d172840 2024-07-31T15:04:56.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.004+0000 7f72240d5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55558d49e370 con 0x55558d193520 2024-07-31T15:04:56.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.004+0000 7f72240d5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55558d4a23b0 con 0x55558d16a660 2024-07-31T15:04:56.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.004+0000 7f72177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d541dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.004+0000 7f7217fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d193520 0x55558d541880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.004+0000 7f72177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d541dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58762/0 (socket says 172.21.15.67:58762) 2024-07-31T15:04:56.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.004+0000 7f72177fe640 1 -- 172.21.15.67:0/346704178 learned_addr learned my addr 172.21.15.67:0/346704178 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:56.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.005+0000 7f72177fe640 1 -- 172.21.15.67:0/346704178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d193520 msgr2=0x55558d541880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:56.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.005+0000 7f72177fe640 1 --2- 172.21.15.67:0/346704178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d193520 0x55558d541880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.005+0000 7f72177fe640 1 -- 172.21.15.67:0/346704178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d172840 msgr2=0x55558d542300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:56.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.005+0000 7f72177fe640 1 --2- 172.21.15.67:0/346704178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d172840 0x55558d542300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.005+0000 7f72177fe640 1 -- 172.21.15.67:0/346704178 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55558d46b040 con 0x55558d16a660 2024-07-31T15:04:56.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.005+0000 7f72177fe640 1 --2- 172.21.15.67:0/346704178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d541dc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f72080e1a50 tx=0x7f72080eab40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=785b552006f7bef3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:56.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.006+0000 7f71fdffb640 1 -- 172.21.15.67:0/346704178 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72080f3070 con 0x55558d16a660 2024-07-31T15:04:56.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.006+0000 7f71fdffb640 1 -- 172.21.15.67:0/346704178 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72080e4db0 con 0x55558d16a660 2024-07-31T15:04:56.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.006+0000 7f7214ff9640 1 --2- 172.21.15.67:0/3898701846 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f72000b3dc0 0x7f72000b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.006+0000 7f71fdffb640 1 -- 172.21.15.67:0/346704178 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72080e69d0 con 0x55558d16a660 2024-07-31T15:04:56.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.006+0000 7f7214ff9640 1 -- 172.21.15.67:0/3898701846 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f72080e4700 con 0x7f7218102090 2024-07-31T15:04:56.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.006+0000 7f7217fff640 1 --2- 172.21.15.67:0/3898701846 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f72000b3dc0 0x7f72000b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.007+0000 7f72240d5740 1 -- 172.21.15.67:0/346704178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 msgr2=0x55558d541dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:56.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.007+0000 7f72240d5740 1 --2- 172.21.15.67:0/346704178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d541dc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f72080e1a50 tx=0x7f72080eab40 comp rx=0 tx=0).stop 2024-07-31T15:04:56.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.007+0000 7f7217fff640 1 --2- 172.21.15.67:0/3898701846 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f72000b3dc0 0x7f72000b62d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7210006010 tx=0x7f7210009860 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:56.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.007+0000 7f72240d5740 1 -- 172.21.15.67:0/346704178 shutdown_connections 2024-07-31T15:04:56.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.007+0000 7f72240d5740 1 --2- 172.21.15.67:0/346704178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d172840 0x55558d542300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.007+0000 7f72240d5740 1 --2- 172.21.15.67:0/346704178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d541dc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.007+0000 7f72240d5740 1 --2- 172.21.15.67:0/346704178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d193520 0x55558d541880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.007+0000 7f72240d5740 1 -- 172.21.15.67:0/346704178 >> 172.21.15.67:0/346704178 conn(0x55558d187910 msgr2=0x55558d53f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:56.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.008+0000 7f72240d5740 1 -- 172.21.15.67:0/346704178 shutdown_connections 2024-07-31T15:04:56.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.008+0000 7f72240d5740 1 -- 172.21.15.67:0/346704178 wait complete. 2024-07-31T15:04:56.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.008+0000 7f72240d5740 1 -- start start 2024-07-31T15:04:56.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.008+0000 7f72240d5740 1 Processor -- start 2024-07-31T15:04:56.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.009+0000 7f72240d5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d16a660 0x55558d54d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.009+0000 7f72240d5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d172840 0x55558d54d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.009+0000 7f72240d5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d193520 0x55558d5593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.009+0000 7f72240d5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55558d301bb0 con 0x55558d193520 2024-07-31T15:04:56.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.009+0000 7f72240d5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55558d440ac0 con 0x55558d16a660 2024-07-31T15:04:56.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.009+0000 7f72240d5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55558d5500b0 con 0x55558d172840 2024-07-31T15:04:56.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.009+0000 7f72240d5740 10 client.0 _ll_register_callbacks cb 0x55558d4493b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:56.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.009+0000 7f72177fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d16a660 0x55558d54d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.009+0000 7f7216ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d172840 0x55558d54d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.009+0000 7f72177fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d16a660 0x55558d54d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45580/0 (socket says 172.21.15.67:45580) 2024-07-31T15:04:56.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.009+0000 7f72177fe640 1 -- 172.21.15.67:0/681636737 learned_addr learned my addr 172.21.15.67:0/681636737 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:56.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.009+0000 7f7216ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d172840 0x55558d54d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58774/0 (socket says 172.21.15.67:58774) 2024-07-31T15:04:56.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.009+0000 7f7217fff640 1 --2- 172.21.15.67:0/681636737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d193520 0x55558d5593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.010+0000 7f72177fe640 1 -- 172.21.15.67:0/681636737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d172840 msgr2=0x55558d54d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:56.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.010+0000 7f72177fe640 1 --2- 172.21.15.67:0/681636737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d172840 0x55558d54d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.010+0000 7f72177fe640 1 -- 172.21.15.67:0/681636737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d193520 msgr2=0x55558d5593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:56.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.010+0000 7f72177fe640 1 --2- 172.21.15.67:0/681636737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d193520 0x55558d5593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.010+0000 7f72177fe640 1 -- 172.21.15.67:0/681636737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55558d1feb00 con 0x55558d16a660 2024-07-31T15:04:56.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.010+0000 7f7216ffd640 1 --2- 172.21.15.67:0/681636737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d172840 0x55558d54d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:56.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.010+0000 7f7217fff640 1 --2- 172.21.15.67:0/681636737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d193520 0x55558d5593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:56.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.010+0000 7f72177fe640 1 --2- 172.21.15.67:0/681636737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d16a660 0x55558d54d430 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f72080a7020 tx=0x7f72080e65d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad03508e3530b1b4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:56.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.010+0000 7f71f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:04:56.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.010+0000 7f71f6ffd640 1 -- 172.21.15.67:0/681636737 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72080f3070 con 0x55558d16a660 2024-07-31T15:04:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.010+0000 7f71f6ffd640 1 -- 172.21.15.67:0/681636737 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f72080e4bb0 con 0x55558d16a660 2024-07-31T15:04:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.010+0000 7f72240d5740 1 -- 172.21.15.67:0/681636737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72080e4db0 con 0x55558d16a660 2024-07-31T15:04:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.011+0000 7f71f6ffd640 1 -- 172.21.15.67:0/681636737 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f72080a6e60 con 0x55558d16a660 2024-07-31T15:04:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.011+0000 7f72240d5740 10 client.28013 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.011+0000 7f72240d5740 1 -- 172.21.15.67:0/681636737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55558d275960 con 0x55558d16a660 2024-07-31T15:04:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.011+0000 7f71f6ffd640 1 -- 172.21.15.67:0/681636737 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72080e4db0 con 0x55558d16a660 2024-07-31T15:04:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.011+0000 7f71f6ffd640 1 -- 172.21.15.67:0/681636737 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f72080a7230 con 0x55558d16a660 2024-07-31T15:04:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.011+0000 7f72240d5740 10 client.28013 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.011+0000 7f72240d5740 10 client.28013 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:04:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.011+0000 7f72240d5740 20 client.28013 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.012+0000 7f72240d5740 1 --2- 172.21.15.67:0/681636737 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55558d151ea0 0x55558d55f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.012+0000 7f72240d5740 4 client.28013 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:04:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.012+0000 7f72240d5740 1 -- 172.21.15.67:0/681636737 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55558d4b0dc0 con 0x55558d151ea0 2024-07-31T15:04:56.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.012+0000 7f7216ffd640 1 --2- 172.21.15.67:0/681636737 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55558d151ea0 0x55558d55f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.013+0000 7f7216ffd640 1 --2- 172.21.15.67:0/681636737 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55558d151ea0 0x55558d55f280 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35b6f8aea825e856 server_cookie=e6d967c1faf40302 in_seq=0 out_seq=0 2024-07-31T15:04:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.013+0000 7f71f6ffd640 10 client.28013 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.015+0000 7f71f6ffd640 1 -- 172.21.15.67:0/681636737 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55558d4b0dc0 con 0x55558d151ea0 2024-07-31T15:04:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.015+0000 7f71f6ffd640 10 client.28013 handle_command_reply: tid=0 2024-07-31T15:04:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.015+0000 7f72240d5740 1 client.28013 shutdown 2024-07-31T15:04:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.015+0000 7f72240d5740 1 -- 172.21.15.67:0/681636737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d16a660 msgr2=0x55558d54d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.015+0000 7f72240d5740 1 --2- 172.21.15.67:0/681636737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d16a660 0x55558d54d430 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f72080a7020 tx=0x7f72080e65d0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.015+0000 7f72240d5740 1 -- 172.21.15.67:0/681636737 shutdown_connections 2024-07-31T15:04:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.015+0000 7f72240d5740 1 --2- 172.21.15.67:0/681636737 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55558d151ea0 0x55558d55f280 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.015+0000 7f72240d5740 1 --2- 172.21.15.67:0/681636737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d193520 0x55558d5593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.015+0000 7f72240d5740 1 --2- 172.21.15.67:0/681636737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d172840 0x55558d54d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.015+0000 7f71f6ffd640 0 client.28013 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:56.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.015+0000 7f72240d5740 1 --2- 172.21.15.67:0/681636737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d16a660 0x55558d54d430 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.015+0000 7f72240d5740 1 -- 172.21.15.67:0/681636737 >> 172.21.15.67:0/681636737 conn(0x55558d187910 msgr2=0x55558d544a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.015+0000 7f72240d5740 1 -- 172.21.15.67:0/681636737 shutdown_connections 2024-07-31T15:04:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.016+0000 7f72240d5740 1 -- 172.21.15.67:0/681636737 wait complete. 2024-07-31T15:04:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.016+0000 7f72240d5740 20 client.28013 trim_cache size 0 max 16384 2024-07-31T15:04:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.043+0000 7f72240d5740 1 Processor -- start 2024-07-31T15:04:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.043+0000 7f72240d5740 1 -- start start 2024-07-31T15:04:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.044+0000 7f72240d5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d543270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.044+0000 7f72240d5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d172840 0x55558d559340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.044+0000 7f72240d5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d193520 0x55558d559880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.044+0000 7f72240d5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f72181ae4a0 con 0x55558d193520 2024-07-31T15:04:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.044+0000 7f72240d5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f72181affe0 con 0x55558d172840 2024-07-31T15:04:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.044+0000 7f72240d5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f72181aeb70 con 0x55558d16a660 2024-07-31T15:04:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.044+0000 7f7217fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d193520 0x55558d559880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.044+0000 7f7217fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d193520 0x55558d559880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45958/0 (socket says 172.21.15.67:45958) 2024-07-31T15:04:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.044+0000 7f7217fff640 1 -- 172.21.15.67:0/1127967515 learned_addr learned my addr 172.21.15.67:0/1127967515 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.044+0000 7f72177fe640 1 --2- 172.21.15.67:0/1127967515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d543270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.044+0000 7f7216ffd640 1 --2- 172.21.15.67:0/1127967515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d172840 0x55558d559340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.045+0000 7f72177fe640 1 -- 172.21.15.67:0/1127967515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d172840 msgr2=0x55558d559340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.045+0000 7f72177fe640 1 --2- 172.21.15.67:0/1127967515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d172840 0x55558d559340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.045+0000 7f72177fe640 1 -- 172.21.15.67:0/1127967515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d193520 msgr2=0x55558d559880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.045+0000 7f72177fe640 1 --2- 172.21.15.67:0/1127967515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d193520 0x55558d559880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.045+0000 7f72177fe640 1 -- 172.21.15.67:0/1127967515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55558d543b70 con 0x55558d16a660 2024-07-31T15:04:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.045+0000 7f72177fe640 1 --2- 172.21.15.67:0/1127967515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d543270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7208009230 tx=0x7f72080eab40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e81438e36f0361b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.045+0000 7f71fd7fa640 1 -- 172.21.15.67:0/1127967515 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72080f3070 con 0x55558d16a660 2024-07-31T15:04:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.045+0000 7f71fd7fa640 1 -- 172.21.15.67:0/1127967515 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72080aaab0 con 0x55558d16a660 2024-07-31T15:04:56.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.045+0000 7f71fd7fa640 1 -- 172.21.15.67:0/1127967515 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72080aa710 con 0x55558d16a660 2024-07-31T15:04:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.046+0000 7f72240d5740 1 -- 172.21.15.67:0/1127967515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 msgr2=0x55558d543270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.046+0000 7f72240d5740 1 --2- 172.21.15.67:0/1127967515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d543270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7208009230 tx=0x7f72080eab40 comp rx=0 tx=0).stop 2024-07-31T15:04:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.047+0000 7f72240d5740 1 -- 172.21.15.67:0/1127967515 shutdown_connections 2024-07-31T15:04:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.047+0000 7f72240d5740 1 --2- 172.21.15.67:0/1127967515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d193520 0x55558d559880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.047+0000 7f72240d5740 1 --2- 172.21.15.67:0/1127967515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d172840 0x55558d559340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.047+0000 7f72240d5740 1 --2- 172.21.15.67:0/1127967515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d543270 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.047+0000 7f72240d5740 1 -- 172.21.15.67:0/1127967515 >> 172.21.15.67:0/1127967515 conn(0x55558d187910 msgr2=0x55558d53f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.047+0000 7f72240d5740 1 -- 172.21.15.67:0/1127967515 shutdown_connections 2024-07-31T15:04:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.047+0000 7f72240d5740 1 -- 172.21.15.67:0/1127967515 wait complete. 2024-07-31T15:04:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.047+0000 7f72240d5740 1 -- start start 2024-07-31T15:04:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.048+0000 7f72240d5740 1 Processor -- start 2024-07-31T15:04:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.048+0000 7f72240d5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d55fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.048+0000 7f72240d5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d172840 0x55558d560200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.048+0000 7f72240d5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d193520 0x55558d5be780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.048+0000 7f72240d5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f72181af910 con 0x55558d172840 2024-07-31T15:04:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.048+0000 7f72240d5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f72181af240 con 0x55558d193520 2024-07-31T15:04:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.048+0000 7f72240d5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f72181ac4e0 con 0x55558d16a660 2024-07-31T15:04:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.048+0000 7f72240d5740 10 client.0 _ll_register_callbacks cb 0x7f72080ea900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.048+0000 7f7216ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d172840 0x55558d560200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.048+0000 7f7216ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d172840 0x55558d560200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45974/0 (socket says 172.21.15.67:45974) 2024-07-31T15:04:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.048+0000 7f7216ffd640 1 -- 172.21.15.67:0/2331942475 learned_addr learned my addr 172.21.15.67:0/2331942475 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.048+0000 7f7217fff640 1 --2- 172.21.15.67:0/2331942475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d193520 0x55558d5be780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.048+0000 7f72177fe640 1 --2- 172.21.15.67:0/2331942475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d55fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.049+0000 7f7216ffd640 1 -- 172.21.15.67:0/2331942475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 msgr2=0x55558d55fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:56.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.049+0000 7f7216ffd640 1 --2- 172.21.15.67:0/2331942475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d55fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.049+0000 7f7216ffd640 1 -- 172.21.15.67:0/2331942475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d193520 msgr2=0x55558d5be780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.049+0000 7f7216ffd640 1 --2- 172.21.15.67:0/2331942475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d193520 0x55558d5be780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.049+0000 7f7216ffd640 1 -- 172.21.15.67:0/2331942475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55558d5bedf0 con 0x55558d172840 2024-07-31T15:04:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.049+0000 7f7216ffd640 1 --2- 172.21.15.67:0/2331942475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d172840 0x55558d560200 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f7218111a90 tx=0x7f720c013a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d897a2deb430a01f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.049+0000 7f71f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:04:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.049+0000 7f71f6ffd640 1 -- 172.21.15.67:0/2331942475 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c004220 con 0x55558d172840 2024-07-31T15:04:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.049+0000 7f71f6ffd640 1 -- 172.21.15.67:0/2331942475 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f720c0043c0 con 0x55558d172840 2024-07-31T15:04:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.049+0000 7f72240d5740 1 -- 172.21.15.67:0/2331942475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72080aaab0 con 0x55558d172840 2024-07-31T15:04:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.050+0000 7f71f6ffd640 1 -- 172.21.15.67:0/2331942475 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f720c005020 con 0x55558d172840 2024-07-31T15:04:56.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.050+0000 7f72240d5740 10 client.18462 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.050+0000 7f72240d5740 1 -- 172.21.15.67:0/2331942475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55558d5c73a0 con 0x55558d172840 2024-07-31T15:04:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.050+0000 7f71f6ffd640 1 -- 172.21.15.67:0/2331942475 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72080aaab0 con 0x55558d172840 2024-07-31T15:04:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.050+0000 7f71f6ffd640 1 -- 172.21.15.67:0/2331942475 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f720c01c550 con 0x55558d172840 2024-07-31T15:04:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.050+0000 7f72240d5740 10 client.18462 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.050+0000 7f72240d5740 10 client.18462 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:04:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.050+0000 7f72240d5740 20 client.18462 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.050+0000 7f72240d5740 1 --2- 172.21.15.67:0/2331942475 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55558d127250 0x55558d5c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.050+0000 7f72240d5740 4 client.18462 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4d8b91f9", "release": true, "await": true}] 2024-07-31T15:04:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.050+0000 7f72240d5740 1 -- 172.21.15.67:0/2331942475 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4d8b91f9", "release": true, "await": true}) -- 0x55558d4ac860 con 0x55558d127250 2024-07-31T15:04:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.051+0000 7f72177fe640 1 --2- 172.21.15.67:0/2331942475 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55558d127250 0x55558d5c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.051+0000 7f72177fe640 1 --2- 172.21.15.67:0/2331942475 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55558d127250 0x55558d5c9b90 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d919ac4ce4b54fe server_cookie=ee0dd278e2b6884c in_seq=0 out_seq=0 2024-07-31T15:04:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:56.051+0000 7f71f6ffd640 10 client.18462 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:56.114 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:04:56.252 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:56.252 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:56.252 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:56.252 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:56.252 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:56.252 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:56.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:56.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:56.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:56 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:04:56.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:56.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:56.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:56.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:56.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:56.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:56.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:56.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:56.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:56.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:56 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:04:56.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:56 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:04:56.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:57 smithi077 ceph-mon[37327]: pgmap v838: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 576 KiB/s wr, 168 op/s 2024-07-31T15:04:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:57.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:57.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:57 smithi067 ceph-mon[32750]: pgmap v838: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 576 KiB/s wr, 168 op/s 2024-07-31T15:04:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:57 smithi073 ceph-mon[36714]: pgmap v838: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 576 KiB/s wr, 168 op/s 2024-07-31T15:04:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.338+0000 7f99a0f50640 1 -- 172.21.15.67:0/3270927245 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f999c1024a0 msgr2=0x7f999c1028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.338+0000 7f99a0f50640 1 --2- 172.21.15.67:0/3270927245 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f999c1024a0 0x7f999c1028b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f998c009900 tx=0x7f998c02a410 comp rx=0 tx=0).stop 2024-07-31T15:04:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.338+0000 7f99a0f50640 1 -- 172.21.15.67:0/3270927245 shutdown_connections 2024-07-31T15:04:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.338+0000 7f99a0f50640 1 --2- 172.21.15.67:0/3270927245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999c108730 0x7f999c10ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.338+0000 7f99a0f50640 1 --2- 172.21.15.67:0/3270927245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f999c107de0 0x7f999c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.338+0000 7f99a0f50640 1 --2- 172.21.15.67:0/3270927245 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f999c1024a0 0x7f999c1028b0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.338+0000 7f99a0f50640 1 -- 172.21.15.67:0/3270927245 >> 172.21.15.67:0/3270927245 conn(0x7f999c079160 msgr2=0x7f999c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.338+0000 7f99a0f50640 1 -- 172.21.15.67:0/3270927245 shutdown_connections 2024-07-31T15:04:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.338+0000 7f99a0f50640 1 -- 172.21.15.67:0/3270927245 wait complete. 2024-07-31T15:04:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.339+0000 7f99a0f50640 1 Processor -- start 2024-07-31T15:04:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.339+0000 7f99a0f50640 1 -- start start 2024-07-31T15:04:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.339+0000 7f99a0f50640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f999c1024a0 0x7f999c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.339+0000 7f99a0f50640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f999c107de0 0x7f999c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.339+0000 7f99a0f50640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999c108730 0x7f999c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.339+0000 7f99a0f50640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f999c112390 con 0x7f999c107de0 2024-07-31T15:04:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.339+0000 7f99a0f50640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f999c112210 con 0x7f999c108730 2024-07-31T15:04:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.339+0000 7f99a0f50640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f999c112510 con 0x7f999c1024a0 2024-07-31T15:04:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f9999d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f999c107de0 0x7f999c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f999a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f999c1024a0 0x7f999c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f9999d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f999c107de0 0x7f999c19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45990/0 (socket says 172.21.15.67:45990) 2024-07-31T15:04:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f9999d9b640 1 -- 172.21.15.67:0/3367085620 learned_addr learned my addr 172.21.15.67:0/3367085620 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f999a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f999c1024a0 0x7f999c199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58800/0 (socket says 172.21.15.67:58800) 2024-07-31T15:04:58.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f999ad9d640 1 --2- 172.21.15.67:0/3367085620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999c108730 0x7f999c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f999ad9d640 1 -- 172.21.15.67:0/3367085620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f999c1024a0 msgr2=0x7f999c199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f999ad9d640 1 --2- 172.21.15.67:0/3367085620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f999c1024a0 0x7f999c199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f999ad9d640 1 -- 172.21.15.67:0/3367085620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f999c107de0 msgr2=0x7f999c19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f999ad9d640 1 --2- 172.21.15.67:0/3367085620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f999c107de0 0x7f999c19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f999ad9d640 1 -- 172.21.15.67:0/3367085620 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c19fec0 con 0x7f999c108730 2024-07-31T15:04:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f9999d9b640 1 --2- 172.21.15.67:0/3367085620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f999c107de0 0x7f999c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f999ad9d640 1 --2- 172.21.15.67:0/3367085620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999c108730 0x7f999c19f790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9994009fd0 tx=0x7f999400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f998b7fe640 1 -- 172.21.15.67:0/3367085620 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999400ce20 con 0x7f999c108730 2024-07-31T15:04:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f998b7fe640 1 -- 172.21.15.67:0/3367085620 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9994005020 con 0x7f999c108730 2024-07-31T15:04:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f99a0f50640 1 -- 172.21.15.67:0/3367085620 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c1a01b0 con 0x7f999c108730 2024-07-31T15:04:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.340+0000 7f998b7fe640 1 -- 172.21.15.67:0/3367085620 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994010690 con 0x7f999c108730 2024-07-31T15:04:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.341+0000 7f99a0f50640 1 -- 172.21.15.67:0/3367085620 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c1a6910 con 0x7f999c108730 2024-07-31T15:04:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.343+0000 7f99a74a1740 1 Processor -- start 2024-07-31T15:04:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.343+0000 7f998b7fe640 1 -- 172.21.15.67:0/3367085620 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f99940054d0 con 0x7f999c108730 2024-07-31T15:04:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.343+0000 7f99a74a1740 1 -- start start 2024-07-31T15:04:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f99a74a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2cf5070 0x55abf309a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f99a74a1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2ccc1b0 0x55abf309aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f99a74a1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 0x55abf309afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f99a74a1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55abf3010b90 con 0x55abf2cd4390 2024-07-31T15:04:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f99a74a1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55abf3000090 con 0x55abf2ccc1b0 2024-07-31T15:04:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f99a74a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55abf30040d0 con 0x55abf2cf5070 2024-07-31T15:04:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f999a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2ccc1b0 0x55abf309aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f999a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2ccc1b0 0x55abf309aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45632/0 (socket says 172.21.15.67:45632) 2024-07-31T15:04:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f999a59c640 1 -- 172.21.15.67:0/810448807 learned_addr learned my addr 172.21.15.67:0/810448807 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f9999d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2cf5070 0x55abf309a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f9999d9b640 1 --2- 172.21.15.67:0/810448807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 0x55abf309afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f999a59c640 1 -- 172.21.15.67:0/810448807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2cf5070 msgr2=0x55abf309a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f999a59c640 1 --2- 172.21.15.67:0/810448807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2cf5070 0x55abf309a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f999a59c640 1 -- 172.21.15.67:0/810448807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 msgr2=0x55abf309afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f999a59c640 1 --2- 172.21.15.67:0/810448807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 0x55abf309afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f999a59c640 1 -- 172.21.15.67:0/810448807 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abf2fa6b70 con 0x55abf2ccc1b0 2024-07-31T15:04:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.344+0000 7f9999d9b640 1 --2- 172.21.15.67:0/810448807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 0x55abf309afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:04:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.345+0000 7f998b7fe640 1 --2- 172.21.15.67:0/3367085620 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99840b3d40 0x7f99840b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.345+0000 7f999a59c640 1 --2- 172.21.15.67:0/810448807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2ccc1b0 0x55abf309aa90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f998c009900 tx=0x7f998c032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ced110c3c40647a4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.345+0000 7f9999d9b640 1 --2- 172.21.15.67:0/3367085620 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99840b3d40 0x7f99840b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.345+0000 7f998b7fe640 1 -- 172.21.15.67:0/3367085620 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f99940e49c0 con 0x7f999c108730 2024-07-31T15:04:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.345+0000 7f9988ff9640 1 -- 172.21.15.67:0/810448807 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c040070 con 0x55abf2ccc1b0 2024-07-31T15:04:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.345+0000 7f9988ff9640 1 -- 172.21.15.67:0/810448807 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f998c004030 con 0x55abf2ccc1b0 2024-07-31T15:04:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.345+0000 7f9988ff9640 1 -- 172.21.15.67:0/810448807 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c035040 con 0x55abf2ccc1b0 2024-07-31T15:04:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.346+0000 7f99a74a1740 1 -- 172.21.15.67:0/810448807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2ccc1b0 msgr2=0x55abf309aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.346+0000 7f99a74a1740 1 --2- 172.21.15.67:0/810448807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2ccc1b0 0x55abf309aa90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f998c009900 tx=0x7f998c032b90 comp rx=0 tx=0).stop 2024-07-31T15:04:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.346+0000 7f9999d9b640 1 --2- 172.21.15.67:0/3367085620 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99840b3d40 0x7f99840b6250 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9990009600 tx=0x7f9990013040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.346+0000 7f99a74a1740 1 -- 172.21.15.67:0/810448807 shutdown_connections 2024-07-31T15:04:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.346+0000 7f99a74a1740 1 --2- 172.21.15.67:0/810448807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 0x55abf309afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.346+0000 7f99a74a1740 1 --2- 172.21.15.67:0/810448807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2ccc1b0 0x55abf309aa90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.346+0000 7f99a74a1740 1 --2- 172.21.15.67:0/810448807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2cf5070 0x55abf309a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.346+0000 7f99a74a1740 1 -- 172.21.15.67:0/810448807 >> 172.21.15.67:0/810448807 conn(0x55abf2ce9460 msgr2=0x55abf3098440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.346+0000 7f99a74a1740 1 -- 172.21.15.67:0/810448807 shutdown_connections 2024-07-31T15:04:58.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.346+0000 7f99a74a1740 1 -- 172.21.15.67:0/810448807 wait complete. 2024-07-31T15:04:58.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.347+0000 7f99a74a1740 1 -- start start 2024-07-31T15:04:58.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.347+0000 7f99a74a1740 1 Processor -- start 2024-07-31T15:04:58.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.347+0000 7f99a74a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2ccc1b0 0x55abf30aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.347+0000 7f99a74a1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 0x55abf30aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.347+0000 7f99a74a1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2cf5070 0x55abf30b6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.347+0000 7f99a74a1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55abf2e63700 con 0x55abf2cd4390 2024-07-31T15:04:58.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.347+0000 7f99a74a1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55abf2fa29c0 con 0x55abf2cf5070 2024-07-31T15:04:58.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.347+0000 7f99a74a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55abf30acfd0 con 0x55abf2ccc1b0 2024-07-31T15:04:58.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.347+0000 7f9999d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 0x55abf30aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.347+0000 7f99a74a1740 10 client.0 _ll_register_callbacks cb 0x55abf2fbd4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:58.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.347+0000 7f999ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2cf5070 0x55abf30b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.347+0000 7f9999d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 0x55abf30aa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46010/0 (socket says 172.21.15.67:46010) 2024-07-31T15:04:58.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.347+0000 7f9999d9b640 1 -- 172.21.15.67:0/3059315213 learned_addr learned my addr 172.21.15.67:0/3059315213 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:58.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.347+0000 7f999a59c640 1 --2- 172.21.15.67:0/3059315213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2ccc1b0 0x55abf30aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.348+0000 7f9999d9b640 1 -- 172.21.15.67:0/3059315213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2ccc1b0 msgr2=0x55abf30aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:58.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.348+0000 7f9999d9b640 1 --2- 172.21.15.67:0/3059315213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2ccc1b0 0x55abf30aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.348+0000 7f9999d9b640 1 -- 172.21.15.67:0/3059315213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2cf5070 msgr2=0x55abf30b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:58.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.348+0000 7f9999d9b640 1 --2- 172.21.15.67:0/3059315213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2cf5070 0x55abf30b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.348+0000 7f9999d9b640 1 -- 172.21.15.67:0/3059315213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abf2d60650 con 0x55abf2cd4390 2024-07-31T15:04:58.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.348+0000 7f999ad9d640 1 --2- 172.21.15.67:0/3059315213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2cf5070 0x55abf30b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:04:58.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.348+0000 7f9999d9b640 1 --2- 172.21.15.67:0/3059315213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 0x55abf30aa9f0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9990020440 tx=0x7f9990004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ecf7e3a75a6bde server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:58.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.348+0000 7f9981ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:04:58.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.348+0000 7f9981ffb640 1 -- 172.21.15.67:0/3059315213 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990030060 con 0x55abf2cd4390 2024-07-31T15:04:58.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.348+0000 7f9981ffb640 1 -- 172.21.15.67:0/3059315213 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f999001a070 con 0x55abf2cd4390 2024-07-31T15:04:58.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.348+0000 7f99a74a1740 1 -- 172.21.15.67:0/3059315213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f998c004030 con 0x55abf2cd4390 2024-07-31T15:04:58.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.348+0000 7f9981ffb640 1 -- 172.21.15.67:0/3059315213 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f9990037050 con 0x55abf2cd4390 2024-07-31T15:04:58.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.348+0000 7f99a74a1740 10 client.18474 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:58.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.348+0000 7f99a74a1740 1 -- 172.21.15.67:0/3059315213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55abf30b9010 con 0x55abf2cd4390 2024-07-31T15:04:58.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.349+0000 7f9981ffb640 1 -- 172.21.15.67:0/3059315213 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f998c004030 con 0x55abf2cd4390 2024-07-31T15:04:58.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.349+0000 7f9981ffb640 1 -- 172.21.15.67:0/3059315213 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f999001e470 con 0x55abf2cd4390 2024-07-31T15:04:58.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.349+0000 7f99a74a1740 10 client.18474 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:58.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.349+0000 7f99a74a1740 10 client.18474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:58.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.349+0000 7f99a74a1740 20 client.18474 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:58.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.349+0000 7f99a74a1740 1 --2- 172.21.15.67:0/3059315213 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55abf2cb39f0 0x55abf30bc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.349+0000 7f99a74a1740 4 client.18474 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:04:58.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.349+0000 7f99a74a1740 1 -- 172.21.15.67:0/3059315213 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abf3012ae0 con 0x55abf2cb39f0 2024-07-31T15:04:58.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.349+0000 7f999a59c640 1 --2- 172.21.15.67:0/3059315213 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55abf2cb39f0 0x55abf30bc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.350+0000 7f9981ffb640 10 client.18474 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:58.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.350+0000 7f999a59c640 1 --2- 172.21.15.67:0/3059315213 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55abf2cb39f0 0x55abf30bc5b0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37c6bce2521b3d7c server_cookie=ca395cbdcf0d2732 in_seq=0 out_seq=0 2024-07-31T15:04:58.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.474+0000 7f9981ffb640 1 -- 172.21.15.67:0/3059315213 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abf3012ae0 con 0x55abf2cb39f0 2024-07-31T15:04:58.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.474+0000 7f9981ffb640 10 client.18474 handle_command_reply: tid=0 2024-07-31T15:04:58.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.474+0000 7f99a74a1740 1 client.18474 shutdown 2024-07-31T15:04:58.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.474+0000 7f99a74a1740 1 -- 172.21.15.67:0/3059315213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 msgr2=0x55abf30aa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:58.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.474+0000 7f99a74a1740 1 --2- 172.21.15.67:0/3059315213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 0x55abf30aa9f0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9990020440 tx=0x7f9990004730 comp rx=0 tx=0).stop 2024-07-31T15:04:58.476 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:58.476 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.477 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:58.477 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.477 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:58.477 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.477 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:58.477 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.477 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:58.478 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.478 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:58.478 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.478 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:58.478 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.478 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:58.478 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.478 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:58.478 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.475+0000 7f99a74a1740 1 -- 172.21.15.67:0/3059315213 shutdown_connections 2024-07-31T15:04:58.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.475+0000 7f99a74a1740 1 --2- 172.21.15.67:0/3059315213 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55abf2cb39f0 0x55abf30bc5b0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.475+0000 7f99a74a1740 1 --2- 172.21.15.67:0/3059315213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2cf5070 0x55abf30b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.475+0000 7f99a74a1740 1 --2- 172.21.15.67:0/3059315213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 0x55abf30aa9f0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.475+0000 7f9981ffb640 0 client.18474 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:58.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.475+0000 7f99a74a1740 1 --2- 172.21.15.67:0/3059315213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2ccc1b0 0x55abf30aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.475+0000 7f99a74a1740 1 -- 172.21.15.67:0/3059315213 >> 172.21.15.67:0/3059315213 conn(0x55abf2ce9460 msgr2=0x55abf3096c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:58.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.475+0000 7f99a74a1740 1 -- 172.21.15.67:0/3059315213 shutdown_connections 2024-07-31T15:04:58.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.475+0000 7f99a74a1740 1 -- 172.21.15.67:0/3059315213 wait complete. 2024-07-31T15:04:58.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.475+0000 7f99a74a1740 20 client.18474 trim_cache size 0 max 16384 2024-07-31T15:04:58.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:58.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:58.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:58.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.488+0000 7f71f6ffd640 1 -- 172.21.15.67:0/2331942475 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55558d4ac860 con 0x55558d127250 2024-07-31T15:04:58.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.488+0000 7f71f6ffd640 10 client.18462 handle_command_reply: tid=0 2024-07-31T15:04:58.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.488+0000 7f72240d5740 1 client.18462 shutdown 2024-07-31T15:04:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.488+0000 7f72240d5740 1 -- 172.21.15.67:0/2331942475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d172840 msgr2=0x55558d560200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.488+0000 7f72240d5740 1 --2- 172.21.15.67:0/2331942475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d172840 0x55558d560200 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f7218111a90 tx=0x7f720c013a70 comp rx=0 tx=0).stop 2024-07-31T15:04:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.489+0000 7f72240d5740 1 -- 172.21.15.67:0/2331942475 shutdown_connections 2024-07-31T15:04:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.489+0000 7f72240d5740 1 --2- 172.21.15.67:0/2331942475 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55558d127250 0x55558d5c9b90 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.489+0000 7f71f6ffd640 0 client.18462 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.489+0000 7f72240d5740 1 --2- 172.21.15.67:0/2331942475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55558d193520 0x55558d5be780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.489+0000 7f72240d5740 1 --2- 172.21.15.67:0/2331942475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55558d172840 0x55558d560200 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.489+0000 7f72240d5740 1 --2- 172.21.15.67:0/2331942475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55558d16a660 0x55558d55fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.489+0000 7f72240d5740 1 -- 172.21.15.67:0/2331942475 >> 172.21.15.67:0/2331942475 conn(0x55558d187910 msgr2=0x55558d54bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:58.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.489+0000 7f72240d5740 1 -- 172.21.15.67:0/2331942475 shutdown_connections 2024-07-31T15:04:58.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.489+0000 7f72240d5740 1 -- 172.21.15.67:0/2331942475 wait complete. 2024-07-31T15:04:58.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.489+0000 7f72240d5740 20 client.18462 trim_cache size 0 max 16384 2024-07-31T15:04:58.492 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:04:58.492 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 63, 2024-07-31T15:04:58.492 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:04:58.492 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 36, 2024-07-31T15:04:58.492 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:04:58.492 INFO:teuthology.orchestra.run.smithi067.stdout: "4d8b91f9": { 2024-07-31T15:04:58.493 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 36, 2024-07-31T15:04:58.493 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:04:58.493 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:04:58.493 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:04:58.493 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:04:58.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:04:58.493 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:04:58.493 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 144.3, 2024-07-31T15:04:58.493 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:04:58.493 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:04:58.493 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:04:58.493 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:04:58.493 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:04:58.494 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:04:58.494 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:04:58.494 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:04:58.494 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:04:58.494 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:04:58.494 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:04:58.494 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:04:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.490+0000 7f721db84640 1 -- 172.21.15.67:0/3898701846 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f72000b3dc0 msgr2=0x7f72000b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.490+0000 7f721db84640 1 --2- 172.21.15.67:0/3898701846 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f72000b3dc0 0x7f72000b62d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7210006010 tx=0x7f7210009860 comp rx=0 tx=0).stop 2024-07-31T15:04:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.490+0000 7f721db84640 1 -- 172.21.15.67:0/3898701846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7218102090 msgr2=0x7f721819c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.490+0000 7f721db84640 1 --2- 172.21.15.67:0/3898701846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7218102090 0x7f721819c020 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f720800e9f0 tx=0x7f720800edf0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.490+0000 7f721db84640 1 -- 172.21.15.67:0/3898701846 shutdown_connections 2024-07-31T15:04:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.490+0000 7f721db84640 1 --2- 172.21.15.67:0/3898701846 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f72000b3dc0 0x7f72000b62d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.490+0000 7f721db84640 1 --2- 172.21.15.67:0/3898701846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f721810fc40 0x7f72181a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.490+0000 7f721db84640 1 --2- 172.21.15.67:0/3898701846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72181029e0 0x7f721819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.490+0000 7f721db84640 1 --2- 172.21.15.67:0/3898701846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7218102090 0x7f721819c020 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.490+0000 7f721db84640 1 -- 172.21.15.67:0/3898701846 >> 172.21.15.67:0/3898701846 conn(0x7f7218079160 msgr2=0x7f7218109770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.490+0000 7f721db84640 1 -- 172.21.15.67:0/3898701846 shutdown_connections 2024-07-31T15:04:58.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.490+0000 7f721db84640 1 -- 172.21.15.67:0/3898701846 wait complete. 2024-07-31T15:04:58.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.497+0000 7f99a74a1740 1 Processor -- start 2024-07-31T15:04:58.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.497+0000 7f99a74a1740 1 -- start start 2024-07-31T15:04:58.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.498+0000 7f99a74a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2ccc1b0 0x55abf3097330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.498+0000 7f99a74a1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 0x55abf3097d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.498+0000 7f99a74a1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2cf5070 0x55abf30982a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.498+0000 7f99a74a1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1ae620 con 0x55abf2cd4390 2024-07-31T15:04:58.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.498+0000 7f99a74a1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1b0160 con 0x55abf2cf5070 2024-07-31T15:04:58.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.498+0000 7f99a74a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1aecf0 con 0x55abf2ccc1b0 2024-07-31T15:04:58.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.498+0000 7f999a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2ccc1b0 0x55abf3097330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.498+0000 7f999a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2ccc1b0 0x55abf3097330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58826/0 (socket says 172.21.15.67:58826) 2024-07-31T15:04:58.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.498+0000 7f999a59c640 1 -- 172.21.15.67:0/1939979101 learned_addr learned my addr 172.21.15.67:0/1939979101 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:58.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.498+0000 7f999a59c640 1 -- 172.21.15.67:0/1939979101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2cf5070 msgr2=0x55abf30982a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:04:58.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.498+0000 7f999a59c640 1 --2- 172.21.15.67:0/1939979101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2cf5070 0x55abf30982a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.498+0000 7f999a59c640 1 -- 172.21.15.67:0/1939979101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 msgr2=0x55abf3097d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:58.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.498+0000 7f999a59c640 1 --2- 172.21.15.67:0/1939979101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 0x55abf3097d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.498+0000 7f999a59c640 1 -- 172.21.15.67:0/1939979101 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abf309b340 con 0x55abf2ccc1b0 2024-07-31T15:04:58.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.499+0000 7f999a59c640 1 --2- 172.21.15.67:0/1939979101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2ccc1b0 0x55abf3097330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f998c005150 tx=0x7f998c005180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ba8c2b0fa6a90d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:58.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.499+0000 7f9983fff640 1 -- 172.21.15.67:0/1939979101 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c035040 con 0x55abf2ccc1b0 2024-07-31T15:04:58.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.499+0000 7f9983fff640 1 -- 172.21.15.67:0/1939979101 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f998c004460 con 0x55abf2ccc1b0 2024-07-31T15:04:58.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.499+0000 7f9983fff640 1 -- 172.21.15.67:0/1939979101 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c040040 con 0x55abf2ccc1b0 2024-07-31T15:04:58.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.500+0000 7f99a74a1740 1 -- 172.21.15.67:0/1939979101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2ccc1b0 msgr2=0x55abf3097330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:58.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.500+0000 7f99a74a1740 1 --2- 172.21.15.67:0/1939979101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2ccc1b0 0x55abf3097330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f998c005150 tx=0x7f998c005180 comp rx=0 tx=0).stop 2024-07-31T15:04:58.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.500+0000 7f99a74a1740 1 -- 172.21.15.67:0/1939979101 shutdown_connections 2024-07-31T15:04:58.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.500+0000 7f99a74a1740 1 --2- 172.21.15.67:0/1939979101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2cf5070 0x55abf30982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.500+0000 7f99a74a1740 1 --2- 172.21.15.67:0/1939979101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cd4390 0x55abf3097d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.500+0000 7f99a74a1740 1 --2- 172.21.15.67:0/1939979101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2ccc1b0 0x55abf3097330 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.500+0000 7f99a74a1740 1 -- 172.21.15.67:0/1939979101 >> 172.21.15.67:0/1939979101 conn(0x55abf2ce9460 msgr2=0x55abf3096c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:58.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.501+0000 7f99a74a1740 1 -- 172.21.15.67:0/1939979101 shutdown_connections 2024-07-31T15:04:58.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.501+0000 7f99a74a1740 1 -- 172.21.15.67:0/1939979101 wait complete. 2024-07-31T15:04:58.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.501+0000 7f99a74a1740 1 -- start start 2024-07-31T15:04:58.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.501+0000 7f99a74a1740 1 Processor -- start 2024-07-31T15:04:58.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f99a74a1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2ccc1b0 0x55abf30a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f99a74a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2cd4390 0x55abf30a0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f99a74a1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cf5070 0x55abf30a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f99a74a1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1afa90 con 0x55abf2cf5070 2024-07-31T15:04:58.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f99a74a1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1af3c0 con 0x55abf2ccc1b0 2024-07-31T15:04:58.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f9999d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2cd4390 0x55abf30a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f99a74a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1ac620 con 0x55abf2cd4390 2024-07-31T15:04:58.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f9999d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2cd4390 0x55abf30a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58834/0 (socket says 172.21.15.67:58834) 2024-07-31T15:04:58.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f9999d9b640 1 -- 172.21.15.67:0/2901416553 learned_addr learned my addr 172.21.15.67:0/2901416553 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:04:58.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f99a74a1740 10 client.0 _ll_register_callbacks cb 0x7f99900115d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:04:58.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f999a59c640 1 --2- 172.21.15.67:0/2901416553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2ccc1b0 0x55abf30a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f999ad9d640 1 --2- 172.21.15.67:0/2901416553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cf5070 0x55abf30a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f999a59c640 1 -- 172.21.15.67:0/2901416553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2cd4390 msgr2=0x55abf30a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f999a59c640 1 --2- 172.21.15.67:0/2901416553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2cd4390 0x55abf30a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f999a59c640 1 -- 172.21.15.67:0/2901416553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cf5070 msgr2=0x55abf30a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:04:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.502+0000 7f999a59c640 1 --2- 172.21.15.67:0/2901416553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cf5070 0x55abf30a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.503+0000 7f999a59c640 1 -- 172.21.15.67:0/2901416553 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abf30a1640 con 0x55abf2ccc1b0 2024-07-31T15:04:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.503+0000 7f999a59c640 1 --2- 172.21.15.67:0/2901416553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2ccc1b0 0x55abf30a44d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f998c033000 tx=0x7f998c004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54ec36d5ec58b991 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:04:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.503+0000 7f9981ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:04:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.503+0000 7f9981ffb640 1 -- 172.21.15.67:0/2901416553 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c040070 con 0x55abf2ccc1b0 2024-07-31T15:04:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.503+0000 7f9981ffb640 1 -- 172.21.15.67:0/2901416553 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f998c031d10 con 0x55abf2ccc1b0 2024-07-31T15:04:58.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.504+0000 7f99a74a1740 1 -- 172.21.15.67:0/2901416553 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f998c004460 con 0x55abf2ccc1b0 2024-07-31T15:04:58.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.504+0000 7f9981ffb640 1 -- 172.21.15.67:0/2901416553 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f998c06a050 con 0x55abf2ccc1b0 2024-07-31T15:04:58.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.504+0000 7f99a74a1740 10 client.28034 fetch_fsmap learned FSMap version 63 2024-07-31T15:04:58.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.504+0000 7f99a74a1740 1 -- 172.21.15.67:0/2901416553 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55abf30b2a40 con 0x55abf2ccc1b0 2024-07-31T15:04:58.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.504+0000 7f9981ffb640 1 -- 172.21.15.67:0/2901416553 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f998c004460 con 0x55abf2ccc1b0 2024-07-31T15:04:58.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.504+0000 7f9981ffb640 1 -- 172.21.15.67:0/2901416553 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f998c057070 con 0x55abf2ccc1b0 2024-07-31T15:04:58.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.505+0000 7f99a74a1740 10 client.28034 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:04:58.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.505+0000 7f99a74a1740 10 client.28034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:04:58.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.505+0000 7f99a74a1740 20 client.28034 populate_metadata read hostname 'smithi067' 2024-07-31T15:04:58.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.505+0000 7f99a74a1740 1 --2- 172.21.15.67:0/2901416553 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55abf2c88da0 0x55abf312b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:04:58.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.505+0000 7f99a74a1740 4 client.28034 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:04:58.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.505+0000 7f99a74a1740 1 -- 172.21.15.67:0/2901416553 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55abf300e580 con 0x55abf2c88da0 2024-07-31T15:04:58.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.505+0000 7f9999d9b640 1 --2- 172.21.15.67:0/2901416553 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55abf2c88da0 0x55abf312b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:04:58.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.505+0000 7f9981ffb640 10 client.28034 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:58.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.506+0000 7f9999d9b640 1 --2- 172.21.15.67:0/2901416553 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55abf2c88da0 0x55abf312b580 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=545026d2630a4d6e server_cookie=a787a02054276248 in_seq=0 out_seq=0 2024-07-31T15:04:58.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.506+0000 7f9981ffb640 1 -- 172.21.15.67:0/2901416553 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x55abf300e580 con 0x55abf2c88da0 2024-07-31T15:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.506+0000 7f9981ffb640 10 client.28034 handle_command_reply: tid=0 2024-07-31T15:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.506+0000 7f99a74a1740 1 client.28034 shutdown 2024-07-31T15:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.506+0000 7f99a74a1740 1 -- 172.21.15.67:0/2901416553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2ccc1b0 msgr2=0x55abf30a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.507+0000 7f99a74a1740 1 --2- 172.21.15.67:0/2901416553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2ccc1b0 0x55abf30a44d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f998c033000 tx=0x7f998c004610 comp rx=0 tx=0).stop 2024-07-31T15:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.507+0000 7f99a74a1740 1 -- 172.21.15.67:0/2901416553 shutdown_connections 2024-07-31T15:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.507+0000 7f99a74a1740 1 --2- 172.21.15.67:0/2901416553 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55abf2c88da0 0x55abf312b580 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.507+0000 7f99a74a1740 1 --2- 172.21.15.67:0/2901416553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55abf2cf5070 0x55abf30a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.507+0000 7f9981ffb640 0 client.28034 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.507+0000 7f99a74a1740 1 --2- 172.21.15.67:0/2901416553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55abf2cd4390 0x55abf30a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.507+0000 7f99a74a1740 1 --2- 172.21.15.67:0/2901416553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55abf2ccc1b0 0x55abf30a44d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.507+0000 7f99a74a1740 1 -- 172.21.15.67:0/2901416553 >> 172.21.15.67:0/2901416553 conn(0x55abf2ce9460 msgr2=0x55abf3097d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.507+0000 7f99a74a1740 1 -- 172.21.15.67:0/2901416553 shutdown_connections 2024-07-31T15:04:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.507+0000 7f99a74a1740 1 -- 172.21.15.67:0/2901416553 wait complete. 2024-07-31T15:04:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.507+0000 7f99a74a1740 20 client.28034 trim_cache size 0 max 16384 2024-07-31T15:04:58.512 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:04:58.512 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (15676 inodes in the stack)", 2024-07-31T15:04:58.512 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T15:04:58.512 INFO:teuthology.orchestra.run.smithi067.stdout: "664fb307-5b5c-42e3-967e-742a749d1599": { 2024-07-31T15:04:58.512 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T15:04:58.512 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "664fb307-5b5c-42e3-967e-742a749d1599", 2024-07-31T15:04:58.513 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T15:04:58.513 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:04:58.513 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:04:58.513 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:04:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.508+0000 7f99a0f50640 1 -- 172.21.15.67:0/3367085620 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99840b3d40 msgr2=0x7f99840b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.508+0000 7f99a0f50640 1 --2- 172.21.15.67:0/3367085620 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99840b3d40 0x7f99840b6250 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9990009600 tx=0x7f9990013040 comp rx=0 tx=0).stop 2024-07-31T15:04:58.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.508+0000 7f99a0f50640 1 -- 172.21.15.67:0/3367085620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999c108730 msgr2=0x7f999c19f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:04:58.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.508+0000 7f99a0f50640 1 --2- 172.21.15.67:0/3367085620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999c108730 0x7f999c19f790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9994009fd0 tx=0x7f999400ef10 comp rx=0 tx=0).stop 2024-07-31T15:04:58.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.508+0000 7f99a0f50640 1 -- 172.21.15.67:0/3367085620 shutdown_connections 2024-07-31T15:04:58.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.508+0000 7f99a0f50640 1 --2- 172.21.15.67:0/3367085620 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f99840b3d40 0x7f99840b6250 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.508+0000 7f99a0f50640 1 --2- 172.21.15.67:0/3367085620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999c108730 0x7f999c19f790 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.508+0000 7f99a0f50640 1 --2- 172.21.15.67:0/3367085620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f999c107de0 0x7f999c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.508+0000 7f99a0f50640 1 --2- 172.21.15.67:0/3367085620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f999c1024a0 0x7f999c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:04:58.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.508+0000 7f99a0f50640 1 -- 172.21.15.67:0/3367085620 >> 172.21.15.67:0/3367085620 conn(0x7f999c079160 msgr2=0x7f999c1061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:04:58.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.509+0000 7f99a0f50640 1 -- 172.21.15.67:0/3367085620 shutdown_connections 2024-07-31T15:04:58.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:04:58.509+0000 7f99a0f50640 1 -- 172.21.15.67:0/3367085620 wait complete. 2024-07-31T15:04:58.601 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/espfix_64.o 2024-07-31T15:04:58.602 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_stat.o 2024-07-31T15:04:58.603 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/aes_generic.o 2024-07-31T15:04:58.604 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/secretmem.o 2024-07-31T15:04:58.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:58.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:58.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:58.644 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-ioprio.o 2024-07-31T15:04:58.647 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/xattr_hurd.o 2024-07-31T15:04:58.656 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/misc.o 2024-07-31T15:04:58.663 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/epoll.o 2024-07-31T15:04:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: pgmap v839: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 572 KiB/s wr, 535 op/s 2024-07-31T15:04:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:59.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:04:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:04:59.341 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/xattr_trusted.o 2024-07-31T15:04:59.359 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/debug.o 2024-07-31T15:04:59.363 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-iolatency.o 2024-07-31T15:04:59.455 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/crc32c_generic.o 2024-07-31T15:04:59.468 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/ksysfs.o 2024-07-31T15:04:59.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: pgmap v839: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 572 KiB/s wr, 535 op/s 2024-07-31T15:04:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:04:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:04:59.491 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/statx.o 2024-07-31T15:04:59.492 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_printk.o 2024-07-31T15:04:59.567 INFO:tasks.cephfs.filesystem:scrub status for tag:664fb307-5b5c-42e3-967e-742a749d1599 - {'path': '/', 'tag': '664fb307-5b5c-42e3-967e-742a749d1599', 'options': 'recursive,force'} 2024-07-31T15:04:59.623 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/memfd.o 2024-07-31T15:04:59.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: pgmap v839: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 572 KiB/s wr, 535 op/s 2024-07-31T15:04:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:59.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:04:59.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:04:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:04:59.904 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/authenc.o 2024-07-31T15:05:00.063 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4d8b91f9', total seconds elapsed: 39.3 2024-07-31T15:05:00.123 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/cgroup/built-in.a 2024-07-31T15:05:00.136 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/xattr_user.o 2024-07-31T15:05:00.189 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/pid_list.o 2024-07-31T15:05:00.254 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/bootflag.o 2024-07-31T15:05:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:00.398 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/net.o 2024-07-31T15:05:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.426+0000 7f59c92f5640 1 -- 172.21.15.67:0/507376725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c410a330 msgr2=0x7f59c410a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.426+0000 7f59c92f5640 1 --2- 172.21.15.67:0/507376725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c410a330 0x7f59c410a740 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f59bc01acb0 tx=0x7f59bc03b9b0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.426+0000 7f59c92f5640 1 -- 172.21.15.67:0/507376725 shutdown_connections 2024-07-31T15:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.426+0000 7f59c92f5640 1 --2- 172.21.15.67:0/507376725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59c411c5d0 0x7f59c411eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.426+0000 7f59c92f5640 1 --2- 172.21.15.67:0/507376725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c410a330 0x7f59c410a740 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.426+0000 7f59c92f5640 1 --2- 172.21.15.67:0/507376725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c41118d0 0x7f59c4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.426+0000 7f59c92f5640 1 -- 172.21.15.67:0/507376725 >> 172.21.15.67:0/507376725 conn(0x7f59c40707c0 msgr2=0x7f59c4070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.427+0000 7f59c92f5640 1 -- 172.21.15.67:0/507376725 shutdown_connections 2024-07-31T15:05:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.427+0000 7f59c92f5640 1 -- 172.21.15.67:0/507376725 wait complete. 2024-07-31T15:05:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.428+0000 7f59c92f5640 1 Processor -- start 2024-07-31T15:05:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.428+0000 7f59c92f5640 1 -- start start 2024-07-31T15:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.429+0000 7f59c92f5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59c410a330 0x7f59c411beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.429+0000 7f59c92f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c41118d0 0x7f59c4118180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.429+0000 7f59c92f5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c411c5d0 0x7f59c4118750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.429+0000 7f59c92f5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41146a0 con 0x7f59c411c5d0 2024-07-31T15:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.429+0000 7f59c92f5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f59c4114520 con 0x7f59c410a330 2024-07-31T15:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.429+0000 7f59c92f5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59c4114820 con 0x7f59c41118d0 2024-07-31T15:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.429+0000 7f59c37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c411c5d0 0x7f59c4118750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.429+0000 7f59c37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c411c5d0 0x7f59c4118750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46036/0 (socket says 172.21.15.67:46036) 2024-07-31T15:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.429+0000 7f59c37fe640 1 -- 172.21.15.67:0/2554241236 learned_addr learned my addr 172.21.15.67:0/2554241236 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.429+0000 7f59c27fc640 1 --2- 172.21.15.67:0/2554241236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c41118d0 0x7f59c4118180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.429+0000 7f59c2ffd640 1 --2- 172.21.15.67:0/2554241236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59c410a330 0x7f59c411beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.430+0000 7f59c27fc640 1 -- 172.21.15.67:0/2554241236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59c410a330 msgr2=0x7f59c411beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.430+0000 7f59c27fc640 1 --2- 172.21.15.67:0/2554241236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59c410a330 0x7f59c411beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.430+0000 7f59c27fc640 1 -- 172.21.15.67:0/2554241236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c411c5d0 msgr2=0x7f59c4118750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.430+0000 7f59c27fc640 1 --2- 172.21.15.67:0/2554241236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c411c5d0 0x7f59c4118750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.430+0000 7f59c27fc640 1 -- 172.21.15.67:0/2554241236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c4118f70 con 0x7f59c41118d0 2024-07-31T15:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.431+0000 7f59c27fc640 1 --2- 172.21.15.67:0/2554241236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c41118d0 0x7f59c4118180 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f59bc03bed0 tx=0x7f59bc040930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.433+0000 7f59b3fff640 1 -- 172.21.15.67:0/2554241236 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc056070 con 0x7f59c41118d0 2024-07-31T15:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.433+0000 7f59b3fff640 1 -- 172.21.15.67:0/2554241236 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59bc047c90 con 0x7f59c41118d0 2024-07-31T15:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.433+0000 7f59c92f5640 1 -- 172.21.15.67:0/2554241236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c41ac810 con 0x7f59c41118d0 2024-07-31T15:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.433+0000 7f59c92f5640 1 -- 172.21.15.67:0/2554241236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c41acd50 con 0x7f59c41118d0 2024-07-31T15:05:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.434+0000 7f59b3fff640 1 -- 172.21.15.67:0/2554241236 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc044040 con 0x7f59c41118d0 2024-07-31T15:05:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.438+0000 7f59d01df740 1 Processor -- start 2024-07-31T15:05:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.439+0000 7f59d01df740 1 -- start start 2024-07-31T15:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.439+0000 7f59d01df740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521c181b0 0x561521fbace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.439+0000 7f59d01df740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521bef2f0 0x561521fbb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.439+0000 7f59d01df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bf74d0 0x561521fbb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.439+0000 7f59d01df740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561521f303b0 con 0x561521bef2f0 2024-07-31T15:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.439+0000 7f59d01df740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561521f1f8b0 con 0x561521c181b0 2024-07-31T15:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.439+0000 7f59d01df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561521f238f0 con 0x561521bf74d0 2024-07-31T15:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.440+0000 7f59b3fff640 1 -- 172.21.15.67:0/2554241236 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f59bc059070 con 0x7f59c41118d0 2024-07-31T15:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.441+0000 7f59c37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521c181b0 0x561521fbace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.441+0000 7f59c37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521c181b0 0x561521fbace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45686/0 (socket says 172.21.15.67:45686) 2024-07-31T15:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.441+0000 7f59c37fe640 1 -- 172.21.15.67:0/2999131529 learned_addr learned my addr 172.21.15.67:0/2999131529 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.442+0000 7f59c37fe640 1 -- 172.21.15.67:0/2999131529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bf74d0 msgr2=0x561521fbb760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.442+0000 7f59c37fe640 1 --2- 172.21.15.67:0/2999131529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bf74d0 0x561521fbb760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.442+0000 7f59c37fe640 1 -- 172.21.15.67:0/2999131529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521bef2f0 msgr2=0x561521fbb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.442+0000 7f59c37fe640 1 --2- 172.21.15.67:0/2999131529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521bef2f0 0x561521fbb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.442+0000 7f59c37fe640 1 -- 172.21.15.67:0/2999131529 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561521ec69a0 con 0x561521c181b0 2024-07-31T15:05:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.442+0000 7f59b3fff640 1 --2- 172.21.15.67:0/2554241236 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f59ac0b3f10 0x7f59ac0b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.443+0000 7f59b3fff640 1 -- 172.21.15.67:0/2554241236 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f59bc115630 con 0x7f59c41118d0 2024-07-31T15:05:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.443+0000 7f59c2ffd640 1 --2- 172.21.15.67:0/2554241236 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f59ac0b3f10 0x7f59ac0b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.443+0000 7f59c37fe640 1 --2- 172.21.15.67:0/2999131529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521c181b0 0x561521fbace0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009dc0 tx=0x7f59b4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c91d0ff40193d750 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.444+0000 7f59b17fa640 1 -- 172.21.15.67:0/2999131529 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b401e070 con 0x561521c181b0 2024-07-31T15:05:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.444+0000 7f59b17fa640 1 -- 172.21.15.67:0/2999131529 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b4019ae0 con 0x561521c181b0 2024-07-31T15:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.445+0000 7f59b17fa640 1 -- 172.21.15.67:0/2999131529 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b400fac0 con 0x561521c181b0 2024-07-31T15:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.446+0000 7f59d01df740 1 -- 172.21.15.67:0/2999131529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521c181b0 msgr2=0x561521fbace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.446+0000 7f59d01df740 1 --2- 172.21.15.67:0/2999131529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521c181b0 0x561521fbace0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009dc0 tx=0x7f59b4011810 comp rx=0 tx=0).stop 2024-07-31T15:05:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.446+0000 7f59d01df740 1 -- 172.21.15.67:0/2999131529 shutdown_connections 2024-07-31T15:05:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.446+0000 7f59d01df740 1 --2- 172.21.15.67:0/2999131529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bf74d0 0x561521fbb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.446+0000 7f59d01df740 1 --2- 172.21.15.67:0/2999131529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521bef2f0 0x561521fbb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.446+0000 7f59d01df740 1 --2- 172.21.15.67:0/2999131529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521c181b0 0x561521fbace0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.446+0000 7f59d01df740 1 -- 172.21.15.67:0/2999131529 >> 172.21.15.67:0/2999131529 conn(0x561521c0c300 msgr2=0x561521fb8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.446+0000 7f59d01df740 1 -- 172.21.15.67:0/2999131529 shutdown_connections 2024-07-31T15:05:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.446+0000 7f59d01df740 1 -- 172.21.15.67:0/2999131529 wait complete. 2024-07-31T15:05:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.447+0000 7f59d01df740 1 -- start start 2024-07-31T15:05:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.447+0000 7f59c2ffd640 1 --2- 172.21.15.67:0/2554241236 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f59ac0b3f10 0x7f59ac0b6420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x561521fb9960 tx=0x7f59b8008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.448+0000 7f59d01df740 1 Processor -- start 2024-07-31T15:05:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.448+0000 7f59d01df740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521bef2f0 0x561521fc6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.449+0000 7f59d01df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bf74d0 0x561521fc6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.449+0000 7f59d01df740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521c181b0 0x561521fd28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.449+0000 7f59c27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bf74d0 0x561521fc6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.449+0000 7f59c27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bf74d0 0x561521fc6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58860/0 (socket says 172.21.15.67:58860) 2024-07-31T15:05:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.449+0000 7f59c27fc640 1 -- 172.21.15.67:0/3740263058 learned_addr learned my addr 172.21.15.67:0/3740263058 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.449+0000 7f59c37fe640 1 --2- 172.21.15.67:0/3740263058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521c181b0 0x561521fd28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.449+0000 7f59d01df740 1 -- 172.21.15.67:0/3740263058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561521e59ab0 con 0x561521bef2f0 2024-07-31T15:05:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.449+0000 7f59d01df740 1 -- 172.21.15.67:0/3740263058 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561521ea7cf0 con 0x561521c181b0 2024-07-31T15:05:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.450+0000 7f59d01df740 1 -- 172.21.15.67:0/3740263058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561521fc9500 con 0x561521bf74d0 2024-07-31T15:05:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.450+0000 7f59d01df740 10 client.0 _ll_register_callbacks cb 0x561521f199f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.450+0000 7f59c27fc640 1 -- 172.21.15.67:0/3740263058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521c181b0 msgr2=0x561521fd28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.450+0000 7f59c27fc640 1 --2- 172.21.15.67:0/3740263058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521c181b0 0x561521fd28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.450+0000 7f59c27fc640 1 -- 172.21.15.67:0/3740263058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521bef2f0 msgr2=0x561521fc6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.450+0000 7f59c27fc640 1 --2- 172.21.15.67:0/3740263058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521bef2f0 0x561521fc6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.450+0000 7f59c27fc640 1 -- 172.21.15.67:0/3740263058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561521c83790 con 0x561521bf74d0 2024-07-31T15:05:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.450+0000 7f59c27fc640 1 --2- 172.21.15.67:0/3740263058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bf74d0 0x561521fc6b10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0dbe00 tx=0x7f59bc0dbd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3df7436b7123089 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.451+0000 7f59d01df740 1 -- 172.21.15.67:0/3740263058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b4019ae0 con 0x561521bf74d0 2024-07-31T15:05:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.455+0000 7f59aa7fc640 10 client.27976 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:05:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.455+0000 7f59aa7fc640 1 -- 172.21.15.67:0/3740263058 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0f6060 con 0x561521bf74d0 2024-07-31T15:05:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.455+0000 7f59aa7fc640 1 -- 172.21.15.67:0/3740263058 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f59bc0d8460 con 0x561521bf74d0 2024-07-31T15:05:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.456+0000 7f59aa7fc640 1 -- 172.21.15.67:0/3740263058 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f59bc0fd050 con 0x561521bf74d0 2024-07-31T15:05:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.456+0000 7f59d01df740 10 client.27976 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.456+0000 7f59d01df740 1 -- 172.21.15.67:0/3740263058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=63}) -- 0x561521cfc620 con 0x561521bf74d0 2024-07-31T15:05:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.456+0000 7f59aa7fc640 1 -- 172.21.15.67:0/3740263058 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f59bc115200 con 0x561521bf74d0 2024-07-31T15:05:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.456+0000 7f59d01df740 10 client.27976 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.456+0000 7f59d01df740 10 client.27976 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:05:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.456+0000 7f59d01df740 20 client.27976 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.457+0000 7f59d01df740 1 --2- 172.21.15.67:0/3740263058 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561521bd6d80 0x561521fd8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.457+0000 7f59d01df740 4 client.27976 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:05:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.457+0000 7f59d01df740 1 -- 172.21.15.67:0/3740263058 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f59b4019ae0 con 0x561521bd6d80 2024-07-31T15:05:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.457+0000 7f59c2ffd640 1 --2- 172.21.15.67:0/3740263058 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561521bd6d80 0x561521fd8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.459+0000 7f59aa7fc640 10 client.27976 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.459+0000 7f59c2ffd640 1 --2- 172.21.15.67:0/3740263058 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561521bd6d80 0x561521fd8700 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=719e7bab711a3edd server_cookie=99e10cbc24948d60 in_seq=0 out_seq=0 2024-07-31T15:05:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.463+0000 7f59aa7fc640 1 -- 172.21.15.67:0/3740263058 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f59b4019ae0 con 0x561521bd6d80 2024-07-31T15:05:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.463+0000 7f59aa7fc640 10 client.27976 handle_command_reply: tid=0 2024-07-31T15:05:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.463+0000 7f59d01df740 1 client.27976 shutdown 2024-07-31T15:05:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.463+0000 7f59d01df740 1 -- 172.21.15.67:0/3740263058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bf74d0 msgr2=0x561521fc6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.463+0000 7f59d01df740 1 --2- 172.21.15.67:0/3740263058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bf74d0 0x561521fc6b10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0dbe00 tx=0x7f59bc0dbd00 comp rx=0 tx=0).stop 2024-07-31T15:05:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.463+0000 7f59d01df740 1 -- 172.21.15.67:0/3740263058 shutdown_connections 2024-07-31T15:05:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.463+0000 7f59d01df740 1 --2- 172.21.15.67:0/3740263058 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561521bd6d80 0x561521fd8700 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.463+0000 7f59d01df740 1 --2- 172.21.15.67:0/3740263058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521c181b0 0x561521fd28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.463+0000 7f59d01df740 1 --2- 172.21.15.67:0/3740263058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bf74d0 0x561521fc6b10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.463+0000 7f59d01df740 1 --2- 172.21.15.67:0/3740263058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521bef2f0 0x561521fc6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.463+0000 7f59d01df740 1 -- 172.21.15.67:0/3740263058 >> 172.21.15.67:0/3740263058 conn(0x561521c0c300 msgr2=0x561521fbbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.463+0000 7f59aa7fc640 0 client.27976 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.463+0000 7f59d01df740 1 -- 172.21.15.67:0/3740263058 shutdown_connections 2024-07-31T15:05:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.464+0000 7f59d01df740 1 -- 172.21.15.67:0/3740263058 wait complete. 2024-07-31T15:05:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.464+0000 7f59d01df740 20 client.27976 trim_cache size 0 max 16384 2024-07-31T15:05:00.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.516+0000 7f59d01df740 1 Processor -- start 2024-07-31T15:05:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.516+0000 7f59d01df740 1 -- start start 2024-07-31T15:05:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.517+0000 7f59d01df740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521c0c300 0x561521fc5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.517+0000 7f59d01df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bef2f0 0x561521fc0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.517+0000 7f59d01df740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521bf74d0 0x561521fbc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.517+0000 7f59d01df740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f59a400fef0 con 0x561521c0c300 2024-07-31T15:05:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.517+0000 7f59d01df740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4011a30 con 0x561521bf74d0 2024-07-31T15:05:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.517+0000 7f59d01df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59a40105c0 con 0x561521bef2f0 2024-07-31T15:05:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.517+0000 7f59c27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bef2f0 0x561521fc0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.517+0000 7f59c37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521bf74d0 0x561521fbc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.517+0000 7f59c2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521c0c300 0x561521fc5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.517+0000 7f59c37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521bf74d0 0x561521fbc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45692/0 (socket says 172.21.15.67:45692) 2024-07-31T15:05:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.517+0000 7f59c37fe640 1 -- 172.21.15.67:0/2076630412 learned_addr learned my addr 172.21.15.67:0/2076630412 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.517+0000 7f59c37fe640 1 -- 172.21.15.67:0/2076630412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bef2f0 msgr2=0x561521fc0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.518+0000 7f59c37fe640 1 --2- 172.21.15.67:0/2076630412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bef2f0 0x561521fc0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.518+0000 7f59c37fe640 1 -- 172.21.15.67:0/2076630412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521c0c300 msgr2=0x561521fc5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.518+0000 7f59c37fe640 1 --2- 172.21.15.67:0/2076630412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521c0c300 0x561521fc5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.518+0000 7f59c37fe640 1 -- 172.21.15.67:0/2076630412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561521fc8ee0 con 0x561521bf74d0 2024-07-31T15:05:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.518+0000 7f59c37fe640 1 --2- 172.21.15.67:0/2076630412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521bf74d0 0x561521fbc3d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f59b400f690 tx=0x7f59b4009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=108283550eeec828 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.521+0000 7f59b0ff9640 1 -- 172.21.15.67:0/2076630412 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b401e040 con 0x561521bf74d0 2024-07-31T15:05:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.521+0000 7f59b0ff9640 1 -- 172.21.15.67:0/2076630412 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b400f790 con 0x561521bf74d0 2024-07-31T15:05:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.521+0000 7f59b0ff9640 1 -- 172.21.15.67:0/2076630412 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b40048e0 con 0x561521bf74d0 2024-07-31T15:05:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.523+0000 7f59d01df740 1 -- 172.21.15.67:0/2076630412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521bf74d0 msgr2=0x561521fbc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.523+0000 7f59d01df740 1 --2- 172.21.15.67:0/2076630412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521bf74d0 0x561521fbc3d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f59b400f690 tx=0x7f59b4009880 comp rx=0 tx=0).stop 2024-07-31T15:05:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.524+0000 7f59d01df740 1 -- 172.21.15.67:0/2076630412 shutdown_connections 2024-07-31T15:05:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.524+0000 7f59d01df740 1 --2- 172.21.15.67:0/2076630412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521bf74d0 0x561521fbc3d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.524+0000 7f59d01df740 1 --2- 172.21.15.67:0/2076630412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521bef2f0 0x561521fc0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.524+0000 7f59d01df740 1 --2- 172.21.15.67:0/2076630412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521c0c300 0x561521fc5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.524+0000 7f59d01df740 1 -- 172.21.15.67:0/2076630412 >> 172.21.15.67:0/2076630412 conn(0x561521fc1ea0 msgr2=0x561521fbbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.525+0000 7f59d01df740 1 -- 172.21.15.67:0/2076630412 shutdown_connections 2024-07-31T15:05:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.525+0000 7f59d01df740 1 -- 172.21.15.67:0/2076630412 wait complete. 2024-07-31T15:05:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.525+0000 7f59d01df740 1 -- start start 2024-07-31T15:05:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.525+0000 7f59d01df740 1 Processor -- start 2024-07-31T15:05:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.526+0000 7f59d01df740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521bef2f0 0x561521fcc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.526+0000 7f59d01df740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521bf74d0 0x561521fcc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.526+0000 7f59d01df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521c0c300 0x561522038e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.526+0000 7f59d01df740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4011360 con 0x561521bef2f0 2024-07-31T15:05:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.526+0000 7f59d01df740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4010c90 con 0x561521bf74d0 2024-07-31T15:05:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.526+0000 7f59d01df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59a400ded0 con 0x561521c0c300 2024-07-31T15:05:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.526+0000 7f59d01df740 10 client.0 _ll_register_callbacks cb 0x7f59bc01a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.526+0000 7f59c37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521c0c300 0x561522038e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.526+0000 7f59c27fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521bf74d0 0x561521fcc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.526+0000 7f59c27fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521bf74d0 0x561521fcc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45700/0 (socket says 172.21.15.67:45700) 2024-07-31T15:05:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.526+0000 7f59c27fc640 1 -- 172.21.15.67:0/2588114583 learned_addr learned my addr 172.21.15.67:0/2588114583 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.527+0000 7f59c27fc640 1 -- 172.21.15.67:0/2588114583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521c0c300 msgr2=0x561522038e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.527+0000 7f59c27fc640 1 --2- 172.21.15.67:0/2588114583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521c0c300 0x561522038e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.527+0000 7f59c27fc640 1 -- 172.21.15.67:0/2588114583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521bef2f0 msgr2=0x561521fcc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.527+0000 7f59c27fc640 1 --2- 172.21.15.67:0/2588114583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521bef2f0 0x561521fcc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.527+0000 7f59c27fc640 1 -- 172.21.15.67:0/2588114583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561522039500 con 0x561521bf74d0 2024-07-31T15:05:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.527+0000 7f59c27fc640 1 --2- 172.21.15.67:0/2588114583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521bf74d0 0x561521fcc6e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f59bc006010 tx=0x7f59bc004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13ab219f41acac6f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.527+0000 7f59aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:05:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.528+0000 7f59aa7fc640 1 -- 172.21.15.67:0/2588114583 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0f6040 con 0x561521bf74d0 2024-07-31T15:05:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.528+0000 7f59d01df740 1 -- 172.21.15.67:0/2588114583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b400f790 con 0x561521bf74d0 2024-07-31T15:05:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.528+0000 7f59aa7fc640 1 -- 172.21.15.67:0/2588114583 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f59bc0d8460 con 0x561521bf74d0 2024-07-31T15:05:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.528+0000 7f59aa7fc640 1 -- 172.21.15.67:0/2588114583 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f59bc0e6070 con 0x561521bf74d0 2024-07-31T15:05:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.543+0000 7f59d01df740 10 client.28052 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.543+0000 7f59d01df740 1 -- 172.21.15.67:0/2588114583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x561522039500 con 0x561521bf74d0 2024-07-31T15:05:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.547+0000 7f59aa7fc640 1 -- 172.21.15.67:0/2588114583 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b400f790 con 0x561521bf74d0 2024-07-31T15:05:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.547+0000 7f59aa7fc640 1 -- 172.21.15.67:0/2588114583 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f59bc0e9070 con 0x561521bf74d0 2024-07-31T15:05:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.547+0000 7f59d01df740 10 client.28052 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.547+0000 7f59d01df740 10 client.28052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:05:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.547+0000 7f59d01df740 20 client.28052 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.547+0000 7f59d01df740 1 --2- 172.21.15.67:0/2588114583 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561521c181b0 0x561522042d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.547+0000 7f59d01df740 4 client.28052 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:05:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.547+0000 7f59d01df740 1 -- 172.21.15.67:0/2588114583 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561521f32300 con 0x561521c181b0 2024-07-31T15:05:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.547+0000 7f59c2ffd640 1 --2- 172.21.15.67:0/2588114583 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561521c181b0 0x561522042d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.548+0000 7f59aa7fc640 10 client.28052 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.549+0000 7f59c2ffd640 1 --2- 172.21.15.67:0/2588114583 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561521c181b0 0x561522042d60 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80178c9b13c444bd server_cookie=75da67d8d447d0dc in_seq=0 out_seq=0 2024-07-31T15:05:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:00.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:00.699 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/ptdump.o 2024-07-31T15:05:00.782 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-iocost.o 2024-07-31T15:05:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.847+0000 7f59aa7fc640 1 -- 172.21.15.67:0/2588114583 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561521f32300 con 0x561521c181b0 2024-07-31T15:05:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.847+0000 7f59aa7fc640 10 client.28052 handle_command_reply: tid=0 2024-07-31T15:05:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.847+0000 7f59d01df740 1 client.28052 shutdown 2024-07-31T15:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.849+0000 7f59d01df740 1 -- 172.21.15.67:0/2588114583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521bf74d0 msgr2=0x561521fcc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.849+0000 7f59d01df740 1 --2- 172.21.15.67:0/2588114583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521bf74d0 0x561521fcc6e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f59bc006010 tx=0x7f59bc004040 comp rx=0 tx=0).stop 2024-07-31T15:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.849+0000 7f59d01df740 1 -- 172.21.15.67:0/2588114583 shutdown_connections 2024-07-31T15:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.849+0000 7f59d01df740 1 --2- 172.21.15.67:0/2588114583 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561521c181b0 0x561522042d60 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.849+0000 7f59d01df740 1 --2- 172.21.15.67:0/2588114583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561521c0c300 0x561522038e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.849+0000 7f59d01df740 1 --2- 172.21.15.67:0/2588114583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561521bf74d0 0x561521fcc6e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.849+0000 7f59d01df740 1 --2- 172.21.15.67:0/2588114583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561521bef2f0 0x561521fcc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.849+0000 7f59aa7fc640 0 client.28052 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.849+0000 7f59d01df740 1 -- 172.21.15.67:0/2588114583 >> 172.21.15.67:0/2588114583 conn(0x561521fc1ea0 msgr2=0x561521fc81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.849+0000 7f59d01df740 1 -- 172.21.15.67:0/2588114583 shutdown_connections 2024-07-31T15:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.849+0000 7f59d01df740 1 -- 172.21.15.67:0/2588114583 wait complete. 2024-07-31T15:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.850+0000 7f59d01df740 20 client.28052 trim_cache size 0 max 16384 2024-07-31T15:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:05:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.858+0000 7f59c92f5640 1 -- 172.21.15.67:0/2554241236 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f59ac0b3f10 msgr2=0x7f59ac0b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.858+0000 7f59c92f5640 1 --2- 172.21.15.67:0/2554241236 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f59ac0b3f10 0x7f59ac0b6420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x561521fb9960 tx=0x7f59b8008040 comp rx=0 tx=0).stop 2024-07-31T15:05:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.858+0000 7f59c92f5640 1 -- 172.21.15.67:0/2554241236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c41118d0 msgr2=0x7f59c4118180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.858+0000 7f59c92f5640 1 --2- 172.21.15.67:0/2554241236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c41118d0 0x7f59c4118180 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f59bc03bed0 tx=0x7f59bc040930 comp rx=0 tx=0).stop 2024-07-31T15:05:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.858+0000 7f59c92f5640 1 -- 172.21.15.67:0/2554241236 shutdown_connections 2024-07-31T15:05:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.858+0000 7f59c92f5640 1 --2- 172.21.15.67:0/2554241236 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f59ac0b3f10 0x7f59ac0b6420 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.858+0000 7f59c92f5640 1 --2- 172.21.15.67:0/2554241236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c411c5d0 0x7f59c4118750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.858+0000 7f59c92f5640 1 --2- 172.21.15.67:0/2554241236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c41118d0 0x7f59c4118180 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.858+0000 7f59c92f5640 1 --2- 172.21.15.67:0/2554241236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59c410a330 0x7f59c411beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.858+0000 7f59c92f5640 1 -- 172.21.15.67:0/2554241236 >> 172.21.15.67:0/2554241236 conn(0x7f59c40707c0 msgr2=0x7f59c411db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.859+0000 7f59c92f5640 1 -- 172.21.15.67:0/2554241236 shutdown_connections 2024-07-31T15:05:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:00.859+0000 7f59c92f5640 1 -- 172.21.15.67:0/2554241236 wait complete. 2024-07-31T15:05:00.972 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/authencesn.o 2024-07-31T15:05:00.998 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/fast_commit.o 2024-07-31T15:05:01.136 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/bpf/core.o 2024-07-31T15:05:01.182 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/e820.o 2024-07-31T15:05:01.213 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_sched_switch.o 2024-07-31T15:05:01.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: pgmap v840: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 378 KiB/s wr, 981 op/s 2024-07-31T15:05:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:01.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: pgmap v840: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 378 KiB/s wr, 981 op/s 2024-07-31T15:05:01.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:01.598 INFO:tasks.workunit.client.0.smithi067.stdout: AR mm/built-in.a 2024-07-31T15:05:01.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: pgmap v840: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 378 KiB/s wr, 981 op/s 2024-07-31T15:05:01.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:01.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:01.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:02.067 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/orphan.o 2024-07-31T15:05:02.076 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/rng.o 2024-07-31T15:05:02.212 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_nop.o 2024-07-31T15:05:02.316 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/msg_ring.o 2024-07-31T15:05:02.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:02.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:02.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:02.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:02.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:02.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:02.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:02.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:02.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:02.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:02.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:02.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:02.646 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/pci-dma.o 2024-07-31T15:05:03.002 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/drbg.o 2024-07-31T15:05:03.004 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/blktrace.o 2024-07-31T15:05:03.224 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_events.o 2024-07-31T15:05:03.251 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/acl.o 2024-07-31T15:05:03.347 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/mq-deadline.o 2024-07-31T15:05:03.511 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/timeout.o 2024-07-31T15:05:03.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: pgmap v841: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 308 KiB/s wr, 1.21k op/s 2024-07-31T15:05:03.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:03.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:03.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:03.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:03.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:03.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:03.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:03.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:03.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.558 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/quirks.o 2024-07-31T15:05:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: pgmap v841: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 308 KiB/s wr, 1.21k op/s 2024-07-31T15:05:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:03.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:03.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:03.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:03.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.673 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/bpf/built-in.a 2024-07-31T15:05:03.733 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/sqpoll.o 2024-07-31T15:05:03.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: pgmap v841: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 308 KiB/s wr, 1.21k op/s 2024-07-31T15:05:03.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:03.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:03.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:03.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:03.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:03.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:03.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:03.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:03.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:03.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.002 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/jitterentropy.o 2024-07-31T15:05:04.087 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/jitterentropy-kcapi.o 2024-07-31T15:05:04.093 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/xattr_security.o 2024-07-31T15:05:04.516 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/ghash-generic.o 2024-07-31T15:05:04.529 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/topology.o 2024-07-31T15:05:04.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:04.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.602 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/kdebugfs.o 2024-07-31T15:05:04.638 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/hash_info.o 2024-07-31T15:05:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:04.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.674 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/kyber-iosched.o 2024-07-31T15:05:04.692 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_export.o 2024-07-31T15:05:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:04.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:04.897 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/fdinfo.o 2024-07-31T15:05:04.935 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/ext4/built-in.a 2024-07-31T15:05:05.053 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/alternative.o 2024-07-31T15:05:05.076 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ramfs/inode.o 2024-07-31T15:05:05.092 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/rsapubkey.asn1.o 2024-07-31T15:05:05.113 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ramfs/file-mmu.o 2024-07-31T15:05:05.148 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/rsaprivkey.asn1.o 2024-07-31T15:05:05.148 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_event_perf.o 2024-07-31T15:05:05.198 INFO:tasks.workunit.client.0.smithi067.stdout: AR crypto/built-in.a 2024-07-31T15:05:05.426 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_events_filter.o 2024-07-31T15:05:05.469 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/hugetlbfs/inode.o 2024-07-31T15:05:05.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: pgmap v842: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 368 KiB/s wr, 1.92k op/s 2024-07-31T15:05:05.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.548 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.548 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.548 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.548 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.548 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.548 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.548 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.548 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.548 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.548 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.548 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.549 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.549 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.549 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.549 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.549 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.549 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.549 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.549 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.549 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.549 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.549 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.550 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.550 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.550 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.550 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.550 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.550 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.550 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.550 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.550 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.550 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.550 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.550 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.551 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.551 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.551 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.551 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.551 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.551 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.551 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.551 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.551 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.551 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.551 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.552 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:05.552 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.562 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.562 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.562 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.562 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.562 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.562 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.562 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.562 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.562 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: pgmap v842: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 368 KiB/s wr, 1.92k op/s 2024-07-31T15:05:05.562 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.563 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.563 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.563 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.563 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.563 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.563 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.563 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.563 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.563 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.563 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.564 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.564 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.564 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.564 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.564 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.564 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.564 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.564 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.564 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.564 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.566 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.566 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.566 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.566 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.566 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.566 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.566 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.566 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.566 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.566 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.567 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.567 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.567 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.567 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.567 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.567 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.567 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.567 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.567 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.567 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.567 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.568 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.568 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.568 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.568 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.568 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:05.568 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.699 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/tctx.o 2024-07-31T15:05:05.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: pgmap v842: 129 pgs: 129 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 368 KiB/s wr, 1.92k op/s 2024-07-31T15:05:05.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:05.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:05.793 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/ramfs/built-in.a 2024-07-31T15:05:05.846 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/poll.o 2024-07-31T15:05:06.003 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/cancel.o 2024-07-31T15:05:06.018 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq-pci.o 2024-07-31T15:05:06.021 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/kbuf.o 2024-07-31T15:05:06.473 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/i8253.o 2024-07-31T15:05:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:06.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:06.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.689 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq-virtio.o 2024-07-31T15:05:06.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:06.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:06.774 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_events_trigger.o 2024-07-31T15:05:06.909 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/rsrc.o 2024-07-31T15:05:06.973 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/hugetlbfs/built-in.a 2024-07-31T15:05:07.018 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_eprobe.o 2024-07-31T15:05:07.025 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/hw_breakpoint.o 2024-07-31T15:05:07.050 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq-debugfs.o 2024-07-31T15:05:07.065 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/cache.o 2024-07-31T15:05:07.089 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/dir.o 2024-07-31T15:05:07.513 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/fatent.o 2024-07-31T15:05:07.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:07.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:07.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:07.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: pgmap v843: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 591 KiB/s wr, 2.58k op/s 2024-07-31T15:05:07.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:07.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:07.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:07.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:07.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:07.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:07.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:05:07.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:07.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:07.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:07.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:07.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:07.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:07.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:07.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: pgmap v843: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 591 KiB/s wr, 2.58k op/s 2024-07-31T15:05:07.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:07.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:07.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:07.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:07.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:07.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:07.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:05:07.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:07.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:07.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:07.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:07.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:07.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:07.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:07.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: pgmap v843: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 591 KiB/s wr, 2.58k op/s 2024-07-31T15:05:07.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:07.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:07.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:07.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:07.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:07.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:07.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:05:07.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:07.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:07.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:07.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:07.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:07.937 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/file.o 2024-07-31T15:05:07.951 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/tsc.o 2024-07-31T15:05:07.952 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-pm.o 2024-07-31T15:05:08.174 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_kprobe.o 2024-07-31T15:05:08.353 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/tsc_msr.o 2024-07-31T15:05:08.535 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/inode.o 2024-07-31T15:05:08.549 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/holder.o 2024-07-31T15:05:08.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:08.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:05:08.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:05:08.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:05:08.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.592 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/rw.o 2024-07-31T15:05:08.601 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/misc.o 2024-07-31T15:05:08.603 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/io_delay.o 2024-07-31T15:05:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:05:08.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:05:08.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:05:08.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:08.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:05:08.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:05:08.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:05:08.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:08.742 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/nfs.o 2024-07-31T15:05:08.859 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/rtc.o 2024-07-31T15:05:08.972 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/resource.o 2024-07-31T15:05:09.124 INFO:tasks.workunit.client.0.smithi067.stdout: AR block/built-in.a 2024-07-31T15:05:09.157 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/namei_vfat.o 2024-07-31T15:05:09.220 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/namei_msdos.o 2024-07-31T15:05:09.399 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/clear_page_64.o 2024-07-31T15:05:09.418 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/math/div64.o 2024-07-31T15:05:09.438 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/cmdline.o 2024-07-31T15:05:09.487 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/math/gcd.o 2024-07-31T15:05:09.513 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/kernel/irqflags.o 2024-07-31T15:05:09.549 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/static_call.o 2024-07-31T15:05:09.549 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/cmpxchg16b_emu.o 2024-07-31T15:05:09.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:09.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:09.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:09 smithi067 ceph-mon[32750]: pgmap v844: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 600 KiB/s wr, 3.40k op/s 2024-07-31T15:05:09.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:05:09.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:09.562 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/math/lcm.o 2024-07-31T15:05:09.581 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/copy_mc.o 2024-07-31T15:05:09.593 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/copy_mc_64.o 2024-07-31T15:05:09.617 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/math/int_pow.o 2024-07-31T15:05:09.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:09.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:09.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:09 smithi073 ceph-mon[36714]: pgmap v844: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 600 KiB/s wr, 3.40k op/s 2024-07-31T15:05:09.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:05:09.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:09.665 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/irqchip/built-in.a 2024-07-31T15:05:09.669 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/math/int_sqrt.o 2024-07-31T15:05:09.699 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/error_report-traces.o 2024-07-31T15:05:09.701 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/bus/mhi/built-in.a 2024-07-31T15:05:09.715 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/bus/built-in.a 2024-07-31T15:05:09.736 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/math/reciprocal_div.o 2024-07-31T15:05:09.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:09.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:09.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:09 smithi077 ceph-mon[37327]: pgmap v844: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 600 KiB/s wr, 3.40k op/s 2024-07-31T15:05:09.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:05:09.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:05:09.746 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pwm/built-in.a 2024-07-31T15:05:09.794 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/msi/pcidev_msi.o 2024-07-31T15:05:09.825 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/math/rational.o 2024-07-31T15:05:09.852 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/opdef.o 2024-07-31T15:05:09.885 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/copy_page_64.o 2024-07-31T15:05:09.919 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/copy_user_64.o 2024-07-31T15:05:09.955 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/process.o 2024-07-31T15:05:09.956 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/copy_user_uncached_64.o 2024-07-31T15:05:10.002 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/cpu.o 2024-07-31T15:05:10.004 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/video/console/dummycon.o 2024-07-31T15:05:10.063 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/fat/built-in.a 2024-07-31T15:05:10.136 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/namei.o 2024-07-31T15:05:10.180 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/math/built-in.a 2024-07-31T15:05:10.243 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/memneq.o 2024-07-31T15:05:10.306 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/power-traces.o 2024-07-31T15:05:10.339 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/msi/api.o 2024-07-31T15:05:10.359 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/csum-copy_64.o 2024-07-31T15:05:10.393 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/csum-partial_64.o 2024-07-31T15:05:10.484 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/csum-wrappers_64.o 2024-07-31T15:05:10.573 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/video/console/vgacon.o 2024-07-31T15:05:10.608 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/utils.o 2024-07-31T15:05:10.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:10 smithi073 ceph-mon[36714]: pgmap v845: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 680 KiB/s wr, 3.85k op/s 2024-07-31T15:05:10.734 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/notif.o 2024-07-31T15:05:10.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:10 smithi077 ceph-mon[37327]: pgmap v845: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 680 KiB/s wr, 3.85k op/s 2024-07-31T15:05:10.743 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/inode.o 2024-07-31T15:05:10.790 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/delay.o 2024-07-31T15:05:10.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:10 smithi067 ceph-mon[32750]: pgmap v845: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 680 KiB/s wr, 3.85k op/s 2024-07-31T15:05:10.957 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/chacha.o 2024-07-31T15:05:10.978 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/msi/msi.o 2024-07-31T15:05:11.072 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/getuser.o 2024-07-31T15:05:11.080 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/aes.o 2024-07-31T15:05:11.112 INFO:tasks.workunit.client.0.smithi067.stdout: GEN arch/x86/lib/inat-tables.c 2024-07-31T15:05:11.141 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/ptrace.o 2024-07-31T15:05:11.159 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/insn-eval.o 2024-07-31T15:05:11.470 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/video/console/built-in.a 2024-07-31T15:05:11.527 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/video/backlight/backlight.o 2024-07-31T15:05:11.571 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/rpm-traces.o 2024-07-31T15:05:11.604 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/arc4.o 2024-07-31T15:05:11.693 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/io-wq.o 2024-07-31T15:05:11.862 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/msi/irqdomain.o 2024-07-31T15:05:11.985 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/dir.o 2024-07-31T15:05:12.228 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/gf128mul.o 2024-07-31T15:05:12.414 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/insn.o 2024-07-31T15:05:12.672 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/tls.o 2024-07-31T15:05:12.848 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/kaslr.o 2024-07-31T15:05:12.870 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pci/msi/built-in.a 2024-07-31T15:05:13.003 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/blake2s.o 2024-07-31T15:05:13.062 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/pcie/portdrv.o 2024-07-31T15:05:13.131 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/memcpy_64.o 2024-07-31T15:05:13.135 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/video/backlight/built-in.a 2024-07-31T15:05:13.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:12 smithi073 ceph-mon[36714]: pgmap v846: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 700 KiB/s wr, 3.57k op/s 2024-07-31T15:05:13.203 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/memmove_64.o 2024-07-31T15:05:13.220 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/util.o 2024-07-31T15:05:13.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:12 smithi077 ceph-mon[37327]: pgmap v846: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 700 KiB/s wr, 3.57k op/s 2024-07-31T15:05:13.242 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/memset_64.o 2024-07-31T15:05:13.251 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_dynevent.o 2024-07-31T15:05:13.261 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/video/fbdev/core/built-in.a 2024-07-31T15:05:13.279 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/misc.o 2024-07-31T15:05:13.286 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/video/fbdev/omap/built-in.a 2024-07-31T15:05:13.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:12 smithi067 ceph-mon[32750]: pgmap v846: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 700 KiB/s wr, 3.57k op/s 2024-07-31T15:05:13.325 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/pc-conf-reg.o 2024-07-31T15:05:13.335 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/video/fbdev/omap2/omapfb/dss/built-in.a 2024-07-31T15:05:13.359 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/video/fbdev/omap2/omapfb/displays/built-in.a 2024-07-31T15:05:13.372 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/video/fbdev/omap2/omapfb/built-in.a 2024-07-31T15:05:13.381 INFO:tasks.workunit.client.0.smithi067.stdout: AR io_uring/built-in.a 2024-07-31T15:05:13.392 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/video/fbdev/omap2/built-in.a 2024-07-31T15:05:13.405 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/blake2s-generic.o 2024-07-31T15:05:13.408 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/video/fbdev/built-in.a 2024-07-31T15:05:13.421 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_probe.o 2024-07-31T15:05:13.437 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/video/aperture.o 2024-07-31T15:05:13.505 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/putuser.o 2024-07-31T15:05:13.615 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/retpoline.o 2024-07-31T15:05:13.636 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/step.o 2024-07-31T15:05:13.670 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/usercopy.o 2024-07-31T15:05:13.782 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/rock.o 2024-07-31T15:05:13.817 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/pcie/rcec.o 2024-07-31T15:05:13.841 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/sha1.o 2024-07-31T15:05:13.978 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/usercopy_64.o 2024-07-31T15:05:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.093+0000 7f782fe9a640 1 -- 172.21.15.67:0/1494658177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78281116d0 msgr2=0x7f7828109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.093+0000 7f782fe9a640 1 --2- 172.21.15.67:0/1494658177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78281116d0 0x7f7828109df0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7820009ca0 tx=0x7f782002a8e0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.094+0000 7f782fe9a640 1 -- 172.21.15.67:0/1494658177 shutdown_connections 2024-07-31T15:05:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.094+0000 7f782fe9a640 1 --2- 172.21.15.67:0/1494658177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f782811c5d0 0x7f782811e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.094+0000 7f782fe9a640 1 --2- 172.21.15.67:0/1494658177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f782810a330 0x7f782810a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.094+0000 7f782fe9a640 1 --2- 172.21.15.67:0/1494658177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78281116d0 0x7f7828109df0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.094+0000 7f782fe9a640 1 -- 172.21.15.67:0/1494658177 >> 172.21.15.67:0/1494658177 conn(0x7f782806e350 msgr2=0x7f782806e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.097+0000 7f782fe9a640 1 -- 172.21.15.67:0/1494658177 shutdown_connections 2024-07-31T15:05:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.097+0000 7f782fe9a640 1 -- 172.21.15.67:0/1494658177 wait complete. 2024-07-31T15:05:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.098+0000 7f782fe9a640 1 Processor -- start 2024-07-31T15:05:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.098+0000 7f782fe9a640 1 -- start start 2024-07-31T15:05:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.098+0000 7f782fe9a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f782810a330 0x7f78281a8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.098+0000 7f782fe9a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78281116d0 0x7f78281a9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.098+0000 7f782fe9a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f782811c5d0 0x7f78281ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.098+0000 7f782fe9a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f78281220d0 con 0x7f782810a330 2024-07-31T15:05:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.098+0000 7f782fe9a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7828121f50 con 0x7f78281116d0 2024-07-31T15:05:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.098+0000 7f782fe9a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7828122250 con 0x7f782811c5d0 2024-07-31T15:05:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.099+0000 7f782dc36640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f782810a330 0x7f78281a8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.099+0000 7f782dc36640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f782810a330 0x7f78281a8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50008/0 (socket says 172.21.15.67:50008) 2024-07-31T15:05:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.099+0000 7f782d435640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78281116d0 0x7f78281a9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.099+0000 7f782dc36640 1 -- 172.21.15.67:0/1091193726 learned_addr learned my addr 172.21.15.67:0/1091193726 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.099+0000 7f782e437640 1 --2- 172.21.15.67:0/1091193726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f782811c5d0 0x7f78281ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.099+0000 7f782d435640 1 -- 172.21.15.67:0/1091193726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f782811c5d0 msgr2=0x7f78281ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.099+0000 7f782d435640 1 --2- 172.21.15.67:0/1091193726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f782811c5d0 0x7f78281ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.099+0000 7f782d435640 1 -- 172.21.15.67:0/1091193726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f782810a330 msgr2=0x7f78281a8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.099+0000 7f782d435640 1 --2- 172.21.15.67:0/1091193726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f782810a330 0x7f78281a8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.099+0000 7f782d435640 1 -- 172.21.15.67:0/1091193726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78281ae9c0 con 0x7f78281116d0 2024-07-31T15:05:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.099+0000 7f782d435640 1 --2- 172.21.15.67:0/1091193726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78281116d0 0x7f78281a9030 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f781800d6a0 tx=0x7f781800db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.100+0000 7f781effd640 1 -- 172.21.15.67:0/1091193726 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818017070 con 0x7f78281116d0 2024-07-31T15:05:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.100+0000 7f781effd640 1 -- 172.21.15.67:0/1091193726 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7818009820 con 0x7f78281116d0 2024-07-31T15:05:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.100+0000 7f781effd640 1 -- 172.21.15.67:0/1091193726 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818004c40 con 0x7f78281116d0 2024-07-31T15:05:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.100+0000 7f782fe9a640 1 -- 172.21.15.67:0/1091193726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78281aecb0 con 0x7f78281116d0 2024-07-31T15:05:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.100+0000 7f782fe9a640 1 -- 172.21.15.67:0/1091193726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7828111db0 con 0x7f78281116d0 2024-07-31T15:05:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.104+0000 7f781effd640 1 -- 172.21.15.67:0/1091193726 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7818005220 con 0x7f78281116d0 2024-07-31T15:05:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.105+0000 7f781effd640 1 --2- 172.21.15.67:0/1091193726 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78140b3db0 0x7f78140b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.105+0000 7f781effd640 1 -- 172.21.15.67:0/1091193726 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f78180e3980 con 0x7f78281116d0 2024-07-31T15:05:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.105+0000 7f782dc36640 1 --2- 172.21.15.67:0/1091193726 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78140b3db0 0x7f78140b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.106+0000 7f782dc36640 1 --2- 172.21.15.67:0/1091193726 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78140b3db0 0x7f78140b62c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7820005e30 tx=0x7f7820000f40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.126+0000 7f7836d8b740 1 Processor -- start 2024-07-31T15:05:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.126+0000 7f7836d8b740 1 -- start start 2024-07-31T15:05:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.126+0000 7f7836d8b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dc6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.126+0000 7f7836d8b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d99fb2f0 0x5653d9dc73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.126+0000 7f7836d8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a034d0 0x5653d9dc7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.126+0000 7f7836d8b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5653d9d3c3b0 con 0x5653d9a241b0 2024-07-31T15:05:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.126+0000 7f7836d8b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5653d9d2b8b0 con 0x5653d99fb2f0 2024-07-31T15:05:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.126+0000 7f7836d8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5653d9d2f8f0 con 0x5653d9a034d0 2024-07-31T15:05:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.126+0000 7f782d435640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d99fb2f0 0x5653d9dc73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.127+0000 7f782d435640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d99fb2f0 0x5653d9dc73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:41524/0 (socket says 172.21.15.67:41524) 2024-07-31T15:05:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.127+0000 7f782d435640 1 -- 172.21.15.67:0/359248323 learned_addr learned my addr 172.21.15.67:0/359248323 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.127+0000 7f782dc36640 1 --2- 172.21.15.67:0/359248323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dc6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.127+0000 7f782e437640 1 --2- 172.21.15.67:0/359248323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a034d0 0x5653d9dc7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.127+0000 7f782d435640 1 -- 172.21.15.67:0/359248323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a034d0 msgr2=0x5653d9dc7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.127+0000 7f782d435640 1 --2- 172.21.15.67:0/359248323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a034d0 0x5653d9dc7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.127+0000 7f782d435640 1 -- 172.21.15.67:0/359248323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 msgr2=0x5653d9dc6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.127+0000 7f782d435640 1 --2- 172.21.15.67:0/359248323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dc6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.127+0000 7f782d435640 1 -- 172.21.15.67:0/359248323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653d9cd29a0 con 0x5653d99fb2f0 2024-07-31T15:05:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.127+0000 7f782dc36640 1 --2- 172.21.15.67:0/359248323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dc6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:05:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.127+0000 7f782d435640 1 --2- 172.21.15.67:0/359248323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d99fb2f0 0x5653d9dc73e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f78180a6350 tx=0x7f78180ae960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f31ca1b5ed1d648 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.128+0000 7f7813fff640 1 -- 172.21.15.67:0/359248323 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818020040 con 0x5653d99fb2f0 2024-07-31T15:05:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.128+0000 7f7813fff640 1 -- 172.21.15.67:0/359248323 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78180e0fe0 con 0x5653d99fb2f0 2024-07-31T15:05:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.133+0000 7f7813fff640 1 -- 172.21.15.67:0/359248323 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78180b1e00 con 0x5653d99fb2f0 2024-07-31T15:05:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.134+0000 7f7836d8b740 1 -- 172.21.15.67:0/359248323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d99fb2f0 msgr2=0x5653d9dc73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.134+0000 7f7836d8b740 1 --2- 172.21.15.67:0/359248323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d99fb2f0 0x5653d9dc73e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f78180a6350 tx=0x7f78180ae960 comp rx=0 tx=0).stop 2024-07-31T15:05:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.134+0000 7f7836d8b740 1 -- 172.21.15.67:0/359248323 shutdown_connections 2024-07-31T15:05:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.134+0000 7f7836d8b740 1 --2- 172.21.15.67:0/359248323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a034d0 0x5653d9dc7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.134+0000 7f7836d8b740 1 --2- 172.21.15.67:0/359248323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d99fb2f0 0x5653d9dc73e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.134+0000 7f7836d8b740 1 --2- 172.21.15.67:0/359248323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dc6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.134+0000 7f7836d8b740 1 -- 172.21.15.67:0/359248323 >> 172.21.15.67:0/359248323 conn(0x5653d9a18300 msgr2=0x5653d9dc4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.134+0000 7f7836d8b740 1 -- 172.21.15.67:0/359248323 shutdown_connections 2024-07-31T15:05:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.134+0000 7f7836d8b740 1 -- 172.21.15.67:0/359248323 wait complete. 2024-07-31T15:05:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.135+0000 7f7836d8b740 1 -- start start 2024-07-31T15:05:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.135+0000 7f7836d8b740 1 Processor -- start 2024-07-31T15:05:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.136+0000 7f7836d8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d99fb2f0 0x5653d9dd29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.136+0000 7f7836d8b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d9a034d0 0x5653d9dd2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.136+0000 7f7836d8b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dde930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.136+0000 7f7836d8b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5653d9c65ab0 con 0x5653d9a241b0 2024-07-31T15:05:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.136+0000 7f7836d8b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5653d9cb3cf0 con 0x5653d9a034d0 2024-07-31T15:05:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.136+0000 7f7836d8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5653d9dd55b0 con 0x5653d99fb2f0 2024-07-31T15:05:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.136+0000 7f7836d8b740 10 client.0 _ll_register_callbacks cb 0x5653d9d259f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.137+0000 7f782d435640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d9a034d0 0x5653d9dd2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.137+0000 7f782d435640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d9a034d0 0x5653d9dd2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:41534/0 (socket says 172.21.15.67:41534) 2024-07-31T15:05:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.137+0000 7f782d435640 1 -- 172.21.15.67:0/151212388 learned_addr learned my addr 172.21.15.67:0/151212388 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.137+0000 7f782dc36640 1 --2- 172.21.15.67:0/151212388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d99fb2f0 0x5653d9dd29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.137+0000 7f782e437640 1 --2- 172.21.15.67:0/151212388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dde930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.137+0000 7f782d435640 1 -- 172.21.15.67:0/151212388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d99fb2f0 msgr2=0x5653d9dd29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.137+0000 7f782d435640 1 --2- 172.21.15.67:0/151212388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d99fb2f0 0x5653d9dd29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.137+0000 7f782d435640 1 -- 172.21.15.67:0/151212388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 msgr2=0x5653d9dde930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.137+0000 7f782d435640 1 --2- 172.21.15.67:0/151212388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dde930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.137+0000 7f782d435640 1 -- 172.21.15.67:0/151212388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653d9a8f790 con 0x5653d9a034d0 2024-07-31T15:05:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.137+0000 7f782e437640 1 --2- 172.21.15.67:0/151212388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dde930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:05:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.138+0000 7f782d435640 1 --2- 172.21.15.67:0/151212388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d9a034d0 0x5653d9dd2f00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f78180b3490 tx=0x7f78180aa0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ea83bcfb33e844a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.138+0000 7f78117fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:05:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.138+0000 7f78117fa640 1 -- 172.21.15.67:0/151212388 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818020050 con 0x5653d9a034d0 2024-07-31T15:05:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.138+0000 7f7836d8b740 1 -- 172.21.15.67:0/151212388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78180e0fe0 con 0x5653d9a034d0 2024-07-31T15:05:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.138+0000 7f78117fa640 1 -- 172.21.15.67:0/151212388 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f78180a9530 con 0x5653d9a034d0 2024-07-31T15:05:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.141+0000 7f78117fa640 1 -- 172.21.15.67:0/151212388 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f78180a9de0 con 0x5653d9a034d0 2024-07-31T15:05:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.142+0000 7f7836d8b740 10 client.28064 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.142+0000 7f7836d8b740 1 -- 172.21.15.67:0/151212388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5653d9b08620 con 0x5653d9a034d0 2024-07-31T15:05:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.142+0000 7f78117fa640 1 -- 172.21.15.67:0/151212388 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78180e0fe0 con 0x5653d9a034d0 2024-07-31T15:05:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.142+0000 7f78117fa640 1 -- 172.21.15.67:0/151212388 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f78180a9130 con 0x5653d9a034d0 2024-07-31T15:05:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.142+0000 7f7836d8b740 10 client.28064 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.142+0000 7f7836d8b740 10 client.28064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:05:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.142+0000 7f7836d8b740 20 client.28064 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.142+0000 7f7836d8b740 1 --2- 172.21.15.67:0/151212388 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653d99e2d80 0x5653d9de4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.142+0000 7f7836d8b740 4 client.28064 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:05:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.142+0000 7f7836d8b740 1 -- 172.21.15.67:0/151212388 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653d9d3e300 con 0x5653d99e2d80 2024-07-31T15:05:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.142+0000 7f782dc36640 1 --2- 172.21.15.67:0/151212388 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653d99e2d80 0x5653d9de4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.143+0000 7f782dc36640 1 --2- 172.21.15.67:0/151212388 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653d99e2d80 0x5653d9de4a50 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=911487c3716aa95f server_cookie=adeee5a386940d1 in_seq=0 out_seq=0 2024-07-31T15:05:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.143+0000 7f78117fa640 10 client.28064 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.145+0000 7f78117fa640 1 -- 172.21.15.67:0/151212388 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653d9d3e300 con 0x5653d99e2d80 2024-07-31T15:05:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.145+0000 7f78117fa640 10 client.28064 handle_command_reply: tid=0 2024-07-31T15:05:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.145+0000 7f7836d8b740 1 client.28064 shutdown 2024-07-31T15:05:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.145+0000 7f7836d8b740 1 -- 172.21.15.67:0/151212388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d9a034d0 msgr2=0x5653d9dd2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.145+0000 7f7836d8b740 1 --2- 172.21.15.67:0/151212388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d9a034d0 0x5653d9dd2f00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f78180b3490 tx=0x7f78180aa0a0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.146+0000 7f7836d8b740 1 -- 172.21.15.67:0/151212388 shutdown_connections 2024-07-31T15:05:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.146+0000 7f7836d8b740 1 --2- 172.21.15.67:0/151212388 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653d99e2d80 0x5653d9de4a50 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.146+0000 7f7836d8b740 1 --2- 172.21.15.67:0/151212388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dde930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.146+0000 7f7836d8b740 1 --2- 172.21.15.67:0/151212388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d9a034d0 0x5653d9dd2f00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.146+0000 7f7836d8b740 1 --2- 172.21.15.67:0/151212388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d99fb2f0 0x5653d9dd29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.146+0000 7f7836d8b740 1 -- 172.21.15.67:0/151212388 >> 172.21.15.67:0/151212388 conn(0x5653d9a18300 msgr2=0x5653d9dc7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.146+0000 7f78117fa640 0 client.28064 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.146+0000 7f7836d8b740 1 -- 172.21.15.67:0/151212388 shutdown_connections 2024-07-31T15:05:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.146+0000 7f7836d8b740 1 -- 172.21.15.67:0/151212388 wait complete. 2024-07-31T15:05:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.146+0000 7f7836d8b740 20 client.28064 trim_cache size 0 max 16384 2024-07-31T15:05:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.179+0000 7f7836d8b740 1 Processor -- start 2024-07-31T15:05:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.179+0000 7f7836d8b740 1 -- start start 2024-07-31T15:05:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.179+0000 7f7836d8b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d99fb2f0 0x5653d9dd2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.179+0000 7f7836d8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a034d0 0x5653d9dccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.179+0000 7f7836d8b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dde720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.179+0000 7f7836d8b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f780c00ff30 con 0x5653d9a241b0 2024-07-31T15:05:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.179+0000 7f7836d8b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f780c011a70 con 0x5653d99fb2f0 2024-07-31T15:05:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.179+0000 7f7836d8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f780c010600 con 0x5653d9a034d0 2024-07-31T15:05:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.180+0000 7f782d435640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a034d0 0x5653d9dccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.180+0000 7f782dc36640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d99fb2f0 0x5653d9dd2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.180+0000 7f782e437640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dde720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.180+0000 7f782dc36640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d99fb2f0 0x5653d9dd2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:41540/0 (socket says 172.21.15.67:41540) 2024-07-31T15:05:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.180+0000 7f782dc36640 1 -- 172.21.15.67:0/3004361965 learned_addr learned my addr 172.21.15.67:0/3004361965 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.180+0000 7f782dc36640 1 -- 172.21.15.67:0/3004361965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a034d0 msgr2=0x5653d9dccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.180+0000 7f782dc36640 1 --2- 172.21.15.67:0/3004361965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a034d0 0x5653d9dccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.180+0000 7f782dc36640 1 -- 172.21.15.67:0/3004361965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 msgr2=0x5653d9dde720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.180+0000 7f782dc36640 1 --2- 172.21.15.67:0/3004361965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dde720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.180+0000 7f782dc36640 1 -- 172.21.15.67:0/3004361965 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653d9dc91a0 con 0x5653d99fb2f0 2024-07-31T15:05:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.180+0000 7f782e437640 1 --2- 172.21.15.67:0/3004361965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dde720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:05:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.181+0000 7f782dc36640 1 --2- 172.21.15.67:0/3004361965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d99fb2f0 0x5653d9dd2e30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f782002aed0 tx=0x7f782003e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ca51bb5c4ea90d5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.182+0000 7f78137fe640 1 -- 172.21.15.67:0/3004361965 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820057070 con 0x5653d99fb2f0 2024-07-31T15:05:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.182+0000 7f78137fe640 1 -- 172.21.15.67:0/3004361965 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7820036040 con 0x5653d99fb2f0 2024-07-31T15:05:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.183+0000 7f7836d8b740 1 -- 172.21.15.67:0/3004361965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d99fb2f0 msgr2=0x5653d9dd2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.183+0000 7f7836d8b740 1 --2- 172.21.15.67:0/3004361965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d99fb2f0 0x5653d9dd2e30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f782002aed0 tx=0x7f782003e000 comp rx=0 tx=0).stop 2024-07-31T15:05:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.184+0000 7f7836d8b740 1 -- 172.21.15.67:0/3004361965 shutdown_connections 2024-07-31T15:05:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.184+0000 7f7836d8b740 1 --2- 172.21.15.67:0/3004361965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d9a241b0 0x5653d9dde720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.184+0000 7f7836d8b740 1 --2- 172.21.15.67:0/3004361965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a034d0 0x5653d9dccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.184+0000 7f7836d8b740 1 --2- 172.21.15.67:0/3004361965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d99fb2f0 0x5653d9dd2e30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.184+0000 7f7836d8b740 1 -- 172.21.15.67:0/3004361965 >> 172.21.15.67:0/3004361965 conn(0x5653d9a18300 msgr2=0x5653d9dd2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:14.189 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/seq/seq.o 2024-07-31T15:05:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.195+0000 7f7836d8b740 1 -- 172.21.15.67:0/3004361965 shutdown_connections 2024-07-31T15:05:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.195+0000 7f7836d8b740 1 -- 172.21.15.67:0/3004361965 wait complete. 2024-07-31T15:05:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.195+0000 7f7836d8b740 1 -- start start 2024-07-31T15:05:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.196+0000 7f7836d8b740 1 Processor -- start 2024-07-31T15:05:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.196+0000 7f7836d8b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d99fb2f0 0x5653d9ddad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.196+0000 7f7836d8b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d9a034d0 0x5653d9dd7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.196+0000 7f7836d8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a241b0 0x5653d9dd7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.196+0000 7f7836d8b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f780c0113a0 con 0x5653d99fb2f0 2024-07-31T15:05:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.196+0000 7f7836d8b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f780c010cd0 con 0x5653d9a034d0 2024-07-31T15:05:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.196+0000 7f7836d8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f780c00df30 con 0x5653d9a241b0 2024-07-31T15:05:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.196+0000 7f7836d8b740 10 client.0 _ll_register_callbacks cb 0x7f78180aa360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.197+0000 7f782d435640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d9a034d0 0x5653d9dd7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.197+0000 7f782d435640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d9a034d0 0x5653d9dd7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:41548/0 (socket says 172.21.15.67:41548) 2024-07-31T15:05:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.197+0000 7f782d435640 1 -- 172.21.15.67:0/472979647 learned_addr learned my addr 172.21.15.67:0/472979647 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.197+0000 7f782e437640 1 --2- 172.21.15.67:0/472979647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a241b0 0x5653d9dd7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.197+0000 7f782d435640 1 -- 172.21.15.67:0/472979647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a241b0 msgr2=0x5653d9dd7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.197+0000 7f782d435640 1 --2- 172.21.15.67:0/472979647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a241b0 0x5653d9dd7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.197+0000 7f782d435640 1 -- 172.21.15.67:0/472979647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d99fb2f0 msgr2=0x5653d9ddad60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:05:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.197+0000 7f782dc36640 1 --2- 172.21.15.67:0/472979647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d99fb2f0 0x5653d9ddad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.198+0000 7f782d435640 1 --2- 172.21.15.67:0/472979647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d99fb2f0 0x5653d9ddad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.198+0000 7f782d435640 1 -- 172.21.15.67:0/472979647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653d9dd82e0 con 0x5653d9a034d0 2024-07-31T15:05:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.198+0000 7f782dc36640 1 --2- 172.21.15.67:0/472979647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d99fb2f0 0x5653d9ddad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:05:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.198+0000 7f782d435640 1 --2- 172.21.15.67:0/472979647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d9a034d0 0x5653d9dd7860 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f78180aa430 tx=0x7f78180aa1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=654dec6828813e38 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.198+0000 7f78117fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:05:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.198+0000 7f78117fa640 1 -- 172.21.15.67:0/472979647 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818020070 con 0x5653d9a034d0 2024-07-31T15:05:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.198+0000 7f78117fa640 1 -- 172.21.15.67:0/472979647 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f78180a8fa0 con 0x5653d9a034d0 2024-07-31T15:05:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.198+0000 7f7836d8b740 1 -- 172.21.15.67:0/472979647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5653d9d39da0 con 0x5653d9a034d0 2024-07-31T15:05:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.198+0000 7f78117fa640 1 -- 172.21.15.67:0/472979647 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f78180a6190 con 0x5653d9a034d0 2024-07-31T15:05:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.198+0000 7f7836d8b740 10 client.28070 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.198+0000 7f7836d8b740 1 -- 172.21.15.67:0/472979647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5653d9e4bf10 con 0x5653d9a034d0 2024-07-31T15:05:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.199+0000 7f78117fa640 1 -- 172.21.15.67:0/472979647 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5653d9d39da0 con 0x5653d9a034d0 2024-07-31T15:05:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.200+0000 7f78117fa640 1 -- 172.21.15.67:0/472979647 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f78180a4a40 con 0x5653d9a034d0 2024-07-31T15:05:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.200+0000 7f7836d8b740 10 client.28070 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.200+0000 7f7836d8b740 10 client.28070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:05:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.200+0000 7f7836d8b740 20 client.28070 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.200+0000 7f7836d8b740 1 --2- 172.21.15.67:0/472979647 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653d99b81c0 0x5653d9e4e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.200+0000 7f7836d8b740 4 client.28070 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:05:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.200+0000 7f7836d8b740 1 -- 172.21.15.67:0/472979647 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653d9d374e0 con 0x5653d99b81c0 2024-07-31T15:05:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.200+0000 7f782dc36640 1 --2- 172.21.15.67:0/472979647 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653d99b81c0 0x5653d9e4e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.201+0000 7f78117fa640 10 client.28070 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:14.201+0000 7f782dc36640 1 --2- 172.21.15.67:0/472979647 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653d99b81c0 0x5653d9e4e360 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a9fbd322fff4e20 server_cookie=da0524da275be6c9 in_seq=0 out_seq=0 2024-07-31T15:05:14.297 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/video/cmdline.o 2024-07-31T15:05:14.358 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/i8237.o 2024-07-31T15:05:14.375 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/sha256.o 2024-07-31T15:05:14.587 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/pcie/aspm.o 2024-07-31T15:05:14.946 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/export.o 2024-07-31T15:05:14.979 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_uprobe.o 2024-07-31T15:05:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.015+0000 7f78117fa640 1 -- 172.21.15.67:0/472979647 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653d9d374e0 con 0x5653d99b81c0 2024-07-31T15:05:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.015+0000 7f78117fa640 10 client.28070 handle_command_reply: tid=0 2024-07-31T15:05:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.015+0000 7f7836d8b740 1 client.28070 shutdown 2024-07-31T15:05:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.015+0000 7f7836d8b740 1 -- 172.21.15.67:0/472979647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d9a034d0 msgr2=0x5653d9dd7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.015+0000 7f7836d8b740 1 --2- 172.21.15.67:0/472979647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d9a034d0 0x5653d9dd7860 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f78180aa430 tx=0x7f78180aa1a0 comp rx=0 tx=0).stop 2024-07-31T15:05:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.015+0000 7f7836d8b740 1 -- 172.21.15.67:0/472979647 shutdown_connections 2024-07-31T15:05:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.015+0000 7f7836d8b740 1 --2- 172.21.15.67:0/472979647 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653d99b81c0 0x5653d9e4e360 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.015+0000 7f7836d8b740 1 --2- 172.21.15.67:0/472979647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653d9a241b0 0x5653d9dd7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.015+0000 7f7836d8b740 1 --2- 172.21.15.67:0/472979647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653d9a034d0 0x5653d9dd7860 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.015+0000 7f7836d8b740 1 --2- 172.21.15.67:0/472979647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653d99fb2f0 0x5653d9ddad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.015+0000 7f7836d8b740 1 -- 172.21.15.67:0/472979647 >> 172.21.15.67:0/472979647 conn(0x5653d9a18300 msgr2=0x5653d9dc45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.015+0000 7f78117fa640 0 client.28070 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.015+0000 7f7836d8b740 1 -- 172.21.15.67:0/472979647 shutdown_connections 2024-07-31T15:05:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.016+0000 7f7836d8b740 1 -- 172.21.15.67:0/472979647 wait complete. 2024-07-31T15:05:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.016+0000 7f7836d8b740 20 client.28070 trim_cache size 0 max 16384 2024-07-31T15:05:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:05:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:05:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:05:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:05:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.029+0000 7f782fe9a640 1 -- 172.21.15.67:0/1091193726 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78140b3db0 msgr2=0x7f78140b62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.030+0000 7f782fe9a640 1 --2- 172.21.15.67:0/1091193726 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78140b3db0 0x7f78140b62c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7820005e30 tx=0x7f7820000f40 comp rx=0 tx=0).stop 2024-07-31T15:05:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.030+0000 7f782fe9a640 1 -- 172.21.15.67:0/1091193726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78281116d0 msgr2=0x7f78281a9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.030+0000 7f782fe9a640 1 --2- 172.21.15.67:0/1091193726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78281116d0 0x7f78281a9030 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f781800d6a0 tx=0x7f781800db60 comp rx=0 tx=0).stop 2024-07-31T15:05:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.030+0000 7f782fe9a640 1 -- 172.21.15.67:0/1091193726 shutdown_connections 2024-07-31T15:05:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.030+0000 7f782fe9a640 1 --2- 172.21.15.67:0/1091193726 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f78140b3db0 0x7f78140b62c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.030+0000 7f782fe9a640 1 --2- 172.21.15.67:0/1091193726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f782811c5d0 0x7f78281ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.030+0000 7f782fe9a640 1 --2- 172.21.15.67:0/1091193726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78281116d0 0x7f78281a9030 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.030+0000 7f782fe9a640 1 --2- 172.21.15.67:0/1091193726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f782810a330 0x7f78281a8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.030+0000 7f782fe9a640 1 -- 172.21.15.67:0/1091193726 >> 172.21.15.67:0/1091193726 conn(0x7f782806e350 msgr2=0x7f7828110350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.030+0000 7f782fe9a640 1 -- 172.21.15.67:0/1091193726 shutdown_connections 2024-07-31T15:05:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:15.030+0000 7f782fe9a640 1 -- 172.21.15.67:0/1091193726 wait complete. 2024-07-31T15:05:15.057 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/msr-smp.o 2024-07-31T15:05:15.075 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/stacktrace.o 2024-07-31T15:05:15.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:14 smithi073 ceph-mon[36714]: pgmap v847: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 955 KiB/s wr, 3.45k op/s 2024-07-31T15:05:15.180 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/seq/seq_lock.o 2024-07-31T15:05:15.197 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/crypto/built-in.a 2024-07-31T15:05:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:14 smithi077 ceph-mon[37327]: pgmap v847: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 955 KiB/s wr, 3.45k op/s 2024-07-31T15:05:15.248 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_inflate/inffast.o 2024-07-31T15:05:15.258 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/video/nomodeset.o 2024-07-31T15:05:15.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:14 smithi067 ceph-mon[32750]: pgmap v847: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 955 KiB/s wr, 3.45k op/s 2024-07-31T15:05:15.321 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/cache-smp.o 2024-07-31T15:05:15.425 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_inflate/inflate.o 2024-07-31T15:05:15.481 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/msr.o 2024-07-31T15:05:15.551 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/seq/seq_clientmgr.o 2024-07-31T15:05:15.557 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/joliet.o 2024-07-31T15:05:15.615 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/video/hdmi.o 2024-07-31T15:05:15.623 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/reboot.o 2024-07-31T15:05:15.893 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/pcie/pme.o 2024-07-31T15:05:15.953 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_inflate/infutil.o 2024-07-31T15:05:16.106 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/compress.o 2024-07-31T15:05:16.177 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/msr-reg.o 2024-07-31T15:05:16.211 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/msr-reg-export.o 2024-07-31T15:05:16.267 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_inflate/inftrees.o 2024-07-31T15:05:16.294 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/video/built-in.a 2024-07-31T15:05:16.325 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/hweight.o 2024-07-31T15:05:16.347 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_inflate/inflate_syms.o 2024-07-31T15:05:16.361 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/iomem.o 2024-07-31T15:05:16.417 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/lib/iomap_copy_64.o 2024-07-31T15:05:16.452 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/lib/inat.o 2024-07-31T15:05:16.478 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/msr.o 2024-07-31T15:05:16.487 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/rethook.o 2024-07-31T15:05:16.571 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pci/pcie/built-in.a 2024-07-31T15:05:16.625 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/lib/lib.a 2024-07-31T15:05:16.673 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/hotplug/pci_hotplug_core.o 2024-07-31T15:05:16.690 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/zlib_inflate/built-in.a 2024-07-31T15:05:16.692 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pci/controller/dwc/built-in.a 2024-07-31T15:05:16.718 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pci/controller/mobiveil/built-in.a 2024-07-31T15:05:16.732 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pci/controller/built-in.a 2024-07-31T15:05:16.736 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_deflate/deflate.o 2024-07-31T15:05:16.737 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/seq/seq_memory.o 2024-07-31T15:05:16.752 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/hotplug/acpi_pcihp.o 2024-07-31T15:05:16.762 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/lib/built-in.a 2024-07-31T15:05:16.789 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_deflate/deftree.o 2024-07-31T15:05:16.873 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/isofs/built-in.a 2024-07-31T15:05:16.989 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/client.o 2024-07-31T15:05:17.070 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpuid.o 2024-07-31T15:05:17.161 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/trace/built-in.a 2024-07-31T15:05:17.212 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/events/core.o 2024-07-31T15:05:17.285 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/early-quirks.o 2024-07-31T15:05:17.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:17 smithi073 ceph-mon[36714]: pgmap v848: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 2.78k op/s 2024-07-31T15:05:17.412 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/smp.o 2024-07-31T15:05:17.414 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pci/hotplug/built-in.a 2024-07-31T15:05:17.437 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/seq/seq_queue.o 2024-07-31T15:05:17.472 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pci/switch/built-in.a 2024-07-31T15:05:17.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:17 smithi077 ceph-mon[37327]: pgmap v848: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 2.78k op/s 2024-07-31T15:05:17.490 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_deflate/deflate_syms.o 2024-07-31T15:05:17.490 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/access.o 2024-07-31T15:05:17.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:17 smithi067 ceph-mon[32750]: pgmap v848: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 2.78k op/s 2024-07-31T15:05:17.578 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/smpboot.o 2024-07-31T15:05:17.834 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/zlib_deflate/built-in.a 2024-07-31T15:05:17.887 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/lzo/lzo1x_compress.o 2024-07-31T15:05:18.116 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/lz4/lz4_decompress.o 2024-07-31T15:05:18.128 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/seq/seq_fifo.o 2024-07-31T15:05:18.245 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/bus.o 2024-07-31T15:05:18.289 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/tsc_sync.o 2024-07-31T15:05:18.352 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/lzo/lzo1x_decompress_safe.o 2024-07-31T15:05:18.446 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/dir.o 2024-07-31T15:05:18.653 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/setup_percpu.o 2024-07-31T15:05:18.683 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/seq/seq_prioq.o 2024-07-31T15:05:18.767 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/lzo/built-in.a 2024-07-31T15:05:18.816 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/zstd_decompress_module.o 2024-07-31T15:05:18.821 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/xz/xz_dec_syms.o 2024-07-31T15:05:18.921 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/probe.o 2024-07-31T15:05:19.153 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/xz/xz_dec_stream.o 2024-07-31T15:05:19.174 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/lz4/built-in.a 2024-07-31T15:05:19.194 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/decompress/huf_decompress.o 2024-07-31T15:05:19.205 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/xz/xz_dec_lzma2.o 2024-07-31T15:05:19.234 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/seq/seq_timer.o 2024-07-31T15:05:19.238 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/mpparse.o 2024-07-31T15:05:19.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:19 smithi073 ceph-mon[36714]: pgmap v849: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 2.15k op/s 2024-07-31T15:05:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:19 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:05:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:19 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:05:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:19 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:05:19.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:19 smithi077 ceph-mon[37327]: pgmap v849: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 2.15k op/s 2024-07-31T15:05:19.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:19 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:05:19.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:19 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:05:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:19 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:05:19.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:19 smithi067 ceph-mon[32750]: pgmap v849: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 2.15k op/s 2024-07-31T15:05:19.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:19 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:05:19.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:19 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:05:19.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:19 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:05:19.608 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/xz/xz_dec_bcj.o 2024-07-31T15:05:19.813 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/trace_clock.o 2024-07-31T15:05:19.869 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/seq/seq_system.o 2024-07-31T15:05:19.945 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/trace.o 2024-07-31T15:05:20.020 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/xz/built-in.a 2024-07-31T15:05:20.076 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/generic_mpih-lshift.o 2024-07-31T15:05:20.094 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/seq/seq_ports.o 2024-07-31T15:05:20.242 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/host-bridge.o 2024-07-31T15:05:20.371 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/seq/seq_info.o 2024-07-31T15:05:20.413 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/rethook.o 2024-07-31T15:05:20.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:20 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:05:20.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:20 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:05:20.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:20 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:05:20.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:20 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:05:20.590 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/generic_mpih-mul1.o 2024-07-31T15:05:20.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:20 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:05:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:20 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:05:20.685 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/file.o 2024-07-31T15:05:20.764 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/seq/seq_dummy.o 2024-07-31T15:05:20.810 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/remove.o 2024-07-31T15:05:20.863 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/pci.o 2024-07-31T15:05:20.929 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/crash_core_64.o 2024-07-31T15:05:21.099 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/generic_mpih-mul2.o 2024-07-31T15:05:21.236 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/decompress/zstd_ddict.o 2024-07-31T15:05:21.278 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/core/seq/built-in.a 2024-07-31T15:05:21.336 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/sound.o 2024-07-31T15:05:21.360 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/decompress/zstd_decompress.o 2024-07-31T15:05:21.391 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/machine_kexec_64.o 2024-07-31T15:05:21.412 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/decompress/zstd_decompress_block.o 2024-07-31T15:05:21.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:21 smithi077 ceph-mon[37327]: pgmap v850: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 1.30k op/s 2024-07-31T15:05:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:21 smithi067 ceph-mon[32750]: pgmap v850: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 1.30k op/s 2024-07-31T15:05:21.619 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/generic_mpih-mul3.o 2024-07-31T15:05:21.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:21 smithi073 ceph-mon[36714]: pgmap v850: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 1.30k op/s 2024-07-31T15:05:21.865 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/events/ring_buffer.o 2024-07-31T15:05:21.895 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/init.o 2024-07-31T15:05:22.021 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/zstd_common_module.o 2024-07-31T15:05:22.145 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/generic_mpih-rshift.o 2024-07-31T15:05:22.162 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/getroot.o 2024-07-31T15:05:22.182 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/kernel/relocate_kernel_64.o 2024-07-31T15:05:22.228 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/crash.o 2024-07-31T15:05:22.374 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/generic_mpih-sub1.o 2024-07-31T15:05:22.638 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/memory.o 2024-07-31T15:05:22.659 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/inode.o 2024-07-31T15:05:22.726 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/events/callchain.o 2024-07-31T15:05:22.812 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/crash_dump_64.o 2024-07-31T15:05:22.874 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/generic_mpih-add1.o 2024-07-31T15:05:23.054 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/control.o 2024-07-31T15:05:23.071 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/super.o 2024-07-31T15:05:23.215 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/pci-driver.o 2024-07-31T15:05:23.342 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/module.o 2024-07-31T15:05:23.373 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/events/hw_breakpoint.o 2024-07-31T15:05:23.390 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/ec.o 2024-07-31T15:05:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:23 smithi077 ceph-mon[37327]: pgmap v851: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 504 op/s 2024-07-31T15:05:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:23 smithi067 ceph-mon[32750]: pgmap v851: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 504 op/s 2024-07-31T15:05:23.616 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/common/debug.o 2024-07-31T15:05:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:23 smithi073 ceph-mon[36714]: pgmap v851: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 504 op/s 2024-07-31T15:05:23.648 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/common/entropy_common.o 2024-07-31T15:05:23.864 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/common/error_private.o 2024-07-31T15:05:23.910 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/common/fse_decompress.o 2024-07-31T15:05:23.962 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/early_printk.o 2024-07-31T15:05:24.274 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/search.o 2024-07-31T15:05:24.291 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/common/zstd_common.o 2024-07-31T15:05:24.338 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/misc.o 2024-07-31T15:05:24.344 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/events/uprobes.o 2024-07-31T15:05:24.364 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpicoder.o 2024-07-31T15:05:24.368 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/hpet.o 2024-07-31T15:05:24.372 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/amd_nb.o 2024-07-31T15:05:24.468 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/zstd/built-in.a 2024-07-31T15:05:24.538 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/fonts/fonts.o 2024-07-31T15:05:24.549 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/io.o 2024-07-31T15:05:24.892 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/fonts/font_8x16.o 2024-07-31T15:05:24.916 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/pci-sysfs.o 2024-07-31T15:05:24.937 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/device.o 2024-07-31T15:05:25.060 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/kvm.o 2024-07-31T15:05:25.082 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-add.o 2024-07-31T15:05:25.230 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/kvmclock.o 2024-07-31T15:05:25.239 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/fonts/built-in.a 2024-07-31T15:05:25.282 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/argv_split.o 2024-07-31T15:05:25.384 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/info.o 2024-07-31T15:05:25.450 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/direct.o 2024-07-31T15:05:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:25 smithi077 ceph-mon[37327]: pgmap v852: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 325 op/s 2024-07-31T15:05:25.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:25 smithi067 ceph-mon[32750]: pgmap v852: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 325 op/s 2024-07-31T15:05:25.604 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/bug.o 2024-07-31T15:05:25.637 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-bit.o 2024-07-31T15:05:25.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:25 smithi073 ceph-mon[36714]: pgmap v852: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 325 op/s 2024-07-31T15:05:25.711 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/events/built-in.a 2024-07-31T15:05:25.749 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/fork.o 2024-07-31T15:05:25.864 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/paravirt.o 2024-07-31T15:05:25.983 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/rom.o 2024-07-31T15:05:26.132 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/pvclock.o 2024-07-31T15:05:26.138 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/isadma.o 2024-07-31T15:05:26.177 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/vmaster.o 2024-07-31T15:05:26.251 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-cmp.o 2024-07-31T15:05:26.555 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/pcspeaker.o 2024-07-31T15:05:26.559 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/setup-res.o 2024-07-31T15:05:26.606 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/net/built-in.a 2024-07-31T15:05:26.668 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/ctljack.o 2024-07-31T15:05:26.712 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/check.o 2024-07-31T15:05:26.717 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/irq.o 2024-07-31T15:05:26.885 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-sub-ui.o 2024-07-31T15:05:26.957 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/vpd.o 2024-07-31T15:05:27.041 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/pagelist.o 2024-07-31T15:05:27.159 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:27 smithi067 ceph-mon[32750]: pgmap v853: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 970 KiB/s wr, 190 op/s 2024-07-31T15:05:27.167 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/jack.o 2024-07-31T15:05:27.279 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/hwdep.o 2024-07-31T15:05:27.290 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/uprobes.o 2024-07-31T15:05:27.388 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/perf_regs.o 2024-07-31T15:05:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:27 smithi077 ceph-mon[37327]: pgmap v853: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 970 KiB/s wr, 190 op/s 2024-07-31T15:05:27.519 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-div.o 2024-07-31T15:05:27.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:27 smithi073 ceph-mon[36714]: pgmap v853: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 970 KiB/s wr, 190 op/s 2024-07-31T15:05:27.814 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/setup-bus.o 2024-07-31T15:05:27.862 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/exec_domain.o 2024-07-31T15:05:27.877 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-inv.o 2024-07-31T15:05:28.022 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/timer.o 2024-07-31T15:05:28.054 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-mod.o 2024-07-31T15:05:28.201 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/tracepoint.o 2024-07-31T15:05:28.229 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/vc.o 2024-07-31T15:05:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.297+0000 7f4ff7d5e640 1 -- 172.21.15.67:0/1594206305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff010d230 msgr2=0x7f4ff0116ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.297+0000 7f4ff7d5e640 1 --2- 172.21.15.67:0/1594206305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff010d230 0x7f4ff0116ca0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800adb0 tx=0x7f4fe802b7a0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.297+0000 7f4ff7d5e640 1 -- 172.21.15.67:0/1594206305 shutdown_connections 2024-07-31T15:05:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.297+0000 7f4ff7d5e640 1 --2- 172.21.15.67:0/1594206305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff010d230 0x7f4ff0116ca0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.297+0000 7f4ff7d5e640 1 --2- 172.21.15.67:0/1594206305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ff010c8e0 0x7f4ff010ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.297+0000 7f4ff7d5e640 1 --2- 172.21.15.67:0/1594206305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ff010a920 0x7f4ff010ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.297+0000 7f4ff7d5e640 1 -- 172.21.15.67:0/1594206305 >> 172.21.15.67:0/1594206305 conn(0x7f4ff006e350 msgr2=0x7f4ff006e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.298+0000 7f4ff7d5e640 1 -- 172.21.15.67:0/1594206305 shutdown_connections 2024-07-31T15:05:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.298+0000 7f4ff7d5e640 1 -- 172.21.15.67:0/1594206305 wait complete. 2024-07-31T15:05:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.299+0000 7f4ff7d5e640 1 Processor -- start 2024-07-31T15:05:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.299+0000 7f4ff7d5e640 1 -- start start 2024-07-31T15:05:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.299+0000 7f4ff7d5e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ff010a920 0x7f4ff01a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.299+0000 7f4ff7d5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ff010c8e0 0x7f4ff01a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.299+0000 7f4ff7d5e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff010d230 0x7f4ff01a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.299+0000 7f4ff7d5e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff0119860 con 0x7f4ff010d230 2024-07-31T15:05:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.299+0000 7f4ff7d5e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff01196e0 con 0x7f4ff010a920 2024-07-31T15:05:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.299+0000 7f4ff7d5e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff01199e0 con 0x7f4ff010c8e0 2024-07-31T15:05:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.300+0000 7f4ff62fb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff010d230 0x7f4ff01a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.300+0000 7f4ff5afa640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ff010a920 0x7f4ff01a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.300+0000 7f4ff62fb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff010d230 0x7f4ff01a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42068/0 (socket says 172.21.15.67:42068) 2024-07-31T15:05:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.300+0000 7f4ff62fb640 1 -- 172.21.15.67:0/957007108 learned_addr learned my addr 172.21.15.67:0/957007108 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.300+0000 7f4ff52f9640 1 --2- 172.21.15.67:0/957007108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ff010c8e0 0x7f4ff01a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.300+0000 7f4ff5afa640 1 -- 172.21.15.67:0/957007108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ff010c8e0 msgr2=0x7f4ff01a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.300+0000 7f4ff5afa640 1 --2- 172.21.15.67:0/957007108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ff010c8e0 0x7f4ff01a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.300+0000 7f4ff5afa640 1 -- 172.21.15.67:0/957007108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff010d230 msgr2=0x7f4ff01a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.300+0000 7f4ff5afa640 1 --2- 172.21.15.67:0/957007108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff010d230 0x7f4ff01a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.300+0000 7f4ff5afa640 1 -- 172.21.15.67:0/957007108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff01b06e0 con 0x7f4ff010a920 2024-07-31T15:05:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.300+0000 7f4ff62fb640 1 --2- 172.21.15.67:0/957007108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff010d230 0x7f4ff01a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:05:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.301+0000 7f4ff5afa640 1 --2- 172.21.15.67:0/957007108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ff010a920 0x7f4ff01a0040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00e9c0 tx=0x7f4fec00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.311+0000 7f4fe6ffd640 1 -- 172.21.15.67:0/957007108 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec00cd00 con 0x7f4ff010a920 2024-07-31T15:05:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.311+0000 7f4ff7d5e640 1 -- 172.21.15.67:0/957007108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff01b09d0 con 0x7f4ff010a920 2024-07-31T15:05:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.311+0000 7f4ff7d5e640 1 -- 172.21.15.67:0/957007108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff01b0f20 con 0x7f4ff010a920 2024-07-31T15:05:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.313+0000 7f4fe6ffd640 1 -- 172.21.15.67:0/957007108 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fec0108e0 con 0x7f4ff010a920 2024-07-31T15:05:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.313+0000 7f4fe6ffd640 1 -- 172.21.15.67:0/957007108 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec018820 con 0x7f4ff010a920 2024-07-31T15:05:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.317+0000 7f4ffec54740 1 Processor -- start 2024-07-31T15:05:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.317+0000 7f4ffec54740 1 -- start start 2024-07-31T15:05:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.317+0000 7f4ffec54740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a08221b0 0x5566a0bc4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.317+0000 7f4ffec54740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a07f92f0 0x5566a0bc53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.317+0000 7f4ffec54740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08014d0 0x5566a0bc5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.317+0000 7f4ffec54740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5566a0b3a3b0 con 0x5566a08221b0 2024-07-31T15:05:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.317+0000 7f4ffec54740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5566a0b298b0 con 0x5566a08014d0 2024-07-31T15:05:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.317+0000 7f4ffec54740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5566a0b2d8f0 con 0x5566a07f92f0 2024-07-31T15:05:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.318+0000 7f4ff62fb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a08221b0 0x5566a0bc4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.318+0000 7f4ff62fb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a08221b0 0x5566a0bc4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42070/0 (socket says 172.21.15.67:42070) 2024-07-31T15:05:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.318+0000 7f4ff52f9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08014d0 0x5566a0bc5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.318+0000 7f4ff62fb640 1 -- 172.21.15.67:0/3225550597 learned_addr learned my addr 172.21.15.67:0/3225550597 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.318+0000 7f4fe6ffd640 1 -- 172.21.15.67:0/957007108 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4fec01f020 con 0x7f4ff010a920 2024-07-31T15:05:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.318+0000 7f4ff52f9640 1 -- 172.21.15.67:0/3225550597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a07f92f0 msgr2=0x5566a0bc53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.318+0000 7f4ff52f9640 1 --2- 172.21.15.67:0/3225550597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a07f92f0 0x5566a0bc53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.318+0000 7f4ff52f9640 1 -- 172.21.15.67:0/3225550597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a08221b0 msgr2=0x5566a0bc4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.318+0000 7f4ff52f9640 1 --2- 172.21.15.67:0/3225550597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a08221b0 0x5566a0bc4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.318+0000 7f4ff52f9640 1 -- 172.21.15.67:0/3225550597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566a0ad09a0 con 0x5566a08014d0 2024-07-31T15:05:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.318+0000 7f4ff62fb640 1 --2- 172.21.15.67:0/3225550597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a08221b0 0x5566a0bc4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:05:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.319+0000 7f4ff52f9640 1 --2- 172.21.15.67:0/3225550597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08014d0 0x5566a0bc5920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4ff006c170 tx=0x7f4fe0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5315a0211532885c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:28.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.319+0000 7f4fe6ffd640 1 --2- 172.21.15.67:0/957007108 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4fdc0b3ef0 0x7f4fdc0b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.319+0000 7f4ff62fb640 1 --2- 172.21.15.67:0/957007108 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4fdc0b3ef0 0x7f4fdc0b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.319+0000 7f4fe6ffd640 1 -- 172.21.15.67:0/957007108 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f4fec0e3f10 con 0x7f4ff010a920 2024-07-31T15:05:28.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.320+0000 7f4ff62fb640 1 --2- 172.21.15.67:0/957007108 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4fdc0b3ef0 0x7f4fdc0b6400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8033ff0 tx=0x7f4fe8004a80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.326+0000 7f4fdbfff640 1 -- 172.21.15.67:0/3225550597 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0004050 con 0x5566a08014d0 2024-07-31T15:05:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.326+0000 7f4fdbfff640 1 -- 172.21.15.67:0/3225550597 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fe00054d0 con 0x5566a08014d0 2024-07-31T15:05:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.326+0000 7f4fdbfff640 1 -- 172.21.15.67:0/3225550597 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0004ca0 con 0x5566a08014d0 2024-07-31T15:05:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.327+0000 7f4ffec54740 1 -- 172.21.15.67:0/3225550597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08014d0 msgr2=0x5566a0bc5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.327+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3225550597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08014d0 0x5566a0bc5920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4ff006c170 tx=0x7f4fe0011810 comp rx=0 tx=0).stop 2024-07-31T15:05:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.337+0000 7f4ffec54740 1 -- 172.21.15.67:0/3225550597 shutdown_connections 2024-07-31T15:05:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.337+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3225550597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08014d0 0x5566a0bc5920 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.337+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3225550597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a07f92f0 0x5566a0bc53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.337+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3225550597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a08221b0 0x5566a0bc4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.337+0000 7f4ffec54740 1 -- 172.21.15.67:0/3225550597 >> 172.21.15.67:0/3225550597 conn(0x5566a0816300 msgr2=0x5566a0bc2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.337+0000 7f4ffec54740 1 -- 172.21.15.67:0/3225550597 shutdown_connections 2024-07-31T15:05:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.337+0000 7f4ffec54740 1 -- 172.21.15.67:0/3225550597 wait complete. 2024-07-31T15:05:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.338+0000 7f4ffec54740 1 -- start start 2024-07-31T15:05:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.338+0000 7f4ffec54740 1 Processor -- start 2024-07-31T15:05:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.338+0000 7f4ffec54740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a07f92f0 0x5566a0bd0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ffec54740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08014d0 0x5566a0bd1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ffec54740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08221b0 0x5566a0be51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ffec54740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5566a0a63ab0 con 0x5566a07f92f0 2024-07-31T15:05:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ffec54740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5566a0ab1cf0 con 0x5566a08221b0 2024-07-31T15:05:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ffec54740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5566a0bdbe30 con 0x5566a08014d0 2024-07-31T15:05:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ffec54740 10 client.0 _ll_register_callbacks cb 0x5566a0b239f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ff5afa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a07f92f0 0x5566a0bd0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ff5afa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a07f92f0 0x5566a0bd0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42080/0 (socket says 172.21.15.67:42080) 2024-07-31T15:05:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ff5afa640 1 -- 172.21.15.67:0/3894095508 learned_addr learned my addr 172.21.15.67:0/3894095508 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ff62fb640 1 --2- 172.21.15.67:0/3894095508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08221b0 0x5566a0be51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ff52f9640 1 --2- 172.21.15.67:0/3894095508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08014d0 0x5566a0bd1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ff5afa640 1 -- 172.21.15.67:0/3894095508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08014d0 msgr2=0x5566a0bd1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ff5afa640 1 --2- 172.21.15.67:0/3894095508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08014d0 0x5566a0bd1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ff5afa640 1 -- 172.21.15.67:0/3894095508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08221b0 msgr2=0x5566a0be51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ff5afa640 1 --2- 172.21.15.67:0/3894095508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08221b0 0x5566a0be51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.339+0000 7f4ff5afa640 1 -- 172.21.15.67:0/3894095508 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566a088d790 con 0x5566a07f92f0 2024-07-31T15:05:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.340+0000 7f4ff52f9640 1 --2- 172.21.15.67:0/3894095508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08014d0 0x5566a0bd1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:05:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.340+0000 7f4ff5afa640 1 --2- 172.21.15.67:0/3894095508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a07f92f0 0x5566a0bd0ad0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0e1350 tx=0x7f4fec0aa680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625897b4c33ee4cc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.348+0000 7f4fd97fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:05:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.348+0000 7f4fd97fa640 1 -- 172.21.15.67:0/3894095508 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0ab370 con 0x5566a07f92f0 2024-07-31T15:05:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.348+0000 7f4ffec54740 1 -- 172.21.15.67:0/3894095508 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe00054d0 con 0x5566a07f92f0 2024-07-31T15:05:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.348+0000 7f4fd97fa640 1 -- 172.21.15.67:0/3894095508 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f4fec0cd2e0 con 0x5566a07f92f0 2024-07-31T15:05:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.350+0000 7f4fd97fa640 1 -- 172.21.15.67:0/3894095508 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f4fec0aac70 con 0x5566a07f92f0 2024-07-31T15:05:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.350+0000 7f4ffec54740 10 client.18510 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.350+0000 7f4ffec54740 1 -- 172.21.15.67:0/3894095508 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5566a0906620 con 0x5566a07f92f0 2024-07-31T15:05:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.350+0000 7f4fd97fa640 1 -- 172.21.15.67:0/3894095508 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fe00054d0 con 0x5566a07f92f0 2024-07-31T15:05:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.350+0000 7f4fd97fa640 1 -- 172.21.15.67:0/3894095508 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f4fec0c6070 con 0x5566a07f92f0 2024-07-31T15:05:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.350+0000 7f4ffec54740 10 client.18510 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.350+0000 7f4ffec54740 10 client.18510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:05:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.350+0000 7f4ffec54740 20 client.18510 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.351+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3894095508 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5566a07e0d80 0x5566a0bd94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.351+0000 7f4ffec54740 4 client.18510 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:05:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.351+0000 7f4ffec54740 1 -- 172.21.15.67:0/3894095508 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566a0b3c300 con 0x5566a07e0d80 2024-07-31T15:05:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.351+0000 7f4ff52f9640 1 --2- 172.21.15.67:0/3894095508 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5566a07e0d80 0x5566a0bd94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.352+0000 7f4fd97fa640 10 client.18510 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.352+0000 7f4ff52f9640 1 --2- 172.21.15.67:0/3894095508 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5566a07e0d80 0x5566a0bd94d0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c56b19375edfc9e3 server_cookie=53a3a0dce89ef346 in_seq=0 out_seq=0 2024-07-31T15:05:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.352+0000 7f4fd97fa640 1 -- 172.21.15.67:0/3894095508 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566a0b3c300 con 0x5566a07e0d80 2024-07-31T15:05:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.352+0000 7f4fd97fa640 10 client.18510 handle_command_reply: tid=0 2024-07-31T15:05:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.352+0000 7f4ffec54740 1 client.18510 shutdown 2024-07-31T15:05:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.353+0000 7f4ffec54740 1 -- 172.21.15.67:0/3894095508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a07f92f0 msgr2=0x5566a0bd0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.353+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3894095508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a07f92f0 0x5566a0bd0ad0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0e1350 tx=0x7f4fec0aa680 comp rx=0 tx=0).stop 2024-07-31T15:05:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.353+0000 7f4ffec54740 1 -- 172.21.15.67:0/3894095508 shutdown_connections 2024-07-31T15:05:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.353+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3894095508 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5566a07e0d80 0x5566a0bd94d0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.353+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3894095508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08221b0 0x5566a0be51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.353+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3894095508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08014d0 0x5566a0bd1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.353+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3894095508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a07f92f0 0x5566a0bd0ad0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.353+0000 7f4ffec54740 1 -- 172.21.15.67:0/3894095508 >> 172.21.15.67:0/3894095508 conn(0x5566a0816300 msgr2=0x5566a0bc5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.353+0000 7f4fd97fa640 0 client.18510 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.360+0000 7f4ffec54740 1 -- 172.21.15.67:0/3894095508 shutdown_connections 2024-07-31T15:05:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.360+0000 7f4ffec54740 1 -- 172.21.15.67:0/3894095508 wait complete. 2024-07-31T15:05:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.360+0000 7f4ffec54740 20 client.18510 trim_cache size 0 max 16384 2024-07-31T15:05:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.409+0000 7f4ffec54740 1 Processor -- start 2024-07-31T15:05:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.423+0000 7f4ffec54740 1 -- start start 2024-07-31T15:05:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.423+0000 7f4ffec54740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08014d0 0x5566a0bca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.423+0000 7f4ffec54740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a08221b0 0x5566a0be4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.424+0000 7f4ff5afa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08014d0 0x5566a0bca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.424+0000 7f4ff5afa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08014d0 0x5566a0bca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:51052/0 (socket says 172.21.15.67:51052) 2024-07-31T15:05:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.424+0000 7f4ffec54740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a07b61c0 0x5566a0bc2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.424+0000 7f4ff52f9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a08221b0 0x5566a0be4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.424+0000 7f4ff5afa640 1 -- 172.21.15.67:0/3535814130 learned_addr learned my addr 172.21.15.67:0/3535814130 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.424+0000 7f4ffec54740 1 -- 172.21.15.67:0/3535814130 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd400ff10 con 0x5566a08221b0 2024-07-31T15:05:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.424+0000 7f4ffec54740 1 -- 172.21.15.67:0/3535814130 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4011a50 con 0x5566a07b61c0 2024-07-31T15:05:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.424+0000 7f4ffec54740 1 -- 172.21.15.67:0/3535814130 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd40105e0 con 0x5566a08014d0 2024-07-31T15:05:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.424+0000 7f4ff62fb640 1 --2- 172.21.15.67:0/3535814130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a07b61c0 0x5566a0bc2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.424+0000 7f4ff5afa640 1 -- 172.21.15.67:0/3535814130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a07b61c0 msgr2=0x5566a0bc2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.425+0000 7f4ff5afa640 1 --2- 172.21.15.67:0/3535814130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a07b61c0 0x5566a0bc2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.425+0000 7f4ff5afa640 1 -- 172.21.15.67:0/3535814130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a08221b0 msgr2=0x5566a0be4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.425+0000 7f4ff5afa640 1 --2- 172.21.15.67:0/3535814130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a08221b0 0x5566a0be4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.425+0000 7f4ff5afa640 1 -- 172.21.15.67:0/3535814130 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566a0bc3730 con 0x5566a08014d0 2024-07-31T15:05:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.425+0000 7f4ff52f9640 1 --2- 172.21.15.67:0/3535814130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a08221b0 0x5566a0be4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:05:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.425+0000 7f4ff5afa640 1 --2- 172.21.15.67:0/3535814130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08014d0 0x5566a0bca670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0e42c0 tx=0x7f4fec0aa6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4b276a8ee517dfb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.435+0000 7f4fdb7fe640 1 -- 172.21.15.67:0/3535814130 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0ab370 con 0x5566a08014d0 2024-07-31T15:05:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.435+0000 7f4fdb7fe640 1 -- 172.21.15.67:0/3535814130 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fec0bfe50 con 0x5566a08014d0 2024-07-31T15:05:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.435+0000 7f4fdb7fe640 1 -- 172.21.15.67:0/3535814130 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0aa100 con 0x5566a08014d0 2024-07-31T15:05:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.436+0000 7f4ffec54740 1 -- 172.21.15.67:0/3535814130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08014d0 msgr2=0x5566a0bca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.436+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3535814130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08014d0 0x5566a0bca670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0e42c0 tx=0x7f4fec0aa6f0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.436+0000 7f4ffec54740 1 -- 172.21.15.67:0/3535814130 shutdown_connections 2024-07-31T15:05:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.436+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3535814130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a07b61c0 0x5566a0bc2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.436+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3535814130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a08221b0 0x5566a0be4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.436+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3535814130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08014d0 0x5566a0bca670 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.436+0000 7f4ffec54740 1 -- 172.21.15.67:0/3535814130 >> 172.21.15.67:0/3535814130 conn(0x5566a0816300 msgr2=0x5566a0bc9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.437+0000 7f4ffec54740 1 -- 172.21.15.67:0/3535814130 shutdown_connections 2024-07-31T15:05:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.437+0000 7f4ffec54740 1 -- 172.21.15.67:0/3535814130 wait complete. 2024-07-31T15:05:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.437+0000 7f4ffec54740 1 -- start start 2024-07-31T15:05:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.437+0000 7f4ffec54740 1 Processor -- start 2024-07-31T15:05:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.438+0000 7f4ffec54740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a07b61c0 0x5566a0bdb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.438+0000 7f4ffec54740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08014d0 0x5566a0be1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.438+0000 7f4ffec54740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08221b0 0x5566a0be1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.438+0000 7f4ffec54740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4011380 con 0x5566a07b61c0 2024-07-31T15:05:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.438+0000 7f4ffec54740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4010cb0 con 0x5566a08014d0 2024-07-31T15:05:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.438+0000 7f4ffec54740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd400df10 con 0x5566a08221b0 2024-07-31T15:05:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.438+0000 7f4ffec54740 10 client.0 _ll_register_callbacks cb 0x7f4fec009390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.439+0000 7f4ff62fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08221b0 0x5566a0be1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.439+0000 7f4ff62fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08221b0 0x5566a0be1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:51066/0 (socket says 172.21.15.67:51066) 2024-07-31T15:05:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.439+0000 7f4ff62fb640 1 -- 172.21.15.67:0/3054354920 learned_addr learned my addr 172.21.15.67:0/3054354920 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.439+0000 7f4ff5afa640 1 --2- 172.21.15.67:0/3054354920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a07b61c0 0x5566a0bdb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.439+0000 7f4ff52f9640 1 --2- 172.21.15.67:0/3054354920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08014d0 0x5566a0be1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.439+0000 7f4ff62fb640 1 -- 172.21.15.67:0/3054354920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08014d0 msgr2=0x5566a0be1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.439+0000 7f4ff62fb640 1 --2- 172.21.15.67:0/3054354920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08014d0 0x5566a0be1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.439+0000 7f4ff62fb640 1 -- 172.21.15.67:0/3054354920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a07b61c0 msgr2=0x5566a0bdb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.439+0000 7f4ff62fb640 1 --2- 172.21.15.67:0/3054354920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a07b61c0 0x5566a0bdb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.439+0000 7f4ff62fb640 1 -- 172.21.15.67:0/3054354920 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566a0bc73d0 con 0x5566a08221b0 2024-07-31T15:05:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.440+0000 7f4ff5afa640 1 --2- 172.21.15.67:0/3054354920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a07b61c0 0x5566a0bdb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:05:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.440+0000 7f4ff62fb640 1 --2- 172.21.15.67:0/3054354920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08221b0 0x5566a0be1760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8004af0 tx=0x7f4fe800a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0ed3971a8fb05b1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.440+0000 7f4fd97fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:05:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.440+0000 7f4fd97fa640 1 -- 172.21.15.67:0/3054354920 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe805a060 con 0x5566a08221b0 2024-07-31T15:05:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.440+0000 7f4ffec54740 1 -- 172.21.15.67:0/3054354920 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fec0bfe50 con 0x5566a08221b0 2024-07-31T15:05:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.441+0000 7f4fd97fa640 1 -- 172.21.15.67:0/3054354920 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f4fe8030920 con 0x5566a08221b0 2024-07-31T15:05:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.441+0000 7f4fd97fa640 1 -- 172.21.15.67:0/3054354920 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f4fe8061050 con 0x5566a08221b0 2024-07-31T15:05:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.441+0000 7f4ffec54740 10 client.28012 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.441+0000 7f4ffec54740 1 -- 172.21.15.67:0/3054354920 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5566a0c5b7a0 con 0x5566a08221b0 2024-07-31T15:05:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.441+0000 7f4fd97fa640 1 -- 172.21.15.67:0/3054354920 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fec0bfe50 con 0x5566a08221b0 2024-07-31T15:05:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.442+0000 7f4fd97fa640 1 -- 172.21.15.67:0/3054354920 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f4fe804ac60 con 0x5566a08221b0 2024-07-31T15:05:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.442+0000 7f4ffec54740 10 client.28012 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.442+0000 7f4ffec54740 10 client.28012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:05:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.442+0000 7f4ffec54740 20 client.28012 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.442+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3054354920 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5566a0c5bf30 0x5566a0c5e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.442+0000 7f4ffec54740 4 client.28012 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:05:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.442+0000 7f4ffec54740 1 -- 172.21.15.67:0/3054354920 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566a0b37da0 con 0x5566a0c5bf30 2024-07-31T15:05:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.442+0000 7f4ff5afa640 1 --2- 172.21.15.67:0/3054354920 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5566a0c5bf30 0x5566a0c5e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.443+0000 7f4fd97fa640 10 client.28012 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:28.443+0000 7f4ff5afa640 1 --2- 172.21.15.67:0/3054354920 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5566a0c5bf30 0x5566a0c5e350 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5cd70818df78132 server_cookie=3464ee8ecb40db61 in_seq=0 out_seq=0 2024-07-31T15:05:28.605 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/panic.o 2024-07-31T15:05:28.633 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cpu.o 2024-07-31T15:05:28.759 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-mul.o 2024-07-31T15:05:28.846 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/itmt.o 2024-07-31T15:05:29.040 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/read.o 2024-07-31T15:05:29.225 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/mmap.o 2024-07-31T15:05:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:29 smithi077 ceph-mon[37327]: pgmap v854: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 706 KiB/s wr, 137 op/s 2024-07-31T15:05:29.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:29 smithi067 ceph-mon[32750]: pgmap v854: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 706 KiB/s wr, 137 op/s 2024-07-31T15:05:29.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:05:29.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:29 smithi073 ceph-mon[36714]: pgmap v854: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 706 KiB/s wr, 137 op/s 2024-07-31T15:05:29.698 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpih-cmp.o 2024-07-31T15:05:29.822 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/setup-irq.o 2024-07-31T15:05:30.052 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/proc.o 2024-07-31T15:05:30.208 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/hrtimer.o 2024-07-31T15:05:30.559 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:05:30.560 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpih-div.o 2024-07-31T15:05:30.898 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/umip.o 2024-07-31T15:05:30.902 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/unwind_orc.o 2024-07-31T15:05:30.905 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/slot.o 2024-07-31T15:05:30.912 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/pci-acpi.o 2024-07-31T15:05:31.044 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/symlink.o 2024-07-31T15:05:31.177 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/pcm.o 2024-07-31T15:05:31.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:31 smithi067 ceph-mon[32750]: pgmap v855: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 400 KiB/s wr, 94 op/s 2024-07-31T15:05:31.330 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/exit.o 2024-07-31T15:05:31.484 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpih-mul.o 2024-07-31T15:05:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:31 smithi077 ceph-mon[37327]: pgmap v855: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 400 KiB/s wr, 94 op/s 2024-07-31T15:05:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:31 smithi073 ceph-mon[36714]: pgmap v855: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 400 KiB/s wr, 94 op/s 2024-07-31T15:05:31.739 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/callthunks.o 2024-07-31T15:05:31.771 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/quirks.o 2024-07-31T15:05:31.782 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/ats.o 2024-07-31T15:05:31.989 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/pci-label.o 2024-07-31T15:05:32.048 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/unlink.o 2024-07-31T15:05:32.117 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/pcm_native.o 2024-07-31T15:05:32.174 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-pow.o 2024-07-31T15:05:32.474 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpiutil.o 2024-07-31T15:05:32.515 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/audit_64.o 2024-07-31T15:05:32.610 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pci/vgaarb.o 2024-07-31T15:05:32.878 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/softirq.o 2024-07-31T15:05:32.899 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/exportfs/expfs.o 2024-07-31T15:05:33.119 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/mmconf-fam10h_64.o 2024-07-31T15:05:33.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.135+0000 7f217f59e640 1 -- 172.21.15.67:0/858673072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2180114d90 msgr2=0x7f2180117280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:33.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.135+0000 7f217f59e640 1 --2- 172.21.15.67:0/858673072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2180114d90 0x7f2180117280 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f2178009ca0 tx=0x7f217802a8f0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.136+0000 7f217f59e640 1 -- 172.21.15.67:0/858673072 shutdown_connections 2024-07-31T15:05:33.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.136+0000 7f217f59e640 1 --2- 172.21.15.67:0/858673072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2180114d90 0x7f2180117280 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.136+0000 7f217f59e640 1 --2- 172.21.15.67:0/858673072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f218010c2b0 0x7f2180114800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.136+0000 7f217f59e640 1 --2- 172.21.15.67:0/858673072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f218010b960 0x7f218010bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.136+0000 7f217f59e640 1 -- 172.21.15.67:0/858673072 >> 172.21.15.67:0/858673072 conn(0x7f218006e880 msgr2=0x7f218006ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:33.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.136+0000 7f217f59e640 1 -- 172.21.15.67:0/858673072 shutdown_connections 2024-07-31T15:05:33.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.136+0000 7f217f59e640 1 -- 172.21.15.67:0/858673072 wait complete. 2024-07-31T15:05:33.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.138+0000 7f217f59e640 1 Processor -- start 2024-07-31T15:05:33.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.138+0000 7f217f59e640 1 -- start start 2024-07-31T15:05:33.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.139+0000 7f217f59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f218010b960 0x7f21801a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.139+0000 7f217f59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f218010c2b0 0x7f21801a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.140+0000 7f217f59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2180114d90 0x7f21801a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.140+0000 7f217f59e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f218011aa50 con 0x7f218010c2b0 2024-07-31T15:05:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.140+0000 7f217dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f218010c2b0 0x7f21801a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.140+0000 7f217dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f218010c2b0 0x7f21801a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51482/0 (socket says 172.21.15.67:51482) 2024-07-31T15:05:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.140+0000 7f217dd9b640 1 -- 172.21.15.67:0/1179443253 learned_addr learned my addr 172.21.15.67:0/1179443253 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.140+0000 7f217f59e640 1 -- 172.21.15.67:0/1179443253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f218011a8d0 con 0x7f218010b960 2024-07-31T15:05:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.140+0000 7f217f59e640 1 -- 172.21.15.67:0/1179443253 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f218011abd0 con 0x7f2180114d90 2024-07-31T15:05:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.140+0000 7f217ed9d640 1 --2- 172.21.15.67:0/1179443253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2180114d90 0x7f21801a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.141+0000 7f217e59c640 1 --2- 172.21.15.67:0/1179443253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f218010b960 0x7f21801a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.141+0000 7f217dd9b640 1 -- 172.21.15.67:0/1179443253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2180114d90 msgr2=0x7f21801a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:33.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.141+0000 7f217dd9b640 1 --2- 172.21.15.67:0/1179443253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2180114d90 0x7f21801a6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.141+0000 7f217dd9b640 1 -- 172.21.15.67:0/1179443253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f218010b960 msgr2=0x7f21801a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:33.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.141+0000 7f217dd9b640 1 --2- 172.21.15.67:0/1179443253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f218010b960 0x7f21801a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.141+0000 7f217dd9b640 1 -- 172.21.15.67:0/1179443253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21801a7100 con 0x7f218010c2b0 2024-07-31T15:05:33.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.141+0000 7f217dd9b640 1 --2- 172.21.15.67:0/1179443253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f218010c2b0 0x7f21801a17f0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f217400b450 tx=0x7f217400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:33.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.143+0000 7f216f7fe640 1 -- 172.21.15.67:0/1179443253 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217401d070 con 0x7f218010c2b0 2024-07-31T15:05:33.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.143+0000 7f216f7fe640 1 -- 172.21.15.67:0/1179443253 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2174018440 con 0x7f218010c2b0 2024-07-31T15:05:33.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.143+0000 7f216f7fe640 1 -- 172.21.15.67:0/1179443253 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174016560 con 0x7f218010c2b0 2024-07-31T15:05:33.145 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/mpi/built-in.a 2024-07-31T15:05:33.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.144+0000 7f217f59e640 1 -- 172.21.15.67:0/1179443253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21801a73f0 con 0x7f218010c2b0 2024-07-31T15:05:33.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.144+0000 7f217f59e640 1 -- 172.21.15.67:0/1179443253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f218010a680 con 0x7f218010c2b0 2024-07-31T15:05:33.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.151+0000 7f218b2fd740 1 Processor -- start 2024-07-31T15:05:33.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.153+0000 7f218b2fd740 1 -- start start 2024-07-31T15:05:33.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.154+0000 7f218b2fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 0x5562687546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.154+0000 7f218b2fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5562683861b0 0x556268754c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.154+0000 7f218b2fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55626838e390 0x556268755140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.154+0000 7f218b2fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5562686cab60 con 0x5562683861b0 2024-07-31T15:05:33.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.154+0000 7f218b2fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5562686ba060 con 0x55626838e390 2024-07-31T15:05:33.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.154+0000 7f217e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5562683861b0 0x556268754c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.154+0000 7f217e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5562683861b0 0x556268754c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51492/0 (socket says 172.21.15.67:51492) 2024-07-31T15:05:33.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.154+0000 7f217e59c640 1 -- 172.21.15.67:0/531927808 learned_addr learned my addr 172.21.15.67:0/531927808 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:33.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.154+0000 7f217ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 0x5562687546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.155+0000 7f218b2fd740 1 -- 172.21.15.67:0/531927808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5562686be0a0 con 0x5562683af070 2024-07-31T15:05:33.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.155+0000 7f217e59c640 1 -- 172.21.15.67:0/531927808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 msgr2=0x5562687546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:33.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.155+0000 7f217e59c640 1 --2- 172.21.15.67:0/531927808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 0x5562687546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.155+0000 7f217e59c640 1 -- 172.21.15.67:0/531927808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55626838e390 msgr2=0x556268755140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:33.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.155+0000 7f217e59c640 1 --2- 172.21.15.67:0/531927808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55626838e390 0x556268755140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.155+0000 7f217e59c640 1 -- 172.21.15.67:0/531927808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556268660b40 con 0x5562683861b0 2024-07-31T15:05:33.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.156+0000 7f217e59c640 1 --2- 172.21.15.67:0/531927808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5562683861b0 0x556268754c00 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f217000bb10 tx=0x7f217000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2184c994a389e81b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:33.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.156+0000 7f216f7fe640 1 -- 172.21.15.67:0/1179443253 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2174020070 con 0x7f218010c2b0 2024-07-31T15:05:33.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.156+0000 7f216cff9640 1 -- 172.21.15.67:0/531927808 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2170017420 con 0x5562683861b0 2024-07-31T15:05:33.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.157+0000 7f216cff9640 1 -- 172.21.15.67:0/531927808 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2170017a70 con 0x5562683861b0 2024-07-31T15:05:33.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.159+0000 7f218b2fd740 1 -- 172.21.15.67:0/531927808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5562683861b0 msgr2=0x556268754c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:33.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.159+0000 7f218b2fd740 1 --2- 172.21.15.67:0/531927808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5562683861b0 0x556268754c00 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f217000bb10 tx=0x7f217000d340 comp rx=0 tx=0).stop 2024-07-31T15:05:33.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.159+0000 7f216f7fe640 1 --2- 172.21.15.67:0/1179443253 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f21680b3f70 0x7f21680b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.159+0000 7f218b2fd740 1 -- 172.21.15.67:0/531927808 shutdown_connections 2024-07-31T15:05:33.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.159+0000 7f218b2fd740 1 --2- 172.21.15.67:0/531927808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55626838e390 0x556268755140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.159+0000 7f218b2fd740 1 --2- 172.21.15.67:0/531927808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5562683861b0 0x556268754c00 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.159+0000 7f218b2fd740 1 --2- 172.21.15.67:0/531927808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 0x5562687546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.159+0000 7f218b2fd740 1 -- 172.21.15.67:0/531927808 >> 172.21.15.67:0/531927808 conn(0x5562683a3460 msgr2=0x5562687525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:33.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.159+0000 7f217ed9d640 1 --2- 172.21.15.67:0/1179443253 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f21680b3f70 0x7f21680b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.160+0000 7f216f7fe640 1 -- 172.21.15.67:0/1179443253 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f21740e7080 con 0x7f218010c2b0 2024-07-31T15:05:33.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.160+0000 7f217ed9d640 1 --2- 172.21.15.67:0/1179443253 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f21680b3f70 0x7f21680b6480 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2178009620 tx=0x7f217803c040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:33.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.161+0000 7f218b2fd740 1 -- 172.21.15.67:0/531927808 shutdown_connections 2024-07-31T15:05:33.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.163+0000 7f218b2fd740 1 -- 172.21.15.67:0/531927808 wait complete. 2024-07-31T15:05:33.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.163+0000 7f218b2fd740 1 -- start start 2024-07-31T15:05:33.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.163+0000 7f218b2fd740 1 Processor -- start 2024-07-31T15:05:33.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.164+0000 7f218b2fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5562683861b0 0x5562687609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.164+0000 7f218b2fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55626838e390 0x556268760f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.164+0000 7f218b2fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 0x556268765fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.164+0000 7f218b2fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55626851d700 con 0x5562683861b0 2024-07-31T15:05:33.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.164+0000 7f218b2fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55626865c990 con 0x55626838e390 2024-07-31T15:05:33.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.164+0000 7f218b2fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55626875adc0 con 0x5562683af070 2024-07-31T15:05:33.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.164+0000 7f218b2fd740 10 client.0 _ll_register_callbacks cb 0x5562686774a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:33.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.165+0000 7f217e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5562683861b0 0x5562687609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.165+0000 7f217e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5562683861b0 0x5562687609a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51494/0 (socket says 172.21.15.67:51494) 2024-07-31T15:05:33.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.165+0000 7f217e59c640 1 -- 172.21.15.67:0/2560411578 learned_addr learned my addr 172.21.15.67:0/2560411578 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:33.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.165+0000 7f217e59c640 1 -- 172.21.15.67:0/2560411578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 msgr2=0x556268765fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:05:33.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.165+0000 7f217e59c640 1 --2- 172.21.15.67:0/2560411578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 0x556268765fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.165+0000 7f217e59c640 1 -- 172.21.15.67:0/2560411578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55626838e390 msgr2=0x556268760f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:05:33.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.165+0000 7f217e59c640 1 --2- 172.21.15.67:0/2560411578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55626838e390 0x556268760f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.165+0000 7f217e59c640 1 -- 172.21.15.67:0/2560411578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55626841a650 con 0x5562683861b0 2024-07-31T15:05:33.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.166+0000 7f217e59c640 1 --2- 172.21.15.67:0/2560411578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5562683861b0 0x5562687609a0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f217000f6e0 tx=0x7f2170004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3ea365cbb9d4c1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:33.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.166+0000 7f2165ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:05:33.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.166+0000 7f2165ffb640 1 -- 172.21.15.67:0/2560411578 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2170026070 con 0x5562683861b0 2024-07-31T15:05:33.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.166+0000 7f218b2fd740 1 -- 172.21.15.67:0/2560411578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2170017a70 con 0x5562683861b0 2024-07-31T15:05:33.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.166+0000 7f2165ffb640 1 -- 172.21.15.67:0/2560411578 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f2170017620 con 0x5562683861b0 2024-07-31T15:05:33.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.167+0000 7f2165ffb640 1 -- 172.21.15.67:0/2560411578 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f2170032050 con 0x5562683861b0 2024-07-31T15:05:33.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.167+0000 7f218b2fd740 10 client.18528 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:33.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.167+0000 7f218b2fd740 1 -- 172.21.15.67:0/2560411578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55626877fef0 con 0x5562683861b0 2024-07-31T15:05:33.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.168+0000 7f2165ffb640 1 -- 172.21.15.67:0/2560411578 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2170017a70 con 0x5562683861b0 2024-07-31T15:05:33.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.168+0000 7f2165ffb640 1 -- 172.21.15.67:0/2560411578 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f2170020410 con 0x5562683861b0 2024-07-31T15:05:33.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.168+0000 7f218b2fd740 10 client.18528 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:33.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.168+0000 7f218b2fd740 10 client.18528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:05:33.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.168+0000 7f218b2fd740 20 client.18528 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:33.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.168+0000 7f218b2fd740 1 --2- 172.21.15.67:0/2560411578 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55626836d9f0 0x556268783820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.168+0000 7f218b2fd740 4 client.18528 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:05:33.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.168+0000 7f218b2fd740 1 -- 172.21.15.67:0/2560411578 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562686ccab0 con 0x55626836d9f0 2024-07-31T15:05:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.170+0000 7f217dd9b640 1 --2- 172.21.15.67:0/2560411578 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55626836d9f0 0x556268783820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.171+0000 7f217dd9b640 1 --2- 172.21.15.67:0/2560411578 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55626836d9f0 0x556268783820 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e67c31562d2c58e server_cookie=ccc903818ee9aa7b in_seq=0 out_seq=0 2024-07-31T15:05:33.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.171+0000 7f2165ffb640 10 client.18528 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:33.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.173+0000 7f2165ffb640 1 -- 172.21.15.67:0/2560411578 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562686ccab0 con 0x55626836d9f0 2024-07-31T15:05:33.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.173+0000 7f2165ffb640 10 client.18528 handle_command_reply: tid=0 2024-07-31T15:05:33.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.173+0000 7f218b2fd740 1 client.18528 shutdown 2024-07-31T15:05:33.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.174+0000 7f218b2fd740 1 -- 172.21.15.67:0/2560411578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5562683861b0 msgr2=0x5562687609a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:33.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.174+0000 7f218b2fd740 1 --2- 172.21.15.67:0/2560411578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5562683861b0 0x5562687609a0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f217000f6e0 tx=0x7f2170004710 comp rx=0 tx=0).stop 2024-07-31T15:05:33.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.175+0000 7f218b2fd740 1 -- 172.21.15.67:0/2560411578 shutdown_connections 2024-07-31T15:05:33.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.175+0000 7f218b2fd740 1 --2- 172.21.15.67:0/2560411578 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55626836d9f0 0x556268783820 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.175+0000 7f218b2fd740 1 --2- 172.21.15.67:0/2560411578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 0x556268765fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.175+0000 7f218b2fd740 1 --2- 172.21.15.67:0/2560411578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55626838e390 0x556268760f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.175+0000 7f218b2fd740 1 --2- 172.21.15.67:0/2560411578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5562683861b0 0x5562687609a0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.175+0000 7f218b2fd740 1 -- 172.21.15.67:0/2560411578 >> 172.21.15.67:0/2560411578 conn(0x5562683a3460 msgr2=0x556268750bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:33.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.175+0000 7f2165ffb640 0 client.18528 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:33.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.176+0000 7f218b2fd740 1 -- 172.21.15.67:0/2560411578 shutdown_connections 2024-07-31T15:05:33.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.176+0000 7f218b2fd740 1 -- 172.21.15.67:0/2560411578 wait complete. 2024-07-31T15:05:33.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.177+0000 7f218b2fd740 20 client.18528 trim_cache size 0 max 16384 2024-07-31T15:05:33.222 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:33 smithi067 ceph-mon[32750]: pgmap v856: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 856 KiB/s wr, 82 op/s 2024-07-31T15:05:33.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.220+0000 7f218b2fd740 1 Processor -- start 2024-07-31T15:05:33.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.220+0000 7f218b2fd740 1 -- start start 2024-07-31T15:05:33.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.221+0000 7f218b2fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562683861b0 0x55626875f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.221+0000 7f218b2fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55626838e390 0x556268751ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.221+0000 7f218b2fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 0x556268752410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.221+0000 7f218b2fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f216000ffa0 con 0x55626838e390 2024-07-31T15:05:33.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.221+0000 7f218b2fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2160011ae0 con 0x5562683861b0 2024-07-31T15:05:33.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.221+0000 7f217e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562683861b0 0x55626875f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.221+0000 7f218b2fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2160010670 con 0x5562683af070 2024-07-31T15:05:33.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.222+0000 7f217dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55626838e390 0x556268751ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.222+0000 7f217ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 0x556268752410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.222+0000 7f217ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 0x556268752410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38732/0 (socket says 172.21.15.67:38732) 2024-07-31T15:05:33.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.222+0000 7f217dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55626838e390 0x556268751ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51504/0 (socket says 172.21.15.67:51504) 2024-07-31T15:05:33.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.222+0000 7f217ed9d640 1 -- 172.21.15.67:0/1793949440 learned_addr learned my addr 172.21.15.67:0/1793949440 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:33.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.222+0000 7f217dd9b640 1 -- 172.21.15.67:0/1793949440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 msgr2=0x556268752410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:33.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.222+0000 7f217dd9b640 1 --2- 172.21.15.67:0/1793949440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 0x556268752410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.222+0000 7f217dd9b640 1 -- 172.21.15.67:0/1793949440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562683861b0 msgr2=0x55626875f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:33.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.222+0000 7f217dd9b640 1 --2- 172.21.15.67:0/1793949440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562683861b0 0x55626875f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.222+0000 7f217dd9b640 1 -- 172.21.15.67:0/1793949440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55626876b080 con 0x55626838e390 2024-07-31T15:05:33.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.223+0000 7f217dd9b640 1 --2- 172.21.15.67:0/1793949440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55626838e390 0x556268751ed0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f21740e4510 tx=0x7f2174018e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5d78ad1bc818a4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:33.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.223+0000 7f2167fff640 1 -- 172.21.15.67:0/1793949440 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21740ada80 con 0x55626838e390 2024-07-31T15:05:33.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.223+0000 7f2167fff640 1 -- 172.21.15.67:0/1793949440 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21740b1fd0 con 0x55626838e390 2024-07-31T15:05:33.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.224+0000 7f2167fff640 1 -- 172.21.15.67:0/1793949440 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21740b65f0 con 0x55626838e390 2024-07-31T15:05:33.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.225+0000 7f218b2fd740 1 -- 172.21.15.67:0/1793949440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55626838e390 msgr2=0x556268751ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:33.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.225+0000 7f218b2fd740 1 --2- 172.21.15.67:0/1793949440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55626838e390 0x556268751ed0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f21740e4510 tx=0x7f2174018e20 comp rx=0 tx=0).stop 2024-07-31T15:05:33.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.227+0000 7f218b2fd740 1 -- 172.21.15.67:0/1793949440 shutdown_connections 2024-07-31T15:05:33.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.227+0000 7f218b2fd740 1 --2- 172.21.15.67:0/1793949440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683af070 0x556268752410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.227+0000 7f218b2fd740 1 --2- 172.21.15.67:0/1793949440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55626838e390 0x556268751ed0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.227+0000 7f218b2fd740 1 --2- 172.21.15.67:0/1793949440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562683861b0 0x55626875f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.227+0000 7f218b2fd740 1 -- 172.21.15.67:0/1793949440 >> 172.21.15.67:0/1793949440 conn(0x5562683a3460 msgr2=0x55626875b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:33.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.230+0000 7f218b2fd740 1 -- 172.21.15.67:0/1793949440 shutdown_connections 2024-07-31T15:05:33.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.230+0000 7f218b2fd740 1 -- 172.21.15.67:0/1793949440 wait complete. 2024-07-31T15:05:33.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.230+0000 7f218b2fd740 1 -- start start 2024-07-31T15:05:33.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.231+0000 7f218b2fd740 1 Processor -- start 2024-07-31T15:05:33.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.231+0000 7f218b2fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683861b0 0x556268792740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.232+0000 7f218b2fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55626838e390 0x556268792ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.232+0000 7f217e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683861b0 0x556268792740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.232+0000 7f217e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683861b0 0x556268792740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38744/0 (socket says 172.21.15.67:38744) 2024-07-31T15:05:33.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.232+0000 7f217e59c640 1 -- 172.21.15.67:0/2890477476 learned_addr learned my addr 172.21.15.67:0/2890477476 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:33.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.232+0000 7f218b2fd740 1 --2- 172.21.15.67:0/2890477476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562683af070 0x5562687e2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.232+0000 7f217dd9b640 1 --2- 172.21.15.67:0/2890477476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55626838e390 0x556268792ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.233+0000 7f217ed9d640 1 --2- 172.21.15.67:0/2890477476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562683af070 0x5562687e2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.232+0000 7f218b2fd740 1 -- 172.21.15.67:0/2890477476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2160011410 con 0x55626838e390 2024-07-31T15:05:33.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.233+0000 7f218b2fd740 1 -- 172.21.15.67:0/2890477476 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2160010d40 con 0x5562683af070 2024-07-31T15:05:33.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.233+0000 7f218b2fd740 1 -- 172.21.15.67:0/2890477476 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f216000dfa0 con 0x5562683861b0 2024-07-31T15:05:33.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.233+0000 7f218b2fd740 10 client.0 _ll_register_callbacks cb 0x7f217000b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:33.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.239+0000 7f217dd9b640 1 -- 172.21.15.67:0/2890477476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683861b0 msgr2=0x556268792740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:33.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.239+0000 7f217dd9b640 1 --2- 172.21.15.67:0/2890477476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683861b0 0x556268792740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.239+0000 7f217dd9b640 1 -- 172.21.15.67:0/2890477476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562683af070 msgr2=0x5562687e2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:33.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.239+0000 7f217dd9b640 1 --2- 172.21.15.67:0/2890477476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562683af070 0x5562687e2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.239+0000 7f217dd9b640 1 -- 172.21.15.67:0/2890477476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562687e3510 con 0x55626838e390 2024-07-31T15:05:33.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.240+0000 7f217dd9b640 1 --2- 172.21.15.67:0/2890477476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55626838e390 0x556268792ce0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x556268746b80 tx=0x7f21740a9f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49ff1b7eef0c3d6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.241+0000 7f2165ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:05:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.243+0000 7f2165ffb640 1 -- 172.21.15.67:0/2890477476 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21740e4890 con 0x55626838e390 2024-07-31T15:05:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.243+0000 7f2165ffb640 1 -- 172.21.15.67:0/2890477476 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f21740b65f0 con 0x55626838e390 2024-07-31T15:05:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.243+0000 7f218b2fd740 1 -- 172.21.15.67:0/2890477476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21740b1fd0 con 0x55626838e390 2024-07-31T15:05:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.243+0000 7f2165ffb640 1 -- 172.21.15.67:0/2890477476 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f21740ada80 con 0x55626838e390 2024-07-31T15:05:33.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.245+0000 7f218b2fd740 10 client.18534 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:33.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.245+0000 7f218b2fd740 1 -- 172.21.15.67:0/2890477476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5562687eba00 con 0x55626838e390 2024-07-31T15:05:33.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.245+0000 7f2165ffb640 1 -- 172.21.15.67:0/2890477476 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21740b67f0 con 0x55626838e390 2024-07-31T15:05:33.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.245+0000 7f2165ffb640 1 -- 172.21.15.67:0/2890477476 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f21740a93f0 con 0x55626838e390 2024-07-31T15:05:33.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.245+0000 7f218b2fd740 10 client.18534 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:33.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.246+0000 7f218b2fd740 10 client.18534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:05:33.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.246+0000 7f218b2fd740 20 client.18534 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:33.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.246+0000 7f218b2fd740 1 --2- 172.21.15.67:0/2890477476 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556268342da0 0x5562687edf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:33.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.246+0000 7f218b2fd740 4 client.18534 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:05:33.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.246+0000 7f218b2fd740 1 -- 172.21.15.67:0/2890477476 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f21740b1fd0 con 0x556268342da0 2024-07-31T15:05:33.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.246+0000 7f217e59c640 1 --2- 172.21.15.67:0/2890477476 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556268342da0 0x5562687edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:33.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.248+0000 7f217e59c640 1 --2- 172.21.15.67:0/2890477476 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556268342da0 0x5562687edf70 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1e7825fa4a5f5e7 server_cookie=34a4799361c2586f in_seq=0 out_seq=0 2024-07-31T15:05:33.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.248+0000 7f2165ffb640 10 client.18534 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.249+0000 7f2165ffb640 1 -- 172.21.15.67:0/2890477476 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f21740b1fd0 con 0x556268342da0 2024-07-31T15:05:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.249+0000 7f2165ffb640 10 client.18534 handle_command_reply: tid=0 2024-07-31T15:05:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.249+0000 7f218b2fd740 1 client.18534 shutdown 2024-07-31T15:05:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.250+0000 7f218b2fd740 1 -- 172.21.15.67:0/2890477476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55626838e390 msgr2=0x556268792ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.250+0000 7f218b2fd740 1 --2- 172.21.15.67:0/2890477476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55626838e390 0x556268792ce0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x556268746b80 tx=0x7f21740a9f20 comp rx=0 tx=0).stop 2024-07-31T15:05:33.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.252+0000 7f218b2fd740 1 -- 172.21.15.67:0/2890477476 shutdown_connections 2024-07-31T15:05:33.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.252+0000 7f218b2fd740 1 --2- 172.21.15.67:0/2890477476 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556268342da0 0x5562687edf70 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.252+0000 7f2165ffb640 0 client.18534 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:33.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.252+0000 7f218b2fd740 1 --2- 172.21.15.67:0/2890477476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562683af070 0x5562687e2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.252+0000 7f218b2fd740 1 --2- 172.21.15.67:0/2890477476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55626838e390 0x556268792ce0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.252+0000 7f218b2fd740 1 --2- 172.21.15.67:0/2890477476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562683861b0 0x556268792740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.253+0000 7f218b2fd740 1 -- 172.21.15.67:0/2890477476 >> 172.21.15.67:0/2890477476 conn(0x5562683a3460 msgr2=0x55626875ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:33.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.253+0000 7f218b2fd740 1 -- 172.21.15.67:0/2890477476 shutdown_connections 2024-07-31T15:05:33.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.253+0000 7f218b2fd740 1 -- 172.21.15.67:0/2890477476 wait complete. 2024-07-31T15:05:33.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.254+0000 7f218b2fd740 20 client.18534 trim_cache size 0 max 16384 2024-07-31T15:05:33.256 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:05:33.256 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T15:05:33.256 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T15:05:33.256 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:05:33.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.255+0000 7f217f59e640 1 -- 172.21.15.67:0/1179443253 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f21680b3f70 msgr2=0x7f21680b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:33.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.255+0000 7f217f59e640 1 --2- 172.21.15.67:0/1179443253 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f21680b3f70 0x7f21680b6480 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2178009620 tx=0x7f217803c040 comp rx=0 tx=0).stop 2024-07-31T15:05:33.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.256+0000 7f217f59e640 1 -- 172.21.15.67:0/1179443253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f218010c2b0 msgr2=0x7f21801a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:33.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.256+0000 7f217f59e640 1 --2- 172.21.15.67:0/1179443253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f218010c2b0 0x7f21801a17f0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f217400b450 tx=0x7f217400b910 comp rx=0 tx=0).stop 2024-07-31T15:05:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.261+0000 7f217f59e640 1 -- 172.21.15.67:0/1179443253 shutdown_connections 2024-07-31T15:05:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.261+0000 7f217f59e640 1 --2- 172.21.15.67:0/1179443253 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f21680b3f70 0x7f21680b6480 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.261+0000 7f217f59e640 1 --2- 172.21.15.67:0/1179443253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2180114d90 0x7f21801a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.261+0000 7f217f59e640 1 --2- 172.21.15.67:0/1179443253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f218010c2b0 0x7f21801a17f0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.261+0000 7f217f59e640 1 --2- 172.21.15.67:0/1179443253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f218010b960 0x7f21801a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.261+0000 7f217f59e640 1 -- 172.21.15.67:0/1179443253 >> 172.21.15.67:0/1179443253 conn(0x7f218006e880 msgr2=0x7f218010c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.262+0000 7f217f59e640 1 -- 172.21.15.67:0/1179443253 shutdown_connections 2024-07-31T15:05:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:33.262+0000 7f217f59e640 1 -- 172.21.15.67:0/1179443253 wait complete. 2024-07-31T15:05:33.421 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/buildid.o 2024-07-31T15:05:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:33 smithi077 ceph-mon[37327]: pgmap v856: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 856 KiB/s wr, 82 op/s 2024-07-31T15:05:33.506 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/write.o 2024-07-31T15:05:33.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:33 smithi073 ceph-mon[36714]: pgmap v856: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 856 KiB/s wr, 82 op/s 2024-07-31T15:05:33.684 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/namespace.o 2024-07-31T15:05:33.767 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pci/built-in.a 2024-07-31T15:05:33.901 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/idle/built-in.a 2024-07-31T15:05:33.962 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/vsmp_64.o 2024-07-31T15:05:33.989 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/char/ipmi/built-in.a 2024-07-31T15:05:34.163 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T15:05:34.163 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:05:34.341 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/exportfs/built-in.a 2024-07-31T15:05:34.409 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/resource.o 2024-07-31T15:05:34.419 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dsargs.o 2024-07-31T15:05:34.561 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/pcm_lib.o 2024-07-31T15:05:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.643+0000 7f4fd97fa640 1 -- 172.21.15.67:0/3054354920 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566a0b37da0 con 0x5566a0c5bf30 2024-07-31T15:05:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.643+0000 7f4fd97fa640 10 client.28012 handle_command_reply: tid=0 2024-07-31T15:05:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.643+0000 7f4ffec54740 1 client.28012 shutdown 2024-07-31T15:05:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.643+0000 7f4ffec54740 1 -- 172.21.15.67:0/3054354920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08221b0 msgr2=0x5566a0be1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.643+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3054354920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08221b0 0x5566a0be1760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8004af0 tx=0x7f4fe800a6d0 comp rx=0 tx=0).stop 2024-07-31T15:05:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.643+0000 7f4ffec54740 1 -- 172.21.15.67:0/3054354920 shutdown_connections 2024-07-31T15:05:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.643+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3054354920 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5566a0c5bf30 0x5566a0c5e350 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.643+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3054354920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566a08221b0 0x5566a0be1760 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.643+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3054354920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566a08014d0 0x5566a0be1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.643+0000 7f4ffec54740 1 --2- 172.21.15.67:0/3054354920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566a07b61c0 0x5566a0bdb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.643+0000 7f4ffec54740 1 -- 172.21.15.67:0/3054354920 >> 172.21.15.67:0/3054354920 conn(0x5566a0816300 msgr2=0x5566a0bcffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.643+0000 7f4fd97fa640 0 client.28012 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.643+0000 7f4ffec54740 1 -- 172.21.15.67:0/3054354920 shutdown_connections 2024-07-31T15:05:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.643+0000 7f4ffec54740 1 -- 172.21.15.67:0/3054354920 wait complete. 2024-07-31T15:05:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.645+0000 7f4ffec54740 20 client.28012 trim_cache size 0 max 16384 2024-07-31T15:05:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:05:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:05:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:05:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:05:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.652+0000 7f4ff7d5e640 1 -- 172.21.15.67:0/957007108 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4fdc0b3ef0 msgr2=0x7f4fdc0b6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.652+0000 7f4ff7d5e640 1 --2- 172.21.15.67:0/957007108 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4fdc0b3ef0 0x7f4fdc0b6400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8033ff0 tx=0x7f4fe8004a80 comp rx=0 tx=0).stop 2024-07-31T15:05:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.652+0000 7f4ff7d5e640 1 -- 172.21.15.67:0/957007108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ff010a920 msgr2=0x7f4ff01a0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.652+0000 7f4ff7d5e640 1 --2- 172.21.15.67:0/957007108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ff010a920 0x7f4ff01a0040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00e9c0 tx=0x7f4fec00ee80 comp rx=0 tx=0).stop 2024-07-31T15:05:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.655+0000 7f4ff7d5e640 1 -- 172.21.15.67:0/957007108 shutdown_connections 2024-07-31T15:05:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.655+0000 7f4ff7d5e640 1 --2- 172.21.15.67:0/957007108 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4fdc0b3ef0 0x7f4fdc0b6400 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.655+0000 7f4ff7d5e640 1 --2- 172.21.15.67:0/957007108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff010d230 0x7f4ff01a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.655+0000 7f4ff7d5e640 1 --2- 172.21.15.67:0/957007108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ff010c8e0 0x7f4ff01a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.655+0000 7f4ff7d5e640 1 --2- 172.21.15.67:0/957007108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ff010a920 0x7f4ff01a0040 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.655+0000 7f4ff7d5e640 1 -- 172.21.15.67:0/957007108 >> 172.21.15.67:0/957007108 conn(0x7f4ff006e350 msgr2=0x7f4ff010bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.656+0000 7f4ff7d5e640 1 -- 172.21.15.67:0/957007108 shutdown_connections 2024-07-31T15:05:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:34.656+0000 7f4ff7d5e640 1 -- 172.21.15.67:0/957007108 wait complete. 2024-07-31T15:05:34.729 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/clz_tab.o 2024-07-31T15:05:34.828 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pnp/pnpacpi/core.o 2024-07-31T15:05:34.831 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/cmdline.o 2024-07-31T15:05:34.867 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/kernel/built-in.a 2024-07-31T15:05:35.076 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/cpumask.o 2024-07-31T15:05:35.144 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dscontrol.o 2024-07-31T15:05:35.148 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:05:35.314 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/built-in.a 2024-07-31T15:05:35.444 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:35 smithi073 ceph-mon[36714]: pgmap v857: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-07-31T15:05:35.481 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:35 smithi067 ceph-mon[32750]: pgmap v857: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-07-31T15:05:35.484 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pnp/pnpacpi/rsparser.o 2024-07-31T15:05:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:35 smithi077 ceph-mon[37327]: pgmap v857: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-07-31T15:05:35.504 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dsdebug.o 2024-07-31T15:05:35.535 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/mount_clnt.o 2024-07-31T15:05:35.575 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/sysctl.o 2024-07-31T15:05:35.729 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/sock.o 2024-07-31T15:05:35.780 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/ctype.o 2024-07-31T15:05:35.829 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/dec_and_lock.o 2024-07-31T15:05:35.858 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dsfield.o 2024-07-31T15:05:36.025 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress.o 2024-07-31T15:05:36.060 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/pcm_misc.o 2024-07-31T15:05:36.108 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress_bunzip2.o 2024-07-31T15:05:36.298 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/request_sock.o 2024-07-31T15:05:36.302 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dsinit.o 2024-07-31T15:05:36.361 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pnp/pnpacpi/built-in.a 2024-07-31T15:05:36.424 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pnp/core.o 2024-07-31T15:05:36.558 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfstrace.o 2024-07-31T15:05:36.652 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/pcm_memory.o 2024-07-31T15:05:36.675 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dsmethod.o 2024-07-31T15:05:36.803 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress_inflate.o 2024-07-31T15:05:37.007 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pnp/card.o 2024-07-31T15:05:37.088 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/capability.o 2024-07-31T15:05:37.138 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dsmthdat.o 2024-07-31T15:05:37.141 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/clntlock.o 2024-07-31T15:05:37.344 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress_unlz4.o 2024-07-31T15:05:37.386 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/memalloc.o 2024-07-31T15:05:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:37 smithi077 ceph-mon[37327]: pgmap v858: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-07-31T15:05:37.495 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:37 smithi067 ceph-mon[32750]: pgmap v858: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-07-31T15:05:37.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.492+0000 7f72df1ed640 1 -- 172.21.15.67:0/502869419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72d8111770 msgr2=0x7f72d810a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:37.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.492+0000 7f72df1ed640 1 --2- 172.21.15.67:0/502869419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72d8111770 0x7f72d810a070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009ca0 tx=0x7f72d002a8f0 comp rx=0 tx=0).stop 2024-07-31T15:05:37.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.502+0000 7f72df1ed640 1 -- 172.21.15.67:0/502869419 shutdown_connections 2024-07-31T15:05:37.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.502+0000 7f72df1ed640 1 --2- 172.21.15.67:0/502869419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d811c5d0 0x7f72d811e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.502+0000 7f72df1ed640 1 --2- 172.21.15.67:0/502869419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72d810a5b0 0x7f72d810a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.502+0000 7f72df1ed640 1 --2- 172.21.15.67:0/502869419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72d8111770 0x7f72d810a070 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.502+0000 7f72df1ed640 1 -- 172.21.15.67:0/502869419 >> 172.21.15.67:0/502869419 conn(0x7f72d806d380 msgr2=0x7f72d806d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:37.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.510+0000 7f72df1ed640 1 -- 172.21.15.67:0/502869419 shutdown_connections 2024-07-31T15:05:37.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.511+0000 7f72df1ed640 1 -- 172.21.15.67:0/502869419 wait complete. 2024-07-31T15:05:37.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.512+0000 7f72df1ed640 1 Processor -- start 2024-07-31T15:05:37.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.512+0000 7f72df1ed640 1 -- start start 2024-07-31T15:05:37.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.513+0000 7f72df1ed640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d810a5b0 0x7f72d81a8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:37.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.513+0000 7f72df1ed640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72d8111770 0x7f72d81a90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:37.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.513+0000 7f72df1ed640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72d811c5d0 0x7f72d81ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:37.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.513+0000 7f72df1ed640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f72d8122150 con 0x7f72d810a5b0 2024-07-31T15:05:37.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.513+0000 7f72df1ed640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f72d8121fd0 con 0x7f72d811c5d0 2024-07-31T15:05:37.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.513+0000 7f72df1ed640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f72d81222d0 con 0x7f72d8111770 2024-07-31T15:05:37.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.517+0000 7f72d7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72d8111770 0x7f72d81a90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.521+0000 7f72d7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72d8111770 0x7f72d81a90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38754/0 (socket says 172.21.15.67:38754) 2024-07-31T15:05:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.521+0000 7f72d7fff640 1 -- 172.21.15.67:0/2263159187 learned_addr learned my addr 172.21.15.67:0/2263159187 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.524+0000 7f72dd78a640 1 --2- 172.21.15.67:0/2263159187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72d811c5d0 0x7f72d81ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.524+0000 7f72dcf89640 1 --2- 172.21.15.67:0/2263159187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d810a5b0 0x7f72d81a8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:37.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.529+0000 7f72dd78a640 1 -- 172.21.15.67:0/2263159187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72d8111770 msgr2=0x7f72d81a90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:37.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.529+0000 7f72dd78a640 1 --2- 172.21.15.67:0/2263159187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72d8111770 0x7f72d81a90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:37.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.529+0000 7f72dd78a640 1 -- 172.21.15.67:0/2263159187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d810a5b0 msgr2=0x7f72d81a8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:37.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.529+0000 7f72dd78a640 1 --2- 172.21.15.67:0/2263159187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d810a5b0 0x7f72d81a8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:37.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.529+0000 7f72dd78a640 1 -- 172.21.15.67:0/2263159187 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d81aea60 con 0x7f72d811c5d0 2024-07-31T15:05:37.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.532+0000 7f72dd78a640 1 --2- 172.21.15.67:0/2263159187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72d811c5d0 0x7f72d81ae360 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d6b0 tx=0x7f72c800db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:37.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.532+0000 7f72d5ffb640 1 -- 172.21.15.67:0/2263159187 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8017070 con 0x7f72d811c5d0 2024-07-31T15:05:37.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.533+0000 7f72df1ed640 1 -- 172.21.15.67:0/2263159187 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d81aed50 con 0x7f72d811c5d0 2024-07-31T15:05:37.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.533+0000 7f72df1ed640 1 -- 172.21.15.67:0/2263159187 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d8111db0 con 0x7f72d811c5d0 2024-07-31T15:05:37.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.533+0000 7f72d5ffb640 1 -- 172.21.15.67:0/2263159187 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72c8009820 con 0x7f72d811c5d0 2024-07-31T15:05:37.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.534+0000 7f72d5ffb640 1 -- 172.21.15.67:0/2263159187 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8004c40 con 0x7f72d811c5d0 2024-07-31T15:05:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.540+0000 7f72d5ffb640 1 -- 172.21.15.67:0/2263159187 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f72c8005220 con 0x7f72d811c5d0 2024-07-31T15:05:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.541+0000 7f72d5ffb640 1 --2- 172.21.15.67:0/2263159187 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f72c00b3f50 0x7f72c00b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.542+0000 7f72d5ffb640 1 -- 172.21.15.67:0/2263159187 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f72c80e3990 con 0x7f72d811c5d0 2024-07-31T15:05:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.542+0000 7f72df1ed640 1 -- 172.21.15.67:0/2263159187 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c40052c0 con 0x7f72d811c5d0 2024-07-31T15:05:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.543+0000 7f72dcf89640 1 --2- 172.21.15.67:0/2263159187 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f72c00b3f50 0x7f72c00b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.544+0000 7f72dcf89640 1 --2- 172.21.15.67:0/2263159187 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f72c00b3f50 0x7f72c00b6460 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f72d0005e30 tx=0x7f72d0005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:37.547+0000 7f72d5ffb640 1 -- 172.21.15.67:0/2263159187 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f72c80aa280 con 0x7f72d811c5d0 2024-07-31T15:05:37.577 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dsobject.o 2024-07-31T15:05:37.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:37 smithi073 ceph-mon[36714]: pgmap v858: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-07-31T15:05:37.720 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pnp/driver.o 2024-07-31T15:05:37.863 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress_unlzma.o 2024-07-31T15:05:37.874 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/ptrace.o 2024-07-31T15:05:38.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:38.066+0000 7f72df1ed640 1 -- 172.21.15.67:0/2263159187 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72c4005880 con 0x7f72d811c5d0 2024-07-31T15:05:38.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:38.071+0000 7f72d5ffb640 1 -- 172.21.15.67:0/2263159187 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+11959 (secure 0 0 0) 0x7f72c80ae7d0 con 0x7f72d811c5d0 2024-07-31T15:05:38.073 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:05:38.074 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":63,"btime":"2024-07-31T15:03:07:099477+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:03:06.671820+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:05:38.074 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 63 2024-07-31T15:05:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:38.076+0000 7f72bf7fe640 1 -- 172.21.15.67:0/2263159187 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f72c00b3f50 msgr2=0x7f72c00b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:38.076+0000 7f72bf7fe640 1 --2- 172.21.15.67:0/2263159187 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f72c00b3f50 0x7f72c00b6460 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f72d0005e30 tx=0x7f72d0005fd0 comp rx=0 tx=0).stop 2024-07-31T15:05:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:38.076+0000 7f72bf7fe640 1 -- 172.21.15.67:0/2263159187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72d811c5d0 msgr2=0x7f72d81ae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:38.076+0000 7f72bf7fe640 1 --2- 172.21.15.67:0/2263159187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72d811c5d0 0x7f72d81ae360 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d6b0 tx=0x7f72c800db70 comp rx=0 tx=0).stop 2024-07-31T15:05:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:38.079+0000 7f72bf7fe640 1 -- 172.21.15.67:0/2263159187 shutdown_connections 2024-07-31T15:05:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:38.079+0000 7f72bf7fe640 1 --2- 172.21.15.67:0/2263159187 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f72c00b3f50 0x7f72c00b6460 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:38.079+0000 7f72bf7fe640 1 --2- 172.21.15.67:0/2263159187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72d811c5d0 0x7f72d81ae360 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:38.079+0000 7f72bf7fe640 1 --2- 172.21.15.67:0/2263159187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72d8111770 0x7f72d81a90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:38.079+0000 7f72bf7fe640 1 --2- 172.21.15.67:0/2263159187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d810a5b0 0x7f72d81a8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:38.079+0000 7f72bf7fe640 1 -- 172.21.15.67:0/2263159187 >> 172.21.15.67:0/2263159187 conn(0x7f72d806d380 msgr2=0x7f72d8110400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:38.080+0000 7f72bf7fe640 1 -- 172.21.15.67:0/2263159187 shutdown_connections 2024-07-31T15:05:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:38.080+0000 7f72bf7fe640 1 -- 172.21.15.67:0/2263159187 wait complete. 2024-07-31T15:05:38.086 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dsopcode.o 2024-07-31T15:05:38.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:38 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2263159187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:05:38.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:38 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2263159187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:05:38.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:38 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2263159187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:05:38.655 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pnp/resource.o 2024-07-31T15:05:38.681 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/pcm_timer.o 2024-07-31T15:05:38.707 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/skbuff.o 2024-07-31T15:05:38.742 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress_unlzo.o 2024-07-31T15:05:38.842 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/clntproc.o 2024-07-31T15:05:39.043 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dspkginit.o 2024-07-31T15:05:39.155 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/user.o 2024-07-31T15:05:39.321 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/core/seq_device.o 2024-07-31T15:05:39.430 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T15:05:39.451 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress_unxz.o 2024-07-31T15:05:39.477 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dsutils.o 2024-07-31T15:05:39.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:39 smithi077 ceph-mon[37327]: pgmap v859: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-07-31T15:05:39.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:39 smithi067 ceph-mon[32750]: pgmap v859: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-07-31T15:05:39.627 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/signal.o 2024-07-31T15:05:39.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:39 smithi073 ceph-mon[36714]: pgmap v859: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-07-31T15:05:39.866 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pnp/manager.o 2024-07-31T15:05:39.994 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/core/built-in.a 2024-07-31T15:05:40.001 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dswexec.o 2024-07-31T15:05:40.008 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress_unzstd.o 2024-07-31T15:05:40.055 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:05:40.064 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/export.o 2024-07-31T15:05:40.143 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/i2c/other/built-in.a 2024-07-31T15:05:40.163 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/i2c/built-in.a 2024-07-31T15:05:40.200 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/drivers/opl3/built-in.a 2024-07-31T15:05:40.206 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/clntxdr.o 2024-07-31T15:05:40.226 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/drivers/opl4/built-in.a 2024-07-31T15:05:40.255 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/drivers/mpu401/built-in.a 2024-07-31T15:05:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/drivers/vx/built-in.a 2024-07-31T15:05:40.308 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/drivers/pcsp/built-in.a 2024-07-31T15:05:40.322 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/drivers/built-in.a 2024-07-31T15:05:40.369 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/isa/ad1816a/built-in.a 2024-07-31T15:05:40.405 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/isa/ad1848/built-in.a 2024-07-31T15:05:40.412 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pnp/support.o 2024-07-31T15:05:40.443 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/isa/cs423x/built-in.a 2024-07-31T15:05:40.444 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dswload.o 2024-07-31T15:05:40.470 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/isa/es1688/built-in.a 2024-07-31T15:05:40.494 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/isa/galaxy/built-in.a 2024-07-31T15:05:40.494 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/dump_stack.o 2024-07-31T15:05:40.518 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/isa/gus/built-in.a 2024-07-31T15:05:40.541 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/isa/msnd/built-in.a 2024-07-31T15:05:40.615 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/isa/opti9xx/built-in.a 2024-07-31T15:05:40.703 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/isa/sb/built-in.a 2024-07-31T15:05:40.725 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/isa/wavefront/built-in.a 2024-07-31T15:05:40.748 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/isa/wss/built-in.a 2024-07-31T15:05:40.762 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/isa/built-in.a 2024-07-31T15:05:40.823 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/ac97/built-in.a 2024-07-31T15:05:40.840 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dswload2.o 2024-07-31T15:05:40.848 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/ali5451/built-in.a 2024-07-31T15:05:40.885 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/asihpi/built-in.a 2024-07-31T15:05:40.892 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pnp/interface.o 2024-07-31T15:05:40.913 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/au88x0/built-in.a 2024-07-31T15:05:40.938 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/aw2/built-in.a 2024-07-31T15:05:40.962 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/ctxfi/built-in.a 2024-07-31T15:05:40.995 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/ca0106/built-in.a 2024-07-31T15:05:41.021 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/cs46xx/built-in.a 2024-07-31T15:05:41.046 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/cs5535audio/built-in.a 2024-07-31T15:05:41.072 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/lola/built-in.a 2024-07-31T15:05:41.096 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/lx6464es/built-in.a 2024-07-31T15:05:41.115 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/earlycpio.o 2024-07-31T15:05:41.129 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/echoaudio/built-in.a 2024-07-31T15:05:41.150 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/host.o 2024-07-31T15:05:41.152 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/emu10k1/built-in.a 2024-07-31T15:05:41.183 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/pci/hda/hda_bind.o 2024-07-31T15:05:41.217 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/extable.o 2024-07-31T15:05:41.274 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dswscope.o 2024-07-31T15:05:41.336 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/sysfs.o 2024-07-31T15:05:41.417 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pnp/quirks.o 2024-07-31T15:05:41.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:41 smithi077 ceph-mon[37327]: pgmap v860: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 254 op/s 2024-07-31T15:05:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:41 smithi067 ceph-mon[32750]: pgmap v860: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 254 op/s 2024-07-31T15:05:41.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:41 smithi073 ceph-mon[36714]: pgmap v860: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 254 op/s 2024-07-31T15:05:41.653 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/dswstate.o 2024-07-31T15:05:41.656 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/flex_proportions.o 2024-07-31T15:05:41.850 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/pci/hda/hda_codec.o 2024-07-31T15:05:42.060 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/sys.o 2024-07-31T15:05:42.080 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/idr.o 2024-07-31T15:05:42.095 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evevent.o 2024-07-31T15:05:42.141 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pnp/system.o 2024-07-31T15:05:42.305 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/svc.o 2024-07-31T15:05:42.330 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/fs_context.o 2024-07-31T15:05:42.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.365+0000 7fab21413640 1 -- 172.21.15.67:0/1248772830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab1c11c5d0 msgr2=0x7fab1c11e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:42.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.365+0000 7fab21413640 1 --2- 172.21.15.67:0/1248772830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab1c11c5d0 0x7fab1c11e9f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fab1001a910 tx=0x7fab1003b420 comp rx=0 tx=0).stop 2024-07-31T15:05:42.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.365+0000 7fab21413640 1 -- 172.21.15.67:0/1248772830 shutdown_connections 2024-07-31T15:05:42.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.365+0000 7fab21413640 1 --2- 172.21.15.67:0/1248772830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab1c11c5d0 0x7fab1c11e9f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.365+0000 7fab21413640 1 --2- 172.21.15.67:0/1248772830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c10a5b0 0x7fab1c10a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.365+0000 7fab21413640 1 --2- 172.21.15.67:0/1248772830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab1c111770 0x7fab1c10a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.365+0000 7fab21413640 1 -- 172.21.15.67:0/1248772830 >> 172.21.15.67:0/1248772830 conn(0x7fab1c06d380 msgr2=0x7fab1c06d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:42.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.365+0000 7fab21413640 1 -- 172.21.15.67:0/1248772830 shutdown_connections 2024-07-31T15:05:42.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.365+0000 7fab21413640 1 -- 172.21.15.67:0/1248772830 wait complete. 2024-07-31T15:05:42.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.366+0000 7fab21413640 1 Processor -- start 2024-07-31T15:05:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.366+0000 7fab21413640 1 -- start start 2024-07-31T15:05:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.367+0000 7fab21413640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c10a5b0 0x7fab1c11be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.367+0000 7fab21413640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab1c111770 0x7fab1c118160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.367+0000 7fab21413640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab1c11c5d0 0x7fab1c118700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.367+0000 7fab21413640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1145f0 con 0x7fab1c10a5b0 2024-07-31T15:05:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.367+0000 7fab21413640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c114470 con 0x7fab1c111770 2024-07-31T15:05:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.367+0000 7fab21413640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c114770 con 0x7fab1c11c5d0 2024-07-31T15:05:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.367+0000 7fab1affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c10a5b0 0x7fab1c11be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.367+0000 7fab1affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c10a5b0 0x7fab1c11be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38292/0 (socket says 172.21.15.67:38292) 2024-07-31T15:05:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.367+0000 7fab1affd640 1 -- 172.21.15.67:0/486348401 learned_addr learned my addr 172.21.15.67:0/486348401 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.368+0000 7fab1affd640 1 -- 172.21.15.67:0/486348401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab1c11c5d0 msgr2=0x7fab1c118700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:05:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.368+0000 7fab1affd640 1 --2- 172.21.15.67:0/486348401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab1c11c5d0 0x7fab1c118700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.368+0000 7fab1affd640 1 -- 172.21.15.67:0/486348401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab1c111770 msgr2=0x7fab1c118160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:05:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.368+0000 7fab1affd640 1 --2- 172.21.15.67:0/486348401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab1c111770 0x7fab1c118160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.368+0000 7fab1affd640 1 -- 172.21.15.67:0/486348401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c118f20 con 0x7fab1c10a5b0 2024-07-31T15:05:42.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.368+0000 7fab1affd640 1 --2- 172.21.15.67:0/486348401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c10a5b0 0x7fab1c11be80 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fab0c005fd0 tx=0x7fab0c00df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:42.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.370+0000 7fab03fff640 1 -- 172.21.15.67:0/486348401 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c017070 con 0x7fab1c10a5b0 2024-07-31T15:05:42.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.370+0000 7fab21413640 1 -- 172.21.15.67:0/486348401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1c1ce700 con 0x7fab1c10a5b0 2024-07-31T15:05:42.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.370+0000 7fab21413640 1 -- 172.21.15.67:0/486348401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1c1cec40 con 0x7fab1c10a5b0 2024-07-31T15:05:42.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.370+0000 7fab03fff640 1 -- 172.21.15.67:0/486348401 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab0c009e50 con 0x7fab1c10a5b0 2024-07-31T15:05:42.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.370+0000 7fab03fff640 1 -- 172.21.15.67:0/486348401 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c003dd0 con 0x7fab1c10a5b0 2024-07-31T15:05:42.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.374+0000 7fab27964740 1 Processor -- start 2024-07-31T15:05:42.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.379+0000 7fab03fff640 1 -- 172.21.15.67:0/486348401 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fab0c004220 con 0x7fab1c10a5b0 2024-07-31T15:05:42.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.380+0000 7fab27964740 1 -- start start 2024-07-31T15:05:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.380+0000 7fab27964740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b5d070 0x55f6e3f02710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.380+0000 7fab27964740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b341b0 0x55f6e3f02c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.380+0000 7fab27964740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b3c390 0x55f6e3f03190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.380+0000 7fab27964740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f6e3e78bb0 con 0x55f6e3b341b0 2024-07-31T15:05:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.380+0000 7fab27964740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f6e3e680b0 con 0x55f6e3b5d070 2024-07-31T15:05:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.380+0000 7fab27964740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f6e3e6c0f0 con 0x55f6e3b3c390 2024-07-31T15:05:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.381+0000 7fab1a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b3c390 0x55f6e3f03190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:42.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.382+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b5d070 0x55f6e3f02710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:42.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.382+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b5d070 0x55f6e3f02710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56204/0 (socket says 172.21.15.67:56204) 2024-07-31T15:05:42.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.382+0000 7fab1b7fe640 1 -- 172.21.15.67:0/2291879463 learned_addr learned my addr 172.21.15.67:0/2291879463 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:42.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.382+0000 7fab1affd640 1 --2- 172.21.15.67:0/2291879463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b341b0 0x55f6e3f02c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:42.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.383+0000 7fab1affd640 1 -- 172.21.15.67:0/2291879463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b3c390 msgr2=0x55f6e3f03190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:42.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.383+0000 7fab1affd640 1 --2- 172.21.15.67:0/2291879463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b3c390 0x55f6e3f03190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.383+0000 7fab1affd640 1 -- 172.21.15.67:0/2291879463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b5d070 msgr2=0x55f6e3f02710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:42.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.383+0000 7fab1affd640 1 --2- 172.21.15.67:0/2291879463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b5d070 0x55f6e3f02710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.383+0000 7fab1affd640 1 -- 172.21.15.67:0/2291879463 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6e3e0eb90 con 0x55f6e3b341b0 2024-07-31T15:05:42.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.384+0000 7fab03fff640 1 --2- 172.21.15.67:0/486348401 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fab040b3f50 0x7fab040b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.384+0000 7fab1b7fe640 1 --2- 172.21.15.67:0/486348401 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fab040b3f50 0x7fab040b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:42.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.384+0000 7fab03fff640 1 -- 172.21.15.67:0/486348401 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fab0c0e4060 con 0x7fab1c10a5b0 2024-07-31T15:05:42.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.385+0000 7fab1b7fe640 1 --2- 172.21.15.67:0/486348401 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fab040b3f50 0x7fab040b6460 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fab1001a6d0 tx=0x7fab10005f30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:42.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.386+0000 7fab1affd640 1 --2- 172.21.15.67:0/2291879463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b341b0 0x55f6e3f02c50 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0e13e0 tx=0x7fab0c0e7f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e171447520c96234 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:42.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.386+0000 7fab017fa640 1 -- 172.21.15.67:0/2291879463 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0a6760 con 0x55f6e3b341b0 2024-07-31T15:05:42.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.387+0000 7fab017fa640 1 -- 172.21.15.67:0/2291879463 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab0c0aa3b0 con 0x55f6e3b341b0 2024-07-31T15:05:42.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.387+0000 7fab017fa640 1 -- 172.21.15.67:0/2291879463 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0aa010 con 0x55f6e3b341b0 2024-07-31T15:05:42.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.388+0000 7fab27964740 1 -- 172.21.15.67:0/2291879463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b341b0 msgr2=0x55f6e3f02c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:42.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.388+0000 7fab27964740 1 --2- 172.21.15.67:0/2291879463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b341b0 0x55f6e3f02c50 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0e13e0 tx=0x7fab0c0e7f90 comp rx=0 tx=0).stop 2024-07-31T15:05:42.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.390+0000 7fab27964740 1 -- 172.21.15.67:0/2291879463 shutdown_connections 2024-07-31T15:05:42.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.390+0000 7fab27964740 1 --2- 172.21.15.67:0/2291879463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b3c390 0x55f6e3f03190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.390+0000 7fab27964740 1 --2- 172.21.15.67:0/2291879463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b341b0 0x55f6e3f02c50 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.390+0000 7fab27964740 1 --2- 172.21.15.67:0/2291879463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b5d070 0x55f6e3f02710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.390+0000 7fab27964740 1 -- 172.21.15.67:0/2291879463 >> 172.21.15.67:0/2291879463 conn(0x55f6e3b51460 msgr2=0x55f6e3f00600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:42.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.394+0000 7fab27964740 1 -- 172.21.15.67:0/2291879463 shutdown_connections 2024-07-31T15:05:42.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.395+0000 7fab27964740 1 -- 172.21.15.67:0/2291879463 wait complete. 2024-07-31T15:05:42.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.395+0000 7fab27964740 1 -- start start 2024-07-31T15:05:42.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.395+0000 7fab27964740 1 Processor -- start 2024-07-31T15:05:42.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.396+0000 7fab27964740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b341b0 0x55f6e3f0e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.396+0000 7fab27964740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b3c390 0x55f6e3f0e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.396+0000 7fab27964740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b5d070 0x55f6e3f1a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.396+0000 7fab27964740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f6e3ccb700 con 0x55f6e3b3c390 2024-07-31T15:05:42.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.396+0000 7fab27964740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f6e3e0a9e0 con 0x55f6e3b341b0 2024-07-31T15:05:42.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.396+0000 7fab27964740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f6e3f10ef0 con 0x55f6e3b5d070 2024-07-31T15:05:42.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.396+0000 7fab27964740 10 client.0 _ll_register_callbacks cb 0x55f6e3e254f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:42.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.399+0000 7fab1a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b3c390 0x55f6e3f0e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:42.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.399+0000 7fab1a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b3c390 0x55f6e3f0e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38322/0 (socket says 172.21.15.67:38322) 2024-07-31T15:05:42.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.399+0000 7fab1a7fc640 1 -- 172.21.15.67:0/2131779840 learned_addr learned my addr 172.21.15.67:0/2131779840 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:42.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.400+0000 7fab1a7fc640 1 -- 172.21.15.67:0/2131779840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b5d070 msgr2=0x55f6e3f1a2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:05:42.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.400+0000 7fab1a7fc640 1 --2- 172.21.15.67:0/2131779840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b5d070 0x55f6e3f1a2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.400+0000 7fab1a7fc640 1 -- 172.21.15.67:0/2131779840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b341b0 msgr2=0x55f6e3f0e2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:05:42.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.400+0000 7fab1a7fc640 1 --2- 172.21.15.67:0/2131779840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b341b0 0x55f6e3f0e2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.400+0000 7fab1a7fc640 1 -- 172.21.15.67:0/2131779840 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6e3bc8650 con 0x55f6e3b3c390 2024-07-31T15:05:42.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.400+0000 7fab1a7fc640 1 --2- 172.21.15.67:0/2131779840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b3c390 0x55f6e3f0e810 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fab1400f5e0 tx=0x7fab1400faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4557bd35f15cbce server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:42.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.401+0000 7faafeff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:05:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.405+0000 7faafeff5640 1 -- 172.21.15.67:0/2131779840 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14004d20 con 0x55f6e3b3c390 2024-07-31T15:05:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.405+0000 7fab27964740 1 -- 172.21.15.67:0/2131779840 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab0c0aa3b0 con 0x55f6e3b3c390 2024-07-31T15:05:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.405+0000 7faafeff5640 1 -- 172.21.15.67:0/2131779840 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fab140117d0 con 0x55f6e3b3c390 2024-07-31T15:05:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.405+0000 7faafeff5640 1 -- 172.21.15.67:0/2131779840 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fab14023050 con 0x55f6e3b3c390 2024-07-31T15:05:42.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.406+0000 7fab27964740 10 client.18552 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:42.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.406+0000 7fab27964740 1 -- 172.21.15.67:0/2131779840 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f6e3f1cff0 con 0x55f6e3b3c390 2024-07-31T15:05:42.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.408+0000 7faafeff5640 1 -- 172.21.15.67:0/2131779840 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab0c0aa3b0 con 0x55f6e3b3c390 2024-07-31T15:05:42.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.408+0000 7faafeff5640 1 -- 172.21.15.67:0/2131779840 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fab1401bb80 con 0x55f6e3b3c390 2024-07-31T15:05:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.412+0000 7fab27964740 10 client.18552 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.412+0000 7fab27964740 10 client.18552 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:05:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.412+0000 7fab27964740 20 client.18552 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.412+0000 7fab27964740 1 --2- 172.21.15.67:0/2131779840 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f6e3b1b9f0 0x55f6e3f20670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.412+0000 7fab27964740 4 client.18552 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:05:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.412+0000 7fab27964740 1 -- 172.21.15.67:0/2131779840 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6e3e7ab00 con 0x55f6e3b1b9f0 2024-07-31T15:05:42.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.417+0000 7fab1affd640 1 --2- 172.21.15.67:0/2131779840 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f6e3b1b9f0 0x55f6e3f20670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:42.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.432+0000 7fab1affd640 1 --2- 172.21.15.67:0/2131779840 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f6e3b1b9f0 0x55f6e3f20670 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c0ef4d8bb9de2337 server_cookie=24d8fa45aeca91af in_seq=0 out_seq=0 2024-07-31T15:05:42.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.433+0000 7faafeff5640 10 client.18552 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:05:42.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.437+0000 7faafeff5640 1 -- 172.21.15.67:0/2131779840 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6e3e7ab00 con 0x55f6e3b1b9f0 2024-07-31T15:05:42.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.437+0000 7faafeff5640 10 client.18552 handle_command_reply: tid=0 2024-07-31T15:05:42.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.438+0000 7fab27964740 1 client.18552 shutdown 2024-07-31T15:05:42.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.448+0000 7fab27964740 1 -- 172.21.15.67:0/2131779840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b3c390 msgr2=0x55f6e3f0e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:42.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.448+0000 7fab27964740 1 --2- 172.21.15.67:0/2131779840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b3c390 0x55f6e3f0e810 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fab1400f5e0 tx=0x7fab1400faa0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.454+0000 7fab27964740 1 -- 172.21.15.67:0/2131779840 shutdown_connections 2024-07-31T15:05:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.454+0000 7fab27964740 1 --2- 172.21.15.67:0/2131779840 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f6e3b1b9f0 0x55f6e3f20670 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.454+0000 7fab27964740 1 --2- 172.21.15.67:0/2131779840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b5d070 0x55f6e3f1a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.454+0000 7fab27964740 1 --2- 172.21.15.67:0/2131779840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b3c390 0x55f6e3f0e810 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.454+0000 7fab27964740 1 --2- 172.21.15.67:0/2131779840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b341b0 0x55f6e3f0e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.454+0000 7faafeff5640 0 client.18552 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:05:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.454+0000 7fab27964740 1 -- 172.21.15.67:0/2131779840 >> 172.21.15.67:0/2131779840 conn(0x55f6e3b51460 msgr2=0x55f6e3f058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.454+0000 7fab27964740 1 -- 172.21.15.67:0/2131779840 shutdown_connections 2024-07-31T15:05:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.454+0000 7fab27964740 1 -- 172.21.15.67:0/2131779840 wait complete. 2024-07-31T15:05:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.454+0000 7fab27964740 20 client.18552 trim_cache size 0 max 16384 2024-07-31T15:05:42.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.484+0000 7fab27964740 1 Processor -- start 2024-07-31T15:05:42.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.484+0000 7fab27964740 1 -- start start 2024-07-31T15:05:42.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.485+0000 7fab27964740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b341b0 0x55f6e3f005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.485+0000 7fab27964740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b3c390 0x55f6e3f07970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.485+0000 7fab27964740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b5d070 0x55f6e3f07eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.485+0000 7fab27964740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fab0800ff70 con 0x55f6e3b3c390 2024-07-31T15:05:42.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.485+0000 7fab27964740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fab08011ab0 con 0x55f6e3b341b0 2024-07-31T15:05:42.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.485+0000 7fab27964740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fab08010640 con 0x55f6e3b5d070 2024-07-31T15:05:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.485+0000 7fab1a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b3c390 0x55f6e3f07970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.485+0000 7fab1b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b5d070 0x55f6e3f07eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.485+0000 7fab1a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b3c390 0x55f6e3f07970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38336/0 (socket says 172.21.15.67:38336) 2024-07-31T15:05:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.485+0000 7fab1a7fc640 1 -- 172.21.15.67:0/3003009899 learned_addr learned my addr 172.21.15.67:0/3003009899 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.485+0000 7fab1affd640 1 --2- 172.21.15.67:0/3003009899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b341b0 0x55f6e3f005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.486+0000 7fab1a7fc640 1 -- 172.21.15.67:0/3003009899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b5d070 msgr2=0x55f6e3f07eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.486+0000 7fab1a7fc640 1 --2- 172.21.15.67:0/3003009899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b5d070 0x55f6e3f07eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.486+0000 7fab1a7fc640 1 -- 172.21.15.67:0/3003009899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b341b0 msgr2=0x55f6e3f005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.486+0000 7fab1a7fc640 1 --2- 172.21.15.67:0/3003009899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b341b0 0x55f6e3f005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.486+0000 7fab1a7fc640 1 -- 172.21.15.67:0/3003009899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6e3f00f70 con 0x55f6e3b3c390 2024-07-31T15:05:42.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.487+0000 7fab1a7fc640 1 --2- 172.21.15.67:0/3003009899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b3c390 0x55f6e3f07970 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fab14002230 tx=0x7fab14010760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555d11874ce0c5d4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:42.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.488+0000 7fab00ff9640 1 -- 172.21.15.67:0/3003009899 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14003b70 con 0x55f6e3b3c390 2024-07-31T15:05:42.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.488+0000 7fab00ff9640 1 -- 172.21.15.67:0/3003009899 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab14010d80 con 0x55f6e3b3c390 2024-07-31T15:05:42.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.489+0000 7fab27964740 1 -- 172.21.15.67:0/3003009899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b3c390 msgr2=0x55f6e3f07970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:42.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.489+0000 7fab27964740 1 --2- 172.21.15.67:0/3003009899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b3c390 0x55f6e3f07970 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fab14002230 tx=0x7fab14010760 comp rx=0 tx=0).stop 2024-07-31T15:05:42.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.490+0000 7fab27964740 1 -- 172.21.15.67:0/3003009899 shutdown_connections 2024-07-31T15:05:42.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.490+0000 7fab27964740 1 --2- 172.21.15.67:0/3003009899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b5d070 0x55f6e3f07eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.490+0000 7fab27964740 1 --2- 172.21.15.67:0/3003009899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b3c390 0x55f6e3f07970 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.490+0000 7fab27964740 1 --2- 172.21.15.67:0/3003009899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b341b0 0x55f6e3f005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.490+0000 7fab27964740 1 -- 172.21.15.67:0/3003009899 >> 172.21.15.67:0/3003009899 conn(0x55f6e3b51460 msgr2=0x55f6e3effe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.490+0000 7fab27964740 1 -- 172.21.15.67:0/3003009899 shutdown_connections 2024-07-31T15:05:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.490+0000 7fab27964740 1 -- 172.21.15.67:0/3003009899 wait complete. 2024-07-31T15:05:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.491+0000 7fab27964740 1 -- start start 2024-07-31T15:05:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.491+0000 7fab27964740 1 Processor -- start 2024-07-31T15:05:42.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.492+0000 7fab27964740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b341b0 0x55f6e3f169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.492+0000 7fab27964740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b3c390 0x55f6e3f071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.492+0000 7fab27964740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b5d070 0x55f6e3f07800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.492+0000 7fab27964740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fab080113e0 con 0x55f6e3b341b0 2024-07-31T15:05:42.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.492+0000 7fab27964740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fab08010d10 con 0x55f6e3b3c390 2024-07-31T15:05:42.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.492+0000 7fab27964740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fab0800df70 con 0x55f6e3b5d070 2024-07-31T15:05:42.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.492+0000 7fab27964740 10 client.0 _ll_register_callbacks cb 0x7fab14009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:42.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.492+0000 7fab1a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b3c390 0x55f6e3f071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:42.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.492+0000 7fab1a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b3c390 0x55f6e3f071b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56224/0 (socket says 172.21.15.67:56224) 2024-07-31T15:05:42.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.492+0000 7fab1a7fc640 1 -- 172.21.15.67:0/3769128572 learned_addr learned my addr 172.21.15.67:0/3769128572 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:42.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.493+0000 7fab1affd640 1 --2- 172.21.15.67:0/3769128572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b341b0 0x55f6e3f169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:42.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.493+0000 7fab1b7fe640 1 --2- 172.21.15.67:0/3769128572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b5d070 0x55f6e3f07800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:42.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.493+0000 7fab1a7fc640 1 -- 172.21.15.67:0/3769128572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b5d070 msgr2=0x55f6e3f07800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.493+0000 7fab1a7fc640 1 --2- 172.21.15.67:0/3769128572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b5d070 0x55f6e3f07800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.493+0000 7fab1a7fc640 1 -- 172.21.15.67:0/3769128572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b341b0 msgr2=0x55f6e3f169f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.493+0000 7fab1a7fc640 1 --2- 172.21.15.67:0/3769128572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b341b0 0x55f6e3f169f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.493+0000 7fab1a7fc640 1 -- 172.21.15.67:0/3769128572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6e3f07f70 con 0x55f6e3b3c390 2024-07-31T15:05:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.494+0000 7fab1a7fc640 1 --2- 172.21.15.67:0/3769128572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b3c390 0x55f6e3f071b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fab14010010 tx=0x7fab14010b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85f3e41386383d15 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.494+0000 7faafeff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:05:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.494+0000 7faafeff5640 1 -- 172.21.15.67:0/3769128572 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14011df0 con 0x55f6e3b3c390 2024-07-31T15:05:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.494+0000 7fab27964740 1 -- 172.21.15.67:0/3769128572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab14010d80 con 0x55f6e3b3c390 2024-07-31T15:05:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.495+0000 7faafeff5640 1 -- 172.21.15.67:0/3769128572 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fab1401b410 con 0x55f6e3b3c390 2024-07-31T15:05:42.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.495+0000 7faafeff5640 1 -- 172.21.15.67:0/3769128572 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fab1403f050 con 0x55f6e3b3c390 2024-07-31T15:05:42.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.497+0000 7fab27964740 10 client.28109 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:42.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.497+0000 7fab27964740 1 -- 172.21.15.67:0/3769128572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f6e3f88e00 con 0x55f6e3b3c390 2024-07-31T15:05:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.502+0000 7faafeff5640 1 -- 172.21.15.67:0/3769128572 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab14010d80 con 0x55f6e3b3c390 2024-07-31T15:05:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.502+0000 7faafeff5640 1 -- 172.21.15.67:0/3769128572 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fab14037020 con 0x55f6e3b3c390 2024-07-31T15:05:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.502+0000 7fab27964740 10 client.28109 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.502+0000 7fab27964740 10 client.28109 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:05:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.502+0000 7fab27964740 20 client.28109 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.502+0000 7fab27964740 1 --2- 172.21.15.67:0/3769128572 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f6e3af0da0 0x55f6e3f8b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.502+0000 7fab27964740 4 client.28109 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:05:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.502+0000 7fab27964740 1 -- 172.21.15.67:0/3769128572 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6e3e765a0 con 0x55f6e3af0da0 2024-07-31T15:05:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.502+0000 7fab1affd640 1 --2- 172.21.15.67:0/3769128572 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f6e3af0da0 0x55f6e3f8b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:42.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.507+0000 7faafeff5640 10 client.28109 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:05:42.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.507+0000 7fab1affd640 1 --2- 172.21.15.67:0/3769128572 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f6e3af0da0 0x55f6e3f8b420 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4709aa50faf3fde1 server_cookie=fe028bc83a47d825 in_seq=0 out_seq=0 2024-07-31T15:05:42.513 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evgpe.o 2024-07-31T15:05:42.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.514+0000 7faafeff5640 1 -- 172.21.15.67:0/3769128572 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6e3e765a0 con 0x55f6e3af0da0 2024-07-31T15:05:42.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.514+0000 7faafeff5640 10 client.28109 handle_command_reply: tid=0 2024-07-31T15:05:42.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.514+0000 7fab27964740 1 client.28109 shutdown 2024-07-31T15:05:42.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.516+0000 7fab27964740 1 -- 172.21.15.67:0/3769128572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b3c390 msgr2=0x55f6e3f071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:42.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.516+0000 7fab27964740 1 --2- 172.21.15.67:0/3769128572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b3c390 0x55f6e3f071b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fab14010010 tx=0x7fab14010b00 comp rx=0 tx=0).stop 2024-07-31T15:05:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.517+0000 7fab27964740 1 -- 172.21.15.67:0/3769128572 shutdown_connections 2024-07-31T15:05:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.517+0000 7fab27964740 1 --2- 172.21.15.67:0/3769128572 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55f6e3af0da0 0x55f6e3f8b420 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.517+0000 7fab27964740 1 --2- 172.21.15.67:0/3769128572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6e3b5d070 0x55f6e3f07800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.517+0000 7fab27964740 1 --2- 172.21.15.67:0/3769128572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6e3b3c390 0x55f6e3f071b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.517+0000 7fab27964740 1 --2- 172.21.15.67:0/3769128572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f6e3b341b0 0x55f6e3f169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.517+0000 7faafeff5640 0 client.28109 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:05:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.517+0000 7fab27964740 1 -- 172.21.15.67:0/3769128572 >> 172.21.15.67:0/3769128572 conn(0x55f6e3b51460 msgr2=0x55f6e3f03720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:42.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.518+0000 7fab27964740 1 -- 172.21.15.67:0/3769128572 shutdown_connections 2024-07-31T15:05:42.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.521+0000 7fab27964740 1 -- 172.21.15.67:0/3769128572 wait complete. 2024-07-31T15:05:42.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.522+0000 7fab27964740 20 client.28109 trim_cache size 0 max 16384 2024-07-31T15:05:42.525 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:05:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.524+0000 7fab21413640 1 -- 172.21.15.67:0/486348401 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fab040b3f50 msgr2=0x7fab040b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.524+0000 7fab21413640 1 --2- 172.21.15.67:0/486348401 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fab040b3f50 0x7fab040b6460 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fab1001a6d0 tx=0x7fab10005f30 comp rx=0 tx=0).stop 2024-07-31T15:05:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.524+0000 7fab21413640 1 -- 172.21.15.67:0/486348401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c10a5b0 msgr2=0x7fab1c11be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.524+0000 7fab21413640 1 --2- 172.21.15.67:0/486348401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c10a5b0 0x7fab1c11be80 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fab0c005fd0 tx=0x7fab0c00df20 comp rx=0 tx=0).stop 2024-07-31T15:05:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.524+0000 7fab21413640 1 -- 172.21.15.67:0/486348401 shutdown_connections 2024-07-31T15:05:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.524+0000 7fab21413640 1 --2- 172.21.15.67:0/486348401 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fab040b3f50 0x7fab040b6460 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.524+0000 7fab21413640 1 --2- 172.21.15.67:0/486348401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab1c11c5d0 0x7fab1c118700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.524+0000 7fab21413640 1 --2- 172.21.15.67:0/486348401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab1c111770 0x7fab1c118160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.524+0000 7fab21413640 1 --2- 172.21.15.67:0/486348401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c10a5b0 0x7fab1c11be80 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.524+0000 7fab21413640 1 -- 172.21.15.67:0/486348401 >> 172.21.15.67:0/486348401 conn(0x7fab1c06d380 msgr2=0x7fab1c110400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:42.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.529+0000 7fab21413640 1 -- 172.21.15.67:0/486348401 shutdown_connections 2024-07-31T15:05:42.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:42.529+0000 7fab21413640 1 -- 172.21.15.67:0/486348401 wait complete. 2024-07-31T15:05:42.604 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pnp/built-in.a 2024-07-31T15:05:42.621 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/irq_regs.o 2024-07-31T15:05:42.664 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/datagram.o 2024-07-31T15:05:42.678 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/stream.o 2024-07-31T15:05:42.779 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/is_single_threaded.o 2024-07-31T15:05:42.970 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evgpeblk.o 2024-07-31T15:05:43.182 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/klist.o 2024-07-31T15:05:43.234 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:43 smithi067 ceph-mon[32750]: pgmap v861: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-07-31T15:05:43.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T15:05:43.437 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/svclock.o 2024-07-31T15:05:43.443 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evgpeinit.o 2024-07-31T15:05:43.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:43 smithi077 ceph-mon[37327]: pgmap v861: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-07-31T15:05:43.613 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/kobject.o 2024-07-31T15:05:43.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:43 smithi073 ceph-mon[36714]: pgmap v861: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-07-31T15:05:43.666 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/pci/hda/hda_jack.o 2024-07-31T15:05:43.687 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/svcshare.o 2024-07-31T15:05:43.784 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/umh.o 2024-07-31T15:05:43.856 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evgpeutil.o 2024-07-31T15:05:43.859 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/scm.o 2024-07-31T15:05:44.027 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfsroot.o 2024-07-31T15:05:44.031 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:05:44.245 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/kobject_uevent.o 2024-07-31T15:05:44.305 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evglock.o 2024-07-31T15:05:44.555 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/workqueue.o 2024-07-31T15:05:44.581 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/pci/hda/hda_auto_parser.o 2024-07-31T15:05:44.583 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/svcproc.o 2024-07-31T15:05:44.602 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/svcsubs.o 2024-07-31T15:05:44.687 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evhandler.o 2024-07-31T15:05:44.911 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/gen_stats.o 2024-07-31T15:05:44.984 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/sysctl.o 2024-07-31T15:05:45.101 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evmisc.o 2024-07-31T15:05:45.192 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-31T15:05:45.193 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.8 2024-07-31T15:05:45.193 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:05:45.248 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/logic_pio.o 2024-07-31T15:05:45.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:45 smithi067 ceph-mon[32750]: pgmap v862: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 258 op/s 2024-07-31T15:05:45.588 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/pci/hda/hda_sysfs.o 2024-07-31T15:05:45.597 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evregion.o 2024-07-31T15:05:45.600 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/gen_estimator.o 2024-07-31T15:05:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:45 smithi073 ceph-mon[36714]: pgmap v862: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 258 op/s 2024-07-31T15:05:45.698 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/mon.o 2024-07-31T15:05:45.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:45 smithi077 ceph-mon[37327]: pgmap v862: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 258 op/s 2024-07-31T15:05:45.800 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evrgnini.o 2024-07-31T15:05:45.861 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs2super.o 2024-07-31T15:05:45.938 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/maple_tree.o 2024-07-31T15:05:46.128 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/net_namespace.o 2024-07-31T15:05:46.222 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/pci/hda/hda_controller.o 2024-07-31T15:05:46.256 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evsci.o 2024-07-31T15:05:46.392 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:05:46.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.456+0000 7f9af9e3a640 1 -- 172.21.15.67:0/1342100195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9af4111770 msgr2=0x7f9af410a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:46.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.456+0000 7f9af9e3a640 1 --2- 172.21.15.67:0/1342100195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9af4111770 0x7f9af410a070 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009ca0 tx=0x7f9ae402a8f0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.463+0000 7f9af9e3a640 1 -- 172.21.15.67:0/1342100195 shutdown_connections 2024-07-31T15:05:46.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.463+0000 7f9af9e3a640 1 --2- 172.21.15.67:0/1342100195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9af411c5d0 0x7f9af411e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.463+0000 7f9af9e3a640 1 --2- 172.21.15.67:0/1342100195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af410a5b0 0x7f9af410a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.463+0000 7f9af9e3a640 1 --2- 172.21.15.67:0/1342100195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9af4111770 0x7f9af410a070 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.463+0000 7f9af9e3a640 1 -- 172.21.15.67:0/1342100195 >> 172.21.15.67:0/1342100195 conn(0x7f9af406d380 msgr2=0x7f9af406d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:46.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.471+0000 7f9af9e3a640 1 -- 172.21.15.67:0/1342100195 shutdown_connections 2024-07-31T15:05:46.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.472+0000 7f9af9e3a640 1 -- 172.21.15.67:0/1342100195 wait complete. 2024-07-31T15:05:46.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.473+0000 7f9af9e3a640 1 Processor -- start 2024-07-31T15:05:46.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.473+0000 7f9af9e3a640 1 -- start start 2024-07-31T15:05:46.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.473+0000 7f9af9e3a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9af410a5b0 0x7f9af411bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.473+0000 7f9af9e3a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9af4111770 0x7f9af41151f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.477+0000 7f9af9e3a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af411c5d0 0x7f9af4115850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.477+0000 7f9af9e3a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4122150 con 0x7f9af4111770 2024-07-31T15:05:46.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.477+0000 7f9af9e3a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4121fd0 con 0x7f9af411c5d0 2024-07-31T15:05:46.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.477+0000 7f9af9e3a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41222d0 con 0x7f9af410a5b0 2024-07-31T15:05:46.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.477+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9af4111770 0x7f9af41151f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:46.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.477+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9af4111770 0x7f9af41151f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38360/0 (socket says 172.21.15.67:38360) 2024-07-31T15:05:46.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.477+0000 7f9af2ffd640 1 -- 172.21.15.67:0/4138012204 learned_addr learned my addr 172.21.15.67:0/4138012204 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:46.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.477+0000 7f9af3fff640 1 --2- 172.21.15.67:0/4138012204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af411c5d0 0x7f9af4115850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:46.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.478+0000 7f9af3fff640 1 -- 172.21.15.67:0/4138012204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9af410a5b0 msgr2=0x7f9af411bf20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:05:46.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.478+0000 7f9af3fff640 1 --2- 172.21.15.67:0/4138012204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9af410a5b0 0x7f9af411bf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.478+0000 7f9af3fff640 1 -- 172.21.15.67:0/4138012204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9af4111770 msgr2=0x7f9af41151f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:46.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.478+0000 7f9af3fff640 1 --2- 172.21.15.67:0/4138012204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9af4111770 0x7f9af41151f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.478+0000 7f9af3fff640 1 -- 172.21.15.67:0/4138012204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af411ab90 con 0x7f9af411c5d0 2024-07-31T15:05:46.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.478+0000 7f9af2ffd640 1 --2- 172.21.15.67:0/4138012204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9af4111770 0x7f9af41151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:05:46.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.479+0000 7f9af3fff640 1 --2- 172.21.15.67:0/4138012204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af411c5d0 0x7f9af4115850 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00c710 tx=0x7f9adc00cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:46.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.481+0000 7f9af0ff9640 1 -- 172.21.15.67:0/4138012204 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc004220 con 0x7f9af411c5d0 2024-07-31T15:05:46.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.481+0000 7f9af0ff9640 1 -- 172.21.15.67:0/4138012204 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9adc00ce20 con 0x7f9af411c5d0 2024-07-31T15:05:46.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.481+0000 7f9af0ff9640 1 -- 172.21.15.67:0/4138012204 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc003d50 con 0x7f9af411c5d0 2024-07-31T15:05:46.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.481+0000 7f9af9e3a640 1 -- 172.21.15.67:0/4138012204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af411ae80 con 0x7f9af411c5d0 2024-07-31T15:05:46.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.481+0000 7f9af9e3a640 1 -- 172.21.15.67:0/4138012204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af411b310 con 0x7f9af411c5d0 2024-07-31T15:05:46.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.488+0000 7f9af0ff9640 1 -- 172.21.15.67:0/4138012204 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f9adc00f450 con 0x7f9af411c5d0 2024-07-31T15:05:46.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.493+0000 7f9b0038b740 1 Processor -- start 2024-07-31T15:05:46.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.497+0000 7f9af0ff9640 1 --2- 172.21.15.67:0/4138012204 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9ae00b3f50 0x7f9ae00b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.497+0000 7f9af0ff9640 1 -- 172.21.15.67:0/4138012204 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f9adc0e3ae0 con 0x7f9af411c5d0 2024-07-31T15:05:46.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.498+0000 7f9b0038b740 1 -- start start 2024-07-31T15:05:46.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.498+0000 7f9af2ffd640 1 --2- 172.21.15.67:0/4138012204 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9ae00b3f50 0x7f9ae00b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:46.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.499+0000 7f9b0038b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562999037070 0x5629993dc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.499+0000 7f9b0038b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56299900e1b0 0x5629993dcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.499+0000 7f9b0038b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993dd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.499+0000 7f9b0038b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562999352bb0 con 0x562999016390 2024-07-31T15:05:46.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.499+0000 7f9b0038b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5629993420b0 con 0x562999037070 2024-07-31T15:05:46.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.499+0000 7f9b0038b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5629993460f0 con 0x56299900e1b0 2024-07-31T15:05:46.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.499+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56299900e1b0 0x5629993dcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:46.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.499+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56299900e1b0 0x5629993dcc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60162/0 (socket says 172.21.15.67:60162) 2024-07-31T15:05:46.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.499+0000 7f9af2ffd640 1 -- 172.21.15.67:0/771319722 learned_addr learned my addr 172.21.15.67:0/771319722 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:46.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.499+0000 7f9af2ffd640 1 --2- 172.21.15.67:0/4138012204 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9ae00b3f50 0x7f9ae00b6460 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9af4110160 tx=0x7f9aec005f50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:46.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.502+0000 7f9af37fe640 1 --2- 172.21.15.67:0/771319722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562999037070 0x5629993dc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:46.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.502+0000 7f9af3fff640 1 --2- 172.21.15.67:0/771319722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993dd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:46.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.503+0000 7f9af2ffd640 1 -- 172.21.15.67:0/771319722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562999037070 msgr2=0x5629993dc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:46.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.503+0000 7f9af2ffd640 1 --2- 172.21.15.67:0/771319722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562999037070 0x5629993dc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.503+0000 7f9af2ffd640 1 -- 172.21.15.67:0/771319722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 msgr2=0x5629993dd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:46.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.503+0000 7f9af2ffd640 1 --2- 172.21.15.67:0/771319722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993dd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.503+0000 7f9af2ffd640 1 -- 172.21.15.67:0/771319722 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629992e8b90 con 0x56299900e1b0 2024-07-31T15:05:46.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.503+0000 7f9af37fe640 1 --2- 172.21.15.67:0/771319722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562999037070 0x5629993dc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:05:46.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.510+0000 7f9af2ffd640 1 --2- 172.21.15.67:0/771319722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56299900e1b0 0x5629993dcc50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00ef10 tx=0x7f9aec011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4cef9efadbcd608 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:46.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.511+0000 7f9ae9ffb640 1 -- 172.21.15.67:0/771319722 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec021070 con 0x56299900e1b0 2024-07-31T15:05:46.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.511+0000 7f9ae9ffb640 1 -- 172.21.15.67:0/771319722 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aec010e40 con 0x56299900e1b0 2024-07-31T15:05:46.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.511+0000 7f9ae9ffb640 1 -- 172.21.15.67:0/771319722 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec01d8c0 con 0x56299900e1b0 2024-07-31T15:05:46.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.513+0000 7f9b0038b740 1 -- 172.21.15.67:0/771319722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56299900e1b0 msgr2=0x5629993dcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:46.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.513+0000 7f9b0038b740 1 --2- 172.21.15.67:0/771319722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56299900e1b0 0x5629993dcc50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00ef10 tx=0x7f9aec011000 comp rx=0 tx=0).stop 2024-07-31T15:05:46.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.513+0000 7f9b0038b740 1 -- 172.21.15.67:0/771319722 shutdown_connections 2024-07-31T15:05:46.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.513+0000 7f9b0038b740 1 --2- 172.21.15.67:0/771319722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993dd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.513+0000 7f9b0038b740 1 --2- 172.21.15.67:0/771319722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56299900e1b0 0x5629993dcc50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.513+0000 7f9b0038b740 1 --2- 172.21.15.67:0/771319722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562999037070 0x5629993dc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.513+0000 7f9b0038b740 1 -- 172.21.15.67:0/771319722 >> 172.21.15.67:0/771319722 conn(0x56299902b460 msgr2=0x5629993da600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:46.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.515+0000 7f9b0038b740 1 -- 172.21.15.67:0/771319722 shutdown_connections 2024-07-31T15:05:46.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.518+0000 7f9b0038b740 1 -- 172.21.15.67:0/771319722 wait complete. 2024-07-31T15:05:46.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.518+0000 7f9b0038b740 1 -- start start 2024-07-31T15:05:46.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.519+0000 7f9b0038b740 1 Processor -- start 2024-07-31T15:05:46.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.519+0000 7f9b0038b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56299900e1b0 0x5629993e8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.519+0000 7f9b0038b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993e88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.519+0000 7f9b0038b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993f4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.519+0000 7f9b0038b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5629991a5700 con 0x562999016390 2024-07-31T15:05:46.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.519+0000 7f9b0038b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5629992e49e0 con 0x56299900e1b0 2024-07-31T15:05:46.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.519+0000 7f9b0038b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5629993eaef0 con 0x562999037070 2024-07-31T15:05:46.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.519+0000 7f9b0038b740 10 client.0 _ll_register_callbacks cb 0x5629992ff4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:46.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.520+0000 7f9af3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:46.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.520+0000 7f9af3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993f4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60178/0 (socket says 172.21.15.67:60178) 2024-07-31T15:05:46.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.520+0000 7f9af3fff640 1 -- 172.21.15.67:0/2795139248 learned_addr learned my addr 172.21.15.67:0/2795139248 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:46.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.520+0000 7f9af37fe640 1 --2- 172.21.15.67:0/2795139248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56299900e1b0 0x5629993e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:46.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.522+0000 7f9af3fff640 1 -- 172.21.15.67:0/2795139248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56299900e1b0 msgr2=0x5629993e8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:46.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.522+0000 7f9af3fff640 1 --2- 172.21.15.67:0/2795139248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56299900e1b0 0x5629993e8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.522+0000 7f9af3fff640 1 -- 172.21.15.67:0/2795139248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 msgr2=0x5629993e88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:46.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.522+0000 7f9af3fff640 1 --2- 172.21.15.67:0/2795139248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993e88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.522+0000 7f9af3fff640 1 -- 172.21.15.67:0/2795139248 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629990a2650 con 0x562999037070 2024-07-31T15:05:46.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.522+0000 7f9af3fff640 1 --2- 172.21.15.67:0/2795139248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993f4270 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0e0ec0 tx=0x7f9adc0a9f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cfcb78cd2fc2f8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:46.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.522+0000 7f9ad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:05:46.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.524+0000 7f9ad2ffd640 1 -- 172.21.15.67:0/2795139248 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0c4e60 con 0x562999037070 2024-07-31T15:05:46.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.524+0000 7f9ad2ffd640 1 -- 172.21.15.67:0/2795139248 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f9adc0a9580 con 0x562999037070 2024-07-31T15:05:46.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.524+0000 7f9b0038b740 1 -- 172.21.15.67:0/2795139248 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562999354b00 con 0x562999037070 2024-07-31T15:05:46.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.525+0000 7f9ad2ffd640 1 -- 172.21.15.67:0/2795139248 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f9adc01d040 con 0x562999037070 2024-07-31T15:05:46.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.525+0000 7f9b0038b740 10 client.28042 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:46.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.525+0000 7f9b0038b740 1 -- 172.21.15.67:0/2795139248 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5629993f7040 con 0x562999037070 2024-07-31T15:05:46.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.526+0000 7f9ad2ffd640 1 -- 172.21.15.67:0/2795139248 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562999354b00 con 0x562999037070 2024-07-31T15:05:46.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.526+0000 7f9ad2ffd640 1 -- 172.21.15.67:0/2795139248 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f9adc020060 con 0x562999037070 2024-07-31T15:05:46.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.526+0000 7f9b0038b740 10 client.28042 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:46.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.526+0000 7f9b0038b740 10 client.28042 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:05:46.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.526+0000 7f9b0038b740 20 client.28042 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:46.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.526+0000 7f9b0038b740 1 --2- 172.21.15.67:0/2795139248 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562998ff59f0 0x5629993faad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.526+0000 7f9b0038b740 4 client.28042 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:05:46.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.526+0000 7f9b0038b740 1 -- 172.21.15.67:0/2795139248 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629993505a0 con 0x562998ff59f0 2024-07-31T15:05:46.531 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/secure_seq.o 2024-07-31T15:05:46.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.531+0000 7f9af37fe640 1 --2- 172.21.15.67:0/2795139248 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562998ff59f0 0x5629993faad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:46.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.540+0000 7f9af37fe640 1 --2- 172.21.15.67:0/2795139248 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562998ff59f0 0x5629993faad0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ac22b2aa8377a28b server_cookie=84aede30d6415f29 in_seq=0 out_seq=0 2024-07-31T15:05:46.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.540+0000 7f9ad2ffd640 10 client.28042 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:05:46.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.541+0000 7f9ad2ffd640 1 -- 172.21.15.67:0/2795139248 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629993505a0 con 0x562998ff59f0 2024-07-31T15:05:46.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.541+0000 7f9ad2ffd640 10 client.28042 handle_command_reply: tid=0 2024-07-31T15:05:46.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.541+0000 7f9b0038b740 1 client.28042 shutdown 2024-07-31T15:05:46.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.541+0000 7f9b0038b740 1 -- 172.21.15.67:0/2795139248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 msgr2=0x5629993f4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:46.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.541+0000 7f9b0038b740 1 --2- 172.21.15.67:0/2795139248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993f4270 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0e0ec0 tx=0x7f9adc0a9f70 comp rx=0 tx=0).stop 2024-07-31T15:05:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.543+0000 7f9b0038b740 1 -- 172.21.15.67:0/2795139248 shutdown_connections 2024-07-31T15:05:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.543+0000 7f9b0038b740 1 --2- 172.21.15.67:0/2795139248 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562998ff59f0 0x5629993faad0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.543+0000 7f9b0038b740 1 --2- 172.21.15.67:0/2795139248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993f4270 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.543+0000 7f9b0038b740 1 --2- 172.21.15.67:0/2795139248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993e88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.543+0000 7f9b0038b740 1 --2- 172.21.15.67:0/2795139248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56299900e1b0 0x5629993e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.543+0000 7f9b0038b740 1 -- 172.21.15.67:0/2795139248 >> 172.21.15.67:0/2795139248 conn(0x56299902b460 msgr2=0x5629993df8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.543+0000 7f9ad2ffd640 0 client.28042 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:05:46.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.544+0000 7f9b0038b740 1 -- 172.21.15.67:0/2795139248 shutdown_connections 2024-07-31T15:05:46.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.544+0000 7f9b0038b740 1 -- 172.21.15.67:0/2795139248 wait complete. 2024-07-31T15:05:46.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.544+0000 7f9b0038b740 20 client.28042 trim_cache size 0 max 16384 2024-07-31T15:05:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.596+0000 7f9b0038b740 1 Processor -- start 2024-07-31T15:05:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.596+0000 7f9b0038b740 1 -- start start 2024-07-31T15:05:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.596+0000 7f9b0038b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56299900e1b0 0x5629993e1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.596+0000 7f9b0038b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993f4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.596+0000 7f9b0038b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.596+0000 7f9b0038b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad400ff80 con 0x562999016390 2024-07-31T15:05:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.596+0000 7f9b0038b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad4011ac0 con 0x56299900e1b0 2024-07-31T15:05:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.596+0000 7f9b0038b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad4010650 con 0x562999037070 2024-07-31T15:05:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.597+0000 7f9af3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.597+0000 7f9af3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993f45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60184/0 (socket says 172.21.15.67:60184) 2024-07-31T15:05:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.597+0000 7f9af3fff640 1 -- 172.21.15.67:0/1093004027 learned_addr learned my addr 172.21.15.67:0/1093004027 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.597+0000 7f9af37fe640 1 --2- 172.21.15.67:0/1093004027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56299900e1b0 0x5629993e1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.597+0000 7f9af3fff640 1 -- 172.21.15.67:0/1093004027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56299900e1b0 msgr2=0x5629993e1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.597+0000 7f9af3fff640 1 --2- 172.21.15.67:0/1093004027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56299900e1b0 0x5629993e1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.597+0000 7f9af3fff640 1 -- 172.21.15.67:0/1093004027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 msgr2=0x5629993f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.597+0000 7f9af3fff640 1 --2- 172.21.15.67:0/1093004027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.597+0000 7f9af3fff640 1 -- 172.21.15.67:0/1093004027 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629993e9420 con 0x562999037070 2024-07-31T15:05:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.597+0000 7f9af3fff640 1 --2- 172.21.15.67:0/1093004027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993f45a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0e1460 tx=0x7f9adc0a9e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87efd2680400610b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.598+0000 7f9ae97fa640 1 -- 172.21.15.67:0/1093004027 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0c4e60 con 0x562999037070 2024-07-31T15:05:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.598+0000 7f9ae97fa640 1 -- 172.21.15.67:0/1093004027 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9adc0c4070 con 0x562999037070 2024-07-31T15:05:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.598+0000 7f9ae97fa640 1 -- 172.21.15.67:0/1093004027 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0a97e0 con 0x562999037070 2024-07-31T15:05:46.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.600+0000 7f9b0038b740 1 -- 172.21.15.67:0/1093004027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 msgr2=0x5629993f45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:46.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.600+0000 7f9b0038b740 1 --2- 172.21.15.67:0/1093004027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993f45a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0e1460 tx=0x7f9adc0a9e30 comp rx=0 tx=0).stop 2024-07-31T15:05:46.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.604+0000 7f9b0038b740 1 -- 172.21.15.67:0/1093004027 shutdown_connections 2024-07-31T15:05:46.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.604+0000 7f9b0038b740 1 --2- 172.21.15.67:0/1093004027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993f45a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.604+0000 7f9b0038b740 1 --2- 172.21.15.67:0/1093004027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.604+0000 7f9b0038b740 1 --2- 172.21.15.67:0/1093004027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56299900e1b0 0x5629993e1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.604+0000 7f9b0038b740 1 -- 172.21.15.67:0/1093004027 >> 172.21.15.67:0/1093004027 conn(0x56299902b460 msgr2=0x5629993e1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:46.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.610+0000 7f9b0038b740 1 -- 172.21.15.67:0/1093004027 shutdown_connections 2024-07-31T15:05:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.612+0000 7f9b0038b740 1 -- 172.21.15.67:0/1093004027 wait complete. 2024-07-31T15:05:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.612+0000 7f9b0038b740 1 -- start start 2024-07-31T15:05:46.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.613+0000 7f9b0038b740 1 Processor -- start 2024-07-31T15:05:46.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.614+0000 7f9b0038b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56299900e1b0 0x5629993f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.614+0000 7f9b0038b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993eb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.614+0000 7f9b0038b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.614+0000 7f9b0038b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad40113f0 con 0x562999016390 2024-07-31T15:05:46.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.614+0000 7f9b0038b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad4010d20 con 0x56299900e1b0 2024-07-31T15:05:46.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.614+0000 7f9b0038b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad400df80 con 0x562999037070 2024-07-31T15:05:46.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.614+0000 7f9b0038b740 10 client.0 _ll_register_callbacks cb 0x7f9adc0aaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:46.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.614+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:46.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.614+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993eb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38378/0 (socket says 172.21.15.67:38378) 2024-07-31T15:05:46.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.614+0000 7f9af2ffd640 1 -- 172.21.15.67:0/885619646 learned_addr learned my addr 172.21.15.67:0/885619646 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:46.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.615+0000 7f9af2ffd640 1 -- 172.21.15.67:0/885619646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 msgr2=0x5629993eb880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:05:46.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.615+0000 7f9af2ffd640 1 --2- 172.21.15.67:0/885619646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993eb880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.615+0000 7f9af2ffd640 1 -- 172.21.15.67:0/885619646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56299900e1b0 msgr2=0x5629993f0a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:05:46.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.615+0000 7f9af2ffd640 1 --2- 172.21.15.67:0/885619646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56299900e1b0 0x5629993f0a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.615+0000 7f9af2ffd640 1 -- 172.21.15.67:0/885619646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562999461290 con 0x562999016390 2024-07-31T15:05:46.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.615+0000 7f9af2ffd640 1 --2- 172.21.15.67:0/885619646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993eb190 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0058c0 tx=0x7f9aec0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75dc040927ab86bd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:46.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.616+0000 7f9ad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:05:46.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.616+0000 7f9ad2ffd640 1 -- 172.21.15.67:0/885619646 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec021070 con 0x562999016390 2024-07-31T15:05:46.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.616+0000 7f9ad2ffd640 1 -- 172.21.15.67:0/885619646 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f9aec004a90 con 0x562999016390 2024-07-31T15:05:46.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.616+0000 7f9b0038b740 1 -- 172.21.15.67:0/885619646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9adc0c4070 con 0x562999016390 2024-07-31T15:05:46.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.620+0000 7f9ad2ffd640 1 -- 172.21.15.67:0/885619646 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f9aec035050 con 0x562999016390 2024-07-31T15:05:46.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.621+0000 7f9b0038b740 10 client.18570 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:46.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.621+0000 7f9b0038b740 1 -- 172.21.15.67:0/885619646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x562999462350 con 0x562999016390 2024-07-31T15:05:46.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.624+0000 7f9ad2ffd640 1 -- 172.21.15.67:0/885619646 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9adc0c4070 con 0x562999016390 2024-07-31T15:05:46.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.624+0000 7f9ad2ffd640 1 -- 172.21.15.67:0/885619646 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f9aec025400 con 0x562999016390 2024-07-31T15:05:46.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.625+0000 7f9b0038b740 10 client.18570 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:46.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.625+0000 7f9b0038b740 10 client.18570 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:05:46.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.625+0000 7f9b0038b740 20 client.18570 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:46.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.625+0000 7f9b0038b740 1 --2- 172.21.15.67:0/885619646 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562998fcada0 0x5629994648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:46.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.625+0000 7f9b0038b740 4 client.18570 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:05:46.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.625+0000 7f9b0038b740 1 -- 172.21.15.67:0/885619646 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56299934dce0 con 0x562998fcada0 2024-07-31T15:05:46.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.629+0000 7f9af37fe640 1 --2- 172.21.15.67:0/885619646 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562998fcada0 0x5629994648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:46.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.643+0000 7f9af37fe640 1 --2- 172.21.15.67:0/885619646 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562998fcada0 0x5629994648c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=26453bfafdb5721c server_cookie=70f2d5e5e9a84296 in_seq=0 out_seq=0 2024-07-31T15:05:46.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.644+0000 7f9ad2ffd640 10 client.18570 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:05:46.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.648+0000 7f9ad2ffd640 1 -- 172.21.15.67:0/885619646 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56299934dce0 con 0x562998fcada0 2024-07-31T15:05:46.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.648+0000 7f9ad2ffd640 10 client.18570 handle_command_reply: tid=0 2024-07-31T15:05:46.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.651+0000 7f9b0038b740 1 client.18570 shutdown 2024-07-31T15:05:46.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.655+0000 7f9b0038b740 1 -- 172.21.15.67:0/885619646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 msgr2=0x5629993eb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:46.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.655+0000 7f9b0038b740 1 --2- 172.21.15.67:0/885619646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993eb190 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0058c0 tx=0x7f9aec0056e0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.660+0000 7f9b0038b740 1 -- 172.21.15.67:0/885619646 shutdown_connections 2024-07-31T15:05:46.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.660+0000 7f9b0038b740 1 --2- 172.21.15.67:0/885619646 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562998fcada0 0x5629994648c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.660+0000 7f9b0038b740 1 --2- 172.21.15.67:0/885619646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562999037070 0x5629993eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.660+0000 7f9b0038b740 1 --2- 172.21.15.67:0/885619646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562999016390 0x5629993eb190 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.660+0000 7f9b0038b740 1 --2- 172.21.15.67:0/885619646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56299900e1b0 0x5629993f0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.660+0000 7f9b0038b740 1 -- 172.21.15.67:0/885619646 >> 172.21.15.67:0/885619646 conn(0x56299902b460 msgr2=0x5629993d9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:46.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.660+0000 7f9ad2ffd640 0 client.18570 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:05:46.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.664+0000 7f9b0038b740 1 -- 172.21.15.67:0/885619646 shutdown_connections 2024-07-31T15:05:46.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.664+0000 7f9b0038b740 1 -- 172.21.15.67:0/885619646 wait complete. 2024-07-31T15:05:46.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.664+0000 7f9b0038b740 20 client.18570 trim_cache size 0 max 16384 2024-07-31T15:05:46.666 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:05:46.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.665+0000 7f9af9e3a640 1 -- 172.21.15.67:0/4138012204 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9ae00b3f50 msgr2=0x7f9ae00b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:46.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.665+0000 7f9af9e3a640 1 --2- 172.21.15.67:0/4138012204 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9ae00b3f50 0x7f9ae00b6460 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9af4110160 tx=0x7f9aec005f50 comp rx=0 tx=0).stop 2024-07-31T15:05:46.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.665+0000 7f9af9e3a640 1 -- 172.21.15.67:0/4138012204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af411c5d0 msgr2=0x7f9af4115850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:46.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.665+0000 7f9af9e3a640 1 --2- 172.21.15.67:0/4138012204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af411c5d0 0x7f9af4115850 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00c710 tx=0x7f9adc00cbd0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.665+0000 7f9af9e3a640 1 -- 172.21.15.67:0/4138012204 shutdown_connections 2024-07-31T15:05:46.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.665+0000 7f9af9e3a640 1 --2- 172.21.15.67:0/4138012204 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9ae00b3f50 0x7f9ae00b6460 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.665+0000 7f9af9e3a640 1 --2- 172.21.15.67:0/4138012204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af411c5d0 0x7f9af4115850 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.665+0000 7f9af9e3a640 1 --2- 172.21.15.67:0/4138012204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9af4111770 0x7f9af41151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.665+0000 7f9af9e3a640 1 --2- 172.21.15.67:0/4138012204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9af410a5b0 0x7f9af411bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:46.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.665+0000 7f9af9e3a640 1 -- 172.21.15.67:0/4138012204 >> 172.21.15.67:0/4138012204 conn(0x7f9af406d380 msgr2=0x7f9af4110400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:46.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.665+0000 7f9af9e3a640 1 -- 172.21.15.67:0/4138012204 shutdown_connections 2024-07-31T15:05:46.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:46.669+0000 7f9af9e3a640 1 -- 172.21.15.67:0/4138012204 wait complete. 2024-07-31T15:05:46.751 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evxface.o 2024-07-31T15:05:46.760 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/trace.o 2024-07-31T15:05:46.810 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/proc.o 2024-07-31T15:05:47.008 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/pid.o 2024-07-31T15:05:47.277 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evxfevnt.o 2024-07-31T15:05:47.362 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/pci/hda/hda_proc.o 2024-07-31T15:05:47.374 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evxfgpe.o 2024-07-31T15:05:47.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:47 smithi067 ceph-mon[32750]: pgmap v863: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 995 KiB/s wr, 228 op/s 2024-07-31T15:05:47.622 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/flow_dissector.o 2024-07-31T15:05:47.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:47 smithi073 ceph-mon[36714]: pgmap v863: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 995 KiB/s wr, 228 op/s 2024-07-31T15:05:47.665 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/evxfregn.o 2024-07-31T15:05:47.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:47 smithi077 ceph-mon[37327]: pgmap v863: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 995 KiB/s wr, 228 op/s 2024-07-31T15:05:47.783 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/xdr.o 2024-07-31T15:05:47.860 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exconcat.o 2024-07-31T15:05:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.869+0000 7fc71359e640 1 -- 172.21.15.67:0/1003376261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc71411c5d0 msgr2=0x7fc71411e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.869+0000 7fc71359e640 1 --2- 172.21.15.67:0/1003376261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc71411c5d0 0x7fc71411e9f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009ca0 tx=0x7fc70c02a8e0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.869+0000 7fc71359e640 1 -- 172.21.15.67:0/1003376261 shutdown_connections 2024-07-31T15:05:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.869+0000 7fc71359e640 1 --2- 172.21.15.67:0/1003376261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc71411c5d0 0x7fc71411e9f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.869+0000 7fc71359e640 1 --2- 172.21.15.67:0/1003376261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71410a330 0x7fc71410a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.869+0000 7fc71359e640 1 --2- 172.21.15.67:0/1003376261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc714111650 0x7fc714109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.869+0000 7fc71359e640 1 -- 172.21.15.67:0/1003376261 >> 172.21.15.67:0/1003376261 conn(0x7fc71406e320 msgr2=0x7fc71406e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.869+0000 7fc71359e640 1 -- 172.21.15.67:0/1003376261 shutdown_connections 2024-07-31T15:05:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.869+0000 7fc71359e640 1 -- 172.21.15.67:0/1003376261 wait complete. 2024-07-31T15:05:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.871+0000 7fc71359e640 1 Processor -- start 2024-07-31T15:05:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.871+0000 7fc71359e640 1 -- start start 2024-07-31T15:05:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.871+0000 7fc71359e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71410a330 0x7fc7141a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.871+0000 7fc71359e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc714111650 0x7fc7141a8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.871+0000 7fc71359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc71411c5d0 0x7fc7141ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.871+0000 7fc71359e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc7141220a0 con 0x7fc71410a330 2024-07-31T15:05:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.871+0000 7fc71359e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc714121f20 con 0x7fc714111650 2024-07-31T15:05:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.871+0000 7fc71359e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc714122220 con 0x7fc71411c5d0 2024-07-31T15:05:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.872+0000 7fc71259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71410a330 0x7fc7141a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.872+0000 7fc71259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71410a330 0x7fc7141a8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38402/0 (socket says 172.21.15.67:38402) 2024-07-31T15:05:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.872+0000 7fc71259c640 1 -- 172.21.15.67:0/2197695753 learned_addr learned my addr 172.21.15.67:0/2197695753 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.872+0000 7fc711d9b640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc714111650 0x7fc7141a8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.872+0000 7fc712d9d640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc71411c5d0 0x7fc7141ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.872+0000 7fc71259c640 1 -- 172.21.15.67:0/2197695753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc71411c5d0 msgr2=0x7fc7141ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.872+0000 7fc71259c640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc71411c5d0 0x7fc7141ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.872+0000 7fc71259c640 1 -- 172.21.15.67:0/2197695753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc714111650 msgr2=0x7fc7141a8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.872+0000 7fc71259c640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc714111650 0x7fc7141a8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.872+0000 7fc71259c640 1 -- 172.21.15.67:0/2197695753 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7141ae8c0 con 0x7fc71410a330 2024-07-31T15:05:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.872+0000 7fc711d9b640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc714111650 0x7fc7141a8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:05:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.873+0000 7fc71259c640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71410a330 0x7fc7141a8a20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc70400c9e0 tx=0x7fc70400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.874+0000 7fc7037fe640 1 -- 172.21.15.67:0/2197695753 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704014070 con 0x7fc71410a330 2024-07-31T15:05:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.874+0000 7fc7037fe640 1 -- 172.21.15.67:0/2197695753 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc704004440 con 0x7fc71410a330 2024-07-31T15:05:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.874+0000 7fc71359e640 1 -- 172.21.15.67:0/2197695753 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7141aebb0 con 0x7fc71410a330 2024-07-31T15:05:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.874+0000 7fc71359e640 1 -- 172.21.15.67:0/2197695753 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc714111db0 con 0x7fc71410a330 2024-07-31T15:05:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.876+0000 7fc7037fe640 1 -- 172.21.15.67:0/2197695753 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704003e60 con 0x7fc71410a330 2024-07-31T15:05:47.879 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs2xdr.o 2024-07-31T15:05:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.878+0000 7fc71fa21740 1 Processor -- start 2024-07-31T15:05:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.879+0000 7fc71fa21740 1 -- start start 2024-07-31T15:05:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.880+0000 7fc71fa21740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b7b1b0 0x55de45f1dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.880+0000 7fc71fa21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b522f0 0x55de45f1e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.880+0000 7fc71fa21740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b5a4d0 0x55de45f1e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.880+0000 7fc71fa21740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55de45e933b0 con 0x55de45b5a4d0 2024-07-31T15:05:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.880+0000 7fc71fa21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55de45e828b0 con 0x55de45b522f0 2024-07-31T15:05:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.880+0000 7fc71fa21740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55de45e868f0 con 0x55de45b7b1b0 2024-07-31T15:05:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.880+0000 7fc711d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b5a4d0 0x55de45f1e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.881+0000 7fc712d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b7b1b0 0x55de45f1dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.881+0000 7fc712d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b7b1b0 0x55de45f1dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60222/0 (socket says 172.21.15.67:60222) 2024-07-31T15:05:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.881+0000 7fc712d9d640 1 -- 172.21.15.67:0/1409581598 learned_addr learned my addr 172.21.15.67:0/1409581598 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.881+0000 7fc7037fe640 1 -- 172.21.15.67:0/2197695753 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc704017070 con 0x7fc71410a330 2024-07-31T15:05:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.882+0000 7fc711d9b640 1 -- 172.21.15.67:0/1409581598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b7b1b0 msgr2=0x55de45f1dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.882+0000 7fc711d9b640 1 --2- 172.21.15.67:0/1409581598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b7b1b0 0x55de45f1dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.882+0000 7fc711d9b640 1 -- 172.21.15.67:0/1409581598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b522f0 msgr2=0x55de45f1e420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:05:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.882+0000 7fc711d9b640 1 --2- 172.21.15.67:0/1409581598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b522f0 0x55de45f1e420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.882+0000 7fc711d9b640 1 -- 172.21.15.67:0/1409581598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de45e299a0 con 0x55de45b5a4d0 2024-07-31T15:05:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.883+0000 7fc7037fe640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6fc0b3f50 0x7fc6fc0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.884+0000 7fc7037fe640 1 -- 172.21.15.67:0/2197695753 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fc7040e3d70 con 0x7fc71410a330 2024-07-31T15:05:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.884+0000 7fc711d9b640 1 --2- 172.21.15.67:0/1409581598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b5a4d0 0x55de45f1e960 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc7080058c0 tx=0x7fc70800fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad296d4640ea3ae server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.885+0000 7fc712d9d640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6fc0b3f50 0x7fc6fc0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.885+0000 7fc700ff9640 1 -- 172.21.15.67:0/1409581598 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70801d070 con 0x55de45b5a4d0 2024-07-31T15:05:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.885+0000 7fc700ff9640 1 -- 172.21.15.67:0/1409581598 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70800dc10 con 0x55de45b5a4d0 2024-07-31T15:05:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.887+0000 7fc712d9d640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6fc0b3f50 0x7fc6fc0b6460 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc70c006010 tx=0x7fc70c02ade0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.887+0000 7fc71fa21740 1 -- 172.21.15.67:0/1409581598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b5a4d0 msgr2=0x55de45f1e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.887+0000 7fc71fa21740 1 --2- 172.21.15.67:0/1409581598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b5a4d0 0x55de45f1e960 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc7080058c0 tx=0x7fc70800fa20 comp rx=0 tx=0).stop 2024-07-31T15:05:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.888+0000 7fc71fa21740 1 -- 172.21.15.67:0/1409581598 shutdown_connections 2024-07-31T15:05:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.888+0000 7fc71fa21740 1 --2- 172.21.15.67:0/1409581598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b5a4d0 0x55de45f1e960 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.888+0000 7fc71fa21740 1 --2- 172.21.15.67:0/1409581598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b522f0 0x55de45f1e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.888+0000 7fc71fa21740 1 --2- 172.21.15.67:0/1409581598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b7b1b0 0x55de45f1dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.888+0000 7fc71fa21740 1 -- 172.21.15.67:0/1409581598 >> 172.21.15.67:0/1409581598 conn(0x55de45b6f300 msgr2=0x55de45f1bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.889+0000 7fc71fa21740 1 -- 172.21.15.67:0/1409581598 shutdown_connections 2024-07-31T15:05:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.889+0000 7fc71fa21740 1 -- 172.21.15.67:0/1409581598 wait complete. 2024-07-31T15:05:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.889+0000 7fc71fa21740 1 -- start start 2024-07-31T15:05:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.890+0000 7fc71fa21740 1 Processor -- start 2024-07-31T15:05:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.890+0000 7fc71fa21740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b522f0 0x55de45f29740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.890+0000 7fc71fa21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b5a4d0 0x55de45f29d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.890+0000 7fc71fa21740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b7b1b0 0x55de45f35a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.890+0000 7fc71fa21740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55de45dbcab0 con 0x55de45b522f0 2024-07-31T15:05:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.890+0000 7fc71fa21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55de45e0acf0 con 0x55de45b5a4d0 2024-07-31T15:05:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.890+0000 7fc71fa21740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55de45f2c6b0 con 0x55de45b7b1b0 2024-07-31T15:05:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.890+0000 7fc71fa21740 10 client.0 _ll_register_callbacks cb 0x55de45e7c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.890+0000 7fc712d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b7b1b0 0x55de45f35a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.891+0000 7fc712d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b7b1b0 0x55de45f35a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60234/0 (socket says 172.21.15.67:60234) 2024-07-31T15:05:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.891+0000 7fc712d9d640 1 -- 172.21.15.67:0/2456979458 learned_addr learned my addr 172.21.15.67:0/2456979458 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.891+0000 7fc711d9b640 1 --2- 172.21.15.67:0/2456979458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b5a4d0 0x55de45f29d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.891+0000 7fc712d9d640 1 -- 172.21.15.67:0/2456979458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b5a4d0 msgr2=0x55de45f29d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.891+0000 7fc712d9d640 1 --2- 172.21.15.67:0/2456979458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b5a4d0 0x55de45f29d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.891+0000 7fc71259c640 1 --2- 172.21.15.67:0/2456979458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b522f0 0x55de45f29740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.891+0000 7fc712d9d640 1 -- 172.21.15.67:0/2456979458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b522f0 msgr2=0x55de45f29740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.891+0000 7fc712d9d640 1 --2- 172.21.15.67:0/2456979458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b522f0 0x55de45f29740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.891+0000 7fc712d9d640 1 -- 172.21.15.67:0/2456979458 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de45be6790 con 0x55de45b7b1b0 2024-07-31T15:05:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.891+0000 7fc71259c640 1 --2- 172.21.15.67:0/2456979458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b522f0 0x55de45f29740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:05:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.891+0000 7fc712d9d640 1 --2- 172.21.15.67:0/2456979458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b7b1b0 0x55de45f35a80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc70c031e90 tx=0x7fc70c02a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de24e09b159bf3ce server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.891+0000 7fc6f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:05:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.892+0000 7fc6f9ffb640 1 -- 172.21.15.67:0/2456979458 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0045c0 con 0x55de45b7b1b0 2024-07-31T15:05:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.892+0000 7fc6f9ffb640 1 -- 172.21.15.67:0/2456979458 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fc70c004760 con 0x55de45b7b1b0 2024-07-31T15:05:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.892+0000 7fc71fa21740 1 -- 172.21.15.67:0/2456979458 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70800dc10 con 0x55de45b7b1b0 2024-07-31T15:05:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.892+0000 7fc6f9ffb640 1 -- 172.21.15.67:0/2456979458 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fc70c0305e0 con 0x55de45b7b1b0 2024-07-31T15:05:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.892+0000 7fc71fa21740 10 client.28054 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.892+0000 7fc71fa21740 1 -- 172.21.15.67:0/2456979458 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55de45c5f620 con 0x55de45b7b1b0 2024-07-31T15:05:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.893+0000 7fc6f9ffb640 1 -- 172.21.15.67:0/2456979458 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70800dc10 con 0x55de45b7b1b0 2024-07-31T15:05:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.893+0000 7fc6f9ffb640 1 -- 172.21.15.67:0/2456979458 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fc70c050070 con 0x55de45b7b1b0 2024-07-31T15:05:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.893+0000 7fc71fa21740 10 client.28054 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.893+0000 7fc71fa21740 10 client.28054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:05:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.893+0000 7fc71fa21740 20 client.28054 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.893+0000 7fc71fa21740 1 --2- 172.21.15.67:0/2456979458 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55de45b39d80 0x55de45f3bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.893+0000 7fc71fa21740 4 client.28054 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:05:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.893+0000 7fc71fa21740 1 -- 172.21.15.67:0/2456979458 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de45e95300 con 0x55de45b39d80 2024-07-31T15:05:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.893+0000 7fc71259c640 1 --2- 172.21.15.67:0/2456979458 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55de45b39d80 0x55de45f3bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.894+0000 7fc6f9ffb640 10 client.28054 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.894+0000 7fc71259c640 1 --2- 172.21.15.67:0/2456979458 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55de45b39d80 0x55de45f3bac0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fea954964357eca server_cookie=aa5dda04c5b40ebe in_seq=0 out_seq=0 2024-07-31T15:05:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.895+0000 7fc6f9ffb640 1 -- 172.21.15.67:0/2456979458 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de45e95300 con 0x55de45b39d80 2024-07-31T15:05:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.895+0000 7fc6f9ffb640 10 client.28054 handle_command_reply: tid=0 2024-07-31T15:05:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.895+0000 7fc71fa21740 1 client.28054 shutdown 2024-07-31T15:05:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.897+0000 7fc71fa21740 1 -- 172.21.15.67:0/2456979458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b7b1b0 msgr2=0x55de45f35a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.897+0000 7fc71fa21740 1 --2- 172.21.15.67:0/2456979458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b7b1b0 0x55de45f35a80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc70c031e90 tx=0x7fc70c02a8e0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.898+0000 7fc71fa21740 1 -- 172.21.15.67:0/2456979458 shutdown_connections 2024-07-31T15:05:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.898+0000 7fc71fa21740 1 --2- 172.21.15.67:0/2456979458 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55de45b39d80 0x55de45f3bac0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.898+0000 7fc71fa21740 1 --2- 172.21.15.67:0/2456979458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b7b1b0 0x55de45f35a80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.898+0000 7fc71fa21740 1 --2- 172.21.15.67:0/2456979458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b5a4d0 0x55de45f29d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.898+0000 7fc71fa21740 1 --2- 172.21.15.67:0/2456979458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b522f0 0x55de45f29740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.898+0000 7fc71fa21740 1 -- 172.21.15.67:0/2456979458 >> 172.21.15.67:0/2456979458 conn(0x55de45b6f300 msgr2=0x55de45f1ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.898+0000 7fc6f9ffb640 0 client.28054 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.898+0000 7fc71fa21740 1 -- 172.21.15.67:0/2456979458 shutdown_connections 2024-07-31T15:05:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.898+0000 7fc71fa21740 1 -- 172.21.15.67:0/2456979458 wait complete. 2024-07-31T15:05:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.898+0000 7fc71fa21740 20 client.28054 trim_cache size 0 max 16384 2024-07-31T15:05:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.934+0000 7fc71fa21740 1 Processor -- start 2024-07-31T15:05:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.934+0000 7fc71fa21740 1 -- start start 2024-07-31T15:05:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.934+0000 7fc71fa21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b522f0 0x55de45f23fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.934+0000 7fc71fa21740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b5a4d0 0x55de45f28610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.934+0000 7fc71fa21740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b7b1b0 0x55de45f1be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.935+0000 7fc71fa21740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f400ff80 con 0x55de45b7b1b0 2024-07-31T15:05:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.935+0000 7fc71fa21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4011ac0 con 0x55de45b522f0 2024-07-31T15:05:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.935+0000 7fc71fa21740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4010650 con 0x55de45b5a4d0 2024-07-31T15:05:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.935+0000 7fc711d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b5a4d0 0x55de45f28610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.935+0000 7fc711d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b5a4d0 0x55de45f28610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60244/0 (socket says 172.21.15.67:60244) 2024-07-31T15:05:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.935+0000 7fc711d9b640 1 -- 172.21.15.67:0/2913425016 learned_addr learned my addr 172.21.15.67:0/2913425016 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.935+0000 7fc71259c640 1 --2- 172.21.15.67:0/2913425016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b522f0 0x55de45f23fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.935+0000 7fc712d9d640 1 --2- 172.21.15.67:0/2913425016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b7b1b0 0x55de45f1be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.936+0000 7fc712d9d640 1 -- 172.21.15.67:0/2913425016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b5a4d0 msgr2=0x55de45f28610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.936+0000 7fc712d9d640 1 --2- 172.21.15.67:0/2913425016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b5a4d0 0x55de45f28610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.936+0000 7fc712d9d640 1 -- 172.21.15.67:0/2913425016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b522f0 msgr2=0x55de45f23fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.936+0000 7fc712d9d640 1 --2- 172.21.15.67:0/2913425016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b522f0 0x55de45f23fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.936+0000 7fc712d9d640 1 -- 172.21.15.67:0/2913425016 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de45f20410 con 0x55de45b7b1b0 2024-07-31T15:05:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.937+0000 7fc712d9d640 1 --2- 172.21.15.67:0/2913425016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b7b1b0 0x55de45f1be00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc70c02aed0 tx=0x7fc70c052440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367f20f24e9c58cc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.938+0000 7fc6fbfff640 1 -- 172.21.15.67:0/2913425016 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c063070 con 0x55de45b7b1b0 2024-07-31T15:05:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.938+0000 7fc6fbfff640 1 -- 172.21.15.67:0/2913425016 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70c036040 con 0x55de45b7b1b0 2024-07-31T15:05:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.939+0000 7fc71fa21740 1 -- 172.21.15.67:0/2913425016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b7b1b0 msgr2=0x55de45f1be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.939+0000 7fc71fa21740 1 --2- 172.21.15.67:0/2913425016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b7b1b0 0x55de45f1be00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc70c02aed0 tx=0x7fc70c052440 comp rx=0 tx=0).stop 2024-07-31T15:05:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.940+0000 7fc71fa21740 1 -- 172.21.15.67:0/2913425016 shutdown_connections 2024-07-31T15:05:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.940+0000 7fc71fa21740 1 --2- 172.21.15.67:0/2913425016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b7b1b0 0x55de45f1be00 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.940+0000 7fc71fa21740 1 --2- 172.21.15.67:0/2913425016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b5a4d0 0x55de45f28610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.940+0000 7fc71fa21740 1 --2- 172.21.15.67:0/2913425016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b522f0 0x55de45f23fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.940+0000 7fc71fa21740 1 -- 172.21.15.67:0/2913425016 >> 172.21.15.67:0/2913425016 conn(0x55de45b6f300 msgr2=0x55de45f1b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.952+0000 7fc71fa21740 1 -- 172.21.15.67:0/2913425016 shutdown_connections 2024-07-31T15:05:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.952+0000 7fc71fa21740 1 -- 172.21.15.67:0/2913425016 wait complete. 2024-07-31T15:05:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.952+0000 7fc71fa21740 1 -- start start 2024-07-31T15:05:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.952+0000 7fc71fa21740 1 Processor -- start 2024-07-31T15:05:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.953+0000 7fc71fa21740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b522f0 0x55de45f30400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.953+0000 7fc71fa21740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b5a4d0 0x55de45f309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.953+0000 7fc71fa21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b0f1c0 0x55de45f9a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.953+0000 7fc71fa21740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f40113f0 con 0x55de45b5a4d0 2024-07-31T15:05:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.953+0000 7fc71fa21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4010d20 con 0x55de45b0f1c0 2024-07-31T15:05:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.953+0000 7fc71fa21740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f400df80 con 0x55de45b522f0 2024-07-31T15:05:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.953+0000 7fc71fa21740 10 client.0 _ll_register_callbacks cb 0x7fc70c0093f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.953+0000 7fc711d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b5a4d0 0x55de45f309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.953+0000 7fc71259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b522f0 0x55de45f30400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.954+0000 7fc71259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b522f0 0x55de45f30400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60248/0 (socket says 172.21.15.67:60248) 2024-07-31T15:05:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.954+0000 7fc711d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b5a4d0 0x55de45f309e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38452/0 (socket says 172.21.15.67:38452) 2024-07-31T15:05:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.954+0000 7fc71259c640 1 -- 172.21.15.67:0/1372100041 learned_addr learned my addr 172.21.15.67:0/1372100041 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.954+0000 7fc712d9d640 1 --2- 172.21.15.67:0/1372100041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b0f1c0 0x55de45f9a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.954+0000 7fc71259c640 1 -- 172.21.15.67:0/1372100041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b0f1c0 msgr2=0x55de45f9a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.954+0000 7fc71259c640 1 --2- 172.21.15.67:0/1372100041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b0f1c0 0x55de45f9a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.954+0000 7fc71259c640 1 -- 172.21.15.67:0/1372100041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b5a4d0 msgr2=0x55de45f309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.954+0000 7fc71259c640 1 --2- 172.21.15.67:0/1372100041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b5a4d0 0x55de45f309e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.954+0000 7fc71259c640 1 -- 172.21.15.67:0/1372100041 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de45f9a920 con 0x55de45b522f0 2024-07-31T15:05:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.954+0000 7fc711d9b640 1 --2- 172.21.15.67:0/1372100041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b5a4d0 0x55de45f309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:05:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.955+0000 7fc71259c640 1 --2- 172.21.15.67:0/1372100041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b522f0 0x55de45f30400 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc7040a7820 tx=0x7fc704003a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6f843d6ce7e1e8e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.964+0000 7fc6f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:05:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.964+0000 7fc6f9ffb640 1 -- 172.21.15.67:0/1372100041 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70401e040 con 0x55de45b522f0 2024-07-31T15:05:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.964+0000 7fc6f9ffb640 1 -- 172.21.15.67:0/1372100041 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fc7040e1370 con 0x55de45b522f0 2024-07-31T15:05:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.964+0000 7fc71fa21740 1 -- 172.21.15.67:0/1372100041 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55de45f25280 con 0x55de45b522f0 2024-07-31T15:05:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.974+0000 7fc6f9ffb640 1 -- 172.21.15.67:0/1372100041 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fc7040e1570 con 0x55de45b522f0 2024-07-31T15:05:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.974+0000 7fc71fa21740 10 client.28060 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.974+0000 7fc71fa21740 1 -- 172.21.15.67:0/1372100041 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55de45fa2f10 con 0x55de45b522f0 2024-07-31T15:05:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.974+0000 7fc6f9ffb640 1 -- 172.21.15.67:0/1372100041 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55de45f25280 con 0x55de45b522f0 2024-07-31T15:05:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.975+0000 7fc6f9ffb640 1 -- 172.21.15.67:0/1372100041 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fc7040a6040 con 0x55de45b522f0 2024-07-31T15:05:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.976+0000 7fc71fa21740 10 client.28060 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.976+0000 7fc71fa21740 10 client.28060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:05:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.976+0000 7fc71fa21740 20 client.28060 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.976+0000 7fc71fa21740 1 --2- 172.21.15.67:0/1372100041 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55de45fa3690 0x55de45fa5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.976+0000 7fc71fa21740 4 client.28060 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:05:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.976+0000 7fc71fa21740 1 -- 172.21.15.67:0/1372100041 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de45e90da0 con 0x55de45fa3690 2024-07-31T15:05:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.976+0000 7fc711d9b640 1 --2- 172.21.15.67:0/1372100041 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55de45fa3690 0x55de45fa5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.977+0000 7fc6f9ffb640 10 client.28060 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:47.977+0000 7fc711d9b640 1 --2- 172.21.15.67:0/1372100041 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55de45fa3690 0x55de45fa5ab0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8057a1db712a297e server_cookie=4d9e2f86620ca50 in_seq=0 out_seq=0 2024-07-31T15:05:48.138 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exconfig.o 2024-07-31T15:05:48.391 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/pci/hda/hda_hwdep.o 2024-07-31T15:05:48.483 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T15:05:48.610 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exconvrt.o 2024-07-31T15:05:48.713 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/task_work.o 2024-07-31T15:05:49.311 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/clnt4xdr.o 2024-07-31T15:05:49.393 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/excreate.o 2024-07-31T15:05:49.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:49 smithi067 ceph-mon[32750]: pgmap v864: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 757 KiB/s wr, 164 op/s 2024-07-31T15:05:49.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.527+0000 7fabece81640 1 -- 172.21.15.67:0/556132188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabe811c660 msgr2=0x7fabe811eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:49.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.527+0000 7fabece81640 1 --2- 172.21.15.67:0/556132188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabe811c660 0x7fabe811eb70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009860 tx=0x7fabd802a380 comp rx=0 tx=0).stop 2024-07-31T15:05:49.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.528+0000 7fabece81640 1 -- 172.21.15.67:0/556132188 shutdown_connections 2024-07-31T15:05:49.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.528+0000 7fabece81640 1 --2- 172.21.15.67:0/556132188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabe811c660 0x7fabe811eb70 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.528+0000 7fabece81640 1 --2- 172.21.15.67:0/556132188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabe810a410 0x7fabe810a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.528+0000 7fabece81640 1 --2- 172.21.15.67:0/556132188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabe8111970 0x7fabe8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.528+0000 7fabece81640 1 -- 172.21.15.67:0/556132188 >> 172.21.15.67:0/556132188 conn(0x7fabe80708f0 msgr2=0x7fabe8070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:49.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.536+0000 7fabece81640 1 -- 172.21.15.67:0/556132188 shutdown_connections 2024-07-31T15:05:49.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.544+0000 7fabece81640 1 -- 172.21.15.67:0/556132188 wait complete. 2024-07-31T15:05:49.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.545+0000 7fabece81640 1 Processor -- start 2024-07-31T15:05:49.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.549+0000 7fabece81640 1 -- start start 2024-07-31T15:05:49.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.549+0000 7fabece81640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabe810a410 0x7fabe8119dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:49.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.549+0000 7fabece81640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabe8111970 0x7fabe8113090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:49.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.549+0000 7fabece81640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabe811c660 0x7fabe81135d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:49.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.549+0000 7fabece81640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81223b0 con 0x7fabe810a410 2024-07-31T15:05:49.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.549+0000 7fabece81640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8122230 con 0x7fabe8111970 2024-07-31T15:05:49.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.549+0000 7fabece81640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8122530 con 0x7fabe811c660 2024-07-31T15:05:49.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.551+0000 7fabe659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabe810a410 0x7fabe8119dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:49.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.552+0000 7fabe659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabe810a410 0x7fabe8119dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38470/0 (socket says 172.21.15.67:38470) 2024-07-31T15:05:49.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.552+0000 7fabe659c640 1 -- 172.21.15.67:0/3807292971 learned_addr learned my addr 172.21.15.67:0/3807292971 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:49.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.552+0000 7fabe659c640 1 -- 172.21.15.67:0/3807292971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabe811c660 msgr2=0x7fabe81135d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:05:49.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.552+0000 7fabe659c640 1 --2- 172.21.15.67:0/3807292971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabe811c660 0x7fabe81135d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.552+0000 7fabe659c640 1 -- 172.21.15.67:0/3807292971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabe8111970 msgr2=0x7fabe8113090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:49.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.553+0000 7fabe5d9b640 1 --2- 172.21.15.67:0/3807292971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabe8111970 0x7fabe8113090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.557+0000 7fabe659c640 1 --2- 172.21.15.67:0/3807292971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabe8111970 0x7fabe8113090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.557+0000 7fabe659c640 1 -- 172.21.15.67:0/3807292971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe8113dd0 con 0x7fabe810a410 2024-07-31T15:05:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.557+0000 7fabe659c640 1 --2- 172.21.15.67:0/3807292971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabe810a410 0x7fabe8119dc0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00b3d0 tx=0x7fabdc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.558+0000 7fabe5d9b640 1 --2- 172.21.15.67:0/3807292971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabe8111970 0x7fabe8113090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:05:49.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.560+0000 7fabd77fe640 1 -- 172.21.15.67:0/3807292971 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc015020 con 0x7fabe810a410 2024-07-31T15:05:49.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.560+0000 7fabd77fe640 1 -- 172.21.15.67:0/3807292971 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabdc011ce0 con 0x7fabe810a410 2024-07-31T15:05:49.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.560+0000 7fabd77fe640 1 -- 172.21.15.67:0/3807292971 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc00f2e0 con 0x7fabe810a410 2024-07-31T15:05:49.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.562+0000 7fabece81640 1 -- 172.21.15.67:0/3807292971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe8118a90 con 0x7fabe810a410 2024-07-31T15:05:49.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.562+0000 7fabece81640 1 -- 172.21.15.67:0/3807292971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe8118f80 con 0x7fabe810a410 2024-07-31T15:05:49.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.568+0000 7fabd57fa640 1 -- 172.21.15.67:0/3807292971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc80052c0 con 0x7fabe810a410 2024-07-31T15:05:49.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.574+0000 7fabd77fe640 1 -- 172.21.15.67:0/3807292971 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fabdc018020 con 0x7fabe810a410 2024-07-31T15:05:49.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.575+0000 7fabd77fe640 1 --2- 172.21.15.67:0/3807292971 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fabd00b3f50 0x7fabd00b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:49.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.576+0000 7fabd77fe640 1 -- 172.21.15.67:0/3807292971 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fabdc0e38a0 con 0x7fabe810a410 2024-07-31T15:05:49.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.576+0000 7fabe5d9b640 1 --2- 172.21.15.67:0/3807292971 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fabd00b3f50 0x7fabd00b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:49.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.581+0000 7fabe5d9b640 1 --2- 172.21.15.67:0/3807292971 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fabd00b3f50 0x7fabd00b6460 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fabe811ef70 tx=0x7fabe00095c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:49.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:49.588+0000 7fabd77fe640 1 -- 172.21.15.67:0/3807292971 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fabdc010e90 con 0x7fabe810a410 2024-07-31T15:05:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.634+0000 7fc6f9ffb640 1 -- 172.21.15.67:0/1372100041 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de45e90da0 con 0x55de45fa3690 2024-07-31T15:05:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.634+0000 7fc6f9ffb640 10 client.28060 handle_command_reply: tid=0 2024-07-31T15:05:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.634+0000 7fc71fa21740 1 client.28060 shutdown 2024-07-31T15:05:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.635+0000 7fc71fa21740 1 -- 172.21.15.67:0/1372100041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b522f0 msgr2=0x55de45f30400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.635+0000 7fc71fa21740 1 --2- 172.21.15.67:0/1372100041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b522f0 0x55de45f30400 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc7040a7820 tx=0x7fc704003a60 comp rx=0 tx=0).stop 2024-07-31T15:05:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.635+0000 7fc71fa21740 1 -- 172.21.15.67:0/1372100041 shutdown_connections 2024-07-31T15:05:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.635+0000 7fc71fa21740 1 --2- 172.21.15.67:0/1372100041 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55de45fa3690 0x55de45fa5ab0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.635+0000 7fc71fa21740 1 --2- 172.21.15.67:0/1372100041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de45b0f1c0 0x55de45f9a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.635+0000 7fc71fa21740 1 --2- 172.21.15.67:0/1372100041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55de45b5a4d0 0x55de45f309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.635+0000 7fc71fa21740 1 --2- 172.21.15.67:0/1372100041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de45b522f0 0x55de45f30400 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.635+0000 7fc71fa21740 1 -- 172.21.15.67:0/1372100041 >> 172.21.15.67:0/1372100041 conn(0x55de45b6f300 msgr2=0x55de45f2aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.635+0000 7fc6f9ffb640 0 client.28060 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.635+0000 7fc71fa21740 1 -- 172.21.15.67:0/1372100041 shutdown_connections 2024-07-31T15:05:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.635+0000 7fc71fa21740 1 -- 172.21.15.67:0/1372100041 wait complete. 2024-07-31T15:05:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.635+0000 7fc71fa21740 20 client.28060 trim_cache size 0 max 16384 2024-07-31T15:05:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:05:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:05:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:05:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:05:49.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:49 smithi073 ceph-mon[36714]: pgmap v864: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 757 KiB/s wr, 164 op/s 2024-07-31T15:05:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.646+0000 7fc71359e640 1 -- 172.21.15.67:0/2197695753 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6fc0b3f50 msgr2=0x7fc6fc0b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.646+0000 7fc71359e640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6fc0b3f50 0x7fc6fc0b6460 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc70c006010 tx=0x7fc70c02ade0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.646+0000 7fc71359e640 1 -- 172.21.15.67:0/2197695753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71410a330 msgr2=0x7fc7141a8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.646+0000 7fc71359e640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71410a330 0x7fc7141a8a20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc70400c9e0 tx=0x7fc70400cea0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.652+0000 7fc71359e640 1 -- 172.21.15.67:0/2197695753 shutdown_connections 2024-07-31T15:05:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.652+0000 7fc71359e640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc6fc0b3f50 0x7fc6fc0b6460 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.652+0000 7fc71359e640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc71411c5d0 0x7fc7141ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.652+0000 7fc71359e640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc714111650 0x7fc7141a8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.652+0000 7fc71359e640 1 --2- 172.21.15.67:0/2197695753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc71410a330 0x7fc7141a8a20 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.652+0000 7fc71359e640 1 -- 172.21.15.67:0/2197695753 >> 172.21.15.67:0/2197695753 conn(0x7fc71406e320 msgr2=0x7fc71411a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.652+0000 7fc71359e640 1 -- 172.21.15.67:0/2197695753 shutdown_connections 2024-07-31T15:05:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:05:49.652+0000 7fc71359e640 1 -- 172.21.15.67:0/2197695753 wait complete. 2024-07-31T15:05:49.665 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:05:49.718 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/pci/hda/hda_intel.o 2024-07-31T15:05:49.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:49 smithi077 ceph-mon[37327]: pgmap v864: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 757 KiB/s wr, 164 op/s 2024-07-31T15:05:49.760 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/extable.o 2024-07-31T15:05:49.891 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs3super.o 2024-07-31T15:05:49.938 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/params.o 2024-07-31T15:05:49.985 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exdebug.o 2024-07-31T15:05:50.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:50.101+0000 7fabd57fa640 1 -- 172.21.15.67:0/3807292971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabc8005880 con 0x7fabe810a410 2024-07-31T15:05:50.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:50.103+0000 7fabd77fe640 1 -- 172.21.15.67:0/3807292971 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+11959 (secure 0 0 0) 0x7fabdc0aa8f0 con 0x7fabe810a410 2024-07-31T15:05:50.113 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:05:50.113 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":63,"btime":"2024-07-31T15:03:07:099477+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:03:06.671820+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/773662454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":773662454},{"type":"v1","addr":"172.21.15.73:6835","nonce":773662454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:05:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 63 2024-07-31T15:05:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:50.125+0000 7fabd57fa640 1 -- 172.21.15.67:0/3807292971 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fabd00b3f50 msgr2=0x7fabd00b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:50.125+0000 7fabd57fa640 1 --2- 172.21.15.67:0/3807292971 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fabd00b3f50 0x7fabd00b6460 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fabe811ef70 tx=0x7fabe00095c0 comp rx=0 tx=0).stop 2024-07-31T15:05:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:50.125+0000 7fabd57fa640 1 -- 172.21.15.67:0/3807292971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabe810a410 msgr2=0x7fabe8119dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:50.125+0000 7fabd57fa640 1 --2- 172.21.15.67:0/3807292971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabe810a410 0x7fabe8119dc0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00b3d0 tx=0x7fabdc00b890 comp rx=0 tx=0).stop 2024-07-31T15:05:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:50.126+0000 7fabd57fa640 1 -- 172.21.15.67:0/3807292971 shutdown_connections 2024-07-31T15:05:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:50.126+0000 7fabd57fa640 1 --2- 172.21.15.67:0/3807292971 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fabd00b3f50 0x7fabd00b6460 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:50.126+0000 7fabd57fa640 1 --2- 172.21.15.67:0/3807292971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabe811c660 0x7fabe81135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:50.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:50.126+0000 7fabd57fa640 1 --2- 172.21.15.67:0/3807292971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabe8111970 0x7fabe8113090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:50.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:50.126+0000 7fabd57fa640 1 --2- 172.21.15.67:0/3807292971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabe810a410 0x7fabe8119dc0 secure :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00b3d0 tx=0x7fabdc00b890 comp rx=0 tx=0).stop 2024-07-31T15:05:50.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:50.126+0000 7fabd57fa640 1 -- 172.21.15.67:0/3807292971 >> 172.21.15.67:0/3807292971 conn(0x7fabe80708f0 msgr2=0x7fabe811ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:50.132+0000 7fabd57fa640 1 -- 172.21.15.67:0/3807292971 shutdown_connections 2024-07-31T15:05:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:50.135+0000 7fabd57fa640 1 -- 172.21.15.67:0/3807292971 wait complete. 2024-07-31T15:05:50.443 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/xdr4.o 2024-07-31T15:05:50.489 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/memcat_p.o 2024-07-31T15:05:50.498 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exdump.o 2024-07-31T15:05:50.514 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:50 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3807292971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:05:50.635 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/sysctl_net_core.o 2024-07-31T15:05:50.639 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/dev.o 2024-07-31T15:05:50.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:50 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3807292971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:05:50.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:50 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3807292971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:05:50.794 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs3client.o 2024-07-31T15:05:50.806 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/kthread.o 2024-07-31T15:05:50.816 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/nmi_backtrace.o 2024-07-31T15:05:50.896 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exfield.o 2024-07-31T15:05:51.027 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/hda/built-in.a 2024-07-31T15:05:51.066 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/ice1712/built-in.a 2024-07-31T15:05:51.089 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/korg1212/built-in.a 2024-07-31T15:05:51.119 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/mixart/built-in.a 2024-07-31T15:05:51.142 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/nm256/built-in.a 2024-07-31T15:05:51.165 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/oxygen/built-in.a 2024-07-31T15:05:51.188 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/pcxhr/built-in.a 2024-07-31T15:05:51.211 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/riptide/built-in.a 2024-07-31T15:05:51.271 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/rme9652/built-in.a 2024-07-31T15:05:51.294 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exfldio.o 2024-07-31T15:05:51.312 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/trident/built-in.a 2024-07-31T15:05:51.374 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/ymfpci/built-in.a 2024-07-31T15:05:51.400 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/vx222/built-in.a 2024-07-31T15:05:51.418 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pci/built-in.a 2024-07-31T15:05:51.423 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/plist.o 2024-07-31T15:05:51.423 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/svc4proc.o 2024-07-31T15:05:51.519 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/radix-tree.o 2024-07-31T15:05:51.548 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/ppc/built-in.a 2024-07-31T15:05:51.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:51 smithi067 ceph-mon[32750]: pgmap v865: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 834 KiB/s wr, 142 op/s 2024-07-31T15:05:51.571 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/arm/built-in.a 2024-07-31T15:05:51.595 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/sh/built-in.a 2024-07-31T15:05:51.628 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/synth/emux/built-in.a 2024-07-31T15:05:51.638 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:51 smithi077 ceph-mon[37327]: pgmap v865: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 834 KiB/s wr, 142 op/s 2024-07-31T15:05:51.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:51 smithi073 ceph-mon[36714]: pgmap v865: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 834 KiB/s wr, 142 op/s 2024-07-31T15:05:51.643 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/synth/built-in.a 2024-07-31T15:05:51.670 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs3proc.o 2024-07-31T15:05:51.682 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/usb/misc/built-in.a 2024-07-31T15:05:51.695 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exmisc.o 2024-07-31T15:05:51.705 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/usb/usx2y/built-in.a 2024-07-31T15:05:51.739 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/usb/caiaq/built-in.a 2024-07-31T15:05:51.782 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0' 2024-07-31T15:05:51.782 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0 2024-07-31T15:05:51.796 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/dev_addr_lists.o 2024-07-31T15:05:51.798 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/usb/6fire/built-in.a 2024-07-31T15:05:51.849 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/usb/hiface/built-in.a 2024-07-31T15:05:51.913 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/usb/bcd2000/built-in.a 2024-07-31T15:05:51.930 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/usb/built-in.a 2024-07-31T15:05:51.933 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/sys_ni.o 2024-07-31T15:05:52.090 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/firewire/built-in.a 2024-07-31T15:05:52.106 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exmutex.o 2024-07-31T15:05:52.168 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/sparc/built-in.a 2024-07-31T15:05:52.210 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/spi/built-in.a 2024-07-31T15:05:52.235 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/parisc/built-in.a 2024-07-31T15:05:52.355 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pcmcia/vx/built-in.a 2024-07-31T15:05:52.358 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/ratelimit.o 2024-07-31T15:05:52.429 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pcmcia/pdaudiocf/built-in.a 2024-07-31T15:05:52.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.435+0000 7fe78b138640 1 -- 172.21.15.67:0/4145728828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe784111970 msgr2=0x7fe784109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:52.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.435+0000 7fe78b138640 1 --2- 172.21.15.67:0/4145728828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe784111970 0x7fe784109df0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe778009900 tx=0x7fe77802a410 comp rx=0 tx=0).stop 2024-07-31T15:05:52.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.435+0000 7fe78b138640 1 -- 172.21.15.67:0/4145728828 shutdown_connections 2024-07-31T15:05:52.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.435+0000 7fe78b138640 1 --2- 172.21.15.67:0/4145728828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe78411c660 0x7fe78411eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.435+0000 7fe78b138640 1 --2- 172.21.15.67:0/4145728828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe78410a410 0x7fe78410a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.435+0000 7fe78b138640 1 --2- 172.21.15.67:0/4145728828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe784111970 0x7fe784109df0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.435+0000 7fe78b138640 1 -- 172.21.15.67:0/4145728828 >> 172.21.15.67:0/4145728828 conn(0x7fe7840708f0 msgr2=0x7fe784070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:52.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.444+0000 7fe78b138640 1 -- 172.21.15.67:0/4145728828 shutdown_connections 2024-07-31T15:05:52.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.444+0000 7fe78b138640 1 -- 172.21.15.67:0/4145728828 wait complete. 2024-07-31T15:05:52.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.445+0000 7fe78b138640 1 Processor -- start 2024-07-31T15:05:52.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.445+0000 7fe78b138640 1 -- start start 2024-07-31T15:05:52.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.445+0000 7fe78b138640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe78410a410 0x7fe7841a8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.445+0000 7fe78b138640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe784111970 0x7fe7841a9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.445+0000 7fe78b138640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe78411c660 0x7fe7841ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.446+0000 7fe7896d5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe78411c660 0x7fe7841ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:52.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.446+0000 7fe7896d5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe78411c660 0x7fe7841ae530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48384/0 (socket says 172.21.15.67:48384) 2024-07-31T15:05:52.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.446+0000 7fe7896d5640 1 -- 172.21.15.67:0/3226634874 learned_addr learned my addr 172.21.15.67:0/3226634874 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:52.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.446+0000 7fe78b138640 1 -- 172.21.15.67:0/3226634874 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe784122340 con 0x7fe78411c660 2024-07-31T15:05:52.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.446+0000 7fe78b138640 1 -- 172.21.15.67:0/3226634874 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841221c0 con 0x7fe78410a410 2024-07-31T15:05:52.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.446+0000 7fe78b138640 1 -- 172.21.15.67:0/3226634874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841224c0 con 0x7fe784111970 2024-07-31T15:05:52.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.446+0000 7fe7896d5640 1 -- 172.21.15.67:0/3226634874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe784111970 msgr2=0x7fe7841a9290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:05:52.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.446+0000 7fe7896d5640 1 --2- 172.21.15.67:0/3226634874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe784111970 0x7fe7841a9290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.446+0000 7fe7896d5640 1 -- 172.21.15.67:0/3226634874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe78410a410 msgr2=0x7fe7841a8cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:05:52.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.446+0000 7fe7896d5640 1 --2- 172.21.15.67:0/3226634874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe78410a410 0x7fe7841a8cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.446+0000 7fe7896d5640 1 -- 172.21.15.67:0/3226634874 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7841aec60 con 0x7fe78411c660 2024-07-31T15:05:52.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.447+0000 7fe7896d5640 1 --2- 172.21.15.67:0/3226634874 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe78411c660 0x7fe7841ae530 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00ec60 tx=0x7fe77c00c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:52.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.447+0000 7fe781ffb640 1 -- 172.21.15.67:0/3226634874 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c004b20 con 0x7fe78411c660 2024-07-31T15:05:52.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.447+0000 7fe78b138640 1 -- 172.21.15.67:0/3226634874 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7841aef50 con 0x7fe78411c660 2024-07-31T15:05:52.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.448+0000 7fe78b138640 1 -- 172.21.15.67:0/3226634874 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe78407b050 con 0x7fe78411c660 2024-07-31T15:05:52.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.451+0000 7fe781ffb640 1 -- 172.21.15.67:0/3226634874 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77c004cc0 con 0x7fe78411c660 2024-07-31T15:05:52.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.451+0000 7fe781ffb640 1 -- 172.21.15.67:0/3226634874 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c0116d0 con 0x7fe78411c660 2024-07-31T15:05:52.453 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/pcmcia/built-in.a 2024-07-31T15:05:52.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.453+0000 7fe781ffb640 1 -- 172.21.15.67:0/3226634874 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe77c0118b0 con 0x7fe78411c660 2024-07-31T15:05:52.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.456+0000 7fe791689740 1 Processor -- start 2024-07-31T15:05:52.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.456+0000 7fe781ffb640 1 --2- 172.21.15.67:0/3226634874 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7740b3ed0 0x7fe7740b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.457+0000 7fe781ffb640 1 -- 172.21.15.67:0/3226634874 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fe77c0e5430 con 0x7fe78411c660 2024-07-31T15:05:52.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.457+0000 7fe791689740 1 -- start start 2024-07-31T15:05:52.459 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/procfs.o 2024-07-31T15:05:52.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.458+0000 7fe791689740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a7b7070 0x55767ab5c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.458+0000 7fe791689740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a78e1b0 0x55767ab5cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.458+0000 7fe791689740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a796390 0x55767ab5d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.458+0000 7fe791689740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55767aad2bb0 con 0x55767a78e1b0 2024-07-31T15:05:52.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.458+0000 7fe791689740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55767aac20b0 con 0x55767a7b7070 2024-07-31T15:05:52.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.458+0000 7fe791689740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55767aac60f0 con 0x55767a796390 2024-07-31T15:05:52.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.459+0000 7fe783fff640 1 --2- 172.21.15.67:0/3226634874 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7740b3ed0 0x7fe7740b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:52.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.459+0000 7fe7896d5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a796390 0x55767ab5d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:52.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.459+0000 7fe788ed4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a7b7070 0x55767ab5c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:52.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.459+0000 7fe7896d5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a796390 0x55767ab5d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43702/0 (socket says 172.21.15.67:43702) 2024-07-31T15:05:52.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.459+0000 7fe788ed4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a7b7070 0x55767ab5c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40518/0 (socket says 172.21.15.67:40518) 2024-07-31T15:05:52.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.459+0000 7fe7896d5640 1 -- 172.21.15.67:0/2109049445 learned_addr learned my addr 172.21.15.67:0/2109049445 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:52.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.459+0000 7fe783fff640 1 --2- 172.21.15.67:0/2109049445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a78e1b0 0x55767ab5cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.459+0000 7fe7896d5640 1 -- 172.21.15.67:0/2109049445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a7b7070 msgr2=0x55767ab5c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.459+0000 7fe7896d5640 1 --2- 172.21.15.67:0/2109049445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a7b7070 0x55767ab5c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.459+0000 7fe7896d5640 1 -- 172.21.15.67:0/2109049445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a78e1b0 msgr2=0x55767ab5cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.459+0000 7fe7896d5640 1 --2- 172.21.15.67:0/2109049445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a78e1b0 0x55767ab5cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.459+0000 7fe7896d5640 1 -- 172.21.15.67:0/2109049445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55767aa68b90 con 0x55767a796390 2024-07-31T15:05:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.460+0000 7fe783fff640 1 --2- 172.21.15.67:0/2109049445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a78e1b0 0x55767ab5cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:05:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.460+0000 7fe783fff640 1 --2- 172.21.15.67:0/3226634874 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7740b3ed0 0x7fe7740b63e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe77000cfd0 tx=0x7fe770005cf0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.461+0000 7fe7896d5640 1 --2- 172.21.15.67:0/2109049445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a796390 0x55767ab5d110 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0a8fe0 tx=0x7fe77c00cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8ed5ba42a851a0e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.461+0000 7fe76effd640 1 -- 172.21.15.67:0/2109049445 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c01f040 con 0x55767a796390 2024-07-31T15:05:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.461+0000 7fe76effd640 1 -- 172.21.15.67:0/2109049445 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77c0abd70 con 0x55767a796390 2024-07-31T15:05:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.462+0000 7fe791689740 1 -- 172.21.15.67:0/2109049445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a796390 msgr2=0x55767ab5d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.462+0000 7fe791689740 1 --2- 172.21.15.67:0/2109049445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a796390 0x55767ab5d110 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0a8fe0 tx=0x7fe77c00cf50 comp rx=0 tx=0).stop 2024-07-31T15:05:52.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.465+0000 7fe791689740 1 -- 172.21.15.67:0/2109049445 shutdown_connections 2024-07-31T15:05:52.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.465+0000 7fe791689740 1 --2- 172.21.15.67:0/2109049445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a796390 0x55767ab5d110 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.465+0000 7fe791689740 1 --2- 172.21.15.67:0/2109049445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a78e1b0 0x55767ab5cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.465+0000 7fe791689740 1 --2- 172.21.15.67:0/2109049445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a7b7070 0x55767ab5c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.465+0000 7fe791689740 1 -- 172.21.15.67:0/2109049445 >> 172.21.15.67:0/2109049445 conn(0x55767a7ab460 msgr2=0x55767ab5a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:52.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.467+0000 7fe791689740 1 -- 172.21.15.67:0/2109049445 shutdown_connections 2024-07-31T15:05:52.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.467+0000 7fe791689740 1 -- 172.21.15.67:0/2109049445 wait complete. 2024-07-31T15:05:52.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.467+0000 7fe791689740 1 -- start start 2024-07-31T15:05:52.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.467+0000 7fe791689740 1 Processor -- start 2024-07-31T15:05:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.468+0000 7fe791689740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a78e1b0 0x55767ab68170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.468+0000 7fe791689740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a796390 0x55767ab686b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.468+0000 7fe791689740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a7b7070 0x55767ab741a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.468+0000 7fe791689740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55767a925700 con 0x55767a796390 2024-07-31T15:05:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.468+0000 7fe791689740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55767aa649e0 con 0x55767a78e1b0 2024-07-31T15:05:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.468+0000 7fe791689740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55767ab6ae00 con 0x55767a7b7070 2024-07-31T15:05:52.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.468+0000 7fe791689740 10 client.0 _ll_register_callbacks cb 0x55767aa7f4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:52.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.469+0000 7fe783fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a796390 0x55767ab686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:52.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.469+0000 7fe783fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a796390 0x55767ab686b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48398/0 (socket says 172.21.15.67:48398) 2024-07-31T15:05:52.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.469+0000 7fe783fff640 1 -- 172.21.15.67:0/1139788933 learned_addr learned my addr 172.21.15.67:0/1139788933 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:52.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.469+0000 7fe783fff640 1 -- 172.21.15.67:0/1139788933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a7b7070 msgr2=0x55767ab741a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:05:52.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.469+0000 7fe783fff640 1 --2- 172.21.15.67:0/1139788933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a7b7070 0x55767ab741a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.469+0000 7fe783fff640 1 -- 172.21.15.67:0/1139788933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a78e1b0 msgr2=0x55767ab68170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:52.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.469+0000 7fe783fff640 1 --2- 172.21.15.67:0/1139788933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a78e1b0 0x55767ab68170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.469+0000 7fe783fff640 1 -- 172.21.15.67:0/1139788933 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55767a822650 con 0x55767a796390 2024-07-31T15:05:52.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.470+0000 7fe783fff640 1 --2- 172.21.15.67:0/1139788933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a796390 0x55767ab686b0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe770010a70 tx=0x7fe7700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66eb370d9d2b8437 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:52.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.470+0000 7fe763fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:05:52.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.470+0000 7fe763fff640 1 -- 172.21.15.67:0/1139788933 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7700040f0 con 0x55767a796390 2024-07-31T15:05:52.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.471+0000 7fe791689740 1 -- 172.21.15.67:0/1139788933 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe77c0abd70 con 0x55767a796390 2024-07-31T15:05:52.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.471+0000 7fe763fff640 1 -- 172.21.15.67:0/1139788933 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fe770004290 con 0x55767a796390 2024-07-31T15:05:52.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.472+0000 7fe763fff640 1 -- 172.21.15.67:0/1139788933 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fe770004490 con 0x55767a796390 2024-07-31T15:05:52.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.475+0000 7fe791689740 10 client.18597 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:52.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.476+0000 7fe791689740 1 -- 172.21.15.67:0/1139788933 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55767ab76e10 con 0x55767a796390 2024-07-31T15:05:52.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.478+0000 7fe763fff640 1 -- 172.21.15.67:0/1139788933 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77c0abd70 con 0x55767a796390 2024-07-31T15:05:52.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.478+0000 7fe763fff640 1 -- 172.21.15.67:0/1139788933 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fe77000f040 con 0x55767a796390 2024-07-31T15:05:52.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.479+0000 7fe791689740 10 client.18597 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:52.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.479+0000 7fe791689740 10 client.18597 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:05:52.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.479+0000 7fe791689740 20 client.18597 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:52.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.479+0000 7fe791689740 1 --2- 172.21.15.67:0/1139788933 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55767a7759f0 0x55767ab7a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.479+0000 7fe791689740 4 client.18597 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:05:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.479+0000 7fe791689740 1 -- 172.21.15.67:0/1139788933 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55767aad4b00 con 0x55767a7759f0 2024-07-31T15:05:52.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.481+0000 7fe788ed4640 1 --2- 172.21.15.67:0/1139788933 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55767a7759f0 0x55767ab7a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:52.491 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:05:52.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.491+0000 7fe788ed4640 1 --2- 172.21.15.67:0/1139788933 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55767a7759f0 0x55767ab7a820 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dab53e97f76f50c4 server_cookie=6f6cc14ddc5c8516 in_seq=0 out_seq=0 2024-07-31T15:05:52.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.491+0000 7fe763fff640 10 client.18597 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:05:52.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.495+0000 7fe763fff640 1 -- 172.21.15.67:0/1139788933 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55767aad4b00 con 0x55767a7759f0 2024-07-31T15:05:52.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.495+0000 7fe763fff640 10 client.18597 handle_command_reply: tid=0 2024-07-31T15:05:52.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.495+0000 7fe791689740 1 client.18597 shutdown 2024-07-31T15:05:52.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.500+0000 7fe791689740 1 -- 172.21.15.67:0/1139788933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a796390 msgr2=0x55767ab686b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:52.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.500+0000 7fe791689740 1 --2- 172.21.15.67:0/1139788933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a796390 0x55767ab686b0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe770010a70 tx=0x7fe7700046c0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.505+0000 7fe791689740 1 -- 172.21.15.67:0/1139788933 shutdown_connections 2024-07-31T15:05:52.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.505+0000 7fe791689740 1 --2- 172.21.15.67:0/1139788933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a7b7070 0x55767ab741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.505+0000 7fe791689740 1 --2- 172.21.15.67:0/1139788933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a796390 0x55767ab686b0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.505+0000 7fe791689740 1 --2- 172.21.15.67:0/1139788933 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55767a7759f0 0x55767ab7a820 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.505+0000 7fe791689740 1 --2- 172.21.15.67:0/1139788933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a78e1b0 0x55767ab68170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.505+0000 7fe791689740 1 -- 172.21.15.67:0/1139788933 >> 172.21.15.67:0/1139788933 conn(0x55767a7ab460 msgr2=0x55767ab5f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:52.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.505+0000 7fe763fff640 0 client.18597 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:05:52.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.510+0000 7fe791689740 1 -- 172.21.15.67:0/1139788933 shutdown_connections 2024-07-31T15:05:52.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.510+0000 7fe791689740 1 -- 172.21.15.67:0/1139788933 wait complete. 2024-07-31T15:05:52.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.511+0000 7fe791689740 20 client.18597 trim_cache size 0 max 16384 2024-07-31T15:05:52.549 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/mips/built-in.a 2024-07-31T15:05:52.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.553+0000 7fe791689740 1 Processor -- start 2024-07-31T15:05:52.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.553+0000 7fe791689740 1 -- start start 2024-07-31T15:05:52.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.554+0000 7fe791689740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a78e1b0 0x55767ab5fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.554+0000 7fe791689740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a796390 0x55767ab73f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.554+0000 7fe791689740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a7b7070 0x55767ab744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.554+0000 7fe791689740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe76800ffd0 con 0x55767a7b7070 2024-07-31T15:05:52.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.554+0000 7fe791689740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe768011b10 con 0x55767a796390 2024-07-31T15:05:52.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.554+0000 7fe791689740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe7680106a0 con 0x55767a78e1b0 2024-07-31T15:05:52.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.554+0000 7fe7896d5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a7b7070 0x55767ab744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:52.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.554+0000 7fe7896d5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a7b7070 0x55767ab744d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48410/0 (socket says 172.21.15.67:48410) 2024-07-31T15:05:52.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.554+0000 7fe7896d5640 1 -- 172.21.15.67:0/950449422 learned_addr learned my addr 172.21.15.67:0/950449422 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:52.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.554+0000 7fe783fff640 1 --2- 172.21.15.67:0/950449422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a796390 0x55767ab73f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:52.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.554+0000 7fe788ed4640 1 --2- 172.21.15.67:0/950449422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a78e1b0 0x55767ab5fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:52.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.555+0000 7fe7896d5640 1 -- 172.21.15.67:0/950449422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a78e1b0 msgr2=0x55767ab5fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:52.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.555+0000 7fe7896d5640 1 --2- 172.21.15.67:0/950449422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a78e1b0 0x55767ab5fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.555+0000 7fe7896d5640 1 -- 172.21.15.67:0/950449422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a796390 msgr2=0x55767ab73f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:52.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.555+0000 7fe7896d5640 1 --2- 172.21.15.67:0/950449422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a796390 0x55767ab73f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.555+0000 7fe7896d5640 1 -- 172.21.15.67:0/950449422 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55767ab68a20 con 0x55767a7b7070 2024-07-31T15:05:52.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.555+0000 7fe7896d5640 1 --2- 172.21.15.67:0/950449422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a7b7070 0x55767ab744d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0aba70 tx=0x7fe77c0ac5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f67d5b9968d9dd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:52.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.556+0000 7fe76e7fc640 1 -- 172.21.15.67:0/950449422 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c01f040 con 0x55767a7b7070 2024-07-31T15:05:52.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.556+0000 7fe76e7fc640 1 -- 172.21.15.67:0/950449422 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77c0cb070 con 0x55767a7b7070 2024-07-31T15:05:52.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.557+0000 7fe76e7fc640 1 -- 172.21.15.67:0/950449422 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c0a7940 con 0x55767a7b7070 2024-07-31T15:05:52.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.560+0000 7fe791689740 1 -- 172.21.15.67:0/950449422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a7b7070 msgr2=0x55767ab744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:52.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.560+0000 7fe791689740 1 --2- 172.21.15.67:0/950449422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a7b7070 0x55767ab744d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0aba70 tx=0x7fe77c0ac5b0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.561+0000 7fe791689740 1 -- 172.21.15.67:0/950449422 shutdown_connections 2024-07-31T15:05:52.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.561+0000 7fe791689740 1 --2- 172.21.15.67:0/950449422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a7b7070 0x55767ab744d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.561+0000 7fe791689740 1 --2- 172.21.15.67:0/950449422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a796390 0x55767ab73f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.561+0000 7fe791689740 1 --2- 172.21.15.67:0/950449422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a78e1b0 0x55767ab5fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.561+0000 7fe791689740 1 -- 172.21.15.67:0/950449422 >> 172.21.15.67:0/950449422 conn(0x55767a7ab460 msgr2=0x55767ab5f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:52.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.563+0000 7fe791689740 1 -- 172.21.15.67:0/950449422 shutdown_connections 2024-07-31T15:05:52.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.565+0000 7fe791689740 1 -- 172.21.15.67:0/950449422 wait complete. 2024-07-31T15:05:52.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.565+0000 7fe791689740 1 -- start start 2024-07-31T15:05:52.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.565+0000 7fe791689740 1 Processor -- start 2024-07-31T15:05:52.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.566+0000 7fe791689740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a78e1b0 0x55767ab6b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.566+0000 7fe791689740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a796390 0x55767ab6ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.566+0000 7fe791689740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a7b7070 0x55767abe0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.566+0000 7fe791689740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe768011440 con 0x55767a796390 2024-07-31T15:05:52.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.566+0000 7fe791689740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe768010d70 con 0x55767a78e1b0 2024-07-31T15:05:52.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.566+0000 7fe791689740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe76800dfd0 con 0x55767a7b7070 2024-07-31T15:05:52.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.566+0000 7fe791689740 10 client.0 _ll_register_callbacks cb 0x7fe77000cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:52.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.567+0000 7fe788ed4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a78e1b0 0x55767ab6b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:52.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.567+0000 7fe788ed4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a78e1b0 0x55767ab6b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40548/0 (socket says 172.21.15.67:40548) 2024-07-31T15:05:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.567+0000 7fe788ed4640 1 -- 172.21.15.67:0/4117851249 learned_addr learned my addr 172.21.15.67:0/4117851249 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.567+0000 7fe7896d5640 1 --2- 172.21.15.67:0/4117851249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a7b7070 0x55767abe0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.567+0000 7fe783fff640 1 --2- 172.21.15.67:0/4117851249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a796390 0x55767ab6ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.568+0000 7fe788ed4640 1 -- 172.21.15.67:0/4117851249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a7b7070 msgr2=0x55767abe0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.568+0000 7fe788ed4640 1 --2- 172.21.15.67:0/4117851249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a7b7070 0x55767abe0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.568+0000 7fe788ed4640 1 -- 172.21.15.67:0/4117851249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a796390 msgr2=0x55767ab6ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.568+0000 7fe788ed4640 1 --2- 172.21.15.67:0/4117851249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a796390 0x55767ab6ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.568+0000 7fe788ed4640 1 -- 172.21.15.67:0/4117851249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55767abe1520 con 0x55767a78e1b0 2024-07-31T15:05:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.568+0000 7fe788ed4640 1 --2- 172.21.15.67:0/4117851249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a78e1b0 0x55767ab6b4d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe7780092a0 tx=0x7fe7780041b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33f1c42bbb45f14d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.568+0000 7fe763fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:05:52.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.570+0000 7fe763fff640 1 -- 172.21.15.67:0/4117851249 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778047030 con 0x55767a78e1b0 2024-07-31T15:05:52.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.570+0000 7fe763fff640 1 -- 172.21.15.67:0/4117851249 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fe7780035f0 con 0x55767a78e1b0 2024-07-31T15:05:52.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.570+0000 7fe791689740 1 -- 172.21.15.67:0/4117851249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe77c0cb070 con 0x55767a78e1b0 2024-07-31T15:05:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.570+0000 7fe763fff640 1 -- 172.21.15.67:0/4117851249 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fe778035040 con 0x55767a78e1b0 2024-07-31T15:05:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.570+0000 7fe791689740 10 client.28145 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.570+0000 7fe791689740 1 -- 172.21.15.67:0/4117851249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55767abe2640 con 0x55767a78e1b0 2024-07-31T15:05:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.571+0000 7fe763fff640 1 -- 172.21.15.67:0/4117851249 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77c0cb070 con 0x55767a78e1b0 2024-07-31T15:05:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.571+0000 7fe763fff640 1 -- 172.21.15.67:0/4117851249 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fe77803c890 con 0x55767a78e1b0 2024-07-31T15:05:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.571+0000 7fe791689740 10 client.28145 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.571+0000 7fe791689740 10 client.28145 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:05:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.571+0000 7fe791689740 20 client.28145 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.571+0000 7fe791689740 1 --2- 172.21.15.67:0/4117851249 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55767a74ada0 0x55767abe4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.571+0000 7fe791689740 4 client.28145 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:05:52.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.571+0000 7fe791689740 1 -- 172.21.15.67:0/4117851249 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55767aad05a0 con 0x55767a74ada0 2024-07-31T15:05:52.574 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exnames.o 2024-07-31T15:05:52.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.573+0000 7fe783fff640 1 --2- 172.21.15.67:0/4117851249 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55767a74ada0 0x55767abe4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:52.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.574+0000 7fe783fff640 1 --2- 172.21.15.67:0/4117851249 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55767a74ada0 0x55767abe4e00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d58f9b74b8133a5f server_cookie=45ce3291b17c60c2 in_seq=0 out_seq=0 2024-07-31T15:05:52.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.574+0000 7fe763fff640 10 client.28145 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:05:52.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.575+0000 7fe763fff640 1 -- 172.21.15.67:0/4117851249 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55767aad05a0 con 0x55767a74ada0 2024-07-31T15:05:52.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.575+0000 7fe763fff640 10 client.28145 handle_command_reply: tid=0 2024-07-31T15:05:52.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.575+0000 7fe791689740 1 client.28145 shutdown 2024-07-31T15:05:52.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.575+0000 7fe791689740 1 -- 172.21.15.67:0/4117851249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a78e1b0 msgr2=0x55767ab6b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:52.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.575+0000 7fe791689740 1 --2- 172.21.15.67:0/4117851249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a78e1b0 0x55767ab6b4d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe7780092a0 tx=0x7fe7780041b0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.575+0000 7fe791689740 1 -- 172.21.15.67:0/4117851249 shutdown_connections 2024-07-31T15:05:52.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.575+0000 7fe791689740 1 --2- 172.21.15.67:0/4117851249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55767a7b7070 0x55767abe0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.575+0000 7fe791689740 1 --2- 172.21.15.67:0/4117851249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55767a796390 0x55767ab6ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.575+0000 7fe791689740 1 --2- 172.21.15.67:0/4117851249 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55767a74ada0 0x55767abe4e00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.575+0000 7fe791689740 1 --2- 172.21.15.67:0/4117851249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55767a78e1b0 0x55767ab6b4d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.575+0000 7fe791689740 1 -- 172.21.15.67:0/4117851249 >> 172.21.15.67:0/4117851249 conn(0x55767a7ab460 msgr2=0x55767ab59d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:52.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.575+0000 7fe763fff640 0 client.28145 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:05:52.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.576+0000 7fe791689740 1 -- 172.21.15.67:0/4117851249 shutdown_connections 2024-07-31T15:05:52.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.576+0000 7fe791689740 1 -- 172.21.15.67:0/4117851249 wait complete. 2024-07-31T15:05:52.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.577+0000 7fe791689740 20 client.28145 trim_cache size 0 max 16384 2024-07-31T15:05:52.579 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:05:52.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.580+0000 7fe78b138640 1 -- 172.21.15.67:0/3226634874 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7740b3ed0 msgr2=0x7fe7740b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:52.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.580+0000 7fe78b138640 1 --2- 172.21.15.67:0/3226634874 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7740b3ed0 0x7fe7740b63e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe77000cfd0 tx=0x7fe770005cf0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.580+0000 7fe78b138640 1 -- 172.21.15.67:0/3226634874 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe78411c660 msgr2=0x7fe7841ae530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:52.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.580+0000 7fe78b138640 1 --2- 172.21.15.67:0/3226634874 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe78411c660 0x7fe7841ae530 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00ec60 tx=0x7fe77c00c640 comp rx=0 tx=0).stop 2024-07-31T15:05:52.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.580+0000 7fe78b138640 1 -- 172.21.15.67:0/3226634874 shutdown_connections 2024-07-31T15:05:52.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.580+0000 7fe78b138640 1 --2- 172.21.15.67:0/3226634874 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe7740b3ed0 0x7fe7740b63e0 secure :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe77000cfd0 tx=0x7fe770005cf0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.580+0000 7fe78b138640 1 --2- 172.21.15.67:0/3226634874 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe78411c660 0x7fe7841ae530 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.580+0000 7fe78b138640 1 --2- 172.21.15.67:0/3226634874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe784111970 0x7fe7841a9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.580+0000 7fe78b138640 1 --2- 172.21.15.67:0/3226634874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe78410a410 0x7fe7841a8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:52.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.580+0000 7fe78b138640 1 -- 172.21.15.67:0/3226634874 >> 172.21.15.67:0/3226634874 conn(0x7fe7840708f0 msgr2=0x7fe78411deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:52.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.581+0000 7fe78b138640 1 -- 172.21.15.67:0/3226634874 shutdown_connections 2024-07-31T15:05:52.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:52.581+0000 7fe78b138640 1 -- 172.21.15.67:0/3226634874 wait complete. 2024-07-31T15:05:52.585 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/soc/built-in.a 2024-07-31T15:05:52.598 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/nsproxy.o 2024-07-31T15:05:52.628 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/rbtree.o 2024-07-31T15:05:52.641 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/atmel/built-in.a 2024-07-31T15:05:52.677 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/hda_bus_type.o 2024-07-31T15:05:52.805 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/dst.o 2024-07-31T15:05:52.872 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs3xdr.o 2024-07-31T15:05:52.975 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/seq_buf.o 2024-07-31T15:05:52.978 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exoparg1.o 2024-07-31T15:05:53.117 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/lockd/built-in.a 2024-07-31T15:05:53.210 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/hdac_bus.o 2024-07-31T15:05:53.227 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nls/nls_base.o 2024-07-31T15:05:53.443 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exoparg2.o 2024-07-31T15:05:53.504 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/notifier.o 2024-07-31T15:05:53.529 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/siphash.o 2024-07-31T15:05:53.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:53 smithi067 ceph-mon[32750]: pgmap v866: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 701 KiB/s wr, 156 op/s 2024-07-31T15:05:53.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:53 smithi073 ceph-mon[36714]: pgmap v866: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 701 KiB/s wr, 156 op/s 2024-07-31T15:05:53.704 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nls/nls_cp437.o 2024-07-31T15:05:53.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:53 smithi077 ceph-mon[37327]: pgmap v866: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 701 KiB/s wr, 156 op/s 2024-07-31T15:05:53.846 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exoparg3.o 2024-07-31T15:05:53.863 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/hdac_device.o 2024-07-31T15:05:53.911 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nls/nls_ascii.o 2024-07-31T15:05:53.980 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/string.o 2024-07-31T15:05:54.078 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/timerqueue.o 2024-07-31T15:05:54.410 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exoparg6.o 2024-07-31T15:05:54.433 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/vsprintf.o 2024-07-31T15:05:54.857 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T15:05:55.219 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nls/nls_iso8859-1.o 2024-07-31T15:05:55.376 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:55 smithi067 ceph-mon[32750]: pgmap v867: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 702 KiB/s wr, 211 op/s 2024-07-31T15:05:55.491 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/win_minmax.o 2024-07-31T15:05:55.510 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/ksysfs.o 2024-07-31T15:05:55.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.514+0000 7f8e118de640 1 -- 172.21.15.67:0/519328194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e0c10a330 msgr2=0x7f8e0c10a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:55.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.514+0000 7f8e118de640 1 --2- 172.21.15.67:0/519328194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e0c10a330 0x7f8e0c10a740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009860 tx=0x7f8df802a380 comp rx=0 tx=0).stop 2024-07-31T15:05:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.517+0000 7f8e118de640 1 -- 172.21.15.67:0/519328194 shutdown_connections 2024-07-31T15:05:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.517+0000 7f8e118de640 1 --2- 172.21.15.67:0/519328194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e0c11c5d0 0x7f8e0c11e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.517+0000 7f8e118de640 1 --2- 172.21.15.67:0/519328194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e0c10a330 0x7f8e0c10a740 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.517+0000 7f8e118de640 1 --2- 172.21.15.67:0/519328194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e0c111730 0x7f8e0c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.517+0000 7f8e118de640 1 -- 172.21.15.67:0/519328194 >> 172.21.15.67:0/519328194 conn(0x7f8e0c06e440 msgr2=0x7f8e0c06e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:55.524 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs3acl.o 2024-07-31T15:05:55.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.524+0000 7f8e118de640 1 -- 172.21.15.67:0/519328194 shutdown_connections 2024-07-31T15:05:55.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.529+0000 7f8e118de640 1 -- 172.21.15.67:0/519328194 wait complete. 2024-07-31T15:05:55.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.530+0000 7f8e118de640 1 Processor -- start 2024-07-31T15:05:55.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.531+0000 7f8e118de640 1 -- start start 2024-07-31T15:05:55.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.531+0000 7f8e118de640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e0c10a330 0x7f8e0c11afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.531+0000 7f8e118de640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e0c111730 0x7f8e0c1142a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.531+0000 7f8e118de640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e0c11c5d0 0x7f8e0c1149f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.531+0000 7f8e118de640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c1221c0 con 0x7f8e0c10a330 2024-07-31T15:05:55.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.531+0000 7f8e118de640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c122040 con 0x7f8e0c11c5d0 2024-07-31T15:05:55.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.531+0000 7f8e118de640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c122340 con 0x7f8e0c111730 2024-07-31T15:05:55.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.532+0000 7f8e108dc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e0c10a330 0x7f8e0c11afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:55.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.532+0000 7f8e108dc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e0c10a330 0x7f8e0c11afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48438/0 (socket says 172.21.15.67:48438) 2024-07-31T15:05:55.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.532+0000 7f8e108dc640 1 -- 172.21.15.67:0/1661852150 learned_addr learned my addr 172.21.15.67:0/1661852150 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:55.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.532+0000 7f8e108dc640 1 -- 172.21.15.67:0/1661852150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e0c111730 msgr2=0x7f8e0c1142a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:05:55.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.532+0000 7f8e108dc640 1 --2- 172.21.15.67:0/1661852150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e0c111730 0x7f8e0c1142a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.532+0000 7f8e108dc640 1 -- 172.21.15.67:0/1661852150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e0c11c5d0 msgr2=0x7f8e0c1149f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:55.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.532+0000 7f8e108dc640 1 --2- 172.21.15.67:0/1661852150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e0c11c5d0 0x7f8e0c1149f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.532+0000 7f8e108dc640 1 -- 172.21.15.67:0/1661852150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c119c40 con 0x7f8e0c10a330 2024-07-31T15:05:55.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.532+0000 7f8e108dc640 1 --2- 172.21.15.67:0/1661852150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e0c10a330 0x7f8e0c11afd0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000b450 tx=0x7f8e0000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:55.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.533+0000 7f8e09ffb640 1 -- 172.21.15.67:0/1661852150 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00015020 con 0x7f8e0c10a330 2024-07-31T15:05:55.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.533+0000 7f8e118de640 1 -- 172.21.15.67:0/1661852150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c119ed0 con 0x7f8e0c10a330 2024-07-31T15:05:55.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.533+0000 7f8e118de640 1 -- 172.21.15.67:0/1661852150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c11a410 con 0x7f8e0c10a330 2024-07-31T15:05:55.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.537+0000 7f8e19091740 1 Processor -- start 2024-07-31T15:05:55.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.537+0000 7f8e09ffb640 1 -- 172.21.15.67:0/1661852150 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e00011d40 con 0x7f8e0c10a330 2024-07-31T15:05:55.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.537+0000 7f8e09ffb640 1 -- 172.21.15.67:0/1661852150 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0000f340 con 0x7f8e0c10a330 2024-07-31T15:05:55.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.545+0000 7f8e19091740 1 -- start start 2024-07-31T15:05:55.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e19091740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b906840 0x55f83bcaca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e19091740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b8dd980 0x55f83bcacfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e19091740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8e5b60 0x55f83bcad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e19091740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f83bc217b0 con 0x55f83b8dd980 2024-07-31T15:05:55.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e19091740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f83bc10cb0 con 0x55f83b8e5b60 2024-07-31T15:05:55.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e19091740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f83bc14cf0 con 0x55f83b906840 2024-07-31T15:05:55.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e09ffb640 1 -- 172.21.15.67:0/1661852150 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8e00018020 con 0x7f8e0c10a330 2024-07-31T15:05:55.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e0bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8e5b60 0x55f83bcad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:55.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e0bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8e5b60 0x55f83bcad4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40560/0 (socket says 172.21.15.67:40560) 2024-07-31T15:05:55.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e0bfff640 1 -- 172.21.15.67:0/1418934347 learned_addr learned my addr 172.21.15.67:0/1418934347 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:55.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e110dd640 1 --2- 172.21.15.67:0/1418934347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b906840 0x55f83bcaca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:55.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e0bfff640 1 -- 172.21.15.67:0/1418934347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b906840 msgr2=0x55f83bcaca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:55.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e0bfff640 1 --2- 172.21.15.67:0/1418934347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b906840 0x55f83bcaca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e0bfff640 1 -- 172.21.15.67:0/1418934347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b8dd980 msgr2=0x55f83bcacfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:55.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e0bfff640 1 --2- 172.21.15.67:0/1418934347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b8dd980 0x55f83bcacfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.546+0000 7f8e0bfff640 1 -- 172.21.15.67:0/1418934347 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83bba60b0 con 0x55f83b8e5b60 2024-07-31T15:05:55.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.547+0000 7f8e0bfff640 1 --2- 172.21.15.67:0/1418934347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8e5b60 0x55f83bcad4e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009780 tx=0x7f8df80303b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=201701178c35fb82 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:55.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.548+0000 7f8df6ffd640 1 -- 172.21.15.67:0/1418934347 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8037040 con 0x55f83b8e5b60 2024-07-31T15:05:55.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.548+0000 7f8df6ffd640 1 -- 172.21.15.67:0/1418934347 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8df8040070 con 0x55f83b8e5b60 2024-07-31T15:05:55.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.548+0000 7f8df6ffd640 1 -- 172.21.15.67:0/1418934347 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df80035f0 con 0x55f83b8e5b60 2024-07-31T15:05:55.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.549+0000 7f8e19091740 1 -- 172.21.15.67:0/1418934347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8e5b60 msgr2=0x55f83bcad4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:55.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.549+0000 7f8e19091740 1 --2- 172.21.15.67:0/1418934347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8e5b60 0x55f83bcad4e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009780 tx=0x7f8df80303b0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.550+0000 7f8e09ffb640 1 --2- 172.21.15.67:0/1661852150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8dfc0d5f50 0x7f8dfc0d8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.550+0000 7f8e09ffb640 1 -- 172.21.15.67:0/1661852150 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f8e000e3c00 con 0x7f8e0c10a330 2024-07-31T15:05:55.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.553+0000 7f8e110dd640 1 --2- 172.21.15.67:0/1661852150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8dfc0d5f50 0x7f8dfc0d8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:55.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.553+0000 7f8e19091740 1 -- 172.21.15.67:0/1418934347 shutdown_connections 2024-07-31T15:05:55.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.553+0000 7f8e19091740 1 --2- 172.21.15.67:0/1418934347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8e5b60 0x55f83bcad4e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.553+0000 7f8e19091740 1 --2- 172.21.15.67:0/1418934347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b8dd980 0x55f83bcacfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.553+0000 7f8e19091740 1 --2- 172.21.15.67:0/1418934347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b906840 0x55f83bcaca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.553+0000 7f8e19091740 1 -- 172.21.15.67:0/1418934347 >> 172.21.15.67:0/1418934347 conn(0x55f83b8fa990 msgr2=0x55f83bcaa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:55.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.557+0000 7f8e19091740 1 -- 172.21.15.67:0/1418934347 shutdown_connections 2024-07-31T15:05:55.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.557+0000 7f8e19091740 1 -- 172.21.15.67:0/1418934347 wait complete. 2024-07-31T15:05:55.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.557+0000 7f8e19091740 1 -- start start 2024-07-31T15:05:55.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.557+0000 7f8e19091740 1 Processor -- start 2024-07-31T15:05:55.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.558+0000 7f8e19091740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8dd980 0x55f83bcb8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.558+0000 7f8e19091740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b8e5b60 0x55f83bcb8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.558+0000 7f8e19091740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b906840 0x55f83bcb8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.558+0000 7f8e19091740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f83ba747a0 con 0x55f83b906840 2024-07-31T15:05:55.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.558+0000 7f8e19091740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f83bbb3320 con 0x55f83b8dd980 2024-07-31T15:05:55.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.558+0000 7f8e19091740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f83bcbb1e0 con 0x55f83b8e5b60 2024-07-31T15:05:55.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.558+0000 7f8e19091740 10 client.0 _ll_register_callbacks cb 0x55f83bbbd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:55.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.559+0000 7f8e110dd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b906840 0x55f83bcb8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:55.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.559+0000 7f8e110dd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b906840 0x55f83bcb8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48460/0 (socket says 172.21.15.67:48460) 2024-07-31T15:05:55.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.559+0000 7f8e110dd640 1 -- 172.21.15.67:0/4138895566 learned_addr learned my addr 172.21.15.67:0/4138895566 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:55.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.559+0000 7f8e110dd640 1 -- 172.21.15.67:0/4138895566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b8e5b60 msgr2=0x55f83bcb8580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:05:55.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.559+0000 7f8e110dd640 1 --2- 172.21.15.67:0/4138895566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b8e5b60 0x55f83bcb8580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.559+0000 7f8e110dd640 1 -- 172.21.15.67:0/4138895566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8dd980 msgr2=0x55f83bcb8040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:05:55.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.559+0000 7f8e110dd640 1 --2- 172.21.15.67:0/4138895566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8dd980 0x55f83bcb8040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.559+0000 7f8e110dd640 1 -- 172.21.15.67:0/4138895566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83bbb74b0 con 0x55f83b906840 2024-07-31T15:05:55.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.560+0000 7f8e110dd640 1 --2- 172.21.15.67:0/4138895566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b906840 0x55f83bcb8ac0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400ea90 tx=0x7f8e0400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af2123b094ec987 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:55.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.562+0000 7f8deffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:05:55.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.562+0000 7f8deffff640 1 -- 172.21.15.67:0/4138895566 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04025060 con 0x55f83b906840 2024-07-31T15:05:55.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.562+0000 7f8e19091740 1 -- 172.21.15.67:0/4138895566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df8040070 con 0x55f83b906840 2024-07-31T15:05:55.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.562+0000 7f8deffff640 1 -- 172.21.15.67:0/4138895566 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f8e0401e3e0 con 0x55f83b906840 2024-07-31T15:05:55.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.568+0000 7f8deffff640 1 -- 172.21.15.67:0/4138895566 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8e0402c050 con 0x55f83b906840 2024-07-31T15:05:55.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.569+0000 7f8e110dd640 1 --2- 172.21.15.67:0/1661852150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8dfc0d5f50 0x7f8dfc0d8460 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x55f83bcc6e80 tx=0x7f8e0401f000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:55.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.569+0000 7f8e19091740 10 client.18612 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:55.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.569+0000 7f8e19091740 1 -- 172.21.15.67:0/4138895566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f83b980b90 con 0x55f83b906840 2024-07-31T15:05:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.578+0000 7f8deffff640 1 -- 172.21.15.67:0/4138895566 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8df8040070 con 0x55f83b906840 2024-07-31T15:05:55.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.578+0000 7f8deffff640 1 -- 172.21.15.67:0/4138895566 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f8e0401fca0 con 0x55f83b906840 2024-07-31T15:05:55.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.580+0000 7f8e19091740 10 client.18612 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:55.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.580+0000 7f8e19091740 10 client.18612 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:05:55.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.580+0000 7f8e19091740 20 client.18612 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:55.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.580+0000 7f8e19091740 1 --2- 172.21.15.67:0/4138895566 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f83b8c51c0 0x55f83bcca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.580+0000 7f8e19091740 4 client.18612 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:05:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.580+0000 7f8e19091740 1 -- 172.21.15.67:0/4138895566 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f83bc23700 con 0x55f83b8c51c0 2024-07-31T15:05:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.580+0000 7f8e108dc640 1 --2- 172.21.15.67:0/4138895566 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f83b8c51c0 0x55f83bcca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.581+0000 7f8e108dc640 1 --2- 172.21.15.67:0/4138895566 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f83b8c51c0 0x55f83bcca740 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f654e160a3e15634 server_cookie=f47e3ae6c4a508b3 in_seq=0 out_seq=0 2024-07-31T15:05:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.581+0000 7f8deffff640 10 client.18612 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.581+0000 7f8deffff640 1 -- 172.21.15.67:0/4138895566 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f83bc23700 con 0x55f83b8c51c0 2024-07-31T15:05:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.582+0000 7f8deffff640 10 client.18612 handle_command_reply: tid=0 2024-07-31T15:05:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.582+0000 7f8e19091740 1 client.18612 shutdown 2024-07-31T15:05:55.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.592+0000 7f8e19091740 1 -- 172.21.15.67:0/4138895566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b906840 msgr2=0x55f83bcb8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:55.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.592+0000 7f8e19091740 1 --2- 172.21.15.67:0/4138895566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b906840 0x55f83bcb8ac0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400ea90 tx=0x7f8e0400ef50 comp rx=0 tx=0).stop 2024-07-31T15:05:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.596+0000 7f8e19091740 1 -- 172.21.15.67:0/4138895566 shutdown_connections 2024-07-31T15:05:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.596+0000 7f8e19091740 1 --2- 172.21.15.67:0/4138895566 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f83b8c51c0 0x55f83bcca740 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.596+0000 7f8e19091740 1 --2- 172.21.15.67:0/4138895566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b906840 0x55f83bcb8ac0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.596+0000 7f8e19091740 1 --2- 172.21.15.67:0/4138895566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b8e5b60 0x55f83bcb8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.596+0000 7f8e19091740 1 --2- 172.21.15.67:0/4138895566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8dd980 0x55f83bcb8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.596+0000 7f8e19091740 1 -- 172.21.15.67:0/4138895566 >> 172.21.15.67:0/4138895566 conn(0x55f83b8fa990 msgr2=0x55f83bcafbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:55.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.600+0000 7f8deffff640 0 client.18612 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:55.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.600+0000 7f8e19091740 1 -- 172.21.15.67:0/4138895566 shutdown_connections 2024-07-31T15:05:55.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.600+0000 7f8e19091740 1 -- 172.21.15.67:0/4138895566 wait complete. 2024-07-31T15:05:55.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.603+0000 7f8e19091740 20 client.18612 trim_cache size 0 max 16384 2024-07-31T15:05:55.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:55 smithi073 ceph-mon[36714]: pgmap v867: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 702 KiB/s wr, 211 op/s 2024-07-31T15:05:55.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.655+0000 7f8e19091740 1 Processor -- start 2024-07-31T15:05:55.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.655+0000 7f8e19091740 1 -- start start 2024-07-31T15:05:55.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.656+0000 7f8e19091740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8dd980 0x55f83bca9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.656+0000 7f8e19091740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b8e5b60 0x55f83bcb9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.656+0000 7f8e19091740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b906840 0x55f83bcae150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.656+0000 7f8e108dc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8dd980 0x55f83bca9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:55.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.657+0000 7f8e108dc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8dd980 0x55f83bca9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40590/0 (socket says 172.21.15.67:40590) 2024-07-31T15:05:55.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.657+0000 7f8e108dc640 1 -- 172.21.15.67:0/2948835649 learned_addr learned my addr 172.21.15.67:0/2948835649 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:55.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.657+0000 7f8e110dd640 1 --2- 172.21.15.67:0/2948835649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b906840 0x55f83bcae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:55.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.657+0000 7f8e19091740 1 -- 172.21.15.67:0/2948835649 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8df000ff60 con 0x55f83b8e5b60 2024-07-31T15:05:55.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.657+0000 7f8e19091740 1 -- 172.21.15.67:0/2948835649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8df0011aa0 con 0x55f83b8dd980 2024-07-31T15:05:55.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.657+0000 7f8e19091740 1 -- 172.21.15.67:0/2948835649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8df0010630 con 0x55f83b906840 2024-07-31T15:05:55.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.657+0000 7f8e0bfff640 1 --2- 172.21.15.67:0/2948835649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b8e5b60 0x55f83bcb9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:55.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.658+0000 7f8e0bfff640 1 -- 172.21.15.67:0/2948835649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b906840 msgr2=0x55f83bcae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:55.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.658+0000 7f8e0bfff640 1 --2- 172.21.15.67:0/2948835649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b906840 0x55f83bcae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.658+0000 7f8e0bfff640 1 -- 172.21.15.67:0/2948835649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8dd980 msgr2=0x55f83bca9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:55.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.658+0000 7f8e0bfff640 1 --2- 172.21.15.67:0/2948835649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8dd980 0x55f83bca9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.658+0000 7f8e0bfff640 1 -- 172.21.15.67:0/2948835649 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83bcb8e20 con 0x55f83b8e5b60 2024-07-31T15:05:55.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.659+0000 7f8e0bfff640 1 --2- 172.21.15.67:0/2948835649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b8e5b60 0x55f83bcb9f20 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f8df803d040 tx=0x7f8df8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2481784dcdfd17a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:55.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.660+0000 7f8df67fc640 1 -- 172.21.15.67:0/2948835649 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8037040 con 0x55f83b8e5b60 2024-07-31T15:05:55.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.660+0000 7f8df67fc640 1 -- 172.21.15.67:0/2948835649 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8df804b070 con 0x55f83b8e5b60 2024-07-31T15:05:55.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.660+0000 7f8df67fc640 1 -- 172.21.15.67:0/2948835649 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df80035f0 con 0x55f83b8e5b60 2024-07-31T15:05:55.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.661+0000 7f8e19091740 1 -- 172.21.15.67:0/2948835649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b8e5b60 msgr2=0x55f83bcb9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:55.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.661+0000 7f8e19091740 1 --2- 172.21.15.67:0/2948835649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b8e5b60 0x55f83bcb9f20 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f8df803d040 tx=0x7f8df8005f90 comp rx=0 tx=0).stop 2024-07-31T15:05:55.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.661+0000 7f8e19091740 1 -- 172.21.15.67:0/2948835649 shutdown_connections 2024-07-31T15:05:55.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.661+0000 7f8e19091740 1 --2- 172.21.15.67:0/2948835649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b906840 0x55f83bcae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.662+0000 7f8e19091740 1 --2- 172.21.15.67:0/2948835649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b8e5b60 0x55f83bcb9f20 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.662+0000 7f8e19091740 1 --2- 172.21.15.67:0/2948835649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8dd980 0x55f83bca9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.662+0000 7f8e19091740 1 -- 172.21.15.67:0/2948835649 >> 172.21.15.67:0/2948835649 conn(0x55f83b8fa990 msgr2=0x55f83bcada40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:55.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.666+0000 7f8e19091740 1 -- 172.21.15.67:0/2948835649 shutdown_connections 2024-07-31T15:05:55.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.666+0000 7f8e19091740 1 -- 172.21.15.67:0/2948835649 wait complete. 2024-07-31T15:05:55.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.666+0000 7f8e19091740 1 -- start start 2024-07-31T15:05:55.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.667+0000 7f8e19091740 1 Processor -- start 2024-07-31T15:05:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.668+0000 7f8e19091740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8dd980 0x55f83bcc0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.668+0000 7f8e19091740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b8e5b60 0x55f83bcbdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.668+0000 7f8e19091740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b906840 0x55f83bcbe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.668+0000 7f8e19091740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8df00113d0 con 0x55f83b906840 2024-07-31T15:05:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.668+0000 7f8e19091740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8df0010d00 con 0x55f83b8dd980 2024-07-31T15:05:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.668+0000 7f8e19091740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8df000df60 con 0x55f83b8e5b60 2024-07-31T15:05:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.668+0000 7f8e19091740 10 client.0 _ll_register_callbacks cb 0x7f8e0400e620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.669+0000 7f8e110dd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b906840 0x55f83bcbe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.669+0000 7f8e110dd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b906840 0x55f83bcbe320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48476/0 (socket says 172.21.15.67:48476) 2024-07-31T15:05:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.669+0000 7f8e110dd640 1 -- 172.21.15.67:0/222581699 learned_addr learned my addr 172.21.15.67:0/222581699 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.669+0000 7f8e110dd640 1 -- 172.21.15.67:0/222581699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b8e5b60 msgr2=0x55f83bcbdde0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:05:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.669+0000 7f8e110dd640 1 --2- 172.21.15.67:0/222581699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b8e5b60 0x55f83bcbdde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.669+0000 7f8e110dd640 1 -- 172.21.15.67:0/222581699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8dd980 msgr2=0x55f83bcc0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:55.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.669+0000 7f8e110dd640 1 --2- 172.21.15.67:0/222581699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8dd980 0x55f83bcc0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:55.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.669+0000 7f8e110dd640 1 -- 172.21.15.67:0/222581699 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83bcbe860 con 0x55f83b906840 2024-07-31T15:05:55.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.670+0000 7f8e110dd640 1 --2- 172.21.15.67:0/222581699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b906840 0x55f83bcbe320 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f8e04031500 tx=0x7f8e04031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633985f71e2fcc6c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:55.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.670+0000 7f8deffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:05:55.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.671+0000 7f8deffff640 1 -- 172.21.15.67:0/222581699 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04018030 con 0x55f83b906840 2024-07-31T15:05:55.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.671+0000 7f8deffff640 1 -- 172.21.15.67:0/222581699 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f8e0401e3e0 con 0x55f83b906840 2024-07-31T15:05:55.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.671+0000 7f8e19091740 1 -- 172.21.15.67:0/222581699 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df804b070 con 0x55f83b906840 2024-07-31T15:05:55.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.671+0000 7f8deffff640 1 -- 172.21.15.67:0/222581699 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8e0401ee40 con 0x55f83b906840 2024-07-31T15:05:55.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.673+0000 7f8e19091740 10 client.18618 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:55.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.673+0000 7f8e19091740 1 -- 172.21.15.67:0/222581699 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f83bd32be0 con 0x55f83b906840 2024-07-31T15:05:55.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.673+0000 7f8deffff640 1 -- 172.21.15.67:0/222581699 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8df804b070 con 0x55f83b906840 2024-07-31T15:05:55.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.673+0000 7f8deffff640 1 -- 172.21.15.67:0/222581699 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f8e0401f7e0 con 0x55f83b906840 2024-07-31T15:05:55.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.674+0000 7f8e19091740 10 client.18618 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:55.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.674+0000 7f8e19091740 10 client.18618 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:05:55.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.674+0000 7f8e19091740 20 client.18618 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:55.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.674+0000 7f8e19091740 1 --2- 172.21.15.67:0/222581699 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f83b89a570 0x55f83bd35450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:55.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.674+0000 7f8e19091740 4 client.18618 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}] 2024-07-31T15:05:55.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.674+0000 7f8e19091740 1 -- 172.21.15.67:0/222581699 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}) -- 0x55f83bc1f1a0 con 0x55f83b89a570 2024-07-31T15:05:55.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.684+0000 7f8e108dc640 1 --2- 172.21.15.67:0/222581699 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f83b89a570 0x55f83bd35450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:55.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.691+0000 7f8e108dc640 1 --2- 172.21.15.67:0/222581699 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f83b89a570 0x55f83bd35450 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f6eba805f725e99 server_cookie=d663db4512151d58 in_seq=0 out_seq=0 2024-07-31T15:05:55.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:55.691+0000 7f8deffff640 10 client.18618 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:05:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:05:55.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:55 smithi077 ceph-mon[37327]: pgmap v867: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 702 KiB/s wr, 211 op/s 2024-07-31T15:05:57.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:57 smithi067 ceph-mon[32750]: pgmap v868: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 590 KiB/s wr, 208 op/s 2024-07-31T15:05:57.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:57 smithi073 ceph-mon[36714]: pgmap v868: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 590 KiB/s wr, 208 op/s 2024-07-31T15:05:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.696+0000 7f2b7dcef640 1 -- 172.21.15.67:0/3609823090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b78102490 msgr2=0x7f2b781028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.696+0000 7f2b7dcef640 1 --2- 172.21.15.67:0/3609823090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b78102490 0x7f2b781028a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009900 tx=0x7f2b6802a3f0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.696+0000 7f2b7dcef640 1 -- 172.21.15.67:0/3609823090 shutdown_connections 2024-07-31T15:05:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.696+0000 7f2b7dcef640 1 --2- 172.21.15.67:0/3609823090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b78108720 0x7f2b7806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.696+0000 7f2b7dcef640 1 --2- 172.21.15.67:0/3609823090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b78107dd0 0x7f2b781081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.696+0000 7f2b7dcef640 1 --2- 172.21.15.67:0/3609823090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b78102490 0x7f2b781028a0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.696+0000 7f2b7dcef640 1 -- 172.21.15.67:0/3609823090 >> 172.21.15.67:0/3609823090 conn(0x7f2b78079160 msgr2=0x7f2b78079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.696+0000 7f2b7dcef640 1 -- 172.21.15.67:0/3609823090 shutdown_connections 2024-07-31T15:05:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.696+0000 7f2b7dcef640 1 -- 172.21.15.67:0/3609823090 wait complete. 2024-07-31T15:05:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.697+0000 7f2b7dcef640 1 Processor -- start 2024-07-31T15:05:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.698+0000 7f2b7dcef640 1 -- start start 2024-07-31T15:05:57.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.698+0000 7f2b7dcef640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b78102490 0x7f2b78114fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.698+0000 7f2b7dcef640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b78107dd0 0x7f2b780743d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.698+0000 7f2b7dcef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b78108720 0x7f2b78111280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.698+0000 7f2b7dcef640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7806c990 con 0x7f2b78107dd0 2024-07-31T15:05:57.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.698+0000 7f2b7dcef640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7806c810 con 0x7f2b78102490 2024-07-31T15:05:57.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.698+0000 7f2b7dcef640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7806cb10 con 0x7f2b78108720 2024-07-31T15:05:57.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.700+0000 7f2b77fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b78108720 0x7f2b78111280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.700+0000 7f2b76ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b78107dd0 0x7f2b780743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.700+0000 7f2b76ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b78107dd0 0x7f2b780743d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48496/0 (socket says 172.21.15.67:48496) 2024-07-31T15:05:57.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.700+0000 7f2b76ffd640 1 -- 172.21.15.67:0/1449254872 learned_addr learned my addr 172.21.15.67:0/1449254872 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.700+0000 7f2b777fe640 1 --2- 172.21.15.67:0/1449254872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b78102490 0x7f2b78114fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.700+0000 7f2b777fe640 1 -- 172.21.15.67:0/1449254872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b78108720 msgr2=0x7f2b78111280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.700+0000 7f2b777fe640 1 --2- 172.21.15.67:0/1449254872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b78108720 0x7f2b78111280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.700+0000 7f2b777fe640 1 -- 172.21.15.67:0/1449254872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b78107dd0 msgr2=0x7f2b780743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.700+0000 7f2b777fe640 1 --2- 172.21.15.67:0/1449254872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b78107dd0 0x7f2b780743d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.700+0000 7f2b777fe640 1 -- 172.21.15.67:0/1449254872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b78074dd0 con 0x7f2b78102490 2024-07-31T15:05:57.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.701+0000 7f2b777fe640 1 --2- 172.21.15.67:0/1449254872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b78102490 0x7f2b78114fa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2b6802a9e0 tx=0x7f2b68003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:57.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.701+0000 7f2b74ff9640 1 -- 172.21.15.67:0/1449254872 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6803f070 con 0x7f2b78102490 2024-07-31T15:05:57.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.701+0000 7f2b74ff9640 1 -- 172.21.15.67:0/1449254872 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b68031b50 con 0x7f2b78102490 2024-07-31T15:05:57.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.701+0000 7f2b74ff9640 1 -- 172.21.15.67:0/1449254872 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68035030 con 0x7f2b78102490 2024-07-31T15:05:57.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.701+0000 7f2b7dcef640 1 -- 172.21.15.67:0/1449254872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b78111820 con 0x7f2b78102490 2024-07-31T15:05:57.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.702+0000 7f2b7dcef640 1 -- 172.21.15.67:0/1449254872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b78111c80 con 0x7f2b78102490 2024-07-31T15:05:57.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.706+0000 7f2b84240740 1 Processor -- start 2024-07-31T15:05:57.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.706+0000 7f2b84240740 1 -- start start 2024-07-31T15:05:57.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.707+0000 7f2b84240740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 0x55ad091e2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.707+0000 7f2b84240740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091e2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.707+0000 7f2b84240740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e1c390 0x55ad091e2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.707+0000 7f2b84240740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ad09158bb0 con 0x55ad08e3d070 2024-07-31T15:05:57.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.707+0000 7f2b84240740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ad091480b0 con 0x55ad08e141b0 2024-07-31T15:05:57.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.707+0000 7f2b84240740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ad0914c0f0 con 0x55ad08e1c390 2024-07-31T15:05:57.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.707+0000 7f2b76ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e1c390 0x55ad091e2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.708+0000 7f2b77fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 0x55ad091e2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.708+0000 7f2b77fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 0x55ad091e2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48502/0 (socket says 172.21.15.67:48502) 2024-07-31T15:05:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.708+0000 7f2b77fff640 1 -- 172.21.15.67:0/1647529075 learned_addr learned my addr 172.21.15.67:0/1647529075 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.708+0000 7f2b74ff9640 1 -- 172.21.15.67:0/1449254872 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2b68042070 con 0x7f2b78102490 2024-07-31T15:05:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.708+0000 7f2b777fe640 1 --2- 172.21.15.67:0/1647529075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091e2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.709+0000 7f2b777fe640 1 -- 172.21.15.67:0/1647529075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e1c390 msgr2=0x55ad091e2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:57.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.709+0000 7f2b777fe640 1 --2- 172.21.15.67:0/1647529075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e1c390 0x55ad091e2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.709+0000 7f2b777fe640 1 -- 172.21.15.67:0/1647529075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 msgr2=0x55ad091e2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:57.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.709+0000 7f2b777fe640 1 --2- 172.21.15.67:0/1647529075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 0x55ad091e2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.709+0000 7f2b777fe640 1 -- 172.21.15.67:0/1647529075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad090eeb90 con 0x55ad08e141b0 2024-07-31T15:05:57.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.710+0000 7f2b777fe640 1 --2- 172.21.15.67:0/1647529075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091e2ab0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2b68101dc0 tx=0x7f2b6810ac60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9223824069ed0997 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:57.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.711+0000 7f2b6dffb640 1 -- 172.21.15.67:0/1647529075 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68113070 con 0x55ad08e141b0 2024-07-31T15:05:57.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.711+0000 7f2b6dffb640 1 -- 172.21.15.67:0/1647529075 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b6810cc60 con 0x55ad08e141b0 2024-07-31T15:05:57.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.711+0000 7f2b6dffb640 1 -- 172.21.15.67:0/1647529075 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6810e380 con 0x55ad08e141b0 2024-07-31T15:05:57.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.712+0000 7f2b74ff9640 1 --2- 172.21.15.67:0/1449254872 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2b640b3f50 0x7f2b640b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.712+0000 7f2b74ff9640 1 -- 172.21.15.67:0/1449254872 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f2b68104a40 con 0x7f2b78102490 2024-07-31T15:05:57.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.714+0000 7f2b84240740 1 -- 172.21.15.67:0/1647529075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 msgr2=0x55ad091e2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:57.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.714+0000 7f2b84240740 1 --2- 172.21.15.67:0/1647529075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091e2ab0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2b68101dc0 tx=0x7f2b6810ac60 comp rx=0 tx=0).stop 2024-07-31T15:05:57.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.715+0000 7f2b77fff640 1 --2- 172.21.15.67:0/1449254872 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2b640b3f50 0x7f2b640b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.715+0000 7f2b84240740 1 -- 172.21.15.67:0/1647529075 shutdown_connections 2024-07-31T15:05:57.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.715+0000 7f2b84240740 1 --2- 172.21.15.67:0/1647529075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e1c390 0x55ad091e2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.715+0000 7f2b84240740 1 --2- 172.21.15.67:0/1647529075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091e2ab0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.715+0000 7f2b84240740 1 --2- 172.21.15.67:0/1647529075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 0x55ad091e2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.715+0000 7f2b84240740 1 -- 172.21.15.67:0/1647529075 >> 172.21.15.67:0/1647529075 conn(0x55ad08e31460 msgr2=0x55ad091e0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:57.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.716+0000 7f2b77fff640 1 --2- 172.21.15.67:0/1449254872 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2b640b3f50 0x7f2b640b6460 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2b70005c40 tx=0x7f2b70009760 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:57.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.716+0000 7f2b84240740 1 -- 172.21.15.67:0/1647529075 shutdown_connections 2024-07-31T15:05:57.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.716+0000 7f2b84240740 1 -- 172.21.15.67:0/1647529075 wait complete. 2024-07-31T15:05:57.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.717+0000 7f2b84240740 1 -- start start 2024-07-31T15:05:57.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.717+0000 7f2b84240740 1 Processor -- start 2024-07-31T15:05:57.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.718+0000 7f2b84240740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e141b0 0x55ad091ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.718+0000 7f2b84240740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e1c390 0x55ad091ee750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.718+0000 7f2b84240740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 0x55ad091f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.719+0000 7f2b84240740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ad08fab700 con 0x55ad08e3d070 2024-07-31T15:05:57.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.719+0000 7f2b84240740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ad090ea9e0 con 0x55ad08e1c390 2024-07-31T15:05:57.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.719+0000 7f2b84240740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ad091f0b10 con 0x55ad08e141b0 2024-07-31T15:05:57.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.719+0000 7f2b84240740 10 client.0 _ll_register_callbacks cb 0x55ad091054f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:57.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.719+0000 7f2b77fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 0x55ad091f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.719+0000 7f2b77fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 0x55ad091f9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48506/0 (socket says 172.21.15.67:48506) 2024-07-31T15:05:57.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.719+0000 7f2b77fff640 1 -- 172.21.15.67:0/3737936171 learned_addr learned my addr 172.21.15.67:0/3737936171 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:57.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.720+0000 7f2b76ffd640 1 --2- 172.21.15.67:0/3737936171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e1c390 0x55ad091ee750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.720+0000 7f2b76ffd640 1 -- 172.21.15.67:0/3737936171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e141b0 msgr2=0x55ad091ee1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:05:57.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.720+0000 7f2b76ffd640 1 --2- 172.21.15.67:0/3737936171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e141b0 0x55ad091ee1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.720+0000 7f2b76ffd640 1 -- 172.21.15.67:0/3737936171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 msgr2=0x55ad091f9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:57.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.720+0000 7f2b76ffd640 1 --2- 172.21.15.67:0/3737936171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 0x55ad091f9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.720+0000 7f2b76ffd640 1 -- 172.21.15.67:0/3737936171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad08ea8650 con 0x55ad08e1c390 2024-07-31T15:05:57.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.721+0000 7f2b76ffd640 1 --2- 172.21.15.67:0/3737936171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e1c390 0x55ad091ee750 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2b7806ab70 tx=0x7f2b60012bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dcff0bbc64d93a4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:57.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.721+0000 7f2b56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:05:57.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.722+0000 7f2b56ffd640 1 -- 172.21.15.67:0/3737936171 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60004050 con 0x55ad08e1c390 2024-07-31T15:05:57.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.722+0000 7f2b84240740 1 -- 172.21.15.67:0/3737936171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b6810cc60 con 0x55ad08e1c390 2024-07-31T15:05:57.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.722+0000 7f2b56ffd640 1 -- 172.21.15.67:0/3737936171 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f2b600041f0 con 0x55ad08e1c390 2024-07-31T15:05:57.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.722+0000 7f2b56ffd640 1 -- 172.21.15.67:0/3737936171 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f2b600043f0 con 0x55ad08e1c390 2024-07-31T15:05:57.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.723+0000 7f2b84240740 10 client.28166 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:57.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.723+0000 7f2b84240740 1 -- 172.21.15.67:0/3737936171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55ad091fcb90 con 0x55ad08e1c390 2024-07-31T15:05:57.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.724+0000 7f2b56ffd640 1 -- 172.21.15.67:0/3737936171 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b6810cc60 con 0x55ad08e1c390 2024-07-31T15:05:57.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.724+0000 7f2b56ffd640 1 -- 172.21.15.67:0/3737936171 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f2b60003b50 con 0x55ad08e1c390 2024-07-31T15:05:57.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.724+0000 7f2b84240740 10 client.28166 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:57.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.724+0000 7f2b84240740 10 client.28166 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:05:57.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.724+0000 7f2b84240740 20 client.28166 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:57.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.724+0000 7f2b84240740 1 --2- 172.21.15.67:0/3737936171 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ad08dfb9f0 0x55ad09200550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.725+0000 7f2b84240740 4 client.28166 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:05:57.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.725+0000 7f2b84240740 1 -- 172.21.15.67:0/3737936171 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad0915ab00 con 0x55ad08dfb9f0 2024-07-31T15:05:57.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.725+0000 7f2b777fe640 1 --2- 172.21.15.67:0/3737936171 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ad08dfb9f0 0x55ad09200550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.726+0000 7f2b777fe640 1 --2- 172.21.15.67:0/3737936171 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ad08dfb9f0 0x55ad09200550 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48a771712f74eec9 server_cookie=7a17c880f67bbdc5 in_seq=0 out_seq=0 2024-07-31T15:05:57.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.726+0000 7f2b56ffd640 10 client.28166 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:05:57.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.732+0000 7f2b56ffd640 1 -- 172.21.15.67:0/3737936171 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad0915ab00 con 0x55ad08dfb9f0 2024-07-31T15:05:57.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.732+0000 7f2b56ffd640 10 client.28166 handle_command_reply: tid=0 2024-07-31T15:05:57.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.733+0000 7f2b84240740 1 client.28166 shutdown 2024-07-31T15:05:57.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.733+0000 7f2b84240740 1 -- 172.21.15.67:0/3737936171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e1c390 msgr2=0x55ad091ee750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:57.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.733+0000 7f2b84240740 1 --2- 172.21.15.67:0/3737936171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e1c390 0x55ad091ee750 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2b7806ab70 tx=0x7f2b60012bf0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.733+0000 7f2b84240740 1 -- 172.21.15.67:0/3737936171 shutdown_connections 2024-07-31T15:05:57.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.733+0000 7f2b84240740 1 --2- 172.21.15.67:0/3737936171 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ad08dfb9f0 0x55ad09200550 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.733+0000 7f2b84240740 1 --2- 172.21.15.67:0/3737936171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 0x55ad091f9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.733+0000 7f2b84240740 1 --2- 172.21.15.67:0/3737936171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e1c390 0x55ad091ee750 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.733+0000 7f2b84240740 1 --2- 172.21.15.67:0/3737936171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e141b0 0x55ad091ee1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.733+0000 7f2b56ffd640 0 client.28166 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:05:57.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.733+0000 7f2b84240740 1 -- 172.21.15.67:0/3737936171 >> 172.21.15.67:0/3737936171 conn(0x55ad08e31460 msgr2=0x55ad091dec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:57.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.734+0000 7f2b84240740 1 -- 172.21.15.67:0/3737936171 shutdown_connections 2024-07-31T15:05:57.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.734+0000 7f2b84240740 1 -- 172.21.15.67:0/3737936171 wait complete. 2024-07-31T15:05:57.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.734+0000 7f2b84240740 20 client.28166 trim_cache size 0 max 16384 2024-07-31T15:05:57.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:57 smithi077 ceph-mon[37327]: pgmap v868: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 590 KiB/s wr, 208 op/s 2024-07-31T15:05:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.756+0000 7f2b84240740 1 Processor -- start 2024-07-31T15:05:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.756+0000 7f2b84240740 1 -- start start 2024-07-31T15:05:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.756+0000 7f2b84240740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.756+0000 7f2b84240740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e1c390 0x55ad091dfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b84240740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 0x55ad091e0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b84240740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781c7ec0 con 0x55ad08e3d070 2024-07-31T15:05:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b84240740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781c9a00 con 0x55ad08e141b0 2024-07-31T15:05:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b84240740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781c8590 con 0x55ad08e1c390 2024-07-31T15:05:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b777fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b777fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091df350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40626/0 (socket says 172.21.15.67:40626) 2024-07-31T15:05:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b777fe640 1 -- 172.21.15.67:0/3743576521 learned_addr learned my addr 172.21.15.67:0/3743576521 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b76ffd640 1 --2- 172.21.15.67:0/3743576521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e1c390 0x55ad091dfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b77fff640 1 --2- 172.21.15.67:0/3743576521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 0x55ad091e0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b777fe640 1 -- 172.21.15.67:0/3743576521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e1c390 msgr2=0x55ad091dfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b777fe640 1 --2- 172.21.15.67:0/3743576521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e1c390 0x55ad091dfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b777fe640 1 -- 172.21.15.67:0/3743576521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 msgr2=0x55ad091e0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b777fe640 1 --2- 172.21.15.67:0/3743576521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 0x55ad091e0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b777fe640 1 -- 172.21.15.67:0/3743576521 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad091e4860 con 0x55ad08e141b0 2024-07-31T15:05:57.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.757+0000 7f2b777fe640 1 --2- 172.21.15.67:0/3743576521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091df350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2b680c7300 tx=0x7f2b6810ac60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=260714b01da2b52e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:57.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.758+0000 7f2b6d7fa640 1 -- 172.21.15.67:0/3743576521 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68113030 con 0x55ad08e141b0 2024-07-31T15:05:57.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.758+0000 7f2b6d7fa640 1 -- 172.21.15.67:0/3743576521 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b680cad90 con 0x55ad08e141b0 2024-07-31T15:05:57.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.758+0000 7f2b6d7fa640 1 -- 172.21.15.67:0/3743576521 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b680cabd0 con 0x55ad08e141b0 2024-07-31T15:05:57.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.759+0000 7f2b84240740 1 -- 172.21.15.67:0/3743576521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 msgr2=0x55ad091df350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:57.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.759+0000 7f2b84240740 1 --2- 172.21.15.67:0/3743576521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091df350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2b680c7300 tx=0x7f2b6810ac60 comp rx=0 tx=0).stop 2024-07-31T15:05:57.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.759+0000 7f2b84240740 1 -- 172.21.15.67:0/3743576521 shutdown_connections 2024-07-31T15:05:57.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.759+0000 7f2b84240740 1 --2- 172.21.15.67:0/3743576521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e3d070 0x55ad091e0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.759+0000 7f2b84240740 1 --2- 172.21.15.67:0/3743576521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e1c390 0x55ad091dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.759+0000 7f2b84240740 1 --2- 172.21.15.67:0/3743576521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091df350 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.759+0000 7f2b84240740 1 -- 172.21.15.67:0/3743576521 >> 172.21.15.67:0/3743576521 conn(0x55ad08e31460 msgr2=0x55ad091dec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:57.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.759+0000 7f2b84240740 1 -- 172.21.15.67:0/3743576521 shutdown_connections 2024-07-31T15:05:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.759+0000 7f2b84240740 1 -- 172.21.15.67:0/3743576521 wait complete. 2024-07-31T15:05:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.759+0000 7f2b84240740 1 -- start start 2024-07-31T15:05:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.760+0000 7f2b84240740 1 Processor -- start 2024-07-31T15:05:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.760+0000 7f2b84240740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091f1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.760+0000 7f2b84240740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e1c390 0x55ad091f1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.760+0000 7f2b84240740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e3d070 0x55ad09266d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.760+0000 7f2b84240740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781c9330 con 0x55ad08e1c390 2024-07-31T15:05:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.760+0000 7f2b84240740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781c8c60 con 0x55ad08e141b0 2024-07-31T15:05:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.760+0000 7f2b84240740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781b2ea0 con 0x55ad08e3d070 2024-07-31T15:05:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.760+0000 7f2b777fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.760+0000 7f2b76ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e1c390 0x55ad091f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.760+0000 7f2b84240740 10 client.0 _ll_register_callbacks cb 0x7f2b60009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:05:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.760+0000 7f2b76ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e1c390 0x55ad091f1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48532/0 (socket says 172.21.15.67:48532) 2024-07-31T15:05:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.760+0000 7f2b76ffd640 1 -- 172.21.15.67:0/2244443848 learned_addr learned my addr 172.21.15.67:0/2244443848 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:05:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.760+0000 7f2b777fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091f1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40640/0 (socket says 172.21.15.67:40640) 2024-07-31T15:05:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.760+0000 7f2b77fff640 1 --2- 172.21.15.67:0/2244443848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e3d070 0x55ad09266d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.761+0000 7f2b777fe640 1 -- 172.21.15.67:0/2244443848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e3d070 msgr2=0x55ad09266d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.761+0000 7f2b777fe640 1 --2- 172.21.15.67:0/2244443848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e3d070 0x55ad09266d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.761+0000 7f2b777fe640 1 -- 172.21.15.67:0/2244443848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e1c390 msgr2=0x55ad091f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:05:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.761+0000 7f2b777fe640 1 --2- 172.21.15.67:0/2244443848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e1c390 0x55ad091f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.761+0000 7f2b777fe640 1 -- 172.21.15.67:0/2244443848 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad09267250 con 0x55ad08e141b0 2024-07-31T15:05:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.761+0000 7f2b76ffd640 1 --2- 172.21.15.67:0/2244443848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e1c390 0x55ad091f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:05:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.761+0000 7f2b777fe640 1 --2- 172.21.15.67:0/2244443848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091f1300 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2b680c84f0 tx=0x7f2b68104c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dda07a1c2ee35590 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:05:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.761+0000 7f2b56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:05:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.761+0000 7f2b56ffd640 1 -- 172.21.15.67:0/2244443848 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68113070 con 0x55ad08e141b0 2024-07-31T15:05:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.761+0000 7f2b84240740 1 -- 172.21.15.67:0/2244443848 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b680cad90 con 0x55ad08e141b0 2024-07-31T15:05:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.761+0000 7f2b56ffd640 1 -- 172.21.15.67:0/2244443848 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f2b6810a300 con 0x55ad08e141b0 2024-07-31T15:05:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.761+0000 7f2b56ffd640 1 -- 172.21.15.67:0/2244443848 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f2b680ca9f0 con 0x55ad08e141b0 2024-07-31T15:05:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.762+0000 7f2b84240740 10 client.28172 fetch_fsmap learned FSMap version 63 2024-07-31T15:05:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.762+0000 7f2b84240740 1 -- 172.21.15.67:0/2244443848 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55ad092684b0 con 0x55ad08e141b0 2024-07-31T15:05:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.762+0000 7f2b56ffd640 1 -- 172.21.15.67:0/2244443848 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b680cad90 con 0x55ad08e141b0 2024-07-31T15:05:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.763+0000 7f2b56ffd640 1 -- 172.21.15.67:0/2244443848 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f2b680c7510 con 0x55ad08e141b0 2024-07-31T15:05:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.763+0000 7f2b84240740 10 client.28172 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:05:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.763+0000 7f2b84240740 10 client.28172 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:05:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.763+0000 7f2b84240740 20 client.28172 populate_metadata read hostname 'smithi067' 2024-07-31T15:05:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.763+0000 7f2b84240740 1 --2- 172.21.15.67:0/2244443848 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ad08dd0da0 0x55ad0926a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:05:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.763+0000 7f2b84240740 4 client.28172 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:05:57.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.763+0000 7f2b84240740 1 -- 172.21.15.67:0/2244443848 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad091565a0 con 0x55ad08dd0da0 2024-07-31T15:05:57.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.763+0000 7f2b76ffd640 1 --2- 172.21.15.67:0/2244443848 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ad08dd0da0 0x55ad0926a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:05:57.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.764+0000 7f2b76ffd640 1 --2- 172.21.15.67:0/2244443848 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ad08dd0da0 0x55ad0926a710 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4186dcfcf412604c server_cookie=d0a161eb63c53711 in_seq=0 out_seq=0 2024-07-31T15:05:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.764+0000 7f2b56ffd640 10 client.28172 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:05:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.765+0000 7f2b56ffd640 1 -- 172.21.15.67:0/2244443848 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad091565a0 con 0x55ad08dd0da0 2024-07-31T15:05:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.765+0000 7f2b56ffd640 10 client.28172 handle_command_reply: tid=0 2024-07-31T15:05:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.765+0000 7f2b84240740 1 client.28172 shutdown 2024-07-31T15:05:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.765+0000 7f2b84240740 1 -- 172.21.15.67:0/2244443848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 msgr2=0x55ad091f1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.765+0000 7f2b84240740 1 --2- 172.21.15.67:0/2244443848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091f1300 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2b680c84f0 tx=0x7f2b68104c90 comp rx=0 tx=0).stop 2024-07-31T15:05:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.766+0000 7f2b84240740 1 -- 172.21.15.67:0/2244443848 shutdown_connections 2024-07-31T15:05:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.766+0000 7f2b84240740 1 --2- 172.21.15.67:0/2244443848 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55ad08dd0da0 0x55ad0926a710 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.766+0000 7f2b56ffd640 0 client.28172 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:05:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.766+0000 7f2b84240740 1 --2- 172.21.15.67:0/2244443848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad08e3d070 0x55ad09266d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.766+0000 7f2b84240740 1 --2- 172.21.15.67:0/2244443848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad08e1c390 0x55ad091f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.766+0000 7f2b84240740 1 --2- 172.21.15.67:0/2244443848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad08e141b0 0x55ad091f1300 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.766+0000 7f2b84240740 1 -- 172.21.15.67:0/2244443848 >> 172.21.15.67:0/2244443848 conn(0x55ad08e31460 msgr2=0x55ad091e7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:57.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.766+0000 7f2b84240740 1 -- 172.21.15.67:0/2244443848 shutdown_connections 2024-07-31T15:05:57.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.766+0000 7f2b84240740 1 -- 172.21.15.67:0/2244443848 wait complete. 2024-07-31T15:05:57.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.766+0000 7f2b84240740 20 client.28172 trim_cache size 0 max 16384 2024-07-31T15:05:57.772 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:05:57.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.767+0000 7f2b7dcef640 1 -- 172.21.15.67:0/1449254872 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2b640b3f50 msgr2=0x7f2b640b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:57.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.767+0000 7f2b7dcef640 1 --2- 172.21.15.67:0/1449254872 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2b640b3f50 0x7f2b640b6460 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2b70005c40 tx=0x7f2b70009760 comp rx=0 tx=0).stop 2024-07-31T15:05:57.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.767+0000 7f2b7dcef640 1 -- 172.21.15.67:0/1449254872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b78102490 msgr2=0x7f2b78114fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:05:57.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.767+0000 7f2b7dcef640 1 --2- 172.21.15.67:0/1449254872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b78102490 0x7f2b78114fa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2b6802a9e0 tx=0x7f2b68003c00 comp rx=0 tx=0).stop 2024-07-31T15:05:57.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.767+0000 7f2b7dcef640 1 -- 172.21.15.67:0/1449254872 shutdown_connections 2024-07-31T15:05:57.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.767+0000 7f2b7dcef640 1 --2- 172.21.15.67:0/1449254872 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2b640b3f50 0x7f2b640b6460 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.767+0000 7f2b7dcef640 1 --2- 172.21.15.67:0/1449254872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b78108720 0x7f2b78111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.767+0000 7f2b7dcef640 1 --2- 172.21.15.67:0/1449254872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b78107dd0 0x7f2b780743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.767+0000 7f2b7dcef640 1 --2- 172.21.15.67:0/1449254872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b78102490 0x7f2b78114fa0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:05:57.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.767+0000 7f2b7dcef640 1 -- 172.21.15.67:0/1449254872 >> 172.21.15.67:0/1449254872 conn(0x7f2b78079160 msgr2=0x7f2b78103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:05:57.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.767+0000 7f2b7dcef640 1 -- 172.21.15.67:0/1449254872 shutdown_connections 2024-07-31T15:05:57.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:05:57.768+0000 7f2b7dcef640 1 -- 172.21.15.67:0/1449254872 wait complete. 2024-07-31T15:05:58.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T15:05:58.853 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:05:59.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:05:59 smithi067 ceph-mon[32750]: pgmap v869: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 229 op/s 2024-07-31T15:05:59.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:05:59 smithi073 ceph-mon[36714]: pgmap v869: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 229 op/s 2024-07-31T15:05:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:05:59 smithi077 ceph-mon[37327]: pgmap v869: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 229 op/s 2024-07-31T15:06:01.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.335+0000 7f39474b5640 1 -- 172.21.15.67:0/2160254246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3940105d10 msgr2=0x7f3940106120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.335+0000 7f39474b5640 1 --2- 172.21.15.67:0/2160254246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3940105d10 0x7f3940106120 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3930009900 tx=0x7f393002a410 comp rx=0 tx=0).stop 2024-07-31T15:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.335+0000 7f39474b5640 1 -- 172.21.15.67:0/2160254246 shutdown_connections 2024-07-31T15:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.335+0000 7f39474b5640 1 --2- 172.21.15.67:0/2160254246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3940106660 0x7f39401100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.335+0000 7f39474b5640 1 --2- 172.21.15.67:0/2160254246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3940105d10 0x7f3940106120 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.335+0000 7f39474b5640 1 --2- 172.21.15.67:0/2160254246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3940101a30 0x7f3940101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.335+0000 7f39474b5640 1 -- 172.21.15.67:0/2160254246 >> 172.21.15.67:0/2160254246 conn(0x7f3940079870 msgr2=0x7f3940079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.335+0000 7f39474b5640 1 -- 172.21.15.67:0/2160254246 shutdown_connections 2024-07-31T15:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.335+0000 7f39474b5640 1 -- 172.21.15.67:0/2160254246 wait complete. 2024-07-31T15:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.336+0000 7f39474b5640 1 Processor -- start 2024-07-31T15:06:01.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.336+0000 7f39474b5640 1 -- start start 2024-07-31T15:06:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.336+0000 7f39474b5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3940101a30 0x7f39401029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.336+0000 7f39474b5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3940105d10 0x7f3940102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.336+0000 7f39474b5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3940106660 0x7f3940103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f39474b5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f39401134d0 con 0x7f3940101a30 2024-07-31T15:06:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f39474b5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3940113350 con 0x7f3940105d10 2024-07-31T15:06:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f39474b5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3940113650 con 0x7f3940106660 2024-07-31T15:06:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f3945251640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3940101a30 0x7f39401029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f3945251640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3940101a30 0x7f39401029d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35272/0 (socket says 172.21.15.67:35272) 2024-07-31T15:06:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f3945251640 1 -- 172.21.15.67:0/510969554 learned_addr learned my addr 172.21.15.67:0/510969554 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f3944a50640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3940105d10 0x7f3940102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f3945a52640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3940106660 0x7f3940103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f3945251640 1 -- 172.21.15.67:0/510969554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3940106660 msgr2=0x7f3940103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f3945251640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3940106660 0x7f3940103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f3945251640 1 -- 172.21.15.67:0/510969554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3940105d10 msgr2=0x7f3940102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f3945251640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3940105d10 0x7f3940102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f3945251640 1 -- 172.21.15.67:0/510969554 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39401a81a0 con 0x7f3940101a30 2024-07-31T15:06:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f3944a50640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3940105d10 0x7f3940102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f3945251640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3940101a30 0x7f39401029d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f393c00e9f0 tx=0x7f393c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f39367fc640 1 -- 172.21.15.67:0/510969554 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c00cd80 con 0x7f3940101a30 2024-07-31T15:06:01.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.337+0000 7f39474b5640 1 -- 172.21.15.67:0/510969554 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39401a8490 con 0x7f3940101a30 2024-07-31T15:06:01.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.338+0000 7f39367fc640 1 -- 172.21.15.67:0/510969554 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f393c0108e0 con 0x7f3940101a30 2024-07-31T15:06:01.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.338+0000 7f39367fc640 1 -- 172.21.15.67:0/510969554 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c018980 con 0x7f3940101a30 2024-07-31T15:06:01.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.338+0000 7f39474b5640 1 -- 172.21.15.67:0/510969554 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39401a89f0 con 0x7f3940101a30 2024-07-31T15:06:01.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.341+0000 7f394da06740 1 Processor -- start 2024-07-31T15:06:01.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.341+0000 7f394da06740 1 -- start start 2024-07-31T15:06:01.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.341+0000 7f394da06740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 0x563dc7ea5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.341+0000 7f39367fc640 1 -- 172.21.15.67:0/510969554 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f393c010430 con 0x7f3940101a30 2024-07-31T15:06:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.341+0000 7f394da06740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7ea5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.341+0000 7f394da06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7adf390 0x563dc7ea5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.341+0000 7f394da06740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563dc7e1bbb0 con 0x563dc7ad71b0 2024-07-31T15:06:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.341+0000 7f394da06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563dc7e0b0b0 con 0x563dc7adf390 2024-07-31T15:06:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.341+0000 7f394da06740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563dc7e0f0f0 con 0x563dc7b00070 2024-07-31T15:06:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.341+0000 7f3945251640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7ea5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.341+0000 7f3945251640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7ea5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35286/0 (socket says 172.21.15.67:35286) 2024-07-31T15:06:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.341+0000 7f3945251640 1 -- 172.21.15.67:0/1642829355 learned_addr learned my addr 172.21.15.67:0/1642829355 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.342+0000 7f3945a52640 1 --2- 172.21.15.67:0/1642829355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 0x563dc7ea5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.342+0000 7f3944a50640 1 --2- 172.21.15.67:0/1642829355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7adf390 0x563dc7ea5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.342+0000 7f3945251640 1 -- 172.21.15.67:0/1642829355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 msgr2=0x563dc7ea5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:01.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.342+0000 7f3945251640 1 --2- 172.21.15.67:0/1642829355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 0x563dc7ea5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.342+0000 7f3945251640 1 -- 172.21.15.67:0/1642829355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7adf390 msgr2=0x563dc7ea5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:01.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.342+0000 7f3945251640 1 --2- 172.21.15.67:0/1642829355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7adf390 0x563dc7ea5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.342+0000 7f3945251640 1 -- 172.21.15.67:0/1642829355 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dc7db1b90 con 0x563dc7ad71b0 2024-07-31T15:06:01.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.342+0000 7f3944a50640 1 --2- 172.21.15.67:0/1642829355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7adf390 0x563dc7ea5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:01.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.342+0000 7f3945251640 1 --2- 172.21.15.67:0/1642829355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7ea5ab0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f393c0e19d0 tx=0x7f393c0ec690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facbec45d8ddf1a0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:01.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.342+0000 7f392b7fe640 1 -- 172.21.15.67:0/1642829355 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c0f3070 con 0x563dc7ad71b0 2024-07-31T15:06:01.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.342+0000 7f392b7fe640 1 -- 172.21.15.67:0/1642829355 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f393c0e4d30 con 0x563dc7ad71b0 2024-07-31T15:06:01.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.342+0000 7f392b7fe640 1 -- 172.21.15.67:0/1642829355 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c0ef6a0 con 0x563dc7ad71b0 2024-07-31T15:06:01.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.342+0000 7f39367fc640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f392c0b3d40 0x7f392c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.343+0000 7f3945a52640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f392c0b3d40 0x7f392c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.343+0000 7f39367fc640 1 -- 172.21.15.67:0/510969554 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f393c0e4680 con 0x7f3940101a30 2024-07-31T15:06:01.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.343+0000 7f394da06740 1 -- 172.21.15.67:0/1642829355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 msgr2=0x563dc7ea5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:01.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.343+0000 7f3945a52640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f392c0b3d40 0x7f392c0b6250 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3938005fd0 tx=0x7f3938009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:01.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.343+0000 7f394da06740 1 --2- 172.21.15.67:0/1642829355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7ea5ab0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f393c0e19d0 tx=0x7f393c0ec690 comp rx=0 tx=0).stop 2024-07-31T15:06:01.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.343+0000 7f394da06740 1 -- 172.21.15.67:0/1642829355 shutdown_connections 2024-07-31T15:06:01.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.343+0000 7f394da06740 1 --2- 172.21.15.67:0/1642829355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7adf390 0x563dc7ea5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.343+0000 7f394da06740 1 --2- 172.21.15.67:0/1642829355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7ea5ab0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.343+0000 7f394da06740 1 --2- 172.21.15.67:0/1642829355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 0x563dc7ea5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.343+0000 7f394da06740 1 -- 172.21.15.67:0/1642829355 >> 172.21.15.67:0/1642829355 conn(0x563dc7af4460 msgr2=0x563dc7ea3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:01.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.343+0000 7f394da06740 1 -- 172.21.15.67:0/1642829355 shutdown_connections 2024-07-31T15:06:01.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.344+0000 7f394da06740 1 -- 172.21.15.67:0/1642829355 wait complete. 2024-07-31T15:06:01.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.344+0000 7f394da06740 1 -- start start 2024-07-31T15:06:01.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.344+0000 7f394da06740 1 Processor -- start 2024-07-31T15:06:01.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.344+0000 7f394da06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7ad71b0 0x563dc7eb1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.344+0000 7f394da06740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7adf390 0x563dc7eb16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.344+0000 7f394da06740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 0x563dc7ebd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.344+0000 7f394da06740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563dc7c6e700 con 0x563dc7adf390 2024-07-31T15:06:01.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.344+0000 7f394da06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563dc7dad9e0 con 0x563dc7ad71b0 2024-07-31T15:06:01.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.344+0000 7f394da06740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563dc7eb3d50 con 0x563dc7b00070 2024-07-31T15:06:01.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.344+0000 7f394da06740 10 client.0 _ll_register_callbacks cb 0x563dc7dc84f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:01.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.344+0000 7f3944a50640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7adf390 0x563dc7eb16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.344+0000 7f3944a50640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7adf390 0x563dc7eb16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35288/0 (socket says 172.21.15.67:35288) 2024-07-31T15:06:01.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.344+0000 7f3944a50640 1 -- 172.21.15.67:0/3784196459 learned_addr learned my addr 172.21.15.67:0/3784196459 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:01.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.345+0000 7f3945251640 1 --2- 172.21.15.67:0/3784196459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7ad71b0 0x563dc7eb1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.345+0000 7f3945a52640 1 --2- 172.21.15.67:0/3784196459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 0x563dc7ebd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.345+0000 7f3944a50640 1 -- 172.21.15.67:0/3784196459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 msgr2=0x563dc7ebd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:01.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.345+0000 7f3944a50640 1 --2- 172.21.15.67:0/3784196459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 0x563dc7ebd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.345+0000 7f3944a50640 1 -- 172.21.15.67:0/3784196459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7ad71b0 msgr2=0x563dc7eb1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:01.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.345+0000 7f3944a50640 1 --2- 172.21.15.67:0/3784196459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7ad71b0 0x563dc7eb1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.345+0000 7f3944a50640 1 -- 172.21.15.67:0/3784196459 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dc7b6b650 con 0x563dc7adf390 2024-07-31T15:06:01.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.345+0000 7f3944a50640 1 --2- 172.21.15.67:0/3784196459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7adf390 0x563dc7eb16b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3930033000 tx=0x7f3930031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a727fdf9caf9c1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:01.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.345+0000 7f3928ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:06:01.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.345+0000 7f3928ff9640 1 -- 172.21.15.67:0/3784196459 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930035040 con 0x563dc7adf390 2024-07-31T15:06:01.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.345+0000 7f3928ff9640 1 -- 172.21.15.67:0/3784196459 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f3930004460 con 0x563dc7adf390 2024-07-31T15:06:01.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.345+0000 7f394da06740 1 -- 172.21.15.67:0/3784196459 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f393c0e4d30 con 0x563dc7adf390 2024-07-31T15:06:01.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.345+0000 7f3928ff9640 1 -- 172.21.15.67:0/3784196459 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f39300318e0 con 0x563dc7adf390 2024-07-31T15:06:01.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.345+0000 7f394da06740 10 client.18648 fetch_fsmap learned FSMap version 63 2024-07-31T15:06:01.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.346+0000 7f394da06740 1 -- 172.21.15.67:0/3784196459 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x563dc7ebfdc0 con 0x563dc7adf390 2024-07-31T15:06:01.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.346+0000 7f3928ff9640 1 -- 172.21.15.67:0/3784196459 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f393c0e4d30 con 0x563dc7adf390 2024-07-31T15:06:01.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.346+0000 7f3928ff9640 1 -- 172.21.15.67:0/3784196459 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f39300035f0 con 0x563dc7adf390 2024-07-31T15:06:01.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.346+0000 7f394da06740 10 client.18648 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:06:01.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.346+0000 7f394da06740 10 client.18648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:01.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.346+0000 7f394da06740 20 client.18648 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:01.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.346+0000 7f394da06740 1 --2- 172.21.15.67:0/3784196459 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563dc7abe9f0 0x563dc7ec3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.346+0000 7f394da06740 4 client.18648 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:06:01.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.346+0000 7f394da06740 1 -- 172.21.15.67:0/3784196459 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dc7e1db00 con 0x563dc7abe9f0 2024-07-31T15:06:01.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.346+0000 7f3945251640 1 --2- 172.21.15.67:0/3784196459 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563dc7abe9f0 0x563dc7ec3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.347+0000 7f3945251640 1 --2- 172.21.15.67:0/3784196459 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563dc7abe9f0 0x563dc7ec3290 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d937efe069eea7b2 server_cookie=22093a4b0f7fe0e4 in_seq=0 out_seq=0 2024-07-31T15:06:01.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.347+0000 7f3928ff9640 10 client.18648 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:01.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.358+0000 7f3928ff9640 1 -- 172.21.15.67:0/3784196459 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563dc7e1db00 con 0x563dc7abe9f0 2024-07-31T15:06:01.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.358+0000 7f3928ff9640 10 client.18648 handle_command_reply: tid=0 2024-07-31T15:06:01.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.358+0000 7f394da06740 1 client.18648 shutdown 2024-07-31T15:06:01.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.358+0000 7f394da06740 1 -- 172.21.15.67:0/3784196459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7adf390 msgr2=0x563dc7eb16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:01.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.358+0000 7f394da06740 1 --2- 172.21.15.67:0/3784196459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7adf390 0x563dc7eb16b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3930033000 tx=0x7f3930031740 comp rx=0 tx=0).stop 2024-07-31T15:06:01.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.358+0000 7f394da06740 1 -- 172.21.15.67:0/3784196459 shutdown_connections 2024-07-31T15:06:01.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.358+0000 7f394da06740 1 --2- 172.21.15.67:0/3784196459 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563dc7abe9f0 0x563dc7ec3290 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.358+0000 7f394da06740 1 --2- 172.21.15.67:0/3784196459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 0x563dc7ebd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.358+0000 7f394da06740 1 --2- 172.21.15.67:0/3784196459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7adf390 0x563dc7eb16b0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.358+0000 7f394da06740 1 --2- 172.21.15.67:0/3784196459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7ad71b0 0x563dc7eb1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.358+0000 7f394da06740 1 -- 172.21.15.67:0/3784196459 >> 172.21.15.67:0/3784196459 conn(0x563dc7af4460 msgr2=0x563dc7ea1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:01.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.358+0000 7f3928ff9640 0 client.18648 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:01.361 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:01 smithi067 ceph-mon[32750]: pgmap v870: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.5 MiB/s wr, 253 op/s 2024-07-31T15:06:01.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.359+0000 7f394da06740 1 -- 172.21.15.67:0/3784196459 shutdown_connections 2024-07-31T15:06:01.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.359+0000 7f394da06740 1 -- 172.21.15.67:0/3784196459 wait complete. 2024-07-31T15:06:01.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.359+0000 7f394da06740 20 client.18648 trim_cache size 0 max 16384 2024-07-31T15:06:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.373+0000 7f394da06740 1 Processor -- start 2024-07-31T15:06:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.373+0000 7f394da06740 1 -- start start 2024-07-31T15:06:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.373+0000 7f394da06740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7ea2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.373+0000 7f394da06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7b00070 0x563dc7ebce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.373+0000 7f394da06740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7a93da0 0x563dc7ebd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.373+0000 7f394da06740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f39401b2a40 con 0x563dc7ad71b0 2024-07-31T15:06:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.373+0000 7f394da06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f39401b4580 con 0x563dc7b00070 2024-07-31T15:06:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.373+0000 7f394da06740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f39401b3110 con 0x563dc7a93da0 2024-07-31T15:06:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f3945251640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7ea2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f3945251640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7ea2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35290/0 (socket says 172.21.15.67:35290) 2024-07-31T15:06:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f3945251640 1 -- 172.21.15.67:0/2880115998 learned_addr learned my addr 172.21.15.67:0/2880115998 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f3944a50640 1 --2- 172.21.15.67:0/2880115998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7b00070 0x563dc7ebce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f3945a52640 1 --2- 172.21.15.67:0/2880115998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7a93da0 0x563dc7ebd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f3945251640 1 -- 172.21.15.67:0/2880115998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7a93da0 msgr2=0x563dc7ebd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f3945251640 1 --2- 172.21.15.67:0/2880115998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7a93da0 0x563dc7ebd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f3945251640 1 -- 172.21.15.67:0/2880115998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7b00070 msgr2=0x563dc7ebce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f3945251640 1 --2- 172.21.15.67:0/2880115998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7b00070 0x563dc7ebce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f3945251640 1 -- 172.21.15.67:0/2880115998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dc7ea83d0 con 0x563dc7ad71b0 2024-07-31T15:06:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f3944a50640 1 --2- 172.21.15.67:0/2880115998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7b00070 0x563dc7ebce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:01.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f3945251640 1 --2- 172.21.15.67:0/2880115998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7ea2350 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f393c0e2060 tx=0x7f393c0e6890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beedb8a6d637a64f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:01.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f392affd640 1 -- 172.21.15.67:0/2880115998 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c0f3070 con 0x563dc7ad71b0 2024-07-31T15:06:01.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f392affd640 1 -- 172.21.15.67:0/2880115998 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f393c0ece90 con 0x563dc7ad71b0 2024-07-31T15:06:01.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.374+0000 7f392affd640 1 -- 172.21.15.67:0/2880115998 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c0aaae0 con 0x563dc7ad71b0 2024-07-31T15:06:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.375+0000 7f394da06740 1 -- 172.21.15.67:0/2880115998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 msgr2=0x563dc7ea2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.375+0000 7f394da06740 1 --2- 172.21.15.67:0/2880115998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7ea2350 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f393c0e2060 tx=0x7f393c0e6890 comp rx=0 tx=0).stop 2024-07-31T15:06:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.375+0000 7f394da06740 1 -- 172.21.15.67:0/2880115998 shutdown_connections 2024-07-31T15:06:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.375+0000 7f394da06740 1 --2- 172.21.15.67:0/2880115998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7a93da0 0x563dc7ebd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.375+0000 7f394da06740 1 --2- 172.21.15.67:0/2880115998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7b00070 0x563dc7ebce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.375+0000 7f394da06740 1 --2- 172.21.15.67:0/2880115998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7ea2350 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.375+0000 7f394da06740 1 -- 172.21.15.67:0/2880115998 >> 172.21.15.67:0/2880115998 conn(0x563dc7af4460 msgr2=0x563dc7ea1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:01.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.375+0000 7f394da06740 1 -- 172.21.15.67:0/2880115998 shutdown_connections 2024-07-31T15:06:01.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.375+0000 7f394da06740 1 -- 172.21.15.67:0/2880115998 wait complete. 2024-07-31T15:06:01.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.376+0000 7f394da06740 1 -- start start 2024-07-31T15:06:01.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.376+0000 7f394da06740 1 Processor -- start 2024-07-31T15:06:01.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.376+0000 7f394da06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7a93da0 0x563dc7eb98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.376+0000 7f394da06740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7eb4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.376+0000 7f394da06740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 0x563dc7eb54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.376+0000 7f394da06740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f39401b3eb0 con 0x563dc7ad71b0 2024-07-31T15:06:01.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.376+0000 7f394da06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f39401b37e0 con 0x563dc7a93da0 2024-07-31T15:06:01.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.376+0000 7f394da06740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f39401b0a40 con 0x563dc7b00070 2024-07-31T15:06:01.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.376+0000 7f394da06740 10 client.0 _ll_register_callbacks cb 0x7f3930009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:01.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.376+0000 7f3944a50640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7eb4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.376+0000 7f3944a50640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7eb4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35294/0 (socket says 172.21.15.67:35294) 2024-07-31T15:06:01.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.376+0000 7f3944a50640 1 -- 172.21.15.67:0/250501886 learned_addr learned my addr 172.21.15.67:0/250501886 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:01.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.376+0000 7f3945a52640 1 --2- 172.21.15.67:0/250501886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 0x563dc7eb54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.377+0000 7f3945251640 1 --2- 172.21.15.67:0/250501886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7a93da0 0x563dc7eb98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.377+0000 7f3944a50640 1 -- 172.21.15.67:0/250501886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 msgr2=0x563dc7eb54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:01.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.377+0000 7f3944a50640 1 --2- 172.21.15.67:0/250501886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 0x563dc7eb54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.377+0000 7f3944a50640 1 -- 172.21.15.67:0/250501886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7a93da0 msgr2=0x563dc7eb98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:01.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.377+0000 7f3944a50640 1 --2- 172.21.15.67:0/250501886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7a93da0 0x563dc7eb98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.377+0000 7f3944a50640 1 -- 172.21.15.67:0/250501886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dc7f2aa00 con 0x563dc7ad71b0 2024-07-31T15:06:01.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.377+0000 7f3944a50640 1 --2- 172.21.15.67:0/250501886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7eb4d90 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3930003b60 tx=0x7f39300314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3dfe724f2187c3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:01.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.377+0000 7f3928ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:06:01.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.377+0000 7f3928ff9640 1 -- 172.21.15.67:0/250501886 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930057060 con 0x563dc7ad71b0 2024-07-31T15:06:01.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.377+0000 7f3928ff9640 1 -- 172.21.15.67:0/250501886 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7f3930031a80 con 0x563dc7ad71b0 2024-07-31T15:06:01.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.377+0000 7f394da06740 1 -- 172.21.15.67:0/250501886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f393c0ece90 con 0x563dc7ad71b0 2024-07-31T15:06:01.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.377+0000 7f3928ff9640 1 -- 172.21.15.67:0/250501886 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f3930035070 con 0x563dc7ad71b0 2024-07-31T15:06:01.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.377+0000 7f394da06740 10 client.18654 fetch_fsmap learned FSMap version 63 2024-07-31T15:06:01.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.377+0000 7f394da06740 1 -- 172.21.15.67:0/250501886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x563dc7f2bb30 con 0x563dc7ad71b0 2024-07-31T15:06:01.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.378+0000 7f3928ff9640 1 -- 172.21.15.67:0/250501886 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f393c0ece90 con 0x563dc7ad71b0 2024-07-31T15:06:01.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.378+0000 7f3928ff9640 1 -- 172.21.15.67:0/250501886 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7f39300327c0 con 0x563dc7ad71b0 2024-07-31T15:06:01.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.378+0000 7f394da06740 10 client.18654 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:06:01.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.378+0000 7f394da06740 10 client.18654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:01.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.378+0000 7f394da06740 20 client.18654 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:01.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.378+0000 7f394da06740 1 --2- 172.21.15.67:0/250501886 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563dc7f2c2b0 0x563dc7f2e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:01.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.378+0000 7f394da06740 4 client.18654 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:06:01.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.378+0000 7f394da06740 1 -- 172.21.15.67:0/250501886 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563dc7e195a0 con 0x563dc7f2c2b0 2024-07-31T15:06:01.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.378+0000 7f3945251640 1 --2- 172.21.15.67:0/250501886 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563dc7f2c2b0 0x563dc7f2e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:01.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.379+0000 7f3928ff9640 10 client.18654 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:01.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.379+0000 7f3945251640 1 --2- 172.21.15.67:0/250501886 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563dc7f2c2b0 0x563dc7f2e6d0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aca1d770b81bcb6 server_cookie=93093d47bb09f12e in_seq=0 out_seq=0 2024-07-31T15:06:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f3928ff9640 1 -- 172.21.15.67:0/250501886 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563dc7e195a0 con 0x563dc7f2c2b0 2024-07-31T15:06:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f3928ff9640 10 client.18654 handle_command_reply: tid=0 2024-07-31T15:06:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f394da06740 1 client.18654 shutdown 2024-07-31T15:06:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f394da06740 1 -- 172.21.15.67:0/250501886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 msgr2=0x563dc7eb4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f394da06740 1 --2- 172.21.15.67:0/250501886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7eb4d90 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3930003b60 tx=0x7f39300314a0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f394da06740 1 -- 172.21.15.67:0/250501886 shutdown_connections 2024-07-31T15:06:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f394da06740 1 --2- 172.21.15.67:0/250501886 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563dc7f2c2b0 0x563dc7f2e6d0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f394da06740 1 --2- 172.21.15.67:0/250501886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563dc7b00070 0x563dc7eb54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f394da06740 1 --2- 172.21.15.67:0/250501886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563dc7ad71b0 0x563dc7eb4d90 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f394da06740 1 --2- 172.21.15.67:0/250501886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563dc7a93da0 0x563dc7eb98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f394da06740 1 -- 172.21.15.67:0/250501886 >> 172.21.15.67:0/250501886 conn(0x563dc7af4460 msgr2=0x563dc7ea65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f3928ff9640 0 client.18654 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f394da06740 1 -- 172.21.15.67:0/250501886 shutdown_connections 2024-07-31T15:06:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f394da06740 1 -- 172.21.15.67:0/250501886 wait complete. 2024-07-31T15:06:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.382+0000 7f394da06740 20 client.18654 trim_cache size 0 max 16384 2024-07-31T15:06:01.387 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:06:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.383+0000 7f39474b5640 1 -- 172.21.15.67:0/510969554 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f392c0b3d40 msgr2=0x7f392c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.383+0000 7f39474b5640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f392c0b3d40 0x7f392c0b6250 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3938005fd0 tx=0x7f3938009500 comp rx=0 tx=0).stop 2024-07-31T15:06:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.383+0000 7f39474b5640 1 -- 172.21.15.67:0/510969554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3940101a30 msgr2=0x7f39401029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.383+0000 7f39474b5640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3940101a30 0x7f39401029d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f393c00e9f0 tx=0x7f393c00eeb0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.383+0000 7f39474b5640 1 -- 172.21.15.67:0/510969554 shutdown_connections 2024-07-31T15:06:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.383+0000 7f39474b5640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f392c0b3d40 0x7f392c0b6250 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.383+0000 7f39474b5640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3940106660 0x7f3940103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.383+0000 7f39474b5640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3940105d10 0x7f3940102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.383+0000 7f39474b5640 1 --2- 172.21.15.67:0/510969554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3940101a30 0x7f39401029d0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.383+0000 7f39474b5640 1 -- 172.21.15.67:0/510969554 >> 172.21.15.67:0/510969554 conn(0x7f3940079870 msgr2=0x7f3940106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.383+0000 7f39474b5640 1 -- 172.21.15.67:0/510969554 shutdown_connections 2024-07-31T15:06:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:01.383+0000 7f39474b5640 1 -- 172.21.15.67:0/510969554 wait complete. 2024-07-31T15:06:01.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:01 smithi073 ceph-mon[36714]: pgmap v870: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.5 MiB/s wr, 253 op/s 2024-07-31T15:06:01.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:01 smithi077 ceph-mon[37327]: pgmap v870: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.5 MiB/s wr, 253 op/s 2024-07-31T15:06:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.776+0000 7fbdedaec640 1 -- 172.21.15.67:0/4213087760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde8107d20 msgr2=0x7fbde8108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.776+0000 7fbdedaec640 1 --2- 172.21.15.67:0/4213087760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde8107d20 0x7fbde8108130 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009900 tx=0x7fbdd002a410 comp rx=0 tx=0).stop 2024-07-31T15:06:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.776+0000 7fbdedaec640 1 -- 172.21.15.67:0/4213087760 shutdown_connections 2024-07-31T15:06:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.776+0000 7fbdedaec640 1 --2- 172.21.15.67:0/4213087760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbde8108670 0x7fbde810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.776+0000 7fbdedaec640 1 --2- 172.21.15.67:0/4213087760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde8107d20 0x7fbde8108130 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.776+0000 7fbdedaec640 1 --2- 172.21.15.67:0/4213087760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde81023f0 0x7fbde8102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.777+0000 7fbdedaec640 1 -- 172.21.15.67:0/4213087760 >> 172.21.15.67:0/4213087760 conn(0x7fbde80fddf0 msgr2=0x7fbde8100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.777+0000 7fbdedaec640 1 -- 172.21.15.67:0/4213087760 shutdown_connections 2024-07-31T15:06:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.777+0000 7fbdedaec640 1 -- 172.21.15.67:0/4213087760 wait complete. 2024-07-31T15:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.777+0000 7fbdedaec640 1 Processor -- start 2024-07-31T15:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.777+0000 7fbdedaec640 1 -- start start 2024-07-31T15:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbdedaec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbde81023f0 0x7fbde80fe320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbdedaec640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde8107d20 0x7fbde80fe860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbdedaec640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde8108670 0x7fbde80feda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbdedaec640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbde8112300 con 0x7fbde8108670 2024-07-31T15:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbdedaec640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbde8112180 con 0x7fbde8107d20 2024-07-31T15:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbdedaec640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbde8112480 con 0x7fbde81023f0 2024-07-31T15:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbde81023f0 0x7fbde80fe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbde6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde8107d20 0x7fbde80fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbde7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde8108670 0x7fbde80feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbde6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde8107d20 0x7fbde80fe860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:33724/0 (socket says 172.21.15.67:33724) 2024-07-31T15:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbde81023f0 0x7fbde80fe320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43552/0 (socket says 172.21.15.67:43552) 2024-07-31T15:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbde6ffd640 1 -- 172.21.15.67:0/3453339076 learned_addr learned my addr 172.21.15.67:0/3453339076 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbde7fff640 1 -- 172.21.15.67:0/3453339076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbde81023f0 msgr2=0x7fbde80fe320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbde7fff640 1 --2- 172.21.15.67:0/3453339076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbde81023f0 0x7fbde80fe320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbde7fff640 1 -- 172.21.15.67:0/3453339076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde8107d20 msgr2=0x7fbde80fe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbde7fff640 1 --2- 172.21.15.67:0/3453339076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde8107d20 0x7fbde80fe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbde7fff640 1 -- 172.21.15.67:0/3453339076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde81a8100 con 0x7fbde8108670 2024-07-31T15:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbde6ffd640 1 --2- 172.21.15.67:0/3453339076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde8107d20 0x7fbde80fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.778+0000 7fbde77fe640 1 --2- 172.21.15.67:0/3453339076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbde81023f0 0x7fbde80fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.779+0000 7fbde7fff640 1 --2- 172.21.15.67:0/3453339076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde8108670 0x7fbde80feda0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009fd0 tx=0x7fbde000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.779+0000 7fbde4ff9640 1 -- 172.21.15.67:0/3453339076 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde000ce20 con 0x7fbde8108670 2024-07-31T15:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.779+0000 7fbde4ff9640 1 -- 172.21.15.67:0/3453339076 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde0005020 con 0x7fbde8108670 2024-07-31T15:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.779+0000 7fbde4ff9640 1 -- 172.21.15.67:0/3453339076 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0010690 con 0x7fbde8108670 2024-07-31T15:06:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.779+0000 7fbdedaec640 1 -- 172.21.15.67:0/3453339076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde81a83f0 con 0x7fbde8108670 2024-07-31T15:06:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.779+0000 7fbdedaec640 1 -- 172.21.15.67:0/3453339076 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde81a8990 con 0x7fbde8108670 2024-07-31T15:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.782+0000 7fbde4ff9640 1 -- 172.21.15.67:0/3453339076 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbde00054d0 con 0x7fbde8108670 2024-07-31T15:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.782+0000 7fbdf49e5740 1 Processor -- start 2024-07-31T15:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.782+0000 7fbdf49e5740 1 -- start start 2024-07-31T15:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbdf49e5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b14b1b0 0x55fa5b4edce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbdf49e5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b1222f0 0x55fa5b4ee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbdf49e5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b12a4d0 0x55fa5b4ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbdf49e5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fa5b4633b0 con 0x55fa5b1222f0 2024-07-31T15:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbdf49e5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fa5b4528b0 con 0x55fa5b14b1b0 2024-07-31T15:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbdf49e5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fa5b4568f0 con 0x55fa5b12a4d0 2024-07-31T15:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b1222f0 0x55fa5b4ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbde6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b14b1b0 0x55fa5b4edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b1222f0 0x55fa5b4ee220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35314/0 (socket says 172.21.15.67:35314) 2024-07-31T15:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbde77fe640 1 -- 172.21.15.67:0/718573301 learned_addr learned my addr 172.21.15.67:0/718573301 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbde6ffd640 1 --2- 172.21.15.67:0/718573301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b12a4d0 0x55fa5b4ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbde77fe640 1 -- 172.21.15.67:0/718573301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b12a4d0 msgr2=0x55fa5b4ee760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbde77fe640 1 --2- 172.21.15.67:0/718573301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b12a4d0 0x55fa5b4ee760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbde77fe640 1 -- 172.21.15.67:0/718573301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b14b1b0 msgr2=0x55fa5b4edce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbde77fe640 1 --2- 172.21.15.67:0/718573301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b14b1b0 0x55fa5b4edce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbde77fe640 1 -- 172.21.15.67:0/718573301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5b3f99a0 con 0x55fa5b1222f0 2024-07-31T15:06:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbde4ff9640 1 --2- 172.21.15.67:0/3453339076 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbdd40b3df0 0x7fbdd40b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbde77fe640 1 --2- 172.21.15.67:0/718573301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b1222f0 0x55fa5b4ee220 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80114f0 tx=0x7fbdd80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d874d382da049b63 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.783+0000 7fbde6ffd640 1 --2- 172.21.15.67:0/3453339076 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbdd40b3df0 0x7fbdd40b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.784+0000 7fbde4ff9640 1 -- 172.21.15.67:0/3453339076 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fbde00e4940 con 0x7fbde8108670 2024-07-31T15:06:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.784+0000 7fbdddffb640 1 -- 172.21.15.67:0/718573301 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd801e070 con 0x55fa5b1222f0 2024-07-31T15:06:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.784+0000 7fbdddffb640 1 -- 172.21.15.67:0/718573301 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd80035f0 con 0x55fa5b1222f0 2024-07-31T15:06:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.784+0000 7fbdddffb640 1 -- 172.21.15.67:0/718573301 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd800f7a0 con 0x55fa5b1222f0 2024-07-31T15:06:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.785+0000 7fbdf49e5740 1 -- 172.21.15.67:0/718573301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b1222f0 msgr2=0x55fa5b4ee220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.785+0000 7fbde6ffd640 1 --2- 172.21.15.67:0/3453339076 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbdd40b3df0 0x7fbdd40b6300 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbde81079e0 tx=0x7fbdd0032f40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.785+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/718573301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b1222f0 0x55fa5b4ee220 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80114f0 tx=0x7fbdd80119b0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.785+0000 7fbdf49e5740 1 -- 172.21.15.67:0/718573301 shutdown_connections 2024-07-31T15:06:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.785+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/718573301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b12a4d0 0x55fa5b4ee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.785+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/718573301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b1222f0 0x55fa5b4ee220 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.785+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/718573301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b14b1b0 0x55fa5b4edce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.785+0000 7fbdf49e5740 1 -- 172.21.15.67:0/718573301 >> 172.21.15.67:0/718573301 conn(0x55fa5b13f300 msgr2=0x55fa5b4ebbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.785+0000 7fbdf49e5740 1 -- 172.21.15.67:0/718573301 shutdown_connections 2024-07-31T15:06:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.785+0000 7fbdf49e5740 1 -- 172.21.15.67:0/718573301 wait complete. 2024-07-31T15:06:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.785+0000 7fbdf49e5740 1 -- start start 2024-07-31T15:06:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.785+0000 7fbdf49e5740 1 Processor -- start 2024-07-31T15:06:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbdf49e5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b1222f0 0x55fa5b4f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbdf49e5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b12a4d0 0x55fa5b4f9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbdf49e5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b14b1b0 0x55fa5b5057b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbdf49e5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fa5b38cab0 con 0x55fa5b12a4d0 2024-07-31T15:06:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbdf49e5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fa5b3dacf0 con 0x55fa5b1222f0 2024-07-31T15:06:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbdf49e5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fa5b4fc460 con 0x55fa5b14b1b0 2024-07-31T15:06:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbdf49e5740 10 client.0 _ll_register_callbacks cb 0x55fa5b44c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbde6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b12a4d0 0x55fa5b4f9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b1222f0 0x55fa5b4f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbde6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b12a4d0 0x55fa5b4f9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35330/0 (socket says 172.21.15.67:35330) 2024-07-31T15:06:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbde6ffd640 1 -- 172.21.15.67:0/620136930 learned_addr learned my addr 172.21.15.67:0/620136930 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbde7fff640 1 --2- 172.21.15.67:0/620136930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b14b1b0 0x55fa5b5057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbde77fe640 1 -- 172.21.15.67:0/620136930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b14b1b0 msgr2=0x55fa5b5057b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbde77fe640 1 --2- 172.21.15.67:0/620136930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b14b1b0 0x55fa5b5057b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbde77fe640 1 -- 172.21.15.67:0/620136930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b12a4d0 msgr2=0x55fa5b4f9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbde77fe640 1 --2- 172.21.15.67:0/620136930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b12a4d0 0x55fa5b4f9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbde77fe640 1 -- 172.21.15.67:0/620136930 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa5b1b6790 con 0x55fa5b1222f0 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.786+0000 7fbde6ffd640 1 --2- 172.21.15.67:0/620136930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b12a4d0 0x55fa5b4f9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.787+0000 7fbde77fe640 1 --2- 172.21.15.67:0/620136930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b1222f0 0x55fa5b4f97d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbdd801c040 tx=0x7fbdd8009d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1af378f7e1483cfc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.787+0000 7fbdc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.787+0000 7fbdc6ffd640 1 -- 172.21.15.67:0/620136930 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd801e040 con 0x55fa5b1222f0 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.787+0000 7fbdc6ffd640 1 -- 172.21.15.67:0/620136930 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fbdd8016050 con 0x55fa5b1222f0 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.787+0000 7fbdf49e5740 1 -- 172.21.15.67:0/620136930 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdd80035f0 con 0x55fa5b1222f0 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.787+0000 7fbdc6ffd640 1 -- 172.21.15.67:0/620136930 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fbdd8031050 con 0x55fa5b1222f0 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.787+0000 7fbdf49e5740 10 client.28202 fetch_fsmap learned FSMap version 63 2024-07-31T15:06:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.787+0000 7fbdf49e5740 1 -- 172.21.15.67:0/620136930 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55fa5b22f620 con 0x55fa5b1222f0 2024-07-31T15:06:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.787+0000 7fbdc6ffd640 1 -- 172.21.15.67:0/620136930 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd80035f0 con 0x55fa5b1222f0 2024-07-31T15:06:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.788+0000 7fbdc6ffd640 1 -- 172.21.15.67:0/620136930 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fbdd8021030 con 0x55fa5b1222f0 2024-07-31T15:06:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.788+0000 7fbdf49e5740 10 client.28202 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:06:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.788+0000 7fbdf49e5740 10 client.28202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.788+0000 7fbdf49e5740 20 client.28202 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.788+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/620136930 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa5b109d80 0x55fa5b50b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.788+0000 7fbdf49e5740 4 client.28202 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:06:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.788+0000 7fbdf49e5740 1 -- 172.21.15.67:0/620136930 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa5b465300 con 0x55fa5b109d80 2024-07-31T15:06:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.788+0000 7fbde6ffd640 1 --2- 172.21.15.67:0/620136930 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa5b109d80 0x55fa5b50b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.788+0000 7fbdc6ffd640 10 client.28202 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.789+0000 7fbde6ffd640 1 --2- 172.21.15.67:0/620136930 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa5b109d80 0x55fa5b50b610 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e22b7ac2a066bdda server_cookie=28161abae6121779 in_seq=0 out_seq=0 2024-07-31T15:06:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.790+0000 7fbdc6ffd640 1 -- 172.21.15.67:0/620136930 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa5b465300 con 0x55fa5b109d80 2024-07-31T15:06:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.790+0000 7fbdc6ffd640 10 client.28202 handle_command_reply: tid=0 2024-07-31T15:06:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.791+0000 7fbdf49e5740 1 client.28202 shutdown 2024-07-31T15:06:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.791+0000 7fbdf49e5740 1 -- 172.21.15.67:0/620136930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b1222f0 msgr2=0x55fa5b4f97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.791+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/620136930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b1222f0 0x55fa5b4f97d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbdd801c040 tx=0x7fbdd8009d40 comp rx=0 tx=0).stop 2024-07-31T15:06:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.791+0000 7fbdf49e5740 1 -- 172.21.15.67:0/620136930 shutdown_connections 2024-07-31T15:06:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.791+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/620136930 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa5b109d80 0x55fa5b50b610 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.791+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/620136930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b14b1b0 0x55fa5b5057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.791+0000 7fbdc6ffd640 0 client.28202 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.791+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/620136930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b12a4d0 0x55fa5b4f9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.791+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/620136930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b1222f0 0x55fa5b4f97d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.791+0000 7fbdf49e5740 1 -- 172.21.15.67:0/620136930 >> 172.21.15.67:0/620136930 conn(0x55fa5b13f300 msgr2=0x55fa5b4eed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.791+0000 7fbdf49e5740 1 -- 172.21.15.67:0/620136930 shutdown_connections 2024-07-31T15:06:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.791+0000 7fbdf49e5740 1 -- 172.21.15.67:0/620136930 wait complete. 2024-07-31T15:06:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.791+0000 7fbdf49e5740 20 client.28202 trim_cache size 0 max 16384 2024-07-31T15:06:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.807+0000 7fbdf49e5740 1 Processor -- start 2024-07-31T15:06:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.807+0000 7fbdf49e5740 1 -- start start 2024-07-31T15:06:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.807+0000 7fbdf49e5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b109d80 0x55fa5b4f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.807+0000 7fbdf49e5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b1222f0 0x55fa5b5055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.807+0000 7fbdf49e5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b12a4d0 0x55fa5b505ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.807+0000 7fbdf49e5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b2a10 con 0x55fa5b109d80 2024-07-31T15:06:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.807+0000 7fbdf49e5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b4550 con 0x55fa5b12a4d0 2024-07-31T15:06:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.807+0000 7fbdf49e5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b30e0 con 0x55fa5b1222f0 2024-07-31T15:06:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.807+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b109d80 0x55fa5b4f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.807+0000 7fbde6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b1222f0 0x55fa5b5055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.808+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b109d80 0x55fa5b4f3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35336/0 (socket says 172.21.15.67:35336) 2024-07-31T15:06:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.808+0000 7fbde77fe640 1 -- 172.21.15.67:0/42224160 learned_addr learned my addr 172.21.15.67:0/42224160 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.808+0000 7fbde7fff640 1 --2- 172.21.15.67:0/42224160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b12a4d0 0x55fa5b505ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.808+0000 7fbde6ffd640 1 -- 172.21.15.67:0/42224160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b12a4d0 msgr2=0x55fa5b505ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.808+0000 7fbde6ffd640 1 --2- 172.21.15.67:0/42224160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b12a4d0 0x55fa5b505ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.808+0000 7fbde6ffd640 1 -- 172.21.15.67:0/42224160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b109d80 msgr2=0x55fa5b4f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.808+0000 7fbde6ffd640 1 --2- 172.21.15.67:0/42224160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b109d80 0x55fa5b4f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.808+0000 7fbde6ffd640 1 -- 172.21.15.67:0/42224160 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5b4f0af0 con 0x55fa5b1222f0 2024-07-31T15:06:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.808+0000 7fbde6ffd640 1 --2- 172.21.15.67:0/42224160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b1222f0 0x55fa5b5055a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0005f70 tx=0x7fbdd0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2abbdbb7fbbd5a52 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.808+0000 7fbddd7fa640 1 -- 172.21.15.67:0/42224160 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0035040 con 0x55fa5b1222f0 2024-07-31T15:06:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.808+0000 7fbddd7fa640 1 -- 172.21.15.67:0/42224160 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd005a030 con 0x55fa5b1222f0 2024-07-31T15:06:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.808+0000 7fbddd7fa640 1 -- 172.21.15.67:0/42224160 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd003d980 con 0x55fa5b1222f0 2024-07-31T15:06:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.809+0000 7fbdf49e5740 1 -- 172.21.15.67:0/42224160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b1222f0 msgr2=0x55fa5b5055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.809+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/42224160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b1222f0 0x55fa5b5055a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0005f70 tx=0x7fbdd0009260 comp rx=0 tx=0).stop 2024-07-31T15:06:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.809+0000 7fbdf49e5740 1 -- 172.21.15.67:0/42224160 shutdown_connections 2024-07-31T15:06:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.809+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/42224160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b12a4d0 0x55fa5b505ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.809+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/42224160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b1222f0 0x55fa5b5055a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.809+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/42224160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b109d80 0x55fa5b4f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.809+0000 7fbdf49e5740 1 -- 172.21.15.67:0/42224160 >> 172.21.15.67:0/42224160 conn(0x55fa5b13f300 msgr2=0x55fa5b4fb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.809+0000 7fbdf49e5740 1 -- 172.21.15.67:0/42224160 shutdown_connections 2024-07-31T15:06:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.809+0000 7fbdf49e5740 1 -- 172.21.15.67:0/42224160 wait complete. 2024-07-31T15:06:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.810+0000 7fbdf49e5740 1 -- start start 2024-07-31T15:06:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.810+0000 7fbdf49e5740 1 Processor -- start 2024-07-31T15:06:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.810+0000 7fbdf49e5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b109d80 0x55fa5b501f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.810+0000 7fbdf49e5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b1222f0 0x55fa5b4fe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.810+0000 7fbdf49e5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b12a4d0 0x55fa5b4fea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.810+0000 7fbdf49e5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b3e80 con 0x55fa5b109d80 2024-07-31T15:06:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.810+0000 7fbdf49e5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b37b0 con 0x55fa5b12a4d0 2024-07-31T15:06:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.810+0000 7fbdf49e5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b0a10 con 0x55fa5b1222f0 2024-07-31T15:06:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.810+0000 7fbdf49e5740 10 client.0 _ll_register_callbacks cb 0x7fbdd801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.810+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b109d80 0x55fa5b501f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.810+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b109d80 0x55fa5b501f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35348/0 (socket says 172.21.15.67:35348) 2024-07-31T15:06:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.810+0000 7fbde77fe640 1 -- 172.21.15.67:0/3242708886 learned_addr learned my addr 172.21.15.67:0/3242708886 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.810+0000 7fbde7fff640 1 --2- 172.21.15.67:0/3242708886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b12a4d0 0x55fa5b4fea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.810+0000 7fbde6ffd640 1 --2- 172.21.15.67:0/3242708886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b1222f0 0x55fa5b4fe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.811+0000 7fbde77fe640 1 -- 172.21.15.67:0/3242708886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b1222f0 msgr2=0x55fa5b4fe3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.811+0000 7fbde77fe640 1 --2- 172.21.15.67:0/3242708886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b1222f0 0x55fa5b4fe3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.811+0000 7fbde77fe640 1 -- 172.21.15.67:0/3242708886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b12a4d0 msgr2=0x55fa5b4fea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.811+0000 7fbde77fe640 1 --2- 172.21.15.67:0/3242708886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b12a4d0 0x55fa5b4fea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.811+0000 7fbde77fe640 1 -- 172.21.15.67:0/3242708886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa5b4f0210 con 0x55fa5b109d80 2024-07-31T15:06:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.811+0000 7fbde77fe640 1 --2- 172.21.15.67:0/3242708886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b109d80 0x55fa5b501f30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009250 tx=0x7fbdd8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafbfaa79e1d8b9e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.811+0000 7fbdc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:06:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.811+0000 7fbdc6ffd640 1 -- 172.21.15.67:0/3242708886 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd801e040 con 0x55fa5b109d80 2024-07-31T15:06:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.811+0000 7fbdc6ffd640 1 -- 172.21.15.67:0/3242708886 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9395+0+0 (secure 0 0 0) 0x7fbdd8011040 con 0x55fa5b109d80 2024-07-31T15:06:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.811+0000 7fbdf49e5740 1 -- 172.21.15.67:0/3242708886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdd005a030 con 0x55fa5b109d80 2024-07-31T15:06:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.811+0000 7fbdc6ffd640 1 -- 172.21.15.67:0/3242708886 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fbdd80045d0 con 0x55fa5b109d80 2024-07-31T15:06:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.811+0000 7fbdf49e5740 10 client.18672 fetch_fsmap learned FSMap version 63 2024-07-31T15:06:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.811+0000 7fbdf49e5740 1 -- 172.21.15.67:0/3242708886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55fa5b572b30 con 0x55fa5b109d80 2024-07-31T15:06:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.811+0000 7fbdc6ffd640 1 -- 172.21.15.67:0/3242708886 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd005a030 con 0x55fa5b109d80 2024-07-31T15:06:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.812+0000 7fbdc6ffd640 1 -- 172.21.15.67:0/3242708886 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 63) ==== 8370+0+0 (secure 0 0 0) 0x7fbdd8021040 con 0x55fa5b109d80 2024-07-31T15:06:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.812+0000 7fbdf49e5740 10 client.18672 fetch_fsmap finished waiting for FSMap version 63 2024-07-31T15:06:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.812+0000 7fbdf49e5740 10 client.18672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.812+0000 7fbdf49e5740 20 client.18672 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.812+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/3242708886 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa5b14b1b0 0x55fa5b575310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.812+0000 7fbdf49e5740 4 client.18672 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:06:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.812+0000 7fbdf49e5740 1 -- 172.21.15.67:0/3242708886 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa5b460da0 con 0x55fa5b14b1b0 2024-07-31T15:06:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.812+0000 7fbde6ffd640 1 --2- 172.21.15.67:0/3242708886 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa5b14b1b0 0x55fa5b575310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.812+0000 7fbdc6ffd640 10 client.18672 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:02.812+0000 7fbde6ffd640 1 --2- 172.21.15.67:0/3242708886 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa5b14b1b0 0x55fa5b575310 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79094712bc10513 server_cookie=8f6113209edb8782 in_seq=0 out_seq=0 2024-07-31T15:06:03.148 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T15:06:03.149 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T15:06:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdc6ffd640 1 -- 172.21.15.67:0/3242708886 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa5b460da0 con 0x55fa5b14b1b0 2024-07-31T15:06:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdc6ffd640 10 client.18672 handle_command_reply: tid=0 2024-07-31T15:06:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdf49e5740 1 client.18672 shutdown 2024-07-31T15:06:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdf49e5740 1 -- 172.21.15.67:0/3242708886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b109d80 msgr2=0x55fa5b501f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/3242708886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b109d80 0x55fa5b501f30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009250 tx=0x7fbdd8003820 comp rx=0 tx=0).stop 2024-07-31T15:06:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdf49e5740 1 -- 172.21.15.67:0/3242708886 shutdown_connections 2024-07-31T15:06:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/3242708886 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55fa5b14b1b0 0x55fa5b575310 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/3242708886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa5b12a4d0 0x55fa5b4fea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/3242708886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa5b1222f0 0x55fa5b4fe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdc6ffd640 0 client.18672 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdf49e5740 1 --2- 172.21.15.67:0/3242708886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fa5b109d80 0x55fa5b501f30 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdf49e5740 1 -- 172.21.15.67:0/3242708886 >> 172.21.15.67:0/3242708886 conn(0x55fa5b13f300 msgr2=0x55fa5b4f2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdf49e5740 1 -- 172.21.15.67:0/3242708886 shutdown_connections 2024-07-31T15:06:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdf49e5740 1 -- 172.21.15.67:0/3242708886 wait complete. 2024-07-31T15:06:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.255+0000 7fbdf49e5740 20 client.18672 trim_cache size 0 max 16384 2024-07-31T15:06:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:06:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:06:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:06:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:06:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.256+0000 7fbdedaec640 1 -- 172.21.15.67:0/3453339076 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbdd40b3df0 msgr2=0x7fbdd40b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.256+0000 7fbdedaec640 1 --2- 172.21.15.67:0/3453339076 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbdd40b3df0 0x7fbdd40b6300 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbde81079e0 tx=0x7fbdd0032f40 comp rx=0 tx=0).stop 2024-07-31T15:06:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.256+0000 7fbdedaec640 1 -- 172.21.15.67:0/3453339076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde8108670 msgr2=0x7fbde80feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.256+0000 7fbdedaec640 1 --2- 172.21.15.67:0/3453339076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde8108670 0x7fbde80feda0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009fd0 tx=0x7fbde000ef10 comp rx=0 tx=0).stop 2024-07-31T15:06:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.256+0000 7fbdedaec640 1 -- 172.21.15.67:0/3453339076 shutdown_connections 2024-07-31T15:06:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.256+0000 7fbdedaec640 1 --2- 172.21.15.67:0/3453339076 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbdd40b3df0 0x7fbdd40b6300 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.256+0000 7fbdedaec640 1 --2- 172.21.15.67:0/3453339076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde8108670 0x7fbde80feda0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.256+0000 7fbdedaec640 1 --2- 172.21.15.67:0/3453339076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde8107d20 0x7fbde80fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.256+0000 7fbdedaec640 1 --2- 172.21.15.67:0/3453339076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbde81023f0 0x7fbde80fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.256+0000 7fbdedaec640 1 -- 172.21.15.67:0/3453339076 >> 172.21.15.67:0/3453339076 conn(0x7fbde80fddf0 msgr2=0x7fbde8100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.256+0000 7fbdedaec640 1 -- 172.21.15.67:0/3453339076 shutdown_connections 2024-07-31T15:06:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:03.256+0000 7fbdedaec640 1 -- 172.21.15.67:0/3453339076 wait complete. 2024-07-31T15:06:03.463 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:06:03.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:03 smithi067 ceph-mon[32750]: pgmap v871: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-07-31T15:06:03.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:03 smithi073 ceph-mon[36714]: pgmap v871: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-07-31T15:06:03.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:03 smithi077 ceph-mon[37327]: pgmap v871: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-07-31T15:06:04.143 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:06:03 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[70341]: ignoring --setuser ceph since I am not root 2024-07-31T15:06:04.143 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:06:03 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[70341]: ignoring --setgroup ceph since I am not root 2024-07-31T15:06:04.143 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:06:03 smithi073 ceph-mds[70345]: -- 172.21.15.73:0/3214782649 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56380e30bc20 con 0x5638124ada80 2024-07-31T15:06:04.143 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:06:03 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[70341]: starting mds.e at 2024-07-31T15:06:04.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:04 smithi067 ceph-mon[32750]: daemon mds.e restarted 2024-07-31T15:06:04.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:04 smithi073 ceph-mon[36714]: daemon mds.e restarted 2024-07-31T15:06:04.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:04 smithi077 ceph-mon[37327]: daemon mds.e restarted 2024-07-31T15:06:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.380+0000 7f383181a640 1 -- 172.21.15.67:0/1037705199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3824095030 msgr2=0x7f3824095440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.380+0000 7f383181a640 1 --2- 172.21.15.67:0/1037705199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3824095030 0x7f3824095440 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f382c065eb0 tx=0x7f382c097cf0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.380+0000 7f383181a640 1 -- 172.21.15.67:0/1037705199 shutdown_connections 2024-07-31T15:06:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.380+0000 7f383181a640 1 --2- 172.21.15.67:0/1037705199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f382409b2c0 0x7f38240a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.380+0000 7f383181a640 1 --2- 172.21.15.67:0/1037705199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f382409a970 0x7f382409ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.380+0000 7f383181a640 1 --2- 172.21.15.67:0/1037705199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3824095030 0x7f3824095440 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.380+0000 7f383181a640 1 -- 172.21.15.67:0/1037705199 >> 172.21.15.67:0/1037705199 conn(0x7f382400cd80 msgr2=0x7f382400d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:05.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.385+0000 7f383181a640 1 -- 172.21.15.67:0/1037705199 shutdown_connections 2024-07-31T15:06:05.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.385+0000 7f383181a640 1 -- 172.21.15.67:0/1037705199 wait complete. 2024-07-31T15:06:05.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.386+0000 7f383181a640 1 Processor -- start 2024-07-31T15:06:05.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.386+0000 7f383181a640 1 -- start start 2024-07-31T15:06:05.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.386+0000 7f383181a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3824095030 0x7f38240a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.386+0000 7f383181a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f382409b2c0 0x7f38240a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.386+0000 7f383181a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38240ae500 0x7f38240ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.386+0000 7f383181a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f38240a3e00 con 0x7f3824095030 2024-07-31T15:06:05.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.386+0000 7f383181a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f38240a3c80 con 0x7f38240ae500 2024-07-31T15:06:05.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.386+0000 7f383181a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38240a3f80 con 0x7f382409b2c0 2024-07-31T15:06:05.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.387+0000 7f3830818640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3824095030 0x7f38240a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.387+0000 7f3830818640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3824095030 0x7f38240a5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35364/0 (socket says 172.21.15.67:35364) 2024-07-31T15:06:05.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.387+0000 7f3830818640 1 -- 172.21.15.67:0/978886319 learned_addr learned my addr 172.21.15.67:0/978886319 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:05.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.387+0000 7f382bfff640 1 --2- 172.21.15.67:0/978886319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f382409b2c0 0x7f38240a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.387+0000 7f3831019640 1 --2- 172.21.15.67:0/978886319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38240ae500 0x7f38240ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.390+0000 7f3831019640 1 -- 172.21.15.67:0/978886319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f382409b2c0 msgr2=0x7f38240a60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:05.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.390+0000 7f3831019640 1 --2- 172.21.15.67:0/978886319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f382409b2c0 0x7f38240a60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.390+0000 7f3831019640 1 -- 172.21.15.67:0/978886319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3824095030 msgr2=0x7f38240a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:05.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.390+0000 7f3831019640 1 --2- 172.21.15.67:0/978886319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3824095030 0x7f38240a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.390+0000 7f3831019640 1 -- 172.21.15.67:0/978886319 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38240af040 con 0x7f38240ae500 2024-07-31T15:06:05.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.395+0000 7f3831019640 1 --2- 172.21.15.67:0/978886319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38240ae500 0x7f38240ae910 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f381800d680 tx=0x7f381800db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.399+0000 7f3829ffb640 1 -- 172.21.15.67:0/978886319 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818017070 con 0x7f38240ae500 2024-07-31T15:06:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.399+0000 7f3829ffb640 1 -- 172.21.15.67:0/978886319 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3818009820 con 0x7f38240ae500 2024-07-31T15:06:05.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.399+0000 7f3829ffb640 1 -- 172.21.15.67:0/978886319 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818004ce0 con 0x7f38240ae500 2024-07-31T15:06:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.399+0000 7f383181a640 1 -- 172.21.15.67:0/978886319 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38240ab380 con 0x7f38240ae500 2024-07-31T15:06:05.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.399+0000 7f383181a640 1 -- 172.21.15.67:0/978886319 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38240ab840 con 0x7f38240ae500 2024-07-31T15:06:05.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.403+0000 7f3838fcd740 1 Processor -- start 2024-07-31T15:06:05.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.403+0000 7f3838fcd740 1 -- start start 2024-07-31T15:06:05.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f3838fcd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560988602350 0x5609889a7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f3838fcd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885d9490 0x5609889a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f3838fcd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x5609889a8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f3838fcd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56098888f2f0 con 0x5609885e1670 2024-07-31T15:06:05.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f3838fcd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56098891da80 con 0x5609885d9490 2024-07-31T15:06:05.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f3838fcd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56098890cf80 con 0x560988602350 2024-07-31T15:06:05.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f382bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560988602350 0x5609889a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f382bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560988602350 0x5609889a7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43612/0 (socket says 172.21.15.67:43612) 2024-07-31T15:06:05.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f382bfff640 1 -- 172.21.15.67:0/686552333 learned_addr learned my addr 172.21.15.67:0/686552333 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:05.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f382bfff640 1 --2- 172.21.15.67:0/686552333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x5609889a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f382bfff640 1 -- 172.21.15.67:0/686552333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885d9490 msgr2=0x5609889a7ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:06:05.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f382bfff640 1 --2- 172.21.15.67:0/686552333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885d9490 0x5609889a7ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f382bfff640 1 -- 172.21.15.67:0/686552333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 msgr2=0x5609889a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:05.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f382bfff640 1 --2- 172.21.15.67:0/686552333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x5609889a8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.404+0000 7f382bfff640 1 -- 172.21.15.67:0/686552333 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56098866d930 con 0x560988602350 2024-07-31T15:06:05.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.405+0000 7f382bfff640 1 --2- 172.21.15.67:0/686552333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560988602350 0x5609889a7580 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3824008810 tx=0x7f382000d240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd70ee058c9fc43 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:05.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.405+0000 7f3816ffd640 1 -- 172.21.15.67:0/686552333 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820016e50 con 0x560988602350 2024-07-31T15:06:05.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.405+0000 7f3816ffd640 1 -- 172.21.15.67:0/686552333 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38200188e0 con 0x560988602350 2024-07-31T15:06:05.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.405+0000 7f3816ffd640 1 -- 172.21.15.67:0/686552333 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38200178a0 con 0x560988602350 2024-07-31T15:06:05.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.406+0000 7f3838fcd740 1 -- 172.21.15.67:0/686552333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560988602350 msgr2=0x5609889a7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:05.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.407+0000 7f3838fcd740 1 --2- 172.21.15.67:0/686552333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560988602350 0x5609889a7580 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3824008810 tx=0x7f382000d240 comp rx=0 tx=0).stop 2024-07-31T15:06:05.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.407+0000 7f3829ffb640 1 -- 172.21.15.67:0/978886319 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f38180099d0 con 0x7f38240ae500 2024-07-31T15:06:05.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.408+0000 7f3829ffb640 1 --2- 172.21.15.67:0/978886319 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f381c0b3f50 0x7f381c0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.408+0000 7f3838fcd740 1 -- 172.21.15.67:0/686552333 shutdown_connections 2024-07-31T15:06:05.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.409+0000 7f3838fcd740 1 --2- 172.21.15.67:0/686552333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x5609889a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.409+0000 7f3838fcd740 1 --2- 172.21.15.67:0/686552333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885d9490 0x5609889a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.409+0000 7f3838fcd740 1 --2- 172.21.15.67:0/686552333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560988602350 0x5609889a7580 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.409+0000 7f3838fcd740 1 -- 172.21.15.67:0/686552333 >> 172.21.15.67:0/686552333 conn(0x5609885f6740 msgr2=0x5609889a5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:05.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.409+0000 7f3829ffb640 1 -- 172.21.15.67:0/978886319 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f38180e3ce0 con 0x7f38240ae500 2024-07-31T15:06:05.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.409+0000 7f3838fcd740 1 -- 172.21.15.67:0/686552333 shutdown_connections 2024-07-31T15:06:05.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.409+0000 7f382bfff640 1 --2- 172.21.15.67:0/978886319 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f381c0b3f50 0x7f381c0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.409+0000 7f3838fcd740 1 -- 172.21.15.67:0/686552333 wait complete. 2024-07-31T15:06:05.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.410+0000 7f382bfff640 1 --2- 172.21.15.67:0/978886319 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f381c0b3f50 0x7f381c0b6460 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3820000ea0 tx=0x7f382000f040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:05.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.410+0000 7f3838fcd740 1 -- start start 2024-07-31T15:06:05.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.411+0000 7f3838fcd740 1 Processor -- start 2024-07-31T15:06:05.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.411+0000 7f3838fcd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885d9490 0x5609889b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.411+0000 7f3838fcd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x5609889b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.411+0000 7f3838fcd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560988602350 0x5609889beed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.411+0000 7f3838fcd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5609888af4f0 con 0x5609885e1670 2024-07-31T15:06:05.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.411+0000 7f3838fcd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560988910fc0 con 0x5609885d9490 2024-07-31T15:06:05.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.411+0000 7f3838fcd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560988770890 con 0x560988602350 2024-07-31T15:06:05.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.411+0000 7f3838fcd740 10 client.0 _ll_register_callbacks cb 0x5609888b9500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.412+0000 7f382bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x5609889b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.412+0000 7f3830818640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885d9490 0x5609889b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.412+0000 7f382bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x5609889b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35388/0 (socket says 172.21.15.67:35388) 2024-07-31T15:06:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.412+0000 7f382bfff640 1 -- 172.21.15.67:0/3143094205 learned_addr learned my addr 172.21.15.67:0/3143094205 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.412+0000 7f3831019640 1 --2- 172.21.15.67:0/3143094205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560988602350 0x5609889beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.412+0000 7f3831019640 1 -- 172.21.15.67:0/3143094205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885d9490 msgr2=0x5609889b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.412+0000 7f3831019640 1 --2- 172.21.15.67:0/3143094205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885d9490 0x5609889b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.412+0000 7f3831019640 1 -- 172.21.15.67:0/3143094205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 msgr2=0x5609889b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:05.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.413+0000 7f3831019640 1 --2- 172.21.15.67:0/3143094205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x5609889b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.413+0000 7f3831019640 1 -- 172.21.15.67:0/3143094205 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609888bec50 con 0x560988602350 2024-07-31T15:06:05.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.413+0000 7f382bfff640 1 --2- 172.21.15.67:0/3143094205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x5609889b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:05.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.413+0000 7f3831019640 1 --2- 172.21.15.67:0/3143094205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560988602350 0x5609889beed0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f38180e1010 tx=0x7f38180a6a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba009fb115e88825 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:05.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.413+0000 7f380bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:06:05.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.413+0000 7f380bfff640 1 -- 172.21.15.67:0/3143094205 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818020070 con 0x560988602350 2024-07-31T15:06:05.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.413+0000 7f380bfff640 1 -- 172.21.15.67:0/3143094205 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f3818018e70 con 0x560988602350 2024-07-31T15:06:05.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.413+0000 7f3838fcd740 1 -- 172.21.15.67:0/3143094205 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38200188e0 con 0x560988602350 2024-07-31T15:06:05.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.415+0000 7f380bfff640 1 -- 172.21.15.67:0/3143094205 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f38180e14a0 con 0x560988602350 2024-07-31T15:06:05.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.415+0000 7f3838fcd740 10 client.28141 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:05.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.415+0000 7f3838fcd740 1 -- 172.21.15.67:0/3143094205 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5609889c1b20 con 0x560988602350 2024-07-31T15:06:05.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.416+0000 7f380bfff640 1 -- 172.21.15.67:0/3143094205 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38200188e0 con 0x560988602350 2024-07-31T15:06:05.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.416+0000 7f380bfff640 1 -- 172.21.15.67:0/3143094205 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f38180a5e40 con 0x560988602350 2024-07-31T15:06:05.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.416+0000 7f3838fcd740 10 client.28141 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:05.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.416+0000 7f3838fcd740 10 client.28141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:05.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.416+0000 7f3838fcd740 20 client.28141 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:05.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.416+0000 7f3838fcd740 1 --2- 172.21.15.67:0/3143094205 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5609885c0cd0 0x5609889c5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.416+0000 7f3838fcd740 4 client.28141 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:06:05.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.416+0000 7f3838fcd740 1 -- 172.21.15.67:0/3143094205 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56098891f9d0 con 0x5609885c0cd0 2024-07-31T15:06:05.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.416+0000 7f3830818640 1 --2- 172.21.15.67:0/3143094205 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5609885c0cd0 0x5609889c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.417+0000 7f3830818640 1 --2- 172.21.15.67:0/3143094205 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5609885c0cd0 0x5609889c5360 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6b2cfcb50090283 server_cookie=1d37f645953361dc in_seq=0 out_seq=0 2024-07-31T15:06:05.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.417+0000 7f380bfff640 10 client.28141 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:05.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:05 smithi067 ceph-mon[32750]: pgmap v872: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-07-31T15:06:05.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:05 smithi067 ceph-mon[32750]: osdmap e113: 12 total, 12 up, 12 in 2024-07-31T15:06:05.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:05 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.73:6834/1516494761,v1:172.21.15.73:6835/1516494761] up:boot 2024-07-31T15:06:05.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:05 smithi067 ceph-mon[32750]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-07-31T15:06:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:05 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T15:06:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:05 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:06:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:05 smithi073 ceph-mon[36714]: pgmap v872: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-07-31T15:06:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:05 smithi073 ceph-mon[36714]: osdmap e113: 12 total, 12 up, 12 in 2024-07-31T15:06:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:05 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.73:6834/1516494761,v1:172.21.15.73:6835/1516494761] up:boot 2024-07-31T15:06:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:05 smithi073 ceph-mon[36714]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-07-31T15:06:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:05 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T15:06:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:05 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:06:05.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.730+0000 7f380bfff640 1 -- 172.21.15.67:0/3143094205 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56098891f9d0 con 0x5609885c0cd0 2024-07-31T15:06:05.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.730+0000 7f380bfff640 10 client.28141 handle_command_reply: tid=0 2024-07-31T15:06:05.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.730+0000 7f3838fcd740 1 client.28141 shutdown 2024-07-31T15:06:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.730+0000 7f3838fcd740 1 -- 172.21.15.67:0/3143094205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560988602350 msgr2=0x5609889beed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.730+0000 7f3838fcd740 1 --2- 172.21.15.67:0/3143094205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560988602350 0x5609889beed0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f38180e1010 tx=0x7f38180a6a10 comp rx=0 tx=0).stop 2024-07-31T15:06:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.730+0000 7f3838fcd740 1 -- 172.21.15.67:0/3143094205 shutdown_connections 2024-07-31T15:06:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.730+0000 7f3838fcd740 1 --2- 172.21.15.67:0/3143094205 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5609885c0cd0 0x5609889c5360 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.730+0000 7f380bfff640 0 client.28141 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.730+0000 7f3838fcd740 1 --2- 172.21.15.67:0/3143094205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560988602350 0x5609889beed0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.730+0000 7f3838fcd740 1 --2- 172.21.15.67:0/3143094205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x5609889b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.730+0000 7f3838fcd740 1 --2- 172.21.15.67:0/3143094205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885d9490 0x5609889b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.730+0000 7f3838fcd740 1 -- 172.21.15.67:0/3143094205 >> 172.21.15.67:0/3143094205 conn(0x5609885f6740 msgr2=0x5609889aa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.730+0000 7f3838fcd740 1 -- 172.21.15.67:0/3143094205 shutdown_connections 2024-07-31T15:06:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.730+0000 7f3838fcd740 1 -- 172.21.15.67:0/3143094205 wait complete. 2024-07-31T15:06:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.731+0000 7f3838fcd740 20 client.28141 trim_cache size 0 max 16384 2024-07-31T15:06:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:05 smithi077 ceph-mon[37327]: pgmap v872: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-07-31T15:06:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:05 smithi077 ceph-mon[37327]: osdmap e113: 12 total, 12 up, 12 in 2024-07-31T15:06:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:05 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.73:6834/1516494761,v1:172.21.15.73:6835/1516494761] up:boot 2024-07-31T15:06:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:05 smithi077 ceph-mon[37327]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-07-31T15:06:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:05 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T15:06:05.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:05 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:06:05.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.766+0000 7f3838fcd740 1 Processor -- start 2024-07-31T15:06:05.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.766+0000 7f3838fcd740 1 -- start start 2024-07-31T15:06:05.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.767+0000 7f3838fcd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885c0cd0 0x5609889aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.767+0000 7f3838fcd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609885d9490 0x5609889a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.767+0000 7f3838fcd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885e1670 0x5609889becc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.767+0000 7f382bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609885d9490 0x5609889a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.767+0000 7f382bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609885d9490 0x5609889a8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43628/0 (socket says 172.21.15.67:43628) 2024-07-31T15:06:05.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.767+0000 7f382bfff640 1 -- 172.21.15.67:0/2939221496 learned_addr learned my addr 172.21.15.67:0/2939221496 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:05.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.767+0000 7f3838fcd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3824152640 con 0x5609885c0cd0 2024-07-31T15:06:05.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.767+0000 7f3838fcd740 1 -- 172.21.15.67:0/2939221496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3824154180 con 0x5609885e1670 2024-07-31T15:06:05.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.767+0000 7f3838fcd740 1 -- 172.21.15.67:0/2939221496 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3824152d10 con 0x5609885d9490 2024-07-31T15:06:05.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.768+0000 7f3830818640 1 --2- 172.21.15.67:0/2939221496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885c0cd0 0x5609889aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.768+0000 7f3831019640 1 --2- 172.21.15.67:0/2939221496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885e1670 0x5609889becc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.768+0000 7f3830818640 1 -- 172.21.15.67:0/2939221496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609885d9490 msgr2=0x5609889a8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.768+0000 7f3830818640 1 --2- 172.21.15.67:0/2939221496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609885d9490 0x5609889a8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.768+0000 7f3830818640 1 -- 172.21.15.67:0/2939221496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885e1670 msgr2=0x5609889becc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.768+0000 7f3830818640 1 --2- 172.21.15.67:0/2939221496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885e1670 0x5609889becc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.768+0000 7f3830818640 1 -- 172.21.15.67:0/2939221496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609889b4190 con 0x5609885c0cd0 2024-07-31T15:06:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.769+0000 7f382bfff640 1 --2- 172.21.15.67:0/2939221496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609885d9490 0x5609889a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.769+0000 7f3831019640 1 --2- 172.21.15.67:0/2939221496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885e1670 0x5609889becc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:06:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.769+0000 7f3830818640 1 --2- 172.21.15.67:0/2939221496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885c0cd0 0x5609889aacf0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f382c0666a0 tx=0x7f382c068fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f194a8c2675819a9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.770+0000 7f38167fc640 1 -- 172.21.15.67:0/2939221496 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c09a3d0 con 0x5609885c0cd0 2024-07-31T15:06:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.770+0000 7f38167fc640 1 -- 172.21.15.67:0/2939221496 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f382c09c040 con 0x5609885c0cd0 2024-07-31T15:06:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.770+0000 7f38167fc640 1 -- 172.21.15.67:0/2939221496 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c098dd0 con 0x5609885c0cd0 2024-07-31T15:06:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.771+0000 7f3838fcd740 1 -- 172.21.15.67:0/2939221496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885c0cd0 msgr2=0x5609889aacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.771+0000 7f3838fcd740 1 --2- 172.21.15.67:0/2939221496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885c0cd0 0x5609889aacf0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f382c0666a0 tx=0x7f382c068fa0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.771+0000 7f3838fcd740 1 -- 172.21.15.67:0/2939221496 shutdown_connections 2024-07-31T15:06:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.771+0000 7f3838fcd740 1 --2- 172.21.15.67:0/2939221496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885e1670 0x5609889becc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.771+0000 7f3838fcd740 1 --2- 172.21.15.67:0/2939221496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609885d9490 0x5609889a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.771+0000 7f3838fcd740 1 --2- 172.21.15.67:0/2939221496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885c0cd0 0x5609889aacf0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.771+0000 7f3838fcd740 1 -- 172.21.15.67:0/2939221496 >> 172.21.15.67:0/2939221496 conn(0x5609885f6740 msgr2=0x5609889a85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.772+0000 7f3838fcd740 1 -- 172.21.15.67:0/2939221496 shutdown_connections 2024-07-31T15:06:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.772+0000 7f3838fcd740 1 -- 172.21.15.67:0/2939221496 wait complete. 2024-07-31T15:06:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.772+0000 7f3838fcd740 1 -- start start 2024-07-31T15:06:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.772+0000 7f3838fcd740 1 Processor -- start 2024-07-31T15:06:05.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.773+0000 7f3838fcd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885c0cd0 0x5609889b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.773+0000 7f3838fcd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609885d9490 0x5609889b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.773+0000 7f3838fcd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x560988a2cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.773+0000 7f3838fcd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3824153ab0 con 0x5609885e1670 2024-07-31T15:06:05.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.773+0000 7f3838fcd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f38241533e0 con 0x5609885c0cd0 2024-07-31T15:06:05.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.773+0000 7f3838fcd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3824150640 con 0x5609885d9490 2024-07-31T15:06:05.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.773+0000 7f3838fcd740 10 client.0 _ll_register_callbacks cb 0x7f3818004e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:05.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.773+0000 7f3830818640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885c0cd0 0x5609889b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.773+0000 7f382bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609885d9490 0x5609889b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.773+0000 7f3831019640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x560988a2cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.773+0000 7f3830818640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885c0cd0 0x5609889b6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:33808/0 (socket says 172.21.15.67:33808) 2024-07-31T15:06:05.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.773+0000 7f3830818640 1 -- 172.21.15.67:0/377933559 learned_addr learned my addr 172.21.15.67:0/377933559 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:05.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.774+0000 7f3831019640 1 -- 172.21.15.67:0/377933559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609885d9490 msgr2=0x5609889b74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:05.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.774+0000 7f3831019640 1 --2- 172.21.15.67:0/377933559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609885d9490 0x5609889b74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.774+0000 7f3831019640 1 -- 172.21.15.67:0/377933559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885c0cd0 msgr2=0x5609889b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:05.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.774+0000 7f3831019640 1 --2- 172.21.15.67:0/377933559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885c0cd0 0x5609889b6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.774+0000 7f3831019640 1 -- 172.21.15.67:0/377933559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609889b7d10 con 0x5609885e1670 2024-07-31T15:06:05.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.774+0000 7f3830818640 1 --2- 172.21.15.67:0/377933559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885c0cd0 0x5609889b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:05.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.774+0000 7f3831019640 1 --2- 172.21.15.67:0/377933559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x560988a2cb30 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f38180b12c0 tx=0x7f38180ae9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb750044aef8827c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:05.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.774+0000 7f380bfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:06:05.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f380bfff640 1 -- 172.21.15.67:0/377933559 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818020070 con 0x5609885e1670 2024-07-31T15:06:05.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f380bfff640 1 -- 172.21.15.67:0/377933559 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f38180a6160 con 0x5609885e1670 2024-07-31T15:06:05.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f382bfff640 1 --2- 172.21.15.67:0/377933559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609885d9490 0x5609889b74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:05.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f3838fcd740 1 -- 172.21.15.67:0/377933559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f382c09c040 con 0x5609885e1670 2024-07-31T15:06:05.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f380bfff640 1 -- 172.21.15.67:0/377933559 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f38180a88a0 con 0x5609885e1670 2024-07-31T15:06:05.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f3838fcd740 10 client.18696 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:05.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f3838fcd740 1 -- 172.21.15.67:0/377933559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x560988a2e090 con 0x5609885e1670 2024-07-31T15:06:05.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f380bfff640 1 -- 172.21.15.67:0/377933559 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f382c09c040 con 0x5609885e1670 2024-07-31T15:06:05.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f380bfff640 1 -- 172.21.15.67:0/377933559 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f38180a5e40 con 0x5609885e1670 2024-07-31T15:06:05.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f3838fcd740 10 client.18696 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:05.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f3838fcd740 10 client.18696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:05.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f3838fcd740 20 client.18696 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:05.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f3838fcd740 1 --2- 172.21.15.67:0/377933559 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560988602350 0x560988a30800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:05.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f3838fcd740 4 client.18696 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T15:06:05.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.775+0000 7f3838fcd740 1 -- 172.21.15.67:0/377933559 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56098891b470 con 0x560988602350 2024-07-31T15:06:05.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.776+0000 7f3830818640 1 --2- 172.21.15.67:0/377933559 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560988602350 0x560988a30800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:05.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.776+0000 7f380bfff640 10 client.18696 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:05.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.776+0000 7f3830818640 1 --2- 172.21.15.67:0/377933559 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560988602350 0x560988a30800 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d73a109aed717f1 server_cookie=4487097068de5743 in_seq=0 out_seq=0 2024-07-31T15:06:05.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.862+0000 7f380bfff640 1 -- 172.21.15.67:0/377933559 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56098891b470 con 0x560988602350 2024-07-31T15:06:05.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.862+0000 7f380bfff640 10 client.18696 handle_command_reply: tid=0 2024-07-31T15:06:05.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.862+0000 7f3838fcd740 1 client.18696 shutdown 2024-07-31T15:06:05.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.862+0000 7f3838fcd740 1 -- 172.21.15.67:0/377933559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 msgr2=0x560988a2cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:05.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.862+0000 7f3838fcd740 1 --2- 172.21.15.67:0/377933559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x560988a2cb30 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f38180b12c0 tx=0x7f38180ae9d0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.863+0000 7f3838fcd740 1 -- 172.21.15.67:0/377933559 shutdown_connections 2024-07-31T15:06:05.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.863+0000 7f3838fcd740 1 --2- 172.21.15.67:0/377933559 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560988602350 0x560988a30800 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.863+0000 7f3838fcd740 1 --2- 172.21.15.67:0/377933559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609885e1670 0x560988a2cb30 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.863+0000 7f3838fcd740 1 --2- 172.21.15.67:0/377933559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609885d9490 0x5609889b74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.863+0000 7f3838fcd740 1 --2- 172.21.15.67:0/377933559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609885c0cd0 0x5609889b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.863+0000 7f3838fcd740 1 -- 172.21.15.67:0/377933559 >> 172.21.15.67:0/377933559 conn(0x5609885f6740 msgr2=0x5609889a85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:05.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.863+0000 7f380bfff640 0 client.18696 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:05.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.863+0000 7f3838fcd740 1 -- 172.21.15.67:0/377933559 shutdown_connections 2024-07-31T15:06:05.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.863+0000 7f3838fcd740 1 -- 172.21.15.67:0/377933559 wait complete. 2024-07-31T15:06:05.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.863+0000 7f3838fcd740 20 client.18696 trim_cache size 0 max 16384 2024-07-31T15:06:05.866 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:06:05.866 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T15:06:05.866 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "6baaca53-fffa-4a41-8804-c4df7946cf59", 2024-07-31T15:06:05.866 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T15:06:05.867 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:06:05.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.864+0000 7f383181a640 1 -- 172.21.15.67:0/978886319 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f381c0b3f50 msgr2=0x7f381c0b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:05.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.864+0000 7f383181a640 1 --2- 172.21.15.67:0/978886319 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f381c0b3f50 0x7f381c0b6460 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3820000ea0 tx=0x7f382000f040 comp rx=0 tx=0).stop 2024-07-31T15:06:05.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.864+0000 7f383181a640 1 -- 172.21.15.67:0/978886319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38240ae500 msgr2=0x7f38240ae910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:05.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.864+0000 7f383181a640 1 --2- 172.21.15.67:0/978886319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38240ae500 0x7f38240ae910 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f381800d680 tx=0x7f381800db40 comp rx=0 tx=0).stop 2024-07-31T15:06:05.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.864+0000 7f383181a640 1 -- 172.21.15.67:0/978886319 shutdown_connections 2024-07-31T15:06:05.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.864+0000 7f383181a640 1 --2- 172.21.15.67:0/978886319 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f381c0b3f50 0x7f381c0b6460 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.864+0000 7f383181a640 1 --2- 172.21.15.67:0/978886319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38240ae500 0x7f38240ae910 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.864+0000 7f383181a640 1 --2- 172.21.15.67:0/978886319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f382409b2c0 0x7f38240a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.864+0000 7f383181a640 1 --2- 172.21.15.67:0/978886319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3824095030 0x7f38240a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:05.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.864+0000 7f383181a640 1 -- 172.21.15.67:0/978886319 >> 172.21.15.67:0/978886319 conn(0x7f382400cd80 msgr2=0x7f3824098ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:05.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.866+0000 7f383181a640 1 -- 172.21.15.67:0/978886319 shutdown_connections 2024-07-31T15:06:05.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:05.866+0000 7f383181a640 1 -- 172.21.15.67:0/978886319 wait complete. 2024-07-31T15:06:06.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:06 smithi067 ceph-mon[32750]: pgmap v874: 129 pgs: 12 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 81 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.2 MiB/s wr, 199 op/s 2024-07-31T15:06:06.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:06 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T15:06:06.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:06.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:06.593 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:06:06.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:06 smithi073 ceph-mon[36714]: pgmap v874: 129 pgs: 12 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 81 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.2 MiB/s wr, 199 op/s 2024-07-31T15:06:06.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:06 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T15:06:06.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:06.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:06 smithi077 ceph-mon[37327]: pgmap v874: 129 pgs: 12 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 81 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.2 MiB/s wr, 199 op/s 2024-07-31T15:06:06.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:06 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T15:06:06.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:06.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:06.934 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:06:08.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:07 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:08.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:07 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:08.175 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:08.176 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:07 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:08.176 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:07 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:08.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:08.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:07 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:08.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:07 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:08.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.926+0000 7f2824a58640 1 -- 172.21.15.67:0/2236667206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2820107610 msgr2=0x7f2820107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:08.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.926+0000 7f2824a58640 1 --2- 172.21.15.67:0/2236667206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2820107610 0x7f2820107af0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f2818009c10 tx=0x7f281802a6f0 comp rx=0 tx=0).stop 2024-07-31T15:06:08.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.929+0000 7f2824a58640 1 -- 172.21.15.67:0/2236667206 shutdown_connections 2024-07-31T15:06:08.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.929+0000 7f2824a58640 1 --2- 172.21.15.67:0/2236667206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2820107610 0x7f2820107af0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:08.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.929+0000 7f2824a58640 1 --2- 172.21.15.67:0/2236667206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2820102b20 0x7f2820107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:08.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.929+0000 7f2824a58640 1 --2- 172.21.15.67:0/2236667206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28201021d0 0x7f28201025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:08.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.929+0000 7f2824a58640 1 -- 172.21.15.67:0/2236667206 >> 172.21.15.67:0/2236667206 conn(0x7f2820079160 msgr2=0x7f2820079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:08.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.932+0000 7f2824a58640 1 -- 172.21.15.67:0/2236667206 shutdown_connections 2024-07-31T15:06:08.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.932+0000 7f2824a58640 1 -- 172.21.15.67:0/2236667206 wait complete. 2024-07-31T15:06:08.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.934+0000 7f2824a58640 1 Processor -- start 2024-07-31T15:06:08.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.934+0000 7f2824a58640 1 -- start start 2024-07-31T15:06:08.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.934+0000 7f2824a58640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28201021d0 0x7f28201a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:08.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.934+0000 7f2824a58640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2820102b20 0x7f28201a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:08.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.934+0000 7f2824a58640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2820107610 0x7f28201a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:08.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.934+0000 7f2824a58640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2820119ab0 con 0x7f28201021d0 2024-07-31T15:06:08.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.934+0000 7f2824a58640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2820119930 con 0x7f2820102b20 2024-07-31T15:06:08.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.934+0000 7f2824a58640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2820119c30 con 0x7f2820107610 2024-07-31T15:06:08.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.935+0000 7f281ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2820107610 0x7f28201a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:08.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.935+0000 7f281ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2820107610 0x7f28201a5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43642/0 (socket says 172.21.15.67:43642) 2024-07-31T15:06:08.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.935+0000 7f281ed9d640 1 -- 172.21.15.67:0/1053503065 learned_addr learned my addr 172.21.15.67:0/1053503065 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:08.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.935+0000 7f281dd9b640 1 --2- 172.21.15.67:0/1053503065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2820102b20 0x7f28201a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:08.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.935+0000 7f281e59c640 1 --2- 172.21.15.67:0/1053503065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28201021d0 0x7f28201a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:08.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.936+0000 7f281dd9b640 1 -- 172.21.15.67:0/1053503065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2820107610 msgr2=0x7f28201a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:08.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.936+0000 7f281dd9b640 1 --2- 172.21.15.67:0/1053503065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2820107610 0x7f28201a5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:08.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.936+0000 7f281dd9b640 1 -- 172.21.15.67:0/1053503065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28201021d0 msgr2=0x7f28201a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:08.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.936+0000 7f281dd9b640 1 --2- 172.21.15.67:0/1053503065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28201021d0 0x7f28201a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:08.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.936+0000 7f281dd9b640 1 -- 172.21.15.67:0/1053503065 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28201a6490 con 0x7f2820102b20 2024-07-31T15:06:08.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.936+0000 7f281dd9b640 1 --2- 172.21.15.67:0/1053503065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2820102b20 0x7f28201a0ac0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f282006a760 tx=0x7f281000db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:08.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.936+0000 7f280f7fe640 1 -- 172.21.15.67:0/1053503065 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810017070 con 0x7f2820102b20 2024-07-31T15:06:08.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.936+0000 7f280f7fe640 1 -- 172.21.15.67:0/1053503065 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2810009820 con 0x7f2820102b20 2024-07-31T15:06:08.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.936+0000 7f280f7fe640 1 -- 172.21.15.67:0/1053503065 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810004b20 con 0x7f2820102b20 2024-07-31T15:06:08.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.936+0000 7f2824a58640 1 -- 172.21.15.67:0/1053503065 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28201a6780 con 0x7f2820102b20 2024-07-31T15:06:08.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.937+0000 7f2824a58640 1 -- 172.21.15.67:0/1053503065 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28201acee0 con 0x7f2820102b20 2024-07-31T15:06:08.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.940+0000 7f280f7fe640 1 -- 172.21.15.67:0/1053503065 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2810004ce0 con 0x7f2820102b20 2024-07-31T15:06:08.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.941+0000 7f280f7fe640 1 --2- 172.21.15.67:0/1053503065 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f28080b3df0 0x7f28080b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:08.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.942+0000 7f280f7fe640 1 -- 172.21.15.67:0/1053503065 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f28100e48a0 con 0x7f2820102b20 2024-07-31T15:06:08.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.942+0000 7f281e59c640 1 --2- 172.21.15.67:0/1053503065 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f28080b3df0 0x7f28080b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:08.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.946+0000 7f281e59c640 1 --2- 172.21.15.67:0/1053503065 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f28080b3df0 0x7f28080b6300 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2814005fd0 tx=0x7f281400b590 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:08.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.946+0000 7f282afa9740 1 Processor -- start 2024-07-31T15:06:08.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.948+0000 7f282afa9740 1 -- start start 2024-07-31T15:06:08.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.948+0000 7f282afa9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc70a16070 0x55cc70dbb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:08.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.948+0000 7f282afa9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709ed1b0 0x55cc70dbba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:08.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.948+0000 7f282afa9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709f5390 0x55cc70dbbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:08.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.948+0000 7f282afa9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cc70d31b90 con 0x55cc70a16070 2024-07-31T15:06:08.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.948+0000 7f282afa9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cc70d21090 con 0x55cc709ed1b0 2024-07-31T15:06:08.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.948+0000 7f282afa9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cc70d250d0 con 0x55cc709f5390 2024-07-31T15:06:08.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.949+0000 7f281ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709f5390 0x55cc70dbbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:08.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.949+0000 7f281ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709f5390 0x55cc70dbbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43656/0 (socket says 172.21.15.67:43656) 2024-07-31T15:06:08.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.949+0000 7f281ed9d640 1 -- 172.21.15.67:0/1553170891 learned_addr learned my addr 172.21.15.67:0/1553170891 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:08.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.950+0000 7f281ed9d640 1 -- 172.21.15.67:0/1553170891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709ed1b0 msgr2=0x55cc70dbba90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:06:08.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.950+0000 7f281ed9d640 1 --2- 172.21.15.67:0/1553170891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709ed1b0 0x55cc70dbba90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:08.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.950+0000 7f281ed9d640 1 -- 172.21.15.67:0/1553170891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc70a16070 msgr2=0x55cc70dbb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:08.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.950+0000 7f281ed9d640 1 --2- 172.21.15.67:0/1553170891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc70a16070 0x55cc70dbb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:08.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.950+0000 7f281ed9d640 1 -- 172.21.15.67:0/1553170891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc70cc7b70 con 0x55cc709f5390 2024-07-31T15:06:08.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.950+0000 7f281ed9d640 1 --2- 172.21.15.67:0/1553170891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709f5390 0x55cc70dbbfd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2818030660 tx=0x7f2818030690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b985bd8690a767de server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:08.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.950+0000 7f280cff9640 1 -- 172.21.15.67:0/1553170891 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818040030 con 0x55cc709f5390 2024-07-31T15:06:08.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.951+0000 7f280cff9640 1 -- 172.21.15.67:0/1553170891 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2818003e20 con 0x55cc709f5390 2024-07-31T15:06:08.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.951+0000 7f280cff9640 1 -- 172.21.15.67:0/1553170891 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818036040 con 0x55cc709f5390 2024-07-31T15:06:08.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.952+0000 7f282afa9740 1 -- 172.21.15.67:0/1553170891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709f5390 msgr2=0x55cc70dbbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:08.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.952+0000 7f282afa9740 1 --2- 172.21.15.67:0/1553170891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709f5390 0x55cc70dbbfd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2818030660 tx=0x7f2818030690 comp rx=0 tx=0).stop 2024-07-31T15:06:08.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.954+0000 7f282afa9740 1 -- 172.21.15.67:0/1553170891 shutdown_connections 2024-07-31T15:06:08.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.954+0000 7f282afa9740 1 --2- 172.21.15.67:0/1553170891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709f5390 0x55cc70dbbfd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:08.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.954+0000 7f282afa9740 1 --2- 172.21.15.67:0/1553170891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709ed1b0 0x55cc70dbba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:08.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.954+0000 7f282afa9740 1 --2- 172.21.15.67:0/1553170891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc70a16070 0x55cc70dbb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:08.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.954+0000 7f282afa9740 1 -- 172.21.15.67:0/1553170891 >> 172.21.15.67:0/1553170891 conn(0x55cc70a0a460 msgr2=0x55cc70db9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:08.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.954+0000 7f282afa9740 1 -- 172.21.15.67:0/1553170891 shutdown_connections 2024-07-31T15:06:08.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.954+0000 7f282afa9740 1 -- 172.21.15.67:0/1553170891 wait complete. 2024-07-31T15:06:08.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.954+0000 7f282afa9740 1 -- start start 2024-07-31T15:06:08.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.955+0000 7f282afa9740 1 Processor -- start 2024-07-31T15:06:08.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.955+0000 7f282afa9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709ed1b0 0x55cc70dc71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:08.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.955+0000 7f282afa9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709f5390 0x55cc70dc7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:08.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.955+0000 7f282afa9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc70a16070 0x55cc70de4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:08.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.955+0000 7f282afa9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cc70b84700 con 0x55cc70a16070 2024-07-31T15:06:08.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.955+0000 7f282afa9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cc70cc39c0 con 0x55cc709f5390 2024-07-31T15:06:08.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.955+0000 7f282afa9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cc70ddace0 con 0x55cc709ed1b0 2024-07-31T15:06:08.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.955+0000 7f282afa9740 10 client.0 _ll_register_callbacks cb 0x55cc70cde4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:08.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.956+0000 7f281e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709ed1b0 0x55cc70dc71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:08.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.956+0000 7f281e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709ed1b0 0x55cc70dc71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43660/0 (socket says 172.21.15.67:43660) 2024-07-31T15:06:08.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.956+0000 7f281e59c640 1 -- 172.21.15.67:0/3736568343 learned_addr learned my addr 172.21.15.67:0/3736568343 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:08.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.956+0000 7f281dd9b640 1 --2- 172.21.15.67:0/3736568343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709f5390 0x55cc70dc7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:08.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.956+0000 7f281ed9d640 1 --2- 172.21.15.67:0/3736568343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc70a16070 0x55cc70de4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:08.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.956+0000 7f281e59c640 1 -- 172.21.15.67:0/3736568343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709f5390 msgr2=0x55cc70dc7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:08.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.956+0000 7f281e59c640 1 --2- 172.21.15.67:0/3736568343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709f5390 0x55cc70dc7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:08.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.956+0000 7f281e59c640 1 -- 172.21.15.67:0/3736568343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc70a16070 msgr2=0x55cc70de4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:08.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.957+0000 7f281e59c640 1 --2- 172.21.15.67:0/3736568343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc70a16070 0x55cc70de4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:08.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.957+0000 7f281e59c640 1 -- 172.21.15.67:0/3736568343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc70a81650 con 0x55cc709ed1b0 2024-07-31T15:06:08.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.957+0000 7f281ed9d640 1 --2- 172.21.15.67:0/3736568343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc70a16070 0x55cc70de4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:06:08.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.958+0000 7f2805ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:06:08.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.958+0000 7f281e59c640 1 --2- 172.21.15.67:0/3736568343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709ed1b0 0x55cc70dc71e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f281400ffb0 tx=0x7f2814011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4c6f01b0e61ff6d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:08.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.958+0000 7f2805ffb640 1 -- 172.21.15.67:0/3736568343 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814022070 con 0x55cc709ed1b0 2024-07-31T15:06:08.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.958+0000 7f282afa9740 1 -- 172.21.15.67:0/3736568343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2818003e20 con 0x55cc709ed1b0 2024-07-31T15:06:08.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.958+0000 7f2805ffb640 1 -- 172.21.15.67:0/3736568343 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f281401e660 con 0x55cc709ed1b0 2024-07-31T15:06:08.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.959+0000 7f2805ffb640 1 -- 172.21.15.67:0/3736568343 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f281402c050 con 0x55cc709ed1b0 2024-07-31T15:06:08.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.959+0000 7f282afa9740 10 client.28159 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:08.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.959+0000 7f282afa9740 1 -- 172.21.15.67:0/3736568343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55cc70de6ee0 con 0x55cc709ed1b0 2024-07-31T15:06:08.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.960+0000 7f2805ffb640 1 -- 172.21.15.67:0/3736568343 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2818003e20 con 0x55cc709ed1b0 2024-07-31T15:06:08.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.961+0000 7f2805ffb640 1 -- 172.21.15.67:0/3736568343 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f2814027930 con 0x55cc709ed1b0 2024-07-31T15:06:08.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.961+0000 7f282afa9740 10 client.28159 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:08.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.961+0000 7f282afa9740 10 client.28159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:08.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.961+0000 7f282afa9740 20 client.28159 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:08.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.961+0000 7f282afa9740 1 --2- 172.21.15.67:0/3736568343 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc709d49f0 0x55cc70dea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:08.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.961+0000 7f282afa9740 4 client.28159 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:06:08.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.961+0000 7f282afa9740 1 -- 172.21.15.67:0/3736568343 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc70d33ae0 con 0x55cc709d49f0 2024-07-31T15:06:08.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.962+0000 7f281dd9b640 1 --2- 172.21.15.67:0/3736568343 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc709d49f0 0x55cc70dea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:08.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.963+0000 7f2805ffb640 10 client.28159 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:08.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:08.964+0000 7f281dd9b640 1 --2- 172.21.15.67:0/3736568343 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc709d49f0 0x55cc70dea320 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6021ad9f059637d0 server_cookie=c515364ad2ed2bae in_seq=0 out_seq=0 2024-07-31T15:06:09.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:09.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:09.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:09.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:09.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: pgmap v875: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 732 KiB/s wr, 216 op/s 2024-07-31T15:06:09.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:09.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:06:09.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:09.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:09.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:09.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:09.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:09.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:09.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:09.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:09.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:09.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f2805ffb640 1 -- 172.21.15.67:0/3736568343 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc70d33ae0 con 0x55cc709d49f0 2024-07-31T15:06:09.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f2805ffb640 10 client.28159 handle_command_reply: tid=0 2024-07-31T15:06:09.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f282afa9740 1 client.28159 shutdown 2024-07-31T15:06:09.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f282afa9740 1 -- 172.21.15.67:0/3736568343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709ed1b0 msgr2=0x55cc70dc71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:09.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f282afa9740 1 --2- 172.21.15.67:0/3736568343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709ed1b0 0x55cc70dc71e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f281400ffb0 tx=0x7f2814011810 comp rx=0 tx=0).stop 2024-07-31T15:06:09.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f282afa9740 1 -- 172.21.15.67:0/3736568343 shutdown_connections 2024-07-31T15:06:09.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f282afa9740 1 --2- 172.21.15.67:0/3736568343 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc709d49f0 0x55cc70dea320 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f282afa9740 1 --2- 172.21.15.67:0/3736568343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc70a16070 0x55cc70de4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f2805ffb640 0 client.28159 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:09.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f282afa9740 1 --2- 172.21.15.67:0/3736568343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709f5390 0x55cc70dc7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f282afa9740 1 --2- 172.21.15.67:0/3736568343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709ed1b0 0x55cc70dc71e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f282afa9740 1 -- 172.21.15.67:0/3736568343 >> 172.21.15.67:0/3736568343 conn(0x55cc70a0a460 msgr2=0x55cc70dbe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:09.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f282afa9740 1 -- 172.21.15.67:0/3736568343 shutdown_connections 2024-07-31T15:06:09.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f282afa9740 1 -- 172.21.15.67:0/3736568343 wait complete. 2024-07-31T15:06:09.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.083+0000 7f282afa9740 20 client.28159 trim_cache size 0 max 16384 2024-07-31T15:06:09.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.103+0000 7f282afa9740 1 Processor -- start 2024-07-31T15:06:09.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.103+0000 7f282afa9740 1 -- start start 2024-07-31T15:06:09.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.103+0000 7f282afa9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709d49f0 0x55cc70de3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:09.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.103+0000 7f282afa9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709ed1b0 0x55cc70dc2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:09.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.103+0000 7f282afa9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc709f5390 0x55cc70db8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:09.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.103+0000 7f282afa9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2800010280 con 0x55cc709f5390 2024-07-31T15:06:09.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.103+0000 7f282afa9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2800011dc0 con 0x55cc709d49f0 2024-07-31T15:06:09.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.103+0000 7f282afa9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2800010950 con 0x55cc709ed1b0 2024-07-31T15:06:09.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.104+0000 7f281ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc709f5390 0x55cc70db8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:09.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.104+0000 7f281e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709d49f0 0x55cc70de3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:09.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.104+0000 7f281e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709d49f0 0x55cc70de3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:33866/0 (socket says 172.21.15.67:33866) 2024-07-31T15:06:09.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.104+0000 7f281e59c640 1 -- 172.21.15.67:0/3064543299 learned_addr learned my addr 172.21.15.67:0/3064543299 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:09.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.104+0000 7f281dd9b640 1 --2- 172.21.15.67:0/3064543299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709ed1b0 0x55cc70dc2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:09.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.104+0000 7f281ed9d640 1 -- 172.21.15.67:0/3064543299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709ed1b0 msgr2=0x55cc70dc2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:09.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.104+0000 7f281ed9d640 1 --2- 172.21.15.67:0/3064543299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709ed1b0 0x55cc70dc2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.104+0000 7f281ed9d640 1 -- 172.21.15.67:0/3064543299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709d49f0 msgr2=0x55cc70de3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:09.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.104+0000 7f281ed9d640 1 --2- 172.21.15.67:0/3064543299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709d49f0 0x55cc70de3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.104+0000 7f281ed9d640 1 -- 172.21.15.67:0/3064543299 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc70dbd840 con 0x55cc709f5390 2024-07-31T15:06:09.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.105+0000 7f281ed9d640 1 --2- 172.21.15.67:0/3064543299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc709f5390 0x55cc70db8e30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f2818030700 tx=0x7f2818000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c2360897ca4de0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:09.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.105+0000 7f2807fff640 1 -- 172.21.15.67:0/3064543299 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818049070 con 0x55cc709f5390 2024-07-31T15:06:09.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.105+0000 7f2807fff640 1 -- 172.21.15.67:0/3064543299 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2818030e20 con 0x55cc709f5390 2024-07-31T15:06:09.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.105+0000 7f2807fff640 1 -- 172.21.15.67:0/3064543299 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818036050 con 0x55cc709f5390 2024-07-31T15:06:09.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.106+0000 7f282afa9740 1 -- 172.21.15.67:0/3064543299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc709f5390 msgr2=0x55cc70db8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:09.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.106+0000 7f282afa9740 1 --2- 172.21.15.67:0/3064543299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc709f5390 0x55cc70db8e30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f2818030700 tx=0x7f2818000f40 comp rx=0 tx=0).stop 2024-07-31T15:06:09.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.106+0000 7f282afa9740 1 -- 172.21.15.67:0/3064543299 shutdown_connections 2024-07-31T15:06:09.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.106+0000 7f282afa9740 1 --2- 172.21.15.67:0/3064543299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc709f5390 0x55cc70db8e30 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.106+0000 7f282afa9740 1 --2- 172.21.15.67:0/3064543299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709ed1b0 0x55cc70dc2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.106+0000 7f282afa9740 1 --2- 172.21.15.67:0/3064543299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709d49f0 0x55cc70de3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.106+0000 7f282afa9740 1 -- 172.21.15.67:0/3064543299 >> 172.21.15.67:0/3064543299 conn(0x55cc70a0a460 msgr2=0x55cc70dc5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:09.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.106+0000 7f282afa9740 1 -- 172.21.15.67:0/3064543299 shutdown_connections 2024-07-31T15:06:09.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.107+0000 7f282afa9740 1 -- 172.21.15.67:0/3064543299 wait complete. 2024-07-31T15:06:09.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.107+0000 7f282afa9740 1 -- start start 2024-07-31T15:06:09.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.107+0000 7f282afa9740 1 Processor -- start 2024-07-31T15:06:09.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.107+0000 7f282afa9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc709d49f0 0x55cc70de0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:09.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.107+0000 7f282afa9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709ed1b0 0x55cc70def4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:09.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f282afa9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709f5390 0x55cc70def9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:09.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f282afa9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f28000116f0 con 0x55cc709d49f0 2024-07-31T15:06:09.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f282afa9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2800011020 con 0x55cc709ed1b0 2024-07-31T15:06:09.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f282afa9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f280000e200 con 0x55cc709f5390 2024-07-31T15:06:09.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f282afa9740 10 client.0 _ll_register_callbacks cb 0x7f2814015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:09.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f281dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709ed1b0 0x55cc70def4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:09.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f281dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709ed1b0 0x55cc70def4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:33874/0 (socket says 172.21.15.67:33874) 2024-07-31T15:06:09.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f281dd9b640 1 -- 172.21.15.67:0/3242413637 learned_addr learned my addr 172.21.15.67:0/3242413637 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:09.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f281e59c640 1 --2- 172.21.15.67:0/3242413637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc709d49f0 0x55cc70de0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:09.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f281ed9d640 1 --2- 172.21.15.67:0/3242413637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709f5390 0x55cc70def9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:09.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f281dd9b640 1 -- 172.21.15.67:0/3242413637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709f5390 msgr2=0x55cc70def9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:09.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f281dd9b640 1 --2- 172.21.15.67:0/3242413637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709f5390 0x55cc70def9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f281dd9b640 1 -- 172.21.15.67:0/3242413637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc709d49f0 msgr2=0x55cc70de0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:09.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f281dd9b640 1 --2- 172.21.15.67:0/3242413637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc709d49f0 0x55cc70de0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.108+0000 7f281dd9b640 1 -- 172.21.15.67:0/3242413637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc70df0160 con 0x55cc709ed1b0 2024-07-31T15:06:09.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.109+0000 7f281dd9b640 1 --2- 172.21.15.67:0/3242413637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709ed1b0 0x55cc70def4b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f28100e1ce0 tx=0x7f28100aa860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8a83d5ab80abc08 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:09.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.109+0000 7f2805ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:06:09.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.109+0000 7f2805ffb640 1 -- 172.21.15.67:0/3242413637 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28100c5060 con 0x55cc709ed1b0 2024-07-31T15:06:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.109+0000 7f2805ffb640 1 -- 172.21.15.67:0/3242413637 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f28100a7380 con 0x55cc709ed1b0 2024-07-31T15:06:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.109+0000 7f282afa9740 1 -- 172.21.15.67:0/3242413637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2818030e20 con 0x55cc709ed1b0 2024-07-31T15:06:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.109+0000 7f2805ffb640 1 -- 172.21.15.67:0/3242413637 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f2810020040 con 0x55cc709ed1b0 2024-07-31T15:06:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.109+0000 7f282afa9740 10 client.28244 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.109+0000 7f282afa9740 1 -- 172.21.15.67:0/3242413637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55cc70e52ab0 con 0x55cc709ed1b0 2024-07-31T15:06:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.109+0000 7f2805ffb640 1 -- 172.21.15.67:0/3242413637 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2818030e20 con 0x55cc709ed1b0 2024-07-31T15:06:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.109+0000 7f2805ffb640 1 -- 172.21.15.67:0/3242413637 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f2810023060 con 0x55cc709ed1b0 2024-07-31T15:06:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.109+0000 7f282afa9740 10 client.28244 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.109+0000 7f282afa9740 10 client.28244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.109+0000 7f282afa9740 20 client.28244 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.109+0000 7f282afa9740 1 --2- 172.21.15.67:0/3242413637 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc70a16070 0x55cc70e55140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.110+0000 7f282afa9740 4 client.28244 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:06:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.110+0000 7f282afa9740 1 -- 172.21.15.67:0/3242413637 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cc70d2f580 con 0x55cc70a16070 2024-07-31T15:06:09.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.110+0000 7f281e59c640 1 --2- 172.21.15.67:0/3242413637 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc70a16070 0x55cc70e55140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:09.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.110+0000 7f2805ffb640 10 client.28244 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:09.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.110+0000 7f281e59c640 1 --2- 172.21.15.67:0/3242413637 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc70a16070 0x55cc70e55140 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2860a6dd2153ff99 server_cookie=4737aa9412625a6e in_seq=0 out_seq=0 2024-07-31T15:06:09.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f2805ffb640 1 -- 172.21.15.67:0/3242413637 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55cc70d2f580 con 0x55cc70a16070 2024-07-31T15:06:09.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f2805ffb640 10 client.28244 handle_command_reply: tid=0 2024-07-31T15:06:09.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f282afa9740 1 client.28244 shutdown 2024-07-31T15:06:09.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f282afa9740 1 -- 172.21.15.67:0/3242413637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709ed1b0 msgr2=0x55cc70def4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f282afa9740 1 --2- 172.21.15.67:0/3242413637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709ed1b0 0x55cc70def4b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f28100e1ce0 tx=0x7f28100aa860 comp rx=0 tx=0).stop 2024-07-31T15:06:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f282afa9740 1 -- 172.21.15.67:0/3242413637 shutdown_connections 2024-07-31T15:06:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f282afa9740 1 --2- 172.21.15.67:0/3242413637 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55cc70a16070 0x55cc70e55140 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f282afa9740 1 --2- 172.21.15.67:0/3242413637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cc709f5390 0x55cc70def9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f282afa9740 1 --2- 172.21.15.67:0/3242413637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc709ed1b0 0x55cc70def4b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f282afa9740 1 --2- 172.21.15.67:0/3242413637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cc709d49f0 0x55cc70de0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f2805ffb640 0 client.28244 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f282afa9740 1 -- 172.21.15.67:0/3242413637 >> 172.21.15.67:0/3242413637 conn(0x55cc70a0a460 msgr2=0x55cc70dbe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f282afa9740 1 -- 172.21.15.67:0/3242413637 shutdown_connections 2024-07-31T15:06:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f282afa9740 1 -- 172.21.15.67:0/3242413637 wait complete. 2024-07-31T15:06:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.184+0000 7f282afa9740 20 client.28244 trim_cache size 0 max 16384 2024-07-31T15:06:09.188 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:06:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (3637 inodes in the stack)", 2024-07-31T15:06:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T15:06:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "6baaca53-fffa-4a41-8804-c4df7946cf59": { 2024-07-31T15:06:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T15:06:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "6baaca53-fffa-4a41-8804-c4df7946cf59", 2024-07-31T15:06:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T15:06:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:09.188 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:06:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.185+0000 7f2824a58640 1 -- 172.21.15.67:0/1053503065 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f28080b3df0 msgr2=0x7f28080b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.185+0000 7f2824a58640 1 --2- 172.21.15.67:0/1053503065 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f28080b3df0 0x7f28080b6300 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2814005fd0 tx=0x7f281400b590 comp rx=0 tx=0).stop 2024-07-31T15:06:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.185+0000 7f2824a58640 1 -- 172.21.15.67:0/1053503065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2820102b20 msgr2=0x7f28201a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.185+0000 7f2824a58640 1 --2- 172.21.15.67:0/1053503065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2820102b20 0x7f28201a0ac0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f282006a760 tx=0x7f281000db30 comp rx=0 tx=0).stop 2024-07-31T15:06:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.185+0000 7f2824a58640 1 -- 172.21.15.67:0/1053503065 shutdown_connections 2024-07-31T15:06:09.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.185+0000 7f2824a58640 1 --2- 172.21.15.67:0/1053503065 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f28080b3df0 0x7f28080b6300 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.185+0000 7f2824a58640 1 --2- 172.21.15.67:0/1053503065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2820107610 0x7f28201a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.185+0000 7f2824a58640 1 --2- 172.21.15.67:0/1053503065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2820102b20 0x7f28201a0ac0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.185+0000 7f2824a58640 1 --2- 172.21.15.67:0/1053503065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28201021d0 0x7f28201a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:09.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.185+0000 7f2824a58640 1 -- 172.21.15.67:0/1053503065 >> 172.21.15.67:0/1053503065 conn(0x7f2820079160 msgr2=0x7f2820105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:09.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.185+0000 7f2824a58640 1 -- 172.21.15.67:0/1053503065 shutdown_connections 2024-07-31T15:06:09.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:09.185+0000 7f2824a58640 1 -- 172.21.15.67:0/1053503065 wait complete. 2024-07-31T15:06:09.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:09.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:09.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:09.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:09.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: pgmap v875: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 732 KiB/s wr, 216 op/s 2024-07-31T15:06:09.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:09.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:06:09.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:09.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:09.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:09.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:09.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:09.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:09.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:09.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:09.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:09.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: pgmap v875: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 732 KiB/s wr, 216 op/s 2024-07-31T15:06:09.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:09.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:06:09.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:09.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:09.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:09.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:09.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:09.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:09.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:09.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:09.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:09.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:09.894 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:06:09 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:06:09.419+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d9b4 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb/ auth{3=1} v3386 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7) n(v4 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:276503 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@5} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce92cb2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v2020>,<0x10000000202/sound v9936>,<0x10000000201/linux-6.5.11 v830>,<0x10000000006/t v586>,<0x10000000005/tmp v553>,<0x100000001fb/client.0 v537>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v519>,<0x10000000001/sv_1 v525>,<0x10000000000/qa v522>,<0x1/volumes v503>]//[]","memoryvalue":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v3386>,<0x10000000202/sound v17237>,<0x10000000201/linux-6.5.11 v2040>,<0x10000000006/t v1164>,<0x10000000005/tmp v1110>,<0x100000001fb/client.0 v1084>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1066>,<0x10000000001/sv_1 v1066>,<0x10000000000/qa v1061>,<0x1/volumes v1021>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:46.317139+0000 53=46+7)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7)","memory_value.rstat":"n(v4 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:06:10.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:10.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:10.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi073 to 176.2M 2024-07-31T15:06:10.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: Unable to set osd_memory_target on smithi073 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-07-31T15:06:10.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:10.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:10.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:10.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:10.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:10.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:10.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:10.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi067 to 176.0M 2024-07-31T15:06:10.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: Unable to set osd_memory_target on smithi067 to 184601804: error parsing value: Value '184601804' is below minimum 939524096 2024-07-31T15:06:10.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:06:10.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:06:10.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:06:10.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:10.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:10.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:10.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:10.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi073 to 176.2M 2024-07-31T15:06:10.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: Unable to set osd_memory_target on smithi073 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-07-31T15:06:10.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:10.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:10.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:10.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:10.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:10.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:10.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:10.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi067 to 176.0M 2024-07-31T15:06:10.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: Unable to set osd_memory_target on smithi067 to 184601804: error parsing value: Value '184601804' is below minimum 939524096 2024-07-31T15:06:10.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:06:10.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:06:10.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:06:10.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:10.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:10.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:09 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi073 to 176.2M 2024-07-31T15:06:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: Unable to set osd_memory_target on smithi073 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-07-31T15:06:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:10.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:10.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:10.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:06:10.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi067 to 176.0M 2024-07-31T15:06:10.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: Unable to set osd_memory_target on smithi067 to 184601804: error parsing value: Value '184601804' is below minimum 939524096 2024-07-31T15:06:10.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:06:10.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:06:10.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:06:10.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:10.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:10.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:10 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:10.586 INFO:tasks.cephfs.filesystem:scrub status for tag:6baaca53-fffa-4a41-8804-c4df7946cf59 - {'path': '/', 'tag': '6baaca53-fffa-4a41-8804-c4df7946cf59', 'options': 'recursive,force'} 2024-07-31T15:06:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:06:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: pgmap v876: 129 pgs: 15 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 77 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 420 KiB/s wr, 195 op/s 2024-07-31T15:06:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:11.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:11.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:06:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: pgmap v876: 129 pgs: 15 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 77 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 420 KiB/s wr, 195 op/s 2024-07-31T15:06:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:11.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:11.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:11.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:11.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:11.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:11.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:11.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:11.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:11.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:06:11.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:11.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:11.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: pgmap v876: 129 pgs: 15 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 77 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 420 KiB/s wr, 195 op/s 2024-07-31T15:06:11.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:11.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:11.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:11.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:11.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:11.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:11.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:11.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:11.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:11.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:11.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:11.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:12.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:12.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:12.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:12.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:12.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:12.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:12.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.100 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:06:12 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:06:12.847+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000ce48 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity/ auth{1=1,2=1} v1695 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3) n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4) old_inodes=3 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:278221 nref=3)) caps={15489=pAsLsXs/-@10} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8aaac580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000ce48:[<0x1000000ce07/integrity v345>,<0x10000000202/security v8766>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000ce48:[<0x1000000ce07/integrity v1695>,<0x10000000202/security v15110>,<0x10000000201/linux-6.5.11 v2044>,<0x10000000006/t v1168>,<0x10000000005/tmp v1114>,<0x100000001fb/client.0 v1088>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1070>,<0x10000000001/sv_1 v1070>,<0x10000000000/qa v1065>,<0x1/volumes v1025>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:59.470106+0000 10=7+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","memory_value.dirstat":"f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3)","memory_value.rstat":"n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:06:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: pgmap v877: 129 pgs: 5 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 114 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 435 KiB/s wr, 207 op/s 2024-07-31T15:06:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:13.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:13.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: pgmap v877: 129 pgs: 5 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 114 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 435 KiB/s wr, 207 op/s 2024-07-31T15:06:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: pgmap v877: 129 pgs: 5 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 114 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 435 KiB/s wr, 207 op/s 2024-07-31T15:06:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.392 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:06:14.068+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000be32 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module/ auth{2=1} v2029 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0) n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:280893 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@8} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce826b7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000be32:[<0x1000000bd0a/module v1134>,<0x10000000202/kernel v5648>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000be32:[<0x1000000bd0a/module v2029>,<0x10000000202/kernel v17462>,<0x10000000201/linux-6.5.11 v2044>,<0x10000000006/t v1168>,<0x10000000005/tmp v1114>,<0x100000001fb/client.0 v1088>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1070>,<0x10000000001/sv_1 v1070>,<0x10000000000/qa v1065>,<0x1/volumes v1025>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:07.168188+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:06:14.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:06:14.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:14.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:06:14.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:14.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:06:14.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: pgmap v878: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 410 KiB/s wr, 243 op/s 2024-07-31T15:06:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:06:15.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:15.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: pgmap v878: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 410 KiB/s wr, 243 op/s 2024-07-31T15:06:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:06:15.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:15.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: pgmap v878: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 410 KiB/s wr, 243 op/s 2024-07-31T15:06:15.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:06:15.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:15.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.699+0000 7f8deffff640 1 -- 172.21.15.67:0/222581699 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f83bc1f1a0 con 0x55f83b89a570 2024-07-31T15:06:15.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.699+0000 7f8deffff640 10 client.18618 handle_command_reply: tid=0 2024-07-31T15:06:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.699+0000 7f8e19091740 1 client.18618 shutdown 2024-07-31T15:06:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.700+0000 7f8e19091740 1 -- 172.21.15.67:0/222581699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b906840 msgr2=0x55f83bcbe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.700+0000 7f8e19091740 1 --2- 172.21.15.67:0/222581699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b906840 0x55f83bcbe320 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f8e04031500 tx=0x7f8e04031980 comp rx=0 tx=0).stop 2024-07-31T15:06:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.700+0000 7f8e19091740 1 -- 172.21.15.67:0/222581699 shutdown_connections 2024-07-31T15:06:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.700+0000 7f8e19091740 1 --2- 172.21.15.67:0/222581699 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f83b89a570 0x55f83bd35450 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.700+0000 7f8e19091740 1 --2- 172.21.15.67:0/222581699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f83b906840 0x55f83bcbe320 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.700+0000 7f8e19091740 1 --2- 172.21.15.67:0/222581699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83b8e5b60 0x55f83bcbdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.700+0000 7f8e19091740 1 --2- 172.21.15.67:0/222581699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f83b8dd980 0x55f83bcc0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.700+0000 7f8e19091740 1 -- 172.21.15.67:0/222581699 >> 172.21.15.67:0/222581699 conn(0x55f83b8fa990 msgr2=0x55f83bcae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.700+0000 7f8deffff640 0 client.18618 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.700+0000 7f8e19091740 1 -- 172.21.15.67:0/222581699 shutdown_connections 2024-07-31T15:06:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.700+0000 7f8e19091740 1 -- 172.21.15.67:0/222581699 wait complete. 2024-07-31T15:06:15.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.700+0000 7f8e19091740 20 client.18618 trim_cache size 0 max 16384 2024-07-31T15:06:15.703 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:06:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-07-31T15:06:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:06:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 38, 2024-07-31T15:06:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:06:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "9986d895": { 2024-07-31T15:06:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 38, 2024-07-31T15:06:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:06:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:06:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:06:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:06:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:06:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:06:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 137.8, 2024-07-31T15:06:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:06:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:06:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:06:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:06:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:06:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:06:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:15.705 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:06:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.701+0000 7f8e118de640 1 -- 172.21.15.67:0/1661852150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8dfc0d5f50 msgr2=0x7f8dfc0d8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.701+0000 7f8e118de640 1 --2- 172.21.15.67:0/1661852150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8dfc0d5f50 0x7f8dfc0d8460 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x55f83bcc6e80 tx=0x7f8e0401f000 comp rx=0 tx=0).stop 2024-07-31T15:06:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.701+0000 7f8e118de640 1 -- 172.21.15.67:0/1661852150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e0c10a330 msgr2=0x7f8e0c11afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.701+0000 7f8e118de640 1 --2- 172.21.15.67:0/1661852150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e0c10a330 0x7f8e0c11afd0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000b450 tx=0x7f8e0000b910 comp rx=0 tx=0).stop 2024-07-31T15:06:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.701+0000 7f8e118de640 1 -- 172.21.15.67:0/1661852150 shutdown_connections 2024-07-31T15:06:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.701+0000 7f8e118de640 1 --2- 172.21.15.67:0/1661852150 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8dfc0d5f50 0x7f8dfc0d8460 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.701+0000 7f8e118de640 1 --2- 172.21.15.67:0/1661852150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e0c11c5d0 0x7f8e0c1149f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.701+0000 7f8e118de640 1 --2- 172.21.15.67:0/1661852150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e0c111730 0x7f8e0c1142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.701+0000 7f8e118de640 1 --2- 172.21.15.67:0/1661852150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e0c10a330 0x7f8e0c11afd0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.701+0000 7f8e118de640 1 -- 172.21.15.67:0/1661852150 >> 172.21.15.67:0/1661852150 conn(0x7f8e0c06e440 msgr2=0x7f8e0c071650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.701+0000 7f8e118de640 1 -- 172.21.15.67:0/1661852150 shutdown_connections 2024-07-31T15:06:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:15.701+0000 7f8e118de640 1 -- 172.21.15.67:0/1661852150 wait complete. 2024-07-31T15:06:15.881 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:06:15.531+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d187 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi/ auth{1=1} v3040 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0) n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1) old_inodes=1 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:288997 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@6} mcw={1=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7c6c1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d187:[<0x1000000d116/ctxfi v1858>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d187:[<0x1000000d116/ctxfi v3040>,<0x1000000cf1b/pci v3598>,<0x10000000202/sound v17470>,<0x10000000201/linux-6.5.11 v2046>,<0x10000000006/t v1170>,<0x10000000005/tmp v1116>,<0x100000001fb/client.0 v1090>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1072>,<0x10000000001/sv_1 v1072>,<0x10000000000/qa v1067>,<0x1/volumes v1027>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.452342+0000 30=30+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:06:15.881 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:06:15.535+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d119 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97/ auth{2=1} v3044 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0) n(v2 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1) old_inodes=2 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:289000 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@6} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce9047e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d119:[<0x1000000d116/ac97 v22>,<0x1000000cf1b/pci v463>,<0x10000000202/sound v8728>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000d119:[<0x1000000d116/ac97 v3044>,<0x1000000cf1b/pci v3598>,<0x10000000202/sound v17470>,<0x10000000201/linux-6.5.11 v2046>,<0x10000000006/t v1170>,<0x10000000005/tmp v1116>,<0x100000001fb/client.0 v1090>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1072>,<0x10000000001/sv_1 v1072>,<0x10000000000/qa v1067>,<0x1/volumes v1027>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:00.491085+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:06:15.881 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:06:15.537+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d1cc [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1/ auth{2=1} v3042 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0) n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:289007 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@6} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8f176c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v1862>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v3042>,<0x1000000cf1b/pci v3598>,<0x10000000202/sound v17470>,<0x10000000201/linux-6.5.11 v2046>,<0x10000000006/t v1170>,<0x10000000005/tmp v1116>,<0x100000001fb/client.0 v1090>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1072>,<0x10000000001/sv_1 v1072>,<0x10000000000/qa v1067>,<0x1/volumes v1027>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.549340+0000 22=22+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:06:15.881 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:06:15.554+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d180 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio/ auth{1=1} v3038 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0) n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1) old_inodes=1 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:289046 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@6} mcw={1=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8ebd2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v1860>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v3038>,<0x1000000cf1b/pci v3598>,<0x10000000202/sound v17470>,<0x10000000201/linux-6.5.11 v2046>,<0x10000000006/t v1170>,<0x10000000005/tmp v1116>,<0x100000001fb/client.0 v1090>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1072>,<0x10000000001/sv_1 v1072>,<0x10000000000/qa v1067>,<0x1/volumes v1027>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.414343+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:06:15.881 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:06:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:06:15.879+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000cde2 [...10,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders/ auth{4=1} v112 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0) n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:280767 nref=3)) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8dc52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v29>,<0x1000000cbec/selinux v646>,<0x10000000202/scripts v8820>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v112>,<0x1000000cbec/selinux v749>,<0x10000000202/scripts v11639>,<0x10000000201/linux-6.5.11 v2048>,<0x10000000006/t v1172>,<0x10000000005/tmp v1118>,<0x100000001fb/client.0 v1092>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1074>,<0x10000000001/sv_1 v1074>,<0x10000000000/qa v1069>,<0x1/volumes v1029>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:58.999116+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:06:16.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:16 smithi067 ceph-mon[32750]: osdmap e114: 12 total, 12 up, 12 in 2024-07-31T15:06:16.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.381+0000 7f6222c94640 1 -- 172.21.15.67:0/2195599804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f621c1023f0 msgr2=0x7f621c102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.381+0000 7f6222c94640 1 --2- 172.21.15.67:0/2195599804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f621c1023f0 0x7f621c102800 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6210009900 tx=0x7f621002a410 comp rx=0 tx=0).stop 2024-07-31T15:06:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.381+0000 7f6222c94640 1 -- 172.21.15.67:0/2195599804 shutdown_connections 2024-07-31T15:06:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.381+0000 7f6222c94640 1 --2- 172.21.15.67:0/2195599804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f621c108670 0x7f621c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.381+0000 7f6222c94640 1 --2- 172.21.15.67:0/2195599804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f621c107d20 0x7f621c108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.381+0000 7f6222c94640 1 --2- 172.21.15.67:0/2195599804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f621c1023f0 0x7f621c102800 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.381+0000 7f6222c94640 1 -- 172.21.15.67:0/2195599804 >> 172.21.15.67:0/2195599804 conn(0x7f621c0fddf0 msgr2=0x7f621c100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.381+0000 7f6222c94640 1 -- 172.21.15.67:0/2195599804 shutdown_connections 2024-07-31T15:06:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.381+0000 7f6222c94640 1 -- 172.21.15.67:0/2195599804 wait complete. 2024-07-31T15:06:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.382+0000 7f6222c94640 1 Processor -- start 2024-07-31T15:06:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.382+0000 7f6222c94640 1 -- start start 2024-07-31T15:06:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.382+0000 7f6222c94640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f621c1023f0 0x7f621c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.382+0000 7f6222c94640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f621c107d20 0x7f621c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.382+0000 7f6222c94640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f621c108670 0x7f621c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.382+0000 7f6222c94640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f621c111230 con 0x7f621c107d20 2024-07-31T15:06:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.382+0000 7f6222c94640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f621c1110b0 con 0x7f621c108670 2024-07-31T15:06:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.382+0000 7f6222c94640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f621c1113b0 con 0x7f621c1023f0 2024-07-31T15:06:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.382+0000 7f6220a30640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f621c1023f0 0x7f621c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.383+0000 7f6220a30640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f621c1023f0 0x7f621c197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44540/0 (socket says 172.21.15.67:44540) 2024-07-31T15:06:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.383+0000 7f6220a30640 1 -- 172.21.15.67:0/3772524437 learned_addr learned my addr 172.21.15.67:0/3772524437 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.383+0000 7f6221231640 1 --2- 172.21.15.67:0/3772524437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f621c108670 0x7f621c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.383+0000 7f621bfff640 1 --2- 172.21.15.67:0/3772524437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f621c107d20 0x7f621c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.383+0000 7f6220a30640 1 -- 172.21.15.67:0/3772524437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f621c108670 msgr2=0x7f621c19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.383+0000 7f6220a30640 1 --2- 172.21.15.67:0/3772524437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f621c108670 0x7f621c19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.383+0000 7f6220a30640 1 -- 172.21.15.67:0/3772524437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f621c107d20 msgr2=0x7f621c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.383+0000 7f6220a30640 1 --2- 172.21.15.67:0/3772524437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f621c107d20 0x7f621c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.383+0000 7f6220a30640 1 -- 172.21.15.67:0/3772524437 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621c19db60 con 0x7f621c1023f0 2024-07-31T15:06:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.383+0000 7f6220a30640 1 --2- 172.21.15.67:0/3772524437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f621c1023f0 0x7f621c197c30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6210005940 tx=0x7f62100040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.384+0000 7f6219ffb640 1 -- 172.21.15.67:0/3772524437 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621003f030 con 0x7f621c1023f0 2024-07-31T15:06:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.384+0000 7f6222c94640 1 -- 172.21.15.67:0/3772524437 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621c19ddf0 con 0x7f621c1023f0 2024-07-31T15:06:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.384+0000 7f6219ffb640 1 -- 172.21.15.67:0/3772524437 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62100044c0 con 0x7f621c1023f0 2024-07-31T15:06:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.384+0000 7f6219ffb640 1 -- 172.21.15.67:0/3772524437 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210035070 con 0x7f621c1023f0 2024-07-31T15:06:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.384+0000 7f6222c94640 1 -- 172.21.15.67:0/3772524437 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621c1a45e0 con 0x7f621c1023f0 2024-07-31T15:06:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.387+0000 7f6219ffb640 1 -- 172.21.15.67:0/3772524437 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6210042030 con 0x7f621c1023f0 2024-07-31T15:06:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.387+0000 7f6229b82740 1 Processor -- start 2024-07-31T15:06:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.387+0000 7f6229b82740 1 -- start start 2024-07-31T15:06:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.387+0000 7f6229b82740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f481b0 0x55bcb92eace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.387+0000 7f6229b82740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92eb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.387+0000 7f6229b82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f274d0 0x55bcb92eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.387+0000 7f6229b82740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bcb92603b0 con 0x55bcb8f481b0 2024-07-31T15:06:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.387+0000 7f6229b82740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bcb924f8b0 con 0x55bcb8f1f2f0 2024-07-31T15:06:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.387+0000 7f6229b82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bcb92538f0 con 0x55bcb8f274d0 2024-07-31T15:06:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f6220a30640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f6221231640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f481b0 0x55bcb92eace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f621bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f274d0 0x55bcb92eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f621bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f274d0 0x55bcb92eb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44550/0 (socket says 172.21.15.67:44550) 2024-07-31T15:06:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f6220a30640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92eb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:55176/0 (socket says 172.21.15.67:55176) 2024-07-31T15:06:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f621bfff640 1 -- 172.21.15.67:0/1882583032 learned_addr learned my addr 172.21.15.67:0/1882583032 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:16.391 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f621bfff640 1 -- 172.21.15.67:0/1882583032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f1f2f0 msgr2=0x55bcb92eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f621bfff640 1 --2- 172.21.15.67:0/1882583032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92eb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f621bfff640 1 -- 172.21.15.67:0/1882583032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f481b0 msgr2=0x55bcb92eace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f621bfff640 1 --2- 172.21.15.67:0/1882583032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f481b0 0x55bcb92eace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f621bfff640 1 -- 172.21.15.67:0/1882583032 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcb91f69a0 con 0x55bcb8f274d0 2024-07-31T15:06:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f6219ffb640 1 --2- 172.21.15.67:0/3772524437 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f620c0b3d40 0x7f620c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f6221231640 1 --2- 172.21.15.67:0/1882583032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f481b0 0x55bcb92eace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f621bfff640 1 --2- 172.21.15.67:0/1882583032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f274d0 0x55bcb92eb760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f620800bc70 tx=0x7f6208005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e6d62e5e53afb68 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f6219ffb640 1 -- 172.21.15.67:0/3772524437 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9395+0+0 (secure 0 0 0) 0x7f62101046e0 con 0x7f621c1023f0 2024-07-31T15:06:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f6221231640 1 --2- 172.21.15.67:0/3772524437 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f620c0b3d40 0x7f620c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.388+0000 7f6206ffd640 1 -- 172.21.15.67:0/1882583032 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208012ea0 con 0x55bcb8f274d0 2024-07-31T15:06:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.389+0000 7f6206ffd640 1 -- 172.21.15.67:0/1882583032 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f620800be20 con 0x55bcb8f274d0 2024-07-31T15:06:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.389+0000 7f6206ffd640 1 -- 172.21.15.67:0/1882583032 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208004c00 con 0x55bcb8f274d0 2024-07-31T15:06:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.390+0000 7f6229b82740 1 -- 172.21.15.67:0/1882583032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f274d0 msgr2=0x55bcb92eb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.390+0000 7f6229b82740 1 --2- 172.21.15.67:0/1882583032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f274d0 0x55bcb92eb760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f620800bc70 tx=0x7f6208005f90 comp rx=0 tx=0).stop 2024-07-31T15:06:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.390+0000 7f6221231640 1 --2- 172.21.15.67:0/3772524437 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f620c0b3d40 0x7f620c0b6250 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6214005fd0 tx=0x7f621400f040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.390+0000 7f6229b82740 1 -- 172.21.15.67:0/1882583032 shutdown_connections 2024-07-31T15:06:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.390+0000 7f6229b82740 1 --2- 172.21.15.67:0/1882583032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f274d0 0x55bcb92eb760 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.390+0000 7f6229b82740 1 --2- 172.21.15.67:0/1882583032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.390+0000 7f6229b82740 1 --2- 172.21.15.67:0/1882583032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f481b0 0x55bcb92eace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.390+0000 7f6229b82740 1 -- 172.21.15.67:0/1882583032 >> 172.21.15.67:0/1882583032 conn(0x55bcb8f3c300 msgr2=0x55bcb92e8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.390+0000 7f6229b82740 1 -- 172.21.15.67:0/1882583032 shutdown_connections 2024-07-31T15:06:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.390+0000 7f6229b82740 1 -- 172.21.15.67:0/1882583032 wait complete. 2024-07-31T15:06:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.390+0000 7f6229b82740 1 -- start start 2024-07-31T15:06:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.390+0000 7f6229b82740 1 Processor -- start 2024-07-31T15:06:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6229b82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6229b82740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6229b82740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f481b0 0x55bcb9302750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6229b82740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bcb9189ab0 con 0x55bcb8f274d0 2024-07-31T15:06:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6229b82740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bcb91d7cf0 con 0x55bcb8f481b0 2024-07-31T15:06:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6229b82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bcb92f9460 con 0x55bcb8f1f2f0 2024-07-31T15:06:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6229b82740 10 client.0 _ll_register_callbacks cb 0x55bcb92499f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6220a30640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f621bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6221231640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f481b0 0x55bcb9302750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6220a30640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92f67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44564/0 (socket says 172.21.15.67:44564) 2024-07-31T15:06:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f621bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55422/0 (socket says 172.21.15.67:55422) 2024-07-31T15:06:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6220a30640 1 -- 172.21.15.67:0/1782396382 learned_addr learned my addr 172.21.15.67:0/1782396382 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6220a30640 1 -- 172.21.15.67:0/1782396382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f481b0 msgr2=0x55bcb9302750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6220a30640 1 --2- 172.21.15.67:0/1782396382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f481b0 0x55bcb9302750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6220a30640 1 -- 172.21.15.67:0/1782396382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 msgr2=0x55bcb92f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6220a30640 1 --2- 172.21.15.67:0/1782396382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.391+0000 7f6220a30640 1 -- 172.21.15.67:0/1782396382 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcb8fb3790 con 0x55bcb8f1f2f0 2024-07-31T15:06:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f6220a30640 1 --2- 172.21.15.67:0/1782396382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92f67e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6210101b20 tx=0x7f62100092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5b5193b724b08c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f61fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:06:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f61fbfff640 1 -- 172.21.15.67:0/1782396382 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210049070 con 0x55bcb8f1f2f0 2024-07-31T15:06:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f61fbfff640 1 -- 172.21.15.67:0/1782396382 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9395+0+0 (secure 0 0 0) 0x7f6210104340 con 0x55bcb8f1f2f0 2024-07-31T15:06:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f6229b82740 1 -- 172.21.15.67:0/1782396382 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f620800be20 con 0x55bcb8f1f2f0 2024-07-31T15:06:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f61fbfff640 1 -- 172.21.15.67:0/1782396382 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f62100ca980 con 0x55bcb8f1f2f0 2024-07-31T15:06:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f6229b82740 10 client.28177 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f6229b82740 1 -- 172.21.15.67:0/1782396382 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55bcb902c620 con 0x55bcb8f1f2f0 2024-07-31T15:06:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f61fbfff640 1 -- 172.21.15.67:0/1782396382 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f620800be20 con 0x55bcb8f1f2f0 2024-07-31T15:06:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f61fbfff640 1 -- 172.21.15.67:0/1782396382 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f62100cb290 con 0x55bcb8f1f2f0 2024-07-31T15:06:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f6229b82740 10 client.28177 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f6229b82740 10 client.28177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f6229b82740 20 client.28177 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f6229b82740 1 --2- 172.21.15.67:0/1782396382 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bcb8f06d80 0x55bcb9308740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f6229b82740 4 client.28177 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:06:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.392+0000 7f6229b82740 1 -- 172.21.15.67:0/1782396382 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcb9262300 con 0x55bcb8f06d80 2024-07-31T15:06:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.393+0000 7f621bfff640 1 --2- 172.21.15.67:0/1782396382 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bcb8f06d80 0x55bcb9308740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.393+0000 7f61fbfff640 10 client.28177 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.393+0000 7f621bfff640 1 --2- 172.21.15.67:0/1782396382 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bcb8f06d80 0x55bcb9308740 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb42648fbaa9e0f6 server_cookie=13d993aa13750327 in_seq=0 out_seq=0 2024-07-31T15:06:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.394+0000 7f61fbfff640 1 -- 172.21.15.67:0/1782396382 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcb9262300 con 0x55bcb8f06d80 2024-07-31T15:06:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.394+0000 7f61fbfff640 10 client.28177 handle_command_reply: tid=0 2024-07-31T15:06:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.394+0000 7f6229b82740 1 client.28177 shutdown 2024-07-31T15:06:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.394+0000 7f6229b82740 1 -- 172.21.15.67:0/1782396382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f1f2f0 msgr2=0x55bcb92f67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.394+0000 7f6229b82740 1 --2- 172.21.15.67:0/1782396382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92f67e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6210101b20 tx=0x7f62100092f0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.394+0000 7f6229b82740 1 -- 172.21.15.67:0/1782396382 shutdown_connections 2024-07-31T15:06:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.394+0000 7f6229b82740 1 --2- 172.21.15.67:0/1782396382 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bcb8f06d80 0x55bcb9308740 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.394+0000 7f6229b82740 1 --2- 172.21.15.67:0/1782396382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f481b0 0x55bcb9302750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.394+0000 7f6229b82740 1 --2- 172.21.15.67:0/1782396382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.394+0000 7f6229b82740 1 --2- 172.21.15.67:0/1782396382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92f67e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.394+0000 7f61fbfff640 0 client.28177 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.394+0000 7f6229b82740 1 -- 172.21.15.67:0/1782396382 >> 172.21.15.67:0/1782396382 conn(0x55bcb8f3c300 msgr2=0x55bcb92e72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.395+0000 7f6229b82740 1 -- 172.21.15.67:0/1782396382 shutdown_connections 2024-07-31T15:06:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.395+0000 7f6229b82740 1 -- 172.21.15.67:0/1782396382 wait complete. 2024-07-31T15:06:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.395+0000 7f6229b82740 20 client.28177 trim_cache size 0 max 16384 2024-07-31T15:06:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.413+0000 7f6229b82740 1 Processor -- start 2024-07-31T15:06:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.413+0000 7f6229b82740 1 -- start start 2024-07-31T15:06:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.414+0000 7f6229b82740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92e8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.414+0000 7f6229b82740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.414+0000 7f6229b82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f481b0 0x55bcb92ec420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.414+0000 7f6229b82740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f621c1ae460 con 0x55bcb8f274d0 2024-07-31T15:06:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.414+0000 7f6229b82740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f621c1affa0 con 0x55bcb8f1f2f0 2024-07-31T15:06:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.414+0000 7f6229b82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f621c1aeb30 con 0x55bcb8f481b0 2024-07-31T15:06:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.414+0000 7f621bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.414+0000 7f621bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92f5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55436/0 (socket says 172.21.15.67:55436) 2024-07-31T15:06:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.414+0000 7f621bfff640 1 -- 172.21.15.67:0/2600891039 learned_addr learned my addr 172.21.15.67:0/2600891039 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.415+0000 7f6221231640 1 --2- 172.21.15.67:0/2600891039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f481b0 0x55bcb92ec420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.415+0000 7f6220a30640 1 --2- 172.21.15.67:0/2600891039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92e8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.415+0000 7f621bfff640 1 -- 172.21.15.67:0/2600891039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f481b0 msgr2=0x55bcb92ec420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.415+0000 7f621bfff640 1 --2- 172.21.15.67:0/2600891039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f481b0 0x55bcb92ec420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.415+0000 7f621bfff640 1 -- 172.21.15.67:0/2600891039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f1f2f0 msgr2=0x55bcb92e8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.415+0000 7f621bfff640 1 --2- 172.21.15.67:0/2600891039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92e8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.415+0000 7f621bfff640 1 -- 172.21.15.67:0/2600891039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcb92ed210 con 0x55bcb8f274d0 2024-07-31T15:06:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.415+0000 7f621bfff640 1 --2- 172.21.15.67:0/2600891039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92f5e40 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f620800bb60 tx=0x7f620800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b81718b308592b1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.416+0000 7f62067fc640 1 -- 172.21.15.67:0/2600891039 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208004590 con 0x55bcb8f274d0 2024-07-31T15:06:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.416+0000 7f62067fc640 1 -- 172.21.15.67:0/2600891039 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f620802b920 con 0x55bcb8f274d0 2024-07-31T15:06:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.416+0000 7f62067fc640 1 -- 172.21.15.67:0/2600891039 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208003a10 con 0x55bcb8f274d0 2024-07-31T15:06:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.417+0000 7f6229b82740 1 -- 172.21.15.67:0/2600891039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 msgr2=0x55bcb92f5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.417+0000 7f6229b82740 1 --2- 172.21.15.67:0/2600891039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92f5e40 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f620800bb60 tx=0x7f620800b790 comp rx=0 tx=0).stop 2024-07-31T15:06:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.417+0000 7f6229b82740 1 -- 172.21.15.67:0/2600891039 shutdown_connections 2024-07-31T15:06:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.417+0000 7f6229b82740 1 --2- 172.21.15.67:0/2600891039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f481b0 0x55bcb92ec420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.417+0000 7f6229b82740 1 --2- 172.21.15.67:0/2600891039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92f5e40 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.417+0000 7f6229b82740 1 --2- 172.21.15.67:0/2600891039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92e8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.417+0000 7f6229b82740 1 -- 172.21.15.67:0/2600891039 >> 172.21.15.67:0/2600891039 conn(0x55bcb8f3c300 msgr2=0x55bcb92ebd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.417+0000 7f6229b82740 1 -- 172.21.15.67:0/2600891039 shutdown_connections 2024-07-31T15:06:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.417+0000 7f6229b82740 1 -- 172.21.15.67:0/2600891039 wait complete. 2024-07-31T15:06:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.418+0000 7f6229b82740 1 -- start start 2024-07-31T15:06:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.418+0000 7f6229b82740 1 Processor -- start 2024-07-31T15:06:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.419+0000 7f6229b82740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92fb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.419+0000 7f6229b82740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.419+0000 7f6229b82740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f481b0 0x55bcb92fc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.419+0000 7f6229b82740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f621c1af8d0 con 0x55bcb8f274d0 2024-07-31T15:06:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.419+0000 7f6229b82740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f621c1af200 con 0x55bcb8f481b0 2024-07-31T15:06:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.419+0000 7f6229b82740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f621c1ac460 con 0x55bcb8f1f2f0 2024-07-31T15:06:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.419+0000 7f6229b82740 10 client.0 _ll_register_callbacks cb 0x7f6210005ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.420+0000 7f6221231640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f481b0 0x55bcb92fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.420+0000 7f6220a30640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.420+0000 7f6221231640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f481b0 0x55bcb92fc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:55204/0 (socket says 172.21.15.67:55204) 2024-07-31T15:06:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.420+0000 7f6221231640 1 -- 172.21.15.67:0/4260903481 learned_addr learned my addr 172.21.15.67:0/4260903481 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.420+0000 7f6220a30640 1 --2- 172.21.15.67:0/4260903481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92fb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44588/0 (socket says 172.21.15.67:44588) 2024-07-31T15:06:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.420+0000 7f6221231640 1 -- 172.21.15.67:0/4260903481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f1f2f0 msgr2=0x55bcb92fb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.420+0000 7f6221231640 1 --2- 172.21.15.67:0/4260903481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92fb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.420+0000 7f6221231640 1 -- 172.21.15.67:0/4260903481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 msgr2=0x55bcb92fbbc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:06:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.420+0000 7f621bfff640 1 --2- 172.21.15.67:0/4260903481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.421+0000 7f6221231640 1 --2- 172.21.15.67:0/4260903481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92fbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.421+0000 7f6221231640 1 -- 172.21.15.67:0/4260903481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcb9377720 con 0x55bcb8f481b0 2024-07-31T15:06:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.421+0000 7f621bfff640 1 --2- 172.21.15.67:0/4260903481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:06:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.422+0000 7f6221231640 1 --2- 172.21.15.67:0/4260903481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f481b0 0x55bcb92fc100 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6214004520 tx=0x7f6214004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85e6024767a95269 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.422+0000 7f61fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:06:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.422+0000 7f61fbfff640 1 -- 172.21.15.67:0/4260903481 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621402c070 con 0x55bcb8f481b0 2024-07-31T15:06:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.422+0000 7f6229b82740 1 -- 172.21.15.67:0/4260903481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f620802b920 con 0x55bcb8f481b0 2024-07-31T15:06:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.422+0000 7f61fbfff640 1 -- 172.21.15.67:0/4260903481 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9395+0+0 (secure 0 0 0) 0x7f6214028460 con 0x55bcb8f481b0 2024-07-31T15:06:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.423+0000 7f61fbfff640 1 -- 172.21.15.67:0/4260903481 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f6214038050 con 0x55bcb8f481b0 2024-07-31T15:06:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.423+0000 7f6229b82740 10 client.28262 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.423+0000 7f6229b82740 1 -- 172.21.15.67:0/4260903481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55bcb9378970 con 0x55bcb8f481b0 2024-07-31T15:06:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.423+0000 7f61fbfff640 1 -- 172.21.15.67:0/4260903481 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f620802b920 con 0x55bcb8f481b0 2024-07-31T15:06:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.423+0000 7f61fbfff640 1 -- 172.21.15.67:0/4260903481 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f6214019530 con 0x55bcb8f481b0 2024-07-31T15:06:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.423+0000 7f6229b82740 10 client.28262 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.423+0000 7f6229b82740 10 client.28262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.423+0000 7f6229b82740 20 client.28262 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.423+0000 7f6229b82740 1 --2- 172.21.15.67:0/4260903481 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bcb8edc1c0 0x55bcb937b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.423+0000 7f6229b82740 4 client.28262 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:06:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.424+0000 7f6229b82740 1 -- 172.21.15.67:0/4260903481 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bcb925dda0 con 0x55bcb8edc1c0 2024-07-31T15:06:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.424+0000 7f6220a30640 1 --2- 172.21.15.67:0/4260903481 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bcb8edc1c0 0x55bcb937b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:16.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.426+0000 7f6220a30640 1 --2- 172.21.15.67:0/4260903481 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bcb8edc1c0 0x55bcb937b190 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19c73e606883c57d server_cookie=5085e422ca733450 in_seq=0 out_seq=0 2024-07-31T15:06:16.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.426+0000 7f61fbfff640 10 client.28262 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:16.455 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9986d895', quiesce took 31.3 seconds. Will release after: -13.5 2024-07-31T15:06:16.456 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:06:16.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:16 smithi077 ceph-mon[37327]: osdmap e114: 12 total, 12 up, 12 in 2024-07-31T15:06:16.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:16.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:16.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:16.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:16.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:16.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:16.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.598+0000 7f61fbfff640 1 -- 172.21.15.67:0/4260903481 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bcb925dda0 con 0x55bcb8edc1c0 2024-07-31T15:06:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.598+0000 7f61fbfff640 10 client.28262 handle_command_reply: tid=0 2024-07-31T15:06:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.598+0000 7f6229b82740 1 client.28262 shutdown 2024-07-31T15:06:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.599+0000 7f6229b82740 1 -- 172.21.15.67:0/4260903481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f481b0 msgr2=0x55bcb92fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.599+0000 7f6229b82740 1 --2- 172.21.15.67:0/4260903481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f481b0 0x55bcb92fc100 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6214004520 tx=0x7f6214004ab0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.599+0000 7f6229b82740 1 -- 172.21.15.67:0/4260903481 shutdown_connections 2024-07-31T15:06:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.599+0000 7f6229b82740 1 --2- 172.21.15.67:0/4260903481 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bcb8edc1c0 0x55bcb937b190 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.599+0000 7f6229b82740 1 --2- 172.21.15.67:0/4260903481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bcb8f481b0 0x55bcb92fc100 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.599+0000 7f6229b82740 1 --2- 172.21.15.67:0/4260903481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcb8f274d0 0x55bcb92fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.599+0000 7f6229b82740 1 --2- 172.21.15.67:0/4260903481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcb8f1f2f0 0x55bcb92fb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.599+0000 7f6229b82740 1 -- 172.21.15.67:0/4260903481 >> 172.21.15.67:0/4260903481 conn(0x55bcb8f3c300 msgr2=0x55bcb92ec580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.599+0000 7f61fbfff640 0 client.28262 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.599+0000 7f6229b82740 1 -- 172.21.15.67:0/4260903481 shutdown_connections 2024-07-31T15:06:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.599+0000 7f6229b82740 1 -- 172.21.15.67:0/4260903481 wait complete. 2024-07-31T15:06:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.599+0000 7f6229b82740 20 client.28262 trim_cache size 0 max 16384 2024-07-31T15:06:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:06:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:06:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:06:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:06:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.600+0000 7f6222c94640 1 -- 172.21.15.67:0/3772524437 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f620c0b3d40 msgr2=0x7f620c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.600+0000 7f6222c94640 1 --2- 172.21.15.67:0/3772524437 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f620c0b3d40 0x7f620c0b6250 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6214005fd0 tx=0x7f621400f040 comp rx=0 tx=0).stop 2024-07-31T15:06:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.600+0000 7f6222c94640 1 -- 172.21.15.67:0/3772524437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f621c1023f0 msgr2=0x7f621c197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.600+0000 7f6222c94640 1 --2- 172.21.15.67:0/3772524437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f621c1023f0 0x7f621c197c30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6210005940 tx=0x7f62100040c0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.601+0000 7f6222c94640 1 -- 172.21.15.67:0/3772524437 shutdown_connections 2024-07-31T15:06:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.601+0000 7f6222c94640 1 --2- 172.21.15.67:0/3772524437 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f620c0b3d40 0x7f620c0b6250 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.601+0000 7f6222c94640 1 --2- 172.21.15.67:0/3772524437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f621c108670 0x7f621c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.601+0000 7f6222c94640 1 --2- 172.21.15.67:0/3772524437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f621c107d20 0x7f621c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.601+0000 7f6222c94640 1 --2- 172.21.15.67:0/3772524437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f621c1023f0 0x7f621c197c30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.601+0000 7f6222c94640 1 -- 172.21.15.67:0/3772524437 >> 172.21.15.67:0/3772524437 conn(0x7f621c0fddf0 msgr2=0x7f621c100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.601+0000 7f6222c94640 1 -- 172.21.15.67:0/3772524437 shutdown_connections 2024-07-31T15:06:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:16.601+0000 7f6222c94640 1 -- 172.21.15.67:0/3772524437 wait complete. 2024-07-31T15:06:16.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:16 smithi073 ceph-mon[36714]: osdmap e114: 12 total, 12 up, 12 in 2024-07-31T15:06:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:16.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:16.862 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:06:17.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: pgmap v880: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 309 KiB/s wr, 404 op/s 2024-07-31T15:06:17.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.270 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:17.270 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: pgmap v880: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 309 KiB/s wr, 404 op/s 2024-07-31T15:06:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:17.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: pgmap v880: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 309 KiB/s wr, 404 op/s 2024-07-31T15:06:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:06:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:18.310 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:18.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:18.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:18.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.709+0000 7f465f9ca640 1 -- 172.21.15.67:0/2378935766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4658109840 msgr2=0x7f4658110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:19.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.709+0000 7f465f9ca640 1 --2- 172.21.15.67:0/2378935766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4658109840 0x7f4658110070 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f4650009ca0 tx=0x7f465002a8c0 comp rx=0 tx=0).stop 2024-07-31T15:06:19.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.709+0000 7f465f9ca640 1 -- 172.21.15.67:0/2378935766 shutdown_connections 2024-07-31T15:06:19.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.709+0000 7f465f9ca640 1 --2- 172.21.15.67:0/2378935766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4658109840 0x7f4658110070 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:19.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.709+0000 7f465f9ca640 1 --2- 172.21.15.67:0/2378935766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4658108ef0 0x7f4658109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:19.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.709+0000 7f465f9ca640 1 --2- 172.21.15.67:0/2378935766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46581035b0 0x7f46581039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:19.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.709+0000 7f465f9ca640 1 -- 172.21.15.67:0/2378935766 >> 172.21.15.67:0/2378935766 conn(0x7f46580ff000 msgr2=0x7f4658101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:19.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.709+0000 7f465f9ca640 1 -- 172.21.15.67:0/2378935766 shutdown_connections 2024-07-31T15:06:19.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.709+0000 7f465f9ca640 1 -- 172.21.15.67:0/2378935766 wait complete. 2024-07-31T15:06:19.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.709+0000 7f465f9ca640 1 Processor -- start 2024-07-31T15:06:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.709+0000 7f465f9ca640 1 -- start start 2024-07-31T15:06:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465f9ca640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46581035b0 0x7f4658199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465f9ca640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4658108ef0 0x7f465819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465f9ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4658109840 0x7f465819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465f9ca640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f46581134f0 con 0x7f46581035b0 2024-07-31T15:06:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465f9ca640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4658113370 con 0x7f4658108ef0 2024-07-31T15:06:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465f9ca640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4658113670 con 0x7f4658109840 2024-07-31T15:06:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465d766640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46581035b0 0x7f4658199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465d766640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46581035b0 0x7f4658199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55452/0 (socket says 172.21.15.67:55452) 2024-07-31T15:06:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465cf65640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4658108ef0 0x7f465819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465d766640 1 -- 172.21.15.67:0/88721323 learned_addr learned my addr 172.21.15.67:0/88721323 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:19.105 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:06:19.105 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"btime":"2024-07-31T15:06:04:323642+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:06:04.323492+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28214":{"gid":28214,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1516494761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1516494761},{"type":"v1","addr":"172.21.15.73:6835","nonce":1516494761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:06:19.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465df67640 1 --2- 172.21.15.67:0/88721323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4658109840 0x7f465819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:19.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465cf65640 1 -- 172.21.15.67:0/88721323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4658109840 msgr2=0x7f465819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:19.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465cf65640 1 --2- 172.21.15.67:0/88721323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4658109840 0x7f465819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:19.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465cf65640 1 -- 172.21.15.67:0/88721323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46581035b0 msgr2=0x7f4658199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:19.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465cf65640 1 --2- 172.21.15.67:0/88721323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46581035b0 0x7f4658199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465cf65640 1 -- 172.21.15.67:0/88721323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465819fdf0 con 0x7f4658108ef0 2024-07-31T15:06:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.710+0000 7f465d766640 1 --2- 172.21.15.67:0/88721323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46581035b0 0x7f4658199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.711+0000 7f465cf65640 1 --2- 172.21.15.67:0/88721323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4658108ef0 0x7f465819a420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f465806c440 tx=0x7f464800db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.711+0000 7f464e7fc640 1 -- 172.21.15.67:0/88721323 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648017070 con 0x7f4658108ef0 2024-07-31T15:06:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.711+0000 7f465f9ca640 1 -- 172.21.15.67:0/88721323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46581a00e0 con 0x7f4658108ef0 2024-07-31T15:06:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.711+0000 7f464e7fc640 1 -- 172.21.15.67:0/88721323 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4648009820 con 0x7f4658108ef0 2024-07-31T15:06:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.711+0000 7f464e7fc640 1 -- 172.21.15.67:0/88721323 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648004ba0 con 0x7f4658108ef0 2024-07-31T15:06:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.711+0000 7f465f9ca640 1 -- 172.21.15.67:0/88721323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46581a6840 con 0x7f4658108ef0 2024-07-31T15:06:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.712+0000 7f465f9ca640 1 -- 172.21.15.67:0/88721323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46581039c0 con 0x7f4658108ef0 2024-07-31T15:06:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.713+0000 7f464e7fc640 1 -- 172.21.15.67:0/88721323 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4648009c60 con 0x7f4658108ef0 2024-07-31T15:06:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.714+0000 7f464e7fc640 1 --2- 172.21.15.67:0/88721323 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46440b3d40 0x7f46440b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.714+0000 7f465d766640 1 --2- 172.21.15.67:0/88721323 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46440b3d40 0x7f46440b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:19.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.714+0000 7f464e7fc640 1 -- 172.21.15.67:0/88721323 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9395+0+0 (secure 0 0 0) 0x7f46480e3fa0 con 0x7f4658108ef0 2024-07-31T15:06:19.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.715+0000 7f465d766640 1 --2- 172.21.15.67:0/88721323 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46440b3d40 0x7f46440b6250 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4654005fd0 tx=0x7f465400b480 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:19.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.717+0000 7f464e7fc640 1 -- 172.21.15.67:0/88721323 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f46480099d0 con 0x7f4658108ef0 2024-07-31T15:06:19.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.997+0000 7f465f9ca640 1 -- 172.21.15.67:0/88721323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f465819afb0 con 0x7f4658108ef0 2024-07-31T15:06:19.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:18.999+0000 7f464e7fc640 1 -- 172.21.15.67:0/88721323 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11962 (secure 0 0 0) 0x7f46480aa8f0 con 0x7f4658108ef0 2024-07-31T15:06:19.107 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-07-31T15:06:19.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:19.001+0000 7f465f9ca640 1 -- 172.21.15.67:0/88721323 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46440b3d40 msgr2=0x7f46440b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:19.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:19.001+0000 7f465f9ca640 1 --2- 172.21.15.67:0/88721323 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46440b3d40 0x7f46440b6250 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4654005fd0 tx=0x7f465400b480 comp rx=0 tx=0).stop 2024-07-31T15:06:19.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:19.001+0000 7f465f9ca640 1 -- 172.21.15.67:0/88721323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4658108ef0 msgr2=0x7f465819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:19.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:19.001+0000 7f465f9ca640 1 --2- 172.21.15.67:0/88721323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4658108ef0 0x7f465819a420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f465806c440 tx=0x7f464800db30 comp rx=0 tx=0).stop 2024-07-31T15:06:19.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:19.001+0000 7f465f9ca640 1 -- 172.21.15.67:0/88721323 shutdown_connections 2024-07-31T15:06:19.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:19.001+0000 7f465f9ca640 1 --2- 172.21.15.67:0/88721323 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46440b3d40 0x7f46440b6250 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:19.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:19.001+0000 7f465f9ca640 1 --2- 172.21.15.67:0/88721323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4658109840 0x7f465819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:19.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:19.001+0000 7f465f9ca640 1 --2- 172.21.15.67:0/88721323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4658108ef0 0x7f465819a420 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:19.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:19.001+0000 7f465f9ca640 1 --2- 172.21.15.67:0/88721323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46581035b0 0x7f4658199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:19.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:19.001+0000 7f465f9ca640 1 -- 172.21.15.67:0/88721323 >> 172.21.15.67:0/88721323 conn(0x7f46580ff000 msgr2=0x7f4658101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:19.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:19.001+0000 7f465f9ca640 1 -- 172.21.15.67:0/88721323 shutdown_connections 2024-07-31T15:06:19.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:19.001+0000 7f465f9ca640 1 -- 172.21.15.67:0/88721323 wait complete. 2024-07-31T15:06:19.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: pgmap v881: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 388 KiB/s wr, 970 op/s 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:19.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:19 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/88721323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:06:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: pgmap v881: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 388 KiB/s wr, 970 op/s 2024-07-31T15:06:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:19.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:19 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/88721323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:06:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: pgmap v881: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 388 KiB/s wr, 970 op/s 2024-07-31T15:06:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:19.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:19.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:19 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/88721323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:06:20.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:20.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:20 smithi077 ceph-mon[37327]: osdmap e115: 12 total, 12 up, 12 in 2024-07-31T15:06:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:20 smithi067 ceph-mon[32750]: osdmap e115: 12 total, 12 up, 12 in 2024-07-31T15:06:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:20 smithi073 ceph-mon[36714]: osdmap e115: 12 total, 12 up, 12 in 2024-07-31T15:06:20.649 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 9986d895 --release --await' 2024-07-31T15:06:20.650 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 9986d895 --release --await 2024-07-31T15:06:20.953 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:06:21.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: pgmap v882: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 442 KiB/s wr, 1.75k op/s 2024-07-31T15:06:21.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.386 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.386 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.386 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:21.386 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: pgmap v882: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 442 KiB/s wr, 1.75k op/s 2024-07-31T15:06:21.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:21.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: pgmap v882: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 442 KiB/s wr, 1.75k op/s 2024-07-31T15:06:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:22.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:22.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:22.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: pgmap v884: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 526 KiB/s wr, 3.70k op/s 2024-07-31T15:06:23.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.431 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.432 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.433 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.433 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.433 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.433 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.433 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:06:23.433 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: pgmap v884: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 526 KiB/s wr, 3.70k op/s 2024-07-31T15:06:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:06:23.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: pgmap v884: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 526 KiB/s wr, 3.70k op/s 2024-07-31T15:06:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:06:23.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:23.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.808+0000 7ff338839640 1 -- 172.21.15.67:0/3647145324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff334108730 msgr2=0x7ff33410ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:23.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.808+0000 7ff338839640 1 --2- 172.21.15.67:0/3647145324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff334108730 0x7ff33410ef60 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009ca0 tx=0x7ff32c02a8c0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.808+0000 7ff338839640 1 -- 172.21.15.67:0/3647145324 shutdown_connections 2024-07-31T15:06:23.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.808+0000 7ff338839640 1 --2- 172.21.15.67:0/3647145324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff334108730 0x7ff33410ef60 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.808+0000 7ff338839640 1 --2- 172.21.15.67:0/3647145324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff334107de0 0x7ff3341081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.808+0000 7ff338839640 1 --2- 172.21.15.67:0/3647145324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3341024a0 0x7ff3341028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.808+0000 7ff338839640 1 -- 172.21.15.67:0/3647145324 >> 172.21.15.67:0/3647145324 conn(0x7ff334079140 msgr2=0x7ff334079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.808+0000 7ff338839640 1 -- 172.21.15.67:0/3647145324 shutdown_connections 2024-07-31T15:06:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.808+0000 7ff338839640 1 -- 172.21.15.67:0/3647145324 wait complete. 2024-07-31T15:06:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.809+0000 7ff338839640 1 Processor -- start 2024-07-31T15:06:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.809+0000 7ff338839640 1 -- start start 2024-07-31T15:06:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.809+0000 7ff338839640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3341024a0 0x7ff334199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.809+0000 7ff338839640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff334107de0 0x7ff33419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.809+0000 7ff338839640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff334108730 0x7ff33419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.809+0000 7ff338839640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff334112400 con 0x7ff3341024a0 2024-07-31T15:06:23.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.809+0000 7ff338839640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff334112280 con 0x7ff334108730 2024-07-31T15:06:23.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.809+0000 7ff338839640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff334112580 con 0x7ff334107de0 2024-07-31T15:06:23.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.809+0000 7ff33259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3341024a0 0x7ff334199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.809+0000 7ff33259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3341024a0 0x7ff334199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42646/0 (socket says 172.21.15.67:42646) 2024-07-31T15:06:23.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.809+0000 7ff33259c640 1 -- 172.21.15.67:0/2885985231 learned_addr learned my addr 172.21.15.67:0/2885985231 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:23.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.809+0000 7ff332d9d640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff334108730 0x7ff33419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.809+0000 7ff331d9b640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff334107de0 0x7ff33419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.810+0000 7ff33259c640 1 -- 172.21.15.67:0/2885985231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff334107de0 msgr2=0x7ff33419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:23.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.810+0000 7ff33259c640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff334107de0 0x7ff33419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.810+0000 7ff33259c640 1 -- 172.21.15.67:0/2885985231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff334108730 msgr2=0x7ff33419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:23.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.810+0000 7ff33259c640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff334108730 0x7ff33419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.810+0000 7ff33259c640 1 -- 172.21.15.67:0/2885985231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33419fdd0 con 0x7ff3341024a0 2024-07-31T15:06:23.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.810+0000 7ff332d9d640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff334108730 0x7ff33419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:06:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.810+0000 7ff331d9b640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff334107de0 0x7ff33419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:06:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.810+0000 7ff33259c640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3341024a0 0x7ff334199e60 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7ff32400e9f0 tx=0x7ff32400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.810+0000 7ff3237fe640 1 -- 172.21.15.67:0/2885985231 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32400cd80 con 0x7ff3341024a0 2024-07-31T15:06:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.810+0000 7ff3237fe640 1 -- 172.21.15.67:0/2885985231 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3240054d0 con 0x7ff3341024a0 2024-07-31T15:06:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.810+0000 7ff338839640 1 -- 172.21.15.67:0/2885985231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3341a00c0 con 0x7ff3341024a0 2024-07-31T15:06:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.810+0000 7ff3237fe640 1 -- 172.21.15.67:0/2885985231 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324010690 con 0x7ff3341024a0 2024-07-31T15:06:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.810+0000 7ff338839640 1 -- 172.21.15.67:0/2885985231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3341a6820 con 0x7ff3341024a0 2024-07-31T15:06:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.813+0000 7ff33ed8a740 1 Processor -- start 2024-07-31T15:06:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.813+0000 7ff33ed8a740 1 -- start start 2024-07-31T15:06:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.813+0000 7ff33ed8a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfecd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.813+0000 7ff33ed8a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfafe660 0x5645cfecd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.813+0000 7ff33ed8a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb06840 0x5645cfecdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.813+0000 7ff3237fe640 1 -- 172.21.15.67:0/2885985231 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff324004e90 con 0x7ff3341024a0 2024-07-31T15:06:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.813+0000 7ff33ed8a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5645cfe42e70 con 0x5645cfb27520 2024-07-31T15:06:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.813+0000 7ff33ed8a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5645cfe32370 con 0x5645cfafe660 2024-07-31T15:06:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.813+0000 7ff33ed8a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5645cfe363b0 con 0x5645cfb06840 2024-07-31T15:06:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.813+0000 7ff332d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfecd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.813+0000 7ff332d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfecd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42654/0 (socket says 172.21.15.67:42654) 2024-07-31T15:06:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.813+0000 7ff332d9d640 1 -- 172.21.15.67:0/3879473503 learned_addr learned my addr 172.21.15.67:0/3879473503 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.813+0000 7ff331d9b640 1 --2- 172.21.15.67:0/3879473503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb06840 0x5645cfecdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.813+0000 7ff33259c640 1 --2- 172.21.15.67:0/3879473503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfafe660 0x5645cfecd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.814+0000 7ff332d9d640 1 -- 172.21.15.67:0/3879473503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb06840 msgr2=0x5645cfecdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.814+0000 7ff332d9d640 1 --2- 172.21.15.67:0/3879473503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb06840 0x5645cfecdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.814+0000 7ff332d9d640 1 -- 172.21.15.67:0/3879473503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfafe660 msgr2=0x5645cfecd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.814+0000 7ff332d9d640 1 --2- 172.21.15.67:0/3879473503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfafe660 0x5645cfecd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.814+0000 7ff332d9d640 1 -- 172.21.15.67:0/3879473503 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645cfdff040 con 0x5645cfb27520 2024-07-31T15:06:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.814+0000 7ff332d9d640 1 --2- 172.21.15.67:0/3879473503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfecd070 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0094a0 tx=0x7ff32c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fd8995dab1ece7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.814+0000 7ff320ff9640 1 -- 172.21.15.67:0/3879473503 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c031db0 con 0x5645cfb27520 2024-07-31T15:06:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.814+0000 7ff320ff9640 1 -- 172.21.15.67:0/3879473503 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff32c040070 con 0x5645cfb27520 2024-07-31T15:06:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.814+0000 7ff3237fe640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff31c0b3d40 0x7ff31c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.814+0000 7ff320ff9640 1 -- 172.21.15.67:0/3879473503 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c004240 con 0x5645cfb27520 2024-07-31T15:06:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.814+0000 7ff33259c640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff31c0b3d40 0x7ff31c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.815+0000 7ff3237fe640 1 -- 172.21.15.67:0/2885985231 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7ff3240e4880 con 0x7ff3341024a0 2024-07-31T15:06:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.815+0000 7ff33ed8a740 1 -- 172.21.15.67:0/3879473503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 msgr2=0x5645cfecd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.815+0000 7ff33259c640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff31c0b3d40 0x7ff31c0b6250 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff3240e1d40 tx=0x7ff324004a90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.815+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/3879473503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfecd070 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0094a0 tx=0x7ff32c009800 comp rx=0 tx=0).stop 2024-07-31T15:06:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.815+0000 7ff33ed8a740 1 -- 172.21.15.67:0/3879473503 shutdown_connections 2024-07-31T15:06:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.815+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/3879473503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb06840 0x5645cfecdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.815+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/3879473503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfafe660 0x5645cfecd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.815+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/3879473503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfecd070 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.815+0000 7ff33ed8a740 1 -- 172.21.15.67:0/3879473503 >> 172.21.15.67:0/3879473503 conn(0x5645cfb1b910 msgr2=0x5645cfecaf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:23.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.815+0000 7ff33ed8a740 1 -- 172.21.15.67:0/3879473503 shutdown_connections 2024-07-31T15:06:23.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.815+0000 7ff33ed8a740 1 -- 172.21.15.67:0/3879473503 wait complete. 2024-07-31T15:06:23.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.816+0000 7ff33ed8a740 1 -- start start 2024-07-31T15:06:23.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.816+0000 7ff33ed8a740 1 Processor -- start 2024-07-31T15:06:23.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.816+0000 7ff33ed8a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfafe660 0x5645cfedd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.816+0000 7ff33ed8a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfb06840 0x5645cfedd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.816+0000 7ff33ed8a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfee8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.816+0000 7ff33ed8a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5645cfc95bb0 con 0x5645cfb27520 2024-07-31T15:06:23.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.816+0000 7ff33ed8a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5645cfdd4ac0 con 0x5645cfb06840 2024-07-31T15:06:23.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.816+0000 7ff33ed8a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5645cfedfb20 con 0x5645cfafe660 2024-07-31T15:06:23.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.816+0000 7ff33ed8a740 10 client.0 _ll_register_callbacks cb 0x5645cfddd3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.816+0000 7ff332d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfee8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff332d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfee8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42658/0 (socket says 172.21.15.67:42658) 2024-07-31T15:06:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff332d9d640 1 -- 172.21.15.67:0/4063408318 learned_addr learned my addr 172.21.15.67:0/4063408318 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff331d9b640 1 --2- 172.21.15.67:0/4063408318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfb06840 0x5645cfedd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff332d9d640 1 --2- 172.21.15.67:0/4063408318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfafe660 0x5645cfedd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff331d9b640 1 -- 172.21.15.67:0/4063408318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfafe660 msgr2=0x5645cfedd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff331d9b640 1 --2- 172.21.15.67:0/4063408318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfafe660 0x5645cfedd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff331d9b640 1 -- 172.21.15.67:0/4063408318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 msgr2=0x5645cfee8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff331d9b640 1 --2- 172.21.15.67:0/4063408318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfee8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff331d9b640 1 -- 172.21.15.67:0/4063408318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645cfb92b00 con 0x5645cfb06840 2024-07-31T15:06:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff332d9d640 1 --2- 172.21.15.67:0/4063408318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfee8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff331d9b640 1 --2- 172.21.15.67:0/4063408318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfb06840 0x5645cfedd600 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff33406b0f0 tx=0x7ff328012b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b94fe58373b3a37 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff319ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:06:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff319ffb640 1 -- 172.21.15.67:0/4063408318 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328004320 con 0x5645cfb06840 2024-07-31T15:06:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff33ed8a740 1 -- 172.21.15.67:0/4063408318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff32c040070 con 0x5645cfb06840 2024-07-31T15:06:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.817+0000 7ff319ffb640 1 -- 172.21.15.67:0/4063408318 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7ff328011970 con 0x5645cfb06840 2024-07-31T15:06:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.818+0000 7ff319ffb640 1 -- 172.21.15.67:0/4063408318 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7ff3280044c0 con 0x5645cfb06840 2024-07-31T15:06:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.818+0000 7ff33ed8a740 10 client.28280 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.818+0000 7ff33ed8a740 1 -- 172.21.15.67:0/4063408318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5645cfc09960 con 0x5645cfb06840 2024-07-31T15:06:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.818+0000 7ff319ffb640 1 -- 172.21.15.67:0/4063408318 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff32c040070 con 0x5645cfb06840 2024-07-31T15:06:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.818+0000 7ff319ffb640 1 -- 172.21.15.67:0/4063408318 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7ff32800f440 con 0x5645cfb06840 2024-07-31T15:06:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.818+0000 7ff33ed8a740 10 client.28280 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.818+0000 7ff33ed8a740 10 client.28280 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:06:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.818+0000 7ff33ed8a740 20 client.28280 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.818+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/4063408318 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5645cfae5ea0 0x5645cfeeefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.818+0000 7ff33ed8a740 4 client.28280 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:06:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.818+0000 7ff33ed8a740 1 -- 172.21.15.67:0/4063408318 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645cfe44dc0 con 0x5645cfae5ea0 2024-07-31T15:06:23.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.819+0000 7ff332d9d640 1 --2- 172.21.15.67:0/4063408318 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5645cfae5ea0 0x5645cfeeefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.819+0000 7ff332d9d640 1 --2- 172.21.15.67:0/4063408318 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5645cfae5ea0 0x5645cfeeefa0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b53df68e1916b625 server_cookie=440fb5ae3683dab5 in_seq=0 out_seq=0 2024-07-31T15:06:23.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.819+0000 7ff319ffb640 10 client.28280 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:23.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff319ffb640 1 -- 172.21.15.67:0/4063408318 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645cfe44dc0 con 0x5645cfae5ea0 2024-07-31T15:06:23.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff319ffb640 10 client.28280 handle_command_reply: tid=0 2024-07-31T15:06:23.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff33ed8a740 1 client.28280 shutdown 2024-07-31T15:06:23.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff33ed8a740 1 -- 172.21.15.67:0/4063408318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfb06840 msgr2=0x5645cfedd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:23.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/4063408318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfb06840 0x5645cfedd600 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff33406b0f0 tx=0x7ff328012b10 comp rx=0 tx=0).stop 2024-07-31T15:06:23.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff33ed8a740 1 -- 172.21.15.67:0/4063408318 shutdown_connections 2024-07-31T15:06:23.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/4063408318 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5645cfae5ea0 0x5645cfeeefa0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/4063408318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfee8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/4063408318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfb06840 0x5645cfedd600 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/4063408318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfafe660 0x5645cfedd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff319ffb640 0 client.28280 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:23.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff33ed8a740 1 -- 172.21.15.67:0/4063408318 >> 172.21.15.67:0/4063408318 conn(0x5645cfb1b910 msgr2=0x5645cfed0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:23.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff33ed8a740 1 -- 172.21.15.67:0/4063408318 shutdown_connections 2024-07-31T15:06:23.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff33ed8a740 1 -- 172.21.15.67:0/4063408318 wait complete. 2024-07-31T15:06:23.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.820+0000 7ff33ed8a740 20 client.28280 trim_cache size 0 max 16384 2024-07-31T15:06:23.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.838+0000 7ff33ed8a740 1 Processor -- start 2024-07-31T15:06:23.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.838+0000 7ff33ed8a740 1 -- start start 2024-07-31T15:06:23.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.838+0000 7ff33ed8a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfafe660 0x5645cfed8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.838+0000 7ff33ed8a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb06840 0x5645cfee8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.838+0000 7ff33ed8a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfece7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.838+0000 7ff33ed8a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff3341ae5a0 con 0x5645cfb27520 2024-07-31T15:06:23.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.838+0000 7ff33ed8a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff3341b00e0 con 0x5645cfafe660 2024-07-31T15:06:23.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.838+0000 7ff33ed8a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff3341aec70 con 0x5645cfb06840 2024-07-31T15:06:23.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.839+0000 7ff332d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfece7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.839+0000 7ff332d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfece7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42668/0 (socket says 172.21.15.67:42668) 2024-07-31T15:06:23.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.839+0000 7ff332d9d640 1 -- 172.21.15.67:0/2288255646 learned_addr learned my addr 172.21.15.67:0/2288255646 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:23.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.839+0000 7ff332d9d640 1 --2- 172.21.15.67:0/2288255646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfafe660 0x5645cfed8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.839+0000 7ff331d9b640 1 --2- 172.21.15.67:0/2288255646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb06840 0x5645cfee8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.839+0000 7ff332d9d640 1 -- 172.21.15.67:0/2288255646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb06840 msgr2=0x5645cfee8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:23.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.839+0000 7ff332d9d640 1 --2- 172.21.15.67:0/2288255646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb06840 0x5645cfee8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.839+0000 7ff332d9d640 1 -- 172.21.15.67:0/2288255646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfafe660 msgr2=0x5645cfed8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:23.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.839+0000 7ff332d9d640 1 --2- 172.21.15.67:0/2288255646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfafe660 0x5645cfed8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.839+0000 7ff332d9d640 1 -- 172.21.15.67:0/2288255646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645cfecf690 con 0x5645cfb27520 2024-07-31T15:06:23.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.839+0000 7ff332d9d640 1 --2- 172.21.15.67:0/2288255646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfece7b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff32c000f80 tx=0x7ff32c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96de354d51847733 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:23.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.839+0000 7ff31bfff640 1 -- 172.21.15.67:0/2288255646 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c003d40 con 0x5645cfb27520 2024-07-31T15:06:23.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.839+0000 7ff31bfff640 1 -- 172.21.15.67:0/2288255646 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff32c059040 con 0x5645cfb27520 2024-07-31T15:06:23.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.839+0000 7ff31bfff640 1 -- 172.21.15.67:0/2288255646 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c02f660 con 0x5645cfb27520 2024-07-31T15:06:23.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.840+0000 7ff33ed8a740 1 -- 172.21.15.67:0/2288255646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 msgr2=0x5645cfece7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:23.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.840+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/2288255646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfece7b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff32c000f80 tx=0x7ff32c009a20 comp rx=0 tx=0).stop 2024-07-31T15:06:23.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.840+0000 7ff33ed8a740 1 -- 172.21.15.67:0/2288255646 shutdown_connections 2024-07-31T15:06:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.840+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/2288255646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfb27520 0x5645cfece7b0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.840+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/2288255646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb06840 0x5645cfee8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.840+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/2288255646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfafe660 0x5645cfed8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.840+0000 7ff33ed8a740 1 -- 172.21.15.67:0/2288255646 >> 172.21.15.67:0/2288255646 conn(0x5645cfb1b910 msgr2=0x5645cfece0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.840+0000 7ff33ed8a740 1 -- 172.21.15.67:0/2288255646 shutdown_connections 2024-07-31T15:06:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.840+0000 7ff33ed8a740 1 -- 172.21.15.67:0/2288255646 wait complete. 2024-07-31T15:06:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.841+0000 7ff33ed8a740 1 -- start start 2024-07-31T15:06:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.841+0000 7ff33ed8a740 1 Processor -- start 2024-07-31T15:06:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.841+0000 7ff33ed8a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfafe660 0x5645cfef4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.841+0000 7ff33ed8a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfb06840 0x5645cfef4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.841+0000 7ff33ed8a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb27520 0x5645cfef4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.841+0000 7ff33ed8a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff3341afa10 con 0x5645cfafe660 2024-07-31T15:06:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.841+0000 7ff33ed8a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff3341af340 con 0x5645cfb06840 2024-07-31T15:06:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.841+0000 7ff33ed8a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff3341ac5a0 con 0x5645cfb27520 2024-07-31T15:06:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.841+0000 7ff33ed8a740 10 client.0 _ll_register_callbacks cb 0x7ff328009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.841+0000 7ff332d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfafe660 0x5645cfef4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.841+0000 7ff332d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfafe660 0x5645cfef4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42680/0 (socket says 172.21.15.67:42680) 2024-07-31T15:06:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.841+0000 7ff332d9d640 1 -- 172.21.15.67:0/797138716 learned_addr learned my addr 172.21.15.67:0/797138716 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.841+0000 7ff332d9d640 1 --2- 172.21.15.67:0/797138716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb27520 0x5645cfef4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.841+0000 7ff331d9b640 1 --2- 172.21.15.67:0/797138716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfb06840 0x5645cfef4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff332d9d640 1 -- 172.21.15.67:0/797138716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb27520 msgr2=0x5645cfef4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff332d9d640 1 --2- 172.21.15.67:0/797138716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb27520 0x5645cfef4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff332d9d640 1 -- 172.21.15.67:0/797138716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfb06840 msgr2=0x5645cfef4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff332d9d640 1 --2- 172.21.15.67:0/797138716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfb06840 0x5645cfef4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff332d9d640 1 -- 172.21.15.67:0/797138716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645cfee4570 con 0x5645cfafe660 2024-07-31T15:06:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff331d9b640 1 --2- 172.21.15.67:0/797138716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfb06840 0x5645cfef4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:06:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff332d9d640 1 --2- 172.21.15.67:0/797138716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfafe660 0x5645cfef4510 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009ca0 tx=0x7ff32c003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70f91bc8216f2aa server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff319ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:06:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff319ffb640 1 -- 172.21.15.67:0/797138716 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c004480 con 0x5645cfafe660 2024-07-31T15:06:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff319ffb640 1 -- 172.21.15.67:0/797138716 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7ff32c031450 con 0x5645cfafe660 2024-07-31T15:06:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff33ed8a740 1 -- 172.21.15.67:0/797138716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff32c059040 con 0x5645cfafe660 2024-07-31T15:06:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff319ffb640 1 -- 172.21.15.67:0/797138716 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7ff32c031650 con 0x5645cfafe660 2024-07-31T15:06:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff33ed8a740 10 client.18747 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff33ed8a740 1 -- 172.21.15.67:0/797138716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5645cff57a60 con 0x5645cfafe660 2024-07-31T15:06:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.842+0000 7ff319ffb640 1 -- 172.21.15.67:0/797138716 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff32c059040 con 0x5645cfafe660 2024-07-31T15:06:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.843+0000 7ff319ffb640 1 -- 172.21.15.67:0/797138716 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7ff32c05c020 con 0x5645cfafe660 2024-07-31T15:06:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.843+0000 7ff33ed8a740 10 client.18747 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:23.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.843+0000 7ff33ed8a740 10 client.18747 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:06:23.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.843+0000 7ff33ed8a740 20 client.18747 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:23.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.843+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/797138716 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5645cfabb250 0x5645cff59ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:23.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.843+0000 7ff33ed8a740 4 client.18747 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9986d895", "release": true, "await": true}] 2024-07-31T15:06:23.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.843+0000 7ff33ed8a740 1 -- 172.21.15.67:0/797138716 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9986d895", "release": true, "await": true}) -- 0x5645cfe40860 con 0x5645cfabb250 2024-07-31T15:06:23.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.843+0000 7ff331d9b640 1 --2- 172.21.15.67:0/797138716 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5645cfabb250 0x5645cff59ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:23.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.843+0000 7ff319ffb640 10 client.18747 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:23.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:23.843+0000 7ff331d9b640 1 --2- 172.21.15.67:0/797138716 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5645cfabb250 0x5645cff59ce0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74b01adf58a9e30b server_cookie=13a0b4d1e8fc107d in_seq=0 out_seq=0 2024-07-31T15:06:24.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:24.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:24.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:06:25.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:25 smithi067 ceph-mon[32750]: pgmap v885: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 488 KiB/s wr, 5.19k op/s 2024-07-31T15:06:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:25 smithi077 ceph-mon[37327]: pgmap v885: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 488 KiB/s wr, 5.19k op/s 2024-07-31T15:06:25.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:25 smithi073 ceph-mon[36714]: pgmap v885: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 488 KiB/s wr, 5.19k op/s 2024-07-31T15:06:26.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.168+0000 7ff319ffb640 1 -- 172.21.15.67:0/797138716 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5645cfe40860 con 0x5645cfabb250 2024-07-31T15:06:26.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.168+0000 7ff319ffb640 10 client.18747 handle_command_reply: tid=0 2024-07-31T15:06:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.168+0000 7ff33ed8a740 1 client.18747 shutdown 2024-07-31T15:06:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.168+0000 7ff33ed8a740 1 -- 172.21.15.67:0/797138716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfafe660 msgr2=0x5645cfef4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.168+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/797138716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfafe660 0x5645cfef4510 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009ca0 tx=0x7ff32c003bf0 comp rx=0 tx=0).stop 2024-07-31T15:06:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.168+0000 7ff33ed8a740 1 -- 172.21.15.67:0/797138716 shutdown_connections 2024-07-31T15:06:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.169+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/797138716 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5645cfabb250 0x5645cff59ce0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.169+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/797138716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645cfb27520 0x5645cfef4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.169+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/797138716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645cfb06840 0x5645cfef4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.169+0000 7ff33ed8a740 1 --2- 172.21.15.67:0/797138716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645cfafe660 0x5645cfef4510 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.169+0000 7ff319ffb640 0 client.18747 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.169+0000 7ff33ed8a740 1 -- 172.21.15.67:0/797138716 >> 172.21.15.67:0/797138716 conn(0x5645cfb1b910 msgr2=0x5645cfed00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.169+0000 7ff33ed8a740 1 -- 172.21.15.67:0/797138716 shutdown_connections 2024-07-31T15:06:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.169+0000 7ff33ed8a740 1 -- 172.21.15.67:0/797138716 wait complete. 2024-07-31T15:06:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.169+0000 7ff33ed8a740 20 client.18747 trim_cache size 0 max 16384 2024-07-31T15:06:26.173 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:06:26.173 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-07-31T15:06:26.173 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:06:26.173 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 40, 2024-07-31T15:06:26.173 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:06:26.173 INFO:teuthology.orchestra.run.smithi067.stdout: "9986d895": { 2024-07-31T15:06:26.173 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 40, 2024-07-31T15:06:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:06:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:06:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:06:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:06:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:06:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:06:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 137.8, 2024-07-31T15:06:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:06:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:06:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:06:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:06:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:06:26.175 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:06:26.175 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:26.175 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:26.175 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:26.175 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:26.175 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:26.175 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:06:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.170+0000 7ff338839640 1 -- 172.21.15.67:0/2885985231 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff31c0b3d40 msgr2=0x7ff31c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.170+0000 7ff338839640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff31c0b3d40 0x7ff31c0b6250 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff3240e1d40 tx=0x7ff324004a90 comp rx=0 tx=0).stop 2024-07-31T15:06:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.170+0000 7ff338839640 1 -- 172.21.15.67:0/2885985231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3341024a0 msgr2=0x7ff334199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.170+0000 7ff338839640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3341024a0 0x7ff334199e60 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7ff32400e9f0 tx=0x7ff32400edf0 comp rx=0 tx=0).stop 2024-07-31T15:06:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.170+0000 7ff338839640 1 -- 172.21.15.67:0/2885985231 shutdown_connections 2024-07-31T15:06:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.170+0000 7ff338839640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff31c0b3d40 0x7ff31c0b6250 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.170+0000 7ff338839640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff334108730 0x7ff33419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.170+0000 7ff338839640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff334107de0 0x7ff33419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.170+0000 7ff338839640 1 --2- 172.21.15.67:0/2885985231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3341024a0 0x7ff334199e60 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.170+0000 7ff338839640 1 -- 172.21.15.67:0/2885985231 >> 172.21.15.67:0/2885985231 conn(0x7ff334079140 msgr2=0x7ff3341013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.170+0000 7ff338839640 1 -- 172.21.15.67:0/2885985231 shutdown_connections 2024-07-31T15:06:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:26.170+0000 7ff338839640 1 -- 172.21.15.67:0/2885985231 wait complete. 2024-07-31T15:06:26.247 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cred.o 2024-07-31T15:06:26.250 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nls/nls_utf8.o 2024-07-31T15:06:26.251 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4proc.o 2024-07-31T15:06:26.264 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/hdac_sysfs.o 2024-07-31T15:06:26.276 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/x86/built-in.a 2024-07-31T15:06:26.287 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4xdr.o 2024-07-31T15:06:26.303 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/xarray.o 2024-07-31T15:06:26.312 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exprep.o 2024-07-31T15:06:26.411 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/netevent.o 2024-07-31T15:06:26.736 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/nls/built-in.a 2024-07-31T15:06:26.744 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exregion.o 2024-07-31T15:06:26.786 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/neighbour.o 2024-07-31T15:06:27.012 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9986d895', total seconds elapsed: 41.8 2024-07-31T15:06:27.013 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.8 2024-07-31T15:06:27.013 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:06:27.143 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/hdac_regmap.o 2024-07-31T15:06:27.191 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exresnte.o 2024-07-31T15:06:27.204 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/reboot.o 2024-07-31T15:06:27.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:27 smithi067 ceph-mon[32750]: pgmap v886: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 430 KiB/s wr, 5.10k op/s 2024-07-31T15:06:27.370 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exresolv.o 2024-07-31T15:06:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:27 smithi077 ceph-mon[37327]: pgmap v886: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 430 KiB/s wr, 5.10k op/s 2024-07-31T15:06:27.494 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/lockref.o 2024-07-31T15:06:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:27 smithi073 ceph-mon[36714]: pgmap v886: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 430 KiB/s wr, 5.10k op/s 2024-07-31T15:06:27.819 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/xen/built-in.a 2024-07-31T15:06:27.867 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/hdac_controller.o 2024-07-31T15:06:27.894 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/virtio/built-in.a 2024-07-31T15:06:27.898 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:06:27.926 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/sound_core.o 2024-07-31T15:06:27.940 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/bcd.o 2024-07-31T15:06:28.063 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exresop.o 2024-07-31T15:06:28.070 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/sort.o 2024-07-31T15:06:28.265 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/parser.o 2024-07-31T15:06:28.460 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exserial.o 2024-07-31T15:06:28.562 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/async.o 2024-07-31T15:06:28.628 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/range.o 2024-07-31T15:06:28.717 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/hdac_stream.o 2024-07-31T15:06:28.727 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/debug_locks.o 2024-07-31T15:06:28.769 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/smpboot.o 2024-07-31T15:06:28.901 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/random32.o 2024-07-31T15:06:28.917 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exstore.o 2024-07-31T15:06:29.037 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/ucount.o 2024-07-31T15:06:29.196 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4state.o 2024-07-31T15:06:29.280 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/bust_spinlocks.o 2024-07-31T15:06:29.299 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/regset.o 2024-07-31T15:06:29.302 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exstoren.o 2024-07-31T15:06:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:29 smithi077 ceph-mon[37327]: pgmap v887: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 130 KiB/s wr, 4.61k op/s 2024-07-31T15:06:29.515 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/ksyms_common.o 2024-07-31T15:06:29.542 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/array.o 2024-07-31T15:06:29.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:29 smithi067 ceph-mon[32750]: pgmap v887: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 130 KiB/s wr, 4.61k op/s 2024-07-31T15:06:29.559 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/rtnetlink.o 2024-07-31T15:06:29.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:29 smithi073 ceph-mon[36714]: pgmap v887: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 130 KiB/s wr, 4.61k op/s 2024-07-31T15:06:29.715 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exstorob.o 2024-07-31T15:06:29.751 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/utils.o 2024-07-31T15:06:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.831+0000 7fa371f04640 1 -- 172.21.15.67:0/3061154120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa36c10c940 msgr2=0x7fa36c07b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.831+0000 7fa371f04640 1 --2- 172.21.15.67:0/3061154120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa36c10c940 0x7fa36c07b0c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa3600097d0 tx=0x7fa36002e940 comp rx=0 tx=0).stop 2024-07-31T15:06:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.832+0000 7fa371f04640 1 -- 172.21.15.67:0/3061154120 shutdown_connections 2024-07-31T15:06:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.832+0000 7fa371f04640 1 --2- 172.21.15.67:0/3061154120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa36c10c940 0x7fa36c07b0c0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.832+0000 7fa371f04640 1 --2- 172.21.15.67:0/3061154120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa36c10bff0 0x7fa36c10c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.832+0000 7fa371f04640 1 --2- 172.21.15.67:0/3061154120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa36c10a090 0x7fa36c10a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.832+0000 7fa371f04640 1 -- 172.21.15.67:0/3061154120 >> 172.21.15.67:0/3061154120 conn(0x7fa36c06e320 msgr2=0x7fa36c06e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.832+0000 7fa371f04640 1 -- 172.21.15.67:0/3061154120 shutdown_connections 2024-07-31T15:06:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.832+0000 7fa371f04640 1 -- 172.21.15.67:0/3061154120 wait complete. 2024-07-31T15:06:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.833+0000 7fa371f04640 1 Processor -- start 2024-07-31T15:06:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.833+0000 7fa371f04640 1 -- start start 2024-07-31T15:06:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.833+0000 7fa371f04640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa36c10a090 0x7fa36c080050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.833+0000 7fa371f04640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa36c10bff0 0x7fa36c080590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.833+0000 7fa371f04640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa36c10c940 0x7fa36c080ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.833+0000 7fa371f04640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c07e5b0 con 0x7fa36c10c940 2024-07-31T15:06:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.833+0000 7fa371f04640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c07e430 con 0x7fa36c10bff0 2024-07-31T15:06:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.833+0000 7fa371f04640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c07e730 con 0x7fa36c10a090 2024-07-31T15:06:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.834+0000 7fa36bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa36c10bff0 0x7fa36c080590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.834+0000 7fa36bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa36c10bff0 0x7fa36c080590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:49312/0 (socket says 172.21.15.67:49312) 2024-07-31T15:06:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.834+0000 7fa370f02640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa36c10a090 0x7fa36c080050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.834+0000 7fa36bfff640 1 -- 172.21.15.67:0/3334567302 learned_addr learned my addr 172.21.15.67:0/3334567302 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.834+0000 7fa371703640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa36c10c940 0x7fa36c080ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.834+0000 7fa36bfff640 1 -- 172.21.15.67:0/3334567302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa36c10a090 msgr2=0x7fa36c080050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.834+0000 7fa36bfff640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa36c10a090 0x7fa36c080050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.834+0000 7fa36bfff640 1 -- 172.21.15.67:0/3334567302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa36c10c940 msgr2=0x7fa36c080ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.834+0000 7fa36bfff640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa36c10c940 0x7fa36c080ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.834+0000 7fa36bfff640 1 -- 172.21.15.67:0/3334567302 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36c1bd310 con 0x7fa36c10bff0 2024-07-31T15:06:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.834+0000 7fa371703640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa36c10c940 0x7fa36c080ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.835+0000 7fa36bfff640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa36c10bff0 0x7fa36c080590 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa36400b990 tx=0x7fa36400be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.848+0000 7fa369ffb640 1 -- 172.21.15.67:0/3334567302 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364019070 con 0x7fa36c10bff0 2024-07-31T15:06:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.848+0000 7fa369ffb640 1 -- 172.21.15.67:0/3334567302 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa364014440 con 0x7fa36c10bff0 2024-07-31T15:06:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.848+0000 7fa369ffb640 1 -- 172.21.15.67:0/3334567302 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364012580 con 0x7fa36c10bff0 2024-07-31T15:06:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.848+0000 7fa371f04640 1 -- 172.21.15.67:0/3334567302 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36c1bd600 con 0x7fa36c10bff0 2024-07-31T15:06:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.848+0000 7fa371f04640 1 -- 172.21.15.67:0/3334567302 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa36c1bdbd0 con 0x7fa36c10bff0 2024-07-31T15:06:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.852+0000 7fa37a05c740 1 Processor -- start 2024-07-31T15:06:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.852+0000 7fa37a05c740 1 -- start start 2024-07-31T15:06:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.852+0000 7fa369ffb640 1 -- 172.21.15.67:0/3334567302 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fa36401c070 con 0x7fa36c10bff0 2024-07-31T15:06:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.852+0000 7fa37a05c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75821b0 0x562ea7924ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa37a05c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75592f0 0x562ea7925420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa37a05c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75614d0 0x562ea7925960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa37a05c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562ea789a3b0 con 0x562ea75821b0 2024-07-31T15:06:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa371703640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75821b0 0x562ea7924ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa37a05c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562ea78898b0 con 0x562ea75592f0 2024-07-31T15:06:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa371703640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75821b0 0x562ea7924ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42702/0 (socket says 172.21.15.67:42702) 2024-07-31T15:06:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa371703640 1 -- 172.21.15.67:0/3880600619 learned_addr learned my addr 172.21.15.67:0/3880600619 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa37a05c740 1 -- 172.21.15.67:0/3880600619 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562ea788d8f0 con 0x562ea75614d0 2024-07-31T15:06:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa370f02640 1 --2- 172.21.15.67:0/3880600619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75592f0 0x562ea7925420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa36bfff640 1 --2- 172.21.15.67:0/3880600619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75614d0 0x562ea7925960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa370f02640 1 -- 172.21.15.67:0/3880600619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75614d0 msgr2=0x562ea7925960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa370f02640 1 --2- 172.21.15.67:0/3880600619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75614d0 0x562ea7925960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa370f02640 1 -- 172.21.15.67:0/3880600619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75821b0 msgr2=0x562ea7924ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa370f02640 1 --2- 172.21.15.67:0/3880600619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75821b0 0x562ea7924ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa370f02640 1 -- 172.21.15.67:0/3880600619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea78309a0 con 0x562ea75592f0 2024-07-31T15:06:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.853+0000 7fa371703640 1 --2- 172.21.15.67:0/3880600619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75821b0 0x562ea7924ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.854+0000 7fa370f02640 1 --2- 172.21.15.67:0/3880600619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75592f0 0x562ea7925420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00db10 tx=0x7fa35c00d680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace92893cb2bf3b0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.855+0000 7fa369ffb640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa3540b3f40 0x7fa3540b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.855+0000 7fa371703640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa3540b3f40 0x7fa3540b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.855+0000 7fa369ffb640 1 -- 172.21.15.67:0/3334567302 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7fa3640e3690 con 0x7fa36c10bff0 2024-07-31T15:06:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.856+0000 7fa371703640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa3540b3f40 0x7fa3540b6450 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa360009720 tx=0x7fa360011040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.866+0000 7fa352ffd640 1 -- 172.21.15.67:0/3880600619 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c01c070 con 0x562ea75592f0 2024-07-31T15:06:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.866+0000 7fa352ffd640 1 -- 172.21.15.67:0/3880600619 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa35c009d00 con 0x562ea75592f0 2024-07-31T15:06:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.867+0000 7fa352ffd640 1 -- 172.21.15.67:0/3880600619 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c0165d0 con 0x562ea75592f0 2024-07-31T15:06:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.868+0000 7fa37a05c740 1 -- 172.21.15.67:0/3880600619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75592f0 msgr2=0x562ea7925420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.868+0000 7fa37a05c740 1 --2- 172.21.15.67:0/3880600619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75592f0 0x562ea7925420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00db10 tx=0x7fa35c00d680 comp rx=0 tx=0).stop 2024-07-31T15:06:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.869+0000 7fa37a05c740 1 -- 172.21.15.67:0/3880600619 shutdown_connections 2024-07-31T15:06:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.869+0000 7fa37a05c740 1 --2- 172.21.15.67:0/3880600619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75614d0 0x562ea7925960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.869+0000 7fa37a05c740 1 --2- 172.21.15.67:0/3880600619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75592f0 0x562ea7925420 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.869+0000 7fa37a05c740 1 --2- 172.21.15.67:0/3880600619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75821b0 0x562ea7924ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.869+0000 7fa37a05c740 1 -- 172.21.15.67:0/3880600619 >> 172.21.15.67:0/3880600619 conn(0x562ea7576300 msgr2=0x562ea7922dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.869+0000 7fa37a05c740 1 -- 172.21.15.67:0/3880600619 shutdown_connections 2024-07-31T15:06:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.869+0000 7fa37a05c740 1 -- 172.21.15.67:0/3880600619 wait complete. 2024-07-31T15:06:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.869+0000 7fa37a05c740 1 -- start start 2024-07-31T15:06:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.869+0000 7fa37a05c740 1 Processor -- start 2024-07-31T15:06:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.870+0000 7fa37a05c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75592f0 0x562ea79309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.870+0000 7fa37a05c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75614d0 0x562ea7930f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.870+0000 7fa37a05c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75821b0 0x562ea793c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.870+0000 7fa37a05c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562ea77c3ab0 con 0x562ea75592f0 2024-07-31T15:06:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.870+0000 7fa37a05c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562ea7811cf0 con 0x562ea75614d0 2024-07-31T15:06:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.870+0000 7fa37a05c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562ea7933660 con 0x562ea75821b0 2024-07-31T15:06:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.870+0000 7fa37a05c740 10 client.0 _ll_register_callbacks cb 0x562ea78839f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.871+0000 7fa370f02640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75592f0 0x562ea79309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.871+0000 7fa370f02640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75592f0 0x562ea79309e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42718/0 (socket says 172.21.15.67:42718) 2024-07-31T15:06:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.871+0000 7fa370f02640 1 -- 172.21.15.67:0/1473779639 learned_addr learned my addr 172.21.15.67:0/1473779639 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.871+0000 7fa371703640 1 --2- 172.21.15.67:0/1473779639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75821b0 0x562ea793c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.871+0000 7fa36bfff640 1 --2- 172.21.15.67:0/1473779639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75614d0 0x562ea7930f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.871+0000 7fa370f02640 1 -- 172.21.15.67:0/1473779639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75821b0 msgr2=0x562ea793c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.871+0000 7fa370f02640 1 --2- 172.21.15.67:0/1473779639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75821b0 0x562ea793c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.871+0000 7fa370f02640 1 -- 172.21.15.67:0/1473779639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75614d0 msgr2=0x562ea7930f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.871+0000 7fa370f02640 1 --2- 172.21.15.67:0/1473779639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75614d0 0x562ea7930f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.871+0000 7fa370f02640 1 -- 172.21.15.67:0/1473779639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ea75ed790 con 0x562ea75592f0 2024-07-31T15:06:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.871+0000 7fa36bfff640 1 --2- 172.21.15.67:0/1473779639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75614d0 0x562ea7930f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:06:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.872+0000 7fa370f02640 1 --2- 172.21.15.67:0/1473779639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75592f0 0x562ea79309e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0056a0 tx=0x7fa35c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e21fcb3fa247ec6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.878+0000 7fa34bfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:06:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.878+0000 7fa34bfff640 1 -- 172.21.15.67:0/1473779639 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c025030 con 0x562ea75592f0 2024-07-31T15:06:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.878+0000 7fa37a05c740 1 -- 172.21.15.67:0/1473779639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa35c009d00 con 0x562ea75592f0 2024-07-31T15:06:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.878+0000 7fa34bfff640 1 -- 172.21.15.67:0/1473779639 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7fa35c009b00 con 0x562ea75592f0 2024-07-31T15:06:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.888+0000 7fa34bfff640 1 -- 172.21.15.67:0/1473779639 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fa35c02f050 con 0x562ea75592f0 2024-07-31T15:06:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.888+0000 7fa37a05c740 10 client.18759 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.888+0000 7fa37a05c740 1 -- 172.21.15.67:0/1473779639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x562ea7666620 con 0x562ea75592f0 2024-07-31T15:06:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.889+0000 7fa34bfff640 1 -- 172.21.15.67:0/1473779639 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa35c009d00 con 0x562ea75592f0 2024-07-31T15:06:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.889+0000 7fa34bfff640 1 -- 172.21.15.67:0/1473779639 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fa35c016070 con 0x562ea75592f0 2024-07-31T15:06:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.889+0000 7fa37a05c740 10 client.18759 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.889+0000 7fa37a05c740 10 client.18759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.889+0000 7fa37a05c740 20 client.18759 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.889+0000 7fa37a05c740 1 --2- 172.21.15.67:0/1473779639 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562ea7540d80 0x562ea79427e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.889+0000 7fa37a05c740 4 client.18759 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:06:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.889+0000 7fa37a05c740 1 -- 172.21.15.67:0/1473779639 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ea789c300 con 0x562ea7540d80 2024-07-31T15:06:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.890+0000 7fa36bfff640 1 --2- 172.21.15.67:0/1473779639 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562ea7540d80 0x562ea79427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.890+0000 7fa34bfff640 10 client.18759 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.891+0000 7fa36bfff640 1 --2- 172.21.15.67:0/1473779639 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562ea7540d80 0x562ea79427e0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d6a32bb18c1bba server_cookie=699dcc1ac71043bd in_seq=0 out_seq=0 2024-07-31T15:06:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.892+0000 7fa34bfff640 1 -- 172.21.15.67:0/1473779639 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ea789c300 con 0x562ea7540d80 2024-07-31T15:06:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.892+0000 7fa34bfff640 10 client.18759 handle_command_reply: tid=0 2024-07-31T15:06:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.892+0000 7fa37a05c740 1 client.18759 shutdown 2024-07-31T15:06:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.892+0000 7fa37a05c740 1 -- 172.21.15.67:0/1473779639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75592f0 msgr2=0x562ea79309e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.892+0000 7fa37a05c740 1 --2- 172.21.15.67:0/1473779639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75592f0 0x562ea79309e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0056a0 tx=0x7fa35c0056d0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.892+0000 7fa37a05c740 1 -- 172.21.15.67:0/1473779639 shutdown_connections 2024-07-31T15:06:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.892+0000 7fa37a05c740 1 --2- 172.21.15.67:0/1473779639 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562ea7540d80 0x562ea79427e0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.892+0000 7fa37a05c740 1 --2- 172.21.15.67:0/1473779639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75821b0 0x562ea793c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.892+0000 7fa37a05c740 1 --2- 172.21.15.67:0/1473779639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75614d0 0x562ea7930f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.892+0000 7fa34bfff640 0 client.18759 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.892+0000 7fa37a05c740 1 --2- 172.21.15.67:0/1473779639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75592f0 0x562ea79309e0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.892+0000 7fa37a05c740 1 -- 172.21.15.67:0/1473779639 >> 172.21.15.67:0/1473779639 conn(0x562ea7576300 msgr2=0x562ea7925f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.892+0000 7fa37a05c740 1 -- 172.21.15.67:0/1473779639 shutdown_connections 2024-07-31T15:06:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.893+0000 7fa37a05c740 1 -- 172.21.15.67:0/1473779639 wait complete. 2024-07-31T15:06:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.893+0000 7fa37a05c740 20 client.18759 trim_cache size 0 max 16384 2024-07-31T15:06:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.941+0000 7fa37a05c740 1 Processor -- start 2024-07-31T15:06:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.941+0000 7fa37a05c740 1 -- start start 2024-07-31T15:06:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.942+0000 7fa37a05c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75592f0 0x562ea79268d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.942+0000 7fa37a05c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75614d0 0x562ea7930040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.942+0000 7fa37a05c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75821b0 0x562ea7932400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.942+0000 7fa37a05c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1c7ca0 con 0x562ea75614d0 2024-07-31T15:06:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.942+0000 7fa36bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75614d0 0x562ea7930040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.942+0000 7fa37a05c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1c97e0 con 0x562ea75592f0 2024-07-31T15:06:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.942+0000 7fa37a05c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1c8370 con 0x562ea75821b0 2024-07-31T15:06:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.942+0000 7fa36bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75614d0 0x562ea7930040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42728/0 (socket says 172.21.15.67:42728) 2024-07-31T15:06:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.942+0000 7fa36bfff640 1 -- 172.21.15.67:0/3776993329 learned_addr learned my addr 172.21.15.67:0/3776993329 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.942+0000 7fa370f02640 1 --2- 172.21.15.67:0/3776993329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75592f0 0x562ea79268d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.943+0000 7fa371703640 1 --2- 172.21.15.67:0/3776993329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75821b0 0x562ea7932400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.943+0000 7fa371703640 1 -- 172.21.15.67:0/3776993329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75592f0 msgr2=0x562ea79268d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.943+0000 7fa371703640 1 --2- 172.21.15.67:0/3776993329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75592f0 0x562ea79268d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.943+0000 7fa371703640 1 -- 172.21.15.67:0/3776993329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75614d0 msgr2=0x562ea7930040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.943+0000 7fa371703640 1 --2- 172.21.15.67:0/3776993329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75614d0 0x562ea7930040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.943+0000 7fa371703640 1 -- 172.21.15.67:0/3776993329 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea79271e0 con 0x562ea75821b0 2024-07-31T15:06:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.944+0000 7fa371703640 1 --2- 172.21.15.67:0/3776993329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75821b0 0x562ea7932400 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa360009230 tx=0x7fa360041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88361ce71070e709 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.945+0000 7fa3527fc640 1 -- 172.21.15.67:0/3776993329 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36000fd60 con 0x562ea75821b0 2024-07-31T15:06:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.945+0000 7fa3527fc640 1 -- 172.21.15.67:0/3776993329 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa360011540 con 0x562ea75821b0 2024-07-31T15:06:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.945+0000 7fa3527fc640 1 -- 172.21.15.67:0/3776993329 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36003e7b0 con 0x562ea75821b0 2024-07-31T15:06:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.946+0000 7fa37a05c740 1 -- 172.21.15.67:0/3776993329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75821b0 msgr2=0x562ea7932400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.946+0000 7fa37a05c740 1 --2- 172.21.15.67:0/3776993329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75821b0 0x562ea7932400 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa360009230 tx=0x7fa360041000 comp rx=0 tx=0).stop 2024-07-31T15:06:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.946+0000 7fa37a05c740 1 -- 172.21.15.67:0/3776993329 shutdown_connections 2024-07-31T15:06:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.946+0000 7fa37a05c740 1 --2- 172.21.15.67:0/3776993329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75821b0 0x562ea7932400 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.946+0000 7fa37a05c740 1 --2- 172.21.15.67:0/3776993329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75614d0 0x562ea7930040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.946+0000 7fa37a05c740 1 --2- 172.21.15.67:0/3776993329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75592f0 0x562ea79268d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.946+0000 7fa37a05c740 1 -- 172.21.15.67:0/3776993329 >> 172.21.15.67:0/3776993329 conn(0x562ea7576300 msgr2=0x562ea7931cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.947+0000 7fa37a05c740 1 -- 172.21.15.67:0/3776993329 shutdown_connections 2024-07-31T15:06:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.947+0000 7fa37a05c740 1 -- 172.21.15.67:0/3776993329 wait complete. 2024-07-31T15:06:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.947+0000 7fa37a05c740 1 -- start start 2024-07-31T15:06:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.947+0000 7fa37a05c740 1 Processor -- start 2024-07-31T15:06:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.948+0000 7fa37a05c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75592f0 0x562ea7945ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.948+0000 7fa37a05c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75614d0 0x562ea79443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.948+0000 7fa37a05c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75821b0 0x562ea7944900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.948+0000 7fa37a05c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1c9110 con 0x562ea75614d0 2024-07-31T15:06:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.948+0000 7fa37a05c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1c8a40 con 0x562ea75821b0 2024-07-31T15:06:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.948+0000 7fa37a05c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1c5ca0 con 0x562ea75592f0 2024-07-31T15:06:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.948+0000 7fa37a05c740 10 client.0 _ll_register_callbacks cb 0x7fa35c01a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.949+0000 7fa36bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75614d0 0x562ea79443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.949+0000 7fa36bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75614d0 0x562ea79443c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42730/0 (socket says 172.21.15.67:42730) 2024-07-31T15:06:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.949+0000 7fa36bfff640 1 -- 172.21.15.67:0/2143830528 learned_addr learned my addr 172.21.15.67:0/2143830528 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.949+0000 7fa370f02640 1 --2- 172.21.15.67:0/2143830528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75592f0 0x562ea7945ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.949+0000 7fa370f02640 1 -- 172.21.15.67:0/2143830528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75821b0 msgr2=0x562ea7944900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:06:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.949+0000 7fa370f02640 1 --2- 172.21.15.67:0/2143830528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75821b0 0x562ea7944900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.949+0000 7fa370f02640 1 -- 172.21.15.67:0/2143830528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75614d0 msgr2=0x562ea79443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.949+0000 7fa370f02640 1 --2- 172.21.15.67:0/2143830528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75614d0 0x562ea79443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.949+0000 7fa370f02640 1 -- 172.21.15.67:0/2143830528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ea7945090 con 0x562ea75592f0 2024-07-31T15:06:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.949+0000 7fa36bfff640 1 --2- 172.21.15.67:0/2143830528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75614d0 0x562ea79443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.950+0000 7fa370f02640 1 --2- 172.21.15.67:0/2143830528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75592f0 0x562ea7945ef0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00a010 tx=0x7fa35c017b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa1fd4c67717d093 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:29.953 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/kasprintf.o 2024-07-31T15:06:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.954+0000 7fa34bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:06:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.954+0000 7fa34bfff640 1 -- 172.21.15.67:0/2143830528 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c037020 con 0x562ea75592f0 2024-07-31T15:06:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.954+0000 7fa37a05c740 1 -- 172.21.15.67:0/2143830528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa360011540 con 0x562ea75592f0 2024-07-31T15:06:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.954+0000 7fa34bfff640 1 -- 172.21.15.67:0/2143830528 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7fa35c009d00 con 0x562ea75592f0 2024-07-31T15:06:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.954+0000 7fa34bfff640 1 -- 172.21.15.67:0/2143830528 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fa35c042050 con 0x562ea75592f0 2024-07-31T15:06:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.954+0000 7fa37a05c740 10 client.28225 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.954+0000 7fa37a05c740 1 -- 172.21.15.67:0/2143830528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x562ea79a9c90 con 0x562ea75592f0 2024-07-31T15:06:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.955+0000 7fa34bfff640 1 -- 172.21.15.67:0/2143830528 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa360011540 con 0x562ea75592f0 2024-07-31T15:06:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.956+0000 7fa34bfff640 1 -- 172.21.15.67:0/2143830528 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fa35c016050 con 0x562ea75592f0 2024-07-31T15:06:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.956+0000 7fa37a05c740 10 client.28225 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.956+0000 7fa37a05c740 10 client.28225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.956+0000 7fa37a05c740 20 client.28225 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.956+0000 7fa37a05c740 1 --2- 172.21.15.67:0/2143830528 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562ea75161c0 0x562ea79ac390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.956+0000 7fa37a05c740 4 client.28225 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:06:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.956+0000 7fa37a05c740 1 -- 172.21.15.67:0/2143830528 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562ea7897da0 con 0x562ea75161c0 2024-07-31T15:06:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.957+0000 7fa36bfff640 1 --2- 172.21.15.67:0/2143830528 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562ea75161c0 0x562ea79ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.957+0000 7fa36bfff640 1 --2- 172.21.15.67:0/2143830528 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562ea75161c0 0x562ea79ac390 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f291fda9d3051c18 server_cookie=e46e67dd0713d955 in_seq=0 out_seq=0 2024-07-31T15:06:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:29.957+0000 7fa34bfff640 10 client.28225 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:30.110 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/groups.o 2024-07-31T15:06:30.224 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exsystem.o 2024-07-31T15:06:30.277 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/bitmap.o 2024-07-31T15:06:30.412 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/hdmi_chmap.o 2024-07-31T15:06:30.423 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/last.o 2024-07-31T15:06:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.462+0000 7fa34bfff640 1 -- 172.21.15.67:0/2143830528 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562ea7897da0 con 0x562ea75161c0 2024-07-31T15:06:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.462+0000 7fa34bfff640 10 client.28225 handle_command_reply: tid=0 2024-07-31T15:06:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.466+0000 7fa37a05c740 1 client.28225 shutdown 2024-07-31T15:06:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.467+0000 7fa37a05c740 1 -- 172.21.15.67:0/2143830528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75592f0 msgr2=0x562ea7945ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.467+0000 7fa37a05c740 1 --2- 172.21.15.67:0/2143830528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75592f0 0x562ea7945ef0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00a010 tx=0x7fa35c017b50 comp rx=0 tx=0).stop 2024-07-31T15:06:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.467+0000 7fa37a05c740 1 -- 172.21.15.67:0/2143830528 shutdown_connections 2024-07-31T15:06:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.467+0000 7fa37a05c740 1 --2- 172.21.15.67:0/2143830528 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x562ea75161c0 0x562ea79ac390 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.467+0000 7fa37a05c740 1 --2- 172.21.15.67:0/2143830528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ea75821b0 0x562ea7944900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.467+0000 7fa37a05c740 1 --2- 172.21.15.67:0/2143830528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ea75614d0 0x562ea79443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.467+0000 7fa37a05c740 1 --2- 172.21.15.67:0/2143830528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ea75592f0 0x562ea7945ef0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.467+0000 7fa37a05c740 1 -- 172.21.15.67:0/2143830528 >> 172.21.15.67:0/2143830528 conn(0x562ea7576300 msgr2=0x562ea79266a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.467+0000 7fa34bfff640 0 client.28225 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.467+0000 7fa37a05c740 1 -- 172.21.15.67:0/2143830528 shutdown_connections 2024-07-31T15:06:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.467+0000 7fa37a05c740 1 -- 172.21.15.67:0/2143830528 wait complete. 2024-07-31T15:06:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.467+0000 7fa37a05c740 20 client.28225 trim_cache size 0 max 16384 2024-07-31T15:06:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:06:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:06:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:06:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:06:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.469+0000 7fa371f04640 1 -- 172.21.15.67:0/3334567302 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa3540b3f40 msgr2=0x7fa3540b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.469+0000 7fa371f04640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa3540b3f40 0x7fa3540b6450 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa360009720 tx=0x7fa360011040 comp rx=0 tx=0).stop 2024-07-31T15:06:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.470+0000 7fa371f04640 1 -- 172.21.15.67:0/3334567302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa36c10bff0 msgr2=0x7fa36c080590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.470+0000 7fa371f04640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa36c10bff0 0x7fa36c080590 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa36400b990 tx=0x7fa36400be50 comp rx=0 tx=0).stop 2024-07-31T15:06:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.470+0000 7fa371f04640 1 -- 172.21.15.67:0/3334567302 shutdown_connections 2024-07-31T15:06:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.470+0000 7fa371f04640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa3540b3f40 0x7fa3540b6450 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.470+0000 7fa371f04640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa36c10c940 0x7fa36c080ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.470+0000 7fa371f04640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa36c10bff0 0x7fa36c080590 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.470+0000 7fa371f04640 1 --2- 172.21.15.67:0/3334567302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa36c10a090 0x7fa36c080050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.470+0000 7fa371f04640 1 -- 172.21.15.67:0/3334567302 >> 172.21.15.67:0/3334567302 conn(0x7fa36c06e320 msgr2=0x7fa36c10d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.473+0000 7fa371f04640 1 -- 172.21.15.67:0/3334567302 shutdown_connections 2024-07-31T15:06:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:30.473+0000 7fa371f04640 1 -- 172.21.15.67:0/3334567302 wait complete. 2024-07-31T15:06:30.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.646+0000 7f1de0547640 1 -- 172.21.15.67:0/2806866973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dd8111970 msgr2=0x7f1dd8109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:30.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.646+0000 7f1de0547640 1 --2- 172.21.15.67:0/2806866973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dd8111970 0x7f1dd8109df0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009900 tx=0x7f1dd402a440 comp rx=0 tx=0).stop 2024-07-31T15:06:30.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.653+0000 7f1de0547640 1 -- 172.21.15.67:0/2806866973 shutdown_connections 2024-07-31T15:06:30.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.653+0000 7f1de0547640 1 --2- 172.21.15.67:0/2806866973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dd811c660 0x7f1dd811eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:30.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.653+0000 7f1de0547640 1 --2- 172.21.15.67:0/2806866973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd810a410 0x7f1dd810a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:30.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.653+0000 7f1de0547640 1 --2- 172.21.15.67:0/2806866973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dd8111970 0x7f1dd8109df0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:30.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.653+0000 7f1de0547640 1 -- 172.21.15.67:0/2806866973 >> 172.21.15.67:0/2806866973 conn(0x7f1dd80708f0 msgr2=0x7f1dd8070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:30.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.662+0000 7f1de0547640 1 -- 172.21.15.67:0/2806866973 shutdown_connections 2024-07-31T15:06:30.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.664+0000 7f1de0547640 1 -- 172.21.15.67:0/2806866973 wait complete. 2024-07-31T15:06:30.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.665+0000 7f1de0547640 1 Processor -- start 2024-07-31T15:06:30.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.665+0000 7f1de0547640 1 -- start start 2024-07-31T15:06:30.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.665+0000 7f1de0547640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd810a410 0x7f1dd811bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:30.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.665+0000 7f1de0547640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dd8111970 0x7f1dd8116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:30.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.665+0000 7f1de0547640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dd811c660 0x7f1dd8116990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:30.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.665+0000 7f1de0547640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81126b0 con 0x7f1dd8111970 2024-07-31T15:06:30.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.665+0000 7f1de0547640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd8112530 con 0x7f1dd810a410 2024-07-31T15:06:30.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.665+0000 7f1de0547640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd8112830 con 0x7f1dd811c660 2024-07-31T15:06:30.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.670+0000 7f1ddeae4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dd811c660 0x7f1dd8116990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:30.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.670+0000 7f1dddae2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dd8111970 0x7f1dd8116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:30.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.670+0000 7f1dde2e3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd810a410 0x7f1dd811bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:30.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.670+0000 7f1dddae2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dd8111970 0x7f1dd8116290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42748/0 (socket says 172.21.15.67:42748) 2024-07-31T15:06:30.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.670+0000 7f1dddae2640 1 -- 172.21.15.67:0/1982413674 learned_addr learned my addr 172.21.15.67:0/1982413674 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:30.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.677+0000 7f1dde2e3640 1 -- 172.21.15.67:0/1982413674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dd811c660 msgr2=0x7f1dd8116990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:30.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.677+0000 7f1dde2e3640 1 --2- 172.21.15.67:0/1982413674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dd811c660 0x7f1dd8116990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:30.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.677+0000 7f1dde2e3640 1 -- 172.21.15.67:0/1982413674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dd8111970 msgr2=0x7f1dd8116290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:30.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.677+0000 7f1dde2e3640 1 --2- 172.21.15.67:0/1982413674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dd8111970 0x7f1dd8116290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:30.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.677+0000 7f1dde2e3640 1 -- 172.21.15.67:0/1982413674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd807b050 con 0x7f1dd810a410 2024-07-31T15:06:30.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.684+0000 7f1dde2e3640 1 --2- 172.21.15.67:0/1982413674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd810a410 0x7f1dd811bfd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40058c0 tx=0x7f1dd40319e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:30.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.685+0000 7f1dcf7fe640 1 -- 172.21.15.67:0/1982413674 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4041020 con 0x7f1dd810a410 2024-07-31T15:06:30.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.685+0000 7f1dcf7fe640 1 -- 172.21.15.67:0/1982413674 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd40046c0 con 0x7f1dd810a410 2024-07-31T15:06:30.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.685+0000 7f1de0547640 1 -- 172.21.15.67:0/1982413674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd807b2e0 con 0x7f1dd810a410 2024-07-31T15:06:30.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.685+0000 7f1de0547640 1 -- 172.21.15.67:0/1982413674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd807b810 con 0x7f1dd810a410 2024-07-31T15:06:30.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.685+0000 7f1dcf7fe640 1 -- 172.21.15.67:0/1982413674 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4036040 con 0x7f1dd810a410 2024-07-31T15:06:30.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.690+0000 7f1de0547640 1 -- 172.21.15.67:0/1982413674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd810b010 con 0x7f1dd810a410 2024-07-31T15:06:30.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.692+0000 7f1dcf7fe640 1 -- 172.21.15.67:0/1982413674 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1dd4044020 con 0x7f1dd810a410 2024-07-31T15:06:30.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.694+0000 7f1dcf7fe640 1 --2- 172.21.15.67:0/1982413674 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1dc40b3e70 0x7f1dc40b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:30.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.694+0000 7f1dcf7fe640 1 -- 172.21.15.67:0/1982413674 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f1dd4104e80 con 0x7f1dd810a410 2024-07-31T15:06:30.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.694+0000 7f1dddae2640 1 --2- 172.21.15.67:0/1982413674 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1dc40b3e70 0x7f1dc40b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:30.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.695+0000 7f1dddae2640 1 --2- 172.21.15.67:0/1982413674 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1dc40b3e70 0x7f1dc40b6380 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8071390 tx=0x7f1dc800a300 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:30.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:30.705+0000 7f1dcf7fe640 1 -- 172.21.15.67:0/1982413674 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f1dd404b020 con 0x7f1dd810a410 2024-07-31T15:06:30.722 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/extrace.o 2024-07-31T15:06:30.891 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/exutils.o 2024-07-31T15:06:31.050 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/scatterlist.o 2024-07-31T15:06:31.054 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/kcmp.o 2024-07-31T15:06:31.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:31.206+0000 7f1de0547640 1 -- 172.21.15.67:0/1982413674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dd807bb30 con 0x7f1dd810a410 2024-07-31T15:06:31.209 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/link_watch.o 2024-07-31T15:06:31.213 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:06:31.213 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"btime":"2024-07-31T15:06:04:323642+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:06:04.323492+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28214":{"gid":28214,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1516494761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1516494761},{"type":"v1","addr":"172.21.15.73:6835","nonce":1516494761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:06:31.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:31.211+0000 7f1dcf7fe640 1 -- 172.21.15.67:0/1982413674 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11962 (secure 0 0 0) 0x7f1dd40cbdf0 con 0x7f1dd810a410 2024-07-31T15:06:31.214 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-07-31T15:06:31.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:31.218+0000 7f1dcd7fa640 1 -- 172.21.15.67:0/1982413674 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1dc40b3e70 msgr2=0x7f1dc40b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:31.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:31.218+0000 7f1dcd7fa640 1 --2- 172.21.15.67:0/1982413674 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1dc40b3e70 0x7f1dc40b6380 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8071390 tx=0x7f1dc800a300 comp rx=0 tx=0).stop 2024-07-31T15:06:31.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:31.218+0000 7f1dcd7fa640 1 -- 172.21.15.67:0/1982413674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd810a410 msgr2=0x7f1dd811bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:31.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:31.218+0000 7f1dcd7fa640 1 --2- 172.21.15.67:0/1982413674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd810a410 0x7f1dd811bfd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40058c0 tx=0x7f1dd40319e0 comp rx=0 tx=0).stop 2024-07-31T15:06:31.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:31.218+0000 7f1dcd7fa640 1 -- 172.21.15.67:0/1982413674 shutdown_connections 2024-07-31T15:06:31.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:31.218+0000 7f1dcd7fa640 1 --2- 172.21.15.67:0/1982413674 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1dc40b3e70 0x7f1dc40b6380 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:31.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:31.218+0000 7f1dcd7fa640 1 --2- 172.21.15.67:0/1982413674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dd811c660 0x7f1dd8116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:31.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:31.218+0000 7f1dcd7fa640 1 --2- 172.21.15.67:0/1982413674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dd8111970 0x7f1dd8116290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:31.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:31.218+0000 7f1dcd7fa640 1 --2- 172.21.15.67:0/1982413674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd810a410 0x7f1dd811bfd0 secure :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40058c0 tx=0x7f1dd40319e0 comp rx=0 tx=0).stop 2024-07-31T15:06:31.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:31.218+0000 7f1dcd7fa640 1 -- 172.21.15.67:0/1982413674 >> 172.21.15.67:0/1982413674 conn(0x7f1dd80708f0 msgr2=0x7f1dd8073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:31.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:31.222+0000 7f1dcd7fa640 1 -- 172.21.15.67:0/1982413674 shutdown_connections 2024-07-31T15:06:31.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:31.224+0000 7f1dcd7fa640 1 -- 172.21.15.67:0/1982413674 wait complete. 2024-07-31T15:06:31.295 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/trace.o 2024-07-31T15:06:31.326 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4renewd.o 2024-07-31T15:06:31.352 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/hdac_component.o 2024-07-31T15:06:31.357 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/hwacpi.o 2024-07-31T15:06:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:31 smithi077 ceph-mon[37327]: pgmap v888: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 147 KiB/s wr, 3.83k op/s 2024-07-31T15:06:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:31 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1982413674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:06:31.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:31 smithi067 ceph-mon[32750]: pgmap v888: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 147 KiB/s wr, 3.83k op/s 2024-07-31T15:06:31.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:31 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1982413674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:06:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:31 smithi073 ceph-mon[36714]: pgmap v888: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 147 KiB/s wr, 3.83k op/s 2024-07-31T15:06:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:31 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1982413674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:06:31.710 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/hwesleep.o 2024-07-31T15:06:31.793 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/freezer.o 2024-07-31T15:06:32.010 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/profile.o 2024-07-31T15:06:32.017 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/list_sort.o 2024-07-31T15:06:32.116 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/hwgpe.o 2024-07-31T15:06:32.118 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/hdac_i915.o 2024-07-31T15:06:32.147 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/uuid.o 2024-07-31T15:06:32.149 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/iov_iter.o 2024-07-31T15:06:32.186 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0' 2024-07-31T15:06:32.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0 2024-07-31T15:06:32.319 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4super.o 2024-07-31T15:06:32.374 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/filter.o 2024-07-31T15:06:32.519 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/sock_diag.o 2024-07-31T15:06:32.639 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/hwregs.o 2024-07-31T15:06:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:06:32.755 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/intel-dsp-config.o 2024-07-31T15:06:32.767 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/stacktrace.o 2024-07-31T15:06:32.981 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/dev_ioctl.o 2024-07-31T15:06:33.129 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/hwsleep.o 2024-07-31T15:06:33.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:33 smithi067 ceph-mon[32750]: pgmap v889: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 183 KiB/s wr, 3.07k op/s 2024-07-31T15:06:33.356 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4file.o 2024-07-31T15:06:33.473 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/dma.o 2024-07-31T15:06:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:33 smithi077 ceph-mon[37327]: pgmap v889: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 183 KiB/s wr, 3.07k op/s 2024-07-31T15:06:33.492 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/intel-nhlt.o 2024-07-31T15:06:33.565 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/hwvalid.o 2024-07-31T15:06:33.616 INFO:tasks.workunit.client.0.smithi067.stdout: CC sound/hda/intel-sdw-acpi.o 2024-07-31T15:06:33.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:33 smithi073 ceph-mon[36714]: pgmap v889: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 183 KiB/s wr, 3.07k op/s 2024-07-31T15:06:33.942 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/smp.o 2024-07-31T15:06:33.952 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/hwxface.o 2024-07-31T15:06:34.006 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/tso.o 2024-07-31T15:06:34.040 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/sock_reuseport.o 2024-07-31T15:06:34.047 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/clz_ctz.o 2024-07-31T15:06:34.082 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/hda/built-in.a 2024-07-31T15:06:34.115 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/bsearch.o 2024-07-31T15:06:34.134 INFO:tasks.workunit.client.0.smithi067.stdout: AR sound/built-in.a 2024-07-31T15:06:34.228 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:34 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:06:34.228 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:34 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:06:34.229 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:34 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:06:34.229 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:34 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:06:34.335 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/hwxfsleep.o 2024-07-31T15:06:34.396 INFO:tasks.workunit.client.0.smithi067.stdout: AR virt/lib/built-in.a 2024-07-31T15:06:34.427 INFO:tasks.workunit.client.0.smithi067.stdout: AR virt/built-in.a 2024-07-31T15:06:34.467 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/hwpci.o 2024-07-31T15:06:34.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:34 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:06:34.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:34 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:06:34.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:34 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:06:34.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:34 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:06:34.569 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/delegation.o 2024-07-31T15:06:34.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:34 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:06:34.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:34 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:06:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:34 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:06:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:34 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:06:34.665 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/find_bit.o 2024-07-31T15:06:34.728 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4idmap.o 2024-07-31T15:06:34.885 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsaccess.o 2024-07-31T15:06:34.926 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/llist.o 2024-07-31T15:06:34.958 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/uid16.o 2024-07-31T15:06:35.019 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/unicode/built-in.a 2024-07-31T15:06:35.048 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/memweight.o 2024-07-31T15:06:35.050 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsalloc.o 2024-07-31T15:06:35.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.064+0000 7f8f5678d640 1 -- 172.21.15.67:0/1631179127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f5010a330 msgr2=0x7f8f5010a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:35.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.064+0000 7f8f5678d640 1 --2- 172.21.15.67:0/1631179127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f5010a330 0x7f8f5010a740 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009900 tx=0x7f8f4402a4a0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.064+0000 7f8f5678d640 1 -- 172.21.15.67:0/1631179127 shutdown_connections 2024-07-31T15:06:35.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.064+0000 7f8f5678d640 1 --2- 172.21.15.67:0/1631179127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f5011c5d0 0x7f8f5011e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.064+0000 7f8f5678d640 1 --2- 172.21.15.67:0/1631179127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f5010a330 0x7f8f5010a740 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.064+0000 7f8f5678d640 1 --2- 172.21.15.67:0/1631179127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f50111780 0x7f8f50109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.064+0000 7f8f5678d640 1 -- 172.21.15.67:0/1631179127 >> 172.21.15.67:0/1631179127 conn(0x7f8f5006e440 msgr2=0x7f8f5006e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:35.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.073+0000 7f8f5678d640 1 -- 172.21.15.67:0/1631179127 shutdown_connections 2024-07-31T15:06:35.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.073+0000 7f8f5678d640 1 -- 172.21.15.67:0/1631179127 wait complete. 2024-07-31T15:06:35.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.074+0000 7f8f5678d640 1 Processor -- start 2024-07-31T15:06:35.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.074+0000 7f8f5678d640 1 -- start start 2024-07-31T15:06:35.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.074+0000 7f8f5678d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f5010a330 0x7f8f501a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.074+0000 7f8f5678d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f50111780 0x7f8f501a9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.074+0000 7f8f5678d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5011c5d0 0x7f8f501ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.074+0000 7f8f5678d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8f501221f0 con 0x7f8f50111780 2024-07-31T15:06:35.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.074+0000 7f8f5678d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8f50122070 con 0x7f8f5011c5d0 2024-07-31T15:06:35.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.074+0000 7f8f5678d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8f50122370 con 0x7f8f5010a330 2024-07-31T15:06:35.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.075+0000 7f8f4f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f50111780 0x7f8f501a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:35.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.075+0000 7f8f4f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f50111780 0x7f8f501a9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37584/0 (socket says 172.21.15.67:37584) 2024-07-31T15:06:35.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.075+0000 7f8f4f7fe640 1 -- 172.21.15.67:0/3333641395 learned_addr learned my addr 172.21.15.67:0/3333641395 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:35.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.076+0000 7f8f54d2a640 1 --2- 172.21.15.67:0/3333641395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5011c5d0 0x7f8f501ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:35.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.076+0000 7f8f4f7fe640 1 -- 172.21.15.67:0/3333641395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f5010a330 msgr2=0x7f8f501a8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:35.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.076+0000 7f8f4f7fe640 1 --2- 172.21.15.67:0/3333641395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f5010a330 0x7f8f501a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.076+0000 7f8f4f7fe640 1 -- 172.21.15.67:0/3333641395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5011c5d0 msgr2=0x7f8f501ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:35.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.076+0000 7f8f4f7fe640 1 --2- 172.21.15.67:0/3333641395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5011c5d0 0x7f8f501ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.076+0000 7f8f4f7fe640 1 -- 172.21.15.67:0/3333641395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f501aeb00 con 0x7f8f50111780 2024-07-31T15:06:35.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.080+0000 7f8f4f7fe640 1 --2- 172.21.15.67:0/3333641395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f50111780 0x7f8f501a9160 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f8f4402aa90 tx=0x7f8f44030a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:35.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.080+0000 7f8f4d7fa640 1 -- 172.21.15.67:0/3333641395 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44040070 con 0x7f8f50111780 2024-07-31T15:06:35.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.081+0000 7f8f4d7fa640 1 -- 172.21.15.67:0/3333641395 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f440035f0 con 0x7f8f50111780 2024-07-31T15:06:35.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.081+0000 7f8f4d7fa640 1 -- 172.21.15.67:0/3333641395 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44035030 con 0x7f8f50111780 2024-07-31T15:06:35.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.082+0000 7f8f5678d640 1 -- 172.21.15.67:0/3333641395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f501aed90 con 0x7f8f50111780 2024-07-31T15:06:35.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.082+0000 7f8f5678d640 1 -- 172.21.15.67:0/3333641395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f501af220 con 0x7f8f50111780 2024-07-31T15:06:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.088+0000 7f8f5ccde740 1 Processor -- start 2024-07-31T15:06:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.088+0000 7f8f5ccde740 1 -- start start 2024-07-31T15:06:35.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.089+0000 7f8f5ccde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866b9840 0x55d986a5fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.089+0000 7f8f5ccde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d986690980 0x55d986a5ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.089+0000 7f8f5ccde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d986698b60 0x55d986a604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.089+0000 7f8f5ccde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d9869d4780 con 0x55d986690980 2024-07-31T15:06:35.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.089+0000 7f8f5ccde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d9869c3c80 con 0x55d9866b9840 2024-07-31T15:06:35.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.089+0000 7f8f5ccde740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d9869c7cc0 con 0x55d986698b60 2024-07-31T15:06:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.092+0000 7f8f54d2a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866b9840 0x55d986a5fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:35.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.093+0000 7f8f54d2a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866b9840 0x55d986a5fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51356/0 (socket says 172.21.15.67:51356) 2024-07-31T15:06:35.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.093+0000 7f8f54d2a640 1 -- 172.21.15.67:0/1142632168 learned_addr learned my addr 172.21.15.67:0/1142632168 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:35.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.094+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/1142632168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d986690980 0x55d986a5ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:35.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.095+0000 7f8f4f7fe640 1 --2- 172.21.15.67:0/1142632168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d986698b60 0x55d986a604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:35.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.095+0000 7f8f4ffff640 1 -- 172.21.15.67:0/1142632168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d986698b60 msgr2=0x55d986a604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:35.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.095+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/1142632168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d986698b60 0x55d986a604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.095+0000 7f8f4ffff640 1 -- 172.21.15.67:0/1142632168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866b9840 msgr2=0x55d986a5fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:35.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.095+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/1142632168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866b9840 0x55d986a5fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.095+0000 7f8f4ffff640 1 -- 172.21.15.67:0/1142632168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d986959080 con 0x55d986690980 2024-07-31T15:06:35.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.096+0000 7f8f4d7fa640 1 -- 172.21.15.67:0/3333641395 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8f4403c6b0 con 0x7f8f50111780 2024-07-31T15:06:35.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.096+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/1142632168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d986690980 0x55d986a5ff70 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009720 tx=0x7f8f40009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6444390064e52917 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:35.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.100+0000 7f8f367fc640 1 -- 172.21.15.67:0/1142632168 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40015020 con 0x55d986690980 2024-07-31T15:06:35.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.100+0000 7f8f367fc640 1 -- 172.21.15.67:0/1142632168 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f40009e30 con 0x55d986690980 2024-07-31T15:06:35.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.100+0000 7f8f367fc640 1 -- 172.21.15.67:0/1142632168 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40010380 con 0x55d986690980 2024-07-31T15:06:35.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.102+0000 7f8f5ccde740 1 -- 172.21.15.67:0/1142632168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d986690980 msgr2=0x55d986a5ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:35.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.102+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/1142632168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d986690980 0x55d986a5ff70 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009720 tx=0x7f8f40009be0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.103+0000 7f8f5ccde740 1 -- 172.21.15.67:0/1142632168 shutdown_connections 2024-07-31T15:06:35.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.103+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/1142632168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d986698b60 0x55d986a604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.103+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/1142632168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d986690980 0x55d986a5ff70 secure :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009720 tx=0x7f8f40009be0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.103+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/1142632168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866b9840 0x55d986a5fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.103+0000 7f8f5ccde740 1 -- 172.21.15.67:0/1142632168 >> 172.21.15.67:0/1142632168 conn(0x55d9866ad990 msgr2=0x55d986a5d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:35.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.105+0000 7f8f4d7fa640 1 --2- 172.21.15.67:0/3333641395 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8f380b3e60 0x7f8f380b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.106+0000 7f8f4d7fa640 1 -- 172.21.15.67:0/3333641395 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f8f44104f60 con 0x7f8f50111780 2024-07-31T15:06:35.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.106+0000 7f8f5ccde740 1 -- 172.21.15.67:0/1142632168 shutdown_connections 2024-07-31T15:06:35.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.110+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/3333641395 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8f380b3e60 0x7f8f380b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:35.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.110+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/3333641395 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8f380b3e60 0x7f8f380b6370 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8f40011c50 tx=0x7f8f40011ab0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:35.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.111+0000 7f8f5ccde740 1 -- 172.21.15.67:0/1142632168 wait complete. 2024-07-31T15:06:35.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.111+0000 7f8f5ccde740 1 -- start start 2024-07-31T15:06:35.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.111+0000 7f8f5ccde740 1 Processor -- start 2024-07-31T15:06:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.112+0000 7f8f5ccde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d986698b60 0x55d986a6b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.112+0000 7f8f5ccde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9866b9840 0x55d986a6b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.112+0000 7f8f5ccde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866781c0 0x55d986a6ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.112+0000 7f8f5ccde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d9868277a0 con 0x55d9866781c0 2024-07-31T15:06:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.112+0000 7f8f5ccde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d9869662f0 con 0x55d986698b60 2024-07-31T15:06:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.112+0000 7f8f5ccde740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d986a6e210 con 0x55d9866b9840 2024-07-31T15:06:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.112+0000 7f8f54d2a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866781c0 0x55d986a6ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.112+0000 7f8f54d2a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866781c0 0x55d986a6ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37604/0 (socket says 172.21.15.67:37604) 2024-07-31T15:06:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.112+0000 7f8f54d2a640 1 -- 172.21.15.67:0/139363757 learned_addr learned my addr 172.21.15.67:0/139363757 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.112+0000 7f8f5ccde740 10 client.0 _ll_register_callbacks cb 0x55d986970110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.113+0000 7f8f4f7fe640 1 --2- 172.21.15.67:0/139363757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9866b9840 0x55d986a6b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.114+0000 7f8f54d2a640 1 -- 172.21.15.67:0/139363757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9866b9840 msgr2=0x55d986a6b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.114+0000 7f8f54d2a640 1 --2- 172.21.15.67:0/139363757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9866b9840 0x55d986a6b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.114+0000 7f8f54d2a640 1 -- 172.21.15.67:0/139363757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d986698b60 msgr2=0x55d986a6b000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:06:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.114+0000 7f8f54d2a640 1 --2- 172.21.15.67:0/139363757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d986698b60 0x55d986a6b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.114+0000 7f8f54d2a640 1 -- 172.21.15.67:0/139363757 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d98696a480 con 0x55d9866781c0 2024-07-31T15:06:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.114+0000 7f8f54d2a640 1 --2- 172.21.15.67:0/139363757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866781c0 0x55d986a6ba80 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8f48011710 tx=0x7f8f48011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d0adc5e140f37a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:35.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.114+0000 7f8f2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:06:35.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.115+0000 7f8f2f7fe640 1 -- 172.21.15.67:0/139363757 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4800fb40 con 0x55d9866781c0 2024-07-31T15:06:35.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.115+0000 7f8f5ccde740 1 -- 172.21.15.67:0/139363757 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f40009e30 con 0x55d9866781c0 2024-07-31T15:06:35.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.115+0000 7f8f2f7fe640 1 -- 172.21.15.67:0/139363757 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f8f4800fce0 con 0x55d9866781c0 2024-07-31T15:06:35.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.115+0000 7f8f2f7fe640 1 -- 172.21.15.67:0/139363757 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f8f48021550 con 0x55d9866781c0 2024-07-31T15:06:35.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.119+0000 7f8f5ccde740 10 client.18783 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:35.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.119+0000 7f8f5ccde740 1 -- 172.21.15.67:0/139363757 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55d986733b90 con 0x55d9866781c0 2024-07-31T15:06:35.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.119+0000 7f8f2f7fe640 1 -- 172.21.15.67:0/139363757 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f40009e30 con 0x55d9866781c0 2024-07-31T15:06:35.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.119+0000 7f8f2f7fe640 1 -- 172.21.15.67:0/139363757 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f8f4800f040 con 0x55d9866781c0 2024-07-31T15:06:35.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.119+0000 7f8f5ccde740 10 client.18783 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:35.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.119+0000 7f8f5ccde740 10 client.18783 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:06:35.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.119+0000 7f8f5ccde740 20 client.18783 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:35.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.120+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/139363757 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d98664d570 0x55d986a9f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.120+0000 7f8f5ccde740 4 client.18783 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:06:35.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.120+0000 7f8f5ccde740 1 -- 172.21.15.67:0/139363757 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9869d66d0 con 0x55d98664d570 2024-07-31T15:06:35.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.123+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/139363757 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d98664d570 0x55d986a9f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:35.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.142+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/139363757 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d98664d570 0x55d986a9f790 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1f38a527030a503 server_cookie=eee655326c313f49 in_seq=0 out_seq=0 2024-07-31T15:06:35.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.143+0000 7f8f2f7fe640 10 client.18783 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.146+0000 7f8f2f7fe640 1 -- 172.21.15.67:0/139363757 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9869d66d0 con 0x55d98664d570 2024-07-31T15:06:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.146+0000 7f8f2f7fe640 10 client.18783 handle_command_reply: tid=0 2024-07-31T15:06:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.146+0000 7f8f5ccde740 1 client.18783 shutdown 2024-07-31T15:06:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.146+0000 7f8f5ccde740 1 -- 172.21.15.67:0/139363757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866781c0 msgr2=0x55d986a6ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.146+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/139363757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866781c0 0x55d986a6ba80 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8f48011710 tx=0x7f8f48011bd0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.146+0000 7f8f5ccde740 1 -- 172.21.15.67:0/139363757 shutdown_connections 2024-07-31T15:06:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.146+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/139363757 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d98664d570 0x55d986a9f790 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.146+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/139363757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866781c0 0x55d986a6ba80 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.146+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/139363757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9866b9840 0x55d986a6b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.146+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/139363757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d986698b60 0x55d986a6b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.146+0000 7f8f5ccde740 1 -- 172.21.15.67:0/139363757 >> 172.21.15.67:0/139363757 conn(0x55d9866ad990 msgr2=0x55d986a62bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.146+0000 7f8f2f7fe640 0 client.18783 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:35.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.148+0000 7f8f5ccde740 1 -- 172.21.15.67:0/139363757 shutdown_connections 2024-07-31T15:06:35.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.151+0000 7f8f5ccde740 1 -- 172.21.15.67:0/139363757 wait complete. 2024-07-31T15:06:35.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.151+0000 7f8f5ccde740 20 client.18783 trim_cache size 0 max 16384 2024-07-31T15:06:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.190+0000 7f8f5ccde740 1 Processor -- start 2024-07-31T15:06:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.190+0000 7f8f5ccde740 1 -- start start 2024-07-31T15:06:35.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.190+0000 7f8f5ccde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866781c0 0x55d986a634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.190+0000 7f8f5ccde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d986698b60 0x55d986a6cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.190+0000 7f8f5ccde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866b9840 0x55d986aa5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.190+0000 7f8f5ccde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c00ffb0 con 0x55d9866b9840 2024-07-31T15:06:35.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.190+0000 7f8f5ccde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c011af0 con 0x55d9866781c0 2024-07-31T15:06:35.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.190+0000 7f8f5ccde740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c010680 con 0x55d986698b60 2024-07-31T15:06:35.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.190+0000 7f8f4ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866781c0 0x55d986a634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:35.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.191+0000 7f8f54d2a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866b9840 0x55d986aa5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:35.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.191+0000 7f8f4ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866781c0 0x55d986a634f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:51364/0 (socket says 172.21.15.67:51364) 2024-07-31T15:06:35.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.191+0000 7f8f4ffff640 1 -- 172.21.15.67:0/1524385728 learned_addr learned my addr 172.21.15.67:0/1524385728 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:35.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.191+0000 7f8f4ffff640 1 -- 172.21.15.67:0/1524385728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d986698b60 msgr2=0x55d986a6cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:35.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.191+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/1524385728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d986698b60 0x55d986a6cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.191+0000 7f8f4ffff640 1 -- 172.21.15.67:0/1524385728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866b9840 msgr2=0x55d986aa5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:35.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.191+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/1524385728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866b9840 0x55d986aa5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.191+0000 7f8f4ffff640 1 -- 172.21.15.67:0/1524385728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d986a61970 con 0x55d9866781c0 2024-07-31T15:06:35.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.191+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/1524385728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866781c0 0x55d986a634f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8f40007fd0 tx=0x7f8f40020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c0fda21b936df74 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:35.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.192+0000 7f8f35ffb640 1 -- 172.21.15.67:0/1524385728 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40015040 con 0x55d9866781c0 2024-07-31T15:06:35.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.192+0000 7f8f35ffb640 1 -- 172.21.15.67:0/1524385728 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f4000bd60 con 0x55d9866781c0 2024-07-31T15:06:35.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.193+0000 7f8f5ccde740 1 -- 172.21.15.67:0/1524385728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866781c0 msgr2=0x55d986a634f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:35.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.193+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/1524385728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866781c0 0x55d986a634f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8f40007fd0 tx=0x7f8f40020000 comp rx=0 tx=0).stop 2024-07-31T15:06:35.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.194+0000 7f8f5ccde740 1 -- 172.21.15.67:0/1524385728 shutdown_connections 2024-07-31T15:06:35.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.194+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/1524385728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866b9840 0x55d986aa5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.194+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/1524385728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d986698b60 0x55d986a6cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.194+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/1524385728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866781c0 0x55d986a634f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.194+0000 7f8f5ccde740 1 -- 172.21.15.67:0/1524385728 >> 172.21.15.67:0/1524385728 conn(0x55d9866ad990 msgr2=0x55d986a6c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:35.198 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/kfifo.o 2024-07-31T15:06:35.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.199+0000 7f8f5ccde740 1 -- 172.21.15.67:0/1524385728 shutdown_connections 2024-07-31T15:06:35.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.199+0000 7f8f5ccde740 1 -- 172.21.15.67:0/1524385728 wait complete. 2024-07-31T15:06:35.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.199+0000 7f8f5ccde740 1 -- start start 2024-07-31T15:06:35.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.200+0000 7f8f5ccde740 1 Processor -- start 2024-07-31T15:06:35.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.200+0000 7f8f5ccde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866781c0 0x55d986a6f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.200+0000 7f8f5ccde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d986698b60 0x55d986a6fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.200+0000 7f8f5ccde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866b9840 0x55d986a6fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.200+0000 7f8f5ccde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c011420 con 0x55d9866b9840 2024-07-31T15:06:35.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.200+0000 7f8f5ccde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c010d50 con 0x55d9866781c0 2024-07-31T15:06:35.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.200+0000 7f8f5ccde740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c00dfb0 con 0x55d986698b60 2024-07-31T15:06:35.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.200+0000 7f8f5ccde740 10 client.0 _ll_register_callbacks cb 0x7f8f48009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:35.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.201+0000 7f8f54d2a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866b9840 0x55d986a6fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:35.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.201+0000 7f8f54d2a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866b9840 0x55d986a6fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37620/0 (socket says 172.21.15.67:37620) 2024-07-31T15:06:35.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.201+0000 7f8f54d2a640 1 -- 172.21.15.67:0/2851273649 learned_addr learned my addr 172.21.15.67:0/2851273649 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:35.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.201+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/2851273649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866781c0 0x55d986a6f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:35.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.201+0000 7f8f4ffff640 1 -- 172.21.15.67:0/2851273649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d986698b60 msgr2=0x55d986a6fab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:06:35.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.201+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/2851273649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d986698b60 0x55d986a6fab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.201+0000 7f8f4ffff640 1 -- 172.21.15.67:0/2851273649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866b9840 msgr2=0x55d986a6fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:35.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.201+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/2851273649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866b9840 0x55d986a6fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:35.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.201+0000 7f8f4ffff640 1 -- 172.21.15.67:0/2851273649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d986b07160 con 0x55d9866781c0 2024-07-31T15:06:35.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.201+0000 7f8f4ffff640 1 --2- 172.21.15.67:0/2851273649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866781c0 0x55d986a6f570 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009010 tx=0x7f8f4000bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85332f7d321b054e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:35.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.201+0000 7f8f2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:06:35.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.203+0000 7f8f2f7fe640 1 -- 172.21.15.67:0/2851273649 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4003d020 con 0x55d9866781c0 2024-07-31T15:06:35.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.203+0000 7f8f2f7fe640 1 -- 172.21.15.67:0/2851273649 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f8f40009e30 con 0x55d9866781c0 2024-07-31T15:06:35.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.204+0000 7f8f5ccde740 1 -- 172.21.15.67:0/2851273649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f4000bd60 con 0x55d9866781c0 2024-07-31T15:06:35.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.204+0000 7f8f2f7fe640 1 -- 172.21.15.67:0/2851273649 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f8f40015040 con 0x55d9866781c0 2024-07-31T15:06:35.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.204+0000 7f8f5ccde740 10 client.28313 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:35.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.204+0000 7f8f5ccde740 1 -- 172.21.15.67:0/2851273649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55d986b083c0 con 0x55d9866781c0 2024-07-31T15:06:35.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.205+0000 7f8f2f7fe640 1 -- 172.21.15.67:0/2851273649 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f4000bd60 con 0x55d9866781c0 2024-07-31T15:06:35.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.205+0000 7f8f2f7fe640 1 -- 172.21.15.67:0/2851273649 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f8f40038050 con 0x55d9866781c0 2024-07-31T15:06:35.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.208+0000 7f8f5ccde740 10 client.28313 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:35.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.208+0000 7f8f5ccde740 10 client.28313 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:06:35.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.208+0000 7f8f5ccde740 20 client.28313 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:35.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.208+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/2851273649 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d986b088e0 0x55d986b0ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:35.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.208+0000 7f8f5ccde740 4 client.28313 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}] 2024-07-31T15:06:35.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.208+0000 7f8f5ccde740 1 -- 172.21.15.67:0/2851273649 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}) -- 0x55d9869d2170 con 0x55d986b088e0 2024-07-31T15:06:35.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.214+0000 7f8f4f7fe640 1 --2- 172.21.15.67:0/2851273649 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d986b088e0 0x55d986b0ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:35.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.218+0000 7f8f4f7fe640 1 --2- 172.21.15.67:0/2851273649 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d986b088e0 0x55d986b0ad00 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dda5a9aed7bb0d6e server_cookie=e20b6c44afb8134d in_seq=0 out_seq=0 2024-07-31T15:06:35.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:35.218+0000 7f8f2f7fe640 10 client.28313 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:35.250 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethernet/eth.o 2024-07-31T15:06:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:35 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:06:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:35 smithi077 ceph-mon[37327]: pgmap v890: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 229 KiB/s wr, 2.16k op/s 2024-07-31T15:06:35.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:35 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:06:35.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:35 smithi073 ceph-mon[36714]: pgmap v890: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 229 KiB/s wr, 2.16k op/s 2024-07-31T15:06:35.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:35 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:06:35.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:35 smithi067 ceph-mon[32750]: pgmap v890: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 229 KiB/s wr, 2.16k op/s 2024-07-31T15:06:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:37 smithi077 ceph-mon[37327]: pgmap v891: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 437 KiB/s wr, 792 op/s 2024-07-31T15:06:37.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:37 smithi067 ceph-mon[32750]: pgmap v891: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 437 KiB/s wr, 792 op/s 2024-07-31T15:06:37.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:37 smithi073 ceph-mon[36714]: pgmap v891: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 437 KiB/s wr, 792 op/s 2024-07-31T15:06:39.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:39 smithi077 ceph-mon[37327]: pgmap v892: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 445 KiB/s wr, 206 op/s 2024-07-31T15:06:39.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:39 smithi067 ceph-mon[32750]: pgmap v892: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 445 KiB/s wr, 206 op/s 2024-07-31T15:06:39.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:39 smithi073 ceph-mon[36714]: pgmap v892: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 445 KiB/s wr, 206 op/s 2024-07-31T15:06:40.587 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:06:40.869 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:06:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:41 smithi067 ceph-mon[32750]: pgmap v893: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 505 KiB/s wr, 109 op/s 2024-07-31T15:06:41.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:41 smithi077 ceph-mon[37327]: pgmap v893: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 505 KiB/s wr, 109 op/s 2024-07-31T15:06:41.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:41 smithi073 ceph-mon[36714]: pgmap v893: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 505 KiB/s wr, 109 op/s 2024-07-31T15:06:42.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.645+0000 7f9a61f77640 1 -- 172.21.15.67:0/2627684152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c107dd0 msgr2=0x7f9a5c1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:42.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.645+0000 7f9a61f77640 1 --2- 172.21.15.67:0/2627684152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c107dd0 0x7f9a5c1081e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009900 tx=0x7f9a5002a410 comp rx=0 tx=0).stop 2024-07-31T15:06:42.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.645+0000 7f9a61f77640 1 -- 172.21.15.67:0/2627684152 shutdown_connections 2024-07-31T15:06:42.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.645+0000 7f9a61f77640 1 --2- 172.21.15.67:0/2627684152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a5c108720 0x7f9a5c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.645+0000 7f9a61f77640 1 --2- 172.21.15.67:0/2627684152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c107dd0 0x7f9a5c1081e0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.645+0000 7f9a61f77640 1 --2- 172.21.15.67:0/2627684152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a5c102490 0x7f9a5c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.645+0000 7f9a61f77640 1 -- 172.21.15.67:0/2627684152 >> 172.21.15.67:0/2627684152 conn(0x7f9a5c079160 msgr2=0x7f9a5c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.645+0000 7f9a61f77640 1 -- 172.21.15.67:0/2627684152 shutdown_connections 2024-07-31T15:06:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.645+0000 7f9a61f77640 1 -- 172.21.15.67:0/2627684152 wait complete. 2024-07-31T15:06:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.646+0000 7f9a61f77640 1 Processor -- start 2024-07-31T15:06:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.646+0000 7f9a61f77640 1 -- start start 2024-07-31T15:06:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.646+0000 7f9a61f77640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a5c102490 0x7f9a5c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.646+0000 7f9a61f77640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c107dd0 0x7f9a5c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.646+0000 7f9a61f77640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a5c108720 0x7f9a5c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.646+0000 7f9a61f77640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c1134b0 con 0x7f9a5c107dd0 2024-07-31T15:06:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.646+0000 7f9a61f77640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c113330 con 0x7f9a5c102490 2024-07-31T15:06:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.646+0000 7f9a61f77640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c113630 con 0x7f9a5c108720 2024-07-31T15:06:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.647+0000 7f9a5b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a5c102490 0x7f9a5c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.647+0000 7f9a5b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a5c102490 0x7f9a5c199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44384/0 (socket says 172.21.15.67:44384) 2024-07-31T15:06:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.647+0000 7f9a5b7fe640 1 -- 172.21.15.67:0/1643062788 learned_addr learned my addr 172.21.15.67:0/1643062788 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.647+0000 7f9a5affd640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c107dd0 0x7f9a5c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.647+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a5c108720 0x7f9a5c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.647+0000 7f9a5b7fe640 1 -- 172.21.15.67:0/1643062788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a5c108720 msgr2=0x7f9a5c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.647+0000 7f9a5b7fe640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a5c108720 0x7f9a5c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.647+0000 7f9a5b7fe640 1 -- 172.21.15.67:0/1643062788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c107dd0 msgr2=0x7f9a5c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.647+0000 7f9a5b7fe640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c107dd0 0x7f9a5c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.647+0000 7f9a5b7fe640 1 -- 172.21.15.67:0/1643062788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5c19ff10 con 0x7f9a5c102490 2024-07-31T15:06:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.647+0000 7f9a5affd640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c107dd0 0x7f9a5c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.647+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a5c108720 0x7f9a5c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:06:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.647+0000 7f9a5b7fe640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a5c102490 0x7f9a5c199fa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00e9f0 tx=0x7f9a4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.647+0000 7f9a58ff9640 1 -- 172.21.15.67:0/1643062788 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c00cd80 con 0x7f9a5c102490 2024-07-31T15:06:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.648+0000 7f9a61f77640 1 -- 172.21.15.67:0/1643062788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5c1a0200 con 0x7f9a5c102490 2024-07-31T15:06:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.648+0000 7f9a58ff9640 1 -- 172.21.15.67:0/1643062788 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a4c0108e0 con 0x7f9a5c102490 2024-07-31T15:06:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.648+0000 7f9a58ff9640 1 -- 172.21.15.67:0/1643062788 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c018980 con 0x7f9a5c102490 2024-07-31T15:06:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.648+0000 7f9a61f77640 1 -- 172.21.15.67:0/1643062788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a5c1a6960 con 0x7f9a5c102490 2024-07-31T15:06:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.650+0000 7f9a684c8740 1 Processor -- start 2024-07-31T15:06:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.650+0000 7f9a684c8740 1 -- start start 2024-07-31T15:06:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a684c8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d41d070 0x55b76d7c2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a684c8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d3f41b0 0x55b76d7c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a684c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 0x55b76d7c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a684c8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b76d738b60 con 0x55b76d3f41b0 2024-07-31T15:06:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a684c8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b76d728060 con 0x55b76d41d070 2024-07-31T15:06:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a684c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b76d72c0a0 con 0x55b76d3fc390 2024-07-31T15:06:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a5bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d41d070 0x55b76d7c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a5bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d41d070 0x55b76d7c2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44400/0 (socket says 172.21.15.67:44400) 2024-07-31T15:06:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a5bfff640 1 -- 172.21.15.67:0/1943953095 learned_addr learned my addr 172.21.15.67:0/1943953095 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a5affd640 1 --2- 172.21.15.67:0/1943953095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 0x55b76d7c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a5bfff640 1 -- 172.21.15.67:0/1943953095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 msgr2=0x55b76d7c2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/1943953095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 0x55b76d7c2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a5bfff640 1 -- 172.21.15.67:0/1943953095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d3f41b0 msgr2=0x55b76d7c2a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:06:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/1943953095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d3f41b0 0x55b76d7c2a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a5bfff640 1 -- 172.21.15.67:0/1943953095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b76d6ceb40 con 0x55b76d41d070 2024-07-31T15:06:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a58ff9640 1 -- 172.21.15.67:0/1643062788 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f9a4c010430 con 0x7f9a5c102490 2024-07-31T15:06:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.651+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/1943953095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d41d070 0x55b76d7c2520 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400ae80 tx=0x7f9a5400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58ce857b5a0229e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.652+0000 7f9a41ffb640 1 -- 172.21.15.67:0/1943953095 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54003b70 con 0x55b76d41d070 2024-07-31T15:06:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.652+0000 7f9a41ffb640 1 -- 172.21.15.67:0/1943953095 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a54003d10 con 0x55b76d41d070 2024-07-31T15:06:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.652+0000 7f9a41ffb640 1 -- 172.21.15.67:0/1943953095 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54010a80 con 0x55b76d41d070 2024-07-31T15:06:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.652+0000 7f9a58ff9640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a440b3d40 0x7f9a440b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.652+0000 7f9a5b7fe640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a440b3d40 0x7f9a440b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.652+0000 7f9a684c8740 1 -- 172.21.15.67:0/1943953095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d41d070 msgr2=0x55b76d7c2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.652+0000 7f9a684c8740 1 --2- 172.21.15.67:0/1943953095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d41d070 0x55b76d7c2520 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400ae80 tx=0x7f9a5400e720 comp rx=0 tx=0).stop 2024-07-31T15:06:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.652+0000 7f9a58ff9640 1 -- 172.21.15.67:0/1643062788 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f9a4c0e4700 con 0x7f9a5c102490 2024-07-31T15:06:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.653+0000 7f9a684c8740 1 -- 172.21.15.67:0/1943953095 shutdown_connections 2024-07-31T15:06:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.653+0000 7f9a684c8740 1 --2- 172.21.15.67:0/1943953095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 0x55b76d7c2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.653+0000 7f9a684c8740 1 --2- 172.21.15.67:0/1943953095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d3f41b0 0x55b76d7c2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.653+0000 7f9a684c8740 1 --2- 172.21.15.67:0/1943953095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d41d070 0x55b76d7c2520 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.653+0000 7f9a684c8740 1 -- 172.21.15.67:0/1943953095 >> 172.21.15.67:0/1943953095 conn(0x55b76d411460 msgr2=0x55b76d7c0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.653+0000 7f9a684c8740 1 -- 172.21.15.67:0/1943953095 shutdown_connections 2024-07-31T15:06:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.653+0000 7f9a684c8740 1 -- 172.21.15.67:0/1943953095 wait complete. 2024-07-31T15:06:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.653+0000 7f9a5b7fe640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a440b3d40 0x7f9a440b6250 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55b76d7c11a0 tx=0x7f9a4c09f850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.653+0000 7f9a684c8740 1 -- start start 2024-07-31T15:06:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.653+0000 7f9a684c8740 1 Processor -- start 2024-07-31T15:06:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.653+0000 7f9a684c8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d3f41b0 0x55b76d7ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a684c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 0x55b76d7ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a684c8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d41d070 0x55b76d7d9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a684c8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b76d58b700 con 0x55b76d41d070 2024-07-31T15:06:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a684c8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b76d6ca990 con 0x55b76d3f41b0 2024-07-31T15:06:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a684c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b76d7d0d00 con 0x55b76d3fc390 2024-07-31T15:06:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a684c8740 10 client.0 _ll_register_callbacks cb 0x55b76d6e54a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a5bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d3f41b0 0x55b76d7ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a5bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d3f41b0 0x55b76d7ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44414/0 (socket says 172.21.15.67:44414) 2024-07-31T15:06:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a5bfff640 1 -- 172.21.15.67:0/3186188076 learned_addr learned my addr 172.21.15.67:0/3186188076 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/3186188076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d41d070 0x55b76d7d9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a5affd640 1 --2- 172.21.15.67:0/3186188076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 0x55b76d7ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a5bfff640 1 -- 172.21.15.67:0/3186188076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 msgr2=0x55b76d7ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/3186188076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 0x55b76d7ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a5bfff640 1 -- 172.21.15.67:0/3186188076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d41d070 msgr2=0x55b76d7d9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/3186188076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d41d070 0x55b76d7d9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a5bfff640 1 -- 172.21.15.67:0/3186188076 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b76d488650 con 0x55b76d3f41b0 2024-07-31T15:06:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/3186188076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d3f41b0 0x55b76d7ce1c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400afd0 tx=0x7f9a540106e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c402f4deac2bfd5d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a3affd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:06:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a3affd640 1 -- 172.21.15.67:0/3186188076 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5401e020 con 0x55b76d3f41b0 2024-07-31T15:06:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a684c8740 1 -- 172.21.15.67:0/3186188076 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a54003d10 con 0x55b76d3f41b0 2024-07-31T15:06:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.654+0000 7f9a3affd640 1 -- 172.21.15.67:0/3186188076 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f9a540040c0 con 0x55b76d3f41b0 2024-07-31T15:06:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.655+0000 7f9a3affd640 1 -- 172.21.15.67:0/3186188076 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f9a54033050 con 0x55b76d3f41b0 2024-07-31T15:06:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.655+0000 7f9a684c8740 10 client.28325 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.655+0000 7f9a684c8740 1 -- 172.21.15.67:0/3186188076 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55b76d7dcd00 con 0x55b76d3f41b0 2024-07-31T15:06:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.655+0000 7f9a3affd640 1 -- 172.21.15.67:0/3186188076 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a54003d10 con 0x55b76d3f41b0 2024-07-31T15:06:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.655+0000 7f9a3affd640 1 -- 172.21.15.67:0/3186188076 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f9a5401a960 con 0x55b76d3f41b0 2024-07-31T15:06:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.655+0000 7f9a684c8740 10 client.28325 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.655+0000 7f9a684c8740 10 client.28325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.655+0000 7f9a684c8740 20 client.28325 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.655+0000 7f9a684c8740 1 --2- 172.21.15.67:0/3186188076 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b76d3db9f0 0x55b76d7e00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.655+0000 7f9a684c8740 4 client.28325 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:06:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.655+0000 7f9a684c8740 1 -- 172.21.15.67:0/3186188076 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b76d73aab0 con 0x55b76d3db9f0 2024-07-31T15:06:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.655+0000 7f9a5affd640 1 --2- 172.21.15.67:0/3186188076 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b76d3db9f0 0x55b76d7e00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.656+0000 7f9a5affd640 1 --2- 172.21.15.67:0/3186188076 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b76d3db9f0 0x55b76d7e00c0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9bc5b7fb34810c6 server_cookie=ebd652d0d858a776 in_seq=0 out_seq=0 2024-07-31T15:06:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.656+0000 7f9a3affd640 10 client.28325 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:42.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a3affd640 1 -- 172.21.15.67:0/3186188076 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b76d73aab0 con 0x55b76d3db9f0 2024-07-31T15:06:42.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a3affd640 10 client.28325 handle_command_reply: tid=0 2024-07-31T15:06:42.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a684c8740 1 client.28325 shutdown 2024-07-31T15:06:42.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a684c8740 1 -- 172.21.15.67:0/3186188076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d3f41b0 msgr2=0x55b76d7ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:42.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a684c8740 1 --2- 172.21.15.67:0/3186188076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d3f41b0 0x55b76d7ce1c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400afd0 tx=0x7f9a540106e0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a684c8740 1 -- 172.21.15.67:0/3186188076 shutdown_connections 2024-07-31T15:06:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a684c8740 1 --2- 172.21.15.67:0/3186188076 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b76d3db9f0 0x55b76d7e00c0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a684c8740 1 --2- 172.21.15.67:0/3186188076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d41d070 0x55b76d7d9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a684c8740 1 --2- 172.21.15.67:0/3186188076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 0x55b76d7ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a3affd640 0 client.28325 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a684c8740 1 --2- 172.21.15.67:0/3186188076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d3f41b0 0x55b76d7ce1c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a684c8740 1 -- 172.21.15.67:0/3186188076 >> 172.21.15.67:0/3186188076 conn(0x55b76d411460 msgr2=0x55b76d7bebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a684c8740 1 -- 172.21.15.67:0/3186188076 shutdown_connections 2024-07-31T15:06:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a684c8740 1 -- 172.21.15.67:0/3186188076 wait complete. 2024-07-31T15:06:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:42.980+0000 7f9a684c8740 20 client.28325 trim_cache size 0 max 16384 2024-07-31T15:06:43.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.000+0000 7f9a684c8740 1 Processor -- start 2024-07-31T15:06:43.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.000+0000 7f9a684c8740 1 -- start start 2024-07-31T15:06:43.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.000+0000 7f9a684c8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d3f41b0 0x55b76d7d9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.000+0000 7f9a684c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 0x55b76d7bfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a684c8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d41d070 0x55b76d7c0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a684c8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c1b0840 con 0x55b76d3f41b0 2024-07-31T15:06:43.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a684c8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c1b2380 con 0x55b76d41d070 2024-07-31T15:06:43.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a684c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c1b0f10 con 0x55b76d3fc390 2024-07-31T15:06:43.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a5bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d3f41b0 0x55b76d7d9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a5bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d3f41b0 0x55b76d7d9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55412/0 (socket says 172.21.15.67:55412) 2024-07-31T15:06:43.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a5bfff640 1 -- 172.21.15.67:0/289064418 learned_addr learned my addr 172.21.15.67:0/289064418 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:43.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a5affd640 1 --2- 172.21.15.67:0/289064418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 0x55b76d7bfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/289064418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d41d070 0x55b76d7c0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a5bfff640 1 -- 172.21.15.67:0/289064418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 msgr2=0x55b76d7bfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:43.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/289064418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 0x55b76d7bfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a5bfff640 1 -- 172.21.15.67:0/289064418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d41d070 msgr2=0x55b76d7c0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:43.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/289064418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d41d070 0x55b76d7c0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a5bfff640 1 -- 172.21.15.67:0/289064418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b76d7c4810 con 0x55b76d3f41b0 2024-07-31T15:06:43.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/289064418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d3f41b0 0x55b76d7d9de0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f9a54000f80 tx=0x7f9a54005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856136fdcd3ff222 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:43.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a417fa640 1 -- 172.21.15.67:0/289064418 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5401e030 con 0x55b76d3f41b0 2024-07-31T15:06:43.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.001+0000 7f9a417fa640 1 -- 172.21.15.67:0/289064418 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a5402d0b0 con 0x55b76d3f41b0 2024-07-31T15:06:43.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.002+0000 7f9a417fa640 1 -- 172.21.15.67:0/289064418 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54024040 con 0x55b76d3f41b0 2024-07-31T15:06:43.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.002+0000 7f9a684c8740 1 -- 172.21.15.67:0/289064418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d3f41b0 msgr2=0x55b76d7d9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:43.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.002+0000 7f9a684c8740 1 --2- 172.21.15.67:0/289064418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d3f41b0 0x55b76d7d9de0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f9a54000f80 tx=0x7f9a54005dd0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.002+0000 7f9a684c8740 1 -- 172.21.15.67:0/289064418 shutdown_connections 2024-07-31T15:06:43.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.002+0000 7f9a684c8740 1 --2- 172.21.15.67:0/289064418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d41d070 0x55b76d7c0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.002+0000 7f9a684c8740 1 --2- 172.21.15.67:0/289064418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3fc390 0x55b76d7bfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.002+0000 7f9a684c8740 1 --2- 172.21.15.67:0/289064418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d3f41b0 0x55b76d7d9de0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.002+0000 7f9a684c8740 1 -- 172.21.15.67:0/289064418 >> 172.21.15.67:0/289064418 conn(0x55b76d411460 msgr2=0x55b76d7cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:43.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.002+0000 7f9a684c8740 1 -- 172.21.15.67:0/289064418 shutdown_connections 2024-07-31T15:06:43.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.003+0000 7f9a684c8740 1 -- 172.21.15.67:0/289064418 wait complete. 2024-07-31T15:06:43.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.003+0000 7f9a684c8740 1 -- start start 2024-07-31T15:06:43.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.003+0000 7f9a684c8740 1 Processor -- start 2024-07-31T15:06:43.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.003+0000 7f9a684c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3f41b0 0x55b76d7d6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.003+0000 7f9a684c8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d3fc390 0x55b76d7d1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.003+0000 7f9a684c8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d41d070 0x55b76d7d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.003+0000 7f9a684c8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c1b1cb0 con 0x55b76d41d070 2024-07-31T15:06:43.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.003+0000 7f9a684c8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c1b15e0 con 0x55b76d3fc390 2024-07-31T15:06:43.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.003+0000 7f9a684c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c1ae840 con 0x55b76d3f41b0 2024-07-31T15:06:43.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.003+0000 7f9a684c8740 10 client.0 _ll_register_callbacks cb 0x7f9a54014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:43.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.003+0000 7f9a5bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d41d070 0x55b76d7d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.003+0000 7f9a5affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d3fc390 0x55b76d7d1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.003+0000 7f9a5bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d41d070 0x55b76d7d2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55424/0 (socket says 172.21.15.67:55424) 2024-07-31T15:06:43.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.003+0000 7f9a5bfff640 1 -- 172.21.15.67:0/1975648788 learned_addr learned my addr 172.21.15.67:0/1975648788 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:43.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.004+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/1975648788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3f41b0 0x55b76d7d6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.004+0000 7f9a5affd640 1 -- 172.21.15.67:0/1975648788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3f41b0 msgr2=0x55b76d7d6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:43.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.004+0000 7f9a5affd640 1 --2- 172.21.15.67:0/1975648788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3f41b0 0x55b76d7d6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.004+0000 7f9a5affd640 1 -- 172.21.15.67:0/1975648788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d41d070 msgr2=0x55b76d7d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:43.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.004+0000 7f9a5affd640 1 --2- 172.21.15.67:0/1975648788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d41d070 0x55b76d7d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.004+0000 7f9a5affd640 1 -- 172.21.15.67:0/1975648788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b76d7d27c0 con 0x55b76d3fc390 2024-07-31T15:06:43.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.004+0000 7f9a5affd640 1 --2- 172.21.15.67:0/1975648788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d3fc390 0x55b76d7d1b10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9a50006010 tx=0x7f9a500042d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=656dba90f831d133 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:43.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.004+0000 7f9a3affd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:06:43.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.004+0000 7f9a3affd640 1 -- 172.21.15.67:0/1975648788 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50035040 con 0x55b76d3fc390 2024-07-31T15:06:43.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.004+0000 7f9a3affd640 1 -- 172.21.15.67:0/1975648788 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f9a50005230 con 0x55b76d3fc390 2024-07-31T15:06:43.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.004+0000 7f9a684c8740 1 -- 172.21.15.67:0/1975648788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a5402d0b0 con 0x55b76d3fc390 2024-07-31T15:06:43.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.004+0000 7f9a3affd640 1 -- 172.21.15.67:0/1975648788 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f9a50052050 con 0x55b76d3fc390 2024-07-31T15:06:43.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.004+0000 7f9a684c8740 10 client.28331 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:43.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.005+0000 7f9a684c8740 1 -- 172.21.15.67:0/1975648788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55b76d8486d0 con 0x55b76d3fc390 2024-07-31T15:06:43.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.005+0000 7f9a3affd640 1 -- 172.21.15.67:0/1975648788 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a5402d0b0 con 0x55b76d3fc390 2024-07-31T15:06:43.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.005+0000 7f9a3affd640 1 -- 172.21.15.67:0/1975648788 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f9a50031880 con 0x55b76d3fc390 2024-07-31T15:06:43.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.005+0000 7f9a684c8740 10 client.28331 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:43.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.005+0000 7f9a684c8740 10 client.28331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:43.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.005+0000 7f9a684c8740 20 client.28331 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:43.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.005+0000 7f9a684c8740 1 --2- 172.21.15.67:0/1975648788 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b76d3b0da0 0x55b76d84af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.005+0000 7f9a684c8740 4 client.28331 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:06:43.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.005+0000 7f9a684c8740 1 -- 172.21.15.67:0/1975648788 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b76d736550 con 0x55b76d3b0da0 2024-07-31T15:06:43.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.005+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/1975648788 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b76d3b0da0 0x55b76d84af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.006+0000 7f9a5bfff640 1 --2- 172.21.15.67:0/1975648788 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b76d3b0da0 0x55b76d84af10 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=489e035bd9661c08 server_cookie=81dcfed2a049b119 in_seq=0 out_seq=0 2024-07-31T15:06:43.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.006+0000 7f9a3affd640 10 client.28331 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:43.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.149+0000 7f9a3affd640 1 -- 172.21.15.67:0/1975648788 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b76d736550 con 0x55b76d3b0da0 2024-07-31T15:06:43.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.149+0000 7f9a3affd640 10 client.28331 handle_command_reply: tid=0 2024-07-31T15:06:43.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.150+0000 7f9a684c8740 1 client.28331 shutdown 2024-07-31T15:06:43.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.150+0000 7f9a684c8740 1 -- 172.21.15.67:0/1975648788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d3fc390 msgr2=0x55b76d7d1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:43.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.150+0000 7f9a684c8740 1 --2- 172.21.15.67:0/1975648788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d3fc390 0x55b76d7d1b10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9a50006010 tx=0x7f9a500042d0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.150+0000 7f9a684c8740 1 -- 172.21.15.67:0/1975648788 shutdown_connections 2024-07-31T15:06:43.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.150+0000 7f9a684c8740 1 --2- 172.21.15.67:0/1975648788 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b76d3b0da0 0x55b76d84af10 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.150+0000 7f9a684c8740 1 --2- 172.21.15.67:0/1975648788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b76d41d070 0x55b76d7d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.150+0000 7f9a684c8740 1 --2- 172.21.15.67:0/1975648788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b76d3fc390 0x55b76d7d1b10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.150+0000 7f9a3affd640 0 client.28331 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:43.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.150+0000 7f9a684c8740 1 --2- 172.21.15.67:0/1975648788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b76d3f41b0 0x55b76d7d6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.150+0000 7f9a684c8740 1 -- 172.21.15.67:0/1975648788 >> 172.21.15.67:0/1975648788 conn(0x55b76d411460 msgr2=0x55b76d7c7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:43.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.150+0000 7f9a684c8740 1 -- 172.21.15.67:0/1975648788 shutdown_connections 2024-07-31T15:06:43.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.150+0000 7f9a684c8740 1 -- 172.21.15.67:0/1975648788 wait complete. 2024-07-31T15:06:43.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.150+0000 7f9a684c8740 20 client.28331 trim_cache size 0 max 16384 2024-07-31T15:06:43.153 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:06:43.154 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T15:06:43.154 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T15:06:43.154 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:06:43.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.151+0000 7f9a61f77640 1 -- 172.21.15.67:0/1643062788 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a440b3d40 msgr2=0x7f9a440b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:43.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.151+0000 7f9a61f77640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a440b3d40 0x7f9a440b6250 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55b76d7c11a0 tx=0x7f9a4c09f850 comp rx=0 tx=0).stop 2024-07-31T15:06:43.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.151+0000 7f9a61f77640 1 -- 172.21.15.67:0/1643062788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a5c102490 msgr2=0x7f9a5c199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:43.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.151+0000 7f9a61f77640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a5c102490 0x7f9a5c199fa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00e9f0 tx=0x7f9a4c00eeb0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.151+0000 7f9a61f77640 1 -- 172.21.15.67:0/1643062788 shutdown_connections 2024-07-31T15:06:43.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.151+0000 7f9a61f77640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f9a440b3d40 0x7f9a440b6250 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.151+0000 7f9a61f77640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a5c108720 0x7f9a5c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.151+0000 7f9a61f77640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c107dd0 0x7f9a5c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.151+0000 7f9a61f77640 1 --2- 172.21.15.67:0/1643062788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a5c102490 0x7f9a5c199fa0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.151+0000 7f9a61f77640 1 -- 172.21.15.67:0/1643062788 >> 172.21.15.67:0/1643062788 conn(0x7f9a5c079160 msgr2=0x7f9a5c104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:43.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.151+0000 7f9a61f77640 1 -- 172.21.15.67:0/1643062788 shutdown_connections 2024-07-31T15:06:43.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:43.151+0000 7f9a61f77640 1 -- 172.21.15.67:0/1643062788 wait complete. 2024-07-31T15:06:43.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:43 smithi067 ceph-mon[32750]: pgmap v894: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 419 KiB/s wr, 104 op/s 2024-07-31T15:06:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.616+0000 7fc381966640 1 -- 172.21.15.67:0/1979994256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37c10f590 msgr2=0x7fc37c10f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.616+0000 7fc381966640 1 --2- 172.21.15.67:0/1979994256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37c10f590 0x7fc37c10f9a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fc370009900 tx=0x7fc37002a410 comp rx=0 tx=0).stop 2024-07-31T15:06:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.616+0000 7fc381966640 1 -- 172.21.15.67:0/1979994256 shutdown_connections 2024-07-31T15:06:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.616+0000 7fc381966640 1 --2- 172.21.15.67:0/1979994256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc37c10fee0 0x7fc37c116710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.616+0000 7fc381966640 1 --2- 172.21.15.67:0/1979994256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37c10f590 0x7fc37c10f9a0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.616+0000 7fc381966640 1 --2- 172.21.15.67:0/1979994256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc37c109b50 0x7fc37c109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.616+0000 7fc381966640 1 -- 172.21.15.67:0/1979994256 >> 172.21.15.67:0/1979994256 conn(0x7fc37c079200 msgr2=0x7fc37c107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.616+0000 7fc381966640 1 -- 172.21.15.67:0/1979994256 shutdown_connections 2024-07-31T15:06:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.616+0000 7fc381966640 1 -- 172.21.15.67:0/1979994256 wait complete. 2024-07-31T15:06:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.617+0000 7fc381966640 1 Processor -- start 2024-07-31T15:06:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.617+0000 7fc381966640 1 -- start start 2024-07-31T15:06:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.617+0000 7fc381966640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc37c109b50 0x7fc37c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc381966640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc37c10f590 0x7fc37c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc381966640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37c10fee0 0x7fc37c1a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc381966640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c119ba0 con 0x7fc37c10fee0 2024-07-31T15:06:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc381966640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c119a20 con 0x7fc37c109b50 2024-07-31T15:06:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc381966640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c119d20 con 0x7fc37c10f590 2024-07-31T15:06:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37c10fee0 0x7fc37c1a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc37affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc37c109b50 0x7fc37c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37c10fee0 0x7fc37c1a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55444/0 (socket says 172.21.15.67:55444) 2024-07-31T15:06:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc37b7fe640 1 -- 172.21.15.67:0/2658551384 learned_addr learned my addr 172.21.15.67:0/2658551384 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc37affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc37c109b50 0x7fc37c1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44444/0 (socket says 172.21.15.67:44444) 2024-07-31T15:06:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc37a7fc640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc37c10f590 0x7fc37c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc37affd640 1 -- 172.21.15.67:0/2658551384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc37c10f590 msgr2=0x7fc37c1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc37affd640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc37c10f590 0x7fc37c1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc37affd640 1 -- 172.21.15.67:0/2658551384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37c10fee0 msgr2=0x7fc37c1a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc37affd640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37c10fee0 0x7fc37c1a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc37affd640 1 -- 172.21.15.67:0/2658551384 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c1a6550 con 0x7fc37c109b50 2024-07-31T15:06:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc37b7fe640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37c10fee0 0x7fc37c1a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc37a7fc640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc37c10f590 0x7fc37c1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:06:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc37affd640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc37c109b50 0x7fc37c1a0620 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc36c00e9f0 tx=0x7fc36c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc380964640 1 -- 172.21.15.67:0/2658551384 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c00ccc0 con 0x7fc37c109b50 2024-07-31T15:06:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc381966640 1 -- 172.21.15.67:0/2658551384 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37c1a6840 con 0x7fc37c109b50 2024-07-31T15:06:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc380964640 1 -- 172.21.15.67:0/2658551384 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc36c0108e0 con 0x7fc37c109b50 2024-07-31T15:06:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.618+0000 7fc380964640 1 -- 172.21.15.67:0/2658551384 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c018880 con 0x7fc37c109b50 2024-07-31T15:06:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.619+0000 7fc381966640 1 -- 172.21.15.67:0/2658551384 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc37c1acfd0 con 0x7fc37c109b50 2024-07-31T15:06:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.621+0000 7fc380964640 1 -- 172.21.15.67:0/2658551384 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc36c010430 con 0x7fc37c109b50 2024-07-31T15:06:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.622+0000 7fc380964640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3640b3d40 0x7fc3640b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.622+0000 7fc38885c740 1 Processor -- start 2024-07-31T15:06:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.622+0000 7fc38885c740 1 -- start start 2024-07-31T15:06:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.622+0000 7fc37a7fc640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3640b3d40 0x7fc3640b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.622+0000 7fc380964640 1 -- 172.21.15.67:0/2658551384 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7fc36c0e4770 con 0x7fc37c109b50 2024-07-31T15:06:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc38885c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14af1b0 0x5610e1851ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc38885c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14862f0 0x5610e1852220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc38885c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e148e4d0 0x5610e1852760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc38885c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5610e17c73b0 con 0x5610e14af1b0 2024-07-31T15:06:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc38885c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5610e17b68b0 con 0x5610e14862f0 2024-07-31T15:06:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc38885c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5610e17ba8f0 con 0x5610e148e4d0 2024-07-31T15:06:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc37affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14af1b0 0x5610e1851ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc37a7fc640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3640b3d40 0x7fc3640b6250 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc370009230 tx=0x7fc370000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc37affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14af1b0 0x5610e1851ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55448/0 (socket says 172.21.15.67:55448) 2024-07-31T15:06:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc37affd640 1 -- 172.21.15.67:0/3393289372 learned_addr learned my addr 172.21.15.67:0/3393289372 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc37a7fc640 1 --2- 172.21.15.67:0/3393289372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14862f0 0x5610e1852220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc37b7fe640 1 --2- 172.21.15.67:0/3393289372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e148e4d0 0x5610e1852760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc37affd640 1 -- 172.21.15.67:0/3393289372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e148e4d0 msgr2=0x5610e1852760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc37affd640 1 --2- 172.21.15.67:0/3393289372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e148e4d0 0x5610e1852760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc37affd640 1 -- 172.21.15.67:0/3393289372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14862f0 msgr2=0x5610e1852220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc37affd640 1 --2- 172.21.15.67:0/3393289372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14862f0 0x5610e1852220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc37affd640 1 -- 172.21.15.67:0/3393289372 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e175d9a0 con 0x5610e14af1b0 2024-07-31T15:06:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc37a7fc640 1 --2- 172.21.15.67:0/3393289372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14862f0 0x5610e1852220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.623+0000 7fc37affd640 1 --2- 172.21.15.67:0/3393289372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14af1b0 0x5610e1851ce0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0a70c0 tx=0x7fc36c0aa8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b42d02f7214cb3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.624+0000 7fc361ffb640 1 -- 172.21.15.67:0/3393289372 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0b40f0 con 0x5610e14af1b0 2024-07-31T15:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.624+0000 7fc361ffb640 1 -- 172.21.15.67:0/3393289372 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc36c010a90 con 0x5610e14af1b0 2024-07-31T15:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.624+0000 7fc38885c740 1 -- 172.21.15.67:0/3393289372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14af1b0 msgr2=0x5610e1851ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.624+0000 7fc38885c740 1 --2- 172.21.15.67:0/3393289372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14af1b0 0x5610e1851ce0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0a70c0 tx=0x7fc36c0aa8c0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.625+0000 7fc38885c740 1 -- 172.21.15.67:0/3393289372 shutdown_connections 2024-07-31T15:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.625+0000 7fc38885c740 1 --2- 172.21.15.67:0/3393289372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e148e4d0 0x5610e1852760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.625+0000 7fc38885c740 1 --2- 172.21.15.67:0/3393289372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14862f0 0x5610e1852220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.625+0000 7fc38885c740 1 --2- 172.21.15.67:0/3393289372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14af1b0 0x5610e1851ce0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.625+0000 7fc38885c740 1 -- 172.21.15.67:0/3393289372 >> 172.21.15.67:0/3393289372 conn(0x5610e14a3300 msgr2=0x5610e184fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.625+0000 7fc38885c740 1 -- 172.21.15.67:0/3393289372 shutdown_connections 2024-07-31T15:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.625+0000 7fc38885c740 1 -- 172.21.15.67:0/3393289372 wait complete. 2024-07-31T15:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.625+0000 7fc38885c740 1 -- start start 2024-07-31T15:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.625+0000 7fc38885c740 1 Processor -- start 2024-07-31T15:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc38885c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e14862f0 0x5610e185d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc38885c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e148e4d0 0x5610e185dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc38885c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14af1b0 0x5610e1869870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc38885c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5610e16f0ab0 con 0x5610e148e4d0 2024-07-31T15:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc38885c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5610e173ecf0 con 0x5610e14af1b0 2024-07-31T15:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc38885c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5610e1860500 con 0x5610e14862f0 2024-07-31T15:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc37a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e148e4d0 0x5610e185dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc37affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e14862f0 0x5610e185d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14af1b0 0x5610e1869870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14af1b0 0x5610e1869870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44476/0 (socket says 172.21.15.67:44476) 2024-07-31T15:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc37a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e148e4d0 0x5610e185dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55454/0 (socket says 172.21.15.67:55454) 2024-07-31T15:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc37b7fe640 1 -- 172.21.15.67:0/1545825996 learned_addr learned my addr 172.21.15.67:0/1545825996 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc37affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e14862f0 0x5610e185d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38544/0 (socket says 172.21.15.67:38544) 2024-07-31T15:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc38885c740 10 client.0 _ll_register_callbacks cb 0x5610e17b09f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc37b7fe640 1 -- 172.21.15.67:0/1545825996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e14862f0 msgr2=0x5610e185d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc37b7fe640 1 --2- 172.21.15.67:0/1545825996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e14862f0 0x5610e185d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc37b7fe640 1 -- 172.21.15.67:0/1545825996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e148e4d0 msgr2=0x5610e185dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc37b7fe640 1 --2- 172.21.15.67:0/1545825996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e148e4d0 0x5610e185dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc37b7fe640 1 -- 172.21.15.67:0/1545825996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e151a790 con 0x5610e14af1b0 2024-07-31T15:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc37b7fe640 1 --2- 172.21.15.67:0/1545825996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14af1b0 0x5610e1869870 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc374010c40 tx=0x7fc374011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a21083bfcc33c748 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.626+0000 7fc35affd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc35affd640 1 -- 172.21.15.67:0/1545825996 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374004b20 con 0x5610e14af1b0 2024-07-31T15:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc35affd640 1 -- 172.21.15.67:0/1545825996 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7fc374004cc0 con 0x5610e14af1b0 2024-07-31T15:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc38885c740 1 -- 172.21.15.67:0/1545825996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc36c010a90 con 0x5610e14af1b0 2024-07-31T15:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc35affd640 1 -- 172.21.15.67:0/1545825996 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fc374024050 con 0x5610e14af1b0 2024-07-31T15:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc38885c740 10 client.28343 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc38885c740 1 -- 172.21.15.67:0/1545825996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5610e1593620 con 0x5610e14af1b0 2024-07-31T15:06:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc35affd640 1 -- 172.21.15.67:0/1545825996 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc36c010a90 con 0x5610e14af1b0 2024-07-31T15:06:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc35affd640 1 -- 172.21.15.67:0/1545825996 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fc37400f450 con 0x5610e14af1b0 2024-07-31T15:06:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc38885c740 10 client.28343 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc38885c740 10 client.28343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc38885c740 20 client.28343 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc38885c740 1 --2- 172.21.15.67:0/1545825996 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5610e146dd80 0x5610e186f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc38885c740 4 client.28343 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:06:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc38885c740 1 -- 172.21.15.67:0/1545825996 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610e17c9300 con 0x5610e146dd80 2024-07-31T15:06:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.627+0000 7fc37affd640 1 --2- 172.21.15.67:0/1545825996 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5610e146dd80 0x5610e186f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.628+0000 7fc35affd640 10 client.28343 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.628+0000 7fc37affd640 1 --2- 172.21.15.67:0/1545825996 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5610e146dd80 0x5610e186f820 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a434baba72e5751 server_cookie=1ca5e3d7ab8bc37a in_seq=0 out_seq=0 2024-07-31T15:06:43.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:43 smithi073 ceph-mon[36714]: pgmap v894: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 419 KiB/s wr, 104 op/s 2024-07-31T15:06:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.649+0000 7fc35affd640 1 -- 172.21.15.67:0/1545825996 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610e17c9300 con 0x5610e146dd80 2024-07-31T15:06:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.649+0000 7fc35affd640 10 client.28343 handle_command_reply: tid=0 2024-07-31T15:06:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.649+0000 7fc38885c740 1 client.28343 shutdown 2024-07-31T15:06:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.650+0000 7fc38885c740 1 -- 172.21.15.67:0/1545825996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14af1b0 msgr2=0x5610e1869870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.650+0000 7fc38885c740 1 --2- 172.21.15.67:0/1545825996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14af1b0 0x5610e1869870 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc374010c40 tx=0x7fc374011810 comp rx=0 tx=0).stop 2024-07-31T15:06:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.650+0000 7fc38885c740 1 -- 172.21.15.67:0/1545825996 shutdown_connections 2024-07-31T15:06:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.650+0000 7fc38885c740 1 --2- 172.21.15.67:0/1545825996 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5610e146dd80 0x5610e186f820 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.650+0000 7fc38885c740 1 --2- 172.21.15.67:0/1545825996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14af1b0 0x5610e1869870 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.650+0000 7fc38885c740 1 --2- 172.21.15.67:0/1545825996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e148e4d0 0x5610e185dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.650+0000 7fc38885c740 1 --2- 172.21.15.67:0/1545825996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e14862f0 0x5610e185d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.650+0000 7fc35affd640 0 client.28343 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.650+0000 7fc38885c740 1 -- 172.21.15.67:0/1545825996 >> 172.21.15.67:0/1545825996 conn(0x5610e14a3300 msgr2=0x5610e1852d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.650+0000 7fc38885c740 1 -- 172.21.15.67:0/1545825996 shutdown_connections 2024-07-31T15:06:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.650+0000 7fc38885c740 1 -- 172.21.15.67:0/1545825996 wait complete. 2024-07-31T15:06:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.650+0000 7fc38885c740 20 client.28343 trim_cache size 0 max 16384 2024-07-31T15:06:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.669+0000 7fc38885c740 1 Processor -- start 2024-07-31T15:06:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.669+0000 7fc38885c740 1 -- start start 2024-07-31T15:06:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.669+0000 7fc38885c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14862f0 0x5610e185f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.669+0000 7fc38885c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e148e4d0 0x5610e1869660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.669+0000 7fc38885c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e14af1b0 0x5610e1869ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.669+0000 7fc38885c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1b6d90 con 0x5610e14862f0 2024-07-31T15:06:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.669+0000 7fc38885c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1b88d0 con 0x5610e148e4d0 2024-07-31T15:06:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.669+0000 7fc38885c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1b7460 con 0x5610e14af1b0 2024-07-31T15:06:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.669+0000 7fc37a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e148e4d0 0x5610e1869660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.669+0000 7fc37affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14862f0 0x5610e185f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.669+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e14af1b0 0x5610e1869ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.669+0000 7fc37a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e148e4d0 0x5610e1869660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44486/0 (socket says 172.21.15.67:44486) 2024-07-31T15:06:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.669+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e14af1b0 0x5610e1869ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38546/0 (socket says 172.21.15.67:38546) 2024-07-31T15:06:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.669+0000 7fc37a7fc640 1 -- 172.21.15.67:0/3072959590 learned_addr learned my addr 172.21.15.67:0/3072959590 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.670+0000 7fc37affd640 1 -- 172.21.15.67:0/3072959590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e14af1b0 msgr2=0x5610e1869ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.670+0000 7fc37affd640 1 --2- 172.21.15.67:0/3072959590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e14af1b0 0x5610e1869ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.670+0000 7fc37affd640 1 -- 172.21.15.67:0/3072959590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e148e4d0 msgr2=0x5610e1869660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.670+0000 7fc37affd640 1 --2- 172.21.15.67:0/3072959590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e148e4d0 0x5610e1869660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.670+0000 7fc37affd640 1 -- 172.21.15.67:0/3072959590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e185c460 con 0x5610e14862f0 2024-07-31T15:06:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.670+0000 7fc37a7fc640 1 --2- 172.21.15.67:0/3072959590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e148e4d0 0x5610e1869660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.670+0000 7fc37affd640 1 --2- 172.21.15.67:0/3072959590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14862f0 0x5610e185f150 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0e2180 tx=0x7fc36c0af7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8bf6842b9e2e68 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.670+0000 7fc3617fa640 1 -- 172.21.15.67:0/3072959590 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c010720 con 0x5610e14862f0 2024-07-31T15:06:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.670+0000 7fc3617fa640 1 -- 172.21.15.67:0/3072959590 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc36c0b40f0 con 0x5610e14862f0 2024-07-31T15:06:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.670+0000 7fc3617fa640 1 -- 172.21.15.67:0/3072959590 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0aa2a0 con 0x5610e14862f0 2024-07-31T15:06:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.671+0000 7fc38885c740 1 -- 172.21.15.67:0/3072959590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14862f0 msgr2=0x5610e185f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.671+0000 7fc38885c740 1 --2- 172.21.15.67:0/3072959590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14862f0 0x5610e185f150 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0e2180 tx=0x7fc36c0af7c0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.671+0000 7fc38885c740 1 -- 172.21.15.67:0/3072959590 shutdown_connections 2024-07-31T15:06:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.671+0000 7fc38885c740 1 --2- 172.21.15.67:0/3072959590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e14af1b0 0x5610e1869ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.671+0000 7fc38885c740 1 --2- 172.21.15.67:0/3072959590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e148e4d0 0x5610e1869660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.671+0000 7fc38885c740 1 --2- 172.21.15.67:0/3072959590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14862f0 0x5610e185f150 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.671+0000 7fc38885c740 1 -- 172.21.15.67:0/3072959590 >> 172.21.15.67:0/3072959590 conn(0x5610e14a3300 msgr2=0x5610e185ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.672+0000 7fc38885c740 1 -- 172.21.15.67:0/3072959590 shutdown_connections 2024-07-31T15:06:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.672+0000 7fc38885c740 1 -- 172.21.15.67:0/3072959590 wait complete. 2024-07-31T15:06:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.672+0000 7fc38885c740 1 -- start start 2024-07-31T15:06:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.672+0000 7fc38885c740 1 Processor -- start 2024-07-31T15:06:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc38885c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14862f0 0x5610e18628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc38885c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e148e4d0 0x5610e1862df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc38885c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14af1b0 0x5610e1863350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc38885c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1b8200 con 0x5610e14af1b0 2024-07-31T15:06:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc38885c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1b7b30 con 0x5610e14862f0 2024-07-31T15:06:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc38885c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1b4d90 con 0x5610e148e4d0 2024-07-31T15:06:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc38885c740 10 client.0 _ll_register_callbacks cb 0x7fc374010510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc37affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14862f0 0x5610e18628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14af1b0 0x5610e1863350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc37affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14862f0 0x5610e18628b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44488/0 (socket says 172.21.15.67:44488) 2024-07-31T15:06:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc37affd640 1 -- 172.21.15.67:0/2157323533 learned_addr learned my addr 172.21.15.67:0/2157323533 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc37a7fc640 1 --2- 172.21.15.67:0/2157323533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e148e4d0 0x5610e1862df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc37b7fe640 1 -- 172.21.15.67:0/2157323533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e148e4d0 msgr2=0x5610e1862df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc37b7fe640 1 --2- 172.21.15.67:0/2157323533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e148e4d0 0x5610e1862df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc37b7fe640 1 -- 172.21.15.67:0/2157323533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14862f0 msgr2=0x5610e18628b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc37b7fe640 1 --2- 172.21.15.67:0/2157323533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14862f0 0x5610e18628b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.673+0000 7fc37b7fe640 1 -- 172.21.15.67:0/2157323533 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e18d6270 con 0x5610e14af1b0 2024-07-31T15:06:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.674+0000 7fc37b7fe640 1 --2- 172.21.15.67:0/2157323533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14af1b0 0x5610e1863350 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc37400fa10 tx=0x7fc374009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02ce05ec3cbe84e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.674+0000 7fc35affd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:06:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.674+0000 7fc35affd640 1 -- 172.21.15.67:0/2157323533 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37400f5d0 con 0x5610e14af1b0 2024-07-31T15:06:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.674+0000 7fc38885c740 1 -- 172.21.15.67:0/2157323533 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc36c0b40f0 con 0x5610e14af1b0 2024-07-31T15:06:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.674+0000 7fc35affd640 1 -- 172.21.15.67:0/2157323533 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7fc374017040 con 0x5610e14af1b0 2024-07-31T15:06:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.674+0000 7fc35affd640 1 -- 172.21.15.67:0/2157323533 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fc374038050 con 0x5610e14af1b0 2024-07-31T15:06:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.674+0000 7fc38885c740 10 client.18822 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.674+0000 7fc38885c740 1 -- 172.21.15.67:0/2157323533 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5610e18d7500 con 0x5610e14af1b0 2024-07-31T15:06:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.674+0000 7fc35affd640 1 -- 172.21.15.67:0/2157323533 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc36c0b40f0 con 0x5610e14af1b0 2024-07-31T15:06:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.675+0000 7fc35affd640 1 -- 172.21.15.67:0/2157323533 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fc374003b70 con 0x5610e14af1b0 2024-07-31T15:06:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.675+0000 7fc38885c740 10 client.18822 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.675+0000 7fc38885c740 10 client.18822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.675+0000 7fc38885c740 20 client.18822 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.675+0000 7fc38885c740 1 --2- 172.21.15.67:0/2157323533 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5610e14431c0 0x5610e18d9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.675+0000 7fc38885c740 4 client.18822 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:06:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.675+0000 7fc38885c740 1 -- 172.21.15.67:0/2157323533 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610e17c4da0 con 0x5610e14431c0 2024-07-31T15:06:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.675+0000 7fc37affd640 1 --2- 172.21.15.67:0/2157323533 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5610e14431c0 0x5610e18d9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.675+0000 7fc35affd640 10 client.18822 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:43.675+0000 7fc37affd640 1 --2- 172.21.15.67:0/2157323533 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5610e14431c0 0x5610e18d9d40 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd8bbb1beb5ca7a server_cookie=631c9f3f4bc4f31e in_seq=0 out_seq=0 2024-07-31T15:06:43.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:43 smithi077 ceph-mon[37327]: pgmap v894: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 419 KiB/s wr, 104 op/s 2024-07-31T15:06:44.025 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T15:06:44.025 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:06:44.522 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:06:45.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:45 smithi067 ceph-mon[32750]: pgmap v895: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 372 KiB/s wr, 85 op/s 2024-07-31T15:06:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:45 smithi073 ceph-mon[36714]: pgmap v895: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 372 KiB/s wr, 85 op/s 2024-07-31T15:06:45.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:45 smithi077 ceph-mon[37327]: pgmap v895: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 372 KiB/s wr, 85 op/s 2024-07-31T15:06:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.316+0000 7fc35affd640 1 -- 172.21.15.67:0/2157323533 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5610e17c4da0 con 0x5610e14431c0 2024-07-31T15:06:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.317+0000 7fc35affd640 10 client.18822 handle_command_reply: tid=0 2024-07-31T15:06:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.317+0000 7fc38885c740 1 client.18822 shutdown 2024-07-31T15:06:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.317+0000 7fc38885c740 1 -- 172.21.15.67:0/2157323533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14af1b0 msgr2=0x5610e1863350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.317+0000 7fc38885c740 1 --2- 172.21.15.67:0/2157323533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14af1b0 0x5610e1863350 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc37400fa10 tx=0x7fc374009920 comp rx=0 tx=0).stop 2024-07-31T15:06:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.317+0000 7fc38885c740 1 -- 172.21.15.67:0/2157323533 shutdown_connections 2024-07-31T15:06:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.317+0000 7fc38885c740 1 --2- 172.21.15.67:0/2157323533 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5610e14431c0 0x5610e18d9d40 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.317+0000 7fc38885c740 1 --2- 172.21.15.67:0/2157323533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5610e14af1b0 0x5610e1863350 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.317+0000 7fc38885c740 1 --2- 172.21.15.67:0/2157323533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610e148e4d0 0x5610e1862df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.317+0000 7fc35affd640 0 client.18822 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.317+0000 7fc38885c740 1 --2- 172.21.15.67:0/2157323533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5610e14862f0 0x5610e18628b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.317+0000 7fc38885c740 1 -- 172.21.15.67:0/2157323533 >> 172.21.15.67:0/2157323533 conn(0x5610e14a3300 msgr2=0x5610e185e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.317+0000 7fc38885c740 1 -- 172.21.15.67:0/2157323533 shutdown_connections 2024-07-31T15:06:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.317+0000 7fc38885c740 1 -- 172.21.15.67:0/2157323533 wait complete. 2024-07-31T15:06:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.317+0000 7fc38885c740 20 client.18822 trim_cache size 0 max 16384 2024-07-31T15:06:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:06:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:06:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:06:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:06:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.318+0000 7fc381966640 1 -- 172.21.15.67:0/2658551384 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3640b3d40 msgr2=0x7fc3640b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.318+0000 7fc381966640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3640b3d40 0x7fc3640b6250 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc370009230 tx=0x7fc370000bc0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.318+0000 7fc381966640 1 -- 172.21.15.67:0/2658551384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc37c109b50 msgr2=0x7fc37c1a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.318+0000 7fc381966640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc37c109b50 0x7fc37c1a0620 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc36c00e9f0 tx=0x7fc36c00eeb0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.318+0000 7fc381966640 1 -- 172.21.15.67:0/2658551384 shutdown_connections 2024-07-31T15:06:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.318+0000 7fc381966640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc3640b3d40 0x7fc3640b6250 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.318+0000 7fc381966640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37c10fee0 0x7fc37c1a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.318+0000 7fc381966640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc37c10f590 0x7fc37c1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.318+0000 7fc381966640 1 --2- 172.21.15.67:0/2658551384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc37c109b50 0x7fc37c1a0620 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.318+0000 7fc381966640 1 -- 172.21.15.67:0/2658551384 >> 172.21.15.67:0/2658551384 conn(0x7fc37c079200 msgr2=0x7fc37c079b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.318+0000 7fc381966640 1 -- 172.21.15.67:0/2658551384 shutdown_connections 2024-07-31T15:06:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:46.318+0000 7fc381966640 1 -- 172.21.15.67:0/2658551384 wait complete. 2024-07-31T15:06:46.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.354+0000 7f0bc1241640 1 -- 172.21.15.67:0/2630944621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc108720 msgr2=0x7f0bbc10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:46.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.354+0000 7f0bc1241640 1 --2- 172.21.15.67:0/2630944621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc108720 0x7f0bbc10ef50 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009900 tx=0x7f0bac02a410 comp rx=0 tx=0).stop 2024-07-31T15:06:46.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.354+0000 7f0bc1241640 1 -- 172.21.15.67:0/2630944621 shutdown_connections 2024-07-31T15:06:46.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.354+0000 7f0bc1241640 1 --2- 172.21.15.67:0/2630944621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc108720 0x7f0bbc10ef50 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.354+0000 7f0bc1241640 1 --2- 172.21.15.67:0/2630944621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0bbc107dd0 0x7f0bbc1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.354+0000 7f0bc1241640 1 --2- 172.21.15.67:0/2630944621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bbc102490 0x7f0bbc1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.354+0000 7f0bc1241640 1 -- 172.21.15.67:0/2630944621 >> 172.21.15.67:0/2630944621 conn(0x7f0bbc079160 msgr2=0x7f0bbc079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:46.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.354+0000 7f0bc1241640 1 -- 172.21.15.67:0/2630944621 shutdown_connections 2024-07-31T15:06:46.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.354+0000 7f0bc1241640 1 -- 172.21.15.67:0/2630944621 wait complete. 2024-07-31T15:06:46.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.355+0000 7f0bc1241640 1 Processor -- start 2024-07-31T15:06:46.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.355+0000 7f0bc1241640 1 -- start start 2024-07-31T15:06:46.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.355+0000 7f0bc1241640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc102490 0x7f0bbc19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:46.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.355+0000 7f0bc1241640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0bbc107dd0 0x7f0bbc19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:46.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.355+0000 7f0bc1241640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bbc108720 0x7f0bbc1a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:46.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.355+0000 7f0bc1241640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc112380 con 0x7f0bbc102490 2024-07-31T15:06:46.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.355+0000 7f0bc1241640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc112200 con 0x7f0bbc107dd0 2024-07-31T15:06:46.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.355+0000 7f0bc1241640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc112500 con 0x7f0bbc108720 2024-07-31T15:06:46.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.355+0000 7f0bbad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc102490 0x7f0bbc19c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:46.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.355+0000 7f0bbad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc102490 0x7f0bbc19c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55508/0 (socket says 172.21.15.67:55508) 2024-07-31T15:06:46.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.355+0000 7f0bba59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0bbc107dd0 0x7f0bbc19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:46.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.356+0000 7f0bbad9d640 1 -- 172.21.15.67:0/1109785069 learned_addr learned my addr 172.21.15.67:0/1109785069 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:46.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.356+0000 7f0bbb59e640 1 --2- 172.21.15.67:0/1109785069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bbc108720 0x7f0bbc1a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:46.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.356+0000 7f0bbad9d640 1 -- 172.21.15.67:0/1109785069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bbc108720 msgr2=0x7f0bbc1a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:46.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.356+0000 7f0bbad9d640 1 --2- 172.21.15.67:0/1109785069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bbc108720 0x7f0bbc1a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.356+0000 7f0bbad9d640 1 -- 172.21.15.67:0/1109785069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0bbc107dd0 msgr2=0x7f0bbc19c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:46.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.356+0000 7f0bbad9d640 1 --2- 172.21.15.67:0/1109785069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0bbc107dd0 0x7f0bbc19c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.356+0000 7f0bbad9d640 1 -- 172.21.15.67:0/1109785069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc1a2040 con 0x7f0bbc102490 2024-07-31T15:06:46.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.356+0000 7f0bba59c640 1 --2- 172.21.15.67:0/1109785069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0bbc107dd0 0x7f0bbc19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:46.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.356+0000 7f0bbad9d640 1 --2- 172.21.15.67:0/1109785069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc102490 0x7f0bbc19c0d0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400cce0 tx=0x7f0bb400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:46.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.356+0000 7f0bb3fff640 1 -- 172.21.15.67:0/1109785069 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4004b20 con 0x7f0bbc102490 2024-07-31T15:06:46.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.356+0000 7f0bb3fff640 1 -- 172.21.15.67:0/1109785069 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb4004cc0 con 0x7f0bbc102490 2024-07-31T15:06:46.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.356+0000 7f0bc1241640 1 -- 172.21.15.67:0/1109785069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bbc1a2330 con 0x7f0bbc102490 2024-07-31T15:06:46.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.356+0000 7f0bb3fff640 1 -- 172.21.15.67:0/1109785069 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400f6b0 con 0x7f0bbc102490 2024-07-31T15:06:46.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.356+0000 7f0bc1241640 1 -- 172.21.15.67:0/1109785069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bbc1a8a90 con 0x7f0bbc102490 2024-07-31T15:06:46.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.358+0000 7f0bc1241640 1 -- 172.21.15.67:0/1109785069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bbc1028a0 con 0x7f0bbc102490 2024-07-31T15:06:46.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.359+0000 7f0bb3fff640 1 -- 172.21.15.67:0/1109785069 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0bb400f850 con 0x7f0bbc102490 2024-07-31T15:06:46.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.360+0000 7f0bb3fff640 1 --2- 172.21.15.67:0/1109785069 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0ba80b3d40 0x7f0ba80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:46.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.360+0000 7f0bba59c640 1 --2- 172.21.15.67:0/1109785069 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0ba80b3d40 0x7f0ba80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:46.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.360+0000 7f0bb3fff640 1 -- 172.21.15.67:0/1109785069 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f0bb40e4320 con 0x7f0bbc102490 2024-07-31T15:06:46.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.360+0000 7f0bba59c640 1 --2- 172.21.15.67:0/1109785069 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0ba80b3d40 0x7f0ba80b6250 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc079680 tx=0x7f0ba4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:46.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.364+0000 7f0bb3fff640 1 -- 172.21.15.67:0/1109785069 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f0bb4014030 con 0x7f0bbc102490 2024-07-31T15:06:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.599+0000 7f0bc1241640 1 -- 172.21.15.67:0/1109785069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bbc19d200 con 0x7f0bbc102490 2024-07-31T15:06:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.599+0000 7f0bb3fff640 1 -- 172.21.15.67:0/1109785069 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11962 (secure 0 0 0) 0x7f0bb40aaca0 con 0x7f0bbc102490 2024-07-31T15:06:46.601 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:06:46.601 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"btime":"2024-07-31T15:06:04:323642+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:06:04.323492+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28214":{"gid":28214,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1516494761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1516494761},{"type":"v1","addr":"172.21.15.73:6835","nonce":1516494761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:06:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-07-31T15:06:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.602+0000 7f0bc1241640 1 -- 172.21.15.67:0/1109785069 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0ba80b3d40 msgr2=0x7f0ba80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.602+0000 7f0bc1241640 1 --2- 172.21.15.67:0/1109785069 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0ba80b3d40 0x7f0ba80b6250 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc079680 tx=0x7f0ba4005f90 comp rx=0 tx=0).stop 2024-07-31T15:06:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.602+0000 7f0bc1241640 1 -- 172.21.15.67:0/1109785069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc102490 msgr2=0x7f0bbc19c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.602+0000 7f0bc1241640 1 --2- 172.21.15.67:0/1109785069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc102490 0x7f0bbc19c0d0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400cce0 tx=0x7f0bb400a620 comp rx=0 tx=0).stop 2024-07-31T15:06:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.602+0000 7f0bc1241640 1 -- 172.21.15.67:0/1109785069 shutdown_connections 2024-07-31T15:06:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.602+0000 7f0bc1241640 1 --2- 172.21.15.67:0/1109785069 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0ba80b3d40 0x7f0ba80b6250 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.602+0000 7f0bc1241640 1 --2- 172.21.15.67:0/1109785069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bbc108720 0x7f0bbc1a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.602+0000 7f0bc1241640 1 --2- 172.21.15.67:0/1109785069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0bbc107dd0 0x7f0bbc19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.602+0000 7f0bc1241640 1 --2- 172.21.15.67:0/1109785069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc102490 0x7f0bbc19c0d0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.602+0000 7f0bc1241640 1 -- 172.21.15.67:0/1109785069 >> 172.21.15.67:0/1109785069 conn(0x7f0bbc079160 msgr2=0x7f0bbc103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.602+0000 7f0bc1241640 1 -- 172.21.15.67:0/1109785069 shutdown_connections 2024-07-31T15:06:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:46.602+0000 7f0bc1241640 1 -- 172.21.15.67:0/1109785069 wait complete. 2024-07-31T15:06:47.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T15:06:47.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:47 smithi067 ceph-mon[32750]: pgmap v896: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 500 KiB/s wr, 89 op/s 2024-07-31T15:06:47.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:47 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1109785069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:06:47.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:47 smithi073 ceph-mon[36714]: pgmap v896: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 500 KiB/s wr, 89 op/s 2024-07-31T15:06:47.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:47 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1109785069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:06:47.677 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:06:47.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:47 smithi077 ceph-mon[37327]: pgmap v896: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 500 KiB/s wr, 89 op/s 2024-07-31T15:06:47.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:47 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1109785069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:06:49.429 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:49 smithi067 ceph-mon[32750]: pgmap v897: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 460 KiB/s wr, 88 op/s 2024-07-31T15:06:49.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:49 smithi073 ceph-mon[36714]: pgmap v897: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 460 KiB/s wr, 88 op/s 2024-07-31T15:06:49.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.676+0000 7f679baec640 1 -- 172.21.15.67:0/3023428764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6794102490 msgr2=0x7f67941028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:49.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.676+0000 7f679baec640 1 --2- 172.21.15.67:0/3023428764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6794102490 0x7f67941028a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6784009ca0 tx=0x7f678402a8c0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.676+0000 7f679baec640 1 -- 172.21.15.67:0/3023428764 shutdown_connections 2024-07-31T15:06:49.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.676+0000 7f679baec640 1 --2- 172.21.15.67:0/3023428764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6794108720 0x7f679410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.676+0000 7f679baec640 1 --2- 172.21.15.67:0/3023428764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6794107dd0 0x7f67941081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.676+0000 7f679baec640 1 --2- 172.21.15.67:0/3023428764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6794102490 0x7f67941028a0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.676+0000 7f679baec640 1 -- 172.21.15.67:0/3023428764 >> 172.21.15.67:0/3023428764 conn(0x7f6794079160 msgr2=0x7f6794079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:49.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.676+0000 7f679baec640 1 -- 172.21.15.67:0/3023428764 shutdown_connections 2024-07-31T15:06:49.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.676+0000 7f679baec640 1 -- 172.21.15.67:0/3023428764 wait complete. 2024-07-31T15:06:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.676+0000 7f679baec640 1 Processor -- start 2024-07-31T15:06:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f679baec640 1 -- start start 2024-07-31T15:06:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f679baec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6794102490 0x7f6794103c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f679baec640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6794107dd0 0x7f6794104140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f679baec640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6794108720 0x7f6794104680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f679baec640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f67941123d0 con 0x7f6794107dd0 2024-07-31T15:06:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f679baec640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6794112250 con 0x7f6794108720 2024-07-31T15:06:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f679baec640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6794112550 con 0x7f6794102490 2024-07-31T15:06:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f6799087640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6794107dd0 0x7f6794104140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f6799087640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6794107dd0 0x7f6794104140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55526/0 (socket says 172.21.15.67:55526) 2024-07-31T15:06:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f6799087640 1 -- 172.21.15.67:0/619951079 learned_addr learned my addr 172.21.15.67:0/619951079 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f6799888640 1 --2- 172.21.15.67:0/619951079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6794102490 0x7f6794103c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f679a089640 1 --2- 172.21.15.67:0/619951079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6794108720 0x7f6794104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f6799087640 1 -- 172.21.15.67:0/619951079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6794102490 msgr2=0x7f6794103c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f6799087640 1 --2- 172.21.15.67:0/619951079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6794102490 0x7f6794103c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f6799087640 1 -- 172.21.15.67:0/619951079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6794108720 msgr2=0x7f6794104680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f6799087640 1 --2- 172.21.15.67:0/619951079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6794108720 0x7f6794104680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.677+0000 7f6799087640 1 -- 172.21.15.67:0/619951079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67941a81b0 con 0x7f6794107dd0 2024-07-31T15:06:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.678+0000 7f6799087640 1 --2- 172.21.15.67:0/619951079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6794107dd0 0x7f6794104140 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f678800e9c0 tx=0x7f678800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.678+0000 7f6792ffd640 1 -- 172.21.15.67:0/619951079 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678800cd80 con 0x7f6794107dd0 2024-07-31T15:06:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.678+0000 7f6792ffd640 1 -- 172.21.15.67:0/619951079 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67880054d0 con 0x7f6794107dd0 2024-07-31T15:06:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.678+0000 7f6792ffd640 1 -- 172.21.15.67:0/619951079 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788010430 con 0x7f6794107dd0 2024-07-31T15:06:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.678+0000 7f679baec640 1 -- 172.21.15.67:0/619951079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67941a84a0 con 0x7f6794107dd0 2024-07-31T15:06:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.678+0000 7f679baec640 1 -- 172.21.15.67:0/619951079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67941a89e0 con 0x7f6794107dd0 2024-07-31T15:06:49.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.681+0000 7f67a203d740 1 Processor -- start 2024-07-31T15:06:49.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.681+0000 7f67a203d740 1 -- start start 2024-07-31T15:06:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.681+0000 7f6792ffd640 1 -- 172.21.15.67:0/619951079 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6788004fb0 con 0x7f6794107dd0 2024-07-31T15:06:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.681+0000 7f67a203d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0ca070 0x56165a46f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.681+0000 7f67a203d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a46fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.681+0000 7f67a203d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0a9390 0x56165a46fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.681+0000 7f67a203d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56165a3e5bb0 con 0x56165a0a9390 2024-07-31T15:06:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.681+0000 7f67a203d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56165a3d50b0 con 0x56165a0a11b0 2024-07-31T15:06:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.681+0000 7f67a203d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56165a3d90f0 con 0x56165a0ca070 2024-07-31T15:06:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.681+0000 7f6799888640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a46fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.681+0000 7f6799087640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0a9390 0x56165a46fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.681+0000 7f6799888640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a46fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44534/0 (socket says 172.21.15.67:44534) 2024-07-31T15:06:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.681+0000 7f6799888640 1 -- 172.21.15.67:0/3279506266 learned_addr learned my addr 172.21.15.67:0/3279506266 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.682+0000 7f679a089640 1 --2- 172.21.15.67:0/3279506266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0ca070 0x56165a46f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.682+0000 7f6799888640 1 -- 172.21.15.67:0/3279506266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0ca070 msgr2=0x56165a46f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.682+0000 7f6799888640 1 --2- 172.21.15.67:0/3279506266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0ca070 0x56165a46f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.682+0000 7f6799888640 1 -- 172.21.15.67:0/3279506266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0a9390 msgr2=0x56165a46fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.682+0000 7f6799888640 1 --2- 172.21.15.67:0/3279506266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0a9390 0x56165a46fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.682+0000 7f6799888640 1 -- 172.21.15.67:0/3279506266 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56165a37bb90 con 0x56165a0a11b0 2024-07-31T15:06:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.682+0000 7f679a089640 1 --2- 172.21.15.67:0/3279506266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0ca070 0x56165a46f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:06:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.682+0000 7f6799888640 1 --2- 172.21.15.67:0/3279506266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a46fab0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f678402ade0 tx=0x7f678402f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f45cdc6927a53e24 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.682+0000 7f6792ffd640 1 --2- 172.21.15.67:0/619951079 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f677c0b3d40 0x7f677c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.682+0000 7f677bfff640 1 -- 172.21.15.67:0/3279506266 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784031cf0 con 0x56165a0a11b0 2024-07-31T15:06:49.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.682+0000 7f677bfff640 1 -- 172.21.15.67:0/3279506266 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6784036030 con 0x56165a0a11b0 2024-07-31T15:06:49.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.682+0000 7f677bfff640 1 -- 172.21.15.67:0/3279506266 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67840042a0 con 0x56165a0a11b0 2024-07-31T15:06:49.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.682+0000 7f679a089640 1 --2- 172.21.15.67:0/619951079 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f677c0b3d40 0x7f677c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.682+0000 7f6792ffd640 1 -- 172.21.15.67:0/619951079 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f67880e4770 con 0x7f6794107dd0 2024-07-31T15:06:49.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.683+0000 7f67a203d740 1 -- 172.21.15.67:0/3279506266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 msgr2=0x56165a46fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:49.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.683+0000 7f679a089640 1 --2- 172.21.15.67:0/619951079 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f677c0b3d40 0x7f677c0b6250 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f678c009cb0 tx=0x7f678c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:49.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.683+0000 7f67a203d740 1 --2- 172.21.15.67:0/3279506266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a46fab0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f678402ade0 tx=0x7f678402f310 comp rx=0 tx=0).stop 2024-07-31T15:06:49.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.684+0000 7f67a203d740 1 -- 172.21.15.67:0/3279506266 shutdown_connections 2024-07-31T15:06:49.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.684+0000 7f67a203d740 1 --2- 172.21.15.67:0/3279506266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0a9390 0x56165a46fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.684+0000 7f67a203d740 1 --2- 172.21.15.67:0/3279506266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a46fab0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.684+0000 7f67a203d740 1 --2- 172.21.15.67:0/3279506266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0ca070 0x56165a46f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.684+0000 7f67a203d740 1 -- 172.21.15.67:0/3279506266 >> 172.21.15.67:0/3279506266 conn(0x56165a0be460 msgr2=0x56165a46d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.684+0000 7f67a203d740 1 -- 172.21.15.67:0/3279506266 shutdown_connections 2024-07-31T15:06:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.684+0000 7f67a203d740 1 -- 172.21.15.67:0/3279506266 wait complete. 2024-07-31T15:06:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.684+0000 7f67a203d740 1 -- start start 2024-07-31T15:06:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.684+0000 7f67a203d740 1 Processor -- start 2024-07-31T15:06:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f67a203d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0a11b0 0x56165a47b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f67a203d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a9390 0x56165a47b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f67a203d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0ca070 0x56165a486e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f67a203d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56165a238700 con 0x56165a0ca070 2024-07-31T15:06:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f67a203d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56165a3779e0 con 0x56165a0a9390 2024-07-31T15:06:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f67a203d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56165a47db70 con 0x56165a0a11b0 2024-07-31T15:06:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f67a203d740 10 client.0 _ll_register_callbacks cb 0x56165a3924f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f679a089640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0ca070 0x56165a486e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f679a089640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0ca070 0x56165a486e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55544/0 (socket says 172.21.15.67:55544) 2024-07-31T15:06:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f679a089640 1 -- 172.21.15.67:0/3209939111 learned_addr learned my addr 172.21.15.67:0/3209939111 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f6799087640 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a9390 0x56165a47b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f6799888640 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0a11b0 0x56165a47b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f679a089640 1 -- 172.21.15.67:0/3209939111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0a11b0 msgr2=0x56165a47b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f679a089640 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0a11b0 0x56165a47b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f679a089640 1 -- 172.21.15.67:0/3209939111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a9390 msgr2=0x56165a47b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f679a089640 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a9390 0x56165a47b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f679a089640 1 -- 172.21.15.67:0/3209939111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56165a135650 con 0x56165a0ca070 2024-07-31T15:06:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f6799888640 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0a11b0 0x56165a47b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:06:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f6799087640 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a9390 0x56165a47b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f679a089640 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0ca070 0x56165a486e90 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f678c004300 tx=0x7f678c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a576b7351413cb3c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.685+0000 7f67797fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:06:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.686+0000 7f67797fa640 1 -- 172.21.15.67:0/3209939111 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c029440 con 0x56165a0ca070 2024-07-31T15:06:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.686+0000 7f67a203d740 1 -- 172.21.15.67:0/3209939111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6784036030 con 0x56165a0ca070 2024-07-31T15:06:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.686+0000 7f67797fa640 1 -- 172.21.15.67:0/3209939111 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f678c01f430 con 0x56165a0ca070 2024-07-31T15:06:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.686+0000 7f67797fa640 1 -- 172.21.15.67:0/3209939111 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f678c01f630 con 0x56165a0ca070 2024-07-31T15:06:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.686+0000 7f67a203d740 10 client.18840 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.686+0000 7f67a203d740 1 -- 172.21.15.67:0/3209939111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x56165a489c20 con 0x56165a0ca070 2024-07-31T15:06:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.686+0000 7f67797fa640 1 -- 172.21.15.67:0/3209939111 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6784036030 con 0x56165a0ca070 2024-07-31T15:06:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.686+0000 7f67797fa640 1 -- 172.21.15.67:0/3209939111 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f678c027070 con 0x56165a0ca070 2024-07-31T15:06:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.686+0000 7f67a203d740 10 client.18840 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.686+0000 7f67a203d740 10 client.18840 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:06:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.686+0000 7f67a203d740 20 client.18840 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.686+0000 7f67a203d740 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x56165a0889f0 0x56165a48d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.686+0000 7f67a203d740 4 client.18840 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:06:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.686+0000 7f67a203d740 1 -- 172.21.15.67:0/3209939111 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56165a3e7b00 con 0x56165a0889f0 2024-07-31T15:06:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.687+0000 7f6799888640 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x56165a0889f0 0x56165a48d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.687+0000 7f6799888640 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x56165a0889f0 0x56165a48d1e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8272e3b87fbf8e2f server_cookie=7647511c2f20c94f in_seq=0 out_seq=0 2024-07-31T15:06:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.687+0000 7f67797fa640 10 client.18840 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:06:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67797fa640 1 -- 172.21.15.67:0/3209939111 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56165a3e7b00 con 0x56165a0889f0 2024-07-31T15:06:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67797fa640 10 client.18840 handle_command_reply: tid=0 2024-07-31T15:06:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67a203d740 1 client.18840 shutdown 2024-07-31T15:06:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67a203d740 1 -- 172.21.15.67:0/3209939111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0ca070 msgr2=0x56165a486e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67a203d740 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0ca070 0x56165a486e90 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f678c004300 tx=0x7f678c004730 comp rx=0 tx=0).stop 2024-07-31T15:06:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67a203d740 1 -- 172.21.15.67:0/3209939111 shutdown_connections 2024-07-31T15:06:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67a203d740 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x56165a0889f0 0x56165a48d1e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67a203d740 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0ca070 0x56165a486e90 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67a203d740 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a9390 0x56165a47b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67a203d740 1 --2- 172.21.15.67:0/3209939111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0a11b0 0x56165a47b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67a203d740 1 -- 172.21.15.67:0/3209939111 >> 172.21.15.67:0/3209939111 conn(0x56165a0be460 msgr2=0x56165a46bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67797fa640 0 client.18840 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:06:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67a203d740 1 -- 172.21.15.67:0/3209939111 shutdown_connections 2024-07-31T15:06:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67a203d740 1 -- 172.21.15.67:0/3209939111 wait complete. 2024-07-31T15:06:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.688+0000 7f67a203d740 20 client.18840 trim_cache size 0 max 16384 2024-07-31T15:06:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.702+0000 7f67a203d740 1 Processor -- start 2024-07-31T15:06:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.702+0000 7f67a203d740 1 -- start start 2024-07-31T15:06:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f67a203d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a46c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f67a203d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0a9390 0x56165a46d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f67a203d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0ca070 0x56165a475690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f67a203d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f67941b29c0 con 0x56165a0a9390 2024-07-31T15:06:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f67a203d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f67941b4500 con 0x56165a0a11b0 2024-07-31T15:06:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f67a203d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67941b3090 con 0x56165a0ca070 2024-07-31T15:06:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f6799087640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0a9390 0x56165a46d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f6799888640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a46c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f6799888640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a46c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44556/0 (socket says 172.21.15.67:44556) 2024-07-31T15:06:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f6799087640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0a9390 0x56165a46d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55556/0 (socket says 172.21.15.67:55556) 2024-07-31T15:06:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f6799888640 1 -- 172.21.15.67:0/1942090305 learned_addr learned my addr 172.21.15.67:0/1942090305 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f679a089640 1 --2- 172.21.15.67:0/1942090305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0ca070 0x56165a475690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f6799888640 1 -- 172.21.15.67:0/1942090305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0ca070 msgr2=0x56165a475690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f6799888640 1 --2- 172.21.15.67:0/1942090305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0ca070 0x56165a475690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f6799888640 1 -- 172.21.15.67:0/1942090305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0a9390 msgr2=0x56165a46d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:49.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f6799888640 1 --2- 172.21.15.67:0/1942090305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0a9390 0x56165a46d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f6799888640 1 -- 172.21.15.67:0/1942090305 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56165a47b9f0 con 0x56165a0a11b0 2024-07-31T15:06:49.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f6799087640 1 --2- 172.21.15.67:0/1942090305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0a9390 0x56165a46d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:06:49.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f679a089640 1 --2- 172.21.15.67:0/1942090305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0ca070 0x56165a475690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:06:49.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.703+0000 7f6799888640 1 --2- 172.21.15.67:0/1942090305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a46c2b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6784003d40 tx=0x7f6784003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2a821054d82729a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:49.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.704+0000 7f677b7fe640 1 -- 172.21.15.67:0/1942090305 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678405f070 con 0x56165a0a11b0 2024-07-31T15:06:49.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.704+0000 7f677b7fe640 1 -- 172.21.15.67:0/1942090305 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6784036040 con 0x56165a0a11b0 2024-07-31T15:06:49.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.704+0000 7f677b7fe640 1 -- 172.21.15.67:0/1942090305 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678402f870 con 0x56165a0a11b0 2024-07-31T15:06:49.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.704+0000 7f67a203d740 1 -- 172.21.15.67:0/1942090305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 msgr2=0x56165a46c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:49.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.704+0000 7f67a203d740 1 --2- 172.21.15.67:0/1942090305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a46c2b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6784003d40 tx=0x7f6784003d70 comp rx=0 tx=0).stop 2024-07-31T15:06:49.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 -- 172.21.15.67:0/1942090305 shutdown_connections 2024-07-31T15:06:49.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 --2- 172.21.15.67:0/1942090305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0ca070 0x56165a475690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 --2- 172.21.15.67:0/1942090305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0a9390 0x56165a46d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 --2- 172.21.15.67:0/1942090305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a46c2b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 -- 172.21.15.67:0/1942090305 >> 172.21.15.67:0/1942090305 conn(0x56165a0be460 msgr2=0x56165a46cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 -- 172.21.15.67:0/1942090305 shutdown_connections 2024-07-31T15:06:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 -- 172.21.15.67:0/1942090305 wait complete. 2024-07-31T15:06:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 -- start start 2024-07-31T15:06:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 Processor -- start 2024-07-31T15:06:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a47e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0a9390 0x56165a47e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0ca070 0x56165a4f3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f67941b3e30 con 0x56165a0ca070 2024-07-31T15:06:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f67941b3760 con 0x56165a0a11b0 2024-07-31T15:06:49.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67941b09c0 con 0x56165a0a9390 2024-07-31T15:06:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.705+0000 7f67a203d740 10 client.0 _ll_register_callbacks cb 0x7f678c009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f679a089640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0ca070 0x56165a4f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f6799888640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a47e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f6799888640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a47e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44570/0 (socket says 172.21.15.67:44570) 2024-07-31T15:06:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f679a089640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0ca070 0x56165a4f3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55566/0 (socket says 172.21.15.67:55566) 2024-07-31T15:06:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f6799888640 1 -- 172.21.15.67:0/2951843811 learned_addr learned my addr 172.21.15.67:0/2951843811 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f6799087640 1 --2- 172.21.15.67:0/2951843811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0a9390 0x56165a47e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f6799888640 1 -- 172.21.15.67:0/2951843811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0a9390 msgr2=0x56165a47e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f6799888640 1 --2- 172.21.15.67:0/2951843811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0a9390 0x56165a47e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f6799888640 1 -- 172.21.15.67:0/2951843811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0ca070 msgr2=0x56165a4f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f6799888640 1 --2- 172.21.15.67:0/2951843811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0ca070 0x56165a4f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f6799888640 1 -- 172.21.15.67:0/2951843811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56165a4f3ea0 con 0x56165a0a11b0 2024-07-31T15:06:49.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f6799888640 1 --2- 172.21.15.67:0/2951843811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a47e1c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f67840317d0 tx=0x7f6784009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c857879c33b56515 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:49.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f67797fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:06:49.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f67797fa640 1 -- 172.21.15.67:0/2951843811 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784068060 con 0x56165a0a11b0 2024-07-31T15:06:49.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f67797fa640 1 -- 172.21.15.67:0/2951843811 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f6784031cf0 con 0x56165a0a11b0 2024-07-31T15:06:49.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.706+0000 7f67a203d740 1 -- 172.21.15.67:0/2951843811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6784036040 con 0x56165a0a11b0 2024-07-31T15:06:49.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.707+0000 7f67797fa640 1 -- 172.21.15.67:0/2951843811 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f678405f050 con 0x56165a0a11b0 2024-07-31T15:06:49.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.707+0000 7f67a203d740 10 client.28373 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:49.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.707+0000 7f67a203d740 1 -- 172.21.15.67:0/2951843811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x56165a4f4f30 con 0x56165a0a11b0 2024-07-31T15:06:49.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.707+0000 7f67797fa640 1 -- 172.21.15.67:0/2951843811 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6784036040 con 0x56165a0a11b0 2024-07-31T15:06:49.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.707+0000 7f67797fa640 1 -- 172.21.15.67:0/2951843811 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f678c009ce0 con 0x56165a0a11b0 2024-07-31T15:06:49.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.707+0000 7f67a203d740 10 client.28373 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:49.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.707+0000 7f67a203d740 10 client.28373 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:06:49.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.707+0000 7f67a203d740 20 client.28373 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:49.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.707+0000 7f67a203d740 1 --2- 172.21.15.67:0/2951843811 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x56165a05dda0 0x56165a4f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:49.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.707+0000 7f67a203d740 4 client.28373 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:06:49.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.707+0000 7f67a203d740 1 -- 172.21.15.67:0/2951843811 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56165a3e35a0 con 0x56165a05dda0 2024-07-31T15:06:49.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.707+0000 7f6799087640 1 --2- 172.21.15.67:0/2951843811 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x56165a05dda0 0x56165a4f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:49.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f6799087640 1 --2- 172.21.15.67:0/2951843811 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x56165a05dda0 0x56165a4f73b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d161d3c395ac87a server_cookie=637ccdca1f6388af in_seq=0 out_seq=0 2024-07-31T15:06:49.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67797fa640 10 client.28373 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:06:49.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67797fa640 1 -- 172.21.15.67:0/2951843811 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56165a3e35a0 con 0x56165a05dda0 2024-07-31T15:06:49.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67797fa640 10 client.28373 handle_command_reply: tid=0 2024-07-31T15:06:49.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67a203d740 1 client.28373 shutdown 2024-07-31T15:06:49.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67a203d740 1 -- 172.21.15.67:0/2951843811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 msgr2=0x56165a47e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:49.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67a203d740 1 --2- 172.21.15.67:0/2951843811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a47e1c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f67840317d0 tx=0x7f6784009800 comp rx=0 tx=0).stop 2024-07-31T15:06:49.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67a203d740 1 -- 172.21.15.67:0/2951843811 shutdown_connections 2024-07-31T15:06:49.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67a203d740 1 --2- 172.21.15.67:0/2951843811 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x56165a05dda0 0x56165a4f73b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67a203d740 1 --2- 172.21.15.67:0/2951843811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56165a0ca070 0x56165a4f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67a203d740 1 --2- 172.21.15.67:0/2951843811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56165a0a9390 0x56165a47e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67797fa640 0 client.28373 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:06:49.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67a203d740 1 --2- 172.21.15.67:0/2951843811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56165a0a11b0 0x56165a47e1c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67a203d740 1 -- 172.21.15.67:0/2951843811 >> 172.21.15.67:0/2951843811 conn(0x56165a0be460 msgr2=0x56165a4793c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:49.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67a203d740 1 -- 172.21.15.67:0/2951843811 shutdown_connections 2024-07-31T15:06:49.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67a203d740 1 -- 172.21.15.67:0/2951843811 wait complete. 2024-07-31T15:06:49.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.708+0000 7f67a203d740 20 client.28373 trim_cache size 0 max 16384 2024-07-31T15:06:49.715 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:06:49.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.709+0000 7f679baec640 1 -- 172.21.15.67:0/619951079 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f677c0b3d40 msgr2=0x7f677c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:49.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.709+0000 7f679baec640 1 --2- 172.21.15.67:0/619951079 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f677c0b3d40 0x7f677c0b6250 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f678c009cb0 tx=0x7f678c005f90 comp rx=0 tx=0).stop 2024-07-31T15:06:49.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.709+0000 7f679baec640 1 -- 172.21.15.67:0/619951079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6794107dd0 msgr2=0x7f6794104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:49.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.709+0000 7f679baec640 1 --2- 172.21.15.67:0/619951079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6794107dd0 0x7f6794104140 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f678800e9c0 tx=0x7f678800ee80 comp rx=0 tx=0).stop 2024-07-31T15:06:49.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.709+0000 7f679baec640 1 -- 172.21.15.67:0/619951079 shutdown_connections 2024-07-31T15:06:49.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.709+0000 7f679baec640 1 --2- 172.21.15.67:0/619951079 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f677c0b3d40 0x7f677c0b6250 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.709+0000 7f679baec640 1 --2- 172.21.15.67:0/619951079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6794108720 0x7f6794104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.709+0000 7f679baec640 1 --2- 172.21.15.67:0/619951079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6794107dd0 0x7f6794104140 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.709+0000 7f679baec640 1 --2- 172.21.15.67:0/619951079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6794102490 0x7f6794103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:49.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.709+0000 7f679baec640 1 -- 172.21.15.67:0/619951079 >> 172.21.15.67:0/619951079 conn(0x7f6794079160 msgr2=0x7f6794106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:49.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.709+0000 7f679baec640 1 -- 172.21.15.67:0/619951079 shutdown_connections 2024-07-31T15:06:49.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:49.709+0000 7f679baec640 1 -- 172.21.15.67:0/619951079 wait complete. 2024-07-31T15:06:49.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:49 smithi077 ceph-mon[37327]: pgmap v897: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 460 KiB/s wr, 88 op/s 2024-07-31T15:06:50.445 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:06:50.446 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-07-31T15:06:50.446 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:06:50.446 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 42, 2024-07-31T15:06:50.446 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:06:50.446 INFO:teuthology.orchestra.run.smithi067.stdout: "cc95e00d": { 2024-07-31T15:06:50.446 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 42, 2024-07-31T15:06:50.446 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:06:50.446 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:06:50.446 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:06:50.446 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:06:50.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:06:50.446 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:06:50.447 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 137.8, 2024-07-31T15:06:50.447 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:06:50.447 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:06:50.447 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:06:50.447 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:06:50.447 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:06:50.447 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:06:50.447 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:50.447 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:50.447 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:50.447 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:50.447 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:06:50.447 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:06:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.442+0000 7f8f2f7fe640 1 -- 172.21.15.67:0/2851273649 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d9869d2170 con 0x55d986b088e0 2024-07-31T15:06:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.442+0000 7f8f2f7fe640 10 client.28313 handle_command_reply: tid=0 2024-07-31T15:06:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.442+0000 7f8f5ccde740 1 client.28313 shutdown 2024-07-31T15:06:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.442+0000 7f8f5ccde740 1 -- 172.21.15.67:0/2851273649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866781c0 msgr2=0x55d986a6f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.442+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/2851273649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866781c0 0x55d986a6f570 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009010 tx=0x7f8f4000bfa0 comp rx=0 tx=0).stop 2024-07-31T15:06:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.442+0000 7f8f5ccde740 1 -- 172.21.15.67:0/2851273649 shutdown_connections 2024-07-31T15:06:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.442+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/2851273649 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d986b088e0 0x55d986b0ad00 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.442+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/2851273649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d9866b9840 0x55d986a6fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.442+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/2851273649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d986698b60 0x55d986a6fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.442+0000 7f8f5ccde740 1 --2- 172.21.15.67:0/2851273649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9866781c0 0x55d986a6f570 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.442+0000 7f8f5ccde740 1 -- 172.21.15.67:0/2851273649 >> 172.21.15.67:0/2851273649 conn(0x55d9866ad990 msgr2=0x55d986a632c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.442+0000 7f8f2f7fe640 0 client.28313 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.442+0000 7f8f5ccde740 1 -- 172.21.15.67:0/2851273649 shutdown_connections 2024-07-31T15:06:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.443+0000 7f8f5ccde740 1 -- 172.21.15.67:0/2851273649 wait complete. 2024-07-31T15:06:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.443+0000 7f8f5ccde740 20 client.28313 trim_cache size 0 max 16384 2024-07-31T15:06:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.443+0000 7f8f5678d640 1 -- 172.21.15.67:0/3333641395 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8f380b3e60 msgr2=0x7f8f380b6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.443+0000 7f8f5678d640 1 --2- 172.21.15.67:0/3333641395 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8f380b3e60 0x7f8f380b6370 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8f40011c50 tx=0x7f8f40011ab0 comp rx=0 tx=0).stop 2024-07-31T15:06:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.443+0000 7f8f5678d640 1 -- 172.21.15.67:0/3333641395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f50111780 msgr2=0x7f8f501a9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.443+0000 7f8f5678d640 1 --2- 172.21.15.67:0/3333641395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f50111780 0x7f8f501a9160 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f8f4402aa90 tx=0x7f8f44030a80 comp rx=0 tx=0).stop 2024-07-31T15:06:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.444+0000 7f8f5678d640 1 -- 172.21.15.67:0/3333641395 shutdown_connections 2024-07-31T15:06:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.444+0000 7f8f5678d640 1 --2- 172.21.15.67:0/3333641395 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8f380b3e60 0x7f8f380b6370 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.444+0000 7f8f5678d640 1 --2- 172.21.15.67:0/3333641395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5011c5d0 0x7f8f501ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.444+0000 7f8f5678d640 1 --2- 172.21.15.67:0/3333641395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f50111780 0x7f8f501a9160 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.444+0000 7f8f5678d640 1 --2- 172.21.15.67:0/3333641395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f5010a330 0x7f8f501a8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.444+0000 7f8f5678d640 1 -- 172.21.15.67:0/3333641395 >> 172.21.15.67:0/3333641395 conn(0x7f8f5006e440 msgr2=0x7f8f5011a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.444+0000 7f8f5678d640 1 -- 172.21.15.67:0/3333641395 shutdown_connections 2024-07-31T15:06:50.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:50.444+0000 7f8f5678d640 1 -- 172.21.15.67:0/3333641395 wait complete. 2024-07-31T15:06:50.615 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T15:06:51.247 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cc95e00d', quiesce took 24.2 seconds. Will release after: -6.399999999999999 2024-07-31T15:06:51.248 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:06:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:06:51.523 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:51 smithi067 ceph-mon[32750]: pgmap v898: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 653 KiB/s wr, 106 op/s 2024-07-31T15:06:51.538 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:51 smithi077 ceph-mon[37327]: pgmap v898: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 653 KiB/s wr, 106 op/s 2024-07-31T15:06:51.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:51 smithi073 ceph-mon[36714]: pgmap v898: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 653 KiB/s wr, 106 op/s 2024-07-31T15:06:52.722 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:06:53.540 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:53 smithi067 ceph-mon[32750]: pgmap v899: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 594 KiB/s wr, 107 op/s 2024-07-31T15:06:53.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:53 smithi073 ceph-mon[36714]: pgmap v899: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 594 KiB/s wr, 107 op/s 2024-07-31T15:06:53.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:53 smithi077 ceph-mon[37327]: pgmap v899: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 594 KiB/s wr, 107 op/s 2024-07-31T15:06:53.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.888+0000 7f78068c3640 1 -- 172.21.15.67:0/2972455706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7800108720 msgr2=0x7f780010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:53.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.888+0000 7f78068c3640 1 --2- 172.21.15.67:0/2972455706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7800108720 0x7f780010ef50 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009ca0 tx=0x7f77f802a8c0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.888+0000 7f78068c3640 1 -- 172.21.15.67:0/2972455706 shutdown_connections 2024-07-31T15:06:53.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.888+0000 7f78068c3640 1 --2- 172.21.15.67:0/2972455706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7800108720 0x7f780010ef50 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.888+0000 7f78068c3640 1 --2- 172.21.15.67:0/2972455706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7800107dd0 0x7f78001081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.888+0000 7f78068c3640 1 --2- 172.21.15.67:0/2972455706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7800102490 0x7f78001028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.888+0000 7f78068c3640 1 -- 172.21.15.67:0/2972455706 >> 172.21.15.67:0/2972455706 conn(0x7f7800079160 msgr2=0x7f7800079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:53.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.888+0000 7f78068c3640 1 -- 172.21.15.67:0/2972455706 shutdown_connections 2024-07-31T15:06:53.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.888+0000 7f78068c3640 1 -- 172.21.15.67:0/2972455706 wait complete. 2024-07-31T15:06:53.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.889+0000 7f78068c3640 1 Processor -- start 2024-07-31T15:06:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.889+0000 7f78068c3640 1 -- start start 2024-07-31T15:06:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.889+0000 7f78068c3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7800102490 0x7f7800198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f78068c3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7800107dd0 0x7f7800199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f78068c3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7800108720 0x7f780019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f78068c3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f78001123a0 con 0x7f7800102490 2024-07-31T15:06:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f78068c3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7800112220 con 0x7f7800107dd0 2024-07-31T15:06:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f78068c3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7800112520 con 0x7f7800108720 2024-07-31T15:06:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f77fffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7800102490 0x7f7800198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f77fffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7800102490 0x7f7800198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34918/0 (socket says 172.21.15.67:34918) 2024-07-31T15:06:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f77fffff640 1 -- 172.21.15.67:0/4147616243 learned_addr learned my addr 172.21.15.67:0/4147616243 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f77ff7fe640 1 --2- 172.21.15.67:0/4147616243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7800107dd0 0x7f7800199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f7804e60640 1 --2- 172.21.15.67:0/4147616243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7800108720 0x7f780019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f77fffff640 1 -- 172.21.15.67:0/4147616243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7800108720 msgr2=0x7f780019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:53.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f77fffff640 1 --2- 172.21.15.67:0/4147616243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7800108720 0x7f780019e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f77fffff640 1 -- 172.21.15.67:0/4147616243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7800107dd0 msgr2=0x7f7800199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:53.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f77fffff640 1 --2- 172.21.15.67:0/4147616243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7800107dd0 0x7f7800199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f77fffff640 1 -- 172.21.15.67:0/4147616243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780019ec20 con 0x7f7800102490 2024-07-31T15:06:53.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.890+0000 7f77fffff640 1 --2- 172.21.15.67:0/4147616243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7800102490 0x7f7800198d70 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f77f000d770 tx=0x7f77f000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:53.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.891+0000 7f77fd7fa640 1 -- 172.21.15.67:0/4147616243 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0017070 con 0x7f7800102490 2024-07-31T15:06:53.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.891+0000 7f78068c3640 1 -- 172.21.15.67:0/4147616243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780019ef10 con 0x7f7800102490 2024-07-31T15:06:53.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.891+0000 7f77fd7fa640 1 -- 172.21.15.67:0/4147616243 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77f0009820 con 0x7f7800102490 2024-07-31T15:06:53.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.891+0000 7f77fd7fa640 1 -- 172.21.15.67:0/4147616243 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0004bc0 con 0x7f7800102490 2024-07-31T15:06:53.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.891+0000 7f78068c3640 1 -- 172.21.15.67:0/4147616243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780006a890 con 0x7f7800102490 2024-07-31T15:06:53.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.893+0000 7f780ce14740 1 Processor -- start 2024-07-31T15:06:53.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.893+0000 7f780ce14740 1 -- start start 2024-07-31T15:06:53.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.894+0000 7f77fd7fa640 1 -- 172.21.15.67:0/4147616243 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f77f0010460 con 0x7f7800102490 2024-07-31T15:06:53.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.894+0000 7f780ce14740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80e1070 0x5653e8486570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.894+0000 7f780ce14740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80b81b0 0x5653e8486ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.894+0000 7f780ce14740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80c0390 0x5653e8486ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.894+0000 7f780ce14740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5653e83fcbb0 con 0x5653e80c0390 2024-07-31T15:06:53.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.894+0000 7f780ce14740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5653e83ec0b0 con 0x5653e80b81b0 2024-07-31T15:06:53.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.894+0000 7f780ce14740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5653e83f00f0 con 0x5653e80e1070 2024-07-31T15:06:53.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.894+0000 7f77ff7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80c0390 0x5653e8486ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.894+0000 7f77ff7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80c0390 0x5653e8486ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34930/0 (socket says 172.21.15.67:34930) 2024-07-31T15:06:53.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.894+0000 7f77ff7fe640 1 -- 172.21.15.67:0/3962801257 learned_addr learned my addr 172.21.15.67:0/3962801257 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:53.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.894+0000 7f7804e60640 1 --2- 172.21.15.67:0/3962801257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80e1070 0x5653e8486570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.894+0000 7f77fffff640 1 --2- 172.21.15.67:0/3962801257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80b81b0 0x5653e8486ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.895+0000 7f77ff7fe640 1 -- 172.21.15.67:0/3962801257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80e1070 msgr2=0x5653e8486570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:53.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.895+0000 7f77ff7fe640 1 --2- 172.21.15.67:0/3962801257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80e1070 0x5653e8486570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.895+0000 7f77ff7fe640 1 -- 172.21.15.67:0/3962801257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80b81b0 msgr2=0x5653e8486ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:53.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.895+0000 7f77ff7fe640 1 --2- 172.21.15.67:0/3962801257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80b81b0 0x5653e8486ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.895+0000 7f77ff7fe640 1 -- 172.21.15.67:0/3962801257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653e8392b90 con 0x5653e80c0390 2024-07-31T15:06:53.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.895+0000 7f77ff7fe640 1 --2- 172.21.15.67:0/3962801257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80c0390 0x5653e8486ff0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f77f400ac70 tx=0x7f77f400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cbb4f5487d827 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:53.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.895+0000 7f77fd7fa640 1 --2- 172.21.15.67:0/4147616243 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f77e80b3d70 0x7f77e80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.895+0000 7f77e67fc640 1 -- 172.21.15.67:0/3962801257 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f401d070 con 0x5653e80c0390 2024-07-31T15:06:53.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.895+0000 7f77e67fc640 1 -- 172.21.15.67:0/3962801257 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77f4018af0 con 0x5653e80c0390 2024-07-31T15:06:53.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.895+0000 7f77e67fc640 1 -- 172.21.15.67:0/3962801257 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4010ae0 con 0x5653e80c0390 2024-07-31T15:06:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.895+0000 7f7804e60640 1 --2- 172.21.15.67:0/4147616243 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f77e80b3d70 0x7f77e80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.895+0000 7f77fd7fa640 1 -- 172.21.15.67:0/4147616243 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f77f00e3d90 con 0x7f7800102490 2024-07-31T15:06:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.896+0000 7f780ce14740 1 -- 172.21.15.67:0/3962801257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80c0390 msgr2=0x5653e8486ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.896+0000 7f7804e60640 1 --2- 172.21.15.67:0/4147616243 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f77e80b3d70 0x7f77e80b6280 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f77f8006010 tx=0x7f77f802f310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.896+0000 7f780ce14740 1 --2- 172.21.15.67:0/3962801257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80c0390 0x5653e8486ff0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f77f400ac70 tx=0x7f77f400f790 comp rx=0 tx=0).stop 2024-07-31T15:06:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.896+0000 7f780ce14740 1 -- 172.21.15.67:0/3962801257 shutdown_connections 2024-07-31T15:06:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.896+0000 7f780ce14740 1 --2- 172.21.15.67:0/3962801257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80c0390 0x5653e8486ff0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.896+0000 7f780ce14740 1 --2- 172.21.15.67:0/3962801257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80b81b0 0x5653e8486ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.896+0000 7f780ce14740 1 --2- 172.21.15.67:0/3962801257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80e1070 0x5653e8486570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.896+0000 7f780ce14740 1 -- 172.21.15.67:0/3962801257 >> 172.21.15.67:0/3962801257 conn(0x5653e80d5460 msgr2=0x5653e8484460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.896+0000 7f780ce14740 1 -- 172.21.15.67:0/3962801257 shutdown_connections 2024-07-31T15:06:53.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.896+0000 7f780ce14740 1 -- 172.21.15.67:0/3962801257 wait complete. 2024-07-31T15:06:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.897+0000 7f780ce14740 1 -- start start 2024-07-31T15:06:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.897+0000 7f780ce14740 1 Processor -- start 2024-07-31T15:06:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.897+0000 7f780ce14740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80b81b0 0x5653e8492140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.897+0000 7f780ce14740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80c0390 0x5653e8492680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.897+0000 7f780ce14740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80e1070 0x5653e849e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.897+0000 7f780ce14740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5653e824f700 con 0x5653e80e1070 2024-07-31T15:06:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.897+0000 7f780ce14740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5653e838e9e0 con 0x5653e80b81b0 2024-07-31T15:06:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.897+0000 7f780ce14740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5653e8494d50 con 0x5653e80c0390 2024-07-31T15:06:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.897+0000 7f780ce14740 10 client.0 _ll_register_callbacks cb 0x5653e83a94f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.898+0000 7f7804e60640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80e1070 0x5653e849e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.898+0000 7f7804e60640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80e1070 0x5653e849e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34940/0 (socket says 172.21.15.67:34940) 2024-07-31T15:06:53.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.898+0000 7f7804e60640 1 -- 172.21.15.67:0/2221173369 learned_addr learned my addr 172.21.15.67:0/2221173369 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:53.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.898+0000 7f77fffff640 1 --2- 172.21.15.67:0/2221173369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80b81b0 0x5653e8492140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.898+0000 7f77ff7fe640 1 --2- 172.21.15.67:0/2221173369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80c0390 0x5653e8492680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.898+0000 7f7804e60640 1 -- 172.21.15.67:0/2221173369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80c0390 msgr2=0x5653e8492680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:53.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.898+0000 7f7804e60640 1 --2- 172.21.15.67:0/2221173369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80c0390 0x5653e8492680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.898+0000 7f7804e60640 1 -- 172.21.15.67:0/2221173369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80b81b0 msgr2=0x5653e8492140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:53.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.898+0000 7f7804e60640 1 --2- 172.21.15.67:0/2221173369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80b81b0 0x5653e8492140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.898+0000 7f7804e60640 1 -- 172.21.15.67:0/2221173369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653e814c650 con 0x5653e80e1070 2024-07-31T15:06:53.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.898+0000 7f7804e60640 1 --2- 172.21.15.67:0/2221173369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80e1070 0x5653e849e070 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f77f8031870 tx=0x7f77f80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bb480536012a89 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:53.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.898+0000 7f77df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:06:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.899+0000 7f77df7fe640 1 -- 172.21.15.67:0/2221173369 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8057060 con 0x5653e80e1070 2024-07-31T15:06:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.899+0000 7f77df7fe640 1 -- 172.21.15.67:0/2221173369 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f77f80044c0 con 0x5653e80e1070 2024-07-31T15:06:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.899+0000 7f780ce14740 1 -- 172.21.15.67:0/2221173369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77f4018af0 con 0x5653e80e1070 2024-07-31T15:06:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.899+0000 7f77df7fe640 1 -- 172.21.15.67:0/2221173369 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f77f805e050 con 0x5653e80e1070 2024-07-31T15:06:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.899+0000 7f780ce14740 10 client.18858 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.899+0000 7f780ce14740 1 -- 172.21.15.67:0/2221173369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5653e84a0da0 con 0x5653e80e1070 2024-07-31T15:06:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.899+0000 7f77df7fe640 1 -- 172.21.15.67:0/2221173369 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77f4018af0 con 0x5653e80e1070 2024-07-31T15:06:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.900+0000 7f77df7fe640 1 -- 172.21.15.67:0/2221173369 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f77f80305c0 con 0x5653e80e1070 2024-07-31T15:06:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.900+0000 7f780ce14740 10 client.18858 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.900+0000 7f780ce14740 10 client.18858 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:06:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.900+0000 7f780ce14740 20 client.18858 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.900+0000 7f780ce14740 1 --2- 172.21.15.67:0/2221173369 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5653e809f9f0 0x5653e84a4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.900+0000 7f780ce14740 4 client.18858 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:06:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.900+0000 7f780ce14740 1 -- 172.21.15.67:0/2221173369 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653e83feb00 con 0x5653e809f9f0 2024-07-31T15:06:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.900+0000 7f77fffff640 1 --2- 172.21.15.67:0/2221173369 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5653e809f9f0 0x5653e84a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.900+0000 7f77df7fe640 10 client.18858 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:06:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.900+0000 7f77fffff640 1 --2- 172.21.15.67:0/2221173369 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5653e809f9f0 0x5653e84a4370 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6dd57e5c7f619525 server_cookie=141b3c363fafa188 in_seq=0 out_seq=0 2024-07-31T15:06:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.901+0000 7f77df7fe640 1 -- 172.21.15.67:0/2221173369 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653e83feb00 con 0x5653e809f9f0 2024-07-31T15:06:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.901+0000 7f77df7fe640 10 client.18858 handle_command_reply: tid=0 2024-07-31T15:06:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.901+0000 7f780ce14740 1 client.18858 shutdown 2024-07-31T15:06:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.902+0000 7f780ce14740 1 -- 172.21.15.67:0/2221173369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80e1070 msgr2=0x5653e849e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.902+0000 7f780ce14740 1 --2- 172.21.15.67:0/2221173369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80e1070 0x5653e849e070 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f77f8031870 tx=0x7f77f80095c0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.902+0000 7f780ce14740 1 -- 172.21.15.67:0/2221173369 shutdown_connections 2024-07-31T15:06:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.902+0000 7f780ce14740 1 --2- 172.21.15.67:0/2221173369 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5653e809f9f0 0x5653e84a4370 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.902+0000 7f780ce14740 1 --2- 172.21.15.67:0/2221173369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80e1070 0x5653e849e070 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.902+0000 7f780ce14740 1 --2- 172.21.15.67:0/2221173369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80c0390 0x5653e8492680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.902+0000 7f780ce14740 1 --2- 172.21.15.67:0/2221173369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80b81b0 0x5653e8492140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.902+0000 7f77df7fe640 0 client.18858 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:06:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.902+0000 7f780ce14740 1 -- 172.21.15.67:0/2221173369 >> 172.21.15.67:0/2221173369 conn(0x5653e80d5460 msgr2=0x5653e8482c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.902+0000 7f780ce14740 1 -- 172.21.15.67:0/2221173369 shutdown_connections 2024-07-31T15:06:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.902+0000 7f780ce14740 1 -- 172.21.15.67:0/2221173369 wait complete. 2024-07-31T15:06:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.902+0000 7f780ce14740 20 client.18858 trim_cache size 0 max 16384 2024-07-31T15:06:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.920+0000 7f780ce14740 1 Processor -- start 2024-07-31T15:06:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.920+0000 7f780ce14740 1 -- start start 2024-07-31T15:06:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.920+0000 7f780ce14740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80b81b0 0x5653e848bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f780ce14740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80c0390 0x5653e84832b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f780ce14740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80e1070 0x5653e8483e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f780ce14740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f78001ae4d0 con 0x5653e80b81b0 2024-07-31T15:06:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f780ce14740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f78001b0010 con 0x5653e80e1070 2024-07-31T15:06:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f780ce14740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78001aeba0 con 0x5653e80c0390 2024-07-31T15:06:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f77fffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80b81b0 0x5653e848bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f77fffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80b81b0 0x5653e848bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34946/0 (socket says 172.21.15.67:34946) 2024-07-31T15:06:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f77fffff640 1 -- 172.21.15.67:0/1181279810 learned_addr learned my addr 172.21.15.67:0/1181279810 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:53.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f77ff7fe640 1 --2- 172.21.15.67:0/1181279810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80c0390 0x5653e84832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f7804e60640 1 --2- 172.21.15.67:0/1181279810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80e1070 0x5653e8483e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f77fffff640 1 -- 172.21.15.67:0/1181279810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80c0390 msgr2=0x5653e84832b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:53.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f77fffff640 1 --2- 172.21.15.67:0/1181279810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80c0390 0x5653e84832b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f77fffff640 1 -- 172.21.15.67:0/1181279810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80e1070 msgr2=0x5653e8483e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:53.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f77fffff640 1 --2- 172.21.15.67:0/1181279810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80e1070 0x5653e8483e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f77fffff640 1 -- 172.21.15.67:0/1181279810 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653e8488860 con 0x5653e80b81b0 2024-07-31T15:06:53.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f77fffff640 1 --2- 172.21.15.67:0/1181279810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80b81b0 0x5653e848bd90 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f77f00aa300 tx=0x7f77f00b3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae359963c95b95 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:53.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.921+0000 7f77e5ffb640 1 -- 172.21.15.67:0/1181279810 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0020040 con 0x5653e80b81b0 2024-07-31T15:06:53.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.922+0000 7f77e5ffb640 1 -- 172.21.15.67:0/1181279810 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77f00e5040 con 0x5653e80b81b0 2024-07-31T15:06:53.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.922+0000 7f77e5ffb640 1 -- 172.21.15.67:0/1181279810 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f00099d0 con 0x5653e80b81b0 2024-07-31T15:06:53.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.922+0000 7f780ce14740 1 -- 172.21.15.67:0/1181279810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80b81b0 msgr2=0x5653e848bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:53.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.922+0000 7f780ce14740 1 --2- 172.21.15.67:0/1181279810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80b81b0 0x5653e848bd90 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f77f00aa300 tx=0x7f77f00b3040 comp rx=0 tx=0).stop 2024-07-31T15:06:53.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.922+0000 7f780ce14740 1 -- 172.21.15.67:0/1181279810 shutdown_connections 2024-07-31T15:06:53.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.922+0000 7f780ce14740 1 --2- 172.21.15.67:0/1181279810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80e1070 0x5653e8483e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.922+0000 7f780ce14740 1 --2- 172.21.15.67:0/1181279810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80c0390 0x5653e84832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.922+0000 7f780ce14740 1 --2- 172.21.15.67:0/1181279810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80b81b0 0x5653e848bd90 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.922+0000 7f780ce14740 1 -- 172.21.15.67:0/1181279810 >> 172.21.15.67:0/1181279810 conn(0x5653e80d5460 msgr2=0x5653e848b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.923+0000 7f780ce14740 1 -- 172.21.15.67:0/1181279810 shutdown_connections 2024-07-31T15:06:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.923+0000 7f780ce14740 1 -- 172.21.15.67:0/1181279810 wait complete. 2024-07-31T15:06:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.923+0000 7f780ce14740 1 -- start start 2024-07-31T15:06:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.923+0000 7f780ce14740 1 Processor -- start 2024-07-31T15:06:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f780ce14740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80b81b0 0x5653e849a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f780ce14740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80c0390 0x5653e8495ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f780ce14740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80e1070 0x5653e8496320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f780ce14740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f78001af940 con 0x5653e80c0390 2024-07-31T15:06:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f780ce14740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f78001af270 con 0x5653e80e1070 2024-07-31T15:06:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f780ce14740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78001ac4d0 con 0x5653e80b81b0 2024-07-31T15:06:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f780ce14740 10 client.0 _ll_register_callbacks cb 0x7f77f8009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f77ff7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80c0390 0x5653e8495ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f77ff7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80c0390 0x5653e8495ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34960/0 (socket says 172.21.15.67:34960) 2024-07-31T15:06:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f77ff7fe640 1 -- 172.21.15.67:0/1941122721 learned_addr learned my addr 172.21.15.67:0/1941122721 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f7804e60640 1 --2- 172.21.15.67:0/1941122721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80e1070 0x5653e8496320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f77fffff640 1 --2- 172.21.15.67:0/1941122721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80b81b0 0x5653e849a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f77ff7fe640 1 -- 172.21.15.67:0/1941122721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80b81b0 msgr2=0x5653e849a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f77ff7fe640 1 --2- 172.21.15.67:0/1941122721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80b81b0 0x5653e849a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f77ff7fe640 1 -- 172.21.15.67:0/1941122721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80e1070 msgr2=0x5653e8496320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f77ff7fe640 1 --2- 172.21.15.67:0/1941122721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80e1070 0x5653e8496320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f77ff7fe640 1 -- 172.21.15.67:0/1941122721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653e8496ab0 con 0x5653e80c0390 2024-07-31T15:06:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f77ff7fe640 1 --2- 172.21.15.67:0/1941122721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80c0390 0x5653e8495ca0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f77f400ac40 tx=0x7f77f40055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2ca3131c5c9a4a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.924+0000 7f77df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:06:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.925+0000 7f77df7fe640 1 -- 172.21.15.67:0/1941122721 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4029070 con 0x5653e80c0390 2024-07-31T15:06:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.925+0000 7f77df7fe640 1 -- 172.21.15.67:0/1941122721 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f77f40109f0 con 0x5653e80c0390 2024-07-31T15:06:53.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.925+0000 7f780ce14740 1 -- 172.21.15.67:0/1941122721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77f00e5040 con 0x5653e80c0390 2024-07-31T15:06:53.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.925+0000 7f77df7fe640 1 -- 172.21.15.67:0/1941122721 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f77f4035050 con 0x5653e80c0390 2024-07-31T15:06:53.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.925+0000 7f780ce14740 10 client.18864 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:53.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.925+0000 7f780ce14740 1 -- 172.21.15.67:0/1941122721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5653e850cbc0 con 0x5653e80c0390 2024-07-31T15:06:53.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.925+0000 7f77df7fe640 1 -- 172.21.15.67:0/1941122721 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77f00e5040 con 0x5653e80c0390 2024-07-31T15:06:53.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.925+0000 7f77df7fe640 1 -- 172.21.15.67:0/1941122721 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f77f4023040 con 0x5653e80c0390 2024-07-31T15:06:53.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.925+0000 7f780ce14740 10 client.18864 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:53.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.925+0000 7f780ce14740 10 client.18864 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:06:53.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.925+0000 7f780ce14740 20 client.18864 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:53.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.925+0000 7f780ce14740 1 --2- 172.21.15.67:0/1941122721 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5653e8074da0 0x5653e850efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:53.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.925+0000 7f780ce14740 4 client.18864 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:06:53.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.925+0000 7f780ce14740 1 -- 172.21.15.67:0/1941122721 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653e83fa5a0 con 0x5653e8074da0 2024-07-31T15:06:53.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f77fffff640 1 --2- 172.21.15.67:0/1941122721 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5653e8074da0 0x5653e850efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:53.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f77fffff640 1 --2- 172.21.15.67:0/1941122721 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5653e8074da0 0x5653e850efc0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e974e999cf58d479 server_cookie=920d45da82976204 in_seq=0 out_seq=0 2024-07-31T15:06:53.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f77df7fe640 10 client.18864 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:06:53.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f77df7fe640 1 -- 172.21.15.67:0/1941122721 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653e83fa5a0 con 0x5653e8074da0 2024-07-31T15:06:53.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f77df7fe640 10 client.18864 handle_command_reply: tid=0 2024-07-31T15:06:53.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f780ce14740 1 client.18864 shutdown 2024-07-31T15:06:53.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f780ce14740 1 -- 172.21.15.67:0/1941122721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80c0390 msgr2=0x5653e8495ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:53.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f780ce14740 1 --2- 172.21.15.67:0/1941122721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80c0390 0x5653e8495ca0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f77f400ac40 tx=0x7f77f40055a0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f780ce14740 1 -- 172.21.15.67:0/1941122721 shutdown_connections 2024-07-31T15:06:53.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f780ce14740 1 --2- 172.21.15.67:0/1941122721 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5653e8074da0 0x5653e850efc0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f780ce14740 1 --2- 172.21.15.67:0/1941122721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653e80e1070 0x5653e8496320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f780ce14740 1 --2- 172.21.15.67:0/1941122721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653e80c0390 0x5653e8495ca0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f780ce14740 1 --2- 172.21.15.67:0/1941122721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653e80b81b0 0x5653e849a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f77df7fe640 0 client.18864 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:06:53.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.926+0000 7f780ce14740 1 -- 172.21.15.67:0/1941122721 >> 172.21.15.67:0/1941122721 conn(0x5653e80d5460 msgr2=0x5653e84875a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f780ce14740 1 -- 172.21.15.67:0/1941122721 shutdown_connections 2024-07-31T15:06:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f780ce14740 1 -- 172.21.15.67:0/1941122721 wait complete. 2024-07-31T15:06:53.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f780ce14740 20 client.18864 trim_cache size 0 max 16384 2024-07-31T15:06:53.933 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:06:53.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f78068c3640 1 -- 172.21.15.67:0/4147616243 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f77e80b3d70 msgr2=0x7f77e80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:53.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f78068c3640 1 --2- 172.21.15.67:0/4147616243 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f77e80b3d70 0x7f77e80b6280 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f77f8006010 tx=0x7f77f802f310 comp rx=0 tx=0).stop 2024-07-31T15:06:53.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f78068c3640 1 -- 172.21.15.67:0/4147616243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7800102490 msgr2=0x7f7800198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:53.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f78068c3640 1 --2- 172.21.15.67:0/4147616243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7800102490 0x7f7800198d70 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f77f000d770 tx=0x7f77f000db70 comp rx=0 tx=0).stop 2024-07-31T15:06:53.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f78068c3640 1 -- 172.21.15.67:0/4147616243 shutdown_connections 2024-07-31T15:06:53.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f78068c3640 1 --2- 172.21.15.67:0/4147616243 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f77e80b3d70 0x7f77e80b6280 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f78068c3640 1 --2- 172.21.15.67:0/4147616243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7800108720 0x7f780019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f78068c3640 1 --2- 172.21.15.67:0/4147616243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7800107dd0 0x7f7800199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f78068c3640 1 --2- 172.21.15.67:0/4147616243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7800102490 0x7f7800198d70 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:53.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f78068c3640 1 -- 172.21.15.67:0/4147616243 >> 172.21.15.67:0/4147616243 conn(0x7f7800079160 msgr2=0x7f7800103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:53.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f78068c3640 1 -- 172.21.15.67:0/4147616243 shutdown_connections 2024-07-31T15:06:53.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:53.927+0000 7f78068c3640 1 -- 172.21.15.67:0/4147616243 wait complete. 2024-07-31T15:06:55.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:55 smithi067 ceph-mon[32750]: pgmap v900: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 719 KiB/s wr, 136 op/s 2024-07-31T15:06:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.553+0000 7fe9935a9640 1 -- 172.21.15.67:0/3185500726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c10d840 msgr2=0x7fe98c10fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.553+0000 7fe9935a9640 1 --2- 172.21.15.67:0/3185500726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c10d840 0x7fe98c10fd30 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe984009ca0 tx=0x7fe98402a8c0 comp rx=0 tx=0).stop 2024-07-31T15:06:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.553+0000 7fe9935a9640 1 -- 172.21.15.67:0/3185500726 shutdown_connections 2024-07-31T15:06:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.553+0000 7fe9935a9640 1 --2- 172.21.15.67:0/3185500726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c10d840 0x7fe98c10fd30 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.553+0000 7fe9935a9640 1 --2- 172.21.15.67:0/3185500726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98c104d30 0x7fe98c10d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.553+0000 7fe9935a9640 1 --2- 172.21.15.67:0/3185500726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c1043e0 0x7fe98c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.553+0000 7fe9935a9640 1 -- 172.21.15.67:0/3185500726 >> 172.21.15.67:0/3185500726 conn(0x7fe98c079160 msgr2=0x7fe98c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.553+0000 7fe9935a9640 1 -- 172.21.15.67:0/3185500726 shutdown_connections 2024-07-31T15:06:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.553+0000 7fe9935a9640 1 -- 172.21.15.67:0/3185500726 wait complete. 2024-07-31T15:06:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.554+0000 7fe9935a9640 1 Processor -- start 2024-07-31T15:06:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.554+0000 7fe9935a9640 1 -- start start 2024-07-31T15:06:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.554+0000 7fe9935a9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c1043e0 0x7fe98c199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.554+0000 7fe9935a9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c104d30 0x7fe98c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.554+0000 7fe9935a9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98c10d840 0x7fe98c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.554+0000 7fe9935a9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1134c0 con 0x7fe98c104d30 2024-07-31T15:06:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.554+0000 7fe9935a9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c113340 con 0x7fe98c1043e0 2024-07-31T15:06:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.554+0000 7fe9935a9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c113640 con 0x7fe98c10d840 2024-07-31T15:06:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.554+0000 7fe990b44640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c104d30 0x7fe98c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.554+0000 7fe990b44640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c104d30 0x7fe98c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34980/0 (socket says 172.21.15.67:34980) 2024-07-31T15:06:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.554+0000 7fe990b44640 1 -- 172.21.15.67:0/2956869563 learned_addr learned my addr 172.21.15.67:0/2956869563 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:55.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.554+0000 7fe991345640 1 --2- 172.21.15.67:0/2956869563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c1043e0 0x7fe98c199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:55.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.555+0000 7fe991b46640 1 --2- 172.21.15.67:0/2956869563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98c10d840 0x7fe98c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:55.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.555+0000 7fe990b44640 1 -- 172.21.15.67:0/2956869563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98c10d840 msgr2=0x7fe98c19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:55.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.555+0000 7fe990b44640 1 --2- 172.21.15.67:0/2956869563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98c10d840 0x7fe98c19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:55.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.555+0000 7fe990b44640 1 -- 172.21.15.67:0/2956869563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c1043e0 msgr2=0x7fe98c199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:55.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.555+0000 7fe990b44640 1 --2- 172.21.15.67:0/2956869563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c1043e0 0x7fe98c199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:55.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.555+0000 7fe990b44640 1 -- 172.21.15.67:0/2956869563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c19fe40 con 0x7fe98c104d30 2024-07-31T15:06:55.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.555+0000 7fe990b44640 1 --2- 172.21.15.67:0/2956869563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c104d30 0x7fe98c19a470 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00e9c0 tx=0x7fe97c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:55.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.555+0000 7fe9827fc640 1 -- 172.21.15.67:0/2956869563 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c00ccc0 con 0x7fe98c104d30 2024-07-31T15:06:55.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.555+0000 7fe9935a9640 1 -- 172.21.15.67:0/2956869563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98c1a0130 con 0x7fe98c104d30 2024-07-31T15:06:55.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.555+0000 7fe9827fc640 1 -- 172.21.15.67:0/2956869563 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe97c0054d0 con 0x7fe98c104d30 2024-07-31T15:06:55.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.555+0000 7fe9827fc640 1 -- 172.21.15.67:0/2956869563 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c010430 con 0x7fe98c104d30 2024-07-31T15:06:55.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.555+0000 7fe9935a9640 1 -- 172.21.15.67:0/2956869563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98c1a6890 con 0x7fe98c104d30 2024-07-31T15:06:55.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.556+0000 7fe9935a9640 1 -- 172.21.15.67:0/2956869563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe98c1047f0 con 0x7fe98c104d30 2024-07-31T15:06:55.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.559+0000 7fe9827fc640 1 -- 172.21.15.67:0/2956869563 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe97c005000 con 0x7fe98c104d30 2024-07-31T15:06:55.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.560+0000 7fe9827fc640 1 --2- 172.21.15.67:0/2956869563 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe9780b3d40 0x7fe9780b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:55.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.560+0000 7fe991345640 1 --2- 172.21.15.67:0/2956869563 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe9780b3d40 0x7fe9780b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:55.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.560+0000 7fe9827fc640 1 -- 172.21.15.67:0/2956869563 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7fe97c0e4800 con 0x7fe98c104d30 2024-07-31T15:06:55.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.561+0000 7fe991345640 1 --2- 172.21.15.67:0/2956869563 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe9780b3d40 0x7fe9780b6250 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe988005fd0 tx=0x7fe98800b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:55.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.564+0000 7fe9827fc640 1 -- 172.21.15.67:0/2956869563 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe97c0555e0 con 0x7fe98c104d30 2024-07-31T15:06:55.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:55 smithi073 ceph-mon[36714]: pgmap v900: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 719 KiB/s wr, 136 op/s 2024-07-31T15:06:55.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:55 smithi077 ceph-mon[37327]: pgmap v900: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 719 KiB/s wr, 136 op/s 2024-07-31T15:06:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.808+0000 7fe9935a9640 1 -- 172.21.15.67:0/2956869563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe98c19b000 con 0x7fe98c104d30 2024-07-31T15:06:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.809+0000 7fe9827fc640 1 -- 172.21.15.67:0/2956869563 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11962 (secure 0 0 0) 0x7fe97c0052f0 con 0x7fe98c104d30 2024-07-31T15:06:55.811 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:06:55.811 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"btime":"2024-07-31T15:06:04:323642+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:06:04.323492+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28214":{"gid":28214,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1516494761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1516494761},{"type":"v1","addr":"172.21.15.73:6835","nonce":1516494761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:06:55.811 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-07-31T15:06:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.812+0000 7fe9935a9640 1 -- 172.21.15.67:0/2956869563 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe9780b3d40 msgr2=0x7fe9780b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.812+0000 7fe9935a9640 1 --2- 172.21.15.67:0/2956869563 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe9780b3d40 0x7fe9780b6250 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe988005fd0 tx=0x7fe98800b410 comp rx=0 tx=0).stop 2024-07-31T15:06:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.812+0000 7fe9935a9640 1 -- 172.21.15.67:0/2956869563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c104d30 msgr2=0x7fe98c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.812+0000 7fe9935a9640 1 --2- 172.21.15.67:0/2956869563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c104d30 0x7fe98c19a470 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00e9c0 tx=0x7fe97c00ee80 comp rx=0 tx=0).stop 2024-07-31T15:06:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.813+0000 7fe9935a9640 1 -- 172.21.15.67:0/2956869563 shutdown_connections 2024-07-31T15:06:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.813+0000 7fe9935a9640 1 --2- 172.21.15.67:0/2956869563 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe9780b3d40 0x7fe9780b6250 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.813+0000 7fe9935a9640 1 --2- 172.21.15.67:0/2956869563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98c10d840 0x7fe98c19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.813+0000 7fe9935a9640 1 --2- 172.21.15.67:0/2956869563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c104d30 0x7fe98c19a470 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.813+0000 7fe9935a9640 1 --2- 172.21.15.67:0/2956869563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c1043e0 0x7fe98c199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.813+0000 7fe9935a9640 1 -- 172.21.15.67:0/2956869563 >> 172.21.15.67:0/2956869563 conn(0x7fe98c079160 msgr2=0x7fe98c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.813+0000 7fe9935a9640 1 -- 172.21.15.67:0/2956869563 shutdown_connections 2024-07-31T15:06:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:55.813+0000 7fe9935a9640 1 -- 172.21.15.67:0/2956869563 wait complete. 2024-07-31T15:06:56.165 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T15:06:56.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:56 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2956869563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:06:56.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:56 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2956869563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:06:56.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:56 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2956869563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:06:56.990 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id cc95e00d --release --await' 2024-07-31T15:06:56.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id cc95e00d --release --await 2024-07-31T15:06:57.080 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:06:57.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:57 smithi067 ceph-mon[32750]: pgmap v901: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 705 KiB/s wr, 148 op/s 2024-07-31T15:06:57.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:57 smithi073 ceph-mon[36714]: pgmap v901: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 705 KiB/s wr, 148 op/s 2024-07-31T15:06:57.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:57 smithi077 ceph-mon[37327]: pgmap v901: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 705 KiB/s wr, 148 op/s 2024-07-31T15:06:57.848 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:06:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.424+0000 7f684275e640 1 -- 172.21.15.67:0/1765702416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683c107d20 msgr2=0x7f683c108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.424+0000 7f684275e640 1 --2- 172.21.15.67:0/1765702416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683c107d20 0x7f683c108130 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f6824009900 tx=0x7f682402a410 comp rx=0 tx=0).stop 2024-07-31T15:06:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.424+0000 7f684275e640 1 -- 172.21.15.67:0/1765702416 shutdown_connections 2024-07-31T15:06:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.424+0000 7f684275e640 1 --2- 172.21.15.67:0/1765702416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683c108670 0x7f683c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.424+0000 7f684275e640 1 --2- 172.21.15.67:0/1765702416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683c107d20 0x7f683c108130 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.424+0000 7f684275e640 1 --2- 172.21.15.67:0/1765702416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f683c1023f0 0x7f683c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.424+0000 7f684275e640 1 -- 172.21.15.67:0/1765702416 >> 172.21.15.67:0/1765702416 conn(0x7f683c0fddf0 msgr2=0x7f683c100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.424+0000 7f684275e640 1 -- 172.21.15.67:0/1765702416 shutdown_connections 2024-07-31T15:06:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.424+0000 7f684275e640 1 -- 172.21.15.67:0/1765702416 wait complete. 2024-07-31T15:06:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.425+0000 7f684275e640 1 Processor -- start 2024-07-31T15:06:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.425+0000 7f684275e640 1 -- start start 2024-07-31T15:06:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.425+0000 7f684275e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683c1023f0 0x7f683c198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.425+0000 7f684275e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683c107d20 0x7f683c199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.425+0000 7f684275e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f683c108670 0x7f683c19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.425+0000 7f684275e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f683c112300 con 0x7f683c1023f0 2024-07-31T15:06:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.425+0000 7f684275e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f683c112180 con 0x7f683c107d20 2024-07-31T15:06:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.425+0000 7f684275e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f683c112480 con 0x7f683c108670 2024-07-31T15:06:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f683bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683c1023f0 0x7f683c198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f683bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683c1023f0 0x7f683c198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34992/0 (socket says 172.21.15.67:34992) 2024-07-31T15:06:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f683bfff640 1 -- 172.21.15.67:0/2928639597 learned_addr learned my addr 172.21.15.67:0/2928639597 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f683bfff640 1 -- 172.21.15.67:0/2928639597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f683c108670 msgr2=0x7f683c19e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f683bfff640 1 --2- 172.21.15.67:0/2928639597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f683c108670 0x7f683c19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f6833fff640 1 --2- 172.21.15.67:0/2928639597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683c107d20 0x7f683c199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f683bfff640 1 -- 172.21.15.67:0/2928639597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683c107d20 msgr2=0x7f683c199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f683bfff640 1 --2- 172.21.15.67:0/2928639597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683c107d20 0x7f683c199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f683bfff640 1 -- 172.21.15.67:0/2928639597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c19ecd0 con 0x7f683c1023f0 2024-07-31T15:06:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f6833fff640 1 --2- 172.21.15.67:0/2928639597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683c107d20 0x7f683c199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:06:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f683bfff640 1 --2- 172.21.15.67:0/2928639597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683c1023f0 0x7f683c198da0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f682c00e9f0 tx=0x7f682c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f6839ffb640 1 -- 172.21.15.67:0/2928639597 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c00cd80 con 0x7f683c1023f0 2024-07-31T15:06:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f6839ffb640 1 -- 172.21.15.67:0/2928639597 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c0054d0 con 0x7f683c1023f0 2024-07-31T15:06:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f684275e640 1 -- 172.21.15.67:0/2928639597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683c19efc0 con 0x7f683c1023f0 2024-07-31T15:06:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f6839ffb640 1 -- 172.21.15.67:0/2928639597 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c010690 con 0x7f683c1023f0 2024-07-31T15:06:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.426+0000 7f684275e640 1 -- 172.21.15.67:0/2928639597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f683c1a5750 con 0x7f683c1023f0 2024-07-31T15:06:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.429+0000 7f6839ffb640 1 -- 172.21.15.67:0/2928639597 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f682c004e90 con 0x7f683c1023f0 2024-07-31T15:06:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.429+0000 7f6849655740 1 Processor -- start 2024-07-31T15:06:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.429+0000 7f6849655740 1 -- start start 2024-07-31T15:06:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.429+0000 7f6849655740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2711b0 0x55557c6136b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.429+0000 7f6849655740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2482f0 0x55557c613bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.429+0000 7f6849655740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2504d0 0x55557c614130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.429+0000 7f6849655740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55557c5893b0 con 0x55557c2482f0 2024-07-31T15:06:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.429+0000 7f6849655740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55557c5788b0 con 0x55557c2504d0 2024-07-31T15:06:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.429+0000 7f6849655740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55557c57c8f0 con 0x55557c2711b0 2024-07-31T15:06:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f683bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2482f0 0x55557c613bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f683bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2482f0 0x55557c613bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35000/0 (socket says 172.21.15.67:35000) 2024-07-31T15:06:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f683bfff640 1 -- 172.21.15.67:0/3669177975 learned_addr learned my addr 172.21.15.67:0/3669177975 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f6833fff640 1 --2- 172.21.15.67:0/3669177975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2504d0 0x55557c614130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f6840cfb640 1 --2- 172.21.15.67:0/3669177975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2711b0 0x55557c6136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f683bfff640 1 -- 172.21.15.67:0/3669177975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2711b0 msgr2=0x55557c6136b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f683bfff640 1 --2- 172.21.15.67:0/3669177975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2711b0 0x55557c6136b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f683bfff640 1 -- 172.21.15.67:0/3669177975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2504d0 msgr2=0x55557c614130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f683bfff640 1 --2- 172.21.15.67:0/3669177975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2504d0 0x55557c614130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f683bfff640 1 -- 172.21.15.67:0/3669177975 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55557c51f9a0 con 0x55557c2482f0 2024-07-31T15:06:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f6839ffb640 1 --2- 172.21.15.67:0/2928639597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68280b3d40 0x7f68280b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f6840cfb640 1 --2- 172.21.15.67:0/2928639597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68280b3d40 0x7f68280b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f683bfff640 1 --2- 172.21.15.67:0/3669177975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2482f0 0x55557c613bf0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f682c0e1c00 tx=0x7f682c0eccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccddd7bf3d8a7cd7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f6839ffb640 1 -- 172.21.15.67:0/2928639597 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f682c0e4880 con 0x7f683c1023f0 2024-07-31T15:06:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.430+0000 7f68327fc640 1 -- 172.21.15.67:0/3669177975 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c0a7010 con 0x55557c2482f0 2024-07-31T15:06:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.431+0000 7f68327fc640 1 -- 172.21.15.67:0/3669177975 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c0aac60 con 0x55557c2482f0 2024-07-31T15:06:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.431+0000 7f68327fc640 1 -- 172.21.15.67:0/3669177975 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c005180 con 0x55557c2482f0 2024-07-31T15:06:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.431+0000 7f6849655740 1 -- 172.21.15.67:0/3669177975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2482f0 msgr2=0x55557c613bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.431+0000 7f6840cfb640 1 --2- 172.21.15.67:0/2928639597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68280b3d40 0x7f68280b6250 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f683400acc0 tx=0x7f683400a350 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.431+0000 7f6849655740 1 --2- 172.21.15.67:0/3669177975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2482f0 0x55557c613bf0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f682c0e1c00 tx=0x7f682c0eccd0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.431+0000 7f6849655740 1 -- 172.21.15.67:0/3669177975 shutdown_connections 2024-07-31T15:06:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.431+0000 7f6849655740 1 --2- 172.21.15.67:0/3669177975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2504d0 0x55557c614130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.431+0000 7f6849655740 1 --2- 172.21.15.67:0/3669177975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2482f0 0x55557c613bf0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.431+0000 7f6849655740 1 --2- 172.21.15.67:0/3669177975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2711b0 0x55557c6136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.431+0000 7f6849655740 1 -- 172.21.15.67:0/3669177975 >> 172.21.15.67:0/3669177975 conn(0x55557c265300 msgr2=0x55557c6115a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.431+0000 7f6849655740 1 -- 172.21.15.67:0/3669177975 shutdown_connections 2024-07-31T15:06:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.431+0000 7f6849655740 1 -- 172.21.15.67:0/3669177975 wait complete. 2024-07-31T15:06:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.432+0000 7f6849655740 1 -- start start 2024-07-31T15:06:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.432+0000 7f6849655740 1 Processor -- start 2024-07-31T15:06:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.432+0000 7f6849655740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2482f0 0x55557c61f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.432+0000 7f6849655740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2504d0 0x55557c61f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.432+0000 7f6849655740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2711b0 0x55557c62b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.432+0000 7f6849655740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55557c4b2ab0 con 0x55557c2482f0 2024-07-31T15:06:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.432+0000 7f6849655740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55557c500cf0 con 0x55557c2504d0 2024-07-31T15:06:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.432+0000 7f6849655740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55557c621e30 con 0x55557c2711b0 2024-07-31T15:06:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.432+0000 7f6849655740 10 client.0 _ll_register_callbacks cb 0x55557c5723e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.432+0000 7f683bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2482f0 0x55557c61f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f683bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2482f0 0x55557c61f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35008/0 (socket says 172.21.15.67:35008) 2024-07-31T15:06:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f683bfff640 1 -- 172.21.15.67:0/4227049448 learned_addr learned my addr 172.21.15.67:0/4227049448 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f6833fff640 1 --2- 172.21.15.67:0/4227049448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2504d0 0x55557c61f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f683bfff640 1 -- 172.21.15.67:0/4227049448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2711b0 msgr2=0x55557c62b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f683bfff640 1 --2- 172.21.15.67:0/4227049448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2711b0 0x55557c62b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f683bfff640 1 -- 172.21.15.67:0/4227049448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2504d0 msgr2=0x55557c61f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f683bfff640 1 --2- 172.21.15.67:0/4227049448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2504d0 0x55557c61f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f683bfff640 1 -- 172.21.15.67:0/4227049448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55557c2dc790 con 0x55557c2482f0 2024-07-31T15:06:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f6833fff640 1 --2- 172.21.15.67:0/4227049448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2504d0 0x55557c61f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:06:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f683bfff640 1 --2- 172.21.15.67:0/4227049448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2482f0 0x55557c61f1f0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f682c0e1d40 tx=0x7f682c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e280f8dbf8a43e11 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f681b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:06:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f681b7fe640 1 -- 172.21.15.67:0/4227049448 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c0aa8c0 con 0x55557c2482f0 2024-07-31T15:06:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f681b7fe640 1 -- 172.21.15.67:0/4227049448 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f682c0ab5c0 con 0x55557c2482f0 2024-07-31T15:06:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f6849655740 1 -- 172.21.15.67:0/4227049448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f682c0aac60 con 0x55557c2482f0 2024-07-31T15:06:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f681b7fe640 1 -- 172.21.15.67:0/4227049448 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f682c0aaaa0 con 0x55557c2482f0 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f6849655740 10 client.18882 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f6849655740 1 -- 172.21.15.67:0/4227049448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55557c355620 con 0x55557c2482f0 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.433+0000 7f681b7fe640 1 -- 172.21.15.67:0/4227049448 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c0aac60 con 0x55557c2482f0 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.434+0000 7f681b7fe640 1 -- 172.21.15.67:0/4227049448 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f682c0b3d70 con 0x55557c2482f0 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.434+0000 7f6849655740 10 client.18882 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.434+0000 7f6849655740 10 client.18882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.434+0000 7f6849655740 20 client.18882 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.434+0000 7f6849655740 1 --2- 172.21.15.67:0/4227049448 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55557c22fd80 0x55557c630f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.434+0000 7f6849655740 4 client.18882 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.434+0000 7f6849655740 1 -- 172.21.15.67:0/4227049448 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55557c58b300 con 0x55557c22fd80 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.434+0000 7f6833fff640 1 --2- 172.21.15.67:0/4227049448 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55557c22fd80 0x55557c630f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.434+0000 7f6833fff640 1 --2- 172.21.15.67:0/4227049448 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55557c22fd80 0x55557c630f90 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e97aebbb67b614 server_cookie=dc23717f45567fa7 in_seq=0 out_seq=0 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.434+0000 7f681b7fe640 10 client.18882 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.435+0000 7f681b7fe640 1 -- 172.21.15.67:0/4227049448 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55557c58b300 con 0x55557c22fd80 2024-07-31T15:06:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.435+0000 7f681b7fe640 10 client.18882 handle_command_reply: tid=0 2024-07-31T15:06:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.435+0000 7f6849655740 1 client.18882 shutdown 2024-07-31T15:06:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.435+0000 7f6849655740 1 -- 172.21.15.67:0/4227049448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2482f0 msgr2=0x55557c61f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.435+0000 7f6849655740 1 --2- 172.21.15.67:0/4227049448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2482f0 0x55557c61f1f0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f682c0e1d40 tx=0x7f682c0048f0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.436+0000 7f6849655740 1 -- 172.21.15.67:0/4227049448 shutdown_connections 2024-07-31T15:06:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.436+0000 7f6849655740 1 --2- 172.21.15.67:0/4227049448 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55557c22fd80 0x55557c630f90 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.436+0000 7f6849655740 1 --2- 172.21.15.67:0/4227049448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2711b0 0x55557c62b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.436+0000 7f6849655740 1 --2- 172.21.15.67:0/4227049448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2504d0 0x55557c61f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.436+0000 7f681b7fe640 0 client.18882 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.436+0000 7f6849655740 1 --2- 172.21.15.67:0/4227049448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2482f0 0x55557c61f1f0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.436+0000 7f6849655740 1 -- 172.21.15.67:0/4227049448 >> 172.21.15.67:0/4227049448 conn(0x55557c265300 msgr2=0x55557c6146e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.436+0000 7f6849655740 1 -- 172.21.15.67:0/4227049448 shutdown_connections 2024-07-31T15:06:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.436+0000 7f6849655740 1 -- 172.21.15.67:0/4227049448 wait complete. 2024-07-31T15:06:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.436+0000 7f6849655740 20 client.18882 trim_cache size 0 max 16384 2024-07-31T15:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f6849655740 1 Processor -- start 2024-07-31T15:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f6849655740 1 -- start start 2024-07-31T15:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f6849655740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c22fd80 0x55557c6198b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f6849655740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2482f0 0x55557c620500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f6849655740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2504d0 0x55557c62b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f6849655740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1ae5f0 con 0x55557c22fd80 2024-07-31T15:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f6849655740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1b0130 con 0x55557c2482f0 2024-07-31T15:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f6849655740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1aecc0 con 0x55557c2504d0 2024-07-31T15:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f683bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c22fd80 0x55557c6198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f683bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c22fd80 0x55557c6198b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35024/0 (socket says 172.21.15.67:35024) 2024-07-31T15:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f683bfff640 1 -- 172.21.15.67:0/2197119488 learned_addr learned my addr 172.21.15.67:0/2197119488 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f6840cfb640 1 --2- 172.21.15.67:0/2197119488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2504d0 0x55557c62b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f6833fff640 1 --2- 172.21.15.67:0/2197119488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2482f0 0x55557c620500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f683bfff640 1 -- 172.21.15.67:0/2197119488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2504d0 msgr2=0x55557c62b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f683bfff640 1 --2- 172.21.15.67:0/2197119488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2504d0 0x55557c62b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f683bfff640 1 -- 172.21.15.67:0/2197119488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2482f0 msgr2=0x55557c620500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f683bfff640 1 --2- 172.21.15.67:0/2197119488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2482f0 0x55557c620500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f683bfff640 1 -- 172.21.15.67:0/2197119488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55557c6213b0 con 0x55557c22fd80 2024-07-31T15:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.452+0000 7f6833fff640 1 --2- 172.21.15.67:0/2197119488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2482f0 0x55557c620500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.453+0000 7f683bfff640 1 --2- 172.21.15.67:0/2197119488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c22fd80 0x55557c6198b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f682c0a71d0 tx=0x7f682c0e4b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413250df17eec6ef server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.453+0000 7f6831ffb640 1 -- 172.21.15.67:0/2197119488 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c0aaf20 con 0x55557c22fd80 2024-07-31T15:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.453+0000 7f6831ffb640 1 -- 172.21.15.67:0/2197119488 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c0ec530 con 0x55557c22fd80 2024-07-31T15:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.453+0000 7f6831ffb640 1 -- 172.21.15.67:0/2197119488 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c0a7010 con 0x55557c22fd80 2024-07-31T15:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.454+0000 7f6849655740 1 -- 172.21.15.67:0/2197119488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c22fd80 msgr2=0x55557c6198b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.454+0000 7f6849655740 1 --2- 172.21.15.67:0/2197119488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c22fd80 0x55557c6198b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f682c0a71d0 tx=0x7f682c0e4b80 comp rx=0 tx=0).stop 2024-07-31T15:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.454+0000 7f6849655740 1 -- 172.21.15.67:0/2197119488 shutdown_connections 2024-07-31T15:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.454+0000 7f6849655740 1 --2- 172.21.15.67:0/2197119488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2504d0 0x55557c62b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.454+0000 7f6849655740 1 --2- 172.21.15.67:0/2197119488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c2482f0 0x55557c620500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.454+0000 7f6849655740 1 --2- 172.21.15.67:0/2197119488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c22fd80 0x55557c6198b0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.454+0000 7f6849655740 1 -- 172.21.15.67:0/2197119488 >> 172.21.15.67:0/2197119488 conn(0x55557c265300 msgr2=0x55557c6191a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.454+0000 7f6849655740 1 -- 172.21.15.67:0/2197119488 shutdown_connections 2024-07-31T15:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.454+0000 7f6849655740 1 -- 172.21.15.67:0/2197119488 wait complete. 2024-07-31T15:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.454+0000 7f6849655740 1 -- start start 2024-07-31T15:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6849655740 1 Processor -- start 2024-07-31T15:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6849655740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c22fd80 0x55557c68b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6849655740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2482f0 0x55557c68bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6849655740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2504d0 0x55557c68d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6849655740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1afa60 con 0x55557c2504d0 2024-07-31T15:06:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6849655740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1af390 con 0x55557c22fd80 2024-07-31T15:06:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6849655740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1ac5f0 con 0x55557c2482f0 2024-07-31T15:06:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6849655740 10 client.0 _ll_register_callbacks cb 0x7f682c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6840cfb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2504d0 0x55557c68d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6840cfb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2504d0 0x55557c68d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35032/0 (socket says 172.21.15.67:35032) 2024-07-31T15:06:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6840cfb640 1 -- 172.21.15.67:0/161433266 learned_addr learned my addr 172.21.15.67:0/161433266 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6833fff640 1 --2- 172.21.15.67:0/161433266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2482f0 0x55557c68bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f683bfff640 1 --2- 172.21.15.67:0/161433266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c22fd80 0x55557c68b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6840cfb640 1 -- 172.21.15.67:0/161433266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2482f0 msgr2=0x55557c68bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6840cfb640 1 --2- 172.21.15.67:0/161433266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2482f0 0x55557c68bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6840cfb640 1 -- 172.21.15.67:0/161433266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c22fd80 msgr2=0x55557c68b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6840cfb640 1 --2- 172.21.15.67:0/161433266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c22fd80 0x55557c68b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.455+0000 7f6840cfb640 1 -- 172.21.15.67:0/161433266 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55557c68ddf0 con 0x55557c2504d0 2024-07-31T15:06:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f6840cfb640 1 --2- 172.21.15.67:0/161433266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2504d0 0x55557c68d750 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6834010ea0 tx=0x7f683401d790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b99130d6f012104 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f681b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:06:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f681b7fe640 1 -- 172.21.15.67:0/161433266 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683401e430 con 0x55557c2504d0 2024-07-31T15:06:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f6849655740 1 -- 172.21.15.67:0/161433266 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f682c0ec530 con 0x55557c2504d0 2024-07-31T15:06:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f681b7fe640 1 -- 172.21.15.67:0/161433266 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f683401e5d0 con 0x55557c2504d0 2024-07-31T15:06:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f681b7fe640 1 -- 172.21.15.67:0/161433266 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f6834031050 con 0x55557c2504d0 2024-07-31T15:06:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f6849655740 10 client.18888 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f6849655740 1 -- 172.21.15.67:0/161433266 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55557c698470 con 0x55557c2504d0 2024-07-31T15:06:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f681b7fe640 1 -- 172.21.15.67:0/161433266 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c0ec530 con 0x55557c2504d0 2024-07-31T15:06:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f681b7fe640 1 -- 172.21.15.67:0/161433266 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f68340267d0 con 0x55557c2504d0 2024-07-31T15:06:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f6849655740 10 client.18888 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f6849655740 10 client.18888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:06:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f6849655740 20 client.18888 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f6849655740 1 --2- 172.21.15.67:0/161433266 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55557c2711b0 0x55557c69ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f6849655740 4 client.18888 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:06:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.456+0000 7f6849655740 1 -- 172.21.15.67:0/161433266 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55557c586da0 con 0x55557c2711b0 2024-07-31T15:06:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.457+0000 7f683bfff640 1 --2- 172.21.15.67:0/161433266 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55557c2711b0 0x55557c69ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.457+0000 7f681b7fe640 10 client.18888 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.457+0000 7f683bfff640 1 --2- 172.21.15.67:0/161433266 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55557c2711b0 0x55557c69ac60 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=104b9fb06893cce3 server_cookie=665e38634e18a6dd in_seq=0 out_seq=0 2024-07-31T15:06:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f681b7fe640 1 -- 172.21.15.67:0/161433266 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55557c586da0 con 0x55557c2711b0 2024-07-31T15:06:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f681b7fe640 10 client.18888 handle_command_reply: tid=0 2024-07-31T15:06:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f6849655740 1 client.18888 shutdown 2024-07-31T15:06:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f6849655740 1 -- 172.21.15.67:0/161433266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2504d0 msgr2=0x55557c68d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f6849655740 1 --2- 172.21.15.67:0/161433266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2504d0 0x55557c68d750 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6834010ea0 tx=0x7f683401d790 comp rx=0 tx=0).stop 2024-07-31T15:06:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f6849655740 1 -- 172.21.15.67:0/161433266 shutdown_connections 2024-07-31T15:06:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f6849655740 1 --2- 172.21.15.67:0/161433266 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55557c2711b0 0x55557c69ac60 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f6849655740 1 --2- 172.21.15.67:0/161433266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557c2504d0 0x55557c68d750 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f6849655740 1 --2- 172.21.15.67:0/161433266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55557c2482f0 0x55557c68bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f681b7fe640 0 client.18888 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:06:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f6849655740 1 --2- 172.21.15.67:0/161433266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55557c22fd80 0x55557c68b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f6849655740 1 -- 172.21.15.67:0/161433266 >> 172.21.15.67:0/161433266 conn(0x55557c265300 msgr2=0x55557c620490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f6849655740 1 -- 172.21.15.67:0/161433266 shutdown_connections 2024-07-31T15:06:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f6849655740 1 -- 172.21.15.67:0/161433266 wait complete. 2024-07-31T15:06:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.517+0000 7f6849655740 20 client.18888 trim_cache size 0 max 16384 2024-07-31T15:06:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:06:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:06:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:06:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:06:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.518+0000 7f684275e640 1 -- 172.21.15.67:0/2928639597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68280b3d40 msgr2=0x7f68280b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.518+0000 7f684275e640 1 --2- 172.21.15.67:0/2928639597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68280b3d40 0x7f68280b6250 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f683400acc0 tx=0x7f683400a350 comp rx=0 tx=0).stop 2024-07-31T15:06:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.518+0000 7f684275e640 1 -- 172.21.15.67:0/2928639597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683c1023f0 msgr2=0x7f683c198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.518+0000 7f684275e640 1 --2- 172.21.15.67:0/2928639597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683c1023f0 0x7f683c198da0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f682c00e9f0 tx=0x7f682c00edf0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.518+0000 7f684275e640 1 -- 172.21.15.67:0/2928639597 shutdown_connections 2024-07-31T15:06:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.518+0000 7f684275e640 1 --2- 172.21.15.67:0/2928639597 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f68280b3d40 0x7f68280b6250 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.518+0000 7f684275e640 1 --2- 172.21.15.67:0/2928639597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f683c108670 0x7f683c19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.518+0000 7f684275e640 1 --2- 172.21.15.67:0/2928639597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683c107d20 0x7f683c199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.518+0000 7f684275e640 1 --2- 172.21.15.67:0/2928639597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683c1023f0 0x7f683c198da0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.518+0000 7f684275e640 1 -- 172.21.15.67:0/2928639597 >> 172.21.15.67:0/2928639597 conn(0x7f683c0fddf0 msgr2=0x7f683c100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.518+0000 7f684275e640 1 -- 172.21.15.67:0/2928639597 shutdown_connections 2024-07-31T15:06:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:06:59.518+0000 7f684275e640 1 -- 172.21.15.67:0/2928639597 wait complete. 2024-07-31T15:06:59.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:06:59 smithi067 ceph-mon[32750]: pgmap v902: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 510 KiB/s wr, 114 op/s 2024-07-31T15:06:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:06:59 smithi073 ceph-mon[36714]: pgmap v902: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 510 KiB/s wr, 114 op/s 2024-07-31T15:06:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:06:59 smithi077 ceph-mon[37327]: pgmap v902: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 510 KiB/s wr, 114 op/s 2024-07-31T15:06:59.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.977+0000 7fe447612640 1 -- 172.21.15.67:0/2118875854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe440108720 msgr2=0x7fe44010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:59.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.977+0000 7fe447612640 1 --2- 172.21.15.67:0/2118875854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe440108720 0x7fe44010ef50 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe438009ca0 tx=0x7fe43802a8c0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.977+0000 7fe447612640 1 -- 172.21.15.67:0/2118875854 shutdown_connections 2024-07-31T15:06:59.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.977+0000 7fe447612640 1 --2- 172.21.15.67:0/2118875854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe440108720 0x7fe44010ef50 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.977+0000 7fe447612640 1 --2- 172.21.15.67:0/2118875854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe440107dd0 0x7fe4401081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.977+0000 7fe447612640 1 --2- 172.21.15.67:0/2118875854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe440102490 0x7fe4401028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.977+0000 7fe447612640 1 -- 172.21.15.67:0/2118875854 >> 172.21.15.67:0/2118875854 conn(0x7fe440079160 msgr2=0x7fe440079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:59.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.978+0000 7fe447612640 1 -- 172.21.15.67:0/2118875854 shutdown_connections 2024-07-31T15:06:59.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.978+0000 7fe447612640 1 -- 172.21.15.67:0/2118875854 wait complete. 2024-07-31T15:06:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.978+0000 7fe447612640 1 Processor -- start 2024-07-31T15:06:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe447612640 1 -- start start 2024-07-31T15:06:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe447612640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe440102490 0x7fe44019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe447612640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe440107dd0 0x7fe44019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe447612640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe440108720 0x7fe44019b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe447612640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401123a0 con 0x7fe440108720 2024-07-31T15:06:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe447612640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe440112220 con 0x7fe440102490 2024-07-31T15:06:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe447612640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe440112520 con 0x7fe440107dd0 2024-07-31T15:06:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe445baf640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe440108720 0x7fe44019b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe445baf640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe440108720 0x7fe44019b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35058/0 (socket says 172.21.15.67:35058) 2024-07-31T15:06:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe445baf640 1 -- 172.21.15.67:0/40515189 learned_addr learned my addr 172.21.15.67:0/40515189 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe444bad640 1 --2- 172.21.15.67:0/40515189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe440107dd0 0x7fe44019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe4453ae640 1 --2- 172.21.15.67:0/40515189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe440102490 0x7fe44019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe445baf640 1 -- 172.21.15.67:0/40515189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe440107dd0 msgr2=0x7fe44019b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe445baf640 1 --2- 172.21.15.67:0/40515189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe440107dd0 0x7fe44019b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe445baf640 1 -- 172.21.15.67:0/40515189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe440102490 msgr2=0x7fe44019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe445baf640 1 --2- 172.21.15.67:0/40515189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe440102490 0x7fe44019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.979+0000 7fe445baf640 1 -- 172.21.15.67:0/40515189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4401a5f10 con 0x7fe440108720 2024-07-31T15:06:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.980+0000 7fe445baf640 1 --2- 172.21.15.67:0/40515189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe440108720 0x7fe44019b6d0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe438005cc0 tx=0x7fe43802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.980+0000 7fe4367fc640 1 -- 172.21.15.67:0/40515189 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438003bf0 con 0x7fe440108720 2024-07-31T15:06:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.980+0000 7fe447612640 1 -- 172.21.15.67:0/40515189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4401a61a0 con 0x7fe440108720 2024-07-31T15:06:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.980+0000 7fe4367fc640 1 -- 172.21.15.67:0/40515189 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe438003d90 con 0x7fe440108720 2024-07-31T15:06:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.980+0000 7fe4367fc640 1 -- 172.21.15.67:0/40515189 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43802fbc0 con 0x7fe440108720 2024-07-31T15:06:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.980+0000 7fe447612640 1 -- 172.21.15.67:0/40515189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4401a66e0 con 0x7fe440108720 2024-07-31T15:06:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.983+0000 7fe44db63740 1 Processor -- start 2024-07-31T15:06:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.983+0000 7fe4367fc640 1 -- 172.21.15.67:0/40515189 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe43802fd60 con 0x7fe440108720 2024-07-31T15:06:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.983+0000 7fe44db63740 1 -- start start 2024-07-31T15:06:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe44db63740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a3a070 0x55ebc5ddf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe44db63740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a111b0 0x55ebc5ddfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe44db63740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a19390 0x55ebc5ddfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe44db63740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ebc5d55bb0 con 0x55ebc5a19390 2024-07-31T15:06:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe44db63740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ebc5d450b0 con 0x55ebc5a3a070 2024-07-31T15:06:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe44db63740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ebc5d490f0 con 0x55ebc5a111b0 2024-07-31T15:06:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe444bad640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a19390 0x55ebc5ddfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe444bad640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a19390 0x55ebc5ddfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35064/0 (socket says 172.21.15.67:35064) 2024-07-31T15:06:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe444bad640 1 -- 172.21.15.67:0/2683148165 learned_addr learned my addr 172.21.15.67:0/2683148165 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe444bad640 1 --2- 172.21.15.67:0/2683148165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a3a070 0x55ebc5ddf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe4453ae640 1 --2- 172.21.15.67:0/2683148165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a111b0 0x55ebc5ddfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe4367fc640 1 --2- 172.21.15.67:0/40515189 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe42c0b3d40 0x7fe42c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe444bad640 1 -- 172.21.15.67:0/2683148165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a111b0 msgr2=0x55ebc5ddfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe444bad640 1 --2- 172.21.15.67:0/2683148165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a111b0 0x55ebc5ddfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe444bad640 1 -- 172.21.15.67:0/2683148165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a3a070 msgr2=0x55ebc5ddf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe444bad640 1 --2- 172.21.15.67:0/2683148165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a3a070 0x55ebc5ddf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe444bad640 1 -- 172.21.15.67:0/2683148165 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebc5cebb90 con 0x55ebc5a19390 2024-07-31T15:06:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.984+0000 7fe445baf640 1 --2- 172.21.15.67:0/40515189 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe42c0b3d40 0x7fe42c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.985+0000 7fe444bad640 1 --2- 172.21.15.67:0/2683148165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a19390 0x55ebc5ddfff0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fe430012770 tx=0x7fe430004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d9013a865da046 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:59.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.985+0000 7fe4367fc640 1 -- 172.21.15.67:0/40515189 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7fe438105870 con 0x7fe440108720 2024-07-31T15:06:59.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.985+0000 7fe42b7fe640 1 -- 172.21.15.67:0/2683148165 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430012cf0 con 0x55ebc5a19390 2024-07-31T15:06:59.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.985+0000 7fe42b7fe640 1 -- 172.21.15.67:0/2683148165 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe430010770 con 0x55ebc5a19390 2024-07-31T15:06:59.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.985+0000 7fe42b7fe640 1 -- 172.21.15.67:0/2683148165 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430003810 con 0x55ebc5a19390 2024-07-31T15:06:59.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.986+0000 7fe44db63740 1 -- 172.21.15.67:0/2683148165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a19390 msgr2=0x55ebc5ddfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:59.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.986+0000 7fe445baf640 1 --2- 172.21.15.67:0/40515189 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe42c0b3d40 0x7fe42c0b6250 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe438102e40 tx=0x7fe4380b7040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:59.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.986+0000 7fe44db63740 1 --2- 172.21.15.67:0/2683148165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a19390 0x55ebc5ddfff0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fe430012770 tx=0x7fe430004730 comp rx=0 tx=0).stop 2024-07-31T15:06:59.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.986+0000 7fe44db63740 1 -- 172.21.15.67:0/2683148165 shutdown_connections 2024-07-31T15:06:59.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.986+0000 7fe44db63740 1 --2- 172.21.15.67:0/2683148165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a19390 0x55ebc5ddfff0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.986+0000 7fe44db63740 1 --2- 172.21.15.67:0/2683148165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a111b0 0x55ebc5ddfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.986+0000 7fe44db63740 1 --2- 172.21.15.67:0/2683148165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a3a070 0x55ebc5ddf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.986+0000 7fe44db63740 1 -- 172.21.15.67:0/2683148165 >> 172.21.15.67:0/2683148165 conn(0x55ebc5a2e460 msgr2=0x55ebc5ddd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.986+0000 7fe44db63740 1 -- 172.21.15.67:0/2683148165 shutdown_connections 2024-07-31T15:06:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.986+0000 7fe44db63740 1 -- 172.21.15.67:0/2683148165 wait complete. 2024-07-31T15:06:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.986+0000 7fe44db63740 1 -- start start 2024-07-31T15:06:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.986+0000 7fe44db63740 1 Processor -- start 2024-07-31T15:06:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe44db63740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 0x55ebc5deb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe44db63740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a19390 0x55ebc5deb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe44db63740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a3a070 0x55ebc5df7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe44db63740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ebc5ba8700 con 0x55ebc5a3a070 2024-07-31T15:06:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe44db63740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ebc5ce79e0 con 0x55ebc5a111b0 2024-07-31T15:06:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe44db63740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ebc5dedd50 con 0x55ebc5a19390 2024-07-31T15:06:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe44db63740 10 client.0 _ll_register_callbacks cb 0x55ebc5d024f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:06:59.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe444bad640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a3a070 0x55ebc5df7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe444bad640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a3a070 0x55ebc5df7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35076/0 (socket says 172.21.15.67:35076) 2024-07-31T15:06:59.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe444bad640 1 -- 172.21.15.67:0/2958711370 learned_addr learned my addr 172.21.15.67:0/2958711370 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:06:59.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe4453ae640 1 --2- 172.21.15.67:0/2958711370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a19390 0x55ebc5deb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe444bad640 1 --2- 172.21.15.67:0/2958711370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 0x55ebc5deb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe444bad640 1 -- 172.21.15.67:0/2958711370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a19390 msgr2=0x55ebc5deb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:59.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe444bad640 1 --2- 172.21.15.67:0/2958711370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a19390 0x55ebc5deb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe444bad640 1 -- 172.21.15.67:0/2958711370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 msgr2=0x55ebc5deb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:06:59.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe444bad640 1 --2- 172.21.15.67:0/2958711370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 0x55ebc5deb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe444bad640 1 -- 172.21.15.67:0/2958711370 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebc5aa5650 con 0x55ebc5a3a070 2024-07-31T15:06:59.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.987+0000 7fe4453ae640 1 --2- 172.21.15.67:0/2958711370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a19390 0x55ebc5deb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:06:59.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe444bad640 1 --2- 172.21.15.67:0/2958711370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a3a070 0x55ebc5df7040 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fe430010eb0 tx=0x7fe43001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5192755b86651938 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:06:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe428ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:06:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe428ff9640 1 -- 172.21.15.67:0/2958711370 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430012cf0 con 0x55ebc5a3a070 2024-07-31T15:06:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe44db63740 1 -- 172.21.15.67:0/2958711370 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe430010770 con 0x55ebc5a3a070 2024-07-31T15:06:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe428ff9640 1 -- 172.21.15.67:0/2958711370 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7fe43001f660 con 0x55ebc5a3a070 2024-07-31T15:06:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe428ff9640 1 -- 172.21.15.67:0/2958711370 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe430034050 con 0x55ebc5a3a070 2024-07-31T15:06:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe44db63740 10 client.18900 fetch_fsmap learned FSMap version 65 2024-07-31T15:06:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe44db63740 1 -- 172.21.15.67:0/2958711370 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55ebc5df9de0 con 0x55ebc5a3a070 2024-07-31T15:06:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe428ff9640 1 -- 172.21.15.67:0/2958711370 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe430010770 con 0x55ebc5a3a070 2024-07-31T15:06:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe428ff9640 1 -- 172.21.15.67:0/2958711370 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fe43002b780 con 0x55ebc5a3a070 2024-07-31T15:06:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe44db63740 10 client.18900 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:06:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe44db63740 10 client.18900 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:06:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe44db63740 20 client.18900 populate_metadata read hostname 'smithi067' 2024-07-31T15:06:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe44db63740 1 --2- 172.21.15.67:0/2958711370 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ebc59f89f0 0x55ebc5dfd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:06:59.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe44db63740 4 client.18900 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:06:59.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.988+0000 7fe44db63740 1 -- 172.21.15.67:0/2958711370 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebc5d57b00 con 0x55ebc59f89f0 2024-07-31T15:06:59.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.989+0000 7fe4453ae640 1 --2- 172.21.15.67:0/2958711370 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ebc59f89f0 0x55ebc5dfd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:06:59.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.989+0000 7fe428ff9640 10 client.18900 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:06:59.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.989+0000 7fe4453ae640 1 --2- 172.21.15.67:0/2958711370 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ebc59f89f0 0x55ebc5dfd140 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70fb6a6ad0234011 server_cookie=da49474198a8cf7a in_seq=0 out_seq=0 2024-07-31T15:06:59.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.990+0000 7fe428ff9640 1 -- 172.21.15.67:0/2958711370 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ebc5d57b00 con 0x55ebc59f89f0 2024-07-31T15:06:59.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.990+0000 7fe428ff9640 10 client.18900 handle_command_reply: tid=0 2024-07-31T15:06:59.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.990+0000 7fe44db63740 1 client.18900 shutdown 2024-07-31T15:06:59.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.990+0000 7fe44db63740 1 -- 172.21.15.67:0/2958711370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a3a070 msgr2=0x55ebc5df7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:06:59.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.990+0000 7fe44db63740 1 --2- 172.21.15.67:0/2958711370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a3a070 0x55ebc5df7040 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fe430010eb0 tx=0x7fe43001b040 comp rx=0 tx=0).stop 2024-07-31T15:06:59.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.990+0000 7fe44db63740 1 -- 172.21.15.67:0/2958711370 shutdown_connections 2024-07-31T15:06:59.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.990+0000 7fe44db63740 1 --2- 172.21.15.67:0/2958711370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a3a070 0x55ebc5df7040 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.990+0000 7fe44db63740 1 --2- 172.21.15.67:0/2958711370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a19390 0x55ebc5deb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.990+0000 7fe44db63740 1 --2- 172.21.15.67:0/2958711370 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ebc59f89f0 0x55ebc5dfd140 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.990+0000 7fe44db63740 1 --2- 172.21.15.67:0/2958711370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 0x55ebc5deb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:06:59.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.990+0000 7fe44db63740 1 -- 172.21.15.67:0/2958711370 >> 172.21.15.67:0/2958711370 conn(0x55ebc5a2e460 msgr2=0x55ebc5de2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:06:59.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.990+0000 7fe428ff9640 0 client.18900 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:06:59.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.990+0000 7fe44db63740 1 -- 172.21.15.67:0/2958711370 shutdown_connections 2024-07-31T15:06:59.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.990+0000 7fe44db63740 1 -- 172.21.15.67:0/2958711370 wait complete. 2024-07-31T15:06:59.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:06:59.991+0000 7fe44db63740 20 client.18900 trim_cache size 0 max 16384 2024-07-31T15:07:00.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.004+0000 7fe44db63740 1 Processor -- start 2024-07-31T15:07:00.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.004+0000 7fe44db63740 1 -- start start 2024-07-31T15:07:00.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe44db63740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 0x55ebc5ddd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe44db63740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a19390 0x55ebc5df6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe44db63740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a3a070 0x55ebc5de47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe44db63740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401b0780 con 0x55ebc5a3a070 2024-07-31T15:07:00.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe44db63740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401b22c0 con 0x55ebc5a111b0 2024-07-31T15:07:00.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe44db63740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401b0e50 con 0x55ebc5a19390 2024-07-31T15:07:00.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe444bad640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a3a070 0x55ebc5de47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe444bad640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a3a070 0x55ebc5de47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35078/0 (socket says 172.21.15.67:35078) 2024-07-31T15:07:00.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe444bad640 1 -- 172.21.15.67:0/3697367518 learned_addr learned my addr 172.21.15.67:0/3697367518 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:00.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe444bad640 1 --2- 172.21.15.67:0/3697367518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 0x55ebc5ddd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe4453ae640 1 --2- 172.21.15.67:0/3697367518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a19390 0x55ebc5df6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe444bad640 1 -- 172.21.15.67:0/3697367518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a19390 msgr2=0x55ebc5df6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:00.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe444bad640 1 --2- 172.21.15.67:0/3697367518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a19390 0x55ebc5df6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe444bad640 1 -- 172.21.15.67:0/3697367518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 msgr2=0x55ebc5ddd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:00.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe444bad640 1 --2- 172.21.15.67:0/3697367518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 0x55ebc5ddd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe444bad640 1 -- 172.21.15.67:0/3697367518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebc5de2470 con 0x55ebc5a3a070 2024-07-31T15:07:00.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.005+0000 7fe4453ae640 1 --2- 172.21.15.67:0/3697367518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a19390 0x55ebc5df6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:07:00.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.006+0000 7fe444bad640 1 --2- 172.21.15.67:0/3697367518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a3a070 0x55ebc5de47f0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe43000bfd0 tx=0x7fe430012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72749980c0431ff0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:00.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.006+0000 7fe42affd640 1 -- 172.21.15.67:0/3697367518 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430010740 con 0x55ebc5a3a070 2024-07-31T15:07:00.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.006+0000 7fe42affd640 1 -- 172.21.15.67:0/3697367518 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe43002bd40 con 0x55ebc5a3a070 2024-07-31T15:07:00.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.006+0000 7fe42affd640 1 -- 172.21.15.67:0/3697367518 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43002b720 con 0x55ebc5a3a070 2024-07-31T15:07:00.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.006+0000 7fe44db63740 1 -- 172.21.15.67:0/3697367518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a3a070 msgr2=0x55ebc5de47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:00.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.006+0000 7fe44db63740 1 --2- 172.21.15.67:0/3697367518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a3a070 0x55ebc5de47f0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe43000bfd0 tx=0x7fe430012520 comp rx=0 tx=0).stop 2024-07-31T15:07:00.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.006+0000 7fe44db63740 1 -- 172.21.15.67:0/3697367518 shutdown_connections 2024-07-31T15:07:00.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.006+0000 7fe44db63740 1 --2- 172.21.15.67:0/3697367518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a3a070 0x55ebc5de47f0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.006+0000 7fe44db63740 1 --2- 172.21.15.67:0/3697367518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a19390 0x55ebc5df6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.006+0000 7fe44db63740 1 --2- 172.21.15.67:0/3697367518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 0x55ebc5ddd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.006+0000 7fe44db63740 1 -- 172.21.15.67:0/3697367518 >> 172.21.15.67:0/3697367518 conn(0x55ebc5a2e460 msgr2=0x55ebc5ddcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:00.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.007+0000 7fe44db63740 1 -- 172.21.15.67:0/3697367518 shutdown_connections 2024-07-31T15:07:00.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.007+0000 7fe44db63740 1 -- 172.21.15.67:0/3697367518 wait complete. 2024-07-31T15:07:00.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.007+0000 7fe44db63740 1 -- start start 2024-07-31T15:07:00.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.007+0000 7fe44db63740 1 Processor -- start 2024-07-31T15:07:00.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe44db63740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 0x55ebc5e023e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe44db63740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a19390 0x55ebc5e02920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe44db63740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a3a070 0x55ebc5e02e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe44db63740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401b1bf0 con 0x55ebc5a19390 2024-07-31T15:07:00.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe44db63740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401b1520 con 0x55ebc5a111b0 2024-07-31T15:07:00.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe44db63740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401ae780 con 0x55ebc5a3a070 2024-07-31T15:07:00.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe44db63740 10 client.0 _ll_register_callbacks cb 0x7fe43000be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:00.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe4453ae640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a19390 0x55ebc5e02920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe4453ae640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a19390 0x55ebc5e02920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35092/0 (socket says 172.21.15.67:35092) 2024-07-31T15:07:00.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe4453ae640 1 -- 172.21.15.67:0/1336068039 learned_addr learned my addr 172.21.15.67:0/1336068039 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:00.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe444bad640 1 --2- 172.21.15.67:0/1336068039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 0x55ebc5e023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe444bad640 1 --2- 172.21.15.67:0/1336068039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a3a070 0x55ebc5e02e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe4453ae640 1 -- 172.21.15.67:0/1336068039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a3a070 msgr2=0x55ebc5e02e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:00.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe4453ae640 1 --2- 172.21.15.67:0/1336068039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a3a070 0x55ebc5e02e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe4453ae640 1 -- 172.21.15.67:0/1336068039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 msgr2=0x55ebc5e023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:00.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe4453ae640 1 --2- 172.21.15.67:0/1336068039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 0x55ebc5e023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe4453ae640 1 -- 172.21.15.67:0/1336068039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebc5df24f0 con 0x55ebc5a19390 2024-07-31T15:07:00.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe444bad640 1 --2- 172.21.15.67:0/1336068039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a3a070 0x55ebc5e02e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:07:00.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe4453ae640 1 --2- 172.21.15.67:0/1336068039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a19390 0x55ebc5e02920 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe44006b430 tx=0x7fe43c00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5a70b8add3b1ad server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:00.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe428ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:07:00.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe428ff9640 1 -- 172.21.15.67:0/1336068039 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c005220 con 0x55ebc5a19390 2024-07-31T15:07:00.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe44db63740 1 -- 172.21.15.67:0/1336068039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe43002bd40 con 0x55ebc5a19390 2024-07-31T15:07:00.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.008+0000 7fe428ff9640 1 -- 172.21.15.67:0/1336068039 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7fe43c0053c0 con 0x55ebc5a19390 2024-07-31T15:07:00.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.009+0000 7fe428ff9640 1 -- 172.21.15.67:0/1336068039 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe43c030050 con 0x55ebc5a19390 2024-07-31T15:07:00.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.009+0000 7fe44db63740 10 client.18906 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:00.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.009+0000 7fe44db63740 1 -- 172.21.15.67:0/1336068039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55ebc5e658f0 con 0x55ebc5a19390 2024-07-31T15:07:00.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.009+0000 7fe428ff9640 1 -- 172.21.15.67:0/1336068039 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe43002bd40 con 0x55ebc5a19390 2024-07-31T15:07:00.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.009+0000 7fe428ff9640 1 -- 172.21.15.67:0/1336068039 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fe43c024030 con 0x55ebc5a19390 2024-07-31T15:07:00.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.009+0000 7fe44db63740 10 client.18906 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:00.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.009+0000 7fe44db63740 10 client.18906 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:07:00.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.009+0000 7fe44db63740 20 client.18906 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:00.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.009+0000 7fe44db63740 1 --2- 172.21.15.67:0/1336068039 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ebc59cdda0 0x55ebc5e68130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.009+0000 7fe44db63740 4 client.18906 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:07:00.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.009+0000 7fe44db63740 1 -- 172.21.15.67:0/1336068039 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ebc5d535a0 con 0x55ebc59cdda0 2024-07-31T15:07:00.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.010+0000 7fe444bad640 1 --2- 172.21.15.67:0/1336068039 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ebc59cdda0 0x55ebc5e68130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.010+0000 7fe444bad640 1 --2- 172.21.15.67:0/1336068039 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ebc59cdda0 0x55ebc5e68130 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f76aadb11bc7a59 server_cookie=b00ca9e6de9d138c in_seq=0 out_seq=0 2024-07-31T15:07:00.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.010+0000 7fe428ff9640 10 client.18906 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:07:00.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.010+0000 7fe428ff9640 1 -- 172.21.15.67:0/1336068039 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ebc5d535a0 con 0x55ebc59cdda0 2024-07-31T15:07:00.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.010+0000 7fe428ff9640 10 client.18906 handle_command_reply: tid=0 2024-07-31T15:07:00.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.010+0000 7fe44db63740 1 client.18906 shutdown 2024-07-31T15:07:00.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.010+0000 7fe44db63740 1 -- 172.21.15.67:0/1336068039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a19390 msgr2=0x55ebc5e02920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:00.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.010+0000 7fe44db63740 1 --2- 172.21.15.67:0/1336068039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a19390 0x55ebc5e02920 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe44006b430 tx=0x7fe43c00beb0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe44db63740 1 -- 172.21.15.67:0/1336068039 shutdown_connections 2024-07-31T15:07:00.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe44db63740 1 --2- 172.21.15.67:0/1336068039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ebc5a3a070 0x55ebc5e02e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe44db63740 1 --2- 172.21.15.67:0/1336068039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebc5a19390 0x55ebc5e02920 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe44db63740 1 --2- 172.21.15.67:0/1336068039 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55ebc59cdda0 0x55ebc5e68130 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe44db63740 1 --2- 172.21.15.67:0/1336068039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc5a111b0 0x55ebc5e023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe44db63740 1 -- 172.21.15.67:0/1336068039 >> 172.21.15.67:0/1336068039 conn(0x55ebc5a2e460 msgr2=0x55ebc5ddbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:00.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe428ff9640 0 client.18906 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:07:00.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe44db63740 1 -- 172.21.15.67:0/1336068039 shutdown_connections 2024-07-31T15:07:00.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe44db63740 1 -- 172.21.15.67:0/1336068039 wait complete. 2024-07-31T15:07:00.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe44db63740 20 client.18906 trim_cache size 0 max 16384 2024-07-31T15:07:00.020 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:07:00.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe447612640 1 -- 172.21.15.67:0/40515189 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe42c0b3d40 msgr2=0x7fe42c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:00.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe447612640 1 --2- 172.21.15.67:0/40515189 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe42c0b3d40 0x7fe42c0b6250 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe438102e40 tx=0x7fe4380b7040 comp rx=0 tx=0).stop 2024-07-31T15:07:00.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe447612640 1 -- 172.21.15.67:0/40515189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe440108720 msgr2=0x7fe44019b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:00.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe447612640 1 --2- 172.21.15.67:0/40515189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe440108720 0x7fe44019b6d0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe438005cc0 tx=0x7fe43802f380 comp rx=0 tx=0).stop 2024-07-31T15:07:00.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe447612640 1 -- 172.21.15.67:0/40515189 shutdown_connections 2024-07-31T15:07:00.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe447612640 1 --2- 172.21.15.67:0/40515189 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe42c0b3d40 0x7fe42c0b6250 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe447612640 1 --2- 172.21.15.67:0/40515189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe440108720 0x7fe44019b6d0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe447612640 1 --2- 172.21.15.67:0/40515189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe440107dd0 0x7fe44019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe447612640 1 --2- 172.21.15.67:0/40515189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe440102490 0x7fe44019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.011+0000 7fe447612640 1 -- 172.21.15.67:0/40515189 >> 172.21.15.67:0/40515189 conn(0x7fe440079160 msgr2=0x7fe440103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:00.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.012+0000 7fe447612640 1 -- 172.21.15.67:0/40515189 shutdown_connections 2024-07-31T15:07:00.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.012+0000 7fe447612640 1 -- 172.21.15.67:0/40515189 wait complete. 2024-07-31T15:07:00.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.819+0000 7fc93a2ad640 1 -- 172.21.15.67:0/1369566482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934107dd0 msgr2=0x7fc9341081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:00.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.819+0000 7fc93a2ad640 1 --2- 172.21.15.67:0/1369566482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934107dd0 0x7fc9341081e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fc928009900 tx=0x7fc92802a3f0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.819+0000 7fc93a2ad640 1 -- 172.21.15.67:0/1369566482 shutdown_connections 2024-07-31T15:07:00.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.819+0000 7fc93a2ad640 1 --2- 172.21.15.67:0/1369566482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc934108720 0x7fc93410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.819+0000 7fc93a2ad640 1 --2- 172.21.15.67:0/1369566482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934107dd0 0x7fc9341081e0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.819+0000 7fc93a2ad640 1 --2- 172.21.15.67:0/1369566482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc934102490 0x7fc9341028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.819+0000 7fc93a2ad640 1 -- 172.21.15.67:0/1369566482 >> 172.21.15.67:0/1369566482 conn(0x7fc934079160 msgr2=0x7fc934079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:00.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.819+0000 7fc93a2ad640 1 -- 172.21.15.67:0/1369566482 shutdown_connections 2024-07-31T15:07:00.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.819+0000 7fc93a2ad640 1 -- 172.21.15.67:0/1369566482 wait complete. 2024-07-31T15:07:00.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.820+0000 7fc93a2ad640 1 Processor -- start 2024-07-31T15:07:00.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.820+0000 7fc93a2ad640 1 -- start start 2024-07-31T15:07:00.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.820+0000 7fc93a2ad640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc934102490 0x7fc934199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.820+0000 7fc93a2ad640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc934107dd0 0x7fc93419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.820+0000 7fc93a2ad640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934108720 0x7fc93419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.820+0000 7fc93a2ad640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341134b0 con 0x7fc934108720 2024-07-31T15:07:00.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.820+0000 7fc93a2ad640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc934113330 con 0x7fc934102490 2024-07-31T15:07:00.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.820+0000 7fc93a2ad640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc934113630 con 0x7fc934107dd0 2024-07-31T15:07:00.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.820+0000 7fc93884a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934108720 0x7fc93419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.820+0000 7fc93884a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934108720 0x7fc93419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35122/0 (socket says 172.21.15.67:35122) 2024-07-31T15:07:00.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.820+0000 7fc93884a640 1 -- 172.21.15.67:0/2311248580 learned_addr learned my addr 172.21.15.67:0/2311248580 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:00.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.820+0000 7fc9337fe640 1 --2- 172.21.15.67:0/2311248580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc934107dd0 0x7fc93419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.820+0000 7fc933fff640 1 --2- 172.21.15.67:0/2311248580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc934102490 0x7fc934199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.821+0000 7fc93884a640 1 -- 172.21.15.67:0/2311248580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc934107dd0 msgr2=0x7fc93419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:00.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.821+0000 7fc93884a640 1 --2- 172.21.15.67:0/2311248580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc934107dd0 0x7fc93419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.821+0000 7fc93884a640 1 -- 172.21.15.67:0/2311248580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc934102490 msgr2=0x7fc934199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:00.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.821+0000 7fc93884a640 1 --2- 172.21.15.67:0/2311248580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc934102490 0x7fc934199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.821+0000 7fc93884a640 1 -- 172.21.15.67:0/2311248580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93419fd30 con 0x7fc934108720 2024-07-31T15:07:00.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.821+0000 7fc93884a640 1 --2- 172.21.15.67:0/2311248580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934108720 0x7fc93419f6c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00cce0 tx=0x7fc92c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:00.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.821+0000 7fc9317fa640 1 -- 172.21.15.67:0/2311248580 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00f450 con 0x7fc934108720 2024-07-31T15:07:00.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.821+0000 7fc9317fa640 1 -- 172.21.15.67:0/2311248580 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc92c00faa0 con 0x7fc934108720 2024-07-31T15:07:00.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.821+0000 7fc93a2ad640 1 -- 172.21.15.67:0/2311248580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9341a0020 con 0x7fc934108720 2024-07-31T15:07:00.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.821+0000 7fc9317fa640 1 -- 172.21.15.67:0/2311248580 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c018a80 con 0x7fc934108720 2024-07-31T15:07:00.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.821+0000 7fc93a2ad640 1 -- 172.21.15.67:0/2311248580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9341a04e0 con 0x7fc934108720 2024-07-31T15:07:00.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.824+0000 7fc9407fe740 1 Processor -- start 2024-07-31T15:07:00.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.824+0000 7fc9317fa640 1 -- 172.21.15.67:0/2311248580 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc92c0213f0 con 0x7fc934108720 2024-07-31T15:07:00.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.824+0000 7fc9407fe740 1 -- start start 2024-07-31T15:07:00.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.824+0000 7fc9407fe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c505b520 0x5628c5401070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.824+0000 7fc9407fe740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5032660 0x5628c54015b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.824+0000 7fc9407fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 0x5628c5401af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.824+0000 7fc9407fe740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5628c5376e70 con 0x5628c5032660 2024-07-31T15:07:00.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.824+0000 7fc9407fe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5628c5366370 con 0x5628c505b520 2024-07-31T15:07:00.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.824+0000 7fc9407fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5628c536a3b0 con 0x5628c503a840 2024-07-31T15:07:00.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.825+0000 7fc933fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5032660 0x5628c54015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.825+0000 7fc933fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5032660 0x5628c54015b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35134/0 (socket says 172.21.15.67:35134) 2024-07-31T15:07:00.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.825+0000 7fc933fff640 1 -- 172.21.15.67:0/4022492268 learned_addr learned my addr 172.21.15.67:0/4022492268 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:00.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.825+0000 7fc9337fe640 1 --2- 172.21.15.67:0/4022492268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c505b520 0x5628c5401070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.825+0000 7fc9337fe640 1 --2- 172.21.15.67:0/4022492268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 0x5628c5401af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.825+0000 7fc933fff640 1 -- 172.21.15.67:0/4022492268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 msgr2=0x5628c5401af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:00.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.825+0000 7fc933fff640 1 --2- 172.21.15.67:0/4022492268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 0x5628c5401af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.825+0000 7fc933fff640 1 -- 172.21.15.67:0/4022492268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c505b520 msgr2=0x5628c5401070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:00.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.825+0000 7fc933fff640 1 --2- 172.21.15.67:0/4022492268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c505b520 0x5628c5401070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.825+0000 7fc933fff640 1 -- 172.21.15.67:0/4022492268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c5333040 con 0x5628c5032660 2024-07-31T15:07:00.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.825+0000 7fc933fff640 1 --2- 172.21.15.67:0/4022492268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5032660 0x5628c54015b0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fc92400ad00 tx=0x7fc92400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be248207a5331cd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:00.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.825+0000 7fc9317fa640 1 --2- 172.21.15.67:0/2311248580 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc91c0b3d70 0x7fc91c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.825+0000 7fc91a7fc640 1 -- 172.21.15.67:0/4022492268 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92401d070 con 0x5628c5032660 2024-07-31T15:07:00.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.825+0000 7fc9337fe640 1 --2- 172.21.15.67:0/2311248580 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc91c0b3d70 0x7fc91c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.826+0000 7fc9317fa640 1 -- 172.21.15.67:0/2311248580 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7fc92c0e4730 con 0x7fc934108720 2024-07-31T15:07:00.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.826+0000 7fc91a7fc640 1 -- 172.21.15.67:0/4022492268 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc924018af0 con 0x5628c5032660 2024-07-31T15:07:00.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.826+0000 7fc91a7fc640 1 -- 172.21.15.67:0/4022492268 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924010af0 con 0x5628c5032660 2024-07-31T15:07:00.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.827+0000 7fc9407fe740 1 -- 172.21.15.67:0/4022492268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5032660 msgr2=0x5628c54015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:00.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.827+0000 7fc9337fe640 1 --2- 172.21.15.67:0/2311248580 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc91c0b3d70 0x7fc91c0b6280 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc928000c00 tx=0x7fc928009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:00.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.827+0000 7fc9407fe740 1 --2- 172.21.15.67:0/4022492268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5032660 0x5628c54015b0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fc92400ad00 tx=0x7fc92400f790 comp rx=0 tx=0).stop 2024-07-31T15:07:00.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.827+0000 7fc9407fe740 1 -- 172.21.15.67:0/4022492268 shutdown_connections 2024-07-31T15:07:00.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.827+0000 7fc9407fe740 1 --2- 172.21.15.67:0/4022492268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 0x5628c5401af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.827+0000 7fc9407fe740 1 --2- 172.21.15.67:0/4022492268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5032660 0x5628c54015b0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.827+0000 7fc9407fe740 1 --2- 172.21.15.67:0/4022492268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c505b520 0x5628c5401070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.827+0000 7fc9407fe740 1 -- 172.21.15.67:0/4022492268 >> 172.21.15.67:0/4022492268 conn(0x5628c504f910 msgr2=0x5628c53fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:00.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.827+0000 7fc9407fe740 1 -- 172.21.15.67:0/4022492268 shutdown_connections 2024-07-31T15:07:00.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.827+0000 7fc9407fe740 1 -- 172.21.15.67:0/4022492268 wait complete. 2024-07-31T15:07:00.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.827+0000 7fc9407fe740 1 -- start start 2024-07-31T15:07:00.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.827+0000 7fc9407fe740 1 Processor -- start 2024-07-31T15:07:00.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc9407fe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c5032660 0x5628c5411070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc9407fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 0x5628c54115b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc9407fe740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c505b520 0x5628c541cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc9407fe740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5628c51c9bb0 con 0x5628c505b520 2024-07-31T15:07:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc9407fe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5628c5308ac0 con 0x5628c5032660 2024-07-31T15:07:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc9407fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5628c5413bb0 con 0x5628c503a840 2024-07-31T15:07:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc9407fe740 10 client.0 _ll_register_callbacks cb 0x5628c53113b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc93884a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c505b520 0x5628c541cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc93884a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c505b520 0x5628c541cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55728/0 (socket says 172.21.15.67:55728) 2024-07-31T15:07:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc93884a640 1 -- 172.21.15.67:0/2453848950 learned_addr learned my addr 172.21.15.67:0/2453848950 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc9337fe640 1 --2- 172.21.15.67:0/2453848950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 0x5628c54115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc933fff640 1 --2- 172.21.15.67:0/2453848950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c5032660 0x5628c5411070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc93884a640 1 -- 172.21.15.67:0/2453848950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 msgr2=0x5628c54115b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc93884a640 1 --2- 172.21.15.67:0/2453848950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 0x5628c54115b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc93884a640 1 -- 172.21.15.67:0/2453848950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c5032660 msgr2=0x5628c5411070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:00.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc93884a640 1 --2- 172.21.15.67:0/2453848950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c5032660 0x5628c5411070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.828+0000 7fc93884a640 1 -- 172.21.15.67:0/2453848950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c50c6b00 con 0x5628c505b520 2024-07-31T15:07:00.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc93884a640 1 --2- 172.21.15.67:0/2453848950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c505b520 0x5628c541cf10 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0aaf90 tx=0x7fc92c00fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9595c17fdf50d3fb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:00.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9137fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:07:00.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9137fe640 1 -- 172.21.15.67:0/2453848950 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0af600 con 0x5628c505b520 2024-07-31T15:07:00.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9137fe640 1 -- 172.21.15.67:0/2453848950 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7fc92c0e4390 con 0x5628c505b520 2024-07-31T15:07:00.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9407fe740 1 -- 172.21.15.67:0/2453848950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc924018af0 con 0x5628c505b520 2024-07-31T15:07:00.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9137fe640 1 -- 172.21.15.67:0/2453848950 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fc92c0e4590 con 0x5628c505b520 2024-07-31T15:07:00.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9407fe740 10 client.18918 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:00.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9407fe740 1 -- 172.21.15.67:0/2453848950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5628c513d960 con 0x5628c505b520 2024-07-31T15:07:00.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9137fe640 1 -- 172.21.15.67:0/2453848950 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc924018af0 con 0x5628c505b520 2024-07-31T15:07:00.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9137fe640 1 -- 172.21.15.67:0/2453848950 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fc92c020030 con 0x5628c505b520 2024-07-31T15:07:00.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9407fe740 10 client.18918 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9407fe740 10 client.18918 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:07:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9407fe740 20 client.18918 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9407fe740 1 --2- 172.21.15.67:0/2453848950 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c5019ea0 0x5628c54232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9407fe740 4 client.18918 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:07:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.829+0000 7fc9407fe740 1 -- 172.21.15.67:0/2453848950 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628c5378dc0 con 0x5628c5019ea0 2024-07-31T15:07:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.830+0000 7fc933fff640 1 --2- 172.21.15.67:0/2453848950 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c5019ea0 0x5628c54232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.830+0000 7fc9137fe640 10 client.18918 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.830+0000 7fc933fff640 1 --2- 172.21.15.67:0/2453848950 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c5019ea0 0x5628c54232c0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96a7701c3ac763d1 server_cookie=1aa0f4680f9c13d in_seq=0 out_seq=0 2024-07-31T15:07:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.831+0000 7fc9137fe640 1 -- 172.21.15.67:0/2453848950 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628c5378dc0 con 0x5628c5019ea0 2024-07-31T15:07:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.831+0000 7fc9137fe640 10 client.18918 handle_command_reply: tid=0 2024-07-31T15:07:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.831+0000 7fc9407fe740 1 client.18918 shutdown 2024-07-31T15:07:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.831+0000 7fc9407fe740 1 -- 172.21.15.67:0/2453848950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c505b520 msgr2=0x5628c541cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.831+0000 7fc9407fe740 1 --2- 172.21.15.67:0/2453848950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c505b520 0x5628c541cf10 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0aaf90 tx=0x7fc92c00fed0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.831+0000 7fc9407fe740 1 -- 172.21.15.67:0/2453848950 shutdown_connections 2024-07-31T15:07:00.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.831+0000 7fc9407fe740 1 --2- 172.21.15.67:0/2453848950 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c5019ea0 0x5628c54232c0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.831+0000 7fc9407fe740 1 --2- 172.21.15.67:0/2453848950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c505b520 0x5628c541cf10 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.831+0000 7fc9137fe640 0 client.18918 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:00.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.831+0000 7fc9407fe740 1 --2- 172.21.15.67:0/2453848950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 0x5628c54115b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.831+0000 7fc9407fe740 1 --2- 172.21.15.67:0/2453848950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c5032660 0x5628c5411070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.831+0000 7fc9407fe740 1 -- 172.21.15.67:0/2453848950 >> 172.21.15.67:0/2453848950 conn(0x5628c504f910 msgr2=0x5628c5402a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:00.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.831+0000 7fc9407fe740 1 -- 172.21.15.67:0/2453848950 shutdown_connections 2024-07-31T15:07:00.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.831+0000 7fc9407fe740 1 -- 172.21.15.67:0/2453848950 wait complete. 2024-07-31T15:07:00.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.832+0000 7fc9407fe740 20 client.18918 trim_cache size 0 max 16384 2024-07-31T15:07:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.850+0000 7fc9407fe740 1 Processor -- start 2024-07-31T15:07:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.850+0000 7fc9407fe740 1 -- start start 2024-07-31T15:07:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.850+0000 7fc9407fe740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5019ea0 0x5628c541cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.850+0000 7fc9407fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c5032660 0x5628c54027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.850+0000 7fc9407fe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c503a840 0x5628c5428890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.850+0000 7fc9407fe740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341ae2a0 con 0x5628c5019ea0 2024-07-31T15:07:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.850+0000 7fc9407fe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341afde0 con 0x5628c503a840 2024-07-31T15:07:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.850+0000 7fc9407fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341ae970 con 0x5628c5032660 2024-07-31T15:07:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.850+0000 7fc933fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5019ea0 0x5628c541cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.850+0000 7fc933fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5019ea0 0x5628c541cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55740/0 (socket says 172.21.15.67:55740) 2024-07-31T15:07:00.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.850+0000 7fc933fff640 1 -- 172.21.15.67:0/1838491387 learned_addr learned my addr 172.21.15.67:0/1838491387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:00.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.850+0000 7fc9337fe640 1 --2- 172.21.15.67:0/1838491387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c5032660 0x5628c54027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.851+0000 7fc93884a640 1 --2- 172.21.15.67:0/1838491387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c503a840 0x5628c5428890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.851+0000 7fc933fff640 1 -- 172.21.15.67:0/1838491387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c5032660 msgr2=0x5628c54027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:00.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.851+0000 7fc933fff640 1 --2- 172.21.15.67:0/1838491387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c5032660 0x5628c54027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.851+0000 7fc933fff640 1 -- 172.21.15.67:0/1838491387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c503a840 msgr2=0x5628c5428890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:00.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.851+0000 7fc933fff640 1 --2- 172.21.15.67:0/1838491387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c503a840 0x5628c5428890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.851+0000 7fc933fff640 1 -- 172.21.15.67:0/1838491387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c540bac0 con 0x5628c5019ea0 2024-07-31T15:07:00.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.851+0000 7fc933fff640 1 --2- 172.21.15.67:0/1838491387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5019ea0 0x5628c541cd00 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fc924010620 tx=0x7fc924005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6def74f750cac5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:00.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.851+0000 7fc919ffb640 1 -- 172.21.15.67:0/1838491387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924033070 con 0x5628c5019ea0 2024-07-31T15:07:00.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.851+0000 7fc919ffb640 1 -- 172.21.15.67:0/1838491387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc92401e040 con 0x5628c5019ea0 2024-07-31T15:07:00.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.851+0000 7fc919ffb640 1 -- 172.21.15.67:0/1838491387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92401d040 con 0x5628c5019ea0 2024-07-31T15:07:00.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.852+0000 7fc9407fe740 1 -- 172.21.15.67:0/1838491387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5019ea0 msgr2=0x5628c541cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:00.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.852+0000 7fc9407fe740 1 --2- 172.21.15.67:0/1838491387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5019ea0 0x5628c541cd00 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fc924010620 tx=0x7fc924005850 comp rx=0 tx=0).stop 2024-07-31T15:07:00.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.852+0000 7fc9407fe740 1 -- 172.21.15.67:0/1838491387 shutdown_connections 2024-07-31T15:07:00.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.852+0000 7fc9407fe740 1 --2- 172.21.15.67:0/1838491387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c503a840 0x5628c5428890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.852+0000 7fc9407fe740 1 --2- 172.21.15.67:0/1838491387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c5032660 0x5628c54027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.852+0000 7fc9407fe740 1 --2- 172.21.15.67:0/1838491387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5019ea0 0x5628c541cd00 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.852+0000 7fc9407fe740 1 -- 172.21.15.67:0/1838491387 >> 172.21.15.67:0/1838491387 conn(0x5628c504f910 msgr2=0x5628c54020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:00.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.852+0000 7fc9407fe740 1 -- 172.21.15.67:0/1838491387 shutdown_connections 2024-07-31T15:07:00.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.852+0000 7fc9407fe740 1 -- 172.21.15.67:0/1838491387 wait complete. 2024-07-31T15:07:00.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.852+0000 7fc9407fe740 1 -- start start 2024-07-31T15:07:00.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.853+0000 7fc9407fe740 1 Processor -- start 2024-07-31T15:07:00.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.853+0000 7fc9407fe740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5019ea0 0x5628c5415790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.853+0000 7fc9407fe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c5032660 0x5628c5415d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.853+0000 7fc9407fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 0x5628c54848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.853+0000 7fc9407fe740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341af710 con 0x5628c5019ea0 2024-07-31T15:07:00.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.853+0000 7fc9407fe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341af040 con 0x5628c5032660 2024-07-31T15:07:00.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.853+0000 7fc9407fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341ac2a0 con 0x5628c503a840 2024-07-31T15:07:00.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.853+0000 7fc9407fe740 10 client.0 _ll_register_callbacks cb 0x7fc92c018420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:00.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.853+0000 7fc933fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5019ea0 0x5628c5415790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.853+0000 7fc933fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5019ea0 0x5628c5415790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55754/0 (socket says 172.21.15.67:55754) 2024-07-31T15:07:00.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.853+0000 7fc933fff640 1 -- 172.21.15.67:0/3864001605 learned_addr learned my addr 172.21.15.67:0/3864001605 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.853+0000 7fc93884a640 1 --2- 172.21.15.67:0/3864001605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 0x5628c54848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.853+0000 7fc9337fe640 1 --2- 172.21.15.67:0/3864001605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c5032660 0x5628c5415d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.854+0000 7fc933fff640 1 -- 172.21.15.67:0/3864001605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 msgr2=0x5628c54848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.854+0000 7fc933fff640 1 --2- 172.21.15.67:0/3864001605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 0x5628c54848c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.854+0000 7fc933fff640 1 -- 172.21.15.67:0/3864001605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c5032660 msgr2=0x5628c5415d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.854+0000 7fc933fff640 1 --2- 172.21.15.67:0/3864001605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c5032660 0x5628c5415d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.854+0000 7fc933fff640 1 -- 172.21.15.67:0/3864001605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c5484f30 con 0x5628c5019ea0 2024-07-31T15:07:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.854+0000 7fc933fff640 1 --2- 172.21.15.67:0/3864001605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5019ea0 0x5628c5415790 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fc92400ac50 tx=0x7fc924018590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf426c2fa6cd0eb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.854+0000 7fc9137fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:07:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.854+0000 7fc9137fe640 1 -- 172.21.15.67:0/3864001605 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924033070 con 0x5628c5019ea0 2024-07-31T15:07:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.854+0000 7fc9137fe640 1 -- 172.21.15.67:0/3864001605 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7fc924018910 con 0x5628c5019ea0 2024-07-31T15:07:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.854+0000 7fc9407fe740 1 -- 172.21.15.67:0/3864001605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5628c5411940 con 0x5628c5019ea0 2024-07-31T15:07:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.854+0000 7fc9137fe640 1 -- 172.21.15.67:0/3864001605 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fc924041050 con 0x5628c5019ea0 2024-07-31T15:07:00.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.854+0000 7fc9407fe740 10 client.18924 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:00.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.854+0000 7fc9407fe740 1 -- 172.21.15.67:0/3864001605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5628c548c360 con 0x5628c5019ea0 2024-07-31T15:07:00.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.854+0000 7fc9137fe640 1 -- 172.21.15.67:0/3864001605 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5628c5411940 con 0x5628c5019ea0 2024-07-31T15:07:00.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.855+0000 7fc9137fe640 1 -- 172.21.15.67:0/3864001605 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fc924010ab0 con 0x5628c5019ea0 2024-07-31T15:07:00.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.855+0000 7fc9407fe740 10 client.18924 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:00.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.855+0000 7fc9407fe740 10 client.18924 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:07:00.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.855+0000 7fc9407fe740 20 client.18924 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:00.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.855+0000 7fc9407fe740 1 --2- 172.21.15.67:0/3864001605 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c505b520 0x5628c548e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:00.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.855+0000 7fc9407fe740 4 client.18924 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cc95e00d", "release": true, "await": true}] 2024-07-31T15:07:00.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.855+0000 7fc9407fe740 1 -- 172.21.15.67:0/3864001605 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cc95e00d", "release": true, "await": true}) -- 0x5628c54106d0 con 0x5628c505b520 2024-07-31T15:07:00.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.855+0000 7fc9337fe640 1 --2- 172.21.15.67:0/3864001605 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c505b520 0x5628c548e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:00.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.855+0000 7fc9337fe640 1 --2- 172.21.15.67:0/3864001605 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c505b520 0x5628c548e7f0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76c18c7c5ed8138c server_cookie=786d9601e372265a in_seq=0 out_seq=0 2024-07-31T15:07:00.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:00.855+0000 7fc9137fe640 10 client.18924 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:01.402 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T15:07:01.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:01 smithi067 ceph-mon[32750]: pgmap v903: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 353 KiB/s wr, 95 op/s 2024-07-31T15:07:01.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:01 smithi073 ceph-mon[36714]: pgmap v903: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 353 KiB/s wr, 95 op/s 2024-07-31T15:07:01.668 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:07:01.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:01 smithi077 ceph-mon[37327]: pgmap v903: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 353 KiB/s wr, 95 op/s 2024-07-31T15:07:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.114+0000 7fc9137fe640 1 -- 172.21.15.67:0/3864001605 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5628c54106d0 con 0x5628c505b520 2024-07-31T15:07:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.114+0000 7fc9137fe640 10 client.18924 handle_command_reply: tid=0 2024-07-31T15:07:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.114+0000 7fc9407fe740 1 client.18924 shutdown 2024-07-31T15:07:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.114+0000 7fc9407fe740 1 -- 172.21.15.67:0/3864001605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5019ea0 msgr2=0x5628c5415790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.114+0000 7fc9407fe740 1 --2- 172.21.15.67:0/3864001605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5019ea0 0x5628c5415790 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fc92400ac50 tx=0x7fc924018590 comp rx=0 tx=0).stop 2024-07-31T15:07:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.115+0000 7fc9407fe740 1 -- 172.21.15.67:0/3864001605 shutdown_connections 2024-07-31T15:07:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.115+0000 7fc9407fe740 1 --2- 172.21.15.67:0/3864001605 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5628c505b520 0x5628c548e7f0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.115+0000 7fc9407fe740 1 --2- 172.21.15.67:0/3864001605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628c503a840 0x5628c54848c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:03.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.115+0000 7fc9407fe740 1 --2- 172.21.15.67:0/3864001605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5628c5032660 0x5628c5415d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:03.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.115+0000 7fc9137fe640 0 client.18924 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:03.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.115+0000 7fc9407fe740 1 --2- 172.21.15.67:0/3864001605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5628c5019ea0 0x5628c5415790 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:03.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.115+0000 7fc9407fe740 1 -- 172.21.15.67:0/3864001605 >> 172.21.15.67:0/3864001605 conn(0x5628c504f910 msgr2=0x5628c53fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:03.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.115+0000 7fc9407fe740 1 -- 172.21.15.67:0/3864001605 shutdown_connections 2024-07-31T15:07:03.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.115+0000 7fc9407fe740 1 -- 172.21.15.67:0/3864001605 wait complete. 2024-07-31T15:07:03.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.115+0000 7fc9407fe740 20 client.18924 trim_cache size 0 max 16384 2024-07-31T15:07:03.118 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 44, 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: "cc95e00d": { 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 44, 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 137.8, 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:07:03.119 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:07:03.120 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:07:03.120 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:07:03.120 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:07:03.120 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:07:03.120 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:07:03.120 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:07:03.120 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:07:03.120 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:07:03.120 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:07:03.120 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:07:03.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.116+0000 7fc93a2ad640 1 -- 172.21.15.67:0/2311248580 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc91c0b3d70 msgr2=0x7fc91c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:03.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.116+0000 7fc93a2ad640 1 --2- 172.21.15.67:0/2311248580 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc91c0b3d70 0x7fc91c0b6280 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc928000c00 tx=0x7fc928009560 comp rx=0 tx=0).stop 2024-07-31T15:07:03.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.116+0000 7fc93a2ad640 1 -- 172.21.15.67:0/2311248580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934108720 msgr2=0x7fc93419f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:03.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.116+0000 7fc93a2ad640 1 --2- 172.21.15.67:0/2311248580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934108720 0x7fc93419f6c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00cce0 tx=0x7fc92c00a620 comp rx=0 tx=0).stop 2024-07-31T15:07:03.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.116+0000 7fc93a2ad640 1 -- 172.21.15.67:0/2311248580 shutdown_connections 2024-07-31T15:07:03.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.116+0000 7fc93a2ad640 1 --2- 172.21.15.67:0/2311248580 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc91c0b3d70 0x7fc91c0b6280 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:03.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.116+0000 7fc93a2ad640 1 --2- 172.21.15.67:0/2311248580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc934108720 0x7fc93419f6c0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:03.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.116+0000 7fc93a2ad640 1 --2- 172.21.15.67:0/2311248580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc934107dd0 0x7fc93419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:03.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.116+0000 7fc93a2ad640 1 --2- 172.21.15.67:0/2311248580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc934102490 0x7fc934199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:03.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.116+0000 7fc93a2ad640 1 -- 172.21.15.67:0/2311248580 >> 172.21.15.67:0/2311248580 conn(0x7fc934079160 msgr2=0x7fc934104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:03.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.117+0000 7fc93a2ad640 1 -- 172.21.15.67:0/2311248580 shutdown_connections 2024-07-31T15:07:03.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.117+0000 7fc93a2ad640 1 -- 172.21.15.67:0/2311248580 wait complete. 2024-07-31T15:07:03.309 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/percpu-refcount.o 2024-07-31T15:07:03.311 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/kallsyms.o 2024-07-31T15:07:03.312 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/fib_notifier.o 2024-07-31T15:07:03.321 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/ethernet/built-in.a 2024-07-31T15:07:03.342 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/rhashtable.o 2024-07-31T15:07:03.345 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsarguments.o 2024-07-31T15:07:03.359 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsconvert.o 2024-07-31T15:07:03.363 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/callback.o 2024-07-31T15:07:03.381 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/callback_xdr.o 2024-07-31T15:07:03.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:03 smithi067 ceph-mon[32750]: pgmap v904: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 187 KiB/s wr, 90 op/s 2024-07-31T15:07:03.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:03 smithi073 ceph-mon[36714]: pgmap v904: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 187 KiB/s wr, 90 op/s 2024-07-31T15:07:03.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:03 smithi077 ceph-mon[37327]: pgmap v904: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 187 KiB/s wr, 90 op/s 2024-07-31T15:07:03.786 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/callback_proc.o 2024-07-31T15:07:03.822 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsdump.o 2024-07-31T15:07:03.968 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/base64.o 2024-07-31T15:07:04.067 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/once.o 2024-07-31T15:07:04.122 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cc95e00d', total seconds elapsed: 37.1 2024-07-31T15:07:04.132 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/xdp.o 2024-07-31T15:07:04.179 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nseval.o 2024-07-31T15:07:04.196 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:07:04.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.728+0000 7f62ba7ba640 1 -- 172.21.15.67:0/2223220235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b410a410 msgr2=0x7f62b410a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:04.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.728+0000 7f62ba7ba640 1 --2- 172.21.15.67:0/2223220235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b410a410 0x7f62b410a820 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009900 tx=0x7f62ac02a440 comp rx=0 tx=0).stop 2024-07-31T15:07:04.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.732+0000 7f62ba7ba640 1 -- 172.21.15.67:0/2223220235 shutdown_connections 2024-07-31T15:07:04.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.732+0000 7f62ba7ba640 1 --2- 172.21.15.67:0/2223220235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b411c660 0x7f62b411eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.732+0000 7f62ba7ba640 1 --2- 172.21.15.67:0/2223220235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b410a410 0x7f62b410a820 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.732+0000 7f62ba7ba640 1 --2- 172.21.15.67:0/2223220235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62b4111970 0x7f62b4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.732+0000 7f62ba7ba640 1 -- 172.21.15.67:0/2223220235 >> 172.21.15.67:0/2223220235 conn(0x7f62b40708f0 msgr2=0x7f62b4070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:04.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.732+0000 7f62ba7ba640 1 -- 172.21.15.67:0/2223220235 shutdown_connections 2024-07-31T15:07:04.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.733+0000 7f62ba7ba640 1 -- 172.21.15.67:0/2223220235 wait complete. 2024-07-31T15:07:04.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.734+0000 7f62ba7ba640 1 Processor -- start 2024-07-31T15:07:04.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.737+0000 7f62ba7ba640 1 -- start start 2024-07-31T15:07:04.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.737+0000 7f62ba7ba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b410a410 0x7f62b41a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.738+0000 7f62ba7ba640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62b4111970 0x7f62b41a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.738+0000 7f62ba7ba640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b411c660 0x7f62b41ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.738+0000 7f62ba7ba640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4122430 con 0x7f62b411c660 2024-07-31T15:07:04.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.738+0000 7f62ba7ba640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f62b41222b0 con 0x7f62b4111970 2024-07-31T15:07:04.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.738+0000 7f62ba7ba640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f62b41225b0 con 0x7f62b410a410 2024-07-31T15:07:04.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.738+0000 7f62b8d57640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b411c660 0x7f62b41ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.738+0000 7f62b8d57640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b411c660 0x7f62b41ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55776/0 (socket says 172.21.15.67:55776) 2024-07-31T15:07:04.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.738+0000 7f62b8d57640 1 -- 172.21.15.67:0/3674079352 learned_addr learned my addr 172.21.15.67:0/3674079352 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:04.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.738+0000 7f62b3fff640 1 --2- 172.21.15.67:0/3674079352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b410a410 0x7f62b41a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.739+0000 7f62b37fe640 1 --2- 172.21.15.67:0/3674079352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62b4111970 0x7f62b41a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.739+0000 7f62b8d57640 1 -- 172.21.15.67:0/3674079352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b410a410 msgr2=0x7f62b41a8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:04.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.739+0000 7f62b8d57640 1 --2- 172.21.15.67:0/3674079352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b410a410 0x7f62b41a8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.739+0000 7f62b8d57640 1 -- 172.21.15.67:0/3674079352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62b4111970 msgr2=0x7f62b41a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:04.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.739+0000 7f62b8d57640 1 --2- 172.21.15.67:0/3674079352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62b4111970 0x7f62b41a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.739+0000 7f62b8d57640 1 -- 172.21.15.67:0/3674079352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b41aee30 con 0x7f62b411c660 2024-07-31T15:07:04.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.740+0000 7f62b8d57640 1 --2- 172.21.15.67:0/3674079352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b411c660 0x7f62b41ae700 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f62a400b990 tx=0x7f62a400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:04.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.740+0000 7f62b17fa640 1 -- 172.21.15.67:0/3674079352 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4019070 con 0x7f62b411c660 2024-07-31T15:07:04.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.740+0000 7f62b17fa640 1 -- 172.21.15.67:0/3674079352 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a400cba0 con 0x7f62b411c660 2024-07-31T15:07:04.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.740+0000 7f62b17fa640 1 -- 172.21.15.67:0/3674079352 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a40075d0 con 0x7f62b411c660 2024-07-31T15:07:04.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.740+0000 7f62ba7ba640 1 -- 172.21.15.67:0/3674079352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b41af120 con 0x7f62b411c660 2024-07-31T15:07:04.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.740+0000 7f62ba7ba640 1 -- 172.21.15.67:0/3674079352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b41b5880 con 0x7f62b411c660 2024-07-31T15:07:04.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.749+0000 7f62b17fa640 1 -- 172.21.15.67:0/3674079352 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f62a401c070 con 0x7f62b411c660 2024-07-31T15:07:04.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.751+0000 7f62b17fa640 1 --2- 172.21.15.67:0/3674079352 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f62a00b3f50 0x7f62a00b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.751+0000 7f62b17fa640 1 -- 172.21.15.67:0/3674079352 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f62a40e47d0 con 0x7f62b411c660 2024-07-31T15:07:04.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.759+0000 7f62c0d0b740 1 Processor -- start 2024-07-31T15:07:04.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.759+0000 7f62c0d0b740 1 -- start start 2024-07-31T15:07:04.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.759+0000 7f62b3fff640 1 --2- 172.21.15.67:0/3674079352 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f62a00b3f50 0x7f62a00b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.759+0000 7f62c0d0b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133ac8070 0x565133e6d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.760+0000 7f62c0d0b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133a9f1b0 0x565133e6dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.760+0000 7f62c0d0b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133aa7390 0x565133e6dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.760+0000 7f62c0d0b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x565133de3bb0 con 0x565133aa7390 2024-07-31T15:07:04.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.760+0000 7f62c0d0b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x565133dd30b0 con 0x565133a9f1b0 2024-07-31T15:07:04.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.760+0000 7f62c0d0b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565133dd70f0 con 0x565133ac8070 2024-07-31T15:07:04.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.760+0000 7f62b8d57640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133aa7390 0x565133e6dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.760+0000 7f62b8d57640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133aa7390 0x565133e6dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55784/0 (socket says 172.21.15.67:55784) 2024-07-31T15:07:04.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.760+0000 7f62b8d57640 1 -- 172.21.15.67:0/251970395 learned_addr learned my addr 172.21.15.67:0/251970395 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:04.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.760+0000 7f62b8d57640 1 -- 172.21.15.67:0/251970395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133ac8070 msgr2=0x565133e6d570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:07:04.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.760+0000 7f62b8d57640 1 --2- 172.21.15.67:0/251970395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133ac8070 0x565133e6d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.760+0000 7f62b8d57640 1 -- 172.21.15.67:0/251970395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133a9f1b0 msgr2=0x565133e6dab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:07:04.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.760+0000 7f62b8d57640 1 --2- 172.21.15.67:0/251970395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133a9f1b0 0x565133e6dab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.760+0000 7f62b8d57640 1 -- 172.21.15.67:0/251970395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565133d79b90 con 0x565133aa7390 2024-07-31T15:07:04.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.761+0000 7f62b8d57640 1 --2- 172.21.15.67:0/251970395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133aa7390 0x565133e6dff0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f62a400bf80 tx=0x7f62a40aafa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7654bd621169a5f3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:04.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.761+0000 7f62aa7fc640 1 -- 172.21.15.67:0/251970395 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a40b4200 con 0x565133aa7390 2024-07-31T15:07:04.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.761+0000 7f62aa7fc640 1 -- 172.21.15.67:0/251970395 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a40a73e0 con 0x565133aa7390 2024-07-31T15:07:04.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.761+0000 7f62aa7fc640 1 -- 172.21.15.67:0/251970395 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a40b3fd0 con 0x565133aa7390 2024-07-31T15:07:04.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.763+0000 7f62c0d0b740 1 -- 172.21.15.67:0/251970395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133aa7390 msgr2=0x565133e6dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:04.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.763+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/251970395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133aa7390 0x565133e6dff0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f62a400bf80 tx=0x7f62a40aafa0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.763+0000 7f62b3fff640 1 --2- 172.21.15.67:0/3674079352 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f62a00b3f50 0x7f62a00b6460 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f629c006fd0 tx=0x7f629c008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:04.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.770+0000 7f62c0d0b740 1 -- 172.21.15.67:0/251970395 shutdown_connections 2024-07-31T15:07:04.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.770+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/251970395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133aa7390 0x565133e6dff0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.770+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/251970395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133a9f1b0 0x565133e6dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.770+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/251970395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133ac8070 0x565133e6d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.770+0000 7f62c0d0b740 1 -- 172.21.15.67:0/251970395 >> 172.21.15.67:0/251970395 conn(0x565133abc460 msgr2=0x565133e6b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.770+0000 7f62c0d0b740 1 -- 172.21.15.67:0/251970395 shutdown_connections 2024-07-31T15:07:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.770+0000 7f62c0d0b740 1 -- 172.21.15.67:0/251970395 wait complete. 2024-07-31T15:07:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.771+0000 7f62c0d0b740 1 -- start start 2024-07-31T15:07:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.771+0000 7f62c0d0b740 1 Processor -- start 2024-07-31T15:07:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.772+0000 7f62c0d0b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133a9f1b0 0x565133e790b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.772+0000 7f62c0d0b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133aa7390 0x565133e795f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.772+0000 7f62c0d0b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133ac8070 0x565133e960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.772+0000 7f62c0d0b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x565133c36700 con 0x565133ac8070 2024-07-31T15:07:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.772+0000 7f62c0d0b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x565133d759e0 con 0x565133a9f1b0 2024-07-31T15:07:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.772+0000 7f62c0d0b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565133e8cd60 con 0x565133aa7390 2024-07-31T15:07:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.772+0000 7f62c0d0b740 10 client.0 _ll_register_callbacks cb 0x565133d904f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.773+0000 7f62b8d57640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133ac8070 0x565133e960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.773+0000 7f62b8d57640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133ac8070 0x565133e960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55792/0 (socket says 172.21.15.67:55792) 2024-07-31T15:07:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.774+0000 7f62b8d57640 1 -- 172.21.15.67:0/1497090080 learned_addr learned my addr 172.21.15.67:0/1497090080 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.774+0000 7f62b3fff640 1 --2- 172.21.15.67:0/1497090080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133a9f1b0 0x565133e790b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.774+0000 7f62b37fe640 1 --2- 172.21.15.67:0/1497090080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133aa7390 0x565133e795f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.775+0000 7f62b8d57640 1 -- 172.21.15.67:0/1497090080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133aa7390 msgr2=0x565133e795f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.775+0000 7f62b8d57640 1 --2- 172.21.15.67:0/1497090080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133aa7390 0x565133e795f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.775+0000 7f62b8d57640 1 -- 172.21.15.67:0/1497090080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133a9f1b0 msgr2=0x565133e790b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.775+0000 7f62b8d57640 1 --2- 172.21.15.67:0/1497090080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133a9f1b0 0x565133e790b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.775+0000 7f62b8d57640 1 -- 172.21.15.67:0/1497090080 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565133b33650 con 0x565133ac8070 2024-07-31T15:07:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.775+0000 7f62b37fe640 1 --2- 172.21.15.67:0/1497090080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133aa7390 0x565133e795f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:07:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.776+0000 7f62b8d57640 1 --2- 172.21.15.67:0/1497090080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133ac8070 0x565133e960d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f62a40a83c0 tx=0x7f62a40b3fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3bf6b27492ca36d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.777+0000 7f629b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:07:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.777+0000 7f629b7fe640 1 -- 172.21.15.67:0/1497090080 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a40c9070 con 0x565133ac8070 2024-07-31T15:07:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.777+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1497090080 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a40a73e0 con 0x565133ac8070 2024-07-31T15:07:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.777+0000 7f629b7fe640 1 -- 172.21.15.67:0/1497090080 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f62a40cc360 con 0x565133ac8070 2024-07-31T15:07:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.778+0000 7f629b7fe640 1 -- 172.21.15.67:0/1497090080 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f62a40bd050 con 0x565133ac8070 2024-07-31T15:07:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.778+0000 7f62c0d0b740 10 client.18936 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.778+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1497090080 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x565133e98d40 con 0x565133ac8070 2024-07-31T15:07:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.779+0000 7f629b7fe640 1 -- 172.21.15.67:0/1497090080 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a40a73e0 con 0x565133ac8070 2024-07-31T15:07:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.779+0000 7f629b7fe640 1 -- 172.21.15.67:0/1497090080 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f62a40a6ac0 con 0x565133ac8070 2024-07-31T15:07:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.781+0000 7f62c0d0b740 10 client.18936 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.781+0000 7f62c0d0b740 10 client.18936 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:07:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.781+0000 7f62c0d0b740 20 client.18936 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.781+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/1497090080 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x565133a869f0 0x565133e9c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.781+0000 7f62c0d0b740 4 client.18936 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:07:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.781+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1497090080 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565133de5b00 con 0x565133a869f0 2024-07-31T15:07:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.783+0000 7f62b3fff640 1 --2- 172.21.15.67:0/1497090080 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x565133a869f0 0x565133e9c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.787+0000 7f62b3fff640 1 --2- 172.21.15.67:0/1497090080 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x565133a869f0 0x565133e9c300 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a985616e4186da5 server_cookie=df25ec84e901e2d4 in_seq=0 out_seq=0 2024-07-31T15:07:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.787+0000 7f629b7fe640 10 client.18936 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:07:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.788+0000 7f629b7fe640 1 -- 172.21.15.67:0/1497090080 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565133de5b00 con 0x565133a869f0 2024-07-31T15:07:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.788+0000 7f629b7fe640 10 client.18936 handle_command_reply: tid=0 2024-07-31T15:07:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.788+0000 7f62c0d0b740 1 client.18936 shutdown 2024-07-31T15:07:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.788+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1497090080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133ac8070 msgr2=0x565133e960d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.788+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/1497090080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133ac8070 0x565133e960d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f62a40a83c0 tx=0x7f62a40b3fa0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.789+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1497090080 shutdown_connections 2024-07-31T15:07:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.789+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/1497090080 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x565133a869f0 0x565133e9c300 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.789+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/1497090080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133ac8070 0x565133e960d0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.789+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/1497090080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133aa7390 0x565133e795f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.789+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/1497090080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133a9f1b0 0x565133e790b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.789+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1497090080 >> 172.21.15.67:0/1497090080 conn(0x565133abc460 msgr2=0x565133e69c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:04.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.789+0000 7f629b7fe640 0 client.18936 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:07:04.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.789+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1497090080 shutdown_connections 2024-07-31T15:07:04.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.789+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1497090080 wait complete. 2024-07-31T15:07:04.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.790+0000 7f62c0d0b740 20 client.18936 trim_cache size 0 max 16384 2024-07-31T15:07:04.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.820+0000 7f62c0d0b740 1 Processor -- start 2024-07-31T15:07:04.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.820+0000 7f62c0d0b740 1 -- start start 2024-07-31T15:07:04.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.821+0000 7f62c0d0b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133a9f1b0 0x565133e6b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.821+0000 7f62c0d0b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133aa7390 0x565133e745a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.821+0000 7f62c0d0b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133ac8070 0x565133e95ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.821+0000 7f62c0d0b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f629400ff80 con 0x565133a9f1b0 2024-07-31T15:07:04.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.821+0000 7f62c0d0b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6294011ac0 con 0x565133ac8070 2024-07-31T15:07:04.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.821+0000 7f62c0d0b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6294010650 con 0x565133aa7390 2024-07-31T15:07:04.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.821+0000 7f62b37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133aa7390 0x565133e745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.821+0000 7f62b8d57640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133ac8070 0x565133e95ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.821+0000 7f62b8d57640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133ac8070 0x565133e95ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40162/0 (socket says 172.21.15.67:40162) 2024-07-31T15:07:04.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.821+0000 7f62b8d57640 1 -- 172.21.15.67:0/3644998833 learned_addr learned my addr 172.21.15.67:0/3644998833 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:04.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.822+0000 7f62b3fff640 1 --2- 172.21.15.67:0/3644998833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133a9f1b0 0x565133e6b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.822+0000 7f62b8d57640 1 -- 172.21.15.67:0/3644998833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133aa7390 msgr2=0x565133e745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:04.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.822+0000 7f62b8d57640 1 --2- 172.21.15.67:0/3644998833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133aa7390 0x565133e745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.822+0000 7f62b8d57640 1 -- 172.21.15.67:0/3644998833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133a9f1b0 msgr2=0x565133e6b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:04.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.822+0000 7f62b8d57640 1 --2- 172.21.15.67:0/3644998833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133a9f1b0 0x565133e6b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.822+0000 7f62b8d57640 1 -- 172.21.15.67:0/3644998833 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565133e7a170 con 0x565133ac8070 2024-07-31T15:07:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.824+0000 7f62b8d57640 1 --2- 172.21.15.67:0/3644998833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133ac8070 0x565133e95ec0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f62a4013000 tx=0x7f62a40b4110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aafbe44a52c779 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.824+0000 7f62a9ffb640 1 -- 172.21.15.67:0/3644998833 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a40c9070 con 0x565133ac8070 2024-07-31T15:07:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.825+0000 7f62a9ffb640 1 -- 172.21.15.67:0/3644998833 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a40c0390 con 0x565133ac8070 2024-07-31T15:07:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.825+0000 7f62a9ffb640 1 -- 172.21.15.67:0/3644998833 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a40cc430 con 0x565133ac8070 2024-07-31T15:07:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.827+0000 7f62c0d0b740 1 -- 172.21.15.67:0/3644998833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133ac8070 msgr2=0x565133e95ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.827+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/3644998833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133ac8070 0x565133e95ec0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f62a4013000 tx=0x7f62a40b4110 comp rx=0 tx=0).stop 2024-07-31T15:07:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.827+0000 7f62c0d0b740 1 -- 172.21.15.67:0/3644998833 shutdown_connections 2024-07-31T15:07:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.827+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/3644998833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133ac8070 0x565133e95ec0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.827+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/3644998833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133aa7390 0x565133e745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.827+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/3644998833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133a9f1b0 0x565133e6b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.827+0000 7f62c0d0b740 1 -- 172.21.15.67:0/3644998833 >> 172.21.15.67:0/3644998833 conn(0x565133abc460 msgr2=0x565133e6ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.830+0000 7f62c0d0b740 1 -- 172.21.15.67:0/3644998833 shutdown_connections 2024-07-31T15:07:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.830+0000 7f62c0d0b740 1 -- 172.21.15.67:0/3644998833 wait complete. 2024-07-31T15:07:04.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.830+0000 7f62c0d0b740 1 -- start start 2024-07-31T15:07:04.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.831+0000 7f62c0d0b740 1 Processor -- start 2024-07-31T15:07:04.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.831+0000 7f62c0d0b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133a9f1b0 0x565133e92030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.831+0000 7f62c0d0b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133aa7390 0x565133e92570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.831+0000 7f62c0d0b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133ac8070 0x565133e94ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.831+0000 7f62c0d0b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f62940113f0 con 0x565133a9f1b0 2024-07-31T15:07:04.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.831+0000 7f62c0d0b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6294010d20 con 0x565133aa7390 2024-07-31T15:07:04.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.831+0000 7f62c0d0b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f629400df80 con 0x565133ac8070 2024-07-31T15:07:04.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.831+0000 7f62c0d0b740 10 client.0 _ll_register_callbacks cb 0x7f62a40aa9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:04.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.834+0000 7f62b8d57640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133ac8070 0x565133e94ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.834+0000 7f62b8d57640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133ac8070 0x565133e94ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60202/0 (socket says 172.21.15.67:60202) 2024-07-31T15:07:04.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.834+0000 7f62b8d57640 1 -- 172.21.15.67:0/1260709970 learned_addr learned my addr 172.21.15.67:0/1260709970 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:04.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.835+0000 7f62b37fe640 1 --2- 172.21.15.67:0/1260709970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133aa7390 0x565133e92570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.835+0000 7f62b8d57640 1 -- 172.21.15.67:0/1260709970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133aa7390 msgr2=0x565133e92570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:04.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.835+0000 7f62b8d57640 1 --2- 172.21.15.67:0/1260709970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133aa7390 0x565133e92570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.835+0000 7f62b8d57640 1 -- 172.21.15.67:0/1260709970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133a9f1b0 msgr2=0x565133e92030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:04.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.835+0000 7f62b8d57640 1 --2- 172.21.15.67:0/1260709970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133a9f1b0 0x565133e92030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.835+0000 7f62b8d57640 1 -- 172.21.15.67:0/1260709970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565133e92bb0 con 0x565133ac8070 2024-07-31T15:07:04.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.836+0000 7f62b8d57640 1 --2- 172.21.15.67:0/1260709970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133ac8070 0x565133e94ef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f62a400bf80 tx=0x7f62a40092b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f8e4b254f97418b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:04.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.836+0000 7f629b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:07:04.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.836+0000 7f629b7fe640 1 -- 172.21.15.67:0/1260709970 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a408a060 con 0x565133ac8070 2024-07-31T15:07:04.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.837+0000 7f629b7fe640 1 -- 172.21.15.67:0/1260709970 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9351+0+0 (secure 0 0 0) 0x7f62a40cc3d0 con 0x565133ac8070 2024-07-31T15:07:04.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.837+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1260709970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a40c0390 con 0x565133ac8070 2024-07-31T15:07:04.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.845+0000 7f629b7fe640 1 -- 172.21.15.67:0/1260709970 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f62a40c9070 con 0x565133ac8070 2024-07-31T15:07:04.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.846+0000 7f62c0d0b740 10 client.28357 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:04.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.846+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1260709970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x565133e92bb0 con 0x565133ac8070 2024-07-31T15:07:04.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.853+0000 7f629b7fe640 1 -- 172.21.15.67:0/1260709970 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a40c0390 con 0x565133ac8070 2024-07-31T15:07:04.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.853+0000 7f629b7fe640 1 -- 172.21.15.67:0/1260709970 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f62a4092c70 con 0x565133ac8070 2024-07-31T15:07:04.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.854+0000 7f62c0d0b740 10 client.28357 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:04.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.854+0000 7f62c0d0b740 10 client.28357 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:07:04.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.854+0000 7f62c0d0b740 20 client.28357 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:04.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.854+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/1260709970 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x565133a5bda0 0x565133f06260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:04.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.854+0000 7f62c0d0b740 4 client.28357 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:07:04.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.854+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1260709970 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565133de15a0 con 0x565133a5bda0 2024-07-31T15:07:04.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.858+0000 7f62b3fff640 1 --2- 172.21.15.67:0/1260709970 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x565133a5bda0 0x565133f06260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.875+0000 7f62b3fff640 1 --2- 172.21.15.67:0/1260709970 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x565133a5bda0 0x565133f06260 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7dd6c98549375a6a server_cookie=a154dfeb1120ad9b in_seq=0 out_seq=0 2024-07-31T15:07:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.877+0000 7f629b7fe640 10 client.28357 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:07:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.881+0000 7f629b7fe640 1 -- 172.21.15.67:0/1260709970 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565133de15a0 con 0x565133a5bda0 2024-07-31T15:07:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.881+0000 7f629b7fe640 10 client.28357 handle_command_reply: tid=0 2024-07-31T15:07:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.882+0000 7f62c0d0b740 1 client.28357 shutdown 2024-07-31T15:07:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.887+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1260709970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133ac8070 msgr2=0x565133e94ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.887+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/1260709970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133ac8070 0x565133e94ef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f62a400bf80 tx=0x7f62a40092b0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.891+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1260709970 shutdown_connections 2024-07-31T15:07:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.891+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/1260709970 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x565133a5bda0 0x565133f06260 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.891+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/1260709970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565133ac8070 0x565133e94ef0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.891+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/1260709970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565133aa7390 0x565133e92570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.891+0000 7f62c0d0b740 1 --2- 172.21.15.67:0/1260709970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565133a9f1b0 0x565133e92030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.891+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1260709970 >> 172.21.15.67:0/1260709970 conn(0x565133abc460 msgr2=0x565133e7a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.891+0000 7f629b7fe640 0 client.28357 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:07:04.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.891+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1260709970 shutdown_connections 2024-07-31T15:07:04.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.891+0000 7f62c0d0b740 1 -- 172.21.15.67:0/1260709970 wait complete. 2024-07-31T15:07:04.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.891+0000 7f62c0d0b740 20 client.28357 trim_cache size 0 max 16384 2024-07-31T15:07:04.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.892+0000 7f62ba7ba640 1 -- 172.21.15.67:0/3674079352 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f62a00b3f50 msgr2=0x7f62a00b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:04.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.892+0000 7f62ba7ba640 1 --2- 172.21.15.67:0/3674079352 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f62a00b3f50 0x7f62a00b6460 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f629c006fd0 tx=0x7f629c008040 comp rx=0 tx=0).stop 2024-07-31T15:07:04.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.892+0000 7f62ba7ba640 1 -- 172.21.15.67:0/3674079352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b411c660 msgr2=0x7f62b41ae700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:04.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.892+0000 7f62ba7ba640 1 --2- 172.21.15.67:0/3674079352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b411c660 0x7f62b41ae700 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f62a400b990 tx=0x7f62a400be50 comp rx=0 tx=0).stop 2024-07-31T15:07:04.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.892+0000 7f62ba7ba640 1 -- 172.21.15.67:0/3674079352 shutdown_connections 2024-07-31T15:07:04.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.892+0000 7f62ba7ba640 1 --2- 172.21.15.67:0/3674079352 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f62a00b3f50 0x7f62a00b6460 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.892+0000 7f62ba7ba640 1 --2- 172.21.15.67:0/3674079352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62b411c660 0x7f62b41ae700 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.892+0000 7f62ba7ba640 1 --2- 172.21.15.67:0/3674079352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62b4111970 0x7f62b41a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.892+0000 7f62ba7ba640 1 --2- 172.21.15.67:0/3674079352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62b410a410 0x7f62b41a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:04.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.892+0000 7f62ba7ba640 1 -- 172.21.15.67:0/3674079352 >> 172.21.15.67:0/3674079352 conn(0x7f62b40708f0 msgr2=0x7f62b411de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:04.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.899+0000 7f62ba7ba640 1 -- 172.21.15.67:0/3674079352 shutdown_connections 2024-07-31T15:07:04.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:03.900+0000 7f62ba7ba640 1 -- 172.21.15.67:0/3674079352 wait complete. 2024-07-31T15:07:04.387 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/amba/built-in.a 2024-07-31T15:07:04.570 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/acct.o 2024-07-31T15:07:04.576 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4namespace.o 2024-07-31T15:07:04.625 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/refcount.o 2024-07-31T15:07:04.748 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/actions/built-in.a 2024-07-31T15:07:04.804 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsinit.o 2024-07-31T15:07:04.833 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/analogbits/built-in.a 2024-07-31T15:07:04.859 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsload.o 2024-07-31T15:07:04.935 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/bcm/built-in.a 2024-07-31T15:07:04.960 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/rcuref.o 2024-07-31T15:07:05.067 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4getroot.o 2024-07-31T15:07:05.154 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T15:07:05.168 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/imgtec/built-in.a 2024-07-31T15:07:05.200 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/usercopy.o 2024-07-31T15:07:05.294 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/imx/built-in.a 2024-07-31T15:07:05.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:05 smithi067 ceph-mon[32750]: pgmap v905: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 292 KiB/s wr, 90 op/s 2024-07-31T15:07:05.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:05 smithi067 ceph-mon[32750]: osdmap e116: 12 total, 12 up, 12 in 2024-07-31T15:07:05.394 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsnames.o 2024-07-31T15:07:05.434 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/ingenic/built-in.a 2024-07-31T15:07:05.554 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/mediatek/built-in.a 2024-07-31T15:07:05.611 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsobject.o 2024-07-31T15:07:05.626 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/microchip/built-in.a 2024-07-31T15:07:05.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:05 smithi073 ceph-mon[36714]: pgmap v905: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 292 KiB/s wr, 90 op/s 2024-07-31T15:07:05.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:05 smithi073 ceph-mon[36714]: osdmap e116: 12 total, 12 up, 12 in 2024-07-31T15:07:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:05 smithi077 ceph-mon[37327]: pgmap v905: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 292 KiB/s wr, 90 op/s 2024-07-31T15:07:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:05 smithi077 ceph-mon[37327]: osdmap e116: 12 total, 12 up, 12 in 2024-07-31T15:07:05.738 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/mstar/built-in.a 2024-07-31T15:07:05.830 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/mvebu/built-in.a 2024-07-31T15:07:05.840 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/errseq.o 2024-07-31T15:07:05.917 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/flow_offload.o 2024-07-31T15:07:05.951 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/ralink/built-in.a 2024-07-31T15:07:05.972 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:07:06.031 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/renesas/built-in.a 2024-07-31T15:07:06.120 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/socfpga/built-in.a 2024-07-31T15:07:06.127 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/crash_core.o 2024-07-31T15:07:06.223 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/bucket_locks.o 2024-07-31T15:07:06.267 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/sprd/built-in.a 2024-07-31T15:07:06.303 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsparse.o 2024-07-31T15:07:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nspredef.o 2024-07-31T15:07:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/starfive/built-in.a 2024-07-31T15:07:06.428 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/sunxi-ng/built-in.a 2024-07-31T15:07:06.441 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4client.o 2024-07-31T15:07:06.512 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/ti/built-in.a 2024-07-31T15:07:06.549 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/versatile/built-in.a 2024-07-31T15:07:06.552 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsprepkg.o 2024-07-31T15:07:06.742 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/xilinx/built-in.a 2024-07-31T15:07:06.792 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clk/built-in.a 2024-07-31T15:07:06.957 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsrepair.o 2024-07-31T15:07:07.010 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma/dw/core.o 2024-07-31T15:07:07.080 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/generic-radix-tree.o 2024-07-31T15:07:07.176 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/string_helpers.o 2024-07-31T15:07:07.242 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/kexec_core.o 2024-07-31T15:07:07.449 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/gro.o 2024-07-31T15:07:07.535 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsrepair2.o 2024-07-31T15:07:07.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:07 smithi067 ceph-mon[32750]: pgmap v907: 129 pgs: 8 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 93 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 192 KiB/s wr, 54 op/s 2024-07-31T15:07:07.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:07 smithi073 ceph-mon[36714]: pgmap v907: 129 pgs: 8 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 93 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 192 KiB/s wr, 54 op/s 2024-07-31T15:07:07.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:07 smithi077 ceph-mon[37327]: pgmap v907: 129 pgs: 8 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 93 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 192 KiB/s wr, 54 op/s 2024-07-31T15:07:07.790 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/hexdump.o 2024-07-31T15:07:07.797 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nssearch.o 2024-07-31T15:07:08.232 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/kstrtox.o 2024-07-31T15:07:08.379 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsutils.o 2024-07-31T15:07:08.468 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/iomap.o 2024-07-31T15:07:08.472 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4session.o 2024-07-31T15:07:08.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:08 smithi067 ceph-mon[32750]: pgmap v908: 129 pgs: 16 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 60 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 206 KiB/s wr, 75 op/s 2024-07-31T15:07:08.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:08 smithi073 ceph-mon[36714]: pgmap v908: 129 pgs: 16 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 60 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 206 KiB/s wr, 75 op/s 2024-07-31T15:07:08.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.658+0000 7f0e083ab640 1 -- 172.21.15.67:0/1603590579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0011c660 msgr2=0x7f0e0011eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:08.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.658+0000 7f0e083ab640 1 --2- 172.21.15.67:0/1603590579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0011c660 0x7f0e0011eb70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009ca0 tx=0x7f0df802a8f0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.659+0000 7f0e083ab640 1 -- 172.21.15.67:0/1603590579 shutdown_connections 2024-07-31T15:07:08.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.659+0000 7f0e083ab640 1 --2- 172.21.15.67:0/1603590579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0011c660 0x7f0e0011eb70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.659+0000 7f0e083ab640 1 --2- 172.21.15.67:0/1603590579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e0010a410 0x7f0e0010a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.659+0000 7f0e083ab640 1 --2- 172.21.15.67:0/1603590579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e00111970 0x7f0e00109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.659+0000 7f0e083ab640 1 -- 172.21.15.67:0/1603590579 >> 172.21.15.67:0/1603590579 conn(0x7f0e000708f0 msgr2=0x7f0e00070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:08.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.662+0000 7f0e083ab640 1 -- 172.21.15.67:0/1603590579 shutdown_connections 2024-07-31T15:07:08.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.663+0000 7f0e083ab640 1 -- 172.21.15.67:0/1603590579 wait complete. 2024-07-31T15:07:08.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.664+0000 7f0e083ab640 1 Processor -- start 2024-07-31T15:07:08.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.664+0000 7f0e083ab640 1 -- start start 2024-07-31T15:07:08.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.664+0000 7f0e083ab640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e0010a410 0x7f0e001a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e083ab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e00111970 0x7f0e001a92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e06147640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e0010a410 0x7f0e001a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:08.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e06147640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e0010a410 0x7f0e001a8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40190/0 (socket says 172.21.15.67:40190) 2024-07-31T15:07:08.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e06147640 1 -- 172.21.15.67:0/3967258980 learned_addr learned my addr 172.21.15.67:0/3967258980 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:08.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e083ab640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0011c660 0x7f0e001ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e083ab640 1 -- 172.21.15.67:0/3967258980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0e00122360 con 0x7f0e00111970 2024-07-31T15:07:08.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e083ab640 1 -- 172.21.15.67:0/3967258980 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0e001221e0 con 0x7f0e0010a410 2024-07-31T15:07:08.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e083ab640 1 -- 172.21.15.67:0/3967258980 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0e001224e0 con 0x7f0e0011c660 2024-07-31T15:07:08.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e06948640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0011c660 0x7f0e001ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:08.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e06147640 1 -- 172.21.15.67:0/3967258980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0011c660 msgr2=0x7f0e001ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:08.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e05946640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e00111970 0x7f0e001a92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:08.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e06147640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0011c660 0x7f0e001ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e06147640 1 -- 172.21.15.67:0/3967258980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e00111970 msgr2=0x7f0e001a92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:08.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e06147640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e00111970 0x7f0e001a92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.665+0000 7f0e06147640 1 -- 172.21.15.67:0/3967258980 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e001aec80 con 0x7f0e0010a410 2024-07-31T15:07:08.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.666+0000 7f0e06147640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e0010a410 0x7f0e001a8d10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0df000b450 tx=0x7f0df000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:08.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.666+0000 7f0df77fe640 1 -- 172.21.15.67:0/3967258980 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0015020 con 0x7f0e0010a410 2024-07-31T15:07:08.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.666+0000 7f0e083ab640 1 -- 172.21.15.67:0/3967258980 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e001aef70 con 0x7f0e0010a410 2024-07-31T15:07:08.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.666+0000 7f0e083ab640 1 -- 172.21.15.67:0/3967258980 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e00075020 con 0x7f0e0010a410 2024-07-31T15:07:08.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.667+0000 7f0df77fe640 1 -- 172.21.15.67:0/3967258980 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df0011d40 con 0x7f0e0010a410 2024-07-31T15:07:08.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.668+0000 7f0df77fe640 1 -- 172.21.15.67:0/3967258980 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df000f340 con 0x7f0e0010a410 2024-07-31T15:07:08.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.681+0000 7f0e0e8fc740 1 Processor -- start 2024-07-31T15:07:08.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.682+0000 7f0e0e8fc740 1 -- start start 2024-07-31T15:07:08.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.682+0000 7f0e0e8fc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d35570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.682+0000 7f0e0e8fc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d35ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.682+0000 7f0e0e8fc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 0x55bb13d35ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.682+0000 7f0e0e8fc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bb13cabbb0 con 0x55bb13990070 2024-07-31T15:07:08.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.682+0000 7f0e0e8fc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bb13c9b0b0 con 0x55bb139671b0 2024-07-31T15:07:08.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.682+0000 7f0e0e8fc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bb13c9f0f0 con 0x55bb1396f390 2024-07-31T15:07:08.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.682+0000 7f0df77fe640 1 -- 172.21.15.67:0/3967258980 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0df0018020 con 0x7f0e0010a410 2024-07-31T15:07:08.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.682+0000 7f0e06147640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d35ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:08.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.683+0000 7f0e06948640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d35570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:08.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.682+0000 7f0e06147640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d35ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40206/0 (socket says 172.21.15.67:40206) 2024-07-31T15:07:08.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.683+0000 7f0e06147640 1 -- 172.21.15.67:0/2877553163 learned_addr learned my addr 172.21.15.67:0/2877553163 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:08.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.685+0000 7f0df77fe640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0dec0b3e20 0x7f0dec0b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.685+0000 7f0df77fe640 1 -- 172.21.15.67:0/3967258980 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9483+0+0 (secure 0 0 0) 0x7f0df00e3d60 con 0x7f0e0010a410 2024-07-31T15:07:08.689 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/apple/built-in.a 2024-07-31T15:07:08.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.688+0000 7f0e06948640 1 -- 172.21.15.67:0/2877553163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 msgr2=0x55bb13d35ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:07:08.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.688+0000 7f0e06948640 1 --2- 172.21.15.67:0/2877553163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 0x55bb13d35ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.688+0000 7f0e06948640 1 -- 172.21.15.67:0/2877553163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 msgr2=0x55bb13d35ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:08.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.688+0000 7f0e06948640 1 --2- 172.21.15.67:0/2877553163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d35ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.688+0000 7f0e06948640 1 -- 172.21.15.67:0/2877553163 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb13c41b90 con 0x55bb13990070 2024-07-31T15:07:08.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.697+0000 7f0e06948640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0dec0b3e20 0x7f0dec0b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:08.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.701+0000 7f0e06948640 1 --2- 172.21.15.67:0/2877553163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d35570 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009bf0 tx=0x7f0df8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33a48db82251d2a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.710+0000 7f0e06948640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0dec0b3e20 0x7f0dec0b6330 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0df804b000 tx=0x7f0df804c000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.710+0000 7f0df4d99640 1 -- 172.21.15.67:0/2877553163 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8031da0 con 0x55bb13990070 2024-07-31T15:07:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.710+0000 7f0df4d99640 1 -- 172.21.15.67:0/2877553163 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df8036040 con 0x55bb13990070 2024-07-31T15:07:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.710+0000 7f0df4d99640 1 -- 172.21.15.67:0/2877553163 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df803a3f0 con 0x55bb13990070 2024-07-31T15:07:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.715+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/2877553163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 msgr2=0x55bb13d35570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.715+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/2877553163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d35570 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009bf0 tx=0x7f0df8009900 comp rx=0 tx=0).stop 2024-07-31T15:07:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.715+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/2877553163 shutdown_connections 2024-07-31T15:07:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.715+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/2877553163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 0x55bb13d35ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.715+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/2877553163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d35ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.715+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/2877553163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d35570 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.715+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/2877553163 >> 172.21.15.67:0/2877553163 conn(0x55bb13984460 msgr2=0x55bb13d33460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.717+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/2877553163 shutdown_connections 2024-07-31T15:07:08.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.728+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/2877553163 wait complete. 2024-07-31T15:07:08.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.729+0000 7f0e0e8fc740 1 -- start start 2024-07-31T15:07:08.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.729+0000 7f0e0e8fc740 1 Processor -- start 2024-07-31T15:07:08.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.732+0000 7f0e0e8fc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.732+0000 7f0e0e8fc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 0x55bb13d41790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.732+0000 7f0e0e8fc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d4d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.732+0000 7f0e0e8fc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bb13afe700 con 0x55bb13990070 2024-07-31T15:07:08.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.732+0000 7f0e0e8fc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bb13c3d9e0 con 0x55bb139671b0 2024-07-31T15:07:08.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.732+0000 7f0e0e8fc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bb13d43d50 con 0x55bb1396f390 2024-07-31T15:07:08.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.732+0000 7f0e0e8fc740 10 client.0 _ll_register_callbacks cb 0x55bb13c584f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:08.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.733+0000 7f0e06948640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d4d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:08.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.733+0000 7f0e06948640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d4d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55842/0 (socket says 172.21.15.67:55842) 2024-07-31T15:07:08.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.733+0000 7f0e06948640 1 -- 172.21.15.67:0/2205660110 learned_addr learned my addr 172.21.15.67:0/2205660110 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:08.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:08 smithi077 ceph-mon[37327]: pgmap v908: 129 pgs: 16 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 60 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 206 KiB/s wr, 75 op/s 2024-07-31T15:07:08.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.733+0000 7f0e06948640 1 -- 172.21.15.67:0/2205660110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 msgr2=0x55bb13d41790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:07:08.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.733+0000 7f0e06948640 1 --2- 172.21.15.67:0/2205660110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 0x55bb13d41790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.733+0000 7f0e06948640 1 -- 172.21.15.67:0/2205660110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 msgr2=0x55bb13d411f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:07:08.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.733+0000 7f0e06948640 1 --2- 172.21.15.67:0/2205660110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d411f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.733+0000 7f0e06948640 1 -- 172.21.15.67:0/2205660110 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb139fb650 con 0x55bb13990070 2024-07-31T15:07:08.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.734+0000 7f0e06948640 1 --2- 172.21.15.67:0/2205660110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d4d0a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009dc0 tx=0x7f0df8030820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21eaf9388b7499ec server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:08.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.734+0000 7f0de9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:07:08.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.734+0000 7f0de9ffb640 1 -- 172.21.15.67:0/2205660110 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df802fc20 con 0x55bb13990070 2024-07-31T15:07:08.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.734+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/2205660110 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df8036040 con 0x55bb13990070 2024-07-31T15:07:08.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.734+0000 7f0de9ffb640 1 -- 172.21.15.67:0/2205660110 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9483+0+0 (secure 0 0 0) 0x7f0df802fdc0 con 0x55bb13990070 2024-07-31T15:07:08.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.735+0000 7f0de9ffb640 1 -- 172.21.15.67:0/2205660110 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f0df804f420 con 0x55bb13990070 2024-07-31T15:07:08.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.739+0000 7f0e0e8fc740 10 client.18948 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:08.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.739+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/2205660110 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55bb13d60e50 con 0x55bb13990070 2024-07-31T15:07:08.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.739+0000 7f0de9ffb640 1 -- 172.21.15.67:0/2205660110 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df8036040 con 0x55bb13990070 2024-07-31T15:07:08.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.739+0000 7f0de9ffb640 1 -- 172.21.15.67:0/2205660110 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f0df8057630 con 0x55bb13990070 2024-07-31T15:07:08.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.739+0000 7f0e0e8fc740 10 client.18948 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:08.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.739+0000 7f0e0e8fc740 10 client.18948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:08.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.739+0000 7f0e0e8fc740 20 client.18948 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:08.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.740+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/2205660110 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb1394e9f0 0x55bb13d643d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.740+0000 7f0e0e8fc740 4 client.18948 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:07:08.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.740+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/2205660110 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb13cadb00 con 0x55bb1394e9f0 2024-07-31T15:07:08.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.740+0000 7f0e06147640 1 --2- 172.21.15.67:0/2205660110 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb1394e9f0 0x55bb13d643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:08.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.740+0000 7f0de9ffb640 10 client.18948 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:08.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.741+0000 7f0e06147640 1 --2- 172.21.15.67:0/2205660110 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb1394e9f0 0x55bb13d643d0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3cc297e126c5cd server_cookie=fde5af98ed43e3cc in_seq=0 out_seq=0 2024-07-31T15:07:08.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.745+0000 7f0de9ffb640 1 -- 172.21.15.67:0/2205660110 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb13cadb00 con 0x55bb1394e9f0 2024-07-31T15:07:08.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.745+0000 7f0de9ffb640 10 client.18948 handle_command_reply: tid=0 2024-07-31T15:07:08.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.745+0000 7f0e0e8fc740 1 client.18948 shutdown 2024-07-31T15:07:08.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.751+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/2205660110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 msgr2=0x55bb13d4d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:08.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.751+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/2205660110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d4d0a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009dc0 tx=0x7f0df8030820 comp rx=0 tx=0).stop 2024-07-31T15:07:08.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.755+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/2205660110 shutdown_connections 2024-07-31T15:07:08.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.755+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/2205660110 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb1394e9f0 0x55bb13d643d0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.755+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/2205660110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d4d0a0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.755+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/2205660110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 0x55bb13d41790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.755+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/2205660110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.755+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/2205660110 >> 172.21.15.67:0/2205660110 conn(0x55bb13984460 msgr2=0x55bb13d31c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:08.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.755+0000 7f0de9ffb640 0 client.18948 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:08.760 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/aspeed/built-in.a 2024-07-31T15:07:08.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.763+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/2205660110 shutdown_connections 2024-07-31T15:07:08.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.771+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/2205660110 wait complete. 2024-07-31T15:07:08.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.771+0000 7f0e0e8fc740 20 client.18948 trim_cache size 0 max 16384 2024-07-31T15:07:08.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.813+0000 7f0e0e8fc740 1 Processor -- start 2024-07-31T15:07:08.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.816+0000 7f0e0e8fc740 1 -- start start 2024-07-31T15:07:08.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.817+0000 7f0e0e8fc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d32350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.817+0000 7f0e0e8fc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 0x55bb13d4ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.817+0000 7f0e06147640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d32350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:08.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.817+0000 7f0e06147640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d32350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40214/0 (socket says 172.21.15.67:40214) 2024-07-31T15:07:08.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.817+0000 7f0e0e8fc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d32e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.817+0000 7f0e05946640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 0x55bb13d4ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:08.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.817+0000 7f0e05946640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 0x55bb13d4ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60248/0 (socket says 172.21.15.67:60248) 2024-07-31T15:07:08.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.818+0000 7f0e06147640 1 -- 172.21.15.67:0/1198450034 learned_addr learned my addr 172.21.15.67:0/1198450034 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:08.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.818+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/1198450034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0de40100b0 con 0x55bb13990070 2024-07-31T15:07:08.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.818+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/1198450034 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0de4011bf0 con 0x55bb139671b0 2024-07-31T15:07:08.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.818+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/1198450034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0de4010780 con 0x55bb1396f390 2024-07-31T15:07:08.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.818+0000 7f0e06948640 1 --2- 172.21.15.67:0/1198450034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d32e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:08.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.819+0000 7f0e06147640 1 -- 172.21.15.67:0/1198450034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 msgr2=0x55bb13d4ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:08.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.819+0000 7f0e06147640 1 --2- 172.21.15.67:0/1198450034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 0x55bb13d4ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.819+0000 7f0e06147640 1 -- 172.21.15.67:0/1198450034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 msgr2=0x55bb13d32e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:08.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.819+0000 7f0e06147640 1 --2- 172.21.15.67:0/1198450034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d32e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.819+0000 7f0e06147640 1 -- 172.21.15.67:0/1198450034 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb13d3c880 con 0x55bb139671b0 2024-07-31T15:07:08.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.819+0000 7f0e05946640 1 --2- 172.21.15.67:0/1198450034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 0x55bb13d4ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:07:08.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.819+0000 7f0e06147640 1 --2- 172.21.15.67:0/1198450034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d32350 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0df0010990 tx=0x7f0df00e6260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d28f65a93e7c87f8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:08.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.823+0000 7f0debfff640 1 -- 172.21.15.67:0/1198450034 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df00aa660 con 0x55bb139671b0 2024-07-31T15:07:08.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.823+0000 7f0debfff640 1 -- 172.21.15.67:0/1198450034 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df00aec60 con 0x55bb139671b0 2024-07-31T15:07:08.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.823+0000 7f0debfff640 1 -- 172.21.15.67:0/1198450034 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df00cbb60 con 0x55bb139671b0 2024-07-31T15:07:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.826+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/1198450034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 msgr2=0x55bb13d32350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.826+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/1198450034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d32350 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0df0010990 tx=0x7f0df00e6260 comp rx=0 tx=0).stop 2024-07-31T15:07:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.826+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/1198450034 shutdown_connections 2024-07-31T15:07:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.826+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/1198450034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb13990070 0x55bb13d32e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.826+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/1198450034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb1396f390 0x55bb13d4ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.826+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/1198450034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d32350 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.826+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/1198450034 >> 172.21.15.67:0/1198450034 conn(0x55bb13984460 msgr2=0x55bb13d31c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.830+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/1198450034 shutdown_connections 2024-07-31T15:07:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.830+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/1198450034 wait complete. 2024-07-31T15:07:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.830+0000 7f0e0e8fc740 1 -- start start 2024-07-31T15:07:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.831+0000 7f0e0e8fc740 1 Processor -- start 2024-07-31T15:07:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.831+0000 7f0e0e8fc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d498a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.832+0000 7f0e0e8fc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb1396f390 0x55bb13d44d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.832+0000 7f0e06147640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.832+0000 7f0e06147640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d498a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40222/0 (socket says 172.21.15.67:40222) 2024-07-31T15:07:08.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.832+0000 7f0e06147640 1 -- 172.21.15.67:0/3704765346 learned_addr learned my addr 172.21.15.67:0/3704765346 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:08.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.832+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/3704765346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb13990070 0x55bb13d45460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.833+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/3704765346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0de4011520 con 0x55bb1396f390 2024-07-31T15:07:08.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.833+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/3704765346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0de4010e50 con 0x55bb139671b0 2024-07-31T15:07:08.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.833+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/3704765346 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0de400e0b0 con 0x55bb13990070 2024-07-31T15:07:08.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.833+0000 7f0e0e8fc740 10 client.0 _ll_register_callbacks cb 0x7f0df0009320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:08.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.834+0000 7f0e06147640 1 -- 172.21.15.67:0/3704765346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb13990070 msgr2=0x55bb13d45460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:08.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.834+0000 7f0e06147640 1 --2- 172.21.15.67:0/3704765346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb13990070 0x55bb13d45460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.834+0000 7f0e06147640 1 -- 172.21.15.67:0/3704765346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb1396f390 msgr2=0x55bb13d44d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:08.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.834+0000 7f0e06147640 1 --2- 172.21.15.67:0/3704765346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb1396f390 0x55bb13d44d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.834+0000 7f0e06147640 1 -- 172.21.15.67:0/3704765346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb13dcbba0 con 0x55bb139671b0 2024-07-31T15:07:08.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.842+0000 7f0e06147640 1 --2- 172.21.15.67:0/3704765346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d498a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0df00aa5f0 tx=0x7f0df00cd290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57e3a2ee642e8531 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:08.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.843+0000 7f0de9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:07:08.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.847+0000 7f0de9ffb640 1 -- 172.21.15.67:0/3704765346 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df00c4070 con 0x55bb139671b0 2024-07-31T15:07:08.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.847+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/3704765346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df00aec60 con 0x55bb139671b0 2024-07-31T15:07:08.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.847+0000 7f0de9ffb640 1 -- 172.21.15.67:0/3704765346 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9483+0+0 (secure 0 0 0) 0x7f0df00cbb60 con 0x55bb139671b0 2024-07-31T15:07:08.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.851+0000 7f0de9ffb640 1 -- 172.21.15.67:0/3704765346 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f0df00e1520 con 0x55bb139671b0 2024-07-31T15:07:08.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.852+0000 7f0e0e8fc740 10 client.28454 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:08.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.852+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/3704765346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55bb13dcccf0 con 0x55bb139671b0 2024-07-31T15:07:08.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.856+0000 7f0de9ffb640 1 -- 172.21.15.67:0/3704765346 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df00aec60 con 0x55bb139671b0 2024-07-31T15:07:08.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.857+0000 7f0de9ffb640 1 -- 172.21.15.67:0/3704765346 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f0df00c9020 con 0x55bb139671b0 2024-07-31T15:07:08.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.857+0000 7f0e0e8fc740 10 client.28454 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:08.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.857+0000 7f0e0e8fc740 10 client.28454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:08.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.857+0000 7f0e0e8fc740 20 client.28454 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:08.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.857+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/3704765346 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb13923da0 0x55bb13dcf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:08.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.858+0000 7f0e0e8fc740 4 client.28454 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:07:08.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.858+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/3704765346 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb13ca95a0 con 0x55bb13923da0 2024-07-31T15:07:08.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.861+0000 7f0e05946640 1 --2- 172.21.15.67:0/3704765346 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb13923da0 0x55bb13dcf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:08.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.870+0000 7f0de9ffb640 10 client.28454 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:08.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.870+0000 7f0e05946640 1 --2- 172.21.15.67:0/3704765346 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb13923da0 0x55bb13dcf2a0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f61b5b87250c63 server_cookie=c3d67448d9820fd3 in_seq=0 out_seq=0 2024-07-31T15:07:08.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.872+0000 7f0de9ffb640 1 -- 172.21.15.67:0/3704765346 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb13ca95a0 con 0x55bb13923da0 2024-07-31T15:07:08.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.872+0000 7f0de9ffb640 10 client.28454 handle_command_reply: tid=0 2024-07-31T15:07:08.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.872+0000 7f0e0e8fc740 1 client.28454 shutdown 2024-07-31T15:07:08.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.873+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/3704765346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 msgr2=0x55bb13d498a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:08.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.873+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/3704765346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d498a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0df00aa5f0 tx=0x7f0df00cd290 comp rx=0 tx=0).stop 2024-07-31T15:07:08.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.874+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/3704765346 shutdown_connections 2024-07-31T15:07:08.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.874+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/3704765346 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bb13923da0 0x55bb13dcf2a0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.874+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/3704765346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb13990070 0x55bb13d45460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.874+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/3704765346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bb1396f390 0x55bb13d44d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.874+0000 7f0e0e8fc740 1 --2- 172.21.15.67:0/3704765346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb139671b0 0x55bb13d498a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.874+0000 7f0de9ffb640 0 client.28454 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:08.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.874+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/3704765346 >> 172.21.15.67:0/3704765346 conn(0x55bb13984460 msgr2=0x55bb13d32d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:08.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.875+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/3704765346 shutdown_connections 2024-07-31T15:07:08.878 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/bcm/bcm63xx/built-in.a 2024-07-31T15:07:08.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.876+0000 7f0e0e8fc740 1 -- 172.21.15.67:0/3704765346 wait complete. 2024-07-31T15:07:08.879 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:07:08.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.877+0000 7f0e0e8fc740 20 client.28454 trim_cache size 0 max 16384 2024-07-31T15:07:08.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.878+0000 7f0e083ab640 1 -- 172.21.15.67:0/3967258980 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0dec0b3e20 msgr2=0x7f0dec0b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:08.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.878+0000 7f0e083ab640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0dec0b3e20 0x7f0dec0b6330 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0df804b000 tx=0x7f0df804c000 comp rx=0 tx=0).stop 2024-07-31T15:07:08.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.878+0000 7f0e083ab640 1 -- 172.21.15.67:0/3967258980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e0010a410 msgr2=0x7f0e001a8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:08.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.878+0000 7f0e083ab640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e0010a410 0x7f0e001a8d10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0df000b450 tx=0x7f0df000b910 comp rx=0 tx=0).stop 2024-07-31T15:07:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.878+0000 7f0e083ab640 1 -- 172.21.15.67:0/3967258980 shutdown_connections 2024-07-31T15:07:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.878+0000 7f0e083ab640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0dec0b3e20 0x7f0dec0b6330 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.878+0000 7f0e083ab640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0011c660 0x7f0e001ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.878+0000 7f0e083ab640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e00111970 0x7f0e001a92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.878+0000 7f0e083ab640 1 --2- 172.21.15.67:0/3967258980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e0010a410 0x7f0e001a8d10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.878+0000 7f0e083ab640 1 -- 172.21.15.67:0/3967258980 >> 172.21.15.67:0/3967258980 conn(0x7f0e000708f0 msgr2=0x7f0e0011dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.878+0000 7f0e083ab640 1 -- 172.21.15.67:0/3967258980 shutdown_connections 2024-07-31T15:07:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:08.878+0000 7f0e083ab640 1 -- 172.21.15.67:0/3967258980 wait complete. 2024-07-31T15:07:08.941 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/bcm/built-in.a 2024-07-31T15:07:08.958 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/virtio/virtio.o 2024-07-31T15:07:08.995 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma/dw/dw.o 2024-07-31T15:07:09.059 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/fsl/built-in.a 2024-07-31T15:07:09.130 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/fujitsu/built-in.a 2024-07-31T15:07:09.130 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nswalk.o 2024-07-31T15:07:09.221 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/imx/built-in.a 2024-07-31T15:07:09.314 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/ixp4xx/built-in.a 2024-07-31T15:07:09.436 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/loongson/built-in.a 2024-07-31T15:07:09.449 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/netdev-genl.o 2024-07-31T15:07:09.467 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/pci_iomap.o 2024-07-31T15:07:09.498 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/mediatek/built-in.a 2024-07-31T15:07:09.593 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/microchip/built-in.a 2024-07-31T15:07:09.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:09 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:07:09.715 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/nuvoton/built-in.a 2024-07-31T15:07:09.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:09 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:07:09.773 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma/dw/idma32.o 2024-07-31T15:07:09.785 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/dns_resolve.o 2024-07-31T15:07:09.793 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/pxa/built-in.a 2024-07-31T15:07:09.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:09 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:07:09.830 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsxfeval.o 2024-07-31T15:07:09.878 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/amlogic/built-in.a 2024-07-31T15:07:09.982 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/qcom/built-in.a 2024-07-31T15:07:10.093 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/renesas/built-in.a 2024-07-31T15:07:10.269 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/kexec.o 2024-07-31T15:07:10.357 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/iomap_copy.o 2024-07-31T15:07:10.437 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma/dw/acpi.o 2024-07-31T15:07:10.545 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/virtio/virtio_ring.o 2024-07-31T15:07:10.550 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsxfname.o 2024-07-31T15:07:10.553 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/rockchip/built-in.a 2024-07-31T15:07:10.623 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/devres.o 2024-07-31T15:07:10.631 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/sifive/built-in.a 2024-07-31T15:07:10.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:10 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:07:10.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:10 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:07:10.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:10 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:07:10.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:10 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:07:10.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:10 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-07-31T15:07:10.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:10 smithi077 ceph-mon[37327]: Unable to set osd_memory_target on smithi077 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-07-31T15:07:10.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:10 smithi077 ceph-mon[37327]: pgmap v909: 129 pgs: 16 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 62 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 253 KiB/s wr, 77 op/s 2024-07-31T15:07:10.746 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/sunxi/built-in.a 2024-07-31T15:07:10.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:10 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:07:10.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:10 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:07:10.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:10 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:07:10.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:10 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:07:10.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:10 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-07-31T15:07:10.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:10 smithi067 ceph-mon[32750]: Unable to set osd_memory_target on smithi077 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-07-31T15:07:10.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:10 smithi067 ceph-mon[32750]: pgmap v909: 129 pgs: 16 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 62 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 253 KiB/s wr, 77 op/s 2024-07-31T15:07:10.838 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/ti/built-in.a 2024-07-31T15:07:10.840 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/netdev-genl-gen.o 2024-07-31T15:07:10.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:07:10.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:07:10.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:07:10.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:10 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:07:10.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:10 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-07-31T15:07:10.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:10 smithi073 ceph-mon[36714]: Unable to set osd_memory_target on smithi077 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-07-31T15:07:10.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:10 smithi073 ceph-mon[36714]: pgmap v909: 129 pgs: 16 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 62 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 253 KiB/s wr, 77 op/s 2024-07-31T15:07:10.939 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/xilinx/built-in.a 2024-07-31T15:07:11.000 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/soc/built-in.a 2024-07-31T15:07:11.032 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T15:07:11.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T15:07:11.109 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/nsxfobj.o 2024-07-31T15:07:11.145 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4trace.o 2024-07-31T15:07:11.358 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/psargs.o 2024-07-31T15:07:11.380 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/compat.o 2024-07-31T15:07:11.457 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/dma/dw/built-in.a 2024-07-31T15:07:11.578 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/utsname.o 2024-07-31T15:07:11.671 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma/hsu/hsu.o 2024-07-31T15:07:11.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:11 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:07:11.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:11 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:07:11.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:11 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:07:11.784 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/gso.o 2024-07-31T15:07:11.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:11 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:07:11.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:11 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:07:11.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:11 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:07:11.806 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/check_signature.o 2024-07-31T15:07:11.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:11 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:07:11.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:11 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:07:11.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:11 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:07:11.915 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:07:12.025 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/psloop.o 2024-07-31T15:07:12.099 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/pid_namespace.o 2024-07-31T15:07:12.107 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/interval_tree.o 2024-07-31T15:07:12.353 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/stop_machine.o 2024-07-31T15:07:12.374 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/assoc_array.o 2024-07-31T15:07:12.392 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/virtio/virtio_anchor.o 2024-07-31T15:07:12.598 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/dma/hsu/built-in.a 2024-07-31T15:07:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.728+0000 7fbd25fc8640 1 -- 172.21.15.67:0/1652480066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd201116d0 msgr2=0x7fbd20109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.728+0000 7fbd25fc8640 1 --2- 172.21.15.67:0/1652480066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd201116d0 0x7fbd20109df0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009860 tx=0x7fbd1002a360 comp rx=0 tx=0).stop 2024-07-31T15:07:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.728+0000 7fbd25fc8640 1 -- 172.21.15.67:0/1652480066 shutdown_connections 2024-07-31T15:07:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.728+0000 7fbd25fc8640 1 --2- 172.21.15.67:0/1652480066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd2011c5d0 0x7fbd2011e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.728+0000 7fbd25fc8640 1 --2- 172.21.15.67:0/1652480066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd2010a330 0x7fbd2010a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.728+0000 7fbd25fc8640 1 --2- 172.21.15.67:0/1652480066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd201116d0 0x7fbd20109df0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.728+0000 7fbd25fc8640 1 -- 172.21.15.67:0/1652480066 >> 172.21.15.67:0/1652480066 conn(0x7fbd2006e350 msgr2=0x7fbd2006e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.732+0000 7fbd25fc8640 1 -- 172.21.15.67:0/1652480066 shutdown_connections 2024-07-31T15:07:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.732+0000 7fbd25fc8640 1 -- 172.21.15.67:0/1652480066 wait complete. 2024-07-31T15:07:12.733 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/dma/idxd/built-in.a 2024-07-31T15:07:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.733+0000 7fbd25fc8640 1 Processor -- start 2024-07-31T15:07:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.733+0000 7fbd25fc8640 1 -- start start 2024-07-31T15:07:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.734+0000 7fbd25fc8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd2010a330 0x7fbd201a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.734+0000 7fbd25fc8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd201116d0 0x7fbd201a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.734+0000 7fbd25fc8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd2011c5d0 0x7fbd201ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.734+0000 7fbd25fc8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbd201220b0 con 0x7fbd2010a330 2024-07-31T15:07:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.734+0000 7fbd25fc8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbd20121f30 con 0x7fbd2011c5d0 2024-07-31T15:07:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.734+0000 7fbd25fc8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd20122230 con 0x7fbd201116d0 2024-07-31T15:07:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.734+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd2011c5d0 0x7fbd201ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.734+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd2011c5d0 0x7fbd201ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44796/0 (socket says 172.21.15.67:44796) 2024-07-31T15:07:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.734+0000 7fbd1ffff640 1 -- 172.21.15.67:0/1958780839 learned_addr learned my addr 172.21.15.67:0/1958780839 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.734+0000 7fbd1effd640 1 --2- 172.21.15.67:0/1958780839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd201116d0 0x7fbd201a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.735+0000 7fbd1ffff640 1 -- 172.21.15.67:0/1958780839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd201116d0 msgr2=0x7fbd201a90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.735+0000 7fbd1ffff640 1 --2- 172.21.15.67:0/1958780839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd201116d0 0x7fbd201a90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.735+0000 7fbd1ffff640 1 -- 172.21.15.67:0/1958780839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd2010a330 msgr2=0x7fbd201a8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.735+0000 7fbd1ffff640 1 --2- 172.21.15.67:0/1958780839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd2010a330 0x7fbd201a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.735+0000 7fbd1ffff640 1 -- 172.21.15.67:0/1958780839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd201aea40 con 0x7fbd2011c5d0 2024-07-31T15:07:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.735+0000 7fbd1effd640 1 --2- 172.21.15.67:0/1958780839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd201116d0 0x7fbd201a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:07:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.735+0000 7fbd1ffff640 1 --2- 172.21.15.67:0/1958780839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd2011c5d0 0x7fbd201ae340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800efc0 tx=0x7fbd1800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:12.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:12 smithi077 ceph-mon[37327]: pgmap v910: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 207 KiB/s wr, 61 op/s 2024-07-31T15:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.742+0000 7fbd1cff9640 1 -- 172.21.15.67:0/1958780839 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18003970 con 0x7fbd2011c5d0 2024-07-31T15:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.742+0000 7fbd25fc8640 1 -- 172.21.15.67:0/1958780839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd201aed30 con 0x7fbd2011c5d0 2024-07-31T15:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.742+0000 7fbd25fc8640 1 -- 172.21.15.67:0/1958780839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd201b54c0 con 0x7fbd2011c5d0 2024-07-31T15:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.742+0000 7fbd1cff9640 1 -- 172.21.15.67:0/1958780839 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd18004020 con 0x7fbd2011c5d0 2024-07-31T15:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.742+0000 7fbd1cff9640 1 -- 172.21.15.67:0/1958780839 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1800b940 con 0x7fbd2011c5d0 2024-07-31T15:07:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.746+0000 7fbd1cff9640 1 -- 172.21.15.67:0/1958780839 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbd1801d020 con 0x7fbd2011c5d0 2024-07-31T15:07:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.747+0000 7fbd2cf98740 1 Processor -- start 2024-07-31T15:07:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.751+0000 7fbd1cff9640 1 --2- 172.21.15.67:0/1958780839 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd080b3f40 0x7fbd080b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.751+0000 7fbd1cff9640 1 -- 172.21.15.67:0/1958780839 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9483+0+0 (secure 0 0 0) 0x7fbd180e5230 con 0x7fbd2011c5d0 2024-07-31T15:07:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.751+0000 7fbd1effd640 1 --2- 172.21.15.67:0/1958780839 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd080b3f40 0x7fbd080b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.753+0000 7fbd2cf98740 1 -- start start 2024-07-31T15:07:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.753+0000 7fbd2cf98740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5580221641b0 0x558022506ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.753+0000 7fbd2cf98740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55802213b2f0 0x5580225073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.753+0000 7fbd2cf98740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221434d0 0x558022507920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.753+0000 7fbd2cf98740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55802247c3b0 con 0x5580221641b0 2024-07-31T15:07:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.753+0000 7fbd2cf98740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55802246b8b0 con 0x55802213b2f0 2024-07-31T15:07:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.753+0000 7fbd2cf98740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55802246f8f0 con 0x5580221434d0 2024-07-31T15:07:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.753+0000 7fbd1effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55802213b2f0 0x5580225073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.753+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221434d0 0x558022507920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.753+0000 7fbd1effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55802213b2f0 0x5580225073e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44812/0 (socket says 172.21.15.67:44812) 2024-07-31T15:07:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.753+0000 7fbd1effd640 1 -- 172.21.15.67:0/3913987999 learned_addr learned my addr 172.21.15.67:0/3913987999 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.753+0000 7fbd1ffff640 1 --2- 172.21.15.67:0/3913987999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221434d0 0x558022507920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38034/0 (socket says 172.21.15.67:38034) 2024-07-31T15:07:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.754+0000 7fbd1effd640 1 -- 172.21.15.67:0/3913987999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221434d0 msgr2=0x558022507920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.754+0000 7fbd1effd640 1 --2- 172.21.15.67:0/3913987999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221434d0 0x558022507920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.754+0000 7fbd1effd640 1 -- 172.21.15.67:0/3913987999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5580221641b0 msgr2=0x558022506ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:07:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.754+0000 7fbd1effd640 1 --2- 172.21.15.67:0/3913987999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5580221641b0 0x558022506ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.754+0000 7fbd1effd640 1 -- 172.21.15.67:0/3913987999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580224129a0 con 0x55802213b2f0 2024-07-31T15:07:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.754+0000 7fbd1ffff640 1 --2- 172.21.15.67:0/3913987999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221434d0 0x558022507920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:07:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.754+0000 7fbd1effd640 1 --2- 172.21.15.67:0/1958780839 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd080b3f40 0x7fbd080b6450 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbd140058c0 tx=0x7fbd14005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.754+0000 7fbd1effd640 1 --2- 172.21.15.67:0/3913987999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55802213b2f0 0x5580225073e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400b960 tx=0x7fbd14012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8e9bf30d06ac627 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.754+0000 7fbd05ffb640 1 -- 172.21.15.67:0/3913987999 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14022070 con 0x55802213b2f0 2024-07-31T15:07:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.755+0000 7fbd05ffb640 1 -- 172.21.15.67:0/3913987999 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd1401eb00 con 0x55802213b2f0 2024-07-31T15:07:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.755+0000 7fbd05ffb640 1 -- 172.21.15.67:0/3913987999 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14010b20 con 0x55802213b2f0 2024-07-31T15:07:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.756+0000 7fbd2cf98740 1 -- 172.21.15.67:0/3913987999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55802213b2f0 msgr2=0x5580225073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.756+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/3913987999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55802213b2f0 0x5580225073e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400b960 tx=0x7fbd14012000 comp rx=0 tx=0).stop 2024-07-31T15:07:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.763+0000 7fbd2cf98740 1 -- 172.21.15.67:0/3913987999 shutdown_connections 2024-07-31T15:07:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.763+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/3913987999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221434d0 0x558022507920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.763+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/3913987999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55802213b2f0 0x5580225073e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.763+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/3913987999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5580221641b0 0x558022506ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.763+0000 7fbd2cf98740 1 -- 172.21.15.67:0/3913987999 >> 172.21.15.67:0/3913987999 conn(0x558022158300 msgr2=0x558022504d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.767+0000 7fbd2cf98740 1 -- 172.21.15.67:0/3913987999 shutdown_connections 2024-07-31T15:07:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.767+0000 7fbd2cf98740 1 -- 172.21.15.67:0/3913987999 wait complete. 2024-07-31T15:07:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.767+0000 7fbd2cf98740 1 -- start start 2024-07-31T15:07:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.767+0000 7fbd2cf98740 1 Processor -- start 2024-07-31T15:07:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.768+0000 7fbd2cf98740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 0x558022512a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.768+0000 7fbd2cf98740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 0x558022512f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.768+0000 7fbd2cf98740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x55802252fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.768+0000 7fbd2cf98740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5580223a5ab0 con 0x55802213b2f0 2024-07-31T15:07:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.768+0000 7fbd2cf98740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5580223f3cf0 con 0x5580221434d0 2024-07-31T15:07:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.768+0000 7fbd2cf98740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558022526630 con 0x5580221641b0 2024-07-31T15:07:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.768+0000 7fbd2cf98740 10 client.0 _ll_register_callbacks cb 0x5580224659f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.768+0000 7fbd1effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 0x558022512f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.769+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x55802252fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.769+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x55802252fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38048/0 (socket says 172.21.15.67:38048) 2024-07-31T15:07:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.769+0000 7fbd1ffff640 1 -- 172.21.15.67:0/1685930191 learned_addr learned my addr 172.21.15.67:0/1685930191 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.769+0000 7fbd1f7fe640 1 --2- 172.21.15.67:0/1685930191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 0x558022512a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.769+0000 7fbd1ffff640 1 -- 172.21.15.67:0/1685930191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 msgr2=0x558022512f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.769+0000 7fbd1ffff640 1 --2- 172.21.15.67:0/1685930191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 0x558022512f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.769+0000 7fbd1ffff640 1 -- 172.21.15.67:0/1685930191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 msgr2=0x558022512a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.769+0000 7fbd1ffff640 1 --2- 172.21.15.67:0/1685930191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 0x558022512a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.769+0000 7fbd1ffff640 1 -- 172.21.15.67:0/1685930191 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580221cf790 con 0x5580221641b0 2024-07-31T15:07:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.769+0000 7fbd1ffff640 1 --2- 172.21.15.67:0/1685930191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x55802252fa00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbd180ab9b0 tx=0x7fbd180e2ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8715428d6d35fb82 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.778+0000 7fbd037f6640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:07:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.778+0000 7fbd037f6640 1 -- 172.21.15.67:0/1685930191 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd180aba20 con 0x5580221641b0 2024-07-31T15:07:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.778+0000 7fbd037f6640 1 -- 172.21.15.67:0/1685930191 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9483+0+0 (secure 0 0 0) 0x7fbd180b4850 con 0x5580221641b0 2024-07-31T15:07:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.778+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1685930191 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd1401eb00 con 0x5580221641b0 2024-07-31T15:07:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.788+0000 7fbd037f6640 1 -- 172.21.15.67:0/1685930191 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fbd180b0020 con 0x5580221641b0 2024-07-31T15:07:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.788+0000 7fbd2cf98740 10 client.28378 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.788+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1685930191 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x558022248620 con 0x5580221641b0 2024-07-31T15:07:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.788+0000 7fbd037f6640 1 -- 172.21.15.67:0/1685930191 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd1401eb00 con 0x5580221641b0 2024-07-31T15:07:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.788+0000 7fbd037f6640 1 -- 172.21.15.67:0/1685930191 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fbd1800b040 con 0x5580221641b0 2024-07-31T15:07:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.790+0000 7fbd2cf98740 10 client.28378 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.790+0000 7fbd2cf98740 10 client.28378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.790+0000 7fbd2cf98740 20 client.28378 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.790+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1685930191 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558022122d80 0x558022535b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.790+0000 7fbd2cf98740 4 client.28378 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:07:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.790+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1685930191 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55802247e300 con 0x558022122d80 2024-07-31T15:07:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.790+0000 7fbd1f7fe640 1 --2- 172.21.15.67:0/1685930191 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558022122d80 0x558022535b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.791+0000 7fbd037f6640 10 client.28378 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.791+0000 7fbd1f7fe640 1 --2- 172.21.15.67:0/1685930191 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558022122d80 0x558022535b90 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79dd2887e5ad148f server_cookie=9e4d03f62b4cb6ad in_seq=0 out_seq=0 2024-07-31T15:07:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.791+0000 7fbd037f6640 1 -- 172.21.15.67:0/1685930191 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55802247e300 con 0x558022122d80 2024-07-31T15:07:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.791+0000 7fbd037f6640 10 client.28378 handle_command_reply: tid=0 2024-07-31T15:07:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.792+0000 7fbd2cf98740 1 client.28378 shutdown 2024-07-31T15:07:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.792+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1685930191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 msgr2=0x55802252fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.792+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1685930191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x55802252fa00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbd180ab9b0 tx=0x7fbd180e2ad0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.792+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1685930191 shutdown_connections 2024-07-31T15:07:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.792+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1685930191 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558022122d80 0x558022535b90 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.792+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1685930191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x55802252fa00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.792+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1685930191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 0x558022512f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.792+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1685930191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 0x558022512a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.792+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1685930191 >> 172.21.15.67:0/1685930191 conn(0x558022158300 msgr2=0x558022507ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.792+0000 7fbd037f6640 0 client.28378 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.792+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1685930191 shutdown_connections 2024-07-31T15:07:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.792+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1685930191 wait complete. 2024-07-31T15:07:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.792+0000 7fbd2cf98740 20 client.28378 trim_cache size 0 max 16384 2024-07-31T15:07:12.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:12 smithi067 ceph-mon[32750]: pgmap v910: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 207 KiB/s wr, 61 op/s 2024-07-31T15:07:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.826+0000 7fbd2cf98740 1 Processor -- start 2024-07-31T15:07:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.826+0000 7fbd2cf98740 1 -- start start 2024-07-31T15:07:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.827+0000 7fbd2cf98740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 0x558022503d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.827+0000 7fbd2cf98740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 0x55802252f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.827+0000 7fbd2cf98740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x5580225085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.827+0000 7fbd2cf98740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c010000 con 0x55802213b2f0 2024-07-31T15:07:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.827+0000 7fbd2cf98740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c011b40 con 0x5580221434d0 2024-07-31T15:07:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.827+0000 7fbd1effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 0x55802252f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.827+0000 7fbd2cf98740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c0106d0 con 0x5580221641b0 2024-07-31T15:07:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.827+0000 7fbd1f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 0x558022503d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.827+0000 7fbd1effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 0x55802252f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44834/0 (socket says 172.21.15.67:44834) 2024-07-31T15:07:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.827+0000 7fbd1f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 0x558022503d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33130/0 (socket says 172.21.15.67:33130) 2024-07-31T15:07:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.827+0000 7fbd1effd640 1 -- 172.21.15.67:0/1937186892 learned_addr learned my addr 172.21.15.67:0/1937186892 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.827+0000 7fbd1ffff640 1 --2- 172.21.15.67:0/1937186892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x5580225085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.827+0000 7fbd1ffff640 1 -- 172.21.15.67:0/1937186892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 msgr2=0x55802252f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.828+0000 7fbd1ffff640 1 --2- 172.21.15.67:0/1937186892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 0x55802252f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.828+0000 7fbd1ffff640 1 -- 172.21.15.67:0/1937186892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 msgr2=0x558022503d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.828+0000 7fbd1ffff640 1 --2- 172.21.15.67:0/1937186892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 0x558022503d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.828+0000 7fbd1ffff640 1 -- 172.21.15.67:0/1937186892 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580225093d0 con 0x5580221641b0 2024-07-31T15:07:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.828+0000 7fbd1effd640 1 --2- 172.21.15.67:0/1937186892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 0x55802252f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:07:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.828+0000 7fbd1ffff640 1 --2- 172.21.15.67:0/1937186892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x5580225085e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbd18004910 tx=0x7fbd18014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c455db74b037333 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:12.831 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/dma/mediatek/built-in.a 2024-07-31T15:07:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.839+0000 7fbd057fa640 1 -- 172.21.15.67:0/1937186892 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd180a71b0 con 0x5580221641b0 2024-07-31T15:07:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.839+0000 7fbd057fa640 1 -- 172.21.15.67:0/1937186892 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd180aba20 con 0x5580221641b0 2024-07-31T15:07:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.839+0000 7fbd057fa640 1 -- 172.21.15.67:0/1937186892 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1800e0b0 con 0x5580221641b0 2024-07-31T15:07:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.840+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1937186892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 msgr2=0x5580225085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.840+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1937186892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x5580225085e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbd18004910 tx=0x7fbd18014000 comp rx=0 tx=0).stop 2024-07-31T15:07:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.840+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1937186892 shutdown_connections 2024-07-31T15:07:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.840+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1937186892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x5580225085e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.840+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1937186892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 0x55802252f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.840+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1937186892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 0x558022503d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.840+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1937186892 >> 172.21.15.67:0/1937186892 conn(0x558022158300 msgr2=0x558022507ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.840+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1937186892 shutdown_connections 2024-07-31T15:07:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.840+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1937186892 wait complete. 2024-07-31T15:07:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.841+0000 7fbd2cf98740 1 -- start start 2024-07-31T15:07:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.841+0000 7fbd2cf98740 1 Processor -- start 2024-07-31T15:07:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.842+0000 7fbd2cf98740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 0x55802252c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.842+0000 7fbd2cf98740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 0x558022528930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.842+0000 7fbd2cf98740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x558022528e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.842+0000 7fbd2cf98740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c011470 con 0x55802213b2f0 2024-07-31T15:07:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.842+0000 7fbd2cf98740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c010da0 con 0x5580221434d0 2024-07-31T15:07:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.842+0000 7fbd2cf98740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c00e000 con 0x5580221641b0 2024-07-31T15:07:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.842+0000 7fbd2cf98740 10 client.0 _ll_register_callbacks cb 0x7fbd180099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.842+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x558022528e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.842+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x558022528e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38068/0 (socket says 172.21.15.67:38068) 2024-07-31T15:07:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.842+0000 7fbd1ffff640 1 -- 172.21.15.67:0/1581346282 learned_addr learned my addr 172.21.15.67:0/1581346282 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.842+0000 7fbd1effd640 1 --2- 172.21.15.67:0/1581346282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 0x558022528930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.843+0000 7fbd1f7fe640 1 --2- 172.21.15.67:0/1581346282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 0x55802252c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.843+0000 7fbd1f7fe640 1 -- 172.21.15.67:0/1581346282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 msgr2=0x558022528e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.843+0000 7fbd1f7fe640 1 --2- 172.21.15.67:0/1581346282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x558022528e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.843+0000 7fbd1f7fe640 1 -- 172.21.15.67:0/1581346282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 msgr2=0x558022528930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.843+0000 7fbd1f7fe640 1 --2- 172.21.15.67:0/1581346282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 0x558022528930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.843+0000 7fbd1f7fe640 1 -- 172.21.15.67:0/1581346282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558022529600 con 0x55802213b2f0 2024-07-31T15:07:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.849+0000 7fbd1f7fe640 1 --2- 172.21.15.67:0/1581346282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 0x55802252c050 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009780 tx=0x7fbd10030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3b84b9a8204de0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.849+0000 7fbd037f6640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:07:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.850+0000 7fbd037f6640 1 -- 172.21.15.67:0/1581346282 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10037040 con 0x55802213b2f0 2024-07-31T15:07:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.850+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1581346282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd180aba20 con 0x55802213b2f0 2024-07-31T15:07:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.850+0000 7fbd037f6640 1 -- 172.21.15.67:0/1581346282 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9483+0+0 (secure 0 0 0) 0x7fbd10030450 con 0x55802213b2f0 2024-07-31T15:07:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.850+0000 7fbd037f6640 1 -- 172.21.15.67:0/1581346282 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fbd100044d0 con 0x55802213b2f0 2024-07-31T15:07:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.850+0000 7fbd2cf98740 10 client.18963 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.851+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1581346282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55802259d280 con 0x55802213b2f0 2024-07-31T15:07:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.851+0000 7fbd037f6640 1 -- 172.21.15.67:0/1581346282 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd180aba20 con 0x55802213b2f0 2024-07-31T15:07:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.851+0000 7fbd037f6640 1 -- 172.21.15.67:0/1581346282 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fbd10004940 con 0x55802213b2f0 2024-07-31T15:07:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.852+0000 7fbd2cf98740 10 client.18963 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.852+0000 7fbd2cf98740 10 client.18963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.852+0000 7fbd2cf98740 20 client.18963 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.852+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1581346282 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5580220f81c0 0x55802259f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.852+0000 7fbd2cf98740 4 client.18963 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:07:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.852+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1581346282 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558022479da0 con 0x5580220f81c0 2024-07-31T15:07:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.852+0000 7fbd1effd640 1 --2- 172.21.15.67:0/1581346282 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5580220f81c0 0x55802259f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.853+0000 7fbd1effd640 1 --2- 172.21.15.67:0/1581346282 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5580220f81c0 0x55802259f9c0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=940cc01551fcea1f server_cookie=3d4741c42d415275 in_seq=0 out_seq=0 2024-07-31T15:07:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:12.859+0000 7fbd037f6640 10 client.18963 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:12.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:12 smithi073 ceph-mon[36714]: pgmap v910: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 207 KiB/s wr, 61 op/s 2024-07-31T15:07:12.935 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/dma/qcom/built-in.a 2024-07-31T15:07:12.963 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/net-sysfs.o 2024-07-31T15:07:12.991 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/dma/ti/built-in.a 2024-07-31T15:07:13.118 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/virtio/virtio_pci_modern_dev.o 2024-07-31T15:07:13.130 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/psobject.o 2024-07-31T15:07:13.291 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/audit.o 2024-07-31T15:07:13.332 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/auditfilter.o 2024-07-31T15:07:13.399 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/bitrev.o 2024-07-31T15:07:13.759 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/dma/xilinx/built-in.a 2024-07-31T15:07:13.821 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma/dmaengine.o 2024-07-31T15:07:13.920 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/psopcode.o 2024-07-31T15:07:14.143 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crc-ccitt.o 2024-07-31T15:07:14.360 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/virtio/virtio_pci_legacy_dev.o 2024-07-31T15:07:14.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.420+0000 7f604537c640 1 -- 172.21.15.67:0/3951432360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6040111970 msgr2=0x7f6040109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:14.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.420+0000 7f604537c640 1 --2- 172.21.15.67:0/3951432360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6040111970 0x7f6040109df0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6030009900 tx=0x7f603002a440 comp rx=0 tx=0).stop 2024-07-31T15:07:14.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.422+0000 7f604537c640 1 -- 172.21.15.67:0/3951432360 shutdown_connections 2024-07-31T15:07:14.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.422+0000 7f604537c640 1 --2- 172.21.15.67:0/3951432360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f604011c660 0x7f604011eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.422+0000 7f604537c640 1 --2- 172.21.15.67:0/3951432360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f604010a410 0x7f604010a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.422+0000 7f604537c640 1 --2- 172.21.15.67:0/3951432360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6040111970 0x7f6040109df0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.422+0000 7f604537c640 1 -- 172.21.15.67:0/3951432360 >> 172.21.15.67:0/3951432360 conn(0x7f60400708f0 msgr2=0x7f6040070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.427+0000 7f604537c640 1 -- 172.21.15.67:0/3951432360 shutdown_connections 2024-07-31T15:07:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.427+0000 7f604537c640 1 -- 172.21.15.67:0/3951432360 wait complete. 2024-07-31T15:07:14.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.428+0000 7f604537c640 1 Processor -- start 2024-07-31T15:07:14.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.429+0000 7f604537c640 1 -- start start 2024-07-31T15:07:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.429+0000 7f604537c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f604010a410 0x7f60401a8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.429+0000 7f604537c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6040111970 0x7f60401a9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.429+0000 7f604537c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f604011c660 0x7f60401ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.429+0000 7f604537c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6040122310 con 0x7f604010a410 2024-07-31T15:07:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.429+0000 7f604537c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6040122190 con 0x7f6040111970 2024-07-31T15:07:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.429+0000 7f604537c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6040122490 con 0x7f604011c660 2024-07-31T15:07:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.430+0000 7f603f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f604011c660 0x7f60401ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.431+0000 7f603f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f604011c660 0x7f60401ae520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38082/0 (socket says 172.21.15.67:38082) 2024-07-31T15:07:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.431+0000 7f603f7fe640 1 -- 172.21.15.67:0/377099618 learned_addr learned my addr 172.21.15.67:0/377099618 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.434+0000 7f603e7fc640 1 --2- 172.21.15.67:0/377099618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6040111970 0x7f60401a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.434+0000 7f603effd640 1 --2- 172.21.15.67:0/377099618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f604010a410 0x7f60401a8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.436+0000 7f603f7fe640 1 -- 172.21.15.67:0/377099618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6040111970 msgr2=0x7f60401a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.436+0000 7f603f7fe640 1 --2- 172.21.15.67:0/377099618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6040111970 0x7f60401a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.436+0000 7f603f7fe640 1 -- 172.21.15.67:0/377099618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f604010a410 msgr2=0x7f60401a8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.436+0000 7f603f7fe640 1 --2- 172.21.15.67:0/377099618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f604010a410 0x7f60401a8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.436+0000 7f603f7fe640 1 -- 172.21.15.67:0/377099618 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60401aec50 con 0x7f604011c660 2024-07-31T15:07:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.436+0000 7f603f7fe640 1 --2- 172.21.15.67:0/377099618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f604011c660 0x7f60401ae520 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f603400d950 tx=0x7f603400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:14.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.441+0000 7f6027fff640 1 -- 172.21.15.67:0/377099618 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034017030 con 0x7f604011c660 2024-07-31T15:07:14.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.441+0000 7f6027fff640 1 -- 172.21.15.67:0/377099618 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6034004020 con 0x7f604011c660 2024-07-31T15:07:14.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.442+0000 7f6027fff640 1 -- 172.21.15.67:0/377099618 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034003b50 con 0x7f604011c660 2024-07-31T15:07:14.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.443+0000 7f604537c640 1 -- 172.21.15.67:0/377099618 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60401aef40 con 0x7f604011c660 2024-07-31T15:07:14.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.443+0000 7f604537c640 1 -- 172.21.15.67:0/377099618 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60401b56a0 con 0x7f604011c660 2024-07-31T15:07:14.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.449+0000 7f6027fff640 1 -- 172.21.15.67:0/377099618 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f603401a070 con 0x7f604011c660 2024-07-31T15:07:14.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.458+0000 7f6027fff640 1 --2- 172.21.15.67:0/377099618 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f60280b3df0 0x7f60280b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.459+0000 7f6027fff640 1 -- 172.21.15.67:0/377099618 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9483+0+0 (secure 0 0 0) 0x7f60340e3260 con 0x7f604011c660 2024-07-31T15:07:14.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.459+0000 7f603effd640 1 --2- 172.21.15.67:0/377099618 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f60280b3df0 0x7f60280b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:14.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.460+0000 7f604b8cd740 1 Processor -- start 2024-07-31T15:07:14.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.460+0000 7f604b8cd740 1 -- start start 2024-07-31T15:07:14.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.460+0000 7f604b8cd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2c21350 0x55f0b2fc63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.460+0000 7f604b8cd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2bf8490 0x55f0b2fc68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.460+0000 7f604b8cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2c00670 0x55f0b2fc6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.460+0000 7f604b8cd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f0b2eae2f0 con 0x55f0b2c21350 2024-07-31T15:07:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.460+0000 7f604b8cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f0b2f3ca80 con 0x55f0b2c00670 2024-07-31T15:07:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.460+0000 7f604b8cd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f0b2f2bf80 con 0x55f0b2bf8490 2024-07-31T15:07:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.461+0000 7f603f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2c00670 0x55f0b2fc6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.461+0000 7f603f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2c00670 0x55f0b2fc6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44880/0 (socket says 172.21.15.67:44880) 2024-07-31T15:07:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.461+0000 7f603f7fe640 1 -- 172.21.15.67:0/3665928848 learned_addr learned my addr 172.21.15.67:0/3665928848 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.461+0000 7f603f7fe640 1 -- 172.21.15.67:0/3665928848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2bf8490 msgr2=0x55f0b2fc68f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:07:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.461+0000 7f603f7fe640 1 --2- 172.21.15.67:0/3665928848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2bf8490 0x55f0b2fc68f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.461+0000 7f603f7fe640 1 -- 172.21.15.67:0/3665928848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2c21350 msgr2=0x55f0b2fc63b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:07:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.461+0000 7f603f7fe640 1 --2- 172.21.15.67:0/3665928848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2c21350 0x55f0b2fc63b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.461+0000 7f603f7fe640 1 -- 172.21.15.67:0/3665928848 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0b2c8c930 con 0x55f0b2c00670 2024-07-31T15:07:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.461+0000 7f603f7fe640 1 --2- 172.21.15.67:0/3665928848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2c00670 0x55f0b2fc6e30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f60340a5b50 tx=0x7f60340aa290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f110afbdb8c48eaf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:14.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.464+0000 7f60257fa640 1 -- 172.21.15.67:0/3665928848 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60340c0020 con 0x55f0b2c00670 2024-07-31T15:07:14.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.464+0000 7f60257fa640 1 -- 172.21.15.67:0/3665928848 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60340b2b80 con 0x55f0b2c00670 2024-07-31T15:07:14.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.464+0000 7f60257fa640 1 -- 172.21.15.67:0/3665928848 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60340041d0 con 0x55f0b2c00670 2024-07-31T15:07:14.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.470+0000 7f604b8cd740 1 -- 172.21.15.67:0/3665928848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2c00670 msgr2=0x55f0b2fc6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:14.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.470+0000 7f604b8cd740 1 --2- 172.21.15.67:0/3665928848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2c00670 0x55f0b2fc6e30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f60340a5b50 tx=0x7f60340aa290 comp rx=0 tx=0).stop 2024-07-31T15:07:14.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.474+0000 7f603effd640 1 --2- 172.21.15.67:0/377099618 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f60280b3df0 0x7f60280b6300 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6030004770 tx=0x7f6030005a00 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:14.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.474+0000 7f604b8cd740 1 -- 172.21.15.67:0/3665928848 shutdown_connections 2024-07-31T15:07:14.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.474+0000 7f604b8cd740 1 --2- 172.21.15.67:0/3665928848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2c00670 0x55f0b2fc6e30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.474+0000 7f604b8cd740 1 --2- 172.21.15.67:0/3665928848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2bf8490 0x55f0b2fc68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.474+0000 7f604b8cd740 1 --2- 172.21.15.67:0/3665928848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2c21350 0x55f0b2fc63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.474+0000 7f604b8cd740 1 -- 172.21.15.67:0/3665928848 >> 172.21.15.67:0/3665928848 conn(0x55f0b2c15740 msgr2=0x55f0b2fc42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:14.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.482+0000 7f604b8cd740 1 -- 172.21.15.67:0/3665928848 shutdown_connections 2024-07-31T15:07:14.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.487+0000 7f604b8cd740 1 -- 172.21.15.67:0/3665928848 wait complete. 2024-07-31T15:07:14.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.487+0000 7f604b8cd740 1 -- start start 2024-07-31T15:07:14.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.487+0000 7f604b8cd740 1 Processor -- start 2024-07-31T15:07:14.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.488+0000 7f604b8cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bf8490 0x55f0b2fd2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.488+0000 7f604b8cd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c00670 0x55f0b2fd2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.488+0000 7f604b8cd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2c21350 0x55f0b2ff9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.488+0000 7f604b8cd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f0b2ece4f0 con 0x55f0b2c21350 2024-07-31T15:07:14.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.488+0000 7f604b8cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f0b2f2ffc0 con 0x55f0b2bf8490 2024-07-31T15:07:14.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.488+0000 7f604b8cd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f0b2d8f890 con 0x55f0b2c00670 2024-07-31T15:07:14.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.488+0000 7f604b8cd740 10 client.0 _ll_register_callbacks cb 0x55f0b2ed8500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:14.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.489+0000 7f603e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c00670 0x55f0b2fd2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:14.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.491+0000 7f603e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c00670 0x55f0b2fd2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38098/0 (socket says 172.21.15.67:38098) 2024-07-31T15:07:14.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.491+0000 7f603e7fc640 1 -- 172.21.15.67:0/2678817993 learned_addr learned my addr 172.21.15.67:0/2678817993 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:14.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.493+0000 7f603f7fe640 1 --2- 172.21.15.67:0/2678817993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2c21350 0x55f0b2ff9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:14.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.495+0000 7f603effd640 1 --2- 172.21.15.67:0/2678817993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bf8490 0x55f0b2fd2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:14.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.497+0000 7f603e7fc640 1 -- 172.21.15.67:0/2678817993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bf8490 msgr2=0x55f0b2fd2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:14.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.497+0000 7f603e7fc640 1 --2- 172.21.15.67:0/2678817993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bf8490 0x55f0b2fd2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.497+0000 7f603e7fc640 1 -- 172.21.15.67:0/2678817993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2c21350 msgr2=0x55f0b2ff9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:14.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.497+0000 7f603e7fc640 1 --2- 172.21.15.67:0/2678817993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2c21350 0x55f0b2ff9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.497+0000 7f603e7fc640 1 -- 172.21.15.67:0/2678817993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0b2eddc50 con 0x55f0b2c00670 2024-07-31T15:07:14.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.497+0000 7f603e7fc640 1 --2- 172.21.15.67:0/2678817993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c00670 0x55f0b2fd2620 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6038011940 tx=0x7f6038011e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7e367e305e7fa6e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:14.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.498+0000 7f6022ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:07:14.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.504+0000 7f6022ff5640 1 -- 172.21.15.67:0/2678817993 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603800fc50 con 0x55f0b2c00670 2024-07-31T15:07:14.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.504+0000 7f6022ff5640 1 -- 172.21.15.67:0/2678817993 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9483+0+0 (secure 0 0 0) 0x7f603800fdf0 con 0x55f0b2c00670 2024-07-31T15:07:14.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.504+0000 7f604b8cd740 1 -- 172.21.15.67:0/2678817993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60340b2b80 con 0x55f0b2c00670 2024-07-31T15:07:14.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.506+0000 7f6022ff5640 1 -- 172.21.15.67:0/2678817993 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f6038024050 con 0x55f0b2c00670 2024-07-31T15:07:14.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.506+0000 7f604b8cd740 10 client.28393 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:14.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.506+0000 7f604b8cd740 1 -- 172.21.15.67:0/2678817993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55f0b3002ba0 con 0x55f0b2c00670 2024-07-31T15:07:14.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.508+0000 7f6022ff5640 1 -- 172.21.15.67:0/2678817993 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60340b2b80 con 0x55f0b2c00670 2024-07-31T15:07:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.508+0000 7f6022ff5640 1 -- 172.21.15.67:0/2678817993 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f603800e5c0 con 0x55f0b2c00670 2024-07-31T15:07:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.508+0000 7f604b8cd740 10 client.28393 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.508+0000 7f604b8cd740 10 client.28393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.508+0000 7f604b8cd740 20 client.28393 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.508+0000 7f604b8cd740 1 --2- 172.21.15.67:0/2678817993 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f0b2bdfcd0 0x55f0b3006140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.508+0000 7f604b8cd740 4 client.28393 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:07:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.508+0000 7f604b8cd740 1 -- 172.21.15.67:0/2678817993 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0b2f3e9d0 con 0x55f0b2bdfcd0 2024-07-31T15:07:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.509+0000 7f603effd640 1 --2- 172.21.15.67:0/2678817993 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f0b2bdfcd0 0x55f0b3006140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.509+0000 7f6022ff5640 10 client.28393 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.509+0000 7f603effd640 1 --2- 172.21.15.67:0/2678817993 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f0b2bdfcd0 0x55f0b3006140 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb39b41525d7dbf4 server_cookie=70f930b62bf6f2bc in_seq=0 out_seq=0 2024-07-31T15:07:14.513 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/psopinfo.o 2024-07-31T15:07:14.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.512+0000 7f6022ff5640 1 -- 172.21.15.67:0/2678817993 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0b2f3e9d0 con 0x55f0b2bdfcd0 2024-07-31T15:07:14.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.512+0000 7f6022ff5640 10 client.28393 handle_command_reply: tid=0 2024-07-31T15:07:14.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.512+0000 7f604b8cd740 1 client.28393 shutdown 2024-07-31T15:07:14.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.518+0000 7f604b8cd740 1 -- 172.21.15.67:0/2678817993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c00670 msgr2=0x55f0b2fd2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:14.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.518+0000 7f604b8cd740 1 --2- 172.21.15.67:0/2678817993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c00670 0x55f0b2fd2620 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6038011940 tx=0x7f6038011e00 comp rx=0 tx=0).stop 2024-07-31T15:07:14.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.520+0000 7f604b8cd740 1 -- 172.21.15.67:0/2678817993 shutdown_connections 2024-07-31T15:07:14.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.520+0000 7f604b8cd740 1 --2- 172.21.15.67:0/2678817993 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f0b2bdfcd0 0x55f0b3006140 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.520+0000 7f604b8cd740 1 --2- 172.21.15.67:0/2678817993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2c21350 0x55f0b2ff9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.520+0000 7f604b8cd740 1 --2- 172.21.15.67:0/2678817993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c00670 0x55f0b2fd2620 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6038011940 tx=0x7f6038011e00 comp rx=0 tx=0).stop 2024-07-31T15:07:14.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.520+0000 7f604b8cd740 1 --2- 172.21.15.67:0/2678817993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bf8490 0x55f0b2fd2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.520+0000 7f604b8cd740 1 -- 172.21.15.67:0/2678817993 >> 172.21.15.67:0/2678817993 conn(0x55f0b2c15740 msgr2=0x55f0b2fc9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:14.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.520+0000 7f6022ff5640 0 client.28393 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.526+0000 7f604b8cd740 1 -- 172.21.15.67:0/2678817993 shutdown_connections 2024-07-31T15:07:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.530+0000 7f604b8cd740 1 -- 172.21.15.67:0/2678817993 wait complete. 2024-07-31T15:07:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.530+0000 7f604b8cd740 20 client.28393 trim_cache size 0 max 16384 2024-07-31T15:07:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.567+0000 7f604b8cd740 1 Processor -- start 2024-07-31T15:07:14.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.571+0000 7f604b8cd740 1 -- start start 2024-07-31T15:07:14.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.571+0000 7f604b8cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bf8490 0x55f0b2fcd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.571+0000 7f604b8cd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c21350 0x55f0b3000ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.571+0000 7f604b8cd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2bb5080 0x55f0b2ff9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.571+0000 7f604b8cd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6040080f20 con 0x55f0b2bb5080 2024-07-31T15:07:14.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.571+0000 7f604b8cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f604007ed10 con 0x55f0b2bf8490 2024-07-31T15:07:14.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.571+0000 7f604b8cd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6040080850 con 0x55f0b2c21350 2024-07-31T15:07:14.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.571+0000 7f603effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bf8490 0x55f0b2fcd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:14.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.572+0000 7f603effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bf8490 0x55f0b2fcd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44898/0 (socket says 172.21.15.67:44898) 2024-07-31T15:07:14.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.572+0000 7f603effd640 1 -- 172.21.15.67:0/411042588 learned_addr learned my addr 172.21.15.67:0/411042588 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:14.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.572+0000 7f603effd640 1 -- 172.21.15.67:0/411042588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c21350 msgr2=0x55f0b3000ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:07:14.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.572+0000 7f603effd640 1 --2- 172.21.15.67:0/411042588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c21350 0x55f0b3000ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.572+0000 7f603effd640 1 -- 172.21.15.67:0/411042588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2bb5080 msgr2=0x55f0b2ff9c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:07:14.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.572+0000 7f603effd640 1 --2- 172.21.15.67:0/411042588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2bb5080 0x55f0b2ff9c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.572+0000 7f603effd640 1 -- 172.21.15.67:0/411042588 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0b2fd0dc0 con 0x55f0b2bf8490 2024-07-31T15:07:14.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.574+0000 7f603effd640 1 --2- 172.21.15.67:0/411042588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bf8490 0x55f0b2fcd610 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6030000c00 tx=0x7f6030040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b57edfa36320171f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.579+0000 7f6024ff9640 1 -- 172.21.15.67:0/411042588 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030035040 con 0x55f0b2bf8490 2024-07-31T15:07:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.579+0000 7f6024ff9640 1 -- 172.21.15.67:0/411042588 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6030056070 con 0x55f0b2bf8490 2024-07-31T15:07:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.579+0000 7f6024ff9640 1 -- 172.21.15.67:0/411042588 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030050570 con 0x55f0b2bf8490 2024-07-31T15:07:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.580+0000 7f604b8cd740 1 -- 172.21.15.67:0/411042588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bf8490 msgr2=0x55f0b2fcd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.580+0000 7f604b8cd740 1 --2- 172.21.15.67:0/411042588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bf8490 0x55f0b2fcd610 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6030000c00 tx=0x7f6030040000 comp rx=0 tx=0).stop 2024-07-31T15:07:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.581+0000 7f604b8cd740 1 -- 172.21.15.67:0/411042588 shutdown_connections 2024-07-31T15:07:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.581+0000 7f604b8cd740 1 --2- 172.21.15.67:0/411042588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2bb5080 0x55f0b2ff9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.581+0000 7f604b8cd740 1 --2- 172.21.15.67:0/411042588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c21350 0x55f0b3000ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.581+0000 7f604b8cd740 1 --2- 172.21.15.67:0/411042588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bf8490 0x55f0b2fcd610 secure :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6030000c00 tx=0x7f6030040000 comp rx=0 tx=0).stop 2024-07-31T15:07:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.581+0000 7f604b8cd740 1 -- 172.21.15.67:0/411042588 >> 172.21.15.67:0/411042588 conn(0x55f0b2c15740 msgr2=0x55f0b2fc3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:14.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.589+0000 7f604b8cd740 1 -- 172.21.15.67:0/411042588 shutdown_connections 2024-07-31T15:07:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.589+0000 7f604b8cd740 1 -- 172.21.15.67:0/411042588 wait complete. 2024-07-31T15:07:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.590+0000 7f604b8cd740 1 -- start start 2024-07-31T15:07:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.590+0000 7f604b8cd740 1 Processor -- start 2024-07-31T15:07:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.591+0000 7f604b8cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bb5080 0x55f0b2ffbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.591+0000 7f604b8cd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c21350 0x55f0b2ffc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.591+0000 7f604b8cd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2fffd00 0x55f0b3000110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.591+0000 7f604b8cd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f604007fab0 con 0x55f0b2fffd00 2024-07-31T15:07:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.591+0000 7f604b8cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f604007f3e0 con 0x55f0b2bb5080 2024-07-31T15:07:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.591+0000 7f604b8cd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6040080180 con 0x55f0b2c21350 2024-07-31T15:07:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.591+0000 7f604b8cd740 10 client.0 _ll_register_callbacks cb 0x7f6038011580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.591+0000 7f603e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c21350 0x55f0b2ffc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:14.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.591+0000 7f603e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c21350 0x55f0b2ffc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38114/0 (socket says 172.21.15.67:38114) 2024-07-31T15:07:14.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.591+0000 7f603e7fc640 1 -- 172.21.15.67:0/1353794428 learned_addr learned my addr 172.21.15.67:0/1353794428 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:14.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.595+0000 7f603effd640 1 --2- 172.21.15.67:0/1353794428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bb5080 0x55f0b2ffbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:14.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.599+0000 7f603effd640 1 -- 172.21.15.67:0/1353794428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c21350 msgr2=0x55f0b2ffc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:14.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.599+0000 7f603effd640 1 --2- 172.21.15.67:0/1353794428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c21350 0x55f0b2ffc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.599+0000 7f603effd640 1 -- 172.21.15.67:0/1353794428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2fffd00 msgr2=0x55f0b3000110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:14.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.599+0000 7f603effd640 1 --2- 172.21.15.67:0/1353794428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2fffd00 0x55f0b3000110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.599+0000 7f603effd640 1 -- 172.21.15.67:0/1353794428 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0b3000650 con 0x55f0b2bb5080 2024-07-31T15:07:14.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.604+0000 7f603effd640 1 --2- 172.21.15.67:0/1353794428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bb5080 0x55f0b2ffbd00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f603004b000 tx=0x7f6030031f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3dd6f278de88d5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.606+0000 7f6022ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:07:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.606+0000 7f6022ff5640 1 -- 172.21.15.67:0/1353794428 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603005e020 con 0x55f0b2bb5080 2024-07-31T15:07:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.606+0000 7f6022ff5640 1 -- 172.21.15.67:0/1353794428 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9483+0+0 (secure 0 0 0) 0x7f60300314a0 con 0x55f0b2bb5080 2024-07-31T15:07:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.608+0000 7f604b8cd740 1 -- 172.21.15.67:0/1353794428 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6030056070 con 0x55f0b2bb5080 2024-07-31T15:07:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.616+0000 7f6022ff5640 1 -- 172.21.15.67:0/1353794428 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f6030035070 con 0x55f0b2bb5080 2024-07-31T15:07:14.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.620+0000 7f604b8cd740 10 client.28487 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:14.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.620+0000 7f604b8cd740 1 -- 172.21.15.67:0/1353794428 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55f0b3068ae0 con 0x55f0b2bb5080 2024-07-31T15:07:14.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.623+0000 7f6022ff5640 1 -- 172.21.15.67:0/1353794428 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6030056070 con 0x55f0b2bb5080 2024-07-31T15:07:14.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.623+0000 7f6022ff5640 1 -- 172.21.15.67:0/1353794428 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f603003c980 con 0x55f0b2bb5080 2024-07-31T15:07:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.626+0000 7f604b8cd740 10 client.28487 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.626+0000 7f604b8cd740 10 client.28487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.626+0000 7f604b8cd740 20 client.28487 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.626+0000 7f604b8cd740 1 --2- 172.21.15.67:0/1353794428 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f0b3069110 0x55f0b306b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.626+0000 7f604b8cd740 4 client.28487 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T15:07:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.626+0000 7f604b8cd740 1 -- 172.21.15.67:0/1353794428 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f0b2f3a470 con 0x55f0b3069110 2024-07-31T15:07:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.627+0000 7f603e7fc640 1 --2- 172.21.15.67:0/1353794428 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f0b3069110 0x55f0b306b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:14.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.627+0000 7f603e7fc640 1 --2- 172.21.15.67:0/1353794428 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f0b3069110 0x55f0b306b530 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bd17a26dc10fb1d server_cookie=8936c3bd3000a3ec in_seq=0 out_seq=0 2024-07-31T15:07:14.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.627+0000 7f6022ff5640 10 client.28487 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.628+0000 7f6022ff5640 1 -- 172.21.15.67:0/1353794428 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f0b2f3a470 con 0x55f0b3069110 2024-07-31T15:07:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.628+0000 7f6022ff5640 10 client.28487 handle_command_reply: tid=0 2024-07-31T15:07:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.629+0000 7f604b8cd740 1 client.28487 shutdown 2024-07-31T15:07:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.631+0000 7f604b8cd740 1 -- 172.21.15.67:0/1353794428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bb5080 msgr2=0x55f0b2ffbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.631+0000 7f604b8cd740 1 --2- 172.21.15.67:0/1353794428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bb5080 0x55f0b2ffbd00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f603004b000 tx=0x7f6030031f10 comp rx=0 tx=0).stop 2024-07-31T15:07:14.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.633+0000 7f604b8cd740 1 -- 172.21.15.67:0/1353794428 shutdown_connections 2024-07-31T15:07:14.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.633+0000 7f604b8cd740 1 --2- 172.21.15.67:0/1353794428 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f0b3069110 0x55f0b306b530 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.633+0000 7f604b8cd740 1 --2- 172.21.15.67:0/1353794428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f0b2fffd00 0x55f0b3000110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.633+0000 7f604b8cd740 1 --2- 172.21.15.67:0/1353794428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0b2c21350 0x55f0b2ffc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.633+0000 7f604b8cd740 1 --2- 172.21.15.67:0/1353794428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0b2bb5080 0x55f0b2ffbd00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.633+0000 7f604b8cd740 1 -- 172.21.15.67:0/1353794428 >> 172.21.15.67:0/1353794428 conn(0x55f0b2c15740 msgr2=0x55f0b2fd3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:14.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.633+0000 7f6022ff5640 0 client.28487 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:14.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.637+0000 7f604b8cd740 1 -- 172.21.15.67:0/1353794428 shutdown_connections 2024-07-31T15:07:14.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.637+0000 7f604b8cd740 1 -- 172.21.15.67:0/1353794428 wait complete. 2024-07-31T15:07:14.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.637+0000 7f604b8cd740 20 client.28487 trim_cache size 0 max 16384 2024-07-31T15:07:14.642 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:07:14.642 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T15:07:14.643 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "d8f6aa19-03ce-4e8d-ba33-a8e8f37936e1", 2024-07-31T15:07:14.643 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T15:07:14.643 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:07:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.640+0000 7f604537c640 1 -- 172.21.15.67:0/377099618 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f60280b3df0 msgr2=0x7f60280b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.640+0000 7f604537c640 1 --2- 172.21.15.67:0/377099618 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f60280b3df0 0x7f60280b6300 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6030004770 tx=0x7f6030005a00 comp rx=0 tx=0).stop 2024-07-31T15:07:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.640+0000 7f604537c640 1 -- 172.21.15.67:0/377099618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f604011c660 msgr2=0x7f60401ae520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.640+0000 7f604537c640 1 --2- 172.21.15.67:0/377099618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f604011c660 0x7f60401ae520 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f603400d950 tx=0x7f603400de10 comp rx=0 tx=0).stop 2024-07-31T15:07:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.640+0000 7f604537c640 1 -- 172.21.15.67:0/377099618 shutdown_connections 2024-07-31T15:07:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.640+0000 7f604537c640 1 --2- 172.21.15.67:0/377099618 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f60280b3df0 0x7f60280b6300 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.640+0000 7f604537c640 1 --2- 172.21.15.67:0/377099618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f604011c660 0x7f60401ae520 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.640+0000 7f604537c640 1 --2- 172.21.15.67:0/377099618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6040111970 0x7f60401a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.640+0000 7f604537c640 1 --2- 172.21.15.67:0/377099618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f604010a410 0x7f60401a8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.640+0000 7f604537c640 1 -- 172.21.15.67:0/377099618 >> 172.21.15.67:0/377099618 conn(0x7f60400708f0 msgr2=0x7f604011dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.640+0000 7f604537c640 1 -- 172.21.15.67:0/377099618 shutdown_connections 2024-07-31T15:07:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:14.640+0000 7f604537c640 1 -- 172.21.15.67:0/377099618 wait complete. 2024-07-31T15:07:14.645 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4sysctl.o 2024-07-31T15:07:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.033+0000 7fbd037f6640 1 -- 172.21.15.67:0/1581346282 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558022479da0 con 0x5580220f81c0 2024-07-31T15:07:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.033+0000 7fbd037f6640 10 client.18963 handle_command_reply: tid=0 2024-07-31T15:07:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.033+0000 7fbd2cf98740 1 client.18963 shutdown 2024-07-31T15:07:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.036+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1581346282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 msgr2=0x55802252c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.036+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1581346282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 0x55802252c050 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009780 tx=0x7fbd10030fc0 comp rx=0 tx=0).stop 2024-07-31T15:07:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.040+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1581346282 shutdown_connections 2024-07-31T15:07:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.040+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1581346282 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5580220f81c0 0x55802259f9c0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.040+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1581346282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580221641b0 0x558022528e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.040+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1581346282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5580221434d0 0x558022528930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.040+0000 7fbd2cf98740 1 --2- 172.21.15.67:0/1581346282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55802213b2f0 0x55802252c050 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.040+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1581346282 >> 172.21.15.67:0/1581346282 conn(0x558022158300 msgr2=0x558022503ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.040+0000 7fbd037f6640 0 client.18963 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.040+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1581346282 shutdown_connections 2024-07-31T15:07:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.040+0000 7fbd2cf98740 1 -- 172.21.15.67:0/1581346282 wait complete. 2024-07-31T15:07:15.045 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/psparse.o 2024-07-31T15:07:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.042+0000 7fbd2cf98740 20 client.18963 trim_cache size 0 max 16384 2024-07-31T15:07:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:07:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:07:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:07:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:07:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.066+0000 7fbd25fc8640 1 -- 172.21.15.67:0/1958780839 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd080b3f40 msgr2=0x7fbd080b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.066+0000 7fbd25fc8640 1 --2- 172.21.15.67:0/1958780839 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd080b3f40 0x7fbd080b6450 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbd140058c0 tx=0x7fbd14005850 comp rx=0 tx=0).stop 2024-07-31T15:07:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.066+0000 7fbd25fc8640 1 -- 172.21.15.67:0/1958780839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd2011c5d0 msgr2=0x7fbd201ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.066+0000 7fbd25fc8640 1 --2- 172.21.15.67:0/1958780839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd2011c5d0 0x7fbd201ae340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800efc0 tx=0x7fbd1800c800 comp rx=0 tx=0).stop 2024-07-31T15:07:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.069+0000 7fbd25fc8640 1 -- 172.21.15.67:0/1958780839 shutdown_connections 2024-07-31T15:07:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.069+0000 7fbd25fc8640 1 --2- 172.21.15.67:0/1958780839 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd080b3f40 0x7fbd080b6450 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.069+0000 7fbd25fc8640 1 --2- 172.21.15.67:0/1958780839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd2011c5d0 0x7fbd201ae340 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.069+0000 7fbd25fc8640 1 --2- 172.21.15.67:0/1958780839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd201116d0 0x7fbd201a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.069+0000 7fbd25fc8640 1 --2- 172.21.15.67:0/1958780839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd2010a330 0x7fbd201a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.069+0000 7fbd25fc8640 1 -- 172.21.15.67:0/1958780839 >> 172.21.15.67:0/1958780839 conn(0x7fbd2006e350 msgr2=0x7fbd20110350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.077+0000 7fbd25fc8640 1 -- 172.21.15.67:0/1958780839 shutdown_connections 2024-07-31T15:07:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:15.081+0000 7fbd25fc8640 1 -- 172.21.15.67:0/1958780839 wait complete. 2024-07-31T15:07:15.108 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:14 smithi073 ceph-mon[36714]: pgmap v911: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 412 KiB/s wr, 59 op/s 2024-07-31T15:07:15.108 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:15.108 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:15.108 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:15.108 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:07:14 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:07:14.689+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000162a [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch/ auth{1=1,2=1,3=1,4=1} v17517 ap=2 RANDEPHEMERALPIN f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22) n(v49 rc2024-07-31T15:05:35.600981+0000 b123297599 18652=17713+939) old_inodes=7 (inest sync->mix g=1,2,3,4) caps={15489=pAsLsXs/-@40} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7c26ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000162a:[<0x10000000202/arch v378>,<0x10000000201/linux-6.5.11 v96>,<0x10000000006/t v68>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v57>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v60>,<0x10000000001/sv_1 v84>,<0x10000000000/qa v70>,<0x1/volumes v70>]//[]","memoryvalue":"(2)0x1000000162a:[<0x10000000202/arch v17517>,<0x10000000201/linux-6.5.11 v2152>,<0x10000000006/t v1253>,<0x10000000005/tmp v1194>,<0x100000001fb/client.0 v1171>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1148>,<0x10000000001/sv_1 v1147>,<0x10000000000/qa v1147>,<0x1/volumes v1104>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:54:49.652308+0000 23=2+21)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:35.600981+0000 b123297599 18652=17713+939)","memory_value.dirstat":"f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22)","memory_value.rstat":"n(v49 rc2024-07-31T15:05:35.600981+0000 b123297599 18652=17713+939)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:07:15.109 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:07:14 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:07:14.790+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d9b4 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb/ auth{3=1} v3876 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7) n(v5 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8) old_inodes=3 (inest sync->mix g=3) caps={15489=pAsLsXsFs/pAsLsXsFsx@8} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce92cb2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v2020>,<0x10000000202/sound v9936>,<0x10000000201/linux-6.5.11 v830>,<0x10000000006/t v586>,<0x10000000005/tmp v553>,<0x100000001fb/client.0 v537>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v519>,<0x10000000001/sv_1 v525>,<0x10000000000/qa v522>,<0x1/volumes v503>]//[]","memoryvalue":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v3876>,<0x10000000202/sound v18372>,<0x10000000201/linux-6.5.11 v2152>,<0x10000000006/t v1253>,<0x10000000005/tmp v1194>,<0x100000001fb/client.0 v1171>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1148>,<0x10000000001/sv_1 v1147>,<0x10000000000/qa v1147>,<0x1/volumes v1104>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:46.317139+0000 53=46+7)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7)","memory_value.rstat":"n(v5 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:07:15.109 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:07:15.106+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c061 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math/ auth{1=1} v1164 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:10.213499+0000 31=31+0) n(v2 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1) old_inodes=2 (inest sync->mix g=1) caps={15489=pAsLsXs/-@6} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8e92fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c061:[<0x1000000bf5f/math v634>,<0x10000000202/lib v5688>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000c061:[<0x1000000bf5f/math v1164>,<0x10000000202/lib v18812>,<0x10000000201/linux-6.5.11 v2152>,<0x10000000006/t v1253>,<0x10000000005/tmp v1194>,<0x100000001fb/client.0 v1171>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1148>,<0x10000000001/sv_1 v1147>,<0x10000000000/qa v1147>,<0x1/volumes v1104>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:10.988109+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:10.213499+0000 31=31+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:07:15.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:14 smithi067 ceph-mon[32750]: pgmap v911: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 412 KiB/s wr, 59 op/s 2024-07-31T15:07:15.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:15.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:15.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:14 smithi077 ceph-mon[37327]: pgmap v911: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 412 KiB/s wr, 59 op/s 2024-07-31T15:07:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:15.316 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crc16.o 2024-07-31T15:07:15.627 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/virtio/virtio_pci_modern.o 2024-07-31T15:07:15.705 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:07:15.712 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/psscope.o 2024-07-31T15:07:15.795 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/auditsc.o 2024-07-31T15:07:15.865 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/pstree.o 2024-07-31T15:07:15.892 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:07:15.535+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000ce48 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity/ auth{1=1,2=1} v1695 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3) n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4) old_inodes=3 (inest sync->mix g=1,2) caps={15489=pAsLsXs/-@10} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8aaac580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000ce48:[<0x1000000ce07/integrity v345>,<0x10000000202/security v8766>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000ce48:[<0x1000000ce07/integrity v1695>,<0x10000000202/security v15110>,<0x10000000201/linux-6.5.11 v2152>,<0x10000000006/t v1253>,<0x10000000005/tmp v1194>,<0x100000001fb/client.0 v1171>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1148>,<0x10000000001/sv_1 v1147>,<0x10000000000/qa v1147>,<0x1/volumes v1104>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:59.470106+0000 10=7+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","memory_value.dirstat":"f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3)","memory_value.rstat":"n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:07:15.918 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC lib/gen_crc32table 2024-07-31T15:07:16.002 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/nfs/built-in.a 2024-07-31T15:07:16.199 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/xxhash.o 2024-07-31T15:07:16.236 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/audit_watch.o 2024-07-31T15:07:16.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T15:07:16.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:16.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:07:16.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:16.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:16.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:07:16.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: osdmap e117: 12 total, 12 up, 12 in 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:16.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:16.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T15:07:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:07:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:07:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:07:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: osdmap e117: 12 total, 12 up, 12 in 2024-07-31T15:07:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:16.350 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/psutils.o 2024-07-31T15:07:16.392 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:07:15.936+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009980 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video/ auth{1=1,4=1} v18105 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:16.340374+0000 28=24+4) n(v11 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28) old_inodes=2 (inest sync->mix g=1,4) caps={15489=pAsLsXs/-@6} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce87325b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009980:[<0x10000001d8f/video v17889>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009980:[<0x10000001d8f/video v18105>,<0x10000000202/drivers v18814>,<0x10000000201/linux-6.5.11 v2152>,<0x10000000006/t v1253>,<0x10000000005/tmp v1194>,<0x100000001fb/client.0 v1171>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1148>,<0x10000000001/sv_1 v1147>,<0x10000000000/qa v1147>,<0x1/volumes v1104>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:35.535846+0000 16=12+4)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:16.340374+0000 28=24+4)","memory_value.rstat":"n(v11 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:07:16.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T15:07:16.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:16.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:07:16.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:16.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:16.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:07:16.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:16.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:16.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:16.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:07:16.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: osdmap e117: 12 total, 12 up, 12 in 2024-07-31T15:07:16.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:16.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:16.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:16.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:16.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:16.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:16.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:16.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:16.406 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/autofs/init.o 2024-07-31T15:07:16.706 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma/virt-dma.o 2024-07-31T15:07:16.738 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:07:16.870 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/genalloc.o 2024-07-31T15:07:16.892 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/pswalk.o 2024-07-31T15:07:16.892 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:07:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:07:16.453+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000be32 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module/ auth{2=1} v2029 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0) n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1) old_inodes=1 (inest sync->mix g=2) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce826b7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000be32:[<0x1000000bd0a/module v1134>,<0x10000000202/kernel v5648>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000be32:[<0x1000000bd0a/module v2029>,<0x10000000202/kernel v18894>,<0x10000000201/linux-6.5.11 v2154>,<0x10000000006/t v1255>,<0x10000000005/tmp v1196>,<0x100000001fb/client.0 v1173>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1150>,<0x10000000001/sv_1 v1149>,<0x10000000000/qa v1149>,<0x1/volumes v1106>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:07.168188+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:07:16.933 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/virtio/virtio_pci_common.o 2024-07-31T15:07:16.957 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/net-procfs.o 2024-07-31T15:07:17.252 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/autofs/inode.o 2024-07-31T15:07:17.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:07:17.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:17 smithi067 ceph-mon[32750]: pgmap v913: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 542 KiB/s wr, 137 op/s 2024-07-31T15:07:17.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:07:17.454 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:07:17 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:07:17.241+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d119 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97/ auth{2=1} v3208 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0) n(v3 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1) old_inodes=3 (inest sync->mix g=2) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce9047e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d119:[<0x1000000d116/ac97 v22>,<0x1000000cf1b/pci v463>,<0x10000000202/sound v8728>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000d119:[<0x1000000d116/ac97 v3208>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v18372>,<0x10000000201/linux-6.5.11 v2154>,<0x10000000006/t v1255>,<0x10000000005/tmp v1196>,<0x100000001fb/client.0 v1173>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1150>,<0x10000000001/sv_1 v1149>,<0x10000000000/qa v1149>,<0x1/volumes v1106>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:00.491085+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:07:17.454 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:07:17 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:07:17.245+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d187 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi/ auth{1=1} v3040 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0) n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1) old_inodes=1 (inest sync->mix g=1) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7c6c1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d187:[<0x1000000d116/ctxfi v1858>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d187:[<0x1000000d116/ctxfi v3040>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v18372>,<0x10000000201/linux-6.5.11 v2154>,<0x10000000006/t v1255>,<0x10000000005/tmp v1196>,<0x100000001fb/client.0 v1173>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1150>,<0x10000000001/sv_1 v1149>,<0x10000000000/qa v1149>,<0x1/volumes v1106>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.452342+0000 30=30+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:07:17.454 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:07:17 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:07:17.248+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d1cc [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1/ auth{2=1} v3042 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0) n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1) old_inodes=1 (inest sync->mix g=2) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8f176c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v1862>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v3042>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v18372>,<0x10000000201/linux-6.5.11 v2154>,<0x10000000006/t v1255>,<0x10000000005/tmp v1196>,<0x100000001fb/client.0 v1173>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1150>,<0x10000000001/sv_1 v1149>,<0x10000000000/qa v1149>,<0x1/volumes v1106>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.549340+0000 22=22+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:07:17.454 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:07:17 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:07:17.264+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d180 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio/ auth{1=1} v3038 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0) n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1) old_inodes=1 (inest sync->mix g=1) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8ebd2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v1860>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v3038>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v18372>,<0x10000000201/linux-6.5.11 v2154>,<0x10000000006/t v1255>,<0x10000000005/tmp v1196>,<0x100000001fb/client.0 v1173>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1150>,<0x10000000001/sv_1 v1149>,<0x10000000000/qa v1149>,<0x1/volumes v1106>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.414343+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:07:17.454 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:07:17.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:17 smithi073 ceph-mon[36714]: pgmap v913: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 542 KiB/s wr, 137 op/s 2024-07-31T15:07:17.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:07:17.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:07:17.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:17 smithi077 ceph-mon[37327]: pgmap v913: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 542 KiB/s wr, 137 op/s 2024-07-31T15:07:17.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:07:17.557 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/psxface.o 2024-07-31T15:07:17.561 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/autofs/root.o 2024-07-31T15:07:17.687 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma/acpi-dma.o 2024-07-31T15:07:17.826 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/percpu_counter.o 2024-07-31T15:07:17.892 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:07:17 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:07:17.452+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000cde2 [...10,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders/ auth{4=1} v112 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0) n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1) old_inodes=2 (inest sync->mix g=4) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8dc52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v29>,<0x1000000cbec/selinux v646>,<0x10000000202/scripts v8820>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v112>,<0x1000000cbec/selinux v749>,<0x10000000202/scripts v11639>,<0x10000000201/linux-6.5.11 v2154>,<0x10000000006/t v1255>,<0x10000000005/tmp v1196>,<0x100000001fb/client.0 v1173>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1150>,<0x10000000001/sv_1 v1149>,<0x10000000000/qa v1149>,<0x1/volumes v1106>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:58.999116+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:07:18.027 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/rsaddr.o 2024-07-31T15:07:18.076 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/netpoll.o 2024-07-31T15:07:18.091 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/autofs/symlink.o 2024-07-31T15:07:18.116 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/audit_fsnotify.o 2024-07-31T15:07:18.128 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/virtio/virtio_pci_legacy.o 2024-07-31T15:07:18.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:18 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:07:18.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:18.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:18.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:18.401 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/syscall.o 2024-07-31T15:07:18.429 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/fib_rules.o 2024-07-31T15:07:18.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:18 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:07:18.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:18.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:18.542 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/rscalc.o 2024-07-31T15:07:18.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:18 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:07:18.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:18.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:18.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:18.708 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/autofs/waitq.o 2024-07-31T15:07:18.750 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/dma/built-in.a 2024-07-31T15:07:18.925 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/vt/vt_ioctl.o 2024-07-31T15:07:18.927 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/audit_tree.o 2024-07-31T15:07:19.035 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/virtio/virtio_input.o 2024-07-31T15:07:19.045 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/errname.o 2024-07-31T15:07:19.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.146+0000 7f481468d640 1 -- 172.21.15.67:0/2387086545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f480c111970 msgr2=0x7f480c109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:19.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.146+0000 7f481468d640 1 --2- 172.21.15.67:0/2387086545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f480c111970 0x7f480c109df0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009860 tx=0x7f47fc02a380 comp rx=0 tx=0).stop 2024-07-31T15:07:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.153+0000 7f481468d640 1 -- 172.21.15.67:0/2387086545 shutdown_connections 2024-07-31T15:07:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.153+0000 7f481468d640 1 --2- 172.21.15.67:0/2387086545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f480c11c660 0x7f480c11eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.153+0000 7f481468d640 1 --2- 172.21.15.67:0/2387086545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f480c10a410 0x7f480c10a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.153+0000 7f481468d640 1 --2- 172.21.15.67:0/2387086545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f480c111970 0x7f480c109df0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.153+0000 7f481468d640 1 -- 172.21.15.67:0/2387086545 >> 172.21.15.67:0/2387086545 conn(0x7f480c0708f0 msgr2=0x7f480c070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:19.161 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/rscreate.o 2024-07-31T15:07:19.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.158+0000 7f481468d640 1 -- 172.21.15.67:0/2387086545 shutdown_connections 2024-07-31T15:07:19.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.162+0000 7f481468d640 1 -- 172.21.15.67:0/2387086545 wait complete. 2024-07-31T15:07:19.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.163+0000 7f481468d640 1 Processor -- start 2024-07-31T15:07:19.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.163+0000 7f481468d640 1 -- start start 2024-07-31T15:07:19.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.163+0000 7f481468d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f480c10a410 0x7f480c1a8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.163+0000 7f481468d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f480c111970 0x7f480c1a9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.163+0000 7f481468d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f480c11c660 0x7f480c1ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.163+0000 7f481468d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f480c122390 con 0x7f480c10a410 2024-07-31T15:07:19.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.163+0000 7f481468d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f480c122210 con 0x7f480c11c660 2024-07-31T15:07:19.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.163+0000 7f481468d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f480c122510 con 0x7f480c111970 2024-07-31T15:07:19.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.163+0000 7f4812429640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f480c10a410 0x7f480c1a8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:19.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.164+0000 7f4812429640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f480c10a410 0x7f480c1a8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33210/0 (socket says 172.21.15.67:33210) 2024-07-31T15:07:19.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.164+0000 7f4812429640 1 -- 172.21.15.67:0/2976628074 learned_addr learned my addr 172.21.15.67:0/2976628074 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:19.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.164+0000 7f4812c2a640 1 --2- 172.21.15.67:0/2976628074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f480c11c660 0x7f480c1ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:19.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.168+0000 7f4812c2a640 1 -- 172.21.15.67:0/2976628074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f480c111970 msgr2=0x7f480c1a9340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:07:19.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.168+0000 7f4812c2a640 1 --2- 172.21.15.67:0/2976628074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f480c111970 0x7f480c1a9340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.168+0000 7f4812c2a640 1 -- 172.21.15.67:0/2976628074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f480c10a410 msgr2=0x7f480c1a8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:19.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.168+0000 7f4812c2a640 1 --2- 172.21.15.67:0/2976628074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f480c10a410 0x7f480c1a8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.169+0000 7f4812c2a640 1 -- 172.21.15.67:0/2976628074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c1aed10 con 0x7f480c11c660 2024-07-31T15:07:19.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.169+0000 7f4812c2a640 1 --2- 172.21.15.67:0/2976628074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f480c11c660 0x7f480c1ae5e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f480400ef00 tx=0x7f480400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:19.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.171+0000 7f48037fe640 1 -- 172.21.15.67:0/2976628074 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804004b20 con 0x7f480c11c660 2024-07-31T15:07:19.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.171+0000 7f48037fe640 1 -- 172.21.15.67:0/2976628074 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4804003e20 con 0x7f480c11c660 2024-07-31T15:07:19.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.171+0000 7f48037fe640 1 -- 172.21.15.67:0/2976628074 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480400b570 con 0x7f480c11c660 2024-07-31T15:07:19.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.176+0000 7f481468d640 1 -- 172.21.15.67:0/2976628074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480c1af000 con 0x7f480c11c660 2024-07-31T15:07:19.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.176+0000 7f481468d640 1 -- 172.21.15.67:0/2976628074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480c07b050 con 0x7f480c11c660 2024-07-31T15:07:19.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.183+0000 7f481abde740 1 Processor -- start 2024-07-31T15:07:19.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.185+0000 7f48037fe640 1 -- 172.21.15.67:0/2976628074 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f480401d020 con 0x7f480c11c660 2024-07-31T15:07:19.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.187+0000 7f48037fe640 1 --2- 172.21.15.67:0/2976628074 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f47f80b3f50 0x7f47f80b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.188+0000 7f48037fe640 1 -- 172.21.15.67:0/2976628074 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9407+0+0 (secure 0 0 0) 0x7f48040e4230 con 0x7f480c11c660 2024-07-31T15:07:19.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.188+0000 7f481abde740 1 -- start start 2024-07-31T15:07:19.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.189+0000 7f481abde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d79070 0x56307911e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.189+0000 7f481abde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d501b0 0x56307911ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.189+0000 7f481abde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d58390 0x56307911efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.189+0000 7f481abde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563079094b60 con 0x563078d501b0 2024-07-31T15:07:19.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.189+0000 7f481abde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563079084060 con 0x563078d79070 2024-07-31T15:07:19.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.189+0000 7f481abde740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5630790880a0 con 0x563078d58390 2024-07-31T15:07:19.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.191+0000 7f4812429640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d79070 0x56307911e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:19.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.191+0000 7f4812429640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d79070 0x56307911e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44928/0 (socket says 172.21.15.67:44928) 2024-07-31T15:07:19.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.191+0000 7f4812429640 1 -- 172.21.15.67:0/1843908376 learned_addr learned my addr 172.21.15.67:0/1843908376 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.191+0000 7f4812c2a640 1 --2- 172.21.15.67:0/1843908376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d58390 0x56307911efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.191+0000 7f4811c28640 1 --2- 172.21.15.67:0/2976628074 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f47f80b3f50 0x7f47f80b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.191+0000 7f4812429640 1 -- 172.21.15.67:0/1843908376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d58390 msgr2=0x56307911efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.191+0000 7f4812429640 1 --2- 172.21.15.67:0/1843908376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d58390 0x56307911efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.191+0000 7f4812429640 1 -- 172.21.15.67:0/1843908376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d501b0 msgr2=0x56307911ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.191+0000 7f4812429640 1 --2- 172.21.15.67:0/1843908376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d501b0 0x56307911ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.191+0000 7f4812429640 1 -- 172.21.15.67:0/1843908376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307902ab40 con 0x563078d79070 2024-07-31T15:07:19.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.192+0000 7f4812429640 1 --2- 172.21.15.67:0/1843908376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d79070 0x56307911e520 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009980 tx=0x7f47fc009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec79c8c3acf3d7b6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.192+0000 7f4811c28640 1 --2- 172.21.15.67:0/2976628074 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f47f80b3f50 0x7f47f80b6460 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x56307911d1a0 tx=0x7f4808008d90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.196+0000 7f4800ff9640 1 -- 172.21.15.67:0/1843908376 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc037040 con 0x563078d79070 2024-07-31T15:07:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.197+0000 7f4800ff9640 1 -- 172.21.15.67:0/1843908376 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47fc038040 con 0x563078d79070 2024-07-31T15:07:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.197+0000 7f4800ff9640 1 -- 172.21.15.67:0/1843908376 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc005280 con 0x563078d79070 2024-07-31T15:07:19.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.199+0000 7f481abde740 1 -- 172.21.15.67:0/1843908376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d79070 msgr2=0x56307911e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:19.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.199+0000 7f481abde740 1 --2- 172.21.15.67:0/1843908376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d79070 0x56307911e520 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009980 tx=0x7f47fc009490 comp rx=0 tx=0).stop 2024-07-31T15:07:19.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.202+0000 7f481abde740 1 -- 172.21.15.67:0/1843908376 shutdown_connections 2024-07-31T15:07:19.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.202+0000 7f481abde740 1 --2- 172.21.15.67:0/1843908376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d58390 0x56307911efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.202+0000 7f481abde740 1 --2- 172.21.15.67:0/1843908376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d501b0 0x56307911ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.202+0000 7f481abde740 1 --2- 172.21.15.67:0/1843908376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d79070 0x56307911e520 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.202+0000 7f481abde740 1 -- 172.21.15.67:0/1843908376 >> 172.21.15.67:0/1843908376 conn(0x563078d6d460 msgr2=0x56307911c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:19.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.202+0000 7f481abde740 1 -- 172.21.15.67:0/1843908376 shutdown_connections 2024-07-31T15:07:19.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.207+0000 7f481abde740 1 -- 172.21.15.67:0/1843908376 wait complete. 2024-07-31T15:07:19.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.208+0000 7f481abde740 1 -- start start 2024-07-31T15:07:19.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.209+0000 7f481abde740 1 Processor -- start 2024-07-31T15:07:19.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.210+0000 7f481abde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d501b0 0x56307912a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.210+0000 7f481abde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d58390 0x56307912a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.210+0000 7f481abde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d79070 0x563079135f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.210+0000 7f481abde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563078ee7700 con 0x563078d58390 2024-07-31T15:07:19.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.210+0000 7f481abde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563079026990 con 0x563078d501b0 2024-07-31T15:07:19.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.210+0000 7f481abde740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56307912cca0 con 0x563078d79070 2024-07-31T15:07:19.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.210+0000 7f481abde740 10 client.0 _ll_register_callbacks cb 0x5630790414a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:19.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.212+0000 7f4811c28640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d58390 0x56307912a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:19.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.212+0000 7f4811c28640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d58390 0x56307912a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33228/0 (socket says 172.21.15.67:33228) 2024-07-31T15:07:19.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.212+0000 7f4811c28640 1 -- 172.21.15.67:0/4272520198 learned_addr learned my addr 172.21.15.67:0/4272520198 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:19.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.216+0000 7f4812429640 1 --2- 172.21.15.67:0/4272520198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d501b0 0x56307912a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:19.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.217+0000 7f4812429640 1 -- 172.21.15.67:0/4272520198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d79070 msgr2=0x563079135f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:19.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.217+0000 7f4812c2a640 1 --2- 172.21.15.67:0/4272520198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d79070 0x563079135f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:19.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.217+0000 7f4812429640 1 --2- 172.21.15.67:0/4272520198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d79070 0x563079135f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.217+0000 7f4812429640 1 -- 172.21.15.67:0/4272520198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d58390 msgr2=0x56307912a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:19.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.217+0000 7f4812429640 1 --2- 172.21.15.67:0/4272520198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d58390 0x56307912a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.217+0000 7f4812429640 1 -- 172.21.15.67:0/4272520198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563078de4650 con 0x563078d501b0 2024-07-31T15:07:19.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.218+0000 7f4812c2a640 1 --2- 172.21.15.67:0/4272520198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d79070 0x563079135f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:07:19.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.218+0000 7f4812429640 1 --2- 172.21.15.67:0/4272520198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d501b0 0x56307912a1b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009830 tx=0x7f47fc030720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b3a3fac4d6c1b5b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:19.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.220+0000 7f47f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:07:19.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.220+0000 7f47f5ffb640 1 -- 172.21.15.67:0/4272520198 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc04c070 con 0x563078d501b0 2024-07-31T15:07:19.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.220+0000 7f47f5ffb640 1 -- 172.21.15.67:0/4272520198 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9407+0+0 (secure 0 0 0) 0x7f47fc005230 con 0x563078d501b0 2024-07-31T15:07:19.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.221+0000 7f481abde740 1 -- 172.21.15.67:0/4272520198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47fc038040 con 0x563078d501b0 2024-07-31T15:07:19.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.226+0000 7f47f5ffb640 1 -- 172.21.15.67:0/4272520198 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f47fc037040 con 0x563078d501b0 2024-07-31T15:07:19.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.226+0000 7f481abde740 10 client.28499 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:19.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.226+0000 7f481abde740 1 -- 172.21.15.67:0/4272520198 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x563079149d40 con 0x563078d501b0 2024-07-31T15:07:19.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.231+0000 7f47f5ffb640 1 -- 172.21.15.67:0/4272520198 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47fc038040 con 0x563078d501b0 2024-07-31T15:07:19.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.231+0000 7f47f5ffb640 1 -- 172.21.15.67:0/4272520198 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f47fc04f020 con 0x563078d501b0 2024-07-31T15:07:19.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.231+0000 7f481abde740 10 client.28499 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:19.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.231+0000 7f481abde740 10 client.28499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:19.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.231+0000 7f481abde740 20 client.28499 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:19.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.231+0000 7f481abde740 1 --2- 172.21.15.67:0/4272520198 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563078d379f0 0x56307914d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.232+0000 7f481abde740 4 client.28499 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:07:19.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.232+0000 7f481abde740 1 -- 172.21.15.67:0/4272520198 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563079096ab0 con 0x563078d379f0 2024-07-31T15:07:19.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.232+0000 7f4811c28640 1 --2- 172.21.15.67:0/4272520198 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563078d379f0 0x56307914d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:19.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.240+0000 7f4811c28640 1 --2- 172.21.15.67:0/4272520198 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563078d379f0 0x56307914d2b0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be70ea2dd47c0bd5 server_cookie=d4135256b0bbbc2d in_seq=0 out_seq=0 2024-07-31T15:07:19.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.243+0000 7f47f5ffb640 10 client.28499 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:19.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.243+0000 7f47f5ffb640 1 -- 172.21.15.67:0/4272520198 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563079096ab0 con 0x563078d379f0 2024-07-31T15:07:19.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.243+0000 7f47f5ffb640 10 client.28499 handle_command_reply: tid=0 2024-07-31T15:07:19.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.243+0000 7f481abde740 1 client.28499 shutdown 2024-07-31T15:07:19.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.243+0000 7f481abde740 1 -- 172.21.15.67:0/4272520198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d501b0 msgr2=0x56307912a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:19.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.243+0000 7f481abde740 1 --2- 172.21.15.67:0/4272520198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d501b0 0x56307912a1b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009830 tx=0x7f47fc030720 comp rx=0 tx=0).stop 2024-07-31T15:07:19.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.243+0000 7f481abde740 1 -- 172.21.15.67:0/4272520198 shutdown_connections 2024-07-31T15:07:19.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.243+0000 7f481abde740 1 --2- 172.21.15.67:0/4272520198 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x563078d379f0 0x56307914d2b0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.243+0000 7f481abde740 1 --2- 172.21.15.67:0/4272520198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d79070 0x563079135f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.243+0000 7f481abde740 1 --2- 172.21.15.67:0/4272520198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d58390 0x56307912a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.243+0000 7f481abde740 1 --2- 172.21.15.67:0/4272520198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d501b0 0x56307912a1b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.243+0000 7f481abde740 1 -- 172.21.15.67:0/4272520198 >> 172.21.15.67:0/4272520198 conn(0x563078d6d460 msgr2=0x56307911abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:19.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.244+0000 7f47f5ffb640 0 client.28499 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:19.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.244+0000 7f481abde740 1 -- 172.21.15.67:0/4272520198 shutdown_connections 2024-07-31T15:07:19.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.244+0000 7f481abde740 1 -- 172.21.15.67:0/4272520198 wait complete. 2024-07-31T15:07:19.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.246+0000 7f481abde740 20 client.28499 trim_cache size 0 max 16384 2024-07-31T15:07:19.263 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/nlattr.o 2024-07-31T15:07:19.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.295+0000 7f481abde740 1 Processor -- start 2024-07-31T15:07:19.323 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.323 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.323 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.323 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.323 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:19.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:19.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: pgmap v914: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 990 KiB/s wr, 517 op/s 2024-07-31T15:07:19.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:19.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:19.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:19.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:19.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.325 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:19.325 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.325 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:19.325 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.325 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.297+0000 7f481abde740 1 -- start start 2024-07-31T15:07:19.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.297+0000 7f481abde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d501b0 0x563079135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.297+0000 7f481abde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d58390 0x56307911c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.297+0000 7f4812429640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d501b0 0x563079135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.297+0000 7f4812429640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d501b0 0x563079135d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38138/0 (socket says 172.21.15.67:38138) 2024-07-31T15:07:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.297+0000 7f4812429640 1 -- 172.21.15.67:0/3184543489 learned_addr learned my addr 172.21.15.67:0/3184543489 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.297+0000 7f481abde740 1 --2- 172.21.15.67:0/3184543489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d79070 0x5630791237a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.297+0000 7f481abde740 1 -- 172.21.15.67:0/3184543489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f480c07c830 con 0x563078d79070 2024-07-31T15:07:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.297+0000 7f481abde740 1 -- 172.21.15.67:0/3184543489 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f480c07e370 con 0x563078d58390 2024-07-31T15:07:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.297+0000 7f481abde740 1 -- 172.21.15.67:0/3184543489 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f480c07cf00 con 0x563078d501b0 2024-07-31T15:07:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.297+0000 7f4811c28640 1 --2- 172.21.15.67:0/3184543489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d58390 0x56307911c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.298+0000 7f4812429640 1 -- 172.21.15.67:0/3184543489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d58390 msgr2=0x56307911c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.298+0000 7f4812429640 1 --2- 172.21.15.67:0/3184543489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d58390 0x56307911c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.298+0000 7f4812429640 1 -- 172.21.15.67:0/3184543489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d79070 msgr2=0x5630791237a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:07:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.298+0000 7f4812429640 1 --2- 172.21.15.67:0/3184543489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d79070 0x5630791237a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.298+0000 7f4812429640 1 -- 172.21.15.67:0/3184543489 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563079120810 con 0x563078d501b0 2024-07-31T15:07:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.302+0000 7f4812429640 1 --2- 172.21.15.67:0/3184543489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d501b0 0x563079135d80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f47fc030490 tx=0x7f47fc030aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6121bf015880652a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.313+0000 7f47f7fff640 1 -- 172.21.15.67:0/3184543489 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc04c070 con 0x563078d501b0 2024-07-31T15:07:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.313+0000 7f47f7fff640 1 -- 172.21.15.67:0/3184543489 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47fc032d50 con 0x563078d501b0 2024-07-31T15:07:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.314+0000 7f481abde740 1 -- 172.21.15.67:0/3184543489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d501b0 msgr2=0x563079135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.314+0000 7f481abde740 1 --2- 172.21.15.67:0/3184543489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d501b0 0x563079135d80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f47fc030490 tx=0x7f47fc030aa0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.315+0000 7f481abde740 1 -- 172.21.15.67:0/3184543489 shutdown_connections 2024-07-31T15:07:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.315+0000 7f481abde740 1 --2- 172.21.15.67:0/3184543489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d79070 0x5630791237a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.315+0000 7f481abde740 1 --2- 172.21.15.67:0/3184543489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d58390 0x56307911c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.315+0000 7f481abde740 1 --2- 172.21.15.67:0/3184543489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d501b0 0x563079135d80 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f47fc030490 tx=0x7f47fc030aa0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.315+0000 7f481abde740 1 -- 172.21.15.67:0/3184543489 >> 172.21.15.67:0/3184543489 conn(0x563078d6d460 msgr2=0x56307911bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.317+0000 7f481abde740 1 -- 172.21.15.67:0/3184543489 shutdown_connections 2024-07-31T15:07:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.317+0000 7f481abde740 1 -- 172.21.15.67:0/3184543489 wait complete. 2024-07-31T15:07:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.317+0000 7f481abde740 1 -- start start 2024-07-31T15:07:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.317+0000 7f481abde740 1 Processor -- start 2024-07-31T15:07:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.318+0000 7f481abde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d58390 0x563079132810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.318+0000 7f481abde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d79070 0x563079152740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.318+0000 7f481abde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d0cda0 0x563079152e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.318+0000 7f481abde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f480c07dca0 con 0x563078d58390 2024-07-31T15:07:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.318+0000 7f481abde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f480c07d5d0 con 0x563078d0cda0 2024-07-31T15:07:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.318+0000 7f481abde740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f480c10a820 con 0x563078d79070 2024-07-31T15:07:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.318+0000 7f481abde740 10 client.0 _ll_register_callbacks cb 0x7f47fc0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.322+0000 7f4811c28640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d79070 0x563079152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.322+0000 7f4811c28640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d79070 0x563079152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:38154/0 (socket says 172.21.15.67:38154) 2024-07-31T15:07:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.322+0000 7f4811c28640 1 -- 172.21.15.67:0/3684782204 learned_addr learned my addr 172.21.15.67:0/3684782204 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.322+0000 7f4812c2a640 1 --2- 172.21.15.67:0/3684782204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d0cda0 0x563079152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.322+0000 7f4812c2a640 1 -- 172.21.15.67:0/3684782204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d79070 msgr2=0x563079152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.322+0000 7f4812c2a640 1 --2- 172.21.15.67:0/3684782204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d79070 0x563079152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.322+0000 7f4812c2a640 1 -- 172.21.15.67:0/3684782204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d58390 msgr2=0x563079132810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.322+0000 7f4812c2a640 1 --2- 172.21.15.67:0/3684782204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d58390 0x563079132810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.322+0000 7f4812c2a640 1 -- 172.21.15.67:0/3684782204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563079131450 con 0x563078d0cda0 2024-07-31T15:07:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.323+0000 7f4812c2a640 1 --2- 172.21.15.67:0/3684782204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d0cda0 0x563079152e50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f48040e15b0 tx=0x7f48040aa960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a125b3bce002ade server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.326+0000 7f47f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:07:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.326+0000 7f47f5ffb640 1 -- 172.21.15.67:0/3684782204 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480401a050 con 0x563078d0cda0 2024-07-31T15:07:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.326+0000 7f481abde740 1 -- 172.21.15.67:0/3684782204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47fc032d50 con 0x563078d0cda0 2024-07-31T15:07:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.326+0000 7f47f5ffb640 1 -- 172.21.15.67:0/3684782204 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9407+0+0 (secure 0 0 0) 0x7f48040b3690 con 0x563078d0cda0 2024-07-31T15:07:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.328+0000 7f47f5ffb640 1 -- 172.21.15.67:0/3684782204 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f48040e1a40 con 0x563078d0cda0 2024-07-31T15:07:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.335+0000 7f481abde740 10 client.28505 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.335+0000 7f481abde740 1 -- 172.21.15.67:0/3684782204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5630791b5c10 con 0x563078d0cda0 2024-07-31T15:07:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.337+0000 7f47f5ffb640 1 -- 172.21.15.67:0/3684782204 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47fc032d50 con 0x563078d0cda0 2024-07-31T15:07:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.337+0000 7f47f5ffb640 1 -- 172.21.15.67:0/3684782204 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f480400b040 con 0x563078d0cda0 2024-07-31T15:07:19.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.337+0000 7f481abde740 10 client.28505 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:19.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.337+0000 7f481abde740 10 client.28505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:19.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.337+0000 7f481abde740 20 client.28505 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:19.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.337+0000 7f481abde740 1 --2- 172.21.15.67:0/3684782204 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5630791b6390 0x5630791b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:19.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.337+0000 7f481abde740 4 client.28505 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:07:19.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.337+0000 7f481abde740 1 -- 172.21.15.67:0/3684782204 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563079092550 con 0x5630791b6390 2024-07-31T15:07:19.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.340+0000 7f4812429640 1 --2- 172.21.15.67:0/3684782204 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5630791b6390 0x5630791b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:19.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.345+0000 7f4812429640 1 --2- 172.21.15.67:0/3684782204 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5630791b6390 0x5630791b87b0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26d674a277ad899c server_cookie=e64839e30683a2f7 in_seq=0 out_seq=0 2024-07-31T15:07:19.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.346+0000 7f47f5ffb640 10 client.28505 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:19.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.351+0000 7f47f5ffb640 1 -- 172.21.15.67:0/3684782204 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x563079092550 con 0x5630791b6390 2024-07-31T15:07:19.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.351+0000 7f47f5ffb640 10 client.28505 handle_command_reply: tid=0 2024-07-31T15:07:19.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.351+0000 7f481abde740 1 client.28505 shutdown 2024-07-31T15:07:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.355+0000 7f481abde740 1 -- 172.21.15.67:0/3684782204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d0cda0 msgr2=0x563079152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.355+0000 7f481abde740 1 --2- 172.21.15.67:0/3684782204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d0cda0 0x563079152e50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f48040e15b0 tx=0x7f48040aa960 comp rx=0 tx=0).stop 2024-07-31T15:07:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.355+0000 7f481abde740 1 -- 172.21.15.67:0/3684782204 shutdown_connections 2024-07-31T15:07:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.355+0000 7f481abde740 1 --2- 172.21.15.67:0/3684782204 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5630791b6390 0x5630791b87b0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.355+0000 7f481abde740 1 --2- 172.21.15.67:0/3684782204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563078d0cda0 0x563079152e50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.355+0000 7f481abde740 1 --2- 172.21.15.67:0/3684782204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563078d79070 0x563079152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.355+0000 7f481abde740 1 --2- 172.21.15.67:0/3684782204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563078d58390 0x563079132810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.355+0000 7f481abde740 1 -- 172.21.15.67:0/3684782204 >> 172.21.15.67:0/3684782204 conn(0x563078d6d460 msgr2=0x56307911f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.355+0000 7f47f5ffb640 0 client.28505 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:19.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.358+0000 7f481abde740 1 -- 172.21.15.67:0/3684782204 shutdown_connections 2024-07-31T15:07:19.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.358+0000 7f481abde740 1 -- 172.21.15.67:0/3684782204 wait complete. 2024-07-31T15:07:19.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.358+0000 7f481abde740 20 client.28505 trim_cache size 0 max 16384 2024-07-31T15:07:19.361 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:07:19.361 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (13486 inodes in the stack)", 2024-07-31T15:07:19.361 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T15:07:19.361 INFO:teuthology.orchestra.run.smithi067.stdout: "d8f6aa19-03ce-4e8d-ba33-a8e8f37936e1": { 2024-07-31T15:07:19.361 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T15:07:19.361 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "d8f6aa19-03ce-4e8d-ba33-a8e8f37936e1", 2024-07-31T15:07:19.362 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T15:07:19.362 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:07:19.362 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:07:19.362 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:07:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.359+0000 7f481468d640 1 -- 172.21.15.67:0/2976628074 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f47f80b3f50 msgr2=0x7f47f80b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.359+0000 7f481468d640 1 --2- 172.21.15.67:0/2976628074 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f47f80b3f50 0x7f47f80b6460 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x56307911d1a0 tx=0x7f4808008d90 comp rx=0 tx=0).stop 2024-07-31T15:07:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.359+0000 7f481468d640 1 -- 172.21.15.67:0/2976628074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f480c11c660 msgr2=0x7f480c1ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.359+0000 7f481468d640 1 --2- 172.21.15.67:0/2976628074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f480c11c660 0x7f480c1ae5e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f480400ef00 tx=0x7f480400c740 comp rx=0 tx=0).stop 2024-07-31T15:07:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.359+0000 7f481468d640 1 -- 172.21.15.67:0/2976628074 shutdown_connections 2024-07-31T15:07:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.359+0000 7f481468d640 1 --2- 172.21.15.67:0/2976628074 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f47f80b3f50 0x7f47f80b6460 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.359+0000 7f481468d640 1 --2- 172.21.15.67:0/2976628074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f480c11c660 0x7f480c1ae5e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.359+0000 7f481468d640 1 --2- 172.21.15.67:0/2976628074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f480c111970 0x7f480c1a9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.359+0000 7f481468d640 1 --2- 172.21.15.67:0/2976628074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f480c10a410 0x7f480c1a8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:19.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.359+0000 7f481468d640 1 -- 172.21.15.67:0/2976628074 >> 172.21.15.67:0/2976628074 conn(0x7f480c0708f0 msgr2=0x7f480c11dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:19.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.359+0000 7f481468d640 1 -- 172.21.15.67:0/2976628074 shutdown_connections 2024-07-31T15:07:19.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:19.360+0000 7f481468d640 1 -- 172.21.15.67:0/2976628074 wait complete. 2024-07-31T15:07:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: pgmap v914: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 990 KiB/s wr, 517 op/s 2024-07-31T15:07:19.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:07:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: pgmap v914: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 990 KiB/s wr, 517 op/s 2024-07-31T15:07:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.074 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/rsdumpinfo.o 2024-07-31T15:07:20.075 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/autofs/expire.o 2024-07-31T15:07:20.079 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/cpu_rmap.o 2024-07-31T15:07:20.283 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/kprobes.o 2024-07-31T15:07:20.287 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/virtio/virtio_dma_buf.o 2024-07-31T15:07:20.386 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/net-traces.o 2024-07-31T15:07:20.394 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/vt/vc_screen.o 2024-07-31T15:07:20.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:20.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:20.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:20.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:20.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:20.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:20.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:20.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:20 smithi077 ceph-mon[37327]: osdmap e118: 12 total, 12 up, 12 in 2024-07-31T15:07:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:20 smithi067 ceph-mon[32750]: osdmap e118: 12 total, 12 up, 12 in 2024-07-31T15:07:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:20 smithi073 ceph-mon[36714]: osdmap e118: 12 total, 12 up, 12 in 2024-07-31T15:07:20.648 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/vt/selection.o 2024-07-31T15:07:20.653 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/rsinfo.o 2024-07-31T15:07:20.658 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/dynamic_queue_limits.o 2024-07-31T15:07:20.947 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/autofs/dev-ioctl.o 2024-07-31T15:07:20.958 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/virtio/built-in.a 2024-07-31T15:07:20.976 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/glob.o 2024-07-31T15:07:20.995 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/selftests.o 2024-07-31T15:07:21.236 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/rsio.o 2024-07-31T15:07:21.415 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/ptp_classifier.o 2024-07-31T15:07:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: pgmap v915: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 841 op/s 2024-07-31T15:07:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.557 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/strncpy_from_user.o 2024-07-31T15:07:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: pgmap v915: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 841 op/s 2024-07-31T15:07:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.678 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/rsirq.o 2024-07-31T15:07:21.688 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/vt/keyboard.o 2024-07-31T15:07:21.706 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/autofs/built-in.a 2024-07-31T15:07:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: pgmap v915: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 841 op/s 2024-07-31T15:07:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:21.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:21.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:21.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:21.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:21.754 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/vfs_super.o 2024-07-31T15:07:21.886 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/seccomp.o 2024-07-31T15:07:22.075 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/rslist.o 2024-07-31T15:07:22.116 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/strnlen_user.o 2024-07-31T15:07:22.405 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/relay.o 2024-07-31T15:07:22.426 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/char/hw_random/core.o 2024-07-31T15:07:22.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:22.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.622 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/rsmemory.o 2024-07-31T15:07:22.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.703 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/net_utils.o 2024-07-31T15:07:22.708 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/vfs_inode.o 2024-07-31T15:07:22.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:22.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:22.907 INFO:tasks.cephfs.filesystem:scrub status for tag:d8f6aa19-03ce-4e8d-ba33-a8e8f37936e1 - {'path': '/', 'tag': 'd8f6aa19-03ce-4e8d-ba33-a8e8f37936e1', 'options': 'recursive,force'} 2024-07-31T15:07:22.997 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/rsmisc.o 2024-07-31T15:07:23.238 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/vt/consolemap.o 2024-07-31T15:07:23.316 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/char/hw_random/via-rng.o 2024-07-31T15:07:23.440 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/sg_pool.o 2024-07-31T15:07:23.526 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/utsname_sysctl.o 2024-07-31T15:07:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: pgmap v917: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 1.96k op/s 2024-07-31T15:07:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:23.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:23.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.594 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/hostfs/built-in.a 2024-07-31T15:07:23.618 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/netprio_cgroup.o 2024-07-31T15:07:23.628 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/rsserial.o 2024-07-31T15:07:23.636 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC drivers/tty/vt/conmakehash 2024-07-31T15:07:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: pgmap v917: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 1.96k op/s 2024-07-31T15:07:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:23.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: pgmap v917: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 1.96k op/s 2024-07-31T15:07:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:23.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:23.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:23.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:23.824 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/char/hw_random/built-in.a 2024-07-31T15:07:23.832 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/vfs_inode_dotl.o 2024-07-31T15:07:23.868 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/delayacct.o 2024-07-31T15:07:23.878 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/debugfs/inode.o 2024-07-31T15:07:23.895 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/char/agp/backend.o 2024-07-31T15:07:24.068 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/rsutils.o 2024-07-31T15:07:24.077 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/stackdepot.o 2024-07-31T15:07:24.358 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/vt/vt.o 2024-07-31T15:07:24.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:24.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.562 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/taskstats.o 2024-07-31T15:07:24.618 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/rsxface.o 2024-07-31T15:07:24.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:24.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.701 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/netclassid_cgroup.o 2024-07-31T15:07:24.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:24.747 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/char/agp/generic.o 2024-07-31T15:07:24.851 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/asn1_decoder.o 2024-07-31T15:07:24.870 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/debugfs/file.o 2024-07-31T15:07:24.929 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/vfs_addr.o 2024-07-31T15:07:25.111 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/tbdata.o 2024-07-31T15:07:25.439 INFO:tasks.workunit.client.0.smithi067.stdout: GEN lib/oid_registry_data.c 2024-07-31T15:07:25.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: pgmap v918: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 2.81k op/s 2024-07-31T15:07:25.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:25.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.559 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/ucs2_string.o 2024-07-31T15:07:25.615 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/tsacct.o 2024-07-31T15:07:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: pgmap v918: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 2.81k op/s 2024-07-31T15:07:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:25.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.673 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/tbfadt.o 2024-07-31T15:07:25.713 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/dst_cache.o 2024-07-31T15:07:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: pgmap v918: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 2.81k op/s 2024-07-31T15:07:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:25.750 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/vfs_file.o 2024-07-31T15:07:25.851 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/char/agp/isoch.o 2024-07-31T15:07:25.867 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/debugfs/built-in.a 2024-07-31T15:07:25.913 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/tbfind.o 2024-07-31T15:07:25.975 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/sbitmap.o 2024-07-31T15:07:26.252 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/group_cpus.o 2024-07-31T15:07:26.333 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/tracepoint.o 2024-07-31T15:07:26.385 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/tbinstal.o 2024-07-31T15:07:26.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:26.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.609 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/vfs_dir.o 2024-07-31T15:07:26.638 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/char/agp/amd64-agp.o 2024-07-31T15:07:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:26.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.729 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/gro_cells.o 2024-07-31T15:07:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:26.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:26.743 INFO:tasks.workunit.client.0.smithi067.stdout: COPY drivers/tty/vt/defkeymap.c 2024-07-31T15:07:26.818 INFO:tasks.workunit.client.0.smithi067.stdout: CONMK drivers/tty/vt/consolemap_deftbl.c 2024-07-31T15:07:26.862 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/vt/defkeymap.o 2024-07-31T15:07:26.919 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/char/agp/intel-agp.o 2024-07-31T15:07:26.935 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/tbprint.o 2024-07-31T15:07:27.059 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/vt/consolemap_deftbl.o 2024-07-31T15:07:27.069 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq_work.o 2024-07-31T15:07:27.104 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/tty/vt/built-in.a 2024-07-31T15:07:27.110 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/lib.a 2024-07-31T15:07:27.243 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/hvc/hvc_console.o 2024-07-31T15:07:27.261 INFO:tasks.workunit.client.0.smithi067.stdout: GEN lib/crc32table.h 2024-07-31T15:07:27.277 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/oid_registry.o 2024-07-31T15:07:27.290 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/vfs_dentry.o 2024-07-31T15:07:27.430 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/tbutils.o 2024-07-31T15:07:27.460 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/tbxface.o 2024-07-31T15:07:27.620 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/core/failover.o 2024-07-31T15:07:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: pgmap v919: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.2 MiB/s wr, 3.02k op/s 2024-07-31T15:07:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: pgmap v919: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.2 MiB/s wr, 3.02k op/s 2024-07-31T15:07:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:27.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.743 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/static_call.o 2024-07-31T15:07:27.760 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crc32.o 2024-07-31T15:07:27.764 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/char/agp/intel-gtt.o 2024-07-31T15:07:27.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: pgmap v919: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.2 MiB/s wr, 3.02k op/s 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:27.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:27.934 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/numa/srat.o 2024-07-31T15:07:27.954 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/v9fs.o 2024-07-31T15:07:27.990 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/tbxfload.o 2024-07-31T15:07:28.181 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/static_call_inline.o 2024-07-31T15:07:28.185 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/tty/hvc/built-in.a 2024-07-31T15:07:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.318+0000 7fb5d2f87640 1 -- 172.21.15.67:0/1701761884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc1116d0 msgr2=0x7fb5cc109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.318+0000 7fb5d2f87640 1 --2- 172.21.15.67:0/1701761884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc1116d0 0x7fb5cc109df0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009ca0 tx=0x7fb5c402a8f0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.320+0000 7fb5d2f87640 1 -- 172.21.15.67:0/1701761884 shutdown_connections 2024-07-31T15:07:28.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.320+0000 7fb5d2f87640 1 --2- 172.21.15.67:0/1701761884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5cc11c5d0 0x7fb5cc11e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.320+0000 7fb5d2f87640 1 --2- 172.21.15.67:0/1701761884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5cc10a330 0x7fb5cc10a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.320+0000 7fb5d2f87640 1 --2- 172.21.15.67:0/1701761884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc1116d0 0x7fb5cc109df0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.320+0000 7fb5d2f87640 1 -- 172.21.15.67:0/1701761884 >> 172.21.15.67:0/1701761884 conn(0x7fb5cc06e350 msgr2=0x7fb5cc06e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:28.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.321+0000 7fb5d2f87640 1 -- 172.21.15.67:0/1701761884 shutdown_connections 2024-07-31T15:07:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.321+0000 7fb5d2f87640 1 -- 172.21.15.67:0/1701761884 wait complete. 2024-07-31T15:07:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.322+0000 7fb5d2f87640 1 Processor -- start 2024-07-31T15:07:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.323+0000 7fb5d2f87640 1 -- start start 2024-07-31T15:07:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.323+0000 7fb5d2f87640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5cc10a330 0x7fb5cc11be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.323+0000 7fb5d2f87640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc1116d0 0x7fb5cc115120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.323+0000 7fb5d2f87640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5cc11c5d0 0x7fb5cc1156c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.323+0000 7fb5d2f87640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc1220b0 con 0x7fb5cc10a330 2024-07-31T15:07:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.323+0000 7fb5d2f87640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc121f30 con 0x7fb5cc11c5d0 2024-07-31T15:07:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.323+0000 7fb5d2f87640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc122230 con 0x7fb5cc1116d0 2024-07-31T15:07:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.324+0000 7fb5cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc1116d0 0x7fb5cc115120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.324+0000 7fb5d1524640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5cc11c5d0 0x7fb5cc1156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.324+0000 7fb5cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc1116d0 0x7fb5cc115120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:53446/0 (socket says 172.21.15.67:53446) 2024-07-31T15:07:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.324+0000 7fb5cbfff640 1 -- 172.21.15.67:0/317475650 learned_addr learned my addr 172.21.15.67:0/317475650 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.324+0000 7fb5d1524640 1 -- 172.21.15.67:0/317475650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc1116d0 msgr2=0x7fb5cc115120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.324+0000 7fb5d1524640 1 --2- 172.21.15.67:0/317475650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc1116d0 0x7fb5cc115120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.324+0000 7fb5d1524640 1 -- 172.21.15.67:0/317475650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5cc10a330 msgr2=0x7fb5cc11be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.324+0000 7fb5d1524640 1 --2- 172.21.15.67:0/317475650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5cc10a330 0x7fb5cc11be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.324+0000 7fb5d1524640 1 -- 172.21.15.67:0/317475650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5cc115ee0 con 0x7fb5cc11c5d0 2024-07-31T15:07:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.324+0000 7fb5d1524640 1 --2- 172.21.15.67:0/317475650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5cc11c5d0 0x7fb5cc1156c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000d680 tx=0x7fb5c000db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.328+0000 7fb5c9ffb640 1 -- 172.21.15.67:0/317475650 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0017070 con 0x7fb5cc11c5d0 2024-07-31T15:07:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.328+0000 7fb5c9ffb640 1 -- 172.21.15.67:0/317475650 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c0009820 con 0x7fb5cc11c5d0 2024-07-31T15:07:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.328+0000 7fb5c9ffb640 1 -- 172.21.15.67:0/317475650 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0004ce0 con 0x7fb5cc11c5d0 2024-07-31T15:07:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.328+0000 7fb5d2f87640 1 -- 172.21.15.67:0/317475650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5cc11ab80 con 0x7fb5cc11c5d0 2024-07-31T15:07:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.328+0000 7fb5d2f87640 1 -- 172.21.15.67:0/317475650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5cc11b080 con 0x7fb5cc11c5d0 2024-07-31T15:07:28.334 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/8250/8250_core.o 2024-07-31T15:07:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.337+0000 7fb5c9ffb640 1 -- 172.21.15.67:0/317475650 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb5c0005220 con 0x7fb5cc11c5d0 2024-07-31T15:07:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.337+0000 7fb5d9e81740 1 Processor -- start 2024-07-31T15:07:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.338+0000 7fb5d9e81740 1 -- start start 2024-07-31T15:07:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.338+0000 7fb5d9e81740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d43c1b0 0x56082d7deea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.338+0000 7fb5d9e81740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d4132f0 0x56082d7df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.338+0000 7fb5d9e81740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d41b4d0 0x56082d7df920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.338+0000 7fb5d9e81740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56082d7543b0 con 0x56082d4132f0 2024-07-31T15:07:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.338+0000 7fb5d9e81740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56082d7438b0 con 0x56082d41b4d0 2024-07-31T15:07:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.338+0000 7fb5d9e81740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56082d7478f0 con 0x56082d43c1b0 2024-07-31T15:07:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.338+0000 7fb5cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d43c1b0 0x56082d7deea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.338+0000 7fb5cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d43c1b0 0x56082d7deea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:53460/0 (socket says 172.21.15.67:53460) 2024-07-31T15:07:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.338+0000 7fb5cbfff640 1 -- 172.21.15.67:0/2561352887 learned_addr learned my addr 172.21.15.67:0/2561352887 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.339+0000 7fb5cbfff640 1 --2- 172.21.15.67:0/2561352887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d41b4d0 0x56082d7df920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.339+0000 7fb5cbfff640 1 -- 172.21.15.67:0/2561352887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d41b4d0 msgr2=0x56082d7df920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.339+0000 7fb5cbfff640 1 --2- 172.21.15.67:0/2561352887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d41b4d0 0x56082d7df920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.339+0000 7fb5cbfff640 1 -- 172.21.15.67:0/2561352887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d4132f0 msgr2=0x56082d7df3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:07:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.339+0000 7fb5cbfff640 1 --2- 172.21.15.67:0/2561352887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d4132f0 0x56082d7df3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.339+0000 7fb5cbfff640 1 -- 172.21.15.67:0/2561352887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082d6ea9a0 con 0x56082d43c1b0 2024-07-31T15:07:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.339+0000 7fb5c9ffb640 1 --2- 172.21.15.67:0/317475650 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5b40b3ed0 0x7fb5b40b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.339+0000 7fb5cbfff640 1 --2- 172.21.15.67:0/2561352887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d43c1b0 0x56082d7deea0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc012890 tx=0x7fb5bc00b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9beba9c62a02125c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.339+0000 7fb5c9ffb640 1 -- 172.21.15.67:0/317475650 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9379+0+0 (secure 0 0 0) 0x7fb5c00e3940 con 0x7fb5cc11c5d0 2024-07-31T15:07:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.341+0000 7fb5d0d23640 1 --2- 172.21.15.67:0/317475650 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5b40b3ed0 0x7fb5b40b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.343+0000 7fb5d0d23640 1 --2- 172.21.15.67:0/317475650 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5b40b3ed0 0x7fb5b40b63e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40094e0 tx=0x7fb5c4009470 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.345+0000 7fb5baffd640 1 -- 172.21.15.67:0/2561352887 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc012e10 con 0x56082d43c1b0 2024-07-31T15:07:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.345+0000 7fb5baffd640 1 -- 172.21.15.67:0/2561352887 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5bc010c20 con 0x56082d43c1b0 2024-07-31T15:07:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.346+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2561352887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d43c1b0 msgr2=0x56082d7deea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.346+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2561352887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d43c1b0 0x56082d7deea0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc012890 tx=0x7fb5bc00b860 comp rx=0 tx=0).stop 2024-07-31T15:07:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.347+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2561352887 shutdown_connections 2024-07-31T15:07:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.347+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2561352887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d41b4d0 0x56082d7df920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.347+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2561352887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d4132f0 0x56082d7df3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.347+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2561352887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d43c1b0 0x56082d7deea0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.347+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2561352887 >> 172.21.15.67:0/2561352887 conn(0x56082d430300 msgr2=0x56082d7dcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.347+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2561352887 shutdown_connections 2024-07-31T15:07:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.347+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2561352887 wait complete. 2024-07-31T15:07:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.348+0000 7fb5d9e81740 1 -- start start 2024-07-31T15:07:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.348+0000 7fb5d9e81740 1 Processor -- start 2024-07-31T15:07:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.349+0000 7fb5d9e81740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d4132f0 0x56082d7ea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.349+0000 7fb5d9e81740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d41b4d0 0x56082d7eae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.349+0000 7fb5d9e81740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d43c1b0 0x56082d7f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.349+0000 7fb5d9e81740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56082d67dab0 con 0x56082d41b4d0 2024-07-31T15:07:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.349+0000 7fb5d9e81740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56082d6cbcf0 con 0x56082d43c1b0 2024-07-31T15:07:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.349+0000 7fb5d9e81740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56082d7ed670 con 0x56082d4132f0 2024-07-31T15:07:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.349+0000 7fb5d9e81740 10 client.0 _ll_register_callbacks cb 0x56082d73d9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.350+0000 7fb5d0d23640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d4132f0 0x56082d7ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.350+0000 7fb5d0d23640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d4132f0 0x56082d7ea950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:53474/0 (socket says 172.21.15.67:53474) 2024-07-31T15:07:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.350+0000 7fb5d0d23640 1 -- 172.21.15.67:0/2416251946 learned_addr learned my addr 172.21.15.67:0/2416251946 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.350+0000 7fb5cbfff640 1 --2- 172.21.15.67:0/2416251946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d41b4d0 0x56082d7eae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.350+0000 7fb5d0d23640 1 -- 172.21.15.67:0/2416251946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d43c1b0 msgr2=0x56082d7f6a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:07:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.350+0000 7fb5d0d23640 1 --2- 172.21.15.67:0/2416251946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d43c1b0 0x56082d7f6a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.350+0000 7fb5d0d23640 1 -- 172.21.15.67:0/2416251946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d41b4d0 msgr2=0x56082d7eae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.350+0000 7fb5d0d23640 1 --2- 172.21.15.67:0/2416251946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d41b4d0 0x56082d7eae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.350+0000 7fb5d0d23640 1 -- 172.21.15.67:0/2416251946 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082d4a7790 con 0x56082d4132f0 2024-07-31T15:07:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.350+0000 7fb5d0d23640 1 --2- 172.21.15.67:0/2416251946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d4132f0 0x56082d7ea950 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4035000 tx=0x7fb5c4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=771cc6c5280a4804 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.351+0000 7fb5affff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:07:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.351+0000 7fb5affff640 1 -- 172.21.15.67:0/2416251946 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40044c0 con 0x56082d4132f0 2024-07-31T15:07:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.351+0000 7fb5affff640 1 -- 172.21.15.67:0/2416251946 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9379+0+0 (secure 0 0 0) 0x7fb5c4004660 con 0x56082d4132f0 2024-07-31T15:07:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.351+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2416251946 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5bc010c20 con 0x56082d4132f0 2024-07-31T15:07:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.352+0000 7fb5affff640 1 -- 172.21.15.67:0/2416251946 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fb5c4058050 con 0x56082d4132f0 2024-07-31T15:07:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.352+0000 7fb5d9e81740 10 client.28417 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.352+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2416251946 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x56082d520620 con 0x56082d4132f0 2024-07-31T15:07:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.352+0000 7fb5affff640 1 -- 172.21.15.67:0/2416251946 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5bc010c20 con 0x56082d4132f0 2024-07-31T15:07:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.352+0000 7fb5affff640 1 -- 172.21.15.67:0/2416251946 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fb5c4030610 con 0x56082d4132f0 2024-07-31T15:07:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.352+0000 7fb5d9e81740 10 client.28417 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.352+0000 7fb5d9e81740 10 client.28417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.352+0000 7fb5d9e81740 20 client.28417 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.352+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2416251946 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56082d3fad80 0x56082d7fcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.352+0000 7fb5d9e81740 4 client.28417 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:07:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.352+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2416251946 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56082d756300 con 0x56082d3fad80 2024-07-31T15:07:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.352+0000 7fb5cbfff640 1 --2- 172.21.15.67:0/2416251946 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56082d3fad80 0x56082d7fcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.353+0000 7fb5affff640 10 client.28417 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.353+0000 7fb5cbfff640 1 --2- 172.21.15.67:0/2416251946 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56082d3fad80 0x56082d7fcf20 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=560d210997b4c81c server_cookie=e442ec19ba2d0825 in_seq=0 out_seq=0 2024-07-31T15:07:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.356+0000 7fb5affff640 1 -- 172.21.15.67:0/2416251946 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56082d756300 con 0x56082d3fad80 2024-07-31T15:07:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.356+0000 7fb5affff640 10 client.28417 handle_command_reply: tid=0 2024-07-31T15:07:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.357+0000 7fb5d9e81740 1 client.28417 shutdown 2024-07-31T15:07:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.357+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2416251946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d4132f0 msgr2=0x56082d7ea950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.357+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2416251946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d4132f0 0x56082d7ea950 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4035000 tx=0x7fb5c4005fd0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.357+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2416251946 shutdown_connections 2024-07-31T15:07:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.357+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2416251946 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56082d3fad80 0x56082d7fcf20 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.357+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2416251946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d43c1b0 0x56082d7f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.357+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2416251946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d41b4d0 0x56082d7eae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.357+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2416251946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d4132f0 0x56082d7ea950 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.357+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2416251946 >> 172.21.15.67:0/2416251946 conn(0x56082d430300 msgr2=0x56082d7dfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.357+0000 7fb5affff640 0 client.28417 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.357+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2416251946 shutdown_connections 2024-07-31T15:07:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.357+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2416251946 wait complete. 2024-07-31T15:07:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.357+0000 7fb5d9e81740 20 client.28417 trim_cache size 0 max 16384 2024-07-31T15:07:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.394+0000 7fb5d9e81740 1 Processor -- start 2024-07-31T15:07:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.394+0000 7fb5d9e81740 1 -- start start 2024-07-31T15:07:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.394+0000 7fb5d9e81740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d4132f0 0x56082d7e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.399+0000 7fb5d9e81740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d41b4d0 0x56082d7dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.399+0000 7fb5d9e81740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d43c1b0 0x56082d7f6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.399+0000 7fb5d9e81740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b000ff40 con 0x56082d4132f0 2024-07-31T15:07:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.399+0000 7fb5d9e81740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b0011a80 con 0x56082d43c1b0 2024-07-31T15:07:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.399+0000 7fb5d9e81740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b0010610 con 0x56082d41b4d0 2024-07-31T15:07:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.399+0000 7fb5cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d41b4d0 0x56082d7dccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.399+0000 7fb5d0d23640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d4132f0 0x56082d7e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.400+0000 7fb5d0d23640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d4132f0 0x56082d7e4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46176/0 (socket says 172.21.15.67:46176) 2024-07-31T15:07:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.400+0000 7fb5d0d23640 1 -- 172.21.15.67:0/1348778851 learned_addr learned my addr 172.21.15.67:0/1348778851 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.400+0000 7fb5d0d23640 1 -- 172.21.15.67:0/1348778851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d41b4d0 msgr2=0x56082d7dccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.402+0000 7fb5d0d23640 1 --2- 172.21.15.67:0/1348778851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d41b4d0 0x56082d7dccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.403+0000 7fb5d0d23640 1 -- 172.21.15.67:0/1348778851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d43c1b0 msgr2=0x56082d7f6830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:07:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.403+0000 7fb5d0d23640 1 --2- 172.21.15.67:0/1348778851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d43c1b0 0x56082d7f6830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.403+0000 7fb5d0d23640 1 -- 172.21.15.67:0/1348778851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082d3fafb0 con 0x56082d4132f0 2024-07-31T15:07:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.403+0000 7fb5d0d23640 1 --2- 172.21.15.67:0/1348778851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d4132f0 0x56082d7e4f70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009850 tx=0x7fb5c4032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434ebac4be55d5f0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.404+0000 7fb5cbfff640 1 --2- 172.21.15.67:0/1348778851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d41b4d0 0x56082d7dccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:07:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.408+0000 7fb5ba7fc640 1 -- 172.21.15.67:0/1348778851 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4030c20 con 0x56082d4132f0 2024-07-31T15:07:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.408+0000 7fb5ba7fc640 1 -- 172.21.15.67:0/1348778851 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c4036050 con 0x56082d4132f0 2024-07-31T15:07:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.408+0000 7fb5ba7fc640 1 -- 172.21.15.67:0/1348778851 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c402f9b0 con 0x56082d4132f0 2024-07-31T15:07:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.410+0000 7fb5d9e81740 1 -- 172.21.15.67:0/1348778851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d4132f0 msgr2=0x56082d7e4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.410+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/1348778851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d4132f0 0x56082d7e4f70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009850 tx=0x7fb5c4032f80 comp rx=0 tx=0).stop 2024-07-31T15:07:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.411+0000 7fb5d9e81740 1 -- 172.21.15.67:0/1348778851 shutdown_connections 2024-07-31T15:07:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.411+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/1348778851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d43c1b0 0x56082d7f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.411+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/1348778851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d41b4d0 0x56082d7dccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.411+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/1348778851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d4132f0 0x56082d7e4f70 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.411+0000 7fb5d9e81740 1 -- 172.21.15.67:0/1348778851 >> 172.21.15.67:0/1348778851 conn(0x56082d430300 msgr2=0x56082d7dc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.411+0000 7fb5d9e81740 1 -- 172.21.15.67:0/1348778851 shutdown_connections 2024-07-31T15:07:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.411+0000 7fb5d9e81740 1 -- 172.21.15.67:0/1348778851 wait complete. 2024-07-31T15:07:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.412+0000 7fb5d9e81740 1 -- start start 2024-07-31T15:07:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.412+0000 7fb5d9e81740 1 Processor -- start 2024-07-31T15:07:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.413+0000 7fb5d9e81740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d4132f0 0x56082d7f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.413+0000 7fb5d9e81740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d41b4d0 0x56082d7f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.413+0000 7fb5d9e81740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d43c1b0 0x56082d7ef970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.414+0000 7fb5cbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d41b4d0 0x56082d7f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.414+0000 7fb5cbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d41b4d0 0x56082d7f30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46188/0 (socket says 172.21.15.67:46188) 2024-07-31T15:07:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.414+0000 7fb5cbfff640 1 -- 172.21.15.67:0/2428001691 learned_addr learned my addr 172.21.15.67:0/2428001691 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.414+0000 7fb5d0d23640 1 --2- 172.21.15.67:0/2428001691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d4132f0 0x56082d7f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.414+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2428001691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b00113b0 con 0x56082d41b4d0 2024-07-31T15:07:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.414+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2428001691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b0010ce0 con 0x56082d43c1b0 2024-07-31T15:07:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.414+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2428001691 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b000df40 con 0x56082d4132f0 2024-07-31T15:07:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.415+0000 7fb5d1524640 1 --2- 172.21.15.67:0/2428001691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d43c1b0 0x56082d7ef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.415+0000 7fb5cbfff640 1 -- 172.21.15.67:0/2428001691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d4132f0 msgr2=0x56082d7f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.415+0000 7fb5cbfff640 1 --2- 172.21.15.67:0/2428001691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d4132f0 0x56082d7f2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.415+0000 7fb5cbfff640 1 -- 172.21.15.67:0/2428001691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d43c1b0 msgr2=0x56082d7ef970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.415+0000 7fb5cbfff640 1 --2- 172.21.15.67:0/2428001691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d43c1b0 0x56082d7ef970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.415+0000 7fb5cbfff640 1 -- 172.21.15.67:0/2428001691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082d7f00b0 con 0x56082d41b4d0 2024-07-31T15:07:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.416+0000 7fb5cbfff640 1 --2- 172.21.15.67:0/2428001691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d41b4d0 0x56082d7f30f0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00b4f0 tx=0x7fb5bc01e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838216e49553ddfd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.416+0000 7fb5d9e81740 10 client.0 _ll_register_callbacks cb 0x7fb5c4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.416+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2428001691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5c4036050 con 0x56082d41b4d0 2024-07-31T15:07:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.418+0000 7fb5affff640 10 client.18993 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:07:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.420+0000 7fb5affff640 1 -- 172.21.15.67:0/2428001691 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc00bca0 con 0x56082d41b4d0 2024-07-31T15:07:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.420+0000 7fb5affff640 1 -- 172.21.15.67:0/2428001691 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9379+0+0 (secure 0 0 0) 0x7fb5bc024040 con 0x56082d41b4d0 2024-07-31T15:07:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.420+0000 7fb5affff640 1 -- 172.21.15.67:0/2428001691 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fb5bc0124f0 con 0x56082d41b4d0 2024-07-31T15:07:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.421+0000 7fb5d9e81740 10 client.18993 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.421+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2428001691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({fsmap=65}) -- 0x56082d863f90 con 0x56082d41b4d0 2024-07-31T15:07:28.424 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/built-in.a 2024-07-31T15:07:28.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.422+0000 7fb5affff640 1 -- 172.21.15.67:0/2428001691 <== mon.0 v2:172.21.15.67:3300/0 4 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fb5bc030030 con 0x56082d41b4d0 2024-07-31T15:07:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.422+0000 7fb5d9e81740 10 client.18993 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.422+0000 7fb5d9e81740 10 client.18993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.422+0000 7fb5d9e81740 20 client.18993 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.422+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2428001691 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56082d3d01c0 0x56082d8663e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.422+0000 7fb5d9e81740 4 client.18993 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:07:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.422+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2428001691 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56082d751da0 con 0x56082d3d01c0 2024-07-31T15:07:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.423+0000 7fb5d0d23640 1 --2- 172.21.15.67:0/2428001691 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56082d3d01c0 0x56082d8663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.423+0000 7fb5d0d23640 1 --2- 172.21.15.67:0/2428001691 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56082d3d01c0 0x56082d8663e0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45f2c7a8f45a8033 server_cookie=d8c8788b3fa5a9ae in_seq=0 out_seq=0 2024-07-31T15:07:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:28.423+0000 7fb5affff640 10 client.18993 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:28.493 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/tbxfroot.o 2024-07-31T15:07:28.581 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/core/built-in.a 2024-07-31T15:07:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:28.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:28.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:28.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.781 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/8250/8250_pnp.o 2024-07-31T15:07:28.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:28.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:28.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:28.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:28.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:28.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:28.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:28.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:28.838 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/fid.o 2024-07-31T15:07:28.901 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/crash_dump.o 2024-07-31T15:07:28.930 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utaddress.o 2024-07-31T15:07:28.972 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/char/agp/built-in.a 2024-07-31T15:07:28.979 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/802/built-in.a 2024-07-31T15:07:29.022 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/acpi/numa/built-in.a 2024-07-31T15:07:29.055 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sched/sch_generic.o 2024-07-31T15:07:29.090 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/char/mem.o 2024-07-31T15:07:29.104 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/8250/8250_port.o 2024-07-31T15:07:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.475+0000 7fb5affff640 1 -- 172.21.15.67:0/2428001691 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56082d751da0 con 0x56082d3d01c0 2024-07-31T15:07:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.475+0000 7fb5affff640 10 client.18993 handle_command_reply: tid=0 2024-07-31T15:07:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.475+0000 7fb5d9e81740 1 client.18993 shutdown 2024-07-31T15:07:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.475+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2428001691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d41b4d0 msgr2=0x56082d7f30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.475+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2428001691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d41b4d0 0x56082d7f30f0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00b4f0 tx=0x7fb5bc01e700 comp rx=0 tx=0).stop 2024-07-31T15:07:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.476+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2428001691 shutdown_connections 2024-07-31T15:07:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.476+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2428001691 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56082d3d01c0 0x56082d8663e0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.476+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2428001691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56082d43c1b0 0x56082d7ef970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.476+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2428001691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56082d41b4d0 0x56082d7f30f0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.476+0000 7fb5d9e81740 1 --2- 172.21.15.67:0/2428001691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56082d4132f0 0x56082d7f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.476+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2428001691 >> 172.21.15.67:0/2428001691 conn(0x56082d430300 msgr2=0x56082d7db350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.476+0000 7fb5affff640 0 client.18993 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.476+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2428001691 shutdown_connections 2024-07-31T15:07:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.476+0000 7fb5d9e81740 1 -- 172.21.15.67:0/2428001691 wait complete. 2024-07-31T15:07:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.476+0000 7fb5d9e81740 20 client.18993 trim_cache size 0 max 16384 2024-07-31T15:07:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:07:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:07:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:07:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:07:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.479+0000 7fb5d2f87640 1 -- 172.21.15.67:0/317475650 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5b40b3ed0 msgr2=0x7fb5b40b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.479+0000 7fb5d2f87640 1 --2- 172.21.15.67:0/317475650 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5b40b3ed0 0x7fb5b40b63e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40094e0 tx=0x7fb5c4009470 comp rx=0 tx=0).stop 2024-07-31T15:07:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.479+0000 7fb5d2f87640 1 -- 172.21.15.67:0/317475650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5cc11c5d0 msgr2=0x7fb5cc1156c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.479+0000 7fb5d2f87640 1 --2- 172.21.15.67:0/317475650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5cc11c5d0 0x7fb5cc1156c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000d680 tx=0x7fb5c000db40 comp rx=0 tx=0).stop 2024-07-31T15:07:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.479+0000 7fb5d2f87640 1 -- 172.21.15.67:0/317475650 shutdown_connections 2024-07-31T15:07:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.479+0000 7fb5d2f87640 1 --2- 172.21.15.67:0/317475650 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb5b40b3ed0 0x7fb5b40b63e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.479+0000 7fb5d2f87640 1 --2- 172.21.15.67:0/317475650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5cc11c5d0 0x7fb5cc1156c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.480+0000 7fb5d2f87640 1 --2- 172.21.15.67:0/317475650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc1116d0 0x7fb5cc115120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.480+0000 7fb5d2f87640 1 --2- 172.21.15.67:0/317475650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5cc10a330 0x7fb5cc11be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.480+0000 7fb5d2f87640 1 -- 172.21.15.67:0/317475650 >> 172.21.15.67:0/317475650 conn(0x7fb5cc06e350 msgr2=0x7fb5cc110350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.480+0000 7fb5d2f87640 1 -- 172.21.15.67:0/317475650 shutdown_connections 2024-07-31T15:07:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:29.480+0000 7fb5d2f87640 1 -- 172.21.15.67:0/317475650 wait complete. 2024-07-31T15:07:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: pgmap v920: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 879 KiB/s wr, 3.91k op/s 2024-07-31T15:07:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:29.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.651 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utalloc.o 2024-07-31T15:07:29.661 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/tracefs/inode.o 2024-07-31T15:07:29.674 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/acpi/pmic/built-in.a 2024-07-31T15:07:29.694 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sched/sch_mq.o 2024-07-31T15:07:29.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: pgmap v920: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 879 KiB/s wr, 3.91k op/s 2024-07-31T15:07:29.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:29.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: pgmap v920: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 879 KiB/s wr, 3.91k op/s 2024-07-31T15:07:29.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:29.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:29.952 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/char/random.o 2024-07-31T15:07:29.967 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/jump_label.o 2024-07-31T15:07:30.026 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utascii.o 2024-07-31T15:07:30.066 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/xattr.o 2024-07-31T15:07:30.414 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/tracefs/built-in.a 2024-07-31T15:07:30.504 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utbuffer.o 2024-07-31T15:07:30.625 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sched/sch_frag.o 2024-07-31T15:07:30.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:30 smithi073 ceph-mon[36714]: pgmap v921: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 764 KiB/s wr, 4.12k op/s 2024-07-31T15:07:30.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:30.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:30.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:30.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:30 smithi077 ceph-mon[37327]: pgmap v921: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 764 KiB/s wr, 4.12k op/s 2024-07-31T15:07:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:30 smithi067 ceph-mon[32750]: pgmap v921: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 764 KiB/s wr, 4.12k op/s 2024-07-31T15:07:30.808 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/context_tracking.o 2024-07-31T15:07:30.857 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] fs/efivarfs/inode.o 2024-07-31T15:07:31.029 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/9p/built-in.a 2024-07-31T15:07:31.045 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] fs/efivarfs/file.o 2024-07-31T15:07:31.080 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/8250/8250_dma.o 2024-07-31T15:07:31.114 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netlink/af_netlink.o 2024-07-31T15:07:31.397 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utcksum.o 2024-07-31T15:07:31.626 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/bpf/built-in.a 2024-07-31T15:07:31.640 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] fs/efivarfs/super.o 2024-07-31T15:07:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:07:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:31.722 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] fs/efivarfs/vars.o 2024-07-31T15:07:31.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:07:31.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:31.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:07:31.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:07:31.819 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utcopy.o 2024-07-31T15:07:31.854 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/char/misc.o 2024-07-31T15:07:31.864 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/iomem.o 2024-07-31T15:07:31.917 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/8250/8250_dwlib.o 2024-07-31T15:07:32.080 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sched/sch_api.o 2024-07-31T15:07:32.278 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sched/sch_blackhole.o 2024-07-31T15:07:32.285 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utexcep.o 2024-07-31T15:07:32.358 INFO:tasks.workunit.client.0.smithi067.stdout: LD [M] fs/efivarfs/efivarfs.o 2024-07-31T15:07:32.397 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/open.o 2024-07-31T15:07:32.419 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/rseq.o 2024-07-31T15:07:32.448 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/char/virtio_console.o 2024-07-31T15:07:32.615 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/8250/8250_pcilib.o 2024-07-31T15:07:32.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:32 smithi073 ceph-mon[36714]: pgmap v922: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.0 MiB/s wr, 4.06k op/s 2024-07-31T15:07:32.671 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utdebug.o 2024-07-31T15:07:32.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:32 smithi077 ceph-mon[37327]: pgmap v922: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.0 MiB/s wr, 4.06k op/s 2024-07-31T15:07:32.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:32 smithi067 ceph-mon[32750]: pgmap v922: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.0 MiB/s wr, 4.06k op/s 2024-07-31T15:07:33.079 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utdecode.o 2024-07-31T15:07:33.184 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/read_write.o 2024-07-31T15:07:33.304 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/built-in.a 2024-07-31T15:07:33.391 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/8250/8250_pci.o 2024-07-31T15:07:33.431 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netlink/genetlink.o 2024-07-31T15:07:33.432 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utdelete.o 2024-07-31T15:07:33.690 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/char/hpet.o 2024-07-31T15:07:33.728 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/8250/8250_exar.o 2024-07-31T15:07:33.854 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/uterror.o 2024-07-31T15:07:33.957 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sched/cls_api.o 2024-07-31T15:07:33.964 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/char/nvram.o 2024-07-31T15:07:34.258 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/uteval.o 2024-07-31T15:07:34.536 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utglobal.o 2024-07-31T15:07:34.581 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/8250/8250_early.o 2024-07-31T15:07:34.638 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/uthex.o 2024-07-31T15:07:34.664 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/8250/8250_lpss.o 2024-07-31T15:07:34.717 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/file_table.o 2024-07-31T15:07:34.755 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/char/built-in.a 2024-07-31T15:07:34.799 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/8250/8250_mid.o 2024-07-31T15:07:34.856 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netlink/policy.o 2024-07-31T15:07:34.932 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/8250/8250_pericom.o 2024-07-31T15:07:35.026 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utids.o 2024-07-31T15:07:35.231 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utinit.o 2024-07-31T15:07:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:34 smithi077 ceph-mon[37327]: pgmap v923: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.0 MiB/s wr, 3.62k op/s 2024-07-31T15:07:35.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:34 smithi067 ceph-mon[32750]: pgmap v923: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.0 MiB/s wr, 3.62k op/s 2024-07-31T15:07:35.307 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utlock.o 2024-07-31T15:07:35.345 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:34 smithi073 ceph-mon[36714]: pgmap v923: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.0 MiB/s wr, 3.62k op/s 2024-07-31T15:07:35.432 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utmath.o 2024-07-31T15:07:35.469 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/super.o 2024-07-31T15:07:35.503 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/char_dev.o 2024-07-31T15:07:35.574 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/tty/serial/8250/built-in.a 2024-07-31T15:07:35.588 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/netlink/built-in.a 2024-07-31T15:07:35.599 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/stat.o 2024-07-31T15:07:35.613 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/serial_core.o 2024-07-31T15:07:35.627 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/exec.o 2024-07-31T15:07:35.654 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utmisc.o 2024-07-31T15:07:35.786 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utmutex.o 2024-07-31T15:07:36.026 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utnonansi.o 2024-07-31T15:07:36.160 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utobject.o 2024-07-31T15:07:36.193 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/pipe.o 2024-07-31T15:07:36.236 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sched/act_api.o 2024-07-31T15:07:36.393 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utosi.o 2024-07-31T15:07:36.520 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/namei.o 2024-07-31T15:07:36.521 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fcntl.o 2024-07-31T15:07:36.623 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utownerid.o 2024-07-31T15:07:36.794 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utpredef.o 2024-07-31T15:07:36.994 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utresdecode.o 2024-07-31T15:07:37.086 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ioctl.o 2024-07-31T15:07:37.126 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/serial_base_bus.o 2024-07-31T15:07:37.159 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utresrc.o 2024-07-31T15:07:37.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:36 smithi077 ceph-mon[37327]: pgmap v924: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 2.92k op/s 2024-07-31T15:07:37.269 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/readdir.o 2024-07-31T15:07:37.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:36 smithi067 ceph-mon[32750]: pgmap v924: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 2.92k op/s 2024-07-31T15:07:37.328 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utstate.o 2024-07-31T15:07:37.353 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/select.o 2024-07-31T15:07:37.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:36 smithi073 ceph-mon[36714]: pgmap v924: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 2.92k op/s 2024-07-31T15:07:37.566 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utstring.o 2024-07-31T15:07:37.711 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utstrsuppt.o 2024-07-31T15:07:37.718 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/serial_ctrl.o 2024-07-31T15:07:37.749 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sched/sch_fifo.o 2024-07-31T15:07:37.953 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/dcache.o 2024-07-31T15:07:37.993 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utstrtoul64.o 2024-07-31T15:07:38.106 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utxface.o 2024-07-31T15:07:38.122 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/inode.o 2024-07-31T15:07:38.257 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/serial_port.o 2024-07-31T15:07:38.359 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utxfinit.o 2024-07-31T15:07:38.510 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utxferror.o 2024-07-31T15:07:38.580 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sched/cls_cgroup.o 2024-07-31T15:07:38.721 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpica/utxfmutex.o 2024-07-31T15:07:38.729 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/attr.o 2024-07-31T15:07:38.811 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/serial/earlycon.o 2024-07-31T15:07:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/bad_inode.o 2024-07-31T15:07:38.996 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/file.o 2024-07-31T15:07:39.092 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/acpi/acpica/built-in.a 2024-07-31T15:07:39.180 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/dptf/int340x_thermal.o 2024-07-31T15:07:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:38 smithi077 ceph-mon[37327]: pgmap v925: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 2.40k op/s 2024-07-31T15:07:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:38 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:07:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:38 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:07:39.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:38 smithi067 ceph-mon[32750]: pgmap v925: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 2.40k op/s 2024-07-31T15:07:39.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:38 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:07:39.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:38 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:07:39.347 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/filesystems.o 2024-07-31T15:07:39.384 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/namespace.o 2024-07-31T15:07:39.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:38 smithi073 ceph-mon[36714]: pgmap v925: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 2.40k op/s 2024-07-31T15:07:39.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:38 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:07:39.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:38 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:07:39.408 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/tables.o 2024-07-31T15:07:39.438 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/tty/serial/built-in.a 2024-07-31T15:07:39.481 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sched/ematch.o 2024-07-31T15:07:39.487 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/tty/ipwireless/built-in.a 2024-07-31T15:07:39.503 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/tty_io.o 2024-07-31T15:07:39.533 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/seq_file.o 2024-07-31T15:07:39.605 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/acpi/dptf/built-in.a 2024-07-31T15:07:39.632 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/blacklist.o 2024-07-31T15:07:40.008 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/xattr.o 2024-07-31T15:07:40.047 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/osi.o 2024-07-31T15:07:40.181 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/osl.o 2024-07-31T15:07:40.213 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/libfs.o 2024-07-31T15:07:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:39 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:07:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:39 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:07:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:39 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:07:40.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:39 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:07:40.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:39 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:07:40.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:39 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:07:40.369 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fs-writeback.o 2024-07-31T15:07:40.370 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/sched/built-in.a 2024-07-31T15:07:40.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:39 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:07:40.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:39 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:07:40.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:39 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:07:40.438 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/ioctl.o 2024-07-31T15:07:40.541 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/utils.o 2024-07-31T15:07:40.922 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/n_tty.o 2024-07-31T15:07:41.039 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/pnode.o 2024-07-31T15:07:41.145 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/reboot.o 2024-07-31T15:07:41.149 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/nvs.o 2024-07-31T15:07:41.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:40 smithi077 ceph-mon[37327]: pgmap v926: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 1.33k op/s 2024-07-31T15:07:41.249 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/splice.o 2024-07-31T15:07:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:40 smithi067 ceph-mon[32750]: pgmap v926: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 1.33k op/s 2024-07-31T15:07:41.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:40 smithi073 ceph-mon[36714]: pgmap v926: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 1.33k op/s 2024-07-31T15:07:41.506 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/sync.o 2024-07-31T15:07:41.645 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/utimes.o 2024-07-31T15:07:41.726 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/wakeup.o 2024-07-31T15:07:41.806 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/sleep.o 2024-07-31T15:07:42.146 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/tty_ioctl.o 2024-07-31T15:07:42.165 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/tty_ldisc.o 2024-07-31T15:07:42.198 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/tty_buffer.o 2024-07-31T15:07:42.231 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/d_path.o 2024-07-31T15:07:42.388 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/stack.o 2024-07-31T15:07:42.416 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/common.o 2024-07-31T15:07:42.648 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/netlink.o 2024-07-31T15:07:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.741+0000 7fd80cb7e640 1 -- 172.21.15.67:0/656490308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd80810d230 msgr2=0x7fd808116ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.741+0000 7fd80cb7e640 1 --2- 172.21.15.67:0/656490308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd80810d230 0x7fd808116ca0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00a910 tx=0x7fd7fc02a610 comp rx=0 tx=0).stop 2024-07-31T15:07:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.743+0000 7fd80cb7e640 1 -- 172.21.15.67:0/656490308 shutdown_connections 2024-07-31T15:07:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.743+0000 7fd80cb7e640 1 --2- 172.21.15.67:0/656490308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd80810d230 0x7fd808116ca0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.743+0000 7fd80cb7e640 1 --2- 172.21.15.67:0/656490308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd80810c8e0 0x7fd80810ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.743+0000 7fd80cb7e640 1 --2- 172.21.15.67:0/656490308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd80810a920 0x7fd80810ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.743+0000 7fd80cb7e640 1 -- 172.21.15.67:0/656490308 >> 172.21.15.67:0/656490308 conn(0x7fd80806e350 msgr2=0x7fd80806e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.743+0000 7fd80cb7e640 1 -- 172.21.15.67:0/656490308 shutdown_connections 2024-07-31T15:07:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.743+0000 7fd80cb7e640 1 -- 172.21.15.67:0/656490308 wait complete. 2024-07-31T15:07:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.744+0000 7fd80cb7e640 1 Processor -- start 2024-07-31T15:07:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.744+0000 7fd80cb7e640 1 -- start start 2024-07-31T15:07:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd80cb7e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd80810a920 0x7fd8081a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd80cb7e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd80810c8e0 0x7fd8081a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd80cb7e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd80810d230 0x7fd8081a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd80cb7e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081198b0 con 0x7fd80810a920 2024-07-31T15:07:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd80cb7e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd808119730 con 0x7fd80810d230 2024-07-31T15:07:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd80cb7e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd808119a30 con 0x7fd80810c8e0 2024-07-31T15:07:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd806d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd80810d230 0x7fd8081a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd805d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd80810c8e0 0x7fd8081a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd806d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd80810d230 0x7fd8081a5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:55018/0 (socket says 172.21.15.67:55018) 2024-07-31T15:07:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd806d9d640 1 -- 172.21.15.67:0/1871290230 learned_addr learned my addr 172.21.15.67:0/1871290230 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd80659c640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd80810a920 0x7fd8081a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd805d9b640 1 -- 172.21.15.67:0/1871290230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd80810d230 msgr2=0x7fd8081a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd805d9b640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd80810d230 0x7fd8081a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd805d9b640 1 -- 172.21.15.67:0/1871290230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd80810a920 msgr2=0x7fd8081a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.745+0000 7fd805d9b640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd80810a920 0x7fd8081a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.746+0000 7fd805d9b640 1 -- 172.21.15.67:0/1871290230 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8081a61a0 con 0x7fd80810c8e0 2024-07-31T15:07:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.746+0000 7fd806d9d640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd80810d230 0x7fd8081a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:07:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.746+0000 7fd80659c640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd80810a920 0x7fd8081a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:07:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.746+0000 7fd805d9b640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd80810c8e0 0x7fd8081a0810 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd80806c170 tx=0x7fd80000e490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.750+0000 7fd7f77fe640 1 -- 172.21.15.67:0/1871290230 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80000eb70 con 0x7fd80810c8e0 2024-07-31T15:07:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.750+0000 7fd7f77fe640 1 -- 172.21.15.67:0/1871290230 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd800003e20 con 0x7fd80810c8e0 2024-07-31T15:07:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.752+0000 7fd7f77fe640 1 -- 172.21.15.67:0/1871290230 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000047a0 con 0x7fd80810c8e0 2024-07-31T15:07:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.752+0000 7fd80cb7e640 1 -- 172.21.15.67:0/1871290230 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8081a6490 con 0x7fd80810c8e0 2024-07-31T15:07:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.752+0000 7fd80cb7e640 1 -- 172.21.15.67:0/1871290230 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8081b1030 con 0x7fd80810c8e0 2024-07-31T15:07:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.756+0000 7fd813a74740 1 Processor -- start 2024-07-31T15:07:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.756+0000 7fd7f77fe640 1 -- 172.21.15.67:0/1871290230 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd80001b030 con 0x7fd80810c8e0 2024-07-31T15:07:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.758+0000 7fd7f77fe640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7f00b3d50 0x7fd7f00b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.758+0000 7fd7f77fe640 1 -- 172.21.15.67:0/1871290230 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9379+0+0 (secure 0 0 0) 0x7fd8000e37d0 con 0x7fd80810c8e0 2024-07-31T15:07:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.758+0000 7fd806d9d640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7f00b3d50 0x7fd7f00b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.758+0000 7fd813a74740 1 -- start start 2024-07-31T15:07:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.759+0000 7fd806d9d640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7f00b3d50 0x7fd7f00b6260 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00a6e0 tx=0x7fd7fc005d50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.759+0000 7fd813a74740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2a21b0 0x56481c644ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.759+0000 7fd813a74740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2792f0 0x56481c6453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.759+0000 7fd813a74740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2814d0 0x56481c645920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.759+0000 7fd813a74740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56481c5ba3b0 con 0x56481c2814d0 2024-07-31T15:07:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.759+0000 7fd813a74740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56481c5a98b0 con 0x56481c2792f0 2024-07-31T15:07:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.759+0000 7fd813a74740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56481c5ad8f0 con 0x56481c2a21b0 2024-07-31T15:07:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.759+0000 7fd806d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2814d0 0x56481c645920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.759+0000 7fd805d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2792f0 0x56481c6453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.759+0000 7fd805d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2792f0 0x56481c6453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:55034/0 (socket says 172.21.15.67:55034) 2024-07-31T15:07:42.762 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/device_sysfs.o 2024-07-31T15:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.759+0000 7fd805d9b640 1 -- 172.21.15.67:0/3769697217 learned_addr learned my addr 172.21.15.67:0/3769697217 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.759+0000 7fd806d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2814d0 0x56481c645920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34206/0 (socket says 172.21.15.67:34206) 2024-07-31T15:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.759+0000 7fd80659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2a21b0 0x56481c644ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.760+0000 7fd805d9b640 1 -- 172.21.15.67:0/3769697217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2a21b0 msgr2=0x56481c644ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.760+0000 7fd805d9b640 1 --2- 172.21.15.67:0/3769697217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2a21b0 0x56481c644ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.760+0000 7fd805d9b640 1 -- 172.21.15.67:0/3769697217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2814d0 msgr2=0x56481c645920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.760+0000 7fd805d9b640 1 --2- 172.21.15.67:0/3769697217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2814d0 0x56481c645920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.760+0000 7fd805d9b640 1 -- 172.21.15.67:0/3769697217 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56481c5509a0 con 0x56481c2792f0 2024-07-31T15:07:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.760+0000 7fd806d9d640 1 --2- 172.21.15.67:0/3769697217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2814d0 0x56481c645920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:07:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.760+0000 7fd805d9b640 1 --2- 172.21.15.67:0/3769697217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2792f0 0x56481c6453e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd8000e0aa0 tx=0x7fd8000a9c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c8e07ea4f1c9e69 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.760+0000 7fd7f4ff9640 1 -- 172.21.15.67:0/3769697217 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000b30d0 con 0x56481c2792f0 2024-07-31T15:07:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.760+0000 7fd7f4ff9640 1 -- 172.21.15.67:0/3769697217 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd80000ed10 con 0x56481c2792f0 2024-07-31T15:07:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.760+0000 7fd7f4ff9640 1 -- 172.21.15.67:0/3769697217 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000e3590 con 0x56481c2792f0 2024-07-31T15:07:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.762+0000 7fd813a74740 1 -- 172.21.15.67:0/3769697217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2792f0 msgr2=0x56481c6453e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.762+0000 7fd813a74740 1 --2- 172.21.15.67:0/3769697217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2792f0 0x56481c6453e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd8000e0aa0 tx=0x7fd8000a9c60 comp rx=0 tx=0).stop 2024-07-31T15:07:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.762+0000 7fd813a74740 1 -- 172.21.15.67:0/3769697217 shutdown_connections 2024-07-31T15:07:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.762+0000 7fd813a74740 1 --2- 172.21.15.67:0/3769697217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2814d0 0x56481c645920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.762+0000 7fd813a74740 1 --2- 172.21.15.67:0/3769697217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2792f0 0x56481c6453e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.762+0000 7fd813a74740 1 --2- 172.21.15.67:0/3769697217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2a21b0 0x56481c644ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.762+0000 7fd813a74740 1 -- 172.21.15.67:0/3769697217 >> 172.21.15.67:0/3769697217 conn(0x56481c296300 msgr2=0x56481c642d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.762+0000 7fd813a74740 1 -- 172.21.15.67:0/3769697217 shutdown_connections 2024-07-31T15:07:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.762+0000 7fd813a74740 1 -- 172.21.15.67:0/3769697217 wait complete. 2024-07-31T15:07:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.763+0000 7fd813a74740 1 -- start start 2024-07-31T15:07:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.763+0000 7fd813a74740 1 Processor -- start 2024-07-31T15:07:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.763+0000 7fd813a74740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2792f0 0x56481c650a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.763+0000 7fd813a74740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2814d0 0x56481c650f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.763+0000 7fd813a74740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2a21b0 0x56481c65c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.763+0000 7fd813a74740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56481c4e3ab0 con 0x56481c2a21b0 2024-07-31T15:07:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.763+0000 7fd813a74740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56481c531cf0 con 0x56481c2792f0 2024-07-31T15:07:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.763+0000 7fd813a74740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56481c653620 con 0x56481c2814d0 2024-07-31T15:07:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.763+0000 7fd813a74740 10 client.0 _ll_register_callbacks cb 0x56481c5a39f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.764+0000 7fd80659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2792f0 0x56481c650a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.764+0000 7fd80659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2792f0 0x56481c650a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:55042/0 (socket says 172.21.15.67:55042) 2024-07-31T15:07:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.764+0000 7fd805d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2814d0 0x56481c650f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.764+0000 7fd80659c640 1 -- 172.21.15.67:0/2018136063 learned_addr learned my addr 172.21.15.67:0/2018136063 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.764+0000 7fd806d9d640 1 --2- 172.21.15.67:0/2018136063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2a21b0 0x56481c65c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.764+0000 7fd805d9b640 1 -- 172.21.15.67:0/2018136063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2792f0 msgr2=0x56481c650a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.765+0000 7fd805d9b640 1 --2- 172.21.15.67:0/2018136063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2792f0 0x56481c650a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.765+0000 7fd805d9b640 1 -- 172.21.15.67:0/2018136063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2a21b0 msgr2=0x56481c65c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.765+0000 7fd805d9b640 1 --2- 172.21.15.67:0/2018136063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2a21b0 0x56481c65c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.765+0000 7fd805d9b640 1 -- 172.21.15.67:0/2018136063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56481c30d790 con 0x56481c2814d0 2024-07-31T15:07:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.765+0000 7fd80659c640 1 --2- 172.21.15.67:0/2018136063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2792f0 0x56481c650a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:07:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.765+0000 7fd806d9d640 1 --2- 172.21.15.67:0/2018136063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2a21b0 0x56481c65c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:07:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.765+0000 7fd805d9b640 1 --2- 172.21.15.67:0/2018136063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2814d0 0x56481c650f90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd8000a9d20 tx=0x7fd8000e0af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e003c0faacd9afd4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.765+0000 7fd7edffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:07:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.766+0000 7fd7edffb640 1 -- 172.21.15.67:0/2018136063 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000c9060 con 0x56481c2814d0 2024-07-31T15:07:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.766+0000 7fd813a74740 1 -- 172.21.15.67:0/2018136063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd80000ed10 con 0x56481c2814d0 2024-07-31T15:07:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.766+0000 7fd7edffb640 1 -- 172.21.15.67:0/2018136063 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9379+0+0 (secure 0 0 0) 0x7fd8000a6610 con 0x56481c2814d0 2024-07-31T15:07:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.766+0000 7fd7edffb640 1 -- 172.21.15.67:0/2018136063 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fd8000cb300 con 0x56481c2814d0 2024-07-31T15:07:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.766+0000 7fd813a74740 10 client.28432 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.766+0000 7fd813a74740 1 -- 172.21.15.67:0/2018136063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x56481c386620 con 0x56481c2814d0 2024-07-31T15:07:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.766+0000 7fd7edffb640 1 -- 172.21.15.67:0/2018136063 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd80000ed10 con 0x56481c2814d0 2024-07-31T15:07:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.767+0000 7fd7edffb640 1 -- 172.21.15.67:0/2018136063 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fd8000c4b60 con 0x56481c2814d0 2024-07-31T15:07:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.767+0000 7fd813a74740 10 client.28432 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.767+0000 7fd813a74740 10 client.28432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.767+0000 7fd813a74740 20 client.28432 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.767+0000 7fd813a74740 1 --2- 172.21.15.67:0/2018136063 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56481c260d80 0x56481c6627c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.767+0000 7fd813a74740 4 client.28432 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:07:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.767+0000 7fd813a74740 1 -- 172.21.15.67:0/2018136063 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56481c5bc300 con 0x56481c260d80 2024-07-31T15:07:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.767+0000 7fd80659c640 1 --2- 172.21.15.67:0/2018136063 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56481c260d80 0x56481c6627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.768+0000 7fd7edffb640 10 client.28432 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.768+0000 7fd80659c640 1 --2- 172.21.15.67:0/2018136063 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56481c260d80 0x56481c6627c0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ef11218edf3c045 server_cookie=f608363bd2242fd9 in_seq=0 out_seq=0 2024-07-31T15:07:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.769+0000 7fd7edffb640 1 -- 172.21.15.67:0/2018136063 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56481c5bc300 con 0x56481c260d80 2024-07-31T15:07:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.769+0000 7fd7edffb640 10 client.28432 handle_command_reply: tid=0 2024-07-31T15:07:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.769+0000 7fd813a74740 1 client.28432 shutdown 2024-07-31T15:07:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.769+0000 7fd813a74740 1 -- 172.21.15.67:0/2018136063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2814d0 msgr2=0x56481c650f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.769+0000 7fd813a74740 1 --2- 172.21.15.67:0/2018136063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2814d0 0x56481c650f90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd8000a9d20 tx=0x7fd8000e0af0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.770+0000 7fd813a74740 1 -- 172.21.15.67:0/2018136063 shutdown_connections 2024-07-31T15:07:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.770+0000 7fd813a74740 1 --2- 172.21.15.67:0/2018136063 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56481c260d80 0x56481c6627c0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.770+0000 7fd813a74740 1 --2- 172.21.15.67:0/2018136063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2a21b0 0x56481c65c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.770+0000 7fd813a74740 1 --2- 172.21.15.67:0/2018136063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2814d0 0x56481c650f90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.770+0000 7fd813a74740 1 --2- 172.21.15.67:0/2018136063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2792f0 0x56481c650a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.770+0000 7fd813a74740 1 -- 172.21.15.67:0/2018136063 >> 172.21.15.67:0/2018136063 conn(0x56481c296300 msgr2=0x56481c645ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.770+0000 7fd7edffb640 0 client.28432 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.770+0000 7fd813a74740 1 -- 172.21.15.67:0/2018136063 shutdown_connections 2024-07-31T15:07:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.770+0000 7fd813a74740 1 -- 172.21.15.67:0/2018136063 wait complete. 2024-07-31T15:07:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.770+0000 7fd813a74740 20 client.28432 trim_cache size 0 max 16384 2024-07-31T15:07:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.830+0000 7fd813a74740 1 Processor -- start 2024-07-31T15:07:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.831+0000 7fd813a74740 1 -- start start 2024-07-31T15:07:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.831+0000 7fd813a74740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2792f0 0x56481c65c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.831+0000 7fd813a74740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2814d0 0x56481c64af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.831+0000 7fd813a74740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2a21b0 0x56481c64f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.831+0000 7fd813a74740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081bad50 con 0x56481c2792f0 2024-07-31T15:07:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.832+0000 7fd80659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2792f0 0x56481c65c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.832+0000 7fd805d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2814d0 0x56481c64af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.832+0000 7fd806d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2a21b0 0x56481c64f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.832+0000 7fd805d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2814d0 0x56481c64af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60620/0 (socket says 172.21.15.67:60620) 2024-07-31T15:07:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.832+0000 7fd805d9b640 1 -- 172.21.15.67:0/1728450171 learned_addr learned my addr 172.21.15.67:0/1728450171 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.834+0000 7fd813a74740 1 -- 172.21.15.67:0/1728450171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081bc890 con 0x56481c2a21b0 2024-07-31T15:07:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.834+0000 7fd813a74740 1 -- 172.21.15.67:0/1728450171 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081bb420 con 0x56481c2814d0 2024-07-31T15:07:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.835+0000 7fd806d9d640 1 -- 172.21.15.67:0/1728450171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2814d0 msgr2=0x56481c64af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.835+0000 7fd806d9d640 1 --2- 172.21.15.67:0/1728450171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2814d0 0x56481c64af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.835+0000 7fd806d9d640 1 -- 172.21.15.67:0/1728450171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2792f0 msgr2=0x56481c65c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.835+0000 7fd806d9d640 1 --2- 172.21.15.67:0/1728450171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2792f0 0x56481c65c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.835+0000 7fd806d9d640 1 -- 172.21.15.67:0/1728450171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56481c647cb0 con 0x56481c2a21b0 2024-07-31T15:07:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.836+0000 7fd80659c640 1 --2- 172.21.15.67:0/1728450171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2792f0 0x56481c65c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:07:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.836+0000 7fd806d9d640 1 --2- 172.21.15.67:0/1728450171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2a21b0 0x56481c64f0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc005de0 tx=0x7fd7fc03e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3870cb2608c7cf9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.836+0000 7fd7effff640 1 -- 172.21.15.67:0/1728450171 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc035040 con 0x56481c2a21b0 2024-07-31T15:07:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.837+0000 7fd7effff640 1 -- 172.21.15.67:0/1728450171 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7fc050070 con 0x56481c2a21b0 2024-07-31T15:07:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.837+0000 7fd7effff640 1 -- 172.21.15.67:0/1728450171 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc04c6a0 con 0x56481c2a21b0 2024-07-31T15:07:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.838+0000 7fd813a74740 1 -- 172.21.15.67:0/1728450171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2a21b0 msgr2=0x56481c64f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.838+0000 7fd813a74740 1 --2- 172.21.15.67:0/1728450171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2a21b0 0x56481c64f0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc005de0 tx=0x7fd7fc03e000 comp rx=0 tx=0).stop 2024-07-31T15:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.840+0000 7fd813a74740 1 -- 172.21.15.67:0/1728450171 shutdown_connections 2024-07-31T15:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.840+0000 7fd813a74740 1 --2- 172.21.15.67:0/1728450171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2a21b0 0x56481c64f0a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.840+0000 7fd813a74740 1 --2- 172.21.15.67:0/1728450171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2814d0 0x56481c64af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.840+0000 7fd813a74740 1 --2- 172.21.15.67:0/1728450171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2792f0 0x56481c65c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.840+0000 7fd813a74740 1 -- 172.21.15.67:0/1728450171 >> 172.21.15.67:0/1728450171 conn(0x56481c296300 msgr2=0x56481c650a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.841+0000 7fd813a74740 1 -- 172.21.15.67:0/1728450171 shutdown_connections 2024-07-31T15:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.841+0000 7fd813a74740 1 -- 172.21.15.67:0/1728450171 wait complete. 2024-07-31T15:07:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.841+0000 7fd813a74740 1 -- start start 2024-07-31T15:07:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.841+0000 7fd813a74740 1 Processor -- start 2024-07-31T15:07:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.842+0000 7fd813a74740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2792f0 0x56481c663ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.842+0000 7fd813a74740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2814d0 0x56481c6643e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.842+0000 7fd813a74740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2a21b0 0x56481c6c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.842+0000 7fd813a74740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081bc1c0 con 0x56481c2814d0 2024-07-31T15:07:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.842+0000 7fd813a74740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081bbaf0 con 0x56481c2a21b0 2024-07-31T15:07:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.842+0000 7fd813a74740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081b8d50 con 0x56481c2792f0 2024-07-31T15:07:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.842+0000 7fd813a74740 10 client.0 _ll_register_callbacks cb 0x7fd8000a9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.842+0000 7fd80659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2792f0 0x56481c663ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.842+0000 7fd805d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2814d0 0x56481c6643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.842+0000 7fd80659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2792f0 0x56481c663ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60628/0 (socket says 172.21.15.67:60628) 2024-07-31T15:07:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.842+0000 7fd80659c640 1 -- 172.21.15.67:0/1815277882 learned_addr learned my addr 172.21.15.67:0/1815277882 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.842+0000 7fd806d9d640 1 --2- 172.21.15.67:0/1815277882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2a21b0 0x56481c6c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.843+0000 7fd806d9d640 1 -- 172.21.15.67:0/1815277882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2792f0 msgr2=0x56481c663ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.843+0000 7fd806d9d640 1 --2- 172.21.15.67:0/1815277882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2792f0 0x56481c663ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.843+0000 7fd806d9d640 1 -- 172.21.15.67:0/1815277882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2814d0 msgr2=0x56481c6643e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.843+0000 7fd806d9d640 1 --2- 172.21.15.67:0/1815277882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2814d0 0x56481c6643e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.843+0000 7fd806d9d640 1 -- 172.21.15.67:0/1815277882 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56481c6c82c0 con 0x56481c2a21b0 2024-07-31T15:07:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.843+0000 7fd805d9b640 1 --2- 172.21.15.67:0/1815277882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2814d0 0x56481c6643e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:07:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.843+0000 7fd806d9d640 1 --2- 172.21.15.67:0/1815277882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2a21b0 0x56481c6c7d00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc02f550 tx=0x7fd7fc00a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d28acbac625b8dd1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.844+0000 7fd7edffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:07:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.844+0000 7fd7edffb640 1 -- 172.21.15.67:0/1815277882 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc035040 con 0x56481c2a21b0 2024-07-31T15:07:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.844+0000 7fd813a74740 1 -- 172.21.15.67:0/1815277882 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7fc050070 con 0x56481c2a21b0 2024-07-31T15:07:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.844+0000 7fd7edffb640 1 -- 172.21.15.67:0/1815277882 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9379+0+0 (secure 0 0 0) 0x7fd7fc055470 con 0x56481c2a21b0 2024-07-31T15:07:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.844+0000 7fd7edffb640 1 -- 172.21.15.67:0/1815277882 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fd7fc069050 con 0x56481c2a21b0 2024-07-31T15:07:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.844+0000 7fd813a74740 10 client.28535 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.844+0000 7fd813a74740 1 -- 172.21.15.67:0/1815277882 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x56481c6c9550 con 0x56481c2a21b0 2024-07-31T15:07:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.845+0000 7fd7edffb640 1 -- 172.21.15.67:0/1815277882 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7fc050070 con 0x56481c2a21b0 2024-07-31T15:07:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.845+0000 7fd7edffb640 1 -- 172.21.15.67:0/1815277882 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fd7fc03a040 con 0x56481c2a21b0 2024-07-31T15:07:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.845+0000 7fd813a74740 10 client.28535 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.845+0000 7fd813a74740 10 client.28535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.845+0000 7fd813a74740 20 client.28535 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.845+0000 7fd813a74740 1 --2- 172.21.15.67:0/1815277882 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56481c2361c0 0x56481c6cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.845+0000 7fd813a74740 4 client.28535 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:07:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.845+0000 7fd813a74740 1 -- 172.21.15.67:0/1815277882 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56481c5b7da0 con 0x56481c2361c0 2024-07-31T15:07:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.845+0000 7fd80659c640 1 --2- 172.21.15.67:0/1815277882 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56481c2361c0 0x56481c6cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.846+0000 7fd7edffb640 10 client.28535 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:42.846+0000 7fd80659c640 1 --2- 172.21.15.67:0/1815277882 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56481c2361c0 0x56481c6cbc90 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6019d0c0fd7e510 server_cookie=6423b42eb7878fe7 in_seq=0 out_seq=0 2024-07-31T15:07:42.867 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/device_pm.o 2024-07-31T15:07:42.886 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/proc.o 2024-07-31T15:07:42.889 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/pci/i386.o 2024-07-31T15:07:42.914 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/tty_port.o 2024-07-31T15:07:42.979 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fs_struct.o 2024-07-31T15:07:43.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:42 smithi077 ceph-mon[37327]: pgmap v927: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 892 op/s 2024-07-31T15:07:43.244 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/pci/init.o 2024-07-31T15:07:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:42 smithi067 ceph-mon[32750]: pgmap v927: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 892 op/s 2024-07-31T15:07:43.371 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/bus.o 2024-07-31T15:07:43.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:42 smithi073 ceph-mon[36714]: pgmap v927: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 892 op/s 2024-07-31T15:07:43.475 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/statfs.o 2024-07-31T15:07:43.528 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/glue.o 2024-07-31T15:07:43.554 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/pci/mmconfig_64.o 2024-07-31T15:07:43.628 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/tty_mutex.o 2024-07-31T15:07:43.903 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/pci/direct.o 2024-07-31T15:07:43.926 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/bitset.o 2024-07-31T15:07:44.346 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fs_pin.o 2024-07-31T15:07:44.355 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/scan.o 2024-07-31T15:07:44.403 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/tty_ldsem.o 2024-07-31T15:07:44.494 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/pci/mmconfig-shared.o 2024-07-31T15:07:44.634 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/pci/fixup.o 2024-07-31T15:07:44.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.634+0000 7fd7edffb640 1 -- 172.21.15.67:0/1815277882 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56481c5b7da0 con 0x56481c2361c0 2024-07-31T15:07:44.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.634+0000 7fd7edffb640 10 client.28535 handle_command_reply: tid=0 2024-07-31T15:07:44.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.634+0000 7fd813a74740 1 client.28535 shutdown 2024-07-31T15:07:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.634+0000 7fd813a74740 1 -- 172.21.15.67:0/1815277882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2a21b0 msgr2=0x56481c6c7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.634+0000 7fd813a74740 1 --2- 172.21.15.67:0/1815277882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2a21b0 0x56481c6c7d00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc02f550 tx=0x7fd7fc00a7a0 comp rx=0 tx=0).stop 2024-07-31T15:07:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.634+0000 7fd813a74740 1 -- 172.21.15.67:0/1815277882 shutdown_connections 2024-07-31T15:07:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.634+0000 7fd813a74740 1 --2- 172.21.15.67:0/1815277882 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56481c2361c0 0x56481c6cbc90 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.634+0000 7fd813a74740 1 --2- 172.21.15.67:0/1815277882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56481c2a21b0 0x56481c6c7d00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.634+0000 7fd813a74740 1 --2- 172.21.15.67:0/1815277882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56481c2814d0 0x56481c6643e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.634+0000 7fd7edffb640 0 client.28535 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.634+0000 7fd813a74740 1 --2- 172.21.15.67:0/1815277882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481c2792f0 0x56481c663ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.634+0000 7fd813a74740 1 -- 172.21.15.67:0/1815277882 >> 172.21.15.67:0/1815277882 conn(0x56481c296300 msgr2=0x56481c64ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.635+0000 7fd813a74740 1 -- 172.21.15.67:0/1815277882 shutdown_connections 2024-07-31T15:07:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.635+0000 7fd813a74740 1 -- 172.21.15.67:0/1815277882 wait complete. 2024-07-31T15:07:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.635+0000 7fd813a74740 20 client.28535 trim_cache size 0 max 16384 2024-07-31T15:07:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:07:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:07:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:07:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:07:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.638+0000 7fd80cb7e640 1 -- 172.21.15.67:0/1871290230 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7f00b3d50 msgr2=0x7fd7f00b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.638+0000 7fd80cb7e640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7f00b3d50 0x7fd7f00b6260 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00a6e0 tx=0x7fd7fc005d50 comp rx=0 tx=0).stop 2024-07-31T15:07:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.638+0000 7fd80cb7e640 1 -- 172.21.15.67:0/1871290230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd80810c8e0 msgr2=0x7fd8081a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.638+0000 7fd80cb7e640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd80810c8e0 0x7fd8081a0810 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd80806c170 tx=0x7fd80000e490 comp rx=0 tx=0).stop 2024-07-31T15:07:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.641+0000 7fd80cb7e640 1 -- 172.21.15.67:0/1871290230 shutdown_connections 2024-07-31T15:07:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.641+0000 7fd80cb7e640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd7f00b3d50 0x7fd7f00b6260 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.641+0000 7fd80cb7e640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd80810d230 0x7fd8081a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.641+0000 7fd80cb7e640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd80810c8e0 0x7fd8081a0810 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.641+0000 7fd80cb7e640 1 --2- 172.21.15.67:0/1871290230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd80810a920 0x7fd8081a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.641+0000 7fd80cb7e640 1 -- 172.21.15.67:0/1871290230 >> 172.21.15.67:0/1871290230 conn(0x7fd80806e350 msgr2=0x7fd80810bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.641+0000 7fd80cb7e640 1 -- 172.21.15.67:0/1871290230 shutdown_connections 2024-07-31T15:07:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:44.641+0000 7fd80cb7e640 1 -- 172.21.15.67:0/1871290230 wait complete. 2024-07-31T15:07:44.925 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/resource.o 2024-07-31T15:07:44.929 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/pci/acpi.o 2024-07-31T15:07:45.070 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nsfs.o 2024-07-31T15:07:45.139 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/tty_baudrate.o 2024-07-31T15:07:45.174 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/strset.o 2024-07-31T15:07:45.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:44 smithi077 ceph-mon[37327]: pgmap v928: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 944 KiB/s wr, 384 op/s 2024-07-31T15:07:45.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:44 smithi067 ceph-mon[32750]: pgmap v928: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 944 KiB/s wr, 384 op/s 2024-07-31T15:07:45.329 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/pci/legacy.o 2024-07-31T15:07:45.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:44 smithi073 ceph-mon[36714]: pgmap v928: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 944 KiB/s wr, 384 op/s 2024-07-31T15:07:45.405 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/pci/irq.o 2024-07-31T15:07:45.550 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpi_processor.o 2024-07-31T15:07:45.596 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/pci/common.o 2024-07-31T15:07:45.622 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/processor_core.o 2024-07-31T15:07:45.643 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fs_types.o 2024-07-31T15:07:45.697 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/tty_jobctrl.o 2024-07-31T15:07:45.883 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/pci/early.o 2024-07-31T15:07:46.087 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fs_context.o 2024-07-31T15:07:46.114 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/linkinfo.o 2024-07-31T15:07:46.138 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/processor_pdc.o 2024-07-31T15:07:46.308 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/pci/bus_numa.o 2024-07-31T15:07:46.342 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/ec.o 2024-07-31T15:07:46.374 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/n_null.o 2024-07-31T15:07:46.411 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/pty.o 2024-07-31T15:07:46.429 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/pci/amd_bus.o 2024-07-31T15:07:46.620 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/dock.o 2024-07-31T15:07:46.885 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/pci_root.o 2024-07-31T15:07:46.887 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fs_parser.o 2024-07-31T15:07:46.899 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/tty_audit.o 2024-07-31T15:07:46.923 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/linkmodes.o 2024-07-31T15:07:47.080 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/pci/built-in.a 2024-07-31T15:07:47.112 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/tty/sysrq.o 2024-07-31T15:07:47.143 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/pci_link.o 2024-07-31T15:07:47.184 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/pci_irq.o 2024-07-31T15:07:47.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:46 smithi077 ceph-mon[37327]: pgmap v929: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 687 KiB/s wr, 119 op/s 2024-07-31T15:07:47.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:46 smithi067 ceph-mon[32750]: pgmap v929: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 687 KiB/s wr, 119 op/s 2024-07-31T15:07:47.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:46 smithi073 ceph-mon[36714]: pgmap v929: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 687 KiB/s wr, 119 op/s 2024-07-31T15:07:47.420 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpi_lpss.o 2024-07-31T15:07:47.478 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fsopen.o 2024-07-31T15:07:47.533 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/init.o 2024-07-31T15:07:47.752 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpi_apd.o 2024-07-31T15:07:47.799 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/rss.o 2024-07-31T15:07:47.840 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpi_platform.o 2024-07-31T15:07:47.886 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpi_pnp.o 2024-07-31T15:07:47.924 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/tty/built-in.a 2024-07-31T15:07:48.005 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/amd/iommu.o 2024-07-31T15:07:48.086 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/power/cpu.o 2024-07-31T15:07:48.134 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/kernel_read_file.o 2024-07-31T15:07:48.202 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/power.o 2024-07-31T15:07:48.205 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/event.o 2024-07-31T15:07:48.539 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/power/hibernate_64.o 2024-07-31T15:07:48.753 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/linkstate.o 2024-07-31T15:07:48.980 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/mnt_idmapping.o 2024-07-31T15:07:49.099 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/evged.o 2024-07-31T15:07:49.182 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/sysfs.o 2024-07-31T15:07:49.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:48 smithi077 ceph-mon[37327]: pgmap v930: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 387 KiB/s wr, 140 op/s 2024-07-31T15:07:49.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:48 smithi067 ceph-mon[32750]: pgmap v930: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 387 KiB/s wr, 140 op/s 2024-07-31T15:07:49.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:48 smithi073 ceph-mon[36714]: pgmap v930: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 387 KiB/s wr, 140 op/s 2024-07-31T15:07:49.670 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/power/hibernate_asm_64.o 2024-07-31T15:07:49.743 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/remap_range.o 2024-07-31T15:07:49.812 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/buffer.o 2024-07-31T15:07:49.924 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/mpage.o 2024-07-31T15:07:49.973 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/property.o 2024-07-31T15:07:50.077 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/debug.o 2024-07-31T15:07:50.159 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/power/hibernate.o 2024-07-31T15:07:50.286 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpi_cmos_rtc.o 2024-07-31T15:07:50.582 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/amd/init.o 2024-07-31T15:07:50.819 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc_namespace.o 2024-07-31T15:07:51.010 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/power/built-in.a 2024-07-31T15:07:51.015 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/x86/apple.o 2024-07-31T15:07:51.027 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/direct-io.o 2024-07-31T15:07:51.037 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/wol.o 2024-07-31T15:07:51.041 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/features.o 2024-07-31T15:07:51.132 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/eventpoll.o 2024-07-31T15:07:51.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:50 smithi077 ceph-mon[37327]: pgmap v931: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 521 KiB/s wr, 144 op/s 2024-07-31T15:07:51.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:50 smithi067 ceph-mon[32750]: pgmap v931: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 521 KiB/s wr, 144 op/s 2024-07-31T15:07:51.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:50 smithi073 ceph-mon[36714]: pgmap v931: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 521 KiB/s wr, 144 op/s 2024-07-31T15:07:51.439 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/anon_inodes.o 2024-07-31T15:07:51.524 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/x86/utils.o 2024-07-31T15:07:51.900 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/privflags.o 2024-07-31T15:07:51.915 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/rings.o 2024-07-31T15:07:52.002 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/signalfd.o 2024-07-31T15:07:52.005 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/x86/s2idle.o 2024-07-31T15:07:52.008 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/timerfd.o 2024-07-31T15:07:52.046 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/host1x/built-in.a 2024-07-31T15:07:52.140 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/tests/built-in.a 2024-07-31T15:07:52.164 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/arm/built-in.a 2024-07-31T15:07:52.186 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/amd/quirks.o 2024-07-31T15:07:52.196 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/display/drm_display_helper_mod.o 2024-07-31T15:07:52.543 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/display/drm_dp_dual_mode_helper.o 2024-07-31T15:07:52.604 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/vga/built-in.a 2024-07-31T15:07:52.619 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/eventfd.o 2024-07-31T15:07:52.679 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/aio.o 2024-07-31T15:07:52.716 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/channels.o 2024-07-31T15:07:52.719 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/debugfs.o 2024-07-31T15:07:52.755 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/amd/io_pgtable.o 2024-07-31T15:07:52.789 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/coalesce.o 2024-07-31T15:07:52.809 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/locks.o 2024-07-31T15:07:52.908 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:07:53.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:52 smithi077 ceph-mon[37327]: pgmap v932: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 997 KiB/s wr, 166 op/s 2024-07-31T15:07:53.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:52 smithi067 ceph-mon[32750]: pgmap v932: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 997 KiB/s wr, 166 op/s 2024-07-31T15:07:53.307 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpi_lpat.o 2024-07-31T15:07:53.334 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/binfmt_misc.o 2024-07-31T15:07:53.367 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/display/drm_dp_helper.o 2024-07-31T15:07:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:52 smithi073 ceph-mon[36714]: pgmap v932: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 997 KiB/s wr, 166 op/s 2024-07-31T15:07:53.560 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/amd/io_pgtable_v2.o 2024-07-31T15:07:53.648 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/pause.o 2024-07-31T15:07:53.747 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/eee.o 2024-07-31T15:07:53.750 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:07:53.845 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpi_lpit.o 2024-07-31T15:07:54.152 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/binfmt_script.o 2024-07-31T15:07:54.231 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/binfmt_elf.o 2024-07-31T15:07:54.316 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/iommu/amd/built-in.a 2024-07-31T15:07:54.318 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/prmt.o 2024-07-31T15:07:54.368 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/intel/dmar.o 2024-07-31T15:07:54.555 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/intel/iommu.o 2024-07-31T15:07:54.595 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpi_pcc.o 2024-07-31T15:07:54.611 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/tsinfo.o 2024-07-31T15:07:54.631 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/intel/pasid.o 2024-07-31T15:07:55.013 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/display/drm_dp_mst_topology.o 2024-07-31T15:07:55.079 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/intel/trace.o 2024-07-31T15:07:55.082 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/ac.o 2024-07-31T15:07:55.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:54 smithi067 ceph-mon[32750]: pgmap v933: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-07-31T15:07:55.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:54 smithi077 ceph-mon[37327]: pgmap v933: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-07-31T15:07:55.382 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/compat_binfmt_elf.o 2024-07-31T15:07:55.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:54 smithi073 ceph-mon[36714]: pgmap v933: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-07-31T15:07:55.460 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/cabletest.o 2024-07-31T15:07:55.507 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/button.o 2024-07-31T15:07:55.590 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/display/drm_dsc_helper.o 2024-07-31T15:07:55.863 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/tunnels.o 2024-07-31T15:07:55.887 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/intel/cap_audit.o 2024-07-31T15:07:56.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.178+0000 7ffb46a7e640 1 -- 172.21.15.67:0/1240647370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb40111790 msgr2=0x7ffb40109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:56.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.179+0000 7ffb46a7e640 1 --2- 172.21.15.67:0/1240647370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb40111790 0x7ffb40109df0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009900 tx=0x7ffb3c02a4a0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.180+0000 7ffb46a7e640 1 -- 172.21.15.67:0/1240647370 shutdown_connections 2024-07-31T15:07:56.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.180+0000 7ffb46a7e640 1 --2- 172.21.15.67:0/1240647370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb4011c5d0 0x7ffb4011e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.180+0000 7ffb46a7e640 1 --2- 172.21.15.67:0/1240647370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb4010a330 0x7ffb4010a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.180+0000 7ffb46a7e640 1 --2- 172.21.15.67:0/1240647370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb40111790 0x7ffb40109df0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.180+0000 7ffb46a7e640 1 -- 172.21.15.67:0/1240647370 >> 172.21.15.67:0/1240647370 conn(0x7ffb4006e420 msgr2=0x7ffb4006e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:56.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.182+0000 7ffb46a7e640 1 -- 172.21.15.67:0/1240647370 shutdown_connections 2024-07-31T15:07:56.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.185+0000 7ffb46a7e640 1 -- 172.21.15.67:0/1240647370 wait complete. 2024-07-31T15:07:56.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.186+0000 7ffb46a7e640 1 Processor -- start 2024-07-31T15:07:56.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.187+0000 7ffb46a7e640 1 -- start start 2024-07-31T15:07:56.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.187+0000 7ffb46a7e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb4010a330 0x7ffb4011a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.187+0000 7ffb46a7e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb40111790 0x7ffb401132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.187+0000 7ffb46a7e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4011c5d0 0x7ffb40113850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.187+0000 7ffb46a7e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffb40122170 con 0x7ffb4010a330 2024-07-31T15:07:56.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.187+0000 7ffb46a7e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffb40121ff0 con 0x7ffb4011c5d0 2024-07-31T15:07:56.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.187+0000 7ffb46a7e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401222f0 con 0x7ffb40111790 2024-07-31T15:07:56.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.188+0000 7ffb4627d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4011c5d0 0x7ffb40113850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:56.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.188+0000 7ffb4627d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4011c5d0 0x7ffb40113850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:55934/0 (socket says 172.21.15.67:55934) 2024-07-31T15:07:56.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.188+0000 7ffb4627d640 1 -- 172.21.15.67:0/3113922903 learned_addr learned my addr 172.21.15.67:0/3113922903 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:56.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.188+0000 7ffb4527b640 1 --2- 172.21.15.67:0/3113922903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb40111790 0x7ffb401132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:56.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.188+0000 7ffb4527b640 1 -- 172.21.15.67:0/3113922903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4011c5d0 msgr2=0x7ffb40113850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:56.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.188+0000 7ffb4527b640 1 --2- 172.21.15.67:0/3113922903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4011c5d0 0x7ffb40113850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.188+0000 7ffb4527b640 1 -- 172.21.15.67:0/3113922903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb4010a330 msgr2=0x7ffb4011a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:56.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.188+0000 7ffb4527b640 1 --2- 172.21.15.67:0/3113922903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb4010a330 0x7ffb4011a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.189+0000 7ffb4527b640 1 -- 172.21.15.67:0/3113922903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb40114070 con 0x7ffb40111790 2024-07-31T15:07:56.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.189+0000 7ffb4527b640 1 --2- 172.21.15.67:0/3113922903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb40111790 0x7ffb401132d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000d6d0 tx=0x7ffb3000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:56.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.190+0000 7ffb36ffd640 1 -- 172.21.15.67:0/3113922903 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30017070 con 0x7ffb40111790 2024-07-31T15:07:56.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.190+0000 7ffb36ffd640 1 -- 172.21.15.67:0/3113922903 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb30009820 con 0x7ffb40111790 2024-07-31T15:07:56.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.190+0000 7ffb36ffd640 1 -- 172.21.15.67:0/3113922903 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30004c40 con 0x7ffb40111790 2024-07-31T15:07:56.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.190+0000 7ffb46a7e640 1 -- 172.21.15.67:0/3113922903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb40118d30 con 0x7ffb40111790 2024-07-31T15:07:56.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.190+0000 7ffb46a7e640 1 -- 172.21.15.67:0/3113922903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb401192a0 con 0x7ffb40111790 2024-07-31T15:07:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.206+0000 7ffb4e231740 1 Processor -- start 2024-07-31T15:07:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.206+0000 7ffb36ffd640 1 -- 172.21.15.67:0/3113922903 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ffb30005220 con 0x7ffb40111790 2024-07-31T15:07:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.207+0000 7ffb36ffd640 1 --2- 172.21.15.67:0/3113922903 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffb2c0b3f10 0x7ffb2c0b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.208+0000 7ffb36ffd640 1 -- 172.21.15.67:0/3113922903 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9379+0+0 (secure 0 0 0) 0x7ffb300e3bd0 con 0x7ffb40111790 2024-07-31T15:07:56.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.211+0000 7ffb4627d640 1 --2- 172.21.15.67:0/3113922903 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffb2c0b3f10 0x7ffb2c0b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.212+0000 7ffb4e231740 1 -- start start 2024-07-31T15:07:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.212+0000 7ffb4e231740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae54384070 0x55ae54729800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.212+0000 7ffb4e231740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae5435b1b0 0x55ae54729d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.212+0000 7ffb4e231740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae54363390 0x55ae5472a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.212+0000 7ffb4e231740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ae5469fb60 con 0x55ae5435b1b0 2024-07-31T15:07:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.212+0000 7ffb4e231740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ae5468f060 con 0x55ae54384070 2024-07-31T15:07:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.212+0000 7ffb4e231740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ae546930a0 con 0x55ae54363390 2024-07-31T15:07:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.213+0000 7ffb45a7c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae54384070 0x55ae54729800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.213+0000 7ffb45a7c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae54384070 0x55ae54729800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:55950/0 (socket says 172.21.15.67:55950) 2024-07-31T15:07:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.213+0000 7ffb45a7c640 1 -- 172.21.15.67:0/3348955261 learned_addr learned my addr 172.21.15.67:0/3348955261 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.213+0000 7ffb45a7c640 1 -- 172.21.15.67:0/3348955261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae54363390 msgr2=0x55ae5472a280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:07:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.213+0000 7ffb45a7c640 1 --2- 172.21.15.67:0/3348955261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae54363390 0x55ae5472a280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.213+0000 7ffb45a7c640 1 -- 172.21.15.67:0/3348955261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae5435b1b0 msgr2=0x55ae54729d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:07:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.213+0000 7ffb45a7c640 1 --2- 172.21.15.67:0/3348955261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae5435b1b0 0x55ae54729d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.213+0000 7ffb45a7c640 1 -- 172.21.15.67:0/3348955261 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae54635b40 con 0x55ae54384070 2024-07-31T15:07:56.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.222+0000 7ffb45a7c640 1 --2- 172.21.15.67:0/3348955261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae54384070 0x55ae54729800 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0040f0 tx=0x7ffb3c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50d437cef5e86858 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:56.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.223+0000 7ffb4627d640 1 --2- 172.21.15.67:0/3113922903 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffb2c0b3f10 0x7ffb2c0b6420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffb4011a770 tx=0x7ffb3800b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:56.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.226+0000 7ffb2bfff640 1 -- 172.21.15.67:0/3348955261 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c03e030 con 0x55ae54384070 2024-07-31T15:07:56.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.226+0000 7ffb2bfff640 1 -- 172.21.15.67:0/3348955261 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb3c0042d0 con 0x55ae54384070 2024-07-31T15:07:56.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.226+0000 7ffb2bfff640 1 -- 172.21.15.67:0/3348955261 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c035040 con 0x55ae54384070 2024-07-31T15:07:56.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.228+0000 7ffb4e231740 1 -- 172.21.15.67:0/3348955261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae54384070 msgr2=0x55ae54729800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:56.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.228+0000 7ffb4e231740 1 --2- 172.21.15.67:0/3348955261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae54384070 0x55ae54729800 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0040f0 tx=0x7ffb3c000bc0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.232+0000 7ffb4e231740 1 -- 172.21.15.67:0/3348955261 shutdown_connections 2024-07-31T15:07:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.232+0000 7ffb4e231740 1 --2- 172.21.15.67:0/3348955261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae54363390 0x55ae5472a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.232+0000 7ffb4e231740 1 --2- 172.21.15.67:0/3348955261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae5435b1b0 0x55ae54729d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.232+0000 7ffb4e231740 1 --2- 172.21.15.67:0/3348955261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae54384070 0x55ae54729800 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.232+0000 7ffb4e231740 1 -- 172.21.15.67:0/3348955261 >> 172.21.15.67:0/3348955261 conn(0x55ae54378460 msgr2=0x55ae547276f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.237+0000 7ffb4e231740 1 -- 172.21.15.67:0/3348955261 shutdown_connections 2024-07-31T15:07:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.237+0000 7ffb4e231740 1 -- 172.21.15.67:0/3348955261 wait complete. 2024-07-31T15:07:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.237+0000 7ffb4e231740 1 -- start start 2024-07-31T15:07:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.237+0000 7ffb4e231740 1 Processor -- start 2024-07-31T15:07:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.239+0000 7ffb4e231740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5435b1b0 0x55ae54735550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.239+0000 7ffb4e231740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae54363390 0x55ae54735af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.239+0000 7ffb4e231740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae54384070 0x55ae5473b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.239+0000 7ffb45a7c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5435b1b0 0x55ae54735550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.239+0000 7ffb45a7c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5435b1b0 0x55ae54735550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:55958/0 (socket says 172.21.15.67:55958) 2024-07-31T15:07:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.239+0000 7ffb45a7c640 1 -- 172.21.15.67:0/4256817914 learned_addr learned my addr 172.21.15.67:0/4256817914 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.239+0000 7ffb4e231740 1 -- 172.21.15.67:0/4256817914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ae544f2700 con 0x55ae54363390 2024-07-31T15:07:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.239+0000 7ffb4e231740 1 -- 172.21.15.67:0/4256817914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ae54631990 con 0x55ae5435b1b0 2024-07-31T15:07:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.240+0000 7ffb4627d640 1 --2- 172.21.15.67:0/4256817914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae54384070 0x55ae5473b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.240+0000 7ffb4e231740 1 -- 172.21.15.67:0/4256817914 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ae54737f80 con 0x55ae54384070 2024-07-31T15:07:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.240+0000 7ffb4e231740 10 client.0 _ll_register_callbacks cb 0x55ae5464c4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.240+0000 7ffb45a7c640 1 -- 172.21.15.67:0/4256817914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae54384070 msgr2=0x55ae5473b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.240+0000 7ffb45a7c640 1 --2- 172.21.15.67:0/4256817914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae54384070 0x55ae5473b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.240+0000 7ffb45a7c640 1 -- 172.21.15.67:0/4256817914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae54363390 msgr2=0x55ae54735af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.240+0000 7ffb45a7c640 1 --2- 172.21.15.67:0/4256817914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae54363390 0x55ae54735af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.240+0000 7ffb45a7c640 1 -- 172.21.15.67:0/4256817914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae543ef650 con 0x55ae5435b1b0 2024-07-31T15:07:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.241+0000 7ffb45a7c640 1 --2- 172.21.15.67:0/4256817914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5435b1b0 0x55ae54735550 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c005a80 tx=0x7ffb3c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99ec55936e1da091 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.241+0000 7ffb4e231740 1 -- 172.21.15.67:0/4256817914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb3c0042d0 con 0x55ae5435b1b0 2024-07-31T15:07:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.241+0000 7ffb297fa640 10 client.28547 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:07:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.241+0000 7ffb297fa640 1 -- 172.21.15.67:0/4256817914 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c035070 con 0x55ae5435b1b0 2024-07-31T15:07:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.241+0000 7ffb297fa640 1 -- 172.21.15.67:0/4256817914 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9379+0+0 (secure 0 0 0) 0x7ffb3c003b90 con 0x55ae5435b1b0 2024-07-31T15:07:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.242+0000 7ffb297fa640 1 -- 172.21.15.67:0/4256817914 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7ffb3c03e040 con 0x55ae5435b1b0 2024-07-31T15:07:56.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.244+0000 7ffb4e231740 10 client.28547 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:56.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.244+0000 7ffb4e231740 1 -- 172.21.15.67:0/4256817914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=65}) -- 0x55ae54743f40 con 0x55ae5435b1b0 2024-07-31T15:07:56.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.249+0000 7ffb297fa640 1 -- 172.21.15.67:0/4256817914 <== mon.2 v2:172.21.15.73:3300/0 4 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7ffb3c005020 con 0x55ae5435b1b0 2024-07-31T15:07:56.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.252+0000 7ffb4e231740 10 client.28547 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:56.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.252+0000 7ffb4e231740 10 client.28547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:56.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.252+0000 7ffb4e231740 20 client.28547 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:56.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.252+0000 7ffb4e231740 1 --2- 172.21.15.67:0/4256817914 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ae543429f0 0x55ae547472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.252+0000 7ffb4e231740 4 client.28547 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:07:56.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.252+0000 7ffb4e231740 1 -- 172.21.15.67:0/4256817914 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffb3c0042d0 con 0x55ae543429f0 2024-07-31T15:07:56.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.257+0000 7ffb4527b640 1 --2- 172.21.15.67:0/4256817914 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ae543429f0 0x55ae547472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:56.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.263+0000 7ffb4527b640 1 --2- 172.21.15.67:0/4256817914 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ae543429f0 0x55ae547472a0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1d39fb931ea9a12 server_cookie=9a0622eea9a8cf9f in_seq=0 out_seq=0 2024-07-31T15:07:56.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.263+0000 7ffb297fa640 10 client.28547 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:56.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.266+0000 7ffb297fa640 1 -- 172.21.15.67:0/4256817914 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ffb3c0042d0 con 0x55ae543429f0 2024-07-31T15:07:56.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.266+0000 7ffb297fa640 10 client.28547 handle_command_reply: tid=0 2024-07-31T15:07:56.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.269+0000 7ffb4e231740 1 client.28547 shutdown 2024-07-31T15:07:56.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.271+0000 7ffb4e231740 1 -- 172.21.15.67:0/4256817914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5435b1b0 msgr2=0x55ae54735550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:56.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.271+0000 7ffb4e231740 1 --2- 172.21.15.67:0/4256817914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5435b1b0 0x55ae54735550 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c005a80 tx=0x7ffb3c003820 comp rx=0 tx=0).stop 2024-07-31T15:07:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.275+0000 7ffb4e231740 1 -- 172.21.15.67:0/4256817914 shutdown_connections 2024-07-31T15:07:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.275+0000 7ffb4e231740 1 --2- 172.21.15.67:0/4256817914 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ae543429f0 0x55ae547472a0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.275+0000 7ffb4e231740 1 --2- 172.21.15.67:0/4256817914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae54384070 0x55ae5473b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.275+0000 7ffb4e231740 1 --2- 172.21.15.67:0/4256817914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae54363390 0x55ae54735af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.275+0000 7ffb297fa640 0 client.28547 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.275+0000 7ffb4e231740 1 --2- 172.21.15.67:0/4256817914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5435b1b0 0x55ae54735550 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.275+0000 7ffb4e231740 1 -- 172.21.15.67:0/4256817914 >> 172.21.15.67:0/4256817914 conn(0x55ae54378460 msgr2=0x55ae5472c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.275+0000 7ffb4e231740 1 -- 172.21.15.67:0/4256817914 shutdown_connections 2024-07-31T15:07:56.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.275+0000 7ffb4e231740 1 -- 172.21.15.67:0/4256817914 wait complete. 2024-07-31T15:07:56.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.276+0000 7ffb4e231740 20 client.28547 trim_cache size 0 max 16384 2024-07-31T15:07:56.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.322+0000 7ffb4e231740 1 Processor -- start 2024-07-31T15:07:56.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.323+0000 7ffb4e231740 1 -- start start 2024-07-31T15:07:56.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.324+0000 7ffb4e231740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae543429f0 0x55ae5473b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.324+0000 7ffb4e231740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5435b1b0 0x55ae5472af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.324+0000 7ffb4e231740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae54363390 0x55ae5472ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.324+0000 7ffb4e231740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401c7de0 con 0x55ae543429f0 2024-07-31T15:07:56.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.324+0000 7ffb4e231740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401c9920 con 0x55ae5435b1b0 2024-07-31T15:07:56.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.324+0000 7ffb4e231740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401c84b0 con 0x55ae54363390 2024-07-31T15:07:56.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.324+0000 7ffb4527b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5435b1b0 0x55ae5472af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:56.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.324+0000 7ffb4527b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5435b1b0 0x55ae5472af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:55968/0 (socket says 172.21.15.67:55968) 2024-07-31T15:07:56.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.324+0000 7ffb4527b640 1 -- 172.21.15.67:0/2409263420 learned_addr learned my addr 172.21.15.67:0/2409263420 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:56.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.324+0000 7ffb4627d640 1 --2- 172.21.15.67:0/2409263420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae54363390 0x55ae5472ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:56.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.325+0000 7ffb4527b640 1 -- 172.21.15.67:0/2409263420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae54363390 msgr2=0x55ae5472ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:56.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.325+0000 7ffb4527b640 1 --2- 172.21.15.67:0/2409263420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae54363390 0x55ae5472ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.325+0000 7ffb4527b640 1 -- 172.21.15.67:0/2409263420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae543429f0 msgr2=0x55ae5473b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:56.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.325+0000 7ffb4527b640 1 --2- 172.21.15.67:0/2409263420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae543429f0 0x55ae5473b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.325+0000 7ffb4527b640 1 -- 172.21.15.67:0/2409263420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae54735e60 con 0x55ae5435b1b0 2024-07-31T15:07:56.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.325+0000 7ffb4527b640 1 --2- 172.21.15.67:0/2409263420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5435b1b0 0x55ae5472af40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009610 tx=0x7ffb300cd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b5f84560a55030f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:56.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.327+0000 7ffb2b7fe640 1 -- 172.21.15.67:0/2409263420 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30020040 con 0x55ae5435b1b0 2024-07-31T15:07:56.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.327+0000 7ffb2b7fe640 1 -- 172.21.15.67:0/2409263420 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb30004de0 con 0x55ae5435b1b0 2024-07-31T15:07:56.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.327+0000 7ffb2b7fe640 1 -- 172.21.15.67:0/2409263420 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb300e11e0 con 0x55ae5435b1b0 2024-07-31T15:07:56.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.328+0000 7ffb4e231740 1 -- 172.21.15.67:0/2409263420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5435b1b0 msgr2=0x55ae5472af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:56.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.328+0000 7ffb4e231740 1 --2- 172.21.15.67:0/2409263420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5435b1b0 0x55ae5472af40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009610 tx=0x7ffb300cd040 comp rx=0 tx=0).stop 2024-07-31T15:07:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.337+0000 7ffb4e231740 1 -- 172.21.15.67:0/2409263420 shutdown_connections 2024-07-31T15:07:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.337+0000 7ffb4e231740 1 --2- 172.21.15.67:0/2409263420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae54363390 0x55ae5472ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.337+0000 7ffb4e231740 1 --2- 172.21.15.67:0/2409263420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5435b1b0 0x55ae5472af40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.337+0000 7ffb4e231740 1 --2- 172.21.15.67:0/2409263420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae543429f0 0x55ae5473b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.337+0000 7ffb4e231740 1 -- 172.21.15.67:0/2409263420 >> 172.21.15.67:0/2409263420 conn(0x55ae54378460 msgr2=0x55ae5472a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:56.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.342+0000 7ffb4e231740 1 -- 172.21.15.67:0/2409263420 shutdown_connections 2024-07-31T15:07:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.346+0000 7ffb4e231740 1 -- 172.21.15.67:0/2409263420 wait complete. 2024-07-31T15:07:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.347+0000 7ffb4e231740 1 -- start start 2024-07-31T15:07:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.347+0000 7ffb4e231740 1 Processor -- start 2024-07-31T15:07:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.348+0000 7ffb4e231740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae543429f0 0x55ae54741f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.348+0000 7ffb4e231740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae5435b1b0 0x55ae5473d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.348+0000 7ffb4e231740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae54363390 0x55ae5473dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.348+0000 7ffb4e231740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401c9250 con 0x55ae54363390 2024-07-31T15:07:56.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.348+0000 7ffb4e231740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401c8b80 con 0x55ae543429f0 2024-07-31T15:07:56.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.348+0000 7ffb4e231740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401c5de0 con 0x55ae5435b1b0 2024-07-31T15:07:56.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.348+0000 7ffb4e231740 10 client.0 _ll_register_callbacks cb 0x7ffb3c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.349+0000 7ffb4527b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae5435b1b0 0x55ae5473d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.349+0000 7ffb4527b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae5435b1b0 0x55ae5473d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:41440/0 (socket says 172.21.15.67:41440) 2024-07-31T15:07:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.349+0000 7ffb4527b640 1 -- 172.21.15.67:0/832639177 learned_addr learned my addr 172.21.15.67:0/832639177 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.351+0000 7ffb4627d640 1 --2- 172.21.15.67:0/832639177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae54363390 0x55ae5473dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.351+0000 7ffb4627d640 1 -- 172.21.15.67:0/832639177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae5435b1b0 msgr2=0x55ae5473d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.351+0000 7ffb4627d640 1 --2- 172.21.15.67:0/832639177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae5435b1b0 0x55ae5473d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.351+0000 7ffb4627d640 1 -- 172.21.15.67:0/832639177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae543429f0 msgr2=0x55ae54741f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.351+0000 7ffb4627d640 1 --2- 172.21.15.67:0/832639177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae543429f0 0x55ae54741f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.351+0000 7ffb4627d640 1 -- 172.21.15.67:0/832639177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae547ae7e0 con 0x55ae54363390 2024-07-31T15:07:56.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.352+0000 7ffb4627d640 1 --2- 172.21.15.67:0/832639177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae54363390 0x55ae5473dc20 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800eb40 tx=0x7ffb38004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d1cafacb86864b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:56.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.356+0000 7ffb297fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:07:56.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.356+0000 7ffb297fa640 1 -- 172.21.15.67:0/832639177 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb38026030 con 0x55ae54363390 2024-07-31T15:07:56.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.356+0000 7ffb297fa640 1 -- 172.21.15.67:0/832639177 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9379+0+0 (secure 0 0 0) 0x7ffb38013ad0 con 0x55ae54363390 2024-07-31T15:07:56.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.356+0000 7ffb4e231740 1 -- 172.21.15.67:0/832639177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb30004de0 con 0x55ae54363390 2024-07-31T15:07:56.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.361+0000 7ffb297fa640 1 -- 172.21.15.67:0/832639177 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7ffb3801b040 con 0x55ae54363390 2024-07-31T15:07:56.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.362+0000 7ffb4e231740 10 client.19014 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:56.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.362+0000 7ffb4e231740 1 -- 172.21.15.67:0/832639177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55ae547afa10 con 0x55ae54363390 2024-07-31T15:07:56.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.366+0000 7ffb297fa640 1 -- 172.21.15.67:0/832639177 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb30004de0 con 0x55ae54363390 2024-07-31T15:07:56.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.366+0000 7ffb297fa640 1 -- 172.21.15.67:0/832639177 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7ffb38035420 con 0x55ae54363390 2024-07-31T15:07:56.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.366+0000 7ffb4e231740 10 client.19014 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:56.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.366+0000 7ffb4e231740 10 client.19014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:56.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.366+0000 7ffb4e231740 20 client.19014 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:56.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.366+0000 7ffb4e231740 1 --2- 172.21.15.67:0/832639177 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ae54384070 0x55ae547b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:56.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.366+0000 7ffb4e231740 4 client.19014 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:07:56.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.366+0000 7ffb4e231740 1 -- 172.21.15.67:0/832639177 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae546a1ab0 con 0x55ae54384070 2024-07-31T15:07:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.383+0000 7ffb45a7c640 1 --2- 172.21.15.67:0/832639177 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ae54384070 0x55ae547b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:56.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.399+0000 7ffb297fa640 10 client.19014 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:56.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.399+0000 7ffb45a7c640 1 --2- 172.21.15.67:0/832639177 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ae54384070 0x55ae547b2230 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61520ed91f5550b server_cookie=4e69ac863dd5c7c3 in_seq=0 out_seq=0 2024-07-31T15:07:56.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.400+0000 7ffb297fa640 1 -- 172.21.15.67:0/832639177 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ae546a1ab0 con 0x55ae54384070 2024-07-31T15:07:56.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.400+0000 7ffb297fa640 10 client.19014 handle_command_reply: tid=0 2024-07-31T15:07:56.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.400+0000 7ffb4e231740 1 client.19014 shutdown 2024-07-31T15:07:56.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.402+0000 7ffb4e231740 1 -- 172.21.15.67:0/832639177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae54363390 msgr2=0x55ae5473dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:56.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.402+0000 7ffb4e231740 1 --2- 172.21.15.67:0/832639177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae54363390 0x55ae5473dc20 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800eb40 tx=0x7ffb38004ad0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.402+0000 7ffb4e231740 1 -- 172.21.15.67:0/832639177 shutdown_connections 2024-07-31T15:07:56.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.402+0000 7ffb4e231740 1 --2- 172.21.15.67:0/832639177 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ae54384070 0x55ae547b2230 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.403+0000 7ffb4e231740 1 --2- 172.21.15.67:0/832639177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae54363390 0x55ae5473dc20 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.403+0000 7ffb4e231740 1 --2- 172.21.15.67:0/832639177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ae5435b1b0 0x55ae5473d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.403+0000 7ffb4e231740 1 --2- 172.21.15.67:0/832639177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae543429f0 0x55ae54741f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.403+0000 7ffb4e231740 1 -- 172.21.15.67:0/832639177 >> 172.21.15.67:0/832639177 conn(0x55ae54378460 msgr2=0x55ae54727690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:56.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.403+0000 7ffb297fa640 0 client.19014 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:56.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.403+0000 7ffb4e231740 1 -- 172.21.15.67:0/832639177 shutdown_connections 2024-07-31T15:07:56.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.403+0000 7ffb4e231740 1 -- 172.21.15.67:0/832639177 wait complete. 2024-07-31T15:07:56.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.403+0000 7ffb4e231740 20 client.19014 trim_cache size 0 max 16384 2024-07-31T15:07:56.405 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:07:56.406 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T15:07:56.406 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T15:07:56.406 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:07:56.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.404+0000 7ffb46a7e640 1 -- 172.21.15.67:0/3113922903 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffb2c0b3f10 msgr2=0x7ffb2c0b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:56.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.404+0000 7ffb46a7e640 1 --2- 172.21.15.67:0/3113922903 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffb2c0b3f10 0x7ffb2c0b6420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffb4011a770 tx=0x7ffb3800b040 comp rx=0 tx=0).stop 2024-07-31T15:07:56.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.404+0000 7ffb46a7e640 1 -- 172.21.15.67:0/3113922903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb40111790 msgr2=0x7ffb401132d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:56.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.404+0000 7ffb46a7e640 1 --2- 172.21.15.67:0/3113922903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb40111790 0x7ffb401132d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000d6d0 tx=0x7ffb3000db90 comp rx=0 tx=0).stop 2024-07-31T15:07:56.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.404+0000 7ffb46a7e640 1 -- 172.21.15.67:0/3113922903 shutdown_connections 2024-07-31T15:07:56.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.404+0000 7ffb46a7e640 1 --2- 172.21.15.67:0/3113922903 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ffb2c0b3f10 0x7ffb2c0b6420 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.404+0000 7ffb46a7e640 1 --2- 172.21.15.67:0/3113922903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4011c5d0 0x7ffb40113850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.404+0000 7ffb46a7e640 1 --2- 172.21.15.67:0/3113922903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb40111790 0x7ffb401132d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.404+0000 7ffb46a7e640 1 --2- 172.21.15.67:0/3113922903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb4010a330 0x7ffb4011a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:56.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.404+0000 7ffb46a7e640 1 -- 172.21.15.67:0/3113922903 >> 172.21.15.67:0/3113922903 conn(0x7ffb4006e420 msgr2=0x7ffb40073680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:56.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.405+0000 7ffb46a7e640 1 -- 172.21.15.67:0/3113922903 shutdown_connections 2024-07-31T15:07:56.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:07:56.405+0000 7ffb46a7e640 1 -- 172.21.15.67:0/3113922903 wait complete. 2024-07-31T15:07:56.422 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/fan_core.o 2024-07-31T15:07:56.879 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/mbcache.o 2024-07-31T15:07:56.960 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/intel/perfmon.o 2024-07-31T15:07:56.978 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/posix_acl.o 2024-07-31T15:07:57.119 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/coredump.o 2024-07-31T15:07:57.236 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/fec.o 2024-07-31T15:07:57.253 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/fan_attr.o 2024-07-31T15:07:57.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:57 smithi067 ceph-mon[32750]: pgmap v934: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-07-31T15:07:57.338 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/display/drm_hdcp_helper.o 2024-07-31T15:07:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:57 smithi073 ceph-mon[36714]: pgmap v934: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-07-31T15:07:57.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:57 smithi077 ceph-mon[37327]: pgmap v934: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-07-31T15:07:57.747 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T15:07:57.748 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:07:57.945 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpi_video.o 2024-07-31T15:07:57.955 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/video_detect.o 2024-07-31T15:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.975+0000 7f340f5bb640 1 -- 172.21.15.67:0/3170539973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f340810b970 msgr2=0x7f340810bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.975+0000 7f340f5bb640 1 --2- 172.21.15.67:0/3170539973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f340810b970 0x7f340810bd80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3400009ca0 tx=0x7f340002a8f0 comp rx=0 tx=0).stop 2024-07-31T15:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.975+0000 7f340f5bb640 1 -- 172.21.15.67:0/3170539973 shutdown_connections 2024-07-31T15:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.975+0000 7f340f5bb640 1 --2- 172.21.15.67:0/3170539973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3408114ea0 0x7f34081173b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.975+0000 7f340f5bb640 1 --2- 172.21.15.67:0/3170539973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f340810c2c0 0x7f3408114800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.975+0000 7f340f5bb640 1 --2- 172.21.15.67:0/3170539973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f340810b970 0x7f340810bd80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.975+0000 7f340f5bb640 1 -- 172.21.15.67:0/3170539973 >> 172.21.15.67:0/3170539973 conn(0x7f34080707c0 msgr2=0x7f3408070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.975+0000 7f340f5bb640 1 -- 172.21.15.67:0/3170539973 shutdown_connections 2024-07-31T15:07:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.975+0000 7f340f5bb640 1 -- 172.21.15.67:0/3170539973 wait complete. 2024-07-31T15:07:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.976+0000 7f340f5bb640 1 Processor -- start 2024-07-31T15:07:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.976+0000 7f340f5bb640 1 -- start start 2024-07-31T15:07:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.977+0000 7f340f5bb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f340810c2c0 0x7f340811bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.977+0000 7f340f5bb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3408114ea0 0x7f340811a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.977+0000 7f340f5bb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340811a990 0x7f340811ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.977+0000 7f340f5bb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f340810af40 con 0x7f340811a990 2024-07-31T15:07:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.977+0000 7f340f5bb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f340810adc0 con 0x7f3408114ea0 2024-07-31T15:07:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.977+0000 7f340f5bb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f340810b0c0 con 0x7f340810c2c0 2024-07-31T15:07:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.977+0000 7f340d357640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f340810c2c0 0x7f340811bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.978+0000 7f340db58640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340811a990 0x7f340811ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.978+0000 7f340db58640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340811a990 0x7f340811ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41150/0 (socket says 172.21.15.67:41150) 2024-07-31T15:07:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.978+0000 7f340db58640 1 -- 172.21.15.67:0/2560640653 learned_addr learned my addr 172.21.15.67:0/2560640653 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.978+0000 7f340cb56640 1 --2- 172.21.15.67:0/2560640653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3408114ea0 0x7f340811a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.979+0000 7f340db58640 1 -- 172.21.15.67:0/2560640653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f340810c2c0 msgr2=0x7f340811bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.979+0000 7f340db58640 1 --2- 172.21.15.67:0/2560640653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f340810c2c0 0x7f340811bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.979+0000 7f340db58640 1 -- 172.21.15.67:0/2560640653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3408114ea0 msgr2=0x7f340811a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.979+0000 7f340db58640 1 --2- 172.21.15.67:0/2560640653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3408114ea0 0x7f340811a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.979+0000 7f340db58640 1 -- 172.21.15.67:0/2560640653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34081a6400 con 0x7f340811a990 2024-07-31T15:07:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.979+0000 7f340db58640 1 --2- 172.21.15.67:0/2560640653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340811a990 0x7f340811ae70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f33f800d750 tx=0x7f33f800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.987+0000 7f33fe7fc640 1 -- 172.21.15.67:0/2560640653 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8017070 con 0x7f340811a990 2024-07-31T15:07:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.987+0000 7f340f5bb640 1 -- 172.21.15.67:0/2560640653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34081a66f0 con 0x7f340811a990 2024-07-31T15:07:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.987+0000 7f340f5bb640 1 -- 172.21.15.67:0/2560640653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34081a6c40 con 0x7f340811a990 2024-07-31T15:07:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.991+0000 7f341658e740 1 Processor -- start 2024-07-31T15:07:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.994+0000 7f341658e740 1 -- start start 2024-07-31T15:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.995+0000 7f341658e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5691191b0 0x55d5694bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.995+0000 7f341658e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f02f0 0x55d5694bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.995+0000 7f341658e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f84d0 0x55d5694bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.995+0000 7f341658e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d5694313b0 con 0x55d5690f02f0 2024-07-31T15:07:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.995+0000 7f341658e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d5694208b0 con 0x55d5690f84d0 2024-07-31T15:07:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.995+0000 7f341658e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d5694248f0 con 0x55d5691191b0 2024-07-31T15:07:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.995+0000 7f33fe7fc640 1 -- 172.21.15.67:0/2560640653 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f80054d0 con 0x7f340811a990 2024-07-31T15:07:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.995+0000 7f33fe7fc640 1 -- 172.21.15.67:0/2560640653 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8004c60 con 0x7f340811a990 2024-07-31T15:07:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.995+0000 7f33fe7fc640 1 -- 172.21.15.67:0/2560640653 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f33f8010460 con 0x7f340811a990 2024-07-31T15:07:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.995+0000 7f340d357640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f02f0 0x55d5694bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.995+0000 7f340cb56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5691191b0 0x55d5694bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.995+0000 7f340cb56640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f84d0 0x55d5694bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.995+0000 7f340d357640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f02f0 0x55d5694bc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41154/0 (socket says 172.21.15.67:41154) 2024-07-31T15:07:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.996+0000 7f340d357640 1 -- 172.21.15.67:0/3718596102 learned_addr learned my addr 172.21.15.67:0/3718596102 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.998+0000 7f33fe7fc640 1 --2- 172.21.15.67:0/2560640653 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33f40b3f80 0x7f33f40b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.998+0000 7f340db58640 1 --2- 172.21.15.67:0/2560640653 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33f40b3f80 0x7f33f40b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.999+0000 7f340d357640 1 -- 172.21.15.67:0/3718596102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5691191b0 msgr2=0x55d5694bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.999+0000 7f340d357640 1 --2- 172.21.15.67:0/3718596102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5691191b0 0x55d5694bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.999+0000 7f340d357640 1 -- 172.21.15.67:0/3718596102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f84d0 msgr2=0x55d5694bc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.999+0000 7f340d357640 1 --2- 172.21.15.67:0/3718596102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f84d0 0x55d5694bc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.999+0000 7f340d357640 1 -- 172.21.15.67:0/3718596102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5693c79a0 con 0x55d5690f02f0 2024-07-31T15:07:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.999+0000 7f340db58640 1 --2- 172.21.15.67:0/2560640653 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33f40b3f80 0x7f33f40b6490 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f33f800d720 tx=0x7f33f80e7f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.999+0000 7f33fe7fc640 1 -- 172.21.15.67:0/2560640653 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9379+0+0 (secure 0 0 0) 0x7f33f80e4c50 con 0x7f340811a990 2024-07-31T15:07:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:57.999+0000 7f340d357640 1 --2- 172.21.15.67:0/3718596102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f02f0 0x55d5694bc220 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f3400009ca0 tx=0x7f3400009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b83d91c6c697d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.000+0000 7f33f37fe640 1 -- 172.21.15.67:0/3718596102 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400003970 con 0x55d5690f02f0 2024-07-31T15:07:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.000+0000 7f33f37fe640 1 -- 172.21.15.67:0/3718596102 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3400036050 con 0x55d5690f02f0 2024-07-31T15:07:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.000+0000 7f33f37fe640 1 -- 172.21.15.67:0/3718596102 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400030850 con 0x55d5690f02f0 2024-07-31T15:07:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.001+0000 7f341658e740 1 -- 172.21.15.67:0/3718596102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f02f0 msgr2=0x55d5694bc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.001+0000 7f341658e740 1 --2- 172.21.15.67:0/3718596102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f02f0 0x55d5694bc220 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f3400009ca0 tx=0x7f3400009470 comp rx=0 tx=0).stop 2024-07-31T15:07:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.007+0000 7f341658e740 1 -- 172.21.15.67:0/3718596102 shutdown_connections 2024-07-31T15:07:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.007+0000 7f341658e740 1 --2- 172.21.15.67:0/3718596102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f84d0 0x55d5694bc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.007+0000 7f341658e740 1 --2- 172.21.15.67:0/3718596102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f02f0 0x55d5694bc220 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.007+0000 7f341658e740 1 --2- 172.21.15.67:0/3718596102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5691191b0 0x55d5694bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.007+0000 7f341658e740 1 -- 172.21.15.67:0/3718596102 >> 172.21.15.67:0/3718596102 conn(0x55d56910d300 msgr2=0x55d5694b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.007+0000 7f341658e740 1 -- 172.21.15.67:0/3718596102 shutdown_connections 2024-07-31T15:07:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.007+0000 7f341658e740 1 -- 172.21.15.67:0/3718596102 wait complete. 2024-07-31T15:07:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.008+0000 7f341658e740 1 -- start start 2024-07-31T15:07:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.008+0000 7f341658e740 1 Processor -- start 2024-07-31T15:07:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.009+0000 7f341658e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5690f02f0 0x55d5694c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.009+0000 7f341658e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f84d0 0x55d5694c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.009+0000 7f341658e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5691191b0 0x55d5694d3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.009+0000 7f341658e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d56935aab0 con 0x55d5691191b0 2024-07-31T15:07:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.009+0000 7f341658e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d5693a8cf0 con 0x55d5690f84d0 2024-07-31T15:07:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.009+0000 7f341658e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d5694ca400 con 0x55d5690f02f0 2024-07-31T15:07:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.009+0000 7f341658e740 10 client.0 _ll_register_callbacks cb 0x55d56941a9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.010+0000 7f340d357640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f84d0 0x55d5694c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.010+0000 7f340d357640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f84d0 0x55d5694c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56014/0 (socket says 172.21.15.67:56014) 2024-07-31T15:07:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.010+0000 7f340d357640 1 -- 172.21.15.67:0/973338590 learned_addr learned my addr 172.21.15.67:0/973338590 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.010+0000 7f340cb56640 1 --2- 172.21.15.67:0/973338590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5691191b0 0x55d5694d3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.010+0000 7f340d357640 1 -- 172.21.15.67:0/973338590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5690f02f0 msgr2=0x55d5694c7850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:07:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.010+0000 7f340d357640 1 --2- 172.21.15.67:0/973338590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5690f02f0 0x55d5694c7850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.010+0000 7f340d357640 1 -- 172.21.15.67:0/973338590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5691191b0 msgr2=0x55d5694d3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.010+0000 7f340d357640 1 --2- 172.21.15.67:0/973338590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5691191b0 0x55d5694d3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.010+0000 7f340d357640 1 -- 172.21.15.67:0/973338590 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d569184790 con 0x55d5690f84d0 2024-07-31T15:07:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.010+0000 7f340cb56640 1 --2- 172.21.15.67:0/973338590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5691191b0 0x55d5694d3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:07:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.011+0000 7f340d357640 1 --2- 172.21.15.67:0/973338590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f84d0 0x55d5694c7d90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f340002aee0 tx=0x7f340002a8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1591a51e4398ae18 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.023+0000 7f33f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:07:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.023+0000 7f33f0ff9640 1 -- 172.21.15.67:0/973338590 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340004a070 con 0x55d5690f84d0 2024-07-31T15:07:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.023+0000 7f341658e740 1 -- 172.21.15.67:0/973338590 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3400036050 con 0x55d5690f84d0 2024-07-31T15:07:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.023+0000 7f33f0ff9640 1 -- 172.21.15.67:0/973338590 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9379+0+0 (secure 0 0 0) 0x7f340002f5a0 con 0x55d5690f84d0 2024-07-31T15:07:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.024+0000 7f33f0ff9640 1 -- 172.21.15.67:0/973338590 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f3400058050 con 0x55d5690f84d0 2024-07-31T15:07:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.024+0000 7f341658e740 10 client.28562 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.024+0000 7f341658e740 1 -- 172.21.15.67:0/973338590 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55d5691fd620 con 0x55d5690f84d0 2024-07-31T15:07:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.024+0000 7f33f0ff9640 1 -- 172.21.15.67:0/973338590 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3400036050 con 0x55d5690f84d0 2024-07-31T15:07:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.024+0000 7f33f0ff9640 1 -- 172.21.15.67:0/973338590 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f34000449b0 con 0x55d5690f84d0 2024-07-31T15:07:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.024+0000 7f341658e740 10 client.28562 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.024+0000 7f341658e740 10 client.28562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.024+0000 7f341658e740 20 client.28562 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.024+0000 7f341658e740 1 --2- 172.21.15.67:0/973338590 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5690d7d80 0x55d5694d95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.024+0000 7f341658e740 4 client.28562 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:07:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.024+0000 7f341658e740 1 -- 172.21.15.67:0/973338590 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d569433300 con 0x55d5690d7d80 2024-07-31T15:07:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.025+0000 7f340cb56640 1 --2- 172.21.15.67:0/973338590 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5690d7d80 0x55d5694d95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.025+0000 7f33f0ff9640 10 client.28562 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.025+0000 7f340cb56640 1 --2- 172.21.15.67:0/973338590 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5690d7d80 0x55d5694d95e0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=380208de3b212891 server_cookie=7f665a5633d5c29 in_seq=0 out_seq=0 2024-07-31T15:07:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.026+0000 7f33f0ff9640 1 -- 172.21.15.67:0/973338590 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d569433300 con 0x55d5690d7d80 2024-07-31T15:07:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.026+0000 7f33f0ff9640 10 client.28562 handle_command_reply: tid=0 2024-07-31T15:07:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.026+0000 7f341658e740 1 client.28562 shutdown 2024-07-31T15:07:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.027+0000 7f341658e740 1 -- 172.21.15.67:0/973338590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f84d0 msgr2=0x55d5694c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.027+0000 7f341658e740 1 --2- 172.21.15.67:0/973338590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f84d0 0x55d5694c7d90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f340002aee0 tx=0x7f340002a8f0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.027+0000 7f341658e740 1 -- 172.21.15.67:0/973338590 shutdown_connections 2024-07-31T15:07:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.027+0000 7f341658e740 1 --2- 172.21.15.67:0/973338590 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5690d7d80 0x55d5694d95e0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.027+0000 7f341658e740 1 --2- 172.21.15.67:0/973338590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5691191b0 0x55d5694d3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.027+0000 7f341658e740 1 --2- 172.21.15.67:0/973338590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f84d0 0x55d5694c7d90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.027+0000 7f341658e740 1 --2- 172.21.15.67:0/973338590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5690f02f0 0x55d5694c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.027+0000 7f341658e740 1 -- 172.21.15.67:0/973338590 >> 172.21.15.67:0/973338590 conn(0x55d56910d300 msgr2=0x55d5694bcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.027+0000 7f33f0ff9640 0 client.28562 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.027+0000 7f341658e740 1 -- 172.21.15.67:0/973338590 shutdown_connections 2024-07-31T15:07:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.027+0000 7f341658e740 1 -- 172.21.15.67:0/973338590 wait complete. 2024-07-31T15:07:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.027+0000 7f341658e740 20 client.28562 trim_cache size 0 max 16384 2024-07-31T15:07:58.075 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/ttm/ttm_tt.o 2024-07-31T15:07:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.081+0000 7f341658e740 1 Processor -- start 2024-07-31T15:07:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.094+0000 7f341658e740 1 -- start start 2024-07-31T15:07:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.094+0000 7f341658e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5690f02f0 0x55d5694bd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.094+0000 7f341658e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f84d0 0x55d5694d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f341658e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5691191b0 0x55d5694d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f340cb56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5690f02f0 0x55d5694bd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f340d357640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f84d0 0x55d5694d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f340d357640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f84d0 0x55d5694d36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41166/0 (socket says 172.21.15.67:41166) 2024-07-31T15:07:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f340cb56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5690f02f0 0x55d5694bd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:41494/0 (socket says 172.21.15.67:41494) 2024-07-31T15:07:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f340d357640 1 -- 172.21.15.67:0/790654989 learned_addr learned my addr 172.21.15.67:0/790654989 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f341658e740 1 -- 172.21.15.67:0/790654989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec00ff30 con 0x55d5690f84d0 2024-07-31T15:07:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f341658e740 1 -- 172.21.15.67:0/790654989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec011a70 con 0x55d5691191b0 2024-07-31T15:07:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f341658e740 1 -- 172.21.15.67:0/790654989 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec010600 con 0x55d5690f02f0 2024-07-31T15:07:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f340cb56640 1 --2- 172.21.15.67:0/790654989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5691191b0 0x55d5694d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f340cb56640 1 -- 172.21.15.67:0/790654989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5691191b0 msgr2=0x55d5694d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f340cb56640 1 --2- 172.21.15.67:0/790654989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5691191b0 0x55d5694d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f340cb56640 1 -- 172.21.15.67:0/790654989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f84d0 msgr2=0x55d5694d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f340cb56640 1 --2- 172.21.15.67:0/790654989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f84d0 0x55d5694d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f340cb56640 1 -- 172.21.15.67:0/790654989 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5694c2df0 con 0x55d5690f02f0 2024-07-31T15:07:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.095+0000 7f340d357640 1 --2- 172.21.15.67:0/790654989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f84d0 0x55d5694d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:07:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.096+0000 7f340cb56640 1 --2- 172.21.15.67:0/790654989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5690f02f0 0x55d5694bd420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55d5694209d0 tx=0x7f3404000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70a5b35cdab02bad server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.109+0000 7f33f2ffd640 1 -- 172.21.15.67:0/790654989 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404011b90 con 0x55d5690f02f0 2024-07-31T15:07:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.109+0000 7f33f2ffd640 1 -- 172.21.15.67:0/790654989 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3404011d30 con 0x55d5690f02f0 2024-07-31T15:07:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.110+0000 7f341658e740 1 -- 172.21.15.67:0/790654989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5690f02f0 msgr2=0x55d5694bd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.110+0000 7f341658e740 1 --2- 172.21.15.67:0/790654989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5690f02f0 0x55d5694bd420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55d5694209d0 tx=0x7f3404000bc0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.110+0000 7f341658e740 1 -- 172.21.15.67:0/790654989 shutdown_connections 2024-07-31T15:07:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.110+0000 7f341658e740 1 --2- 172.21.15.67:0/790654989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5691191b0 0x55d5694d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.110+0000 7f341658e740 1 --2- 172.21.15.67:0/790654989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f84d0 0x55d5694d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.110+0000 7f341658e740 1 --2- 172.21.15.67:0/790654989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5690f02f0 0x55d5694bd420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.110+0000 7f341658e740 1 -- 172.21.15.67:0/790654989 >> 172.21.15.67:0/790654989 conn(0x55d56910d300 msgr2=0x55d5694bcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.111+0000 7f341658e740 1 -- 172.21.15.67:0/790654989 shutdown_connections 2024-07-31T15:07:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.111+0000 7f341658e740 1 -- 172.21.15.67:0/790654989 wait complete. 2024-07-31T15:07:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.111+0000 7f341658e740 1 -- start start 2024-07-31T15:07:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.111+0000 7f341658e740 1 Processor -- start 2024-07-31T15:07:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.112+0000 7f341658e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f02f0 0x55d5694dae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.112+0000 7f341658e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f84d0 0x55d5694db400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.112+0000 7f341658e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5691191b0 0x55d569538b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.112+0000 7f341658e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec0113a0 con 0x55d5690f84d0 2024-07-31T15:07:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.112+0000 7f341658e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec010cd0 con 0x55d5690f02f0 2024-07-31T15:07:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.112+0000 7f341658e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec00df30 con 0x55d5691191b0 2024-07-31T15:07:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.112+0000 7f341658e740 10 client.0 _ll_register_callbacks cb 0x7f3400009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:07:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.113+0000 7f340d357640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f84d0 0x55d5694db400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.113+0000 7f340d357640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f84d0 0x55d5694db400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41176/0 (socket says 172.21.15.67:41176) 2024-07-31T15:07:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.113+0000 7f340d357640 1 -- 172.21.15.67:0/803965525 learned_addr learned my addr 172.21.15.67:0/803965525 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:07:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.113+0000 7f340cb56640 1 --2- 172.21.15.67:0/803965525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f02f0 0x55d5694dae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.113+0000 7f340cb56640 1 --2- 172.21.15.67:0/803965525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5691191b0 0x55d569538b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.114+0000 7f340d357640 1 -- 172.21.15.67:0/803965525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5691191b0 msgr2=0x55d569538b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.114+0000 7f340d357640 1 --2- 172.21.15.67:0/803965525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5691191b0 0x55d569538b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.114+0000 7f340d357640 1 -- 172.21.15.67:0/803965525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f02f0 msgr2=0x55d5694dae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:07:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.114+0000 7f340d357640 1 --2- 172.21.15.67:0/803965525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f02f0 0x55d5694dae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.114+0000 7f340d357640 1 -- 172.21.15.67:0/803965525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5695391d0 con 0x55d5690f84d0 2024-07-31T15:07:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.114+0000 7f340d357640 1 --2- 172.21.15.67:0/803965525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f84d0 0x55d5694db400 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f3400009bf0 tx=0x7f3400030a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3074025b5ebe35f3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:07:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.121+0000 7f33f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:07:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.121+0000 7f33f0ff9640 1 -- 172.21.15.67:0/803965525 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400031960 con 0x55d5690f84d0 2024-07-31T15:07:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.121+0000 7f33f0ff9640 1 -- 172.21.15.67:0/803965525 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9379+0+0 (secure 0 0 0) 0x7f3400004800 con 0x55d5690f84d0 2024-07-31T15:07:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.121+0000 7f341658e740 1 -- 172.21.15.67:0/803965525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3404011d30 con 0x55d5690f84d0 2024-07-31T15:07:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.129+0000 7f33f0ff9640 1 -- 172.21.15.67:0/803965525 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f3400065050 con 0x55d5690f84d0 2024-07-31T15:07:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.129+0000 7f341658e740 10 client.19032 fetch_fsmap learned FSMap version 65 2024-07-31T15:07:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.129+0000 7f341658e740 1 -- 172.21.15.67:0/803965525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55d569540500 con 0x55d5690f84d0 2024-07-31T15:07:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.130+0000 7f33f0ff9640 1 -- 172.21.15.67:0/803965525 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3404011d30 con 0x55d5690f84d0 2024-07-31T15:07:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.130+0000 7f33f0ff9640 1 -- 172.21.15.67:0/803965525 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f340002f5a0 con 0x55d5690f84d0 2024-07-31T15:07:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.131+0000 7f341658e740 10 client.19032 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:07:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.131+0000 7f341658e740 10 client.19032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:07:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.131+0000 7f341658e740 20 client.19032 populate_metadata read hostname 'smithi067' 2024-07-31T15:07:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.131+0000 7f341658e740 1 --2- 172.21.15.67:0/803965525 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5690ad1c0 0x55d569542ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:07:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.131+0000 7f341658e740 4 client.19032 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:07:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.131+0000 7f341658e740 1 -- 172.21.15.67:0/803965525 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d56942eda0 con 0x55d5690ad1c0 2024-07-31T15:07:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.132+0000 7f340cb56640 1 --2- 172.21.15.67:0/803965525 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5690ad1c0 0x55d569542ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:07:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.132+0000 7f33f0ff9640 10 client.19032 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:58.132+0000 7f340cb56640 1 --2- 172.21.15.67:0/803965525 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5690ad1c0 0x55d569542ab0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc7853272d58607f server_cookie=fbd5eaf0c313111f in_seq=0 out_seq=0 2024-07-31T15:07:58.219 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/display/drm_hdmi_helper.o 2024-07-31T15:07:58.307 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/iommu/intel/built-in.a 2024-07-31T15:07:58.352 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/drop_caches.o 2024-07-31T15:07:58.527 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/eeprom.o 2024-07-31T15:07:58.638 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/sysctls.o 2024-07-31T15:07:58.724 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:07:58.979 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fhandle.o 2024-07-31T15:07:59.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:07:59 smithi067 ceph-mon[32750]: pgmap v935: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-07-31T15:07:59.226 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/stats.o 2024-07-31T15:07:59.269 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/iommu/arm/arm-smmu/built-in.a 2024-07-31T15:07:59.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.288+0000 7f33f0ff9640 1 -- 172.21.15.67:0/803965525 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d56942eda0 con 0x55d5690ad1c0 2024-07-31T15:07:59.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.288+0000 7f33f0ff9640 10 client.19032 handle_command_reply: tid=0 2024-07-31T15:07:59.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.288+0000 7f341658e740 1 client.19032 shutdown 2024-07-31T15:07:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.289+0000 7f341658e740 1 -- 172.21.15.67:0/803965525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f84d0 msgr2=0x55d5694db400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.289+0000 7f341658e740 1 --2- 172.21.15.67:0/803965525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f84d0 0x55d5694db400 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f3400009bf0 tx=0x7f3400030a60 comp rx=0 tx=0).stop 2024-07-31T15:07:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.290+0000 7f341658e740 1 -- 172.21.15.67:0/803965525 shutdown_connections 2024-07-31T15:07:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.290+0000 7f341658e740 1 --2- 172.21.15.67:0/803965525 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5690ad1c0 0x55d569542ab0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.290+0000 7f341658e740 1 --2- 172.21.15.67:0/803965525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5691191b0 0x55d569538b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.290+0000 7f341658e740 1 --2- 172.21.15.67:0/803965525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5690f84d0 0x55d5694db400 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.290+0000 7f341658e740 1 --2- 172.21.15.67:0/803965525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5690f02f0 0x55d5694dae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.290+0000 7f341658e740 1 -- 172.21.15.67:0/803965525 >> 172.21.15.67:0/803965525 conn(0x55d56910d300 msgr2=0x55d5694c5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.290+0000 7f33f0ff9640 0 client.19032 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:07:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.290+0000 7f341658e740 1 -- 172.21.15.67:0/803965525 shutdown_connections 2024-07-31T15:07:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.290+0000 7f341658e740 1 -- 172.21.15.67:0/803965525 wait complete. 2024-07-31T15:07:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.290+0000 7f341658e740 20 client.19032 trim_cache size 0 max 16384 2024-07-31T15:07:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:07:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:07:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:07:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:07:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.292+0000 7f340f5bb640 1 -- 172.21.15.67:0/2560640653 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33f40b3f80 msgr2=0x7f33f40b6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.292+0000 7f340f5bb640 1 --2- 172.21.15.67:0/2560640653 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33f40b3f80 0x7f33f40b6490 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f33f800d720 tx=0x7f33f80e7f90 comp rx=0 tx=0).stop 2024-07-31T15:07:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.292+0000 7f340f5bb640 1 -- 172.21.15.67:0/2560640653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340811a990 msgr2=0x7f340811ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:07:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.292+0000 7f340f5bb640 1 --2- 172.21.15.67:0/2560640653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340811a990 0x7f340811ae70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f33f800d750 tx=0x7f33f800dc10 comp rx=0 tx=0).stop 2024-07-31T15:07:59.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.293+0000 7f340f5bb640 1 -- 172.21.15.67:0/2560640653 shutdown_connections 2024-07-31T15:07:59.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.293+0000 7f340f5bb640 1 --2- 172.21.15.67:0/2560640653 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33f40b3f80 0x7f33f40b6490 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:59.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.293+0000 7f340f5bb640 1 --2- 172.21.15.67:0/2560640653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340811a990 0x7f340811ae70 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:59.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.293+0000 7f340f5bb640 1 --2- 172.21.15.67:0/2560640653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3408114ea0 0x7f340811a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:59.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.293+0000 7f340f5bb640 1 --2- 172.21.15.67:0/2560640653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f340810c2c0 0x7f340811bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:07:59.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.293+0000 7f340f5bb640 1 -- 172.21.15.67:0/2560640653 >> 172.21.15.67:0/2560640653 conn(0x7f34080707c0 msgr2=0x7f340810c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:07:59.295 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.294+0000 7f340f5bb640 1 -- 172.21.15.67:0/2560640653 shutdown_connections 2024-07-31T15:07:59.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:07:59.294+0000 7f340f5bb640 1 -- 172.21.15.67:0/2560640653 wait complete. 2024-07-31T15:07:59.310 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/iommu/arm/arm-smmu-v3/built-in.a 2024-07-31T15:07:59.326 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/iommu/arm/built-in.a 2024-07-31T15:07:59.388 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/iommu/iommufd/built-in.a 2024-07-31T15:07:59.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:07:59 smithi073 ceph-mon[36714]: pgmap v935: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-07-31T15:07:59.409 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/iommu.o 2024-07-31T15:07:59.412 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/phc_vclocks.o 2024-07-31T15:07:59.413 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/display/drm_scdc_helper.o 2024-07-31T15:07:59.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:07:59 smithi077 ceph-mon[37327]: pgmap v935: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-07-31T15:07:59.528 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/ttm/ttm_bo.o 2024-07-31T15:07:59.539 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/processor_driver.o 2024-07-31T15:07:59.782 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/built-in.a 2024-07-31T15:08:00.060 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/processor_thermal.o 2024-07-31T15:08:00.083 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/iommu-traces.o 2024-07-31T15:08:00.127 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/display/built-in.a 2024-07-31T15:08:00.175 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/mm.o 2024-07-31T15:08:00.241 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/iommu-sysfs.o 2024-07-31T15:08:00.246 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/ttm/ttm_bo_util.o 2024-07-31T15:08:00.373 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/dma-iommu.o 2024-07-31T15:08:00.462 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_driver.o 2024-07-31T15:08:00.623 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/processor_idle.o 2024-07-31T15:08:00.884 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/ttm/ttm_bo_vm.o 2024-07-31T15:08:00.942 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/processor_throttling.o 2024-07-31T15:08:01.033 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/io-pgtable.o 2024-07-31T15:08:01.102 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/module.o 2024-07-31T15:08:01.147 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/renesas/rcar-du/built-in.a 2024-07-31T15:08:01.169 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/renesas/built-in.a 2024-07-31T15:08:01.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.185+0000 7feaf8b7d640 1 -- 172.21.15.67:0/2790475113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf411c660 msgr2=0x7feaf411eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:01.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.185+0000 7feaf8b7d640 1 --2- 172.21.15.67:0/2790475113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf411c660 0x7feaf411eb70 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7feae4009900 tx=0x7feae402a400 comp rx=0 tx=0).stop 2024-07-31T15:08:01.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.194+0000 7feaf8b7d640 1 -- 172.21.15.67:0/2790475113 shutdown_connections 2024-07-31T15:08:01.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.194+0000 7feaf8b7d640 1 --2- 172.21.15.67:0/2790475113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf411c660 0x7feaf411eb70 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:01.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.194+0000 7feaf8b7d640 1 --2- 172.21.15.67:0/2790475113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf410a410 0x7feaf410a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:01.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.194+0000 7feaf8b7d640 1 --2- 172.21.15.67:0/2790475113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf4111970 0x7feaf4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:01.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.194+0000 7feaf8b7d640 1 -- 172.21.15.67:0/2790475113 >> 172.21.15.67:0/2790475113 conn(0x7feaf40708f0 msgr2=0x7feaf4070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:01.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.196+0000 7feaf8b7d640 1 -- 172.21.15.67:0/2790475113 shutdown_connections 2024-07-31T15:08:01.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.198+0000 7feaf8b7d640 1 -- 172.21.15.67:0/2790475113 wait complete. 2024-07-31T15:08:01.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.199+0000 7feaf8b7d640 1 Processor -- start 2024-07-31T15:08:01.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.199+0000 7feaf8b7d640 1 -- start start 2024-07-31T15:08:01.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.200+0000 7feaf8b7d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf410a410 0x7feaf4119e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:01.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.200+0000 7feaf8b7d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf4111970 0x7feaf4113110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:01.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.200+0000 7feaf8b7d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf411c660 0x7feaf41136b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:01.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.200+0000 7feaf8b7d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feaf41222d0 con 0x7feaf4111970 2024-07-31T15:08:01.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.200+0000 7feaf8b7d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feaf4122150 con 0x7feaf411c660 2024-07-31T15:08:01.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.200+0000 7feaf8b7d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feaf4122450 con 0x7feaf410a410 2024-07-31T15:08:01.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.200+0000 7feaf1d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf4111970 0x7feaf4113110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:01.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.200+0000 7feaf1d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf4111970 0x7feaf4113110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37898/0 (socket says 172.21.15.67:37898) 2024-07-31T15:08:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.200+0000 7feaf1d9b640 1 -- 172.21.15.67:0/3272610911 learned_addr learned my addr 172.21.15.67:0/3272610911 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.204+0000 7feaf259c640 1 --2- 172.21.15.67:0/3272610911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf410a410 0x7feaf4119e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.204+0000 7feaf2d9d640 1 --2- 172.21.15.67:0/3272610911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf411c660 0x7feaf41136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.205+0000 7feaf1d9b640 1 -- 172.21.15.67:0/3272610911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf410a410 msgr2=0x7feaf4119e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.205+0000 7feaf1d9b640 1 --2- 172.21.15.67:0/3272610911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf410a410 0x7feaf4119e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.205+0000 7feaf1d9b640 1 -- 172.21.15.67:0/3272610911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf411c660 msgr2=0x7feaf41136b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.205+0000 7feaf1d9b640 1 --2- 172.21.15.67:0/3272610911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf411c660 0x7feaf41136b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.205+0000 7feaf1d9b640 1 -- 172.21.15.67:0/3272610911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf4113ed0 con 0x7feaf4111970 2024-07-31T15:08:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.205+0000 7feaf1d9b640 1 --2- 172.21.15.67:0/3272610911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf4111970 0x7feaf4113110 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7feaec00eb90 tx=0x7feaec00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.206+0000 7feae37fe640 1 -- 172.21.15.67:0/3272610911 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec003b70 con 0x7feaf4111970 2024-07-31T15:08:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.206+0000 7feaf8b7d640 1 -- 172.21.15.67:0/3272610911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf4118b70 con 0x7feaf4111970 2024-07-31T15:08:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.206+0000 7feaf8b7d640 1 -- 172.21.15.67:0/3272610911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf4119080 con 0x7feaf4111970 2024-07-31T15:08:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.206+0000 7feae37fe640 1 -- 172.21.15.67:0/3272610911 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaec00ee90 con 0x7feaf4111970 2024-07-31T15:08:01.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.206+0000 7feae37fe640 1 -- 172.21.15.67:0/3272610911 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec010720 con 0x7feaf4111970 2024-07-31T15:08:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.213+0000 7feae37fe640 1 -- 172.21.15.67:0/3272610911 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7feaec0109c0 con 0x7feaf4111970 2024-07-31T15:08:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.220+0000 7feaf8b7d640 1 -- 172.21.15.67:0/3272610911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead40052c0 con 0x7feaf4111970 2024-07-31T15:08:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.222+0000 7feae37fe640 1 --2- 172.21.15.67:0/3272610911 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feadc0b3f50 0x7feadc0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.223+0000 7feae37fe640 1 -- 172.21.15.67:0/3272610911 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9379+0+0 (secure 0 0 0) 0x7feaec0e5800 con 0x7feaf4111970 2024-07-31T15:08:01.237 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:01 smithi067 ceph-mon[32750]: pgmap v936: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-07-31T15:08:01.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.230+0000 7feaf259c640 1 --2- 172.21.15.67:0/3272610911 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feadc0b3f50 0x7feadc0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:01.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.231+0000 7feaf259c640 1 --2- 172.21.15.67:0/3272610911 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feadc0b3f50 0x7feadc0b6460 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7feae8009b50 tx=0x7feae8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:01.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.238+0000 7feae37fe640 1 -- 172.21.15.67:0/3272610911 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7feaec0ac750 con 0x7feaf4111970 2024-07-31T15:08:01.269 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/omapdrm/built-in.a 2024-07-31T15:08:01.338 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/connector/cn_queue.o 2024-07-31T15:08:01.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:01 smithi073 ceph-mon[36714]: pgmap v936: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-07-31T15:08:01.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:01 smithi077 ceph-mon[37327]: pgmap v936: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-07-31T15:08:01.695 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/ttm/ttm_module.o 2024-07-31T15:08:01.720 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/connector/connector.o 2024-07-31T15:08:01.722 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/iommu/iova.o 2024-07-31T15:08:01.729 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/processor_perflib.o 2024-07-31T15:08:01.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.730+0000 7feaf8b7d640 1 -- 172.21.15.67:0/3272610911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fead4005880 con 0x7feaf4111970 2024-07-31T15:08:01.734 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:08:01.734 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"btime":"2024-07-31T15:06:04:323642+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:06:04.323492+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28214":{"gid":28214,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1516494761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1516494761},{"type":"v1","addr":"172.21.15.73:6835","nonce":1516494761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:08:01.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.732+0000 7feae37fe640 1 -- 172.21.15.67:0/3272610911 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11962 (secure 0 0 0) 0x7feaec0ac050 con 0x7feaf4111970 2024-07-31T15:08:01.735 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-07-31T15:08:01.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.736+0000 7feaf8b7d640 1 -- 172.21.15.67:0/3272610911 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feadc0b3f50 msgr2=0x7feadc0b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:01.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.736+0000 7feaf8b7d640 1 --2- 172.21.15.67:0/3272610911 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feadc0b3f50 0x7feadc0b6460 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7feae8009b50 tx=0x7feae8005f90 comp rx=0 tx=0).stop 2024-07-31T15:08:01.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.736+0000 7feaf8b7d640 1 -- 172.21.15.67:0/3272610911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf4111970 msgr2=0x7feaf4113110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:01.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.736+0000 7feaf8b7d640 1 --2- 172.21.15.67:0/3272610911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf4111970 0x7feaf4113110 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7feaec00eb90 tx=0x7feaec00c6a0 comp rx=0 tx=0).stop 2024-07-31T15:08:01.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.736+0000 7feaf8b7d640 1 -- 172.21.15.67:0/3272610911 shutdown_connections 2024-07-31T15:08:01.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.736+0000 7feaf8b7d640 1 --2- 172.21.15.67:0/3272610911 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feadc0b3f50 0x7feadc0b6460 secure :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7feae8009b50 tx=0x7feae8005f90 comp rx=0 tx=0).stop 2024-07-31T15:08:01.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.736+0000 7feaf8b7d640 1 --2- 172.21.15.67:0/3272610911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf411c660 0x7feaf41136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:01.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.736+0000 7feaf8b7d640 1 --2- 172.21.15.67:0/3272610911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf4111970 0x7feaf4113110 secure :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7feaec00eb90 tx=0x7feaec00c6a0 comp rx=0 tx=0).stop 2024-07-31T15:08:01.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.736+0000 7feaf8b7d640 1 --2- 172.21.15.67:0/3272610911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf410a410 0x7feaf4119e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:01.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.736+0000 7feaf8b7d640 1 -- 172.21.15.67:0/3272610911 >> 172.21.15.67:0/3272610911 conn(0x7feaf40708f0 msgr2=0x7feaf411ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:01.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.739+0000 7feaf8b7d640 1 -- 172.21.15.67:0/3272610911 shutdown_connections 2024-07-31T15:08:01.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:01.742+0000 7feaf8b7d640 1 -- 172.21.15.67:0/3272610911 wait complete. 2024-07-31T15:08:02.020 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/connector/cn_proc.o 2024-07-31T15:08:02.023 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/pse-pd.o 2024-07-31T15:08:02.117 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/ttm/ttm_execbuf_util.o 2024-07-31T15:08:02.276 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/ttm/ttm_range_manager.o 2024-07-31T15:08:02.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:02 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3272610911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:08:02.374 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/container.o 2024-07-31T15:08:02.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:02 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3272610911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:08:02.438 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/iommu/built-in.a 2024-07-31T15:08:02.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:02 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3272610911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:08:02.546 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_drm_client.o 2024-07-31T15:08:02.552 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_config.o 2024-07-31T15:08:02.572 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/power/sysfs.o 2024-07-31T15:08:02.754 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/firmware_loader/builtin/main.o 2024-07-31T15:08:02.804 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/thermal.o 2024-07-31T15:08:02.833 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_getparam.o 2024-07-31T15:08:02.853 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/ttm/ttm_resource.o 2024-07-31T15:08:02.882 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ethtool/plca.o 2024-07-31T15:08:02.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T15:08:02.924 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/connector/built-in.a 2024-07-31T15:08:02.998 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/ttm/ttm_pool.o 2024-07-31T15:08:03.095 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/base/firmware_loader/builtin/built-in.a 2024-07-31T15:08:03.097 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/power/generic_ops.o 2024-07-31T15:08:03.168 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/firmware_loader/main.o 2024-07-31T15:08:03.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:03 smithi067 ceph-mon[32750]: pgmap v937: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 175 op/s 2024-07-31T15:08:03.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:03 smithi073 ceph-mon[36714]: pgmap v937: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 175 op/s 2024-07-31T15:08:03.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:03 smithi077 ceph-mon[37327]: pgmap v937: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 175 op/s 2024-07-31T15:08:03.489 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/ttm/ttm_device.o 2024-07-31T15:08:03.550 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:08:03.555 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/acpi_memhotplug.o 2024-07-31T15:08:03.564 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/power/common.o 2024-07-31T15:08:03.648 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/power/qos.o 2024-07-31T15:08:03.684 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_ioctl.o 2024-07-31T15:08:03.762 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/ethtool/built-in.a 2024-07-31T15:08:03.842 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_irq.o 2024-07-31T15:08:03.855 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/core.o 2024-07-31T15:08:04.002 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/ioapic.o 2024-07-31T15:08:04.072 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/power/runtime.o 2024-07-31T15:08:04.245 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/ttm/ttm_sys_manager.o 2024-07-31T15:08:04.267 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/base/firmware_loader/built-in.a 2024-07-31T15:08:04.434 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/battery.o 2024-07-31T15:08:04.634 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/power/wakeirq.o 2024-07-31T15:08:04.749 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/ttm/ttm_agp_backend.o 2024-07-31T15:08:05.094 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/bgrt.o 2024-07-31T15:08:05.281 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/netfilter/nf_defrag_ipv4.o 2024-07-31T15:08:05.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:05 smithi073 ceph-mon[36714]: pgmap v938: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 232 op/s 2024-07-31T15:08:05.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:05 smithi073 ceph-mon[36714]: osdmap e119: 12 total, 12 up, 12 in 2024-07-31T15:08:05.406 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/route.o 2024-07-31T15:08:05.449 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:05 smithi067 ceph-mon[32750]: pgmap v938: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 232 op/s 2024-07-31T15:08:05.449 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:05 smithi067 ceph-mon[32750]: osdmap e119: 12 total, 12 up, 12 in 2024-07-31T15:08:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:05 smithi077 ceph-mon[37327]: pgmap v938: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 232 op/s 2024-07-31T15:08:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:05 smithi077 ceph-mon[37327]: osdmap e119: 12 total, 12 up, 12 in 2024-07-31T15:08:05.603 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/power/main.o 2024-07-31T15:08:05.814 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/cppc_acpi.o 2024-07-31T15:08:05.842 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_log.o 2024-07-31T15:08:05.876 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/ttm/built-in.a 2024-07-31T15:08:06.057 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/acpi/spcr.o 2024-07-31T15:08:06.083 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_mitigations.o 2024-07-31T15:08:06.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.109+0000 7f64f224c640 1 -- 172.21.15.67:0/1630923375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64ec07ba50 msgr2=0x7f64ec07be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:06.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.109+0000 7f64f224c640 1 --2- 172.21.15.67:0/1630923375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64ec07ba50 0x7f64ec07be60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009900 tx=0x7f64dc02a420 comp rx=0 tx=0).stop 2024-07-31T15:08:06.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.110+0000 7f64f224c640 1 -- 172.21.15.67:0/1630923375 shutdown_connections 2024-07-31T15:08:06.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.110+0000 7f64f224c640 1 --2- 172.21.15.67:0/1630923375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64ec081ce0 0x7f64ec082500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.110+0000 7f64f224c640 1 --2- 172.21.15.67:0/1630923375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64ec081390 0x7f64ec0817a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.110+0000 7f64f224c640 1 --2- 172.21.15.67:0/1630923375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64ec07ba50 0x7f64ec07be60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.111+0000 7f64f224c640 1 -- 172.21.15.67:0/1630923375 >> 172.21.15.67:0/1630923375 conn(0x7f64ec077b20 msgr2=0x7f64ec107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:06.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.112+0000 7f64f224c640 1 -- 172.21.15.67:0/1630923375 shutdown_connections 2024-07-31T15:08:06.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.112+0000 7f64f224c640 1 -- 172.21.15.67:0/1630923375 wait complete. 2024-07-31T15:08:06.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.113+0000 7f64f224c640 1 Processor -- start 2024-07-31T15:08:06.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.113+0000 7f64f224c640 1 -- start start 2024-07-31T15:08:06.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.114+0000 7f64f224c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64ec07ba50 0x7f64ec1a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.114+0000 7f64f224c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64ec081390 0x7f64ec1a9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.114+0000 7f64f224c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64ec081ce0 0x7f64ec1ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.114+0000 7f64f224c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec122340 con 0x7f64ec07ba50 2024-07-31T15:08:06.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.114+0000 7f64f224c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1221c0 con 0x7f64ec081390 2024-07-31T15:08:06.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.114+0000 7f64f224c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1224c0 con 0x7f64ec081ce0 2024-07-31T15:08:06.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.114+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64ec07ba50 0x7f64ec1a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:06.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.114+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64ec07ba50 0x7f64ec1a8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37916/0 (socket says 172.21.15.67:37916) 2024-07-31T15:08:06.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.114+0000 7f64eb7fe640 1 -- 172.21.15.67:0/3621157317 learned_addr learned my addr 172.21.15.67:0/3621157317 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:06.119 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/tilcdc/built-in.a 2024-07-31T15:08:06.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.115+0000 7f64eb7fe640 1 -- 172.21.15.67:0/3621157317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64ec081ce0 msgr2=0x7f64ec1ae510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:08:06.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.115+0000 7f64eb7fe640 1 --2- 172.21.15.67:0/3621157317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64ec081ce0 0x7f64ec1ae510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.115+0000 7f64eb7fe640 1 -- 172.21.15.67:0/3621157317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64ec081390 msgr2=0x7f64ec1a9270 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:08:06.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.115+0000 7f64eb7fe640 1 --2- 172.21.15.67:0/3621157317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64ec081390 0x7f64ec1a9270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.115+0000 7f64eb7fe640 1 -- 172.21.15.67:0/3621157317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec1aec40 con 0x7f64ec07ba50 2024-07-31T15:08:06.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.116+0000 7f64eb7fe640 1 --2- 172.21.15.67:0/3621157317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64ec07ba50 0x7f64ec1a8cd0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f64dc02a990 tx=0x7f64dc003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:06.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.116+0000 7f64e8ff9640 1 -- 172.21.15.67:0/3621157317 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc03f070 con 0x7f64ec07ba50 2024-07-31T15:08:06.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.116+0000 7f64f224c640 1 -- 172.21.15.67:0/3621157317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ec1aeed0 con 0x7f64ec07ba50 2024-07-31T15:08:06.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.116+0000 7f64f224c640 1 -- 172.21.15.67:0/3621157317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ec10f7e0 con 0x7f64ec07ba50 2024-07-31T15:08:06.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.116+0000 7f64e8ff9640 1 -- 172.21.15.67:0/3621157317 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc03bd20 con 0x7f64ec07ba50 2024-07-31T15:08:06.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.118+0000 7f64e8ff9640 1 -- 172.21.15.67:0/3621157317 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc035030 con 0x7f64ec07ba50 2024-07-31T15:08:06.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.129+0000 7f64f879d740 1 Processor -- start 2024-07-31T15:08:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.142+0000 7f64e8ff9640 1 -- 172.21.15.67:0/3621157317 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f64dc042070 con 0x7f64ec07ba50 2024-07-31T15:08:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.143+0000 7f64e8ff9640 1 --2- 172.21.15.67:0/3621157317 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f64d40b3f50 0x7f64d40b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.143+0000 7f64e8ff9640 1 -- 172.21.15.67:0/3621157317 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f64dc104ad0 con 0x7f64ec07ba50 2024-07-31T15:08:06.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.143+0000 7f64f879d740 1 -- start start 2024-07-31T15:08:06.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.144+0000 7f64f879d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba39570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.144+0000 7f64f879d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b66b1b0 0x561a2ba39ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.144+0000 7f64f879d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b673390 0x561a2ba39ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.144+0000 7f64f879d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561a2b9afbb0 con 0x561a2b694070 2024-07-31T15:08:06.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.144+0000 7f64f879d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561a2b99f0b0 con 0x561a2b66b1b0 2024-07-31T15:08:06.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.144+0000 7f64f879d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561a2b9a30f0 con 0x561a2b673390 2024-07-31T15:08:06.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.148+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba39570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:06.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.148+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba39570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37926/0 (socket says 172.21.15.67:37926) 2024-07-31T15:08:06.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.148+0000 7f64eb7fe640 1 -- 172.21.15.67:0/3992649293 learned_addr learned my addr 172.21.15.67:0/3992649293 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:06.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.149+0000 7f64ebfff640 1 --2- 172.21.15.67:0/3621157317 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f64d40b3f50 0x7f64d40b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:06.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.152+0000 7f64eb7fe640 1 -- 172.21.15.67:0/3992649293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b673390 msgr2=0x561a2ba39ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:06.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.152+0000 7f64eb7fe640 1 --2- 172.21.15.67:0/3992649293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b673390 0x561a2ba39ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.152+0000 7f64eb7fe640 1 -- 172.21.15.67:0/3992649293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b66b1b0 msgr2=0x561a2ba39ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:08:06.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.152+0000 7f64eb7fe640 1 --2- 172.21.15.67:0/3992649293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b66b1b0 0x561a2ba39ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.152+0000 7f64eb7fe640 1 -- 172.21.15.67:0/3992649293 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a2b945b90 con 0x561a2b694070 2024-07-31T15:08:06.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.152+0000 7f64eb7fe640 1 --2- 172.21.15.67:0/3992649293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba39570 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0cafc0 tx=0x7f64dc0cacd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8aef90d8c2df08 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:06.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.153+0000 7f64d1ffb640 1 -- 172.21.15.67:0/3992649293 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0cb2c0 con 0x561a2b694070 2024-07-31T15:08:06.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.153+0000 7f64d1ffb640 1 -- 172.21.15.67:0/3992649293 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc0cf8c0 con 0x561a2b694070 2024-07-31T15:08:06.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.154+0000 7f64f879d740 1 -- 172.21.15.67:0/3992649293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 msgr2=0x561a2ba39570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:06.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.154+0000 7f64f879d740 1 --2- 172.21.15.67:0/3992649293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba39570 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0cafc0 tx=0x7f64dc0cacd0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.160+0000 7f64f879d740 1 -- 172.21.15.67:0/3992649293 shutdown_connections 2024-07-31T15:08:06.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.160+0000 7f64f879d740 1 --2- 172.21.15.67:0/3992649293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b673390 0x561a2ba39ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.160+0000 7f64f879d740 1 --2- 172.21.15.67:0/3992649293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b66b1b0 0x561a2ba39ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.160+0000 7f64f879d740 1 --2- 172.21.15.67:0/3992649293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba39570 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.160+0000 7f64f879d740 1 -- 172.21.15.67:0/3992649293 >> 172.21.15.67:0/3992649293 conn(0x561a2b688460 msgr2=0x561a2ba37460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:06.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.163+0000 7f64ebfff640 1 --2- 172.21.15.67:0/3621157317 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f64d40b3f50 0x7f64d40b6460 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f64e4005c40 tx=0x7f64e400e6c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:06.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.167+0000 7f64f879d740 1 -- 172.21.15.67:0/3992649293 shutdown_connections 2024-07-31T15:08:06.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.174+0000 7f64f879d740 1 -- 172.21.15.67:0/3992649293 wait complete. 2024-07-31T15:08:06.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.174+0000 7f64f879d740 1 -- start start 2024-07-31T15:08:06.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.174+0000 7f64f879d740 1 Processor -- start 2024-07-31T15:08:06.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.178+0000 7f64f879d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b66b1b0 0x561a2ba45140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.178+0000 7f64f879d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b673390 0x561a2ba45680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.178+0000 7f64f879d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b694070 0x561a2ba50e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.178+0000 7f64f879d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561a2b802700 con 0x561a2b673390 2024-07-31T15:08:06.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.178+0000 7f64f879d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561a2b9419e0 con 0x561a2b66b1b0 2024-07-31T15:08:06.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.178+0000 7f64f879d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561a2ba47b10 con 0x561a2b694070 2024-07-31T15:08:06.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.178+0000 7f64f879d740 10 client.0 _ll_register_callbacks cb 0x561a2b95c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:06.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.179+0000 7f64eaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b673390 0x561a2ba45680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:06.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.179+0000 7f64eaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b673390 0x561a2ba45680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37930/0 (socket says 172.21.15.67:37930) 2024-07-31T15:08:06.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.179+0000 7f64eaffd640 1 -- 172.21.15.67:0/124317683 learned_addr learned my addr 172.21.15.67:0/124317683 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:06.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.180+0000 7f64eaffd640 1 -- 172.21.15.67:0/124317683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b694070 msgr2=0x561a2ba50e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:08:06.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.180+0000 7f64eaffd640 1 --2- 172.21.15.67:0/124317683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b694070 0x561a2ba50e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.180+0000 7f64eaffd640 1 -- 172.21.15.67:0/124317683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b66b1b0 msgr2=0x561a2ba45140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:08:06.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.180+0000 7f64eaffd640 1 --2- 172.21.15.67:0/124317683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b66b1b0 0x561a2ba45140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.180+0000 7f64eaffd640 1 -- 172.21.15.67:0/124317683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a2b6ff650 con 0x561a2b673390 2024-07-31T15:08:06.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.180+0000 7f64eaffd640 1 --2- 172.21.15.67:0/124317683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b673390 0x561a2ba45680 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f64e000bb70 tx=0x7f64e000d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108d7745aced5d38 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:06.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.180+0000 7f64caffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:08:06.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.183+0000 7f64caffd640 1 -- 172.21.15.67:0/124317683 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0016070 con 0x561a2b673390 2024-07-31T15:08:06.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.183+0000 7f64f879d740 1 -- 172.21.15.67:0/124317683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561a2b9b1b00 con 0x561a2b673390 2024-07-31T15:08:06.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.183+0000 7f64caffd640 1 -- 172.21.15.67:0/124317683 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f64e0005220 con 0x561a2b673390 2024-07-31T15:08:06.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.184+0000 7f64caffd640 1 -- 172.21.15.67:0/124317683 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f64e0021050 con 0x561a2b673390 2024-07-31T15:08:06.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.187+0000 7f64f879d740 10 client.19050 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:06.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.187+0000 7f64f879d740 1 -- 172.21.15.67:0/124317683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x561a2ba53ca0 con 0x561a2b673390 2024-07-31T15:08:06.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.196+0000 7f64caffd640 1 -- 172.21.15.67:0/124317683 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561a2b9b1b00 con 0x561a2b673390 2024-07-31T15:08:06.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.197+0000 7f64caffd640 1 -- 172.21.15.67:0/124317683 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f64e0010460 con 0x561a2b673390 2024-07-31T15:08:06.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.197+0000 7f64f879d740 10 client.19050 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:06.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.197+0000 7f64f879d740 10 client.19050 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:08:06.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.197+0000 7f64f879d740 20 client.19050 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:06.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.197+0000 7f64f879d740 1 --2- 172.21.15.67:0/124317683 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x561a2b6529f0 0x561a2ba57660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.197+0000 7f64f879d740 4 client.19050 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:08:06.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.197+0000 7f64f879d740 1 -- 172.21.15.67:0/124317683 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a2b9ad5a0 con 0x561a2b6529f0 2024-07-31T15:08:06.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.201+0000 7f64eb7fe640 1 --2- 172.21.15.67:0/124317683 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x561a2b6529f0 0x561a2ba57660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.211+0000 7f64eb7fe640 1 --2- 172.21.15.67:0/124317683 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x561a2b6529f0 0x561a2ba57660 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=30a3edcc721317ce server_cookie=476bb576c2ff679c in_seq=0 out_seq=0 2024-07-31T15:08:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.211+0000 7f64caffd640 10 client.19050 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:08:06.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.213+0000 7f64caffd640 1 -- 172.21.15.67:0/124317683 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a2b9ad5a0 con 0x561a2b6529f0 2024-07-31T15:08:06.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.213+0000 7f64caffd640 10 client.19050 handle_command_reply: tid=0 2024-07-31T15:08:06.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.214+0000 7f64f879d740 1 client.19050 shutdown 2024-07-31T15:08:06.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.216+0000 7f64f879d740 1 -- 172.21.15.67:0/124317683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b673390 msgr2=0x561a2ba45680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:06.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.216+0000 7f64f879d740 1 --2- 172.21.15.67:0/124317683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b673390 0x561a2ba45680 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f64e000bb70 tx=0x7f64e000d6b0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.223 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/virtio/virtgpu_drv.o 2024-07-31T15:08:06.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.220+0000 7f64f879d740 1 -- 172.21.15.67:0/124317683 shutdown_connections 2024-07-31T15:08:06.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.220+0000 7f64f879d740 1 --2- 172.21.15.67:0/124317683 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x561a2b6529f0 0x561a2ba57660 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.220+0000 7f64f879d740 1 --2- 172.21.15.67:0/124317683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b694070 0x561a2ba50e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.220+0000 7f64f879d740 1 --2- 172.21.15.67:0/124317683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b673390 0x561a2ba45680 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.220+0000 7f64f879d740 1 --2- 172.21.15.67:0/124317683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b66b1b0 0x561a2ba45140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.220+0000 7f64f879d740 1 -- 172.21.15.67:0/124317683 >> 172.21.15.67:0/124317683 conn(0x561a2b688460 msgr2=0x561a2ba35c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:06.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.220+0000 7f64caffd640 0 client.19050 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:08:06.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.222+0000 7f64f879d740 1 -- 172.21.15.67:0/124317683 shutdown_connections 2024-07-31T15:08:06.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.222+0000 7f64f879d740 1 -- 172.21.15.67:0/124317683 wait complete. 2024-07-31T15:08:06.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.224+0000 7f64f879d740 20 client.19050 trim_cache size 0 max 16384 2024-07-31T15:08:06.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.256+0000 7f64f879d740 1 Processor -- start 2024-07-31T15:08:06.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.259+0000 7f64f879d740 1 -- start start 2024-07-31T15:08:06.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.259+0000 7f64f879d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b66b1b0 0x561a2ba3e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.259+0000 7f64f879d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b673390 0x561a2ba3ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.259+0000 7f64f879d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba50c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.260+0000 7f64f879d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f64d800ff80 con 0x561a2b694070 2024-07-31T15:08:06.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.260+0000 7f64f879d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f64d8011ac0 con 0x561a2b673390 2024-07-31T15:08:06.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.260+0000 7f64f879d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64d8010650 con 0x561a2b66b1b0 2024-07-31T15:08:06.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.260+0000 7f64ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba50c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:06.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.260+0000 7f64ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba50c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37946/0 (socket says 172.21.15.67:37946) 2024-07-31T15:08:06.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.260+0000 7f64ebfff640 1 -- 172.21.15.67:0/2351249393 learned_addr learned my addr 172.21.15.67:0/2351249393 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:06.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.260+0000 7f64ebfff640 1 -- 172.21.15.67:0/2351249393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b66b1b0 msgr2=0x561a2ba3e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:06.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.260+0000 7f64ebfff640 1 --2- 172.21.15.67:0/2351249393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b66b1b0 0x561a2ba3e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.260+0000 7f64ebfff640 1 -- 172.21.15.67:0/2351249393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b673390 msgr2=0x561a2ba3ebc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:08:06.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.260+0000 7f64ebfff640 1 --2- 172.21.15.67:0/2351249393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b673390 0x561a2ba3ebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.260+0000 7f64ebfff640 1 -- 172.21.15.67:0/2351249393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a2ba3bd70 con 0x561a2b694070 2024-07-31T15:08:06.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.261+0000 7f64ebfff640 1 --2- 172.21.15.67:0/2351249393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba50c90 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f64e4005c10 tx=0x7f64e4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d857bc9d6ef8ceb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:06.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.263+0000 7f64d17fa640 1 -- 172.21.15.67:0/2351249393 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e401f480 con 0x561a2b694070 2024-07-31T15:08:06.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.263+0000 7f64d17fa640 1 -- 172.21.15.67:0/2351249393 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e4024070 con 0x561a2b694070 2024-07-31T15:08:06.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.263+0000 7f64d17fa640 1 -- 172.21.15.67:0/2351249393 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4012a90 con 0x561a2b694070 2024-07-31T15:08:06.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.274+0000 7f64f879d740 1 -- 172.21.15.67:0/2351249393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 msgr2=0x561a2ba50c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:06.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.274+0000 7f64f879d740 1 --2- 172.21.15.67:0/2351249393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba50c90 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f64e4005c10 tx=0x7f64e4013000 comp rx=0 tx=0).stop 2024-07-31T15:08:06.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.274+0000 7f64f879d740 1 -- 172.21.15.67:0/2351249393 shutdown_connections 2024-07-31T15:08:06.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.274+0000 7f64f879d740 1 --2- 172.21.15.67:0/2351249393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba50c90 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.274+0000 7f64f879d740 1 --2- 172.21.15.67:0/2351249393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b673390 0x561a2ba3ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.274+0000 7f64f879d740 1 --2- 172.21.15.67:0/2351249393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b66b1b0 0x561a2ba3e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.274+0000 7f64f879d740 1 -- 172.21.15.67:0/2351249393 >> 172.21.15.67:0/2351249393 conn(0x561a2b688460 msgr2=0x561a2ba437c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.282+0000 7f64f879d740 1 -- 172.21.15.67:0/2351249393 shutdown_connections 2024-07-31T15:08:06.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.287+0000 7f64f879d740 1 -- 172.21.15.67:0/2351249393 wait complete. 2024-07-31T15:08:06.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.287+0000 7f64f879d740 1 -- start start 2024-07-31T15:08:06.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.288+0000 7f64f879d740 1 Processor -- start 2024-07-31T15:08:06.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.289+0000 7f64f879d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b66b1b0 0x561a2ba4d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.289+0000 7f64f879d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b673390 0x561a2ba4d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.289+0000 7f64f879d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba48710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.289+0000 7f64f879d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f64d80113f0 con 0x561a2b694070 2024-07-31T15:08:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.289+0000 7f64f879d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f64d8010d20 con 0x561a2b66b1b0 2024-07-31T15:08:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.289+0000 7f64f879d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64d800e010 con 0x561a2b673390 2024-07-31T15:08:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.289+0000 7f64f879d740 10 client.0 _ll_register_callbacks cb 0x7f64e0005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.290+0000 7f64ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba48710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.290+0000 7f64ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba48710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37948/0 (socket says 172.21.15.67:37948) 2024-07-31T15:08:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.290+0000 7f64ebfff640 1 -- 172.21.15.67:0/1867030978 learned_addr learned my addr 172.21.15.67:0/1867030978 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.290+0000 7f64ebfff640 1 -- 172.21.15.67:0/1867030978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b673390 msgr2=0x561a2ba4d6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:08:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.290+0000 7f64ebfff640 1 --2- 172.21.15.67:0/1867030978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b673390 0x561a2ba4d6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.290+0000 7f64ebfff640 1 -- 172.21.15.67:0/1867030978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b66b1b0 msgr2=0x561a2ba4d100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:08:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.290+0000 7f64ebfff640 1 --2- 172.21.15.67:0/1867030978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b66b1b0 0x561a2ba4d100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.290+0000 7f64ebfff640 1 -- 172.21.15.67:0/1867030978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a2ba48ea0 con 0x561a2b694070 2024-07-31T15:08:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.291+0000 7f64ebfff640 1 --2- 172.21.15.67:0/1867030978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba48710 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009a50 tx=0x7f64e4005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c407ab889a504b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.291+0000 7f64caffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:08:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.291+0000 7f64caffd640 1 -- 172.21.15.67:0/1867030978 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e401f480 con 0x561a2b694070 2024-07-31T15:08:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.292+0000 7f64f879d740 1 -- 172.21.15.67:0/1867030978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e4024070 con 0x561a2b694070 2024-07-31T15:08:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.292+0000 7f64caffd640 1 -- 172.21.15.67:0/1867030978 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f64e401f620 con 0x561a2b694070 2024-07-31T15:08:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.292+0000 7f64caffd640 1 -- 172.21.15.67:0/1867030978 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f64e401f820 con 0x561a2b694070 2024-07-31T15:08:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.292+0000 7f64f879d740 10 client.19056 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.292+0000 7f64f879d740 1 -- 172.21.15.67:0/1867030978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x561a2babf990 con 0x561a2b694070 2024-07-31T15:08:06.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.293+0000 7f64caffd640 1 -- 172.21.15.67:0/1867030978 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e4024070 con 0x561a2b694070 2024-07-31T15:08:06.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.294+0000 7f64caffd640 1 -- 172.21.15.67:0/1867030978 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f64e400f040 con 0x561a2b694070 2024-07-31T15:08:06.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.295+0000 7f64f879d740 10 client.19056 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:06.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.295+0000 7f64f879d740 10 client.19056 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:08:06.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.295+0000 7f64f879d740 20 client.19056 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:06.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.295+0000 7f64f879d740 1 --2- 172.21.15.67:0/1867030978 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x561a2b627da0 0x561a2bac20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:06.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.295+0000 7f64f879d740 4 client.19056 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:08:06.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.295+0000 7f64f879d740 1 -- 172.21.15.67:0/1867030978 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a2b9aace0 con 0x561a2b627da0 2024-07-31T15:08:06.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.301+0000 7f64eb7fe640 1 --2- 172.21.15.67:0/1867030978 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x561a2b627da0 0x561a2bac20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:06.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.313+0000 7f64eb7fe640 1 --2- 172.21.15.67:0/1867030978 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x561a2b627da0 0x561a2bac20a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5161e3961dc2e56f server_cookie=c7548da7e2f14784 in_seq=0 out_seq=0 2024-07-31T15:08:06.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.314+0000 7f64caffd640 10 client.19056 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:08:06.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.314+0000 7f64caffd640 1 -- 172.21.15.67:0/1867030978 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a2b9aace0 con 0x561a2b627da0 2024-07-31T15:08:06.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.314+0000 7f64caffd640 10 client.19056 handle_command_reply: tid=0 2024-07-31T15:08:06.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.314+0000 7f64f879d740 1 client.19056 shutdown 2024-07-31T15:08:06.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.321+0000 7f64f879d740 1 -- 172.21.15.67:0/1867030978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 msgr2=0x561a2ba48710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:06.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.321+0000 7f64f879d740 1 --2- 172.21.15.67:0/1867030978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba48710 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009a50 tx=0x7f64e4005a50 comp rx=0 tx=0).stop 2024-07-31T15:08:06.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.321+0000 7f64f879d740 1 -- 172.21.15.67:0/1867030978 shutdown_connections 2024-07-31T15:08:06.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.321+0000 7f64f879d740 1 --2- 172.21.15.67:0/1867030978 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x561a2b627da0 0x561a2bac20a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.321+0000 7f64f879d740 1 --2- 172.21.15.67:0/1867030978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a2b694070 0x561a2ba48710 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.321+0000 7f64f879d740 1 --2- 172.21.15.67:0/1867030978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a2b673390 0x561a2ba4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.321+0000 7f64f879d740 1 --2- 172.21.15.67:0/1867030978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a2b66b1b0 0x561a2ba4d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.321+0000 7f64caffd640 0 client.19056 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:08:06.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.321+0000 7f64f879d740 1 -- 172.21.15.67:0/1867030978 >> 172.21.15.67:0/1867030978 conn(0x561a2b688460 msgr2=0x561a2ba3e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:06.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.321+0000 7f64f879d740 1 -- 172.21.15.67:0/1867030978 shutdown_connections 2024-07-31T15:08:06.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.321+0000 7f64f879d740 1 -- 172.21.15.67:0/1867030978 wait complete. 2024-07-31T15:08:06.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.321+0000 7f64f879d740 20 client.19056 trim_cache size 0 max 16384 2024-07-31T15:08:06.324 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:08:06.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.322+0000 7f64f224c640 1 -- 172.21.15.67:0/3621157317 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f64d40b3f50 msgr2=0x7f64d40b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:06.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.322+0000 7f64f224c640 1 --2- 172.21.15.67:0/3621157317 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f64d40b3f50 0x7f64d40b6460 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f64e4005c40 tx=0x7f64e400e6c0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.322+0000 7f64f224c640 1 -- 172.21.15.67:0/3621157317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64ec07ba50 msgr2=0x7f64ec1a8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:06.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.322+0000 7f64f224c640 1 --2- 172.21.15.67:0/3621157317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64ec07ba50 0x7f64ec1a8cd0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f64dc02a990 tx=0x7f64dc003cd0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.322+0000 7f64f224c640 1 -- 172.21.15.67:0/3621157317 shutdown_connections 2024-07-31T15:08:06.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.322+0000 7f64f224c640 1 --2- 172.21.15.67:0/3621157317 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f64d40b3f50 0x7f64d40b6460 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.322+0000 7f64f224c640 1 --2- 172.21.15.67:0/3621157317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64ec081ce0 0x7f64ec1ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.322+0000 7f64f224c640 1 --2- 172.21.15.67:0/3621157317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64ec081390 0x7f64ec1a9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.322+0000 7f64f224c640 1 --2- 172.21.15.67:0/3621157317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64ec07ba50 0x7f64ec1a8cd0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:06.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.322+0000 7f64f224c640 1 -- 172.21.15.67:0/3621157317 >> 172.21.15.67:0/3621157317 conn(0x7f64ec077b20 msgr2=0x7f64ec077f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:06.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.323+0000 7f64f224c640 1 -- 172.21.15.67:0/3621157317 shutdown_connections 2024-07-31T15:08:06.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:06.323+0000 7f64f224c640 1 -- 172.21.15.67:0/3621157317 wait complete. 2024-07-31T15:08:07.017 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/netfilter/nf_reject_ipv4.o 2024-07-31T15:08:07.274 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/netfilter/ip_tables.o 2024-07-31T15:08:07.315 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_module.o 2024-07-31T15:08:07.318 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T15:08:07.381 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/acpi/built-in.a 2024-07-31T15:08:07.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:07 smithi073 ceph-mon[36714]: pgmap v940: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 232 op/s 2024-07-31T15:08:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:07 smithi077 ceph-mon[37327]: pgmap v940: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 232 op/s 2024-07-31T15:08:07.493 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:07 smithi067 ceph-mon[32750]: pgmap v940: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 232 op/s 2024-07-31T15:08:07.621 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/virtio/virtgpu_kms.o 2024-07-31T15:08:07.654 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/block/loop.o 2024-07-31T15:08:07.780 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_queue.o 2024-07-31T15:08:08.022 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/power/wakeup.o 2024-07-31T15:08:08.534 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:08:08.932 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/block/virtio_blk.o 2024-07-31T15:08:08.999 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_params.o 2024-07-31T15:08:09.051 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/virtio/virtgpu_gem.o 2024-07-31T15:08:09.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:09 smithi073 ceph-mon[36714]: pgmap v941: 129 pgs: 14 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 72 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 882 KiB/s wr, 226 op/s 2024-07-31T15:08:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:09 smithi077 ceph-mon[37327]: pgmap v941: 129 pgs: 14 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 72 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 882 KiB/s wr, 226 op/s 2024-07-31T15:08:09.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:09 smithi067 ceph-mon[32750]: pgmap v941: 129 pgs: 14 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 72 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 882 KiB/s wr, 226 op/s 2024-07-31T15:08:09.555 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/netfilter/iptable_filter.o 2024-07-31T15:08:09.556 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_sockopt.o 2024-07-31T15:08:09.747 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/virtio/virtgpu_vram.o 2024-07-31T15:08:10.046 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/power/wakeup_stats.o 2024-07-31T15:08:10.067 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/misc/eeprom/built-in.a 2024-07-31T15:08:10.237 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-07-31T15:08:10.238 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:08:10.296 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/mfd/built-in.a 2024-07-31T15:08:10.299 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_pci.o 2024-07-31T15:08:10.369 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/inetpeer.o 2024-07-31T15:08:10.379 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/misc/cb710/built-in.a 2024-07-31T15:08:10.484 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/block/built-in.a 2024-07-31T15:08:10.494 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/misc/ti-st/built-in.a 2024-07-31T15:08:10.636 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/misc/lis3lv02d/built-in.a 2024-07-31T15:08:10.696 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/nfc/built-in.a 2024-07-31T15:08:10.732 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/misc/cardreader/built-in.a 2024-07-31T15:08:10.753 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/power/trace.o 2024-07-31T15:08:10.773 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/virtio/virtgpu_display.o 2024-07-31T15:08:10.795 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/misc/built-in.a 2024-07-31T15:08:10.860 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/dax/hmem/built-in.a 2024-07-31T15:08:10.860 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/utils.o 2024-07-31T15:08:10.873 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/netfilter/iptable_mangle.o 2024-07-31T15:08:10.904 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/dax/built-in.a 2024-07-31T15:08:10.932 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma-buf/dma-buf.o 2024-07-31T15:08:10.977 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma-buf/dma-fence.o 2024-07-31T15:08:11.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.030+0000 7f096d171640 1 -- 172.21.15.67:0/1142966602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f096811c660 msgr2=0x7f096811eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:11.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.030+0000 7f096d171640 1 --2- 172.21.15.67:0/1142966602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f096811c660 0x7f096811eb70 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f096001acb0 tx=0x7f096003b920 comp rx=0 tx=0).stop 2024-07-31T15:08:11.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.030+0000 7f096d171640 1 -- 172.21.15.67:0/1142966602 shutdown_connections 2024-07-31T15:08:11.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.030+0000 7f096d171640 1 --2- 172.21.15.67:0/1142966602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f096811c660 0x7f096811eb70 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.030+0000 7f096d171640 1 --2- 172.21.15.67:0/1142966602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f096810a410 0x7f096810a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.030+0000 7f096d171640 1 --2- 172.21.15.67:0/1142966602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0968111970 0x7f0968109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.030+0000 7f096d171640 1 -- 172.21.15.67:0/1142966602 >> 172.21.15.67:0/1142966602 conn(0x7f09680708f0 msgr2=0x7f0968070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:11.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.043+0000 7f096d171640 1 -- 172.21.15.67:0/1142966602 shutdown_connections 2024-07-31T15:08:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.052+0000 7f096d171640 1 -- 172.21.15.67:0/1142966602 wait complete. 2024-07-31T15:08:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.053+0000 7f096d171640 1 Processor -- start 2024-07-31T15:08:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.053+0000 7f096d171640 1 -- start start 2024-07-31T15:08:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.053+0000 7f096d171640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f096810a410 0x7f0968117f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.053+0000 7f096d171640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0968111970 0x7f0968118520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.053+0000 7f096d171640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09681a58e0 0x7f09681a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.054+0000 7f096d171640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f09681146a0 con 0x7f0968111970 2024-07-31T15:08:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.054+0000 7f096d171640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0968114520 con 0x7f096810a410 2024-07-31T15:08:11.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.054+0000 7f096d171640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0968114820 con 0x7f09681a58e0 2024-07-31T15:08:11.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.058+0000 7f096759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09681a58e0 0x7f09681a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:11.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.058+0000 7f0966d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f096810a410 0x7f0968117f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:11.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.060+0000 7f096759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09681a58e0 0x7f09681a5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:46798/0 (socket says 172.21.15.67:46798) 2024-07-31T15:08:11.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.060+0000 7f096759e640 1 -- 172.21.15.67:0/109868840 learned_addr learned my addr 172.21.15.67:0/109868840 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:11.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.061+0000 7f0966d9d640 1 -- 172.21.15.67:0/109868840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09681a58e0 msgr2=0x7f09681a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:11.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.061+0000 7f0966d9d640 1 --2- 172.21.15.67:0/109868840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09681a58e0 0x7f09681a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.061+0000 7f0966d9d640 1 -- 172.21.15.67:0/109868840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0968111970 msgr2=0x7f0968118520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:11.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.061+0000 7f0966d9d640 1 --2- 172.21.15.67:0/109868840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0968111970 0x7f0968118520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.061+0000 7f0966d9d640 1 -- 172.21.15.67:0/109868840 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09681a64c0 con 0x7f096810a410 2024-07-31T15:08:11.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.067+0000 7f0966d9d640 1 --2- 172.21.15.67:0/109868840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f096810a410 0x7f0968117f80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f095800c970 tx=0x7f095800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.072+0000 7f094ffff640 1 -- 172.21.15.67:0/109868840 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958007bb0 con 0x7f096810a410 2024-07-31T15:08:11.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.072+0000 7f096d171640 1 -- 172.21.15.67:0/109868840 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09681aca60 con 0x7f096810a410 2024-07-31T15:08:11.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.072+0000 7f096d171640 1 -- 172.21.15.67:0/109868840 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09681aceb0 con 0x7f096810a410 2024-07-31T15:08:11.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.074+0000 7f094ffff640 1 -- 172.21.15.67:0/109868840 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0958007d50 con 0x7f096810a410 2024-07-31T15:08:11.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.074+0000 7f094ffff640 1 -- 172.21.15.67:0/109868840 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958018820 con 0x7f096810a410 2024-07-31T15:08:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.080+0000 7f09736c2740 1 Processor -- start 2024-07-31T15:08:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.080+0000 7f09736c2740 1 -- start start 2024-07-31T15:08:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.081+0000 7f09736c2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd367570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.081+0000 7f09736c2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcf991b0 0x562ddd367ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.081+0000 7f09736c2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcfa1390 0x562ddd367ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.081+0000 7f09736c2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562ddd2ddbb0 con 0x562ddcfa1390 2024-07-31T15:08:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.081+0000 7f09736c2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562ddd2cd0b0 con 0x562ddcfc2070 2024-07-31T15:08:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.081+0000 7f09736c2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562ddd2d10f0 con 0x562ddcf991b0 2024-07-31T15:08:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.081+0000 7f096759e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd367570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.081+0000 7f096759e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd367570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48232/0 (socket says 172.21.15.67:48232) 2024-07-31T15:08:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.081+0000 7f096759e640 1 -- 172.21.15.67:0/2763963079 learned_addr learned my addr 172.21.15.67:0/2763963079 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.082+0000 7f096759e640 1 -- 172.21.15.67:0/2763963079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcf991b0 msgr2=0x562ddd367ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:08:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.082+0000 7f096759e640 1 --2- 172.21.15.67:0/2763963079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcf991b0 0x562ddd367ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.082+0000 7f096759e640 1 -- 172.21.15.67:0/2763963079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcfa1390 msgr2=0x562ddd367ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:08:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.082+0000 7f096759e640 1 --2- 172.21.15.67:0/2763963079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcfa1390 0x562ddd367ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.082+0000 7f096759e640 1 -- 172.21.15.67:0/2763963079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ddd273b90 con 0x562ddcfc2070 2024-07-31T15:08:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.082+0000 7f096759e640 1 --2- 172.21.15.67:0/2763963079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd367570 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0960040b60 tx=0x7f096003be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2643e72aceec3c55 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:11.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.085+0000 7f094ffff640 1 -- 172.21.15.67:0/109868840 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f09580189c0 con 0x7f096810a410 2024-07-31T15:08:11.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.086+0000 7f094ffff640 1 --2- 172.21.15.67:0/109868840 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09500b3e80 0x7f09500b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.086+0000 7f094ffff640 1 -- 172.21.15.67:0/109868840 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f09580e4210 con 0x7f096810a410 2024-07-31T15:08:11.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.094+0000 7f094d7fa640 1 -- 172.21.15.67:0/2763963079 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960045040 con 0x562ddcfc2070 2024-07-31T15:08:11.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.094+0000 7f094d7fa640 1 -- 172.21.15.67:0/2763963079 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0960040820 con 0x562ddcfc2070 2024-07-31T15:08:11.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.094+0000 7f094d7fa640 1 -- 172.21.15.67:0/2763963079 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096004a420 con 0x562ddcfc2070 2024-07-31T15:08:11.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.096+0000 7f09736c2740 1 -- 172.21.15.67:0/2763963079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 msgr2=0x562ddd367570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:11.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.096+0000 7f09736c2740 1 --2- 172.21.15.67:0/2763963079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd367570 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0960040b60 tx=0x7f096003be20 comp rx=0 tx=0).stop 2024-07-31T15:08:11.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.097+0000 7f09736c2740 1 -- 172.21.15.67:0/2763963079 shutdown_connections 2024-07-31T15:08:11.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.097+0000 7f09736c2740 1 --2- 172.21.15.67:0/2763963079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcfa1390 0x562ddd367ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.097+0000 7f09736c2740 1 --2- 172.21.15.67:0/2763963079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcf991b0 0x562ddd367ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.097+0000 7f09736c2740 1 --2- 172.21.15.67:0/2763963079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd367570 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.097+0000 7f09736c2740 1 -- 172.21.15.67:0/2763963079 >> 172.21.15.67:0/2763963079 conn(0x562ddcfb6460 msgr2=0x562ddd365460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:11.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.099+0000 7f0966d9d640 1 --2- 172.21.15.67:0/109868840 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09500b3e80 0x7f09500b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:11.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.100+0000 7f09736c2740 1 -- 172.21.15.67:0/2763963079 shutdown_connections 2024-07-31T15:08:11.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.103+0000 7f0966d9d640 1 --2- 172.21.15.67:0/109868840 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09500b3e80 0x7f09500b6390 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x562ddd3661f0 tx=0x7f0958005d80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:11.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.103+0000 7f09736c2740 1 -- 172.21.15.67:0/2763963079 wait complete. 2024-07-31T15:08:11.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.103+0000 7f09736c2740 1 -- start start 2024-07-31T15:08:11.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.103+0000 7f09736c2740 1 Processor -- start 2024-07-31T15:08:11.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.104+0000 7f09736c2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcf991b0 0x562ddd3731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.104+0000 7f09736c2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcfa1390 0x562ddd373740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.104+0000 7f09736c2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd37f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.104+0000 7f09736c2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562ddd130700 con 0x562ddcf991b0 2024-07-31T15:08:11.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.104+0000 7f09736c2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562ddd26f9e0 con 0x562ddcfc2070 2024-07-31T15:08:11.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.104+0000 7f09736c2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562ddd375cf0 con 0x562ddcfa1390 2024-07-31T15:08:11.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.104+0000 7f09736c2740 10 client.0 _ll_register_callbacks cb 0x562ddd28a4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.108+0000 7f096759e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd37f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.108+0000 7f096759e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd37f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48238/0 (socket says 172.21.15.67:48238) 2024-07-31T15:08:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.108+0000 7f096759e640 1 -- 172.21.15.67:0/1189134328 learned_addr learned my addr 172.21.15.67:0/1189134328 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.108+0000 7f096759e640 1 --2- 172.21.15.67:0/1189134328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcf991b0 0x562ddd3731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.112+0000 7f096759e640 1 -- 172.21.15.67:0/1189134328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcfa1390 msgr2=0x562ddd373740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:08:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.112+0000 7f096759e640 1 --2- 172.21.15.67:0/1189134328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcfa1390 0x562ddd373740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.112+0000 7f096759e640 1 -- 172.21.15.67:0/1189134328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcf991b0 msgr2=0x562ddd3731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.112+0000 7f096759e640 1 --2- 172.21.15.67:0/1189134328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcf991b0 0x562ddd3731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.112+0000 7f096759e640 1 -- 172.21.15.67:0/1189134328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ddd02d650 con 0x562ddcfc2070 2024-07-31T15:08:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.116+0000 7f096759e640 1 --2- 172.21.15.67:0/1189134328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd37f070 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f096001aa30 tx=0x7f0960041a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad4d8c1ec720db45 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:11.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.118+0000 7f094aff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:08:11.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.122+0000 7f094aff5640 1 -- 172.21.15.67:0/1189134328 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960045070 con 0x562ddcfc2070 2024-07-31T15:08:11.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.122+0000 7f094aff5640 1 -- 172.21.15.67:0/1189134328 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f0960019040 con 0x562ddcfc2070 2024-07-31T15:08:11.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.125+0000 7f09736c2740 1 -- 172.21.15.67:0/1189134328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0960040820 con 0x562ddcfc2070 2024-07-31T15:08:11.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.130+0000 7f094aff5640 1 -- 172.21.15.67:0/1189134328 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f0960040580 con 0x562ddcfc2070 2024-07-31T15:08:11.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.130+0000 7f09736c2740 10 client.28586 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:11.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.130+0000 7f09736c2740 1 -- 172.21.15.67:0/1189134328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x562ddd3a3df0 con 0x562ddcfc2070 2024-07-31T15:08:11.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.135+0000 7f094aff5640 1 -- 172.21.15.67:0/1189134328 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0960040820 con 0x562ddcfc2070 2024-07-31T15:08:11.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.135+0000 7f094aff5640 1 -- 172.21.15.67:0/1189134328 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f096005b070 con 0x562ddcfc2070 2024-07-31T15:08:11.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.135+0000 7f09736c2740 10 client.28586 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:11.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.135+0000 7f09736c2740 10 client.28586 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:08:11.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.135+0000 7f09736c2740 20 client.28586 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:11.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.135+0000 7f09736c2740 1 --2- 172.21.15.67:0/1189134328 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562ddcf809f0 0x562ddd3a7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.135+0000 7f09736c2740 4 client.28586 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:08:11.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.135+0000 7f09736c2740 1 -- 172.21.15.67:0/1189134328 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ddd2dfb00 con 0x562ddcf809f0 2024-07-31T15:08:11.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.138+0000 7f096659c640 1 --2- 172.21.15.67:0/1189134328 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562ddcf809f0 0x562ddd3a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:11.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.149+0000 7f096659c640 1 --2- 172.21.15.67:0/1189134328 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562ddcf809f0 0x562ddd3a7300 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=91babad32d965b04 server_cookie=983ee9c3b3d1465e in_seq=0 out_seq=0 2024-07-31T15:08:11.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.151+0000 7f094aff5640 10 client.28586 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:08:11.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.155+0000 7f094aff5640 1 -- 172.21.15.67:0/1189134328 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ddd2dfb00 con 0x562ddcf809f0 2024-07-31T15:08:11.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.155+0000 7f094aff5640 10 client.28586 handle_command_reply: tid=0 2024-07-31T15:08:11.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.155+0000 7f09736c2740 1 client.28586 shutdown 2024-07-31T15:08:11.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.155+0000 7f09736c2740 1 -- 172.21.15.67:0/1189134328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 msgr2=0x562ddd37f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:11.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.155+0000 7f09736c2740 1 --2- 172.21.15.67:0/1189134328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd37f070 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f096001aa30 tx=0x7f0960041a90 comp rx=0 tx=0).stop 2024-07-31T15:08:11.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.159+0000 7f09736c2740 1 -- 172.21.15.67:0/1189134328 shutdown_connections 2024-07-31T15:08:11.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.159+0000 7f09736c2740 1 --2- 172.21.15.67:0/1189134328 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562ddcf809f0 0x562ddd3a7300 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.159+0000 7f09736c2740 1 --2- 172.21.15.67:0/1189134328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd37f070 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.159+0000 7f09736c2740 1 --2- 172.21.15.67:0/1189134328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcfa1390 0x562ddd373740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.159+0000 7f09736c2740 1 --2- 172.21.15.67:0/1189134328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcf991b0 0x562ddd3731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.159+0000 7f09736c2740 1 -- 172.21.15.67:0/1189134328 >> 172.21.15.67:0/1189134328 conn(0x562ddcfb6460 msgr2=0x562ddd363c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.159+0000 7f094aff5640 0 client.28586 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:08:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.171+0000 7f09736c2740 1 -- 172.21.15.67:0/1189134328 shutdown_connections 2024-07-31T15:08:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.171+0000 7f09736c2740 1 -- 172.21.15.67:0/1189134328 wait complete. 2024-07-31T15:08:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.171+0000 7f09736c2740 20 client.28586 trim_cache size 0 max 16384 2024-07-31T15:08:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.201+0000 7f09736c2740 1 Processor -- start 2024-07-31T15:08:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.201+0000 7f09736c2740 1 -- start start 2024-07-31T15:08:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.202+0000 7f09736c2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcf991b0 0x562ddd3654f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.202+0000 7f09736c2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcfa1390 0x562ddd374bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.202+0000 7f096759e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcf991b0 0x562ddd3654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.202+0000 7f09736c2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd37ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.202+0000 7f09736c2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f09540100b0 con 0x562ddcf991b0 2024-07-31T15:08:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.202+0000 7f09736c2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0954011bf0 con 0x562ddcfc2070 2024-07-31T15:08:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.202+0000 7f09736c2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0954010780 con 0x562ddcfa1390 2024-07-31T15:08:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.203+0000 7f096759e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd37ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.203+0000 7f096759e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd37ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48240/0 (socket says 172.21.15.67:48240) 2024-07-31T15:08:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.203+0000 7f096759e640 1 -- 172.21.15.67:0/614199658 learned_addr learned my addr 172.21.15.67:0/614199658 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.207+0000 7f096759e640 1 -- 172.21.15.67:0/614199658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcfa1390 msgr2=0x562ddd374bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:08:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.207+0000 7f096759e640 1 --2- 172.21.15.67:0/614199658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcfa1390 0x562ddd374bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.207+0000 7f096759e640 1 -- 172.21.15.67:0/614199658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcf991b0 msgr2=0x562ddd3654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:11.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.207+0000 7f096759e640 1 --2- 172.21.15.67:0/614199658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcf991b0 0x562ddd3654f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.207+0000 7f096759e640 1 -- 172.21.15.67:0/614199658 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ddd360c70 con 0x562ddcfc2070 2024-07-31T15:08:11.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.207+0000 7f096759e640 1 --2- 172.21.15.67:0/614199658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd37ee60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f096001a2c0 tx=0x7f09600419f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22ac1a741749a865 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:11.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.211+0000 7f094cff9640 1 -- 172.21.15.67:0/614199658 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960045030 con 0x562ddcfc2070 2024-07-31T15:08:11.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.211+0000 7f094cff9640 1 -- 172.21.15.67:0/614199658 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f096001a5d0 con 0x562ddcfc2070 2024-07-31T15:08:11.218 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:11 smithi067 ceph-mon[32750]: pgmap v942: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 740 KiB/s wr, 212 op/s 2024-07-31T15:08:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.212+0000 7f09736c2740 1 -- 172.21.15.67:0/614199658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 msgr2=0x562ddd37ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.212+0000 7f09736c2740 1 --2- 172.21.15.67:0/614199658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd37ee60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f096001a2c0 tx=0x7f09600419f0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.214+0000 7f09736c2740 1 -- 172.21.15.67:0/614199658 shutdown_connections 2024-07-31T15:08:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.214+0000 7f09736c2740 1 --2- 172.21.15.67:0/614199658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfc2070 0x562ddd37ee60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.214+0000 7f09736c2740 1 --2- 172.21.15.67:0/614199658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcfa1390 0x562ddd374bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.214+0000 7f09736c2740 1 --2- 172.21.15.67:0/614199658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcf991b0 0x562ddd3654f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.214+0000 7f09736c2740 1 -- 172.21.15.67:0/614199658 >> 172.21.15.67:0/614199658 conn(0x562ddcfb6460 msgr2=0x562ddd3744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:11.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.219+0000 7f09736c2740 1 -- 172.21.15.67:0/614199658 shutdown_connections 2024-07-31T15:08:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.219+0000 7f09736c2740 1 -- 172.21.15.67:0/614199658 wait complete. 2024-07-31T15:08:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.219+0000 7f09736c2740 1 -- start start 2024-07-31T15:08:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.219+0000 7f09736c2740 1 Processor -- start 2024-07-31T15:08:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.220+0000 7f09736c2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcf991b0 0x562ddd3767e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.220+0000 7f09736c2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfa1390 0x562ddd376dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.220+0000 7f09736c2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcfc2070 0x562ddd407b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.220+0000 7f09736c2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0954011520 con 0x562ddcfc2070 2024-07-31T15:08:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.220+0000 7f09736c2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0954010e50 con 0x562ddcfa1390 2024-07-31T15:08:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.220+0000 7f09736c2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f095400e0b0 con 0x562ddcf991b0 2024-07-31T15:08:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.220+0000 7f09736c2740 10 client.0 _ll_register_callbacks cb 0x7f0960040c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.226+0000 7f096759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcf991b0 0x562ddd3767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.226+0000 7f096759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcf991b0 0x562ddd3767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:46830/0 (socket says 172.21.15.67:46830) 2024-07-31T15:08:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.226+0000 7f096759e640 1 -- 172.21.15.67:0/3596436651 learned_addr learned my addr 172.21.15.67:0/3596436651 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.226+0000 7f096759e640 1 --2- 172.21.15.67:0/3596436651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcfc2070 0x562ddd407b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.226+0000 7f096659c640 1 --2- 172.21.15.67:0/3596436651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfa1390 0x562ddd376dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.227+0000 7f096759e640 1 -- 172.21.15.67:0/3596436651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfa1390 msgr2=0x562ddd376dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.227+0000 7f096759e640 1 --2- 172.21.15.67:0/3596436651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfa1390 0x562ddd376dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.227+0000 7f096759e640 1 -- 172.21.15.67:0/3596436651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcfc2070 msgr2=0x562ddd407b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.227+0000 7f096759e640 1 --2- 172.21.15.67:0/3596436651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcfc2070 0x562ddd407b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.227+0000 7f096759e640 1 -- 172.21.15.67:0/3596436651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ddd408170 con 0x562ddcf991b0 2024-07-31T15:08:11.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.236+0000 7f096759e640 1 --2- 172.21.15.67:0/3596436651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcf991b0 0x562ddd3767e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0960005d50 tx=0x7f0960041950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5751ce839a545cad server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:11.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.238+0000 7f094aff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:08:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.245+0000 7f094aff5640 1 -- 172.21.15.67:0/3596436651 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960045040 con 0x562ddcf991b0 2024-07-31T15:08:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.245+0000 7f094aff5640 1 -- 172.21.15.67:0/3596436651 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f0960041210 con 0x562ddcf991b0 2024-07-31T15:08:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.247+0000 7f09736c2740 1 -- 172.21.15.67:0/3596436651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562ddd2db5a0 con 0x562ddcf991b0 2024-07-31T15:08:11.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.252+0000 7f094aff5640 1 -- 172.21.15.67:0/3596436651 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f0960004b40 con 0x562ddcf991b0 2024-07-31T15:08:11.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.254+0000 7f09736c2740 10 client.28480 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:11.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.254+0000 7f09736c2740 1 -- 172.21.15.67:0/3596436651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x562ddd40f490 con 0x562ddcf991b0 2024-07-31T15:08:11.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.260+0000 7f094aff5640 1 -- 172.21.15.67:0/3596436651 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562ddd2db5a0 con 0x562ddcf991b0 2024-07-31T15:08:11.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.260+0000 7f094aff5640 1 -- 172.21.15.67:0/3596436651 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f096005b070 con 0x562ddcf991b0 2024-07-31T15:08:11.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.262+0000 7f09736c2740 10 client.28480 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:11.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.262+0000 7f09736c2740 10 client.28480 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:08:11.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.262+0000 7f09736c2740 20 client.28480 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:11.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.262+0000 7f09736c2740 1 --2- 172.21.15.67:0/3596436651 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562ddcf55da0 0x562ddd4117a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:11.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.262+0000 7f09736c2740 4 client.28480 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:08:11.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.262+0000 7f09736c2740 1 -- 172.21.15.67:0/3596436651 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ddd2d8ce0 con 0x562ddcf55da0 2024-07-31T15:08:11.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.266+0000 7f096659c640 1 --2- 172.21.15.67:0/3596436651 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562ddcf55da0 0x562ddd4117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:11.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.280+0000 7f094aff5640 10 client.28480 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:08:11.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.280+0000 7f096659c640 1 --2- 172.21.15.67:0/3596436651 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562ddcf55da0 0x562ddd4117a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ea4327ae14bb44ba server_cookie=730c5af79420b04c in_seq=0 out_seq=0 2024-07-31T15:08:11.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.280+0000 7f094aff5640 1 -- 172.21.15.67:0/3596436651 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ddd2d8ce0 con 0x562ddcf55da0 2024-07-31T15:08:11.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.280+0000 7f094aff5640 10 client.28480 handle_command_reply: tid=0 2024-07-31T15:08:11.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.280+0000 7f09736c2740 1 client.28480 shutdown 2024-07-31T15:08:11.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.280+0000 7f09736c2740 1 -- 172.21.15.67:0/3596436651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcf991b0 msgr2=0x562ddd3767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:11.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.280+0000 7f09736c2740 1 --2- 172.21.15.67:0/3596436651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcf991b0 0x562ddd3767e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0960005d50 tx=0x7f0960041950 comp rx=0 tx=0).stop 2024-07-31T15:08:11.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.280+0000 7f09736c2740 1 -- 172.21.15.67:0/3596436651 shutdown_connections 2024-07-31T15:08:11.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.280+0000 7f09736c2740 1 --2- 172.21.15.67:0/3596436651 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x562ddcf55da0 0x562ddd4117a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.280+0000 7f09736c2740 1 --2- 172.21.15.67:0/3596436651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562ddcfc2070 0x562ddd407b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.287 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:08:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.280+0000 7f09736c2740 1 --2- 172.21.15.67:0/3596436651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ddcfa1390 0x562ddd376dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.280+0000 7f094aff5640 0 client.28480 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:08:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.280+0000 7f09736c2740 1 --2- 172.21.15.67:0/3596436651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ddcf991b0 0x562ddd3767e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.281+0000 7f09736c2740 1 -- 172.21.15.67:0/3596436651 >> 172.21.15.67:0/3596436651 conn(0x562ddcfb6460 msgr2=0x562ddd3652c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.281+0000 7f09736c2740 1 -- 172.21.15.67:0/3596436651 shutdown_connections 2024-07-31T15:08:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.281+0000 7f09736c2740 1 -- 172.21.15.67:0/3596436651 wait complete. 2024-07-31T15:08:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.281+0000 7f09736c2740 20 client.28480 trim_cache size 0 max 16384 2024-07-31T15:08:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.282+0000 7f096d171640 1 -- 172.21.15.67:0/109868840 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09500b3e80 msgr2=0x7f09500b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.282+0000 7f096d171640 1 --2- 172.21.15.67:0/109868840 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09500b3e80 0x7f09500b6390 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x562ddd3661f0 tx=0x7f0958005d80 comp rx=0 tx=0).stop 2024-07-31T15:08:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.282+0000 7f096d171640 1 -- 172.21.15.67:0/109868840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f096810a410 msgr2=0x7f0968117f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.282+0000 7f096d171640 1 --2- 172.21.15.67:0/109868840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f096810a410 0x7f0968117f80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f095800c970 tx=0x7f095800ce30 comp rx=0 tx=0).stop 2024-07-31T15:08:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.282+0000 7f096d171640 1 -- 172.21.15.67:0/109868840 shutdown_connections 2024-07-31T15:08:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.282+0000 7f096d171640 1 --2- 172.21.15.67:0/109868840 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f09500b3e80 0x7f09500b6390 secure :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0x562ddd3661f0 tx=0x7f0958005d80 comp rx=0 tx=0).stop 2024-07-31T15:08:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.282+0000 7f096d171640 1 --2- 172.21.15.67:0/109868840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09681a58e0 0x7f09681a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.282+0000 7f096d171640 1 --2- 172.21.15.67:0/109868840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0968111970 0x7f0968118520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.282+0000 7f096d171640 1 --2- 172.21.15.67:0/109868840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f096810a410 0x7f0968117f80 secure :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f095800c970 tx=0x7f095800ce30 comp rx=0 tx=0).stop 2024-07-31T15:08:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.282+0000 7f096d171640 1 -- 172.21.15.67:0/109868840 >> 172.21.15.67:0/109868840 conn(0x7f09680708f0 msgr2=0x7f096810b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.285+0000 7f096d171640 1 -- 172.21.15.67:0/109868840 shutdown_connections 2024-07-31T15:08:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:11.285+0000 7f096d171640 1 -- 172.21.15.67:0/109868840 wait complete. 2024-07-31T15:08:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:08:11.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:11 smithi073 ceph-mon[36714]: pgmap v942: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 740 KiB/s wr, 212 op/s 2024-07-31T15:08:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:11 smithi077 ceph-mon[37327]: pgmap v942: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 740 KiB/s wr, 212 op/s 2024-07-31T15:08:11.813 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_scatterlist.o 2024-07-31T15:08:11.842 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/base/power/built-in.a 2024-07-31T15:08:12.005 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/netfilter/ipt_REJECT.o 2024-07-31T15:08:12.199 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] net/ipv4/netfilter/iptable_nat.o 2024-07-31T15:08:12.226 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/regmap/regmap.o 2024-07-31T15:08:12.368 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T15:08:12.383 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/regmap/regcache.o 2024-07-31T15:08:12.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:12 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:08:12.448 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:12 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:08:12.449 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma-buf/dma-fence-array.o 2024-07-31T15:08:12.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:12 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:08:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.582+0000 7fc0d0c60640 1 -- 172.21.15.67:0/1642163478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0cc11c5d0 msgr2=0x7fc0cc11e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.582+0000 7fc0d0c60640 1 --2- 172.21.15.67:0/1642163478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0cc11c5d0 0x7fc0cc11e9f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009ca0 tx=0x7fc0c402a8e0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.589+0000 7fc0d0c60640 1 -- 172.21.15.67:0/1642163478 shutdown_connections 2024-07-31T15:08:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.589+0000 7fc0d0c60640 1 --2- 172.21.15.67:0/1642163478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0cc11c5d0 0x7fc0cc11e9f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.589+0000 7fc0d0c60640 1 --2- 172.21.15.67:0/1642163478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0cc10a330 0x7fc0cc10a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.589+0000 7fc0d0c60640 1 --2- 172.21.15.67:0/1642163478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0cc111650 0x7fc0cc109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.589+0000 7fc0d0c60640 1 -- 172.21.15.67:0/1642163478 >> 172.21.15.67:0/1642163478 conn(0x7fc0cc06e320 msgr2=0x7fc0cc06e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.591+0000 7fc0d0c60640 1 -- 172.21.15.67:0/1642163478 shutdown_connections 2024-07-31T15:08:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.591+0000 7fc0d0c60640 1 -- 172.21.15.67:0/1642163478 wait complete. 2024-07-31T15:08:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.592+0000 7fc0d0c60640 1 Processor -- start 2024-07-31T15:08:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.592+0000 7fc0d0c60640 1 -- start start 2024-07-31T15:08:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.593+0000 7fc0d0c60640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0cc10a330 0x7fc0cc1a8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.593+0000 7fc0d0c60640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0cc111650 0x7fc0cc1a9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.593+0000 7fc0d0c60640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0cc11c5d0 0x7fc0cc1ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.593+0000 7fc0d0c60640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc122140 con 0x7fc0cc111650 2024-07-31T15:08:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.593+0000 7fc0d0c60640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc121fc0 con 0x7fc0cc11c5d0 2024-07-31T15:08:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.593+0000 7fc0d0c60640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc1222c0 con 0x7fc0cc10a330 2024-07-31T15:08:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.593+0000 7fc0cb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0cc10a330 0x7fc0cc1a8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.593+0000 7fc0cb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0cc10a330 0x7fc0cc1a8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:46834/0 (socket says 172.21.15.67:46834) 2024-07-31T15:08:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.593+0000 7fc0cb7fe640 1 -- 172.21.15.67:0/815819844 learned_addr learned my addr 172.21.15.67:0/815819844 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.593+0000 7fc0cbfff640 1 --2- 172.21.15.67:0/815819844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0cc11c5d0 0x7fc0cc1ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.594+0000 7fc0cb7fe640 1 -- 172.21.15.67:0/815819844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0cc11c5d0 msgr2=0x7fc0cc1ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.594+0000 7fc0cb7fe640 1 --2- 172.21.15.67:0/815819844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0cc11c5d0 0x7fc0cc1ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.594+0000 7fc0cb7fe640 1 -- 172.21.15.67:0/815819844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0cc111650 msgr2=0x7fc0cc1a9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.594+0000 7fc0cb7fe640 1 --2- 172.21.15.67:0/815819844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0cc111650 0x7fc0cc1a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.594+0000 7fc0cb7fe640 1 -- 172.21.15.67:0/815819844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc1aead0 con 0x7fc0cc10a330 2024-07-31T15:08:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.594+0000 7fc0cb7fe640 1 --2- 172.21.15.67:0/815819844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0cc10a330 0x7fc0cc1a8ba0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00b450 tx=0x7fc0bc00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.594+0000 7fc0c8ff9640 1 -- 172.21.15.67:0/815819844 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc015020 con 0x7fc0cc10a330 2024-07-31T15:08:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.594+0000 7fc0d0c60640 1 -- 172.21.15.67:0/815819844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0cc1aedc0 con 0x7fc0cc10a330 2024-07-31T15:08:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.595+0000 7fc0d0c60640 1 -- 172.21.15.67:0/815819844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0cc07b050 con 0x7fc0cc10a330 2024-07-31T15:08:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.595+0000 7fc0c8ff9640 1 -- 172.21.15.67:0/815819844 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0bc011d40 con 0x7fc0cc10a330 2024-07-31T15:08:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.595+0000 7fc0c8ff9640 1 -- 172.21.15.67:0/815819844 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc00f340 con 0x7fc0cc10a330 2024-07-31T15:08:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.601+0000 7fc0d8dbe740 1 Processor -- start 2024-07-31T15:08:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.601+0000 7fc0d8dbe740 1 -- start start 2024-07-31T15:08:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.601+0000 7fc0d8dbe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c451b0 0x55f839fe7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.601+0000 7fc0d8dbe740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c1c2f0 0x55f839fe83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.601+0000 7fc0d8dbe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c244d0 0x55f839fe8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.601+0000 7fc0d8dbe740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f839f5d3b0 con 0x55f839c1c2f0 2024-07-31T15:08:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.601+0000 7fc0d8dbe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f839f4c8b0 con 0x55f839c451b0 2024-07-31T15:08:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.601+0000 7fc0d8dbe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f839f508f0 con 0x55f839c244d0 2024-07-31T15:08:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.602+0000 7fc0cbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c451b0 0x55f839fe7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.602+0000 7fc0cbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c451b0 0x55f839fe7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48288/0 (socket says 172.21.15.67:48288) 2024-07-31T15:08:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.602+0000 7fc0cbfff640 1 -- 172.21.15.67:0/640870172 learned_addr learned my addr 172.21.15.67:0/640870172 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.602+0000 7fc0cbfff640 1 -- 172.21.15.67:0/640870172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c244d0 msgr2=0x55f839fe8920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:08:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.602+0000 7fc0cbfff640 1 --2- 172.21.15.67:0/640870172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c244d0 0x55f839fe8920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.602+0000 7fc0cbfff640 1 -- 172.21.15.67:0/640870172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c1c2f0 msgr2=0x55f839fe83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.602+0000 7fc0cbfff640 1 --2- 172.21.15.67:0/640870172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c1c2f0 0x55f839fe83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.602+0000 7fc0cbfff640 1 -- 172.21.15.67:0/640870172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f839ef39a0 con 0x55f839c451b0 2024-07-31T15:08:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.602+0000 7fc0c8ff9640 1 -- 172.21.15.67:0/815819844 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc0bc018020 con 0x7fc0cc10a330 2024-07-31T15:08:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.603+0000 7fc0cbfff640 1 --2- 172.21.15.67:0/640870172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c451b0 0x55f839fe7ea0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4031520 tx=0x7fc0c4031550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2febf9e39b165ab7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.611+0000 7fc0c8ff9640 1 --2- 172.21.15.67:0/815819844 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc0b40b3f40 0x7fc0b40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.611+0000 7fc0c8ff9640 1 -- 172.21.15.67:0/815819844 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fc0bc0e3fe0 con 0x7fc0cc10a330 2024-07-31T15:08:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.613+0000 7fc0cb7fe640 1 --2- 172.21.15.67:0/815819844 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc0b40b3f40 0x7fc0b40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.617+0000 7fc0cb7fe640 1 --2- 172.21.15.67:0/815819844 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc0b40b3f40 0x7fc0b40b6450 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55f839fe6b20 tx=0x7fc0bc0046c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.618+0000 7fc0b1ffb640 1 -- 172.21.15.67:0/640870172 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4061070 con 0x55f839c451b0 2024-07-31T15:08:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.618+0000 7fc0b1ffb640 1 -- 172.21.15.67:0/640870172 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c4036040 con 0x55f839c451b0 2024-07-31T15:08:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.618+0000 7fc0b1ffb640 1 -- 172.21.15.67:0/640870172 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4004440 con 0x55f839c451b0 2024-07-31T15:08:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.619+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/640870172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c451b0 msgr2=0x55f839fe7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.619+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/640870172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c451b0 0x55f839fe7ea0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4031520 tx=0x7fc0c4031550 comp rx=0 tx=0).stop 2024-07-31T15:08:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.621+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/640870172 shutdown_connections 2024-07-31T15:08:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.621+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/640870172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c244d0 0x55f839fe8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.621+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/640870172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c1c2f0 0x55f839fe83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.621+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/640870172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c451b0 0x55f839fe7ea0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.621+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/640870172 >> 172.21.15.67:0/640870172 conn(0x55f839c39300 msgr2=0x55f839fe5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.624+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/640870172 shutdown_connections 2024-07-31T15:08:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.628+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/640870172 wait complete. 2024-07-31T15:08:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.630+0000 7fc0d8dbe740 1 -- start start 2024-07-31T15:08:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.630+0000 7fc0d8dbe740 1 Processor -- start 2024-07-31T15:08:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.631+0000 7fc0d8dbe740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c1c2f0 0x55f839ff3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.631+0000 7fc0d8dbe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c244d0 0x55f839ff44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.631+0000 7fc0d8dbe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c451b0 0x55f839ff94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.631+0000 7fc0d8dbe740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f839e86ab0 con 0x55f839c1c2f0 2024-07-31T15:08:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.631+0000 7fc0d8dbe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f839ed4cf0 con 0x55f839c244d0 2024-07-31T15:08:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.631+0000 7fc0d8dbe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f839fee3b0 con 0x55f839c451b0 2024-07-31T15:08:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.631+0000 7fc0d8dbe740 10 client.0 _ll_register_callbacks cb 0x55f839f469f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.632+0000 7fc0caffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c244d0 0x55f839ff44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.632+0000 7fc0caffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c244d0 0x55f839ff44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48296/0 (socket says 172.21.15.67:48296) 2024-07-31T15:08:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.632+0000 7fc0caffd640 1 -- 172.21.15.67:0/2096754028 learned_addr learned my addr 172.21.15.67:0/2096754028 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.632+0000 7fc0cbfff640 1 --2- 172.21.15.67:0/2096754028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c451b0 0x55f839ff94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.632+0000 7fc0cbfff640 1 -- 172.21.15.67:0/2096754028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c244d0 msgr2=0x55f839ff44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.632+0000 7fc0cbfff640 1 --2- 172.21.15.67:0/2096754028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c244d0 0x55f839ff44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.632+0000 7fc0cbfff640 1 -- 172.21.15.67:0/2096754028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c1c2f0 msgr2=0x55f839ff3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.632+0000 7fc0cbfff640 1 --2- 172.21.15.67:0/2096754028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c1c2f0 0x55f839ff3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.632+0000 7fc0cbfff640 1 -- 172.21.15.67:0/2096754028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f839cb0790 con 0x55f839c451b0 2024-07-31T15:08:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.633+0000 7fc0cbfff640 1 --2- 172.21.15.67:0/2096754028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c451b0 0x55f839ff94e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4031fd0 tx=0x7fc0c4009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8700ff2208da7bda server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.646+0000 7fc0aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:08:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.646+0000 7fc0aaffd640 1 -- 172.21.15.67:0/2096754028 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4061070 con 0x55f839c451b0 2024-07-31T15:08:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.646+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/2096754028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0c4036040 con 0x55f839c451b0 2024-07-31T15:08:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.647+0000 7fc0aaffd640 1 -- 172.21.15.67:0/2096754028 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fc0c40041c0 con 0x55f839c451b0 2024-07-31T15:08:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.647+0000 7fc0aaffd640 1 -- 172.21.15.67:0/2096754028 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fc0c40043c0 con 0x55f839c451b0 2024-07-31T15:08:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.647+0000 7fc0d8dbe740 10 client.28486 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.647+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/2096754028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55f839d29620 con 0x55f839c451b0 2024-07-31T15:08:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.647+0000 7fc0aaffd640 1 -- 172.21.15.67:0/2096754028 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c4036040 con 0x55f839c451b0 2024-07-31T15:08:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.648+0000 7fc0aaffd640 1 -- 172.21.15.67:0/2096754028 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fc0c4030d30 con 0x55f839c451b0 2024-07-31T15:08:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.648+0000 7fc0d8dbe740 10 client.28486 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.648+0000 7fc0d8dbe740 10 client.28486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:08:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.648+0000 7fc0d8dbe740 20 client.28486 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.648+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/2096754028 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f839c03d80 0x55f83a0057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.648+0000 7fc0d8dbe740 4 client.28486 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:08:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.648+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/2096754028 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f839f5f300 con 0x55f839c03d80 2024-07-31T15:08:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.648+0000 7fc0caffd640 1 --2- 172.21.15.67:0/2096754028 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f839c03d80 0x55f83a0057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.649+0000 7fc0aaffd640 10 client.28486 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.650+0000 7fc0caffd640 1 --2- 172.21.15.67:0/2096754028 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f839c03d80 0x55f83a0057e0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54efa7232fced87c server_cookie=6f17c70147b9126 in_seq=0 out_seq=0 2024-07-31T15:08:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.650+0000 7fc0aaffd640 1 -- 172.21.15.67:0/2096754028 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f839f5f300 con 0x55f839c03d80 2024-07-31T15:08:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.650+0000 7fc0aaffd640 10 client.28486 handle_command_reply: tid=0 2024-07-31T15:08:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.650+0000 7fc0d8dbe740 1 client.28486 shutdown 2024-07-31T15:08:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.662+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/2096754028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c451b0 msgr2=0x55f839ff94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.662+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/2096754028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c451b0 0x55f839ff94e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4031fd0 tx=0x7fc0c4009880 comp rx=0 tx=0).stop 2024-07-31T15:08:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.662+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/2096754028 shutdown_connections 2024-07-31T15:08:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.662+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/2096754028 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f839c03d80 0x55f83a0057e0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.662+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/2096754028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c451b0 0x55f839ff94e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.662+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/2096754028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c244d0 0x55f839ff44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.662+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/2096754028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c1c2f0 0x55f839ff3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.662+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/2096754028 >> 172.21.15.67:0/2096754028 conn(0x55f839c39300 msgr2=0x55f839fe9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.662+0000 7fc0aaffd640 0 client.28486 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.667+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/2096754028 shutdown_connections 2024-07-31T15:08:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.667+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/2096754028 wait complete. 2024-07-31T15:08:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.667+0000 7fc0d8dbe740 20 client.28486 trim_cache size 0 max 16384 2024-07-31T15:08:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.708+0000 7fc0d8dbe740 1 Processor -- start 2024-07-31T15:08:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.715+0000 7fc0d8dbe740 1 -- start start 2024-07-31T15:08:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.715+0000 7fc0d8dbe740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c1c2f0 0x55f839ff2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.715+0000 7fc0d8dbe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c244d0 0x55f839ff2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.715+0000 7fc0d8dbe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c451b0 0x55f83a00c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.715+0000 7fc0d8dbe740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b8010090 con 0x55f839c1c2f0 2024-07-31T15:08:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.715+0000 7fc0d8dbe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b8011bd0 con 0x55f839c244d0 2024-07-31T15:08:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.715+0000 7fc0d8dbe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b8010760 con 0x55f839c451b0 2024-07-31T15:08:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.715+0000 7fc0caffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c244d0 0x55f839ff2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.716+0000 7fc0caffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c244d0 0x55f839ff2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48304/0 (socket says 172.21.15.67:48304) 2024-07-31T15:08:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.716+0000 7fc0caffd640 1 -- 172.21.15.67:0/3342759930 learned_addr learned my addr 172.21.15.67:0/3342759930 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.716+0000 7fc0caffd640 1 -- 172.21.15.67:0/3342759930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c451b0 msgr2=0x55f83a00c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.716+0000 7fc0caffd640 1 --2- 172.21.15.67:0/3342759930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c451b0 0x55f83a00c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.716+0000 7fc0caffd640 1 -- 172.21.15.67:0/3342759930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c1c2f0 msgr2=0x55f839ff2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.716+0000 7fc0caffd640 1 --2- 172.21.15.67:0/3342759930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c1c2f0 0x55f839ff2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.716+0000 7fc0caffd640 1 -- 172.21.15.67:0/3342759930 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f839c03fb0 con 0x55f839c244d0 2024-07-31T15:08:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.716+0000 7fc0caffd640 1 --2- 172.21.15.67:0/3342759930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c244d0 0x55f839ff2a80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00091e0 tx=0x7fc0c0009210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=470a7d4267228c31 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.717+0000 7fc0b17fa640 1 -- 172.21.15.67:0/3342759930 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0015a90 con 0x55f839c244d0 2024-07-31T15:08:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.717+0000 7fc0b17fa640 1 -- 172.21.15.67:0/3342759930 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c0015c30 con 0x55f839c244d0 2024-07-31T15:08:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.717+0000 7fc0b17fa640 1 -- 172.21.15.67:0/3342759930 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0006810 con 0x55f839c244d0 2024-07-31T15:08:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.720+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/3342759930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c244d0 msgr2=0x55f839ff2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.720+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/3342759930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c244d0 0x55f839ff2a80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00091e0 tx=0x7fc0c0009210 comp rx=0 tx=0).stop 2024-07-31T15:08:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.720+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/3342759930 shutdown_connections 2024-07-31T15:08:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.720+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/3342759930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c451b0 0x55f83a00c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.720+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/3342759930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c244d0 0x55f839ff2a80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.720+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/3342759930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c1c2f0 0x55f839ff2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.720+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/3342759930 >> 172.21.15.67:0/3342759930 conn(0x55f839c39300 msgr2=0x55f839feee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.735+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/3342759930 shutdown_connections 2024-07-31T15:08:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.735+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/3342759930 wait complete. 2024-07-31T15:08:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.735+0000 7fc0d8dbe740 1 -- start start 2024-07-31T15:08:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.736+0000 7fc0d8dbe740 1 Processor -- start 2024-07-31T15:08:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.736+0000 7fc0d8dbe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c1c2f0 0x55f83a009790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.736+0000 7fc0d8dbe740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c244d0 0x55f83a009d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.736+0000 7fc0d8dbe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c451b0 0x55f83a06b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.736+0000 7fc0d8dbe740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b8011500 con 0x55f839c244d0 2024-07-31T15:08:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.736+0000 7fc0d8dbe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b8010e30 con 0x55f839c451b0 2024-07-31T15:08:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.736+0000 7fc0d8dbe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b800e090 con 0x55f839c1c2f0 2024-07-31T15:08:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.736+0000 7fc0d8dbe740 10 client.0 _ll_register_callbacks cb 0x7fc0c4031470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.737+0000 7fc0cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c1c2f0 0x55f83a009790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.737+0000 7fc0cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c1c2f0 0x55f83a009790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:46860/0 (socket says 172.21.15.67:46860) 2024-07-31T15:08:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.737+0000 7fc0cbfff640 1 -- 172.21.15.67:0/1324220406 learned_addr learned my addr 172.21.15.67:0/1324220406 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.737+0000 7fc0cbfff640 1 --2- 172.21.15.67:0/1324220406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c451b0 0x55f83a06b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.737+0000 7fc0cbfff640 1 -- 172.21.15.67:0/1324220406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c451b0 msgr2=0x55f83a06b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.737+0000 7fc0cbfff640 1 --2- 172.21.15.67:0/1324220406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c451b0 0x55f83a06b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.737+0000 7fc0cbfff640 1 -- 172.21.15.67:0/1324220406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c244d0 msgr2=0x55f83a009d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.737+0000 7fc0cbfff640 1 --2- 172.21.15.67:0/1324220406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c244d0 0x55f83a009d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.737+0000 7fc0cbfff640 1 -- 172.21.15.67:0/1324220406 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83a06b8f0 con 0x55f839c1c2f0 2024-07-31T15:08:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.738+0000 7fc0cbfff640 1 --2- 172.21.15.67:0/1324220406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c1c2f0 0x55f83a009790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009620 tx=0x7fc0c40021d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9685274da5d34ba server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.751+0000 7fc0aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:08:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.751+0000 7fc0aaffd640 1 -- 172.21.15.67:0/1324220406 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4061070 con 0x55f839c1c2f0 2024-07-31T15:08:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.751+0000 7fc0aaffd640 1 -- 172.21.15.67:0/1324220406 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fc0c4066d20 con 0x55f839c1c2f0 2024-07-31T15:08:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.751+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/1324220406 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0c0015c30 con 0x55f839c1c2f0 2024-07-31T15:08:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.752+0000 7fc0aaffd640 1 -- 172.21.15.67:0/1324220406 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fc0c408d050 con 0x55f839c1c2f0 2024-07-31T15:08:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.759+0000 7fc0d8dbe740 10 client.28489 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.759+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/1324220406 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55f83a06ca20 con 0x55f839c1c2f0 2024-07-31T15:08:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.759+0000 7fc0aaffd640 1 -- 172.21.15.67:0/1324220406 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c0015c30 con 0x55f839c1c2f0 2024-07-31T15:08:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.759+0000 7fc0aaffd640 1 -- 172.21.15.67:0/1324220406 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fc0c4030ae0 con 0x55f839c1c2f0 2024-07-31T15:08:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.759+0000 7fc0d8dbe740 10 client.28489 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.759+0000 7fc0d8dbe740 10 client.28489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:08:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.759+0000 7fc0d8dbe740 20 client.28489 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.759+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/1324220406 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f839bd91c0 0x55f83a06f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.759+0000 7fc0d8dbe740 4 client.28489 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:08:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.759+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/1324220406 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f839fed780 con 0x55f839bd91c0 2024-07-31T15:08:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.760+0000 7fc0caffd640 1 --2- 172.21.15.67:0/1324220406 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f839bd91c0 0x55f83a06f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.760+0000 7fc0aaffd640 10 client.28489 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:12.760+0000 7fc0caffd640 1 --2- 172.21.15.67:0/1324220406 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f839bd91c0 0x55f83a06f210 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35b383f06268ef67 server_cookie=4bf8a54eb27848b7 in_seq=0 out_seq=0 2024-07-31T15:08:12.867 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nfnetlink.o 2024-07-31T15:08:13.267 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma-buf/dma-fence-chain.o 2024-07-31T15:08:13.481 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:13 smithi067 ceph-mon[32750]: pgmap v943: 129 pgs: 10 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 86 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 323 KiB/s wr, 154 op/s 2024-07-31T15:08:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:13 smithi077 ceph-mon[37327]: pgmap v943: 129 pgs: 10 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 86 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 323 KiB/s wr, 154 op/s 2024-07-31T15:08:13.499 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/virtio/virtgpu_vq.o 2024-07-31T15:08:13.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:13 smithi073 ceph-mon[36714]: pgmap v943: 129 pgs: 10 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 86 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 323 KiB/s wr, 154 op/s 2024-07-31T15:08:13.910 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nfnetlink_log.o 2024-07-31T15:08:13.939 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:08:14.157 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_suspend.o 2024-07-31T15:08:14.309 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/ipv4/netfilter/built-in.a 2024-07-31T15:08:14.404 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma-buf/dma-fence-unwrap.o 2024-07-31T15:08:14.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:08:14.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:08:14.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:14 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:08:14.452 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/protocol.o 2024-07-31T15:08:14.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:08:14.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:08:14.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:14 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:08:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:08:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:08:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:14 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:08:14.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.947+0000 7f465bfff640 1 -- 172.21.15.67:0/2309978220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f465c115f40 msgr2=0x7f465c118360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:14.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.948+0000 7f465bfff640 1 --2- 172.21.15.67:0/2309978220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f465c115f40 0x7f465c118360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f464c009900 tx=0x7f464c02a4d0 comp rx=0 tx=0).stop 2024-07-31T15:08:14.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.955+0000 7f465bfff640 1 -- 172.21.15.67:0/2309978220 shutdown_connections 2024-07-31T15:08:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.955+0000 7f465bfff640 1 --2- 172.21.15.67:0/2309978220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f465c115f40 0x7f465c118360 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.955+0000 7f465bfff640 1 --2- 172.21.15.67:0/2309978220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465c10a740 0x7f465c115a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.955+0000 7f465bfff640 1 --2- 172.21.15.67:0/2309978220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f465c109df0 0x7f465c10a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.955+0000 7f465bfff640 1 -- 172.21.15.67:0/2309978220 >> 172.21.15.67:0/2309978220 conn(0x7f465c06e420 msgr2=0x7f465c06e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.955+0000 7f465bfff640 1 -- 172.21.15.67:0/2309978220 shutdown_connections 2024-07-31T15:08:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.955+0000 7f465bfff640 1 -- 172.21.15.67:0/2309978220 wait complete. 2024-07-31T15:08:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.957+0000 7f465bfff640 1 Processor -- start 2024-07-31T15:08:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.957+0000 7f465bfff640 1 -- start start 2024-07-31T15:08:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.958+0000 7f465bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f465c109df0 0x7f465c1a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.958+0000 7f465bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f465c10a740 0x7f465c1a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.958+0000 7f465affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f465c109df0 0x7f465c1a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.959+0000 7f465affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f465c109df0 0x7f465c1a25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48322/0 (socket says 172.21.15.67:48322) 2024-07-31T15:08:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.959+0000 7f465affd640 1 -- 172.21.15.67:0/2451925480 learned_addr learned my addr 172.21.15.67:0/2451925480 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.959+0000 7f465bfff640 1 --2- 172.21.15.67:0/2451925480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465c115f40 0x7f465c1a7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.959+0000 7f465a7fc640 1 --2- 172.21.15.67:0/2451925480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f465c10a740 0x7f465c1a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.959+0000 7f465bfff640 1 -- 172.21.15.67:0/2451925480 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f465c11bbb0 con 0x7f465c115f40 2024-07-31T15:08:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.959+0000 7f465bfff640 1 -- 172.21.15.67:0/2451925480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f465c11ba30 con 0x7f465c109df0 2024-07-31T15:08:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.959+0000 7f465bfff640 1 -- 172.21.15.67:0/2451925480 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f465c11bd30 con 0x7f465c10a740 2024-07-31T15:08:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.964+0000 7f465affd640 1 -- 172.21.15.67:0/2451925480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f465c10a740 msgr2=0x7f465c1a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.964+0000 7f465affd640 1 --2- 172.21.15.67:0/2451925480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f465c10a740 0x7f465c1a2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.964+0000 7f465affd640 1 -- 172.21.15.67:0/2451925480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465c115f40 msgr2=0x7f465c1a7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.964+0000 7f465affd640 1 --2- 172.21.15.67:0/2451925480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465c115f40 0x7f465c1a7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.964+0000 7f465affd640 1 -- 172.21.15.67:0/2451925480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465c1a8530 con 0x7f465c109df0 2024-07-31T15:08:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.966+0000 7f465affd640 1 --2- 172.21.15.67:0/2451925480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f465c109df0 0x7f465c1a25c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4654009fa0 tx=0x7f465400be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.970+0000 7f466084a640 1 -- 172.21.15.67:0/2451925480 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465401d070 con 0x7f465c109df0 2024-07-31T15:08:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.970+0000 7f466084a640 1 -- 172.21.15.67:0/2451925480 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46540188f0 con 0x7f465c109df0 2024-07-31T15:08:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.970+0000 7f465bfff640 1 -- 172.21.15.67:0/2451925480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465c1a8820 con 0x7f465c109df0 2024-07-31T15:08:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.970+0000 7f465bfff640 1 -- 172.21.15.67:0/2451925480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f465c10b790 con 0x7f465c109df0 2024-07-31T15:08:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.972+0000 7f466084a640 1 -- 172.21.15.67:0/2451925480 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654007980 con 0x7f465c109df0 2024-07-31T15:08:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.976+0000 7f465bfff640 1 -- 172.21.15.67:0/2451925480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465c10ae00 con 0x7f465c109df0 2024-07-31T15:08:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.979+0000 7f466084a640 1 -- 172.21.15.67:0/2451925480 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4654020070 con 0x7f465c109df0 2024-07-31T15:08:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.981+0000 7f466084a640 1 --2- 172.21.15.67:0/2451925480 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46400b3e20 0x7f46400b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.981+0000 7f466084a640 1 -- 172.21.15.67:0/2451925480 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f46540e7b10 con 0x7f465c109df0 2024-07-31T15:08:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.982+0000 7f465a7fc640 1 --2- 172.21.15.67:0/2451925480 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46400b3e20 0x7f46400b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.983+0000 7f465a7fc640 1 --2- 172.21.15.67:0/2451925480 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46400b3e20 0x7f46400b6330 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f465c117c30 tx=0x7f4648009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:14.986+0000 7f466084a640 1 -- 172.21.15.67:0/2451925480 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f46540aeb10 con 0x7f465c109df0 2024-07-31T15:08:15.036 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_core.o 2024-07-31T15:08:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.152+0000 7fc0aaffd640 1 -- 172.21.15.67:0/1324220406 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f839fed780 con 0x55f839bd91c0 2024-07-31T15:08:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.152+0000 7fc0aaffd640 10 client.28489 handle_command_reply: tid=0 2024-07-31T15:08:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.155+0000 7fc0d8dbe740 1 client.28489 shutdown 2024-07-31T15:08:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.155+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/1324220406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c1c2f0 msgr2=0x55f83a009790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.155+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/1324220406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c1c2f0 0x55f83a009790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009620 tx=0x7fc0c40021d0 comp rx=0 tx=0).stop 2024-07-31T15:08:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.155+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/1324220406 shutdown_connections 2024-07-31T15:08:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.155+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/1324220406 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f839bd91c0 0x55f83a06f210 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.155+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/1324220406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f839c451b0 0x55f83a06b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.155+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/1324220406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f839c244d0 0x55f83a009d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.155+0000 7fc0d8dbe740 1 --2- 172.21.15.67:0/1324220406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839c1c2f0 0x55f83a009790 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.155+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/1324220406 >> 172.21.15.67:0/1324220406 conn(0x55f839c39300 msgr2=0x55f839ff2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.155+0000 7fc0aaffd640 0 client.28489 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.155+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/1324220406 shutdown_connections 2024-07-31T15:08:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.157+0000 7fc0d8dbe740 1 -- 172.21.15.67:0/1324220406 wait complete. 2024-07-31T15:08:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.157+0000 7fc0d8dbe740 20 client.28489 trim_cache size 0 max 16384 2024-07-31T15:08:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:08:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:08:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:08:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:08:15.161 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma-buf/dma-resv.o 2024-07-31T15:08:15.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.174+0000 7fc0d0c60640 1 -- 172.21.15.67:0/815819844 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc0b40b3f40 msgr2=0x7fc0b40b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:15.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.174+0000 7fc0d0c60640 1 --2- 172.21.15.67:0/815819844 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc0b40b3f40 0x7fc0b40b6450 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55f839fe6b20 tx=0x7fc0bc0046c0 comp rx=0 tx=0).stop 2024-07-31T15:08:15.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.174+0000 7fc0d0c60640 1 -- 172.21.15.67:0/815819844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0cc10a330 msgr2=0x7fc0cc1a8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:15.175 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.174+0000 7fc0d0c60640 1 --2- 172.21.15.67:0/815819844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0cc10a330 0x7fc0cc1a8ba0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00b450 tx=0x7fc0bc00b910 comp rx=0 tx=0).stop 2024-07-31T15:08:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.178+0000 7fc0d0c60640 1 -- 172.21.15.67:0/815819844 shutdown_connections 2024-07-31T15:08:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.178+0000 7fc0d0c60640 1 --2- 172.21.15.67:0/815819844 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc0b40b3f40 0x7fc0b40b6450 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.178+0000 7fc0d0c60640 1 --2- 172.21.15.67:0/815819844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0cc11c5d0 0x7fc0cc1ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.178+0000 7fc0d0c60640 1 --2- 172.21.15.67:0/815819844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0cc111650 0x7fc0cc1a9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.178+0000 7fc0d0c60640 1 --2- 172.21.15.67:0/815819844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0cc10a330 0x7fc0cc1a8ba0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.178+0000 7fc0d0c60640 1 -- 172.21.15.67:0/815819844 >> 172.21.15.67:0/815819844 conn(0x7fc0cc06e320 msgr2=0x7fc0cc11a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.178+0000 7fc0d0c60640 1 -- 172.21.15.67:0/815819844 shutdown_connections 2024-07-31T15:08:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:15.178+0000 7fc0d0c60640 1 -- 172.21.15.67:0/815819844 wait complete. 2024-07-31T15:08:15.223 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_standalone.o 2024-07-31T15:08:15.252 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_switcheroo.o 2024-07-31T15:08:15.267 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/virtio/virtgpu_fence.o 2024-07-31T15:08:15.276 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:15 smithi067 ceph-mon[32750]: pgmap v944: 129 pgs: 8 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 94 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 554 KiB/s wr, 85 op/s 2024-07-31T15:08:15.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:15.512+0000 7f465bfff640 1 -- 172.21.15.67:0/2451925480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f465c10baa0 con 0x7f465c109df0 2024-07-31T15:08:15.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:15.518+0000 7f466084a640 1 -- 172.21.15.67:0/2451925480 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11962 (secure 0 0 0) 0x7f46540ae410 con 0x7f465c109df0 2024-07-31T15:08:15.521 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:08:15.522 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"btime":"2024-07-31T15:06:04:323642+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:06:04.323492+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28214":{"gid":28214,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1516494761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1516494761},{"type":"v1","addr":"172.21.15.73:6835","nonce":1516494761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:08:15.522 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-07-31T15:08:15.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:15.526+0000 7f46527fc640 1 -- 172.21.15.67:0/2451925480 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46400b3e20 msgr2=0x7f46400b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:15.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:15.526+0000 7f46527fc640 1 --2- 172.21.15.67:0/2451925480 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46400b3e20 0x7f46400b6330 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f465c117c30 tx=0x7f4648009110 comp rx=0 tx=0).stop 2024-07-31T15:08:15.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:15.526+0000 7f46527fc640 1 -- 172.21.15.67:0/2451925480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f465c109df0 msgr2=0x7f465c1a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:15.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:15.526+0000 7f46527fc640 1 --2- 172.21.15.67:0/2451925480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f465c109df0 0x7f465c1a25c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4654009fa0 tx=0x7f465400be20 comp rx=0 tx=0).stop 2024-07-31T15:08:15.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:15.531+0000 7f46527fc640 1 -- 172.21.15.67:0/2451925480 shutdown_connections 2024-07-31T15:08:15.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:15.531+0000 7f46527fc640 1 --2- 172.21.15.67:0/2451925480 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f46400b3e20 0x7f46400b6330 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:15.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:15.531+0000 7f46527fc640 1 --2- 172.21.15.67:0/2451925480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465c115f40 0x7f465c1a7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:15.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:15.531+0000 7f46527fc640 1 --2- 172.21.15.67:0/2451925480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f465c10a740 0x7f465c1a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:15.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:15.531+0000 7f46527fc640 1 --2- 172.21.15.67:0/2451925480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f465c109df0 0x7f465c1a25c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:15.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:15.531+0000 7f46527fc640 1 -- 172.21.15.67:0/2451925480 >> 172.21.15.67:0/2451925480 conn(0x7f465c06e420 msgr2=0x7f465c10fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:15.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:15.536+0000 7f46527fc640 1 -- 172.21.15.67:0/2451925480 shutdown_connections 2024-07-31T15:08:15.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:15.536+0000 7f46527fc640 1 -- 172.21.15.67:0/2451925480 wait complete. 2024-07-31T15:08:15.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:15 smithi073 ceph-mon[36714]: pgmap v944: 129 pgs: 8 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 94 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 554 KiB/s wr, 85 op/s 2024-07-31T15:08:15.732 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/ip_input.o 2024-07-31T15:08:15.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:15 smithi077 ceph-mon[37327]: pgmap v944: 129 pgs: 8 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 94 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 554 KiB/s wr, 85 op/s 2024-07-31T15:08:15.938 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/regmap/regcache-rbtree.o 2024-07-31T15:08:16.109 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/dma-buf/sync_file.o 2024-07-31T15:08:16.453 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_sysfs.o 2024-07-31T15:08:16.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:16 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2451925480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:08:16.581 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/virtio/virtgpu_object.o 2024-07-31T15:08:16.596 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/ip_fragment.o 2024-07-31T15:08:16.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:16 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2451925480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:08:16.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:16 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2451925480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:08:16.925 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/cxl/core/built-in.a 2024-07-31T15:08:16.928 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-07-31T15:08:16.928 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-07-31T15:08:16.943 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/dma-buf/built-in.a 2024-07-31T15:08:16.950 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/cxl/built-in.a 2024-07-31T15:08:16.972 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/regmap/regcache-flat.o 2024-07-31T15:08:16.988 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/regmap/regcache-maple.o 2024-07-31T15:08:17.046 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_expect.o 2024-07-31T15:08:17.449 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/imx/built-in.a 2024-07-31T15:08:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:17 smithi067 ceph-mon[32750]: pgmap v945: 129 pgs: 5 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 109 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 739 KiB/s wr, 129 op/s 2024-07-31T15:08:17.594 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/i2c/built-in.a 2024-07-31T15:08:17.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:17 smithi073 ceph-mon[36714]: pgmap v945: 129 pgs: 5 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 109 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 739 KiB/s wr, 129 op/s 2024-07-31T15:08:17.667 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/virtio/virtgpu_debugfs.o 2024-07-31T15:08:17.675 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/panel/built-in.a 2024-07-31T15:08:17.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:17 smithi077 ceph-mon[37327]: pgmap v945: 129 pgs: 5 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 109 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 739 KiB/s wr, 129 op/s 2024-07-31T15:08:17.748 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/bridge/analogix/built-in.a 2024-07-31T15:08:17.777 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/hisilicon/built-in.a 2024-07-31T15:08:17.787 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/bridge/cadence/built-in.a 2024-07-31T15:08:17.790 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_helper.o 2024-07-31T15:08:17.818 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_utils.o 2024-07-31T15:08:17.822 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/mxsfb/built-in.a 2024-07-31T15:08:17.849 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/bridge/imx/built-in.a 2024-07-31T15:08:17.854 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:08:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.657+0000 7fe8f04b3640 1 -- 172.21.15.67:0/3383508346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8e811c660 msgr2=0x7fe8e811eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.657+0000 7fe8f04b3640 1 --2- 172.21.15.67:0/3383508346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8e811c660 0x7fe8e811eb70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009860 tx=0x7fe8dc02a360 comp rx=0 tx=0).stop 2024-07-31T15:08:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.658+0000 7fe8f04b3640 1 -- 172.21.15.67:0/3383508346 shutdown_connections 2024-07-31T15:08:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.658+0000 7fe8f04b3640 1 --2- 172.21.15.67:0/3383508346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8e811c660 0x7fe8e811eb70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.658+0000 7fe8f04b3640 1 --2- 172.21.15.67:0/3383508346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8e810a410 0x7fe8e810a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.658+0000 7fe8f04b3640 1 --2- 172.21.15.67:0/3383508346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8e8111970 0x7fe8e8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.658+0000 7fe8f04b3640 1 -- 172.21.15.67:0/3383508346 >> 172.21.15.67:0/3383508346 conn(0x7fe8e80708f0 msgr2=0x7fe8e8070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.667+0000 7fe8f04b3640 1 -- 172.21.15.67:0/3383508346 shutdown_connections 2024-07-31T15:08:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.671+0000 7fe8f04b3640 1 -- 172.21.15.67:0/3383508346 wait complete. 2024-07-31T15:08:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.672+0000 7fe8f04b3640 1 Processor -- start 2024-07-31T15:08:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.673+0000 7fe8f04b3640 1 -- start start 2024-07-31T15:08:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.673+0000 7fe8f04b3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8e810a410 0x7fe8e8117f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.673+0000 7fe8f04b3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8e8111970 0x7fe8e8118530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.673+0000 7fe8f04b3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8e81a5910 0x7fe8e81a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.673+0000 7fe8f04b3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81146c0 con 0x7fe8e8111970 2024-07-31T15:08:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.673+0000 7fe8f04b3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8114540 con 0x7fe8e810a410 2024-07-31T15:08:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.673+0000 7fe8f04b3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8114840 con 0x7fe8e81a5910 2024-07-31T15:08:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.677+0000 7fe8eda4e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8e8111970 0x7fe8e8118530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.677+0000 7fe8eda4e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8e8111970 0x7fe8e8118530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36840/0 (socket says 172.21.15.67:36840) 2024-07-31T15:08:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.677+0000 7fe8eda4e640 1 -- 172.21.15.67:0/3726232464 learned_addr learned my addr 172.21.15.67:0/3726232464 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.677+0000 7fe8eea50640 1 --2- 172.21.15.67:0/3726232464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8e81a5910 0x7fe8e81a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.680+0000 7fe8eda4e640 1 -- 172.21.15.67:0/3726232464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8e81a5910 msgr2=0x7fe8e81a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.680+0000 7fe8eda4e640 1 --2- 172.21.15.67:0/3726232464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8e81a5910 0x7fe8e81a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.680+0000 7fe8eda4e640 1 -- 172.21.15.67:0/3726232464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8e810a410 msgr2=0x7fe8e8117f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:08:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.680+0000 7fe8eda4e640 1 --2- 172.21.15.67:0/3726232464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8e810a410 0x7fe8e8117f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.680+0000 7fe8eda4e640 1 -- 172.21.15.67:0/3726232464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e81a6560 con 0x7fe8e8111970 2024-07-31T15:08:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.681+0000 7fe8eda4e640 1 --2- 172.21.15.67:0/3726232464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8e8111970 0x7fe8e8118530 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe8e001fde0 tx=0x7fe8e001d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.686+0000 7fe8db7fe640 1 -- 172.21.15.67:0/3726232464 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e001de60 con 0x7fe8e8111970 2024-07-31T15:08:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.686+0000 7fe8db7fe640 1 -- 172.21.15.67:0/3726232464 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e001ae50 con 0x7fe8e8111970 2024-07-31T15:08:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.686+0000 7fe8db7fe640 1 -- 172.21.15.67:0/3726232464 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e001c580 con 0x7fe8e8111970 2024-07-31T15:08:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.688+0000 7fe8f04b3640 1 -- 172.21.15.67:0/3726232464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e81aca70 con 0x7fe8e8111970 2024-07-31T15:08:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.688+0000 7fe8f04b3640 1 -- 172.21.15.67:0/3726232464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e81aced0 con 0x7fe8e8111970 2024-07-31T15:08:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.695+0000 7fe8f6a04740 1 Processor -- start 2024-07-31T15:08:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.696+0000 7fe8db7fe640 1 -- 172.21.15.67:0/3726232464 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe8e002e020 con 0x7fe8e8111970 2024-07-31T15:08:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.698+0000 7fe8f6a04740 1 -- start start 2024-07-31T15:08:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.698+0000 7fe8f6a04740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef73070 0x558e8f318570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.698+0000 7fe8f6a04740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef4a1b0 0x558e8f318ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.698+0000 7fe8f6a04740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef52390 0x558e8f318ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.698+0000 7fe8f6a04740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558e8f28ebb0 con 0x558e8ef52390 2024-07-31T15:08:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.698+0000 7fe8f6a04740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558e8f27e0b0 con 0x558e8ef4a1b0 2024-07-31T15:08:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.698+0000 7fe8f6a04740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558e8f2820f0 con 0x558e8ef73070 2024-07-31T15:08:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.699+0000 7fe8ee24f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef4a1b0 0x558e8f318ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.699+0000 7fe8ee24f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef4a1b0 0x558e8f318ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48348/0 (socket says 172.21.15.67:48348) 2024-07-31T15:08:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.699+0000 7fe8ee24f640 1 -- 172.21.15.67:0/2101686587 learned_addr learned my addr 172.21.15.67:0/2101686587 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.701+0000 7fe8db7fe640 1 --2- 172.21.15.67:0/3726232464 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe8d40b3f40 0x7fe8d40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.701+0000 7fe8eda4e640 1 --2- 172.21.15.67:0/2101686587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef52390 0x558e8f318ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.703+0000 7fe8eda4e640 1 -- 172.21.15.67:0/2101686587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef73070 msgr2=0x558e8f318570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.703+0000 7fe8db7fe640 1 -- 172.21.15.67:0/3726232464 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fe8e00f5070 con 0x7fe8e8111970 2024-07-31T15:08:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.700+0000 7fe8eea50640 1 --2- 172.21.15.67:0/2101686587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef73070 0x558e8f318570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.704+0000 7fe8eda4e640 1 --2- 172.21.15.67:0/2101686587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef73070 0x558e8f318570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.704+0000 7fe8eda4e640 1 -- 172.21.15.67:0/2101686587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef4a1b0 msgr2=0x558e8f318ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.704+0000 7fe8eda4e640 1 --2- 172.21.15.67:0/2101686587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef4a1b0 0x558e8f318ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.704+0000 7fe8eda4e640 1 -- 172.21.15.67:0/2101686587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e8f224b90 con 0x558e8ef52390 2024-07-31T15:08:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.704+0000 7fe8eea50640 1 --2- 172.21.15.67:0/2101686587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef73070 0x558e8f318570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:08:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.704+0000 7fe8eea50640 1 --2- 172.21.15.67:0/3726232464 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe8d40b3f40 0x7fe8d40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.705+0000 7fe8eda4e640 1 --2- 172.21.15.67:0/2101686587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef52390 0x558e8f318ff0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00f23a0 tx=0x7fe8e0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfbbbacfbd13b32 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.705+0000 7fe8eea50640 1 --2- 172.21.15.67:0/3726232464 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe8d40b3f40 0x7fe8d40b6450 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0092e0 tx=0x7fe8dc005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.706+0000 7fe8d8d99640 1 -- 172.21.15.67:0/2101686587 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00bb860 con 0x558e8ef52390 2024-07-31T15:08:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.706+0000 7fe8d8d99640 1 -- 172.21.15.67:0/2101686587 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e00bfe60 con 0x558e8ef52390 2024-07-31T15:08:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.707+0000 7fe8f6a04740 1 -- 172.21.15.67:0/2101686587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef52390 msgr2=0x558e8f318ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.707+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/2101686587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef52390 0x558e8f318ff0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00f23a0 tx=0x7fe8e0000f40 comp rx=0 tx=0).stop 2024-07-31T15:08:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.709+0000 7fe8f6a04740 1 -- 172.21.15.67:0/2101686587 shutdown_connections 2024-07-31T15:08:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.709+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/2101686587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef52390 0x558e8f318ff0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.709+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/2101686587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef4a1b0 0x558e8f318ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.709+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/2101686587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef73070 0x558e8f318570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.709+0000 7fe8f6a04740 1 -- 172.21.15.67:0/2101686587 >> 172.21.15.67:0/2101686587 conn(0x558e8ef67460 msgr2=0x558e8f316460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:17.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.709+0000 7fe8f6a04740 1 -- 172.21.15.67:0/2101686587 shutdown_connections 2024-07-31T15:08:17.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.709+0000 7fe8f6a04740 1 -- 172.21.15.67:0/2101686587 wait complete. 2024-07-31T15:08:17.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.710+0000 7fe8f6a04740 1 -- start start 2024-07-31T15:08:17.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.710+0000 7fe8f6a04740 1 Processor -- start 2024-07-31T15:08:17.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.711+0000 7fe8f6a04740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef4a1b0 0x558e8f324180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.711+0000 7fe8f6a04740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef52390 0x558e8f324700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.711+0000 7fe8f6a04740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef73070 0x558e8f32ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.711+0000 7fe8f6a04740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558e8f0e1700 con 0x558e8ef73070 2024-07-31T15:08:17.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.711+0000 7fe8f6a04740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558e8f2209e0 con 0x558e8ef4a1b0 2024-07-31T15:08:17.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.711+0000 7fe8f6a04740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558e8f326cf0 con 0x558e8ef52390 2024-07-31T15:08:17.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.711+0000 7fe8f6a04740 10 client.0 _ll_register_callbacks cb 0x558e8f23b4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:17.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.716+0000 7fe8eea50640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef73070 0x558e8f32ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.716+0000 7fe8eea50640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef73070 0x558e8f32ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36844/0 (socket says 172.21.15.67:36844) 2024-07-31T15:08:17.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.716+0000 7fe8eea50640 1 -- 172.21.15.67:0/310612518 learned_addr learned my addr 172.21.15.67:0/310612518 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:17.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.716+0000 7fe8ee24f640 1 --2- 172.21.15.67:0/310612518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef4a1b0 0x558e8f324180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.716+0000 7fe8eda4e640 1 --2- 172.21.15.67:0/310612518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef52390 0x558e8f324700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.717+0000 7fe8ee24f640 1 -- 172.21.15.67:0/310612518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef52390 msgr2=0x558e8f324700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:17.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.717+0000 7fe8ee24f640 1 --2- 172.21.15.67:0/310612518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef52390 0x558e8f324700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.717+0000 7fe8ee24f640 1 -- 172.21.15.67:0/310612518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef73070 msgr2=0x558e8f32ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:17.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.717+0000 7fe8ee24f640 1 --2- 172.21.15.67:0/310612518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef73070 0x558e8f32ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.717+0000 7fe8ee24f640 1 -- 172.21.15.67:0/310612518 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e8efde650 con 0x558e8ef4a1b0 2024-07-31T15:08:17.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.717+0000 7fe8eea50640 1 --2- 172.21.15.67:0/310612518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef73070 0x558e8f32ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:17.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.718+0000 7fe8ee24f640 1 --2- 172.21.15.67:0/310612518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef4a1b0 0x558e8f324180 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009f20 tx=0x7fe8e400e660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=326ba11c268e9d1c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:17.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.718+0000 7fe8d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:08:17.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.720+0000 7fe8d1ffb640 1 -- 172.21.15.67:0/310612518 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4017070 con 0x558e8ef4a1b0 2024-07-31T15:08:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.720+0000 7fe8d1ffb640 1 -- 172.21.15.67:0/310612518 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fe8e4013470 con 0x558e8ef4a1b0 2024-07-31T15:08:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.720+0000 7fe8f6a04740 1 -- 172.21.15.67:0/310612518 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e00bfe60 con 0x558e8ef4a1b0 2024-07-31T15:08:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.721+0000 7fe8d1ffb640 1 -- 172.21.15.67:0/310612518 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe8e4023050 con 0x558e8ef4a1b0 2024-07-31T15:08:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.723+0000 7fe8f6a04740 10 client.28619 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.723+0000 7fe8f6a04740 1 -- 172.21.15.67:0/310612518 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x558e8f332d20 con 0x558e8ef4a1b0 2024-07-31T15:08:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.727+0000 7fe8d1ffb640 1 -- 172.21.15.67:0/310612518 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e00bfe60 con 0x558e8ef4a1b0 2024-07-31T15:08:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.727+0000 7fe8d1ffb640 1 -- 172.21.15.67:0/310612518 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fe8e4012850 con 0x558e8ef4a1b0 2024-07-31T15:08:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.727+0000 7fe8f6a04740 10 client.28619 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.727+0000 7fe8f6a04740 10 client.28619 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:08:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.727+0000 7fe8f6a04740 20 client.28619 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.727+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/310612518 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558e8ef319f0 0x558e8f336690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.727+0000 7fe8f6a04740 4 client.28619 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:08:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.727+0000 7fe8f6a04740 1 -- 172.21.15.67:0/310612518 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e8f290b00 con 0x558e8ef319f0 2024-07-31T15:08:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.730+0000 7fe8eda4e640 1 --2- 172.21.15.67:0/310612518 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558e8ef319f0 0x558e8f336690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.748+0000 7fe8eda4e640 1 --2- 172.21.15.67:0/310612518 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558e8ef319f0 0x558e8f336690 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e077084acbaa3d6a server_cookie=d8e90b73f43d1a84 in_seq=0 out_seq=0 2024-07-31T15:08:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.748+0000 7fe8d1ffb640 10 client.28619 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:08:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.754+0000 7fe8d1ffb640 1 -- 172.21.15.67:0/310612518 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e8f290b00 con 0x558e8ef319f0 2024-07-31T15:08:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.754+0000 7fe8d1ffb640 10 client.28619 handle_command_reply: tid=0 2024-07-31T15:08:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.755+0000 7fe8f6a04740 1 client.28619 shutdown 2024-07-31T15:08:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.756+0000 7fe8f6a04740 1 -- 172.21.15.67:0/310612518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef4a1b0 msgr2=0x558e8f324180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.756+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/310612518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef4a1b0 0x558e8f324180 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009f20 tx=0x7fe8e400e660 comp rx=0 tx=0).stop 2024-07-31T15:08:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.756+0000 7fe8f6a04740 1 -- 172.21.15.67:0/310612518 shutdown_connections 2024-07-31T15:08:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.756+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/310612518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef73070 0x558e8f32ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.756+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/310612518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef52390 0x558e8f324700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.756+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/310612518 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558e8ef319f0 0x558e8f336690 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.756+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/310612518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef4a1b0 0x558e8f324180 secure :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009f20 tx=0x7fe8e400e660 comp rx=0 tx=0).stop 2024-07-31T15:08:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.756+0000 7fe8f6a04740 1 -- 172.21.15.67:0/310612518 >> 172.21.15.67:0/310612518 conn(0x558e8ef67460 msgr2=0x558e8f314c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.762+0000 7fe8d1ffb640 0 client.28619 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:08:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.762+0000 7fe8f6a04740 1 -- 172.21.15.67:0/310612518 shutdown_connections 2024-07-31T15:08:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.762+0000 7fe8f6a04740 1 -- 172.21.15.67:0/310612518 wait complete. 2024-07-31T15:08:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.763+0000 7fe8f6a04740 20 client.28619 trim_cache size 0 max 16384 2024-07-31T15:08:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.782+0000 7fe8f6a04740 1 Processor -- start 2024-07-31T15:08:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.782+0000 7fe8f6a04740 1 -- start start 2024-07-31T15:08:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.782+0000 7fe8f6a04740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef52390 0x558e8f315350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.782+0000 7fe8f6a04740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef73070 0x558e8f31d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.782+0000 7fe8f6a04740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef06da0 0x558e8f31dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.783+0000 7fe8f6a04740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81b7300 con 0x558e8ef06da0 2024-07-31T15:08:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.783+0000 7fe8f6a04740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81b8e40 con 0x558e8ef73070 2024-07-31T15:08:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.783+0000 7fe8f6a04740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81b79d0 con 0x558e8ef52390 2024-07-31T15:08:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.783+0000 7fe8eda4e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef73070 0x558e8f31d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.783+0000 7fe8eda4e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef73070 0x558e8f31d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48374/0 (socket says 172.21.15.67:48374) 2024-07-31T15:08:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.783+0000 7fe8eda4e640 1 -- 172.21.15.67:0/4167734612 learned_addr learned my addr 172.21.15.67:0/4167734612 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.783+0000 7fe8eea50640 1 --2- 172.21.15.67:0/4167734612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef06da0 0x558e8f31dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.783+0000 7fe8eda4e640 1 -- 172.21.15.67:0/4167734612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef52390 msgr2=0x558e8f315350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.783+0000 7fe8ee24f640 1 --2- 172.21.15.67:0/4167734612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef52390 0x558e8f315350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.783+0000 7fe8eda4e640 1 --2- 172.21.15.67:0/4167734612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef52390 0x558e8f315350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.783+0000 7fe8eda4e640 1 -- 172.21.15.67:0/4167734612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef06da0 msgr2=0x558e8f31dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.783+0000 7fe8eda4e640 1 --2- 172.21.15.67:0/4167734612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef06da0 0x558e8f31dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.783+0000 7fe8eda4e640 1 -- 172.21.15.67:0/4167734612 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e8ef31c20 con 0x558e8ef73070 2024-07-31T15:08:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.783+0000 7fe8eea50640 1 --2- 172.21.15.67:0/4167734612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef06da0 0x558e8f31dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.784+0000 7fe8eda4e640 1 --2- 172.21.15.67:0/4167734612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef73070 0x558e8f31d680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00bb560 tx=0x7fe8e00bb080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d18ad8b09f2b7dd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.784+0000 7fe8d3fff640 1 -- 172.21.15.67:0/4167734612 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00d6190 con 0x558e8ef73070 2024-07-31T15:08:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.784+0000 7fe8d3fff640 1 -- 172.21.15.67:0/4167734612 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e00bb370 con 0x558e8ef73070 2024-07-31T15:08:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.784+0000 7fe8d3fff640 1 -- 172.21.15.67:0/4167734612 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00b7af0 con 0x558e8ef73070 2024-07-31T15:08:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.785+0000 7fe8f6a04740 1 -- 172.21.15.67:0/4167734612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef73070 msgr2=0x558e8f31d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.785+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/4167734612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef73070 0x558e8f31d680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00bb560 tx=0x7fe8e00bb080 comp rx=0 tx=0).stop 2024-07-31T15:08:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.786+0000 7fe8f6a04740 1 -- 172.21.15.67:0/4167734612 shutdown_connections 2024-07-31T15:08:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.786+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/4167734612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef06da0 0x558e8f31dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.786+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/4167734612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef73070 0x558e8f31d680 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.786+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/4167734612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef52390 0x558e8f315350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.786+0000 7fe8f6a04740 1 -- 172.21.15.67:0/4167734612 >> 172.21.15.67:0/4167734612 conn(0x558e8ef67460 msgr2=0x558e8f314c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.787+0000 7fe8f6a04740 1 -- 172.21.15.67:0/4167734612 shutdown_connections 2024-07-31T15:08:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.787+0000 7fe8f6a04740 1 -- 172.21.15.67:0/4167734612 wait complete. 2024-07-31T15:08:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.787+0000 7fe8f6a04740 1 -- start start 2024-07-31T15:08:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.788+0000 7fe8f6a04740 1 Processor -- start 2024-07-31T15:08:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.788+0000 7fe8f6a04740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef06da0 0x558e8f32c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.789+0000 7fe8f6a04740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef52390 0x558e8f3278f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.789+0000 7fe8ee24f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef06da0 0x558e8f32c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.789+0000 7fe8ee24f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef06da0 0x558e8f32c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:48390/0 (socket says 172.21.15.67:48390) 2024-07-31T15:08:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.789+0000 7fe8eda4e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef52390 0x558e8f3278f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.789+0000 7fe8eda4e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef52390 0x558e8f3278f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36862/0 (socket says 172.21.15.67:36862) 2024-07-31T15:08:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.789+0000 7fe8f6a04740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef73070 0x558e8f327e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.789+0000 7fe8f6a04740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81b8770 con 0x558e8ef52390 2024-07-31T15:08:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.789+0000 7fe8f6a04740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81b80a0 con 0x558e8ef06da0 2024-07-31T15:08:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.789+0000 7fe8f6a04740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81b5300 con 0x558e8ef73070 2024-07-31T15:08:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.789+0000 7fe8f6a04740 10 client.0 _ll_register_callbacks cb 0x7fe8e40092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.790+0000 7fe8eea50640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef73070 0x558e8f327e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.790+0000 7fe8ee24f640 1 -- 172.21.15.67:0/4247815376 learned_addr learned my addr 172.21.15.67:0/4247815376 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.790+0000 7fe8eea50640 1 --2- 172.21.15.67:0/4247815376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef73070 0x558e8f327e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:46934/0 (socket says 172.21.15.67:46934) 2024-07-31T15:08:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.790+0000 7fe8eea50640 1 -- 172.21.15.67:0/4247815376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef06da0 msgr2=0x558e8f32c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.790+0000 7fe8eea50640 1 --2- 172.21.15.67:0/4247815376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef06da0 0x558e8f32c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.790+0000 7fe8eea50640 1 -- 172.21.15.67:0/4247815376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef52390 msgr2=0x558e8f3278f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.790+0000 7fe8eea50640 1 --2- 172.21.15.67:0/4247815376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef52390 0x558e8f3278f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.790+0000 7fe8eea50640 1 -- 172.21.15.67:0/4247815376 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e8f3283f0 con 0x558e8ef73070 2024-07-31T15:08:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.791+0000 7fe8eea50640 1 --2- 172.21.15.67:0/4247815376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef73070 0x558e8f327e30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc035040 tx=0x7fe8dc030810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8998aea1678177de server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.791+0000 7fe8d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:08:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.792+0000 7fe8d1ffb640 1 -- 172.21.15.67:0/4247815376 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc04e020 con 0x558e8ef73070 2024-07-31T15:08:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.792+0000 7fe8d1ffb640 1 -- 172.21.15.67:0/4247815376 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fe8dc0048c0 con 0x558e8ef73070 2024-07-31T15:08:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.792+0000 7fe8f6a04740 1 -- 172.21.15.67:0/4247815376 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e00bb370 con 0x558e8ef73070 2024-07-31T15:08:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.793+0000 7fe8d1ffb640 1 -- 172.21.15.67:0/4247815376 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe8dc037040 con 0x558e8ef73070 2024-07-31T15:08:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.793+0000 7fe8f6a04740 10 client.28504 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.793+0000 7fe8f6a04740 1 -- 172.21.15.67:0/4247815376 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x558e8f39ec00 con 0x558e8ef73070 2024-07-31T15:08:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.794+0000 7fe8d1ffb640 1 -- 172.21.15.67:0/4247815376 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc044070 con 0x558e8ef73070 2024-07-31T15:08:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.794+0000 7fe8d1ffb640 1 -- 172.21.15.67:0/4247815376 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fe8dc049070 con 0x558e8ef73070 2024-07-31T15:08:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.794+0000 7fe8f6a04740 10 client.28504 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.794+0000 7fe8f6a04740 10 client.28504 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:08:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.794+0000 7fe8f6a04740 20 client.28504 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.794+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/4247815376 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558e8f39f3b0 0x558e8f3a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.794+0000 7fe8f6a04740 4 client.28504 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:08:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.794+0000 7fe8f6a04740 1 -- 172.21.15.67:0/4247815376 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe8e00bb370 con 0x558e8f39f3b0 2024-07-31T15:08:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.794+0000 7fe8ee24f640 1 --2- 172.21.15.67:0/4247815376 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558e8f39f3b0 0x558e8f3a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.796+0000 7fe8ee24f640 1 --2- 172.21.15.67:0/4247815376 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558e8f39f3b0 0x558e8f3a17d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c900cf198439488a server_cookie=28a510433709a95b in_seq=0 out_seq=0 2024-07-31T15:08:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.796+0000 7fe8d1ffb640 10 client.28504 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:08:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.797+0000 7fe8d1ffb640 1 -- 172.21.15.67:0/4247815376 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe8e00bb370 con 0x558e8f39f3b0 2024-07-31T15:08:17.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.797+0000 7fe8d1ffb640 10 client.28504 handle_command_reply: tid=0 2024-07-31T15:08:17.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.797+0000 7fe8f6a04740 1 client.28504 shutdown 2024-07-31T15:08:17.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.798+0000 7fe8f6a04740 1 -- 172.21.15.67:0/4247815376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef73070 msgr2=0x558e8f327e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:17.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.798+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/4247815376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef73070 0x558e8f327e30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc035040 tx=0x7fe8dc030810 comp rx=0 tx=0).stop 2024-07-31T15:08:17.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.798+0000 7fe8f6a04740 1 -- 172.21.15.67:0/4247815376 shutdown_connections 2024-07-31T15:08:17.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.798+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/4247815376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e8ef73070 0x558e8f327e30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.798+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/4247815376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e8ef52390 0x558e8f3278f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.798+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/4247815376 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x558e8f39f3b0 0x558e8f3a17d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.798+0000 7fe8f6a04740 1 --2- 172.21.15.67:0/4247815376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e8ef06da0 0x558e8f32c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.798+0000 7fe8d1ffb640 0 client.28504 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:08:17.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.798+0000 7fe8f6a04740 1 -- 172.21.15.67:0/4247815376 >> 172.21.15.67:0/4247815376 conn(0x558e8ef67460 msgr2=0x558e8f31d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:17.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.798+0000 7fe8f6a04740 1 -- 172.21.15.67:0/4247815376 shutdown_connections 2024-07-31T15:08:17.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.798+0000 7fe8f6a04740 1 -- 172.21.15.67:0/4247815376 wait complete. 2024-07-31T15:08:17.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.803+0000 7fe8f6a04740 20 client.28504 trim_cache size 0 max 16384 2024-07-31T15:08:17.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.806+0000 7fe8f04b3640 1 -- 172.21.15.67:0/3726232464 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe8d40b3f40 msgr2=0x7fe8d40b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:17.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.806+0000 7fe8f04b3640 1 --2- 172.21.15.67:0/3726232464 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe8d40b3f40 0x7fe8d40b6450 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0092e0 tx=0x7fe8dc005f90 comp rx=0 tx=0).stop 2024-07-31T15:08:17.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.806+0000 7fe8f04b3640 1 -- 172.21.15.67:0/3726232464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8e8111970 msgr2=0x7fe8e8118530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:17.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.806+0000 7fe8f04b3640 1 --2- 172.21.15.67:0/3726232464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8e8111970 0x7fe8e8118530 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe8e001fde0 tx=0x7fe8e001d750 comp rx=0 tx=0).stop 2024-07-31T15:08:17.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.806+0000 7fe8f04b3640 1 -- 172.21.15.67:0/3726232464 shutdown_connections 2024-07-31T15:08:17.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.806+0000 7fe8f04b3640 1 --2- 172.21.15.67:0/3726232464 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe8d40b3f40 0x7fe8d40b6450 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.806+0000 7fe8f04b3640 1 --2- 172.21.15.67:0/3726232464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8e81a5910 0x7fe8e81a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.806+0000 7fe8f04b3640 1 --2- 172.21.15.67:0/3726232464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8e8111970 0x7fe8e8118530 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.806+0000 7fe8f04b3640 1 --2- 172.21.15.67:0/3726232464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8e810a410 0x7fe8e8117f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:17.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.806+0000 7fe8f04b3640 1 -- 172.21.15.67:0/3726232464 >> 172.21.15.67:0/3726232464 conn(0x7fe8e80708f0 msgr2=0x7fe8e810b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:17.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.807+0000 7fe8f04b3640 1 -- 172.21.15.67:0/3726232464 shutdown_connections 2024-07-31T15:08:17.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:17.807+0000 7fe8f04b3640 1 -- 172.21.15.67:0/3726232464 wait complete. 2024-07-31T15:08:17.872 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/ip_forward.o 2024-07-31T15:08:17.875 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/regmap/regmap-debugfs.o 2024-07-31T15:08:17.906 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/bridge/synopsys/built-in.a 2024-07-31T15:08:17.922 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/bridge/built-in.a 2024-07-31T15:08:17.945 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/xfrm/xfrm_policy.o 2024-07-31T15:08:17.980 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/intel_clock_gating.o 2024-07-31T15:08:18.359 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/virtio/virtgpu_plane.o 2024-07-31T15:08:18.488 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/intel_device_info.o 2024-07-31T15:08:18.629 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/base/regmap/built-in.a 2024-07-31T15:08:18.650 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/unix/af_unix.o 2024-07-31T15:08:18.719 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/base/test/built-in.a 2024-07-31T15:08:18.733 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/component.o 2024-07-31T15:08:18.900 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T15:08:18.929 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_proto.o 2024-07-31T15:08:18.952 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:08:19.032 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/ip_options.o 2024-07-31T15:08:19.102 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/virtio/virtgpu_ioctl.o 2024-07-31T15:08:19.434 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/core.o 2024-07-31T15:08:19.449 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:19 smithi067 ceph-mon[32750]: pgmap v946: 129 pgs: 3 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 117 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-07-31T15:08:19.558 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/intel_memory_region.o 2024-07-31T15:08:19.562 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_proto_generic.o 2024-07-31T15:08:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:19 smithi073 ceph-mon[36714]: pgmap v946: 129 pgs: 3 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 117 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-07-31T15:08:19.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:19 smithi077 ceph-mon[37327]: pgmap v946: 129 pgs: 3 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 117 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-07-31T15:08:20.006 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/virtio/virtgpu_prime.o 2024-07-31T15:08:20.192 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/ip_output.o 2024-07-31T15:08:20.195 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_proto_tcp.o 2024-07-31T15:08:20.437 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/intel_pcode.o 2024-07-31T15:08:20.531 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/intel_region_ttm.o 2024-07-31T15:08:20.549 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:08:20.677 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/virtio/virtgpu_trace_points.o 2024-07-31T15:08:20.750 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/unix/garbage.o 2024-07-31T15:08:20.793 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/xfrm/xfrm_state.o 2024-07-31T15:08:21.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:21 smithi067 ceph-mon[32750]: pgmap v947: 129 pgs: 3 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 117 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 208 op/s 2024-07-31T15:08:21.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:21 smithi067 ceph-mon[32750]: osdmap e120: 12 total, 12 up, 12 in 2024-07-31T15:08:21.358 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/intel_runtime_pm.o 2024-07-31T15:08:21.406 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/virtio/virtgpu_submit.o 2024-07-31T15:08:21.447 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/bus.o 2024-07-31T15:08:21.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:21 smithi077 ceph-mon[37327]: pgmap v947: 129 pgs: 3 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 117 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 208 op/s 2024-07-31T15:08:21.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:21 smithi077 ceph-mon[37327]: osdmap e120: 12 total, 12 up, 12 in 2024-07-31T15:08:21.537 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/xfrm/xfrm_hash.o 2024-07-31T15:08:21.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:21 smithi073 ceph-mon[36714]: pgmap v947: 129 pgs: 3 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 117 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 208 op/s 2024-07-31T15:08:21.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:21 smithi073 ceph-mon[36714]: osdmap e120: 12 total, 12 up, 12 in 2024-07-31T15:08:21.657 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_proto_udp.o 2024-07-31T15:08:21.661 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/unix/sysctl_net_unix.o 2024-07-31T15:08:22.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.051+0000 7f933a319640 1 -- 172.21.15.67:0/871958494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f933410d110 msgr2=0x7f933410d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:22.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.051+0000 7f933a319640 1 --2- 172.21.15.67:0/871958494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f933410d110 0x7f933410d520 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f932c013490 tx=0x7f932c032af0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.051+0000 7f933a319640 1 -- 172.21.15.67:0/871958494 shutdown_connections 2024-07-31T15:08:22.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.051+0000 7f933a319640 1 --2- 172.21.15.67:0/871958494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f933410da60 0x7f9334110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.051+0000 7f933a319640 1 --2- 172.21.15.67:0/871958494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f933410d110 0x7f933410d520 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.051+0000 7f933a319640 1 --2- 172.21.15.67:0/871958494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9334109df0 0x7f933410a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.051+0000 7f933a319640 1 -- 172.21.15.67:0/871958494 >> 172.21.15.67:0/871958494 conn(0x7f933406e440 msgr2=0x7f933406e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:22.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.053+0000 7f933a319640 1 -- 172.21.15.67:0/871958494 shutdown_connections 2024-07-31T15:08:22.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.053+0000 7f933a319640 1 -- 172.21.15.67:0/871958494 wait complete. 2024-07-31T15:08:22.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.054+0000 7f933a319640 1 Processor -- start 2024-07-31T15:08:22.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.054+0000 7f933a319640 1 -- start start 2024-07-31T15:08:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.054+0000 7f933a319640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9334109df0 0x7f93341b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.054+0000 7f933a319640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f933410d110 0x7f93341b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.054+0000 7f933a319640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f933410da60 0x7f93341b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.054+0000 7f933a319640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f93341199c0 con 0x7f9334109df0 2024-07-31T15:08:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.054+0000 7f933a319640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9334119840 con 0x7f933410d110 2024-07-31T15:08:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.054+0000 7f933a319640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9334119b40 con 0x7f933410da60 2024-07-31T15:08:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.055+0000 7f9338b16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f933410d110 0x7f93341b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.055+0000 7f9338b16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f933410d110 0x7f93341b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:41962/0 (socket says 172.21.15.67:41962) 2024-07-31T15:08:22.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.055+0000 7f9338b16640 1 -- 172.21.15.67:0/94404725 learned_addr learned my addr 172.21.15.67:0/94404725 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:22.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.055+0000 7f9339317640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9334109df0 0x7f93341b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.055+0000 7f9339b18640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f933410da60 0x7f93341b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.055+0000 7f9338b16640 1 -- 172.21.15.67:0/94404725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f933410da60 msgr2=0x7f93341b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.055+0000 7f9338b16640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f933410da60 0x7f93341b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.055+0000 7f9338b16640 1 -- 172.21.15.67:0/94404725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9334109df0 msgr2=0x7f93341b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.055+0000 7f9338b16640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9334109df0 0x7f93341b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.055+0000 7f9338b16640 1 -- 172.21.15.67:0/94404725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93341b7210 con 0x7f933410d110 2024-07-31T15:08:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.055+0000 7f9339317640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9334109df0 0x7f93341b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.056+0000 7f9338b16640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f933410d110 0x7f93341b1840 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f932c0330e0 tx=0x7f932c034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.056+0000 7f932a7fc640 1 -- 172.21.15.67:0/94404725 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c044070 con 0x7f933410d110 2024-07-31T15:08:22.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.057+0000 7f932a7fc640 1 -- 172.21.15.67:0/94404725 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f932c003e40 con 0x7f933410d110 2024-07-31T15:08:22.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.057+0000 7f932a7fc640 1 -- 172.21.15.67:0/94404725 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c03a040 con 0x7f933410d110 2024-07-31T15:08:22.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.060+0000 7f933a319640 1 -- 172.21.15.67:0/94404725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93341b74a0 con 0x7f933410d110 2024-07-31T15:08:22.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.060+0000 7f933a319640 1 -- 172.21.15.67:0/94404725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93341b7900 con 0x7f933410d110 2024-07-31T15:08:22.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.064+0000 7f9341acc740 1 Processor -- start 2024-07-31T15:08:22.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.064+0000 7f9341acc740 1 -- start start 2024-07-31T15:08:22.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.065+0000 7f9341acc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744cc0840 0x559745066a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.065+0000 7f9341acc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744c97980 0x559745066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.065+0000 7f9341acc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x5597450674b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.065+0000 7f9341acc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559744fdb780 con 0x559744c97980 2024-07-31T15:08:22.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.065+0000 7f9341acc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559744fcac80 con 0x559744c9fb60 2024-07-31T15:08:22.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.065+0000 7f9341acc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559744fcecc0 con 0x559744cc0840 2024-07-31T15:08:22.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.065+0000 7f9338b16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x5597450674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.065+0000 7f9338b16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x5597450674b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:41974/0 (socket says 172.21.15.67:41974) 2024-07-31T15:08:22.072 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/ip_sockglue.o 2024-07-31T15:08:22.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.065+0000 7f9338b16640 1 -- 172.21.15.67:0/3836606078 learned_addr learned my addr 172.21.15.67:0/3836606078 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:22.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.065+0000 7f9339317640 1 --2- 172.21.15.67:0/3836606078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744c97980 0x559745066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.065+0000 7f9339b18640 1 --2- 172.21.15.67:0/3836606078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744cc0840 0x559745066a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.066+0000 7f9338b16640 1 -- 172.21.15.67:0/3836606078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744cc0840 msgr2=0x559745066a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:22.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.066+0000 7f9338b16640 1 --2- 172.21.15.67:0/3836606078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744cc0840 0x559745066a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.066+0000 7f9338b16640 1 -- 172.21.15.67:0/3836606078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744c97980 msgr2=0x559745066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:22.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.066+0000 7f9338b16640 1 --2- 172.21.15.67:0/3836606078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744c97980 0x559745066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.066+0000 7f9338b16640 1 -- 172.21.15.67:0/3836606078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559744f60080 con 0x559744c9fb60 2024-07-31T15:08:22.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.066+0000 7f9339317640 1 --2- 172.21.15.67:0/3836606078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744c97980 0x559745066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:22.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.069+0000 7f9338b16640 1 --2- 172.21.15.67:0/3836606078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x5597450674b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f932c032b30 tx=0x7f932c00f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=968e24b9c1e431be server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:22.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.069+0000 7f932a7fc640 1 -- 172.21.15.67:0/94404725 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f932c03d070 con 0x7f933410d110 2024-07-31T15:08:22.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.070+0000 7f931f7fe640 1 -- 172.21.15.67:0/3836606078 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c10c5b0 con 0x559744c9fb60 2024-07-31T15:08:22.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.070+0000 7f931f7fe640 1 -- 172.21.15.67:0/3836606078 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f932c10cc80 con 0x559744c9fb60 2024-07-31T15:08:22.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.070+0000 7f931f7fe640 1 -- 172.21.15.67:0/3836606078 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c10b250 con 0x559744c9fb60 2024-07-31T15:08:22.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.071+0000 7f932a7fc640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93200b3e80 0x7f93200b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.071+0000 7f9341acc740 1 -- 172.21.15.67:0/3836606078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 msgr2=0x5597450674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:22.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.071+0000 7f9341acc740 1 --2- 172.21.15.67:0/3836606078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x5597450674b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f932c032b30 tx=0x7f932c00f7f0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.071+0000 7f932a7fc640 1 -- 172.21.15.67:0/94404725 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9407+0+0 (secure 0 0 0) 0x7f932c105460 con 0x7f933410d110 2024-07-31T15:08:22.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.074+0000 7f9339b18640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93200b3e80 0x7f93200b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.075+0000 7f9341acc740 1 -- 172.21.15.67:0/3836606078 shutdown_connections 2024-07-31T15:08:22.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.075+0000 7f9341acc740 1 --2- 172.21.15.67:0/3836606078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x5597450674b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.075+0000 7f9341acc740 1 --2- 172.21.15.67:0/3836606078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744c97980 0x559745066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.075+0000 7f9341acc740 1 --2- 172.21.15.67:0/3836606078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744cc0840 0x559745066a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.075+0000 7f9341acc740 1 -- 172.21.15.67:0/3836606078 >> 172.21.15.67:0/3836606078 conn(0x559744cb4990 msgr2=0x559745064920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:22.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.076+0000 7f9339b18640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93200b3e80 0x7f93200b6390 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f933410ab30 tx=0x7f933000b2e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:22.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.076+0000 7f9341acc740 1 -- 172.21.15.67:0/3836606078 shutdown_connections 2024-07-31T15:08:22.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.078+0000 7f9341acc740 1 -- 172.21.15.67:0/3836606078 wait complete. 2024-07-31T15:08:22.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.079+0000 7f9341acc740 1 -- start start 2024-07-31T15:08:22.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.079+0000 7f9341acc740 1 Processor -- start 2024-07-31T15:08:22.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.080+0000 7f9341acc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744c97980 0x5597450720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.080+0000 7f9341acc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x5597450725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.080+0000 7f9341acc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744cc0840 0x559745072b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.080+0000 7f9338b16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x5597450725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.080+0000 7f9338b16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x5597450725f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:41984/0 (socket says 172.21.15.67:41984) 2024-07-31T15:08:22.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.080+0000 7f9339317640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744c97980 0x5597450720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.081+0000 7f9339317640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744c97980 0x5597450720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52414/0 (socket says 172.21.15.67:52414) 2024-07-31T15:08:22.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.081+0000 7f9341acc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559744e2e7a0 con 0x559744c97980 2024-07-31T15:08:22.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.081+0000 7f9341acc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559744f6d2f0 con 0x559744c9fb60 2024-07-31T15:08:22.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.081+0000 7f9341acc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559745074fd0 con 0x559744cc0840 2024-07-31T15:08:22.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.081+0000 7f9338b16640 1 -- 172.21.15.67:0/362883612 learned_addr learned my addr 172.21.15.67:0/362883612 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:22.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.082+0000 7f9339b18640 1 --2- 172.21.15.67:0/362883612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744cc0840 0x559745072b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.083+0000 7f9339b18640 1 -- 172.21.15.67:0/362883612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 msgr2=0x5597450725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:22.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.083+0000 7f9339b18640 1 --2- 172.21.15.67:0/362883612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x5597450725f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.083+0000 7f9339b18640 1 -- 172.21.15.67:0/362883612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744c97980 msgr2=0x5597450720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:22.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.083+0000 7f9339b18640 1 --2- 172.21.15.67:0/362883612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744c97980 0x5597450720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.083+0000 7f9339b18640 1 -- 172.21.15.67:0/362883612 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559744f71480 con 0x559744cc0840 2024-07-31T15:08:22.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.083+0000 7f9339317640 1 --2- 172.21.15.67:0/362883612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744c97980 0x5597450720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:22.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.081+0000 7f9341acc740 10 client.0 _ll_register_callbacks cb 0x559744f77110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:22.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.084+0000 7f9339b18640 1 --2- 172.21.15.67:0/362883612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744cc0840 0x559745072b30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f933000bf90 tx=0x7f9330004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e913a00a6aaaee4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:22.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.084+0000 7f9341acc740 1 -- 172.21.15.67:0/362883612 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f932c10cc80 con 0x559744cc0840 2024-07-31T15:08:22.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.085+0000 7f931cff9640 10 client.28516 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:08:22.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.086+0000 7f931cff9640 1 -- 172.21.15.67:0/362883612 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933001fc00 con 0x559744cc0840 2024-07-31T15:08:22.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.086+0000 7f931cff9640 1 -- 172.21.15.67:0/362883612 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9407+0+0 (secure 0 0 0) 0x7f9330017030 con 0x559744cc0840 2024-07-31T15:08:22.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.086+0000 7f931cff9640 1 -- 172.21.15.67:0/362883612 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f933001fda0 con 0x559744cc0840 2024-07-31T15:08:22.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.087+0000 7f9341acc740 10 client.28516 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:22.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.087+0000 7f9341acc740 1 -- 172.21.15.67:0/362883612 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=65}) -- 0x559744d3ab90 con 0x559744cc0840 2024-07-31T15:08:22.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.089+0000 7f931cff9640 1 -- 172.21.15.67:0/362883612 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f9330005020 con 0x559744cc0840 2024-07-31T15:08:22.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.089+0000 7f9341acc740 10 client.28516 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.089+0000 7f9341acc740 10 client.28516 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:08:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.089+0000 7f9341acc740 20 client.28516 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.089+0000 7f9341acc740 1 --2- 172.21.15.67:0/362883612 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559744c7f1c0 0x559745084220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.089+0000 7f9341acc740 4 client.28516 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:08:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.089+0000 7f9341acc740 1 -- 172.21.15.67:0/362883612 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559744fdd6d0 con 0x559744c7f1c0 2024-07-31T15:08:22.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.091+0000 7f9339317640 1 --2- 172.21.15.67:0/362883612 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559744c7f1c0 0x559745084220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.092+0000 7f9339317640 1 --2- 172.21.15.67:0/362883612 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559744c7f1c0 0x559745084220 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a58a5d17a7b8654 server_cookie=8ac9374815762cb7 in_seq=0 out_seq=0 2024-07-31T15:08:22.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.092+0000 7f931cff9640 10 client.28516 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:22.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.093+0000 7f931cff9640 1 -- 172.21.15.67:0/362883612 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559744fdd6d0 con 0x559744c7f1c0 2024-07-31T15:08:22.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.093+0000 7f931cff9640 10 client.28516 handle_command_reply: tid=0 2024-07-31T15:08:22.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.093+0000 7f9341acc740 1 client.28516 shutdown 2024-07-31T15:08:22.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.093+0000 7f9341acc740 1 -- 172.21.15.67:0/362883612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744cc0840 msgr2=0x559745072b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:22.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.093+0000 7f9341acc740 1 --2- 172.21.15.67:0/362883612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744cc0840 0x559745072b30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f933000bf90 tx=0x7f9330004730 comp rx=0 tx=0).stop 2024-07-31T15:08:22.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.094+0000 7f9341acc740 1 -- 172.21.15.67:0/362883612 shutdown_connections 2024-07-31T15:08:22.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.094+0000 7f9341acc740 1 --2- 172.21.15.67:0/362883612 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559744c7f1c0 0x559745084220 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.094+0000 7f9341acc740 1 --2- 172.21.15.67:0/362883612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744cc0840 0x559745072b30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.094+0000 7f9341acc740 1 --2- 172.21.15.67:0/362883612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x5597450725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.094+0000 7f9341acc740 1 --2- 172.21.15.67:0/362883612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744c97980 0x5597450720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.094+0000 7f9341acc740 1 -- 172.21.15.67:0/362883612 >> 172.21.15.67:0/362883612 conn(0x559744cb4990 msgr2=0x559745068420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.094+0000 7f931cff9640 0 client.28516 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.095+0000 7f9341acc740 1 -- 172.21.15.67:0/362883612 shutdown_connections 2024-07-31T15:08:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.095+0000 7f9341acc740 1 -- 172.21.15.67:0/362883612 wait complete. 2024-07-31T15:08:22.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.095+0000 7f9341acc740 20 client.28516 trim_cache size 0 max 16384 2024-07-31T15:08:22.119 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/dd.o 2024-07-31T15:08:22.131 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/virtio/built-in.a 2024-07-31T15:08:22.137 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/syscore.o 2024-07-31T15:08:22.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.136+0000 7f9341acc740 1 Processor -- start 2024-07-31T15:08:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.136+0000 7f9341acc740 1 -- start start 2024-07-31T15:08:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.136+0000 7f9341acc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744c97980 0x559745072590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.136+0000 7f9341acc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x559745072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.136+0000 7f9341acc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744cc0840 0x559745089a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.136+0000 7f9341acc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f93341aedd0 con 0x559744cc0840 2024-07-31T15:08:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.136+0000 7f9341acc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93341da740 con 0x559744c9fb60 2024-07-31T15:08:22.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.136+0000 7f9341acc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93341af4a0 con 0x559744c97980 2024-07-31T15:08:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.137+0000 7f9338b16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x559745072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.137+0000 7f9339317640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744c97980 0x559745072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.137+0000 7f9338b16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x559745072ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:41986/0 (socket says 172.21.15.67:41986) 2024-07-31T15:08:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.137+0000 7f9338b16640 1 -- 172.21.15.67:0/1689062072 learned_addr learned my addr 172.21.15.67:0/1689062072 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.137+0000 7f9339b18640 1 --2- 172.21.15.67:0/1689062072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744cc0840 0x559745089a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.137+0000 7f9338b16640 1 -- 172.21.15.67:0/1689062072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744c97980 msgr2=0x559745072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.137+0000 7f9338b16640 1 --2- 172.21.15.67:0/1689062072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744c97980 0x559745072590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.137+0000 7f9338b16640 1 -- 172.21.15.67:0/1689062072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744cc0840 msgr2=0x559745089a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.137+0000 7f9338b16640 1 --2- 172.21.15.67:0/1689062072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744cc0840 0x559745089a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.137+0000 7f9338b16640 1 -- 172.21.15.67:0/1689062072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559745068d20 con 0x559744c9fb60 2024-07-31T15:08:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.137+0000 7f9339b18640 1 --2- 172.21.15.67:0/1689062072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744cc0840 0x559745089a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:08:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.138+0000 7f9338b16640 1 --2- 172.21.15.67:0/1689062072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x559745072ad0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f932c107000 tx=0x7f932c0110e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1bc52773ea6c724 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.138+0000 7f931effd640 1 -- 172.21.15.67:0/1689062072 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c0cbc90 con 0x559744c9fb60 2024-07-31T15:08:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.138+0000 7f931effd640 1 -- 172.21.15.67:0/1689062072 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f932c0cb9d0 con 0x559744c9fb60 2024-07-31T15:08:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.139+0000 7f9341acc740 1 -- 172.21.15.67:0/1689062072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 msgr2=0x559745072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.140+0000 7f9341acc740 1 --2- 172.21.15.67:0/1689062072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x559745072ad0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f932c107000 tx=0x7f932c0110e0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.140+0000 7f9341acc740 1 -- 172.21.15.67:0/1689062072 shutdown_connections 2024-07-31T15:08:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.140+0000 7f9341acc740 1 --2- 172.21.15.67:0/1689062072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744cc0840 0x559745089a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.140+0000 7f9341acc740 1 --2- 172.21.15.67:0/1689062072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c9fb60 0x559745072ad0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.140+0000 7f9341acc740 1 --2- 172.21.15.67:0/1689062072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744c97980 0x559745072590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.140+0000 7f9341acc740 1 -- 172.21.15.67:0/1689062072 >> 172.21.15.67:0/1689062072 conn(0x559744cb4990 msgr2=0x559745071e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.140+0000 7f9341acc740 1 -- 172.21.15.67:0/1689062072 shutdown_connections 2024-07-31T15:08:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.140+0000 7f9341acc740 1 -- 172.21.15.67:0/1689062072 wait complete. 2024-07-31T15:08:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.141+0000 7f9341acc740 1 -- start start 2024-07-31T15:08:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.141+0000 7f9341acc740 1 Processor -- start 2024-07-31T15:08:22.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.142+0000 7f9341acc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c97980 0x55974507a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.142+0000 7f9341acc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744c9fb60 0x559745075a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.142+0000 7f9341acc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744cc0840 0x559745075f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.142+0000 7f9341acc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f93341da070 con 0x559744cc0840 2024-07-31T15:08:22.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.142+0000 7f9341acc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93341afb70 con 0x559744c97980 2024-07-31T15:08:22.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.142+0000 7f9341acc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93341b23d0 con 0x559744c9fb60 2024-07-31T15:08:22.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.142+0000 7f9341acc740 10 client.0 _ll_register_callbacks cb 0x7f933000bdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:22.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.142+0000 7f9339317640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c97980 0x55974507a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.142+0000 7f9339317640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c97980 0x55974507a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:41994/0 (socket says 172.21.15.67:41994) 2024-07-31T15:08:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.142+0000 7f9339317640 1 -- 172.21.15.67:0/688316852 learned_addr learned my addr 172.21.15.67:0/688316852 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.142+0000 7f9338b16640 1 --2- 172.21.15.67:0/688316852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744c9fb60 0x559745075a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.142+0000 7f9339b18640 1 --2- 172.21.15.67:0/688316852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744cc0840 0x559745075f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.143+0000 7f9338b16640 1 -- 172.21.15.67:0/688316852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c97980 msgr2=0x55974507a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.143+0000 7f9338b16640 1 --2- 172.21.15.67:0/688316852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c97980 0x55974507a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.143+0000 7f9338b16640 1 -- 172.21.15.67:0/688316852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744cc0840 msgr2=0x559745075f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.143+0000 7f9338b16640 1 --2- 172.21.15.67:0/688316852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744cc0840 0x559745075f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:22.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.143+0000 7f9338b16640 1 -- 172.21.15.67:0/688316852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559745076530 con 0x559744c9fb60 2024-07-31T15:08:22.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.144+0000 7f9338b16640 1 --2- 172.21.15.67:0/688316852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744c9fb60 0x559745075a00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f932c0c7f40 tx=0x7f932c0d0350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=893b07b16002661e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:22.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.145+0000 7f931cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:08:22.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.145+0000 7f931cff9640 1 -- 172.21.15.67:0/688316852 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c0e5070 con 0x559744c9fb60 2024-07-31T15:08:22.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.145+0000 7f931cff9640 1 -- 172.21.15.67:0/688316852 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9407+0+0 (secure 0 0 0) 0x7f932c10dce0 con 0x559744c9fb60 2024-07-31T15:08:22.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.145+0000 7f9341acc740 1 -- 172.21.15.67:0/688316852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f932c0cb9d0 con 0x559744c9fb60 2024-07-31T15:08:22.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.147+0000 7f9339317640 1 --2- 172.21.15.67:0/688316852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c97980 0x55974507a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:22.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.147+0000 7f931cff9640 1 -- 172.21.15.67:0/688316852 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f932c0cb810 con 0x559744c9fb60 2024-07-31T15:08:22.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.147+0000 7f9341acc740 10 client.28522 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:22.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.147+0000 7f9341acc740 1 -- 172.21.15.67:0/688316852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5597450ec6e0 con 0x559744c9fb60 2024-07-31T15:08:22.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.148+0000 7f931cff9640 1 -- 172.21.15.67:0/688316852 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f932c0cb9d0 con 0x559744c9fb60 2024-07-31T15:08:22.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.148+0000 7f931cff9640 1 -- 172.21.15.67:0/688316852 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f932c047030 con 0x559744c9fb60 2024-07-31T15:08:22.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.148+0000 7f9341acc740 10 client.28522 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:22.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.148+0000 7f9341acc740 10 client.28522 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:08:22.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.148+0000 7f9341acc740 20 client.28522 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:22.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.148+0000 7f9341acc740 1 --2- 172.21.15.67:0/688316852 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559744c54570 0x5597450eeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:22.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.149+0000 7f9341acc740 4 client.28522 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-07-31T15:08:22.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.149+0000 7f9341acc740 1 -- 172.21.15.67:0/688316852 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x559744fd9170 con 0x559744c54570 2024-07-31T15:08:22.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.149+0000 7f9339317640 1 --2- 172.21.15.67:0/688316852 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559744c54570 0x5597450eeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:22.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.150+0000 7f9339317640 1 --2- 172.21.15.67:0/688316852 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559744c54570 0x5597450eeaf0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab34e3110d7e12b8 server_cookie=158d298ef7128818 in_seq=0 out_seq=0 2024-07-31T15:08:22.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:22.150+0000 7f931cff9640 10 client.28522 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:22.216 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/xfrm/xfrm_input.o 2024-07-31T15:08:23.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.215+0000 7f1eb1943640 1 -- 172.21.15.67:0/3113691312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eac104290 msgr2=0x7f1eac1046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:23.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.215+0000 7f1eb1943640 1 --2- 172.21.15.67:0/3113691312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eac104290 0x7f1eac1046a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009900 tx=0x7f1e9c02a410 comp rx=0 tx=0).stop 2024-07-31T15:08:23.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.215+0000 7f1eb1943640 1 -- 172.21.15.67:0/3113691312 shutdown_connections 2024-07-31T15:08:23.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.215+0000 7f1eb1943640 1 --2- 172.21.15.67:0/3113691312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eac10d840 0x7f1eac10fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.215+0000 7f1eb1943640 1 --2- 172.21.15.67:0/3113691312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1eac104cc0 0x7f1eac10d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.215+0000 7f1eb1943640 1 --2- 172.21.15.67:0/3113691312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eac104290 0x7f1eac1046a0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.215+0000 7f1eb1943640 1 -- 172.21.15.67:0/3113691312 >> 172.21.15.67:0/3113691312 conn(0x7f1eac079160 msgr2=0x7f1eac079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:23.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.215+0000 7f1eb1943640 1 -- 172.21.15.67:0/3113691312 shutdown_connections 2024-07-31T15:08:23.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.215+0000 7f1eb1943640 1 -- 172.21.15.67:0/3113691312 wait complete. 2024-07-31T15:08:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1eb1943640 1 Processor -- start 2024-07-31T15:08:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1eb1943640 1 -- start start 2024-07-31T15:08:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1eb1943640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eac104290 0x7f1eac199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1eb1943640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1eac104cc0 0x7f1eac19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1eb1943640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eac10d840 0x7f1eac19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1eb1943640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1134b0 con 0x7f1eac104290 2024-07-31T15:08:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1eb1943640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac113330 con 0x7f1eac10d840 2024-07-31T15:08:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1eb1943640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac113630 con 0x7f1eac104cc0 2024-07-31T15:08:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1eaaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eac104290 0x7f1eac199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1ea3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1eac104cc0 0x7f1eac19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1eaaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eac104290 0x7f1eac199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52456/0 (socket says 172.21.15.67:52456) 2024-07-31T15:08:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1ea3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1eac104cc0 0x7f1eac19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43748/0 (socket says 172.21.15.67:43748) 2024-07-31T15:08:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1eaaffd640 1 -- 172.21.15.67:0/2093352366 learned_addr learned my addr 172.21.15.67:0/2093352366 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1eab7fe640 1 --2- 172.21.15.67:0/2093352366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eac10d840 0x7f1eac19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1ea3fff640 1 -- 172.21.15.67:0/2093352366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eac10d840 msgr2=0x7f1eac19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1ea3fff640 1 --2- 172.21.15.67:0/2093352366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eac10d840 0x7f1eac19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1ea3fff640 1 -- 172.21.15.67:0/2093352366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eac104290 msgr2=0x7f1eac199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1ea3fff640 1 --2- 172.21.15.67:0/2093352366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eac104290 0x7f1eac199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.216+0000 7f1ea3fff640 1 -- 172.21.15.67:0/2093352366 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eac19fe10 con 0x7f1eac104cc0 2024-07-31T15:08:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.217+0000 7f1ea3fff640 1 --2- 172.21.15.67:0/2093352366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1eac104cc0 0x7f1eac19a440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1eac06a760 tx=0x7f1e9400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.217+0000 7f1eb0941640 1 -- 172.21.15.67:0/2093352366 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9400cc40 con 0x7f1eac104cc0 2024-07-31T15:08:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.217+0000 7f1eb1943640 1 -- 172.21.15.67:0/2093352366 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eac1a0100 con 0x7f1eac104cc0 2024-07-31T15:08:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.217+0000 7f1eb0941640 1 -- 172.21.15.67:0/2093352366 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e9400cde0 con 0x7f1eac104cc0 2024-07-31T15:08:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.217+0000 7f1eb1943640 1 -- 172.21.15.67:0/2093352366 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eac1a6860 con 0x7f1eac104cc0 2024-07-31T15:08:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.217+0000 7f1eb0941640 1 -- 172.21.15.67:0/2093352366 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e94004a00 con 0x7f1eac104cc0 2024-07-31T15:08:23.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.219+0000 7f1eb7e94740 1 Processor -- start 2024-07-31T15:08:23.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.219+0000 7f1eb7e94740 1 -- start start 2024-07-31T15:08:23.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eb7e94740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 0x55c10cfd1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eb0941640 1 -- 172.21.15.67:0/2093352366 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1e94004ba0 con 0x7f1eac104cc0 2024-07-31T15:08:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eb7e94740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc031b0 0x55c10cfd1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eb7e94740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc0b390 0x55c10cfd2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eb7e94740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c10cf47bb0 con 0x55c10cc0b390 2024-07-31T15:08:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eb7e94740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c10cf370b0 con 0x55c10cc031b0 2024-07-31T15:08:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eb7e94740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c10cf3b0f0 con 0x55c10cc2c070 2024-07-31T15:08:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eab7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 0x55c10cfd1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1ea3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc0b390 0x55c10cfd2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eab7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 0x55c10cfd1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43754/0 (socket says 172.21.15.67:43754) 2024-07-31T15:08:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eab7fe640 1 -- 172.21.15.67:0/4052773483 learned_addr learned my addr 172.21.15.67:0/4052773483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eaaffd640 1 --2- 172.21.15.67:0/4052773483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc031b0 0x55c10cfd1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eab7fe640 1 -- 172.21.15.67:0/4052773483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc031b0 msgr2=0x55c10cfd1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eab7fe640 1 --2- 172.21.15.67:0/4052773483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc031b0 0x55c10cfd1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eab7fe640 1 -- 172.21.15.67:0/4052773483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc0b390 msgr2=0x55c10cfd2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eab7fe640 1 --2- 172.21.15.67:0/4052773483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc0b390 0x55c10cfd2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.220+0000 7f1eab7fe640 1 -- 172.21.15.67:0/4052773483 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c10ceddb90 con 0x55c10cc2c070 2024-07-31T15:08:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.221+0000 7f1eab7fe640 1 --2- 172.21.15.67:0/4052773483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 0x55c10cfd1690 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009fd0 tx=0x7f1ea40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b82583366d099ba server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.221+0000 7f1ea1ffb640 1 -- 172.21.15.67:0/4052773483 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4004d20 con 0x55c10cc2c070 2024-07-31T15:08:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.221+0000 7f1ea1ffb640 1 -- 172.21.15.67:0/4052773483 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea4004050 con 0x55c10cc2c070 2024-07-31T15:08:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.221+0000 7f1ea1ffb640 1 -- 172.21.15.67:0/4052773483 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4005260 con 0x55c10cc2c070 2024-07-31T15:08:23.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.221+0000 7f1eb0941640 1 --2- 172.21.15.67:0/2093352366 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1e980b3d40 0x7f1e980b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.222+0000 7f1eaaffd640 1 --2- 172.21.15.67:0/2093352366 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1e980b3d40 0x7f1e980b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.222+0000 7f1eb0941640 1 -- 172.21.15.67:0/2093352366 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9407+0+0 (secure 0 0 0) 0x7f1e940e42b0 con 0x7f1eac104cc0 2024-07-31T15:08:23.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.222+0000 7f1eb7e94740 1 -- 172.21.15.67:0/4052773483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 msgr2=0x55c10cfd1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:23.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.222+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/4052773483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 0x55c10cfd1690 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009fd0 tx=0x7f1ea40118d0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.222+0000 7f1eb7e94740 1 -- 172.21.15.67:0/4052773483 shutdown_connections 2024-07-31T15:08:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.222+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/4052773483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc0b390 0x55c10cfd2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.222+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/4052773483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc031b0 0x55c10cfd1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.222+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/4052773483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 0x55c10cfd1690 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.222+0000 7f1eb7e94740 1 -- 172.21.15.67:0/4052773483 >> 172.21.15.67:0/4052773483 conn(0x55c10cc20460 msgr2=0x55c10cfcf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.222+0000 7f1eaaffd640 1 --2- 172.21.15.67:0/2093352366 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1e980b3d40 0x7f1e980b6250 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c031440 tx=0x7f1e9c0058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.222+0000 7f1eb7e94740 1 -- 172.21.15.67:0/4052773483 shutdown_connections 2024-07-31T15:08:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.222+0000 7f1eb7e94740 1 -- 172.21.15.67:0/4052773483 wait complete. 2024-07-31T15:08:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1eb7e94740 1 -- start start 2024-07-31T15:08:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1eb7e94740 1 Processor -- start 2024-07-31T15:08:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1eb7e94740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc031b0 0x55c10cfdd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1eb7e94740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc0b390 0x55c10cfdd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1eb7e94740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc2c070 0x55c10cfddb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1eb7e94740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c10cd9a700 con 0x55c10cc0b390 2024-07-31T15:08:23.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1eb7e94740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c10ced99e0 con 0x55c10cc2c070 2024-07-31T15:08:23.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1eb7e94740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c10cfe41d0 con 0x55c10cc031b0 2024-07-31T15:08:23.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1eb7e94740 10 client.0 _ll_register_callbacks cb 0x55c10cef44f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:23.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1ea3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc0b390 0x55c10cfdd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1eab7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc2c070 0x55c10cfddb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1eaaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc031b0 0x55c10cfdd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1ea3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc0b390 0x55c10cfdd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52476/0 (socket says 172.21.15.67:52476) 2024-07-31T15:08:23.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1eab7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc2c070 0x55c10cfddb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:42038/0 (socket says 172.21.15.67:42038) 2024-07-31T15:08:23.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.223+0000 7f1ea3fff640 1 -- 172.21.15.67:0/2349440483 learned_addr learned my addr 172.21.15.67:0/2349440483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:23.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.224+0000 7f1eab7fe640 1 -- 172.21.15.67:0/2349440483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc031b0 msgr2=0x55c10cfdd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:23.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.224+0000 7f1eab7fe640 1 --2- 172.21.15.67:0/2349440483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc031b0 0x55c10cfdd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.224+0000 7f1eab7fe640 1 -- 172.21.15.67:0/2349440483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc0b390 msgr2=0x55c10cfdd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.224+0000 7f1eab7fe640 1 --2- 172.21.15.67:0/2349440483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc0b390 0x55c10cfdd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.224+0000 7f1eab7fe640 1 -- 172.21.15.67:0/2349440483 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c10cc97650 con 0x55c10cc2c070 2024-07-31T15:08:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.224+0000 7f1ea3fff640 1 --2- 172.21.15.67:0/2349440483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc0b390 0x55c10cfdd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.224+0000 7f1eab7fe640 1 --2- 172.21.15.67:0/2349440483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc2c070 0x55c10cfddb10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4005c10 tx=0x7f1ea4004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593451200ea43559 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.224+0000 7f1e8affd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:08:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.224+0000 7f1e8affd640 1 -- 172.21.15.67:0/2349440483 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4028020 con 0x55c10cc2c070 2024-07-31T15:08:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.224+0000 7f1eb7e94740 1 -- 172.21.15.67:0/2349440483 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea4004050 con 0x55c10cc2c070 2024-07-31T15:08:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.224+0000 7f1e8affd640 1 -- 172.21.15.67:0/2349440483 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9407+0+0 (secure 0 0 0) 0x7f1ea4004410 con 0x55c10cc2c070 2024-07-31T15:08:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.225+0000 7f1e8affd640 1 -- 172.21.15.67:0/2349440483 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f1ea4004e60 con 0x55c10cc2c070 2024-07-31T15:08:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.225+0000 7f1eb7e94740 10 client.28655 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.225+0000 7f1eb7e94740 1 -- 172.21.15.67:0/2349440483 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55c10cff45b0 con 0x55c10cc2c070 2024-07-31T15:08:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.225+0000 7f1e8affd640 1 -- 172.21.15.67:0/2349440483 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea4004050 con 0x55c10cc2c070 2024-07-31T15:08:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.225+0000 7f1e8affd640 1 -- 172.21.15.67:0/2349440483 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f1ea4023450 con 0x55c10cc2c070 2024-07-31T15:08:23.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.225+0000 7f1eb7e94740 10 client.28655 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:23.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.225+0000 7f1eb7e94740 10 client.28655 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:08:23.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.225+0000 7f1eb7e94740 20 client.28655 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:23.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.225+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/2349440483 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c10cbea9f0 0x55c10cff7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.225+0000 7f1eb7e94740 4 client.28655 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:08:23.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.225+0000 7f1eb7e94740 1 -- 172.21.15.67:0/2349440483 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c10cf49b00 con 0x55c10cbea9f0 2024-07-31T15:08:23.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.225+0000 7f1eaaffd640 1 --2- 172.21.15.67:0/2349440483 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c10cbea9f0 0x55c10cff7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.226+0000 7f1e8affd640 10 client.28655 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:08:23.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.226+0000 7f1eaaffd640 1 --2- 172.21.15.67:0/2349440483 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c10cbea9f0 0x55c10cff7910 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50ffad313f89be7e server_cookie=75268ebb9830f51c in_seq=0 out_seq=0 2024-07-31T15:08:23.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.383+0000 7f1e8affd640 1 -- 172.21.15.67:0/2349440483 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c10cf49b00 con 0x55c10cbea9f0 2024-07-31T15:08:23.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.383+0000 7f1e8affd640 10 client.28655 handle_command_reply: tid=0 2024-07-31T15:08:23.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.383+0000 7f1eb7e94740 1 client.28655 shutdown 2024-07-31T15:08:23.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.383+0000 7f1eb7e94740 1 -- 172.21.15.67:0/2349440483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc2c070 msgr2=0x55c10cfddb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:23.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.383+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/2349440483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc2c070 0x55c10cfddb10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4005c10 tx=0x7f1ea4004c70 comp rx=0 tx=0).stop 2024-07-31T15:08:23.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.383+0000 7f1eb7e94740 1 -- 172.21.15.67:0/2349440483 shutdown_connections 2024-07-31T15:08:23.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.383+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/2349440483 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c10cbea9f0 0x55c10cff7910 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.383+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/2349440483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc2c070 0x55c10cfddb10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.383+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/2349440483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc0b390 0x55c10cfdd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.383+0000 7f1e8affd640 0 client.28655 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:08:23.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.383+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/2349440483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc031b0 0x55c10cfdd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.383+0000 7f1eb7e94740 1 -- 172.21.15.67:0/2349440483 >> 172.21.15.67:0/2349440483 conn(0x55c10cc20460 msgr2=0x55c10cfcdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:23.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.383+0000 7f1eb7e94740 1 -- 172.21.15.67:0/2349440483 shutdown_connections 2024-07-31T15:08:23.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.384+0000 7f1eb7e94740 1 -- 172.21.15.67:0/2349440483 wait complete. 2024-07-31T15:08:23.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.384+0000 7f1eb7e94740 20 client.28655 trim_cache size 0 max 16384 2024-07-31T15:08:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.399+0000 7f1eb7e94740 1 Processor -- start 2024-07-31T15:08:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.399+0000 7f1eb7e94740 1 -- start start 2024-07-31T15:08:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.399+0000 7f1eb7e94740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc031b0 0x55c10cfcf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.399+0000 7f1eb7e94740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc0b390 0x55c10cffcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.399+0000 7f1eb7e94740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 0x55c10cffd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.399+0000 7f1eb7e94740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1ae660 con 0x55c10cc031b0 2024-07-31T15:08:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.399+0000 7f1eb7e94740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1b01a0 con 0x55c10cc0b390 2024-07-31T15:08:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1eb7e94740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1aed30 con 0x55c10cc2c070 2024-07-31T15:08:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1eaaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc031b0 0x55c10cfcf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1ea3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc0b390 0x55c10cffcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1eaaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc031b0 0x55c10cfcf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52486/0 (socket says 172.21.15.67:52486) 2024-07-31T15:08:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1eaaffd640 1 -- 172.21.15.67:0/3868025331 learned_addr learned my addr 172.21.15.67:0/3868025331 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1eab7fe640 1 --2- 172.21.15.67:0/3868025331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 0x55c10cffd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1ea3fff640 1 -- 172.21.15.67:0/3868025331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 msgr2=0x55c10cffd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1ea3fff640 1 --2- 172.21.15.67:0/3868025331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 0x55c10cffd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1ea3fff640 1 -- 172.21.15.67:0/3868025331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc031b0 msgr2=0x55c10cfcf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1ea3fff640 1 --2- 172.21.15.67:0/3868025331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc031b0 0x55c10cfcf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1ea3fff640 1 -- 172.21.15.67:0/3868025331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c10cfd3ba0 con 0x55c10cc0b390 2024-07-31T15:08:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1eaaffd640 1 --2- 172.21.15.67:0/3868025331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc031b0 0x55c10cfcf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1ea3fff640 1 --2- 172.21.15.67:0/3868025331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc0b390 0x55c10cffcfc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1e9401e000 tx=0x7f1e94005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6783521a1f23821 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1ea17fa640 1 -- 172.21.15.67:0/3868025331 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9401c040 con 0x55c10cc0b390 2024-07-31T15:08:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.400+0000 7f1ea17fa640 1 -- 172.21.15.67:0/3868025331 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e940aa660 con 0x55c10cc0b390 2024-07-31T15:08:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.401+0000 7f1ea17fa640 1 -- 172.21.15.67:0/3868025331 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e940aa2c0 con 0x55c10cc0b390 2024-07-31T15:08:23.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.401+0000 7f1eb7e94740 1 -- 172.21.15.67:0/3868025331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc0b390 msgr2=0x55c10cffcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:23.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.401+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/3868025331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc0b390 0x55c10cffcfc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1e9401e000 tx=0x7f1e94005f10 comp rx=0 tx=0).stop 2024-07-31T15:08:23.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.401+0000 7f1eb7e94740 1 -- 172.21.15.67:0/3868025331 shutdown_connections 2024-07-31T15:08:23.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.401+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/3868025331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 0x55c10cffd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.401+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/3868025331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc0b390 0x55c10cffcfc0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.401+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/3868025331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc031b0 0x55c10cfcf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.401+0000 7f1eb7e94740 1 -- 172.21.15.67:0/3868025331 >> 172.21.15.67:0/3868025331 conn(0x55c10cc20460 msgr2=0x55c10cfceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.402+0000 7f1eb7e94740 1 -- 172.21.15.67:0/3868025331 shutdown_connections 2024-07-31T15:08:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.402+0000 7f1eb7e94740 1 -- 172.21.15.67:0/3868025331 wait complete. 2024-07-31T15:08:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.402+0000 7f1eb7e94740 1 -- start start 2024-07-31T15:08:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.402+0000 7f1eb7e94740 1 Processor -- start 2024-07-31T15:08:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.402+0000 7f1eb7e94740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc031b0 0x55c10cff2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.402+0000 7f1eb7e94740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc0b390 0x55c10cfed6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.402+0000 7f1eb7e94740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 0x55c10cfedc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.402+0000 7f1eb7e94740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1afad0 con 0x55c10cc031b0 2024-07-31T15:08:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.402+0000 7f1eb7e94740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1af400 con 0x55c10cc0b390 2024-07-31T15:08:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.402+0000 7f1eb7e94740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1ac640 con 0x55c10cc2c070 2024-07-31T15:08:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1eb7e94740 10 client.0 _ll_register_callbacks cb 0x7f1e940e7cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1eaaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc031b0 0x55c10cff2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1ea3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc0b390 0x55c10cfed6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1ea3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc0b390 0x55c10cfed6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:42044/0 (socket says 172.21.15.67:42044) 2024-07-31T15:08:23.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1ea3fff640 1 -- 172.21.15.67:0/428568568 learned_addr learned my addr 172.21.15.67:0/428568568 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:23.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1eaaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc031b0 0x55c10cff2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52498/0 (socket says 172.21.15.67:52498) 2024-07-31T15:08:23.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1eab7fe640 1 --2- 172.21.15.67:0/428568568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 0x55c10cfedc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1ea3fff640 1 -- 172.21.15.67:0/428568568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 msgr2=0x55c10cfedc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:23.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1ea3fff640 1 --2- 172.21.15.67:0/428568568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 0x55c10cfedc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1ea3fff640 1 -- 172.21.15.67:0/428568568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc031b0 msgr2=0x55c10cff2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:23.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1ea3fff640 1 --2- 172.21.15.67:0/428568568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc031b0 0x55c10cff2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1ea3fff640 1 -- 172.21.15.67:0/428568568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c10cfee140 con 0x55c10cc0b390 2024-07-31T15:08:23.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1ea3fff640 1 --2- 172.21.15.67:0/428568568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc0b390 0x55c10cfed6c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400c6a0 tx=0x7f1e940aafe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e87faa1088f15ae4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:23.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1e8affd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:08:23.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1e8affd640 1 -- 172.21.15.67:0/428568568 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e940a62e0 con 0x55c10cc0b390 2024-07-31T15:08:23.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1e8affd640 1 -- 172.21.15.67:0/428568568 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9407+0+0 (secure 0 0 0) 0x7f1e940b3980 con 0x55c10cc0b390 2024-07-31T15:08:23.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.403+0000 7f1eb7e94740 1 -- 172.21.15.67:0/428568568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e940aa660 con 0x55c10cc0b390 2024-07-31T15:08:23.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.404+0000 7f1e8affd640 1 -- 172.21.15.67:0/428568568 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f1e940aa2c0 con 0x55c10cc0b390 2024-07-31T15:08:23.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.404+0000 7f1eb7e94740 10 client.28661 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:23.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.404+0000 7f1eb7e94740 1 -- 172.21.15.67:0/428568568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55c10d05fb90 con 0x55c10cc0b390 2024-07-31T15:08:23.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.404+0000 7f1e8affd640 1 -- 172.21.15.67:0/428568568 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e940aa660 con 0x55c10cc0b390 2024-07-31T15:08:23.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.404+0000 7f1e8affd640 1 -- 172.21.15.67:0/428568568 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f1e940a64c0 con 0x55c10cc0b390 2024-07-31T15:08:23.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.404+0000 7f1eb7e94740 10 client.28661 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:23.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.404+0000 7f1eb7e94740 10 client.28661 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:08:23.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.404+0000 7f1eb7e94740 20 client.28661 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:23.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.404+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/428568568 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c10cbbfda0 0x55c10d062290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:23.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.404+0000 7f1eb7e94740 4 client.28661 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:08:23.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.404+0000 7f1eb7e94740 1 -- 172.21.15.67:0/428568568 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c10cf455a0 con 0x55c10cbbfda0 2024-07-31T15:08:23.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.404+0000 7f1eaaffd640 1 --2- 172.21.15.67:0/428568568 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c10cbbfda0 0x55c10d062290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:23.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.406+0000 7f1eaaffd640 1 --2- 172.21.15.67:0/428568568 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c10cbbfda0 0x55c10d062290 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43a297614077b103 server_cookie=17edad6f0a744948 in_seq=0 out_seq=0 2024-07-31T15:08:23.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.406+0000 7f1e8affd640 10 client.28661 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:08:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:23 smithi077 ceph-mon[37327]: pgmap v949: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.4 MiB/s wr, 253 op/s 2024-07-31T15:08:23.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1e8affd640 1 -- 172.21.15.67:0/428568568 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c10cf455a0 con 0x55c10cbbfda0 2024-07-31T15:08:23.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1e8affd640 10 client.28661 handle_command_reply: tid=0 2024-07-31T15:08:23.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1eb7e94740 1 client.28661 shutdown 2024-07-31T15:08:23.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:23 smithi067 ceph-mon[32750]: pgmap v949: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.4 MiB/s wr, 253 op/s 2024-07-31T15:08:23.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1eb7e94740 1 -- 172.21.15.67:0/428568568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc0b390 msgr2=0x55c10cfed6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:23.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/428568568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc0b390 0x55c10cfed6c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400c6a0 tx=0x7f1e940aafe0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1eb7e94740 1 -- 172.21.15.67:0/428568568 shutdown_connections 2024-07-31T15:08:23.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/428568568 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55c10cbbfda0 0x55c10d062290 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/428568568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10cc2c070 0x55c10cfedc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/428568568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c10cc0b390 0x55c10cfed6c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1e8affd640 0 client.28661 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:08:23.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1eb7e94740 1 --2- 172.21.15.67:0/428568568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c10cc031b0 0x55c10cff2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1eb7e94740 1 -- 172.21.15.67:0/428568568 >> 172.21.15.67:0/428568568 conn(0x55c10cc20460 msgr2=0x55c10cfd26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:23.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1eb7e94740 1 -- 172.21.15.67:0/428568568 shutdown_connections 2024-07-31T15:08:23.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1eb7e94740 1 -- 172.21.15.67:0/428568568 wait complete. 2024-07-31T15:08:23.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.495+0000 7f1eb7e94740 20 client.28661 trim_cache size 0 max 16384 2024-07-31T15:08:23.499 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:08:23.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.496+0000 7f1eb1943640 1 -- 172.21.15.67:0/2093352366 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1e980b3d40 msgr2=0x7f1e980b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:23.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.496+0000 7f1eb1943640 1 --2- 172.21.15.67:0/2093352366 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1e980b3d40 0x7f1e980b6250 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c031440 tx=0x7f1e9c0058d0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.496+0000 7f1eb1943640 1 -- 172.21.15.67:0/2093352366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1eac104cc0 msgr2=0x7f1eac19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:23.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.496+0000 7f1eb1943640 1 --2- 172.21.15.67:0/2093352366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1eac104cc0 0x7f1eac19a440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1eac06a760 tx=0x7f1e9400ee50 comp rx=0 tx=0).stop 2024-07-31T15:08:23.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.496+0000 7f1eb1943640 1 -- 172.21.15.67:0/2093352366 shutdown_connections 2024-07-31T15:08:23.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.496+0000 7f1eb1943640 1 --2- 172.21.15.67:0/2093352366 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1e980b3d40 0x7f1e980b6250 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.496+0000 7f1eb1943640 1 --2- 172.21.15.67:0/2093352366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eac10d840 0x7f1eac19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.496+0000 7f1eb1943640 1 --2- 172.21.15.67:0/2093352366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1eac104cc0 0x7f1eac19a440 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.496+0000 7f1eb1943640 1 --2- 172.21.15.67:0/2093352366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eac104290 0x7f1eac199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:23.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.496+0000 7f1eb1943640 1 -- 172.21.15.67:0/2093352366 >> 172.21.15.67:0/2093352366 conn(0x7f1eac079160 msgr2=0x7f1eac100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:23.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.496+0000 7f1eb1943640 1 -- 172.21.15.67:0/2093352366 shutdown_connections 2024-07-31T15:08:23.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:23.496+0000 7f1eb1943640 1 -- 172.21.15.67:0/2093352366 wait complete. 2024-07-31T15:08:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:23 smithi073 ceph-mon[36714]: pgmap v949: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.4 MiB/s wr, 253 op/s 2024-07-31T15:08:24.333 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T15:08:24.643 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:08:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:25 smithi067 ceph-mon[32750]: pgmap v950: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.1 MiB/s wr, 225 op/s 2024-07-31T15:08:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:25 smithi077 ceph-mon[37327]: pgmap v950: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.1 MiB/s wr, 225 op/s 2024-07-31T15:08:25.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:25 smithi073 ceph-mon[36714]: pgmap v950: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.1 MiB/s wr, 225 op/s 2024-07-31T15:08:26.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:26 smithi067 ceph-mon[32750]: osdmap e121: 12 total, 12 up, 12 in 2024-07-31T15:08:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.450+0000 7feb10ee3640 1 -- 172.21.15.67:0/3094897592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb0c1024e0 msgr2=0x7feb0c1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.450+0000 7feb10ee3640 1 --2- 172.21.15.67:0/3094897592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb0c1024e0 0x7feb0c1028f0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7feafc009900 tx=0x7feafc02a410 comp rx=0 tx=0).stop 2024-07-31T15:08:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.450+0000 7feb10ee3640 1 -- 172.21.15.67:0/3094897592 shutdown_connections 2024-07-31T15:08:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.450+0000 7feb10ee3640 1 --2- 172.21.15.67:0/3094897592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb0c108720 0x7feb0c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.450+0000 7feb10ee3640 1 --2- 172.21.15.67:0/3094897592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb0c107dd0 0x7feb0c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.450+0000 7feb10ee3640 1 --2- 172.21.15.67:0/3094897592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb0c1024e0 0x7feb0c1028f0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.450+0000 7feb10ee3640 1 -- 172.21.15.67:0/3094897592 >> 172.21.15.67:0/3094897592 conn(0x7feb0c0790b0 msgr2=0x7feb0c0794c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.450+0000 7feb10ee3640 1 -- 172.21.15.67:0/3094897592 shutdown_connections 2024-07-31T15:08:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.450+0000 7feb10ee3640 1 -- 172.21.15.67:0/3094897592 wait complete. 2024-07-31T15:08:26.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.451+0000 7feb10ee3640 1 Processor -- start 2024-07-31T15:08:26.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.451+0000 7feb10ee3640 1 -- start start 2024-07-31T15:08:26.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.451+0000 7feb10ee3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb0c1024e0 0x7feb0c0ff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.451+0000 7feb10ee3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb0c107dd0 0x7feb0c0ff630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.451+0000 7feb10ee3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb0c108720 0x7feb0c0ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.451+0000 7feb10ee3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c1123d0 con 0x7feb0c108720 2024-07-31T15:08:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.451+0000 7feb10ee3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c112250 con 0x7feb0c1024e0 2024-07-31T15:08:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.451+0000 7feb10ee3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c112550 con 0x7feb0c107dd0 2024-07-31T15:08:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.451+0000 7feb0ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb0c108720 0x7feb0c0ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.451+0000 7feb0a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb0c1024e0 0x7feb0c0ff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.451+0000 7feb0ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb0c108720 0x7feb0c0ffb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52516/0 (socket says 172.21.15.67:52516) 2024-07-31T15:08:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.451+0000 7feb0ad9d640 1 -- 172.21.15.67:0/2451075991 learned_addr learned my addr 172.21.15.67:0/2451075991 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.451+0000 7feb09d9b640 1 --2- 172.21.15.67:0/2451075991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb0c107dd0 0x7feb0c0ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.452+0000 7feb0a59c640 1 -- 172.21.15.67:0/2451075991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb0c107dd0 msgr2=0x7feb0c0ff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.452+0000 7feb0a59c640 1 --2- 172.21.15.67:0/2451075991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb0c107dd0 0x7feb0c0ff630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.452+0000 7feb0a59c640 1 -- 172.21.15.67:0/2451075991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb0c108720 msgr2=0x7feb0c0ffb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.452+0000 7feb0a59c640 1 --2- 172.21.15.67:0/2451075991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb0c108720 0x7feb0c0ffb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.452+0000 7feb0a59c640 1 -- 172.21.15.67:0/2451075991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c1a81d0 con 0x7feb0c1024e0 2024-07-31T15:08:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.452+0000 7feb0ad9d640 1 --2- 172.21.15.67:0/2451075991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb0c108720 0x7feb0c0ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.452+0000 7feb0a59c640 1 --2- 172.21.15.67:0/2451075991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb0c1024e0 0x7feb0c0ff0f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feafc02a930 tx=0x7feafc031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.452+0000 7feafb7fe640 1 -- 172.21.15.67:0/2451075991 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc03f030 con 0x7feb0c1024e0 2024-07-31T15:08:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.452+0000 7feb10ee3640 1 -- 172.21.15.67:0/2451075991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0c1a8460 con 0x7feb0c1024e0 2024-07-31T15:08:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.452+0000 7feafb7fe640 1 -- 172.21.15.67:0/2451075991 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feafc004490 con 0x7feb0c1024e0 2024-07-31T15:08:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.452+0000 7feafb7fe640 1 -- 172.21.15.67:0/2451075991 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc035070 con 0x7feb0c1024e0 2024-07-31T15:08:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.452+0000 7feb10ee3640 1 -- 172.21.15.67:0/2451075991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0c1a8940 con 0x7feb0c1024e0 2024-07-31T15:08:26.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb17434740 1 Processor -- start 2024-07-31T15:08:26.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb17434740 1 -- start start 2024-07-31T15:08:26.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb17434740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c639b070 0x5592c6740570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb17434740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 0x5592c6740ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb17434740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c6740ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb17434740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5592c66b6bb0 con 0x5592c639b070 2024-07-31T15:08:26.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb17434740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5592c66a60b0 con 0x5592c63721b0 2024-07-31T15:08:26.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb17434740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5592c66aa0f0 con 0x5592c637a390 2024-07-31T15:08:26.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb0ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c639b070 0x5592c6740570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb0ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c639b070 0x5592c6740570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52520/0 (socket says 172.21.15.67:52520) 2024-07-31T15:08:26.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb0ad9d640 1 -- 172.21.15.67:0/3922773116 learned_addr learned my addr 172.21.15.67:0/3922773116 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:26.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb09d9b640 1 --2- 172.21.15.67:0/3922773116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c6740ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feafb7fe640 1 -- 172.21.15.67:0/2451075991 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7feafc042030 con 0x7feb0c1024e0 2024-07-31T15:08:26.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb0ad9d640 1 -- 172.21.15.67:0/3922773116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 msgr2=0x5592c6740ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:26.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb0ad9d640 1 --2- 172.21.15.67:0/3922773116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c6740ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb0ad9d640 1 -- 172.21.15.67:0/3922773116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 msgr2=0x5592c6740ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:26.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.455+0000 7feb0ad9d640 1 --2- 172.21.15.67:0/3922773116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 0x5592c6740ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.456+0000 7feb0ad9d640 1 -- 172.21.15.67:0/3922773116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c664cb90 con 0x5592c639b070 2024-07-31T15:08:26.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.456+0000 7feb09d9b640 1 --2- 172.21.15.67:0/3922773116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c6740ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:26.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.456+0000 7feb0ad9d640 1 --2- 172.21.15.67:0/3922773116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c639b070 0x5592c6740570 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7feb040118b0 tx=0x7feb04011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a20bcaaae2c233 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:26.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.456+0000 7feaf8ff9640 1 -- 172.21.15.67:0/3922773116 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0401e070 con 0x5592c639b070 2024-07-31T15:08:26.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.456+0000 7feaf8ff9640 1 -- 172.21.15.67:0/3922773116 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0400fe70 con 0x5592c639b070 2024-07-31T15:08:26.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.456+0000 7feaf8ff9640 1 -- 172.21.15.67:0/3922773116 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04019d40 con 0x5592c639b070 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.457+0000 7feafb7fe640 1 --2- 172.21.15.67:0/2451075991 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feaf40b3d40 0x7feaf40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.457+0000 7feb17434740 1 -- 172.21.15.67:0/3922773116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c639b070 msgr2=0x5592c6740570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.457+0000 7feb17434740 1 --2- 172.21.15.67:0/3922773116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c639b070 0x5592c6740570 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7feb040118b0 tx=0x7feb04011d70 comp rx=0 tx=0).stop 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.457+0000 7feb0a59c640 1 --2- 172.21.15.67:0/2451075991 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feaf40b3d40 0x7feaf40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.457+0000 7feb17434740 1 -- 172.21.15.67:0/3922773116 shutdown_connections 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.457+0000 7feb17434740 1 --2- 172.21.15.67:0/3922773116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c6740ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.457+0000 7feb17434740 1 --2- 172.21.15.67:0/3922773116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 0x5592c6740ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.457+0000 7feb17434740 1 --2- 172.21.15.67:0/3922773116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c639b070 0x5592c6740570 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.457+0000 7feb17434740 1 -- 172.21.15.67:0/3922773116 >> 172.21.15.67:0/3922773116 conn(0x5592c638f460 msgr2=0x5592c673e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.457+0000 7feafb7fe640 1 -- 172.21.15.67:0/2451075991 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7feafc104280 con 0x7feb0c1024e0 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.457+0000 7feb17434740 1 -- 172.21.15.67:0/3922773116 shutdown_connections 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.457+0000 7feb17434740 1 -- 172.21.15.67:0/3922773116 wait complete. 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.457+0000 7feb0a59c640 1 --2- 172.21.15.67:0/2451075991 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feaf40b3d40 0x7feaf40b6250 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x5592c673f1f0 tx=0x7feafc03b560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.458+0000 7feb17434740 1 -- start start 2024-07-31T15:08:26.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.458+0000 7feb17434740 1 Processor -- start 2024-07-31T15:08:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.458+0000 7feb17434740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c63721b0 0x5592c674c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.458+0000 7feb17434740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c674c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.458+0000 7feb17434740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c639b070 0x5592c67580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.458+0000 7feb17434740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5592c6509700 con 0x5592c63721b0 2024-07-31T15:08:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.458+0000 7feb17434740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5592c66489e0 con 0x5592c639b070 2024-07-31T15:08:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.458+0000 7feb17434740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5592c674ed50 con 0x5592c637a390 2024-07-31T15:08:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.458+0000 7feb17434740 10 client.0 _ll_register_callbacks cb 0x5592c66634f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.458+0000 7feb0ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c63721b0 0x5592c674c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.458+0000 7feb0ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c63721b0 0x5592c674c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52522/0 (socket says 172.21.15.67:52522) 2024-07-31T15:08:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.458+0000 7feb0ad9d640 1 -- 172.21.15.67:0/265234197 learned_addr learned my addr 172.21.15.67:0/265234197 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.458+0000 7feb09d9b640 1 --2- 172.21.15.67:0/265234197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c674c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.458+0000 7feb0ad9d640 1 --2- 172.21.15.67:0/265234197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c639b070 0x5592c67580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.459+0000 7feb0ad9d640 1 -- 172.21.15.67:0/265234197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 msgr2=0x5592c674c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.459+0000 7feb0ad9d640 1 --2- 172.21.15.67:0/265234197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c674c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.459+0000 7feb0ad9d640 1 -- 172.21.15.67:0/265234197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c639b070 msgr2=0x5592c67580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.459+0000 7feb0ad9d640 1 --2- 172.21.15.67:0/265234197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c639b070 0x5592c67580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.459+0000 7feb0ad9d640 1 -- 172.21.15.67:0/265234197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c6406650 con 0x5592c63721b0 2024-07-31T15:08:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.459+0000 7feb09d9b640 1 --2- 172.21.15.67:0/265234197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c674c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.459+0000 7feb0ad9d640 1 --2- 172.21.15.67:0/265234197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c63721b0 0x5592c674c0c0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7feb0401aef0 tx=0x7feb04019940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a924c25c3417fd48 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.459+0000 7feaf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:08:26.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.459+0000 7feaf1ffb640 1 -- 172.21.15.67:0/265234197 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0401e030 con 0x5592c63721b0 2024-07-31T15:08:26.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.459+0000 7feaf1ffb640 1 -- 172.21.15.67:0/265234197 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7feb04011110 con 0x5592c63721b0 2024-07-31T15:08:26.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.459+0000 7feb17434740 1 -- 172.21.15.67:0/265234197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0400fe70 con 0x5592c63721b0 2024-07-31T15:08:26.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.459+0000 7feaf1ffb640 1 -- 172.21.15.67:0/265234197 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7feb04036050 con 0x5592c63721b0 2024-07-31T15:08:26.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.459+0000 7feb17434740 10 client.19125 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:26.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.459+0000 7feb17434740 1 -- 172.21.15.67:0/265234197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5592c675acf0 con 0x5592c63721b0 2024-07-31T15:08:26.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.460+0000 7feaf1ffb640 1 -- 172.21.15.67:0/265234197 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0400fe70 con 0x5592c63721b0 2024-07-31T15:08:26.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.460+0000 7feaf1ffb640 1 -- 172.21.15.67:0/265234197 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7feb04010c10 con 0x5592c63721b0 2024-07-31T15:08:26.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.460+0000 7feb17434740 10 client.19125 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:26.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.460+0000 7feb17434740 10 client.19125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:08:26.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.460+0000 7feb17434740 20 client.19125 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:26.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.460+0000 7feb17434740 1 --2- 172.21.15.67:0/265234197 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5592c63599f0 0x5592c675e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.460+0000 7feb17434740 4 client.19125 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:08:26.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.460+0000 7feb17434740 1 -- 172.21.15.67:0/265234197 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592c66b8b00 con 0x5592c63599f0 2024-07-31T15:08:26.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.460+0000 7feb09d9b640 1 --2- 172.21.15.67:0/265234197 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5592c63599f0 0x5592c675e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.460+0000 7feb09d9b640 1 --2- 172.21.15.67:0/265234197 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5592c63599f0 0x5592c675e160 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=194eb7201876aadc server_cookie=613abc90a97a1f3f in_seq=0 out_seq=0 2024-07-31T15:08:26.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.460+0000 7feaf1ffb640 10 client.19125 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:26.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:26 smithi077 ceph-mon[37327]: osdmap e121: 12 total, 12 up, 12 in 2024-07-31T15:08:26.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:26 smithi073 ceph-mon[36714]: osdmap e121: 12 total, 12 up, 12 in 2024-07-31T15:08:26.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.788+0000 7feaf1ffb640 1 -- 172.21.15.67:0/265234197 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592c66b8b00 con 0x5592c63599f0 2024-07-31T15:08:26.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.788+0000 7feaf1ffb640 10 client.19125 handle_command_reply: tid=0 2024-07-31T15:08:26.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.788+0000 7feb17434740 1 client.19125 shutdown 2024-07-31T15:08:26.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.788+0000 7feb17434740 1 -- 172.21.15.67:0/265234197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c63721b0 msgr2=0x5592c674c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:26.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.788+0000 7feb17434740 1 --2- 172.21.15.67:0/265234197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c63721b0 0x5592c674c0c0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7feb0401aef0 tx=0x7feb04019940 comp rx=0 tx=0).stop 2024-07-31T15:08:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.788+0000 7feb17434740 1 -- 172.21.15.67:0/265234197 shutdown_connections 2024-07-31T15:08:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.788+0000 7feb17434740 1 --2- 172.21.15.67:0/265234197 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5592c63599f0 0x5592c675e160 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.788+0000 7feb17434740 1 --2- 172.21.15.67:0/265234197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c639b070 0x5592c67580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.788+0000 7feb17434740 1 --2- 172.21.15.67:0/265234197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c674c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.788+0000 7feb17434740 1 --2- 172.21.15.67:0/265234197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c63721b0 0x5592c674c0c0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.788+0000 7feb17434740 1 -- 172.21.15.67:0/265234197 >> 172.21.15.67:0/265234197 conn(0x5592c638f460 msgr2=0x5592c6743750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.788+0000 7feaf1ffb640 0 client.19125 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.788+0000 7feb17434740 1 -- 172.21.15.67:0/265234197 shutdown_connections 2024-07-31T15:08:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.789+0000 7feb17434740 1 -- 172.21.15.67:0/265234197 wait complete. 2024-07-31T15:08:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.789+0000 7feb17434740 20 client.19125 trim_cache size 0 max 16384 2024-07-31T15:08:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.803+0000 7feb17434740 1 Processor -- start 2024-07-31T15:08:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.803+0000 7feb17434740 1 -- start start 2024-07-31T15:08:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.803+0000 7feb17434740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 0x5592c6743d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.803+0000 7feb17434740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c637a390 0x5592c67457f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.803+0000 7feb17434740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c639b070 0x5592c6745d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.803+0000 7feb17434740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c1b2a50 con 0x5592c637a390 2024-07-31T15:08:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.803+0000 7feb17434740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c1b4590 con 0x5592c63721b0 2024-07-31T15:08:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.803+0000 7feb17434740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c1b3120 con 0x5592c639b070 2024-07-31T15:08:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.803+0000 7feb09d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c637a390 0x5592c67457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.803+0000 7feb09d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c637a390 0x5592c67457f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52524/0 (socket says 172.21.15.67:52524) 2024-07-31T15:08:26.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.803+0000 7feb0ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 0x5592c6743d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.803+0000 7feb09d9b640 1 -- 172.21.15.67:0/1868649711 learned_addr learned my addr 172.21.15.67:0/1868649711 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:26.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.803+0000 7feb0ad9d640 1 --2- 172.21.15.67:0/1868649711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c639b070 0x5592c6745d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.804+0000 7feb09d9b640 1 -- 172.21.15.67:0/1868649711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c639b070 msgr2=0x5592c6745d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:26.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.804+0000 7feb09d9b640 1 --2- 172.21.15.67:0/1868649711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c639b070 0x5592c6745d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.804+0000 7feb09d9b640 1 -- 172.21.15.67:0/1868649711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 msgr2=0x5592c6743d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:26.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.804+0000 7feb09d9b640 1 --2- 172.21.15.67:0/1868649711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 0x5592c6743d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.804+0000 7feb09d9b640 1 -- 172.21.15.67:0/1868649711 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c674cd50 con 0x5592c637a390 2024-07-31T15:08:26.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.804+0000 7feb0ad9d640 1 --2- 172.21.15.67:0/1868649711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 0x5592c6743d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:26.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.804+0000 7feb09d9b640 1 --2- 172.21.15.67:0/1868649711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c637a390 0x5592c67457f0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7feb0c06b3a0 tx=0x7feb00005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbe516008aa631b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:26.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.804+0000 7feaf3fff640 1 -- 172.21.15.67:0/1868649711 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb000122d0 con 0x5592c637a390 2024-07-31T15:08:26.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.804+0000 7feaf3fff640 1 -- 172.21.15.67:0/1868649711 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0000fd50 con 0x5592c637a390 2024-07-31T15:08:26.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.804+0000 7feaf3fff640 1 -- 172.21.15.67:0/1868649711 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00010d00 con 0x5592c637a390 2024-07-31T15:08:26.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.805+0000 7feb17434740 1 -- 172.21.15.67:0/1868649711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c637a390 msgr2=0x5592c67457f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:26.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.805+0000 7feb17434740 1 --2- 172.21.15.67:0/1868649711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c637a390 0x5592c67457f0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7feb0c06b3a0 tx=0x7feb00005f90 comp rx=0 tx=0).stop 2024-07-31T15:08:26.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.805+0000 7feb17434740 1 -- 172.21.15.67:0/1868649711 shutdown_connections 2024-07-31T15:08:26.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.805+0000 7feb17434740 1 --2- 172.21.15.67:0/1868649711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c639b070 0x5592c6745d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.805+0000 7feb17434740 1 --2- 172.21.15.67:0/1868649711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c637a390 0x5592c67457f0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.805+0000 7feb17434740 1 --2- 172.21.15.67:0/1868649711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 0x5592c6743d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.805+0000 7feb17434740 1 -- 172.21.15.67:0/1868649711 >> 172.21.15.67:0/1868649711 conn(0x5592c638f460 msgr2=0x5592c6743670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.805+0000 7feb17434740 1 -- 172.21.15.67:0/1868649711 shutdown_connections 2024-07-31T15:08:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.805+0000 7feb17434740 1 -- 172.21.15.67:0/1868649711 wait complete. 2024-07-31T15:08:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb17434740 1 -- start start 2024-07-31T15:08:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb17434740 1 Processor -- start 2024-07-31T15:08:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb17434740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 0x5592c673cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb17434740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c673d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb17434740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c639b070 0x5592c673d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb17434740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c1b3ec0 con 0x5592c639b070 2024-07-31T15:08:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb17434740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c1b37f0 con 0x5592c63721b0 2024-07-31T15:08:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb17434740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c1b0a50 con 0x5592c637a390 2024-07-31T15:08:26.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb17434740 10 client.0 _ll_register_callbacks cb 0x7feb0401ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:26.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb0ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c639b070 0x5592c673d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb09d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c673d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb0ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 0x5592c673cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb0ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 0x5592c673cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:42108/0 (socket says 172.21.15.67:42108) 2024-07-31T15:08:26.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb0ad9d640 1 -- 172.21.15.67:0/797438450 learned_addr learned my addr 172.21.15.67:0/797438450 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:26.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.806+0000 7feb09d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c673d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43860/0 (socket says 172.21.15.67:43860) 2024-07-31T15:08:26.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.807+0000 7feb0ad9d640 1 -- 172.21.15.67:0/797438450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 msgr2=0x5592c673d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:26.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.807+0000 7feb0ad9d640 1 --2- 172.21.15.67:0/797438450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c673d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.807+0000 7feb0ad9d640 1 -- 172.21.15.67:0/797438450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c639b070 msgr2=0x5592c673d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:26.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.807+0000 7feb0ad9d640 1 --2- 172.21.15.67:0/797438450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c639b070 0x5592c673d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.807+0000 7feb0ad9d640 1 -- 172.21.15.67:0/797438450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c6753080 con 0x5592c63721b0 2024-07-31T15:08:26.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.807+0000 7feb0ad9d640 1 --2- 172.21.15.67:0/797438450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 0x5592c673cc70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feb0401aef0 tx=0x7feb04000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=594a654ae1580ade server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:26.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.807+0000 7feaf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:08:26.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.807+0000 7feaf1ffb640 1 -- 172.21.15.67:0/797438450 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0401e040 con 0x5592c63721b0 2024-07-31T15:08:26.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.807+0000 7feb17434740 1 -- 172.21.15.67:0/797438450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0000fd50 con 0x5592c63721b0 2024-07-31T15:08:26.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.807+0000 7feaf1ffb640 1 -- 172.21.15.67:0/797438450 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7feb04028be0 con 0x5592c63721b0 2024-07-31T15:08:26.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.807+0000 7feaf1ffb640 1 -- 172.21.15.67:0/797438450 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7feb0404c050 con 0x5592c63721b0 2024-07-31T15:08:26.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.807+0000 7feb17434740 10 client.28676 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:26.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.807+0000 7feb17434740 1 -- 172.21.15.67:0/797438450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5592c67542c0 con 0x5592c63721b0 2024-07-31T15:08:26.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.808+0000 7feaf1ffb640 1 -- 172.21.15.67:0/797438450 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0000fd50 con 0x5592c63721b0 2024-07-31T15:08:26.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.808+0000 7feaf1ffb640 1 -- 172.21.15.67:0/797438450 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7feb04011170 con 0x5592c63721b0 2024-07-31T15:08:26.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.808+0000 7feb17434740 10 client.28676 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:26.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.808+0000 7feb17434740 10 client.28676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:08:26.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.808+0000 7feb17434740 20 client.28676 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:26.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.808+0000 7feb17434740 1 --2- 172.21.15.67:0/797438450 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5592c632eda0 0x5592c67c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:26.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.808+0000 7feb17434740 4 client.28676 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:08:26.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.808+0000 7feb17434740 1 -- 172.21.15.67:0/797438450 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592c66b45a0 con 0x5592c632eda0 2024-07-31T15:08:26.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.808+0000 7feb09d9b640 1 --2- 172.21.15.67:0/797438450 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5592c632eda0 0x5592c67c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:26.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.808+0000 7feb09d9b640 1 --2- 172.21.15.67:0/797438450 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5592c632eda0 0x5592c67c8740 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bf19264f1bec0fb server_cookie=3490cfb3cc25df40 in_seq=0 out_seq=0 2024-07-31T15:08:26.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.808+0000 7feaf1ffb640 10 client.28676 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feaf1ffb640 1 -- 172.21.15.67:0/797438450 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592c66b45a0 con 0x5592c632eda0 2024-07-31T15:08:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feaf1ffb640 10 client.28676 handle_command_reply: tid=0 2024-07-31T15:08:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feb17434740 1 client.28676 shutdown 2024-07-31T15:08:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feb17434740 1 -- 172.21.15.67:0/797438450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 msgr2=0x5592c673cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feb17434740 1 --2- 172.21.15.67:0/797438450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 0x5592c673cc70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feb0401aef0 tx=0x7feb04000f40 comp rx=0 tx=0).stop 2024-07-31T15:08:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feb17434740 1 -- 172.21.15.67:0/797438450 shutdown_connections 2024-07-31T15:08:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feb17434740 1 --2- 172.21.15.67:0/797438450 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5592c632eda0 0x5592c67c8740 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feb17434740 1 --2- 172.21.15.67:0/797438450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5592c639b070 0x5592c673d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feb17434740 1 --2- 172.21.15.67:0/797438450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c637a390 0x5592c673d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feaf1ffb640 0 client.28676 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feb17434740 1 --2- 172.21.15.67:0/797438450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5592c63721b0 0x5592c673cc70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feb17434740 1 -- 172.21.15.67:0/797438450 >> 172.21.15.67:0/797438450 conn(0x5592c638f460 msgr2=0x5592c67414c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feb17434740 1 -- 172.21.15.67:0/797438450 shutdown_connections 2024-07-31T15:08:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feb17434740 1 -- 172.21.15.67:0/797438450 wait complete. 2024-07-31T15:08:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.847+0000 7feb17434740 20 client.28676 trim_cache size 0 max 16384 2024-07-31T15:08:26.850 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:08:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.848+0000 7feb10ee3640 1 -- 172.21.15.67:0/2451075991 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feaf40b3d40 msgr2=0x7feaf40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.848+0000 7feb10ee3640 1 --2- 172.21.15.67:0/2451075991 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feaf40b3d40 0x7feaf40b6250 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x5592c673f1f0 tx=0x7feafc03b560 comp rx=0 tx=0).stop 2024-07-31T15:08:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.848+0000 7feb10ee3640 1 -- 172.21.15.67:0/2451075991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb0c1024e0 msgr2=0x7feb0c0ff0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.848+0000 7feb10ee3640 1 --2- 172.21.15.67:0/2451075991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb0c1024e0 0x7feb0c0ff0f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feafc02a930 tx=0x7feafc031a20 comp rx=0 tx=0).stop 2024-07-31T15:08:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.848+0000 7feb10ee3640 1 -- 172.21.15.67:0/2451075991 shutdown_connections 2024-07-31T15:08:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.848+0000 7feb10ee3640 1 --2- 172.21.15.67:0/2451075991 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feaf40b3d40 0x7feaf40b6250 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.848+0000 7feb10ee3640 1 --2- 172.21.15.67:0/2451075991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb0c108720 0x7feb0c0ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.848+0000 7feb10ee3640 1 --2- 172.21.15.67:0/2451075991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb0c107dd0 0x7feb0c0ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.848+0000 7feb10ee3640 1 --2- 172.21.15.67:0/2451075991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb0c1024e0 0x7feb0c0ff0f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:26.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.848+0000 7feb10ee3640 1 -- 172.21.15.67:0/2451075991 >> 172.21.15.67:0/2451075991 conn(0x7feb0c0790b0 msgr2=0x7feb0c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:26.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.848+0000 7feb10ee3640 1 -- 172.21.15.67:0/2451075991 shutdown_connections 2024-07-31T15:08:26.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:26.848+0000 7feb10ee3640 1 -- 172.21.15.67:0/2451075991 wait complete. 2024-07-31T15:08:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:27 smithi077 ceph-mon[37327]: pgmap v952: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 145 KiB/s wr, 66 op/s 2024-07-31T15:08:27.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:27 smithi067 ceph-mon[32750]: pgmap v952: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 145 KiB/s wr, 66 op/s 2024-07-31T15:08:27.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:27 smithi073 ceph-mon[36714]: pgmap v952: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 145 KiB/s wr, 66 op/s 2024-07-31T15:08:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.315+0000 7f1e534a9640 1 -- 172.21.15.67:0/1608725686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c103500 msgr2=0x7f1e4c103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.315+0000 7f1e534a9640 1 --2- 172.21.15.67:0/1608725686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c103500 0x7f1e4c103910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009860 tx=0x7f1e4802a340 comp rx=0 tx=0).stop 2024-07-31T15:08:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.316+0000 7f1e534a9640 1 -- 172.21.15.67:0/1608725686 shutdown_connections 2024-07-31T15:08:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.316+0000 7f1e534a9640 1 --2- 172.21.15.67:0/1608725686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e4c109780 0x7f1e4c10ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.316+0000 7f1e534a9640 1 --2- 172.21.15.67:0/1608725686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e4c108e30 0x7f1e4c109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.316+0000 7f1e534a9640 1 --2- 172.21.15.67:0/1608725686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c103500 0x7f1e4c103910 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.316+0000 7f1e534a9640 1 -- 172.21.15.67:0/1608725686 >> 172.21.15.67:0/1608725686 conn(0x7f1e4c0fef00 msgr2=0x7f1e4c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.316+0000 7f1e534a9640 1 -- 172.21.15.67:0/1608725686 shutdown_connections 2024-07-31T15:08:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.316+0000 7f1e534a9640 1 -- 172.21.15.67:0/1608725686 wait complete. 2024-07-31T15:08:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.316+0000 7f1e534a9640 1 Processor -- start 2024-07-31T15:08:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.316+0000 7f1e534a9640 1 -- start start 2024-07-31T15:08:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e534a9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e4c103500 0x7f1e4c199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e534a9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e4c108e30 0x7f1e4c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e534a9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c109780 0x7f1e4c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e534a9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c113450 con 0x7f1e4c108e30 2024-07-31T15:08:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e534a9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1132d0 con 0x7f1e4c109780 2024-07-31T15:08:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e534a9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1135d0 con 0x7f1e4c103500 2024-07-31T15:08:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e50a44640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e4c108e30 0x7f1e4c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e50a44640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e4c108e30 0x7f1e4c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52554/0 (socket says 172.21.15.67:52554) 2024-07-31T15:08:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e50a44640 1 -- 172.21.15.67:0/3779193107 learned_addr learned my addr 172.21.15.67:0/3779193107 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e51a46640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c109780 0x7f1e4c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e51245640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e4c103500 0x7f1e4c199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e50a44640 1 -- 172.21.15.67:0/3779193107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e4c103500 msgr2=0x7f1e4c199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e50a44640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e4c103500 0x7f1e4c199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e50a44640 1 -- 172.21.15.67:0/3779193107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c109780 msgr2=0x7f1e4c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e50a44640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c109780 0x7f1e4c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e50a44640 1 -- 172.21.15.67:0/3779193107 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c19fe00 con 0x7f1e4c108e30 2024-07-31T15:08:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e51245640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e4c103500 0x7f1e4c199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e51a46640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c109780 0x7f1e4c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.317+0000 7f1e50a44640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e4c108e30 0x7f1e4c19a470 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00e9c0 tx=0x7f1e3c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.318+0000 7f1e427fc640 1 -- 172.21.15.67:0/3779193107 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00ccc0 con 0x7f1e4c108e30 2024-07-31T15:08:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.318+0000 7f1e534a9640 1 -- 172.21.15.67:0/3779193107 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4c1a00f0 con 0x7f1e4c108e30 2024-07-31T15:08:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.318+0000 7f1e427fc640 1 -- 172.21.15.67:0/3779193107 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e3c0054d0 con 0x7f1e4c108e30 2024-07-31T15:08:28.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.318+0000 7f1e427fc640 1 -- 172.21.15.67:0/3779193107 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c010430 con 0x7f1e4c108e30 2024-07-31T15:08:28.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.318+0000 7f1e534a9640 1 -- 172.21.15.67:0/3779193107 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4c1a6880 con 0x7f1e4c108e30 2024-07-31T15:08:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.320+0000 7f1e5a3a3740 1 Processor -- start 2024-07-31T15:08:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e427fc640 1 -- 172.21.15.67:0/3779193107 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1e3c005020 con 0x7f1e4c108e30 2024-07-31T15:08:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e5a3a3740 1 -- start start 2024-07-31T15:08:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e5a3a3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff841b0 0x55d770326ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e5a3a3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff5b2f0 0x55d770327220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e5a3a3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff634d0 0x55d770327760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e5a3a3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d77029c3b0 con 0x55d76ff5b2f0 2024-07-31T15:08:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e5a3a3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d77028b8b0 con 0x55d76ff634d0 2024-07-31T15:08:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e5a3a3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d77028f8f0 con 0x55d76ff841b0 2024-07-31T15:08:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e51245640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff5b2f0 0x55d770327220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e50a44640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff634d0 0x55d770327760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e51245640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff5b2f0 0x55d770327220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52570/0 (socket says 172.21.15.67:52570) 2024-07-31T15:08:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e51245640 1 -- 172.21.15.67:0/1790286775 learned_addr learned my addr 172.21.15.67:0/1790286775 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e51a46640 1 --2- 172.21.15.67:0/1790286775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff841b0 0x55d770326ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e51245640 1 -- 172.21.15.67:0/1790286775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff841b0 msgr2=0x55d770326ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e51245640 1 --2- 172.21.15.67:0/1790286775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff841b0 0x55d770326ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e51245640 1 -- 172.21.15.67:0/1790286775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff634d0 msgr2=0x55d770327760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e51245640 1 --2- 172.21.15.67:0/1790286775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff634d0 0x55d770327760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.321+0000 7f1e51245640 1 -- 172.21.15.67:0/1790286775 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7702329a0 con 0x55d76ff5b2f0 2024-07-31T15:08:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.322+0000 7f1e427fc640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1e380b3d40 0x7f1e380b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.322+0000 7f1e51245640 1 --2- 172.21.15.67:0/1790286775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff5b2f0 0x55d770327220 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009860 tx=0x7f1e48030b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd11217eeb1b816e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.322+0000 7f1e51a46640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1e380b3d40 0x7f1e380b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.322+0000 7f1e427fc640 1 -- 172.21.15.67:0/3779193107 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f1e3c0e4750 con 0x7f1e4c108e30 2024-07-31T15:08:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.322+0000 7f1e377fe640 1 -- 172.21.15.67:0/1790286775 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48037040 con 0x55d76ff5b2f0 2024-07-31T15:08:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.322+0000 7f1e377fe640 1 -- 172.21.15.67:0/1790286775 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e48030450 con 0x55d76ff5b2f0 2024-07-31T15:08:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.322+0000 7f1e377fe640 1 -- 172.21.15.67:0/1790286775 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48005580 con 0x55d76ff5b2f0 2024-07-31T15:08:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.323+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/1790286775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff5b2f0 msgr2=0x55d770327220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.323+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/1790286775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff5b2f0 0x55d770327220 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009860 tx=0x7f1e48030b50 comp rx=0 tx=0).stop 2024-07-31T15:08:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.323+0000 7f1e51a46640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1e380b3d40 0x7f1e380b6250 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1e44005fd0 tx=0x7f1e440095c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.323+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/1790286775 shutdown_connections 2024-07-31T15:08:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.323+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/1790286775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff634d0 0x55d770327760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.323+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/1790286775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff5b2f0 0x55d770327220 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.323+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/1790286775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff841b0 0x55d770326ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.323+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/1790286775 >> 172.21.15.67:0/1790286775 conn(0x55d76ff78300 msgr2=0x55d770324bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.323+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/1790286775 shutdown_connections 2024-07-31T15:08:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.323+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/1790286775 wait complete. 2024-07-31T15:08:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.323+0000 7f1e5a3a3740 1 -- start start 2024-07-31T15:08:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.323+0000 7f1e5a3a3740 1 Processor -- start 2024-07-31T15:08:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e5a3a3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff5b2f0 0x55d7703327c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e5a3a3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff634d0 0x55d770332d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e5a3a3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff841b0 0x55d77033e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e5a3a3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d7701c5ab0 con 0x55d76ff634d0 2024-07-31T15:08:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e5a3a3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d770213cf0 con 0x55d76ff841b0 2024-07-31T15:08:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e5a3a3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d770335400 con 0x55d76ff5b2f0 2024-07-31T15:08:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e5a3a3740 10 client.0 _ll_register_callbacks cb 0x55d7702859f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e50a44640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff634d0 0x55d770332d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e51a46640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff841b0 0x55d77033e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e50a44640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff634d0 0x55d770332d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52586/0 (socket says 172.21.15.67:52586) 2024-07-31T15:08:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e50a44640 1 -- 172.21.15.67:0/3484319645 learned_addr learned my addr 172.21.15.67:0/3484319645 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e51245640 1 --2- 172.21.15.67:0/3484319645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff5b2f0 0x55d7703327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e50a44640 1 -- 172.21.15.67:0/3484319645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff5b2f0 msgr2=0x55d7703327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e50a44640 1 --2- 172.21.15.67:0/3484319645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff5b2f0 0x55d7703327c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e50a44640 1 -- 172.21.15.67:0/3484319645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff841b0 msgr2=0x55d77033e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e50a44640 1 --2- 172.21.15.67:0/3484319645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff841b0 0x55d77033e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e50a44640 1 -- 172.21.15.67:0/3484319645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d76ffef790 con 0x55d76ff634d0 2024-07-31T15:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e51a46640 1 --2- 172.21.15.67:0/3484319645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff841b0 0x55d77033e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e50a44640 1 --2- 172.21.15.67:0/3484319645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff634d0 0x55d770332d00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0e1ba0 tx=0x7f1e3c0a6f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a227e032474317f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e34ff9640 1 -- 172.21.15.67:0/3484319645 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00ce60 con 0x55d76ff634d0 2024-07-31T15:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e34ff9640 1 -- 172.21.15.67:0/3484319645 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f1e3c0e4390 con 0x55d76ff634d0 2024-07-31T15:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.324+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/3484319645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e48030450 con 0x55d76ff634d0 2024-07-31T15:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.325+0000 7f1e34ff9640 1 -- 172.21.15.67:0/3484319645 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f1e3c005310 con 0x55d76ff634d0 2024-07-31T15:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.325+0000 7f1e5a3a3740 10 client.19143 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.325+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/3484319645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55d770068620 con 0x55d76ff634d0 2024-07-31T15:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.325+0000 7f1e34ff9640 1 -- 172.21.15.67:0/3484319645 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e48030450 con 0x55d76ff634d0 2024-07-31T15:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.325+0000 7f1e34ff9640 1 -- 172.21.15.67:0/3484319645 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f1e3c0e4070 con 0x55d76ff634d0 2024-07-31T15:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.325+0000 7f1e5a3a3740 10 client.19143 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.325+0000 7f1e5a3a3740 10 client.19143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:08:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.325+0000 7f1e5a3a3740 20 client.19143 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.325+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/3484319645 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d76ff42d80 0x55d7703447a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.325+0000 7f1e5a3a3740 4 client.19143 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:08:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.325+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/3484319645 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d77029e300 con 0x55d76ff42d80 2024-07-31T15:08:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.325+0000 7f1e51245640 1 --2- 172.21.15.67:0/3484319645 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d76ff42d80 0x55d7703447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.326+0000 7f1e34ff9640 10 client.19143 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.326+0000 7f1e51245640 1 --2- 172.21.15.67:0/3484319645 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d76ff42d80 0x55d7703447a0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2af9ecae505c23ad server_cookie=1c68b638cb843645 in_seq=0 out_seq=0 2024-07-31T15:08:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.446+0000 7f1e34ff9640 1 -- 172.21.15.67:0/3484319645 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d77029e300 con 0x55d76ff42d80 2024-07-31T15:08:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.446+0000 7f1e34ff9640 10 client.19143 handle_command_reply: tid=0 2024-07-31T15:08:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.446+0000 7f1e5a3a3740 1 client.19143 shutdown 2024-07-31T15:08:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.446+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/3484319645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff634d0 msgr2=0x55d770332d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.446+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/3484319645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff634d0 0x55d770332d00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0e1ba0 tx=0x7f1e3c0a6f40 comp rx=0 tx=0).stop 2024-07-31T15:08:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.446+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/3484319645 shutdown_connections 2024-07-31T15:08:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.446+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/3484319645 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d76ff42d80 0x55d7703447a0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.446+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/3484319645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff841b0 0x55d77033e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.446+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/3484319645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff634d0 0x55d770332d00 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.446+0000 7f1e34ff9640 0 client.19143 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.446+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/3484319645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff5b2f0 0x55d7703327c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.446+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/3484319645 >> 172.21.15.67:0/3484319645 conn(0x55d76ff78300 msgr2=0x55d7703232b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.446+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/3484319645 shutdown_connections 2024-07-31T15:08:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.446+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/3484319645 wait complete. 2024-07-31T15:08:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.447+0000 7f1e5a3a3740 20 client.19143 trim_cache size 0 max 16384 2024-07-31T15:08:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.462+0000 7f1e5a3a3740 1 Processor -- start 2024-07-31T15:08:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.462+0000 7f1e5a3a3740 1 -- start start 2024-07-31T15:08:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.462+0000 7f1e5a3a3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff5b2f0 0x55d770331e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.462+0000 7f1e5a3a3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff634d0 0x55d770328420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.462+0000 7f1e5a3a3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff841b0 0x55d77033e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e5a3a3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1ae460 con 0x55d76ff841b0 2024-07-31T15:08:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e5a3a3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1affa0 con 0x55d76ff5b2f0 2024-07-31T15:08:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e5a3a3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1aeb30 con 0x55d76ff634d0 2024-07-31T15:08:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e51a46640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff841b0 0x55d77033e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e51a46640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff841b0 0x55d77033e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52600/0 (socket says 172.21.15.67:52600) 2024-07-31T15:08:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e51a46640 1 -- 172.21.15.67:0/2444496173 learned_addr learned my addr 172.21.15.67:0/2444496173 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e51245640 1 --2- 172.21.15.67:0/2444496173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff5b2f0 0x55d770331e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e50a44640 1 --2- 172.21.15.67:0/2444496173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff634d0 0x55d770328420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e51a46640 1 -- 172.21.15.67:0/2444496173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff634d0 msgr2=0x55d770328420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e51a46640 1 --2- 172.21.15.67:0/2444496173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff634d0 0x55d770328420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e51a46640 1 -- 172.21.15.67:0/2444496173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff5b2f0 msgr2=0x55d770331e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e51a46640 1 --2- 172.21.15.67:0/2444496173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff5b2f0 0x55d770331e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e51a46640 1 -- 172.21.15.67:0/2444496173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d770344b10 con 0x55d76ff841b0 2024-07-31T15:08:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e50a44640 1 --2- 172.21.15.67:0/2444496173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff634d0 0x55d770328420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e51a46640 1 --2- 172.21.15.67:0/2444496173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff841b0 0x55d77033e540 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f1e44003f60 tx=0x7f1e44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a00dcfe15579066 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.463+0000 7f1e36ffd640 1 -- 172.21.15.67:0/2444496173 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44020450 con 0x55d76ff841b0 2024-07-31T15:08:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.464+0000 7f1e36ffd640 1 -- 172.21.15.67:0/2444496173 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e44030070 con 0x55d76ff841b0 2024-07-31T15:08:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.464+0000 7f1e36ffd640 1 -- 172.21.15.67:0/2444496173 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4401b7a0 con 0x55d76ff841b0 2024-07-31T15:08:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.464+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/2444496173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff841b0 msgr2=0x55d77033e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.464+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/2444496173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff841b0 0x55d77033e540 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f1e44003f60 tx=0x7f1e44017000 comp rx=0 tx=0).stop 2024-07-31T15:08:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.464+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/2444496173 shutdown_connections 2024-07-31T15:08:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.464+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/2444496173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff841b0 0x55d77033e540 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.464+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/2444496173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff634d0 0x55d770328420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.464+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/2444496173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff5b2f0 0x55d770331e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.465+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/2444496173 >> 172.21.15.67:0/2444496173 conn(0x55d76ff78300 msgr2=0x55d770327d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.465+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/2444496173 shutdown_connections 2024-07-31T15:08:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.465+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/2444496173 wait complete. 2024-07-31T15:08:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.465+0000 7f1e5a3a3740 1 -- start start 2024-07-31T15:08:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.465+0000 7f1e5a3a3740 1 Processor -- start 2024-07-31T15:08:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.465+0000 7f1e5a3a3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff5b2f0 0x55d77033ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.465+0000 7f1e5a3a3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff634d0 0x55d770337680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.465+0000 7f1e5a3a3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff841b0 0x55d770337bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.465+0000 7f1e5a3a3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1af8d0 con 0x55d76ff5b2f0 2024-07-31T15:08:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.465+0000 7f1e5a3a3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1af200 con 0x55d76ff634d0 2024-07-31T15:08:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.465+0000 7f1e5a3a3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1ac460 con 0x55d76ff841b0 2024-07-31T15:08:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.465+0000 7f1e5a3a3740 10 client.0 _ll_register_callbacks cb 0x7f1e3c0ea240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e51245640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff5b2f0 0x55d77033ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e51245640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff5b2f0 0x55d77033ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52616/0 (socket says 172.21.15.67:52616) 2024-07-31T15:08:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e50a44640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff634d0 0x55d770337680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e51245640 1 -- 172.21.15.67:0/1773211989 learned_addr learned my addr 172.21.15.67:0/1773211989 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e51a46640 1 --2- 172.21.15.67:0/1773211989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff841b0 0x55d770337bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e51245640 1 -- 172.21.15.67:0/1773211989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff841b0 msgr2=0x55d770337bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e51245640 1 --2- 172.21.15.67:0/1773211989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff841b0 0x55d770337bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e51245640 1 -- 172.21.15.67:0/1773211989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff634d0 msgr2=0x55d770337680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e51245640 1 --2- 172.21.15.67:0/1773211989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff634d0 0x55d770337680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e51245640 1 -- 172.21.15.67:0/1773211989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d770338330 con 0x55d76ff5b2f0 2024-07-31T15:08:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e51a46640 1 --2- 172.21.15.67:0/1773211989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff841b0 0x55d770337bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e50a44640 1 --2- 172.21.15.67:0/1773211989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff634d0 0x55d770337680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e51245640 1 --2- 172.21.15.67:0/1773211989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff5b2f0 0x55d77033ada0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x55d770325960 tx=0x7f1e48005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732ba4251f9e2013 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:08:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e34ff9640 1 -- 172.21.15.67:0/1773211989 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48037040 con 0x55d76ff5b2f0 2024-07-31T15:08:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/1773211989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e44030070 con 0x55d76ff5b2f0 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e34ff9640 1 -- 172.21.15.67:0/1773211989 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f1e48004230 con 0x55d76ff5b2f0 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e34ff9640 1 -- 172.21.15.67:0/1773211989 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f1e48030910 con 0x55d76ff5b2f0 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.466+0000 7f1e5a3a3740 10 client.19149 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.467+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/1773211989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55d7703abe50 con 0x55d76ff5b2f0 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.467+0000 7f1e34ff9640 1 -- 172.21.15.67:0/1773211989 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e44030070 con 0x55d76ff5b2f0 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.467+0000 7f1e34ff9640 1 -- 172.21.15.67:0/1773211989 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f1e480037b0 con 0x55d76ff5b2f0 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.467+0000 7f1e5a3a3740 10 client.19149 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.467+0000 7f1e5a3a3740 10 client.19149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.467+0000 7f1e5a3a3740 20 client.19149 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.467+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/1773211989 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d76ff181c0 0x55d7703ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.467+0000 7f1e5a3a3740 4 client.19149 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.467+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/1773211989 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d770299da0 con 0x55d76ff181c0 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.467+0000 7f1e50a44640 1 --2- 172.21.15.67:0/1773211989 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d76ff181c0 0x55d7703ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.467+0000 7f1e34ff9640 10 client.19149 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:28.467+0000 7f1e50a44640 1 --2- 172.21.15.67:0/1773211989 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d76ff181c0 0x55d7703ae130 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0da8e3c75f9dc2e server_cookie=d7c2e57399c3dbb1 in_seq=0 out_seq=0 2024-07-31T15:08:28.765 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T15:08:28.766 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T15:08:29.159 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:08:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:29 smithi077 ceph-mon[37327]: pgmap v953: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 214 KiB/s wr, 51 op/s 2024-07-31T15:08:29.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:29 smithi067 ceph-mon[32750]: pgmap v953: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 214 KiB/s wr, 51 op/s 2024-07-31T15:08:29.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:29 smithi073 ceph-mon[36714]: pgmap v953: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 214 KiB/s wr, 51 op/s 2024-07-31T15:08:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.970+0000 7f187ef66640 1 -- 172.21.15.67:0/3922718745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1878108ed0 msgr2=0x7f18781092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.970+0000 7f187ef66640 1 --2- 172.21.15.67:0/3922718745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1878108ed0 0x7f18781092e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f186c009900 tx=0x7f186c02a410 comp rx=0 tx=0).stop 2024-07-31T15:08:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.970+0000 7f187ef66640 1 -- 172.21.15.67:0/3922718745 shutdown_connections 2024-07-31T15:08:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.970+0000 7f187ef66640 1 --2- 172.21.15.67:0/3922718745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1878109820 0x7f1878110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.970+0000 7f187ef66640 1 --2- 172.21.15.67:0/3922718745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1878108ed0 0x7f18781092e0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.970+0000 7f187ef66640 1 --2- 172.21.15.67:0/3922718745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1878103590 0x7f18781039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.970+0000 7f187ef66640 1 -- 172.21.15.67:0/3922718745 >> 172.21.15.67:0/3922718745 conn(0x7f187807b280 msgr2=0x7f187807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.970+0000 7f187ef66640 1 -- 172.21.15.67:0/3922718745 shutdown_connections 2024-07-31T15:08:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.970+0000 7f187ef66640 1 -- 172.21.15.67:0/3922718745 wait complete. 2024-07-31T15:08:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187ef66640 1 Processor -- start 2024-07-31T15:08:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187ef66640 1 -- start start 2024-07-31T15:08:30.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187ef66640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1878103590 0x7f1878199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:30.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187ef66640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1878108ed0 0x7f187819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:30.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187ef66640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1878109820 0x7f187819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187ef66640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f18781134b0 con 0x7f1878109820 2024-07-31T15:08:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187ef66640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1878113330 con 0x7f1878103590 2024-07-31T15:08:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187ef66640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1878113630 con 0x7f1878108ed0 2024-07-31T15:08:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187cd02640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1878103590 0x7f1878199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187d503640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1878109820 0x7f187819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f1877fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1878108ed0 0x7f187819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187cd02640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1878103590 0x7f1878199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56628/0 (socket says 172.21.15.67:56628) 2024-07-31T15:08:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187d503640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1878109820 0x7f187819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46194/0 (socket says 172.21.15.67:46194) 2024-07-31T15:08:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187d503640 1 -- 172.21.15.67:0/1899584244 learned_addr learned my addr 172.21.15.67:0/1899584244 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187cd02640 1 -- 172.21.15.67:0/1899584244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1878108ed0 msgr2=0x7f187819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.971+0000 7f187cd02640 1 --2- 172.21.15.67:0/1899584244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1878108ed0 0x7f187819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.972+0000 7f187cd02640 1 -- 172.21.15.67:0/1899584244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1878109820 msgr2=0x7f187819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.972+0000 7f187cd02640 1 --2- 172.21.15.67:0/1899584244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1878109820 0x7f187819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.972+0000 7f187cd02640 1 -- 172.21.15.67:0/1899584244 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187819ff10 con 0x7f1878103590 2024-07-31T15:08:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.972+0000 7f187cd02640 1 --2- 172.21.15.67:0/1899584244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1878103590 0x7f1878199fa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f186800d720 tx=0x7f186800dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.972+0000 7f1875ffb640 1 -- 172.21.15.67:0/1899584244 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868017070 con 0x7f1878103590 2024-07-31T15:08:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.972+0000 7f187ef66640 1 -- 172.21.15.67:0/1899584244 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18781a0200 con 0x7f1878103590 2024-07-31T15:08:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.972+0000 7f1875ffb640 1 -- 172.21.15.67:0/1899584244 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f186800de30 con 0x7f1878103590 2024-07-31T15:08:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.972+0000 7f1875ffb640 1 -- 172.21.15.67:0/1899584244 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680047a0 con 0x7f1878103590 2024-07-31T15:08:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.972+0000 7f187ef66640 1 -- 172.21.15.67:0/1899584244 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18781a6960 con 0x7f1878103590 2024-07-31T15:08:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f1875ffb640 1 -- 172.21.15.67:0/1899584244 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f18680099d0 con 0x7f1878103590 2024-07-31T15:08:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f18854b7740 1 Processor -- start 2024-07-31T15:08:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f18854b7740 1 -- start start 2024-07-31T15:08:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f18854b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc449e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f18854b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc449e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f18854b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 0x556fc449ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f18854b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556fc43862f0 con 0x556fc40d0490 2024-07-31T15:08:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f18854b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556fc4414a80 con 0x556fc40f9350 2024-07-31T15:08:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f18854b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556fc4403f80 con 0x556fc40d8670 2024-07-31T15:08:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f187cd02640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc449e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f187d503640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc449e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f1877fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 0x556fc449ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f187d503640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc449e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56632/0 (socket says 172.21.15.67:56632) 2024-07-31T15:08:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f1877fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 0x556fc449ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:42690/0 (socket says 172.21.15.67:42690) 2024-07-31T15:08:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.975+0000 7f187d503640 1 -- 172.21.15.67:0/1732030559 learned_addr learned my addr 172.21.15.67:0/1732030559 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.976+0000 7f1875ffb640 1 --2- 172.21.15.67:0/1899584244 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18600b3d40 0x7f18600b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.976+0000 7f187d503640 1 -- 172.21.15.67:0/1732030559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 msgr2=0x556fc449ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.976+0000 7f187d503640 1 --2- 172.21.15.67:0/1732030559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 0x556fc449ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.976+0000 7f187d503640 1 -- 172.21.15.67:0/1732030559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 msgr2=0x556fc449e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.976+0000 7f187d503640 1 --2- 172.21.15.67:0/1732030559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc449e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.976+0000 7f187d503640 1 -- 172.21.15.67:0/1732030559 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fc4164930 con 0x556fc40f9350 2024-07-31T15:08:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.976+0000 7f187d503640 1 --2- 172.21.15.67:0/1899584244 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18600b3d40 0x7f18600b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.976+0000 7f1875ffb640 1 -- 172.21.15.67:0/1899584244 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f18680e3f10 con 0x7f1878103590 2024-07-31T15:08:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.976+0000 7f187d503640 1 --2- 172.21.15.67:0/1732030559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc449e3b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1870011470 tx=0x7f1870011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8e04eb3391f6537 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.976+0000 7f185effd640 1 -- 172.21.15.67:0/1732030559 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187000fa60 con 0x556fc40f9350 2024-07-31T15:08:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.976+0000 7f185effd640 1 -- 172.21.15.67:0/1732030559 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18700183b0 con 0x556fc40f9350 2024-07-31T15:08:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.976+0000 7f185effd640 1 -- 172.21.15.67:0/1732030559 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187000e5b0 con 0x556fc40f9350 2024-07-31T15:08:30.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.977+0000 7f18854b7740 1 -- 172.21.15.67:0/1732030559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 msgr2=0x556fc449e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:30.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.977+0000 7f18854b7740 1 --2- 172.21.15.67:0/1732030559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc449e3b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1870011470 tx=0x7f1870011930 comp rx=0 tx=0).stop 2024-07-31T15:08:30.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.977+0000 7f187d503640 1 --2- 172.21.15.67:0/1899584244 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18600b3d40 0x7f18600b6250 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1870020000 tx=0x7f1870021000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:30.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.977+0000 7f18854b7740 1 -- 172.21.15.67:0/1732030559 shutdown_connections 2024-07-31T15:08:30.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.977+0000 7f18854b7740 1 --2- 172.21.15.67:0/1732030559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 0x556fc449ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.977+0000 7f18854b7740 1 --2- 172.21.15.67:0/1732030559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc449e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.977+0000 7f18854b7740 1 --2- 172.21.15.67:0/1732030559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc449e3b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.977+0000 7f18854b7740 1 -- 172.21.15.67:0/1732030559 >> 172.21.15.67:0/1732030559 conn(0x556fc40ed740 msgr2=0x556fc449c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:30.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.977+0000 7f18854b7740 1 -- 172.21.15.67:0/1732030559 shutdown_connections 2024-07-31T15:08:30.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.977+0000 7f18854b7740 1 -- 172.21.15.67:0/1732030559 wait complete. 2024-07-31T15:08:30.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.978+0000 7f18854b7740 1 -- start start 2024-07-31T15:08:30.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.978+0000 7f18854b7740 1 Processor -- start 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.978+0000 7f18854b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d0490 0x556fc44a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.978+0000 7f18854b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d8670 0x556fc44aa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.978+0000 7f18854b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc44b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.978+0000 7f18854b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556fc43a64f0 con 0x556fc40d8670 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.978+0000 7f18854b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556fc4407fc0 con 0x556fc40f9350 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.978+0000 7f18854b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556fc4267890 con 0x556fc40d0490 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.978+0000 7f18854b7740 10 client.0 _ll_register_callbacks cb 0x556fc43b0500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.978+0000 7f1877fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d8670 0x556fc44aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.978+0000 7f1877fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d8670 0x556fc44aa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46212/0 (socket says 172.21.15.67:46212) 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.978+0000 7f1877fff640 1 -- 172.21.15.67:0/13666589 learned_addr learned my addr 172.21.15.67:0/13666589 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.978+0000 7f187cd02640 1 --2- 172.21.15.67:0/13666589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d0490 0x556fc44a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.978+0000 7f187d503640 1 --2- 172.21.15.67:0/13666589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc44b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.979+0000 7f1877fff640 1 -- 172.21.15.67:0/13666589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d0490 msgr2=0x556fc44a9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.979+0000 7f1877fff640 1 --2- 172.21.15.67:0/13666589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d0490 0x556fc44a9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.979+0000 7f1877fff640 1 -- 172.21.15.67:0/13666589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 msgr2=0x556fc44b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.979+0000 7f1877fff640 1 --2- 172.21.15.67:0/13666589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc44b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.979+0000 7f1877fff640 1 -- 172.21.15.67:0/13666589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fc43b5c50 con 0x556fc40d8670 2024-07-31T15:08:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.979+0000 7f187cd02640 1 --2- 172.21.15.67:0/13666589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d0490 0x556fc44a9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.979+0000 7f1877fff640 1 --2- 172.21.15.67:0/13666589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d8670 0x556fc44aa120 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f186c009900 tx=0x7f186c032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc832c3e1dcd129 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.979+0000 7f1857fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:08:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.979+0000 7f1857fff640 1 -- 172.21.15.67:0/13666589 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c035040 con 0x556fc40d8670 2024-07-31T15:08:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.979+0000 7f1857fff640 1 -- 172.21.15.67:0/13666589 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f186c0317a0 con 0x556fc40d8670 2024-07-31T15:08:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.979+0000 7f18854b7740 1 -- 172.21.15.67:0/13666589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18700183b0 con 0x556fc40d8670 2024-07-31T15:08:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.979+0000 7f1857fff640 1 -- 172.21.15.67:0/13666589 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f186c04f050 con 0x556fc40d8670 2024-07-31T15:08:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.979+0000 7f18854b7740 10 client.19161 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.979+0000 7f18854b7740 1 -- 172.21.15.67:0/13666589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x556fc44b88a0 con 0x556fc40d8670 2024-07-31T15:08:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.980+0000 7f1857fff640 1 -- 172.21.15.67:0/13666589 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18700183b0 con 0x556fc40d8670 2024-07-31T15:08:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.980+0000 7f1857fff640 1 -- 172.21.15.67:0/13666589 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f186c004750 con 0x556fc40d8670 2024-07-31T15:08:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.980+0000 7f18854b7740 10 client.19161 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.980+0000 7f18854b7740 10 client.19161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:08:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.980+0000 7f18854b7740 20 client.19161 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.980+0000 7f18854b7740 1 --2- 172.21.15.67:0/13666589 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556fc40b7cd0 0x556fc44bbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.980+0000 7f18854b7740 4 client.19161 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:08:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.980+0000 7f18854b7740 1 -- 172.21.15.67:0/13666589 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fc44169d0 con 0x556fc40b7cd0 2024-07-31T15:08:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.980+0000 7f187cd02640 1 --2- 172.21.15.67:0/13666589 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556fc40b7cd0 0x556fc44bbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.980+0000 7f1857fff640 10 client.19161 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.980+0000 7f187cd02640 1 --2- 172.21.15.67:0/13666589 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556fc40b7cd0 0x556fc44bbdd0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9de5c0623610b4a server_cookie=78578c5f962535ca in_seq=0 out_seq=0 2024-07-31T15:08:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f1857fff640 1 -- 172.21.15.67:0/13666589 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fc44169d0 con 0x556fc40b7cd0 2024-07-31T15:08:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f1857fff640 10 client.19161 handle_command_reply: tid=0 2024-07-31T15:08:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f18854b7740 1 client.19161 shutdown 2024-07-31T15:08:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f18854b7740 1 -- 172.21.15.67:0/13666589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d8670 msgr2=0x556fc44aa120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f18854b7740 1 --2- 172.21.15.67:0/13666589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d8670 0x556fc44aa120 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f186c009900 tx=0x7f186c032f60 comp rx=0 tx=0).stop 2024-07-31T15:08:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f18854b7740 1 -- 172.21.15.67:0/13666589 shutdown_connections 2024-07-31T15:08:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f18854b7740 1 --2- 172.21.15.67:0/13666589 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556fc40b7cd0 0x556fc44bbdd0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f18854b7740 1 --2- 172.21.15.67:0/13666589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc44b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f18854b7740 1 --2- 172.21.15.67:0/13666589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d8670 0x556fc44aa120 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f18854b7740 1 --2- 172.21.15.67:0/13666589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d0490 0x556fc44a9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f18854b7740 1 -- 172.21.15.67:0/13666589 >> 172.21.15.67:0/13666589 conn(0x556fc40ed740 msgr2=0x556fc44a1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f1857fff640 0 client.19161 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f18854b7740 1 -- 172.21.15.67:0/13666589 shutdown_connections 2024-07-31T15:08:30.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f18854b7740 1 -- 172.21.15.67:0/13666589 wait complete. 2024-07-31T15:08:30.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:30.990+0000 7f18854b7740 20 client.19161 trim_cache size 0 max 16384 2024-07-31T15:08:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.009+0000 7f18854b7740 1 Processor -- start 2024-07-31T15:08:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.009+0000 7f18854b7740 1 -- start start 2024-07-31T15:08:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.009+0000 7f18854b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc449faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.009+0000 7f18854b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 0x556fc44b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.009+0000 7f18854b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc44b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.009+0000 7f18854b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f18781ae6a0 con 0x556fc40d0490 2024-07-31T15:08:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.009+0000 7f18854b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f18781b01e0 con 0x556fc40f9350 2024-07-31T15:08:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.009+0000 7f18854b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f18781aed70 con 0x556fc40d8670 2024-07-31T15:08:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.009+0000 7f187cd02640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc449faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.010+0000 7f187cd02640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc449faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46214/0 (socket says 172.21.15.67:46214) 2024-07-31T15:08:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.010+0000 7f187cd02640 1 -- 172.21.15.67:0/3926886185 learned_addr learned my addr 172.21.15.67:0/3926886185 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.010+0000 7f187d503640 1 --2- 172.21.15.67:0/3926886185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc44b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.010+0000 7f1877fff640 1 --2- 172.21.15.67:0/3926886185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 0x556fc44b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.010+0000 7f187cd02640 1 -- 172.21.15.67:0/3926886185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 msgr2=0x556fc44b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.010+0000 7f187cd02640 1 --2- 172.21.15.67:0/3926886185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 0x556fc44b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.010+0000 7f187cd02640 1 -- 172.21.15.67:0/3926886185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 msgr2=0x556fc44b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.010+0000 7f187cd02640 1 --2- 172.21.15.67:0/3926886185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc44b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.010+0000 7f187cd02640 1 -- 172.21.15.67:0/3926886185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fc449c7e0 con 0x556fc40d0490 2024-07-31T15:08:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.010+0000 7f187cd02640 1 --2- 172.21.15.67:0/3926886185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc449faf0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f18680e1900 tx=0x7f18680e6490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88eba2a2288fe69 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.010+0000 7f185e7fc640 1 -- 172.21.15.67:0/3926886185 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680bfe60 con 0x556fc40d0490 2024-07-31T15:08:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.010+0000 7f185e7fc640 1 -- 172.21.15.67:0/3926886185 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1868005020 con 0x556fc40d0490 2024-07-31T15:08:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.010+0000 7f185e7fc640 1 -- 172.21.15.67:0/3926886185 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680aedc0 con 0x556fc40d0490 2024-07-31T15:08:31.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.011+0000 7f18854b7740 1 -- 172.21.15.67:0/3926886185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 msgr2=0x556fc449faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:31.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.011+0000 7f18854b7740 1 --2- 172.21.15.67:0/3926886185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc449faf0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f18680e1900 tx=0x7f18680e6490 comp rx=0 tx=0).stop 2024-07-31T15:08:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.011+0000 7f18854b7740 1 -- 172.21.15.67:0/3926886185 shutdown_connections 2024-07-31T15:08:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.011+0000 7f18854b7740 1 --2- 172.21.15.67:0/3926886185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc44b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.011+0000 7f18854b7740 1 --2- 172.21.15.67:0/3926886185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 0x556fc44b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.011+0000 7f18854b7740 1 --2- 172.21.15.67:0/3926886185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc449faf0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.011+0000 7f18854b7740 1 -- 172.21.15.67:0/3926886185 >> 172.21.15.67:0/3926886185 conn(0x556fc40ed740 msgr2=0x556fc449f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.011+0000 7f18854b7740 1 -- 172.21.15.67:0/3926886185 shutdown_connections 2024-07-31T15:08:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.011+0000 7f18854b7740 1 -- 172.21.15.67:0/3926886185 wait complete. 2024-07-31T15:08:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.012+0000 7f18854b7740 1 -- start start 2024-07-31T15:08:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.012+0000 7f18854b7740 1 Processor -- start 2024-07-31T15:08:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.012+0000 7f18854b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc44b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.012+0000 7f18854b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 0x556fc44ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.012+0000 7f18854b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc44ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.012+0000 7f18854b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f18781afb10 con 0x556fc40d0490 2024-07-31T15:08:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.012+0000 7f18854b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f18781af440 con 0x556fc40f9350 2024-07-31T15:08:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.012+0000 7f18854b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f18781ac6a0 con 0x556fc40d8670 2024-07-31T15:08:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.012+0000 7f18854b7740 10 client.0 _ll_register_callbacks cb 0x7f186c032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.012+0000 7f187cd02640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc44b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.012+0000 7f187cd02640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc44b2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46224/0 (socket says 172.21.15.67:46224) 2024-07-31T15:08:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.012+0000 7f187cd02640 1 -- 172.21.15.67:0/38071820 learned_addr learned my addr 172.21.15.67:0/38071820 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.012+0000 7f1877fff640 1 --2- 172.21.15.67:0/38071820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 0x556fc44ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.012+0000 7f187d503640 1 --2- 172.21.15.67:0/38071820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc44ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f187cd02640 1 -- 172.21.15.67:0/38071820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 msgr2=0x556fc44ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f187cd02640 1 --2- 172.21.15.67:0/38071820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 0x556fc44ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f187cd02640 1 -- 172.21.15.67:0/38071820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 msgr2=0x556fc44ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f187cd02640 1 --2- 172.21.15.67:0/38071820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc44ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f187cd02640 1 -- 172.21.15.67:0/38071820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fc4523560 con 0x556fc40d0490 2024-07-31T15:08:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f187cd02640 1 --2- 172.21.15.67:0/38071820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc44b2530 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f1868005fd0 tx=0x7f18680e12a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a94bf36ed15e09 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f1857fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:08:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f1857fff640 1 -- 172.21.15.67:0/38071820 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680bfe60 con 0x556fc40d0490 2024-07-31T15:08:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f1857fff640 1 -- 172.21.15.67:0/38071820 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f18680e1580 con 0x556fc40d0490 2024-07-31T15:08:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f18854b7740 1 -- 172.21.15.67:0/38071820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556fc4412470 con 0x556fc40d0490 2024-07-31T15:08:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f1857fff640 1 -- 172.21.15.67:0/38071820 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f18680b31b0 con 0x556fc40d0490 2024-07-31T15:08:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f18854b7740 10 client.19167 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f18854b7740 1 -- 172.21.15.67:0/38071820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x556fc4524670 con 0x556fc40d0490 2024-07-31T15:08:31.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f1857fff640 1 -- 172.21.15.67:0/38071820 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556fc4412470 con 0x556fc40d0490 2024-07-31T15:08:31.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.013+0000 7f1857fff640 1 -- 172.21.15.67:0/38071820 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f18680a6a50 con 0x556fc40d0490 2024-07-31T15:08:31.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.014+0000 7f18854b7740 10 client.19167 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:31.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.014+0000 7f18854b7740 10 client.19167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:08:31.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.014+0000 7f18854b7740 20 client.19167 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:31.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.014+0000 7f18854b7740 1 --2- 172.21.15.67:0/38071820 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556fc408d080 0x556fc4526e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:31.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.014+0000 7f18854b7740 4 client.19167 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T15:08:31.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.014+0000 7f18854b7740 1 -- 172.21.15.67:0/38071820 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556fc440fbb0 con 0x556fc408d080 2024-07-31T15:08:31.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.014+0000 7f1877fff640 1 --2- 172.21.15.67:0/38071820 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556fc408d080 0x556fc4526e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:31.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.014+0000 7f1877fff640 1 --2- 172.21.15.67:0/38071820 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556fc408d080 0x556fc4526e60 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86bd3fdaca639a69 server_cookie=c9843001e4ca45c6 in_seq=0 out_seq=0 2024-07-31T15:08:31.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.014+0000 7f1857fff640 10 client.19167 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:31.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.080+0000 7f1857fff640 1 -- 172.21.15.67:0/38071820 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556fc440fbb0 con 0x556fc408d080 2024-07-31T15:08:31.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.080+0000 7f1857fff640 10 client.19167 handle_command_reply: tid=0 2024-07-31T15:08:31.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.080+0000 7f18854b7740 1 client.19167 shutdown 2024-07-31T15:08:31.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.080+0000 7f18854b7740 1 -- 172.21.15.67:0/38071820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 msgr2=0x556fc44b2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:31.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.080+0000 7f18854b7740 1 --2- 172.21.15.67:0/38071820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc44b2530 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f1868005fd0 tx=0x7f18680e12a0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.080+0000 7f18854b7740 1 -- 172.21.15.67:0/38071820 shutdown_connections 2024-07-31T15:08:31.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.080+0000 7f18854b7740 1 --2- 172.21.15.67:0/38071820 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x556fc408d080 0x556fc4526e60 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.080+0000 7f18854b7740 1 --2- 172.21.15.67:0/38071820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc40f9350 0x556fc44ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.080+0000 7f1857fff640 0 client.19167 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:31.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.080+0000 7f18854b7740 1 --2- 172.21.15.67:0/38071820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556fc40d8670 0x556fc44ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.080+0000 7f18854b7740 1 --2- 172.21.15.67:0/38071820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556fc40d0490 0x556fc44b2530 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.080+0000 7f18854b7740 1 -- 172.21.15.67:0/38071820 >> 172.21.15.67:0/38071820 conn(0x556fc40ed740 msgr2=0x556fc449f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:31.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.081+0000 7f18854b7740 1 -- 172.21.15.67:0/38071820 shutdown_connections 2024-07-31T15:08:31.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.081+0000 7f18854b7740 1 -- 172.21.15.67:0/38071820 wait complete. 2024-07-31T15:08:31.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.081+0000 7f18854b7740 20 client.19167 trim_cache size 0 max 16384 2024-07-31T15:08:31.084 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:08:31.084 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T15:08:31.084 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "b25ef0d8-8f9f-44f2-8ecd-c31be64da7e9", 2024-07-31T15:08:31.084 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T15:08:31.084 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:08:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.081+0000 7f187ef66640 1 -- 172.21.15.67:0/1899584244 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18600b3d40 msgr2=0x7f18600b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.081+0000 7f187ef66640 1 --2- 172.21.15.67:0/1899584244 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18600b3d40 0x7f18600b6250 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1870020000 tx=0x7f1870021000 comp rx=0 tx=0).stop 2024-07-31T15:08:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.081+0000 7f187ef66640 1 -- 172.21.15.67:0/1899584244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1878103590 msgr2=0x7f1878199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.081+0000 7f187ef66640 1 --2- 172.21.15.67:0/1899584244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1878103590 0x7f1878199fa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f186800d720 tx=0x7f186800dbe0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.081+0000 7f187ef66640 1 -- 172.21.15.67:0/1899584244 shutdown_connections 2024-07-31T15:08:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.081+0000 7f187ef66640 1 --2- 172.21.15.67:0/1899584244 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18600b3d40 0x7f18600b6250 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.081+0000 7f187ef66640 1 --2- 172.21.15.67:0/1899584244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1878109820 0x7f187819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.081+0000 7f187ef66640 1 --2- 172.21.15.67:0/1899584244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1878108ed0 0x7f187819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.081+0000 7f187ef66640 1 --2- 172.21.15.67:0/1899584244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1878103590 0x7f1878199fa0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.081+0000 7f187ef66640 1 -- 172.21.15.67:0/1899584244 >> 172.21.15.67:0/1899584244 conn(0x7f187807b280 msgr2=0x7f1878107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.082+0000 7f187ef66640 1 -- 172.21.15.67:0/1899584244 shutdown_connections 2024-07-31T15:08:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:31.082+0000 7f187ef66640 1 -- 172.21.15.67:0/1899584244 wait complete. 2024-07-31T15:08:31.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:31 smithi067 ceph-mon[32750]: pgmap v954: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 209 KiB/s wr, 63 op/s 2024-07-31T15:08:31.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:31 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T15:08:31.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:31.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:31 smithi077 ceph-mon[37327]: pgmap v954: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 209 KiB/s wr, 63 op/s 2024-07-31T15:08:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:31 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T15:08:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:31.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.619+0000 7f1e34ff9640 1 -- 172.21.15.67:0/1773211989 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d770299da0 con 0x55d76ff181c0 2024-07-31T15:08:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.619+0000 7f1e34ff9640 10 client.19149 handle_command_reply: tid=0 2024-07-31T15:08:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.620+0000 7f1e5a3a3740 1 client.19149 shutdown 2024-07-31T15:08:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.620+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/1773211989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff5b2f0 msgr2=0x55d77033ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.620+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/1773211989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff5b2f0 0x55d77033ada0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x55d770325960 tx=0x7f1e48005e10 comp rx=0 tx=0).stop 2024-07-31T15:08:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.620+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/1773211989 shutdown_connections 2024-07-31T15:08:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.620+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/1773211989 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d76ff181c0 0x55d7703ae130 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.620+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/1773211989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d76ff841b0 0x55d770337bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.620+0000 7f1e34ff9640 0 client.19149 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.620+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/1773211989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d76ff634d0 0x55d770337680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.620+0000 7f1e5a3a3740 1 --2- 172.21.15.67:0/1773211989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d76ff5b2f0 0x55d77033ada0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.620+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/1773211989 >> 172.21.15.67:0/1773211989 conn(0x55d76ff78300 msgr2=0x55d7703286d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.620+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/1773211989 shutdown_connections 2024-07-31T15:08:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.620+0000 7f1e5a3a3740 1 -- 172.21.15.67:0/1773211989 wait complete. 2024-07-31T15:08:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.620+0000 7f1e5a3a3740 20 client.19149 trim_cache size 0 max 16384 2024-07-31T15:08:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:08:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:08:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:08:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:08:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.621+0000 7f1e534a9640 1 -- 172.21.15.67:0/3779193107 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1e380b3d40 msgr2=0x7f1e380b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.621+0000 7f1e534a9640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1e380b3d40 0x7f1e380b6250 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1e44005fd0 tx=0x7f1e440095c0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.621+0000 7f1e534a9640 1 -- 172.21.15.67:0/3779193107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e4c108e30 msgr2=0x7f1e4c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.621+0000 7f1e534a9640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e4c108e30 0x7f1e4c19a470 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00e9c0 tx=0x7f1e3c00ee80 comp rx=0 tx=0).stop 2024-07-31T15:08:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.621+0000 7f1e534a9640 1 -- 172.21.15.67:0/3779193107 shutdown_connections 2024-07-31T15:08:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.621+0000 7f1e534a9640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f1e380b3d40 0x7f1e380b6250 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.621+0000 7f1e534a9640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c109780 0x7f1e4c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.621+0000 7f1e534a9640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e4c108e30 0x7f1e4c19a470 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.621+0000 7f1e534a9640 1 --2- 172.21.15.67:0/3779193107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e4c103500 0x7f1e4c199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.621+0000 7f1e534a9640 1 -- 172.21.15.67:0/3779193107 >> 172.21.15.67:0/3779193107 conn(0x7f1e4c0fef00 msgr2=0x7f1e4c101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.621+0000 7f1e534a9640 1 -- 172.21.15.67:0/3779193107 shutdown_connections 2024-07-31T15:08:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:31.621+0000 7f1e534a9640 1 -- 172.21.15.67:0/3779193107 wait complete. 2024-07-31T15:08:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:31 smithi073 ceph-mon[36714]: pgmap v954: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 209 KiB/s wr, 63 op/s 2024-07-31T15:08:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:31 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T15:08:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:31.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:08:32.154 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:08:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:32.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:32.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:32.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:32.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:32.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:32.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:32.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.642 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:08:32.302+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d9b4 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb/ auth{3=1} v3876 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7) n(v5 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8) old_inodes=3 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:379332 nref=3)) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce92cb2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v2020>,<0x10000000202/sound v9936>,<0x10000000201/linux-6.5.11 v830>,<0x10000000006/t v586>,<0x10000000005/tmp v553>,<0x100000001fb/client.0 v537>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v519>,<0x10000000001/sv_1 v525>,<0x10000000000/qa v522>,<0x1/volumes v503>]//[]","memoryvalue":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v3876>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2328>,<0x10000000006/t v1395>,<0x10000000005/tmp v1333>,<0x100000001fb/client.0 v1308>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1277>,<0x10000000001/sv_1 v1278>,<0x10000000000/qa v1278>,<0x1/volumes v1231>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:46.317139+0000 53=46+7)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7)","memory_value.rstat":"n(v5 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:08:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:32.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:32.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:32.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:32.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: pgmap v955: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 199 KiB/s wr, 98 op/s 2024-07-31T15:08:33.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:33.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:33.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:33 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:08:33.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:33.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:33.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: pgmap v955: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 199 KiB/s wr, 98 op/s 2024-07-31T15:08:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:33.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:33 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:08:33.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:33.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:33.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:33.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:33.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:33.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:33.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: pgmap v955: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 199 KiB/s wr, 98 op/s 2024-07-31T15:08:33.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:33.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:33.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:33.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:08:33.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:08:33 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:08:33.599+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000162a [...1b,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch/ auth{1=1,2=1,3=1,4=1} v20647 ap=3 RANDEPHEMERALPIN f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22) n(v51 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939) old_inodes=6 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:379234 nref=4)) caps={15489=pAsLsXsFs/-@48} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7c26ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000162a:[<0x10000000202/arch v378>,<0x10000000201/linux-6.5.11 v96>,<0x10000000006/t v68>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v57>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v60>,<0x10000000001/sv_1 v84>,<0x10000000000/qa v70>,<0x1/volumes v70>]//[]","memoryvalue":"(2)0x1000000162a:[<0x10000000202/arch v20647>,<0x10000000201/linux-6.5.11 v2330>,<0x10000000006/t v1397>,<0x10000000005/tmp v1335>,<0x100000001fb/client.0 v1310>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1279>,<0x10000000001/sv_1 v1280>,<0x10000000000/qa v1280>,<0x1/volumes v1233>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:54:49.652308+0000 23=2+21)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939)","memory_value.dirstat":"f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22)","memory_value.rstat":"n(v51 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:08:34.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.318+0000 7f11c8829640 1 -- 172.21.15.67:0/1670365124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108720 msgr2=0x7f11c410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:34.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.318+0000 7f11c8829640 1 --2- 172.21.15.67:0/1670365124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108720 0x7f11c410ef50 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009ca0 tx=0x7f11bc02a8c0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.318+0000 7f11c8829640 1 -- 172.21.15.67:0/1670365124 shutdown_connections 2024-07-31T15:08:34.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.318+0000 7f11c8829640 1 --2- 172.21.15.67:0/1670365124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108720 0x7f11c410ef50 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.318+0000 7f11c8829640 1 --2- 172.21.15.67:0/1670365124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11c4107dd0 0x7f11c41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.318+0000 7f11c8829640 1 --2- 172.21.15.67:0/1670365124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c4102490 0x7f11c41028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.318+0000 7f11c8829640 1 -- 172.21.15.67:0/1670365124 >> 172.21.15.67:0/1670365124 conn(0x7f11c4079160 msgr2=0x7f11c4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:34.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.318+0000 7f11c8829640 1 -- 172.21.15.67:0/1670365124 shutdown_connections 2024-07-31T15:08:34.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.318+0000 7f11c8829640 1 -- 172.21.15.67:0/1670365124 wait complete. 2024-07-31T15:08:34.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.319+0000 7f11c8829640 1 Processor -- start 2024-07-31T15:08:34.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.319+0000 7f11c8829640 1 -- start start 2024-07-31T15:08:34.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.319+0000 7f11c8829640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11c4102490 0x7f11c4199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.319+0000 7f11c8829640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c4107dd0 0x7f11c419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.319+0000 7f11c8829640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108720 0x7f11c419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.319+0000 7f11c8829640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41123a0 con 0x7f11c4108720 2024-07-31T15:08:34.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.319+0000 7f11c8829640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4112220 con 0x7f11c4102490 2024-07-31T15:08:34.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.319+0000 7f11c8829640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4112520 con 0x7f11c4107dd0 2024-07-31T15:08:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.319+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c4107dd0 0x7f11c419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.319+0000 7f11c2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108720 0x7f11c419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.319+0000 7f11c259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11c4102490 0x7f11c4199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.319+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c4107dd0 0x7f11c419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:42758/0 (socket says 172.21.15.67:42758) 2024-07-31T15:08:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.319+0000 7f11bbfff640 1 -- 172.21.15.67:0/3084969066 learned_addr learned my addr 172.21.15.67:0/3084969066 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.320+0000 7f11c259c640 1 -- 172.21.15.67:0/3084969066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c4107dd0 msgr2=0x7f11c419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.320+0000 7f11c259c640 1 --2- 172.21.15.67:0/3084969066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c4107dd0 0x7f11c419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.320+0000 7f11c259c640 1 -- 172.21.15.67:0/3084969066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108720 msgr2=0x7f11c419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.320+0000 7f11c259c640 1 --2- 172.21.15.67:0/3084969066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108720 0x7f11c419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.320+0000 7f11c259c640 1 -- 172.21.15.67:0/3084969066 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c419fe50 con 0x7f11c4102490 2024-07-31T15:08:34.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.320+0000 7f11bbfff640 1 --2- 172.21.15.67:0/3084969066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c4107dd0 0x7f11c419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:34.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.320+0000 7f11c259c640 1 --2- 172.21.15.67:0/3084969066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11c4102490 0x7f11c4199ee0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f11b400e9f0 tx=0x7f11b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:34.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.320+0000 7f11bb7fe640 1 -- 172.21.15.67:0/3084969066 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b400ccc0 con 0x7f11c4102490 2024-07-31T15:08:34.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.320+0000 7f11c8829640 1 -- 172.21.15.67:0/3084969066 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c41a0140 con 0x7f11c4102490 2024-07-31T15:08:34.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.320+0000 7f11bb7fe640 1 -- 172.21.15.67:0/3084969066 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11b4005020 con 0x7f11c4102490 2024-07-31T15:08:34.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.320+0000 7f11bb7fe640 1 -- 172.21.15.67:0/3084969066 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4004a00 con 0x7f11c4102490 2024-07-31T15:08:34.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.320+0000 7f11c8829640 1 -- 172.21.15.67:0/3084969066 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c41a68a0 con 0x7f11c4102490 2024-07-31T15:08:34.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.323+0000 7f11ced7a740 1 Processor -- start 2024-07-31T15:08:34.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.323+0000 7f11bb7fe640 1 -- 172.21.15.67:0/3084969066 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f11b4004ba0 con 0x7f11c4102490 2024-07-31T15:08:34.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.323+0000 7f11ced7a740 1 -- start start 2024-07-31T15:08:34.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11ced7a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b53a070 0x555c1b8df550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11ced7a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b5111b0 0x555c1b8dfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11ced7a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b519390 0x555c1b8dffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11ced7a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555c1b855b90 con 0x555c1b5111b0 2024-07-31T15:08:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11ced7a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555c1b845090 con 0x555c1b519390 2024-07-31T15:08:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11ced7a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555c1b8490d0 con 0x555c1b53a070 2024-07-31T15:08:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b519390 0x555c1b8dffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11c259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b5111b0 0x555c1b8dfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b519390 0x555c1b8dffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56690/0 (socket says 172.21.15.67:56690) 2024-07-31T15:08:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11bbfff640 1 -- 172.21.15.67:0/3534098879 learned_addr learned my addr 172.21.15.67:0/3534098879 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11c2d9d640 1 --2- 172.21.15.67:0/3534098879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b53a070 0x555c1b8df550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11bbfff640 1 -- 172.21.15.67:0/3534098879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b53a070 msgr2=0x555c1b8df550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11bbfff640 1 --2- 172.21.15.67:0/3534098879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b53a070 0x555c1b8df550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11bbfff640 1 -- 172.21.15.67:0/3534098879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b5111b0 msgr2=0x555c1b8dfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11bbfff640 1 --2- 172.21.15.67:0/3534098879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b5111b0 0x555c1b8dfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11bbfff640 1 -- 172.21.15.67:0/3534098879 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1b7ebb70 con 0x555c1b519390 2024-07-31T15:08:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11bb7fe640 1 --2- 172.21.15.67:0/3084969066 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11b00b3d40 0x7f11b00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.324+0000 7f11bbfff640 1 --2- 172.21.15.67:0/3534098879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b519390 0x555c1b8dffd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f11ac011510 tx=0x7f11ac0119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4470f65016b36fed server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.325+0000 7f11c2d9d640 1 --2- 172.21.15.67:0/3084969066 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11b00b3d40 0x7f11b00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.325+0000 7f11b8ff9640 1 -- 172.21.15.67:0/3534098879 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac01e070 con 0x555c1b519390 2024-07-31T15:08:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.325+0000 7f11b8ff9640 1 -- 172.21.15.67:0/3534098879 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11ac00fa70 con 0x555c1b519390 2024-07-31T15:08:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.325+0000 7f11b8ff9640 1 -- 172.21.15.67:0/3534098879 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac019bf0 con 0x555c1b519390 2024-07-31T15:08:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.325+0000 7f11bb7fe640 1 -- 172.21.15.67:0/3084969066 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f11b40e4300 con 0x7f11c4102490 2024-07-31T15:08:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.326+0000 7f11ced7a740 1 -- 172.21.15.67:0/3534098879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b519390 msgr2=0x555c1b8dffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.326+0000 7f11c2d9d640 1 --2- 172.21.15.67:0/3084969066 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11b00b3d40 0x7f11b00b6250 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f11bc02f500 tx=0x7f11bc02f310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.326+0000 7f11ced7a740 1 --2- 172.21.15.67:0/3534098879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b519390 0x555c1b8dffd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f11ac011510 tx=0x7f11ac0119d0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.326+0000 7f11ced7a740 1 -- 172.21.15.67:0/3534098879 shutdown_connections 2024-07-31T15:08:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.326+0000 7f11ced7a740 1 --2- 172.21.15.67:0/3534098879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b519390 0x555c1b8dffd0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.326+0000 7f11ced7a740 1 --2- 172.21.15.67:0/3534098879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b5111b0 0x555c1b8dfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.326+0000 7f11ced7a740 1 --2- 172.21.15.67:0/3534098879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b53a070 0x555c1b8df550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.326+0000 7f11ced7a740 1 -- 172.21.15.67:0/3534098879 >> 172.21.15.67:0/3534098879 conn(0x555c1b52e460 msgr2=0x555c1b8dd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.326+0000 7f11ced7a740 1 -- 172.21.15.67:0/3534098879 shutdown_connections 2024-07-31T15:08:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.326+0000 7f11ced7a740 1 -- 172.21.15.67:0/3534098879 wait complete. 2024-07-31T15:08:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.326+0000 7f11ced7a740 1 -- start start 2024-07-31T15:08:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.326+0000 7f11ced7a740 1 Processor -- start 2024-07-31T15:08:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11ced7a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b5111b0 0x555c1b8eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11ced7a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b519390 0x555c1b8eb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11ced7a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b53a070 0x555c1b8f7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11ced7a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555c1b6a8700 con 0x555c1b5111b0 2024-07-31T15:08:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11ced7a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555c1b7e79c0 con 0x555c1b519390 2024-07-31T15:08:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11ced7a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555c1b8edd30 con 0x555c1b53a070 2024-07-31T15:08:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11c259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b5111b0 0x555c1b8eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11c259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b5111b0 0x555c1b8eb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46272/0 (socket says 172.21.15.67:46272) 2024-07-31T15:08:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11ced7a740 10 client.0 _ll_register_callbacks cb 0x555c1b8024d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11c259c640 1 -- 172.21.15.67:0/1818196005 learned_addr learned my addr 172.21.15.67:0/1818196005 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11bbfff640 1 --2- 172.21.15.67:0/1818196005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b519390 0x555c1b8eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11c2d9d640 1 --2- 172.21.15.67:0/1818196005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b53a070 0x555c1b8f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11c259c640 1 -- 172.21.15.67:0/1818196005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b53a070 msgr2=0x555c1b8f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11c259c640 1 --2- 172.21.15.67:0/1818196005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b53a070 0x555c1b8f7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11c259c640 1 -- 172.21.15.67:0/1818196005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b519390 msgr2=0x555c1b8eb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11c259c640 1 --2- 172.21.15.67:0/1818196005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b519390 0x555c1b8eb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11c259c640 1 -- 172.21.15.67:0/1818196005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1b5a5650 con 0x555c1b5111b0 2024-07-31T15:08:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11c2d9d640 1 --2- 172.21.15.67:0/1818196005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b53a070 0x555c1b8f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11c259c640 1 --2- 172.21.15.67:0/1818196005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b5111b0 0x555c1b8eb0f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f11b40aa8b0 tx=0x7f11b4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466d2b5a38bee4da server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.327+0000 7f11a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:08:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.328+0000 7f11a9ffb640 1 -- 172.21.15.67:0/1818196005 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b401c070 con 0x555c1b5111b0 2024-07-31T15:08:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.328+0000 7f11a9ffb640 1 -- 172.21.15.67:0/1818196005 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f11b40a6e40 con 0x555c1b5111b0 2024-07-31T15:08:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.328+0000 7f11ced7a740 1 -- 172.21.15.67:0/1818196005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11ac00fa70 con 0x555c1b5111b0 2024-07-31T15:08:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.328+0000 7f11a9ffb640 1 -- 172.21.15.67:0/1818196005 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f11b40aa500 con 0x555c1b5111b0 2024-07-31T15:08:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.328+0000 7f11ced7a740 10 client.19179 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.328+0000 7f11ced7a740 1 -- 172.21.15.67:0/1818196005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x555c1b8f9df0 con 0x555c1b5111b0 2024-07-31T15:08:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.328+0000 7f11a9ffb640 1 -- 172.21.15.67:0/1818196005 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11ac00fa70 con 0x555c1b5111b0 2024-07-31T15:08:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.328+0000 7f11a9ffb640 1 -- 172.21.15.67:0/1818196005 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f11b40c2020 con 0x555c1b5111b0 2024-07-31T15:08:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.328+0000 7f11ced7a740 10 client.19179 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.328+0000 7f11ced7a740 10 client.19179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:08:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.328+0000 7f11ced7a740 20 client.19179 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.328+0000 7f11ced7a740 1 --2- 172.21.15.67:0/1818196005 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c1b4f89f0 0x555c1b8fd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.328+0000 7f11ced7a740 4 client.19179 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:08:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.328+0000 7f11ced7a740 1 -- 172.21.15.67:0/1818196005 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c1b857ae0 con 0x555c1b4f89f0 2024-07-31T15:08:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.329+0000 7f11bbfff640 1 --2- 172.21.15.67:0/1818196005 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c1b4f89f0 0x555c1b8fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.329+0000 7f11bbfff640 1 --2- 172.21.15.67:0/1818196005 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c1b4f89f0 0x555c1b8fd250 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76eaa43571dc9a30 server_cookie=3b75d2fe7b083c in_seq=0 out_seq=0 2024-07-31T15:08:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.329+0000 7f11a9ffb640 10 client.19179 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:34.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.357+0000 7f11a9ffb640 1 -- 172.21.15.67:0/1818196005 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c1b857ae0 con 0x555c1b4f89f0 2024-07-31T15:08:34.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.357+0000 7f11a9ffb640 10 client.19179 handle_command_reply: tid=0 2024-07-31T15:08:34.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.357+0000 7f11ced7a740 1 client.19179 shutdown 2024-07-31T15:08:34.359 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:34.359 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:34.359 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:34 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:08:34.359 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:34.359 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:34.360 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:34.360 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:34.360 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:34.360 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:34.360 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:34.360 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:34.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.357+0000 7f11ced7a740 1 -- 172.21.15.67:0/1818196005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b5111b0 msgr2=0x555c1b8eb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:34.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.357+0000 7f11ced7a740 1 --2- 172.21.15.67:0/1818196005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b5111b0 0x555c1b8eb0f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f11b40aa8b0 tx=0x7f11b4009f60 comp rx=0 tx=0).stop 2024-07-31T15:08:34.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.357+0000 7f11ced7a740 1 -- 172.21.15.67:0/1818196005 shutdown_connections 2024-07-31T15:08:34.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.357+0000 7f11ced7a740 1 --2- 172.21.15.67:0/1818196005 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c1b4f89f0 0x555c1b8fd250 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.357+0000 7f11ced7a740 1 --2- 172.21.15.67:0/1818196005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b53a070 0x555c1b8f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.357+0000 7f11ced7a740 1 --2- 172.21.15.67:0/1818196005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b519390 0x555c1b8eb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.357+0000 7f11ced7a740 1 --2- 172.21.15.67:0/1818196005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b5111b0 0x555c1b8eb0f0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.357+0000 7f11a9ffb640 0 client.19179 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:34.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.357+0000 7f11ced7a740 1 -- 172.21.15.67:0/1818196005 >> 172.21.15.67:0/1818196005 conn(0x555c1b52e460 msgr2=0x555c1b8dbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:34.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.358+0000 7f11ced7a740 1 -- 172.21.15.67:0/1818196005 shutdown_connections 2024-07-31T15:08:34.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.358+0000 7f11ced7a740 1 -- 172.21.15.67:0/1818196005 wait complete. 2024-07-31T15:08:34.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.358+0000 7f11ced7a740 20 client.19179 trim_cache size 0 max 16384 2024-07-31T15:08:34.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.379+0000 7f11ced7a740 1 Processor -- start 2024-07-31T15:08:34.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.379+0000 7f11ced7a740 1 -- start start 2024-07-31T15:08:34.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.379+0000 7f11ced7a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b4f89f0 0x555c1b8dc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.379+0000 7f11ced7a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b5111b0 0x555c1b8e6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.379+0000 7f11ced7a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b519390 0x555c1b8f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.379+0000 7f11ced7a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41b0740 con 0x555c1b519390 2024-07-31T15:08:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.379+0000 7f11ced7a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41b2280 con 0x555c1b5111b0 2024-07-31T15:08:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.379+0000 7f11ced7a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41b0e10 con 0x555c1b4f89f0 2024-07-31T15:08:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.379+0000 7f11c2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b519390 0x555c1b8f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.379+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b5111b0 0x555c1b8e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.379+0000 7f11c259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b4f89f0 0x555c1b8dc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.379+0000 7f11c2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b519390 0x555c1b8f6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46278/0 (socket says 172.21.15.67:46278) 2024-07-31T15:08:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.379+0000 7f11c2d9d640 1 -- 172.21.15.67:0/3041141634 learned_addr learned my addr 172.21.15.67:0/3041141634 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.380+0000 7f11c2d9d640 1 -- 172.21.15.67:0/3041141634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b4f89f0 msgr2=0x555c1b8dc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:34.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.380+0000 7f11c2d9d640 1 --2- 172.21.15.67:0/3041141634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b4f89f0 0x555c1b8dc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.380+0000 7f11c2d9d640 1 -- 172.21.15.67:0/3041141634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b5111b0 msgr2=0x555c1b8e6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:34.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.380+0000 7f11c2d9d640 1 --2- 172.21.15.67:0/3041141634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b5111b0 0x555c1b8e6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.380+0000 7f11c2d9d640 1 -- 172.21.15.67:0/3041141634 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1b8dd510 con 0x555c1b519390 2024-07-31T15:08:34.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.380+0000 7f11bbfff640 1 --2- 172.21.15.67:0/3041141634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b5111b0 0x555c1b8e6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:34.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.380+0000 7f11c2d9d640 1 --2- 172.21.15.67:0/3041141634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b519390 0x555c1b8f6e10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f11bc035000 tx=0x7f11bc041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9dd9a0a1ff5e0e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:34.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.380+0000 7f11abfff640 1 -- 172.21.15.67:0/3041141634 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc003970 con 0x555c1b519390 2024-07-31T15:08:34.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.380+0000 7f11abfff640 1 -- 172.21.15.67:0/3041141634 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11bc036050 con 0x555c1b519390 2024-07-31T15:08:34.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.380+0000 7f11abfff640 1 -- 172.21.15.67:0/3041141634 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc02f660 con 0x555c1b519390 2024-07-31T15:08:34.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.381+0000 7f11ced7a740 1 -- 172.21.15.67:0/3041141634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b519390 msgr2=0x555c1b8f6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:34.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.381+0000 7f11ced7a740 1 --2- 172.21.15.67:0/3041141634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b519390 0x555c1b8f6e10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f11bc035000 tx=0x7f11bc041000 comp rx=0 tx=0).stop 2024-07-31T15:08:34.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.381+0000 7f11ced7a740 1 -- 172.21.15.67:0/3041141634 shutdown_connections 2024-07-31T15:08:34.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.381+0000 7f11ced7a740 1 --2- 172.21.15.67:0/3041141634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b519390 0x555c1b8f6e10 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.381+0000 7f11ced7a740 1 --2- 172.21.15.67:0/3041141634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b5111b0 0x555c1b8e6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.381+0000 7f11ced7a740 1 --2- 172.21.15.67:0/3041141634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b4f89f0 0x555c1b8dc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.381+0000 7f11ced7a740 1 -- 172.21.15.67:0/3041141634 >> 172.21.15.67:0/3041141634 conn(0x555c1b52e460 msgr2=0x555c1b8dbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:34.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.381+0000 7f11ced7a740 1 -- 172.21.15.67:0/3041141634 shutdown_connections 2024-07-31T15:08:34.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.381+0000 7f11ced7a740 1 -- 172.21.15.67:0/3041141634 wait complete. 2024-07-31T15:08:34.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.382+0000 7f11ced7a740 1 -- start start 2024-07-31T15:08:34.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.382+0000 7f11ced7a740 1 Processor -- start 2024-07-31T15:08:34.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.382+0000 7f11ced7a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b4f89f0 0x555c1b8ef190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.382+0000 7f11ced7a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b5111b0 0x555c1b8ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.382+0000 7f11ced7a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b519390 0x555c1b9649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.382+0000 7f11ced7a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41b1bb0 con 0x555c1b4f89f0 2024-07-31T15:08:34.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.382+0000 7f11ced7a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41b14e0 con 0x555c1b5111b0 2024-07-31T15:08:34.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.382+0000 7f11ced7a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41ae740 con 0x555c1b519390 2024-07-31T15:08:34.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.382+0000 7f11ced7a740 10 client.0 _ll_register_callbacks cb 0x7f11b40e7b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:34.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11c2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b519390 0x555c1b9649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11c259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b4f89f0 0x555c1b8ef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11c2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b519390 0x555c1b9649e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:42794/0 (socket says 172.21.15.67:42794) 2024-07-31T15:08:34.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11c2d9d640 1 -- 172.21.15.67:0/3480695844 learned_addr learned my addr 172.21.15.67:0/3480695844 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:34.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11bbfff640 1 --2- 172.21.15.67:0/3480695844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b5111b0 0x555c1b8ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11c2d9d640 1 -- 172.21.15.67:0/3480695844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b5111b0 msgr2=0x555c1b8ef6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:34.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11c2d9d640 1 --2- 172.21.15.67:0/3480695844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b5111b0 0x555c1b8ef6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11c2d9d640 1 -- 172.21.15.67:0/3480695844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b4f89f0 msgr2=0x555c1b8ef190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:34.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11c2d9d640 1 --2- 172.21.15.67:0/3480695844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b4f89f0 0x555c1b8ef190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11c2d9d640 1 -- 172.21.15.67:0/3480695844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1b8effa0 con 0x555c1b519390 2024-07-31T15:08:34.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11c259c640 1 --2- 172.21.15.67:0/3480695844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b4f89f0 0x555c1b8ef190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:34.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11bbfff640 1 --2- 172.21.15.67:0/3480695844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b5111b0 0x555c1b8ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:34.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11c2d9d640 1 --2- 172.21.15.67:0/3480695844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b519390 0x555c1b9649e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f11bc031820 tx=0x7f11bc02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbf6823c99e9fcb1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:34.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:08:34.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11a9ffb640 1 -- 172.21.15.67:0/3480695844 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc02fb50 con 0x555c1b519390 2024-07-31T15:08:34.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11a9ffb640 1 -- 172.21.15.67:0/3480695844 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f11bc02fcf0 con 0x555c1b519390 2024-07-31T15:08:34.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.383+0000 7f11ced7a740 1 -- 172.21.15.67:0/3480695844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11bc036050 con 0x555c1b519390 2024-07-31T15:08:34.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.384+0000 7f11a9ffb640 1 -- 172.21.15.67:0/3480695844 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f11bc0305c0 con 0x555c1b519390 2024-07-31T15:08:34.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.384+0000 7f11ced7a740 10 client.28612 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:34.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.384+0000 7f11ced7a740 1 -- 172.21.15.67:0/3480695844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x555c1b965fb0 con 0x555c1b519390 2024-07-31T15:08:34.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.384+0000 7f11a9ffb640 1 -- 172.21.15.67:0/3480695844 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11bc036050 con 0x555c1b519390 2024-07-31T15:08:34.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.384+0000 7f11a9ffb640 1 -- 172.21.15.67:0/3480695844 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f11bc05d570 con 0x555c1b519390 2024-07-31T15:08:34.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.384+0000 7f11ced7a740 10 client.28612 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:34.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.384+0000 7f11ced7a740 10 client.28612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:08:34.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.384+0000 7f11ced7a740 20 client.28612 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:34.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.384+0000 7f11ced7a740 1 --2- 172.21.15.67:0/3480695844 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c1b53a070 0x555c1b9685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:34.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.384+0000 7f11ced7a740 4 client.28612 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:08:34.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.384+0000 7f11ced7a740 1 -- 172.21.15.67:0/3480695844 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555c1b853580 con 0x555c1b53a070 2024-07-31T15:08:34.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.384+0000 7f11c259c640 1 --2- 172.21.15.67:0/3480695844 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c1b53a070 0x555c1b9685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:34.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.385+0000 7f11a9ffb640 10 client.28612 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:34.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.385+0000 7f11c259c640 1 --2- 172.21.15.67:0/3480695844 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c1b53a070 0x555c1b9685e0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfc9ded37d6419ad server_cookie=bf92918660d8b613 in_seq=0 out_seq=0 2024-07-31T15:08:34.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.410+0000 7f11a9ffb640 1 -- 172.21.15.67:0/3480695844 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x555c1b853580 con 0x555c1b53a070 2024-07-31T15:08:34.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.410+0000 7f11a9ffb640 10 client.28612 handle_command_reply: tid=0 2024-07-31T15:08:34.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.410+0000 7f11ced7a740 1 client.28612 shutdown 2024-07-31T15:08:34.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.411+0000 7f11ced7a740 1 -- 172.21.15.67:0/3480695844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b519390 msgr2=0x555c1b9649e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:34.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.411+0000 7f11ced7a740 1 --2- 172.21.15.67:0/3480695844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b519390 0x555c1b9649e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f11bc031820 tx=0x7f11bc02a8c0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.411+0000 7f11ced7a740 1 -- 172.21.15.67:0/3480695844 shutdown_connections 2024-07-31T15:08:34.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.411+0000 7f11ced7a740 1 --2- 172.21.15.67:0/3480695844 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c1b53a070 0x555c1b9685e0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.411+0000 7f11ced7a740 1 --2- 172.21.15.67:0/3480695844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c1b519390 0x555c1b9649e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.411+0000 7f11ced7a740 1 --2- 172.21.15.67:0/3480695844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c1b5111b0 0x555c1b8ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.411+0000 7f11ced7a740 1 --2- 172.21.15.67:0/3480695844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c1b4f89f0 0x555c1b8ef190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.411+0000 7f11ced7a740 1 -- 172.21.15.67:0/3480695844 >> 172.21.15.67:0/3480695844 conn(0x555c1b52e460 msgr2=0x555c1b8e0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:34.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.411+0000 7f11a9ffb640 0 client.28612 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:34.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.411+0000 7f11ced7a740 1 -- 172.21.15.67:0/3480695844 shutdown_connections 2024-07-31T15:08:34.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.411+0000 7f11ced7a740 1 -- 172.21.15.67:0/3480695844 wait complete. 2024-07-31T15:08:34.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.411+0000 7f11ced7a740 20 client.28612 trim_cache size 0 max 16384 2024-07-31T15:08:34.414 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:08:34.415 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (4845 inodes in the stack)", 2024-07-31T15:08:34.415 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T15:08:34.415 INFO:teuthology.orchestra.run.smithi067.stdout: "b25ef0d8-8f9f-44f2-8ecd-c31be64da7e9": { 2024-07-31T15:08:34.415 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T15:08:34.415 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "b25ef0d8-8f9f-44f2-8ecd-c31be64da7e9", 2024-07-31T15:08:34.415 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T15:08:34.415 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:08:34.415 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:08:34.415 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:08:34.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.412+0000 7f11c8829640 1 -- 172.21.15.67:0/3084969066 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11b00b3d40 msgr2=0x7f11b00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:34.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.412+0000 7f11c8829640 1 --2- 172.21.15.67:0/3084969066 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11b00b3d40 0x7f11b00b6250 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f11bc02f500 tx=0x7f11bc02f310 comp rx=0 tx=0).stop 2024-07-31T15:08:34.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.412+0000 7f11c8829640 1 -- 172.21.15.67:0/3084969066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11c4102490 msgr2=0x7f11c4199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:34.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.412+0000 7f11c8829640 1 --2- 172.21.15.67:0/3084969066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11c4102490 0x7f11c4199ee0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f11b400e9f0 tx=0x7f11b400eeb0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.412+0000 7f11c8829640 1 -- 172.21.15.67:0/3084969066 shutdown_connections 2024-07-31T15:08:34.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.412+0000 7f11c8829640 1 --2- 172.21.15.67:0/3084969066 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f11b00b3d40 0x7f11b00b6250 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.412+0000 7f11c8829640 1 --2- 172.21.15.67:0/3084969066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108720 0x7f11c419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.412+0000 7f11c8829640 1 --2- 172.21.15.67:0/3084969066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c4107dd0 0x7f11c419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.412+0000 7f11c8829640 1 --2- 172.21.15.67:0/3084969066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11c4102490 0x7f11c4199ee0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:34.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.412+0000 7f11c8829640 1 -- 172.21.15.67:0/3084969066 >> 172.21.15.67:0/3084969066 conn(0x7f11c4079160 msgr2=0x7f11c406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:34.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.412+0000 7f11c8829640 1 -- 172.21.15.67:0/3084969066 shutdown_connections 2024-07-31T15:08:34.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:34.412+0000 7f11c8829640 1 -- 172.21.15.67:0/3084969066 wait complete. 2024-07-31T15:08:34.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:34 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:08:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:34.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:34 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:08:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:34.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.241 INFO:tasks.cephfs.filesystem:scrub status for tag:b25ef0d8-8f9f-44f2-8ecd-c31be64da7e9 - {'path': '/', 'tag': 'b25ef0d8-8f9f-44f2-8ecd-c31be64da7e9', 'options': 'recursive,force'} 2024-07-31T15:08:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: pgmap v956: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 270 KiB/s wr, 162 op/s 2024-07-31T15:08:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:35.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:35.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:35.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:35.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:35.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:35.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:35.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:35.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:35.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:35.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:35.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:35.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:35.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: pgmap v956: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 270 KiB/s wr, 162 op/s 2024-07-31T15:08:35.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:35.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:35.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:35.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:35.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.568 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:35.568 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.568 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:35.568 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.568 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:35.568 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.568 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: pgmap v956: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 270 KiB/s wr, 162 op/s 2024-07-31T15:08:35.568 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:35.568 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.569 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:35.569 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:35.569 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:35.569 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.569 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.569 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.569 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:35.569 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.569 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:35.569 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.569 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:35.569 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.569 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:35.570 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.570 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:35.570 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.570 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:35.570 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.570 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:35.570 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:35.892 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:08:35 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:08:35.766+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c061 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math/ auth{1=1} v1164 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:10.213499+0000 31=31+0) n(v2 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1) old_inodes=2 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:381387 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@9} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8e92fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c061:[<0x1000000bf5f/math v634>,<0x10000000202/lib v5688>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000c061:[<0x1000000bf5f/math v1164>,<0x10000000202/lib v21040>,<0x10000000201/linux-6.5.11 v2332>,<0x10000000006/t v1399>,<0x10000000005/tmp v1337>,<0x100000001fb/client.0 v1312>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1281>,<0x10000000001/sv_1 v1282>,<0x10000000000/qa v1282>,<0x1/volumes v1237>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:10.988109+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:10.213499+0000 31=31+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:08:36.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:36.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:36.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:36.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:36.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:36.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:36.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:36.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:36.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:36.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:36.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:36.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:36.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:36.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:36.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:08:36.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:36 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:08:36.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:36.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:36.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:36.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:36.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:36.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:36.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:36.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:36.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:36.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:36.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:36.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:36.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:36.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:36.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:08:36.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:36 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:08:36.642 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:08:36.176+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000ce48 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity/ auth{1=1,2=1} v1695 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3) n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4) old_inodes=3 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:381849 nref=3)) caps={15489=pAsLsXs/-@10} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8aaac580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000ce48:[<0x1000000ce07/integrity v345>,<0x10000000202/security v8766>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000ce48:[<0x1000000ce07/integrity v1695>,<0x10000000202/security v15110>,<0x10000000201/linux-6.5.11 v2332>,<0x10000000006/t v1399>,<0x10000000005/tmp v1337>,<0x100000001fb/client.0 v1312>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1281>,<0x10000000001/sv_1 v1282>,<0x10000000000/qa v1282>,<0x1/volumes v1237>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:59.470106+0000 10=7+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","memory_value.dirstat":"f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3)","memory_value.rstat":"n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:08:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:08:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:08:37.237 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:08:36 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:08:36.897+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009980 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video/ auth{1=1,4=1} v18105 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:16.340374+0000 28=24+4) n(v11 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28) old_inodes=2 (inest sync->mix g=1,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:386638 nref=3)) caps={15489=pAsLsXs/-@6} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce87325b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009980:[<0x10000001d8f/video v17889>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009980:[<0x10000001d8f/video v18105>,<0x10000000202/drivers v21042>,<0x10000000201/linux-6.5.11 v2334>,<0x10000000006/t v1401>,<0x10000000005/tmp v1339>,<0x100000001fb/client.0 v1314>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1283>,<0x10000000001/sv_1 v1284>,<0x10000000000/qa v1284>,<0x1/volumes v1239>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:35.535846+0000 16=12+4)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:16.340374+0000 28=24+4)","memory_value.rstat":"n(v11 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:08:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: pgmap v957: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 265 KiB/s wr, 179 op/s 2024-07-31T15:08:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:37.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:37 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:08:37.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: pgmap v957: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 265 KiB/s wr, 179 op/s 2024-07-31T15:08:37.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:37.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:08:37.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: pgmap v957: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 265 KiB/s wr, 179 op/s 2024-07-31T15:08:37.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:37.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:37.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:37 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:08:38.142 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:08:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:08:37.880+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000be32 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module/ auth{2=1} v2029 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0) n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:400341 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@12} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce826b7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000be32:[<0x1000000bd0a/module v1134>,<0x10000000202/kernel v5648>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000be32:[<0x1000000bd0a/module v2029>,<0x10000000202/kernel v21206>,<0x10000000201/linux-6.5.11 v2334>,<0x10000000006/t v1401>,<0x10000000005/tmp v1339>,<0x100000001fb/client.0 v1314>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1283>,<0x10000000001/sv_1 v1284>,<0x10000000000/qa v1284>,<0x1/volumes v1239>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:07.168188+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:08:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:38.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:38 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:08:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:38 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:08:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:38.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:38.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:38 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:08:39.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: pgmap v958: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 304 KiB/s wr, 321 op/s 2024-07-31T15:08:39.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:39.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.582 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.582 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: pgmap v958: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 304 KiB/s wr, 321 op/s 2024-07-31T15:08:39.582 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.582 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.582 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.582 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.582 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.582 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.582 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.582 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.582 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.582 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.582 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.583 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:39.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.584 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: pgmap v958: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 304 KiB/s wr, 321 op/s 2024-07-31T15:08:39.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:39.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:39.892 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:08:39.579+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d187 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi/ auth{1=1} v3040 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0) n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1) old_inodes=1 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:392303 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7c6c1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d187:[<0x1000000d116/ctxfi v1858>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d187:[<0x1000000d116/ctxfi v3040>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2340>,<0x10000000006/t v1405>,<0x10000000005/tmp v1341>,<0x100000001fb/client.0 v1318>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1287>,<0x10000000001/sv_1 v1286>,<0x10000000000/qa v1286>,<0x1/volumes v1243>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.452342+0000 30=30+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:08:39.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:08:39.590+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d119 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97/ auth{2=1} v3208 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0) n(v3 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1) old_inodes=3 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:392306 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce9047e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d119:[<0x1000000d116/ac97 v22>,<0x1000000cf1b/pci v463>,<0x10000000202/sound v8728>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000d119:[<0x1000000d116/ac97 v3208>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2340>,<0x10000000006/t v1405>,<0x10000000005/tmp v1341>,<0x100000001fb/client.0 v1318>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1287>,<0x10000000001/sv_1 v1286>,<0x10000000000/qa v1286>,<0x1/volumes v1243>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:00.491085+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:08:39.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:08:39.606+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d1cc [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1/ auth{2=1} v3042 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0) n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:392313 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8f176c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v1862>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v3042>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2340>,<0x10000000006/t v1405>,<0x10000000005/tmp v1341>,<0x100000001fb/client.0 v1318>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1287>,<0x10000000001/sv_1 v1286>,<0x10000000000/qa v1286>,<0x1/volumes v1243>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.549340+0000 22=22+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:08:39.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:08:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:08:39.670+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d180 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio/ auth{1=1} v3038 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0) n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1) old_inodes=1 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:392352 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8ebd2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v1860>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v3038>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2340>,<0x10000000006/t v1405>,<0x10000000005/tmp v1341>,<0x100000001fb/client.0 v1318>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1287>,<0x10000000001/sv_1 v1286>,<0x10000000000/qa v1286>,<0x1/volumes v1243>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.414343+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:08:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:40 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:08:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:40 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:08:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:40 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:08:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:40 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:08:40.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:40 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:08:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:40 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:08:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:40 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:08:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:40 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:08:40.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:40 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:08:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:40 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:08:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:40 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:08:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:40 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:08:41.393 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:08:41 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:08:41.086+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000cde2 [...10,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders/ auth{4=1} v112 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0) n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:393144 nref=3)) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8dc52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v29>,<0x1000000cbec/selinux v646>,<0x10000000202/scripts v8820>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v112>,<0x1000000cbec/selinux v749>,<0x10000000202/scripts v11639>,<0x10000000201/linux-6.5.11 v2342>,<0x10000000006/t v1407>,<0x10000000005/tmp v1344>,<0x100000001fb/client.0 v1320>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1289>,<0x10000000001/sv_1 v1288>,<0x10000000000/qa v1289>,<0x1/volumes v1245>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:58.999116+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:08:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:41 smithi073 ceph-mon[36714]: pgmap v959: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 466 KiB/s wr, 474 op/s 2024-07-31T15:08:41.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:41 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:08:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:41 smithi067 ceph-mon[32750]: pgmap v959: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 466 KiB/s wr, 474 op/s 2024-07-31T15:08:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:41 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:08:41.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:41 smithi077 ceph-mon[37327]: pgmap v959: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 466 KiB/s wr, 474 op/s 2024-07-31T15:08:41.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:41 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:08:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f931cff9640 1 -- 172.21.15.67:0/688316852 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559744fd9170 con 0x559744c54570 2024-07-31T15:08:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f931cff9640 10 client.28522 handle_command_reply: tid=0 2024-07-31T15:08:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f9341acc740 1 client.28522 shutdown 2024-07-31T15:08:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f9341acc740 1 -- 172.21.15.67:0/688316852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744c9fb60 msgr2=0x559745075a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f9341acc740 1 --2- 172.21.15.67:0/688316852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744c9fb60 0x559745075a00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f932c0c7f40 tx=0x7f932c0d0350 comp rx=0 tx=0).stop 2024-07-31T15:08:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f9341acc740 1 -- 172.21.15.67:0/688316852 shutdown_connections 2024-07-31T15:08:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f9341acc740 1 --2- 172.21.15.67:0/688316852 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559744c54570 0x5597450eeaf0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f9341acc740 1 --2- 172.21.15.67:0/688316852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559744cc0840 0x559745075f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f9341acc740 1 --2- 172.21.15.67:0/688316852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559744c9fb60 0x559745075a00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f9341acc740 1 --2- 172.21.15.67:0/688316852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559744c97980 0x55974507a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f931cff9640 0 client.28522 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f9341acc740 1 -- 172.21.15.67:0/688316852 >> 172.21.15.67:0/688316852 conn(0x559744cb4990 msgr2=0x559745072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f9341acc740 1 -- 172.21.15.67:0/688316852 shutdown_connections 2024-07-31T15:08:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f9341acc740 1 -- 172.21.15.67:0/688316852 wait complete. 2024-07-31T15:08:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.742+0000 7f9341acc740 20 client.28522 trim_cache size 0 max 16384 2024-07-31T15:08:41.745 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:08:41.745 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-07-31T15:08:41.746 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:08:41.746 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 46, 2024-07-31T15:08:41.746 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:08:41.746 INFO:teuthology.orchestra.run.smithi067.stdout: "8bcd794b": { 2024-07-31T15:08:41.746 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 46, 2024-07-31T15:08:41.746 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:08:41.746 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:08:41.746 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:08:41.746 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:08:41.746 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:08:41.747 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:08:41.747 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 130.0, 2024-07-31T15:08:41.747 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:08:41.747 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:08:41.747 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:08:41.747 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:08:41.747 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:08:41.747 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:08:41.747 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:08:41.747 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:08:41.747 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:08:41.747 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:08:41.747 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:08:41.748 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:08:41.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.743+0000 7f933a319640 1 -- 172.21.15.67:0/94404725 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93200b3e80 msgr2=0x7f93200b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:41.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.743+0000 7f933a319640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93200b3e80 0x7f93200b6390 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f933410ab30 tx=0x7f933000b2e0 comp rx=0 tx=0).stop 2024-07-31T15:08:41.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.743+0000 7f933a319640 1 -- 172.21.15.67:0/94404725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f933410d110 msgr2=0x7f93341b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:41.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.743+0000 7f933a319640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f933410d110 0x7f93341b1840 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f932c0330e0 tx=0x7f932c034670 comp rx=0 tx=0).stop 2024-07-31T15:08:41.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.743+0000 7f933a319640 1 -- 172.21.15.67:0/94404725 shutdown_connections 2024-07-31T15:08:41.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.743+0000 7f933a319640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93200b3e80 0x7f93200b6390 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:41.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.743+0000 7f933a319640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f933410da60 0x7f93341b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.743+0000 7f933a319640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f933410d110 0x7f93341b1840 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.743+0000 7f933a319640 1 --2- 172.21.15.67:0/94404725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9334109df0 0x7f93341b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.743+0000 7f933a319640 1 -- 172.21.15.67:0/94404725 >> 172.21.15.67:0/94404725 conn(0x7f933406e440 msgr2=0x7f9334073660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.743+0000 7f933a319640 1 -- 172.21.15.67:0/94404725 shutdown_connections 2024-07-31T15:08:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:41.743+0000 7f933a319640 1 -- 172.21.15.67:0/94404725 wait complete. 2024-07-31T15:08:42.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:42.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:42.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:42.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:42.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:42.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:42.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:42.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:42.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:42.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:42.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:42.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: pgmap v960: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 524 KiB/s wr, 689 op/s 2024-07-31T15:08:43.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:43.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.557 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8bcd794b', quiesce took 33.3 seconds. Will release after: -23.299999999999997 2024-07-31T15:08:43.558 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:08:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: pgmap v960: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 524 KiB/s wr, 689 op/s 2024-07-31T15:08:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:43.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: pgmap v960: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 524 KiB/s wr, 689 op/s 2024-07-31T15:08:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:43.868 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:08:44.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.748+0000 7f68156dc640 1 -- 172.21.15.67:0/388800684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68101097c0 msgr2=0x7f681010fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.748+0000 7f68156dc640 1 --2- 172.21.15.67:0/388800684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68101097c0 0x7f681010fff0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6808009ca0 tx=0x7f680802a8c0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.748+0000 7f68156dc640 1 -- 172.21.15.67:0/388800684 shutdown_connections 2024-07-31T15:08:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.748+0000 7f68156dc640 1 --2- 172.21.15.67:0/388800684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68101097c0 0x7f681010fff0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.748+0000 7f68156dc640 1 --2- 172.21.15.67:0/388800684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6810108e70 0x7f6810109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.748+0000 7f68156dc640 1 --2- 172.21.15.67:0/388800684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6810103540 0x7f6810103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.748+0000 7f68156dc640 1 -- 172.21.15.67:0/388800684 >> 172.21.15.67:0/388800684 conn(0x7f68100fefa0 msgr2=0x7f68101013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.748+0000 7f68156dc640 1 -- 172.21.15.67:0/388800684 shutdown_connections 2024-07-31T15:08:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.748+0000 7f68156dc640 1 -- 172.21.15.67:0/388800684 wait complete. 2024-07-31T15:08:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.749+0000 7f68156dc640 1 Processor -- start 2024-07-31T15:08:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.749+0000 7f68156dc640 1 -- start start 2024-07-31T15:08:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f68156dc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6810103540 0x7f6810075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f68156dc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6810108e70 0x7f6810075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f68156dc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68101097c0 0x7f6810075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f68156dc640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6810113490 con 0x7f6810103540 2024-07-31T15:08:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f68156dc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6810113310 con 0x7f6810108e70 2024-07-31T15:08:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f68156dc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6810113610 con 0x7f68101097c0 2024-07-31T15:08:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f680e7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6810108e70 0x7f6810075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f680f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68101097c0 0x7f6810075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f680e7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6810108e70 0x7f6810075680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:39944/0 (socket says 172.21.15.67:39944) 2024-07-31T15:08:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f680e7fc640 1 -- 172.21.15.67:0/2990731111 learned_addr learned my addr 172.21.15.67:0/2990731111 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f680effd640 1 --2- 172.21.15.67:0/2990731111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6810103540 0x7f6810075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f680e7fc640 1 -- 172.21.15.67:0/2990731111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68101097c0 msgr2=0x7f6810075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f680e7fc640 1 --2- 172.21.15.67:0/2990731111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68101097c0 0x7f6810075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f680e7fc640 1 -- 172.21.15.67:0/2990731111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6810103540 msgr2=0x7f6810075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f680e7fc640 1 --2- 172.21.15.67:0/2990731111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6810103540 0x7f6810075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f680e7fc640 1 -- 172.21.15.67:0/2990731111 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68101a8090 con 0x7f6810108e70 2024-07-31T15:08:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.750+0000 7f680f7fe640 1 --2- 172.21.15.67:0/2990731111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68101097c0 0x7f6810075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.751+0000 7f680e7fc640 1 --2- 172.21.15.67:0/2990731111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6810108e70 0x7f6810075680 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f680400e9f0 tx=0x7f680400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.751+0000 7f67f7fff640 1 -- 172.21.15.67:0/2990731111 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680400ccc0 con 0x7f6810108e70 2024-07-31T15:08:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.751+0000 7f68156dc640 1 -- 172.21.15.67:0/2990731111 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68101a8380 con 0x7f6810108e70 2024-07-31T15:08:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.751+0000 7f68156dc640 1 -- 172.21.15.67:0/2990731111 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68101a8920 con 0x7f6810108e70 2024-07-31T15:08:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.751+0000 7f67f7fff640 1 -- 172.21.15.67:0/2990731111 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68040108e0 con 0x7f6810108e70 2024-07-31T15:08:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.751+0000 7f67f7fff640 1 -- 172.21.15.67:0/2990731111 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804018860 con 0x7f6810108e70 2024-07-31T15:08:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.754+0000 7f681c5ce740 1 Processor -- start 2024-07-31T15:08:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.754+0000 7f681c5ce740 1 -- start start 2024-07-31T15:08:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.754+0000 7f681c5ce740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22a41b0 0x557cb2646ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.754+0000 7f681c5ce740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb227b2f0 0x557cb2647220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.754+0000 7f681c5ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 0x557cb2647760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.754+0000 7f681c5ce740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557cb25bc3b0 con 0x557cb227b2f0 2024-07-31T15:08:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.754+0000 7f681c5ce740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557cb25ab8b0 con 0x557cb22a41b0 2024-07-31T15:08:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.754+0000 7f681c5ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557cb25af8f0 con 0x557cb22834d0 2024-07-31T15:08:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.754+0000 7f680f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22a41b0 0x557cb2646ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.754+0000 7f680effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb227b2f0 0x557cb2647220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.754+0000 7f680f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22a41b0 0x557cb2646ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:39952/0 (socket says 172.21.15.67:39952) 2024-07-31T15:08:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.754+0000 7f680f7fe640 1 -- 172.21.15.67:0/3016639505 learned_addr learned my addr 172.21.15.67:0/3016639505 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.755+0000 7f680f7fe640 1 -- 172.21.15.67:0/3016639505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 msgr2=0x557cb2647760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:08:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.755+0000 7f680f7fe640 1 --2- 172.21.15.67:0/3016639505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 0x557cb2647760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.755+0000 7f680f7fe640 1 -- 172.21.15.67:0/3016639505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb227b2f0 msgr2=0x557cb2647220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.755+0000 7f680f7fe640 1 --2- 172.21.15.67:0/3016639505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb227b2f0 0x557cb2647220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.755+0000 7f680f7fe640 1 -- 172.21.15.67:0/3016639505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cb25529a0 con 0x557cb22a41b0 2024-07-31T15:08:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.755+0000 7f67f7fff640 1 -- 172.21.15.67:0/2990731111 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6804010430 con 0x7f6810108e70 2024-07-31T15:08:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.755+0000 7f680f7fe640 1 --2- 172.21.15.67:0/3016639505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22a41b0 0x557cb2646ce0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f68080094c0 tx=0x7f680802f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e1d708465170667 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.755+0000 7f67f57fa640 1 -- 172.21.15.67:0/3016639505 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808031650 con 0x557cb22a41b0 2024-07-31T15:08:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.755+0000 7f67f57fa640 1 -- 172.21.15.67:0/3016639505 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6808036040 con 0x557cb22a41b0 2024-07-31T15:08:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.755+0000 7f67f57fa640 1 -- 172.21.15.67:0/3016639505 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808004440 con 0x557cb22a41b0 2024-07-31T15:08:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.756+0000 7f67f7fff640 1 --2- 172.21.15.67:0/2990731111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f67f80b3d70 0x7f67f80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.756+0000 7f680effd640 1 --2- 172.21.15.67:0/2990731111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f67f80b3d70 0x7f67f80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.756+0000 7f67f7fff640 1 -- 172.21.15.67:0/2990731111 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f68040e45b0 con 0x7f6810108e70 2024-07-31T15:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.756+0000 7f681c5ce740 1 -- 172.21.15.67:0/3016639505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22a41b0 msgr2=0x557cb2646ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.756+0000 7f681c5ce740 1 --2- 172.21.15.67:0/3016639505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22a41b0 0x557cb2646ce0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f68080094c0 tx=0x7f680802f310 comp rx=0 tx=0).stop 2024-07-31T15:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.756+0000 7f681c5ce740 1 -- 172.21.15.67:0/3016639505 shutdown_connections 2024-07-31T15:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.756+0000 7f681c5ce740 1 --2- 172.21.15.67:0/3016639505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 0x557cb2647760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.756+0000 7f681c5ce740 1 --2- 172.21.15.67:0/3016639505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb227b2f0 0x557cb2647220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.756+0000 7f681c5ce740 1 --2- 172.21.15.67:0/3016639505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22a41b0 0x557cb2646ce0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.756+0000 7f681c5ce740 1 -- 172.21.15.67:0/3016639505 >> 172.21.15.67:0/3016639505 conn(0x557cb2298300 msgr2=0x557cb2644bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.756+0000 7f680effd640 1 --2- 172.21.15.67:0/2990731111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f67f80b3d70 0x7f67f80b6280 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x557cb2645960 tx=0x7f6800005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.756+0000 7f681c5ce740 1 -- 172.21.15.67:0/3016639505 shutdown_connections 2024-07-31T15:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.756+0000 7f681c5ce740 1 -- 172.21.15.67:0/3016639505 wait complete. 2024-07-31T15:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.757+0000 7f681c5ce740 1 -- start start 2024-07-31T15:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.757+0000 7f681c5ce740 1 Processor -- start 2024-07-31T15:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.757+0000 7f681c5ce740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb227b2f0 0x557cb26528f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.757+0000 7f681c5ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 0x557cb2652e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.757+0000 7f681c5ce740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb22a41b0 0x557cb265e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.757+0000 7f681c5ce740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557cb24e5ab0 con 0x557cb22a41b0 2024-07-31T15:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.757+0000 7f681c5ce740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557cb2533cf0 con 0x557cb227b2f0 2024-07-31T15:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.757+0000 7f681c5ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557cb2655280 con 0x557cb22834d0 2024-07-31T15:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.757+0000 7f681c5ce740 10 client.0 _ll_register_callbacks cb 0x557cb25a59f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.757+0000 7f680effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb227b2f0 0x557cb26528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.757+0000 7f680effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb227b2f0 0x557cb26528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:39956/0 (socket says 172.21.15.67:39956) 2024-07-31T15:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.757+0000 7f680effd640 1 -- 172.21.15.67:0/2048821357 learned_addr learned my addr 172.21.15.67:0/2048821357 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.757+0000 7f680e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 0x557cb2652e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.757+0000 7f680f7fe640 1 --2- 172.21.15.67:0/2048821357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb22a41b0 0x557cb265e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.758+0000 7f680effd640 1 -- 172.21.15.67:0/2048821357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 msgr2=0x557cb2652e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.758+0000 7f680effd640 1 --2- 172.21.15.67:0/2048821357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 0x557cb2652e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.758+0000 7f680effd640 1 -- 172.21.15.67:0/2048821357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb22a41b0 msgr2=0x557cb265e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.758+0000 7f680effd640 1 --2- 172.21.15.67:0/2048821357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb22a41b0 0x557cb265e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.758+0000 7f680effd640 1 -- 172.21.15.67:0/2048821357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cb230f790 con 0x557cb227b2f0 2024-07-31T15:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.758+0000 7f680f7fe640 1 --2- 172.21.15.67:0/2048821357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb22a41b0 0x557cb265e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.758+0000 7f680effd640 1 --2- 172.21.15.67:0/2048821357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb227b2f0 0x557cb26528f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6800013010 tx=0x7f6800004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ba42a091ef87842 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.758+0000 7f67f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.759+0000 7f67f2ff5640 1 -- 172.21.15.67:0/2048821357 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680001e680 con 0x557cb227b2f0 2024-07-31T15:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.759+0000 7f67f2ff5640 1 -- 172.21.15.67:0/2048821357 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f680001e820 con 0x557cb227b2f0 2024-07-31T15:08:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.759+0000 7f681c5ce740 1 -- 172.21.15.67:0/2048821357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6808036040 con 0x557cb227b2f0 2024-07-31T15:08:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.759+0000 7f67f2ff5640 1 -- 172.21.15.67:0/2048821357 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f680001ea20 con 0x557cb227b2f0 2024-07-31T15:08:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.759+0000 7f681c5ce740 10 client.28742 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.759+0000 7f681c5ce740 1 -- 172.21.15.67:0/2048821357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x557cb2388620 con 0x557cb227b2f0 2024-07-31T15:08:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.759+0000 7f67f2ff5640 1 -- 172.21.15.67:0/2048821357 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6808036040 con 0x557cb227b2f0 2024-07-31T15:08:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.759+0000 7f67f2ff5640 1 -- 172.21.15.67:0/2048821357 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f6800027070 con 0x557cb227b2f0 2024-07-31T15:08:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.759+0000 7f681c5ce740 10 client.28742 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.759+0000 7f681c5ce740 10 client.28742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:08:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.759+0000 7f681c5ce740 20 client.28742 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.759+0000 7f681c5ce740 1 --2- 172.21.15.67:0/2048821357 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557cb2262d80 0x557cb2664670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.760+0000 7f681c5ce740 4 client.28742 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:08:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.760+0000 7f681c5ce740 1 -- 172.21.15.67:0/2048821357 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cb25be300 con 0x557cb2262d80 2024-07-31T15:08:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.760+0000 7f680e7fc640 1 --2- 172.21.15.67:0/2048821357 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557cb2262d80 0x557cb2664670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.760+0000 7f67f2ff5640 10 client.28742 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.760+0000 7f680e7fc640 1 --2- 172.21.15.67:0/2048821357 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557cb2262d80 0x557cb2664670 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe0564222ff76ac server_cookie=6e96b90e184561ba in_seq=0 out_seq=0 2024-07-31T15:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.761+0000 7f67f2ff5640 1 -- 172.21.15.67:0/2048821357 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cb25be300 con 0x557cb2262d80 2024-07-31T15:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.761+0000 7f67f2ff5640 10 client.28742 handle_command_reply: tid=0 2024-07-31T15:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.761+0000 7f681c5ce740 1 client.28742 shutdown 2024-07-31T15:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.761+0000 7f681c5ce740 1 -- 172.21.15.67:0/2048821357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb227b2f0 msgr2=0x557cb26528f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.761+0000 7f681c5ce740 1 --2- 172.21.15.67:0/2048821357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb227b2f0 0x557cb26528f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6800013010 tx=0x7f6800004730 comp rx=0 tx=0).stop 2024-07-31T15:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.761+0000 7f681c5ce740 1 -- 172.21.15.67:0/2048821357 shutdown_connections 2024-07-31T15:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.761+0000 7f681c5ce740 1 --2- 172.21.15.67:0/2048821357 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557cb2262d80 0x557cb2664670 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.761+0000 7f681c5ce740 1 --2- 172.21.15.67:0/2048821357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb22a41b0 0x557cb265e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.761+0000 7f681c5ce740 1 --2- 172.21.15.67:0/2048821357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 0x557cb2652e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.761+0000 7f67f2ff5640 0 client.28742 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.761+0000 7f681c5ce740 1 --2- 172.21.15.67:0/2048821357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb227b2f0 0x557cb26528f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.761+0000 7f681c5ce740 1 -- 172.21.15.67:0/2048821357 >> 172.21.15.67:0/2048821357 conn(0x557cb2298300 msgr2=0x557cb26432b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.761+0000 7f681c5ce740 1 -- 172.21.15.67:0/2048821357 shutdown_connections 2024-07-31T15:08:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.761+0000 7f681c5ce740 1 -- 172.21.15.67:0/2048821357 wait complete. 2024-07-31T15:08:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.762+0000 7f681c5ce740 20 client.28742 trim_cache size 0 max 16384 2024-07-31T15:08:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.781+0000 7f681c5ce740 1 Processor -- start 2024-07-31T15:08:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.781+0000 7f681c5ce740 1 -- start start 2024-07-31T15:08:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.781+0000 7f681c5ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb227b2f0 0x557cb26439c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.781+0000 7f681c5ce740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22834d0 0x557cb2652bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.781+0000 7f681c5ce740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb22a41b0 0x557cb26444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.781+0000 7f681c5ce740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f68101b2980 con 0x557cb22a41b0 2024-07-31T15:08:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.781+0000 7f681c5ce740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f68101b44c0 con 0x557cb22834d0 2024-07-31T15:08:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.781+0000 7f681c5ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68101b3050 con 0x557cb227b2f0 2024-07-31T15:08:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.782+0000 7f680effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb227b2f0 0x557cb26439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.782+0000 7f680effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb227b2f0 0x557cb26439c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34876/0 (socket says 172.21.15.67:34876) 2024-07-31T15:08:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.782+0000 7f680effd640 1 -- 172.21.15.67:0/2811559099 learned_addr learned my addr 172.21.15.67:0/2811559099 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.782+0000 7f680e7fc640 1 --2- 172.21.15.67:0/2811559099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22834d0 0x557cb2652bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.782+0000 7f680f7fe640 1 --2- 172.21.15.67:0/2811559099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb22a41b0 0x557cb26444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.782+0000 7f680effd640 1 -- 172.21.15.67:0/2811559099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22834d0 msgr2=0x557cb2652bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.782+0000 7f680effd640 1 --2- 172.21.15.67:0/2811559099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22834d0 0x557cb2652bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.782+0000 7f680effd640 1 -- 172.21.15.67:0/2811559099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb22a41b0 msgr2=0x557cb26444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.782+0000 7f680effd640 1 --2- 172.21.15.67:0/2811559099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb22a41b0 0x557cb26444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.782+0000 7f680effd640 1 -- 172.21.15.67:0/2811559099 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cb26454d0 con 0x557cb227b2f0 2024-07-31T15:08:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.782+0000 7f680effd640 1 --2- 172.21.15.67:0/2811559099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb227b2f0 0x557cb26439c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6800004450 tx=0x7f6800009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79e550d914aa2f01 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.782+0000 7f67f4ff9640 1 -- 172.21.15.67:0/2811559099 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680001e680 con 0x557cb227b2f0 2024-07-31T15:08:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.783+0000 7f67f4ff9640 1 -- 172.21.15.67:0/2811559099 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f680001e820 con 0x557cb227b2f0 2024-07-31T15:08:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.783+0000 7f67f4ff9640 1 -- 172.21.15.67:0/2811559099 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680001f450 con 0x557cb227b2f0 2024-07-31T15:08:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.784+0000 7f681c5ce740 1 -- 172.21.15.67:0/2811559099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb227b2f0 msgr2=0x557cb26439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.784+0000 7f681c5ce740 1 --2- 172.21.15.67:0/2811559099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb227b2f0 0x557cb26439c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6800004450 tx=0x7f6800009f20 comp rx=0 tx=0).stop 2024-07-31T15:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.784+0000 7f681c5ce740 1 -- 172.21.15.67:0/2811559099 shutdown_connections 2024-07-31T15:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.784+0000 7f681c5ce740 1 --2- 172.21.15.67:0/2811559099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb22a41b0 0x557cb26444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.784+0000 7f681c5ce740 1 --2- 172.21.15.67:0/2811559099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22834d0 0x557cb2652bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.784+0000 7f681c5ce740 1 --2- 172.21.15.67:0/2811559099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb227b2f0 0x557cb26439c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.784+0000 7f681c5ce740 1 -- 172.21.15.67:0/2811559099 >> 172.21.15.67:0/2811559099 conn(0x557cb2298300 msgr2=0x557cb26432b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.784+0000 7f681c5ce740 1 -- 172.21.15.67:0/2811559099 shutdown_connections 2024-07-31T15:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.784+0000 7f681c5ce740 1 -- 172.21.15.67:0/2811559099 wait complete. 2024-07-31T15:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.784+0000 7f681c5ce740 1 -- start start 2024-07-31T15:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.784+0000 7f681c5ce740 1 Processor -- start 2024-07-31T15:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f681c5ce740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb227b2f0 0x557cb265a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f681c5ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 0x557cb265ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f681c5ce740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22a41b0 0x557cb2657610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f681c5ce740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f68101b3df0 con 0x557cb227b2f0 2024-07-31T15:08:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f681c5ce740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f68101b3720 con 0x557cb22a41b0 2024-07-31T15:08:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f681c5ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68101b0980 con 0x557cb22834d0 2024-07-31T15:08:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f681c5ce740 10 client.0 _ll_register_callbacks cb 0x7f6800009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f680e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 0x557cb265ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f680e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 0x557cb265ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34888/0 (socket says 172.21.15.67:34888) 2024-07-31T15:08:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f680e7fc640 1 -- 172.21.15.67:0/3443396314 learned_addr learned my addr 172.21.15.67:0/3443396314 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f680effd640 1 --2- 172.21.15.67:0/3443396314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb227b2f0 0x557cb265a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f680f7fe640 1 --2- 172.21.15.67:0/3443396314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22a41b0 0x557cb2657610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f680e7fc640 1 -- 172.21.15.67:0/3443396314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22a41b0 msgr2=0x557cb2657610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f680e7fc640 1 --2- 172.21.15.67:0/3443396314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22a41b0 0x557cb2657610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f680e7fc640 1 -- 172.21.15.67:0/3443396314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb227b2f0 msgr2=0x557cb265a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f680e7fc640 1 --2- 172.21.15.67:0/3443396314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb227b2f0 0x557cb265a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f680e7fc640 1 -- 172.21.15.67:0/3443396314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cb2657da0 con 0x557cb22834d0 2024-07-31T15:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f680e7fc640 1 --2- 172.21.15.67:0/3443396314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 0x557cb265ada0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f68040e1a00 tx=0x7f680400ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3e3b6cc066000ae server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.785+0000 7f67f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.786+0000 7f67f2ff5640 1 -- 172.21.15.67:0/3443396314 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680408f070 con 0x557cb22834d0 2024-07-31T15:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.786+0000 7f681c5ce740 1 -- 172.21.15.67:0/3443396314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f680001e820 con 0x557cb22834d0 2024-07-31T15:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.786+0000 7f67f2ff5640 1 -- 172.21.15.67:0/3443396314 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f68040a71a0 con 0x557cb22834d0 2024-07-31T15:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.786+0000 7f67f2ff5640 1 -- 172.21.15.67:0/3443396314 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f68040b3f10 con 0x557cb22834d0 2024-07-31T15:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.786+0000 7f681c5ce740 10 client.28627 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.786+0000 7f681c5ce740 1 -- 172.21.15.67:0/3443396314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x557cb26cb9f0 con 0x557cb22834d0 2024-07-31T15:08:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.786+0000 7f67f2ff5640 1 -- 172.21.15.67:0/3443396314 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f680001e820 con 0x557cb22834d0 2024-07-31T15:08:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.786+0000 7f67f2ff5640 1 -- 172.21.15.67:0/3443396314 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f68040a6980 con 0x557cb22834d0 2024-07-31T15:08:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.786+0000 7f681c5ce740 10 client.28627 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.786+0000 7f681c5ce740 10 client.28627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:08:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.786+0000 7f681c5ce740 20 client.28627 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.786+0000 7f681c5ce740 1 --2- 172.21.15.67:0/3443396314 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557cb22381c0 0x557cb26ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.787+0000 7f681c5ce740 4 client.28627 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:08:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.787+0000 7f681c5ce740 1 -- 172.21.15.67:0/3443396314 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557cb25b9da0 con 0x557cb22381c0 2024-07-31T15:08:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.787+0000 7f680effd640 1 --2- 172.21.15.67:0/3443396314 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557cb22381c0 0x557cb26ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.787+0000 7f67f2ff5640 10 client.28627 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.787+0000 7f680effd640 1 --2- 172.21.15.67:0/3443396314 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557cb22381c0 0x557cb26ce120 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83d7c7408757eced server_cookie=1075309bb9bff1fb in_seq=0 out_seq=0 2024-07-31T15:08:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.959+0000 7f67f2ff5640 1 -- 172.21.15.67:0/3443396314 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557cb25b9da0 con 0x557cb22381c0 2024-07-31T15:08:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.959+0000 7f67f2ff5640 10 client.28627 handle_command_reply: tid=0 2024-07-31T15:08:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.959+0000 7f681c5ce740 1 client.28627 shutdown 2024-07-31T15:08:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.959+0000 7f681c5ce740 1 -- 172.21.15.67:0/3443396314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 msgr2=0x557cb265ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.959+0000 7f681c5ce740 1 --2- 172.21.15.67:0/3443396314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 0x557cb265ada0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f68040e1a00 tx=0x7f680400ce60 comp rx=0 tx=0).stop 2024-07-31T15:08:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.959+0000 7f681c5ce740 1 -- 172.21.15.67:0/3443396314 shutdown_connections 2024-07-31T15:08:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.959+0000 7f681c5ce740 1 --2- 172.21.15.67:0/3443396314 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557cb22381c0 0x557cb26ce120 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.959+0000 7f681c5ce740 1 --2- 172.21.15.67:0/3443396314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cb22a41b0 0x557cb2657610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.959+0000 7f67f2ff5640 0 client.28627 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.959+0000 7f681c5ce740 1 --2- 172.21.15.67:0/3443396314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557cb22834d0 0x557cb265ada0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.959+0000 7f681c5ce740 1 --2- 172.21.15.67:0/3443396314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cb227b2f0 0x557cb265a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.959+0000 7f681c5ce740 1 -- 172.21.15.67:0/3443396314 >> 172.21.15.67:0/3443396314 conn(0x557cb2298300 msgr2=0x557cb26444b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.959+0000 7f681c5ce740 1 -- 172.21.15.67:0/3443396314 shutdown_connections 2024-07-31T15:08:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.960+0000 7f681c5ce740 1 -- 172.21.15.67:0/3443396314 wait complete. 2024-07-31T15:08:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.960+0000 7f681c5ce740 20 client.28627 trim_cache size 0 max 16384 2024-07-31T15:08:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:08:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:08:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:08:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:08:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.961+0000 7f68156dc640 1 -- 172.21.15.67:0/2990731111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f67f80b3d70 msgr2=0x7f67f80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.961+0000 7f68156dc640 1 --2- 172.21.15.67:0/2990731111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f67f80b3d70 0x7f67f80b6280 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x557cb2645960 tx=0x7f6800005f90 comp rx=0 tx=0).stop 2024-07-31T15:08:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.961+0000 7f68156dc640 1 -- 172.21.15.67:0/2990731111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6810108e70 msgr2=0x7f6810075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.961+0000 7f68156dc640 1 --2- 172.21.15.67:0/2990731111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6810108e70 0x7f6810075680 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f680400e9f0 tx=0x7f680400eeb0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.961+0000 7f68156dc640 1 -- 172.21.15.67:0/2990731111 shutdown_connections 2024-07-31T15:08:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.961+0000 7f68156dc640 1 --2- 172.21.15.67:0/2990731111 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f67f80b3d70 0x7f67f80b6280 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.961+0000 7f68156dc640 1 --2- 172.21.15.67:0/2990731111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68101097c0 0x7f6810075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.961+0000 7f68156dc640 1 --2- 172.21.15.67:0/2990731111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6810108e70 0x7f6810075680 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.961+0000 7f68156dc640 1 --2- 172.21.15.67:0/2990731111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6810103540 0x7f6810075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.961+0000 7f68156dc640 1 -- 172.21.15.67:0/2990731111 >> 172.21.15.67:0/2990731111 conn(0x7f68100fefa0 msgr2=0x7f68101063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.961+0000 7f68156dc640 1 -- 172.21.15.67:0/2990731111 shutdown_connections 2024-07-31T15:08:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:44.961+0000 7f68156dc640 1 -- 172.21.15.67:0/2990731111 wait complete. 2024-07-31T15:08:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: pgmap v961: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 801 KiB/s wr, 1.01k op/s 2024-07-31T15:08:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:45.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: pgmap v961: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 801 KiB/s wr, 1.01k op/s 2024-07-31T15:08:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:45.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: pgmap v961: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 801 KiB/s wr, 1.01k op/s 2024-07-31T15:08:45.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:45.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:45.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.076+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1000947977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c109ec0 msgr2=0x7f909c10a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:46.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.076+0000 7f90a2c1c640 1 --2- 172.21.15.67:0/1000947977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c109ec0 0x7f909c10a2d0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f9090009900 tx=0x7f909002a410 comp rx=0 tx=0).stop 2024-07-31T15:08:46.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.076+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1000947977 shutdown_connections 2024-07-31T15:08:46.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.076+0000 7f90a2c1c640 1 --2- 172.21.15.67:0/1000947977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f909c10cc40 0x7f909c1166d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:46.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.076+0000 7f90a2c1c640 1 --2- 172.21.15.67:0/1000947977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f909c10c2f0 0x7f909c10c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:46.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.076+0000 7f90a2c1c640 1 --2- 172.21.15.67:0/1000947977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c109ec0 0x7f909c10a2d0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:46.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.076+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1000947977 >> 172.21.15.67:0/1000947977 conn(0x7f909c079100 msgr2=0x7f909c07b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:46.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.076+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1000947977 shutdown_connections 2024-07-31T15:08:46.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.076+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1000947977 wait complete. 2024-07-31T15:08:46.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.077+0000 7f90a2c1c640 1 Processor -- start 2024-07-31T15:08:46.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.078+0000 7f90a2c1c640 1 -- start start 2024-07-31T15:08:46.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.078+0000 7f90a2c1c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f909c109ec0 0x7f909c1a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:46.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.078+0000 7f90a2c1c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c10c2f0 0x7f909c1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:46.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.078+0000 7f90a2c1c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f909c10cc40 0x7f909c1a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:46.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.078+0000 7f90a2c1c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f909c119b10 con 0x7f909c10c2f0 2024-07-31T15:08:46.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.078+0000 7f90a2c1c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f909c119990 con 0x7f909c109ec0 2024-07-31T15:08:46.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.078+0000 7f90a11b9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f909c10cc40 0x7f909c1a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:46.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.078+0000 7f90a2c1c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f909c119c90 con 0x7f909c10cc40 2024-07-31T15:08:46.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.078+0000 7f90a09b8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f909c109ec0 0x7f909c1a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:46.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.078+0000 7f90a09b8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f909c109ec0 0x7f909c1a0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:39990/0 (socket says 172.21.15.67:39990) 2024-07-31T15:08:46.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.078+0000 7f90a09b8640 1 -- 172.21.15.67:0/1945505060 learned_addr learned my addr 172.21.15.67:0/1945505060 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:46.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.078+0000 7f909bfff640 1 --2- 172.21.15.67:0/1945505060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c10c2f0 0x7f909c1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:46.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.079+0000 7f909bfff640 1 -- 172.21.15.67:0/1945505060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f909c10cc40 msgr2=0x7f909c1a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:46.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.079+0000 7f909bfff640 1 --2- 172.21.15.67:0/1945505060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f909c10cc40 0x7f909c1a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:46.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.079+0000 7f909bfff640 1 -- 172.21.15.67:0/1945505060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f909c109ec0 msgr2=0x7f909c1a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:46.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.079+0000 7f909bfff640 1 --2- 172.21.15.67:0/1945505060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f909c109ec0 0x7f909c1a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:46.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.079+0000 7f909bfff640 1 -- 172.21.15.67:0/1945505060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909c1a64b0 con 0x7f909c10c2f0 2024-07-31T15:08:46.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.079+0000 7f90a09b8640 1 --2- 172.21.15.67:0/1945505060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f909c109ec0 0x7f909c1a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:46.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.079+0000 7f909bfff640 1 --2- 172.21.15.67:0/1945505060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c10c2f0 0x7f909c1a0ae0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f909c06b560 tx=0x7f908800db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:46.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.079+0000 7f9099ffb640 1 -- 172.21.15.67:0/1945505060 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088017070 con 0x7f909c10c2f0 2024-07-31T15:08:46.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.080+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1945505060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909c1a67a0 con 0x7f909c10c2f0 2024-07-31T15:08:46.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.080+0000 7f9099ffb640 1 -- 172.21.15.67:0/1945505060 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9088009820 con 0x7f909c10c2f0 2024-07-31T15:08:46.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.080+0000 7f9099ffb640 1 -- 172.21.15.67:0/1945505060 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088004b90 con 0x7f909c10c2f0 2024-07-31T15:08:46.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.080+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1945505060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f909c1acf00 con 0x7f909c10c2f0 2024-07-31T15:08:46.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.082+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1945505060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f909c10a2d0 con 0x7f909c10c2f0 2024-07-31T15:08:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.084+0000 7f9099ffb640 1 -- 172.21.15.67:0/1945505060 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f9088004d30 con 0x7f909c10c2f0 2024-07-31T15:08:46.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.085+0000 7f9099ffb640 1 --2- 172.21.15.67:0/1945505060 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f908c0b3dc0 0x7f908c0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.085+0000 7f90a09b8640 1 --2- 172.21.15.67:0/1945505060 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f908c0b3dc0 0x7f908c0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.085+0000 7f9099ffb640 1 -- 172.21.15.67:0/1945505060 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f90880e3f70 con 0x7f909c10c2f0 2024-07-31T15:08:46.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.086+0000 7f90a09b8640 1 --2- 172.21.15.67:0/1945505060 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f908c0b3dc0 0x7f908c0b62d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f90900096c0 tx=0x7f90900058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:46.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.089+0000 7f9099ffb640 1 -- 172.21.15.67:0/1945505060 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f90880099d0 con 0x7f909c10c2f0 2024-07-31T15:08:46.280 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:08:45 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:08:45.952+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000948f [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250/ auth{4=1} v716 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:35.607536+0000 75=75+0) n(v5 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:425393 nref=3)) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce836e4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000948f:[<0x1000000948d/8250 v533>,<0x1000000945c/serial v657>,<0x10000001d8f/tty v18356>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x1000000948f:[<0x1000000948d/8250 v716>,<0x1000000945c/serial v1492>,<0x10000001d8f/tty v18901>,<0x10000000202/drivers v21237>,<0x10000000201/linux-6.5.11 v2352>,<0x10000000006/t v1412>,<0x10000000005/tmp v1348>,<0x100000001fb/client.0 v1324>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1291>,<0x10000000001/sv_1 v1292>,<0x10000000000/qa v1293>,<0x1/volumes v1247>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:33.063897+0000 47=47+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:35.607536+0000 75=75+0)","memory_value.rstat":"n(v5 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:08:46.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.373+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1945505060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f909c1a1670 con 0x7f909c10c2f0 2024-07-31T15:08:46.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.373+0000 7f9099ffb640 1 -- 172.21.15.67:0/1945505060 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11962 (secure 0 0 0) 0x7f909c1a1670 con 0x7f909c10c2f0 2024-07-31T15:08:46.375 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:08:46.375 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"btime":"2024-07-31T15:06:04:323642+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:06:04.323492+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28214":{"gid":28214,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1516494761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1516494761},{"type":"v1","addr":"172.21.15.73:6835","nonce":1516494761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:08:46.376 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-07-31T15:08:46.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.376+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1945505060 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f908c0b3dc0 msgr2=0x7f908c0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:46.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.376+0000 7f90a2c1c640 1 --2- 172.21.15.67:0/1945505060 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f908c0b3dc0 0x7f908c0b62d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f90900096c0 tx=0x7f90900058d0 comp rx=0 tx=0).stop 2024-07-31T15:08:46.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.376+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1945505060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c10c2f0 msgr2=0x7f909c1a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:46.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.376+0000 7f90a2c1c640 1 --2- 172.21.15.67:0/1945505060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c10c2f0 0x7f909c1a0ae0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f909c06b560 tx=0x7f908800db30 comp rx=0 tx=0).stop 2024-07-31T15:08:46.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.376+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1945505060 shutdown_connections 2024-07-31T15:08:46.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.376+0000 7f90a2c1c640 1 --2- 172.21.15.67:0/1945505060 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f908c0b3dc0 0x7f908c0b62d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:46.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.376+0000 7f90a2c1c640 1 --2- 172.21.15.67:0/1945505060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f909c10cc40 0x7f909c1a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:46.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.376+0000 7f90a2c1c640 1 --2- 172.21.15.67:0/1945505060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c10c2f0 0x7f909c1a0ae0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:46.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.376+0000 7f90a2c1c640 1 --2- 172.21.15.67:0/1945505060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f909c109ec0 0x7f909c1a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:46.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.376+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1945505060 >> 172.21.15.67:0/1945505060 conn(0x7f909c079100 msgr2=0x7f909c10eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:46.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.376+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1945505060 shutdown_connections 2024-07-31T15:08:46.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:46.376+0000 7f90a2c1c640 1 -- 172.21.15.67:0/1945505060 wait complete. 2024-07-31T15:08:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:46.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:46 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:08:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:08:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:46 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:08:47.088 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 8bcd794b --release --await' 2024-07-31T15:08:47.088 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 8bcd794b --release --await 2024-07-31T15:08:47.281 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:08:46 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:08:46.953+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000439c [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du/ auth{2=1} v303 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:01.164016+0000 34=34+0) n(v3 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:429586 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@6} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce83f19600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000439c:[<0x10000004399/rcar-du v163>,<0x10000002d23/renesas v2988>,<0x10000002d21/drm v1196>,<0x10000001d8f/gpu v9523>,<0x10000000202/drivers v2799>,<0x10000000201/linux-6.5.11 v334>,<0x10000000006/t v232>,<0x10000000005/tmp v202>,<0x100000001fb/client.0 v201>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v184>,<0x10000000001/sv_1 v213>,<0x10000000000/qa v197>,<0x1/volumes v198>]//[]","memoryvalue":"(2)0x1000000439c:[<0x10000004399/rcar-du v303>,<0x10000002d23/renesas v3147>,<0x10000002d21/drm v1391>,<0x10000001d8f/gpu v18675>,<0x10000000202/drivers v21237>,<0x10000000201/linux-6.5.11 v2352>,<0x10000000006/t v1412>,<0x10000000005/tmp v1348>,<0x100000001fb/client.0 v1324>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1291>,<0x10000000001/sv_1 v1292>,<0x10000000000/qa v1293>,<0x1/volumes v1247>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:38.082295+0000 30=30+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:01.164016+0000 34=34+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:08:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:08:47.525 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.525 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.525 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.525 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: pgmap v962: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 962 KiB/s wr, 1.47k op/s 2024-07-31T15:08:47.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1945505060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:08:47.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.530 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:47.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:08:47.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: pgmap v962: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 962 KiB/s wr, 1.47k op/s 2024-07-31T15:08:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1945505060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:08:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:47.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:08:47.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: pgmap v962: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 962 KiB/s wr, 1.47k op/s 2024-07-31T15:08:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1945505060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:47.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:47.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:08:47.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:48.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: pgmap v963: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.0 MiB/s wr, 2.98k op/s 2024-07-31T15:08:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:49.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: pgmap v963: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.0 MiB/s wr, 2.98k op/s 2024-07-31T15:08:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:49.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: pgmap v963: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.0 MiB/s wr, 2.98k op/s 2024-07-31T15:08:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:49.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:49.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.054+0000 7f086fc7b640 1 -- 172.21.15.67:0/2894840337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08681035e0 msgr2=0x7f08681039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.054+0000 7f086fc7b640 1 --2- 172.21.15.67:0/2894840337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08681035e0 0x7f08681039f0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f0864009900 tx=0x7f086402a410 comp rx=0 tx=0).stop 2024-07-31T15:08:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.054+0000 7f086fc7b640 1 -- 172.21.15.67:0/2894840337 shutdown_connections 2024-07-31T15:08:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.054+0000 7f086fc7b640 1 --2- 172.21.15.67:0/2894840337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0868109870 0x7f08681100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.054+0000 7f086fc7b640 1 --2- 172.21.15.67:0/2894840337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0868108f20 0x7f0868109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.054+0000 7f086fc7b640 1 --2- 172.21.15.67:0/2894840337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08681035e0 0x7f08681039f0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.054+0000 7f086fc7b640 1 -- 172.21.15.67:0/2894840337 >> 172.21.15.67:0/2894840337 conn(0x7f08680ff030 msgr2=0x7f0868101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.054+0000 7f086fc7b640 1 -- 172.21.15.67:0/2894840337 shutdown_connections 2024-07-31T15:08:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.054+0000 7f086fc7b640 1 -- 172.21.15.67:0/2894840337 wait complete. 2024-07-31T15:08:50.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.054+0000 7f086fc7b640 1 Processor -- start 2024-07-31T15:08:50.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.054+0000 7f086fc7b640 1 -- start start 2024-07-31T15:08:50.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086fc7b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08681035e0 0x7f0868075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086fc7b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0868108f20 0x7f0868075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086fc7b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0868109870 0x7f0868075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086fc7b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0868113520 con 0x7f08681035e0 2024-07-31T15:08:50.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086fc7b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f08681133a0 con 0x7f0868109870 2024-07-31T15:08:50.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086fc7b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08681136a0 con 0x7f0868108f20 2024-07-31T15:08:50.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086da17640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08681035e0 0x7f0868075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086da17640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08681035e0 0x7f0868075140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36762/0 (socket says 172.21.15.67:36762) 2024-07-31T15:08:50.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086da17640 1 -- 172.21.15.67:0/3215257598 learned_addr learned my addr 172.21.15.67:0/3215257598 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:50.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086d216640 1 --2- 172.21.15.67:0/3215257598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0868108f20 0x7f0868075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086e218640 1 --2- 172.21.15.67:0/3215257598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0868109870 0x7f0868075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086da17640 1 -- 172.21.15.67:0/3215257598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0868108f20 msgr2=0x7f0868075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:50.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086da17640 1 --2- 172.21.15.67:0/3215257598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0868108f20 0x7f0868075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086da17640 1 -- 172.21.15.67:0/3215257598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0868109870 msgr2=0x7f0868075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086da17640 1 --2- 172.21.15.67:0/3215257598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0868109870 0x7f0868075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086da17640 1 -- 172.21.15.67:0/3215257598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08681a8270 con 0x7f08681035e0 2024-07-31T15:08:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.055+0000 7f086da17640 1 --2- 172.21.15.67:0/3215257598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08681035e0 0x7f0868075140 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0864005940 tx=0x7f08640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.056+0000 7f085effd640 1 -- 172.21.15.67:0/3215257598 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086403f030 con 0x7f08681035e0 2024-07-31T15:08:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.056+0000 7f086fc7b640 1 -- 172.21.15.67:0/3215257598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08681a8500 con 0x7f08681035e0 2024-07-31T15:08:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.056+0000 7f085effd640 1 -- 172.21.15.67:0/3215257598 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0864003cb0 con 0x7f08681035e0 2024-07-31T15:08:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.056+0000 7f085effd640 1 -- 172.21.15.67:0/3215257598 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864035070 con 0x7f08681035e0 2024-07-31T15:08:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.056+0000 7f086fc7b640 1 -- 172.21.15.67:0/3215257598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08681a89e0 con 0x7f08681035e0 2024-07-31T15:08:50.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.058+0000 7f085effd640 1 -- 172.21.15.67:0/3215257598 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0864042030 con 0x7f08681035e0 2024-07-31T15:08:50.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.059+0000 7f08761cc740 1 Processor -- start 2024-07-31T15:08:50.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.059+0000 7f08761cc740 1 -- start start 2024-07-31T15:08:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.059+0000 7f08761cc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc57b9520 0x557dc5b5f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.059+0000 7f08761cc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc5790660 0x557dc5b5f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.059+0000 7f08761cc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5798840 0x557dc5b5faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.059+0000 7f08761cc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557dc5ad4e70 con 0x557dc5790660 2024-07-31T15:08:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.059+0000 7f08761cc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557dc5ac4370 con 0x557dc5798840 2024-07-31T15:08:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.059+0000 7f08761cc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557dc5ac83b0 con 0x557dc57b9520 2024-07-31T15:08:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.059+0000 7f086da17640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc5790660 0x557dc5b5f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.059+0000 7f086e218640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc57b9520 0x557dc5b5f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.059+0000 7f086da17640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc5790660 0x557dc5b5f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36778/0 (socket says 172.21.15.67:36778) 2024-07-31T15:08:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.059+0000 7f086da17640 1 -- 172.21.15.67:0/1434528358 learned_addr learned my addr 172.21.15.67:0/1434528358 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.059+0000 7f086d216640 1 --2- 172.21.15.67:0/1434528358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5798840 0x557dc5b5faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.060+0000 7f086da17640 1 -- 172.21.15.67:0/1434528358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc57b9520 msgr2=0x557dc5b5f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.060+0000 7f086da17640 1 --2- 172.21.15.67:0/1434528358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc57b9520 0x557dc5b5f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.060+0000 7f086da17640 1 -- 172.21.15.67:0/1434528358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5798840 msgr2=0x557dc5b5faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.060+0000 7f086da17640 1 --2- 172.21.15.67:0/1434528358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5798840 0x557dc5b5faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.060+0000 7f086da17640 1 -- 172.21.15.67:0/1434528358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dc5a91040 con 0x557dc5790660 2024-07-31T15:08:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.060+0000 7f086d216640 1 --2- 172.21.15.67:0/1434528358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5798840 0x557dc5b5faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.060+0000 7f086e218640 1 --2- 172.21.15.67:0/1434528358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc57b9520 0x557dc5b5f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.060+0000 7f085effd640 1 --2- 172.21.15.67:0/3215257598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f08540b3d40 0x7f08540b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.060+0000 7f086da17640 1 --2- 172.21.15.67:0/1434528358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc5790660 0x557dc5b5f5b0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0864101ec0 tx=0x7f0864108e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08935899e11881b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.060+0000 7f086e218640 1 --2- 172.21.15.67:0/3215257598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f08540b3d40 0x7f08540b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.060+0000 7f0853fff640 1 -- 172.21.15.67:0/1434528358 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08640c7380 con 0x557dc5790660 2024-07-31T15:08:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.060+0000 7f085effd640 1 -- 172.21.15.67:0/3215257598 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f0864104b60 con 0x7f08681035e0 2024-07-31T15:08:50.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.060+0000 7f0853fff640 1 -- 172.21.15.67:0/1434528358 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08640cafd0 con 0x557dc5790660 2024-07-31T15:08:50.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.060+0000 7f0853fff640 1 -- 172.21.15.67:0/1434528358 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08640cac30 con 0x557dc5790660 2024-07-31T15:08:50.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.061+0000 7f08761cc740 1 -- 172.21.15.67:0/1434528358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc5790660 msgr2=0x557dc5b5f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:50.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.061+0000 7f08761cc740 1 --2- 172.21.15.67:0/1434528358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc5790660 0x557dc5b5f5b0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0864101ec0 tx=0x7f0864108e30 comp rx=0 tx=0).stop 2024-07-31T15:08:50.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.061+0000 7f086e218640 1 --2- 172.21.15.67:0/3215257598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f08540b3d40 0x7f08540b6250 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0860005fd0 tx=0x7f08600096a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:50.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.061+0000 7f08761cc740 1 -- 172.21.15.67:0/1434528358 shutdown_connections 2024-07-31T15:08:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.061+0000 7f08761cc740 1 --2- 172.21.15.67:0/1434528358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5798840 0x557dc5b5faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.061+0000 7f08761cc740 1 --2- 172.21.15.67:0/1434528358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc5790660 0x557dc5b5f5b0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.061+0000 7f08761cc740 1 --2- 172.21.15.67:0/1434528358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc57b9520 0x557dc5b5f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.061+0000 7f08761cc740 1 -- 172.21.15.67:0/1434528358 >> 172.21.15.67:0/1434528358 conn(0x557dc57ad910 msgr2=0x557dc5b5cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.061+0000 7f08761cc740 1 -- 172.21.15.67:0/1434528358 shutdown_connections 2024-07-31T15:08:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.061+0000 7f08761cc740 1 -- 172.21.15.67:0/1434528358 wait complete. 2024-07-31T15:08:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.061+0000 7f08761cc740 1 -- start start 2024-07-31T15:08:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f08761cc740 1 Processor -- start 2024-07-31T15:08:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f08761cc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc5790660 0x557dc5b734d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f08761cc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5798840 0x557dc5b73a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f08761cc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc57b9520 0x557dc5b7eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f08761cc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557dc5927bb0 con 0x557dc57b9520 2024-07-31T15:08:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f08761cc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557dc5a66ac0 con 0x557dc5798840 2024-07-31T15:08:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f08761cc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557dc5b75d40 con 0x557dc5790660 2024-07-31T15:08:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f08761cc740 10 client.0 _ll_register_callbacks cb 0x557dc5a6f3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f086e218640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc57b9520 0x557dc5b7eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f086e218640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc57b9520 0x557dc5b7eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36788/0 (socket says 172.21.15.67:36788) 2024-07-31T15:08:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f086e218640 1 -- 172.21.15.67:0/3887851944 learned_addr learned my addr 172.21.15.67:0/3887851944 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f086d216640 1 --2- 172.21.15.67:0/3887851944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5798840 0x557dc5b73a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f086da17640 1 --2- 172.21.15.67:0/3887851944 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc5790660 0x557dc5b734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f086e218640 1 -- 172.21.15.67:0/3887851944 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc5790660 msgr2=0x557dc5b734d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f086e218640 1 --2- 172.21.15.67:0/3887851944 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc5790660 0x557dc5b734d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f086e218640 1 -- 172.21.15.67:0/3887851944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5798840 msgr2=0x557dc5b73a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f086e218640 1 --2- 172.21.15.67:0/3887851944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5798840 0x557dc5b73a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f086e218640 1 -- 172.21.15.67:0/3887851944 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dc5824b00 con 0x557dc57b9520 2024-07-31T15:08:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.062+0000 7f086da17640 1 --2- 172.21.15.67:0/3887851944 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc5790660 0x557dc5b734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:50.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.063+0000 7f086e218640 1 --2- 172.21.15.67:0/3887851944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc57b9520 0x557dc5b7eff0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f0860009fd0 tx=0x7f0860004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be862ea1bcc0c92 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:50.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.063+0000 7f08517fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:08:50.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.063+0000 7f08517fa640 1 -- 172.21.15.67:0/3887851944 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086002c060 con 0x557dc57b9520 2024-07-31T15:08:50.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.063+0000 7f08517fa640 1 -- 172.21.15.67:0/3887851944 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f0860017030 con 0x557dc57b9520 2024-07-31T15:08:50.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.063+0000 7f08761cc740 1 -- 172.21.15.67:0/3887851944 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08640cafd0 con 0x557dc57b9520 2024-07-31T15:08:50.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.063+0000 7f08517fa640 1 -- 172.21.15.67:0/3887851944 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f0860033050 con 0x557dc57b9520 2024-07-31T15:08:50.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.063+0000 7f08761cc740 10 client.19221 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:50.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.063+0000 7f08761cc740 1 -- 172.21.15.67:0/3887851944 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x557dc589b960 con 0x557dc57b9520 2024-07-31T15:08:50.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.063+0000 7f08517fa640 1 -- 172.21.15.67:0/3887851944 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08640cafd0 con 0x557dc57b9520 2024-07-31T15:08:50.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.063+0000 7f08517fa640 1 -- 172.21.15.67:0/3887851944 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f0860027a70 con 0x557dc57b9520 2024-07-31T15:08:50.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.064+0000 7f08761cc740 10 client.19221 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:50.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.064+0000 7f08761cc740 10 client.19221 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.064+0000 7f08761cc740 20 client.19221 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.064+0000 7f08761cc740 1 --2- 172.21.15.67:0/3887851944 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557dc5777ea0 0x557dc5b85120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.064+0000 7f08761cc740 4 client.19221 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.064+0000 7f08761cc740 1 -- 172.21.15.67:0/3887851944 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dc5ad6dc0 con 0x557dc5777ea0 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.064+0000 7f086da17640 1 --2- 172.21.15.67:0/3887851944 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557dc5777ea0 0x557dc5b85120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.064+0000 7f08517fa640 10 client.19221 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.064+0000 7f086da17640 1 --2- 172.21.15.67:0/3887851944 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557dc5777ea0 0x557dc5b85120 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5bd3c091e6f6155 server_cookie=fbf2866e6ee93c03 in_seq=0 out_seq=0 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.065+0000 7f08517fa640 1 -- 172.21.15.67:0/3887851944 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557dc5ad6dc0 con 0x557dc5777ea0 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.065+0000 7f08517fa640 10 client.19221 handle_command_reply: tid=0 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.065+0000 7f08761cc740 1 client.19221 shutdown 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.065+0000 7f08761cc740 1 -- 172.21.15.67:0/3887851944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc57b9520 msgr2=0x557dc5b7eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.065+0000 7f08761cc740 1 --2- 172.21.15.67:0/3887851944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc57b9520 0x557dc5b7eff0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f0860009fd0 tx=0x7f0860004ab0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.065+0000 7f08761cc740 1 -- 172.21.15.67:0/3887851944 shutdown_connections 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.065+0000 7f08761cc740 1 --2- 172.21.15.67:0/3887851944 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557dc5777ea0 0x557dc5b85120 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.065+0000 7f08761cc740 1 --2- 172.21.15.67:0/3887851944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc57b9520 0x557dc5b7eff0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.065+0000 7f08761cc740 1 --2- 172.21.15.67:0/3887851944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5798840 0x557dc5b73a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.065+0000 7f08761cc740 1 --2- 172.21.15.67:0/3887851944 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc5790660 0x557dc5b734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.065+0000 7f08761cc740 1 -- 172.21.15.67:0/3887851944 >> 172.21.15.67:0/3887851944 conn(0x557dc57ad910 msgr2=0x557dc5b5b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.065+0000 7f08517fa640 0 client.19221 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.065+0000 7f08761cc740 1 -- 172.21.15.67:0/3887851944 shutdown_connections 2024-07-31T15:08:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.065+0000 7f08761cc740 1 -- 172.21.15.67:0/3887851944 wait complete. 2024-07-31T15:08:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.066+0000 7f08761cc740 20 client.19221 trim_cache size 0 max 16384 2024-07-31T15:08:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.083+0000 7f08761cc740 1 Processor -- start 2024-07-31T15:08:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.083+0000 7f08761cc740 1 -- start start 2024-07-31T15:08:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f08761cc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc57ad910 0x557dc5b607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f08761cc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5790660 0x557dc5b8a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f08761cc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc5798840 0x557dc5b8ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f08761cc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f08681ae6e0 con 0x557dc57ad910 2024-07-31T15:08:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f08761cc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f08681b0220 con 0x557dc5790660 2024-07-31T15:08:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f08761cc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08681aedb0 con 0x557dc5798840 2024-07-31T15:08:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f086da17640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc57ad910 0x557dc5b607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f086da17640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc57ad910 0x557dc5b607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36802/0 (socket says 172.21.15.67:36802) 2024-07-31T15:08:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f086da17640 1 -- 172.21.15.67:0/3753508728 learned_addr learned my addr 172.21.15.67:0/3753508728 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f086d216640 1 --2- 172.21.15.67:0/3753508728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5790660 0x557dc5b8a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f086e218640 1 --2- 172.21.15.67:0/3753508728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc5798840 0x557dc5b8ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f086da17640 1 -- 172.21.15.67:0/3753508728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc5798840 msgr2=0x557dc5b8ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f086da17640 1 --2- 172.21.15.67:0/3753508728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc5798840 0x557dc5b8ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f086da17640 1 -- 172.21.15.67:0/3753508728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5790660 msgr2=0x557dc5b8a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f086da17640 1 --2- 172.21.15.67:0/3753508728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5790660 0x557dc5b8a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f086da17640 1 -- 172.21.15.67:0/3753508728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dc5b61360 con 0x557dc57ad910 2024-07-31T15:08:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f086e218640 1 --2- 172.21.15.67:0/3753508728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc5798840 0x557dc5b8ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:08:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f086d216640 1 --2- 172.21.15.67:0/3753508728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5790660 0x557dc5b8a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f086da17640 1 --2- 172.21.15.67:0/3753508728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc57ad910 0x557dc5b607b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0864102000 tx=0x7f0864104e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767dad78101a4e79 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.084+0000 7f08537fe640 1 -- 172.21.15.67:0/3753508728 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864049070 con 0x557dc57ad910 2024-07-31T15:08:50.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.085+0000 7f08537fe640 1 -- 172.21.15.67:0/3753508728 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08640cafd0 con 0x557dc57ad910 2024-07-31T15:08:50.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.085+0000 7f08537fe640 1 -- 172.21.15.67:0/3753508728 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08640cae10 con 0x557dc57ad910 2024-07-31T15:08:50.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.085+0000 7f08761cc740 1 -- 172.21.15.67:0/3753508728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc57ad910 msgr2=0x557dc5b607b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.085+0000 7f08761cc740 1 --2- 172.21.15.67:0/3753508728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc57ad910 0x557dc5b607b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0864102000 tx=0x7f0864104e60 comp rx=0 tx=0).stop 2024-07-31T15:08:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.085+0000 7f08761cc740 1 -- 172.21.15.67:0/3753508728 shutdown_connections 2024-07-31T15:08:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.085+0000 7f08761cc740 1 --2- 172.21.15.67:0/3753508728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc5798840 0x557dc5b8ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.085+0000 7f08761cc740 1 --2- 172.21.15.67:0/3753508728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5790660 0x557dc5b8a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.085+0000 7f08761cc740 1 --2- 172.21.15.67:0/3753508728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc57ad910 0x557dc5b607b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.085+0000 7f08761cc740 1 -- 172.21.15.67:0/3753508728 >> 172.21.15.67:0/3753508728 conn(0x557dc5b85470 msgr2=0x557dc5b600a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.085+0000 7f08761cc740 1 -- 172.21.15.67:0/3753508728 shutdown_connections 2024-07-31T15:08:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.085+0000 7f08761cc740 1 -- 172.21.15.67:0/3753508728 wait complete. 2024-07-31T15:08:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.086+0000 7f08761cc740 1 -- start start 2024-07-31T15:08:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.086+0000 7f08761cc740 1 Processor -- start 2024-07-31T15:08:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.086+0000 7f08761cc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5790660 0x557dc5b770f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.086+0000 7f08761cc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc5798840 0x557dc5b77630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.086+0000 7f08761cc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc57ad910 0x557dc5b77bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.086+0000 7f08761cc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f08681afb50 con 0x557dc5798840 2024-07-31T15:08:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.086+0000 7f08761cc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f08681af480 con 0x557dc5790660 2024-07-31T15:08:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.086+0000 7f08761cc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08681ac700 con 0x557dc57ad910 2024-07-31T15:08:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.086+0000 7f08761cc740 10 client.0 _ll_register_callbacks cb 0x7f0860009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.086+0000 7f086d216640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc5798840 0x557dc5b77630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.086+0000 7f086d216640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc5798840 0x557dc5b77630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36814/0 (socket says 172.21.15.67:36814) 2024-07-31T15:08:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.086+0000 7f086d216640 1 -- 172.21.15.67:0/1010714894 learned_addr learned my addr 172.21.15.67:0/1010714894 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.086+0000 7f086da17640 1 --2- 172.21.15.67:0/1010714894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5790660 0x557dc5b770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.086+0000 7f086e218640 1 --2- 172.21.15.67:0/1010714894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc57ad910 0x557dc5b77bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.087+0000 7f086d216640 1 -- 172.21.15.67:0/1010714894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc57ad910 msgr2=0x557dc5b77bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.087+0000 7f086d216640 1 --2- 172.21.15.67:0/1010714894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc57ad910 0x557dc5b77bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.087+0000 7f086d216640 1 -- 172.21.15.67:0/1010714894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5790660 msgr2=0x557dc5b770f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.087+0000 7f086d216640 1 --2- 172.21.15.67:0/1010714894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5790660 0x557dc5b770f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.087+0000 7f086d216640 1 -- 172.21.15.67:0/1010714894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dc5becb50 con 0x557dc5798840 2024-07-31T15:08:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.087+0000 7f086da17640 1 --2- 172.21.15.67:0/1010714894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5790660 0x557dc5b770f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.087+0000 7f086d216640 1 --2- 172.21.15.67:0/1010714894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc5798840 0x557dc5b77630 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f085800b9f0 tx=0x7f085800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b779fd4375df8967 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.087+0000 7f08517fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:08:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.087+0000 7f08517fa640 1 -- 172.21.15.67:0/1010714894 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858005220 con 0x557dc5798840 2024-07-31T15:08:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.087+0000 7f08517fa640 1 -- 172.21.15.67:0/1010714894 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f08580053c0 con 0x557dc5798840 2024-07-31T15:08:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.087+0000 7f08761cc740 1 -- 172.21.15.67:0/1010714894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08640cafd0 con 0x557dc5798840 2024-07-31T15:08:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.087+0000 7f08517fa640 1 -- 172.21.15.67:0/1010714894 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f0858027050 con 0x557dc5798840 2024-07-31T15:08:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.087+0000 7f08761cc740 10 client.19227 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.087+0000 7f08761cc740 1 -- 172.21.15.67:0/1010714894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x557dc5bedde0 con 0x557dc5798840 2024-07-31T15:08:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.088+0000 7f08517fa640 1 -- 172.21.15.67:0/1010714894 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08640cafd0 con 0x557dc5798840 2024-07-31T15:08:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.088+0000 7f08517fa640 1 -- 172.21.15.67:0/1010714894 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f085801b070 con 0x557dc5798840 2024-07-31T15:08:50.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.088+0000 7f08761cc740 10 client.19227 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:50.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.088+0000 7f08761cc740 10 client.19227 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:08:50.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.088+0000 7f08761cc740 20 client.19227 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:50.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.088+0000 7f08761cc740 1 --2- 172.21.15.67:0/1010714894 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557dc57b9520 0x557dc5bf0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:50.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.088+0000 7f08761cc740 4 client.19227 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8bcd794b", "release": true, "await": true}] 2024-07-31T15:08:50.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.088+0000 7f08761cc740 1 -- 172.21.15.67:0/1010714894 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8bcd794b", "release": true, "await": true}) -- 0x557dc5ad2860 con 0x557dc57b9520 2024-07-31T15:08:50.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.088+0000 7f086da17640 1 --2- 172.21.15.67:0/1010714894 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557dc57b9520 0x557dc5bf0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:50.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.089+0000 7f08517fa640 10 client.19227 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:50.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:50.089+0000 7f086da17640 1 --2- 172.21.15.67:0/1010714894 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557dc57b9520 0x557dc5bf0650 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28baaf114d8cfafd server_cookie=de35561b067e2939 in_seq=0 out_seq=0 2024-07-31T15:08:50.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:50 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:50.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:50 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:50.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:50 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:08:50.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:50 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:50.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:50 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:50.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:50 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:50.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:50 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:08:50.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:50 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:50.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:50 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:50.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:50 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:50.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:50 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:08:50.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:50 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:08:51.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:51 smithi067 ceph-mon[32750]: pgmap v964: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 996 KiB/s wr, 3.58k op/s 2024-07-31T15:08:51.638 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:51 smithi077 ceph-mon[37327]: pgmap v964: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 996 KiB/s wr, 3.58k op/s 2024-07-31T15:08:51.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:51 smithi073 ceph-mon[36714]: pgmap v964: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 996 KiB/s wr, 3.58k op/s 2024-07-31T15:08:52.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.520+0000 7f08517fa640 1 -- 172.21.15.67:0/1010714894 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557dc5ad2860 con 0x557dc57b9520 2024-07-31T15:08:52.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.520+0000 7f08517fa640 10 client.19227 handle_command_reply: tid=0 2024-07-31T15:08:52.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.520+0000 7f08761cc740 1 client.19227 shutdown 2024-07-31T15:08:52.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.520+0000 7f08761cc740 1 -- 172.21.15.67:0/1010714894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc5798840 msgr2=0x557dc5b77630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:52.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.520+0000 7f08761cc740 1 --2- 172.21.15.67:0/1010714894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc5798840 0x557dc5b77630 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f085800b9f0 tx=0x7f085800beb0 comp rx=0 tx=0).stop 2024-07-31T15:08:52.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.520+0000 7f08761cc740 1 -- 172.21.15.67:0/1010714894 shutdown_connections 2024-07-31T15:08:52.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.520+0000 7f08761cc740 1 --2- 172.21.15.67:0/1010714894 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557dc57b9520 0x557dc5bf0650 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:52.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.520+0000 7f08761cc740 1 --2- 172.21.15.67:0/1010714894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dc57ad910 0x557dc5b77bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:52.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.520+0000 7f08517fa640 0 client.19227 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:52.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.520+0000 7f08761cc740 1 --2- 172.21.15.67:0/1010714894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557dc5798840 0x557dc5b77630 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:52.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.520+0000 7f08761cc740 1 --2- 172.21.15.67:0/1010714894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557dc5790660 0x557dc5b770f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:52.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.520+0000 7f08761cc740 1 -- 172.21.15.67:0/1010714894 >> 172.21.15.67:0/1010714894 conn(0x557dc5b85470 msgr2=0x557dc5b71750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:52.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.521+0000 7f08761cc740 1 -- 172.21.15.67:0/1010714894 shutdown_connections 2024-07-31T15:08:52.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.521+0000 7f08761cc740 1 -- 172.21.15.67:0/1010714894 wait complete. 2024-07-31T15:08:52.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.521+0000 7f08761cc740 20 client.19227 trim_cache size 0 max 16384 2024-07-31T15:08:52.525 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:08:52.525 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-07-31T15:08:52.525 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:08:52.525 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 48, 2024-07-31T15:08:52.525 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:08:52.525 INFO:teuthology.orchestra.run.smithi067.stdout: "8bcd794b": { 2024-07-31T15:08:52.525 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 48, 2024-07-31T15:08:52.525 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:08:52.525 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:08:52.525 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:08:52.525 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:08:52.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:08:52.525 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:08:52.526 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 130.0, 2024-07-31T15:08:52.526 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:08:52.526 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:08:52.526 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:08:52.526 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:08:52.526 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:08:52.526 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:08:52.526 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:08:52.526 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:08:52.526 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:08:52.526 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:08:52.526 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:08:52.527 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:08:52.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.522+0000 7f086fc7b640 1 -- 172.21.15.67:0/3215257598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f08540b3d40 msgr2=0x7f08540b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:52.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.522+0000 7f086fc7b640 1 --2- 172.21.15.67:0/3215257598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f08540b3d40 0x7f08540b6250 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0860005fd0 tx=0x7f08600096a0 comp rx=0 tx=0).stop 2024-07-31T15:08:52.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.522+0000 7f086fc7b640 1 -- 172.21.15.67:0/3215257598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08681035e0 msgr2=0x7f0868075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:52.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.522+0000 7f086fc7b640 1 --2- 172.21.15.67:0/3215257598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08681035e0 0x7f0868075140 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0864005940 tx=0x7f08640042f0 comp rx=0 tx=0).stop 2024-07-31T15:08:52.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.522+0000 7f086fc7b640 1 -- 172.21.15.67:0/3215257598 shutdown_connections 2024-07-31T15:08:52.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.522+0000 7f086fc7b640 1 --2- 172.21.15.67:0/3215257598 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f08540b3d40 0x7f08540b6250 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:52.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.522+0000 7f086fc7b640 1 --2- 172.21.15.67:0/3215257598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0868109870 0x7f0868075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:52.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.522+0000 7f086fc7b640 1 --2- 172.21.15.67:0/3215257598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0868108f20 0x7f0868075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:52.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.522+0000 7f086fc7b640 1 --2- 172.21.15.67:0/3215257598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08681035e0 0x7f0868075140 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:52.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.522+0000 7f086fc7b640 1 -- 172.21.15.67:0/3215257598 >> 172.21.15.67:0/3215257598 conn(0x7f08680ff030 msgr2=0x7f0868101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:52.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.522+0000 7f086fc7b640 1 -- 172.21.15.67:0/3215257598 shutdown_connections 2024-07-31T15:08:52.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:08:52.522+0000 7f086fc7b640 1 -- 172.21.15.67:0/3215257598 wait complete. 2024-07-31T15:08:52.600 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/unix/scm.o 2024-07-31T15:08:52.600 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/inet_timewait_sock.o 2024-07-31T15:08:52.601 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/xfrm/xfrm_output.o 2024-07-31T15:08:52.601 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/inet_connection_sock.o 2024-07-31T15:08:52.603 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_proto_icmp.o 2024-07-31T15:08:52.603 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/driver.o 2024-07-31T15:08:52.604 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/inet_hashtables.o 2024-07-31T15:08:52.796 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/intel_sbi.o 2024-07-31T15:08:53.049 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/class.o 2024-07-31T15:08:53.302 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:53 smithi067 ceph-mon[32750]: pgmap v965: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 793 KiB/s wr, 4.18k op/s 2024-07-31T15:08:53.377 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8bcd794b', total seconds elapsed: 43.1 2024-07-31T15:08:53.472 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/unix/built-in.a 2024-07-31T15:08:53.586 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/intel_step.o 2024-07-31T15:08:53.589 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_extend.o 2024-07-31T15:08:53.601 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/netfilter/ip6_tables.o 2024-07-31T15:08:53.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:53 smithi073 ceph-mon[36714]: pgmap v965: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 793 KiB/s wr, 4.18k op/s 2024-07-31T15:08:53.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:53 smithi077 ceph-mon[37327]: pgmap v965: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 793 KiB/s wr, 4.18k op/s 2024-07-31T15:08:53.745 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/platform.o 2024-07-31T15:08:53.781 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/xfrm/xfrm_sysctl.o 2024-07-31T15:08:53.791 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/cpu.o 2024-07-31T15:08:54.209 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/intel_uncore.o 2024-07-31T15:08:54.235 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp.o 2024-07-31T15:08:54.427 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/intel_wakeref.o 2024-07-31T15:08:54.572 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_acct.o 2024-07-31T15:08:54.582 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/packet/af_packet.o 2024-07-31T15:08:54.630 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/xfrm/xfrm_replay.o 2024-07-31T15:08:54.730 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/firmware.o 2024-07-31T15:08:54.995 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/netfilter/ip6table_filter.o 2024-07-31T15:08:55.100 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/init.o 2024-07-31T15:08:55.270 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/vlv_sideband.o 2024-07-31T15:08:55.444 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_seqadj.o 2024-07-31T15:08:55.636 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/map.o 2024-07-31T15:08:55.639 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/xfrm/xfrm_device.o 2024-07-31T15:08:55.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:55 smithi073 ceph-mon[36714]: pgmap v966: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 706 KiB/s wr, 4.27k op/s 2024-07-31T15:08:55.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:55 smithi077 ceph-mon[37327]: pgmap v966: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 706 KiB/s wr, 4.27k op/s 2024-07-31T15:08:55.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:55 smithi067 ceph-mon[32750]: pgmap v966: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 706 KiB/s wr, 4.27k op/s 2024-07-31T15:08:55.983 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/netfilter/ip6table_mangle.o 2024-07-31T15:08:56.070 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/devres.o 2024-07-31T15:08:56.125 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/netfilter/nf_defrag_ipv6_hooks.o 2024-07-31T15:08:56.323 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/vlv_suspend.o 2024-07-31T15:08:56.491 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/xfrm/xfrm_algo.o 2024-07-31T15:08:56.516 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_proto_icmpv6.o 2024-07-31T15:08:56.763 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/attribute_container.o 2024-07-31T15:08:56.980 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp_input.o 2024-07-31T15:08:57.017 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp_output.o 2024-07-31T15:08:57.058 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/packet/built-in.a 2024-07-31T15:08:57.101 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/dsa/built-in.a 2024-07-31T15:08:57.150 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/netfilter/nf_conntrack_reasm.o 2024-07-31T15:08:57.161 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_gss/auth_gss.o 2024-07-31T15:08:57.240 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/transport_class.o 2024-07-31T15:08:57.298 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/soc/intel_dram.o 2024-07-31T15:08:57.426 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/xfrm/xfrm_user.o 2024-07-31T15:08:57.622 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_netlink.o 2024-07-31T15:08:57.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:57 smithi073 ceph-mon[36714]: pgmap v967: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 389 KiB/s wr, 3.91k op/s 2024-07-31T15:08:57.655 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/topology.o 2024-07-31T15:08:57.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:57 smithi077 ceph-mon[37327]: pgmap v967: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 389 KiB/s wr, 3.91k op/s 2024-07-31T15:08:57.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:57 smithi067 ceph-mon[32750]: pgmap v967: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 389 KiB/s wr, 3.91k op/s 2024-07-31T15:08:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.232+0000 7f933d406640 1 -- 172.21.15.67:0/3655335140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f933810a330 msgr2=0x7f933810a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.232+0000 7f933d406640 1 --2- 172.21.15.67:0/3655335140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f933810a330 0x7f933810a740 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9330009ca0 tx=0x7f933002a980 comp rx=0 tx=0).stop 2024-07-31T15:08:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.234+0000 7f933d406640 1 -- 172.21.15.67:0/3655335140 shutdown_connections 2024-07-31T15:08:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.234+0000 7f933d406640 1 --2- 172.21.15.67:0/3655335140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f933811c5d0 0x7f933811eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.234+0000 7f933d406640 1 --2- 172.21.15.67:0/3655335140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f933810a330 0x7f933810a740 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.234+0000 7f933d406640 1 --2- 172.21.15.67:0/3655335140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93381118d0 0x7f9338109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.234+0000 7f933d406640 1 -- 172.21.15.67:0/3655335140 >> 172.21.15.67:0/3655335140 conn(0x7f93380707c0 msgr2=0x7f9338070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.240+0000 7f933d406640 1 -- 172.21.15.67:0/3655335140 shutdown_connections 2024-07-31T15:08:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.240+0000 7f933d406640 1 -- 172.21.15.67:0/3655335140 wait complete. 2024-07-31T15:08:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.241+0000 7f933d406640 1 Processor -- start 2024-07-31T15:08:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.241+0000 7f933d406640 1 -- start start 2024-07-31T15:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.241+0000 7f933d406640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f933810a330 0x7f93381a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.241+0000 7f933d406640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93381118d0 0x7f93381a9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.241+0000 7f933d406640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f933811c5d0 0x7f93381ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.241+0000 7f933d406640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9338122360 con 0x7f933811c5d0 2024-07-31T15:08:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.241+0000 7f933d406640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93381221e0 con 0x7f933810a330 2024-07-31T15:08:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.241+0000 7f933d406640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93381224e0 con 0x7f93381118d0 2024-07-31T15:08:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.242+0000 7f9336ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f933810a330 0x7f93381a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.242+0000 7f9336ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f933810a330 0x7f93381a8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35798/0 (socket says 172.21.15.67:35798) 2024-07-31T15:08:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.242+0000 7f9336ffd640 1 -- 172.21.15.67:0/2624989241 learned_addr learned my addr 172.21.15.67:0/2624989241 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.242+0000 7f93377fe640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f933811c5d0 0x7f93381ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.242+0000 7f93367fc640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93381118d0 0x7f93381a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.242+0000 7f9336ffd640 1 -- 172.21.15.67:0/2624989241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93381118d0 msgr2=0x7f93381a9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.242+0000 7f9336ffd640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93381118d0 0x7f93381a9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.242+0000 7f9336ffd640 1 -- 172.21.15.67:0/2624989241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f933811c5d0 msgr2=0x7f93381ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.242+0000 7f9336ffd640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f933811c5d0 0x7f93381ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.242+0000 7f9336ffd640 1 -- 172.21.15.67:0/2624989241 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93381aed10 con 0x7f933810a330 2024-07-31T15:08:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.243+0000 7f93377fe640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f933811c5d0 0x7f93381ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.243+0000 7f93367fc640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93381118d0 0x7f93381a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:08:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.243+0000 7f9336ffd640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f933810a330 0x7f93381a8de0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f932800b4a0 tx=0x7f932800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.248+0000 7f931ffff640 1 -- 172.21.15.67:0/2624989241 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328014070 con 0x7f933810a330 2024-07-31T15:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.248+0000 7f933d406640 1 -- 172.21.15.67:0/2624989241 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93381af000 con 0x7f933810a330 2024-07-31T15:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.248+0000 7f933d406640 1 -- 172.21.15.67:0/2624989241 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93381b5790 con 0x7f933810a330 2024-07-31T15:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.248+0000 7f931ffff640 1 -- 172.21.15.67:0/2624989241 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9328010ac0 con 0x7f933810a330 2024-07-31T15:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.248+0000 7f931ffff640 1 -- 172.21.15.67:0/2624989241 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328017be0 con 0x7f933810a330 2024-07-31T15:08:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.252+0000 7f931ffff640 1 -- 172.21.15.67:0/2624989241 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f9328017070 con 0x7f933810a330 2024-07-31T15:08:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.253+0000 7f93443d4740 1 Processor -- start 2024-07-31T15:08:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.254+0000 7f931ffff640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93200b3f50 0x7f93200b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.254+0000 7f931ffff640 1 -- 172.21.15.67:0/2624989241 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f93280e4510 con 0x7f933810a330 2024-07-31T15:08:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.254+0000 7f93377fe640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93200b3f50 0x7f93200b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.255+0000 7f93443d4740 1 -- start start 2024-07-31T15:08:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.255+0000 7f93443d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491681b0 0x555a4950ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.255+0000 7f93443d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a4950b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.255+0000 7f93443d4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491474d0 0x555a4950b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.255+0000 7f93443d4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555a494803b0 con 0x555a491474d0 2024-07-31T15:08:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.255+0000 7f93443d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555a4946f8b0 con 0x555a491681b0 2024-07-31T15:08:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.255+0000 7f93443d4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555a494738f0 con 0x555a4913f2f0 2024-07-31T15:08:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.255+0000 7f93377fe640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93200b3f50 0x7f93200b6460 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f932c006fd0 tx=0x7f932c009060 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.256+0000 7f9336ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491681b0 0x555a4950ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.256+0000 7f9336ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491681b0 0x555a4950ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35808/0 (socket says 172.21.15.67:35808) 2024-07-31T15:08:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.256+0000 7f9336ffd640 1 -- 172.21.15.67:0/4288440456 learned_addr learned my addr 172.21.15.67:0/4288440456 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.256+0000 7f93367fc640 1 --2- 172.21.15.67:0/4288440456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a4950b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.256+0000 7f9336ffd640 1 -- 172.21.15.67:0/4288440456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 msgr2=0x555a4950b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.256+0000 7f9336ffd640 1 --2- 172.21.15.67:0/4288440456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a4950b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.256+0000 7f9336ffd640 1 -- 172.21.15.67:0/4288440456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491474d0 msgr2=0x555a4950b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.256+0000 7f9336ffd640 1 --2- 172.21.15.67:0/4288440456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491474d0 0x555a4950b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.256+0000 7f9336ffd640 1 -- 172.21.15.67:0/4288440456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a494169a0 con 0x555a491681b0 2024-07-31T15:08:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.256+0000 7f9336ffd640 1 --2- 172.21.15.67:0/4288440456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491681b0 0x555a4950ace0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f93280a80e0 tx=0x7f93280047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5729f2c5a95cdf3e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.258+0000 7f931d7fa640 1 -- 172.21.15.67:0/4288440456 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932801e040 con 0x555a491681b0 2024-07-31T15:08:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.258+0000 7f931d7fa640 1 -- 172.21.15.67:0/4288440456 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93280af470 con 0x555a491681b0 2024-07-31T15:08:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.258+0000 7f931d7fa640 1 -- 172.21.15.67:0/4288440456 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328017360 con 0x555a491681b0 2024-07-31T15:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.259+0000 7f93443d4740 1 -- 172.21.15.67:0/4288440456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491681b0 msgr2=0x555a4950ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.259+0000 7f93443d4740 1 --2- 172.21.15.67:0/4288440456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491681b0 0x555a4950ace0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f93280a80e0 tx=0x7f93280047d0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.259+0000 7f93443d4740 1 -- 172.21.15.67:0/4288440456 shutdown_connections 2024-07-31T15:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.259+0000 7f93443d4740 1 --2- 172.21.15.67:0/4288440456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491474d0 0x555a4950b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.259+0000 7f93443d4740 1 --2- 172.21.15.67:0/4288440456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a4950b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.259+0000 7f93443d4740 1 --2- 172.21.15.67:0/4288440456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491681b0 0x555a4950ace0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.259+0000 7f93443d4740 1 -- 172.21.15.67:0/4288440456 >> 172.21.15.67:0/4288440456 conn(0x555a4915c300 msgr2=0x555a49508bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:58.263 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/container.o 2024-07-31T15:08:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.264+0000 7f93443d4740 1 -- 172.21.15.67:0/4288440456 shutdown_connections 2024-07-31T15:08:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.264+0000 7f93443d4740 1 -- 172.21.15.67:0/4288440456 wait complete. 2024-07-31T15:08:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.265+0000 7f93443d4740 1 -- start start 2024-07-31T15:08:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.265+0000 7f93443d4740 1 Processor -- start 2024-07-31T15:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.265+0000 7f93443d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a49516860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.265+0000 7f93443d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491474d0 0x555a49516da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.265+0000 7f93443d4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491681b0 0x555a49522800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.265+0000 7f93443d4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555a493a9ab0 con 0x555a491681b0 2024-07-31T15:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.265+0000 7f93443d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555a493f7cf0 con 0x555a491474d0 2024-07-31T15:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.265+0000 7f93443d4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555a495194b0 con 0x555a4913f2f0 2024-07-31T15:08:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.265+0000 7f93443d4740 10 client.0 _ll_register_callbacks cb 0x555a494699f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.266+0000 7f93367fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491474d0 0x555a49516da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.266+0000 7f93367fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491474d0 0x555a49516da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35824/0 (socket says 172.21.15.67:35824) 2024-07-31T15:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.266+0000 7f93377fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491681b0 0x555a49522800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.266+0000 7f93367fc640 1 -- 172.21.15.67:0/961372439 learned_addr learned my addr 172.21.15.67:0/961372439 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.266+0000 7f9336ffd640 1 --2- 172.21.15.67:0/961372439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a49516860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.266+0000 7f93367fc640 1 -- 172.21.15.67:0/961372439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 msgr2=0x555a49516860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.266+0000 7f93367fc640 1 --2- 172.21.15.67:0/961372439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a49516860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.266+0000 7f93367fc640 1 -- 172.21.15.67:0/961372439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491681b0 msgr2=0x555a49522800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.266+0000 7f93367fc640 1 --2- 172.21.15.67:0/961372439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491681b0 0x555a49522800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.266+0000 7f93367fc640 1 -- 172.21.15.67:0/961372439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a491d3790 con 0x555a491474d0 2024-07-31T15:08:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.267+0000 7f93367fc640 1 --2- 172.21.15.67:0/961372439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491474d0 0x555a49516da0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9330009ca0 tx=0x7f9330003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d44c4d3c6997b33e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.267+0000 7f931aff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.267+0000 7f931aff5640 1 -- 172.21.15.67:0/961372439 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330034070 con 0x555a491474d0 2024-07-31T15:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.267+0000 7f93443d4740 1 -- 172.21.15.67:0/961372439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93280af470 con 0x555a491474d0 2024-07-31T15:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.267+0000 7f931aff5640 1 -- 172.21.15.67:0/961372439 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f933002f3d0 con 0x555a491474d0 2024-07-31T15:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.267+0000 7f931aff5640 1 -- 172.21.15.67:0/961372439 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f933004e050 con 0x555a491474d0 2024-07-31T15:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.267+0000 7f93443d4740 10 client.28781 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.267+0000 7f93443d4740 1 -- 172.21.15.67:0/961372439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x555a4924c620 con 0x555a491474d0 2024-07-31T15:08:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.268+0000 7f931aff5640 1 -- 172.21.15.67:0/961372439 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93280af470 con 0x555a491474d0 2024-07-31T15:08:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.272+0000 7f931aff5640 1 -- 172.21.15.67:0/961372439 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f9330004040 con 0x555a491474d0 2024-07-31T15:08:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.273+0000 7f93443d4740 10 client.28781 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.273+0000 7f93443d4740 10 client.28781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:08:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.273+0000 7f93443d4740 20 client.28781 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.273+0000 7f93443d4740 1 --2- 172.21.15.67:0/961372439 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555a49126d80 0x555a49539c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.273+0000 7f93443d4740 4 client.28781 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.273+0000 7f93443d4740 1 -- 172.21.15.67:0/961372439 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a49482300 con 0x555a49126d80 2024-07-31T15:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.273+0000 7f9336ffd640 1 --2- 172.21.15.67:0/961372439 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555a49126d80 0x555a49539c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.273+0000 7f931aff5640 10 client.28781 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.273+0000 7f9336ffd640 1 --2- 172.21.15.67:0/961372439 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555a49126d80 0x555a49539c80 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3549d44d2efbc0dd server_cookie=a9da5e05d73eb981 in_seq=0 out_seq=0 2024-07-31T15:08:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.274+0000 7f931aff5640 1 -- 172.21.15.67:0/961372439 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a49482300 con 0x555a49126d80 2024-07-31T15:08:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.274+0000 7f931aff5640 10 client.28781 handle_command_reply: tid=0 2024-07-31T15:08:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.274+0000 7f93443d4740 1 client.28781 shutdown 2024-07-31T15:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.274+0000 7f93443d4740 1 -- 172.21.15.67:0/961372439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491474d0 msgr2=0x555a49516da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.274+0000 7f93443d4740 1 --2- 172.21.15.67:0/961372439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491474d0 0x555a49516da0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9330009ca0 tx=0x7f9330003ba0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.274+0000 7f93443d4740 1 -- 172.21.15.67:0/961372439 shutdown_connections 2024-07-31T15:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.275+0000 7f93443d4740 1 --2- 172.21.15.67:0/961372439 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555a49126d80 0x555a49539c80 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.275+0000 7f93443d4740 1 --2- 172.21.15.67:0/961372439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491681b0 0x555a49522800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.275+0000 7f93443d4740 1 --2- 172.21.15.67:0/961372439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491474d0 0x555a49516da0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.275+0000 7f93443d4740 1 --2- 172.21.15.67:0/961372439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a49516860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.275+0000 7f93443d4740 1 -- 172.21.15.67:0/961372439 >> 172.21.15.67:0/961372439 conn(0x555a4915c300 msgr2=0x555a495072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.275+0000 7f931aff5640 0 client.28781 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.275+0000 7f93443d4740 1 -- 172.21.15.67:0/961372439 shutdown_connections 2024-07-31T15:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.275+0000 7f93443d4740 1 -- 172.21.15.67:0/961372439 wait complete. 2024-07-31T15:08:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.275+0000 7f93443d4740 20 client.28781 trim_cache size 0 max 16384 2024-07-31T15:08:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.308+0000 7f93443d4740 1 Processor -- start 2024-07-31T15:08:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.308+0000 7f93443d4740 1 -- start start 2024-07-31T15:08:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.308+0000 7f93443d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a495225f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.308+0000 7f93443d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491474d0 0x555a49516860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.308+0000 7f93443d4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491681b0 0x555a49516da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.308+0000 7f9336ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a495225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.308+0000 7f93367fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491474d0 0x555a49516860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.308+0000 7f9336ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a495225f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49646/0 (socket says 172.21.15.67:49646) 2024-07-31T15:08:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.308+0000 7f93367fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491474d0 0x555a49516860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35838/0 (socket says 172.21.15.67:35838) 2024-07-31T15:08:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.309+0000 7f93377fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491681b0 0x555a49516da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.309+0000 7f93377fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491681b0 0x555a49516da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51030/0 (socket says 172.21.15.67:51030) 2024-07-31T15:08:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.309+0000 7f9336ffd640 1 -- 172.21.15.67:0/4265735741 learned_addr learned my addr 172.21.15.67:0/4265735741 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.309+0000 7f93443d4740 1 -- 172.21.15.67:0/4265735741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f932400fd10 con 0x555a491681b0 2024-07-31T15:08:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.309+0000 7f93443d4740 1 -- 172.21.15.67:0/4265735741 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9324011850 con 0x555a491474d0 2024-07-31T15:08:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.309+0000 7f93443d4740 1 -- 172.21.15.67:0/4265735741 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93240103e0 con 0x555a4913f2f0 2024-07-31T15:08:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.309+0000 7f93367fc640 1 -- 172.21.15.67:0/4265735741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 msgr2=0x555a495225f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.309+0000 7f93367fc640 1 --2- 172.21.15.67:0/4265735741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a495225f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.309+0000 7f93367fc640 1 -- 172.21.15.67:0/4265735741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491681b0 msgr2=0x555a49516da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.309+0000 7f93367fc640 1 --2- 172.21.15.67:0/4265735741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491681b0 0x555a49516da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.309+0000 7f93367fc640 1 -- 172.21.15.67:0/4265735741 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a4950cfe0 con 0x555a491474d0 2024-07-31T15:08:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.309+0000 7f9336ffd640 1 --2- 172.21.15.67:0/4265735741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a495225f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.309+0000 7f93377fe640 1 --2- 172.21.15.67:0/4265735741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491681b0 0x555a49516da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.310+0000 7f93367fc640 1 --2- 172.21.15.67:0/4265735741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491474d0 0x555a49516860 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f933002af40 tx=0x7f933002a980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f229979748a4dae server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.312+0000 7f931cff9640 1 -- 172.21.15.67:0/4265735741 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933003e070 con 0x555a491474d0 2024-07-31T15:08:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.312+0000 7f931cff9640 1 -- 172.21.15.67:0/4265735741 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9330004680 con 0x555a491474d0 2024-07-31T15:08:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.314+0000 7f93443d4740 1 -- 172.21.15.67:0/4265735741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491474d0 msgr2=0x555a49516860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.314+0000 7f93443d4740 1 --2- 172.21.15.67:0/4265735741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491474d0 0x555a49516860 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f933002af40 tx=0x7f933002a980 comp rx=0 tx=0).stop 2024-07-31T15:08:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.318+0000 7f93443d4740 1 -- 172.21.15.67:0/4265735741 shutdown_connections 2024-07-31T15:08:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.318+0000 7f93443d4740 1 --2- 172.21.15.67:0/4265735741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491681b0 0x555a49516da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.318+0000 7f93443d4740 1 --2- 172.21.15.67:0/4265735741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491474d0 0x555a49516860 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.318+0000 7f93443d4740 1 --2- 172.21.15.67:0/4265735741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a495225f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.318+0000 7f93443d4740 1 -- 172.21.15.67:0/4265735741 >> 172.21.15.67:0/4265735741 conn(0x555a4915c300 msgr2=0x555a49511830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.319+0000 7f93443d4740 1 -- 172.21.15.67:0/4265735741 shutdown_connections 2024-07-31T15:08:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.319+0000 7f93443d4740 1 -- 172.21.15.67:0/4265735741 wait complete. 2024-07-31T15:08:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.319+0000 7f93443d4740 1 -- start start 2024-07-31T15:08:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.319+0000 7f93443d4740 1 Processor -- start 2024-07-31T15:08:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.320+0000 7f93443d4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a4951e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.320+0000 7f93443d4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491474d0 0x555a4951eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.320+0000 7f93443d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491681b0 0x555a4951b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.320+0000 7f93443d4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9324011180 con 0x555a491474d0 2024-07-31T15:08:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.320+0000 7f93443d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9324010ab0 con 0x555a491681b0 2024-07-31T15:08:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.320+0000 7f93443d4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f932400dda0 con 0x555a4913f2f0 2024-07-31T15:08:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.320+0000 7f93443d4740 10 client.0 _ll_register_callbacks cb 0x7f9330009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:08:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.320+0000 7f9336ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a4951e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.320+0000 7f9336ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a4951e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49656/0 (socket says 172.21.15.67:49656) 2024-07-31T15:08:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.320+0000 7f9336ffd640 1 -- 172.21.15.67:0/3624816151 learned_addr learned my addr 172.21.15.67:0/3624816151 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:08:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.320+0000 7f93367fc640 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491474d0 0x555a4951eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.321+0000 7f93377fe640 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491681b0 0x555a4951b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.321+0000 7f9336ffd640 1 -- 172.21.15.67:0/3624816151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491681b0 msgr2=0x555a4951b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.321+0000 7f9336ffd640 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491681b0 0x555a4951b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.321+0000 7f9336ffd640 1 -- 172.21.15.67:0/3624816151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491474d0 msgr2=0x555a4951eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:08:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.321+0000 7f9336ffd640 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491474d0 0x555a4951eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.321+0000 7f9336ffd640 1 -- 172.21.15.67:0/3624816151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a4951bc70 con 0x555a4913f2f0 2024-07-31T15:08:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.322+0000 7f93367fc640 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491474d0 0x555a4951eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:08:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.322+0000 7f93377fe640 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491681b0 0x555a4951b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:08:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.322+0000 7f9336ffd640 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a4951e760 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9328017780 tx=0x7f93280ab5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a0352821830b8b0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:08:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.322+0000 7f931aff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:08:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.322+0000 7f931aff5640 1 -- 172.21.15.67:0/3624816151 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280bf070 con 0x555a4913f2f0 2024-07-31T15:08:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.323+0000 7f93443d4740 1 -- 172.21.15.67:0/3624816151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9330004680 con 0x555a4913f2f0 2024-07-31T15:08:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.323+0000 7f931aff5640 1 -- 172.21.15.67:0/3624816151 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9379+0+0 (secure 0 0 0) 0x7f93280a7460 con 0x555a4913f2f0 2024-07-31T15:08:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.323+0000 7f931aff5640 1 -- 172.21.15.67:0/3624816151 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f93280bae80 con 0x555a4913f2f0 2024-07-31T15:08:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.323+0000 7f93443d4740 10 client.28660 fetch_fsmap learned FSMap version 65 2024-07-31T15:08:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.323+0000 7f93443d4740 1 -- 172.21.15.67:0/3624816151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x555a495a0ac0 con 0x555a4913f2f0 2024-07-31T15:08:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.324+0000 7f931aff5640 1 -- 172.21.15.67:0/3624816151 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9330004680 con 0x555a4913f2f0 2024-07-31T15:08:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.324+0000 7f931aff5640 1 -- 172.21.15.67:0/3624816151 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f93280aa0d0 con 0x555a4913f2f0 2024-07-31T15:08:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.324+0000 7f93443d4740 10 client.28660 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:08:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.324+0000 7f93443d4740 10 client.28660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:08:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.324+0000 7f93443d4740 20 client.28660 populate_metadata read hostname 'smithi067' 2024-07-31T15:08:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.324+0000 7f93443d4740 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555a490fc1c0 0x555a495a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:08:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.324+0000 7f93443d4740 4 client.28660 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:08:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.324+0000 7f93443d4740 1 -- 172.21.15.67:0/3624816151 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a4947dda0 con 0x555a490fc1c0 2024-07-31T15:08:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.324+0000 7f93367fc640 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555a490fc1c0 0x555a495a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:08:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.325+0000 7f931aff5640 10 client.28660 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:58.325+0000 7f93367fc640 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555a490fc1c0 0x555a495a2d60 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=566c2bccd5afef81 server_cookie=fa6a1a1dae0cb9ac in_seq=0 out_seq=0 2024-07-31T15:08:58.347 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/soc/intel_gmch.o 2024-07-31T15:08:58.351 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/netfilter/nf_reject_ipv6.o 2024-07-31T15:08:58.639 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/property.o 2024-07-31T15:08:58.926 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_gss/gss_generic_token.o 2024-07-31T15:08:59.246 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/soc/intel_pch.o 2024-07-31T15:08:59.627 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_ftp.o 2024-07-31T15:08:59.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:59 smithi073 ceph-mon[36714]: pgmap v968: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 159 KiB/s wr, 3.40k op/s 2024-07-31T15:08:59.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:08:59 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:08:59.644 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/xfrm/built-in.a 2024-07-31T15:08:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.645+0000 7f931aff5640 1 -- 172.21.15.67:0/3624816151 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a4947dda0 con 0x555a490fc1c0 2024-07-31T15:08:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.645+0000 7f931aff5640 10 client.28660 handle_command_reply: tid=0 2024-07-31T15:08:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.645+0000 7f93443d4740 1 client.28660 shutdown 2024-07-31T15:08:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.646+0000 7f93443d4740 1 -- 172.21.15.67:0/3624816151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 msgr2=0x555a4951e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.646+0000 7f93443d4740 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a4951e760 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9328017780 tx=0x7f93280ab5f0 comp rx=0 tx=0).stop 2024-07-31T15:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.646+0000 7f93443d4740 1 -- 172.21.15.67:0/3624816151 shutdown_connections 2024-07-31T15:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.646+0000 7f93443d4740 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555a490fc1c0 0x555a495a2d60 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.646+0000 7f93443d4740 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a491681b0 0x555a4951b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.646+0000 7f93443d4740 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555a491474d0 0x555a4951eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.646+0000 7f93443d4740 1 --2- 172.21.15.67:0/3624816151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a4913f2f0 0x555a4951e760 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.646+0000 7f93443d4740 1 -- 172.21.15.67:0/3624816151 >> 172.21.15.67:0/3624816151 conn(0x555a4915c300 msgr2=0x555a495167e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.646+0000 7f931aff5640 0 client.28660 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.646+0000 7f93443d4740 1 -- 172.21.15.67:0/3624816151 shutdown_connections 2024-07-31T15:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.646+0000 7f93443d4740 1 -- 172.21.15.67:0/3624816151 wait complete. 2024-07-31T15:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.647+0000 7f93443d4740 20 client.28660 trim_cache size 0 max 16384 2024-07-31T15:08:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:08:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:08:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:08:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:08:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.648+0000 7f933d406640 1 -- 172.21.15.67:0/2624989241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93200b3f50 msgr2=0x7f93200b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.648+0000 7f933d406640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93200b3f50 0x7f93200b6460 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f932c006fd0 tx=0x7f932c009060 comp rx=0 tx=0).stop 2024-07-31T15:08:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.648+0000 7f933d406640 1 -- 172.21.15.67:0/2624989241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f933810a330 msgr2=0x7f93381a8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:08:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.648+0000 7f933d406640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f933810a330 0x7f93381a8de0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f932800b4a0 tx=0x7f932800b960 comp rx=0 tx=0).stop 2024-07-31T15:08:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.648+0000 7f933d406640 1 -- 172.21.15.67:0/2624989241 shutdown_connections 2024-07-31T15:08:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.648+0000 7f933d406640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f93200b3f50 0x7f93200b6460 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.649+0000 7f933d406640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f933811c5d0 0x7f93381ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.649+0000 7f933d406640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93381118d0 0x7f93381a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.649+0000 7f933d406640 1 --2- 172.21.15.67:0/2624989241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f933810a330 0x7f93381a8de0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:08:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.649+0000 7f933d406640 1 -- 172.21.15.67:0/2624989241 >> 172.21.15.67:0/2624989241 conn(0x7f93380707c0 msgr2=0x7f933811a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:08:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.649+0000 7f933d406640 1 -- 172.21.15.67:0/2624989241 shutdown_connections 2024-07-31T15:08:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:08:59.649+0000 7f933d406640 1 -- 172.21.15.67:0/2624989241 wait complete. 2024-07-31T15:08:59.657 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/cacheinfo.o 2024-07-31T15:08:59.687 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_gss/gss_mech_switch.o 2024-07-31T15:08:59.690 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/netfilter/ip6t_ipv6header.o 2024-07-31T15:08:59.713 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_irc.o 2024-07-31T15:08:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:59 smithi077 ceph-mon[37327]: pgmap v968: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 159 KiB/s wr, 3.40k op/s 2024-07-31T15:08:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:08:59 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:08:59.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:59 smithi067 ceph-mon[32750]: pgmap v968: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 159 KiB/s wr, 3.40k op/s 2024-07-31T15:08:59.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:08:59 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:09:00.085 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/swnode.o 2024-07-31T15:09:00.333 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/auxiliary.o 2024-07-31T15:09:00.482 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_memcpy.o 2024-07-31T15:09:00.484 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp_timer.o 2024-07-31T15:09:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:00 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:09:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:00 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:09:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:00 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:09:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:00 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:09:00.677 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_conntrack_sip.o 2024-07-31T15:09:00.687 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_mm.o 2024-07-31T15:09:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:00 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:09:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:00 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:09:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:00 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:09:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:00 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:09:00.765 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/netfilter/ip6t_REJECT.o 2024-07-31T15:09:00.768 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/devtmpfs.o 2024-07-31T15:09:00.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:00 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:09:00.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:00 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:09:00.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:00 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:09:00.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:00 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:09:00.808 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_gss/svcauth_gss.o 2024-07-31T15:09:00.838 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_nat_core.o 2024-07-31T15:09:00.917 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/node.o 2024-07-31T15:09:01.488 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_sw_fence.o 2024-07-31T15:09:01.526 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/module.o 2024-07-31T15:09:01.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:01 smithi073 ceph-mon[36714]: pgmap v969: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 78 KiB/s wr, 1.87k op/s 2024-07-31T15:09:01.643 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/platform-msi.o 2024-07-31T15:09:01.696 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp_ipv4.o 2024-07-31T15:09:01.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:01 smithi077 ceph-mon[37327]: pgmap v969: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 78 KiB/s wr, 1.87k op/s 2024-07-31T15:09:01.742 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/ipv6/netfilter/built-in.a 2024-07-31T15:09:01.803 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/af_inet6.o 2024-07-31T15:09:01.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:01 smithi067 ceph-mon[32750]: pgmap v969: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 78 KiB/s wr, 1.87k op/s 2024-07-31T15:09:01.909 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/anycast.o 2024-07-31T15:09:02.072 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_sw_fence_work.o 2024-07-31T15:09:02.177 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/physical_location.o 2024-07-31T15:09:02.256 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_nat_proto.o 2024-07-31T15:09:02.363 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_nat_helper.o 2024-07-31T15:09:02.808 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_gss/gss_rpc_upcall.o 2024-07-31T15:09:02.878 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_syncmap.o 2024-07-31T15:09:02.904 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/base/trace.o 2024-07-31T15:09:03.305 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_user_extensions.o 2024-07-31T15:09:03.315 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_ioc32.o 2024-07-31T15:09:03.507 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/ip6_output.o 2024-07-31T15:09:03.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:03 smithi073 ceph-mon[36714]: pgmap v970: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 257 KiB/s wr, 1.15k op/s 2024-07-31T15:09:03.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:03 smithi077 ceph-mon[37327]: pgmap v970: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 257 KiB/s wr, 1.15k op/s 2024-07-31T15:09:03.769 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/base/built-in.a 2024-07-31T15:09:03.803 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_debugfs.o 2024-07-31T15:09:03.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:03 smithi067 ceph-mon[32750]: pgmap v970: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 257 KiB/s wr, 1.15k op/s 2024-07-31T15:09:03.870 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_gss/gss_rpc_xdr.o 2024-07-31T15:09:03.898 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_nat_masquerade.o 2024-07-31T15:09:03.906 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_debugfs_params.o 2024-07-31T15:09:03.934 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/macintosh/mac_hid.o 2024-07-31T15:09:04.155 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_display_debugfs.o 2024-07-31T15:09:04.279 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp_minisocks.o 2024-07-31T15:09:04.572 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/macintosh/built-in.a 2024-07-31T15:09:04.947 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/scsi/pcmcia/built-in.a 2024-07-31T15:09:04.980 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_pipe_crc.o 2024-07-31T15:09:05.029 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi.o 2024-07-31T15:09:05.235 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_nat_ftp.o 2024-07-31T15:09:05.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:09:05.252 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_gss/trace.o 2024-07-31T15:09:05.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:05 smithi073 ceph-mon[36714]: pgmap v971: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 497 KiB/s wr, 441 op/s 2024-07-31T15:09:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:05 smithi073 ceph-mon[36714]: osdmap e122: 12 total, 12 up, 12 in 2024-07-31T15:09:05.679 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_pmu.o 2024-07-31T15:09:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:05 smithi077 ceph-mon[37327]: pgmap v971: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 497 KiB/s wr, 441 op/s 2024-07-31T15:09:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:05 smithi077 ceph-mon[37327]: osdmap e122: 12 total, 12 up, 12 in 2024-07-31T15:09:05.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:05 smithi067 ceph-mon[32750]: pgmap v971: 129 pgs: 129 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 497 KiB/s wr, 441 op/s 2024-07-31T15:09:05.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:05 smithi067 ceph-mon[32750]: osdmap e122: 12 total, 12 up, 12 in 2024-07-31T15:09:06.100 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp_cong.o 2024-07-31T15:09:06.470 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/ip6_input.o 2024-07-31T15:09:06.476 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:09:06.843 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_nat_irc.o 2024-07-31T15:09:06.954 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/gen2_engine_cs.o 2024-07-31T15:09:07.246 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/hosts.o 2024-07-31T15:09:07.284 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_ioctl.o 2024-07-31T15:09:07.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:07 smithi067 ceph-mon[32750]: pgmap v973: 129 pgs: 5 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 109 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 982 KiB/s wr, 148 op/s 2024-07-31T15:09:07.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:07 smithi073 ceph-mon[36714]: pgmap v973: 129 pgs: 5 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 109 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 982 KiB/s wr, 148 op/s 2024-07-31T15:09:07.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:07 smithi077 ceph-mon[37327]: pgmap v973: 129 pgs: 5 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 109 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 982 KiB/s wr, 148 op/s 2024-07-31T15:09:08.020 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/nf_nat_sip.o 2024-07-31T15:09:08.205 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/x_tables.o 2024-07-31T15:09:08.208 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp_metrics.o 2024-07-31T15:09:08.482 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/tiny/built-in.a 2024-07-31T15:09:08.525 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_gss/gss_krb5_mech.o 2024-07-31T15:09:08.535 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp_fastopen.o 2024-07-31T15:09:08.538 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/addrconf.o 2024-07-31T15:09:08.701 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsicam.o 2024-07-31T15:09:09.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.116+0000 7f37325f2640 1 -- 172.21.15.67:0/2172307620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f372c11c660 msgr2=0x7f372c11eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:09.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.116+0000 7f37325f2640 1 --2- 172.21.15.67:0/2172307620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f372c11c660 0x7f372c11eb70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3724009ca0 tx=0x7f372402a8f0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.123+0000 7f37325f2640 1 -- 172.21.15.67:0/2172307620 shutdown_connections 2024-07-31T15:09:09.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.123+0000 7f37325f2640 1 --2- 172.21.15.67:0/2172307620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f372c11c660 0x7f372c11eb70 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.123+0000 7f37325f2640 1 --2- 172.21.15.67:0/2172307620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f372c10a410 0x7f372c10a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.123+0000 7f37325f2640 1 --2- 172.21.15.67:0/2172307620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f372c111970 0x7f372c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.123+0000 7f37325f2640 1 -- 172.21.15.67:0/2172307620 >> 172.21.15.67:0/2172307620 conn(0x7f372c0708f0 msgr2=0x7f372c070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:09.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.132+0000 7f37325f2640 1 -- 172.21.15.67:0/2172307620 shutdown_connections 2024-07-31T15:09:09.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.133+0000 7f37325f2640 1 -- 172.21.15.67:0/2172307620 wait complete. 2024-07-31T15:09:09.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.135+0000 7f37325f2640 1 Processor -- start 2024-07-31T15:09:09.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.142+0000 7f37325f2640 1 -- start start 2024-07-31T15:09:09.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.143+0000 7f37325f2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f372c10a410 0x7f372c1a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.143+0000 7f37325f2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f372c111970 0x7f372c1a9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.143+0000 7f37325f2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f372c11c660 0x7f372c1ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.143+0000 7f37325f2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f372c122350 con 0x7f372c111970 2024-07-31T15:09:09.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.143+0000 7f37325f2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1221d0 con 0x7f372c11c660 2024-07-31T15:09:09.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.143+0000 7f37325f2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1224d0 con 0x7f372c10a410 2024-07-31T15:09:09.146 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/gen6_engine_cs.o 2024-07-31T15:09:09.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.149+0000 7f372b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f372c111970 0x7f372c1a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:09.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.149+0000 7f3730b8f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f372c11c660 0x7f372c1ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:09.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.149+0000 7f372b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f372c111970 0x7f372c1a9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49354/0 (socket says 172.21.15.67:49354) 2024-07-31T15:09:09.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.149+0000 7f372b7fe640 1 -- 172.21.15.67:0/3448032312 learned_addr learned my addr 172.21.15.67:0/3448032312 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:09.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.150+0000 7f372bfff640 1 --2- 172.21.15.67:0/3448032312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f372c10a410 0x7f372c1a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:09.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.158+0000 7f372b7fe640 1 -- 172.21.15.67:0/3448032312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f372c10a410 msgr2=0x7f372c1a8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:09.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.158+0000 7f372b7fe640 1 --2- 172.21.15.67:0/3448032312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f372c10a410 0x7f372c1a8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.158+0000 7f372b7fe640 1 -- 172.21.15.67:0/3448032312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f372c11c660 msgr2=0x7f372c1ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:09.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.158+0000 7f372b7fe640 1 --2- 172.21.15.67:0/3448032312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f372c11c660 0x7f372c1ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.158+0000 7f372b7fe640 1 -- 172.21.15.67:0/3448032312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c1aecd0 con 0x7f372c111970 2024-07-31T15:09:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.161+0000 7f372bfff640 1 --2- 172.21.15.67:0/3448032312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f372c10a410 0x7f372c1a8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.171+0000 7f372b7fe640 1 --2- 172.21.15.67:0/3448032312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f372c111970 0x7f372c1a9300 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f372000d740 tx=0x7f372000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.178+0000 7f37297fa640 1 -- 172.21.15.67:0/3448032312 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720017070 con 0x7f372c111970 2024-07-31T15:09:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.178+0000 7f37297fa640 1 -- 172.21.15.67:0/3448032312 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3720009cd0 con 0x7f372c111970 2024-07-31T15:09:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.178+0000 7f37297fa640 1 -- 172.21.15.67:0/3448032312 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720003b50 con 0x7f372c111970 2024-07-31T15:09:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.178+0000 7f37325f2640 1 -- 172.21.15.67:0/3448032312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372c1aefc0 con 0x7f372c111970 2024-07-31T15:09:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.178+0000 7f37325f2640 1 -- 172.21.15.67:0/3448032312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372c1b5720 con 0x7f372c111970 2024-07-31T15:09:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.181+0000 7f3738b43740 1 Processor -- start 2024-07-31T15:09:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.188+0000 7f3738b43740 1 -- start start 2024-07-31T15:09:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.191+0000 7f3738b43740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e21685070 0x557e21a2a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.191+0000 7f3738b43740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e2165c1b0 0x557e21a2aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.191+0000 7f3738b43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21664390 0x557e21a2afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.191+0000 7f3738b43740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557e219a0b60 con 0x557e21685070 2024-07-31T15:09:09.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.191+0000 7f3738b43740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557e21990060 con 0x557e2165c1b0 2024-07-31T15:09:09.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.191+0000 7f3738b43740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557e219940a0 con 0x557e21664390 2024-07-31T15:09:09.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.193+0000 7f372bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e2165c1b0 0x557e21a2aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:09.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.193+0000 7f372bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e2165c1b0 0x557e21a2aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37604/0 (socket says 172.21.15.67:37604) 2024-07-31T15:09:09.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.193+0000 7f372bfff640 1 -- 172.21.15.67:0/2277246437 learned_addr learned my addr 172.21.15.67:0/2277246437 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:09.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.195+0000 7f372b7fe640 1 --2- 172.21.15.67:0/2277246437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21664390 0x557e21a2afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:09.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.197+0000 7f372bfff640 1 -- 172.21.15.67:0/2277246437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21664390 msgr2=0x557e21a2afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:09.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.198+0000 7f372bfff640 1 --2- 172.21.15.67:0/2277246437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21664390 0x557e21a2afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.198+0000 7f372bfff640 1 -- 172.21.15.67:0/2277246437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e21685070 msgr2=0x557e21a2a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:09.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.198+0000 7f372bfff640 1 --2- 172.21.15.67:0/2277246437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e21685070 0x557e21a2a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.198+0000 7f372bfff640 1 -- 172.21.15.67:0/2277246437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e21936b40 con 0x557e2165c1b0 2024-07-31T15:09:09.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.200+0000 7f37297fa640 1 -- 172.21.15.67:0/3448032312 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3720003cf0 con 0x7f372c111970 2024-07-31T15:09:09.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.201+0000 7f37297fa640 1 --2- 172.21.15.67:0/3448032312 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37140b3f50 0x7f37140b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.201+0000 7f37297fa640 1 -- 172.21.15.67:0/3448032312 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9483+0+0 (secure 0 0 0) 0x7f37200e6b00 con 0x7f372c111970 2024-07-31T15:09:09.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.206+0000 7f3730b8f640 1 --2- 172.21.15.67:0/3448032312 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37140b3f50 0x7f37140b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:09.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.206+0000 7f372bfff640 1 --2- 172.21.15.67:0/2277246437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e2165c1b0 0x557e21a2aa60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f371c009e70 tx=0x7f371c00e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afad891f955782ff server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:09.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.207+0000 7f3730b8f640 1 --2- 172.21.15.67:0/3448032312 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37140b3f50 0x7f37140b6460 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3724006010 tx=0x7f3724009840 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:09.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.211+0000 7f37127fc640 1 -- 172.21.15.67:0/2277246437 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c017070 con 0x557e2165c1b0 2024-07-31T15:09:09.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.211+0000 7f37127fc640 1 -- 172.21.15.67:0/2277246437 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f371c013920 con 0x557e2165c1b0 2024-07-31T15:09:09.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.211+0000 7f37127fc640 1 -- 172.21.15.67:0/2277246437 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c012870 con 0x557e2165c1b0 2024-07-31T15:09:09.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.213+0000 7f3738b43740 1 -- 172.21.15.67:0/2277246437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e2165c1b0 msgr2=0x557e21a2aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:09.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.213+0000 7f3738b43740 1 --2- 172.21.15.67:0/2277246437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e2165c1b0 0x557e21a2aa60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f371c009e70 tx=0x7f371c00e5a0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.215+0000 7f3738b43740 1 -- 172.21.15.67:0/2277246437 shutdown_connections 2024-07-31T15:09:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.215+0000 7f3738b43740 1 --2- 172.21.15.67:0/2277246437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21664390 0x557e21a2afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.215+0000 7f3738b43740 1 --2- 172.21.15.67:0/2277246437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e2165c1b0 0x557e21a2aa60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.215+0000 7f3738b43740 1 --2- 172.21.15.67:0/2277246437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e21685070 0x557e21a2a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.215+0000 7f3738b43740 1 -- 172.21.15.67:0/2277246437 >> 172.21.15.67:0/2277246437 conn(0x557e21679460 msgr2=0x557e21a28410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:09.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.226+0000 7f3738b43740 1 -- 172.21.15.67:0/2277246437 shutdown_connections 2024-07-31T15:09:09.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.226+0000 7f3738b43740 1 -- 172.21.15.67:0/2277246437 wait complete. 2024-07-31T15:09:09.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.227+0000 7f3738b43740 1 -- start start 2024-07-31T15:09:09.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.227+0000 7f3738b43740 1 Processor -- start 2024-07-31T15:09:09.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.228+0000 7f3738b43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e2165c1b0 0x557e21a36160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.228+0000 7f3738b43740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e21664390 0x557e21a366c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.228+0000 7f3738b43740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21685070 0x557e21a42080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.228+0000 7f3738b43740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557e217f3700 con 0x557e21664390 2024-07-31T15:09:09.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.228+0000 7f3738b43740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557e21932990 con 0x557e21685070 2024-07-31T15:09:09.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.228+0000 7f3738b43740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557e21a38d00 con 0x557e2165c1b0 2024-07-31T15:09:09.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.228+0000 7f3738b43740 10 client.0 _ll_register_callbacks cb 0x557e2194d4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:09.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.228+0000 7f372b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e21664390 0x557e21a366c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:09.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.228+0000 7f372b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e21664390 0x557e21a366c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49374/0 (socket says 172.21.15.67:49374) 2024-07-31T15:09:09.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.228+0000 7f372b7fe640 1 -- 172.21.15.67:0/982772661 learned_addr learned my addr 172.21.15.67:0/982772661 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:09.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.229+0000 7f372b7fe640 1 -- 172.21.15.67:0/982772661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e2165c1b0 msgr2=0x557e21a36160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:09:09.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.229+0000 7f372b7fe640 1 --2- 172.21.15.67:0/982772661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e2165c1b0 0x557e21a36160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.229+0000 7f372b7fe640 1 -- 172.21.15.67:0/982772661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21685070 msgr2=0x557e21a42080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:09:09.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.229+0000 7f372b7fe640 1 --2- 172.21.15.67:0/982772661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21685070 0x557e21a42080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.229+0000 7f372b7fe640 1 -- 172.21.15.67:0/982772661 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e216f0650 con 0x557e21664390 2024-07-31T15:09:09.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.229+0000 7f372b7fe640 1 --2- 172.21.15.67:0/982772661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e21664390 0x557e21a366c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f37200acfd0 tx=0x7f37200e4390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86430a771dbe40a9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:09.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.235+0000 7f370b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:09.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.235+0000 7f370b7fe640 1 -- 172.21.15.67:0/982772661 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720020040 con 0x557e21664390 2024-07-31T15:09:09.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.235+0000 7f3738b43740 1 -- 172.21.15.67:0/982772661 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f371c013920 con 0x557e21664390 2024-07-31T15:09:09.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.235+0000 7f370b7fe640 1 -- 172.21.15.67:0/982772661 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9483+0+0 (secure 0 0 0) 0x7f37200b6100 con 0x557e21664390 2024-07-31T15:09:09.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.246+0000 7f370b7fe640 1 -- 172.21.15.67:0/982772661 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f37200ad9d0 con 0x557e21664390 2024-07-31T15:09:09.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.247+0000 7f3738b43740 10 client.19248 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:09.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.247+0000 7f3738b43740 1 -- 172.21.15.67:0/982772661 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x557e21a55de0 con 0x557e21664390 2024-07-31T15:09:09.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.250+0000 7f370b7fe640 1 -- 172.21.15.67:0/982772661 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f371c013920 con 0x557e21664390 2024-07-31T15:09:09.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.250+0000 7f370b7fe640 1 -- 172.21.15.67:0/982772661 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f37200abfa0 con 0x557e21664390 2024-07-31T15:09:09.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.250+0000 7f3738b43740 10 client.19248 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:09.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.250+0000 7f3738b43740 10 client.19248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:09.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.250+0000 7f3738b43740 20 client.19248 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:09.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.250+0000 7f3738b43740 1 --2- 172.21.15.67:0/982772661 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e216439f0 0x557e21a59430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.251+0000 7f3738b43740 4 client.19248 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:09:09.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.251+0000 7f3738b43740 1 -- 172.21.15.67:0/982772661 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e219a2ab0 con 0x557e216439f0 2024-07-31T15:09:09.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.254+0000 7f372bfff640 1 --2- 172.21.15.67:0/982772661 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e216439f0 0x557e21a59430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:09.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.271+0000 7f372bfff640 1 --2- 172.21.15.67:0/982772661 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e216439f0 0x557e21a59430 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f5bd7430fb63305 server_cookie=36c5c8098a896aba in_seq=0 out_seq=0 2024-07-31T15:09:09.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.271+0000 7f370b7fe640 10 client.19248 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:09.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.275+0000 7f370b7fe640 1 -- 172.21.15.67:0/982772661 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e219a2ab0 con 0x557e216439f0 2024-07-31T15:09:09.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.275+0000 7f370b7fe640 10 client.19248 handle_command_reply: tid=0 2024-07-31T15:09:09.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.278+0000 7f3738b43740 1 client.19248 shutdown 2024-07-31T15:09:09.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.279+0000 7f3738b43740 1 -- 172.21.15.67:0/982772661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e21664390 msgr2=0x557e21a366c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:09.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.279+0000 7f3738b43740 1 --2- 172.21.15.67:0/982772661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e21664390 0x557e21a366c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f37200acfd0 tx=0x7f37200e4390 comp rx=0 tx=0).stop 2024-07-31T15:09:09.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.284+0000 7f3738b43740 1 -- 172.21.15.67:0/982772661 shutdown_connections 2024-07-31T15:09:09.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.284+0000 7f3738b43740 1 --2- 172.21.15.67:0/982772661 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e216439f0 0x557e21a59430 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.284+0000 7f3738b43740 1 --2- 172.21.15.67:0/982772661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21685070 0x557e21a42080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.284+0000 7f3738b43740 1 --2- 172.21.15.67:0/982772661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e21664390 0x557e21a366c0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.284+0000 7f3738b43740 1 --2- 172.21.15.67:0/982772661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e2165c1b0 0x557e21a36160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.284+0000 7f3738b43740 1 -- 172.21.15.67:0/982772661 >> 172.21.15.67:0/982772661 conn(0x557e21679460 msgr2=0x557e21a26bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:09.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.284+0000 7f370b7fe640 0 client.19248 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:09.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.284+0000 7f3738b43740 1 -- 172.21.15.67:0/982772661 shutdown_connections 2024-07-31T15:09:09.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.288+0000 7f3738b43740 1 -- 172.21.15.67:0/982772661 wait complete. 2024-07-31T15:09:09.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.292+0000 7f3738b43740 20 client.19248 trim_cache size 0 max 16384 2024-07-31T15:09:09.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.333+0000 7f3738b43740 1 Processor -- start 2024-07-31T15:09:09.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.336+0000 7f3738b43740 1 -- start start 2024-07-31T15:09:09.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.336+0000 7f3738b43740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e2165c1b0 0x557e21a28510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.336+0000 7f3738b43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21664390 0x557e21a41e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.336+0000 7f3738b43740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21685070 0x557e21a2f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.336+0000 7f3738b43740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3718010060 con 0x557e2165c1b0 2024-07-31T15:09:09.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.336+0000 7f3738b43740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3718011ba0 con 0x557e21685070 2024-07-31T15:09:09.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.336+0000 7f3738b43740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3718010730 con 0x557e21664390 2024-07-31T15:09:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.336+0000 7f372b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21664390 0x557e21a41e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.336+0000 7f372b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21664390 0x557e21a41e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35168/0 (socket says 172.21.15.67:35168) 2024-07-31T15:09:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.336+0000 7f372b7fe640 1 -- 172.21.15.67:0/818939795 learned_addr learned my addr 172.21.15.67:0/818939795 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.337+0000 7f372b7fe640 1 -- 172.21.15.67:0/818939795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21685070 msgr2=0x557e21a2f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.337+0000 7f372b7fe640 1 --2- 172.21.15.67:0/818939795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21685070 0x557e21a2f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.337+0000 7f372b7fe640 1 -- 172.21.15.67:0/818939795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e2165c1b0 msgr2=0x557e21a28510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.337+0000 7f372b7fe640 1 --2- 172.21.15.67:0/818939795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e2165c1b0 0x557e21a28510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.337+0000 7f372b7fe640 1 -- 172.21.15.67:0/818939795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e21a37220 con 0x557e21664390 2024-07-31T15:09:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.337+0000 7f372b7fe640 1 --2- 172.21.15.67:0/818939795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21664390 0x557e21a41e70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f37200aa540 tx=0x7f37200ad2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd915b97ade7182c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.337+0000 7f3711ffb640 1 -- 172.21.15.67:0/818939795 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720014070 con 0x557e21664390 2024-07-31T15:09:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.337+0000 7f3711ffb640 1 -- 172.21.15.67:0/818939795 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37200b6380 con 0x557e21664390 2024-07-31T15:09:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.337+0000 7f3711ffb640 1 -- 172.21.15.67:0/818939795 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720020030 con 0x557e21664390 2024-07-31T15:09:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.341+0000 7f3738b43740 1 -- 172.21.15.67:0/818939795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21664390 msgr2=0x557e21a41e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.341+0000 7f3738b43740 1 --2- 172.21.15.67:0/818939795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21664390 0x557e21a41e70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f37200aa540 tx=0x7f37200ad2d0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.351+0000 7f3738b43740 1 -- 172.21.15.67:0/818939795 shutdown_connections 2024-07-31T15:09:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.351+0000 7f3738b43740 1 --2- 172.21.15.67:0/818939795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21685070 0x557e21a2f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.351+0000 7f3738b43740 1 --2- 172.21.15.67:0/818939795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21664390 0x557e21a41e70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.351+0000 7f3738b43740 1 --2- 172.21.15.67:0/818939795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e2165c1b0 0x557e21a28510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.351+0000 7f3738b43740 1 -- 172.21.15.67:0/818939795 >> 172.21.15.67:0/818939795 conn(0x557e21679460 msgr2=0x557e21a27e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:09.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.363+0000 7f3738b43740 1 -- 172.21.15.67:0/818939795 shutdown_connections 2024-07-31T15:09:09.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.363+0000 7f3738b43740 1 -- 172.21.15.67:0/818939795 wait complete. 2024-07-31T15:09:09.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.363+0000 7f3738b43740 1 -- start start 2024-07-31T15:09:09.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.364+0000 7f3738b43740 1 Processor -- start 2024-07-31T15:09:09.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.364+0000 7f3738b43740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e2165c1b0 0x557e21a3a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.365+0000 7f3738b43740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21664390 0x557e21a3a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.365+0000 7f3738b43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21685070 0x557e21a3aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.365+0000 7f372b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21664390 0x557e21a3a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:09.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.365+0000 7f372b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21664390 0x557e21a3a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37616/0 (socket says 172.21.15.67:37616) 2024-07-31T15:09:09.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.365+0000 7f372b7fe640 1 -- 172.21.15.67:0/2369224743 learned_addr learned my addr 172.21.15.67:0/2369224743 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:09.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.365+0000 7f3730b8f640 1 --2- 172.21.15.67:0/2369224743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21685070 0x557e21a3aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:09.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.365+0000 7f3738b43740 1 -- 172.21.15.67:0/2369224743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f37180114d0 con 0x557e2165c1b0 2024-07-31T15:09:09.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.365+0000 7f3738b43740 1 -- 172.21.15.67:0/2369224743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3718010e00 con 0x557e21664390 2024-07-31T15:09:09.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.366+0000 7f372bfff640 1 --2- 172.21.15.67:0/2369224743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e2165c1b0 0x557e21a3a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:09.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.366+0000 7f3738b43740 1 -- 172.21.15.67:0/2369224743 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f371800e060 con 0x557e21685070 2024-07-31T15:09:09.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.366+0000 7f372b7fe640 1 -- 172.21.15.67:0/2369224743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21685070 msgr2=0x557e21a3aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:09.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.366+0000 7f372b7fe640 1 --2- 172.21.15.67:0/2369224743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21685070 0x557e21a3aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.366+0000 7f372b7fe640 1 -- 172.21.15.67:0/2369224743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e2165c1b0 msgr2=0x557e21a3a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:09.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.366+0000 7f372b7fe640 1 --2- 172.21.15.67:0/2369224743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e2165c1b0 0x557e21a3a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.366+0000 7f372b7fe640 1 -- 172.21.15.67:0/2369224743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e21ac0d70 con 0x557e21664390 2024-07-31T15:09:09.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.367+0000 7f372b7fe640 1 --2- 172.21.15.67:0/2369224743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21664390 0x557e21a3a580 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3720012040 tx=0x7f37200ce540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb785327c3584f0d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:09.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.367+0000 7f3738b43740 10 client.0 _ll_register_callbacks cb 0x7f37200046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:09.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.367+0000 7f3738b43740 1 -- 172.21.15.67:0/2369224743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37200b6380 con 0x557e21664390 2024-07-31T15:09:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.370+0000 7f370b7fe640 10 client.28799 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:09:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.370+0000 7f370b7fe640 1 -- 172.21.15.67:0/2369224743 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720014030 con 0x557e21664390 2024-07-31T15:09:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.370+0000 7f370b7fe640 1 -- 172.21.15.67:0/2369224743 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9483+0+0 (secure 0 0 0) 0x7f37200ac530 con 0x557e21664390 2024-07-31T15:09:09.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.382+0000 7f370b7fe640 1 -- 172.21.15.67:0/2369224743 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f3720020070 con 0x557e21664390 2024-07-31T15:09:09.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.383+0000 7f3738b43740 10 client.28799 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:09.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.383+0000 7f3738b43740 1 -- 172.21.15.67:0/2369224743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=65}) -- 0x557e21ac1bb0 con 0x557e21664390 2024-07-31T15:09:09.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.390+0000 7f370b7fe640 1 -- 172.21.15.67:0/2369224743 <== mon.2 v2:172.21.15.73:3300/0 4 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f37200b2a60 con 0x557e21664390 2024-07-31T15:09:09.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.391+0000 7f3738b43740 10 client.28799 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:09.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.391+0000 7f3738b43740 10 client.28799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:09.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.391+0000 7f3738b43740 20 client.28799 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:09.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.391+0000 7f3738b43740 1 --2- 172.21.15.67:0/2369224743 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e21618da0 0x557e21ac4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:09.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.391+0000 7f3738b43740 4 client.28799 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:09:09.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.391+0000 7f3738b43740 1 -- 172.21.15.67:0/2369224743 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f37200b6380 con 0x557e21618da0 2024-07-31T15:09:09.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.395+0000 7f372bfff640 1 --2- 172.21.15.67:0/2369224743 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e21618da0 0x557e21ac4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:09.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.420+0000 7f372bfff640 1 --2- 172.21.15.67:0/2369224743 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e21618da0 0x557e21ac4080 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45a7bd21f8465fe0 server_cookie=28f20da832f597dc in_seq=0 out_seq=0 2024-07-31T15:09:09.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.421+0000 7f370b7fe640 10 client.28799 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:09.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.425+0000 7f370b7fe640 1 -- 172.21.15.67:0/2369224743 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f37200b6380 con 0x557e21618da0 2024-07-31T15:09:09.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.425+0000 7f370b7fe640 10 client.28799 handle_command_reply: tid=0 2024-07-31T15:09:09.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.425+0000 7f3738b43740 1 client.28799 shutdown 2024-07-31T15:09:09.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.425+0000 7f3738b43740 1 -- 172.21.15.67:0/2369224743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21664390 msgr2=0x557e21a3a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:09.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.425+0000 7f3738b43740 1 --2- 172.21.15.67:0/2369224743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21664390 0x557e21a3a580 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3720012040 tx=0x7f37200ce540 comp rx=0 tx=0).stop 2024-07-31T15:09:09.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.425+0000 7f3738b43740 1 -- 172.21.15.67:0/2369224743 shutdown_connections 2024-07-31T15:09:09.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.425+0000 7f3738b43740 1 --2- 172.21.15.67:0/2369224743 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e21618da0 0x557e21ac4080 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.425+0000 7f3738b43740 1 --2- 172.21.15.67:0/2369224743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e21685070 0x557e21a3aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.425+0000 7f3738b43740 1 --2- 172.21.15.67:0/2369224743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e21664390 0x557e21a3a580 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.425+0000 7f3738b43740 1 --2- 172.21.15.67:0/2369224743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e2165c1b0 0x557e21a3a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.425+0000 7f3738b43740 1 -- 172.21.15.67:0/2369224743 >> 172.21.15.67:0/2369224743 conn(0x557e21679460 msgr2=0x557e21a2f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:09.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.425+0000 7f370b7fe640 0 client.28799 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:09.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.426+0000 7f3738b43740 1 -- 172.21.15.67:0/2369224743 shutdown_connections 2024-07-31T15:09:09.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.426+0000 7f3738b43740 1 -- 172.21.15.67:0/2369224743 wait complete. 2024-07-31T15:09:09.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.426+0000 7f3738b43740 20 client.28799 trim_cache size 0 max 16384 2024-07-31T15:09:09.428 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:09:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T15:09:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T15:09:09.428 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:09:09.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.427+0000 7f37325f2640 1 -- 172.21.15.67:0/3448032312 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37140b3f50 msgr2=0x7f37140b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.427+0000 7f37325f2640 1 --2- 172.21.15.67:0/3448032312 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37140b3f50 0x7f37140b6460 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3724006010 tx=0x7f3724009840 comp rx=0 tx=0).stop 2024-07-31T15:09:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.427+0000 7f37325f2640 1 -- 172.21.15.67:0/3448032312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f372c111970 msgr2=0x7f372c1a9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.427+0000 7f37325f2640 1 --2- 172.21.15.67:0/3448032312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f372c111970 0x7f372c1a9300 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f372000d740 tx=0x7f372000db40 comp rx=0 tx=0).stop 2024-07-31T15:09:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.427+0000 7f37325f2640 1 -- 172.21.15.67:0/3448032312 shutdown_connections 2024-07-31T15:09:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.427+0000 7f37325f2640 1 --2- 172.21.15.67:0/3448032312 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37140b3f50 0x7f37140b6460 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.427+0000 7f37325f2640 1 --2- 172.21.15.67:0/3448032312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f372c11c660 0x7f372c1ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.427+0000 7f37325f2640 1 --2- 172.21.15.67:0/3448032312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f372c111970 0x7f372c1a9300 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.427+0000 7f37325f2640 1 --2- 172.21.15.67:0/3448032312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f372c10a410 0x7f372c1a8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.427+0000 7f37325f2640 1 -- 172.21.15.67:0/3448032312 >> 172.21.15.67:0/3448032312 conn(0x7f372c0708f0 msgr2=0x7f372c11ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.427+0000 7f37325f2640 1 -- 172.21.15.67:0/3448032312 shutdown_connections 2024-07-31T15:09:09.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:09.427+0000 7f37325f2640 1 -- 172.21.15.67:0/3448032312 wait complete. 2024-07-31T15:09:09.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:09 smithi067 ceph-mon[32750]: pgmap v974: 129 pgs: 15 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 73 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-07-31T15:09:09.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:09 smithi073 ceph-mon[36714]: pgmap v974: 129 pgs: 15 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 73 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-07-31T15:09:09.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:09 smithi077 ceph-mon[37327]: pgmap v974: 129 pgs: 15 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 73 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-07-31T15:09:09.755 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_gss/gss_krb5_seal.o 2024-07-31T15:09:09.923 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_gss/gss_krb5_unseal.o 2024-07-31T15:09:09.930 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_error.o 2024-07-31T15:09:10.079 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_gss/gss_krb5_seqnum.o 2024-07-31T15:09:10.271 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T15:09:10.272 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:09:10.356 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp_rate.o 2024-07-31T15:09:10.704 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/nvme/host/built-in.a 2024-07-31T15:09:10.842 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/nvme/target/built-in.a 2024-07-31T15:09:10.899 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/nvme/built-in.a 2024-07-31T15:09:10.917 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/xlnx/built-in.a 2024-07-31T15:09:10.964 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/clnt.o 2024-07-31T15:09:11.026 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/core.o 2024-07-31T15:09:11.044 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/xt_tcpudp.o 2024-07-31T15:09:11.099 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_gss/gss_krb5_wrap.o 2024-07-31T15:09:11.229 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/gen6_ppgtt.o 2024-07-31T15:09:11.273 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:09:11.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:11 smithi073 ceph-mon[36714]: pgmap v975: 129 pgs: 18 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 68 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-07-31T15:09:11.733 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:11 smithi067 ceph-mon[32750]: pgmap v975: 129 pgs: 18 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 68 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-07-31T15:09:11.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:11 smithi077 ceph-mon[37327]: pgmap v975: 129 pgs: 18 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 68 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-07-31T15:09:11.949 INFO:tasks.quiescer.fs.[cephfs]:Total duration (36.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-31T15:09:11.949 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.1 2024-07-31T15:09:11.949 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:09:11.958 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp_recovery.o 2024-07-31T15:09:12.256 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_gss/gss_krb5_crypto.o 2024-07-31T15:09:12.317 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_lib.o 2024-07-31T15:09:12.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:12 smithi067 ceph-mon[32750]: pgmap v976: 129 pgs: 12 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 83 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-07-31T15:09:12.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:12 smithi077 ceph-mon[37327]: pgmap v976: 129 pgs: 12 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 83 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-07-31T15:09:12.771 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/xt_CONNSECMARK.o 2024-07-31T15:09:12.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:12 smithi073 ceph-mon[36714]: pgmap v976: 129 pgs: 12 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 83 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-07-31T15:09:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.068+0000 7fda32067640 1 -- 172.21.15.67:0/2766262778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda2c10a090 msgr2=0x7fda2c10a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.068+0000 7fda32067640 1 --2- 172.21.15.67:0/2766262778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda2c10a090 0x7fda2c10a4a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fda28008410 tx=0x7fda2802e2f0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.092+0000 7fda32067640 1 -- 172.21.15.67:0/2766262778 shutdown_connections 2024-07-31T15:09:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.092+0000 7fda32067640 1 --2- 172.21.15.67:0/2766262778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda2c10c940 0x7fda2c07b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.092+0000 7fda32067640 1 --2- 172.21.15.67:0/2766262778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda2c10bff0 0x7fda2c10c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.092+0000 7fda32067640 1 --2- 172.21.15.67:0/2766262778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda2c10a090 0x7fda2c10a4a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.092+0000 7fda32067640 1 -- 172.21.15.67:0/2766262778 >> 172.21.15.67:0/2766262778 conn(0x7fda2c06e320 msgr2=0x7fda2c06e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.092+0000 7fda32067640 1 -- 172.21.15.67:0/2766262778 shutdown_connections 2024-07-31T15:09:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.092+0000 7fda32067640 1 -- 172.21.15.67:0/2766262778 wait complete. 2024-07-31T15:09:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.093+0000 7fda32067640 1 Processor -- start 2024-07-31T15:09:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.093+0000 7fda32067640 1 -- start start 2024-07-31T15:09:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.093+0000 7fda32067640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda2c10a090 0x7fda2c07ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.094+0000 7fda32067640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda2c10bff0 0x7fda2c0804e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.094+0000 7fda32067640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda2c10c940 0x7fda2c080a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.094+0000 7fda32067640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c07e4f0 con 0x7fda2c10a090 2024-07-31T15:09:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.094+0000 7fda32067640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c07e370 con 0x7fda2c10bff0 2024-07-31T15:09:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.094+0000 7fda32067640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c07e670 con 0x7fda2c10c940 2024-07-31T15:09:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.094+0000 7fda31866640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda2c10c940 0x7fda2c080a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.094+0000 7fda31866640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda2c10c940 0x7fda2c080a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34434/0 (socket says 172.21.15.67:34434) 2024-07-31T15:09:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.094+0000 7fda31866640 1 -- 172.21.15.67:0/2915381756 learned_addr learned my addr 172.21.15.67:0/2915381756 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.095+0000 7fda31866640 1 -- 172.21.15.67:0/2915381756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda2c10bff0 msgr2=0x7fda2c0804e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:09:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.099+0000 7fda30864640 1 --2- 172.21.15.67:0/2915381756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda2c10bff0 0x7fda2c0804e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.099+0000 7fda31866640 1 --2- 172.21.15.67:0/2915381756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda2c10bff0 0x7fda2c0804e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.099+0000 7fda31866640 1 -- 172.21.15.67:0/2915381756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda2c10a090 msgr2=0x7fda2c07ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.099+0000 7fda31866640 1 --2- 172.21.15.67:0/2915381756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda2c10a090 0x7fda2c07ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.099+0000 7fda31866640 1 -- 172.21.15.67:0/2915381756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c1bd240 con 0x7fda2c10c940 2024-07-31T15:09:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.108+0000 7fda31866640 1 --2- 172.21.15.67:0/2915381756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda2c10c940 0x7fda2c080a20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00a850 tx=0x7fda1c00ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.108+0000 7fda227fc640 1 -- 172.21.15.67:0/2915381756 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c0079b0 con 0x7fda2c10c940 2024-07-31T15:09:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.108+0000 7fda227fc640 1 -- 172.21.15.67:0/2915381756 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda1c005020 con 0x7fda2c10c940 2024-07-31T15:09:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.108+0000 7fda227fc640 1 -- 172.21.15.67:0/2915381756 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c004cd0 con 0x7fda2c10c940 2024-07-31T15:09:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.114+0000 7fda32067640 1 -- 172.21.15.67:0/2915381756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2c1bd530 con 0x7fda2c10c940 2024-07-31T15:09:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.114+0000 7fda32067640 1 -- 172.21.15.67:0/2915381756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2c1bda30 con 0x7fda2c10c940 2024-07-31T15:09:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.123+0000 7fda227fc640 1 -- 172.21.15.67:0/2915381756 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fda1c014410 con 0x7fda2c10c940 2024-07-31T15:09:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.124+0000 7fda227fc640 1 --2- 172.21.15.67:0/2915381756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fda180b3f50 0x7fda180b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.124+0000 7fda227fc640 1 -- 172.21.15.67:0/2915381756 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9483+0+0 (secure 0 0 0) 0x7fda1c0e4800 con 0x7fda2c10c940 2024-07-31T15:09:13.136 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:09:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.131+0000 7fda3a1c9740 1 Processor -- start 2024-07-31T15:09:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.139+0000 7fda31065640 1 --2- 172.21.15.67:0/2915381756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fda180b3f50 0x7fda180b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.147+0000 7fda3a1c9740 1 -- start start 2024-07-31T15:09:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.147+0000 7fda3a1c9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673391b0 0x55f9676dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.147+0000 7fda3a1c9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 0x55f9676dc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.147+0000 7fda3a1c9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9673184d0 0x55f9676dc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.147+0000 7fda3a1c9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f9676513b0 con 0x55f9673102f0 2024-07-31T15:09:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.147+0000 7fda3a1c9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f9676408b0 con 0x55f9673184d0 2024-07-31T15:09:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.147+0000 7fda3a1c9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f9676448f0 con 0x55f9673391b0 2024-07-31T15:09:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.152+0000 7fda31065640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673391b0 0x55f9676dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.152+0000 7fda31065640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673391b0 0x55f9676dbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34450/0 (socket says 172.21.15.67:34450) 2024-07-31T15:09:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.152+0000 7fda31065640 1 -- 172.21.15.67:0/2531925048 learned_addr learned my addr 172.21.15.67:0/2531925048 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.156+0000 7fda31065640 1 --2- 172.21.15.67:0/2915381756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fda180b3f50 0x7fda180b6460 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fda28036d30 tx=0x7fda28036d60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.156+0000 7fda31065640 1 -- 172.21.15.67:0/2531925048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9673184d0 msgr2=0x55f9676dc960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:09:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.156+0000 7fda31065640 1 --2- 172.21.15.67:0/2531925048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9673184d0 0x55f9676dc960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.156+0000 7fda31065640 1 -- 172.21.15.67:0/2531925048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 msgr2=0x55f9676dc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.156+0000 7fda31065640 1 --2- 172.21.15.67:0/2531925048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 0x55f9676dc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.156+0000 7fda31065640 1 -- 172.21.15.67:0/2531925048 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9675e79a0 con 0x55f9673391b0 2024-07-31T15:09:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.156+0000 7fda31065640 1 --2- 172.21.15.67:0/2531925048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673391b0 0x55f9676dbee0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fda28036dd0 tx=0x7fda28049000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fad1c203d702ba2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.161+0000 7fda177fe640 1 -- 172.21.15.67:0/2531925048 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28004040 con 0x55f9673391b0 2024-07-31T15:09:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.161+0000 7fda177fe640 1 -- 172.21.15.67:0/2531925048 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda280041e0 con 0x55f9673391b0 2024-07-31T15:09:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.162+0000 7fda3a1c9740 1 -- 172.21.15.67:0/2531925048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673391b0 msgr2=0x55f9676dbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.162+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/2531925048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673391b0 0x55f9676dbee0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fda28036dd0 tx=0x7fda28049000 comp rx=0 tx=0).stop 2024-07-31T15:09:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.162+0000 7fda3a1c9740 1 -- 172.21.15.67:0/2531925048 shutdown_connections 2024-07-31T15:09:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.162+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/2531925048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9673184d0 0x55f9676dc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.162+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/2531925048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 0x55f9676dc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.162+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/2531925048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673391b0 0x55f9676dbee0 secure :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fda28036dd0 tx=0x7fda28049000 comp rx=0 tx=0).stop 2024-07-31T15:09:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.162+0000 7fda3a1c9740 1 -- 172.21.15.67:0/2531925048 >> 172.21.15.67:0/2531925048 conn(0x55f96732d300 msgr2=0x55f9676d9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:13.179 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.178+0000 7fda3a1c9740 1 -- 172.21.15.67:0/2531925048 shutdown_connections 2024-07-31T15:09:13.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.178+0000 7fda3a1c9740 1 -- 172.21.15.67:0/2531925048 wait complete. 2024-07-31T15:09:13.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.178+0000 7fda3a1c9740 1 -- start start 2024-07-31T15:09:13.180 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.178+0000 7fda3a1c9740 1 Processor -- start 2024-07-31T15:09:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.179+0000 7fda3a1c9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 0x55f9676e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.179+0000 7fda3a1c9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673184d0 0x55f9676e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.179+0000 7fda3a1c9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9672f7d80 0x55f9676fc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.179+0000 7fda3a1c9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f96757aab0 con 0x55f9673102f0 2024-07-31T15:09:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.179+0000 7fda3a1c9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f9675c8cf0 con 0x55f9672f7d80 2024-07-31T15:09:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.179+0000 7fda3a1c9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f9676ea650 con 0x55f9673184d0 2024-07-31T15:09:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.179+0000 7fda3a1c9740 10 client.0 _ll_register_callbacks cb 0x55f96763a9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.179+0000 7fda30864640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673184d0 0x55f9676e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.179+0000 7fda30864640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673184d0 0x55f9676e7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34452/0 (socket says 172.21.15.67:34452) 2024-07-31T15:09:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.179+0000 7fda30864640 1 -- 172.21.15.67:0/2435424776 learned_addr learned my addr 172.21.15.67:0/2435424776 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.179+0000 7fda31866640 1 --2- 172.21.15.67:0/2435424776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9672f7d80 0x55f9676fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.180+0000 7fda31065640 1 --2- 172.21.15.67:0/2435424776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 0x55f9676e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.180+0000 7fda31866640 1 -- 172.21.15.67:0/2435424776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673184d0 msgr2=0x55f9676e7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.180+0000 7fda31866640 1 --2- 172.21.15.67:0/2435424776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673184d0 0x55f9676e7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.180+0000 7fda31866640 1 -- 172.21.15.67:0/2435424776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 msgr2=0x55f9676e7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.180+0000 7fda31866640 1 --2- 172.21.15.67:0/2435424776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 0x55f9676e7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.180+0000 7fda31866640 1 -- 172.21.15.67:0/2435424776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9673a4790 con 0x55f9672f7d80 2024-07-31T15:09:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.180+0000 7fda31065640 1 --2- 172.21.15.67:0/2435424776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 0x55f9676e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:09:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.194+0000 7fda31866640 1 --2- 172.21.15.67:0/2435424776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9672f7d80 0x55f9676fc230 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0e1b30 tx=0x7fda1c0aaff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fa1a9c95e9e9970 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.194+0000 7fda14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:09:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.194+0000 7fda14ff9640 1 -- 172.21.15.67:0/2435424776 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c0c4020 con 0x55f9672f7d80 2024-07-31T15:09:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.194+0000 7fda14ff9640 1 -- 172.21.15.67:0/2435424776 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9483+0+0 (secure 0 0 0) 0x7fda1c0e4300 con 0x55f9672f7d80 2024-07-31T15:09:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.194+0000 7fda3a1c9740 1 -- 172.21.15.67:0/2435424776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda280041e0 con 0x55f9672f7d80 2024-07-31T15:09:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.195+0000 7fda14ff9640 1 -- 172.21.15.67:0/2435424776 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fda1c0e1fc0 con 0x55f9672f7d80 2024-07-31T15:09:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.195+0000 7fda3a1c9740 10 client.28805 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.195+0000 7fda3a1c9740 1 -- 172.21.15.67:0/2435424776 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55f96741d620 con 0x55f9672f7d80 2024-07-31T15:09:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.195+0000 7fda14ff9640 1 -- 172.21.15.67:0/2435424776 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda280041e0 con 0x55f9672f7d80 2024-07-31T15:09:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.196+0000 7fda14ff9640 1 -- 172.21.15.67:0/2435424776 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fda1c0a6960 con 0x55f9672f7d80 2024-07-31T15:09:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.196+0000 7fda3a1c9740 10 client.28805 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.196+0000 7fda3a1c9740 10 client.28805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.196+0000 7fda3a1c9740 20 client.28805 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.196+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/2435424776 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f9672cd1c0 0x55f9676f0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.197+0000 7fda3a1c9740 4 client.28805 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:09:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.197+0000 7fda3a1c9740 1 -- 172.21.15.67:0/2435424776 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f967653300 con 0x55f9672cd1c0 2024-07-31T15:09:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.197+0000 7fda31065640 1 --2- 172.21.15.67:0/2435424776 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f9672cd1c0 0x55f9676f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.197+0000 7fda14ff9640 10 client.28805 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.197+0000 7fda31065640 1 --2- 172.21.15.67:0/2435424776 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f9672cd1c0 0x55f9676f0500 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d98a7086b9a6470 server_cookie=991a6bd546d3d4ce in_seq=0 out_seq=0 2024-07-31T15:09:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.198+0000 7fda14ff9640 1 -- 172.21.15.67:0/2435424776 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f967653300 con 0x55f9672cd1c0 2024-07-31T15:09:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.198+0000 7fda14ff9640 10 client.28805 handle_command_reply: tid=0 2024-07-31T15:09:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.198+0000 7fda3a1c9740 1 client.28805 shutdown 2024-07-31T15:09:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.208+0000 7fda3a1c9740 1 -- 172.21.15.67:0/2435424776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9672f7d80 msgr2=0x55f9676fc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.208+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/2435424776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9672f7d80 0x55f9676fc230 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0e1b30 tx=0x7fda1c0aaff0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.212+0000 7fda3a1c9740 1 -- 172.21.15.67:0/2435424776 shutdown_connections 2024-07-31T15:09:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.212+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/2435424776 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f9672cd1c0 0x55f9676f0500 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.212+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/2435424776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9672f7d80 0x55f9676fc230 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.212+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/2435424776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673184d0 0x55f9676e7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.212+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/2435424776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 0x55f9676e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.212+0000 7fda3a1c9740 1 -- 172.21.15.67:0/2435424776 >> 172.21.15.67:0/2435424776 conn(0x55f96732d300 msgr2=0x55f9676dcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.212+0000 7fda14ff9640 0 client.28805 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.212+0000 7fda3a1c9740 1 -- 172.21.15.67:0/2435424776 shutdown_connections 2024-07-31T15:09:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.212+0000 7fda3a1c9740 1 -- 172.21.15.67:0/2435424776 wait complete. 2024-07-31T15:09:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.219+0000 7fda3a1c9740 20 client.28805 trim_cache size 0 max 16384 2024-07-31T15:09:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.275+0000 7fda3a1c9740 1 Processor -- start 2024-07-31T15:09:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.281+0000 7fda3a1c9740 1 -- start start 2024-07-31T15:09:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.281+0000 7fda3a1c9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9672f7d80 0x55f9676e2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.282+0000 7fda3a1c9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 0x55f9676e7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.282+0000 7fda3a1c9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9673184d0 0x55f9676dd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.282+0000 7fda3a1c9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fda1000ff60 con 0x55f9673102f0 2024-07-31T15:09:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.282+0000 7fda3a1c9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fda10011aa0 con 0x55f9673184d0 2024-07-31T15:09:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.282+0000 7fda3a1c9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda10010630 con 0x55f9672f7d80 2024-07-31T15:09:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.286+0000 7fda30864640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 0x55f9676e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.286+0000 7fda30864640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 0x55f9676e7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46478/0 (socket says 172.21.15.67:46478) 2024-07-31T15:09:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.286+0000 7fda30864640 1 -- 172.21.15.67:0/3799191863 learned_addr learned my addr 172.21.15.67:0/3799191863 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.288+0000 7fda31065640 1 --2- 172.21.15.67:0/3799191863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9672f7d80 0x55f9676e2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.288+0000 7fda31866640 1 --2- 172.21.15.67:0/3799191863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9673184d0 0x55f9676dd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.289+0000 7fda31065640 1 -- 172.21.15.67:0/3799191863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9673184d0 msgr2=0x55f9676dd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.289+0000 7fda31065640 1 --2- 172.21.15.67:0/3799191863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9673184d0 0x55f9676dd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.289+0000 7fda31065640 1 -- 172.21.15.67:0/3799191863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 msgr2=0x55f9676e7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.289+0000 7fda31065640 1 --2- 172.21.15.67:0/3799191863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 0x55f9676e7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.289+0000 7fda31065640 1 -- 172.21.15.67:0/3799191863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9672cd3f0 con 0x55f9672f7d80 2024-07-31T15:09:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.289+0000 7fda31866640 1 --2- 172.21.15.67:0/3799191863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9673184d0 0x55f9676dd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:09:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.289+0000 7fda31065640 1 --2- 172.21.15.67:0/3799191863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9672f7d80 0x55f9676e2020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fda28040000 tx=0x7fda280057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f3647654c39f582 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.291+0000 7fda16ffd640 1 -- 172.21.15.67:0/3799191863 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28054040 con 0x55f9672f7d80 2024-07-31T15:09:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.291+0000 7fda16ffd640 1 -- 172.21.15.67:0/3799191863 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda28004450 con 0x55f9672f7d80 2024-07-31T15:09:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.291+0000 7fda16ffd640 1 -- 172.21.15.67:0/3799191863 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda280041e0 con 0x55f9672f7d80 2024-07-31T15:09:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.296+0000 7fda3a1c9740 1 -- 172.21.15.67:0/3799191863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9672f7d80 msgr2=0x55f9676e2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.296+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/3799191863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9672f7d80 0x55f9676e2020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fda28040000 tx=0x7fda280057f0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.296+0000 7fda3a1c9740 1 -- 172.21.15.67:0/3799191863 shutdown_connections 2024-07-31T15:09:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.296+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/3799191863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9673184d0 0x55f9676dd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.296+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/3799191863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673102f0 0x55f9676e7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.296+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/3799191863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9672f7d80 0x55f9676e2020 secure :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fda28040000 tx=0x7fda280057f0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.296+0000 7fda3a1c9740 1 -- 172.21.15.67:0/3799191863 >> 172.21.15.67:0/3799191863 conn(0x55f96732d300 msgr2=0x55f9676dcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.299+0000 7fda3a1c9740 1 -- 172.21.15.67:0/3799191863 shutdown_connections 2024-07-31T15:09:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.299+0000 7fda3a1c9740 1 -- 172.21.15.67:0/3799191863 wait complete. 2024-07-31T15:09:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.300+0000 7fda3a1c9740 1 -- start start 2024-07-31T15:09:13.302 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.300+0000 7fda3a1c9740 1 Processor -- start 2024-07-31T15:09:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.300+0000 7fda3a1c9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673102f0 0x55f9676f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.300+0000 7fda3a1c9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673184d0 0x55f9676f6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.300+0000 7fda3a1c9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9676f2d10 0x55f9676f3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.300+0000 7fda3a1c9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fda100113d0 con 0x55f9673184d0 2024-07-31T15:09:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.300+0000 7fda3a1c9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fda10010d00 con 0x55f9676f2d10 2024-07-31T15:09:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.300+0000 7fda3a1c9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda1000df60 con 0x55f9673102f0 2024-07-31T15:09:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.300+0000 7fda3a1c9740 10 client.0 _ll_register_callbacks cb 0x7fda1c0aae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.301+0000 7fda31866640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9676f2d10 0x55f9676f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.301+0000 7fda31866640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9676f2d10 0x55f9676f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:43106/0 (socket says 172.21.15.67:43106) 2024-07-31T15:09:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.301+0000 7fda31866640 1 -- 172.21.15.67:0/1407889575 learned_addr learned my addr 172.21.15.67:0/1407889575 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.301+0000 7fda31065640 1 --2- 172.21.15.67:0/1407889575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673102f0 0x55f9676f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.301+0000 7fda31866640 1 -- 172.21.15.67:0/1407889575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673102f0 msgr2=0x55f9676f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.301+0000 7fda31866640 1 --2- 172.21.15.67:0/1407889575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673102f0 0x55f9676f5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.301+0000 7fda31866640 1 -- 172.21.15.67:0/1407889575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673184d0 msgr2=0x55f9676f6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.301+0000 7fda31866640 1 --2- 172.21.15.67:0/1407889575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673184d0 0x55f9676f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.301+0000 7fda31866640 1 -- 172.21.15.67:0/1407889575 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9676f3810 con 0x55f9676f2d10 2024-07-31T15:09:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.301+0000 7fda31065640 1 --2- 172.21.15.67:0/1407889575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673102f0 0x55f9676f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.302+0000 7fda31866640 1 --2- 172.21.15.67:0/1407889575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9676f2d10 0x55f9676f3120 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0ab8a0 tx=0x7fda1c0aa690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2996e2423b906cb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.311+0000 7fda14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:09:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.311+0000 7fda14ff9640 1 -- 172.21.15.67:0/1407889575 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c0c4040 con 0x55f9676f2d10 2024-07-31T15:09:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.311+0000 7fda14ff9640 1 -- 172.21.15.67:0/1407889575 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9483+0+0 (secure 0 0 0) 0x7fda1c0aa250 con 0x55f9676f2d10 2024-07-31T15:09:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.316+0000 7fda3a1c9740 1 -- 172.21.15.67:0/1407889575 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda28004450 con 0x55f9676f2d10 2024-07-31T15:09:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.317+0000 7fda14ff9640 1 -- 172.21.15.67:0/1407889575 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fda1c0bc050 con 0x55f9676f2d10 2024-07-31T15:09:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.317+0000 7fda3a1c9740 10 client.28811 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.317+0000 7fda3a1c9740 1 -- 172.21.15.67:0/1407889575 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55f9677724f0 con 0x55f9676f2d10 2024-07-31T15:09:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.317+0000 7fda14ff9640 1 -- 172.21.15.67:0/1407889575 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda28004450 con 0x55f9676f2d10 2024-07-31T15:09:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.318+0000 7fda14ff9640 1 -- 172.21.15.67:0/1407889575 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fda1c0b5070 con 0x55f9676f2d10 2024-07-31T15:09:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.318+0000 7fda3a1c9740 10 client.28811 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.318+0000 7fda3a1c9740 10 client.28811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.318+0000 7fda3a1c9740 20 client.28811 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.318+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/1407889575 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f967772c20 0x55f967775060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.318+0000 7fda3a1c9740 4 client.28811 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:09:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.318+0000 7fda3a1c9740 1 -- 172.21.15.67:0/1407889575 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f96764eda0 con 0x55f967772c20 2024-07-31T15:09:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.318+0000 7fda31065640 1 --2- 172.21.15.67:0/1407889575 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f967772c20 0x55f967775060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.319+0000 7fda14ff9640 10 client.28811 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:13.319+0000 7fda31065640 1 --2- 172.21.15.67:0/1407889575 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f967772c20 0x55f967775060 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f586a6eae06243 server_cookie=f3a9ba52496bdf6c in_seq=0 out_seq=0 2024-07-31T15:09:13.870 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_gss/gss_krb5_keys.o 2024-07-31T15:09:13.882 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp_ulp.o 2024-07-31T15:09:13.882 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/gen7_renderclear.o 2024-07-31T15:09:14.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:13 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:09:14.143 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:13 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:09:14.212 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/addrlabel.o 2024-07-31T15:09:14.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:13 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:09:14.256 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/xt_NFLOG.o 2024-07-31T15:09:15.018 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/sysfs.o 2024-07-31T15:09:15.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:14 smithi067 ceph-mon[32750]: pgmap v977: 129 pgs: 8 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 95 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 825 KiB/s wr, 67 op/s 2024-07-31T15:09:15.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.812+0000 7f842ffff640 1 -- 172.21.15.67:0/4214883209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f843010a330 msgr2=0x7f843010a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:15.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.812+0000 7f842ffff640 1 --2- 172.21.15.67:0/4214883209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f843010a330 0x7f843010a740 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8420009900 tx=0x7f842002a4d0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.815+0000 7f842ffff640 1 -- 172.21.15.67:0/4214883209 shutdown_connections 2024-07-31T15:09:15.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.815+0000 7f842ffff640 1 --2- 172.21.15.67:0/4214883209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f843011c5d0 0x7f843011e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.815+0000 7f842ffff640 1 --2- 172.21.15.67:0/4214883209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f843010a330 0x7f843010a740 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.815+0000 7f842ffff640 1 --2- 172.21.15.67:0/4214883209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8430111790 0x7f8430109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.815+0000 7f842ffff640 1 -- 172.21.15.67:0/4214883209 >> 172.21.15.67:0/4214883209 conn(0x7f843006e420 msgr2=0x7f843006e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.818+0000 7f842ffff640 1 -- 172.21.15.67:0/4214883209 shutdown_connections 2024-07-31T15:09:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.818+0000 7f842ffff640 1 -- 172.21.15.67:0/4214883209 wait complete. 2024-07-31T15:09:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.819+0000 7f842ffff640 1 Processor -- start 2024-07-31T15:09:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.819+0000 7f842ffff640 1 -- start start 2024-07-31T15:09:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.819+0000 7f842ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f843010a330 0x7f84301a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.819+0000 7f842ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8430111790 0x7f84301a90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.819+0000 7f842ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f843011c5d0 0x7f84301ae390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.819+0000 7f842ffff640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f84301220e0 con 0x7f843011c5d0 2024-07-31T15:09:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.819+0000 7f842ffff640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8430121f60 con 0x7f8430111790 2024-07-31T15:09:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.819+0000 7f842ffff640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8430122260 con 0x7f843010a330 2024-07-31T15:09:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.819+0000 7f842effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f843010a330 0x7f84301a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:15.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.820+0000 7f842effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f843010a330 0x7f84301a8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34490/0 (socket says 172.21.15.67:34490) 2024-07-31T15:09:15.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.820+0000 7f842effd640 1 -- 172.21.15.67:0/1891206840 learned_addr learned my addr 172.21.15.67:0/1891206840 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:15.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.820+0000 7f842e7fc640 1 --2- 172.21.15.67:0/1891206840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8430111790 0x7f84301a90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:15.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.823+0000 7f842effd640 1 -- 172.21.15.67:0/1891206840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8430111790 msgr2=0x7f84301a90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:15.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.823+0000 7f842effd640 1 --2- 172.21.15.67:0/1891206840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8430111790 0x7f84301a90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.823+0000 7f842effd640 1 -- 172.21.15.67:0/1891206840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f843011c5d0 msgr2=0x7f84301ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:15.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.823+0000 7f842effd640 1 --2- 172.21.15.67:0/1891206840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f843011c5d0 0x7f84301ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.823+0000 7f842effd640 1 -- 172.21.15.67:0/1891206840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84301aeac0 con 0x7f843010a330 2024-07-31T15:09:15.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.823+0000 7f842effd640 1 --2- 172.21.15.67:0/1891206840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f843010a330 0x7f84301a8b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f842400e9f0 tx=0x7f842400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:15.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.829+0000 7f843490b640 1 -- 172.21.15.67:0/1891206840 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842400cd80 con 0x7f843010a330 2024-07-31T15:09:15.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.829+0000 7f842ffff640 1 -- 172.21.15.67:0/1891206840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84301aedb0 con 0x7f843010a330 2024-07-31T15:09:15.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.829+0000 7f842ffff640 1 -- 172.21.15.67:0/1891206840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843007b050 con 0x7f843010a330 2024-07-31T15:09:15.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.830+0000 7f843490b640 1 -- 172.21.15.67:0/1891206840 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84240108e0 con 0x7f843010a330 2024-07-31T15:09:15.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.830+0000 7f843490b640 1 -- 172.21.15.67:0/1891206840 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424018880 con 0x7f843010a330 2024-07-31T15:09:15.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.838+0000 7f84167fc640 1 -- 172.21.15.67:0/1891206840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841c0052c0 con 0x7f843010a330 2024-07-31T15:09:15.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.844+0000 7f843490b640 1 -- 172.21.15.67:0/1891206840 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8424010430 con 0x7f843010a330 2024-07-31T15:09:15.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.845+0000 7f843490b640 1 --2- 172.21.15.67:0/1891206840 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f84180b3f50 0x7f84180b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:15.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.845+0000 7f843490b640 1 -- 172.21.15.67:0/1891206840 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9483+0+0 (secure 0 0 0) 0x7f84240e4860 con 0x7f843010a330 2024-07-31T15:09:15.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.847+0000 7f842e7fc640 1 --2- 172.21.15.67:0/1891206840 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f84180b3f50 0x7f84180b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:15.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.850+0000 7f842e7fc640 1 --2- 172.21.15.67:0/1891206840 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f84180b3f50 0x7f84180b6460 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f843011bb60 tx=0x7f8420000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:15.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:14.851+0000 7f843490b640 1 -- 172.21.15.67:0/1891206840 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f84240555e0 con 0x7f843010a330 2024-07-31T15:09:15.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:14 smithi073 ceph-mon[36714]: pgmap v977: 129 pgs: 8 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 95 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 825 KiB/s wr, 67 op/s 2024-07-31T15:09:15.149 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/constants.o 2024-07-31T15:09:15.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:14 smithi077 ceph-mon[37327]: pgmap v977: 129 pgs: 8 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 95 active+clean; 1.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 825 KiB/s wr, 67 op/s 2024-07-31T15:09:15.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:15.361+0000 7f84167fc640 1 -- 172.21.15.67:0/1891206840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f841c005880 con 0x7f843010a330 2024-07-31T15:09:15.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:15.363+0000 7f843490b640 1 -- 172.21.15.67:0/1891206840 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11962 (secure 0 0 0) 0x7f84240ab750 con 0x7f843010a330 2024-07-31T15:09:15.366 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:09:15.366 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"btime":"2024-07-31T15:06:04:323642+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:06:04.323492+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28214":{"gid":28214,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1516494761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1516494761},{"type":"v1","addr":"172.21.15.73:6835","nonce":1516494761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:09:15.366 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-07-31T15:09:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:15.376+0000 7f84167fc640 1 -- 172.21.15.67:0/1891206840 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f84180b3f50 msgr2=0x7f84180b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:15.376+0000 7f84167fc640 1 --2- 172.21.15.67:0/1891206840 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f84180b3f50 0x7f84180b6460 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f843011bb60 tx=0x7f8420000bc0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:15.376+0000 7f84167fc640 1 -- 172.21.15.67:0/1891206840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f843010a330 msgr2=0x7f84301a8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:15.376+0000 7f84167fc640 1 --2- 172.21.15.67:0/1891206840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f843010a330 0x7f84301a8b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f842400e9f0 tx=0x7f842400eeb0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:15.376+0000 7f84167fc640 1 -- 172.21.15.67:0/1891206840 shutdown_connections 2024-07-31T15:09:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:15.376+0000 7f84167fc640 1 --2- 172.21.15.67:0/1891206840 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f84180b3f50 0x7f84180b6460 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:15.376+0000 7f84167fc640 1 --2- 172.21.15.67:0/1891206840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f843011c5d0 0x7f84301ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:15.376+0000 7f84167fc640 1 --2- 172.21.15.67:0/1891206840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8430111790 0x7f84301a90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:15.376+0000 7f84167fc640 1 --2- 172.21.15.67:0/1891206840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f843010a330 0x7f84301a8b50 secure :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f842400e9f0 tx=0x7f842400eeb0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:15.376+0000 7f84167fc640 1 -- 172.21.15.67:0/1891206840 >> 172.21.15.67:0/1891206840 conn(0x7f843006e420 msgr2=0x7f8430071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:15.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:15.378+0000 7f84167fc640 1 -- 172.21.15.67:0/1891206840 shutdown_connections 2024-07-31T15:09:15.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:15.378+0000 7f84167fc640 1 -- 172.21.15.67:0/1891206840 wait complete. 2024-07-31T15:09:15.402 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/xprt.o 2024-07-31T15:09:15.428 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/gen8_engine_cs.o 2024-07-31T15:09:15.451 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/sunrpc/auth_gss/built-in.a 2024-07-31T15:09:15.547 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/gen8_ppgtt.o 2024-07-31T15:09:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.568+0000 7fda14ff9640 1 -- 172.21.15.67:0/1407889575 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f96764eda0 con 0x55f967772c20 2024-07-31T15:09:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.568+0000 7fda14ff9640 10 client.28811 handle_command_reply: tid=0 2024-07-31T15:09:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.568+0000 7fda3a1c9740 1 client.28811 shutdown 2024-07-31T15:09:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.576+0000 7fda3a1c9740 1 -- 172.21.15.67:0/1407889575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9676f2d10 msgr2=0x55f9676f3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.576+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/1407889575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9676f2d10 0x55f9676f3120 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0ab8a0 tx=0x7fda1c0aa690 comp rx=0 tx=0).stop 2024-07-31T15:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.580+0000 7fda3a1c9740 1 -- 172.21.15.67:0/1407889575 shutdown_connections 2024-07-31T15:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.580+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/1407889575 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55f967772c20 0x55f967775060 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.580+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/1407889575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9676f2d10 0x55f9676f3120 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.580+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/1407889575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9673184d0 0x55f9676f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.580+0000 7fda3a1c9740 1 --2- 172.21.15.67:0/1407889575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9673102f0 0x55f9676f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.580+0000 7fda3a1c9740 1 -- 172.21.15.67:0/1407889575 >> 172.21.15.67:0/1407889575 conn(0x55f96732d300 msgr2=0x55f9676dd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.580+0000 7fda14ff9640 0 client.28811 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.580+0000 7fda3a1c9740 1 -- 172.21.15.67:0/1407889575 shutdown_connections 2024-07-31T15:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.581+0000 7fda3a1c9740 1 -- 172.21.15.67:0/1407889575 wait complete. 2024-07-31T15:09:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.581+0000 7fda3a1c9740 20 client.28811 trim_cache size 0 max 16384 2024-07-31T15:09:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:09:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:09:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:09:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:09:15.591 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/xt_SECMARK.o 2024-07-31T15:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.594+0000 7fda32067640 1 -- 172.21.15.67:0/2915381756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fda180b3f50 msgr2=0x7fda180b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.594+0000 7fda32067640 1 --2- 172.21.15.67:0/2915381756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fda180b3f50 0x7fda180b6460 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fda28036d30 tx=0x7fda28036d60 comp rx=0 tx=0).stop 2024-07-31T15:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.594+0000 7fda32067640 1 -- 172.21.15.67:0/2915381756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda2c10c940 msgr2=0x7fda2c080a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.594+0000 7fda32067640 1 --2- 172.21.15.67:0/2915381756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda2c10c940 0x7fda2c080a20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00a850 tx=0x7fda1c00ad10 comp rx=0 tx=0).stop 2024-07-31T15:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.595+0000 7fda32067640 1 -- 172.21.15.67:0/2915381756 shutdown_connections 2024-07-31T15:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.595+0000 7fda32067640 1 --2- 172.21.15.67:0/2915381756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fda180b3f50 0x7fda180b6460 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.595+0000 7fda32067640 1 --2- 172.21.15.67:0/2915381756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda2c10c940 0x7fda2c080a20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.598+0000 7fda32067640 1 --2- 172.21.15.67:0/2915381756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda2c10bff0 0x7fda2c0804e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.598+0000 7fda32067640 1 --2- 172.21.15.67:0/2915381756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda2c10a090 0x7fda2c07ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.598+0000 7fda32067640 1 -- 172.21.15.67:0/2915381756 >> 172.21.15.67:0/2915381756 conn(0x7fda2c06e320 msgr2=0x7fda2c10d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.598+0000 7fda32067640 1 -- 172.21.15.67:0/2915381756 shutdown_connections 2024-07-31T15:09:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:15.598+0000 7fda32067640 1 -- 172.21.15.67:0/2915381756 wait complete. 2024-07-31T15:09:15.635 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp_offload.o 2024-07-31T15:09:16.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:15 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1891206840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:09:16.199 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/route.o 2024-07-31T15:09:16.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:15 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1891206840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:09:16.311 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T15:09:16.370 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_lib_dma.o 2024-07-31T15:09:16.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:15 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1891206840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:09:16.918 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/xt_TCPMSS.o 2024-07-31T15:09:17.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.001+0000 7f531a829640 1 -- 172.21.15.67:0/1847984213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53141099f0 msgr2=0x7f5314109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:17.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.001+0000 7f531a829640 1 --2- 172.21.15.67:0/1847984213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53141099f0 0x7f5314109e00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f530c009ca0 tx=0x7f530c02a8f0 comp rx=0 tx=0).stop 2024-07-31T15:09:17.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.002+0000 7f531a829640 1 -- 172.21.15.67:0/1847984213 shutdown_connections 2024-07-31T15:09:17.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.002+0000 7f531a829640 1 --2- 172.21.15.67:0/1847984213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f531410fc80 0x7f53141164b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:17.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.002+0000 7f531a829640 1 --2- 172.21.15.67:0/1847984213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f531410f330 0x7f531410f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:17.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.002+0000 7f531a829640 1 --2- 172.21.15.67:0/1847984213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53141099f0 0x7f5314109e00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:17.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.002+0000 7f531a829640 1 -- 172.21.15.67:0/1847984213 >> 172.21.15.67:0/1847984213 conn(0x7f5314078ff0 msgr2=0x7f5314107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:17.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.007+0000 7f531a829640 1 -- 172.21.15.67:0/1847984213 shutdown_connections 2024-07-31T15:09:17.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.007+0000 7f531a829640 1 -- 172.21.15.67:0/1847984213 wait complete. 2024-07-31T15:09:17.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.008+0000 7f531a829640 1 Processor -- start 2024-07-31T15:09:17.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.011+0000 7f531a829640 1 -- start start 2024-07-31T15:09:17.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.011+0000 7f531a829640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53141099f0 0x7f53141a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:17.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.012+0000 7f531a829640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f531410f330 0x7f53141a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:17.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.012+0000 7f531a829640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531410fc80 0x7f53141a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:17.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.012+0000 7f531a829640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5314119990 con 0x7f53141099f0 2024-07-31T15:09:17.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.012+0000 7f531a829640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5314119810 con 0x7f531410f330 2024-07-31T15:09:17.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.012+0000 7f531a829640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5314119b10 con 0x7f531410fc80 2024-07-31T15:09:17.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.012+0000 7f5319827640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53141099f0 0x7f53141a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:17.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.012+0000 7f5319827640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53141099f0 0x7f53141a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46536/0 (socket says 172.21.15.67:46536) 2024-07-31T15:09:17.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.012+0000 7f5319827640 1 -- 172.21.15.67:0/2928182798 learned_addr learned my addr 172.21.15.67:0/2928182798 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:17.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.013+0000 7f5319827640 1 -- 172.21.15.67:0/2928182798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531410fc80 msgr2=0x7f53141a5b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:09:17.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.013+0000 7f5319827640 1 --2- 172.21.15.67:0/2928182798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531410fc80 0x7f53141a5b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.013+0000 7f5319827640 1 -- 172.21.15.67:0/2928182798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f531410f330 msgr2=0x7f53141a08c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:09:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.013+0000 7f5319827640 1 --2- 172.21.15.67:0/2928182798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f531410f330 0x7f53141a08c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.013+0000 7f5319827640 1 -- 172.21.15.67:0/2928182798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53141a6290 con 0x7f53141099f0 2024-07-31T15:09:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.014+0000 7f5319827640 1 --2- 172.21.15.67:0/2928182798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53141099f0 0x7f53141a0320 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f530c02aee0 tx=0x7f530c0305f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.014+0000 7f530affd640 1 -- 172.21.15.67:0/2928182798 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c030960 con 0x7f53141099f0 2024-07-31T15:09:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.014+0000 7f530affd640 1 -- 172.21.15.67:0/2928182798 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f530c02d070 con 0x7f53141099f0 2024-07-31T15:09:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.014+0000 7f530affd640 1 -- 172.21.15.67:0/2928182798 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c03be50 con 0x7f53141099f0 2024-07-31T15:09:17.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.018+0000 7f531a829640 1 -- 172.21.15.67:0/2928182798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53141a6520 con 0x7f53141099f0 2024-07-31T15:09:17.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.018+0000 7f531a829640 1 -- 172.21.15.67:0/2928182798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53141acce0 con 0x7f53141099f0 2024-07-31T15:09:17.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.021+0000 7f531a829640 1 -- 172.21.15.67:0/2928182798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f531410ac70 con 0x7f53141099f0 2024-07-31T15:09:17.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.033+0000 7f530affd640 1 -- 172.21.15.67:0/2928182798 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f530c03b5b0 con 0x7f53141099f0 2024-07-31T15:09:17.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.037+0000 7f530affd640 1 --2- 172.21.15.67:0/2928182798 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f53000b40f0 0x7f53000b6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:17.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.038+0000 7f530affd640 1 -- 172.21.15.67:0/2928182798 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9483+0+0 (secure 0 0 0) 0x7f530c1053c0 con 0x7f53141099f0 2024-07-31T15:09:17.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.038+0000 7f530affd640 1 -- 172.21.15.67:0/2928182798 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f530c049060 con 0x7f53141099f0 2024-07-31T15:09:17.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.042+0000 7f5319026640 1 --2- 172.21.15.67:0/2928182798 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f53000b40f0 0x7f53000b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:17.048 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:09:17.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.043+0000 7f5319026640 1 --2- 172.21.15.67:0/2928182798 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f53000b40f0 0x7f53000b6600 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f531410fa50 tx=0x7f5310005b80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:17.126 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/radiotap.o 2024-07-31T15:09:17.190 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_scan.o 2024-07-31T15:09:17.201 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/socklib.o 2024-07-31T15:09:17.202 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:16 smithi067 ceph-mon[32750]: pgmap v978: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-07-31T15:09:17.202 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:09:17.202 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:09:17.202 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:16 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:09:17.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:16 smithi077 ceph-mon[37327]: pgmap v978: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-07-31T15:09:17.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:09:17.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:09:17.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:16 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:09:17.255 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp_plb.o 2024-07-31T15:09:17.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:16 smithi073 ceph-mon[36714]: pgmap v978: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-07-31T15:09:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:09:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:09:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:16 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:09:17.526 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_breadcrumbs.o 2024-07-31T15:09:17.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.553+0000 7f531a829640 1 -- 172.21.15.67:0/2928182798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5314109e00 con 0x7f53141099f0 2024-07-31T15:09:17.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.556+0000 7f530affd640 1 -- 172.21.15.67:0/2928182798 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11962 (secure 0 0 0) 0x7f530c0cc440 con 0x7f53141099f0 2024-07-31T15:09:17.559 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:09:17.559 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"btime":"2024-07-31T15:06:04:323642+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:06:04.323492+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28214":{"gid":28214,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1516494761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1516494761},{"type":"v1","addr":"172.21.15.73:6835","nonce":1516494761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:09:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-07-31T15:09:17.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.571+0000 7f531a829640 1 -- 172.21.15.67:0/2928182798 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f53000b40f0 msgr2=0x7f53000b6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:17.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.571+0000 7f531a829640 1 --2- 172.21.15.67:0/2928182798 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f53000b40f0 0x7f53000b6600 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f531410fa50 tx=0x7f5310005b80 comp rx=0 tx=0).stop 2024-07-31T15:09:17.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.571+0000 7f531a829640 1 -- 172.21.15.67:0/2928182798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53141099f0 msgr2=0x7f53141a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:17.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.571+0000 7f531a829640 1 --2- 172.21.15.67:0/2928182798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53141099f0 0x7f53141a0320 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f530c02aee0 tx=0x7f530c0305f0 comp rx=0 tx=0).stop 2024-07-31T15:09:17.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.578+0000 7f531a829640 1 -- 172.21.15.67:0/2928182798 shutdown_connections 2024-07-31T15:09:17.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.578+0000 7f531a829640 1 --2- 172.21.15.67:0/2928182798 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f53000b40f0 0x7f53000b6600 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:17.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.578+0000 7f531a829640 1 --2- 172.21.15.67:0/2928182798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531410fc80 0x7f53141a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:17.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.578+0000 7f531a829640 1 --2- 172.21.15.67:0/2928182798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f531410f330 0x7f53141a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:17.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.578+0000 7f531a829640 1 --2- 172.21.15.67:0/2928182798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53141099f0 0x7f53141a0320 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:17.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.578+0000 7f531a829640 1 -- 172.21.15.67:0/2928182798 >> 172.21.15.67:0/2928182798 conn(0x7f5314078ff0 msgr2=0x7f5314107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:17.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.582+0000 7f531a829640 1 -- 172.21.15.67:0/2928182798 shutdown_connections 2024-07-31T15:09:17.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:17.582+0000 7f531a829640 1 -- 172.21.15.67:0/2928182798 wait complete. 2024-07-31T15:09:17.937 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_context.o 2024-07-31T15:09:17.979 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/util.o 2024-07-31T15:09:18.074 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/xt_conntrack.o 2024-07-31T15:09:18.109 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/xprtsock.o 2024-07-31T15:09:18.230 INFO:tasks.workunit.client.0.smithi067.stdout: GEN drivers/scsi/scsi_devinfo_tbl.c 2024-07-31T15:09:18.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:17 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2928182798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:09:18.240 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_devinfo.o 2024-07-31T15:09:18.274 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:17 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2928182798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:09:18.333 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/datagram.o 2024-07-31T15:09:18.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:17 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2928182798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:09:18.704 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/raw.o 2024-07-31T15:09:18.957 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_sysctl.o 2024-07-31T15:09:19.041 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_context_sseu.o 2024-07-31T15:09:19.060 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/xt_policy.o 2024-07-31T15:09:19.202 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:18 smithi067 ceph-mon[32750]: pgmap v979: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 926 KiB/s wr, 132 op/s 2024-07-31T15:09:19.205 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 138.1 --await-for 45.0' 2024-07-31T15:09:19.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 138.1 --await-for 45.0 2024-07-31T15:09:19.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:18 smithi077 ceph-mon[37327]: pgmap v979: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 926 KiB/s wr, 132 op/s 2024-07-31T15:09:19.327 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netfilter/xt_state.o 2024-07-31T15:09:19.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:18 smithi073 ceph-mon[36714]: pgmap v979: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 926 KiB/s wr, 132 op/s 2024-07-31T15:09:19.598 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_proc.o 2024-07-31T15:09:19.841 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/ip6_fib.o 2024-07-31T15:09:19.955 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_engine_cs.o 2024-07-31T15:09:20.068 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] net/netfilter/nf_log_syslog.o 2024-07-31T15:09:20.117 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/udp.o 2024-07-31T15:09:20.431 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] net/netfilter/xt_mark.o 2024-07-31T15:09:20.453 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_debugfs.o 2024-07-31T15:09:20.459 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/reg.o 2024-07-31T15:09:20.499 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:09:20.537 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/sched.o 2024-07-31T15:09:20.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.548+0000 7f06397eb640 1 -- 172.21.15.67:0/2955265171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0634111790 msgr2=0x7f0634109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:20.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.548+0000 7f06397eb640 1 --2- 172.21.15.67:0/2955265171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0634111790 0x7f0634109df0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f062c009be0 tx=0x7f062c02a770 comp rx=0 tx=0).stop 2024-07-31T15:09:20.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.548+0000 7f06397eb640 1 -- 172.21.15.67:0/2955265171 shutdown_connections 2024-07-31T15:09:20.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.548+0000 7f06397eb640 1 --2- 172.21.15.67:0/2955265171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063411c5d0 0x7f063411e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.548+0000 7f06397eb640 1 --2- 172.21.15.67:0/2955265171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f063410a330 0x7f063410a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.548+0000 7f06397eb640 1 --2- 172.21.15.67:0/2955265171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0634111790 0x7f0634109df0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.548+0000 7f06397eb640 1 -- 172.21.15.67:0/2955265171 >> 172.21.15.67:0/2955265171 conn(0x7f063406e420 msgr2=0x7f063406e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:20.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.550+0000 7f06397eb640 1 -- 172.21.15.67:0/2955265171 shutdown_connections 2024-07-31T15:09:20.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.550+0000 7f06397eb640 1 -- 172.21.15.67:0/2955265171 wait complete. 2024-07-31T15:09:20.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.551+0000 7f06397eb640 1 Processor -- start 2024-07-31T15:09:20.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.552+0000 7f06397eb640 1 -- start start 2024-07-31T15:09:20.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.552+0000 7f06397eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063410a330 0x7f06341a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.552+0000 7f06397eb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0634111790 0x7f06341a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.552+0000 7f06397eb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f063411c5d0 0x7f06341ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.552+0000 7f06397eb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f06341220f0 con 0x7f063411c5d0 2024-07-31T15:09:20.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.552+0000 7f06397eb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0634121f70 con 0x7f0634111790 2024-07-31T15:09:20.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.552+0000 7f06397eb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0634122270 con 0x7f063410a330 2024-07-31T15:09:20.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.553+0000 7f0638fea640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f063411c5d0 0x7f06341ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:20.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.553+0000 7f0638fea640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f063411c5d0 0x7f06341ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46552/0 (socket says 172.21.15.67:46552) 2024-07-31T15:09:20.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.553+0000 7f0638fea640 1 -- 172.21.15.67:0/3254147581 learned_addr learned my addr 172.21.15.67:0/3254147581 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:20.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.556+0000 7f0633fff640 1 --2- 172.21.15.67:0/3254147581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063410a330 0x7f06341a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:20.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.561+0000 7f0638fea640 1 -- 172.21.15.67:0/3254147581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063410a330 msgr2=0x7f06341a8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:20.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.561+0000 7f0638fea640 1 --2- 172.21.15.67:0/3254147581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063410a330 0x7f06341a8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.561+0000 7f0638fea640 1 -- 172.21.15.67:0/3254147581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0634111790 msgr2=0x7f06341a90a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:09:20.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.561+0000 7f0638fea640 1 --2- 172.21.15.67:0/3254147581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0634111790 0x7f06341a90a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.561+0000 7f0638fea640 1 -- 172.21.15.67:0/3254147581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06341aea40 con 0x7f063411c5d0 2024-07-31T15:09:20.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.561+0000 7f0638fea640 1 --2- 172.21.15.67:0/3254147581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f063411c5d0 0x7f06341ae340 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f062400e970 tx=0x7f062400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:20.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.563+0000 7f06317fa640 1 -- 172.21.15.67:0/3254147581 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062400ccc0 con 0x7f063411c5d0 2024-07-31T15:09:20.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.563+0000 7f06317fa640 1 -- 172.21.15.67:0/3254147581 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f062400ce60 con 0x7f063411c5d0 2024-07-31T15:09:20.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.563+0000 7f06317fa640 1 -- 172.21.15.67:0/3254147581 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624010430 con 0x7f063411c5d0 2024-07-31T15:09:20.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.564+0000 7f06397eb640 1 -- 172.21.15.67:0/3254147581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06341aed30 con 0x7f063411c5d0 2024-07-31T15:09:20.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.564+0000 7f06397eb640 1 -- 172.21.15.67:0/3254147581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0634111db0 con 0x7f063411c5d0 2024-07-31T15:09:20.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.569+0000 7f0640f9e740 1 Processor -- start 2024-07-31T15:09:20.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.576+0000 7f06317fa640 1 -- 172.21.15.67:0/3254147581 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f06240046c0 con 0x7f063411c5d0 2024-07-31T15:09:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.578+0000 7f0640f9e740 1 -- start start 2024-07-31T15:09:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.578+0000 7f0640f9e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 0x557a3f7a2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.578+0000 7f0640f9e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.578+0000 7f0640f9e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3dc390 0x557a3f7a3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.578+0000 7f0640f9e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557a3f718bb0 con 0x557a3f3fd070 2024-07-31T15:09:20.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.578+0000 7f0640f9e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a3f7080b0 con 0x557a3f3d41b0 2024-07-31T15:09:20.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.578+0000 7f0640f9e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a3f70c0f0 con 0x557a3f3dc390 2024-07-31T15:09:20.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.579+0000 7f0633fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:20.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.579+0000 7f0633fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7a2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:43158/0 (socket says 172.21.15.67:43158) 2024-07-31T15:09:20.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.579+0000 7f0633fff640 1 -- 172.21.15.67:0/3227986771 learned_addr learned my addr 172.21.15.67:0/3227986771 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:20.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.579+0000 7f06317fa640 1 --2- 172.21.15.67:0/3254147581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f061c0b3f50 0x7f061c0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.579+0000 7f06317fa640 1 -- 172.21.15.67:0/3254147581 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9395+0+0 (secure 0 0 0) 0x7f06240e4820 con 0x7f063411c5d0 2024-07-31T15:09:20.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.581+0000 7f06337fe640 1 --2- 172.21.15.67:0/3227986771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 0x557a3f7a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:20.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.581+0000 7f06337fe640 1 --2- 172.21.15.67:0/3227986771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3dc390 0x557a3f7a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:20.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.582+0000 7f0633fff640 1 -- 172.21.15.67:0/3227986771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3dc390 msgr2=0x557a3f7a3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:20.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.582+0000 7f0633fff640 1 --2- 172.21.15.67:0/3227986771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3dc390 0x557a3f7a3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.582+0000 7f0633fff640 1 -- 172.21.15.67:0/3227986771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 msgr2=0x557a3f7a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:20.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.582+0000 7f0633fff640 1 --2- 172.21.15.67:0/3227986771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 0x557a3f7a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.582+0000 7f0633fff640 1 -- 172.21.15.67:0/3227986771 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a3f6aeb90 con 0x557a3f3d41b0 2024-07-31T15:09:20.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.582+0000 7f0633fff640 1 --2- 172.21.15.67:0/3227986771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7a2c50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f062c009b30 tx=0x7f062c0303a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c4cfff7f5404125 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:20.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.580+0000 7f0638fea640 1 --2- 172.21.15.67:0/3254147581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f061c0b3f50 0x7f061c0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:20.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.583+0000 7f0638fea640 1 --2- 172.21.15.67:0/3254147581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f061c0b3f50 0x7f061c0b6460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f06240aaff0 tx=0x7f06240056d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.590+0000 7f06337fe640 1 --2- 172.21.15.67:0/3227986771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 0x557a3f7a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:20.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.590+0000 7f061a7fc640 1 -- 172.21.15.67:0/3227986771 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c042070 con 0x557a3f3d41b0 2024-07-31T15:09:20.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.590+0000 7f061a7fc640 1 -- 172.21.15.67:0/3227986771 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f062c030da0 con 0x557a3f3d41b0 2024-07-31T15:09:20.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.590+0000 7f061a7fc640 1 -- 172.21.15.67:0/3227986771 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c036040 con 0x557a3f3d41b0 2024-07-31T15:09:20.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.595+0000 7f0640f9e740 1 -- 172.21.15.67:0/3227986771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 msgr2=0x557a3f7a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:20.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.595+0000 7f0640f9e740 1 --2- 172.21.15.67:0/3227986771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7a2c50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f062c009b30 tx=0x7f062c0303a0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.599+0000 7f0640f9e740 1 -- 172.21.15.67:0/3227986771 shutdown_connections 2024-07-31T15:09:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.599+0000 7f0640f9e740 1 --2- 172.21.15.67:0/3227986771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3dc390 0x557a3f7a3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.599+0000 7f0640f9e740 1 --2- 172.21.15.67:0/3227986771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7a2c50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.599+0000 7f0640f9e740 1 --2- 172.21.15.67:0/3227986771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 0x557a3f7a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.599+0000 7f0640f9e740 1 -- 172.21.15.67:0/3227986771 >> 172.21.15.67:0/3227986771 conn(0x557a3f3f1460 msgr2=0x557a3f7a0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.599+0000 7f0640f9e740 1 -- 172.21.15.67:0/3227986771 shutdown_connections 2024-07-31T15:09:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.603+0000 7f0640f9e740 1 -- 172.21.15.67:0/3227986771 wait complete. 2024-07-31T15:09:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.603+0000 7f0640f9e740 1 -- start start 2024-07-31T15:09:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.603+0000 7f0640f9e740 1 Processor -- start 2024-07-31T15:09:20.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.607+0000 7f0640f9e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.607+0000 7f0640f9e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3dc390 0x557a3f7ae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.607+0000 7f0640f9e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 0x557a3f7ba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.607+0000 7f0640f9e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557a3f56b700 con 0x557a3f3fd070 2024-07-31T15:09:20.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.607+0000 7f0640f9e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a3f6aa9e0 con 0x557a3f3d41b0 2024-07-31T15:09:20.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.607+0000 7f0640f9e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a3f7b0e90 con 0x557a3f3dc390 2024-07-31T15:09:20.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.607+0000 7f0640f9e740 10 client.0 _ll_register_callbacks cb 0x557a3f6c54f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:20.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.608+0000 7f06337fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 0x557a3f7ba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:20.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.608+0000 7f06337fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 0x557a3f7ba180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46576/0 (socket says 172.21.15.67:46576) 2024-07-31T15:09:20.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.608+0000 7f06337fe640 1 -- 172.21.15.67:0/1356241794 learned_addr learned my addr 172.21.15.67:0/1356241794 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:20.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.608+0000 7f06337fe640 1 --2- 172.21.15.67:0/1356241794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:20.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.608+0000 7f06337fe640 1 -- 172.21.15.67:0/1356241794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3dc390 msgr2=0x557a3f7ae970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:09:20.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.608+0000 7f06337fe640 1 --2- 172.21.15.67:0/1356241794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3dc390 0x557a3f7ae970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.608+0000 7f06337fe640 1 -- 172.21.15.67:0/1356241794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 msgr2=0x557a3f7ba180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:20.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.608+0000 7f06337fe640 1 --2- 172.21.15.67:0/1356241794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 0x557a3f7ba180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.608+0000 7f06337fe640 1 -- 172.21.15.67:0/1356241794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a3f468650 con 0x557a3f3d41b0 2024-07-31T15:09:20.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.608+0000 7f06337fe640 1 --2- 172.21.15.67:0/1356241794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7ae3b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0628014f30 tx=0x7f062800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffe26e6bd5f907d4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:20.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.612+0000 7f06137fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:09:20.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.612+0000 7f06137fe640 1 -- 172.21.15.67:0/1356241794 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628012650 con 0x557a3f3d41b0 2024-07-31T15:09:20.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.612+0000 7f06137fe640 1 -- 172.21.15.67:0/1356241794 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9395+0+0 (secure 0 0 0) 0x7f06280127f0 con 0x557a3f3d41b0 2024-07-31T15:09:20.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.613+0000 7f0640f9e740 1 -- 172.21.15.67:0/1356241794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f062c030da0 con 0x557a3f3d41b0 2024-07-31T15:09:20.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.618+0000 7f06137fe640 1 -- 172.21.15.67:0/1356241794 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f0628027050 con 0x557a3f3d41b0 2024-07-31T15:09:20.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.619+0000 7f0640f9e740 10 client.28826 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:20.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.619+0000 7f0640f9e740 1 -- 172.21.15.67:0/1356241794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x557a3f7bcf90 con 0x557a3f3d41b0 2024-07-31T15:09:20.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.626+0000 7f06137fe640 1 -- 172.21.15.67:0/1356241794 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f062c030da0 con 0x557a3f3d41b0 2024-07-31T15:09:20.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.626+0000 7f06137fe640 1 -- 172.21.15.67:0/1356241794 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f062800fa10 con 0x557a3f3d41b0 2024-07-31T15:09:20.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.627+0000 7f0640f9e740 10 client.28826 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:20.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.627+0000 7f0640f9e740 10 client.28826 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:09:20.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.627+0000 7f0640f9e740 20 client.28826 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:20.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.627+0000 7f0640f9e740 1 --2- 172.21.15.67:0/1356241794 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x557a3f3bb9f0 0x557a3f7c04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.627+0000 7f0640f9e740 4 client.28826 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:09:20.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.627+0000 7f0640f9e740 1 -- 172.21.15.67:0/1356241794 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a3f71ab00 con 0x557a3f3bb9f0 2024-07-31T15:09:20.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.627+0000 7f0633fff640 1 --2- 172.21.15.67:0/1356241794 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x557a3f3bb9f0 0x557a3f7c04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:20.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.628+0000 7f0633fff640 1 --2- 172.21.15.67:0/1356241794 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x557a3f3bb9f0 0x557a3f7c04a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=600d89b531631817 server_cookie=c324c5efb9ee1e63 in_seq=0 out_seq=0 2024-07-31T15:09:20.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.631+0000 7f06137fe640 10 client.28826 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:09:20.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.635+0000 7f06137fe640 1 -- 172.21.15.67:0/1356241794 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a3f71ab00 con 0x557a3f3bb9f0 2024-07-31T15:09:20.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.635+0000 7f06137fe640 10 client.28826 handle_command_reply: tid=0 2024-07-31T15:09:20.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.637+0000 7f0640f9e740 1 client.28826 shutdown 2024-07-31T15:09:20.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.647+0000 7f0640f9e740 1 -- 172.21.15.67:0/1356241794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 msgr2=0x557a3f7ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:20.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.647+0000 7f0640f9e740 1 --2- 172.21.15.67:0/1356241794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7ae3b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0628014f30 tx=0x7f062800b260 comp rx=0 tx=0).stop 2024-07-31T15:09:20.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.654+0000 7f0640f9e740 1 -- 172.21.15.67:0/1356241794 shutdown_connections 2024-07-31T15:09:20.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.654+0000 7f0640f9e740 1 --2- 172.21.15.67:0/1356241794 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x557a3f3bb9f0 0x557a3f7c04a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.654+0000 7f0640f9e740 1 --2- 172.21.15.67:0/1356241794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 0x557a3f7ba180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.654+0000 7f0640f9e740 1 --2- 172.21.15.67:0/1356241794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3dc390 0x557a3f7ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.654+0000 7f0640f9e740 1 --2- 172.21.15.67:0/1356241794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7ae3b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.654+0000 7f0640f9e740 1 -- 172.21.15.67:0/1356241794 >> 172.21.15.67:0/1356241794 conn(0x557a3f3f1460 msgr2=0x557a3f7a58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:20.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.655+0000 7f06137fe640 0 client.28826 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:09:20.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.659+0000 7f0640f9e740 1 -- 172.21.15.67:0/1356241794 shutdown_connections 2024-07-31T15:09:20.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.663+0000 7f0640f9e740 1 -- 172.21.15.67:0/1356241794 wait complete. 2024-07-31T15:09:20.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.663+0000 7f0640f9e740 20 client.28826 trim_cache size 0 max 16384 2024-07-31T15:09:20.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.696+0000 7f0640f9e740 1 Processor -- start 2024-07-31T15:09:20.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.696+0000 7f0640f9e740 1 -- start start 2024-07-31T15:09:20.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.696+0000 7f0640f9e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.696+0000 7f0640f9e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3dc390 0x557a3f7c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.696+0000 7f0640f9e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 0x557a3f7c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.696+0000 7f0640f9e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f06341c7ec0 con 0x557a3f3fd070 2024-07-31T15:09:20.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.696+0000 7f0640f9e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f06341c9a00 con 0x557a3f3d41b0 2024-07-31T15:09:20.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.696+0000 7f0640f9e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06341c8590 con 0x557a3f3dc390 2024-07-31T15:09:20.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.697+0000 7f06337fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:20.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.697+0000 7f0633fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3dc390 0x557a3f7c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:20.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.697+0000 7f06337fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7a3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:43162/0 (socket says 172.21.15.67:43162) 2024-07-31T15:09:20.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.697+0000 7f06337fe640 1 -- 172.21.15.67:0/897822846 learned_addr learned my addr 172.21.15.67:0/897822846 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:20.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.697+0000 7f06337fe640 1 --2- 172.21.15.67:0/897822846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 0x557a3f7c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:20.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.697+0000 7f06337fe640 1 -- 172.21.15.67:0/897822846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3dc390 msgr2=0x557a3f7c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:20.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.697+0000 7f06337fe640 1 --2- 172.21.15.67:0/897822846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3dc390 0x557a3f7c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.697+0000 7f06337fe640 1 -- 172.21.15.67:0/897822846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 msgr2=0x557a3f7c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:20.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.697+0000 7f06337fe640 1 --2- 172.21.15.67:0/897822846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 0x557a3f7c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.697+0000 7f06337fe640 1 -- 172.21.15.67:0/897822846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a3f7a4a00 con 0x557a3f3d41b0 2024-07-31T15:09:20.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.698+0000 7f06337fe640 1 --2- 172.21.15.67:0/897822846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7a3e50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0628000c00 tx=0x7f0628005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e75741ec860c50a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:20.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.705+0000 7f0619ffb640 1 -- 172.21.15.67:0/897822846 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628014a70 con 0x557a3f3d41b0 2024-07-31T15:09:20.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.705+0000 7f0619ffb640 1 -- 172.21.15.67:0/897822846 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f062800f8f0 con 0x557a3f3d41b0 2024-07-31T15:09:20.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.705+0000 7f0619ffb640 1 -- 172.21.15.67:0/897822846 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06280128c0 con 0x557a3f3d41b0 2024-07-31T15:09:20.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.708+0000 7f0640f9e740 1 -- 172.21.15.67:0/897822846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 msgr2=0x557a3f7a3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:20.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.708+0000 7f0640f9e740 1 --2- 172.21.15.67:0/897822846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7a3e50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0628000c00 tx=0x7f0628005970 comp rx=0 tx=0).stop 2024-07-31T15:09:20.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.712+0000 7f0640f9e740 1 -- 172.21.15.67:0/897822846 shutdown_connections 2024-07-31T15:09:20.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.712+0000 7f0640f9e740 1 --2- 172.21.15.67:0/897822846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3fd070 0x557a3f7c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.712+0000 7f0640f9e740 1 --2- 172.21.15.67:0/897822846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3dc390 0x557a3f7c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.712+0000 7f0640f9e740 1 --2- 172.21.15.67:0/897822846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3d41b0 0x557a3f7a3e50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.712+0000 7f0640f9e740 1 -- 172.21.15.67:0/897822846 >> 172.21.15.67:0/897822846 conn(0x557a3f3f1460 msgr2=0x557a3f7a3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:20.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.724+0000 7f0640f9e740 1 -- 172.21.15.67:0/897822846 shutdown_connections 2024-07-31T15:09:20.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.728+0000 7f0640f9e740 1 -- 172.21.15.67:0/897822846 wait complete. 2024-07-31T15:09:20.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.728+0000 7f0640f9e740 1 -- start start 2024-07-31T15:09:20.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.728+0000 7f0640f9e740 1 Processor -- start 2024-07-31T15:09:20.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.729+0000 7f0640f9e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3d41b0 0x557a3f7b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.729+0000 7f0640f9e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3dc390 0x557a3f7b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.729+0000 7f0640f9e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3fd070 0x557a3f7b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.729+0000 7f0640f9e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f06341c9330 con 0x557a3f3dc390 2024-07-31T15:09:20.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.729+0000 7f0640f9e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f06341c8c60 con 0x557a3f3fd070 2024-07-31T15:09:20.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.729+0000 7f0640f9e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f063410ae40 con 0x557a3f3d41b0 2024-07-31T15:09:20.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.729+0000 7f0640f9e740 10 client.0 _ll_register_callbacks cb 0x7f06280144c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:20.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.733+0000 7f0633fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3dc390 0x557a3f7b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:20.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.733+0000 7f0633fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3dc390 0x557a3f7b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46594/0 (socket says 172.21.15.67:46594) 2024-07-31T15:09:20.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.733+0000 7f0633fff640 1 -- 172.21.15.67:0/2726957379 learned_addr learned my addr 172.21.15.67:0/2726957379 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:20.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.734+0000 7f0633fff640 1 -- 172.21.15.67:0/2726957379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3d41b0 msgr2=0x557a3f7b1a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:09:20.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.734+0000 7f0633fff640 1 --2- 172.21.15.67:0/2726957379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3d41b0 0x557a3f7b1a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.734+0000 7f0633fff640 1 -- 172.21.15.67:0/2726957379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3fd070 msgr2=0x557a3f7b5e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:09:20.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.734+0000 7f0633fff640 1 --2- 172.21.15.67:0/2726957379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3fd070 0x557a3f7b5e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.734+0000 7f0633fff640 1 -- 172.21.15.67:0/2726957379 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a3f7b64d0 con 0x557a3f3dc390 2024-07-31T15:09:20.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.734+0000 7f0633fff640 1 --2- 172.21.15.67:0/2726957379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3dc390 0x557a3f7b1fa0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f062c031960 tx=0x7f062c0318f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332ef54a5318bed6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:20.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.735+0000 7f06137fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:20.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.735+0000 7f06137fe640 1 -- 172.21.15.67:0/2726957379 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c036040 con 0x557a3f3dc390 2024-07-31T15:09:20.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.735+0000 7f0640f9e740 1 -- 172.21.15.67:0/2726957379 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f062800f8f0 con 0x557a3f3dc390 2024-07-31T15:09:20.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.735+0000 7f06137fe640 1 -- 172.21.15.67:0/2726957379 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9395+0+0 (secure 0 0 0) 0x7f062c031440 con 0x557a3f3dc390 2024-07-31T15:09:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.739+0000 7f06137fe640 1 -- 172.21.15.67:0/2726957379 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f062c042040 con 0x557a3f3dc390 2024-07-31T15:09:20.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.744+0000 7f0640f9e740 10 client.19284 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:20.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.744+0000 7f0640f9e740 1 -- 172.21.15.67:0/2726957379 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x557a3f8286d0 con 0x557a3f3dc390 2024-07-31T15:09:20.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.748+0000 7f06137fe640 1 -- 172.21.15.67:0/2726957379 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f062800f8f0 con 0x557a3f3dc390 2024-07-31T15:09:20.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.748+0000 7f06137fe640 1 -- 172.21.15.67:0/2726957379 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f062c03c450 con 0x557a3f3dc390 2024-07-31T15:09:20.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.748+0000 7f0640f9e740 10 client.19284 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:20.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.748+0000 7f0640f9e740 10 client.19284 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:09:20.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.748+0000 7f0640f9e740 20 client.19284 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:20.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.748+0000 7f0640f9e740 1 --2- 172.21.15.67:0/2726957379 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x557a3f390da0 0x557a3f82ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:20.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.748+0000 7f0640f9e740 4 client.19284 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:09:20.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.748+0000 7f0640f9e740 1 -- 172.21.15.67:0/2726957379 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a3f7165a0 con 0x557a3f390da0 2024-07-31T15:09:20.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.753+0000 7f06337fe640 1 --2- 172.21.15.67:0/2726957379 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x557a3f390da0 0x557a3f82ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:20.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.764+0000 7f06337fe640 1 --2- 172.21.15.67:0/2726957379 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x557a3f390da0 0x557a3f82ad70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ca7548ea8a3a7372 server_cookie=e4672cc951c18985 in_seq=0 out_seq=0 2024-07-31T15:09:20.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.765+0000 7f06137fe640 10 client.19284 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:09:20.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.772+0000 7f06137fe640 1 -- 172.21.15.67:0/2726957379 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a3f7165a0 con 0x557a3f390da0 2024-07-31T15:09:20.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.772+0000 7f06137fe640 10 client.19284 handle_command_reply: tid=0 2024-07-31T15:09:20.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.772+0000 7f0640f9e740 1 client.19284 shutdown 2024-07-31T15:09:20.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.778+0000 7f0640f9e740 1 -- 172.21.15.67:0/2726957379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3dc390 msgr2=0x557a3f7b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:20.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.778+0000 7f0640f9e740 1 --2- 172.21.15.67:0/2726957379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3dc390 0x557a3f7b1fa0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f062c031960 tx=0x7f062c0318f0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.778+0000 7f0640f9e740 1 -- 172.21.15.67:0/2726957379 shutdown_connections 2024-07-31T15:09:20.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.778+0000 7f0640f9e740 1 --2- 172.21.15.67:0/2726957379 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x557a3f390da0 0x557a3f82ad70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.778+0000 7f0640f9e740 1 --2- 172.21.15.67:0/2726957379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a3f3fd070 0x557a3f7b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.778+0000 7f0640f9e740 1 --2- 172.21.15.67:0/2726957379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557a3f3dc390 0x557a3f7b1fa0 secure :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f062c031960 tx=0x7f062c0318f0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.778+0000 7f0640f9e740 1 --2- 172.21.15.67:0/2726957379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a3f3d41b0 0x557a3f7b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.778+0000 7f0640f9e740 1 -- 172.21.15.67:0/2726957379 >> 172.21.15.67:0/2726957379 conn(0x557a3f3f1460 msgr2=0x557a3f7a3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:20.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.778+0000 7f06137fe640 0 client.19284 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:09:20.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.781+0000 7f0640f9e740 1 -- 172.21.15.67:0/2726957379 shutdown_connections 2024-07-31T15:09:20.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.781+0000 7f0640f9e740 1 -- 172.21.15.67:0/2726957379 wait complete. 2024-07-31T15:09:20.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.781+0000 7f0640f9e740 20 client.19284 trim_cache size 0 max 16384 2024-07-31T15:09:20.785 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:09:20.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.784+0000 7f06397eb640 1 -- 172.21.15.67:0/3254147581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f061c0b3f50 msgr2=0x7f061c0b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:20.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.784+0000 7f06397eb640 1 --2- 172.21.15.67:0/3254147581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f061c0b3f50 0x7f061c0b6460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f06240aaff0 tx=0x7f06240056d0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.784+0000 7f06397eb640 1 -- 172.21.15.67:0/3254147581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f063411c5d0 msgr2=0x7f06341ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:20.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.784+0000 7f06397eb640 1 --2- 172.21.15.67:0/3254147581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f063411c5d0 0x7f06341ae340 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f062400e970 tx=0x7f062400ed70 comp rx=0 tx=0).stop 2024-07-31T15:09:20.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.785+0000 7f06397eb640 1 -- 172.21.15.67:0/3254147581 shutdown_connections 2024-07-31T15:09:20.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.785+0000 7f06397eb640 1 --2- 172.21.15.67:0/3254147581 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f061c0b3f50 0x7f061c0b6460 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.785+0000 7f06397eb640 1 --2- 172.21.15.67:0/3254147581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f063411c5d0 0x7f06341ae340 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.785+0000 7f06397eb640 1 --2- 172.21.15.67:0/3254147581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0634111790 0x7f06341a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.785+0000 7f06397eb640 1 --2- 172.21.15.67:0/3254147581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063410a330 0x7f06341a8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:20.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.785+0000 7f06397eb640 1 -- 172.21.15.67:0/3254147581 >> 172.21.15.67:0/3254147581 conn(0x7f063406e420 msgr2=0x7f0634071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:20.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.785+0000 7f06397eb640 1 -- 172.21.15.67:0/3254147581 shutdown_connections 2024-07-31T15:09:20.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:20.785+0000 7f06397eb640 1 -- 172.21.15.67:0/3254147581 wait complete. 2024-07-31T15:09:21.085 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_trace.o 2024-07-31T15:09:21.233 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] net/netfilter/xt_nat.o 2024-07-31T15:09:21.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:21 smithi077 ceph-mon[37327]: pgmap v980: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 738 KiB/s wr, 136 op/s 2024-07-31T15:09:21.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:21 smithi077 ceph-mon[37327]: osdmap e123: 12 total, 12 up, 12 in 2024-07-31T15:09:21.509 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/scan.o 2024-07-31T15:09:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:21 smithi067 ceph-mon[32750]: pgmap v980: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 738 KiB/s wr, 136 op/s 2024-07-31T15:09:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:21 smithi067 ceph-mon[32750]: osdmap e123: 12 total, 12 up, 12 in 2024-07-31T15:09:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:21 smithi073 ceph-mon[36714]: pgmap v980: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 738 KiB/s wr, 136 op/s 2024-07-31T15:09:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:21 smithi073 ceph-mon[36714]: osdmap e123: 12 total, 12 up, 12 in 2024-07-31T15:09:21.647 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T15:09:21.655 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/ipv6_sockglue.o 2024-07-31T15:09:21.775 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_logging.o 2024-07-31T15:09:21.889 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_engine_heartbeat.o 2024-07-31T15:09:22.226 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] net/netfilter/xt_LOG.o 2024-07-31T15:09:22.524 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_pm.o 2024-07-31T15:09:22.697 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/udplite.o 2024-07-31T15:09:22.997 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_engine_pm.o 2024-07-31T15:09:23.007 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:09:23.198 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/nl80211.o 2024-07-31T15:09:23.269 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_bsg.o 2024-07-31T15:09:23.289 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/ndisc.o 2024-07-31T15:09:23.303 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] net/netfilter/xt_MASQUERADE.o 2024-07-31T15:09:23.366 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:23 smithi067 ceph-mon[32750]: pgmap v982: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 877 KiB/s wr, 156 op/s 2024-07-31T15:09:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:23 smithi077 ceph-mon[37327]: pgmap v982: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 877 KiB/s wr, 156 op/s 2024-07-31T15:09:23.585 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/udp_offload.o 2024-07-31T15:09:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:23 smithi073 ceph-mon[36714]: pgmap v982: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 877 KiB/s wr, 156 op/s 2024-07-31T15:09:23.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.893+0000 7f4284898640 1 -- 172.21.15.67:0/1924911869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4280111780 msgr2=0x7f4280109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:23.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.893+0000 7f4284898640 1 --2- 172.21.15.67:0/1924911869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4280111780 0x7f4280109df0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4270009860 tx=0x7f427002a3d0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.903+0000 7f4284898640 1 -- 172.21.15.67:0/1924911869 shutdown_connections 2024-07-31T15:09:23.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.903+0000 7f4284898640 1 --2- 172.21.15.67:0/1924911869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f428011c5d0 0x7f428011e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.903+0000 7f4284898640 1 --2- 172.21.15.67:0/1924911869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f428010a330 0x7f428010a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.903+0000 7f4284898640 1 --2- 172.21.15.67:0/1924911869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4280111780 0x7f4280109df0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.903+0000 7f4284898640 1 -- 172.21.15.67:0/1924911869 >> 172.21.15.67:0/1924911869 conn(0x7f428006e440 msgr2=0x7f428006e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:23.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.907+0000 7f4284898640 1 -- 172.21.15.67:0/1924911869 shutdown_connections 2024-07-31T15:09:23.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.911+0000 7f4284898640 1 -- 172.21.15.67:0/1924911869 wait complete. 2024-07-31T15:09:23.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.912+0000 7f4284898640 1 Processor -- start 2024-07-31T15:09:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.914+0000 7f4284898640 1 -- start start 2024-07-31T15:09:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.914+0000 7f4284898640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f428010a330 0x7f428011aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.914+0000 7f4284898640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4280111780 0x7f4280114190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.914+0000 7f4284898640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f428011c5d0 0x7f4280114790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.914+0000 7f4284898640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4280122150 con 0x7f4280111780 2024-07-31T15:09:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.914+0000 7f4284898640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4280121fd0 con 0x7f428010a330 2024-07-31T15:09:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.914+0000 7f4284898640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f42801222d0 con 0x7f428011c5d0 2024-07-31T15:09:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.916+0000 7f427dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4280111780 0x7f4280114190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.916+0000 7f427dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4280111780 0x7f4280114190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58092/0 (socket says 172.21.15.67:58092) 2024-07-31T15:09:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.916+0000 7f427dd9b640 1 -- 172.21.15.67:0/1729460590 learned_addr learned my addr 172.21.15.67:0/1729460590 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.916+0000 7f427ed9d640 1 --2- 172.21.15.67:0/1729460590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f428011c5d0 0x7f4280114790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.916+0000 7f427e59c640 1 --2- 172.21.15.67:0/1729460590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f428010a330 0x7f428011aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:23.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.917+0000 7f427dd9b640 1 -- 172.21.15.67:0/1729460590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f428011c5d0 msgr2=0x7f4280114790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:23.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.917+0000 7f427dd9b640 1 --2- 172.21.15.67:0/1729460590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f428011c5d0 0x7f4280114790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.917+0000 7f427dd9b640 1 -- 172.21.15.67:0/1729460590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f428010a330 msgr2=0x7f428011aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:23.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.917+0000 7f427dd9b640 1 --2- 172.21.15.67:0/1729460590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f428010a330 0x7f428011aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.917+0000 7f427dd9b640 1 -- 172.21.15.67:0/1729460590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4280114fb0 con 0x7f4280111780 2024-07-31T15:09:23.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.917+0000 7f427dd9b640 1 --2- 172.21.15.67:0/1729460590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4280111780 0x7f4280114190 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f427400e9f0 tx=0x7f427400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:23.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.917+0000 7f426f7fe640 1 -- 172.21.15.67:0/1729460590 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427400cd80 con 0x7f4280111780 2024-07-31T15:09:23.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.918+0000 7f4284898640 1 -- 172.21.15.67:0/1729460590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4280119c50 con 0x7f4280111780 2024-07-31T15:09:23.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.918+0000 7f4284898640 1 -- 172.21.15.67:0/1729460590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428011a190 con 0x7f4280111780 2024-07-31T15:09:23.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.921+0000 7f428ade9740 1 Processor -- start 2024-07-31T15:09:23.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.923+0000 7f428ade9740 1 -- start start 2024-07-31T15:09:23.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.923+0000 7f428ade9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb2e840 0x559d3eed4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:23.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.923+0000 7f428ade9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb05980 0x559d3eed4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:23.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.923+0000 7f428ade9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb0db60 0x559d3eed54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:23.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.923+0000 7f428ade9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559d3ee49780 con 0x559d3eb0db60 2024-07-31T15:09:23.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.923+0000 7f428ade9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559d3ee38c80 con 0x559d3eb2e840 2024-07-31T15:09:23.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.923+0000 7f428ade9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559d3ee3ccc0 con 0x559d3eb05980 2024-07-31T15:09:23.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.925+0000 7f427ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb2e840 0x559d3eed4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:23.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.925+0000 7f427ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb2e840 0x559d3eed4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:43488/0 (socket says 172.21.15.67:43488) 2024-07-31T15:09:23.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.925+0000 7f427ed9d640 1 -- 172.21.15.67:0/197435205 learned_addr learned my addr 172.21.15.67:0/197435205 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:23.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.925+0000 7f427e59c640 1 --2- 172.21.15.67:0/197435205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb05980 0x559d3eed4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.931+0000 7f427dd9b640 1 --2- 172.21.15.67:0/197435205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb0db60 0x559d3eed54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.931+0000 7f426f7fe640 1 -- 172.21.15.67:0/1729460590 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42740108e0 con 0x7f4280111780 2024-07-31T15:09:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.931+0000 7f426f7fe640 1 -- 172.21.15.67:0/1729460590 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42740188e0 con 0x7f4280111780 2024-07-31T15:09:23.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.931+0000 7f427ed9d640 1 -- 172.21.15.67:0/197435205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb05980 msgr2=0x559d3eed4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:23.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.931+0000 7f427ed9d640 1 --2- 172.21.15.67:0/197435205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb05980 0x559d3eed4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.931+0000 7f427ed9d640 1 -- 172.21.15.67:0/197435205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb0db60 msgr2=0x559d3eed54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:23.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.931+0000 7f427ed9d640 1 --2- 172.21.15.67:0/197435205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb0db60 0x559d3eed54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.933+0000 7f427ed9d640 1 -- 172.21.15.67:0/197435205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d3edce080 con 0x559d3eb2e840 2024-07-31T15:09:23.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.936+0000 7f427dd9b640 1 --2- 172.21.15.67:0/197435205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb0db60 0x559d3eed54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:09:23.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.937+0000 7f427ed9d640 1 --2- 172.21.15.67:0/197435205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb2e840 0x559d3eed4a30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4278009d50 tx=0x7f427800f7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c9ce8bfd0774f91 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:23.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.938+0000 7f426cff9640 1 -- 172.21.15.67:0/197435205 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427801c070 con 0x559d3eb2e840 2024-07-31T15:09:23.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.938+0000 7f426cff9640 1 -- 172.21.15.67:0/197435205 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4278009e40 con 0x559d3eb2e840 2024-07-31T15:09:23.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.938+0000 7f426cff9640 1 -- 172.21.15.67:0/197435205 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278017a80 con 0x559d3eb2e840 2024-07-31T15:09:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.938+0000 7f426f7fe640 1 -- 172.21.15.67:0/1729460590 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4274010430 con 0x7f4280111780 2024-07-31T15:09:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.940+0000 7f426f7fe640 1 --2- 172.21.15.67:0/1729460590 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42680b3f50 0x7f42680b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.940+0000 7f427e59c640 1 --2- 172.21.15.67:0/1729460590 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42680b3f50 0x7f42680b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:23.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.941+0000 7f428ade9740 1 -- 172.21.15.67:0/197435205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb2e840 msgr2=0x559d3eed4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:23.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.941+0000 7f428ade9740 1 --2- 172.21.15.67:0/197435205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb2e840 0x559d3eed4a30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4278009d50 tx=0x7f427800f7d0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.941+0000 7f427e59c640 1 --2- 172.21.15.67:0/1729460590 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42680b3f50 0x7f42680b6460 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4270005eb0 tx=0x7f4270000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:23.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.941+0000 7f428ade9740 1 -- 172.21.15.67:0/197435205 shutdown_connections 2024-07-31T15:09:23.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.941+0000 7f428ade9740 1 --2- 172.21.15.67:0/197435205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb0db60 0x559d3eed54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.941+0000 7f428ade9740 1 --2- 172.21.15.67:0/197435205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb05980 0x559d3eed4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.941+0000 7f428ade9740 1 --2- 172.21.15.67:0/197435205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb2e840 0x559d3eed4a30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.941+0000 7f428ade9740 1 -- 172.21.15.67:0/197435205 >> 172.21.15.67:0/197435205 conn(0x559d3eb22990 msgr2=0x559d3eed2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:23.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.941+0000 7f426f7fe640 1 -- 172.21.15.67:0/1729460590 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9395+0+0 (secure 0 0 0) 0x7f42740e7a70 con 0x7f4280111780 2024-07-31T15:09:23.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.943+0000 7f428ade9740 1 -- 172.21.15.67:0/197435205 shutdown_connections 2024-07-31T15:09:23.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.943+0000 7f428ade9740 1 -- 172.21.15.67:0/197435205 wait complete. 2024-07-31T15:09:23.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.943+0000 7f428ade9740 1 -- start start 2024-07-31T15:09:23.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.943+0000 7f428ade9740 1 Processor -- start 2024-07-31T15:09:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.944+0000 7f428ade9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb05980 0x559d3eee0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.944+0000 7f428ade9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb0db60 0x559d3eee0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.944+0000 7f428ade9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb2e840 0x559d3eee0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.944+0000 7f428ade9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559d3ec9c7a0 con 0x559d3eb2e840 2024-07-31T15:09:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.944+0000 7f428ade9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559d3eddb2f0 con 0x559d3eb05980 2024-07-31T15:09:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.944+0000 7f428ade9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559d3eee3210 con 0x559d3eb0db60 2024-07-31T15:09:23.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.944+0000 7f428ade9740 10 client.0 _ll_register_callbacks cb 0x559d3ede5110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.945+0000 7f427ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb2e840 0x559d3eee0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.945+0000 7f427ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb2e840 0x559d3eee0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58098/0 (socket says 172.21.15.67:58098) 2024-07-31T15:09:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.945+0000 7f427ed9d640 1 -- 172.21.15.67:0/328641243 learned_addr learned my addr 172.21.15.67:0/328641243 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.945+0000 7f427ed9d640 1 -- 172.21.15.67:0/328641243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb0db60 msgr2=0x559d3eee0550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:09:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.945+0000 7f427ed9d640 1 --2- 172.21.15.67:0/328641243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb0db60 0x559d3eee0550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.945+0000 7f427ed9d640 1 -- 172.21.15.67:0/328641243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb05980 msgr2=0x559d3eee0010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:09:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.945+0000 7f427ed9d640 1 --2- 172.21.15.67:0/328641243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb05980 0x559d3eee0010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.945+0000 7f427ed9d640 1 -- 172.21.15.67:0/328641243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d3eddf480 con 0x559d3eb2e840 2024-07-31T15:09:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.946+0000 7f427ed9d640 1 --2- 172.21.15.67:0/328641243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb2e840 0x559d3eee0a90 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f4278006010 tx=0x7f4278009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea37830e1c5359d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.946+0000 7f4265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.946+0000 7f4265ffb640 1 -- 172.21.15.67:0/328641243 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427801c040 con 0x559d3eb2e840 2024-07-31T15:09:23.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.946+0000 7f428ade9740 1 -- 172.21.15.67:0/328641243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559d3ee4b6d0 con 0x559d3eb2e840 2024-07-31T15:09:23.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.947+0000 7f4265ffb640 1 -- 172.21.15.67:0/328641243 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9395+0+0 (secure 0 0 0) 0x7f4278004ae0 con 0x559d3eb2e840 2024-07-31T15:09:23.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.951+0000 7f4265ffb640 1 -- 172.21.15.67:0/328641243 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f4278017e60 con 0x559d3eb2e840 2024-07-31T15:09:23.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.951+0000 7f428ade9740 10 client.19293 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:23.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.951+0000 7f428ade9740 1 -- 172.21.15.67:0/328641243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x559d3eba8b90 con 0x559d3eb2e840 2024-07-31T15:09:23.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.951+0000 7f4265ffb640 1 -- 172.21.15.67:0/328641243 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559d3ee4b6d0 con 0x559d3eb2e840 2024-07-31T15:09:23.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.952+0000 7f4265ffb640 1 -- 172.21.15.67:0/328641243 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f427801f070 con 0x559d3eb2e840 2024-07-31T15:09:23.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.952+0000 7f428ade9740 10 client.19293 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:23.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.952+0000 7f428ade9740 10 client.19293 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:09:23.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.952+0000 7f428ade9740 20 client.19293 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:23.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.952+0000 7f428ade9740 1 --2- 172.21.15.67:0/328641243 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559d3eaed1c0 0x559d3eef2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:23.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.952+0000 7f428ade9740 4 client.19293 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:09:23.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.952+0000 7f428ade9740 1 -- 172.21.15.67:0/328641243 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d3ee47170 con 0x559d3eaed1c0 2024-07-31T15:09:23.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.956+0000 7f427e59c640 1 --2- 172.21.15.67:0/328641243 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559d3eaed1c0 0x559d3eef2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:23.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.957+0000 7f427e59c640 1 --2- 172.21.15.67:0/328641243 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559d3eaed1c0 0x559d3eef2510 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8360af96f3cf58f4 server_cookie=5d0e57fa3d2c03f6 in_seq=0 out_seq=0 2024-07-31T15:09:23.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.957+0000 7f4265ffb640 10 client.19293 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:23.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.958+0000 7f4265ffb640 1 -- 172.21.15.67:0/328641243 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d3ee47170 con 0x559d3eaed1c0 2024-07-31T15:09:23.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.958+0000 7f4265ffb640 10 client.19293 handle_command_reply: tid=0 2024-07-31T15:09:23.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.958+0000 7f428ade9740 1 client.19293 shutdown 2024-07-31T15:09:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.963+0000 7f428ade9740 1 -- 172.21.15.67:0/328641243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb2e840 msgr2=0x559d3eee0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.963+0000 7f428ade9740 1 --2- 172.21.15.67:0/328641243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb2e840 0x559d3eee0a90 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f4278006010 tx=0x7f4278009ac0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.968+0000 7f428ade9740 1 -- 172.21.15.67:0/328641243 shutdown_connections 2024-07-31T15:09:23.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.968+0000 7f428ade9740 1 --2- 172.21.15.67:0/328641243 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559d3eaed1c0 0x559d3eef2510 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.968+0000 7f428ade9740 1 --2- 172.21.15.67:0/328641243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb2e840 0x559d3eee0a90 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.968+0000 7f428ade9740 1 --2- 172.21.15.67:0/328641243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb0db60 0x559d3eee0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.968+0000 7f428ade9740 1 --2- 172.21.15.67:0/328641243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb05980 0x559d3eee0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:23.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.968+0000 7f428ade9740 1 -- 172.21.15.67:0/328641243 >> 172.21.15.67:0/328641243 conn(0x559d3eb22990 msgr2=0x559d3eed7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:23.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.969+0000 7f4265ffb640 0 client.19293 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:23.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.969+0000 7f428ade9740 1 -- 172.21.15.67:0/328641243 shutdown_connections 2024-07-31T15:09:23.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.973+0000 7f428ade9740 1 -- 172.21.15.67:0/328641243 wait complete. 2024-07-31T15:09:23.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:23.973+0000 7f428ade9740 20 client.19293 trim_cache size 0 max 16384 2024-07-31T15:09:23.975 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_common.o 2024-07-31T15:09:24.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.021+0000 7f428ade9740 1 Processor -- start 2024-07-31T15:09:24.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.021+0000 7f428ade9740 1 -- start start 2024-07-31T15:09:24.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.022+0000 7f428ade9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb05980 0x559d3eed1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:24.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.022+0000 7f428ade9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb0db60 0x559d3eed6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.022+0000 7f428ade9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb2e840 0x559d3eee1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.022+0000 7f427e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb05980 0x559d3eed1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.022+0000 7f427e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb05980 0x559d3eed1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37378/0 (socket says 172.21.15.67:37378) 2024-07-31T15:09:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.022+0000 7f427dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb0db60 0x559d3eed6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.022+0000 7f427e59c640 1 -- 172.21.15.67:0/1128641388 learned_addr learned my addr 172.21.15.67:0/1128641388 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.022+0000 7f428ade9740 1 -- 172.21.15.67:0/1128641388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f426000ff40 con 0x559d3eb0db60 2024-07-31T15:09:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.022+0000 7f428ade9740 1 -- 172.21.15.67:0/1128641388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4260011a80 con 0x559d3eb2e840 2024-07-31T15:09:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.022+0000 7f428ade9740 1 -- 172.21.15.67:0/1128641388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4260010610 con 0x559d3eb05980 2024-07-31T15:09:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.023+0000 7f427ed9d640 1 --2- 172.21.15.67:0/1128641388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb2e840 0x559d3eee1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:24.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.023+0000 7f427ed9d640 1 -- 172.21.15.67:0/1128641388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb05980 msgr2=0x559d3eed1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:24.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.023+0000 7f427ed9d640 1 --2- 172.21.15.67:0/1128641388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb05980 0x559d3eed1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:24.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.023+0000 7f427ed9d640 1 -- 172.21.15.67:0/1128641388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb0db60 msgr2=0x559d3eed6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:24.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.023+0000 7f427ed9d640 1 --2- 172.21.15.67:0/1128641388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb0db60 0x559d3eed6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:24.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.023+0000 7f427ed9d640 1 -- 172.21.15.67:0/1128641388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d3eee2750 con 0x559d3eb2e840 2024-07-31T15:09:24.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.028+0000 7f427ed9d640 1 --2- 172.21.15.67:0/1128641388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb2e840 0x559d3eee1c40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4278002230 tx=0x7f42780053a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a96b954b0692308 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:24.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.032+0000 7f4267fff640 1 -- 172.21.15.67:0/1128641388 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278037070 con 0x559d3eb2e840 2024-07-31T15:09:24.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.032+0000 7f4267fff640 1 -- 172.21.15.67:0/1128641388 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4278004f90 con 0x559d3eb2e840 2024-07-31T15:09:24.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.033+0000 7f428ade9740 1 -- 172.21.15.67:0/1128641388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb2e840 msgr2=0x559d3eee1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:24.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.033+0000 7f428ade9740 1 --2- 172.21.15.67:0/1128641388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb2e840 0x559d3eee1c40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4278002230 tx=0x7f42780053a0 comp rx=0 tx=0).stop 2024-07-31T15:09:24.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.037+0000 7f428ade9740 1 -- 172.21.15.67:0/1128641388 shutdown_connections 2024-07-31T15:09:24.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.037+0000 7f428ade9740 1 --2- 172.21.15.67:0/1128641388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb2e840 0x559d3eee1c40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:24.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.037+0000 7f428ade9740 1 --2- 172.21.15.67:0/1128641388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb0db60 0x559d3eed6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:24.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.037+0000 7f428ade9740 1 --2- 172.21.15.67:0/1128641388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb05980 0x559d3eed1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:24.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.037+0000 7f428ade9740 1 -- 172.21.15.67:0/1128641388 >> 172.21.15.67:0/1128641388 conn(0x559d3eb22990 msgr2=0x559d3eee1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:24.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.044+0000 7f428ade9740 1 -- 172.21.15.67:0/1128641388 shutdown_connections 2024-07-31T15:09:24.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.044+0000 7f428ade9740 1 -- 172.21.15.67:0/1128641388 wait complete. 2024-07-31T15:09:24.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.045+0000 7f428ade9740 1 -- start start 2024-07-31T15:09:24.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.045+0000 7f428ade9740 1 Processor -- start 2024-07-31T15:09:24.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.045+0000 7f428ade9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb05980 0x559d3eee5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:24.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.045+0000 7f428ade9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb0db60 0x559d3eee6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:24.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.045+0000 7f428ade9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb2e840 0x559d3ef58ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:24.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.045+0000 7f428ade9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f42600113b0 con 0x559d3eb05980 2024-07-31T15:09:24.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.045+0000 7f428ade9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4260010ce0 con 0x559d3eb0db60 2024-07-31T15:09:24.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.045+0000 7f428ade9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f426000df40 con 0x559d3eb2e840 2024-07-31T15:09:24.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.045+0000 7f428ade9740 10 client.0 _ll_register_callbacks cb 0x7f4278009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:24.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.046+0000 7f427e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb05980 0x559d3eee5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:24.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.046+0000 7f427e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb05980 0x559d3eee5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58116/0 (socket says 172.21.15.67:58116) 2024-07-31T15:09:24.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.046+0000 7f427e59c640 1 -- 172.21.15.67:0/2289939752 learned_addr learned my addr 172.21.15.67:0/2289939752 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:24.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.046+0000 7f427e59c640 1 -- 172.21.15.67:0/2289939752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb2e840 msgr2=0x559d3ef58ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:09:24.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.046+0000 7f427e59c640 1 --2- 172.21.15.67:0/2289939752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb2e840 0x559d3ef58ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:24.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.046+0000 7f427e59c640 1 -- 172.21.15.67:0/2289939752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb0db60 msgr2=0x559d3eee6020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:09:24.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.046+0000 7f427e59c640 1 --2- 172.21.15.67:0/2289939752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb0db60 0x559d3eee6020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:24.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.046+0000 7f427e59c640 1 -- 172.21.15.67:0/2289939752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d3ef591e0 con 0x559d3eb05980 2024-07-31T15:09:24.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.047+0000 7f427e59c640 1 --2- 172.21.15.67:0/2289939752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb05980 0x559d3eee5ae0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f427002a9c0 tx=0x7f4270005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527162f8da6d6224 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:24.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.048+0000 7f4265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:24.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.048+0000 7f4265ffb640 1 -- 172.21.15.67:0/2289939752 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42700046c0 con 0x559d3eb05980 2024-07-31T15:09:24.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.049+0000 7f4265ffb640 1 -- 172.21.15.67:0/2289939752 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9395+0+0 (secure 0 0 0) 0x7f4270004860 con 0x559d3eb05980 2024-07-31T15:09:24.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.053+0000 7f428ade9740 1 -- 172.21.15.67:0/2289939752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4278004f90 con 0x559d3eb05980 2024-07-31T15:09:24.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.055+0000 7f4265ffb640 1 -- 172.21.15.67:0/2289939752 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f4270004a60 con 0x559d3eb05980 2024-07-31T15:09:24.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.055+0000 7f428ade9740 10 client.19299 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:24.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.055+0000 7f428ade9740 1 -- 172.21.15.67:0/2289939752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x559d3ef7c4e0 con 0x559d3eb05980 2024-07-31T15:09:24.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.056+0000 7f4265ffb640 1 -- 172.21.15.67:0/2289939752 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4278004f90 con 0x559d3eb05980 2024-07-31T15:09:24.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.056+0000 7f4265ffb640 1 -- 172.21.15.67:0/2289939752 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f4270057070 con 0x559d3eb05980 2024-07-31T15:09:24.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.056+0000 7f428ade9740 10 client.19299 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:24.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.056+0000 7f428ade9740 10 client.19299 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:09:24.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.056+0000 7f428ade9740 20 client.19299 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:24.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.056+0000 7f428ade9740 1 --2- 172.21.15.67:0/2289939752 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559d3eac2570 0x559d3ef7ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:24.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.056+0000 7f428ade9740 4 client.19299 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.1, "await_for": 45.0}] 2024-07-31T15:09:24.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.056+0000 7f428ade9740 1 -- 172.21.15.67:0/2289939752 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.1, "await_for": 45.0}) -- 0x559d3ee448b0 con 0x559d3eac2570 2024-07-31T15:09:24.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.060+0000 7f427dd9b640 1 --2- 172.21.15.67:0/2289939752 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559d3eac2570 0x559d3ef7ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:24.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.071+0000 7f427dd9b640 1 --2- 172.21.15.67:0/2289939752 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559d3eac2570 0x559d3ef7ede0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2177ec3e0a8ea98d server_cookie=1d16715d647ffd04 in_seq=0 out_seq=0 2024-07-31T15:09:24.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:24.071+0000 7f4265ffb640 10 client.19299 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:24.099 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_engine_user.o 2024-07-31T15:09:25.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:25 smithi067 ceph-mon[32750]: pgmap v983: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 821 KiB/s wr, 154 op/s 2024-07-31T15:09:25.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:25 smithi067 ceph-mon[32750]: osdmap e124: 12 total, 12 up, 12 in 2024-07-31T15:09:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:25 smithi077 ceph-mon[37327]: pgmap v983: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 821 KiB/s wr, 154 op/s 2024-07-31T15:09:25.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:25 smithi077 ceph-mon[37327]: osdmap e124: 12 total, 12 up, 12 in 2024-07-31T15:09:25.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.533+0000 7f5a68b3f640 1 -- 172.21.15.67:0/2103629285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a64108720 msgr2=0x7f5a6410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:25.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.533+0000 7f5a68b3f640 1 --2- 172.21.15.67:0/2103629285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a64108720 0x7f5a6410ef50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009ca0 tx=0x7f5a5c02a8c0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.534+0000 7f5a68b3f640 1 -- 172.21.15.67:0/2103629285 shutdown_connections 2024-07-31T15:09:25.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.534+0000 7f5a68b3f640 1 --2- 172.21.15.67:0/2103629285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a64108720 0x7f5a6410ef50 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.534+0000 7f5a68b3f640 1 --2- 172.21.15.67:0/2103629285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a64107dd0 0x7f5a641081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.534+0000 7f5a68b3f640 1 --2- 172.21.15.67:0/2103629285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a64102490 0x7f5a641028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.534+0000 7f5a68b3f640 1 -- 172.21.15.67:0/2103629285 >> 172.21.15.67:0/2103629285 conn(0x7f5a64079160 msgr2=0x7f5a64079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:25.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.534+0000 7f5a68b3f640 1 -- 172.21.15.67:0/2103629285 shutdown_connections 2024-07-31T15:09:25.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.534+0000 7f5a68b3f640 1 -- 172.21.15.67:0/2103629285 wait complete. 2024-07-31T15:09:25.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.534+0000 7f5a68b3f640 1 Processor -- start 2024-07-31T15:09:25.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.534+0000 7f5a68b3f640 1 -- start start 2024-07-31T15:09:25.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a68b3f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a64102490 0x7f5a64104b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a68b3f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a64107dd0 0x7f5a64105050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a68b3f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a64108720 0x7f5a64105590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a68b3f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641123a0 con 0x7f5a64107dd0 2024-07-31T15:09:25.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a68b3f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5a64112220 con 0x7f5a64102490 2024-07-31T15:09:25.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a68b3f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a64112520 con 0x7f5a64108720 2024-07-31T15:09:25.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a6259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a64102490 0x7f5a64104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a5bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a64107dd0 0x7f5a64105050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a6259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a64102490 0x7f5a64104b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:43536/0 (socket says 172.21.15.67:43536) 2024-07-31T15:09:25.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a6259c640 1 -- 172.21.15.67:0/2443193104 learned_addr learned my addr 172.21.15.67:0/2443193104 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:25.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a62d9d640 1 --2- 172.21.15.67:0/2443193104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a64108720 0x7f5a64105590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a6259c640 1 -- 172.21.15.67:0/2443193104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a64108720 msgr2=0x7f5a64105590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:25.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a6259c640 1 --2- 172.21.15.67:0/2443193104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a64108720 0x7f5a64105590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a6259c640 1 -- 172.21.15.67:0/2443193104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a64107dd0 msgr2=0x7f5a64105050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:25.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a6259c640 1 --2- 172.21.15.67:0/2443193104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a64107dd0 0x7f5a64105050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.535+0000 7f5a6259c640 1 -- 172.21.15.67:0/2443193104 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a641a81b0 con 0x7f5a64102490 2024-07-31T15:09:25.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.536+0000 7f5a6259c640 1 --2- 172.21.15.67:0/2443193104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a64102490 0x7f5a64104b10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400e9f0 tx=0x7f5a5400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:25.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.536+0000 7f5a5b7fe640 1 -- 172.21.15.67:0/2443193104 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5400cd80 con 0x7f5a64102490 2024-07-31T15:09:25.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.536+0000 7f5a68b3f640 1 -- 172.21.15.67:0/2443193104 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a641a84a0 con 0x7f5a64102490 2024-07-31T15:09:25.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.536+0000 7f5a5b7fe640 1 -- 172.21.15.67:0/2443193104 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a540108e0 con 0x7f5a64102490 2024-07-31T15:09:25.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.536+0000 7f5a5b7fe640 1 -- 172.21.15.67:0/2443193104 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54018980 con 0x7f5a64102490 2024-07-31T15:09:25.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.536+0000 7f5a68b3f640 1 -- 172.21.15.67:0/2443193104 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a641a8a30 con 0x7f5a64102490 2024-07-31T15:09:25.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.538+0000 7f5a5b7fe640 1 -- 172.21.15.67:0/2443193104 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5a54010430 con 0x7f5a64102490 2024-07-31T15:09:25.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.539+0000 7f5a5b7fe640 1 --2- 172.21.15.67:0/2443193104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5a500b3dc0 0x7f5a500b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.539+0000 7f5a5bfff640 1 --2- 172.21.15.67:0/2443193104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5a500b3dc0 0x7f5a500b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.539+0000 7f5a5b7fe640 1 -- 172.21.15.67:0/2443193104 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f5a540e45e0 con 0x7f5a64102490 2024-07-31T15:09:25.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.540+0000 7f5a6f090740 1 Processor -- start 2024-07-31T15:09:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.540+0000 7f5a6f090740 1 -- start start 2024-07-31T15:09:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.540+0000 7f5a5bfff640 1 --2- 172.21.15.67:0/2443193104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5a500b3dc0 0x7f5a500b62d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5a64079a50 tx=0x7f5a4c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.540+0000 7f5a6f090740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7daed070 0x560b7de92570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.540+0000 7f5a6f090740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 0x560b7de92ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.540+0000 7f5a6f090740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7dacc390 0x560b7de92ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.540+0000 7f5a6f090740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560b7de08bb0 con 0x560b7dacc390 2024-07-31T15:09:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.540+0000 7f5a6f090740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560b7ddf80b0 con 0x560b7daed070 2024-07-31T15:09:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.540+0000 7f5a6f090740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560b7ddfc0f0 con 0x560b7dac41b0 2024-07-31T15:09:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.540+0000 7f5a6259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7daed070 0x560b7de92570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.540+0000 7f5a5bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 0x560b7de92ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.540+0000 7f5a6259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7daed070 0x560b7de92570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:43538/0 (socket says 172.21.15.67:43538) 2024-07-31T15:09:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.540+0000 7f5a6259c640 1 -- 172.21.15.67:0/662813166 learned_addr learned my addr 172.21.15.67:0/662813166 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.540+0000 7f5a62d9d640 1 --2- 172.21.15.67:0/662813166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7dacc390 0x560b7de92ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.541+0000 7f5a6259c640 1 -- 172.21.15.67:0/662813166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 msgr2=0x560b7de92ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:25.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.541+0000 7f5a6259c640 1 --2- 172.21.15.67:0/662813166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 0x560b7de92ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.541+0000 7f5a6259c640 1 -- 172.21.15.67:0/662813166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7dacc390 msgr2=0x560b7de92ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:25.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.541+0000 7f5a6259c640 1 --2- 172.21.15.67:0/662813166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7dacc390 0x560b7de92ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.541+0000 7f5a6259c640 1 -- 172.21.15.67:0/662813166 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b7dd9eb90 con 0x560b7daed070 2024-07-31T15:09:25.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.541+0000 7f5a5bfff640 1 --2- 172.21.15.67:0/662813166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 0x560b7de92ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:25.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.541+0000 7f5a6259c640 1 --2- 172.21.15.67:0/662813166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7daed070 0x560b7de92570 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5a540a6fe0 tx=0x7f5a540aaf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84a4ab901b669b3a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:25.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.541+0000 7f5a58ff9640 1 -- 172.21.15.67:0/662813166 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54010a90 con 0x560b7daed070 2024-07-31T15:09:25.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.541+0000 7f5a58ff9640 1 -- 172.21.15.67:0/662813166 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a540e1c70 con 0x560b7daed070 2024-07-31T15:09:25.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.541+0000 7f5a58ff9640 1 -- 172.21.15.67:0/662813166 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54018b20 con 0x560b7daed070 2024-07-31T15:09:25.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.542+0000 7f5a6f090740 1 -- 172.21.15.67:0/662813166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7daed070 msgr2=0x560b7de92570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:25.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.542+0000 7f5a6f090740 1 --2- 172.21.15.67:0/662813166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7daed070 0x560b7de92570 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5a540a6fe0 tx=0x7f5a540aaf60 comp rx=0 tx=0).stop 2024-07-31T15:09:25.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.542+0000 7f5a6f090740 1 -- 172.21.15.67:0/662813166 shutdown_connections 2024-07-31T15:09:25.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.542+0000 7f5a6f090740 1 --2- 172.21.15.67:0/662813166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7dacc390 0x560b7de92ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.542+0000 7f5a6f090740 1 --2- 172.21.15.67:0/662813166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 0x560b7de92ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.542+0000 7f5a6f090740 1 --2- 172.21.15.67:0/662813166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7daed070 0x560b7de92570 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.542+0000 7f5a6f090740 1 -- 172.21.15.67:0/662813166 >> 172.21.15.67:0/662813166 conn(0x560b7dae1460 msgr2=0x560b7de90460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.542+0000 7f5a6f090740 1 -- 172.21.15.67:0/662813166 shutdown_connections 2024-07-31T15:09:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.542+0000 7f5a6f090740 1 -- 172.21.15.67:0/662813166 wait complete. 2024-07-31T15:09:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.543+0000 7f5a6f090740 1 -- start start 2024-07-31T15:09:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.543+0000 7f5a6f090740 1 Processor -- start 2024-07-31T15:09:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.543+0000 7f5a6f090740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7dac41b0 0x560b7de9e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a6f090740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7de9e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a6f090740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7daed070 0x560b7deaa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a6f090740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560b7dc5b700 con 0x560b7dac41b0 2024-07-31T15:09:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a6f090740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560b7dd9a9e0 con 0x560b7dacc390 2024-07-31T15:09:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a6f090740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560b7dea0d50 con 0x560b7daed070 2024-07-31T15:09:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a6f090740 10 client.0 _ll_register_callbacks cb 0x560b7ddb54f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a5bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7de9e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a6259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7dac41b0 0x560b7de9e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a62d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7daed070 0x560b7deaa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a5bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7de9e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:43550/0 (socket says 172.21.15.67:43550) 2024-07-31T15:09:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a5bfff640 1 -- 172.21.15.67:0/2303205058 learned_addr learned my addr 172.21.15.67:0/2303205058 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a6259c640 1 -- 172.21.15.67:0/2303205058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7daed070 msgr2=0x560b7deaa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a6259c640 1 --2- 172.21.15.67:0/2303205058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7daed070 0x560b7deaa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a6259c640 1 -- 172.21.15.67:0/2303205058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 msgr2=0x560b7de9e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a6259c640 1 --2- 172.21.15.67:0/2303205058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7de9e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a6259c640 1 -- 172.21.15.67:0/2303205058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b7db58650 con 0x560b7dac41b0 2024-07-31T15:09:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a5bfff640 1 --2- 172.21.15.67:0/2303205058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7de9e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a6259c640 1 --2- 172.21.15.67:0/2303205058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7dac41b0 0x560b7de9e080 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f5a540e1f70 tx=0x7f5a540e1990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f41c62f31700722 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.544+0000 7f5a49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.545+0000 7f5a49ffb640 1 -- 172.21.15.67:0/2303205058 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a540c4020 con 0x560b7dac41b0 2024-07-31T15:09:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.545+0000 7f5a49ffb640 1 -- 172.21.15.67:0/2303205058 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f5a540a71f0 con 0x560b7dac41b0 2024-07-31T15:09:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.545+0000 7f5a6f090740 1 -- 172.21.15.67:0/2303205058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a540e1c70 con 0x560b7dac41b0 2024-07-31T15:09:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.545+0000 7f5a49ffb640 1 -- 172.21.15.67:0/2303205058 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f5a540cc0d0 con 0x560b7dac41b0 2024-07-31T15:09:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.545+0000 7f5a6f090740 10 client.19311 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.545+0000 7f5a6f090740 1 -- 172.21.15.67:0/2303205058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x560b7deaccb0 con 0x560b7dac41b0 2024-07-31T15:09:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.545+0000 7f5a49ffb640 1 -- 172.21.15.67:0/2303205058 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a540e1c70 con 0x560b7dac41b0 2024-07-31T15:09:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.545+0000 7f5a49ffb640 1 -- 172.21.15.67:0/2303205058 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f5a540a68d0 con 0x560b7dac41b0 2024-07-31T15:09:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.546+0000 7f5a6f090740 10 client.19311 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.546+0000 7f5a6f090740 10 client.19311 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:09:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.546+0000 7f5a6f090740 20 client.19311 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.546+0000 7f5a6f090740 1 --2- 172.21.15.67:0/2303205058 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560b7daab9f0 0x560b7deb0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.546+0000 7f5a6f090740 4 client.19311 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:09:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.546+0000 7f5a6f090740 1 -- 172.21.15.67:0/2303205058 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b7de0ab00 con 0x560b7daab9f0 2024-07-31T15:09:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.546+0000 7f5a5bfff640 1 --2- 172.21.15.67:0/2303205058 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560b7daab9f0 0x560b7deb0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.546+0000 7f5a49ffb640 10 client.19311 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:09:25.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.546+0000 7f5a5bfff640 1 --2- 172.21.15.67:0/2303205058 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560b7daab9f0 0x560b7deb0010 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c2b50b44ed41d69 server_cookie=723d712876ac2ca2 in_seq=0 out_seq=0 2024-07-31T15:09:25.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:25 smithi073 ceph-mon[36714]: pgmap v983: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 821 KiB/s wr, 154 op/s 2024-07-31T15:09:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:25 smithi073 ceph-mon[36714]: osdmap e124: 12 total, 12 up, 12 in 2024-07-31T15:09:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.763+0000 7f5a49ffb640 1 -- 172.21.15.67:0/2303205058 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b7de0ab00 con 0x560b7daab9f0 2024-07-31T15:09:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.763+0000 7f5a49ffb640 10 client.19311 handle_command_reply: tid=0 2024-07-31T15:09:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.763+0000 7f5a6f090740 1 client.19311 shutdown 2024-07-31T15:09:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.763+0000 7f5a6f090740 1 -- 172.21.15.67:0/2303205058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7dac41b0 msgr2=0x560b7de9e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.763+0000 7f5a6f090740 1 --2- 172.21.15.67:0/2303205058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7dac41b0 0x560b7de9e080 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f5a540e1f70 tx=0x7f5a540e1990 comp rx=0 tx=0).stop 2024-07-31T15:09:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.763+0000 7f5a6f090740 1 -- 172.21.15.67:0/2303205058 shutdown_connections 2024-07-31T15:09:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.763+0000 7f5a6f090740 1 --2- 172.21.15.67:0/2303205058 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560b7daab9f0 0x560b7deb0010 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.763+0000 7f5a6f090740 1 --2- 172.21.15.67:0/2303205058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7daed070 0x560b7deaa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.763+0000 7f5a6f090740 1 --2- 172.21.15.67:0/2303205058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7de9e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.763+0000 7f5a6f090740 1 --2- 172.21.15.67:0/2303205058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7dac41b0 0x560b7de9e080 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.764+0000 7f5a6f090740 1 -- 172.21.15.67:0/2303205058 >> 172.21.15.67:0/2303205058 conn(0x560b7dae1460 msgr2=0x560b7de8ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.764+0000 7f5a49ffb640 0 client.19311 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:09:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.764+0000 7f5a6f090740 1 -- 172.21.15.67:0/2303205058 shutdown_connections 2024-07-31T15:09:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.764+0000 7f5a6f090740 1 -- 172.21.15.67:0/2303205058 wait complete. 2024-07-31T15:09:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.764+0000 7f5a6f090740 20 client.19311 trim_cache size 0 max 16384 2024-07-31T15:09:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.781+0000 7f5a6f090740 1 Processor -- start 2024-07-31T15:09:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.781+0000 7f5a6f090740 1 -- start start 2024-07-31T15:09:25.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.781+0000 7f5a6f090740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 0x560b7de90560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.781+0000 7f5a6f090740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7de98690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.781+0000 7f5a6f090740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7daed070 0x560b7dea9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.782+0000 7f5a6f090740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641b2a60 con 0x560b7daed070 2024-07-31T15:09:25.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.782+0000 7f5a62d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7daed070 0x560b7dea9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.782+0000 7f5a6259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 0x560b7de90560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.782+0000 7f5a62d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7daed070 0x560b7dea9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58146/0 (socket says 172.21.15.67:58146) 2024-07-31T15:09:25.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.782+0000 7f5a62d9d640 1 -- 172.21.15.67:0/3049662033 learned_addr learned my addr 172.21.15.67:0/3049662033 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:25.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.782+0000 7f5a6f090740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641b45a0 con 0x560b7dacc390 2024-07-31T15:09:25.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.782+0000 7f5a6f090740 1 -- 172.21.15.67:0/3049662033 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641b3130 con 0x560b7dac41b0 2024-07-31T15:09:25.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.782+0000 7f5a5bfff640 1 --2- 172.21.15.67:0/3049662033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7de98690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.782+0000 7f5a62d9d640 1 -- 172.21.15.67:0/3049662033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 msgr2=0x560b7de90560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:25.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.782+0000 7f5a62d9d640 1 --2- 172.21.15.67:0/3049662033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 0x560b7de90560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.782+0000 7f5a62d9d640 1 -- 172.21.15.67:0/3049662033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 msgr2=0x560b7de98690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:25.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.782+0000 7f5a62d9d640 1 --2- 172.21.15.67:0/3049662033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7de98690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.782+0000 7f5a62d9d640 1 -- 172.21.15.67:0/3049662033 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b7de99710 con 0x560b7daed070 2024-07-31T15:09:25.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.783+0000 7f5a62d9d640 1 --2- 172.21.15.67:0/3049662033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7daed070 0x560b7dea9fb0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c035000 tx=0x7f5a5c005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474726c60369d70 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:25.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.783+0000 7f5a4bfff640 1 -- 172.21.15.67:0/3049662033 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c031df0 con 0x560b7daed070 2024-07-31T15:09:25.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.783+0000 7f5a4bfff640 1 -- 172.21.15.67:0/3049662033 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a5c04b070 con 0x560b7daed070 2024-07-31T15:09:25.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.783+0000 7f5a4bfff640 1 -- 172.21.15.67:0/3049662033 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c004330 con 0x560b7daed070 2024-07-31T15:09:25.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.784+0000 7f5a6f090740 1 -- 172.21.15.67:0/3049662033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7daed070 msgr2=0x560b7dea9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:25.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.784+0000 7f5a6f090740 1 --2- 172.21.15.67:0/3049662033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7daed070 0x560b7dea9fb0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c035000 tx=0x7f5a5c005c30 comp rx=0 tx=0).stop 2024-07-31T15:09:25.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.784+0000 7f5a6f090740 1 -- 172.21.15.67:0/3049662033 shutdown_connections 2024-07-31T15:09:25.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.784+0000 7f5a6f090740 1 --2- 172.21.15.67:0/3049662033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7daed070 0x560b7dea9fb0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.784+0000 7f5a6f090740 1 --2- 172.21.15.67:0/3049662033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7de98690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.784+0000 7f5a6f090740 1 --2- 172.21.15.67:0/3049662033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 0x560b7de90560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.784+0000 7f5a6f090740 1 -- 172.21.15.67:0/3049662033 >> 172.21.15.67:0/3049662033 conn(0x560b7dae1460 msgr2=0x560b7de8fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:25.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.784+0000 7f5a6f090740 1 -- 172.21.15.67:0/3049662033 shutdown_connections 2024-07-31T15:09:25.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.784+0000 7f5a6f090740 1 -- 172.21.15.67:0/3049662033 wait complete. 2024-07-31T15:09:25.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a6f090740 1 -- start start 2024-07-31T15:09:25.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a6f090740 1 Processor -- start 2024-07-31T15:09:25.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a6f090740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 0x560b7deb0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a6f090740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7deb07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a6f090740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7daed070 0x560b7df16550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a6f090740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641b3ed0 con 0x560b7daed070 2024-07-31T15:09:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a6f090740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641b3800 con 0x560b7dacc390 2024-07-31T15:09:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a6f090740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641b0a60 con 0x560b7dac41b0 2024-07-31T15:09:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a6f090740 10 client.0 _ll_register_callbacks cb 0x7f5a540107f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a5bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7deb07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a62d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7daed070 0x560b7df16550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a6259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 0x560b7deb0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a5bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7deb07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:43570/0 (socket says 172.21.15.67:43570) 2024-07-31T15:09:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a5bfff640 1 -- 172.21.15.67:0/2643811561 learned_addr learned my addr 172.21.15.67:0/2643811561 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a6259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 0x560b7deb0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37444/0 (socket says 172.21.15.67:37444) 2024-07-31T15:09:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.785+0000 7f5a62d9d640 1 --2- 172.21.15.67:0/2643811561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7daed070 0x560b7df16550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58156/0 (socket says 172.21.15.67:58156) 2024-07-31T15:09:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.786+0000 7f5a5bfff640 1 -- 172.21.15.67:0/2643811561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 msgr2=0x560b7deb0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.786+0000 7f5a5bfff640 1 --2- 172.21.15.67:0/2643811561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 0x560b7deb0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.786+0000 7f5a5bfff640 1 -- 172.21.15.67:0/2643811561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7daed070 msgr2=0x560b7df16550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:25.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.786+0000 7f5a5bfff640 1 --2- 172.21.15.67:0/2643811561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7daed070 0x560b7df16550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.786+0000 7f5a5bfff640 1 -- 172.21.15.67:0/2643811561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b7df16a90 con 0x560b7dacc390 2024-07-31T15:09:25.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.786+0000 7f5a5bfff640 1 --2- 172.21.15.67:0/2643811561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7deb07d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c003730 tx=0x7f5a4c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb159657c02f9c98 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:25.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.786+0000 7f5a49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:09:25.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.786+0000 7f5a49ffb640 1 -- 172.21.15.67:0/2643811561 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c005220 con 0x560b7dacc390 2024-07-31T15:09:25.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.786+0000 7f5a6f090740 1 -- 172.21.15.67:0/2643811561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a5c04b070 con 0x560b7dacc390 2024-07-31T15:09:25.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.786+0000 7f5a49ffb640 1 -- 172.21.15.67:0/2643811561 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f5a4c0053c0 con 0x560b7dacc390 2024-07-31T15:09:25.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.786+0000 7f5a49ffb640 1 -- 172.21.15.67:0/2643811561 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f5a4c037050 con 0x560b7dacc390 2024-07-31T15:09:25.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.786+0000 7f5a6f090740 10 client.28856 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:25.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.786+0000 7f5a6f090740 1 -- 172.21.15.67:0/2643811561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x560b7df17bb0 con 0x560b7dacc390 2024-07-31T15:09:25.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.787+0000 7f5a49ffb640 1 -- 172.21.15.67:0/2643811561 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a5c04b070 con 0x560b7dacc390 2024-07-31T15:09:25.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.787+0000 7f5a49ffb640 1 -- 172.21.15.67:0/2643811561 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f5a4c004a00 con 0x560b7dacc390 2024-07-31T15:09:25.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.787+0000 7f5a6f090740 10 client.28856 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:25.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.787+0000 7f5a6f090740 10 client.28856 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:09:25.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.787+0000 7f5a6f090740 20 client.28856 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:25.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.787+0000 7f5a6f090740 1 --2- 172.21.15.67:0/2643811561 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560b7da80da0 0x560b7df1a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:25.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.787+0000 7f5a6f090740 4 client.28856 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:09:25.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.787+0000 7f5a6f090740 1 -- 172.21.15.67:0/2643811561 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b7de065a0 con 0x560b7da80da0 2024-07-31T15:09:25.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.787+0000 7f5a6259c640 1 --2- 172.21.15.67:0/2643811561 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560b7da80da0 0x560b7df1a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:25.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.788+0000 7f5a6259c640 1 --2- 172.21.15.67:0/2643811561 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560b7da80da0 0x560b7df1a3d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ac53ee3ac333705 server_cookie=7a2cd46c41aab26e in_seq=0 out_seq=0 2024-07-31T15:09:25.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.788+0000 7f5a49ffb640 10 client.28856 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:09:25.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a49ffb640 1 -- 172.21.15.67:0/2643811561 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b7de065a0 con 0x560b7da80da0 2024-07-31T15:09:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a49ffb640 10 client.28856 handle_command_reply: tid=0 2024-07-31T15:09:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a6f090740 1 client.28856 shutdown 2024-07-31T15:09:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a6f090740 1 -- 172.21.15.67:0/2643811561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 msgr2=0x560b7deb07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a6f090740 1 --2- 172.21.15.67:0/2643811561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7deb07d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c003730 tx=0x7f5a4c004730 comp rx=0 tx=0).stop 2024-07-31T15:09:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a6f090740 1 -- 172.21.15.67:0/2643811561 shutdown_connections 2024-07-31T15:09:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a6f090740 1 --2- 172.21.15.67:0/2643811561 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x560b7da80da0 0x560b7df1a3d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a6f090740 1 --2- 172.21.15.67:0/2643811561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560b7daed070 0x560b7df16550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a49ffb640 0 client.28856 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:09:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a6f090740 1 --2- 172.21.15.67:0/2643811561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b7dacc390 0x560b7deb07d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a6f090740 1 --2- 172.21.15.67:0/2643811561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b7dac41b0 0x560b7deb0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a6f090740 1 -- 172.21.15.67:0/2643811561 >> 172.21.15.67:0/2643811561 conn(0x560b7dae1460 msgr2=0x560b7de8fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a6f090740 1 -- 172.21.15.67:0/2643811561 shutdown_connections 2024-07-31T15:09:25.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a6f090740 1 -- 172.21.15.67:0/2643811561 wait complete. 2024-07-31T15:09:25.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.941+0000 7f5a6f090740 20 client.28856 trim_cache size 0 max 16384 2024-07-31T15:09:25.944 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:09:25.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.942+0000 7f5a68b3f640 1 -- 172.21.15.67:0/2443193104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5a500b3dc0 msgr2=0x7f5a500b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.942+0000 7f5a68b3f640 1 --2- 172.21.15.67:0/2443193104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5a500b3dc0 0x7f5a500b62d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5a64079a50 tx=0x7f5a4c005f90 comp rx=0 tx=0).stop 2024-07-31T15:09:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.942+0000 7f5a68b3f640 1 -- 172.21.15.67:0/2443193104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a64102490 msgr2=0x7f5a64104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.942+0000 7f5a68b3f640 1 --2- 172.21.15.67:0/2443193104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a64102490 0x7f5a64104b10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400e9f0 tx=0x7f5a5400eeb0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.942+0000 7f5a68b3f640 1 -- 172.21.15.67:0/2443193104 shutdown_connections 2024-07-31T15:09:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.942+0000 7f5a68b3f640 1 --2- 172.21.15.67:0/2443193104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5a500b3dc0 0x7f5a500b62d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.942+0000 7f5a68b3f640 1 --2- 172.21.15.67:0/2443193104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a64108720 0x7f5a64105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.942+0000 7f5a68b3f640 1 --2- 172.21.15.67:0/2443193104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a64107dd0 0x7f5a64105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.942+0000 7f5a68b3f640 1 --2- 172.21.15.67:0/2443193104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a64102490 0x7f5a64104b10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.942+0000 7f5a68b3f640 1 -- 172.21.15.67:0/2443193104 >> 172.21.15.67:0/2443193104 conn(0x7f5a64079160 msgr2=0x7f5a6406ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.942+0000 7f5a68b3f640 1 -- 172.21.15.67:0/2443193104 shutdown_connections 2024-07-31T15:09:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:25.942+0000 7f5a68b3f640 1 -- 172.21.15.67:0/2443193104 wait complete. 2024-07-31T15:09:26.793 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T15:09:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:09:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:27 smithi077 ceph-mon[37327]: pgmap v985: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 224 KiB/s wr, 67 op/s 2024-07-31T15:09:27.508 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:27 smithi067 ceph-mon[32750]: pgmap v985: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 224 KiB/s wr, 67 op/s 2024-07-31T15:09:27.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:27 smithi073 ceph-mon[36714]: pgmap v985: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 224 KiB/s wr, 67 op/s 2024-07-31T15:09:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.738+0000 7f0010115640 1 -- 172.21.15.67:0/4136239475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f000810d7a0 msgr2=0x7f000810fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.738+0000 7f0010115640 1 --2- 172.21.15.67:0/4136239475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f000810d7a0 0x7f000810fc90 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0000009ca0 tx=0x7f000002a8e0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.738+0000 7f0010115640 1 -- 172.21.15.67:0/4136239475 shutdown_connections 2024-07-31T15:09:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.738+0000 7f0010115640 1 --2- 172.21.15.67:0/4136239475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f000810d7a0 0x7f000810fc90 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.738+0000 7f0010115640 1 --2- 172.21.15.67:0/4136239475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0008104c90 0x7f000810d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.738+0000 7f0010115640 1 --2- 172.21.15.67:0/4136239475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0008104340 0x7f0008104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.738+0000 7f0010115640 1 -- 172.21.15.67:0/4136239475 >> 172.21.15.67:0/4136239475 conn(0x7f00080fddf0 msgr2=0x7f0008100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.738+0000 7f0010115640 1 -- 172.21.15.67:0/4136239475 shutdown_connections 2024-07-31T15:09:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.738+0000 7f0010115640 1 -- 172.21.15.67:0/4136239475 wait complete. 2024-07-31T15:09:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f0010115640 1 Processor -- start 2024-07-31T15:09:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f0010115640 1 -- start start 2024-07-31T15:09:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f0010115640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0008104340 0x7f000819c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f0010115640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0008104c90 0x7f000819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f0010115640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f000810d7a0 0x7f00081a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f0010115640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00081134a0 con 0x7f000810d7a0 2024-07-31T15:09:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f0010115640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0008113320 con 0x7f0008104c90 2024-07-31T15:09:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f0010115640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0008113620 con 0x7f0008104340 2024-07-31T15:09:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f000e6b2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f000810d7a0 0x7f00081a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f000e6b2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f000810d7a0 0x7f00081a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58180/0 (socket says 172.21.15.67:58180) 2024-07-31T15:09:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f000e6b2640 1 -- 172.21.15.67:0/2348283962 learned_addr learned my addr 172.21.15.67:0/2348283962 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f000d6b0640 1 --2- 172.21.15.67:0/2348283962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0008104c90 0x7f000819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f000deb1640 1 --2- 172.21.15.67:0/2348283962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0008104340 0x7f000819c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f000e6b2640 1 -- 172.21.15.67:0/2348283962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0008104340 msgr2=0x7f000819c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f000e6b2640 1 --2- 172.21.15.67:0/2348283962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0008104340 0x7f000819c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f000e6b2640 1 -- 172.21.15.67:0/2348283962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0008104c90 msgr2=0x7f000819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.739+0000 7f000e6b2640 1 --2- 172.21.15.67:0/2348283962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0008104c90 0x7f000819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.740+0000 7f000e6b2640 1 -- 172.21.15.67:0/2348283962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00081a1ff0 con 0x7f000810d7a0 2024-07-31T15:09:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.740+0000 7f000e6b2640 1 --2- 172.21.15.67:0/2348283962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f000810d7a0 0x7f00081a18f0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0000005cc0 tx=0x7f0000031e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.740+0000 7efffeffd640 1 -- 172.21.15.67:0/2348283962 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000003cf0 con 0x7f000810d7a0 2024-07-31T15:09:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.740+0000 7efffeffd640 1 -- 172.21.15.67:0/2348283962 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00000044f0 con 0x7f000810d7a0 2024-07-31T15:09:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.740+0000 7f0010115640 1 -- 172.21.15.67:0/2348283962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00081a2280 con 0x7f000810d7a0 2024-07-31T15:09:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.740+0000 7efffeffd640 1 -- 172.21.15.67:0/2348283962 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000030c60 con 0x7f000810d7a0 2024-07-31T15:09:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.740+0000 7f0010115640 1 -- 172.21.15.67:0/2348283962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00081a8a70 con 0x7f000810d7a0 2024-07-31T15:09:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.743+0000 7f0017007740 1 Processor -- start 2024-07-31T15:09:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.744+0000 7efffeffd640 1 -- 172.21.15.67:0/2348283962 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0000004040 con 0x7f000810d7a0 2024-07-31T15:09:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.744+0000 7f0017007740 1 -- start start 2024-07-31T15:09:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.744+0000 7f0017007740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c9d1b0 0x56491603fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.744+0000 7f0017007740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c742f0 0x564916040220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.744+0000 7f0017007740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c7c4d0 0x564916040760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.744+0000 7f0017007740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564915fb53b0 con 0x564915c742f0 2024-07-31T15:09:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.744+0000 7f0017007740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564915fa48b0 con 0x564915c7c4d0 2024-07-31T15:09:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.744+0000 7f0017007740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564915fa88f0 con 0x564915c9d1b0 2024-07-31T15:09:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.744+0000 7f000deb1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c742f0 0x564916040220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.744+0000 7f000deb1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c742f0 0x564916040220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58194/0 (socket says 172.21.15.67:58194) 2024-07-31T15:09:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.744+0000 7f000d6b0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c7c4d0 0x564916040760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.744+0000 7f000deb1640 1 -- 172.21.15.67:0/1708186079 learned_addr learned my addr 172.21.15.67:0/1708186079 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.745+0000 7f000d6b0640 1 --2- 172.21.15.67:0/1708186079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c9d1b0 0x56491603fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.745+0000 7f000deb1640 1 -- 172.21.15.67:0/1708186079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c9d1b0 msgr2=0x56491603fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.745+0000 7f000deb1640 1 --2- 172.21.15.67:0/1708186079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c9d1b0 0x56491603fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.745+0000 7f000deb1640 1 -- 172.21.15.67:0/1708186079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c7c4d0 msgr2=0x564916040760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.745+0000 7f000deb1640 1 --2- 172.21.15.67:0/1708186079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c7c4d0 0x564916040760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.745+0000 7f000deb1640 1 -- 172.21.15.67:0/1708186079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564915f4b9a0 con 0x564915c742f0 2024-07-31T15:09:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.745+0000 7f000d6b0640 1 --2- 172.21.15.67:0/1708186079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c7c4d0 0x564916040760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.745+0000 7efffeffd640 1 --2- 172.21.15.67:0/2348283962 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efff40b3dc0 0x7efff40b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.745+0000 7f000d6b0640 1 --2- 172.21.15.67:0/2348283962 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efff40b3dc0 0x7efff40b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.745+0000 7efffeffd640 1 -- 172.21.15.67:0/2348283962 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f00001053b0 con 0x7f000810d7a0 2024-07-31T15:09:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.745+0000 7f000deb1640 1 --2- 172.21.15.67:0/1708186079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c742f0 0x564916040220 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0004009a40 tx=0x7f0004009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995c795aa6d70e29 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.745+0000 7efff3fff640 1 -- 172.21.15.67:0/1708186079 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004017070 con 0x564915c742f0 2024-07-31T15:09:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.746+0000 7efff3fff640 1 -- 172.21.15.67:0/1708186079 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0004013470 con 0x564915c742f0 2024-07-31T15:09:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.746+0000 7efff3fff640 1 -- 172.21.15.67:0/1708186079 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004012630 con 0x564915c742f0 2024-07-31T15:09:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.746+0000 7f0017007740 1 -- 172.21.15.67:0/1708186079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c742f0 msgr2=0x564916040220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.746+0000 7f0017007740 1 --2- 172.21.15.67:0/1708186079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c742f0 0x564916040220 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0004009a40 tx=0x7f0004009f00 comp rx=0 tx=0).stop 2024-07-31T15:09:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.746+0000 7f000d6b0640 1 --2- 172.21.15.67:0/2348283962 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efff40b3dc0 0x7efff40b62d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efff8009fd0 tx=0x7efff80096c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.746+0000 7f0017007740 1 -- 172.21.15.67:0/1708186079 shutdown_connections 2024-07-31T15:09:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.746+0000 7f0017007740 1 --2- 172.21.15.67:0/1708186079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c7c4d0 0x564916040760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.746+0000 7f0017007740 1 --2- 172.21.15.67:0/1708186079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c742f0 0x564916040220 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.746+0000 7f0017007740 1 --2- 172.21.15.67:0/1708186079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c9d1b0 0x56491603fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.747+0000 7f0017007740 1 -- 172.21.15.67:0/1708186079 >> 172.21.15.67:0/1708186079 conn(0x564915c91300 msgr2=0x56491603dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.747+0000 7f0017007740 1 -- 172.21.15.67:0/1708186079 shutdown_connections 2024-07-31T15:09:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.747+0000 7f0017007740 1 -- 172.21.15.67:0/1708186079 wait complete. 2024-07-31T15:09:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.747+0000 7f0017007740 1 -- start start 2024-07-31T15:09:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.747+0000 7f0017007740 1 Processor -- start 2024-07-31T15:09:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.747+0000 7f0017007740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c7c4d0 0x56491604b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.747+0000 7f0017007740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c9d1b0 0x56491604bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.747+0000 7f0017007740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c5bd80 0x5649160577b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.747+0000 7f0017007740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564915edeab0 con 0x564915c9d1b0 2024-07-31T15:09:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.747+0000 7f0017007740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564915f2ccf0 con 0x564915c5bd80 2024-07-31T15:09:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.747+0000 7f0017007740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56491604e460 con 0x564915c7c4d0 2024-07-31T15:09:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7f0017007740 10 client.0 _ll_register_callbacks cb 0x564915f9e9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7f000d6b0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c9d1b0 0x56491604bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7f000d6b0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c9d1b0 0x56491604bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58208/0 (socket says 172.21.15.67:58208) 2024-07-31T15:09:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7f000d6b0640 1 -- 172.21.15.67:0/173204951 learned_addr learned my addr 172.21.15.67:0/173204951 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7f000e6b2640 1 --2- 172.21.15.67:0/173204951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c5bd80 0x5649160577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7f000d6b0640 1 -- 172.21.15.67:0/173204951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c7c4d0 msgr2=0x56491604b830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:09:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7f000d6b0640 1 --2- 172.21.15.67:0/173204951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c7c4d0 0x56491604b830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7f000d6b0640 1 -- 172.21.15.67:0/173204951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c5bd80 msgr2=0x5649160577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7f000d6b0640 1 --2- 172.21.15.67:0/173204951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c5bd80 0x5649160577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7f000d6b0640 1 -- 172.21.15.67:0/173204951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564915d08790 con 0x564915c9d1b0 2024-07-31T15:09:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7f000d6b0640 1 --2- 172.21.15.67:0/173204951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c9d1b0 0x56491604bd70 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7efff8011aa0 tx=0x7efff8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd4147511c875da server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7efff17fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7efff17fa640 1 -- 172.21.15.67:0/173204951 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8005020 con 0x564915c9d1b0 2024-07-31T15:09:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7efff17fa640 1 -- 172.21.15.67:0/173204951 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7efff80051c0 con 0x564915c9d1b0 2024-07-31T15:09:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.748+0000 7f0017007740 1 -- 172.21.15.67:0/173204951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0004013470 con 0x564915c9d1b0 2024-07-31T15:09:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.749+0000 7efff17fa640 1 -- 172.21.15.67:0/173204951 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7efff8033050 con 0x564915c9d1b0 2024-07-31T15:09:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.749+0000 7f0017007740 10 client.19329 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.749+0000 7f0017007740 1 -- 172.21.15.67:0/173204951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x564915d81620 con 0x564915c9d1b0 2024-07-31T15:09:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.749+0000 7efff17fa640 1 -- 172.21.15.67:0/173204951 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0004013470 con 0x564915c9d1b0 2024-07-31T15:09:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.749+0000 7efff17fa640 1 -- 172.21.15.67:0/173204951 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7efff8027030 con 0x564915c9d1b0 2024-07-31T15:09:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.749+0000 7f0017007740 10 client.19329 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.749+0000 7f0017007740 10 client.19329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.749+0000 7f0017007740 20 client.19329 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.749+0000 7f0017007740 1 --2- 172.21.15.67:0/173204951 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564915c311c0 0x56491605d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.749+0000 7f0017007740 4 client.19329 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:09:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.749+0000 7f0017007740 1 -- 172.21.15.67:0/173204951 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564915fb7300 con 0x564915c311c0 2024-07-31T15:09:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.764+0000 7f000deb1640 1 --2- 172.21.15.67:0/173204951 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564915c311c0 0x56491605d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.764+0000 7f000deb1640 1 --2- 172.21.15.67:0/173204951 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564915c311c0 0x56491605d970 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d4b6fde9225ef server_cookie=da0aafe27f1b2f1e in_seq=0 out_seq=0 2024-07-31T15:09:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.764+0000 7efff17fa640 10 client.19329 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.895+0000 7efff17fa640 1 -- 172.21.15.67:0/173204951 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564915fb7300 con 0x564915c311c0 2024-07-31T15:09:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.896+0000 7efff17fa640 10 client.19329 handle_command_reply: tid=0 2024-07-31T15:09:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.896+0000 7f0017007740 1 client.19329 shutdown 2024-07-31T15:09:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.896+0000 7f0017007740 1 -- 172.21.15.67:0/173204951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c9d1b0 msgr2=0x56491604bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.896+0000 7f0017007740 1 --2- 172.21.15.67:0/173204951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c9d1b0 0x56491604bd70 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7efff8011aa0 tx=0x7efff8004730 comp rx=0 tx=0).stop 2024-07-31T15:09:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.896+0000 7f0017007740 1 -- 172.21.15.67:0/173204951 shutdown_connections 2024-07-31T15:09:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.896+0000 7f0017007740 1 --2- 172.21.15.67:0/173204951 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564915c311c0 0x56491605d970 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.896+0000 7f0017007740 1 --2- 172.21.15.67:0/173204951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c5bd80 0x5649160577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.896+0000 7f0017007740 1 --2- 172.21.15.67:0/173204951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c9d1b0 0x56491604bd70 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.896+0000 7efff17fa640 0 client.19329 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.896+0000 7f0017007740 1 --2- 172.21.15.67:0/173204951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c7c4d0 0x56491604b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.896+0000 7f0017007740 1 -- 172.21.15.67:0/173204951 >> 172.21.15.67:0/173204951 conn(0x564915c91300 msgr2=0x564916040d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.896+0000 7f0017007740 1 -- 172.21.15.67:0/173204951 shutdown_connections 2024-07-31T15:09:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.896+0000 7f0017007740 1 -- 172.21.15.67:0/173204951 wait complete. 2024-07-31T15:09:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.896+0000 7f0017007740 20 client.19329 trim_cache size 0 max 16384 2024-07-31T15:09:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.928+0000 7f0017007740 1 Processor -- start 2024-07-31T15:09:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.928+0000 7f0017007740 1 -- start start 2024-07-31T15:09:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.928+0000 7f0017007740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c311c0 0x564916041420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.928+0000 7f0017007740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c5bd80 0x5649160575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.928+0000 7f0017007740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c7c4d0 0x564916057ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.928+0000 7f0017007740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b2920 con 0x564915c5bd80 2024-07-31T15:09:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.928+0000 7f0017007740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b4460 con 0x564915c7c4d0 2024-07-31T15:09:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.928+0000 7f0017007740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b2ff0 con 0x564915c311c0 2024-07-31T15:09:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.928+0000 7f000d6b0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c5bd80 0x5649160575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.928+0000 7f000d6b0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c5bd80 0x5649160575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58222/0 (socket says 172.21.15.67:58222) 2024-07-31T15:09:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.928+0000 7f000d6b0640 1 -- 172.21.15.67:0/1110018607 learned_addr learned my addr 172.21.15.67:0/1110018607 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.928+0000 7f000e6b2640 1 --2- 172.21.15.67:0/1110018607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c7c4d0 0x564916057ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.928+0000 7f000deb1640 1 --2- 172.21.15.67:0/1110018607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c311c0 0x564916041420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.929+0000 7f000d6b0640 1 -- 172.21.15.67:0/1110018607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c311c0 msgr2=0x564916041420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.929+0000 7f000d6b0640 1 --2- 172.21.15.67:0/1110018607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c311c0 0x564916041420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.929+0000 7f000d6b0640 1 -- 172.21.15.67:0/1110018607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c7c4d0 msgr2=0x564916057ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.929+0000 7f000d6b0640 1 --2- 172.21.15.67:0/1110018607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c7c4d0 0x564916057ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.929+0000 7f000d6b0640 1 -- 172.21.15.67:0/1110018607 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564916042af0 con 0x564915c5bd80 2024-07-31T15:09:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.929+0000 7f000d6b0640 1 --2- 172.21.15.67:0/1110018607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c5bd80 0x5649160575a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7efff8009fa0 tx=0x7efff8003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b013c54703ad1d53 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.929+0000 7efff37fe640 1 -- 172.21.15.67:0/1110018607 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff801e9c0 con 0x564915c5bd80 2024-07-31T15:09:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.929+0000 7efff37fe640 1 -- 172.21.15.67:0/1110018607 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff80054d0 con 0x564915c5bd80 2024-07-31T15:09:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.929+0000 7efff37fe640 1 -- 172.21.15.67:0/1110018607 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8004a90 con 0x564915c5bd80 2024-07-31T15:09:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.930+0000 7f0017007740 1 -- 172.21.15.67:0/1110018607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c5bd80 msgr2=0x5649160575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.930+0000 7f0017007740 1 --2- 172.21.15.67:0/1110018607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c5bd80 0x5649160575a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7efff8009fa0 tx=0x7efff8003ea0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.931+0000 7f0017007740 1 -- 172.21.15.67:0/1110018607 shutdown_connections 2024-07-31T15:09:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.931+0000 7f0017007740 1 --2- 172.21.15.67:0/1110018607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c7c4d0 0x564916057ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.931+0000 7f0017007740 1 --2- 172.21.15.67:0/1110018607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c5bd80 0x5649160575a0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.931+0000 7f0017007740 1 --2- 172.21.15.67:0/1110018607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c311c0 0x564916041420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.931+0000 7f0017007740 1 -- 172.21.15.67:0/1110018607 >> 172.21.15.67:0/1110018607 conn(0x564915c91300 msgr2=0x564916040d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.931+0000 7f0017007740 1 -- 172.21.15.67:0/1110018607 shutdown_connections 2024-07-31T15:09:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.931+0000 7f0017007740 1 -- 172.21.15.67:0/1110018607 wait complete. 2024-07-31T15:09:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.931+0000 7f0017007740 1 -- start start 2024-07-31T15:09:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.931+0000 7f0017007740 1 Processor -- start 2024-07-31T15:09:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f0017007740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c311c0 0x564916053f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f0017007740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c5bd80 0x564916050420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f0017007740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c7c4d0 0x564916050b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f0017007740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b3d90 con 0x564915c311c0 2024-07-31T15:09:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f0017007740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b36c0 con 0x564915c5bd80 2024-07-31T15:09:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f0017007740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b0920 con 0x564915c7c4d0 2024-07-31T15:09:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f0017007740 10 client.0 _ll_register_callbacks cb 0x7efff8003f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f000deb1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c311c0 0x564916053f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f000deb1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c311c0 0x564916053f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58234/0 (socket says 172.21.15.67:58234) 2024-07-31T15:09:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f000deb1640 1 -- 172.21.15.67:0/1301895760 learned_addr learned my addr 172.21.15.67:0/1301895760 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f000d6b0640 1 --2- 172.21.15.67:0/1301895760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c5bd80 0x564916050420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f000e6b2640 1 --2- 172.21.15.67:0/1301895760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c7c4d0 0x564916050b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f000deb1640 1 -- 172.21.15.67:0/1301895760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c7c4d0 msgr2=0x564916050b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f000deb1640 1 --2- 172.21.15.67:0/1301895760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c7c4d0 0x564916050b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f000deb1640 1 -- 172.21.15.67:0/1301895760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c5bd80 msgr2=0x564916050420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f000deb1640 1 --2- 172.21.15.67:0/1301895760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c5bd80 0x564916050420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f000deb1640 1 -- 172.21.15.67:0/1301895760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649160c41a0 con 0x564915c311c0 2024-07-31T15:09:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f000d6b0640 1 --2- 172.21.15.67:0/1301895760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c5bd80 0x564916050420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7f000deb1640 1 --2- 172.21.15.67:0/1301895760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c311c0 0x564916053f70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f0004009a40 tx=0x7f0004013d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a24e2e30ede120 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.932+0000 7efff17fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.933+0000 7efff17fa640 1 -- 172.21.15.67:0/1301895760 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004017040 con 0x564915c311c0 2024-07-31T15:09:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.933+0000 7efff17fa640 1 -- 172.21.15.67:0/1301895760 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f0004013470 con 0x564915c311c0 2024-07-31T15:09:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.933+0000 7f0017007740 1 -- 172.21.15.67:0/1301895760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff80054d0 con 0x564915c311c0 2024-07-31T15:09:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.933+0000 7efff17fa640 1 -- 172.21.15.67:0/1301895760 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f0004034050 con 0x564915c311c0 2024-07-31T15:09:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.933+0000 7f0017007740 10 client.19335 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.933+0000 7f0017007740 1 -- 172.21.15.67:0/1301895760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5649160c5280 con 0x564915c311c0 2024-07-31T15:09:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.933+0000 7efff17fa640 1 -- 172.21.15.67:0/1301895760 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff80054d0 con 0x564915c311c0 2024-07-31T15:09:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.933+0000 7efff17fa640 1 -- 172.21.15.67:0/1301895760 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f0004004450 con 0x564915c311c0 2024-07-31T15:09:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.933+0000 7f0017007740 10 client.19335 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.933+0000 7f0017007740 10 client.19335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.933+0000 7f0017007740 20 client.19335 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.933+0000 7f0017007740 1 --2- 172.21.15.67:0/1301895760 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564915c9d1b0 0x5649160c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.933+0000 7f0017007740 4 client.19335 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:09:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.933+0000 7f0017007740 1 -- 172.21.15.67:0/1301895760 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564915fb2da0 con 0x564915c9d1b0 2024-07-31T15:09:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.934+0000 7f000d6b0640 1 --2- 172.21.15.67:0/1301895760 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564915c9d1b0 0x5649160c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.934+0000 7f000d6b0640 1 --2- 172.21.15.67:0/1301895760 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564915c9d1b0 0x5649160c7960 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fc3c19fdaac4114 server_cookie=747c0adea56cbf8b in_seq=0 out_seq=0 2024-07-31T15:09:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:28.934+0000 7efff17fa640 10 client.19335 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.952+0000 7f08fe93e640 1 -- 172.21.15.67:0/892807925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08f8108720 msgr2=0x7f08f810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.952+0000 7f08fe93e640 1 --2- 172.21.15.67:0/892807925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08f8108720 0x7f08f810ef50 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009ca0 tx=0x7f08f002a8c0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.952+0000 7f08fe93e640 1 -- 172.21.15.67:0/892807925 shutdown_connections 2024-07-31T15:09:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.952+0000 7f08fe93e640 1 --2- 172.21.15.67:0/892807925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08f8108720 0x7f08f810ef50 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.952+0000 7f08fe93e640 1 --2- 172.21.15.67:0/892807925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08f8107dd0 0x7f08f81081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.952+0000 7f08fe93e640 1 --2- 172.21.15.67:0/892807925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08f8102490 0x7f08f81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.952+0000 7f08fe93e640 1 -- 172.21.15.67:0/892807925 >> 172.21.15.67:0/892807925 conn(0x7f08f8079160 msgr2=0x7f08f8079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.952+0000 7f08fe93e640 1 -- 172.21.15.67:0/892807925 shutdown_connections 2024-07-31T15:09:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.952+0000 7f08fe93e640 1 -- 172.21.15.67:0/892807925 wait complete. 2024-07-31T15:09:28.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.953+0000 7f08fe93e640 1 Processor -- start 2024-07-31T15:09:28.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.953+0000 7f08fe93e640 1 -- start start 2024-07-31T15:09:28.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.953+0000 7f08fe93e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08f8102490 0x7f08f819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.953+0000 7f08fe93e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08f8107dd0 0x7f08f819b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.953+0000 7f08fe93e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08f8108720 0x7f08f819b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.953+0000 7f08f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08f8107dd0 0x7f08f819b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.953+0000 7f08fe93e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f08f81123a0 con 0x7f08f8107dd0 2024-07-31T15:09:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.953+0000 7f08f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08f8107dd0 0x7f08f819b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58258/0 (socket says 172.21.15.67:58258) 2024-07-31T15:09:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.954+0000 7f08fe93e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f08f8112220 con 0x7f08f8108720 2024-07-31T15:09:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.954+0000 7f08f77fe640 1 -- 172.21.15.67:0/3600055982 learned_addr learned my addr 172.21.15.67:0/3600055982 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.954+0000 7f08fe93e640 1 -- 172.21.15.67:0/3600055982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08f8112520 con 0x7f08f8102490 2024-07-31T15:09:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.954+0000 7f08fcedb640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08f8108720 0x7f08f819b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.954+0000 7f08f7fff640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08f8102490 0x7f08f819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.954+0000 7f08f7fff640 1 -- 172.21.15.67:0/3600055982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08f8108720 msgr2=0x7f08f819b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.954+0000 7f08f7fff640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08f8108720 0x7f08f819b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.954+0000 7f08f7fff640 1 -- 172.21.15.67:0/3600055982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08f8107dd0 msgr2=0x7f08f819b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.954+0000 7f08f7fff640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08f8107dd0 0x7f08f819b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.954+0000 7f08f7fff640 1 -- 172.21.15.67:0/3600055982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f81a5f40 con 0x7f08f8102490 2024-07-31T15:09:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.954+0000 7f08f77fe640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08f8107dd0 0x7f08f819b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.954+0000 7f08f7fff640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08f8102490 0x7f08f819acb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f08e800e9f0 tx=0x7f08e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.954+0000 7f08f57fa640 1 -- 172.21.15.67:0/3600055982 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e800cd80 con 0x7f08f8102490 2024-07-31T15:09:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.955+0000 7f08fe93e640 1 -- 172.21.15.67:0/3600055982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f81a6230 con 0x7f08f8102490 2024-07-31T15:09:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.955+0000 7f08f57fa640 1 -- 172.21.15.67:0/3600055982 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08e80035f0 con 0x7f08f8102490 2024-07-31T15:09:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.955+0000 7f08f57fa640 1 -- 172.21.15.67:0/3600055982 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8010670 con 0x7f08f8102490 2024-07-31T15:09:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.955+0000 7f08fe93e640 1 -- 172.21.15.67:0/3600055982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f81a6770 con 0x7f08f8102490 2024-07-31T15:09:28.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.957+0000 7f0904e8f740 1 Processor -- start 2024-07-31T15:09:28.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.957+0000 7f08f57fa640 1 -- 172.21.15.67:0/3600055982 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f08e80037a0 con 0x7f08f8102490 2024-07-31T15:09:28.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.957+0000 7f0904e8f740 1 -- start start 2024-07-31T15:09:28.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f0904e8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507af070 0x555e50b54570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f0904e8f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e507861b0 0x555e50b54ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f0904e8f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5078e390 0x555e50b54ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f0904e8f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555e50acabb0 con 0x555e5078e390 2024-07-31T15:09:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f0904e8f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555e50aba0b0 con 0x555e507861b0 2024-07-31T15:09:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f0904e8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e50abe0f0 con 0x555e507af070 2024-07-31T15:09:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f08fcedb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507af070 0x555e50b54570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f08f7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e507861b0 0x555e50b54ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f08f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5078e390 0x555e50b54ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f08fcedb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507af070 0x555e50b54570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37498/0 (socket says 172.21.15.67:37498) 2024-07-31T15:09:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f08fcedb640 1 -- 172.21.15.67:0/3517431806 learned_addr learned my addr 172.21.15.67:0/3517431806 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f08fcedb640 1 -- 172.21.15.67:0/3517431806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e507861b0 msgr2=0x555e50b54ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f08fcedb640 1 --2- 172.21.15.67:0/3517431806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e507861b0 0x555e50b54ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f08fcedb640 1 -- 172.21.15.67:0/3517431806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5078e390 msgr2=0x555e50b54ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:28.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f08fcedb640 1 --2- 172.21.15.67:0/3517431806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5078e390 0x555e50b54ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f08fcedb640 1 -- 172.21.15.67:0/3517431806 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e50a60b90 con 0x555e507af070 2024-07-31T15:09:28.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.958+0000 7f08f57fa640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f08e40b3d70 0x7f08e40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.959+0000 7f08fcedb640 1 --2- 172.21.15.67:0/3517431806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507af070 0x555e50b54570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08f002f410 tx=0x7f08f002f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=295eb5376c4123 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:28.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.959+0000 7f08f7fff640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f08e40b3d70 0x7f08e40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.959+0000 7f08ee7fc640 1 -- 172.21.15.67:0/3517431806 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0031cf0 con 0x555e507af070 2024-07-31T15:09:28.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.959+0000 7f08ee7fc640 1 -- 172.21.15.67:0/3517431806 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f0036030 con 0x555e507af070 2024-07-31T15:09:28.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.959+0000 7f08ee7fc640 1 -- 172.21.15.67:0/3517431806 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f00042a0 con 0x555e507af070 2024-07-31T15:09:28.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.959+0000 7f08f57fa640 1 -- 172.21.15.67:0/3600055982 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f08e80e4590 con 0x7f08f8102490 2024-07-31T15:09:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.960+0000 7f0904e8f740 1 -- 172.21.15.67:0/3517431806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507af070 msgr2=0x555e50b54570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.960+0000 7f0904e8f740 1 --2- 172.21.15.67:0/3517431806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507af070 0x555e50b54570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08f002f410 tx=0x7f08f002f440 comp rx=0 tx=0).stop 2024-07-31T15:09:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.960+0000 7f08f7fff640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f08e40b3d70 0x7f08e40b6280 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f08e80e1a50 tx=0x7f08e80e6040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.960+0000 7f0904e8f740 1 -- 172.21.15.67:0/3517431806 shutdown_connections 2024-07-31T15:09:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.960+0000 7f0904e8f740 1 --2- 172.21.15.67:0/3517431806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5078e390 0x555e50b54ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.960+0000 7f0904e8f740 1 --2- 172.21.15.67:0/3517431806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e507861b0 0x555e50b54ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.960+0000 7f0904e8f740 1 --2- 172.21.15.67:0/3517431806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507af070 0x555e50b54570 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.960+0000 7f0904e8f740 1 -- 172.21.15.67:0/3517431806 >> 172.21.15.67:0/3517431806 conn(0x555e507a3460 msgr2=0x555e50b52460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:28.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.960+0000 7f0904e8f740 1 -- 172.21.15.67:0/3517431806 shutdown_connections 2024-07-31T15:09:28.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.960+0000 7f0904e8f740 1 -- 172.21.15.67:0/3517431806 wait complete. 2024-07-31T15:09:28.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.960+0000 7f0904e8f740 1 -- start start 2024-07-31T15:09:28.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f0904e8f740 1 Processor -- start 2024-07-31T15:09:28.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f0904e8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507861b0 0x555e50b600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f0904e8f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5078e390 0x555e50b60630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f0904e8f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e507af070 0x555e50b6beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f0904e8f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555e5091d700 con 0x555e5078e390 2024-07-31T15:09:28.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f0904e8f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555e50a5c9e0 con 0x555e507af070 2024-07-31T15:09:28.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f0904e8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e50b62bc0 con 0x555e507861b0 2024-07-31T15:09:28.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f08f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5078e390 0x555e50b60630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f08f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5078e390 0x555e50b60630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58278/0 (socket says 172.21.15.67:58278) 2024-07-31T15:09:28.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f08f77fe640 1 -- 172.21.15.67:0/2205132926 learned_addr learned my addr 172.21.15.67:0/2205132926 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:28.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f08fcedb640 1 --2- 172.21.15.67:0/2205132926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507861b0 0x555e50b600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f0904e8f740 10 client.0 _ll_register_callbacks cb 0x555e50a774f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:28.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f08fcedb640 1 --2- 172.21.15.67:0/2205132926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e507af070 0x555e50b6beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f08f77fe640 1 -- 172.21.15.67:0/2205132926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507861b0 msgr2=0x555e50b600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:28.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.961+0000 7f08f77fe640 1 --2- 172.21.15.67:0/2205132926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507861b0 0x555e50b600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.962+0000 7f08f77fe640 1 -- 172.21.15.67:0/2205132926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e507af070 msgr2=0x555e50b6beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:28.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.962+0000 7f08f77fe640 1 --2- 172.21.15.67:0/2205132926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e507af070 0x555e50b6beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.962+0000 7f08f77fe640 1 -- 172.21.15.67:0/2205132926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e5081a650 con 0x555e5078e390 2024-07-31T15:09:28.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.962+0000 7f08fcedb640 1 --2- 172.21.15.67:0/2205132926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507861b0 0x555e50b600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.962+0000 7f08f77fe640 1 --2- 172.21.15.67:0/2205132926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5078e390 0x555e50b60630 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f08e0012d20 tx=0x7f08e000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fb3841de69c951 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.962+0000 7f08d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.962+0000 7f08d77fe640 1 -- 172.21.15.67:0/2205132926 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e000fe20 con 0x555e5078e390 2024-07-31T15:09:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.962+0000 7f08d77fe640 1 -- 172.21.15.67:0/2205132926 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f08e00047a0 con 0x555e5078e390 2024-07-31T15:09:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.962+0000 7f0904e8f740 1 -- 172.21.15.67:0/2205132926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f0036030 con 0x555e5078e390 2024-07-31T15:09:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.962+0000 7f08d77fe640 1 -- 172.21.15.67:0/2205132926 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f08e00049a0 con 0x555e5078e390 2024-07-31T15:09:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.962+0000 7f0904e8f740 10 client.19347 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.962+0000 7f0904e8f740 1 -- 172.21.15.67:0/2205132926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x555e50b6ec00 con 0x555e5078e390 2024-07-31T15:09:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.962+0000 7f08d77fe640 1 -- 172.21.15.67:0/2205132926 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f0036030 con 0x555e5078e390 2024-07-31T15:09:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.963+0000 7f08d77fe640 1 -- 172.21.15.67:0/2205132926 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f08e0010570 con 0x555e5078e390 2024-07-31T15:09:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.963+0000 7f0904e8f740 10 client.19347 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.963+0000 7f0904e8f740 10 client.19347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:09:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.963+0000 7f0904e8f740 20 client.19347 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.963+0000 7f0904e8f740 1 --2- 172.21.15.67:0/2205132926 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555e5076d9f0 0x555e50b72100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.963+0000 7f0904e8f740 4 client.19347 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:09:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.963+0000 7f0904e8f740 1 -- 172.21.15.67:0/2205132926 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e50accb00 con 0x555e5076d9f0 2024-07-31T15:09:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.963+0000 7f08fcedb640 1 --2- 172.21.15.67:0/2205132926 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555e5076d9f0 0x555e50b72100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.963+0000 7f08d77fe640 10 client.19347 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:09:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.963+0000 7f08fcedb640 1 --2- 172.21.15.67:0/2205132926 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555e5076d9f0 0x555e50b72100 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce5a6ebe6cd2e2b9 server_cookie=1a2a98b65c75e4bf in_seq=0 out_seq=0 2024-07-31T15:09:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.964+0000 7f08d77fe640 1 -- 172.21.15.67:0/2205132926 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e50accb00 con 0x555e5076d9f0 2024-07-31T15:09:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.964+0000 7f08d77fe640 10 client.19347 handle_command_reply: tid=0 2024-07-31T15:09:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.964+0000 7f0904e8f740 1 client.19347 shutdown 2024-07-31T15:09:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.964+0000 7f0904e8f740 1 -- 172.21.15.67:0/2205132926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5078e390 msgr2=0x555e50b60630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.964+0000 7f0904e8f740 1 --2- 172.21.15.67:0/2205132926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5078e390 0x555e50b60630 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f08e0012d20 tx=0x7f08e000f790 comp rx=0 tx=0).stop 2024-07-31T15:09:28.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.964+0000 7f0904e8f740 1 -- 172.21.15.67:0/2205132926 shutdown_connections 2024-07-31T15:09:28.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.964+0000 7f0904e8f740 1 --2- 172.21.15.67:0/2205132926 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555e5076d9f0 0x555e50b72100 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.964+0000 7f08d77fe640 0 client.19347 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:09:28.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.964+0000 7f0904e8f740 1 --2- 172.21.15.67:0/2205132926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e507af070 0x555e50b6beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.964+0000 7f0904e8f740 1 --2- 172.21.15.67:0/2205132926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5078e390 0x555e50b60630 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.964+0000 7f0904e8f740 1 --2- 172.21.15.67:0/2205132926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507861b0 0x555e50b600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.964+0000 7f0904e8f740 1 -- 172.21.15.67:0/2205132926 >> 172.21.15.67:0/2205132926 conn(0x555e507a3460 msgr2=0x555e50b57750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:28.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.965+0000 7f0904e8f740 1 -- 172.21.15.67:0/2205132926 shutdown_connections 2024-07-31T15:09:28.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.965+0000 7f0904e8f740 1 -- 172.21.15.67:0/2205132926 wait complete. 2024-07-31T15:09:28.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.965+0000 7f0904e8f740 20 client.19347 trim_cache size 0 max 16384 2024-07-31T15:09:28.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.991+0000 7f0904e8f740 1 Processor -- start 2024-07-31T15:09:28.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.991+0000 7f0904e8f740 1 -- start start 2024-07-31T15:09:28.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.991+0000 7f0904e8f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5076d9f0 0x555e50b5ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.991+0000 7f0904e8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507861b0 0x555e50b57d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.991+0000 7f0904e8f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e5078e390 0x555e50b6bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:28.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.991+0000 7f0904e8f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f08f81b06f0 con 0x555e5076d9f0 2024-07-31T15:09:28.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.991+0000 7f0904e8f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f08f81b2230 con 0x555e5078e390 2024-07-31T15:09:28.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.991+0000 7f0904e8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08f81b0dc0 con 0x555e507861b0 2024-07-31T15:09:28.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.992+0000 7f08fcedb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5076d9f0 0x555e50b5ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.992+0000 7f08fcedb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5076d9f0 0x555e50b5ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58290/0 (socket says 172.21.15.67:58290) 2024-07-31T15:09:28.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.992+0000 7f08fcedb640 1 -- 172.21.15.67:0/3123798117 learned_addr learned my addr 172.21.15.67:0/3123798117 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:28.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.992+0000 7f08f77fe640 1 --2- 172.21.15.67:0/3123798117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507861b0 0x555e50b57d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.992+0000 7f08fcedb640 1 --2- 172.21.15.67:0/3123798117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e5078e390 0x555e50b6bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:28.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.992+0000 7f08fcedb640 1 -- 172.21.15.67:0/3123798117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507861b0 msgr2=0x555e50b57d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:28.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.992+0000 7f08fcedb640 1 --2- 172.21.15.67:0/3123798117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507861b0 0x555e50b57d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.992+0000 7f08fcedb640 1 -- 172.21.15.67:0/3123798117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e5078e390 msgr2=0x555e50b6bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:28.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.992+0000 7f08fcedb640 1 --2- 172.21.15.67:0/3123798117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e5078e390 0x555e50b6bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.992+0000 7f08fcedb640 1 -- 172.21.15.67:0/3123798117 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e50b611b0 con 0x555e5076d9f0 2024-07-31T15:09:28.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.992+0000 7f08fcedb640 1 --2- 172.21.15.67:0/3123798117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5076d9f0 0x555e50b5ede0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f08f0006010 tx=0x7f08f002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e5e403822375b7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:28.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.992+0000 7f08edffb640 1 -- 172.21.15.67:0/3123798117 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0003a70 con 0x555e5076d9f0 2024-07-31T15:09:28.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.993+0000 7f08edffb640 1 -- 172.21.15.67:0/3123798117 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f0036040 con 0x555e5076d9f0 2024-07-31T15:09:28.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.993+0000 7f08edffb640 1 -- 172.21.15.67:0/3123798117 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0004640 con 0x555e5076d9f0 2024-07-31T15:09:28.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.993+0000 7f0904e8f740 1 -- 172.21.15.67:0/3123798117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5076d9f0 msgr2=0x555e50b5ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:28.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.993+0000 7f0904e8f740 1 --2- 172.21.15.67:0/3123798117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5076d9f0 0x555e50b5ede0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f08f0006010 tx=0x7f08f002f3a0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.993+0000 7f0904e8f740 1 -- 172.21.15.67:0/3123798117 shutdown_connections 2024-07-31T15:09:28.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.993+0000 7f0904e8f740 1 --2- 172.21.15.67:0/3123798117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e5078e390 0x555e50b6bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.993+0000 7f0904e8f740 1 --2- 172.21.15.67:0/3123798117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e507861b0 0x555e50b57d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.994+0000 7f0904e8f740 1 --2- 172.21.15.67:0/3123798117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5076d9f0 0x555e50b5ede0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:28.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.994+0000 7f0904e8f740 1 -- 172.21.15.67:0/3123798117 >> 172.21.15.67:0/3123798117 conn(0x555e507a3460 msgr2=0x555e50b57670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.994+0000 7f0904e8f740 1 -- 172.21.15.67:0/3123798117 shutdown_connections 2024-07-31T15:09:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.994+0000 7f0904e8f740 1 -- 172.21.15.67:0/3123798117 wait complete. 2024-07-31T15:09:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.994+0000 7f0904e8f740 1 -- start start 2024-07-31T15:09:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.994+0000 7f0904e8f740 1 Processor -- start 2024-07-31T15:09:29.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.994+0000 7f0904e8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e5076d9f0 0x555e50b68550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:29.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.994+0000 7f0904e8f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e507861b0 0x555e50b63aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:29.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.994+0000 7f0904e8f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e5078e390 0x555e50b64040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:29.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.994+0000 7f0904e8f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f08f81b1b60 con 0x555e507861b0 2024-07-31T15:09:29.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.994+0000 7f0904e8f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f08f81b1490 con 0x555e5078e390 2024-07-31T15:09:29.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.994+0000 7f0904e8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08f81ae6f0 con 0x555e5076d9f0 2024-07-31T15:09:29.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f0904e8f740 10 client.0 _ll_register_callbacks cb 0x7f08e0009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:29.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08fcedb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e5078e390 0x555e50b64040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:29.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e507861b0 0x555e50b63aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:29.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08fcedb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e5078e390 0x555e50b64040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:43676/0 (socket says 172.21.15.67:43676) 2024-07-31T15:09:29.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08fcedb640 1 -- 172.21.15.67:0/336752191 learned_addr learned my addr 172.21.15.67:0/336752191 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:29.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08fcedb640 1 --2- 172.21.15.67:0/336752191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e5076d9f0 0x555e50b68550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:29.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08f77fe640 1 -- 172.21.15.67:0/336752191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e5076d9f0 msgr2=0x555e50b68550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:29.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08f77fe640 1 --2- 172.21.15.67:0/336752191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e5076d9f0 0x555e50b68550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:29.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08f77fe640 1 -- 172.21.15.67:0/336752191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e5078e390 msgr2=0x555e50b64040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:29.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08f77fe640 1 --2- 172.21.15.67:0/336752191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e5078e390 0x555e50b64040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:29.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08f77fe640 1 -- 172.21.15.67:0/336752191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e50b647d0 con 0x555e507861b0 2024-07-31T15:09:29.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08fcedb640 1 --2- 172.21.15.67:0/336752191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e5078e390 0x555e50b64040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:29.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08f77fe640 1 --2- 172.21.15.67:0/336752191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e507861b0 0x555e50b63aa0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f08e00109e0 tx=0x7f08e000fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7890d756f8118810 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08d77fe640 1 -- 172.21.15.67:0/336752191 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e002d020 con 0x555e507861b0 2024-07-31T15:09:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f0904e8f740 1 -- 172.21.15.67:0/336752191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f0036040 con 0x555e507861b0 2024-07-31T15:09:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.995+0000 7f08d77fe640 1 -- 172.21.15.67:0/336752191 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f08e0010de0 con 0x555e507861b0 2024-07-31T15:09:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.996+0000 7f08d77fe640 1 -- 172.21.15.67:0/336752191 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f08e0038050 con 0x555e507861b0 2024-07-31T15:09:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.996+0000 7f0904e8f740 10 client.19353 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.996+0000 7f0904e8f740 1 -- 172.21.15.67:0/336752191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x555e50bda800 con 0x555e507861b0 2024-07-31T15:09:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.996+0000 7f08d77fe640 1 -- 172.21.15.67:0/336752191 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f0036040 con 0x555e507861b0 2024-07-31T15:09:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.996+0000 7f08d77fe640 1 -- 172.21.15.67:0/336752191 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f08e001cae0 con 0x555e507861b0 2024-07-31T15:09:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.996+0000 7f0904e8f740 10 client.19353 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.996+0000 7f0904e8f740 10 client.19353 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:09:29.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.996+0000 7f0904e8f740 20 client.19353 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:29.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.996+0000 7f0904e8f740 1 --2- 172.21.15.67:0/336752191 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555e507af070 0x555e50bdcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:29.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.996+0000 7f0904e8f740 4 client.19353 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:09:29.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.996+0000 7f0904e8f740 1 -- 172.21.15.67:0/336752191 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e50ac85a0 con 0x555e507af070 2024-07-31T15:09:29.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.996+0000 7f08fcedb640 1 --2- 172.21.15.67:0/336752191 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555e507af070 0x555e50bdcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:29.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f08d77fe640 10 client.19353 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:09:29.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f08fcedb640 1 --2- 172.21.15.67:0/336752191 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555e507af070 0x555e50bdcfb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=64bbf45ccc61a7cf server_cookie=ab4ce5deddde4c5d in_seq=0 out_seq=0 2024-07-31T15:09:29.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f08d77fe640 1 -- 172.21.15.67:0/336752191 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e50ac85a0 con 0x555e507af070 2024-07-31T15:09:29.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f08d77fe640 10 client.19353 handle_command_reply: tid=0 2024-07-31T15:09:29.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f0904e8f740 1 client.19353 shutdown 2024-07-31T15:09:29.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f0904e8f740 1 -- 172.21.15.67:0/336752191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e507861b0 msgr2=0x555e50b63aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:29.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f0904e8f740 1 --2- 172.21.15.67:0/336752191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e507861b0 0x555e50b63aa0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f08e00109e0 tx=0x7f08e000fd40 comp rx=0 tx=0).stop 2024-07-31T15:09:29.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f0904e8f740 1 -- 172.21.15.67:0/336752191 shutdown_connections 2024-07-31T15:09:29.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f0904e8f740 1 --2- 172.21.15.67:0/336752191 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555e507af070 0x555e50bdcfb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:29.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f0904e8f740 1 --2- 172.21.15.67:0/336752191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e5078e390 0x555e50b64040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:29.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f0904e8f740 1 --2- 172.21.15.67:0/336752191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e507861b0 0x555e50b63aa0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:29.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f0904e8f740 1 --2- 172.21.15.67:0/336752191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e5076d9f0 0x555e50b68550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:29.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f08d77fe640 0 client.19353 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:09:29.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f0904e8f740 1 -- 172.21.15.67:0/336752191 >> 172.21.15.67:0/336752191 conn(0x555e507a3460 msgr2=0x555e50b555a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:29.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.997+0000 7f0904e8f740 1 -- 172.21.15.67:0/336752191 shutdown_connections 2024-07-31T15:09:29.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.998+0000 7f0904e8f740 1 -- 172.21.15.67:0/336752191 wait complete. 2024-07-31T15:09:29.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.998+0000 7f0904e8f740 20 client.19353 trim_cache size 0 max 16384 2024-07-31T15:09:29.004 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:09:29.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.998+0000 7f08fe93e640 1 -- 172.21.15.67:0/3600055982 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f08e40b3d70 msgr2=0x7f08e40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:29.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.998+0000 7f08fe93e640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f08e40b3d70 0x7f08e40b6280 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f08e80e1a50 tx=0x7f08e80e6040 comp rx=0 tx=0).stop 2024-07-31T15:09:29.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.998+0000 7f08fe93e640 1 -- 172.21.15.67:0/3600055982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08f8102490 msgr2=0x7f08f819acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:29.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.998+0000 7f08fe93e640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08f8102490 0x7f08f819acb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f08e800e9f0 tx=0x7f08e800eeb0 comp rx=0 tx=0).stop 2024-07-31T15:09:29.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.998+0000 7f08fe93e640 1 -- 172.21.15.67:0/3600055982 shutdown_connections 2024-07-31T15:09:29.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.998+0000 7f08fe93e640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f08e40b3d70 0x7f08e40b6280 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:29.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.998+0000 7f08fe93e640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08f8108720 0x7f08f819b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:29.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.998+0000 7f08fe93e640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08f8107dd0 0x7f08f819b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:29.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.998+0000 7f08fe93e640 1 --2- 172.21.15.67:0/3600055982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08f8102490 0x7f08f819acb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:29.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.998+0000 7f08fe93e640 1 -- 172.21.15.67:0/3600055982 >> 172.21.15.67:0/3600055982 conn(0x7f08f8079160 msgr2=0x7f08f8104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:29.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.998+0000 7f08fe93e640 1 -- 172.21.15.67:0/3600055982 shutdown_connections 2024-07-31T15:09:29.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:28.998+0000 7f08fe93e640 1 -- 172.21.15.67:0/3600055982 wait complete. 2024-07-31T15:09:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:29 smithi077 ceph-mon[37327]: pgmap v986: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 187 KiB/s wr, 36 op/s 2024-07-31T15:09:29.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:29 smithi067 ceph-mon[32750]: pgmap v986: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 187 KiB/s wr, 36 op/s 2024-07-31T15:09:29.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:29 smithi073 ceph-mon[36714]: pgmap v986: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 187 KiB/s wr, 36 op/s 2024-07-31T15:09:29.888 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T15:09:30.155 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:09:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:31 smithi077 ceph-mon[37327]: pgmap v987: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 166 KiB/s wr, 38 op/s 2024-07-31T15:09:31.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:31 smithi067 ceph-mon[32750]: pgmap v987: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 166 KiB/s wr, 38 op/s 2024-07-31T15:09:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:31 smithi073 ceph-mon[36714]: pgmap v987: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 166 KiB/s wr, 38 op/s 2024-07-31T15:09:31.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.980+0000 7fc55d497640 1 -- 172.21.15.67:0/3774666863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc558109820 msgr2=0x7fc558110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:31.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.980+0000 7fc55d497640 1 --2- 172.21.15.67:0/3774666863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc558109820 0x7fc558110050 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fc550009ca0 tx=0x7fc55002a8c0 comp rx=0 tx=0).stop 2024-07-31T15:09:31.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.980+0000 7fc55d497640 1 -- 172.21.15.67:0/3774666863 shutdown_connections 2024-07-31T15:09:31.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.980+0000 7fc55d497640 1 --2- 172.21.15.67:0/3774666863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc558109820 0x7fc558110050 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:31.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.980+0000 7fc55d497640 1 --2- 172.21.15.67:0/3774666863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc558108ed0 0x7fc5581092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:31.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.980+0000 7fc55d497640 1 --2- 172.21.15.67:0/3774666863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc558103590 0x7fc5581039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:31.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.980+0000 7fc55d497640 1 -- 172.21.15.67:0/3774666863 >> 172.21.15.67:0/3774666863 conn(0x7fc55807b280 msgr2=0x7fc55807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:31.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.980+0000 7fc55d497640 1 -- 172.21.15.67:0/3774666863 shutdown_connections 2024-07-31T15:09:31.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.980+0000 7fc55d497640 1 -- 172.21.15.67:0/3774666863 wait complete. 2024-07-31T15:09:31.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.980+0000 7fc55d497640 1 Processor -- start 2024-07-31T15:09:31.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc55d497640 1 -- start start 2024-07-31T15:09:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc55d497640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc558103590 0x7fc558199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc55d497640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc558108ed0 0x7fc55819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc55d497640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc558109820 0x7fc55819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc55d497640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc5581134f0 con 0x7fc558108ed0 2024-07-31T15:09:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc55d497640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc558113370 con 0x7fc558109820 2024-07-31T15:09:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc55d497640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc558113670 con 0x7fc558103590 2024-07-31T15:09:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc5567fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc558108ed0 0x7fc55819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc5567fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc558108ed0 0x7fc55819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49660/0 (socket says 172.21.15.67:49660) 2024-07-31T15:09:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc5567fc640 1 -- 172.21.15.67:0/3998508286 learned_addr learned my addr 172.21.15.67:0/3998508286 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc5577fe640 1 --2- 172.21.15.67:0/3998508286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc558109820 0x7fc55819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc556ffd640 1 --2- 172.21.15.67:0/3998508286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc558103590 0x7fc558199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc5567fc640 1 -- 172.21.15.67:0/3998508286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc558103590 msgr2=0x7fc558199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc5567fc640 1 --2- 172.21.15.67:0/3998508286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc558103590 0x7fc558199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc5567fc640 1 -- 172.21.15.67:0/3998508286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc558109820 msgr2=0x7fc55819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc5567fc640 1 --2- 172.21.15.67:0/3998508286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc558109820 0x7fc55819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.981+0000 7fc5567fc640 1 -- 172.21.15.67:0/3998508286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55819fe10 con 0x7fc558108ed0 2024-07-31T15:09:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.982+0000 7fc5567fc640 1 --2- 172.21.15.67:0/3998508286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc558108ed0 0x7fc55819a440 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00e9f0 tx=0x7fc54c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.982+0000 7fc53ffff640 1 -- 172.21.15.67:0/3998508286 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c00ccc0 con 0x7fc558108ed0 2024-07-31T15:09:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.982+0000 7fc53ffff640 1 -- 172.21.15.67:0/3998508286 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc54c0108e0 con 0x7fc558108ed0 2024-07-31T15:09:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.982+0000 7fc55d497640 1 -- 172.21.15.67:0/3998508286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5581a0100 con 0x7fc558108ed0 2024-07-31T15:09:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.982+0000 7fc53ffff640 1 -- 172.21.15.67:0/3998508286 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c018860 con 0x7fc558108ed0 2024-07-31T15:09:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.982+0000 7fc55d497640 1 -- 172.21.15.67:0/3998508286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5581a6860 con 0x7fc558108ed0 2024-07-31T15:09:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.984+0000 7fc5639e8740 1 Processor -- start 2024-07-31T15:09:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.984+0000 7fc5639e8740 1 -- start start 2024-07-31T15:09:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc5639e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012f7070 0x56320169c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc5639e8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x56320169cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc5639e8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012d6390 0x56320169cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc5639e8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563201612bb0 con 0x5632012ce1b0 2024-07-31T15:09:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc5639e8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5632016020b0 con 0x5632012d6390 2024-07-31T15:09:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc5639e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5632016060f0 con 0x5632012f7070 2024-07-31T15:09:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc53ffff640 1 -- 172.21.15.67:0/3998508286 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc54c018a00 con 0x7fc558108ed0 2024-07-31T15:09:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc556ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x56320169cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc5577fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012f7070 0x56320169c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc556ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x56320169cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49672/0 (socket says 172.21.15.67:49672) 2024-07-31T15:09:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc5577fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012f7070 0x56320169c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50164/0 (socket says 172.21.15.67:50164) 2024-07-31T15:09:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc556ffd640 1 -- 172.21.15.67:0/1757295185 learned_addr learned my addr 172.21.15.67:0/1757295185 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc5567fc640 1 --2- 172.21.15.67:0/1757295185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012d6390 0x56320169cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc556ffd640 1 -- 172.21.15.67:0/1757295185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012f7070 msgr2=0x56320169c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc556ffd640 1 --2- 172.21.15.67:0/1757295185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012f7070 0x56320169c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc556ffd640 1 -- 172.21.15.67:0/1757295185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012d6390 msgr2=0x56320169cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc556ffd640 1 --2- 172.21.15.67:0/1757295185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012d6390 0x56320169cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc556ffd640 1 -- 172.21.15.67:0/1757295185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632015a8b90 con 0x5632012ce1b0 2024-07-31T15:09:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc5567fc640 1 --2- 172.21.15.67:0/1757295185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012d6390 0x56320169cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.985+0000 7fc556ffd640 1 --2- 172.21.15.67:0/1757295185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x56320169cab0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fc54800cbe0 tx=0x7fc54800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b76c2a7162ccd6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.986+0000 7fc53d7fa640 1 -- 172.21.15.67:0/1757295185 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548010630 con 0x5632012ce1b0 2024-07-31T15:09:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.986+0000 7fc53d7fa640 1 -- 172.21.15.67:0/1757295185 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc548010c80 con 0x5632012ce1b0 2024-07-31T15:09:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.986+0000 7fc53d7fa640 1 -- 172.21.15.67:0/1757295185 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54801ab10 con 0x5632012ce1b0 2024-07-31T15:09:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.986+0000 7fc53ffff640 1 --2- 172.21.15.67:0/3998508286 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc5400b3e40 0x7fc5400b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.986+0000 7fc5577fe640 1 --2- 172.21.15.67:0/3998508286 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc5400b3e40 0x7fc5400b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.986+0000 7fc53ffff640 1 -- 172.21.15.67:0/3998508286 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7fc54c0e46c0 con 0x7fc558108ed0 2024-07-31T15:09:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.987+0000 7fc5639e8740 1 -- 172.21.15.67:0/1757295185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 msgr2=0x56320169cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.987+0000 7fc5639e8740 1 --2- 172.21.15.67:0/1757295185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x56320169cab0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fc54800cbe0 tx=0x7fc54800e720 comp rx=0 tx=0).stop 2024-07-31T15:09:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.987+0000 7fc5639e8740 1 -- 172.21.15.67:0/1757295185 shutdown_connections 2024-07-31T15:09:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.987+0000 7fc5639e8740 1 --2- 172.21.15.67:0/1757295185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012d6390 0x56320169cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.987+0000 7fc5639e8740 1 --2- 172.21.15.67:0/1757295185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x56320169cab0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.987+0000 7fc5639e8740 1 --2- 172.21.15.67:0/1757295185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012f7070 0x56320169c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.987+0000 7fc5639e8740 1 -- 172.21.15.67:0/1757295185 >> 172.21.15.67:0/1757295185 conn(0x5632012eb460 msgr2=0x56320169a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.987+0000 7fc5639e8740 1 -- 172.21.15.67:0/1757295185 shutdown_connections 2024-07-31T15:09:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.987+0000 7fc5577fe640 1 --2- 172.21.15.67:0/3998508286 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc5400b3e40 0x7fc5400b6350 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc550006010 tx=0x7fc55002f3b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.987+0000 7fc5639e8740 1 -- 172.21.15.67:0/1757295185 wait complete. 2024-07-31T15:09:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.987+0000 7fc5639e8740 1 -- start start 2024-07-31T15:09:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.987+0000 7fc5639e8740 1 Processor -- start 2024-07-31T15:09:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc5639e8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x5632016a8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc5639e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012d6390 0x5632016a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc5639e8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012f7070 0x5632016b4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc5639e8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563201465700 con 0x5632012ce1b0 2024-07-31T15:09:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc5639e8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5632015a49e0 con 0x5632012f7070 2024-07-31T15:09:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc5639e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5632016aad50 con 0x5632012d6390 2024-07-31T15:09:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc5639e8740 10 client.0 _ll_register_callbacks cb 0x5632015bf4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc556ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x5632016a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc556ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x5632016a8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49688/0 (socket says 172.21.15.67:49688) 2024-07-31T15:09:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc556ffd640 1 -- 172.21.15.67:0/256894219 learned_addr learned my addr 172.21.15.67:0/256894219 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc5577fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012f7070 0x5632016b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc5567fc640 1 --2- 172.21.15.67:0/256894219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012d6390 0x5632016a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:32.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc556ffd640 1 -- 172.21.15.67:0/256894219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012d6390 msgr2=0x5632016a86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:32.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc556ffd640 1 --2- 172.21.15.67:0/256894219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012d6390 0x5632016a86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc556ffd640 1 -- 172.21.15.67:0/256894219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012f7070 msgr2=0x5632016b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:32.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc556ffd640 1 --2- 172.21.15.67:0/256894219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012f7070 0x5632016b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc556ffd640 1 -- 172.21.15.67:0/256894219 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563201362650 con 0x5632012ce1b0 2024-07-31T15:09:32.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc556ffd640 1 --2- 172.21.15.67:0/256894219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x5632016a8190 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fc54800c920 tx=0x7fc548005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9324ff3ef75167b0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:32.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.988+0000 7fc53aff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:32.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.989+0000 7fc53aff5640 1 -- 172.21.15.67:0/256894219 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54802a060 con 0x5632012ce1b0 2024-07-31T15:09:32.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.989+0000 7fc5639e8740 1 -- 172.21.15.67:0/256894219 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc548010c80 con 0x5632012ce1b0 2024-07-31T15:09:32.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.989+0000 7fc53aff5640 1 -- 172.21.15.67:0/256894219 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7fc54801a8d0 con 0x5632012ce1b0 2024-07-31T15:09:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.989+0000 7fc53aff5640 1 -- 172.21.15.67:0/256894219 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fc548031050 con 0x5632012ce1b0 2024-07-31T15:09:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.989+0000 7fc5639e8740 10 client.19365 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.989+0000 7fc5639e8740 1 -- 172.21.15.67:0/256894219 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5632016b6da0 con 0x5632012ce1b0 2024-07-31T15:09:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.989+0000 7fc53aff5640 1 -- 172.21.15.67:0/256894219 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc548010c80 con 0x5632012ce1b0 2024-07-31T15:09:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.989+0000 7fc53aff5640 1 -- 172.21.15.67:0/256894219 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fc548025420 con 0x5632012ce1b0 2024-07-31T15:09:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.989+0000 7fc5639e8740 10 client.19365 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.989+0000 7fc5639e8740 10 client.19365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:09:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.989+0000 7fc5639e8740 20 client.19365 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.989+0000 7fc5639e8740 1 --2- 172.21.15.67:0/256894219 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632012b59f0 0x5632016ba1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.989+0000 7fc5639e8740 4 client.19365 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:09:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.989+0000 7fc5639e8740 1 -- 172.21.15.67:0/256894219 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563201614b00 con 0x5632012b59f0 2024-07-31T15:09:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.990+0000 7fc5567fc640 1 --2- 172.21.15.67:0/256894219 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632012b59f0 0x5632016ba1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.990+0000 7fc5567fc640 1 --2- 172.21.15.67:0/256894219 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632012b59f0 0x5632016ba1b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b6cb7482f3782a7 server_cookie=55c805b2a6bd5baf in_seq=0 out_seq=0 2024-07-31T15:09:32.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.990+0000 7fc53aff5640 10 client.19365 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:09:32.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc53aff5640 1 -- 172.21.15.67:0/256894219 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563201614b00 con 0x5632012b59f0 2024-07-31T15:09:32.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc53aff5640 10 client.19365 handle_command_reply: tid=0 2024-07-31T15:09:32.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc5639e8740 1 client.19365 shutdown 2024-07-31T15:09:32.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc5639e8740 1 -- 172.21.15.67:0/256894219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 msgr2=0x5632016a8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:32.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc5639e8740 1 --2- 172.21.15.67:0/256894219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x5632016a8190 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fc54800c920 tx=0x7fc548005740 comp rx=0 tx=0).stop 2024-07-31T15:09:32.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc5639e8740 1 -- 172.21.15.67:0/256894219 shutdown_connections 2024-07-31T15:09:32.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc5639e8740 1 --2- 172.21.15.67:0/256894219 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632012b59f0 0x5632016ba1b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc5639e8740 1 --2- 172.21.15.67:0/256894219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012f7070 0x5632016b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc5639e8740 1 --2- 172.21.15.67:0/256894219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012d6390 0x5632016a86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc5639e8740 1 --2- 172.21.15.67:0/256894219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x5632016a8190 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc53aff5640 0 client.19365 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:09:32.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc5639e8740 1 -- 172.21.15.67:0/256894219 >> 172.21.15.67:0/256894219 conn(0x5632012eb460 msgr2=0x563201698c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:32.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc5639e8740 1 -- 172.21.15.67:0/256894219 shutdown_connections 2024-07-31T15:09:32.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc5639e8740 1 -- 172.21.15.67:0/256894219 wait complete. 2024-07-31T15:09:32.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:31.991+0000 7fc5639e8740 20 client.19365 trim_cache size 0 max 16384 2024-07-31T15:09:32.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.005+0000 7fc5639e8740 1 Processor -- start 2024-07-31T15:09:32.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.005+0000 7fc5639e8740 1 -- start start 2024-07-31T15:09:32.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.005+0000 7fc5639e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012b59f0 0x5632016b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:32.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.005+0000 7fc5639e8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012ce1b0 0x56320169dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:32.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.005+0000 7fc5639e8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012d6390 0x5632016bd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:32.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.005+0000 7fc5639e8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc5581b06f0 con 0x5632012d6390 2024-07-31T15:09:32.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.005+0000 7fc5639e8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc5581b2230 con 0x5632012ce1b0 2024-07-31T15:09:32.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.005+0000 7fc5639e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc5581b0dc0 con 0x5632012b59f0 2024-07-31T15:09:32.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.005+0000 7fc5577fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012d6390 0x5632016bd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:32.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.006+0000 7fc5577fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012d6390 0x5632016bd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49692/0 (socket says 172.21.15.67:49692) 2024-07-31T15:09:32.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.006+0000 7fc5577fe640 1 -- 172.21.15.67:0/1205776014 learned_addr learned my addr 172.21.15.67:0/1205776014 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:32.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.006+0000 7fc5567fc640 1 --2- 172.21.15.67:0/1205776014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012ce1b0 0x56320169dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:32.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.006+0000 7fc556ffd640 1 --2- 172.21.15.67:0/1205776014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012b59f0 0x5632016b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:32.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.006+0000 7fc5577fe640 1 -- 172.21.15.67:0/1205776014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012b59f0 msgr2=0x5632016b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:32.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.006+0000 7fc5577fe640 1 --2- 172.21.15.67:0/1205776014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012b59f0 0x5632016b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.006+0000 7fc5577fe640 1 -- 172.21.15.67:0/1205776014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012ce1b0 msgr2=0x56320169dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:32.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.006+0000 7fc5577fe640 1 --2- 172.21.15.67:0/1205776014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012ce1b0 0x56320169dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.006+0000 7fc5577fe640 1 -- 172.21.15.67:0/1205776014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320169f3d0 con 0x5632012d6390 2024-07-31T15:09:32.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.006+0000 7fc5567fc640 1 --2- 172.21.15.67:0/1205776014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012ce1b0 0x56320169dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:32.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.006+0000 7fc5577fe640 1 --2- 172.21.15.67:0/1205776014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012d6390 0x5632016bd1d0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fc55002ade0 tx=0x7fc550044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cebaffb37305995 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:32.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.006+0000 7fc53cff9640 1 -- 172.21.15.67:0/1205776014 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc550003af0 con 0x5632012d6390 2024-07-31T15:09:32.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.006+0000 7fc53cff9640 1 -- 172.21.15.67:0/1205776014 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc550036050 con 0x5632012d6390 2024-07-31T15:09:32.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.006+0000 7fc53cff9640 1 -- 172.21.15.67:0/1205776014 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55002f990 con 0x5632012d6390 2024-07-31T15:09:32.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.007+0000 7fc5639e8740 1 -- 172.21.15.67:0/1205776014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012d6390 msgr2=0x5632016bd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:32.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.007+0000 7fc5639e8740 1 --2- 172.21.15.67:0/1205776014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012d6390 0x5632016bd1d0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fc55002ade0 tx=0x7fc550044000 comp rx=0 tx=0).stop 2024-07-31T15:09:32.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.007+0000 7fc5639e8740 1 -- 172.21.15.67:0/1205776014 shutdown_connections 2024-07-31T15:09:32.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.007+0000 7fc5639e8740 1 --2- 172.21.15.67:0/1205776014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012d6390 0x5632016bd1d0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.007+0000 7fc5639e8740 1 --2- 172.21.15.67:0/1205776014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012ce1b0 0x56320169dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.007+0000 7fc5639e8740 1 --2- 172.21.15.67:0/1205776014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012b59f0 0x5632016b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.007+0000 7fc5639e8740 1 -- 172.21.15.67:0/1205776014 >> 172.21.15.67:0/1205776014 conn(0x5632012eb460 msgr2=0x56320169d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:32.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.007+0000 7fc5639e8740 1 -- 172.21.15.67:0/1205776014 shutdown_connections 2024-07-31T15:09:32.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.007+0000 7fc5639e8740 1 -- 172.21.15.67:0/1205776014 wait complete. 2024-07-31T15:09:32.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.007+0000 7fc5639e8740 1 -- start start 2024-07-31T15:09:32.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5639e8740 1 Processor -- start 2024-07-31T15:09:32.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5639e8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012b59f0 0x5632016bb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:32.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5639e8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x5632016bb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:32.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5639e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012d6390 0x5632016bbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:32.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5639e8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc5581b1b60 con 0x5632012ce1b0 2024-07-31T15:09:32.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5639e8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc5581b1490 con 0x5632012b59f0 2024-07-31T15:09:32.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5639e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc5581ae6f0 con 0x5632012d6390 2024-07-31T15:09:32.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5639e8740 10 client.0 _ll_register_callbacks cb 0x7fc548014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:32.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5567fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x5632016bb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:32.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5567fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x5632016bb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49704/0 (socket says 172.21.15.67:49704) 2024-07-31T15:09:32.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc556ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012b59f0 0x5632016bb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:32.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5567fc640 1 -- 172.21.15.67:0/2093878995 learned_addr learned my addr 172.21.15.67:0/2093878995 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:32.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5577fe640 1 --2- 172.21.15.67:0/2093878995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012d6390 0x5632016bbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:32.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5567fc640 1 -- 172.21.15.67:0/2093878995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012d6390 msgr2=0x5632016bbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:32.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5567fc640 1 --2- 172.21.15.67:0/2093878995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012d6390 0x5632016bbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5567fc640 1 -- 172.21.15.67:0/2093878995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012b59f0 msgr2=0x5632016bb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:32.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5567fc640 1 --2- 172.21.15.67:0/2093878995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012b59f0 0x5632016bb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5567fc640 1 -- 172.21.15.67:0/2093878995 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563201721880 con 0x5632012ce1b0 2024-07-31T15:09:32.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc5577fe640 1 --2- 172.21.15.67:0/2093878995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012d6390 0x5632016bbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:09:32.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.008+0000 7fc556ffd640 1 --2- 172.21.15.67:0/2093878995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012b59f0 0x5632016bb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:32.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc5567fc640 1 --2- 172.21.15.67:0/2093878995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x5632016bb790 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0a81a0 tx=0x7fc54c009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e4eaab46424d87 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:32.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc53aff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:32.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc53aff5640 1 -- 172.21.15.67:0/2093878995 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c08be80 con 0x5632012ce1b0 2024-07-31T15:09:32.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc53aff5640 1 -- 172.21.15.67:0/2093878995 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7fc54c0a71c0 con 0x5632012ce1b0 2024-07-31T15:09:32.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc5639e8740 1 -- 172.21.15.67:0/2093878995 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc550036050 con 0x5632012ce1b0 2024-07-31T15:09:32.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc53aff5640 1 -- 172.21.15.67:0/2093878995 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fc54c0aaf30 con 0x5632012ce1b0 2024-07-31T15:09:32.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc5639e8740 10 client.19371 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:32.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc5639e8740 1 -- 172.21.15.67:0/2093878995 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x563201722b60 con 0x5632012ce1b0 2024-07-31T15:09:32.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc53aff5640 1 -- 172.21.15.67:0/2093878995 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc550036050 con 0x5632012ce1b0 2024-07-31T15:09:32.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc53aff5640 1 -- 172.21.15.67:0/2093878995 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fc54c0b3b50 con 0x5632012ce1b0 2024-07-31T15:09:32.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc5639e8740 10 client.19371 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:32.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc5639e8740 10 client.19371 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:09:32.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc5639e8740 20 client.19371 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:32.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc5639e8740 1 --2- 172.21.15.67:0/2093878995 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632012f7070 0x5632017252d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:32.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc5639e8740 4 client.19371 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:09:32.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.009+0000 7fc5639e8740 1 -- 172.21.15.67:0/2093878995 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632016105a0 con 0x5632012f7070 2024-07-31T15:09:32.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc556ffd640 1 --2- 172.21.15.67:0/2093878995 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632012f7070 0x5632017252d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:32.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc556ffd640 1 --2- 172.21.15.67:0/2093878995 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632012f7070 0x5632017252d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4fc0c2a7b1555352 server_cookie=df3ab80f932cda18 in_seq=0 out_seq=0 2024-07-31T15:09:32.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc53aff5640 10 client.19371 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:09:32.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc53aff5640 1 -- 172.21.15.67:0/2093878995 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632016105a0 con 0x5632012f7070 2024-07-31T15:09:32.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc53aff5640 10 client.19371 handle_command_reply: tid=0 2024-07-31T15:09:32.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc5639e8740 1 client.19371 shutdown 2024-07-31T15:09:32.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc5639e8740 1 -- 172.21.15.67:0/2093878995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 msgr2=0x5632016bb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:32.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc5639e8740 1 --2- 172.21.15.67:0/2093878995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x5632016bb790 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0a81a0 tx=0x7fc54c009740 comp rx=0 tx=0).stop 2024-07-31T15:09:32.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc5639e8740 1 -- 172.21.15.67:0/2093878995 shutdown_connections 2024-07-31T15:09:32.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc5639e8740 1 --2- 172.21.15.67:0/2093878995 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x5632012f7070 0x5632017252d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc5639e8740 1 --2- 172.21.15.67:0/2093878995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5632012d6390 0x5632016bbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc5639e8740 1 --2- 172.21.15.67:0/2093878995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5632012ce1b0 0x5632016bb790 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc5639e8740 1 --2- 172.21.15.67:0/2093878995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632012b59f0 0x5632016bb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc5639e8740 1 -- 172.21.15.67:0/2093878995 >> 172.21.15.67:0/2093878995 conn(0x5632012eb460 msgr2=0x56320169d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:32.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.010+0000 7fc53aff5640 0 client.19371 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:09:32.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc5639e8740 1 -- 172.21.15.67:0/2093878995 shutdown_connections 2024-07-31T15:09:32.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc5639e8740 1 -- 172.21.15.67:0/2093878995 wait complete. 2024-07-31T15:09:32.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc5639e8740 20 client.19371 trim_cache size 0 max 16384 2024-07-31T15:09:32.018 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:09:32.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc55d497640 1 -- 172.21.15.67:0/3998508286 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc5400b3e40 msgr2=0x7fc5400b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:32.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc55d497640 1 --2- 172.21.15.67:0/3998508286 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc5400b3e40 0x7fc5400b6350 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc550006010 tx=0x7fc55002f3b0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc55d497640 1 -- 172.21.15.67:0/3998508286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc558108ed0 msgr2=0x7fc55819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:32.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc55d497640 1 --2- 172.21.15.67:0/3998508286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc558108ed0 0x7fc55819a440 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00e9f0 tx=0x7fc54c00eeb0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc55d497640 1 -- 172.21.15.67:0/3998508286 shutdown_connections 2024-07-31T15:09:32.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc55d497640 1 --2- 172.21.15.67:0/3998508286 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc5400b3e40 0x7fc5400b6350 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc55d497640 1 --2- 172.21.15.67:0/3998508286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc558109820 0x7fc55819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc55d497640 1 --2- 172.21.15.67:0/3998508286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc558108ed0 0x7fc55819a440 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc55d497640 1 --2- 172.21.15.67:0/3998508286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc558103590 0x7fc558199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc55d497640 1 -- 172.21.15.67:0/3998508286 >> 172.21.15.67:0/3998508286 conn(0x7fc55807b280 msgr2=0x7fc558106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:32.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc55d497640 1 -- 172.21.15.67:0/3998508286 shutdown_connections 2024-07-31T15:09:32.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:32.011+0000 7fc55d497640 1 -- 172.21.15.67:0/3998508286 wait complete. 2024-07-31T15:09:32.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T15:09:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7efff17fa640 1 -- 172.21.15.67:0/1301895760 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564915fb2da0 con 0x564915c9d1b0 2024-07-31T15:09:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7efff17fa640 10 client.19335 handle_command_reply: tid=0 2024-07-31T15:09:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7f0017007740 1 client.19335 shutdown 2024-07-31T15:09:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7f0017007740 1 -- 172.21.15.67:0/1301895760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c311c0 msgr2=0x564916053f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7f0017007740 1 --2- 172.21.15.67:0/1301895760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c311c0 0x564916053f70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f0004009a40 tx=0x7f0004013d00 comp rx=0 tx=0).stop 2024-07-31T15:09:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7f0017007740 1 -- 172.21.15.67:0/1301895760 shutdown_connections 2024-07-31T15:09:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7f0017007740 1 --2- 172.21.15.67:0/1301895760 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x564915c9d1b0 0x5649160c7960 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7f0017007740 1 --2- 172.21.15.67:0/1301895760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564915c7c4d0 0x564916050b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7f0017007740 1 --2- 172.21.15.67:0/1301895760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564915c5bd80 0x564916050420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7efff17fa640 0 client.19335 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7f0017007740 1 --2- 172.21.15.67:0/1301895760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564915c311c0 0x564916053f70 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7f0017007740 1 -- 172.21.15.67:0/1301895760 >> 172.21.15.67:0/1301895760 conn(0x564915c91300 msgr2=0x56491604ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7f0017007740 1 -- 172.21.15.67:0/1301895760 shutdown_connections 2024-07-31T15:09:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7f0017007740 1 -- 172.21.15.67:0/1301895760 wait complete. 2024-07-31T15:09:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.772+0000 7f0017007740 20 client.19335 trim_cache size 0 max 16384 2024-07-31T15:09:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:09:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:09:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:09:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:09:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.773+0000 7f0010115640 1 -- 172.21.15.67:0/2348283962 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efff40b3dc0 msgr2=0x7efff40b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.773+0000 7f0010115640 1 --2- 172.21.15.67:0/2348283962 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efff40b3dc0 0x7efff40b62d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efff8009fd0 tx=0x7efff80096c0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.773+0000 7f0010115640 1 -- 172.21.15.67:0/2348283962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f000810d7a0 msgr2=0x7f00081a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.773+0000 7f0010115640 1 --2- 172.21.15.67:0/2348283962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f000810d7a0 0x7f00081a18f0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0000005cc0 tx=0x7f0000031e40 comp rx=0 tx=0).stop 2024-07-31T15:09:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.773+0000 7f0010115640 1 -- 172.21.15.67:0/2348283962 shutdown_connections 2024-07-31T15:09:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.773+0000 7f0010115640 1 --2- 172.21.15.67:0/2348283962 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efff40b3dc0 0x7efff40b62d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.773+0000 7f0010115640 1 --2- 172.21.15.67:0/2348283962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f000810d7a0 0x7f00081a18f0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.773+0000 7f0010115640 1 --2- 172.21.15.67:0/2348283962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0008104c90 0x7f000819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.773+0000 7f0010115640 1 --2- 172.21.15.67:0/2348283962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0008104340 0x7f000819c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.773+0000 7f0010115640 1 -- 172.21.15.67:0/2348283962 >> 172.21.15.67:0/2348283962 conn(0x7f00080fddf0 msgr2=0x7f0008101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.773+0000 7f0010115640 1 -- 172.21.15.67:0/2348283962 shutdown_connections 2024-07-31T15:09:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:32.773+0000 7f0010115640 1 -- 172.21.15.67:0/2348283962 wait complete. 2024-07-31T15:09:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:09:33.477 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:33 smithi067 ceph-mon[32750]: pgmap v988: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 234 KiB/s wr, 70 op/s 2024-07-31T15:09:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:33 smithi077 ceph-mon[37327]: pgmap v988: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 234 KiB/s wr, 70 op/s 2024-07-31T15:09:33.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:33 smithi073 ceph-mon[36714]: pgmap v988: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 234 KiB/s wr, 70 op/s 2024-07-31T15:09:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.834+0000 7f5959d8e640 1 -- 172.21.15.67:0/4114992866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5954108720 msgr2=0x7f595410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.834+0000 7f5959d8e640 1 --2- 172.21.15.67:0/4114992866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5954108720 0x7f595410ef50 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f594c009ca0 tx=0x7f594c02a8c0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.834+0000 7f5959d8e640 1 -- 172.21.15.67:0/4114992866 shutdown_connections 2024-07-31T15:09:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.834+0000 7f5959d8e640 1 --2- 172.21.15.67:0/4114992866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5954108720 0x7f595410ef50 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.834+0000 7f5959d8e640 1 --2- 172.21.15.67:0/4114992866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5954107dd0 0x7f59541081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.834+0000 7f5959d8e640 1 --2- 172.21.15.67:0/4114992866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5954102490 0x7f59541028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.834+0000 7f5959d8e640 1 -- 172.21.15.67:0/4114992866 >> 172.21.15.67:0/4114992866 conn(0x7f5954079160 msgr2=0x7f5954079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.834+0000 7f5959d8e640 1 -- 172.21.15.67:0/4114992866 shutdown_connections 2024-07-31T15:09:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.834+0000 7f5959d8e640 1 -- 172.21.15.67:0/4114992866 wait complete. 2024-07-31T15:09:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.835+0000 7f5959d8e640 1 Processor -- start 2024-07-31T15:09:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.835+0000 7f5959d8e640 1 -- start start 2024-07-31T15:09:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.835+0000 7f5959d8e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5954102490 0x7f5954199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.835+0000 7f5959d8e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5954107dd0 0x7f595419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.835+0000 7f5959d8e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5954108720 0x7f595419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.835+0000 7f5959d8e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5954112380 con 0x7f5954108720 2024-07-31T15:09:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.835+0000 7f5959d8e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5954112200 con 0x7f5954102490 2024-07-31T15:09:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.835+0000 7f5959d8e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5954112500 con 0x7f5954107dd0 2024-07-31T15:09:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.835+0000 7f5953fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5954108720 0x7f595419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.835+0000 7f5953fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5954108720 0x7f595419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49708/0 (socket says 172.21.15.67:49708) 2024-07-31T15:09:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.835+0000 7f5953fff640 1 -- 172.21.15.67:0/2838483547 learned_addr learned my addr 172.21.15.67:0/2838483547 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.835+0000 7f59537fe640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5954102490 0x7f5954199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.835+0000 7f5952ffd640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5954107dd0 0x7f595419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.836+0000 7f5953fff640 1 -- 172.21.15.67:0/2838483547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5954107dd0 msgr2=0x7f595419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.836+0000 7f5953fff640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5954107dd0 0x7f595419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.836+0000 7f5953fff640 1 -- 172.21.15.67:0/2838483547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5954102490 msgr2=0x7f5954199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.836+0000 7f5953fff640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5954102490 0x7f5954199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.836+0000 7f5953fff640 1 -- 172.21.15.67:0/2838483547 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595419fd30 con 0x7f5954108720 2024-07-31T15:09:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.836+0000 7f59537fe640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5954102490 0x7f5954199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.836+0000 7f5953fff640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5954108720 0x7f595419f6c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f594c0098d0 tx=0x7f594c0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.836+0000 7f5950ff9640 1 -- 172.21.15.67:0/2838483547 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c003b70 con 0x7f5954108720 2024-07-31T15:09:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.836+0000 7f5950ff9640 1 -- 172.21.15.67:0/2838483547 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f594c003d10 con 0x7f5954108720 2024-07-31T15:09:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.836+0000 7f5959d8e640 1 -- 172.21.15.67:0/2838483547 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595419ffc0 con 0x7f5954108720 2024-07-31T15:09:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.836+0000 7f5950ff9640 1 -- 172.21.15.67:0/2838483547 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c02f9d0 con 0x7f5954108720 2024-07-31T15:09:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.836+0000 7f5959d8e640 1 -- 172.21.15.67:0/2838483547 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59541a0500 con 0x7f5954108720 2024-07-31T15:09:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.839+0000 7f59602df740 1 Processor -- start 2024-07-31T15:09:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.839+0000 7f5950ff9640 1 -- 172.21.15.67:0/2838483547 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f594c02fb70 con 0x7f5954108720 2024-07-31T15:09:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.839+0000 7f59602df740 1 -- start start 2024-07-31T15:09:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.839+0000 7f59602df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e49b070 0x56260e840690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.839+0000 7f59602df740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e4721b0 0x56260e840bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.839+0000 7f59602df740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e47a390 0x56260e841110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.839+0000 7f59602df740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56260e7b6bb0 con 0x56260e47a390 2024-07-31T15:09:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.839+0000 7f59602df740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56260e7a60b0 con 0x56260e4721b0 2024-07-31T15:09:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.839+0000 7f59602df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56260e7aa0f0 con 0x56260e49b070 2024-07-31T15:09:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.839+0000 7f5952ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e47a390 0x56260e841110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.839+0000 7f59537fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e4721b0 0x56260e840bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.839+0000 7f5952ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e47a390 0x56260e841110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49716/0 (socket says 172.21.15.67:49716) 2024-07-31T15:09:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.839+0000 7f5952ffd640 1 -- 172.21.15.67:0/1769172627 learned_addr learned my addr 172.21.15.67:0/1769172627 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.839+0000 7f5952ffd640 1 --2- 172.21.15.67:0/1769172627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e49b070 0x56260e840690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.840+0000 7f59537fe640 1 -- 172.21.15.67:0/1769172627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e49b070 msgr2=0x56260e840690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.840+0000 7f59537fe640 1 --2- 172.21.15.67:0/1769172627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e49b070 0x56260e840690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.840+0000 7f59537fe640 1 -- 172.21.15.67:0/1769172627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e47a390 msgr2=0x56260e841110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.840+0000 7f59537fe640 1 --2- 172.21.15.67:0/1769172627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e47a390 0x56260e841110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.840+0000 7f5950ff9640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f59400b3d70 0x7f59400b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.840+0000 7f59537fe640 1 -- 172.21.15.67:0/1769172627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260e74cb90 con 0x56260e4721b0 2024-07-31T15:09:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.840+0000 7f5952ffd640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f59400b3d70 0x7f59400b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.840+0000 7f59537fe640 1 --2- 172.21.15.67:0/1769172627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e4721b0 0x56260e840bd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f594400afc0 tx=0x7f594400f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bb0daeda1278144 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.840+0000 7f5950ff9640 1 -- 172.21.15.67:0/2838483547 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f594c105790 con 0x7f5954108720 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.840+0000 7f5949ffb640 1 -- 172.21.15.67:0/1769172627 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944005220 con 0x56260e4721b0 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.840+0000 7f5949ffb640 1 -- 172.21.15.67:0/1769172627 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5944004c50 con 0x56260e4721b0 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.840+0000 7f5949ffb640 1 -- 172.21.15.67:0/1769172627 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944010bd0 con 0x56260e4721b0 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.841+0000 7f59602df740 1 -- 172.21.15.67:0/1769172627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e4721b0 msgr2=0x56260e840bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.841+0000 7f5952ffd640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f59400b3d70 0x7f59400b6280 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f593c009fd0 tx=0x7f593c004730 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.841+0000 7f59602df740 1 --2- 172.21.15.67:0/1769172627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e4721b0 0x56260e840bd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f594400afc0 tx=0x7f594400f850 comp rx=0 tx=0).stop 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.841+0000 7f59602df740 1 -- 172.21.15.67:0/1769172627 shutdown_connections 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.841+0000 7f59602df740 1 --2- 172.21.15.67:0/1769172627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e47a390 0x56260e841110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.841+0000 7f59602df740 1 --2- 172.21.15.67:0/1769172627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e4721b0 0x56260e840bd0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.841+0000 7f59602df740 1 --2- 172.21.15.67:0/1769172627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e49b070 0x56260e840690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.841+0000 7f59602df740 1 -- 172.21.15.67:0/1769172627 >> 172.21.15.67:0/1769172627 conn(0x56260e48f460 msgr2=0x56260e83e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.841+0000 7f59602df740 1 -- 172.21.15.67:0/1769172627 shutdown_connections 2024-07-31T15:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.841+0000 7f59602df740 1 -- 172.21.15.67:0/1769172627 wait complete. 2024-07-31T15:09:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.841+0000 7f59602df740 1 -- start start 2024-07-31T15:09:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.841+0000 7f59602df740 1 Processor -- start 2024-07-31T15:09:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f59602df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e4721b0 0x56260e850610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f59602df740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e47a390 0x56260e850b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f59602df740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e49b070 0x56260e85c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f59602df740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56260e609700 con 0x56260e49b070 2024-07-31T15:09:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f59602df740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56260e7489e0 con 0x56260e47a390 2024-07-31T15:09:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f59602df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56260e853110 con 0x56260e4721b0 2024-07-31T15:09:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f59602df740 10 client.0 _ll_register_callbacks cb 0x56260e7634f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f5953fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e49b070 0x56260e85c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f5953fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e49b070 0x56260e85c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49720/0 (socket says 172.21.15.67:49720) 2024-07-31T15:09:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f5953fff640 1 -- 172.21.15.67:0/845503813 learned_addr learned my addr 172.21.15.67:0/845503813 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f5952ffd640 1 --2- 172.21.15.67:0/845503813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e47a390 0x56260e850b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f59537fe640 1 --2- 172.21.15.67:0/845503813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e4721b0 0x56260e850610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f5953fff640 1 -- 172.21.15.67:0/845503813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e4721b0 msgr2=0x56260e850610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:34.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f5953fff640 1 --2- 172.21.15.67:0/845503813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e4721b0 0x56260e850610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f5953fff640 1 -- 172.21.15.67:0/845503813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e47a390 msgr2=0x56260e850b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:34.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f5953fff640 1 --2- 172.21.15.67:0/845503813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e47a390 0x56260e850b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f5953fff640 1 -- 172.21.15.67:0/845503813 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260e506650 con 0x56260e49b070 2024-07-31T15:09:34.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f5953fff640 1 --2- 172.21.15.67:0/845503813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e49b070 0x56260e85c440 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f594c0c81f0 tx=0x7f594c0cb9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8de773aa75420d1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:34.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.842+0000 7f5932ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:34.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.843+0000 7f5932ffd640 1 -- 172.21.15.67:0/845503813 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c0d0770 con 0x56260e49b070 2024-07-31T15:09:34.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.843+0000 7f59602df740 1 -- 172.21.15.67:0/845503813 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5944004c50 con 0x56260e49b070 2024-07-31T15:09:34.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.843+0000 7f5932ffd640 1 -- 172.21.15.67:0/845503813 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f594c0c8400 con 0x56260e49b070 2024-07-31T15:09:34.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.843+0000 7f5932ffd640 1 -- 172.21.15.67:0/845503813 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f594c0d5220 con 0x56260e49b070 2024-07-31T15:09:34.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.843+0000 7f59602df740 10 client.19383 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:34.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.843+0000 7f59602df740 1 -- 172.21.15.67:0/845503813 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x56260e85f0e0 con 0x56260e49b070 2024-07-31T15:09:34.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.843+0000 7f5932ffd640 1 -- 172.21.15.67:0/845503813 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5944004c50 con 0x56260e49b070 2024-07-31T15:09:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.844+0000 7f5932ffd640 1 -- 172.21.15.67:0/845503813 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f594c104070 con 0x56260e49b070 2024-07-31T15:09:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.844+0000 7f59602df740 10 client.19383 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.844+0000 7f59602df740 10 client.19383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.844+0000 7f59602df740 20 client.19383 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.844+0000 7f59602df740 1 --2- 172.21.15.67:0/845503813 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56260e4599f0 0x56260e8625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.844+0000 7f59602df740 4 client.19383 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:09:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.844+0000 7f59602df740 1 -- 172.21.15.67:0/845503813 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56260e7b8b00 con 0x56260e4599f0 2024-07-31T15:09:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.844+0000 7f59537fe640 1 --2- 172.21.15.67:0/845503813 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56260e4599f0 0x56260e8625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.844+0000 7f5932ffd640 10 client.19383 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.844+0000 7f59537fe640 1 --2- 172.21.15.67:0/845503813 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56260e4599f0 0x56260e8625e0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad82a83103d87055 server_cookie=69e481828a54ec16 in_seq=0 out_seq=0 2024-07-31T15:09:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f5932ffd640 1 -- 172.21.15.67:0/845503813 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56260e7b8b00 con 0x56260e4599f0 2024-07-31T15:09:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f5932ffd640 10 client.19383 handle_command_reply: tid=0 2024-07-31T15:09:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f59602df740 1 client.19383 shutdown 2024-07-31T15:09:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f59602df740 1 -- 172.21.15.67:0/845503813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e49b070 msgr2=0x56260e85c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f59602df740 1 --2- 172.21.15.67:0/845503813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e49b070 0x56260e85c440 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f594c0c81f0 tx=0x7f594c0cb9a0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f59602df740 1 -- 172.21.15.67:0/845503813 shutdown_connections 2024-07-31T15:09:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f59602df740 1 --2- 172.21.15.67:0/845503813 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56260e4599f0 0x56260e8625e0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f59602df740 1 --2- 172.21.15.67:0/845503813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e49b070 0x56260e85c440 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f59602df740 1 --2- 172.21.15.67:0/845503813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e47a390 0x56260e850b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f5932ffd640 0 client.19383 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:34.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f59602df740 1 --2- 172.21.15.67:0/845503813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e4721b0 0x56260e850610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f59602df740 1 -- 172.21.15.67:0/845503813 >> 172.21.15.67:0/845503813 conn(0x56260e48f460 msgr2=0x56260e842080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:34.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f59602df740 1 -- 172.21.15.67:0/845503813 shutdown_connections 2024-07-31T15:09:34.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f59602df740 1 -- 172.21.15.67:0/845503813 wait complete. 2024-07-31T15:09:34.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.933+0000 7f59602df740 20 client.19383 trim_cache size 0 max 16384 2024-07-31T15:09:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.947+0000 7f59602df740 1 Processor -- start 2024-07-31T15:09:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.947+0000 7f59602df740 1 -- start start 2024-07-31T15:09:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.947+0000 7f59602df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e4721b0 0x56260e85c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.947+0000 7f59602df740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e47a390 0x56260e85c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.947+0000 7f59602df740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e49b070 0x56260e867da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.947+0000 7f59602df740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f59541ae3e0 con 0x56260e47a390 2024-07-31T15:09:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.947+0000 7f59602df740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f59541aff20 con 0x56260e49b070 2024-07-31T15:09:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.947+0000 7f59602df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59541aeab0 con 0x56260e4721b0 2024-07-31T15:09:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.947+0000 7f5952ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e47a390 0x56260e85c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.947+0000 7f5952ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e47a390 0x56260e85c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49734/0 (socket says 172.21.15.67:49734) 2024-07-31T15:09:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.947+0000 7f5952ffd640 1 -- 172.21.15.67:0/1037107695 learned_addr learned my addr 172.21.15.67:0/1037107695 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.948+0000 7f5953fff640 1 --2- 172.21.15.67:0/1037107695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e49b070 0x56260e867da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.948+0000 7f59537fe640 1 --2- 172.21.15.67:0/1037107695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e4721b0 0x56260e85c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.948+0000 7f5952ffd640 1 -- 172.21.15.67:0/1037107695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e4721b0 msgr2=0x56260e85c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.948+0000 7f5952ffd640 1 --2- 172.21.15.67:0/1037107695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e4721b0 0x56260e85c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.948+0000 7f5952ffd640 1 -- 172.21.15.67:0/1037107695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e49b070 msgr2=0x56260e867da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.948+0000 7f5952ffd640 1 --2- 172.21.15.67:0/1037107695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e49b070 0x56260e867da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.948+0000 7f5952ffd640 1 -- 172.21.15.67:0/1037107695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260e843450 con 0x56260e47a390 2024-07-31T15:09:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.948+0000 7f59537fe640 1 --2- 172.21.15.67:0/1037107695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e4721b0 0x56260e85c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:09:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.948+0000 7f5952ffd640 1 --2- 172.21.15.67:0/1037107695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e47a390 0x56260e85c770 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f593c003670 tx=0x7f593c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e79b0d714c777c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:34.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.948+0000 7f59497fa640 1 -- 172.21.15.67:0/1037107695 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c005020 con 0x56260e47a390 2024-07-31T15:09:34.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.948+0000 7f59497fa640 1 -- 172.21.15.67:0/1037107695 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f593c0047a0 con 0x56260e47a390 2024-07-31T15:09:34.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.948+0000 7f59497fa640 1 -- 172.21.15.67:0/1037107695 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c01e590 con 0x56260e47a390 2024-07-31T15:09:34.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.949+0000 7f59602df740 1 -- 172.21.15.67:0/1037107695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e47a390 msgr2=0x56260e85c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:34.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.949+0000 7f59602df740 1 --2- 172.21.15.67:0/1037107695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e47a390 0x56260e85c770 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f593c003670 tx=0x7f593c013040 comp rx=0 tx=0).stop 2024-07-31T15:09:34.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.949+0000 7f59602df740 1 -- 172.21.15.67:0/1037107695 shutdown_connections 2024-07-31T15:09:34.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.949+0000 7f59602df740 1 --2- 172.21.15.67:0/1037107695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e49b070 0x56260e867da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.949+0000 7f59602df740 1 --2- 172.21.15.67:0/1037107695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e47a390 0x56260e85c770 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.949+0000 7f59602df740 1 --2- 172.21.15.67:0/1037107695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e4721b0 0x56260e85c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.949+0000 7f59602df740 1 -- 172.21.15.67:0/1037107695 >> 172.21.15.67:0/1037107695 conn(0x56260e48f460 msgr2=0x56260e84ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:34.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.949+0000 7f59602df740 1 -- 172.21.15.67:0/1037107695 shutdown_connections 2024-07-31T15:09:34.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.949+0000 7f59602df740 1 -- 172.21.15.67:0/1037107695 wait complete. 2024-07-31T15:09:34.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.950+0000 7f59602df740 1 -- start start 2024-07-31T15:09:34.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.950+0000 7f59602df740 1 Processor -- start 2024-07-31T15:09:34.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.950+0000 7f59602df740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e4721b0 0x56260e854900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.950+0000 7f59602df740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e47a390 0x56260e854e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.950+0000 7f59602df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e49b070 0x56260e8c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.950+0000 7f59602df740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f59541af850 con 0x56260e4721b0 2024-07-31T15:09:34.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.950+0000 7f59602df740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f59541af180 con 0x56260e47a390 2024-07-31T15:09:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.950+0000 7f59602df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59541ac3e0 con 0x56260e49b070 2024-07-31T15:09:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.950+0000 7f59602df740 10 client.0 _ll_register_callbacks cb 0x7f594c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.950+0000 7f59537fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e4721b0 0x56260e854900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.950+0000 7f59537fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e4721b0 0x56260e854900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49740/0 (socket says 172.21.15.67:49740) 2024-07-31T15:09:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.950+0000 7f59537fe640 1 -- 172.21.15.67:0/1593832291 learned_addr learned my addr 172.21.15.67:0/1593832291 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.950+0000 7f5952ffd640 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e47a390 0x56260e854e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f5953fff640 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e49b070 0x56260e8c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f59537fe640 1 -- 172.21.15.67:0/1593832291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e49b070 msgr2=0x56260e8c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f59537fe640 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e49b070 0x56260e8c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f59537fe640 1 -- 172.21.15.67:0/1593832291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e47a390 msgr2=0x56260e854e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f59537fe640 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e47a390 0x56260e854e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f59537fe640 1 -- 172.21.15.67:0/1593832291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260e8ca2b0 con 0x56260e4721b0 2024-07-31T15:09:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f5953fff640 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e49b070 0x56260e8c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:09:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f5952ffd640 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e47a390 0x56260e854e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f59537fe640 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e4721b0 0x56260e854900 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f594400aee0 tx=0x7f594400aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac003df3d805073 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f5932ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f5932ffd640 1 -- 172.21.15.67:0/1593832291 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944010700 con 0x56260e4721b0 2024-07-31T15:09:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f5932ffd640 1 -- 172.21.15.67:0/1593832291 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f59440108a0 con 0x56260e4721b0 2024-07-31T15:09:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f59602df740 1 -- 172.21.15.67:0/1593832291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f593c0047a0 con 0x56260e4721b0 2024-07-31T15:09:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f5932ffd640 1 -- 172.21.15.67:0/1593832291 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f594403c050 con 0x56260e4721b0 2024-07-31T15:09:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f59602df740 10 client.19389 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.951+0000 7f59602df740 1 -- 172.21.15.67:0/1593832291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x56260e8cb4f0 con 0x56260e4721b0 2024-07-31T15:09:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.952+0000 7f5932ffd640 1 -- 172.21.15.67:0/1593832291 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f593c0047a0 con 0x56260e4721b0 2024-07-31T15:09:34.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.952+0000 7f5932ffd640 1 -- 172.21.15.67:0/1593832291 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f5944018bd0 con 0x56260e4721b0 2024-07-31T15:09:34.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.952+0000 7f59602df740 10 client.19389 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:34.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.952+0000 7f59602df740 10 client.19389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:34.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.952+0000 7f59602df740 20 client.19389 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:34.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.952+0000 7f59602df740 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56260e42eda0 0x56260e8cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:34.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.952+0000 7f59602df740 4 client.19389 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:09:34.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.952+0000 7f59602df740 1 -- 172.21.15.67:0/1593832291 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56260e7b45a0 con 0x56260e42eda0 2024-07-31T15:09:34.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.952+0000 7f5952ffd640 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56260e42eda0 0x56260e8cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:34.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.953+0000 7f5932ffd640 10 client.19389 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:34.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.953+0000 7f5952ffd640 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56260e42eda0 0x56260e8cdb50 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724d87f3b6fce606 server_cookie=4fcd7a8eb71db125 in_seq=0 out_seq=0 2024-07-31T15:09:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f5932ffd640 1 -- 172.21.15.67:0/1593832291 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56260e7b45a0 con 0x56260e42eda0 2024-07-31T15:09:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f5932ffd640 10 client.19389 handle_command_reply: tid=0 2024-07-31T15:09:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f59602df740 1 client.19389 shutdown 2024-07-31T15:09:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f59602df740 1 -- 172.21.15.67:0/1593832291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e4721b0 msgr2=0x56260e854900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f59602df740 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e4721b0 0x56260e854900 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f594400aee0 tx=0x7f594400aac0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f59602df740 1 -- 172.21.15.67:0/1593832291 shutdown_connections 2024-07-31T15:09:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f59602df740 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56260e42eda0 0x56260e8cdb50 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f59602df740 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56260e49b070 0x56260e8c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f59602df740 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56260e47a390 0x56260e854e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f59602df740 1 --2- 172.21.15.67:0/1593832291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56260e4721b0 0x56260e854900 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f59602df740 1 -- 172.21.15.67:0/1593832291 >> 172.21.15.67:0/1593832291 conn(0x56260e48f460 msgr2=0x56260e8416c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f5932ffd640 0 client.19389 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f59602df740 1 -- 172.21.15.67:0/1593832291 shutdown_connections 2024-07-31T15:09:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f59602df740 1 -- 172.21.15.67:0/1593832291 wait complete. 2024-07-31T15:09:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.969+0000 7f59602df740 20 client.19389 trim_cache size 0 max 16384 2024-07-31T15:09:34.972 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:09:34.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.970+0000 7f5959d8e640 1 -- 172.21.15.67:0/2838483547 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f59400b3d70 msgr2=0x7f59400b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:34.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.970+0000 7f5959d8e640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f59400b3d70 0x7f59400b6280 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f593c009fd0 tx=0x7f593c004730 comp rx=0 tx=0).stop 2024-07-31T15:09:34.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.970+0000 7f5959d8e640 1 -- 172.21.15.67:0/2838483547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5954108720 msgr2=0x7f595419f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:34.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.970+0000 7f5959d8e640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5954108720 0x7f595419f6c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f594c0098d0 tx=0x7f594c0317b0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.970+0000 7f5959d8e640 1 -- 172.21.15.67:0/2838483547 shutdown_connections 2024-07-31T15:09:34.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.970+0000 7f5959d8e640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f59400b3d70 0x7f59400b6280 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.970+0000 7f5959d8e640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5954108720 0x7f595419f6c0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.970+0000 7f5959d8e640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5954107dd0 0x7f595419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.970+0000 7f5959d8e640 1 --2- 172.21.15.67:0/2838483547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5954102490 0x7f5954199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:34.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.970+0000 7f5959d8e640 1 -- 172.21.15.67:0/2838483547 >> 172.21.15.67:0/2838483547 conn(0x7f5954079160 msgr2=0x7f595406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:34.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.970+0000 7f5959d8e640 1 -- 172.21.15.67:0/2838483547 shutdown_connections 2024-07-31T15:09:34.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:34.970+0000 7f5959d8e640 1 -- 172.21.15.67:0/2838483547 wait complete. 2024-07-31T15:09:35.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:35 smithi067 ceph-mon[32750]: pgmap v989: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 267 KiB/s wr, 90 op/s 2024-07-31T15:09:35.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:35 smithi073 ceph-mon[36714]: pgmap v989: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 267 KiB/s wr, 90 op/s 2024-07-31T15:09:35.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:35 smithi077 ceph-mon[37327]: pgmap v989: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 267 KiB/s wr, 90 op/s 2024-07-31T15:09:36.793 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T15:09:36.793 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T15:09:37.077 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:09:37.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:37 smithi067 ceph-mon[32750]: pgmap v990: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 465 KiB/s wr, 98 op/s 2024-07-31T15:09:37.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:37 smithi073 ceph-mon[36714]: pgmap v990: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 465 KiB/s wr, 98 op/s 2024-07-31T15:09:37.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:37 smithi077 ceph-mon[37327]: pgmap v990: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 465 KiB/s wr, 98 op/s 2024-07-31T15:09:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.838+0000 7f8a426a6640 1 -- 172.21.15.67:0/4180807039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a3c107e60 msgr2=0x7f8a3c108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.838+0000 7f8a426a6640 1 --2- 172.21.15.67:0/4180807039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a3c107e60 0x7f8a3c108270 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009900 tx=0x7f8a2402a410 comp rx=0 tx=0).stop 2024-07-31T15:09:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.838+0000 7f8a426a6640 1 -- 172.21.15.67:0/4180807039 shutdown_connections 2024-07-31T15:09:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.838+0000 7f8a426a6640 1 --2- 172.21.15.67:0/4180807039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c1087b0 0x7f8a3c10efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.838+0000 7f8a426a6640 1 --2- 172.21.15.67:0/4180807039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a3c107e60 0x7f8a3c108270 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.838+0000 7f8a426a6640 1 --2- 172.21.15.67:0/4180807039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a3c102520 0x7f8a3c102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.838+0000 7f8a426a6640 1 -- 172.21.15.67:0/4180807039 >> 172.21.15.67:0/4180807039 conn(0x7f8a3c0791a0 msgr2=0x7f8a3c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.838+0000 7f8a426a6640 1 -- 172.21.15.67:0/4180807039 shutdown_connections 2024-07-31T15:09:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.838+0000 7f8a426a6640 1 -- 172.21.15.67:0/4180807039 wait complete. 2024-07-31T15:09:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a426a6640 1 Processor -- start 2024-07-31T15:09:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a426a6640 1 -- start start 2024-07-31T15:09:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a426a6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a3c102520 0x7f8a3c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a426a6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a3c107e60 0x7f8a3c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a426a6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c1087b0 0x7f8a3c199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a426a6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c112440 con 0x7f8a3c102520 2024-07-31T15:09:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a426a6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1122c0 con 0x7f8a3c107e60 2024-07-31T15:09:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a426a6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1125c0 con 0x7f8a3c1087b0 2024-07-31T15:09:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a3bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a3c102520 0x7f8a3c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a3bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a3c102520 0x7f8a3c198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49766/0 (socket says 172.21.15.67:49766) 2024-07-31T15:09:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a3bfff640 1 -- 172.21.15.67:0/1013702221 learned_addr learned my addr 172.21.15.67:0/1013702221 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a40c43640 1 --2- 172.21.15.67:0/1013702221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c1087b0 0x7f8a3c199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a33fff640 1 --2- 172.21.15.67:0/1013702221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a3c107e60 0x7f8a3c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a3bfff640 1 -- 172.21.15.67:0/1013702221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c1087b0 msgr2=0x7f8a3c199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a3bfff640 1 --2- 172.21.15.67:0/1013702221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c1087b0 0x7f8a3c199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a3bfff640 1 -- 172.21.15.67:0/1013702221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a3c107e60 msgr2=0x7f8a3c1990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a3bfff640 1 --2- 172.21.15.67:0/1013702221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a3c107e60 0x7f8a3c1990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.839+0000 7f8a3bfff640 1 -- 172.21.15.67:0/1013702221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c1a8230 con 0x7f8a3c102520 2024-07-31T15:09:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.840+0000 7f8a3bfff640 1 --2- 172.21.15.67:0/1013702221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a3c102520 0x7f8a3c198b80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00ea60 tx=0x7f8a2c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.840+0000 7f8a39ffb640 1 -- 172.21.15.67:0/1013702221 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c00cde0 con 0x7f8a3c102520 2024-07-31T15:09:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.840+0000 7f8a39ffb640 1 -- 172.21.15.67:0/1013702221 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a2c0035f0 con 0x7f8a3c102520 2024-07-31T15:09:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.840+0000 7f8a426a6640 1 -- 172.21.15.67:0/1013702221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c1a8520 con 0x7f8a3c102520 2024-07-31T15:09:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.840+0000 7f8a39ffb640 1 -- 172.21.15.67:0/1013702221 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c010690 con 0x7f8a3c102520 2024-07-31T15:09:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.840+0000 7f8a426a6640 1 -- 172.21.15.67:0/1013702221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c1a8a80 con 0x7f8a3c102520 2024-07-31T15:09:38.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.842+0000 7f8a48bf7740 1 Processor -- start 2024-07-31T15:09:38.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a48bf7740 1 -- start start 2024-07-31T15:09:38.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a48bf7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237971350 0x55d237d163b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a48bf7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 0x55d237d168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a39ffb640 1 -- 172.21.15.67:0/1013702221 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8a2c010830 con 0x7f8a3c102520 2024-07-31T15:09:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a48bf7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237950670 0x55d237d16e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a48bf7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d237bfe2f0 con 0x55d237971350 2024-07-31T15:09:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a48bf7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d237c8ca80 con 0x55d237948490 2024-07-31T15:09:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a48bf7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d237c7bf80 con 0x55d237950670 2024-07-31T15:09:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a40c43640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237971350 0x55d237d163b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a40c43640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237971350 0x55d237d163b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49770/0 (socket says 172.21.15.67:49770) 2024-07-31T15:09:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a40c43640 1 -- 172.21.15.67:0/310645034 learned_addr learned my addr 172.21.15.67:0/310645034 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a3bfff640 1 --2- 172.21.15.67:0/310645034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 0x55d237d168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a33fff640 1 --2- 172.21.15.67:0/310645034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237950670 0x55d237d16e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a3bfff640 1 -- 172.21.15.67:0/310645034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237950670 msgr2=0x55d237d16e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a3bfff640 1 --2- 172.21.15.67:0/310645034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237950670 0x55d237d16e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a3bfff640 1 -- 172.21.15.67:0/310645034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237971350 msgr2=0x55d237d163b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a3bfff640 1 --2- 172.21.15.67:0/310645034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237971350 0x55d237d163b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a3bfff640 1 -- 172.21.15.67:0/310645034 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2379dc930 con 0x55d237948490 2024-07-31T15:09:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.843+0000 7f8a40c43640 1 --2- 172.21.15.67:0/310645034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237971350 0x55d237d163b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.844+0000 7f8a3bfff640 1 --2- 172.21.15.67:0/310645034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 0x55d237d168f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0e1940 tx=0x7f8a2c0eaa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e240c9168dad999 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.844+0000 7f8a327fc640 1 -- 172.21.15.67:0/310645034 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0f7070 con 0x55d237948490 2024-07-31T15:09:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.844+0000 7f8a327fc640 1 -- 172.21.15.67:0/310645034 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a2c0e4dc0 con 0x55d237948490 2024-07-31T15:09:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.844+0000 7f8a327fc640 1 -- 172.21.15.67:0/310645034 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0f2a20 con 0x55d237948490 2024-07-31T15:09:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.844+0000 7f8a39ffb640 1 --2- 172.21.15.67:0/1013702221 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a280b3dc0 0x7f8a280b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.844+0000 7f8a40c43640 1 --2- 172.21.15.67:0/1013702221 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a280b3dc0 0x7f8a280b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.844+0000 7f8a39ffb640 1 -- 172.21.15.67:0/1013702221 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f8a2c0e4530 con 0x7f8a3c102520 2024-07-31T15:09:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.845+0000 7f8a48bf7740 1 -- 172.21.15.67:0/310645034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 msgr2=0x55d237d168f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.845+0000 7f8a40c43640 1 --2- 172.21.15.67:0/1013702221 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a280b3dc0 0x7f8a280b62d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8a34011890 tx=0x7f8a34009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.845+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/310645034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 0x55d237d168f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0e1940 tx=0x7f8a2c0eaa10 comp rx=0 tx=0).stop 2024-07-31T15:09:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.845+0000 7f8a48bf7740 1 -- 172.21.15.67:0/310645034 shutdown_connections 2024-07-31T15:09:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.845+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/310645034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237950670 0x55d237d16e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.845+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/310645034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 0x55d237d168f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.845+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/310645034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237971350 0x55d237d163b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.845+0000 7f8a48bf7740 1 -- 172.21.15.67:0/310645034 >> 172.21.15.67:0/310645034 conn(0x55d237965740 msgr2=0x55d237d142a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.845+0000 7f8a48bf7740 1 -- 172.21.15.67:0/310645034 shutdown_connections 2024-07-31T15:09:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.845+0000 7f8a48bf7740 1 -- 172.21.15.67:0/310645034 wait complete. 2024-07-31T15:09:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.845+0000 7f8a48bf7740 1 -- start start 2024-07-31T15:09:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.845+0000 7f8a48bf7740 1 Processor -- start 2024-07-31T15:09:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a48bf7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237948490 0x55d237d21c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a48bf7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237950670 0x55d237d22250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a48bf7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237971350 0x55d237d2dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a48bf7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d237c1e4f0 con 0x55d237948490 2024-07-31T15:09:38.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a48bf7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d237c7ffc0 con 0x55d237950670 2024-07-31T15:09:38.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a48bf7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d237adf890 con 0x55d237971350 2024-07-31T15:09:38.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a3bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237948490 0x55d237d21c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a33fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237950670 0x55d237d22250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a33fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237950670 0x55d237d22250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:35270/0 (socket says 172.21.15.67:35270) 2024-07-31T15:09:38.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a33fff640 1 -- 172.21.15.67:0/3060985701 learned_addr learned my addr 172.21.15.67:0/3060985701 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:38.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a48bf7740 10 client.0 _ll_register_callbacks cb 0x55d237c28500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:38.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a40c43640 1 --2- 172.21.15.67:0/3060985701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237971350 0x55d237d2dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a33fff640 1 -- 172.21.15.67:0/3060985701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237971350 msgr2=0x55d237d2dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:38.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a33fff640 1 --2- 172.21.15.67:0/3060985701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237971350 0x55d237d2dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a33fff640 1 -- 172.21.15.67:0/3060985701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237948490 msgr2=0x55d237d21c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:38.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a33fff640 1 --2- 172.21.15.67:0/3060985701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237948490 0x55d237d21c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.846+0000 7f8a33fff640 1 -- 172.21.15.67:0/3060985701 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d237c2dc50 con 0x55d237950670 2024-07-31T15:09:38.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a33fff640 1 --2- 172.21.15.67:0/3060985701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237950670 0x55d237d22250 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009900 tx=0x7f8a24032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49504975c6cdae43 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:38.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:09:38.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a1b7fe640 1 -- 172.21.15.67:0/3060985701 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24035040 con 0x55d237950670 2024-07-31T15:09:38.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a1b7fe640 1 -- 172.21.15.67:0/3060985701 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f8a24031760 con 0x55d237950670 2024-07-31T15:09:38.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3060985701 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a2c0e4dc0 con 0x55d237950670 2024-07-31T15:09:38.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a1b7fe640 1 -- 172.21.15.67:0/3060985701 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f8a2404f050 con 0x55d237950670 2024-07-31T15:09:38.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a48bf7740 10 client.28940 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:38.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3060985701 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55d237d30930 con 0x55d237950670 2024-07-31T15:09:38.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a1b7fe640 1 -- 172.21.15.67:0/3060985701 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a2c0e4dc0 con 0x55d237950670 2024-07-31T15:09:38.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a1b7fe640 1 -- 172.21.15.67:0/3060985701 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f8a24004750 con 0x55d237950670 2024-07-31T15:09:38.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a48bf7740 10 client.28940 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:38.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a48bf7740 10 client.28940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:38.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a48bf7740 20 client.28940 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:38.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/3060985701 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d23792fcd0 0x55d237d33eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a48bf7740 4 client.28940 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:09:38.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.847+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3060985701 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d237c8e9d0 con 0x55d23792fcd0 2024-07-31T15:09:38.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.848+0000 7f8a3bfff640 1 --2- 172.21.15.67:0/3060985701 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d23792fcd0 0x55d237d33eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.848+0000 7f8a1b7fe640 10 client.28940 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:38.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.848+0000 7f8a3bfff640 1 --2- 172.21.15.67:0/3060985701 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d23792fcd0 0x55d237d33eb0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1de8ca1a44b65b7e server_cookie=2b22310e76fd493c in_seq=0 out_seq=0 2024-07-31T15:09:38.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a1b7fe640 1 -- 172.21.15.67:0/3060985701 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d237c8e9d0 con 0x55d23792fcd0 2024-07-31T15:09:38.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a1b7fe640 10 client.28940 handle_command_reply: tid=0 2024-07-31T15:09:38.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a48bf7740 1 client.28940 shutdown 2024-07-31T15:09:38.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3060985701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237950670 msgr2=0x55d237d22250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:38.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/3060985701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237950670 0x55d237d22250 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009900 tx=0x7f8a24032e90 comp rx=0 tx=0).stop 2024-07-31T15:09:38.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3060985701 shutdown_connections 2024-07-31T15:09:38.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/3060985701 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d23792fcd0 0x55d237d33eb0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/3060985701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237971350 0x55d237d2dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/3060985701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237950670 0x55d237d22250 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a1b7fe640 0 client.28940 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:38.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/3060985701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237948490 0x55d237d21c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3060985701 >> 172.21.15.67:0/3060985701 conn(0x55d237965740 msgr2=0x55d237d19540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:38.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3060985701 shutdown_connections 2024-07-31T15:09:38.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3060985701 wait complete. 2024-07-31T15:09:38.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.875+0000 7f8a48bf7740 20 client.28940 trim_cache size 0 max 16384 2024-07-31T15:09:38.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.893+0000 7f8a48bf7740 1 Processor -- start 2024-07-31T15:09:38.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.893+0000 7f8a48bf7740 1 -- start start 2024-07-31T15:09:38.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.893+0000 7f8a48bf7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 0x55d237d22250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.893+0000 7f8a48bf7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237950670 0x55d237d13280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.893+0000 7f8a48bf7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237971350 0x55d237d238a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.893+0000 7f8a48bf7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1b2970 con 0x55d237971350 2024-07-31T15:09:38.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.893+0000 7f8a48bf7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1b44b0 con 0x55d237948490 2024-07-31T15:09:38.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.893+0000 7f8a48bf7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1b3040 con 0x55d237950670 2024-07-31T15:09:38.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.893+0000 7f8a40c43640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237971350 0x55d237d238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.893+0000 7f8a40c43640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237971350 0x55d237d238a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49788/0 (socket says 172.21.15.67:49788) 2024-07-31T15:09:38.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.893+0000 7f8a40c43640 1 -- 172.21.15.67:0/1178653445 learned_addr learned my addr 172.21.15.67:0/1178653445 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:38.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.894+0000 7f8a3bfff640 1 --2- 172.21.15.67:0/1178653445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 0x55d237d22250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.894+0000 7f8a33fff640 1 --2- 172.21.15.67:0/1178653445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237950670 0x55d237d13280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.894+0000 7f8a40c43640 1 -- 172.21.15.67:0/1178653445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237950670 msgr2=0x55d237d13280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:38.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.894+0000 7f8a40c43640 1 --2- 172.21.15.67:0/1178653445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237950670 0x55d237d13280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.894+0000 7f8a40c43640 1 -- 172.21.15.67:0/1178653445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 msgr2=0x55d237d22250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:38.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.894+0000 7f8a40c43640 1 --2- 172.21.15.67:0/1178653445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 0x55d237d22250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.894+0000 7f8a40c43640 1 -- 172.21.15.67:0/1178653445 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d237d186a0 con 0x55d237971350 2024-07-31T15:09:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.894+0000 7f8a40c43640 1 --2- 172.21.15.67:0/1178653445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237971350 0x55d237d238a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400fad0 tx=0x7f8a34016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedb922847466e4b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.894+0000 7f8a31ffb640 1 -- 172.21.15.67:0/1178653445 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34004b20 con 0x55d237971350 2024-07-31T15:09:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.894+0000 7f8a31ffb640 1 -- 172.21.15.67:0/1178653445 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a34004cc0 con 0x55d237971350 2024-07-31T15:09:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.894+0000 7f8a31ffb640 1 -- 172.21.15.67:0/1178653445 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34005240 con 0x55d237971350 2024-07-31T15:09:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.895+0000 7f8a48bf7740 1 -- 172.21.15.67:0/1178653445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237971350 msgr2=0x55d237d238a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.895+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/1178653445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237971350 0x55d237d238a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400fad0 tx=0x7f8a34016000 comp rx=0 tx=0).stop 2024-07-31T15:09:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.895+0000 7f8a48bf7740 1 -- 172.21.15.67:0/1178653445 shutdown_connections 2024-07-31T15:09:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.895+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/1178653445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237971350 0x55d237d238a0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.895+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/1178653445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237950670 0x55d237d13280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.895+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/1178653445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 0x55d237d22250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.895+0000 7f8a48bf7740 1 -- 172.21.15.67:0/1178653445 >> 172.21.15.67:0/1178653445 conn(0x55d237965740 msgr2=0x55d237d23190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.895+0000 7f8a48bf7740 1 -- 172.21.15.67:0/1178653445 shutdown_connections 2024-07-31T15:09:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.895+0000 7f8a48bf7740 1 -- 172.21.15.67:0/1178653445 wait complete. 2024-07-31T15:09:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.896+0000 7f8a48bf7740 1 -- start start 2024-07-31T15:09:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.896+0000 7f8a48bf7740 1 Processor -- start 2024-07-31T15:09:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.896+0000 7f8a48bf7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 0x55d237d25d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.896+0000 7f8a48bf7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237950670 0x55d237d26280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.896+0000 7f8a48bf7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237971350 0x55d237d267c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.896+0000 7f8a48bf7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1b3de0 con 0x55d237950670 2024-07-31T15:09:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.896+0000 7f8a48bf7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1b3710 con 0x55d237948490 2024-07-31T15:09:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.896+0000 7f8a48bf7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1b0970 con 0x55d237971350 2024-07-31T15:09:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.896+0000 7f8a48bf7740 10 client.0 _ll_register_callbacks cb 0x7f8a24005ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:38.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.896+0000 7f8a33fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237950670 0x55d237d26280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.896+0000 7f8a33fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237950670 0x55d237d26280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49796/0 (socket says 172.21.15.67:49796) 2024-07-31T15:09:38.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.896+0000 7f8a3bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 0x55d237d25d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.896+0000 7f8a33fff640 1 -- 172.21.15.67:0/3289684980 learned_addr learned my addr 172.21.15.67:0/3289684980 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:38.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.896+0000 7f8a40c43640 1 --2- 172.21.15.67:0/3289684980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237971350 0x55d237d267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a33fff640 1 -- 172.21.15.67:0/3289684980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237971350 msgr2=0x55d237d267c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:38.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a33fff640 1 --2- 172.21.15.67:0/3289684980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237971350 0x55d237d267c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a33fff640 1 -- 172.21.15.67:0/3289684980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 msgr2=0x55d237d25d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:38.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a33fff640 1 --2- 172.21.15.67:0/3289684980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 0x55d237d25d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a33fff640 1 -- 172.21.15.67:0/3289684980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d237d9b7a0 con 0x55d237950670 2024-07-31T15:09:38.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a40c43640 1 --2- 172.21.15.67:0/3289684980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237971350 0x55d237d267c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:09:38.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a3bfff640 1 --2- 172.21.15.67:0/3289684980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 0x55d237d25d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a33fff640 1 --2- 172.21.15.67:0/3289684980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237950670 0x55d237d26280 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f8a240096c0 tx=0x7f8a2402ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4023a57ec7feda74 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a1b7fe640 1 -- 172.21.15.67:0/3289684980 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24058060 con 0x55d237950670 2024-07-31T15:09:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3289684980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a34004cc0 con 0x55d237950670 2024-07-31T15:09:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a1b7fe640 1 -- 172.21.15.67:0/3289684980 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f8a24031560 con 0x55d237950670 2024-07-31T15:09:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a1b7fe640 1 -- 172.21.15.67:0/3289684980 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f8a24035050 con 0x55d237950670 2024-07-31T15:09:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a48bf7740 10 client.19407 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3289684980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55d237d9ca60 con 0x55d237950670 2024-07-31T15:09:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.897+0000 7f8a1b7fe640 1 -- 172.21.15.67:0/3289684980 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a34004cc0 con 0x55d237950670 2024-07-31T15:09:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.898+0000 7f8a1b7fe640 1 -- 172.21.15.67:0/3289684980 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f8a24031760 con 0x55d237950670 2024-07-31T15:09:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.898+0000 7f8a48bf7740 10 client.19407 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:38.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.898+0000 7f8a48bf7740 10 client.19407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:38.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.898+0000 7f8a48bf7740 20 client.19407 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:38.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.898+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/3289684980 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d237905080 0x55d237d9f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:38.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.898+0000 7f8a48bf7740 4 client.19407 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T15:09:38.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.898+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3289684980 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d237c8a470 con 0x55d237905080 2024-07-31T15:09:38.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.898+0000 7f8a3bfff640 1 --2- 172.21.15.67:0/3289684980 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d237905080 0x55d237d9f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:38.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.898+0000 7f8a3bfff640 1 --2- 172.21.15.67:0/3289684980 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d237905080 0x55d237d9f250 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa14d54cf3973b server_cookie=8b9e0a0c3ad56639 in_seq=0 out_seq=0 2024-07-31T15:09:38.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.898+0000 7f8a1b7fe640 10 client.19407 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:38.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a1b7fe640 1 -- 172.21.15.67:0/3289684980 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d237c8a470 con 0x55d237905080 2024-07-31T15:09:38.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a1b7fe640 10 client.19407 handle_command_reply: tid=0 2024-07-31T15:09:38.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a48bf7740 1 client.19407 shutdown 2024-07-31T15:09:38.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3289684980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237950670 msgr2=0x55d237d26280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:38.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/3289684980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237950670 0x55d237d26280 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f8a240096c0 tx=0x7f8a2402ab10 comp rx=0 tx=0).stop 2024-07-31T15:09:38.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3289684980 shutdown_connections 2024-07-31T15:09:38.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/3289684980 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d237905080 0x55d237d9f250 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/3289684980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d237971350 0x55d237d267c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/3289684980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d237950670 0x55d237d26280 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a48bf7740 1 --2- 172.21.15.67:0/3289684980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d237948490 0x55d237d25d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3289684980 >> 172.21.15.67:0/3289684980 conn(0x55d237965740 msgr2=0x55d237d13050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:38.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a1b7fe640 0 client.19407 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:38.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3289684980 shutdown_connections 2024-07-31T15:09:38.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a48bf7740 1 -- 172.21.15.67:0/3289684980 wait complete. 2024-07-31T15:09:38.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a48bf7740 20 client.19407 trim_cache size 0 max 16384 2024-07-31T15:09:38.925 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:09:38.925 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T15:09:38.925 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "10ad993e-6cce-4560-a896-fd6e65de949f", 2024-07-31T15:09:38.925 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T15:09:38.925 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:09:38.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a426a6640 1 -- 172.21.15.67:0/1013702221 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a280b3dc0 msgr2=0x7f8a280b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:38.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a426a6640 1 --2- 172.21.15.67:0/1013702221 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a280b3dc0 0x7f8a280b62d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8a34011890 tx=0x7f8a34009500 comp rx=0 tx=0).stop 2024-07-31T15:09:38.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a426a6640 1 -- 172.21.15.67:0/1013702221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a3c102520 msgr2=0x7f8a3c198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:38.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a426a6640 1 --2- 172.21.15.67:0/1013702221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a3c102520 0x7f8a3c198b80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00ea60 tx=0x7f8a2c00ef20 comp rx=0 tx=0).stop 2024-07-31T15:09:38.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a426a6640 1 -- 172.21.15.67:0/1013702221 shutdown_connections 2024-07-31T15:09:38.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a426a6640 1 --2- 172.21.15.67:0/1013702221 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8a280b3dc0 0x7f8a280b62d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a426a6640 1 --2- 172.21.15.67:0/1013702221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c1087b0 0x7f8a3c199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a426a6640 1 --2- 172.21.15.67:0/1013702221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a3c107e60 0x7f8a3c1990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.922+0000 7f8a426a6640 1 --2- 172.21.15.67:0/1013702221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a3c102520 0x7f8a3c198b80 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:38.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.923+0000 7f8a426a6640 1 -- 172.21.15.67:0/1013702221 >> 172.21.15.67:0/1013702221 conn(0x7f8a3c0791a0 msgr2=0x7f8a3c105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:38.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.923+0000 7f8a426a6640 1 -- 172.21.15.67:0/1013702221 shutdown_connections 2024-07-31T15:09:38.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:38.923+0000 7f8a426a6640 1 -- 172.21.15.67:0/1013702221 wait complete. 2024-07-31T15:09:39.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:39 smithi067 ceph-mon[32750]: pgmap v991: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 652 KiB/s wr, 94 op/s 2024-07-31T15:09:39.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:39 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T15:09:39.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:39.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:39.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:39 smithi073 ceph-mon[36714]: pgmap v991: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 652 KiB/s wr, 94 op/s 2024-07-31T15:09:39.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:39 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T15:09:39.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:39.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:39.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:39 smithi077 ceph-mon[37327]: pgmap v991: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 652 KiB/s wr, 94 op/s 2024-07-31T15:09:39.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:39 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T15:09:39.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:39.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.140 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:09:40.142 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:39.735+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000162a [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch/ auth{1=1,2=1,3=1,4=1} v21244 ap=3 RANDEPHEMERALPIN f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22) n(v53 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939) old_inodes=7 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:432201 nref=3)) caps={15489=pAsLsXsFs/-@56} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7c26ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000162a:[<0x10000000202/arch v378>,<0x10000000201/linux-6.5.11 v96>,<0x10000000006/t v68>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v57>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v60>,<0x10000000001/sv_1 v84>,<0x10000000000/qa v70>,<0x1/volumes v70>]//[]","memoryvalue":"(2)0x1000000162a:[<0x10000000202/arch v21244>,<0x10000000201/linux-6.5.11 v2434>,<0x10000000006/t v1492>,<0x10000000005/tmp v1420>,<0x100000001fb/client.0 v1396>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1358>,<0x10000000001/sv_1 v1362>,<0x10000000000/qa v1356>,<0x1/volumes v1312>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:54:49.652308+0000 23=2+21)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939)","memory_value.dirstat":"f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22)","memory_value.rstat":"n(v53 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:40.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:39.791+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c276 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802/ auth{3=1} v10568 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:29.002671+0000 14=14+0) n(v3 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:433371 nref=3)) caps={15489=pAsLsXs/-@11} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8fd23180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c276:[<0x1000000c25e/802 v68>,<0x10000000202/net v7765>,<0x10000000201/linux-6.5.11 v689>,<0x10000000006/t v498>,<0x10000000005/tmp v466>,<0x100000001fb/client.0 v452>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v438>,<0x10000000001/sv_1 v453>,<0x10000000000/qa v441>,<0x1/volumes v433>]//[]","memoryvalue":"(2)0x1000000c276:[<0x1000000c25e/802 v10568>,<0x10000000202/net v21414>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:49.293317+0000 10=10+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:29.002671+0000 14=14+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:40.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:39.832+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d9b4 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb/ auth{3=1} v3876 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7) n(v5 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8) old_inodes=3 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:434190 nref=3)) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce92cb2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v2020>,<0x10000000202/sound v9936>,<0x10000000201/linux-6.5.11 v830>,<0x10000000006/t v586>,<0x10000000005/tmp v553>,<0x100000001fb/client.0 v537>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v519>,<0x10000000001/sv_1 v525>,<0x10000000000/qa v522>,<0x1/volumes v503>]//[]","memoryvalue":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v3876>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:46.317139+0000 53=46+7)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7)","memory_value.rstat":"n(v5 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:40.455 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:09:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:09:40.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:09:40.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:09:40.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:40.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f4265ffb640 1 -- 172.21.15.67:0/2289939752 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559d3ee448b0 con 0x559d3eac2570 2024-07-31T15:09:40.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f4265ffb640 10 client.19299 handle_command_reply: tid=0 2024-07-31T15:09:40.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f428ade9740 1 client.19299 shutdown 2024-07-31T15:09:40.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f428ade9740 1 -- 172.21.15.67:0/2289939752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb05980 msgr2=0x559d3eee5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:40.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f428ade9740 1 --2- 172.21.15.67:0/2289939752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb05980 0x559d3eee5ae0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f427002a9c0 tx=0x7f4270005f90 comp rx=0 tx=0).stop 2024-07-31T15:09:40.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f428ade9740 1 -- 172.21.15.67:0/2289939752 shutdown_connections 2024-07-31T15:09:40.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f428ade9740 1 --2- 172.21.15.67:0/2289939752 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559d3eac2570 0x559d3ef7ede0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:40.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f428ade9740 1 --2- 172.21.15.67:0/2289939752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d3eb2e840 0x559d3ef58ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:40.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f428ade9740 1 --2- 172.21.15.67:0/2289939752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d3eb0db60 0x559d3eee6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:40.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f4265ffb640 0 client.19299 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:40.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f428ade9740 1 --2- 172.21.15.67:0/2289939752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559d3eb05980 0x559d3eee5ae0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:40.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f428ade9740 1 -- 172.21.15.67:0/2289939752 >> 172.21.15.67:0/2289939752 conn(0x559d3eb22990 msgr2=0x559d3eed61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:40.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f428ade9740 1 -- 172.21.15.67:0/2289939752 shutdown_connections 2024-07-31T15:09:40.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f428ade9740 1 -- 172.21.15.67:0/2289939752 wait complete. 2024-07-31T15:09:40.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.555+0000 7f428ade9740 20 client.19299 trim_cache size 0 max 16384 2024-07-31T15:09:40.559 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:09:40.559 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-07-31T15:09:40.559 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:09:40.559 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 50, 2024-07-31T15:09:40.560 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:09:40.560 INFO:teuthology.orchestra.run.smithi067.stdout: "7e631bf4": { 2024-07-31T15:09:40.560 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 50, 2024-07-31T15:09:40.560 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:09:40.560 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:09:40.560 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:09:40.560 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:09:40.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:09:40.560 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:09:40.560 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 138.1, 2024-07-31T15:09:40.560 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:09:40.560 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:09:40.560 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:09:40.561 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:09:40.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:09:40.561 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:09:40.561 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:09:40.561 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:09:40.561 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:09:40.561 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:09:40.561 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:09:40.561 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:09:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.556+0000 7f4284898640 1 -- 172.21.15.67:0/1729460590 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42680b3f50 msgr2=0x7f42680b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.556+0000 7f4284898640 1 --2- 172.21.15.67:0/1729460590 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42680b3f50 0x7f42680b6460 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4270005eb0 tx=0x7f4270000bc0 comp rx=0 tx=0).stop 2024-07-31T15:09:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.556+0000 7f4284898640 1 -- 172.21.15.67:0/1729460590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4280111780 msgr2=0x7f4280114190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.556+0000 7f4284898640 1 --2- 172.21.15.67:0/1729460590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4280111780 0x7f4280114190 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f427400e9f0 tx=0x7f427400eeb0 comp rx=0 tx=0).stop 2024-07-31T15:09:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.556+0000 7f4284898640 1 -- 172.21.15.67:0/1729460590 shutdown_connections 2024-07-31T15:09:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.556+0000 7f4284898640 1 --2- 172.21.15.67:0/1729460590 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f42680b3f50 0x7f42680b6460 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.556+0000 7f4284898640 1 --2- 172.21.15.67:0/1729460590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f428011c5d0 0x7f4280114790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.556+0000 7f4284898640 1 --2- 172.21.15.67:0/1729460590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4280111780 0x7f4280114190 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.556+0000 7f4284898640 1 --2- 172.21.15.67:0/1729460590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f428010a330 0x7f428011aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.556+0000 7f4284898640 1 -- 172.21.15.67:0/1729460590 >> 172.21.15.67:0/1729460590 conn(0x7f428006e440 msgr2=0x7f4280071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.556+0000 7f4284898640 1 -- 172.21.15.67:0/1729460590 shutdown_connections 2024-07-31T15:09:40.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:40.556+0000 7f4284898640 1 -- 172.21.15.67:0/1729460590 wait complete. 2024-07-31T15:09:40.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:09:40.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:09:40.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:09:40.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.605 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:40.605 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:09:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:09:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:09:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:40.869 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:40.599+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c061 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math/ auth{1=1} v1164 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:10.213499+0000 31=31+0) n(v2 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1) old_inodes=2 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:432885 nref=3)) caps={15489=pAsLsXs/-@10} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8e92fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c061:[<0x1000000bf5f/math v634>,<0x10000000202/lib v5688>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000c061:[<0x1000000bf5f/math v1164>,<0x10000000202/lib v21040>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:10.988109+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:10.213499+0000 31=31+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:41.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:40.868+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000ce48 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity/ auth{1=1,2=1} v1695 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3) n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4) old_inodes=3 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:435037 nref=3)) caps={15489=pAsLsXs/-@10} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8aaac580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000ce48:[<0x1000000ce07/integrity v345>,<0x10000000202/security v8766>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000ce48:[<0x1000000ce07/integrity v1695>,<0x10000000202/security v15110>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:59.470106+0000 10=7+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","memory_value.dirstat":"f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3)","memory_value.rstat":"n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:41.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:40.967+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009eb8 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs/ auth{4=1} v4394 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:25.901734+0000 12=12+0) n(v3 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:435521 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce809d7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009eb8:[<0x10000009d36/debugfs v3846>,<0x10000000202/fs v3830>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009eb8:[<0x10000009d36/debugfs v4394>,<0x10000000202/fs v21227>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:39.315767+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:25.901734+0000 12=12+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:41.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:40 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:40.992+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000a492 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs/ auth{1=1} v5396 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:30.806634+0000 8=8+0) n(v3 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1) old_inodes=2 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:435594 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8aca9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000a492:[<0x10000009d36/tracefs v3850>,<0x10000000202/fs v3830>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x1000000a492:[<0x10000009d36/tracefs v5396>,<0x10000000202/fs v21227>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:41.554720+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:30.806634+0000 8=8+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:41.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:41.034+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002351 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char/ auth{2=1} v18176 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:34.793553+0000 56=49+7) n(v10 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10) old_inodes=4 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:445983 nref=3)) caps={15489=pAsLsXs/-@11} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7ef94100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000002351:[<0x10000001d8f/char v2490>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x10000002351:[<0x10000001d8f/char v18176>,<0x10000000202/drivers v21416>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:02.131047+0000 40=33+7)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:34.793553+0000 56=49+7)","memory_value.rstat":"n(v10 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:41.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: pgmap v992: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 715 KiB/s wr, 98 op/s 2024-07-31T15:09:41.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:41.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:41 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.514 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:41.165+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000243a [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk/ auth{2=1,4=1} fragtree_t(*^3) v18679 ap=3 RANDEPHEMERALPIN f(v3 m2024-07-31T15:07:07.059118+0000 137=88+49) n(v9 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50) old_inodes=2 (inest sync->mix g=2,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:446065 nref=3)) caps={15489=pAsLsXs/-@8} | dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7fa16680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000243a:[<0x10000001d8f/clk v2484>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x1000000243a:[<0x10000001d8f/clk v18679>,<0x10000000202/drivers v21416>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 133=84+49)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50)","memory_value.dirstat":"f(v3 m2024-07-31T15:07:07.059118+0000 137=88+49)","memory_value.rstat":"n(v9 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:41.515 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:41.213+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009980 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video/ auth{1=1,4=1} v18105 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:16.340374+0000 28=24+4) n(v11 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28) old_inodes=2 (inest sync->mix g=1,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:446106 nref=3)) caps={15489=pAsLsXs/-@6} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce87325b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009980:[<0x10000001d8f/video v17889>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009980:[<0x10000001d8f/video v18105>,<0x10000000202/drivers v21416>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:35.535846+0000 16=12+4)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:16.340374+0000 28=24+4)","memory_value.rstat":"n(v11 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:41.515 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.516 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.516 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.516 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: pgmap v992: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 715 KiB/s wr, 98 op/s 2024-07-31T15:09:41.516 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.516 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.516 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.516 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.516 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.516 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.516 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.517 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.518 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.518 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.518 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.518 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.518 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.518 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.518 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.519 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.520 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.520 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.520 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.520 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.520 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:41.520 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.520 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.520 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.524 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7e631bf4', quiesce took 29.6 seconds. Will release after: -11.5 2024-07-31T15:09:41.524 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:09:41.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: pgmap v992: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 715 KiB/s wr, 98 op/s 2024-07-31T15:09:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:41.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:41 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:09:41.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:41 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:41.513+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000be32 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module/ auth{2=1} v2029 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0) n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:440974 nref=3)) caps={15489=pAsLsXs/-@13} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce826b7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000be32:[<0x1000000bd0a/module v1134>,<0x10000000202/kernel v5648>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000be32:[<0x1000000bd0a/module v2029>,<0x10000000202/kernel v21206>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:07.168188+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:42.393 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:42.119+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d187 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi/ auth{1=1} v3040 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0) n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1) old_inodes=1 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:443805 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7c6c1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d187:[<0x1000000d116/ctxfi v1858>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d187:[<0x1000000d116/ctxfi v3040>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.452342+0000 30=30+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:42.393 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:42.122+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d119 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97/ auth{2=1} v3208 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0) n(v3 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1) old_inodes=3 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:443808 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce9047e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d119:[<0x1000000d116/ac97 v22>,<0x1000000cf1b/pci v463>,<0x10000000202/sound v8728>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000d119:[<0x1000000d116/ac97 v3208>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:00.491085+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:42.393 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:42.128+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d1cc [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1/ auth{2=1} v3042 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0) n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:443815 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8f176c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v1862>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v3042>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.549340+0000 22=22+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:42.393 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:42.151+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d180 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio/ auth{1=1} v3038 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0) n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1) old_inodes=1 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:443854 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8ebd2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v1860>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v3038>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.414343+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:42.394 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:42.367+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000cde2 [...10,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders/ auth{4=1} v112 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0) n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:443586 nref=3)) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8dc52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v29>,<0x1000000cbec/selinux v646>,<0x10000000202/scripts v8820>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v112>,<0x1000000cbec/selinux v749>,<0x10000000202/scripts v11639>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:58.999116+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:42.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:42.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:42.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:42.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:42.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:42.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:42.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:42.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:42.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:42.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:42.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:42.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:42.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:42.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:09:42.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:09:42.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:09:42.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:09:42.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:42 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:09:42.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:42.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:42.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:42.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:42.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:42.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:42.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:42.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:42.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:09:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:09:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:09:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:09:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:42 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:09:42.473 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:09:42.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:09:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:09:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:09:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:09:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:42 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:09:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: pgmap v993: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 709 KiB/s wr, 212 op/s 2024-07-31T15:09:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:43.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:09:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:43.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.332+0000 7fbac94a2640 1 -- 172.21.15.67:0/3498531267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac4102490 msgr2=0x7fbac41028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:43.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.332+0000 7fbac94a2640 1 --2- 172.21.15.67:0/3498531267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac4102490 0x7fbac41028a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009900 tx=0x7fbab402a410 comp rx=0 tx=0).stop 2024-07-31T15:09:43.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.333+0000 7fbac94a2640 1 -- 172.21.15.67:0/3498531267 shutdown_connections 2024-07-31T15:09:43.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.333+0000 7fbac94a2640 1 --2- 172.21.15.67:0/3498531267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbac4108720 0x7fbac410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.333+0000 7fbac94a2640 1 --2- 172.21.15.67:0/3498531267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbac4107dd0 0x7fbac41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.333+0000 7fbac94a2640 1 --2- 172.21.15.67:0/3498531267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac4102490 0x7fbac41028a0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.333+0000 7fbac94a2640 1 -- 172.21.15.67:0/3498531267 >> 172.21.15.67:0/3498531267 conn(0x7fbac4079160 msgr2=0x7fbac4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:43.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.333+0000 7fbac94a2640 1 -- 172.21.15.67:0/3498531267 shutdown_connections 2024-07-31T15:09:43.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.333+0000 7fbac94a2640 1 -- 172.21.15.67:0/3498531267 wait complete. 2024-07-31T15:09:43.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.334+0000 7fbac94a2640 1 Processor -- start 2024-07-31T15:09:43.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.334+0000 7fbac94a2640 1 -- start start 2024-07-31T15:09:43.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.334+0000 7fbac94a2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac4102490 0x7fbac419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.334+0000 7fbac94a2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbac4107dd0 0x7fbac419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.334+0000 7fbac94a2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbac4108720 0x7fbac419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.334+0000 7fbac94a2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbac4112380 con 0x7fbac4102490 2024-07-31T15:09:43.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.334+0000 7fbac94a2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbac4112200 con 0x7fbac4108720 2024-07-31T15:09:43.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.334+0000 7fbac94a2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbac4112500 con 0x7fbac4107dd0 2024-07-31T15:09:43.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.334+0000 7fbac2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac4102490 0x7fbac419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.334+0000 7fbac2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac4102490 0x7fbac419acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39668/0 (socket says 172.21.15.67:39668) 2024-07-31T15:09:43.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.334+0000 7fbac27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbac4107dd0 0x7fbac419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.334+0000 7fbac2ffd640 1 -- 172.21.15.67:0/1454492584 learned_addr learned my addr 172.21.15.67:0/1454492584 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:43.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.334+0000 7fbac37fe640 1 --2- 172.21.15.67:0/1454492584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbac4108720 0x7fbac419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.335+0000 7fbac27fc640 1 -- 172.21.15.67:0/1454492584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbac4108720 msgr2=0x7fbac419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:43.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.335+0000 7fbac27fc640 1 --2- 172.21.15.67:0/1454492584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbac4108720 0x7fbac419b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.335+0000 7fbac27fc640 1 -- 172.21.15.67:0/1454492584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac4102490 msgr2=0x7fbac419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:43.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.335+0000 7fbac27fc640 1 --2- 172.21.15.67:0/1454492584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac4102490 0x7fbac419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.335+0000 7fbac27fc640 1 -- 172.21.15.67:0/1454492584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac41a5f60 con 0x7fbac4107dd0 2024-07-31T15:09:43.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.335+0000 7fbac2ffd640 1 --2- 172.21.15.67:0/1454492584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac4102490 0x7fbac419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:43.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.335+0000 7fbac27fc640 1 --2- 172.21.15.67:0/1454492584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbac4107dd0 0x7fbac419b210 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbab8005fd0 tx=0x7fbab800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:43.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.335+0000 7fbaabfff640 1 -- 172.21.15.67:0/1454492584 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab8009c00 con 0x7fbac4107dd0 2024-07-31T15:09:43.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.335+0000 7fbac94a2640 1 -- 172.21.15.67:0/1454492584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac41a6250 con 0x7fbac4107dd0 2024-07-31T15:09:43.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.335+0000 7fbaabfff640 1 -- 172.21.15.67:0/1454492584 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbab8009da0 con 0x7fbac4107dd0 2024-07-31T15:09:43.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.335+0000 7fbac94a2640 1 -- 172.21.15.67:0/1454492584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac41a67e0 con 0x7fbac4107dd0 2024-07-31T15:09:43.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.335+0000 7fbaabfff640 1 -- 172.21.15.67:0/1454492584 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab8003e40 con 0x7fbac4107dd0 2024-07-31T15:09:43.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.338+0000 7fbacf9f3740 1 Processor -- start 2024-07-31T15:09:43.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.338+0000 7fbacf9f3740 1 -- start start 2024-07-31T15:09:43.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.338+0000 7fbaabfff640 1 -- 172.21.15.67:0/1454492584 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbab8010460 con 0x7fbac4107dd0 2024-07-31T15:09:43.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbacf9f3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028da0070 0x55d029145550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbacf9f3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d029145a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbacf9f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028d7f390 0x55d029145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbacf9f3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d0290bbb90 con 0x55d028da0070 2024-07-31T15:09:43.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbacf9f3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d0290ab090 con 0x55d028d771b0 2024-07-31T15:09:43.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbacf9f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d0290af0d0 con 0x55d028d7f390 2024-07-31T15:09:43.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbac37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028da0070 0x55d029145550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbac37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028da0070 0x55d029145550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39678/0 (socket says 172.21.15.67:39678) 2024-07-31T15:09:43.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbac37fe640 1 -- 172.21.15.67:0/2177050096 learned_addr learned my addr 172.21.15.67:0/2177050096 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:43.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbac37fe640 1 -- 172.21.15.67:0/2177050096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028d7f390 msgr2=0x55d029145fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:09:43.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbac37fe640 1 --2- 172.21.15.67:0/2177050096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028d7f390 0x55d029145fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbac37fe640 1 -- 172.21.15.67:0/2177050096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 msgr2=0x55d029145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:43.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbac2ffd640 1 --2- 172.21.15.67:0/2177050096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d029145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbac37fe640 1 --2- 172.21.15.67:0/2177050096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d029145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.339+0000 7fbac37fe640 1 -- 172.21.15.67:0/2177050096 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d029051b70 con 0x55d028da0070 2024-07-31T15:09:43.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.340+0000 7fbac2ffd640 1 --2- 172.21.15.67:0/2177050096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d029145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:09:43.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.340+0000 7fbaabfff640 1 --2- 172.21.15.67:0/1454492584 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbaac0b3d70 0x7fbaac0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.340+0000 7fbac2ffd640 1 --2- 172.21.15.67:0/1454492584 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbaac0b3d70 0x7fbaac0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.340+0000 7fbaabfff640 1 -- 172.21.15.67:0/1454492584 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7fbab80e3e90 con 0x7fbac4107dd0 2024-07-31T15:09:43.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.340+0000 7fbac37fe640 1 --2- 172.21.15.67:0/2177050096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028da0070 0x55d029145550 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009f50 tx=0x7fbabc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758141849be57e93 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:43.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.340+0000 7fbaa97fa640 1 -- 172.21.15.67:0/2177050096 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc004d20 con 0x55d028da0070 2024-07-31T15:09:43.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.340+0000 7fbaa97fa640 1 -- 172.21.15.67:0/2177050096 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbabc004030 con 0x55d028da0070 2024-07-31T15:09:43.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.340+0000 7fbaa97fa640 1 -- 172.21.15.67:0/2177050096 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc005260 con 0x55d028da0070 2024-07-31T15:09:43.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.341+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2177050096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028da0070 msgr2=0x55d029145550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:43.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.341+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/2177050096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028da0070 0x55d029145550 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009f50 tx=0x7fbabc0118d0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.341+0000 7fbac2ffd640 1 --2- 172.21.15.67:0/1454492584 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbaac0b3d70 0x7fbaac0b6280 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009230 tx=0x7fbab40058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:43.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.342+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2177050096 shutdown_connections 2024-07-31T15:09:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.342+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/2177050096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028d7f390 0x55d029145fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.342+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/2177050096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d029145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.342+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/2177050096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028da0070 0x55d029145550 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.342+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2177050096 >> 172.21.15.67:0/2177050096 conn(0x55d028d94460 msgr2=0x55d029143440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.342+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2177050096 shutdown_connections 2024-07-31T15:09:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.342+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2177050096 wait complete. 2024-07-31T15:09:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.342+0000 7fbacf9f3740 1 -- start start 2024-07-31T15:09:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.342+0000 7fbacf9f3740 1 Processor -- start 2024-07-31T15:09:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbacf9f3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d029155600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbacf9f3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028d7f390 0x55d029155be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbacf9f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028da0070 0x55d0291613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbacf9f3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d028f0e700 con 0x55d028d7f390 2024-07-31T15:09:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbacf9f3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d02904d9c0 con 0x55d028d771b0 2024-07-31T15:09:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbacf9f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d029158090 con 0x55d028da0070 2024-07-31T15:09:43.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbacf9f3740 10 client.0 _ll_register_callbacks cb 0x55d0290684d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:43.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbac2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d029155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbac37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028da0070 0x55d0291613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbac2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d029155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59912/0 (socket says 172.21.15.67:59912) 2024-07-31T15:09:43.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbac2ffd640 1 -- 172.21.15.67:0/3675627385 learned_addr learned my addr 172.21.15.67:0/3675627385 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:43.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbac37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028da0070 0x55d0291613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:40796/0 (socket says 172.21.15.67:40796) 2024-07-31T15:09:43.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbac27fc640 1 --2- 172.21.15.67:0/3675627385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028d7f390 0x55d029155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbac2ffd640 1 -- 172.21.15.67:0/3675627385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028da0070 msgr2=0x55d0291613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:43.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbac2ffd640 1 --2- 172.21.15.67:0/3675627385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028da0070 0x55d0291613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbac2ffd640 1 -- 172.21.15.67:0/3675627385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028d7f390 msgr2=0x55d029155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:43.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbac2ffd640 1 --2- 172.21.15.67:0/3675627385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028d7f390 0x55d029155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbac2ffd640 1 -- 172.21.15.67:0/3675627385 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d028e0b650 con 0x55d028d771b0 2024-07-31T15:09:43.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbac37fe640 1 --2- 172.21.15.67:0/3675627385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028da0070 0x55d0291613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:43.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbac27fc640 1 --2- 172.21.15.67:0/3675627385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028d7f390 0x55d029155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:43.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbac2ffd640 1 --2- 172.21.15.67:0/3675627385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d029155600 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009a20 tx=0x7fbab402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd2307a4b90cc5f5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:43.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.343+0000 7fbaa6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:09:43.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.344+0000 7fbaa6ff5640 1 -- 172.21.15.67:0/3675627385 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab404c030 con 0x55d028d771b0 2024-07-31T15:09:43.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.344+0000 7fbaa6ff5640 1 -- 172.21.15.67:0/3675627385 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7fbab40035f0 con 0x55d028d771b0 2024-07-31T15:09:43.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.344+0000 7fbacf9f3740 1 -- 172.21.15.67:0/3675627385 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbabc004030 con 0x55d028d771b0 2024-07-31T15:09:43.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.344+0000 7fbaa6ff5640 1 -- 172.21.15.67:0/3675627385 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fbab4059050 con 0x55d028d771b0 2024-07-31T15:09:43.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.344+0000 7fbacf9f3740 10 client.28955 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:43.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.344+0000 7fbacf9f3740 1 -- 172.21.15.67:0/3675627385 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55d029164150 con 0x55d028d771b0 2024-07-31T15:09:43.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.344+0000 7fbaa6ff5640 1 -- 172.21.15.67:0/3675627385 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbabc004030 con 0x55d028d771b0 2024-07-31T15:09:43.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.344+0000 7fbaa6ff5640 1 -- 172.21.15.67:0/3675627385 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fbab40487f0 con 0x55d028d771b0 2024-07-31T15:09:43.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.344+0000 7fbacf9f3740 10 client.28955 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:43.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.344+0000 7fbacf9f3740 10 client.28955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:43.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.344+0000 7fbacf9f3740 20 client.28955 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:43.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.344+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/3675627385 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d028d5e9f0 0x55d0291676f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.344+0000 7fbacf9f3740 4 client.28955 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:09:43.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.344+0000 7fbacf9f3740 1 -- 172.21.15.67:0/3675627385 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0290bdae0 con 0x55d028d5e9f0 2024-07-31T15:09:43.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.345+0000 7fbac27fc640 1 --2- 172.21.15.67:0/3675627385 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d028d5e9f0 0x55d0291676f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.345+0000 7fbaa6ff5640 10 client.28955 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:43.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.345+0000 7fbac27fc640 1 --2- 172.21.15.67:0/3675627385 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d028d5e9f0 0x55d0291676f0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86e633a7cafbfad5 server_cookie=9c5dd30d36215657 in_seq=0 out_seq=0 2024-07-31T15:09:43.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbaa6ff5640 1 -- 172.21.15.67:0/3675627385 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0290bdae0 con 0x55d028d5e9f0 2024-07-31T15:09:43.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbaa6ff5640 10 client.28955 handle_command_reply: tid=0 2024-07-31T15:09:43.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbacf9f3740 1 client.28955 shutdown 2024-07-31T15:09:43.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbacf9f3740 1 -- 172.21.15.67:0/3675627385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 msgr2=0x55d029155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:43.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/3675627385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d029155600 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009a20 tx=0x7fbab402a410 comp rx=0 tx=0).stop 2024-07-31T15:09:43.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbacf9f3740 1 -- 172.21.15.67:0/3675627385 shutdown_connections 2024-07-31T15:09:43.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/3675627385 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d028d5e9f0 0x55d0291676f0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/3675627385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028da0070 0x55d0291613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/3675627385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028d7f390 0x55d029155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbaa6ff5640 0 client.28955 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:43.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/3675627385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d029155600 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbacf9f3740 1 -- 172.21.15.67:0/3675627385 >> 172.21.15.67:0/3675627385 conn(0x55d028d94460 msgr2=0x55d029146f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:43.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbacf9f3740 1 -- 172.21.15.67:0/3675627385 shutdown_connections 2024-07-31T15:09:43.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbacf9f3740 1 -- 172.21.15.67:0/3675627385 wait complete. 2024-07-31T15:09:43.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.347+0000 7fbacf9f3740 20 client.28955 trim_cache size 0 max 16384 2024-07-31T15:09:43.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.368+0000 7fbacf9f3740 1 Processor -- start 2024-07-31T15:09:43.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.368+0000 7fbacf9f3740 1 -- start start 2024-07-31T15:09:43.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.368+0000 7fbacf9f3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d0291505b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.368+0000 7fbacf9f3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028d7f390 0x55d0291611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.368+0000 7fbacf9f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028da0070 0x55d029146c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.368+0000 7fbacf9f3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbac41ae5b0 con 0x55d028d7f390 2024-07-31T15:09:43.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.368+0000 7fbacf9f3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbac41b00f0 con 0x55d028d771b0 2024-07-31T15:09:43.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.368+0000 7fbacf9f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbac41aec80 con 0x55d028da0070 2024-07-31T15:09:43.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.369+0000 7fbac2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d0291505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.369+0000 7fbac27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028d7f390 0x55d0291611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.369+0000 7fbac37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028da0070 0x55d029146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.369+0000 7fbac2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d0291505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59914/0 (socket says 172.21.15.67:59914) 2024-07-31T15:09:43.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.369+0000 7fbac2ffd640 1 -- 172.21.15.67:0/2529161531 learned_addr learned my addr 172.21.15.67:0/2529161531 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:43.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.369+0000 7fbac37fe640 1 -- 172.21.15.67:0/2529161531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 msgr2=0x55d0291505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:43.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.369+0000 7fbac37fe640 1 --2- 172.21.15.67:0/2529161531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d0291505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.369+0000 7fbac37fe640 1 -- 172.21.15.67:0/2529161531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028d7f390 msgr2=0x55d0291611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:43.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.369+0000 7fbac37fe640 1 --2- 172.21.15.67:0/2529161531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028d7f390 0x55d0291611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.369+0000 7fbac37fe640 1 -- 172.21.15.67:0/2529161531 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d029150c90 con 0x55d028da0070 2024-07-31T15:09:43.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.369+0000 7fbac2ffd640 1 --2- 172.21.15.67:0/2529161531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d0291505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:43.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.369+0000 7fbac27fc640 1 --2- 172.21.15.67:0/2529161531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028d7f390 0x55d0291611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:43.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.369+0000 7fbac37fe640 1 --2- 172.21.15.67:0/2529161531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028da0070 0x55d029146c90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbabc011dd0 tx=0x7fbabc009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=295149890a210a48 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:43.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.370+0000 7fbaa8ff9640 1 -- 172.21.15.67:0/2529161531 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc004bc0 con 0x55d028da0070 2024-07-31T15:09:43.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.370+0000 7fbaa8ff9640 1 -- 172.21.15.67:0/2529161531 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbabc004d60 con 0x55d028da0070 2024-07-31T15:09:43.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.370+0000 7fbaa8ff9640 1 -- 172.21.15.67:0/2529161531 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc005300 con 0x55d028da0070 2024-07-31T15:09:43.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.370+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2529161531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028da0070 msgr2=0x55d029146c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:43.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.370+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/2529161531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028da0070 0x55d029146c90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbabc011dd0 tx=0x7fbabc009c60 comp rx=0 tx=0).stop 2024-07-31T15:09:43.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2529161531 shutdown_connections 2024-07-31T15:09:43.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/2529161531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028da0070 0x55d029146c90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/2529161531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028d7f390 0x55d0291611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/2529161531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d771b0 0x55d0291505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2529161531 >> 172.21.15.67:0/2529161531 conn(0x55d028d94460 msgr2=0x55d029146580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:43.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2529161531 shutdown_connections 2024-07-31T15:09:43.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2529161531 wait complete. 2024-07-31T15:09:43.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 -- start start 2024-07-31T15:09:43.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 Processor -- start 2024-07-31T15:09:43.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028d771b0 0x55d02915d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d7f390 0x55d02915dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028da0070 0x55d029158d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbac41afa20 con 0x55d028da0070 2024-07-31T15:09:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbac41af350 con 0x55d028d7f390 2024-07-31T15:09:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.371+0000 7fbacf9f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbac41ac5b0 con 0x55d028d771b0 2024-07-31T15:09:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbacf9f3740 10 client.0 _ll_register_callbacks cb 0x7fbab4040050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbac2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028d771b0 0x55d02915d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbac37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028da0070 0x55d029158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbac2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028d771b0 0x55d02915d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:40814/0 (socket says 172.21.15.67:40814) 2024-07-31T15:09:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbac2ffd640 1 -- 172.21.15.67:0/2535015724 learned_addr learned my addr 172.21.15.67:0/2535015724 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbac27fc640 1 --2- 172.21.15.67:0/2535015724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d7f390 0x55d02915dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbac37fe640 1 -- 172.21.15.67:0/2535015724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028d771b0 msgr2=0x55d02915d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbac37fe640 1 --2- 172.21.15.67:0/2535015724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028d771b0 0x55d02915d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbac37fe640 1 -- 172.21.15.67:0/2535015724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d7f390 msgr2=0x55d02915dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbac37fe640 1 --2- 172.21.15.67:0/2535015724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d7f390 0x55d02915dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbac37fe640 1 -- 172.21.15.67:0/2535015724 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0291594a0 con 0x55d028da0070 2024-07-31T15:09:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbac37fe640 1 --2- 172.21.15.67:0/2535015724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028da0070 0x55d029158d10 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fbabc011dd0 tx=0x7fbabc004110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da68f64dd8648d12 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbaa6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbaa6ff5640 1 -- 172.21.15.67:0/2535015724 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc004bc0 con 0x55d028da0070 2024-07-31T15:09:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbaa6ff5640 1 -- 172.21.15.67:0/2535015724 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7fbabc005200 con 0x55d028da0070 2024-07-31T15:09:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.372+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2535015724 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbabc004d60 con 0x55d028da0070 2024-07-31T15:09:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.373+0000 7fbaa6ff5640 1 -- 172.21.15.67:0/2535015724 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fbabc005400 con 0x55d028da0070 2024-07-31T15:09:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.373+0000 7fbacf9f3740 10 client.19425 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.373+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2535015724 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55d0291cfa70 con 0x55d028da0070 2024-07-31T15:09:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.373+0000 7fbaa6ff5640 1 -- 172.21.15.67:0/2535015724 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbabc004d60 con 0x55d028da0070 2024-07-31T15:09:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.373+0000 7fbaa6ff5640 1 -- 172.21.15.67:0/2535015724 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fbabc018070 con 0x55d028da0070 2024-07-31T15:09:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.373+0000 7fbacf9f3740 10 client.19425 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.373+0000 7fbacf9f3740 10 client.19425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.373+0000 7fbacf9f3740 20 client.19425 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.373+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/2535015724 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d028d33da0 0x55d0291d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.373+0000 7fbacf9f3740 4 client.19425 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:09:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.373+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2535015724 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d0290b9580 con 0x55d028d33da0 2024-07-31T15:09:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.373+0000 7fbac2ffd640 1 --2- 172.21.15.67:0/2535015724 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d028d33da0 0x55d0291d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.374+0000 7fbac2ffd640 1 --2- 172.21.15.67:0/2535015724 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d028d33da0 0x55d0291d1ef0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0d0e53d61b32cc7 server_cookie=ca515012177e3e9 in_seq=0 out_seq=0 2024-07-31T15:09:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.374+0000 7fbaa6ff5640 10 client.19425 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.375+0000 7fbaa6ff5640 1 -- 172.21.15.67:0/2535015724 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x55d0290b9580 con 0x55d028d33da0 2024-07-31T15:09:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.375+0000 7fbaa6ff5640 10 client.19425 handle_command_reply: tid=0 2024-07-31T15:09:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.375+0000 7fbacf9f3740 1 client.19425 shutdown 2024-07-31T15:09:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.375+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2535015724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028da0070 msgr2=0x55d029158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.375+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/2535015724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028da0070 0x55d029158d10 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fbabc011dd0 tx=0x7fbabc004110 comp rx=0 tx=0).stop 2024-07-31T15:09:43.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.375+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2535015724 shutdown_connections 2024-07-31T15:09:43.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.375+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/2535015724 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d028d33da0 0x55d0291d1ef0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.375+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/2535015724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d028da0070 0x55d029158d10 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.375+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/2535015724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d028d7f390 0x55d02915dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.375+0000 7fbacf9f3740 1 --2- 172.21.15.67:0/2535015724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d028d771b0 0x55d02915d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.375+0000 7fbaa6ff5640 0 client.19425 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:43.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.375+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2535015724 >> 172.21.15.67:0/2535015724 conn(0x55d028d94460 msgr2=0x55d029146580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:43.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.375+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2535015724 shutdown_connections 2024-07-31T15:09:43.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.376+0000 7fbacf9f3740 1 -- 172.21.15.67:0/2535015724 wait complete. 2024-07-31T15:09:43.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.376+0000 7fbacf9f3740 20 client.19425 trim_cache size 0 max 16384 2024-07-31T15:09:43.383 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:09:43.383 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (14747 inodes in the stack)", 2024-07-31T15:09:43.383 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T15:09:43.383 INFO:teuthology.orchestra.run.smithi067.stdout: "10ad993e-6cce-4560-a896-fd6e65de949f": { 2024-07-31T15:09:43.383 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T15:09:43.383 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "10ad993e-6cce-4560-a896-fd6e65de949f", 2024-07-31T15:09:43.383 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T15:09:43.383 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:09:43.383 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:09:43.383 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:09:43.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.376+0000 7fbac94a2640 1 -- 172.21.15.67:0/1454492584 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbaac0b3d70 msgr2=0x7fbaac0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:43.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.376+0000 7fbac94a2640 1 --2- 172.21.15.67:0/1454492584 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbaac0b3d70 0x7fbaac0b6280 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009230 tx=0x7fbab40058d0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.376+0000 7fbac94a2640 1 -- 172.21.15.67:0/1454492584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbac4107dd0 msgr2=0x7fbac419b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.376+0000 7fbac94a2640 1 --2- 172.21.15.67:0/1454492584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbac4107dd0 0x7fbac419b210 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbab8005fd0 tx=0x7fbab800dde0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.376+0000 7fbac94a2640 1 -- 172.21.15.67:0/1454492584 shutdown_connections 2024-07-31T15:09:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.376+0000 7fbac94a2640 1 --2- 172.21.15.67:0/1454492584 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbaac0b3d70 0x7fbaac0b6280 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.376+0000 7fbac94a2640 1 --2- 172.21.15.67:0/1454492584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbac4108720 0x7fbac419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.377+0000 7fbac94a2640 1 --2- 172.21.15.67:0/1454492584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbac4107dd0 0x7fbac419b210 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.377+0000 7fbac94a2640 1 --2- 172.21.15.67:0/1454492584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac4102490 0x7fbac419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.377+0000 7fbac94a2640 1 -- 172.21.15.67:0/1454492584 >> 172.21.15.67:0/1454492584 conn(0x7fbac4079160 msgr2=0x7fbac4100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.377+0000 7fbac94a2640 1 -- 172.21.15.67:0/1454492584 shutdown_connections 2024-07-31T15:09:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:43.377+0000 7fbac94a2640 1 -- 172.21.15.67:0/1454492584 wait complete. 2024-07-31T15:09:43.624 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:43.313+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000021f0 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader/ auth{2=1} v893 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:04.418950+0000 19=18+1) n(v5 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:476740 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@8} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7dc21b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000021f0:[<0x100000021db/firmware_loader v414>,<0x10000001d8f/base v2486>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x100000021f0:[<0x100000021db/firmware_loader v893>,<0x10000001d8f/base v18747>,<0x10000000202/drivers v21416>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:01.469061+0000 13=12+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:04.418950+0000 19=18+1)","memory_value.rstat":"n(v5 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:43.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: pgmap v993: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 709 KiB/s wr, 212 op/s 2024-07-31T15:09:43.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:43.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:43.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:43.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:43.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:09:43.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:43.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.626 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.626 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:43.626 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.626 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:43.626 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:43.626 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.626 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:43.626 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.626 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:43.626 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.626 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.626 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:43.626 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.627 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:43.627 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: pgmap v993: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 709 KiB/s wr, 212 op/s 2024-07-31T15:09:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:09:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:43.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:43.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:43.622+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000008a08 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip/ auth{3=1} v2681 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:09.647065+0000 7=7+0) n(v2 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:454691 nref=3)) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8c1aa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000008a08:[<0x1000000894f/microchip v613>,<0x10000001d8f/soc v16560>,<0x10000000202/drivers v3182>,<0x10000000201/linux-6.5.11 v445>,<0x10000000006/t v329>,<0x10000000005/tmp v299>,<0x100000001fb/client.0 v296>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v274>,<0x10000000001/sv_1 v299>,<0x10000000000/qa v280>,<0x1/volumes v280>]//[]","memoryvalue":"(2)0x10000008a08:[<0x1000000894f/microchip v2681>,<0x10000001d8f/soc v17222>,<0x10000000202/drivers v21416>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:51.041772+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:09.647065+0000 7=7+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:43.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:43.741+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001fc7 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica/ auth{2=1} fragtree_t(*^3) v1950 ap=3 RANDEPHEMERALPIN f(v10 m2024-07-31T15:07:39.153464+0000 516=516+0) n(v16 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1) old_inodes=4 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:477545 nref=3)) caps={15489=pAsLsXs/-@28} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7f0ba680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000001fc7:[<0x10000001fad/acpica v1734>,<0x10000001d8f/acpi v2434>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x10000001fc7:[<0x10000001fad/acpica v1950>,<0x10000001d8f/acpi v18672>,<0x10000000202/drivers v21416>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:07:37.321501+0000 494=494+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1)","memory_value.dirstat":"f(v10 m2024-07-31T15:07:39.153464+0000 516=516+0)","memory_value.rstat":"n(v16 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:43.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:43.801+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009462 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc/ auth{3=1} v1424 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:28.223687+0000 22=22+0) n(v4 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:477693 nref=3)) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce85c96b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009462:[<0x1000000945c/hvc v352>,<0x10000001d8f/tty v18356>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009462:[<0x1000000945c/hvc v1424>,<0x10000001d8f/tty v18901>,<0x10000000202/drivers v21416>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:32.992899+0000 16=16+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:28.223687+0000 22=22+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:43.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:43 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:43.863+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005d80 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader/ auth{4=1} v1324 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:10.777820+0000 21=21+0) n(v3 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1) old_inodes=1 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:477822 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7f8f5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000005d80:[<0x10000005d5d/cardreader v104>,<0x10000001d8f/misc v13245>,<0x10000000202/drivers v2910>,<0x10000000201/linux-6.5.11 v379>,<0x10000000006/t v275>,<0x10000000005/tmp v244>,<0x100000001fb/client.0 v243>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v221>,<0x10000000001/sv_1 v249>,<0x10000000000/qa v233>,<0x1/volumes v234>]//[]","memoryvalue":"(2)0x10000005d80:[<0x10000005d5d/cardreader v1324>,<0x10000001d8f/misc v18516>,<0x10000000202/drivers v21416>,<0x10000000201/linux-6.5.11 v2437>,<0x10000000006/t v1494>,<0x10000000005/tmp v1423>,<0x100000001fb/client.0 v1398>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1361>,<0x10000000001/sv_1 v1364>,<0x10000000000/qa v1358>,<0x1/volumes v1316>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:21.430390+0000 17=17+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:10.777820+0000 21=21+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:44.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:44 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:44.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:44 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:09:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:44.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:44.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:44 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.435 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.435 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.435 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.435 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.435 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.435 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.435 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: pgmap v994: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 674 KiB/s wr, 584 op/s 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.440 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.440 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.440 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.440 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.440 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.440 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:45.440 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:45 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.638 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.639 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.639 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.639 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.639 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.639 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.639 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.639 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.639 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.639 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.639 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.639 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.639 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: pgmap v994: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 674 KiB/s wr, 584 op/s 2024-07-31T15:09:45.639 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.640 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.640 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.640 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.640 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.640 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.640 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.640 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.640 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.640 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.640 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.640 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.640 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.641 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:45.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: pgmap v994: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 674 KiB/s wr, 584 op/s 2024-07-31T15:09:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:45.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:45 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.891+0000 7fa87247e640 1 -- 172.21.15.67:0/3442335918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa86c101a50 msgr2=0x7fa86c101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.891+0000 7fa87247e640 1 --2- 172.21.15.67:0/3442335918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa86c101a50 0x7fa86c101e60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009900 tx=0x7fa85c02a410 comp rx=0 tx=0).stop 2024-07-31T15:09:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.891+0000 7fa87247e640 1 -- 172.21.15.67:0/3442335918 shutdown_connections 2024-07-31T15:09:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.891+0000 7fa87247e640 1 --2- 172.21.15.67:0/3442335918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa86c1098e0 0x7fa86c1100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.891+0000 7fa87247e640 1 --2- 172.21.15.67:0/3442335918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa86c108f90 0x7fa86c1093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.891+0000 7fa87247e640 1 --2- 172.21.15.67:0/3442335918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa86c101a50 0x7fa86c101e60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.891+0000 7fa87247e640 1 -- 172.21.15.67:0/3442335918 >> 172.21.15.67:0/3442335918 conn(0x7fa86c079880 msgr2=0x7fa86c079c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.891+0000 7fa87247e640 1 -- 172.21.15.67:0/3442335918 shutdown_connections 2024-07-31T15:09:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.891+0000 7fa87247e640 1 -- 172.21.15.67:0/3442335918 wait complete. 2024-07-31T15:09:45.894 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:45 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:45.636+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000948f [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250/ auth{4=1} v716 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:35.607536+0000 75=75+0) n(v5 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:479009 nref=3)) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce836e4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000948f:[<0x1000000948d/8250 v533>,<0x1000000945c/serial v657>,<0x10000001d8f/tty v18356>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x1000000948f:[<0x1000000948d/8250 v716>,<0x1000000945c/serial v1492>,<0x10000001d8f/tty v18901>,<0x10000000202/drivers v21424>,<0x10000000201/linux-6.5.11 v2439>,<0x10000000006/t v1496>,<0x10000000005/tmp v1425>,<0x100000001fb/client.0 v1400>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1363>,<0x10000000001/sv_1 v1366>,<0x10000000000/qa v1360>,<0x1/volumes v1318>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:33.063897+0000 47=47+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:35.607536+0000 75=75+0)","memory_value.rstat":"n(v5 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.892+0000 7fa87247e640 1 Processor -- start 2024-07-31T15:09:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.892+0000 7fa87247e640 1 -- start start 2024-07-31T15:09:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.892+0000 7fa87247e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa86c101a50 0x7fa86c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.892+0000 7fa87247e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa86c108f90 0x7fa86c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.892+0000 7fa87247e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa86c1098e0 0x7fa86c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.892+0000 7fa87247e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c113560 con 0x7fa86c101a50 2024-07-31T15:09:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.892+0000 7fa87247e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1133e0 con 0x7fa86c108f90 2024-07-31T15:09:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.892+0000 7fa87247e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1136e0 con 0x7fa86c1098e0 2024-07-31T15:09:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.893+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa86c101a50 0x7fa86c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.893+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa86c108f90 0x7fa86c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.893+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa86c101a50 0x7fa86c19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39708/0 (socket says 172.21.15.67:39708) 2024-07-31T15:09:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.893+0000 7fa86bfff640 1 -- 172.21.15.67:0/192942069 learned_addr learned my addr 172.21.15.67:0/192942069 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.893+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa86c108f90 0x7fa86c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59936/0 (socket says 172.21.15.67:59936) 2024-07-31T15:09:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.893+0000 7fa870a1b640 1 --2- 172.21.15.67:0/192942069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa86c1098e0 0x7fa86c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.893+0000 7fa86bfff640 1 -- 172.21.15.67:0/192942069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa86c1098e0 msgr2=0x7fa86c19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.893+0000 7fa86bfff640 1 --2- 172.21.15.67:0/192942069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa86c1098e0 0x7fa86c19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.893+0000 7fa86bfff640 1 -- 172.21.15.67:0/192942069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa86c108f90 msgr2=0x7fa86c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.893+0000 7fa86bfff640 1 --2- 172.21.15.67:0/192942069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa86c108f90 0x7fa86c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.893+0000 7fa86bfff640 1 -- 172.21.15.67:0/192942069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86c19ff50 con 0x7fa86c101a50 2024-07-31T15:09:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.893+0000 7fa86b7fe640 1 --2- 172.21.15.67:0/192942069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa86c108f90 0x7fa86c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.893+0000 7fa86bfff640 1 --2- 172.21.15.67:0/192942069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa86c101a50 0x7fa86c19a020 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa85c032fd0 tx=0x7fa85c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.893+0000 7fa8697fa640 1 -- 172.21.15.67:0/192942069 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c03f030 con 0x7fa86c101a50 2024-07-31T15:09:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.894+0000 7fa8697fa640 1 -- 172.21.15.67:0/192942069 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa85c004490 con 0x7fa86c101a50 2024-07-31T15:09:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.894+0000 7fa87247e640 1 -- 172.21.15.67:0/192942069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86c1a01e0 con 0x7fa86c101a50 2024-07-31T15:09:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.894+0000 7fa8697fa640 1 -- 172.21.15.67:0/192942069 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c035070 con 0x7fa86c101a50 2024-07-31T15:09:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.894+0000 7fa87247e640 1 -- 172.21.15.67:0/192942069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa86c1a69d0 con 0x7fa86c101a50 2024-07-31T15:09:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.897+0000 7fa879375740 1 Processor -- start 2024-07-31T15:09:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.897+0000 7fa8697fa640 1 -- 172.21.15.67:0/192942069 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fa85c042030 con 0x7fa86c101a50 2024-07-31T15:09:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.897+0000 7fa879375740 1 -- start start 2024-07-31T15:09:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.897+0000 7fa879375740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880dc1b0 0x56078847ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.897+0000 7fa879375740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880b32f0 0x56078847f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.897+0000 7fa879375740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x56078847f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.897+0000 7fa879375740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5607883f43b0 con 0x5607880bb4d0 2024-07-31T15:09:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.897+0000 7fa879375740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5607883e38b0 con 0x5607880dc1b0 2024-07-31T15:09:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.897+0000 7fa879375740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5607883e78f0 con 0x5607880b32f0 2024-07-31T15:09:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.898+0000 7fa870a1b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880dc1b0 0x56078847ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.898+0000 7fa870a1b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880dc1b0 0x56078847ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59942/0 (socket says 172.21.15.67:59942) 2024-07-31T15:09:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.898+0000 7fa870a1b640 1 -- 172.21.15.67:0/3925401791 learned_addr learned my addr 172.21.15.67:0/3925401791 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.898+0000 7fa86b7fe640 1 --2- 172.21.15.67:0/3925401791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x56078847f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.898+0000 7fa86bfff640 1 --2- 172.21.15.67:0/3925401791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880b32f0 0x56078847f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.898+0000 7fa870a1b640 1 -- 172.21.15.67:0/3925401791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880b32f0 msgr2=0x56078847f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.898+0000 7fa870a1b640 1 --2- 172.21.15.67:0/3925401791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880b32f0 0x56078847f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.898+0000 7fa870a1b640 1 -- 172.21.15.67:0/3925401791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 msgr2=0x56078847f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.898+0000 7fa870a1b640 1 --2- 172.21.15.67:0/3925401791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x56078847f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.898+0000 7fa870a1b640 1 -- 172.21.15.67:0/3925401791 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56078838a9a0 con 0x5607880dc1b0 2024-07-31T15:09:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.898+0000 7fa86b7fe640 1 --2- 172.21.15.67:0/3925401791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x56078847f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.898+0000 7fa86bfff640 1 --2- 172.21.15.67:0/3925401791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880b32f0 0x56078847f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:09:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.898+0000 7fa8697fa640 1 --2- 172.21.15.67:0/192942069 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa8540b3d40 0x7fa8540b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.899+0000 7fa870a1b640 1 --2- 172.21.15.67:0/3925401791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880dc1b0 0x56078847ece0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa864009fd0 tx=0x7fa8640118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f48de0004aa6cea2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.899+0000 7fa86bfff640 1 --2- 172.21.15.67:0/192942069 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa8540b3d40 0x7fa8540b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.899+0000 7fa8527fc640 1 -- 172.21.15.67:0/3925401791 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864004d20 con 0x5607880dc1b0 2024-07-31T15:09:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.899+0000 7fa8527fc640 1 -- 172.21.15.67:0/3925401791 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa864004050 con 0x5607880dc1b0 2024-07-31T15:09:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.899+0000 7fa8527fc640 1 -- 172.21.15.67:0/3925401791 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864005280 con 0x5607880dc1b0 2024-07-31T15:09:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.899+0000 7fa8697fa640 1 -- 172.21.15.67:0/192942069 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7fa85c104690 con 0x7fa86c101a50 2024-07-31T15:09:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.900+0000 7fa879375740 1 -- 172.21.15.67:0/3925401791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880dc1b0 msgr2=0x56078847ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.900+0000 7fa86bfff640 1 --2- 172.21.15.67:0/192942069 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa8540b3d40 0x7fa8540b6250 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x56078847d960 tx=0x7fa85c10a260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.900+0000 7fa879375740 1 --2- 172.21.15.67:0/3925401791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880dc1b0 0x56078847ece0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa864009fd0 tx=0x7fa8640118d0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.900+0000 7fa879375740 1 -- 172.21.15.67:0/3925401791 shutdown_connections 2024-07-31T15:09:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.900+0000 7fa879375740 1 --2- 172.21.15.67:0/3925401791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x56078847f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.900+0000 7fa879375740 1 --2- 172.21.15.67:0/3925401791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880b32f0 0x56078847f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.900+0000 7fa879375740 1 --2- 172.21.15.67:0/3925401791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880dc1b0 0x56078847ece0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.900+0000 7fa879375740 1 -- 172.21.15.67:0/3925401791 >> 172.21.15.67:0/3925401791 conn(0x5607880d0300 msgr2=0x56078847cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.900+0000 7fa879375740 1 -- 172.21.15.67:0/3925401791 shutdown_connections 2024-07-31T15:09:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.900+0000 7fa879375740 1 -- 172.21.15.67:0/3925401791 wait complete. 2024-07-31T15:09:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.901+0000 7fa879375740 1 -- start start 2024-07-31T15:09:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.901+0000 7fa879375740 1 Processor -- start 2024-07-31T15:09:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa879375740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880b32f0 0x56078848e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa879375740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x56078848eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa879375740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 0x56078849ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa879375740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56078831dab0 con 0x5607880bb4d0 2024-07-31T15:09:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa879375740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56078836bcf0 con 0x5607880b32f0 2024-07-31T15:09:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa879375740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560788491840 con 0x5607880dc1b0 2024-07-31T15:09:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa879375740 10 client.0 _ll_register_callbacks cb 0x5607883dd9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa870a1b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880b32f0 0x56078848e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x56078848eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x56078848eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39734/0 (socket says 172.21.15.67:39734) 2024-07-31T15:09:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa870a1b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880b32f0 0x56078848e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59944/0 (socket says 172.21.15.67:59944) 2024-07-31T15:09:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa86b7fe640 1 -- 172.21.15.67:0/1139550213 learned_addr learned my addr 172.21.15.67:0/1139550213 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa870a1b640 1 --2- 172.21.15.67:0/1139550213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 0x56078849ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa870a1b640 1 -- 172.21.15.67:0/1139550213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 msgr2=0x56078849ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa870a1b640 1 --2- 172.21.15.67:0/1139550213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 0x56078849ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.902+0000 7fa870a1b640 1 -- 172.21.15.67:0/1139550213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 msgr2=0x56078848eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.903+0000 7fa870a1b640 1 --2- 172.21.15.67:0/1139550213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x56078848eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.903+0000 7fa870a1b640 1 -- 172.21.15.67:0/1139550213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560788147790 con 0x5607880b32f0 2024-07-31T15:09:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.903+0000 7fa86b7fe640 1 --2- 172.21.15.67:0/1139550213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x56078848eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.903+0000 7fa870a1b640 1 --2- 172.21.15.67:0/1139550213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880b32f0 0x56078848e930 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa864003fd0 tx=0x7fa864005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd14adb3fce0c54b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.903+0000 7fa84b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:09:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.903+0000 7fa84b7fe640 1 -- 172.21.15.67:0/1139550213 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86402c020 con 0x5607880b32f0 2024-07-31T15:09:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.903+0000 7fa84b7fe640 1 -- 172.21.15.67:0/1139550213 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7fa86400f4e0 con 0x5607880b32f0 2024-07-31T15:09:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.903+0000 7fa879375740 1 -- 172.21.15.67:0/1139550213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa864004050 con 0x5607880b32f0 2024-07-31T15:09:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.903+0000 7fa84b7fe640 1 -- 172.21.15.67:0/1139550213 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fa864037050 con 0x5607880b32f0 2024-07-31T15:09:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.903+0000 7fa879375740 10 client.28973 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.903+0000 7fa879375740 1 -- 172.21.15.67:0/1139550213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5607881c0620 con 0x5607880b32f0 2024-07-31T15:09:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.904+0000 7fa84b7fe640 1 -- 172.21.15.67:0/1139550213 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa864004050 con 0x5607880b32f0 2024-07-31T15:09:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.904+0000 7fa84b7fe640 1 -- 172.21.15.67:0/1139550213 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fa8640285e0 con 0x5607880b32f0 2024-07-31T15:09:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.904+0000 7fa879375740 10 client.28973 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.904+0000 7fa879375740 10 client.28973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.904+0000 7fa879375740 20 client.28973 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.904+0000 7fa879375740 1 --2- 172.21.15.67:0/1139550213 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56078809ad80 0x5607884a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.904+0000 7fa879375740 4 client.28973 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:09:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.904+0000 7fa879375740 1 -- 172.21.15.67:0/1139550213 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607883f6300 con 0x56078809ad80 2024-07-31T15:09:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.904+0000 7fa86b7fe640 1 --2- 172.21.15.67:0/1139550213 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56078809ad80 0x5607884a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.905+0000 7fa84b7fe640 10 client.28973 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.905+0000 7fa86b7fe640 1 --2- 172.21.15.67:0/1139550213 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56078809ad80 0x5607884a0a10 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aabcc8b5f55424e server_cookie=1fded9bdbfd78d44 in_seq=0 out_seq=0 2024-07-31T15:09:45.913 INFO:tasks.cephfs.filesystem:scrub status for tag:10ad993e-6cce-4560-a896-fd6e65de949f - {'path': '/', 'tag': '10ad993e-6cce-4560-a896-fd6e65de949f', 'options': 'recursive,force'} 2024-07-31T15:09:45.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.906+0000 7fa84b7fe640 1 -- 172.21.15.67:0/1139550213 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607883f6300 con 0x56078809ad80 2024-07-31T15:09:45.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.906+0000 7fa84b7fe640 10 client.28973 handle_command_reply: tid=0 2024-07-31T15:09:45.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.906+0000 7fa879375740 1 client.28973 shutdown 2024-07-31T15:09:45.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.906+0000 7fa879375740 1 -- 172.21.15.67:0/1139550213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880b32f0 msgr2=0x56078848e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:45.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.906+0000 7fa879375740 1 --2- 172.21.15.67:0/1139550213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880b32f0 0x56078848e930 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa864003fd0 tx=0x7fa864005bd0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.906+0000 7fa879375740 1 -- 172.21.15.67:0/1139550213 shutdown_connections 2024-07-31T15:09:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.906+0000 7fa879375740 1 --2- 172.21.15.67:0/1139550213 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56078809ad80 0x5607884a0a10 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.906+0000 7fa879375740 1 --2- 172.21.15.67:0/1139550213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 0x56078849ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.906+0000 7fa879375740 1 --2- 172.21.15.67:0/1139550213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x56078848eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.906+0000 7fa879375740 1 --2- 172.21.15.67:0/1139550213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880b32f0 0x56078848e930 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.906+0000 7fa879375740 1 -- 172.21.15.67:0/1139550213 >> 172.21.15.67:0/1139550213 conn(0x5607880d0300 msgr2=0x5607884806d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.906+0000 7fa84b7fe640 0 client.28973 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.907+0000 7fa879375740 1 -- 172.21.15.67:0/1139550213 shutdown_connections 2024-07-31T15:09:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.907+0000 7fa879375740 1 -- 172.21.15.67:0/1139550213 wait complete. 2024-07-31T15:09:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.907+0000 7fa879375740 20 client.28973 trim_cache size 0 max 16384 2024-07-31T15:09:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.926+0000 7fa879375740 1 Processor -- start 2024-07-31T15:09:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.926+0000 7fa879375740 1 -- start start 2024-07-31T15:09:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.926+0000 7fa879375740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880b32f0 0x56078849a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.926+0000 7fa879375740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880bb4d0 0x56078849af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.926+0000 7fa879375740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 0x5607884a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.926+0000 7fa879375740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1ae560 con 0x5607880b32f0 2024-07-31T15:09:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.926+0000 7fa879375740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1b00a0 con 0x5607880bb4d0 2024-07-31T15:09:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.926+0000 7fa879375740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1aec30 con 0x5607880dc1b0 2024-07-31T15:09:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.926+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880bb4d0 0x56078849af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.926+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880bb4d0 0x56078849af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59948/0 (socket says 172.21.15.67:59948) 2024-07-31T15:09:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.926+0000 7fa86b7fe640 1 -- 172.21.15.67:0/634002722 learned_addr learned my addr 172.21.15.67:0/634002722 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.927+0000 7fa870a1b640 1 --2- 172.21.15.67:0/634002722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880b32f0 0x56078849a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.927+0000 7fa870a1b640 1 --2- 172.21.15.67:0/634002722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 0x5607884a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.927+0000 7fa86b7fe640 1 -- 172.21.15.67:0/634002722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 msgr2=0x5607884a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.927+0000 7fa86b7fe640 1 --2- 172.21.15.67:0/634002722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 0x5607884a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.927+0000 7fa86b7fe640 1 -- 172.21.15.67:0/634002722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880b32f0 msgr2=0x56078849a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.927+0000 7fa86b7fe640 1 --2- 172.21.15.67:0/634002722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880b32f0 0x56078849a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.927+0000 7fa86b7fe640 1 -- 172.21.15.67:0/634002722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56078847fb20 con 0x5607880bb4d0 2024-07-31T15:09:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.927+0000 7fa86b7fe640 1 --2- 172.21.15.67:0/634002722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880bb4d0 0x56078849af30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa860005e00 tx=0x7fa860005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5379e7d38318463 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.927+0000 7fa851ffb640 1 -- 172.21.15.67:0/634002722 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86000fa20 con 0x5607880bb4d0 2024-07-31T15:09:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.927+0000 7fa851ffb640 1 -- 172.21.15.67:0/634002722 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa86000fbc0 con 0x5607880bb4d0 2024-07-31T15:09:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.927+0000 7fa851ffb640 1 -- 172.21.15.67:0/634002722 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860003ce0 con 0x5607880bb4d0 2024-07-31T15:09:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.928+0000 7fa879375740 1 -- 172.21.15.67:0/634002722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880bb4d0 msgr2=0x56078849af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.928+0000 7fa879375740 1 --2- 172.21.15.67:0/634002722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880bb4d0 0x56078849af30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa860005e00 tx=0x7fa860005f90 comp rx=0 tx=0).stop 2024-07-31T15:09:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.928+0000 7fa879375740 1 -- 172.21.15.67:0/634002722 shutdown_connections 2024-07-31T15:09:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.928+0000 7fa879375740 1 --2- 172.21.15.67:0/634002722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 0x5607884a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.928+0000 7fa879375740 1 --2- 172.21.15.67:0/634002722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880bb4d0 0x56078849af30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.928+0000 7fa879375740 1 --2- 172.21.15.67:0/634002722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880b32f0 0x56078849a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.928+0000 7fa879375740 1 -- 172.21.15.67:0/634002722 >> 172.21.15.67:0/634002722 conn(0x5607880d0300 msgr2=0x560788484720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.928+0000 7fa879375740 1 -- 172.21.15.67:0/634002722 shutdown_connections 2024-07-31T15:09:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.928+0000 7fa879375740 1 -- 172.21.15.67:0/634002722 wait complete. 2024-07-31T15:09:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.929+0000 7fa879375740 1 -- start start 2024-07-31T15:09:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.929+0000 7fa879375740 1 Processor -- start 2024-07-31T15:09:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.929+0000 7fa879375740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880b32f0 0x5607884971d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.929+0000 7fa879375740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x560788493b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.929+0000 7fa879375740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 0x560788494070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.929+0000 7fa879375740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1af9d0 con 0x5607880bb4d0 2024-07-31T15:09:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.929+0000 7fa879375740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1af300 con 0x5607880b32f0 2024-07-31T15:09:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.929+0000 7fa879375740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1ac560 con 0x5607880dc1b0 2024-07-31T15:09:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa879375740 10 client.0 _ll_register_callbacks cb 0x7fa860009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x560788493b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa870a1b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880b32f0 0x5607884971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x560788493b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39748/0 (socket says 172.21.15.67:39748) 2024-07-31T15:09:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa86b7fe640 1 -- 172.21.15.67:0/2639021571 learned_addr learned my addr 172.21.15.67:0/2639021571 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa870a1b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880b32f0 0x5607884971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59964/0 (socket says 172.21.15.67:59964) 2024-07-31T15:09:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa870a1b640 1 --2- 172.21.15.67:0/2639021571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 0x560788494070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa86b7fe640 1 -- 172.21.15.67:0/2639021571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 msgr2=0x560788494070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa86b7fe640 1 --2- 172.21.15.67:0/2639021571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 0x560788494070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa86b7fe640 1 -- 172.21.15.67:0/2639021571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880b32f0 msgr2=0x5607884971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa86b7fe640 1 --2- 172.21.15.67:0/2639021571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880b32f0 0x5607884971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa86b7fe640 1 -- 172.21.15.67:0/2639021571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560788481210 con 0x5607880bb4d0 2024-07-31T15:09:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa870a1b640 1 --2- 172.21.15.67:0/2639021571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880b32f0 0x5607884971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa86b7fe640 1 --2- 172.21.15.67:0/2639021571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x560788493b30 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa8600039c0 tx=0x7fa86000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fae680067a4c0fb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.930+0000 7fa84b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.931+0000 7fa84b7fe640 1 -- 172.21.15.67:0/2639021571 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86000fd70 con 0x5607880bb4d0 2024-07-31T15:09:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.931+0000 7fa879375740 1 -- 172.21.15.67:0/2639021571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa86000fbc0 con 0x5607880bb4d0 2024-07-31T15:09:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.931+0000 7fa84b7fe640 1 -- 172.21.15.67:0/2639021571 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7fa860005050 con 0x5607880bb4d0 2024-07-31T15:09:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.931+0000 7fa84b7fe640 1 -- 172.21.15.67:0/2639021571 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fa86003b050 con 0x5607880bb4d0 2024-07-31T15:09:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.931+0000 7fa879375740 10 client.19443 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.931+0000 7fa879375740 1 -- 172.21.15.67:0/2639021571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x560788507de0 con 0x5607880bb4d0 2024-07-31T15:09:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.931+0000 7fa84b7fe640 1 -- 172.21.15.67:0/2639021571 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa86000fbc0 con 0x5607880bb4d0 2024-07-31T15:09:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.931+0000 7fa84b7fe640 1 -- 172.21.15.67:0/2639021571 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fa8600354e0 con 0x5607880bb4d0 2024-07-31T15:09:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.931+0000 7fa879375740 10 client.19443 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.931+0000 7fa879375740 10 client.19443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.931+0000 7fa879375740 20 client.19443 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.931+0000 7fa879375740 1 --2- 172.21.15.67:0/2639021571 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5607880701c0 0x56078850a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.931+0000 7fa879375740 4 client.19443 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:09:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.931+0000 7fa879375740 1 -- 172.21.15.67:0/2639021571 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607883f1da0 con 0x5607880701c0 2024-07-31T15:09:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.932+0000 7fa870a1b640 1 --2- 172.21.15.67:0/2639021571 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5607880701c0 0x56078850a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.932+0000 7fa84b7fe640 10 client.19443 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.932+0000 7fa870a1b640 1 --2- 172.21.15.67:0/2639021571 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5607880701c0 0x56078850a580 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e71a1e382b072360 server_cookie=ed74ef66fa52ae7b in_seq=0 out_seq=0 2024-07-31T15:09:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.999+0000 7fa84b7fe640 1 -- 172.21.15.67:0/2639021571 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5607883f1da0 con 0x5607880701c0 2024-07-31T15:09:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.999+0000 7fa84b7fe640 10 client.19443 handle_command_reply: tid=0 2024-07-31T15:09:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:45.999+0000 7fa879375740 1 client.19443 shutdown 2024-07-31T15:09:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.000+0000 7fa879375740 1 -- 172.21.15.67:0/2639021571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 msgr2=0x560788493b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.000+0000 7fa879375740 1 --2- 172.21.15.67:0/2639021571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x560788493b30 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa8600039c0 tx=0x7fa86000fab0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.000+0000 7fa879375740 1 -- 172.21.15.67:0/2639021571 shutdown_connections 2024-07-31T15:09:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.000+0000 7fa879375740 1 --2- 172.21.15.67:0/2639021571 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5607880701c0 0x56078850a580 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.000+0000 7fa879375740 1 --2- 172.21.15.67:0/2639021571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607880dc1b0 0x560788494070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.000+0000 7fa879375740 1 --2- 172.21.15.67:0/2639021571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5607880bb4d0 0x560788493b30 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.000+0000 7fa879375740 1 --2- 172.21.15.67:0/2639021571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607880b32f0 0x5607884971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.000+0000 7fa879375740 1 -- 172.21.15.67:0/2639021571 >> 172.21.15.67:0/2639021571 conn(0x5607880d0300 msgr2=0x56078848de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.000+0000 7fa84b7fe640 0 client.19443 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.000+0000 7fa879375740 1 -- 172.21.15.67:0/2639021571 shutdown_connections 2024-07-31T15:09:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.000+0000 7fa879375740 1 -- 172.21.15.67:0/2639021571 wait complete. 2024-07-31T15:09:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.000+0000 7fa879375740 20 client.19443 trim_cache size 0 max 16384 2024-07-31T15:09:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:09:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:09:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:09:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:09:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.001+0000 7fa87247e640 1 -- 172.21.15.67:0/192942069 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa8540b3d40 msgr2=0x7fa8540b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.001+0000 7fa87247e640 1 --2- 172.21.15.67:0/192942069 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa8540b3d40 0x7fa8540b6250 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x56078847d960 tx=0x7fa85c10a260 comp rx=0 tx=0).stop 2024-07-31T15:09:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.001+0000 7fa87247e640 1 -- 172.21.15.67:0/192942069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa86c101a50 msgr2=0x7fa86c19a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.001+0000 7fa87247e640 1 --2- 172.21.15.67:0/192942069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa86c101a50 0x7fa86c19a020 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa85c032fd0 tx=0x7fa85c031a20 comp rx=0 tx=0).stop 2024-07-31T15:09:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.001+0000 7fa87247e640 1 -- 172.21.15.67:0/192942069 shutdown_connections 2024-07-31T15:09:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.001+0000 7fa87247e640 1 --2- 172.21.15.67:0/192942069 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa8540b3d40 0x7fa8540b6250 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.001+0000 7fa87247e640 1 --2- 172.21.15.67:0/192942069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa86c1098e0 0x7fa86c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.001+0000 7fa87247e640 1 --2- 172.21.15.67:0/192942069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa86c108f90 0x7fa86c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.001+0000 7fa87247e640 1 --2- 172.21.15.67:0/192942069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa86c101a50 0x7fa86c19a020 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.001+0000 7fa87247e640 1 -- 172.21.15.67:0/192942069 >> 172.21.15.67:0/192942069 conn(0x7fa86c079880 msgr2=0x7fa86c106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.001+0000 7fa87247e640 1 -- 172.21.15.67:0/192942069 shutdown_connections 2024-07-31T15:09:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:46.001+0000 7fa87247e640 1 -- 172.21.15.67:0/192942069 wait complete. 2024-07-31T15:09:46.534 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:09:46.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.539 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.540 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.540 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.540 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:46.540 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:46 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:09:46.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:46.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.642+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/2652754484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f44102490 msgr2=0x7f7f441028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.642+0000 7f7f4bf1e640 1 --2- 172.21.15.67:0/2652754484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f44102490 0x7f7f441028a0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009900 tx=0x7f7f4002a410 comp rx=0 tx=0).stop 2024-07-31T15:09:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.642+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/2652754484 shutdown_connections 2024-07-31T15:09:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.642+0000 7f7f4bf1e640 1 --2- 172.21.15.67:0/2652754484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f44108720 0x7f7f4410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.642+0000 7f7f4bf1e640 1 --2- 172.21.15.67:0/2652754484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f44107dd0 0x7f7f441081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.642+0000 7f7f4bf1e640 1 --2- 172.21.15.67:0/2652754484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f44102490 0x7f7f441028a0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.642+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/2652754484 >> 172.21.15.67:0/2652754484 conn(0x7f7f44079160 msgr2=0x7f7f44079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.642+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/2652754484 shutdown_connections 2024-07-31T15:09:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.642+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/2652754484 wait complete. 2024-07-31T15:09:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.643+0000 7f7f4bf1e640 1 Processor -- start 2024-07-31T15:09:46.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.643+0000 7f7f4bf1e640 1 -- start start 2024-07-31T15:09:46.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.643+0000 7f7f4bf1e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f44102490 0x7f7f44199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:46.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.643+0000 7f7f4bf1e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f44107dd0 0x7f7f4419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:46.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.643+0000 7f7f4bf1e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f44108720 0x7f7f4419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:46.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.643+0000 7f7f4bf1e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7f441123d0 con 0x7f7f44108720 2024-07-31T15:09:46.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.644+0000 7f7f4bf1e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7f44112250 con 0x7f7f44107dd0 2024-07-31T15:09:46.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.644+0000 7f7f49cba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f44102490 0x7f7f44199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:46.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.644+0000 7f7f494b9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f44107dd0 0x7f7f4419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:46.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.644+0000 7f7f494b9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f44107dd0 0x7f7f4419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59986/0 (socket says 172.21.15.67:59986) 2024-07-31T15:09:46.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.644+0000 7f7f494b9640 1 -- 172.21.15.67:0/1212722313 learned_addr learned my addr 172.21.15.67:0/1212722313 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:46.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.644+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/1212722313 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f44112550 con 0x7f7f44102490 2024-07-31T15:09:46.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.644+0000 7f7f4a4bb640 1 --2- 172.21.15.67:0/1212722313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f44108720 0x7f7f4419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:46.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.644+0000 7f7f494b9640 1 -- 172.21.15.67:0/1212722313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f44102490 msgr2=0x7f7f44199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:46.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.644+0000 7f7f494b9640 1 --2- 172.21.15.67:0/1212722313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f44102490 0x7f7f44199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.644+0000 7f7f494b9640 1 -- 172.21.15.67:0/1212722313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f44108720 msgr2=0x7f7f4419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:46.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.644+0000 7f7f494b9640 1 --2- 172.21.15.67:0/1212722313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f44108720 0x7f7f4419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.644+0000 7f7f494b9640 1 -- 172.21.15.67:0/1212722313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4419fe10 con 0x7f7f44107dd0 2024-07-31T15:09:46.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.644+0000 7f7f49cba640 1 --2- 172.21.15.67:0/1212722313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f44102490 0x7f7f44199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:46.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.644+0000 7f7f494b9640 1 --2- 172.21.15.67:0/1212722313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f44107dd0 0x7f7f4419a490 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400e9c0 tx=0x7f7f3400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:46.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.645+0000 7f7f3affd640 1 -- 172.21.15.67:0/1212722313 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3400cd80 con 0x7f7f44107dd0 2024-07-31T15:09:46.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.645+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/1212722313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f441a0100 con 0x7f7f44107dd0 2024-07-31T15:09:46.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.645+0000 7f7f3affd640 1 -- 172.21.15.67:0/1212722313 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f340054d0 con 0x7f7f44107dd0 2024-07-31T15:09:46.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.645+0000 7f7f3affd640 1 -- 172.21.15.67:0/1212722313 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34010430 con 0x7f7f44107dd0 2024-07-31T15:09:46.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.645+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/1212722313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f441a68b0 con 0x7f7f44107dd0 2024-07-31T15:09:46.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.646+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/1212722313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f441028a0 con 0x7f7f44107dd0 2024-07-31T15:09:46.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.648+0000 7f7f3affd640 1 -- 172.21.15.67:0/1212722313 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7f34004fb0 con 0x7f7f44107dd0 2024-07-31T15:09:46.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.649+0000 7f7f3affd640 1 --2- 172.21.15.67:0/1212722313 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7f300b3dc0 0x7f7f300b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:46.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.649+0000 7f7f49cba640 1 --2- 172.21.15.67:0/1212722313 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7f300b3dc0 0x7f7f300b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:46.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.649+0000 7f7f3affd640 1 -- 172.21.15.67:0/1212722313 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f7f340e48d0 con 0x7f7f44107dd0 2024-07-31T15:09:46.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.649+0000 7f7f49cba640 1 --2- 172.21.15.67:0/1212722313 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7f300b3dc0 0x7f7f300b62d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7f40005ab0 tx=0x7f7f40005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:46.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.652+0000 7f7f3affd640 1 -- 172.21.15.67:0/1212722313 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f7f340555e0 con 0x7f7f44107dd0 2024-07-31T15:09:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:09:46.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:46.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:46 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:46.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.939+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/1212722313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f4419b020 con 0x7f7f44107dd0 2024-07-31T15:09:46.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.940+0000 7f7f3affd640 1 -- 172.21.15.67:0/1212722313 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11962 (secure 0 0 0) 0x7f7f340ab8a0 con 0x7f7f44107dd0 2024-07-31T15:09:46.941 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:09:46.941 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"btime":"2024-07-31T15:06:04:323642+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:06:04.323492+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28214":{"gid":28214,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1516494761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1516494761},{"type":"v1","addr":"172.21.15.73:6835","nonce":1516494761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:09:46.942 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-07-31T15:09:46.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.942+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/1212722313 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7f300b3dc0 msgr2=0x7f7f300b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:46.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.942+0000 7f7f4bf1e640 1 --2- 172.21.15.67:0/1212722313 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7f300b3dc0 0x7f7f300b62d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7f40005ab0 tx=0x7f7f40005a20 comp rx=0 tx=0).stop 2024-07-31T15:09:46.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.942+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/1212722313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f44107dd0 msgr2=0x7f7f4419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.942+0000 7f7f4bf1e640 1 --2- 172.21.15.67:0/1212722313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f44107dd0 0x7f7f4419a490 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400e9c0 tx=0x7f7f3400ee80 comp rx=0 tx=0).stop 2024-07-31T15:09:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.942+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/1212722313 shutdown_connections 2024-07-31T15:09:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.942+0000 7f7f4bf1e640 1 --2- 172.21.15.67:0/1212722313 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7f300b3dc0 0x7f7f300b62d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.942+0000 7f7f4bf1e640 1 --2- 172.21.15.67:0/1212722313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f44108720 0x7f7f4419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.942+0000 7f7f4bf1e640 1 --2- 172.21.15.67:0/1212722313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f44107dd0 0x7f7f4419a490 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.942+0000 7f7f4bf1e640 1 --2- 172.21.15.67:0/1212722313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f44102490 0x7f7f44199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.942+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/1212722313 >> 172.21.15.67:0/1212722313 conn(0x7f7f44079160 msgr2=0x7f7f44106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.942+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/1212722313 shutdown_connections 2024-07-31T15:09:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:46.942+0000 7f7f4bf1e640 1 -- 172.21.15.67:0/1212722313 wait complete. 2024-07-31T15:09:47.142 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:09:46 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:09:46.674+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000439c [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du/ auth{2=1} v303 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:01.164016+0000 34=34+0) n(v3 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:484109 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce83f19600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000439c:[<0x10000004399/rcar-du v163>,<0x10000002d23/renesas v2988>,<0x10000002d21/drm v1196>,<0x10000001d8f/gpu v9523>,<0x10000000202/drivers v2799>,<0x10000000201/linux-6.5.11 v334>,<0x10000000006/t v232>,<0x10000000005/tmp v202>,<0x100000001fb/client.0 v201>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v184>,<0x10000000001/sv_1 v213>,<0x10000000000/qa v197>,<0x1/volumes v198>]//[]","memoryvalue":"(2)0x1000000439c:[<0x10000004399/rcar-du v303>,<0x10000002d23/renesas v3147>,<0x10000002d21/drm v1457>,<0x10000001d8f/gpu v18750>,<0x10000000202/drivers v21433>,<0x10000000201/linux-6.5.11 v2445>,<0x10000000006/t v1498>,<0x10000000005/tmp v1428>,<0x100000001fb/client.0 v1403>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1367>,<0x10000000001/sv_1 v1368>,<0x10000000000/qa v1362>,<0x1/volumes v1321>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:38.082295+0000 30=30+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:01.164016+0000 34=34+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:09:47.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: pgmap v995: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 646 KiB/s wr, 1.46k op/s 2024-07-31T15:09:47.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1212722313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:09:47.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:47.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:47 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: pgmap v995: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 646 KiB/s wr, 1.46k op/s 2024-07-31T15:09:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:09:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1212722313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:09:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:47.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:47 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: pgmap v995: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 646 KiB/s wr, 1.46k op/s 2024-07-31T15:09:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:09:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1212722313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:09:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:47.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:47 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:47.751 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 7e631bf4 --release --await' 2024-07-31T15:09:47.751 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 7e631bf4 --release --await 2024-07-31T15:09:48.095 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:09:48.496 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:48.502 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:48 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:48.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:48 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:48.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:48.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:48 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: pgmap v996: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 562 KiB/s wr, 3.02k op/s 2024-07-31T15:09:49.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.537 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:09:49.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:49 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: pgmap v996: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 562 KiB/s wr, 3.02k op/s 2024-07-31T15:09:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:09:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:49 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: pgmap v996: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 562 KiB/s wr, 3.02k op/s 2024-07-31T15:09:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:09:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:49.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:49 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:09:50.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.035+0000 7fe2a1068640 1 -- 172.21.15.67:0/2662912064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe29c075ab0 msgr2=0x7fe29c10f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:50.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.035+0000 7fe2a1068640 1 --2- 172.21.15.67:0/2662912064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe29c075ab0 0x7fe29c10f650 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fe284009900 tx=0x7fe28402a410 comp rx=0 tx=0).stop 2024-07-31T15:09:50.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.035+0000 7fe2a1068640 1 -- 172.21.15.67:0/2662912064 shutdown_connections 2024-07-31T15:09:50.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.035+0000 7fe2a1068640 1 --2- 172.21.15.67:0/2662912064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c10fb90 0x7fe29c111fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.035+0000 7fe2a1068640 1 --2- 172.21.15.67:0/2662912064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe29c075ab0 0x7fe29c10f650 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.035+0000 7fe2a1068640 1 --2- 172.21.15.67:0/2662912064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c075160 0x7fe29c075570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.035+0000 7fe2a1068640 1 -- 172.21.15.67:0/2662912064 >> 172.21.15.67:0/2662912064 conn(0x7fe29c0ff040 msgr2=0x7fe29c101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:50.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.035+0000 7fe2a1068640 1 -- 172.21.15.67:0/2662912064 shutdown_connections 2024-07-31T15:09:50.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.035+0000 7fe2a1068640 1 -- 172.21.15.67:0/2662912064 wait complete. 2024-07-31T15:09:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.036+0000 7fe2a1068640 1 Processor -- start 2024-07-31T15:09:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe2a1068640 1 -- start start 2024-07-31T15:09:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe2a1068640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c075160 0x7fe29c19c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe2a1068640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe29c075ab0 0x7fe29c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe2a1068640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c10fb90 0x7fe29c1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe2a1068640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1156a0 con 0x7fe29c075ab0 2024-07-31T15:09:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe2a1068640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c115520 con 0x7fe29c10fb90 2024-07-31T15:09:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe2a1068640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c115820 con 0x7fe29c075160 2024-07-31T15:09:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe29a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe29c075ab0 0x7fe29c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe29a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe29c075ab0 0x7fe29c19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39774/0 (socket says 172.21.15.67:39774) 2024-07-31T15:09:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe29a59c640 1 -- 172.21.15.67:0/13047653 learned_addr learned my addr 172.21.15.67:0/13047653 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe29ad9d640 1 --2- 172.21.15.67:0/13047653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c075160 0x7fe29c19c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe29b59e640 1 --2- 172.21.15.67:0/13047653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c10fb90 0x7fe29c1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe29a59c640 1 -- 172.21.15.67:0/13047653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c075160 msgr2=0x7fe29c19c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe29a59c640 1 --2- 172.21.15.67:0/13047653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c075160 0x7fe29c19c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.037+0000 7fe29a59c640 1 -- 172.21.15.67:0/13047653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c10fb90 msgr2=0x7fe29c1a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.038+0000 7fe29a59c640 1 --2- 172.21.15.67:0/13047653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c10fb90 0x7fe29c1a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.038+0000 7fe29a59c640 1 -- 172.21.15.67:0/13047653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29c1a20a0 con 0x7fe29c075ab0 2024-07-31T15:09:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.038+0000 7fe29a59c640 1 --2- 172.21.15.67:0/13047653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe29c075ab0 0x7fe29c19c6d0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fe28402a930 tx=0x7fe284004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.038+0000 7fe293fff640 1 -- 172.21.15.67:0/13047653 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28403f030 con 0x7fe29c075ab0 2024-07-31T15:09:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.038+0000 7fe2a1068640 1 -- 172.21.15.67:0/13047653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29c1a2330 con 0x7fe29c075ab0 2024-07-31T15:09:50.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.038+0000 7fe293fff640 1 -- 172.21.15.67:0/13047653 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe284003aa0 con 0x7fe29c075ab0 2024-07-31T15:09:50.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.038+0000 7fe293fff640 1 -- 172.21.15.67:0/13047653 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284035070 con 0x7fe29c075ab0 2024-07-31T15:09:50.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.038+0000 7fe2a1068640 1 -- 172.21.15.67:0/13047653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe29c1a8af0 con 0x7fe29c075ab0 2024-07-31T15:09:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.041+0000 7fe2a75b9740 1 Processor -- start 2024-07-31T15:09:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.041+0000 7fe2a75b9740 1 -- start start 2024-07-31T15:09:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe293fff640 1 -- 172.21.15.67:0/13047653 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe284042030 con 0x7fe29c075ab0 2024-07-31T15:09:50.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe2a75b9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc1a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe2a75b9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b84b660 0x55ad5bc1a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe2a75b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b853840 0x55ad5bc1aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe2a75b9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ad5bb8fe70 con 0x55ad5b874520 2024-07-31T15:09:50.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe2a75b9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ad5bb7f370 con 0x55ad5b84b660 2024-07-31T15:09:50.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe2a75b9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ad5bb833b0 con 0x55ad5b853840 2024-07-31T15:09:50.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe29b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc1a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe29b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc1a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39780/0 (socket says 172.21.15.67:39780) 2024-07-31T15:09:50.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe29b59e640 1 -- 172.21.15.67:0/159324118 learned_addr learned my addr 172.21.15.67:0/159324118 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:50.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe29a59c640 1 --2- 172.21.15.67:0/159324118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b853840 0x55ad5bc1aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe29ad9d640 1 --2- 172.21.15.67:0/159324118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b84b660 0x55ad5bc1a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe29b59e640 1 -- 172.21.15.67:0/159324118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b853840 msgr2=0x55ad5bc1aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:50.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe29b59e640 1 --2- 172.21.15.67:0/159324118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b853840 0x55ad5bc1aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe29b59e640 1 -- 172.21.15.67:0/159324118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b84b660 msgr2=0x55ad5bc1a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:50.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe29b59e640 1 --2- 172.21.15.67:0/159324118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b84b660 0x55ad5bc1a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.042+0000 7fe29b59e640 1 -- 172.21.15.67:0/159324118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad5bb4c040 con 0x55ad5b874520 2024-07-31T15:09:50.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.043+0000 7fe29b59e640 1 --2- 172.21.15.67:0/159324118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc1a070 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0039f0 tx=0x7fe28c003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5866da4c2f9e816 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:50.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.043+0000 7fe2917fa640 1 -- 172.21.15.67:0/159324118 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c01a470 con 0x55ad5b874520 2024-07-31T15:09:50.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.043+0000 7fe293fff640 1 --2- 172.21.15.67:0/13047653 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2880b3d40 0x7fe2880b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.043+0000 7fe2917fa640 1 -- 172.21.15.67:0/159324118 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe28c015070 con 0x55ad5b874520 2024-07-31T15:09:50.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.043+0000 7fe2917fa640 1 -- 172.21.15.67:0/159324118 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0107e0 con 0x55ad5b874520 2024-07-31T15:09:50.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.043+0000 7fe29ad9d640 1 --2- 172.21.15.67:0/13047653 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2880b3d40 0x7fe2880b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.043+0000 7fe293fff640 1 -- 172.21.15.67:0/13047653 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7fe284104950 con 0x7fe29c075ab0 2024-07-31T15:09:50.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.044+0000 7fe2a75b9740 1 -- 172.21.15.67:0/159324118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 msgr2=0x55ad5bc1a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:50.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.044+0000 7fe29ad9d640 1 --2- 172.21.15.67:0/13047653 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2880b3d40 0x7fe2880b6250 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55ad5bc18cf0 tx=0x7fe29400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:50.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.044+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/159324118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc1a070 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0039f0 tx=0x7fe28c003eb0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.044+0000 7fe2a75b9740 1 -- 172.21.15.67:0/159324118 shutdown_connections 2024-07-31T15:09:50.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.044+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/159324118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b853840 0x55ad5bc1aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.044+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/159324118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b84b660 0x55ad5bc1a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.044+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/159324118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc1a070 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.044+0000 7fe2a75b9740 1 -- 172.21.15.67:0/159324118 >> 172.21.15.67:0/159324118 conn(0x55ad5b868910 msgr2=0x55ad5bc17f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:50.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.044+0000 7fe2a75b9740 1 -- 172.21.15.67:0/159324118 shutdown_connections 2024-07-31T15:09:50.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.044+0000 7fe2a75b9740 1 -- 172.21.15.67:0/159324118 wait complete. 2024-07-31T15:09:50.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.045+0000 7fe2a75b9740 1 -- start start 2024-07-31T15:09:50.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.045+0000 7fe2a75b9740 1 Processor -- start 2024-07-31T15:09:50.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.045+0000 7fe2a75b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b84b660 0x55ad5bc25bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.045+0000 7fe2a75b9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b853840 0x55ad5bc26130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.045+0000 7fe2a75b9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc31bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.045+0000 7fe2a75b9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ad5b9e2bb0 con 0x55ad5b874520 2024-07-31T15:09:50.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.045+0000 7fe2a75b9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ad5bb21ac0 con 0x55ad5b853840 2024-07-31T15:09:50.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.045+0000 7fe2a75b9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ad5bc288a0 con 0x55ad5b84b660 2024-07-31T15:09:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.045+0000 7fe2a75b9740 10 client.0 _ll_register_callbacks cb 0x55ad5bb2a3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.045+0000 7fe29ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b84b660 0x55ad5bc25bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.045+0000 7fe29b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc31bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.045+0000 7fe29ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b84b660 0x55ad5bc25bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:40900/0 (socket says 172.21.15.67:40900) 2024-07-31T15:09:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.045+0000 7fe29ad9d640 1 -- 172.21.15.67:0/1873288842 learned_addr learned my addr 172.21.15.67:0/1873288842 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.045+0000 7fe29a59c640 1 --2- 172.21.15.67:0/1873288842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b853840 0x55ad5bc26130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.046+0000 7fe29ad9d640 1 -- 172.21.15.67:0/1873288842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b853840 msgr2=0x55ad5bc26130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.046+0000 7fe29ad9d640 1 --2- 172.21.15.67:0/1873288842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b853840 0x55ad5bc26130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.046+0000 7fe29ad9d640 1 -- 172.21.15.67:0/1873288842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 msgr2=0x55ad5bc31bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.046+0000 7fe29ad9d640 1 --2- 172.21.15.67:0/1873288842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc31bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.046+0000 7fe29ad9d640 1 -- 172.21.15.67:0/1873288842 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad5b8dfb00 con 0x55ad5b84b660 2024-07-31T15:09:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.046+0000 7fe29ad9d640 1 --2- 172.21.15.67:0/1873288842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b84b660 0x55ad5bc25bf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe294003cb0 tx=0x7fe2940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb8848e0f2db861 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.046+0000 7fe2827fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:09:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.046+0000 7fe2827fc640 1 -- 172.21.15.67:0/1873288842 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29402c060 con 0x55ad5b84b660 2024-07-31T15:09:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.046+0000 7fe2a75b9740 1 -- 172.21.15.67:0/1873288842 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe28c015070 con 0x55ad5b84b660 2024-07-31T15:09:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.046+0000 7fe2827fc640 1 -- 172.21.15.67:0/1873288842 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7fe294017030 con 0x55ad5b84b660 2024-07-31T15:09:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.046+0000 7fe2827fc640 1 -- 172.21.15.67:0/1873288842 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe294033050 con 0x55ad5b84b660 2024-07-31T15:09:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.047+0000 7fe2a75b9740 10 client.28870 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.047+0000 7fe2a75b9740 1 -- 172.21.15.67:0/1873288842 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55ad5b956960 con 0x55ad5b84b660 2024-07-31T15:09:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.047+0000 7fe2827fc640 1 -- 172.21.15.67:0/1873288842 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe28c015070 con 0x55ad5b84b660 2024-07-31T15:09:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.047+0000 7fe2827fc640 1 -- 172.21.15.67:0/1873288842 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fe29401c480 con 0x55ad5b84b660 2024-07-31T15:09:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.047+0000 7fe2a75b9740 10 client.28870 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.047+0000 7fe2a75b9740 10 client.28870 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:09:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.047+0000 7fe2a75b9740 20 client.28870 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.047+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/1873288842 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ad5b832ea0 0x55ad5bc37c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.047+0000 7fe2a75b9740 4 client.28870 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:09:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.047+0000 7fe2a75b9740 1 -- 172.21.15.67:0/1873288842 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad5bb91dc0 con 0x55ad5b832ea0 2024-07-31T15:09:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.047+0000 7fe29a59c640 1 --2- 172.21.15.67:0/1873288842 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ad5b832ea0 0x55ad5bc37c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.048+0000 7fe29a59c640 1 --2- 172.21.15.67:0/1873288842 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ad5b832ea0 0x55ad5bc37c00 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1ce22d7e16f6b4 server_cookie=62f7ca5e683046ee in_seq=0 out_seq=0 2024-07-31T15:09:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.048+0000 7fe2827fc640 10 client.28870 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2827fc640 1 -- 172.21.15.67:0/1873288842 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad5bb91dc0 con 0x55ad5b832ea0 2024-07-31T15:09:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2827fc640 10 client.28870 handle_command_reply: tid=0 2024-07-31T15:09:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2a75b9740 1 client.28870 shutdown 2024-07-31T15:09:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2a75b9740 1 -- 172.21.15.67:0/1873288842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b84b660 msgr2=0x55ad5bc25bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/1873288842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b84b660 0x55ad5bc25bf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe294003cb0 tx=0x7fe2940046c0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2a75b9740 1 -- 172.21.15.67:0/1873288842 shutdown_connections 2024-07-31T15:09:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/1873288842 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ad5b832ea0 0x55ad5bc37c00 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/1873288842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc31bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/1873288842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b853840 0x55ad5bc26130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2827fc640 0 client.28870 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/1873288842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b84b660 0x55ad5bc25bf0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2a75b9740 1 -- 172.21.15.67:0/1873288842 >> 172.21.15.67:0/1873288842 conn(0x55ad5b868910 msgr2=0x55ad5bc1b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2a75b9740 1 -- 172.21.15.67:0/1873288842 shutdown_connections 2024-07-31T15:09:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2a75b9740 1 -- 172.21.15.67:0/1873288842 wait complete. 2024-07-31T15:09:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.049+0000 7fe2a75b9740 20 client.28870 trim_cache size 0 max 16384 2024-07-31T15:09:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.067+0000 7fe2a75b9740 1 Processor -- start 2024-07-31T15:09:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.067+0000 7fe2a75b9740 1 -- start start 2024-07-31T15:09:50.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe2a75b9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b84b660 0x55ad5bc20190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe2a75b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b853840 0x55ad5bc1b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe2a75b9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc319e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe2a75b9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1b29d0 con 0x55ad5b874520 2024-07-31T15:09:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe2a75b9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1b4510 con 0x55ad5b84b660 2024-07-31T15:09:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe2a75b9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1b30a0 con 0x55ad5b853840 2024-07-31T15:09:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe29b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe29b59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc319e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39788/0 (socket says 172.21.15.67:39788) 2024-07-31T15:09:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe29b59e640 1 -- 172.21.15.67:0/3151899272 learned_addr learned my addr 172.21.15.67:0/3151899272 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe29a59c640 1 --2- 172.21.15.67:0/3151899272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b853840 0x55ad5bc1b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe29ad9d640 1 --2- 172.21.15.67:0/3151899272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b84b660 0x55ad5bc20190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe29b59e640 1 -- 172.21.15.67:0/3151899272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b853840 msgr2=0x55ad5bc1b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe29b59e640 1 --2- 172.21.15.67:0/3151899272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b853840 0x55ad5bc1b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe29b59e640 1 -- 172.21.15.67:0/3151899272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b84b660 msgr2=0x55ad5bc20190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe29b59e640 1 --2- 172.21.15.67:0/3151899272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b84b660 0x55ad5bc20190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe29b59e640 1 -- 172.21.15.67:0/3151899272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad5bc27fd0 con 0x55ad5b874520 2024-07-31T15:09:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe29a59c640 1 --2- 172.21.15.67:0/3151899272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b853840 0x55ad5bc1b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:09:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.068+0000 7fe29ad9d640 1 --2- 172.21.15.67:0/3151899272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b84b660 0x55ad5bc20190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:09:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.069+0000 7fe29b59e640 1 --2- 172.21.15.67:0/3151899272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc319e0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fe28c011fa0 tx=0x7fe28c009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90aec1db3feb7bc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.069+0000 7fe290ff9640 1 -- 172.21.15.67:0/3151899272 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c011440 con 0x55ad5b874520 2024-07-31T15:09:50.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.069+0000 7fe290ff9640 1 -- 172.21.15.67:0/3151899272 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe28c015040 con 0x55ad5b874520 2024-07-31T15:09:50.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.069+0000 7fe290ff9640 1 -- 172.21.15.67:0/3151899272 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c01a970 con 0x55ad5b874520 2024-07-31T15:09:50.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.070+0000 7fe2a75b9740 1 -- 172.21.15.67:0/3151899272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 msgr2=0x55ad5bc319e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:50.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.070+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/3151899272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc319e0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fe28c011fa0 tx=0x7fe28c009b00 comp rx=0 tx=0).stop 2024-07-31T15:09:50.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.070+0000 7fe2a75b9740 1 -- 172.21.15.67:0/3151899272 shutdown_connections 2024-07-31T15:09:50.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.070+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/3151899272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b874520 0x55ad5bc319e0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.070+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/3151899272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b853840 0x55ad5bc1b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.070+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/3151899272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b84b660 0x55ad5bc20190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.070+0000 7fe2a75b9740 1 -- 172.21.15.67:0/3151899272 >> 172.21.15.67:0/3151899272 conn(0x55ad5b868910 msgr2=0x55ad5bc1b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:50.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.070+0000 7fe2a75b9740 1 -- 172.21.15.67:0/3151899272 shutdown_connections 2024-07-31T15:09:50.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.070+0000 7fe2a75b9740 1 -- 172.21.15.67:0/3151899272 wait complete. 2024-07-31T15:09:50.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.070+0000 7fe2a75b9740 1 -- start start 2024-07-31T15:09:50.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.070+0000 7fe2a75b9740 1 Processor -- start 2024-07-31T15:09:50.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe2a75b9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b84b660 0x55ad5bc2e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe2a75b9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b853840 0x55ad5bc29580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe2a75b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b874520 0x55ad5bc29ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe2a75b9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1b3e40 con 0x55ad5b84b660 2024-07-31T15:09:50.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe2a75b9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1b3770 con 0x55ad5b853840 2024-07-31T15:09:50.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe2a75b9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1b09d0 con 0x55ad5b874520 2024-07-31T15:09:50.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe2a75b9740 10 client.0 _ll_register_callbacks cb 0x7fe294005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:50.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe29ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b84b660 0x55ad5bc2e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe29ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b84b660 0x55ad5bc2e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39794/0 (socket says 172.21.15.67:39794) 2024-07-31T15:09:50.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe29ad9d640 1 -- 172.21.15.67:0/885233004 learned_addr learned my addr 172.21.15.67:0/885233004 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:50.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe29a59c640 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b853840 0x55ad5bc29580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe29b59e640 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b874520 0x55ad5bc29ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe29ad9d640 1 -- 172.21.15.67:0/885233004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b874520 msgr2=0x55ad5bc29ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:50.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe29ad9d640 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b874520 0x55ad5bc29ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe29ad9d640 1 -- 172.21.15.67:0/885233004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b853840 msgr2=0x55ad5bc29580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:50.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe29ad9d640 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b853840 0x55ad5bc29580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:50.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe29ad9d640 1 -- 172.21.15.67:0/885233004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad5bc2a100 con 0x55ad5b84b660 2024-07-31T15:09:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe29b59e640 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b874520 0x55ad5bc29ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:09:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe29a59c640 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b853840 0x55ad5bc29580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:09:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe29ad9d640 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b84b660 0x55ad5bc2e0d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fe294018040 tx=0x7fe29401cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e9e9727e13cc2c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.071+0000 7fe2827fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.072+0000 7fe2827fc640 1 -- 172.21.15.67:0/885233004 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29402c020 con 0x55ad5b84b660 2024-07-31T15:09:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.072+0000 7fe2827fc640 1 -- 172.21.15.67:0/885233004 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7fe294005240 con 0x55ad5b84b660 2024-07-31T15:09:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.072+0000 7fe2a75b9740 1 -- 172.21.15.67:0/885233004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe28c015040 con 0x55ad5b84b660 2024-07-31T15:09:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.072+0000 7fe2827fc640 1 -- 172.21.15.67:0/885233004 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe29401ce60 con 0x55ad5b84b660 2024-07-31T15:09:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.072+0000 7fe2a75b9740 10 client.19467 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.072+0000 7fe2a75b9740 1 -- 172.21.15.67:0/885233004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55ad5bca0170 con 0x55ad5b84b660 2024-07-31T15:09:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.072+0000 7fe2827fc640 1 -- 172.21.15.67:0/885233004 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe28c015040 con 0x55ad5b84b660 2024-07-31T15:09:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.072+0000 7fe2827fc640 1 -- 172.21.15.67:0/885233004 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fe294027410 con 0x55ad5b84b660 2024-07-31T15:09:50.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.072+0000 7fe2a75b9740 10 client.19467 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:50.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.072+0000 7fe2a75b9740 10 client.19467 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:09:50.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.072+0000 7fe2a75b9740 20 client.19467 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:50.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.072+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ad5b808250 0x55ad5bca28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:50.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.072+0000 7fe2a75b9740 4 client.19467 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7e631bf4", "release": true, "await": true}] 2024-07-31T15:09:50.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.072+0000 7fe2a75b9740 1 -- 172.21.15.67:0/885233004 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7e631bf4", "release": true, "await": true}) -- 0x55ad5bb8d860 con 0x55ad5b808250 2024-07-31T15:09:50.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.073+0000 7fe29a59c640 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ad5b808250 0x55ad5bca28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:50.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.073+0000 7fe29a59c640 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ad5b808250 0x55ad5bca28c0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13136680f10ff982 server_cookie=45c6bd7b5e6a04b9 in_seq=0 out_seq=0 2024-07-31T15:09:50.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:50.073+0000 7fe2827fc640 10 client.19467 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:51.539 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:51 smithi077 ceph-mon[37327]: pgmap v997: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 269 KiB/s wr, 3.78k op/s 2024-07-31T15:09:51.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:51 smithi067 ceph-mon[32750]: pgmap v997: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 269 KiB/s wr, 3.78k op/s 2024-07-31T15:09:51.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:51 smithi073 ceph-mon[36714]: pgmap v997: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 269 KiB/s wr, 3.78k op/s 2024-07-31T15:09:52.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.478+0000 7fe2827fc640 1 -- 172.21.15.67:0/885233004 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ad5bb8d860 con 0x55ad5b808250 2024-07-31T15:09:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.478+0000 7fe2827fc640 10 client.19467 handle_command_reply: tid=0 2024-07-31T15:09:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.478+0000 7fe2a75b9740 1 client.19467 shutdown 2024-07-31T15:09:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.479+0000 7fe2a75b9740 1 -- 172.21.15.67:0/885233004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b84b660 msgr2=0x55ad5bc2e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.479+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b84b660 0x55ad5bc2e0d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fe294018040 tx=0x7fe29401cd40 comp rx=0 tx=0).stop 2024-07-31T15:09:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.479+0000 7fe2a75b9740 1 -- 172.21.15.67:0/885233004 shutdown_connections 2024-07-31T15:09:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.479+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55ad5b808250 0x55ad5bca28c0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.479+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ad5b874520 0x55ad5bc29ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.479+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad5b853840 0x55ad5bc29580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.479+0000 7fe2a75b9740 1 --2- 172.21.15.67:0/885233004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ad5b84b660 0x55ad5bc2e0d0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.479+0000 7fe2a75b9740 1 -- 172.21.15.67:0/885233004 >> 172.21.15.67:0/885233004 conn(0x55ad5b868910 msgr2=0x55ad5bc1b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.479+0000 7fe2827fc640 0 client.19467 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.479+0000 7fe2a75b9740 1 -- 172.21.15.67:0/885233004 shutdown_connections 2024-07-31T15:09:52.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.479+0000 7fe2a75b9740 1 -- 172.21.15.67:0/885233004 wait complete. 2024-07-31T15:09:52.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.479+0000 7fe2a75b9740 20 client.19467 trim_cache size 0 max 16384 2024-07-31T15:09:52.482 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:09:52.482 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-07-31T15:09:52.482 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:09:52.482 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 52, 2024-07-31T15:09:52.482 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:09:52.482 INFO:teuthology.orchestra.run.smithi067.stdout: "7e631bf4": { 2024-07-31T15:09:52.483 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 52, 2024-07-31T15:09:52.483 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:09:52.483 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:09:52.483 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:09:52.483 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:09:52.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:09:52.483 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:09:52.483 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 138.1, 2024-07-31T15:09:52.483 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:09:52.483 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:09:52.483 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:09:52.483 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:09:52.483 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:09:52.484 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:09:52.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:09:52.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:09:52.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:09:52.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:09:52.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:09:52.484 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:09:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.480+0000 7fe2a1068640 1 -- 172.21.15.67:0/13047653 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2880b3d40 msgr2=0x7fe2880b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.480+0000 7fe2a1068640 1 --2- 172.21.15.67:0/13047653 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2880b3d40 0x7fe2880b6250 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55ad5bc18cf0 tx=0x7fe29400b390 comp rx=0 tx=0).stop 2024-07-31T15:09:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.480+0000 7fe2a1068640 1 -- 172.21.15.67:0/13047653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe29c075ab0 msgr2=0x7fe29c19c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.480+0000 7fe2a1068640 1 --2- 172.21.15.67:0/13047653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe29c075ab0 0x7fe29c19c6d0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fe28402a930 tx=0x7fe284004270 comp rx=0 tx=0).stop 2024-07-31T15:09:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.480+0000 7fe2a1068640 1 -- 172.21.15.67:0/13047653 shutdown_connections 2024-07-31T15:09:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.480+0000 7fe2a1068640 1 --2- 172.21.15.67:0/13047653 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe2880b3d40 0x7fe2880b6250 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.480+0000 7fe2a1068640 1 --2- 172.21.15.67:0/13047653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c10fb90 0x7fe29c1a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.480+0000 7fe2a1068640 1 --2- 172.21.15.67:0/13047653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe29c075ab0 0x7fe29c19c6d0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.480+0000 7fe2a1068640 1 --2- 172.21.15.67:0/13047653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c075160 0x7fe29c19c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.480+0000 7fe2a1068640 1 -- 172.21.15.67:0/13047653 >> 172.21.15.67:0/13047653 conn(0x7fe29c0ff040 msgr2=0x7fe29c109770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.480+0000 7fe2a1068640 1 -- 172.21.15.67:0/13047653 shutdown_connections 2024-07-31T15:09:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:52.480+0000 7fe2a1068640 1 -- 172.21.15.67:0/13047653 wait complete. 2024-07-31T15:09:52.533 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_transport_spi.o 2024-07-31T15:09:52.540 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_execlists_submission.o 2024-07-31T15:09:52.555 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/arp.o 2024-07-31T15:09:52.590 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] net/netfilter/xt_addrtype.o 2024-07-31T15:09:52.599 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/udp.o 2024-07-31T15:09:52.608 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_ggtt.o 2024-07-31T15:09:52.612 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/mlme.o 2024-07-31T15:09:52.628 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth.o 2024-07-31T15:09:53.356 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7e631bf4', total seconds elapsed: 41.4 2024-07-31T15:09:53.356 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.1 2024-07-31T15:09:53.357 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:09:53.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:53 smithi067 ceph-mon[32750]: pgmap v998: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 215 KiB/s wr, 4.53k op/s 2024-07-31T15:09:53.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:53 smithi073 ceph-mon[36714]: pgmap v998: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 215 KiB/s wr, 4.53k op/s 2024-07-31T15:09:53.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:53 smithi077 ceph-mon[37327]: pgmap v998: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 215 KiB/s wr, 4.53k op/s 2024-07-31T15:09:53.790 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/virtio_scsi.o 2024-07-31T15:09:53.879 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/sd.o 2024-07-31T15:09:53.893 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/netfilter/built-in.a 2024-07-31T15:09:53.904 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:09:54.062 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/main.o 2024-07-31T15:09:54.079 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/icmp.o 2024-07-31T15:09:54.186 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_null.o 2024-07-31T15:09:54.468 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_ggtt_fencing.o 2024-07-31T15:09:54.566 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/ibss.o 2024-07-31T15:09:54.602 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/udplite.o 2024-07-31T15:09:54.717 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/status.o 2024-07-31T15:09:55.073 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_tls.o 2024-07-31T15:09:55.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:55 smithi067 ceph-mon[32750]: pgmap v999: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 213 KiB/s wr, 4.63k op/s 2024-07-31T15:09:55.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:55 smithi073 ceph-mon[36714]: pgmap v999: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 213 KiB/s wr, 4.63k op/s 2024-07-31T15:09:55.659 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/raw.o 2024-07-31T15:09:55.663 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gt.o 2024-07-31T15:09:55.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:55 smithi077 ceph-mon[37327]: pgmap v999: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 213 KiB/s wr, 4.63k op/s 2024-07-31T15:09:55.865 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/devinet.o 2024-07-31T15:09:56.044 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/auth_unix.o 2024-07-31T15:09:56.046 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/sme.o 2024-07-31T15:09:56.260 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/sr.o 2024-07-31T15:09:56.281 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/svc.o 2024-07-31T15:09:56.511 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/driver-ops.o 2024-07-31T15:09:56.946 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.o 2024-07-31T15:09:57.048 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/icmp.o 2024-07-31T15:09:57.081 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/sr_ioctl.o 2024-07-31T15:09:57.102 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/svcsock.o 2024-07-31T15:09:57.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:57 smithi067 ceph-mon[32750]: pgmap v1000: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 123 KiB/s wr, 4.23k op/s 2024-07-31T15:09:57.586 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/af_inet.o 2024-07-31T15:09:57.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:57 smithi073 ceph-mon[36714]: pgmap v1000: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 123 KiB/s wr, 4.23k op/s 2024-07-31T15:09:57.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:57 smithi077 ceph-mon[37327]: pgmap v1000: 129 pgs: 129 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 123 KiB/s wr, 4.23k op/s 2024-07-31T15:09:57.858 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/sr_vendor.o 2024-07-31T15:09:57.870 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_clock_utils.o 2024-07-31T15:09:58.151 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/svcauth.o 2024-07-31T15:09:58.174 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/chan.o 2024-07-31T15:09:58.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.205+0000 7f694fac9640 1 -- 172.21.15.67:0/533377554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6948111770 msgr2=0x7f694810a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:58.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.205+0000 7f694fac9640 1 --2- 172.21.15.67:0/533377554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6948111770 0x7f694810a070 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f694401a910 tx=0x7f694403b450 comp rx=0 tx=0).stop 2024-07-31T15:09:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.206+0000 7f694fac9640 1 -- 172.21.15.67:0/533377554 shutdown_connections 2024-07-31T15:09:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.206+0000 7f694fac9640 1 --2- 172.21.15.67:0/533377554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f694811c5d0 0x7f694811e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.206+0000 7f694fac9640 1 --2- 172.21.15.67:0/533377554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f694810a5b0 0x7f694810a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.206+0000 7f694fac9640 1 --2- 172.21.15.67:0/533377554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6948111770 0x7f694810a070 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.206+0000 7f694fac9640 1 -- 172.21.15.67:0/533377554 >> 172.21.15.67:0/533377554 conn(0x7f694806d380 msgr2=0x7f694806d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:58.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.207+0000 7f694fac9640 1 -- 172.21.15.67:0/533377554 shutdown_connections 2024-07-31T15:09:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.207+0000 7f694fac9640 1 -- 172.21.15.67:0/533377554 wait complete. 2024-07-31T15:09:58.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.208+0000 7f694fac9640 1 Processor -- start 2024-07-31T15:09:58.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.208+0000 7f694fac9640 1 -- start start 2024-07-31T15:09:58.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.208+0000 7f694fac9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f694810a5b0 0x7f694811be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:58.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.208+0000 7f694fac9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6948111770 0x7f6948118140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:58.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.208+0000 7f694fac9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f694811c5d0 0x7f69481186c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:58.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.208+0000 7f694fac9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6948114640 con 0x7f694810a5b0 2024-07-31T15:09:58.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.208+0000 7f694fac9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69481144c0 con 0x7f6948111770 2024-07-31T15:09:58.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.208+0000 7f694fac9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69481147c0 con 0x7f694811c5d0 2024-07-31T15:09:58.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.209+0000 7f694d064640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6948111770 0x7f6948118140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:58.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.209+0000 7f694d064640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6948111770 0x7f6948118140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56588/0 (socket says 172.21.15.67:56588) 2024-07-31T15:09:58.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.209+0000 7f694d064640 1 -- 172.21.15.67:0/879336309 learned_addr learned my addr 172.21.15.67:0/879336309 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:58.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.209+0000 7f694d865640 1 --2- 172.21.15.67:0/879336309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f694810a5b0 0x7f694811be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:58.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.209+0000 7f694d064640 1 -- 172.21.15.67:0/879336309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f694811c5d0 msgr2=0x7f69481186c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:09:58.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.209+0000 7f694d064640 1 --2- 172.21.15.67:0/879336309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f694811c5d0 0x7f69481186c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:58.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.210+0000 7f694d064640 1 -- 172.21.15.67:0/879336309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f694810a5b0 msgr2=0x7f694811be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:58.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.210+0000 7f694d064640 1 --2- 172.21.15.67:0/879336309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f694810a5b0 0x7f694811be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:58.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.210+0000 7f694d064640 1 -- 172.21.15.67:0/879336309 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6948118e50 con 0x7f6948111770 2024-07-31T15:09:58.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.210+0000 7f694d865640 1 --2- 172.21.15.67:0/879336309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f694810a5b0 0x7f694811be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:09:58.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.210+0000 7f694d064640 1 --2- 172.21.15.67:0/879336309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6948111770 0x7f6948118140 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f694000ea50 tx=0x7f694000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.214+0000 7f693effd640 1 -- 172.21.15.67:0/879336309 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694000cdc0 con 0x7f6948111770 2024-07-31T15:09:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.214+0000 7f694fac9640 1 -- 172.21.15.67:0/879336309 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69481ac630 con 0x7f6948111770 2024-07-31T15:09:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.214+0000 7f694fac9640 1 -- 172.21.15.67:0/879336309 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69481acb70 con 0x7f6948111770 2024-07-31T15:09:58.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.215+0000 7f693effd640 1 -- 172.21.15.67:0/879336309 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6940005020 con 0x7f6948111770 2024-07-31T15:09:58.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.215+0000 7f694fac9640 1 -- 172.21.15.67:0/879336309 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f694810a9c0 con 0x7f6948111770 2024-07-31T15:09:58.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.221+0000 7f693effd640 1 -- 172.21.15.67:0/879336309 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69400116d0 con 0x7f6948111770 2024-07-31T15:09:58.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.221+0000 7f693effd640 1 -- 172.21.15.67:0/879336309 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f694003a030 con 0x7f6948111770 2024-07-31T15:09:58.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.222+0000 7f693effd640 1 --2- 172.21.15.67:0/879336309 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69340b3f80 0x7f69340b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:58.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.223+0000 7f693effd640 1 -- 172.21.15.67:0/879336309 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f6940107c40 con 0x7f6948111770 2024-07-31T15:09:58.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.224+0000 7f694d865640 1 --2- 172.21.15.67:0/879336309 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69340b3f80 0x7f69340b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:58.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.231+0000 7f693effd640 1 -- 172.21.15.67:0/879336309 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f69400cecb0 con 0x7f6948111770 2024-07-31T15:09:58.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.241+0000 7f694d865640 1 --2- 172.21.15.67:0/879336309 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69340b3f80 0x7f69340b6490 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6944006010 tx=0x7f6944000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:58.433 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/mcast.o 2024-07-31T15:09:58.437 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/sta_info.o 2024-07-31T15:09:58.650 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/sg.o 2024-07-31T15:09:58.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.762+0000 7f694fac9640 1 -- 172.21.15.67:0/879336309 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6948119440 con 0x7f6948111770 2024-07-31T15:09:58.766 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:09:58.767 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"btime":"2024-07-31T15:06:04:323642+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:06:04.323492+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28214":{"gid":28214,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1516494761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1516494761},{"type":"v1","addr":"172.21.15.73:6835","nonce":1516494761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:09:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.764+0000 7f693effd640 1 -- 172.21.15.67:0/879336309 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11962 (secure 0 0 0) 0x7f69400ce5b0 con 0x7f6948111770 2024-07-31T15:09:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-07-31T15:09:58.772 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_debugfs.o 2024-07-31T15:09:58.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.773+0000 7f693cf79640 1 -- 172.21.15.67:0/879336309 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69340b3f80 msgr2=0x7f69340b6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:58.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.773+0000 7f693cf79640 1 --2- 172.21.15.67:0/879336309 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69340b3f80 0x7f69340b6490 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6944006010 tx=0x7f6944000bc0 comp rx=0 tx=0).stop 2024-07-31T15:09:58.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.773+0000 7f693cf79640 1 -- 172.21.15.67:0/879336309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6948111770 msgr2=0x7f6948118140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:58.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.773+0000 7f693cf79640 1 --2- 172.21.15.67:0/879336309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6948111770 0x7f6948118140 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f694000ea50 tx=0x7f694000ef10 comp rx=0 tx=0).stop 2024-07-31T15:09:58.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.773+0000 7f693cf79640 1 -- 172.21.15.67:0/879336309 shutdown_connections 2024-07-31T15:09:58.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.773+0000 7f693cf79640 1 --2- 172.21.15.67:0/879336309 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f69340b3f80 0x7f69340b6490 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:58.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.773+0000 7f693cf79640 1 --2- 172.21.15.67:0/879336309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f694811c5d0 0x7f69481186c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:58.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.773+0000 7f693cf79640 1 --2- 172.21.15.67:0/879336309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6948111770 0x7f6948118140 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:58.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.773+0000 7f693cf79640 1 --2- 172.21.15.67:0/879336309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f694810a5b0 0x7f694811be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:58.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.773+0000 7f693cf79640 1 -- 172.21.15.67:0/879336309 >> 172.21.15.67:0/879336309 conn(0x7f694806d380 msgr2=0x7f6948110400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:58.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.774+0000 7f693cf79640 1 -- 172.21.15.67:0/879336309 shutdown_connections 2024-07-31T15:09:58.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:09:58.774+0000 7f693cf79640 1 -- 172.21.15.67:0/879336309 wait complete. 2024-07-31T15:09:59.307 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/svcauth_unix.o 2024-07-31T15:09:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.317+0000 7f98a13c8640 1 -- 172.21.15.67:0/1912978727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c115f40 msgr2=0x7f989c118360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.317+0000 7f98a13c8640 1 --2- 172.21.15.67:0/1912978727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c115f40 0x7f989c118360 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9894009ca0 tx=0x7f989402a8c0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.317+0000 7f98a13c8640 1 -- 172.21.15.67:0/1912978727 shutdown_connections 2024-07-31T15:09:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.317+0000 7f98a13c8640 1 --2- 172.21.15.67:0/1912978727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c115f40 0x7f989c118360 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.317+0000 7f98a13c8640 1 --2- 172.21.15.67:0/1912978727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f989c10a740 0x7f989c115a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.317+0000 7f98a13c8640 1 --2- 172.21.15.67:0/1912978727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f989c109df0 0x7f989c10a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.317+0000 7f98a13c8640 1 -- 172.21.15.67:0/1912978727 >> 172.21.15.67:0/1912978727 conn(0x7f989c06e350 msgr2=0x7f989c06e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.318+0000 7f98a13c8640 1 -- 172.21.15.67:0/1912978727 shutdown_connections 2024-07-31T15:09:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.318+0000 7f98a13c8640 1 -- 172.21.15.67:0/1912978727 wait complete. 2024-07-31T15:09:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.319+0000 7f98a13c8640 1 Processor -- start 2024-07-31T15:09:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.319+0000 7f98a13c8640 1 -- start start 2024-07-31T15:09:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.319+0000 7f98a13c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c109df0 0x7f989c112fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.319+0000 7f98a13c8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f989c10a740 0x7f989c1134f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.319+0000 7f98a13c8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f989c115f40 0x7f989c113a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.320+0000 7f98a13c8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f989c10d300 con 0x7f989c115f40 2024-07-31T15:09:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.320+0000 7f98a13c8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f989c10d180 con 0x7f989c109df0 2024-07-31T15:09:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.320+0000 7f98a13c8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f989c10d480 con 0x7f989c10a740 2024-07-31T15:09:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.320+0000 7f989a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f989c10a740 0x7f989c1134f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.320+0000 7f989a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f989c10a740 0x7f989c1134f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44378/0 (socket says 172.21.15.67:44378) 2024-07-31T15:09:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.320+0000 7f989a7fc640 1 -- 172.21.15.67:0/70866266 learned_addr learned my addr 172.21.15.67:0/70866266 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.320+0000 7f989b7fe640 1 --2- 172.21.15.67:0/70866266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f989c115f40 0x7f989c113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.320+0000 7f989affd640 1 --2- 172.21.15.67:0/70866266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c109df0 0x7f989c112fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.321+0000 7f989b7fe640 1 -- 172.21.15.67:0/70866266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f989c10a740 msgr2=0x7f989c1134f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.321+0000 7f989b7fe640 1 --2- 172.21.15.67:0/70866266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f989c10a740 0x7f989c1134f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.321+0000 7f989b7fe640 1 -- 172.21.15.67:0/70866266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c109df0 msgr2=0x7f989c112fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.321+0000 7f989b7fe640 1 --2- 172.21.15.67:0/70866266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c109df0 0x7f989c112fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.321+0000 7f989b7fe640 1 -- 172.21.15.67:0/70866266 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989c11e470 con 0x7f989c115f40 2024-07-31T15:09:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.322+0000 7f989b7fe640 1 --2- 172.21.15.67:0/70866266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f989c115f40 0x7f989c113a30 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f9894005e30 tx=0x7f98940319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.335+0000 7f9883fff640 1 -- 172.21.15.67:0/70866266 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894003cf0 con 0x7f989c115f40 2024-07-31T15:09:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.335+0000 7f9883fff640 1 -- 172.21.15.67:0/70866266 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98940044f0 con 0x7f989c115f40 2024-07-31T15:09:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.335+0000 7f9883fff640 1 -- 172.21.15.67:0/70866266 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989402fc30 con 0x7f989c115f40 2024-07-31T15:09:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.335+0000 7f98a13c8640 1 -- 172.21.15.67:0/70866266 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989c11e700 con 0x7f989c115f40 2024-07-31T15:09:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.335+0000 7f98a13c8640 1 -- 172.21.15.67:0/70866266 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f989c11ec00 con 0x7f989c115f40 2024-07-31T15:09:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.339+0000 7f98a8398740 1 Processor -- start 2024-07-31T15:09:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.342+0000 7f9883fff640 1 -- 172.21.15.67:0/70866266 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f9894004040 con 0x7f989c115f40 2024-07-31T15:09:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.342+0000 7f98a8398740 1 -- start start 2024-07-31T15:09:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.342+0000 7f98a8398740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 0x55b435613ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.343+0000 7f98a8398740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352482f0 0x55b4356143e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.343+0000 7f98a8398740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352504d0 0x55b435614920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.343+0000 7f98a8398740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b4355893b0 con 0x55b4352482f0 2024-07-31T15:09:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.343+0000 7f98a8398740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b4355788b0 con 0x55b4352504d0 2024-07-31T15:09:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.343+0000 7f98a8398740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b43557c8f0 con 0x55b4352711b0 2024-07-31T15:09:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.343+0000 7f989affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352482f0 0x55b4356143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.343+0000 7f989affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352482f0 0x55b4356143e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56010/0 (socket says 172.21.15.67:56010) 2024-07-31T15:09:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.343+0000 7f989affd640 1 -- 172.21.15.67:0/3610562613 learned_addr learned my addr 172.21.15.67:0/3610562613 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.343+0000 7f989a7fc640 1 --2- 172.21.15.67:0/3610562613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 0x55b435613ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.344+0000 7f989a7fc640 1 --2- 172.21.15.67:0/3610562613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352504d0 0x55b435614920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.344+0000 7f989affd640 1 -- 172.21.15.67:0/3610562613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 msgr2=0x55b435613ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.344+0000 7f989affd640 1 --2- 172.21.15.67:0/3610562613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 0x55b435613ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.344+0000 7f989affd640 1 -- 172.21.15.67:0/3610562613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352504d0 msgr2=0x55b435614920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.344+0000 7f989affd640 1 --2- 172.21.15.67:0/3610562613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352504d0 0x55b435614920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.344+0000 7f989affd640 1 -- 172.21.15.67:0/3610562613 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b43551f9a0 con 0x55b4352482f0 2024-07-31T15:09:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.345+0000 7f989affd640 1 --2- 172.21.15.67:0/3610562613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352482f0 0x55b4356143e0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9890009ce0 tx=0x7f9890011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aeac04b8842e7a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.345+0000 7f98817fa640 1 -- 172.21.15.67:0/3610562613 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890005220 con 0x55b4352482f0 2024-07-31T15:09:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.345+0000 7f98817fa640 1 -- 172.21.15.67:0/3610562613 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9890004c50 con 0x55b4352482f0 2024-07-31T15:09:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.345+0000 7f98817fa640 1 -- 172.21.15.67:0/3610562613 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98900197f0 con 0x55b4352482f0 2024-07-31T15:09:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.346+0000 7f9883fff640 1 --2- 172.21.15.67:0/70866266 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98840b3e80 0x7f98840b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.346+0000 7f9883fff640 1 -- 172.21.15.67:0/70866266 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f9894105800 con 0x7f989c115f40 2024-07-31T15:09:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.347+0000 7f989a7fc640 1 --2- 172.21.15.67:0/70866266 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98840b3e80 0x7f98840b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.347+0000 7f98a8398740 1 -- 172.21.15.67:0/3610562613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352482f0 msgr2=0x55b4356143e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.347+0000 7f98a8398740 1 --2- 172.21.15.67:0/3610562613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352482f0 0x55b4356143e0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9890009ce0 tx=0x7f9890011810 comp rx=0 tx=0).stop 2024-07-31T15:09:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.348+0000 7f989a7fc640 1 --2- 172.21.15.67:0/70866266 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98840b3e80 0x7f98840b6390 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f988c0058c0 tx=0x7f988c004730 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.354+0000 7f98a8398740 1 -- 172.21.15.67:0/3610562613 shutdown_connections 2024-07-31T15:09:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.354+0000 7f98a8398740 1 --2- 172.21.15.67:0/3610562613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352504d0 0x55b435614920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.354+0000 7f98a8398740 1 --2- 172.21.15.67:0/3610562613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352482f0 0x55b4356143e0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.354+0000 7f98a8398740 1 --2- 172.21.15.67:0/3610562613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 0x55b435613ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.354+0000 7f98a8398740 1 -- 172.21.15.67:0/3610562613 >> 172.21.15.67:0/3610562613 conn(0x55b435265300 msgr2=0x55b435611d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.354+0000 7f98a8398740 1 -- 172.21.15.67:0/3610562613 shutdown_connections 2024-07-31T15:09:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.354+0000 7f98a8398740 1 -- 172.21.15.67:0/3610562613 wait complete. 2024-07-31T15:09:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.354+0000 7f98a8398740 1 -- start start 2024-07-31T15:09:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.355+0000 7f98a8398740 1 Processor -- start 2024-07-31T15:09:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.355+0000 7f98a8398740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352482f0 0x55b43561f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.355+0000 7f98a8398740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352504d0 0x55b43561fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.355+0000 7f98a8398740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 0x55b43562b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.355+0000 7f98a8398740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b4354b2ab0 con 0x55b4352504d0 2024-07-31T15:09:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.355+0000 7f98a8398740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b435500cf0 con 0x55b4352482f0 2024-07-31T15:09:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.355+0000 7f98a8398740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b435622620 con 0x55b4352711b0 2024-07-31T15:09:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.355+0000 7f98a8398740 10 client.0 _ll_register_callbacks cb 0x55b4355729f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.356+0000 7f989a7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352504d0 0x55b43561fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.356+0000 7f989affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352482f0 0x55b43561f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.356+0000 7f989b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 0x55b43562b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.356+0000 7f989b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 0x55b43562b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44382/0 (socket says 172.21.15.67:44382) 2024-07-31T15:09:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.356+0000 7f989b7fe640 1 -- 172.21.15.67:0/407050215 learned_addr learned my addr 172.21.15.67:0/407050215 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.357+0000 7f989a7fc640 1 -- 172.21.15.67:0/407050215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 msgr2=0x55b43562b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.357+0000 7f989a7fc640 1 --2- 172.21.15.67:0/407050215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 0x55b43562b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.357+0000 7f989a7fc640 1 -- 172.21.15.67:0/407050215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352482f0 msgr2=0x55b43561f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.357+0000 7f989a7fc640 1 --2- 172.21.15.67:0/407050215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352482f0 0x55b43561f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.357+0000 7f989a7fc640 1 -- 172.21.15.67:0/407050215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4352dc790 con 0x55b4352504d0 2024-07-31T15:09:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.357+0000 7f989a7fc640 1 --2- 172.21.15.67:0/407050215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352504d0 0x55b43561fee0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f988c011fe0 tx=0x7f988c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbc4027014237e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.366+0000 7f987eff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:09:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.366+0000 7f987eff5640 1 -- 172.21.15.67:0/407050215 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c022610 con 0x55b4352504d0 2024-07-31T15:09:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.366+0000 7f98a8398740 1 -- 172.21.15.67:0/407050215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9890004c50 con 0x55b4352504d0 2024-07-31T15:09:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.366+0000 7f987eff5640 1 -- 172.21.15.67:0/407050215 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f988c0227b0 con 0x55b4352504d0 2024-07-31T15:09:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.367+0000 7f987eff5640 1 -- 172.21.15.67:0/407050215 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f988c033050 con 0x55b4352504d0 2024-07-31T15:09:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.367+0000 7f98a8398740 10 client.19482 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.367+0000 7f98a8398740 1 -- 172.21.15.67:0/407050215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55b435355620 con 0x55b4352504d0 2024-07-31T15:09:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.368+0000 7f987eff5640 1 -- 172.21.15.67:0/407050215 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9890004c50 con 0x55b4352504d0 2024-07-31T15:09:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.368+0000 7f987eff5640 1 -- 172.21.15.67:0/407050215 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f988c01e070 con 0x55b4352504d0 2024-07-31T15:09:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.368+0000 7f98a8398740 10 client.19482 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.368+0000 7f98a8398740 10 client.19482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.368+0000 7f98a8398740 20 client.19482 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.368+0000 7f98a8398740 1 --2- 172.21.15.67:0/407050215 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b43522fd80 0x55b435631e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.368+0000 7f98a8398740 4 client.19482 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:09:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.368+0000 7f98a8398740 1 -- 172.21.15.67:0/407050215 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b43558b300 con 0x55b43522fd80 2024-07-31T15:09:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.368+0000 7f989affd640 1 --2- 172.21.15.67:0/407050215 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b43522fd80 0x55b435631e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.369+0000 7f989affd640 1 --2- 172.21.15.67:0/407050215 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b43522fd80 0x55b435631e20 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5858072e151ffe83 server_cookie=de7a433b5fc25665 in_seq=0 out_seq=0 2024-07-31T15:09:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.370+0000 7f987eff5640 10 client.19482 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.375+0000 7f987eff5640 1 -- 172.21.15.67:0/407050215 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b43558b300 con 0x55b43522fd80 2024-07-31T15:09:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.375+0000 7f987eff5640 10 client.19482 handle_command_reply: tid=0 2024-07-31T15:09:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.375+0000 7f98a8398740 1 client.19482 shutdown 2024-07-31T15:09:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.375+0000 7f98a8398740 1 -- 172.21.15.67:0/407050215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352504d0 msgr2=0x55b43561fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.375+0000 7f98a8398740 1 --2- 172.21.15.67:0/407050215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352504d0 0x55b43561fee0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f988c011fe0 tx=0x7f988c00a7d0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.376+0000 7f98a8398740 1 -- 172.21.15.67:0/407050215 shutdown_connections 2024-07-31T15:09:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.376+0000 7f98a8398740 1 --2- 172.21.15.67:0/407050215 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b43522fd80 0x55b435631e20 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.376+0000 7f98a8398740 1 --2- 172.21.15.67:0/407050215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 0x55b43562b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.376+0000 7f987eff5640 0 client.19482 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.376+0000 7f98a8398740 1 --2- 172.21.15.67:0/407050215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352504d0 0x55b43561fee0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.376+0000 7f98a8398740 1 --2- 172.21.15.67:0/407050215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352482f0 0x55b43561f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.376+0000 7f98a8398740 1 -- 172.21.15.67:0/407050215 >> 172.21.15.67:0/407050215 conn(0x55b435265300 msgr2=0x55b435614ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.376+0000 7f98a8398740 1 -- 172.21.15.67:0/407050215 shutdown_connections 2024-07-31T15:09:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.376+0000 7f98a8398740 1 -- 172.21.15.67:0/407050215 wait complete. 2024-07-31T15:09:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.376+0000 7f98a8398740 20 client.19482 trim_cache size 0 max 16384 2024-07-31T15:09:59.401 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/igmp.o 2024-07-31T15:09:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.413+0000 7f98a8398740 1 Processor -- start 2024-07-31T15:09:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.413+0000 7f98a8398740 1 -- start start 2024-07-31T15:09:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.414+0000 7f98a8398740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352482f0 0x55b435610d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.414+0000 7f98a8398740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352504d0 0x55b43561f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.414+0000 7f98a8398740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352711b0 0x55b4356213c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.414+0000 7f989affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352482f0 0x55b435610d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.414+0000 7f989a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352504d0 0x55b43561f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.414+0000 7f989affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352482f0 0x55b435610d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44394/0 (socket says 172.21.15.67:44394) 2024-07-31T15:09:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.414+0000 7f989a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352504d0 0x55b43561f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56618/0 (socket says 172.21.15.67:56618) 2024-07-31T15:09:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.414+0000 7f989b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352711b0 0x55b4356213c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.414+0000 7f989b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352711b0 0x55b4356213c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56032/0 (socket says 172.21.15.67:56032) 2024-07-31T15:09:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.414+0000 7f989affd640 1 -- 172.21.15.67:0/1117878455 learned_addr learned my addr 172.21.15.67:0/1117878455 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.414+0000 7f98a8398740 1 -- 172.21.15.67:0/1117878455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1c7c10 con 0x55b4352711b0 2024-07-31T15:09:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.414+0000 7f98a8398740 1 -- 172.21.15.67:0/1117878455 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1c9750 con 0x55b4352504d0 2024-07-31T15:09:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.414+0000 7f98a8398740 1 -- 172.21.15.67:0/1117878455 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1c82e0 con 0x55b4352482f0 2024-07-31T15:09:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.415+0000 7f989a7fc640 1 -- 172.21.15.67:0/1117878455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352482f0 msgr2=0x55b435610d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.415+0000 7f989a7fc640 1 --2- 172.21.15.67:0/1117878455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352482f0 0x55b435610d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.415+0000 7f989a7fc640 1 -- 172.21.15.67:0/1117878455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352711b0 msgr2=0x55b4356213c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.415+0000 7f989a7fc640 1 --2- 172.21.15.67:0/1117878455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352711b0 0x55b4356213c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.415+0000 7f989a7fc640 1 -- 172.21.15.67:0/1117878455 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b435620630 con 0x55b4352504d0 2024-07-31T15:09:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.415+0000 7f989b7fe640 1 --2- 172.21.15.67:0/1117878455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352711b0 0x55b4356213c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:09:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.415+0000 7f989a7fc640 1 --2- 172.21.15.67:0/1117878455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352504d0 0x55b43561f000 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f988c00b010 tx=0x7f988c02af20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29c03109eaa6f904 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.416+0000 7f9880ff9640 1 -- 172.21.15.67:0/1117878455 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c02a600 con 0x55b4352504d0 2024-07-31T15:09:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.416+0000 7f9880ff9640 1 -- 172.21.15.67:0/1117878455 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f988c03c030 con 0x55b4352504d0 2024-07-31T15:09:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.416+0000 7f9880ff9640 1 -- 172.21.15.67:0/1117878455 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c0047a0 con 0x55b4352504d0 2024-07-31T15:09:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.417+0000 7f98a8398740 1 -- 172.21.15.67:0/1117878455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352504d0 msgr2=0x55b43561f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.417+0000 7f98a8398740 1 --2- 172.21.15.67:0/1117878455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352504d0 0x55b43561f000 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f988c00b010 tx=0x7f988c02af20 comp rx=0 tx=0).stop 2024-07-31T15:09:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.417+0000 7f98a8398740 1 -- 172.21.15.67:0/1117878455 shutdown_connections 2024-07-31T15:09:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.417+0000 7f98a8398740 1 --2- 172.21.15.67:0/1117878455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352711b0 0x55b4356213c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.417+0000 7f98a8398740 1 --2- 172.21.15.67:0/1117878455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352504d0 0x55b43561f000 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.417+0000 7f98a8398740 1 --2- 172.21.15.67:0/1117878455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352482f0 0x55b435610d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.417+0000 7f98a8398740 1 -- 172.21.15.67:0/1117878455 >> 172.21.15.67:0/1117878455 conn(0x55b435265300 msgr2=0x55b435620cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.418+0000 7f98a8398740 1 -- 172.21.15.67:0/1117878455 shutdown_connections 2024-07-31T15:09:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.418+0000 7f98a8398740 1 -- 172.21.15.67:0/1117878455 wait complete. 2024-07-31T15:09:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.418+0000 7f98a8398740 1 -- start start 2024-07-31T15:09:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.418+0000 7f98a8398740 1 Processor -- start 2024-07-31T15:09:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.419+0000 7f98a8398740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352482f0 0x55b435627ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.419+0000 7f98a8398740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352504d0 0x55b435624890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.419+0000 7f98a8398740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 0x55b435624dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.419+0000 7f98a8398740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1c9080 con 0x55b4352504d0 2024-07-31T15:09:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.419+0000 7f98a8398740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1c89b0 con 0x55b4352482f0 2024-07-31T15:09:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.419+0000 7f98a8398740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1142b0 con 0x55b4352711b0 2024-07-31T15:09:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.419+0000 7f98a8398740 10 client.0 _ll_register_callbacks cb 0x7f988c011c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:09:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.419+0000 7f989b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 0x55b435624dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.419+0000 7f989b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 0x55b435624dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:44404/0 (socket says 172.21.15.67:44404) 2024-07-31T15:09:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.419+0000 7f989b7fe640 1 -- 172.21.15.67:0/3407383201 learned_addr learned my addr 172.21.15.67:0/3407383201 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:09:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.419+0000 7f989affd640 1 --2- 172.21.15.67:0/3407383201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352482f0 0x55b435627ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.420+0000 7f989affd640 1 -- 172.21.15.67:0/3407383201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 msgr2=0x55b435624dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:09:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.420+0000 7f989affd640 1 --2- 172.21.15.67:0/3407383201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 0x55b435624dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.420+0000 7f989affd640 1 -- 172.21.15.67:0/3407383201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352504d0 msgr2=0x55b435624890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:09:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.420+0000 7f989a7fc640 1 --2- 172.21.15.67:0/3407383201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352504d0 0x55b435624890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.420+0000 7f989affd640 1 --2- 172.21.15.67:0/3407383201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352504d0 0x55b435624890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.420+0000 7f989affd640 1 -- 172.21.15.67:0/3407383201 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b435625310 con 0x55b4352482f0 2024-07-31T15:09:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.420+0000 7f989a7fc640 1 --2- 172.21.15.67:0/3407383201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352504d0 0x55b435624890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:09:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.421+0000 7f989affd640 1 --2- 172.21.15.67:0/3407383201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352482f0 0x55b435627ca0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9890009820 tx=0x7f9890009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36eeae1f84192f40 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:09:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.421+0000 7f987eff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:09:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.421+0000 7f987eff5640 1 -- 172.21.15.67:0/3407383201 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98900098d0 con 0x55b4352482f0 2024-07-31T15:09:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.421+0000 7f98a8398740 1 -- 172.21.15.67:0/3407383201 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f988c03c030 con 0x55b4352482f0 2024-07-31T15:09:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.421+0000 7f987eff5640 1 -- 172.21.15.67:0/3407383201 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f98900051b0 con 0x55b4352482f0 2024-07-31T15:09:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.421+0000 7f987eff5640 1 -- 172.21.15.67:0/3407383201 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f989003a050 con 0x55b4352482f0 2024-07-31T15:09:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.422+0000 7f98a8398740 10 client.29021 fetch_fsmap learned FSMap version 65 2024-07-31T15:09:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.422+0000 7f98a8398740 1 -- 172.21.15.67:0/3407383201 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55b435699240 con 0x55b4352482f0 2024-07-31T15:09:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.422+0000 7f987eff5640 1 -- 172.21.15.67:0/3407383201 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f988c03c030 con 0x55b4352482f0 2024-07-31T15:09:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.422+0000 7f987eff5640 1 -- 172.21.15.67:0/3407383201 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f989002d3b0 con 0x55b4352482f0 2024-07-31T15:09:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.422+0000 7f98a8398740 10 client.29021 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:09:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.422+0000 7f98a8398740 10 client.29021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:09:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.422+0000 7f98a8398740 20 client.29021 populate_metadata read hostname 'smithi067' 2024-07-31T15:09:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.422+0000 7f98a8398740 1 --2- 172.21.15.67:0/3407383201 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4352051c0 0x55b43569b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:09:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.422+0000 7f98a8398740 4 client.29021 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:09:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.422+0000 7f98a8398740 1 -- 172.21.15.67:0/3407383201 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b43561b2d0 con 0x55b4352051c0 2024-07-31T15:09:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.423+0000 7f989a7fc640 1 --2- 172.21.15.67:0/3407383201 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4352051c0 0x55b43569b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:09:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.423+0000 7f989a7fc640 1 --2- 172.21.15.67:0/3407383201 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4352051c0 0x55b43569b800 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5346043fe66b5684 server_cookie=2608c9567d9220d1 in_seq=0 out_seq=0 2024-07-31T15:09:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.423+0000 7f987eff5640 10 client.29021 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:59.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:59 smithi067 ceph-mon[32750]: pgmap v1001: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 277 KiB/s wr, 3.33k op/s 2024-07-31T15:09:59.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:59 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/879336309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:09:59.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:09:59 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:09:59.621 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/ethtool.o 2024-07-31T15:09:59.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:59 smithi073 ceph-mon[36714]: pgmap v1001: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 277 KiB/s wr, 3.33k op/s 2024-07-31T15:09:59.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:59 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/879336309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:09:59.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:09:59 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:09:59.681 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 138.1 --await-for 45.0' 2024-07-31T15:09:59.681 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 138.1 --await-for 45.0 2024-07-31T15:09:59.728 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_engines_debugfs.o 2024-07-31T15:09:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:59 smithi077 ceph-mon[37327]: pgmap v1001: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 277 KiB/s wr, 3.33k op/s 2024-07-31T15:09:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:59 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/879336309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:09:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:09:59 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:09:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.894+0000 7f987eff5640 1 -- 172.21.15.67:0/3407383201 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b43561b2d0 con 0x55b4352051c0 2024-07-31T15:09:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.894+0000 7f987eff5640 10 client.29021 handle_command_reply: tid=0 2024-07-31T15:09:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.894+0000 7f98a8398740 1 client.29021 shutdown 2024-07-31T15:09:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.899+0000 7f98a8398740 1 -- 172.21.15.67:0/3407383201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352482f0 msgr2=0x55b435627ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.899+0000 7f98a8398740 1 --2- 172.21.15.67:0/3407383201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352482f0 0x55b435627ca0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9890009820 tx=0x7f9890009640 comp rx=0 tx=0).stop 2024-07-31T15:09:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.899+0000 7f98a8398740 1 -- 172.21.15.67:0/3407383201 shutdown_connections 2024-07-31T15:09:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.899+0000 7f98a8398740 1 --2- 172.21.15.67:0/3407383201 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4352051c0 0x55b43569b800 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.899+0000 7f98a8398740 1 --2- 172.21.15.67:0/3407383201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4352711b0 0x55b435624dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.899+0000 7f98a8398740 1 --2- 172.21.15.67:0/3407383201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4352504d0 0x55b435624890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.899+0000 7f98a8398740 1 --2- 172.21.15.67:0/3407383201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4352482f0 0x55b435627ca0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.899+0000 7f98a8398740 1 -- 172.21.15.67:0/3407383201 >> 172.21.15.67:0/3407383201 conn(0x55b435265300 msgr2=0x55b435615830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.899+0000 7f987eff5640 0 client.29021 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:09:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.899+0000 7f98a8398740 1 -- 172.21.15.67:0/3407383201 shutdown_connections 2024-07-31T15:09:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.900+0000 7f98a8398740 1 -- 172.21.15.67:0/3407383201 wait complete. 2024-07-31T15:09:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.900+0000 7f98a8398740 20 client.29021 trim_cache size 0 max 16384 2024-07-31T15:09:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:09:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:09:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:09:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:09:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.910+0000 7f98a13c8640 1 -- 172.21.15.67:0/70866266 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98840b3e80 msgr2=0x7f98840b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.910+0000 7f98a13c8640 1 --2- 172.21.15.67:0/70866266 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98840b3e80 0x7f98840b6390 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f988c0058c0 tx=0x7f988c004730 comp rx=0 tx=0).stop 2024-07-31T15:09:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.910+0000 7f98a13c8640 1 -- 172.21.15.67:0/70866266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f989c115f40 msgr2=0x7f989c113a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:09:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.910+0000 7f98a13c8640 1 --2- 172.21.15.67:0/70866266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f989c115f40 0x7f989c113a30 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f9894005e30 tx=0x7f98940319f0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.911+0000 7f98a13c8640 1 -- 172.21.15.67:0/70866266 shutdown_connections 2024-07-31T15:09:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.911+0000 7f98a13c8640 1 --2- 172.21.15.67:0/70866266 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f98840b3e80 0x7f98840b6390 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.911+0000 7f98a13c8640 1 --2- 172.21.15.67:0/70866266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f989c115f40 0x7f989c113a30 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.911+0000 7f98a13c8640 1 --2- 172.21.15.67:0/70866266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f989c10a740 0x7f989c1134f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.911+0000 7f98a13c8640 1 --2- 172.21.15.67:0/70866266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c109df0 0x7f989c112fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:09:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.911+0000 7f98a13c8640 1 -- 172.21.15.67:0/70866266 >> 172.21.15.67:0/70866266 conn(0x7f989c06e350 msgr2=0x7f989c11c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:09:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.911+0000 7f98a13c8640 1 -- 172.21.15.67:0/70866266 shutdown_connections 2024-07-31T15:09:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:09:59.911+0000 7f98a13c8640 1 -- 172.21.15.67:0/70866266 wait complete. 2024-07-31T15:10:00.384 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/mesh.o 2024-07-31T15:10:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:10:00.640 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/scsi/scsi_sysfs.o 2024-07-31T15:10:00.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:00 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:10:00.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:00 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:10:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:00 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:10:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:00 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:10:00.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:00 smithi073 ceph-mon[36714]: overall HEALTH_OK 2024-07-31T15:10:00.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:00 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:10:00.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:00 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:10:00.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:00 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:10:00.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:00 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:10:00.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:00 smithi067 ceph-mon[32750]: overall HEALTH_OK 2024-07-31T15:10:00.728 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_irq.o 2024-07-31T15:10:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:00 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:10:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:00 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:10:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:00 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:10:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:00 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:10:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:00 smithi077 ceph-mon[37327]: overall HEALTH_OK 2024-07-31T15:10:01.050 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/reassembly.o 2024-07-31T15:10:01.325 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/ap.o 2024-07-31T15:10:01.346 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/addr.o 2024-07-31T15:10:01.350 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/wep.o 2024-07-31T15:10:01.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:01 smithi067 ceph-mon[32750]: pgmap v1002: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 213 KiB/s wr, 1.78k op/s 2024-07-31T15:10:01.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:01 smithi073 ceph-mon[36714]: pgmap v1002: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 213 KiB/s wr, 1.78k op/s 2024-07-31T15:10:01.653 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/fib_frontend.o 2024-07-31T15:10:01.734 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/scsi/built-in.a 2024-07-31T15:10:01.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:01 smithi077 ceph-mon[37327]: pgmap v1002: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 213 KiB/s wr, 1.78k op/s 2024-07-31T15:10:01.750 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_mcr.o 2024-07-31T15:10:01.810 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/libata-core.o 2024-07-31T15:10:01.973 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/trace.o 2024-07-31T15:10:02.293 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/tcp_ipv6.o 2024-07-31T15:10:02.348 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/aead_api.o 2024-07-31T15:10:02.350 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/rpcb_clnt.o 2024-07-31T15:10:02.728 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_pm.o 2024-07-31T15:10:02.757 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/ping.o 2024-07-31T15:10:02.895 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/wpa.o 2024-07-31T15:10:03.147 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/fib_semantics.o 2024-07-31T15:10:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.457+0000 7f8f40a9f640 1 -- 172.21.15.67:0/2795385780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f3c108d80 msgr2=0x7f8f3c109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.457+0000 7f8f40a9f640 1 --2- 172.21.15.67:0/2795385780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f3c108d80 0x7f8f3c109190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009900 tx=0x7f8f3002a440 comp rx=0 tx=0).stop 2024-07-31T15:10:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.464+0000 7f8f40a9f640 1 -- 172.21.15.67:0/2795385780 shutdown_connections 2024-07-31T15:10:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.464+0000 7f8f40a9f640 1 --2- 172.21.15.67:0/2795385780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f3c1096d0 0x7f8f3c10ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.464+0000 7f8f40a9f640 1 --2- 172.21.15.67:0/2795385780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f3c108d80 0x7f8f3c109190 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.464+0000 7f8f40a9f640 1 --2- 172.21.15.67:0/2795385780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f3c103440 0x7f8f3c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.464+0000 7f8f40a9f640 1 -- 172.21.15.67:0/2795385780 >> 172.21.15.67:0/2795385780 conn(0x7f8f3c0feed0 msgr2=0x7f8f3c101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.464+0000 7f8f40a9f640 1 -- 172.21.15.67:0/2795385780 shutdown_connections 2024-07-31T15:10:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.464+0000 7f8f40a9f640 1 -- 172.21.15.67:0/2795385780 wait complete. 2024-07-31T15:10:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.465+0000 7f8f40a9f640 1 Processor -- start 2024-07-31T15:10:03.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.466+0000 7f8f40a9f640 1 -- start start 2024-07-31T15:10:03.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.466+0000 7f8f40a9f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f3c103440 0x7f8f3c118ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.467+0000 7f8f40a9f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f3c108d80 0x7f8f3c112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.467+0000 7f8f40a9f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f3c1096d0 0x7f8f3c1132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.467+0000 7f8f40a9f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c111290 con 0x7f8f3c108d80 2024-07-31T15:10:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.467+0000 7f8f40a9f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c111110 con 0x7f8f3c1096d0 2024-07-31T15:10:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.468+0000 7f8f40a9f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c111410 con 0x7f8f3c103440 2024-07-31T15:10:03.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.468+0000 7f8f39d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f3c108d80 0x7f8f3c112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:03.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.468+0000 7f8f39d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f3c108d80 0x7f8f3c112d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57906/0 (socket says 172.21.15.67:57906) 2024-07-31T15:10:03.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.468+0000 7f8f39d9b640 1 -- 172.21.15.67:0/2683761725 learned_addr learned my addr 172.21.15.67:0/2683761725 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.469+0000 7f8f39d9b640 1 -- 172.21.15.67:0/2683761725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f3c103440 msgr2=0x7f8f3c118ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:10:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.469+0000 7f8f39d9b640 1 --2- 172.21.15.67:0/2683761725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f3c103440 0x7f8f3c118ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.469+0000 7f8f39d9b640 1 -- 172.21.15.67:0/2683761725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f3c1096d0 msgr2=0x7f8f3c1132c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.469+0000 7f8f39d9b640 1 --2- 172.21.15.67:0/2683761725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f3c1096d0 0x7f8f3c1132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.470+0000 7f8f39d9b640 1 -- 172.21.15.67:0/2683761725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c1139f0 con 0x7f8f3c108d80 2024-07-31T15:10:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.470+0000 7f8f39d9b640 1 --2- 172.21.15.67:0/2683761725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f3c108d80 0x7f8f3c112d80 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f8f3002a9b0 tx=0x7f8f30003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.471+0000 7f8f2b7fe640 1 -- 172.21.15.67:0/2683761725 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3003f070 con 0x7f8f3c108d80 2024-07-31T15:10:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.472+0000 7f8f2b7fe640 1 -- 172.21.15.67:0/2683761725 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f3003bd20 con 0x7f8f3c108d80 2024-07-31T15:10:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.472+0000 7f8f2b7fe640 1 -- 172.21.15.67:0/2683761725 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30035030 con 0x7f8f3c108d80 2024-07-31T15:10:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.473+0000 7f8f40a9f640 1 -- 172.21.15.67:0/2683761725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3c1b50b0 con 0x7f8f3c108d80 2024-07-31T15:10:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.473+0000 7f8f40a9f640 1 -- 172.21.15.67:0/2683761725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3c1b54d0 con 0x7f8f3c108d80 2024-07-31T15:10:03.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.479+0000 7f8f2b7fe640 1 -- 172.21.15.67:0/2683761725 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8f30042070 con 0x7f8f3c108d80 2024-07-31T15:10:03.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.480+0000 7f8f2b7fe640 1 --2- 172.21.15.67:0/2683761725 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8f240b3f40 0x7f8f240b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.481+0000 7f8f2b7fe640 1 -- 172.21.15.67:0/2683761725 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f8f30104b00 con 0x7f8f3c108d80 2024-07-31T15:10:03.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.487+0000 7f8f46ff0740 1 Processor -- start 2024-07-31T15:10:03.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.482+0000 7f8f3a59c640 1 --2- 172.21.15.67:0/2683761725 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8f240b3f40 0x7f8f240b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:03.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.494+0000 7f8f3a59c640 1 --2- 172.21.15.67:0/2683761725 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8f240b3f40 0x7f8f240b6450 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c118810 tx=0x7f8f34009250 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:03.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.495+0000 7f8f46ff0740 1 -- start start 2024-07-31T15:10:03.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.495+0000 7f8f46ff0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbba4840 0x55d4bbf4aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.495+0000 7f8f46ff0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb7b980 0x55d4bbf4af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.495+0000 7f8f46ff0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb83b60 0x55d4bbf4b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.495+0000 7f8f46ff0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d4bbebf780 con 0x55d4bbba4840 2024-07-31T15:10:03.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.495+0000 7f8f46ff0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d4bbeaec80 con 0x55d4bbb83b60 2024-07-31T15:10:03.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.495+0000 7f8f46ff0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d4bbeb2cc0 con 0x55d4bbb7b980 2024-07-31T15:10:03.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.496+0000 7f8f3a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbba4840 0x55d4bbf4aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:03.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.496+0000 7f8f3a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbba4840 0x55d4bbf4aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57908/0 (socket says 172.21.15.67:57908) 2024-07-31T15:10:03.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.496+0000 7f8f3a59c640 1 -- 172.21.15.67:0/44140302 learned_addr learned my addr 172.21.15.67:0/44140302 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:03.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.496+0000 7f8f3ad9d640 1 --2- 172.21.15.67:0/44140302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb83b60 0x55d4bbf4b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:03.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.496+0000 7f8f39d9b640 1 --2- 172.21.15.67:0/44140302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb7b980 0x55d4bbf4af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.497+0000 7f8f3a59c640 1 -- 172.21.15.67:0/44140302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb7b980 msgr2=0x55d4bbf4af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.497+0000 7f8f3a59c640 1 --2- 172.21.15.67:0/44140302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb7b980 0x55d4bbf4af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.497+0000 7f8f3a59c640 1 -- 172.21.15.67:0/44140302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb83b60 msgr2=0x55d4bbf4b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.497+0000 7f8f3a59c640 1 --2- 172.21.15.67:0/44140302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb83b60 0x55d4bbf4b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.497+0000 7f8f3a59c640 1 -- 172.21.15.67:0/44140302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4bbe44080 con 0x55d4bbba4840 2024-07-31T15:10:03.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.497+0000 7f8f3ad9d640 1 --2- 172.21.15.67:0/44140302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb83b60 0x55d4bbf4b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:10:03.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.501+0000 7f8f3a59c640 1 --2- 172.21.15.67:0/44140302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbba4840 0x55d4bbf4aa30 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f8f34018710 tx=0x7f8f3400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a550ec50b2543788 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.505+0000 7f8f28ff9640 1 -- 172.21.15.67:0/44140302 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34019620 con 0x55d4bbba4840 2024-07-31T15:10:03.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.505+0000 7f8f28ff9640 1 -- 172.21.15.67:0/44140302 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f34018e40 con 0x55d4bbba4840 2024-07-31T15:10:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.506+0000 7f8f46ff0740 1 -- 172.21.15.67:0/44140302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbba4840 msgr2=0x55d4bbf4aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.506+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/44140302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbba4840 0x55d4bbf4aa30 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f8f34018710 tx=0x7f8f3400f000 comp rx=0 tx=0).stop 2024-07-31T15:10:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.507+0000 7f8f46ff0740 1 -- 172.21.15.67:0/44140302 shutdown_connections 2024-07-31T15:10:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.507+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/44140302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb83b60 0x55d4bbf4b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.507+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/44140302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb7b980 0x55d4bbf4af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.507+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/44140302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbba4840 0x55d4bbf4aa30 secure :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f8f34018710 tx=0x7f8f3400f000 comp rx=0 tx=0).stop 2024-07-31T15:10:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.507+0000 7f8f46ff0740 1 -- 172.21.15.67:0/44140302 >> 172.21.15.67:0/44140302 conn(0x55d4bbb98990 msgr2=0x55d4bbf48920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:03.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.513+0000 7f8f46ff0740 1 -- 172.21.15.67:0/44140302 shutdown_connections 2024-07-31T15:10:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.517+0000 7f8f46ff0740 1 -- 172.21.15.67:0/44140302 wait complete. 2024-07-31T15:10:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.517+0000 7f8f46ff0740 1 -- start start 2024-07-31T15:10:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.518+0000 7f8f46ff0740 1 Processor -- start 2024-07-31T15:10:03.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.519+0000 7f8f46ff0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb7b980 0x55d4bbf55ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.519+0000 7f8f46ff0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb83b60 0x55d4bbf56530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.519+0000 7f8f46ff0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb631c0 0x55d4bbf56a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.519+0000 7f8f46ff0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d4bbd127a0 con 0x55d4bbb83b60 2024-07-31T15:10:03.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.519+0000 7f8f46ff0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d4bbe512f0 con 0x55d4bbb7b980 2024-07-31T15:10:03.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.519+0000 7f8f46ff0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d4bbf59260 con 0x55d4bbb631c0 2024-07-31T15:10:03.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.519+0000 7f8f46ff0740 10 client.0 _ll_register_callbacks cb 0x55d4bbe5b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.523+0000 7f8f3a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb7b980 0x55d4bbf55ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.523+0000 7f8f3a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb7b980 0x55d4bbf55ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59300/0 (socket says 172.21.15.67:59300) 2024-07-31T15:10:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.523+0000 7f8f3a59c640 1 -- 172.21.15.67:0/674413319 learned_addr learned my addr 172.21.15.67:0/674413319 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.523+0000 7f8f39d9b640 1 --2- 172.21.15.67:0/674413319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb83b60 0x55d4bbf56530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.523+0000 7f8f3a59c640 1 -- 172.21.15.67:0/674413319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb631c0 msgr2=0x55d4bbf56a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:10:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.523+0000 7f8f3a59c640 1 --2- 172.21.15.67:0/674413319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb631c0 0x55d4bbf56a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.523+0000 7f8f3a59c640 1 -- 172.21.15.67:0/674413319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb83b60 msgr2=0x55d4bbf56530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.523+0000 7f8f3a59c640 1 --2- 172.21.15.67:0/674413319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb83b60 0x55d4bbf56530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.523+0000 7f8f3a59c640 1 -- 172.21.15.67:0/674413319 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4bbe55480 con 0x55d4bbb7b980 2024-07-31T15:10:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.524+0000 7f8f3a59c640 1 --2- 172.21.15.67:0/674413319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb7b980 0x55d4bbf55ff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8f34016770 tx=0x7f8f3401aba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca6250e8cfee9cb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.524+0000 7f8f21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:10:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.528+0000 7f8f21ffb640 1 -- 172.21.15.67:0/674413319 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3402f020 con 0x55d4bbb7b980 2024-07-31T15:10:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.528+0000 7f8f46ff0740 1 -- 172.21.15.67:0/674413319 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d4bbec16d0 con 0x55d4bbb7b980 2024-07-31T15:10:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.528+0000 7f8f21ffb640 1 -- 172.21.15.67:0/674413319 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f8f3402b600 con 0x55d4bbb7b980 2024-07-31T15:10:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.533+0000 7f8f21ffb640 1 -- 172.21.15.67:0/674413319 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f8f3402b800 con 0x55d4bbb7b980 2024-07-31T15:10:03.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.537+0000 7f8f46ff0740 10 client.29024 fetch_fsmap learned FSMap version 65 2024-07-31T15:10:03.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.537+0000 7f8f46ff0740 1 -- 172.21.15.67:0/674413319 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55d4bbc1eb90 con 0x55d4bbb7b980 2024-07-31T15:10:03.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.541+0000 7f8f21ffb640 1 -- 172.21.15.67:0/674413319 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d4bbec16d0 con 0x55d4bbb7b980 2024-07-31T15:10:03.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.541+0000 7f8f21ffb640 1 -- 172.21.15.67:0/674413319 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f8f34021530 con 0x55d4bbb7b980 2024-07-31T15:10:03.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.541+0000 7f8f46ff0740 10 client.29024 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:10:03.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.541+0000 7f8f46ff0740 10 client.29024 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:10:03.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.541+0000 7f8f46ff0740 20 client.29024 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:03.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.541+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/674413319 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d4bbb38570 0x55d4bbf684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.541+0000 7f8f46ff0740 4 client.29024 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:10:03.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.541+0000 7f8f46ff0740 1 -- 172.21.15.67:0/674413319 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4bbebd170 con 0x55d4bbb38570 2024-07-31T15:10:03.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.543+0000 7f8f39d9b640 1 --2- 172.21.15.67:0/674413319 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d4bbb38570 0x55d4bbf684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:03.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.543+0000 7f8f21ffb640 10 client.29024 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:03.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.543+0000 7f8f39d9b640 1 --2- 172.21.15.67:0/674413319 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d4bbb38570 0x55d4bbf684e0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56b43d724b8cba22 server_cookie=7f032015d05452e3 in_seq=0 out_seq=0 2024-07-31T15:10:03.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.544+0000 7f8f21ffb640 1 -- 172.21.15.67:0/674413319 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4bbebd170 con 0x55d4bbb38570 2024-07-31T15:10:03.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.544+0000 7f8f21ffb640 10 client.29024 handle_command_reply: tid=0 2024-07-31T15:10:03.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.545+0000 7f8f46ff0740 1 client.29024 shutdown 2024-07-31T15:10:03.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.545+0000 7f8f46ff0740 1 -- 172.21.15.67:0/674413319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb7b980 msgr2=0x55d4bbf55ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:03.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.545+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/674413319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb7b980 0x55d4bbf55ff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8f34016770 tx=0x7f8f3401aba0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.553+0000 7f8f46ff0740 1 -- 172.21.15.67:0/674413319 shutdown_connections 2024-07-31T15:10:03.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.553+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/674413319 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d4bbb38570 0x55d4bbf684e0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.553+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/674413319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb631c0 0x55d4bbf56a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.553+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/674413319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb83b60 0x55d4bbf56530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.553+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/674413319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb7b980 0x55d4bbf55ff0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.553+0000 7f8f46ff0740 1 -- 172.21.15.67:0/674413319 >> 172.21.15.67:0/674413319 conn(0x55d4bbb98990 msgr2=0x55d4bbf4dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:03.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.553+0000 7f8f21ffb640 0 client.29024 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:03.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.554+0000 7f8f46ff0740 1 -- 172.21.15.67:0/674413319 shutdown_connections 2024-07-31T15:10:03.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.555+0000 7f8f46ff0740 1 -- 172.21.15.67:0/674413319 wait complete. 2024-07-31T15:10:03.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.556+0000 7f8f46ff0740 20 client.29024 trim_cache size 0 max 16384 2024-07-31T15:10:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.595+0000 7f8f46ff0740 1 Processor -- start 2024-07-31T15:10:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.596+0000 7f8f46ff0740 1 -- start start 2024-07-31T15:10:03.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:03 smithi067 ceph-mon[32750]: pgmap v1003: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 385 KiB/s wr, 1.05k op/s 2024-07-31T15:10:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.596+0000 7f8f46ff0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb631c0 0x55d4bbf56700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.596+0000 7f8f46ff0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb7b980 0x55d4bbf6dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.596+0000 7f8f46ff0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb83b60 0x55d4bbf6e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.596+0000 7f8f46ff0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c00fef0 con 0x55d4bbb631c0 2024-07-31T15:10:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.596+0000 7f8f46ff0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c011a30 con 0x55d4bbb83b60 2024-07-31T15:10:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.596+0000 7f8f3ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb83b60 0x55d4bbf6e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.596+0000 7f8f3a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb631c0 0x55d4bbf56700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.596+0000 7f8f3ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb83b60 0x55d4bbf6e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59310/0 (socket says 172.21.15.67:59310) 2024-07-31T15:10:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.596+0000 7f8f3ad9d640 1 -- 172.21.15.67:0/3520935821 learned_addr learned my addr 172.21.15.67:0/3520935821 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.597+0000 7f8f46ff0740 1 -- 172.21.15.67:0/3520935821 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c0105c0 con 0x55d4bbb7b980 2024-07-31T15:10:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.598+0000 7f8f3a59c640 1 -- 172.21.15.67:0/3520935821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb7b980 msgr2=0x55d4bbf6dd40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:10:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.598+0000 7f8f3a59c640 1 --2- 172.21.15.67:0/3520935821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb7b980 0x55d4bbf6dd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.598+0000 7f8f3a59c640 1 -- 172.21.15.67:0/3520935821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb83b60 msgr2=0x55d4bbf6e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.598+0000 7f8f3a59c640 1 --2- 172.21.15.67:0/3520935821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb83b60 0x55d4bbf6e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.598+0000 7f8f3a59c640 1 -- 172.21.15.67:0/3520935821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4bbf49710 con 0x55d4bbb631c0 2024-07-31T15:10:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.598+0000 7f8f3a59c640 1 --2- 172.21.15.67:0/3520935821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb631c0 0x55d4bbf56700 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f8f34021a70 tx=0x7f8f340092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30e743427a7fe7b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.608+0000 7f8f23fff640 1 -- 172.21.15.67:0/3520935821 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34040020 con 0x55d4bbb631c0 2024-07-31T15:10:03.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.608+0000 7f8f23fff640 1 -- 172.21.15.67:0/3520935821 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f34021af0 con 0x55d4bbb631c0 2024-07-31T15:10:03.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.608+0000 7f8f23fff640 1 -- 172.21.15.67:0/3520935821 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3402f070 con 0x55d4bbb631c0 2024-07-31T15:10:03.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.610+0000 7f8f46ff0740 1 -- 172.21.15.67:0/3520935821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb631c0 msgr2=0x55d4bbf56700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:03.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.610+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/3520935821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb631c0 0x55d4bbf56700 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f8f34021a70 tx=0x7f8f340092e0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.619+0000 7f8f46ff0740 1 -- 172.21.15.67:0/3520935821 shutdown_connections 2024-07-31T15:10:03.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.619+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/3520935821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb83b60 0x55d4bbf6e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.619+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/3520935821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb7b980 0x55d4bbf6dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.619+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/3520935821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb631c0 0x55d4bbf56700 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.619+0000 7f8f46ff0740 1 -- 172.21.15.67:0/3520935821 >> 172.21.15.67:0/3520935821 conn(0x55d4bbb98990 msgr2=0x55d4bbf55ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:03.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.626+0000 7f8f46ff0740 1 -- 172.21.15.67:0/3520935821 shutdown_connections 2024-07-31T15:10:03.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.628+0000 7f8f46ff0740 1 -- 172.21.15.67:0/3520935821 wait complete. 2024-07-31T15:10:03.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.628+0000 7f8f46ff0740 1 -- start start 2024-07-31T15:10:03.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.629+0000 7f8f46ff0740 1 Processor -- start 2024-07-31T15:10:03.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.634+0000 7f8f46ff0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb631c0 0x55d4bbf5e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.635+0000 7f8f46ff0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb7b980 0x55d4bbf5bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.635+0000 7f8f3a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb631c0 0x55d4bbf5e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:03.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.635+0000 7f8f3a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb631c0 0x55d4bbf5e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59324/0 (socket says 172.21.15.67:59324) 2024-07-31T15:10:03.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.635+0000 7f8f3a59c640 1 -- 172.21.15.67:0/2979579818 learned_addr learned my addr 172.21.15.67:0/2979579818 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:03.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.635+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb83b60 0x55d4bbf5c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.635+0000 7f8f39d9b640 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb7b980 0x55d4bbf5bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:03.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.636+0000 7f8f46ff0740 1 -- 172.21.15.67:0/2979579818 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c011360 con 0x55d4bbb83b60 2024-07-31T15:10:03.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.636+0000 7f8f46ff0740 1 -- 172.21.15.67:0/2979579818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c010c90 con 0x55d4bbb631c0 2024-07-31T15:10:03.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.636+0000 7f8f46ff0740 1 -- 172.21.15.67:0/2979579818 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c00def0 con 0x55d4bbb7b980 2024-07-31T15:10:03.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.636+0000 7f8f3ad9d640 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb83b60 0x55d4bbf5c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:03.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.636+0000 7f8f46ff0740 10 client.0 _ll_register_callbacks cb 0x7f8f3401a830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:03.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.636+0000 7f8f3a59c640 1 -- 172.21.15.67:0/2979579818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb7b980 msgr2=0x55d4bbf5bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:03.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.636+0000 7f8f3a59c640 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb7b980 0x55d4bbf5bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.636+0000 7f8f3a59c640 1 -- 172.21.15.67:0/2979579818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb83b60 msgr2=0x55d4bbf5c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:03.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.636+0000 7f8f3a59c640 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb83b60 0x55d4bbf5c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:03.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.636+0000 7f8f3a59c640 1 -- 172.21.15.67:0/2979579818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4bbf5c950 con 0x55d4bbb631c0 2024-07-31T15:10:03.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.637+0000 7f8f46ff0740 1 -- 172.21.15.67:0/2979579818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f34021af0 con 0x55d4bbb631c0 2024-07-31T15:10:03.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.637+0000 7f8f3a59c640 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb631c0 0x55d4bbf5e9b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400a550 tx=0x7f8f34016610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17dd77793a280fc1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:03.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.637+0000 7f8f3ad9d640 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb83b60 0x55d4bbf5c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:10:03.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.637+0000 7f8f21ffb640 10 client.29030 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:10:03.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.637+0000 7f8f21ffb640 1 -- 172.21.15.67:0/2979579818 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34040030 con 0x55d4bbb631c0 2024-07-31T15:10:03.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.637+0000 7f8f21ffb640 1 -- 172.21.15.67:0/2979579818 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9351+0+0 (secure 0 0 0) 0x7f8f340215f0 con 0x55d4bbb631c0 2024-07-31T15:10:03.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.638+0000 7f8f21ffb640 1 -- 172.21.15.67:0/2979579818 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f8f3402f070 con 0x55d4bbb631c0 2024-07-31T15:10:03.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:03 smithi073 ceph-mon[36714]: pgmap v1003: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 385 KiB/s wr, 1.05k op/s 2024-07-31T15:10:03.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.639+0000 7f8f46ff0740 10 client.29030 fetch_fsmap learned FSMap version 65 2024-07-31T15:10:03.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.639+0000 7f8f46ff0740 1 -- 172.21.15.67:0/2979579818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=65}) -- 0x55d4bbfd0840 con 0x55d4bbb631c0 2024-07-31T15:10:03.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.646+0000 7f8f21ffb640 1 -- 172.21.15.67:0/2979579818 <== mon.2 v2:172.21.15.73:3300/0 4 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f8f3402b9a0 con 0x55d4bbb631c0 2024-07-31T15:10:03.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.647+0000 7f8f46ff0740 10 client.29030 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:10:03.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.647+0000 7f8f46ff0740 10 client.29030 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:10:03.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.647+0000 7f8f46ff0740 20 client.29030 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:03.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.647+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d4bbfd0e40 0x55d4bbfd3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:03.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.647+0000 7f8f46ff0740 4 client.29030 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.1, "await_for": 45.0}] 2024-07-31T15:10:03.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.647+0000 7f8f46ff0740 1 -- 172.21.15.67:0/2979579818 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.1, "await_for": 45.0}) -- 0x7f8f34021af0 con 0x55d4bbfd0e40 2024-07-31T15:10:03.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.651+0000 7f8f39d9b640 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d4bbfd0e40 0x55d4bbfd3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:03.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.664+0000 7f8f39d9b640 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d4bbfd0e40 0x55d4bbfd3260 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3adc4a193d99378c server_cookie=dfe4d5035129d3a9 in_seq=0 out_seq=0 2024-07-31T15:10:03.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:03.665+0000 7f8f21ffb640 10 client.29030 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:03.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:03 smithi077 ceph-mon[37327]: pgmap v1003: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 385 KiB/s wr, 1.05k op/s 2024-07-31T15:10:05.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:05 smithi073 ceph-mon[36714]: pgmap v1004: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 485 KiB/s wr, 313 op/s 2024-07-31T15:10:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:05 smithi073 ceph-mon[36714]: osdmap e125: 12 total, 12 up, 12 in 2024-07-31T15:10:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:05 smithi077 ceph-mon[37327]: pgmap v1004: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 485 KiB/s wr, 313 op/s 2024-07-31T15:10:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:05 smithi077 ceph-mon[37327]: osdmap e125: 12 total, 12 up, 12 in 2024-07-31T15:10:05.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:05 smithi067 ceph-mon[32750]: pgmap v1004: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 485 KiB/s wr, 313 op/s 2024-07-31T15:10:05.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:05 smithi067 ceph-mon[32750]: osdmap e125: 12 total, 12 up, 12 in 2024-07-31T15:10:07.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:07 smithi073 ceph-mon[36714]: pgmap v1006: 129 pgs: 7 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 102 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 668 KiB/s wr, 122 op/s 2024-07-31T15:10:07.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:07 smithi077 ceph-mon[37327]: pgmap v1006: 129 pgs: 7 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 102 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 668 KiB/s wr, 122 op/s 2024-07-31T15:10:07.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:07 smithi067 ceph-mon[32750]: pgmap v1006: 129 pgs: 7 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 102 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 668 KiB/s wr, 122 op/s 2024-07-31T15:10:09.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:09 smithi073 ceph-mon[36714]: pgmap v1007: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 684 KiB/s wr, 122 op/s 2024-07-31T15:10:09.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:09 smithi077 ceph-mon[37327]: pgmap v1007: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 684 KiB/s wr, 122 op/s 2024-07-31T15:10:09.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:09 smithi067 ceph-mon[32750]: pgmap v1007: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 684 KiB/s wr, 122 op/s 2024-07-31T15:10:11.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:11 smithi073 ceph-mon[36714]: pgmap v1008: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 723 KiB/s wr, 118 op/s 2024-07-31T15:10:11.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:11 smithi077 ceph-mon[37327]: pgmap v1008: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 723 KiB/s wr, 118 op/s 2024-07-31T15:10:11.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:11 smithi067 ceph-mon[32750]: pgmap v1008: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 723 KiB/s wr, 118 op/s 2024-07-31T15:10:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.057+0000 7fdebb9bb640 1 -- 172.21.15.67:0/694581981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdeb4102bb0 msgr2=0x7fdeb4102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.057+0000 7fdebb9bb640 1 --2- 172.21.15.67:0/694581981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdeb4102bb0 0x7fdeb4102fc0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009900 tx=0x7fdeb002a410 comp rx=0 tx=0).stop 2024-07-31T15:10:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.057+0000 7fdebb9bb640 1 -- 172.21.15.67:0/694581981 shutdown_connections 2024-07-31T15:10:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.057+0000 7fdebb9bb640 1 --2- 172.21.15.67:0/694581981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdeb4105460 0x7fdeb410eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.057+0000 7fdebb9bb640 1 --2- 172.21.15.67:0/694581981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeb4104b10 0x7fdeb4104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.057+0000 7fdebb9bb640 1 --2- 172.21.15.67:0/694581981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdeb4102bb0 0x7fdeb4102fc0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.057+0000 7fdebb9bb640 1 -- 172.21.15.67:0/694581981 >> 172.21.15.67:0/694581981 conn(0x7fdeb40798d0 msgr2=0x7fdeb4079ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.057+0000 7fdebb9bb640 1 -- 172.21.15.67:0/694581981 shutdown_connections 2024-07-31T15:10:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.057+0000 7fdebb9bb640 1 -- 172.21.15.67:0/694581981 wait complete. 2024-07-31T15:10:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdebb9bb640 1 Processor -- start 2024-07-31T15:10:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdebb9bb640 1 -- start start 2024-07-31T15:10:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdebb9bb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeb4102bb0 0x7fdeb4199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdebb9bb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdeb4104b10 0x7fdeb419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdebb9bb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdeb4105460 0x7fdeb419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdebb9bb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb4112350 con 0x7fdeb4104b10 2024-07-31T15:10:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdebb9bb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41121d0 con 0x7fdeb4102bb0 2024-07-31T15:10:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdebb9bb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41124d0 con 0x7fdeb4105460 2024-07-31T15:10:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdeb8f56640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdeb4104b10 0x7fdeb419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdeb8f56640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdeb4104b10 0x7fdeb419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35522/0 (socket says 172.21.15.67:35522) 2024-07-31T15:10:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdeb8f56640 1 -- 172.21.15.67:0/2743525215 learned_addr learned my addr 172.21.15.67:0/2743525215 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdeb9757640 1 --2- 172.21.15.67:0/2743525215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeb4102bb0 0x7fdeb4199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdeb9f58640 1 --2- 172.21.15.67:0/2743525215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdeb4105460 0x7fdeb419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdeb8f56640 1 -- 172.21.15.67:0/2743525215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdeb4105460 msgr2=0x7fdeb419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdeb8f56640 1 --2- 172.21.15.67:0/2743525215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdeb4105460 0x7fdeb419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdeb8f56640 1 -- 172.21.15.67:0/2743525215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeb4102bb0 msgr2=0x7fdeb4199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdeb8f56640 1 --2- 172.21.15.67:0/2743525215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeb4102bb0 0x7fdeb4199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.058+0000 7fdeb8f56640 1 -- 172.21.15.67:0/2743525215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb419fdf0 con 0x7fdeb4104b10 2024-07-31T15:10:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.059+0000 7fdeb8f56640 1 --2- 172.21.15.67:0/2743525215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdeb4104b10 0x7fdeb419a460 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fdea400e9f0 tx=0x7fdea400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.059+0000 7fdeaa7fc640 1 -- 172.21.15.67:0/2743525215 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea400ccc0 con 0x7fdeb4104b10 2024-07-31T15:10:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.059+0000 7fdeaa7fc640 1 -- 172.21.15.67:0/2743525215 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea40108e0 con 0x7fdeb4104b10 2024-07-31T15:10:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.059+0000 7fdebb9bb640 1 -- 172.21.15.67:0/2743525215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb41a00e0 con 0x7fdeb4104b10 2024-07-31T15:10:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.059+0000 7fdeaa7fc640 1 -- 172.21.15.67:0/2743525215 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4018860 con 0x7fdeb4104b10 2024-07-31T15:10:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.059+0000 7fdebb9bb640 1 -- 172.21.15.67:0/2743525215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb41a6870 con 0x7fdeb4104b10 2024-07-31T15:10:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.062+0000 7fdec2992740 1 Processor -- start 2024-07-31T15:10:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.062+0000 7fdec2992740 1 -- start start 2024-07-31T15:10:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.062+0000 7fdeaa7fc640 1 -- 172.21.15.67:0/2743525215 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fdea4010430 con 0x7fdeb4104b10 2024-07-31T15:10:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.062+0000 7fdec2992740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f2b1b0 0x55b4072cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.062+0000 7fdec2992740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f022f0 0x55b4072ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.062+0000 7fdec2992740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f0a4d0 0x55b4072ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.062+0000 7fdec2992740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b4072433b0 con 0x55b406f2b1b0 2024-07-31T15:10:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.062+0000 7fdec2992740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b4072328b0 con 0x55b406f0a4d0 2024-07-31T15:10:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.062+0000 7fdec2992740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b4072368f0 con 0x55b406f022f0 2024-07-31T15:10:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.062+0000 7fdeb9f58640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f2b1b0 0x55b4072cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.062+0000 7fdeb8f56640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f0a4d0 0x55b4072ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.062+0000 7fdeb9f58640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f2b1b0 0x55b4072cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35528/0 (socket says 172.21.15.67:35528) 2024-07-31T15:10:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.062+0000 7fdeb9f58640 1 -- 172.21.15.67:0/2005235609 learned_addr learned my addr 172.21.15.67:0/2005235609 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.063+0000 7fdeb9757640 1 --2- 172.21.15.67:0/2005235609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f022f0 0x55b4072ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.063+0000 7fdeb8f56640 1 -- 172.21.15.67:0/2005235609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f022f0 msgr2=0x55b4072ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.063+0000 7fdeb8f56640 1 --2- 172.21.15.67:0/2005235609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f022f0 0x55b4072ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.063+0000 7fdeb8f56640 1 -- 172.21.15.67:0/2005235609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f2b1b0 msgr2=0x55b4072cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.063+0000 7fdeb8f56640 1 --2- 172.21.15.67:0/2005235609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f2b1b0 0x55b4072cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.063+0000 7fdeb8f56640 1 -- 172.21.15.67:0/2005235609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4071d99a0 con 0x55b406f0a4d0 2024-07-31T15:10:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.063+0000 7fdeb9f58640 1 --2- 172.21.15.67:0/2005235609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f2b1b0 0x55b4072cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.063+0000 7fdeb8f56640 1 --2- 172.21.15.67:0/2005235609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f0a4d0 0x55b4072ce760 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdea40e1a90 tx=0x7fdea40ec690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63381cf9650d78fe server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.063+0000 7fdeaa7fc640 1 --2- 172.21.15.67:0/2743525215 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdea00b3d40 0x7fdea00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.063+0000 7fde9f7fe640 1 -- 172.21.15.67:0/2005235609 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea40f3070 con 0x55b406f0a4d0 2024-07-31T15:10:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.063+0000 7fde9f7fe640 1 -- 172.21.15.67:0/2005235609 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea40aaa60 con 0x55b406f0a4d0 2024-07-31T15:10:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.063+0000 7fdeb9f58640 1 --2- 172.21.15.67:0/2743525215 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdea00b3d40 0x7fdea00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.063+0000 7fde9f7fe640 1 -- 172.21.15.67:0/2005235609 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea406ced0 con 0x55b406f0a4d0 2024-07-31T15:10:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.064+0000 7fdeaa7fc640 1 -- 172.21.15.67:0/2743525215 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7fdea40e4700 con 0x7fdeb4104b10 2024-07-31T15:10:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.064+0000 7fdec2992740 1 -- 172.21.15.67:0/2005235609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f0a4d0 msgr2=0x55b4072ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.064+0000 7fdec2992740 1 --2- 172.21.15.67:0/2005235609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f0a4d0 0x55b4072ce760 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdea40e1a90 tx=0x7fdea40ec690 comp rx=0 tx=0).stop 2024-07-31T15:10:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.064+0000 7fdeb9f58640 1 --2- 172.21.15.67:0/2743525215 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdea00b3d40 0x7fdea00b6250 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009fd0 tx=0x7fdeac0098a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.064+0000 7fdec2992740 1 -- 172.21.15.67:0/2005235609 shutdown_connections 2024-07-31T15:10:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.064+0000 7fdec2992740 1 --2- 172.21.15.67:0/2005235609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f0a4d0 0x55b4072ce760 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.064+0000 7fdec2992740 1 --2- 172.21.15.67:0/2005235609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f022f0 0x55b4072ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.064+0000 7fdec2992740 1 --2- 172.21.15.67:0/2005235609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f2b1b0 0x55b4072cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.064+0000 7fdec2992740 1 -- 172.21.15.67:0/2005235609 >> 172.21.15.67:0/2005235609 conn(0x55b406f1f300 msgr2=0x55b4072cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.064+0000 7fdec2992740 1 -- 172.21.15.67:0/2005235609 shutdown_connections 2024-07-31T15:10:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.065+0000 7fdec2992740 1 -- 172.21.15.67:0/2005235609 wait complete. 2024-07-31T15:10:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.065+0000 7fdec2992740 1 -- start start 2024-07-31T15:10:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.065+0000 7fdec2992740 1 Processor -- start 2024-07-31T15:10:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.065+0000 7fdec2992740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f022f0 0x55b4072dd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdec2992740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f0a4d0 0x55b4072ddf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdec2992740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f2b1b0 0x55b4072e9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdec2992740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b40716cab0 con 0x55b406f022f0 2024-07-31T15:10:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdec2992740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b4071bacf0 con 0x55b406f0a4d0 2024-07-31T15:10:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdec2992740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b4072e0840 con 0x55b406f2b1b0 2024-07-31T15:10:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdec2992740 10 client.0 _ll_register_callbacks cb 0x55b40722c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdeb9757640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f022f0 0x55b4072dd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdeb9757640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f022f0 0x55b4072dd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35532/0 (socket says 172.21.15.67:35532) 2024-07-31T15:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdeb9757640 1 -- 172.21.15.67:0/222780225 learned_addr learned my addr 172.21.15.67:0/222780225 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdeb8f56640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f0a4d0 0x55b4072ddf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdeb9f58640 1 --2- 172.21.15.67:0/222780225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f2b1b0 0x55b4072e9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdeb9757640 1 -- 172.21.15.67:0/222780225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f2b1b0 msgr2=0x55b4072e9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdeb9757640 1 --2- 172.21.15.67:0/222780225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f2b1b0 0x55b4072e9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdeb9757640 1 -- 172.21.15.67:0/222780225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f0a4d0 msgr2=0x55b4072ddf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdeb9757640 1 --2- 172.21.15.67:0/222780225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f0a4d0 0x55b4072ddf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdeb9757640 1 -- 172.21.15.67:0/222780225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b406f96790 con 0x55b406f022f0 2024-07-31T15:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdeb8f56640 1 --2- 172.21.15.67:0/222780225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f0a4d0 0x55b4072ddf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fdeb9757640 1 --2- 172.21.15.67:0/222780225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f022f0 0x55b4072dd990 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009900 tx=0x7fdeb00316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd50e1d65d1fcf0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fde9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.066+0000 7fde9cff9640 1 -- 172.21.15.67:0/222780225 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0035040 con 0x55b406f022f0 2024-07-31T15:10:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.067+0000 7fde9cff9640 1 -- 172.21.15.67:0/222780225 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7fdeb0004040 con 0x55b406f022f0 2024-07-31T15:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.067+0000 7fdec2992740 1 -- 172.21.15.67:0/222780225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea40aaa60 con 0x55b406f022f0 2024-07-31T15:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.067+0000 7fde9cff9640 1 -- 172.21.15.67:0/222780225 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fdeb004f050 con 0x55b406f022f0 2024-07-31T15:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.067+0000 7fdec2992740 10 client.19509 fetch_fsmap learned FSMap version 65 2024-07-31T15:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.067+0000 7fdec2992740 1 -- 172.21.15.67:0/222780225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55b40700f620 con 0x55b406f022f0 2024-07-31T15:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.067+0000 7fde9cff9640 1 -- 172.21.15.67:0/222780225 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea40aaa60 con 0x55b406f022f0 2024-07-31T15:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.067+0000 7fde9cff9640 1 -- 172.21.15.67:0/222780225 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fdeb0004750 con 0x55b406f022f0 2024-07-31T15:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.067+0000 7fdec2992740 10 client.19509 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.067+0000 7fdec2992740 10 client.19509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.067+0000 7fdec2992740 20 client.19509 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.067+0000 7fdec2992740 1 --2- 172.21.15.67:0/222780225 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b406ee9d80 0x55b4072efb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.067+0000 7fdec2992740 4 client.19509 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.067+0000 7fdec2992740 1 -- 172.21.15.67:0/222780225 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b407245300 con 0x55b406ee9d80 2024-07-31T15:10:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.068+0000 7fdeb8f56640 1 --2- 172.21.15.67:0/222780225 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b406ee9d80 0x55b4072efb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.068+0000 7fdeb8f56640 1 --2- 172.21.15.67:0/222780225 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b406ee9d80 0x55b4072efb90 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522366085b7332cf server_cookie=f24c51241798ab14 in_seq=0 out_seq=0 2024-07-31T15:10:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.068+0000 7fde9cff9640 10 client.19509 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:13.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:13 smithi073 ceph-mon[36714]: pgmap v1009: 129 pgs: 9 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 94 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 792 KiB/s wr, 84 op/s 2024-07-31T15:10:13.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:13 smithi077 ceph-mon[37327]: pgmap v1009: 129 pgs: 9 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 94 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 792 KiB/s wr, 84 op/s 2024-07-31T15:10:13.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:13 smithi067 ceph-mon[32750]: pgmap v1009: 129 pgs: 9 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 94 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 792 KiB/s wr, 84 op/s 2024-07-31T15:10:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.833+0000 7fde9cff9640 1 -- 172.21.15.67:0/222780225 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b407245300 con 0x55b406ee9d80 2024-07-31T15:10:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.833+0000 7fde9cff9640 10 client.19509 handle_command_reply: tid=0 2024-07-31T15:10:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.833+0000 7fdec2992740 1 client.19509 shutdown 2024-07-31T15:10:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.833+0000 7fdec2992740 1 -- 172.21.15.67:0/222780225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f022f0 msgr2=0x55b4072dd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.833+0000 7fdec2992740 1 --2- 172.21.15.67:0/222780225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f022f0 0x55b4072dd990 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009900 tx=0x7fdeb00316d0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.833+0000 7fdec2992740 1 -- 172.21.15.67:0/222780225 shutdown_connections 2024-07-31T15:10:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.833+0000 7fdec2992740 1 --2- 172.21.15.67:0/222780225 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b406ee9d80 0x55b4072efb90 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.833+0000 7fdec2992740 1 --2- 172.21.15.67:0/222780225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f2b1b0 0x55b4072e9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.833+0000 7fdec2992740 1 --2- 172.21.15.67:0/222780225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f0a4d0 0x55b4072ddf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.833+0000 7fdec2992740 1 --2- 172.21.15.67:0/222780225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f022f0 0x55b4072dd990 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.833+0000 7fde9cff9640 0 client.19509 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.833+0000 7fdec2992740 1 -- 172.21.15.67:0/222780225 >> 172.21.15.67:0/222780225 conn(0x55b406f1f300 msgr2=0x55b4072ca2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.834+0000 7fdec2992740 1 -- 172.21.15.67:0/222780225 shutdown_connections 2024-07-31T15:10:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.834+0000 7fdec2992740 1 -- 172.21.15.67:0/222780225 wait complete. 2024-07-31T15:10:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.834+0000 7fdec2992740 20 client.19509 trim_cache size 0 max 16384 2024-07-31T15:10:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.850+0000 7fdec2992740 1 Processor -- start 2024-07-31T15:10:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.851+0000 7fdec2992740 1 -- start start 2024-07-31T15:10:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.851+0000 7fdec2992740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f022f0 0x55b4072ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.851+0000 7fdec2992740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f0a4d0 0x55b4072e9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.851+0000 7fdec2992740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f2b1b0 0x55b4072e9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.851+0000 7fdec2992740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41ae6f0 con 0x55b406f0a4d0 2024-07-31T15:10:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.851+0000 7fdec2992740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41b0230 con 0x55b406f022f0 2024-07-31T15:10:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.851+0000 7fdec2992740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41aedc0 con 0x55b406f2b1b0 2024-07-31T15:10:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.851+0000 7fdeb8f56640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f0a4d0 0x55b4072e9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.851+0000 7fdeb8f56640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f0a4d0 0x55b4072e9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35538/0 (socket says 172.21.15.67:35538) 2024-07-31T15:10:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.851+0000 7fdeb9757640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f022f0 0x55b4072ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.851+0000 7fdeb8f56640 1 -- 172.21.15.67:0/2006182821 learned_addr learned my addr 172.21.15.67:0/2006182821 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.851+0000 7fdeb9f58640 1 --2- 172.21.15.67:0/2006182821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f2b1b0 0x55b4072e9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.852+0000 7fdeb9757640 1 -- 172.21.15.67:0/2006182821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f2b1b0 msgr2=0x55b4072e9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.852+0000 7fdeb9757640 1 --2- 172.21.15.67:0/2006182821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f2b1b0 0x55b4072e9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.852+0000 7fdeb9757640 1 -- 172.21.15.67:0/2006182821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f0a4d0 msgr2=0x55b4072e9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.852+0000 7fdeb9757640 1 --2- 172.21.15.67:0/2006182821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f0a4d0 0x55b4072e9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.852+0000 7fdeb9757640 1 -- 172.21.15.67:0/2006182821 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4072dc8d0 con 0x55b406f022f0 2024-07-31T15:10:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.852+0000 7fdeb8f56640 1 --2- 172.21.15.67:0/2006182821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f0a4d0 0x55b4072e9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.852+0000 7fdeb9757640 1 --2- 172.21.15.67:0/2006182821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f022f0 0x55b4072ca9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0004cc0 tx=0x7fdeb002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb98631fd5035d2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.852+0000 7fde9effd640 1 -- 172.21.15.67:0/2006182821 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0035030 con 0x55b406f022f0 2024-07-31T15:10:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.852+0000 7fde9effd640 1 -- 172.21.15.67:0/2006182821 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeb0031740 con 0x55b406f022f0 2024-07-31T15:10:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.852+0000 7fde9effd640 1 -- 172.21.15.67:0/2006182821 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0005570 con 0x55b406f022f0 2024-07-31T15:10:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.853+0000 7fdec2992740 1 -- 172.21.15.67:0/2006182821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f022f0 msgr2=0x55b4072ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.853+0000 7fdec2992740 1 --2- 172.21.15.67:0/2006182821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f022f0 0x55b4072ca9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0004cc0 tx=0x7fdeb002a410 comp rx=0 tx=0).stop 2024-07-31T15:10:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.853+0000 7fdec2992740 1 -- 172.21.15.67:0/2006182821 shutdown_connections 2024-07-31T15:10:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.853+0000 7fdec2992740 1 --2- 172.21.15.67:0/2006182821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f2b1b0 0x55b4072e9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.853+0000 7fdec2992740 1 --2- 172.21.15.67:0/2006182821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f0a4d0 0x55b4072e9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.853+0000 7fdec2992740 1 --2- 172.21.15.67:0/2006182821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f022f0 0x55b4072ca9c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.853+0000 7fdec2992740 1 -- 172.21.15.67:0/2006182821 >> 172.21.15.67:0/2006182821 conn(0x55b406f1f300 msgr2=0x55b4072ca2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.853+0000 7fdec2992740 1 -- 172.21.15.67:0/2006182821 shutdown_connections 2024-07-31T15:10:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.853+0000 7fdec2992740 1 -- 172.21.15.67:0/2006182821 wait complete. 2024-07-31T15:10:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.854+0000 7fdec2992740 1 -- start start 2024-07-31T15:10:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.854+0000 7fdec2992740 1 Processor -- start 2024-07-31T15:10:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.854+0000 7fdec2992740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f022f0 0x55b4072e2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.854+0000 7fdec2992740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f0a4d0 0x55b4072e3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.854+0000 7fdec2992740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f2b1b0 0x55b407356360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.854+0000 7fdec2992740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41afb60 con 0x55b406f022f0 2024-07-31T15:10:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.854+0000 7fdec2992740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41af490 con 0x55b406f2b1b0 2024-07-31T15:10:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.854+0000 7fdec2992740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41ac6f0 con 0x55b406f0a4d0 2024-07-31T15:10:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdec2992740 10 client.0 _ll_register_callbacks cb 0x7fdeb0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdeb9757640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f022f0 0x55b4072e2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdeb9f58640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f2b1b0 0x55b407356360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdeb9757640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f022f0 0x55b4072e2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35540/0 (socket says 172.21.15.67:35540) 2024-07-31T15:10:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdeb9f58640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f2b1b0 0x55b407356360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:50502/0 (socket says 172.21.15.67:50502) 2024-07-31T15:10:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdeb9757640 1 -- 172.21.15.67:0/1239161871 learned_addr learned my addr 172.21.15.67:0/1239161871 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdeb8f56640 1 --2- 172.21.15.67:0/1239161871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f0a4d0 0x55b4072e3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdeb9757640 1 -- 172.21.15.67:0/1239161871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f0a4d0 msgr2=0x55b4072e3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdeb9757640 1 --2- 172.21.15.67:0/1239161871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f0a4d0 0x55b4072e3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdeb9757640 1 -- 172.21.15.67:0/1239161871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f2b1b0 msgr2=0x55b407356360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdeb9757640 1 --2- 172.21.15.67:0/1239161871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f2b1b0 0x55b407356360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdeb9757640 1 -- 172.21.15.67:0/1239161871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4072e3b10 con 0x55b406f022f0 2024-07-31T15:10:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdeb8f56640 1 --2- 172.21.15.67:0/1239161871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f0a4d0 0x55b4072e3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdeb9f58640 1 --2- 172.21.15.67:0/1239161871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f2b1b0 0x55b407356360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fdeb9757640 1 --2- 172.21.15.67:0/1239161871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f022f0 0x55b4072e2d00 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0004970 tx=0x7fdeb0005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6017c5f3f1547ee server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.855+0000 7fde9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:10:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.856+0000 7fde9cff9640 1 -- 172.21.15.67:0/1239161871 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0062070 con 0x55b406f022f0 2024-07-31T15:10:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.856+0000 7fde9cff9640 1 -- 172.21.15.67:0/1239161871 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7fdeb00052f0 con 0x55b406f022f0 2024-07-31T15:10:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.856+0000 7fdec2992740 1 -- 172.21.15.67:0/1239161871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb0031740 con 0x55b406f022f0 2024-07-31T15:10:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.856+0000 7fde9cff9640 1 -- 172.21.15.67:0/1239161871 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fdeb006e050 con 0x55b406f022f0 2024-07-31T15:10:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.856+0000 7fdec2992740 10 client.19515 fetch_fsmap learned FSMap version 65 2024-07-31T15:10:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.856+0000 7fdec2992740 1 -- 172.21.15.67:0/1239161871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55b407357930 con 0x55b406f022f0 2024-07-31T15:10:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.856+0000 7fde9cff9640 1 -- 172.21.15.67:0/1239161871 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeb0031740 con 0x55b406f022f0 2024-07-31T15:10:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.856+0000 7fde9cff9640 1 -- 172.21.15.67:0/1239161871 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fdeb00035f0 con 0x55b406f022f0 2024-07-31T15:10:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.857+0000 7fdec2992740 10 client.19515 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:10:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.857+0000 7fdec2992740 10 client.19515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.857+0000 7fdec2992740 20 client.19515 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.857+0000 7fdec2992740 1 --2- 172.21.15.67:0/1239161871 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b406ebf1c0 0x55b407359fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.857+0000 7fdec2992740 4 client.19515 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:10:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.857+0000 7fdec2992740 1 -- 172.21.15.67:0/1239161871 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b407240da0 con 0x55b406ebf1c0 2024-07-31T15:10:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.857+0000 7fdeb8f56640 1 --2- 172.21.15.67:0/1239161871 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b406ebf1c0 0x55b407359fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.857+0000 7fdeb8f56640 1 --2- 172.21.15.67:0/1239161871 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b406ebf1c0 0x55b407359fe0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19c2f855b77026f0 server_cookie=28ce09473a4eb0ad in_seq=0 out_seq=0 2024-07-31T15:10:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:13.857+0000 7fde9cff9640 10 client.19515 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:15.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:15 smithi073 ceph-mon[36714]: pgmap v1010: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 663 KiB/s wr, 61 op/s 2024-07-31T15:10:15.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:15 smithi073 ceph-mon[36714]: osdmap e126: 12 total, 12 up, 12 in 2024-07-31T15:10:15.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:15 smithi077 ceph-mon[37327]: pgmap v1010: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 663 KiB/s wr, 61 op/s 2024-07-31T15:10:15.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:15 smithi077 ceph-mon[37327]: osdmap e126: 12 total, 12 up, 12 in 2024-07-31T15:10:15.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:15 smithi067 ceph-mon[32750]: pgmap v1010: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 663 KiB/s wr, 61 op/s 2024-07-31T15:10:15.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:15 smithi067 ceph-mon[32750]: osdmap e126: 12 total, 12 up, 12 in 2024-07-31T15:10:15.914 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:10:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.136+0000 7fde9cff9640 1 -- 172.21.15.67:0/1239161871 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b407240da0 con 0x55b406ebf1c0 2024-07-31T15:10:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.136+0000 7fde9cff9640 10 client.19515 handle_command_reply: tid=0 2024-07-31T15:10:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.136+0000 7fdec2992740 1 client.19515 shutdown 2024-07-31T15:10:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.136+0000 7fdec2992740 1 -- 172.21.15.67:0/1239161871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f022f0 msgr2=0x55b4072e2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.136+0000 7fdec2992740 1 --2- 172.21.15.67:0/1239161871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f022f0 0x55b4072e2d00 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0004970 tx=0x7fdeb0005db0 comp rx=0 tx=0).stop 2024-07-31T15:10:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.137+0000 7fdec2992740 1 -- 172.21.15.67:0/1239161871 shutdown_connections 2024-07-31T15:10:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.137+0000 7fdec2992740 1 --2- 172.21.15.67:0/1239161871 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b406ebf1c0 0x55b407359fe0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.137+0000 7fdec2992740 1 --2- 172.21.15.67:0/1239161871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b406f2b1b0 0x55b407356360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.137+0000 7fdec2992740 1 --2- 172.21.15.67:0/1239161871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b406f0a4d0 0x55b4072e3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.137+0000 7fdec2992740 1 --2- 172.21.15.67:0/1239161871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b406f022f0 0x55b4072e2d00 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.137+0000 7fdec2992740 1 -- 172.21.15.67:0/1239161871 >> 172.21.15.67:0/1239161871 conn(0x55b406f1f300 msgr2=0x55b4072ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.137+0000 7fde9cff9640 0 client.19515 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.137+0000 7fdec2992740 1 -- 172.21.15.67:0/1239161871 shutdown_connections 2024-07-31T15:10:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.137+0000 7fdec2992740 1 -- 172.21.15.67:0/1239161871 wait complete. 2024-07-31T15:10:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.137+0000 7fdec2992740 20 client.19515 trim_cache size 0 max 16384 2024-07-31T15:10:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:10:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:10:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:10:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:10:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.137+0000 7fdebb9bb640 1 -- 172.21.15.67:0/2743525215 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdea00b3d40 msgr2=0x7fdea00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.137+0000 7fdebb9bb640 1 --2- 172.21.15.67:0/2743525215 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdea00b3d40 0x7fdea00b6250 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009fd0 tx=0x7fdeac0098a0 comp rx=0 tx=0).stop 2024-07-31T15:10:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.137+0000 7fdebb9bb640 1 -- 172.21.15.67:0/2743525215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdeb4104b10 msgr2=0x7fdeb419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.137+0000 7fdebb9bb640 1 --2- 172.21.15.67:0/2743525215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdeb4104b10 0x7fdeb419a460 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fdea400e9f0 tx=0x7fdea400eeb0 comp rx=0 tx=0).stop 2024-07-31T15:10:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.138+0000 7fdebb9bb640 1 -- 172.21.15.67:0/2743525215 shutdown_connections 2024-07-31T15:10:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.138+0000 7fdebb9bb640 1 --2- 172.21.15.67:0/2743525215 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdea00b3d40 0x7fdea00b6250 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.138+0000 7fdebb9bb640 1 --2- 172.21.15.67:0/2743525215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdeb4105460 0x7fdeb419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.138+0000 7fdebb9bb640 1 --2- 172.21.15.67:0/2743525215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdeb4104b10 0x7fdeb419a460 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.138+0000 7fdebb9bb640 1 --2- 172.21.15.67:0/2743525215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeb4102bb0 0x7fdeb4199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.138+0000 7fdebb9bb640 1 -- 172.21.15.67:0/2743525215 >> 172.21.15.67:0/2743525215 conn(0x7fdeb40798d0 msgr2=0x7fdeb4106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.138+0000 7fdebb9bb640 1 -- 172.21.15.67:0/2743525215 shutdown_connections 2024-07-31T15:10:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:16.138+0000 7fdebb9bb640 1 -- 172.21.15.67:0/2743525215 wait complete. 2024-07-31T15:10:16.201 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:10:17.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:17 smithi073 ceph-mon[36714]: pgmap v1012: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 699 KiB/s wr, 62 op/s 2024-07-31T15:10:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:17 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:10:17.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:17 smithi067 ceph-mon[32750]: pgmap v1012: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 699 KiB/s wr, 62 op/s 2024-07-31T15:10:17.676 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:17 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:10:17.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:17 smithi077 ceph-mon[37327]: pgmap v1012: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 699 KiB/s wr, 62 op/s 2024-07-31T15:10:17.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:17 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:10:17.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.898+0000 7fafbe5ec640 1 -- 172.21.15.67:0/890199690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb8108ed0 msgr2=0x7fafb81092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:17.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.898+0000 7fafbe5ec640 1 --2- 172.21.15.67:0/890199690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb8108ed0 0x7fafb81092e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fafac009900 tx=0x7fafac02a410 comp rx=0 tx=0).stop 2024-07-31T15:10:17.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.898+0000 7fafbe5ec640 1 -- 172.21.15.67:0/890199690 shutdown_connections 2024-07-31T15:10:17.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.898+0000 7fafbe5ec640 1 --2- 172.21.15.67:0/890199690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafb8109820 0x7fafb8110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:17.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.898+0000 7fafbe5ec640 1 --2- 172.21.15.67:0/890199690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb8108ed0 0x7fafb81092e0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:17.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.898+0000 7fafbe5ec640 1 --2- 172.21.15.67:0/890199690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafb8103590 0x7fafb81039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.898+0000 7fafbe5ec640 1 -- 172.21.15.67:0/890199690 >> 172.21.15.67:0/890199690 conn(0x7fafb807b280 msgr2=0x7fafb807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.898+0000 7fafbe5ec640 1 -- 172.21.15.67:0/890199690 shutdown_connections 2024-07-31T15:10:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.898+0000 7fafbe5ec640 1 -- 172.21.15.67:0/890199690 wait complete. 2024-07-31T15:10:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.899+0000 7fafbe5ec640 1 Processor -- start 2024-07-31T15:10:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.899+0000 7fafbe5ec640 1 -- start start 2024-07-31T15:10:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.900+0000 7fafbe5ec640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafb8103590 0x7fafb8075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.900+0000 7fafbe5ec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafb8108ed0 0x7fafb8075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.900+0000 7fafbe5ec640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb8109820 0x7fafb8075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.900+0000 7fafbe5ec640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81134b0 con 0x7fafb8109820 2024-07-31T15:10:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.900+0000 7fafbe5ec640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8113330 con 0x7fafb8103590 2024-07-31T15:10:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.900+0000 7fafbe5ec640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8113630 con 0x7fafb8108ed0 2024-07-31T15:10:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.900+0000 7fafbcb89640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb8109820 0x7fafb8075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.900+0000 7fafb7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafb8103590 0x7fafb8075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.900+0000 7fafbcb89640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb8109820 0x7fafb8075bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35566/0 (socket says 172.21.15.67:35566) 2024-07-31T15:10:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.900+0000 7fafbcb89640 1 -- 172.21.15.67:0/100366453 learned_addr learned my addr 172.21.15.67:0/100366453 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.900+0000 7fafb77fe640 1 --2- 172.21.15.67:0/100366453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafb8108ed0 0x7fafb8075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.901+0000 7fafb7fff640 1 -- 172.21.15.67:0/100366453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafb8108ed0 msgr2=0x7fafb8075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.901+0000 7fafb7fff640 1 --2- 172.21.15.67:0/100366453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafb8108ed0 0x7fafb8075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.901+0000 7fafb7fff640 1 -- 172.21.15.67:0/100366453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb8109820 msgr2=0x7fafb8075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.901+0000 7fafb7fff640 1 --2- 172.21.15.67:0/100366453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb8109820 0x7fafb8075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.901+0000 7fafb7fff640 1 -- 172.21.15.67:0/100366453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb81a8200 con 0x7fafb8103590 2024-07-31T15:10:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.901+0000 7fafbcb89640 1 --2- 172.21.15.67:0/100366453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb8109820 0x7fafb8075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.901+0000 7fafb7fff640 1 --2- 172.21.15.67:0/100366453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafb8103590 0x7fafb8075140 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fafa800e9f0 tx=0x7fafa800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:17.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.901+0000 7fafb57fa640 1 -- 172.21.15.67:0/100366453 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa800ccc0 con 0x7fafb8103590 2024-07-31T15:10:17.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.901+0000 7fafb57fa640 1 -- 172.21.15.67:0/100366453 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafa80108e0 con 0x7fafb8103590 2024-07-31T15:10:17.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.901+0000 7fafbe5ec640 1 -- 172.21.15.67:0/100366453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb81a84f0 con 0x7fafb8103590 2024-07-31T15:10:17.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.901+0000 7fafb57fa640 1 -- 172.21.15.67:0/100366453 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8018880 con 0x7fafb8103590 2024-07-31T15:10:17.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.901+0000 7fafbe5ec640 1 -- 172.21.15.67:0/100366453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb81a8a00 con 0x7fafb8103590 2024-07-31T15:10:17.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.904+0000 7fafc4b3d740 1 Processor -- start 2024-07-31T15:10:17.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.904+0000 7fafc4b3d740 1 -- start start 2024-07-31T15:10:17.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.904+0000 7fafc4b3d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a2a070 0x5584c5dcf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:17.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.904+0000 7fafc4b3d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a011b0 0x5584c5dcfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:17.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.904+0000 7fafc4b3d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a09390 0x5584c5dcffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:17.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.904+0000 7fafb57fa640 1 -- 172.21.15.67:0/100366453 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fafa8010430 con 0x7fafb8103590 2024-07-31T15:10:17.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.904+0000 7fafc4b3d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5584c5d45b90 con 0x5584c5a09390 2024-07-31T15:10:17.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.904+0000 7fafc4b3d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5584c5d35090 con 0x5584c5a011b0 2024-07-31T15:10:17.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.904+0000 7fafc4b3d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5584c5d390d0 con 0x5584c5a2a070 2024-07-31T15:10:17.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.904+0000 7fafbcb89640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a2a070 0x5584c5dcf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:17.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.904+0000 7fafbcb89640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a2a070 0x5584c5dcf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:45522/0 (socket says 172.21.15.67:45522) 2024-07-31T15:10:17.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.904+0000 7fafbcb89640 1 -- 172.21.15.67:0/3004638599 learned_addr learned my addr 172.21.15.67:0/3004638599 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:17.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.904+0000 7fafb7fff640 1 --2- 172.21.15.67:0/3004638599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a011b0 0x5584c5dcfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:17.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.904+0000 7fafb77fe640 1 --2- 172.21.15.67:0/3004638599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a09390 0x5584c5dcffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:17.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.905+0000 7fafbcb89640 1 -- 172.21.15.67:0/3004638599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a011b0 msgr2=0x5584c5dcfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:17.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.905+0000 7fafbcb89640 1 --2- 172.21.15.67:0/3004638599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a011b0 0x5584c5dcfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:17.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.905+0000 7fafbcb89640 1 -- 172.21.15.67:0/3004638599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a09390 msgr2=0x5584c5dcffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:17.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.905+0000 7fafbcb89640 1 --2- 172.21.15.67:0/3004638599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a09390 0x5584c5dcffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:17.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.905+0000 7fafbcb89640 1 -- 172.21.15.67:0/3004638599 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584c5cdbb70 con 0x5584c5a2a070 2024-07-31T15:10:17.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.905+0000 7fafb7fff640 1 --2- 172.21.15.67:0/3004638599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a011b0 0x5584c5dcfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:10:17.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.905+0000 7fafbcb89640 1 --2- 172.21.15.67:0/3004638599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a2a070 0x5584c5dcf550 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fafb0011850 tx=0x7fafb0011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a060e60e71be5bbe server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:17.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.905+0000 7faf9e7fc640 1 -- 172.21.15.67:0/3004638599 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0011330 con 0x5584c5a2a070 2024-07-31T15:10:17.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.905+0000 7faf9e7fc640 1 -- 172.21.15.67:0/3004638599 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb000fe50 con 0x5584c5a2a070 2024-07-31T15:10:17.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.905+0000 7faf9e7fc640 1 -- 172.21.15.67:0/3004638599 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb00193e0 con 0x5584c5a2a070 2024-07-31T15:10:17.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.905+0000 7fafb57fa640 1 --2- 172.21.15.67:0/100366453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fafa00b3d40 0x7fafa00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:17.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.906+0000 7fafb7fff640 1 --2- 172.21.15.67:0/100366453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fafa00b3d40 0x7fafa00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:17.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.906+0000 7fafb57fa640 1 -- 172.21.15.67:0/100366453 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9407+0+0 (secure 0 0 0) 0x7fafa80e46f0 con 0x7fafb8103590 2024-07-31T15:10:17.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.906+0000 7fafc4b3d740 1 -- 172.21.15.67:0/3004638599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a2a070 msgr2=0x5584c5dcf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:17.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.906+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/3004638599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a2a070 0x5584c5dcf550 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fafb0011850 tx=0x7fafb0011d10 comp rx=0 tx=0).stop 2024-07-31T15:10:17.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.906+0000 7fafb7fff640 1 --2- 172.21.15.67:0/100366453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fafa00b3d40 0x7fafa00b6250 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fafa80e1b80 tx=0x7fafa8009700 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:17.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.906+0000 7fafc4b3d740 1 -- 172.21.15.67:0/3004638599 shutdown_connections 2024-07-31T15:10:17.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.906+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/3004638599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a09390 0x5584c5dcffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:17.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.906+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/3004638599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a011b0 0x5584c5dcfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:17.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.906+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/3004638599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a2a070 0x5584c5dcf550 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:17.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.906+0000 7fafc4b3d740 1 -- 172.21.15.67:0/3004638599 >> 172.21.15.67:0/3004638599 conn(0x5584c5a1e460 msgr2=0x5584c5dcd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:17.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.906+0000 7fafc4b3d740 1 -- 172.21.15.67:0/3004638599 shutdown_connections 2024-07-31T15:10:17.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.906+0000 7fafc4b3d740 1 -- 172.21.15.67:0/3004638599 wait complete. 2024-07-31T15:10:17.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.906+0000 7fafc4b3d740 1 -- start start 2024-07-31T15:10:17.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.907+0000 7fafc4b3d740 1 Processor -- start 2024-07-31T15:10:17.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.907+0000 7fafc4b3d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a011b0 0x5584c5ddb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:17.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.907+0000 7fafc4b3d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a09390 0x5584c5ddb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:17.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.907+0000 7fafc4b3d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a2a070 0x5584c5de7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:17.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.907+0000 7fafc4b3d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5584c5b98700 con 0x5584c5a011b0 2024-07-31T15:10:17.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.907+0000 7fafc4b3d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5584c5cd79c0 con 0x5584c5a09390 2024-07-31T15:10:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.907+0000 7fafc4b3d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5584c5dddd30 con 0x5584c5a2a070 2024-07-31T15:10:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.907+0000 7fafc4b3d740 10 client.0 _ll_register_callbacks cb 0x5584c5cf24d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.907+0000 7fafbcb89640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a011b0 0x5584c5ddb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.907+0000 7fafbcb89640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a011b0 0x5584c5ddb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35578/0 (socket says 172.21.15.67:35578) 2024-07-31T15:10:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.907+0000 7fafb77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a09390 0x5584c5ddb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.907+0000 7fafbcb89640 1 -- 172.21.15.67:0/2455424888 learned_addr learned my addr 172.21.15.67:0/2455424888 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7fafbcb89640 1 --2- 172.21.15.67:0/2455424888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a2a070 0x5584c5de7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7fafb77fe640 1 -- 172.21.15.67:0/2455424888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a2a070 msgr2=0x5584c5de7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7fafb77fe640 1 --2- 172.21.15.67:0/2455424888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a2a070 0x5584c5de7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7fafb77fe640 1 -- 172.21.15.67:0/2455424888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a011b0 msgr2=0x5584c5ddb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7fafb77fe640 1 --2- 172.21.15.67:0/2455424888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a011b0 0x5584c5ddb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7fafb77fe640 1 -- 172.21.15.67:0/2455424888 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584c5a95650 con 0x5584c5a09390 2024-07-31T15:10:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7fafbcb89640 1 --2- 172.21.15.67:0/2455424888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a011b0 0x5584c5ddb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7fafb77fe640 1 --2- 172.21.15.67:0/2455424888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a09390 0x5584c5ddb550 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fafac02a930 tx=0x7fafac032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddb3fc0530011dfe server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7faf977fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:10:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7faf977fe640 1 -- 172.21.15.67:0/2455424888 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac035040 con 0x5584c5a09390 2024-07-31T15:10:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7faf977fe640 1 -- 172.21.15.67:0/2455424888 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9407+0+0 (secure 0 0 0) 0x7fafac031760 con 0x5584c5a09390 2024-07-31T15:10:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7fafc4b3d740 1 -- 172.21.15.67:0/2455424888 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb000fe50 con 0x5584c5a09390 2024-07-31T15:10:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7faf977fe640 1 -- 172.21.15.67:0/2455424888 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fafac004040 con 0x5584c5a09390 2024-07-31T15:10:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7fafc4b3d740 10 client.29060 fetch_fsmap learned FSMap version 65 2024-07-31T15:10:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.908+0000 7fafc4b3d740 1 -- 172.21.15.67:0/2455424888 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5584c5de9c50 con 0x5584c5a09390 2024-07-31T15:10:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.909+0000 7faf977fe640 1 -- 172.21.15.67:0/2455424888 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb000fe50 con 0x5584c5a09390 2024-07-31T15:10:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.909+0000 7faf977fe640 1 -- 172.21.15.67:0/2455424888 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fafac004750 con 0x5584c5a09390 2024-07-31T15:10:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.909+0000 7fafc4b3d740 10 client.29060 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:10:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.909+0000 7fafc4b3d740 10 client.29060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:17.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.909+0000 7fafc4b3d740 20 client.29060 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:17.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.909+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/2455424888 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584c59e89f0 0x5584c5ded140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:17.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.909+0000 7fafc4b3d740 4 client.29060 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:10:17.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.909+0000 7fafc4b3d740 1 -- 172.21.15.67:0/2455424888 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584c5d47ae0 con 0x5584c59e89f0 2024-07-31T15:10:17.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.909+0000 7fafbcb89640 1 --2- 172.21.15.67:0/2455424888 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584c59e89f0 0x5584c5ded140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:17.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.909+0000 7faf977fe640 10 client.29060 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:17.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:17.909+0000 7fafbcb89640 1 --2- 172.21.15.67:0/2455424888 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584c59e89f0 0x5584c5ded140 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f34fc51026eb3aa server_cookie=862f52a500faa7e1 in_seq=0 out_seq=0 2024-07-31T15:10:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7faf977fe640 1 -- 172.21.15.67:0/2455424888 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584c5d47ae0 con 0x5584c59e89f0 2024-07-31T15:10:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7faf977fe640 10 client.29060 handle_command_reply: tid=0 2024-07-31T15:10:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7fafc4b3d740 1 client.29060 shutdown 2024-07-31T15:10:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7fafc4b3d740 1 -- 172.21.15.67:0/2455424888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a09390 msgr2=0x5584c5ddb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/2455424888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a09390 0x5584c5ddb550 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fafac02a930 tx=0x7fafac032e90 comp rx=0 tx=0).stop 2024-07-31T15:10:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7fafc4b3d740 1 -- 172.21.15.67:0/2455424888 shutdown_connections 2024-07-31T15:10:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/2455424888 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584c59e89f0 0x5584c5ded140 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/2455424888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a2a070 0x5584c5de7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/2455424888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a09390 0x5584c5ddb550 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7faf977fe640 0 client.29060 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/2455424888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a011b0 0x5584c5ddb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7fafc4b3d740 1 -- 172.21.15.67:0/2455424888 >> 172.21.15.67:0/2455424888 conn(0x5584c5a1e460 msgr2=0x5584c5dcbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7fafc4b3d740 1 -- 172.21.15.67:0/2455424888 shutdown_connections 2024-07-31T15:10:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7fafc4b3d740 1 -- 172.21.15.67:0/2455424888 wait complete. 2024-07-31T15:10:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.143+0000 7fafc4b3d740 20 client.29060 trim_cache size 0 max 16384 2024-07-31T15:10:18.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.162+0000 7fafc4b3d740 1 Processor -- start 2024-07-31T15:10:18.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.162+0000 7fafc4b3d740 1 -- start start 2024-07-31T15:10:18.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.162+0000 7fafc4b3d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c59e89f0 0x5584c5ddca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.162+0000 7fafc4b3d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a011b0 0x5584c5de6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.162+0000 7fafc4b3d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a09390 0x5584c5de7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.162+0000 7fafc4b3d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81b2a70 con 0x5584c5a09390 2024-07-31T15:10:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.162+0000 7fafc4b3d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81b45b0 con 0x5584c59e89f0 2024-07-31T15:10:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.162+0000 7fafc4b3d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81b3140 con 0x5584c5a011b0 2024-07-31T15:10:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.162+0000 7fafbcb89640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a09390 0x5584c5de7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.162+0000 7fafbcb89640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a09390 0x5584c5de7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35588/0 (socket says 172.21.15.67:35588) 2024-07-31T15:10:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.162+0000 7fafbcb89640 1 -- 172.21.15.67:0/1145163314 learned_addr learned my addr 172.21.15.67:0/1145163314 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.162+0000 7fafbcb89640 1 --2- 172.21.15.67:0/1145163314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c59e89f0 0x5584c5ddca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.162+0000 7fafb77fe640 1 --2- 172.21.15.67:0/1145163314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a011b0 0x5584c5de6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.163+0000 7fafbcb89640 1 -- 172.21.15.67:0/1145163314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a011b0 msgr2=0x5584c5de6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:18.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.163+0000 7fafbcb89640 1 --2- 172.21.15.67:0/1145163314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a011b0 0x5584c5de6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.163+0000 7fafbcb89640 1 -- 172.21.15.67:0/1145163314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c59e89f0 msgr2=0x5584c5ddca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:18.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.163+0000 7fafbcb89640 1 --2- 172.21.15.67:0/1145163314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c59e89f0 0x5584c5ddca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.163+0000 7fafbcb89640 1 -- 172.21.15.67:0/1145163314 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584c5ddb8c0 con 0x5584c5a09390 2024-07-31T15:10:18.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.163+0000 7fafbcb89640 1 --2- 172.21.15.67:0/1145163314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a09390 0x5584c5de7350 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009aa0 tx=0x7fafb0010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e1fd8cf013dcc2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:18.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.163+0000 7faf9dffb640 1 -- 172.21.15.67:0/1145163314 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb001f040 con 0x5584c5a09390 2024-07-31T15:10:18.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.163+0000 7faf9dffb640 1 -- 172.21.15.67:0/1145163314 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb000fc90 con 0x5584c5a09390 2024-07-31T15:10:18.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.163+0000 7faf9dffb640 1 -- 172.21.15.67:0/1145163314 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0005020 con 0x5584c5a09390 2024-07-31T15:10:18.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.164+0000 7fafc4b3d740 1 -- 172.21.15.67:0/1145163314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a09390 msgr2=0x5584c5de7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:18.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.164+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/1145163314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a09390 0x5584c5de7350 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009aa0 tx=0x7fafb0010b30 comp rx=0 tx=0).stop 2024-07-31T15:10:18.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.164+0000 7fafc4b3d740 1 -- 172.21.15.67:0/1145163314 shutdown_connections 2024-07-31T15:10:18.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.164+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/1145163314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c5a09390 0x5584c5de7350 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.164+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/1145163314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a011b0 0x5584c5de6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.164+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/1145163314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c59e89f0 0x5584c5ddca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.164+0000 7fafc4b3d740 1 -- 172.21.15.67:0/1145163314 >> 172.21.15.67:0/1145163314 conn(0x5584c5a1e460 msgr2=0x5584c5ddc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.164+0000 7fafc4b3d740 1 -- 172.21.15.67:0/1145163314 shutdown_connections 2024-07-31T15:10:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.164+0000 7fafc4b3d740 1 -- 172.21.15.67:0/1145163314 wait complete. 2024-07-31T15:10:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.164+0000 7fafc4b3d740 1 -- start start 2024-07-31T15:10:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.165+0000 7fafc4b3d740 1 Processor -- start 2024-07-31T15:10:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.165+0000 7fafc4b3d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c59e89f0 0x5584c5de35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.165+0000 7fafc4b3d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a011b0 0x5584c5ddea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.165+0000 7fafc4b3d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a09390 0x5584c5ddefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.165+0000 7fafc4b3d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81b3ee0 con 0x5584c59e89f0 2024-07-31T15:10:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.165+0000 7fafc4b3d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81b3810 con 0x5584c5a09390 2024-07-31T15:10:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.165+0000 7fafc4b3d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81b0a50 con 0x5584c5a011b0 2024-07-31T15:10:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.165+0000 7fafc4b3d740 10 client.0 _ll_register_callbacks cb 0x7fafac032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.165+0000 7fafbcb89640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c59e89f0 0x5584c5de35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.165+0000 7fafbcb89640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c59e89f0 0x5584c5de35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35592/0 (socket says 172.21.15.67:35592) 2024-07-31T15:10:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.165+0000 7fafbcb89640 1 -- 172.21.15.67:0/3267843700 learned_addr learned my addr 172.21.15.67:0/3267843700 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.165+0000 7fafb77fe640 1 --2- 172.21.15.67:0/3267843700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a011b0 0x5584c5ddea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.165+0000 7fafbcb89640 1 --2- 172.21.15.67:0/3267843700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a09390 0x5584c5ddefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.166+0000 7fafbcb89640 1 -- 172.21.15.67:0/3267843700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a011b0 msgr2=0x5584c5ddea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.166+0000 7fafbcb89640 1 --2- 172.21.15.67:0/3267843700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a011b0 0x5584c5ddea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.166+0000 7fafbcb89640 1 -- 172.21.15.67:0/3267843700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a09390 msgr2=0x5584c5ddefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.166+0000 7fafbcb89640 1 --2- 172.21.15.67:0/3267843700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a09390 0x5584c5ddefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.166+0000 7fafbcb89640 1 -- 172.21.15.67:0/3267843700 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584c5ddf6e0 con 0x5584c59e89f0 2024-07-31T15:10:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.166+0000 7fafbcb89640 1 --2- 172.21.15.67:0/3267843700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c59e89f0 0x5584c5de35e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009730 tx=0x7fafb00111a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb40a8dab72cdc1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.166+0000 7faf977fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:10:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.166+0000 7faf977fe640 1 -- 172.21.15.67:0/3267843700 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb00049d0 con 0x5584c59e89f0 2024-07-31T15:10:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.166+0000 7faf977fe640 1 -- 172.21.15.67:0/3267843700 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9407+0+0 (secure 0 0 0) 0x7fafb003d070 con 0x5584c59e89f0 2024-07-31T15:10:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.166+0000 7fafc4b3d740 1 -- 172.21.15.67:0/3267843700 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb000fc90 con 0x5584c59e89f0 2024-07-31T15:10:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.166+0000 7faf977fe640 1 -- 172.21.15.67:0/3267843700 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fafb004b050 con 0x5584c59e89f0 2024-07-31T15:10:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.166+0000 7fafc4b3d740 10 client.19533 fetch_fsmap learned FSMap version 65 2024-07-31T15:10:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.166+0000 7fafc4b3d740 1 -- 172.21.15.67:0/3267843700 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5584c5e557f0 con 0x5584c59e89f0 2024-07-31T15:10:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.166+0000 7faf977fe640 1 -- 172.21.15.67:0/3267843700 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb000fc90 con 0x5584c59e89f0 2024-07-31T15:10:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.167+0000 7faf977fe640 1 -- 172.21.15.67:0/3267843700 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fafb0018070 con 0x5584c59e89f0 2024-07-31T15:10:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.167+0000 7fafc4b3d740 10 client.19533 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:10:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.167+0000 7fafc4b3d740 10 client.19533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.167+0000 7fafc4b3d740 20 client.19533 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.167+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/3267843700 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584c5a2a070 0x5584c5e57c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.167+0000 7fafc4b3d740 4 client.19533 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:10:18.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.167+0000 7fafc4b3d740 1 -- 172.21.15.67:0/3267843700 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5584c5d43580 con 0x5584c5a2a070 2024-07-31T15:10:18.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.167+0000 7fafb77fe640 1 --2- 172.21.15.67:0/3267843700 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584c5a2a070 0x5584c5e57c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:18.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.168+0000 7faf977fe640 10 client.19533 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:18.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.168+0000 7fafb77fe640 1 --2- 172.21.15.67:0/3267843700 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584c5a2a070 0x5584c5e57c10 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f56938ddc8ac008 server_cookie=53dc2f75e9f076ba in_seq=0 out_seq=0 2024-07-31T15:10:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7faf977fe640 1 -- 172.21.15.67:0/3267843700 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5584c5d43580 con 0x5584c5a2a070 2024-07-31T15:10:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7faf977fe640 10 client.19533 handle_command_reply: tid=0 2024-07-31T15:10:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7fafc4b3d740 1 client.19533 shutdown 2024-07-31T15:10:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7fafc4b3d740 1 -- 172.21.15.67:0/3267843700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c59e89f0 msgr2=0x5584c5de35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/3267843700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c59e89f0 0x5584c5de35e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009730 tx=0x7fafb00111a0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7fafc4b3d740 1 -- 172.21.15.67:0/3267843700 shutdown_connections 2024-07-31T15:10:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/3267843700 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5584c5a2a070 0x5584c5e57c10 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/3267843700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5584c5a09390 0x5584c5ddefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/3267843700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5584c5a011b0 0x5584c5ddea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7fafc4b3d740 1 --2- 172.21.15.67:0/3267843700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5584c59e89f0 0x5584c5de35e0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7faf977fe640 0 client.19533 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:18.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7fafc4b3d740 1 -- 172.21.15.67:0/3267843700 >> 172.21.15.67:0/3267843700 conn(0x5584c5a1e460 msgr2=0x5584c5dd0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:18.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7fafc4b3d740 1 -- 172.21.15.67:0/3267843700 shutdown_connections 2024-07-31T15:10:18.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7fafc4b3d740 1 -- 172.21.15.67:0/3267843700 wait complete. 2024-07-31T15:10:18.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.184+0000 7fafc4b3d740 20 client.19533 trim_cache size 0 max 16384 2024-07-31T15:10:18.187 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:10:18.187 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T15:10:18.187 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T15:10:18.187 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:10:18.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.185+0000 7fafbe5ec640 1 -- 172.21.15.67:0/100366453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fafa00b3d40 msgr2=0x7fafa00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:18.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.185+0000 7fafbe5ec640 1 --2- 172.21.15.67:0/100366453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fafa00b3d40 0x7fafa00b6250 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fafa80e1b80 tx=0x7fafa8009700 comp rx=0 tx=0).stop 2024-07-31T15:10:18.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.185+0000 7fafbe5ec640 1 -- 172.21.15.67:0/100366453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafb8103590 msgr2=0x7fafb8075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:18.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.185+0000 7fafbe5ec640 1 --2- 172.21.15.67:0/100366453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafb8103590 0x7fafb8075140 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fafa800e9f0 tx=0x7fafa800eeb0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.185+0000 7fafbe5ec640 1 -- 172.21.15.67:0/100366453 shutdown_connections 2024-07-31T15:10:18.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.185+0000 7fafbe5ec640 1 --2- 172.21.15.67:0/100366453 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fafa00b3d40 0x7fafa00b6250 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.185+0000 7fafbe5ec640 1 --2- 172.21.15.67:0/100366453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb8109820 0x7fafb8075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.185+0000 7fafbe5ec640 1 --2- 172.21.15.67:0/100366453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafb8108ed0 0x7fafb8075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.185+0000 7fafbe5ec640 1 --2- 172.21.15.67:0/100366453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafb8103590 0x7fafb8075140 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:18.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.185+0000 7fafbe5ec640 1 -- 172.21.15.67:0/100366453 >> 172.21.15.67:0/100366453 conn(0x7fafb807b280 msgr2=0x7fafb8107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:18.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.185+0000 7fafbe5ec640 1 -- 172.21.15.67:0/100366453 shutdown_connections 2024-07-31T15:10:18.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:18.185+0000 7fafbe5ec640 1 -- 172.21.15.67:0/100366453 wait complete. 2024-07-31T15:10:18.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:18 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:18.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:18 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:18.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:18 smithi077 ceph-mon[37327]: pgmap v1013: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 504 KiB/s wr, 66 op/s 2024-07-31T15:10:18.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:18 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:18.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:18 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:18.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:18 smithi067 ceph-mon[32750]: pgmap v1013: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 504 KiB/s wr, 66 op/s 2024-07-31T15:10:18.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:18 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:18.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:18 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:18.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:18 smithi073 ceph-mon[36714]: pgmap v1013: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 504 KiB/s wr, 66 op/s 2024-07-31T15:10:19.484 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T15:10:19.484 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:10:19.767 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:10:20.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:20.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:20.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:20.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:19 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:20.150 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:20.150 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:20.150 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:20.150 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:19 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:20.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:20.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:20.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:20.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:19 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:21.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.103+0000 7f8f21ffb640 1 -- 172.21.15.67:0/2979579818 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f8f34021af0 con 0x55d4bbfd0e40 2024-07-31T15:10:21.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.103+0000 7f8f21ffb640 10 client.29030 handle_command_reply: tid=0 2024-07-31T15:10:21.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.103+0000 7f8f46ff0740 1 client.29030 shutdown 2024-07-31T15:10:21.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.104+0000 7f8f46ff0740 1 -- 172.21.15.67:0/2979579818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb631c0 msgr2=0x55d4bbf5e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:21.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.104+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb631c0 0x55d4bbf5e9b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400a550 tx=0x7f8f34016610 comp rx=0 tx=0).stop 2024-07-31T15:10:21.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.104+0000 7f8f46ff0740 1 -- 172.21.15.67:0/2979579818 shutdown_connections 2024-07-31T15:10:21.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.104+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d4bbfd0e40 0x55d4bbfd3260 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:21.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.104+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d4bbb83b60 0x55d4bbf5c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:21.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.104+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4bbb7b980 0x55d4bbf5bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:21.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.104+0000 7f8f46ff0740 1 --2- 172.21.15.67:0/2979579818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4bbb631c0 0x55d4bbf5e9b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:21.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.104+0000 7f8f21ffb640 0 client.29030 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:21.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.104+0000 7f8f46ff0740 1 -- 172.21.15.67:0/2979579818 >> 172.21.15.67:0/2979579818 conn(0x55d4bbb98990 msgr2=0x55d4bbf61410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:21.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.104+0000 7f8f46ff0740 1 -- 172.21.15.67:0/2979579818 shutdown_connections 2024-07-31T15:10:21.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.104+0000 7f8f46ff0740 1 -- 172.21.15.67:0/2979579818 wait complete. 2024-07-31T15:10:21.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.104+0000 7f8f46ff0740 20 client.29030 trim_cache size 0 max 16384 2024-07-31T15:10:21.107 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:10:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-07-31T15:10:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:10:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 54, 2024-07-31T15:10:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:10:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "dd84c0d1": { 2024-07-31T15:10:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 54, 2024-07-31T15:10:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:10:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:10:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:10:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:10:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:10:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:10:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 138.1, 2024-07-31T15:10:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:10:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:10:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:10:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:10:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:10:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:10:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:10:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:10:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:10:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:10:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:10:21.109 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:10:21.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.105+0000 7f8f40a9f640 1 -- 172.21.15.67:0/2683761725 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8f240b3f40 msgr2=0x7f8f240b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:21.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.105+0000 7f8f40a9f640 1 --2- 172.21.15.67:0/2683761725 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8f240b3f40 0x7f8f240b6450 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c118810 tx=0x7f8f34009250 comp rx=0 tx=0).stop 2024-07-31T15:10:21.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.105+0000 7f8f40a9f640 1 -- 172.21.15.67:0/2683761725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f3c108d80 msgr2=0x7f8f3c112d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:21.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.105+0000 7f8f40a9f640 1 --2- 172.21.15.67:0/2683761725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f3c108d80 0x7f8f3c112d80 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f8f3002a9b0 tx=0x7f8f30003d60 comp rx=0 tx=0).stop 2024-07-31T15:10:21.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.105+0000 7f8f40a9f640 1 -- 172.21.15.67:0/2683761725 shutdown_connections 2024-07-31T15:10:21.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.105+0000 7f8f40a9f640 1 --2- 172.21.15.67:0/2683761725 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8f240b3f40 0x7f8f240b6450 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:21.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.105+0000 7f8f40a9f640 1 --2- 172.21.15.67:0/2683761725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f3c1096d0 0x7f8f3c1132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:21.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.105+0000 7f8f40a9f640 1 --2- 172.21.15.67:0/2683761725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f3c108d80 0x7f8f3c112d80 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:21.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.105+0000 7f8f40a9f640 1 --2- 172.21.15.67:0/2683761725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f3c103440 0x7f8f3c118ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:21.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.105+0000 7f8f40a9f640 1 -- 172.21.15.67:0/2683761725 >> 172.21.15.67:0/2683761725 conn(0x7f8f3c0feed0 msgr2=0x7f8f3c1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:21.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.105+0000 7f8f40a9f640 1 -- 172.21.15.67:0/2683761725 shutdown_connections 2024-07-31T15:10:21.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:21.105+0000 7f8f40a9f640 1 -- 172.21.15.67:0/2683761725 wait complete. 2024-07-31T15:10:21.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:21 smithi077 ceph-mon[37327]: pgmap v1014: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 418 KiB/s wr, 69 op/s 2024-07-31T15:10:21.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:21 smithi077 ceph-mon[37327]: osdmap e127: 12 total, 12 up, 12 in 2024-07-31T15:10:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:21 smithi067 ceph-mon[32750]: pgmap v1014: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 418 KiB/s wr, 69 op/s 2024-07-31T15:10:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:21 smithi067 ceph-mon[32750]: osdmap e127: 12 total, 12 up, 12 in 2024-07-31T15:10:21.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:21 smithi073 ceph-mon[36714]: pgmap v1014: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 418 KiB/s wr, 69 op/s 2024-07-31T15:10:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:21 smithi073 ceph-mon[36714]: osdmap e127: 12 total, 12 up, 12 in 2024-07-31T15:10:21.919 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'dd84c0d1', quiesce took 28.6 seconds. Will release after: -10.5 2024-07-31T15:10:21.919 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:10:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.153+0000 7fb266717640 1 -- 172.21.15.67:0/603876300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb260108740 msgr2=0x7fb26010ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.153+0000 7fb266717640 1 --2- 172.21.15.67:0/603876300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb260108740 0x7fb26010ef70 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fb258009ca0 tx=0x7fb25802a8c0 comp rx=0 tx=0).stop 2024-07-31T15:10:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.153+0000 7fb266717640 1 -- 172.21.15.67:0/603876300 shutdown_connections 2024-07-31T15:10:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.153+0000 7fb266717640 1 --2- 172.21.15.67:0/603876300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb260108740 0x7fb26010ef70 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.153+0000 7fb266717640 1 --2- 172.21.15.67:0/603876300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb260107df0 0x7fb260108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:22.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.153+0000 7fb266717640 1 --2- 172.21.15.67:0/603876300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2601024b0 0x7fb2601028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:22.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.153+0000 7fb266717640 1 -- 172.21.15.67:0/603876300 >> 172.21.15.67:0/603876300 conn(0x7fb260079170 msgr2=0x7fb260079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:22.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.155+0000 7fb266717640 1 -- 172.21.15.67:0/603876300 shutdown_connections 2024-07-31T15:10:22.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.155+0000 7fb266717640 1 -- 172.21.15.67:0/603876300 wait complete. 2024-07-31T15:10:22.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.155+0000 7fb266717640 1 Processor -- start 2024-07-31T15:10:22.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.155+0000 7fb266717640 1 -- start start 2024-07-31T15:10:22.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb266717640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2601024b0 0x7fb260198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:22.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb266717640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb260107df0 0x7fb260199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:22.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb266717640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb260108740 0x7fb260199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:22.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb266717640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb2601123c0 con 0x7fb260107df0 2024-07-31T15:10:22.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb266717640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb260112240 con 0x7fb260108740 2024-07-31T15:10:22.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb266717640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb260112540 con 0x7fb2601024b0 2024-07-31T15:10:22.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb257fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb260107df0 0x7fb260199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:22.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb257fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb260107df0 0x7fb260199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44274/0 (socket says 172.21.15.67:44274) 2024-07-31T15:10:22.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb257fff640 1 -- 172.21.15.67:0/1441801894 learned_addr learned my addr 172.21.15.67:0/1441801894 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:22.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb25ffff640 1 --2- 172.21.15.67:0/1441801894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2601024b0 0x7fb260198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:22.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb264cb4640 1 --2- 172.21.15.67:0/1441801894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb260108740 0x7fb260199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:22.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb257fff640 1 -- 172.21.15.67:0/1441801894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2601024b0 msgr2=0x7fb260198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:22.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb257fff640 1 --2- 172.21.15.67:0/1441801894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2601024b0 0x7fb260198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:22.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb257fff640 1 -- 172.21.15.67:0/1441801894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb260108740 msgr2=0x7fb260199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:22.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb257fff640 1 --2- 172.21.15.67:0/1441801894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb260108740 0x7fb260199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:22.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb257fff640 1 -- 172.21.15.67:0/1441801894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2601a8160 con 0x7fb260107df0 2024-07-31T15:10:22.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.156+0000 7fb257fff640 1 --2- 172.21.15.67:0/1441801894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb260107df0 0x7fb260199020 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fb24800d740 tx=0x7fb24800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:22.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.157+0000 7fb25dffb640 1 -- 172.21.15.67:0/1441801894 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248017070 con 0x7fb260107df0 2024-07-31T15:10:22.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.157+0000 7fb25dffb640 1 -- 172.21.15.67:0/1441801894 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb24800de50 con 0x7fb260107df0 2024-07-31T15:10:22.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.157+0000 7fb266717640 1 -- 172.21.15.67:0/1441801894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2601a83f0 con 0x7fb260107df0 2024-07-31T15:10:22.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.157+0000 7fb25dffb640 1 -- 172.21.15.67:0/1441801894 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248004ce0 con 0x7fb260107df0 2024-07-31T15:10:22.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.157+0000 7fb266717640 1 -- 172.21.15.67:0/1441801894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2601a8930 con 0x7fb260107df0 2024-07-31T15:10:22.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.158+0000 7fb266717640 1 -- 172.21.15.67:0/1441801894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb260103730 con 0x7fb260107df0 2024-07-31T15:10:22.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.160+0000 7fb25dffb640 1 -- 172.21.15.67:0/1441801894 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb248004e80 con 0x7fb260107df0 2024-07-31T15:10:22.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.161+0000 7fb25dffb640 1 --2- 172.21.15.67:0/1441801894 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb24c0b3dc0 0x7fb24c0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:22.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.161+0000 7fb25ffff640 1 --2- 172.21.15.67:0/1441801894 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb24c0b3dc0 0x7fb24c0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:22.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.161+0000 7fb25dffb640 1 -- 172.21.15.67:0/1441801894 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9379+0+0 (secure 0 0 0) 0x7fb2480e40f0 con 0x7fb260107df0 2024-07-31T15:10:22.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.162+0000 7fb25ffff640 1 --2- 172.21.15.67:0/1441801894 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb24c0b3dc0 0x7fb24c0b62d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb250005e10 tx=0x7fb250005da0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:22.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.164+0000 7fb25dffb640 1 -- 172.21.15.67:0/1441801894 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb2480ab120 con 0x7fb260107df0 2024-07-31T15:10:22.283 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:10:22.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.423+0000 7fb266717640 1 -- 172.21.15.67:0/1441801894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb26006b980 con 0x7fb260107df0 2024-07-31T15:10:22.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.424+0000 7fb25dffb640 1 -- 172.21.15.67:0/1441801894 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11962 (secure 0 0 0) 0x7fb26006b980 con 0x7fb260107df0 2024-07-31T15:10:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:10:22.425 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"btime":"2024-07-31T15:06:04:323642+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:06:04.323492+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28214":{"gid":28214,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1516494761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1516494761},{"type":"v1","addr":"172.21.15.73:6835","nonce":1516494761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:10:22.426 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-07-31T15:10:22.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.426+0000 7fb266717640 1 -- 172.21.15.67:0/1441801894 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb24c0b3dc0 msgr2=0x7fb24c0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:22.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.426+0000 7fb266717640 1 --2- 172.21.15.67:0/1441801894 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb24c0b3dc0 0x7fb24c0b62d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb250005e10 tx=0x7fb250005da0 comp rx=0 tx=0).stop 2024-07-31T15:10:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.426+0000 7fb266717640 1 -- 172.21.15.67:0/1441801894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb260107df0 msgr2=0x7fb260199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.426+0000 7fb266717640 1 --2- 172.21.15.67:0/1441801894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb260107df0 0x7fb260199020 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fb24800d740 tx=0x7fb24800dc00 comp rx=0 tx=0).stop 2024-07-31T15:10:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.426+0000 7fb266717640 1 -- 172.21.15.67:0/1441801894 shutdown_connections 2024-07-31T15:10:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.426+0000 7fb266717640 1 --2- 172.21.15.67:0/1441801894 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb24c0b3dc0 0x7fb24c0b62d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.426+0000 7fb266717640 1 --2- 172.21.15.67:0/1441801894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb260108740 0x7fb260199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.426+0000 7fb266717640 1 --2- 172.21.15.67:0/1441801894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb260107df0 0x7fb260199020 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.426+0000 7fb266717640 1 --2- 172.21.15.67:0/1441801894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2601024b0 0x7fb260198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.426+0000 7fb266717640 1 -- 172.21.15.67:0/1441801894 >> 172.21.15.67:0/1441801894 conn(0x7fb260079170 msgr2=0x7fb260104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.426+0000 7fb266717640 1 -- 172.21.15.67:0/1441801894 shutdown_connections 2024-07-31T15:10:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:22.426+0000 7fb266717640 1 -- 172.21.15.67:0/1441801894 wait complete. 2024-07-31T15:10:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:23 smithi077 ceph-mon[37327]: pgmap v1016: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 221 KiB/s wr, 72 op/s 2024-07-31T15:10:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:23 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1441801894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:10:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:10:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:10:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:23 smithi067 ceph-mon[32750]: pgmap v1016: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 221 KiB/s wr, 72 op/s 2024-07-31T15:10:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:23 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1441801894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:10:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:10:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:10:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:23 smithi073 ceph-mon[36714]: pgmap v1016: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 221 KiB/s wr, 72 op/s 2024-07-31T15:10:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:23 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1441801894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:10:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:10:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:10:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:10:23.651 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T15:10:24.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.580+0000 7f89a739d640 1 -- 172.21.15.67:0/1806881394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89a0108530 msgr2=0x7f89a010ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:24.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.580+0000 7f89a739d640 1 --2- 172.21.15.67:0/1806881394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89a0108530 0x7f89a010ed60 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f8998009ca0 tx=0x7f899802a8c0 comp rx=0 tx=0).stop 2024-07-31T15:10:24.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.580+0000 7f89a739d640 1 -- 172.21.15.67:0/1806881394 shutdown_connections 2024-07-31T15:10:24.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.580+0000 7f89a739d640 1 --2- 172.21.15.67:0/1806881394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89a0108530 0x7f89a010ed60 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:24.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.580+0000 7f89a739d640 1 --2- 172.21.15.67:0/1806881394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89a0107be0 0x7f89a0107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:24.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.580+0000 7f89a739d640 1 --2- 172.21.15.67:0/1806881394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a01022a0 0x7f89a01026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:24.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.580+0000 7f89a739d640 1 -- 172.21.15.67:0/1806881394 >> 172.21.15.67:0/1806881394 conn(0x7f89a00fdcf0 msgr2=0x7f89a0100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:24.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.580+0000 7f89a739d640 1 -- 172.21.15.67:0/1806881394 shutdown_connections 2024-07-31T15:10:24.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.580+0000 7f89a739d640 1 -- 172.21.15.67:0/1806881394 wait complete. 2024-07-31T15:10:24.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.581+0000 7f89a739d640 1 Processor -- start 2024-07-31T15:10:24.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a739d640 1 -- start start 2024-07-31T15:10:24.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a739d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a01022a0 0x7f89a019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:24.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a739d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89a0107be0 0x7f89a019b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:24.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a739d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89a0108530 0x7f89a019b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:24.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a739d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f89a0112200 con 0x7f89a0108530 2024-07-31T15:10:24.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a739d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f89a0112080 con 0x7f89a01022a0 2024-07-31T15:10:24.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a739d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f89a0112380 con 0x7f89a0107be0 2024-07-31T15:10:24.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a6b9c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89a0108530 0x7f89a019b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:24.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a5b9a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89a0107be0 0x7f89a019b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:24.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a6b9c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89a0108530 0x7f89a019b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44294/0 (socket says 172.21.15.67:44294) 2024-07-31T15:10:24.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a6b9c640 1 -- 172.21.15.67:0/1334276081 learned_addr learned my addr 172.21.15.67:0/1334276081 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:24.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a639b640 1 --2- 172.21.15.67:0/1334276081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a01022a0 0x7f89a019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:24.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a6b9c640 1 -- 172.21.15.67:0/1334276081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89a0107be0 msgr2=0x7f89a019b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:24.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a6b9c640 1 --2- 172.21.15.67:0/1334276081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89a0107be0 0x7f89a019b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:24.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a6b9c640 1 -- 172.21.15.67:0/1334276081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a01022a0 msgr2=0x7f89a019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:24.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.582+0000 7f89a6b9c640 1 --2- 172.21.15.67:0/1334276081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a01022a0 0x7f89a019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:24.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.583+0000 7f89a6b9c640 1 -- 172.21.15.67:0/1334276081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a01a5d60 con 0x7f89a0108530 2024-07-31T15:10:24.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.583+0000 7f89a5b9a640 1 --2- 172.21.15.67:0/1334276081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89a0107be0 0x7f89a019b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:24.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.583+0000 7f89a6b9c640 1 --2- 172.21.15.67:0/1334276081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89a0108530 0x7f89a019b550 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f8998005cc0 tx=0x7f899802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:24.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.583+0000 7f89937fe640 1 -- 172.21.15.67:0/1334276081 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998003bf0 con 0x7f89a0108530 2024-07-31T15:10:24.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.583+0000 7f89a739d640 1 -- 172.21.15.67:0/1334276081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a01a5ff0 con 0x7f89a0108530 2024-07-31T15:10:24.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.583+0000 7f89937fe640 1 -- 172.21.15.67:0/1334276081 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8998003d90 con 0x7f89a0108530 2024-07-31T15:10:24.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.583+0000 7f89937fe640 1 -- 172.21.15.67:0/1334276081 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899802fbc0 con 0x7f89a0108530 2024-07-31T15:10:24.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.583+0000 7f89a739d640 1 -- 172.21.15.67:0/1334276081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a01a64a0 con 0x7f89a0108530 2024-07-31T15:10:24.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.584+0000 7f89a739d640 1 -- 172.21.15.67:0/1334276081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a01026b0 con 0x7f89a0108530 2024-07-31T15:10:24.587 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:10:24.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.586+0000 7f89937fe640 1 -- 172.21.15.67:0/1334276081 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f899802fd60 con 0x7f89a0108530 2024-07-31T15:10:24.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.587+0000 7f89937fe640 1 --2- 172.21.15.67:0/1334276081 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f898c0b3d70 0x7f898c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:24.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.587+0000 7f89a639b640 1 --2- 172.21.15.67:0/1334276081 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f898c0b3d70 0x7f898c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:24.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.587+0000 7f89937fe640 1 -- 172.21.15.67:0/1334276081 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9379+0+0 (secure 0 0 0) 0x7f8998105600 con 0x7f89a0108530 2024-07-31T15:10:24.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.588+0000 7f89a639b640 1 --2- 172.21.15.67:0/1334276081 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f898c0b3d70 0x7f898c0b6280 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f899c0058c0 tx=0x7f899c00a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:24.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.591+0000 7f89937fe640 1 -- 172.21.15.67:0/1334276081 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f89980cc6c0 con 0x7f89a0108530 2024-07-31T15:10:24.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.844+0000 7f89a739d640 1 -- 172.21.15.67:0/1334276081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89a002e0c0 con 0x7f89a0108530 2024-07-31T15:10:24.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.845+0000 7f89937fe640 1 -- 172.21.15.67:0/1334276081 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11962 (secure 0 0 0) 0x7f89980cbfc0 con 0x7f89a0108530 2024-07-31T15:10:24.847 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:10:24.847 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"btime":"2024-07-31T15:06:04:323642+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:06:04.323492+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28214":{"gid":28214,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1516494761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1516494761},{"type":"v1","addr":"172.21.15.73:6835","nonce":1516494761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24634,24646,24650,24655,24671]},"id":1}]} 2024-07-31T15:10:24.847 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-07-31T15:10:24.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.847+0000 7f89a739d640 1 -- 172.21.15.67:0/1334276081 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f898c0b3d70 msgr2=0x7f898c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:24.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.847+0000 7f89a739d640 1 --2- 172.21.15.67:0/1334276081 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f898c0b3d70 0x7f898c0b6280 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f899c0058c0 tx=0x7f899c00a2d0 comp rx=0 tx=0).stop 2024-07-31T15:10:24.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.847+0000 7f89a739d640 1 -- 172.21.15.67:0/1334276081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89a0108530 msgr2=0x7f89a019b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:24.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.847+0000 7f89a739d640 1 --2- 172.21.15.67:0/1334276081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89a0108530 0x7f89a019b550 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f8998005cc0 tx=0x7f899802f380 comp rx=0 tx=0).stop 2024-07-31T15:10:24.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.847+0000 7f89a739d640 1 -- 172.21.15.67:0/1334276081 shutdown_connections 2024-07-31T15:10:24.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.847+0000 7f89a739d640 1 --2- 172.21.15.67:0/1334276081 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f898c0b3d70 0x7f898c0b6280 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:24.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.847+0000 7f89a739d640 1 --2- 172.21.15.67:0/1334276081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89a0108530 0x7f89a019b550 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:24.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.847+0000 7f89a739d640 1 --2- 172.21.15.67:0/1334276081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89a0107be0 0x7f89a019b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:24.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.847+0000 7f89a739d640 1 --2- 172.21.15.67:0/1334276081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89a01022a0 0x7f89a019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:24.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.847+0000 7f89a739d640 1 -- 172.21.15.67:0/1334276081 >> 172.21.15.67:0/1334276081 conn(0x7f89a00fdcf0 msgr2=0x7f89a0100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:24.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.848+0000 7f89a739d640 1 -- 172.21.15.67:0/1334276081 shutdown_connections 2024-07-31T15:10:24.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:24.848+0000 7f89a739d640 1 -- 172.21.15.67:0/1334276081 wait complete. 2024-07-31T15:10:25.511 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:25 smithi067 ceph-mon[32750]: pgmap v1017: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 204 KiB/s wr, 71 op/s 2024-07-31T15:10:25.511 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:25 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1334276081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:10:25.541 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id dd84c0d1 --release --await' 2024-07-31T15:10:25.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id dd84c0d1 --release --await 2024-07-31T15:10:25.637 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:25 smithi077 ceph-mon[37327]: pgmap v1017: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 204 KiB/s wr, 71 op/s 2024-07-31T15:10:25.638 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:25 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1334276081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:10:25.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:25 smithi073 ceph-mon[36714]: pgmap v1017: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 204 KiB/s wr, 71 op/s 2024-07-31T15:10:25.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:25 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1334276081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:10:26.479 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:10:27.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.093+0000 7f2c8e6e6640 1 -- 172.21.15.67:0/1887941397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c88108720 msgr2=0x7f2c8810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:27.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.093+0000 7f2c8e6e6640 1 --2- 172.21.15.67:0/1887941397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c88108720 0x7f2c8810ef50 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009ca0 tx=0x7f2c8002a8c0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.093+0000 7f2c8e6e6640 1 -- 172.21.15.67:0/1887941397 shutdown_connections 2024-07-31T15:10:27.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.093+0000 7f2c8e6e6640 1 --2- 172.21.15.67:0/1887941397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c88108720 0x7f2c8810ef50 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.093+0000 7f2c8e6e6640 1 --2- 172.21.15.67:0/1887941397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c88107dd0 0x7f2c881081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.093+0000 7f2c8e6e6640 1 --2- 172.21.15.67:0/1887941397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c88102490 0x7f2c881028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.093+0000 7f2c8e6e6640 1 -- 172.21.15.67:0/1887941397 >> 172.21.15.67:0/1887941397 conn(0x7f2c88079160 msgr2=0x7f2c88079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.093+0000 7f2c8e6e6640 1 -- 172.21.15.67:0/1887941397 shutdown_connections 2024-07-31T15:10:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.093+0000 7f2c8e6e6640 1 -- 172.21.15.67:0/1887941397 wait complete. 2024-07-31T15:10:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.093+0000 7f2c8e6e6640 1 Processor -- start 2024-07-31T15:10:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.093+0000 7f2c8e6e6640 1 -- start start 2024-07-31T15:10:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c8e6e6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c88102490 0x7f2c8810f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c8e6e6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c88107dd0 0x7f2c8810ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c8e6e6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c88108720 0x7f2c88110460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c8e6e6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2c881134b0 con 0x7f2c88107dd0 2024-07-31T15:10:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c8e6e6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2c88113330 con 0x7f2c88108720 2024-07-31T15:10:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c8e6e6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2c88113630 con 0x7f2c88102490 2024-07-31T15:10:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c877fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c88107dd0 0x7f2c8810ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c877fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c88107dd0 0x7f2c8810ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44312/0 (socket says 172.21.15.67:44312) 2024-07-31T15:10:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c877fe640 1 -- 172.21.15.67:0/641475208 learned_addr learned my addr 172.21.15.67:0/641475208 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c8cc83640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c88108720 0x7f2c88110460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c87fff640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c88102490 0x7f2c8810f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c877fe640 1 -- 172.21.15.67:0/641475208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c88102490 msgr2=0x7f2c8810f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c877fe640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c88102490 0x7f2c8810f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c877fe640 1 -- 172.21.15.67:0/641475208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c88108720 msgr2=0x7f2c88110460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c877fe640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c88108720 0x7f2c88110460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c877fe640 1 -- 172.21.15.67:0/641475208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c881a8100 con 0x7f2c88107dd0 2024-07-31T15:10:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c8cc83640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c88108720 0x7f2c88110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:10:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.094+0000 7f2c877fe640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c88107dd0 0x7f2c8810ff20 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00e9f0 tx=0x7f2c7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:27.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.095+0000 7f2c857fa640 1 -- 172.21.15.67:0/641475208 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c00ccc0 con 0x7f2c88107dd0 2024-07-31T15:10:27.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.095+0000 7f2c857fa640 1 -- 172.21.15.67:0/641475208 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7c0054d0 con 0x7f2c88107dd0 2024-07-31T15:10:27.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.095+0000 7f2c8e6e6640 1 -- 172.21.15.67:0/641475208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c881a83f0 con 0x7f2c88107dd0 2024-07-31T15:10:27.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.095+0000 7f2c857fa640 1 -- 172.21.15.67:0/641475208 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c010430 con 0x7f2c88107dd0 2024-07-31T15:10:27.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.095+0000 7f2c8e6e6640 1 -- 172.21.15.67:0/641475208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c881a88e0 con 0x7f2c88107dd0 2024-07-31T15:10:27.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.097+0000 7f2c94c37740 1 Processor -- start 2024-07-31T15:10:27.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.097+0000 7f2c94c37740 1 -- start start 2024-07-31T15:10:27.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.097+0000 7f2c857fa640 1 -- 172.21.15.67:0/641475208 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2c7c0046c0 con 0x7f2c88107dd0 2024-07-31T15:10:27.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c94c37740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff9b070 0x55d2f0340570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c94c37740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff721b0 0x55d2f0340ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c94c37740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff7a390 0x55d2f0340ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c94c37740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d2f02b6bb0 con 0x55d2eff9b070 2024-07-31T15:10:27.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c94c37740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d2f02a60b0 con 0x55d2eff7a390 2024-07-31T15:10:27.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c94c37740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d2f02aa0f0 con 0x55d2eff721b0 2024-07-31T15:10:27.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c8cc83640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff9b070 0x55d2f0340570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c8cc83640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff9b070 0x55d2f0340570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44316/0 (socket says 172.21.15.67:44316) 2024-07-31T15:10:27.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c8cc83640 1 -- 172.21.15.67:0/2688405996 learned_addr learned my addr 172.21.15.67:0/2688405996 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:27.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c877fe640 1 --2- 172.21.15.67:0/2688405996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff7a390 0x55d2f0340ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c87fff640 1 --2- 172.21.15.67:0/2688405996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff721b0 0x55d2f0340ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c8cc83640 1 -- 172.21.15.67:0/2688405996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff721b0 msgr2=0x55d2f0340ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:27.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c8cc83640 1 --2- 172.21.15.67:0/2688405996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff721b0 0x55d2f0340ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c8cc83640 1 -- 172.21.15.67:0/2688405996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff7a390 msgr2=0x55d2f0340ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:27.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c8cc83640 1 --2- 172.21.15.67:0/2688405996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff7a390 0x55d2f0340ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c8cc83640 1 -- 172.21.15.67:0/2688405996 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2f024cb90 con 0x55d2eff9b070 2024-07-31T15:10:27.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c877fe640 1 --2- 172.21.15.67:0/2688405996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff7a390 0x55d2f0340ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:10:27.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c857fa640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2c700b3d40 0x7f2c700b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.098+0000 7f2c8cc83640 1 --2- 172.21.15.67:0/2688405996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff9b070 0x55d2f0340570 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009470 tx=0x7f2c800094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1234e350d39c7cbe server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:27.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.099+0000 7f2c87fff640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2c700b3d40 0x7f2c700b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.099+0000 7f2c857fa640 1 -- 172.21.15.67:0/641475208 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9379+0+0 (secure 0 0 0) 0x7f2c7c0e4810 con 0x7f2c88107dd0 2024-07-31T15:10:27.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.099+0000 7f2c6e7fc640 1 -- 172.21.15.67:0/2688405996 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80031cf0 con 0x55d2eff9b070 2024-07-31T15:10:27.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.099+0000 7f2c6e7fc640 1 -- 172.21.15.67:0/2688405996 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c80036040 con 0x55d2eff9b070 2024-07-31T15:10:27.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.099+0000 7f2c6e7fc640 1 -- 172.21.15.67:0/2688405996 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c800042a0 con 0x55d2eff9b070 2024-07-31T15:10:27.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.100+0000 7f2c94c37740 1 -- 172.21.15.67:0/2688405996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff9b070 msgr2=0x55d2f0340570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:27.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.100+0000 7f2c87fff640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2c700b3d40 0x7f2c700b6250 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x55d2f033f1f0 tx=0x7f2c78005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:27.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.100+0000 7f2c94c37740 1 --2- 172.21.15.67:0/2688405996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff9b070 0x55d2f0340570 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009470 tx=0x7f2c800094a0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.100+0000 7f2c94c37740 1 -- 172.21.15.67:0/2688405996 shutdown_connections 2024-07-31T15:10:27.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.100+0000 7f2c94c37740 1 --2- 172.21.15.67:0/2688405996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff7a390 0x55d2f0340ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.100+0000 7f2c94c37740 1 --2- 172.21.15.67:0/2688405996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff721b0 0x55d2f0340ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.100+0000 7f2c94c37740 1 --2- 172.21.15.67:0/2688405996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff9b070 0x55d2f0340570 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.100+0000 7f2c94c37740 1 -- 172.21.15.67:0/2688405996 >> 172.21.15.67:0/2688405996 conn(0x55d2eff8f460 msgr2=0x55d2f033e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:27.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.100+0000 7f2c94c37740 1 -- 172.21.15.67:0/2688405996 shutdown_connections 2024-07-31T15:10:27.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.100+0000 7f2c94c37740 1 -- 172.21.15.67:0/2688405996 wait complete. 2024-07-31T15:10:27.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.100+0000 7f2c94c37740 1 -- start start 2024-07-31T15:10:27.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.100+0000 7f2c94c37740 1 Processor -- start 2024-07-31T15:10:27.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c94c37740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff721b0 0x55d2f034c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c94c37740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff7a390 0x55d2f034c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c94c37740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff9b070 0x55d2f0357fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c94c37740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d2f0109700 con 0x55d2eff7a390 2024-07-31T15:10:27.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c94c37740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d2f02489e0 con 0x55d2eff721b0 2024-07-31T15:10:27.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c94c37740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d2f034ecf0 con 0x55d2eff9b070 2024-07-31T15:10:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c94c37740 10 client.0 _ll_register_callbacks cb 0x55d2f02634f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c877fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff7a390 0x55d2f034c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c877fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff7a390 0x55d2f034c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44322/0 (socket says 172.21.15.67:44322) 2024-07-31T15:10:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c877fe640 1 -- 172.21.15.67:0/2162420388 learned_addr learned my addr 172.21.15.67:0/2162420388 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c8cc83640 1 --2- 172.21.15.67:0/2162420388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff9b070 0x55d2f0357fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c87fff640 1 --2- 172.21.15.67:0/2162420388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff721b0 0x55d2f034c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c877fe640 1 -- 172.21.15.67:0/2162420388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff9b070 msgr2=0x55d2f0357fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c877fe640 1 --2- 172.21.15.67:0/2162420388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff9b070 0x55d2f0357fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c877fe640 1 -- 172.21.15.67:0/2162420388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff721b0 msgr2=0x55d2f034c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c877fe640 1 --2- 172.21.15.67:0/2162420388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff721b0 0x55d2f034c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c877fe640 1 -- 172.21.15.67:0/2162420388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2f0006650 con 0x55d2eff7a390 2024-07-31T15:10:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c877fe640 1 --2- 172.21.15.67:0/2162420388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff7a390 0x55d2f034c680 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00c6a0 tx=0x7f2c7c0aaa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07fad337a8c7faf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:27.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c677fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:10:27.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.101+0000 7f2c677fe640 1 -- 172.21.15.67:0/2162420388 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c0aabd0 con 0x55d2eff7a390 2024-07-31T15:10:27.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.102+0000 7f2c677fe640 1 -- 172.21.15.67:0/2162420388 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9379+0+0 (secure 0 0 0) 0x7f2c7c0ab530 con 0x55d2eff7a390 2024-07-31T15:10:27.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.102+0000 7f2c94c37740 1 -- 172.21.15.67:0/2162420388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c80036040 con 0x55d2eff7a390 2024-07-31T15:10:27.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.102+0000 7f2c677fe640 1 -- 172.21.15.67:0/2162420388 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f2c7c0ab7c0 con 0x55d2eff7a390 2024-07-31T15:10:27.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.102+0000 7f2c94c37740 10 client.19557 fetch_fsmap learned FSMap version 65 2024-07-31T15:10:27.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.102+0000 7f2c94c37740 1 -- 172.21.15.67:0/2162420388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55d2f035ad10 con 0x55d2eff7a390 2024-07-31T15:10:27.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.102+0000 7f2c677fe640 1 -- 172.21.15.67:0/2162420388 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c80036040 con 0x55d2eff7a390 2024-07-31T15:10:27.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.102+0000 7f2c677fe640 1 -- 172.21.15.67:0/2162420388 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f2c7c005000 con 0x55d2eff7a390 2024-07-31T15:10:27.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.102+0000 7f2c94c37740 10 client.19557 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:10:27.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.102+0000 7f2c94c37740 10 client.19557 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:10:27.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.102+0000 7f2c94c37740 20 client.19557 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:27.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.102+0000 7f2c94c37740 1 --2- 172.21.15.67:0/2162420388 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55d2eff599f0 0x55d2f035e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.102+0000 7f2c94c37740 4 client.19557 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:10:27.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.102+0000 7f2c94c37740 1 -- 172.21.15.67:0/2162420388 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2f02b8b00 con 0x55d2eff599f0 2024-07-31T15:10:27.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.102+0000 7f2c87fff640 1 --2- 172.21.15.67:0/2162420388 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55d2eff599f0 0x55d2f035e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.103+0000 7f2c677fe640 10 client.19557 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:10:27.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.103+0000 7f2c87fff640 1 --2- 172.21.15.67:0/2162420388 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55d2eff599f0 0x55d2f035e220 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=238fe266e8728ce7 server_cookie=91c25a9216894542 in_seq=0 out_seq=0 2024-07-31T15:10:27.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c677fe640 1 -- 172.21.15.67:0/2162420388 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2f02b8b00 con 0x55d2eff599f0 2024-07-31T15:10:27.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c677fe640 10 client.19557 handle_command_reply: tid=0 2024-07-31T15:10:27.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c94c37740 1 client.19557 shutdown 2024-07-31T15:10:27.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c94c37740 1 -- 172.21.15.67:0/2162420388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff7a390 msgr2=0x55d2f034c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:27.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c94c37740 1 --2- 172.21.15.67:0/2162420388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff7a390 0x55d2f034c680 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00c6a0 tx=0x7f2c7c0aaa10 comp rx=0 tx=0).stop 2024-07-31T15:10:27.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c94c37740 1 -- 172.21.15.67:0/2162420388 shutdown_connections 2024-07-31T15:10:27.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c94c37740 1 --2- 172.21.15.67:0/2162420388 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55d2eff599f0 0x55d2f035e220 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c94c37740 1 --2- 172.21.15.67:0/2162420388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff9b070 0x55d2f0357fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c94c37740 1 --2- 172.21.15.67:0/2162420388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff7a390 0x55d2f034c680 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c677fe640 0 client.19557 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:10:27.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c94c37740 1 --2- 172.21.15.67:0/2162420388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff721b0 0x55d2f034c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c94c37740 1 -- 172.21.15.67:0/2162420388 >> 172.21.15.67:0/2162420388 conn(0x55d2eff8f460 msgr2=0x55d2f0343750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:27.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c94c37740 1 -- 172.21.15.67:0/2162420388 shutdown_connections 2024-07-31T15:10:27.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c94c37740 1 -- 172.21.15.67:0/2162420388 wait complete. 2024-07-31T15:10:27.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.104+0000 7f2c94c37740 20 client.19557 trim_cache size 0 max 16384 2024-07-31T15:10:27.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.119+0000 7f2c94c37740 1 Processor -- start 2024-07-31T15:10:27.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.119+0000 7f2c94c37740 1 -- start start 2024-07-31T15:10:27.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.119+0000 7f2c94c37740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff721b0 0x55d2f0357dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.119+0000 7f2c94c37740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff7a390 0x55d2f033d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.119+0000 7f2c94c37740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff9b070 0x55d2f0345650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.119+0000 7f2c94c37740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2c881b29c0 con 0x55d2eff9b070 2024-07-31T15:10:27.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.119+0000 7f2c94c37740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2c881b4500 con 0x55d2eff721b0 2024-07-31T15:10:27.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.119+0000 7f2c94c37740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2c881b3090 con 0x55d2eff7a390 2024-07-31T15:10:27.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.119+0000 7f2c8cc83640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff9b070 0x55d2f0345650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.119+0000 7f2c8cc83640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff9b070 0x55d2f0345650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44324/0 (socket says 172.21.15.67:44324) 2024-07-31T15:10:27.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.119+0000 7f2c8cc83640 1 -- 172.21.15.67:0/3091937385 learned_addr learned my addr 172.21.15.67:0/3091937385 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:27.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.119+0000 7f2c877fe640 1 --2- 172.21.15.67:0/3091937385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff7a390 0x55d2f033d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.119+0000 7f2c87fff640 1 --2- 172.21.15.67:0/3091937385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff721b0 0x55d2f0357dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.120+0000 7f2c8cc83640 1 -- 172.21.15.67:0/3091937385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff7a390 msgr2=0x55d2f033d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:27.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.120+0000 7f2c8cc83640 1 --2- 172.21.15.67:0/3091937385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff7a390 0x55d2f033d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.120+0000 7f2c8cc83640 1 -- 172.21.15.67:0/3091937385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff721b0 msgr2=0x55d2f0357dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:27.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.120+0000 7f2c8cc83640 1 --2- 172.21.15.67:0/3091937385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff721b0 0x55d2f0357dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.120+0000 7f2c8cc83640 1 -- 172.21.15.67:0/3091937385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2f03433d0 con 0x55d2eff9b070 2024-07-31T15:10:27.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.120+0000 7f2c8cc83640 1 --2- 172.21.15.67:0/3091937385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff9b070 0x55d2f0345650 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009800 tx=0x7f2c8002afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdbd22fb21bc165 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.120+0000 7f2c6dffb640 1 -- 172.21.15.67:0/3091937385 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80053020 con 0x55d2eff9b070 2024-07-31T15:10:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.120+0000 7f2c6dffb640 1 -- 172.21.15.67:0/3091937385 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c80036040 con 0x55d2eff9b070 2024-07-31T15:10:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.120+0000 7f2c6dffb640 1 -- 172.21.15.67:0/3091937385 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8004a040 con 0x55d2eff9b070 2024-07-31T15:10:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.122+0000 7f2c94c37740 1 -- 172.21.15.67:0/3091937385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff9b070 msgr2=0x55d2f0345650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.122+0000 7f2c94c37740 1 --2- 172.21.15.67:0/3091937385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff9b070 0x55d2f0345650 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009800 tx=0x7f2c8002afc0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.122+0000 7f2c94c37740 1 -- 172.21.15.67:0/3091937385 shutdown_connections 2024-07-31T15:10:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.122+0000 7f2c94c37740 1 --2- 172.21.15.67:0/3091937385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff9b070 0x55d2f0345650 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.122+0000 7f2c94c37740 1 --2- 172.21.15.67:0/3091937385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff7a390 0x55d2f033d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.122+0000 7f2c94c37740 1 --2- 172.21.15.67:0/3091937385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff721b0 0x55d2f0357dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.122+0000 7f2c94c37740 1 -- 172.21.15.67:0/3091937385 >> 172.21.15.67:0/3091937385 conn(0x55d2eff8f460 msgr2=0x55d2f033cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:27.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.122+0000 7f2c94c37740 1 -- 172.21.15.67:0/3091937385 shutdown_connections 2024-07-31T15:10:27.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.122+0000 7f2c94c37740 1 -- 172.21.15.67:0/3091937385 wait complete. 2024-07-31T15:10:27.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.122+0000 7f2c94c37740 1 -- start start 2024-07-31T15:10:27.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.123+0000 7f2c94c37740 1 Processor -- start 2024-07-31T15:10:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.123+0000 7f2c94c37740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff721b0 0x55d2f0353f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.123+0000 7f2c94c37740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff7a390 0x55d2f0354490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.123+0000 7f2c94c37740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff9b070 0x55d2f034ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.123+0000 7f2c94c37740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2c881b3e30 con 0x55d2eff721b0 2024-07-31T15:10:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.123+0000 7f2c94c37740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2c881b3760 con 0x55d2eff7a390 2024-07-31T15:10:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.123+0000 7f2c94c37740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2c881b09c0 con 0x55d2eff9b070 2024-07-31T15:10:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.123+0000 7f2c94c37740 10 client.0 _ll_register_callbacks cb 0x7f2c7c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.123+0000 7f2c87fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff721b0 0x55d2f0353f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.123+0000 7f2c87fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff721b0 0x55d2f0353f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44332/0 (socket says 172.21.15.67:44332) 2024-07-31T15:10:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.123+0000 7f2c87fff640 1 -- 172.21.15.67:0/3148557988 learned_addr learned my addr 172.21.15.67:0/3148557988 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.123+0000 7f2c8cc83640 1 --2- 172.21.15.67:0/3148557988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff9b070 0x55d2f034ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.123+0000 7f2c877fe640 1 --2- 172.21.15.67:0/3148557988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff7a390 0x55d2f0354490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.124+0000 7f2c87fff640 1 -- 172.21.15.67:0/3148557988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff9b070 msgr2=0x55d2f034ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:27.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.124+0000 7f2c87fff640 1 --2- 172.21.15.67:0/3148557988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff9b070 0x55d2f034ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.124+0000 7f2c87fff640 1 -- 172.21.15.67:0/3148557988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff7a390 msgr2=0x55d2f0354490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:27.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.124+0000 7f2c87fff640 1 --2- 172.21.15.67:0/3148557988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff7a390 0x55d2f0354490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.124+0000 7f2c87fff640 1 -- 172.21.15.67:0/3148557988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2f0354ad0 con 0x55d2eff721b0 2024-07-31T15:10:27.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.124+0000 7f2c87fff640 1 --2- 172.21.15.67:0/3148557988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff721b0 0x55d2f0353f50 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x55d2f034e790 tx=0x7f2c780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776b64e679aee0de server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:27.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.124+0000 7f2c677fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:10:27.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.124+0000 7f2c677fe640 1 -- 172.21.15.67:0/3148557988 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7801fe10 con 0x55d2eff721b0 2024-07-31T15:10:27.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.124+0000 7f2c677fe640 1 -- 172.21.15.67:0/3148557988 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9379+0+0 (secure 0 0 0) 0x7f2c78004030 con 0x55d2eff721b0 2024-07-31T15:10:27.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.124+0000 7f2c94c37740 1 -- 172.21.15.67:0/3148557988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c80036040 con 0x55d2eff721b0 2024-07-31T15:10:27.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.125+0000 7f2c677fe640 1 -- 172.21.15.67:0/3148557988 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f2c7803a050 con 0x55d2eff721b0 2024-07-31T15:10:27.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.125+0000 7f2c94c37740 10 client.19563 fetch_fsmap learned FSMap version 65 2024-07-31T15:10:27.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.125+0000 7f2c94c37740 1 -- 172.21.15.67:0/3148557988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55d2f03cf730 con 0x55d2eff721b0 2024-07-31T15:10:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.125+0000 7f2c677fe640 1 -- 172.21.15.67:0/3148557988 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c80036040 con 0x55d2eff721b0 2024-07-31T15:10:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.125+0000 7f2c677fe640 1 -- 172.21.15.67:0/3148557988 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f2c78033060 con 0x55d2eff721b0 2024-07-31T15:10:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.125+0000 7f2c94c37740 10 client.19563 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:10:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.125+0000 7f2c94c37740 10 client.19563 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:10:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.125+0000 7f2c94c37740 20 client.19563 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.125+0000 7f2c94c37740 1 --2- 172.21.15.67:0/3148557988 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55d2eff2eda0 0x55d2f03d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.125+0000 7f2c94c37740 4 client.19563 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:10:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.125+0000 7f2c94c37740 1 -- 172.21.15.67:0/3148557988 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2f034ca10 con 0x55d2eff2eda0 2024-07-31T15:10:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.125+0000 7f2c877fe640 1 --2- 172.21.15.67:0/3148557988 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55d2eff2eda0 0x55d2f03d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.126+0000 7f2c677fe640 10 client.19563 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:10:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.126+0000 7f2c877fe640 1 --2- 172.21.15.67:0/3148557988 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55d2eff2eda0 0x55d2f03d1f20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d4e5eb85321939c server_cookie=a19ca75760ec971c in_seq=0 out_seq=0 2024-07-31T15:10:27.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.126+0000 7f2c677fe640 1 -- 172.21.15.67:0/3148557988 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2f034ca10 con 0x55d2eff2eda0 2024-07-31T15:10:27.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.126+0000 7f2c677fe640 10 client.19563 handle_command_reply: tid=0 2024-07-31T15:10:27.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.126+0000 7f2c94c37740 1 client.19563 shutdown 2024-07-31T15:10:27.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.126+0000 7f2c94c37740 1 -- 172.21.15.67:0/3148557988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff721b0 msgr2=0x55d2f0353f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:27.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.126+0000 7f2c94c37740 1 --2- 172.21.15.67:0/3148557988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff721b0 0x55d2f0353f50 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x55d2f034e790 tx=0x7f2c780046c0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.126+0000 7f2c94c37740 1 -- 172.21.15.67:0/3148557988 shutdown_connections 2024-07-31T15:10:27.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.126+0000 7f2c94c37740 1 --2- 172.21.15.67:0/3148557988 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55d2eff2eda0 0x55d2f03d1f20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.126+0000 7f2c94c37740 1 --2- 172.21.15.67:0/3148557988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2eff9b070 0x55d2f034ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.132 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:10:27.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.126+0000 7f2c94c37740 1 --2- 172.21.15.67:0/3148557988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2eff7a390 0x55d2f0354490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.126+0000 7f2c94c37740 1 --2- 172.21.15.67:0/3148557988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d2eff721b0 0x55d2f0353f50 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.126+0000 7f2c94c37740 1 -- 172.21.15.67:0/3148557988 >> 172.21.15.67:0/3148557988 conn(0x55d2eff8f460 msgr2=0x55d2f0341580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:27.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.126+0000 7f2c677fe640 0 client.19563 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:10:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.127+0000 7f2c94c37740 1 -- 172.21.15.67:0/3148557988 shutdown_connections 2024-07-31T15:10:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.127+0000 7f2c94c37740 1 -- 172.21.15.67:0/3148557988 wait complete. 2024-07-31T15:10:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.127+0000 7f2c94c37740 20 client.19563 trim_cache size 0 max 16384 2024-07-31T15:10:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.127+0000 7f2c8e6e6640 1 -- 172.21.15.67:0/641475208 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2c700b3d40 msgr2=0x7f2c700b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.127+0000 7f2c8e6e6640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2c700b3d40 0x7f2c700b6250 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x55d2f033f1f0 tx=0x7f2c78005f90 comp rx=0 tx=0).stop 2024-07-31T15:10:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.127+0000 7f2c8e6e6640 1 -- 172.21.15.67:0/641475208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c88107dd0 msgr2=0x7f2c8810ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.127+0000 7f2c8e6e6640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c88107dd0 0x7f2c8810ff20 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00e9f0 tx=0x7f2c7c00eeb0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.127+0000 7f2c8e6e6640 1 -- 172.21.15.67:0/641475208 shutdown_connections 2024-07-31T15:10:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.127+0000 7f2c8e6e6640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2c700b3d40 0x7f2c700b6250 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.127+0000 7f2c8e6e6640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c88108720 0x7f2c88110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.127+0000 7f2c8e6e6640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c88107dd0 0x7f2c8810ff20 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.127+0000 7f2c8e6e6640 1 --2- 172.21.15.67:0/641475208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c88102490 0x7f2c8810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.127+0000 7f2c8e6e6640 1 -- 172.21.15.67:0/641475208 >> 172.21.15.67:0/641475208 conn(0x7f2c88079160 msgr2=0x7f2c88104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.128+0000 7f2c8e6e6640 1 -- 172.21.15.67:0/641475208 shutdown_connections 2024-07-31T15:10:27.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:27.128+0000 7f2c8e6e6640 1 -- 172.21.15.67:0/641475208 wait complete. 2024-07-31T15:10:27.399 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:27 smithi067 ceph-mon[32750]: pgmap v1018: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 80 KiB/s wr, 53 op/s 2024-07-31T15:10:27.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:27 smithi073 ceph-mon[36714]: pgmap v1018: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 80 KiB/s wr, 53 op/s 2024-07-31T15:10:27.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:27 smithi077 ceph-mon[37327]: pgmap v1018: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 80 KiB/s wr, 53 op/s 2024-07-31T15:10:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.242+0000 7f3cabfa9640 1 -- 172.21.15.67:0/548331620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca4108670 msgr2=0x7f3ca410eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.242+0000 7f3cabfa9640 1 --2- 172.21.15.67:0/548331620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca4108670 0x7f3ca410eea0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009ca0 tx=0x7f3c9c02a8c0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.242+0000 7f3cabfa9640 1 -- 172.21.15.67:0/548331620 shutdown_connections 2024-07-31T15:10:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.242+0000 7f3cabfa9640 1 --2- 172.21.15.67:0/548331620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca4108670 0x7f3ca410eea0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.242+0000 7f3cabfa9640 1 --2- 172.21.15.67:0/548331620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ca4107d20 0x7f3ca4108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.242+0000 7f3cabfa9640 1 --2- 172.21.15.67:0/548331620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ca41023f0 0x7f3ca4102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.242+0000 7f3cabfa9640 1 -- 172.21.15.67:0/548331620 >> 172.21.15.67:0/548331620 conn(0x7f3ca40fddf0 msgr2=0x7f3ca4100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.243+0000 7f3cabfa9640 1 -- 172.21.15.67:0/548331620 shutdown_connections 2024-07-31T15:10:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.243+0000 7f3cabfa9640 1 -- 172.21.15.67:0/548331620 wait complete. 2024-07-31T15:10:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.243+0000 7f3cabfa9640 1 Processor -- start 2024-07-31T15:10:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.243+0000 7f3cabfa9640 1 -- start start 2024-07-31T15:10:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3cabfa9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ca41023f0 0x7f3ca419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3cabfa9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ca4107d20 0x7f3ca419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3cabfa9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca4108670 0x7f3ca419b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3cabfa9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4112340 con 0x7f3ca4108670 2024-07-31T15:10:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3cabfa9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41121c0 con 0x7f3ca4107d20 2024-07-31T15:10:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3cabfa9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41124c0 con 0x7f3ca41023f0 2024-07-31T15:10:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3caa546640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca4108670 0x7f3ca419b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3caa546640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca4108670 0x7f3ca419b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44352/0 (socket says 172.21.15.67:44352) 2024-07-31T15:10:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3caa546640 1 -- 172.21.15.67:0/3434635164 learned_addr learned my addr 172.21.15.67:0/3434635164 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3ca9544640 1 --2- 172.21.15.67:0/3434635164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ca4107d20 0x7f3ca419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3ca9d45640 1 --2- 172.21.15.67:0/3434635164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ca41023f0 0x7f3ca419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3caa546640 1 -- 172.21.15.67:0/3434635164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ca41023f0 msgr2=0x7f3ca419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3caa546640 1 --2- 172.21.15.67:0/3434635164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ca41023f0 0x7f3ca419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3caa546640 1 -- 172.21.15.67:0/3434635164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ca4107d20 msgr2=0x7f3ca419b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3caa546640 1 --2- 172.21.15.67:0/3434635164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ca4107d20 0x7f3ca419b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.244+0000 7f3caa546640 1 -- 172.21.15.67:0/3434635164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca41a5ee0 con 0x7f3ca4108670 2024-07-31T15:10:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.245+0000 7f3caa546640 1 --2- 172.21.15.67:0/3434635164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca4108670 0x7f3ca419b650 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c005cc0 tx=0x7f3c9c02f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.248+0000 7f3c9affd640 1 -- 172.21.15.67:0/3434635164 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c003cf0 con 0x7f3ca4108670 2024-07-31T15:10:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.248+0000 7f3c9affd640 1 -- 172.21.15.67:0/3434635164 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c9c0044f0 con 0x7f3ca4108670 2024-07-31T15:10:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.248+0000 7f3cabfa9640 1 -- 172.21.15.67:0/3434635164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca41a6170 con 0x7f3ca4108670 2024-07-31T15:10:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.248+0000 7f3c9affd640 1 -- 172.21.15.67:0/3434635164 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c02fd10 con 0x7f3ca4108670 2024-07-31T15:10:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.248+0000 7f3cabfa9640 1 -- 172.21.15.67:0/3434635164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca41a6710 con 0x7f3ca4108670 2024-07-31T15:10:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.251+0000 7f3c9affd640 1 -- 172.21.15.67:0/3434635164 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3c9c0305c0 con 0x7f3ca4108670 2024-07-31T15:10:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.251+0000 7f3cb2f7b740 1 Processor -- start 2024-07-31T15:10:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.251+0000 7f3cb2f7b740 1 -- start start 2024-07-31T15:10:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3cb2f7b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b515e1b0 0x55b4b5500ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3cb2f7b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b51352f0 0x55b4b5501220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3cb2f7b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b513d4d0 0x55b4b5501760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3cb2f7b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b4b54763b0 con 0x55b4b51352f0 2024-07-31T15:10:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3cb2f7b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b4b54658b0 con 0x55b4b513d4d0 2024-07-31T15:10:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3cb2f7b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b4b54698f0 con 0x55b4b515e1b0 2024-07-31T15:10:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3ca9d45640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b51352f0 0x55b4b5501220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3ca9d45640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b51352f0 0x55b4b5501220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44366/0 (socket says 172.21.15.67:44366) 2024-07-31T15:10:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3ca9d45640 1 -- 172.21.15.67:0/1954851890 learned_addr learned my addr 172.21.15.67:0/1954851890 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3ca9544640 1 --2- 172.21.15.67:0/1954851890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b513d4d0 0x55b4b5501760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3c9affd640 1 --2- 172.21.15.67:0/3434635164 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3c900b3dc0 0x7f3c900b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3caa546640 1 --2- 172.21.15.67:0/3434635164 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3c900b3dc0 0x7f3c900b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3ca9544640 1 --2- 172.21.15.67:0/1954851890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b515e1b0 0x55b4b5500ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3ca9d45640 1 -- 172.21.15.67:0/1954851890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b515e1b0 msgr2=0x55b4b5500ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3ca9d45640 1 --2- 172.21.15.67:0/1954851890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b515e1b0 0x55b4b5500ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3ca9d45640 1 -- 172.21.15.67:0/1954851890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b513d4d0 msgr2=0x55b4b5501760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3ca9d45640 1 --2- 172.21.15.67:0/1954851890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b513d4d0 0x55b4b5501760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3c9affd640 1 -- 172.21.15.67:0/3434635164 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9379+0+0 (secure 0 0 0) 0x7f3c9c105a20 con 0x7f3ca4108670 2024-07-31T15:10:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3ca9d45640 1 -- 172.21.15.67:0/1954851890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4b540c9a0 con 0x55b4b51352f0 2024-07-31T15:10:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.252+0000 7f3ca9544640 1 --2- 172.21.15.67:0/1954851890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b513d4d0 0x55b4b5501760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.253+0000 7f3ca9d45640 1 --2- 172.21.15.67:0/1954851890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b51352f0 0x55b4b5501220 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000abe0 tx=0x7f3ca000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5036f85f5138bc8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.253+0000 7f3caa546640 1 --2- 172.21.15.67:0/3434635164 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3c900b3dc0 0x7f3c900b62d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0cc310 tx=0x7f3c9c0996d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.253+0000 7f3c8ffff640 1 -- 172.21.15.67:0/1954851890 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0005220 con 0x55b4b51352f0 2024-07-31T15:10:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.253+0000 7f3c8ffff640 1 -- 172.21.15.67:0/1954851890 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca00047a0 con 0x55b4b51352f0 2024-07-31T15:10:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.253+0000 7f3c8ffff640 1 -- 172.21.15.67:0/1954851890 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0018660 con 0x55b4b51352f0 2024-07-31T15:10:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.254+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/1954851890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b51352f0 msgr2=0x55b4b5501220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.254+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/1954851890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b51352f0 0x55b4b5501220 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000abe0 tx=0x7f3ca000f790 comp rx=0 tx=0).stop 2024-07-31T15:10:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.254+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/1954851890 shutdown_connections 2024-07-31T15:10:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.254+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/1954851890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b513d4d0 0x55b4b5501760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.254+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/1954851890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b51352f0 0x55b4b5501220 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.254+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/1954851890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b515e1b0 0x55b4b5500ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.254+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/1954851890 >> 172.21.15.67:0/1954851890 conn(0x55b4b5152300 msgr2=0x55b4b54febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.254+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/1954851890 shutdown_connections 2024-07-31T15:10:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.254+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/1954851890 wait complete. 2024-07-31T15:10:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.254+0000 7f3cb2f7b740 1 -- start start 2024-07-31T15:10:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.254+0000 7f3cb2f7b740 1 Processor -- start 2024-07-31T15:10:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3cb2f7b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b51352f0 0x55b4b550c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3cb2f7b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b513d4d0 0x55b4b550ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3cb2f7b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b515e1b0 0x55b4b5518780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3cb2f7b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b4b539fab0 con 0x55b4b515e1b0 2024-07-31T15:10:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3cb2f7b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b4b53edcf0 con 0x55b4b513d4d0 2024-07-31T15:10:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3cb2f7b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b4b550f460 con 0x55b4b51352f0 2024-07-31T15:10:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3cb2f7b740 10 client.0 _ll_register_callbacks cb 0x55b4b545f9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3ca9544640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b515e1b0 0x55b4b5518780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3ca9544640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b515e1b0 0x55b4b5518780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44382/0 (socket says 172.21.15.67:44382) 2024-07-31T15:10:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3ca9544640 1 -- 172.21.15.67:0/4122574262 learned_addr learned my addr 172.21.15.67:0/4122574262 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3ca9d45640 1 --2- 172.21.15.67:0/4122574262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b513d4d0 0x55b4b550ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3ca9544640 1 --2- 172.21.15.67:0/4122574262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b51352f0 0x55b4b550c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3ca9544640 1 -- 172.21.15.67:0/4122574262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b51352f0 msgr2=0x55b4b550c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3ca9544640 1 --2- 172.21.15.67:0/4122574262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b51352f0 0x55b4b550c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3ca9544640 1 -- 172.21.15.67:0/4122574262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b513d4d0 msgr2=0x55b4b550ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3ca9544640 1 --2- 172.21.15.67:0/4122574262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b513d4d0 0x55b4b550ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3ca9544640 1 -- 172.21.15.67:0/4122574262 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4b51c9790 con 0x55b4b515e1b0 2024-07-31T15:10:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3ca9544640 1 --2- 172.21.15.67:0/4122574262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b515e1b0 0x55b4b5518780 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400a870 tx=0x7f3c94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7b0308841ffda3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.255+0000 7f3c8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:10:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.256+0000 7f3c8d7fa640 1 -- 172.21.15.67:0/4122574262 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94005020 con 0x55b4b515e1b0 2024-07-31T15:10:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.256+0000 7f3c8d7fa640 1 -- 172.21.15.67:0/4122574262 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9379+0+0 (secure 0 0 0) 0x7f3c940051c0 con 0x55b4b515e1b0 2024-07-31T15:10:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.256+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/4122574262 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ca00047a0 con 0x55b4b515e1b0 2024-07-31T15:10:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.256+0000 7f3c8d7fa640 1 -- 172.21.15.67:0/4122574262 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f3c94027050 con 0x55b4b515e1b0 2024-07-31T15:10:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.256+0000 7f3cb2f7b740 10 client.19575 fetch_fsmap learned FSMap version 65 2024-07-31T15:10:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.256+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/4122574262 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55b4b5242620 con 0x55b4b515e1b0 2024-07-31T15:10:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.256+0000 7f3c8d7fa640 1 -- 172.21.15.67:0/4122574262 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca00047a0 con 0x55b4b515e1b0 2024-07-31T15:10:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.256+0000 7f3c8d7fa640 1 -- 172.21.15.67:0/4122574262 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f3c9400f420 con 0x55b4b515e1b0 2024-07-31T15:10:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.256+0000 7f3cb2f7b740 10 client.19575 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:10:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.256+0000 7f3cb2f7b740 10 client.19575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.256+0000 7f3cb2f7b740 20 client.19575 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.256+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/4122574262 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4b511cd80 0x55b4b551e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.256+0000 7f3cb2f7b740 4 client.19575 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:10:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.256+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/4122574262 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4b5478300 con 0x55b4b511cd80 2024-07-31T15:10:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.257+0000 7f3ca9d45640 1 --2- 172.21.15.67:0/4122574262 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4b511cd80 0x55b4b551e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.257+0000 7f3c8d7fa640 10 client.19575 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.257+0000 7f3ca9d45640 1 --2- 172.21.15.67:0/4122574262 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4b511cd80 0x55b4b551e710 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98962cdd7460b896 server_cookie=6a5ab178a709abab in_seq=0 out_seq=0 2024-07-31T15:10:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3c8d7fa640 1 -- 172.21.15.67:0/4122574262 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4b5478300 con 0x55b4b511cd80 2024-07-31T15:10:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3c8d7fa640 10 client.19575 handle_command_reply: tid=0 2024-07-31T15:10:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3cb2f7b740 1 client.19575 shutdown 2024-07-31T15:10:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/4122574262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b515e1b0 msgr2=0x55b4b5518780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/4122574262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b515e1b0 0x55b4b5518780 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400a870 tx=0x7f3c94004730 comp rx=0 tx=0).stop 2024-07-31T15:10:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/4122574262 shutdown_connections 2024-07-31T15:10:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/4122574262 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4b511cd80 0x55b4b551e710 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/4122574262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b515e1b0 0x55b4b5518780 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3c8d7fa640 0 client.19575 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/4122574262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b513d4d0 0x55b4b550ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/4122574262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b51352f0 0x55b4b550c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/4122574262 >> 172.21.15.67:0/4122574262 conn(0x55b4b5152300 msgr2=0x55b4b54fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/4122574262 shutdown_connections 2024-07-31T15:10:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/4122574262 wait complete. 2024-07-31T15:10:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.258+0000 7f3cb2f7b740 20 client.19575 trim_cache size 0 max 16384 2024-07-31T15:10:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3cb2f7b740 1 Processor -- start 2024-07-31T15:10:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3cb2f7b740 1 -- start start 2024-07-31T15:10:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3cb2f7b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b511cd80 0x55b4b5506e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3cb2f7b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b51352f0 0x55b4b550bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3cb2f7b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b513d4d0 0x55b4b54fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3cb2f7b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41b0780 con 0x55b4b513d4d0 2024-07-31T15:10:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3cb2f7b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41b22c0 con 0x55b4b511cd80 2024-07-31T15:10:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3cb2f7b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41b0e50 con 0x55b4b51352f0 2024-07-31T15:10:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3ca9544640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b513d4d0 0x55b4b54fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3ca9544640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b513d4d0 0x55b4b54fe8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44384/0 (socket says 172.21.15.67:44384) 2024-07-31T15:10:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3ca9544640 1 -- 172.21.15.67:0/3149539787 learned_addr learned my addr 172.21.15.67:0/3149539787 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3ca9544640 1 --2- 172.21.15.67:0/3149539787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b511cd80 0x55b4b5506e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3ca9d45640 1 --2- 172.21.15.67:0/3149539787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b51352f0 0x55b4b550bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3ca9544640 1 -- 172.21.15.67:0/3149539787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b51352f0 msgr2=0x55b4b550bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3ca9544640 1 --2- 172.21.15.67:0/3149539787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b51352f0 0x55b4b550bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3ca9544640 1 -- 172.21.15.67:0/3149539787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b511cd80 msgr2=0x55b4b5506e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3ca9544640 1 --2- 172.21.15.67:0/3149539787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b511cd80 0x55b4b5506e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.274+0000 7f3ca9544640 1 -- 172.21.15.67:0/3149539787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4b550aa40 con 0x55b4b513d4d0 2024-07-31T15:10:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.275+0000 7f3ca9544640 1 --2- 172.21.15.67:0/3149539787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b513d4d0 0x55b4b54fe8b0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f3c940058c0 tx=0x7f3c940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4697d3da994d5bab server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.275+0000 7f3c8f7fe640 1 -- 172.21.15.67:0/3149539787 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9400f420 con 0x55b4b513d4d0 2024-07-31T15:10:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.275+0000 7f3c8f7fe640 1 -- 172.21.15.67:0/3149539787 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c9400f5c0 con 0x55b4b513d4d0 2024-07-31T15:10:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.275+0000 7f3c8f7fe640 1 -- 172.21.15.67:0/3149539787 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94005280 con 0x55b4b513d4d0 2024-07-31T15:10:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.275+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/3149539787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b513d4d0 msgr2=0x55b4b54fe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.275+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/3149539787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b513d4d0 0x55b4b54fe8b0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f3c940058c0 tx=0x7f3c940046c0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.275+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/3149539787 shutdown_connections 2024-07-31T15:10:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.275+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/3149539787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b513d4d0 0x55b4b54fe8b0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.275+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/3149539787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b51352f0 0x55b4b550bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.275+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/3149539787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b511cd80 0x55b4b5506e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.275+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/3149539787 >> 172.21.15.67:0/3149539787 conn(0x55b4b5152300 msgr2=0x55b4b54fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.276+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/3149539787 shutdown_connections 2024-07-31T15:10:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.276+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/3149539787 wait complete. 2024-07-31T15:10:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.276+0000 7f3cb2f7b740 1 -- start start 2024-07-31T15:10:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.276+0000 7f3cb2f7b740 1 Processor -- start 2024-07-31T15:10:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.276+0000 7f3cb2f7b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b511cd80 0x55b4b5514f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.276+0000 7f3cb2f7b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b51352f0 0x55b4b55116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.276+0000 7f3cb2f7b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b513d4d0 0x55b4b5511d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.276+0000 7f3cb2f7b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41b1bf0 con 0x55b4b511cd80 2024-07-31T15:10:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.276+0000 7f3cb2f7b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41b1520 con 0x55b4b51352f0 2024-07-31T15:10:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.276+0000 7f3cb2f7b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41ae780 con 0x55b4b513d4d0 2024-07-31T15:10:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3cb2f7b740 10 client.0 _ll_register_callbacks cb 0x7f3c9400a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3ca9544640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b511cd80 0x55b4b5514f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3ca9544640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b511cd80 0x55b4b5514f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44398/0 (socket says 172.21.15.67:44398) 2024-07-31T15:10:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3ca9544640 1 -- 172.21.15.67:0/2582042143 learned_addr learned my addr 172.21.15.67:0/2582042143 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3ca9d45640 1 --2- 172.21.15.67:0/2582042143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b51352f0 0x55b4b55116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3ca9544640 1 --2- 172.21.15.67:0/2582042143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b513d4d0 0x55b4b5511d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3ca9544640 1 -- 172.21.15.67:0/2582042143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b513d4d0 msgr2=0x55b4b5511d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3ca9544640 1 --2- 172.21.15.67:0/2582042143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b513d4d0 0x55b4b5511d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3ca9544640 1 -- 172.21.15.67:0/2582042143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b51352f0 msgr2=0x55b4b55116b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3ca9544640 1 --2- 172.21.15.67:0/2582042143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b51352f0 0x55b4b55116b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3ca9544640 1 -- 172.21.15.67:0/2582042143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4b55124c0 con 0x55b4b511cd80 2024-07-31T15:10:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3ca9544640 1 --2- 172.21.15.67:0/2582042143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b511cd80 0x55b4b5514f10 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400a790 tx=0x7f3c9400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a7475e5be261de server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3c8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:10:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3c8d7fa640 1 -- 172.21.15.67:0/2582042143 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9403d060 con 0x55b4b511cd80 2024-07-31T15:10:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3c8d7fa640 1 -- 172.21.15.67:0/2582042143 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9379+0+0 (secure 0 0 0) 0x7f3c94005230 con 0x55b4b511cd80 2024-07-31T15:10:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/2582042143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c9400f5c0 con 0x55b4b511cd80 2024-07-31T15:10:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3c8d7fa640 1 -- 172.21.15.67:0/2582042143 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f3c94030030 con 0x55b4b511cd80 2024-07-31T15:10:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3cb2f7b740 10 client.19581 fetch_fsmap learned FSMap version 65 2024-07-31T15:10:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.277+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/2582042143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55b4b5585f80 con 0x55b4b511cd80 2024-07-31T15:10:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.278+0000 7f3c8d7fa640 1 -- 172.21.15.67:0/2582042143 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c9400f5c0 con 0x55b4b511cd80 2024-07-31T15:10:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.278+0000 7f3c8d7fa640 1 -- 172.21.15.67:0/2582042143 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f3c94020440 con 0x55b4b511cd80 2024-07-31T15:10:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.278+0000 7f3cb2f7b740 10 client.19581 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:10:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.278+0000 7f3cb2f7b740 10 client.19581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.278+0000 7f3cb2f7b740 20 client.19581 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.278+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/2582042143 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4b515e1b0 0x55b4b55885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.278+0000 7f3cb2f7b740 4 client.19581 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:10:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.278+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/2582042143 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b4b5473da0 con 0x55b4b515e1b0 2024-07-31T15:10:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.278+0000 7f3ca9d45640 1 --2- 172.21.15.67:0/2582042143 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4b515e1b0 0x55b4b55885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.279+0000 7f3ca9d45640 1 --2- 172.21.15.67:0/2582042143 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4b515e1b0 0x55b4b55885d0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2239af70e973ee6c server_cookie=2568b7a6e63ff34c in_seq=0 out_seq=0 2024-07-31T15:10:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.279+0000 7f3c8d7fa640 10 client.19581 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.331+0000 7f3c8d7fa640 1 -- 172.21.15.67:0/2582042143 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b4b5473da0 con 0x55b4b515e1b0 2024-07-31T15:10:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.331+0000 7f3c8d7fa640 10 client.19581 handle_command_reply: tid=0 2024-07-31T15:10:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.331+0000 7f3cb2f7b740 1 client.19581 shutdown 2024-07-31T15:10:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.331+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/2582042143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b511cd80 msgr2=0x55b4b5514f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.331+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/2582042143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b511cd80 0x55b4b5514f10 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400a790 tx=0x7f3c9400fab0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/2582042143 shutdown_connections 2024-07-31T15:10:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/2582042143 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b4b515e1b0 0x55b4b55885d0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/2582042143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b4b513d4d0 0x55b4b5511d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/2582042143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4b51352f0 0x55b4b55116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3c8d7fa640 0 client.19581 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cb2f7b740 1 --2- 172.21.15.67:0/2582042143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b511cd80 0x55b4b5514f10 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/2582042143 >> 172.21.15.67:0/2582042143 conn(0x55b4b5152300 msgr2=0x55b4b5501d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/2582042143 shutdown_connections 2024-07-31T15:10:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cb2f7b740 1 -- 172.21.15.67:0/2582042143 wait complete. 2024-07-31T15:10:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cb2f7b740 20 client.19581 trim_cache size 0 max 16384 2024-07-31T15:10:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:10:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:10:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:10:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:10:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cabfa9640 1 -- 172.21.15.67:0/3434635164 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3c900b3dc0 msgr2=0x7f3c900b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cabfa9640 1 --2- 172.21.15.67:0/3434635164 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3c900b3dc0 0x7f3c900b62d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0cc310 tx=0x7f3c9c0996d0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cabfa9640 1 -- 172.21.15.67:0/3434635164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca4108670 msgr2=0x7f3ca419b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cabfa9640 1 --2- 172.21.15.67:0/3434635164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca4108670 0x7f3ca419b650 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c005cc0 tx=0x7f3c9c02f470 comp rx=0 tx=0).stop 2024-07-31T15:10:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cabfa9640 1 -- 172.21.15.67:0/3434635164 shutdown_connections 2024-07-31T15:10:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cabfa9640 1 --2- 172.21.15.67:0/3434635164 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3c900b3dc0 0x7f3c900b62d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cabfa9640 1 --2- 172.21.15.67:0/3434635164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca4108670 0x7f3ca419b650 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cabfa9640 1 --2- 172.21.15.67:0/3434635164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ca4107d20 0x7f3ca419b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cabfa9640 1 --2- 172.21.15.67:0/3434635164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ca41023f0 0x7f3ca419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.332+0000 7f3cabfa9640 1 -- 172.21.15.67:0/3434635164 >> 172.21.15.67:0/3434635164 conn(0x7f3ca40fddf0 msgr2=0x7f3ca41072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.333+0000 7f3cabfa9640 1 -- 172.21.15.67:0/3434635164 shutdown_connections 2024-07-31T15:10:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:29.333+0000 7f3cabfa9640 1 -- 172.21.15.67:0/3434635164 wait complete. 2024-07-31T15:10:29.398 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:29 smithi067 ceph-mon[32750]: pgmap v1019: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 75 KiB/s wr, 38 op/s 2024-07-31T15:10:29.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.403+0000 7faa41dac640 1 -- 172.21.15.67:0/142112386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa3c107dd0 msgr2=0x7faa3c1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.403+0000 7faa41dac640 1 --2- 172.21.15.67:0/142112386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa3c107dd0 0x7faa3c1081e0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7faa30009900 tx=0x7faa3002a410 comp rx=0 tx=0).stop 2024-07-31T15:10:29.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.403+0000 7faa41dac640 1 -- 172.21.15.67:0/142112386 shutdown_connections 2024-07-31T15:10:29.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.403+0000 7faa41dac640 1 --2- 172.21.15.67:0/142112386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa3c108720 0x7faa3c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.403+0000 7faa41dac640 1 --2- 172.21.15.67:0/142112386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa3c107dd0 0x7faa3c1081e0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.403+0000 7faa41dac640 1 --2- 172.21.15.67:0/142112386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa3c102490 0x7faa3c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.403+0000 7faa41dac640 1 -- 172.21.15.67:0/142112386 >> 172.21.15.67:0/142112386 conn(0x7faa3c079190 msgr2=0x7faa3c0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:29.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.403+0000 7faa41dac640 1 -- 172.21.15.67:0/142112386 shutdown_connections 2024-07-31T15:10:29.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.403+0000 7faa41dac640 1 -- 172.21.15.67:0/142112386 wait complete. 2024-07-31T15:10:29.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.403+0000 7faa41dac640 1 Processor -- start 2024-07-31T15:10:29.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa41dac640 1 -- start start 2024-07-31T15:10:29.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa41dac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa3c102490 0x7faa3c199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa41dac640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa3c107dd0 0x7faa3c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa41dac640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa3c108720 0x7faa3c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa41dac640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c113500 con 0x7faa3c107dd0 2024-07-31T15:10:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa41dac640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c113380 con 0x7faa3c108720 2024-07-31T15:10:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa41dac640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c113680 con 0x7faa3c102490 2024-07-31T15:10:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa3affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa3c107dd0 0x7faa3c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa3affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa3c107dd0 0x7faa3c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44408/0 (socket says 172.21.15.67:44408) 2024-07-31T15:10:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa3affd640 1 -- 172.21.15.67:0/1455570856 learned_addr learned my addr 172.21.15.67:0/1455570856 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa3b7fe640 1 --2- 172.21.15.67:0/1455570856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa3c102490 0x7faa3c199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa3bfff640 1 --2- 172.21.15.67:0/1455570856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa3c108720 0x7faa3c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa3affd640 1 -- 172.21.15.67:0/1455570856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa3c102490 msgr2=0x7faa3c199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa3affd640 1 --2- 172.21.15.67:0/1455570856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa3c102490 0x7faa3c199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa3affd640 1 -- 172.21.15.67:0/1455570856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa3c108720 msgr2=0x7faa3c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa3affd640 1 --2- 172.21.15.67:0/1455570856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa3c108720 0x7faa3c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.404+0000 7faa3affd640 1 -- 172.21.15.67:0/1455570856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3c19fe80 con 0x7faa3c107dd0 2024-07-31T15:10:29.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.405+0000 7faa3affd640 1 --2- 172.21.15.67:0/1455570856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa3c107dd0 0x7faa3c19a4b0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7faa30006010 tx=0x7faa30004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:29.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.405+0000 7faa38ff9640 1 -- 172.21.15.67:0/1455570856 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3003f030 con 0x7faa3c107dd0 2024-07-31T15:10:29.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.405+0000 7faa38ff9640 1 -- 172.21.15.67:0/1455570856 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa30003aa0 con 0x7faa3c107dd0 2024-07-31T15:10:29.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.405+0000 7faa38ff9640 1 -- 172.21.15.67:0/1455570856 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa30035070 con 0x7faa3c107dd0 2024-07-31T15:10:29.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.405+0000 7faa41dac640 1 -- 172.21.15.67:0/1455570856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa3c1a0110 con 0x7faa3c107dd0 2024-07-31T15:10:29.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.405+0000 7faa41dac640 1 -- 172.21.15.67:0/1455570856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa3c1a68d0 con 0x7faa3c107dd0 2024-07-31T15:10:29.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.408+0000 7faa482fd740 1 Processor -- start 2024-07-31T15:10:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.408+0000 7faa482fd740 1 -- start start 2024-07-31T15:10:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.408+0000 7faa38ff9640 1 -- 172.21.15.67:0/1455570856 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7faa30042030 con 0x7faa3c107dd0 2024-07-31T15:10:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.408+0000 7faa482fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5b1e520 0x5567c5ec4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.408+0000 7faa482fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5af5660 0x5567c5ec45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.408+0000 7faa482fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5afd840 0x5567c5ec4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.408+0000 7faa482fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5567c5e39e70 con 0x5567c5af5660 2024-07-31T15:10:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.408+0000 7faa482fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5567c5e29370 con 0x5567c5b1e520 2024-07-31T15:10:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.408+0000 7faa482fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5567c5e2d3b0 con 0x5567c5afd840 2024-07-31T15:10:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.408+0000 7faa3b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5af5660 0x5567c5ec45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.408+0000 7faa3b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5af5660 0x5567c5ec45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44416/0 (socket says 172.21.15.67:44416) 2024-07-31T15:10:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.408+0000 7faa3b7fe640 1 -- 172.21.15.67:0/2984529052 learned_addr learned my addr 172.21.15.67:0/2984529052 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.408+0000 7faa3bfff640 1 --2- 172.21.15.67:0/2984529052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5b1e520 0x5567c5ec4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.408+0000 7faa3affd640 1 --2- 172.21.15.67:0/2984529052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5afd840 0x5567c5ec4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.409+0000 7faa3b7fe640 1 -- 172.21.15.67:0/2984529052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5afd840 msgr2=0x5567c5ec4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.409+0000 7faa3b7fe640 1 --2- 172.21.15.67:0/2984529052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5afd840 0x5567c5ec4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.409+0000 7faa3b7fe640 1 -- 172.21.15.67:0/2984529052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5b1e520 msgr2=0x5567c5ec4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.409+0000 7faa3b7fe640 1 --2- 172.21.15.67:0/2984529052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5b1e520 0x5567c5ec4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.409+0000 7faa3b7fe640 1 -- 172.21.15.67:0/2984529052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c5df6040 con 0x5567c5af5660 2024-07-31T15:10:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.409+0000 7faa3affd640 1 --2- 172.21.15.67:0/2984529052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5afd840 0x5567c5ec4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:10:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.409+0000 7faa38ff9640 1 --2- 172.21.15.67:0/1455570856 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faa240b3dc0 0x7faa240b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.409+0000 7faa3b7fe640 1 --2- 172.21.15.67:0/2984529052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5af5660 0x5567c5ec45b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00bd10 tx=0x7faa2c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f48ab4a9ab1ac7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.409+0000 7faa3bfff640 1 --2- 172.21.15.67:0/1455570856 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faa240b3dc0 0x7faa240b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.409+0000 7faa38ff9640 1 -- 172.21.15.67:0/1455570856 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9379+0+0 (secure 0 0 0) 0x7faa301043c0 con 0x7faa3c107dd0 2024-07-31T15:10:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.409+0000 7faa21ffb640 1 -- 172.21.15.67:0/2984529052 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c004030 con 0x5567c5af5660 2024-07-31T15:10:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.409+0000 7faa21ffb640 1 -- 172.21.15.67:0/2984529052 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2c0035f0 con 0x5567c5af5660 2024-07-31T15:10:29.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.409+0000 7faa21ffb640 1 -- 172.21.15.67:0/2984529052 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c011770 con 0x5567c5af5660 2024-07-31T15:10:29.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.410+0000 7faa482fd740 1 -- 172.21.15.67:0/2984529052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5af5660 msgr2=0x5567c5ec45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.410+0000 7faa3bfff640 1 --2- 172.21.15.67:0/1455570856 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faa240b3dc0 0x7faa240b62d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7faa34005fd0 tx=0x7faa34009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:29.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.410+0000 7faa482fd740 1 --2- 172.21.15.67:0/2984529052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5af5660 0x5567c5ec45b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00bd10 tx=0x7faa2c0107a0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.410+0000 7faa482fd740 1 -- 172.21.15.67:0/2984529052 shutdown_connections 2024-07-31T15:10:29.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.410+0000 7faa482fd740 1 --2- 172.21.15.67:0/2984529052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5afd840 0x5567c5ec4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.410+0000 7faa482fd740 1 --2- 172.21.15.67:0/2984529052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5af5660 0x5567c5ec45b0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.410+0000 7faa482fd740 1 --2- 172.21.15.67:0/2984529052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5b1e520 0x5567c5ec4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.410+0000 7faa482fd740 1 -- 172.21.15.67:0/2984529052 >> 172.21.15.67:0/2984529052 conn(0x5567c5b12910 msgr2=0x5567c5ec1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.410+0000 7faa482fd740 1 -- 172.21.15.67:0/2984529052 shutdown_connections 2024-07-31T15:10:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.410+0000 7faa482fd740 1 -- 172.21.15.67:0/2984529052 wait complete. 2024-07-31T15:10:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.411+0000 7faa482fd740 1 -- start start 2024-07-31T15:10:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.411+0000 7faa482fd740 1 Processor -- start 2024-07-31T15:10:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.411+0000 7faa482fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5af5660 0x5567c5ecfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.411+0000 7faa482fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5afd840 0x5567c5ed01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.411+0000 7faa482fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5b1e520 0x5567c5edbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.411+0000 7faa482fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5567c5c8cbb0 con 0x5567c5afd840 2024-07-31T15:10:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.411+0000 7faa482fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5567c5dcbac0 con 0x5567c5b1e520 2024-07-31T15:10:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.411+0000 7faa482fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5567c5ed2850 con 0x5567c5af5660 2024-07-31T15:10:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa3affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5afd840 0x5567c5ed01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa3affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5afd840 0x5567c5ed01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44426/0 (socket says 172.21.15.67:44426) 2024-07-31T15:10:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa3affd640 1 -- 172.21.15.67:0/1342689534 learned_addr learned my addr 172.21.15.67:0/1342689534 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa482fd740 10 client.0 _ll_register_callbacks cb 0x5567c5dd43b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa3bfff640 1 --2- 172.21.15.67:0/1342689534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5b1e520 0x5567c5edbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa3b7fe640 1 --2- 172.21.15.67:0/1342689534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5af5660 0x5567c5ecfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa3affd640 1 -- 172.21.15.67:0/1342689534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5af5660 msgr2=0x5567c5ecfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa3affd640 1 --2- 172.21.15.67:0/1342689534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5af5660 0x5567c5ecfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa3affd640 1 -- 172.21.15.67:0/1342689534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5b1e520 msgr2=0x5567c5edbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa3affd640 1 --2- 172.21.15.67:0/1342689534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5b1e520 0x5567c5edbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa3affd640 1 -- 172.21.15.67:0/1342689534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c5b89b00 con 0x5567c5afd840 2024-07-31T15:10:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa3affd640 1 --2- 172.21.15.67:0/1342689534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5afd840 0x5567c5ed01d0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7faa30031440 tx=0x7faa30003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115f9072ec8aedb7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa1affd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:10:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa1affd640 1 -- 172.21.15.67:0/1342689534 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa30049070 con 0x5567c5afd840 2024-07-31T15:10:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa482fd740 1 -- 172.21.15.67:0/1342689534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa2c0035f0 con 0x5567c5afd840 2024-07-31T15:10:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa1affd640 1 -- 172.21.15.67:0/1342689534 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9379+0+0 (secure 0 0 0) 0x7faa30104120 con 0x5567c5afd840 2024-07-31T15:10:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.412+0000 7faa1affd640 1 -- 172.21.15.67:0/1342689534 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7faa300ed040 con 0x5567c5afd840 2024-07-31T15:10:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.413+0000 7faa482fd740 10 client.19593 fetch_fsmap learned FSMap version 65 2024-07-31T15:10:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.413+0000 7faa482fd740 1 -- 172.21.15.67:0/1342689534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5567c5c00960 con 0x5567c5afd840 2024-07-31T15:10:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.413+0000 7faa1affd640 1 -- 172.21.15.67:0/1342689534 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2c0035f0 con 0x5567c5afd840 2024-07-31T15:10:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.413+0000 7faa1affd640 1 -- 172.21.15.67:0/1342689534 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7faa3004c060 con 0x5567c5afd840 2024-07-31T15:10:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.413+0000 7faa482fd740 10 client.19593 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:10:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.413+0000 7faa482fd740 10 client.19593 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:10:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.413+0000 7faa482fd740 20 client.19593 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.413+0000 7faa482fd740 1 --2- 172.21.15.67:0/1342689534 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5567c5adcea0 0x5567c5ee1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.413+0000 7faa482fd740 4 client.19593 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:10:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.413+0000 7faa482fd740 1 -- 172.21.15.67:0/1342689534 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567c5e3bdc0 con 0x5567c5adcea0 2024-07-31T15:10:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.413+0000 7faa3b7fe640 1 --2- 172.21.15.67:0/1342689534 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5567c5adcea0 0x5567c5ee1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.414+0000 7faa3b7fe640 1 --2- 172.21.15.67:0/1342689534 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5567c5adcea0 0x5567c5ee1bf0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa26532bebcc908 server_cookie=d373a8d14ed56a72 in_seq=0 out_seq=0 2024-07-31T15:10:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.414+0000 7faa1affd640 10 client.19593 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.414+0000 7faa1affd640 1 -- 172.21.15.67:0/1342689534 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567c5e3bdc0 con 0x5567c5adcea0 2024-07-31T15:10:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.414+0000 7faa1affd640 10 client.19593 handle_command_reply: tid=0 2024-07-31T15:10:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.415+0000 7faa482fd740 1 client.19593 shutdown 2024-07-31T15:10:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.415+0000 7faa482fd740 1 -- 172.21.15.67:0/1342689534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5afd840 msgr2=0x5567c5ed01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.415+0000 7faa482fd740 1 --2- 172.21.15.67:0/1342689534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5afd840 0x5567c5ed01d0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7faa30031440 tx=0x7faa30003f50 comp rx=0 tx=0).stop 2024-07-31T15:10:29.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.415+0000 7faa482fd740 1 -- 172.21.15.67:0/1342689534 shutdown_connections 2024-07-31T15:10:29.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.415+0000 7faa482fd740 1 --2- 172.21.15.67:0/1342689534 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5567c5adcea0 0x5567c5ee1bf0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.415+0000 7faa482fd740 1 --2- 172.21.15.67:0/1342689534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5b1e520 0x5567c5edbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.415+0000 7faa1affd640 0 client.19593 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:29.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.415+0000 7faa482fd740 1 --2- 172.21.15.67:0/1342689534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5afd840 0x5567c5ed01d0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.415+0000 7faa482fd740 1 --2- 172.21.15.67:0/1342689534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5af5660 0x5567c5ecfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.415+0000 7faa482fd740 1 -- 172.21.15.67:0/1342689534 >> 172.21.15.67:0/1342689534 conn(0x5567c5b12910 msgr2=0x5567c5ec7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:29.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.415+0000 7faa482fd740 1 -- 172.21.15.67:0/1342689534 shutdown_connections 2024-07-31T15:10:29.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.415+0000 7faa482fd740 1 -- 172.21.15.67:0/1342689534 wait complete. 2024-07-31T15:10:29.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.415+0000 7faa482fd740 20 client.19593 trim_cache size 0 max 16384 2024-07-31T15:10:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.433+0000 7faa482fd740 1 Processor -- start 2024-07-31T15:10:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.433+0000 7faa482fd740 1 -- start start 2024-07-31T15:10:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa482fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5adcea0 0x5567c5ec1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa482fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5af5660 0x5567c5ece310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa482fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5afd840 0x5567c5edb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa482fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c1ae5f0 con 0x5567c5af5660 2024-07-31T15:10:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa482fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c1b0130 con 0x5567c5adcea0 2024-07-31T15:10:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa482fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c1aecc0 con 0x5567c5afd840 2024-07-31T15:10:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa3b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5adcea0 0x5567c5ec1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa3affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5af5660 0x5567c5ece310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa3b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5adcea0 0x5567c5ec1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59354/0 (socket says 172.21.15.67:59354) 2024-07-31T15:10:29.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa3b7fe640 1 -- 172.21.15.67:0/755465558 learned_addr learned my addr 172.21.15.67:0/755465558 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:29.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa3bfff640 1 --2- 172.21.15.67:0/755465558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5afd840 0x5567c5edb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa3b7fe640 1 -- 172.21.15.67:0/755465558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5afd840 msgr2=0x5567c5edb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa3b7fe640 1 --2- 172.21.15.67:0/755465558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5afd840 0x5567c5edb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa3b7fe640 1 -- 172.21.15.67:0/755465558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5af5660 msgr2=0x5567c5ece310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa3b7fe640 1 --2- 172.21.15.67:0/755465558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5af5660 0x5567c5ece310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa3b7fe640 1 -- 172.21.15.67:0/755465558 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c5ec9860 con 0x5567c5adcea0 2024-07-31T15:10:29.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.434+0000 7faa3b7fe640 1 --2- 172.21.15.67:0/755465558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5adcea0 0x5567c5ec1fb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00bc30 tx=0x7faa2c003de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=794a8e2b364f1fbc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:29.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.435+0000 7faa217fa640 1 -- 172.21.15.67:0/755465558 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c005240 con 0x5567c5adcea0 2024-07-31T15:10:29.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.435+0000 7faa217fa640 1 -- 172.21.15.67:0/755465558 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2c003e50 con 0x5567c5adcea0 2024-07-31T15:10:29.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.435+0000 7faa217fa640 1 -- 172.21.15.67:0/755465558 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c0116a0 con 0x5567c5adcea0 2024-07-31T15:10:29.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.435+0000 7faa482fd740 1 -- 172.21.15.67:0/755465558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5adcea0 msgr2=0x5567c5ec1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:29.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.435+0000 7faa482fd740 1 --2- 172.21.15.67:0/755465558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5adcea0 0x5567c5ec1fb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00bc30 tx=0x7faa2c003de0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.436+0000 7faa482fd740 1 -- 172.21.15.67:0/755465558 shutdown_connections 2024-07-31T15:10:29.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.436+0000 7faa482fd740 1 --2- 172.21.15.67:0/755465558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5afd840 0x5567c5edb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.436+0000 7faa482fd740 1 --2- 172.21.15.67:0/755465558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5af5660 0x5567c5ece310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.436+0000 7faa482fd740 1 --2- 172.21.15.67:0/755465558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5adcea0 0x5567c5ec1fb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.436+0000 7faa482fd740 1 -- 172.21.15.67:0/755465558 >> 172.21.15.67:0/755465558 conn(0x5567c5b12910 msgr2=0x5567c5ec18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:29.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.436+0000 7faa482fd740 1 -- 172.21.15.67:0/755465558 shutdown_connections 2024-07-31T15:10:29.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.436+0000 7faa482fd740 1 -- 172.21.15.67:0/755465558 wait complete. 2024-07-31T15:10:29.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.436+0000 7faa482fd740 1 -- start start 2024-07-31T15:10:29.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.436+0000 7faa482fd740 1 Processor -- start 2024-07-31T15:10:29.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa482fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5adcea0 0x5567c5ed4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa482fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5af5660 0x5567c5ed46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa482fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5afd840 0x5567c5f493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa482fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c1afa60 con 0x5567c5afd840 2024-07-31T15:10:29.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa482fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c1af390 con 0x5567c5af5660 2024-07-31T15:10:29.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa482fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c1ac5f0 con 0x5567c5adcea0 2024-07-31T15:10:29.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa3bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5afd840 0x5567c5f493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa482fd740 10 client.0 _ll_register_callbacks cb 0x7faa30005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa3affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5af5660 0x5567c5ed46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa3b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5adcea0 0x5567c5ed4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa3bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5afd840 0x5567c5f493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44448/0 (socket says 172.21.15.67:44448) 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa3bfff640 1 -- 172.21.15.67:0/1153137256 learned_addr learned my addr 172.21.15.67:0/1153137256 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa3bfff640 1 -- 172.21.15.67:0/1153137256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5adcea0 msgr2=0x5567c5ed4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa3bfff640 1 --2- 172.21.15.67:0/1153137256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5adcea0 0x5567c5ed4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa3bfff640 1 -- 172.21.15.67:0/1153137256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5af5660 msgr2=0x5567c5ed46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa3bfff640 1 --2- 172.21.15.67:0/1153137256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5af5660 0x5567c5ed46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa3bfff640 1 -- 172.21.15.67:0/1153137256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c5ece9f0 con 0x5567c5afd840 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa3affd640 1 --2- 172.21.15.67:0/1153137256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5af5660 0x5567c5ed46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa3bfff640 1 --2- 172.21.15.67:0/1153137256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5afd840 0x5567c5f493c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7faa34004330 tx=0x7faa34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3c5170fb1db483 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.437+0000 7faa1affd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.438+0000 7faa1affd640 1 -- 172.21.15.67:0/1153137256 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3400fc70 con 0x5567c5afd840 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.438+0000 7faa482fd740 1 -- 172.21.15.67:0/1153137256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa2c003e50 con 0x5567c5afd840 2024-07-31T15:10:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.438+0000 7faa1affd640 1 -- 172.21.15.67:0/1153137256 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9379+0+0 (secure 0 0 0) 0x7faa34004b20 con 0x5567c5afd840 2024-07-31T15:10:29.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.438+0000 7faa1affd640 1 -- 172.21.15.67:0/1153137256 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7faa34034050 con 0x5567c5afd840 2024-07-31T15:10:29.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.438+0000 7faa482fd740 10 client.19599 fetch_fsmap learned FSMap version 65 2024-07-31T15:10:29.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.438+0000 7faa482fd740 1 -- 172.21.15.67:0/1153137256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5567c5f4aad0 con 0x5567c5afd840 2024-07-31T15:10:29.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.438+0000 7faa1affd640 1 -- 172.21.15.67:0/1153137256 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2c003e50 con 0x5567c5afd840 2024-07-31T15:10:29.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.438+0000 7faa1affd640 1 -- 172.21.15.67:0/1153137256 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7faa34005740 con 0x5567c5afd840 2024-07-31T15:10:29.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.438+0000 7faa482fd740 10 client.19599 fetch_fsmap finished waiting for FSMap version 65 2024-07-31T15:10:29.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.438+0000 7faa482fd740 10 client.19599 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:10:29.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.438+0000 7faa482fd740 20 client.19599 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:29.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.438+0000 7faa482fd740 1 --2- 172.21.15.67:0/1153137256 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5567c5b1e520 0x5567c5f4d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:29.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.438+0000 7faa482fd740 4 client.19599 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "dd84c0d1", "release": true, "await": true}] 2024-07-31T15:10:29.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.438+0000 7faa482fd740 1 -- 172.21.15.67:0/1153137256 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "dd84c0d1", "release": true, "await": true}) -- 0x5567c5e37860 con 0x5567c5b1e520 2024-07-31T15:10:29.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.439+0000 7faa3b7fe640 1 --2- 172.21.15.67:0/1153137256 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5567c5b1e520 0x5567c5f4d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:29.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.439+0000 7faa1affd640 10 client.19599 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:29.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:29.439+0000 7faa3b7fe640 1 --2- 172.21.15.67:0/1153137256 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5567c5b1e520 0x5567c5f4d340 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f50dd5c8ec4420 server_cookie=f395e7529eb7e157 in_seq=0 out_seq=0 2024-07-31T15:10:29.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:29 smithi073 ceph-mon[36714]: pgmap v1019: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 75 KiB/s wr, 38 op/s 2024-07-31T15:10:29.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:29 smithi077 ceph-mon[37327]: pgmap v1019: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 75 KiB/s wr, 38 op/s 2024-07-31T15:10:29.867 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T15:10:29.893 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:10:29 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[70341]: ignoring --setuser ceph since I am not root 2024-07-31T15:10:29.893 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:10:29 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[70341]: ignoring --setgroup ceph since I am not root 2024-07-31T15:10:29.893 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:10:29 smithi073 ceph-mds[70345]: -- 172.21.15.73:0/4168885173 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56398d9cf860 con 0x563991b71a80 2024-07-31T15:10:29.893 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:10:29 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[70341]: starting mds.e at 2024-07-31T15:10:30.135 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:10:30.514 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:30 smithi067 ceph-mon[32750]: daemon mds.e restarted 2024-07-31T15:10:30.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:30 smithi067 ceph-mon[32750]: osdmap e128: 12 total, 12 up, 12 in 2024-07-31T15:10:30.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:30 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.73:6834/631186801,v1:172.21.15.73:6835/631186801] up:boot 2024-07-31T15:10:30.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:30 smithi067 ceph-mon[32750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:10:30.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:30 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T15:10:30.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:30 smithi073 ceph-mon[36714]: daemon mds.e restarted 2024-07-31T15:10:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:30 smithi073 ceph-mon[36714]: osdmap e128: 12 total, 12 up, 12 in 2024-07-31T15:10:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:30 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.73:6834/631186801,v1:172.21.15.73:6835/631186801] up:boot 2024-07-31T15:10:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:30 smithi073 ceph-mon[36714]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:10:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:30 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T15:10:30.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:30 smithi077 ceph-mon[37327]: daemon mds.e restarted 2024-07-31T15:10:30.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:30 smithi077 ceph-mon[37327]: osdmap e128: 12 total, 12 up, 12 in 2024-07-31T15:10:30.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:30 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.73:6834/631186801,v1:172.21.15.73:6835/631186801] up:boot 2024-07-31T15:10:30.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:30 smithi077 ceph-mon[37327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-31T15:10:30.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:30 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T15:10:31.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:31 smithi067 ceph-mon[32750]: pgmap v1020: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 69 KiB/s wr, 30 op/s 2024-07-31T15:10:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:31 smithi073 ceph-mon[36714]: pgmap v1020: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 69 KiB/s wr, 30 op/s 2024-07-31T15:10:31.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:31 smithi077 ceph-mon[37327]: pgmap v1020: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 69 KiB/s wr, 30 op/s 2024-07-31T15:10:31.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.857+0000 7faa1affd640 1 -- 172.21.15.67:0/1153137256 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5567c5e37860 con 0x5567c5b1e520 2024-07-31T15:10:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.857+0000 7faa1affd640 10 client.19599 handle_command_reply: tid=0 2024-07-31T15:10:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.857+0000 7faa482fd740 1 client.19599 shutdown 2024-07-31T15:10:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.858+0000 7faa482fd740 1 -- 172.21.15.67:0/1153137256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5afd840 msgr2=0x5567c5f493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.858+0000 7faa482fd740 1 --2- 172.21.15.67:0/1153137256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5afd840 0x5567c5f493c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7faa34004330 tx=0x7faa34004a40 comp rx=0 tx=0).stop 2024-07-31T15:10:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.858+0000 7faa482fd740 1 -- 172.21.15.67:0/1153137256 shutdown_connections 2024-07-31T15:10:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.858+0000 7faa482fd740 1 --2- 172.21.15.67:0/1153137256 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5567c5b1e520 0x5567c5f4d340 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.858+0000 7faa482fd740 1 --2- 172.21.15.67:0/1153137256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5567c5afd840 0x5567c5f493c0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.858+0000 7faa482fd740 1 --2- 172.21.15.67:0/1153137256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567c5af5660 0x5567c5ed46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.858+0000 7faa482fd740 1 --2- 172.21.15.67:0/1153137256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567c5adcea0 0x5567c5ed4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.858+0000 7faa482fd740 1 -- 172.21.15.67:0/1153137256 >> 172.21.15.67:0/1153137256 conn(0x5567c5b12910 msgr2=0x5567c5ec18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.858+0000 7faa1affd640 0 client.19599 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:31.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.859+0000 7faa482fd740 1 -- 172.21.15.67:0/1153137256 shutdown_connections 2024-07-31T15:10:31.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.859+0000 7faa482fd740 1 -- 172.21.15.67:0/1153137256 wait complete. 2024-07-31T15:10:31.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.859+0000 7faa482fd740 20 client.19599 trim_cache size 0 max 16384 2024-07-31T15:10:31.863 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:10:31.863 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 66, 2024-07-31T15:10:31.863 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:10:31.863 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 56, 2024-07-31T15:10:31.863 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:10:31.863 INFO:teuthology.orchestra.run.smithi067.stdout: "dd84c0d1": { 2024-07-31T15:10:31.863 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 56, 2024-07-31T15:10:31.863 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:10:31.863 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:10:31.863 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:10:31.863 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:10:31.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:10:31.863 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:10:31.864 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 138.1, 2024-07-31T15:10:31.864 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:10:31.864 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:10:31.864 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:10:31.864 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:10:31.864 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:10:31.864 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:10:31.864 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:10:31.864 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:10:31.864 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:10:31.864 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:10:31.864 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:10:31.864 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:10:31.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.860+0000 7faa41dac640 1 -- 172.21.15.67:0/1455570856 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faa240b3dc0 msgr2=0x7faa240b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:31.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.860+0000 7faa41dac640 1 --2- 172.21.15.67:0/1455570856 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faa240b3dc0 0x7faa240b62d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7faa34005fd0 tx=0x7faa34009500 comp rx=0 tx=0).stop 2024-07-31T15:10:31.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.860+0000 7faa41dac640 1 -- 172.21.15.67:0/1455570856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa3c107dd0 msgr2=0x7faa3c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:31.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.860+0000 7faa41dac640 1 --2- 172.21.15.67:0/1455570856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa3c107dd0 0x7faa3c19a4b0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7faa30006010 tx=0x7faa30004060 comp rx=0 tx=0).stop 2024-07-31T15:10:31.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.860+0000 7faa41dac640 1 -- 172.21.15.67:0/1455570856 shutdown_connections 2024-07-31T15:10:31.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.860+0000 7faa41dac640 1 --2- 172.21.15.67:0/1455570856 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faa240b3dc0 0x7faa240b62d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.860+0000 7faa41dac640 1 --2- 172.21.15.67:0/1455570856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa3c108720 0x7faa3c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.860+0000 7faa41dac640 1 --2- 172.21.15.67:0/1455570856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa3c107dd0 0x7faa3c19a4b0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.860+0000 7faa41dac640 1 --2- 172.21.15.67:0/1455570856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa3c102490 0x7faa3c199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.860+0000 7faa41dac640 1 -- 172.21.15.67:0/1455570856 >> 172.21.15.67:0/1455570856 conn(0x7faa3c079190 msgr2=0x7faa3c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:31.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.860+0000 7faa41dac640 1 -- 172.21.15.67:0/1455570856 shutdown_connections 2024-07-31T15:10:31.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.860+0000 7faa41dac640 1 -- 172.21.15.67:0/1455570856 wait complete. 2024-07-31T15:10:31.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.930+0000 7f65a14c2640 1 -- 172.21.15.67:0/3038897068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f659c1042f0 msgr2=0x7f659c104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:31.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.930+0000 7f65a14c2640 1 --2- 172.21.15.67:0/3038897068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f659c1042f0 0x7f659c104700 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f658c009900 tx=0x7f658c02a410 comp rx=0 tx=0).stop 2024-07-31T15:10:31.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.930+0000 7f65a14c2640 1 -- 172.21.15.67:0/3038897068 shutdown_connections 2024-07-31T15:10:31.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.930+0000 7f65a14c2640 1 --2- 172.21.15.67:0/3038897068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f659c10d8a0 0x7f659c10fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.930+0000 7f65a14c2640 1 --2- 172.21.15.67:0/3038897068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f659c104d20 0x7f659c10d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.930+0000 7f65a14c2640 1 --2- 172.21.15.67:0/3038897068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f659c1042f0 0x7f659c104700 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.930+0000 7f65a14c2640 1 -- 172.21.15.67:0/3038897068 >> 172.21.15.67:0/3038897068 conn(0x7f659c079190 msgr2=0x7f659c0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:31.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.930+0000 7f65a14c2640 1 -- 172.21.15.67:0/3038897068 shutdown_connections 2024-07-31T15:10:31.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.930+0000 7f65a14c2640 1 -- 172.21.15.67:0/3038897068 wait complete. 2024-07-31T15:10:31.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.931+0000 7f65a14c2640 1 Processor -- start 2024-07-31T15:10:31.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.931+0000 7f65a14c2640 1 -- start start 2024-07-31T15:10:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.931+0000 7f65a14c2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f659c1042f0 0x7f659c199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.931+0000 7f65a14c2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f659c104d20 0x7f659c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.931+0000 7f65a14c2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f659c10d8a0 0x7f659c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.931+0000 7f65a14c2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1134f0 con 0x7f659c10d8a0 2024-07-31T15:10:31.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.931+0000 7f65a14c2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f659c113370 con 0x7f659c104d20 2024-07-31T15:10:31.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.931+0000 7f65a14c2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f659c113670 con 0x7f659c1042f0 2024-07-31T15:10:31.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.931+0000 7f659affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f659c1042f0 0x7f659c199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.931+0000 7f659b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f659c10d8a0 0x7f659c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.931+0000 7f659affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f659c1042f0 0x7f659c199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34782/0 (socket says 172.21.15.67:34782) 2024-07-31T15:10:31.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.931+0000 7f659affd640 1 -- 172.21.15.67:0/1968954390 learned_addr learned my addr 172.21.15.67:0/1968954390 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:31.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.931+0000 7f6593fff640 1 --2- 172.21.15.67:0/1968954390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f659c104d20 0x7f659c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.932+0000 7f6593fff640 1 -- 172.21.15.67:0/1968954390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f659c1042f0 msgr2=0x7f659c199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:31.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.932+0000 7f6593fff640 1 --2- 172.21.15.67:0/1968954390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f659c1042f0 0x7f659c199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.932+0000 7f6593fff640 1 -- 172.21.15.67:0/1968954390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f659c10d8a0 msgr2=0x7f659c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:31.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.932+0000 7f6593fff640 1 --2- 172.21.15.67:0/1968954390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f659c10d8a0 0x7f659c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.932+0000 7f6593fff640 1 -- 172.21.15.67:0/1968954390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c19fe70 con 0x7f659c104d20 2024-07-31T15:10:31.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.932+0000 7f659b7fe640 1 --2- 172.21.15.67:0/1968954390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f659c10d8a0 0x7f659c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:31.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.932+0000 7f6593fff640 1 --2- 172.21.15.67:0/1968954390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f659c104d20 0x7f659c19a4a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f658400e9c0 tx=0x7f658400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:31.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.932+0000 7f6598ff9640 1 -- 172.21.15.67:0/1968954390 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658400ccc0 con 0x7f659c104d20 2024-07-31T15:10:31.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.932+0000 7f6598ff9640 1 -- 172.21.15.67:0/1968954390 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6584005020 con 0x7f659c104d20 2024-07-31T15:10:31.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.932+0000 7f6598ff9640 1 -- 172.21.15.67:0/1968954390 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584004a00 con 0x7f659c104d20 2024-07-31T15:10:31.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.932+0000 7f65a14c2640 1 -- 172.21.15.67:0/1968954390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659c1a0160 con 0x7f659c104d20 2024-07-31T15:10:31.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.932+0000 7f65a14c2640 1 -- 172.21.15.67:0/1968954390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659c1a68c0 con 0x7f659c104d20 2024-07-31T15:10:31.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.935+0000 7f65a7a13740 1 Processor -- start 2024-07-31T15:10:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.935+0000 7f65a7a13740 1 -- start start 2024-07-31T15:10:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f65a7a13740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e30070 0x55ff971d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f65a7a13740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96e071b0 0x55ff971d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f65a7a13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 0x55ff971d6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f65a7a13740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ff9714bbb0 con 0x55ff96e071b0 2024-07-31T15:10:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f65a7a13740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ff9713b0b0 con 0x55ff96e30070 2024-07-31T15:10:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f65a7a13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ff9713f0f0 con 0x55ff96e0f390 2024-07-31T15:10:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f6598ff9640 1 -- 172.21.15.67:0/1968954390 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f65840051d0 con 0x7f659c104d20 2024-07-31T15:10:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f659b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e30070 0x55ff971d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f659b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e30070 0x55ff971d5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45792/0 (socket says 172.21.15.67:45792) 2024-07-31T15:10:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f659b7fe640 1 -- 172.21.15.67:0/3611500346 learned_addr learned my addr 172.21.15.67:0/3611500346 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f659affd640 1 --2- 172.21.15.67:0/3611500346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96e071b0 0x55ff971d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f6593fff640 1 --2- 172.21.15.67:0/3611500346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 0x55ff971d6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f659b7fe640 1 -- 172.21.15.67:0/3611500346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 msgr2=0x55ff971d6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f659b7fe640 1 --2- 172.21.15.67:0/3611500346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 0x55ff971d6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f659b7fe640 1 -- 172.21.15.67:0/3611500346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96e071b0 msgr2=0x55ff971d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f659b7fe640 1 --2- 172.21.15.67:0/3611500346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96e071b0 0x55ff971d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.936+0000 7f659b7fe640 1 -- 172.21.15.67:0/3611500346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff970e1b90 con 0x55ff96e30070 2024-07-31T15:10:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.937+0000 7f659b7fe640 1 --2- 172.21.15.67:0/3611500346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e30070 0x55ff971d5690 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f65940117e0 tx=0x7f6594011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7986c0e995ad04e5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.937+0000 7f65917fa640 1 -- 172.21.15.67:0/3611500346 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594011310 con 0x55ff96e30070 2024-07-31T15:10:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.937+0000 7f65917fa640 1 -- 172.21.15.67:0/3611500346 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6594003e20 con 0x55ff96e30070 2024-07-31T15:10:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.937+0000 7f6598ff9640 1 --2- 172.21.15.67:0/1968954390 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65880b3e40 0x7f65880b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.937+0000 7f65917fa640 1 -- 172.21.15.67:0/3611500346 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594004490 con 0x55ff96e30070 2024-07-31T15:10:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.937+0000 7f659affd640 1 --2- 172.21.15.67:0/1968954390 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65880b3e40 0x7f65880b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.937+0000 7f6598ff9640 1 -- 172.21.15.67:0/1968954390 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f65840e4290 con 0x7f659c104d20 2024-07-31T15:10:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.938+0000 7f65a7a13740 1 -- 172.21.15.67:0/3611500346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e30070 msgr2=0x55ff971d5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.938+0000 7f659affd640 1 --2- 172.21.15.67:0/1968954390 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65880b3e40 0x7f65880b6350 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x55ff971d4310 tx=0x7f658c005890 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.938+0000 7f65a7a13740 1 --2- 172.21.15.67:0/3611500346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e30070 0x55ff971d5690 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f65940117e0 tx=0x7f6594011ca0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.938+0000 7f65a7a13740 1 -- 172.21.15.67:0/3611500346 shutdown_connections 2024-07-31T15:10:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.938+0000 7f65a7a13740 1 --2- 172.21.15.67:0/3611500346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 0x55ff971d6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.938+0000 7f65a7a13740 1 --2- 172.21.15.67:0/3611500346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96e071b0 0x55ff971d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.938+0000 7f65a7a13740 1 --2- 172.21.15.67:0/3611500346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e30070 0x55ff971d5690 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.938+0000 7f65a7a13740 1 -- 172.21.15.67:0/3611500346 >> 172.21.15.67:0/3611500346 conn(0x55ff96e24460 msgr2=0x55ff971d3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.939+0000 7f65a7a13740 1 -- 172.21.15.67:0/3611500346 shutdown_connections 2024-07-31T15:10:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.939+0000 7f65a7a13740 1 -- 172.21.15.67:0/3611500346 wait complete. 2024-07-31T15:10:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.939+0000 7f65a7a13740 1 -- start start 2024-07-31T15:10:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.939+0000 7f65a7a13740 1 Processor -- start 2024-07-31T15:10:31.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f65a7a13740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e071b0 0x55ff971e12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f65a7a13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 0x55ff971e1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f65a7a13740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96e30070 0x55ff971ed1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f65a7a13740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ff96f9e700 con 0x55ff96e30070 2024-07-31T15:10:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f65a7a13740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ff970dd9e0 con 0x55ff96e071b0 2024-07-31T15:10:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f65a7a13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ff971e3e70 con 0x55ff96e0f390 2024-07-31T15:10:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f65a7a13740 10 client.0 _ll_register_callbacks cb 0x55ff970f84f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f659affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e071b0 0x55ff971e12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f659affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e071b0 0x55ff971e12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45798/0 (socket says 172.21.15.67:45798) 2024-07-31T15:10:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f659affd640 1 -- 172.21.15.67:0/981655210 learned_addr learned my addr 172.21.15.67:0/981655210 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f659b7fe640 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96e30070 0x55ff971ed1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f6593fff640 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 0x55ff971e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f659affd640 1 -- 172.21.15.67:0/981655210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 msgr2=0x55ff971e1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f659affd640 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 0x55ff971e1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f659affd640 1 -- 172.21.15.67:0/981655210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96e30070 msgr2=0x55ff971ed1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f659affd640 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96e30070 0x55ff971ed1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f659affd640 1 -- 172.21.15.67:0/981655210 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff96e9b650 con 0x55ff96e071b0 2024-07-31T15:10:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f659b7fe640 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96e30070 0x55ff971ed1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.940+0000 7f6593fff640 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 0x55ff971e1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:10:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.941+0000 7f659affd640 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e071b0 0x55ff971e12d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f658c006010 tx=0x7f658c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1604b16f4427b0f8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.941+0000 7f65827fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:10:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.941+0000 7f65827fc640 1 -- 172.21.15.67:0/981655210 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c057060 con 0x55ff96e071b0 2024-07-31T15:10:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.941+0000 7f65a7a13740 1 -- 172.21.15.67:0/981655210 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6594003e20 con 0x55ff96e071b0 2024-07-31T15:10:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.941+0000 7f65827fc640 1 -- 172.21.15.67:0/981655210 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f658c004250 con 0x55ff96e071b0 2024-07-31T15:10:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.941+0000 7f65827fc640 1 -- 172.21.15.67:0/981655210 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f658c05e050 con 0x55ff96e071b0 2024-07-31T15:10:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.941+0000 7f65a7a13740 10 client.29141 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.941+0000 7f65a7a13740 1 -- 172.21.15.67:0/981655210 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55ff971eff60 con 0x55ff96e071b0 2024-07-31T15:10:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.941+0000 7f65827fc640 1 -- 172.21.15.67:0/981655210 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6594003e20 con 0x55ff96e071b0 2024-07-31T15:10:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.941+0000 7f65827fc640 1 -- 172.21.15.67:0/981655210 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f658c003680 con 0x55ff96e071b0 2024-07-31T15:10:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.941+0000 7f65a7a13740 10 client.29141 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.941+0000 7f65a7a13740 10 client.29141 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:10:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.941+0000 7f65a7a13740 20 client.29141 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.941+0000 7f65a7a13740 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55ff96dee9f0 0x55ff971f3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.942+0000 7f65a7a13740 4 client.29141 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:10:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.942+0000 7f65a7a13740 1 -- 172.21.15.67:0/981655210 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff9714db00 con 0x55ff96dee9f0 2024-07-31T15:10:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.942+0000 7f6593fff640 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55ff96dee9f0 0x55ff971f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.942+0000 7f6593fff640 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55ff96dee9f0 0x55ff971f3500 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1e191e608dcecb91 server_cookie=664967dd21600fc in_seq=0 out_seq=0 2024-07-31T15:10:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.942+0000 7f65827fc640 10 client.29141 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:10:31.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.944+0000 7f65827fc640 1 -- 172.21.15.67:0/981655210 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff9714db00 con 0x55ff96dee9f0 2024-07-31T15:10:31.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.944+0000 7f65827fc640 10 client.29141 handle_command_reply: tid=0 2024-07-31T15:10:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.944+0000 7f65a7a13740 1 client.29141 shutdown 2024-07-31T15:10:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.944+0000 7f65a7a13740 1 -- 172.21.15.67:0/981655210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e071b0 msgr2=0x55ff971e12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.944+0000 7f65a7a13740 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e071b0 0x55ff971e12d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f658c006010 tx=0x7f658c009260 comp rx=0 tx=0).stop 2024-07-31T15:10:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.944+0000 7f65a7a13740 1 -- 172.21.15.67:0/981655210 shutdown_connections 2024-07-31T15:10:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.944+0000 7f65a7a13740 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55ff96dee9f0 0x55ff971f3500 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.944+0000 7f65a7a13740 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96e30070 0x55ff971ed1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.944+0000 7f65827fc640 0 client.29141 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:10:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.944+0000 7f65a7a13740 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 0x55ff971e1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.944+0000 7f65a7a13740 1 --2- 172.21.15.67:0/981655210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e071b0 0x55ff971e12d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.944+0000 7f65a7a13740 1 -- 172.21.15.67:0/981655210 >> 172.21.15.67:0/981655210 conn(0x55ff96e24460 msgr2=0x55ff971d8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.944+0000 7f65a7a13740 1 -- 172.21.15.67:0/981655210 shutdown_connections 2024-07-31T15:10:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.944+0000 7f65a7a13740 1 -- 172.21.15.67:0/981655210 wait complete. 2024-07-31T15:10:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.945+0000 7f65a7a13740 20 client.29141 trim_cache size 0 max 16384 2024-07-31T15:10:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.959+0000 7f65a7a13740 1 Processor -- start 2024-07-31T15:10:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.959+0000 7f65a7a13740 1 -- start start 2024-07-31T15:10:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.959+0000 7f65a7a13740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96dee9f0 0x55ff971d3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.959+0000 7f65a7a13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e071b0 0x55ff971ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.959+0000 7f65a7a13740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e0f390 0x55ff971da720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.959+0000 7f65a7a13740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1ae4a0 con 0x55ff96dee9f0 2024-07-31T15:10:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.959+0000 7f65a7a13740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1affe0 con 0x55ff96e0f390 2024-07-31T15:10:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.959+0000 7f65a7a13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1aeb70 con 0x55ff96e071b0 2024-07-31T15:10:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.959+0000 7f659affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96dee9f0 0x55ff971d3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.960+0000 7f659affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96dee9f0 0x55ff971d3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44924/0 (socket says 172.21.15.67:44924) 2024-07-31T15:10:31.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.960+0000 7f659b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e0f390 0x55ff971da720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.960+0000 7f659affd640 1 -- 172.21.15.67:0/3608695488 learned_addr learned my addr 172.21.15.67:0/3608695488 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:31.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.960+0000 7f6593fff640 1 --2- 172.21.15.67:0/3608695488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e071b0 0x55ff971ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.960+0000 7f659affd640 1 -- 172.21.15.67:0/3608695488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e071b0 msgr2=0x55ff971ecfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:31.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.960+0000 7f659affd640 1 --2- 172.21.15.67:0/3608695488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e071b0 0x55ff971ecfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.960+0000 7f659affd640 1 -- 172.21.15.67:0/3608695488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e0f390 msgr2=0x55ff971da720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:31.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.960+0000 7f659affd640 1 --2- 172.21.15.67:0/3608695488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e0f390 0x55ff971da720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.960+0000 7f659affd640 1 -- 172.21.15.67:0/3608695488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff971dc960 con 0x55ff96dee9f0 2024-07-31T15:10:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.960+0000 7f659b7fe640 1 --2- 172.21.15.67:0/3608695488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e0f390 0x55ff971da720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.960+0000 7f659affd640 1 --2- 172.21.15.67:0/3608695488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96dee9f0 0x55ff971d3680 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f658c031440 tx=0x7f658c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2036342a0c5d367b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.960+0000 7f6590ff9640 1 -- 172.21.15.67:0/3608695488 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c057070 con 0x55ff96dee9f0 2024-07-31T15:10:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.961+0000 7f6590ff9640 1 -- 172.21.15.67:0/3608695488 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f658c004250 con 0x55ff96dee9f0 2024-07-31T15:10:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.961+0000 7f6590ff9640 1 -- 172.21.15.67:0/3608695488 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c035040 con 0x55ff96dee9f0 2024-07-31T15:10:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.961+0000 7f65a7a13740 1 -- 172.21.15.67:0/3608695488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96dee9f0 msgr2=0x55ff971d3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.961+0000 7f65a7a13740 1 --2- 172.21.15.67:0/3608695488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96dee9f0 0x55ff971d3680 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f658c031440 tx=0x7f658c009520 comp rx=0 tx=0).stop 2024-07-31T15:10:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 -- 172.21.15.67:0/3608695488 shutdown_connections 2024-07-31T15:10:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 --2- 172.21.15.67:0/3608695488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e0f390 0x55ff971da720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 --2- 172.21.15.67:0/3608695488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e071b0 0x55ff971ecfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 --2- 172.21.15.67:0/3608695488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96dee9f0 0x55ff971d3680 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 -- 172.21.15.67:0/3608695488 >> 172.21.15.67:0/3608695488 conn(0x55ff96e24460 msgr2=0x55ff971d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 -- 172.21.15.67:0/3608695488 shutdown_connections 2024-07-31T15:10:31.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 -- 172.21.15.67:0/3608695488 wait complete. 2024-07-31T15:10:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 -- start start 2024-07-31T15:10:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 Processor -- start 2024-07-31T15:10:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96dee9f0 0x55ff971e9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e071b0 0x55ff971d25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 0x55ff971d2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1af910 con 0x55ff96dee9f0 2024-07-31T15:10:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1af240 con 0x55ff96e071b0 2024-07-31T15:10:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1ac480 con 0x55ff96e0f390 2024-07-31T15:10:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.962+0000 7f65a7a13740 10 client.0 _ll_register_callbacks cb 0x7f658c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.963+0000 7f659affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96dee9f0 0x55ff971e9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.963+0000 7f6593fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e071b0 0x55ff971d25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.963+0000 7f659affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96dee9f0 0x55ff971e9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44928/0 (socket says 172.21.15.67:44928) 2024-07-31T15:10:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.963+0000 7f659affd640 1 -- 172.21.15.67:0/3965841369 learned_addr learned my addr 172.21.15.67:0/3965841369 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.963+0000 7f659b7fe640 1 --2- 172.21.15.67:0/3965841369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 0x55ff971d2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.963+0000 7f659b7fe640 1 -- 172.21.15.67:0/3965841369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e071b0 msgr2=0x55ff971d25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.963+0000 7f659b7fe640 1 --2- 172.21.15.67:0/3965841369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e071b0 0x55ff971d25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.963+0000 7f659b7fe640 1 -- 172.21.15.67:0/3965841369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96dee9f0 msgr2=0x55ff971e9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.963+0000 7f659b7fe640 1 --2- 172.21.15.67:0/3965841369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96dee9f0 0x55ff971e9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.963+0000 7f659b7fe640 1 -- 172.21.15.67:0/3965841369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff971d30e0 con 0x55ff96e0f390 2024-07-31T15:10:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.963+0000 7f659b7fe640 1 --2- 172.21.15.67:0/3965841369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 0x55ff971d2af0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6594006010 tx=0x7f6594011130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7c3c4d02c500d66 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.963+0000 7f65827fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:10:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f65827fc640 1 -- 172.21.15.67:0/3965841369 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659402d060 con 0x55ff96e0f390 2024-07-31T15:10:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f65a7a13740 1 -- 172.21.15.67:0/3965841369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f658c004250 con 0x55ff96e0f390 2024-07-31T15:10:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f65827fc640 1 -- 172.21.15.67:0/3965841369 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f6594011360 con 0x55ff96e0f390 2024-07-31T15:10:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f65827fc640 1 -- 172.21.15.67:0/3965841369 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f659400f9a0 con 0x55ff96e0f390 2024-07-31T15:10:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f65a7a13740 10 client.29014 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:31.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f65a7a13740 1 -- 172.21.15.67:0/3965841369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55ff971e6ed0 con 0x55ff96e0f390 2024-07-31T15:10:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f65827fc640 1 -- 172.21.15.67:0/3965841369 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f658c004250 con 0x55ff96e0f390 2024-07-31T15:10:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f65827fc640 1 -- 172.21.15.67:0/3965841369 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f6594018020 con 0x55ff96e0f390 2024-07-31T15:10:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f65a7a13740 10 client.29014 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f65a7a13740 10 client.29014 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:10:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f65a7a13740 20 client.29014 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f65a7a13740 1 --2- 172.21.15.67:0/3965841369 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55ff96e30070 0x55ff97266850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f65a7a13740 4 client.29014 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:10:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f65a7a13740 1 -- 172.21.15.67:0/3965841369 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff971495a0 con 0x55ff96e30070 2024-07-31T15:10:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.964+0000 7f659affd640 1 --2- 172.21.15.67:0/3965841369 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55ff96e30070 0x55ff97266850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.965+0000 7f65827fc640 10 client.29014 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:10:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.965+0000 7f659affd640 1 --2- 172.21.15.67:0/3965841369 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55ff96e30070 0x55ff97266850 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46d4f497f2b9cb31 server_cookie=55e63f662700e431 in_seq=0 out_seq=0 2024-07-31T15:10:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.965+0000 7f65827fc640 1 -- 172.21.15.67:0/3965841369 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff971495a0 con 0x55ff96e30070 2024-07-31T15:10:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.965+0000 7f65827fc640 10 client.29014 handle_command_reply: tid=0 2024-07-31T15:10:31.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.965+0000 7f65a7a13740 1 client.29014 shutdown 2024-07-31T15:10:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.965+0000 7f65a7a13740 1 -- 172.21.15.67:0/3965841369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 msgr2=0x55ff971d2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.965+0000 7f65a7a13740 1 --2- 172.21.15.67:0/3965841369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 0x55ff971d2af0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6594006010 tx=0x7f6594011130 comp rx=0 tx=0).stop 2024-07-31T15:10:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.965+0000 7f65a7a13740 1 -- 172.21.15.67:0/3965841369 shutdown_connections 2024-07-31T15:10:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.965+0000 7f65a7a13740 1 --2- 172.21.15.67:0/3965841369 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55ff96e30070 0x55ff97266850 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.965+0000 7f65a7a13740 1 --2- 172.21.15.67:0/3965841369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff96e0f390 0x55ff971d2af0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.965+0000 7f65a7a13740 1 --2- 172.21.15.67:0/3965841369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff96e071b0 0x55ff971d25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.965+0000 7f65a7a13740 1 --2- 172.21.15.67:0/3965841369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff96dee9f0 0x55ff971e9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.965+0000 7f65a7a13740 1 -- 172.21.15.67:0/3965841369 >> 172.21.15.67:0/3965841369 conn(0x55ff96e24460 msgr2=0x55ff971d66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.965+0000 7f65827fc640 0 client.29014 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:10:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.966+0000 7f65a7a13740 1 -- 172.21.15.67:0/3965841369 shutdown_connections 2024-07-31T15:10:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.966+0000 7f65a7a13740 1 -- 172.21.15.67:0/3965841369 wait complete. 2024-07-31T15:10:31.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.966+0000 7f65a7a13740 20 client.29014 trim_cache size 0 max 16384 2024-07-31T15:10:31.975 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:10:31.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.966+0000 7f65a14c2640 1 -- 172.21.15.67:0/1968954390 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65880b3e40 msgr2=0x7f65880b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:31.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.966+0000 7f65a14c2640 1 --2- 172.21.15.67:0/1968954390 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65880b3e40 0x7f65880b6350 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x55ff971d4310 tx=0x7f658c005890 comp rx=0 tx=0).stop 2024-07-31T15:10:31.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.966+0000 7f65a14c2640 1 -- 172.21.15.67:0/1968954390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f659c104d20 msgr2=0x7f659c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:31.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.966+0000 7f65a14c2640 1 --2- 172.21.15.67:0/1968954390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f659c104d20 0x7f659c19a4a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f658400e9c0 tx=0x7f658400ee80 comp rx=0 tx=0).stop 2024-07-31T15:10:31.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.966+0000 7f65a14c2640 1 -- 172.21.15.67:0/1968954390 shutdown_connections 2024-07-31T15:10:31.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.966+0000 7f65a14c2640 1 --2- 172.21.15.67:0/1968954390 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65880b3e40 0x7f65880b6350 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.966+0000 7f65a14c2640 1 --2- 172.21.15.67:0/1968954390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f659c10d8a0 0x7f659c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.966+0000 7f65a14c2640 1 --2- 172.21.15.67:0/1968954390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f659c104d20 0x7f659c19a4a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.966+0000 7f65a14c2640 1 --2- 172.21.15.67:0/1968954390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f659c1042f0 0x7f659c199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:31.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.966+0000 7f65a14c2640 1 -- 172.21.15.67:0/1968954390 >> 172.21.15.67:0/1968954390 conn(0x7f659c079190 msgr2=0x7f659c1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:31.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.967+0000 7f65a14c2640 1 -- 172.21.15.67:0/1968954390 shutdown_connections 2024-07-31T15:10:31.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:31.967+0000 7f65a14c2640 1 -- 172.21.15.67:0/1968954390 wait complete. 2024-07-31T15:10:32.372 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/fib_trie.o 2024-07-31T15:10:32.374 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/scan.o 2024-07-31T15:10:32.408 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/exthdrs.o 2024-07-31T15:10:32.435 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/timer.o 2024-07-31T15:10:32.447 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/fib_notifier.o 2024-07-31T15:10:32.468 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/ocb.o 2024-07-31T15:10:32.474 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/libata-scsi.o 2024-07-31T15:10:32.554 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_pm_debugfs.o 2024-07-31T15:10:32.727 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'dd84c0d1', total seconds elapsed: 39.4 2024-07-31T15:10:33.260 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:33 smithi067 ceph-mon[32750]: pgmap v1022: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-07-31T15:10:33.339 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/xdr.o 2024-07-31T15:10:33.342 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/offchannel.o 2024-07-31T15:10:33.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:33 smithi073 ceph-mon[36714]: pgmap v1022: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-07-31T15:10:33.655 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T15:10:33.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:33 smithi077 ceph-mon[37327]: pgmap v1022: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-07-31T15:10:33.850 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/datagram.o 2024-07-31T15:10:33.853 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_pm_irq.o 2024-07-31T15:10:33.903 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/pmsr.o 2024-07-31T15:10:34.169 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/libata-eh.o 2024-07-31T15:10:34.277 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/inet_fragment.o 2024-07-31T15:10:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:10:34.380 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/ip6_flowlabel.o 2024-07-31T15:10:34.688 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_requests.o 2024-07-31T15:10:35.107 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/ht.o 2024-07-31T15:10:35.142 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_sysfs.o 2024-07-31T15:10:35.407 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/sunrpc_syms.o 2024-07-31T15:10:35.511 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/inet6_connection_sock.o 2024-07-31T15:10:35.519 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/ping.o 2024-07-31T15:10:35.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:35 smithi067 ceph-mon[32750]: pgmap v1023: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 241 KiB/s wr, 41 op/s 2024-07-31T15:10:35.582 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_sysfs_pm.o 2024-07-31T15:10:35.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:35 smithi073 ceph-mon[36714]: pgmap v1023: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 241 KiB/s wr, 41 op/s 2024-07-31T15:10:35.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:35 smithi077 ceph-mon[37327]: pgmap v1023: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 241 KiB/s wr, 41 op/s 2024-07-31T15:10:35.864 INFO:tasks.workunit.client.0.smithi067.stdout: GEN net/wireless/shipped-certs.c 2024-07-31T15:10:35.911 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/wireless/shipped-certs.o 2024-07-31T15:10:35.957 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gtt.o 2024-07-31T15:10:36.365 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/cache.o 2024-07-31T15:10:36.366 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/libata-transport.o 2024-07-31T15:10:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.627+0000 7f29f8310640 1 -- 172.21.15.67:0/2785435841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29f010a5b0 msgr2=0x7f29f010a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.627+0000 7f29f8310640 1 --2- 172.21.15.67:0/2785435841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29f010a5b0 0x7f29f010a9c0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f29ec01a910 tx=0x7f29ec03b450 comp rx=0 tx=0).stop 2024-07-31T15:10:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.629+0000 7f29f8310640 1 -- 172.21.15.67:0/2785435841 shutdown_connections 2024-07-31T15:10:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.629+0000 7f29f8310640 1 --2- 172.21.15.67:0/2785435841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29f011c5d0 0x7f29f011e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.629+0000 7f29f8310640 1 --2- 172.21.15.67:0/2785435841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29f010a5b0 0x7f29f010a9c0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.629+0000 7f29f8310640 1 --2- 172.21.15.67:0/2785435841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f0111770 0x7f29f010a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.629+0000 7f29f8310640 1 -- 172.21.15.67:0/2785435841 >> 172.21.15.67:0/2785435841 conn(0x7f29f006d380 msgr2=0x7f29f006d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:36.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.630+0000 7f29f8310640 1 -- 172.21.15.67:0/2785435841 shutdown_connections 2024-07-31T15:10:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.630+0000 7f29f8310640 1 -- 172.21.15.67:0/2785435841 wait complete. 2024-07-31T15:10:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.631+0000 7f29f8310640 1 Processor -- start 2024-07-31T15:10:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.631+0000 7f29f8310640 1 -- start start 2024-07-31T15:10:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.631+0000 7f29f8310640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29f010a5b0 0x7f29f011beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.631+0000 7f29f8310640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f0111770 0x7f29f0118190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.631+0000 7f29f8310640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29f011c5d0 0x7f29f0118760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.631+0000 7f29f8310640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01145f0 con 0x7f29f011c5d0 2024-07-31T15:10:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.631+0000 7f29f8310640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0114470 con 0x7f29f0111770 2024-07-31T15:10:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.631+0000 7f29f8310640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0114770 con 0x7f29f010a5b0 2024-07-31T15:10:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.631+0000 7f29f58ab640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f0111770 0x7f29f0118190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.631+0000 7f29f58ab640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f0111770 0x7f29f0118190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:45830/0 (socket says 172.21.15.67:45830) 2024-07-31T15:10:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.631+0000 7f29f58ab640 1 -- 172.21.15.67:0/14825544 learned_addr learned my addr 172.21.15.67:0/14825544 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.632+0000 7f29f60ac640 1 --2- 172.21.15.67:0/14825544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29f010a5b0 0x7f29f011beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.632+0000 7f29f58ab640 1 -- 172.21.15.67:0/14825544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29f010a5b0 msgr2=0x7f29f011beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.632+0000 7f29f58ab640 1 --2- 172.21.15.67:0/14825544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29f010a5b0 0x7f29f011beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.632+0000 7f29f58ab640 1 -- 172.21.15.67:0/14825544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29f011c5d0 msgr2=0x7f29f0118760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.632+0000 7f29f58ab640 1 --2- 172.21.15.67:0/14825544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29f011c5d0 0x7f29f0118760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.632+0000 7f29f58ab640 1 -- 172.21.15.67:0/14825544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f0118f80 con 0x7f29f0111770 2024-07-31T15:10:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.632+0000 7f29f58ab640 1 --2- 172.21.15.67:0/14825544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f0111770 0x7f29f0118190 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f29ec006010 tx=0x7f29ec004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.637+0000 7f29e77fe640 1 -- 172.21.15.67:0/14825544 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec04f070 con 0x7f29f0111770 2024-07-31T15:10:36.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.637+0000 7f29f8310640 1 -- 172.21.15.67:0/14825544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f01ac6f0 con 0x7f29f0111770 2024-07-31T15:10:36.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.638+0000 7f29f8310640 1 -- 172.21.15.67:0/14825544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f01acba0 con 0x7f29f0111770 2024-07-31T15:10:36.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.638+0000 7f29e77fe640 1 -- 172.21.15.67:0/14825544 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29ec0409c0 con 0x7f29f0111770 2024-07-31T15:10:36.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.638+0000 7f29e77fe640 1 -- 172.21.15.67:0/14825544 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec046030 con 0x7f29f0111770 2024-07-31T15:10:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.642+0000 7f29fe861740 1 Processor -- start 2024-07-31T15:10:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.644+0000 7f29fe861740 1 -- start start 2024-07-31T15:10:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.644+0000 7f29fe861740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713cb4070 0x555714059710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.644+0000 7f29fe861740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714059c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.646+0000 7f29fe861740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x55571405a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.646+0000 7f29fe861740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555713fcfbb0 con 0x555713c93390 2024-07-31T15:10:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.646+0000 7f29fe861740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555713fbf0b0 con 0x555713cb4070 2024-07-31T15:10:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.646+0000 7f29fe861740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555713fc30f0 con 0x555713c8b1b0 2024-07-31T15:10:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.647+0000 7f29f60ac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714059c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.647+0000 7f29f60ac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714059c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34862/0 (socket says 172.21.15.67:34862) 2024-07-31T15:10:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.647+0000 7f29f60ac640 1 -- 172.21.15.67:0/1471695007 learned_addr learned my addr 172.21.15.67:0/1471695007 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.647+0000 7f29f58ab640 1 --2- 172.21.15.67:0/1471695007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x55571405a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.647+0000 7f29f60ac640 1 -- 172.21.15.67:0/1471695007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713cb4070 msgr2=0x555714059710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:10:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.647+0000 7f29f60ac640 1 --2- 172.21.15.67:0/1471695007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713cb4070 0x555714059710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.647+0000 7f29f60ac640 1 -- 172.21.15.67:0/1471695007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 msgr2=0x55571405a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.647+0000 7f29f60ac640 1 --2- 172.21.15.67:0/1471695007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x55571405a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.647+0000 7f29f60ac640 1 -- 172.21.15.67:0/1471695007 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555713f65b90 con 0x555713c8b1b0 2024-07-31T15:10:36.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.647+0000 7f29f58ab640 1 --2- 172.21.15.67:0/1471695007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x55571405a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:10:36.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.647+0000 7f29f60ac640 1 --2- 172.21.15.67:0/1471695007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714059c50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f29e8003a70 tx=0x7f29e8003f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=139d324c1c2dc3bd server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:36.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.648+0000 7f29e77fe640 1 -- 172.21.15.67:0/14825544 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f29ec052070 con 0x7f29f0111770 2024-07-31T15:10:36.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.649+0000 7f29e4d99640 1 -- 172.21.15.67:0/1471695007 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8011440 con 0x555713c8b1b0 2024-07-31T15:10:36.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.649+0000 7f29e77fe640 1 --2- 172.21.15.67:0/14825544 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f29dc0b3f50 0x7f29dc0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.649+0000 7f29e77fe640 1 -- 172.21.15.67:0/14825544 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f29ec116550 con 0x7f29f0111770 2024-07-31T15:10:36.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.650+0000 7f29e4d99640 1 -- 172.21.15.67:0/1471695007 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29e8015030 con 0x555713c8b1b0 2024-07-31T15:10:36.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.652+0000 7f29fe861740 1 -- 172.21.15.67:0/1471695007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 msgr2=0x555714059c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:36.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.652+0000 7f29fe861740 1 --2- 172.21.15.67:0/1471695007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714059c50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f29e8003a70 tx=0x7f29e8003f30 comp rx=0 tx=0).stop 2024-07-31T15:10:36.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.652+0000 7f29f68ad640 1 --2- 172.21.15.67:0/14825544 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f29dc0b3f50 0x7f29dc0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:36.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.652+0000 7f29fe861740 1 -- 172.21.15.67:0/1471695007 shutdown_connections 2024-07-31T15:10:36.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.652+0000 7f29fe861740 1 --2- 172.21.15.67:0/1471695007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x55571405a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.652+0000 7f29fe861740 1 --2- 172.21.15.67:0/1471695007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714059c50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.652+0000 7f29fe861740 1 --2- 172.21.15.67:0/1471695007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713cb4070 0x555714059710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.652+0000 7f29fe861740 1 -- 172.21.15.67:0/1471695007 >> 172.21.15.67:0/1471695007 conn(0x555713ca8460 msgr2=0x555714057600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:36.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.653+0000 7f29f68ad640 1 --2- 172.21.15.67:0/14825544 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f29dc0b3f50 0x7f29dc0b6460 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f29e000a740 tx=0x7f29e0005b40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:36.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.653+0000 7f29fe861740 1 -- 172.21.15.67:0/1471695007 shutdown_connections 2024-07-31T15:10:36.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.653+0000 7f29fe861740 1 -- 172.21.15.67:0/1471695007 wait complete. 2024-07-31T15:10:36.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.654+0000 7f29fe861740 1 -- start start 2024-07-31T15:10:36.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.654+0000 7f29fe861740 1 Processor -- start 2024-07-31T15:10:36.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.654+0000 7f29fe861740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714065250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.654+0000 7f29fe861740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x555714065790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.654+0000 7f29fe861740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713cb4070 0x555714071310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.654+0000 7f29fe861740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555713e22700 con 0x555713c93390 2024-07-31T15:10:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.654+0000 7f29fe861740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555713f619e0 con 0x555713cb4070 2024-07-31T15:10:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.654+0000 7f29fe861740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555714067f40 con 0x555713c8b1b0 2024-07-31T15:10:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.654+0000 7f29fe861740 10 client.0 _ll_register_callbacks cb 0x555713f7c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.655+0000 7f29f60ac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714065250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.655+0000 7f29f58ab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x555714065790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.655+0000 7f29f58ab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x555714065790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44950/0 (socket says 172.21.15.67:44950) 2024-07-31T15:10:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.655+0000 7f29f58ab640 1 -- 172.21.15.67:0/854200075 learned_addr learned my addr 172.21.15.67:0/854200075 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.655+0000 7f29f60ac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714065250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34866/0 (socket says 172.21.15.67:34866) 2024-07-31T15:10:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.655+0000 7f29f68ad640 1 --2- 172.21.15.67:0/854200075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713cb4070 0x555714071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.656+0000 7f29f68ad640 1 -- 172.21.15.67:0/854200075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 msgr2=0x555714065250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.656+0000 7f29f68ad640 1 --2- 172.21.15.67:0/854200075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714065250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.656+0000 7f29f68ad640 1 -- 172.21.15.67:0/854200075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 msgr2=0x555714065790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.656+0000 7f29f68ad640 1 --2- 172.21.15.67:0/854200075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x555714065790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.656+0000 7f29f68ad640 1 -- 172.21.15.67:0/854200075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555713d1f650 con 0x555713cb4070 2024-07-31T15:10:36.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.656+0000 7f29f68ad640 1 --2- 172.21.15.67:0/854200075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713cb4070 0x555714071310 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f29e000f6d0 tx=0x7f29e00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ff9bac548f0b30 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:36.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.656+0000 7f29d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:10:36.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.660+0000 7f29f58ab640 1 --2- 172.21.15.67:0/854200075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x555714065790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:36.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.660+0000 7f29d9ffb640 1 -- 172.21.15.67:0/854200075 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0020070 con 0x555713cb4070 2024-07-31T15:10:36.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.660+0000 7f29fe861740 1 -- 172.21.15.67:0/854200075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29e8015030 con 0x555713cb4070 2024-07-31T15:10:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.660+0000 7f29d9ffb640 1 -- 172.21.15.67:0/854200075 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f29e0004220 con 0x555713cb4070 2024-07-31T15:10:36.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.667+0000 7f29d9ffb640 1 -- 172.21.15.67:0/854200075 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f29e0004440 con 0x555713cb4070 2024-07-31T15:10:36.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.667+0000 7f29fe861740 10 client.29156 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:36.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.667+0000 7f29fe861740 1 -- 172.21.15.67:0/854200075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x555714095fc0 con 0x555713cb4070 2024-07-31T15:10:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.667+0000 7f29d9ffb640 1 -- 172.21.15.67:0/854200075 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29e8015030 con 0x555713cb4070 2024-07-31T15:10:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.667+0000 7f29d9ffb640 1 -- 172.21.15.67:0/854200075 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f29e001b070 con 0x555713cb4070 2024-07-31T15:10:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.667+0000 7f29fe861740 10 client.29156 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.667+0000 7f29fe861740 10 client.29156 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:10:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.667+0000 7f29fe861740 20 client.29156 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.667+0000 7f29fe861740 1 --2- 172.21.15.67:0/854200075 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555713c729f0 0x555714099860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.667+0000 7f29fe861740 4 client.29156 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:10:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.667+0000 7f29fe861740 1 -- 172.21.15.67:0/854200075 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555713fd1b00 con 0x555713c729f0 2024-07-31T15:10:36.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.672+0000 7f29f60ac640 1 --2- 172.21.15.67:0/854200075 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555713c729f0 0x555714099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:36.683 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/udp_offload.o 2024-07-31T15:10:36.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.684+0000 7f29f60ac640 1 --2- 172.21.15.67:0/854200075 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555713c729f0 0x555714099860 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7fbae0a58cfbad40 server_cookie=9557dd1421898883 in_seq=0 out_seq=0 2024-07-31T15:10:36.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.684+0000 7f29d9ffb640 10 client.29156 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:10:36.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.685+0000 7f29d9ffb640 1 -- 172.21.15.67:0/854200075 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555713fd1b00 con 0x555713c729f0 2024-07-31T15:10:36.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.685+0000 7f29d9ffb640 10 client.29156 handle_command_reply: tid=0 2024-07-31T15:10:36.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.685+0000 7f29fe861740 1 client.29156 shutdown 2024-07-31T15:10:36.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.686+0000 7f29fe861740 1 -- 172.21.15.67:0/854200075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713cb4070 msgr2=0x555714071310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:36.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.686+0000 7f29fe861740 1 --2- 172.21.15.67:0/854200075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713cb4070 0x555714071310 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f29e000f6d0 tx=0x7f29e00046c0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.688+0000 7f29fe861740 1 -- 172.21.15.67:0/854200075 shutdown_connections 2024-07-31T15:10:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.688+0000 7f29fe861740 1 --2- 172.21.15.67:0/854200075 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555713c729f0 0x555714099860 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.688+0000 7f29fe861740 1 --2- 172.21.15.67:0/854200075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713cb4070 0x555714071310 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.688+0000 7f29fe861740 1 --2- 172.21.15.67:0/854200075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x555714065790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.688+0000 7f29fe861740 1 --2- 172.21.15.67:0/854200075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714065250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.688+0000 7f29fe861740 1 -- 172.21.15.67:0/854200075 >> 172.21.15.67:0/854200075 conn(0x555713ca8460 msgr2=0x55571405c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.688+0000 7f29d9ffb640 0 client.29156 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:10:36.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.690+0000 7f29fe861740 1 -- 172.21.15.67:0/854200075 shutdown_connections 2024-07-31T15:10:36.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.691+0000 7f29fe861740 1 -- 172.21.15.67:0/854200075 wait complete. 2024-07-31T15:10:36.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.691+0000 7f29fe861740 20 client.29156 trim_cache size 0 max 16384 2024-07-31T15:10:36.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.722+0000 7f29fe861740 1 Processor -- start 2024-07-31T15:10:36.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.722+0000 7f29fe861740 1 -- start start 2024-07-31T15:10:36.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.722+0000 7f29fe861740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714057630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.723+0000 7f29fe861740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713c93390 0x55571405f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.723+0000 7f29f60ac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714057630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:36.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.723+0000 7f29f60ac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714057630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34880/0 (socket says 172.21.15.67:34880) 2024-07-31T15:10:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.723+0000 7f29f60ac640 1 -- 172.21.15.67:0/287314474 learned_addr learned my addr 172.21.15.67:0/287314474 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.723+0000 7f29f58ab640 1 --2- 172.21.15.67:0/287314474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713c93390 0x55571405f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.723+0000 7f29fe861740 1 --2- 172.21.15.67:0/287314474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713cb4070 0x555714071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.724+0000 7f29f68ad640 1 --2- 172.21.15.67:0/287314474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713cb4070 0x555714071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.724+0000 7f29fe861740 1 -- 172.21.15.67:0/287314474 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f29d400fea0 con 0x555713cb4070 2024-07-31T15:10:36.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.724+0000 7f29fe861740 1 -- 172.21.15.67:0/287314474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f29d40119e0 con 0x555713c93390 2024-07-31T15:10:36.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.724+0000 7f29fe861740 1 -- 172.21.15.67:0/287314474 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4010570 con 0x555713c8b1b0 2024-07-31T15:10:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.725+0000 7f29f58ab640 1 -- 172.21.15.67:0/287314474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 msgr2=0x555714057630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.725+0000 7f29f58ab640 1 --2- 172.21.15.67:0/287314474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714057630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.725+0000 7f29f58ab640 1 -- 172.21.15.67:0/287314474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713cb4070 msgr2=0x555714071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.725+0000 7f29f58ab640 1 --2- 172.21.15.67:0/287314474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713cb4070 0x555714071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.725+0000 7f29f58ab640 1 -- 172.21.15.67:0/287314474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55571405ba00 con 0x555713c93390 2024-07-31T15:10:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.725+0000 7f29f68ad640 1 --2- 172.21.15.67:0/287314474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713cb4070 0x555714071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.726+0000 7f29f58ab640 1 --2- 172.21.15.67:0/287314474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713c93390 0x55571405f830 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0dcaf0 tx=0x7f29ec019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d73b02e5e98df5bf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:36.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.726+0000 7f29dbfff640 1 -- 172.21.15.67:0/287314474 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec059040 con 0x555713c93390 2024-07-31T15:10:36.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.726+0000 7f29dbfff640 1 -- 172.21.15.67:0/287314474 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29ec0dd4f0 con 0x555713c93390 2024-07-31T15:10:36.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.728+0000 7f29fe861740 1 -- 172.21.15.67:0/287314474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713c93390 msgr2=0x55571405f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:36.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.728+0000 7f29fe861740 1 --2- 172.21.15.67:0/287314474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713c93390 0x55571405f830 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0dcaf0 tx=0x7f29ec019000 comp rx=0 tx=0).stop 2024-07-31T15:10:36.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.728+0000 7f29fe861740 1 -- 172.21.15.67:0/287314474 shutdown_connections 2024-07-31T15:10:36.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.728+0000 7f29fe861740 1 --2- 172.21.15.67:0/287314474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713cb4070 0x555714071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.728+0000 7f29fe861740 1 --2- 172.21.15.67:0/287314474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713c93390 0x55571405f830 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.728+0000 7f29fe861740 1 --2- 172.21.15.67:0/287314474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x555714057630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.728+0000 7f29fe861740 1 -- 172.21.15.67:0/287314474 >> 172.21.15.67:0/287314474 conn(0x555713ca8460 msgr2=0x555714056f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:36.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.730+0000 7f29fe861740 1 -- 172.21.15.67:0/287314474 shutdown_connections 2024-07-31T15:10:36.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.734+0000 7f29fe861740 1 -- 172.21.15.67:0/287314474 wait complete. 2024-07-31T15:10:36.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.734+0000 7f29fe861740 1 -- start start 2024-07-31T15:10:36.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.734+0000 7f29fe861740 1 Processor -- start 2024-07-31T15:10:36.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.735+0000 7f29fe861740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x55571406d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.735+0000 7f29fe861740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x555714068230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.735+0000 7f29fe861740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713cb4070 0x555714068770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.735+0000 7f29fe861740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4011310 con 0x555713c93390 2024-07-31T15:10:36.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.735+0000 7f29fe861740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4010c40 con 0x555713cb4070 2024-07-31T15:10:36.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.735+0000 7f29fe861740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f29d400de80 con 0x555713c8b1b0 2024-07-31T15:10:36.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.735+0000 7f29fe861740 10 client.0 _ll_register_callbacks cb 0x7f29e000a3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:36.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.735+0000 7f29f58ab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x555714068230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:36.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.736+0000 7f29f58ab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x555714068230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44968/0 (socket says 172.21.15.67:44968) 2024-07-31T15:10:36.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.736+0000 7f29f58ab640 1 -- 172.21.15.67:0/353120547 learned_addr learned my addr 172.21.15.67:0/353120547 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:36.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.736+0000 7f29f58ab640 1 -- 172.21.15.67:0/353120547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 msgr2=0x55571406d960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:10:36.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.736+0000 7f29f58ab640 1 --2- 172.21.15.67:0/353120547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x55571406d960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.736+0000 7f29f58ab640 1 -- 172.21.15.67:0/353120547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713cb4070 msgr2=0x555714068770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:36.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.736+0000 7f29f58ab640 1 --2- 172.21.15.67:0/353120547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713cb4070 0x555714068770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.736+0000 7f29f58ab640 1 -- 172.21.15.67:0/353120547 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555714068f00 con 0x555713c93390 2024-07-31T15:10:36.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.737+0000 7f29f58ab640 1 --2- 172.21.15.67:0/353120547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x555714068230 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f29ec000c00 tx=0x7f29ec0e5d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffaee79e69535f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.742+0000 7f29d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:10:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.742+0000 7f29d9ffb640 1 -- 172.21.15.67:0/353120547 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec0fb020 con 0x555713c93390 2024-07-31T15:10:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.742+0000 7f29d9ffb640 1 -- 172.21.15.67:0/353120547 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f29ec040720 con 0x555713c93390 2024-07-31T15:10:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.742+0000 7f29fe861740 1 -- 172.21.15.67:0/353120547 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29ec0dd4f0 con 0x555713c93390 2024-07-31T15:10:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.743+0000 7f29d9ffb640 1 -- 172.21.15.67:0/353120547 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f29ec059070 con 0x555713c93390 2024-07-31T15:10:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.743+0000 7f29fe861740 10 client.19635 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.743+0000 7f29fe861740 1 -- 172.21.15.67:0/353120547 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x555714100f70 con 0x555713c93390 2024-07-31T15:10:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.744+0000 7f29d9ffb640 1 -- 172.21.15.67:0/353120547 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29ec0dd4f0 con 0x555713c93390 2024-07-31T15:10:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.744+0000 7f29d9ffb640 1 -- 172.21.15.67:0/353120547 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f29ec0ff050 con 0x555713c93390 2024-07-31T15:10:36.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.744+0000 7f29fe861740 10 client.19635 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:36.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.744+0000 7f29fe861740 10 client.19635 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:10:36.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.744+0000 7f29fe861740 20 client.19635 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:36.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.744+0000 7f29fe861740 1 --2- 172.21.15.67:0/353120547 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555713c47da0 0x555714103760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:36.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.744+0000 7f29fe861740 4 client.19635 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:10:36.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.744+0000 7f29fe861740 1 -- 172.21.15.67:0/353120547 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555713fcd5a0 con 0x555713c47da0 2024-07-31T15:10:36.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.745+0000 7f29f60ac640 1 --2- 172.21.15.67:0/353120547 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555713c47da0 0x555714103760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:36.750 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/agg-tx.o 2024-07-31T15:10:36.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.750+0000 7f29f60ac640 1 --2- 172.21.15.67:0/353120547 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555713c47da0 0x555714103760 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c50866ec412bc51 server_cookie=acaae601dd1c40e2 in_seq=0 out_seq=0 2024-07-31T15:10:36.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.750+0000 7f29d9ffb640 10 client.19635 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:10:36.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.750+0000 7f29d9ffb640 1 -- 172.21.15.67:0/353120547 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555713fcd5a0 con 0x555713c47da0 2024-07-31T15:10:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.750+0000 7f29d9ffb640 10 client.19635 handle_command_reply: tid=0 2024-07-31T15:10:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.750+0000 7f29fe861740 1 client.19635 shutdown 2024-07-31T15:10:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.752+0000 7f29fe861740 1 -- 172.21.15.67:0/353120547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 msgr2=0x555714068230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.752+0000 7f29fe861740 1 --2- 172.21.15.67:0/353120547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x555714068230 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f29ec000c00 tx=0x7f29ec0e5d80 comp rx=0 tx=0).stop 2024-07-31T15:10:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.753+0000 7f29fe861740 1 -- 172.21.15.67:0/353120547 shutdown_connections 2024-07-31T15:10:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.753+0000 7f29fe861740 1 --2- 172.21.15.67:0/353120547 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x555713c47da0 0x555714103760 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.753+0000 7f29fe861740 1 --2- 172.21.15.67:0/353120547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555713cb4070 0x555714068770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.753+0000 7f29fe861740 1 --2- 172.21.15.67:0/353120547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555713c93390 0x555714068230 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.753+0000 7f29fe861740 1 --2- 172.21.15.67:0/353120547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555713c8b1b0 0x55571406d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.753+0000 7f29fe861740 1 -- 172.21.15.67:0/353120547 >> 172.21.15.67:0/353120547 conn(0x555713ca8460 msgr2=0x55571405a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.753+0000 7f29d9ffb640 0 client.19635 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:10:36.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.753+0000 7f29fe861740 1 -- 172.21.15.67:0/353120547 shutdown_connections 2024-07-31T15:10:36.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.753+0000 7f29fe861740 1 -- 172.21.15.67:0/353120547 wait complete. 2024-07-31T15:10:36.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.754+0000 7f29fe861740 20 client.19635 trim_cache size 0 max 16384 2024-07-31T15:10:36.755 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:10:36.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.755+0000 7f29f8310640 1 -- 172.21.15.67:0/14825544 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f29dc0b3f50 msgr2=0x7f29dc0b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:36.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.755+0000 7f29f8310640 1 --2- 172.21.15.67:0/14825544 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f29dc0b3f50 0x7f29dc0b6460 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f29e000a740 tx=0x7f29e0005b40 comp rx=0 tx=0).stop 2024-07-31T15:10:36.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.755+0000 7f29f8310640 1 -- 172.21.15.67:0/14825544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f0111770 msgr2=0x7f29f0118190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:36.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.755+0000 7f29f8310640 1 --2- 172.21.15.67:0/14825544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f0111770 0x7f29f0118190 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f29ec006010 tx=0x7f29ec004a30 comp rx=0 tx=0).stop 2024-07-31T15:10:36.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.755+0000 7f29f8310640 1 -- 172.21.15.67:0/14825544 shutdown_connections 2024-07-31T15:10:36.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.755+0000 7f29f8310640 1 --2- 172.21.15.67:0/14825544 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f29dc0b3f50 0x7f29dc0b6460 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.755+0000 7f29f8310640 1 --2- 172.21.15.67:0/14825544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29f011c5d0 0x7f29f0118760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.755+0000 7f29f8310640 1 --2- 172.21.15.67:0/14825544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29f0111770 0x7f29f0118190 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.755+0000 7f29f8310640 1 --2- 172.21.15.67:0/14825544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29f010a5b0 0x7f29f011beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:36.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.755+0000 7f29f8310640 1 -- 172.21.15.67:0/14825544 >> 172.21.15.67:0/14825544 conn(0x7f29f006d380 msgr2=0x7f29f0110400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:36.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.758+0000 7f29f8310640 1 -- 172.21.15.67:0/14825544 shutdown_connections 2024-07-31T15:10:36.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:36.758+0000 7f29f8310640 1 -- 172.21.15.67:0/14825544 wait complete. 2024-07-31T15:10:36.902 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/wireless/built-in.a 2024-07-31T15:10:36.951 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/libata-trace.o 2024-07-31T15:10:36.988 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/ip_tunnel_core.o 2024-07-31T15:10:37.064 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/libata-sata.o 2024-07-31T15:10:37.281 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/gre_offload.o 2024-07-31T15:10:37.310 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_llc.o 2024-07-31T15:10:37.509 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:37 smithi067 ceph-mon[32750]: pgmap v1024: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 217 KiB/s wr, 41 op/s 2024-07-31T15:10:37.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T15:10:37.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:37 smithi073 ceph-mon[36714]: pgmap v1024: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 217 KiB/s wr, 41 op/s 2024-07-31T15:10:37.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:37 smithi077 ceph-mon[37327]: pgmap v1024: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 217 KiB/s wr, 41 op/s 2024-07-31T15:10:37.765 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/seg6.o 2024-07-31T15:10:37.778 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_lrc.o 2024-07-31T15:10:38.086 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/libata-sff.o 2024-07-31T15:10:38.228 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/metrics.o 2024-07-31T15:10:38.252 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:10:38.306 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/libata-pmp.o 2024-07-31T15:10:38.362 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netlabel/netlabel_user.o 2024-07-31T15:10:38.515 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/agg-rx.o 2024-07-31T15:10:38.562 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/rpc_pipe.o 2024-07-31T15:10:38.722 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/fib6_notifier.o 2024-07-31T15:10:39.158 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_migrate.o 2024-07-31T15:10:39.227 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_mocs.o 2024-07-31T15:10:39.245 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netlabel/netlabel_kapi.o 2024-07-31T15:10:39.265 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/netlink.o 2024-07-31T15:10:39.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:39 smithi067 ceph-mon[32750]: pgmap v1025: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 418 KiB/s wr, 54 op/s 2024-07-31T15:10:39.578 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/rpl.o 2024-07-31T15:10:39.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:39 smithi073 ceph-mon[36714]: pgmap v1025: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 418 KiB/s wr, 54 op/s 2024-07-31T15:10:39.680 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/libata-acpi.o 2024-07-31T15:10:39.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:39 smithi077 ceph-mon[37327]: pgmap v1025: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 418 KiB/s wr, 54 op/s 2024-07-31T15:10:39.992 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/sysfs.o 2024-07-31T15:10:40.070 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netlabel/netlabel_domainhash.o 2024-07-31T15:10:40.109 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/vht.o 2024-07-31T15:10:40.123 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/nexthop.o 2024-07-31T15:10:40.367 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_ppgtt.o 2024-07-31T15:10:40.387 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/he.o 2024-07-31T15:10:40.452 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/ioam6.o 2024-07-31T15:10:40.652 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/libata-pata-timings.o 2024-07-31T15:10:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.664+0000 7fbe0ecac640 1 -- 172.21.15.67:0/3196476212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe0811c5d0 msgr2=0x7fbe0811e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.664+0000 7fbe0ecac640 1 --2- 172.21.15.67:0/3196476212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe0811c5d0 0x7fbe0811e9f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009860 tx=0x7fbdf802a410 comp rx=0 tx=0).stop 2024-07-31T15:10:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.664+0000 7fbe0ecac640 1 -- 172.21.15.67:0/3196476212 shutdown_connections 2024-07-31T15:10:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.664+0000 7fbe0ecac640 1 --2- 172.21.15.67:0/3196476212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe0811c5d0 0x7fbe0811e9f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.664+0000 7fbe0ecac640 1 --2- 172.21.15.67:0/3196476212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe0810a330 0x7fbe0810a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.664+0000 7fbe0ecac640 1 --2- 172.21.15.67:0/3196476212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe08111790 0x7fbe08109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.664+0000 7fbe0ecac640 1 -- 172.21.15.67:0/3196476212 >> 172.21.15.67:0/3196476212 conn(0x7fbe0806e420 msgr2=0x7fbe0806e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.667+0000 7fbe0ecac640 1 -- 172.21.15.67:0/3196476212 shutdown_connections 2024-07-31T15:10:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.667+0000 7fbe0ecac640 1 -- 172.21.15.67:0/3196476212 wait complete. 2024-07-31T15:10:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.668+0000 7fbe0ecac640 1 Processor -- start 2024-07-31T15:10:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.668+0000 7fbe0ecac640 1 -- start start 2024-07-31T15:10:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.668+0000 7fbe0ecac640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe0810a330 0x7fbe081a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.668+0000 7fbe0ecac640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe08111790 0x7fbe081a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.668+0000 7fbe0ecac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe0811c5d0 0x7fbe081ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.668+0000 7fbe0ecac640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08122090 con 0x7fbe08111790 2024-07-31T15:10:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.668+0000 7fbe0ecac640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08121f10 con 0x7fbe0810a330 2024-07-31T15:10:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.668+0000 7fbe0ecac640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08122210 con 0x7fbe0811c5d0 2024-07-31T15:10:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.669+0000 7fbe0d4a9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe08111790 0x7fbe081a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.669+0000 7fbe0d4a9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe08111790 0x7fbe081a9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44984/0 (socket says 172.21.15.67:44984) 2024-07-31T15:10:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.669+0000 7fbe0d4a9640 1 -- 172.21.15.67:0/1552500762 learned_addr learned my addr 172.21.15.67:0/1552500762 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.669+0000 7fbe0e4ab640 1 --2- 172.21.15.67:0/1552500762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe0811c5d0 0x7fbe081ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.669+0000 7fbe0dcaa640 1 --2- 172.21.15.67:0/1552500762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe0810a330 0x7fbe081a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.670+0000 7fbe0e4ab640 1 -- 172.21.15.67:0/1552500762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe0810a330 msgr2=0x7fbe081a8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.670+0000 7fbe0e4ab640 1 --2- 172.21.15.67:0/1552500762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe0810a330 0x7fbe081a8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.670+0000 7fbe0e4ab640 1 -- 172.21.15.67:0/1552500762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe08111790 msgr2=0x7fbe081a9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.670+0000 7fbe0e4ab640 1 --2- 172.21.15.67:0/1552500762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe08111790 0x7fbe081a9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.670+0000 7fbe0e4ab640 1 -- 172.21.15.67:0/1552500762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe081ae930 con 0x7fbe0811c5d0 2024-07-31T15:10:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.674+0000 7fbe0e4ab640 1 --2- 172.21.15.67:0/1552500762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe0811c5d0 0x7fbe081ae2c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbdf802a930 tx=0x7fbdf8030ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:40.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.679+0000 7fbdfeffd640 1 -- 172.21.15.67:0/1552500762 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8005390 con 0x7fbe0811c5d0 2024-07-31T15:10:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.679+0000 7fbdfeffd640 1 -- 172.21.15.67:0/1552500762 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdf8005530 con 0x7fbe0811c5d0 2024-07-31T15:10:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.679+0000 7fbdfeffd640 1 -- 172.21.15.67:0/1552500762 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8004920 con 0x7fbe0811c5d0 2024-07-31T15:10:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.679+0000 7fbe0ecac640 1 -- 172.21.15.67:0/1552500762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe081aebc0 con 0x7fbe0811c5d0 2024-07-31T15:10:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.679+0000 7fbe0ecac640 1 -- 172.21.15.67:0/1552500762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe081af050 con 0x7fbe0811c5d0 2024-07-31T15:10:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.698+0000 7fbe1645f740 1 Processor -- start 2024-07-31T15:10:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.700+0000 7fbe1645f740 1 -- start start 2024-07-31T15:10:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.700+0000 7fbe1645f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abce070 0x55fe2af73710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.700+0000 7fbe1645f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2aba51b0 0x55fe2af73c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.700+0000 7fbe1645f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2abad390 0x55fe2af74190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.700+0000 7fbe1645f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fe2aee9bb0 con 0x55fe2abad390 2024-07-31T15:10:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.700+0000 7fbe1645f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fe2aed90b0 con 0x55fe2aba51b0 2024-07-31T15:10:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.700+0000 7fbe1645f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fe2aedd0f0 con 0x55fe2abce070 2024-07-31T15:10:40.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.701+0000 7fbdfeffd640 1 -- 172.21.15.67:0/1552500762 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbdf8039040 con 0x7fbe0811c5d0 2024-07-31T15:10:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.701+0000 7fbe0d4a9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abce070 0x55fe2af73710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.701+0000 7fbe0d4a9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abce070 0x55fe2af73710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34926/0 (socket says 172.21.15.67:34926) 2024-07-31T15:10:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.701+0000 7fbe0d4a9640 1 -- 172.21.15.67:0/3697031906 learned_addr learned my addr 172.21.15.67:0/3697031906 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.701+0000 7fbe0d4a9640 1 --2- 172.21.15.67:0/3697031906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2abad390 0x55fe2af74190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.702+0000 7fbe0d4a9640 1 -- 172.21.15.67:0/3697031906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abce070 msgr2=0x55fe2af73710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.702+0000 7fbe0d4a9640 1 --2- 172.21.15.67:0/3697031906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abce070 0x55fe2af73710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.702+0000 7fbe0d4a9640 1 -- 172.21.15.67:0/3697031906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2aba51b0 msgr2=0x55fe2af73c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:10:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.702+0000 7fbe0d4a9640 1 --2- 172.21.15.67:0/3697031906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2aba51b0 0x55fe2af73c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.702+0000 7fbe0d4a9640 1 -- 172.21.15.67:0/3697031906 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe2ae7fb90 con 0x55fe2abad390 2024-07-31T15:10:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.702+0000 7fbdfeffd640 1 --2- 172.21.15.67:0/1552500762 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbdf40b3f40 0x7fbdf40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.702+0000 7fbdfeffd640 1 -- 172.21.15.67:0/1552500762 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7fbdf8105cc0 con 0x7fbe0811c5d0 2024-07-31T15:10:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.702+0000 7fbe0d4a9640 1 --2- 172.21.15.67:0/3697031906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2abad390 0x55fe2af74190 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fbe04012680 tx=0x7fbe0400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fff6f1f55bf5b8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.703+0000 7fbe0dcaa640 1 --2- 172.21.15.67:0/1552500762 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbdf40b3f40 0x7fbdf40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.703+0000 7fbdf3fff640 1 -- 172.21.15.67:0/3697031906 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04004230 con 0x55fe2abad390 2024-07-31T15:10:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.703+0000 7fbdf3fff640 1 -- 172.21.15.67:0/3697031906 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe040043d0 con 0x55fe2abad390 2024-07-31T15:10:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.703+0000 7fbdf3fff640 1 -- 172.21.15.67:0/3697031906 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe040035f0 con 0x55fe2abad390 2024-07-31T15:10:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.704+0000 7fbe0dcaa640 1 --2- 172.21.15.67:0/1552500762 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbdf40b3f40 0x7fbdf40b6450 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbe00007350 tx=0x7fbe000072e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.707+0000 7fbe1645f740 1 -- 172.21.15.67:0/3697031906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2abad390 msgr2=0x55fe2af74190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.707+0000 7fbe1645f740 1 --2- 172.21.15.67:0/3697031906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2abad390 0x55fe2af74190 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fbe04012680 tx=0x7fbe0400b480 comp rx=0 tx=0).stop 2024-07-31T15:10:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.712+0000 7fbe1645f740 1 -- 172.21.15.67:0/3697031906 shutdown_connections 2024-07-31T15:10:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.712+0000 7fbe1645f740 1 --2- 172.21.15.67:0/3697031906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2abad390 0x55fe2af74190 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.712+0000 7fbe1645f740 1 --2- 172.21.15.67:0/3697031906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2aba51b0 0x55fe2af73c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.712+0000 7fbe1645f740 1 --2- 172.21.15.67:0/3697031906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abce070 0x55fe2af73710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.712+0000 7fbe1645f740 1 -- 172.21.15.67:0/3697031906 >> 172.21.15.67:0/3697031906 conn(0x55fe2abc2460 msgr2=0x55fe2af71600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:40.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.717+0000 7fbe1645f740 1 -- 172.21.15.67:0/3697031906 shutdown_connections 2024-07-31T15:10:40.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.720+0000 7fbe1645f740 1 -- 172.21.15.67:0/3697031906 wait complete. 2024-07-31T15:10:40.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.721+0000 7fbe1645f740 1 -- start start 2024-07-31T15:10:40.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.721+0000 7fbe1645f740 1 Processor -- start 2024-07-31T15:10:40.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.722+0000 7fbe1645f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2aba51b0 0x55fe2af7f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.722+0000 7fbe1645f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2abad390 0x55fe2af7f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.722+0000 7fbe1645f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2abce070 0x55fe2af8b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.722+0000 7fbe1645f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fe2ad3c700 con 0x55fe2abce070 2024-07-31T15:10:40.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.722+0000 7fbe1645f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fe2ae7b9e0 con 0x55fe2abad390 2024-07-31T15:10:40.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.722+0000 7fbe1645f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fe2af81ef0 con 0x55fe2aba51b0 2024-07-31T15:10:40.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.722+0000 7fbe1645f740 10 client.0 _ll_register_callbacks cb 0x55fe2ae964f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:40.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.726+0000 7fbe0dcaa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2aba51b0 0x55fe2af7f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.726+0000 7fbe0dcaa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2aba51b0 0x55fe2af7f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:34934/0 (socket says 172.21.15.67:34934) 2024-07-31T15:10:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.726+0000 7fbe0dcaa640 1 -- 172.21.15.67:0/3073444099 learned_addr learned my addr 172.21.15.67:0/3073444099 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.726+0000 7fbe0e4ab640 1 --2- 172.21.15.67:0/3073444099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2abce070 0x55fe2af8b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.726+0000 7fbe0d4a9640 1 --2- 172.21.15.67:0/3073444099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2abad390 0x55fe2af7f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.727+0000 7fbe0e4ab640 1 -- 172.21.15.67:0/3073444099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2aba51b0 msgr2=0x55fe2af7f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.727+0000 7fbe0e4ab640 1 --2- 172.21.15.67:0/3073444099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2aba51b0 0x55fe2af7f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.727+0000 7fbe0e4ab640 1 -- 172.21.15.67:0/3073444099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2abad390 msgr2=0x55fe2af7f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.727+0000 7fbe0e4ab640 1 --2- 172.21.15.67:0/3073444099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2abad390 0x55fe2af7f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.727+0000 7fbe0e4ab640 1 -- 172.21.15.67:0/3073444099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe2ac39650 con 0x55fe2abce070 2024-07-31T15:10:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.727+0000 7fbe0e4ab640 1 --2- 172.21.15.67:0/3073444099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2abce070 0x55fe2af8b270 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fbdf81030a0 tx=0x7fbdf80cc070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab1fc790ba7b2ab server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.728+0000 7fbdf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:10:40.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.728+0000 7fbdf17fa640 1 -- 172.21.15.67:0/3073444099 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf80e6060 con 0x55fe2abce070 2024-07-31T15:10:40.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.728+0000 7fbe1645f740 1 -- 172.21.15.67:0/3073444099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe040043d0 con 0x55fe2abce070 2024-07-31T15:10:40.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.728+0000 7fbdf17fa640 1 -- 172.21.15.67:0/3073444099 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7fbdf8031a50 con 0x55fe2abce070 2024-07-31T15:10:40.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.728+0000 7fbdf17fa640 1 -- 172.21.15.67:0/3073444099 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fbdf8049040 con 0x55fe2abce070 2024-07-31T15:10:40.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.734+0000 7fbe1645f740 10 client.19644 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:40.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.734+0000 7fbe1645f740 1 -- 172.21.15.67:0/3073444099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55fe2af8e010 con 0x55fe2abce070 2024-07-31T15:10:40.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.734+0000 7fbdf17fa640 1 -- 172.21.15.67:0/3073444099 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe040043d0 con 0x55fe2abce070 2024-07-31T15:10:40.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.734+0000 7fbdf17fa640 1 -- 172.21.15.67:0/3073444099 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fbdf80d5180 con 0x55fe2abce070 2024-07-31T15:10:40.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.734+0000 7fbe1645f740 10 client.19644 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:40.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.734+0000 7fbe1645f740 10 client.19644 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:10:40.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.734+0000 7fbe1645f740 20 client.19644 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:40.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.734+0000 7fbe1645f740 1 --2- 172.21.15.67:0/3073444099 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55fe2ab8c9f0 0x55fe2af916c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.735+0000 7fbe1645f740 4 client.19644 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:10:40.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.735+0000 7fbe1645f740 1 -- 172.21.15.67:0/3073444099 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe2aeebb00 con 0x55fe2ab8c9f0 2024-07-31T15:10:40.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.736+0000 7fbe0dcaa640 1 --2- 172.21.15.67:0/3073444099 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55fe2ab8c9f0 0x55fe2af916c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:40.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.756+0000 7fbe0dcaa640 1 --2- 172.21.15.67:0/3073444099 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55fe2ab8c9f0 0x55fe2af916c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7eb20e83bd1eaad server_cookie=c2383422e6f052e in_seq=0 out_seq=0 2024-07-31T15:10:40.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.757+0000 7fbdf17fa640 10 client.19644 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:10:40.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.761+0000 7fbdf17fa640 1 -- 172.21.15.67:0/3073444099 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe2aeebb00 con 0x55fe2ab8c9f0 2024-07-31T15:10:40.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.761+0000 7fbdf17fa640 10 client.19644 handle_command_reply: tid=0 2024-07-31T15:10:40.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.761+0000 7fbe1645f740 1 client.19644 shutdown 2024-07-31T15:10:40.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.766+0000 7fbe1645f740 1 -- 172.21.15.67:0/3073444099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2abce070 msgr2=0x55fe2af8b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:40.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.766+0000 7fbe1645f740 1 --2- 172.21.15.67:0/3073444099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2abce070 0x55fe2af8b270 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fbdf81030a0 tx=0x7fbdf80cc070 comp rx=0 tx=0).stop 2024-07-31T15:10:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.766+0000 7fbe1645f740 1 -- 172.21.15.67:0/3073444099 shutdown_connections 2024-07-31T15:10:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.766+0000 7fbe1645f740 1 --2- 172.21.15.67:0/3073444099 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55fe2ab8c9f0 0x55fe2af916c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.766+0000 7fbe1645f740 1 --2- 172.21.15.67:0/3073444099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2abce070 0x55fe2af8b270 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.766+0000 7fbe1645f740 1 --2- 172.21.15.67:0/3073444099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2abad390 0x55fe2af7f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.766+0000 7fbe1645f740 1 --2- 172.21.15.67:0/3073444099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2aba51b0 0x55fe2af7f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.766+0000 7fbe1645f740 1 -- 172.21.15.67:0/3073444099 >> 172.21.15.67:0/3073444099 conn(0x55fe2abc2460 msgr2=0x55fe2af768a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.766+0000 7fbdf17fa640 0 client.19644 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:10:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.766+0000 7fbe1645f740 1 -- 172.21.15.67:0/3073444099 shutdown_connections 2024-07-31T15:10:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.766+0000 7fbe1645f740 1 -- 172.21.15.67:0/3073444099 wait complete. 2024-07-31T15:10:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.766+0000 7fbe1645f740 20 client.19644 trim_cache size 0 max 16384 2024-07-31T15:10:40.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.800+0000 7fbe1645f740 1 Processor -- start 2024-07-31T15:10:40.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.805+0000 7fbe1645f740 1 -- start start 2024-07-31T15:10:40.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.805+0000 7fbe1645f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2abc2460 0x55fe2af78f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.805+0000 7fbe1645f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2aba51b0 0x55fe2af8b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.805+0000 7fbe1645f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abad390 0x55fe2af70f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.805+0000 7fbe1645f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec00fe60 con 0x55fe2aba51b0 2024-07-31T15:10:40.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.805+0000 7fbe1645f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec0119a0 con 0x55fe2abc2460 2024-07-31T15:10:40.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.805+0000 7fbe1645f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec010530 con 0x55fe2abad390 2024-07-31T15:10:40.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.806+0000 7fbe0e4ab640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abad390 0x55fe2af70f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:40.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.806+0000 7fbe0d4a9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2aba51b0 0x55fe2af8b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:40.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.806+0000 7fbe0d4a9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2aba51b0 0x55fe2af8b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45006/0 (socket says 172.21.15.67:45006) 2024-07-31T15:10:40.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.806+0000 7fbe0d4a9640 1 -- 172.21.15.67:0/2875993364 learned_addr learned my addr 172.21.15.67:0/2875993364 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:40.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.806+0000 7fbe0d4a9640 1 -- 172.21.15.67:0/2875993364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abad390 msgr2=0x55fe2af70f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:40.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.806+0000 7fbe0d4a9640 1 --2- 172.21.15.67:0/2875993364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abad390 0x55fe2af70f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.807+0000 7fbe0d4a9640 1 -- 172.21.15.67:0/2875993364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2abc2460 msgr2=0x55fe2af78f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:10:40.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.807+0000 7fbe0d4a9640 1 --2- 172.21.15.67:0/2875993364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2abc2460 0x55fe2af78f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.807+0000 7fbe0d4a9640 1 -- 172.21.15.67:0/2875993364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe2af7f6c0 con 0x55fe2aba51b0 2024-07-31T15:10:40.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.807+0000 7fbe0d4a9640 1 --2- 172.21.15.67:0/2875993364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2aba51b0 0x55fe2af8b060 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fbe0401b000 tx=0x7fbe0400bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa70e7448afda352 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:40.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.810+0000 7fbdf37fe640 1 -- 172.21.15.67:0/2875993364 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe040052c0 con 0x55fe2aba51b0 2024-07-31T15:10:40.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.810+0000 7fbdf37fe640 1 -- 172.21.15.67:0/2875993364 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe04005460 con 0x55fe2aba51b0 2024-07-31T15:10:40.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.810+0000 7fbdf37fe640 1 -- 172.21.15.67:0/2875993364 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0401d810 con 0x55fe2aba51b0 2024-07-31T15:10:40.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.813+0000 7fbe1645f740 1 -- 172.21.15.67:0/2875993364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2aba51b0 msgr2=0x55fe2af8b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:40.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.813+0000 7fbe1645f740 1 --2- 172.21.15.67:0/2875993364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2aba51b0 0x55fe2af8b060 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fbe0401b000 tx=0x7fbe0400bb60 comp rx=0 tx=0).stop 2024-07-31T15:10:40.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.813+0000 7fbe1645f740 1 -- 172.21.15.67:0/2875993364 shutdown_connections 2024-07-31T15:10:40.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.813+0000 7fbe1645f740 1 --2- 172.21.15.67:0/2875993364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abad390 0x55fe2af70f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.813+0000 7fbe1645f740 1 --2- 172.21.15.67:0/2875993364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2aba51b0 0x55fe2af8b060 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.813+0000 7fbe1645f740 1 --2- 172.21.15.67:0/2875993364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2abc2460 0x55fe2af78f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.813+0000 7fbe1645f740 1 -- 172.21.15.67:0/2875993364 >> 172.21.15.67:0/2875993364 conn(0x55fe2af79b80 msgr2=0x55fe2af78820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:40.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.821+0000 7fbe1645f740 1 -- 172.21.15.67:0/2875993364 shutdown_connections 2024-07-31T15:10:40.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.825+0000 7fbe1645f740 1 -- 172.21.15.67:0/2875993364 wait complete. 2024-07-31T15:10:40.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.825+0000 7fbe1645f740 1 -- start start 2024-07-31T15:10:40.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.825+0000 7fbe1645f740 1 Processor -- start 2024-07-31T15:10:40.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.826+0000 7fbe1645f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2aba51b0 0x55fe2af87940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.826+0000 7fbe1645f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2abad390 0x55fe2af82e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.826+0000 7fbe1645f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abc2460 0x55fe2af83440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.826+0000 7fbe1645f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec0112d0 con 0x55fe2aba51b0 2024-07-31T15:10:40.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.826+0000 7fbe1645f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec010c00 con 0x55fe2abad390 2024-07-31T15:10:40.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.826+0000 7fbe1645f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec00de40 con 0x55fe2abc2460 2024-07-31T15:10:40.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.826+0000 7fbe1645f740 10 client.0 _ll_register_callbacks cb 0x7fbdf8005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:40.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.828+0000 7fbe0dcaa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2aba51b0 0x55fe2af87940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:40.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.828+0000 7fbe0dcaa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2aba51b0 0x55fe2af87940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47644/0 (socket says 172.21.15.67:47644) 2024-07-31T15:10:40.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.828+0000 7fbe0dcaa640 1 -- 172.21.15.67:0/1922577453 learned_addr learned my addr 172.21.15.67:0/1922577453 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:40.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.829+0000 7fbe0dcaa640 1 -- 172.21.15.67:0/1922577453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abc2460 msgr2=0x55fe2af83440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:10:40.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.829+0000 7fbe0dcaa640 1 --2- 172.21.15.67:0/1922577453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abc2460 0x55fe2af83440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.829+0000 7fbe0dcaa640 1 -- 172.21.15.67:0/1922577453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2abad390 msgr2=0x55fe2af82e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:10:40.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.829+0000 7fbe0dcaa640 1 --2- 172.21.15.67:0/1922577453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2abad390 0x55fe2af82e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.829+0000 7fbe0dcaa640 1 -- 172.21.15.67:0/1922577453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe2af83ba0 con 0x55fe2aba51b0 2024-07-31T15:10:40.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.829+0000 7fbe0dcaa640 1 --2- 172.21.15.67:0/1922577453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2aba51b0 0x55fe2af87940 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fbe00010a80 tx=0x7fbe00004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5893f7d0d296f6df server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:40.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.831+0000 7fbdf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:10:40.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.835+0000 7fbdf17fa640 1 -- 172.21.15.67:0/1922577453 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0001ecc0 con 0x55fe2aba51b0 2024-07-31T15:10:40.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.835+0000 7fbe1645f740 1 -- 172.21.15.67:0/1922577453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe04005460 con 0x55fe2aba51b0 2024-07-31T15:10:40.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.835+0000 7fbdf17fa640 1 -- 172.21.15.67:0/1922577453 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7fbe000262d0 con 0x55fe2aba51b0 2024-07-31T15:10:40.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.839+0000 7fbdf17fa640 1 -- 172.21.15.67:0/1922577453 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fbe0001ee60 con 0x55fe2aba51b0 2024-07-31T15:10:40.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.840+0000 7fbe1645f740 10 client.19650 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:40.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.840+0000 7fbe1645f740 1 -- 172.21.15.67:0/1922577453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55fe2aff9ef0 con 0x55fe2aba51b0 2024-07-31T15:10:40.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.843+0000 7fbdf17fa640 1 -- 172.21.15.67:0/1922577453 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe04005460 con 0x55fe2aba51b0 2024-07-31T15:10:40.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.843+0000 7fbdf17fa640 1 -- 172.21.15.67:0/1922577453 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fbe0001d560 con 0x55fe2aba51b0 2024-07-31T15:10:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.844+0000 7fbe1645f740 10 client.19650 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.844+0000 7fbe1645f740 10 client.19650 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:10:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.844+0000 7fbe1645f740 20 client.19650 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.844+0000 7fbe1645f740 1 --2- 172.21.15.67:0/1922577453 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55fe2abce070 0x55fe2affc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.844+0000 7fbe1645f740 4 client.19650 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:10:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.844+0000 7fbe1645f740 1 -- 172.21.15.67:0/1922577453 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe2aee75a0 con 0x55fe2abce070 2024-07-31T15:10:40.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.852+0000 7fbe0d4a9640 1 --2- 172.21.15.67:0/1922577453 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55fe2abce070 0x55fe2affc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:40.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.872+0000 7fbe0d4a9640 1 --2- 172.21.15.67:0/1922577453 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55fe2abce070 0x55fe2affc6c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2458f4c206f6aea9 server_cookie=b4007ef4c91ed7a6 in_seq=0 out_seq=0 2024-07-31T15:10:40.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.873+0000 7fbdf17fa640 10 client.19650 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:10:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.877+0000 7fbdf17fa640 1 -- 172.21.15.67:0/1922577453 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe2aee75a0 con 0x55fe2abce070 2024-07-31T15:10:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.877+0000 7fbdf17fa640 10 client.19650 handle_command_reply: tid=0 2024-07-31T15:10:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.877+0000 7fbe1645f740 1 client.19650 shutdown 2024-07-31T15:10:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.886+0000 7fbe1645f740 1 -- 172.21.15.67:0/1922577453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2aba51b0 msgr2=0x55fe2af87940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.886+0000 7fbe1645f740 1 --2- 172.21.15.67:0/1922577453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2aba51b0 0x55fe2af87940 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fbe00010a80 tx=0x7fbe00004ab0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.886+0000 7fbe1645f740 1 -- 172.21.15.67:0/1922577453 shutdown_connections 2024-07-31T15:10:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.886+0000 7fbe1645f740 1 --2- 172.21.15.67:0/1922577453 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55fe2abce070 0x55fe2affc6c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.886+0000 7fbe1645f740 1 --2- 172.21.15.67:0/1922577453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe2abc2460 0x55fe2af83440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.886+0000 7fbe1645f740 1 --2- 172.21.15.67:0/1922577453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe2abad390 0x55fe2af82e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.886+0000 7fbe1645f740 1 --2- 172.21.15.67:0/1922577453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fe2aba51b0 0x55fe2af87940 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.886+0000 7fbe1645f740 1 -- 172.21.15.67:0/1922577453 >> 172.21.15.67:0/1922577453 conn(0x55fe2af79b80 msgr2=0x55fe2af70f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.886+0000 7fbdf17fa640 0 client.19650 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:10:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.886+0000 7fbe1645f740 1 -- 172.21.15.67:0/1922577453 shutdown_connections 2024-07-31T15:10:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.886+0000 7fbe1645f740 1 -- 172.21.15.67:0/1922577453 wait complete. 2024-07-31T15:10:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.886+0000 7fbe1645f740 20 client.19650 trim_cache size 0 max 16384 2024-07-31T15:10:40.889 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:10:40.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.891+0000 7fbe0ecac640 1 -- 172.21.15.67:0/1552500762 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbdf40b3f40 msgr2=0x7fbdf40b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:40.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.891+0000 7fbe0ecac640 1 --2- 172.21.15.67:0/1552500762 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbdf40b3f40 0x7fbdf40b6450 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbe00007350 tx=0x7fbe000072e0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.891+0000 7fbe0ecac640 1 -- 172.21.15.67:0/1552500762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe0811c5d0 msgr2=0x7fbe081ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:40.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.891+0000 7fbe0ecac640 1 --2- 172.21.15.67:0/1552500762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe0811c5d0 0x7fbe081ae2c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbdf802a930 tx=0x7fbdf8030ec0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.893+0000 7fbe0ecac640 1 -- 172.21.15.67:0/1552500762 shutdown_connections 2024-07-31T15:10:40.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.893+0000 7fbe0ecac640 1 --2- 172.21.15.67:0/1552500762 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbdf40b3f40 0x7fbdf40b6450 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.893+0000 7fbe0ecac640 1 --2- 172.21.15.67:0/1552500762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe0811c5d0 0x7fbe081ae2c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.893+0000 7fbe0ecac640 1 --2- 172.21.15.67:0/1552500762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe08111790 0x7fbe081a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.893+0000 7fbe0ecac640 1 --2- 172.21.15.67:0/1552500762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe0810a330 0x7fbe081a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:40.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.893+0000 7fbe0ecac640 1 -- 172.21.15.67:0/1552500762 >> 172.21.15.67:0/1552500762 conn(0x7fbe0806e420 msgr2=0x7fbe08071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:40.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.894+0000 7fbe0ecac640 1 -- 172.21.15.67:0/1552500762 shutdown_connections 2024-07-31T15:10:40.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:40.898+0000 7fbe0ecac640 1 -- 172.21.15.67:0/1552500762 wait complete. 2024-07-31T15:10:41.029 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/svc_xprt.o 2024-07-31T15:10:41.181 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netlabel/netlabel_addrlist.o 2024-07-31T15:10:41.346 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/ahci.o 2024-07-31T15:10:41.362 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/xprtmultipath.o 2024-07-31T15:10:41.365 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_rc6.o 2024-07-31T15:10:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:41 smithi067 ceph-mon[32750]: pgmap v1026: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 467 KiB/s wr, 56 op/s 2024-07-31T15:10:41.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:41 smithi073 ceph-mon[36714]: pgmap v1026: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 467 KiB/s wr, 56 op/s 2024-07-31T15:10:41.705 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/s1g.o 2024-07-31T15:10:41.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:41 smithi077 ceph-mon[37327]: pgmap v1026: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 467 KiB/s wr, 56 op/s 2024-07-31T15:10:41.822 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T15:10:41.889 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/sysctl_net_ipv6.o 2024-07-31T15:10:42.147 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netlabel/netlabel_mgmt.o 2024-07-31T15:10:42.177 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/udp_tunnel_stub.o 2024-07-31T15:10:42.472 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_region_lmem.o 2024-07-31T15:10:42.477 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netlabel/netlabel_unlabeled.o 2024-07-31T15:10:42.481 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/libahci.o 2024-07-31T15:10:42.501 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:10:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.649+0000 7f19fb0e3640 1 -- 172.21.15.67:0/1011593349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19f410c2c0 msgr2=0x7f19f4114800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.649+0000 7f19fb0e3640 1 --2- 172.21.15.67:0/1011593349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19f410c2c0 0x7f19f4114800 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009be0 tx=0x7f19ec02a6c0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.654+0000 7f19fb0e3640 1 -- 172.21.15.67:0/1011593349 shutdown_connections 2024-07-31T15:10:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.654+0000 7f19fb0e3640 1 --2- 172.21.15.67:0/1011593349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19f4114ea0 0x7f19f41173b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.654+0000 7f19fb0e3640 1 --2- 172.21.15.67:0/1011593349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19f410c2c0 0x7f19f4114800 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.654+0000 7f19fb0e3640 1 --2- 172.21.15.67:0/1011593349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f410b970 0x7f19f410bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.654+0000 7f19fb0e3640 1 -- 172.21.15.67:0/1011593349 >> 172.21.15.67:0/1011593349 conn(0x7f19f40707c0 msgr2=0x7f19f4070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.654+0000 7f19fb0e3640 1 -- 172.21.15.67:0/1011593349 shutdown_connections 2024-07-31T15:10:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.654+0000 7f19fb0e3640 1 -- 172.21.15.67:0/1011593349 wait complete. 2024-07-31T15:10:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.655+0000 7f19fb0e3640 1 Processor -- start 2024-07-31T15:10:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.655+0000 7f19fb0e3640 1 -- start start 2024-07-31T15:10:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.656+0000 7f19fb0e3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19f410b970 0x7f19f41a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.656+0000 7f19fb0e3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19f410c2c0 0x7f19f41a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.656+0000 7f19fb0e3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f4114ea0 0x7f19f41a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.656+0000 7f19fb0e3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f19f411ab70 con 0x7f19f4114ea0 2024-07-31T15:10:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.656+0000 7f19fb0e3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f19f411a9f0 con 0x7f19f410b970 2024-07-31T15:10:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.656+0000 7f19fb0e3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19f411acf0 con 0x7f19f410c2c0 2024-07-31T15:10:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.656+0000 7f19f9680640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f4114ea0 0x7f19f41a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.656+0000 7f19f3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19f410c2c0 0x7f19f41a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.656+0000 7f19f3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19f410c2c0 0x7f19f41a3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:36284/0 (socket says 172.21.15.67:36284) 2024-07-31T15:10:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.656+0000 7f19f3fff640 1 -- 172.21.15.67:0/2292262406 learned_addr learned my addr 172.21.15.67:0/2292262406 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.656+0000 7f19f8e7f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19f410b970 0x7f19f41a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.656+0000 7f19f3fff640 1 -- 172.21.15.67:0/2292262406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19f410b970 msgr2=0x7f19f41a34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.656+0000 7f19f3fff640 1 --2- 172.21.15.67:0/2292262406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19f410b970 0x7f19f41a34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.656+0000 7f19f3fff640 1 -- 172.21.15.67:0/2292262406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f4114ea0 msgr2=0x7f19f41a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.657+0000 7f19f3fff640 1 --2- 172.21.15.67:0/2292262406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f4114ea0 0x7f19f41a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.657+0000 7f19f3fff640 1 -- 172.21.15.67:0/2292262406 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f41ae750 con 0x7f19f410c2c0 2024-07-31T15:10:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.657+0000 7f19f8e7f640 1 --2- 172.21.15.67:0/2292262406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19f410b970 0x7f19f41a34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.657+0000 7f19f9680640 1 --2- 172.21.15.67:0/2292262406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f4114ea0 0x7f19f41a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.657+0000 7f19f3fff640 1 --2- 172.21.15.67:0/2292262406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19f410c2c0 0x7f19f41a3a00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f19ec02acb0 tx=0x7f19ec003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.664+0000 7f19f1ffb640 1 -- 172.21.15.67:0/2292262406 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec040020 con 0x7f19f410c2c0 2024-07-31T15:10:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.664+0000 7f19f1ffb640 1 -- 172.21.15.67:0/2292262406 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19ec003e40 con 0x7f19f410c2c0 2024-07-31T15:10:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.664+0000 7f19f1ffb640 1 -- 172.21.15.67:0/2292262406 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec036040 con 0x7f19f410c2c0 2024-07-31T15:10:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.664+0000 7f19fb0e3640 1 -- 172.21.15.67:0/2292262406 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f41a12b0 con 0x7f19f410c2c0 2024-07-31T15:10:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.664+0000 7f19fb0e3640 1 -- 172.21.15.67:0/2292262406 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f41a1750 con 0x7f19f410c2c0 2024-07-31T15:10:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.668+0000 7f19f1ffb640 1 -- 172.21.15.67:0/2292262406 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f19ec043020 con 0x7f19f410c2c0 2024-07-31T15:10:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.669+0000 7f19f1ffb640 1 --2- 172.21.15.67:0/2292262406 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19dc0b3ed0 0x7f19dc0b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.670+0000 7f19f1ffb640 1 -- 172.21.15.67:0/2292262406 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f19ec105c50 con 0x7f19f410c2c0 2024-07-31T15:10:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.670+0000 7f19f8e7f640 1 --2- 172.21.15.67:0/2292262406 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19dc0b3ed0 0x7f19dc0b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.681+0000 7f19f8e7f640 1 --2- 172.21.15.67:0/2292262406 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19dc0b3ed0 0x7f19dc0b63e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f19f410b1e0 tx=0x7f19e401a040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.681+0000 7f1a01fd4740 1 Processor -- start 2024-07-31T15:10:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.681+0000 7f1a01fd4740 1 -- start start 2024-07-31T15:10:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.681+0000 7f1a01fd4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d2a1b0 0x5581290ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.681+0000 7f1a01fd4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d012f0 0x5581290cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.681+0000 7f1a01fd4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.681+0000 7f1a01fd4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5581290423b0 con 0x558128d094d0 2024-07-31T15:10:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.681+0000 7f1a01fd4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5581290318b0 con 0x558128d2a1b0 2024-07-31T15:10:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.682+0000 7f1a01fd4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5581290358f0 con 0x558128d012f0 2024-07-31T15:10:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.682+0000 7f19f3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d012f0 0x5581290cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.682+0000 7f19f3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d012f0 0x5581290cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:36300/0 (socket says 172.21.15.67:36300) 2024-07-31T15:10:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.682+0000 7f19f3fff640 1 -- 172.21.15.67:0/2102129107 learned_addr learned my addr 172.21.15.67:0/2102129107 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.682+0000 7f19f9680640 1 --2- 172.21.15.67:0/2102129107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.682+0000 7f19f8e7f640 1 --2- 172.21.15.67:0/2102129107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d2a1b0 0x5581290ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.682+0000 7f19f8e7f640 1 -- 172.21.15.67:0/2102129107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d012f0 msgr2=0x5581290cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.682+0000 7f19f8e7f640 1 --2- 172.21.15.67:0/2102129107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d012f0 0x5581290cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.682+0000 7f19f8e7f640 1 -- 172.21.15.67:0/2102129107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 msgr2=0x5581290cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.682+0000 7f19f8e7f640 1 --2- 172.21.15.67:0/2102129107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.682+0000 7f19f8e7f640 1 -- 172.21.15.67:0/2102129107 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558128fd89a0 con 0x558128d2a1b0 2024-07-31T15:10:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.683+0000 7f19f8e7f640 1 --2- 172.21.15.67:0/2102129107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d2a1b0 0x5581290ccce0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f19e400fc20 tx=0x7f19e401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f163214600b239f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.692+0000 7f19e2ffd640 1 -- 172.21.15.67:0/2102129107 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e4011cc0 con 0x558128d2a1b0 2024-07-31T15:10:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.692+0000 7f19e2ffd640 1 -- 172.21.15.67:0/2102129107 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19e40054d0 con 0x558128d2a1b0 2024-07-31T15:10:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.692+0000 7f19e2ffd640 1 -- 172.21.15.67:0/2102129107 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e4004d10 con 0x558128d2a1b0 2024-07-31T15:10:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.693+0000 7f1a01fd4740 1 -- 172.21.15.67:0/2102129107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d2a1b0 msgr2=0x5581290ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.693+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/2102129107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d2a1b0 0x5581290ccce0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f19e400fc20 tx=0x7f19e401c000 comp rx=0 tx=0).stop 2024-07-31T15:10:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.693+0000 7f1a01fd4740 1 -- 172.21.15.67:0/2102129107 shutdown_connections 2024-07-31T15:10:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.693+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/2102129107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.693+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/2102129107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d012f0 0x5581290cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.693+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/2102129107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d2a1b0 0x5581290ccce0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.693+0000 7f1a01fd4740 1 -- 172.21.15.67:0/2102129107 >> 172.21.15.67:0/2102129107 conn(0x558128d1e300 msgr2=0x5581290cabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.697+0000 7f1a01fd4740 1 -- 172.21.15.67:0/2102129107 shutdown_connections 2024-07-31T15:10:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.697+0000 7f1a01fd4740 1 -- 172.21.15.67:0/2102129107 wait complete. 2024-07-31T15:10:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.697+0000 7f1a01fd4740 1 -- start start 2024-07-31T15:10:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.697+0000 7f1a01fd4740 1 Processor -- start 2024-07-31T15:10:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.698+0000 7f1a01fd4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d012f0 0x5581290d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.698+0000 7f1a01fd4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d094d0 0x5581290d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.698+0000 7f1a01fd4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d2a1b0 0x5581290f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.698+0000 7f1a01fd4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558128f6bab0 con 0x558128d012f0 2024-07-31T15:10:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.698+0000 7f1a01fd4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558128fb9cf0 con 0x558128d2a1b0 2024-07-31T15:10:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.698+0000 7f1a01fd4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5581290ec470 con 0x558128d094d0 2024-07-31T15:10:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.698+0000 7f1a01fd4740 10 client.0 _ll_register_callbacks cb 0x55812902b9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.698+0000 7f19f8e7f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d012f0 0x5581290d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.698+0000 7f19f9680640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d2a1b0 0x5581290f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.699+0000 7f19f3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d094d0 0x5581290d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.699+0000 7f19f9680640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d2a1b0 0x5581290f57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52872/0 (socket says 172.21.15.67:52872) 2024-07-31T15:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.699+0000 7f19f9680640 1 -- 172.21.15.67:0/2490352320 learned_addr learned my addr 172.21.15.67:0/2490352320 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.699+0000 7f19f3fff640 1 -- 172.21.15.67:0/2490352320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d2a1b0 msgr2=0x5581290f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.699+0000 7f19f3fff640 1 --2- 172.21.15.67:0/2490352320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d2a1b0 0x5581290f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.699+0000 7f19f3fff640 1 -- 172.21.15.67:0/2490352320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d012f0 msgr2=0x5581290d87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.699+0000 7f19f3fff640 1 --2- 172.21.15.67:0/2490352320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d012f0 0x5581290d87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.699+0000 7f19f3fff640 1 -- 172.21.15.67:0/2490352320 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558128d95790 con 0x558128d094d0 2024-07-31T15:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.699+0000 7f19f8e7f640 1 --2- 172.21.15.67:0/2490352320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d012f0 0x5581290d87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:42.702 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.700+0000 7f19f3fff640 1 --2- 172.21.15.67:0/2490352320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d094d0 0x5581290d8d30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f19ec006010 tx=0x7f19ec0ccaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bbe80d6e1019e09 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:42.712 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.711+0000 7f19d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:10:42.712 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.711+0000 7f19d7fff640 1 -- 172.21.15.67:0/2490352320 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0ccc50 con 0x558128d094d0 2024-07-31T15:10:42.712 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.711+0000 7f1a01fd4740 1 -- 172.21.15.67:0/2490352320 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e40054d0 con 0x558128d094d0 2024-07-31T15:10:42.712 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.711+0000 7f19d7fff640 1 -- 172.21.15.67:0/2490352320 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f19ec031850 con 0x558128d094d0 2024-07-31T15:10:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.715+0000 7f19d7fff640 1 -- 172.21.15.67:0/2490352320 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f19ec0cc550 con 0x558128d094d0 2024-07-31T15:10:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.715+0000 7f1a01fd4740 10 client.29053 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.715+0000 7f1a01fd4740 1 -- 172.21.15.67:0/2490352320 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x558128e0e620 con 0x558128d094d0 2024-07-31T15:10:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.715+0000 7f19d7fff640 1 -- 172.21.15.67:0/2490352320 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19ec0ea080 con 0x558128d094d0 2024-07-31T15:10:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.715+0000 7f19d7fff640 1 -- 172.21.15.67:0/2490352320 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f19ec0c7ec0 con 0x558128d094d0 2024-07-31T15:10:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.715+0000 7f1a01fd4740 10 client.29053 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.715+0000 7f1a01fd4740 10 client.29053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.716+0000 7f1a01fd4740 20 client.29053 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.716+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/2490352320 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558128ce8d80 0x5581290fb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.716+0000 7f1a01fd4740 4 client.29053 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:10:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.716+0000 7f1a01fd4740 1 -- 172.21.15.67:0/2490352320 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558129044300 con 0x558128ce8d80 2024-07-31T15:10:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.716+0000 7f19f8e7f640 1 --2- 172.21.15.67:0/2490352320 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558128ce8d80 0x5581290fb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.717+0000 7f19d7fff640 10 client.29053 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.717+0000 7f19f8e7f640 1 --2- 172.21.15.67:0/2490352320 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558128ce8d80 0x5581290fb900 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be6bea7ac4318912 server_cookie=976305a36e8535f in_seq=0 out_seq=0 2024-07-31T15:10:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.717+0000 7f19d7fff640 1 -- 172.21.15.67:0/2490352320 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558129044300 con 0x558128ce8d80 2024-07-31T15:10:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.717+0000 7f19d7fff640 10 client.29053 handle_command_reply: tid=0 2024-07-31T15:10:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.718+0000 7f1a01fd4740 1 client.29053 shutdown 2024-07-31T15:10:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.718+0000 7f1a01fd4740 1 -- 172.21.15.67:0/2490352320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d094d0 msgr2=0x5581290d8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.718+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/2490352320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d094d0 0x5581290d8d30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f19ec006010 tx=0x7f19ec0ccaf0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.718+0000 7f1a01fd4740 1 -- 172.21.15.67:0/2490352320 shutdown_connections 2024-07-31T15:10:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.718+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/2490352320 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558128ce8d80 0x5581290fb900 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.718+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/2490352320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d2a1b0 0x5581290f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.718+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/2490352320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d094d0 0x5581290d8d30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.718+0000 7f19d7fff640 0 client.29053 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.718+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/2490352320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d012f0 0x5581290d87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.718+0000 7f1a01fd4740 1 -- 172.21.15.67:0/2490352320 >> 172.21.15.67:0/2490352320 conn(0x558128d1e300 msgr2=0x5581290cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.718+0000 7f1a01fd4740 1 -- 172.21.15.67:0/2490352320 shutdown_connections 2024-07-31T15:10:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.718+0000 7f1a01fd4740 1 -- 172.21.15.67:0/2490352320 wait complete. 2024-07-31T15:10:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.718+0000 7f1a01fd4740 20 client.29053 trim_cache size 0 max 16384 2024-07-31T15:10:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.757+0000 7f1a01fd4740 1 Processor -- start 2024-07-31T15:10:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.758+0000 7f1a01fd4740 1 -- start start 2024-07-31T15:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.758+0000 7f1a01fd4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d012f0 0x5581290d81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.758+0000 7f1a01fd4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.759+0000 7f19f8e7f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d012f0 0x5581290d81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.759+0000 7f19f8e7f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d012f0 0x5581290d81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52882/0 (socket says 172.21.15.67:52882) 2024-07-31T15:10:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.759+0000 7f19f3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.759+0000 7f19f3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290f55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47668/0 (socket says 172.21.15.67:47668) 2024-07-31T15:10:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.759+0000 7f19f8e7f640 1 -- 172.21.15.67:0/3385927318 learned_addr learned my addr 172.21.15.67:0/3385927318 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.759+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/3385927318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d2a1b0 0x5581290f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.759+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3385927318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f19d800fdf0 con 0x558128d094d0 2024-07-31T15:10:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.759+0000 7f19f9680640 1 --2- 172.21.15.67:0/3385927318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d2a1b0 0x5581290f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.759+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3385927318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f19d8011930 con 0x558128d012f0 2024-07-31T15:10:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.759+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3385927318 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19d80104c0 con 0x558128d2a1b0 2024-07-31T15:10:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.760+0000 7f19f9680640 1 -- 172.21.15.67:0/3385927318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d012f0 msgr2=0x5581290d81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.760+0000 7f19f9680640 1 --2- 172.21.15.67:0/3385927318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d012f0 0x5581290d81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.760+0000 7f19f9680640 1 -- 172.21.15.67:0/3385927318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 msgr2=0x5581290f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.760+0000 7f19f9680640 1 --2- 172.21.15.67:0/3385927318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290f55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.760+0000 7f19f9680640 1 -- 172.21.15.67:0/3385927318 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581290d6ac0 con 0x558128d2a1b0 2024-07-31T15:10:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.760+0000 7f19f3fff640 1 --2- 172.21.15.67:0/3385927318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.761+0000 7f19f9680640 1 --2- 172.21.15.67:0/3385927318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d2a1b0 0x5581290f5af0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f19e8012770 tx=0x7f19e8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd6d303ae147bd4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.773+0000 7f19e27fc640 1 -- 172.21.15.67:0/3385927318 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8022070 con 0x558128d2a1b0 2024-07-31T15:10:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.773+0000 7f19e27fc640 1 -- 172.21.15.67:0/3385927318 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19e800f920 con 0x558128d2a1b0 2024-07-31T15:10:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.774+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3385927318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d2a1b0 msgr2=0x5581290f5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.774+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/3385927318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d2a1b0 0x5581290f5af0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f19e8012770 tx=0x7f19e8009260 comp rx=0 tx=0).stop 2024-07-31T15:10:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.774+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3385927318 shutdown_connections 2024-07-31T15:10:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.774+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/3385927318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d2a1b0 0x5581290f5af0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.774+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/3385927318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.774+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/3385927318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d012f0 0x5581290d81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.774+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3385927318 >> 172.21.15.67:0/3385927318 conn(0x558128d1e300 msgr2=0x5581290d7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.774+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3385927318 shutdown_connections 2024-07-31T15:10:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.774+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3385927318 wait complete. 2024-07-31T15:10:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.775+0000 7f1a01fd4740 1 -- start start 2024-07-31T15:10:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.775+0000 7f1a01fd4740 1 Processor -- start 2024-07-31T15:10:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.775+0000 7f1a01fd4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d012f0 0x5581290f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.775+0000 7f1a01fd4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290f0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.775+0000 7f1a01fd4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d2a1b0 0x5581290f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.775+0000 7f1a01fd4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f19d8011260 con 0x558128d094d0 2024-07-31T15:10:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.775+0000 7f1a01fd4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f19d8010b90 con 0x558128d012f0 2024-07-31T15:10:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.775+0000 7f1a01fd4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19d800ddd0 con 0x558128d2a1b0 2024-07-31T15:10:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.776+0000 7f1a01fd4740 10 client.0 _ll_register_callbacks cb 0x7f19ec0cc410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.776+0000 7f19f3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.776+0000 7f19f8e7f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d012f0 0x5581290f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.776+0000 7f19f3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290f0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47676/0 (socket says 172.21.15.67:47676) 2024-07-31T15:10:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.776+0000 7f19f3fff640 1 -- 172.21.15.67:0/3152248766 learned_addr learned my addr 172.21.15.67:0/3152248766 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.776+0000 7f19f9680640 1 --2- 172.21.15.67:0/3152248766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d2a1b0 0x5581290f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.776+0000 7f19f8e7f640 1 -- 172.21.15.67:0/3152248766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d2a1b0 msgr2=0x5581290f0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.776+0000 7f19f8e7f640 1 --2- 172.21.15.67:0/3152248766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d2a1b0 0x5581290f0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.776+0000 7f19f8e7f640 1 -- 172.21.15.67:0/3152248766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 msgr2=0x5581290f0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.776+0000 7f19f8e7f640 1 --2- 172.21.15.67:0/3152248766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290f0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.776+0000 7f19f8e7f640 1 -- 172.21.15.67:0/3152248766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581290f1330 con 0x558128d012f0 2024-07-31T15:10:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.776+0000 7f19f9680640 1 --2- 172.21.15.67:0/3152248766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d2a1b0 0x5581290f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:10:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.776+0000 7f19f3fff640 1 --2- 172.21.15.67:0/3152248766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:10:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.777+0000 7f19f8e7f640 1 --2- 172.21.15.67:0/3152248766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d012f0 0x5581290f1d30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f19e4011a10 tx=0x7f19e4005680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e832f8f2b423cde9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.788+0000 7f19d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:10:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.788+0000 7f19d7fff640 1 -- 172.21.15.67:0/3152248766 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e4004ab0 con 0x558128d012f0 2024-07-31T15:10:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.788+0000 7f19d7fff640 1 -- 172.21.15.67:0/3152248766 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f19e4027040 con 0x558128d012f0 2024-07-31T15:10:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.788+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3152248766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e800f920 con 0x558128d012f0 2024-07-31T15:10:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.788+0000 7f19d7fff640 1 -- 172.21.15.67:0/3152248766 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f19e4004fe0 con 0x558128d012f0 2024-07-31T15:10:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.788+0000 7f1a01fd4740 10 client.29186 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.788+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3152248766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x558129162f50 con 0x558128d012f0 2024-07-31T15:10:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.789+0000 7f19d7fff640 1 -- 172.21.15.67:0/3152248766 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19e800f920 con 0x558128d012f0 2024-07-31T15:10:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.790+0000 7f19d7fff640 1 -- 172.21.15.67:0/3152248766 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f19e4021070 con 0x558128d012f0 2024-07-31T15:10:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.790+0000 7f1a01fd4740 10 client.29186 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.790+0000 7f1a01fd4740 10 client.29186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.790+0000 7f1a01fd4740 20 client.29186 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.790+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/3152248766 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558128cbe1c0 0x5581291654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.790+0000 7f1a01fd4740 4 client.29186 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:10:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.790+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3152248766 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55812903fda0 con 0x558128cbe1c0 2024-07-31T15:10:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.791+0000 7f19f3fff640 1 --2- 172.21.15.67:0/3152248766 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558128cbe1c0 0x5581291654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.791+0000 7f19d7fff640 10 client.29186 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:42.791+0000 7f19f3fff640 1 --2- 172.21.15.67:0/3152248766 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558128cbe1c0 0x5581291654d0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1d8d47fbd6d5e8f server_cookie=358382b8c3e92eab in_seq=0 out_seq=0 2024-07-31T15:10:42.861 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/xfrm6_policy.o 2024-07-31T15:10:43.007 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/stats.o 2024-07-31T15:10:43.184 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/ip_tunnel.o 2024-07-31T15:10:43.229 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/ibss.o 2024-07-31T15:10:43.276 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/sysctl_net_ipv4.o 2024-07-31T15:10:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:43 smithi067 ceph-mon[32750]: pgmap v1027: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 492 KiB/s wr, 61 op/s 2024-07-31T15:10:43.463 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_renderstate.o 2024-07-31T15:10:43.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:43 smithi073 ceph-mon[36714]: pgmap v1027: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 492 KiB/s wr, 61 op/s 2024-07-31T15:10:43.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:43 smithi077 ceph-mon[37327]: pgmap v1027: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 492 KiB/s wr, 61 op/s 2024-07-31T15:10:43.768 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netlabel/netlabel_cipso_v4.o 2024-07-31T15:10:43.892 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/xfrm6_state.o 2024-07-31T15:10:43.971 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/ata_piix.o 2024-07-31T15:10:44.674 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sunrpc/sysctl.o 2024-07-31T15:10:44.725 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_reset.o 2024-07-31T15:10:44.741 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/proc.o 2024-07-31T15:10:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.927+0000 7f19d7fff640 1 -- 172.21.15.67:0/3152248766 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55812903fda0 con 0x558128cbe1c0 2024-07-31T15:10:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.927+0000 7f19d7fff640 10 client.29186 handle_command_reply: tid=0 2024-07-31T15:10:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.927+0000 7f1a01fd4740 1 client.29186 shutdown 2024-07-31T15:10:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.927+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3152248766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d012f0 msgr2=0x5581290f1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.927+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/3152248766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d012f0 0x5581290f1d30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f19e4011a10 tx=0x7f19e4005680 comp rx=0 tx=0).stop 2024-07-31T15:10:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.927+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3152248766 shutdown_connections 2024-07-31T15:10:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.927+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/3152248766 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558128cbe1c0 0x5581291654d0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.927+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/3152248766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558128d2a1b0 0x5581290f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.927+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/3152248766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558128d094d0 0x5581290f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.927+0000 7f1a01fd4740 1 --2- 172.21.15.67:0/3152248766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558128d012f0 0x5581290f1d30 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.927+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3152248766 >> 172.21.15.67:0/3152248766 conn(0x558128d1e300 msgr2=0x5581290d98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.928+0000 7f19d7fff640 0 client.29186 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.928+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3152248766 shutdown_connections 2024-07-31T15:10:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.928+0000 7f1a01fd4740 1 -- 172.21.15.67:0/3152248766 wait complete. 2024-07-31T15:10:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.928+0000 7f1a01fd4740 20 client.29186 trim_cache size 0 max 16384 2024-07-31T15:10:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:10:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:10:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:10:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:10:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.954+0000 7f19fb0e3640 1 -- 172.21.15.67:0/2292262406 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19dc0b3ed0 msgr2=0x7f19dc0b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.954+0000 7f19fb0e3640 1 --2- 172.21.15.67:0/2292262406 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19dc0b3ed0 0x7f19dc0b63e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f19f410b1e0 tx=0x7f19e401a040 comp rx=0 tx=0).stop 2024-07-31T15:10:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.954+0000 7f19fb0e3640 1 -- 172.21.15.67:0/2292262406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19f410c2c0 msgr2=0x7f19f41a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.954+0000 7f19fb0e3640 1 --2- 172.21.15.67:0/2292262406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19f410c2c0 0x7f19f41a3a00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f19ec02acb0 tx=0x7f19ec003c10 comp rx=0 tx=0).stop 2024-07-31T15:10:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.954+0000 7f19fb0e3640 1 -- 172.21.15.67:0/2292262406 shutdown_connections 2024-07-31T15:10:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.954+0000 7f19fb0e3640 1 --2- 172.21.15.67:0/2292262406 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f19dc0b3ed0 0x7f19dc0b63e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.954+0000 7f19fb0e3640 1 --2- 172.21.15.67:0/2292262406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19f4114ea0 0x7f19f41a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.954+0000 7f19fb0e3640 1 --2- 172.21.15.67:0/2292262406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19f410c2c0 0x7f19f41a3a00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.954+0000 7f19fb0e3640 1 --2- 172.21.15.67:0/2292262406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19f410b970 0x7f19f41a34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.954+0000 7f19fb0e3640 1 -- 172.21.15.67:0/2292262406 >> 172.21.15.67:0/2292262406 conn(0x7f19f40707c0 msgr2=0x7f19f4072a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.954+0000 7f19fb0e3640 1 -- 172.21.15.67:0/2292262406 shutdown_connections 2024-07-31T15:10:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:44.954+0000 7f19fb0e3640 1 -- 172.21.15.67:0/2292262406 wait complete. 2024-07-31T15:10:44.998 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/xfrm6_input.o 2024-07-31T15:10:45.083 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/netlabel/netlabel_calipso.o 2024-07-31T15:10:45.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.094+0000 7f3eb56b5640 1 -- 172.21.15.67:0/1880253675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3eb010b880 msgr2=0x7f3eb01168a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:45.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.094+0000 7f3eb56b5640 1 --2- 172.21.15.67:0/1880253675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3eb010b880 0x7f3eb01168a0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009be0 tx=0x7f3ea802a6e0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.095+0000 7f3eb56b5640 1 -- 172.21.15.67:0/1880253675 shutdown_connections 2024-07-31T15:10:45.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.095+0000 7f3eb56b5640 1 --2- 172.21.15.67:0/1880253675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3eb010b880 0x7f3eb01168a0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.095+0000 7f3eb56b5640 1 --2- 172.21.15.67:0/1880253675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3eb010af30 0x7f3eb010b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.095+0000 7f3eb56b5640 1 --2- 172.21.15.67:0/1880253675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb01055f0 0x7f3eb0105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.095+0000 7f3eb56b5640 1 -- 172.21.15.67:0/1880253675 >> 172.21.15.67:0/1880253675 conn(0x7f3eb0101060 msgr2=0x7f3eb01034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:45.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.098+0000 7f3eb56b5640 1 -- 172.21.15.67:0/1880253675 shutdown_connections 2024-07-31T15:10:45.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.100+0000 7f3eb56b5640 1 -- 172.21.15.67:0/1880253675 wait complete. 2024-07-31T15:10:45.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.101+0000 7f3eb56b5640 1 Processor -- start 2024-07-31T15:10:45.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.102+0000 7f3eb56b5640 1 -- start start 2024-07-31T15:10:45.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.102+0000 7f3eb56b5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb01055f0 0x7f3eb0116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.102+0000 7f3eb56b5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3eb010af30 0x7f3eb010f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.102+0000 7f3eb56b5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3eb010b880 0x7f3eb010f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.102+0000 7f3eb56b5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb0119960 con 0x7f3eb010af30 2024-07-31T15:10:45.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.102+0000 7f3eb56b5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb01197e0 con 0x7f3eb010b880 2024-07-31T15:10:45.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.102+0000 7f3eb56b5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb0119ae0 con 0x7f3eb01055f0 2024-07-31T15:10:45.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.105+0000 7f3eaffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb01055f0 0x7f3eb0116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.109+0000 7f3eaffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb01055f0 0x7f3eb0116130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:36340/0 (socket says 172.21.15.67:36340) 2024-07-31T15:10:45.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.109+0000 7f3eaffff640 1 -- 172.21.15.67:0/2302789898 learned_addr learned my addr 172.21.15.67:0/2302789898 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:45.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.109+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3eb010b880 0x7f3eb010f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.116+0000 7f3eaf7fe640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3eb010af30 0x7f3eb010f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.116+0000 7f3eaffff640 1 -- 172.21.15.67:0/2302789898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3eb010b880 msgr2=0x7f3eb010f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:45.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.116+0000 7f3eaffff640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3eb010b880 0x7f3eb010f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.116+0000 7f3eaffff640 1 -- 172.21.15.67:0/2302789898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3eb010af30 msgr2=0x7f3eb010f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:45.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.116+0000 7f3eaffff640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3eb010af30 0x7f3eb010f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.116+0000 7f3eaffff640 1 -- 172.21.15.67:0/2302789898 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb01101c0 con 0x7f3eb01055f0 2024-07-31T15:10:45.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.116+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3eb010b880 0x7f3eb010f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:10:45.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.121+0000 7f3eaffff640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb01055f0 0x7f3eb0116130 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400b450 tx=0x7f3ea400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:45.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.121+0000 7f3eaf7fe640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3eb010af30 0x7f3eb010f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:10:45.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.124+0000 7f3ead7fa640 1 -- 172.21.15.67:0/2302789898 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4015020 con 0x7f3eb01055f0 2024-07-31T15:10:45.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.124+0000 7f3eb56b5640 1 -- 172.21.15.67:0/2302789898 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb0114e60 con 0x7f3eb01055f0 2024-07-31T15:10:45.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.124+0000 7f3eb56b5640 1 -- 172.21.15.67:0/2302789898 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb01153c0 con 0x7f3eb01055f0 2024-07-31T15:10:45.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.125+0000 7f3ead7fa640 1 -- 172.21.15.67:0/2302789898 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea4011d40 con 0x7f3eb01055f0 2024-07-31T15:10:45.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.127+0000 7f3ead7fa640 1 -- 172.21.15.67:0/2302789898 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea400f340 con 0x7f3eb01055f0 2024-07-31T15:10:45.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.128+0000 7f3ebce68740 1 Processor -- start 2024-07-31T15:10:45.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.132+0000 7f3ebce68740 1 -- start start 2024-07-31T15:10:45.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.132+0000 7f3ebce68740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 0x5574c89e7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.132+0000 7f3ebce68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c86191b0 0x5574c89e7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.132+0000 7f3ebce68740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c8621390 0x5574c89e8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.132+0000 7f3ebce68740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5574c895dbb0 con 0x5574c8642070 2024-07-31T15:10:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.132+0000 7f3ebce68740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5574c894d0b0 con 0x5574c8621390 2024-07-31T15:10:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.132+0000 7f3ebce68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5574c89510f0 con 0x5574c86191b0 2024-07-31T15:10:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.133+0000 7f3eb4eb4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 0x5574c89e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.133+0000 7f3eb4eb4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 0x5574c89e7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47714/0 (socket says 172.21.15.67:47714) 2024-07-31T15:10:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.133+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/940059953 learned_addr learned my addr 172.21.15.67:0/940059953 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.133+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/940059953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c86191b0 msgr2=0x5574c89e7c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:10:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.133+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/940059953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c86191b0 0x5574c89e7c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.133+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/940059953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c8621390 msgr2=0x5574c89e8190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:10:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.133+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/940059953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c8621390 0x5574c89e8190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.134+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/940059953 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c88f3b90 con 0x5574c8642070 2024-07-31T15:10:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.134+0000 7f3ead7fa640 1 -- 172.21.15.67:0/2302789898 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3ea4018020 con 0x7f3eb01055f0 2024-07-31T15:10:45.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.135+0000 7f3ead7fa640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e980b3e20 0x7f3e980b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.135+0000 7f3eaffff640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e980b3e20 0x7f3e980b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.136+0000 7f3ead7fa640 1 -- 172.21.15.67:0/2302789898 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f3ea40e3c90 con 0x7f3eb01055f0 2024-07-31T15:10:45.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.136+0000 7f3eaffff640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e980b3e20 0x7f3e980b6330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x5574c89e6390 tx=0x7f3ea40ae040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:45.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.138+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/940059953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 0x5574c89e7710 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009b30 tx=0x7f3ea80305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5d393fdb3fa37b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:45.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.140+0000 7f3e967fc640 1 -- 172.21.15.67:0/940059953 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8041020 con 0x5574c8642070 2024-07-31T15:10:45.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.140+0000 7f3e967fc640 1 -- 172.21.15.67:0/940059953 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea8037040 con 0x5574c8642070 2024-07-31T15:10:45.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.140+0000 7f3e967fc640 1 -- 172.21.15.67:0/940059953 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8036070 con 0x5574c8642070 2024-07-31T15:10:45.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.141+0000 7f3ebce68740 1 -- 172.21.15.67:0/940059953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 msgr2=0x5574c89e7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:45.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.141+0000 7f3ebce68740 1 --2- 172.21.15.67:0/940059953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 0x5574c89e7710 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009b30 tx=0x7f3ea80305c0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.145+0000 7f3ebce68740 1 -- 172.21.15.67:0/940059953 shutdown_connections 2024-07-31T15:10:45.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.145+0000 7f3ebce68740 1 --2- 172.21.15.67:0/940059953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c8621390 0x5574c89e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.145+0000 7f3ebce68740 1 --2- 172.21.15.67:0/940059953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c86191b0 0x5574c89e7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.145+0000 7f3ebce68740 1 --2- 172.21.15.67:0/940059953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 0x5574c89e7710 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.145+0000 7f3ebce68740 1 -- 172.21.15.67:0/940059953 >> 172.21.15.67:0/940059953 conn(0x5574c8636460 msgr2=0x5574c89e5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:45.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.148+0000 7f3ebce68740 1 -- 172.21.15.67:0/940059953 shutdown_connections 2024-07-31T15:10:45.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.152+0000 7f3ebce68740 1 -- 172.21.15.67:0/940059953 wait complete. 2024-07-31T15:10:45.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.153+0000 7f3ebce68740 1 -- start start 2024-07-31T15:10:45.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.153+0000 7f3ebce68740 1 Processor -- start 2024-07-31T15:10:45.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.154+0000 7f3ebce68740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c86191b0 0x5574c89f3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.154+0000 7f3ebce68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 0x5574c89f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.154+0000 7f3ebce68740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 0x5574c89ff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.154+0000 7f3ebce68740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5574c87b0700 con 0x5574c8642070 2024-07-31T15:10:45.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.154+0000 7f3ebce68740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5574c88ef9e0 con 0x5574c86191b0 2024-07-31T15:10:45.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.154+0000 7f3ebce68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5574c89f5ef0 con 0x5574c8621390 2024-07-31T15:10:45.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.154+0000 7f3ebce68740 10 client.0 _ll_register_callbacks cb 0x5574c890a4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:45.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.155+0000 7f3eb4eb4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c86191b0 0x5574c89f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.155+0000 7f3eb4eb4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c86191b0 0x5574c89f3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52918/0 (socket says 172.21.15.67:52918) 2024-07-31T15:10:45.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.155+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/2342342625 learned_addr learned my addr 172.21.15.67:0/2342342625 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:45.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.155+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/2342342625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 0x5574c89ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.155+0000 7f3eaf7fe640 1 --2- 172.21.15.67:0/2342342625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 0x5574c89f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.156+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/2342342625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 msgr2=0x5574c89f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:45.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.156+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/2342342625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 0x5574c89f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.156+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/2342342625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 msgr2=0x5574c89ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:45.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.156+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/2342342625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 0x5574c89ff270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.156+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/2342342625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c86ad650 con 0x5574c86191b0 2024-07-31T15:10:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.158+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/2342342625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c86191b0 0x5574c89f3370 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80099a0 tx=0x7f3ea8033040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c4b8662939b495c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.158+0000 7f3e8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:10:45.160 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/xfrm6_output.o 2024-07-31T15:10:45.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.158+0000 7f3e8f7fe640 1 -- 172.21.15.67:0/2342342625 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8041030 con 0x5574c86191b0 2024-07-31T15:10:45.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.158+0000 7f3e8f7fe640 1 -- 172.21.15.67:0/2342342625 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f3ea8003c30 con 0x5574c86191b0 2024-07-31T15:10:45.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.159+0000 7f3ebce68740 1 -- 172.21.15.67:0/2342342625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea8037040 con 0x5574c86191b0 2024-07-31T15:10:45.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.159+0000 7f3e8f7fe640 1 -- 172.21.15.67:0/2342342625 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f3ea8055050 con 0x5574c86191b0 2024-07-31T15:10:45.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.159+0000 7f3ebce68740 10 client.29192 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:45.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.159+0000 7f3ebce68740 1 -- 172.21.15.67:0/2342342625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5574c8a01fe0 con 0x5574c86191b0 2024-07-31T15:10:45.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.160+0000 7f3e8f7fe640 1 -- 172.21.15.67:0/2342342625 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea8037040 con 0x5574c86191b0 2024-07-31T15:10:45.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.161+0000 7f3e8f7fe640 1 -- 172.21.15.67:0/2342342625 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f3ea803d780 con 0x5574c86191b0 2024-07-31T15:10:45.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.161+0000 7f3ebce68740 10 client.29192 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:45.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.161+0000 7f3ebce68740 10 client.29192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:45.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.161+0000 7f3ebce68740 20 client.29192 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:45.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.162+0000 7f3ebce68740 1 --2- 172.21.15.67:0/2342342625 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574c86009f0 0x5574c8a05340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.162+0000 7f3ebce68740 4 client.29192 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:10:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.162+0000 7f3ebce68740 1 -- 172.21.15.67:0/2342342625 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574c895fb00 con 0x5574c86009f0 2024-07-31T15:10:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.162+0000 7f3eaf7fe640 1 --2- 172.21.15.67:0/2342342625 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574c86009f0 0x5574c8a05340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.168+0000 7f3eaf7fe640 1 --2- 172.21.15.67:0/2342342625 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574c86009f0 0x5574c8a05340 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c91dd8946877325 server_cookie=93b433f968b215ac in_seq=0 out_seq=0 2024-07-31T15:10:45.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.168+0000 7f3e8f7fe640 10 client.29192 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.169+0000 7f3e8f7fe640 1 -- 172.21.15.67:0/2342342625 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574c895fb00 con 0x5574c86009f0 2024-07-31T15:10:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.169+0000 7f3e8f7fe640 10 client.29192 handle_command_reply: tid=0 2024-07-31T15:10:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.169+0000 7f3ebce68740 1 client.29192 shutdown 2024-07-31T15:10:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.170+0000 7f3ebce68740 1 -- 172.21.15.67:0/2342342625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c86191b0 msgr2=0x5574c89f3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.170+0000 7f3ebce68740 1 --2- 172.21.15.67:0/2342342625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c86191b0 0x5574c89f3370 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80099a0 tx=0x7f3ea8033040 comp rx=0 tx=0).stop 2024-07-31T15:10:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.170+0000 7f3ebce68740 1 -- 172.21.15.67:0/2342342625 shutdown_connections 2024-07-31T15:10:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.170+0000 7f3ebce68740 1 --2- 172.21.15.67:0/2342342625 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574c86009f0 0x5574c8a05340 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.170+0000 7f3ebce68740 1 --2- 172.21.15.67:0/2342342625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 0x5574c89ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.170+0000 7f3ebce68740 1 --2- 172.21.15.67:0/2342342625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 0x5574c89f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.170+0000 7f3ebce68740 1 --2- 172.21.15.67:0/2342342625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c86191b0 0x5574c89f3370 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.170+0000 7f3ebce68740 1 -- 172.21.15.67:0/2342342625 >> 172.21.15.67:0/2342342625 conn(0x5574c8636460 msgr2=0x5574c89ea8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.173+0000 7f3e8f7fe640 0 client.29192 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.174+0000 7f3ebce68740 1 -- 172.21.15.67:0/2342342625 shutdown_connections 2024-07-31T15:10:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.175+0000 7f3ebce68740 1 -- 172.21.15.67:0/2342342625 wait complete. 2024-07-31T15:10:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.175+0000 7f3ebce68740 20 client.29192 trim_cache size 0 max 16384 2024-07-31T15:10:45.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.213+0000 7f3ebce68740 1 Processor -- start 2024-07-31T15:10:45.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.217+0000 7f3ebce68740 1 -- start start 2024-07-31T15:10:45.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.217+0000 7f3ebce68740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c86191b0 0x5574c89e5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.217+0000 7f3ebce68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 0x5574c89ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.217+0000 7f3ebce68740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c8642070 0x5574c89ec820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.217+0000 7f3ebce68740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c00fe20 con 0x5574c86191b0 2024-07-31T15:10:45.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.217+0000 7f3ebce68740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c011960 con 0x5574c8642070 2024-07-31T15:10:45.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.217+0000 7f3ebce68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c0104f0 con 0x5574c8621390 2024-07-31T15:10:45.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.218+0000 7f3eb4eb4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c86191b0 0x5574c89e5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.218+0000 7f3eb4eb4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c86191b0 0x5574c89e5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47742/0 (socket says 172.21.15.67:47742) 2024-07-31T15:10:45.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.218+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/2884229996 learned_addr learned my addr 172.21.15.67:0/2884229996 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:45.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.218+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/2884229996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c8642070 0x5574c89ec820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.219+0000 7f3eaf7fe640 1 --2- 172.21.15.67:0/2884229996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 0x5574c89ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.222+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/2884229996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 msgr2=0x5574c89ff1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:45.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.222+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/2884229996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 0x5574c89ff1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.222+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/2884229996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c86191b0 msgr2=0x5574c89e5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:45.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.222+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/2884229996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c86191b0 0x5574c89e5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.222+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/2884229996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c89f4430 con 0x5574c8642070 2024-07-31T15:10:45.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.226+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/2884229996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c8642070 0x5574c89ec820 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8030500 tx=0x7f3ea8030e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a538329e04283256 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:45.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.232+0000 7f3e95ffb640 1 -- 172.21.15.67:0/2884229996 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8041030 con 0x5574c8642070 2024-07-31T15:10:45.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.232+0000 7f3e95ffb640 1 -- 172.21.15.67:0/2884229996 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea803d930 con 0x5574c8642070 2024-07-31T15:10:45.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.232+0000 7f3e95ffb640 1 -- 172.21.15.67:0/2884229996 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8036070 con 0x5574c8642070 2024-07-31T15:10:45.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.234+0000 7f3ebce68740 1 -- 172.21.15.67:0/2884229996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c8642070 msgr2=0x5574c89ec820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:45.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.234+0000 7f3ebce68740 1 --2- 172.21.15.67:0/2884229996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c8642070 0x5574c89ec820 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8030500 tx=0x7f3ea8030e20 comp rx=0 tx=0).stop 2024-07-31T15:10:45.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.242+0000 7f3ebce68740 1 -- 172.21.15.67:0/2884229996 shutdown_connections 2024-07-31T15:10:45.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.242+0000 7f3ebce68740 1 --2- 172.21.15.67:0/2884229996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c8642070 0x5574c89ec820 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.242+0000 7f3ebce68740 1 --2- 172.21.15.67:0/2884229996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 0x5574c89ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.242+0000 7f3ebce68740 1 --2- 172.21.15.67:0/2884229996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c86191b0 0x5574c89e5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.242+0000 7f3ebce68740 1 -- 172.21.15.67:0/2884229996 >> 172.21.15.67:0/2884229996 conn(0x5574c8636460 msgr2=0x5574c89e4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:45.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.250+0000 7f3ebce68740 1 -- 172.21.15.67:0/2884229996 shutdown_connections 2024-07-31T15:10:45.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.254+0000 7f3ebce68740 1 -- 172.21.15.67:0/2884229996 wait complete. 2024-07-31T15:10:45.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.254+0000 7f3ebce68740 1 -- start start 2024-07-31T15:10:45.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.255+0000 7f3ebce68740 1 Processor -- start 2024-07-31T15:10:45.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.256+0000 7f3ebce68740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c86191b0 0x5574c8a087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.256+0000 7f3ebce68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 0x5574c8a05ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.256+0000 7f3ebce68740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 0x5574c8a06220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.256+0000 7f3ebce68740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c011290 con 0x5574c8642070 2024-07-31T15:10:45.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.256+0000 7f3ebce68740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c010bc0 con 0x5574c86191b0 2024-07-31T15:10:45.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.257+0000 7f3ebce68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c00de00 con 0x5574c8621390 2024-07-31T15:10:45.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.257+0000 7f3eb4eb4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c86191b0 0x5574c8a087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.257+0000 7f3eb4eb4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c86191b0 0x5574c8a087c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52936/0 (socket says 172.21.15.67:52936) 2024-07-31T15:10:45.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.257+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/3937808316 learned_addr learned my addr 172.21.15.67:0/3937808316 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:45.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.257+0000 7f3ebce68740 10 client.0 _ll_register_callbacks cb 0x7f3ea0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:45.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.257+0000 7f3eaf7fe640 1 --2- 172.21.15.67:0/3937808316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 0x5574c8a05ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.257+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/3937808316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 msgr2=0x5574c8a05ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:45.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.257+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/3937808316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 0x5574c8a05ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.257+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/3937808316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 msgr2=0x5574c8a06220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:45.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.257+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/3937808316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 0x5574c8a06220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.257+0000 7f3eb4eb4640 1 -- 172.21.15.67:0/3937808316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c8a06760 con 0x5574c86191b0 2024-07-31T15:10:45.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.261+0000 7f3eb4eb4640 1 --2- 172.21.15.67:0/3937808316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c86191b0 0x5574c8a087c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80099a0 tx=0x7f3ea8030fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bafea6a65ce62ca1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:45.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.263+0000 7f3e8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:10:45.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.267+0000 7f3e8f7fe640 1 -- 172.21.15.67:0/3937808316 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8036040 con 0x5574c86191b0 2024-07-31T15:10:45.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.267+0000 7f3e8f7fe640 1 -- 172.21.15.67:0/3937808316 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f3ea8031730 con 0x5574c86191b0 2024-07-31T15:10:45.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.267+0000 7f3ebce68740 1 -- 172.21.15.67:0/3937808316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea803d930 con 0x5574c86191b0 2024-07-31T15:10:45.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.274+0000 7f3e8f7fe640 1 -- 172.21.15.67:0/3937808316 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f3ea8031b40 con 0x5574c86191b0 2024-07-31T15:10:45.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.275+0000 7f3ebce68740 10 client.29198 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:45.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.275+0000 7f3ebce68740 1 -- 172.21.15.67:0/3937808316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5574c8a6d520 con 0x5574c86191b0 2024-07-31T15:10:45.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.283+0000 7f3e8f7fe640 1 -- 172.21.15.67:0/3937808316 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea803d930 con 0x5574c86191b0 2024-07-31T15:10:45.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.283+0000 7f3e8f7fe640 1 -- 172.21.15.67:0/3937808316 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f3ea8046ca0 con 0x5574c86191b0 2024-07-31T15:10:45.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.283+0000 7f3ebce68740 10 client.29198 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:45.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.283+0000 7f3ebce68740 10 client.29198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:45.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.283+0000 7f3ebce68740 20 client.29198 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:45.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.283+0000 7f3ebce68740 1 --2- 172.21.15.67:0/3937808316 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574c85d5da0 0x5574c8a6fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.283+0000 7f3ebce68740 4 client.29198 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:10:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.283+0000 7f3ebce68740 1 -- 172.21.15.67:0/3937808316 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574c895b5a0 con 0x5574c85d5da0 2024-07-31T15:10:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.284+0000 7f3eaf7fe640 1 --2- 172.21.15.67:0/3937808316 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574c85d5da0 0x5574c8a6fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:45.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.290+0000 7f3eaf7fe640 1 --2- 172.21.15.67:0/3937808316 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574c85d5da0 0x5574c8a6fa70 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df471b31ed7be7f8 server_cookie=8ca5bd6177d5887 in_seq=0 out_seq=0 2024-07-31T15:10:45.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.290+0000 7f3e8f7fe640 10 client.29198 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:45.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.293+0000 7f3e8f7fe640 1 -- 172.21.15.67:0/3937808316 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574c895b5a0 con 0x5574c85d5da0 2024-07-31T15:10:45.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.293+0000 7f3e8f7fe640 10 client.29198 handle_command_reply: tid=0 2024-07-31T15:10:45.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.293+0000 7f3ebce68740 1 client.29198 shutdown 2024-07-31T15:10:45.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.302+0000 7f3ebce68740 1 -- 172.21.15.67:0/3937808316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c86191b0 msgr2=0x5574c8a087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:45.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.302+0000 7f3ebce68740 1 --2- 172.21.15.67:0/3937808316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c86191b0 0x5574c8a087c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80099a0 tx=0x7f3ea8030fd0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.309+0000 7f3ebce68740 1 -- 172.21.15.67:0/3937808316 shutdown_connections 2024-07-31T15:10:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.309+0000 7f3ebce68740 1 --2- 172.21.15.67:0/3937808316 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5574c85d5da0 0x5574c8a6fa70 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.309+0000 7f3ebce68740 1 --2- 172.21.15.67:0/3937808316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574c8642070 0x5574c8a06220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.309+0000 7f3ebce68740 1 --2- 172.21.15.67:0/3937808316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c8621390 0x5574c8a05ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.309+0000 7f3e8f7fe640 0 client.29198 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:45.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.309+0000 7f3ebce68740 1 --2- 172.21.15.67:0/3937808316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574c86191b0 0x5574c8a087c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.309+0000 7f3ebce68740 1 -- 172.21.15.67:0/3937808316 >> 172.21.15.67:0/3937808316 conn(0x5574c8636460 msgr2=0x5574c89ec820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:45.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.310+0000 7f3ebce68740 1 -- 172.21.15.67:0/3937808316 shutdown_connections 2024-07-31T15:10:45.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.310+0000 7f3ebce68740 1 -- 172.21.15.67:0/3937808316 wait complete. 2024-07-31T15:10:45.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.310+0000 7f3ebce68740 20 client.29198 trim_cache size 0 max 16384 2024-07-31T15:10:45.312 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:10:45.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.311+0000 7f3eb56b5640 1 -- 172.21.15.67:0/2302789898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e980b3e20 msgr2=0x7f3e980b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:45.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.311+0000 7f3eb56b5640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e980b3e20 0x7f3e980b6330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x5574c89e6390 tx=0x7f3ea40ae040 comp rx=0 tx=0).stop 2024-07-31T15:10:45.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.311+0000 7f3eb56b5640 1 -- 172.21.15.67:0/2302789898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb01055f0 msgr2=0x7f3eb0116130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:45.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.311+0000 7f3eb56b5640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb01055f0 0x7f3eb0116130 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400b450 tx=0x7f3ea400b910 comp rx=0 tx=0).stop 2024-07-31T15:10:45.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.311+0000 7f3eb56b5640 1 -- 172.21.15.67:0/2302789898 shutdown_connections 2024-07-31T15:10:45.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.311+0000 7f3eb56b5640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e980b3e20 0x7f3e980b6330 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.311+0000 7f3eb56b5640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3eb010b880 0x7f3eb010f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.311+0000 7f3eb56b5640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3eb010af30 0x7f3eb010f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.311+0000 7f3eb56b5640 1 --2- 172.21.15.67:0/2302789898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb01055f0 0x7f3eb0116130 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:45.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.311+0000 7f3eb56b5640 1 -- 172.21.15.67:0/2302789898 >> 172.21.15.67:0/2302789898 conn(0x7f3eb0101060 msgr2=0x7f3eb0109490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:45.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.311+0000 7f3eb56b5640 1 -- 172.21.15.67:0/2302789898 shutdown_connections 2024-07-31T15:10:45.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:45.311+0000 7f3eb56b5640 1 -- 172.21.15.67:0/2302789898 wait complete. 2024-07-31T15:10:45.341 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:45 smithi067 ceph-mon[32750]: pgmap v1028: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 504 KiB/s wr, 56 op/s 2024-07-31T15:10:45.344 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/pata_amd.o 2024-07-31T15:10:45.499 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/iface.o 2024-07-31T15:10:45.536 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/sunrpc/built-in.a 2024-07-31T15:10:45.624 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/xfrm6_protocol.o 2024-07-31T15:10:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:45 smithi073 ceph-mon[36714]: pgmap v1028: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 504 KiB/s wr, 56 op/s 2024-07-31T15:10:45.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:45 smithi077 ceph-mon[37327]: pgmap v1028: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 504 KiB/s wr, 56 op/s 2024-07-31T15:10:46.007 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/netlabel/built-in.a 2024-07-31T15:10:46.048 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/fib_rules.o 2024-07-31T15:10:46.059 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/rfkill/core.o 2024-07-31T15:10:46.146 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/netfilter.o 2024-07-31T15:10:46.153 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/pata_oldpiix.o 2024-07-31T15:10:46.221 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_ring.o 2024-07-31T15:10:46.229 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ata/pata_sch.o 2024-07-31T15:10:46.598 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/proc.o 2024-07-31T15:10:46.837 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/syncookies.o 2024-07-31T15:10:46.889 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/rfkill/input.o 2024-07-31T15:10:46.914 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/ata/built-in.a 2024-07-31T15:10:46.996 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/gud/built-in.a 2024-07-31T15:10:47.052 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/phy/mdio-boardinfo.o 2024-07-31T15:10:47.073 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_ring_submission.o 2024-07-31T15:10:47.083 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/ipmr.o 2024-07-31T15:10:47.334 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/pse-pd/built-in.a 2024-07-31T15:10:47.348 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/phy/mdio_devres.o 2024-07-31T15:10:47.367 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T15:10:47.367 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T15:10:47.453 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/rfkill/built-in.a 2024-07-31T15:10:47.523 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/phy/phy.o 2024-07-31T15:10:47.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:47 smithi073 ceph-mon[36714]: pgmap v1029: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 395 KiB/s wr, 69 op/s 2024-07-31T15:10:47.662 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/link.o 2024-07-31T15:10:47.684 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/phy/phy-c45.o 2024-07-31T15:10:47.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:47 smithi077 ceph-mon[37327]: pgmap v1029: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 395 KiB/s wr, 69 op/s 2024-07-31T15:10:47.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:47 smithi067 ceph-mon[32750]: pgmap v1029: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 395 KiB/s wr, 69 op/s 2024-07-31T15:10:47.813 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/rate.o 2024-07-31T15:10:47.942 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/calipso.o 2024-07-31T15:10:47.990 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:10:48.081 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firewire/init_ohci1394_dma.o 2024-07-31T15:10:48.368 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_rps.o 2024-07-31T15:10:48.699 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/firewire/built-in.a 2024-07-31T15:10:48.735 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/phy/phy-core.o 2024-07-31T15:10:48.759 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cdrom/cdrom.o 2024-07-31T15:10:48.808 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/ah6.o 2024-07-31T15:10:49.136 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/ipmr_base.o 2024-07-31T15:10:49.244 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/syncookies.o 2024-07-31T15:10:49.382 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tunnel4.o 2024-07-31T15:10:49.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:49 smithi067 ceph-mon[32750]: pgmap v1030: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 513 KiB/s wr, 93 op/s 2024-07-31T15:10:49.613 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/michael.o 2024-07-31T15:10:49.617 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/phy/phy_device.o 2024-07-31T15:10:49.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:49 smithi073 ceph-mon[36714]: pgmap v1030: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 513 KiB/s wr, 93 op/s 2024-07-31T15:10:49.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:49 smithi077 ceph-mon[37327]: pgmap v1030: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 513 KiB/s wr, 93 op/s 2024-07-31T15:10:49.928 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/esp6.o 2024-07-31T15:10:50.069 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_sa_media.o 2024-07-31T15:10:50.274 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/cdrom/built-in.a 2024-07-31T15:10:50.282 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/9p/mod.o 2024-07-31T15:10:50.327 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/9p/client.o 2024-07-31T15:10:50.329 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/9p/error.o 2024-07-31T15:10:50.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.386+0000 7f5e5f7fe640 1 -- 172.21.15.67:0/681764140 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58031d20 con 0x7f5e601022a0 2024-07-31T15:10:50.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.386+0000 7f5e65988640 1 -- 172.21.15.67:0/681764140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e601022a0 msgr2=0x7f5e601026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:50.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.386+0000 7f5e65988640 1 --2- 172.21.15.67:0/681764140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e601022a0 0x7f5e601026b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009ca0 tx=0x7f5e5802a8e0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.386+0000 7f5e65988640 1 -- 172.21.15.67:0/681764140 shutdown_connections 2024-07-31T15:10:50.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.386+0000 7f5e65988640 1 --2- 172.21.15.67:0/681764140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e60108530 0x7f5e6010ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.386+0000 7f5e65988640 1 --2- 172.21.15.67:0/681764140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60107be0 0x7f5e60107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.386+0000 7f5e65988640 1 --2- 172.21.15.67:0/681764140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e601022a0 0x7f5e601026b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.386+0000 7f5e65988640 1 -- 172.21.15.67:0/681764140 >> 172.21.15.67:0/681764140 conn(0x7f5e600fdcf0 msgr2=0x7f5e60100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:50.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.387+0000 7f5e65988640 1 -- 172.21.15.67:0/681764140 shutdown_connections 2024-07-31T15:10:50.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.387+0000 7f5e65988640 1 -- 172.21.15.67:0/681764140 wait complete. 2024-07-31T15:10:50.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.388+0000 7f5e65988640 1 Processor -- start 2024-07-31T15:10:50.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.388+0000 7f5e65988640 1 -- start start 2024-07-31T15:10:50.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.391+0000 7f5e65988640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e601022a0 0x7f5e6006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.391+0000 7f5e65988640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e60107be0 0x7f5e6006f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.391+0000 7f5e65988640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60108530 0x7f5e601ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.391+0000 7f5e65988640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6006d5b0 con 0x7f5e60108530 2024-07-31T15:10:50.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.391+0000 7f5e65988640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6006d430 con 0x7f5e601022a0 2024-07-31T15:10:50.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.391+0000 7f5e65988640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6006d730 con 0x7f5e60107be0 2024-07-31T15:10:50.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.392+0000 7f5e5ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e60107be0 0x7f5e6006f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.392+0000 7f5e65187640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60108530 0x7f5e601ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.392+0000 7f5e65187640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60108530 0x7f5e601ac0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47774/0 (socket says 172.21.15.67:47774) 2024-07-31T15:10:50.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.392+0000 7f5e65187640 1 -- 172.21.15.67:0/4005640779 learned_addr learned my addr 172.21.15.67:0/4005640779 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:50.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.392+0000 7f5e64986640 1 --2- 172.21.15.67:0/4005640779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e601022a0 0x7f5e6006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.393+0000 7f5e65187640 1 -- 172.21.15.67:0/4005640779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e60107be0 msgr2=0x7f5e6006f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:50.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.393+0000 7f5e65187640 1 --2- 172.21.15.67:0/4005640779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e60107be0 0x7f5e6006f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.393+0000 7f5e65187640 1 -- 172.21.15.67:0/4005640779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e601022a0 msgr2=0x7f5e6006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:50.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.393+0000 7f5e65187640 1 --2- 172.21.15.67:0/4005640779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e601022a0 0x7f5e6006f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.393+0000 7f5e65187640 1 -- 172.21.15.67:0/4005640779 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6006fec0 con 0x7f5e60108530 2024-07-31T15:10:50.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.393+0000 7f5e65187640 1 --2- 172.21.15.67:0/4005640779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60108530 0x7f5e601ac0e0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f5e48007800 tx=0x7f5e48007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:50.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.394+0000 7f5e5dffb640 1 -- 172.21.15.67:0/4005640779 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48019030 con 0x7f5e60108530 2024-07-31T15:10:50.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.394+0000 7f5e65988640 1 -- 172.21.15.67:0/4005640779 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e601ac6e0 con 0x7f5e60108530 2024-07-31T15:10:50.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.394+0000 7f5e5dffb640 1 -- 172.21.15.67:0/4005640779 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e4800fdd0 con 0x7f5e60108530 2024-07-31T15:10:50.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.394+0000 7f5e5dffb640 1 -- 172.21.15.67:0/4005640779 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48004ad0 con 0x7f5e60108530 2024-07-31T15:10:50.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.394+0000 7f5e65988640 1 -- 172.21.15.67:0/4005640779 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e601acc20 con 0x7f5e60108530 2024-07-31T15:10:50.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.401+0000 7f5e5dffb640 1 -- 172.21.15.67:0/4005640779 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5e4801c070 con 0x7f5e60108530 2024-07-31T15:10:50.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.402+0000 7f5e5dffb640 1 --2- 172.21.15.67:0/4005640779 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5e4c0b3e70 0x7f5e4c0b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.402+0000 7f5e5dffb640 1 -- 172.21.15.67:0/4005640779 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f5e480e4160 con 0x7f5e60108530 2024-07-31T15:10:50.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.403+0000 7f5e64986640 1 --2- 172.21.15.67:0/4005640779 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5e4c0b3e70 0x7f5e4c0b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.404+0000 7f5e64986640 1 --2- 172.21.15.67:0/4005640779 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5e4c0b3e70 0x7f5e4c0b6380 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5e58006010 tx=0x7f5e58000f40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:50.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.408+0000 7f5e6d13b740 1 Processor -- start 2024-07-31T15:10:50.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.408+0000 7f5e6d13b740 1 -- start start 2024-07-31T15:10:50.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.408+0000 7f5e6d13b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096c13350 0x560096fb8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.408+0000 7f5e6d13b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 0x560096fb8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.408+0000 7f5e6d13b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096bf2670 0x560096fb9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.408+0000 7f5e6d13b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560096ea02f0 con 0x560096bf2670 2024-07-31T15:10:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.408+0000 7f5e6d13b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560096f2ea80 con 0x560096bea490 2024-07-31T15:10:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.408+0000 7f5e6d13b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560096f1df80 con 0x560096c13350 2024-07-31T15:10:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.409+0000 7f5e65187640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096bf2670 0x560096fb9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.409+0000 7f5e64986640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096c13350 0x560096fb8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.409+0000 7f5e65187640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096bf2670 0x560096fb9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47780/0 (socket says 172.21.15.67:47780) 2024-07-31T15:10:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.409+0000 7f5e65187640 1 -- 172.21.15.67:0/691922502 learned_addr learned my addr 172.21.15.67:0/691922502 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.409+0000 7f5e65187640 1 -- 172.21.15.67:0/691922502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096c13350 msgr2=0x560096fb8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.409+0000 7f5e65187640 1 --2- 172.21.15.67:0/691922502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096c13350 0x560096fb8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.409+0000 7f5e65187640 1 -- 172.21.15.67:0/691922502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 msgr2=0x560096fb8ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:10:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.409+0000 7f5e65187640 1 --2- 172.21.15.67:0/691922502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 0x560096fb8ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.409+0000 7f5e65187640 1 -- 172.21.15.67:0/691922502 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560096c7e930 con 0x560096bf2670 2024-07-31T15:10:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.411+0000 7f5e64986640 1 --2- 172.21.15.67:0/691922502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096c13350 0x560096fb8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:10:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.411+0000 7f5e65187640 1 --2- 172.21.15.67:0/691922502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096bf2670 0x560096fb9000 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5e480aa990 tx=0x7f5e480aa510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57058f4f00159e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:50.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.413+0000 7f5e56ffd640 1 -- 172.21.15.67:0/691922502 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e480b3400 con 0x560096bf2670 2024-07-31T15:10:50.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.413+0000 7f5e56ffd640 1 -- 172.21.15.67:0/691922502 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e480aaa00 con 0x560096bf2670 2024-07-31T15:10:50.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.413+0000 7f5e56ffd640 1 -- 172.21.15.67:0/691922502 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e480e17c0 con 0x560096bf2670 2024-07-31T15:10:50.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.414+0000 7f5e6d13b740 1 -- 172.21.15.67:0/691922502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096bf2670 msgr2=0x560096fb9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:50.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.414+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/691922502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096bf2670 0x560096fb9000 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5e480aa990 tx=0x7f5e480aa510 comp rx=0 tx=0).stop 2024-07-31T15:10:50.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.415+0000 7f5e6d13b740 1 -- 172.21.15.67:0/691922502 shutdown_connections 2024-07-31T15:10:50.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.415+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/691922502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096bf2670 0x560096fb9000 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.415+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/691922502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 0x560096fb8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.415+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/691922502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096c13350 0x560096fb8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.415+0000 7f5e6d13b740 1 -- 172.21.15.67:0/691922502 >> 172.21.15.67:0/691922502 conn(0x560096c07740 msgr2=0x560096fb6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:50.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.415+0000 7f5e6d13b740 1 -- 172.21.15.67:0/691922502 shutdown_connections 2024-07-31T15:10:50.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.415+0000 7f5e6d13b740 1 -- 172.21.15.67:0/691922502 wait complete. 2024-07-31T15:10:50.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.416+0000 7f5e6d13b740 1 -- start start 2024-07-31T15:10:50.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.416+0000 7f5e6d13b740 1 Processor -- start 2024-07-31T15:10:50.418 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/ipconfig.o 2024-07-31T15:10:50.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.417+0000 7f5e6d13b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bea490 0x560096fc4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.417+0000 7f5e6d13b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096bf2670 0x560096fc4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.417+0000 7f5e6d13b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096c13350 0x560096fc9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.417+0000 7f5e6d13b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560096ec04f0 con 0x560096bf2670 2024-07-31T15:10:50.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.417+0000 7f5e6d13b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560096f21fc0 con 0x560096c13350 2024-07-31T15:10:50.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.417+0000 7f5e6d13b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560096d81890 con 0x560096bea490 2024-07-31T15:10:50.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.417+0000 7f5e6d13b740 10 client.0 _ll_register_callbacks cb 0x560096eca500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:50.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.417+0000 7f5e5ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096bf2670 0x560096fc4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.417+0000 7f5e5ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096bf2670 0x560096fc4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47794/0 (socket says 172.21.15.67:47794) 2024-07-31T15:10:50.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.417+0000 7f5e5ffff640 1 -- 172.21.15.67:0/1173584533 learned_addr learned my addr 172.21.15.67:0/1173584533 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:50.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.418+0000 7f5e65187640 1 --2- 172.21.15.67:0/1173584533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096c13350 0x560096fc9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.418+0000 7f5e64986640 1 --2- 172.21.15.67:0/1173584533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bea490 0x560096fc4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.419+0000 7f5e65187640 1 -- 172.21.15.67:0/1173584533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bea490 msgr2=0x560096fc4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:50.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.419+0000 7f5e65187640 1 --2- 172.21.15.67:0/1173584533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bea490 0x560096fc4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.419+0000 7f5e65187640 1 -- 172.21.15.67:0/1173584533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096bf2670 msgr2=0x560096fc4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:50.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.419+0000 7f5e65187640 1 --2- 172.21.15.67:0/1173584533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096bf2670 0x560096fc4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.419+0000 7f5e65187640 1 -- 172.21.15.67:0/1173584533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560096ecfc50 con 0x560096c13350 2024-07-31T15:10:50.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.420+0000 7f5e65187640 1 --2- 172.21.15.67:0/1173584533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096c13350 0x560096fc9e40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5e480aa940 tx=0x7f5e480b3540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc912fc2ed6bf9b1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:50.422 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/tkip.o 2024-07-31T15:10:50.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.422+0000 7f5e3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:10:50.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.422+0000 7f5e3ffff640 1 -- 172.21.15.67:0/1173584533 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e480c9070 con 0x560096c13350 2024-07-31T15:10:50.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.422+0000 7f5e3ffff640 1 -- 172.21.15.67:0/1173584533 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f5e4800f920 con 0x560096c13350 2024-07-31T15:10:50.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.423+0000 7f5e6d13b740 1 -- 172.21.15.67:0/1173584533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e480aaa00 con 0x560096c13350 2024-07-31T15:10:50.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.423+0000 7f5e3ffff640 1 -- 172.21.15.67:0/1173584533 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f5e480e19d0 con 0x560096c13350 2024-07-31T15:10:50.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.423+0000 7f5e6d13b740 10 client.29207 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:50.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.423+0000 7f5e6d13b740 1 -- 172.21.15.67:0/1173584533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x560096ecfc50 con 0x560096c13350 2024-07-31T15:10:50.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.423+0000 7f5e3ffff640 1 -- 172.21.15.67:0/1173584533 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e480aaa00 con 0x560096c13350 2024-07-31T15:10:50.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.423+0000 7f5e3ffff640 1 -- 172.21.15.67:0/1173584533 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f5e48026060 con 0x560096c13350 2024-07-31T15:10:50.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.423+0000 7f5e6d13b740 10 client.29207 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:50.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.423+0000 7f5e6d13b740 10 client.29207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:50.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.423+0000 7f5e6d13b740 20 client.29207 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:50.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.424+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/1173584533 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560096bd1cd0 0x560096fd5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.424+0000 7f5e6d13b740 4 client.29207 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:10:50.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.424+0000 7f5e6d13b740 1 -- 172.21.15.67:0/1173584533 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560096f309d0 con 0x560096bd1cd0 2024-07-31T15:10:50.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.424+0000 7f5e64986640 1 --2- 172.21.15.67:0/1173584533 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560096bd1cd0 0x560096fd5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.425+0000 7f5e64986640 1 --2- 172.21.15.67:0/1173584533 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560096bd1cd0 0x560096fd5e50 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fb1ae2c357e9788 server_cookie=4635d8df598c1c13 in_seq=0 out_seq=0 2024-07-31T15:10:50.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.425+0000 7f5e3ffff640 10 client.29207 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:50.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.426+0000 7f5e3ffff640 1 -- 172.21.15.67:0/1173584533 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560096f309d0 con 0x560096bd1cd0 2024-07-31T15:10:50.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.426+0000 7f5e3ffff640 10 client.29207 handle_command_reply: tid=0 2024-07-31T15:10:50.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.426+0000 7f5e6d13b740 1 client.29207 shutdown 2024-07-31T15:10:50.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.427+0000 7f5e6d13b740 1 -- 172.21.15.67:0/1173584533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096c13350 msgr2=0x560096fc9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:50.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.427+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/1173584533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096c13350 0x560096fc9e40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5e480aa940 tx=0x7f5e480b3540 comp rx=0 tx=0).stop 2024-07-31T15:10:50.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.427+0000 7f5e6d13b740 1 -- 172.21.15.67:0/1173584533 shutdown_connections 2024-07-31T15:10:50.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.427+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/1173584533 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560096bd1cd0 0x560096fd5e50 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.427+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/1173584533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096c13350 0x560096fc9e40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.427+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/1173584533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096bf2670 0x560096fc4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.427+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/1173584533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bea490 0x560096fc4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.427+0000 7f5e6d13b740 1 -- 172.21.15.67:0/1173584533 >> 172.21.15.67:0/1173584533 conn(0x560096c07740 msgr2=0x560096fbb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:50.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.427+0000 7f5e3ffff640 0 client.29207 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:50.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.428+0000 7f5e6d13b740 1 -- 172.21.15.67:0/1173584533 shutdown_connections 2024-07-31T15:10:50.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.428+0000 7f5e6d13b740 1 -- 172.21.15.67:0/1173584533 wait complete. 2024-07-31T15:10:50.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.429+0000 7f5e6d13b740 20 client.29207 trim_cache size 0 max 16384 2024-07-31T15:10:50.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.469+0000 7f5e6d13b740 1 Processor -- start 2024-07-31T15:10:50.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.469+0000 7f5e6d13b740 1 -- start start 2024-07-31T15:10:50.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.470+0000 7f5e6d13b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 0x560096fb53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.470+0000 7f5e6d13b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bf2670 0x560096fbd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.470+0000 7f5e6d13b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096c13350 0x560096fbdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.470+0000 7f5e6d13b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601b6cb0 con 0x560096c13350 2024-07-31T15:10:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.471+0000 7f5e6d13b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601b87f0 con 0x560096bea490 2024-07-31T15:10:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.471+0000 7f5e6d13b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601b7380 con 0x560096bf2670 2024-07-31T15:10:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.471+0000 7f5e64986640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 0x560096fb53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.471+0000 7f5e64986640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 0x560096fb53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52972/0 (socket says 172.21.15.67:52972) 2024-07-31T15:10:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.471+0000 7f5e64986640 1 -- 172.21.15.67:0/123092531 learned_addr learned my addr 172.21.15.67:0/123092531 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.471+0000 7f5e65187640 1 --2- 172.21.15.67:0/123092531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096c13350 0x560096fbdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.471+0000 7f5e5ffff640 1 --2- 172.21.15.67:0/123092531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bf2670 0x560096fbd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.475+0000 7f5e65187640 1 -- 172.21.15.67:0/123092531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bf2670 msgr2=0x560096fbd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.475+0000 7f5e65187640 1 --2- 172.21.15.67:0/123092531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bf2670 0x560096fbd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.475+0000 7f5e65187640 1 -- 172.21.15.67:0/123092531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 msgr2=0x560096fb53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.475+0000 7f5e65187640 1 --2- 172.21.15.67:0/123092531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 0x560096fb53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.475+0000 7f5e65187640 1 -- 172.21.15.67:0/123092531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560096ecfc50 con 0x560096c13350 2024-07-31T15:10:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.476+0000 7f5e65187640 1 --2- 172.21.15.67:0/123092531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096c13350 0x560096fbdd40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f5e480a61b0 tx=0x7f5e480054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f985b2418b0754d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.477+0000 7f5e567fc640 1 -- 172.21.15.67:0/123092531 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e480c9070 con 0x560096c13350 2024-07-31T15:10:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.477+0000 7f5e567fc640 1 -- 172.21.15.67:0/123092531 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e480af000 con 0x560096c13350 2024-07-31T15:10:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.478+0000 7f5e6d13b740 1 -- 172.21.15.67:0/123092531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096c13350 msgr2=0x560096fbdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.478+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/123092531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096c13350 0x560096fbdd40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f5e480a61b0 tx=0x7f5e480054e0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.479+0000 7f5e6d13b740 1 -- 172.21.15.67:0/123092531 shutdown_connections 2024-07-31T15:10:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.479+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/123092531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096c13350 0x560096fbdd40 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.479+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/123092531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bf2670 0x560096fbd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.479+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/123092531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 0x560096fb53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.479+0000 7f5e6d13b740 1 -- 172.21.15.67:0/123092531 >> 172.21.15.67:0/123092531 conn(0x560096c07740 msgr2=0x560096fb6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.485+0000 7f5e6d13b740 1 -- 172.21.15.67:0/123092531 shutdown_connections 2024-07-31T15:10:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.489+0000 7f5e6d13b740 1 -- 172.21.15.67:0/123092531 wait complete. 2024-07-31T15:10:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.489+0000 7f5e6d13b740 1 -- start start 2024-07-31T15:10:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.489+0000 7f5e6d13b740 1 Processor -- start 2024-07-31T15:10:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.490+0000 7f5e6d13b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 0x560096fcb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.490+0000 7f5e6d13b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bf2670 0x560096fcbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.490+0000 7f5e6d13b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096c13350 0x56009703c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.490+0000 7f5e6d13b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601b8120 con 0x560096c13350 2024-07-31T15:10:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.490+0000 7f5e6d13b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601b7a50 con 0x560096bea490 2024-07-31T15:10:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.490+0000 7f5e6d13b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601b4c90 con 0x560096bf2670 2024-07-31T15:10:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.490+0000 7f5e6d13b740 10 client.0 _ll_register_callbacks cb 0x7f5e480b3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.490+0000 7f5e64986640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 0x560096fcb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.491+0000 7f5e64986640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 0x560096fcb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52976/0 (socket says 172.21.15.67:52976) 2024-07-31T15:10:50.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.491+0000 7f5e64986640 1 -- 172.21.15.67:0/2602701405 learned_addr learned my addr 172.21.15.67:0/2602701405 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:50.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.491+0000 7f5e65187640 1 --2- 172.21.15.67:0/2602701405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096c13350 0x56009703c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.491+0000 7f5e5ffff640 1 --2- 172.21.15.67:0/2602701405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bf2670 0x560096fcbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.492+0000 7f5e65187640 1 -- 172.21.15.67:0/2602701405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bf2670 msgr2=0x560096fcbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:50.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.492+0000 7f5e65187640 1 --2- 172.21.15.67:0/2602701405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bf2670 0x560096fcbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.492+0000 7f5e65187640 1 -- 172.21.15.67:0/2602701405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 msgr2=0x560096fcb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:50.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.492+0000 7f5e65187640 1 --2- 172.21.15.67:0/2602701405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 0x560096fcb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.492+0000 7f5e65187640 1 -- 172.21.15.67:0/2602701405 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009703c9f0 con 0x560096c13350 2024-07-31T15:10:50.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.492+0000 7f5e65187640 1 --2- 172.21.15.67:0/2602701405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096c13350 0x56009703c380 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5e480a61b0 tx=0x7f5e48090610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928927198aa9bbb9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:50.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.493+0000 7f5e3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:10:50.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.493+0000 7f5e3ffff640 1 -- 172.21.15.67:0/2602701405 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e480c9070 con 0x560096c13350 2024-07-31T15:10:50.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.493+0000 7f5e6d13b740 1 -- 172.21.15.67:0/2602701405 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e480af000 con 0x560096c13350 2024-07-31T15:10:50.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.493+0000 7f5e3ffff640 1 -- 172.21.15.67:0/2602701405 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f5e4800fb30 con 0x560096c13350 2024-07-31T15:10:50.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.494+0000 7f5e3ffff640 1 -- 172.21.15.67:0/2602701405 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f5e480aaa00 con 0x560096c13350 2024-07-31T15:10:50.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.494+0000 7f5e6d13b740 10 client.19698 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:50.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.494+0000 7f5e6d13b740 1 -- 172.21.15.67:0/2602701405 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56009703c9f0 con 0x560096c13350 2024-07-31T15:10:50.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.495+0000 7f5e3ffff640 1 -- 172.21.15.67:0/2602701405 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e480af000 con 0x560096c13350 2024-07-31T15:10:50.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.495+0000 7f5e3ffff640 1 -- 172.21.15.67:0/2602701405 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f5e480a96d0 con 0x560096c13350 2024-07-31T15:10:50.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.495+0000 7f5e6d13b740 10 client.19698 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:50.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.495+0000 7f5e6d13b740 10 client.19698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:50.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.495+0000 7f5e6d13b740 20 client.19698 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:50.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.495+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/2602701405 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560096ba7080 0x56009703ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:50.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.495+0000 7f5e6d13b740 4 client.19698 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T15:10:50.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.495+0000 7f5e6d13b740 1 -- 172.21.15.67:0/2602701405 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560096f2c470 con 0x560096ba7080 2024-07-31T15:10:50.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.495+0000 7f5e64986640 1 --2- 172.21.15.67:0/2602701405 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560096ba7080 0x56009703ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.509+0000 7f5e64986640 1 --2- 172.21.15.67:0/2602701405 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560096ba7080 0x56009703ff50 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bd6324207a3a177 server_cookie=cf4025273b911759 in_seq=0 out_seq=0 2024-07-31T15:10:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.509+0000 7f5e3ffff640 10 client.19698 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.513+0000 7f5e3ffff640 1 -- 172.21.15.67:0/2602701405 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560096f2c470 con 0x560096ba7080 2024-07-31T15:10:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.513+0000 7f5e3ffff640 10 client.19698 handle_command_reply: tid=0 2024-07-31T15:10:50.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.513+0000 7f5e6d13b740 1 client.19698 shutdown 2024-07-31T15:10:50.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.522+0000 7f5e6d13b740 1 -- 172.21.15.67:0/2602701405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096c13350 msgr2=0x56009703c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:50.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.522+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/2602701405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096c13350 0x56009703c380 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5e480a61b0 tx=0x7f5e48090610 comp rx=0 tx=0).stop 2024-07-31T15:10:50.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.522+0000 7f5e6d13b740 1 -- 172.21.15.67:0/2602701405 shutdown_connections 2024-07-31T15:10:50.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.522+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/2602701405 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560096ba7080 0x56009703ff50 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.522+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/2602701405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560096c13350 0x56009703c380 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.522+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/2602701405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560096bf2670 0x560096fcbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.522+0000 7f5e6d13b740 1 --2- 172.21.15.67:0/2602701405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560096bea490 0x560096fcb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.522+0000 7f5e6d13b740 1 -- 172.21.15.67:0/2602701405 >> 172.21.15.67:0/2602701405 conn(0x560096c07740 msgr2=0x560096fb6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:50.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.525+0000 7f5e3ffff640 0 client.19698 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:50.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.529+0000 7f5e6d13b740 1 -- 172.21.15.67:0/2602701405 shutdown_connections 2024-07-31T15:10:50.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.531+0000 7f5e6d13b740 1 -- 172.21.15.67:0/2602701405 wait complete. 2024-07-31T15:10:50.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.535+0000 7f5e6d13b740 20 client.19698 trim_cache size 0 max 16384 2024-07-31T15:10:50.539 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:10:50.539 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T15:10:50.539 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "c05a848d-c8e4-481b-ad13-32de04ecf834", 2024-07-31T15:10:50.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T15:10:50.539 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:10:50.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.538+0000 7f5e65988640 1 -- 172.21.15.67:0/4005640779 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5e4c0b3e70 msgr2=0x7f5e4c0b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:50.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.538+0000 7f5e65988640 1 --2- 172.21.15.67:0/4005640779 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5e4c0b3e70 0x7f5e4c0b6380 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5e58006010 tx=0x7f5e58000f40 comp rx=0 tx=0).stop 2024-07-31T15:10:50.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.539+0000 7f5e65988640 1 -- 172.21.15.67:0/4005640779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60108530 msgr2=0x7f5e601ac0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:50.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.539+0000 7f5e65988640 1 --2- 172.21.15.67:0/4005640779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60108530 0x7f5e601ac0e0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f5e48007800 tx=0x7f5e48007cc0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.543+0000 7f5e65988640 1 -- 172.21.15.67:0/4005640779 shutdown_connections 2024-07-31T15:10:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.543+0000 7f5e65988640 1 --2- 172.21.15.67:0/4005640779 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5e4c0b3e70 0x7f5e4c0b6380 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.543+0000 7f5e65988640 1 --2- 172.21.15.67:0/4005640779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60108530 0x7f5e601ac0e0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.543+0000 7f5e65988640 1 --2- 172.21.15.67:0/4005640779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e60107be0 0x7f5e6006f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.543+0000 7f5e65988640 1 --2- 172.21.15.67:0/4005640779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e601022a0 0x7f5e6006f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.543+0000 7f5e65988640 1 -- 172.21.15.67:0/4005640779 >> 172.21.15.67:0/4005640779 conn(0x7f5e600fdcf0 msgr2=0x7f5e60100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:50.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.544+0000 7f5e65988640 1 -- 172.21.15.67:0/4005640779 shutdown_connections 2024-07-31T15:10:50.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:50.544+0000 7f5e65988640 1 -- 172.21.15.67:0/4005640779 wait complete. 2024-07-31T15:10:50.819 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/aes_cmac.o 2024-07-31T15:10:50.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:50 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:50.574+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000162a [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch/ auth{1=1,2=1,3=1,4=1} v21244 ap=2 RANDEPHEMERALPIN f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22) n(v53 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939) old_inodes=7 (inest sync->mix g=1,2,3,4) caps={15489=pAsLsXsFs/-@60} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7c26ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000162a:[<0x10000000202/arch v378>,<0x10000000201/linux-6.5.11 v96>,<0x10000000006/t v68>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v57>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v60>,<0x10000000001/sv_1 v84>,<0x10000000000/qa v70>,<0x1/volumes v70>]//[]","memoryvalue":"(2)0x1000000162a:[<0x10000000202/arch v21244>,<0x10000000201/linux-6.5.11 v2540>,<0x10000000006/t v1586>,<0x10000000005/tmp v1513>,<0x100000001fb/client.0 v1488>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1451>,<0x10000000001/sv_1 v1452>,<0x10000000000/qa v1442>,<0x1/volumes v1395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:54:49.652308+0000 23=2+21)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939)","memory_value.dirstat":"f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22)","memory_value.rstat":"n(v53 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:50.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:50 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:50.638+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c276 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802/ auth{3=1} v10568 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:29.002671+0000 14=14+0) n(v3 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1) old_inodes=2 (inest sync->mix g=3) caps={15489=pAsLsXs/-@11} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8fd23180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c276:[<0x1000000c25e/802 v68>,<0x10000000202/net v7765>,<0x10000000201/linux-6.5.11 v689>,<0x10000000006/t v498>,<0x10000000005/tmp v466>,<0x100000001fb/client.0 v452>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v438>,<0x10000000001/sv_1 v453>,<0x10000000000/qa v441>,<0x1/volumes v433>]//[]","memoryvalue":"(2)0x1000000c276:[<0x1000000c25e/802 v10568>,<0x10000000202/net v21607>,<0x10000000201/linux-6.5.11 v2540>,<0x10000000006/t v1586>,<0x10000000005/tmp v1513>,<0x100000001fb/client.0 v1488>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1451>,<0x10000000001/sv_1 v1452>,<0x10000000000/qa v1442>,<0x1/volumes v1395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:49.293317+0000 10=10+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:29.002671+0000 14=14+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:50.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:50 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:50.641+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c836 [...1f,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet/ auth{4=1} v12171 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:57.081877+0000 11=11+0) n(v2 rc2024-07-31T15:08:57.083877+0000 b263808 12=11+1) old_inodes=2 (inest sync->mix g=4) caps={15489=pAsLsXs/-@6} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8c20b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c836:[<0x1000000c25e/packet v6437>,<0x10000000202/net v8736>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000c836:[<0x1000000c25e/packet v12171>,<0x10000000202/net v21607>,<0x10000000201/linux-6.5.11 v2540>,<0x10000000006/t v1586>,<0x10000000005/tmp v1513>,<0x100000001fb/client.0 v1488>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1451>,<0x10000000001/sv_1 v1452>,<0x10000000000/qa v1442>,<0x1/volumes v1395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:56.791162+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:57.083877+0000 b263808 12=11+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:57.081877+0000 11=11+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:08:57.083877+0000 b263808 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:50.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:50 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:50.642+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c9e2 [...1f,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix/ auth{2=1} v12135 ap=2 RANDEPHEMERALPIN f(v2 m2024-07-31T15:08:53.494950+0000 21=21+0) n(v5 rc2024-07-31T15:08:53.495950+0000 b445955 22=21+1) old_inodes=3 (inest sync->mix g=2) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8591ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c9e2:[<0x1000000c25e/unix v6743>,<0x10000000202/net v8736>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000c9e2:[<0x1000000c25e/unix v12135>,<0x10000000202/net v21607>,<0x10000000201/linux-6.5.11 v2540>,<0x10000000006/t v1586>,<0x10000000005/tmp v1513>,<0x100000001fb/client.0 v1488>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1451>,<0x10000000001/sv_1 v1452>,<0x10000000000/qa v1442>,<0x1/volumes v1395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:08:21.655599+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:53.495950+0000 b445955 22=21+1)","memory_value.dirstat":"f(v2 m2024-07-31T15:08:53.494950+0000 21=21+0)","memory_value.rstat":"n(v5 rc2024-07-31T15:08:53.495950+0000 b445955 22=21+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:50.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:50 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:50.672+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d9b4 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb/ auth{3=1} v3876 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7) n(v5 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8) old_inodes=3 (inest sync->mix g=3) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce92cb2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v2020>,<0x10000000202/sound v9936>,<0x10000000201/linux-6.5.11 v830>,<0x10000000006/t v586>,<0x10000000005/tmp v553>,<0x100000001fb/client.0 v537>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v519>,<0x10000000001/sv_1 v525>,<0x10000000000/qa v522>,<0x1/volumes v503>]//[]","memoryvalue":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v3876>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2540>,<0x10000000006/t v1586>,<0x10000000005/tmp v1513>,<0x100000001fb/client.0 v1488>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1451>,<0x10000000001/sv_1 v1452>,<0x10000000000/qa v1442>,<0x1/volumes v1395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:46.317139+0000 53=46+7)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7)","memory_value.rstat":"n(v5 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:50.971 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/dns_resolver/dns_key.o 2024-07-31T15:10:51.104 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_sseu.o 2024-07-31T15:10:51.154 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:50 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:50.898+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c061 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math/ auth{1=1} v1164 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:10.213499+0000 31=31+0) n(v2 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1) old_inodes=2 (inest sync->mix g=1) caps={15489=pAsLsXs/-@10} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8e92fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c061:[<0x1000000bf5f/math v634>,<0x10000000202/lib v5688>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000c061:[<0x1000000bf5f/math v1164>,<0x10000000202/lib v21040>,<0x10000000201/linux-6.5.11 v2540>,<0x10000000006/t v1586>,<0x10000000005/tmp v1513>,<0x100000001fb/client.0 v1488>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1451>,<0x10000000001/sv_1 v1452>,<0x10000000000/qa v1442>,<0x1/volumes v1395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:10.988109+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:10.213499+0000 31=31+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:51.155 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:51.079+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000ce48 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity/ auth{1=1,2=1} v1695 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3) n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4) old_inodes=3 (inest sync->mix g=1,2) caps={15489=pAsLsXs/-@10} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8aaac580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000ce48:[<0x1000000ce07/integrity v345>,<0x10000000202/security v8766>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000ce48:[<0x1000000ce07/integrity v1695>,<0x10000000202/security v15110>,<0x10000000201/linux-6.5.11 v2540>,<0x10000000006/t v1586>,<0x10000000005/tmp v1513>,<0x100000001fb/client.0 v1488>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1451>,<0x10000000001/sv_1 v1452>,<0x10000000000/qa v1442>,<0x1/volumes v1395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:59.470106+0000 10=7+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","memory_value.dirstat":"f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3)","memory_value.rstat":"n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:51.155 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:51.153+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009eb8 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs/ auth{4=1} v4394 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:25.901734+0000 12=12+0) n(v3 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1) old_inodes=2 (inest sync->mix g=4) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce809d7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009eb8:[<0x10000009d36/debugfs v3846>,<0x10000000202/fs v3830>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009eb8:[<0x10000009d36/debugfs v4394>,<0x10000000202/fs v21227>,<0x10000000201/linux-6.5.11 v2540>,<0x10000000006/t v1586>,<0x10000000005/tmp v1513>,<0x100000001fb/client.0 v1488>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1451>,<0x10000000001/sv_1 v1452>,<0x10000000000/qa v1442>,<0x1/volumes v1395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:39.315767+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:25.901734+0000 12=12+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:51.476 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:10:51.501 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/phy/linkmode.o 2024-07-31T15:10:51.634 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: pgmap v1031: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 359 KiB/s wr, 97 op/s 2024-07-31T15:10:51.634 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T15:10:51.634 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:51.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:51.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:51.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:51.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:51.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:51.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c836 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c9e2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:51.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:51.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:51.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:51.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:51 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.643 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:51.174+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000a492 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs/ auth{1=1} v5396 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:30.806634+0000 8=8+0) n(v3 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1) old_inodes=2 (inest sync->mix g=1) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8aca9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000a492:[<0x10000009d36/tracefs v3850>,<0x10000000202/fs v3830>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x1000000a492:[<0x10000009d36/tracefs v5396>,<0x10000000202/fs v21227>,<0x10000000201/linux-6.5.11 v2540>,<0x10000000006/t v1586>,<0x10000000005/tmp v1513>,<0x100000001fb/client.0 v1488>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1451>,<0x10000000001/sv_1 v1452>,<0x10000000000/qa v1442>,<0x1/volumes v1395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:41.554720+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:30.806634+0000 8=8+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:51.643 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:51.198+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002351 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char/ auth{2=1} v18176 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:34.793553+0000 56=49+7) n(v10 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10) old_inodes=4 (inest sync->mix g=2) caps={15489=pAsLsXs/-@11} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7ef94100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000002351:[<0x10000001d8f/char v2490>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x10000002351:[<0x10000001d8f/char v18176>,<0x10000000202/drivers v21605>,<0x10000000201/linux-6.5.11 v2540>,<0x10000000006/t v1586>,<0x10000000005/tmp v1513>,<0x100000001fb/client.0 v1488>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1451>,<0x10000000001/sv_1 v1452>,<0x10000000000/qa v1442>,<0x1/volumes v1395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:02.131047+0000 40=33+7)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:34.793553+0000 56=49+7)","memory_value.rstat":"n(v10 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:51.643 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:51.280+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000243a [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk/ auth{2=1,4=1} fragtree_t(*^3) v18679 ap=2 RANDEPHEMERALPIN f(v3 m2024-07-31T15:07:07.059118+0000 137=88+49) n(v9 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50) old_inodes=2 (inest sync->mix g=2,4) caps={15489=pAsLsXs/-@8} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7fa16680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000243a:[<0x10000001d8f/clk v2484>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x1000000243a:[<0x10000001d8f/clk v18679>,<0x10000000202/drivers v21605>,<0x10000000201/linux-6.5.11 v2540>,<0x10000000006/t v1586>,<0x10000000005/tmp v1513>,<0x100000001fb/client.0 v1488>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1451>,<0x10000000001/sv_1 v1452>,<0x10000000000/qa v1442>,<0x1/volumes v1395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 133=84+49)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50)","memory_value.dirstat":"f(v3 m2024-07-31T15:07:07.059118+0000 137=88+49)","memory_value.rstat":"n(v9 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:51.643 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:51.308+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000008596 [...21,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/scsi/ auth{1=6,2=1,3=1,4=1} fragtree_t(*^3) v18555 ap=2 RANDEPHEMERALPIN f(v4 m2024-07-31T15:10:01.775559+0000 269=232+37) n(v11 rc2024-07-31T15:10:01.777559+0000 b31477139 971=924+47) old_inodes=2 (inest sync->mix g=1,2,3,4) caps={15489=pAsLsXsFs/pAsLsXsFsx@11} mcw={1=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce864ab600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000008596:[<0x10000001d8f/scsi v16994>,<0x10000000202/drivers v3182>,<0x10000000201/linux-6.5.11 v445>,<0x10000000006/t v329>,<0x10000000005/tmp v299>,<0x100000001fb/client.0 v296>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v274>,<0x10000000001/sv_1 v299>,<0x10000000000/qa v280>,<0x1/volumes v280>]//[]","memoryvalue":"(2)0x10000008596:[<0x10000001d8f/scsi v18555>,<0x10000000202/drivers v21605>,<0x10000000201/linux-6.5.11 v2540>,<0x10000000006/t v1586>,<0x10000000005/tmp v1513>,<0x100000001fb/client.0 v1488>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1451>,<0x10000000001/sv_1 v1452>,<0x10000000000/qa v1442>,<0x1/volumes v1395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:10:00.625582+0000 265=228+37)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:10:01.777559+0000 b31477139 971=924+47)","memory_value.dirstat":"f(v4 m2024-07-31T15:10:01.775559+0000 269=232+37)","memory_value.rstat":"n(v11 rc2024-07-31T15:10:01.777559+0000 b31477139 971=924+47)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:51.643 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:51.310+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009980 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video/ auth{1=1,4=1} v18105 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:16.340374+0000 28=24+4) n(v11 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28) old_inodes=2 (inest sync->mix g=1,4) caps={15489=pAsLsXs/-@6} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce87325b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009980:[<0x10000001d8f/video v17889>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009980:[<0x10000001d8f/video v18105>,<0x10000000202/drivers v21605>,<0x10000000201/linux-6.5.11 v2540>,<0x10000000006/t v1586>,<0x10000000005/tmp v1513>,<0x100000001fb/client.0 v1488>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1451>,<0x10000000001/sv_1 v1452>,<0x10000000000/qa v1442>,<0x1/volumes v1395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:35.535846+0000 16=12+4)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:16.340374+0000 28=24+4)","memory_value.rstat":"n(v11 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:51.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: pgmap v1031: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 359 KiB/s wr, 97 op/s 2024-07-31T15:10:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T15:10:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:51.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c836 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c9e2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:51.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:51 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: pgmap v1031: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 359 KiB/s wr, 97 op/s 2024-07-31T15:10:51.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T15:10:51.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:51.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:51.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:51.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:51.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:51.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:51.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c836 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c9e2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:51.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:51.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:51.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:51.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:51.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:51 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:10:51.843 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/sit.o 2024-07-31T15:10:52.023 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/addrconf_core.o 2024-07-31T15:10:52.373 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/9p/protocol.o 2024-07-31T15:10:52.590 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:10:52.595 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/phy/mdio_bus.o 2024-07-31T15:10:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:10:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000008596 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/scsi) see mds.b log and `damage ls` output for details 2024-07-31T15:10:52.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:10:52.694 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/dns_resolver/dns_query.o 2024-07-31T15:10:52.702 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.702 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.702 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.702 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.702 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.702 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.702 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:52.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:10:52.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000008596 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/scsi) see mds.b log and `damage ls` output for details 2024-07-31T15:10:52.704 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:52 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:10:52.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:52.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:52.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:10:52.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000008596 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/scsi) see mds.b log and `damage ls` output for details 2024-07-31T15:10:52.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:52 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:10:52.825 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/netfilter.o 2024-07-31T15:10:52.876 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/aes_gmac.o 2024-07-31T15:10:53.096 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_sseu_debugfs.o 2024-07-31T15:10:53.142 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:52 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:52.702+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000be32 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module/ auth{2=1} v2029 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0) n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1) old_inodes=1 (inest sync->mix g=2) caps={15489=pAsLsXs/-@13} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce826b7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000be32:[<0x1000000bd0a/module v1134>,<0x10000000202/kernel v5648>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000be32:[<0x1000000bd0a/module v2029>,<0x10000000202/kernel v21206>,<0x10000000201/linux-6.5.11 v2540>,<0x10000000006/t v1586>,<0x10000000005/tmp v1513>,<0x100000001fb/client.0 v1488>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1451>,<0x10000000001/sv_1 v1452>,<0x10000000000/qa v1442>,<0x1/volumes v1395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:07.168188+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:53.509 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/9p/trans_common.o 2024-07-31T15:10:53.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:53 smithi073 ceph-mon[36714]: pgmap v1032: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 488 KiB/s wr, 177 op/s 2024-07-31T15:10:53.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:53 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:10:53.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:53 smithi077 ceph-mon[37327]: pgmap v1032: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 488 KiB/s wr, 177 op/s 2024-07-31T15:10:53.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:53 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:10:53.782 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/dns_resolver/built-in.a 2024-07-31T15:10:53.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:53 smithi067 ceph-mon[32750]: pgmap v1032: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 488 KiB/s wr, 177 op/s 2024-07-31T15:10:53.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:53 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:10:53.826 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/exthdrs_core.o 2024-07-31T15:10:53.878 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/handshake/genl.o 2024-07-31T15:10:54.088 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/fils_aead.o 2024-07-31T15:10:54.104 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/devres.o 2024-07-31T15:10:54.116 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/tcp_cubic.o 2024-07-31T15:10:54.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:53 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:53.755+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d187 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi/ auth{1=1} v3040 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0) n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1) old_inodes=1 (inest sync->mix g=1) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7c6c1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d187:[<0x1000000d116/ctxfi v1858>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d187:[<0x1000000d116/ctxfi v3040>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2544>,<0x10000000006/t v1590>,<0x10000000005/tmp v1517>,<0x100000001fb/client.0 v1492>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1455>,<0x10000000001/sv_1 v1456>,<0x10000000000/qa v1446>,<0x1/volumes v1400>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.452342+0000 30=30+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:54.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:53 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:53.765+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d119 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97/ auth{2=1} v3208 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0) n(v3 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1) old_inodes=3 (inest sync->mix g=2) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce9047e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d119:[<0x1000000d116/ac97 v22>,<0x1000000cf1b/pci v463>,<0x10000000202/sound v8728>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000d119:[<0x1000000d116/ac97 v3208>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2544>,<0x10000000006/t v1590>,<0x10000000005/tmp v1517>,<0x100000001fb/client.0 v1492>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1455>,<0x10000000001/sv_1 v1456>,<0x10000000000/qa v1446>,<0x1/volumes v1400>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:00.491085+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:54.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:53 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:53.770+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d1cc [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1/ auth{2=1} v3042 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0) n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1) old_inodes=1 (inest sync->mix g=2) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8f176c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v1862>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v3042>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2544>,<0x10000000006/t v1590>,<0x10000000005/tmp v1517>,<0x100000001fb/client.0 v1492>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1455>,<0x10000000001/sv_1 v1456>,<0x10000000000/qa v1446>,<0x1/volumes v1400>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.549340+0000 22=22+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:54.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:53 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:53.795+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d180 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio/ auth{1=1} v3038 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0) n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1) old_inodes=1 (inest sync->mix g=1) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8ebd2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v1860>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v3038>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2544>,<0x10000000006/t v1590>,<0x10000000005/tmp v1517>,<0x100000001fb/client.0 v1492>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1455>,<0x10000000001/sv_1 v1456>,<0x10000000000/qa v1446>,<0x1/volumes v1400>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.414343+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:54.299 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/9p/trans_fd.o 2024-07-31T15:10:54.616 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_timeline.o 2024-07-31T15:10:54.643 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:54 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:54.187+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000cde2 [...10,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders/ auth{4=1} v112 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0) n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1) old_inodes=2 (inest sync->mix g=4) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8dc52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v29>,<0x1000000cbec/selinux v646>,<0x10000000202/scripts v8820>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v112>,<0x1000000cbec/selinux v749>,<0x10000000202/scripts v11639>,<0x10000000201/linux-6.5.11 v2546>,<0x10000000006/t v1592>,<0x10000000005/tmp v1519>,<0x100000001fb/client.0 v1494>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1457>,<0x10000000001/sv_1 v1458>,<0x10000000000/qa v1448>,<0x1/volumes v1402>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:58.999116+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:54.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:54 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:54 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:54 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:54 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:54 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:54.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:54 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.676 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/phy/mdio_device.o 2024-07-31T15:10:54.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:54 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:54 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:54 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:54 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:54 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:54.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:54 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:54 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:54 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:54 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:54 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:54 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:54.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:54 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:10:54.818 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/handshake/netlink.o 2024-07-31T15:10:55.059 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/phy/swphy.o 2024-07-31T15:10:55.130 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/ip6_checksum.o 2024-07-31T15:10:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.344+0000 7f90f0890640 1 -- 172.21.15.67:0/3819413263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90ec10c100 msgr2=0x7f90ec10c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.344+0000 7f90f0890640 1 --2- 172.21.15.67:0/3819413263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90ec10c100 0x7f90ec10c510 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009ca0 tx=0x7f90e402a8d0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.346+0000 7f90f0890640 1 -- 172.21.15.67:0/3819413263 shutdown_connections 2024-07-31T15:10:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.346+0000 7f90f0890640 1 --2- 172.21.15.67:0/3819413263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90ec10ca50 0x7f90ec07b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.346+0000 7f90f0890640 1 --2- 172.21.15.67:0/3819413263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90ec10c100 0x7f90ec10c510 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.346+0000 7f90f0890640 1 --2- 172.21.15.67:0/3819413263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90ec10a080 0x7f90ec10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.346+0000 7f90f0890640 1 -- 172.21.15.67:0/3819413263 >> 172.21.15.67:0/3819413263 conn(0x7f90ec06d380 msgr2=0x7f90ec06d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:55.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.354+0000 7f90f0890640 1 -- 172.21.15.67:0/3819413263 shutdown_connections 2024-07-31T15:10:55.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.354+0000 7f90f0890640 1 -- 172.21.15.67:0/3819413263 wait complete. 2024-07-31T15:10:55.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.355+0000 7f90f0890640 1 Processor -- start 2024-07-31T15:10:55.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.356+0000 7f90f0890640 1 -- start start 2024-07-31T15:10:55.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.356+0000 7f90f0890640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90ec10a080 0x7f90ec075620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.356+0000 7f90f0890640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90ec10c100 0x7f90ec075b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.356+0000 7f90f0890640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90ec10ca50 0x7f90ec0760a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.356+0000 7f90f0890640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec07d600 con 0x7f90ec10c100 2024-07-31T15:10:55.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.356+0000 7f90f0890640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec07d480 con 0x7f90ec10a080 2024-07-31T15:10:55.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.356+0000 7f90f0890640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec07d780 con 0x7f90ec10ca50 2024-07-31T15:10:55.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.358+0000 7f90ea59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90ec10a080 0x7f90ec075620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.358+0000 7f90ead9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90ec10ca50 0x7f90ec0760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.358+0000 7f90ea59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90ec10a080 0x7f90ec075620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59772/0 (socket says 172.21.15.67:59772) 2024-07-31T15:10:55.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.358+0000 7f90ea59c640 1 -- 172.21.15.67:0/2496829619 learned_addr learned my addr 172.21.15.67:0/2496829619 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:55.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.360+0000 7f90e9d9b640 1 --2- 172.21.15.67:0/2496829619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90ec10c100 0x7f90ec075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.361+0000 7f90e9d9b640 1 -- 172.21.15.67:0/2496829619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90ec10ca50 msgr2=0x7f90ec0760a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:55.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.361+0000 7f90e9d9b640 1 --2- 172.21.15.67:0/2496829619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90ec10ca50 0x7f90ec0760a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.361+0000 7f90e9d9b640 1 -- 172.21.15.67:0/2496829619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90ec10a080 msgr2=0x7f90ec075620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:55.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.361+0000 7f90e9d9b640 1 --2- 172.21.15.67:0/2496829619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90ec10a080 0x7f90ec075620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.361+0000 7f90e9d9b640 1 -- 172.21.15.67:0/2496829619 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ec1386c0 con 0x7f90ec10c100 2024-07-31T15:10:55.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.361+0000 7f90e9d9b640 1 --2- 172.21.15.67:0/2496829619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90ec10c100 0x7f90ec075b60 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f90e402aec0 tx=0x7f90e4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:55.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.366+0000 7f90db7fe640 1 -- 172.21.15.67:0/2496829619 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4004240 con 0x7f90ec10c100 2024-07-31T15:10:55.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.366+0000 7f90f0890640 1 -- 172.21.15.67:0/2496829619 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ec138950 con 0x7f90ec10c100 2024-07-31T15:10:55.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.366+0000 7f90f0890640 1 -- 172.21.15.67:0/2496829619 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ec138e60 con 0x7f90ec10c100 2024-07-31T15:10:55.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.386+0000 7f90f6de1740 1 Processor -- start 2024-07-31T15:10:55.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.388+0000 7f90db7fe640 1 -- 172.21.15.67:0/2496829619 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e40305d0 con 0x7f90ec10c100 2024-07-31T15:10:55.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.388+0000 7f90db7fe640 1 -- 172.21.15.67:0/2496829619 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e403c670 con 0x7f90ec10c100 2024-07-31T15:10:55.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.388+0000 7f90f6de1740 1 -- start start 2024-07-31T15:10:55.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.393+0000 7f90db7fe640 1 -- 172.21.15.67:0/2496829619 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f90e4031ab0 con 0x7f90ec10c100 2024-07-31T15:10:55.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.394+0000 7f90db7fe640 1 --2- 172.21.15.67:0/2496829619 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90d40b3f40 0x7f90d40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.395+0000 7f90f6de1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e05f070 0x555e1e404800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.395+0000 7f90f6de1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e0361b0 0x555e1e404d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.395+0000 7f90f6de1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 0x555e1e405280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.395+0000 7f90f6de1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555e1e37ab60 con 0x555e1e0361b0 2024-07-31T15:10:55.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.395+0000 7f90f6de1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555e1e36a060 con 0x555e1e03e390 2024-07-31T15:10:55.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.395+0000 7f90f6de1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e1e36e0a0 con 0x555e1e05f070 2024-07-31T15:10:55.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.396+0000 7f90e9d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e0361b0 0x555e1e404d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.396+0000 7f90e9d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e0361b0 0x555e1e404d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43856/0 (socket says 172.21.15.67:43856) 2024-07-31T15:10:55.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.396+0000 7f90e9d9b640 1 -- 172.21.15.67:0/3443504567 learned_addr learned my addr 172.21.15.67:0/3443504567 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:55.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.398+0000 7f90ead9d640 1 --2- 172.21.15.67:0/3443504567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 0x555e1e405280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.398+0000 7f90ead9d640 1 --2- 172.21.15.67:0/2496829619 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90d40b3f40 0x7f90d40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.398+0000 7f90ea59c640 1 --2- 172.21.15.67:0/3443504567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e05f070 0x555e1e404800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.399+0000 7f90ead9d640 1 -- 172.21.15.67:0/3443504567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e05f070 msgr2=0x555e1e404800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:55.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.399+0000 7f90ead9d640 1 --2- 172.21.15.67:0/3443504567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e05f070 0x555e1e404800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.399+0000 7f90ead9d640 1 -- 172.21.15.67:0/3443504567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e0361b0 msgr2=0x555e1e404d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:55.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.399+0000 7f90ead9d640 1 --2- 172.21.15.67:0/3443504567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e0361b0 0x555e1e404d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.399+0000 7f90ead9d640 1 -- 172.21.15.67:0/3443504567 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e1e310b40 con 0x555e1e03e390 2024-07-31T15:10:55.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.399+0000 7f90ead9d640 1 --2- 172.21.15.67:0/3443504567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 0x555e1e405280 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f90e0012570 tx=0x7f90e0012a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c5973a298084543 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:55.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.400+0000 7f90db7fe640 1 -- 172.21.15.67:0/2496829619 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f90e4106a50 con 0x7f90ec10c100 2024-07-31T15:10:55.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.400+0000 7f90d8ff9640 1 -- 172.21.15.67:0/3443504567 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e00109b0 con 0x555e1e03e390 2024-07-31T15:10:55.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.400+0000 7f90d8ff9640 1 -- 172.21.15.67:0/3443504567 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e001d400 con 0x555e1e03e390 2024-07-31T15:10:55.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.401+0000 7f90f6de1740 1 -- 172.21.15.67:0/3443504567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 msgr2=0x555e1e405280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:55.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.401+0000 7f90f6de1740 1 --2- 172.21.15.67:0/3443504567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 0x555e1e405280 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f90e0012570 tx=0x7f90e0012a30 comp rx=0 tx=0).stop 2024-07-31T15:10:55.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.402+0000 7f90ead9d640 1 --2- 172.21.15.67:0/2496829619 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90d40b3f40 0x7f90d40b6450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f90e0020000 tx=0x7f90e001c8c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:55.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.402+0000 7f90f6de1740 1 -- 172.21.15.67:0/3443504567 shutdown_connections 2024-07-31T15:10:55.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.402+0000 7f90f6de1740 1 --2- 172.21.15.67:0/3443504567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 0x555e1e405280 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.402+0000 7f90f6de1740 1 --2- 172.21.15.67:0/3443504567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e0361b0 0x555e1e404d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.402+0000 7f90f6de1740 1 --2- 172.21.15.67:0/3443504567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e05f070 0x555e1e404800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.402+0000 7f90f6de1740 1 -- 172.21.15.67:0/3443504567 >> 172.21.15.67:0/3443504567 conn(0x555e1e053460 msgr2=0x555e1e4026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:55.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.405+0000 7f90f6de1740 1 -- 172.21.15.67:0/3443504567 shutdown_connections 2024-07-31T15:10:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.405+0000 7f90f6de1740 1 -- 172.21.15.67:0/3443504567 wait complete. 2024-07-31T15:10:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.406+0000 7f90f6de1740 1 -- start start 2024-07-31T15:10:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.406+0000 7f90f6de1740 1 Processor -- start 2024-07-31T15:10:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.407+0000 7f90f6de1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e0361b0 0x555e1e4104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.407+0000 7f90f6de1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e03e390 0x555e1e410a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.407+0000 7f90f6de1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e05f070 0x555e1e41c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.407+0000 7f90f6de1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555e1e1cd700 con 0x555e1e0361b0 2024-07-31T15:10:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.407+0000 7f90f6de1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555e1e30c990 con 0x555e1e05f070 2024-07-31T15:10:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.407+0000 7f90f6de1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e1e413030 con 0x555e1e03e390 2024-07-31T15:10:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.407+0000 7f90ead9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e05f070 0x555e1e41c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.407+0000 7f90ead9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e05f070 0x555e1e41c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59788/0 (socket says 172.21.15.67:59788) 2024-07-31T15:10:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.407+0000 7f90ead9d640 1 -- 172.21.15.67:0/3555402695 learned_addr learned my addr 172.21.15.67:0/3555402695 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.407+0000 7f90f6de1740 10 client.0 _ll_register_callbacks cb 0x555e1e3274a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.407+0000 7f90ead9d640 1 -- 172.21.15.67:0/3555402695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e03e390 msgr2=0x555e1e410a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:10:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.408+0000 7f90ead9d640 1 --2- 172.21.15.67:0/3555402695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e03e390 0x555e1e410a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.408+0000 7f90ead9d640 1 -- 172.21.15.67:0/3555402695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e0361b0 msgr2=0x555e1e4104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.408+0000 7f90ead9d640 1 --2- 172.21.15.67:0/3555402695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e0361b0 0x555e1e4104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.408+0000 7f90ead9d640 1 -- 172.21.15.67:0/3555402695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e1e0ca650 con 0x555e1e05f070 2024-07-31T15:10:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.408+0000 7f90ead9d640 1 --2- 172.21.15.67:0/3555402695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e05f070 0x555e1e41c400 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f90e001d000 tx=0x7f90e000b600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3e684017631b2a8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.419+0000 7f90d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:10:55.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.419+0000 7f90d1ffb640 1 -- 172.21.15.67:0/3555402695 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0005450 con 0x555e1e05f070 2024-07-31T15:10:55.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.420+0000 7f90f6de1740 1 -- 172.21.15.67:0/3555402695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e1e37cab0 con 0x555e1e05f070 2024-07-31T15:10:55.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.420+0000 7f90d1ffb640 1 -- 172.21.15.67:0/3555402695 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f90e00055f0 con 0x555e1e05f070 2024-07-31T15:10:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.430+0000 7f90d1ffb640 1 -- 172.21.15.67:0/3555402695 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f90e0040050 con 0x555e1e05f070 2024-07-31T15:10:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.430+0000 7f90f6de1740 10 client.29225 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.430+0000 7f90f6de1740 1 -- 172.21.15.67:0/3555402695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x555e1e41f230 con 0x555e1e05f070 2024-07-31T15:10:55.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.434+0000 7f90d1ffb640 1 -- 172.21.15.67:0/3555402695 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555e1e37cab0 con 0x555e1e05f070 2024-07-31T15:10:55.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.434+0000 7f90d1ffb640 1 -- 172.21.15.67:0/3555402695 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f90e0010980 con 0x555e1e05f070 2024-07-31T15:10:55.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.434+0000 7f90f6de1740 10 client.29225 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:55.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.434+0000 7f90f6de1740 10 client.29225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:55.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.434+0000 7f90f6de1740 20 client.29225 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:55.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.434+0000 7f90f6de1740 1 --2- 172.21.15.67:0/3555402695 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e1e01d9f0 0x555e1e422c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.434+0000 7f90f6de1740 4 client.29225 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:10:55.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.434+0000 7f90f6de1740 1 -- 172.21.15.67:0/3555402695 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e1e378550 con 0x555e1e01d9f0 2024-07-31T15:10:55.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.434+0000 7f90ea59c640 1 --2- 172.21.15.67:0/3555402695 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e1e01d9f0 0x555e1e422c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.435+0000 7f90d1ffb640 10 client.29225 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:55.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.436+0000 7f90ea59c640 1 --2- 172.21.15.67:0/3555402695 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e1e01d9f0 0x555e1e422c40 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f61c42e79a7d8f7 server_cookie=756693bd6d426df2 in_seq=0 out_seq=0 2024-07-31T15:10:55.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.439+0000 7f90d1ffb640 1 -- 172.21.15.67:0/3555402695 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e1e378550 con 0x555e1e01d9f0 2024-07-31T15:10:55.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.439+0000 7f90d1ffb640 10 client.29225 handle_command_reply: tid=0 2024-07-31T15:10:55.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.439+0000 7f90f6de1740 1 client.29225 shutdown 2024-07-31T15:10:55.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.439+0000 7f90f6de1740 1 -- 172.21.15.67:0/3555402695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e05f070 msgr2=0x555e1e41c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:55.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.439+0000 7f90f6de1740 1 --2- 172.21.15.67:0/3555402695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e05f070 0x555e1e41c400 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f90e001d000 tx=0x7f90e000b600 comp rx=0 tx=0).stop 2024-07-31T15:10:55.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.439+0000 7f90f6de1740 1 -- 172.21.15.67:0/3555402695 shutdown_connections 2024-07-31T15:10:55.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.439+0000 7f90f6de1740 1 --2- 172.21.15.67:0/3555402695 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e1e01d9f0 0x555e1e422c40 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.439+0000 7f90f6de1740 1 --2- 172.21.15.67:0/3555402695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e05f070 0x555e1e41c400 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.439+0000 7f90f6de1740 1 --2- 172.21.15.67:0/3555402695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e03e390 0x555e1e410a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.439+0000 7f90f6de1740 1 --2- 172.21.15.67:0/3555402695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e0361b0 0x555e1e4104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.439+0000 7f90f6de1740 1 -- 172.21.15.67:0/3555402695 >> 172.21.15.67:0/3555402695 conn(0x555e1e053460 msgr2=0x555e1e407990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:55.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.439+0000 7f90d1ffb640 0 client.29225 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:55.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.443+0000 7f90f6de1740 1 -- 172.21.15.67:0/3555402695 shutdown_connections 2024-07-31T15:10:55.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.443+0000 7f90f6de1740 1 -- 172.21.15.67:0/3555402695 wait complete. 2024-07-31T15:10:55.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.443+0000 7f90f6de1740 20 client.29225 trim_cache size 0 max 16384 2024-07-31T15:10:55.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.494+0000 7f90f6de1740 1 Processor -- start 2024-07-31T15:10:55.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.494+0000 7f90f6de1740 1 -- start start 2024-07-31T15:10:55.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.494+0000 7f90f6de1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e0361b0 0x555e1e40a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.494+0000 7f90f6de1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 0x555e1e41c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.495+0000 7f90ea59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e0361b0 0x555e1e40a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.495+0000 7f90ea59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e0361b0 0x555e1e40a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50408/0 (socket says 172.21.15.67:50408) 2024-07-31T15:10:55.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.495+0000 7f90ea59c640 1 -- 172.21.15.67:0/1287045356 learned_addr learned my addr 172.21.15.67:0/1287045356 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:55.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.495+0000 7f90e9d9b640 1 --2- 172.21.15.67:0/1287045356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 0x555e1e41c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.495+0000 7f90f6de1740 1 --2- 172.21.15.67:0/1287045356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e05f070 0x555e1e428440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.495+0000 7f90f6de1740 1 -- 172.21.15.67:0/1287045356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc00fea0 con 0x555e1e05f070 2024-07-31T15:10:55.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.495+0000 7f90f6de1740 1 -- 172.21.15.67:0/1287045356 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc0119e0 con 0x555e1e03e390 2024-07-31T15:10:55.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.495+0000 7f90f6de1740 1 -- 172.21.15.67:0/1287045356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc010570 con 0x555e1e0361b0 2024-07-31T15:10:55.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.496+0000 7f90ead9d640 1 --2- 172.21.15.67:0/1287045356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e05f070 0x555e1e428440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.496+0000 7f90ea59c640 1 -- 172.21.15.67:0/1287045356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 msgr2=0x555e1e41c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:55.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.496+0000 7f90ea59c640 1 --2- 172.21.15.67:0/1287045356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 0x555e1e41c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.496+0000 7f90ea59c640 1 -- 172.21.15.67:0/1287045356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e05f070 msgr2=0x555e1e428440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:55.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.496+0000 7f90ea59c640 1 --2- 172.21.15.67:0/1287045356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e05f070 0x555e1e428440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.496+0000 7f90ea59c640 1 -- 172.21.15.67:0/1287045356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e1e40f250 con 0x555e1e0361b0 2024-07-31T15:10:55.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.497+0000 7f90ea59c640 1 --2- 172.21.15.67:0/1287045356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e0361b0 0x555e1e40a020 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f90dc000c00 tx=0x7f90dc010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f32c97841c7023ba server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.500+0000 7f90d3fff640 1 -- 172.21.15.67:0/1287045356 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc022070 con 0x555e1e0361b0 2024-07-31T15:10:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.500+0000 7f90d3fff640 1 -- 172.21.15.67:0/1287045356 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90dc01cb00 con 0x555e1e0361b0 2024-07-31T15:10:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.500+0000 7f90d3fff640 1 -- 172.21.15.67:0/1287045356 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc01bb50 con 0x555e1e0361b0 2024-07-31T15:10:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.502+0000 7f90f6de1740 1 -- 172.21.15.67:0/1287045356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e0361b0 msgr2=0x555e1e40a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.502+0000 7f90f6de1740 1 --2- 172.21.15.67:0/1287045356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e0361b0 0x555e1e40a020 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f90dc000c00 tx=0x7f90dc010800 comp rx=0 tx=0).stop 2024-07-31T15:10:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.507+0000 7f90f6de1740 1 -- 172.21.15.67:0/1287045356 shutdown_connections 2024-07-31T15:10:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.507+0000 7f90f6de1740 1 --2- 172.21.15.67:0/1287045356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e05f070 0x555e1e428440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.507+0000 7f90f6de1740 1 --2- 172.21.15.67:0/1287045356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 0x555e1e41c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.507+0000 7f90f6de1740 1 --2- 172.21.15.67:0/1287045356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e0361b0 0x555e1e40a020 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.507+0000 7f90f6de1740 1 -- 172.21.15.67:0/1287045356 >> 172.21.15.67:0/1287045356 conn(0x555e1e053460 msgr2=0x555e1e409910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.512+0000 7f90f6de1740 1 -- 172.21.15.67:0/1287045356 shutdown_connections 2024-07-31T15:10:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.512+0000 7f90f6de1740 1 -- 172.21.15.67:0/1287045356 wait complete. 2024-07-31T15:10:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.512+0000 7f90f6de1740 1 -- start start 2024-07-31T15:10:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.513+0000 7f90f6de1740 1 Processor -- start 2024-07-31T15:10:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.513+0000 7f90f6de1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e0361b0 0x555e1e413320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.514+0000 7f90f6de1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 0x555e1e413860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.514+0000 7f90ea59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e0361b0 0x555e1e413320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.514+0000 7f90ea59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e0361b0 0x555e1e413320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50416/0 (socket says 172.21.15.67:50416) 2024-07-31T15:10:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.514+0000 7f90ea59c640 1 -- 172.21.15.67:0/1359755032 learned_addr learned my addr 172.21.15.67:0/1359755032 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.514+0000 7f90e9d9b640 1 --2- 172.21.15.67:0/1359755032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 0x555e1e413860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.514+0000 7f90f6de1740 1 --2- 172.21.15.67:0/1359755032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e05f070 0x555e1e413da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.515+0000 7f90f6de1740 1 -- 172.21.15.67:0/1359755032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc011310 con 0x555e1e05f070 2024-07-31T15:10:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.515+0000 7f90f6de1740 1 -- 172.21.15.67:0/1359755032 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc010c40 con 0x555e1e03e390 2024-07-31T15:10:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.515+0000 7f90f6de1740 1 -- 172.21.15.67:0/1359755032 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc00de80 con 0x555e1e0361b0 2024-07-31T15:10:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.515+0000 7f90f6de1740 10 client.0 _ll_register_callbacks cb 0x7f90e410c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.515+0000 7f90ea59c640 1 -- 172.21.15.67:0/1359755032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 msgr2=0x555e1e413860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.515+0000 7f90ea59c640 1 --2- 172.21.15.67:0/1359755032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 0x555e1e413860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.515+0000 7f90ea59c640 1 -- 172.21.15.67:0/1359755032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e05f070 msgr2=0x555e1e413da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.515+0000 7f90ea59c640 1 --2- 172.21.15.67:0/1359755032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e05f070 0x555e1e413da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.515+0000 7f90ea59c640 1 -- 172.21.15.67:0/1359755032 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e1e489400 con 0x555e1e0361b0 2024-07-31T15:10:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.516+0000 7f90ea59c640 1 --2- 172.21.15.67:0/1359755032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e0361b0 0x555e1e413320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f90dc006010 tx=0x7f90dc01b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=805941494637894c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.516+0000 7f90e9d9b640 1 --2- 172.21.15.67:0/1359755032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 0x555e1e413860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:10:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.516+0000 7f90f6de1740 1 -- 172.21.15.67:0/1359755032 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90dc01cb00 con 0x555e1e0361b0 2024-07-31T15:10:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.517+0000 7f90d1ffb640 10 client.29101 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:10:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.517+0000 7f90d1ffb640 1 -- 172.21.15.67:0/1359755032 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc02d060 con 0x555e1e0361b0 2024-07-31T15:10:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.517+0000 7f90d1ffb640 1 -- 172.21.15.67:0/1359755032 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f90dc004440 con 0x555e1e0361b0 2024-07-31T15:10:55.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.517+0000 7f90d1ffb640 1 -- 172.21.15.67:0/1359755032 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f90dc034050 con 0x555e1e0361b0 2024-07-31T15:10:55.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.519+0000 7f90f6de1740 10 client.29101 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:55.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.519+0000 7f90f6de1740 1 -- 172.21.15.67:0/1359755032 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=66}) -- 0x555e1e48a530 con 0x555e1e0361b0 2024-07-31T15:10:55.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.524+0000 7f90d1ffb640 1 -- 172.21.15.67:0/1359755032 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f90dc01c450 con 0x555e1e0361b0 2024-07-31T15:10:55.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.524+0000 7f90f6de1740 10 client.29101 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:55.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.524+0000 7f90f6de1740 10 client.29101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:55.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.524+0000 7f90f6de1740 20 client.29101 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:55.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.524+0000 7f90f6de1740 1 --2- 172.21.15.67:0/1359755032 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e1dff2da0 0x555e1e48c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:55.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.524+0000 7f90f6de1740 4 client.29101 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:10:55.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.524+0000 7f90f6de1740 1 -- 172.21.15.67:0/1359755032 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f90dc01cb00 con 0x555e1dff2da0 2024-07-31T15:10:55.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.524+0000 7f90e9d9b640 1 --2- 172.21.15.67:0/1359755032 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e1dff2da0 0x555e1e48c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:55.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.531+0000 7f90d1ffb640 10 client.29101 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:55.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.531+0000 7f90e9d9b640 1 --2- 172.21.15.67:0/1359755032 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e1dff2da0 0x555e1e48c990 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f68d7aa7aac61b8 server_cookie=662da3bde3fcfe54 in_seq=0 out_seq=0 2024-07-31T15:10:55.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.532+0000 7f90d1ffb640 1 -- 172.21.15.67:0/1359755032 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x7f90dc01cb00 con 0x555e1dff2da0 2024-07-31T15:10:55.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.532+0000 7f90d1ffb640 10 client.29101 handle_command_reply: tid=0 2024-07-31T15:10:55.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.535+0000 7f90f6de1740 1 client.29101 shutdown 2024-07-31T15:10:55.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.541+0000 7f90f6de1740 1 -- 172.21.15.67:0/1359755032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e0361b0 msgr2=0x555e1e413320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:55.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.541+0000 7f90f6de1740 1 --2- 172.21.15.67:0/1359755032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e0361b0 0x555e1e413320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f90dc006010 tx=0x7f90dc01b5e0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.545+0000 7f90f6de1740 1 -- 172.21.15.67:0/1359755032 shutdown_connections 2024-07-31T15:10:55.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.545+0000 7f90f6de1740 1 --2- 172.21.15.67:0/1359755032 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e1dff2da0 0x555e1e48c990 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.545+0000 7f90f6de1740 1 --2- 172.21.15.67:0/1359755032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e1e05f070 0x555e1e413da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.545+0000 7f90f6de1740 1 --2- 172.21.15.67:0/1359755032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e1e03e390 0x555e1e413860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.545+0000 7f90f6de1740 1 --2- 172.21.15.67:0/1359755032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e1e0361b0 0x555e1e413320 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.545+0000 7f90f6de1740 1 -- 172.21.15.67:0/1359755032 >> 172.21.15.67:0/1359755032 conn(0x555e1e053460 msgr2=0x555e1e405830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:55.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.545+0000 7f90d1ffb640 0 client.29101 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:55.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.546+0000 7f90f6de1740 1 -- 172.21.15.67:0/1359755032 shutdown_connections 2024-07-31T15:10:55.552 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:10:55.552 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (16240 inodes in the stack)", 2024-07-31T15:10:55.552 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T15:10:55.552 INFO:teuthology.orchestra.run.smithi067.stdout: "c05a848d-c8e4-481b-ad13-32de04ecf834": { 2024-07-31T15:10:55.552 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T15:10:55.552 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "c05a848d-c8e4-481b-ad13-32de04ecf834", 2024-07-31T15:10:55.552 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T15:10:55.552 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:10:55.552 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:10:55.553 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:10:55.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.548+0000 7f90f6de1740 1 -- 172.21.15.67:0/1359755032 wait complete. 2024-07-31T15:10:55.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.549+0000 7f90f6de1740 20 client.29101 trim_cache size 0 max 16384 2024-07-31T15:10:55.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.550+0000 7f90f0890640 1 -- 172.21.15.67:0/2496829619 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90d40b3f40 msgr2=0x7f90d40b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:55.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.550+0000 7f90f0890640 1 --2- 172.21.15.67:0/2496829619 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90d40b3f40 0x7f90d40b6450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f90e0020000 tx=0x7f90e001c8c0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.550+0000 7f90f0890640 1 -- 172.21.15.67:0/2496829619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90ec10c100 msgr2=0x7f90ec075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:55.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.550+0000 7f90f0890640 1 --2- 172.21.15.67:0/2496829619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90ec10c100 0x7f90ec075b60 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f90e402aec0 tx=0x7f90e4003f10 comp rx=0 tx=0).stop 2024-07-31T15:10:55.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.551+0000 7f90f0890640 1 -- 172.21.15.67:0/2496829619 shutdown_connections 2024-07-31T15:10:55.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.551+0000 7f90f0890640 1 --2- 172.21.15.67:0/2496829619 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90d40b3f40 0x7f90d40b6450 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.551+0000 7f90f0890640 1 --2- 172.21.15.67:0/2496829619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90ec10ca50 0x7f90ec0760a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.551+0000 7f90f0890640 1 --2- 172.21.15.67:0/2496829619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90ec10c100 0x7f90ec075b60 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.551+0000 7f90f0890640 1 --2- 172.21.15.67:0/2496829619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90ec10a080 0x7f90ec075620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:55.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.551+0000 7f90f0890640 1 -- 172.21.15.67:0/2496829619 >> 172.21.15.67:0/2496829619 conn(0x7f90ec06d380 msgr2=0x7f90ec10e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:55.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.551+0000 7f90f0890640 1 -- 172.21.15.67:0/2496829619 shutdown_connections 2024-07-31T15:10:55.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:10:55.551+0000 7f90f0890640 1 -- 172.21.15.67:0/2496829619 wait complete. 2024-07-31T15:10:55.576 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/cfg.o 2024-07-31T15:10:55.620 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/cipso_ipv4.o 2024-07-31T15:10:55.640 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/phy/fixed_phy.o 2024-07-31T15:10:55.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:55.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: pgmap v1033: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 787 KiB/s wr, 388 op/s 2024-07-31T15:10:55.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:55.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:55.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:55.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:55.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:55.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:55.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:55.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:55 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:55.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: pgmap v1033: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 787 KiB/s wr, 388 op/s 2024-07-31T15:10:55.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:55.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:55.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:55.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:55.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:55.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:55.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:55.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:55 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:55.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: pgmap v1033: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 787 KiB/s wr, 388 op/s 2024-07-31T15:10:55.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:55.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:55.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:55.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:55.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:55.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:55.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:55.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:55.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:55 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.142 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/phy/realtek.o 2024-07-31T15:10:56.147 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_wopcm.o 2024-07-31T15:10:56.156 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/handshake/request.o 2024-07-31T15:10:56.275 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/9p/trans_virtio.o 2024-07-31T15:10:56.497 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/ip6_icmp.o 2024-07-31T15:10:56.690 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/output_core.o 2024-07-31T15:10:56.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:56.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:56.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:56.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:56.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:56.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:56.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:56.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:56 smithi077 ceph-mon[37327]: pgmap v1034: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 738 op/s 2024-07-31T15:10:56.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:56.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:56.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:56.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:56.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:56.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:56.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:56.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:56 smithi067 ceph-mon[32750]: pgmap v1034: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 738 op/s 2024-07-31T15:10:56.892 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:56.460+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000021f0 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader/ auth{2=1} v893 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:04.418950+0000 19=18+1) n(v5 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2) old_inodes=1 (inest sync->mix g=2) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7dc21b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000021f0:[<0x100000021db/firmware_loader v414>,<0x10000001d8f/base v2486>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x100000021f0:[<0x100000021db/firmware_loader v893>,<0x10000001d8f/base v18747>,<0x10000000202/drivers v21625>,<0x10000000201/linux-6.5.11 v2546>,<0x10000000006/t v1594>,<0x10000000005/tmp v1523>,<0x100000001fb/client.0 v1497>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1460>,<0x10000000001/sv_1 v1461>,<0x10000000000/qa v1451>,<0x1/volumes v1404>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:01.469061+0000 13=12+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:04.418950+0000 19=18+1)","memory_value.rstat":"n(v5 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:56.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:56.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:56.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:56 smithi073 ceph-mon[36714]: pgmap v1034: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 738 op/s 2024-07-31T15:10:57.061 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_workarounds.o 2024-07-31T15:10:57.335 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:57.066+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000008a08 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip/ auth{3=1} v2681 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:09.647065+0000 7=7+0) n(v2 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1) old_inodes=2 (inest sync->mix g=3) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8c1aa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000008a08:[<0x1000000894f/microchip v613>,<0x10000001d8f/soc v16560>,<0x10000000202/drivers v3182>,<0x10000000201/linux-6.5.11 v445>,<0x10000000006/t v329>,<0x10000000005/tmp v299>,<0x100000001fb/client.0 v296>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v274>,<0x10000000001/sv_1 v299>,<0x10000000000/qa v280>,<0x1/volumes v280>]//[]","memoryvalue":"(2)0x10000008a08:[<0x1000000894f/microchip v2681>,<0x10000001d8f/soc v17222>,<0x10000000202/drivers v21641>,<0x10000000201/linux-6.5.11 v2551>,<0x10000000006/t v1598>,<0x10000000005/tmp v1525>,<0x100000001fb/client.0 v1499>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1462>,<0x10000000001/sv_1 v1463>,<0x10000000000/qa v1453>,<0x1/volumes v1406>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:51.041772+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:09.647065+0000 7=7+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:57.376 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/phy/built-in.a 2024-07-31T15:10:57.494 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/handshake/tlshd.o 2024-07-31T15:10:57.495 INFO:tasks.cephfs.filesystem:scrub status for tag:c05a848d-c8e4-481b-ad13-32de04ecf834 - {'path': '/', 'tag': 'c05a848d-c8e4-481b-ad13-32de04ecf834', 'options': 'recursive,force'} 2024-07-31T15:10:57.517 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/mdio/acpi_mdio.o 2024-07-31T15:10:57.616 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:57.334+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001fc7 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica/ auth{2=1} fragtree_t(*^3) v1950 ap=2 RANDEPHEMERALPIN f(v10 m2024-07-31T15:07:39.153464+0000 516=516+0) n(v16 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1) old_inodes=4 (inest sync->mix g=2) caps={15489=pAsLsXs/-@28} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7f0ba680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000001fc7:[<0x10000001fad/acpica v1734>,<0x10000001d8f/acpi v2434>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x10000001fc7:[<0x10000001fad/acpica v1950>,<0x10000001d8f/acpi v18672>,<0x10000000202/drivers v21641>,<0x10000000201/linux-6.5.11 v2551>,<0x10000000006/t v1598>,<0x10000000005/tmp v1525>,<0x100000001fb/client.0 v1499>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1462>,<0x10000000001/sv_1 v1463>,<0x10000000000/qa v1453>,<0x1/volumes v1406>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:07:37.321501+0000 494=494+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1)","memory_value.dirstat":"f(v10 m2024-07-31T15:07:39.153464+0000 516=516+0)","memory_value.rstat":"n(v16 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:57.616 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:57.490+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009462 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc/ auth{3=1} v1424 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:28.223687+0000 22=22+0) n(v4 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1) old_inodes=2 (inest sync->mix g=3) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce85c96b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009462:[<0x1000000945c/hvc v352>,<0x10000001d8f/tty v18356>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009462:[<0x1000000945c/hvc v1424>,<0x10000001d8f/tty v18901>,<0x10000000202/drivers v21641>,<0x10000000201/linux-6.5.11 v2551>,<0x10000000006/t v1598>,<0x10000000005/tmp v1525>,<0x100000001fb/client.0 v1499>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1462>,<0x10000000001/sv_1 v1463>,<0x10000000000/qa v1453>,<0x1/volumes v1406>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:32.992899+0000 16=16+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:28.223687+0000 22=22+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:57.616 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.616 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.616 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.616 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.617 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.617 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.617 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.617 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.617 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.617 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.617 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.617 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.617 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:10:57.617 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.617 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.617 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.617 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.618 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.618 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.618 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.618 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.618 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.618 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.618 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.618 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.618 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.618 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.618 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.618 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.618 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.619 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.619 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:57.619 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.619 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:10:57.655 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/xfrm4_policy.o 2024-07-31T15:10:57.672 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/9p/built-in.a 2024-07-31T15:10:57.686 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/pcs/built-in.a 2024-07-31T15:10:57.711 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/mdio/fwnode_mdio.o 2024-07-31T15:10:57.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:10:57.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:57.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:57 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:10:57.783 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/3com/built-in.a 2024-07-31T15:10:57.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:10:57.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:57.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:57.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:57 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:10:57.811 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/protocol.o 2024-07-31T15:10:57.886 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/8390/built-in.a 2024-07-31T15:10:57.892 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:10:57 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:10:57.614+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005d80 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader/ auth{4=1} v1324 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:10.777820+0000 21=21+0) n(v3 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1) old_inodes=1 (inest sync->mix g=4) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7f8f5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000005d80:[<0x10000005d5d/cardreader v104>,<0x10000001d8f/misc v13245>,<0x10000000202/drivers v2910>,<0x10000000201/linux-6.5.11 v379>,<0x10000000006/t v275>,<0x10000000005/tmp v244>,<0x100000001fb/client.0 v243>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v221>,<0x10000000001/sv_1 v249>,<0x10000000000/qa v233>,<0x1/volumes v234>]//[]","memoryvalue":"(2)0x10000005d80:[<0x10000005d5d/cardreader v1324>,<0x10000001d8f/misc v18516>,<0x10000000202/drivers v21641>,<0x10000000201/linux-6.5.11 v2551>,<0x10000000006/t v1598>,<0x10000000005/tmp v1525>,<0x100000001fb/client.0 v1499>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1462>,<0x10000000001/sv_1 v1463>,<0x10000000000/qa v1453>,<0x1/volumes v1406>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:21.430390+0000 17=17+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:10.777820+0000 21=21+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:10:57.981 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/adaptec/built-in.a 2024-07-31T15:10:58.086 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/agere/built-in.a 2024-07-31T15:10:58.220 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/alacritech/built-in.a 2024-07-31T15:10:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.303+0000 7f4a1f59e640 1 -- 172.21.15.67:0/1702856696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a2010c0e0 msgr2=0x7f4a20114800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.303+0000 7f4a1f59e640 1 --2- 172.21.15.67:0/1702856696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a2010c0e0 0x7f4a20114800 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009900 tx=0x7f4a1402a4a0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.304+0000 7f4a1f59e640 1 -- 172.21.15.67:0/1702856696 shutdown_connections 2024-07-31T15:10:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.304+0000 7f4a1f59e640 1 --2- 172.21.15.67:0/1702856696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a20114d40 0x7f4a201171c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.304+0000 7f4a1f59e640 1 --2- 172.21.15.67:0/1702856696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a2010c0e0 0x7f4a20114800 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.304+0000 7f4a1f59e640 1 --2- 172.21.15.67:0/1702856696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a2010b790 0x7f4a2010bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.304+0000 7f4a1f59e640 1 -- 172.21.15.67:0/1702856696 >> 172.21.15.67:0/1702856696 conn(0x7f4a2006e320 msgr2=0x7f4a2006e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.304+0000 7f4a1f59e640 1 -- 172.21.15.67:0/1702856696 shutdown_connections 2024-07-31T15:10:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.305+0000 7f4a1f59e640 1 -- 172.21.15.67:0/1702856696 wait complete. 2024-07-31T15:10:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.306+0000 7f4a1f59e640 1 Processor -- start 2024-07-31T15:10:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.306+0000 7f4a1f59e640 1 -- start start 2024-07-31T15:10:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1f59e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a2010b790 0x7f4a2010b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1f59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a2010c0e0 0x7f4a201a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1f59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a20114d40 0x7f4a201a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1f59e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2011a9c0 con 0x7f4a2010b790 2024-07-31T15:10:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1f59e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2011a840 con 0x7f4a20114d40 2024-07-31T15:10:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1f59e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2011ab40 con 0x7f4a2010c0e0 2024-07-31T15:10:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a2010c0e0 0x7f4a201a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a2010c0e0 0x7f4a201a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50434/0 (socket says 172.21.15.67:50434) 2024-07-31T15:10:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1dd9b640 1 -- 172.21.15.67:0/2370750730 learned_addr learned my addr 172.21.15.67:0/2370750730 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1ed9d640 1 --2- 172.21.15.67:0/2370750730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a20114d40 0x7f4a201a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1dd9b640 1 -- 172.21.15.67:0/2370750730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a20114d40 msgr2=0x7f4a201a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1dd9b640 1 --2- 172.21.15.67:0/2370750730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a20114d40 0x7f4a201a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1dd9b640 1 -- 172.21.15.67:0/2370750730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a2010b790 msgr2=0x7f4a2010b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1dd9b640 1 --2- 172.21.15.67:0/2370750730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a2010b790 0x7f4a2010b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.307+0000 7f4a1dd9b640 1 -- 172.21.15.67:0/2370750730 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a201a1d40 con 0x7f4a2010c0e0 2024-07-31T15:10:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.308+0000 7f4a1dd9b640 1 --2- 172.21.15.67:0/2370750730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a2010c0e0 0x7f4a201a1080 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4a14006010 tx=0x7f4a14031ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:58.311 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/alteon/built-in.a 2024-07-31T15:10:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.318+0000 7f4a0f7fe640 1 -- 172.21.15.67:0/2370750730 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14040030 con 0x7f4a2010c0e0 2024-07-31T15:10:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.318+0000 7f4a0f7fe640 1 -- 172.21.15.67:0/2370750730 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a1403c8f0 con 0x7f4a2010c0e0 2024-07-31T15:10:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.319+0000 7f4a0f7fe640 1 -- 172.21.15.67:0/2370750730 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14005180 con 0x7f4a2010c0e0 2024-07-31T15:10:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.319+0000 7f4a1f59e640 1 -- 172.21.15.67:0/2370750730 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a201a60d0 con 0x7f4a2010c0e0 2024-07-31T15:10:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.319+0000 7f4a1f59e640 1 -- 172.21.15.67:0/2370750730 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a201a6610 con 0x7f4a2010c0e0 2024-07-31T15:10:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.323+0000 7f4a2bd6b740 1 Processor -- start 2024-07-31T15:10:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.326+0000 7f4a2bd6b740 1 -- start start 2024-07-31T15:10:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.326+0000 7f4a2bd6b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3aa01b0 0x5556a3e42ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.327+0000 7f4a2bd6b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a772f0 0x5556a3e433e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.327+0000 7f4a1ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3aa01b0 0x5556a3e42ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.327+0000 7f4a1ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3aa01b0 0x5556a3e42ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50440/0 (socket says 172.21.15.67:50440) 2024-07-31T15:10:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.327+0000 7f4a1ed9d640 1 -- 172.21.15.67:0/4055767051 learned_addr learned my addr 172.21.15.67:0/4055767051 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.327+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/4055767051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a7f4d0 0x5556a3e43920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.327+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/4055767051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5556a3db83b0 con 0x5556a3a772f0 2024-07-31T15:10:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.327+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/4055767051 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5556a3da78b0 con 0x5556a3a7f4d0 2024-07-31T15:10:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.327+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/4055767051 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5556a3dab8f0 con 0x5556a3aa01b0 2024-07-31T15:10:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.327+0000 7f4a1dd9b640 1 --2- 172.21.15.67:0/4055767051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a7f4d0 0x5556a3e43920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.328+0000 7f4a0f7fe640 1 -- 172.21.15.67:0/2370750730 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4a14043030 con 0x7f4a2010c0e0 2024-07-31T15:10:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.328+0000 7f4a1ed9d640 1 -- 172.21.15.67:0/4055767051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a7f4d0 msgr2=0x5556a3e43920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.328+0000 7f4a1ed9d640 1 --2- 172.21.15.67:0/4055767051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a7f4d0 0x5556a3e43920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.328+0000 7f4a1ed9d640 1 -- 172.21.15.67:0/4055767051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a772f0 msgr2=0x5556a3e433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.328+0000 7f4a1ed9d640 1 --2- 172.21.15.67:0/4055767051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a772f0 0x5556a3e433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.328+0000 7f4a1ed9d640 1 -- 172.21.15.67:0/4055767051 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556a3d4e9a0 con 0x5556a3aa01b0 2024-07-31T15:10:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.328+0000 7f4a1ed9d640 1 --2- 172.21.15.67:0/4055767051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3aa01b0 0x5556a3e42ea0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009cd0 tx=0x7f4a10011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e627bc34dccf02e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.330+0000 7f4a0f7fe640 1 --2- 172.21.15.67:0/2370750730 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4a080b3d50 0x7f4a080b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.331+0000 7f4a0cff9640 1 -- 172.21.15.67:0/4055767051 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1001e070 con 0x5556a3aa01b0 2024-07-31T15:10:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.331+0000 7f4a0f7fe640 1 -- 172.21.15.67:0/2370750730 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f4a14104d20 con 0x7f4a2010c0e0 2024-07-31T15:10:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.332+0000 7f4a0cff9640 1 -- 172.21.15.67:0/4055767051 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a10019ae0 con 0x5556a3aa01b0 2024-07-31T15:10:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.332+0000 7f4a1e59c640 1 --2- 172.21.15.67:0/2370750730 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4a080b3d50 0x7f4a080b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.334+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/4055767051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3aa01b0 msgr2=0x5556a3e42ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.334+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/4055767051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3aa01b0 0x5556a3e42ea0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009cd0 tx=0x7f4a10011810 comp rx=0 tx=0).stop 2024-07-31T15:10:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.334+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/4055767051 shutdown_connections 2024-07-31T15:10:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.334+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/4055767051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a7f4d0 0x5556a3e43920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.334+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/4055767051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a772f0 0x5556a3e433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.334+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/4055767051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3aa01b0 0x5556a3e42ea0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.334+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/4055767051 >> 172.21.15.67:0/4055767051 conn(0x5556a3a94300 msgr2=0x5556a3e40d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.337+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/4055767051 shutdown_connections 2024-07-31T15:10:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.337+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/4055767051 wait complete. 2024-07-31T15:10:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.337+0000 7f4a2bd6b740 1 -- start start 2024-07-31T15:10:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.337+0000 7f4a2bd6b740 1 Processor -- start 2024-07-31T15:10:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.338+0000 7f4a2bd6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a772f0 0x5556a3e4ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.338+0000 7f4a2bd6b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a7f4d0 0x5556a3e4ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.338+0000 7f4a2bd6b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3aa01b0 0x5556a3e5a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.338+0000 7f4a2bd6b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5556a3ce1ab0 con 0x5556a3a7f4d0 2024-07-31T15:10:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.338+0000 7f4a2bd6b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5556a3d2fcf0 con 0x5556a3a772f0 2024-07-31T15:10:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.338+0000 7f4a1ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3aa01b0 0x5556a3e5a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.338+0000 7f4a1ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3aa01b0 0x5556a3e5a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50444/0 (socket says 172.21.15.67:50444) 2024-07-31T15:10:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.338+0000 7f4a1ed9d640 1 -- 172.21.15.67:0/2755298737 learned_addr learned my addr 172.21.15.67:0/2755298737 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.338+0000 7f4a2bd6b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5556a3e51670 con 0x5556a3aa01b0 2024-07-31T15:10:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.339+0000 7f4a1dd9b640 1 --2- 172.21.15.67:0/2755298737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a7f4d0 0x5556a3e4ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.339+0000 7f4a1ed9d640 1 -- 172.21.15.67:0/2755298737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a772f0 msgr2=0x5556a3e4ea30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:10:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.339+0000 7f4a1ed9d640 1 --2- 172.21.15.67:0/2755298737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a772f0 0x5556a3e4ea30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.339+0000 7f4a1ed9d640 1 -- 172.21.15.67:0/2755298737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a7f4d0 msgr2=0x5556a3e4ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.339+0000 7f4a1ed9d640 1 --2- 172.21.15.67:0/2755298737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a7f4d0 0x5556a3e4ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.339+0000 7f4a1ed9d640 1 -- 172.21.15.67:0/2755298737 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556a3b0b790 con 0x5556a3aa01b0 2024-07-31T15:10:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.339+0000 7f4a1ed9d640 1 --2- 172.21.15.67:0/2755298737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3aa01b0 0x5556a3e5a9e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000e5e0 tx=0x7f4a1000e610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d91575b63b73d96b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.339+0000 7f4a2bd6b740 10 client.0 _ll_register_callbacks cb 0x5556a3da19f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.340+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2755298737 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5556a3dba300 con 0x5556a3aa01b0 2024-07-31T15:10:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.340+0000 7f4a05ffb640 10 client.29113 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:10:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.340+0000 7f4a05ffb640 1 -- 172.21.15.67:0/2755298737 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10027060 con 0x5556a3aa01b0 2024-07-31T15:10:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.340+0000 7f4a05ffb640 1 -- 172.21.15.67:0/2755298737 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f4a100052d0 con 0x5556a3aa01b0 2024-07-31T15:10:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.340+0000 7f4a05ffb640 1 -- 172.21.15.67:0/2755298737 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f4a1002e050 con 0x5556a3aa01b0 2024-07-31T15:10:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.341+0000 7f4a2bd6b740 10 client.29113 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.341+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2755298737 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=66}) -- 0x5556a3b84620 con 0x5556a3aa01b0 2024-07-31T15:10:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.341+0000 7f4a05ffb640 1 -- 172.21.15.67:0/2755298737 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f4a10021030 con 0x5556a3aa01b0 2024-07-31T15:10:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.341+0000 7f4a2bd6b740 10 client.29113 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.341+0000 7f4a2bd6b740 10 client.29113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.341+0000 7f4a2bd6b740 20 client.29113 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.341+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/2755298737 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5556a3a5ed80 0x5556a3e60cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.341+0000 7f4a2bd6b740 4 client.29113 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:10:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.341+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2755298737 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556a3dba300 con 0x5556a3a5ed80 2024-07-31T15:10:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.341+0000 7f4a1e59c640 1 --2- 172.21.15.67:0/2755298737 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5556a3a5ed80 0x5556a3e60cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.342+0000 7f4a1e59c640 1 --2- 172.21.15.67:0/2755298737 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5556a3a5ed80 0x5556a3e60cb0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63109d7e23666cb server_cookie=916b566f76446b70 in_seq=0 out_seq=0 2024-07-31T15:10:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.342+0000 7f4a05ffb640 10 client.29113 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.343+0000 7f4a05ffb640 1 -- 172.21.15.67:0/2755298737 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5556a3dba300 con 0x5556a3a5ed80 2024-07-31T15:10:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.343+0000 7f4a05ffb640 10 client.29113 handle_command_reply: tid=0 2024-07-31T15:10:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.343+0000 7f4a2bd6b740 1 client.29113 shutdown 2024-07-31T15:10:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.345+0000 7f4a1e59c640 1 --2- 172.21.15.67:0/2370750730 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4a080b3d50 0x7f4a080b6260 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x5556a3e51070 tx=0x7f4a18009250 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.345+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2755298737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3aa01b0 msgr2=0x5556a3e5a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.345+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/2755298737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3aa01b0 0x5556a3e5a9e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000e5e0 tx=0x7f4a1000e610 comp rx=0 tx=0).stop 2024-07-31T15:10:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.345+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2755298737 shutdown_connections 2024-07-31T15:10:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.345+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/2755298737 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5556a3a5ed80 0x5556a3e60cb0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.345+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/2755298737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3aa01b0 0x5556a3e5a9e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.345+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/2755298737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a7f4d0 0x5556a3e4ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.345+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/2755298737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a772f0 0x5556a3e4ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.345+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2755298737 >> 172.21.15.67:0/2755298737 conn(0x5556a3a94300 msgr2=0x5556a3e43ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.345+0000 7f4a05ffb640 0 client.29113 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.345+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2755298737 shutdown_connections 2024-07-31T15:10:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.346+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2755298737 wait complete. 2024-07-31T15:10:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.346+0000 7f4a2bd6b740 20 client.29113 trim_cache size 0 max 16384 2024-07-31T15:10:58.357 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/amazon/built-in.a 2024-07-31T15:10:58.392 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/amd/built-in.a 2024-07-31T15:10:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.401+0000 7f4a2bd6b740 1 Processor -- start 2024-07-31T15:10:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.401+0000 7f4a2bd6b740 1 -- start start 2024-07-31T15:10:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.401+0000 7f4a2bd6b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a772f0 0x5556a3e48fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.402+0000 7f4a2bd6b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3a7f4d0 0x5556a3e4eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.402+0000 7f4a2bd6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3aa01b0 0x5556a3e5a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.402+0000 7f4a2bd6b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201c7930 con 0x5556a3a772f0 2024-07-31T15:10:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.402+0000 7f4a1ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3aa01b0 0x5556a3e5a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.402+0000 7f4a1ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3aa01b0 0x5556a3e5a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59850/0 (socket says 172.21.15.67:59850) 2024-07-31T15:10:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.402+0000 7f4a1ed9d640 1 -- 172.21.15.67:0/3013440281 learned_addr learned my addr 172.21.15.67:0/3013440281 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.402+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/3013440281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201c9470 con 0x5556a3aa01b0 2024-07-31T15:10:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.402+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/3013440281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201c8000 con 0x5556a3a7f4d0 2024-07-31T15:10:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.403+0000 7f4a1ed9d640 1 -- 172.21.15.67:0/3013440281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3a7f4d0 msgr2=0x5556a3e4eea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:10:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.403+0000 7f4a1ed9d640 1 --2- 172.21.15.67:0/3013440281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3a7f4d0 0x5556a3e4eea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.403+0000 7f4a1ed9d640 1 -- 172.21.15.67:0/3013440281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a772f0 msgr2=0x5556a3e48fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:10:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.405+0000 7f4a1e59c640 1 --2- 172.21.15.67:0/3013440281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a772f0 0x5556a3e48fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.405+0000 7f4a1ed9d640 1 --2- 172.21.15.67:0/3013440281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a772f0 0x5556a3e48fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.406+0000 7f4a1ed9d640 1 -- 172.21.15.67:0/3013440281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556a3a5efb0 con 0x5556a3aa01b0 2024-07-31T15:10:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.406+0000 7f4a1ed9d640 1 --2- 172.21.15.67:0/3013440281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3aa01b0 0x5556a3e5a7d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4a10000c00 tx=0x7f4a10004000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40d175a36bbfb333 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.407+0000 7f4a07fff640 1 -- 172.21.15.67:0/3013440281 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1001e040 con 0x5556a3aa01b0 2024-07-31T15:10:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.407+0000 7f4a07fff640 1 -- 172.21.15.67:0/3013440281 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a100052d0 con 0x5556a3aa01b0 2024-07-31T15:10:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.408+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/3013440281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3aa01b0 msgr2=0x5556a3e5a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:10:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.408+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/3013440281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3aa01b0 0x5556a3e5a7d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4a10000c00 tx=0x7f4a10004000 comp rx=0 tx=0).stop 2024-07-31T15:10:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.408+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/3013440281 shutdown_connections 2024-07-31T15:10:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.408+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/3013440281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3aa01b0 0x5556a3e5a7d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.408+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/3013440281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3a7f4d0 0x5556a3e4eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.408+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/3013440281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a772f0 0x5556a3e48fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.408+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/3013440281 >> 172.21.15.67:0/3013440281 conn(0x5556a3a94300 msgr2=0x5556a3e4e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:10:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.415+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/3013440281 shutdown_connections 2024-07-31T15:10:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.415+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/3013440281 wait complete. 2024-07-31T15:10:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.415+0000 7f4a2bd6b740 1 -- start start 2024-07-31T15:10:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.415+0000 7f4a2bd6b740 1 Processor -- start 2024-07-31T15:10:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.416+0000 7f4a2bd6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a772f0 0x5556a3e5a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.416+0000 7f4a2bd6b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3a7f4d0 0x5556a3e5ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.416+0000 7f4a2bd6b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a341c0 0x5556a3e5b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.416+0000 7f4a2bd6b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201c8da0 con 0x5556a3a341c0 2024-07-31T15:10:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.416+0000 7f4a2bd6b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201c86d0 con 0x5556a3a772f0 2024-07-31T15:10:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.416+0000 7f4a2bd6b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a20074dd0 con 0x5556a3a7f4d0 2024-07-31T15:10:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.416+0000 7f4a2bd6b740 10 client.0 _ll_register_callbacks cb 0x7f4a1001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:10:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.416+0000 7f4a1e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a772f0 0x5556a3e5a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:58.418 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/aquantia/built-in.a 2024-07-31T15:10:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.416+0000 7f4a1e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a772f0 0x5556a3e5a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:59862/0 (socket says 172.21.15.67:59862) 2024-07-31T15:10:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.416+0000 7f4a1e59c640 1 -- 172.21.15.67:0/2376011631 learned_addr learned my addr 172.21.15.67:0/2376011631 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:10:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.416+0000 7f4a1dd9b640 1 --2- 172.21.15.67:0/2376011631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3a7f4d0 0x5556a3e5ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.417+0000 7f4a1e59c640 1 -- 172.21.15.67:0/2376011631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3a7f4d0 msgr2=0x5556a3e5ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.417+0000 7f4a1e59c640 1 --2- 172.21.15.67:0/2376011631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3a7f4d0 0x5556a3e5ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.417+0000 7f4a1e59c640 1 -- 172.21.15.67:0/2376011631 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a341c0 msgr2=0x5556a3e5b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:10:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.417+0000 7f4a1e59c640 1 --2- 172.21.15.67:0/2376011631 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a341c0 0x5556a3e5b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:10:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.417+0000 7f4a1e59c640 1 -- 172.21.15.67:0/2376011631 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556a3ec6520 con 0x5556a3a772f0 2024-07-31T15:10:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.418+0000 7f4a1e59c640 1 --2- 172.21.15.67:0/2376011631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a772f0 0x5556a3e5a650 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800c7a0 tx=0x7f4a18004c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c06c631e1610568a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:10:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.422+0000 7f4a05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:10:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.422+0000 7f4a05ffb640 1 -- 172.21.15.67:0/2376011631 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1801d050 con 0x5556a3a772f0 2024-07-31T15:10:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.422+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2376011631 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5556a3db5da0 con 0x5556a3a772f0 2024-07-31T15:10:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.422+0000 7f4a05ffb640 1 -- 172.21.15.67:0/2376011631 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9307+0+0 (secure 0 0 0) 0x7f4a1800d9b0 con 0x5556a3a772f0 2024-07-31T15:10:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.423+0000 7f4a05ffb640 1 -- 172.21.15.67:0/2376011631 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f4a18032050 con 0x5556a3a772f0 2024-07-31T15:10:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.423+0000 7f4a2bd6b740 10 client.29243 fetch_fsmap learned FSMap version 66 2024-07-31T15:10:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.423+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2376011631 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5556a3ec78a0 con 0x5556a3a772f0 2024-07-31T15:10:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.427+0000 7f4a05ffb640 1 -- 172.21.15.67:0/2376011631 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5556a3db5da0 con 0x5556a3a772f0 2024-07-31T15:10:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.428+0000 7f4a05ffb640 1 -- 172.21.15.67:0/2376011631 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f4a180293e0 con 0x5556a3a772f0 2024-07-31T15:10:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.428+0000 7f4a2bd6b740 10 client.29243 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:10:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.428+0000 7f4a2bd6b740 10 client.29243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:10:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.428+0000 7f4a2bd6b740 20 client.29243 populate_metadata read hostname 'smithi067' 2024-07-31T15:10:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.428+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/2376011631 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5556a3ec7ff0 0x5556a3eca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:10:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.428+0000 7f4a2bd6b740 4 client.29243 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:10:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.428+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2376011631 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556a3db34e0 con 0x5556a3ec7ff0 2024-07-31T15:10:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.429+0000 7f4a1dd9b640 1 --2- 172.21.15.67:0/2376011631 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5556a3ec7ff0 0x5556a3eca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:10:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.430+0000 7f4a05ffb640 10 client.29243 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:10:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:10:58.430+0000 7f4a1dd9b640 1 --2- 172.21.15.67:0/2376011631 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5556a3ec7ff0 0x5556a3eca470 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=707b19c701c80425 server_cookie=78ca9ce298db051e in_seq=0 out_seq=0 2024-07-31T15:10:58.481 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/arc/built-in.a 2024-07-31T15:10:58.502 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/asix/built-in.a 2024-07-31T15:10:58.538 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/ip6_offload.o 2024-07-31T15:10:58.545 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/handshake/trace.o 2024-07-31T15:10:58.564 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/atheros/built-in.a 2024-07-31T15:10:58.578 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/mdio/built-in.a 2024-07-31T15:10:58.624 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/cadence/built-in.a 2024-07-31T15:10:58.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:10:58.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:10:58.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:10:58.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: pgmap v1035: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 1.30k op/s 2024-07-31T15:10:58.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:58.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:58 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.771 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/xfrm4_state.o 2024-07-31T15:10:58.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:10:58.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:10:58.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:10:58.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: pgmap v1035: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 1.30k op/s 2024-07-31T15:10:58.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:58.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:58 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.859 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/broadcom/tg3.o 2024-07-31T15:10:58.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:10:58.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:10:58.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:10:58.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:58.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: pgmap v1035: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 1.30k op/s 2024-07-31T15:10:58.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:58.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:58 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.464 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/shmem_utils.o 2024-07-31T15:10:59.471 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/ethtool.o 2024-07-31T15:10:59.506 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/usb/built-in.a 2024-07-31T15:10:59.576 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/rx.o 2024-07-31T15:10:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:59.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:59.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:59.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:59.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:59.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:59.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:59.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:59.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:10:59.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:10:59 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.796 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/handshake/built-in.a 2024-07-31T15:10:59.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:59.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:59.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:59.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:59.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:59.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:10:59.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:10:59 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.886 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/sysfs_engines.o 2024-07-31T15:10:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:59.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:59.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:59.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:59.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:59.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:59.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:59.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:10:59.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:10:59 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:10:59.980 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/xfrm4_input.o 2024-07-31T15:11:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.111+0000 7f4a05ffb640 1 -- 172.21.15.67:0/2376011631 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5556a3db34e0 con 0x5556a3ec7ff0 2024-07-31T15:11:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.112+0000 7f4a05ffb640 10 client.29243 handle_command_reply: tid=0 2024-07-31T15:11:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.112+0000 7f4a2bd6b740 1 client.29243 shutdown 2024-07-31T15:11:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.113+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2376011631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a772f0 msgr2=0x5556a3e5a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.113+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/2376011631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a772f0 0x5556a3e5a650 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800c7a0 tx=0x7f4a18004c30 comp rx=0 tx=0).stop 2024-07-31T15:11:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.113+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2376011631 shutdown_connections 2024-07-31T15:11:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.113+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/2376011631 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5556a3ec7ff0 0x5556a3eca470 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.113+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/2376011631 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556a3a341c0 0x5556a3e5b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.113+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/2376011631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556a3a7f4d0 0x5556a3e5ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.113+0000 7f4a05ffb640 0 client.29243 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.113+0000 7f4a2bd6b740 1 --2- 172.21.15.67:0/2376011631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556a3a772f0 0x5556a3e5a650 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.113+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2376011631 >> 172.21.15.67:0/2376011631 conn(0x5556a3a94300 msgr2=0x5556a3e4fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.114+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2376011631 shutdown_connections 2024-07-31T15:11:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.114+0000 7f4a2bd6b740 1 -- 172.21.15.67:0/2376011631 wait complete. 2024-07-31T15:11:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.114+0000 7f4a2bd6b740 20 client.29243 trim_cache size 0 max 16384 2024-07-31T15:11:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:11:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:11:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:11:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:11:00.117 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/admtek/built-in.a 2024-07-31T15:11:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.119+0000 7f4a1f59e640 1 -- 172.21.15.67:0/2370750730 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4a080b3d50 msgr2=0x7f4a080b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.119+0000 7f4a1f59e640 1 --2- 172.21.15.67:0/2370750730 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4a080b3d50 0x7f4a080b6260 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x5556a3e51070 tx=0x7f4a18009250 comp rx=0 tx=0).stop 2024-07-31T15:11:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.119+0000 7f4a1f59e640 1 -- 172.21.15.67:0/2370750730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a2010c0e0 msgr2=0x7f4a201a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.119+0000 7f4a1f59e640 1 --2- 172.21.15.67:0/2370750730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a2010c0e0 0x7f4a201a1080 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4a14006010 tx=0x7f4a14031ba0 comp rx=0 tx=0).stop 2024-07-31T15:11:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.121+0000 7f4a1f59e640 1 -- 172.21.15.67:0/2370750730 shutdown_connections 2024-07-31T15:11:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.121+0000 7f4a1f59e640 1 --2- 172.21.15.67:0/2370750730 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4a080b3d50 0x7f4a080b6260 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.121+0000 7f4a1f59e640 1 --2- 172.21.15.67:0/2370750730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a20114d40 0x7f4a201a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.121+0000 7f4a1f59e640 1 --2- 172.21.15.67:0/2370750730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a2010c0e0 0x7f4a201a1080 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.121+0000 7f4a1f59e640 1 --2- 172.21.15.67:0/2370750730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a2010b790 0x7f4a2010b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.121+0000 7f4a1f59e640 1 -- 172.21.15.67:0/2370750730 >> 172.21.15.67:0/2370750730 conn(0x7f4a2006e320 msgr2=0x7f4a2010c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.123+0000 7f4a1f59e640 1 -- 172.21.15.67:0/2370750730 shutdown_connections 2024-07-31T15:11:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:00.123+0000 7f4a1f59e640 1 -- 172.21.15.67:0/2370750730 wait complete. 2024-07-31T15:11:00.167 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/ath/built-in.a 2024-07-31T15:11:00.200 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/atmel/built-in.a 2024-07-31T15:11:00.208 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/tcpv6_offload.o 2024-07-31T15:11:00.247 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/broadcom/built-in.a 2024-07-31T15:11:00.287 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/cisco/built-in.a 2024-07-31T15:11:00.327 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/intel/built-in.a 2024-07-31T15:11:00.368 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/intersil/built-in.a 2024-07-31T15:11:00.413 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/marvell/built-in.a 2024-07-31T15:11:00.460 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/mediatek/built-in.a 2024-07-31T15:11:00.563 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/microchip/built-in.a 2024-07-31T15:11:00.608 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/purelifi/built-in.a 2024-07-31T15:11:00.655 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/quantenna/built-in.a 2024-07-31T15:11:00.702 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/ralink/built-in.a 2024-07-31T15:11:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:00 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:00 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:00.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:00 smithi077 ceph-mon[37327]: pgmap v1036: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 1.49k op/s 2024-07-31T15:11:00.738 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/realtek/built-in.a 2024-07-31T15:11:00.788 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/rsi/built-in.a 2024-07-31T15:11:00.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:00 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:00.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:00 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:00.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:00 smithi067 ceph-mon[32750]: pgmap v1036: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 1.49k op/s 2024-07-31T15:11:00.842 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/silabs/built-in.a 2024-07-31T15:11:00.863 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_ggtt_gmch.o 2024-07-31T15:11:00.886 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/st/built-in.a 2024-07-31T15:11:00.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:00 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:00.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:00 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:00.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:00 smithi073 ceph-mon[36714]: pgmap v1036: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 1.49k op/s 2024-07-31T15:11:00.999 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/ti/built-in.a 2024-07-31T15:11:01.026 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/gen6_renderstate.o 2024-07-31T15:11:01.106 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/zydas/built-in.a 2024-07-31T15:11:01.137 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/legacy/built-in.a 2024-07-31T15:11:01.175 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/xfrm4_output.o 2024-07-31T15:11:01.181 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/virtual/built-in.a 2024-07-31T15:11:01.201 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/wireless/built-in.a 2024-07-31T15:11:01.244 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/spectmgmt.o 2024-07-31T15:11:01.314 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/mii.o 2024-07-31T15:11:01.411 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/exthdrs_offload.o 2024-07-31T15:11:01.432 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/gen7_renderstate.o 2024-07-31T15:11:01.788 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/gen8_renderstate.o 2024-07-31T15:11:01.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:01.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:01.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:01.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:01.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:01.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:01.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:01.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:01.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:01.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:01.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:01 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:01.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:01.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:01.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:01.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:01.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:01.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:01.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:01.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:01.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:01.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:01 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.905 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/gen9_renderstate.o 2024-07-31T15:11:01.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:01.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:01.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:01.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:01.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:01.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:01.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:01.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:01.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:01.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:01.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:01.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:01 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.199 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_busy.o 2024-07-31T15:11:02.252 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv4/xfrm4_protocol.o 2024-07-31T15:11:02.323 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/inet6_hashtables.o 2024-07-31T15:11:02.343 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/ipv6/mcast_snoop.o 2024-07-31T15:11:02.384 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/socket.o 2024-07-31T15:11:02.401 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/tx.o 2024-07-31T15:11:02.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: pgmap v1037: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 2.14k op/s 2024-07-31T15:11:02.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:02.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:02 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.892 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:11:02.443+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000948f [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250/ auth{4=1} v716 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:35.607536+0000 75=75+0) n(v5 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1) old_inodes=2 (inest sync->mix g=4) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce836e4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000948f:[<0x1000000948d/8250 v533>,<0x1000000945c/serial v657>,<0x10000001d8f/tty v18356>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x1000000948f:[<0x1000000948d/8250 v716>,<0x1000000945c/serial v1492>,<0x10000001d8f/tty v18901>,<0x10000000202/drivers v21665>,<0x10000000201/linux-6.5.11 v2561>,<0x10000000006/t v1607>,<0x10000000005/tmp v1534>,<0x100000001fb/client.0 v1509>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1471>,<0x10000000001/sv_1 v1472>,<0x10000000000/qa v1462>,<0x1/volumes v1414>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:33.063897+0000 47=47+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:35.607536+0000 75=75+0)","memory_value.rstat":"n(v5 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:11:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: pgmap v1037: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 2.14k op/s 2024-07-31T15:11:02.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:02.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:02 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: pgmap v1037: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 2.14k op/s 2024-07-31T15:11:02.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:02.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:02.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:02 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.389 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_clflush.o 2024-07-31T15:11:03.544 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_context.o 2024-07-31T15:11:03.661 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/ipv4/built-in.a 2024-07-31T15:11:03.759 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/key.o 2024-07-31T15:11:03.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:11:03.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:03.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:03 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.859 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/compat.o 2024-07-31T15:11:03.865 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/ipv6/built-in.a 2024-07-31T15:11:03.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:11:03.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:03.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:03 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:11:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:03.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:03.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:03 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.043 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/sysctl_net.o 2024-07-31T15:11:04.442 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_create.o 2024-07-31T15:11:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:04.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:04.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:04.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:04.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:04.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:04.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:04.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:04 smithi067 ceph-mon[32750]: pgmap v1038: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 2.71k op/s 2024-07-31T15:11:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:04.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:04.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:04.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:04.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:04.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:04.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:04.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:04.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:04.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:04.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:04.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:04.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:04.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:04 smithi073 ceph-mon[36714]: pgmap v1038: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 2.71k op/s 2024-07-31T15:11:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:04.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:04.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:04.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:04.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:04 smithi077 ceph-mon[37327]: pgmap v1038: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 2.71k op/s 2024-07-31T15:11:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/brocade/built-in.a 2024-07-31T15:11:05.218 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/util.o 2024-07-31T15:11:05.371 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/cavium/common/built-in.a 2024-07-31T15:11:05.404 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/cavium/thunder/built-in.a 2024-07-31T15:11:05.497 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/cavium/liquidio/built-in.a 2024-07-31T15:11:05.549 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/solomon/built-in.a 2024-07-31T15:11:05.596 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_aperture.o 2024-07-31T15:11:05.601 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/cavium/octeon/built-in.a 2024-07-31T15:11:05.678 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/cavium/built-in.a 2024-07-31T15:11:05.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:05.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:05.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:05.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:05.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:05.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:05.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:05.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:05.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: osdmap e129: 12 total, 12 up, 12 in 2024-07-31T15:11:05.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.882 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/chelsio/built-in.a 2024-07-31T15:11:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:05.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: osdmap e129: 12 total, 12 up, 12 in 2024-07-31T15:11:05.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.968 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/cisco/built-in.a 2024-07-31T15:11:05.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:05.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: osdmap e129: 12 total, 12 up, 12 in 2024-07-31T15:11:05.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:06.032 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/cortina/built-in.a 2024-07-31T15:11:06.045 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_atomic.o 2024-07-31T15:11:06.092 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_atomic_uapi.o 2024-07-31T15:11:06.108 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_dmabuf.o 2024-07-31T15:11:06.530 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/loopback.o 2024-07-31T15:11:06.584 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/netconsole.o 2024-07-31T15:11:06.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:06 smithi067 ceph-mon[32750]: pgmap v1040: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 3.03k op/s 2024-07-31T15:11:06.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:06 smithi073 ceph-mon[36714]: pgmap v1040: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 3.03k op/s 2024-07-31T15:11:06.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:06 smithi077 ceph-mon[37327]: pgmap v1040: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 3.03k op/s 2024-07-31T15:11:07.660 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_auth.o 2024-07-31T15:11:07.775 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_domain.o 2024-07-31T15:11:07.940 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_execbuffer.o 2024-07-31T15:11:07.983 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_blend.o 2024-07-31T15:11:08.381 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/wme.o 2024-07-31T15:11:08.625 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_bridge.o 2024-07-31T15:11:08.642 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:11:08 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:11:08.238+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000439c [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du/ auth{2=1} v303 ap=2 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:01.164016+0000 34=34+0) n(v3 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1) old_inodes=1 (inest sync->mix g=2) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce83f19600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000439c:[<0x10000004399/rcar-du v163>,<0x10000002d23/renesas v2988>,<0x10000002d21/drm v1196>,<0x10000001d8f/gpu v9523>,<0x10000000202/drivers v2799>,<0x10000000201/linux-6.5.11 v334>,<0x10000000006/t v232>,<0x10000000005/tmp v202>,<0x100000001fb/client.0 v201>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v184>,<0x10000000001/sv_1 v213>,<0x10000000000/qa v197>,<0x1/volumes v198>]//[]","memoryvalue":"(2)0x1000000439c:[<0x10000004399/rcar-du v303>,<0x10000002d23/renesas v3147>,<0x10000002d21/drm v1562>,<0x10000001d8f/gpu v18847>,<0x10000000202/drivers v21717>,<0x10000000201/linux-6.5.11 v2573>,<0x10000000006/t v1620>,<0x10000000005/tmp v1547>,<0x100000001fb/client.0 v1521>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1484>,<0x10000000001/sv_1 v1485>,<0x10000000000/qa v1475>,<0x1/volumes v1426>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:38.082295+0000 30=30+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:01.164016+0000 34=34+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:11:08.890 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/chan.o 2024-07-31T15:11:09.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:09 smithi067 ceph-mon[32750]: pgmap v1041: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.0 MiB/s wr, 2.95k op/s 2024-07-31T15:11:09.374 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/virtio_net.o 2024-07-31T15:11:09.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:09 smithi073 ceph-mon[36714]: pgmap v1041: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.0 MiB/s wr, 2.95k op/s 2024-07-31T15:11:09.418 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_cache.o 2024-07-31T15:11:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:09 smithi077 ceph-mon[37327]: pgmap v1041: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.0 MiB/s wr, 2.95k op/s 2024-07-31T15:11:10.254 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_client.o 2024-07-31T15:11:10.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:10 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:10.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:10 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:11:10.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:10 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:10.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:10 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:10.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:10 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:10.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:10 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:10.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:10 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:10.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:10 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:10.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:10 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:10.361 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/auxdisplay/built-in.a 2024-07-31T15:11:10.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:10 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:10 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:11:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:10 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:10 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:10 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:10 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:10 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:10 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:10.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:10 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:10.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:10 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:10.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:10 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:11:10.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:10 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:10.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:10 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:10.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:10 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:10.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:10 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:10.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:10 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:10.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:10 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:10.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:10 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:10.509 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pcmcia/cs.o 2024-07-31T15:11:10.523 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/broadcom/built-in.a 2024-07-31T15:11:10.546 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pcmcia/socket_sysfs.o 2024-07-31T15:11:10.599 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/common/common.o 2024-07-31T15:11:11.282 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_internal.o 2024-07-31T15:11:11.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:11.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:11 smithi067 ceph-mon[32750]: pgmap v1042: 129 pgs: 17 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 60 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 997 KiB/s wr, 2.77k op/s 2024-07-31T15:11:11.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:11.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:11.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:11.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:11.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:11.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:11 smithi073 ceph-mon[36714]: pgmap v1042: 129 pgs: 17 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 60 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 997 KiB/s wr, 2.77k op/s 2024-07-31T15:11:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:11.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.423 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/common/debug.o 2024-07-31T15:11:11.456 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/dec/tulip/built-in.a 2024-07-31T15:11:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:11 smithi077 ceph-mon[37327]: pgmap v1042: 129 pgs: 17 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 60 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 997 KiB/s wr, 2.77k op/s 2024-07-31T15:11:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:11.526 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/dec/built-in.a 2024-07-31T15:11:11.633 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/usb.o 2024-07-31T15:11:11.703 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/dlink/built-in.a 2024-07-31T15:11:11.791 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/emulex/built-in.a 2024-07-31T15:11:11.865 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_client_modeset.o 2024-07-31T15:11:11.887 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/engleder/built-in.a 2024-07-31T15:11:11.963 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/ezchip/built-in.a 2024-07-31T15:11:11.994 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/trace.o 2024-07-31T15:11:12.015 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/usb/common/built-in.a 2024-07-31T15:11:12.016 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/fujitsu/built-in.a 2024-07-31T15:11:12.020 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pcmcia/cardbus.o 2024-07-31T15:11:12.107 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/fungible/built-in.a 2024-07-31T15:11:12.196 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/google/built-in.a 2024-07-31T15:11:12.214 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/usb/phy/built-in.a 2024-07-31T15:11:12.318 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/huawei/built-in.a 2024-07-31T15:11:12.318 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/mon/mon_main.o 2024-07-31T15:11:12.326 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_object.o 2024-07-31T15:11:12.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:12.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:12.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:12.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:12.440 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000/e1000_main.o 2024-07-31T15:11:12.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:12.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:12.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:12.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:12.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:12.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:12.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:12.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:12.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:12.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:12.824 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pcmcia/ds.o 2024-07-31T15:11:12.962 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/hub.o 2024-07-31T15:11:13.179 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000/e1000_hw.o 2024-07-31T15:11:13.349 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/mon/mon_stat.o 2024-07-31T15:11:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: pgmap v1043: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 590 KiB/s wr, 2.06k op/s 2024-07-31T15:11:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:13.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:13.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.418+0000 7ff66a068640 1 -- 172.21.15.67:0/2127654965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66410fd80 msgr2=0x7ff66407b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.418+0000 7ff66a068640 1 --2- 172.21.15.67:0/2127654965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66410fd80 0x7ff66407b0c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009ca0 tx=0x7ff65c02a8f0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.418+0000 7ff66a068640 1 -- 172.21.15.67:0/2127654965 shutdown_connections 2024-07-31T15:11:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.418+0000 7ff66a068640 1 --2- 172.21.15.67:0/2127654965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66410fd80 0x7ff66407b0c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.418+0000 7ff66a068640 1 --2- 172.21.15.67:0/2127654965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff66410f430 0x7ff66410f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.418+0000 7ff66a068640 1 --2- 172.21.15.67:0/2127654965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66410b460 0x7ff66410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.418+0000 7ff66a068640 1 -- 172.21.15.67:0/2127654965 >> 172.21.15.67:0/2127654965 conn(0x7ff6640707c0 msgr2=0x7ff664070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.419+0000 7ff66a068640 1 -- 172.21.15.67:0/2127654965 shutdown_connections 2024-07-31T15:11:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.419+0000 7ff66a068640 1 -- 172.21.15.67:0/2127654965 wait complete. 2024-07-31T15:11:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.420+0000 7ff66a068640 1 Processor -- start 2024-07-31T15:11:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.420+0000 7ff66a068640 1 -- start start 2024-07-31T15:11:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.420+0000 7ff66a068640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66410b460 0x7ff66407ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.420+0000 7ff66a068640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66410f430 0x7ff664080520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.420+0000 7ff66a068640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff66410fd80 0x7ff664080a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.420+0000 7ff66a068640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff66407e550 con 0x7ff66410fd80 2024-07-31T15:11:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.420+0000 7ff66a068640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff66407e3d0 con 0x7ff66410f430 2024-07-31T15:11:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.420+0000 7ff66a068640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff66407e6d0 con 0x7ff66410b460 2024-07-31T15:11:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.421+0000 7ff6637fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66410b460 0x7ff66407ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.421+0000 7ff662ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66410f430 0x7ff664080520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.421+0000 7ff6637fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66410b460 0x7ff66407ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:47902/0 (socket says 172.21.15.67:47902) 2024-07-31T15:11:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.421+0000 7ff6637fe640 1 -- 172.21.15.67:0/2323300665 learned_addr learned my addr 172.21.15.67:0/2323300665 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.421+0000 7ff662ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66410f430 0x7ff664080520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46516/0 (socket says 172.21.15.67:46516) 2024-07-31T15:11:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.421+0000 7ff663fff640 1 --2- 172.21.15.67:0/2323300665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff66410fd80 0x7ff664080a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.421+0000 7ff6637fe640 1 -- 172.21.15.67:0/2323300665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66410f430 msgr2=0x7ff664080520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.421+0000 7ff6637fe640 1 --2- 172.21.15.67:0/2323300665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66410f430 0x7ff664080520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.421+0000 7ff6637fe640 1 -- 172.21.15.67:0/2323300665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff66410fd80 msgr2=0x7ff664080a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.421+0000 7ff6637fe640 1 --2- 172.21.15.67:0/2323300665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff66410fd80 0x7ff664080a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.421+0000 7ff6637fe640 1 -- 172.21.15.67:0/2323300665 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff664132700 con 0x7ff66410b460 2024-07-31T15:11:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.421+0000 7ff6637fe640 1 --2- 172.21.15.67:0/2323300665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66410b460 0x7ff66407ffe0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff65800b450 tx=0x7ff65800b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.430+0000 7ff660ff9640 1 -- 172.21.15.67:0/2323300665 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658015020 con 0x7ff66410b460 2024-07-31T15:11:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.430+0000 7ff660ff9640 1 -- 172.21.15.67:0/2323300665 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff658011ce0 con 0x7ff66410b460 2024-07-31T15:11:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.432+0000 7ff660ff9640 1 -- 172.21.15.67:0/2323300665 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658010de0 con 0x7ff66410b460 2024-07-31T15:11:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.432+0000 7ff66a068640 1 -- 172.21.15.67:0/2323300665 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6641329f0 con 0x7ff66410b460 2024-07-31T15:11:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.432+0000 7ff66a068640 1 -- 172.21.15.67:0/2323300665 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff664132f40 con 0x7ff66410b460 2024-07-31T15:11:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.435+0000 7ff671035740 1 Processor -- start 2024-07-31T15:11:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.436+0000 7ff671035740 1 -- start start 2024-07-31T15:11:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.436+0000 7ff671035740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36991b0 0x559ba3a3bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.436+0000 7ff671035740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba36702f0 0x559ba3a3c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.436+0000 7ff671035740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a3c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.436+0000 7ff671035740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559ba39b13b0 con 0x559ba36991b0 2024-07-31T15:11:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.437+0000 7ff671035740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559ba39a08b0 con 0x559ba36702f0 2024-07-31T15:11:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.437+0000 7ff671035740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559ba39a48f0 con 0x559ba36784d0 2024-07-31T15:11:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.437+0000 7ff662ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a3c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.437+0000 7ff662ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a3c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:47904/0 (socket says 172.21.15.67:47904) 2024-07-31T15:11:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.437+0000 7ff662ffd640 1 -- 172.21.15.67:0/2535398921 learned_addr learned my addr 172.21.15.67:0/2535398921 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.437+0000 7ff662ffd640 1 -- 172.21.15.67:0/2535398921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba36702f0 msgr2=0x559ba3a3c220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:11:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.437+0000 7ff662ffd640 1 --2- 172.21.15.67:0/2535398921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba36702f0 0x559ba3a3c220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.437+0000 7ff662ffd640 1 -- 172.21.15.67:0/2535398921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36991b0 msgr2=0x559ba3a3bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.437+0000 7ff662ffd640 1 --2- 172.21.15.67:0/2535398921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36991b0 0x559ba3a3bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.437+0000 7ff662ffd640 1 -- 172.21.15.67:0/2535398921 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ba39479a0 con 0x559ba36784d0 2024-07-31T15:11:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.438+0000 7ff662ffd640 1 --2- 172.21.15.67:0/2535398921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a3c760 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff65400ac60 tx=0x7ff65400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a32b94d4f7c4b922 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.439+0000 7ff660ff9640 1 -- 172.21.15.67:0/2323300665 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff658018020 con 0x7ff66410b460 2024-07-31T15:11:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.440+0000 7ff660ff9640 1 --2- 172.21.15.67:0/2323300665 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff64c0b3da0 0x7ff64c0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.440+0000 7ff660ff9640 1 -- 172.21.15.67:0/2323300665 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7ff6580e3940 con 0x7ff66410b460 2024-07-31T15:11:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.443+0000 7ff663fff640 1 --2- 172.21.15.67:0/2323300665 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff64c0b3da0 0x7ff64c0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.455+0000 7ff663fff640 1 --2- 172.21.15.67:0/2323300665 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff64c0b3da0 0x7ff64c0b62b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff65c0094e0 tx=0x7ff65c02adf0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.457+0000 7ff649ffb640 1 -- 172.21.15.67:0/2535398921 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65401d070 con 0x559ba36784d0 2024-07-31T15:11:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.457+0000 7ff649ffb640 1 -- 172.21.15.67:0/2535398921 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff654018af0 con 0x559ba36784d0 2024-07-31T15:11:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.457+0000 7ff649ffb640 1 -- 172.21.15.67:0/2535398921 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654010b00 con 0x559ba36784d0 2024-07-31T15:11:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.458+0000 7ff671035740 1 -- 172.21.15.67:0/2535398921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 msgr2=0x559ba3a3c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.458+0000 7ff671035740 1 --2- 172.21.15.67:0/2535398921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a3c760 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff65400ac60 tx=0x7ff65400f790 comp rx=0 tx=0).stop 2024-07-31T15:11:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.462+0000 7ff671035740 1 -- 172.21.15.67:0/2535398921 shutdown_connections 2024-07-31T15:11:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.462+0000 7ff671035740 1 --2- 172.21.15.67:0/2535398921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a3c760 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.462+0000 7ff671035740 1 --2- 172.21.15.67:0/2535398921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba36702f0 0x559ba3a3c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.462+0000 7ff671035740 1 --2- 172.21.15.67:0/2535398921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36991b0 0x559ba3a3bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.462+0000 7ff671035740 1 -- 172.21.15.67:0/2535398921 >> 172.21.15.67:0/2535398921 conn(0x559ba368d300 msgr2=0x559ba3a39bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.463+0000 7ff671035740 1 -- 172.21.15.67:0/2535398921 shutdown_connections 2024-07-31T15:11:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.463+0000 7ff671035740 1 -- 172.21.15.67:0/2535398921 wait complete. 2024-07-31T15:11:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.463+0000 7ff671035740 1 -- start start 2024-07-31T15:11:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.464+0000 7ff671035740 1 Processor -- start 2024-07-31T15:11:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.464+0000 7ff671035740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba36702f0 0x559ba3a47940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.464+0000 7ff671035740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a47f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.464+0000 7ff671035740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36991b0 0x559ba3a53780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.464+0000 7ff671035740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559ba38daab0 con 0x559ba36991b0 2024-07-31T15:11:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.464+0000 7ff671035740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559ba3928cf0 con 0x559ba36702f0 2024-07-31T15:11:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.464+0000 7ff671035740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559ba3a4a460 con 0x559ba36784d0 2024-07-31T15:11:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.464+0000 7ff671035740 10 client.0 _ll_register_callbacks cb 0x559ba399a9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.465+0000 7ff662ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a47f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.465+0000 7ff6637fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba36702f0 0x559ba3a47940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.465+0000 7ff6637fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba36702f0 0x559ba3a47940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46522/0 (socket says 172.21.15.67:46522) 2024-07-31T15:11:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.465+0000 7ff6637fe640 1 -- 172.21.15.67:0/2750202770 learned_addr learned my addr 172.21.15.67:0/2750202770 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.465+0000 7ff6637fe640 1 -- 172.21.15.67:0/2750202770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 msgr2=0x559ba3a47f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.465+0000 7ff6637fe640 1 --2- 172.21.15.67:0/2750202770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a47f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.465+0000 7ff6637fe640 1 -- 172.21.15.67:0/2750202770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36991b0 msgr2=0x559ba3a53780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.465+0000 7ff6637fe640 1 --2- 172.21.15.67:0/2750202770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36991b0 0x559ba3a53780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.465+0000 7ff6637fe640 1 -- 172.21.15.67:0/2750202770 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ba3704790 con 0x559ba36702f0 2024-07-31T15:11:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.466+0000 7ff6637fe640 1 --2- 172.21.15.67:0/2750202770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba36702f0 0x559ba3a47940 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff6580e12d0 tx=0x7ff658010440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0d16434a2d086d3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.467+0000 7ff642ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:11:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.467+0000 7ff642ffd640 1 -- 172.21.15.67:0/2750202770 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6580c3070 con 0x559ba36702f0 2024-07-31T15:11:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.467+0000 7ff671035740 1 -- 172.21.15.67:0/2750202770 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff654018af0 con 0x559ba36702f0 2024-07-31T15:11:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.467+0000 7ff642ffd640 1 -- 172.21.15.67:0/2750202770 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7ff658011830 con 0x559ba36702f0 2024-07-31T15:11:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.467+0000 7ff642ffd640 1 -- 172.21.15.67:0/2750202770 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7ff6580ae8a0 con 0x559ba36702f0 2024-07-31T15:11:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.468+0000 7ff671035740 10 client.29252 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.468+0000 7ff671035740 1 -- 172.21.15.67:0/2750202770 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x559ba377d620 con 0x559ba36702f0 2024-07-31T15:11:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.471+0000 7ff642ffd640 1 -- 172.21.15.67:0/2750202770 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff654018af0 con 0x559ba36702f0 2024-07-31T15:11:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.471+0000 7ff642ffd640 1 -- 172.21.15.67:0/2750202770 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7ff658022060 con 0x559ba36702f0 2024-07-31T15:11:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.471+0000 7ff671035740 10 client.29252 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.471+0000 7ff671035740 10 client.29252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:11:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.471+0000 7ff671035740 20 client.29252 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.471+0000 7ff671035740 1 --2- 172.21.15.67:0/2750202770 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559ba3657d80 0x559ba3a598d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.471+0000 7ff671035740 4 client.29252 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:11:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.471+0000 7ff671035740 1 -- 172.21.15.67:0/2750202770 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ba39b3300 con 0x559ba3657d80 2024-07-31T15:11:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.471+0000 7ff662ffd640 1 --2- 172.21.15.67:0/2750202770 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559ba3657d80 0x559ba3a598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.472+0000 7ff642ffd640 10 client.29252 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.472+0000 7ff662ffd640 1 --2- 172.21.15.67:0/2750202770 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559ba3657d80 0x559ba3a598d0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a85b910085ade587 server_cookie=a821248d8f4bb30d in_seq=0 out_seq=0 2024-07-31T15:11:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.473+0000 7ff642ffd640 1 -- 172.21.15.67:0/2750202770 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ba39b3300 con 0x559ba3657d80 2024-07-31T15:11:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.473+0000 7ff642ffd640 10 client.29252 handle_command_reply: tid=0 2024-07-31T15:11:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.473+0000 7ff671035740 1 client.29252 shutdown 2024-07-31T15:11:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.474+0000 7ff671035740 1 -- 172.21.15.67:0/2750202770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba36702f0 msgr2=0x559ba3a47940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.474+0000 7ff671035740 1 --2- 172.21.15.67:0/2750202770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba36702f0 0x559ba3a47940 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff6580e12d0 tx=0x7ff658010440 comp rx=0 tx=0).stop 2024-07-31T15:11:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.474+0000 7ff671035740 1 -- 172.21.15.67:0/2750202770 shutdown_connections 2024-07-31T15:11:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.474+0000 7ff671035740 1 --2- 172.21.15.67:0/2750202770 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559ba3657d80 0x559ba3a598d0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.474+0000 7ff671035740 1 --2- 172.21.15.67:0/2750202770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36991b0 0x559ba3a53780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.474+0000 7ff671035740 1 --2- 172.21.15.67:0/2750202770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a47f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.474+0000 7ff671035740 1 --2- 172.21.15.67:0/2750202770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba36702f0 0x559ba3a47940 secure :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff6580e12d0 tx=0x7ff658010440 comp rx=0 tx=0).stop 2024-07-31T15:11:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.474+0000 7ff642ffd640 0 client.29252 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.474+0000 7ff671035740 1 -- 172.21.15.67:0/2750202770 >> 172.21.15.67:0/2750202770 conn(0x559ba368d300 msgr2=0x559ba3a382b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:13.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.478+0000 7ff671035740 1 -- 172.21.15.67:0/2750202770 shutdown_connections 2024-07-31T15:11:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.478+0000 7ff671035740 1 -- 172.21.15.67:0/2750202770 wait complete. 2024-07-31T15:11:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.478+0000 7ff671035740 20 client.29252 trim_cache size 0 max 16384 2024-07-31T15:11:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: pgmap v1043: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 590 KiB/s wr, 2.06k op/s 2024-07-31T15:11:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:13.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.508 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_lmem.o 2024-07-31T15:11:13.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: pgmap v1043: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 590 KiB/s wr, 2.06k op/s 2024-07-31T15:11:13.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:13.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.573+0000 7ff671035740 1 Processor -- start 2024-07-31T15:11:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.580+0000 7ff671035740 1 -- start start 2024-07-31T15:11:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.580+0000 7ff671035740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36784d0 0x559ba3a389c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.580+0000 7ff671035740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36991b0 0x559ba3a53570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.580+0000 7ff671035740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba362d1c0 0x559ba3a40da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.580+0000 7ff671035740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641dae60 con 0x559ba36784d0 2024-07-31T15:11:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.580+0000 7ff671035740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641d8c50 con 0x559ba362d1c0 2024-07-31T15:11:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.580+0000 7ff671035740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641da790 con 0x559ba36991b0 2024-07-31T15:11:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.581+0000 7ff662ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36991b0 0x559ba3a53570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.581+0000 7ff662ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36991b0 0x559ba3a53570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:47932/0 (socket says 172.21.15.67:47932) 2024-07-31T15:11:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.581+0000 7ff662ffd640 1 -- 172.21.15.67:0/1323520422 learned_addr learned my addr 172.21.15.67:0/1323520422 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.581+0000 7ff663fff640 1 --2- 172.21.15.67:0/1323520422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba362d1c0 0x559ba3a40da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.581+0000 7ff663fff640 1 -- 172.21.15.67:0/1323520422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36991b0 msgr2=0x559ba3a53570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.581+0000 7ff663fff640 1 --2- 172.21.15.67:0/1323520422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36991b0 0x559ba3a53570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.581+0000 7ff663fff640 1 -- 172.21.15.67:0/1323520422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36784d0 msgr2=0x559ba3a389c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.581+0000 7ff663fff640 1 --2- 172.21.15.67:0/1323520422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36784d0 0x559ba3a389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.581+0000 7ff663fff640 1 -- 172.21.15.67:0/1323520422 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ba3a42f80 con 0x559ba362d1c0 2024-07-31T15:11:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.582+0000 7ff663fff640 1 --2- 172.21.15.67:0/1323520422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba362d1c0 0x559ba3a40da0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff65c006010 tx=0x7ff65c03e070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=270e518d9ddfa6c3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.582+0000 7ff6497fa640 1 -- 172.21.15.67:0/1323520422 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c031950 con 0x559ba362d1c0 2024-07-31T15:11:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.583+0000 7ff6497fa640 1 -- 172.21.15.67:0/1323520422 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff65c036040 con 0x559ba362d1c0 2024-07-31T15:11:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.583+0000 7ff6497fa640 1 -- 172.21.15.67:0/1323520422 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c02f8a0 con 0x559ba362d1c0 2024-07-31T15:11:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.584+0000 7ff671035740 1 -- 172.21.15.67:0/1323520422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba362d1c0 msgr2=0x559ba3a40da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.584+0000 7ff671035740 1 --2- 172.21.15.67:0/1323520422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba362d1c0 0x559ba3a40da0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff65c006010 tx=0x7ff65c03e070 comp rx=0 tx=0).stop 2024-07-31T15:11:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.590+0000 7ff671035740 1 -- 172.21.15.67:0/1323520422 shutdown_connections 2024-07-31T15:11:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.590+0000 7ff671035740 1 --2- 172.21.15.67:0/1323520422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba362d1c0 0x559ba3a40da0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.590+0000 7ff671035740 1 --2- 172.21.15.67:0/1323520422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36991b0 0x559ba3a53570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.590+0000 7ff671035740 1 --2- 172.21.15.67:0/1323520422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36784d0 0x559ba3a389c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.590+0000 7ff671035740 1 -- 172.21.15.67:0/1323520422 >> 172.21.15.67:0/1323520422 conn(0x559ba368d300 msgr2=0x559ba3a382b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.594+0000 7ff671035740 1 -- 172.21.15.67:0/1323520422 shutdown_connections 2024-07-31T15:11:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.594+0000 7ff671035740 1 -- 172.21.15.67:0/1323520422 wait complete. 2024-07-31T15:11:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.595+0000 7ff671035740 1 -- start start 2024-07-31T15:11:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.595+0000 7ff671035740 1 Processor -- start 2024-07-31T15:11:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.596+0000 7ff671035740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba362d1c0 0x559ba3a4ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.596+0000 7ff671035740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a4c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.596+0000 7ff671035740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36991b0 0x559ba3a4ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff6637fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba362d1c0 0x559ba3a4ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff6637fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba362d1c0 0x559ba3a4ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46552/0 (socket says 172.21.15.67:46552) 2024-07-31T15:11:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff662ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a4c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff662ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a4c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:47946/0 (socket says 172.21.15.67:47946) 2024-07-31T15:11:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff663fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36991b0 0x559ba3a4ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff671035740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641d99f0 con 0x559ba36991b0 2024-07-31T15:11:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff671035740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641d9320 con 0x559ba362d1c0 2024-07-31T15:11:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff671035740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641da0c0 con 0x559ba36784d0 2024-07-31T15:11:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff6637fe640 1 -- 172.21.15.67:0/441966163 learned_addr learned my addr 172.21.15.67:0/441966163 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff671035740 10 client.0 _ll_register_callbacks cb 0x7ff658010f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff6637fe640 1 -- 172.21.15.67:0/441966163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 msgr2=0x559ba3a4c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff6637fe640 1 --2- 172.21.15.67:0/441966163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a4c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff6637fe640 1 -- 172.21.15.67:0/441966163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36991b0 msgr2=0x559ba3a4ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff6637fe640 1 --2- 172.21.15.67:0/441966163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36991b0 0x559ba3a4ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.597+0000 7ff6637fe640 1 -- 172.21.15.67:0/441966163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ba3ac0060 con 0x559ba362d1c0 2024-07-31T15:11:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.598+0000 7ff6637fe640 1 --2- 172.21.15.67:0/441966163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba362d1c0 0x559ba3a4ffd0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff658009fd0 tx=0x7ff6580aa180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bafe5f0b382d28d3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.605+0000 7ff642ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:11:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.605+0000 7ff642ffd640 1 -- 172.21.15.67:0/441966163 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6580c3070 con 0x559ba362d1c0 2024-07-31T15:11:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.605+0000 7ff671035740 1 -- 172.21.15.67:0/441966163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff65c036040 con 0x559ba362d1c0 2024-07-31T15:11:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.605+0000 7ff642ffd640 1 -- 172.21.15.67:0/441966163 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7ff658018dd0 con 0x559ba362d1c0 2024-07-31T15:11:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.605+0000 7ff642ffd640 1 -- 172.21.15.67:0/441966163 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7ff6580a6160 con 0x559ba362d1c0 2024-07-31T15:11:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.605+0000 7ff671035740 10 client.29258 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.605+0000 7ff671035740 1 -- 172.21.15.67:0/441966163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x559ba3ac11a0 con 0x559ba362d1c0 2024-07-31T15:11:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.617+0000 7ff642ffd640 1 -- 172.21.15.67:0/441966163 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff65c036040 con 0x559ba362d1c0 2024-07-31T15:11:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.617+0000 7ff642ffd640 1 -- 172.21.15.67:0/441966163 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7ff6580afa30 con 0x559ba362d1c0 2024-07-31T15:11:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.617+0000 7ff671035740 10 client.29258 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.617+0000 7ff671035740 10 client.29258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:11:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.617+0000 7ff671035740 20 client.29258 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.617+0000 7ff671035740 1 --2- 172.21.15.67:0/441966163 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559ba3ac1770 0x559ba3ac3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.617+0000 7ff671035740 4 client.29258 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:11:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.617+0000 7ff671035740 1 -- 172.21.15.67:0/441966163 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ba39aeda0 con 0x559ba3ac1770 2024-07-31T15:11:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.617+0000 7ff662ffd640 1 --2- 172.21.15.67:0/441966163 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559ba3ac1770 0x559ba3ac3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.618+0000 7ff642ffd640 10 client.29258 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:13.618+0000 7ff662ffd640 1 --2- 172.21.15.67:0/441966163 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559ba3ac1770 0x559ba3ac3b90 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc39a914ea3473ed server_cookie=3bb65f2d956ccdd9 in_seq=0 out_seq=0 2024-07-31T15:11:14.158 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pcmcia/pcmcia_resource.o 2024-07-31T15:11:14.347 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/mon/mon_text.o 2024-07-31T15:11:14.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:14.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:14.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:14.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:14.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:14.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:14.891 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/pci-quirks.o 2024-07-31T15:11:15.051 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_mman.o 2024-07-31T15:11:15.327 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/mon/mon_bin.o 2024-07-31T15:11:15.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:15 smithi073 ceph-mon[36714]: pgmap v1044: 129 pgs: 7 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 101 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 486 KiB/s wr, 1.50k op/s 2024-07-31T15:11:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.456+0000 7ff642ffd640 1 -- 172.21.15.67:0/441966163 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ba39aeda0 con 0x559ba3ac1770 2024-07-31T15:11:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.456+0000 7ff642ffd640 10 client.29258 handle_command_reply: tid=0 2024-07-31T15:11:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.456+0000 7ff671035740 1 client.29258 shutdown 2024-07-31T15:11:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.465+0000 7ff671035740 1 -- 172.21.15.67:0/441966163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba362d1c0 msgr2=0x559ba3a4ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.465+0000 7ff671035740 1 --2- 172.21.15.67:0/441966163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba362d1c0 0x559ba3a4ffd0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff658009fd0 tx=0x7ff6580aa180 comp rx=0 tx=0).stop 2024-07-31T15:11:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.465+0000 7ff671035740 1 -- 172.21.15.67:0/441966163 shutdown_connections 2024-07-31T15:11:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.465+0000 7ff671035740 1 --2- 172.21.15.67:0/441966163 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x559ba3ac1770 0x559ba3ac3b90 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.465+0000 7ff671035740 1 --2- 172.21.15.67:0/441966163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559ba36991b0 0x559ba3a4ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.465+0000 7ff671035740 1 --2- 172.21.15.67:0/441966163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ba36784d0 0x559ba3a4c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.465+0000 7ff671035740 1 --2- 172.21.15.67:0/441966163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ba362d1c0 0x559ba3a4ffd0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.465+0000 7ff671035740 1 -- 172.21.15.67:0/441966163 >> 172.21.15.67:0/441966163 conn(0x559ba368d300 msgr2=0x559ba3a40da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.465+0000 7ff642ffd640 0 client.29258 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.465+0000 7ff671035740 1 -- 172.21.15.67:0/441966163 shutdown_connections 2024-07-31T15:11:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.465+0000 7ff671035740 1 -- 172.21.15.67:0/441966163 wait complete. 2024-07-31T15:11:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.466+0000 7ff671035740 20 client.29258 trim_cache size 0 max 16384 2024-07-31T15:11:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:11:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:11:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:11:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:11:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.482+0000 7ff66a068640 1 -- 172.21.15.67:0/2323300665 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff64c0b3da0 msgr2=0x7ff64c0b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.482+0000 7ff66a068640 1 --2- 172.21.15.67:0/2323300665 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff64c0b3da0 0x7ff64c0b62b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff65c0094e0 tx=0x7ff65c02adf0 comp rx=0 tx=0).stop 2024-07-31T15:11:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.482+0000 7ff66a068640 1 -- 172.21.15.67:0/2323300665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66410b460 msgr2=0x7ff66407ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.482+0000 7ff66a068640 1 --2- 172.21.15.67:0/2323300665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66410b460 0x7ff66407ffe0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff65800b450 tx=0x7ff65800b850 comp rx=0 tx=0).stop 2024-07-31T15:11:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.482+0000 7ff66a068640 1 -- 172.21.15.67:0/2323300665 shutdown_connections 2024-07-31T15:11:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.482+0000 7ff66a068640 1 --2- 172.21.15.67:0/2323300665 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff64c0b3da0 0x7ff64c0b62b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.482+0000 7ff66a068640 1 --2- 172.21.15.67:0/2323300665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff66410fd80 0x7ff664080a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.482+0000 7ff66a068640 1 --2- 172.21.15.67:0/2323300665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff66410f430 0x7ff664080520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.482+0000 7ff66a068640 1 --2- 172.21.15.67:0/2323300665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff66410b460 0x7ff66407ffe0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.482+0000 7ff66a068640 1 -- 172.21.15.67:0/2323300665 >> 172.21.15.67:0/2323300665 conn(0x7ff6640707c0 msgr2=0x7ff66410e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:15.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:15 smithi077 ceph-mon[37327]: pgmap v1044: 129 pgs: 7 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 101 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 486 KiB/s wr, 1.50k op/s 2024-07-31T15:11:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.486+0000 7ff66a068640 1 -- 172.21.15.67:0/2323300665 shutdown_connections 2024-07-31T15:11:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:15.486+0000 7ff66a068640 1 -- 172.21.15.67:0/2323300665 wait complete. 2024-07-31T15:11:15.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:15 smithi067 ceph-mon[32750]: pgmap v1044: 129 pgs: 7 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 101 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 486 KiB/s wr, 1.50k op/s 2024-07-31T15:11:15.566 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pcmcia/cistpl.o 2024-07-31T15:11:15.671 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/ehci-hcd.o 2024-07-31T15:11:16.026 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000/e1000_ethtool.o 2024-07-31T15:11:16.286 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/class/usblp.o 2024-07-31T15:11:16.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:16.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:16.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:16.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:16.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:16.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:16.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:16.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:16.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:16.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:16.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:16.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:16.688 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_pages.o 2024-07-31T15:11:16.693 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/usb/mon/built-in.a 2024-07-31T15:11:16.785 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/ehci-pci.o 2024-07-31T15:11:16.972 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pcmcia/pcmcia_cis.o 2024-07-31T15:11:17.266 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/usb/class/built-in.a 2024-07-31T15:11:17.363 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/storage/scsiglue.o 2024-07-31T15:11:17.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: pgmap v1045: 129 pgs: 4 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 108 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 440 KiB/s wr, 1.42k op/s 2024-07-31T15:11:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:17.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:17.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:17.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: pgmap v1045: 129 pgs: 4 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 108 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 440 KiB/s wr, 1.42k op/s 2024-07-31T15:11:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:17.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:17.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: pgmap v1045: 129 pgs: 4 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 108 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 440 KiB/s wr, 1.42k op/s 2024-07-31T15:11:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:17.592 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/storage/protocol.o 2024-07-31T15:11:17.625 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/hcd.o 2024-07-31T15:11:17.741 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000/e1000_param.o 2024-07-31T15:11:17.808 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_phys.o 2024-07-31T15:11:17.930 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pcmcia/rsrc_mgr.o 2024-07-31T15:11:18.179 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/storage/transport.o 2024-07-31T15:11:18.237 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_pm.o 2024-07-31T15:11:18.331 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/ohci-hcd.o 2024-07-31T15:11:18.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:18.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:18.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.507 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pcmcia/rsrc_nonstatic.o 2024-07-31T15:11:18.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:18.714 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_region.o 2024-07-31T15:11:18.716 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/intel/e1000/built-in.a 2024-07-31T15:11:19.113 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/storage/usb.o 2024-07-31T15:11:19.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:11:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:19 smithi073 ceph-mon[36714]: pgmap v1046: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 434 KiB/s wr, 1.47k op/s 2024-07-31T15:11:19.396 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/mlme.o 2024-07-31T15:11:19.409 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pcmcia/yenta_socket.o 2024-07-31T15:11:19.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:11:19.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:19.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:19.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:19 smithi077 ceph-mon[37327]: pgmap v1046: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 434 KiB/s wr, 1.47k op/s 2024-07-31T15:11:19.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:11:19.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:19.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:11:19.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:19 smithi067 ceph-mon[32750]: pgmap v1046: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 434 KiB/s wr, 1.47k op/s 2024-07-31T15:11:19.632 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/urb.o 2024-07-31T15:11:19.635 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000e/82571.o 2024-07-31T15:11:19.660 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_shmem.o 2024-07-31T15:11:19.779 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_shrinker.o 2024-07-31T15:11:20.119 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/storage/initializers.o 2024-07-31T15:11:20.647 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/message.o 2024-07-31T15:11:20.662 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/ohci-pci.o 2024-07-31T15:11:20.738 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pcmcia/built-in.a 2024-07-31T15:11:20.805 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/serio/serio.o 2024-07-31T15:11:20.851 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/storage/sierra_ms.o 2024-07-31T15:11:20.957 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000e/ich8lan.o 2024-07-31T15:11:20.979 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/keyboard/atkbd.o 2024-07-31T15:11:20.991 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_stolen.o 2024-07-31T15:11:21.275 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/uhci-hcd.o 2024-07-31T15:11:21.442 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/serio/i8042.o 2024-07-31T15:11:21.483 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/storage/option_ms.o 2024-07-31T15:11:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:21 smithi067 ceph-mon[32750]: pgmap v1047: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 349 KiB/s wr, 1.12k op/s 2024-07-31T15:11:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:21 smithi067 ceph-mon[32750]: osdmap e130: 12 total, 12 up, 12 in 2024-07-31T15:11:21.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:21 smithi073 ceph-mon[36714]: pgmap v1047: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 349 KiB/s wr, 1.12k op/s 2024-07-31T15:11:21.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:21 smithi073 ceph-mon[36714]: osdmap e130: 12 total, 12 up, 12 in 2024-07-31T15:11:21.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:21 smithi077 ceph-mon[37327]: pgmap v1047: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 349 KiB/s wr, 1.12k op/s 2024-07-31T15:11:21.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:21 smithi077 ceph-mon[37327]: osdmap e130: 12 total, 12 up, 12 in 2024-07-31T15:11:21.870 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/input/keyboard/built-in.a 2024-07-31T15:11:21.881 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/driver.o 2024-07-31T15:11:21.945 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/mouse/psmouse-base.o 2024-07-31T15:11:22.020 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_throttle.o 2024-07-31T15:11:22.091 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/storage/usual-tables.o 2024-07-31T15:11:22.486 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/serio/serport.o 2024-07-31T15:11:22.618 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/usb/storage/built-in.a 2024-07-31T15:11:22.658 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_tiling.o 2024-07-31T15:11:22.677 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/xhci.o 2024-07-31T15:11:22.813 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/config.o 2024-07-31T15:11:22.850 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000e/80003es2lan.o 2024-07-31T15:11:22.938 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/mouse/synaptics.o 2024-07-31T15:11:22.942 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/mouse/focaltech.o 2024-07-31T15:11:23.157 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/serio/libps2.o 2024-07-31T15:11:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:23 smithi067 ceph-mon[32750]: pgmap v1049: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 572 KiB/s wr, 1.47k op/s 2024-07-31T15:11:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:11:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:23 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:23.574 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_ttm.o 2024-07-31T15:11:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:23 smithi073 ceph-mon[36714]: pgmap v1049: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 572 KiB/s wr, 1.47k op/s 2024-07-31T15:11:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:11:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:23 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:23.654 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/mouse/alps.o 2024-07-31T15:11:23.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:23 smithi077 ceph-mon[37327]: pgmap v1049: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 572 KiB/s wr, 1.47k op/s 2024-07-31T15:11:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:11:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:23 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:23.859 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/file.o 2024-07-31T15:11:23.917 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/tdls.o 2024-07-31T15:11:23.954 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/input/serio/built-in.a 2024-07-31T15:11:24.013 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/mouse/byd.o 2024-07-31T15:11:24.046 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000e/mac.o 2024-07-31T15:11:24.210 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/rtc/lib.o 2024-07-31T15:11:24.546 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/buffer.o 2024-07-31T15:11:24.595 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/rtc/class.o 2024-07-31T15:11:24.840 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_ttm_move.o 2024-07-31T15:11:24.848 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_ttm_pm.o 2024-07-31T15:11:25.126 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/mouse/logips2pp.o 2024-07-31T15:11:25.146 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/xhci-mem.o 2024-07-31T15:11:25.182 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/sysfs.o 2024-07-31T15:11:25.209 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000e/manage.o 2024-07-31T15:11:25.220 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/rtc/interface.o 2024-07-31T15:11:25.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:24 smithi077 ceph-mon[37327]: pgmap v1050: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 1.38k op/s 2024-07-31T15:11:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:11:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:11:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:24 smithi067 ceph-mon[32750]: pgmap v1050: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 1.38k op/s 2024-07-31T15:11:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:11:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:11:25.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:24 smithi073 ceph-mon[36714]: pgmap v1050: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 1.38k op/s 2024-07-31T15:11:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:11:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:11:25.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:11:25.694 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/mouse/lifebook.o 2024-07-31T15:11:25.705 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_userptr.o 2024-07-31T15:11:25.900 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/i2c/algos/i2c-algo-bit.o 2024-07-31T15:11:25.972 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/ocb.o 2024-07-31T15:11:26.014 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/endpoint.o 2024-07-31T15:11:26.120 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000e/nvm.o 2024-07-31T15:11:26.250 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/mouse/trackpoint.o 2024-07-31T15:11:26.350 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/rtc/nvmem.o 2024-07-31T15:11:26.574 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/devio.o 2024-07-31T15:11:26.689 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/i2c/algos/built-in.a 2024-07-31T15:11:26.702 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/xhci-ext-caps.o 2024-07-31T15:11:26.729 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_wait.o 2024-07-31T15:11:26.742 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/i2c/busses/i2c-i801.o 2024-07-31T15:11:26.848 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/rtc/dev.o 2024-07-31T15:11:26.877 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/mouse/cypress_ps2.o 2024-07-31T15:11:27.070 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000e/phy.o 2024-07-31T15:11:27.294 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/xhci-ring.o 2024-07-31T15:11:27.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:27 smithi067 ceph-mon[32750]: pgmap v1051: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 1.27k op/s 2024-07-31T15:11:27.385 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/airtime.o 2024-07-31T15:11:27.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:27 smithi073 ceph-mon[36714]: pgmap v1051: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 1.27k op/s 2024-07-31T15:11:27.449 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/rtc/proc.o 2024-07-31T15:11:27.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:27 smithi077 ceph-mon[37327]: pgmap v1051: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 1.27k op/s 2024-07-31T15:11:27.496 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:11:27.562 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gem/i915_gemfs.o 2024-07-31T15:11:27.568 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/mouse/psmouse-smbus.o 2024-07-31T15:11:27.963 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/rtc/sysfs.o 2024-07-31T15:11:27.970 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/i2c/busses/built-in.a 2024-07-31T15:11:28.046 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:11:28.048 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/i2c/muxes/built-in.a 2024-07-31T15:11:28.072 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/i2c/i2c-boardinfo.o 2024-07-31T15:11:28.221 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/notify.o 2024-07-31T15:11:28.334 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_active.o 2024-07-31T15:11:28.350 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/input/mouse/built-in.a 2024-07-31T15:11:28.492 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/input/joystick/built-in.a 2024-07-31T15:11:28.516 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/input/tablet/built-in.a 2024-07-31T15:11:28.536 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000e/param.o 2024-07-31T15:11:28.548 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/input/touchscreen/built-in.a 2024-07-31T15:11:28.565 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/rtc/rtc-mc146818-lib.o 2024-07-31T15:11:28.634 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/input/misc/built-in.a 2024-07-31T15:11:28.693 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/input.o 2024-07-31T15:11:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.720+0000 7fd08baa9640 1 -- 172.21.15.67:0/3358772722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd08411c5d0 msgr2=0x7fd08411e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.720+0000 7fd08baa9640 1 --2- 172.21.15.67:0/3358772722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd08411c5d0 0x7fd08411e9f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009be0 tx=0x7fd07c02a6c0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.721+0000 7fd08baa9640 1 -- 172.21.15.67:0/3358772722 shutdown_connections 2024-07-31T15:11:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.721+0000 7fd08baa9640 1 --2- 172.21.15.67:0/3358772722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd08411c5d0 0x7fd08411e9f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.721+0000 7fd08baa9640 1 --2- 172.21.15.67:0/3358772722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd08410a330 0x7fd08410a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.721+0000 7fd08baa9640 1 --2- 172.21.15.67:0/3358772722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0841116d0 0x7fd084109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.721+0000 7fd08baa9640 1 -- 172.21.15.67:0/3358772722 >> 172.21.15.67:0/3358772722 conn(0x7fd08406e350 msgr2=0x7fd08406e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.721+0000 7fd08baa9640 1 -- 172.21.15.67:0/3358772722 shutdown_connections 2024-07-31T15:11:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.721+0000 7fd08baa9640 1 -- 172.21.15.67:0/3358772722 wait complete. 2024-07-31T15:11:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.722+0000 7fd08baa9640 1 Processor -- start 2024-07-31T15:11:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.723+0000 7fd08baa9640 1 -- start start 2024-07-31T15:11:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.723+0000 7fd08baa9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd08410a330 0x7fd0841a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.723+0000 7fd08baa9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0841116d0 0x7fd0841a8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.723+0000 7fd08baa9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd08411c5d0 0x7fd0841ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.723+0000 7fd08baa9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841220c0 con 0x7fd0841116d0 2024-07-31T15:11:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.723+0000 7fd08baa9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd084121f40 con 0x7fd08410a330 2024-07-31T15:11:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.723+0000 7fd08baa9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd084122240 con 0x7fd08411c5d0 2024-07-31T15:11:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.723+0000 7fd089845640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd08410a330 0x7fd0841a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.723+0000 7fd089845640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd08410a330 0x7fd0841a8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40384/0 (socket says 172.21.15.67:40384) 2024-07-31T15:11:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.723+0000 7fd089845640 1 -- 172.21.15.67:0/1116439820 learned_addr learned my addr 172.21.15.67:0/1116439820 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.723+0000 7fd08a046640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd08411c5d0 0x7fd0841ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.723+0000 7fd089044640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0841116d0 0x7fd0841a8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.724+0000 7fd089845640 1 -- 172.21.15.67:0/1116439820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd08411c5d0 msgr2=0x7fd0841ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.724+0000 7fd089845640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd08411c5d0 0x7fd0841ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.724+0000 7fd089845640 1 -- 172.21.15.67:0/1116439820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0841116d0 msgr2=0x7fd0841a8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.724+0000 7fd089845640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0841116d0 0x7fd0841a8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.724+0000 7fd089845640 1 -- 172.21.15.67:0/1116439820 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0841ae950 con 0x7fd08410a330 2024-07-31T15:11:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.724+0000 7fd089044640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0841116d0 0x7fd0841a8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:11:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.724+0000 7fd089845640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd08410a330 0x7fd0841a8a20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd08000e940 tx=0x7fd08000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.724+0000 7fd07affd640 1 -- 172.21.15.67:0/1116439820 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08000ccc0 con 0x7fd08410a330 2024-07-31T15:11:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.724+0000 7fd08baa9640 1 -- 172.21.15.67:0/1116439820 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0841aec40 con 0x7fd08410a330 2024-07-31T15:11:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.725+0000 7fd08baa9640 1 -- 172.21.15.67:0/1116439820 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd084111db0 con 0x7fd08410a330 2024-07-31T15:11:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.725+0000 7fd07affd640 1 -- 172.21.15.67:0/1116439820 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0800054d0 con 0x7fd08410a330 2024-07-31T15:11:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.725+0000 7fd07affd640 1 -- 172.21.15.67:0/1116439820 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080010430 con 0x7fd08410a330 2024-07-31T15:11:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.729+0000 7fd0929a2740 1 Processor -- start 2024-07-31T15:11:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.730+0000 7fd0929a2740 1 -- start start 2024-07-31T15:11:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.733+0000 7fd07affd640 1 -- 172.21.15.67:0/1116439820 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd0800105d0 con 0x7fd08410a330 2024-07-31T15:11:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.734+0000 7fd0929a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc32ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.734+0000 7fd0929a2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c8672f0 0x55d96cc333e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.734+0000 7fd0929a2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c86f4d0 0x55d96cc33920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.734+0000 7fd0929a2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d96cba83b0 con 0x55d96c86f4d0 2024-07-31T15:11:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.734+0000 7fd0929a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d96cb978b0 con 0x55d96c8901b0 2024-07-31T15:11:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.734+0000 7fd0929a2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d96cb9b8f0 con 0x55d96c8672f0 2024-07-31T15:11:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.734+0000 7fd08a046640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc32ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.734+0000 7fd08a046640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc32ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:40390/0 (socket says 172.21.15.67:40390) 2024-07-31T15:11:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.734+0000 7fd08a046640 1 -- 172.21.15.67:0/863482802 learned_addr learned my addr 172.21.15.67:0/863482802 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.734+0000 7fd089044640 1 --2- 172.21.15.67:0/863482802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c86f4d0 0x55d96cc33920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.734+0000 7fd089845640 1 --2- 172.21.15.67:0/863482802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c8672f0 0x55d96cc333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.735+0000 7fd07affd640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd0700b3ed0 0x7fd0700b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.735+0000 7fd08a046640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd0700b3ed0 0x7fd0700b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.736+0000 7fd07affd640 1 -- 172.21.15.67:0/1116439820 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fd0800e43e0 con 0x7fd08410a330 2024-07-31T15:11:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.736+0000 7fd08a046640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd0700b3ed0 0x7fd0700b63e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009be0 tx=0x7fd07c030b10 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.736+0000 7fd089044640 1 -- 172.21.15.67:0/863482802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c8672f0 msgr2=0x55d96cc333e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.736+0000 7fd089044640 1 --2- 172.21.15.67:0/863482802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c8672f0 0x55d96cc333e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.736+0000 7fd089044640 1 -- 172.21.15.67:0/863482802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 msgr2=0x55d96cc32ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.736+0000 7fd089044640 1 --2- 172.21.15.67:0/863482802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc32ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.737+0000 7fd089044640 1 -- 172.21.15.67:0/863482802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96cb3e9a0 con 0x55d96c86f4d0 2024-07-31T15:11:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.737+0000 7fd08a046640 1 --2- 172.21.15.67:0/863482802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc32ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.737+0000 7fd089044640 1 --2- 172.21.15.67:0/863482802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c86f4d0 0x55d96cc33920 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd08406c170 tx=0x7fd0740117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfed3a295f58de8e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.738+0000 7fd06ffff640 1 -- 172.21.15.67:0/863482802 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd074012450 con 0x55d96c86f4d0 2024-07-31T15:11:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.738+0000 7fd06ffff640 1 -- 172.21.15.67:0/863482802 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd074012aa0 con 0x55d96c86f4d0 2024-07-31T15:11:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.739+0000 7fd06ffff640 1 -- 172.21.15.67:0/863482802 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07400f9f0 con 0x55d96c86f4d0 2024-07-31T15:11:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.741+0000 7fd0929a2740 1 -- 172.21.15.67:0/863482802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c86f4d0 msgr2=0x55d96cc33920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.741+0000 7fd0929a2740 1 --2- 172.21.15.67:0/863482802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c86f4d0 0x55d96cc33920 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd08406c170 tx=0x7fd0740117b0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.747+0000 7fd0929a2740 1 -- 172.21.15.67:0/863482802 shutdown_connections 2024-07-31T15:11:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.747+0000 7fd0929a2740 1 --2- 172.21.15.67:0/863482802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c86f4d0 0x55d96cc33920 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.747+0000 7fd0929a2740 1 --2- 172.21.15.67:0/863482802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c8672f0 0x55d96cc333e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.747+0000 7fd0929a2740 1 --2- 172.21.15.67:0/863482802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc32ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.747+0000 7fd0929a2740 1 -- 172.21.15.67:0/863482802 >> 172.21.15.67:0/863482802 conn(0x55d96c884300 msgr2=0x55d96cc30d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.747+0000 7fd0929a2740 1 -- 172.21.15.67:0/863482802 shutdown_connections 2024-07-31T15:11:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.747+0000 7fd0929a2740 1 -- 172.21.15.67:0/863482802 wait complete. 2024-07-31T15:11:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.748+0000 7fd0929a2740 1 -- start start 2024-07-31T15:11:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.748+0000 7fd0929a2740 1 Processor -- start 2024-07-31T15:11:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.748+0000 7fd0929a2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c8672f0 0x55d96cc3ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.748+0000 7fd0929a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c86f4d0 0x55d96cc3ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.749+0000 7fd0929a2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8901b0 0x55d96cc4a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.749+0000 7fd0929a2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d96cad1ab0 con 0x55d96c8901b0 2024-07-31T15:11:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.749+0000 7fd0929a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d96cb1fcf0 con 0x55d96c86f4d0 2024-07-31T15:11:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.749+0000 7fd0929a2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d96cc41620 con 0x55d96c8672f0 2024-07-31T15:11:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.749+0000 7fd0929a2740 10 client.0 _ll_register_callbacks cb 0x55d96cb919f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.749+0000 7fd089845640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c8672f0 0x55d96cc3ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.749+0000 7fd089845640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c8672f0 0x55d96cc3ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:56324/0 (socket says 172.21.15.67:56324) 2024-07-31T15:11:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.749+0000 7fd089845640 1 -- 172.21.15.67:0/2800038300 learned_addr learned my addr 172.21.15.67:0/2800038300 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.749+0000 7fd089044640 1 --2- 172.21.15.67:0/2800038300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c86f4d0 0x55d96cc3ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.749+0000 7fd08a046640 1 --2- 172.21.15.67:0/2800038300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8901b0 0x55d96cc4a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.749+0000 7fd089845640 1 -- 172.21.15.67:0/2800038300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c86f4d0 msgr2=0x55d96cc3ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.749+0000 7fd089845640 1 --2- 172.21.15.67:0/2800038300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c86f4d0 0x55d96cc3ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.749+0000 7fd089845640 1 -- 172.21.15.67:0/2800038300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8901b0 msgr2=0x55d96cc4a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.750+0000 7fd089845640 1 --2- 172.21.15.67:0/2800038300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8901b0 0x55d96cc4a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.750+0000 7fd089845640 1 -- 172.21.15.67:0/2800038300 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96c8fb790 con 0x55d96c8672f0 2024-07-31T15:11:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.750+0000 7fd08a046640 1 --2- 172.21.15.67:0/2800038300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8901b0 0x55d96cc4a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.751+0000 7fd089845640 1 --2- 172.21.15.67:0/2800038300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c8672f0 0x55d96cc3ea30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd0800e1dc0 tx=0x7fd0800aaaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b551cd4f45cb8da server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.751+0000 7fd06d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:11:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.751+0000 7fd06d7fa640 1 -- 172.21.15.67:0/2800038300 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0800ab5b0 con 0x55d96c8672f0 2024-07-31T15:11:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.751+0000 7fd0929a2740 1 -- 172.21.15.67:0/2800038300 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd074012aa0 con 0x55d96c8672f0 2024-07-31T15:11:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.751+0000 7fd06d7fa640 1 -- 172.21.15.67:0/2800038300 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fd0800b38a0 con 0x55d96c8672f0 2024-07-31T15:11:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.751+0000 7fd06d7fa640 1 -- 172.21.15.67:0/2800038300 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fd0800aac60 con 0x55d96c8672f0 2024-07-31T15:11:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.751+0000 7fd0929a2740 10 client.29143 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.751+0000 7fd0929a2740 1 -- 172.21.15.67:0/2800038300 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55d96c974620 con 0x55d96c8672f0 2024-07-31T15:11:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.752+0000 7fd06d7fa640 1 -- 172.21.15.67:0/2800038300 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd074012aa0 con 0x55d96c8672f0 2024-07-31T15:11:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.752+0000 7fd06d7fa640 1 -- 172.21.15.67:0/2800038300 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fd0800aa3b0 con 0x55d96c8672f0 2024-07-31T15:11:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.752+0000 7fd0929a2740 10 client.29143 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.752+0000 7fd0929a2740 10 client.29143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:11:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.752+0000 7fd0929a2740 20 client.29143 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.752+0000 7fd0929a2740 1 --2- 172.21.15.67:0/2800038300 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d96c84ed80 0x55d96cc50a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.752+0000 7fd0929a2740 4 client.29143 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:11:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.752+0000 7fd0929a2740 1 -- 172.21.15.67:0/2800038300 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d96cbaa300 con 0x55d96c84ed80 2024-07-31T15:11:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.752+0000 7fd089044640 1 --2- 172.21.15.67:0/2800038300 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d96c84ed80 0x55d96cc50a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.753+0000 7fd06d7fa640 10 client.29143 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.753+0000 7fd089044640 1 --2- 172.21.15.67:0/2800038300 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d96c84ed80 0x55d96cc50a20 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5f75e692ed5156f server_cookie=7d41b99556f8ee71 in_seq=0 out_seq=0 2024-07-31T15:11:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.754+0000 7fd06d7fa640 1 -- 172.21.15.67:0/2800038300 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d96cbaa300 con 0x55d96c84ed80 2024-07-31T15:11:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.754+0000 7fd06d7fa640 10 client.29143 handle_command_reply: tid=0 2024-07-31T15:11:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.754+0000 7fd0929a2740 1 client.29143 shutdown 2024-07-31T15:11:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.754+0000 7fd0929a2740 1 -- 172.21.15.67:0/2800038300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c8672f0 msgr2=0x55d96cc3ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.754+0000 7fd0929a2740 1 --2- 172.21.15.67:0/2800038300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c8672f0 0x55d96cc3ea30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd0800e1dc0 tx=0x7fd0800aaaa0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.756+0000 7fd0929a2740 1 -- 172.21.15.67:0/2800038300 shutdown_connections 2024-07-31T15:11:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.756+0000 7fd0929a2740 1 --2- 172.21.15.67:0/2800038300 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d96c84ed80 0x55d96cc50a20 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.756+0000 7fd0929a2740 1 --2- 172.21.15.67:0/2800038300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8901b0 0x55d96cc4a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.756+0000 7fd0929a2740 1 --2- 172.21.15.67:0/2800038300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c86f4d0 0x55d96cc3ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.756+0000 7fd0929a2740 1 --2- 172.21.15.67:0/2800038300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c8672f0 0x55d96cc3ea30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.756+0000 7fd06d7fa640 0 client.29143 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.756+0000 7fd0929a2740 1 -- 172.21.15.67:0/2800038300 >> 172.21.15.67:0/2800038300 conn(0x55d96c884300 msgr2=0x55d96cc33ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.756+0000 7fd0929a2740 1 -- 172.21.15.67:0/2800038300 shutdown_connections 2024-07-31T15:11:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.756+0000 7fd0929a2740 1 -- 172.21.15.67:0/2800038300 wait complete. 2024-07-31T15:11:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.756+0000 7fd0929a2740 20 client.29143 trim_cache size 0 max 16384 2024-07-31T15:11:28.760 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/i2c/i2c-core-base.o 2024-07-31T15:11:28.782 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/generic.o 2024-07-31T15:11:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.792+0000 7fd0929a2740 1 Processor -- start 2024-07-31T15:11:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.793+0000 7fd0929a2740 1 -- start start 2024-07-31T15:11:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.793+0000 7fd0929a2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8672f0 0x55d96cc3d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.793+0000 7fd0929a2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c86f4d0 0x55d96cc38f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.793+0000 7fd0929a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc3eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.793+0000 7fd0929a2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd06800fda0 con 0x55d96c8672f0 2024-07-31T15:11:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.793+0000 7fd0929a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd0680118e0 con 0x55d96c8901b0 2024-07-31T15:11:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.793+0000 7fd0929a2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd068010470 con 0x55d96c86f4d0 2024-07-31T15:11:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.793+0000 7fd08a046640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc3eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.794+0000 7fd089845640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8672f0 0x55d96cc3d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.794+0000 7fd089845640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8672f0 0x55d96cc3d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59120/0 (socket says 172.21.15.67:59120) 2024-07-31T15:11:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.794+0000 7fd089845640 1 -- 172.21.15.67:0/636301936 learned_addr learned my addr 172.21.15.67:0/636301936 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.794+0000 7fd089044640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c86f4d0 0x55d96cc38f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.794+0000 7fd08a046640 1 -- 172.21.15.67:0/636301936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c86f4d0 msgr2=0x55d96cc38f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.794+0000 7fd08a046640 1 --2- 172.21.15.67:0/636301936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c86f4d0 0x55d96cc38f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.794+0000 7fd08a046640 1 -- 172.21.15.67:0/636301936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8672f0 msgr2=0x55d96cc3d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.794+0000 7fd08a046640 1 --2- 172.21.15.67:0/636301936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8672f0 0x55d96cc3d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.794+0000 7fd08a046640 1 -- 172.21.15.67:0/636301936 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96cc3c790 con 0x55d96c8901b0 2024-07-31T15:11:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.795+0000 7fd089845640 1 --2- 172.21.15.67:0/636301936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8672f0 0x55d96cc3d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.795+0000 7fd08a046640 1 --2- 172.21.15.67:0/636301936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc3eea0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009b30 tx=0x7fd07c03e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd37979b44a986cb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.796+0000 7fd06f7fe640 1 -- 172.21.15.67:0/636301936 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c036040 con 0x55d96c8901b0 2024-07-31T15:11:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.796+0000 7fd06f7fe640 1 -- 172.21.15.67:0/636301936 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd07c004370 con 0x55d96c8901b0 2024-07-31T15:11:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.796+0000 7fd06f7fe640 1 -- 172.21.15.67:0/636301936 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0319d0 con 0x55d96c8901b0 2024-07-31T15:11:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.797+0000 7fd0929a2740 1 -- 172.21.15.67:0/636301936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 msgr2=0x55d96cc3eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.797+0000 7fd0929a2740 1 --2- 172.21.15.67:0/636301936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc3eea0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009b30 tx=0x7fd07c03e000 comp rx=0 tx=0).stop 2024-07-31T15:11:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.798+0000 7fd0929a2740 1 -- 172.21.15.67:0/636301936 shutdown_connections 2024-07-31T15:11:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.798+0000 7fd0929a2740 1 --2- 172.21.15.67:0/636301936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc3eea0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.798+0000 7fd0929a2740 1 --2- 172.21.15.67:0/636301936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c86f4d0 0x55d96cc38f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.799+0000 7fd0929a2740 1 --2- 172.21.15.67:0/636301936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8672f0 0x55d96cc3d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.799+0000 7fd0929a2740 1 -- 172.21.15.67:0/636301936 >> 172.21.15.67:0/636301936 conn(0x55d96c884300 msgr2=0x55d96cc3e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.799+0000 7fd0929a2740 1 -- 172.21.15.67:0/636301936 shutdown_connections 2024-07-31T15:11:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.799+0000 7fd0929a2740 1 -- 172.21.15.67:0/636301936 wait complete. 2024-07-31T15:11:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.800+0000 7fd0929a2740 1 -- start start 2024-07-31T15:11:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.800+0000 7fd0929a2740 1 Processor -- start 2024-07-31T15:11:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.803+0000 7fd0929a2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8672f0 0x55d96cc47140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.803+0000 7fd0929a2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c86f4d0 0x55d96cc438a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.803+0000 7fd0929a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc43f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.803+0000 7fd089845640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8672f0 0x55d96cc47140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.803+0000 7fd0929a2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd068011210 con 0x55d96c8672f0 2024-07-31T15:11:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.803+0000 7fd089845640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8672f0 0x55d96cc47140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59136/0 (socket says 172.21.15.67:59136) 2024-07-31T15:11:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.804+0000 7fd089845640 1 -- 172.21.15.67:0/1163365391 learned_addr learned my addr 172.21.15.67:0/1163365391 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.804+0000 7fd0929a2740 1 -- 172.21.15.67:0/1163365391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd068010b40 con 0x55d96c8901b0 2024-07-31T15:11:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.804+0000 7fd089044640 1 --2- 172.21.15.67:0/1163365391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c86f4d0 0x55d96cc438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.804+0000 7fd08a046640 1 --2- 172.21.15.67:0/1163365391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc43f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.804+0000 7fd0929a2740 1 -- 172.21.15.67:0/1163365391 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd06800dda0 con 0x55d96c86f4d0 2024-07-31T15:11:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.804+0000 7fd089044640 1 -- 172.21.15.67:0/1163365391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 msgr2=0x55d96cc43f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.804+0000 7fd089044640 1 --2- 172.21.15.67:0/1163365391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc43f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.804+0000 7fd089044640 1 -- 172.21.15.67:0/1163365391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8672f0 msgr2=0x55d96cc47140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.804+0000 7fd089044640 1 --2- 172.21.15.67:0/1163365391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8672f0 0x55d96cc47140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.804+0000 7fd089044640 1 -- 172.21.15.67:0/1163365391 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96ccb71a0 con 0x55d96c86f4d0 2024-07-31T15:11:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.804+0000 7fd089845640 1 --2- 172.21.15.67:0/1163365391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8672f0 0x55d96cc47140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.805+0000 7fd0929a2740 10 client.0 _ll_register_callbacks cb 0x7fd08000ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.805+0000 7fd0929a2740 1 -- 172.21.15.67:0/1163365391 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd07c004370 con 0x55d96c86f4d0 2024-07-31T15:11:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.805+0000 7fd089044640 1 --2- 172.21.15.67:0/1163365391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c86f4d0 0x55d96cc438a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd074000c00 tx=0x7fd0740057d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de8699e8b3ae2d19 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.812+0000 7fd06d7fa640 10 client.29149 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:11:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.812+0000 7fd06d7fa640 1 -- 172.21.15.67:0/1163365391 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd074004250 con 0x55d96c86f4d0 2024-07-31T15:11:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.812+0000 7fd06d7fa640 1 -- 172.21.15.67:0/1163365391 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fd074020040 con 0x55d96c86f4d0 2024-07-31T15:11:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.812+0000 7fd06d7fa640 1 -- 172.21.15.67:0/1163365391 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fd0740043f0 con 0x55d96c86f4d0 2024-07-31T15:11:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.812+0000 7fd0929a2740 10 client.29149 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.812+0000 7fd0929a2740 1 -- 172.21.15.67:0/1163365391 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=66}) -- 0x55d96ccb7f90 con 0x55d96c86f4d0 2024-07-31T15:11:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.823+0000 7fd06d7fa640 1 -- 172.21.15.67:0/1163365391 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fd0740035f0 con 0x55d96c86f4d0 2024-07-31T15:11:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.823+0000 7fd0929a2740 10 client.29149 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.823+0000 7fd0929a2740 10 client.29149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:11:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.823+0000 7fd0929a2740 20 client.29149 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.823+0000 7fd0929a2740 1 --2- 172.21.15.67:0/1163365391 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d96c8241c0 0x55d96ccba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.823+0000 7fd0929a2740 4 client.29149 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:11:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.823+0000 7fd0929a2740 1 -- 172.21.15.67:0/1163365391 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d96cc3a360 con 0x55d96c8241c0 2024-07-31T15:11:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.823+0000 7fd089845640 1 --2- 172.21.15.67:0/1163365391 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d96c8241c0 0x55d96ccba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.824+0000 7fd089845640 1 --2- 172.21.15.67:0/1163365391 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d96c8241c0 0x55d96ccba4a0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb1b9ee9b877b70 server_cookie=1fb57a2587f7232a in_seq=0 out_seq=0 2024-07-31T15:11:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:28.824+0000 7fd06d7fa640 10 client.29149 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:29.039 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:29 smithi067 ceph-mon[32750]: pgmap v1052: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 906 KiB/s wr, 664 op/s 2024-07-31T15:11:29.392 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/eht.o 2024-07-31T15:11:29.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:29 smithi073 ceph-mon[36714]: pgmap v1052: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 906 KiB/s wr, 664 op/s 2024-07-31T15:11:29.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:29 smithi077 ceph-mon[37327]: pgmap v1052: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 906 KiB/s wr, 664 op/s 2024-07-31T15:11:29.903 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/quirks.o 2024-07-31T15:11:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.943+0000 7fd06d7fa640 1 -- 172.21.15.67:0/1163365391 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d96cc3a360 con 0x55d96c8241c0 2024-07-31T15:11:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.943+0000 7fd06d7fa640 10 client.29149 handle_command_reply: tid=0 2024-07-31T15:11:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.943+0000 7fd0929a2740 1 client.29149 shutdown 2024-07-31T15:11:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.943+0000 7fd0929a2740 1 -- 172.21.15.67:0/1163365391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c86f4d0 msgr2=0x55d96cc438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.943+0000 7fd0929a2740 1 --2- 172.21.15.67:0/1163365391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c86f4d0 0x55d96cc438a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd074000c00 tx=0x7fd0740057d0 comp rx=0 tx=0).stop 2024-07-31T15:11:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.943+0000 7fd0929a2740 1 -- 172.21.15.67:0/1163365391 shutdown_connections 2024-07-31T15:11:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.943+0000 7fd0929a2740 1 --2- 172.21.15.67:0/1163365391 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d96c8241c0 0x55d96ccba4a0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.944+0000 7fd0929a2740 1 --2- 172.21.15.67:0/1163365391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d96c8901b0 0x55d96cc43f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.944+0000 7fd0929a2740 1 --2- 172.21.15.67:0/1163365391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d96c86f4d0 0x55d96cc438a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.944+0000 7fd0929a2740 1 --2- 172.21.15.67:0/1163365391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d96c8672f0 0x55d96cc47140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.944+0000 7fd0929a2740 1 -- 172.21.15.67:0/1163365391 >> 172.21.15.67:0/1163365391 conn(0x55d96c884300 msgr2=0x55d96cc33ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.944+0000 7fd06d7fa640 0 client.29149 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.944+0000 7fd0929a2740 1 -- 172.21.15.67:0/1163365391 shutdown_connections 2024-07-31T15:11:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.944+0000 7fd0929a2740 1 -- 172.21.15.67:0/1163365391 wait complete. 2024-07-31T15:11:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.944+0000 7fd0929a2740 20 client.29149 trim_cache size 0 max 16384 2024-07-31T15:11:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:11:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:11:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:11:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:11:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.945+0000 7fd08baa9640 1 -- 172.21.15.67:0/1116439820 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd0700b3ed0 msgr2=0x7fd0700b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.945+0000 7fd08baa9640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd0700b3ed0 0x7fd0700b63e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009be0 tx=0x7fd07c030b10 comp rx=0 tx=0).stop 2024-07-31T15:11:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.945+0000 7fd08baa9640 1 -- 172.21.15.67:0/1116439820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd08410a330 msgr2=0x7fd0841a8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.945+0000 7fd08baa9640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd08410a330 0x7fd0841a8a20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd08000e940 tx=0x7fd08000ee00 comp rx=0 tx=0).stop 2024-07-31T15:11:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.946+0000 7fd08baa9640 1 -- 172.21.15.67:0/1116439820 shutdown_connections 2024-07-31T15:11:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.946+0000 7fd08baa9640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd0700b3ed0 0x7fd0700b63e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.946+0000 7fd08baa9640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd08411c5d0 0x7fd0841ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.946+0000 7fd08baa9640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0841116d0 0x7fd0841a8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.946+0000 7fd08baa9640 1 --2- 172.21.15.67:0/1116439820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd08410a330 0x7fd0841a8a20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.946+0000 7fd08baa9640 1 -- 172.21.15.67:0/1116439820 >> 172.21.15.67:0/1116439820 conn(0x7fd08406e350 msgr2=0x7fd084110350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.946+0000 7fd08baa9640 1 -- 172.21.15.67:0/1116439820 shutdown_connections 2024-07-31T15:11:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:29.946+0000 7fd08baa9640 1 -- 172.21.15.67:0/1116439820 wait complete. 2024-07-31T15:11:29.953 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/xhci-hub.o 2024-07-31T15:11:29.965 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000e/ethtool.o 2024-07-31T15:11:29.989 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_cmd_parser.o 2024-07-31T15:11:30.034 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/rtc/rtc-cmos.o 2024-07-31T15:11:30.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:30 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:11:30.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:30 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:11:30.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:30 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:11:30.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:30 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:11:30.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:30 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:11:30.386 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/led.o 2024-07-31T15:11:30.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:30 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:11:30.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:30 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:11:30.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:30 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:11:30.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:30 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:11:30.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:30 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:11:30.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:30 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:11:30.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:30 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:11:30.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:30 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:11:30.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:30 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:11:30.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:30 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:11:30.588 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/devices.o 2024-07-31T15:11:30.699 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/input-compat.o 2024-07-31T15:11:30.722 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/i2c/i2c-core-smbus.o 2024-07-31T15:11:30.900 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/rtc/built-in.a 2024-07-31T15:11:30.957 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/input-mt.o 2024-07-31T15:11:31.081 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_deps.o 2024-07-31T15:11:31.175 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:31 smithi067 ceph-mon[32750]: pgmap v1053: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 935 KiB/s wr, 470 op/s 2024-07-31T15:11:31.296 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/i3c/built-in.a 2024-07-31T15:11:31.325 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/input-poller.o 2024-07-31T15:11:31.360 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/phy.o 2024-07-31T15:11:31.385 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/pm.o 2024-07-31T15:11:31.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:31 smithi073 ceph-mon[36714]: pgmap v1053: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 935 KiB/s wr, 470 op/s 2024-07-31T15:11:31.404 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/xhci-dbg.o 2024-07-31T15:11:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:31 smithi077 ceph-mon[37327]: pgmap v1053: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 935 KiB/s wr, 470 op/s 2024-07-31T15:11:31.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.540+0000 7f1397ba4640 1 -- 172.21.15.67:0/1919778603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f139011c660 msgr2=0x7f139011eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:31.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.540+0000 7f1397ba4640 1 --2- 172.21.15.67:0/1919778603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f139011c660 0x7f139011eb70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f138801acb0 tx=0x7f138803b900 comp rx=0 tx=0).stop 2024-07-31T15:11:31.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.543+0000 7f1397ba4640 1 -- 172.21.15.67:0/1919778603 shutdown_connections 2024-07-31T15:11:31.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.543+0000 7f1397ba4640 1 --2- 172.21.15.67:0/1919778603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f139011c660 0x7f139011eb70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.543+0000 7f1397ba4640 1 --2- 172.21.15.67:0/1919778603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f139010a410 0x7f139010a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.543+0000 7f1397ba4640 1 --2- 172.21.15.67:0/1919778603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1390111970 0x7f1390109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.543+0000 7f1397ba4640 1 -- 172.21.15.67:0/1919778603 >> 172.21.15.67:0/1919778603 conn(0x7f13900708f0 msgr2=0x7f1390070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:31.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.554+0000 7f1397ba4640 1 -- 172.21.15.67:0/1919778603 shutdown_connections 2024-07-31T15:11:31.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.554+0000 7f1397ba4640 1 -- 172.21.15.67:0/1919778603 wait complete. 2024-07-31T15:11:31.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.555+0000 7f1397ba4640 1 Processor -- start 2024-07-31T15:11:31.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.559+0000 7f1397ba4640 1 -- start start 2024-07-31T15:11:31.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.559+0000 7f1397ba4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f139010a410 0x7f139011bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.560+0000 7f1397ba4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1390111970 0x7f1390118080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.560+0000 7f1397ba4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f139011c660 0x7f13901185c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.560+0000 7f1397ba4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f13901145f0 con 0x7f139011c660 2024-07-31T15:11:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.560+0000 7f1397ba4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1390114470 con 0x7f139010a410 2024-07-31T15:11:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.560+0000 7f1396141640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f139011c660 0x7f13901185c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.561+0000 7f1396141640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f139011c660 0x7f13901185c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43332/0 (socket says 172.21.15.67:43332) 2024-07-31T15:11:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.561+0000 7f1397ba4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1390114770 con 0x7f1390111970 2024-07-31T15:11:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.561+0000 7f1396141640 1 -- 172.21.15.67:0/659046756 learned_addr learned my addr 172.21.15.67:0/659046756 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:31.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.565+0000 7f139513f640 1 --2- 172.21.15.67:0/659046756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1390111970 0x7f1390118080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.565+0000 7f1395940640 1 --2- 172.21.15.67:0/659046756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f139010a410 0x7f139011bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.568+0000 7f1396141640 1 -- 172.21.15.67:0/659046756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1390111970 msgr2=0x7f1390118080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:31.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.568+0000 7f1396141640 1 --2- 172.21.15.67:0/659046756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1390111970 0x7f1390118080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.568+0000 7f1396141640 1 -- 172.21.15.67:0/659046756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f139010a410 msgr2=0x7f139011bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:31.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.568+0000 7f1396141640 1 --2- 172.21.15.67:0/659046756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f139010a410 0x7f139011bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.568+0000 7f1396141640 1 -- 172.21.15.67:0/659046756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1390118cd0 con 0x7f139011c660 2024-07-31T15:11:31.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.569+0000 7f1396141640 1 --2- 172.21.15.67:0/659046756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f139011c660 0x7f13901185c0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f1388040d10 tx=0x7f1388040d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:31.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.574+0000 7f1386ffd640 1 -- 172.21.15.67:0/659046756 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388045070 con 0x7f139011c660 2024-07-31T15:11:31.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.574+0000 7f1386ffd640 1 -- 172.21.15.67:0/659046756 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1388040e60 con 0x7f139011c660 2024-07-31T15:11:31.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.574+0000 7f1386ffd640 1 -- 172.21.15.67:0/659046756 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138804a7b0 con 0x7f139011c660 2024-07-31T15:11:31.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.576+0000 7f1397ba4640 1 -- 172.21.15.67:0/659046756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13901ac800 con 0x7f139011c660 2024-07-31T15:11:31.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.576+0000 7f1397ba4640 1 -- 172.21.15.67:0/659046756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13901acba0 con 0x7f139011c660 2024-07-31T15:11:31.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.580+0000 7f139e0f5740 1 Processor -- start 2024-07-31T15:11:31.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.582+0000 7f139e0f5740 1 -- start start 2024-07-31T15:11:31.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.582+0000 7f139e0f5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 0x55e27fa29550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.582+0000 7f139e0f5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f65b1b0 0x55e27fa29a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.582+0000 7f139e0f5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f663390 0x55e27fa29fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.582+0000 7f139e0f5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e27f99fb90 con 0x55e27f684070 2024-07-31T15:11:31.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.582+0000 7f139e0f5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e27f98f090 con 0x55e27f663390 2024-07-31T15:11:31.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.582+0000 7f139e0f5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e27f9930d0 con 0x55e27f65b1b0 2024-07-31T15:11:31.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.583+0000 7f1386ffd640 1 -- 172.21.15.67:0/659046756 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1388048020 con 0x7f139011c660 2024-07-31T15:11:31.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.584+0000 7f139513f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 0x55e27fa29550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.584+0000 7f139513f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 0x55e27fa29550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43338/0 (socket says 172.21.15.67:43338) 2024-07-31T15:11:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.584+0000 7f139513f640 1 -- 172.21.15.67:0/1105306935 learned_addr learned my addr 172.21.15.67:0/1105306935 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.584+0000 7f1386ffd640 1 --2- 172.21.15.67:0/659046756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f137c0b3f50 0x7f137c0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.584+0000 7f1395940640 1 --2- 172.21.15.67:0/1105306935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f65b1b0 0x55e27fa29a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.584+0000 7f139513f640 1 --2- 172.21.15.67:0/1105306935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f663390 0x55e27fa29fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.584+0000 7f1386ffd640 1 -- 172.21.15.67:0/659046756 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7f1388116320 con 0x7f139011c660 2024-07-31T15:11:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.584+0000 7f139513f640 1 -- 172.21.15.67:0/1105306935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f65b1b0 msgr2=0x55e27fa29a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.584+0000 7f139513f640 1 --2- 172.21.15.67:0/1105306935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f65b1b0 0x55e27fa29a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.584+0000 7f139513f640 1 -- 172.21.15.67:0/1105306935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f663390 msgr2=0x55e27fa29fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.584+0000 7f139513f640 1 --2- 172.21.15.67:0/1105306935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f663390 0x55e27fa29fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.584+0000 7f139513f640 1 -- 172.21.15.67:0/1105306935 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e27f935b70 con 0x55e27f684070 2024-07-31T15:11:31.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.587+0000 7f139513f640 1 --2- 172.21.15.67:0/1105306935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 0x55e27fa29550 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f138000af90 tx=0x7f138000a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588c5cc26c0bb926 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:31.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.588+0000 7f1396141640 1 --2- 172.21.15.67:0/659046756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f137c0b3f50 0x7f137c0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.589+0000 7f1396141640 1 --2- 172.21.15.67:0/659046756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f137c0b3f50 0x7f137c0b6460 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1388040380 tx=0x7f138804d040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:31.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.589+0000 7f137bfff640 1 -- 172.21.15.67:0/1105306935 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380019030 con 0x55e27f684070 2024-07-31T15:11:31.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.590+0000 7f137bfff640 1 -- 172.21.15.67:0/1105306935 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13800054d0 con 0x55e27f684070 2024-07-31T15:11:31.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.590+0000 7f137bfff640 1 -- 172.21.15.67:0/1105306935 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380011520 con 0x55e27f684070 2024-07-31T15:11:31.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.591+0000 7f139e0f5740 1 -- 172.21.15.67:0/1105306935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 msgr2=0x55e27fa29550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:31.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.591+0000 7f139e0f5740 1 --2- 172.21.15.67:0/1105306935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 0x55e27fa29550 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f138000af90 tx=0x7f138000a780 comp rx=0 tx=0).stop 2024-07-31T15:11:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.595+0000 7f139e0f5740 1 -- 172.21.15.67:0/1105306935 shutdown_connections 2024-07-31T15:11:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.595+0000 7f139e0f5740 1 --2- 172.21.15.67:0/1105306935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f663390 0x55e27fa29fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.595+0000 7f139e0f5740 1 --2- 172.21.15.67:0/1105306935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f65b1b0 0x55e27fa29a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.595+0000 7f139e0f5740 1 --2- 172.21.15.67:0/1105306935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 0x55e27fa29550 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.595+0000 7f139e0f5740 1 -- 172.21.15.67:0/1105306935 >> 172.21.15.67:0/1105306935 conn(0x55e27f678460 msgr2=0x55e27fa27440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:31.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.603+0000 7f139e0f5740 1 -- 172.21.15.67:0/1105306935 shutdown_connections 2024-07-31T15:11:31.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.607+0000 7f139e0f5740 1 -- 172.21.15.67:0/1105306935 wait complete. 2024-07-31T15:11:31.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.607+0000 7f139e0f5740 1 -- start start 2024-07-31T15:11:31.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.607+0000 7f139e0f5740 1 Processor -- start 2024-07-31T15:11:31.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.612+0000 7f139e0f5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f65b1b0 0x55e27fa35130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.612+0000 7f139e0f5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f663390 0x55e27fa35690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.612+0000 7f139e0f5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 0x55e27fa41020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.612+0000 7f139e0f5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e27f7f2700 con 0x55e27f684070 2024-07-31T15:11:31.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.612+0000 7f139e0f5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e27f9319c0 con 0x55e27f663390 2024-07-31T15:11:31.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.612+0000 7f139e0f5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e27fa37d30 con 0x55e27f65b1b0 2024-07-31T15:11:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.612+0000 7f139e0f5740 10 client.0 _ll_register_callbacks cb 0x55e27f94c4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.613+0000 7f1395940640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f663390 0x55e27fa35690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.613+0000 7f139513f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f65b1b0 0x55e27fa35130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.613+0000 7f139513f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f65b1b0 0x55e27fa35130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:54428/0 (socket says 172.21.15.67:54428) 2024-07-31T15:11:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.613+0000 7f139513f640 1 -- 172.21.15.67:0/882615393 learned_addr learned my addr 172.21.15.67:0/882615393 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.613+0000 7f139513f640 1 --2- 172.21.15.67:0/882615393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 0x55e27fa41020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.614+0000 7f139513f640 1 -- 172.21.15.67:0/882615393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f663390 msgr2=0x55e27fa35690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.614+0000 7f139513f640 1 --2- 172.21.15.67:0/882615393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f663390 0x55e27fa35690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.614+0000 7f139513f640 1 -- 172.21.15.67:0/882615393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 msgr2=0x55e27fa41020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.614+0000 7f139513f640 1 --2- 172.21.15.67:0/882615393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 0x55e27fa41020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.614+0000 7f139513f640 1 -- 172.21.15.67:0/882615393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e27f6ef650 con 0x55e27f65b1b0 2024-07-31T15:11:31.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.619+0000 7f139513f640 1 --2- 172.21.15.67:0/882615393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f65b1b0 0x55e27fa35130 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f138000a660 tx=0x7f1380004b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2a640b5372f6000 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:31.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.619+0000 7f13797fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:11:31.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.623+0000 7f13797fa640 1 -- 172.21.15.67:0/882615393 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380019040 con 0x55e27f65b1b0 2024-07-31T15:11:31.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.623+0000 7f139e0f5740 1 -- 172.21.15.67:0/882615393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13800054d0 con 0x55e27f65b1b0 2024-07-31T15:11:31.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.623+0000 7f13797fa640 1 -- 172.21.15.67:0/882615393 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7f1380005000 con 0x55e27f65b1b0 2024-07-31T15:11:31.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.623+0000 7f13797fa640 1 -- 172.21.15.67:0/882615393 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f1380032050 con 0x55e27f65b1b0 2024-07-31T15:11:31.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.625+0000 7f139e0f5740 10 client.29161 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:31.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.625+0000 7f139e0f5740 1 -- 172.21.15.67:0/882615393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55e27fa43d70 con 0x55e27f65b1b0 2024-07-31T15:11:31.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.628+0000 7f13797fa640 1 -- 172.21.15.67:0/882615393 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13800054d0 con 0x55e27f65b1b0 2024-07-31T15:11:31.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.628+0000 7f13797fa640 1 -- 172.21.15.67:0/882615393 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f1380011cb0 con 0x55e27f65b1b0 2024-07-31T15:11:31.633 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000e/netdev.o 2024-07-31T15:11:31.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.632+0000 7f139e0f5740 10 client.29161 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:31.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.632+0000 7f139e0f5740 10 client.29161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:11:31.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.632+0000 7f139e0f5740 20 client.29161 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:31.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.632+0000 7f139e0f5740 1 --2- 172.21.15.67:0/882615393 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e27f6429f0 0x55e27fa47180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.632+0000 7f139e0f5740 4 client.29161 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:11:31.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.632+0000 7f139e0f5740 1 -- 172.21.15.67:0/882615393 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e27f9a1ae0 con 0x55e27f6429f0 2024-07-31T15:11:31.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.637+0000 7f1395940640 1 --2- 172.21.15.67:0/882615393 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e27f6429f0 0x55e27fa47180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.641+0000 7f1395940640 1 --2- 172.21.15.67:0/882615393 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e27f6429f0 0x55e27fa47180 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b39b01063c7c109 server_cookie=80bcc6dbd0561d8b in_seq=0 out_seq=0 2024-07-31T15:11:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.642+0000 7f13797fa640 10 client.29161 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.646+0000 7f13797fa640 1 -- 172.21.15.67:0/882615393 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e27f9a1ae0 con 0x55e27f6429f0 2024-07-31T15:11:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.646+0000 7f13797fa640 10 client.29161 handle_command_reply: tid=0 2024-07-31T15:11:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.646+0000 7f139e0f5740 1 client.29161 shutdown 2024-07-31T15:11:31.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.648+0000 7f139e0f5740 1 -- 172.21.15.67:0/882615393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f65b1b0 msgr2=0x55e27fa35130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:31.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.648+0000 7f139e0f5740 1 --2- 172.21.15.67:0/882615393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f65b1b0 0x55e27fa35130 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f138000a660 tx=0x7f1380004b30 comp rx=0 tx=0).stop 2024-07-31T15:11:31.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.648+0000 7f139e0f5740 1 -- 172.21.15.67:0/882615393 shutdown_connections 2024-07-31T15:11:31.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.648+0000 7f139e0f5740 1 --2- 172.21.15.67:0/882615393 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e27f6429f0 0x55e27fa47180 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.649+0000 7f139e0f5740 1 --2- 172.21.15.67:0/882615393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 0x55e27fa41020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.649+0000 7f139e0f5740 1 --2- 172.21.15.67:0/882615393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f663390 0x55e27fa35690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.649+0000 7f139e0f5740 1 --2- 172.21.15.67:0/882615393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f65b1b0 0x55e27fa35130 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.649+0000 7f139e0f5740 1 -- 172.21.15.67:0/882615393 >> 172.21.15.67:0/882615393 conn(0x55e27f678460 msgr2=0x55e27fa25c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.649+0000 7f13797fa640 0 client.29161 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.649+0000 7f139e0f5740 1 -- 172.21.15.67:0/882615393 shutdown_connections 2024-07-31T15:11:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.649+0000 7f139e0f5740 1 -- 172.21.15.67:0/882615393 wait complete. 2024-07-31T15:11:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.649+0000 7f139e0f5740 20 client.29161 trim_cache size 0 max 16384 2024-07-31T15:11:31.665 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/port.o 2024-07-31T15:11:31.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.693+0000 7f139e0f5740 1 Processor -- start 2024-07-31T15:11:31.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.693+0000 7f139e0f5740 1 -- start start 2024-07-31T15:11:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.693+0000 7f139e0f5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f65b1b0 0x55e27fa40e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.693+0000 7f139e0f5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f663390 0x55e27fa2e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.693+0000 7f139e0f5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 0x55e27fa2eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.693+0000 7f139e0f5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f13901b6dd0 con 0x55e27f684070 2024-07-31T15:11:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.693+0000 7f139e0f5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f13901b8910 con 0x55e27f65b1b0 2024-07-31T15:11:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.693+0000 7f139e0f5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13901b74a0 con 0x55e27f663390 2024-07-31T15:11:31.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.695+0000 7f139513f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f65b1b0 0x55e27fa40e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.695+0000 7f139513f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f65b1b0 0x55e27fa40e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:55374/0 (socket says 172.21.15.67:55374) 2024-07-31T15:11:31.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.695+0000 7f139513f640 1 -- 172.21.15.67:0/1816787601 learned_addr learned my addr 172.21.15.67:0/1816787601 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.695+0000 7f139513f640 1 --2- 172.21.15.67:0/1816787601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 0x55e27fa2eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.695+0000 7f139513f640 1 -- 172.21.15.67:0/1816787601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f663390 msgr2=0x55e27fa2e660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:11:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.695+0000 7f139513f640 1 --2- 172.21.15.67:0/1816787601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f663390 0x55e27fa2e660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.695+0000 7f139513f640 1 -- 172.21.15.67:0/1816787601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 msgr2=0x55e27fa2eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.695+0000 7f139513f640 1 --2- 172.21.15.67:0/1816787601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 0x55e27fa2eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.695+0000 7f139513f640 1 -- 172.21.15.67:0/1816787601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e27fa2c3b0 con 0x55e27f65b1b0 2024-07-31T15:11:31.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.702+0000 7f139513f640 1 --2- 172.21.15.67:0/1816787601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f65b1b0 0x55e27fa40e10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f138000ac80 tx=0x7f1380000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92852cc6006aa512 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:31.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.708+0000 7f137b7fe640 1 -- 172.21.15.67:0/1816787601 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380019040 con 0x55e27f65b1b0 2024-07-31T15:11:31.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.708+0000 7f137b7fe640 1 -- 172.21.15.67:0/1816787601 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f138000a740 con 0x55e27f65b1b0 2024-07-31T15:11:31.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.708+0000 7f137b7fe640 1 -- 172.21.15.67:0/1816787601 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138002e930 con 0x55e27f65b1b0 2024-07-31T15:11:31.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.710+0000 7f139e0f5740 1 -- 172.21.15.67:0/1816787601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f65b1b0 msgr2=0x55e27fa40e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:31.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.710+0000 7f139e0f5740 1 --2- 172.21.15.67:0/1816787601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f65b1b0 0x55e27fa40e10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f138000ac80 tx=0x7f1380000bc0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.714+0000 7f139e0f5740 1 -- 172.21.15.67:0/1816787601 shutdown_connections 2024-07-31T15:11:31.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.714+0000 7f139e0f5740 1 --2- 172.21.15.67:0/1816787601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f684070 0x55e27fa2eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.714+0000 7f139e0f5740 1 --2- 172.21.15.67:0/1816787601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f663390 0x55e27fa2e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.714+0000 7f139e0f5740 1 --2- 172.21.15.67:0/1816787601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f65b1b0 0x55e27fa40e10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.714+0000 7f139e0f5740 1 -- 172.21.15.67:0/1816787601 >> 172.21.15.67:0/1816787601 conn(0x55e27f678460 msgr2=0x55e27fa25c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:31.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.719+0000 7f139e0f5740 1 -- 172.21.15.67:0/1816787601 shutdown_connections 2024-07-31T15:11:31.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.720+0000 7f139e0f5740 1 -- 172.21.15.67:0/1816787601 wait complete. 2024-07-31T15:11:31.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.720+0000 7f139e0f5740 1 -- start start 2024-07-31T15:11:31.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.721+0000 7f139e0f5740 1 Processor -- start 2024-07-31T15:11:31.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.721+0000 7f139e0f5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f65b1b0 0x55e27fa38810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.721+0000 7f139e0f5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f663390 0x55e27fa38dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.721+0000 7f139e0f5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f684070 0x55e27faa7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.721+0000 7f139e0f5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f13901b8240 con 0x55e27f65b1b0 2024-07-31T15:11:31.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.721+0000 7f139e0f5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f13901b7b70 con 0x55e27f684070 2024-07-31T15:11:31.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.721+0000 7f139e0f5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13901b4dd0 con 0x55e27f663390 2024-07-31T15:11:31.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.721+0000 7f139e0f5740 10 client.0 _ll_register_callbacks cb 0x7f1380013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:31.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.722+0000 7f1395940640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f663390 0x55e27fa38dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.722+0000 7f1395940640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f663390 0x55e27fa38dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:54452/0 (socket says 172.21.15.67:54452) 2024-07-31T15:11:31.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.722+0000 7f1395940640 1 -- 172.21.15.67:0/219764528 learned_addr learned my addr 172.21.15.67:0/219764528 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:31.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.722+0000 7f139513f640 1 --2- 172.21.15.67:0/219764528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f684070 0x55e27faa7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.722+0000 7f139513f640 1 --2- 172.21.15.67:0/219764528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f65b1b0 0x55e27fa38810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.722+0000 7f1395940640 1 -- 172.21.15.67:0/219764528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f684070 msgr2=0x55e27faa7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:31.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.722+0000 7f1395940640 1 --2- 172.21.15.67:0/219764528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f684070 0x55e27faa7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.722+0000 7f1395940640 1 -- 172.21.15.67:0/219764528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f65b1b0 msgr2=0x55e27fa38810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:31.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.722+0000 7f1395940640 1 --2- 172.21.15.67:0/219764528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f65b1b0 0x55e27fa38810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.722+0000 7f1395940640 1 -- 172.21.15.67:0/219764528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e27faa7dc0 con 0x55e27f663390 2024-07-31T15:11:31.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.727+0000 7f1395940640 1 --2- 172.21.15.67:0/219764528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f663390 0x55e27fa38dd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55e27fa281d0 tx=0x7f138c013b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91bc8e771d6a746e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:31.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.728+0000 7f13797fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:11:31.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.732+0000 7f13797fa640 1 -- 172.21.15.67:0/219764528 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c00fad0 con 0x55e27f663390 2024-07-31T15:11:31.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.732+0000 7f139e0f5740 1 -- 172.21.15.67:0/219764528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e27f99d580 con 0x55e27f663390 2024-07-31T15:11:31.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.732+0000 7f13797fa640 1 -- 172.21.15.67:0/219764528 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7f138c00fc70 con 0x55e27f663390 2024-07-31T15:11:31.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.740+0000 7f13797fa640 1 -- 172.21.15.67:0/219764528 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f138c049450 con 0x55e27f663390 2024-07-31T15:11:31.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.740+0000 7f139e0f5740 10 client.29164 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:31.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.740+0000 7f139e0f5740 1 -- 172.21.15.67:0/219764528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55e27faaf130 con 0x55e27f663390 2024-07-31T15:11:31.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.748+0000 7f13797fa640 1 -- 172.21.15.67:0/219764528 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e27f99d580 con 0x55e27f663390 2024-07-31T15:11:31.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.748+0000 7f13797fa640 1 -- 172.21.15.67:0/219764528 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f138c011680 con 0x55e27f663390 2024-07-31T15:11:31.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.749+0000 7f139e0f5740 10 client.29164 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:31.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.749+0000 7f139e0f5740 10 client.29164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:11:31.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.749+0000 7f139e0f5740 20 client.29164 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:31.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.749+0000 7f139e0f5740 1 --2- 172.21.15.67:0/219764528 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e27f617da0 0x55e27fab18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:31.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.749+0000 7f139e0f5740 4 client.29164 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:11:31.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.749+0000 7f139e0f5740 1 -- 172.21.15.67:0/219764528 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e27f99acc0 con 0x55e27f617da0 2024-07-31T15:11:31.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.755+0000 7f139513f640 1 --2- 172.21.15.67:0/219764528 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e27f617da0 0x55e27fab18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:31.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.769+0000 7f139513f640 1 --2- 172.21.15.67:0/219764528 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e27f617da0 0x55e27fab18f0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b36cad31fa0d6bb0 server_cookie=634bb0b6ef502659 in_seq=0 out_seq=0 2024-07-31T15:11:31.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.770+0000 7f13797fa640 10 client.29164 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:31.778 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:11:31.778 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T15:11:31.778 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T15:11:31.778 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:11:31.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f13797fa640 1 -- 172.21.15.67:0/219764528 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e27f99acc0 con 0x55e27f617da0 2024-07-31T15:11:31.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f13797fa640 10 client.29164 handle_command_reply: tid=0 2024-07-31T15:11:31.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f139e0f5740 1 client.29164 shutdown 2024-07-31T15:11:31.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f139e0f5740 1 -- 172.21.15.67:0/219764528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f663390 msgr2=0x55e27fa38dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:31.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f139e0f5740 1 --2- 172.21.15.67:0/219764528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f663390 0x55e27fa38dd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55e27fa281d0 tx=0x7f138c013b20 comp rx=0 tx=0).stop 2024-07-31T15:11:31.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f139e0f5740 1 -- 172.21.15.67:0/219764528 shutdown_connections 2024-07-31T15:11:31.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f139e0f5740 1 --2- 172.21.15.67:0/219764528 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55e27f617da0 0x55e27fab18f0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f139e0f5740 1 --2- 172.21.15.67:0/219764528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e27f684070 0x55e27faa7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f139e0f5740 1 --2- 172.21.15.67:0/219764528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e27f663390 0x55e27fa38dd0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f139e0f5740 1 --2- 172.21.15.67:0/219764528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e27f65b1b0 0x55e27fa38810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f139e0f5740 1 -- 172.21.15.67:0/219764528 >> 172.21.15.67:0/219764528 conn(0x55e27f678460 msgr2=0x55e27fa2e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:31.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f13797fa640 0 client.29164 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:31.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f139e0f5740 1 -- 172.21.15.67:0/219764528 shutdown_connections 2024-07-31T15:11:31.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f139e0f5740 1 -- 172.21.15.67:0/219764528 wait complete. 2024-07-31T15:11:31.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.775+0000 7f139e0f5740 20 client.29164 trim_cache size 0 max 16384 2024-07-31T15:11:31.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.778+0000 7f1397ba4640 1 -- 172.21.15.67:0/659046756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f137c0b3f50 msgr2=0x7f137c0b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:31.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.778+0000 7f1397ba4640 1 --2- 172.21.15.67:0/659046756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f137c0b3f50 0x7f137c0b6460 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1388040380 tx=0x7f138804d040 comp rx=0 tx=0).stop 2024-07-31T15:11:31.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.778+0000 7f1397ba4640 1 -- 172.21.15.67:0/659046756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f139011c660 msgr2=0x7f13901185c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:31.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.778+0000 7f1397ba4640 1 --2- 172.21.15.67:0/659046756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f139011c660 0x7f13901185c0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f1388040d10 tx=0x7f1388040d40 comp rx=0 tx=0).stop 2024-07-31T15:11:31.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.778+0000 7f1397ba4640 1 -- 172.21.15.67:0/659046756 shutdown_connections 2024-07-31T15:11:31.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.778+0000 7f1397ba4640 1 --2- 172.21.15.67:0/659046756 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f137c0b3f50 0x7f137c0b6460 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.778+0000 7f1397ba4640 1 --2- 172.21.15.67:0/659046756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f139011c660 0x7f13901185c0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.778+0000 7f1397ba4640 1 --2- 172.21.15.67:0/659046756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1390111970 0x7f1390118080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.778+0000 7f1397ba4640 1 --2- 172.21.15.67:0/659046756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f139010a410 0x7f139011bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:31.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.779+0000 7f1397ba4640 1 -- 172.21.15.67:0/659046756 >> 172.21.15.67:0/659046756 conn(0x7f13900708f0 msgr2=0x7f139010b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:31.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.779+0000 7f1397ba4640 1 -- 172.21.15.67:0/659046756 shutdown_connections 2024-07-31T15:11:31.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:31.779+0000 7f1397ba4640 1 -- 172.21.15.67:0/659046756 wait complete. 2024-07-31T15:11:31.868 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_gem_evict.o 2024-07-31T15:11:31.914 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/i2c/i2c-core-acpi.o 2024-07-31T15:11:32.015 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/ff-core.o 2024-07-31T15:11:32.030 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/xhci-trace.o 2024-07-31T15:11:32.037 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/xhci-debugfs.o 2024-07-31T15:11:32.592 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/hcd-pci.o 2024-07-31T15:11:32.759 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/touchscreen.o 2024-07-31T15:11:32.784 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/i2c/i2c-smbus.o 2024-07-31T15:11:32.802 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T15:11:32.803 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:11:32.993 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_gem_gtt.o 2024-07-31T15:11:32.995 INFO:tasks.workunit.client.0.smithi067.stdout: CC net/mac80211/rc80211_minstrel_ht.o 2024-07-31T15:11:33.085 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/net_failover.o 2024-07-31T15:11:33.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:33 smithi073 ceph-mon[36714]: pgmap v1054: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 728 KiB/s wr, 362 op/s 2024-07-31T15:11:33.485 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/ff-memless.o 2024-07-31T15:11:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:33 smithi077 ceph-mon[37327]: pgmap v1054: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 728 KiB/s wr, 362 op/s 2024-07-31T15:11:33.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:33 smithi067 ceph-mon[32750]: pgmap v1054: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 728 KiB/s wr, 362 op/s 2024-07-31T15:11:33.535 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/core/usb-acpi.o 2024-07-31T15:11:33.539 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/i2c/built-in.a 2024-07-31T15:11:33.598 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/sparse-keymap.o 2024-07-31T15:11:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:11:33.777 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/host/xhci-pci.o 2024-07-31T15:11:34.023 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_gem_ww.o 2024-07-31T15:11:34.368 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_gem.o 2024-07-31T15:11:34.414 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/vivaldi-fmap.o 2024-07-31T15:11:34.415 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_query.o 2024-07-31T15:11:34.420 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/usb/core/built-in.a 2024-07-31T15:11:34.768 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/usb/misc/built-in.a 2024-07-31T15:11:34.848 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/usb/early/ehci-dbgp.o 2024-07-31T15:11:34.871 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/input-leds.o 2024-07-31T15:11:35.265 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/usb/host/built-in.a 2024-07-31T15:11:35.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:35 smithi067 ceph-mon[32750]: pgmap v1055: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 869 KiB/s wr, 272 op/s 2024-07-31T15:11:35.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:35 smithi073 ceph-mon[36714]: pgmap v1055: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 869 KiB/s wr, 272 op/s 2024-07-31T15:11:35.412 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/input/evdev.o 2024-07-31T15:11:35.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:35 smithi077 ceph-mon[37327]: pgmap v1055: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 869 KiB/s wr, 272 op/s 2024-07-31T15:11:35.751 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e1000e/ptp.o 2024-07-31T15:11:35.759 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/usb/early/built-in.a 2024-07-31T15:11:35.787 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/usb/built-in.a 2024-07-31T15:11:35.865 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_request.o 2024-07-31T15:11:35.907 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/i2c/built-in.a 2024-07-31T15:11:35.939 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/tuners/built-in.a 2024-07-31T15:11:35.984 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pps/clients/built-in.a 2024-07-31T15:11:36.075 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pps/generators/built-in.a 2024-07-31T15:11:36.136 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pps/pps.o 2024-07-31T15:11:36.173 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/rc/keymaps/built-in.a 2024-07-31T15:11:36.221 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/rc/built-in.a 2024-07-31T15:11:36.315 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/mac80211/built-in.a 2024-07-31T15:11:36.401 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/common/b2c2/built-in.a 2024-07-31T15:11:36.444 INFO:tasks.workunit.client.0.smithi067.stdout: AR net/built-in.a 2024-07-31T15:11:36.456 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/common/saa7146/built-in.a 2024-07-31T15:11:36.480 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/common/siano/built-in.a 2024-07-31T15:11:36.507 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/common/v4l2-tpg/built-in.a 2024-07-31T15:11:36.527 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_scheduler.o 2024-07-31T15:11:36.533 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/common/videobuf2/built-in.a 2024-07-31T15:11:36.548 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/common/built-in.a 2024-07-31T15:11:36.565 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/input/built-in.a 2024-07-31T15:11:36.603 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/allegro-dvt/built-in.a 2024-07-31T15:11:36.619 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/amlogic/meson-ge2d/built-in.a 2024-07-31T15:11:36.621 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_trace_points.o 2024-07-31T15:11:36.632 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/amlogic/built-in.a 2024-07-31T15:11:36.655 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/ttpci/built-in.a 2024-07-31T15:11:36.663 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/amphion/built-in.a 2024-07-31T15:11:36.687 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/b2c2/built-in.a 2024-07-31T15:11:36.691 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/aspeed/built-in.a 2024-07-31T15:11:36.716 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/pluto2/built-in.a 2024-07-31T15:11:36.718 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/atmel/built-in.a 2024-07-31T15:11:36.738 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/dm1105/built-in.a 2024-07-31T15:11:36.740 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/cadence/built-in.a 2024-07-31T15:11:36.759 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pps/kapi.o 2024-07-31T15:11:36.761 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/pt1/built-in.a 2024-07-31T15:11:36.763 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/chips-media/built-in.a 2024-07-31T15:11:36.768 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/intel/e1000e/built-in.a 2024-07-31T15:11:36.787 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/pt3/built-in.a 2024-07-31T15:11:36.787 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/intel/built-in.a 2024-07-31T15:11:36.802 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/intel/e100.o 2024-07-31T15:11:36.822 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/marvell/built-in.a 2024-07-31T15:11:36.827 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/mantis/built-in.a 2024-07-31T15:11:36.920 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/mediatek/jpeg/built-in.a 2024-07-31T15:11:36.921 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/ngene/built-in.a 2024-07-31T15:11:36.949 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/ddbridge/built-in.a 2024-07-31T15:11:36.959 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/mediatek/mdp/built-in.a 2024-07-31T15:11:36.975 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/mediatek/vcodec/built-in.a 2024-07-31T15:11:36.981 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/saa7146/built-in.a 2024-07-31T15:11:36.982 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/smipcie/built-in.a 2024-07-31T15:11:37.013 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/mediatek/vpu/built-in.a 2024-07-31T15:11:37.016 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/netup_unidvb/built-in.a 2024-07-31T15:11:37.030 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/intel/ipu3/built-in.a 2024-07-31T15:11:37.038 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/mediatek/mdp3/built-in.a 2024-07-31T15:11:37.046 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/intel/built-in.a 2024-07-31T15:11:37.051 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/mediatek/built-in.a 2024-07-31T15:11:37.058 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/usb/b2c2/built-in.a 2024-07-31T15:11:37.067 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/pci/built-in.a 2024-07-31T15:11:37.088 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/usb/dvb-usb/built-in.a 2024-07-31T15:11:37.095 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/microchip/built-in.a 2024-07-31T15:11:37.115 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/usb/dvb-usb-v2/built-in.a 2024-07-31T15:11:37.125 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/usb/s2255/built-in.a 2024-07-31T15:11:37.131 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/nvidia/tegra-vde/built-in.a 2024-07-31T15:11:37.142 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/nvidia/built-in.a 2024-07-31T15:11:37.151 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/usb/ttusb-budget/built-in.a 2024-07-31T15:11:37.154 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/usb/siano/built-in.a 2024-07-31T15:11:37.186 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/usb/ttusb-dec/built-in.a 2024-07-31T15:11:37.187 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/nxp/dw100/built-in.a 2024-07-31T15:11:37.195 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/qcom/camss/built-in.a 2024-07-31T15:11:37.203 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/usb/built-in.a 2024-07-31T15:11:37.211 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/nxp/imx-jpeg/built-in.a 2024-07-31T15:11:37.224 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/qcom/venus/built-in.a 2024-07-31T15:11:37.234 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/nxp/imx8-isi/built-in.a 2024-07-31T15:11:37.238 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/qcom/built-in.a 2024-07-31T15:11:37.248 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/nxp/built-in.a 2024-07-31T15:11:37.255 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/pps/sysfs.o 2024-07-31T15:11:37.263 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_color_mgmt.o 2024-07-31T15:11:37.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.269+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1521154070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b44111970 msgr2=0x7f5b44109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:37.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.269+0000 7f5b4bd4f640 1 --2- 172.21.15.67:0/1521154070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b44111970 0x7f5b44109df0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009900 tx=0x7f5b3402a400 comp rx=0 tx=0).stop 2024-07-31T15:11:37.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.269+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1521154070 shutdown_connections 2024-07-31T15:11:37.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.269+0000 7f5b4bd4f640 1 --2- 172.21.15.67:0/1521154070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4411c660 0x7f5b4411eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:37.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.269+0000 7f5b4bd4f640 1 --2- 172.21.15.67:0/1521154070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b4410a410 0x7f5b4410a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:37.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.269+0000 7f5b4bd4f640 1 --2- 172.21.15.67:0/1521154070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b44111970 0x7f5b44109df0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:37.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.269+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1521154070 >> 172.21.15.67:0/1521154070 conn(0x7f5b440708f0 msgr2=0x7f5b44070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:37.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.270+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1521154070 shutdown_connections 2024-07-31T15:11:37.272 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:37 smithi067 ceph-mon[32750]: pgmap v1056: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 551 KiB/s wr, 160 op/s 2024-07-31T15:11:37.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.270+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1521154070 wait complete. 2024-07-31T15:11:37.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.271+0000 7f5b4bd4f640 1 Processor -- start 2024-07-31T15:11:37.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.271+0000 7f5b4bd4f640 1 -- start start 2024-07-31T15:11:37.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.272+0000 7f5b4bd4f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b4410a410 0x7f5b4411bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:37.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.272+0000 7f5b4bd4f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b44111970 0x7f5b441161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:37.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.272+0000 7f5b4bd4f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4411c660 0x7f5b441167a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:37.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.272+0000 7f5b4bd4f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441126b0 con 0x7f5b44111970 2024-07-31T15:11:37.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.272+0000 7f5b4bd4f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5b44112530 con 0x7f5b4410a410 2024-07-31T15:11:37.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.272+0000 7f5b4bd4f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5b44112830 con 0x7f5b4411c660 2024-07-31T15:11:37.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.273+0000 7f5b49aeb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b4410a410 0x7f5b4411bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:37.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.273+0000 7f5b49aeb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b4410a410 0x7f5b4411bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:55394/0 (socket says 172.21.15.67:55394) 2024-07-31T15:11:37.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.273+0000 7f5b49aeb640 1 -- 172.21.15.67:0/1907698405 learned_addr learned my addr 172.21.15.67:0/1907698405 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:37.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.273+0000 7f5b492ea640 1 --2- 172.21.15.67:0/1907698405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b44111970 0x7f5b441161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:37.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.273+0000 7f5b4a2ec640 1 --2- 172.21.15.67:0/1907698405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4411c660 0x7f5b441167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:37.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.274+0000 7f5b49aeb640 1 -- 172.21.15.67:0/1907698405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4411c660 msgr2=0x7f5b441167a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:37.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.274+0000 7f5b49aeb640 1 --2- 172.21.15.67:0/1907698405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4411c660 0x7f5b441167a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:37.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.274+0000 7f5b49aeb640 1 -- 172.21.15.67:0/1907698405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b44111970 msgr2=0x7f5b441161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:37.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.274+0000 7f5b49aeb640 1 --2- 172.21.15.67:0/1907698405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b44111970 0x7f5b441161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:37.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.274+0000 7f5b49aeb640 1 -- 172.21.15.67:0/1907698405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b44116f30 con 0x7f5b4410a410 2024-07-31T15:11:37.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.275+0000 7f5b49aeb640 1 --2- 172.21.15.67:0/1907698405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b4410a410 0x7f5b4411bee0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5b34031440 tx=0x7f5b34003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:37.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.275+0000 7f5b3affd640 1 -- 172.21.15.67:0/1907698405 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3403f070 con 0x7f5b4410a410 2024-07-31T15:11:37.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.275+0000 7f5b3affd640 1 -- 172.21.15.67:0/1907698405 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b34003e40 con 0x7f5b4410a410 2024-07-31T15:11:37.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.277+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1907698405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b44127ab0 con 0x7f5b4410a410 2024-07-31T15:11:37.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.277+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1907698405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b44127f40 con 0x7f5b4410a410 2024-07-31T15:11:37.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.277+0000 7f5b3affd640 1 -- 172.21.15.67:0/1907698405 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34035030 con 0x7f5b4410a410 2024-07-31T15:11:37.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.279+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1907698405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b4410b010 con 0x7f5b4410a410 2024-07-31T15:11:37.281 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/renesas/rzg2l-cru/built-in.a 2024-07-31T15:11:37.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.282+0000 7f5b3affd640 1 -- 172.21.15.67:0/1907698405 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5b34042070 con 0x7f5b4410a410 2024-07-31T15:11:37.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.283+0000 7f5b3affd640 1 --2- 172.21.15.67:0/1907698405 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5b300b3da0 0x7f5b300b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:37.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.284+0000 7f5b492ea640 1 --2- 172.21.15.67:0/1907698405 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5b300b3da0 0x7f5b300b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:37.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.284+0000 7f5b3affd640 1 -- 172.21.15.67:0/1907698405 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7f5b34104960 con 0x7f5b4410a410 2024-07-31T15:11:37.287 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/renesas/rcar-vin/built-in.a 2024-07-31T15:11:37.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.286+0000 7f5b492ea640 1 --2- 172.21.15.67:0/1907698405 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5b300b3da0 0x7f5b300b62b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5b4411ee80 tx=0x7f5b40005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:37.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.289+0000 7f5b3affd640 1 -- 172.21.15.67:0/1907698405 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5b34049030 con 0x7f5b4410a410 2024-07-31T15:11:37.322 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/renesas/vsp1/built-in.a 2024-07-31T15:11:37.346 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/renesas/built-in.a 2024-07-31T15:11:37.351 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/rockchip/rga/built-in.a 2024-07-31T15:11:37.374 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_ttm_buddy_manager.o 2024-07-31T15:11:37.380 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_vma.o 2024-07-31T15:11:37.389 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/mmc/siano/built-in.a 2024-07-31T15:11:37.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:37 smithi073 ceph-mon[36714]: pgmap v1056: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 551 KiB/s wr, 160 op/s 2024-07-31T15:11:37.397 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/rockchip/rkisp1/built-in.a 2024-07-31T15:11:37.412 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/mmc/built-in.a 2024-07-31T15:11:37.421 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/rockchip/built-in.a 2024-07-31T15:11:37.433 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_vma_resource.o 2024-07-31T15:11:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:37 smithi077 ceph-mon[37327]: pgmap v1056: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 551 KiB/s wr, 160 op/s 2024-07-31T15:11:37.563 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/samsung/exynos-gsc/built-in.a 2024-07-31T15:11:37.644 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/samsung/exynos4-is/built-in.a 2024-07-31T15:11:37.673 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/pps/built-in.a 2024-07-31T15:11:37.675 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/samsung/s3c-camif/built-in.a 2024-07-31T15:11:37.700 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/samsung/s5p-g2d/built-in.a 2024-07-31T15:11:37.721 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/samsung/s5p-jpeg/built-in.a 2024-07-31T15:11:37.735 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ptp/ptp_clock.o 2024-07-31T15:11:37.786 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/samsung/s5p-mfc/built-in.a 2024-07-31T15:11:37.810 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:11:37.810 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":66,"btime":"2024-07-31T15:10:30:235518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:10:30.235494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29129":{"gid":29129,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/631186801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":631186801},{"type":"v1","addr":"172.21.15.73:6835","nonce":631186801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:11:37.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.806+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1907698405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b44128260 con 0x7f5b4410a410 2024-07-31T15:11:37.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.808+0000 7f5b3affd640 1 -- 172.21.15.67:0/1907698405 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11959 (secure 0 0 0) 0x7f5b340cb900 con 0x7f5b4410a410 2024-07-31T15:11:37.811 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 66 2024-07-31T15:11:37.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.813+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1907698405 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5b300b3da0 msgr2=0x7f5b300b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:37.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.813+0000 7f5b4bd4f640 1 --2- 172.21.15.67:0/1907698405 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5b300b3da0 0x7f5b300b62b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5b4411ee80 tx=0x7f5b40005f90 comp rx=0 tx=0).stop 2024-07-31T15:11:37.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.813+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1907698405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b4410a410 msgr2=0x7f5b4411bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.813+0000 7f5b4bd4f640 1 --2- 172.21.15.67:0/1907698405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b4410a410 0x7f5b4411bee0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5b34031440 tx=0x7f5b34003cd0 comp rx=0 tx=0).stop 2024-07-31T15:11:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.813+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1907698405 shutdown_connections 2024-07-31T15:11:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.813+0000 7f5b4bd4f640 1 --2- 172.21.15.67:0/1907698405 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5b300b3da0 0x7f5b300b62b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.814+0000 7f5b4bd4f640 1 --2- 172.21.15.67:0/1907698405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4411c660 0x7f5b441167a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.814+0000 7f5b4bd4f640 1 --2- 172.21.15.67:0/1907698405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b44111970 0x7f5b441161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.814+0000 7f5b4bd4f640 1 --2- 172.21.15.67:0/1907698405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b4410a410 0x7f5b4411bee0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.814+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1907698405 >> 172.21.15.67:0/1907698405 conn(0x7f5b440708f0 msgr2=0x7f5b44073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.814+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1907698405 shutdown_connections 2024-07-31T15:11:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:37.814+0000 7f5b4bd4f640 1 -- 172.21.15.67:0/1907698405 wait complete. 2024-07-31T15:11:37.837 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/samsung/built-in.a 2024-07-31T15:11:37.939 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/st/sti/bdisp/built-in.a 2024-07-31T15:11:37.970 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/st/sti/c8sectpfe/built-in.a 2024-07-31T15:11:37.985 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/st/sti/delta/built-in.a 2024-07-31T15:11:37.985 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_gsc_fw.o 2024-07-31T15:11:38.009 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/st/sti/hva/built-in.a 2024-07-31T15:11:38.021 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/st/stm32/built-in.a 2024-07-31T15:11:38.029 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_gsc_proxy.o 2024-07-31T15:11:38.033 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/st/built-in.a 2024-07-31T15:11:38.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:38 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1907698405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:11:38.088 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/sunxi/sun4i-csi/built-in.a 2024-07-31T15:11:38.114 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/ti/am437x/built-in.a 2024-07-31T15:11:38.115 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/sunxi/sun6i-csi/built-in.a 2024-07-31T15:11:38.137 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/ti/cal/built-in.a 2024-07-31T15:11:38.139 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/sunxi/sun6i-mipi-csi2/built-in.a 2024-07-31T15:11:38.165 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/ti/vpe/built-in.a 2024-07-31T15:11:38.165 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/built-in.a 2024-07-31T15:11:38.190 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/sunxi/sun8i-di/built-in.a 2024-07-31T15:11:38.190 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/ti/davinci/built-in.a 2024-07-31T15:11:38.216 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/sunxi/sun8i-rotate/built-in.a 2024-07-31T15:11:38.218 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/ti/omap/built-in.a 2024-07-31T15:11:38.230 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/sunxi/built-in.a 2024-07-31T15:11:38.242 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/ti/omap3isp/built-in.a 2024-07-31T15:11:38.258 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/ti/built-in.a 2024-07-31T15:11:38.267 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/verisilicon/built-in.a 2024-07-31T15:11:38.291 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/via/built-in.a 2024-07-31T15:11:38.299 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/xilinx/built-in.a 2024-07-31T15:11:38.309 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_gsc_uc.o 2024-07-31T15:11:38.310 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/platform/built-in.a 2024-07-31T15:11:38.345 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_connector.o 2024-07-31T15:11:38.364 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/intel/built-in.a 2024-07-31T15:11:38.381 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/firewire/built-in.a 2024-07-31T15:11:38.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:38 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1907698405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:11:38.409 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/spi/built-in.a 2024-07-31T15:11:38.415 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/i825xx/built-in.a 2024-07-31T15:11:38.439 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/test-drivers/built-in.a 2024-07-31T15:11:38.440 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/microsoft/built-in.a 2024-07-31T15:11:38.453 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/media/built-in.a 2024-07-31T15:11:38.467 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/litex/built-in.a 2024-07-31T15:11:38.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:38 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1907698405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:11:38.508 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/marvell/octeon_ep/built-in.a 2024-07-31T15:11:38.511 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/marvell/octeontx2/built-in.a 2024-07-31T15:11:38.546 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/marvell/prestera/built-in.a 2024-07-31T15:11:38.550 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ptp/ptp_chardev.o 2024-07-31T15:11:38.563 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/marvell/sky2.o 2024-07-31T15:11:38.571 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T15:11:38.572 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/power/supply/power_supply_core.o 2024-07-31T15:11:38.911 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_gsc_uc_heci_cmd_submit.o 2024-07-31T15:11:38.938 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc.o 2024-07-31T15:11:39.013 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hwmon/hwmon.o 2024-07-31T15:11:39.252 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/mellanox/built-in.a 2024-07-31T15:11:39.298 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:11:39.349 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/micrel/built-in.a 2024-07-31T15:11:39.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:39 smithi073 ceph-mon[36714]: pgmap v1057: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-07-31T15:11:39.393 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ptp/ptp_sysfs.o 2024-07-31T15:11:39.429 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:39 smithi067 ceph-mon[32750]: pgmap v1057: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-07-31T15:11:39.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:39 smithi077 ceph-mon[37327]: pgmap v1057: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-07-31T15:11:39.810 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_crtc.o 2024-07-31T15:11:39.930 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_ads.o 2024-07-31T15:11:39.957 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.0 2024-07-31T15:11:39.957 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:11:40.276 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/microchip/built-in.a 2024-07-31T15:11:40.292 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/hwmon/built-in.a 2024-07-31T15:11:40.343 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/power/supply/power_supply_sysfs.o 2024-07-31T15:11:40.345 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/mscc/built-in.a 2024-07-31T15:11:40.414 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_capture.o 2024-07-31T15:11:40.459 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/myricom/built-in.a 2024-07-31T15:11:40.537 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_ct.o 2024-07-31T15:11:40.590 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ptp/ptp_vclock.o 2024-07-31T15:11:40.794 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/natsemi/built-in.a 2024-07-31T15:11:40.817 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_displayid.o 2024-07-31T15:11:40.855 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_debugfs.o 2024-07-31T15:11:41.057 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/power/supply/power_supply_leds.o 2024-07-31T15:11:41.200 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:11:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:41 smithi067 ceph-mon[32750]: pgmap v1058: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-07-31T15:11:41.336 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ptp/ptp_kvm_x86.o 2024-07-31T15:11:41.359 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/power/supply/power_supply_hwmon.o 2024-07-31T15:11:41.369 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_fw.o 2024-07-31T15:11:41.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:41 smithi073 ceph-mon[36714]: pgmap v1058: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-07-31T15:11:41.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:41 smithi077 ceph-mon[37327]: pgmap v1058: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-07-31T15:11:41.577 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_hwconfig.o 2024-07-31T15:11:41.656 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_log.o 2024-07-31T15:11:41.765 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_log_debugfs.o 2024-07-31T15:11:41.830 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/power/supply/built-in.a 2024-07-31T15:11:41.912 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/power/built-in.a 2024-07-31T15:11:41.964 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_rc.o 2024-07-31T15:11:41.977 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_drv.o 2024-07-31T15:11:41.996 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/ptp/ptp_kvm_common.o 2024-07-31T15:11:42.264 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/marvell/built-in.a 2024-07-31T15:11:42.303 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.o 2024-07-31T15:11:42.336 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/neterion/built-in.a 2024-07-31T15:11:42.389 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/netronome/built-in.a 2024-07-31T15:11:42.395 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_submission.o 2024-07-31T15:11:42.427 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/ni/built-in.a 2024-07-31T15:11:42.478 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/nvidia/forcedeth.o 2024-07-31T15:11:42.606 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_huc.o 2024-07-31T15:11:42.647 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:11:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.255+0000 7efd843b9640 1 -- 172.21.15.67:0/1238873349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd740956a0 msgr2=0x7efd74099bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.255+0000 7efd843b9640 1 --2- 172.21.15.67:0/1238873349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd740956a0 0x7efd74099bd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efd6c009930 tx=0x7efd6c02a440 comp rx=0 tx=0).stop 2024-07-31T15:11:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.258+0000 7efd843b9640 1 -- 172.21.15.67:0/1238873349 shutdown_connections 2024-07-31T15:11:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.258+0000 7efd843b9640 1 --2- 172.21.15.67:0/1238873349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd7409a190 0x7efd7409a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.258+0000 7efd843b9640 1 --2- 172.21.15.67:0/1238873349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd740956a0 0x7efd74099bd0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.258+0000 7efd843b9640 1 --2- 172.21.15.67:0/1238873349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd74094d50 0x7efd74095160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.258+0000 7efd843b9640 1 -- 172.21.15.67:0/1238873349 >> 172.21.15.67:0/1238873349 conn(0x7efd7400cd80 msgr2=0x7efd7400d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.259+0000 7efd843b9640 1 -- 172.21.15.67:0/1238873349 shutdown_connections 2024-07-31T15:11:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.259+0000 7efd843b9640 1 -- 172.21.15.67:0/1238873349 wait complete. 2024-07-31T15:11:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.260+0000 7efd843b9640 1 Processor -- start 2024-07-31T15:11:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.261+0000 7efd843b9640 1 -- start start 2024-07-31T15:11:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.261+0000 7efd843b9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd74094d50 0x7efd741330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.261+0000 7efd843b9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd740956a0 0x7efd74133640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.262+0000 7efd82155640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd74094d50 0x7efd741330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.262+0000 7efd82155640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd74094d50 0x7efd741330a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49220/0 (socket says 172.21.15.67:49220) 2024-07-31T15:11:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.262+0000 7efd82155640 1 -- 172.21.15.67:0/4125147550 learned_addr learned my addr 172.21.15.67:0/4125147550 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.262+0000 7efd843b9640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd7409a190 0x7efd741388e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.262+0000 7efd82956640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd7409a190 0x7efd741388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.262+0000 7efd81954640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd740956a0 0x7efd74133640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.263+0000 7efd843b9640 1 -- 172.21.15.67:0/4125147550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efd740ac6b0 con 0x7efd7409a190 2024-07-31T15:11:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.263+0000 7efd843b9640 1 -- 172.21.15.67:0/4125147550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efd740ac530 con 0x7efd740956a0 2024-07-31T15:11:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.263+0000 7efd843b9640 1 -- 172.21.15.67:0/4125147550 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd740ac830 con 0x7efd74094d50 2024-07-31T15:11:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.265+0000 7efd82155640 1 -- 172.21.15.67:0/4125147550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd740956a0 msgr2=0x7efd74133640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.265+0000 7efd82155640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd740956a0 0x7efd74133640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.265+0000 7efd82155640 1 -- 172.21.15.67:0/4125147550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd7409a190 msgr2=0x7efd741388e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.265+0000 7efd82155640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd7409a190 0x7efd741388e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.265+0000 7efd82155640 1 -- 172.21.15.67:0/4125147550 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd74138f50 con 0x7efd74094d50 2024-07-31T15:11:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.265+0000 7efd82956640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd7409a190 0x7efd741388e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:11:42.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.265+0000 7efd82155640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd74094d50 0x7efd741330a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efd7800c920 tx=0x7efd7800cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.266+0000 7efd737fe640 1 -- 172.21.15.67:0/4125147550 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78007c70 con 0x7efd74094d50 2024-07-31T15:11:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.266+0000 7efd737fe640 1 -- 172.21.15.67:0/4125147550 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd78007e10 con 0x7efd74094d50 2024-07-31T15:11:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.267+0000 7efd737fe640 1 -- 172.21.15.67:0/4125147550 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78018820 con 0x7efd74094d50 2024-07-31T15:11:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.268+0000 7efd843b9640 1 -- 172.21.15.67:0/4125147550 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd74139240 con 0x7efd74094d50 2024-07-31T15:11:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.268+0000 7efd843b9640 1 -- 172.21.15.67:0/4125147550 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd74139720 con 0x7efd74094d50 2024-07-31T15:11:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.276+0000 7efd8a90a740 1 Processor -- start 2024-07-31T15:11:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.276+0000 7efd8a90a740 1 -- start start 2024-07-31T15:11:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.277+0000 7efd8a90a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f58070 0x563ec72fd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.277+0000 7efd737fe640 1 -- 172.21.15.67:0/4125147550 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7efd780189c0 con 0x7efd74094d50 2024-07-31T15:11:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.278+0000 7efd8a90a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 0x563ec72fdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.278+0000 7efd8a90a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f37390 0x563ec72fe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.279+0000 7efd82155640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 0x563ec72fdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.279+0000 7efd82956640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f58070 0x563ec72fd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.279+0000 7efd82956640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f58070 0x563ec72fd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42234/0 (socket says 172.21.15.67:42234) 2024-07-31T15:11:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.279+0000 7efd8a90a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563ec7273bb0 con 0x563ec6f58070 2024-07-31T15:11:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.279+0000 7efd82155640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 0x563ec72fdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49222/0 (socket says 172.21.15.67:49222) 2024-07-31T15:11:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.279+0000 7efd8a90a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563ec72630b0 con 0x563ec6f37390 2024-07-31T15:11:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.279+0000 7efd8a90a740 1 -- 172.21.15.67:0/4074379404 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563ec72670f0 con 0x563ec6f2f1b0 2024-07-31T15:11:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.279+0000 7efd82155640 1 -- 172.21.15.67:0/4074379404 learned_addr learned my addr 172.21.15.67:0/4074379404 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.280+0000 7efd81954640 1 --2- 172.21.15.67:0/4074379404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f37390 0x563ec72fe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.280+0000 7efd737fe640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efd680b3df0 0x7efd680b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.281+0000 7efd82956640 1 -- 172.21.15.67:0/4074379404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 msgr2=0x563ec72fdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.281+0000 7efd82956640 1 --2- 172.21.15.67:0/4074379404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 0x563ec72fdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.281+0000 7efd82956640 1 -- 172.21.15.67:0/4074379404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f37390 msgr2=0x563ec72fe190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.281+0000 7efd82956640 1 --2- 172.21.15.67:0/4074379404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f37390 0x563ec72fe190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.281+0000 7efd82956640 1 -- 172.21.15.67:0/4074379404 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ec7209b90 con 0x563ec6f58070 2024-07-31T15:11:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.281+0000 7efd82956640 1 --2- 172.21.15.67:0/4074379404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f58070 0x563ec72fd710 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7efd7c000b60 tx=0x7efd7c069d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c728ee0783be5e1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.282+0000 7efd82956640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efd680b3df0 0x7efd680b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.282+0000 7efd737fe640 1 -- 172.21.15.67:0/4125147550 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7efd780e3d40 con 0x7efd74094d50 2024-07-31T15:11:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.282+0000 7efd82155640 1 --2- 172.21.15.67:0/4074379404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 0x563ec72fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:11:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.283+0000 7efd82956640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efd680b3df0 0x7efd680b6300 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7efd7c07ffd0 tx=0x7efd7c07ff60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.283+0000 7efd70d99640 1 -- 172.21.15.67:0/4074379404 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c080810 con 0x563ec6f58070 2024-07-31T15:11:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.283+0000 7efd70d99640 1 -- 172.21.15.67:0/4074379404 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd7c087070 con 0x563ec6f58070 2024-07-31T15:11:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.284+0000 7efd70d99640 1 -- 172.21.15.67:0/4074379404 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c07fbd0 con 0x563ec6f58070 2024-07-31T15:11:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.284+0000 7efd8a90a740 1 -- 172.21.15.67:0/4074379404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f58070 msgr2=0x563ec72fd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.284+0000 7efd8a90a740 1 --2- 172.21.15.67:0/4074379404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f58070 0x563ec72fd710 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7efd7c000b60 tx=0x7efd7c069d50 comp rx=0 tx=0).stop 2024-07-31T15:11:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.285+0000 7efd8a90a740 1 -- 172.21.15.67:0/4074379404 shutdown_connections 2024-07-31T15:11:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.285+0000 7efd8a90a740 1 --2- 172.21.15.67:0/4074379404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f37390 0x563ec72fe190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.285+0000 7efd8a90a740 1 --2- 172.21.15.67:0/4074379404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 0x563ec72fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.285+0000 7efd8a90a740 1 --2- 172.21.15.67:0/4074379404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f58070 0x563ec72fd710 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.285+0000 7efd8a90a740 1 -- 172.21.15.67:0/4074379404 >> 172.21.15.67:0/4074379404 conn(0x563ec6f4c460 msgr2=0x563ec72fb600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.285+0000 7efd8a90a740 1 -- 172.21.15.67:0/4074379404 shutdown_connections 2024-07-31T15:11:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.285+0000 7efd8a90a740 1 -- 172.21.15.67:0/4074379404 wait complete. 2024-07-31T15:11:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.286+0000 7efd8a90a740 1 -- start start 2024-07-31T15:11:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.286+0000 7efd8a90a740 1 Processor -- start 2024-07-31T15:11:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.287+0000 7efd8a90a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f2f1b0 0x563ec7309280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.287+0000 7efd8a90a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f37390 0x563ec73097c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.287+0000 7efd8a90a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f58070 0x563ec7315240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.288+0000 7efd8a90a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563ec70c6700 con 0x563ec6f37390 2024-07-31T15:11:42.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.288+0000 7efd8a90a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563ec72059e0 con 0x563ec6f2f1b0 2024-07-31T15:11:42.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.288+0000 7efd8a90a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563ec730bef0 con 0x563ec6f58070 2024-07-31T15:11:42.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.288+0000 7efd8a90a740 10 client.0 _ll_register_callbacks cb 0x563ec72204f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:42.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.288+0000 7efd82155640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f2f1b0 0x563ec7309280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.289+0000 7efd81954640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f37390 0x563ec73097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.289+0000 7efd81954640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f37390 0x563ec73097c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42236/0 (socket says 172.21.15.67:42236) 2024-07-31T15:11:42.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.289+0000 7efd81954640 1 -- 172.21.15.67:0/2141105173 learned_addr learned my addr 172.21.15.67:0/2141105173 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:42.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.289+0000 7efd82956640 1 --2- 172.21.15.67:0/2141105173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f58070 0x563ec7315240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.290+0000 7efd81954640 1 -- 172.21.15.67:0/2141105173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f58070 msgr2=0x563ec7315240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:42.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.290+0000 7efd81954640 1 --2- 172.21.15.67:0/2141105173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f58070 0x563ec7315240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.290+0000 7efd81954640 1 -- 172.21.15.67:0/2141105173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f2f1b0 msgr2=0x563ec7309280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:42.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.290+0000 7efd81954640 1 --2- 172.21.15.67:0/2141105173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f2f1b0 0x563ec7309280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.290+0000 7efd81954640 1 -- 172.21.15.67:0/2141105173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ec6fc3650 con 0x563ec6f37390 2024-07-31T15:11:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.290+0000 7efd81954640 1 --2- 172.21.15.67:0/2141105173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f37390 0x563ec73097c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7efd6c031440 tx=0x7efd6c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240da01d64dfc7a9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.291+0000 7efd65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:11:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.295+0000 7efd65ffb640 1 -- 172.21.15.67:0/2141105173 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c035040 con 0x563ec6f37390 2024-07-31T15:11:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.295+0000 7efd65ffb640 1 -- 172.21.15.67:0/2141105173 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7efd6c003a30 con 0x563ec6f37390 2024-07-31T15:11:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.295+0000 7efd8a90a740 1 -- 172.21.15.67:0/2141105173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd7c087070 con 0x563ec6f37390 2024-07-31T15:11:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.295+0000 7efd65ffb640 1 -- 172.21.15.67:0/2141105173 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7efd6c003c30 con 0x563ec6f37390 2024-07-31T15:11:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.296+0000 7efd8a90a740 10 client.19764 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.296+0000 7efd8a90a740 1 -- 172.21.15.67:0/2141105173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x563ec7317ea0 con 0x563ec6f37390 2024-07-31T15:11:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.296+0000 7efd65ffb640 1 -- 172.21.15.67:0/2141105173 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd7c087070 con 0x563ec6f37390 2024-07-31T15:11:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.297+0000 7efd65ffb640 1 -- 172.21.15.67:0/2141105173 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7efd6c03c480 con 0x563ec6f37390 2024-07-31T15:11:42.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.298+0000 7efd8a90a740 10 client.19764 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.298+0000 7efd8a90a740 10 client.19764 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:11:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.298+0000 7efd8a90a740 20 client.19764 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.298+0000 7efd8a90a740 1 --2- 172.21.15.67:0/2141105173 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563ec6f169f0 0x563ec731b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.298+0000 7efd8a90a740 4 client.19764 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:11:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.298+0000 7efd8a90a740 1 -- 172.21.15.67:0/2141105173 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ec7275b00 con 0x563ec6f169f0 2024-07-31T15:11:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.299+0000 7efd82155640 1 --2- 172.21.15.67:0/2141105173 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563ec6f169f0 0x563ec731b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.302+0000 7efd82155640 1 --2- 172.21.15.67:0/2141105173 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563ec6f169f0 0x563ec731b810 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6b3b4be3d6fb5225 server_cookie=7a6565d8446effe0 in_seq=0 out_seq=0 2024-07-31T15:11:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.302+0000 7efd65ffb640 10 client.19764 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:11:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.303+0000 7efd65ffb640 1 -- 172.21.15.67:0/2141105173 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ec7275b00 con 0x563ec6f169f0 2024-07-31T15:11:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.303+0000 7efd65ffb640 10 client.19764 handle_command_reply: tid=0 2024-07-31T15:11:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.303+0000 7efd8a90a740 1 client.19764 shutdown 2024-07-31T15:11:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.303+0000 7efd8a90a740 1 -- 172.21.15.67:0/2141105173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f37390 msgr2=0x563ec73097c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.303+0000 7efd8a90a740 1 --2- 172.21.15.67:0/2141105173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f37390 0x563ec73097c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7efd6c031440 tx=0x7efd6c004600 comp rx=0 tx=0).stop 2024-07-31T15:11:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.304+0000 7efd8a90a740 1 -- 172.21.15.67:0/2141105173 shutdown_connections 2024-07-31T15:11:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.304+0000 7efd8a90a740 1 --2- 172.21.15.67:0/2141105173 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563ec6f169f0 0x563ec731b810 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.304+0000 7efd8a90a740 1 --2- 172.21.15.67:0/2141105173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f58070 0x563ec7315240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.304+0000 7efd8a90a740 1 --2- 172.21.15.67:0/2141105173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f37390 0x563ec73097c0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.304+0000 7efd8a90a740 1 --2- 172.21.15.67:0/2141105173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f2f1b0 0x563ec7309280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.304+0000 7efd8a90a740 1 -- 172.21.15.67:0/2141105173 >> 172.21.15.67:0/2141105173 conn(0x563ec6f4c460 msgr2=0x563ec73008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.305+0000 7efd65ffb640 0 client.19764 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:11:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.306+0000 7efd8a90a740 1 -- 172.21.15.67:0/2141105173 shutdown_connections 2024-07-31T15:11:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.306+0000 7efd8a90a740 1 -- 172.21.15.67:0/2141105173 wait complete. 2024-07-31T15:11:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.312+0000 7efd8a90a740 20 client.19764 trim_cache size 0 max 16384 2024-07-31T15:11:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.355+0000 7efd8a90a740 1 Processor -- start 2024-07-31T15:11:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.356+0000 7efd8a90a740 1 -- start start 2024-07-31T15:11:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.356+0000 7efd8a90a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 0x563ec72fa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.356+0000 7efd8a90a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f37390 0x563ec72fee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.357+0000 7efd8a90a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f58070 0x563ec7315030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.357+0000 7efd8a90a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efd74149760 con 0x563ec6f58070 2024-07-31T15:11:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.357+0000 7efd81954640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f37390 0x563ec72fee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.357+0000 7efd81954640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f37390 0x563ec72fee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44298/0 (socket says 172.21.15.67:44298) 2024-07-31T15:11:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.357+0000 7efd81954640 1 -- 172.21.15.67:0/1737056798 learned_addr learned my addr 172.21.15.67:0/1737056798 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.358+0000 7efd8a90a740 1 -- 172.21.15.67:0/1737056798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efd7414b2a0 con 0x563ec6f37390 2024-07-31T15:11:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.358+0000 7efd8a90a740 1 -- 172.21.15.67:0/1737056798 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd74149e30 con 0x563ec6f2f1b0 2024-07-31T15:11:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.358+0000 7efd81954640 1 -- 172.21.15.67:0/1737056798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 msgr2=0x563ec72fa600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:11:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.358+0000 7efd81954640 1 --2- 172.21.15.67:0/1737056798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 0x563ec72fa600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.358+0000 7efd81954640 1 -- 172.21.15.67:0/1737056798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f58070 msgr2=0x563ec7315030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:11:42.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.358+0000 7efd81954640 1 --2- 172.21.15.67:0/1737056798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f58070 0x563ec7315030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.358+0000 7efd81954640 1 -- 172.21.15.67:0/1737056798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ec730a340 con 0x563ec6f37390 2024-07-31T15:11:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.359+0000 7efd81954640 1 --2- 172.21.15.67:0/1737056798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f37390 0x563ec72fee50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efd6c005ab0 tx=0x7efd6c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=677596861405c71a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.360+0000 7efd67fff640 1 -- 172.21.15.67:0/1737056798 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c035030 con 0x563ec6f37390 2024-07-31T15:11:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.360+0000 7efd67fff640 1 -- 172.21.15.67:0/1737056798 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd6c03c440 con 0x563ec6f37390 2024-07-31T15:11:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.360+0000 7efd67fff640 1 -- 172.21.15.67:0/1737056798 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c03cca0 con 0x563ec6f37390 2024-07-31T15:11:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.362+0000 7efd8a90a740 1 -- 172.21.15.67:0/1737056798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f37390 msgr2=0x563ec72fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.362+0000 7efd8a90a740 1 --2- 172.21.15.67:0/1737056798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f37390 0x563ec72fee50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efd6c005ab0 tx=0x7efd6c000bc0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.362+0000 7efd8a90a740 1 -- 172.21.15.67:0/1737056798 shutdown_connections 2024-07-31T15:11:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.362+0000 7efd8a90a740 1 --2- 172.21.15.67:0/1737056798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f58070 0x563ec7315030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.362+0000 7efd8a90a740 1 --2- 172.21.15.67:0/1737056798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f37390 0x563ec72fee50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.362+0000 7efd8a90a740 1 --2- 172.21.15.67:0/1737056798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 0x563ec72fa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.362+0000 7efd8a90a740 1 -- 172.21.15.67:0/1737056798 >> 172.21.15.67:0/1737056798 conn(0x563ec6f4c460 msgr2=0x563ec72fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.362+0000 7efd8a90a740 1 -- 172.21.15.67:0/1737056798 shutdown_connections 2024-07-31T15:11:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.362+0000 7efd8a90a740 1 -- 172.21.15.67:0/1737056798 wait complete. 2024-07-31T15:11:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.363+0000 7efd8a90a740 1 -- start start 2024-07-31T15:11:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.363+0000 7efd8a90a740 1 Processor -- start 2024-07-31T15:11:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.364+0000 7efd8a90a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 0x563ec730d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.365+0000 7efd8a90a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f37390 0x563ec730d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.365+0000 7efd81954640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f37390 0x563ec730d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.365+0000 7efd81954640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f37390 0x563ec730d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42252/0 (socket says 172.21.15.67:42252) 2024-07-31T15:11:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.365+0000 7efd81954640 1 -- 172.21.15.67:0/4273988139 learned_addr learned my addr 172.21.15.67:0/4273988139 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.365+0000 7efd8a90a740 1 --2- 172.21.15.67:0/4273988139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f58070 0x563ec730dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.366+0000 7efd8a90a740 1 -- 172.21.15.67:0/4273988139 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efd7414abd0 con 0x563ec6f37390 2024-07-31T15:11:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.366+0000 7efd8a90a740 1 -- 172.21.15.67:0/4273988139 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efd7414a500 con 0x563ec6f58070 2024-07-31T15:11:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.366+0000 7efd82155640 1 --2- 172.21.15.67:0/4273988139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 0x563ec730d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.366+0000 7efd82956640 1 --2- 172.21.15.67:0/4273988139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f58070 0x563ec730dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.366+0000 7efd8a90a740 1 -- 172.21.15.67:0/4273988139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd74147760 con 0x563ec6f2f1b0 2024-07-31T15:11:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.366+0000 7efd8a90a740 10 client.0 _ll_register_callbacks cb 0x7efd6c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.367+0000 7efd82155640 1 -- 172.21.15.67:0/4273988139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f58070 msgr2=0x563ec730dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.367+0000 7efd82155640 1 --2- 172.21.15.67:0/4273988139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f58070 0x563ec730dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.367+0000 7efd82155640 1 -- 172.21.15.67:0/4273988139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f37390 msgr2=0x563ec730d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.367+0000 7efd82155640 1 --2- 172.21.15.67:0/4273988139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f37390 0x563ec730d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.367+0000 7efd82155640 1 -- 172.21.15.67:0/4273988139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ec7382fe0 con 0x563ec6f2f1b0 2024-07-31T15:11:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.368+0000 7efd82155640 1 --2- 172.21.15.67:0/4273988139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 0x563ec730d050 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efd780aa300 tx=0x7efd7800fd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d74bbba3fe7b19 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.369+0000 7efd65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:11:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.369+0000 7efd65ffb640 1 -- 172.21.15.67:0/4273988139 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd780aad00 con 0x563ec6f2f1b0 2024-07-31T15:11:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.369+0000 7efd65ffb640 1 -- 172.21.15.67:0/4273988139 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7efd780a6890 con 0x563ec6f2f1b0 2024-07-31T15:11:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.369+0000 7efd8a90a740 1 -- 172.21.15.67:0/4273988139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd6c03c440 con 0x563ec6f2f1b0 2024-07-31T15:11:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.370+0000 7efd65ffb640 1 -- 172.21.15.67:0/4273988139 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7efd780aa600 con 0x563ec6f2f1b0 2024-07-31T15:11:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.370+0000 7efd8a90a740 10 client.29179 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.370+0000 7efd8a90a740 1 -- 172.21.15.67:0/4273988139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x563ec7384240 con 0x563ec6f2f1b0 2024-07-31T15:11:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.371+0000 7efd65ffb640 1 -- 172.21.15.67:0/4273988139 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd780aec00 con 0x563ec6f2f1b0 2024-07-31T15:11:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.371+0000 7efd65ffb640 1 -- 172.21.15.67:0/4273988139 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7efd780a92d0 con 0x563ec6f2f1b0 2024-07-31T15:11:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.371+0000 7efd8a90a740 10 client.29179 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.371+0000 7efd8a90a740 10 client.29179 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:11:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.371+0000 7efd8a90a740 20 client.29179 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.371+0000 7efd8a90a740 1 --2- 172.21.15.67:0/4273988139 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563ec6eebda0 0x563ec73869b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.371+0000 7efd8a90a740 4 client.29179 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:11:42.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.371+0000 7efd8a90a740 1 -- 172.21.15.67:0/4273988139 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7efd6c03c440 con 0x563ec6eebda0 2024-07-31T15:11:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.374+0000 7efd81954640 1 --2- 172.21.15.67:0/4273988139 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563ec6eebda0 0x563ec73869b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.386+0000 7efd81954640 1 --2- 172.21.15.67:0/4273988139 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563ec6eebda0 0x563ec73869b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d24cc97f9551e58f server_cookie=6c6b29442d19711c in_seq=0 out_seq=0 2024-07-31T15:11:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.386+0000 7efd65ffb640 10 client.29179 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:11:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.387+0000 7efd65ffb640 1 -- 172.21.15.67:0/4273988139 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7efd6c03c440 con 0x563ec6eebda0 2024-07-31T15:11:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.387+0000 7efd65ffb640 10 client.29179 handle_command_reply: tid=0 2024-07-31T15:11:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.387+0000 7efd8a90a740 1 client.29179 shutdown 2024-07-31T15:11:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.389+0000 7efd8a90a740 1 -- 172.21.15.67:0/4273988139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 msgr2=0x563ec730d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.389+0000 7efd8a90a740 1 --2- 172.21.15.67:0/4273988139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 0x563ec730d050 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efd780aa300 tx=0x7efd7800fd60 comp rx=0 tx=0).stop 2024-07-31T15:11:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.389+0000 7efd8a90a740 1 -- 172.21.15.67:0/4273988139 shutdown_connections 2024-07-31T15:11:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.389+0000 7efd8a90a740 1 --2- 172.21.15.67:0/4273988139 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563ec6eebda0 0x563ec73869b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.389+0000 7efd8a90a740 1 --2- 172.21.15.67:0/4273988139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ec6f58070 0x563ec730dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.389+0000 7efd65ffb640 0 client.29179 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:11:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.389+0000 7efd8a90a740 1 --2- 172.21.15.67:0/4273988139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563ec6f37390 0x563ec730d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.390+0000 7efd8a90a740 1 --2- 172.21.15.67:0/4273988139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ec6f2f1b0 0x563ec730d050 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.390+0000 7efd8a90a740 1 -- 172.21.15.67:0/4273988139 >> 172.21.15.67:0/4273988139 conn(0x563ec6f4c460 msgr2=0x563ec72fa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.390+0000 7efd8a90a740 1 -- 172.21.15.67:0/4273988139 shutdown_connections 2024-07-31T15:11:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.390+0000 7efd8a90a740 1 -- 172.21.15.67:0/4273988139 wait complete. 2024-07-31T15:11:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.390+0000 7efd8a90a740 20 client.29179 trim_cache size 0 max 16384 2024-07-31T15:11:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.393+0000 7efd843b9640 1 -- 172.21.15.67:0/4125147550 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efd680b3df0 msgr2=0x7efd680b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.393+0000 7efd843b9640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efd680b3df0 0x7efd680b6300 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7efd7c07ffd0 tx=0x7efd7c07ff60 comp rx=0 tx=0).stop 2024-07-31T15:11:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.393+0000 7efd843b9640 1 -- 172.21.15.67:0/4125147550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd74094d50 msgr2=0x7efd741330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.393+0000 7efd843b9640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd74094d50 0x7efd741330a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efd7800c920 tx=0x7efd7800cde0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.393+0000 7efd843b9640 1 -- 172.21.15.67:0/4125147550 shutdown_connections 2024-07-31T15:11:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.393+0000 7efd843b9640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7efd680b3df0 0x7efd680b6300 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.393+0000 7efd843b9640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd7409a190 0x7efd741388e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.393+0000 7efd843b9640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd740956a0 0x7efd74133640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.393+0000 7efd843b9640 1 --2- 172.21.15.67:0/4125147550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd74094d50 0x7efd741330a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:42.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.393+0000 7efd843b9640 1 -- 172.21.15.67:0/4125147550 >> 172.21.15.67:0/4125147550 conn(0x7efd7400cd80 msgr2=0x7efd74097ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:42.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.399+0000 7efd843b9640 1 -- 172.21.15.67:0/4125147550 shutdown_connections 2024-07-31T15:11:42.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:42.402+0000 7efd843b9640 1 -- 172.21.15.67:0/4125147550 wait complete. 2024-07-31T15:11:42.686 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/ptp/built-in.a 2024-07-31T15:11:42.693 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_huc_debugfs.o 2024-07-31T15:11:42.751 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/thermal/broadcom/built-in.a 2024-07-31T15:11:42.774 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/thermal/samsung/built-in.a 2024-07-31T15:11:42.802 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/thermal/intel/intel_tcc.o 2024-07-31T15:11:42.839 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/thermal/st/built-in.a 2024-07-31T15:11:42.858 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/thermal/qcom/built-in.a 2024-07-31T15:11:42.871 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/watchdog/built-in.a 2024-07-31T15:11:42.901 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpufreq/cpufreq.o 2024-07-31T15:11:42.910 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/md.o 2024-07-31T15:11:43.001 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/thermal/intel/therm_throt.o 2024-07-31T15:11:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.272+0000 7fde71ea9640 1 -- 172.21.15.67:0/461051833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c10f430 msgr2=0x7fde6c10f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.272+0000 7fde71ea9640 1 --2- 172.21.15.67:0/461051833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c10f430 0x7fde6c10f840 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fde64009f00 tx=0x7fde6402ab60 comp rx=0 tx=0).stop 2024-07-31T15:11:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.272+0000 7fde71ea9640 1 -- 172.21.15.67:0/461051833 shutdown_connections 2024-07-31T15:11:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.272+0000 7fde71ea9640 1 --2- 172.21.15.67:0/461051833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde6c10fd80 0x7fde6c07b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.272+0000 7fde71ea9640 1 --2- 172.21.15.67:0/461051833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c10f430 0x7fde6c10f840 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.272+0000 7fde71ea9640 1 --2- 172.21.15.67:0/461051833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde6c10b460 0x7fde6c10b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.272+0000 7fde71ea9640 1 -- 172.21.15.67:0/461051833 >> 172.21.15.67:0/461051833 conn(0x7fde6c0707c0 msgr2=0x7fde6c070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.281+0000 7fde71ea9640 1 -- 172.21.15.67:0/461051833 shutdown_connections 2024-07-31T15:11:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.281+0000 7fde71ea9640 1 -- 172.21.15.67:0/461051833 wait complete. 2024-07-31T15:11:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.282+0000 7fde71ea9640 1 Processor -- start 2024-07-31T15:11:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.282+0000 7fde71ea9640 1 -- start start 2024-07-31T15:11:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.283+0000 7fde71ea9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde6c10b460 0x7fde6c080010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.283+0000 7fde71ea9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c10f430 0x7fde6c080550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.283+0000 7fde71ea9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde6c10fd80 0x7fde6c080a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.283+0000 7fde71ea9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c07e500 con 0x7fde6c10f430 2024-07-31T15:11:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.283+0000 7fde71ea9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c07e380 con 0x7fde6c10b460 2024-07-31T15:11:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.283+0000 7fde71ea9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c07e680 con 0x7fde6c10fd80 2024-07-31T15:11:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.283+0000 7fde6affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c10f430 0x7fde6c080550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.283+0000 7fde6affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c10f430 0x7fde6c080550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42278/0 (socket says 172.21.15.67:42278) 2024-07-31T15:11:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.283+0000 7fde6affd640 1 -- 172.21.15.67:0/656469261 learned_addr learned my addr 172.21.15.67:0/656469261 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.283+0000 7fde6bfff640 1 --2- 172.21.15.67:0/656469261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde6c10fd80 0x7fde6c080a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.283+0000 7fde6bfff640 1 -- 172.21.15.67:0/656469261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde6c10b460 msgr2=0x7fde6c080010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:11:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.283+0000 7fde6bfff640 1 --2- 172.21.15.67:0/656469261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde6c10b460 0x7fde6c080010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.283+0000 7fde6bfff640 1 -- 172.21.15.67:0/656469261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c10f430 msgr2=0x7fde6c080550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.283+0000 7fde6bfff640 1 --2- 172.21.15.67:0/656469261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c10f430 0x7fde6c080550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.284+0000 7fde6bfff640 1 -- 172.21.15.67:0/656469261 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6c138840 con 0x7fde6c10fd80 2024-07-31T15:11:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.284+0000 7fde6affd640 1 --2- 172.21.15.67:0/656469261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c10f430 0x7fde6c080550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.284+0000 7fde6bfff640 1 --2- 172.21.15.67:0/656469261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde6c10fd80 0x7fde6c080a90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fde6000d680 tx=0x7fde6000db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.290+0000 7fde68ff9640 1 -- 172.21.15.67:0/656469261 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60017070 con 0x7fde6c10fd80 2024-07-31T15:11:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.291+0000 7fde71ea9640 1 -- 172.21.15.67:0/656469261 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6c138b30 con 0x7fde6c10fd80 2024-07-31T15:11:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.291+0000 7fde71ea9640 1 -- 172.21.15.67:0/656469261 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6c139100 con 0x7fde6c10fd80 2024-07-31T15:11:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.291+0000 7fde68ff9640 1 -- 172.21.15.67:0/656469261 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde60009820 con 0x7fde6c10fd80 2024-07-31T15:11:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.295+0000 7fde68ff9640 1 -- 172.21.15.67:0/656469261 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60004ce0 con 0x7fde6c10fd80 2024-07-31T15:11:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.295+0000 7fde68ff9640 1 -- 172.21.15.67:0/656469261 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fde60010460 con 0x7fde6c10fd80 2024-07-31T15:11:43.297 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.295+0000 7fde78da5740 1 Processor -- start 2024-07-31T15:11:43.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:43 smithi067 ceph-mon[32750]: pgmap v1059: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.5 MiB/s wr, 238 op/s 2024-07-31T15:11:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.307+0000 7fde78da5740 1 -- start start 2024-07-31T15:11:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.307+0000 7fde78da5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 0x558a58acdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.307+0000 7fde78da5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ace220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.307+0000 7fde78da5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ace760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.307+0000 7fde78da5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558a58a433b0 con 0x558a587022f0 2024-07-31T15:11:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.307+0000 7fde78da5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558a58a328b0 con 0x558a5870a4d0 2024-07-31T15:11:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.307+0000 7fde78da5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558a58a368f0 con 0x558a5872b1b0 2024-07-31T15:11:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.307+0000 7fde6b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ace220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.308+0000 7fde6affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 0x558a58acdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.308+0000 7fde6affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 0x558a58acdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49276/0 (socket says 172.21.15.67:49276) 2024-07-31T15:11:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.308+0000 7fde6affd640 1 -- 172.21.15.67:0/3712138904 learned_addr learned my addr 172.21.15.67:0/3712138904 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.308+0000 7fde6affd640 1 --2- 172.21.15.67:0/3712138904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ace760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.308+0000 7fde6b7fe640 1 -- 172.21.15.67:0/3712138904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 msgr2=0x558a58acdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.308+0000 7fde6b7fe640 1 --2- 172.21.15.67:0/3712138904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 0x558a58acdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.308+0000 7fde6b7fe640 1 -- 172.21.15.67:0/3712138904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 msgr2=0x558a58ace760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.308+0000 7fde6b7fe640 1 --2- 172.21.15.67:0/3712138904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ace760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.308+0000 7fde6b7fe640 1 -- 172.21.15.67:0/3712138904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a589d99a0 con 0x558a587022f0 2024-07-31T15:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.308+0000 7fde68ff9640 1 --2- 172.21.15.67:0/656469261 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fde540b3ea0 0x7fde540b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.309+0000 7fde68ff9640 1 -- 172.21.15.67:0/656469261 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fde600e4b90 con 0x7fde6c10fd80 2024-07-31T15:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.309+0000 7fde6affd640 1 --2- 172.21.15.67:0/656469261 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fde540b3ea0 0x7fde540b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.309+0000 7fde6b7fe640 1 --2- 172.21.15.67:0/3712138904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ace220 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0096b0 tx=0x7fde5c009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b8a83f9979dd4f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.310+0000 7fde6affd640 1 --2- 172.21.15.67:0/656469261 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fde540b3ea0 0x7fde540b63b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fde64009a60 tx=0x7fde64005bd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.319+0000 7fde51ffb640 1 -- 172.21.15.67:0/3712138904 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c015020 con 0x558a587022f0 2024-07-31T15:11:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.319+0000 7fde51ffb640 1 -- 172.21.15.67:0/3712138904 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde5c00bd70 con 0x558a587022f0 2024-07-31T15:11:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.321+0000 7fde78da5740 1 -- 172.21.15.67:0/3712138904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 msgr2=0x558a58ace220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.321+0000 7fde78da5740 1 --2- 172.21.15.67:0/3712138904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ace220 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0096b0 tx=0x7fde5c009b70 comp rx=0 tx=0).stop 2024-07-31T15:11:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.321+0000 7fde78da5740 1 -- 172.21.15.67:0/3712138904 shutdown_connections 2024-07-31T15:11:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.321+0000 7fde78da5740 1 --2- 172.21.15.67:0/3712138904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ace760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.321+0000 7fde78da5740 1 --2- 172.21.15.67:0/3712138904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ace220 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.321+0000 7fde78da5740 1 --2- 172.21.15.67:0/3712138904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 0x558a58acdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.321+0000 7fde78da5740 1 -- 172.21.15.67:0/3712138904 >> 172.21.15.67:0/3712138904 conn(0x558a5871f300 msgr2=0x558a58acbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.321+0000 7fde78da5740 1 -- 172.21.15.67:0/3712138904 shutdown_connections 2024-07-31T15:11:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.321+0000 7fde78da5740 1 -- 172.21.15.67:0/3712138904 wait complete. 2024-07-31T15:11:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.322+0000 7fde78da5740 1 -- start start 2024-07-31T15:11:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.322+0000 7fde78da5740 1 Processor -- start 2024-07-31T15:11:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.322+0000 7fde78da5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ad9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.322+0000 7fde78da5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ad99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.322+0000 7fde78da5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 0x558a58ae5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.322+0000 7fde78da5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558a5896cab0 con 0x558a587022f0 2024-07-31T15:11:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.322+0000 7fde78da5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558a589bacf0 con 0x558a5870a4d0 2024-07-31T15:11:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.322+0000 7fde78da5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558a58adc4b0 con 0x558a5872b1b0 2024-07-31T15:11:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.323+0000 7fde78da5740 10 client.0 _ll_register_callbacks cb 0x558a58a2c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.323+0000 7fde6b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ad9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.323+0000 7fde6b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ad9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42304/0 (socket says 172.21.15.67:42304) 2024-07-31T15:11:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.323+0000 7fde6b7fe640 1 -- 172.21.15.67:0/2696289603 learned_addr learned my addr 172.21.15.67:0/2696289603 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.323+0000 7fde6affd640 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ad99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.323+0000 7fde6bfff640 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 0x558a58ae5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.323+0000 7fde6affd640 1 -- 172.21.15.67:0/2696289603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 msgr2=0x558a58ae5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.323+0000 7fde6affd640 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 0x558a58ae5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.323+0000 7fde6affd640 1 -- 172.21.15.67:0/2696289603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 msgr2=0x558a58ad9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.323+0000 7fde6affd640 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ad9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.323+0000 7fde6affd640 1 -- 172.21.15.67:0/2696289603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a58796790 con 0x558a5870a4d0 2024-07-31T15:11:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.323+0000 7fde6bfff640 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 0x558a58ae5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.323+0000 7fde6b7fe640 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ad9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.324+0000 7fde6affd640 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ad99c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fde6402b150 tx=0x7fde640095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1211a0123e2ca2b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.332+0000 7fde4f7f6640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:11:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.332+0000 7fde4f7f6640 1 -- 172.21.15.67:0/2696289603 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64043920 con 0x558a5870a4d0 2024-07-31T15:11:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.332+0000 7fde78da5740 1 -- 172.21.15.67:0/2696289603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde5c00bd70 con 0x558a5870a4d0 2024-07-31T15:11:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.332+0000 7fde4f7f6640 1 -- 172.21.15.67:0/2696289603 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fde64043ac0 con 0x558a5870a4d0 2024-07-31T15:11:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.332+0000 7fde4f7f6640 1 -- 172.21.15.67:0/2696289603 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fde6405a7d0 con 0x558a5870a4d0 2024-07-31T15:11:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.333+0000 7fde78da5740 10 client.29321 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.333+0000 7fde78da5740 1 -- 172.21.15.67:0/2696289603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x558a5880f620 con 0x558a5870a4d0 2024-07-31T15:11:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.333+0000 7fde4f7f6640 1 -- 172.21.15.67:0/2696289603 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde5c00bd70 con 0x558a5870a4d0 2024-07-31T15:11:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.333+0000 7fde4f7f6640 1 -- 172.21.15.67:0/2696289603 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fde6400b040 con 0x558a5870a4d0 2024-07-31T15:11:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.333+0000 7fde78da5740 10 client.29321 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.333+0000 7fde78da5740 10 client.29321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:11:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.333+0000 7fde78da5740 20 client.29321 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.333+0000 7fde78da5740 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558a586e9d80 0x558a58aeb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.333+0000 7fde78da5740 4 client.29321 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:11:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.333+0000 7fde78da5740 1 -- 172.21.15.67:0/2696289603 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a58a45300 con 0x558a586e9d80 2024-07-31T15:11:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.333+0000 7fde6b7fe640 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558a586e9d80 0x558a58aeb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.336+0000 7fde6b7fe640 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558a586e9d80 0x558a58aeb7b0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1340c016b303a10d server_cookie=b413c4242b17e44f in_seq=0 out_seq=0 2024-07-31T15:11:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.338+0000 7fde4f7f6640 10 client.29321 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.338+0000 7fde4f7f6640 1 -- 172.21.15.67:0/2696289603 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a58a45300 con 0x558a586e9d80 2024-07-31T15:11:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.338+0000 7fde4f7f6640 10 client.29321 handle_command_reply: tid=0 2024-07-31T15:11:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.338+0000 7fde78da5740 1 client.29321 shutdown 2024-07-31T15:11:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.339+0000 7fde78da5740 1 -- 172.21.15.67:0/2696289603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 msgr2=0x558a58ad99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.339+0000 7fde78da5740 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ad99c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fde6402b150 tx=0x7fde640095c0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.339+0000 7fde78da5740 1 -- 172.21.15.67:0/2696289603 shutdown_connections 2024-07-31T15:11:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.339+0000 7fde78da5740 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558a586e9d80 0x558a58aeb7b0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.339+0000 7fde78da5740 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 0x558a58ae5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.339+0000 7fde78da5740 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ad99c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.339+0000 7fde78da5740 1 --2- 172.21.15.67:0/2696289603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ad9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.339+0000 7fde4f7f6640 0 client.29321 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.339+0000 7fde78da5740 1 -- 172.21.15.67:0/2696289603 >> 172.21.15.67:0/2696289603 conn(0x558a5871f300 msgr2=0x558a58aced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.339+0000 7fde78da5740 1 -- 172.21.15.67:0/2696289603 shutdown_connections 2024-07-31T15:11:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.339+0000 7fde78da5740 1 -- 172.21.15.67:0/2696289603 wait complete. 2024-07-31T15:11:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.339+0000 7fde78da5740 20 client.29321 trim_cache size 0 max 16384 2024-07-31T15:11:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.391+0000 7fde78da5740 1 Processor -- start 2024-07-31T15:11:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.391+0000 7fde78da5740 1 -- start start 2024-07-31T15:11:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.391+0000 7fde78da5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a587022f0 0x558a58ad7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.391+0000 7fde78da5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ad8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.392+0000 7fde78da5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a5872b1b0 0x558a58ae5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.392+0000 7fde6affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ad8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.392+0000 7fde6affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ad8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44346/0 (socket says 172.21.15.67:44346) 2024-07-31T15:11:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.392+0000 7fde6b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a587022f0 0x558a58ad7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.392+0000 7fde6b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a587022f0 0x558a58ad7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49292/0 (socket says 172.21.15.67:49292) 2024-07-31T15:11:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.392+0000 7fde6bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a5872b1b0 0x558a58ae5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.392+0000 7fde6bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a5872b1b0 0x558a58ae5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42320/0 (socket says 172.21.15.67:42320) 2024-07-31T15:11:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.392+0000 7fde6affd640 1 -- 172.21.15.67:0/1443756052 learned_addr learned my addr 172.21.15.67:0/1443756052 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.392+0000 7fde78da5740 1 -- 172.21.15.67:0/1443756052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fde5800ff80 con 0x558a5872b1b0 2024-07-31T15:11:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.393+0000 7fde78da5740 1 -- 172.21.15.67:0/1443756052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fde58011ac0 con 0x558a5870a4d0 2024-07-31T15:11:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.393+0000 7fde78da5740 1 -- 172.21.15.67:0/1443756052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fde58010650 con 0x558a587022f0 2024-07-31T15:11:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.393+0000 7fde6affd640 1 -- 172.21.15.67:0/1443756052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a587022f0 msgr2=0x558a58ad7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.393+0000 7fde6affd640 1 --2- 172.21.15.67:0/1443756052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a587022f0 0x558a58ad7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.393+0000 7fde6affd640 1 -- 172.21.15.67:0/1443756052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a5872b1b0 msgr2=0x558a58ae5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.393+0000 7fde6affd640 1 --2- 172.21.15.67:0/1443756052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a5872b1b0 0x558a58ae5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.393+0000 7fde6affd640 1 -- 172.21.15.67:0/1443756052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a58ada300 con 0x558a5870a4d0 2024-07-31T15:11:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.394+0000 7fde6bfff640 1 --2- 172.21.15.67:0/1443756052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a5872b1b0 0x558a58ae5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.394+0000 7fde6affd640 1 --2- 172.21.15.67:0/1443756052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ad8220 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fde64006010 tx=0x7fde64052fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20eb4fe6ede8b131 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.394+0000 7fde517fa640 1 -- 172.21.15.67:0/1443756052 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64069070 con 0x558a5870a4d0 2024-07-31T15:11:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.394+0000 7fde517fa640 1 -- 172.21.15.67:0/1443756052 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde64056070 con 0x558a5870a4d0 2024-07-31T15:11:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.394+0000 7fde517fa640 1 -- 172.21.15.67:0/1443756052 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64042be0 con 0x558a5870a4d0 2024-07-31T15:11:43.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:43 smithi073 ceph-mon[36714]: pgmap v1059: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.5 MiB/s wr, 238 op/s 2024-07-31T15:11:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.396+0000 7fde78da5740 1 -- 172.21.15.67:0/1443756052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 msgr2=0x558a58ad8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.396+0000 7fde78da5740 1 --2- 172.21.15.67:0/1443756052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ad8220 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fde64006010 tx=0x7fde64052fd0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.396+0000 7fde78da5740 1 -- 172.21.15.67:0/1443756052 shutdown_connections 2024-07-31T15:11:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.396+0000 7fde78da5740 1 --2- 172.21.15.67:0/1443756052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a5872b1b0 0x558a58ae5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.396+0000 7fde78da5740 1 --2- 172.21.15.67:0/1443756052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ad8220 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.396+0000 7fde78da5740 1 --2- 172.21.15.67:0/1443756052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a587022f0 0x558a58ad7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.396+0000 7fde78da5740 1 -- 172.21.15.67:0/1443756052 >> 172.21.15.67:0/1443756052 conn(0x558a5871f300 msgr2=0x558a58ad45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.396+0000 7fde78da5740 1 -- 172.21.15.67:0/1443756052 shutdown_connections 2024-07-31T15:11:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.396+0000 7fde78da5740 1 -- 172.21.15.67:0/1443756052 wait complete. 2024-07-31T15:11:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.397+0000 7fde78da5740 1 -- start start 2024-07-31T15:11:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.398+0000 7fde78da5740 1 Processor -- start 2024-07-31T15:11:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.398+0000 7fde78da5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ae2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.399+0000 7fde6b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ae2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.399+0000 7fde6b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ae2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42322/0 (socket says 172.21.15.67:42322) 2024-07-31T15:11:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.399+0000 7fde6b7fe640 1 -- 172.21.15.67:0/2714450765 learned_addr learned my addr 172.21.15.67:0/2714450765 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.399+0000 7fde78da5740 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ade7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.399+0000 7fde78da5740 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 0x558a58adeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.399+0000 7fde78da5740 1 -- 172.21.15.67:0/2714450765 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fde580113f0 con 0x558a587022f0 2024-07-31T15:11:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.399+0000 7fde6affd640 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ade7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.399+0000 7fde78da5740 1 -- 172.21.15.67:0/2714450765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fde58010d20 con 0x558a5870a4d0 2024-07-31T15:11:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.399+0000 7fde78da5740 1 -- 172.21.15.67:0/2714450765 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fde5800df80 con 0x558a5872b1b0 2024-07-31T15:11:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.400+0000 7fde6bfff640 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 0x558a58adeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.400+0000 7fde78da5740 10 client.0 _ll_register_callbacks cb 0x7fde64034c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.400+0000 7fde6affd640 1 -- 172.21.15.67:0/2714450765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 msgr2=0x558a58adeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.400+0000 7fde6affd640 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 0x558a58adeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.400+0000 7fde6affd640 1 -- 172.21.15.67:0/2714450765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 msgr2=0x558a58ae2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.400+0000 7fde6affd640 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ae2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.400+0000 7fde6affd640 1 -- 172.21.15.67:0/2714450765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a58b51f80 con 0x558a5870a4d0 2024-07-31T15:11:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.400+0000 7fde6b7fe640 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ae2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.400+0000 7fde78da5740 1 -- 172.21.15.67:0/2714450765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde64056070 con 0x558a5870a4d0 2024-07-31T15:11:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.401+0000 7fde6affd640 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ade7b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x558a58adc6e0 tx=0x7fde64009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1283cca1cbd06179 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.401+0000 7fde4f7f6640 10 client.29327 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:11:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.401+0000 7fde4f7f6640 1 -- 172.21.15.67:0/2714450765 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64074060 con 0x558a5870a4d0 2024-07-31T15:11:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.401+0000 7fde4f7f6640 1 -- 172.21.15.67:0/2714450765 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fde6405ab70 con 0x558a5870a4d0 2024-07-31T15:11:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.401+0000 7fde4f7f6640 1 -- 172.21.15.67:0/2714450765 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fde64069040 con 0x558a5870a4d0 2024-07-31T15:11:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.401+0000 7fde78da5740 10 client.29327 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.401+0000 7fde78da5740 1 -- 172.21.15.67:0/2714450765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=66}) -- 0x558a58b52f40 con 0x558a5870a4d0 2024-07-31T15:11:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.402+0000 7fde4f7f6640 1 -- 172.21.15.67:0/2714450765 <== mon.2 v2:172.21.15.73:3300/0 4 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fde64077050 con 0x558a5870a4d0 2024-07-31T15:11:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.402+0000 7fde78da5740 10 client.29327 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.402+0000 7fde78da5740 10 client.29327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:11:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.402+0000 7fde78da5740 20 client.29327 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.402+0000 7fde78da5740 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558a586bf1c0 0x558a58b552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.402+0000 7fde78da5740 4 client.29327 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:11:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.402+0000 7fde78da5740 1 -- 172.21.15.67:0/2714450765 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a58a40da0 con 0x558a586bf1c0 2024-07-31T15:11:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.402+0000 7fde6b7fe640 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558a586bf1c0 0x558a58b552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.403+0000 7fde4f7f6640 10 client.29327 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:43.403+0000 7fde6b7fe640 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558a586bf1c0 0x558a58b552c0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ee474e5b5fda71 server_cookie=6d356071e1c9ef38 in_seq=0 out_seq=0 2024-07-31T15:11:43.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:43 smithi077 ceph-mon[37327]: pgmap v1059: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.5 MiB/s wr, 238 op/s 2024-07-31T15:11:43.542 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/md-bitmap.o 2024-07-31T15:11:43.604 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T15:11:43.622 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_huc_fw.o 2024-07-31T15:11:43.727 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/md-autodetect.o 2024-07-31T15:11:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.573+0000 7fde4f7f6640 1 -- 172.21.15.67:0/2714450765 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a58a40da0 con 0x558a586bf1c0 2024-07-31T15:11:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.573+0000 7fde4f7f6640 10 client.29327 handle_command_reply: tid=0 2024-07-31T15:11:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.573+0000 7fde78da5740 1 client.29327 shutdown 2024-07-31T15:11:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.574+0000 7fde78da5740 1 -- 172.21.15.67:0/2714450765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 msgr2=0x558a58ade7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.574+0000 7fde78da5740 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ade7b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x558a58adc6e0 tx=0x7fde64009630 comp rx=0 tx=0).stop 2024-07-31T15:11:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.574+0000 7fde78da5740 1 -- 172.21.15.67:0/2714450765 shutdown_connections 2024-07-31T15:11:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.574+0000 7fde78da5740 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x558a586bf1c0 0x558a58b552c0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.574+0000 7fde78da5740 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a5872b1b0 0x558a58adeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.574+0000 7fde78da5740 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a5870a4d0 0x558a58ade7b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.574+0000 7fde78da5740 1 --2- 172.21.15.67:0/2714450765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a587022f0 0x558a58ae2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.574+0000 7fde78da5740 1 -- 172.21.15.67:0/2714450765 >> 172.21.15.67:0/2714450765 conn(0x558a5871f300 msgr2=0x558a58ad45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.574+0000 7fde4f7f6640 0 client.29327 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.574+0000 7fde78da5740 1 -- 172.21.15.67:0/2714450765 shutdown_connections 2024-07-31T15:11:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.575+0000 7fde78da5740 1 -- 172.21.15.67:0/2714450765 wait complete. 2024-07-31T15:11:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.575+0000 7fde78da5740 20 client.29327 trim_cache size 0 max 16384 2024-07-31T15:11:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:11:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:11:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:11:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:11:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.575+0000 7fde71ea9640 1 -- 172.21.15.67:0/656469261 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fde540b3ea0 msgr2=0x7fde540b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.576+0000 7fde71ea9640 1 --2- 172.21.15.67:0/656469261 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fde540b3ea0 0x7fde540b63b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fde64009a60 tx=0x7fde64005bd0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.576+0000 7fde71ea9640 1 -- 172.21.15.67:0/656469261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde6c10fd80 msgr2=0x7fde6c080a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.576+0000 7fde71ea9640 1 --2- 172.21.15.67:0/656469261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde6c10fd80 0x7fde6c080a90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fde6000d680 tx=0x7fde6000db40 comp rx=0 tx=0).stop 2024-07-31T15:11:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.576+0000 7fde71ea9640 1 -- 172.21.15.67:0/656469261 shutdown_connections 2024-07-31T15:11:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.576+0000 7fde71ea9640 1 --2- 172.21.15.67:0/656469261 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fde540b3ea0 0x7fde540b63b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.576+0000 7fde71ea9640 1 --2- 172.21.15.67:0/656469261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde6c10fd80 0x7fde6c080a90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.576+0000 7fde71ea9640 1 --2- 172.21.15.67:0/656469261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde6c10f430 0x7fde6c080550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.576+0000 7fde71ea9640 1 --2- 172.21.15.67:0/656469261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde6c10b460 0x7fde6c080010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.576+0000 7fde71ea9640 1 -- 172.21.15.67:0/656469261 >> 172.21.15.67:0/656469261 conn(0x7fde6c0707c0 msgr2=0x7fde6c10d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.579+0000 7fde71ea9640 1 -- 172.21.15.67:0/656469261 shutdown_connections 2024-07-31T15:11:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:44.579+0000 7fde71ea9640 1 -- 172.21.15.67:0/656469261 wait complete. 2024-07-31T15:11:44.709 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_uc.o 2024-07-31T15:11:44.910 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] drivers/thermal/intel/x86_pkg_temp_thermal.o 2024-07-31T15:11:44.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.926+0000 7f37c699b640 1 -- 172.21.15.67:0/2852696082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c0107650 msgr2=0x7f37c0107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:44.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.926+0000 7f37c699b640 1 --2- 172.21.15.67:0/2852696082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c0107650 0x7f37c0107b30 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f37b401aba0 tx=0x7f37b403b6f0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.929+0000 7f37c699b640 1 -- 172.21.15.67:0/2852696082 shutdown_connections 2024-07-31T15:11:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.929+0000 7f37c699b640 1 --2- 172.21.15.67:0/2852696082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c0107650 0x7f37c0107b30 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.929+0000 7f37c699b640 1 --2- 172.21.15.67:0/2852696082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37c0102b60 0x7f37c0107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.929+0000 7f37c699b640 1 --2- 172.21.15.67:0/2852696082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c0102210 0x7f37c0102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.929+0000 7f37c699b640 1 -- 172.21.15.67:0/2852696082 >> 172.21.15.67:0/2852696082 conn(0x7f37c0079210 msgr2=0x7f37c0079620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:44.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.937+0000 7f37c699b640 1 -- 172.21.15.67:0/2852696082 shutdown_connections 2024-07-31T15:11:44.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.940+0000 7f37c699b640 1 -- 172.21.15.67:0/2852696082 wait complete. 2024-07-31T15:11:44.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.941+0000 7f37c699b640 1 Processor -- start 2024-07-31T15:11:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.943+0000 7f37c699b640 1 -- start start 2024-07-31T15:11:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.944+0000 7f37c699b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c0102210 0x7f37c010ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.944+0000 7f37c699b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37c0102b60 0x7f37c010f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.944+0000 7f37c699b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c0107650 0x7f37c0073c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.944+0000 7f37c699b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f37c010afa0 con 0x7f37c0102210 2024-07-31T15:11:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.944+0000 7f37c699b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f37c010ae20 con 0x7f37c0107650 2024-07-31T15:11:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.944+0000 7f37c699b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f37c010b120 con 0x7f37c0102b60 2024-07-31T15:11:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.945+0000 7f37bf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37c0102b60 0x7f37c010f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.945+0000 7f37bf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37c0102b60 0x7f37c010f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49312/0 (socket says 172.21.15.67:49312) 2024-07-31T15:11:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.945+0000 7f37bf7fe640 1 -- 172.21.15.67:0/3978128390 learned_addr learned my addr 172.21.15.67:0/3978128390 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.946+0000 7f37c4f38640 1 --2- 172.21.15.67:0/3978128390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c0107650 0x7f37c0073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.946+0000 7f37bffff640 1 --2- 172.21.15.67:0/3978128390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c0102210 0x7f37c010ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.947+0000 7f37bffff640 1 -- 172.21.15.67:0/3978128390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37c0102b60 msgr2=0x7f37c010f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.947+0000 7f37bffff640 1 --2- 172.21.15.67:0/3978128390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37c0102b60 0x7f37c010f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.947+0000 7f37bffff640 1 -- 172.21.15.67:0/3978128390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c0107650 msgr2=0x7f37c0073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:44.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.947+0000 7f37bffff640 1 --2- 172.21.15.67:0/3978128390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c0107650 0x7f37c0073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:44.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.947+0000 7f37bffff640 1 -- 172.21.15.67:0/3978128390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c00743d0 con 0x7f37c0102210 2024-07-31T15:11:44.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.947+0000 7f37bffff640 1 --2- 172.21.15.67:0/3978128390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c0102210 0x7f37c010ef20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f37b800bdf0 tx=0x7f37b8009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:44.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.952+0000 7f37bd7fa640 1 -- 172.21.15.67:0/3978128390 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8019070 con 0x7f37c0102210 2024-07-31T15:11:44.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.952+0000 7f37bd7fa640 1 -- 172.21.15.67:0/3978128390 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b80148f0 con 0x7f37c0102210 2024-07-31T15:11:44.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.952+0000 7f37bd7fa640 1 -- 172.21.15.67:0/3978128390 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b80128e0 con 0x7f37c0102210 2024-07-31T15:11:44.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.953+0000 7f37c699b640 1 -- 172.21.15.67:0/3978128390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c00746c0 con 0x7f37c0102210 2024-07-31T15:11:44.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.953+0000 7f37c699b640 1 -- 172.21.15.67:0/3978128390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c01acfd0 con 0x7f37c0102210 2024-07-31T15:11:44.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.957+0000 7f37c699b640 1 -- 172.21.15.67:0/3978128390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ac0052c0 con 0x7f37c0102210 2024-07-31T15:11:44.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.960+0000 7f37bd7fa640 1 -- 172.21.15.67:0/3978128390 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f37b801c070 con 0x7f37c0102210 2024-07-31T15:11:44.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.961+0000 7f37bd7fa640 1 --2- 172.21.15.67:0/3978128390 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37a80b3f50 0x7f37a80b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:44.964 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_uc_debugfs.o 2024-07-31T15:11:44.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.961+0000 7f37bd7fa640 1 -- 172.21.15.67:0/3978128390 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7f37b80e37d0 con 0x7f37c0102210 2024-07-31T15:11:44.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.962+0000 7f37bf7fe640 1 --2- 172.21.15.67:0/3978128390 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37a80b3f50 0x7f37a80b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:44.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.963+0000 7f37bf7fe640 1 --2- 172.21.15.67:0/3978128390 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37a80b3f50 0x7f37a80b6460 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f37b0006fd0 tx=0x7f37b0008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:44.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:44.969+0000 7f37bd7fa640 1 -- 172.21.15.67:0/3978128390 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f37b8012340 con 0x7f37c0102210 2024-07-31T15:11:44.985 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:11:45.355 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:45 smithi067 ceph-mon[32750]: pgmap v1060: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 238 op/s 2024-07-31T15:11:45.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:45 smithi073 ceph-mon[36714]: pgmap v1060: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 238 op/s 2024-07-31T15:11:45.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:45 smithi077 ceph-mon[37327]: pgmap v1060: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 238 op/s 2024-07-31T15:11:45.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:45.493+0000 7f37c699b640 1 -- 172.21.15.67:0/3978128390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37ac005880 con 0x7f37c0102210 2024-07-31T15:11:45.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:45.495+0000 7f37bd7fa640 1 -- 172.21.15.67:0/3978128390 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11959 (secure 0 0 0) 0x7f37b80aa1d0 con 0x7f37c0102210 2024-07-31T15:11:45.497 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:11:45.497 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 66 2024-07-31T15:11:45.498 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":66,"btime":"2024-07-31T15:10:30:235518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:10:30.235494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29129":{"gid":29129,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/631186801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":631186801},{"type":"v1","addr":"172.21.15.73:6835","nonce":631186801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:11:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:45.502+0000 7f37c699b640 1 -- 172.21.15.67:0/3978128390 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37a80b3f50 msgr2=0x7f37a80b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:45.502+0000 7f37c699b640 1 --2- 172.21.15.67:0/3978128390 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37a80b3f50 0x7f37a80b6460 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f37b0006fd0 tx=0x7f37b0008040 comp rx=0 tx=0).stop 2024-07-31T15:11:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:45.502+0000 7f37c699b640 1 -- 172.21.15.67:0/3978128390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c0102210 msgr2=0x7f37c010ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:45.502+0000 7f37c699b640 1 --2- 172.21.15.67:0/3978128390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c0102210 0x7f37c010ef20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f37b800bdf0 tx=0x7f37b8009f70 comp rx=0 tx=0).stop 2024-07-31T15:11:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:45.507+0000 7f37c699b640 1 -- 172.21.15.67:0/3978128390 shutdown_connections 2024-07-31T15:11:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:45.507+0000 7f37c699b640 1 --2- 172.21.15.67:0/3978128390 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f37a80b3f50 0x7f37a80b6460 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:45.507+0000 7f37c699b640 1 --2- 172.21.15.67:0/3978128390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c0107650 0x7f37c0073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:45.507+0000 7f37c699b640 1 --2- 172.21.15.67:0/3978128390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37c0102b60 0x7f37c010f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:45.507+0000 7f37c699b640 1 --2- 172.21.15.67:0/3978128390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c0102210 0x7f37c010ef20 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:45.507+0000 7f37c699b640 1 -- 172.21.15.67:0/3978128390 >> 172.21.15.67:0/3978128390 conn(0x7f37c0079210 msgr2=0x7f37c0105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:45.508+0000 7f37c699b640 1 -- 172.21.15.67:0/3978128390 shutdown_connections 2024-07-31T15:11:45.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:45.509+0000 7f37c699b640 1 -- 172.21.15.67:0/3978128390 wait complete. 2024-07-31T15:11:45.535 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_uc_fw.o 2024-07-31T15:11:45.554 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpufreq/freq_table.o 2024-07-31T15:11:45.556 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/nvidia/built-in.a 2024-07-31T15:11:45.622 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/oki-semi/built-in.a 2024-07-31T15:11:45.646 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/packetengines/built-in.a 2024-07-31T15:11:45.670 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/qlogic/built-in.a 2024-07-31T15:11:45.691 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/thermal/intel/built-in.a 2024-07-31T15:11:45.704 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/qualcomm/emac/built-in.a 2024-07-31T15:11:45.718 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/qualcomm/built-in.a 2024-07-31T15:11:45.736 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/thermal/tegra/built-in.a 2024-07-31T15:11:45.763 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/realtek/8139too.o 2024-07-31T15:11:45.771 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/thermal/mediatek/built-in.a 2024-07-31T15:11:45.789 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/thermal/thermal_core.o 2024-07-31T15:11:45.825 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/realtek/r8169_main.o 2024-07-31T15:11:45.841 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/realtek/r8169_firmware.o 2024-07-31T15:11:45.887 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/gt/intel_gsc.o 2024-07-31T15:11:46.048 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpufreq/cpufreq_performance.o 2024-07-31T15:11:46.258 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/net/ethernet/realtek/r8169_phy_config.o 2024-07-31T15:11:46.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:46 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3978128390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:11:46.434 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpufreq/cpufreq_userspace.o 2024-07-31T15:11:46.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:46 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3978128390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:11:46.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:46 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3978128390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:11:46.677 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_hwmon.o 2024-07-31T15:11:46.695 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0' 2024-07-31T15:11:46.695 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0 2024-07-31T15:11:46.844 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/hsw_ips.o 2024-07-31T15:11:46.910 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpufreq/cpufreq_ondemand.o 2024-07-31T15:11:46.980 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpufreq/cpufreq_governor.o 2024-07-31T15:11:47.022 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/thermal/thermal_sysfs.o 2024-07-31T15:11:47.314 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/thermal/thermal_trip.o 2024-07-31T15:11:47.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:47 smithi073 ceph-mon[36714]: pgmap v1061: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-07-31T15:11:47.446 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:11:47.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:47 smithi077 ceph-mon[37327]: pgmap v1061: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-07-31T15:11:47.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:47 smithi067 ceph-mon[32750]: pgmap v1061: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-07-31T15:11:47.630 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpufreq/cpufreq_governor_attr_set.o 2024-07-31T15:11:47.650 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpufreq/acpi-cpufreq.o 2024-07-31T15:11:47.780 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_atomic.o 2024-07-31T15:11:47.787 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/thermal/thermal_helpers.o 2024-07-31T15:11:47.792 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpuidle/governors/menu.o 2024-07-31T15:11:48.107 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpufreq/intel_pstate.o 2024-07-31T15:11:48.117 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_atomic_plane.o 2024-07-31T15:11:48.277 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/thermal/thermal_hwmon.o 2024-07-31T15:11:48.288 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpuidle/governors/haltpoll.o 2024-07-31T15:11:48.358 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/realtek/built-in.a 2024-07-31T15:11:48.410 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm.o 2024-07-31T15:11:48.416 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/renesas/built-in.a 2024-07-31T15:11:48.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.385+0000 7f87fa6ea640 1 -- 172.21.15.67:0/2610920075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f4111770 msgr2=0x7f87f410a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:48.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.385+0000 7f87fa6ea640 1 --2- 172.21.15.67:0/2610920075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f4111770 0x7f87f410a070 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009ca0 tx=0x7f87ec02a8b0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.388+0000 7f87fa6ea640 1 -- 172.21.15.67:0/2610920075 shutdown_connections 2024-07-31T15:11:48.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.388+0000 7f87fa6ea640 1 --2- 172.21.15.67:0/2610920075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f411c5d0 0x7f87f411e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.388+0000 7f87fa6ea640 1 --2- 172.21.15.67:0/2610920075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87f410a5b0 0x7f87f410a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.389+0000 7f87fa6ea640 1 --2- 172.21.15.67:0/2610920075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f4111770 0x7f87f410a070 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.389+0000 7f87fa6ea640 1 -- 172.21.15.67:0/2610920075 >> 172.21.15.67:0/2610920075 conn(0x7f87f406d380 msgr2=0x7f87f406d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:48.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.392+0000 7f87fa6ea640 1 -- 172.21.15.67:0/2610920075 shutdown_connections 2024-07-31T15:11:48.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.392+0000 7f87fa6ea640 1 -- 172.21.15.67:0/2610920075 wait complete. 2024-07-31T15:11:48.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.393+0000 7f87fa6ea640 1 Processor -- start 2024-07-31T15:11:48.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.393+0000 7f87fa6ea640 1 -- start start 2024-07-31T15:11:48.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.393+0000 7f87fa6ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f410a5b0 0x7f87f411bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.393+0000 7f87fa6ea640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87f4111770 0x7f87f4115200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.393+0000 7f87fa6ea640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f411c5d0 0x7f87f4115880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.393+0000 7f87fa6ea640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f87f4122150 con 0x7f87f411c5d0 2024-07-31T15:11:48.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.393+0000 7f87fa6ea640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f87f4121fd0 con 0x7f87f4111770 2024-07-31T15:11:48.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.393+0000 7f87fa6ea640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87f41222d0 con 0x7f87f410a5b0 2024-07-31T15:11:48.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.394+0000 7f87f8c87640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f411c5d0 0x7f87f4115880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:48.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.394+0000 7f87f8c87640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f411c5d0 0x7f87f4115880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42366/0 (socket says 172.21.15.67:42366) 2024-07-31T15:11:48.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.394+0000 7f87f8c87640 1 -- 172.21.15.67:0/2018209639 learned_addr learned my addr 172.21.15.67:0/2018209639 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:48.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.395+0000 7f87f8c87640 1 -- 172.21.15.67:0/2018209639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f410a5b0 msgr2=0x7f87f411bf30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:11:48.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.395+0000 7f87f8c87640 1 --2- 172.21.15.67:0/2018209639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f410a5b0 0x7f87f411bf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.395+0000 7f87f8c87640 1 -- 172.21.15.67:0/2018209639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87f4111770 msgr2=0x7f87f4115200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:11:48.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.395+0000 7f87f8c87640 1 --2- 172.21.15.67:0/2018209639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87f4111770 0x7f87f4115200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.395+0000 7f87f8c87640 1 -- 172.21.15.67:0/2018209639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f411aba0 con 0x7f87f411c5d0 2024-07-31T15:11:48.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.396+0000 7f87f8c87640 1 --2- 172.21.15.67:0/2018209639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f411c5d0 0x7f87f4115880 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f87e400d750 tx=0x7f87e400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:48.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.397+0000 7f87f17fa640 1 -- 172.21.15.67:0/2018209639 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4017070 con 0x7f87f411c5d0 2024-07-31T15:11:48.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.397+0000 7f87fa6ea640 1 -- 172.21.15.67:0/2018209639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f411ae90 con 0x7f87f411c5d0 2024-07-31T15:11:48.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.397+0000 7f87fa6ea640 1 -- 172.21.15.67:0/2018209639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f411b3d0 con 0x7f87f411c5d0 2024-07-31T15:11:48.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.399+0000 7f87f17fa640 1 -- 172.21.15.67:0/2018209639 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e40054d0 con 0x7f87f411c5d0 2024-07-31T15:11:48.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.399+0000 7f87f17fa640 1 -- 172.21.15.67:0/2018209639 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4004c60 con 0x7f87f411c5d0 2024-07-31T15:11:48.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.402+0000 7f8800c3b740 1 Processor -- start 2024-07-31T15:11:48.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.404+0000 7f87f17fa640 1 -- 172.21.15.67:0/2018209639 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f87e4010460 con 0x7f87f411c5d0 2024-07-31T15:11:48.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.405+0000 7f8800c3b740 1 -- start start 2024-07-31T15:11:48.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.405+0000 7f8800c3b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305da070 0x55f93097f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.405+0000 7f8800c3b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305b11b0 0x55f93097fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.405+0000 7f8800c3b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b9390 0x55f930980190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.405+0000 7f8800c3b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f9308f5bb0 con 0x55f9305da070 2024-07-31T15:11:48.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.405+0000 7f8800c3b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f9308e50b0 con 0x55f9305b9390 2024-07-31T15:11:48.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.405+0000 7f8800c3b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f9308e90f0 con 0x55f9305b11b0 2024-07-31T15:11:48.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.406+0000 7f87f37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305da070 0x55f93097f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:48.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.406+0000 7f87f37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305da070 0x55f93097f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42374/0 (socket says 172.21.15.67:42374) 2024-07-31T15:11:48.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.406+0000 7f87f37fe640 1 -- 172.21.15.67:0/3374129149 learned_addr learned my addr 172.21.15.67:0/3374129149 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:48.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.407+0000 7f87f37fe640 1 -- 172.21.15.67:0/3374129149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305b11b0 msgr2=0x55f93097fc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:11:48.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.407+0000 7f87f37fe640 1 --2- 172.21.15.67:0/3374129149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305b11b0 0x55f93097fc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.407+0000 7f87f37fe640 1 -- 172.21.15.67:0/3374129149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b9390 msgr2=0x55f930980190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:11:48.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.407+0000 7f87f37fe640 1 --2- 172.21.15.67:0/3374129149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b9390 0x55f930980190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.407+0000 7f87f37fe640 1 -- 172.21.15.67:0/3374129149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f93088bb90 con 0x55f9305da070 2024-07-31T15:11:48.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.409+0000 7f87f37fe640 1 --2- 172.21.15.67:0/3374129149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305da070 0x55f93097f710 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f87e800b9f0 tx=0x7f87e800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34e5fddcfb8c33a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:48.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.410+0000 7f87f17fa640 1 --2- 172.21.15.67:0/2018209639 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f87dc0b3f00 0x7f87dc0b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.410+0000 7f87da7fc640 1 -- 172.21.15.67:0/3374129149 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8016070 con 0x55f9305da070 2024-07-31T15:11:48.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.410+0000 7f87da7fc640 1 -- 172.21.15.67:0/3374129149 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e80054d0 con 0x55f9305da070 2024-07-31T15:11:48.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.411+0000 7f87f17fa640 1 -- 172.21.15.67:0/2018209639 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7f87e40e3b10 con 0x7f87f411c5d0 2024-07-31T15:11:48.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.411+0000 7f8800c3b740 1 -- 172.21.15.67:0/3374129149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305da070 msgr2=0x55f93097f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:48.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.411+0000 7f8800c3b740 1 --2- 172.21.15.67:0/3374129149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305da070 0x55f93097f710 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f87e800b9f0 tx=0x7f87e800beb0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.411+0000 7f87f3fff640 1 --2- 172.21.15.67:0/2018209639 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f87dc0b3f00 0x7f87dc0b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.411+0000 7f8800c3b740 1 -- 172.21.15.67:0/3374129149 shutdown_connections 2024-07-31T15:11:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.411+0000 7f8800c3b740 1 --2- 172.21.15.67:0/3374129149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b9390 0x55f930980190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.411+0000 7f8800c3b740 1 --2- 172.21.15.67:0/3374129149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305b11b0 0x55f93097fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.411+0000 7f8800c3b740 1 --2- 172.21.15.67:0/3374129149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305da070 0x55f93097f710 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.411+0000 7f8800c3b740 1 -- 172.21.15.67:0/3374129149 >> 172.21.15.67:0/3374129149 conn(0x55f9305ce460 msgr2=0x55f93097d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.412+0000 7f87f3fff640 1 --2- 172.21.15.67:0/2018209639 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f87dc0b3f00 0x7f87dc0b6410 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f87ec02aea0 tx=0x7f87ec005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.413+0000 7f8800c3b740 1 -- 172.21.15.67:0/3374129149 shutdown_connections 2024-07-31T15:11:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.413+0000 7f8800c3b740 1 -- 172.21.15.67:0/3374129149 wait complete. 2024-07-31T15:11:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.414+0000 7f8800c3b740 1 -- start start 2024-07-31T15:11:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.415+0000 7f8800c3b740 1 Processor -- start 2024-07-31T15:11:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.416+0000 7f8800c3b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305b11b0 0x55f93098b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.416+0000 7f8800c3b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305b9390 0x55f93098b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.416+0000 7f87f3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305b11b0 0x55f93098b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:48.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.416+0000 7f87f3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305b11b0 0x55f93098b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42386/0 (socket says 172.21.15.67:42386) 2024-07-31T15:11:48.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.416+0000 7f87f3fff640 1 -- 172.21.15.67:0/2532578513 learned_addr learned my addr 172.21.15.67:0/2532578513 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:48.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.416+0000 7f8800c3b740 1 --2- 172.21.15.67:0/2532578513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305da070 0x55f930997310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.416+0000 7f8800c3b740 1 -- 172.21.15.67:0/2532578513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f930748700 con 0x55f9305b11b0 2024-07-31T15:11:48.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.416+0000 7f8800c3b740 1 -- 172.21.15.67:0/2532578513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f9308879e0 con 0x55f9305da070 2024-07-31T15:11:48.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.416+0000 7f8800c3b740 1 -- 172.21.15.67:0/2532578513 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f93098df40 con 0x55f9305b9390 2024-07-31T15:11:48.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.417+0000 7f8800c3b740 10 client.0 _ll_register_callbacks cb 0x55f9308a24f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:48.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.417+0000 7f87f8c87640 1 --2- 172.21.15.67:0/2532578513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305da070 0x55f930997310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:48.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.417+0000 7f87f3fff640 1 -- 172.21.15.67:0/2532578513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305b9390 msgr2=0x55f93098b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:48.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.417+0000 7f87f3fff640 1 --2- 172.21.15.67:0/2532578513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305b9390 0x55f93098b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.417+0000 7f87f3fff640 1 -- 172.21.15.67:0/2532578513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305da070 msgr2=0x55f930997310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:48.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.417+0000 7f87f3fff640 1 --2- 172.21.15.67:0/2532578513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305da070 0x55f930997310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.417+0000 7f87f3fff640 1 -- 172.21.15.67:0/2532578513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f930645650 con 0x55f9305b11b0 2024-07-31T15:11:48.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.418+0000 7f87f3fff640 1 --2- 172.21.15.67:0/2532578513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305b11b0 0x55f93098b2b0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f87ec035000 tx=0x7f87ec02a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c6f22e857336a9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:48.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.418+0000 7f87d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:11:48.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.418+0000 7f87d37fe640 1 -- 172.21.15.67:0/2532578513 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec004450 con 0x55f9305b11b0 2024-07-31T15:11:48.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.418+0000 7f8800c3b740 1 -- 172.21.15.67:0/2532578513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e80054d0 con 0x55f9305b11b0 2024-07-31T15:11:48.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.418+0000 7f87d37fe640 1 -- 172.21.15.67:0/2532578513 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7f87ec0045f0 con 0x55f9305b11b0 2024-07-31T15:11:48.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.419+0000 7f87d37fe640 1 -- 172.21.15.67:0/2532578513 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f87ec059050 con 0x55f9305b11b0 2024-07-31T15:11:48.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.419+0000 7f8800c3b740 10 client.19803 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:48.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.419+0000 7f8800c3b740 1 -- 172.21.15.67:0/2532578513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55f93099a010 con 0x55f9305b11b0 2024-07-31T15:11:48.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.420+0000 7f87d37fe640 1 -- 172.21.15.67:0/2532578513 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e80054d0 con 0x55f9305b11b0 2024-07-31T15:11:48.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.420+0000 7f87d37fe640 1 -- 172.21.15.67:0/2532578513 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f87ec050400 con 0x55f9305b11b0 2024-07-31T15:11:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.420+0000 7f8800c3b740 10 client.19803 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.420+0000 7f8800c3b740 10 client.19803 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:11:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.420+0000 7f8800c3b740 20 client.19803 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.421+0000 7f8800c3b740 1 --2- 172.21.15.67:0/2532578513 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f9305989f0 0x55f93099d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.421+0000 7f8800c3b740 4 client.19803 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:11:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.421+0000 7f8800c3b740 1 -- 172.21.15.67:0/2532578513 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9308f7b00 con 0x55f9305989f0 2024-07-31T15:11:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.421+0000 7f87f37fe640 1 --2- 172.21.15.67:0/2532578513 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f9305989f0 0x55f93099d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.422+0000 7f87f37fe640 1 --2- 172.21.15.67:0/2532578513 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f9305989f0 0x55f93099d640 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b72669f9996ff10 server_cookie=570591e18143ef8b in_seq=0 out_seq=0 2024-07-31T15:11:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.422+0000 7f87d37fe640 10 client.19803 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:11:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.423+0000 7f87d37fe640 1 -- 172.21.15.67:0/2532578513 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9308f7b00 con 0x55f9305989f0 2024-07-31T15:11:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.423+0000 7f87d37fe640 10 client.19803 handle_command_reply: tid=0 2024-07-31T15:11:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.423+0000 7f8800c3b740 1 client.19803 shutdown 2024-07-31T15:11:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.424+0000 7f8800c3b740 1 -- 172.21.15.67:0/2532578513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305b11b0 msgr2=0x55f93098b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.424+0000 7f8800c3b740 1 --2- 172.21.15.67:0/2532578513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305b11b0 0x55f93098b2b0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f87ec035000 tx=0x7f87ec02a8b0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.424+0000 7f8800c3b740 1 -- 172.21.15.67:0/2532578513 shutdown_connections 2024-07-31T15:11:48.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.424+0000 7f8800c3b740 1 --2- 172.21.15.67:0/2532578513 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f9305989f0 0x55f93099d640 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.424+0000 7f8800c3b740 1 --2- 172.21.15.67:0/2532578513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305da070 0x55f930997310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.424+0000 7f8800c3b740 1 --2- 172.21.15.67:0/2532578513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305b9390 0x55f93098b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.424+0000 7f8800c3b740 1 --2- 172.21.15.67:0/2532578513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305b11b0 0x55f93098b2b0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.424+0000 7f8800c3b740 1 -- 172.21.15.67:0/2532578513 >> 172.21.15.67:0/2532578513 conn(0x55f9305ce460 msgr2=0x55f9309828f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:48.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.425+0000 7f87d37fe640 0 client.19803 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:11:48.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.425+0000 7f8800c3b740 1 -- 172.21.15.67:0/2532578513 shutdown_connections 2024-07-31T15:11:48.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.426+0000 7f8800c3b740 1 -- 172.21.15.67:0/2532578513 wait complete. 2024-07-31T15:11:48.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.426+0000 7f8800c3b740 20 client.19803 trim_cache size 0 max 16384 2024-07-31T15:11:48.460 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/rdc/built-in.a 2024-07-31T15:11:48.469 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/rocker/built-in.a 2024-07-31T15:11:48.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.470+0000 7f8800c3b740 1 Processor -- start 2024-07-31T15:11:48.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.472+0000 7f8800c3b740 1 -- start start 2024-07-31T15:11:48.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.472+0000 7f8800c3b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b11b0 0x55f93098cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.472+0000 7f8800c3b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305b9390 0x55f930997100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.472+0000 7f8800c3b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305da070 0x55f930997640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.472+0000 7f8800c3b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f87e000fe50 con 0x55f9305b9390 2024-07-31T15:11:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.472+0000 7f8800c3b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f87e0011990 con 0x55f9305b11b0 2024-07-31T15:11:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.472+0000 7f8800c3b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87e0010520 con 0x55f9305da070 2024-07-31T15:11:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.473+0000 7f87f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b11b0 0x55f93098cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.473+0000 7f87f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b11b0 0x55f93098cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:44392/0 (socket says 172.21.15.67:44392) 2024-07-31T15:11:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.473+0000 7f87f3fff640 1 -- 172.21.15.67:0/4220886236 learned_addr learned my addr 172.21.15.67:0/4220886236 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.473+0000 7f87f8c87640 1 --2- 172.21.15.67:0/4220886236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305da070 0x55f930997640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.473+0000 7f87f37fe640 1 --2- 172.21.15.67:0/4220886236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305b9390 0x55f930997100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.473+0000 7f87f37fe640 1 -- 172.21.15.67:0/4220886236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305da070 msgr2=0x55f930997640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.473+0000 7f87f37fe640 1 --2- 172.21.15.67:0/4220886236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305da070 0x55f930997640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.473+0000 7f87f37fe640 1 -- 172.21.15.67:0/4220886236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b11b0 msgr2=0x55f93098cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:48.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.473+0000 7f87f37fe640 1 --2- 172.21.15.67:0/4220886236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b11b0 0x55f93098cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.473+0000 7f87f37fe640 1 -- 172.21.15.67:0/4220886236 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f93098c150 con 0x55f9305b9390 2024-07-31T15:11:48.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.474+0000 7f87f37fe640 1 --2- 172.21.15.67:0/4220886236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305b9390 0x55f930997100 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x55f93099dda0 tx=0x7f87e800df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f8a6b8ed34997b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:48.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.474+0000 7f87d9ffb640 1 -- 172.21.15.67:0/4220886236 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8028020 con 0x55f9305b9390 2024-07-31T15:11:48.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.474+0000 7f87d9ffb640 1 -- 172.21.15.67:0/4220886236 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e80054b0 con 0x55f9305b9390 2024-07-31T15:11:48.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.475+0000 7f87d9ffb640 1 -- 172.21.15.67:0/4220886236 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8016040 con 0x55f9305b9390 2024-07-31T15:11:48.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.476+0000 7f8800c3b740 1 -- 172.21.15.67:0/4220886236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305b9390 msgr2=0x55f930997100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:48.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.476+0000 7f8800c3b740 1 --2- 172.21.15.67:0/4220886236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305b9390 0x55f930997100 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x55f93099dda0 tx=0x7f87e800df90 comp rx=0 tx=0).stop 2024-07-31T15:11:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.476+0000 7f8800c3b740 1 -- 172.21.15.67:0/4220886236 shutdown_connections 2024-07-31T15:11:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.476+0000 7f8800c3b740 1 --2- 172.21.15.67:0/4220886236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305da070 0x55f930997640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.476+0000 7f8800c3b740 1 --2- 172.21.15.67:0/4220886236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305b9390 0x55f930997100 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.476+0000 7f8800c3b740 1 --2- 172.21.15.67:0/4220886236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b11b0 0x55f93098cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.477+0000 7f8800c3b740 1 -- 172.21.15.67:0/4220886236 >> 172.21.15.67:0/4220886236 conn(0x55f9305ce460 msgr2=0x55f93098c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.477+0000 7f8800c3b740 1 -- 172.21.15.67:0/4220886236 shutdown_connections 2024-07-31T15:11:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.477+0000 7f8800c3b740 1 -- 172.21.15.67:0/4220886236 wait complete. 2024-07-31T15:11:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.478+0000 7f8800c3b740 1 -- start start 2024-07-31T15:11:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.478+0000 7f8800c3b740 1 Processor -- start 2024-07-31T15:11:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.479+0000 7f8800c3b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305b11b0 0x55f93098ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.479+0000 7f8800c3b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b9390 0x55f93098ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.479+0000 7f8800c3b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305da070 0x55f930a03dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.480+0000 7f87f8c87640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305da070 0x55f930a03dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.480+0000 7f87f8c87640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305da070 0x55f930a03dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42406/0 (socket says 172.21.15.67:42406) 2024-07-31T15:11:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.480+0000 7f87f8c87640 1 -- 172.21.15.67:0/2650203536 learned_addr learned my addr 172.21.15.67:0/2650203536 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.480+0000 7f8800c3b740 1 -- 172.21.15.67:0/2650203536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f87e00112c0 con 0x55f9305da070 2024-07-31T15:11:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.480+0000 7f8800c3b740 1 -- 172.21.15.67:0/2650203536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f87e0010bf0 con 0x55f9305b9390 2024-07-31T15:11:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.480+0000 7f8800c3b740 1 -- 172.21.15.67:0/2650203536 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87e000de50 con 0x55f9305b11b0 2024-07-31T15:11:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.481+0000 7f87f37fe640 1 --2- 172.21.15.67:0/2650203536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b9390 0x55f93098ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.481+0000 7f87f3fff640 1 --2- 172.21.15.67:0/2650203536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305b11b0 0x55f93098ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.481+0000 7f87f8c87640 1 -- 172.21.15.67:0/2650203536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305b11b0 msgr2=0x55f93098ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.481+0000 7f87f8c87640 1 --2- 172.21.15.67:0/2650203536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305b11b0 0x55f93098ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.481+0000 7f87f8c87640 1 -- 172.21.15.67:0/2650203536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b9390 msgr2=0x55f93098ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.481+0000 7f87f8c87640 1 --2- 172.21.15.67:0/2650203536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b9390 0x55f93098ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.481+0000 7f87f8c87640 1 -- 172.21.15.67:0/2650203536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f930a04350 con 0x55f9305da070 2024-07-31T15:11:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.482+0000 7f87f8c87640 1 --2- 172.21.15.67:0/2650203536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305da070 0x55f930a03dc0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f87e400b470 tx=0x7f87e40aa9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79a8b9ac81fc764 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.482+0000 7f8800c3b740 10 client.0 _ll_register_callbacks cb 0x7f87ec009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.482+0000 7f8800c3b740 1 -- 172.21.15.67:0/2650203536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e80054b0 con 0x55f9305da070 2024-07-31T15:11:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.483+0000 7f87d37fe640 10 client.19809 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:11:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.483+0000 7f87d37fe640 1 -- 172.21.15.67:0/2650203536 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e40c4060 con 0x55f9305da070 2024-07-31T15:11:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.483+0000 7f87d37fe640 1 -- 172.21.15.67:0/2650203536 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7f87e40a9610 con 0x55f9305da070 2024-07-31T15:11:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.483+0000 7f87d37fe640 1 -- 172.21.15.67:0/2650203536 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f87e4020040 con 0x55f9305da070 2024-07-31T15:11:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.486+0000 7f8800c3b740 10 client.19809 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.486+0000 7f8800c3b740 1 -- 172.21.15.67:0/2650203536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({fsmap=66}) -- 0x55f930a05500 con 0x55f9305da070 2024-07-31T15:11:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.487+0000 7f87d37fe640 1 -- 172.21.15.67:0/2650203536 <== mon.0 v2:172.21.15.67:3300/0 4 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f87e40cb330 con 0x55f9305da070 2024-07-31T15:11:48.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.487+0000 7f8800c3b740 10 client.19809 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:48.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.487+0000 7f8800c3b740 10 client.19809 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:11:48.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.487+0000 7f8800c3b740 20 client.19809 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:48.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.487+0000 7f8800c3b740 1 --2- 172.21.15.67:0/2650203536 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f93056dda0 0x55f930a07b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:48.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.487+0000 7f8800c3b740 4 client.19809 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:11:48.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.487+0000 7f8800c3b740 1 -- 172.21.15.67:0/2650203536 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9308f35a0 con 0x55f93056dda0 2024-07-31T15:11:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.488+0000 7f87f3fff640 1 --2- 172.21.15.67:0/2650203536 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f93056dda0 0x55f930a07b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.488+0000 7f87f3fff640 1 --2- 172.21.15.67:0/2650203536 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f93056dda0 0x55f930a07b10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=22be3bc96395ca09 server_cookie=dd1f10b5508b2845 in_seq=0 out_seq=0 2024-07-31T15:11:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.489+0000 7f87d37fe640 10 client.19809 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:11:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.489+0000 7f87d37fe640 1 -- 172.21.15.67:0/2650203536 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9308f35a0 con 0x55f93056dda0 2024-07-31T15:11:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.489+0000 7f87d37fe640 10 client.19809 handle_command_reply: tid=0 2024-07-31T15:11:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.489+0000 7f8800c3b740 1 client.19809 shutdown 2024-07-31T15:11:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.489+0000 7f8800c3b740 1 -- 172.21.15.67:0/2650203536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305da070 msgr2=0x55f930a03dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.490+0000 7f8800c3b740 1 --2- 172.21.15.67:0/2650203536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305da070 0x55f930a03dc0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f87e400b470 tx=0x7f87e40aa9d0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.491+0000 7f8800c3b740 1 -- 172.21.15.67:0/2650203536 shutdown_connections 2024-07-31T15:11:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.491+0000 7f8800c3b740 1 --2- 172.21.15.67:0/2650203536 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55f93056dda0 0x55f930a07b10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.491+0000 7f8800c3b740 1 --2- 172.21.15.67:0/2650203536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9305da070 0x55f930a03dc0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.491+0000 7f87d37fe640 0 client.19809 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:11:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.491+0000 7f8800c3b740 1 --2- 172.21.15.67:0/2650203536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f9305b9390 0x55f93098ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.491+0000 7f8800c3b740 1 --2- 172.21.15.67:0/2650203536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9305b11b0 0x55f93098ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.491+0000 7f8800c3b740 1 -- 172.21.15.67:0/2650203536 >> 172.21.15.67:0/2650203536 conn(0x55f9305ce460 msgr2=0x55f930980740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.491+0000 7f8800c3b740 1 -- 172.21.15.67:0/2650203536 shutdown_connections 2024-07-31T15:11:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.491+0000 7f8800c3b740 1 -- 172.21.15.67:0/2650203536 wait complete. 2024-07-31T15:11:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.492+0000 7f8800c3b740 20 client.19809 trim_cache size 0 max 16384 2024-07-31T15:11:48.495 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:11:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.495+0000 7f87fa6ea640 1 -- 172.21.15.67:0/2018209639 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f87dc0b3f00 msgr2=0x7f87dc0b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.495+0000 7f87fa6ea640 1 --2- 172.21.15.67:0/2018209639 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f87dc0b3f00 0x7f87dc0b6410 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f87ec02aea0 tx=0x7f87ec005fd0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.495+0000 7f87fa6ea640 1 -- 172.21.15.67:0/2018209639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f411c5d0 msgr2=0x7f87f4115880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.495+0000 7f87fa6ea640 1 --2- 172.21.15.67:0/2018209639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f411c5d0 0x7f87f4115880 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f87e400d750 tx=0x7f87e400dc10 comp rx=0 tx=0).stop 2024-07-31T15:11:48.498 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/samsung/built-in.a 2024-07-31T15:11:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.495+0000 7f87fa6ea640 1 -- 172.21.15.67:0/2018209639 shutdown_connections 2024-07-31T15:11:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.495+0000 7f87fa6ea640 1 --2- 172.21.15.67:0/2018209639 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f87dc0b3f00 0x7f87dc0b6410 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.495+0000 7f87fa6ea640 1 --2- 172.21.15.67:0/2018209639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f411c5d0 0x7f87f4115880 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.495+0000 7f87fa6ea640 1 --2- 172.21.15.67:0/2018209639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87f4111770 0x7f87f4115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.495+0000 7f87fa6ea640 1 --2- 172.21.15.67:0/2018209639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87f410a5b0 0x7f87f411bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.496+0000 7f87fa6ea640 1 -- 172.21.15.67:0/2018209639 >> 172.21.15.67:0/2018209639 conn(0x7f87f406d380 msgr2=0x7f87f4110400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.497+0000 7f87fa6ea640 1 -- 172.21.15.67:0/2018209639 shutdown_connections 2024-07-31T15:11:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:48.497+0000 7f87fa6ea640 1 -- 172.21.15.67:0/2018209639 wait complete. 2024-07-31T15:11:48.511 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/seeq/built-in.a 2024-07-31T15:11:48.536 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-table.o 2024-07-31T15:11:48.538 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/silan/built-in.a 2024-07-31T15:11:48.571 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/sis/built-in.a 2024-07-31T15:11:48.596 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/sfc/built-in.a 2024-07-31T15:11:48.620 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/smsc/built-in.a 2024-07-31T15:11:48.643 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/socionext/built-in.a 2024-07-31T15:11:48.668 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/stmicro/built-in.a 2024-07-31T15:11:48.692 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/sun/built-in.a 2024-07-31T15:11:48.718 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/tehuti/built-in.a 2024-07-31T15:11:48.740 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/ti/built-in.a 2024-07-31T15:11:48.771 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/vertexcom/built-in.a 2024-07-31T15:11:48.772 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/thermal/gov_step_wise.o 2024-07-31T15:11:48.796 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/via/built-in.a 2024-07-31T15:11:48.824 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/wangxun/built-in.a 2024-07-31T15:11:48.832 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_audio.o 2024-07-31T15:11:48.843 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/wiznet/built-in.a 2024-07-31T15:11:48.870 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/xilinx/built-in.a 2024-07-31T15:11:48.897 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/xircom/built-in.a 2024-07-31T15:11:48.921 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/synopsys/built-in.a 2024-07-31T15:11:48.945 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/pensando/built-in.a 2024-07-31T15:11:48.966 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/ethernet/built-in.a 2024-07-31T15:11:49.028 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/cpuidle/governors/built-in.a 2024-07-31T15:11:49.116 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpuidle/cpuidle.o 2024-07-31T15:11:49.201 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/thermal/gov_user_space.o 2024-07-31T15:11:49.206 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/net/built-in.a 2024-07-31T15:11:49.289 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/mmc/built-in.a 2024-07-31T15:11:49.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:49 smithi067 ceph-mon[32750]: pgmap v1062: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 252 op/s 2024-07-31T15:11:49.313 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/ufs/built-in.a 2024-07-31T15:11:49.352 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_bios.o 2024-07-31T15:11:49.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:49 smithi073 ceph-mon[36714]: pgmap v1062: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 252 op/s 2024-07-31T15:11:49.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:49 smithi077 ceph-mon[37327]: pgmap v1062: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 252 op/s 2024-07-31T15:11:49.503 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpuidle/driver.o 2024-07-31T15:11:49.602 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/thermal/built-in.a 2024-07-31T15:11:49.637 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpuidle/governor.o 2024-07-31T15:11:49.689 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/cpufreq/built-in.a 2024-07-31T15:11:49.722 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpuidle/sysfs.o 2024-07-31T15:11:49.737 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpuidle/poll_state.o 2024-07-31T15:11:49.959 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/cpuidle/cpuidle-haltpoll.o 2024-07-31T15:11:50.004 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-target.o 2024-07-31T15:11:50.014 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-linear.o 2024-07-31T15:11:50.050 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-stripe.o 2024-07-31T15:11:50.091 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_bw.o 2024-07-31T15:11:50.238 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T15:11:50.254 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_cdclk.o 2024-07-31T15:11:50.275 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-ioctl.o 2024-07-31T15:11:50.475 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/cpuidle/built-in.a 2024-07-31T15:11:50.572 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_color.o 2024-07-31T15:11:50.694 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-io.o 2024-07-31T15:11:50.804 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-kcopyd.o 2024-07-31T15:11:50.817 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-sysfs.o 2024-07-31T15:11:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.910+0000 7fbd4d5c4640 1 -- 172.21.15.67:0/3791599044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4810c2b0 msgr2=0x7fbd48114800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.910+0000 7fbd4d5c4640 1 --2- 172.21.15.67:0/3791599044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4810c2b0 0x7fbd48114800 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009ca0 tx=0x7fbd4002aa10 comp rx=0 tx=0).stop 2024-07-31T15:11:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.911+0000 7fbd4d5c4640 1 -- 172.21.15.67:0/3791599044 shutdown_connections 2024-07-31T15:11:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.911+0000 7fbd4d5c4640 1 --2- 172.21.15.67:0/3791599044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd4810a700 0x7fbd4810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.911+0000 7fbd4d5c4640 1 --2- 172.21.15.67:0/3791599044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4810c2b0 0x7fbd48114800 secure :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009ca0 tx=0x7fbd4002aa10 comp rx=0 tx=0).stop 2024-07-31T15:11:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.911+0000 7fbd4d5c4640 1 --2- 172.21.15.67:0/3791599044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd4810b960 0x7fbd4810bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.911+0000 7fbd4d5c4640 1 -- 172.21.15.67:0/3791599044 >> 172.21.15.67:0/3791599044 conn(0x7fbd4806e440 msgr2=0x7fbd4806e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:50.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.918+0000 7fbd4d5c4640 1 -- 172.21.15.67:0/3791599044 shutdown_connections 2024-07-31T15:11:50.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.918+0000 7fbd4d5c4640 1 -- 172.21.15.67:0/3791599044 wait complete. 2024-07-31T15:11:50.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.919+0000 7fbd4d5c4640 1 Processor -- start 2024-07-31T15:11:50.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.923+0000 7fbd4d5c4640 1 -- start start 2024-07-31T15:11:50.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.923+0000 7fbd4d5c4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4810a700 0x7fbd48111de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:50.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.923+0000 7fbd4d5c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd4810b960 0x7fbd48112320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:50.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.923+0000 7fbd4d5c4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd48112860 0x7fbd481aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:50.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.923+0000 7fbd4d5c4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4811de90 con 0x7fbd4810a700 2024-07-31T15:11:50.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.923+0000 7fbd4d5c4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4811dd10 con 0x7fbd48112860 2024-07-31T15:11:50.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.923+0000 7fbd4d5c4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4811e010 con 0x7fbd4810b960 2024-07-31T15:11:50.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.924+0000 7fbd467fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd4810b960 0x7fbd48112320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:50.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.924+0000 7fbd46ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4810a700 0x7fbd48111de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:50.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.924+0000 7fbd467fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd4810b960 0x7fbd48112320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58618/0 (socket says 172.21.15.67:58618) 2024-07-31T15:11:50.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.924+0000 7fbd467fc640 1 -- 172.21.15.67:0/4218391096 learned_addr learned my addr 172.21.15.67:0/4218391096 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:50.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.924+0000 7fbd477fe640 1 --2- 172.21.15.67:0/4218391096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd48112860 0x7fbd481aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:50.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.925+0000 7fbd477fe640 1 -- 172.21.15.67:0/4218391096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd4810b960 msgr2=0x7fbd48112320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:50.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.925+0000 7fbd477fe640 1 --2- 172.21.15.67:0/4218391096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd4810b960 0x7fbd48112320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:50.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.925+0000 7fbd477fe640 1 -- 172.21.15.67:0/4218391096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4810a700 msgr2=0x7fbd48111de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:50.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.925+0000 7fbd477fe640 1 --2- 172.21.15.67:0/4218391096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4810a700 0x7fbd48111de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:50.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.925+0000 7fbd477fe640 1 -- 172.21.15.67:0/4218391096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd481aabd0 con 0x7fbd48112860 2024-07-31T15:11:50.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.925+0000 7fbd477fe640 1 --2- 172.21.15.67:0/4218391096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd48112860 0x7fbd481aa3b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00b3e0 tx=0x7fbd3c00b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:50.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.926+0000 7fbd2ffff640 1 -- 172.21.15.67:0/4218391096 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c015020 con 0x7fbd48112860 2024-07-31T15:11:50.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.926+0000 7fbd2ffff640 1 -- 172.21.15.67:0/4218391096 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd3c011ce0 con 0x7fbd48112860 2024-07-31T15:11:50.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.929+0000 7fbd46ffd640 1 --2- 172.21.15.67:0/4218391096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4810a700 0x7fbd48111de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:50.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.930+0000 7fbd2ffff640 1 -- 172.21.15.67:0/4218391096 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c010e20 con 0x7fbd48112860 2024-07-31T15:11:50.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.933+0000 7fbd4d5c4640 1 -- 172.21.15.67:0/4218391096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd481aae60 con 0x7fbd48112860 2024-07-31T15:11:50.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.933+0000 7fbd4d5c4640 1 -- 172.21.15.67:0/4218391096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd481ab3a0 con 0x7fbd48112860 2024-07-31T15:11:50.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.939+0000 7fbd53b15740 1 Processor -- start 2024-07-31T15:11:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.943+0000 7fbd2ffff640 1 -- 172.21.15.67:0/4218391096 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbd3c018020 con 0x7fbd48112860 2024-07-31T15:11:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.945+0000 7fbd2ffff640 1 --2- 172.21.15.67:0/4218391096 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd300b3ce0 0x7fbd300b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.945+0000 7fbd2ffff640 1 -- 172.21.15.67:0/4218391096 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fbd3c0a6750 con 0x7fbd48112860 2024-07-31T15:11:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.946+0000 7fbd53b15740 1 -- start start 2024-07-31T15:11:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.946+0000 7fbd467fc640 1 --2- 172.21.15.67:0/4218391096 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd300b3ce0 0x7fbd300b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.946+0000 7fbd53b15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 0x560c7af14a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.946+0000 7fbd53b15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab45980 0x560c7af14fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.946+0000 7fbd53b15740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab4db60 0x560c7af154e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.946+0000 7fbd53b15740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560c7ae897b0 con 0x560c7ab4db60 2024-07-31T15:11:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.946+0000 7fbd53b15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560c7ae78cb0 con 0x560c7ab6e840 2024-07-31T15:11:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.946+0000 7fbd53b15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560c7ae7ccf0 con 0x560c7ab45980 2024-07-31T15:11:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.946+0000 7fbd477fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab4db60 0x560c7af154e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.946+0000 7fbd477fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab4db60 0x560c7af154e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37178/0 (socket says 172.21.15.67:37178) 2024-07-31T15:11:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.946+0000 7fbd477fe640 1 -- 172.21.15.67:0/3422946052 learned_addr learned my addr 172.21.15.67:0/3422946052 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.947+0000 7fbd467fc640 1 --2- 172.21.15.67:0/3422946052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab45980 0x560c7af14fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.947+0000 7fbd477fe640 1 -- 172.21.15.67:0/3422946052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab45980 msgr2=0x560c7af14fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.947+0000 7fbd477fe640 1 --2- 172.21.15.67:0/3422946052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab45980 0x560c7af14fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.947+0000 7fbd477fe640 1 -- 172.21.15.67:0/3422946052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 msgr2=0x560c7af14a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:11:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.947+0000 7fbd477fe640 1 --2- 172.21.15.67:0/3422946052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 0x560c7af14a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.947+0000 7fbd477fe640 1 -- 172.21.15.67:0/3422946052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7ae0e0b0 con 0x560c7ab4db60 2024-07-31T15:11:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.948+0000 7fbd477fe640 1 --2- 172.21.15.67:0/3422946052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab4db60 0x560c7af154e0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c006fd0 tx=0x7fbd3c0af880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87800eb5b28cf44 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.948+0000 7fbd467fc640 1 --2- 172.21.15.67:0/3422946052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab45980 0x560c7af14fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:11:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.948+0000 7fbd467fc640 1 --2- 172.21.15.67:0/4218391096 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd300b3ce0 0x7fbd300b61f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009a60 tx=0x7fbd40005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:50.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.956+0000 7fbd2d7fa640 1 -- 172.21.15.67:0/3422946052 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0a6950 con 0x560c7ab4db60 2024-07-31T15:11:50.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.956+0000 7fbd2d7fa640 1 -- 172.21.15.67:0/3422946052 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd3c0aa860 con 0x560c7ab4db60 2024-07-31T15:11:50.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.956+0000 7fbd2d7fa640 1 -- 172.21.15.67:0/3422946052 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0a6af0 con 0x560c7ab4db60 2024-07-31T15:11:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.958+0000 7fbd53b15740 1 -- 172.21.15.67:0/3422946052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab4db60 msgr2=0x560c7af154e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.958+0000 7fbd53b15740 1 --2- 172.21.15.67:0/3422946052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab4db60 0x560c7af154e0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c006fd0 tx=0x7fbd3c0af880 comp rx=0 tx=0).stop 2024-07-31T15:11:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.960+0000 7fbd53b15740 1 -- 172.21.15.67:0/3422946052 shutdown_connections 2024-07-31T15:11:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.960+0000 7fbd53b15740 1 --2- 172.21.15.67:0/3422946052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab4db60 0x560c7af154e0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.960+0000 7fbd53b15740 1 --2- 172.21.15.67:0/3422946052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab45980 0x560c7af14fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.960+0000 7fbd53b15740 1 --2- 172.21.15.67:0/3422946052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 0x560c7af14a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.960+0000 7fbd53b15740 1 -- 172.21.15.67:0/3422946052 >> 172.21.15.67:0/3422946052 conn(0x560c7ab62990 msgr2=0x560c7af12950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:50.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.965+0000 7fbd53b15740 1 -- 172.21.15.67:0/3422946052 shutdown_connections 2024-07-31T15:11:50.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.966+0000 7fbd53b15740 1 -- 172.21.15.67:0/3422946052 wait complete. 2024-07-31T15:11:50.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.966+0000 7fbd53b15740 1 -- start start 2024-07-31T15:11:50.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.967+0000 7fbd53b15740 1 Processor -- start 2024-07-31T15:11:50.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.971+0000 7fbd53b15740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab45980 0x560c7af20030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:50.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.971+0000 7fbd53b15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab4db60 0x560c7af20570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:50.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.971+0000 7fbd53b15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 0x560c7af20ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:50.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.971+0000 7fbd53b15740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560c7acdc7a0 con 0x560c7ab45980 2024-07-31T15:11:50.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.971+0000 7fbd53b15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560c7ae1b320 con 0x560c7ab6e840 2024-07-31T15:11:50.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.971+0000 7fbd53b15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560c7af23000 con 0x560c7ab4db60 2024-07-31T15:11:50.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.971+0000 7fbd53b15740 10 client.0 _ll_register_callbacks cb 0x560c7ae25140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:50.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.972+0000 7fbd46ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab45980 0x560c7af20030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:50.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.972+0000 7fbd46ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab45980 0x560c7af20030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37186/0 (socket says 172.21.15.67:37186) 2024-07-31T15:11:50.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.972+0000 7fbd46ffd640 1 -- 172.21.15.67:0/3125137769 learned_addr learned my addr 172.21.15.67:0/3125137769 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:50.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.972+0000 7fbd467fc640 1 --2- 172.21.15.67:0/3125137769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab4db60 0x560c7af20570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:50.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.973+0000 7fbd46ffd640 1 -- 172.21.15.67:0/3125137769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab4db60 msgr2=0x560c7af20570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:50.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.973+0000 7fbd46ffd640 1 --2- 172.21.15.67:0/3125137769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab4db60 0x560c7af20570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:50.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.973+0000 7fbd46ffd640 1 -- 172.21.15.67:0/3125137769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 msgr2=0x560c7af20ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:11:50.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.973+0000 7fbd46ffd640 1 --2- 172.21.15.67:0/3125137769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 0x560c7af20ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:50.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.973+0000 7fbd46ffd640 1 -- 172.21.15.67:0/3125137769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7ae1f4b0 con 0x560c7ab45980 2024-07-31T15:11:50.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.973+0000 7fbd46ffd640 1 --2- 172.21.15.67:0/3125137769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab45980 0x560c7af20030 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800e930 tx=0x7fbd3800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfaf6b5f6cac29a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:50.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.975+0000 7fbd2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:11:50.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.975+0000 7fbd2aff5640 1 -- 172.21.15.67:0/3125137769 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd38017070 con 0x560c7ab45980 2024-07-31T15:11:50.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.975+0000 7fbd53b15740 1 -- 172.21.15.67:0/3125137769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd3c0aa860 con 0x560c7ab45980 2024-07-31T15:11:50.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.975+0000 7fbd2aff5640 1 -- 172.21.15.67:0/3125137769 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fbd3800c770 con 0x560c7ab45980 2024-07-31T15:11:50.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.976+0000 7fbd2aff5640 1 -- 172.21.15.67:0/3125137769 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fbd38023050 con 0x560c7ab45980 2024-07-31T15:11:50.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.976+0000 7fbd53b15740 10 client.19821 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:50.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.976+0000 7fbd53b15740 1 -- 172.21.15.67:0/3125137769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x560c7abe8b90 con 0x560c7ab45980 2024-07-31T15:11:50.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.978+0000 7fbd2aff5640 1 -- 172.21.15.67:0/3125137769 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd3c0aa860 con 0x560c7ab45980 2024-07-31T15:11:50.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.978+0000 7fbd2aff5640 1 -- 172.21.15.67:0/3125137769 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fbd38013c90 con 0x560c7ab45980 2024-07-31T15:11:50.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.981+0000 7fbd53b15740 10 client.19821 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:50.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.981+0000 7fbd53b15740 10 client.19821 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:11:50.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.981+0000 7fbd53b15740 20 client.19821 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:50.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.981+0000 7fbd53b15740 1 --2- 172.21.15.67:0/3125137769 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560c7ab2d1c0 0x560c7af31f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.981+0000 7fbd53b15740 4 client.19821 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:11:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.981+0000 7fbd53b15740 1 -- 172.21.15.67:0/3125137769 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c7ae8b700 con 0x560c7ab2d1c0 2024-07-31T15:11:50.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.986+0000 7fbd467fc640 1 --2- 172.21.15.67:0/3125137769 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560c7ab2d1c0 0x560c7af31f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:50.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.997+0000 7fbd2aff5640 10 client.19821 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:50.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.997+0000 7fbd467fc640 1 --2- 172.21.15.67:0/3125137769 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560c7ab2d1c0 0x560c7af31f90 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e319ceafda3d4850 server_cookie=a4d946dc5613c5e0 in_seq=0 out_seq=0 2024-07-31T15:11:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.998+0000 7fbd2aff5640 1 -- 172.21.15.67:0/3125137769 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c7ae8b700 con 0x560c7ab2d1c0 2024-07-31T15:11:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:50.998+0000 7fbd2aff5640 10 client.19821 handle_command_reply: tid=0 2024-07-31T15:11:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.001+0000 7fbd53b15740 1 client.19821 shutdown 2024-07-31T15:11:51.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.017+0000 7fbd53b15740 1 -- 172.21.15.67:0/3125137769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab45980 msgr2=0x560c7af20030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:51.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.017+0000 7fbd53b15740 1 --2- 172.21.15.67:0/3125137769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab45980 0x560c7af20030 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800e930 tx=0x7fbd3800edf0 comp rx=0 tx=0).stop 2024-07-31T15:11:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.022+0000 7fbd53b15740 1 -- 172.21.15.67:0/3125137769 shutdown_connections 2024-07-31T15:11:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.022+0000 7fbd53b15740 1 --2- 172.21.15.67:0/3125137769 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560c7ab2d1c0 0x560c7af31f90 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.022+0000 7fbd53b15740 1 --2- 172.21.15.67:0/3125137769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 0x560c7af20ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.022+0000 7fbd53b15740 1 --2- 172.21.15.67:0/3125137769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab4db60 0x560c7af20570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.022+0000 7fbd53b15740 1 --2- 172.21.15.67:0/3125137769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab45980 0x560c7af20030 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.022+0000 7fbd53b15740 1 -- 172.21.15.67:0/3125137769 >> 172.21.15.67:0/3125137769 conn(0x560c7ab62990 msgr2=0x560c7af17c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:51.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.022+0000 7fbd2aff5640 0 client.19821 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:51.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.025+0000 7fbd53b15740 1 -- 172.21.15.67:0/3125137769 shutdown_connections 2024-07-31T15:11:51.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.025+0000 7fbd53b15740 1 -- 172.21.15.67:0/3125137769 wait complete. 2024-07-31T15:11:51.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.025+0000 7fbd53b15740 20 client.19821 trim_cache size 0 max 16384 2024-07-31T15:11:51.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.071+0000 7fbd53b15740 1 Processor -- start 2024-07-31T15:11:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.075+0000 7fbd53b15740 1 -- start start 2024-07-31T15:11:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.075+0000 7fbd53b15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab45980 0x560c7af16450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.075+0000 7fbd53b15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab4db60 0x560c7af21c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.075+0000 7fbd53b15740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab6e840 0x560c7af377a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.075+0000 7fbd53b15740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481c8210 con 0x560c7ab6e840 2024-07-31T15:11:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.075+0000 7fbd53b15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481c9d50 con 0x560c7ab45980 2024-07-31T15:11:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.075+0000 7fbd53b15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481c88e0 con 0x560c7ab4db60 2024-07-31T15:11:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.075+0000 7fbd46ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab45980 0x560c7af16450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.075+0000 7fbd46ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab45980 0x560c7af16450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:53610/0 (socket says 172.21.15.67:53610) 2024-07-31T15:11:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.075+0000 7fbd46ffd640 1 -- 172.21.15.67:0/1106023862 learned_addr learned my addr 172.21.15.67:0/1106023862 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.076+0000 7fbd477fe640 1 --2- 172.21.15.67:0/1106023862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab6e840 0x560c7af377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.076+0000 7fbd477fe640 1 -- 172.21.15.67:0/1106023862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab4db60 msgr2=0x560c7af21c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.076+0000 7fbd477fe640 1 --2- 172.21.15.67:0/1106023862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab4db60 0x560c7af21c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.076+0000 7fbd477fe640 1 -- 172.21.15.67:0/1106023862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab45980 msgr2=0x560c7af16450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.076+0000 7fbd477fe640 1 --2- 172.21.15.67:0/1106023862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab45980 0x560c7af16450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.076+0000 7fbd477fe640 1 -- 172.21.15.67:0/1106023862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7af1e530 con 0x560c7ab6e840 2024-07-31T15:11:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.077+0000 7fbd477fe640 1 --2- 172.21.15.67:0/1106023862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab6e840 0x560c7af377a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0aaa10 tx=0x7fbd3c00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e8f7469fe1eea6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.077+0000 7fbd2cff9640 1 -- 172.21.15.67:0/1106023862 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0c9070 con 0x560c7ab6e840 2024-07-31T15:11:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.078+0000 7fbd2cff9640 1 -- 172.21.15.67:0/1106023862 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd3c0b3210 con 0x560c7ab6e840 2024-07-31T15:11:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.078+0000 7fbd2cff9640 1 -- 172.21.15.67:0/1106023862 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0a6cc0 con 0x560c7ab6e840 2024-07-31T15:11:51.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.080+0000 7fbd53b15740 1 -- 172.21.15.67:0/1106023862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab6e840 msgr2=0x560c7af377a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:51.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.080+0000 7fbd53b15740 1 --2- 172.21.15.67:0/1106023862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab6e840 0x560c7af377a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0aaa10 tx=0x7fbd3c00bde0 comp rx=0 tx=0).stop 2024-07-31T15:11:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.084+0000 7fbd53b15740 1 -- 172.21.15.67:0/1106023862 shutdown_connections 2024-07-31T15:11:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.084+0000 7fbd53b15740 1 --2- 172.21.15.67:0/1106023862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab6e840 0x560c7af377a0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.084+0000 7fbd53b15740 1 --2- 172.21.15.67:0/1106023862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab4db60 0x560c7af21c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.084+0000 7fbd53b15740 1 --2- 172.21.15.67:0/1106023862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab45980 0x560c7af16450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.084+0000 7fbd53b15740 1 -- 172.21.15.67:0/1106023862 >> 172.21.15.67:0/1106023862 conn(0x560c7ab62990 msgr2=0x560c7af21550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:51.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.092+0000 7fbd53b15740 1 -- 172.21.15.67:0/1106023862 shutdown_connections 2024-07-31T15:11:51.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.093+0000 7fbd53b15740 1 -- 172.21.15.67:0/1106023862 wait complete. 2024-07-31T15:11:51.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.093+0000 7fbd53b15740 1 -- start start 2024-07-31T15:11:51.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.094+0000 7fbd53b15740 1 Processor -- start 2024-07-31T15:11:51.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.094+0000 7fbd53b15740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab45980 0x560c7af26260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:51.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.095+0000 7fbd53b15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab4db60 0x560c7af267a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:51.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.095+0000 7fbd53b15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 0x560c7af26ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:51.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.095+0000 7fbd53b15740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481c9680 con 0x560c7ab45980 2024-07-31T15:11:51.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.095+0000 7fbd53b15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481c8fb0 con 0x560c7ab6e840 2024-07-31T15:11:51.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.095+0000 7fbd53b15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481c6210 con 0x560c7ab4db60 2024-07-31T15:11:51.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.095+0000 7fbd53b15740 10 client.0 _ll_register_callbacks cb 0x7fbd3800e570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:51.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.096+0000 7fbd477fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 0x560c7af26ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:51.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.096+0000 7fbd477fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 0x560c7af26ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:53626/0 (socket says 172.21.15.67:53626) 2024-07-31T15:11:51.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.096+0000 7fbd477fe640 1 -- 172.21.15.67:0/1672966608 learned_addr learned my addr 172.21.15.67:0/1672966608 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:51.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.096+0000 7fbd467fc640 1 --2- 172.21.15.67:0/1672966608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab4db60 0x560c7af267a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:51.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.097+0000 7fbd477fe640 1 -- 172.21.15.67:0/1672966608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab4db60 msgr2=0x560c7af267a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:51.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.097+0000 7fbd477fe640 1 --2- 172.21.15.67:0/1672966608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab4db60 0x560c7af267a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:51.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.097+0000 7fbd477fe640 1 -- 172.21.15.67:0/1672966608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab45980 msgr2=0x560c7af26260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:51.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.097+0000 7fbd477fe640 1 --2- 172.21.15.67:0/1672966608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab45980 0x560c7af26260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:51.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.097+0000 7fbd477fe640 1 -- 172.21.15.67:0/1672966608 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7af998b0 con 0x560c7ab6e840 2024-07-31T15:11:51.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.102+0000 7fbd477fe640 1 --2- 172.21.15.67:0/1672966608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 0x560c7af26ce0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0a62d0 tx=0x7fbd3c0a6d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd8798e2ec71ce57 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:51.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.102+0000 7fbd2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:11:51.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.103+0000 7fbd2aff5640 1 -- 172.21.15.67:0/1672966608 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0c9070 con 0x560c7ab6e840 2024-07-31T15:11:51.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.103+0000 7fbd2aff5640 1 -- 172.21.15.67:0/1672966608 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fbd3c0a5e50 con 0x560c7ab6e840 2024-07-31T15:11:51.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.104+0000 7fbd53b15740 1 -- 172.21.15.67:0/1672966608 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd3c0b3210 con 0x560c7ab6e840 2024-07-31T15:11:51.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.107+0000 7fbd2aff5640 1 -- 172.21.15.67:0/1672966608 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fbd3c0a6e60 con 0x560c7ab6e840 2024-07-31T15:11:51.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.109+0000 7fbd53b15740 10 client.29351 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:51.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.109+0000 7fbd53b15740 1 -- 172.21.15.67:0/1672966608 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x560c7af9ab10 con 0x560c7ab6e840 2024-07-31T15:11:51.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.115+0000 7fbd2aff5640 1 -- 172.21.15.67:0/1672966608 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd3c0b3210 con 0x560c7ab6e840 2024-07-31T15:11:51.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.115+0000 7fbd2aff5640 1 -- 172.21.15.67:0/1672966608 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fbd3c022060 con 0x560c7ab6e840 2024-07-31T15:11:51.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.116+0000 7fbd53b15740 10 client.29351 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:51.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.116+0000 7fbd53b15740 10 client.29351 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:11:51.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.116+0000 7fbd53b15740 20 client.29351 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:51.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.116+0000 7fbd53b15740 1 --2- 172.21.15.67:0/1672966608 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560c7ab02570 0x560c7af9cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:51.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.116+0000 7fbd53b15740 4 client.29351 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}] 2024-07-31T15:11:51.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.116+0000 7fbd53b15740 1 -- 172.21.15.67:0/1672966608 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}) -- 0x560c7ae871a0 con 0x560c7ab02570 2024-07-31T15:11:51.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.117+0000 7fbd46ffd640 1 --2- 172.21.15.67:0/1672966608 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560c7ab02570 0x560c7af9cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:51.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.127+0000 7fbd46ffd640 1 --2- 172.21.15.67:0/1672966608 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560c7ab02570 0x560c7af9cc10 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce838762804fee4b server_cookie=b0b392e57816a948 in_seq=0 out_seq=0 2024-07-31T15:11:51.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:51.127+0000 7fbd2aff5640 10 client.29351 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:51.138 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:11:51.348 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:51 smithi067 ceph-mon[32750]: pgmap v1063: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 949 KiB/s wr, 234 op/s 2024-07-31T15:11:51.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:51 smithi073 ceph-mon[36714]: pgmap v1063: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 949 KiB/s wr, 234 op/s 2024-07-31T15:11:51.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:51 smithi077 ceph-mon[37327]: pgmap v1063: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 949 KiB/s wr, 234 op/s 2024-07-31T15:11:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.094+0000 7f07fd3a9640 1 -- 172.21.15.67:0/1132564163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f8107dd0 msgr2=0x7f07f81081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.094+0000 7f07fd3a9640 1 --2- 172.21.15.67:0/1132564163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f8107dd0 0x7f07f81081e0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009900 tx=0x7f07e002a410 comp rx=0 tx=0).stop 2024-07-31T15:11:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.094+0000 7f07fd3a9640 1 -- 172.21.15.67:0/1132564163 shutdown_connections 2024-07-31T15:11:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.094+0000 7f07fd3a9640 1 --2- 172.21.15.67:0/1132564163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f8108720 0x7f07f810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.094+0000 7f07fd3a9640 1 --2- 172.21.15.67:0/1132564163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f8107dd0 0x7f07f81081e0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.094+0000 7f07fd3a9640 1 --2- 172.21.15.67:0/1132564163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07f8102490 0x7f07f81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.094+0000 7f07fd3a9640 1 -- 172.21.15.67:0/1132564163 >> 172.21.15.67:0/1132564163 conn(0x7f07f8079160 msgr2=0x7f07f8079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.094+0000 7f07fd3a9640 1 -- 172.21.15.67:0/1132564163 shutdown_connections 2024-07-31T15:11:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.094+0000 7f07fd3a9640 1 -- 172.21.15.67:0/1132564163 wait complete. 2024-07-31T15:11:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.095+0000 7f07fd3a9640 1 Processor -- start 2024-07-31T15:11:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.095+0000 7f07fd3a9640 1 -- start start 2024-07-31T15:11:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.095+0000 7f07fd3a9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f8102490 0x7f07f819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.095+0000 7f07fd3a9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07f8107dd0 0x7f07f819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.095+0000 7f07fd3a9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f8108720 0x7f07f819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.095+0000 7f07fd3a9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f07f8112370 con 0x7f07f8108720 2024-07-31T15:11:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.095+0000 7f07fd3a9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f07f81121f0 con 0x7f07f8102490 2024-07-31T15:11:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.095+0000 7f07fd3a9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07f81124f0 con 0x7f07f8107dd0 2024-07-31T15:11:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.095+0000 7f07f6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f8102490 0x7f07f819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.095+0000 7f07ee7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07f8107dd0 0x7f07f819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.095+0000 7f07f6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f8102490 0x7f07f819ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:53632/0 (socket says 172.21.15.67:53632) 2024-07-31T15:11:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.095+0000 7f07f6ffd640 1 -- 172.21.15.67:0/3824171205 learned_addr learned my addr 172.21.15.67:0/3824171205 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.095+0000 7f07f77fe640 1 --2- 172.21.15.67:0/3824171205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f8108720 0x7f07f819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.096+0000 7f07f6ffd640 1 -- 172.21.15.67:0/3824171205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07f8107dd0 msgr2=0x7f07f819b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.096+0000 7f07f6ffd640 1 --2- 172.21.15.67:0/3824171205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07f8107dd0 0x7f07f819b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.096+0000 7f07f6ffd640 1 -- 172.21.15.67:0/3824171205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f8108720 msgr2=0x7f07f819b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.096+0000 7f07f6ffd640 1 --2- 172.21.15.67:0/3824171205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f8108720 0x7f07f819b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.096+0000 7f07f6ffd640 1 -- 172.21.15.67:0/3824171205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f8198af0 con 0x7f07f8102490 2024-07-31T15:11:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.096+0000 7f07f6ffd640 1 --2- 172.21.15.67:0/3824171205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f8102490 0x7f07f819ad00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f07e800e9f0 tx=0x7f07e800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.096+0000 7f07f4ff9640 1 -- 172.21.15.67:0/3824171205 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e800cd80 con 0x7f07f8102490 2024-07-31T15:11:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.096+0000 7f07fd3a9640 1 -- 172.21.15.67:0/3824171205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f8198de0 con 0x7f07f8102490 2024-07-31T15:11:53.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.096+0000 7f07f4ff9640 1 -- 172.21.15.67:0/3824171205 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07e80054d0 con 0x7f07f8102490 2024-07-31T15:11:53.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.096+0000 7f07f4ff9640 1 -- 172.21.15.67:0/3824171205 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8010690 con 0x7f07f8102490 2024-07-31T15:11:53.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.097+0000 7f07fd3a9640 1 -- 172.21.15.67:0/3824171205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f81992f0 con 0x7f07f8102490 2024-07-31T15:11:53.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.099+0000 7f07f4ff9640 1 -- 172.21.15.67:0/3824171205 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f07e8004e70 con 0x7f07f8102490 2024-07-31T15:11:53.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.100+0000 7f08038fa740 1 Processor -- start 2024-07-31T15:11:53.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.100+0000 7f08038fa740 1 -- start start 2024-07-31T15:11:53.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.100+0000 7f08038fa740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede89070 0x5566ee22e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.100+0000 7f08038fa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede601b0 0x5566ee22eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.100+0000 7f08038fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 0x5566ee22eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.100+0000 7f08038fa740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5566ee1a4bb0 con 0x5566ede89070 2024-07-31T15:11:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.100+0000 7f08038fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5566ee1940b0 con 0x5566ede68390 2024-07-31T15:11:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.100+0000 7f08038fa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5566ee1980f0 con 0x5566ede601b0 2024-07-31T15:11:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.100+0000 7f07f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede89070 0x5566ee22e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.100+0000 7f07f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede89070 0x5566ee22e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37222/0 (socket says 172.21.15.67:37222) 2024-07-31T15:11:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.100+0000 7f07f77fe640 1 -- 172.21.15.67:0/2470461632 learned_addr learned my addr 172.21.15.67:0/2470461632 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.100+0000 7f07f6ffd640 1 --2- 172.21.15.67:0/2470461632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede601b0 0x5566ee22eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.100+0000 7f07ee7fc640 1 --2- 172.21.15.67:0/2470461632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 0x5566ee22eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.101+0000 7f07f77fe640 1 -- 172.21.15.67:0/2470461632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede601b0 msgr2=0x5566ee22eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.101+0000 7f07f77fe640 1 --2- 172.21.15.67:0/2470461632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede601b0 0x5566ee22eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.101+0000 7f07f77fe640 1 -- 172.21.15.67:0/2470461632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 msgr2=0x5566ee22eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.101+0000 7f07f77fe640 1 --2- 172.21.15.67:0/2470461632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 0x5566ee22eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.101+0000 7f07f77fe640 1 -- 172.21.15.67:0/2470461632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ee13ab90 con 0x5566ede89070 2024-07-31T15:11:53.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.101+0000 7f07f77fe640 1 --2- 172.21.15.67:0/2470461632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede89070 0x5566ee22e570 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009fd0 tx=0x7f07f00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ecdedba28511d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:53.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.101+0000 7f07f4ff9640 1 --2- 172.21.15.67:0/3824171205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07e40b3d70 0x7f07e40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.101+0000 7f07ed7fa640 1 -- 172.21.15.67:0/2470461632 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0004d20 con 0x5566ede89070 2024-07-31T15:11:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.101+0000 7f07f6ffd640 1 --2- 172.21.15.67:0/3824171205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07e40b3d70 0x7f07e40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.101+0000 7f07ed7fa640 1 -- 172.21.15.67:0/2470461632 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f0004050 con 0x5566ede89070 2024-07-31T15:11:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.101+0000 7f07ed7fa640 1 -- 172.21.15.67:0/2470461632 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0005280 con 0x5566ede89070 2024-07-31T15:11:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.102+0000 7f07f4ff9640 1 -- 172.21.15.67:0/3824171205 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7f07e80e4740 con 0x7f07f8102490 2024-07-31T15:11:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.102+0000 7f08038fa740 1 -- 172.21.15.67:0/2470461632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede89070 msgr2=0x5566ee22e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.102+0000 7f07f6ffd640 1 --2- 172.21.15.67:0/3824171205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07e40b3d70 0x7f07e40b6280 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f07e80e1bd0 tx=0x7f07e80ea550 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.102+0000 7f08038fa740 1 --2- 172.21.15.67:0/2470461632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede89070 0x5566ee22e570 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009fd0 tx=0x7f07f00118d0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.102+0000 7f08038fa740 1 -- 172.21.15.67:0/2470461632 shutdown_connections 2024-07-31T15:11:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.102+0000 7f08038fa740 1 --2- 172.21.15.67:0/2470461632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 0x5566ee22eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.102+0000 7f08038fa740 1 --2- 172.21.15.67:0/2470461632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede601b0 0x5566ee22eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.102+0000 7f08038fa740 1 --2- 172.21.15.67:0/2470461632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede89070 0x5566ee22e570 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.102+0000 7f08038fa740 1 -- 172.21.15.67:0/2470461632 >> 172.21.15.67:0/2470461632 conn(0x5566ede7d460 msgr2=0x5566ee22c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.102+0000 7f08038fa740 1 -- 172.21.15.67:0/2470461632 shutdown_connections 2024-07-31T15:11:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.102+0000 7f08038fa740 1 -- 172.21.15.67:0/2470461632 wait complete. 2024-07-31T15:11:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.103+0000 7f08038fa740 1 -- start start 2024-07-31T15:11:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.103+0000 7f08038fa740 1 Processor -- start 2024-07-31T15:11:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.103+0000 7f08038fa740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede601b0 0x5566ee23a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.103+0000 7f08038fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 0x5566ee23a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.103+0000 7f08038fa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 0x5566ee246040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.103+0000 7f08038fa740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5566edff7700 con 0x5566ede601b0 2024-07-31T15:11:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.103+0000 7f08038fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5566ee1369e0 con 0x5566ede68390 2024-07-31T15:11:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.103+0000 7f08038fa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5566ee23cd50 con 0x5566ede89070 2024-07-31T15:11:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.103+0000 7f08038fa740 10 client.0 _ll_register_callbacks cb 0x5566ee1514f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.103+0000 7f07f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede601b0 0x5566ee23a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.104+0000 7f07f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede601b0 0x5566ee23a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37224/0 (socket says 172.21.15.67:37224) 2024-07-31T15:11:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.104+0000 7f07f77fe640 1 -- 172.21.15.67:0/2375479985 learned_addr learned my addr 172.21.15.67:0/2375479985 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.104+0000 7f07f77fe640 1 --2- 172.21.15.67:0/2375479985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 0x5566ee246040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.104+0000 7f07ee7fc640 1 --2- 172.21.15.67:0/2375479985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 0x5566ee23a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.104+0000 7f07ee7fc640 1 -- 172.21.15.67:0/2375479985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 msgr2=0x5566ee246040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:53.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.104+0000 7f07ee7fc640 1 --2- 172.21.15.67:0/2375479985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 0x5566ee246040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.104+0000 7f07ee7fc640 1 -- 172.21.15.67:0/2375479985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede601b0 msgr2=0x5566ee23a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:53.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.104+0000 7f07ee7fc640 1 --2- 172.21.15.67:0/2375479985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede601b0 0x5566ee23a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.104+0000 7f07ee7fc640 1 -- 172.21.15.67:0/2375479985 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566edef4650 con 0x5566ede68390 2024-07-31T15:11:53.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.104+0000 7f07ee7fc640 1 --2- 172.21.15.67:0/2375479985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 0x5566ee23a680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f07e0006010 tx=0x7f07e0032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffa82685f2558d1c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:53.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.104+0000 7f07de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:11:53.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.104+0000 7f07de7fc640 1 -- 172.21.15.67:0/2375479985 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0035040 con 0x5566ede68390 2024-07-31T15:11:53.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.104+0000 7f07de7fc640 1 -- 172.21.15.67:0/2375479985 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7f07e0031760 con 0x5566ede68390 2024-07-31T15:11:53.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.104+0000 7f08038fa740 1 -- 172.21.15.67:0/2375479985 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07f0004050 con 0x5566ede68390 2024-07-31T15:11:53.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.105+0000 7f07de7fc640 1 -- 172.21.15.67:0/2375479985 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f07e0004040 con 0x5566ede68390 2024-07-31T15:11:53.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.105+0000 7f08038fa740 10 client.29363 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:53.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.105+0000 7f08038fa740 1 -- 172.21.15.67:0/2375479985 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5566ee248d70 con 0x5566ede68390 2024-07-31T15:11:53.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.105+0000 7f07de7fc640 1 -- 172.21.15.67:0/2375479985 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f0004050 con 0x5566ede68390 2024-07-31T15:11:53.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.105+0000 7f07de7fc640 1 -- 172.21.15.67:0/2375479985 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f07e0004750 con 0x5566ede68390 2024-07-31T15:11:53.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.105+0000 7f08038fa740 10 client.29363 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:53.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.105+0000 7f08038fa740 10 client.29363 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:11:53.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.105+0000 7f08038fa740 20 client.29363 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:53.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.105+0000 7f08038fa740 1 --2- 172.21.15.67:0/2375479985 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5566ede479f0 0x5566ee24c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.105+0000 7f08038fa740 4 client.29363 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:11:53.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.105+0000 7f08038fa740 1 -- 172.21.15.67:0/2375479985 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566ee1a6b00 con 0x5566ede479f0 2024-07-31T15:11:53.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.105+0000 7f07f77fe640 1 --2- 172.21.15.67:0/2375479985 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5566ede479f0 0x5566ee24c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.106+0000 7f07f77fe640 1 --2- 172.21.15.67:0/2375479985 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5566ede479f0 0x5566ee24c260 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a1ce0e2536f5299 server_cookie=202af519c7d78a2a in_seq=0 out_seq=0 2024-07-31T15:11:53.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.106+0000 7f07de7fc640 10 client.29363 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:11:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:53 smithi073 ceph-mon[36714]: pgmap v1064: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 923 KiB/s wr, 229 op/s 2024-07-31T15:11:53.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:53 smithi077 ceph-mon[37327]: pgmap v1064: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 923 KiB/s wr, 229 op/s 2024-07-31T15:11:53.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:53 smithi067 ceph-mon[32750]: pgmap v1064: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 923 KiB/s wr, 229 op/s 2024-07-31T15:11:53.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.658+0000 7f07de7fc640 1 -- 172.21.15.67:0/2375479985 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566ee1a6b00 con 0x5566ede479f0 2024-07-31T15:11:53.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.658+0000 7f07de7fc640 10 client.29363 handle_command_reply: tid=0 2024-07-31T15:11:53.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.658+0000 7f08038fa740 1 client.29363 shutdown 2024-07-31T15:11:53.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.659+0000 7f08038fa740 1 -- 172.21.15.67:0/2375479985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 msgr2=0x5566ee23a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:53.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.659+0000 7f08038fa740 1 --2- 172.21.15.67:0/2375479985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 0x5566ee23a680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f07e0006010 tx=0x7f07e0032e90 comp rx=0 tx=0).stop 2024-07-31T15:11:53.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.659+0000 7f08038fa740 1 -- 172.21.15.67:0/2375479985 shutdown_connections 2024-07-31T15:11:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.659+0000 7f08038fa740 1 --2- 172.21.15.67:0/2375479985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 0x5566ee246040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.659+0000 7f08038fa740 1 --2- 172.21.15.67:0/2375479985 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5566ede479f0 0x5566ee24c260 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.659+0000 7f08038fa740 1 --2- 172.21.15.67:0/2375479985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 0x5566ee23a680 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.659+0000 7f08038fa740 1 --2- 172.21.15.67:0/2375479985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede601b0 0x5566ee23a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.659+0000 7f07de7fc640 0 client.29363 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:11:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.659+0000 7f08038fa740 1 -- 172.21.15.67:0/2375479985 >> 172.21.15.67:0/2375479985 conn(0x5566ede7d460 msgr2=0x5566ee22ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.659+0000 7f08038fa740 1 -- 172.21.15.67:0/2375479985 shutdown_connections 2024-07-31T15:11:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.659+0000 7f08038fa740 1 -- 172.21.15.67:0/2375479985 wait complete. 2024-07-31T15:11:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.659+0000 7f08038fa740 20 client.29363 trim_cache size 0 max 16384 2024-07-31T15:11:53.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.673+0000 7f08038fa740 1 Processor -- start 2024-07-31T15:11:53.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.673+0000 7f08038fa740 1 -- start start 2024-07-31T15:11:53.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.673+0000 7f08038fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede601b0 0x5566ee22b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.673+0000 7f08038fa740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede68390 0x5566ee245e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.673+0000 7f08038fa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 0x5566ee233680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.673+0000 7f08038fa740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f07f81b29d0 con 0x5566ede68390 2024-07-31T15:11:53.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.673+0000 7f08038fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f07f81b4510 con 0x5566ede601b0 2024-07-31T15:11:53.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.673+0000 7f08038fa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07f81b30a0 con 0x5566ede89070 2024-07-31T15:11:53.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.673+0000 7f07ee7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede68390 0x5566ee245e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.673+0000 7f07ee7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede68390 0x5566ee245e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37234/0 (socket says 172.21.15.67:37234) 2024-07-31T15:11:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.673+0000 7f07ee7fc640 1 -- 172.21.15.67:0/2982715467 learned_addr learned my addr 172.21.15.67:0/2982715467 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.673+0000 7f07f77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede601b0 0x5566ee22b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.673+0000 7f07f77fe640 1 --2- 172.21.15.67:0/2982715467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 0x5566ee233680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.674+0000 7f07ee7fc640 1 -- 172.21.15.67:0/2982715467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 msgr2=0x5566ee233680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.674+0000 7f07ee7fc640 1 --2- 172.21.15.67:0/2982715467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 0x5566ee233680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.674+0000 7f07ee7fc640 1 -- 172.21.15.67:0/2982715467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede601b0 msgr2=0x5566ee22b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.674+0000 7f07ee7fc640 1 --2- 172.21.15.67:0/2982715467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede601b0 0x5566ee22b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.674+0000 7f07ee7fc640 1 -- 172.21.15.67:0/2982715467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ee230860 con 0x5566ede68390 2024-07-31T15:11:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.674+0000 7f07ee7fc640 1 --2- 172.21.15.67:0/2982715467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede68390 0x5566ee245e30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f07e0000c00 tx=0x7f07e0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5789169774316eae server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.674+0000 7f07ecff9640 1 -- 172.21.15.67:0/2982715467 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0035070 con 0x5566ede68390 2024-07-31T15:11:53.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.674+0000 7f07ecff9640 1 -- 172.21.15.67:0/2982715467 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07e00048c0 con 0x5566ede68390 2024-07-31T15:11:53.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.674+0000 7f07ecff9640 1 -- 172.21.15.67:0/2982715467 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0004d90 con 0x5566ede68390 2024-07-31T15:11:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.675+0000 7f08038fa740 1 -- 172.21.15.67:0/2982715467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede68390 msgr2=0x5566ee245e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.675+0000 7f08038fa740 1 --2- 172.21.15.67:0/2982715467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede68390 0x5566ee245e30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f07e0000c00 tx=0x7f07e0005850 comp rx=0 tx=0).stop 2024-07-31T15:11:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.675+0000 7f08038fa740 1 -- 172.21.15.67:0/2982715467 shutdown_connections 2024-07-31T15:11:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.675+0000 7f08038fa740 1 --2- 172.21.15.67:0/2982715467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 0x5566ee233680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.675+0000 7f08038fa740 1 --2- 172.21.15.67:0/2982715467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede68390 0x5566ee245e30 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.675+0000 7f08038fa740 1 --2- 172.21.15.67:0/2982715467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede601b0 0x5566ee22b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.675+0000 7f08038fa740 1 -- 172.21.15.67:0/2982715467 >> 172.21.15.67:0/2982715467 conn(0x5566ede7d460 msgr2=0x5566ee22ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.675+0000 7f08038fa740 1 -- 172.21.15.67:0/2982715467 shutdown_connections 2024-07-31T15:11:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.675+0000 7f08038fa740 1 -- 172.21.15.67:0/2982715467 wait complete. 2024-07-31T15:11:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.675+0000 7f08038fa740 1 -- start start 2024-07-31T15:11:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f08038fa740 1 Processor -- start 2024-07-31T15:11:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f08038fa740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede601b0 0x5566ee23dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f08038fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 0x5566ee23e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f08038fa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 0x5566ee23e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f08038fa740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f07f81b3e40 con 0x5566ede601b0 2024-07-31T15:11:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f08038fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f07f81b3770 con 0x5566ede68390 2024-07-31T15:11:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f08038fa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07f81b09d0 con 0x5566ede89070 2024-07-31T15:11:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f08038fa740 10 client.0 _ll_register_callbacks cb 0x7f07e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f07f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede601b0 0x5566ee23dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f07f77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede601b0 0x5566ee23dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37238/0 (socket says 172.21.15.67:37238) 2024-07-31T15:11:53.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f07f77fe640 1 -- 172.21.15.67:0/2174253785 learned_addr learned my addr 172.21.15.67:0/2174253785 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:53.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f07ee7fc640 1 --2- 172.21.15.67:0/2174253785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 0x5566ee23e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f07f77fe640 1 --2- 172.21.15.67:0/2174253785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 0x5566ee23e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f07f77fe640 1 -- 172.21.15.67:0/2174253785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 msgr2=0x5566ee23e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:53.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f07f77fe640 1 --2- 172.21.15.67:0/2174253785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 0x5566ee23e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f07f77fe640 1 -- 172.21.15.67:0/2174253785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 msgr2=0x5566ee23e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:53.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f07f77fe640 1 --2- 172.21.15.67:0/2174253785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 0x5566ee23e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f07f77fe640 1 -- 172.21.15.67:0/2174253785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ee2b3a40 con 0x5566ede601b0 2024-07-31T15:11:53.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.676+0000 7f07ee7fc640 1 --2- 172.21.15.67:0/2174253785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 0x5566ee23e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:53.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f07f77fe640 1 --2- 172.21.15.67:0/2174253785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede601b0 0x5566ee23dda0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f07f0013010 tx=0x7f07f0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eada437f93ae3c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:53.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f07de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:11:53.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f07de7fc640 1 -- 172.21.15.67:0/2174253785 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0004210 con 0x5566ede601b0 2024-07-31T15:11:53.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f07de7fc640 1 -- 172.21.15.67:0/2174253785 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7f07f00043b0 con 0x5566ede601b0 2024-07-31T15:11:53.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f08038fa740 1 -- 172.21.15.67:0/2174253785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07e00048c0 con 0x5566ede601b0 2024-07-31T15:11:53.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f07de7fc640 1 -- 172.21.15.67:0/2174253785 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f07f00045b0 con 0x5566ede601b0 2024-07-31T15:11:53.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f08038fa740 10 client.19842 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:53.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f08038fa740 1 -- 172.21.15.67:0/2174253785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5566ee2b4bf0 con 0x5566ede601b0 2024-07-31T15:11:53.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f07de7fc640 1 -- 172.21.15.67:0/2174253785 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07e00048c0 con 0x5566ede601b0 2024-07-31T15:11:53.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f07de7fc640 1 -- 172.21.15.67:0/2174253785 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f07f0028450 con 0x5566ede601b0 2024-07-31T15:11:53.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f08038fa740 10 client.19842 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:53.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f08038fa740 10 client.19842 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:11:53.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f08038fa740 20 client.19842 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:53.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f08038fa740 1 --2- 172.21.15.67:0/2174253785 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5566ede1cda0 0x5566ee2b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:53.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f08038fa740 4 client.19842 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:11:53.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.677+0000 7f08038fa740 1 -- 172.21.15.67:0/2174253785 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566ee1a25a0 con 0x5566ede1cda0 2024-07-31T15:11:53.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.678+0000 7f07ee7fc640 1 --2- 172.21.15.67:0/2174253785 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5566ede1cda0 0x5566ee2b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:53.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.678+0000 7f07de7fc640 10 client.19842 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:11:53.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.678+0000 7f07ee7fc640 1 --2- 172.21.15.67:0/2174253785 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5566ede1cda0 0x5566ee2b71d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f3dcfc1a92fbc5f server_cookie=5f5772224eb00dde in_seq=0 out_seq=0 2024-07-31T15:11:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.719+0000 7f07de7fc640 1 -- 172.21.15.67:0/2174253785 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566ee1a25a0 con 0x5566ede1cda0 2024-07-31T15:11:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.719+0000 7f07de7fc640 10 client.19842 handle_command_reply: tid=0 2024-07-31T15:11:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.719+0000 7f08038fa740 1 client.19842 shutdown 2024-07-31T15:11:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.719+0000 7f08038fa740 1 -- 172.21.15.67:0/2174253785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede601b0 msgr2=0x5566ee23dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.719+0000 7f08038fa740 1 --2- 172.21.15.67:0/2174253785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede601b0 0x5566ee23dda0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f07f0013010 tx=0x7f07f0013040 comp rx=0 tx=0).stop 2024-07-31T15:11:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.719+0000 7f08038fa740 1 -- 172.21.15.67:0/2174253785 shutdown_connections 2024-07-31T15:11:53.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.719+0000 7f08038fa740 1 --2- 172.21.15.67:0/2174253785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566ede89070 0x5566ee23e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.719+0000 7f08038fa740 1 --2- 172.21.15.67:0/2174253785 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5566ede1cda0 0x5566ee2b71d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.719+0000 7f08038fa740 1 --2- 172.21.15.67:0/2174253785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566ede68390 0x5566ee23e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.719+0000 7f08038fa740 1 --2- 172.21.15.67:0/2174253785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5566ede601b0 0x5566ee23dda0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.719+0000 7f07de7fc640 0 client.19842 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:11:53.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.719+0000 7f08038fa740 1 -- 172.21.15.67:0/2174253785 >> 172.21.15.67:0/2174253785 conn(0x5566ede7d460 msgr2=0x5566ee233680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:53.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.719+0000 7f08038fa740 1 -- 172.21.15.67:0/2174253785 shutdown_connections 2024-07-31T15:11:53.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.719+0000 7f08038fa740 1 -- 172.21.15.67:0/2174253785 wait complete. 2024-07-31T15:11:53.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.720+0000 7f08038fa740 20 client.19842 trim_cache size 0 max 16384 2024-07-31T15:11:53.723 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:11:53.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.720+0000 7f07fd3a9640 1 -- 172.21.15.67:0/3824171205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07e40b3d70 msgr2=0x7f07e40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:53.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.720+0000 7f07fd3a9640 1 --2- 172.21.15.67:0/3824171205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07e40b3d70 0x7f07e40b6280 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f07e80e1bd0 tx=0x7f07e80ea550 comp rx=0 tx=0).stop 2024-07-31T15:11:53.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.720+0000 7f07fd3a9640 1 -- 172.21.15.67:0/3824171205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f8102490 msgr2=0x7f07f819ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:53.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.720+0000 7f07fd3a9640 1 --2- 172.21.15.67:0/3824171205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f8102490 0x7f07f819ad00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f07e800e9f0 tx=0x7f07e800edf0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.720+0000 7f07fd3a9640 1 -- 172.21.15.67:0/3824171205 shutdown_connections 2024-07-31T15:11:53.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.720+0000 7f07fd3a9640 1 --2- 172.21.15.67:0/3824171205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07e40b3d70 0x7f07e40b6280 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.720+0000 7f07fd3a9640 1 --2- 172.21.15.67:0/3824171205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f8108720 0x7f07f819b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.720+0000 7f07fd3a9640 1 --2- 172.21.15.67:0/3824171205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07f8107dd0 0x7f07f819b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.720+0000 7f07fd3a9640 1 --2- 172.21.15.67:0/3824171205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f8102490 0x7f07f819ad00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:53.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.720+0000 7f07fd3a9640 1 -- 172.21.15.67:0/3824171205 >> 172.21.15.67:0/3824171205 conn(0x7f07f8079160 msgr2=0x7f07f8103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:53.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.720+0000 7f07fd3a9640 1 -- 172.21.15.67:0/3824171205 shutdown_connections 2024-07-31T15:11:53.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:53.720+0000 7f07fd3a9640 1 -- 172.21.15.67:0/3824171205 wait complete. 2024-07-31T15:11:54.582 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T15:11:54.905 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:11:55.289 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:55 smithi067 ceph-mon[32750]: pgmap v1065: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 898 KiB/s wr, 227 op/s 2024-07-31T15:11:55.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:55 smithi073 ceph-mon[36714]: pgmap v1065: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 898 KiB/s wr, 227 op/s 2024-07-31T15:11:55.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:55 smithi077 ceph-mon[37327]: pgmap v1065: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 898 KiB/s wr, 227 op/s 2024-07-31T15:11:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.580+0000 7fae05014640 1 -- 172.21.15.67:0/3087126555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae001024d0 msgr2=0x7fae001028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.580+0000 7fae05014640 1 --2- 172.21.15.67:0/3087126555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae001024d0 0x7fae001028e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fade8009900 tx=0x7fade802a410 comp rx=0 tx=0).stop 2024-07-31T15:11:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.580+0000 7fae05014640 1 -- 172.21.15.67:0/3087126555 shutdown_connections 2024-07-31T15:11:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.580+0000 7fae05014640 1 --2- 172.21.15.67:0/3087126555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae00108760 0x7fae0010ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.580+0000 7fae05014640 1 --2- 172.21.15.67:0/3087126555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae00107e10 0x7fae00108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.580+0000 7fae05014640 1 --2- 172.21.15.67:0/3087126555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae001024d0 0x7fae001028e0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.580+0000 7fae05014640 1 -- 172.21.15.67:0/3087126555 >> 172.21.15.67:0/3087126555 conn(0x7fae000791e0 msgr2=0x7fae000795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:56.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.580+0000 7fae05014640 1 -- 172.21.15.67:0/3087126555 shutdown_connections 2024-07-31T15:11:56.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.580+0000 7fae05014640 1 -- 172.21.15.67:0/3087126555 wait complete. 2024-07-31T15:11:56.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.581+0000 7fae05014640 1 Processor -- start 2024-07-31T15:11:56.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.581+0000 7fae05014640 1 -- start start 2024-07-31T15:11:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.581+0000 7fae05014640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae001024d0 0x7fae00198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.581+0000 7fae05014640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae00107e10 0x7fae00199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.581+0000 7fae05014640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae00108760 0x7fae0019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.581+0000 7fae05014640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fae00112410 con 0x7fae001024d0 2024-07-31T15:11:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.581+0000 7fae05014640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fae00112290 con 0x7fae00107e10 2024-07-31T15:11:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.581+0000 7fae05014640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae00112590 con 0x7fae00108760 2024-07-31T15:11:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.582+0000 7fadfed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae001024d0 0x7fae00198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.582+0000 7fadfed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae001024d0 0x7fae00198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37262/0 (socket says 172.21.15.67:37262) 2024-07-31T15:11:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.582+0000 7fadfed9d640 1 -- 172.21.15.67:0/2106256267 learned_addr learned my addr 172.21.15.67:0/2106256267 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.582+0000 7fadfe59c640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae00107e10 0x7fae00199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.582+0000 7fadff59e640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae00108760 0x7fae0019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.582+0000 7fadfe59c640 1 -- 172.21.15.67:0/2106256267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae00108760 msgr2=0x7fae0019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.582+0000 7fadfe59c640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae00108760 0x7fae0019e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.582+0000 7fadfe59c640 1 -- 172.21.15.67:0/2106256267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae001024d0 msgr2=0x7fae00198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:56.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.582+0000 7fadfe59c640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae001024d0 0x7fae00198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.582+0000 7fadfe59c640 1 -- 172.21.15.67:0/2106256267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0019ee50 con 0x7fae00107e10 2024-07-31T15:11:56.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.582+0000 7fadfed9d640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae001024d0 0x7fae00198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:56.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.582+0000 7fadfe59c640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae00107e10 0x7fae00199480 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fae0006a760 tx=0x7fadf000cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:56.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.582+0000 7fadf7fff640 1 -- 172.21.15.67:0/2106256267 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0007bb0 con 0x7fae00107e10 2024-07-31T15:11:56.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.582+0000 7fadf7fff640 1 -- 172.21.15.67:0/2106256267 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadf0007d50 con 0x7fae00107e10 2024-07-31T15:11:56.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.583+0000 7fae05014640 1 -- 172.21.15.67:0/2106256267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0019f140 con 0x7fae00107e10 2024-07-31T15:11:56.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.583+0000 7fadf7fff640 1 -- 172.21.15.67:0/2106256267 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0004de0 con 0x7fae00107e10 2024-07-31T15:11:56.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.583+0000 7fae05014640 1 -- 172.21.15.67:0/2106256267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae001a58a0 con 0x7fae00107e10 2024-07-31T15:11:56.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.585+0000 7fae0b565740 1 Processor -- start 2024-07-31T15:11:56.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.585+0000 7fadf7fff640 1 -- 172.21.15.67:0/2106256267 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fadf0004fc0 con 0x7fae00107e10 2024-07-31T15:11:56.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.585+0000 7fae0b565740 1 -- start start 2024-07-31T15:11:56.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.585+0000 7fae0b565740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a86c070 0x55eb0ac11570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.585+0000 7fae0b565740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac11ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.585+0000 7fae0b565740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a84b390 0x55eb0ac11ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.585+0000 7fae0b565740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55eb0ab87bb0 con 0x55eb0a8431b0 2024-07-31T15:11:56.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fae0b565740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55eb0ab770b0 con 0x55eb0a86c070 2024-07-31T15:11:56.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fae0b565740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55eb0ab7b0f0 con 0x55eb0a84b390 2024-07-31T15:11:56.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadfed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac11ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadff59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a86c070 0x55eb0ac11570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadfed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac11ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37276/0 (socket says 172.21.15.67:37276) 2024-07-31T15:11:56.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadff59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a86c070 0x55eb0ac11570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:53692/0 (socket says 172.21.15.67:53692) 2024-07-31T15:11:56.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadfed9d640 1 -- 172.21.15.67:0/875197027 learned_addr learned my addr 172.21.15.67:0/875197027 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:56.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadfe59c640 1 --2- 172.21.15.67:0/875197027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a84b390 0x55eb0ac11ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadff59e640 1 -- 172.21.15.67:0/875197027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a84b390 msgr2=0x55eb0ac11ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:56.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadff59e640 1 --2- 172.21.15.67:0/875197027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a84b390 0x55eb0ac11ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadff59e640 1 -- 172.21.15.67:0/875197027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 msgr2=0x55eb0ac11ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:56.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadff59e640 1 --2- 172.21.15.67:0/875197027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac11ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadff59e640 1 -- 172.21.15.67:0/875197027 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb0ab1db90 con 0x55eb0a86c070 2024-07-31T15:11:56.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadfed9d640 1 --2- 172.21.15.67:0/875197027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac11ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:56.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadf7fff640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fadec0b3d40 0x7fadec0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadff59e640 1 --2- 172.21.15.67:0/875197027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a86c070 0x55eb0ac11570 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009e50 tx=0x7fadf80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7a3edf69854937 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:56.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadfed9d640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fadec0b3d40 0x7fadec0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.586+0000 7fadf7fff640 1 -- 172.21.15.67:0/2106256267 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fadf00e4070 con 0x7fae00107e10 2024-07-31T15:11:56.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.587+0000 7fadf57fa640 1 -- 172.21.15.67:0/875197027 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8003b70 con 0x55eb0a86c070 2024-07-31T15:11:56.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.587+0000 7fadf57fa640 1 -- 172.21.15.67:0/875197027 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadf8015070 con 0x55eb0a86c070 2024-07-31T15:11:56.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.587+0000 7fadf57fa640 1 -- 172.21.15.67:0/875197027 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8011740 con 0x55eb0a86c070 2024-07-31T15:11:56.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.587+0000 7fae0b565740 1 -- 172.21.15.67:0/875197027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a86c070 msgr2=0x55eb0ac11570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:56.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.587+0000 7fadfed9d640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fadec0b3d40 0x7fadec0b6250 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x55eb0ac101f0 tx=0x7fade80058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:56.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.587+0000 7fae0b565740 1 --2- 172.21.15.67:0/875197027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a86c070 0x55eb0ac11570 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009e50 tx=0x7fadf80107a0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.587+0000 7fae0b565740 1 -- 172.21.15.67:0/875197027 shutdown_connections 2024-07-31T15:11:56.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.587+0000 7fae0b565740 1 --2- 172.21.15.67:0/875197027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a84b390 0x55eb0ac11ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.587+0000 7fae0b565740 1 --2- 172.21.15.67:0/875197027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac11ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.587+0000 7fae0b565740 1 --2- 172.21.15.67:0/875197027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a86c070 0x55eb0ac11570 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.587+0000 7fae0b565740 1 -- 172.21.15.67:0/875197027 >> 172.21.15.67:0/875197027 conn(0x55eb0a860460 msgr2=0x55eb0ac0f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:56.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.588+0000 7fae0b565740 1 -- 172.21.15.67:0/875197027 shutdown_connections 2024-07-31T15:11:56.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.588+0000 7fae0b565740 1 -- 172.21.15.67:0/875197027 wait complete. 2024-07-31T15:11:56.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.588+0000 7fae0b565740 1 -- start start 2024-07-31T15:11:56.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.588+0000 7fae0b565740 1 Processor -- start 2024-07-31T15:11:56.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.588+0000 7fae0b565740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac1d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.588+0000 7fae0b565740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a84b390 0x55eb0ac1d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.588+0000 7fae0b565740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a86c070 0x55eb0ac290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.588+0000 7fae0b565740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55eb0a9da700 con 0x55eb0a8431b0 2024-07-31T15:11:56.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.588+0000 7fae0b565740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55eb0ab199e0 con 0x55eb0a84b390 2024-07-31T15:11:56.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.588+0000 7fae0b565740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55eb0ac1fd50 con 0x55eb0a86c070 2024-07-31T15:11:56.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.588+0000 7fae0b565740 10 client.0 _ll_register_callbacks cb 0x55eb0ab344f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:56.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fadfe59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a84b390 0x55eb0ac1d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fadff59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a86c070 0x55eb0ac290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fadfed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac1d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fadff59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a86c070 0x55eb0ac290a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:58770/0 (socket says 172.21.15.67:58770) 2024-07-31T15:11:56.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fadff59e640 1 -- 172.21.15.67:0/79109565 learned_addr learned my addr 172.21.15.67:0/79109565 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:56.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fadff59e640 1 -- 172.21.15.67:0/79109565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a84b390 msgr2=0x55eb0ac1d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:56.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fadff59e640 1 --2- 172.21.15.67:0/79109565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a84b390 0x55eb0ac1d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fadff59e640 1 -- 172.21.15.67:0/79109565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 msgr2=0x55eb0ac1d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:56.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fadff59e640 1 --2- 172.21.15.67:0/79109565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac1d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fadff59e640 1 -- 172.21.15.67:0/79109565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb0a8d7650 con 0x55eb0a86c070 2024-07-31T15:11:56.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fadfe59c640 1 --2- 172.21.15.67:0/79109565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a84b390 0x55eb0ac1d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:56.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fadff59e640 1 --2- 172.21.15.67:0/79109565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a86c070 0x55eb0ac290a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fadf801e6c0 tx=0x7fadf801e6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a27d705f28c9637 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:56.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fade67fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:11:56.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fade67fc640 1 -- 172.21.15.67:0/79109565 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf802c060 con 0x55eb0a86c070 2024-07-31T15:11:56.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fade67fc640 1 -- 172.21.15.67:0/79109565 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fadf801e910 con 0x55eb0a86c070 2024-07-31T15:11:56.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.589+0000 7fae0b565740 1 -- 172.21.15.67:0/79109565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadf8015070 con 0x55eb0a86c070 2024-07-31T15:11:56.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.590+0000 7fade67fc640 1 -- 172.21.15.67:0/79109565 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fadf8033050 con 0x55eb0a86c070 2024-07-31T15:11:56.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.590+0000 7fae0b565740 10 client.29242 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:56.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.590+0000 7fae0b565740 1 -- 172.21.15.67:0/79109565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55eb0ac2bdf0 con 0x55eb0a86c070 2024-07-31T15:11:56.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.590+0000 7fade67fc640 1 -- 172.21.15.67:0/79109565 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadf8015070 con 0x55eb0a86c070 2024-07-31T15:11:56.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.590+0000 7fade67fc640 1 -- 172.21.15.67:0/79109565 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fadf801c840 con 0x55eb0a86c070 2024-07-31T15:11:56.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.590+0000 7fae0b565740 10 client.29242 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:56.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.590+0000 7fae0b565740 10 client.29242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:11:56.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.590+0000 7fae0b565740 20 client.29242 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:56.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.590+0000 7fae0b565740 1 --2- 172.21.15.67:0/79109565 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55eb0a82a9f0 0x55eb0ac2f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.590+0000 7fae0b565740 4 client.29242 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:11:56.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.590+0000 7fae0b565740 1 -- 172.21.15.67:0/79109565 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb0ab89b00 con 0x55eb0a82a9f0 2024-07-31T15:11:56.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.591+0000 7fadfed9d640 1 --2- 172.21.15.67:0/79109565 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55eb0a82a9f0 0x55eb0ac2f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.591+0000 7fade67fc640 10 client.29242 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:11:56.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.591+0000 7fadfed9d640 1 --2- 172.21.15.67:0/79109565 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55eb0a82a9f0 0x55eb0ac2f200 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b7340ce697a8773d server_cookie=95bc980f4a56481e in_seq=0 out_seq=0 2024-07-31T15:11:56.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fade67fc640 1 -- 172.21.15.67:0/79109565 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb0ab89b00 con 0x55eb0a82a9f0 2024-07-31T15:11:56.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fade67fc640 10 client.29242 handle_command_reply: tid=0 2024-07-31T15:11:56.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fae0b565740 1 client.29242 shutdown 2024-07-31T15:11:56.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fae0b565740 1 -- 172.21.15.67:0/79109565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a86c070 msgr2=0x55eb0ac290a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:56.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fae0b565740 1 --2- 172.21.15.67:0/79109565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a86c070 0x55eb0ac290a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fadf801e6c0 tx=0x7fadf801e6f0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fae0b565740 1 -- 172.21.15.67:0/79109565 shutdown_connections 2024-07-31T15:11:56.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fae0b565740 1 --2- 172.21.15.67:0/79109565 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55eb0a82a9f0 0x55eb0ac2f200 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fae0b565740 1 --2- 172.21.15.67:0/79109565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a86c070 0x55eb0ac290a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fade67fc640 0 client.29242 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:11:56.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fae0b565740 1 --2- 172.21.15.67:0/79109565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a84b390 0x55eb0ac1d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fae0b565740 1 --2- 172.21.15.67:0/79109565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac1d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fae0b565740 1 -- 172.21.15.67:0/79109565 >> 172.21.15.67:0/79109565 conn(0x55eb0a860460 msgr2=0x55eb0ac0dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:56.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fae0b565740 1 -- 172.21.15.67:0/79109565 shutdown_connections 2024-07-31T15:11:56.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fae0b565740 1 -- 172.21.15.67:0/79109565 wait complete. 2024-07-31T15:11:56.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.592+0000 7fae0b565740 20 client.29242 trim_cache size 0 max 16384 2024-07-31T15:11:56.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fae0b565740 1 Processor -- start 2024-07-31T15:11:56.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fae0b565740 1 -- start start 2024-07-31T15:11:56.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fae0b565740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a8431b0 0x55eb0ac0e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fae0b565740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a84b390 0x55eb0ac29010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fae0b565740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a86c070 0x55eb0ac0ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fae0b565740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fae001af6d0 con 0x55eb0a84b390 2024-07-31T15:11:56.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fae0b565740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fae001b1210 con 0x55eb0a86c070 2024-07-31T15:11:56.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fae0b565740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae001afda0 con 0x55eb0a8431b0 2024-07-31T15:11:56.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fadfe59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a84b390 0x55eb0ac29010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fadfe59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a84b390 0x55eb0ac29010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37288/0 (socket says 172.21.15.67:37288) 2024-07-31T15:11:56.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fadfe59c640 1 -- 172.21.15.67:0/1277599457 learned_addr learned my addr 172.21.15.67:0/1277599457 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:56.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fadff59e640 1 --2- 172.21.15.67:0/1277599457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a86c070 0x55eb0ac0ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fadfed9d640 1 --2- 172.21.15.67:0/1277599457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a8431b0 0x55eb0ac0e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fadfe59c640 1 -- 172.21.15.67:0/1277599457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a8431b0 msgr2=0x55eb0ac0e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:56.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.606+0000 7fadfe59c640 1 --2- 172.21.15.67:0/1277599457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a8431b0 0x55eb0ac0e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.607+0000 7fadfe59c640 1 -- 172.21.15.67:0/1277599457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a86c070 msgr2=0x55eb0ac0ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:56.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.607+0000 7fadfe59c640 1 --2- 172.21.15.67:0/1277599457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a86c070 0x55eb0ac0ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.607+0000 7fadfe59c640 1 -- 172.21.15.67:0/1277599457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb0ac143d0 con 0x55eb0a84b390 2024-07-31T15:11:56.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.607+0000 7fadfed9d640 1 --2- 172.21.15.67:0/1277599457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a8431b0 0x55eb0ac0e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:11:56.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.607+0000 7fadff59e640 1 --2- 172.21.15.67:0/1277599457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a86c070 0x55eb0ac0ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:56.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.607+0000 7fadfe59c640 1 --2- 172.21.15.67:0/1277599457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a84b390 0x55eb0ac29010 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fadf00e1a50 tx=0x7fadf0092040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f00f7b4d50b978d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:56.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.607+0000 7fadf4ff9640 1 -- 172.21.15.67:0/1277599457 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf001d070 con 0x55eb0a84b390 2024-07-31T15:11:56.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.607+0000 7fadf4ff9640 1 -- 172.21.15.67:0/1277599457 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadf00b3310 con 0x55eb0a84b390 2024-07-31T15:11:56.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.607+0000 7fadf4ff9640 1 -- 172.21.15.67:0/1277599457 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf00e1e00 con 0x55eb0a84b390 2024-07-31T15:11:56.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.608+0000 7fae0b565740 1 -- 172.21.15.67:0/1277599457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a84b390 msgr2=0x55eb0ac29010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:56.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.608+0000 7fae0b565740 1 --2- 172.21.15.67:0/1277599457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a84b390 0x55eb0ac29010 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fadf00e1a50 tx=0x7fadf0092040 comp rx=0 tx=0).stop 2024-07-31T15:11:56.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.608+0000 7fae0b565740 1 -- 172.21.15.67:0/1277599457 shutdown_connections 2024-07-31T15:11:56.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.608+0000 7fae0b565740 1 --2- 172.21.15.67:0/1277599457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a86c070 0x55eb0ac0ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.608+0000 7fae0b565740 1 --2- 172.21.15.67:0/1277599457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a84b390 0x55eb0ac29010 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.608+0000 7fae0b565740 1 --2- 172.21.15.67:0/1277599457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a8431b0 0x55eb0ac0e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.608+0000 7fae0b565740 1 -- 172.21.15.67:0/1277599457 >> 172.21.15.67:0/1277599457 conn(0x55eb0a860460 msgr2=0x55eb0ac0dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:56.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.608+0000 7fae0b565740 1 -- 172.21.15.67:0/1277599457 shutdown_connections 2024-07-31T15:11:56.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.608+0000 7fae0b565740 1 -- 172.21.15.67:0/1277599457 wait complete. 2024-07-31T15:11:56.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.608+0000 7fae0b565740 1 -- start start 2024-07-31T15:11:56.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.608+0000 7fae0b565740 1 Processor -- start 2024-07-31T15:11:56.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fae0b565740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac329f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fae0b565740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a84b390 0x55eb0ac2fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fae0b565740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a86c070 0x55eb0ac30510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fae0b565740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fae001b0b40 con 0x55eb0a8431b0 2024-07-31T15:11:56.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fae0b565740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fae001b0470 con 0x55eb0a84b390 2024-07-31T15:11:56.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fae0b565740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae001ad6d0 con 0x55eb0a86c070 2024-07-31T15:11:56.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fae0b565740 10 client.0 _ll_register_callbacks cb 0x7fade8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:56.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fadfed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fadfed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac329f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37300/0 (socket says 172.21.15.67:37300) 2024-07-31T15:11:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fadfed9d640 1 -- 172.21.15.67:0/3160615425 learned_addr learned my addr 172.21.15.67:0/3160615425 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fadfe59c640 1 --2- 172.21.15.67:0/3160615425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a84b390 0x55eb0ac2fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fadff59e640 1 --2- 172.21.15.67:0/3160615425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a86c070 0x55eb0ac30510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fadfe59c640 1 -- 172.21.15.67:0/3160615425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a86c070 msgr2=0x55eb0ac30510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fadfe59c640 1 --2- 172.21.15.67:0/3160615425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a86c070 0x55eb0ac30510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fadfe59c640 1 -- 172.21.15.67:0/3160615425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 msgr2=0x55eb0ac329f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fadfe59c640 1 --2- 172.21.15.67:0/3160615425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac329f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fadfe59c640 1 -- 172.21.15.67:0/3160615425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb0ac30c80 con 0x55eb0a84b390 2024-07-31T15:11:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fadfed9d640 1 --2- 172.21.15.67:0/3160615425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fadfe59c640 1 --2- 172.21.15.67:0/3160615425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a84b390 0x55eb0ac2fec0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55eb0ac1f520 tx=0x7fadf00e1530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=275783cc27be289f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.609+0000 7fade67fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:11:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.610+0000 7fade67fc640 1 -- 172.21.15.67:0/3160615425 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf001d070 con 0x55eb0a84b390 2024-07-31T15:11:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.610+0000 7fade67fc640 1 -- 172.21.15.67:0/3160615425 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fadf00e6340 con 0x55eb0a84b390 2024-07-31T15:11:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.610+0000 7fae0b565740 1 -- 172.21.15.67:0/3160615425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadf00b3310 con 0x55eb0a84b390 2024-07-31T15:11:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.610+0000 7fade67fc640 1 -- 172.21.15.67:0/3160615425 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fadf00ca050 con 0x55eb0a84b390 2024-07-31T15:11:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.610+0000 7fae0b565740 10 client.29387 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.610+0000 7fae0b565740 1 -- 172.21.15.67:0/3160615425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55eb0ac9bba0 con 0x55eb0a84b390 2024-07-31T15:11:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.610+0000 7fade67fc640 1 -- 172.21.15.67:0/3160615425 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadf00b3310 con 0x55eb0a84b390 2024-07-31T15:11:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.610+0000 7fade67fc640 1 -- 172.21.15.67:0/3160615425 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fadf00a9b70 con 0x55eb0a84b390 2024-07-31T15:11:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.610+0000 7fae0b565740 10 client.29387 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.610+0000 7fae0b565740 10 client.29387 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:11:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.610+0000 7fae0b565740 20 client.29387 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.610+0000 7fae0b565740 1 --2- 172.21.15.67:0/3160615425 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55eb0a7ffda0 0x55eb0ac9e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.610+0000 7fae0b565740 4 client.29387 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:11:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.610+0000 7fae0b565740 1 -- 172.21.15.67:0/3160615425 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb0ac1daa0 con 0x55eb0a7ffda0 2024-07-31T15:11:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fadfed9d640 1 --2- 172.21.15.67:0/3160615425 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55eb0a7ffda0 0x55eb0ac9e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fadfed9d640 1 --2- 172.21.15.67:0/3160615425 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55eb0a7ffda0 0x55eb0ac9e310 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f45fa96b977d182 server_cookie=69c2fbfb59503dee in_seq=0 out_seq=0 2024-07-31T15:11:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fade67fc640 10 client.29387 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:11:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fade67fc640 1 -- 172.21.15.67:0/3160615425 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb0ac1daa0 con 0x55eb0a7ffda0 2024-07-31T15:11:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fade67fc640 10 client.29387 handle_command_reply: tid=0 2024-07-31T15:11:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fae0b565740 1 client.29387 shutdown 2024-07-31T15:11:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fae0b565740 1 -- 172.21.15.67:0/3160615425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a84b390 msgr2=0x55eb0ac2fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fae0b565740 1 --2- 172.21.15.67:0/3160615425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a84b390 0x55eb0ac2fec0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55eb0ac1f520 tx=0x7fadf00e1530 comp rx=0 tx=0).stop 2024-07-31T15:11:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fae0b565740 1 -- 172.21.15.67:0/3160615425 shutdown_connections 2024-07-31T15:11:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fae0b565740 1 --2- 172.21.15.67:0/3160615425 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55eb0a7ffda0 0x55eb0ac9e310 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fae0b565740 1 --2- 172.21.15.67:0/3160615425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb0a86c070 0x55eb0ac30510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fae0b565740 1 --2- 172.21.15.67:0/3160615425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb0a84b390 0x55eb0ac2fec0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fade67fc640 0 client.29387 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:11:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fae0b565740 1 --2- 172.21.15.67:0/3160615425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb0a8431b0 0x55eb0ac329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fae0b565740 1 -- 172.21.15.67:0/3160615425 >> 172.21.15.67:0/3160615425 conn(0x55eb0a860460 msgr2=0x55eb0ac16680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.611+0000 7fae0b565740 1 -- 172.21.15.67:0/3160615425 shutdown_connections 2024-07-31T15:11:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.612+0000 7fae0b565740 1 -- 172.21.15.67:0/3160615425 wait complete. 2024-07-31T15:11:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.612+0000 7fae0b565740 20 client.29387 trim_cache size 0 max 16384 2024-07-31T15:11:56.623 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:11:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.612+0000 7fae05014640 1 -- 172.21.15.67:0/2106256267 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fadec0b3d40 msgr2=0x7fadec0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.612+0000 7fae05014640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fadec0b3d40 0x7fadec0b6250 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x55eb0ac101f0 tx=0x7fade80058d0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.612+0000 7fae05014640 1 -- 172.21.15.67:0/2106256267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae00107e10 msgr2=0x7fae00199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.612+0000 7fae05014640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae00107e10 0x7fae00199480 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fae0006a760 tx=0x7fadf000cdd0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.612+0000 7fae05014640 1 -- 172.21.15.67:0/2106256267 shutdown_connections 2024-07-31T15:11:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.612+0000 7fae05014640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fadec0b3d40 0x7fadec0b6250 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.612+0000 7fae05014640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae00108760 0x7fae0019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.612+0000 7fae05014640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae00107e10 0x7fae00199480 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.612+0000 7fae05014640 1 --2- 172.21.15.67:0/2106256267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae001024d0 0x7fae00198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.612+0000 7fae05014640 1 -- 172.21.15.67:0/2106256267 >> 172.21.15.67:0/2106256267 conn(0x7fae000791e0 msgr2=0x7fae00106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.612+0000 7fae05014640 1 -- 172.21.15.67:0/2106256267 shutdown_connections 2024-07-31T15:11:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:56.613+0000 7fae05014640 1 -- 172.21.15.67:0/2106256267 wait complete. 2024-07-31T15:11:57.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:57 smithi067 ceph-mon[32750]: pgmap v1066: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 478 KiB/s wr, 193 op/s 2024-07-31T15:11:57.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:57 smithi073 ceph-mon[36714]: pgmap v1066: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 478 KiB/s wr, 193 op/s 2024-07-31T15:11:57.422 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T15:11:57.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:57 smithi077 ceph-mon[37327]: pgmap v1066: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 478 KiB/s wr, 193 op/s 2024-07-31T15:11:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:11:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.739+0000 7f20a0292640 1 -- 172.21.15.67:0/1250640824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20980751e0 msgr2=0x7f20980755f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.739+0000 7f20a0292640 1 --2- 172.21.15.67:0/1250640824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20980751e0 0x7f20980755f0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f2094009900 tx=0x7f209402a410 comp rx=0 tx=0).stop 2024-07-31T15:11:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.739+0000 7f20a0292640 1 -- 172.21.15.67:0/1250640824 shutdown_connections 2024-07-31T15:11:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.739+0000 7f20a0292640 1 --2- 172.21.15.67:0/1250640824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f209810fa70 0x7f2098111ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.739+0000 7f20a0292640 1 --2- 172.21.15.67:0/1250640824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2098075b30 0x7f209810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.739+0000 7f20a0292640 1 --2- 172.21.15.67:0/1250640824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20980751e0 0x7f20980755f0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.739+0000 7f20a0292640 1 -- 172.21.15.67:0/1250640824 >> 172.21.15.67:0/1250640824 conn(0x7f20980fef80 msgr2=0x7f20981013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.739+0000 7f20a0292640 1 -- 172.21.15.67:0/1250640824 shutdown_connections 2024-07-31T15:11:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.739+0000 7f20a0292640 1 -- 172.21.15.67:0/1250640824 wait complete. 2024-07-31T15:11:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.740+0000 7f20a0292640 1 Processor -- start 2024-07-31T15:11:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.740+0000 7f20a0292640 1 -- start start 2024-07-31T15:11:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.740+0000 7f20a0292640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20980751e0 0x7f209819c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.740+0000 7f20a0292640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2098075b30 0x7f209819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.740+0000 7f20a0292640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209810fa70 0x7f20981a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.740+0000 7f20a0292640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2098115680 con 0x7f20980751e0 2024-07-31T15:11:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.740+0000 7f20a0292640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2098115500 con 0x7f2098075b30 2024-07-31T15:11:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.740+0000 7f20a0292640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2098115800 con 0x7f209810fa70 2024-07-31T15:11:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f209e02e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20980751e0 0x7f209819c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f209d82d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2098075b30 0x7f209819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f209d82d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2098075b30 0x7f209819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:53724/0 (socket says 172.21.15.67:53724) 2024-07-31T15:11:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f209e02e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20980751e0 0x7f209819c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37310/0 (socket says 172.21.15.67:37310) 2024-07-31T15:11:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f209d82d640 1 -- 172.21.15.67:0/1029814168 learned_addr learned my addr 172.21.15.67:0/1029814168 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f209e82f640 1 --2- 172.21.15.67:0/1029814168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209810fa70 0x7f20981a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f209d82d640 1 -- 172.21.15.67:0/1029814168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209810fa70 msgr2=0x7f20981a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f209d82d640 1 --2- 172.21.15.67:0/1029814168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209810fa70 0x7f20981a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f209d82d640 1 -- 172.21.15.67:0/1029814168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20980751e0 msgr2=0x7f209819c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f209d82d640 1 --2- 172.21.15.67:0/1029814168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20980751e0 0x7f209819c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f209d82d640 1 -- 172.21.15.67:0/1029814168 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20981a2010 con 0x7f2098075b30 2024-07-31T15:11:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f209d82d640 1 --2- 172.21.15.67:0/1029814168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2098075b30 0x7f209819c680 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f208c00e9c0 tx=0x7f208c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f208b7fe640 1 -- 172.21.15.67:0/1029814168 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c00ccc0 con 0x7f2098075b30 2024-07-31T15:11:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f20a0292640 1 -- 172.21.15.67:0/1029814168 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20981a2300 con 0x7f2098075b30 2024-07-31T15:11:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f208b7fe640 1 -- 172.21.15.67:0/1029814168 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f208c0054d0 con 0x7f2098075b30 2024-07-31T15:11:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f208b7fe640 1 -- 172.21.15.67:0/1029814168 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c010430 con 0x7f2098075b30 2024-07-31T15:11:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.741+0000 7f20a0292640 1 -- 172.21.15.67:0/1029814168 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20981a8a90 con 0x7f2098075b30 2024-07-31T15:11:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.744+0000 7f20a7188740 1 Processor -- start 2024-07-31T15:11:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.744+0000 7f20a7188740 1 -- start start 2024-07-31T15:11:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f20a7188740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af3e1b0 0x55633b2e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f20a7188740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af152f0 0x55633b2e1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f20a7188740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af1d4d0 0x55633b2e1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f20a7188740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55633b2563b0 con 0x55633af1d4d0 2024-07-31T15:11:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f20a7188740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55633b2458b0 con 0x55633af3e1b0 2024-07-31T15:11:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f20a7188740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55633b2498f0 con 0x55633af152f0 2024-07-31T15:11:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f208b7fe640 1 -- 172.21.15.67:0/1029814168 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f208c005020 con 0x7f2098075b30 2024-07-31T15:11:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f209e82f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af3e1b0 0x55633b2e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f209e82f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af3e1b0 0x55633b2e0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:53732/0 (socket says 172.21.15.67:53732) 2024-07-31T15:11:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f209e82f640 1 -- 172.21.15.67:0/502785749 learned_addr learned my addr 172.21.15.67:0/502785749 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f209e02e640 1 --2- 172.21.15.67:0/502785749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af152f0 0x55633b2e1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f209d82d640 1 --2- 172.21.15.67:0/502785749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af1d4d0 0x55633b2e1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f209e82f640 1 -- 172.21.15.67:0/502785749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af152f0 msgr2=0x55633b2e1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f209e82f640 1 --2- 172.21.15.67:0/502785749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af152f0 0x55633b2e1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f209e82f640 1 -- 172.21.15.67:0/502785749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af1d4d0 msgr2=0x55633b2e1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f209e82f640 1 --2- 172.21.15.67:0/502785749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af1d4d0 0x55633b2e1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f209e82f640 1 -- 172.21.15.67:0/502785749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55633b1ec9a0 con 0x55633af3e1b0 2024-07-31T15:11:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f209d82d640 1 --2- 172.21.15.67:0/502785749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af1d4d0 0x55633b2e1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:11:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.745+0000 7f209e82f640 1 --2- 172.21.15.67:0/502785749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af3e1b0 0x55633b2e0ce0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2090010400 tx=0x7f20900108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef42f1f86b2fe2e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.746+0000 7f2088bfe640 1 -- 172.21.15.67:0/502785749 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090003b70 con 0x55633af3e1b0 2024-07-31T15:11:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.746+0000 7f2088bfe640 1 -- 172.21.15.67:0/502785749 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090003d10 con 0x55633af3e1b0 2024-07-31T15:11:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.746+0000 7f2088bfe640 1 -- 172.21.15.67:0/502785749 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090011a40 con 0x55633af3e1b0 2024-07-31T15:11:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.746+0000 7f208b7fe640 1 --2- 172.21.15.67:0/1029814168 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f20840b3d40 0x7f20840b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.746+0000 7f209e02e640 1 --2- 172.21.15.67:0/1029814168 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f20840b3d40 0x7f20840b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.746+0000 7f208b7fe640 1 -- 172.21.15.67:0/1029814168 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7f208c0e4760 con 0x7f2098075b30 2024-07-31T15:11:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.746+0000 7f20a7188740 1 -- 172.21.15.67:0/502785749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af3e1b0 msgr2=0x55633b2e0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.746+0000 7f209e02e640 1 --2- 172.21.15.67:0/1029814168 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f20840b3d40 0x7f20840b6250 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2094031440 tx=0x7f2094009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.746+0000 7f20a7188740 1 --2- 172.21.15.67:0/502785749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af3e1b0 0x55633b2e0ce0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2090010400 tx=0x7f20900108c0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.746+0000 7f20a7188740 1 -- 172.21.15.67:0/502785749 shutdown_connections 2024-07-31T15:11:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.746+0000 7f20a7188740 1 --2- 172.21.15.67:0/502785749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af1d4d0 0x55633b2e1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.746+0000 7f20a7188740 1 --2- 172.21.15.67:0/502785749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af152f0 0x55633b2e1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.746+0000 7f20a7188740 1 --2- 172.21.15.67:0/502785749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af3e1b0 0x55633b2e0ce0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.747+0000 7f20a7188740 1 -- 172.21.15.67:0/502785749 >> 172.21.15.67:0/502785749 conn(0x55633af32300 msgr2=0x55633b2debd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.747+0000 7f20a7188740 1 -- 172.21.15.67:0/502785749 shutdown_connections 2024-07-31T15:11:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.747+0000 7f20a7188740 1 -- 172.21.15.67:0/502785749 wait complete. 2024-07-31T15:11:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.747+0000 7f20a7188740 1 -- start start 2024-07-31T15:11:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.747+0000 7f20a7188740 1 Processor -- start 2024-07-31T15:11:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.747+0000 7f20a7188740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af152f0 0x55633b2ec850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.747+0000 7f20a7188740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af1d4d0 0x55633b2ecd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.747+0000 7f20a7188740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af3e1b0 0x55633b2f8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.747+0000 7f20a7188740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55633b17fab0 con 0x55633af152f0 2024-07-31T15:11:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.747+0000 7f20a7188740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55633b1cdcf0 con 0x55633af1d4d0 2024-07-31T15:11:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.747+0000 7f20a7188740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55633b2ef460 con 0x55633af3e1b0 2024-07-31T15:11:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f209e02e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af152f0 0x55633b2ec850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f20a7188740 10 client.0 _ll_register_callbacks cb 0x55633b23f9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f209d82d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af1d4d0 0x55633b2ecd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f209e02e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af152f0 0x55633b2ec850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37314/0 (socket says 172.21.15.67:37314) 2024-07-31T15:11:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f209e02e640 1 -- 172.21.15.67:0/3294532560 learned_addr learned my addr 172.21.15.67:0/3294532560 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f209e82f640 1 --2- 172.21.15.67:0/3294532560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af3e1b0 0x55633b2f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f209d82d640 1 -- 172.21.15.67:0/3294532560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af3e1b0 msgr2=0x55633b2f8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f209d82d640 1 --2- 172.21.15.67:0/3294532560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af3e1b0 0x55633b2f8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f209d82d640 1 -- 172.21.15.67:0/3294532560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af152f0 msgr2=0x55633b2ec850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f209d82d640 1 --2- 172.21.15.67:0/3294532560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af152f0 0x55633b2ec850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f209d82d640 1 -- 172.21.15.67:0/3294532560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55633afa9790 con 0x55633af1d4d0 2024-07-31T15:11:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f209d82d640 1 --2- 172.21.15.67:0/3294532560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af1d4d0 0x55633b2ecd90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f208c0e1b40 tx=0x7f208c0aa770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca5b6c1ac6c22f37 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f2081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:11:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f2081ffb640 1 -- 172.21.15.67:0/3294532560 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c005680 con 0x55633af1d4d0 2024-07-31T15:11:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f2081ffb640 1 -- 172.21.15.67:0/3294532560 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7f208c0b3c20 con 0x55633af1d4d0 2024-07-31T15:11:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f20a7188740 1 -- 172.21.15.67:0/3294532560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2090003d10 con 0x55633af1d4d0 2024-07-31T15:11:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f2081ffb640 1 -- 172.21.15.67:0/3294532560 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f208c0aab10 con 0x55633af1d4d0 2024-07-31T15:11:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f20a7188740 10 client.29399 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.748+0000 7f20a7188740 1 -- 172.21.15.67:0/3294532560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55633b022620 con 0x55633af1d4d0 2024-07-31T15:11:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.749+0000 7f2081ffb640 1 -- 172.21.15.67:0/3294532560 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090003d10 con 0x55633af1d4d0 2024-07-31T15:11:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.749+0000 7f2081ffb640 1 -- 172.21.15.67:0/3294532560 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f208c0a6970 con 0x55633af1d4d0 2024-07-31T15:11:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.749+0000 7f20a7188740 10 client.29399 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.749+0000 7f20a7188740 10 client.29399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:11:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.749+0000 7f20a7188740 20 client.29399 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.749+0000 7f20a7188740 1 --2- 172.21.15.67:0/3294532560 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55633aefcd80 0x55633b2fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.749+0000 7f20a7188740 4 client.29399 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:11:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.749+0000 7f20a7188740 1 -- 172.21.15.67:0/3294532560 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55633b258300 con 0x55633aefcd80 2024-07-31T15:11:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.749+0000 7f209e02e640 1 --2- 172.21.15.67:0/3294532560 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55633aefcd80 0x55633b2fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.749+0000 7f2081ffb640 10 client.29399 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.749+0000 7f209e02e640 1 --2- 172.21.15.67:0/3294532560 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55633aefcd80 0x55633b2fe770 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1a895d29f69d05d server_cookie=73ae565926963fa1 in_seq=0 out_seq=0 2024-07-31T15:11:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.879+0000 7f2081ffb640 1 -- 172.21.15.67:0/3294532560 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55633b258300 con 0x55633aefcd80 2024-07-31T15:11:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.879+0000 7f2081ffb640 10 client.29399 handle_command_reply: tid=0 2024-07-31T15:11:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.879+0000 7f20a7188740 1 client.29399 shutdown 2024-07-31T15:11:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.879+0000 7f20a7188740 1 -- 172.21.15.67:0/3294532560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af1d4d0 msgr2=0x55633b2ecd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.879+0000 7f20a7188740 1 --2- 172.21.15.67:0/3294532560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af1d4d0 0x55633b2ecd90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f208c0e1b40 tx=0x7f208c0aa770 comp rx=0 tx=0).stop 2024-07-31T15:11:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.880+0000 7f20a7188740 1 -- 172.21.15.67:0/3294532560 shutdown_connections 2024-07-31T15:11:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.880+0000 7f20a7188740 1 --2- 172.21.15.67:0/3294532560 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55633aefcd80 0x55633b2fe770 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.880+0000 7f20a7188740 1 --2- 172.21.15.67:0/3294532560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af3e1b0 0x55633b2f8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.880+0000 7f20a7188740 1 --2- 172.21.15.67:0/3294532560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af1d4d0 0x55633b2ecd90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.880+0000 7f2081ffb640 0 client.29399 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.880+0000 7f20a7188740 1 --2- 172.21.15.67:0/3294532560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af152f0 0x55633b2ec850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.880+0000 7f20a7188740 1 -- 172.21.15.67:0/3294532560 >> 172.21.15.67:0/3294532560 conn(0x55633af32300 msgr2=0x55633b2e1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.880+0000 7f20a7188740 1 -- 172.21.15.67:0/3294532560 shutdown_connections 2024-07-31T15:11:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.880+0000 7f20a7188740 1 -- 172.21.15.67:0/3294532560 wait complete. 2024-07-31T15:11:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.880+0000 7f20a7188740 20 client.29399 trim_cache size 0 max 16384 2024-07-31T15:11:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.895+0000 7f20a7188740 1 Processor -- start 2024-07-31T15:11:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.895+0000 7f20a7188740 1 -- start start 2024-07-31T15:11:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.895+0000 7f20a7188740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af152f0 0x55633b2e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.895+0000 7f20a7188740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af1d4d0 0x55633b2e6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.895+0000 7f20a7188740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af3e1b0 0x55633b2e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.895+0000 7f20a7188740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b2a10 con 0x55633af3e1b0 2024-07-31T15:11:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.895+0000 7f20a7188740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b4550 con 0x55633af1d4d0 2024-07-31T15:11:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.895+0000 7f20a7188740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b30e0 con 0x55633af152f0 2024-07-31T15:11:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.895+0000 7f209e82f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af3e1b0 0x55633b2e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.896+0000 7f209e82f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af3e1b0 0x55633b2e1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37330/0 (socket says 172.21.15.67:37330) 2024-07-31T15:11:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.896+0000 7f209e82f640 1 -- 172.21.15.67:0/4203355492 learned_addr learned my addr 172.21.15.67:0/4203355492 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.896+0000 7f209d82d640 1 --2- 172.21.15.67:0/4203355492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af1d4d0 0x55633b2e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.896+0000 7f209e02e640 1 --2- 172.21.15.67:0/4203355492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af152f0 0x55633b2e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.896+0000 7f209e82f640 1 -- 172.21.15.67:0/4203355492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af152f0 msgr2=0x55633b2e5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.896+0000 7f209e82f640 1 --2- 172.21.15.67:0/4203355492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af152f0 0x55633b2e5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.896+0000 7f209e82f640 1 -- 172.21.15.67:0/4203355492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af1d4d0 msgr2=0x55633b2e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.896+0000 7f209e82f640 1 --2- 172.21.15.67:0/4203355492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af1d4d0 0x55633b2e6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.896+0000 7f209e82f640 1 -- 172.21.15.67:0/4203355492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55633b2eab20 con 0x55633af3e1b0 2024-07-31T15:11:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.896+0000 7f209d82d640 1 --2- 172.21.15.67:0/4203355492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af1d4d0 0x55633b2e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.896+0000 7f209e82f640 1 --2- 172.21.15.67:0/4203355492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af3e1b0 0x55633b2e1d10 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f209000a010 tx=0x7f20900116a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d394837aa8391a6d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.896+0000 7f2083fff640 1 -- 172.21.15.67:0/4203355492 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090004c00 con 0x55633af3e1b0 2024-07-31T15:11:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.896+0000 7f2083fff640 1 -- 172.21.15.67:0/4203355492 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090003e20 con 0x55633af3e1b0 2024-07-31T15:11:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.896+0000 7f2083fff640 1 -- 172.21.15.67:0/4203355492 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090004020 con 0x55633af3e1b0 2024-07-31T15:11:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.897+0000 7f20a7188740 1 -- 172.21.15.67:0/4203355492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af3e1b0 msgr2=0x55633b2e1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.897+0000 7f20a7188740 1 --2- 172.21.15.67:0/4203355492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af3e1b0 0x55633b2e1d10 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f209000a010 tx=0x7f20900116a0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.897+0000 7f20a7188740 1 -- 172.21.15.67:0/4203355492 shutdown_connections 2024-07-31T15:11:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.897+0000 7f20a7188740 1 --2- 172.21.15.67:0/4203355492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af3e1b0 0x55633b2e1d10 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.897+0000 7f20a7188740 1 --2- 172.21.15.67:0/4203355492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af1d4d0 0x55633b2e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.897+0000 7f20a7188740 1 --2- 172.21.15.67:0/4203355492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af152f0 0x55633b2e5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.897+0000 7f20a7188740 1 -- 172.21.15.67:0/4203355492 >> 172.21.15.67:0/4203355492 conn(0x55633af32300 msgr2=0x55633b2ec850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.897+0000 7f20a7188740 1 -- 172.21.15.67:0/4203355492 shutdown_connections 2024-07-31T15:11:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.897+0000 7f20a7188740 1 -- 172.21.15.67:0/4203355492 wait complete. 2024-07-31T15:11:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.897+0000 7f20a7188740 1 -- start start 2024-07-31T15:11:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f20a7188740 1 Processor -- start 2024-07-31T15:11:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f20a7188740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af152f0 0x55633b2f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f20a7188740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af1d4d0 0x55633b2f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f20a7188740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af3e1b0 0x55633b364f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f20a7188740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b3e80 con 0x55633af3e1b0 2024-07-31T15:11:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f20a7188740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b37b0 con 0x55633af152f0 2024-07-31T15:11:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f20a7188740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b0a10 con 0x55633af1d4d0 2024-07-31T15:11:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f20a7188740 10 client.0 _ll_register_callbacks cb 0x7f208c0096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f209e82f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af3e1b0 0x55633b364f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f209e82f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af3e1b0 0x55633b364f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37340/0 (socket says 172.21.15.67:37340) 2024-07-31T15:11:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f209d82d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af1d4d0 0x55633b2f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f209e82f640 1 -- 172.21.15.67:0/1119405632 learned_addr learned my addr 172.21.15.67:0/1119405632 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f209e02e640 1 --2- 172.21.15.67:0/1119405632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af152f0 0x55633b2f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f209e82f640 1 -- 172.21.15.67:0/1119405632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af1d4d0 msgr2=0x55633b2f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f209e82f640 1 --2- 172.21.15.67:0/1119405632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af1d4d0 0x55633b2f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f209e82f640 1 -- 172.21.15.67:0/1119405632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af152f0 msgr2=0x55633b2f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f209e82f640 1 --2- 172.21.15.67:0/1119405632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af152f0 0x55633b2f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f209e82f640 1 -- 172.21.15.67:0/1119405632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55633b3654f0 con 0x55633af3e1b0 2024-07-31T15:11:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f209e02e640 1 --2- 172.21.15.67:0/1119405632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af152f0 0x55633b2f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.898+0000 7f209d82d640 1 --2- 172.21.15.67:0/1119405632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af1d4d0 0x55633b2f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f209e82f640 1 --2- 172.21.15.67:0/1119405632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af3e1b0 0x55633b364f30 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2090000f80 tx=0x7f2090009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79521ff6981a2571 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f2081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:11:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f2081ffb640 1 -- 172.21.15.67:0/1119405632 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209002e670 con 0x55633af3e1b0 2024-07-31T15:11:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f2081ffb640 1 -- 172.21.15.67:0/1119405632 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7f209002e810 con 0x55633af3e1b0 2024-07-31T15:11:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f20a7188740 1 -- 172.21.15.67:0/1119405632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2090003e20 con 0x55633af3e1b0 2024-07-31T15:11:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f2081ffb640 1 -- 172.21.15.67:0/1119405632 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f2090003c40 con 0x55633af3e1b0 2024-07-31T15:11:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f20a7188740 10 client.19875 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f20a7188740 1 -- 172.21.15.67:0/1119405632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55633b366780 con 0x55633af3e1b0 2024-07-31T15:11:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f2081ffb640 1 -- 172.21.15.67:0/1119405632 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090003e20 con 0x55633af3e1b0 2024-07-31T15:11:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f2081ffb640 1 -- 172.21.15.67:0/1119405632 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f2090024070 con 0x55633af3e1b0 2024-07-31T15:11:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f20a7188740 10 client.19875 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f20a7188740 10 client.19875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:11:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f20a7188740 20 client.19875 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f20a7188740 1 --2- 172.21.15.67:0/1119405632 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55633aed21c0 0x55633b368c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f20a7188740 4 client.19875 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:11:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.899+0000 7f20a7188740 1 -- 172.21.15.67:0/1119405632 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55633b253da0 con 0x55633aed21c0 2024-07-31T15:11:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.900+0000 7f209e02e640 1 --2- 172.21.15.67:0/1119405632 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55633aed21c0 0x55633b368c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.900+0000 7f2081ffb640 10 client.19875 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:11:57.900+0000 7f209e02e640 1 --2- 172.21.15.67:0/1119405632 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55633aed21c0 0x55633b368c30 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d2437629509ce8 server_cookie=939dedc5fc557032 in_seq=0 out_seq=0 2024-07-31T15:11:59.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:11:59 smithi067 ceph-mon[32750]: pgmap v1067: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 512 KiB/s wr, 157 op/s 2024-07-31T15:11:59.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:11:59 smithi073 ceph-mon[36714]: pgmap v1067: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 512 KiB/s wr, 157 op/s 2024-07-31T15:11:59.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.421+0000 7feafd3da640 1 -- 172.21.15.67:0/2327103651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf81043e0 msgr2=0x7feaf81047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.421+0000 7feafd3da640 1 --2- 172.21.15.67:0/2327103651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf81043e0 0x7feaf81047f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7feae8009900 tx=0x7feae802a410 comp rx=0 tx=0).stop 2024-07-31T15:11:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.421+0000 7feafd3da640 1 -- 172.21.15.67:0/2327103651 shutdown_connections 2024-07-31T15:11:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.421+0000 7feafd3da640 1 --2- 172.21.15.67:0/2327103651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf810d840 0x7feaf810fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.421+0000 7feafd3da640 1 --2- 172.21.15.67:0/2327103651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf8104d30 0x7feaf810d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.421+0000 7feafd3da640 1 --2- 172.21.15.67:0/2327103651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf81043e0 0x7feaf81047f0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.421+0000 7feafd3da640 1 -- 172.21.15.67:0/2327103651 >> 172.21.15.67:0/2327103651 conn(0x7feaf8079160 msgr2=0x7feaf8079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.421+0000 7feafd3da640 1 -- 172.21.15.67:0/2327103651 shutdown_connections 2024-07-31T15:11:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.421+0000 7feafd3da640 1 -- 172.21.15.67:0/2327103651 wait complete. 2024-07-31T15:11:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.422+0000 7feafd3da640 1 Processor -- start 2024-07-31T15:11:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.422+0000 7feafd3da640 1 -- start start 2024-07-31T15:11:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.422+0000 7feafd3da640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf81043e0 0x7feaf8102120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.422+0000 7feafd3da640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf8104d30 0x7feaf8102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.422+0000 7feafd3da640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf810d840 0x7feaf8102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.422+0000 7feafd3da640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feaf81134b0 con 0x7feaf810d840 2024-07-31T15:11:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.422+0000 7feafd3da640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8113330 con 0x7feaf81043e0 2024-07-31T15:11:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.422+0000 7feafd3da640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8113630 con 0x7feaf8104d30 2024-07-31T15:11:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.422+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf810d840 0x7feaf8102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.422+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf810d840 0x7feaf8102ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37360/0 (socket says 172.21.15.67:37360) 2024-07-31T15:11:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.422+0000 7feaf77fe640 1 -- 172.21.15.67:0/990980439 learned_addr learned my addr 172.21.15.67:0/990980439 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.422+0000 7feaf6ffd640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf81043e0 0x7feaf8102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.423+0000 7feaf67fc640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf8104d30 0x7feaf8102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.423+0000 7feaf6ffd640 1 -- 172.21.15.67:0/990980439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf8104d30 msgr2=0x7feaf8102660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.423+0000 7feaf6ffd640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf8104d30 0x7feaf8102660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.423+0000 7feaf6ffd640 1 -- 172.21.15.67:0/990980439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf810d840 msgr2=0x7feaf8102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.423+0000 7feaf6ffd640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf810d840 0x7feaf8102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.423+0000 7feaf6ffd640 1 -- 172.21.15.67:0/990980439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf81a82a0 con 0x7feaf81043e0 2024-07-31T15:11:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.423+0000 7feaf77fe640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf810d840 0x7feaf8102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:11:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.423+0000 7feaf6ffd640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf81043e0 0x7feaf8102120 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7feae8006010 tx=0x7feae8004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.423+0000 7feadffff640 1 -- 172.21.15.67:0/990980439 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae803f070 con 0x7feaf81043e0 2024-07-31T15:11:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.423+0000 7feadffff640 1 -- 172.21.15.67:0/990980439 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feae8003ca0 con 0x7feaf81043e0 2024-07-31T15:11:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.423+0000 7feadffff640 1 -- 172.21.15.67:0/990980439 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8035070 con 0x7feaf81043e0 2024-07-31T15:11:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.423+0000 7feafd3da640 1 -- 172.21.15.67:0/990980439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf81a8530 con 0x7feaf81043e0 2024-07-31T15:11:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.423+0000 7feafd3da640 1 -- 172.21.15.67:0/990980439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf81a89c0 con 0x7feaf81043e0 2024-07-31T15:11:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.426+0000 7feadffff640 1 -- 172.21.15.67:0/990980439 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7feae803b460 con 0x7feaf81043e0 2024-07-31T15:11:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.426+0000 7feb0392b740 1 Processor -- start 2024-07-31T15:11:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.426+0000 7feb0392b740 1 -- start start 2024-07-31T15:11:59.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feb0392b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd506070 0x5612dd8ab690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feb0392b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4dd1b0 0x5612dd8abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feb0392b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8ac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feb0392b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5612dd821bb0 con 0x5612dd4e5390 2024-07-31T15:11:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feb0392b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5612dd8110b0 con 0x5612dd506070 2024-07-31T15:11:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feb0392b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5612dd8150f0 con 0x5612dd4dd1b0 2024-07-31T15:11:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd506070 0x5612dd8ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd506070 0x5612dd8ab690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:53776/0 (socket says 172.21.15.67:53776) 2024-07-31T15:11:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feaf67fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feaf77fe640 1 -- 172.21.15.67:0/4280245539 learned_addr learned my addr 172.21.15.67:0/4280245539 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feaf6ffd640 1 --2- 172.21.15.67:0/4280245539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4dd1b0 0x5612dd8abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feadffff640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feae00b3d40 0x7feae00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feaf67fc640 1 -- 172.21.15.67:0/4280245539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4dd1b0 msgr2=0x5612dd8abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feaf67fc640 1 --2- 172.21.15.67:0/4280245539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4dd1b0 0x5612dd8abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feaf67fc640 1 -- 172.21.15.67:0/4280245539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd506070 msgr2=0x5612dd8ab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feaf67fc640 1 --2- 172.21.15.67:0/4280245539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd506070 0x5612dd8ab690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.427+0000 7feaf67fc640 1 -- 172.21.15.67:0/4280245539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612dd7b7b90 con 0x5612dd4e5390 2024-07-31T15:11:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.428+0000 7feaf77fe640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feae00b3d40 0x7feae00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.428+0000 7feadffff640 1 -- 172.21.15.67:0/990980439 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7feae81047c0 con 0x7feaf81043e0 2024-07-31T15:11:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.428+0000 7feaf67fc640 1 --2- 172.21.15.67:0/4280245539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8ac110 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7feaf806a4d0 tx=0x7feaec00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e703430d727eff server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.428+0000 7feadd7fa640 1 -- 172.21.15.67:0/4280245539 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec010e20 con 0x5612dd4e5390 2024-07-31T15:11:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.428+0000 7feadd7fa640 1 -- 172.21.15.67:0/4280245539 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaec0035f0 con 0x5612dd4e5390 2024-07-31T15:11:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.428+0000 7feadd7fa640 1 -- 172.21.15.67:0/4280245539 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec0044e0 con 0x5612dd4e5390 2024-07-31T15:11:59.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.428+0000 7feaf77fe640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feae00b3d40 0x7feae00b6250 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7feaf00116f0 tx=0x7feaf0004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.429+0000 7feb0392b740 1 -- 172.21.15.67:0/4280245539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 msgr2=0x5612dd8ac110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.429+0000 7feb0392b740 1 --2- 172.21.15.67:0/4280245539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8ac110 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7feaf806a4d0 tx=0x7feaec00bf90 comp rx=0 tx=0).stop 2024-07-31T15:11:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.429+0000 7feb0392b740 1 -- 172.21.15.67:0/4280245539 shutdown_connections 2024-07-31T15:11:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.429+0000 7feb0392b740 1 --2- 172.21.15.67:0/4280245539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8ac110 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.429+0000 7feb0392b740 1 --2- 172.21.15.67:0/4280245539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4dd1b0 0x5612dd8abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.429+0000 7feb0392b740 1 --2- 172.21.15.67:0/4280245539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd506070 0x5612dd8ab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.429+0000 7feb0392b740 1 -- 172.21.15.67:0/4280245539 >> 172.21.15.67:0/4280245539 conn(0x5612dd4fa460 msgr2=0x5612dd8a9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:59.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.429+0000 7feb0392b740 1 -- 172.21.15.67:0/4280245539 shutdown_connections 2024-07-31T15:11:59.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.429+0000 7feb0392b740 1 -- 172.21.15.67:0/4280245539 wait complete. 2024-07-31T15:11:59.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.430+0000 7feb0392b740 1 -- start start 2024-07-31T15:11:59.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.430+0000 7feb0392b740 1 Processor -- start 2024-07-31T15:11:59.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.430+0000 7feb0392b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4dd1b0 0x5612dd8b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.430+0000 7feb0392b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8b7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.430+0000 7feb0392b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd506070 0x5612dd8c3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.430+0000 7feb0392b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5612dd674700 con 0x5612dd4e5390 2024-07-31T15:11:59.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.430+0000 7feb0392b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5612dd7b39e0 con 0x5612dd4dd1b0 2024-07-31T15:11:59.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.430+0000 7feb0392b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5612dd8b9e70 con 0x5612dd506070 2024-07-31T15:11:59.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.430+0000 7feb0392b740 10 client.0 _ll_register_callbacks cb 0x5612dd7ce4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:59.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.430+0000 7feaf67fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.430+0000 7feaf67fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8b7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37374/0 (socket says 172.21.15.67:37374) 2024-07-31T15:11:59.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.430+0000 7feaf6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4dd1b0 0x5612dd8b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.430+0000 7feaf67fc640 1 -- 172.21.15.67:0/2721468781 learned_addr learned my addr 172.21.15.67:0/2721468781 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:59.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.430+0000 7feaf77fe640 1 --2- 172.21.15.67:0/2721468781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd506070 0x5612dd8c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.431+0000 7feaf67fc640 1 -- 172.21.15.67:0/2721468781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd506070 msgr2=0x5612dd8c3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:59.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.431+0000 7feaf67fc640 1 --2- 172.21.15.67:0/2721468781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd506070 0x5612dd8c3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.431+0000 7feaf67fc640 1 -- 172.21.15.67:0/2721468781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4dd1b0 msgr2=0x5612dd8b7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:59.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.431+0000 7feaf67fc640 1 --2- 172.21.15.67:0/2721468781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4dd1b0 0x5612dd8b7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.431+0000 7feaf67fc640 1 -- 172.21.15.67:0/2721468781 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612dd571650 con 0x5612dd4e5390 2024-07-31T15:11:59.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.431+0000 7feaf67fc640 1 --2- 172.21.15.67:0/2721468781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8b7740 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7feaec003f70 tx=0x7feaec010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f379b835aa766bd8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:59.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.431+0000 7feadaff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:11:59.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.431+0000 7feadaff5640 1 -- 172.21.15.67:0/2721468781 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec02a060 con 0x5612dd4e5390 2024-07-31T15:11:59.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.431+0000 7feadaff5640 1 -- 172.21.15.67:0/2721468781 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7feaec004000 con 0x5612dd4e5390 2024-07-31T15:11:59.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.431+0000 7feb0392b740 1 -- 172.21.15.67:0/2721468781 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaec0035f0 con 0x5612dd4e5390 2024-07-31T15:11:59.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.431+0000 7feadaff5640 1 -- 172.21.15.67:0/2721468781 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7feaec031050 con 0x5612dd4e5390 2024-07-31T15:11:59.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.431+0000 7feb0392b740 10 client.19887 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:59.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.431+0000 7feb0392b740 1 -- 172.21.15.67:0/2721468781 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5612dd8c5e30 con 0x5612dd4e5390 2024-07-31T15:11:59.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.432+0000 7feadaff5640 1 -- 172.21.15.67:0/2721468781 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaec0035f0 con 0x5612dd4e5390 2024-07-31T15:11:59.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.432+0000 7feadaff5640 1 -- 172.21.15.67:0/2721468781 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7feaec0046c0 con 0x5612dd4e5390 2024-07-31T15:11:59.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.432+0000 7feb0392b740 10 client.19887 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:59.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.432+0000 7feb0392b740 10 client.19887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:11:59.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.432+0000 7feb0392b740 20 client.19887 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:59.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.432+0000 7feb0392b740 1 --2- 172.21.15.67:0/2721468781 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5612dd4c49f0 0x5612dd8c9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.432+0000 7feb0392b740 4 client.19887 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:11:59.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.432+0000 7feb0392b740 1 -- 172.21.15.67:0/2721468781 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612dd823b00 con 0x5612dd4c49f0 2024-07-31T15:11:59.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.432+0000 7feaf6ffd640 1 --2- 172.21.15.67:0/2721468781 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5612dd4c49f0 0x5612dd8c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.433+0000 7feaf6ffd640 1 --2- 172.21.15.67:0/2721468781 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5612dd4c49f0 0x5612dd8c9190 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e20d57b82be560 server_cookie=95b2776598b155b in_seq=0 out_seq=0 2024-07-31T15:11:59.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.433+0000 7feadaff5640 10 client.19887 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:59.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:11:59 smithi077 ceph-mon[37327]: pgmap v1067: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 512 KiB/s wr, 157 op/s 2024-07-31T15:11:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feadaff5640 1 -- 172.21.15.67:0/2721468781 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612dd823b00 con 0x5612dd4c49f0 2024-07-31T15:11:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feadaff5640 10 client.19887 handle_command_reply: tid=0 2024-07-31T15:11:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feb0392b740 1 client.19887 shutdown 2024-07-31T15:11:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feb0392b740 1 -- 172.21.15.67:0/2721468781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 msgr2=0x5612dd8b7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feb0392b740 1 --2- 172.21.15.67:0/2721468781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8b7740 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7feaec003f70 tx=0x7feaec010e50 comp rx=0 tx=0).stop 2024-07-31T15:11:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feb0392b740 1 -- 172.21.15.67:0/2721468781 shutdown_connections 2024-07-31T15:11:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feb0392b740 1 --2- 172.21.15.67:0/2721468781 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5612dd4c49f0 0x5612dd8c9190 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feb0392b740 1 --2- 172.21.15.67:0/2721468781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd506070 0x5612dd8c3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feb0392b740 1 --2- 172.21.15.67:0/2721468781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8b7740 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feb0392b740 1 --2- 172.21.15.67:0/2721468781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4dd1b0 0x5612dd8b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feadaff5640 0 client.19887 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:59.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feb0392b740 1 -- 172.21.15.67:0/2721468781 >> 172.21.15.67:0/2721468781 conn(0x5612dd4fa460 msgr2=0x5612dd8ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:59.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feb0392b740 1 -- 172.21.15.67:0/2721468781 shutdown_connections 2024-07-31T15:11:59.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feb0392b740 1 -- 172.21.15.67:0/2721468781 wait complete. 2024-07-31T15:11:59.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.671+0000 7feb0392b740 20 client.19887 trim_cache size 0 max 16384 2024-07-31T15:11:59.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feb0392b740 1 Processor -- start 2024-07-31T15:11:59.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feb0392b740 1 -- start start 2024-07-31T15:11:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feb0392b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4c49f0 0x5612dd8a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feb0392b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4dd1b0 0x5612dd8c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feb0392b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4e5390 0x5612dd8c3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feb0392b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feaf81b2a20 con 0x5612dd4dd1b0 2024-07-31T15:11:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feb0392b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feaf81b4560 con 0x5612dd4c49f0 2024-07-31T15:11:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feb0392b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feaf81b30f0 con 0x5612dd4e5390 2024-07-31T15:11:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feaf67fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4dd1b0 0x5612dd8c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feaf67fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4dd1b0 0x5612dd8c30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37384/0 (socket says 172.21.15.67:37384) 2024-07-31T15:11:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feaf67fc640 1 -- 172.21.15.67:0/1668915823 learned_addr learned my addr 172.21.15.67:0/1668915823 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feaf6ffd640 1 --2- 172.21.15.67:0/1668915823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4c49f0 0x5612dd8a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feaf77fe640 1 --2- 172.21.15.67:0/1668915823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4e5390 0x5612dd8c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feaf67fc640 1 -- 172.21.15.67:0/1668915823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4e5390 msgr2=0x5612dd8c3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feaf67fc640 1 --2- 172.21.15.67:0/1668915823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4e5390 0x5612dd8c3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feaf67fc640 1 -- 172.21.15.67:0/1668915823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4c49f0 msgr2=0x5612dd8a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feaf67fc640 1 --2- 172.21.15.67:0/1668915823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4c49f0 0x5612dd8a95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.686+0000 7feaf67fc640 1 -- 172.21.15.67:0/1668915823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612dd8a4fa0 con 0x5612dd4dd1b0 2024-07-31T15:11:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.687+0000 7feaf67fc640 1 --2- 172.21.15.67:0/1668915823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4dd1b0 0x5612dd8c30d0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7feaec004ac0 tx=0x7feaec003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58abd63f104f8cc3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.687+0000 7feadcff9640 1 -- 172.21.15.67:0/1668915823 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec02a070 con 0x5612dd4dd1b0 2024-07-31T15:11:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.687+0000 7feadcff9640 1 -- 172.21.15.67:0/1668915823 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaec003b70 con 0x5612dd4dd1b0 2024-07-31T15:11:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.687+0000 7feadcff9640 1 -- 172.21.15.67:0/1668915823 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec003fa0 con 0x5612dd4dd1b0 2024-07-31T15:11:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.688+0000 7feb0392b740 1 -- 172.21.15.67:0/1668915823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4dd1b0 msgr2=0x5612dd8c30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.688+0000 7feb0392b740 1 --2- 172.21.15.67:0/1668915823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4dd1b0 0x5612dd8c30d0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7feaec004ac0 tx=0x7feaec003ee0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.688+0000 7feb0392b740 1 -- 172.21.15.67:0/1668915823 shutdown_connections 2024-07-31T15:11:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.688+0000 7feb0392b740 1 --2- 172.21.15.67:0/1668915823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4e5390 0x5612dd8c3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.688+0000 7feb0392b740 1 --2- 172.21.15.67:0/1668915823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4dd1b0 0x5612dd8c30d0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.688+0000 7feb0392b740 1 --2- 172.21.15.67:0/1668915823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4c49f0 0x5612dd8a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.688+0000 7feb0392b740 1 -- 172.21.15.67:0/1668915823 >> 172.21.15.67:0/1668915823 conn(0x5612dd4fa460 msgr2=0x5612dd8a8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.688+0000 7feb0392b740 1 -- 172.21.15.67:0/1668915823 shutdown_connections 2024-07-31T15:11:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.688+0000 7feb0392b740 1 -- 172.21.15.67:0/1668915823 wait complete. 2024-07-31T15:11:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.688+0000 7feb0392b740 1 -- start start 2024-07-31T15:11:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.688+0000 7feb0392b740 1 Processor -- start 2024-07-31T15:11:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feb0392b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4c49f0 0x5612dd8cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feb0392b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4dd1b0 0x5612dd8c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feb0392b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feb0392b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feaf81b3e90 con 0x5612dd4e5390 2024-07-31T15:11:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feb0392b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feaf81b37c0 con 0x5612dd4dd1b0 2024-07-31T15:11:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feb0392b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feaf81b0a20 con 0x5612dd4c49f0 2024-07-31T15:11:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feb0392b740 10 client.0 _ll_register_callbacks cb 0x7feae810a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:11:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feaf67fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4dd1b0 0x5612dd8c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37394/0 (socket says 172.21.15.67:37394) 2024-07-31T15:11:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feaf77fe640 1 -- 172.21.15.67:0/3480366872 learned_addr learned my addr 172.21.15.67:0/3480366872 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:11:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feaf67fc640 1 --2- 172.21.15.67:0/3480366872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4dd1b0 0x5612dd8c7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:53798/0 (socket says 172.21.15.67:53798) 2024-07-31T15:11:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feaf6ffd640 1 --2- 172.21.15.67:0/3480366872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4c49f0 0x5612dd8cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feaf77fe640 1 -- 172.21.15.67:0/3480366872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4c49f0 msgr2=0x5612dd8cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feaf77fe640 1 --2- 172.21.15.67:0/3480366872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4c49f0 0x5612dd8cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feaf77fe640 1 -- 172.21.15.67:0/3480366872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4dd1b0 msgr2=0x5612dd8c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:11:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feaf77fe640 1 --2- 172.21.15.67:0/3480366872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4dd1b0 0x5612dd8c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.689+0000 7feaf77fe640 1 -- 172.21.15.67:0/3480366872 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612dd8c88a0 con 0x5612dd4e5390 2024-07-31T15:11:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.690+0000 7feaf77fe640 1 --2- 172.21.15.67:0/3480366872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8c81b0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7feaf0011130 tx=0x7feaf0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40db88529638e879 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:11:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.690+0000 7feadaff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:11:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.690+0000 7feadaff5640 1 -- 172.21.15.67:0/3480366872 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf002f020 con 0x5612dd4e5390 2024-07-31T15:11:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.690+0000 7feadaff5640 1 -- 172.21.15.67:0/3480366872 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7feaf002b680 con 0x5612dd4e5390 2024-07-31T15:11:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.690+0000 7feb0392b740 1 -- 172.21.15.67:0/3480366872 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaec003b70 con 0x5612dd4e5390 2024-07-31T15:11:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.690+0000 7feadaff5640 1 -- 172.21.15.67:0/3480366872 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7feaf003a050 con 0x5612dd4e5390 2024-07-31T15:11:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.690+0000 7feb0392b740 10 client.19893 fetch_fsmap learned FSMap version 66 2024-07-31T15:11:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.690+0000 7feb0392b740 1 -- 172.21.15.67:0/3480366872 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5612dd931590 con 0x5612dd4e5390 2024-07-31T15:11:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.690+0000 7feadaff5640 1 -- 172.21.15.67:0/3480366872 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaec003b70 con 0x5612dd4e5390 2024-07-31T15:11:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.691+0000 7feadaff5640 1 -- 172.21.15.67:0/3480366872 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7feaf002a970 con 0x5612dd4e5390 2024-07-31T15:11:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.691+0000 7feb0392b740 10 client.19893 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:11:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.691+0000 7feb0392b740 10 client.19893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:11:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.691+0000 7feb0392b740 20 client.19893 populate_metadata read hostname 'smithi067' 2024-07-31T15:11:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.691+0000 7feb0392b740 1 --2- 172.21.15.67:0/3480366872 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5612dd506070 0x5612dd933e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:11:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.691+0000 7feb0392b740 4 client.19893 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:11:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.691+0000 7feb0392b740 1 -- 172.21.15.67:0/3480366872 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612dd81f5a0 con 0x5612dd506070 2024-07-31T15:11:59.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.691+0000 7feaf6ffd640 1 --2- 172.21.15.67:0/3480366872 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5612dd506070 0x5612dd933e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:11:59.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.691+0000 7feaf6ffd640 1 --2- 172.21.15.67:0/3480366872 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5612dd506070 0x5612dd933e00 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=546d0a71b3deb827 server_cookie=998f5bfb3f84b242 in_seq=0 out_seq=0 2024-07-31T15:11:59.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.691+0000 7feadaff5640 10 client.19893 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:59.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feadaff5640 1 -- 172.21.15.67:0/3480366872 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612dd81f5a0 con 0x5612dd506070 2024-07-31T15:11:59.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feadaff5640 10 client.19893 handle_command_reply: tid=0 2024-07-31T15:11:59.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feb0392b740 1 client.19893 shutdown 2024-07-31T15:11:59.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feb0392b740 1 -- 172.21.15.67:0/3480366872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 msgr2=0x5612dd8c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:59.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feb0392b740 1 --2- 172.21.15.67:0/3480366872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8c81b0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7feaf0011130 tx=0x7feaf0004ab0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feb0392b740 1 -- 172.21.15.67:0/3480366872 shutdown_connections 2024-07-31T15:11:59.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feb0392b740 1 --2- 172.21.15.67:0/3480366872 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5612dd506070 0x5612dd933e00 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feb0392b740 1 --2- 172.21.15.67:0/3480366872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5612dd4e5390 0x5612dd8c81b0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feb0392b740 1 --2- 172.21.15.67:0/3480366872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612dd4dd1b0 0x5612dd8c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feb0392b740 1 --2- 172.21.15.67:0/3480366872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612dd4c49f0 0x5612dd8cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feb0392b740 1 -- 172.21.15.67:0/3480366872 >> 172.21.15.67:0/3480366872 conn(0x5612dd4fa460 msgr2=0x5612dd8ac6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:59.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feadaff5640 0 client.19893 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:11:59.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feb0392b740 1 -- 172.21.15.67:0/3480366872 shutdown_connections 2024-07-31T15:11:59.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feb0392b740 1 -- 172.21.15.67:0/3480366872 wait complete. 2024-07-31T15:11:59.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feb0392b740 20 client.19893 trim_cache size 0 max 16384 2024-07-31T15:11:59.953 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:11:59.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feafd3da640 1 -- 172.21.15.67:0/990980439 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feae00b3d40 msgr2=0x7feae00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:59.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feafd3da640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feae00b3d40 0x7feae00b6250 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7feaf00116f0 tx=0x7feaf0004a40 comp rx=0 tx=0).stop 2024-07-31T15:11:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feafd3da640 1 -- 172.21.15.67:0/990980439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf81043e0 msgr2=0x7feaf8102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:11:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.950+0000 7feafd3da640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf81043e0 0x7feaf8102120 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7feae8006010 tx=0x7feae8004060 comp rx=0 tx=0).stop 2024-07-31T15:11:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.951+0000 7feafd3da640 1 -- 172.21.15.67:0/990980439 shutdown_connections 2024-07-31T15:11:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.951+0000 7feafd3da640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7feae00b3d40 0x7feae00b6250 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.951+0000 7feafd3da640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feaf810d840 0x7feaf8102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.951+0000 7feafd3da640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf8104d30 0x7feaf8102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.951+0000 7feafd3da640 1 --2- 172.21.15.67:0/990980439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feaf81043e0 0x7feaf8102120 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:11:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.951+0000 7feafd3da640 1 -- 172.21.15.67:0/990980439 >> 172.21.15.67:0/990980439 conn(0x7feaf8079160 msgr2=0x7feaf81002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:11:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.951+0000 7feafd3da640 1 -- 172.21.15.67:0/990980439 shutdown_connections 2024-07-31T15:11:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:11:59.951+0000 7feafd3da640 1 -- 172.21.15.67:0/990980439 wait complete. 2024-07-31T15:12:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f2081ffb640 1 -- 172.21.15.67:0/1119405632 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55633b253da0 con 0x55633aed21c0 2024-07-31T15:12:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f2081ffb640 10 client.19875 handle_command_reply: tid=0 2024-07-31T15:12:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f20a7188740 1 client.19875 shutdown 2024-07-31T15:12:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f20a7188740 1 -- 172.21.15.67:0/1119405632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af3e1b0 msgr2=0x55633b364f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f20a7188740 1 --2- 172.21.15.67:0/1119405632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af3e1b0 0x55633b364f30 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2090000f80 tx=0x7f2090009bd0 comp rx=0 tx=0).stop 2024-07-31T15:12:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f20a7188740 1 -- 172.21.15.67:0/1119405632 shutdown_connections 2024-07-31T15:12:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f20a7188740 1 --2- 172.21.15.67:0/1119405632 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55633aed21c0 0x55633b368c30 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f20a7188740 1 --2- 172.21.15.67:0/1119405632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55633af3e1b0 0x55633b364f30 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f20a7188740 1 --2- 172.21.15.67:0/1119405632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55633af1d4d0 0x55633b2f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f20a7188740 1 --2- 172.21.15.67:0/1119405632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55633af152f0 0x55633b2f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f2081ffb640 0 client.19875 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f20a7188740 1 -- 172.21.15.67:0/1119405632 >> 172.21.15.67:0/1119405632 conn(0x55633af32300 msgr2=0x55633b2e5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f20a7188740 1 -- 172.21.15.67:0/1119405632 shutdown_connections 2024-07-31T15:12:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f20a7188740 1 -- 172.21.15.67:0/1119405632 wait complete. 2024-07-31T15:12:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.103+0000 7f20a7188740 20 client.19875 trim_cache size 0 max 16384 2024-07-31T15:12:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:12:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:12:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:12:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:12:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.104+0000 7f20a0292640 1 -- 172.21.15.67:0/1029814168 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f20840b3d40 msgr2=0x7f20840b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.104+0000 7f20a0292640 1 --2- 172.21.15.67:0/1029814168 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f20840b3d40 0x7f20840b6250 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2094031440 tx=0x7f2094009560 comp rx=0 tx=0).stop 2024-07-31T15:12:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.104+0000 7f20a0292640 1 -- 172.21.15.67:0/1029814168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2098075b30 msgr2=0x7f209819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.104+0000 7f20a0292640 1 --2- 172.21.15.67:0/1029814168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2098075b30 0x7f209819c680 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f208c00e9c0 tx=0x7f208c00ee80 comp rx=0 tx=0).stop 2024-07-31T15:12:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.104+0000 7f20a0292640 1 -- 172.21.15.67:0/1029814168 shutdown_connections 2024-07-31T15:12:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.104+0000 7f20a0292640 1 --2- 172.21.15.67:0/1029814168 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f20840b3d40 0x7f20840b6250 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.104+0000 7f20a0292640 1 --2- 172.21.15.67:0/1029814168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209810fa70 0x7f20981a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.104+0000 7f20a0292640 1 --2- 172.21.15.67:0/1029814168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2098075b30 0x7f209819c680 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.104+0000 7f20a0292640 1 --2- 172.21.15.67:0/1029814168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20980751e0 0x7f209819c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.104+0000 7f20a0292640 1 -- 172.21.15.67:0/1029814168 >> 172.21.15.67:0/1029814168 conn(0x7f20980fef80 msgr2=0x7f20981095f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:01.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.104+0000 7f20a0292640 1 -- 172.21.15.67:0/1029814168 shutdown_connections 2024-07-31T15:12:01.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:01.104+0000 7f20a0292640 1 -- 172.21.15.67:0/1029814168 wait complete. 2024-07-31T15:12:01.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:01 smithi073 ceph-mon[36714]: pgmap v1068: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 505 KiB/s wr, 118 op/s 2024-07-31T15:12:01.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:01 smithi077 ceph-mon[37327]: pgmap v1068: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 505 KiB/s wr, 118 op/s 2024-07-31T15:12:01.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:01 smithi067 ceph-mon[32750]: pgmap v1068: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 505 KiB/s wr, 118 op/s 2024-07-31T15:12:02.247 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T15:12:02.247 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T15:12:02.535 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:12:03.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:03 smithi067 ceph-mon[32750]: pgmap v1069: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 572 KiB/s wr, 124 op/s 2024-07-31T15:12:03.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:03 smithi073 ceph-mon[36714]: pgmap v1069: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 572 KiB/s wr, 124 op/s 2024-07-31T15:12:03.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:03 smithi077 ceph-mon[37327]: pgmap v1069: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 572 KiB/s wr, 124 op/s 2024-07-31T15:12:04.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.351+0000 7fd4d2be7640 1 -- 172.21.15.67:0/1498296236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc108760 msgr2=0x7fd4cc10ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:04.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.351+0000 7fd4d2be7640 1 --2- 172.21.15.67:0/1498296236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc108760 0x7fd4cc10ef90 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009ca0 tx=0x7fd4c402a8c0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.351+0000 7fd4d2be7640 1 -- 172.21.15.67:0/1498296236 shutdown_connections 2024-07-31T15:12:04.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.351+0000 7fd4d2be7640 1 --2- 172.21.15.67:0/1498296236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc108760 0x7fd4cc10ef90 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.351+0000 7fd4d2be7640 1 --2- 172.21.15.67:0/1498296236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4cc107e10 0x7fd4cc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.351+0000 7fd4d2be7640 1 --2- 172.21.15.67:0/1498296236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4cc1024d0 0x7fd4cc1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.351+0000 7fd4d2be7640 1 -- 172.21.15.67:0/1498296236 >> 172.21.15.67:0/1498296236 conn(0x7fd4cc079190 msgr2=0x7fd4cc0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:04.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.351+0000 7fd4d2be7640 1 -- 172.21.15.67:0/1498296236 shutdown_connections 2024-07-31T15:12:04.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.351+0000 7fd4d2be7640 1 -- 172.21.15.67:0/1498296236 wait complete. 2024-07-31T15:12:04.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.352+0000 7fd4d2be7640 1 Processor -- start 2024-07-31T15:12:04.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.352+0000 7fd4d2be7640 1 -- start start 2024-07-31T15:12:04.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.352+0000 7fd4d2be7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4cc1024d0 0x7fd4cc198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.352+0000 7fd4d2be7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4cc107e10 0x7fd4cc1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.352+0000 7fd4d2be7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc108760 0x7fd4cc19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.352+0000 7fd4d2be7640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1123c0 con 0x7fd4cc108760 2024-07-31T15:12:04.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.352+0000 7fd4d2be7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc112240 con 0x7fd4cc1024d0 2024-07-31T15:12:04.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.352+0000 7fd4d2be7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc112540 con 0x7fd4cc107e10 2024-07-31T15:12:04.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.352+0000 7fd4d1184640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc108760 0x7fd4cc19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.352+0000 7fd4d0983640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4cc1024d0 0x7fd4cc198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.352+0000 7fd4d1184640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc108760 0x7fd4cc19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43292/0 (socket says 172.21.15.67:43292) 2024-07-31T15:12:04.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.352+0000 7fd4d1184640 1 -- 172.21.15.67:0/2938528431 learned_addr learned my addr 172.21.15.67:0/2938528431 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:04.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.352+0000 7fd4cbfff640 1 --2- 172.21.15.67:0/2938528431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4cc107e10 0x7fd4cc1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.353+0000 7fd4d1184640 1 -- 172.21.15.67:0/2938528431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4cc107e10 msgr2=0x7fd4cc1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:04.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.353+0000 7fd4d1184640 1 --2- 172.21.15.67:0/2938528431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4cc107e10 0x7fd4cc1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.353+0000 7fd4d1184640 1 -- 172.21.15.67:0/2938528431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4cc1024d0 msgr2=0x7fd4cc198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:04.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.353+0000 7fd4d1184640 1 --2- 172.21.15.67:0/2938528431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4cc1024d0 0x7fd4cc198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.353+0000 7fd4d1184640 1 -- 172.21.15.67:0/2938528431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc19edc0 con 0x7fd4cc108760 2024-07-31T15:12:04.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.353+0000 7fd4cbfff640 1 --2- 172.21.15.67:0/2938528431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4cc107e10 0x7fd4cc1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:12:04.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.353+0000 7fd4d1184640 1 --2- 172.21.15.67:0/2938528431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc108760 0x7fd4cc19e690 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40098d0 tx=0x7fd4c40317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:04.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.353+0000 7fd4c9ffb640 1 -- 172.21.15.67:0/2938528431 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4003b70 con 0x7fd4cc108760 2024-07-31T15:12:04.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.353+0000 7fd4d2be7640 1 -- 172.21.15.67:0/2938528431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4cc19f050 con 0x7fd4cc108760 2024-07-31T15:12:04.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.353+0000 7fd4c9ffb640 1 -- 172.21.15.67:0/2938528431 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c4003d10 con 0x7fd4cc108760 2024-07-31T15:12:04.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.353+0000 7fd4c9ffb640 1 -- 172.21.15.67:0/2938528431 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c402fa20 con 0x7fd4cc108760 2024-07-31T15:12:04.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.353+0000 7fd4d2be7640 1 -- 172.21.15.67:0/2938528431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4cc1a5810 con 0x7fd4cc108760 2024-07-31T15:12:04.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.356+0000 7fd4d9138740 1 Processor -- start 2024-07-31T15:12:04.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.356+0000 7fd4c9ffb640 1 -- 172.21.15.67:0/2938528431 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd4c402fc40 con 0x7fd4cc108760 2024-07-31T15:12:04.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.356+0000 7fd4d9138740 1 -- start start 2024-07-31T15:12:04.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.356+0000 7fd4d9138740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.356+0000 7fd4d9138740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.356+0000 7fd4d9138740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 0x5622606c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.356+0000 7fd4d9138740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5622605ad2f0 con 0x5622602f7490 2024-07-31T15:12:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.356+0000 7fd4d9138740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56226063ba80 con 0x562260320350 2024-07-31T15:12:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.356+0000 7fd4d9138740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56226062af80 con 0x5622602ff670 2024-07-31T15:12:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.356+0000 7fd4d0983640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.356+0000 7fd4cbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.356+0000 7fd4d0983640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606c58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43304/0 (socket says 172.21.15.67:43304) 2024-07-31T15:12:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.356+0000 7fd4d0983640 1 -- 172.21.15.67:0/2762444576 learned_addr learned my addr 172.21.15.67:0/2762444576 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.357+0000 7fd4cbfff640 1 --2- 172.21.15.67:0/2762444576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 0x5622606c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.357+0000 7fd4d0983640 1 -- 172.21.15.67:0/2762444576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 msgr2=0x5622606c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.357+0000 7fd4d0983640 1 --2- 172.21.15.67:0/2762444576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 0x5622606c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.357+0000 7fd4d0983640 1 -- 172.21.15.67:0/2762444576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 msgr2=0x5622606c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.357+0000 7fd4d0983640 1 --2- 172.21.15.67:0/2762444576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606c53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.357+0000 7fd4d0983640 1 -- 172.21.15.67:0/2762444576 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56226038b930 con 0x5622602f7490 2024-07-31T15:12:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.357+0000 7fd4c9ffb640 1 --2- 172.21.15.67:0/2938528431 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd4b40b3d40 0x7fd4b40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.357+0000 7fd4cbfff640 1 --2- 172.21.15.67:0/2938528431 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd4b40b3d40 0x7fd4b40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.357+0000 7fd4d0983640 1 --2- 172.21.15.67:0/2762444576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606c58f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000bd10 tx=0x7fd4c00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61e9af0ae24929b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.357+0000 7fd4c9ffb640 1 -- 172.21.15.67:0/2938528431 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fd4c4105770 con 0x7fd4cc108760 2024-07-31T15:12:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.357+0000 7fd4b2ffd640 1 -- 172.21.15.67:0/2762444576 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0004030 con 0x5622602f7490 2024-07-31T15:12:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.357+0000 7fd4b2ffd640 1 -- 172.21.15.67:0/2762444576 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c00035f0 con 0x5622602f7490 2024-07-31T15:12:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.357+0000 7fd4b2ffd640 1 -- 172.21.15.67:0/2762444576 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0011680 con 0x5622602f7490 2024-07-31T15:12:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.358+0000 7fd4d9138740 1 -- 172.21.15.67:0/2762444576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 msgr2=0x5622606c58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.358+0000 7fd4cbfff640 1 --2- 172.21.15.67:0/2938528431 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd4b40b3d40 0x7fd4b40b6250 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc1058d0 tx=0x7fd4bc009600 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.358+0000 7fd4d9138740 1 --2- 172.21.15.67:0/2762444576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606c58f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000bd10 tx=0x7fd4c00107a0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.358+0000 7fd4d9138740 1 -- 172.21.15.67:0/2762444576 shutdown_connections 2024-07-31T15:12:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.358+0000 7fd4d9138740 1 --2- 172.21.15.67:0/2762444576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 0x5622606c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.358+0000 7fd4d9138740 1 --2- 172.21.15.67:0/2762444576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606c58f0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.358+0000 7fd4d9138740 1 --2- 172.21.15.67:0/2762444576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.358+0000 7fd4d9138740 1 -- 172.21.15.67:0/2762444576 >> 172.21.15.67:0/2762444576 conn(0x562260314740 msgr2=0x5622606c32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:04.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.358+0000 7fd4d9138740 1 -- 172.21.15.67:0/2762444576 shutdown_connections 2024-07-31T15:12:04.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.358+0000 7fd4d9138740 1 -- 172.21.15.67:0/2762444576 wait complete. 2024-07-31T15:12:04.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.359+0000 7fd4d9138740 1 -- start start 2024-07-31T15:12:04.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.359+0000 7fd4d9138740 1 Processor -- start 2024-07-31T15:12:04.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.359+0000 7fd4d9138740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602f7490 0x5622606d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.359+0000 7fd4d9138740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602ff670 0x5622606d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.359+0000 7fd4d9138740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606def40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.359+0000 7fd4d9138740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5622605cd4f0 con 0x5622602ff670 2024-07-31T15:12:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.359+0000 7fd4d9138740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56226062efc0 con 0x562260320350 2024-07-31T15:12:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.359+0000 7fd4d9138740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56226048e890 con 0x5622602f7490 2024-07-31T15:12:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.359+0000 7fd4d9138740 10 client.0 _ll_register_callbacks cb 0x5622605d7500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.359+0000 7fd4cbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602ff670 0x5622606d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.359+0000 7fd4cbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602ff670 0x5622606d1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43306/0 (socket says 172.21.15.67:43306) 2024-07-31T15:12:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.359+0000 7fd4cbfff640 1 -- 172.21.15.67:0/2874082025 learned_addr learned my addr 172.21.15.67:0/2874082025 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.359+0000 7fd4d1184640 1 --2- 172.21.15.67:0/2874082025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606def40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.360+0000 7fd4d0983640 1 --2- 172.21.15.67:0/2874082025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602f7490 0x5622606d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.360+0000 7fd4cbfff640 1 -- 172.21.15.67:0/2874082025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602f7490 msgr2=0x5622606d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.360+0000 7fd4cbfff640 1 --2- 172.21.15.67:0/2874082025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602f7490 0x5622606d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.360+0000 7fd4cbfff640 1 -- 172.21.15.67:0/2874082025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 msgr2=0x5622606def40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.360+0000 7fd4cbfff640 1 --2- 172.21.15.67:0/2874082025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606def40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.360+0000 7fd4cbfff640 1 -- 172.21.15.67:0/2874082025 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622605dcc50 con 0x5622602ff670 2024-07-31T15:12:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.360+0000 7fd4d1184640 1 --2- 172.21.15.67:0/2874082025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606def40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:12:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.360+0000 7fd4cbfff640 1 --2- 172.21.15.67:0/2874082025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602ff670 0x5622606d1500 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc012b20 tx=0x7fd4bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a565ba12ddb3263 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.360+0000 7fd4abfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:12:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.360+0000 7fd4abfff640 1 -- 172.21.15.67:0/2874082025 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0239b0 con 0x5622602ff670 2024-07-31T15:12:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.360+0000 7fd4abfff640 1 -- 172.21.15.67:0/2874082025 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fd4bc023b50 con 0x5622602ff670 2024-07-31T15:12:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.360+0000 7fd4d9138740 1 -- 172.21.15.67:0/2874082025 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c00035f0 con 0x5622602ff670 2024-07-31T15:12:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.360+0000 7fd4abfff640 1 -- 172.21.15.67:0/2874082025 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fd4bc023d90 con 0x5622602ff670 2024-07-31T15:12:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.361+0000 7fd4d9138740 10 client.19905 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.361+0000 7fd4d9138740 1 -- 172.21.15.67:0/2874082025 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5622606e1cc0 con 0x5622602ff670 2024-07-31T15:12:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.361+0000 7fd4abfff640 1 -- 172.21.15.67:0/2874082025 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c00035f0 con 0x5622602ff670 2024-07-31T15:12:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.361+0000 7fd4abfff640 1 -- 172.21.15.67:0/2874082025 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fd4bc02a070 con 0x5622602ff670 2024-07-31T15:12:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.361+0000 7fd4d9138740 10 client.19905 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.361+0000 7fd4d9138740 10 client.19905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:12:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.361+0000 7fd4d9138740 20 client.19905 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.361+0000 7fd4d9138740 1 --2- 172.21.15.67:0/2874082025 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5622602decd0 0x5622606e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.361+0000 7fd4d9138740 4 client.19905 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:12:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.361+0000 7fd4d9138740 1 -- 172.21.15.67:0/2874082025 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56226063d9d0 con 0x5622602decd0 2024-07-31T15:12:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.361+0000 7fd4d0983640 1 --2- 172.21.15.67:0/2874082025 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5622602decd0 0x5622606e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.362+0000 7fd4d0983640 1 --2- 172.21.15.67:0/2874082025 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5622602decd0 0x5622606e52c0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d64ca6e62b939e70 server_cookie=f736b73e7599c348 in_seq=0 out_seq=0 2024-07-31T15:12:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.362+0000 7fd4abfff640 10 client.19905 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.363+0000 7fd4abfff640 1 -- 172.21.15.67:0/2874082025 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56226063d9d0 con 0x5622602decd0 2024-07-31T15:12:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.363+0000 7fd4abfff640 10 client.19905 handle_command_reply: tid=0 2024-07-31T15:12:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.363+0000 7fd4d9138740 1 client.19905 shutdown 2024-07-31T15:12:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.364+0000 7fd4d9138740 1 -- 172.21.15.67:0/2874082025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602ff670 msgr2=0x5622606d1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.364+0000 7fd4d9138740 1 --2- 172.21.15.67:0/2874082025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602ff670 0x5622606d1500 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc012b20 tx=0x7fd4bc004730 comp rx=0 tx=0).stop 2024-07-31T15:12:04.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.364+0000 7fd4d9138740 1 -- 172.21.15.67:0/2874082025 shutdown_connections 2024-07-31T15:12:04.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.364+0000 7fd4d9138740 1 --2- 172.21.15.67:0/2874082025 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5622602decd0 0x5622606e52c0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.364+0000 7fd4d9138740 1 --2- 172.21.15.67:0/2874082025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606def40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.364+0000 7fd4d9138740 1 --2- 172.21.15.67:0/2874082025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602ff670 0x5622606d1500 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.364+0000 7fd4abfff640 0 client.19905 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:04.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.364+0000 7fd4d9138740 1 --2- 172.21.15.67:0/2874082025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602f7490 0x5622606d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.364+0000 7fd4d9138740 1 -- 172.21.15.67:0/2874082025 >> 172.21.15.67:0/2874082025 conn(0x562260314740 msgr2=0x5622606c8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:04.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.364+0000 7fd4d9138740 1 -- 172.21.15.67:0/2874082025 shutdown_connections 2024-07-31T15:12:04.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.364+0000 7fd4d9138740 1 -- 172.21.15.67:0/2874082025 wait complete. 2024-07-31T15:12:04.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.364+0000 7fd4d9138740 20 client.19905 trim_cache size 0 max 16384 2024-07-31T15:12:04.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d9138740 1 Processor -- start 2024-07-31T15:12:04.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d9138740 1 -- start start 2024-07-31T15:12:04.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d9138740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606c3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d9138740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 0x5622606eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d9138740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606eb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d9138740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1ae610 con 0x5622602f7490 2024-07-31T15:12:04.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d9138740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1b0150 con 0x562260320350 2024-07-31T15:12:04.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d9138740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1aece0 con 0x5622602ff670 2024-07-31T15:12:04.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d0983640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606c3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d0983640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606c3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43310/0 (socket says 172.21.15.67:43310) 2024-07-31T15:12:04.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d0983640 1 -- 172.21.15.67:0/2495410847 learned_addr learned my addr 172.21.15.67:0/2495410847 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:04.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d1184640 1 --2- 172.21.15.67:0/2495410847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606eb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4cbfff640 1 --2- 172.21.15.67:0/2495410847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 0x5622606eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d0983640 1 -- 172.21.15.67:0/2495410847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 msgr2=0x5622606eaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:04.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d0983640 1 --2- 172.21.15.67:0/2495410847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 0x5622606eaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d0983640 1 -- 172.21.15.67:0/2495410847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 msgr2=0x5622606eb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:04.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d0983640 1 --2- 172.21.15.67:0/2495410847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606eb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.383+0000 7fd4d0983640 1 -- 172.21.15.67:0/2495410847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622606d2080 con 0x5622602f7490 2024-07-31T15:12:04.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.384+0000 7fd4d0983640 1 --2- 172.21.15.67:0/2495410847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606c3220 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x5622606c4030 tx=0x7fd4c000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1105b26cd62b536 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:04.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.384+0000 7fd4b27fc640 1 -- 172.21.15.67:0/2495410847 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0005240 con 0x5622602f7490 2024-07-31T15:12:04.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.384+0000 7fd4b27fc640 1 -- 172.21.15.67:0/2495410847 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c00053e0 con 0x5622602f7490 2024-07-31T15:12:04.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.384+0000 7fd4b27fc640 1 -- 172.21.15.67:0/2495410847 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c00116a0 con 0x5622602f7490 2024-07-31T15:12:04.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.385+0000 7fd4d9138740 1 -- 172.21.15.67:0/2495410847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 msgr2=0x5622606c3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.385+0000 7fd4d9138740 1 --2- 172.21.15.67:0/2495410847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606c3220 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x5622606c4030 tx=0x7fd4c000b840 comp rx=0 tx=0).stop 2024-07-31T15:12:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.385+0000 7fd4d9138740 1 -- 172.21.15.67:0/2495410847 shutdown_connections 2024-07-31T15:12:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.385+0000 7fd4d9138740 1 --2- 172.21.15.67:0/2495410847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606eb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.385+0000 7fd4d9138740 1 --2- 172.21.15.67:0/2495410847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 0x5622606eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.385+0000 7fd4d9138740 1 --2- 172.21.15.67:0/2495410847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606c3220 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.385+0000 7fd4d9138740 1 -- 172.21.15.67:0/2495410847 >> 172.21.15.67:0/2495410847 conn(0x562260314740 msgr2=0x5622606c2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.385+0000 7fd4d9138740 1 -- 172.21.15.67:0/2495410847 shutdown_connections 2024-07-31T15:12:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.385+0000 7fd4d9138740 1 -- 172.21.15.67:0/2495410847 wait complete. 2024-07-31T15:12:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.385+0000 7fd4d9138740 1 -- start start 2024-07-31T15:12:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.385+0000 7fd4d9138740 1 Processor -- start 2024-07-31T15:12:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d9138740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606db280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d9138740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 0x5622606d6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d9138740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606d6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d9138740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1afa80 con 0x5622602f7490 2024-07-31T15:12:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d9138740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1af3b0 con 0x562260320350 2024-07-31T15:12:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d9138740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1ac610 con 0x5622602ff670 2024-07-31T15:12:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d9138740 10 client.0 _ll_register_callbacks cb 0x7fd4bc012980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d0983640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606db280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d1184640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606d6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d0983640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606db280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43316/0 (socket says 172.21.15.67:43316) 2024-07-31T15:12:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d0983640 1 -- 172.21.15.67:0/24853231 learned_addr learned my addr 172.21.15.67:0/24853231 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4cbfff640 1 --2- 172.21.15.67:0/24853231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 0x5622606d6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d0983640 1 -- 172.21.15.67:0/24853231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 msgr2=0x5622606d6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d0983640 1 --2- 172.21.15.67:0/24853231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 0x5622606d6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d0983640 1 -- 172.21.15.67:0/24853231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 msgr2=0x5622606d6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d0983640 1 --2- 172.21.15.67:0/24853231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606d6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d0983640 1 -- 172.21.15.67:0/24853231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622606d71b0 con 0x5622602f7490 2024-07-31T15:12:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4cbfff640 1 --2- 172.21.15.67:0/24853231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 0x5622606d6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:12:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d1184640 1 --2- 172.21.15.67:0/24853231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606d6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:12:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4d0983640 1 --2- 172.21.15.67:0/24853231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606db280 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0003df0 tx=0x7fd4c0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3397865973c790d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.386+0000 7fd4abfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:12:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.387+0000 7fd4abfff640 1 -- 172.21.15.67:0/24853231 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0037070 con 0x5622602f7490 2024-07-31T15:12:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.387+0000 7fd4abfff640 1 -- 172.21.15.67:0/24853231 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9395+0+0 (secure 0 0 0) 0x7fd4c00037f0 con 0x5622602f7490 2024-07-31T15:12:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.387+0000 7fd4d9138740 1 -- 172.21.15.67:0/24853231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c00053e0 con 0x5622602f7490 2024-07-31T15:12:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.387+0000 7fd4abfff640 1 -- 172.21.15.67:0/24853231 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fd4c0005590 con 0x5622602f7490 2024-07-31T15:12:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.387+0000 7fd4d9138740 10 client.19911 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.387+0000 7fd4d9138740 1 -- 172.21.15.67:0/24853231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56226074d710 con 0x5622602f7490 2024-07-31T15:12:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.387+0000 7fd4abfff640 1 -- 172.21.15.67:0/24853231 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c00053e0 con 0x5622602f7490 2024-07-31T15:12:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.387+0000 7fd4abfff640 1 -- 172.21.15.67:0/24853231 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fd4c0011ab0 con 0x5622602f7490 2024-07-31T15:12:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.387+0000 7fd4d9138740 10 client.19911 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.387+0000 7fd4d9138740 10 client.19911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:12:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.387+0000 7fd4d9138740 20 client.19911 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.387+0000 7fd4d9138740 1 --2- 172.21.15.67:0/24853231 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5622602b4080 0x56226074fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.387+0000 7fd4d9138740 4 client.19911 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T15:12:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.387+0000 7fd4d9138740 1 -- 172.21.15.67:0/24853231 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562260639470 con 0x5622602b4080 2024-07-31T15:12:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.388+0000 7fd4cbfff640 1 --2- 172.21.15.67:0/24853231 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5622602b4080 0x56226074fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.388+0000 7fd4abfff640 10 client.19911 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:04.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.388+0000 7fd4cbfff640 1 --2- 172.21.15.67:0/24853231 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5622602b4080 0x56226074fc80 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bb74f0e90730cd6 server_cookie=e83c3421f01879e in_seq=0 out_seq=0 2024-07-31T15:12:04.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4abfff640 1 -- 172.21.15.67:0/24853231 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562260639470 con 0x5622602b4080 2024-07-31T15:12:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4abfff640 10 client.19911 handle_command_reply: tid=0 2024-07-31T15:12:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4d9138740 1 client.19911 shutdown 2024-07-31T15:12:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4d9138740 1 -- 172.21.15.67:0/24853231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 msgr2=0x5622606db280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4d9138740 1 --2- 172.21.15.67:0/24853231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606db280 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0003df0 tx=0x7fd4c0010d50 comp rx=0 tx=0).stop 2024-07-31T15:12:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4d9138740 1 -- 172.21.15.67:0/24853231 shutdown_connections 2024-07-31T15:12:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4d9138740 1 --2- 172.21.15.67:0/24853231 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5622602b4080 0x56226074fc80 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4d9138740 1 --2- 172.21.15.67:0/24853231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562260320350 0x5622606d6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4d9138740 1 --2- 172.21.15.67:0/24853231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622602ff670 0x5622606d6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4abfff640 0 client.19911 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4d9138740 1 --2- 172.21.15.67:0/24853231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5622602f7490 0x5622606db280 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4d9138740 1 -- 172.21.15.67:0/24853231 >> 172.21.15.67:0/24853231 conn(0x562260314740 msgr2=0x5622606c18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4d9138740 1 -- 172.21.15.67:0/24853231 shutdown_connections 2024-07-31T15:12:04.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4d9138740 1 -- 172.21.15.67:0/24853231 wait complete. 2024-07-31T15:12:04.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.390+0000 7fd4d9138740 20 client.19911 trim_cache size 0 max 16384 2024-07-31T15:12:04.396 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:12:04.396 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T15:12:04.396 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "1b31e70a-1dca-4e81-8b70-471a68a8f057", 2024-07-31T15:12:04.396 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T15:12:04.396 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:12:04.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.391+0000 7fd4d2be7640 1 -- 172.21.15.67:0/2938528431 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd4b40b3d40 msgr2=0x7fd4b40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:04.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.391+0000 7fd4d2be7640 1 --2- 172.21.15.67:0/2938528431 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd4b40b3d40 0x7fd4b40b6250 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc1058d0 tx=0x7fd4bc009600 comp rx=0 tx=0).stop 2024-07-31T15:12:04.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.391+0000 7fd4d2be7640 1 -- 172.21.15.67:0/2938528431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc108760 msgr2=0x7fd4cc19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:04.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.391+0000 7fd4d2be7640 1 --2- 172.21.15.67:0/2938528431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc108760 0x7fd4cc19e690 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40098d0 tx=0x7fd4c40317b0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.391+0000 7fd4d2be7640 1 -- 172.21.15.67:0/2938528431 shutdown_connections 2024-07-31T15:12:04.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.391+0000 7fd4d2be7640 1 --2- 172.21.15.67:0/2938528431 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fd4b40b3d40 0x7fd4b40b6250 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.391+0000 7fd4d2be7640 1 --2- 172.21.15.67:0/2938528431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc108760 0x7fd4cc19e690 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.391+0000 7fd4d2be7640 1 --2- 172.21.15.67:0/2938528431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4cc107e10 0x7fd4cc1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.391+0000 7fd4d2be7640 1 --2- 172.21.15.67:0/2938528431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4cc1024d0 0x7fd4cc198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:04.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.391+0000 7fd4d2be7640 1 -- 172.21.15.67:0/2938528431 >> 172.21.15.67:0/2938528431 conn(0x7fd4cc079190 msgr2=0x7fd4cc103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:04.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.391+0000 7fd4d2be7640 1 -- 172.21.15.67:0/2938528431 shutdown_connections 2024-07-31T15:12:04.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:04.391+0000 7fd4d2be7640 1 -- 172.21.15.67:0/2938528431 wait complete. 2024-07-31T15:12:04.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:04 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:04.466+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000162a [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch/ auth{1=1,2=1,3=1,4=1} v21244 ap=3 RANDEPHEMERALPIN f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22) n(v53 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939) old_inodes=7 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:538997 nref=4)) caps={15489=pAsLsXs/-@64} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7c26ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000162a:[<0x10000000202/arch v378>,<0x10000000201/linux-6.5.11 v96>,<0x10000000006/t v68>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v57>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v60>,<0x10000000001/sv_1 v84>,<0x10000000000/qa v70>,<0x1/volumes v70>]//[]","memoryvalue":"(2)0x1000000162a:[<0x10000000202/arch v21244>,<0x10000000201/linux-6.5.11 v2681>,<0x10000000006/t v1719>,<0x10000000005/tmp v1644>,<0x100000001fb/client.0 v1615>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1577>,<0x10000000001/sv_1 v1578>,<0x10000000000/qa v1568>,<0x1/volumes v1520>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:54:49.652308+0000 23=2+21)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939)","memory_value.dirstat":"f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22)","memory_value.rstat":"n(v53 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:04.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:04 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:04.576+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c276 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802/ auth{3=1} v10568 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:29.002671+0000 14=14+0) n(v3 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:540781 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@17} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8fd23180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c276:[<0x1000000c25e/802 v68>,<0x10000000202/net v7765>,<0x10000000201/linux-6.5.11 v689>,<0x10000000006/t v498>,<0x10000000005/tmp v466>,<0x100000001fb/client.0 v452>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v438>,<0x10000000001/sv_1 v453>,<0x10000000000/qa v441>,<0x1/volumes v433>]//[]","memoryvalue":"(2)0x1000000c276:[<0x1000000c25e/802 v10568>,<0x10000000202/net v21952>,<0x10000000201/linux-6.5.11 v2681>,<0x10000000006/t v1719>,<0x10000000005/tmp v1644>,<0x100000001fb/client.0 v1615>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1577>,<0x10000000001/sv_1 v1578>,<0x10000000000/qa v1568>,<0x1/volumes v1520>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:49.293317+0000 10=10+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:29.002671+0000 14=14+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:04.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:04 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:04.585+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c9e2 [...1f,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix/ auth{2=1} v12135 ap=3 RANDEPHEMERALPIN f(v2 m2024-07-31T15:08:53.494950+0000 21=21+0) n(v5 rc2024-07-31T15:08:53.495950+0000 b445955 22=21+1) old_inodes=3 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:540797 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@15} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8591ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c9e2:[<0x1000000c25e/unix v6743>,<0x10000000202/net v8736>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000c9e2:[<0x1000000c25e/unix v12135>,<0x10000000202/net v21952>,<0x10000000201/linux-6.5.11 v2681>,<0x10000000006/t v1719>,<0x10000000005/tmp v1644>,<0x100000001fb/client.0 v1615>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1577>,<0x10000000001/sv_1 v1578>,<0x10000000000/qa v1568>,<0x1/volumes v1520>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:08:21.655599+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:53.495950+0000 b445955 22=21+1)","memory_value.dirstat":"f(v2 m2024-07-31T15:08:53.494950+0000 21=21+0)","memory_value.rstat":"n(v5 rc2024-07-31T15:08:53.495950+0000 b445955 22=21+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:04.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:04 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:04.586+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c836 [...1f,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet/ auth{4=1} v12171 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:57.081877+0000 11=11+0) n(v2 rc2024-07-31T15:08:57.083877+0000 b263808 12=11+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:540795 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@12} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8c20b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c836:[<0x1000000c25e/packet v6437>,<0x10000000202/net v8736>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000c836:[<0x1000000c25e/packet v12171>,<0x10000000202/net v21952>,<0x10000000201/linux-6.5.11 v2681>,<0x10000000006/t v1719>,<0x10000000005/tmp v1644>,<0x100000001fb/client.0 v1615>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1577>,<0x10000000001/sv_1 v1578>,<0x10000000000/qa v1568>,<0x1/volumes v1520>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:56.791162+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:57.083877+0000 b263808 12=11+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:57.081877+0000 11=11+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:08:57.083877+0000 b263808 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:04.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:04 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:04.629+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d9b4 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb/ auth{3=1} v3876 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7) n(v5 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8) old_inodes=3 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:541511 nref=3)) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce92cb2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v2020>,<0x10000000202/sound v9936>,<0x10000000201/linux-6.5.11 v830>,<0x10000000006/t v586>,<0x10000000005/tmp v553>,<0x100000001fb/client.0 v537>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v519>,<0x10000000001/sv_1 v525>,<0x10000000000/qa v522>,<0x1/volumes v503>]//[]","memoryvalue":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v3876>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2681>,<0x10000000006/t v1719>,<0x10000000005/tmp v1644>,<0x100000001fb/client.0 v1615>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1577>,<0x10000000001/sv_1 v1578>,<0x10000000000/qa v1568>,<0x1/volumes v1520>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:46.317139+0000 53=46+7)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7)","memory_value.rstat":"n(v5 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:05.203 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:12:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: pgmap v1070: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 460 KiB/s wr, 179 op/s 2024-07-31T15:12:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:05 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: pgmap v1070: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 460 KiB/s wr, 179 op/s 2024-07-31T15:12:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:05.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:05.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:05.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:05.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:05.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:05.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:05 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.612 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:12:05.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: pgmap v1070: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 460 KiB/s wr, 179 op/s 2024-07-31T15:12:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:05.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:05.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:05.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:05.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:05 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T15:12:06.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:06.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:06.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:06.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:06.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:06.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.491 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c9e2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c836 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:06.492 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:06.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:06.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.493 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:06.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.494 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:06 smithi077 ceph-mon[37327]: osdmap e131: 12 total, 12 up, 12 in 2024-07-31T15:12:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T15:12:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:06.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:06.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c9e2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c836 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:06 smithi067 ceph-mon[32750]: osdmap e131: 12 total, 12 up, 12 in 2024-07-31T15:12:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T15:12:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c9e2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c836 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:12:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:06.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:06 smithi073 ceph-mon[36714]: osdmap e131: 12 total, 12 up, 12 in 2024-07-31T15:12:07.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:07 smithi067 ceph-mon[32750]: pgmap v1072: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 544 KiB/s wr, 215 op/s 2024-07-31T15:12:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:07 smithi077 ceph-mon[37327]: pgmap v1072: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 544 KiB/s wr, 215 op/s 2024-07-31T15:12:07.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:07 smithi073 ceph-mon[36714]: pgmap v1072: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 544 KiB/s wr, 215 op/s 2024-07-31T15:12:07.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.835+0000 7f1523e61640 1 -- 172.21.15.67:0/3163107367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f151c109860 msgr2=0x7f151c110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:07.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.835+0000 7f1523e61640 1 --2- 172.21.15.67:0/3163107367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f151c109860 0x7f151c110090 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1514009be0 tx=0x7f151402a710 comp rx=0 tx=0).stop 2024-07-31T15:12:07.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.835+0000 7f1523e61640 1 -- 172.21.15.67:0/3163107367 shutdown_connections 2024-07-31T15:12:07.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.835+0000 7f1523e61640 1 --2- 172.21.15.67:0/3163107367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f151c109860 0x7f151c110090 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.835+0000 7f1523e61640 1 --2- 172.21.15.67:0/3163107367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f151c108f10 0x7f151c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.835+0000 7f1523e61640 1 --2- 172.21.15.67:0/3163107367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f151c1035d0 0x7f151c1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.835+0000 7f1523e61640 1 -- 172.21.15.67:0/3163107367 >> 172.21.15.67:0/3163107367 conn(0x7f151c0ff060 msgr2=0x7f151c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:07.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.836+0000 7f1523e61640 1 -- 172.21.15.67:0/3163107367 shutdown_connections 2024-07-31T15:12:07.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.836+0000 7f1523e61640 1 -- 172.21.15.67:0/3163107367 wait complete. 2024-07-31T15:12:07.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.837+0000 7f1523e61640 1 Processor -- start 2024-07-31T15:12:07.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.837+0000 7f1523e61640 1 -- start start 2024-07-31T15:12:07.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.837+0000 7f1523e61640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f151c1035d0 0x7f151c199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.837+0000 7f1523e61640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f151c108f10 0x7f151c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.837+0000 7f1523e61640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f151c109860 0x7f151c19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.837+0000 7f1521bfd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f151c1035d0 0x7f151c199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.837+0000 7f1521bfd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f151c1035d0 0x7f151c199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:50414/0 (socket says 172.21.15.67:50414) 2024-07-31T15:12:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.837+0000 7f15213fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f151c108f10 0x7f151c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.837+0000 7f15213fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f151c108f10 0x7f151c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:48606/0 (socket says 172.21.15.67:48606) 2024-07-31T15:12:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.838+0000 7f15223fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f151c109860 0x7f151c19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.838+0000 7f15223fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f151c109860 0x7f151c19f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43342/0 (socket says 172.21.15.67:43342) 2024-07-31T15:12:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.838+0000 7f1521bfd640 1 -- 172.21.15.67:0/4201269932 learned_addr learned my addr 172.21.15.67:0/4201269932 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.838+0000 7f1523e61640 1 -- 172.21.15.67:0/4201269932 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f151c113510 con 0x7f151c109860 2024-07-31T15:12:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.838+0000 7f1523e61640 1 -- 172.21.15.67:0/4201269932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f151c113390 con 0x7f151c1035d0 2024-07-31T15:12:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.838+0000 7f1523e61640 1 -- 172.21.15.67:0/4201269932 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f151c113690 con 0x7f151c108f10 2024-07-31T15:12:07.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.838+0000 7f15213fc640 1 -- 172.21.15.67:0/4201269932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f151c1035d0 msgr2=0x7f151c199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:07.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.838+0000 7f15213fc640 1 --2- 172.21.15.67:0/4201269932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f151c1035d0 0x7f151c199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.838+0000 7f15213fc640 1 -- 172.21.15.67:0/4201269932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f151c109860 msgr2=0x7f151c19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:07.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.838+0000 7f15213fc640 1 --2- 172.21.15.67:0/4201269932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f151c109860 0x7f151c19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.838+0000 7f15213fc640 1 -- 172.21.15.67:0/4201269932 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151c19fc00 con 0x7f151c108f10 2024-07-31T15:12:07.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.838+0000 7f15213fc640 1 --2- 172.21.15.67:0/4201269932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f151c108f10 0x7f151c19a230 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f151c06a450 tx=0x7f150c00db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:07.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.839+0000 7f1512ffd640 1 -- 172.21.15.67:0/4201269932 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c018070 con 0x7f151c108f10 2024-07-31T15:12:07.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.839+0000 7f1523e61640 1 -- 172.21.15.67:0/4201269932 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151c19fef0 con 0x7f151c108f10 2024-07-31T15:12:07.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.839+0000 7f1512ffd640 1 -- 172.21.15.67:0/4201269932 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f150c012760 con 0x7f151c108f10 2024-07-31T15:12:07.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.839+0000 7f1512ffd640 1 -- 172.21.15.67:0/4201269932 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c01c660 con 0x7f151c108f10 2024-07-31T15:12:07.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.839+0000 7f1523e61640 1 -- 172.21.15.67:0/4201269932 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151c075020 con 0x7f151c108f10 2024-07-31T15:12:07.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.843+0000 7f152a3b2740 1 Processor -- start 2024-07-31T15:12:07.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.843+0000 7f152a3b2740 1 -- start start 2024-07-31T15:12:07.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.843+0000 7f152a3b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd71070 0x56051d116640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:07.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.844+0000 7f152a3b2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd481b0 0x56051d116b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:07.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.844+0000 7f152a3b2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd50390 0x56051d1170c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:07.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.844+0000 7f152a3b2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56051d08cb60 con 0x56051cd481b0 2024-07-31T15:12:07.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.844+0000 7f152a3b2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56051d07c060 con 0x56051cd50390 2024-07-31T15:12:07.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.844+0000 7f152a3b2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56051d0800a0 con 0x56051cd71070 2024-07-31T15:12:07.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.844+0000 7f15223fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd71070 0x56051d116640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:07.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.844+0000 7f15223fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd71070 0x56051d116640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:48612/0 (socket says 172.21.15.67:48612) 2024-07-31T15:12:07.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.844+0000 7f15223fe640 1 -- 172.21.15.67:0/1644497557 learned_addr learned my addr 172.21.15.67:0/1644497557 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:07.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.844+0000 7f1512ffd640 1 -- 172.21.15.67:0/4201269932 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f150c009a20 con 0x7f151c108f10 2024-07-31T15:12:07.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.845+0000 7f15213fc640 1 --2- 172.21.15.67:0/1644497557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd50390 0x56051d1170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:07.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.845+0000 7f15223fe640 1 -- 172.21.15.67:0/1644497557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd50390 msgr2=0x56051d1170c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:07.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.845+0000 7f15223fe640 1 --2- 172.21.15.67:0/1644497557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd50390 0x56051d1170c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.845+0000 7f15223fe640 1 -- 172.21.15.67:0/1644497557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd481b0 msgr2=0x56051d116b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:12:07.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.845+0000 7f15223fe640 1 --2- 172.21.15.67:0/1644497557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd481b0 0x56051d116b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.845+0000 7f15223fe640 1 -- 172.21.15.67:0/1644497557 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56051d022b40 con 0x56051cd71070 2024-07-31T15:12:07.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.845+0000 7f15213fc640 1 --2- 172.21.15.67:0/1644497557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd50390 0x56051d1170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:12:07.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.845+0000 7f15223fe640 1 --2- 172.21.15.67:0/1644497557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd71070 0x56051d116640 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f151402ad00 tx=0x7f1514030fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac9ff53d40bdd8e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:07.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.846+0000 7f1507fff640 1 -- 172.21.15.67:0/1644497557 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514041070 con 0x56051cd71070 2024-07-31T15:12:07.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.846+0000 7f1507fff640 1 -- 172.21.15.67:0/1644497557 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1514030620 con 0x56051cd71070 2024-07-31T15:12:07.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.846+0000 7f1507fff640 1 -- 172.21.15.67:0/1644497557 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514036040 con 0x56051cd71070 2024-07-31T15:12:07.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.846+0000 7f1512ffd640 1 --2- 172.21.15.67:0/4201269932 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15080b3dc0 0x7f15080b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:07.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.846+0000 7f1512ffd640 1 -- 172.21.15.67:0/4201269932 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f150c0e7d80 con 0x7f151c108f10 2024-07-31T15:12:07.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.847+0000 7f152a3b2740 1 -- 172.21.15.67:0/1644497557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd71070 msgr2=0x56051d116640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:07.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.847+0000 7f152a3b2740 1 --2- 172.21.15.67:0/1644497557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd71070 0x56051d116640 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f151402ad00 tx=0x7f1514030fd0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.847+0000 7f1521bfd640 1 --2- 172.21.15.67:0/4201269932 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15080b3dc0 0x7f15080b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:07.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.847+0000 7f152a3b2740 1 -- 172.21.15.67:0/1644497557 shutdown_connections 2024-07-31T15:12:07.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.847+0000 7f152a3b2740 1 --2- 172.21.15.67:0/1644497557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd50390 0x56051d1170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.847+0000 7f152a3b2740 1 --2- 172.21.15.67:0/1644497557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd481b0 0x56051d116b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.847+0000 7f152a3b2740 1 --2- 172.21.15.67:0/1644497557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd71070 0x56051d116640 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.847+0000 7f152a3b2740 1 -- 172.21.15.67:0/1644497557 >> 172.21.15.67:0/1644497557 conn(0x56051cd65460 msgr2=0x56051d114530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:07.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.848+0000 7f152a3b2740 1 -- 172.21.15.67:0/1644497557 shutdown_connections 2024-07-31T15:12:07.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.850+0000 7f1521bfd640 1 --2- 172.21.15.67:0/4201269932 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15080b3dc0 0x7f15080b62d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x56051d1152c0 tx=0x7f1518005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:07.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.850+0000 7f152a3b2740 1 -- 172.21.15.67:0/1644497557 wait complete. 2024-07-31T15:12:07.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.851+0000 7f152a3b2740 1 -- start start 2024-07-31T15:12:07.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.851+0000 7f152a3b2740 1 Processor -- start 2024-07-31T15:12:07.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.852+0000 7f152a3b2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd481b0 0x56051d122250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:07.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.852+0000 7f152a3b2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd50390 0x56051d1227d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:07.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.852+0000 7f152a3b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd71070 0x56051d12e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:07.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.853+0000 7f152a3b2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56051cedf700 con 0x56051cd481b0 2024-07-31T15:12:07.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.853+0000 7f152a3b2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56051d01e990 con 0x56051cd50390 2024-07-31T15:12:07.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.853+0000 7f15223fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd71070 0x56051d12e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:07.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.853+0000 7f15223fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd71070 0x56051d12e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:48624/0 (socket says 172.21.15.67:48624) 2024-07-31T15:12:07.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.853+0000 7f15223fe640 1 -- 172.21.15.67:0/2398079465 learned_addr learned my addr 172.21.15.67:0/2398079465 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:07.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.853+0000 7f152a3b2740 1 -- 172.21.15.67:0/2398079465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56051d124dc0 con 0x56051cd71070 2024-07-31T15:12:07.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.853+0000 7f152a3b2740 10 client.0 _ll_register_callbacks cb 0x56051d0394a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:07.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.853+0000 7f1521bfd640 1 --2- 172.21.15.67:0/2398079465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd481b0 0x56051d122250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:07.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.854+0000 7f15223fe640 1 -- 172.21.15.67:0/2398079465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd50390 msgr2=0x56051d1227d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:12:07.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.854+0000 7f15223fe640 1 --2- 172.21.15.67:0/2398079465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd50390 0x56051d1227d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.854+0000 7f15223fe640 1 -- 172.21.15.67:0/2398079465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd481b0 msgr2=0x56051d122250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:07.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.854+0000 7f15223fe640 1 --2- 172.21.15.67:0/2398079465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd481b0 0x56051d122250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.854+0000 7f15223fe640 1 -- 172.21.15.67:0/2398079465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56051cddc650 con 0x56051cd71070 2024-07-31T15:12:07.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.854+0000 7f15223fe640 1 --2- 172.21.15.67:0/2398079465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd71070 0x56051d12e0b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1514009560 tx=0x7f1514009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=552330f99e3d328c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:07.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.860+0000 7f15057fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:12:07.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.860+0000 7f15057fa640 1 -- 172.21.15.67:0/2398079465 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514036070 con 0x56051cd71070 2024-07-31T15:12:07.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.860+0000 7f152a3b2740 1 -- 172.21.15.67:0/2398079465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1514030620 con 0x56051cd71070 2024-07-31T15:12:07.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.860+0000 7f15057fa640 1 -- 172.21.15.67:0/2398079465 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f1514030d00 con 0x56051cd71070 2024-07-31T15:12:07.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.860+0000 7f15057fa640 1 -- 172.21.15.67:0/2398079465 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f1514058050 con 0x56051cd71070 2024-07-31T15:12:07.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.860+0000 7f152a3b2740 10 client.29314 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:07.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.861+0000 7f152a3b2740 1 -- 172.21.15.67:0/2398079465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56051d130d90 con 0x56051cd71070 2024-07-31T15:12:07.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.864+0000 7f15057fa640 1 -- 172.21.15.67:0/2398079465 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1514030620 con 0x56051cd71070 2024-07-31T15:12:07.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.865+0000 7f15057fa640 1 -- 172.21.15.67:0/2398079465 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f1514004490 con 0x56051cd71070 2024-07-31T15:12:07.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.865+0000 7f152a3b2740 10 client.29314 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:07.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.865+0000 7f152a3b2740 10 client.29314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:12:07.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.865+0000 7f152a3b2740 20 client.29314 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:07.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.865+0000 7f152a3b2740 1 --2- 172.21.15.67:0/2398079465 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56051cd2f9f0 0x56051d134380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:07.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.865+0000 7f152a3b2740 4 client.29314 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:12:07.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.865+0000 7f152a3b2740 1 -- 172.21.15.67:0/2398079465 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56051d08eab0 con 0x56051cd2f9f0 2024-07-31T15:12:07.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.866+0000 7f1521bfd640 1 --2- 172.21.15.67:0/2398079465 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56051cd2f9f0 0x56051d134380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:07.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.867+0000 7f1521bfd640 1 --2- 172.21.15.67:0/2398079465 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56051cd2f9f0 0x56051d134380 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baea3cb364c8334e server_cookie=e46257c4575a670e in_seq=0 out_seq=0 2024-07-31T15:12:07.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.867+0000 7f15057fa640 10 client.29314 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:07.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.955+0000 7f15057fa640 1 -- 172.21.15.67:0/2398079465 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56051d08eab0 con 0x56051cd2f9f0 2024-07-31T15:12:07.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.955+0000 7f15057fa640 10 client.29314 handle_command_reply: tid=0 2024-07-31T15:12:07.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.955+0000 7f152a3b2740 1 client.29314 shutdown 2024-07-31T15:12:07.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.955+0000 7f152a3b2740 1 -- 172.21.15.67:0/2398079465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd71070 msgr2=0x56051d12e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:07.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.955+0000 7f152a3b2740 1 --2- 172.21.15.67:0/2398079465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd71070 0x56051d12e0b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1514009560 tx=0x7f1514009480 comp rx=0 tx=0).stop 2024-07-31T15:12:07.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.956+0000 7f152a3b2740 1 -- 172.21.15.67:0/2398079465 shutdown_connections 2024-07-31T15:12:07.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.956+0000 7f152a3b2740 1 --2- 172.21.15.67:0/2398079465 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56051cd2f9f0 0x56051d134380 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.956+0000 7f152a3b2740 1 --2- 172.21.15.67:0/2398079465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd71070 0x56051d12e0b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.956+0000 7f152a3b2740 1 --2- 172.21.15.67:0/2398079465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd50390 0x56051d1227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.956+0000 7f152a3b2740 1 --2- 172.21.15.67:0/2398079465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd481b0 0x56051d122250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.956+0000 7f152a3b2740 1 -- 172.21.15.67:0/2398079465 >> 172.21.15.67:0/2398079465 conn(0x56051cd65460 msgr2=0x56051d112cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:07.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.956+0000 7f15057fa640 0 client.29314 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:07.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.956+0000 7f152a3b2740 1 -- 172.21.15.67:0/2398079465 shutdown_connections 2024-07-31T15:12:07.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.956+0000 7f152a3b2740 1 -- 172.21.15.67:0/2398079465 wait complete. 2024-07-31T15:12:07.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.956+0000 7f152a3b2740 20 client.29314 trim_cache size 0 max 16384 2024-07-31T15:12:07.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.990+0000 7f152a3b2740 1 Processor -- start 2024-07-31T15:12:07.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.991+0000 7f152a3b2740 1 -- start start 2024-07-31T15:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.991+0000 7f152a3b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd481b0 0x56051d11b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.991+0000 7f152a3b2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd50390 0x56051d11bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.991+0000 7f152a3b2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd71070 0x56051d113e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.991+0000 7f152a3b2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1ae5c0 con 0x56051cd50390 2024-07-31T15:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.991+0000 7f152a3b2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1b0100 con 0x56051cd71070 2024-07-31T15:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.991+0000 7f152a3b2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1aec90 con 0x56051cd481b0 2024-07-31T15:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.992+0000 7f15213fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd50390 0x56051d11bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:07.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.992+0000 7f15213fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd50390 0x56051d11bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43358/0 (socket says 172.21.15.67:43358) 2024-07-31T15:12:07.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.992+0000 7f15213fc640 1 -- 172.21.15.67:0/3149366543 learned_addr learned my addr 172.21.15.67:0/3149366543 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:07.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.993+0000 7f15213fc640 1 -- 172.21.15.67:0/3149366543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd481b0 msgr2=0x56051d11b750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:12:07.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.993+0000 7f15213fc640 1 --2- 172.21.15.67:0/3149366543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd481b0 0x56051d11b750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.993+0000 7f15213fc640 1 -- 172.21.15.67:0/3149366543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd71070 msgr2=0x56051d113e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:12:07.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.993+0000 7f15213fc640 1 --2- 172.21.15.67:0/3149366543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd71070 0x56051d113e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:07.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.993+0000 7f15213fc640 1 -- 172.21.15.67:0/3149366543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56051cd2fc20 con 0x56051cd50390 2024-07-31T15:12:07.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.993+0000 7f15213fc640 1 --2- 172.21.15.67:0/3149366543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd50390 0x56051d11bc90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f150c0ab9a0 tx=0x7f150c0adfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4251b8ae7d21f891 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:07.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.994+0000 7f15077fe640 1 -- 172.21.15.67:0/3149366543 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c0ae730 con 0x56051cd50390 2024-07-31T15:12:07.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.994+0000 7f15077fe640 1 -- 172.21.15.67:0/3149366543 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f150c0a8e50 con 0x56051cd50390 2024-07-31T15:12:07.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.994+0000 7f15077fe640 1 -- 172.21.15.67:0/3149366543 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c0e5330 con 0x56051cd50390 2024-07-31T15:12:07.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.995+0000 7f152a3b2740 1 -- 172.21.15.67:0/3149366543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd50390 msgr2=0x56051d11bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:07.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.995+0000 7f152a3b2740 1 --2- 172.21.15.67:0/3149366543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd50390 0x56051d11bc90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f150c0ab9a0 tx=0x7f150c0adfb0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.999+0000 7f152a3b2740 1 -- 172.21.15.67:0/3149366543 shutdown_connections 2024-07-31T15:12:08.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.999+0000 7f152a3b2740 1 --2- 172.21.15.67:0/3149366543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd71070 0x56051d113e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.999+0000 7f152a3b2740 1 --2- 172.21.15.67:0/3149366543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd50390 0x56051d11bc90 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.999+0000 7f152a3b2740 1 --2- 172.21.15.67:0/3149366543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd481b0 0x56051d11b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:07.999+0000 7f152a3b2740 1 -- 172.21.15.67:0/3149366543 >> 172.21.15.67:0/3149366543 conn(0x56051cd65460 msgr2=0x56051d112cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:08.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.000+0000 7f152a3b2740 1 -- 172.21.15.67:0/3149366543 shutdown_connections 2024-07-31T15:12:08.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.000+0000 7f152a3b2740 1 -- 172.21.15.67:0/3149366543 wait complete. 2024-07-31T15:12:08.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.001+0000 7f152a3b2740 1 -- start start 2024-07-31T15:12:08.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.001+0000 7f152a3b2740 1 Processor -- start 2024-07-31T15:12:08.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f152a3b2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd481b0 0x56051d1253e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:08.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f152a3b2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd50390 0x56051d125960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:08.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f152a3b2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd71070 0x56051d1a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:08.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f152a3b2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1afa30 con 0x56051cd71070 2024-07-31T15:12:08.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f152a3b2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1af360 con 0x56051cd481b0 2024-07-31T15:12:08.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f152a3b2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1ac650 con 0x56051cd50390 2024-07-31T15:12:08.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f152a3b2740 10 client.0 _ll_register_callbacks cb 0x7f1514009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:08.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f15213fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd50390 0x56051d125960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:08.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f15213fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd50390 0x56051d125960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:48628/0 (socket says 172.21.15.67:48628) 2024-07-31T15:12:08.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f15213fc640 1 -- 172.21.15.67:0/2409722375 learned_addr learned my addr 172.21.15.67:0/2409722375 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:08.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f15213fc640 1 -- 172.21.15.67:0/2409722375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd481b0 msgr2=0x56051d1253e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:12:08.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f15213fc640 1 --2- 172.21.15.67:0/2409722375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd481b0 0x56051d1253e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f15213fc640 1 -- 172.21.15.67:0/2409722375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd71070 msgr2=0x56051d1a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:08.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f15213fc640 1 --2- 172.21.15.67:0/2409722375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd71070 0x56051d1a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.002+0000 7f15213fc640 1 -- 172.21.15.67:0/2409722375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56051d1b4860 con 0x56051cd50390 2024-07-31T15:12:08.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.003+0000 7f15213fc640 1 --2- 172.21.15.67:0/2409722375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd50390 0x56051d125960 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f150c0e7040 tx=0x7f150c0097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a7b8716ddd62e77 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:08.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.003+0000 7f15057fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:12:08.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.003+0000 7f15057fa640 1 -- 172.21.15.67:0/2409722375 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c0b7730 con 0x56051cd50390 2024-07-31T15:12:08.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.003+0000 7f152a3b2740 1 -- 172.21.15.67:0/2409722375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f150c0a8e50 con 0x56051cd50390 2024-07-31T15:12:08.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.003+0000 7f15057fa640 1 -- 172.21.15.67:0/2409722375 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f150c0aa9c0 con 0x56051cd50390 2024-07-31T15:12:08.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.004+0000 7f15057fa640 1 -- 172.21.15.67:0/2409722375 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f150c0ae730 con 0x56051cd50390 2024-07-31T15:12:08.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.004+0000 7f152a3b2740 10 client.29317 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:08.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.004+0000 7f152a3b2740 1 -- 172.21.15.67:0/2409722375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56051d1b59c0 con 0x56051cd50390 2024-07-31T15:12:08.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.004+0000 7f15057fa640 1 -- 172.21.15.67:0/2409722375 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f150c0a8e50 con 0x56051cd50390 2024-07-31T15:12:08.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.004+0000 7f15057fa640 1 -- 172.21.15.67:0/2409722375 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f150c0ec040 con 0x56051cd50390 2024-07-31T15:12:08.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.004+0000 7f152a3b2740 10 client.29317 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:08.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.004+0000 7f152a3b2740 10 client.29317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:12:08.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.004+0000 7f152a3b2740 20 client.29317 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:08.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.005+0000 7f152a3b2740 1 --2- 172.21.15.67:0/2409722375 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56051cd04da0 0x56051d1b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:08.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.005+0000 7f152a3b2740 4 client.29317 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:12:08.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.005+0000 7f152a3b2740 1 -- 172.21.15.67:0/2409722375 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56051d08a550 con 0x56051cd04da0 2024-07-31T15:12:08.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.006+0000 7f1521bfd640 1 --2- 172.21.15.67:0/2409722375 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56051cd04da0 0x56051d1b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:08.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.008+0000 7f1521bfd640 1 --2- 172.21.15.67:0/2409722375 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56051cd04da0 0x56051d1b7fd0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95c1265c20788fbf server_cookie=a7f3c6a1cb1ec8a6 in_seq=0 out_seq=0 2024-07-31T15:12:08.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.008+0000 7f15057fa640 10 client.29317 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:08.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.111+0000 7f15057fa640 1 -- 172.21.15.67:0/2409722375 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x56051d08a550 con 0x56051cd04da0 2024-07-31T15:12:08.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.111+0000 7f15057fa640 10 client.29317 handle_command_reply: tid=0 2024-07-31T15:12:08.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.111+0000 7f152a3b2740 1 client.29317 shutdown 2024-07-31T15:12:08.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.112+0000 7f152a3b2740 1 -- 172.21.15.67:0/2409722375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd50390 msgr2=0x56051d125960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:08.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.112+0000 7f152a3b2740 1 --2- 172.21.15.67:0/2409722375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd50390 0x56051d125960 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f150c0e7040 tx=0x7f150c0097b0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.112+0000 7f152a3b2740 1 -- 172.21.15.67:0/2409722375 shutdown_connections 2024-07-31T15:12:08.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.112+0000 7f152a3b2740 1 --2- 172.21.15.67:0/2409722375 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56051cd04da0 0x56051d1b7fd0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.112+0000 7f152a3b2740 1 --2- 172.21.15.67:0/2409722375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56051cd71070 0x56051d1a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.112+0000 7f152a3b2740 1 --2- 172.21.15.67:0/2409722375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56051cd50390 0x56051d125960 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.112+0000 7f152a3b2740 1 --2- 172.21.15.67:0/2409722375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56051cd481b0 0x56051d1253e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.112+0000 7f152a3b2740 1 -- 172.21.15.67:0/2409722375 >> 172.21.15.67:0/2409722375 conn(0x56051cd65460 msgr2=0x56051d11b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:08.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.112+0000 7f15057fa640 0 client.29317 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:08.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.112+0000 7f152a3b2740 1 -- 172.21.15.67:0/2409722375 shutdown_connections 2024-07-31T15:12:08.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.112+0000 7f152a3b2740 1 -- 172.21.15.67:0/2409722375 wait complete. 2024-07-31T15:12:08.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.112+0000 7f152a3b2740 20 client.29317 trim_cache size 0 max 16384 2024-07-31T15:12:08.116 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:12:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (5379 inodes in the stack)", 2024-07-31T15:12:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T15:12:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "1b31e70a-1dca-4e81-8b70-471a68a8f057": { 2024-07-31T15:12:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T15:12:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "1b31e70a-1dca-4e81-8b70-471a68a8f057", 2024-07-31T15:12:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T15:12:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:12:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:12:08.117 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:12:08.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.113+0000 7f1523e61640 1 -- 172.21.15.67:0/4201269932 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15080b3dc0 msgr2=0x7f15080b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:08.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.113+0000 7f1523e61640 1 --2- 172.21.15.67:0/4201269932 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15080b3dc0 0x7f15080b62d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x56051d1152c0 tx=0x7f1518005f90 comp rx=0 tx=0).stop 2024-07-31T15:12:08.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.113+0000 7f1523e61640 1 -- 172.21.15.67:0/4201269932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f151c108f10 msgr2=0x7f151c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:08.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.113+0000 7f1523e61640 1 --2- 172.21.15.67:0/4201269932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f151c108f10 0x7f151c19a230 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f151c06a450 tx=0x7f150c00db30 comp rx=0 tx=0).stop 2024-07-31T15:12:08.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.113+0000 7f1523e61640 1 -- 172.21.15.67:0/4201269932 shutdown_connections 2024-07-31T15:12:08.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.113+0000 7f1523e61640 1 --2- 172.21.15.67:0/4201269932 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f15080b3dc0 0x7f15080b62d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.113+0000 7f1523e61640 1 --2- 172.21.15.67:0/4201269932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f151c109860 0x7f151c19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.113+0000 7f1523e61640 1 --2- 172.21.15.67:0/4201269932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f151c108f10 0x7f151c19a230 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.113+0000 7f1523e61640 1 --2- 172.21.15.67:0/4201269932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f151c1035d0 0x7f151c199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:08.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.113+0000 7f1523e61640 1 -- 172.21.15.67:0/4201269932 >> 172.21.15.67:0/4201269932 conn(0x7f151c0ff060 msgr2=0x7f151c106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:08.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.114+0000 7f1523e61640 1 -- 172.21.15.67:0/4201269932 shutdown_connections 2024-07-31T15:12:08.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:08.114+0000 7f1523e61640 1 -- 172.21.15.67:0/4201269932 wait complete. 2024-07-31T15:12:08.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:08.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:08.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:08 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:08.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:08 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:08.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:08.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:08.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:08 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:08.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:08 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:08.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:08 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:08.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:08 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:09.070 INFO:tasks.cephfs.filesystem:scrub status for tag:1b31e70a-1dca-4e81-8b70-471a68a8f057 - {'path': '/', 'tag': '1b31e70a-1dca-4e81-8b70-471a68a8f057', 'options': 'recursive,force'} 2024-07-31T15:12:09.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:09 smithi067 ceph-mon[32750]: pgmap v1073: 129 pgs: 13 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 76 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 976 KiB/s wr, 306 op/s 2024-07-31T15:12:09.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:09.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:09.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:09 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:09.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:09 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:09.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:09 smithi073 ceph-mon[36714]: pgmap v1073: 129 pgs: 13 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 76 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 976 KiB/s wr, 306 op/s 2024-07-31T15:12:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:09 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:09.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:09 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:09.393 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:09 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:09.111+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c061 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math/ auth{1=1} v1164 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:10.213499+0000 31=31+0) n(v2 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1) old_inodes=2 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:542516 nref=3)) caps={15489=pAsLsXs/-@10} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8e92fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c061:[<0x1000000bf5f/math v634>,<0x10000000202/lib v5688>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000c061:[<0x1000000bf5f/math v1164>,<0x10000000202/lib v21040>,<0x10000000201/linux-6.5.11 v2689>,<0x10000000006/t v1727>,<0x10000000005/tmp v1652>,<0x100000001fb/client.0 v1623>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1585>,<0x10000000001/sv_1 v1586>,<0x10000000000/qa v1576>,<0x1/volumes v1529>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:10.988109+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:10.213499+0000 31=31+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:09 smithi077 ceph-mon[37327]: pgmap v1073: 129 pgs: 13 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 76 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 976 KiB/s wr, 306 op/s 2024-07-31T15:12:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:09 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:09 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:10.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:10 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:10.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:10 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:10.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:10 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:10.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:10 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:10.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:10 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:12:10.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:10 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:10.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:10 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:10.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:10 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:10.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:10 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:10.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:10 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:12:10.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:10 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:10.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:10 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:10.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:10 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:10.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:10 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:10.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:10 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:12:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:11 smithi077 ceph-mon[37327]: pgmap v1074: 129 pgs: 17 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 63 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1003 KiB/s wr, 292 op/s 2024-07-31T15:12:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:11 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:11.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:11 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:11.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:11 smithi067 ceph-mon[32750]: pgmap v1074: 129 pgs: 17 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 63 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1003 KiB/s wr, 292 op/s 2024-07-31T15:12:11.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:11.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:11.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:11.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:11.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:11 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:11.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:11 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:11.623 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:11 smithi073 ceph-mon[36714]: pgmap v1074: 129 pgs: 17 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 63 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1003 KiB/s wr, 292 op/s 2024-07-31T15:12:11.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:11.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:11.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:11.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:11.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:11 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:11.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:11 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:11.624 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:11 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:11.417+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000ce48 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity/ auth{1=1,2=1} v1695 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3) n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4) old_inodes=3 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:543294 nref=3)) caps={15489=pAsLsXs/-@10} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8aaac580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000ce48:[<0x1000000ce07/integrity v345>,<0x10000000202/security v8766>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000ce48:[<0x1000000ce07/integrity v1695>,<0x10000000202/security v15110>,<0x10000000201/linux-6.5.11 v2696>,<0x10000000006/t v1734>,<0x10000000005/tmp v1660>,<0x100000001fb/client.0 v1629>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1592>,<0x10000000001/sv_1 v1593>,<0x10000000000/qa v1584>,<0x1/volumes v1535>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:59.470106+0000 10=7+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","memory_value.dirstat":"f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3)","memory_value.rstat":"n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:11.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:11 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:11.621+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009eb8 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs/ auth{4=1} v4394 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:25.901734+0000 12=12+0) n(v3 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:543778 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce809d7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009eb8:[<0x10000009d36/debugfs v3846>,<0x10000000202/fs v3830>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009eb8:[<0x10000009d36/debugfs v4394>,<0x10000000202/fs v21227>,<0x10000000201/linux-6.5.11 v2696>,<0x10000000006/t v1734>,<0x10000000005/tmp v1660>,<0x100000001fb/client.0 v1629>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1592>,<0x10000000001/sv_1 v1593>,<0x10000000000/qa v1584>,<0x1/volumes v1535>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:39.315767+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:25.901734+0000 12=12+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:11.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:11 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:11.701+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000a492 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs/ auth{1=1} v5396 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:30.806634+0000 8=8+0) n(v3 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1) old_inodes=2 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:543851 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8aca9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000a492:[<0x10000009d36/tracefs v3850>,<0x10000000202/fs v3830>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x1000000a492:[<0x10000009d36/tracefs v5396>,<0x10000000202/fs v21227>,<0x10000000201/linux-6.5.11 v2696>,<0x10000000006/t v1734>,<0x10000000005/tmp v1660>,<0x100000001fb/client.0 v1629>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1592>,<0x10000000001/sv_1 v1593>,<0x10000000000/qa v1584>,<0x1/volumes v1535>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:41.554720+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:30.806634+0000 8=8+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:11.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:11 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:11.770+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002351 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char/ auth{2=1} v18176 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:34.793553+0000 56=49+7) n(v10 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10) old_inodes=4 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:553151 nref=3)) caps={15489=pAsLsXs/-@11} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7ef94100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000002351:[<0x10000001d8f/char v2490>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x10000002351:[<0x10000001d8f/char v18176>,<0x10000000202/drivers v21969>,<0x10000000201/linux-6.5.11 v2696>,<0x10000000006/t v1734>,<0x10000000005/tmp v1660>,<0x100000001fb/client.0 v1629>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1592>,<0x10000000001/sv_1 v1593>,<0x10000000000/qa v1584>,<0x1/volumes v1535>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:02.131047+0000 40=33+7)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:34.793553+0000 56=49+7)","memory_value.rstat":"n(v10 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:12.229 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:11 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:11.936+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000243a [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk/ auth{2=1,4=1} fragtree_t(*^3) v18679 ap=3 RANDEPHEMERALPIN f(v3 m2024-07-31T15:07:07.059118+0000 137=88+49) n(v9 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50) old_inodes=2 (inest sync->mix g=2,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:553233 nref=3)) caps={15489=pAsLsXs/-@8} | dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7fa16680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000243a:[<0x10000001d8f/clk v2484>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x1000000243a:[<0x10000001d8f/clk v18679>,<0x10000000202/drivers v21971>,<0x10000000201/linux-6.5.11 v2696>,<0x10000000006/t v1734>,<0x10000000005/tmp v1660>,<0x100000001fb/client.0 v1629>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1592>,<0x10000000001/sv_1 v1593>,<0x10000000000/qa v1584>,<0x1/volumes v1535>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 133=84+49)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50)","memory_value.dirstat":"f(v3 m2024-07-31T15:07:07.059118+0000 137=88+49)","memory_value.rstat":"n(v9 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:12.229 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:12.002+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009980 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video/ auth{1=1,4=1} v18105 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:16.340374+0000 28=24+4) n(v11 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28) old_inodes=2 (inest sync->mix g=1,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:553274 nref=3)) caps={15489=pAsLsXs/-@6} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce87325b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009980:[<0x10000001d8f/video v17889>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009980:[<0x10000001d8f/video v18105>,<0x10000000202/drivers v21971>,<0x10000000201/linux-6.5.11 v2696>,<0x10000000006/t v1734>,<0x10000000005/tmp v1660>,<0x100000001fb/client.0 v1629>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1592>,<0x10000000001/sv_1 v1593>,<0x10000000000/qa v1584>,<0x1/volumes v1535>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:35.535846+0000 16=12+4)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:16.340374+0000 28=24+4)","memory_value.rstat":"n(v11 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:12.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:12.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:12.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:12.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:12.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.489 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:12.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:12.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:12.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:12 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:12.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:12 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:12.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:12.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:12.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:12.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:12.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:12.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:12.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:12.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:12.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:12.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:12.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:12.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:12.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:12.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:12 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:13.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: pgmap v1075: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 882 KiB/s wr, 286 op/s 2024-07-31T15:12:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:13.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:13 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:13.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:13.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: pgmap v1075: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 882 KiB/s wr, 286 op/s 2024-07-31T15:12:13.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:13.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:13.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:13.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:13.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:13.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:13.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:13.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:13.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: pgmap v1075: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 882 KiB/s wr, 286 op/s 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:13.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:13.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:13.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:13 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.142 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:13.679+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000be32 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module/ auth{2=1} v2029 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0) n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:550828 nref=3)) caps={15489=pAsLsXs/-@13} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce826b7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000be32:[<0x1000000bd0a/module v1134>,<0x10000000202/kernel v5648>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000be32:[<0x1000000bd0a/module v2029>,<0x10000000202/kernel v21206>,<0x10000000201/linux-6.5.11 v2696>,<0x10000000006/t v1734>,<0x10000000005/tmp v1660>,<0x100000001fb/client.0 v1629>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1592>,<0x10000000001/sv_1 v1593>,<0x10000000000/qa v1584>,<0x1/volumes v1535>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:07.168188+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.232+0000 7f718242c640 1 -- 172.21.15.67:0/1125270865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f717c108e70 msgr2=0x7f717c109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.232+0000 7f718242c640 1 --2- 172.21.15.67:0/1125270865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f717c108e70 0x7f717c109280 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f7170009900 tx=0x7f717002a410 comp rx=0 tx=0).stop 2024-07-31T15:12:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.232+0000 7f718242c640 1 -- 172.21.15.67:0/1125270865 shutdown_connections 2024-07-31T15:12:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.232+0000 7f718242c640 1 --2- 172.21.15.67:0/1125270865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f717c1097c0 0x7f717c10fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.232+0000 7f718242c640 1 --2- 172.21.15.67:0/1125270865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f717c108e70 0x7f717c109280 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.232+0000 7f718242c640 1 --2- 172.21.15.67:0/1125270865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f717c103540 0x7f717c103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.232+0000 7f718242c640 1 -- 172.21.15.67:0/1125270865 >> 172.21.15.67:0/1125270865 conn(0x7f717c0fefa0 msgr2=0x7f717c1013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.232+0000 7f718242c640 1 -- 172.21.15.67:0/1125270865 shutdown_connections 2024-07-31T15:12:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.232+0000 7f718242c640 1 -- 172.21.15.67:0/1125270865 wait complete. 2024-07-31T15:12:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f718242c640 1 Processor -- start 2024-07-31T15:12:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f718242c640 1 -- start start 2024-07-31T15:12:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f718242c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f717c103540 0x7f717c199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f718242c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f717c108e70 0x7f717c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f718242c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f717c1097c0 0x7f717c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f718242c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f717c113430 con 0x7f717c108e70 2024-07-31T15:12:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f718242c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1132b0 con 0x7f717c103540 2024-07-31T15:12:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f718242c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1135b0 con 0x7f717c1097c0 2024-07-31T15:12:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f717b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f717c108e70 0x7f717c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f717bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f717c103540 0x7f717c199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f717bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f717c103540 0x7f717c199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46914/0 (socket says 172.21.15.67:46914) 2024-07-31T15:12:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f717bfff640 1 -- 172.21.15.67:0/1989717104 learned_addr learned my addr 172.21.15.67:0/1989717104 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f71809c9640 1 --2- 172.21.15.67:0/1989717104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f717c1097c0 0x7f717c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f717bfff640 1 -- 172.21.15.67:0/1989717104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f717c1097c0 msgr2=0x7f717c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f717bfff640 1 --2- 172.21.15.67:0/1989717104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f717c1097c0 0x7f717c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f717bfff640 1 -- 172.21.15.67:0/1989717104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f717c108e70 msgr2=0x7f717c19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f717bfff640 1 --2- 172.21.15.67:0/1989717104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f717c108e70 0x7f717c19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.233+0000 7f717bfff640 1 -- 172.21.15.67:0/1989717104 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c19fda0 con 0x7f717c103540 2024-07-31T15:12:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.234+0000 7f717b7fe640 1 --2- 172.21.15.67:0/1989717104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f717c108e70 0x7f717c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:12:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.234+0000 7f717bfff640 1 --2- 172.21.15.67:0/1989717104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f717c103540 0x7f717c199e70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f716c00e9f0 tx=0x7f716c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.234+0000 7f71797fa640 1 -- 172.21.15.67:0/1989717104 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c00cd80 con 0x7f717c103540 2024-07-31T15:12:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.234+0000 7f71797fa640 1 -- 172.21.15.67:0/1989717104 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f716c0108e0 con 0x7f717c103540 2024-07-31T15:12:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.234+0000 7f71797fa640 1 -- 172.21.15.67:0/1989717104 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c018980 con 0x7f717c103540 2024-07-31T15:12:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.234+0000 7f718242c640 1 -- 172.21.15.67:0/1989717104 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c1a0090 con 0x7f717c103540 2024-07-31T15:12:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.234+0000 7f718242c640 1 -- 172.21.15.67:0/1989717104 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c1a6820 con 0x7f717c103540 2024-07-31T15:12:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.237+0000 7f71797fa640 1 -- 172.21.15.67:0/1989717104 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f716c010430 con 0x7f717c103540 2024-07-31T15:12:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.237+0000 7f71893fe740 1 Processor -- start 2024-07-31T15:12:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f71893fe740 1 -- start start 2024-07-31T15:12:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f71893fe740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7b91b0 0x56047ab5b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f71893fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7902f0 0x56047ab5bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f71893fe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 0x56047ab5c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f71893fe740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56047aad13b0 con 0x56047a7b91b0 2024-07-31T15:12:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f71797fa640 1 --2- 172.21.15.67:0/1989717104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f71640b3d40 0x7f71640b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f71893fe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56047aac08b0 con 0x56047a7984d0 2024-07-31T15:12:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f71893fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56047aac48f0 con 0x56047a7902f0 2024-07-31T15:12:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f71809c9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7b91b0 0x56047ab5b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f717bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7902f0 0x56047ab5bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f71809c9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7b91b0 0x56047ab5b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49162/0 (socket says 172.21.15.67:49162) 2024-07-31T15:12:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f717bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7902f0 0x56047ab5bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49566/0 (socket says 172.21.15.67:49566) 2024-07-31T15:12:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f71809c9640 1 -- 172.21.15.67:0/2586500098 learned_addr learned my addr 172.21.15.67:0/2586500098 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f717b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 0x56047ab5c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f71809c9640 1 --2- 172.21.15.67:0/1989717104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f71640b3d40 0x7f71640b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.238+0000 7f71797fa640 1 -- 172.21.15.67:0/1989717104 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f716c0e46d0 con 0x7f717c103540 2024-07-31T15:12:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.239+0000 7f717b7fe640 1 -- 172.21.15.67:0/2586500098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7902f0 msgr2=0x56047ab5bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.239+0000 7f717b7fe640 1 --2- 172.21.15.67:0/2586500098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7902f0 0x56047ab5bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.239+0000 7f717b7fe640 1 -- 172.21.15.67:0/2586500098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7b91b0 msgr2=0x56047ab5b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.239+0000 7f717b7fe640 1 --2- 172.21.15.67:0/2586500098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7b91b0 0x56047ab5b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.239+0000 7f717b7fe640 1 -- 172.21.15.67:0/2586500098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56047aa679a0 con 0x56047a7984d0 2024-07-31T15:12:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.239+0000 7f717b7fe640 1 --2- 172.21.15.67:0/2586500098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 0x56047ab5c130 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f717c069f60 tx=0x7f7170000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2efca603ddfcbca7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.239+0000 7f71809c9640 1 --2- 172.21.15.67:0/1989717104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f71640b3d40 0x7f71640b6250 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f71740099e0 tx=0x7f7174009c80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.240+0000 7f71627fc640 1 -- 172.21.15.67:0/2586500098 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7170035040 con 0x56047a7984d0 2024-07-31T15:12:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.240+0000 7f71627fc640 1 -- 172.21.15.67:0/2586500098 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7170031720 con 0x56047a7984d0 2024-07-31T15:12:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.240+0000 7f71627fc640 1 -- 172.21.15.67:0/2586500098 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7170004770 con 0x56047a7984d0 2024-07-31T15:12:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.241+0000 7f71893fe740 1 -- 172.21.15.67:0/2586500098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 msgr2=0x56047ab5c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.241+0000 7f71893fe740 1 --2- 172.21.15.67:0/2586500098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 0x56047ab5c130 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f717c069f60 tx=0x7f7170000bc0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.241+0000 7f71893fe740 1 -- 172.21.15.67:0/2586500098 shutdown_connections 2024-07-31T15:12:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.241+0000 7f71893fe740 1 --2- 172.21.15.67:0/2586500098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 0x56047ab5c130 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.241+0000 7f71893fe740 1 --2- 172.21.15.67:0/2586500098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7902f0 0x56047ab5bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.241+0000 7f71893fe740 1 --2- 172.21.15.67:0/2586500098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7b91b0 0x56047ab5b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.241+0000 7f71893fe740 1 -- 172.21.15.67:0/2586500098 >> 172.21.15.67:0/2586500098 conn(0x56047a7ad300 msgr2=0x56047ab595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.241+0000 7f71893fe740 1 -- 172.21.15.67:0/2586500098 shutdown_connections 2024-07-31T15:12:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.241+0000 7f71893fe740 1 -- 172.21.15.67:0/2586500098 wait complete. 2024-07-31T15:12:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.242+0000 7f71893fe740 1 -- start start 2024-07-31T15:12:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.242+0000 7f71893fe740 1 Processor -- start 2024-07-31T15:12:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.242+0000 7f71893fe740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7902f0 0x56047ab67220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.242+0000 7f71893fe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 0x56047ab67760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.242+0000 7f71893fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.242+0000 7f71893fe740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56047a9faab0 con 0x56047a7902f0 2024-07-31T15:12:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.242+0000 7f71893fe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56047aa48cf0 con 0x56047a7984d0 2024-07-31T15:12:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.242+0000 7f71893fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56047ab69dd0 con 0x56047a7b91b0 2024-07-31T15:12:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.242+0000 7f71893fe740 10 client.0 _ll_register_callbacks cb 0x56047aaba3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.242+0000 7f71809c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.242+0000 7f717bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7902f0 0x56047ab67220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.242+0000 7f71809c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49570/0 (socket says 172.21.15.67:49570) 2024-07-31T15:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.242+0000 7f71809c9640 1 -- 172.21.15.67:0/3377337197 learned_addr learned my addr 172.21.15.67:0/3377337197 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.243+0000 7f717b7fe640 1 --2- 172.21.15.67:0/3377337197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 0x56047ab67760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.243+0000 7f71809c9640 1 -- 172.21.15.67:0/3377337197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 msgr2=0x56047ab67760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.243+0000 7f71809c9640 1 --2- 172.21.15.67:0/3377337197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 0x56047ab67760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.243+0000 7f71809c9640 1 -- 172.21.15.67:0/3377337197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7902f0 msgr2=0x56047ab67220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.243+0000 7f71809c9640 1 --2- 172.21.15.67:0/3377337197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7902f0 0x56047ab67220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.243+0000 7f71809c9640 1 -- 172.21.15.67:0/3377337197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56047a824790 con 0x56047a7b91b0 2024-07-31T15:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.243+0000 7f71809c9640 1 --2- 172.21.15.67:0/3377337197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab730c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7174003e70 tx=0x7f7174009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89ad643306676882 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.243+0000 7f715b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.243+0000 7f715b7fe640 1 -- 172.21.15.67:0/3377337197 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174020870 con 0x56047a7b91b0 2024-07-31T15:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.243+0000 7f715b7fe640 1 -- 172.21.15.67:0/3377337197 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f7174020a10 con 0x56047a7b91b0 2024-07-31T15:12:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.243+0000 7f71893fe740 1 -- 172.21.15.67:0/3377337197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7170031720 con 0x56047a7b91b0 2024-07-31T15:12:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.250+0000 7f715b7fe640 1 -- 172.21.15.67:0/3377337197 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f7174020c10 con 0x56047a7b91b0 2024-07-31T15:12:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.250+0000 7f71893fe740 10 client.29329 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.250+0000 7f71893fe740 1 -- 172.21.15.67:0/3377337197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56047a89d620 con 0x56047a7b91b0 2024-07-31T15:12:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.250+0000 7f715b7fe640 1 -- 172.21.15.67:0/3377337197 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7170031720 con 0x56047a7b91b0 2024-07-31T15:12:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.250+0000 7f715b7fe640 1 -- 172.21.15.67:0/3377337197 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f7174027020 con 0x56047a7b91b0 2024-07-31T15:12:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.251+0000 7f71893fe740 10 client.29329 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.251+0000 7f71893fe740 10 client.29329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:12:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.251+0000 7f71893fe740 20 client.29329 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.251+0000 7f71893fe740 1 --2- 172.21.15.67:0/3377337197 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56047a777d80 0x56047ab792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.251+0000 7f71893fe740 4 client.29329 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:12:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.251+0000 7f71893fe740 1 -- 172.21.15.67:0/3377337197 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56047aad3300 con 0x56047a777d80 2024-07-31T15:12:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.251+0000 7f717bfff640 1 --2- 172.21.15.67:0/3377337197 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56047a777d80 0x56047ab792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.251+0000 7f715b7fe640 10 client.29329 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.251+0000 7f717bfff640 1 --2- 172.21.15.67:0/3377337197 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56047a777d80 0x56047ab792d0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c65204824c072f7e server_cookie=7650ea959b701152 in_seq=0 out_seq=0 2024-07-31T15:12:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.261+0000 7f715b7fe640 1 -- 172.21.15.67:0/3377337197 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56047aad3300 con 0x56047a777d80 2024-07-31T15:12:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.261+0000 7f715b7fe640 10 client.29329 handle_command_reply: tid=0 2024-07-31T15:12:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.261+0000 7f71893fe740 1 client.29329 shutdown 2024-07-31T15:12:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.262+0000 7f71893fe740 1 -- 172.21.15.67:0/3377337197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 msgr2=0x56047ab730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.262+0000 7f71893fe740 1 --2- 172.21.15.67:0/3377337197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab730c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7174003e70 tx=0x7f7174009b30 comp rx=0 tx=0).stop 2024-07-31T15:12:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.262+0000 7f71893fe740 1 -- 172.21.15.67:0/3377337197 shutdown_connections 2024-07-31T15:12:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.262+0000 7f71893fe740 1 --2- 172.21.15.67:0/3377337197 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56047a777d80 0x56047ab792d0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.262+0000 7f71893fe740 1 --2- 172.21.15.67:0/3377337197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab730c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.262+0000 7f71893fe740 1 --2- 172.21.15.67:0/3377337197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 0x56047ab67760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.262+0000 7f71893fe740 1 --2- 172.21.15.67:0/3377337197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7902f0 0x56047ab67220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.262+0000 7f715b7fe640 0 client.29329 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.262+0000 7f71893fe740 1 -- 172.21.15.67:0/3377337197 >> 172.21.15.67:0/3377337197 conn(0x56047a7ad300 msgr2=0x56047ab5c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.262+0000 7f71893fe740 1 -- 172.21.15.67:0/3377337197 shutdown_connections 2024-07-31T15:12:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.262+0000 7f71893fe740 1 -- 172.21.15.67:0/3377337197 wait complete. 2024-07-31T15:12:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.262+0000 7f71893fe740 20 client.29329 trim_cache size 0 max 16384 2024-07-31T15:12:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.282+0000 7f71893fe740 1 Processor -- start 2024-07-31T15:12:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.282+0000 7f71893fe740 1 -- start start 2024-07-31T15:12:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.282+0000 7f71893fe740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7902f0 0x56047ab68ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.282+0000 7f71893fe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 0x56047ab66880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.282+0000 7f71893fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab7e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.282+0000 7f71893fe740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1ae6c0 con 0x56047a7902f0 2024-07-31T15:12:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.282+0000 7f71893fe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1b0200 con 0x56047a7984d0 2024-07-31T15:12:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.282+0000 7f71893fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1aed90 con 0x56047a7b91b0 2024-07-31T15:12:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.282+0000 7f717bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7902f0 0x56047ab68ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.282+0000 7f717bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7902f0 0x56047ab68ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49170/0 (socket says 172.21.15.67:49170) 2024-07-31T15:12:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.282+0000 7f717bfff640 1 -- 172.21.15.67:0/3460064634 learned_addr learned my addr 172.21.15.67:0/3460064634 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.282+0000 7f71809c9640 1 --2- 172.21.15.67:0/3460064634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab7e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.282+0000 7f717b7fe640 1 --2- 172.21.15.67:0/3460064634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 0x56047ab66880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.283+0000 7f717bfff640 1 -- 172.21.15.67:0/3460064634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 msgr2=0x56047ab7e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.283+0000 7f717bfff640 1 --2- 172.21.15.67:0/3460064634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab7e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.283+0000 7f717bfff640 1 -- 172.21.15.67:0/3460064634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 msgr2=0x56047ab66880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.283+0000 7f717bfff640 1 --2- 172.21.15.67:0/3460064634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 0x56047ab66880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.283+0000 7f717bfff640 1 -- 172.21.15.67:0/3460064634 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56047ab5d1b0 con 0x56047a7902f0 2024-07-31T15:12:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.283+0000 7f71809c9640 1 --2- 172.21.15.67:0/3460064634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab7e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:12:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.283+0000 7f717bfff640 1 --2- 172.21.15.67:0/3460064634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7902f0 0x56047ab68ef0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f716c0a8240 tx=0x7f716c018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ee36307298dfd4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.283+0000 7f7161ffb640 1 -- 172.21.15.67:0/3460064634 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c0c4070 con 0x56047a7902f0 2024-07-31T15:12:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.283+0000 7f7161ffb640 1 -- 172.21.15.67:0/3460064634 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f716c0aafd0 con 0x56047a7902f0 2024-07-31T15:12:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.283+0000 7f7161ffb640 1 -- 172.21.15.67:0/3460064634 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c0a7260 con 0x56047a7902f0 2024-07-31T15:12:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.284+0000 7f71893fe740 1 -- 172.21.15.67:0/3460064634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7902f0 msgr2=0x56047ab68ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.284+0000 7f71893fe740 1 --2- 172.21.15.67:0/3460064634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7902f0 0x56047ab68ef0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f716c0a8240 tx=0x7f716c018420 comp rx=0 tx=0).stop 2024-07-31T15:12:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.284+0000 7f71893fe740 1 -- 172.21.15.67:0/3460064634 shutdown_connections 2024-07-31T15:12:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.284+0000 7f71893fe740 1 --2- 172.21.15.67:0/3460064634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab7e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.284+0000 7f71893fe740 1 --2- 172.21.15.67:0/3460064634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7984d0 0x56047ab66880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.284+0000 7f71893fe740 1 --2- 172.21.15.67:0/3460064634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7902f0 0x56047ab68ef0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.284+0000 7f71893fe740 1 -- 172.21.15.67:0/3460064634 >> 172.21.15.67:0/3460064634 conn(0x56047a7ad300 msgr2=0x56047ab607e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.284+0000 7f71893fe740 1 -- 172.21.15.67:0/3460064634 shutdown_connections 2024-07-31T15:12:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.285+0000 7f71893fe740 1 -- 172.21.15.67:0/3460064634 wait complete. 2024-07-31T15:12:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.285+0000 7f71893fe740 1 -- start start 2024-07-31T15:12:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.285+0000 7f71893fe740 1 Processor -- start 2024-07-31T15:12:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f71893fe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7902f0 0x56047ab6f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f71893fe740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7984d0 0x56047ab6bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f71893fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab6c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f71893fe740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1afb30 con 0x56047a7984d0 2024-07-31T15:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f717bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7902f0 0x56047ab6f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f717bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7902f0 0x56047ab6f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46962/0 (socket says 172.21.15.67:46962) 2024-07-31T15:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f717bfff640 1 -- 172.21.15.67:0/167501744 learned_addr learned my addr 172.21.15.67:0/167501744 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f71893fe740 1 -- 172.21.15.67:0/167501744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1af460 con 0x56047a7902f0 2024-07-31T15:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f71809c9640 1 --2- 172.21.15.67:0/167501744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab6c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f717b7fe640 1 --2- 172.21.15.67:0/167501744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7984d0 0x56047ab6bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f71893fe740 1 -- 172.21.15.67:0/167501744 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1ac6c0 con 0x56047a7b91b0 2024-07-31T15:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f71893fe740 10 client.0 _ll_register_callbacks cb 0x7f7174003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f717bfff640 1 -- 172.21.15.67:0/167501744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 msgr2=0x56047ab6c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f717bfff640 1 --2- 172.21.15.67:0/167501744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab6c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f717bfff640 1 -- 172.21.15.67:0/167501744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7984d0 msgr2=0x56047ab6bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f717bfff640 1 --2- 172.21.15.67:0/167501744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7984d0 0x56047ab6bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.286+0000 7f717bfff640 1 -- 172.21.15.67:0/167501744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56047ab6cc40 con 0x56047a7902f0 2024-07-31T15:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.287+0000 7f717bfff640 1 --2- 172.21.15.67:0/167501744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7902f0 0x56047ab6f6a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f716c0aa8b0 tx=0x7f716c0b4040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca16b62e09cd9314 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.287+0000 7f715b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.287+0000 7f715b7fe640 1 -- 172.21.15.67:0/167501744 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c0c4040 con 0x56047a7902f0 2024-07-31T15:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.287+0000 7f715b7fe640 1 -- 172.21.15.67:0/167501744 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f716c0e1ce0 con 0x56047a7902f0 2024-07-31T15:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.287+0000 7f71893fe740 1 -- 172.21.15.67:0/167501744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f716c0aafd0 con 0x56047a7902f0 2024-07-31T15:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.287+0000 7f715b7fe640 1 -- 172.21.15.67:0/167501744 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f716c0bd050 con 0x56047a7902f0 2024-07-31T15:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.287+0000 7f71893fe740 10 client.29465 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.287+0000 7f71893fe740 1 -- 172.21.15.67:0/167501744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56047abe07b0 con 0x56047a7902f0 2024-07-31T15:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.287+0000 7f715b7fe640 1 -- 172.21.15.67:0/167501744 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f716c0aafd0 con 0x56047a7902f0 2024-07-31T15:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.287+0000 7f715b7fe640 1 -- 172.21.15.67:0/167501744 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f716c0e4290 con 0x56047a7902f0 2024-07-31T15:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.287+0000 7f71893fe740 10 client.29465 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.287+0000 7f71893fe740 10 client.29465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.287+0000 7f71893fe740 20 client.29465 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.288+0000 7f71893fe740 1 --2- 172.21.15.67:0/167501744 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56047a74d1c0 0x56047abe2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.288+0000 7f71893fe740 4 client.29465 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.288+0000 7f71893fe740 1 -- 172.21.15.67:0/167501744 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56047aaceda0 con 0x56047a74d1c0 2024-07-31T15:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.288+0000 7f717b7fe640 1 --2- 172.21.15.67:0/167501744 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56047a74d1c0 0x56047abe2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.288+0000 7f717b7fe640 1 --2- 172.21.15.67:0/167501744 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56047a74d1c0 0x56047abe2e50 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9a1b23146ef505a server_cookie=b443723a1ef578dc in_seq=0 out_seq=0 2024-07-31T15:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.288+0000 7f715b7fe640 10 client.29465 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:14.509 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:14.509 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.509 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.510 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:14.510 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.510 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:14.510 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.510 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:14.510 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.510 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:14.510 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.510 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:14.510 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.510 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.510 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:14.511 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.511 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:14.511 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.511 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:14.511 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.511 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:14.511 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:14 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:14.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:14.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:14 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f715b7fe640 1 -- 172.21.15.67:0/167501744 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56047aaceda0 con 0x56047a74d1c0 2024-07-31T15:12:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f715b7fe640 10 client.29465 handle_command_reply: tid=0 2024-07-31T15:12:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f71893fe740 1 client.29465 shutdown 2024-07-31T15:12:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f71893fe740 1 -- 172.21.15.67:0/167501744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7902f0 msgr2=0x56047ab6f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f71893fe740 1 --2- 172.21.15.67:0/167501744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7902f0 0x56047ab6f6a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f716c0aa8b0 tx=0x7f716c0b4040 comp rx=0 tx=0).stop 2024-07-31T15:12:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f71893fe740 1 -- 172.21.15.67:0/167501744 shutdown_connections 2024-07-31T15:12:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f71893fe740 1 --2- 172.21.15.67:0/167501744 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56047a74d1c0 0x56047abe2e50 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f71893fe740 1 --2- 172.21.15.67:0/167501744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56047a7b91b0 0x56047ab6c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f71893fe740 1 --2- 172.21.15.67:0/167501744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56047a7984d0 0x56047ab6bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f715b7fe640 0 client.29465 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f71893fe740 1 --2- 172.21.15.67:0/167501744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56047a7902f0 0x56047ab6f6a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f71893fe740 1 -- 172.21.15.67:0/167501744 >> 172.21.15.67:0/167501744 conn(0x56047a7ad300 msgr2=0x56047ab68c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f71893fe740 1 -- 172.21.15.67:0/167501744 shutdown_connections 2024-07-31T15:12:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f71893fe740 1 -- 172.21.15.67:0/167501744 wait complete. 2024-07-31T15:12:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.645+0000 7f71893fe740 20 client.29465 trim_cache size 0 max 16384 2024-07-31T15:12:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:12:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:12:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:12:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:12:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.646+0000 7f718242c640 1 -- 172.21.15.67:0/1989717104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f71640b3d40 msgr2=0x7f71640b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.646+0000 7f718242c640 1 --2- 172.21.15.67:0/1989717104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f71640b3d40 0x7f71640b6250 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f71740099e0 tx=0x7f7174009c80 comp rx=0 tx=0).stop 2024-07-31T15:12:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.646+0000 7f718242c640 1 -- 172.21.15.67:0/1989717104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f717c103540 msgr2=0x7f717c199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.646+0000 7f718242c640 1 --2- 172.21.15.67:0/1989717104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f717c103540 0x7f717c199e70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f716c00e9f0 tx=0x7f716c00eeb0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.646+0000 7f718242c640 1 -- 172.21.15.67:0/1989717104 shutdown_connections 2024-07-31T15:12:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.646+0000 7f718242c640 1 --2- 172.21.15.67:0/1989717104 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f71640b3d40 0x7f71640b6250 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.646+0000 7f718242c640 1 --2- 172.21.15.67:0/1989717104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f717c1097c0 0x7f717c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.646+0000 7f718242c640 1 --2- 172.21.15.67:0/1989717104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f717c108e70 0x7f717c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.646+0000 7f718242c640 1 --2- 172.21.15.67:0/1989717104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f717c103540 0x7f717c199e70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.646+0000 7f718242c640 1 -- 172.21.15.67:0/1989717104 >> 172.21.15.67:0/1989717104 conn(0x7f717c0fefa0 msgr2=0x7f717c1013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.646+0000 7f718242c640 1 -- 172.21.15.67:0/1989717104 shutdown_connections 2024-07-31T15:12:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:14.646+0000 7f718242c640 1 -- 172.21.15.67:0/1989717104 wait complete. 2024-07-31T15:12:14.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd2aff5640 1 -- 172.21.15.67:0/1672966608 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560c7ae871a0 con 0x560c7ab02570 2024-07-31T15:12:14.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd2aff5640 10 client.29351 handle_command_reply: tid=0 2024-07-31T15:12:14.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd53b15740 1 client.29351 shutdown 2024-07-31T15:12:14.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd53b15740 1 -- 172.21.15.67:0/1672966608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 msgr2=0x560c7af26ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:14.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd53b15740 1 --2- 172.21.15.67:0/1672966608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 0x560c7af26ce0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0a62d0 tx=0x7fbd3c0a6d30 comp rx=0 tx=0).stop 2024-07-31T15:12:14.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd53b15740 1 -- 172.21.15.67:0/1672966608 shutdown_connections 2024-07-31T15:12:14.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd53b15740 1 --2- 172.21.15.67:0/1672966608 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x560c7ab02570 0x560c7af9cc10 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd53b15740 1 --2- 172.21.15.67:0/1672966608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c7ab6e840 0x560c7af26ce0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd53b15740 1 --2- 172.21.15.67:0/1672966608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c7ab4db60 0x560c7af267a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd53b15740 1 --2- 172.21.15.67:0/1672966608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560c7ab45980 0x560c7af26260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd2aff5640 0 client.29351 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:14.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd53b15740 1 -- 172.21.15.67:0/1672966608 >> 172.21.15.67:0/1672966608 conn(0x560c7ab62990 msgr2=0x560c7af21e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:14.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd53b15740 1 -- 172.21.15.67:0/1672966608 shutdown_connections 2024-07-31T15:12:14.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd53b15740 1 -- 172.21.15.67:0/1672966608 wait complete. 2024-07-31T15:12:14.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.712+0000 7fbd53b15740 20 client.29351 trim_cache size 0 max 16384 2024-07-31T15:12:14.716 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:12:14.716 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 66, 2024-07-31T15:12:14.716 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:12:14.716 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 58, 2024-07-31T15:12:14.716 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:12:14.716 INFO:teuthology.orchestra.run.smithi067.stdout: "53a8a385": { 2024-07-31T15:12:14.716 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 58, 2024-07-31T15:12:14.716 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:12:14.717 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:12:14.717 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:12:14.717 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:12:14.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:12:14.717 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:12:14.717 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 137.0, 2024-07-31T15:12:14.717 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:12:14.717 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:12:14.717 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:12:14.717 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:12:14.717 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:12:14.717 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:12:14.717 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:12:14.718 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:12:14.718 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:12:14.718 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:12:14.718 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:12:14.718 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:12:14.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.713+0000 7fbd4d5c4640 1 -- 172.21.15.67:0/4218391096 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd300b3ce0 msgr2=0x7fbd300b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.713+0000 7fbd4d5c4640 1 --2- 172.21.15.67:0/4218391096 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd300b3ce0 0x7fbd300b61f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009a60 tx=0x7fbd40005fd0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.713+0000 7fbd4d5c4640 1 -- 172.21.15.67:0/4218391096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd48112860 msgr2=0x7fbd481aa3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.713+0000 7fbd4d5c4640 1 --2- 172.21.15.67:0/4218391096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd48112860 0x7fbd481aa3b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00b3e0 tx=0x7fbd3c00b8a0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.713+0000 7fbd4d5c4640 1 -- 172.21.15.67:0/4218391096 shutdown_connections 2024-07-31T15:12:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.713+0000 7fbd4d5c4640 1 --2- 172.21.15.67:0/4218391096 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbd300b3ce0 0x7fbd300b61f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.713+0000 7fbd4d5c4640 1 --2- 172.21.15.67:0/4218391096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd48112860 0x7fbd481aa3b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.713+0000 7fbd4d5c4640 1 --2- 172.21.15.67:0/4218391096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd4810b960 0x7fbd48112320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.713+0000 7fbd4d5c4640 1 --2- 172.21.15.67:0/4218391096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd4810a700 0x7fbd48111de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.713+0000 7fbd4d5c4640 1 -- 172.21.15.67:0/4218391096 >> 172.21.15.67:0/4218391096 conn(0x7fbd4806e440 msgr2=0x7fbd48071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.713+0000 7fbd4d5c4640 1 -- 172.21.15.67:0/4218391096 shutdown_connections 2024-07-31T15:12:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:14.713+0000 7fbd4d5c4640 1 -- 172.21.15.67:0/4218391096 wait complete. 2024-07-31T15:12:14.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:14.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:14.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:14.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:14.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:14.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:14.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:14.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:14.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:14.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:14.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:14.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:14 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:12:15.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:15.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: pgmap v1076: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.3 MiB/s wr, 529 op/s 2024-07-31T15:12:15.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:15.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:15.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:15.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:15.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:15.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:15.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:15.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:15.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:15.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:15.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:15.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:15.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:15 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.565 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '53a8a385', quiesce took 35.6 seconds. Will release after: -18.6 2024-07-31T15:12:15.565 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:12:15.643 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:15.238+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d187 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi/ auth{1=1} v3040 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0) n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1) old_inodes=1 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:558842 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7c6c1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d187:[<0x1000000d116/ctxfi v1858>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d187:[<0x1000000d116/ctxfi v3040>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2701>,<0x10000000006/t v1739>,<0x10000000005/tmp v1665>,<0x100000001fb/client.0 v1633>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1598>,<0x10000000001/sv_1 v1598>,<0x10000000000/qa v1589>,<0x1/volumes v1539>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.452342+0000 30=30+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:15.643 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:15.240+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d119 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97/ auth{2=1} v3208 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0) n(v3 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1) old_inodes=3 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:558845 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce9047e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d119:[<0x1000000d116/ac97 v22>,<0x1000000cf1b/pci v463>,<0x10000000202/sound v8728>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000d119:[<0x1000000d116/ac97 v3208>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2701>,<0x10000000006/t v1739>,<0x10000000005/tmp v1665>,<0x100000001fb/client.0 v1633>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1598>,<0x10000000001/sv_1 v1598>,<0x10000000000/qa v1589>,<0x1/volumes v1539>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:00.491085+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:15.643 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:15.243+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d1cc [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1/ auth{2=1} v3042 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0) n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:558852 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8f176c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v1862>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v3042>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2701>,<0x10000000006/t v1739>,<0x10000000005/tmp v1665>,<0x100000001fb/client.0 v1633>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1598>,<0x10000000001/sv_1 v1598>,<0x10000000000/qa v1589>,<0x1/volumes v1539>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.549340+0000 22=22+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:15.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:12:15.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:15.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: pgmap v1076: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.3 MiB/s wr, 529 op/s 2024-07-31T15:12:15.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:15.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:15.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:15.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:15.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:15.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:15.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:15.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:15.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:15.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:15.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:15.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:15.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.646 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:15.251+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d180 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio/ auth{1=1} v3038 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0) n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1) old_inodes=1 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:558891 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8ebd2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v1860>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v3038>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2701>,<0x10000000006/t v1739>,<0x10000000005/tmp v1665>,<0x100000001fb/client.0 v1633>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1598>,<0x10000000001/sv_1 v1598>,<0x10000000000/qa v1589>,<0x1/volumes v1539>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.414343+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:15.646 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:15.351+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000cde2 [...10,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders/ auth{4=1} v112 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0) n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:545760 nref=3)) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8dc52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v29>,<0x1000000cbec/selinux v646>,<0x10000000202/scripts v8820>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v112>,<0x1000000cbec/selinux v749>,<0x10000000202/scripts v11639>,<0x10000000201/linux-6.5.11 v2701>,<0x10000000006/t v1739>,<0x10000000005/tmp v1665>,<0x100000001fb/client.0 v1633>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1598>,<0x10000000001/sv_1 v1598>,<0x10000000000/qa v1589>,<0x1/volumes v1539>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:58.999116+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:15.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:12:15.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:15.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: pgmap v1076: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.3 MiB/s wr, 529 op/s 2024-07-31T15:12:15.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:15.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:15.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:15.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:15.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:15.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:15.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:15.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:15.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:15.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:15.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:15.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:15.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:15 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:12:16.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:12:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:12:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:12:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:16.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:16 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.607 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:16.241+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000008171 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/power/supply/ auth{3=1} fragtree_t(*^3) v450 ap=3 RANDEPHEMERALPIN f(v2 m2024-07-31T15:11:41.904516+0000 130=130+0) n(v4 rc2024-07-31T15:11:41.905516+0000 b2648100 131=130+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:576864 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@6} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce89c56c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000008171:[<0x10000008145/supply v416>,<0x10000001d8f/power v16677>,<0x10000000202/drivers v3182>,<0x10000000201/linux-6.5.11 v445>,<0x10000000006/t v329>,<0x10000000005/tmp v299>,<0x100000001fb/client.0 v296>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v274>,<0x10000000001/sv_1 v299>,<0x10000000000/qa v280>,<0x1/volumes v280>]//[]","memoryvalue":"(2)0x10000008171:[<0x10000008145/supply v450>,<0x10000001d8f/power v19017>,<0x10000000202/drivers v21984>,<0x10000000201/linux-6.5.11 v2701>,<0x10000000006/t v1739>,<0x10000000005/tmp v1665>,<0x100000001fb/client.0 v1633>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1598>,<0x10000000001/sv_1 v1598>,<0x10000000000/qa v1589>,<0x1/volumes v1539>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 118=118+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:41.905516+0000 b2648100 131=130+1)","memory_value.dirstat":"f(v2 m2024-07-31T15:11:41.904516+0000 130=130+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:11:41.905516+0000 b2648100 131=130+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:16.607 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:16.256+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000021f0 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader/ auth{2=1} v893 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:04.418950+0000 19=18+1) n(v5 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:560750 nref=3)) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7dc21b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000021f0:[<0x100000021db/firmware_loader v414>,<0x10000001d8f/base v2486>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x100000021f0:[<0x100000021db/firmware_loader v893>,<0x10000001d8f/base v18747>,<0x10000000202/drivers v21984>,<0x10000000201/linux-6.5.11 v2701>,<0x10000000006/t v1739>,<0x10000000005/tmp v1665>,<0x100000001fb/client.0 v1633>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1598>,<0x10000000001/sv_1 v1598>,<0x10000000000/qa v1589>,<0x1/volumes v1539>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:01.469061+0000 13=12+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:04.418950+0000 19=18+1)","memory_value.rstat":"n(v5 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:16.607 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:16.319+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000006f7c [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/phy/ auth{3=1} fragtree_t(*^3) v3665 ap=3 RANDEPHEMERALPIN f(v3 m2024-07-31T15:10:56.134450+0000 111=110+1) n(v7 rc2024-07-31T15:10:57.456423+0000 b2820277 123=121+2) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:577097 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@11} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce86f32b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000006f7c:[<0x100000060b2/phy v1160>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x10000006f7c:[<0x100000060b2/phy v3665>,<0x10000001d8f/net v18816>,<0x10000000202/drivers v21984>,<0x10000000201/linux-6.5.11 v2701>,<0x10000000006/t v1739>,<0x10000000005/tmp v1665>,<0x100000001fb/client.0 v1633>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1598>,<0x10000000001/sv_1 v1598>,<0x10000000000/qa v1589>,<0x1/volumes v1539>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:10:56.134450+0000 103=102+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:10:57.456423+0000 b2820277 123=121+2)","memory_value.dirstat":"f(v3 m2024-07-31T15:10:56.134450+0000 111=110+1)","memory_value.rstat":"n(v7 rc2024-07-31T15:10:57.456423+0000 b2820277 123=121+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:16.608 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:16.358+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000093bf [...25,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/thermal/intel/ auth{4=1} v992 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:45.716439+0000 29=28+1) n(v6 rc2024-07-31T15:11:45.722439+0000 b416255 48=46+2) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:576806 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@6} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce81e7d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000093bf:[<0x100000093a3/intel v467>,<0x10000001d8f/thermal v17380>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x100000093bf:[<0x100000093a3/intel v992>,<0x10000001d8f/thermal v18812>,<0x10000000202/drivers v21984>,<0x10000000201/linux-6.5.11 v2701>,<0x10000000006/t v1739>,<0x10000000005/tmp v1665>,<0x100000001fb/client.0 v1633>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1598>,<0x10000000001/sv_1 v1598>,<0x10000000000/qa v1589>,<0x1/volumes v1539>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:32.789903+0000 17=16+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:45.722439+0000 b416255 48=46+2)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:45.716439+0000 29=28+1)","memory_value.rstat":"n(v6 rc2024-07-31T15:11:45.722439+0000 b416255 48=46+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:16.608 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:16.605+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000008a08 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip/ auth{3=1} v2681 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:09.647065+0000 7=7+0) n(v2 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:562428 nref=3)) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8c1aa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000008a08:[<0x1000000894f/microchip v613>,<0x10000001d8f/soc v16560>,<0x10000000202/drivers v3182>,<0x10000000201/linux-6.5.11 v445>,<0x10000000006/t v329>,<0x10000000005/tmp v299>,<0x100000001fb/client.0 v296>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v274>,<0x10000000001/sv_1 v299>,<0x10000000000/qa v280>,<0x1/volumes v280>]//[]","memoryvalue":"(2)0x10000008a08:[<0x1000000894f/microchip v2681>,<0x10000001d8f/soc v17222>,<0x10000000202/drivers v21984>,<0x10000000201/linux-6.5.11 v2701>,<0x10000000006/t v1739>,<0x10000000005/tmp v1665>,<0x100000001fb/client.0 v1633>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1598>,<0x10000000001/sv_1 v1598>,<0x10000000000/qa v1589>,<0x1/volumes v1539>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:51.041772+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:09.647065+0000 7=7+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:16.608 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.608 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.608 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.608 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.608 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:12:16.608 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.608 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:12:16.608 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.608 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.609 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.610 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.610 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.610 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.610 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.610 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.610 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.610 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.610 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.610 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.610 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.610 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.610 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.610 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.611 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.611 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.611 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:16.611 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:12:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:12:16.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:12:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:16.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:16 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:16.862 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:16.729+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001fc7 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica/ auth{2=1} fragtree_t(*^3) v1950 ap=3 RANDEPHEMERALPIN f(v10 m2024-07-31T15:07:39.153464+0000 516=516+0) n(v16 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1) old_inodes=4 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:563100 nref=3)) caps={15489=pAsLsXs/-@28} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7f0ba680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000001fc7:[<0x10000001fad/acpica v1734>,<0x10000001d8f/acpi v2434>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x10000001fc7:[<0x10000001fad/acpica v1950>,<0x10000001d8f/acpi v18672>,<0x10000000202/drivers v21984>,<0x10000000201/linux-6.5.11 v2701>,<0x10000000006/t v1739>,<0x10000000005/tmp v1665>,<0x100000001fb/client.0 v1633>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1598>,<0x10000000001/sv_1 v1598>,<0x10000000000/qa v1589>,<0x1/volumes v1539>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:07:37.321501+0000 494=494+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1)","memory_value.dirstat":"f(v10 m2024-07-31T15:07:39.153464+0000 516=516+0)","memory_value.rstat":"n(v16 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:16.862 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:16.786+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009462 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc/ auth{3=1} v1424 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:28.223687+0000 22=22+0) n(v4 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:563927 nref=3)) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce85c96b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009462:[<0x1000000945c/hvc v352>,<0x10000001d8f/tty v18356>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009462:[<0x1000000945c/hvc v1424>,<0x10000001d8f/tty v18901>,<0x10000000202/drivers v21984>,<0x10000000201/linux-6.5.11 v2701>,<0x10000000006/t v1739>,<0x10000000005/tmp v1665>,<0x100000001fb/client.0 v1633>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1598>,<0x10000000001/sv_1 v1598>,<0x10000000000/qa v1589>,<0x1/volumes v1539>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:32.992899+0000 16=16+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:28.223687+0000 22=22+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:16.862 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:16.860+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005d80 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader/ auth{4=1} v1324 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:10.777820+0000 21=21+0) n(v3 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1) old_inodes=1 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:566582 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7f8f5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000005d80:[<0x10000005d5d/cardreader v104>,<0x10000001d8f/misc v13245>,<0x10000000202/drivers v2910>,<0x10000000201/linux-6.5.11 v379>,<0x10000000006/t v275>,<0x10000000005/tmp v244>,<0x100000001fb/client.0 v243>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v221>,<0x10000000001/sv_1 v249>,<0x10000000000/qa v233>,<0x1/volumes v234>]//[]","memoryvalue":"(2)0x10000005d80:[<0x10000005d5d/cardreader v1324>,<0x10000001d8f/misc v18516>,<0x10000000202/drivers v21984>,<0x10000000201/linux-6.5.11 v2701>,<0x10000000006/t v1739>,<0x10000000005/tmp v1665>,<0x100000001fb/client.0 v1633>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1598>,<0x10000000001/sv_1 v1598>,<0x10000000000/qa v1589>,<0x1/volumes v1539>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:21.430390+0000 17=17+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:10.777820+0000 21=21+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:17.142 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:16.976+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000054d1 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media/common/v4l2-tpg/ auth{4=1} v342 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:36.520626+0000 8=8+0) n(v3 rc2024-07-31T15:11:36.521626+0000 b175839 9=8+1) old_inodes=1 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:579215 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@6} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7d165080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000054d1:[<0x100000054a1/v4l2-tpg v130>,<0x10000005460/common v1799>,<0x10000001d8f/media v12887>,<0x10000000202/drivers v2910>,<0x10000000201/linux-6.5.11 v379>,<0x10000000006/t v275>,<0x10000000005/tmp v244>,<0x100000001fb/client.0 v243>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v221>,<0x10000000001/sv_1 v249>,<0x10000000000/qa v233>,<0x1/volumes v234>]//[]","memoryvalue":"(2)0x100000054d1:[<0x100000054a1/v4l2-tpg v342>,<0x10000005460/common v3457>,<0x10000001d8f/media v18769>,<0x10000000202/drivers v21984>,<0x10000000201/linux-6.5.11 v2701>,<0x10000000006/t v1739>,<0x10000000005/tmp v1665>,<0x100000001fb/client.0 v1633>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1598>,<0x10000000001/sv_1 v1598>,<0x10000000000/qa v1589>,<0x1/volumes v1539>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:12.842569+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:36.521626+0000 b175839 9=8+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:36.520626+0000 8=8+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:11:36.521626+0000 b175839 9=8+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:17.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: pgmap v1077: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.1 MiB/s wr, 653 op/s 2024-07-31T15:12:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000008171 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/power/supply) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000006f7c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/phy) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000093bf (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/thermal/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000054d1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media/common/v4l2-tpg) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:17.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: pgmap v1077: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.1 MiB/s wr, 653 op/s 2024-07-31T15:12:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000008171 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/power/supply) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000006f7c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/phy) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000093bf (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/thermal/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000054d1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media/common/v4l2-tpg) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:17.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: pgmap v1077: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.1 MiB/s wr, 653 op/s 2024-07-31T15:12:17.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000008171 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/power/supply) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000006f7c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/phy) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000093bf (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/thermal/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:17.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000054d1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media/common/v4l2-tpg) see mds.b log and `damage ls` output for details 2024-07-31T15:12:17.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:17.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:18.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.620 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.620 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.620 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.620 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.622 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.622 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.622 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.622 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.622 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.622 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.622 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.622 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.622 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.622 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.622 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.622 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.623 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.623 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.623 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.623 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.623 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.623 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.623 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.623 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.623 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.623 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.623 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.623 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.623 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.624 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:18.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:18.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:18.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:18.617+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000074a1 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/wireless/marvell/ auth{2=1} v1362 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:00.438362+0000 10=7+3) n(v6 rc2024-07-31T15:11:00.441362+0000 b1951078 100=96+4) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:581454 nref=3)) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8700c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000074a1:[<0x1000000708f/marvell v338>,<0x100000060b2/wireless v1686>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x100000074a1:[<0x1000000708f/marvell v1362>,<0x100000060b2/wireless v3767>,<0x10000001d8f/net v18816>,<0x10000000202/drivers v21986>,<0x10000000201/linux-6.5.11 v2703>,<0x10000000006/t v1741>,<0x10000000005/tmp v1667>,<0x100000001fb/client.0 v1635>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1600>,<0x10000000001/sv_1 v1600>,<0x10000000000/qa v1591>,<0x1/volumes v1541>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:33.434139+0000 6=3+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:00.441362+0000 b1951078 100=96+4)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:00.438362+0000 10=7+3)","memory_value.rstat":"n(v6 rc2024-07-31T15:11:00.441362+0000 b1951078 100=96+4)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:18.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:18.668+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000006af6 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/microsoft/ auth{4=1} v2613 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:38.452587+0000 7=6+1) n(v5 rc2024-07-31T15:11:38.453587+0000 b144243 15=13+2) old_inodes=1 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:581544 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@6} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce81722680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000006af6:[<0x1000000622a/microsoft v2181>,<0x100000060b2/ethernet v1684>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x10000006af6:[<0x1000000622a/microsoft v2613>,<0x100000060b2/ethernet v3773>,<0x10000001d8f/net v18816>,<0x10000000202/drivers v21986>,<0x10000000201/linux-6.5.11 v2703>,<0x10000000006/t v1741>,<0x10000000005/tmp v1667>,<0x100000001fb/client.0 v1635>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1600>,<0x10000000001/sv_1 v1600>,<0x10000000000/qa v1591>,<0x1/volumes v1541>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:28.579240+0000 3=2+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:38.453587+0000 b144243 15=13+2)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:38.452587+0000 7=6+1)","memory_value.rstat":"n(v5 rc2024-07-31T15:11:38.453587+0000 b144243 15=13+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:18.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:18 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:18.681+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000006bc8 [...25,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/pensando/ auth{2=1} v2564 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:48.960373+0000 7=6+1) n(v5 rc2024-07-31T15:11:48.962373+0000 b410323 33=31+2) old_inodes=2 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:581571 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@6} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce80ef8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000006bc8:[<0x1000000622a/pensando v2175>,<0x100000060b2/ethernet v1684>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x10000006bc8:[<0x1000000622a/pensando v2564>,<0x100000060b2/ethernet v3773>,<0x10000001d8f/net v18816>,<0x10000000202/drivers v21986>,<0x10000000201/linux-6.5.11 v2703>,<0x10000000006/t v1741>,<0x10000000005/tmp v1667>,<0x100000001fb/client.0 v1635>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1600>,<0x10000000001/sv_1 v1600>,<0x10000000000/qa v1591>,<0x1/volumes v1541>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:29.021231+0000 3=2+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:48.962373+0000 b410323 33=31+2)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:48.960373+0000 7=6+1)","memory_value.rstat":"n(v5 rc2024-07-31T15:11:48.962373+0000 b410323 33=31+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.262+0000 7ff00a983640 1 -- 172.21.15.67:0/2471061018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff004108720 msgr2=0x7ff00410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.262+0000 7ff00a983640 1 --2- 172.21.15.67:0/2471061018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff004108720 0x7ff00410ef50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7feffc009ca0 tx=0x7feffc02a8c0 comp rx=0 tx=0).stop 2024-07-31T15:12:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.262+0000 7ff00a983640 1 -- 172.21.15.67:0/2471061018 shutdown_connections 2024-07-31T15:12:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.262+0000 7ff00a983640 1 --2- 172.21.15.67:0/2471061018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff004108720 0x7ff00410ef50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.262+0000 7ff00a983640 1 --2- 172.21.15.67:0/2471061018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff004107dd0 0x7ff0041081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.262+0000 7ff00a983640 1 --2- 172.21.15.67:0/2471061018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff004102490 0x7ff0041028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.262+0000 7ff00a983640 1 -- 172.21.15.67:0/2471061018 >> 172.21.15.67:0/2471061018 conn(0x7ff004079160 msgr2=0x7ff004079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.262+0000 7ff00a983640 1 -- 172.21.15.67:0/2471061018 shutdown_connections 2024-07-31T15:12:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.263+0000 7ff00a983640 1 -- 172.21.15.67:0/2471061018 wait complete. 2024-07-31T15:12:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.263+0000 7ff00a983640 1 Processor -- start 2024-07-31T15:12:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.263+0000 7ff00a983640 1 -- start start 2024-07-31T15:12:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff00a983640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff004102490 0x7ff004198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff00a983640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff004107dd0 0x7ff004199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff00a983640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff004108720 0x7ff00419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff00a983640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041123a0 con 0x7ff004107dd0 2024-07-31T15:12:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff00a983640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff004112220 con 0x7ff004102490 2024-07-31T15:12:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff00a983640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff004112520 con 0x7ff004108720 2024-07-31T15:12:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff008f20640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff004108720 0x7ff00419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff003fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff004102490 0x7ff004198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff003fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff004102490 0x7ff004198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46984/0 (socket says 172.21.15.67:46984) 2024-07-31T15:12:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff003fff640 1 -- 172.21.15.67:0/2244665880 learned_addr learned my addr 172.21.15.67:0/2244665880 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff008f20640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff004108720 0x7ff00419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49604/0 (socket says 172.21.15.67:49604) 2024-07-31T15:12:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff003fff640 1 -- 172.21.15.67:0/2244665880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff004108720 msgr2=0x7ff00419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff003fff640 1 --2- 172.21.15.67:0/2244665880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff004108720 0x7ff00419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff003fff640 1 -- 172.21.15.67:0/2244665880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff004107dd0 msgr2=0x7ff004199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff003fff640 1 --2- 172.21.15.67:0/2244665880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff004107dd0 0x7ff004199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.264+0000 7ff003fff640 1 -- 172.21.15.67:0/2244665880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00419ece0 con 0x7ff004102490 2024-07-31T15:12:19.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.265+0000 7ff003fff640 1 --2- 172.21.15.67:0/2244665880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff004102490 0x7ff004198d70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7feff400d770 tx=0x7feff400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:19.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.267+0000 7ff0017fa640 1 -- 172.21.15.67:0/2244665880 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4017070 con 0x7ff004102490 2024-07-31T15:12:19.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.268+0000 7ff0017fa640 1 -- 172.21.15.67:0/2244665880 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff40054d0 con 0x7ff004102490 2024-07-31T15:12:19.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.268+0000 7ff00a983640 1 -- 172.21.15.67:0/2244665880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00419efd0 con 0x7ff004102490 2024-07-31T15:12:19.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.268+0000 7ff0017fa640 1 -- 172.21.15.67:0/2244665880 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4004c30 con 0x7ff004102490 2024-07-31T15:12:19.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.268+0000 7ff00a983640 1 -- 172.21.15.67:0/2244665880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00406a870 con 0x7ff004102490 2024-07-31T15:12:19.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.269+0000 7ff00a983640 1 -- 172.21.15.67:0/2244665880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff004103710 con 0x7ff004102490 2024-07-31T15:12:19.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.271+0000 7ff0017fa640 1 -- 172.21.15.67:0/2244665880 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7feff4004dd0 con 0x7ff004102490 2024-07-31T15:12:19.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.271+0000 7ff0017fa640 1 --2- 172.21.15.67:0/2244665880 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fefec0b3dc0 0x7fefec0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:19.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.272+0000 7ff0037fe640 1 --2- 172.21.15.67:0/2244665880 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fefec0b3dc0 0x7fefec0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:19.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.272+0000 7ff0017fa640 1 -- 172.21.15.67:0/2244665880 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7feff40e41e0 con 0x7ff004102490 2024-07-31T15:12:19.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.272+0000 7ff0037fe640 1 --2- 172.21.15.67:0/2244665880 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fefec0b3dc0 0x7fefec0b62d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7feff8005d00 tx=0x7feff8005c50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:19.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.277+0000 7ff0017fa640 1 -- 172.21.15.67:0/2244665880 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7feff40ab1b0 con 0x7ff004102490 2024-07-31T15:12:19.392 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:19.129+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000948f [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250/ auth{4=1} v716 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:35.607536+0000 75=75+0) n(v5 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:566425 nref=3)) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce836e4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000948f:[<0x1000000948d/8250 v533>,<0x1000000945c/serial v657>,<0x10000001d8f/tty v18356>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x1000000948f:[<0x1000000948d/8250 v716>,<0x1000000945c/serial v1492>,<0x10000001d8f/tty v18901>,<0x10000000202/drivers v21986>,<0x10000000201/linux-6.5.11 v2703>,<0x10000000006/t v1741>,<0x10000000005/tmp v1667>,<0x100000001fb/client.0 v1635>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1600>,<0x10000000001/sv_1 v1600>,<0x10000000000/qa v1591>,<0x1/volumes v1541>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:33.063897+0000 47=47+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:35.607536+0000 75=75+0)","memory_value.rstat":"n(v5 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: pgmap v1078: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1010 KiB/s wr, 1.58k op/s 2024-07-31T15:12:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.394 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000074a1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/wireless/marvell) see mds.b log and `damage ls` output for details 2024-07-31T15:12:19.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.395 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000006af6 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/microsoft) see mds.b log and `damage ls` output for details 2024-07-31T15:12:19.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000006bc8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/pensando) see mds.b log and `damage ls` output for details 2024-07-31T15:12:19.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:19.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.396 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:12:19.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: pgmap v1078: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1010 KiB/s wr, 1.58k op/s 2024-07-31T15:12:19.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000074a1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/wireless/marvell) see mds.b log and `damage ls` output for details 2024-07-31T15:12:19.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000006af6 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/microsoft) see mds.b log and `damage ls` output for details 2024-07-31T15:12:19.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000006bc8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/pensando) see mds.b log and `damage ls` output for details 2024-07-31T15:12:19.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:19.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:12:19.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.560+0000 7ff00a983640 1 -- 172.21.15.67:0/2244665880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff00406b3e0 con 0x7ff004102490 2024-07-31T15:12:19.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.561+0000 7ff0017fa640 1 -- 172.21.15.67:0/2244665880 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11959 (secure 0 0 0) 0x7feff40aaab0 con 0x7ff004102490 2024-07-31T15:12:19.563 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:12:19.563 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":66,"btime":"2024-07-31T15:10:30:235518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:10:30.235494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29129":{"gid":29129,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/631186801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":631186801},{"type":"v1","addr":"172.21.15.73:6835","nonce":631186801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:12:19.563 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 66 2024-07-31T15:12:19.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.563+0000 7ff00a983640 1 -- 172.21.15.67:0/2244665880 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fefec0b3dc0 msgr2=0x7fefec0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:19.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.563+0000 7ff00a983640 1 --2- 172.21.15.67:0/2244665880 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fefec0b3dc0 0x7fefec0b62d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7feff8005d00 tx=0x7feff8005c50 comp rx=0 tx=0).stop 2024-07-31T15:12:19.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.563+0000 7ff00a983640 1 -- 172.21.15.67:0/2244665880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff004102490 msgr2=0x7ff004198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:19.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.563+0000 7ff00a983640 1 --2- 172.21.15.67:0/2244665880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff004102490 0x7ff004198d70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7feff400d770 tx=0x7feff400dc30 comp rx=0 tx=0).stop 2024-07-31T15:12:19.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.564+0000 7ff00a983640 1 -- 172.21.15.67:0/2244665880 shutdown_connections 2024-07-31T15:12:19.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.564+0000 7ff00a983640 1 --2- 172.21.15.67:0/2244665880 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fefec0b3dc0 0x7fefec0b62d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:19.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.564+0000 7ff00a983640 1 --2- 172.21.15.67:0/2244665880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff004108720 0x7ff00419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:19.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.564+0000 7ff00a983640 1 --2- 172.21.15.67:0/2244665880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff004107dd0 0x7ff004199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:19.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.564+0000 7ff00a983640 1 --2- 172.21.15.67:0/2244665880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff004102490 0x7ff004198d70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:19.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.564+0000 7ff00a983640 1 -- 172.21.15.67:0/2244665880 >> 172.21.15.67:0/2244665880 conn(0x7ff004079160 msgr2=0x7ff004103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:19.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.564+0000 7ff00a983640 1 -- 172.21.15.67:0/2244665880 shutdown_connections 2024-07-31T15:12:19.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:19.564+0000 7ff00a983640 1 -- 172.21.15.67:0/2244665880 wait complete. 2024-07-31T15:12:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: pgmap v1078: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1010 KiB/s wr, 1.58k op/s 2024-07-31T15:12:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000074a1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/wireless/marvell) see mds.b log and `damage ls` output for details 2024-07-31T15:12:19.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000006af6 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/microsoft) see mds.b log and `damage ls` output for details 2024-07-31T15:12:19.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000006bc8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/pensando) see mds.b log and `damage ls` output for details 2024-07-31T15:12:19.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:19.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:19.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:12:20.142 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:19 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:19.796+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000006803 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/marvell/octeontx2/ auth{2=1} v383 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:38.527585+0000 8=6+2) n(v5 rc2024-07-31T15:11:38.528585+0000 b1832909 74=71+3) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:589280 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@6} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8265c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000006803:[<0x100000067e0/octeontx2 v159>,<0x1000000622a/marvell v2175>,<0x100000060b2/ethernet v1684>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x10000006803:[<0x100000067e0/octeontx2 v383>,<0x1000000622a/marvell v2657>,<0x100000060b2/ethernet v3783>,<0x10000001d8f/net v18821>,<0x10000000202/drivers v21991>,<0x10000000201/linux-6.5.11 v2708>,<0x10000000006/t v1747>,<0x10000000005/tmp v1670>,<0x100000001fb/client.0 v1637>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1602>,<0x10000000001/sv_1 v1603>,<0x10000000000/qa v1594>,<0x1/volumes v1543>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:27.598261+0000 4=2+2)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:38.528585+0000 b1832909 74=71+3)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:38.527585+0000 8=6+2)","memory_value.rstat":"n(v5 rc2024-07-31T15:11:38.528585+0000 b1832909 74=71+3)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:20.335 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 53a8a385 --release --await' 2024-07-31T15:12:20.336 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 53a8a385 --release --await 2024-07-31T15:12:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2244665880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:12:20.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000006803 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/marvell/octeontx2) see mds.b log and `damage ls` output for details 2024-07-31T15:12:20.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:20.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:20 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:12:20.642 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:12:20.161+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000439c [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du/ auth{2=1} v303 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:01.164016+0000 34=34+0) n(v3 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:590732 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce83f19600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000439c:[<0x10000004399/rcar-du v163>,<0x10000002d23/renesas v2988>,<0x10000002d21/drm v1196>,<0x10000001d8f/gpu v9523>,<0x10000000202/drivers v2799>,<0x10000000201/linux-6.5.11 v334>,<0x10000000006/t v232>,<0x10000000005/tmp v202>,<0x100000001fb/client.0 v201>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v184>,<0x10000000001/sv_1 v213>,<0x10000000000/qa v197>,<0x1/volumes v198>]//[]","memoryvalue":"(2)0x1000000439c:[<0x10000004399/rcar-du v303>,<0x10000002d23/renesas v3147>,<0x10000002d21/drm v1869>,<0x10000001d8f/gpu v18947>,<0x10000000202/drivers v21991>,<0x10000000201/linux-6.5.11 v2708>,<0x10000000006/t v1747>,<0x10000000005/tmp v1670>,<0x100000001fb/client.0 v1637>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1602>,<0x10000000001/sv_1 v1603>,<0x10000000000/qa v1594>,<0x1/volumes v1543>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:38.082295+0000 30=30+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:01.164016+0000 34=34+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:12:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.649 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:12:20.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2244665880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:12:20.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000006803 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/marvell/octeontx2) see mds.b log and `damage ls` output for details 2024-07-31T15:12:20.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:20.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:20 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:12:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2244665880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:12:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000006803 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/marvell/octeontx2) see mds.b log and `damage ls` output for details 2024-07-31T15:12:20.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:20.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:20.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:20 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:12:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: pgmap v1079: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 791 KiB/s wr, 1.95k op/s 2024-07-31T15:12:21.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: osdmap e132: 12 total, 12 up, 12 in 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:21.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.640 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.640 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.640 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.640 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.640 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.640 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.640 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.641 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.641 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: pgmap v1079: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 791 KiB/s wr, 1.95k op/s 2024-07-31T15:12:21.641 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.641 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.641 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.641 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.641 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.641 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.641 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.641 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.641 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.642 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.642 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.642 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.642 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.642 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.642 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: osdmap e132: 12 total, 12 up, 12 in 2024-07-31T15:12:21.642 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.643 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.643 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.643 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.643 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.643 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.643 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.643 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.643 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.643 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.643 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.643 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.643 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.644 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.644 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.644 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.644 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.644 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.644 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.644 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.644 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.644 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.644 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.644 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.644 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.644 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.644 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.645 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.645 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.645 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.645 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.645 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.645 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.645 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.645 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.645 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.645 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.645 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.645 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.645 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.645 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.646 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.646 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:21.646 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: pgmap v1079: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 791 KiB/s wr, 1.95k op/s 2024-07-31T15:12:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: osdmap e132: 12 total, 12 up, 12 in 2024-07-31T15:12:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:21.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:21.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.483 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.492 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.492 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.492 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.492 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:22.492 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.492 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.749 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.750 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.750 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.750 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.750 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.750 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.750 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.750 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.750 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.750 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.750 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:22.750 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.750 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:22.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.481+0000 7f51c1799640 1 -- 172.21.15.67:0/3182581265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51bc102490 msgr2=0x7f51bc1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:22.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.481+0000 7f51c1799640 1 --2- 172.21.15.67:0/3182581265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51bc102490 0x7f51bc1028a0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009900 tx=0x7f51ac02a410 comp rx=0 tx=0).stop 2024-07-31T15:12:22.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.481+0000 7f51c1799640 1 -- 172.21.15.67:0/3182581265 shutdown_connections 2024-07-31T15:12:22.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.481+0000 7f51c1799640 1 --2- 172.21.15.67:0/3182581265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51bc108720 0x7f51bc10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.481+0000 7f51c1799640 1 --2- 172.21.15.67:0/3182581265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51bc107dd0 0x7f51bc1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.481+0000 7f51c1799640 1 --2- 172.21.15.67:0/3182581265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51bc102490 0x7f51bc1028a0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.481+0000 7f51c1799640 1 -- 172.21.15.67:0/3182581265 >> 172.21.15.67:0/3182581265 conn(0x7f51bc079160 msgr2=0x7f51bc079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:22.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.481+0000 7f51c1799640 1 -- 172.21.15.67:0/3182581265 shutdown_connections 2024-07-31T15:12:22.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.481+0000 7f51c1799640 1 -- 172.21.15.67:0/3182581265 wait complete. 2024-07-31T15:12:22.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.482+0000 7f51c1799640 1 Processor -- start 2024-07-31T15:12:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.482+0000 7f51c1799640 1 -- start start 2024-07-31T15:12:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.482+0000 7f51c1799640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51bc102490 0x7f51bc198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.482+0000 7f51c1799640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51bc107dd0 0x7f51bc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.482+0000 7f51c1799640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51bc108720 0x7f51bc19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.482+0000 7f51c1799640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc112380 con 0x7f51bc108720 2024-07-31T15:12:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.482+0000 7f51c1799640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc112200 con 0x7f51bc102490 2024-07-31T15:12:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.482+0000 7f51c1799640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc112500 con 0x7f51bc107dd0 2024-07-31T15:12:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51bb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51bc108720 0x7f51bc19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51bb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51bc108720 0x7f51bc19e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49276/0 (socket says 172.21.15.67:49276) 2024-07-31T15:12:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51bb7fe640 1 -- 172.21.15.67:0/1901350796 learned_addr learned my addr 172.21.15.67:0/1901350796 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51baffd640 1 --2- 172.21.15.67:0/1901350796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51bc102490 0x7f51bc198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51ba7fc640 1 --2- 172.21.15.67:0/1901350796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51bc107dd0 0x7f51bc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51bb7fe640 1 -- 172.21.15.67:0/1901350796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51bc107dd0 msgr2=0x7f51bc199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51bb7fe640 1 --2- 172.21.15.67:0/1901350796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51bc107dd0 0x7f51bc199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51bb7fe640 1 -- 172.21.15.67:0/1901350796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51bc102490 msgr2=0x7f51bc198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51bb7fe640 1 --2- 172.21.15.67:0/1901350796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51bc102490 0x7f51bc198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51bb7fe640 1 -- 172.21.15.67:0/1901350796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51bc19ec40 con 0x7f51bc108720 2024-07-31T15:12:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51bb7fe640 1 --2- 172.21.15.67:0/1901350796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51bc108720 0x7f51bc19e5d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f51b400eaa0 tx=0x7f51b400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51a3fff640 1 -- 172.21.15.67:0/1901350796 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b4004d20 con 0x7f51bc108720 2024-07-31T15:12:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51c1799640 1 -- 172.21.15.67:0/1901350796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51bc19ef30 con 0x7f51bc108720 2024-07-31T15:12:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51a3fff640 1 -- 172.21.15.67:0/1901350796 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51b4005020 con 0x7f51bc108720 2024-07-31T15:12:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51a3fff640 1 -- 172.21.15.67:0/1901350796 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b40116b0 con 0x7f51bc108720 2024-07-31T15:12:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.483+0000 7f51c1799640 1 -- 172.21.15.67:0/1901350796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51bc1a5750 con 0x7f51bc108720 2024-07-31T15:12:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.486+0000 7f51c7cea740 1 Processor -- start 2024-07-31T15:12:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.486+0000 7f51a3fff640 1 -- 172.21.15.67:0/1901350796 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f51b40051d0 con 0x7f51bc108720 2024-07-31T15:12:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.486+0000 7f51c7cea740 1 -- start start 2024-07-31T15:12:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51c7cea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f2433520 0x55d5f27d9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51c7cea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f240a660 0x55d5f27d95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51c7cea740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27d9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51c7cea740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d5f274ee70 con 0x55d5f240a660 2024-07-31T15:12:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51c7cea740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d5f273e370 con 0x55d5f2412840 2024-07-31T15:12:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51c7cea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d5f27423b0 con 0x55d5f2433520 2024-07-31T15:12:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51baffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f240a660 0x55d5f27d95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51baffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f240a660 0x55d5f27d95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49284/0 (socket says 172.21.15.67:49284) 2024-07-31T15:12:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51baffd640 1 -- 172.21.15.67:0/2871139312 learned_addr learned my addr 172.21.15.67:0/2871139312 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51ba7fc640 1 --2- 172.21.15.67:0/2871139312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27d9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51ba7fc640 1 --2- 172.21.15.67:0/2871139312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f2433520 0x55d5f27d9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51baffd640 1 -- 172.21.15.67:0/2871139312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f2433520 msgr2=0x55d5f27d9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51baffd640 1 --2- 172.21.15.67:0/2871139312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f2433520 0x55d5f27d9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51baffd640 1 -- 172.21.15.67:0/2871139312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 msgr2=0x55d5f27d9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51baffd640 1 --2- 172.21.15.67:0/2871139312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27d9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51baffd640 1 -- 172.21.15.67:0/2871139312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f270b040 con 0x55d5f240a660 2024-07-31T15:12:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51ba7fc640 1 --2- 172.21.15.67:0/2871139312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f2433520 0x55d5f27d9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:12:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51a3fff640 1 --2- 172.21.15.67:0/1901350796 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f51a40b3d70 0x7f51a40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51baffd640 1 --2- 172.21.15.67:0/2871139312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f240a660 0x55d5f27d95b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009900 tx=0x7f51ac032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4ca36d51cbb690 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.487+0000 7f51ba7fc640 1 --2- 172.21.15.67:0/1901350796 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f51a40b3d70 0x7f51a40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.488+0000 7f51a3fff640 1 -- 172.21.15.67:0/1901350796 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f51b40e44a0 con 0x7f51bc108720 2024-07-31T15:12:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.488+0000 7f51a17fa640 1 -- 172.21.15.67:0/2871139312 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac040070 con 0x55d5f240a660 2024-07-31T15:12:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.488+0000 7f51a17fa640 1 -- 172.21.15.67:0/2871139312 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ac0316a0 con 0x55d5f240a660 2024-07-31T15:12:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.488+0000 7f51a17fa640 1 -- 172.21.15.67:0/2871139312 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac035040 con 0x55d5f240a660 2024-07-31T15:12:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.489+0000 7f51c7cea740 1 -- 172.21.15.67:0/2871139312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f240a660 msgr2=0x55d5f27d95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.489+0000 7f51ba7fc640 1 --2- 172.21.15.67:0/1901350796 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f51a40b3d70 0x7f51a40b6280 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f51bc100680 tx=0x7f51b000a6c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.489+0000 7f51c7cea740 1 --2- 172.21.15.67:0/2871139312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f240a660 0x55d5f27d95b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009900 tx=0x7f51ac032e60 comp rx=0 tx=0).stop 2024-07-31T15:12:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.489+0000 7f51c7cea740 1 -- 172.21.15.67:0/2871139312 shutdown_connections 2024-07-31T15:12:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.489+0000 7f51c7cea740 1 --2- 172.21.15.67:0/2871139312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27d9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.489+0000 7f51c7cea740 1 --2- 172.21.15.67:0/2871139312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f240a660 0x55d5f27d95b0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.489+0000 7f51c7cea740 1 --2- 172.21.15.67:0/2871139312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f2433520 0x55d5f27d9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.489+0000 7f51c7cea740 1 -- 172.21.15.67:0/2871139312 >> 172.21.15.67:0/2871139312 conn(0x55d5f2427910 msgr2=0x55d5f27d6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.489+0000 7f51c7cea740 1 -- 172.21.15.67:0/2871139312 shutdown_connections 2024-07-31T15:12:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.489+0000 7f51c7cea740 1 -- 172.21.15.67:0/2871139312 wait complete. 2024-07-31T15:12:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.489+0000 7f51c7cea740 1 -- start start 2024-07-31T15:12:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.489+0000 7f51c7cea740 1 Processor -- start 2024-07-31T15:12:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51c7cea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f240a660 0x55d5f27e6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51c7cea740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27e7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51c7cea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f2433520 0x55d5f27f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51c7cea740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d5f25a1bb0 con 0x55d5f2433520 2024-07-31T15:12:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51c7cea740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d5f26e0ac0 con 0x55d5f2412840 2024-07-31T15:12:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51c7cea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d5f27e9950 con 0x55d5f240a660 2024-07-31T15:12:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51c7cea740 10 client.0 _ll_register_callbacks cb 0x55d5f26e93b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51bb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f2433520 0x55d5f27f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51bb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f2433520 0x55d5f27f2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49294/0 (socket says 172.21.15.67:49294) 2024-07-31T15:12:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51bb7fe640 1 -- 172.21.15.67:0/1611354779 learned_addr learned my addr 172.21.15.67:0/1611354779 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51baffd640 1 --2- 172.21.15.67:0/1611354779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f240a660 0x55d5f27e6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51ba7fc640 1 --2- 172.21.15.67:0/1611354779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51bb7fe640 1 -- 172.21.15.67:0/1611354779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f240a660 msgr2=0x55d5f27e6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51bb7fe640 1 --2- 172.21.15.67:0/1611354779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f240a660 0x55d5f27e6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51bb7fe640 1 -- 172.21.15.67:0/1611354779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 msgr2=0x55d5f27e7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51bb7fe640 1 --2- 172.21.15.67:0/1611354779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27e7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51bb7fe640 1 -- 172.21.15.67:0/1611354779 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5f249eb00 con 0x55d5f2433520 2024-07-31T15:12:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51ba7fc640 1 --2- 172.21.15.67:0/1611354779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27e7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:12:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51bb7fe640 1 --2- 172.21.15.67:0/1611354779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f2433520 0x55d5f27f2c80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f51b40a6dc0 tx=0x7f51b4005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac999d107ff25ca server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f519eff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:12:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f519eff5640 1 -- 172.21.15.67:0/1611354779 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b401f040 con 0x55d5f2433520 2024-07-31T15:12:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f519eff5640 1 -- 172.21.15.67:0/1611354779 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f51b40c5580 con 0x55d5f2433520 2024-07-31T15:12:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.490+0000 7f51c7cea740 1 -- 172.21.15.67:0/1611354779 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51ac0316a0 con 0x55d5f2433520 2024-07-31T15:12:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.491+0000 7f519eff5640 1 -- 172.21.15.67:0/1611354779 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f51b40af340 con 0x55d5f2433520 2024-07-31T15:12:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.491+0000 7f51c7cea740 10 client.19956 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.491+0000 7f51c7cea740 1 -- 172.21.15.67:0/1611354779 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55d5f2515960 con 0x55d5f2433520 2024-07-31T15:12:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.491+0000 7f519eff5640 1 -- 172.21.15.67:0/1611354779 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ac0316a0 con 0x55d5f2433520 2024-07-31T15:12:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.491+0000 7f519eff5640 1 -- 172.21.15.67:0/1611354779 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f51b400f040 con 0x55d5f2433520 2024-07-31T15:12:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.491+0000 7f51c7cea740 10 client.19956 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.491+0000 7f51c7cea740 10 client.19956 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:12:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.491+0000 7f51c7cea740 20 client.19956 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.491+0000 7f51c7cea740 1 --2- 172.21.15.67:0/1611354779 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5f23f1ea0 0x55d5f27f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.491+0000 7f51c7cea740 4 client.19956 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:12:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.491+0000 7f51c7cea740 1 -- 172.21.15.67:0/1611354779 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5f2750dc0 con 0x55d5f23f1ea0 2024-07-31T15:12:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.491+0000 7f51baffd640 1 --2- 172.21.15.67:0/1611354779 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5f23f1ea0 0x55d5f27f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.492+0000 7f51baffd640 1 --2- 172.21.15.67:0/1611354779 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5f23f1ea0 0x55d5f27f8c30 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=198aa6ebd57f6012 server_cookie=1f2ece657725fb7a in_seq=0 out_seq=0 2024-07-31T15:12:22.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.492+0000 7f519eff5640 10 client.19956 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:22.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.492+0000 7f519eff5640 1 -- 172.21.15.67:0/1611354779 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5f2750dc0 con 0x55d5f23f1ea0 2024-07-31T15:12:22.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.492+0000 7f519eff5640 10 client.19956 handle_command_reply: tid=0 2024-07-31T15:12:22.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.492+0000 7f51c7cea740 1 client.19956 shutdown 2024-07-31T15:12:22.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.492+0000 7f51c7cea740 1 -- 172.21.15.67:0/1611354779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f2433520 msgr2=0x55d5f27f2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:22.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.492+0000 7f51c7cea740 1 --2- 172.21.15.67:0/1611354779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f2433520 0x55d5f27f2c80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f51b40a6dc0 tx=0x7f51b4005720 comp rx=0 tx=0).stop 2024-07-31T15:12:22.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.493+0000 7f51c7cea740 1 -- 172.21.15.67:0/1611354779 shutdown_connections 2024-07-31T15:12:22.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.493+0000 7f51c7cea740 1 --2- 172.21.15.67:0/1611354779 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5f23f1ea0 0x55d5f27f8c30 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.493+0000 7f51c7cea740 1 --2- 172.21.15.67:0/1611354779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f2433520 0x55d5f27f2c80 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.493+0000 7f51c7cea740 1 --2- 172.21.15.67:0/1611354779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27e7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.493+0000 7f519eff5640 0 client.19956 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:22.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.493+0000 7f51c7cea740 1 --2- 172.21.15.67:0/1611354779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f240a660 0x55d5f27e6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.493+0000 7f51c7cea740 1 -- 172.21.15.67:0/1611354779 >> 172.21.15.67:0/1611354779 conn(0x55d5f2427910 msgr2=0x55d5f27daa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.493+0000 7f51c7cea740 1 -- 172.21.15.67:0/1611354779 shutdown_connections 2024-07-31T15:12:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.493+0000 7f51c7cea740 1 -- 172.21.15.67:0/1611354779 wait complete. 2024-07-31T15:12:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.493+0000 7f51c7cea740 20 client.19956 trim_cache size 0 max 16384 2024-07-31T15:12:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.511+0000 7f51c7cea740 1 Processor -- start 2024-07-31T15:12:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.511+0000 7f51c7cea740 1 -- start start 2024-07-31T15:12:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51c7cea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f240a660 0x55d5f27f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51c7cea740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27fe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51c7cea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f2433520 0x55d5f27fe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51c7cea740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc1ae470 con 0x55d5f240a660 2024-07-31T15:12:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51c7cea740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc1affb0 con 0x55d5f2412840 2024-07-31T15:12:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51c7cea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc1aeb40 con 0x55d5f2433520 2024-07-31T15:12:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51baffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f240a660 0x55d5f27f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51baffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f240a660 0x55d5f27f2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49310/0 (socket says 172.21.15.67:49310) 2024-07-31T15:12:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51baffd640 1 -- 172.21.15.67:0/2959934811 learned_addr learned my addr 172.21.15.67:0/2959934811 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51bb7fe640 1 --2- 172.21.15.67:0/2959934811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f2433520 0x55d5f27fe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51ba7fc640 1 --2- 172.21.15.67:0/2959934811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27fe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51baffd640 1 -- 172.21.15.67:0/2959934811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f2433520 msgr2=0x55d5f27fe930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51baffd640 1 --2- 172.21.15.67:0/2959934811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f2433520 0x55d5f27fe930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51baffd640 1 -- 172.21.15.67:0/2959934811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 msgr2=0x55d5f27fe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51baffd640 1 --2- 172.21.15.67:0/2959934811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27fe3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51baffd640 1 -- 172.21.15.67:0/2959934811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f27d9eb0 con 0x55d5f240a660 2024-07-31T15:12:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51ba7fc640 1 --2- 172.21.15.67:0/2959934811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:12:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.512+0000 7f51baffd640 1 --2- 172.21.15.67:0/2959934811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f240a660 0x55d5f27f2a70 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x55d5f27d7cf0 tx=0x7f51ac005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39a8068ba6242e5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.513+0000 7f51a0ff9640 1 -- 172.21.15.67:0/2959934811 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac035040 con 0x55d5f240a660 2024-07-31T15:12:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.513+0000 7f51a0ff9640 1 -- 172.21.15.67:0/2959934811 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ac0319b0 con 0x55d5f240a660 2024-07-31T15:12:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.513+0000 7f51a0ff9640 1 -- 172.21.15.67:0/2959934811 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac040040 con 0x55d5f240a660 2024-07-31T15:12:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.514+0000 7f51c7cea740 1 -- 172.21.15.67:0/2959934811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f240a660 msgr2=0x55d5f27f2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.514+0000 7f51c7cea740 1 --2- 172.21.15.67:0/2959934811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f240a660 0x55d5f27f2a70 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x55d5f27d7cf0 tx=0x7f51ac005340 comp rx=0 tx=0).stop 2024-07-31T15:12:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.514+0000 7f51c7cea740 1 -- 172.21.15.67:0/2959934811 shutdown_connections 2024-07-31T15:12:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.514+0000 7f51c7cea740 1 --2- 172.21.15.67:0/2959934811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f2433520 0x55d5f27fe930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.514+0000 7f51c7cea740 1 --2- 172.21.15.67:0/2959934811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.514+0000 7f51c7cea740 1 --2- 172.21.15.67:0/2959934811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f240a660 0x55d5f27f2a70 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.514+0000 7f51c7cea740 1 -- 172.21.15.67:0/2959934811 >> 172.21.15.67:0/2959934811 conn(0x55d5f2427910 msgr2=0x55d5f27d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.514+0000 7f51c7cea740 1 -- 172.21.15.67:0/2959934811 shutdown_connections 2024-07-31T15:12:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.514+0000 7f51c7cea740 1 -- 172.21.15.67:0/2959934811 wait complete. 2024-07-31T15:12:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.514+0000 7f51c7cea740 1 -- start start 2024-07-31T15:12:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.514+0000 7f51c7cea740 1 Processor -- start 2024-07-31T15:12:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51c7cea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f240a660 0x55d5f27eb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51c7cea740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27ebb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51c7cea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f2433520 0x55d5f285a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51c7cea740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc1af8e0 con 0x55d5f2433520 2024-07-31T15:12:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51c7cea740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc1af210 con 0x55d5f2412840 2024-07-31T15:12:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51c7cea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc1ac470 con 0x55d5f240a660 2024-07-31T15:12:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51c7cea740 10 client.0 _ll_register_callbacks cb 0x7f51b40a6c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51bb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f2433520 0x55d5f285a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51bb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f2433520 0x55d5f285a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49326/0 (socket says 172.21.15.67:49326) 2024-07-31T15:12:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51bb7fe640 1 -- 172.21.15.67:0/2642030325 learned_addr learned my addr 172.21.15.67:0/2642030325 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51baffd640 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f240a660 0x55d5f27eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51ba7fc640 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27ebb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51bb7fe640 1 -- 172.21.15.67:0/2642030325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f240a660 msgr2=0x55d5f27eb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51bb7fe640 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f240a660 0x55d5f27eb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51bb7fe640 1 -- 172.21.15.67:0/2642030325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 msgr2=0x55d5f27ebb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:22.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51bb7fe640 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27ebb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:22.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51bb7fe640 1 -- 172.21.15.67:0/2642030325 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5f285aae0 con 0x55d5f2433520 2024-07-31T15:12:22.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51baffd640 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f240a660 0x55d5f27eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:12:22.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51ba7fc640 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27ebb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:12:22.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f51bb7fe640 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f2433520 0x55d5f285a470 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f51b40e1930 tx=0x7f51b40c5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4224abd16286e7db server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:22.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.515+0000 7f519eff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:12:22.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.516+0000 7f519eff5640 1 -- 172.21.15.67:0/2642030325 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b401f040 con 0x55d5f2433520 2024-07-31T15:12:22.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.516+0000 7f519eff5640 1 -- 172.21.15.67:0/2642030325 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f51b40a9fa0 con 0x55d5f2433520 2024-07-31T15:12:22.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.516+0000 7f51c7cea740 1 -- 172.21.15.67:0/2642030325 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51ac0319b0 con 0x55d5f2433520 2024-07-31T15:12:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.516+0000 7f519eff5640 1 -- 172.21.15.67:0/2642030325 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f51b40aab10 con 0x55d5f2433520 2024-07-31T15:12:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.516+0000 7f51c7cea740 10 client.19962 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.516+0000 7f51c7cea740 1 -- 172.21.15.67:0/2642030325 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55d5f2861ef0 con 0x55d5f2433520 2024-07-31T15:12:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.516+0000 7f519eff5640 1 -- 172.21.15.67:0/2642030325 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ac0319b0 con 0x55d5f2433520 2024-07-31T15:12:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.516+0000 7f519eff5640 1 -- 172.21.15.67:0/2642030325 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f51b40a9aa0 con 0x55d5f2433520 2024-07-31T15:12:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.516+0000 7f51c7cea740 10 client.19962 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.516+0000 7f51c7cea740 10 client.19962 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:12:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.516+0000 7f51c7cea740 20 client.19962 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.516+0000 7f51c7cea740 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5f23c7250 0x55d5f2864570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.516+0000 7f51c7cea740 4 client.19962 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "53a8a385", "release": true, "await": true}] 2024-07-31T15:12:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.516+0000 7f51c7cea740 1 -- 172.21.15.67:0/2642030325 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "53a8a385", "release": true, "await": true}) -- 0x55d5f274c860 con 0x55d5f23c7250 2024-07-31T15:12:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.517+0000 7f51baffd640 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5f23c7250 0x55d5f2864570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:22.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.517+0000 7f519eff5640 10 client.19962 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:22.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:22.517+0000 7f51baffd640 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5f23c7250 0x55d5f2864570 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac64a30b549e8e0 server_cookie=83d35645403e91b5 in_seq=0 out_seq=0 2024-07-31T15:12:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:23 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:12:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:23 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:23 smithi067 ceph-mon[32750]: pgmap v1081: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 803 KiB/s wr, 3.39k op/s 2024-07-31T15:12:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:23 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:12:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:23 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:23 smithi073 ceph-mon[36714]: pgmap v1081: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 803 KiB/s wr, 3.39k op/s 2024-07-31T15:12:23.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:23.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:23 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:12:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:23 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:12:23.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:23 smithi077 ceph-mon[37327]: pgmap v1081: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 803 KiB/s wr, 3.39k op/s 2024-07-31T15:12:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.032+0000 7f519eff5640 1 -- 172.21.15.67:0/2642030325 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d5f274c860 con 0x55d5f23c7250 2024-07-31T15:12:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.032+0000 7f519eff5640 10 client.19962 handle_command_reply: tid=0 2024-07-31T15:12:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.032+0000 7f51c7cea740 1 client.19962 shutdown 2024-07-31T15:12:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.033+0000 7f51c7cea740 1 -- 172.21.15.67:0/2642030325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f2433520 msgr2=0x55d5f285a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.033+0000 7f51c7cea740 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f2433520 0x55d5f285a470 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f51b40e1930 tx=0x7f51b40c5790 comp rx=0 tx=0).stop 2024-07-31T15:12:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.033+0000 7f51c7cea740 1 -- 172.21.15.67:0/2642030325 shutdown_connections 2024-07-31T15:12:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.033+0000 7f51c7cea740 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55d5f23c7250 0x55d5f2864570 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.033+0000 7f51c7cea740 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d5f2433520 0x55d5f285a470 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.033+0000 7f51c7cea740 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5f2412840 0x55d5f27ebb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.033+0000 7f519eff5640 0 client.19962 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:25.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.033+0000 7f51c7cea740 1 --2- 172.21.15.67:0/2642030325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5f240a660 0x55d5f27eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:25.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.033+0000 7f51c7cea740 1 -- 172.21.15.67:0/2642030325 >> 172.21.15.67:0/2642030325 conn(0x55d5f2427910 msgr2=0x55d5f27da0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:25.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.033+0000 7f51c7cea740 1 -- 172.21.15.67:0/2642030325 shutdown_connections 2024-07-31T15:12:25.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.033+0000 7f51c7cea740 1 -- 172.21.15.67:0/2642030325 wait complete. 2024-07-31T15:12:25.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.033+0000 7f51c7cea740 20 client.19962 trim_cache size 0 max 16384 2024-07-31T15:12:25.036 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:12:25.036 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 66, 2024-07-31T15:12:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:12:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 60, 2024-07-31T15:12:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:12:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "53a8a385": { 2024-07-31T15:12:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 60, 2024-07-31T15:12:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:12:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:12:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:12:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:12:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:12:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:12:25.038 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 137.0, 2024-07-31T15:12:25.038 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:12:25.038 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:12:25.038 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:12:25.038 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:12:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:12:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:12:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:12:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:12:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:12:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:12:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:12:25.039 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:12:25.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.034+0000 7f51c1799640 1 -- 172.21.15.67:0/1901350796 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f51a40b3d70 msgr2=0x7f51a40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:25.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.034+0000 7f51c1799640 1 --2- 172.21.15.67:0/1901350796 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f51a40b3d70 0x7f51a40b6280 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f51bc100680 tx=0x7f51b000a6c0 comp rx=0 tx=0).stop 2024-07-31T15:12:25.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.034+0000 7f51c1799640 1 -- 172.21.15.67:0/1901350796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51bc108720 msgr2=0x7f51bc19e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:25.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.034+0000 7f51c1799640 1 --2- 172.21.15.67:0/1901350796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51bc108720 0x7f51bc19e5d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f51b400eaa0 tx=0x7f51b400eea0 comp rx=0 tx=0).stop 2024-07-31T15:12:25.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.035+0000 7f51c1799640 1 -- 172.21.15.67:0/1901350796 shutdown_connections 2024-07-31T15:12:25.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.035+0000 7f51c1799640 1 --2- 172.21.15.67:0/1901350796 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f51a40b3d70 0x7f51a40b6280 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:25.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.035+0000 7f51c1799640 1 --2- 172.21.15.67:0/1901350796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51bc108720 0x7f51bc19e5d0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:25.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.035+0000 7f51c1799640 1 --2- 172.21.15.67:0/1901350796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51bc107dd0 0x7f51bc199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:25.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.035+0000 7f51c1799640 1 --2- 172.21.15.67:0/1901350796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51bc102490 0x7f51bc198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.035+0000 7f51c1799640 1 -- 172.21.15.67:0/1901350796 >> 172.21.15.67:0/1901350796 conn(0x7f51bc079160 msgr2=0x7f51bc100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.035+0000 7f51c1799640 1 -- 172.21.15.67:0/1901350796 shutdown_connections 2024-07-31T15:12:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:25.035+0000 7f51c1799640 1 -- 172.21.15.67:0/1901350796 wait complete. 2024-07-31T15:12:25.488 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_combo_phy.o 2024-07-31T15:12:25.490 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_dumb_buffers.o 2024-07-31T15:12:25.542 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-stats.o 2024-07-31T15:12:25.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:25 smithi067 ceph-mon[32750]: pgmap v1082: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 402 KiB/s wr, 5.29k op/s 2024-07-31T15:12:25.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:25 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:12:25.558 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/leds/trigger/built-in.a 2024-07-31T15:12:25.590 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/leds/blink/built-in.a 2024-07-31T15:12:25.608 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/leds/simple/built-in.a 2024-07-31T15:12:25.620 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/leds/led-core.o 2024-07-31T15:12:25.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:25 smithi073 ceph-mon[36714]: pgmap v1082: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 402 KiB/s wr, 5.29k op/s 2024-07-31T15:12:25.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:25 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:12:25.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:25 smithi077 ceph-mon[37327]: pgmap v1082: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 402 KiB/s wr, 5.29k op/s 2024-07-31T15:12:25.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:25 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:12:25.765 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '53a8a385', total seconds elapsed: 45.8 2024-07-31T15:12:25.807 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_connector.o 2024-07-31T15:12:25.808 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_crtc.o 2024-07-31T15:12:25.811 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_crtc_state_dump.o 2024-07-31T15:12:25.874 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/firmware/arm_ffa/built-in.a 2024-07-31T15:12:25.928 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/firmware/arm_scmi/built-in.a 2024-07-31T15:12:25.955 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/firmware/broadcom/built-in.a 2024-07-31T15:12:25.998 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/firmware/cirrus/built-in.a 2024-07-31T15:12:26.034 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/firmware/meson/built-in.a 2024-07-31T15:12:26.136 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/leds/led-class.o 2024-07-31T15:12:26.209 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/leds/led-triggers.o 2024-07-31T15:12:26.262 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/efi-stub-helper.o 2024-07-31T15:12:26.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:12:26.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:12:26.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:12:26.584 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/gop.o 2024-07-31T15:12:26.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:12:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:12:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:12:26.689 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-rq.o 2024-07-31T15:12:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:12:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:12:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:12:26.771 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_cursor.o 2024-07-31T15:12:26.799 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_display.o 2024-07-31T15:12:26.849 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/leds/built-in.a 2024-07-31T15:12:26.915 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/crypto/stm32/built-in.a 2024-07-31T15:12:26.928 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/secureboot.o 2024-07-31T15:12:26.942 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/crypto/xilinx/built-in.a 2024-07-31T15:12:26.974 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/crypto/hisilicon/built-in.a 2024-07-31T15:12:27.009 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/crypto/starfive/built-in.a 2024-07-31T15:12:27.013 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/crypto/intel/keembay/built-in.a 2024-07-31T15:12:27.032 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_display_driver.o 2024-07-31T15:12:27.048 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/crypto/intel/ixp4xx/built-in.a 2024-07-31T15:12:27.063 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/crypto/intel/built-in.a 2024-07-31T15:12:27.084 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/crypto/built-in.a 2024-07-31T15:12:27.127 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/clocksource/acpi_pm.o 2024-07-31T15:12:27.190 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/usbhid/hid-core.o 2024-07-31T15:12:27.350 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/tpm.o 2024-07-31T15:12:27.454 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-io-rewind.o 2024-07-31T15:12:27.631 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-builtin.o 2024-07-31T15:12:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:27 smithi073 ceph-mon[36714]: pgmap v1083: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 385 KiB/s wr, 5.38k op/s 2024-07-31T15:12:27.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:27 smithi077 ceph-mon[37327]: pgmap v1083: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 385 KiB/s wr, 5.38k op/s 2024-07-31T15:12:27.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:27 smithi067 ceph-mon[32750]: pgmap v1083: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 385 KiB/s wr, 5.38k op/s 2024-07-31T15:12:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.828+0000 7f4a93c77640 1 -- 172.21.15.67:0/1986603437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8c1116d0 msgr2=0x7f4a8c109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.828+0000 7f4a93c77640 1 --2- 172.21.15.67:0/1986603437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8c1116d0 0x7f4a8c109df0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009f00 tx=0x7f4a8402ab60 comp rx=0 tx=0).stop 2024-07-31T15:12:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.828+0000 7f4a93c77640 1 -- 172.21.15.67:0/1986603437 shutdown_connections 2024-07-31T15:12:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.828+0000 7f4a93c77640 1 --2- 172.21.15.67:0/1986603437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8c11c5d0 0x7f4a8c11e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.828+0000 7f4a93c77640 1 --2- 172.21.15.67:0/1986603437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a8c10a330 0x7f4a8c10a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.828+0000 7f4a93c77640 1 --2- 172.21.15.67:0/1986603437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8c1116d0 0x7f4a8c109df0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.828+0000 7f4a93c77640 1 -- 172.21.15.67:0/1986603437 >> 172.21.15.67:0/1986603437 conn(0x7f4a8c06e350 msgr2=0x7f4a8c06e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.828+0000 7f4a93c77640 1 -- 172.21.15.67:0/1986603437 shutdown_connections 2024-07-31T15:12:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.828+0000 7f4a93c77640 1 -- 172.21.15.67:0/1986603437 wait complete. 2024-07-31T15:12:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.829+0000 7f4a93c77640 1 Processor -- start 2024-07-31T15:12:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.830+0000 7f4a93c77640 1 -- start start 2024-07-31T15:12:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.830+0000 7f4a93c77640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8c10a330 0x7f4a8c11be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.830+0000 7f4a93c77640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8c1116d0 0x7f4a8c115100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.830+0000 7f4a93c77640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a8c11c5d0 0x7f4a8c115680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.830+0000 7f4a93c77640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c1221c0 con 0x7f4a8c10a330 2024-07-31T15:12:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.830+0000 7f4a93c77640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c122040 con 0x7f4a8c1116d0 2024-07-31T15:12:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.830+0000 7f4a93c77640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c122340 con 0x7f4a8c11c5d0 2024-07-31T15:12:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.830+0000 7f4a92214640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a8c11c5d0 0x7f4a8c115680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.831+0000 7f4a92214640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a8c11c5d0 0x7f4a8c115680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49424/0 (socket says 172.21.15.67:49424) 2024-07-31T15:12:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.831+0000 7f4a92214640 1 -- 172.21.15.67:0/3814976465 learned_addr learned my addr 172.21.15.67:0/3814976465 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.831+0000 7f4a91a13640 1 --2- 172.21.15.67:0/3814976465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8c10a330 0x7f4a8c11be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.831+0000 7f4a92214640 1 -- 172.21.15.67:0/3814976465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8c1116d0 msgr2=0x7f4a8c115100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:12:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.831+0000 7f4a92214640 1 --2- 172.21.15.67:0/3814976465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8c1116d0 0x7f4a8c115100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.831+0000 7f4a92214640 1 -- 172.21.15.67:0/3814976465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8c10a330 msgr2=0x7f4a8c11be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.831+0000 7f4a92214640 1 --2- 172.21.15.67:0/3814976465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8c10a330 0x7f4a8c11be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.832+0000 7f4a92214640 1 -- 172.21.15.67:0/3814976465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c115ea0 con 0x7f4a8c11c5d0 2024-07-31T15:12:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.832+0000 7f4a91a13640 1 --2- 172.21.15.67:0/3814976465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8c10a330 0x7f4a8c11be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:12:27.834 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/clocksource/i8253.o 2024-07-31T15:12:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.832+0000 7f4a92214640 1 --2- 172.21.15.67:0/3814976465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a8c11c5d0 0x7f4a8c115680 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800b3f0 tx=0x7f4a8800b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.832+0000 7f4a82ffd640 1 -- 172.21.15.67:0/3814976465 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88015020 con 0x7f4a8c11c5d0 2024-07-31T15:12:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.832+0000 7f4a93c77640 1 -- 172.21.15.67:0/3814976465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8c11ab60 con 0x7f4a8c11c5d0 2024-07-31T15:12:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.833+0000 7f4a93c77640 1 -- 172.21.15.67:0/3814976465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a8c11b100 con 0x7f4a8c11c5d0 2024-07-31T15:12:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.833+0000 7f4a82ffd640 1 -- 172.21.15.67:0/3814976465 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a88011ce0 con 0x7f4a8c11c5d0 2024-07-31T15:12:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.835+0000 7f4a82ffd640 1 -- 172.21.15.67:0/3814976465 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88010dc0 con 0x7f4a8c11c5d0 2024-07-31T15:12:27.839 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.837+0000 7f4a82ffd640 1 -- 172.21.15.67:0/3814976465 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4a88018020 con 0x7f4a8c11c5d0 2024-07-31T15:12:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.839+0000 7f4a82ffd640 1 --2- 172.21.15.67:0/3814976465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4a780b3e80 0x7f4a780b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.839+0000 7f4a91a13640 1 --2- 172.21.15.67:0/3814976465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4a780b3e80 0x7f4a780b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.839+0000 7f4a82ffd640 1 -- 172.21.15.67:0/3814976465 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f4a880e3de0 con 0x7f4a8c11c5d0 2024-07-31T15:12:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.840+0000 7f4a9ab6a740 1 Processor -- start 2024-07-31T15:12:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.840+0000 7f4a9ab6a740 1 -- start start 2024-07-31T15:12:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.840+0000 7f4a91a13640 1 --2- 172.21.15.67:0/3814976465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4a780b3e80 0x7f4a780b6390 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4a84036040 tx=0x7f4a84000f40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.840+0000 7f4a9ab6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282261b0 0x5636285c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.840+0000 7f4a9ab6a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.840+0000 7f4a9ab6a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282054d0 0x5636285c9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a9ab6a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56362853e3b0 con 0x5636282054d0 2024-07-31T15:12:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a9ab6a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56362852d8b0 con 0x5636281fd2f0 2024-07-31T15:12:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a9ab6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5636285318f0 con 0x5636282261b0 2024-07-31T15:12:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a91a13640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282261b0 0x5636285c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a91a13640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282261b0 0x5636285c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49440/0 (socket says 172.21.15.67:49440) 2024-07-31T15:12:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a91a13640 1 -- 172.21.15.67:0/3824165069 learned_addr learned my addr 172.21.15.67:0/3824165069 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a92214640 1 --2- 172.21.15.67:0/3824165069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282054d0 0x5636285c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a91212640 1 --2- 172.21.15.67:0/3824165069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a91a13640 1 -- 172.21.15.67:0/3824165069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 msgr2=0x5636285c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a91a13640 1 --2- 172.21.15.67:0/3824165069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a91a13640 1 -- 172.21.15.67:0/3824165069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282054d0 msgr2=0x5636285c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a91a13640 1 --2- 172.21.15.67:0/3824165069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282054d0 0x5636285c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a91a13640 1 -- 172.21.15.67:0/3824165069 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636284d49a0 con 0x5636282261b0 2024-07-31T15:12:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a91212640 1 --2- 172.21.15.67:0/3824165069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:12:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.841+0000 7f4a92214640 1 --2- 172.21.15.67:0/3824165069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282054d0 0x5636285c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:12:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.842+0000 7f4a91a13640 1 --2- 172.21.15.67:0/3824165069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282261b0 0x5636285c8ea0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4a84034520 tx=0x7f4a8403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e33542edef5781 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.842+0000 7f4a77fff640 1 -- 172.21.15.67:0/3824165069 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8402ba00 con 0x5636282261b0 2024-07-31T15:12:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.842+0000 7f4a77fff640 1 -- 172.21.15.67:0/3824165069 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a84051070 con 0x5636282261b0 2024-07-31T15:12:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.842+0000 7f4a77fff640 1 -- 172.21.15.67:0/3824165069 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8403f440 con 0x5636282261b0 2024-07-31T15:12:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.843+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/3824165069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282261b0 msgr2=0x5636285c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.843+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/3824165069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282261b0 0x5636285c8ea0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4a84034520 tx=0x7f4a8403f000 comp rx=0 tx=0).stop 2024-07-31T15:12:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.843+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/3824165069 shutdown_connections 2024-07-31T15:12:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.843+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/3824165069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282054d0 0x5636285c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.843+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/3824165069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.843+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/3824165069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282261b0 0x5636285c8ea0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.843+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/3824165069 >> 172.21.15.67:0/3824165069 conn(0x56362821a300 msgr2=0x5636285c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.843+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/3824165069 shutdown_connections 2024-07-31T15:12:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.844+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/3824165069 wait complete. 2024-07-31T15:12:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.844+0000 7f4a9ab6a740 1 -- start start 2024-07-31T15:12:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.844+0000 7f4a9ab6a740 1 Processor -- start 2024-07-31T15:12:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.845+0000 7f4a9ab6a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285d4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.845+0000 7f4a9ab6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 0x5636285d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.845+0000 7f4a9ab6a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.845+0000 7f4a9ab6a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563628467ab0 con 0x5636282261b0 2024-07-31T15:12:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.845+0000 7f4a9ab6a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5636284b5cf0 con 0x5636281fd2f0 2024-07-31T15:12:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.845+0000 7f4a9ab6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5636285d75c0 con 0x5636282054d0 2024-07-31T15:12:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.845+0000 7f4a9ab6a740 10 client.0 _ll_register_callbacks cb 0x5636285279f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.845+0000 7f4a91a13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285d4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.845+0000 7f4a91a13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285d4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:55794/0 (socket says 172.21.15.67:55794) 2024-07-31T15:12:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.845+0000 7f4a91a13640 1 -- 172.21.15.67:0/1240227279 learned_addr learned my addr 172.21.15.67:0/1240227279 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.845+0000 7f4a91212640 1 --2- 172.21.15.67:0/1240227279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 0x5636285d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.846+0000 7f4a91a13640 1 -- 172.21.15.67:0/1240227279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 msgr2=0x5636285d4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.846+0000 7f4a91a13640 1 --2- 172.21.15.67:0/1240227279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 0x5636285d4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.846+0000 7f4a91a13640 1 -- 172.21.15.67:0/1240227279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 msgr2=0x5636285e0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.846+0000 7f4a92214640 1 --2- 172.21.15.67:0/1240227279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.846+0000 7f4a91a13640 1 --2- 172.21.15.67:0/1240227279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.846+0000 7f4a91a13640 1 -- 172.21.15.67:0/1240227279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563628291790 con 0x5636281fd2f0 2024-07-31T15:12:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.846+0000 7f4a92214640 1 --2- 172.21.15.67:0/1240227279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:12:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.846+0000 7f4a91a13640 1 --2- 172.21.15.67:0/1240227279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285d4a10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4a8402aba0 tx=0x7f4a8402c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d2ce69f08feb899 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.846+0000 7f4a757fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:12:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.846+0000 7f4a757fa640 1 -- 172.21.15.67:0/1240227279 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8402bc10 con 0x5636281fd2f0 2024-07-31T15:12:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.847+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/1240227279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a84051070 con 0x5636281fd2f0 2024-07-31T15:12:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.847+0000 7f4a757fa640 1 -- 172.21.15.67:0/1240227279 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f4a8402bdb0 con 0x5636281fd2f0 2024-07-31T15:12:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.847+0000 7f4a757fa640 1 -- 172.21.15.67:0/1240227279 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f4a84068050 con 0x5636281fd2f0 2024-07-31T15:12:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.847+0000 7f4a9ab6a740 10 client.29492 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.847+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/1240227279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56362830a620 con 0x5636281fd2f0 2024-07-31T15:12:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.848+0000 7f4a757fa640 1 -- 172.21.15.67:0/1240227279 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a84051070 con 0x5636281fd2f0 2024-07-31T15:12:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.848+0000 7f4a757fa640 1 -- 172.21.15.67:0/1240227279 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f4a8400b040 con 0x5636281fd2f0 2024-07-31T15:12:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.851+0000 7f4a9ab6a740 10 client.29492 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.851+0000 7f4a9ab6a740 10 client.29492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:12:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.851+0000 7f4a9ab6a740 20 client.29492 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.851+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/1240227279 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5636281e4d80 0x5636285e6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.851+0000 7f4a9ab6a740 4 client.29492 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:12:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.851+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/1240227279 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563628540300 con 0x5636281e4d80 2024-07-31T15:12:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.851+0000 7f4a91212640 1 --2- 172.21.15.67:0/1240227279 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5636281e4d80 0x5636285e6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.852+0000 7f4a757fa640 10 client.29492 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.852+0000 7f4a91212640 1 --2- 172.21.15.67:0/1240227279 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5636281e4d80 0x5636285e6710 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6484a39cee12937 server_cookie=1ccb2594c6f47b42 in_seq=0 out_seq=0 2024-07-31T15:12:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.854+0000 7f4a757fa640 1 -- 172.21.15.67:0/1240227279 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563628540300 con 0x5636281e4d80 2024-07-31T15:12:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.854+0000 7f4a757fa640 10 client.29492 handle_command_reply: tid=0 2024-07-31T15:12:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.854+0000 7f4a9ab6a740 1 client.29492 shutdown 2024-07-31T15:12:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.854+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/1240227279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 msgr2=0x5636285d4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.854+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/1240227279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285d4a10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4a8402aba0 tx=0x7f4a8402c6e0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.854+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/1240227279 shutdown_connections 2024-07-31T15:12:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.854+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/1240227279 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5636281e4d80 0x5636285e6710 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.854+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/1240227279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.854+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/1240227279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 0x5636285d4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.854+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/1240227279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285d4a10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.854+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/1240227279 >> 172.21.15.67:0/1240227279 conn(0x56362821a300 msgr2=0x5636285c9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.854+0000 7f4a757fa640 0 client.29492 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.854+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/1240227279 shutdown_connections 2024-07-31T15:12:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.855+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/1240227279 wait complete. 2024-07-31T15:12:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.855+0000 7f4a9ab6a740 20 client.29492 trim_cache size 0 max 16384 2024-07-31T15:12:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.889+0000 7f4a9ab6a740 1 Processor -- start 2024-07-31T15:12:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.889+0000 7f4a9ab6a740 1 -- start start 2024-07-31T15:12:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.889+0000 7f4a9ab6a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285d4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.889+0000 7f4a9ab6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 0x5636285cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.889+0000 7f4a9ab6a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.889+0000 7f4a9ab6a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7000fec0 con 0x5636282261b0 2024-07-31T15:12:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.889+0000 7f4a9ab6a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4a70011a00 con 0x5636281fd2f0 2024-07-31T15:12:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.889+0000 7f4a9ab6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a70010590 con 0x5636282054d0 2024-07-31T15:12:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.889+0000 7f4a91a13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285d4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.890+0000 7f4a91a13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285d4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:55808/0 (socket says 172.21.15.67:55808) 2024-07-31T15:12:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.890+0000 7f4a91a13640 1 -- 172.21.15.67:0/3123787942 learned_addr learned my addr 172.21.15.67:0/3123787942 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.890+0000 7f4a91212640 1 --2- 172.21.15.67:0/3123787942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 0x5636285cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.890+0000 7f4a92214640 1 --2- 172.21.15.67:0/3123787942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.890+0000 7f4a91a13640 1 -- 172.21.15.67:0/3123787942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 msgr2=0x5636285cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.890+0000 7f4a91a13640 1 --2- 172.21.15.67:0/3123787942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 0x5636285cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.890+0000 7f4a91a13640 1 -- 172.21.15.67:0/3123787942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 msgr2=0x5636285e08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.890+0000 7f4a91a13640 1 --2- 172.21.15.67:0/3123787942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.890+0000 7f4a91a13640 1 -- 172.21.15.67:0/3123787942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636285d2ce0 con 0x5636281fd2f0 2024-07-31T15:12:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.890+0000 7f4a92214640 1 --2- 172.21.15.67:0/3123787942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:12:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.890+0000 7f4a91a13640 1 --2- 172.21.15.67:0/3123787942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285d4e80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4a8404c470 tx=0x7f4a840349e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=391caac32dac24e5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.891+0000 7f4a777fe640 1 -- 172.21.15.67:0/3123787942 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84034d00 con 0x5636281fd2f0 2024-07-31T15:12:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.891+0000 7f4a777fe640 1 -- 172.21.15.67:0/3123787942 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a84051040 con 0x5636281fd2f0 2024-07-31T15:12:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.891+0000 7f4a777fe640 1 -- 172.21.15.67:0/3123787942 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8402bce0 con 0x5636281fd2f0 2024-07-31T15:12:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.892+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/3123787942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 msgr2=0x5636285d4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.892+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/3123787942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285d4e80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4a8404c470 tx=0x7f4a840349e0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.892+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/3123787942 shutdown_connections 2024-07-31T15:12:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.892+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/3123787942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.892+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/3123787942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 0x5636285cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.892+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/3123787942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285d4e80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.892+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/3123787942 >> 172.21.15.67:0/3123787942 conn(0x56362821a300 msgr2=0x5636285d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.892+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/3123787942 shutdown_connections 2024-07-31T15:12:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.892+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/3123787942 wait complete. 2024-07-31T15:12:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.893+0000 7f4a9ab6a740 1 -- start start 2024-07-31T15:12:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.893+0000 7f4a9ab6a740 1 Processor -- start 2024-07-31T15:12:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.894+0000 7f4a9ab6a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285e9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.894+0000 7f4a9ab6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 0x5636285e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.894+0000 7f4a9ab6a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.894+0000 7f4a9ab6a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4a70011330 con 0x5636282261b0 2024-07-31T15:12:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.894+0000 7f4a9ab6a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4a70010c60 con 0x5636281fd2f0 2024-07-31T15:12:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.894+0000 7f4a9ab6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7000dec0 con 0x5636282054d0 2024-07-31T15:12:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.894+0000 7f4a9ab6a740 10 client.0 _ll_register_callbacks cb 0x7f4a88012ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.894+0000 7f4a92214640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.894+0000 7f4a91a13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285e9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.894+0000 7f4a92214640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49370/0 (socket says 172.21.15.67:49370) 2024-07-31T15:12:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.894+0000 7f4a92214640 1 -- 172.21.15.67:0/2875538999 learned_addr learned my addr 172.21.15.67:0/2875538999 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.894+0000 7f4a91212640 1 --2- 172.21.15.67:0/2875538999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 0x5636285e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.895+0000 7f4a91a13640 1 -- 172.21.15.67:0/2875538999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 msgr2=0x5636285e82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.895+0000 7f4a91a13640 1 --2- 172.21.15.67:0/2875538999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 0x5636285e82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.895+0000 7f4a91a13640 1 -- 172.21.15.67:0/2875538999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 msgr2=0x5636285e87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.895+0000 7f4a91a13640 1 --2- 172.21.15.67:0/2875538999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.895+0000 7f4a91a13640 1 -- 172.21.15.67:0/2875538999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636285e8f20 con 0x5636281fd2f0 2024-07-31T15:12:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.895+0000 7f4a92214640 1 --2- 172.21.15.67:0/2875538999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:12:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.895+0000 7f4a91a13640 1 --2- 172.21.15.67:0/2875538999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285e9dd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009f00 tx=0x7f4a8402b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17fe4bf1905ee292 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.896+0000 7f4a757fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:12:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.896+0000 7f4a757fa640 1 -- 172.21.15.67:0/2875538999 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84073070 con 0x5636281fd2f0 2024-07-31T15:12:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.896+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/2875538999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a84051040 con 0x5636281fd2f0 2024-07-31T15:12:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.896+0000 7f4a757fa640 1 -- 172.21.15.67:0/2875538999 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f4a8404cb00 con 0x5636281fd2f0 2024-07-31T15:12:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.896+0000 7f4a757fa640 1 -- 172.21.15.67:0/2875538999 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f4a84080050 con 0x5636281fd2f0 2024-07-31T15:12:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.897+0000 7f4a9ab6a740 10 client.29498 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.897+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/2875538999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56362864db60 con 0x5636281fd2f0 2024-07-31T15:12:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.897+0000 7f4a757fa640 1 -- 172.21.15.67:0/2875538999 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a84051040 con 0x5636281fd2f0 2024-07-31T15:12:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.898+0000 7f4a757fa640 1 -- 172.21.15.67:0/2875538999 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f4a8400b040 con 0x5636281fd2f0 2024-07-31T15:12:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.898+0000 7f4a9ab6a740 10 client.29498 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.898+0000 7f4a9ab6a740 10 client.29498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:12:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.898+0000 7f4a9ab6a740 20 client.29498 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.898+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/2875538999 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5636281ba1c0 0x563628650090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.899+0000 7f4a9ab6a740 4 client.29498 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:12:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.899+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/2875538999 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56362853bda0 con 0x5636281ba1c0 2024-07-31T15:12:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.899+0000 7f4a91212640 1 --2- 172.21.15.67:0/2875538999 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5636281ba1c0 0x563628650090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.899+0000 7f4a91212640 1 --2- 172.21.15.67:0/2875538999 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5636281ba1c0 0x563628650090 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50b1a98a3d51430 server_cookie=e9137b3457b0fa3e in_seq=0 out_seq=0 2024-07-31T15:12:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:27.899+0000 7f4a757fa640 10 client.29498 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:28.085 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_display_irq.o 2024-07-31T15:12:28.110 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/file.o 2024-07-31T15:12:28.233 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_display_power.o 2024-07-31T15:12:28.255 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-raid1.o 2024-07-31T15:12:28.311 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/clocksource/built-in.a 2024-07-31T15:12:28.332 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-log.o 2024-07-31T15:12:29.010 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/mem.o 2024-07-31T15:12:29.326 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/usbhid/hiddev.o 2024-07-31T15:12:29.438 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/platform/x86/amd/built-in.a 2024-07-31T15:12:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.451+0000 7f4a757fa640 1 -- 172.21.15.67:0/2875538999 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56362853bda0 con 0x5636281ba1c0 2024-07-31T15:12:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.451+0000 7f4a757fa640 10 client.29498 handle_command_reply: tid=0 2024-07-31T15:12:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.456+0000 7f4a9ab6a740 1 client.29498 shutdown 2024-07-31T15:12:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.456+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/2875538999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 msgr2=0x5636285e9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.456+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/2875538999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285e9dd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009f00 tx=0x7f4a8402b640 comp rx=0 tx=0).stop 2024-07-31T15:12:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.456+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/2875538999 shutdown_connections 2024-07-31T15:12:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.456+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/2875538999 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5636281ba1c0 0x563628650090 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.456+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/2875538999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636282261b0 0x5636285e87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.456+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/2875538999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636282054d0 0x5636285e82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.456+0000 7f4a9ab6a740 1 --2- 172.21.15.67:0/2875538999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636281fd2f0 0x5636285e9dd0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.456+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/2875538999 >> 172.21.15.67:0/2875538999 conn(0x56362821a300 msgr2=0x5636285c5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.456+0000 7f4a757fa640 0 client.29498 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.456+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/2875538999 shutdown_connections 2024-07-31T15:12:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.456+0000 7f4a9ab6a740 1 -- 172.21.15.67:0/2875538999 wait complete. 2024-07-31T15:12:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.457+0000 7f4a9ab6a740 20 client.29498 trim_cache size 0 max 16384 2024-07-31T15:12:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:12:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:12:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:12:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:12:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.475+0000 7f4a93c77640 1 -- 172.21.15.67:0/3814976465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4a780b3e80 msgr2=0x7f4a780b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.475+0000 7f4a93c77640 1 --2- 172.21.15.67:0/3814976465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4a780b3e80 0x7f4a780b6390 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4a84036040 tx=0x7f4a84000f40 comp rx=0 tx=0).stop 2024-07-31T15:12:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.475+0000 7f4a93c77640 1 -- 172.21.15.67:0/3814976465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a8c11c5d0 msgr2=0x7f4a8c115680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.475+0000 7f4a93c77640 1 --2- 172.21.15.67:0/3814976465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a8c11c5d0 0x7f4a8c115680 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800b3f0 tx=0x7f4a8800b8b0 comp rx=0 tx=0).stop 2024-07-31T15:12:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.475+0000 7f4a93c77640 1 -- 172.21.15.67:0/3814976465 shutdown_connections 2024-07-31T15:12:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.475+0000 7f4a93c77640 1 --2- 172.21.15.67:0/3814976465 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4a780b3e80 0x7f4a780b6390 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.476+0000 7f4a93c77640 1 --2- 172.21.15.67:0/3814976465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a8c11c5d0 0x7f4a8c115680 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.476+0000 7f4a93c77640 1 --2- 172.21.15.67:0/3814976465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a8c1116d0 0x7f4a8c115100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.476+0000 7f4a93c77640 1 --2- 172.21.15.67:0/3814976465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8c10a330 0x7f4a8c11be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.476+0000 7f4a93c77640 1 -- 172.21.15.67:0/3814976465 >> 172.21.15.67:0/3814976465 conn(0x7f4a8c06e350 msgr2=0x7f4a8c110350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.476+0000 7f4a93c77640 1 -- 172.21.15.67:0/3814976465 shutdown_connections 2024-07-31T15:12:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:29.476+0000 7f4a93c77640 1 -- 172.21.15.67:0/3814976465 wait complete. 2024-07-31T15:12:29.563 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/platform/x86/intel/built-in.a 2024-07-31T15:12:29.641 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/platform/x86/wmi.o 2024-07-31T15:12:29.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:29 smithi073 ceph-mon[36714]: pgmap v1084: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 317 KiB/s wr, 4.23k op/s 2024-07-31T15:12:29.716 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:29 smithi067 ceph-mon[32750]: pgmap v1084: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 317 KiB/s wr, 4.23k op/s 2024-07-31T15:12:29.726 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/platform/x86/wmi-bmof.o 2024-07-31T15:12:29.733 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/random.o 2024-07-31T15:12:29.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:29 smithi077 ceph-mon[37327]: pgmap v1084: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 317 KiB/s wr, 4.23k op/s 2024-07-31T15:12:29.949 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-region-hash.o 2024-07-31T15:12:30.194 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/platform/x86/eeepc-laptop.o 2024-07-31T15:12:30.261 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/usbhid/hid-pidff.o 2024-07-31T15:12:30.309 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/platform/x86/p2sb.o 2024-07-31T15:12:30.432 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/randomalloc.o 2024-07-31T15:12:30.498 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_display_power_map.o 2024-07-31T15:12:30.592 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_display_power_well.o 2024-07-31T15:12:30.728 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/md/dm-zero.o 2024-07-31T15:12:30.878 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_display_reset.o 2024-07-31T15:12:31.049 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/pci.o 2024-07-31T15:12:31.184 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/platform/x86/built-in.a 2024-07-31T15:12:31.217 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/hid/usbhid/built-in.a 2024-07-31T15:12:31.230 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/platform/surface/built-in.a 2024-07-31T15:12:31.249 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/platform/built-in.a 2024-07-31T15:12:31.249 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-core.o 2024-07-31T15:12:31.298 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/md/built-in.a 2024-07-31T15:12:31.307 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/mailbox/mailbox.o 2024-07-31T15:12:31.334 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/mailbox/pcc.o 2024-07-31T15:12:31.503 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_display_rps.o 2024-07-31T15:12:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:31 smithi073 ceph-mon[36714]: pgmap v1085: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 116 KiB/s wr, 3.69k op/s 2024-07-31T15:12:31.718 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/skip_spaces.o 2024-07-31T15:12:31.720 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dmc.o 2024-07-31T15:12:31.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:31 smithi077 ceph-mon[37327]: pgmap v1085: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 116 KiB/s wr, 3.69k op/s 2024-07-31T15:12:31.766 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/lib-cmdline.o 2024-07-31T15:12:31.785 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dpio_phy.o 2024-07-31T15:12:31.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:31 smithi067 ceph-mon[32750]: pgmap v1085: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 116 KiB/s wr, 3.69k op/s 2024-07-31T15:12:31.822 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dpll.o 2024-07-31T15:12:31.880 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/lib-ctype.o 2024-07-31T15:12:31.894 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/mailbox/built-in.a 2024-07-31T15:12:31.917 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/alignedmem.o 2024-07-31T15:12:31.967 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/firmware/imx/built-in.a 2024-07-31T15:12:31.980 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/relocate.o 2024-07-31T15:12:32.268 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dpll_mgr.o 2024-07-31T15:12:32.428 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dpt.o 2024-07-31T15:12:32.567 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/printk.o 2024-07-31T15:12:32.656 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/vsprintf.o 2024-07-31T15:12:32.697 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-input.o 2024-07-31T15:12:32.896 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/libstub/x86-stub.o 2024-07-31T15:12:32.940 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_drrs.o 2024-07-31T15:12:33.132 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dsb.o 2024-07-31T15:12:33.187 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/alignedmem.stub.o 2024-07-31T15:12:33.229 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/efi-stub-helper.stub.o 2024-07-31T15:12:33.254 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/file.stub.o 2024-07-31T15:12:33.278 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/gop.stub.o 2024-07-31T15:12:33.302 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/lib-cmdline.stub.o 2024-07-31T15:12:33.336 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/lib-ctype.stub.o 2024-07-31T15:12:33.363 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_fb.o 2024-07-31T15:12:33.428 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_fb_pin.o 2024-07-31T15:12:33.519 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_fbc.o 2024-07-31T15:12:33.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:33 smithi073 ceph-mon[36714]: pgmap v1086: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 427 KiB/s wr, 2.93k op/s 2024-07-31T15:12:33.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:33 smithi077 ceph-mon[37327]: pgmap v1086: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 427 KiB/s wr, 2.93k op/s 2024-07-31T15:12:33.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:33 smithi067 ceph-mon[32750]: pgmap v1086: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 427 KiB/s wr, 2.93k op/s 2024-07-31T15:12:33.817 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/mem.stub.o 2024-07-31T15:12:33.840 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/pci.stub.o 2024-07-31T15:12:33.863 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/printk.stub.o 2024-07-31T15:12:33.886 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/random.stub.o 2024-07-31T15:12:33.910 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/randomalloc.stub.o 2024-07-31T15:12:33.933 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/relocate.stub.o 2024-07-31T15:12:33.957 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/secureboot.stub.o 2024-07-31T15:12:33.979 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/skip_spaces.stub.o 2024-07-31T15:12:34.001 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/tpm.stub.o 2024-07-31T15:12:34.028 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_fdi.o 2024-07-31T15:12:34.028 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/vsprintf.stub.o 2024-07-31T15:12:34.050 INFO:tasks.workunit.client.0.smithi067.stdout: STUBCPY drivers/firmware/efi/libstub/x86-stub.stub.o 2024-07-31T15:12:34.076 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/firmware/efi/libstub/lib.a 2024-07-31T15:12:34.107 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_fifo_underrun.o 2024-07-31T15:12:34.152 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/efi-bgrt.o 2024-07-31T15:12:34.384 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_frontbuffer.o 2024-07-31T15:12:34.539 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-quirks.o 2024-07-31T15:12:34.588 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_global_state.o 2024-07-31T15:12:34.715 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/efi.o 2024-07-31T15:12:34.987 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_hdcp.o 2024-07-31T15:12:35.168 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_hdcp_gsc.o 2024-07-31T15:12:35.254 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-debug.o 2024-07-31T15:12:35.279 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hidraw.o 2024-07-31T15:12:35.344 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-generic.o 2024-07-31T15:12:35.487 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_hotplug.o 2024-07-31T15:12:35.586 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_hotplug_irq.o 2024-07-31T15:12:35.613 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/vars.o 2024-07-31T15:12:35.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:35 smithi073 ceph-mon[36714]: pgmap v1087: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 584 KiB/s wr, 2.25k op/s 2024-07-31T15:12:35.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:35 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:12:35.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:35 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:12:35.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:35 smithi077 ceph-mon[37327]: pgmap v1087: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 584 KiB/s wr, 2.25k op/s 2024-07-31T15:12:35.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:35 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:12:35.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:35 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:12:35.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:35 smithi067 ceph-mon[32750]: pgmap v1087: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 584 KiB/s wr, 2.25k op/s 2024-07-31T15:12:35.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:35 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:12:35.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:35 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:12:35.951 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-a4tech.o 2024-07-31T15:12:36.045 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-apple.o 2024-07-31T15:12:36.065 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-belkin.o 2024-07-31T15:12:36.143 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/reboot.o 2024-07-31T15:12:36.152 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_hti.o 2024-07-31T15:12:36.582 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-cherry.o 2024-07-31T15:12:36.593 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_load_detect.o 2024-07-31T15:12:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:36 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:12:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:36 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:12:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:36 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:12:36.652 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/memattr.o 2024-07-31T15:12:36.682 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-chicony.o 2024-07-31T15:12:36.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:36 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:12:36.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:36 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:12:36.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:36 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:12:36.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:36 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:12:36.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:36 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:12:36.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:36 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:12:36.810 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-cypress.o 2024-07-31T15:12:36.816 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_lpe_audio.o 2024-07-31T15:12:36.986 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_modeset_lock.o 2024-07-31T15:12:37.016 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_modeset_verify.o 2024-07-31T15:12:37.203 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-ezkey.o 2024-07-31T15:12:37.257 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/tpm.o 2024-07-31T15:12:37.367 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/memmap.o 2024-07-31T15:12:37.438 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-gyration.o 2024-07-31T15:12:37.520 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_modeset_setup.o 2024-07-31T15:12:37.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:37 smithi073 ceph-mon[36714]: pgmap v1088: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 565 KiB/s wr, 401 op/s 2024-07-31T15:12:37.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:37 smithi077 ceph-mon[37327]: pgmap v1088: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 565 KiB/s wr, 401 op/s 2024-07-31T15:12:37.789 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_overlay.o 2024-07-31T15:12:37.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:37 smithi067 ceph-mon[32750]: pgmap v1088: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 565 KiB/s wr, 401 op/s 2024-07-31T15:12:37.811 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-ite.o 2024-07-31T15:12:37.846 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_pch_display.o 2024-07-31T15:12:37.878 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/esrt.o 2024-07-31T15:12:38.012 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/runtime-wrappers.o 2024-07-31T15:12:38.015 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_pch_refclk.o 2024-07-31T15:12:38.075 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-kensington.o 2024-07-31T15:12:38.447 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-lg.o 2024-07-31T15:12:38.520 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/efi/earlycon.o 2024-07-31T15:12:38.682 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-lgff.o 2024-07-31T15:12:38.834 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_plane_initial.o 2024-07-31T15:12:38.859 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_pmdemand.o 2024-07-31T15:12:39.065 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_psr.o 2024-07-31T15:12:39.072 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:12:39.208 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-lg4ff.o 2024-07-31T15:12:39.210 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_quirks.o 2024-07-31T15:12:39.302 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/firmware/efi/built-in.a 2024-07-31T15:12:39.320 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-lg-g15.o 2024-07-31T15:12:39.346 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-microsoft.o 2024-07-31T15:12:39.440 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/firmware/psci/built-in.a 2024-07-31T15:12:39.511 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/firmware/smccc/built-in.a 2024-07-31T15:12:39.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:39 smithi067 ceph-mon[32750]: pgmap v1089: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 662 KiB/s wr, 182 op/s 2024-07-31T15:12:39.579 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:12:39.583 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/firmware/tegra/built-in.a 2024-07-31T15:12:39.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:39 smithi073 ceph-mon[36714]: pgmap v1089: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 662 KiB/s wr, 182 op/s 2024-07-31T15:12:39.661 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/firmware/xilinx/built-in.a 2024-07-31T15:12:39.677 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/dmi_scan.o 2024-07-31T15:12:39.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:39 smithi077 ceph-mon[37327]: pgmap v1089: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 662 KiB/s wr, 182 op/s 2024-07-31T15:12:39.910 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/dmi-id.o 2024-07-31T15:12:40.085 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/firmware/memmap.o 2024-07-31T15:12:40.169 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_sprite.o 2024-07-31T15:12:40.259 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-monterey.o 2024-07-31T15:12:40.265 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-ntrig.o 2024-07-31T15:12:40.324 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_sprite_uapi.o 2024-07-31T15:12:40.397 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_tc.o 2024-07-31T15:12:40.566 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_vblank.o 2024-07-31T15:12:40.631 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/firmware/built-in.a 2024-07-31T15:12:40.689 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/perf/built-in.a 2024-07-31T15:12:40.714 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/hwtracing/intel_th/built-in.a 2024-07-31T15:12:40.738 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/android/built-in.a 2024-07-31T15:12:40.751 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_vga.o 2024-07-31T15:12:40.881 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_wm.o 2024-07-31T15:12:41.137 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-pl.o 2024-07-31T15:12:41.249 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/i9xx_plane.o 2024-07-31T15:12:41.262 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/i9xx_wm.o 2024-07-31T15:12:41.581 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:41 smithi067 ceph-mon[32750]: pgmap v1090: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 661 KiB/s wr, 122 op/s 2024-07-31T15:12:41.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:41 smithi073 ceph-mon[36714]: pgmap v1090: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 661 KiB/s wr, 122 op/s 2024-07-31T15:12:41.657 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/skl_scaler.o 2024-07-31T15:12:41.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:41 smithi077 ceph-mon[37327]: pgmap v1090: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 661 KiB/s wr, 122 op/s 2024-07-31T15:12:41.758 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/skl_universal_plane.o 2024-07-31T15:12:41.852 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-petalynx.o 2024-07-31T15:12:41.911 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/skl_watermark.o 2024-07-31T15:12:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.925+0000 7f7fce266640 1 -- 172.21.15.67:0/2400506629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fc8111970 msgr2=0x7f7fc8109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.925+0000 7f7fce266640 1 --2- 172.21.15.67:0/2400506629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fc8111970 0x7f7fc8109df0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009900 tx=0x7f7fbc02a440 comp rx=0 tx=0).stop 2024-07-31T15:12:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.925+0000 7f7fce266640 1 -- 172.21.15.67:0/2400506629 shutdown_connections 2024-07-31T15:12:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.925+0000 7f7fce266640 1 --2- 172.21.15.67:0/2400506629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc811c660 0x7f7fc811eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.925+0000 7f7fce266640 1 --2- 172.21.15.67:0/2400506629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc810a410 0x7f7fc810a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.925+0000 7f7fce266640 1 --2- 172.21.15.67:0/2400506629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fc8111970 0x7f7fc8109df0 secure :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009900 tx=0x7f7fbc02a440 comp rx=0 tx=0).stop 2024-07-31T15:12:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.925+0000 7f7fce266640 1 -- 172.21.15.67:0/2400506629 >> 172.21.15.67:0/2400506629 conn(0x7f7fc80708f0 msgr2=0x7f7fc8070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.926+0000 7f7fce266640 1 -- 172.21.15.67:0/2400506629 shutdown_connections 2024-07-31T15:12:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.926+0000 7f7fce266640 1 -- 172.21.15.67:0/2400506629 wait complete. 2024-07-31T15:12:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.927+0000 7f7fce266640 1 Processor -- start 2024-07-31T15:12:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.927+0000 7f7fce266640 1 -- start start 2024-07-31T15:12:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.927+0000 7f7fce266640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc810a410 0x7f7fc81a8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.927+0000 7f7fce266640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc811c660 0x7f7fc81a93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.927+0000 7f7fce266640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fc81ae5b0 0x7f7fc81ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.927+0000 7f7fce266640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8122320 con 0x7f7fc81ae5b0 2024-07-31T15:12:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.927+0000 7f7fce266640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81221a0 con 0x7f7fc810a410 2024-07-31T15:12:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.927+0000 7f7fce266640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81224a0 con 0x7f7fc811c660 2024-07-31T15:12:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.928+0000 7f7fcc803640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fc81ae5b0 0x7f7fc81ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.928+0000 7f7fcc803640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fc81ae5b0 0x7f7fc81ae9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34338/0 (socket says 172.21.15.67:34338) 2024-07-31T15:12:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.928+0000 7f7fcc803640 1 -- 172.21.15.67:0/603023870 learned_addr learned my addr 172.21.15.67:0/603023870 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.928+0000 7f7fcc803640 1 -- 172.21.15.67:0/603023870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc811c660 msgr2=0x7f7fc81a93d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:12:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.928+0000 7f7fcc803640 1 --2- 172.21.15.67:0/603023870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc811c660 0x7f7fc81a93d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.928+0000 7f7fcc803640 1 -- 172.21.15.67:0/603023870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc810a410 msgr2=0x7f7fc81a8e30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:12:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.928+0000 7f7fcc803640 1 --2- 172.21.15.67:0/603023870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc810a410 0x7f7fc81a8e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.928+0000 7f7fcc803640 1 -- 172.21.15.67:0/603023870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc81af200 con 0x7f7fc81ae5b0 2024-07-31T15:12:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.929+0000 7f7fcc803640 1 --2- 172.21.15.67:0/603023870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fc81ae5b0 0x7f7fc81ae9e0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000eb10 tx=0x7f7fc000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.931+0000 7f7fc57fa640 1 -- 172.21.15.67:0/603023870 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0004b20 con 0x7f7fc81ae5b0 2024-07-31T15:12:41.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.931+0000 7f7fc57fa640 1 -- 172.21.15.67:0/603023870 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc0004cc0 con 0x7f7fc81ae5b0 2024-07-31T15:12:41.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.931+0000 7f7fc57fa640 1 -- 172.21.15.67:0/603023870 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0010670 con 0x7f7fc81ae5b0 2024-07-31T15:12:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.934+0000 7f7fce266640 1 -- 172.21.15.67:0/603023870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc8111db0 con 0x7f7fc81ae5b0 2024-07-31T15:12:41.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.940+0000 7f7fce266640 1 -- 172.21.15.67:0/603023870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc81122a0 con 0x7f7fc81ae5b0 2024-07-31T15:12:41.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.940+0000 7f7fc57fa640 1 -- 172.21.15.67:0/603023870 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7fc0010810 con 0x7f7fc81ae5b0 2024-07-31T15:12:41.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.942+0000 7f7fc57fa640 1 --2- 172.21.15.67:0/603023870 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7fb00b3ed0 0x7f7fb00b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:41.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.942+0000 7f7fc7fff640 1 --2- 172.21.15.67:0/603023870 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7fb00b3ed0 0x7f7fb00b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:41.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.942+0000 7f7fc57fa640 1 -- 172.21.15.67:0/603023870 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f7fc00e8060 con 0x7f7fc81ae5b0 2024-07-31T15:12:41.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.949+0000 7f7fd47b7740 1 Processor -- start 2024-07-31T15:12:41.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.950+0000 7f7fc7fff640 1 --2- 172.21.15.67:0/603023870 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7fb00b3ed0 0x7f7fb00b63e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009230 tx=0x7f7fbc005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:41.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.950+0000 7f7fd47b7740 1 -- start start 2024-07-31T15:12:41.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.950+0000 7f7fd47b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 0x56311b182550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:41.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.950+0000 7f7fd47b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adb41b0 0x56311b182a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:41.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.950+0000 7f7fd47b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311adbc390 0x56311b182fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:41.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.950+0000 7f7fd47b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56311b0f8b90 con 0x56311adbc390 2024-07-31T15:12:41.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.951+0000 7f7fd47b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56311b0e8090 con 0x56311adb41b0 2024-07-31T15:12:41.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.951+0000 7f7fd47b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56311b0ec0d0 con 0x56311addd070 2024-07-31T15:12:41.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.951+0000 7f7fc7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 0x56311b182550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:41.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.951+0000 7f7fcc803640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311adbc390 0x56311b182fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:41.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.951+0000 7f7fcc803640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311adbc390 0x56311b182fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34352/0 (socket says 172.21.15.67:34352) 2024-07-31T15:12:41.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.951+0000 7f7fcc803640 1 -- 172.21.15.67:0/1953697817 learned_addr learned my addr 172.21.15.67:0/1953697817 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:41.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.951+0000 7f7fcc803640 1 -- 172.21.15.67:0/1953697817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 msgr2=0x56311b182550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:41.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.951+0000 7f7fcc803640 1 --2- 172.21.15.67:0/1953697817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 0x56311b182550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:41.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.951+0000 7f7fcc803640 1 -- 172.21.15.67:0/1953697817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adb41b0 msgr2=0x56311b182a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:12:41.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.951+0000 7f7fcc803640 1 --2- 172.21.15.67:0/1953697817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adb41b0 0x56311b182a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:41.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.951+0000 7f7fcc803640 1 -- 172.21.15.67:0/1953697817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56311b08eb70 con 0x56311adbc390 2024-07-31T15:12:41.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.952+0000 7f7fcc803640 1 --2- 172.21.15.67:0/1953697817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311adbc390 0x56311b182fd0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00aa980 tx=0x7f7fc00aa5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32fb5fd95ef0ef3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:41.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.952+0000 7f7fae7fc640 1 -- 172.21.15.67:0/1953697817 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00aa790 con 0x56311adbc390 2024-07-31T15:12:41.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.952+0000 7f7fae7fc640 1 -- 172.21.15.67:0/1953697817 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc00cd040 con 0x56311adbc390 2024-07-31T15:12:41.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.953+0000 7f7fae7fc640 1 -- 172.21.15.67:0/1953697817 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00b3d30 con 0x56311adbc390 2024-07-31T15:12:41.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.957+0000 7f7fd47b7740 1 -- 172.21.15.67:0/1953697817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311adbc390 msgr2=0x56311b182fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:41.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.957+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/1953697817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311adbc390 0x56311b182fd0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00aa980 tx=0x7f7fc00aa5d0 comp rx=0 tx=0).stop 2024-07-31T15:12:41.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.963+0000 7f7fd47b7740 1 -- 172.21.15.67:0/1953697817 shutdown_connections 2024-07-31T15:12:41.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.963+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/1953697817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311adbc390 0x56311b182fd0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:41.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.963+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/1953697817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adb41b0 0x56311b182a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:41.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.963+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/1953697817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 0x56311b182550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:41.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.963+0000 7f7fd47b7740 1 -- 172.21.15.67:0/1953697817 >> 172.21.15.67:0/1953697817 conn(0x56311add1460 msgr2=0x56311b180440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:41.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.966+0000 7f7fd47b7740 1 -- 172.21.15.67:0/1953697817 shutdown_connections 2024-07-31T15:12:41.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.966+0000 7f7fd47b7740 1 -- 172.21.15.67:0/1953697817 wait complete. 2024-07-31T15:12:41.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.966+0000 7f7fd47b7740 1 -- start start 2024-07-31T15:12:41.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.967+0000 7f7fd47b7740 1 Processor -- start 2024-07-31T15:12:41.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.968+0000 7f7fd47b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311adb41b0 0x56311b18e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:41.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.968+0000 7f7fd47b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adbc390 0x56311b18e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:41.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.968+0000 7f7fd47b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 0x56311b19a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:41.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.968+0000 7f7fd47b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56311af4b700 con 0x56311adb41b0 2024-07-31T15:12:41.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.968+0000 7f7fd47b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56311b08a9c0 con 0x56311adbc390 2024-07-31T15:12:41.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.968+0000 7f7fd47b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56311b190cd0 con 0x56311addd070 2024-07-31T15:12:41.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.968+0000 7f7fd47b7740 10 client.0 _ll_register_callbacks cb 0x56311b0a54d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:41.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.968+0000 7f7fc77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adbc390 0x56311b18e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:41.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.968+0000 7f7fc77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adbc390 0x56311b18e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60922/0 (socket says 172.21.15.67:60922) 2024-07-31T15:12:41.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.968+0000 7f7fc77fe640 1 -- 172.21.15.67:0/2521917867 learned_addr learned my addr 172.21.15.67:0/2521917867 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:41.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.973+0000 7f7fc77fe640 1 -- 172.21.15.67:0/2521917867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 msgr2=0x56311b19a040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:12:41.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.973+0000 7f7fc77fe640 1 --2- 172.21.15.67:0/2521917867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 0x56311b19a040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:41.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.973+0000 7f7fc77fe640 1 -- 172.21.15.67:0/2521917867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311adb41b0 msgr2=0x56311b18e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:41.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.973+0000 7f7fc77fe640 1 --2- 172.21.15.67:0/2521917867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311adb41b0 0x56311b18e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:41.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.973+0000 7f7fc77fe640 1 -- 172.21.15.67:0/2521917867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56311ae48650 con 0x56311adbc390 2024-07-31T15:12:41.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.980+0000 7f7fc77fe640 1 --2- 172.21.15.67:0/2521917867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adbc390 0x56311b18e690 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800af70 tx=0x7f7fb800c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f73a7473e080d543 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:41.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.980+0000 7f7fa77fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:12:41.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.981+0000 7f7fa77fe640 1 -- 172.21.15.67:0/2521917867 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8004230 con 0x56311adbc390 2024-07-31T15:12:41.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.981+0000 7f7fa77fe640 1 -- 172.21.15.67:0/2521917867 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f7fb80043d0 con 0x56311adbc390 2024-07-31T15:12:41.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.982+0000 7f7fd47b7740 1 -- 172.21.15.67:0/2521917867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc00cd040 con 0x56311adbc390 2024-07-31T15:12:41.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.989+0000 7f7fa77fe640 1 -- 172.21.15.67:0/2521917867 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f7fb801f050 con 0x56311adbc390 2024-07-31T15:12:41.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.990+0000 7f7fd47b7740 10 client.29504 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:41.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.990+0000 7f7fd47b7740 1 -- 172.21.15.67:0/2521917867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56311b19cdf0 con 0x56311adbc390 2024-07-31T15:12:41.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.995+0000 7f7fa77fe640 1 -- 172.21.15.67:0/2521917867 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc00cd040 con 0x56311adbc390 2024-07-31T15:12:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.995+0000 7f7fa77fe640 1 -- 172.21.15.67:0/2521917867 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f7fb8003ab0 con 0x56311adbc390 2024-07-31T15:12:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.995+0000 7f7fd47b7740 10 client.29504 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.995+0000 7f7fd47b7740 10 client.29504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:12:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.995+0000 7f7fd47b7740 20 client.29504 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.995+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/2521917867 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56311ad9b9f0 0x56311b1a0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.995+0000 7f7fd47b7740 4 client.29504 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:12:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.995+0000 7f7fd47b7740 1 -- 172.21.15.67:0/2521917867 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56311b0faae0 con 0x56311ad9b9f0 2024-07-31T15:12:41.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.997+0000 7f7fc7fff640 1 --2- 172.21.15.67:0/2521917867 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56311ad9b9f0 0x56311b1a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:41.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.998+0000 7f7fc7fff640 1 --2- 172.21.15.67:0/2521917867 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56311ad9b9f0 0x56311b1a0840 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9b6c2d2b48915ae server_cookie=7e85a6bdf6f736f2 in_seq=0 out_seq=0 2024-07-31T15:12:42.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:41.998+0000 7f7fa77fe640 10 client.29504 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:42.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.001+0000 7f7fa77fe640 1 -- 172.21.15.67:0/2521917867 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56311b0faae0 con 0x56311ad9b9f0 2024-07-31T15:12:42.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.001+0000 7f7fa77fe640 10 client.29504 handle_command_reply: tid=0 2024-07-31T15:12:42.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.001+0000 7f7fd47b7740 1 client.29504 shutdown 2024-07-31T15:12:42.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.004+0000 7f7fd47b7740 1 -- 172.21.15.67:0/2521917867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adbc390 msgr2=0x56311b18e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:42.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.004+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/2521917867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adbc390 0x56311b18e690 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800af70 tx=0x7f7fb800c760 comp rx=0 tx=0).stop 2024-07-31T15:12:42.006 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_acpi.o 2024-07-31T15:12:42.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.007+0000 7f7fd47b7740 1 -- 172.21.15.67:0/2521917867 shutdown_connections 2024-07-31T15:12:42.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.007+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/2521917867 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56311ad9b9f0 0x56311b1a0840 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.007+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/2521917867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 0x56311b19a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.007+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/2521917867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adbc390 0x56311b18e690 secure :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800af70 tx=0x7f7fb800c760 comp rx=0 tx=0).stop 2024-07-31T15:12:42.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.007+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/2521917867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311adb41b0 0x56311b18e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.007+0000 7f7fd47b7740 1 -- 172.21.15.67:0/2521917867 >> 172.21.15.67:0/2521917867 conn(0x56311add1460 msgr2=0x56311b17ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:42.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.007+0000 7f7fa77fe640 0 client.29504 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:42.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.009+0000 7f7fd47b7740 1 -- 172.21.15.67:0/2521917867 shutdown_connections 2024-07-31T15:12:42.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.009+0000 7f7fd47b7740 1 -- 172.21.15.67:0/2521917867 wait complete. 2024-07-31T15:12:42.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.009+0000 7f7fd47b7740 20 client.29504 trim_cache size 0 max 16384 2024-07-31T15:12:42.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.048+0000 7f7fd47b7740 1 Processor -- start 2024-07-31T15:12:42.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.048+0000 7f7fd47b7740 1 -- start start 2024-07-31T15:12:42.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.049+0000 7f7fd47b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adb41b0 0x56311b187d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.049+0000 7f7fd47b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 0x56311b199e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.049+0000 7f7fd47b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311ad70da0 0x56311b19a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.049+0000 7f7fd47b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81c8680 con 0x56311ad70da0 2024-07-31T15:12:42.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.049+0000 7f7fd47b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81ca1c0 con 0x56311adb41b0 2024-07-31T15:12:42.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.049+0000 7f7fd47b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81c8d50 con 0x56311addd070 2024-07-31T15:12:42.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.049+0000 7f7fc77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 0x56311b199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.049+0000 7f7fc7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adb41b0 0x56311b187d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.049+0000 7f7fc77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 0x56311b199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49956/0 (socket says 172.21.15.67:49956) 2024-07-31T15:12:42.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.049+0000 7f7fc77fe640 1 -- 172.21.15.67:0/2003033839 learned_addr learned my addr 172.21.15.67:0/2003033839 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:42.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.049+0000 7f7fcc803640 1 --2- 172.21.15.67:0/2003033839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311ad70da0 0x56311b19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.050+0000 7f7fc77fe640 1 -- 172.21.15.67:0/2003033839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adb41b0 msgr2=0x56311b187d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:42.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.050+0000 7f7fc77fe640 1 --2- 172.21.15.67:0/2003033839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adb41b0 0x56311b187d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.050+0000 7f7fc77fe640 1 -- 172.21.15.67:0/2003033839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311ad70da0 msgr2=0x56311b19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:42.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.050+0000 7f7fc77fe640 1 --2- 172.21.15.67:0/2003033839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311ad70da0 0x56311b19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.050+0000 7f7fc77fe640 1 -- 172.21.15.67:0/2003033839 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56311b190530 con 0x56311addd070 2024-07-31T15:12:42.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.050+0000 7f7fc77fe640 1 --2- 172.21.15.67:0/2003033839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 0x56311b199e30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8000c00 tx=0x7f7fb800e430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7d57caf42875786 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:42.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.050+0000 7f7fadffb640 1 -- 172.21.15.67:0/2003033839 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8028070 con 0x56311addd070 2024-07-31T15:12:42.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.050+0000 7f7fadffb640 1 -- 172.21.15.67:0/2003033839 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fb8005520 con 0x56311addd070 2024-07-31T15:12:42.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.052+0000 7f7fd47b7740 1 -- 172.21.15.67:0/2003033839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 msgr2=0x56311b199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:42.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.052+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/2003033839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 0x56311b199e30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8000c00 tx=0x7f7fb800e430 comp rx=0 tx=0).stop 2024-07-31T15:12:42.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.055+0000 7f7fd47b7740 1 -- 172.21.15.67:0/2003033839 shutdown_connections 2024-07-31T15:12:42.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.055+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/2003033839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311ad70da0 0x56311b19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.055+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/2003033839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311addd070 0x56311b199e30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.055+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/2003033839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311adb41b0 0x56311b187d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.055+0000 7f7fd47b7740 1 -- 172.21.15.67:0/2003033839 >> 172.21.15.67:0/2003033839 conn(0x56311add1460 msgr2=0x56311b187660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.059+0000 7f7fd47b7740 1 -- 172.21.15.67:0/2003033839 shutdown_connections 2024-07-31T15:12:42.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.064+0000 7f7fd47b7740 1 -- 172.21.15.67:0/2003033839 wait complete. 2024-07-31T15:12:42.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.064+0000 7f7fd47b7740 1 -- start start 2024-07-31T15:12:42.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.064+0000 7f7fd47b7740 1 Processor -- start 2024-07-31T15:12:42.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.067+0000 7f7fd47b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311ad70da0 0x56311b196810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.067+0000 7f7fd47b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311adb41b0 0x56311b191cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.067+0000 7f7fd47b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311addd070 0x56311b192390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.067+0000 7f7fd47b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81c9af0 con 0x56311addd070 2024-07-31T15:12:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.067+0000 7f7fd47b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81c9420 con 0x56311ad70da0 2024-07-31T15:12:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.067+0000 7f7fd47b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81af3d0 con 0x56311adb41b0 2024-07-31T15:12:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.067+0000 7f7fd47b7740 10 client.0 _ll_register_callbacks cb 0x7f7fb8005dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.068+0000 7f7fc7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311ad70da0 0x56311b196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.068+0000 7f7fcc803640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311addd070 0x56311b192390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.068+0000 7f7fc7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311ad70da0 0x56311b196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60944/0 (socket says 172.21.15.67:60944) 2024-07-31T15:12:42.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.068+0000 7f7fc7fff640 1 -- 172.21.15.67:0/1517572585 learned_addr learned my addr 172.21.15.67:0/1517572585 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:42.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.069+0000 7f7fc77fe640 1 --2- 172.21.15.67:0/1517572585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311adb41b0 0x56311b191cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.072+0000 7f7fcc803640 1 -- 172.21.15.67:0/1517572585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311adb41b0 msgr2=0x56311b191cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.072+0000 7f7fcc803640 1 --2- 172.21.15.67:0/1517572585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311adb41b0 0x56311b191cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.072+0000 7f7fcc803640 1 -- 172.21.15.67:0/1517572585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311ad70da0 msgr2=0x56311b196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.072+0000 7f7fcc803640 1 --2- 172.21.15.67:0/1517572585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311ad70da0 0x56311b196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.072+0000 7f7fcc803640 1 -- 172.21.15.67:0/1517572585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56311b208010 con 0x56311addd070 2024-07-31T15:12:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.073+0000 7f7fcc803640 1 --2- 172.21.15.67:0/1517572585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311addd070 0x56311b192390 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00049d0 tx=0x7f7fc00b3ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68155ab28397421b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:42.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.074+0000 7f7fa77fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:12:42.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.074+0000 7f7fa77fe640 1 -- 172.21.15.67:0/1517572585 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00c9020 con 0x56311addd070 2024-07-31T15:12:42.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.074+0000 7f7fa77fe640 1 -- 172.21.15.67:0/1517572585 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f7fc00aa0f0 con 0x56311addd070 2024-07-31T15:12:42.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.075+0000 7f7fd47b7740 1 -- 172.21.15.67:0/1517572585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fb8005520 con 0x56311addd070 2024-07-31T15:12:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.078+0000 7f7fa77fe640 1 -- 172.21.15.67:0/1517572585 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f7fc00aa790 con 0x56311addd070 2024-07-31T15:12:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.079+0000 7f7fd47b7740 10 client.19992 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:42.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.079+0000 7f7fd47b7740 1 -- 172.21.15.67:0/1517572585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56311b2090f0 con 0x56311addd070 2024-07-31T15:12:42.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.082+0000 7f7fa77fe640 1 -- 172.21.15.67:0/1517572585 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fb8005520 con 0x56311addd070 2024-07-31T15:12:42.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.082+0000 7f7fa77fe640 1 -- 172.21.15.67:0/1517572585 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f7fc00a65f0 con 0x56311addd070 2024-07-31T15:12:42.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.083+0000 7f7fd47b7740 10 client.19992 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:42.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.083+0000 7f7fd47b7740 10 client.19992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:12:42.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.083+0000 7f7fd47b7740 20 client.19992 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:42.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.083+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/1517572585 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56311b209320 0x56311b20b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.083+0000 7f7fd47b7740 4 client.19992 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:12:42.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.083+0000 7f7fd47b7740 1 -- 172.21.15.67:0/1517572585 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56311b0f6580 con 0x56311b209320 2024-07-31T15:12:42.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.087+0000 7f7fc7fff640 1 --2- 172.21.15.67:0/1517572585 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56311b209320 0x56311b20b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.087+0000 7f7fa77fe640 10 client.19992 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:42.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.087+0000 7f7fc7fff640 1 --2- 172.21.15.67:0/1517572585 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56311b209320 0x56311b20b740 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6755d8344c74cd45 server_cookie=8e88fa274b9ce3ab in_seq=0 out_seq=0 2024-07-31T15:12:42.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.091+0000 7f7fa77fe640 1 -- 172.21.15.67:0/1517572585 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56311b0f6580 con 0x56311b209320 2024-07-31T15:12:42.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.091+0000 7f7fa77fe640 10 client.19992 handle_command_reply: tid=0 2024-07-31T15:12:42.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.093+0000 7f7fd47b7740 1 client.19992 shutdown 2024-07-31T15:12:42.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.099+0000 7f7fd47b7740 1 -- 172.21.15.67:0/1517572585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311addd070 msgr2=0x56311b192390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:42.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.099+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/1517572585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311addd070 0x56311b192390 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00049d0 tx=0x7f7fc00b3ab0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.103+0000 7f7fd47b7740 1 -- 172.21.15.67:0/1517572585 shutdown_connections 2024-07-31T15:12:42.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.103+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/1517572585 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56311b209320 0x56311b20b740 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.103+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/1517572585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56311addd070 0x56311b192390 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.103+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/1517572585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56311adb41b0 0x56311b191cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.103+0000 7f7fd47b7740 1 --2- 172.21.15.67:0/1517572585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56311ad70da0 0x56311b196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.103+0000 7f7fd47b7740 1 -- 172.21.15.67:0/1517572585 >> 172.21.15.67:0/1517572585 conn(0x56311add1460 msgr2=0x56311b17fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:42.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.104+0000 7f7fa77fe640 0 client.19992 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:42.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.104+0000 7f7fd47b7740 1 -- 172.21.15.67:0/1517572585 shutdown_connections 2024-07-31T15:12:42.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.104+0000 7f7fd47b7740 1 -- 172.21.15.67:0/1517572585 wait complete. 2024-07-31T15:12:42.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.105+0000 7f7fd47b7740 20 client.19992 trim_cache size 0 max 16384 2024-07-31T15:12:42.108 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:12:42.108 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T15:12:42.109 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T15:12:42.109 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:12:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.106+0000 7f7fce266640 1 -- 172.21.15.67:0/603023870 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7fb00b3ed0 msgr2=0x7f7fb00b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:42.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.106+0000 7f7fce266640 1 --2- 172.21.15.67:0/603023870 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7fb00b3ed0 0x7f7fb00b63e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009230 tx=0x7f7fbc005fd0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.106+0000 7f7fce266640 1 -- 172.21.15.67:0/603023870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fc81ae5b0 msgr2=0x7f7fc81ae9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:42.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.106+0000 7f7fce266640 1 --2- 172.21.15.67:0/603023870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fc81ae5b0 0x7f7fc81ae9e0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000eb10 tx=0x7f7fc000c2d0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.106+0000 7f7fce266640 1 -- 172.21.15.67:0/603023870 shutdown_connections 2024-07-31T15:12:42.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.106+0000 7f7fce266640 1 --2- 172.21.15.67:0/603023870 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7fb00b3ed0 0x7f7fb00b63e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.106+0000 7f7fce266640 1 --2- 172.21.15.67:0/603023870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fc81ae5b0 0x7f7fc81ae9e0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.106+0000 7f7fce266640 1 --2- 172.21.15.67:0/603023870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc811c660 0x7f7fc81a93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.106+0000 7f7fce266640 1 --2- 172.21.15.67:0/603023870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc810a410 0x7f7fc81a8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.106+0000 7f7fce266640 1 -- 172.21.15.67:0/603023870 >> 172.21.15.67:0/603023870 conn(0x7f7fc80708f0 msgr2=0x7f7fc811cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:42.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.106+0000 7f7fce266640 1 -- 172.21.15.67:0/603023870 shutdown_connections 2024-07-31T15:12:42.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:42.107+0000 7f7fce266640 1 -- 172.21.15.67:0/603023870 wait complete. 2024-07-31T15:12:42.143 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_opregion.o 2024-07-31T15:12:42.631 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-redragon.o 2024-07-31T15:12:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.794+0000 7f7c7e28c640 1 -- 172.21.15.67:0/270444069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7810b970 msgr2=0x7f7c7810bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.794+0000 7f7c7e28c640 1 --2- 172.21.15.67:0/270444069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7810b970 0x7f7c7810bd80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009900 tx=0x7f7c6802a410 comp rx=0 tx=0).stop 2024-07-31T15:12:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.797+0000 7f7c7e28c640 1 -- 172.21.15.67:0/270444069 shutdown_connections 2024-07-31T15:12:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.797+0000 7f7c7e28c640 1 --2- 172.21.15.67:0/270444069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c78114ea0 0x7f7c781173b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.797+0000 7f7c7e28c640 1 --2- 172.21.15.67:0/270444069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c7810c2c0 0x7f7c78114800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.797+0000 7f7c7e28c640 1 --2- 172.21.15.67:0/270444069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7810b970 0x7f7c7810bd80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.797+0000 7f7c7e28c640 1 -- 172.21.15.67:0/270444069 >> 172.21.15.67:0/270444069 conn(0x7f7c780707c0 msgr2=0x7f7c78070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.808+0000 7f7c7e28c640 1 -- 172.21.15.67:0/270444069 shutdown_connections 2024-07-31T15:12:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.808+0000 7f7c7e28c640 1 -- 172.21.15.67:0/270444069 wait complete. 2024-07-31T15:12:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.809+0000 7f7c7e28c640 1 Processor -- start 2024-07-31T15:12:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.810+0000 7f7c7e28c640 1 -- start start 2024-07-31T15:12:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.810+0000 7f7c7e28c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c7810b970 0x7f7c7810b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.810+0000 7f7c7e28c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7810c2c0 0x7f7c781a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.810+0000 7f7c7e28c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c78114ea0 0x7f7c781a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.810+0000 7f7c7e28c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7811ab70 con 0x7f7c7810b970 2024-07-31T15:12:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.810+0000 7f7c7e28c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7811a9f0 con 0x7f7c7810c2c0 2024-07-31T15:12:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.810+0000 7f7c7e28c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7811acf0 con 0x7f7c78114ea0 2024-07-31T15:12:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.810+0000 7f7c7c829640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c78114ea0 0x7f7c781a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.810+0000 7f7c7c829640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c78114ea0 0x7f7c781a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49974/0 (socket says 172.21.15.67:49974) 2024-07-31T15:12:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.811+0000 7f7c7c829640 1 -- 172.21.15.67:0/2296973446 learned_addr learned my addr 172.21.15.67:0/2296973446 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.811+0000 7f7c777fe640 1 --2- 172.21.15.67:0/2296973446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7810c2c0 0x7f7c781a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.811+0000 7f7c777fe640 1 -- 172.21.15.67:0/2296973446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c78114ea0 msgr2=0x7f7c781a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.811+0000 7f7c777fe640 1 --2- 172.21.15.67:0/2296973446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c78114ea0 0x7f7c781a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.811+0000 7f7c777fe640 1 -- 172.21.15.67:0/2296973446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c7810b970 msgr2=0x7f7c7810b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.811+0000 7f7c777fe640 1 --2- 172.21.15.67:0/2296973446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c7810b970 0x7f7c7810b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.811+0000 7f7c777fe640 1 -- 172.21.15.67:0/2296973446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c781a1e50 con 0x7f7c7810c2c0 2024-07-31T15:12:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.811+0000 7f7c777fe640 1 --2- 172.21.15.67:0/2296973446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7810c2c0 0x7f7c781a1270 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00e9c0 tx=0x7f7c6c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.825+0000 7f7c757fa640 1 -- 172.21.15.67:0/2296973446 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00ccc0 con 0x7f7c7810c2c0 2024-07-31T15:12:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.825+0000 7f7c7e28c640 1 -- 172.21.15.67:0/2296973446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c781a62c0 con 0x7f7c7810c2c0 2024-07-31T15:12:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.825+0000 7f7c7e28c640 1 -- 172.21.15.67:0/2296973446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c781a6810 con 0x7f7c7810c2c0 2024-07-31T15:12:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.825+0000 7f7c757fa640 1 -- 172.21.15.67:0/2296973446 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c6c0108e0 con 0x7f7c7810c2c0 2024-07-31T15:12:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.825+0000 7f7c757fa640 1 -- 172.21.15.67:0/2296973446 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c0187c0 con 0x7f7c7810c2c0 2024-07-31T15:12:42.829 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/dvo_ch7017.o 2024-07-31T15:12:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.830+0000 7f7c757fa640 1 -- 172.21.15.67:0/2296973446 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7c6c018960 con 0x7f7c7810c2c0 2024-07-31T15:12:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.830+0000 7f7c8517c740 1 Processor -- start 2024-07-31T15:12:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.830+0000 7f7c8517c740 1 -- start start 2024-07-31T15:12:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.831+0000 7f7c8517c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x557337380ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.831+0000 7f7c8517c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fb52f0 0x557337381220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.831+0000 7f7c8517c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 0x557337381760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.831+0000 7f7c8517c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5573372f63b0 con 0x557336fb52f0 2024-07-31T15:12:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.831+0000 7f7c7c829640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x557337380ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.831+0000 7f7c7c829640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x557337380ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60958/0 (socket says 172.21.15.67:60958) 2024-07-31T15:12:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.831+0000 7f7c7c829640 1 -- 172.21.15.67:0/167645437 learned_addr learned my addr 172.21.15.67:0/167645437 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.831+0000 7f7c8517c740 1 -- 172.21.15.67:0/167645437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5573372e58b0 con 0x557336fde1b0 2024-07-31T15:12:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.831+0000 7f7c8517c740 1 -- 172.21.15.67:0/167645437 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5573372e98f0 con 0x557336fbd4d0 2024-07-31T15:12:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.831+0000 7f7c77fff640 1 --2- 172.21.15.67:0/167645437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fb52f0 0x557337381220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.832+0000 7f7c7c829640 1 -- 172.21.15.67:0/167645437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 msgr2=0x557337381760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:12:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.832+0000 7f7c7c829640 1 --2- 172.21.15.67:0/167645437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 0x557337381760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.832+0000 7f7c7c829640 1 -- 172.21.15.67:0/167645437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fb52f0 msgr2=0x557337381220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.832+0000 7f7c7c829640 1 --2- 172.21.15.67:0/167645437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fb52f0 0x557337381220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.832+0000 7f7c7c829640 1 -- 172.21.15.67:0/167645437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55733728c9a0 con 0x557336fde1b0 2024-07-31T15:12:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.832+0000 7f7c77fff640 1 --2- 172.21.15.67:0/167645437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fb52f0 0x557337381220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:12:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.832+0000 7f7c7c829640 1 --2- 172.21.15.67:0/167645437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x557337380ce0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000f3f0 tx=0x7f7c7000f8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c39edf15d817d5dc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.833+0000 7f7c757fa640 1 --2- 172.21.15.67:0/2296973446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c600b3f50 0x7f7c600b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.833+0000 7f7c757fa640 1 -- 172.21.15.67:0/2296973446 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f7c6c010430 con 0x7f7c7810c2c0 2024-07-31T15:12:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.833+0000 7f7c77fff640 1 --2- 172.21.15.67:0/2296973446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c600b3f50 0x7f7c600b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.834+0000 7f7c77fff640 1 --2- 172.21.15.67:0/2296973446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c600b3f50 0x7f7c600b6460 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55733737f960 tx=0x7f7c68031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.843+0000 7f7c667fc640 1 -- 172.21.15.67:0/167645437 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70004d20 con 0x557336fde1b0 2024-07-31T15:12:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.843+0000 7f7c667fc640 1 -- 172.21.15.67:0/167645437 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c70014070 con 0x557336fde1b0 2024-07-31T15:12:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.844+0000 7f7c8517c740 1 -- 172.21.15.67:0/167645437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 msgr2=0x557337380ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.844+0000 7f7c8517c740 1 --2- 172.21.15.67:0/167645437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x557337380ce0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000f3f0 tx=0x7f7c7000f8b0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.844+0000 7f7c8517c740 1 -- 172.21.15.67:0/167645437 shutdown_connections 2024-07-31T15:12:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.844+0000 7f7c8517c740 1 --2- 172.21.15.67:0/167645437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 0x557337381760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.844+0000 7f7c8517c740 1 --2- 172.21.15.67:0/167645437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fb52f0 0x557337381220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.844+0000 7f7c8517c740 1 --2- 172.21.15.67:0/167645437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x557337380ce0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.844+0000 7f7c8517c740 1 -- 172.21.15.67:0/167645437 >> 172.21.15.67:0/167645437 conn(0x557336fd2300 msgr2=0x55733737ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.844+0000 7f7c8517c740 1 -- 172.21.15.67:0/167645437 shutdown_connections 2024-07-31T15:12:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.844+0000 7f7c8517c740 1 -- 172.21.15.67:0/167645437 wait complete. 2024-07-31T15:12:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.845+0000 7f7c8517c740 1 -- start start 2024-07-31T15:12:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.845+0000 7f7c8517c740 1 Processor -- start 2024-07-31T15:12:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.846+0000 7f7c8517c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fb52f0 0x55733738c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.846+0000 7f7c8517c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fbd4d0 0x55733738cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.846+0000 7f7c8517c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x557337398880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.846+0000 7f7c8517c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55733721fab0 con 0x557336fbd4d0 2024-07-31T15:12:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.846+0000 7f7c8517c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55733726dcf0 con 0x557336fde1b0 2024-07-31T15:12:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.846+0000 7f7c8517c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55733738f4b0 con 0x557336fb52f0 2024-07-31T15:12:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.846+0000 7f7c8517c740 10 client.0 _ll_register_callbacks cb 0x5573372df9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.846+0000 7f7c77fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fb52f0 0x55733738c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.847+0000 7f7c77fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fb52f0 0x55733738c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:49990/0 (socket says 172.21.15.67:49990) 2024-07-31T15:12:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.847+0000 7f7c77fff640 1 -- 172.21.15.67:0/1847775762 learned_addr learned my addr 172.21.15.67:0/1847775762 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.847+0000 7f7c777fe640 1 --2- 172.21.15.67:0/1847775762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fbd4d0 0x55733738cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.847+0000 7f7c7c829640 1 --2- 172.21.15.67:0/1847775762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x557337398880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.847+0000 7f7c77fff640 1 -- 172.21.15.67:0/1847775762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 msgr2=0x557337398880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.847+0000 7f7c77fff640 1 --2- 172.21.15.67:0/1847775762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x557337398880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.847+0000 7f7c77fff640 1 -- 172.21.15.67:0/1847775762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fbd4d0 msgr2=0x55733738cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.847+0000 7f7c77fff640 1 --2- 172.21.15.67:0/1847775762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fbd4d0 0x55733738cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.847+0000 7f7c77fff640 1 -- 172.21.15.67:0/1847775762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557337049790 con 0x557336fb52f0 2024-07-31T15:12:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.848+0000 7f7c777fe640 1 --2- 172.21.15.67:0/1847775762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fbd4d0 0x55733738cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:12:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.848+0000 7f7c77fff640 1 --2- 172.21.15.67:0/1847775762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fb52f0 0x55733738c540 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7c68033000 tx=0x7f7c68005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fef8baa6fb859d9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.857+0000 7f7c5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:12:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.857+0000 7f7c5b7fe640 1 -- 172.21.15.67:0/1847775762 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68035040 con 0x557336fb52f0 2024-07-31T15:12:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.857+0000 7f7c8517c740 1 -- 172.21.15.67:0/1847775762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c70014070 con 0x557336fb52f0 2024-07-31T15:12:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.857+0000 7f7c5b7fe640 1 -- 172.21.15.67:0/1847775762 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f7c68004ad0 con 0x557336fb52f0 2024-07-31T15:12:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.858+0000 7f7c5b7fe640 1 -- 172.21.15.67:0/1847775762 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f7c68056050 con 0x557336fb52f0 2024-07-31T15:12:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.858+0000 7f7c8517c740 10 client.29383 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.858+0000 7f7c8517c740 1 -- 172.21.15.67:0/1847775762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5573370c2620 con 0x557336fb52f0 2024-07-31T15:12:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.861+0000 7f7c5b7fe640 1 -- 172.21.15.67:0/1847775762 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c70014070 con 0x557336fb52f0 2024-07-31T15:12:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.862+0000 7f7c5b7fe640 1 -- 172.21.15.67:0/1847775762 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f7c6803a040 con 0x557336fb52f0 2024-07-31T15:12:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.862+0000 7f7c8517c740 10 client.29383 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.862+0000 7f7c8517c740 10 client.29383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:12:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.862+0000 7f7c8517c740 20 client.29383 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.862+0000 7f7c8517c740 1 --2- 172.21.15.67:0/1847775762 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557336f9cd80 0x55733739ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.862+0000 7f7c8517c740 4 client.29383 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:12:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.862+0000 7f7c8517c740 1 -- 172.21.15.67:0/1847775762 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573372f8300 con 0x557336f9cd80 2024-07-31T15:12:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.862+0000 7f7c777fe640 1 --2- 172.21.15.67:0/1847775762 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557336f9cd80 0x55733739ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.863+0000 7f7c5b7fe640 10 client.29383 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.863+0000 7f7c777fe640 1 --2- 172.21.15.67:0/1847775762 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557336f9cd80 0x55733739ec40 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c1d1be7a5b610c server_cookie=baa1365df16e9dcc in_seq=0 out_seq=0 2024-07-31T15:12:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.864+0000 7f7c5b7fe640 1 -- 172.21.15.67:0/1847775762 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573372f8300 con 0x557336f9cd80 2024-07-31T15:12:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.864+0000 7f7c5b7fe640 10 client.29383 handle_command_reply: tid=0 2024-07-31T15:12:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.864+0000 7f7c8517c740 1 client.29383 shutdown 2024-07-31T15:12:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.864+0000 7f7c8517c740 1 -- 172.21.15.67:0/1847775762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fb52f0 msgr2=0x55733738c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.864+0000 7f7c8517c740 1 --2- 172.21.15.67:0/1847775762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fb52f0 0x55733738c540 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7c68033000 tx=0x7f7c68005fd0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.865+0000 7f7c8517c740 1 -- 172.21.15.67:0/1847775762 shutdown_connections 2024-07-31T15:12:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.865+0000 7f7c8517c740 1 --2- 172.21.15.67:0/1847775762 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557336f9cd80 0x55733739ec40 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.865+0000 7f7c8517c740 1 --2- 172.21.15.67:0/1847775762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x557337398880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.865+0000 7f7c8517c740 1 --2- 172.21.15.67:0/1847775762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fbd4d0 0x55733738cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.865+0000 7f7c8517c740 1 --2- 172.21.15.67:0/1847775762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fb52f0 0x55733738c540 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.865+0000 7f7c8517c740 1 -- 172.21.15.67:0/1847775762 >> 172.21.15.67:0/1847775762 conn(0x557336fd2300 msgr2=0x557337381d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.865+0000 7f7c5b7fe640 0 client.29383 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.875+0000 7f7c8517c740 1 -- 172.21.15.67:0/1847775762 shutdown_connections 2024-07-31T15:12:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.875+0000 7f7c8517c740 1 -- 172.21.15.67:0/1847775762 wait complete. 2024-07-31T15:12:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.875+0000 7f7c8517c740 20 client.29383 trim_cache size 0 max 16384 2024-07-31T15:12:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.943+0000 7f7c8517c740 1 Processor -- start 2024-07-31T15:12:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.943+0000 7f7c8517c740 1 -- start start 2024-07-31T15:12:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.944+0000 7f7c8517c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fb52f0 0x557337386e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.944+0000 7f7c8517c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 0x55733737ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.944+0000 7f7c8517c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fde1b0 0x557337398670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.944+0000 7f7c8517c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c00ffa0 con 0x557336fde1b0 2024-07-31T15:12:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.944+0000 7f7c77fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fb52f0 0x557337386e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.944+0000 7f7c77fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fb52f0 0x557337386e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60982/0 (socket says 172.21.15.67:60982) 2024-07-31T15:12:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.944+0000 7f7c77fff640 1 -- 172.21.15.67:0/884137322 learned_addr learned my addr 172.21.15.67:0/884137322 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.944+0000 7f7c777fe640 1 --2- 172.21.15.67:0/884137322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 0x55733737ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.944+0000 7f7c7c829640 1 --2- 172.21.15.67:0/884137322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fde1b0 0x557337398670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.944+0000 7f7c8517c740 1 -- 172.21.15.67:0/884137322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c011ae0 con 0x557336fb52f0 2024-07-31T15:12:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.944+0000 7f7c8517c740 1 -- 172.21.15.67:0/884137322 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c010670 con 0x557336fbd4d0 2024-07-31T15:12:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.945+0000 7f7c77fff640 1 -- 172.21.15.67:0/884137322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 msgr2=0x55733737ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.945+0000 7f7c77fff640 1 --2- 172.21.15.67:0/884137322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 0x55733737ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.945+0000 7f7c77fff640 1 -- 172.21.15.67:0/884137322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fde1b0 msgr2=0x557337398670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.945+0000 7f7c77fff640 1 --2- 172.21.15.67:0/884137322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fde1b0 0x557337398670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.945+0000 7f7c77fff640 1 -- 172.21.15.67:0/884137322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55733738c2c0 con 0x557336fb52f0 2024-07-31T15:12:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.945+0000 7f7c7c829640 1 --2- 172.21.15.67:0/884137322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fde1b0 0x557337398670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:12:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.945+0000 7f7c77fff640 1 --2- 172.21.15.67:0/884137322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fb52f0 0x557337386e20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7c6802a9f0 tx=0x7f7c680053e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aa6d641d3de748a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.946+0000 7f7c65ffb640 1 -- 172.21.15.67:0/884137322 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68004ad0 con 0x557336fb52f0 2024-07-31T15:12:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.946+0000 7f7c65ffb640 1 -- 172.21.15.67:0/884137322 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c6804d070 con 0x557336fb52f0 2024-07-31T15:12:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.946+0000 7f7c65ffb640 1 -- 172.21.15.67:0/884137322 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68003850 con 0x557336fb52f0 2024-07-31T15:12:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.948+0000 7f7c8517c740 1 -- 172.21.15.67:0/884137322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fb52f0 msgr2=0x557337386e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.948+0000 7f7c8517c740 1 --2- 172.21.15.67:0/884137322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fb52f0 0x557337386e20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7c6802a9f0 tx=0x7f7c680053e0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.953+0000 7f7c8517c740 1 -- 172.21.15.67:0/884137322 shutdown_connections 2024-07-31T15:12:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.953+0000 7f7c8517c740 1 --2- 172.21.15.67:0/884137322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fde1b0 0x557337398670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.953+0000 7f7c8517c740 1 --2- 172.21.15.67:0/884137322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 0x55733737ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.953+0000 7f7c8517c740 1 --2- 172.21.15.67:0/884137322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fb52f0 0x557337386e20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.953+0000 7f7c8517c740 1 -- 172.21.15.67:0/884137322 >> 172.21.15.67:0/884137322 conn(0x557336fd2300 msgr2=0x55733737e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.954+0000 7f7c8517c740 1 -- 172.21.15.67:0/884137322 shutdown_connections 2024-07-31T15:12:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.954+0000 7f7c8517c740 1 -- 172.21.15.67:0/884137322 wait complete. 2024-07-31T15:12:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.955+0000 7f7c8517c740 1 -- start start 2024-07-31T15:12:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.955+0000 7f7c8517c740 1 Processor -- start 2024-07-31T15:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.956+0000 7f7c8517c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fb52f0 0x5573373947e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.956+0000 7f7c8517c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 0x557337394d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.956+0000 7f7c8517c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x5573373917b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.956+0000 7f7c8517c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c011410 con 0x557336fb52f0 2024-07-31T15:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.956+0000 7f7c777fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 0x557337394d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.956+0000 7f7c777fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 0x557337394d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50008/0 (socket says 172.21.15.67:50008) 2024-07-31T15:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.956+0000 7f7c777fe640 1 -- 172.21.15.67:0/2789277439 learned_addr learned my addr 172.21.15.67:0/2789277439 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.957+0000 7f7c8517c740 1 -- 172.21.15.67:0/2789277439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c010d40 con 0x557336fde1b0 2024-07-31T15:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.957+0000 7f7c7c829640 1 --2- 172.21.15.67:0/2789277439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x5573373917b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.957+0000 7f7c8517c740 1 -- 172.21.15.67:0/2789277439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c00dfa0 con 0x557336fbd4d0 2024-07-31T15:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.957+0000 7f7c8517c740 10 client.0 _ll_register_callbacks cb 0x7f7c6c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:42.961 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T15:12:42.961 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.957+0000 7f7c777fe640 1 -- 172.21.15.67:0/2789277439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 msgr2=0x5573373917b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.957+0000 7f7c777fe640 1 --2- 172.21.15.67:0/2789277439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x5573373917b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.957+0000 7f7c777fe640 1 -- 172.21.15.67:0/2789277439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fb52f0 msgr2=0x5573373947e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.957+0000 7f7c777fe640 1 --2- 172.21.15.67:0/2789277439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fb52f0 0x5573373947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.957+0000 7f7c777fe640 1 -- 172.21.15.67:0/2789277439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557337391d20 con 0x557336fbd4d0 2024-07-31T15:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.957+0000 7f7c7c829640 1 --2- 172.21.15.67:0/2789277439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x5573373917b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.958+0000 7f7c777fe640 1 --2- 172.21.15.67:0/2789277439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 0x557337394d20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0abdb0 tx=0x7f7c6c01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52686bdf4163e83c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.962+0000 7f7c5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:12:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.962+0000 7f7c5b7fe640 1 -- 172.21.15.67:0/2789277439 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c0c5020 con 0x557336fbd4d0 2024-07-31T15:12:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.962+0000 7f7c5b7fe640 1 -- 172.21.15.67:0/2789277439 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f7c6c0a8410 con 0x557336fbd4d0 2024-07-31T15:12:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.963+0000 7f7c8517c740 1 -- 172.21.15.67:0/2789277439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c6804d070 con 0x557336fbd4d0 2024-07-31T15:12:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.963+0000 7f7c5b7fe640 1 -- 172.21.15.67:0/2789277439 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f7c6c0e2df0 con 0x557336fbd4d0 2024-07-31T15:12:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.963+0000 7f7c8517c740 10 client.29389 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.963+0000 7f7c8517c740 1 -- 172.21.15.67:0/2789277439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x557337405bc0 con 0x557336fbd4d0 2024-07-31T15:12:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.964+0000 7f7c5b7fe640 1 -- 172.21.15.67:0/2789277439 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c6804d070 con 0x557336fbd4d0 2024-07-31T15:12:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.965+0000 7f7c5b7fe640 1 -- 172.21.15.67:0/2789277439 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f7c6c0a7790 con 0x557336fbd4d0 2024-07-31T15:12:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.965+0000 7f7c8517c740 10 client.29389 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.965+0000 7f7c8517c740 10 client.29389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:12:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.965+0000 7f7c8517c740 20 client.29389 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.965+0000 7f7c8517c740 1 --2- 172.21.15.67:0/2789277439 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557336f721c0 0x557337408300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.965+0000 7f7c8517c740 4 client.29389 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:12:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.965+0000 7f7c8517c740 1 -- 172.21.15.67:0/2789277439 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5573372f3da0 con 0x557336f721c0 2024-07-31T15:12:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.966+0000 7f7c77fff640 1 --2- 172.21.15.67:0/2789277439 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557336f721c0 0x557337408300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.966+0000 7f7c77fff640 1 --2- 172.21.15.67:0/2789277439 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557336f721c0 0x557337408300 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4596ed355903fe4c server_cookie=f043a85612b67372 in_seq=0 out_seq=0 2024-07-31T15:12:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:42.966+0000 7f7c5b7fe640 10 client.29389 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:43.469 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/dvo_ch7xxx.o 2024-07-31T15:12:43.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:43 smithi067 ceph-mon[32750]: pgmap v1091: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 630 KiB/s wr, 125 op/s 2024-07-31T15:12:43.561 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-samsung.o 2024-07-31T15:12:43.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:43 smithi073 ceph-mon[36714]: pgmap v1091: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 630 KiB/s wr, 125 op/s 2024-07-31T15:12:43.708 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/dvo_ivch.o 2024-07-31T15:12:43.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:43 smithi077 ceph-mon[37327]: pgmap v1091: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 630 KiB/s wr, 125 op/s 2024-07-31T15:12:44.185 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:12:44.549 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-sony.o 2024-07-31T15:12:44.829 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-sunplus.o 2024-07-31T15:12:45.049 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/dvo_ns2501.o 2024-07-31T15:12:45.114 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/dvo_sil164.o 2024-07-31T15:12:45.155 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/dvo_tfp410.o 2024-07-31T15:12:45.424 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/g4x_dp.o 2024-07-31T15:12:45.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:45 smithi073 ceph-mon[36714]: pgmap v1092: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 519 KiB/s wr, 99 op/s 2024-07-31T15:12:45.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:45 smithi077 ceph-mon[37327]: pgmap v1092: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 519 KiB/s wr, 99 op/s 2024-07-31T15:12:45.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:45 smithi067 ceph-mon[32750]: pgmap v1092: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 519 KiB/s wr, 99 op/s 2024-07-31T15:12:45.815 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/hid/hid-topseed.o 2024-07-31T15:12:46.015 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/g4x_hdmi.o 2024-07-31T15:12:46.169 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/icl_dsi.o 2024-07-31T15:12:46.284 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_backlight.o 2024-07-31T15:12:46.355 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_crt.o 2024-07-31T15:12:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.370+0000 7f7c5b7fe640 1 -- 172.21.15.67:0/2789277439 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5573372f3da0 con 0x557336f721c0 2024-07-31T15:12:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.370+0000 7f7c5b7fe640 10 client.29389 handle_command_reply: tid=0 2024-07-31T15:12:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.370+0000 7f7c8517c740 1 client.29389 shutdown 2024-07-31T15:12:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.370+0000 7f7c8517c740 1 -- 172.21.15.67:0/2789277439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 msgr2=0x557337394d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.370+0000 7f7c8517c740 1 --2- 172.21.15.67:0/2789277439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 0x557337394d20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0abdb0 tx=0x7f7c6c01f000 comp rx=0 tx=0).stop 2024-07-31T15:12:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.371+0000 7f7c8517c740 1 -- 172.21.15.67:0/2789277439 shutdown_connections 2024-07-31T15:12:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.371+0000 7f7c8517c740 1 --2- 172.21.15.67:0/2789277439 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557336f721c0 0x557337408300 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.371+0000 7f7c8517c740 1 --2- 172.21.15.67:0/2789277439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557336fde1b0 0x5573373917b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.371+0000 7f7c8517c740 1 --2- 172.21.15.67:0/2789277439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557336fbd4d0 0x557337394d20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.371+0000 7f7c8517c740 1 --2- 172.21.15.67:0/2789277439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557336fb52f0 0x5573373947e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.371+0000 7f7c8517c740 1 -- 172.21.15.67:0/2789277439 >> 172.21.15.67:0/2789277439 conn(0x557336fd2300 msgr2=0x55733738d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.371+0000 7f7c5b7fe640 0 client.29389 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.371+0000 7f7c8517c740 1 -- 172.21.15.67:0/2789277439 shutdown_connections 2024-07-31T15:12:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.371+0000 7f7c8517c740 1 -- 172.21.15.67:0/2789277439 wait complete. 2024-07-31T15:12:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.371+0000 7f7c8517c740 20 client.29389 trim_cache size 0 max 16384 2024-07-31T15:12:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:12:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:12:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:12:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:12:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.379+0000 7f7c7e28c640 1 -- 172.21.15.67:0/2296973446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c600b3f50 msgr2=0x7f7c600b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.379+0000 7f7c7e28c640 1 --2- 172.21.15.67:0/2296973446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c600b3f50 0x7f7c600b6460 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55733737f960 tx=0x7f7c68031400 comp rx=0 tx=0).stop 2024-07-31T15:12:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.379+0000 7f7c7e28c640 1 -- 172.21.15.67:0/2296973446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7810c2c0 msgr2=0x7f7c781a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.379+0000 7f7c7e28c640 1 --2- 172.21.15.67:0/2296973446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7810c2c0 0x7f7c781a1270 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00e9c0 tx=0x7f7c6c00ee80 comp rx=0 tx=0).stop 2024-07-31T15:12:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.380+0000 7f7c7e28c640 1 -- 172.21.15.67:0/2296973446 shutdown_connections 2024-07-31T15:12:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.380+0000 7f7c7e28c640 1 --2- 172.21.15.67:0/2296973446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7c600b3f50 0x7f7c600b6460 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.380+0000 7f7c7e28c640 1 --2- 172.21.15.67:0/2296973446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c78114ea0 0x7f7c781a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.380+0000 7f7c7e28c640 1 --2- 172.21.15.67:0/2296973446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7810c2c0 0x7f7c781a1270 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.380+0000 7f7c7e28c640 1 --2- 172.21.15.67:0/2296973446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c7810b970 0x7f7c7810b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.380+0000 7f7c7e28c640 1 -- 172.21.15.67:0/2296973446 >> 172.21.15.67:0/2296973446 conn(0x7f7c780707c0 msgr2=0x7f7c78072930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.380+0000 7f7c7e28c640 1 -- 172.21.15.67:0/2296973446 shutdown_connections 2024-07-31T15:12:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:46.380+0000 7f7c7e28c640 1 -- 172.21.15.67:0/2296973446 wait complete. 2024-07-31T15:12:46.455 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_edid.o 2024-07-31T15:12:46.564 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_cx0_phy.o 2024-07-31T15:12:46.777 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/hid/built-in.a 2024-07-31T15:12:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.844+0000 7f891e710640 1 -- 172.21.15.67:0/488072580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891810a080 msgr2=0x7f891810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.844+0000 7f891e710640 1 --2- 172.21.15.67:0/488072580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891810a080 0x7f891810a490 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f8910009ca0 tx=0x7f891002a8f0 comp rx=0 tx=0).stop 2024-07-31T15:12:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.844+0000 7f891e710640 1 -- 172.21.15.67:0/488072580 shutdown_connections 2024-07-31T15:12:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.844+0000 7f891e710640 1 --2- 172.21.15.67:0/488072580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f891810ca50 0x7f891807b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.844+0000 7f891e710640 1 --2- 172.21.15.67:0/488072580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891810c100 0x7f891810c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.844+0000 7f891e710640 1 --2- 172.21.15.67:0/488072580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891810a080 0x7f891810a490 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.844+0000 7f891e710640 1 -- 172.21.15.67:0/488072580 >> 172.21.15.67:0/488072580 conn(0x7f891806d380 msgr2=0x7f891806d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.845+0000 7f891e710640 1 -- 172.21.15.67:0/488072580 shutdown_connections 2024-07-31T15:12:46.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.845+0000 7f891e710640 1 -- 172.21.15.67:0/488072580 wait complete. 2024-07-31T15:12:46.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.846+0000 7f891e710640 1 Processor -- start 2024-07-31T15:12:46.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.846+0000 7f891e710640 1 -- start start 2024-07-31T15:12:46.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.846+0000 7f891e710640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891810a080 0x7f8918075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:46.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.846+0000 7f891e710640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891810c100 0x7f8918075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:46.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.846+0000 7f891e710640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f891810ca50 0x7f8918076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.846+0000 7f891e710640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f891807d600 con 0x7f891810a080 2024-07-31T15:12:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.846+0000 7f891e710640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f891807d480 con 0x7f891810ca50 2024-07-31T15:12:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.846+0000 7f891e710640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f891807d780 con 0x7f891810c100 2024-07-31T15:12:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.847+0000 7f8917fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891810a080 0x7f8918075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.847+0000 7f891ccad640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f891810ca50 0x7f8918076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.847+0000 7f8917fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891810a080 0x7f8918075580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34452/0 (socket says 172.21.15.67:34452) 2024-07-31T15:12:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.847+0000 7f8917fff640 1 -- 172.21.15.67:0/943590942 learned_addr learned my addr 172.21.15.67:0/943590942 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.847+0000 7f8917fff640 1 -- 172.21.15.67:0/943590942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891810c100 msgr2=0x7f8918075ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:12:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.847+0000 7f8917fff640 1 --2- 172.21.15.67:0/943590942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891810c100 0x7f8918075ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.847+0000 7f8917fff640 1 -- 172.21.15.67:0/943590942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f891810ca50 msgr2=0x7f8918076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.847+0000 7f8917fff640 1 --2- 172.21.15.67:0/943590942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f891810ca50 0x7f8918076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.847+0000 7f8917fff640 1 -- 172.21.15.67:0/943590942 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8918138670 con 0x7f891810a080 2024-07-31T15:12:46.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.848+0000 7f8917fff640 1 --2- 172.21.15.67:0/943590942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891810a080 0x7f8918075580 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f8910030e70 tx=0x7f8910030ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:46.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.849+0000 7f89157fa640 1 -- 172.21.15.67:0/943590942 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89100319d0 con 0x7f891810a080 2024-07-31T15:12:46.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.849+0000 7f891e710640 1 -- 172.21.15.67:0/943590942 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8918138900 con 0x7f891810a080 2024-07-31T15:12:46.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.849+0000 7f891e710640 1 -- 172.21.15.67:0/943590942 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8918138e40 con 0x7f891810a080 2024-07-31T15:12:46.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.851+0000 7f89157fa640 1 -- 172.21.15.67:0/943590942 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f891002d070 con 0x7f891810a080 2024-07-31T15:12:46.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.852+0000 7f89157fa640 1 -- 172.21.15.67:0/943590942 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891003ba00 con 0x7f891810a080 2024-07-31T15:12:46.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.855+0000 7f891e710640 1 -- 172.21.15.67:0/943590942 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f891810a490 con 0x7f891810a080 2024-07-31T15:12:46.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.866+0000 7f89157fa640 1 -- 172.21.15.67:0/943590942 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f891003bba0 con 0x7f891810a080 2024-07-31T15:12:46.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.867+0000 7f89157fa640 1 --2- 172.21.15.67:0/943590942 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f89000b3db0 0x7f89000b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:46.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.867+0000 7f89157fa640 1 -- 172.21.15.67:0/943590942 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f8910105c70 con 0x7f891810a080 2024-07-31T15:12:46.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.868+0000 7f89157fa640 1 -- 172.21.15.67:0/943590942 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f891004b060 con 0x7f891810a080 2024-07-31T15:12:46.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.868+0000 7f89177fe640 1 --2- 172.21.15.67:0/943590942 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f89000b3db0 0x7f89000b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:46.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:46.869+0000 7f89177fe640 1 --2- 172.21.15.67:0/943590942 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f89000b3db0 0x7f89000b62c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8918110120 tx=0x7f890c00b370 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:47.038 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/nvmem/layouts/built-in.a 2024-07-31T15:12:47.112 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/nvmem/core.o 2024-07-31T15:12:47.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:47.350+0000 7f891e710640 1 -- 172.21.15.67:0/943590942 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8918076880 con 0x7f891810a080 2024-07-31T15:12:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:47.355+0000 7f89157fa640 1 -- 172.21.15.67:0/943590942 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11959 (secure 0 0 0) 0x7f89100cccc0 con 0x7f891810a080 2024-07-31T15:12:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:12:47.391 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":66,"btime":"2024-07-31T15:10:30:235518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:10:30.235494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29129":{"gid":29129,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/631186801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":631186801},{"type":"v1","addr":"172.21.15.73:6835","nonce":631186801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:12:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 66 2024-07-31T15:12:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:47.369+0000 7f891e710640 1 -- 172.21.15.67:0/943590942 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f89000b3db0 msgr2=0x7f89000b62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:47.369+0000 7f891e710640 1 --2- 172.21.15.67:0/943590942 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f89000b3db0 0x7f89000b62c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8918110120 tx=0x7f890c00b370 comp rx=0 tx=0).stop 2024-07-31T15:12:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:47.369+0000 7f891e710640 1 -- 172.21.15.67:0/943590942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891810a080 msgr2=0x7f8918075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:47.369+0000 7f891e710640 1 --2- 172.21.15.67:0/943590942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891810a080 0x7f8918075580 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f8910030e70 tx=0x7f8910030ea0 comp rx=0 tx=0).stop 2024-07-31T15:12:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:47.369+0000 7f891e710640 1 -- 172.21.15.67:0/943590942 shutdown_connections 2024-07-31T15:12:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:47.369+0000 7f891e710640 1 --2- 172.21.15.67:0/943590942 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f89000b3db0 0x7f89000b62c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:47.369+0000 7f891e710640 1 --2- 172.21.15.67:0/943590942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f891810ca50 0x7f8918076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:47.369+0000 7f891e710640 1 --2- 172.21.15.67:0/943590942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891810c100 0x7f8918075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:47.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:47.369+0000 7f891e710640 1 --2- 172.21.15.67:0/943590942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891810a080 0x7f8918075580 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:47.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:47.369+0000 7f891e710640 1 -- 172.21.15.67:0/943590942 >> 172.21.15.67:0/943590942 conn(0x7f891806d380 msgr2=0x7f891810e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:47.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:47.369+0000 7f891e710640 1 -- 172.21.15.67:0/943590942 shutdown_connections 2024-07-31T15:12:47.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:47.369+0000 7f891e710640 1 -- 172.21.15.67:0/943590942 wait complete. 2024-07-31T15:12:47.570 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_ddi.o 2024-07-31T15:12:47.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:47 smithi073 ceph-mon[36714]: pgmap v1093: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 548 KiB/s wr, 80 op/s 2024-07-31T15:12:47.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:47 smithi077 ceph-mon[37327]: pgmap v1093: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 548 KiB/s wr, 80 op/s 2024-07-31T15:12:47.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:47 smithi067 ceph-mon[32750]: pgmap v1093: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 548 KiB/s wr, 80 op/s 2024-07-31T15:12:47.981 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_encoder.o 2024-07-31T15:12:48.021 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_ddi_buf_trans.o 2024-07-31T15:12:48.350 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_file.o 2024-07-31T15:12:48.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:48 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/943590942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:12:48.608 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T15:12:48.621 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/nvmem/built-in.a 2024-07-31T15:12:48.639 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_fourcc.o 2024-07-31T15:12:48.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:48 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/943590942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:12:48.663 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_display_device.o 2024-07-31T15:12:48.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:48 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/943590942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:12:48.815 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_display_trace.o 2024-07-31T15:12:48.841 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_framebuffer.o 2024-07-31T15:12:49.194 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_gem.o 2024-07-31T15:12:49.285 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:12:49.370 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_ioctl.o 2024-07-31T15:12:49.408 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_lease.o 2024-07-31T15:12:49.525 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dkl_phy.o 2024-07-31T15:12:49.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:49 smithi073 ceph-mon[36714]: pgmap v1094: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.2 MiB/s wr, 212 op/s 2024-07-31T15:12:49.681 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:49 smithi067 ceph-mon[32750]: pgmap v1094: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.2 MiB/s wr, 212 op/s 2024-07-31T15:12:49.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:49 smithi077 ceph-mon[37327]: pgmap v1094: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.2 MiB/s wr, 212 op/s 2024-07-31T15:12:49.754 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dp.o 2024-07-31T15:12:49.864 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_managed.o 2024-07-31T15:12:50.154 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_mm.o 2024-07-31T15:12:50.206 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_mode_config.o 2024-07-31T15:12:50.315 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_mode_object.o 2024-07-31T15:12:50.406 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dp_aux.o 2024-07-31T15:12:50.506 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_modes.o 2024-07-31T15:12:50.925 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dp_aux_backlight.o 2024-07-31T15:12:50.971 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_modeset_lock.o 2024-07-31T15:12:50.992 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_plane.o 2024-07-31T15:12:51.080 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_prime.o 2024-07-31T15:12:51.084 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dp_hdcp.o 2024-07-31T15:12:51.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:51 smithi073 ceph-mon[36714]: pgmap v1095: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.4 MiB/s wr, 215 op/s 2024-07-31T15:12:51.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.645+0000 7f87be1c6640 1 -- 172.21.15.67:0/787523928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87b8101c60 msgr2=0x7f87b8102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:51.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.645+0000 7f87be1c6640 1 --2- 172.21.15.67:0/787523928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87b8101c60 0x7f87b8102070 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f87a400dbf0 tx=0x7f87a4029ed0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.647+0000 7f87be1c6640 1 -- 172.21.15.67:0/787523928 shutdown_connections 2024-07-31T15:12:51.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.647+0000 7f87be1c6640 1 --2- 172.21.15.67:0/787523928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87b8107ea0 0x7f87b810e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.647+0000 7f87be1c6640 1 --2- 172.21.15.67:0/787523928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b8107550 0x7f87b8107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.647+0000 7f87be1c6640 1 --2- 172.21.15.67:0/787523928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87b8101c60 0x7f87b8102070 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.647+0000 7f87be1c6640 1 -- 172.21.15.67:0/787523928 >> 172.21.15.67:0/787523928 conn(0x7f87b8077690 msgr2=0x7f87b8077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:51.649 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:51 smithi067 ceph-mon[32750]: pgmap v1095: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.4 MiB/s wr, 215 op/s 2024-07-31T15:12:51.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.650+0000 7f87be1c6640 1 -- 172.21.15.67:0/787523928 shutdown_connections 2024-07-31T15:12:51.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.650+0000 7f87be1c6640 1 -- 172.21.15.67:0/787523928 wait complete. 2024-07-31T15:12:51.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.651+0000 7f87be1c6640 1 Processor -- start 2024-07-31T15:12:51.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.651+0000 7f87be1c6640 1 -- start start 2024-07-31T15:12:51.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.652+0000 7f87be1c6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b8101c60 0x7f87b81a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.652+0000 7f87be1c6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87b8107550 0x7f87b81a9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.652+0000 7f87be1c6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87b8107ea0 0x7f87b81ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.652+0000 7f87be1c6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81112e0 con 0x7f87b8101c60 2024-07-31T15:12:51.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.652+0000 7f87be1c6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8111160 con 0x7f87b8107ea0 2024-07-31T15:12:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.652+0000 7f87be1c6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8111460 con 0x7f87b8107550 2024-07-31T15:12:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.652+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b8101c60 0x7f87b81a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.652+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b8101c60 0x7f87b81a8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37660/0 (socket says 172.21.15.67:37660) 2024-07-31T15:12:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.652+0000 7f87b77fe640 1 -- 172.21.15.67:0/968527123 learned_addr learned my addr 172.21.15.67:0/968527123 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.652+0000 7f87b7fff640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87b8107ea0 0x7f87b81ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.652+0000 7f87b6ffd640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87b8107550 0x7f87b81a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.653+0000 7f87b7fff640 1 -- 172.21.15.67:0/968527123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87b8107550 msgr2=0x7f87b81a9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.653+0000 7f87b7fff640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87b8107550 0x7f87b81a9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.653+0000 7f87b7fff640 1 -- 172.21.15.67:0/968527123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b8101c60 msgr2=0x7f87b81a8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.653+0000 7f87b7fff640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b8101c60 0x7f87b81a8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.653+0000 7f87b7fff640 1 -- 172.21.15.67:0/968527123 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b81aebf0 con 0x7f87b8107ea0 2024-07-31T15:12:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.653+0000 7f87b7fff640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87b8107ea0 0x7f87b81ae4c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f87b00042c0 tx=0x7f87b00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.656+0000 7f87b77fe640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b8101c60 0x7f87b81a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:12:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.656+0000 7f87b4ff9640 1 -- 172.21.15.67:0/968527123 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b000dc40 con 0x7f87b8107ea0 2024-07-31T15:12:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.656+0000 7f87be1c6640 1 -- 172.21.15.67:0/968527123 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b81aeee0 con 0x7f87b8107ea0 2024-07-31T15:12:51.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.656+0000 7f87be1c6640 1 -- 172.21.15.67:0/968527123 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b8197980 con 0x7f87b8107ea0 2024-07-31T15:12:51.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.662+0000 7f87c4717740 1 Processor -- start 2024-07-31T15:12:51.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.663+0000 7f87c4717740 1 -- start start 2024-07-31T15:12:51.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.663+0000 7f87b4ff9640 1 -- 172.21.15.67:0/968527123 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b000dde0 con 0x7f87b8107ea0 2024-07-31T15:12:51.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.663+0000 7f87b4ff9640 1 -- 172.21.15.67:0/968527123 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0005280 con 0x7f87b8107ea0 2024-07-31T15:12:51.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.663+0000 7f87b4ff9640 1 -- 172.21.15.67:0/968527123 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f87b00054e0 con 0x7f87b8107ea0 2024-07-31T15:12:51.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.664+0000 7f87c4717740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d54070 0x55c1d30f9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.664+0000 7f87c4717740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d2b1b0 0x55c1d30f9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.664+0000 7f87c4717740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d30fa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.664+0000 7f87c4717740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c1d306fbb0 con 0x55c1d2d33390 2024-07-31T15:12:51.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.664+0000 7f87c4717740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c1d305f0b0 con 0x55c1d2d2b1b0 2024-07-31T15:12:51.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.664+0000 7f87c4717740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c1d30630f0 con 0x55c1d2d54070 2024-07-31T15:12:51.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.665+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d2b1b0 0x55c1d30f9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:51.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.665+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d2b1b0 0x55c1d30f9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56782/0 (socket says 172.21.15.67:56782) 2024-07-31T15:12:51.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.665+0000 7f87b77fe640 1 -- 172.21.15.67:0/1990612885 learned_addr learned my addr 172.21.15.67:0/1990612885 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:51.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.665+0000 7f87b6ffd640 1 --2- 172.21.15.67:0/1990612885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d30fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:51.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.666+0000 7f87b77fe640 1 -- 172.21.15.67:0/1990612885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d54070 msgr2=0x55c1d30f9690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:12:51.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.666+0000 7f87b77fe640 1 --2- 172.21.15.67:0/1990612885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d54070 0x55c1d30f9690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.666+0000 7f87b77fe640 1 -- 172.21.15.67:0/1990612885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 msgr2=0x55c1d30fa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:51.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.666+0000 7f87b77fe640 1 --2- 172.21.15.67:0/1990612885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d30fa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.666+0000 7f87b77fe640 1 -- 172.21.15.67:0/1990612885 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d3005b90 con 0x55c1d2d2b1b0 2024-07-31T15:12:51.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.666+0000 7f87b6ffd640 1 --2- 172.21.15.67:0/1990612885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d30fa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:12:51.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.666+0000 7f87b77fe640 1 --2- 172.21.15.67:0/1990612885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d2b1b0 0x55c1d30f9bd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f87a4032620 tx=0x7f87a40323f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0e5b625ce9f2db1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:51.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.667+0000 7f87adffb640 1 -- 172.21.15.67:0/1990612885 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a400d7a0 con 0x55c1d2d2b1b0 2024-07-31T15:12:51.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.667+0000 7f87adffb640 1 -- 172.21.15.67:0/1990612885 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a4005040 con 0x55c1d2d2b1b0 2024-07-31T15:12:51.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.667+0000 7f87adffb640 1 -- 172.21.15.67:0/1990612885 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a402ab90 con 0x55c1d2d2b1b0 2024-07-31T15:12:51.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.668+0000 7f87c4717740 1 -- 172.21.15.67:0/1990612885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d2b1b0 msgr2=0x55c1d30f9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:51.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.668+0000 7f87c4717740 1 --2- 172.21.15.67:0/1990612885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d2b1b0 0x55c1d30f9bd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f87a4032620 tx=0x7f87a40323f0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.671+0000 7f87b4ff9640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f879c0b3f70 0x7f879c0b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.671+0000 7f87b6ffd640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f879c0b3f70 0x7f879c0b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:51.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.671+0000 7f87b4ff9640 1 -- 172.21.15.67:0/968527123 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f87b00e4a60 con 0x7f87b8107ea0 2024-07-31T15:12:51.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.671+0000 7f87c4717740 1 -- 172.21.15.67:0/1990612885 shutdown_connections 2024-07-31T15:12:51.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.671+0000 7f87c4717740 1 --2- 172.21.15.67:0/1990612885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d30fa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.671+0000 7f87c4717740 1 --2- 172.21.15.67:0/1990612885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d2b1b0 0x55c1d30f9bd0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.671+0000 7f87c4717740 1 --2- 172.21.15.67:0/1990612885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d54070 0x55c1d30f9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.671+0000 7f87c4717740 1 -- 172.21.15.67:0/1990612885 >> 172.21.15.67:0/1990612885 conn(0x55c1d2d48460 msgr2=0x55c1d30f7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:51.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.672+0000 7f87b6ffd640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f879c0b3f70 0x7f879c0b6480 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f87b8107300 tx=0x7f87a800b510 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:51.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.673+0000 7f87c4717740 1 -- 172.21.15.67:0/1990612885 shutdown_connections 2024-07-31T15:12:51.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.673+0000 7f87c4717740 1 -- 172.21.15.67:0/1990612885 wait complete. 2024-07-31T15:12:51.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.673+0000 7f87c4717740 1 -- start start 2024-07-31T15:12:51.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.674+0000 7f87c4717740 1 Processor -- start 2024-07-31T15:12:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.674+0000 7f87c4717740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d2b1b0 0x55c1d31052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.674+0000 7f87c4717740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d3105800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.674+0000 7f87c4717740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d54070 0x55c1d3111190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.674+0000 7f87c4717740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c1d2ec2700 con 0x55c1d2d33390 2024-07-31T15:12:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.674+0000 7f87c4717740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c1d30019e0 con 0x55c1d2d2b1b0 2024-07-31T15:12:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.674+0000 7f87c4717740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c1d3107e10 con 0x55c1d2d54070 2024-07-31T15:12:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.675+0000 7f87c4717740 10 client.0 _ll_register_callbacks cb 0x55c1d301c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.675+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d2b1b0 0x55c1d31052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.675+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d3105800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:51.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.676+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d3105800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37690/0 (socket says 172.21.15.67:37690) 2024-07-31T15:12:51.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.676+0000 7f87b6ffd640 1 -- 172.21.15.67:0/85420003 learned_addr learned my addr 172.21.15.67:0/85420003 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:51.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.676+0000 7f87b7fff640 1 --2- 172.21.15.67:0/85420003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d54070 0x55c1d3111190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.676+0000 7f87b6ffd640 1 -- 172.21.15.67:0/85420003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d54070 msgr2=0x55c1d3111190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.676+0000 7f87b6ffd640 1 --2- 172.21.15.67:0/85420003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d54070 0x55c1d3111190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.676+0000 7f87b6ffd640 1 -- 172.21.15.67:0/85420003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d2b1b0 msgr2=0x55c1d31052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.676+0000 7f87b6ffd640 1 --2- 172.21.15.67:0/85420003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d2b1b0 0x55c1d31052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.676+0000 7f87b6ffd640 1 -- 172.21.15.67:0/85420003 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d2dbf650 con 0x55c1d2d33390 2024-07-31T15:12:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.677+0000 7f87b6ffd640 1 --2- 172.21.15.67:0/85420003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d3105800 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f87a800bbd0 tx=0x7f87a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa3aa3bd36f080b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:51.680 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_print.o 2024-07-31T15:12:51.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.680+0000 7f879affd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:12:51.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.680+0000 7f879affd640 1 -- 172.21.15.67:0/85420003 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a801c110 con 0x55c1d2d33390 2024-07-31T15:12:51.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.680+0000 7f879affd640 1 -- 172.21.15.67:0/85420003 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f87a8004030 con 0x55c1d2d33390 2024-07-31T15:12:51.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.680+0000 7f87c4717740 1 -- 172.21.15.67:0/85420003 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87a4005040 con 0x55c1d2d33390 2024-07-31T15:12:51.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.681+0000 7f879affd640 1 -- 172.21.15.67:0/85420003 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f87a8004230 con 0x55c1d2d33390 2024-07-31T15:12:51.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.682+0000 7f87c4717740 10 client.20019 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:51.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.682+0000 7f87c4717740 1 -- 172.21.15.67:0/85420003 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55c1d3113f20 con 0x55c1d2d33390 2024-07-31T15:12:51.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.683+0000 7f879affd640 1 -- 172.21.15.67:0/85420003 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a4005040 con 0x55c1d2d33390 2024-07-31T15:12:51.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.684+0000 7f879affd640 1 -- 172.21.15.67:0/85420003 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f87a800b040 con 0x55c1d2d33390 2024-07-31T15:12:51.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.684+0000 7f87c4717740 10 client.20019 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:51.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.684+0000 7f87c4717740 10 client.20019 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:12:51.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.684+0000 7f87c4717740 20 client.20019 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:51.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.684+0000 7f87c4717740 1 --2- 172.21.15.67:0/85420003 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55c1d2d129f0 0x55c1d31174a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.684+0000 7f87c4717740 4 client.20019 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:12:51.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.684+0000 7f87c4717740 1 -- 172.21.15.67:0/85420003 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1d3071b00 con 0x55c1d2d129f0 2024-07-31T15:12:51.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.685+0000 7f87b77fe640 1 --2- 172.21.15.67:0/85420003 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55c1d2d129f0 0x55c1d31174a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:51.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.686+0000 7f87b77fe640 1 --2- 172.21.15.67:0/85420003 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55c1d2d129f0 0x55c1d31174a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=354a3d15e4b8a452 server_cookie=69e2cd51aaa1c2ea in_seq=0 out_seq=0 2024-07-31T15:12:51.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.686+0000 7f879affd640 10 client.20019 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:12:51.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.687+0000 7f879affd640 1 -- 172.21.15.67:0/85420003 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1d3071b00 con 0x55c1d2d129f0 2024-07-31T15:12:51.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.687+0000 7f879affd640 10 client.20019 handle_command_reply: tid=0 2024-07-31T15:12:51.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.687+0000 7f87c4717740 1 client.20019 shutdown 2024-07-31T15:12:51.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.688+0000 7f87c4717740 1 -- 172.21.15.67:0/85420003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 msgr2=0x55c1d3105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:51.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.688+0000 7f87c4717740 1 --2- 172.21.15.67:0/85420003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d3105800 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f87a800bbd0 tx=0x7f87a80046c0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.688+0000 7f87c4717740 1 -- 172.21.15.67:0/85420003 shutdown_connections 2024-07-31T15:12:51.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.688+0000 7f87c4717740 1 --2- 172.21.15.67:0/85420003 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55c1d2d129f0 0x55c1d31174a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.688+0000 7f87c4717740 1 --2- 172.21.15.67:0/85420003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d54070 0x55c1d3111190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.688+0000 7f87c4717740 1 --2- 172.21.15.67:0/85420003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d3105800 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.688+0000 7f87c4717740 1 --2- 172.21.15.67:0/85420003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d2b1b0 0x55c1d31052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.688+0000 7f87c4717740 1 -- 172.21.15.67:0/85420003 >> 172.21.15.67:0/85420003 conn(0x55c1d2d48460 msgr2=0x55c1d30f5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:51.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.689+0000 7f879affd640 0 client.20019 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:12:51.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.689+0000 7f87c4717740 1 -- 172.21.15.67:0/85420003 shutdown_connections 2024-07-31T15:12:51.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.689+0000 7f87c4717740 1 -- 172.21.15.67:0/85420003 wait complete. 2024-07-31T15:12:51.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.689+0000 7f87c4717740 20 client.20019 trim_cache size 0 max 16384 2024-07-31T15:12:51.699 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dp_link_training.o 2024-07-31T15:12:51.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.719+0000 7f87c4717740 1 Processor -- start 2024-07-31T15:12:51.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.719+0000 7f87c4717740 1 -- start start 2024-07-31T15:12:51.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.720+0000 7f87c4717740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d2b1b0 0x55c1d30f6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.720+0000 7f87c4717740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d30fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.720+0000 7f87c4717740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d54070 0x55c1d30fece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.720+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d30fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.720+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d30fe7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37692/0 (socket says 172.21.15.67:37692) 2024-07-31T15:12:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.721+0000 7f87b6ffd640 1 -- 172.21.15.67:0/141404322 learned_addr learned my addr 172.21.15.67:0/141404322 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.720+0000 7f87c4717740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f87a000ff40 con 0x55c1d2d33390 2024-07-31T15:12:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.721+0000 7f87c4717740 1 -- 172.21.15.67:0/141404322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0011a80 con 0x55c1d2d54070 2024-07-31T15:12:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.721+0000 7f87c4717740 1 -- 172.21.15.67:0/141404322 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0010610 con 0x55c1d2d2b1b0 2024-07-31T15:12:51.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.721+0000 7f87b6ffd640 1 -- 172.21.15.67:0/141404322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d2b1b0 msgr2=0x55c1d30f6410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:12:51.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.721+0000 7f87b6ffd640 1 --2- 172.21.15.67:0/141404322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d2b1b0 0x55c1d30f6410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.721+0000 7f87b6ffd640 1 -- 172.21.15.67:0/141404322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d54070 msgr2=0x55c1d30fece0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:12:51.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.721+0000 7f87b6ffd640 1 --2- 172.21.15.67:0/141404322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d54070 0x55c1d30fece0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.721+0000 7f87b6ffd640 1 -- 172.21.15.67:0/141404322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d2d12c20 con 0x55c1d2d33390 2024-07-31T15:12:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.722+0000 7f87b6ffd640 1 --2- 172.21.15.67:0/141404322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d30fe7a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f87a800c010 tx=0x7f87a800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b996a8a9be713a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:51.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.724+0000 7f87ad7fa640 1 -- 172.21.15.67:0/141404322 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8004730 con 0x55c1d2d33390 2024-07-31T15:12:51.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.724+0000 7f87ad7fa640 1 -- 172.21.15.67:0/141404322 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a8033070 con 0x55c1d2d33390 2024-07-31T15:12:51.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.725+0000 7f87ad7fa640 1 -- 172.21.15.67:0/141404322 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8004c70 con 0x55c1d2d33390 2024-07-31T15:12:51.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.726+0000 7f87c4717740 1 -- 172.21.15.67:0/141404322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 msgr2=0x55c1d30fe7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:51.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.726+0000 7f87c4717740 1 --2- 172.21.15.67:0/141404322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d30fe7a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f87a800c010 tx=0x7f87a800bfa0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.727+0000 7f87c4717740 1 -- 172.21.15.67:0/141404322 shutdown_connections 2024-07-31T15:12:51.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.727+0000 7f87c4717740 1 --2- 172.21.15.67:0/141404322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d54070 0x55c1d30fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.727+0000 7f87c4717740 1 --2- 172.21.15.67:0/141404322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d33390 0x55c1d30fe7a0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.727+0000 7f87c4717740 1 --2- 172.21.15.67:0/141404322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d2b1b0 0x55c1d30f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.727+0000 7f87c4717740 1 -- 172.21.15.67:0/141404322 >> 172.21.15.67:0/141404322 conn(0x55c1d2d48460 msgr2=0x55c1d30f5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:51.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.727+0000 7f87c4717740 1 -- 172.21.15.67:0/141404322 shutdown_connections 2024-07-31T15:12:51.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.728+0000 7f87c4717740 1 -- 172.21.15.67:0/141404322 wait complete. 2024-07-31T15:12:51.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.728+0000 7f87c4717740 1 -- start start 2024-07-31T15:12:51.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.729+0000 7f87c4717740 1 Processor -- start 2024-07-31T15:12:51.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.729+0000 7f87c4717740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d2b1b0 0x55c1d310d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.729+0000 7f87c4717740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d33390 0x55c1d3108bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.730+0000 7f87c4717740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d54070 0x55c1d3109110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.730+0000 7f87c4717740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f87a00113b0 con 0x55c1d2d54070 2024-07-31T15:12:51.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.730+0000 7f87c4717740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0010ce0 con 0x55c1d2d33390 2024-07-31T15:12:51.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.730+0000 7f87c4717740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87a000df40 con 0x55c1d2d2b1b0 2024-07-31T15:12:51.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.730+0000 7f87c4717740 10 client.0 _ll_register_callbacks cb 0x7f87a8005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.730+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d33390 0x55c1d3108bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.730+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d33390 0x55c1d3108bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56816/0 (socket says 172.21.15.67:56816) 2024-07-31T15:12:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.730+0000 7f87b6ffd640 1 -- 172.21.15.67:0/1087815609 learned_addr learned my addr 172.21.15.67:0/1087815609 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.730+0000 7f87b7fff640 1 --2- 172.21.15.67:0/1087815609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d54070 0x55c1d3109110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:51.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.731+0000 7f87b7fff640 1 -- 172.21.15.67:0/1087815609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d2b1b0 msgr2=0x55c1d310d720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:12:51.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.731+0000 7f87b7fff640 1 --2- 172.21.15.67:0/1087815609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d2b1b0 0x55c1d310d720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.731+0000 7f87b7fff640 1 -- 172.21.15.67:0/1087815609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d33390 msgr2=0x55c1d3108bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:51.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.731+0000 7f87b7fff640 1 --2- 172.21.15.67:0/1087815609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d33390 0x55c1d3108bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.731+0000 7f87b7fff640 1 -- 172.21.15.67:0/1087815609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d3109800 con 0x55c1d2d54070 2024-07-31T15:12:51.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.731+0000 7f87b7fff640 1 --2- 172.21.15.67:0/1087815609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d54070 0x55c1d3109110 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f87b00e1e60 tx=0x7f87b0005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d7ab8edb253b75 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:51.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.731+0000 7f879affd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:12:51.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.731+0000 7f879affd640 1 -- 172.21.15.67:0/1087815609 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b00aba20 con 0x55c1d2d54070 2024-07-31T15:12:51.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.732+0000 7f879affd640 1 -- 172.21.15.67:0/1087815609 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f87b00aa790 con 0x55c1d2d54070 2024-07-31T15:12:51.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.732+0000 7f87c4717740 1 -- 172.21.15.67:0/1087815609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87a8033070 con 0x55c1d2d54070 2024-07-31T15:12:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.732+0000 7f879affd640 1 -- 172.21.15.67:0/1087815609 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f87b00c5e60 con 0x55c1d2d54070 2024-07-31T15:12:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.732+0000 7f87c4717740 10 client.20025 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.732+0000 7f87c4717740 1 -- 172.21.15.67:0/1087815609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55c1d317fb10 con 0x55c1d2d54070 2024-07-31T15:12:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.733+0000 7f879affd640 1 -- 172.21.15.67:0/1087815609 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b00ab320 con 0x55c1d2d54070 2024-07-31T15:12:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.733+0000 7f879affd640 1 -- 172.21.15.67:0/1087815609 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f87b00a9ff0 con 0x55c1d2d54070 2024-07-31T15:12:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.733+0000 7f87c4717740 10 client.20025 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.733+0000 7f87c4717740 10 client.20025 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:12:51.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.733+0000 7f87c4717740 20 client.20025 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:51.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.733+0000 7f87c4717740 1 --2- 172.21.15.67:0/1087815609 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55c1d2ce7da0 0x55c1d3182250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:51.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.734+0000 7f87c4717740 4 client.20025 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:12:51.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.734+0000 7f87c4717740 1 -- 172.21.15.67:0/1087815609 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f87a8033070 con 0x55c1d2ce7da0 2024-07-31T15:12:51.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.734+0000 7f87b77fe640 1 --2- 172.21.15.67:0/1087815609 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55c1d2ce7da0 0x55c1d3182250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:51.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:51 smithi077 ceph-mon[37327]: pgmap v1095: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.4 MiB/s wr, 215 op/s 2024-07-31T15:12:51.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.742+0000 7f87b77fe640 1 --2- 172.21.15.67:0/1087815609 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55c1d2ce7da0 0x55c1d3182250 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ae6269eec4b158 server_cookie=3a42876fe28a356f in_seq=0 out_seq=0 2024-07-31T15:12:51.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.742+0000 7f879affd640 10 client.20025 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:12:51.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.744+0000 7f879affd640 1 -- 172.21.15.67:0/1087815609 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f87a8033070 con 0x55c1d2ce7da0 2024-07-31T15:12:51.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.744+0000 7f879affd640 10 client.20025 handle_command_reply: tid=0 2024-07-31T15:12:51.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.744+0000 7f87c4717740 1 client.20025 shutdown 2024-07-31T15:12:51.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.746+0000 7f87c4717740 1 -- 172.21.15.67:0/1087815609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d54070 msgr2=0x55c1d3109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:51.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.746+0000 7f87c4717740 1 --2- 172.21.15.67:0/1087815609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d54070 0x55c1d3109110 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f87b00e1e60 tx=0x7f87b0005b10 comp rx=0 tx=0).stop 2024-07-31T15:12:51.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.746+0000 7f87c4717740 1 -- 172.21.15.67:0/1087815609 shutdown_connections 2024-07-31T15:12:51.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.746+0000 7f87c4717740 1 --2- 172.21.15.67:0/1087815609 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x55c1d2ce7da0 0x55c1d3182250 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.746+0000 7f87c4717740 1 --2- 172.21.15.67:0/1087815609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1d2d54070 0x55c1d3109110 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.746+0000 7f879affd640 0 client.20025 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:12:51.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.746+0000 7f87c4717740 1 --2- 172.21.15.67:0/1087815609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d2d33390 0x55c1d3108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.746+0000 7f87c4717740 1 --2- 172.21.15.67:0/1087815609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1d2d2b1b0 0x55c1d310d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.746+0000 7f87c4717740 1 -- 172.21.15.67:0/1087815609 >> 172.21.15.67:0/1087815609 conn(0x55c1d2d48460 msgr2=0x55c1d30fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:51.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.747+0000 7f87c4717740 1 -- 172.21.15.67:0/1087815609 shutdown_connections 2024-07-31T15:12:51.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.747+0000 7f87c4717740 1 -- 172.21.15.67:0/1087815609 wait complete. 2024-07-31T15:12:51.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.747+0000 7f87c4717740 20 client.20025 trim_cache size 0 max 16384 2024-07-31T15:12:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:12:51.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.750+0000 7f87be1c6640 1 -- 172.21.15.67:0/968527123 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f879c0b3f70 msgr2=0x7f879c0b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:51.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.750+0000 7f87be1c6640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f879c0b3f70 0x7f879c0b6480 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f87b8107300 tx=0x7f87a800b510 comp rx=0 tx=0).stop 2024-07-31T15:12:51.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.750+0000 7f87be1c6640 1 -- 172.21.15.67:0/968527123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87b8107ea0 msgr2=0x7f87b81ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:51.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.750+0000 7f87be1c6640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87b8107ea0 0x7f87b81ae4c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f87b00042c0 tx=0x7f87b00046c0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.753 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_property.o 2024-07-31T15:12:51.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.753+0000 7f87be1c6640 1 -- 172.21.15.67:0/968527123 shutdown_connections 2024-07-31T15:12:51.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.753+0000 7f87be1c6640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f879c0b3f70 0x7f879c0b6480 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.753+0000 7f87be1c6640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87b8107ea0 0x7f87b81ae4c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.753+0000 7f87be1c6640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87b8107550 0x7f87b81a9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.753+0000 7f87be1c6640 1 --2- 172.21.15.67:0/968527123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87b8101c60 0x7f87b81a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:51.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.753+0000 7f87be1c6640 1 -- 172.21.15.67:0/968527123 >> 172.21.15.67:0/968527123 conn(0x7f87b8077690 msgr2=0x7f87b81055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:51.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.754+0000 7f87be1c6640 1 -- 172.21.15.67:0/968527123 shutdown_connections 2024-07-31T15:12:51.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:51.754+0000 7f87be1c6640 1 -- 172.21.15.67:0/968527123 wait complete. 2024-07-31T15:12:51.906 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_syncobj.o 2024-07-31T15:12:51.964 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dp_mst.o 2024-07-31T15:12:52.075 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_sysfs.o 2024-07-31T15:12:52.270 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_trace_points.o 2024-07-31T15:12:52.308 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dsi.o 2024-07-31T15:12:52.559 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dsi_dcs_backlight.o 2024-07-31T15:12:52.563 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_vblank.o 2024-07-31T15:12:52.693 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T15:12:52.912 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_vblank_work.o 2024-07-31T15:12:53.009 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_vma_manager.o 2024-07-31T15:12:53.048 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_writeback.o 2024-07-31T15:12:53.213 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_ioc32.o 2024-07-31T15:12:53.264 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dsi_vbt.o 2024-07-31T15:12:53.339 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_dvo.o 2024-07-31T15:12:53.417 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:12:53.538 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_gmbus.o 2024-07-31T15:12:53.612 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_panel.o 2024-07-31T15:12:53.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:53 smithi073 ceph-mon[36714]: pgmap v1096: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.4 MiB/s wr, 235 op/s 2024-07-31T15:12:53.648 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_pci.o 2024-07-31T15:12:53.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:53 smithi067 ceph-mon[32750]: pgmap v1096: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.4 MiB/s wr, 235 op/s 2024-07-31T15:12:53.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:53 smithi077 ceph-mon[37327]: pgmap v1096: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.4 MiB/s wr, 235 op/s 2024-07-31T15:12:53.750 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_debugfs.o 2024-07-31T15:12:53.773 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_debugfs_crc.o 2024-07-31T15:12:53.904 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_panel_orientation_quirks.o 2024-07-31T15:12:54.256 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_buddy.o 2024-07-31T15:12:54.305 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_gem_shmem_helper.o 2024-07-31T15:12:54.491 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_atomic_helper.o 2024-07-31T15:12:54.503 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_atomic_state_helper.o 2024-07-31T15:12:54.508 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_bridge_connector.o 2024-07-31T15:12:54.514 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_crtc_helper.o 2024-07-31T15:12:54.618 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_hdmi.o 2024-07-31T15:12:54.941 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_lspcon.o 2024-07-31T15:12:55.050 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_damage_helper.o 2024-07-31T15:12:55.144 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_encoder_slave.o 2024-07-31T15:12:55.147 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_flip_work.o 2024-07-31T15:12:55.260 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_format_helper.o 2024-07-31T15:12:55.378 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_gem_atomic_helper.o 2024-07-31T15:12:55.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:55 smithi067 ceph-mon[32750]: pgmap v1097: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.5 MiB/s wr, 245 op/s 2024-07-31T15:12:55.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:55 smithi073 ceph-mon[36714]: pgmap v1097: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.5 MiB/s wr, 245 op/s 2024-07-31T15:12:55.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:55 smithi077 ceph-mon[37327]: pgmap v1097: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.5 MiB/s wr, 245 op/s 2024-07-31T15:12:55.765 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_gem_framebuffer_helper.o 2024-07-31T15:12:55.783 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_kms_helper_common.o 2024-07-31T15:12:55.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.871+0000 7f15119b4640 1 -- 172.21.15.67:0/2870593387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f150c11c5d0 msgr2=0x7f150c11e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:55.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.871+0000 7f15119b4640 1 --2- 172.21.15.67:0/2870593387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f150c11c5d0 0x7f150c11e9f0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f1504009900 tx=0x7f150402a4d0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.873+0000 7f15119b4640 1 -- 172.21.15.67:0/2870593387 shutdown_connections 2024-07-31T15:12:55.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.873+0000 7f15119b4640 1 --2- 172.21.15.67:0/2870593387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f150c11c5d0 0x7f150c11e9f0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.873+0000 7f15119b4640 1 --2- 172.21.15.67:0/2870593387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150c10a330 0x7f150c10a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.873+0000 7f15119b4640 1 --2- 172.21.15.67:0/2870593387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f150c111790 0x7f150c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.873+0000 7f15119b4640 1 -- 172.21.15.67:0/2870593387 >> 172.21.15.67:0/2870593387 conn(0x7f150c06e420 msgr2=0x7f150c06e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:55.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.874+0000 7f15119b4640 1 -- 172.21.15.67:0/2870593387 shutdown_connections 2024-07-31T15:12:55.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.874+0000 7f15119b4640 1 -- 172.21.15.67:0/2870593387 wait complete. 2024-07-31T15:12:55.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.875+0000 7f15119b4640 1 Processor -- start 2024-07-31T15:12:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.876+0000 7f15119b4640 1 -- start start 2024-07-31T15:12:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.876+0000 7f15119b4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150c10a330 0x7f150c119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.876+0000 7f15119b4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f150c111790 0x7f150c1132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.876+0000 7f15119b4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f150c11c5d0 0x7f150c113820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.876+0000 7f15119b4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f150c122090 con 0x7f150c11c5d0 2024-07-31T15:12:55.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.876+0000 7f15119b4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f150c121f10 con 0x7f150c111790 2024-07-31T15:12:55.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.876+0000 7f15119b4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f150c122210 con 0x7f150c10a330 2024-07-31T15:12:55.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.878+0000 7f15111b3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f150c11c5d0 0x7f150c113820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.878+0000 7f15111b3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f150c11c5d0 0x7f150c113820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37728/0 (socket says 172.21.15.67:37728) 2024-07-31T15:12:55.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.878+0000 7f15111b3640 1 -- 172.21.15.67:0/3883188904 learned_addr learned my addr 172.21.15.67:0/3883188904 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:55.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.878+0000 7f15109b2640 1 --2- 172.21.15.67:0/3883188904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150c10a330 0x7f150c119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.878+0000 7f15111b3640 1 -- 172.21.15.67:0/3883188904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150c10a330 msgr2=0x7f150c119ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:55.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.878+0000 7f15111b3640 1 --2- 172.21.15.67:0/3883188904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150c10a330 0x7f150c119ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.878+0000 7f15111b3640 1 -- 172.21.15.67:0/3883188904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f150c111790 msgr2=0x7f150c1132c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:12:55.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.878+0000 7f15111b3640 1 --2- 172.21.15.67:0/3883188904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f150c111790 0x7f150c1132c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.878+0000 7f15111b3640 1 -- 172.21.15.67:0/3883188904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c114040 con 0x7f150c11c5d0 2024-07-31T15:12:55.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.879+0000 7f15111b3640 1 --2- 172.21.15.67:0/3883188904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f150c11c5d0 0x7f150c113820 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1504005ab0 tx=0x7f1504031c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:55.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.879+0000 7f1509ffb640 1 -- 172.21.15.67:0/3883188904 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504040030 con 0x7f150c11c5d0 2024-07-31T15:12:55.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.880+0000 7f15119b4640 1 -- 172.21.15.67:0/3883188904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150c118c60 con 0x7f150c11c5d0 2024-07-31T15:12:55.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.880+0000 7f15119b4640 1 -- 172.21.15.67:0/3883188904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150c119140 con 0x7f150c11c5d0 2024-07-31T15:12:55.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.883+0000 7f1509ffb640 1 -- 172.21.15.67:0/3883188904 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f150403c8f0 con 0x7f150c11c5d0 2024-07-31T15:12:55.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.883+0000 7f1509ffb640 1 -- 172.21.15.67:0/3883188904 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15040449f0 con 0x7f150c11c5d0 2024-07-31T15:12:55.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.884+0000 7f1519167740 1 Processor -- start 2024-07-31T15:12:55.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.889+0000 7f1519167740 1 -- start start 2024-07-31T15:12:55.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.890+0000 7f1509ffb640 1 -- 172.21.15.67:0/3883188904 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1504043030 con 0x7f150c11c5d0 2024-07-31T15:12:55.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.890+0000 7f1519167740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561355204070 0x5613555a9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.890+0000 7f1519167740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613551db1b0 0x5613555a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.890+0000 7f1519167740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613551e3390 0x5613555aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.890+0000 7f1519167740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56135551fbb0 con 0x5613551e3390 2024-07-31T15:12:55.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.890+0000 7f1519167740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56135550f0b0 con 0x5613551db1b0 2024-07-31T15:12:55.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.890+0000 7f1519167740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5613555130f0 con 0x561355204070 2024-07-31T15:12:55.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.890+0000 7f15109b2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613551db1b0 0x5613555a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.890+0000 7f15109b2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613551db1b0 0x5613555a9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56832/0 (socket says 172.21.15.67:56832) 2024-07-31T15:12:55.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.891+0000 7f150bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561355204070 0x5613555a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.891+0000 7f150bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561355204070 0x5613555a9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60532/0 (socket says 172.21.15.67:60532) 2024-07-31T15:12:55.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.891+0000 7f150bfff640 1 -- 172.21.15.67:0/444910797 learned_addr learned my addr 172.21.15.67:0/444910797 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:55.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.891+0000 7f150bfff640 1 --2- 172.21.15.67:0/444910797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613551e3390 0x5613555aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.891+0000 7f150bfff640 1 -- 172.21.15.67:0/444910797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613551db1b0 msgr2=0x5613555a9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:55.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.893+0000 7f1509ffb640 1 --2- 172.21.15.67:0/3883188904 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14f80b3f50 0x7f14f80b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.894+0000 7f1509ffb640 1 -- 172.21.15.67:0/3883188904 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f1504104bb0 con 0x7f150c11c5d0 2024-07-31T15:12:55.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.895+0000 7f150bfff640 1 --2- 172.21.15.67:0/444910797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613551db1b0 0x5613555a9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.895+0000 7f150bfff640 1 -- 172.21.15.67:0/444910797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613551e3390 msgr2=0x5613555aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:55.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.895+0000 7f150bfff640 1 --2- 172.21.15.67:0/444910797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613551e3390 0x5613555aa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.895+0000 7f150bfff640 1 -- 172.21.15.67:0/444910797 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613554b5b90 con 0x561355204070 2024-07-31T15:12:55.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.895+0000 7f15109b2640 1 --2- 172.21.15.67:0/444910797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613551db1b0 0x5613555a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:12:55.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.895+0000 7f150bfff640 1 --2- 172.21.15.67:0/444910797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561355204070 0x5613555a9700 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f14fc012a10 tx=0x7f14fc009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db6d1ba042b1d7c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:55.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.894+0000 7f15111b3640 1 --2- 172.21.15.67:0/3883188904 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14f80b3f50 0x7f14f80b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.896+0000 7f15111b3640 1 --2- 172.21.15.67:0/3883188904 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14f80b3f50 0x7f14f80b6460 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f15040c7470 tx=0x7f15040444c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:55.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.898+0000 7f1502ffd640 1 -- 172.21.15.67:0/444910797 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc004d20 con 0x561355204070 2024-07-31T15:12:55.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.898+0000 7f1502ffd640 1 -- 172.21.15.67:0/444910797 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14fc00db90 con 0x561355204070 2024-07-31T15:12:55.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.900+0000 7f1519167740 1 -- 172.21.15.67:0/444910797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561355204070 msgr2=0x5613555a9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:55.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.900+0000 7f1519167740 1 --2- 172.21.15.67:0/444910797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561355204070 0x5613555a9700 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f14fc012a10 tx=0x7f14fc009610 comp rx=0 tx=0).stop 2024-07-31T15:12:55.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.901+0000 7f1519167740 1 -- 172.21.15.67:0/444910797 shutdown_connections 2024-07-31T15:12:55.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.901+0000 7f1519167740 1 --2- 172.21.15.67:0/444910797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613551e3390 0x5613555aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.901+0000 7f1519167740 1 --2- 172.21.15.67:0/444910797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613551db1b0 0x5613555a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.901+0000 7f1519167740 1 --2- 172.21.15.67:0/444910797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561355204070 0x5613555a9700 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.901+0000 7f1519167740 1 -- 172.21.15.67:0/444910797 >> 172.21.15.67:0/444910797 conn(0x5613551f8460 msgr2=0x5613555a75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:55.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.901+0000 7f1519167740 1 -- 172.21.15.67:0/444910797 shutdown_connections 2024-07-31T15:12:55.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.901+0000 7f1519167740 1 -- 172.21.15.67:0/444910797 wait complete. 2024-07-31T15:12:55.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.902+0000 7f1519167740 1 -- start start 2024-07-31T15:12:55.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.902+0000 7f1519167740 1 Processor -- start 2024-07-31T15:12:55.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.903+0000 7f1519167740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613551db1b0 0x5613555b5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.903+0000 7f1519167740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5613551e3390 0x5613555b5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.903+0000 7f1519167740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561355204070 0x5613555c1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.903+0000 7f1519167740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561355372700 con 0x561355204070 2024-07-31T15:12:55.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.903+0000 7f1519167740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5613554b19e0 con 0x5613551db1b0 2024-07-31T15:12:55.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.903+0000 7f1519167740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5613555b7f30 con 0x5613551e3390 2024-07-31T15:12:55.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.904+0000 7f1519167740 10 client.0 _ll_register_callbacks cb 0x5613554cc4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:55.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.907+0000 7f15109b2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5613551e3390 0x5613555b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.907+0000 7f15109b2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5613551e3390 0x5613555b5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:60548/0 (socket says 172.21.15.67:60548) 2024-07-31T15:12:55.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.907+0000 7f15109b2640 1 -- 172.21.15.67:0/3444613977 learned_addr learned my addr 172.21.15.67:0/3444613977 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:55.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.907+0000 7f150bfff640 1 --2- 172.21.15.67:0/3444613977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613551db1b0 0x5613555b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.909+0000 7f15109b2640 1 -- 172.21.15.67:0/3444613977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613551db1b0 msgr2=0x5613555b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:55.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.909+0000 7f15109b2640 1 --2- 172.21.15.67:0/3444613977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613551db1b0 0x5613555b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.909+0000 7f15109b2640 1 -- 172.21.15.67:0/3444613977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561355204070 msgr2=0x5613555c1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:55.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.909+0000 7f15109b2640 1 --2- 172.21.15.67:0/3444613977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561355204070 0x5613555c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.909+0000 7f15109b2640 1 -- 172.21.15.67:0/3444613977 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56135526f650 con 0x5613551e3390 2024-07-31T15:12:55.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.910+0000 7f15109b2640 1 --2- 172.21.15.67:0/3444613977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5613551e3390 0x5613555b5790 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f14f400afd0 tx=0x7f14f400f870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72d367a05f24d3ec server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:55.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.911+0000 7f14ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:12:55.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.912+0000 7f14ebfff640 1 -- 172.21.15.67:0/3444613977 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4005220 con 0x5613551e3390 2024-07-31T15:12:55.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.913+0000 7f1519167740 1 -- 172.21.15.67:0/3444613977 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14fc00db90 con 0x5613551e3390 2024-07-31T15:12:55.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.913+0000 7f14ebfff640 1 -- 172.21.15.67:0/3444613977 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f14f40053c0 con 0x5613551e3390 2024-07-31T15:12:55.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.914+0000 7f14ebfff640 1 -- 172.21.15.67:0/3444613977 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f14f40055c0 con 0x5613551e3390 2024-07-31T15:12:55.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.914+0000 7f1519167740 10 client.29413 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:55.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.914+0000 7f1519167740 1 -- 172.21.15.67:0/3444613977 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5613555c3f00 con 0x5613551e3390 2024-07-31T15:12:55.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.922+0000 7f14ebfff640 1 -- 172.21.15.67:0/3444613977 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14fc00db90 con 0x5613551e3390 2024-07-31T15:12:55.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.922+0000 7f14ebfff640 1 -- 172.21.15.67:0/3444613977 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f14f4011c80 con 0x5613551e3390 2024-07-31T15:12:55.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.923+0000 7f1519167740 10 client.29413 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:55.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.923+0000 7f1519167740 10 client.29413 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:12:55.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.923+0000 7f1519167740 20 client.29413 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:55.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.923+0000 7f1519167740 1 --2- 172.21.15.67:0/3444613977 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5613551c29f0 0x5613555c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.923+0000 7f1519167740 4 client.29413 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:12:55.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.923+0000 7f1519167740 1 -- 172.21.15.67:0/3444613977 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561355521b00 con 0x5613551c29f0 2024-07-31T15:12:55.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.924+0000 7f150bfff640 1 --2- 172.21.15.67:0/3444613977 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5613551c29f0 0x5613555c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.924+0000 7f150bfff640 1 --2- 172.21.15.67:0/3444613977 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5613551c29f0 0x5613555c7840 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b8f604455b645d1 server_cookie=ca891f18b68fe6d2 in_seq=0 out_seq=0 2024-07-31T15:12:55.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.925+0000 7f14ebfff640 10 client.29413 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:12:55.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.925+0000 7f14ebfff640 1 -- 172.21.15.67:0/3444613977 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561355521b00 con 0x5613551c29f0 2024-07-31T15:12:55.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.926+0000 7f14ebfff640 10 client.29413 handle_command_reply: tid=0 2024-07-31T15:12:55.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.927+0000 7f1519167740 1 client.29413 shutdown 2024-07-31T15:12:55.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.927+0000 7f1519167740 1 -- 172.21.15.67:0/3444613977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5613551e3390 msgr2=0x5613555b5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:55.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.927+0000 7f1519167740 1 --2- 172.21.15.67:0/3444613977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5613551e3390 0x5613555b5790 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f14f400afd0 tx=0x7f14f400f870 comp rx=0 tx=0).stop 2024-07-31T15:12:55.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.927+0000 7f1519167740 1 -- 172.21.15.67:0/3444613977 shutdown_connections 2024-07-31T15:12:55.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.927+0000 7f1519167740 1 --2- 172.21.15.67:0/3444613977 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5613551c29f0 0x5613555c7840 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.927+0000 7f1519167740 1 --2- 172.21.15.67:0/3444613977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561355204070 0x5613555c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.927+0000 7f1519167740 1 --2- 172.21.15.67:0/3444613977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5613551e3390 0x5613555b5790 secure :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f14f400afd0 tx=0x7f14f400f870 comp rx=0 tx=0).stop 2024-07-31T15:12:55.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.927+0000 7f1519167740 1 --2- 172.21.15.67:0/3444613977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613551db1b0 0x5613555b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.927+0000 7f1519167740 1 -- 172.21.15.67:0/3444613977 >> 172.21.15.67:0/3444613977 conn(0x5613551f8460 msgr2=0x5613555a5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:55.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.927+0000 7f14ebfff640 0 client.29413 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:12:55.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.928+0000 7f1519167740 1 -- 172.21.15.67:0/3444613977 shutdown_connections 2024-07-31T15:12:55.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.928+0000 7f1519167740 1 -- 172.21.15.67:0/3444613977 wait complete. 2024-07-31T15:12:55.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.928+0000 7f1519167740 20 client.29413 trim_cache size 0 max 16384 2024-07-31T15:12:55.948 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_modeset_helper.o 2024-07-31T15:12:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.958+0000 7f1519167740 1 Processor -- start 2024-07-31T15:12:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.958+0000 7f1519167740 1 -- start start 2024-07-31T15:12:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.959+0000 7f1519167740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613551db1b0 0x5613555a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.959+0000 7f1519167740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561355204070 0x5613555a6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.959+0000 7f1519167740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561355197da0 0x5613555a7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.959+0000 7f1519167740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1c7ce0 con 0x5613551db1b0 2024-07-31T15:12:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.959+0000 7f150bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613551db1b0 0x5613555a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.959+0000 7f150bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613551db1b0 0x5613555a6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37766/0 (socket says 172.21.15.67:37766) 2024-07-31T15:12:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.959+0000 7f150bfff640 1 -- 172.21.15.67:0/1888354740 learned_addr learned my addr 172.21.15.67:0/1888354740 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.959+0000 7f1519167740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1c9820 con 0x561355204070 2024-07-31T15:12:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.960+0000 7f1519167740 1 -- 172.21.15.67:0/1888354740 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1c83b0 con 0x561355197da0 2024-07-31T15:12:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.960+0000 7f15109b2640 1 --2- 172.21.15.67:0/1888354740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561355204070 0x5613555a6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.960+0000 7f150bfff640 1 --2- 172.21.15.67:0/1888354740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561355197da0 0x5613555a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.961+0000 7f150bfff640 1 -- 172.21.15.67:0/1888354740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561355197da0 msgr2=0x5613555a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.961+0000 7f150bfff640 1 --2- 172.21.15.67:0/1888354740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561355197da0 0x5613555a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.961+0000 7f150bfff640 1 -- 172.21.15.67:0/1888354740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561355204070 msgr2=0x5613555a6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.961+0000 7f150bfff640 1 --2- 172.21.15.67:0/1888354740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561355204070 0x5613555a6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.961+0000 7f150bfff640 1 -- 172.21.15.67:0/1888354740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613555b6310 con 0x5613551db1b0 2024-07-31T15:12:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.962+0000 7f150bfff640 1 --2- 172.21.15.67:0/1888354740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613551db1b0 0x5613555a6350 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00a010 tx=0x7f14fc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d950ec9bba7ba87f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.962+0000 7f15027fc640 1 -- 172.21.15.67:0/1888354740 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc005980 con 0x5613551db1b0 2024-07-31T15:12:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.963+0000 7f15027fc640 1 -- 172.21.15.67:0/1888354740 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14fc009d10 con 0x5613551db1b0 2024-07-31T15:12:55.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.963+0000 7f15027fc640 1 -- 172.21.15.67:0/1888354740 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc02b3c0 con 0x5613551db1b0 2024-07-31T15:12:55.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.964+0000 7f1519167740 1 -- 172.21.15.67:0/1888354740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613551db1b0 msgr2=0x5613555a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:55.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.965+0000 7f1519167740 1 --2- 172.21.15.67:0/1888354740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613551db1b0 0x5613555a6350 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00a010 tx=0x7f14fc014040 comp rx=0 tx=0).stop 2024-07-31T15:12:55.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.965+0000 7f1519167740 1 -- 172.21.15.67:0/1888354740 shutdown_connections 2024-07-31T15:12:55.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.965+0000 7f1519167740 1 --2- 172.21.15.67:0/1888354740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561355197da0 0x5613555a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.966+0000 7f1519167740 1 --2- 172.21.15.67:0/1888354740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561355204070 0x5613555a6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.966+0000 7f1519167740 1 --2- 172.21.15.67:0/1888354740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613551db1b0 0x5613555a6350 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.966+0000 7f1519167740 1 -- 172.21.15.67:0/1888354740 >> 172.21.15.67:0/1888354740 conn(0x5613551f8460 msgr2=0x5613555a5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:55.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.966+0000 7f1519167740 1 -- 172.21.15.67:0/1888354740 shutdown_connections 2024-07-31T15:12:55.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.966+0000 7f1519167740 1 -- 172.21.15.67:0/1888354740 wait complete. 2024-07-31T15:12:55.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.967+0000 7f1519167740 1 -- start start 2024-07-31T15:12:55.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.968+0000 7f1519167740 1 Processor -- start 2024-07-31T15:12:55.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.969+0000 7f1519167740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561355197da0 0x5613555b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.969+0000 7f1519167740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5613551db1b0 0x5613555b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.969+0000 7f150bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561355197da0 0x5613555b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.969+0000 7f150bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561355197da0 0x5613555b87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37780/0 (socket says 172.21.15.67:37780) 2024-07-31T15:12:55.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.970+0000 7f150bfff640 1 -- 172.21.15.67:0/386901357 learned_addr learned my addr 172.21.15.67:0/386901357 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:55.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.970+0000 7f1519167740 1 --2- 172.21.15.67:0/386901357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561355204070 0x56135562dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.970+0000 7f1519167740 1 -- 172.21.15.67:0/386901357 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1c9150 con 0x561355197da0 2024-07-31T15:12:55.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.970+0000 7f1519167740 1 -- 172.21.15.67:0/386901357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1c8a80 con 0x561355204070 2024-07-31T15:12:55.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.970+0000 7f1519167740 1 -- 172.21.15.67:0/386901357 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1c5ce0 con 0x5613551db1b0 2024-07-31T15:12:55.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.971+0000 7f15109b2640 1 --2- 172.21.15.67:0/386901357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5613551db1b0 0x5613555b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.971+0000 7f1519167740 10 client.0 _ll_register_callbacks cb 0x7f14f400a370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:55.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.971+0000 7f150bfff640 1 --2- 172.21.15.67:0/386901357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561355204070 0x56135562dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.972+0000 7f150bfff640 1 -- 172.21.15.67:0/386901357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5613551db1b0 msgr2=0x5613555b8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:55.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.972+0000 7f150bfff640 1 --2- 172.21.15.67:0/386901357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5613551db1b0 0x5613555b8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.972+0000 7f150bfff640 1 -- 172.21.15.67:0/386901357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561355204070 msgr2=0x56135562dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:55.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.972+0000 7f150bfff640 1 --2- 172.21.15.67:0/386901357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561355204070 0x56135562dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.972+0000 7f150bfff640 1 -- 172.21.15.67:0/386901357 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56135562e520 con 0x561355197da0 2024-07-31T15:12:55.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.973+0000 7f150bfff640 1 --2- 172.21.15.67:0/386901357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561355197da0 0x5613555b87a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009bb0 tx=0x7f14fc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc28ab72d380ca26 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:55.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.975+0000 7f14ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:12:55.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.975+0000 7f14ebfff640 1 -- 172.21.15.67:0/386901357 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc021070 con 0x561355197da0 2024-07-31T15:12:55.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.975+0000 7f14ebfff640 1 -- 172.21.15.67:0/386901357 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f14fc03f070 con 0x561355197da0 2024-07-31T15:12:55.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.975+0000 7f1519167740 1 -- 172.21.15.67:0/386901357 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14fc009d10 con 0x561355197da0 2024-07-31T15:12:55.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.975+0000 7f14ebfff640 1 -- 172.21.15.67:0/386901357 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f14fc04b050 con 0x561355197da0 2024-07-31T15:12:55.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.976+0000 7f1519167740 10 client.20040 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:55.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.976+0000 7f1519167740 1 -- 172.21.15.67:0/386901357 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56135562f750 con 0x561355197da0 2024-07-31T15:12:55.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.980+0000 7f14ebfff640 1 -- 172.21.15.67:0/386901357 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14fc040040 con 0x561355197da0 2024-07-31T15:12:55.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.982+0000 7f14ebfff640 1 -- 172.21.15.67:0/386901357 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f14fc043020 con 0x561355197da0 2024-07-31T15:12:55.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.982+0000 7f1519167740 10 client.20040 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:55.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.982+0000 7f1519167740 10 client.20040 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:12:55.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.982+0000 7f1519167740 20 client.20040 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:55.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.982+0000 7f1519167740 1 --2- 172.21.15.67:0/386901357 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x56135562fbe0 0x561355632000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:55.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.982+0000 7f1519167740 4 client.20040 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:12:55.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.982+0000 7f1519167740 1 -- 172.21.15.67:0/386901357 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f14fc009d10 con 0x56135562fbe0 2024-07-31T15:12:55.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.983+0000 7f15109b2640 1 --2- 172.21.15.67:0/386901357 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x56135562fbe0 0x561355632000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:55.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.983+0000 7f15109b2640 1 --2- 172.21.15.67:0/386901357 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x56135562fbe0 0x561355632000 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=56602a2d2d62d8da server_cookie=f6432adb8e5c84b0 in_seq=0 out_seq=0 2024-07-31T15:12:55.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.983+0000 7f14ebfff640 10 client.20040 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:12:55.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.984+0000 7f14ebfff640 1 -- 172.21.15.67:0/386901357 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f14fc009d10 con 0x56135562fbe0 2024-07-31T15:12:55.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.984+0000 7f14ebfff640 10 client.20040 handle_command_reply: tid=0 2024-07-31T15:12:55.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.984+0000 7f1519167740 1 client.20040 shutdown 2024-07-31T15:12:55.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.984+0000 7f1519167740 1 -- 172.21.15.67:0/386901357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561355197da0 msgr2=0x5613555b87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:55.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.984+0000 7f1519167740 1 --2- 172.21.15.67:0/386901357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561355197da0 0x5613555b87a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009bb0 tx=0x7f14fc005bd0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.990+0000 7f1519167740 1 -- 172.21.15.67:0/386901357 shutdown_connections 2024-07-31T15:12:55.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.990+0000 7f1519167740 1 --2- 172.21.15.67:0/386901357 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x56135562fbe0 0x561355632000 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.990+0000 7f1519167740 1 --2- 172.21.15.67:0/386901357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561355204070 0x56135562dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.990+0000 7f1519167740 1 --2- 172.21.15.67:0/386901357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5613551db1b0 0x5613555b8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.990+0000 7f1519167740 1 --2- 172.21.15.67:0/386901357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561355197da0 0x5613555b87a0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:55.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.990+0000 7f1519167740 1 -- 172.21.15.67:0/386901357 >> 172.21.15.67:0/386901357 conn(0x5613551f8460 msgr2=0x5613555a6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:55.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.991+0000 7f14ebfff640 0 client.20040 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:12:55.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:55.996+0000 7f1519167740 1 -- 172.21.15.67:0/386901357 shutdown_connections 2024-07-31T15:12:56.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:56.000+0000 7f1519167740 1 -- 172.21.15.67:0/386901357 wait complete. 2024-07-31T15:12:56.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:56.004+0000 7f1519167740 20 client.20040 trim_cache size 0 max 16384 2024-07-31T15:12:56.005 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:12:56.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:56.005+0000 7f15119b4640 1 -- 172.21.15.67:0/3883188904 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14f80b3f50 msgr2=0x7f14f80b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:56.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:56.005+0000 7f15119b4640 1 --2- 172.21.15.67:0/3883188904 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14f80b3f50 0x7f14f80b6460 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f15040c7470 tx=0x7f15040444c0 comp rx=0 tx=0).stop 2024-07-31T15:12:56.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:56.005+0000 7f15119b4640 1 -- 172.21.15.67:0/3883188904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f150c11c5d0 msgr2=0x7f150c113820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:56.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:56.005+0000 7f15119b4640 1 --2- 172.21.15.67:0/3883188904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f150c11c5d0 0x7f150c113820 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1504005ab0 tx=0x7f1504031c60 comp rx=0 tx=0).stop 2024-07-31T15:12:56.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:56.005+0000 7f15119b4640 1 -- 172.21.15.67:0/3883188904 shutdown_connections 2024-07-31T15:12:56.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:56.005+0000 7f15119b4640 1 --2- 172.21.15.67:0/3883188904 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f14f80b3f50 0x7f14f80b6460 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:56.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:56.005+0000 7f15119b4640 1 --2- 172.21.15.67:0/3883188904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f150c11c5d0 0x7f150c113820 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:56.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:56.005+0000 7f15119b4640 1 --2- 172.21.15.67:0/3883188904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f150c111790 0x7f150c1132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:56.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:56.005+0000 7f15119b4640 1 --2- 172.21.15.67:0/3883188904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150c10a330 0x7f150c119ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:56.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:56.005+0000 7f15119b4640 1 -- 172.21.15.67:0/3883188904 >> 172.21.15.67:0/3883188904 conn(0x7f150c06e420 msgr2=0x7f150c071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:56.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:56.006+0000 7f15119b4640 1 -- 172.21.15.67:0/3883188904 shutdown_connections 2024-07-31T15:12:56.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:12:56.006+0000 7f15119b4640 1 -- 172.21.15.67:0/3883188904 wait complete. 2024-07-31T15:12:56.109 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_plane_helper.o 2024-07-31T15:12:56.119 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_probe_helper.o 2024-07-31T15:12:56.137 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_lvds.o 2024-07-31T15:12:56.212 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_rect.o 2024-07-31T15:12:56.376 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_self_refresh_helper.o 2024-07-31T15:12:56.566 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_simple_kms_helper.o 2024-07-31T15:12:56.643 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_panel.o 2024-07-31T15:12:56.692 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/bridge/panel.o 2024-07-31T15:12:56.740 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/drm_mipi_dsi.o 2024-07-31T15:12:56.762 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_pps.o 2024-07-31T15:12:56.969 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T15:12:57.014 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_qp_tables.o 2024-07-31T15:12:57.054 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_sdvo.o 2024-07-31T15:12:57.250 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_snps_phy.o 2024-07-31T15:12:57.337 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.7 2024-07-31T15:12:57.337 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:12:57.358 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_tv.o 2024-07-31T15:12:57.380 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_vdsc.o 2024-07-31T15:12:57.399 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/intel_vrr.o 2024-07-31T15:12:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:12:57.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:57 smithi073 ceph-mon[36714]: pgmap v1098: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-07-31T15:12:57.694 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/vlv_dsi.o 2024-07-31T15:12:57.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:57 smithi077 ceph-mon[37327]: pgmap v1098: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-07-31T15:12:57.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:57 smithi067 ceph-mon[32750]: pgmap v1098: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-07-31T15:12:57.900 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/display/vlv_dsi_pll.o 2024-07-31T15:12:58.634 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_perf.o 2024-07-31T15:12:58.636 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/pxp/intel_pxp.o 2024-07-31T15:12:58.860 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/pxp/intel_pxp_tee.o 2024-07-31T15:12:59.017 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/pxp/intel_pxp_huc.o 2024-07-31T15:12:59.251 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:12:59.284 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_gpu_error.o 2024-07-31T15:12:59.379 INFO:tasks.workunit.client.0.smithi067.stdout: CC drivers/gpu/drm/i915/i915_vgpu.o 2024-07-31T15:12:59.718 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:12:59 smithi067 ceph-mon[32750]: pgmap v1099: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.1 MiB/s wr, 202 op/s 2024-07-31T15:12:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.733+0000 7f6bea615640 1 -- 172.21.15.67:0/103601134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6be41116d0 msgr2=0x7f6be4109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.734+0000 7f6bea615640 1 --2- 172.21.15.67:0/103601134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6be41116d0 0x7f6be4109df0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009860 tx=0x7f6bd402a380 comp rx=0 tx=0).stop 2024-07-31T15:12:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.734+0000 7f6bea615640 1 -- 172.21.15.67:0/103601134 shutdown_connections 2024-07-31T15:12:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.734+0000 7f6bea615640 1 --2- 172.21.15.67:0/103601134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6be411c5d0 0x7f6be411e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.734+0000 7f6bea615640 1 --2- 172.21.15.67:0/103601134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6be410a330 0x7f6be410a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.734+0000 7f6bea615640 1 --2- 172.21.15.67:0/103601134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6be41116d0 0x7f6be4109df0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.734+0000 7f6bea615640 1 -- 172.21.15.67:0/103601134 >> 172.21.15.67:0/103601134 conn(0x7f6be406e350 msgr2=0x7f6be406e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.734+0000 7f6bea615640 1 -- 172.21.15.67:0/103601134 shutdown_connections 2024-07-31T15:12:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.734+0000 7f6bea615640 1 -- 172.21.15.67:0/103601134 wait complete. 2024-07-31T15:12:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:12:59 smithi077 ceph-mon[37327]: pgmap v1099: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.1 MiB/s wr, 202 op/s 2024-07-31T15:12:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.735+0000 7f6bea615640 1 Processor -- start 2024-07-31T15:12:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.735+0000 7f6bea615640 1 -- start start 2024-07-31T15:12:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.735+0000 7f6bea615640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6be410a330 0x7f6be41a8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.735+0000 7f6bea615640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6be41116d0 0x7f6be41a9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.735+0000 7f6bea615640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6be411c5d0 0x7f6be41ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.735+0000 7f6bea615640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6be4122120 con 0x7f6be410a330 2024-07-31T15:12:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.735+0000 7f6bea615640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6be4121fa0 con 0x7f6be41116d0 2024-07-31T15:12:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.735+0000 7f6bea615640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6be41222a0 con 0x7f6be411c5d0 2024-07-31T15:12:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.736+0000 7f6be3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6be410a330 0x7f6be41a8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.736+0000 7f6be3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6be410a330 0x7f6be41a8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37810/0 (socket says 172.21.15.67:37810) 2024-07-31T15:12:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.736+0000 7f6be3fff640 1 -- 172.21.15.67:0/3034333460 learned_addr learned my addr 172.21.15.67:0/3034333460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.736+0000 7f6be8bb2640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6be411c5d0 0x7f6be41ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.736+0000 7f6be37fe640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6be41116d0 0x7f6be41a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.736+0000 7f6be3fff640 1 -- 172.21.15.67:0/3034333460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6be411c5d0 msgr2=0x7f6be41ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.736+0000 7f6be3fff640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6be411c5d0 0x7f6be41ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.736+0000 7f6be3fff640 1 -- 172.21.15.67:0/3034333460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6be41116d0 msgr2=0x7f6be41a9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.736+0000 7f6be3fff640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6be41116d0 0x7f6be41a9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.736+0000 7f6be3fff640 1 -- 172.21.15.67:0/3034333460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be41aeab0 con 0x7f6be410a330 2024-07-31T15:12:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.736+0000 7f6be8bb2640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6be411c5d0 0x7f6be41ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:12:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.736+0000 7f6be37fe640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6be41116d0 0x7f6be41a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:12:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.737+0000 7f6be3fff640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6be410a330 0x7f6be41a8b80 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4031000 tx=0x7f6bd4032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.747+0000 7f6be17fa640 1 -- 172.21.15.67:0/3034333460 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4040020 con 0x7f6be410a330 2024-07-31T15:12:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.747+0000 7f6be17fa640 1 -- 172.21.15.67:0/3034333460 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bd4038040 con 0x7f6be410a330 2024-07-31T15:12:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.747+0000 7f6be17fa640 1 -- 172.21.15.67:0/3034333460 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4037070 con 0x7f6be410a330 2024-07-31T15:12:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.747+0000 7f6bea615640 1 -- 172.21.15.67:0/3034333460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be41aed40 con 0x7f6be410a330 2024-07-31T15:12:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.747+0000 7f6bea615640 1 -- 172.21.15.67:0/3034333460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be41b5530 con 0x7f6be410a330 2024-07-31T15:12:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.751+0000 7f6bf15df740 1 Processor -- start 2024-07-31T15:12:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.752+0000 7f6be17fa640 1 -- 172.21.15.67:0/3034333460 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6bd4043020 con 0x7f6be410a330 2024-07-31T15:12:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.752+0000 7f6bf15df740 1 -- start start 2024-07-31T15:12:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.752+0000 7f6bf15df740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb421b0 0x55988cee4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.753+0000 7f6bf15df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb192f0 0x55988cee4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.753+0000 7f6bf15df740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb214d0 0x55988cee52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.753+0000 7f6bf15df740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55988ce5a3b0 con 0x55988cb214d0 2024-07-31T15:12:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.753+0000 7f6bf15df740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55988ce498b0 con 0x55988cb421b0 2024-07-31T15:12:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.753+0000 7f6bf15df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55988ce4d8f0 con 0x55988cb192f0 2024-07-31T15:12:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.753+0000 7f6be37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb214d0 0x55988cee52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.753+0000 7f6be8bb2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb421b0 0x55988cee4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.753+0000 7f6be8bb2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb421b0 0x55988cee4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56890/0 (socket says 172.21.15.67:56890) 2024-07-31T15:12:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.753+0000 7f6be8bb2640 1 -- 172.21.15.67:0/448246537 learned_addr learned my addr 172.21.15.67:0/448246537 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.753+0000 7f6be3fff640 1 --2- 172.21.15.67:0/448246537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb192f0 0x55988cee4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.753+0000 7f6be37fe640 1 -- 172.21.15.67:0/448246537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb192f0 msgr2=0x55988cee4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.753+0000 7f6be37fe640 1 --2- 172.21.15.67:0/448246537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb192f0 0x55988cee4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.753+0000 7f6be37fe640 1 -- 172.21.15.67:0/448246537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb421b0 msgr2=0x55988cee4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.753+0000 7f6be37fe640 1 --2- 172.21.15.67:0/448246537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb421b0 0x55988cee4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.754+0000 7f6be37fe640 1 -- 172.21.15.67:0/448246537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55988cdf09a0 con 0x55988cb214d0 2024-07-31T15:12:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.754+0000 7f6be37fe640 1 --2- 172.21.15.67:0/448246537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb214d0 0x55988cee52f0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80058c0 tx=0x7f6bd800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10f15bde6804c53 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.754+0000 7f6be17fa640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6bcc0b3ec0 0x7f6bcc0b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.755+0000 7f6bca7fc640 1 -- 172.21.15.67:0/448246537 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8017420 con 0x55988cb214d0 2024-07-31T15:12:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.755+0000 7f6bca7fc640 1 -- 172.21.15.67:0/448246537 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bd8017a70 con 0x55988cb214d0 2024-07-31T15:12:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.755+0000 7f6be8bb2640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6bcc0b3ec0 0x7f6bcc0b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.756+0000 7f6bf15df740 1 -- 172.21.15.67:0/448246537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb214d0 msgr2=0x55988cee52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.756+0000 7f6be8bb2640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6bcc0b3ec0 0x7f6bcc0b63d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00ea20 tx=0x7f6bdc010040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.756+0000 7f6bf15df740 1 --2- 172.21.15.67:0/448246537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb214d0 0x55988cee52f0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80058c0 tx=0x7f6bd800d710 comp rx=0 tx=0).stop 2024-07-31T15:12:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.756+0000 7f6bf15df740 1 -- 172.21.15.67:0/448246537 shutdown_connections 2024-07-31T15:12:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.756+0000 7f6bf15df740 1 --2- 172.21.15.67:0/448246537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb214d0 0x55988cee52f0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.756+0000 7f6bf15df740 1 --2- 172.21.15.67:0/448246537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb192f0 0x55988cee4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.756+0000 7f6bf15df740 1 --2- 172.21.15.67:0/448246537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb421b0 0x55988cee4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.756+0000 7f6bf15df740 1 -- 172.21.15.67:0/448246537 >> 172.21.15.67:0/448246537 conn(0x55988cb36300 msgr2=0x55988cee2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.756+0000 7f6be17fa640 1 -- 172.21.15.67:0/3034333460 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f6bd4104240 con 0x7f6be410a330 2024-07-31T15:12:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.757+0000 7f6bf15df740 1 -- 172.21.15.67:0/448246537 shutdown_connections 2024-07-31T15:12:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.757+0000 7f6bf15df740 1 -- 172.21.15.67:0/448246537 wait complete. 2024-07-31T15:12:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.757+0000 7f6bf15df740 1 -- start start 2024-07-31T15:12:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.757+0000 7f6bf15df740 1 Processor -- start 2024-07-31T15:12:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.758+0000 7f6bf15df740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb192f0 0x55988cef0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.758+0000 7f6bf15df740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb214d0 0x55988cef05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.758+0000 7f6bf15df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 0x55988cefc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.758+0000 7f6bf15df740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55988cd83ab0 con 0x55988cb192f0 2024-07-31T15:12:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.758+0000 7f6bf15df740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55988cdd1cf0 con 0x55988cb214d0 2024-07-31T15:12:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.758+0000 7f6bf15df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55988cef3040 con 0x55988cb421b0 2024-07-31T15:12:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.758+0000 7f6bf15df740 10 client.0 _ll_register_callbacks cb 0x55988ce433e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.759+0000 7f6be37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb214d0 0x55988cef05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.759+0000 7f6be37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb214d0 0x55988cef05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56894/0 (socket says 172.21.15.67:56894) 2024-07-31T15:12:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.759+0000 7f6be37fe640 1 -- 172.21.15.67:0/3224408564 learned_addr learned my addr 172.21.15.67:0/3224408564 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.759+0000 7f6be8bb2640 1 --2- 172.21.15.67:0/3224408564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 0x55988cefc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.759+0000 7f6be3fff640 1 --2- 172.21.15.67:0/3224408564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb192f0 0x55988cef0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.760+0000 7f6be8bb2640 1 -- 172.21.15.67:0/3224408564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb214d0 msgr2=0x55988cef05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.760+0000 7f6be8bb2640 1 --2- 172.21.15.67:0/3224408564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb214d0 0x55988cef05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.760+0000 7f6be8bb2640 1 -- 172.21.15.67:0/3224408564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb192f0 msgr2=0x55988cef0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.760+0000 7f6be8bb2640 1 --2- 172.21.15.67:0/3224408564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb192f0 0x55988cef0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.760+0000 7f6be8bb2640 1 -- 172.21.15.67:0/3224408564 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988cbad790 con 0x55988cb421b0 2024-07-31T15:12:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.761+0000 7f6be3fff640 1 --2- 172.21.15.67:0/3224408564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb192f0 0x55988cef0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:12:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.761+0000 7f6be8bb2640 1 --2- 172.21.15.67:0/3224408564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 0x55988cefc380 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00d7b0 tx=0x7f6bdc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2466272c4b937f5a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.761+0000 7f6bc7ff7640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:12:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.762+0000 7f6bc7ff7640 1 -- 172.21.15.67:0/3224408564 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc004d20 con 0x55988cb421b0 2024-07-31T15:12:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.762+0000 7f6bf15df740 1 -- 172.21.15.67:0/3224408564 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bd8017a70 con 0x55988cb421b0 2024-07-31T15:12:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.762+0000 7f6bc7ff7640 1 -- 172.21.15.67:0/3224408564 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f6bdc01a660 con 0x55988cb421b0 2024-07-31T15:12:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.762+0000 7f6bc7ff7640 1 -- 172.21.15.67:0/3224408564 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f6bdc030050 con 0x55988cb421b0 2024-07-31T15:12:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.762+0000 7f6bf15df740 10 client.29431 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.762+0000 7f6bf15df740 1 -- 172.21.15.67:0/3224408564 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55988cc26620 con 0x55988cb421b0 2024-07-31T15:12:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.763+0000 7f6bc7ff7640 1 -- 172.21.15.67:0/3224408564 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bd8017a70 con 0x55988cb421b0 2024-07-31T15:12:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.763+0000 7f6bc7ff7640 1 -- 172.21.15.67:0/3224408564 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f6bdc00b040 con 0x55988cb421b0 2024-07-31T15:12:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.764+0000 7f6bf15df740 10 client.29431 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.764+0000 7f6bf15df740 10 client.29431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:12:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.764+0000 7f6bf15df740 20 client.29431 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.764+0000 7f6bf15df740 1 --2- 172.21.15.67:0/3224408564 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55988cb00d80 0x55988cf137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.764+0000 7f6bf15df740 4 client.29431 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:12:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.764+0000 7f6bf15df740 1 -- 172.21.15.67:0/3224408564 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55988ce5c300 con 0x55988cb00d80 2024-07-31T15:12:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.764+0000 7f6be3fff640 1 --2- 172.21.15.67:0/3224408564 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55988cb00d80 0x55988cf137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.764+0000 7f6bc7ff7640 10 client.29431 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.765+0000 7f6be3fff640 1 --2- 172.21.15.67:0/3224408564 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55988cb00d80 0x55988cf137d0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69c91e0f7e8653bc server_cookie=7d6b32fe9c17039b in_seq=0 out_seq=0 2024-07-31T15:12:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.766+0000 7f6bc7ff7640 1 -- 172.21.15.67:0/3224408564 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55988ce5c300 con 0x55988cb00d80 2024-07-31T15:12:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.766+0000 7f6bc7ff7640 10 client.29431 handle_command_reply: tid=0 2024-07-31T15:12:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.766+0000 7f6bf15df740 1 client.29431 shutdown 2024-07-31T15:12:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.766+0000 7f6bf15df740 1 -- 172.21.15.67:0/3224408564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 msgr2=0x55988cefc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.766+0000 7f6bf15df740 1 --2- 172.21.15.67:0/3224408564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 0x55988cefc380 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00d7b0 tx=0x7f6bdc004ab0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.766+0000 7f6bf15df740 1 -- 172.21.15.67:0/3224408564 shutdown_connections 2024-07-31T15:12:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.767+0000 7f6bf15df740 1 --2- 172.21.15.67:0/3224408564 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55988cb00d80 0x55988cf137d0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.767+0000 7f6bf15df740 1 --2- 172.21.15.67:0/3224408564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 0x55988cefc380 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.767+0000 7f6bf15df740 1 --2- 172.21.15.67:0/3224408564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb214d0 0x55988cef05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.767+0000 7f6bf15df740 1 --2- 172.21.15.67:0/3224408564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb192f0 0x55988cef0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.767+0000 7f6bf15df740 1 -- 172.21.15.67:0/3224408564 >> 172.21.15.67:0/3224408564 conn(0x55988cb36300 msgr2=0x55988cee58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.767+0000 7f6bc7ff7640 0 client.29431 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.767+0000 7f6bf15df740 1 -- 172.21.15.67:0/3224408564 shutdown_connections 2024-07-31T15:12:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.767+0000 7f6bf15df740 1 -- 172.21.15.67:0/3224408564 wait complete. 2024-07-31T15:12:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.767+0000 7f6bf15df740 20 client.29431 trim_cache size 0 max 16384 2024-07-31T15:12:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.801+0000 7f6bf15df740 1 Processor -- start 2024-07-31T15:12:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.801+0000 7f6bf15df740 1 -- start start 2024-07-31T15:12:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.802+0000 7f6bf15df740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb192f0 0x55988cefc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.802+0000 7f6bf15df740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb214d0 0x55988ceef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.802+0000 7f6be3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb192f0 0x55988cefc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.802+0000 7f6be3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb192f0 0x55988cefc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56896/0 (socket says 172.21.15.67:56896) 2024-07-31T15:12:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.802+0000 7f6be3fff640 1 -- 172.21.15.67:0/1241215210 learned_addr learned my addr 172.21.15.67:0/1241215210 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.802+0000 7f6bf15df740 1 --2- 172.21.15.67:0/1241215210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 0x55988ceefa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.803+0000 7f6bf15df740 1 -- 172.21.15.67:0/1241215210 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6be41b8f10 con 0x55988cb214d0 2024-07-31T15:12:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.803+0000 7f6be37fe640 1 --2- 172.21.15.67:0/1241215210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb214d0 0x55988ceef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.803+0000 7f6bf15df740 1 -- 172.21.15.67:0/1241215210 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6be41b6d00 con 0x55988cb192f0 2024-07-31T15:12:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.803+0000 7f6be8bb2640 1 --2- 172.21.15.67:0/1241215210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 0x55988ceefa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.803+0000 7f6bf15df740 1 -- 172.21.15.67:0/1241215210 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6be41b8840 con 0x55988cb421b0 2024-07-31T15:12:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.803+0000 7f6be3fff640 1 -- 172.21.15.67:0/1241215210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 msgr2=0x55988ceefa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.803+0000 7f6be3fff640 1 --2- 172.21.15.67:0/1241215210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 0x55988ceefa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.803+0000 7f6be3fff640 1 -- 172.21.15.67:0/1241215210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb214d0 msgr2=0x55988ceef540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.804+0000 7f6be3fff640 1 --2- 172.21.15.67:0/1241215210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb214d0 0x55988ceef540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.804+0000 7f6be3fff640 1 -- 172.21.15.67:0/1241215210 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55988cee6da0 con 0x55988cb192f0 2024-07-31T15:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.804+0000 7f6be37fe640 1 --2- 172.21.15.67:0/1241215210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb214d0 0x55988ceef540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.804+0000 7f6be3fff640 1 --2- 172.21.15.67:0/1241215210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb192f0 0x55988cefc170 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4000c00 tx=0x7f6bd40050e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4184e2207021070d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.804+0000 7f6bc9ffb640 1 -- 172.21.15.67:0/1241215210 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd40c6400 con 0x55988cb192f0 2024-07-31T15:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.805+0000 7f6bc9ffb640 1 -- 172.21.15.67:0/1241215210 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bd4009470 con 0x55988cb192f0 2024-07-31T15:12:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.806+0000 7f6bf15df740 1 -- 172.21.15.67:0/1241215210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb192f0 msgr2=0x55988cefc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:12:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.806+0000 7f6bf15df740 1 --2- 172.21.15.67:0/1241215210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb192f0 0x55988cefc170 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4000c00 tx=0x7f6bd40050e0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.806+0000 7f6bf15df740 1 -- 172.21.15.67:0/1241215210 shutdown_connections 2024-07-31T15:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.806+0000 7f6bf15df740 1 --2- 172.21.15.67:0/1241215210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 0x55988ceefa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.806+0000 7f6bf15df740 1 --2- 172.21.15.67:0/1241215210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb214d0 0x55988ceef540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.806+0000 7f6bf15df740 1 --2- 172.21.15.67:0/1241215210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb192f0 0x55988cefc170 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.806+0000 7f6bf15df740 1 -- 172.21.15.67:0/1241215210 >> 172.21.15.67:0/1241215210 conn(0x55988cb36300 msgr2=0x55988ceee8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.806+0000 7f6bf15df740 1 -- 172.21.15.67:0/1241215210 shutdown_connections 2024-07-31T15:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.807+0000 7f6bf15df740 1 -- 172.21.15.67:0/1241215210 wait complete. 2024-07-31T15:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.807+0000 7f6bf15df740 1 -- start start 2024-07-31T15:12:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.807+0000 7f6bf15df740 1 Processor -- start 2024-07-31T15:12:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6bf15df740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb192f0 0x55988cef8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6bf15df740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb214d0 0x55988cef52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6bf15df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 0x55988cef57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6bf15df740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6be41b7aa0 con 0x55988cb192f0 2024-07-31T15:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6bf15df740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6be41b73d0 con 0x55988cb214d0 2024-07-31T15:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6bf15df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6be41b8170 con 0x55988cb421b0 2024-07-31T15:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6bf15df740 10 client.0 _ll_register_callbacks cb 0x7f6bdc00e740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6be37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb214d0 0x55988cef52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6be37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb214d0 0x55988cef52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56906/0 (socket says 172.21.15.67:56906) 2024-07-31T15:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6be37fe640 1 -- 172.21.15.67:0/2478658239 learned_addr learned my addr 172.21.15.67:0/2478658239 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6be3fff640 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb192f0 0x55988cef8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6be8bb2640 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 0x55988cef57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6be37fe640 1 -- 172.21.15.67:0/2478658239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 msgr2=0x55988cef57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6be37fe640 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 0x55988cef57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6be37fe640 1 -- 172.21.15.67:0/2478658239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb192f0 msgr2=0x55988cef8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.808+0000 7f6be37fe640 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb192f0 0x55988cef8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.809+0000 7f6be37fe640 1 -- 172.21.15.67:0/2478658239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988cef5de0 con 0x55988cb214d0 2024-07-31T15:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.809+0000 7f6be3fff640 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb192f0 0x55988cef8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.809+0000 7f6be8bb2640 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 0x55988cef57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.809+0000 7f6be37fe640 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb214d0 0x55988cef52b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800f590 tx=0x7f6bd800f720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bbef0977a4fd2e0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.809+0000 7f6bc7ff7640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.809+0000 7f6bc7ff7640 1 -- 172.21.15.67:0/2478658239 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8017760 con 0x55988cb214d0 2024-07-31T15:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.809+0000 7f6bf15df740 1 -- 172.21.15.67:0/2478658239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bd4009470 con 0x55988cb214d0 2024-07-31T15:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.809+0000 7f6bc7ff7640 1 -- 172.21.15.67:0/2478658239 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f6bd8017900 con 0x55988cb214d0 2024-07-31T15:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.810+0000 7f6bc7ff7640 1 -- 172.21.15.67:0/2478658239 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f6bd8017b00 con 0x55988cb214d0 2024-07-31T15:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.810+0000 7f6bf15df740 10 client.29573 fetch_fsmap learned FSMap version 66 2024-07-31T15:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.810+0000 7f6bf15df740 1 -- 172.21.15.67:0/2478658239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55988cf7ad10 con 0x55988cb214d0 2024-07-31T15:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.810+0000 7f6bc7ff7640 1 -- 172.21.15.67:0/2478658239 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bd4009470 con 0x55988cb214d0 2024-07-31T15:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.810+0000 7f6bc7ff7640 1 -- 172.21.15.67:0/2478658239 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f6bd8023420 con 0x55988cb214d0 2024-07-31T15:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.810+0000 7f6bf15df740 10 client.29573 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.810+0000 7f6bf15df740 10 client.29573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.810+0000 7f6bf15df740 20 client.29573 populate_metadata read hostname 'smithi067' 2024-07-31T15:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.810+0000 7f6bf15df740 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55988cad61c0 0x55988cf7d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:12:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.810+0000 7f6bf15df740 4 client.29573 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:12:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.810+0000 7f6bf15df740 1 -- 172.21.15.67:0/2478658239 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55988ce57da0 con 0x55988cad61c0 2024-07-31T15:12:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.811+0000 7f6be3fff640 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55988cad61c0 0x55988cf7d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:12:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.811+0000 7f6bc7ff7640 10 client.29573 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:12:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:12:59.811+0000 7f6be3fff640 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55988cad61c0 0x55988cf7d400 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f68b2e2bfcdfabbf server_cookie=fafa1d1f89375d2e in_seq=0 out_seq=0 2024-07-31T15:12:59.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:12:59 smithi073 ceph-mon[36714]: pgmap v1099: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.1 MiB/s wr, 202 op/s 2024-07-31T15:13:00.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.669+0000 7f2d988c0640 1 -- 172.21.15.67:0/1184954681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d9410a330 msgr2=0x7f2d9410a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:00.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.669+0000 7f2d988c0640 1 --2- 172.21.15.67:0/1184954681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d9410a330 0x7f2d9410a740 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009be0 tx=0x7f2d8c02a790 comp rx=0 tx=0).stop 2024-07-31T15:13:00.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.674+0000 7f2d988c0640 1 -- 172.21.15.67:0/1184954681 shutdown_connections 2024-07-31T15:13:00.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.674+0000 7f2d988c0640 1 --2- 172.21.15.67:0/1184954681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d9411c5d0 0x7f2d9411e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.674+0000 7f2d988c0640 1 --2- 172.21.15.67:0/1184954681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d9410a330 0x7f2d9410a740 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.674+0000 7f2d988c0640 1 --2- 172.21.15.67:0/1184954681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d94111790 0x7f2d94109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.674+0000 7f2d988c0640 1 -- 172.21.15.67:0/1184954681 >> 172.21.15.67:0/1184954681 conn(0x7f2d9406e420 msgr2=0x7f2d9406e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:00.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.680+0000 7f2d988c0640 1 -- 172.21.15.67:0/1184954681 shutdown_connections 2024-07-31T15:13:00.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.682+0000 7f2d988c0640 1 -- 172.21.15.67:0/1184954681 wait complete. 2024-07-31T15:13:00.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.683+0000 7f2d988c0640 1 Processor -- start 2024-07-31T15:13:00.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.688+0000 7f2d988c0640 1 -- start start 2024-07-31T15:13:00.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.689+0000 7f2d988c0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d9410a330 0x7f2d941a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.689+0000 7f2d988c0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d94111790 0x7f2d941a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.689+0000 7f2d988c0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d9411c5d0 0x7f2d941ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.689+0000 7f2d988c0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94122170 con 0x7f2d9410a330 2024-07-31T15:13:00.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.689+0000 7f2d988c0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94121ff0 con 0x7f2d9411c5d0 2024-07-31T15:13:00.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.689+0000 7f2d988c0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941222f0 con 0x7f2d94111790 2024-07-31T15:13:00.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.690+0000 7f2d937fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d9410a330 0x7f2d941a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.690+0000 7f2d937fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d9410a330 0x7f2d941a8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37852/0 (socket says 172.21.15.67:37852) 2024-07-31T15:13:00.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.690+0000 7f2d937fe640 1 -- 172.21.15.67:0/406814980 learned_addr learned my addr 172.21.15.67:0/406814980 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:00.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.691+0000 7f2d93fff640 1 --2- 172.21.15.67:0/406814980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d9411c5d0 0x7f2d941ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.691+0000 7f2d92ffd640 1 --2- 172.21.15.67:0/406814980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d94111790 0x7f2d941a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.691+0000 7f2d937fe640 1 -- 172.21.15.67:0/406814980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d94111790 msgr2=0x7f2d941a91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.691+0000 7f2d937fe640 1 --2- 172.21.15.67:0/406814980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d94111790 0x7f2d941a91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.691+0000 7f2d937fe640 1 -- 172.21.15.67:0/406814980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d9411c5d0 msgr2=0x7f2d941ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.691+0000 7f2d937fe640 1 --2- 172.21.15.67:0/406814980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d9411c5d0 0x7f2d941ae440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.691+0000 7f2d937fe640 1 -- 172.21.15.67:0/406814980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d941aeb70 con 0x7f2d9410a330 2024-07-31T15:13:00.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.693+0000 7f2d937fe640 1 --2- 172.21.15.67:0/406814980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d9410a330 0x7f2d941a8c00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400b450 tx=0x7f2d8400b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.701+0000 7f2d90ff9640 1 -- 172.21.15.67:0/406814980 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84015020 con 0x7f2d9410a330 2024-07-31T15:13:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.701+0000 7f2d90ff9640 1 -- 172.21.15.67:0/406814980 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d84011ce0 con 0x7f2d9410a330 2024-07-31T15:13:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.701+0000 7f2d90ff9640 1 -- 172.21.15.67:0/406814980 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84010de0 con 0x7f2d9410a330 2024-07-31T15:13:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.702+0000 7f2d988c0640 1 -- 172.21.15.67:0/406814980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d941aee60 con 0x7f2d9410a330 2024-07-31T15:13:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.702+0000 7f2d988c0640 1 -- 172.21.15.67:0/406814980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d941b55c0 con 0x7f2d9410a330 2024-07-31T15:13:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.708+0000 7f2da0073740 1 Processor -- start 2024-07-31T15:13:00.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.714+0000 7f2da0073740 1 -- start start 2024-07-31T15:13:00.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.714+0000 7f2da0073740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca28070 0x55710cdcd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.714+0000 7f2da0073740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9ff1b0 0x55710cdcdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.714+0000 7f2da0073740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca07390 0x55710cdce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.714+0000 7f2da0073740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55710cd43bb0 con 0x55710ca28070 2024-07-31T15:13:00.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.714+0000 7f2da0073740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55710cd330b0 con 0x55710ca07390 2024-07-31T15:13:00.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.714+0000 7f2da0073740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55710cd370f0 con 0x55710c9ff1b0 2024-07-31T15:13:00.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.717+0000 7f2d93fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca28070 0x55710cdcd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.717+0000 7f2d93fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca28070 0x55710cdcd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37854/0 (socket says 172.21.15.67:37854) 2024-07-31T15:13:00.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.717+0000 7f2d93fff640 1 -- 172.21.15.67:0/3823819214 learned_addr learned my addr 172.21.15.67:0/3823819214 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:00.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.718+0000 7f2d92ffd640 1 --2- 172.21.15.67:0/3823819214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca07390 0x55710cdce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.718+0000 7f2d90ff9640 1 -- 172.21.15.67:0/406814980 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2d84018020 con 0x7f2d9410a330 2024-07-31T15:13:00.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.719+0000 7f2d90ff9640 1 --2- 172.21.15.67:0/406814980 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2d7c0b3f50 0x7f2d7c0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.719+0000 7f2d90ff9640 1 -- 172.21.15.67:0/406814980 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f2d840e38f0 con 0x7f2d9410a330 2024-07-31T15:13:00.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.720+0000 7f2d93fff640 1 --2- 172.21.15.67:0/406814980 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2d7c0b3f50 0x7f2d7c0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.720+0000 7f2d93fff640 1 -- 172.21.15.67:0/3823819214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9ff1b0 msgr2=0x55710cdcdd90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:13:00.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.720+0000 7f2d93fff640 1 --2- 172.21.15.67:0/3823819214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9ff1b0 0x55710cdcdd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.720+0000 7f2d93fff640 1 -- 172.21.15.67:0/3823819214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca07390 msgr2=0x55710cdce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:00.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.720+0000 7f2d93fff640 1 --2- 172.21.15.67:0/3823819214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca07390 0x55710cdce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.720+0000 7f2d93fff640 1 -- 172.21.15.67:0/3823819214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55710ccd9b90 con 0x55710ca28070 2024-07-31T15:13:00.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.723+0000 7f2d93fff640 1 --2- 172.21.15.67:0/3823819214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca28070 0x55710cdcd850 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f2d88011800 tx=0x7f2d88011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364fe42614d81ffe server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:00.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.724+0000 7f2d93fff640 1 --2- 172.21.15.67:0/406814980 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2d7c0b3f50 0x7f2d7c0b6460 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2d8801e000 tx=0x7f2d88020000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:00.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.724+0000 7f2d79ffb640 1 -- 172.21.15.67:0/3823819214 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8801a040 con 0x55710ca28070 2024-07-31T15:13:00.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.724+0000 7f2d79ffb640 1 -- 172.21.15.67:0/3823819214 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d88022070 con 0x55710ca28070 2024-07-31T15:13:00.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.726+0000 7f2da0073740 1 -- 172.21.15.67:0/3823819214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca28070 msgr2=0x55710cdcd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:00.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.726+0000 7f2da0073740 1 --2- 172.21.15.67:0/3823819214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca28070 0x55710cdcd850 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f2d88011800 tx=0x7f2d88011cc0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.728+0000 7f2da0073740 1 -- 172.21.15.67:0/3823819214 shutdown_connections 2024-07-31T15:13:00.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.728+0000 7f2da0073740 1 --2- 172.21.15.67:0/3823819214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca07390 0x55710cdce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.728+0000 7f2da0073740 1 --2- 172.21.15.67:0/3823819214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9ff1b0 0x55710cdcdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.728+0000 7f2da0073740 1 --2- 172.21.15.67:0/3823819214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca28070 0x55710cdcd850 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.728+0000 7f2da0073740 1 -- 172.21.15.67:0/3823819214 >> 172.21.15.67:0/3823819214 conn(0x55710ca1c460 msgr2=0x55710cdcb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:00.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.732+0000 7f2da0073740 1 -- 172.21.15.67:0/3823819214 shutdown_connections 2024-07-31T15:13:00.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.736+0000 7f2da0073740 1 -- 172.21.15.67:0/3823819214 wait complete. 2024-07-31T15:13:00.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.736+0000 7f2da0073740 1 -- start start 2024-07-31T15:13:00.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.736+0000 7f2da0073740 1 Processor -- start 2024-07-31T15:13:00.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.740+0000 7f2da0073740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca07390 0x55710cdd9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.740+0000 7f2da0073740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca28070 0x55710cdd9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.740+0000 7f2da0073740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9e69f0 0x55710cdf6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.740+0000 7f2da0073740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55710cb96700 con 0x55710ca07390 2024-07-31T15:13:00.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.740+0000 7f2da0073740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55710ccd59e0 con 0x55710ca28070 2024-07-31T15:13:00.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.740+0000 7f2da0073740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55710cded030 con 0x55710c9e69f0 2024-07-31T15:13:00.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.740+0000 7f2da0073740 10 client.0 _ll_register_callbacks cb 0x55710ccf04f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:00.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.741+0000 7f2d92ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca28070 0x55710cdd9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.741+0000 7f2d93fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9e69f0 0x55710cdf6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.741+0000 7f2d92ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca28070 0x55710cdd9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56948/0 (socket says 172.21.15.67:56948) 2024-07-31T15:13:00.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.741+0000 7f2d92ffd640 1 -- 172.21.15.67:0/2550367522 learned_addr learned my addr 172.21.15.67:0/2550367522 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:00.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.741+0000 7f2d937fe640 1 --2- 172.21.15.67:0/2550367522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca07390 0x55710cdd9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.745+0000 7f2d937fe640 1 -- 172.21.15.67:0/2550367522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9e69f0 msgr2=0x55710cdf6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.745+0000 7f2d937fe640 1 --2- 172.21.15.67:0/2550367522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9e69f0 0x55710cdf6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.745+0000 7f2d937fe640 1 -- 172.21.15.67:0/2550367522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca28070 msgr2=0x55710cdd9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.745+0000 7f2d937fe640 1 --2- 172.21.15.67:0/2550367522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca28070 0x55710cdd9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.745+0000 7f2d937fe640 1 -- 172.21.15.67:0/2550367522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55710ca93650 con 0x55710ca07390 2024-07-31T15:13:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.745+0000 7f2d937fe640 1 --2- 172.21.15.67:0/2550367522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca07390 0x55710cdd9480 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f2d840e0d80 tx=0x7f2d840105b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875cf1f430b022ec server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.745+0000 7f2d72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:13:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.750+0000 7f2d72ffd640 1 -- 172.21.15.67:0/2550367522 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d840aa240 con 0x55710ca07390 2024-07-31T15:13:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.750+0000 7f2da0073740 1 -- 172.21.15.67:0/2550367522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d88022070 con 0x55710ca07390 2024-07-31T15:13:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.750+0000 7f2d72ffd640 1 -- 172.21.15.67:0/2550367522 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f2d840a6830 con 0x55710ca07390 2024-07-31T15:13:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.756+0000 7f2d72ffd640 1 -- 172.21.15.67:0/2550367522 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f2d840ca050 con 0x55710ca07390 2024-07-31T15:13:00.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.759+0000 7f2da0073740 10 client.20067 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:00.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.759+0000 7f2da0073740 1 -- 172.21.15.67:0/2550367522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55710cdf91d0 con 0x55710ca07390 2024-07-31T15:13:00.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.763+0000 7f2d72ffd640 1 -- 172.21.15.67:0/2550367522 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d88022070 con 0x55710ca07390 2024-07-31T15:13:00.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.763+0000 7f2d72ffd640 1 -- 172.21.15.67:0/2550367522 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f2d840cb240 con 0x55710ca07390 2024-07-31T15:13:00.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.765+0000 7f2da0073740 10 client.20067 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:00.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.765+0000 7f2da0073740 10 client.20067 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:13:00.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.765+0000 7f2da0073740 20 client.20067 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:00.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.765+0000 7f2da0073740 1 --2- 172.21.15.67:0/2550367522 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55710c9bbda0 0x55710cdfc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.765+0000 7f2da0073740 4 client.20067 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:13:00.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.765+0000 7f2da0073740 1 -- 172.21.15.67:0/2550367522 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55710cd45b00 con 0x55710c9bbda0 2024-07-31T15:13:00.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.766+0000 7f2d92ffd640 1 --2- 172.21.15.67:0/2550367522 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55710c9bbda0 0x55710cdfc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.769+0000 7f2d92ffd640 1 --2- 172.21.15.67:0/2550367522 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55710c9bbda0 0x55710cdfc720 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a199c39e658f189f server_cookie=61f22c4dbcae239d in_seq=0 out_seq=0 2024-07-31T15:13:00.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.771+0000 7f2d72ffd640 10 client.20067 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:13:00.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.771+0000 7f2d72ffd640 1 -- 172.21.15.67:0/2550367522 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55710cd45b00 con 0x55710c9bbda0 2024-07-31T15:13:00.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.771+0000 7f2d72ffd640 10 client.20067 handle_command_reply: tid=0 2024-07-31T15:13:00.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.771+0000 7f2da0073740 1 client.20067 shutdown 2024-07-31T15:13:00.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.775+0000 7f2da0073740 1 -- 172.21.15.67:0/2550367522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca07390 msgr2=0x55710cdd9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:00.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.775+0000 7f2da0073740 1 --2- 172.21.15.67:0/2550367522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca07390 0x55710cdd9480 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f2d840e0d80 tx=0x7f2d840105b0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.775+0000 7f2da0073740 1 -- 172.21.15.67:0/2550367522 shutdown_connections 2024-07-31T15:13:00.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.775+0000 7f2da0073740 1 --2- 172.21.15.67:0/2550367522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9e69f0 0x55710cdf6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.775+0000 7f2da0073740 1 --2- 172.21.15.67:0/2550367522 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55710c9bbda0 0x55710cdfc720 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.775+0000 7f2da0073740 1 --2- 172.21.15.67:0/2550367522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca28070 0x55710cdd9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.775+0000 7f2da0073740 1 --2- 172.21.15.67:0/2550367522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca07390 0x55710cdd9480 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.775+0000 7f2da0073740 1 -- 172.21.15.67:0/2550367522 >> 172.21.15.67:0/2550367522 conn(0x55710ca1c460 msgr2=0x55710cdd0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:00.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.775+0000 7f2d72ffd640 0 client.20067 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:13:00.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.779+0000 7f2da0073740 1 -- 172.21.15.67:0/2550367522 shutdown_connections 2024-07-31T15:13:00.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.779+0000 7f2da0073740 1 -- 172.21.15.67:0/2550367522 wait complete. 2024-07-31T15:13:00.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.779+0000 7f2da0073740 20 client.20067 trim_cache size 0 max 16384 2024-07-31T15:13:00.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.812+0000 7f2da0073740 1 Processor -- start 2024-07-31T15:13:00.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.814+0000 7f2da0073740 1 -- start start 2024-07-31T15:13:00.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.814+0000 7f2da0073740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710c9e69f0 0x55710cdf61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.818+0000 7f2da0073740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710ca07390 0x55710cddaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.818+0000 7f2da0073740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca28070 0x55710ce01f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.818+0000 7f2da0073740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8000fee0 con 0x55710c9e69f0 2024-07-31T15:13:00.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.818+0000 7f2da0073740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2d80011a20 con 0x55710ca28070 2024-07-31T15:13:00.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.818+0000 7f2da0073740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d800105b0 con 0x55710ca07390 2024-07-31T15:13:00.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.825+0000 7f2d937fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710c9e69f0 0x55710cdf61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.825+0000 7f2d937fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710c9e69f0 0x55710cdf61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37870/0 (socket says 172.21.15.67:37870) 2024-07-31T15:13:00.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.825+0000 7f2d937fe640 1 -- 172.21.15.67:0/1905507847 learned_addr learned my addr 172.21.15.67:0/1905507847 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:00.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.826+0000 7f2d92ffd640 1 --2- 172.21.15.67:0/1905507847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710ca07390 0x55710cddaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.826+0000 7f2d93fff640 1 --2- 172.21.15.67:0/1905507847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca28070 0x55710ce01f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.835+0000 7f2d92ffd640 1 -- 172.21.15.67:0/1905507847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca28070 msgr2=0x55710ce01f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.835+0000 7f2d92ffd640 1 --2- 172.21.15.67:0/1905507847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca28070 0x55710ce01f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.835+0000 7f2d92ffd640 1 -- 172.21.15.67:0/1905507847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710c9e69f0 msgr2=0x55710cdf61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.835+0000 7f2d92ffd640 1 --2- 172.21.15.67:0/1905507847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710c9e69f0 0x55710cdf61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.835+0000 7f2d92ffd640 1 -- 172.21.15.67:0/1905507847 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55710cdcfd70 con 0x55710ca07390 2024-07-31T15:13:00.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.844+0000 7f2d92ffd640 1 --2- 172.21.15.67:0/1905507847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710ca07390 0x55710cddaeb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55710cdfcf00 tx=0x7f2d8c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=180f45e2452d754e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:00.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.847+0000 7f2d797fa640 1 -- 172.21.15.67:0/1905507847 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c04d030 con 0x55710ca07390 2024-07-31T15:13:00.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.847+0000 7f2d797fa640 1 -- 172.21.15.67:0/1905507847 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d8c030da0 con 0x55710ca07390 2024-07-31T15:13:00.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.847+0000 7f2d797fa640 1 -- 172.21.15.67:0/1905507847 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c036040 con 0x55710ca07390 2024-07-31T15:13:00.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.848+0000 7f2da0073740 1 -- 172.21.15.67:0/1905507847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710ca07390 msgr2=0x55710cddaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:00.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.848+0000 7f2da0073740 1 --2- 172.21.15.67:0/1905507847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710ca07390 0x55710cddaeb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55710cdfcf00 tx=0x7f2d8c000f40 comp rx=0 tx=0).stop 2024-07-31T15:13:00.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.850+0000 7f2da0073740 1 -- 172.21.15.67:0/1905507847 shutdown_connections 2024-07-31T15:13:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.850+0000 7f2da0073740 1 --2- 172.21.15.67:0/1905507847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca28070 0x55710ce01f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.850+0000 7f2da0073740 1 --2- 172.21.15.67:0/1905507847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710ca07390 0x55710cddaeb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.850+0000 7f2da0073740 1 --2- 172.21.15.67:0/1905507847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710c9e69f0 0x55710cdf61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.850+0000 7f2da0073740 1 -- 172.21.15.67:0/1905507847 >> 172.21.15.67:0/1905507847 conn(0x55710ca1c460 msgr2=0x55710cdda7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:00.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.854+0000 7f2da0073740 1 -- 172.21.15.67:0/1905507847 shutdown_connections 2024-07-31T15:13:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.855+0000 7f2da0073740 1 -- 172.21.15.67:0/1905507847 wait complete. 2024-07-31T15:13:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.855+0000 7f2da0073740 1 -- start start 2024-07-31T15:13:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.856+0000 7f2da0073740 1 Processor -- start 2024-07-31T15:13:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.856+0000 7f2da0073740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9e69f0 0x55710cdf27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.857+0000 7f2da0073740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca07390 0x55710cdedc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.857+0000 7f2da0073740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca28070 0x55710cdee190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.857+0000 7f2da0073740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2d80011350 con 0x55710ca28070 2024-07-31T15:13:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.857+0000 7f2da0073740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2d80010c80 con 0x55710ca07390 2024-07-31T15:13:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.857+0000 7f2da0073740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8000dee0 con 0x55710c9e69f0 2024-07-31T15:13:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.857+0000 7f2da0073740 10 client.0 _ll_register_callbacks cb 0x7f2d8400bf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.862+0000 7f2d92ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca07390 0x55710cdedc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.862+0000 7f2d92ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca07390 0x55710cdedc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60134/0 (socket says 172.21.15.67:60134) 2024-07-31T15:13:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.862+0000 7f2d92ffd640 1 -- 172.21.15.67:0/1979966522 learned_addr learned my addr 172.21.15.67:0/1979966522 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.864+0000 7f2d93fff640 1 --2- 172.21.15.67:0/1979966522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca28070 0x55710cdee190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.865+0000 7f2d937fe640 1 --2- 172.21.15.67:0/1979966522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9e69f0 0x55710cdf27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.867+0000 7f2d937fe640 1 -- 172.21.15.67:0/1979966522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca07390 msgr2=0x55710cdedc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.867+0000 7f2d937fe640 1 --2- 172.21.15.67:0/1979966522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca07390 0x55710cdedc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.867+0000 7f2d937fe640 1 -- 172.21.15.67:0/1979966522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca28070 msgr2=0x55710cdee190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.867+0000 7f2d937fe640 1 --2- 172.21.15.67:0/1979966522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca28070 0x55710cdee190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.867+0000 7f2d937fe640 1 -- 172.21.15.67:0/1979966522 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55710cdee6d0 con 0x55710c9e69f0 2024-07-31T15:13:00.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.876+0000 7f2d937fe640 1 --2- 172.21.15.67:0/1979966522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9e69f0 0x55710cdf27a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2d840e0db0 tx=0x7f2d840e1100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca1e346be1976f5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:00.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.876+0000 7f2d72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:13:00.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.880+0000 7f2d72ffd640 1 -- 172.21.15.67:0/1979966522 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d840aa010 con 0x55710c9e69f0 2024-07-31T15:13:00.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.880+0000 7f2d72ffd640 1 -- 172.21.15.67:0/1979966522 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f2d840c4040 con 0x55710c9e69f0 2024-07-31T15:13:00.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.880+0000 7f2da0073740 1 -- 172.21.15.67:0/1979966522 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d8c030da0 con 0x55710c9e69f0 2024-07-31T15:13:00.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.883+0000 7f2d72ffd640 1 -- 172.21.15.67:0/1979966522 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f2d840a9b90 con 0x55710c9e69f0 2024-07-31T15:13:00.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.883+0000 7f2da0073740 10 client.29443 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:00.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.883+0000 7f2da0073740 1 -- 172.21.15.67:0/1979966522 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55710ce64bd0 con 0x55710c9e69f0 2024-07-31T15:13:00.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.887+0000 7f2d72ffd640 1 -- 172.21.15.67:0/1979966522 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d8c030da0 con 0x55710c9e69f0 2024-07-31T15:13:00.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.888+0000 7f2d72ffd640 1 -- 172.21.15.67:0/1979966522 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f2d840a5bb0 con 0x55710c9e69f0 2024-07-31T15:13:00.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.888+0000 7f2da0073740 10 client.29443 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:00.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.888+0000 7f2da0073740 10 client.29443 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:13:00.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.888+0000 7f2da0073740 20 client.29443 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:00.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.888+0000 7f2da0073740 1 --2- 172.21.15.67:0/1979966522 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55710ce65350 0x55710ce67770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:00.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.888+0000 7f2da0073740 4 client.29443 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:13:00.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.888+0000 7f2da0073740 1 -- 172.21.15.67:0/1979966522 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55710cd415a0 con 0x55710ce65350 2024-07-31T15:13:00.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.889+0000 7f2d92ffd640 1 --2- 172.21.15.67:0/1979966522 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55710ce65350 0x55710ce67770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:00.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.895+0000 7f2d72ffd640 10 client.29443 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:13:00.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.895+0000 7f2d92ffd640 1 --2- 172.21.15.67:0/1979966522 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55710ce65350 0x55710ce67770 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a98989fc6365b2f server_cookie=f33f7d83cd989fce in_seq=0 out_seq=0 2024-07-31T15:13:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.896+0000 7f2d72ffd640 1 -- 172.21.15.67:0/1979966522 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55710cd415a0 con 0x55710ce65350 2024-07-31T15:13:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.896+0000 7f2d72ffd640 10 client.29443 handle_command_reply: tid=0 2024-07-31T15:13:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.896+0000 7f2da0073740 1 client.29443 shutdown 2024-07-31T15:13:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.897+0000 7f2da0073740 1 -- 172.21.15.67:0/1979966522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9e69f0 msgr2=0x55710cdf27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.897+0000 7f2da0073740 1 --2- 172.21.15.67:0/1979966522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9e69f0 0x55710cdf27a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2d840e0db0 tx=0x7f2d840e1100 comp rx=0 tx=0).stop 2024-07-31T15:13:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.897+0000 7f2da0073740 1 -- 172.21.15.67:0/1979966522 shutdown_connections 2024-07-31T15:13:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.897+0000 7f2da0073740 1 --2- 172.21.15.67:0/1979966522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55710ca28070 0x55710cdee190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.897+0000 7f2da0073740 1 --2- 172.21.15.67:0/1979966522 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x55710ce65350 0x55710ce67770 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.897+0000 7f2da0073740 1 --2- 172.21.15.67:0/1979966522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55710ca07390 0x55710cdedc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.897+0000 7f2da0073740 1 --2- 172.21.15.67:0/1979966522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55710c9e69f0 0x55710cdf27a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.897+0000 7f2da0073740 1 -- 172.21.15.67:0/1979966522 >> 172.21.15.67:0/1979966522 conn(0x55710ca1c460 msgr2=0x55710cdcb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:00.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.897+0000 7f2d72ffd640 0 client.29443 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:13:00.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.897+0000 7f2da0073740 1 -- 172.21.15.67:0/1979966522 shutdown_connections 2024-07-31T15:13:00.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.898+0000 7f2da0073740 1 -- 172.21.15.67:0/1979966522 wait complete. 2024-07-31T15:13:00.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.899+0000 7f2da0073740 20 client.29443 trim_cache size 0 max 16384 2024-07-31T15:13:00.900 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:13:00.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.899+0000 7f2d988c0640 1 -- 172.21.15.67:0/406814980 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2d7c0b3f50 msgr2=0x7f2d7c0b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:00.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.899+0000 7f2d988c0640 1 --2- 172.21.15.67:0/406814980 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2d7c0b3f50 0x7f2d7c0b6460 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2d8801e000 tx=0x7f2d88020000 comp rx=0 tx=0).stop 2024-07-31T15:13:00.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.899+0000 7f2d988c0640 1 -- 172.21.15.67:0/406814980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d9410a330 msgr2=0x7f2d941a8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:00.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.900+0000 7f2d988c0640 1 --2- 172.21.15.67:0/406814980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d9410a330 0x7f2d941a8c00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400b450 tx=0x7f2d8400b850 comp rx=0 tx=0).stop 2024-07-31T15:13:00.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.900+0000 7f2d988c0640 1 -- 172.21.15.67:0/406814980 shutdown_connections 2024-07-31T15:13:00.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.900+0000 7f2d988c0640 1 --2- 172.21.15.67:0/406814980 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f2d7c0b3f50 0x7f2d7c0b6460 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.900+0000 7f2d988c0640 1 --2- 172.21.15.67:0/406814980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d9411c5d0 0x7f2d941ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.900+0000 7f2d988c0640 1 --2- 172.21.15.67:0/406814980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d94111790 0x7f2d941a91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.900+0000 7f2d988c0640 1 --2- 172.21.15.67:0/406814980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d9410a330 0x7f2d941a8c00 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.900+0000 7f2d988c0640 1 -- 172.21.15.67:0/406814980 >> 172.21.15.67:0/406814980 conn(0x7f2d9406e420 msgr2=0x7f2d94071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:00.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.900+0000 7f2d988c0640 1 -- 172.21.15.67:0/406814980 shutdown_connections 2024-07-31T15:13:00.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:00.900+0000 7f2d988c0640 1 -- 172.21.15.67:0/406814980 wait complete. 2024-07-31T15:13:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.961+0000 7f6bc7ff7640 1 -- 172.21.15.67:0/2478658239 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55988ce57da0 con 0x55988cad61c0 2024-07-31T15:13:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.961+0000 7f6bc7ff7640 10 client.29573 handle_command_reply: tid=0 2024-07-31T15:13:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.961+0000 7f6bf15df740 1 client.29573 shutdown 2024-07-31T15:13:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.961+0000 7f6bf15df740 1 -- 172.21.15.67:0/2478658239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb214d0 msgr2=0x55988cef52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.961+0000 7f6bf15df740 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb214d0 0x55988cef52b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800f590 tx=0x7f6bd800f720 comp rx=0 tx=0).stop 2024-07-31T15:13:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.962+0000 7f6bf15df740 1 -- 172.21.15.67:0/2478658239 shutdown_connections 2024-07-31T15:13:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.962+0000 7f6bf15df740 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55988cad61c0 0x55988cf7d400 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.962+0000 7f6bf15df740 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988cb421b0 0x55988cef57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.962+0000 7f6bf15df740 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988cb214d0 0x55988cef52b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.962+0000 7f6bf15df740 1 --2- 172.21.15.67:0/2478658239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55988cb192f0 0x55988cef8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.962+0000 7f6bf15df740 1 -- 172.21.15.67:0/2478658239 >> 172.21.15.67:0/2478658239 conn(0x55988cb36300 msgr2=0x55988cee58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.962+0000 7f6bc7ff7640 0 client.29573 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.962+0000 7f6bf15df740 1 -- 172.21.15.67:0/2478658239 shutdown_connections 2024-07-31T15:13:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.962+0000 7f6bf15df740 1 -- 172.21.15.67:0/2478658239 wait complete. 2024-07-31T15:13:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.962+0000 7f6bf15df740 20 client.29573 trim_cache size 0 max 16384 2024-07-31T15:13:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:13:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:13:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:13:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:13:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.963+0000 7f6bea615640 1 -- 172.21.15.67:0/3034333460 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6bcc0b3ec0 msgr2=0x7f6bcc0b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.963+0000 7f6bea615640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6bcc0b3ec0 0x7f6bcc0b63d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00ea20 tx=0x7f6bdc010040 comp rx=0 tx=0).stop 2024-07-31T15:13:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.963+0000 7f6bea615640 1 -- 172.21.15.67:0/3034333460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6be410a330 msgr2=0x7f6be41a8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.963+0000 7f6bea615640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6be410a330 0x7f6be41a8b80 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4031000 tx=0x7f6bd4032cd0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.963+0000 7f6bea615640 1 -- 172.21.15.67:0/3034333460 shutdown_connections 2024-07-31T15:13:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.963+0000 7f6bea615640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6bcc0b3ec0 0x7f6bcc0b63d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.963+0000 7f6bea615640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6be411c5d0 0x7f6be41ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.963+0000 7f6bea615640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6be41116d0 0x7f6be41a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.963+0000 7f6bea615640 1 --2- 172.21.15.67:0/3034333460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6be410a330 0x7f6be41a8b80 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.963+0000 7f6bea615640 1 -- 172.21.15.67:0/3034333460 >> 172.21.15.67:0/3034333460 conn(0x7f6be406e350 msgr2=0x7f6be4110350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.963+0000 7f6bea615640 1 -- 172.21.15.67:0/3034333460 shutdown_connections 2024-07-31T15:13:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:00.963+0000 7f6bea615640 1 -- 172.21.15.67:0/3034333460 wait complete. 2024-07-31T15:13:01.128 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/i915/built-in.a 2024-07-31T15:13:01.282 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/drm/built-in.a 2024-07-31T15:13:01.445 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/gpu/built-in.a 2024-07-31T15:13:01.507 INFO:tasks.workunit.client.0.smithi067.stdout: AR drivers/built-in.a 2024-07-31T15:13:01.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:01 smithi067 ceph-mon[32750]: pgmap v1100: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 513 KiB/s wr, 70 op/s 2024-07-31T15:13:01.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:01 smithi077 ceph-mon[37327]: pgmap v1100: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 513 KiB/s wr, 70 op/s 2024-07-31T15:13:01.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:01 smithi073 ceph-mon[36714]: pgmap v1100: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 513 KiB/s wr, 70 op/s 2024-07-31T15:13:02.108 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T15:13:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.610+0000 7fbaf572e640 1 -- 172.21.15.67:0/2928172388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaf0101b00 msgr2=0x7fbaf0101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.610+0000 7fbaf572e640 1 --2- 172.21.15.67:0/2928172388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaf0101b00 0x7fbaf0101f10 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009900 tx=0x7fbae002a410 comp rx=0 tx=0).stop 2024-07-31T15:13:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.610+0000 7fbaf572e640 1 -- 172.21.15.67:0/2928172388 shutdown_connections 2024-07-31T15:13:02.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.610+0000 7fbaf572e640 1 --2- 172.21.15.67:0/2928172388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaf0109830 0x7fbaf0110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.610+0000 7fbaf572e640 1 --2- 172.21.15.67:0/2928172388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbaf0108ee0 0x7fbaf01092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.610+0000 7fbaf572e640 1 --2- 172.21.15.67:0/2928172388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaf0101b00 0x7fbaf0101f10 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.610+0000 7fbaf572e640 1 -- 172.21.15.67:0/2928172388 >> 172.21.15.67:0/2928172388 conn(0x7fbaf0077690 msgr2=0x7fbaf0077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.610+0000 7fbaf572e640 1 -- 172.21.15.67:0/2928172388 shutdown_connections 2024-07-31T15:13:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.610+0000 7fbaf572e640 1 -- 172.21.15.67:0/2928172388 wait complete. 2024-07-31T15:13:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.610+0000 7fbaf572e640 1 Processor -- start 2024-07-31T15:13:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaf572e640 1 -- start start 2024-07-31T15:13:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaf572e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaf0101b00 0x7fbaf019c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaf572e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaf0108ee0 0x7fbaf019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaf572e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbaf0109830 0x7fbaf01a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaf572e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf0113490 con 0x7fbaf0101b00 2024-07-31T15:13:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaf572e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf0113310 con 0x7fbaf0109830 2024-07-31T15:13:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaf572e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf0113610 con 0x7fbaf0108ee0 2024-07-31T15:13:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaeeffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaf0101b00 0x7fbaf019c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaeeffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaf0101b00 0x7fbaf019c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53296/0 (socket says 172.21.15.67:53296) 2024-07-31T15:13:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaeeffd640 1 -- 172.21.15.67:0/1782080565 learned_addr learned my addr 172.21.15.67:0/1782080565 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaef7fe640 1 --2- 172.21.15.67:0/1782080565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbaf0109830 0x7fbaf01a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaeeffd640 1 -- 172.21.15.67:0/1782080565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaf0108ee0 msgr2=0x7fbaf019c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:13:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaeeffd640 1 --2- 172.21.15.67:0/1782080565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaf0108ee0 0x7fbaf019c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaeeffd640 1 -- 172.21.15.67:0/1782080565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbaf0109830 msgr2=0x7fbaf01a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.611+0000 7fbaeeffd640 1 --2- 172.21.15.67:0/1782080565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbaf0109830 0x7fbaf01a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.612+0000 7fbaeeffd640 1 -- 172.21.15.67:0/1782080565 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf01a2090 con 0x7fbaf0101b00 2024-07-31T15:13:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.612+0000 7fbaeeffd640 1 --2- 172.21.15.67:0/1782080565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaf0101b00 0x7fbaf019c120 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fbae0005940 tx=0x7fbae00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.612+0000 7fbad7fff640 1 -- 172.21.15.67:0/1782080565 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae003f030 con 0x7fbaf0101b00 2024-07-31T15:13:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.612+0000 7fbad7fff640 1 -- 172.21.15.67:0/1782080565 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbae0003cb0 con 0x7fbaf0101b00 2024-07-31T15:13:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.612+0000 7fbaf572e640 1 -- 172.21.15.67:0/1782080565 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf01a2320 con 0x7fbaf0101b00 2024-07-31T15:13:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.612+0000 7fbad7fff640 1 -- 172.21.15.67:0/1782080565 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0035070 con 0x7fbaf0101b00 2024-07-31T15:13:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.612+0000 7fbaf572e640 1 -- 172.21.15.67:0/1782080565 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf01a8ae0 con 0x7fbaf0101b00 2024-07-31T15:13:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.613+0000 7fbaf572e640 1 -- 172.21.15.67:0/1782080565 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf0101f10 con 0x7fbaf0101b00 2024-07-31T15:13:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.616+0000 7fbad7fff640 1 -- 172.21.15.67:0/1782080565 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbae0042030 con 0x7fbaf0101b00 2024-07-31T15:13:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.617+0000 7fbad7fff640 1 --2- 172.21.15.67:0/1782080565 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbad80b3d40 0x7fbad80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.617+0000 7fbaee7fc640 1 --2- 172.21.15.67:0/1782080565 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbad80b3d40 0x7fbad80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.617+0000 7fbad7fff640 1 -- 172.21.15.67:0/1782080565 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7fbae0104f30 con 0x7fbaf0101b00 2024-07-31T15:13:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.617+0000 7fbaee7fc640 1 --2- 172.21.15.67:0/1782080565 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbad80b3d40 0x7fbad80b6250 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbae40058c0 tx=0x7fbae4005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.620+0000 7fbad7fff640 1 -- 172.21.15.67:0/1782080565 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fbae00cbed0 con 0x7fbaf0101b00 2024-07-31T15:13:02.654 INFO:tasks.workunit.client.0.smithi067.stdout: AR built-in.a 2024-07-31T15:13:02.697 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:13:02.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.876+0000 7fbaf572e640 1 -- 172.21.15.67:0/1782080565 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbaf01a8e10 con 0x7fbaf0101b00 2024-07-31T15:13:02.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.877+0000 7fbad7fff640 1 -- 172.21.15.67:0/1782080565 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11959 (secure 0 0 0) 0x7fbae00d41a0 con 0x7fbaf0101b00 2024-07-31T15:13:02.878 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:13:02.879 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":66,"btime":"2024-07-31T15:10:30:235518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:10:30.235494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29129":{"gid":29129,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/631186801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":631186801},{"type":"v1","addr":"172.21.15.73:6835","nonce":631186801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:13:02.879 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 66 2024-07-31T15:13:02.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.879+0000 7fbaf572e640 1 -- 172.21.15.67:0/1782080565 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbad80b3d40 msgr2=0x7fbad80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.879+0000 7fbaf572e640 1 --2- 172.21.15.67:0/1782080565 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbad80b3d40 0x7fbad80b6250 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbae40058c0 tx=0x7fbae4005850 comp rx=0 tx=0).stop 2024-07-31T15:13:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.879+0000 7fbaf572e640 1 -- 172.21.15.67:0/1782080565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaf0101b00 msgr2=0x7fbaf019c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.879+0000 7fbaf572e640 1 --2- 172.21.15.67:0/1782080565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaf0101b00 0x7fbaf019c120 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fbae0005940 tx=0x7fbae00042f0 comp rx=0 tx=0).stop 2024-07-31T15:13:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.879+0000 7fbaf572e640 1 -- 172.21.15.67:0/1782080565 shutdown_connections 2024-07-31T15:13:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.879+0000 7fbaf572e640 1 --2- 172.21.15.67:0/1782080565 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbad80b3d40 0x7fbad80b6250 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.879+0000 7fbaf572e640 1 --2- 172.21.15.67:0/1782080565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbaf0109830 0x7fbaf01a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.879+0000 7fbaf572e640 1 --2- 172.21.15.67:0/1782080565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaf0108ee0 0x7fbaf019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.879+0000 7fbaf572e640 1 --2- 172.21.15.67:0/1782080565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaf0101b00 0x7fbaf019c120 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.879+0000 7fbaf572e640 1 -- 172.21.15.67:0/1782080565 >> 172.21.15.67:0/1782080565 conn(0x7fbaf0077690 msgr2=0x7fbaf0106cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.879+0000 7fbaf572e640 1 -- 172.21.15.67:0/1782080565 shutdown_connections 2024-07-31T15:13:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:02.879+0000 7fbaf572e640 1 -- 172.21.15.67:0/1782080565 wait complete. 2024-07-31T15:13:03.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:03 smithi067 ceph-mon[32750]: pgmap v1101: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 289 KiB/s wr, 99 op/s 2024-07-31T15:13:03.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:03 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1782080565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:13:03.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:03 smithi077 ceph-mon[37327]: pgmap v1101: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 289 KiB/s wr, 99 op/s 2024-07-31T15:13:03.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:03 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1782080565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:13:03.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:03 smithi073 ceph-mon[36714]: pgmap v1101: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 289 KiB/s wr, 99 op/s 2024-07-31T15:13:03.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:03 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1782080565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:13:04.619 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 131.7 --await-for 45.0' 2024-07-31T15:13:04.619 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 131.7 --await-for 45.0 2024-07-31T15:13:05.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.154+0000 7f90ab334640 1 -- 172.21.15.67:0/332523272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90a41029d0 msgr2=0x7f90a410f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:05.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.154+0000 7f90ab334640 1 --2- 172.21.15.67:0/332523272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90a41029d0 0x7f90a410f6f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f9094009900 tx=0x7f909402a480 comp rx=0 tx=0).stop 2024-07-31T15:13:05.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.154+0000 7f90ab334640 1 -- 172.21.15.67:0/332523272 shutdown_connections 2024-07-31T15:13:05.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.154+0000 7f90ab334640 1 --2- 172.21.15.67:0/332523272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90a410fc30 0x7f90a4112050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.154+0000 7f90ab334640 1 --2- 172.21.15.67:0/332523272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90a41029d0 0x7f90a410f6f0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.154+0000 7f90ab334640 1 --2- 172.21.15.67:0/332523272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a4102080 0x7f90a4102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.154+0000 7f90ab334640 1 -- 172.21.15.67:0/332523272 >> 172.21.15.67:0/332523272 conn(0x7f90a4079160 msgr2=0x7f90a4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:05.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.155+0000 7f90ab334640 1 -- 172.21.15.67:0/332523272 shutdown_connections 2024-07-31T15:13:05.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.155+0000 7f90ab334640 1 -- 172.21.15.67:0/332523272 wait complete. 2024-07-31T15:13:05.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.155+0000 7f90ab334640 1 Processor -- start 2024-07-31T15:13:05.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90ab334640 1 -- start start 2024-07-31T15:13:05.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90ab334640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a4102080 0x7f90a419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90ab334640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90a41029d0 0x7f90a419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90ab334640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90a410fc30 0x7f90a41a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90ab334640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41156c0 con 0x7f90a410fc30 2024-07-31T15:13:05.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90ab334640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f90a4115540 con 0x7f90a4102080 2024-07-31T15:13:05.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90ab334640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f90a4115840 con 0x7f90a41029d0 2024-07-31T15:13:05.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90a90d0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a4102080 0x7f90a419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90a98d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90a410fc30 0x7f90a41a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90a90d0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a4102080 0x7f90a419c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60170/0 (socket says 172.21.15.67:60170) 2024-07-31T15:13:05.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90a90d0640 1 -- 172.21.15.67:0/437636095 learned_addr learned my addr 172.21.15.67:0/437636095 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:05.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90a88cf640 1 --2- 172.21.15.67:0/437636095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90a41029d0 0x7f90a419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90a90d0640 1 -- 172.21.15.67:0/437636095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90a41029d0 msgr2=0x7f90a419c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:05.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90a90d0640 1 --2- 172.21.15.67:0/437636095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90a41029d0 0x7f90a419c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90a90d0640 1 -- 172.21.15.67:0/437636095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90a410fc30 msgr2=0x7f90a41a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:05.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90a90d0640 1 --2- 172.21.15.67:0/437636095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90a410fc30 0x7f90a41a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90a90d0640 1 -- 172.21.15.67:0/437636095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a41a2020 con 0x7f90a4102080 2024-07-31T15:13:05.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.156+0000 7f90a98d1640 1 --2- 172.21.15.67:0/437636095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90a410fc30 0x7f90a41a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:05.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.157+0000 7f90a90d0640 1 --2- 172.21.15.67:0/437636095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a4102080 0x7f90a419c0b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f90a000e9f0 tx=0x7f90a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:05.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.157+0000 7f909a7fc640 1 -- 172.21.15.67:0/437636095 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a000cd80 con 0x7f90a4102080 2024-07-31T15:13:05.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.157+0000 7f90ab334640 1 -- 172.21.15.67:0/437636095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a41a2310 con 0x7f90a4102080 2024-07-31T15:13:05.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.157+0000 7f909a7fc640 1 -- 172.21.15.67:0/437636095 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a00108e0 con 0x7f90a4102080 2024-07-31T15:13:05.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.157+0000 7f909a7fc640 1 -- 172.21.15.67:0/437636095 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0018980 con 0x7f90a4102080 2024-07-31T15:13:05.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.157+0000 7f90ab334640 1 -- 172.21.15.67:0/437636095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a41a8a70 con 0x7f90a4102080 2024-07-31T15:13:05.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.160+0000 7f909a7fc640 1 -- 172.21.15.67:0/437636095 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f90a0010430 con 0x7f90a4102080 2024-07-31T15:13:05.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.160+0000 7f90b1885740 1 Processor -- start 2024-07-31T15:13:05.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.160+0000 7f90b1885740 1 -- start start 2024-07-31T15:13:05.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.160+0000 7f90b1885740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3c1070 0x558dca766570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.161+0000 7f90b1885740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca3981b0 0x558dca766ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.161+0000 7f90b1885740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca766ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.161+0000 7f90b1885740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558dca6dcbb0 con 0x558dca3c1070 2024-07-31T15:13:05.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.161+0000 7f90b1885740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558dca6cc0b0 con 0x558dca3a0390 2024-07-31T15:13:05.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.161+0000 7f90b1885740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558dca6d00f0 con 0x558dca3981b0 2024-07-31T15:13:05.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.161+0000 7f90a98d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3c1070 0x558dca766570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.161+0000 7f90a90d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca3981b0 0x558dca766ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.161+0000 7f90a90d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca3981b0 0x558dca766ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35968/0 (socket says 172.21.15.67:35968) 2024-07-31T15:13:05.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.161+0000 7f90a90d0640 1 -- 172.21.15.67:0/3280041103 learned_addr learned my addr 172.21.15.67:0/3280041103 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:05.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.161+0000 7f90a88cf640 1 --2- 172.21.15.67:0/3280041103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca766ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.161+0000 7f90a88cf640 1 -- 172.21.15.67:0/3280041103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca3981b0 msgr2=0x558dca766ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:05.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.161+0000 7f90a88cf640 1 --2- 172.21.15.67:0/3280041103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca3981b0 0x558dca766ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.161+0000 7f90a88cf640 1 -- 172.21.15.67:0/3280041103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3c1070 msgr2=0x558dca766570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:05.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.161+0000 7f90a88cf640 1 --2- 172.21.15.67:0/3280041103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3c1070 0x558dca766570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.162+0000 7f90a88cf640 1 -- 172.21.15.67:0/3280041103 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dca672b90 con 0x558dca3a0390 2024-07-31T15:13:05.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.162+0000 7f909a7fc640 1 --2- 172.21.15.67:0/437636095 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90900b3d40 0x7f90900b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.162+0000 7f90a88cf640 1 --2- 172.21.15.67:0/3280041103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca766ff0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9094009230 tx=0x7f9094032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b727d4cc536af1ad server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:05.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.162+0000 7f90a98d1640 1 --2- 172.21.15.67:0/437636095 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90900b3d40 0x7f90900b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.162+0000 7f909a7fc640 1 -- 172.21.15.67:0/437636095 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f90a00e4680 con 0x7f90a4102080 2024-07-31T15:13:05.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.162+0000 7f908f7fe640 1 -- 172.21.15.67:0/3280041103 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094040070 con 0x558dca3a0390 2024-07-31T15:13:05.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.162+0000 7f908f7fe640 1 -- 172.21.15.67:0/3280041103 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9094004030 con 0x558dca3a0390 2024-07-31T15:13:05.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.162+0000 7f908f7fe640 1 -- 172.21.15.67:0/3280041103 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094035040 con 0x558dca3a0390 2024-07-31T15:13:05.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.163+0000 7f90b1885740 1 -- 172.21.15.67:0/3280041103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 msgr2=0x558dca766ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:05.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.163+0000 7f90b1885740 1 --2- 172.21.15.67:0/3280041103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca766ff0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9094009230 tx=0x7f9094032b90 comp rx=0 tx=0).stop 2024-07-31T15:13:05.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.163+0000 7f90a98d1640 1 --2- 172.21.15.67:0/437636095 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90900b3d40 0x7f90900b6250 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f909c009fd0 tx=0x7f909c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:05.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.163+0000 7f90b1885740 1 -- 172.21.15.67:0/3280041103 shutdown_connections 2024-07-31T15:13:05.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.163+0000 7f90b1885740 1 --2- 172.21.15.67:0/3280041103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca766ff0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.163+0000 7f90b1885740 1 --2- 172.21.15.67:0/3280041103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca3981b0 0x558dca766ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.163+0000 7f90b1885740 1 --2- 172.21.15.67:0/3280041103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3c1070 0x558dca766570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.163+0000 7f90b1885740 1 -- 172.21.15.67:0/3280041103 >> 172.21.15.67:0/3280041103 conn(0x558dca3b5460 msgr2=0x558dca764460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:05.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.164+0000 7f90b1885740 1 -- 172.21.15.67:0/3280041103 shutdown_connections 2024-07-31T15:13:05.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.164+0000 7f90b1885740 1 -- 172.21.15.67:0/3280041103 wait complete. 2024-07-31T15:13:05.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.164+0000 7f90b1885740 1 -- start start 2024-07-31T15:13:05.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.164+0000 7f90b1885740 1 Processor -- start 2024-07-31T15:13:05.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90b1885740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca3981b0 0x558dca772180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90b1885740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca772700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90b1885740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3c1070 0x558dca77dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90b1885740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558dca52f700 con 0x558dca3c1070 2024-07-31T15:13:05.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90b1885740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558dca66e9e0 con 0x558dca3a0390 2024-07-31T15:13:05.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90b1885740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558dca774cf0 con 0x558dca3981b0 2024-07-31T15:13:05.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90b1885740 10 client.0 _ll_register_callbacks cb 0x558dca6894f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:05.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90a88cf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca772700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90a90d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca3981b0 0x558dca772180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90a88cf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca772700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60186/0 (socket says 172.21.15.67:60186) 2024-07-31T15:13:05.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90a88cf640 1 -- 172.21.15.67:0/2482555346 learned_addr learned my addr 172.21.15.67:0/2482555346 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:05.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90a98d1640 1 --2- 172.21.15.67:0/2482555346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3c1070 0x558dca77dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90a90d0640 1 -- 172.21.15.67:0/2482555346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 msgr2=0x558dca772700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:05.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90a90d0640 1 --2- 172.21.15.67:0/2482555346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca772700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90a90d0640 1 -- 172.21.15.67:0/2482555346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3c1070 msgr2=0x558dca77dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:05.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90a90d0640 1 --2- 172.21.15.67:0/2482555346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3c1070 0x558dca77dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90a90d0640 1 -- 172.21.15.67:0/2482555346 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dca42c650 con 0x558dca3981b0 2024-07-31T15:13:05.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.165+0000 7f90a98d1640 1 --2- 172.21.15.67:0/2482555346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3c1070 0x558dca77dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:13:05.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.166+0000 7f90a90d0640 1 --2- 172.21.15.67:0/2482555346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca3981b0 0x558dca772180 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x558dca7651f0 tx=0x7f90a0010cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e06b669bd4a7019 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:05.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.166+0000 7f908cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:13:05.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.166+0000 7f908cff9640 1 -- 172.21.15.67:0/2482555346 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a00aa740 con 0x558dca3981b0 2024-07-31T15:13:05.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.166+0000 7f90b1885740 1 -- 172.21.15.67:0/2482555346 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9094004030 con 0x558dca3981b0 2024-07-31T15:13:05.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.166+0000 7f908cff9640 1 -- 172.21.15.67:0/2482555346 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f90a00e42b0 con 0x558dca3981b0 2024-07-31T15:13:05.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.166+0000 7f908cff9640 1 -- 172.21.15.67:0/2482555346 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f90a0010720 con 0x558dca3981b0 2024-07-31T15:13:05.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.166+0000 7f90b1885740 10 client.29455 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:05.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.166+0000 7f90b1885740 1 -- 172.21.15.67:0/2482555346 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x558dca780e00 con 0x558dca3981b0 2024-07-31T15:13:05.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.166+0000 7f908cff9640 1 -- 172.21.15.67:0/2482555346 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9094004030 con 0x558dca3981b0 2024-07-31T15:13:05.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.167+0000 7f908cff9640 1 -- 172.21.15.67:0/2482555346 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f90a00e7ce0 con 0x558dca3981b0 2024-07-31T15:13:05.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.167+0000 7f90b1885740 10 client.29455 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:05.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.167+0000 7f90b1885740 10 client.29455 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:13:05.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.167+0000 7f90b1885740 20 client.29455 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:05.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.167+0000 7f90b1885740 1 --2- 172.21.15.67:0/2482555346 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x558dca37f9f0 0x558dca7842d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.167+0000 7f90b1885740 4 client.29455 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:13:05.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.167+0000 7f90b1885740 1 -- 172.21.15.67:0/2482555346 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dca6deb00 con 0x558dca37f9f0 2024-07-31T15:13:05.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.167+0000 7f90a88cf640 1 --2- 172.21.15.67:0/2482555346 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x558dca37f9f0 0x558dca7842d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.167+0000 7f908cff9640 10 client.29455 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:13:05.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.167+0000 7f90a88cf640 1 --2- 172.21.15.67:0/2482555346 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x558dca37f9f0 0x558dca7842d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2291fdc0b30d81e7 server_cookie=5a1242dfe18157bd in_seq=0 out_seq=0 2024-07-31T15:13:05.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.168+0000 7f908cff9640 1 -- 172.21.15.67:0/2482555346 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558dca6deb00 con 0x558dca37f9f0 2024-07-31T15:13:05.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.168+0000 7f908cff9640 10 client.29455 handle_command_reply: tid=0 2024-07-31T15:13:05.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.168+0000 7f90b1885740 1 client.29455 shutdown 2024-07-31T15:13:05.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.168+0000 7f90b1885740 1 -- 172.21.15.67:0/2482555346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca3981b0 msgr2=0x558dca772180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:05.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.168+0000 7f90b1885740 1 --2- 172.21.15.67:0/2482555346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca3981b0 0x558dca772180 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x558dca7651f0 tx=0x7f90a0010cc0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.168+0000 7f90b1885740 1 -- 172.21.15.67:0/2482555346 shutdown_connections 2024-07-31T15:13:05.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.168+0000 7f90b1885740 1 --2- 172.21.15.67:0/2482555346 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x558dca37f9f0 0x558dca7842d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.168+0000 7f90b1885740 1 --2- 172.21.15.67:0/2482555346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3c1070 0x558dca77dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.168+0000 7f90b1885740 1 --2- 172.21.15.67:0/2482555346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca772700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.168+0000 7f90b1885740 1 --2- 172.21.15.67:0/2482555346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca3981b0 0x558dca772180 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.168+0000 7f908cff9640 0 client.29455 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:13:05.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.168+0000 7f90b1885740 1 -- 172.21.15.67:0/2482555346 >> 172.21.15.67:0/2482555346 conn(0x558dca3b5460 msgr2=0x558dca769750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:05.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.169+0000 7f90b1885740 1 -- 172.21.15.67:0/2482555346 shutdown_connections 2024-07-31T15:13:05.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.169+0000 7f90b1885740 1 -- 172.21.15.67:0/2482555346 wait complete. 2024-07-31T15:13:05.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.169+0000 7f90b1885740 20 client.29455 trim_cache size 0 max 16384 2024-07-31T15:13:05.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.184+0000 7f90b1885740 1 Processor -- start 2024-07-31T15:13:05.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.184+0000 7f90b1885740 1 -- start start 2024-07-31T15:13:05.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90b1885740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca37f9f0 0x558dca77ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90b1885740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3981b0 0x558dca763350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90b1885740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca76b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90b1885740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41b2970 con 0x558dca3981b0 2024-07-31T15:13:05.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90b1885740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41b44b0 con 0x558dca3a0390 2024-07-31T15:13:05.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90b1885740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41b3040 con 0x558dca37f9f0 2024-07-31T15:13:05.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90a90d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca37f9f0 0x558dca77ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90a88cf640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3981b0 0x558dca763350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90a98d1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca76b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90a90d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca37f9f0 0x558dca77ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:35992/0 (socket says 172.21.15.67:35992) 2024-07-31T15:13:05.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90a90d0640 1 -- 172.21.15.67:0/1772030904 learned_addr learned my addr 172.21.15.67:0/1772030904 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:05.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90a98d1640 1 -- 172.21.15.67:0/1772030904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca37f9f0 msgr2=0x558dca77ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:05.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90a98d1640 1 --2- 172.21.15.67:0/1772030904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca37f9f0 0x558dca77ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90a98d1640 1 -- 172.21.15.67:0/1772030904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3981b0 msgr2=0x558dca763350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:05.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90a98d1640 1 --2- 172.21.15.67:0/1772030904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3981b0 0x558dca763350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90a98d1640 1 -- 172.21.15.67:0/1772030904 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dca770480 con 0x558dca3a0390 2024-07-31T15:13:05.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90a90d0640 1 --2- 172.21.15.67:0/1772030904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca37f9f0 0x558dca77ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.185+0000 7f90a98d1640 1 --2- 172.21.15.67:0/1772030904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca76b650 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f909c0124c0 tx=0x7f909c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d22bb21b1ccdd5e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.186+0000 7f908effd640 1 -- 172.21.15.67:0/1772030904 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c003e00 con 0x558dca3a0390 2024-07-31T15:13:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.186+0000 7f908effd640 1 -- 172.21.15.67:0/1772030904 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f909c004050 con 0x558dca3a0390 2024-07-31T15:13:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.186+0000 7f908effd640 1 -- 172.21.15.67:0/1772030904 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c005310 con 0x558dca3a0390 2024-07-31T15:13:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.187+0000 7f90b1885740 1 -- 172.21.15.67:0/1772030904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 msgr2=0x558dca76b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.187+0000 7f90b1885740 1 --2- 172.21.15.67:0/1772030904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca76b650 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f909c0124c0 tx=0x7f909c017000 comp rx=0 tx=0).stop 2024-07-31T15:13:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.187+0000 7f90b1885740 1 -- 172.21.15.67:0/1772030904 shutdown_connections 2024-07-31T15:13:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.187+0000 7f90b1885740 1 --2- 172.21.15.67:0/1772030904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3a0390 0x558dca76b650 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.187+0000 7f90b1885740 1 --2- 172.21.15.67:0/1772030904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3981b0 0x558dca763350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.187+0000 7f90b1885740 1 --2- 172.21.15.67:0/1772030904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca37f9f0 0x558dca77ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.187+0000 7f90b1885740 1 -- 172.21.15.67:0/1772030904 >> 172.21.15.67:0/1772030904 conn(0x558dca3b5460 msgr2=0x558dca762c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.187+0000 7f90b1885740 1 -- 172.21.15.67:0/1772030904 shutdown_connections 2024-07-31T15:13:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.187+0000 7f90b1885740 1 -- 172.21.15.67:0/1772030904 wait complete. 2024-07-31T15:13:05.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.187+0000 7f90b1885740 1 -- start start 2024-07-31T15:13:05.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.188+0000 7f90b1885740 1 Processor -- start 2024-07-31T15:13:05.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.188+0000 7f90b1885740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca37f9f0 0x558dca779f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.188+0000 7f90b1885740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3981b0 0x558dca77a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.188+0000 7f90b1885740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3a0390 0x558dca774f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.188+0000 7f90b1885740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41b3de0 con 0x558dca3a0390 2024-07-31T15:13:05.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.188+0000 7f90b1885740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41b3710 con 0x558dca3981b0 2024-07-31T15:13:05.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.188+0000 7f90b1885740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41b0970 con 0x558dca37f9f0 2024-07-31T15:13:05.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.188+0000 7f90b1885740 10 client.0 _ll_register_callbacks cb 0x7f90a0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.188+0000 7f90a88cf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3981b0 0x558dca77a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.188+0000 7f90a88cf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3981b0 0x558dca77a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60202/0 (socket says 172.21.15.67:60202) 2024-07-31T15:13:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.188+0000 7f90a98d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3a0390 0x558dca774f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.188+0000 7f90a88cf640 1 -- 172.21.15.67:0/3436881742 learned_addr learned my addr 172.21.15.67:0/3436881742 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.188+0000 7f90a90d0640 1 --2- 172.21.15.67:0/3436881742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca37f9f0 0x558dca779f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.189+0000 7f90a88cf640 1 -- 172.21.15.67:0/3436881742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca37f9f0 msgr2=0x558dca779f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.189+0000 7f90a88cf640 1 --2- 172.21.15.67:0/3436881742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca37f9f0 0x558dca779f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.189+0000 7f90a88cf640 1 -- 172.21.15.67:0/3436881742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3a0390 msgr2=0x558dca774f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.189+0000 7f90a88cf640 1 --2- 172.21.15.67:0/3436881742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3a0390 0x558dca774f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.189+0000 7f90a88cf640 1 -- 172.21.15.67:0/3436881742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dca77aa20 con 0x558dca3981b0 2024-07-31T15:13:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.189+0000 7f90a90d0640 1 --2- 172.21.15.67:0/3436881742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca37f9f0 0x558dca779f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.189+0000 7f90a88cf640 1 --2- 172.21.15.67:0/3436881742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3981b0 0x558dca77a490 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9094005150 tx=0x7f9094009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5abc8c533650ae4f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:05.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.189+0000 7f908cff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:13:05.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.189+0000 7f908cff9640 1 -- 172.21.15.67:0/3436881742 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909405c060 con 0x558dca3981b0 2024-07-31T15:13:05.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.189+0000 7f908cff9640 1 -- 172.21.15.67:0/3436881742 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f9094004470 con 0x558dca3981b0 2024-07-31T15:13:05.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.189+0000 7f90b1885740 1 -- 172.21.15.67:0/3436881742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f909c004050 con 0x558dca3981b0 2024-07-31T15:13:05.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.190+0000 7f908cff9640 1 -- 172.21.15.67:0/3436881742 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f9094063050 con 0x558dca3981b0 2024-07-31T15:13:05.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.190+0000 7f90b1885740 10 client.29615 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:05.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.190+0000 7f90b1885740 1 -- 172.21.15.67:0/3436881742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x558dca7ecf20 con 0x558dca3981b0 2024-07-31T15:13:05.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.190+0000 7f908cff9640 1 -- 172.21.15.67:0/3436881742 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f909c004050 con 0x558dca3981b0 2024-07-31T15:13:05.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.190+0000 7f908cff9640 1 -- 172.21.15.67:0/3436881742 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f9094031630 con 0x558dca3981b0 2024-07-31T15:13:05.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.190+0000 7f90b1885740 10 client.29615 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:05.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.190+0000 7f90b1885740 10 client.29615 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:13:05.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.190+0000 7f90b1885740 20 client.29615 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:05.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.190+0000 7f90b1885740 1 --2- 172.21.15.67:0/3436881742 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x558dca3c1070 0x558dca7ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:05.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.190+0000 7f90b1885740 4 client.29615 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:13:05.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.190+0000 7f90b1885740 1 -- 172.21.15.67:0/3436881742 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558dca6da5a0 con 0x558dca3c1070 2024-07-31T15:13:05.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.191+0000 7f90a90d0640 1 --2- 172.21.15.67:0/3436881742 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x558dca3c1070 0x558dca7ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:05.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.191+0000 7f908cff9640 10 client.29615 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:13:05.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.191+0000 7f90a90d0640 1 --2- 172.21.15.67:0/3436881742 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x558dca3c1070 0x558dca7ef2c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2811e2d4bc71802 server_cookie=32077b774a588da9 in_seq=0 out_seq=0 2024-07-31T15:13:05.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.191+0000 7f908cff9640 1 -- 172.21.15.67:0/3436881742 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558dca6da5a0 con 0x558dca3c1070 2024-07-31T15:13:05.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.191+0000 7f908cff9640 10 client.29615 handle_command_reply: tid=0 2024-07-31T15:13:05.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.191+0000 7f90b1885740 1 client.29615 shutdown 2024-07-31T15:13:05.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.192+0000 7f90b1885740 1 -- 172.21.15.67:0/3436881742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3981b0 msgr2=0x558dca77a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:05.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.192+0000 7f90b1885740 1 --2- 172.21.15.67:0/3436881742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3981b0 0x558dca77a490 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9094005150 tx=0x7f9094009260 comp rx=0 tx=0).stop 2024-07-31T15:13:05.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.192+0000 7f90b1885740 1 -- 172.21.15.67:0/3436881742 shutdown_connections 2024-07-31T15:13:05.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.192+0000 7f90b1885740 1 --2- 172.21.15.67:0/3436881742 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x558dca3c1070 0x558dca7ef2c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.192+0000 7f90b1885740 1 --2- 172.21.15.67:0/3436881742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558dca3a0390 0x558dca774f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.192+0000 7f90b1885740 1 --2- 172.21.15.67:0/3436881742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dca3981b0 0x558dca77a490 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.192+0000 7f908cff9640 0 client.29615 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:13:05.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.192+0000 7f90b1885740 1 --2- 172.21.15.67:0/3436881742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dca37f9f0 0x558dca779f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.192+0000 7f90b1885740 1 -- 172.21.15.67:0/3436881742 >> 172.21.15.67:0/3436881742 conn(0x558dca3b5460 msgr2=0x558dca7675a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:05.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.192+0000 7f90b1885740 1 -- 172.21.15.67:0/3436881742 shutdown_connections 2024-07-31T15:13:05.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.192+0000 7f90b1885740 1 -- 172.21.15.67:0/3436881742 wait complete. 2024-07-31T15:13:05.197 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:13:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.193+0000 7f90b1885740 20 client.29615 trim_cache size 0 max 16384 2024-07-31T15:13:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.193+0000 7f90ab334640 1 -- 172.21.15.67:0/437636095 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90900b3d40 msgr2=0x7f90900b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.193+0000 7f90ab334640 1 --2- 172.21.15.67:0/437636095 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90900b3d40 0x7f90900b6250 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f909c009fd0 tx=0x7f909c009500 comp rx=0 tx=0).stop 2024-07-31T15:13:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.193+0000 7f90ab334640 1 -- 172.21.15.67:0/437636095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a4102080 msgr2=0x7f90a419c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.193+0000 7f90ab334640 1 --2- 172.21.15.67:0/437636095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a4102080 0x7f90a419c0b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f90a000e9f0 tx=0x7f90a000eeb0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.193+0000 7f90ab334640 1 -- 172.21.15.67:0/437636095 shutdown_connections 2024-07-31T15:13:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.193+0000 7f90ab334640 1 --2- 172.21.15.67:0/437636095 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f90900b3d40 0x7f90900b6250 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.193+0000 7f90ab334640 1 --2- 172.21.15.67:0/437636095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90a410fc30 0x7f90a41a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.193+0000 7f90ab334640 1 --2- 172.21.15.67:0/437636095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90a41029d0 0x7f90a419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.193+0000 7f90ab334640 1 --2- 172.21.15.67:0/437636095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a4102080 0x7f90a419c0b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.193+0000 7f90ab334640 1 -- 172.21.15.67:0/437636095 >> 172.21.15.67:0/437636095 conn(0x7f90a4079160 msgr2=0x7f90a4109740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:05.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.194+0000 7f90ab334640 1 -- 172.21.15.67:0/437636095 shutdown_connections 2024-07-31T15:13:05.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:05.194+0000 7f90ab334640 1 -- 172.21.15.67:0/437636095 wait complete. 2024-07-31T15:13:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:13:05.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:05 smithi067 ceph-mon[32750]: pgmap v1102: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 475 KiB/s wr, 168 op/s 2024-07-31T15:13:05.694 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:05 smithi073 ceph-mon[36714]: pgmap v1102: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 475 KiB/s wr, 168 op/s 2024-07-31T15:13:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:05 smithi077 ceph-mon[37327]: pgmap v1102: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 475 KiB/s wr, 168 op/s 2024-07-31T15:13:05.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T15:13:06.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:06 smithi067 ceph-mon[32750]: osdmap e133: 12 total, 12 up, 12 in 2024-07-31T15:13:06.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:06 smithi077 ceph-mon[37327]: osdmap e133: 12 total, 12 up, 12 in 2024-07-31T15:13:06.878 INFO:tasks.workunit.client.0.smithi067.stdout: AR vmlinux.a 2024-07-31T15:13:06.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:06 smithi073 ceph-mon[36714]: osdmap e133: 12 total, 12 up, 12 in 2024-07-31T15:13:07.453 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:07 smithi067 ceph-mon[32750]: pgmap v1104: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 509 KiB/s wr, 251 op/s 2024-07-31T15:13:07.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:07 smithi077 ceph-mon[37327]: pgmap v1104: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 509 KiB/s wr, 251 op/s 2024-07-31T15:13:07.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:07 smithi073 ceph-mon[36714]: pgmap v1104: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 509 KiB/s wr, 251 op/s 2024-07-31T15:13:08.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.063+0000 7fa27b7fe640 1 -- 172.21.15.67:0/2278732525 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27402f340 con 0x7fa27c10ce80 2024-07-31T15:13:08.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.064+0000 7fa28190a640 1 -- 172.21.15.67:0/2278732525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa27c10ce80 msgr2=0x7fa27c1168f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:08.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.064+0000 7fa28190a640 1 --2- 172.21.15.67:0/2278732525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa27c10ce80 0x7fa27c1168f0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fa274009ca0 tx=0x7fa27402a8f0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.064+0000 7fa28190a640 1 -- 172.21.15.67:0/2278732525 shutdown_connections 2024-07-31T15:13:08.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.064+0000 7fa28190a640 1 --2- 172.21.15.67:0/2278732525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa27c10ce80 0x7fa27c1168f0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.064+0000 7fa28190a640 1 --2- 172.21.15.67:0/2278732525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c10c530 0x7fa27c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.064+0000 7fa28190a640 1 --2- 172.21.15.67:0/2278732525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa27c10a080 0x7fa27c10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.064+0000 7fa28190a640 1 -- 172.21.15.67:0/2278732525 >> 172.21.15.67:0/2278732525 conn(0x7fa27c06e8c0 msgr2=0x7fa27c06ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:08.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.066+0000 7fa28190a640 1 -- 172.21.15.67:0/2278732525 shutdown_connections 2024-07-31T15:13:08.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.066+0000 7fa28190a640 1 -- 172.21.15.67:0/2278732525 wait complete. 2024-07-31T15:13:08.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.067+0000 7fa28190a640 1 Processor -- start 2024-07-31T15:13:08.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.067+0000 7fa28190a640 1 -- start start 2024-07-31T15:13:08.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.067+0000 7fa28190a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa27c10a080 0x7fa27c1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.067+0000 7fa28190a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa27c10c530 0x7fa27c1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.067+0000 7fa28190a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c10ce80 0x7fa27c1a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.067+0000 7fa28190a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c119a60 con 0x7fa27c10a080 2024-07-31T15:13:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.067+0000 7fa28190a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1198e0 con 0x7fa27c10c530 2024-07-31T15:13:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.067+0000 7fa28190a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c119be0 con 0x7fa27c10ce80 2024-07-31T15:13:08.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.067+0000 7fa27bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa27c10c530 0x7fa27c1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.068+0000 7fa27bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa27c10c530 0x7fa27c1a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60218/0 (socket says 172.21.15.67:60218) 2024-07-31T15:13:08.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.068+0000 7fa27bfff640 1 -- 172.21.15.67:0/987175375 learned_addr learned my addr 172.21.15.67:0/987175375 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:08.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.068+0000 7fa281109640 1 --2- 172.21.15.67:0/987175375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c10ce80 0x7fa27c1a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.068+0000 7fa280908640 1 --2- 172.21.15.67:0/987175375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa27c10a080 0x7fa27c1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.068+0000 7fa27bfff640 1 -- 172.21.15.67:0/987175375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c10ce80 msgr2=0x7fa27c1a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.068+0000 7fa27bfff640 1 --2- 172.21.15.67:0/987175375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c10ce80 0x7fa27c1a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.068+0000 7fa27bfff640 1 -- 172.21.15.67:0/987175375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa27c10a080 msgr2=0x7fa27c1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.068+0000 7fa27bfff640 1 --2- 172.21.15.67:0/987175375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa27c10a080 0x7fa27c1a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.068+0000 7fa27bfff640 1 -- 172.21.15.67:0/987175375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27c1a6350 con 0x7fa27c10c530 2024-07-31T15:13:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.069+0000 7fa27bfff640 1 --2- 172.21.15.67:0/987175375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa27c10c530 0x7fa27c1a0980 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa27000b450 tx=0x7fa27000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.069+0000 7fa279ffb640 1 -- 172.21.15.67:0/987175375 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27001d070 con 0x7fa27c10c530 2024-07-31T15:13:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.069+0000 7fa28190a640 1 -- 172.21.15.67:0/987175375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27c1a6640 con 0x7fa27c10c530 2024-07-31T15:13:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.069+0000 7fa28190a640 1 -- 172.21.15.67:0/987175375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27c07b050 con 0x7fa27c10c530 2024-07-31T15:13:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.070+0000 7fa279ffb640 1 -- 172.21.15.67:0/987175375 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa270018440 con 0x7fa27c10c530 2024-07-31T15:13:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.070+0000 7fa279ffb640 1 -- 172.21.15.67:0/987175375 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2700165a0 con 0x7fa27c10c530 2024-07-31T15:13:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.074+0000 7fa2890bd740 1 Processor -- start 2024-07-31T15:13:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.074+0000 7fa2890bd740 1 -- start start 2024-07-31T15:13:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.075+0000 7fa2890bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e5886b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.075+0000 7fa2890bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 0x555e5886b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.075+0000 7fa2890bd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 0x555e5886bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.075+0000 7fa2890bd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555e587d77b0 con 0x555e5849bb60 2024-07-31T15:13:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.075+0000 7fa2890bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555e587c6cb0 con 0x555e58493980 2024-07-31T15:13:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.075+0000 7fa2890bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e587cacf0 con 0x555e584bc840 2024-07-31T15:13:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.076+0000 7fa280908640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 0x555e5886b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.076+0000 7fa280908640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 0x555e5886b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60234/0 (socket says 172.21.15.67:60234) 2024-07-31T15:13:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.076+0000 7fa280908640 1 -- 172.21.15.67:0/420725159 learned_addr learned my addr 172.21.15.67:0/420725159 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.077+0000 7fa281109640 1 --2- 172.21.15.67:0/420725159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e5886b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.076+0000 7fa279ffb640 1 -- 172.21.15.67:0/987175375 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fa270020070 con 0x7fa27c10c530 2024-07-31T15:13:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.077+0000 7fa27bfff640 1 --2- 172.21.15.67:0/420725159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 0x555e5886bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.078+0000 7fa279ffb640 1 --2- 172.21.15.67:0/987175375 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa26c0b3df0 0x7fa26c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.078+0000 7fa279ffb640 1 -- 172.21.15.67:0/987175375 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7fa2700e2fc0 con 0x7fa27c10c530 2024-07-31T15:13:08.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.079+0000 7fa281109640 1 -- 172.21.15.67:0/420725159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 msgr2=0x555e5886b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:08.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.079+0000 7fa281109640 1 --2- 172.21.15.67:0/420725159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 0x555e5886b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.079+0000 7fa281109640 1 -- 172.21.15.67:0/420725159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 msgr2=0x555e5886bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:08.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.079+0000 7fa281109640 1 --2- 172.21.15.67:0/420725159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 0x555e5886bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.079+0000 7fa281109640 1 -- 172.21.15.67:0/420725159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5875c0b0 con 0x555e584bc840 2024-07-31T15:13:08.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.080+0000 7fa281109640 1 --2- 172.21.15.67:0/420725159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e5886b270 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa274006010 tx=0x7fa274009940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8edb76699934202 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:08.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.080+0000 7fa281109640 1 --2- 172.21.15.67:0/987175375 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa26c0b3df0 0x7fa26c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.080+0000 7fa266ffd640 1 -- 172.21.15.67:0/420725159 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27404d070 con 0x555e584bc840 2024-07-31T15:13:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.081+0000 7fa266ffd640 1 -- 172.21.15.67:0/420725159 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa274036040 con 0x555e584bc840 2024-07-31T15:13:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.082+0000 7fa2890bd740 1 -- 172.21.15.67:0/420725159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 msgr2=0x555e5886b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.082+0000 7fa2890bd740 1 --2- 172.21.15.67:0/420725159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e5886b270 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa274006010 tx=0x7fa274009940 comp rx=0 tx=0).stop 2024-07-31T15:13:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.082+0000 7fa281109640 1 --2- 172.21.15.67:0/987175375 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa26c0b3df0 0x7fa26c0b6300 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa27402ae60 tx=0x7fa274004940 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.086+0000 7fa2890bd740 1 -- 172.21.15.67:0/420725159 shutdown_connections 2024-07-31T15:13:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.086+0000 7fa2890bd740 1 --2- 172.21.15.67:0/420725159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 0x555e5886bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.086+0000 7fa2890bd740 1 --2- 172.21.15.67:0/420725159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 0x555e5886b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.086+0000 7fa2890bd740 1 --2- 172.21.15.67:0/420725159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e5886b270 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.086+0000 7fa2890bd740 1 -- 172.21.15.67:0/420725159 >> 172.21.15.67:0/420725159 conn(0x555e584b0990 msgr2=0x555e58869160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.095+0000 7fa2890bd740 1 -- 172.21.15.67:0/420725159 shutdown_connections 2024-07-31T15:13:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.102+0000 7fa2890bd740 1 -- 172.21.15.67:0/420725159 wait complete. 2024-07-31T15:13:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.102+0000 7fa2890bd740 1 -- start start 2024-07-31T15:13:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.102+0000 7fa2890bd740 1 Processor -- start 2024-07-31T15:13:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.104+0000 7fa2890bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 0x555e5886fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.104+0000 7fa2890bd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 0x555e58870370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.104+0000 7fa2890bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e58873e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.104+0000 7fa2890bd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555e5862a7a0 con 0x555e5849bb60 2024-07-31T15:13:08.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.104+0000 7fa2890bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555e58769320 con 0x555e58493980 2024-07-31T15:13:08.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.104+0000 7fa2890bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e58868590 con 0x555e584bc840 2024-07-31T15:13:08.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.104+0000 7fa2890bd740 10 client.0 _ll_register_callbacks cb 0x555e58773140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:08.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.105+0000 7fa27bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 0x555e58870370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.105+0000 7fa27bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 0x555e58870370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53370/0 (socket says 172.21.15.67:53370) 2024-07-31T15:13:08.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.105+0000 7fa27bfff640 1 -- 172.21.15.67:0/3363273492 learned_addr learned my addr 172.21.15.67:0/3363273492 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:08.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.105+0000 7fa281109640 1 --2- 172.21.15.67:0/3363273492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e58873e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.106+0000 7fa281109640 1 -- 172.21.15.67:0/3363273492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 msgr2=0x555e5886fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.106+0000 7fa281109640 1 --2- 172.21.15.67:0/3363273492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 0x555e5886fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.106+0000 7fa281109640 1 -- 172.21.15.67:0/3363273492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 msgr2=0x555e58870370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.106+0000 7fa281109640 1 --2- 172.21.15.67:0/3363273492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 0x555e58870370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.106+0000 7fa281109640 1 -- 172.21.15.67:0/3363273492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e5876d4b0 con 0x555e584bc840 2024-07-31T15:13:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.106+0000 7fa281109640 1 --2- 172.21.15.67:0/3363273492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e58873e00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa274009bf0 tx=0x7fa274041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82e4e389769b740e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.110+0000 7fa25bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:13:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.110+0000 7fa25bfff640 1 -- 172.21.15.67:0/3363273492 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27404d040 con 0x555e584bc840 2024-07-31T15:13:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.110+0000 7fa2890bd740 1 -- 172.21.15.67:0/3363273492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa274036040 con 0x555e584bc840 2024-07-31T15:13:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.111+0000 7fa25bfff640 1 -- 172.21.15.67:0/3363273492 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7fa274031480 con 0x555e584bc840 2024-07-31T15:13:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.114+0000 7fa25bfff640 1 -- 172.21.15.67:0/3363273492 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fa274066050 con 0x555e584bc840 2024-07-31T15:13:08.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.123+0000 7fa2890bd740 10 client.29470 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:08.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.123+0000 7fa2890bd740 1 -- 172.21.15.67:0/3363273492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x555e58536b90 con 0x555e584bc840 2024-07-31T15:13:08.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.125+0000 7fa25bfff640 1 -- 172.21.15.67:0/3363273492 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa274036040 con 0x555e584bc840 2024-07-31T15:13:08.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.125+0000 7fa25bfff640 1 -- 172.21.15.67:0/3363273492 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fa274043050 con 0x555e584bc840 2024-07-31T15:13:08.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.125+0000 7fa2890bd740 10 client.29470 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:08.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.125+0000 7fa2890bd740 10 client.29470 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:13:08.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.125+0000 7fa2890bd740 20 client.29470 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:08.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.125+0000 7fa2890bd740 1 --2- 172.21.15.67:0/3363273492 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e5847b1c0 0x555e58880350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.125+0000 7fa2890bd740 4 client.29470 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:13:08.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.125+0000 7fa2890bd740 1 -- 172.21.15.67:0/3363273492 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e587d9700 con 0x555e5847b1c0 2024-07-31T15:13:08.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.128+0000 7fa280908640 1 --2- 172.21.15.67:0/3363273492 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e5847b1c0 0x555e58880350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.148 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:13:08.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.130+0000 7fa280908640 1 --2- 172.21.15.67:0/3363273492 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e5847b1c0 0x555e58880350 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5c7f0b4bbfaa8ea server_cookie=6ba39ad2d58670a6 in_seq=0 out_seq=0 2024-07-31T15:13:08.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.130+0000 7fa25bfff640 10 client.29470 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:08.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.134+0000 7fa25bfff640 1 -- 172.21.15.67:0/3363273492 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e587d9700 con 0x555e5847b1c0 2024-07-31T15:13:08.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.134+0000 7fa25bfff640 10 client.29470 handle_command_reply: tid=0 2024-07-31T15:13:08.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.134+0000 7fa2890bd740 1 client.29470 shutdown 2024-07-31T15:13:08.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.137+0000 7fa2890bd740 1 -- 172.21.15.67:0/3363273492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 msgr2=0x555e58873e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:08.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.137+0000 7fa2890bd740 1 --2- 172.21.15.67:0/3363273492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e58873e00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa274009bf0 tx=0x7fa274041000 comp rx=0 tx=0).stop 2024-07-31T15:13:08.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.138+0000 7fa2890bd740 1 -- 172.21.15.67:0/3363273492 shutdown_connections 2024-07-31T15:13:08.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.138+0000 7fa2890bd740 1 --2- 172.21.15.67:0/3363273492 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e5847b1c0 0x555e58880350 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.138+0000 7fa2890bd740 1 --2- 172.21.15.67:0/3363273492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e58873e00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.138+0000 7fa2890bd740 1 --2- 172.21.15.67:0/3363273492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 0x555e58870370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.138+0000 7fa2890bd740 1 --2- 172.21.15.67:0/3363273492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 0x555e5886fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.138+0000 7fa2890bd740 1 -- 172.21.15.67:0/3363273492 >> 172.21.15.67:0/3363273492 conn(0x555e584b0990 msgr2=0x555e5886cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:08.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.138+0000 7fa25bfff640 0 client.29470 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:08.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.139+0000 7fa2890bd740 1 -- 172.21.15.67:0/3363273492 shutdown_connections 2024-07-31T15:13:08.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.140+0000 7fa2890bd740 1 -- 172.21.15.67:0/3363273492 wait complete. 2024-07-31T15:13:08.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.140+0000 7fa2890bd740 20 client.29470 trim_cache size 0 max 16384 2024-07-31T15:13:08.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.190+0000 7fa2890bd740 1 Processor -- start 2024-07-31T15:13:08.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.192+0000 7fa2890bd740 1 -- start start 2024-07-31T15:13:08.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.193+0000 7fa2890bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 0x555e58863b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.193+0000 7fa2890bd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 0x555e58861f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.193+0000 7fa2890bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e58862470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.193+0000 7fa2890bd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa2600100f0 con 0x555e5849bb60 2024-07-31T15:13:08.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.193+0000 7fa2890bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa260011c30 con 0x555e58493980 2024-07-31T15:13:08.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.193+0000 7fa2890bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2600107c0 con 0x555e584bc840 2024-07-31T15:13:08.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.193+0000 7fa280908640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 0x555e58863b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.193+0000 7fa280908640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 0x555e58863b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60254/0 (socket says 172.21.15.67:60254) 2024-07-31T15:13:08.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.193+0000 7fa280908640 1 -- 172.21.15.67:0/565727196 learned_addr learned my addr 172.21.15.67:0/565727196 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:08.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.193+0000 7fa281109640 1 --2- 172.21.15.67:0/565727196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e58862470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.194+0000 7fa281109640 1 -- 172.21.15.67:0/565727196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 msgr2=0x555e58863b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:08.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.194+0000 7fa281109640 1 --2- 172.21.15.67:0/565727196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 0x555e58863b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.194+0000 7fa281109640 1 -- 172.21.15.67:0/565727196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 msgr2=0x555e58861f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:08.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.194+0000 7fa281109640 1 --2- 172.21.15.67:0/565727196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 0x555e58861f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.194+0000 7fa281109640 1 -- 172.21.15.67:0/565727196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5886c0b0 con 0x555e584bc840 2024-07-31T15:13:08.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.197+0000 7fa281109640 1 --2- 172.21.15.67:0/565727196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e58862470 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa274006010 tx=0x7fa27402a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a8eec5fb0478bc6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:08.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.198+0000 7fa2667fc640 1 -- 172.21.15.67:0/565727196 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27404d070 con 0x555e584bc840 2024-07-31T15:13:08.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.198+0000 7fa2667fc640 1 -- 172.21.15.67:0/565727196 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa274036070 con 0x555e584bc840 2024-07-31T15:13:08.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.198+0000 7fa2667fc640 1 -- 172.21.15.67:0/565727196 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274030c10 con 0x555e584bc840 2024-07-31T15:13:08.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.199+0000 7fa2890bd740 1 -- 172.21.15.67:0/565727196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 msgr2=0x555e58862470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:08.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.199+0000 7fa2890bd740 1 --2- 172.21.15.67:0/565727196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e58862470 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa274006010 tx=0x7fa27402a8f0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.203+0000 7fa2890bd740 1 -- 172.21.15.67:0/565727196 shutdown_connections 2024-07-31T15:13:08.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.203+0000 7fa2890bd740 1 --2- 172.21.15.67:0/565727196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e584bc840 0x555e58862470 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.203+0000 7fa2890bd740 1 --2- 172.21.15.67:0/565727196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e5849bb60 0x555e58861f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.203+0000 7fa2890bd740 1 --2- 172.21.15.67:0/565727196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e58493980 0x555e58863b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.203+0000 7fa2890bd740 1 -- 172.21.15.67:0/565727196 >> 172.21.15.67:0/565727196 conn(0x555e584b0990 msgr2=0x555e5886ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:08.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.205+0000 7fa2890bd740 1 -- 172.21.15.67:0/565727196 shutdown_connections 2024-07-31T15:13:08.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.207+0000 7fa2890bd740 1 -- 172.21.15.67:0/565727196 wait complete. 2024-07-31T15:13:08.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.207+0000 7fa2890bd740 1 -- start start 2024-07-31T15:13:08.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.207+0000 7fa2890bd740 1 Processor -- start 2024-07-31T15:13:08.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.208+0000 7fa2890bd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e58493980 0x555e5887a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.208+0000 7fa2890bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e5849bb60 0x555e5887abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.208+0000 7fa2890bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e584bc840 0x555e58875c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.208+0000 7fa2890bd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa260011560 con 0x555e58493980 2024-07-31T15:13:08.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.208+0000 7fa2890bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa260010e90 con 0x555e584bc840 2024-07-31T15:13:08.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.208+0000 7fa2890bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa26000e0f0 con 0x555e5849bb60 2024-07-31T15:13:08.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.209+0000 7fa2890bd740 10 client.0 _ll_register_callbacks cb 0x7fa274009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:08.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.209+0000 7fa280908640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e58493980 0x555e5887a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.209+0000 7fa280908640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e58493980 0x555e5887a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53386/0 (socket says 172.21.15.67:53386) 2024-07-31T15:13:08.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.209+0000 7fa280908640 1 -- 172.21.15.67:0/3595834195 learned_addr learned my addr 172.21.15.67:0/3595834195 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:08.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.210+0000 7fa281109640 1 --2- 172.21.15.67:0/3595834195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e584bc840 0x555e58875c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.213+0000 7fa27bfff640 1 --2- 172.21.15.67:0/3595834195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e5849bb60 0x555e5887abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.213+0000 7fa280908640 1 -- 172.21.15.67:0/3595834195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e5849bb60 msgr2=0x555e5887abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:08.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.213+0000 7fa280908640 1 --2- 172.21.15.67:0/3595834195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e5849bb60 0x555e5887abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.213+0000 7fa280908640 1 -- 172.21.15.67:0/3595834195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e584bc840 msgr2=0x555e58875c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:08.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.213+0000 7fa280908640 1 --2- 172.21.15.67:0/3595834195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e584bc840 0x555e58875c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:08.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.213+0000 7fa280908640 1 -- 172.21.15.67:0/3595834195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e58876420 con 0x555e58493980 2024-07-31T15:13:08.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.214+0000 7fa280908640 1 --2- 172.21.15.67:0/3595834195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e58493980 0x555e5887a630 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fa268000f10 tx=0x7fa268039440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446ccdeef4d620df server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.215+0000 7fa25bfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:13:08.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.218+0000 7fa25bfff640 1 -- 172.21.15.67:0/3595834195 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268037660 con 0x555e58493980 2024-07-31T15:13:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.218+0000 7fa25bfff640 1 -- 172.21.15.67:0/3595834195 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7fa268037800 con 0x555e58493980 2024-07-31T15:13:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.219+0000 7fa2890bd740 1 -- 172.21.15.67:0/3595834195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa274036070 con 0x555e58493980 2024-07-31T15:13:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.220+0000 7fa25bfff640 1 -- 172.21.15.67:0/3595834195 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fa268037a00 con 0x555e58493980 2024-07-31T15:13:08.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.222+0000 7fa2890bd740 10 client.20109 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:08.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.222+0000 7fa2890bd740 1 -- 172.21.15.67:0/3595834195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x555e588e86e0 con 0x555e58493980 2024-07-31T15:13:08.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.225+0000 7fa25bfff640 1 -- 172.21.15.67:0/3595834195 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa274036070 con 0x555e58493980 2024-07-31T15:13:08.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.225+0000 7fa25bfff640 1 -- 172.21.15.67:0/3595834195 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fa268040440 con 0x555e58493980 2024-07-31T15:13:08.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.225+0000 7fa2890bd740 10 client.20109 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:08.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.225+0000 7fa2890bd740 10 client.20109 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:13:08.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.225+0000 7fa2890bd740 20 client.20109 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:08.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.225+0000 7fa2890bd740 1 --2- 172.21.15.67:0/3595834195 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e58450570 0x555e588eaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:08.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.225+0000 7fa2890bd740 4 client.20109 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.7, "await_for": 45.0}] 2024-07-31T15:13:08.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.225+0000 7fa2890bd740 1 -- 172.21.15.67:0/3595834195 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.7, "await_for": 45.0}) -- 0x555e587d51a0 con 0x555e58450570 2024-07-31T15:13:08.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.227+0000 7fa27bfff640 1 --2- 172.21.15.67:0/3595834195 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e58450570 0x555e588eaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:08.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.234+0000 7fa27bfff640 1 --2- 172.21.15.67:0/3595834195 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e58450570 0x555e588eaf70 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=627eb16c4d6b3a7e server_cookie=aef9781a3c94c62f in_seq=0 out_seq=0 2024-07-31T15:13:08.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:08.235+0000 7fa25bfff640 10 client.20109 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:09.709 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:09 smithi067 ceph-mon[32750]: pgmap v1105: 129 pgs: 15 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 67 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 753 KiB/s wr, 371 op/s 2024-07-31T15:13:09.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:09 smithi077 ceph-mon[37327]: pgmap v1105: 129 pgs: 15 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 67 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 753 KiB/s wr, 371 op/s 2024-07-31T15:13:09.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:09 smithi073 ceph-mon[36714]: pgmap v1105: 129 pgs: 15 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 67 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 753 KiB/s wr, 371 op/s 2024-07-31T15:13:10.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.541+0000 7fcbfcda5640 1 -- 172.21.15.67:0/1706222949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf81022b0 msgr2=0x7fcbf81026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:10.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.541+0000 7fcbfcda5640 1 --2- 172.21.15.67:0/1706222949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf81022b0 0x7fcbf81026c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009900 tx=0x7fcbe802a3f0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.541+0000 7fcbfcda5640 1 -- 172.21.15.67:0/1706222949 shutdown_connections 2024-07-31T15:13:10.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.541+0000 7fcbfcda5640 1 --2- 172.21.15.67:0/1706222949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf8108540 0x7fcbf810ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.541+0000 7fcbfcda5640 1 --2- 172.21.15.67:0/1706222949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbf8107bf0 0x7fcbf8108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.541+0000 7fcbfcda5640 1 --2- 172.21.15.67:0/1706222949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf81022b0 0x7fcbf81026c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.541+0000 7fcbfcda5640 1 -- 172.21.15.67:0/1706222949 >> 172.21.15.67:0/1706222949 conn(0x7fcbf80fdd00 msgr2=0x7fcbf8100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:10.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.541+0000 7fcbfcda5640 1 -- 172.21.15.67:0/1706222949 shutdown_connections 2024-07-31T15:13:10.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.541+0000 7fcbfcda5640 1 -- 172.21.15.67:0/1706222949 wait complete. 2024-07-31T15:13:10.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.542+0000 7fcbfcda5640 1 Processor -- start 2024-07-31T15:13:10.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.542+0000 7fcbfcda5640 1 -- start start 2024-07-31T15:13:10.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.542+0000 7fcbfcda5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf81022b0 0x7fcbf8199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.542+0000 7fcbfcda5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbf8107bf0 0x7fcbf819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.542+0000 7fcbfcda5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf8108540 0x7fcbf819f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.542+0000 7fcbfcda5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf8113320 con 0x7fcbf81022b0 2024-07-31T15:13:10.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.542+0000 7fcbfcda5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81131a0 con 0x7fcbf8108540 2024-07-31T15:13:10.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.542+0000 7fcbf77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf81022b0 0x7fcbf8199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.543+0000 7fcbf77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf81022b0 0x7fcbf8199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53404/0 (socket says 172.21.15.67:53404) 2024-07-31T15:13:10.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.543+0000 7fcbf77fe640 1 -- 172.21.15.67:0/1678526733 learned_addr learned my addr 172.21.15.67:0/1678526733 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:10.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.543+0000 7fcbf7fff640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf8108540 0x7fcbf819f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.543+0000 7fcbfcda5640 1 -- 172.21.15.67:0/1678526733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81134a0 con 0x7fcbf8107bf0 2024-07-31T15:13:10.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.543+0000 7fcbf6ffd640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbf8107bf0 0x7fcbf819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.543+0000 7fcbf7fff640 1 -- 172.21.15.67:0/1678526733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbf8107bf0 msgr2=0x7fcbf819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:10.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.543+0000 7fcbf7fff640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbf8107bf0 0x7fcbf819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.543+0000 7fcbf7fff640 1 -- 172.21.15.67:0/1678526733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf81022b0 msgr2=0x7fcbf8199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:10.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.543+0000 7fcbf7fff640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf81022b0 0x7fcbf8199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.543+0000 7fcbf7fff640 1 -- 172.21.15.67:0/1678526733 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf819fc60 con 0x7fcbf8108540 2024-07-31T15:13:10.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.543+0000 7fcbf77fe640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf81022b0 0x7fcbf8199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:10.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.543+0000 7fcbf7fff640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf8108540 0x7fcbf819f530 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000ec50 tx=0x7fcbf000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:10.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.544+0000 7fcbf4ff9640 1 -- 172.21.15.67:0/1678526733 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0003b70 con 0x7fcbf8108540 2024-07-31T15:13:10.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.544+0000 7fcbf4ff9640 1 -- 172.21.15.67:0/1678526733 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf0003d10 con 0x7fcbf8108540 2024-07-31T15:13:10.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.544+0000 7fcbf4ff9640 1 -- 172.21.15.67:0/1678526733 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0010830 con 0x7fcbf8108540 2024-07-31T15:13:10.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.544+0000 7fcbfcda5640 1 -- 172.21.15.67:0/1678526733 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf819ff50 con 0x7fcbf8108540 2024-07-31T15:13:10.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.544+0000 7fcbfcda5640 1 -- 172.21.15.67:0/1678526733 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf81a66b0 con 0x7fcbf8108540 2024-07-31T15:13:10.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.546+0000 7fcc04558740 1 Processor -- start 2024-07-31T15:13:10.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.547+0000 7fcbf4ff9640 1 -- 172.21.15.67:0/1678526733 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fcbf00109d0 con 0x7fcbf8108540 2024-07-31T15:13:10.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.547+0000 7fcc04558740 1 -- start start 2024-07-31T15:13:10.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.547+0000 7fcc04558740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4b7070 0x55841d85c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.547+0000 7fcc04558740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d48e1b0 0x55841d85cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.547+0000 7fcc04558740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d496390 0x55841d85d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.547+0000 7fcc04558740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55841d7d2bb0 con 0x55841d48e1b0 2024-07-31T15:13:10.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.547+0000 7fcc04558740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55841d7c20b0 con 0x55841d496390 2024-07-31T15:13:10.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.547+0000 7fcc04558740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55841d7c60f0 con 0x55841d4b7070 2024-07-31T15:13:10.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.547+0000 7fcbf6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4b7070 0x55841d85c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.548+0000 7fcbf6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4b7070 0x55841d85c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:36066/0 (socket says 172.21.15.67:36066) 2024-07-31T15:13:10.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.548+0000 7fcbf6ffd640 1 -- 172.21.15.67:0/2065978740 learned_addr learned my addr 172.21.15.67:0/2065978740 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:10.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.548+0000 7fcbf6ffd640 1 --2- 172.21.15.67:0/2065978740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d496390 0x55841d85d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.548+0000 7fcbf6ffd640 1 -- 172.21.15.67:0/2065978740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4b7070 msgr2=0x55841d85c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:10.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.548+0000 7fcbf6ffd640 1 --2- 172.21.15.67:0/2065978740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4b7070 0x55841d85c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.548+0000 7fcbf6ffd640 1 -- 172.21.15.67:0/2065978740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d48e1b0 msgr2=0x55841d85cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:10.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.548+0000 7fcbf4ff9640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcbe00b3dc0 0x7fcbe00b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.548+0000 7fcbf4ff9640 1 -- 172.21.15.67:0/1678526733 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7fcbf00e4930 con 0x7fcbf8108540 2024-07-31T15:13:10.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.549+0000 7fcbf77fe640 1 --2- 172.21.15.67:0/2065978740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d48e1b0 0x55841d85cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.549+0000 7fcbf6ffd640 1 --2- 172.21.15.67:0/2065978740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d48e1b0 0x55841d85cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.549+0000 7fcbf6ffd640 1 -- 172.21.15.67:0/2065978740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55841d768b90 con 0x55841d496390 2024-07-31T15:13:10.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.549+0000 7fcbf6ffd640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcbe00b3dc0 0x7fcbe00b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.549+0000 7fcbf6ffd640 1 --2- 172.21.15.67:0/2065978740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d496390 0x55841d85d170 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0117a0 tx=0x7fcbec005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ad50394026a0e6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:10.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.549+0000 7fcbf77fe640 1 --2- 172.21.15.67:0/2065978740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d48e1b0 0x55841d85cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:13:10.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.549+0000 7fcbddffb640 1 -- 172.21.15.67:0/2065978740 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec027070 con 0x55841d496390 2024-07-31T15:13:10.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.549+0000 7fcbddffb640 1 -- 172.21.15.67:0/2065978740 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbec00fcf0 con 0x55841d496390 2024-07-31T15:13:10.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.549+0000 7fcbddffb640 1 -- 172.21.15.67:0/2065978740 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0203b0 con 0x55841d496390 2024-07-31T15:13:10.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.551+0000 7fcc04558740 1 -- 172.21.15.67:0/2065978740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d496390 msgr2=0x55841d85d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:10.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.551+0000 7fcc04558740 1 --2- 172.21.15.67:0/2065978740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d496390 0x55841d85d170 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0117a0 tx=0x7fcbec005f90 comp rx=0 tx=0).stop 2024-07-31T15:13:10.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.551+0000 7fcc04558740 1 -- 172.21.15.67:0/2065978740 shutdown_connections 2024-07-31T15:13:10.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.551+0000 7fcc04558740 1 --2- 172.21.15.67:0/2065978740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d496390 0x55841d85d170 secure :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0117a0 tx=0x7fcbec005f90 comp rx=0 tx=0).stop 2024-07-31T15:13:10.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.551+0000 7fcc04558740 1 --2- 172.21.15.67:0/2065978740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d48e1b0 0x55841d85cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.551+0000 7fcc04558740 1 --2- 172.21.15.67:0/2065978740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4b7070 0x55841d85c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.551+0000 7fcc04558740 1 -- 172.21.15.67:0/2065978740 >> 172.21.15.67:0/2065978740 conn(0x55841d4ab460 msgr2=0x55841d85a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:10.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.551+0000 7fcbf6ffd640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcbe00b3dc0 0x7fcbe00b62d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcbec024000 tx=0x7fcbec0209e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:10.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.551+0000 7fcc04558740 1 -- 172.21.15.67:0/2065978740 shutdown_connections 2024-07-31T15:13:10.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.551+0000 7fcc04558740 1 -- 172.21.15.67:0/2065978740 wait complete. 2024-07-31T15:13:10.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.552+0000 7fcc04558740 1 -- start start 2024-07-31T15:13:10.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.552+0000 7fcc04558740 1 Processor -- start 2024-07-31T15:13:10.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcc04558740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d48e1b0 0x55841d86c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcc04558740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d4b7070 0x55841d86cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcc04558740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d4759f0 0x55841d872500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcc04558740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55841d625700 con 0x55841d4b7070 2024-07-31T15:13:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcc04558740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55841d7649e0 con 0x55841d4759f0 2024-07-31T15:13:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcc04558740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55841d86f200 con 0x55841d48e1b0 2024-07-31T15:13:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcbf6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d4b7070 0x55841d86cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcc04558740 10 client.0 _ll_register_callbacks cb 0x55841d77f4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcbf6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d4b7070 0x55841d86cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53416/0 (socket says 172.21.15.67:53416) 2024-07-31T15:13:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcbf6ffd640 1 -- 172.21.15.67:0/2569574836 learned_addr learned my addr 172.21.15.67:0/2569574836 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcbf7fff640 1 --2- 172.21.15.67:0/2569574836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d4759f0 0x55841d872500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcbf7fff640 1 -- 172.21.15.67:0/2569574836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d48e1b0 msgr2=0x55841d86c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcbf7fff640 1 --2- 172.21.15.67:0/2569574836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d48e1b0 0x55841d86c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcbf7fff640 1 -- 172.21.15.67:0/2569574836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d4b7070 msgr2=0x55841d86cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcbf7fff640 1 --2- 172.21.15.67:0/2569574836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d4b7070 0x55841d86cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.553+0000 7fcbf7fff640 1 -- 172.21.15.67:0/2569574836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55841d522650 con 0x55841d4759f0 2024-07-31T15:13:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.554+0000 7fcbf7fff640 1 --2- 172.21.15.67:0/2569574836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d4759f0 0x55841d872500 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00e1c30 tx=0x7fcbf00aa890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f971f8d36d54bee server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.554+0000 7fcbd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:13:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.554+0000 7fcbd6ffd640 1 -- 172.21.15.67:0/2569574836 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00c4020 con 0x55841d4759f0 2024-07-31T15:13:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.554+0000 7fcbd6ffd640 1 -- 172.21.15.67:0/2569574836 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7fcbf0004d50 con 0x55841d4759f0 2024-07-31T15:13:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.554+0000 7fcc04558740 1 -- 172.21.15.67:0/2569574836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbec00fcf0 con 0x55841d4759f0 2024-07-31T15:13:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.554+0000 7fcbd6ffd640 1 -- 172.21.15.67:0/2569574836 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fcbf00ab820 con 0x55841d4759f0 2024-07-31T15:13:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.554+0000 7fcc04558740 10 client.29639 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.554+0000 7fcc04558740 1 -- 172.21.15.67:0/2569574836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55841d862a80 con 0x55841d4759f0 2024-07-31T15:13:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.554+0000 7fcbd6ffd640 1 -- 172.21.15.67:0/2569574836 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf00c9020 con 0x55841d4759f0 2024-07-31T15:13:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.554+0000 7fcbd6ffd640 1 -- 172.21.15.67:0/2569574836 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fcbf00a6a90 con 0x55841d4759f0 2024-07-31T15:13:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.555+0000 7fcc04558740 10 client.29639 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.555+0000 7fcc04558740 10 client.29639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.555+0000 7fcc04558740 20 client.29639 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.555+0000 7fcc04558740 1 --2- 172.21.15.67:0/2569574836 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55841d44ada0 0x55841d865ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.555+0000 7fcc04558740 4 client.29639 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:13:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.555+0000 7fcc04558740 1 -- 172.21.15.67:0/2569574836 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcbec00fcf0 con 0x55841d44ada0 2024-07-31T15:13:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.555+0000 7fcbf77fe640 1 --2- 172.21.15.67:0/2569574836 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55841d44ada0 0x55841d865ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.555+0000 7fcbd6ffd640 10 client.29639 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.556+0000 7fcbf77fe640 1 --2- 172.21.15.67:0/2569574836 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55841d44ada0 0x55841d865ff0 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6def3eb13471f69 server_cookie=d074c010800b5c95 in_seq=0 out_seq=0 2024-07-31T15:13:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.556+0000 7fcbd6ffd640 1 -- 172.21.15.67:0/2569574836 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fcbec00fcf0 con 0x55841d44ada0 2024-07-31T15:13:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.556+0000 7fcbd6ffd640 10 client.29639 handle_command_reply: tid=0 2024-07-31T15:13:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.556+0000 7fcc04558740 1 client.29639 shutdown 2024-07-31T15:13:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.556+0000 7fcc04558740 1 -- 172.21.15.67:0/2569574836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d4759f0 msgr2=0x55841d872500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.556+0000 7fcc04558740 1 --2- 172.21.15.67:0/2569574836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d4759f0 0x55841d872500 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00e1c30 tx=0x7fcbf00aa890 comp rx=0 tx=0).stop 2024-07-31T15:13:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.557+0000 7fcc04558740 1 -- 172.21.15.67:0/2569574836 shutdown_connections 2024-07-31T15:13:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.557+0000 7fcc04558740 1 --2- 172.21.15.67:0/2569574836 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55841d44ada0 0x55841d865ff0 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.557+0000 7fcbd6ffd640 0 client.29639 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.557+0000 7fcc04558740 1 --2- 172.21.15.67:0/2569574836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d4759f0 0x55841d872500 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.557+0000 7fcc04558740 1 --2- 172.21.15.67:0/2569574836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d4b7070 0x55841d86cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.557+0000 7fcc04558740 1 --2- 172.21.15.67:0/2569574836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d48e1b0 0x55841d86c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.557+0000 7fcc04558740 1 -- 172.21.15.67:0/2569574836 >> 172.21.15.67:0/2569574836 conn(0x55841d4ab460 msgr2=0x55841d858c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.557+0000 7fcc04558740 1 -- 172.21.15.67:0/2569574836 shutdown_connections 2024-07-31T15:13:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.557+0000 7fcc04558740 1 -- 172.21.15.67:0/2569574836 wait complete. 2024-07-31T15:13:10.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.557+0000 7fcc04558740 20 client.29639 trim_cache size 0 max 16384 2024-07-31T15:13:10.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.583+0000 7fcc04558740 1 Processor -- start 2024-07-31T15:13:10.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.584+0000 7fcc04558740 1 -- start start 2024-07-31T15:13:10.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.584+0000 7fcc04558740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4759f0 0x55841d872470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.584+0000 7fcc04558740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d48e1b0 0x55841d85de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.584+0000 7fcc04558740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d4b7070 0x55841d880220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.584+0000 7fcbf77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4759f0 0x55841d872470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.584+0000 7fcbf77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4759f0 0x55841d872470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:36092/0 (socket says 172.21.15.67:36092) 2024-07-31T15:13:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.584+0000 7fcbf6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d48e1b0 0x55841d85de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.584+0000 7fcbf6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d48e1b0 0x55841d85de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:60308/0 (socket says 172.21.15.67:60308) 2024-07-31T15:13:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.585+0000 7fcbf77fe640 1 -- 172.21.15.67:0/749083148 learned_addr learned my addr 172.21.15.67:0/749083148 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.585+0000 7fcc04558740 1 -- 172.21.15.67:0/749083148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81b6cd0 con 0x55841d4b7070 2024-07-31T15:13:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.585+0000 7fcc04558740 1 -- 172.21.15.67:0/749083148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81b8810 con 0x55841d48e1b0 2024-07-31T15:13:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.585+0000 7fcc04558740 1 -- 172.21.15.67:0/749083148 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81b73a0 con 0x55841d4759f0 2024-07-31T15:13:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.585+0000 7fcbf7fff640 1 --2- 172.21.15.67:0/749083148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d4b7070 0x55841d880220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.585+0000 7fcbf6ffd640 1 -- 172.21.15.67:0/749083148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4759f0 msgr2=0x55841d872470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.585+0000 7fcbf6ffd640 1 --2- 172.21.15.67:0/749083148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4759f0 0x55841d872470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.585+0000 7fcbf6ffd640 1 -- 172.21.15.67:0/749083148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d4b7070 msgr2=0x55841d880220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.585+0000 7fcbf6ffd640 1 --2- 172.21.15.67:0/749083148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d4b7070 0x55841d880220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.585+0000 7fcbf6ffd640 1 -- 172.21.15.67:0/749083148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55841d85ef80 con 0x55841d48e1b0 2024-07-31T15:13:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.585+0000 7fcbf6ffd640 1 --2- 172.21.15.67:0/749083148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d48e1b0 0x55841d85de30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55841d855ec0 tx=0x7fcbec028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=422a7bd2dda5ccd0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.586+0000 7fcbdd7fa640 1 -- 172.21.15.67:0/749083148 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec04c070 con 0x55841d48e1b0 2024-07-31T15:13:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.586+0000 7fcbdd7fa640 1 -- 172.21.15.67:0/749083148 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbec021b50 con 0x55841d48e1b0 2024-07-31T15:13:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.587+0000 7fcbdd7fa640 1 -- 172.21.15.67:0/749083148 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec027030 con 0x55841d48e1b0 2024-07-31T15:13:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.587+0000 7fcc04558740 1 -- 172.21.15.67:0/749083148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d48e1b0 msgr2=0x55841d85de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.587+0000 7fcc04558740 1 --2- 172.21.15.67:0/749083148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d48e1b0 0x55841d85de30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55841d855ec0 tx=0x7fcbec028000 comp rx=0 tx=0).stop 2024-07-31T15:13:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.588+0000 7fcc04558740 1 -- 172.21.15.67:0/749083148 shutdown_connections 2024-07-31T15:13:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.588+0000 7fcc04558740 1 --2- 172.21.15.67:0/749083148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d4b7070 0x55841d880220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.588+0000 7fcc04558740 1 --2- 172.21.15.67:0/749083148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d48e1b0 0x55841d85de30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.588+0000 7fcc04558740 1 --2- 172.21.15.67:0/749083148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4759f0 0x55841d872470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.588+0000 7fcc04558740 1 -- 172.21.15.67:0/749083148 >> 172.21.15.67:0/749083148 conn(0x55841d4ab460 msgr2=0x55841d85d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:10.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.589+0000 7fcc04558740 1 -- 172.21.15.67:0/749083148 shutdown_connections 2024-07-31T15:13:10.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.589+0000 7fcc04558740 1 -- 172.21.15.67:0/749083148 wait complete. 2024-07-31T15:13:10.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.590+0000 7fcc04558740 1 -- start start 2024-07-31T15:13:10.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.590+0000 7fcc04558740 1 Processor -- start 2024-07-31T15:13:10.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.590+0000 7fcc04558740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4759f0 0x55841d880810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.590+0000 7fcc04558740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d48e1b0 0x55841d87dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.590+0000 7fcc04558740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d4b7070 0x55841d87e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.590+0000 7fcc04558740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81b8140 con 0x55841d48e1b0 2024-07-31T15:13:10.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.590+0000 7fcc04558740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81b7a70 con 0x55841d4b7070 2024-07-31T15:13:10.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.590+0000 7fcc04558740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81b4cd0 con 0x55841d4759f0 2024-07-31T15:13:10.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.590+0000 7fcc04558740 10 client.0 _ll_register_callbacks cb 0x7fcbf0005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:10.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.591+0000 7fcbf6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d48e1b0 0x55841d87dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.591+0000 7fcbf6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d48e1b0 0x55841d87dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53444/0 (socket says 172.21.15.67:53444) 2024-07-31T15:13:10.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.591+0000 7fcbf6ffd640 1 -- 172.21.15.67:0/1850653800 learned_addr learned my addr 172.21.15.67:0/1850653800 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:10.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.591+0000 7fcbf77fe640 1 --2- 172.21.15.67:0/1850653800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4759f0 0x55841d880810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.591+0000 7fcbf7fff640 1 --2- 172.21.15.67:0/1850653800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d4b7070 0x55841d87e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.592+0000 7fcbf7fff640 1 -- 172.21.15.67:0/1850653800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4759f0 msgr2=0x55841d880810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.592+0000 7fcbf7fff640 1 --2- 172.21.15.67:0/1850653800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4759f0 0x55841d880810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.592+0000 7fcbf7fff640 1 -- 172.21.15.67:0/1850653800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d48e1b0 msgr2=0x55841d87dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.592+0000 7fcbf7fff640 1 --2- 172.21.15.67:0/1850653800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d48e1b0 0x55841d87dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.592+0000 7fcbf7fff640 1 -- 172.21.15.67:0/1850653800 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55841d87e860 con 0x55841d4b7070 2024-07-31T15:13:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.592+0000 7fcbf7fff640 1 --2- 172.21.15.67:0/1850653800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d4b7070 0x55841d87e220 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00ab0b0 tx=0x7fcbf00aa6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=403376ff64c62b29 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.593+0000 7fcbd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:13:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.593+0000 7fcbd6ffd640 1 -- 172.21.15.67:0/1850653800 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00c4040 con 0x55841d4b7070 2024-07-31T15:13:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.593+0000 7fcbd6ffd640 1 -- 172.21.15.67:0/1850653800 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7fcbf00a7710 con 0x55841d4b7070 2024-07-31T15:13:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.593+0000 7fcc04558740 1 -- 172.21.15.67:0/1850653800 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbec021b50 con 0x55841d4b7070 2024-07-31T15:13:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.594+0000 7fcbd6ffd640 1 -- 172.21.15.67:0/1850653800 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fcbf0004660 con 0x55841d4b7070 2024-07-31T15:13:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.594+0000 7fcc04558740 10 client.29645 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.594+0000 7fcc04558740 1 -- 172.21.15.67:0/1850653800 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55841d8e3070 con 0x55841d4b7070 2024-07-31T15:13:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.594+0000 7fcbd6ffd640 1 -- 172.21.15.67:0/1850653800 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbec021b50 con 0x55841d4b7070 2024-07-31T15:13:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.594+0000 7fcbd6ffd640 1 -- 172.21.15.67:0/1850653800 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fcbf00b0ac0 con 0x55841d4b7070 2024-07-31T15:13:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.594+0000 7fcc04558740 10 client.29645 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.594+0000 7fcc04558740 10 client.29645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.594+0000 7fcc04558740 20 client.29645 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.594+0000 7fcc04558740 1 --2- 172.21.15.67:0/1850653800 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55841d8e3740 0x55841d8e5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.594+0000 7fcc04558740 4 client.29645 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:13:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.595+0000 7fcc04558740 1 -- 172.21.15.67:0/1850653800 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55841d7d4b00 con 0x55841d8e3740 2024-07-31T15:13:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.595+0000 7fcbf77fe640 1 --2- 172.21.15.67:0/1850653800 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55841d8e3740 0x55841d8e5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.595+0000 7fcbd6ffd640 10 client.29645 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.595+0000 7fcbf77fe640 1 --2- 172.21.15.67:0/1850653800 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55841d8e3740 0x55841d8e5b60 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=847b1e68cc289724 server_cookie=704805524f8c7cc4 in_seq=0 out_seq=0 2024-07-31T15:13:10.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.597+0000 7fcbd6ffd640 1 -- 172.21.15.67:0/1850653800 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55841d7d4b00 con 0x55841d8e3740 2024-07-31T15:13:10.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.597+0000 7fcbd6ffd640 10 client.29645 handle_command_reply: tid=0 2024-07-31T15:13:10.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.597+0000 7fcc04558740 1 client.29645 shutdown 2024-07-31T15:13:10.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.597+0000 7fcc04558740 1 -- 172.21.15.67:0/1850653800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d4b7070 msgr2=0x55841d87e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:10.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.597+0000 7fcc04558740 1 --2- 172.21.15.67:0/1850653800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d4b7070 0x55841d87e220 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00ab0b0 tx=0x7fcbf00aa6b0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.600+0000 7fcc04558740 1 -- 172.21.15.67:0/1850653800 shutdown_connections 2024-07-31T15:13:10.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.600+0000 7fcc04558740 1 --2- 172.21.15.67:0/1850653800 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55841d8e3740 0x55841d8e5b60 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.600+0000 7fcc04558740 1 --2- 172.21.15.67:0/1850653800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55841d4b7070 0x55841d87e220 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.600+0000 7fcc04558740 1 --2- 172.21.15.67:0/1850653800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55841d48e1b0 0x55841d87dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.600+0000 7fcc04558740 1 --2- 172.21.15.67:0/1850653800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55841d4759f0 0x55841d880810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.600+0000 7fcc04558740 1 -- 172.21.15.67:0/1850653800 >> 172.21.15.67:0/1850653800 conn(0x55841d4ab460 msgr2=0x55841d85e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:10.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.600+0000 7fcbd6ffd640 0 client.29645 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:10.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.601+0000 7fcc04558740 1 -- 172.21.15.67:0/1850653800 shutdown_connections 2024-07-31T15:13:10.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.601+0000 7fcc04558740 1 -- 172.21.15.67:0/1850653800 wait complete. 2024-07-31T15:13:10.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.601+0000 7fcc04558740 20 client.29645 trim_cache size 0 max 16384 2024-07-31T15:13:10.603 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:13:10.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.603+0000 7fcbfcda5640 1 -- 172.21.15.67:0/1678526733 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcbe00b3dc0 msgr2=0x7fcbe00b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:10.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.603+0000 7fcbfcda5640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcbe00b3dc0 0x7fcbe00b62d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcbec024000 tx=0x7fcbec0209e0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.603+0000 7fcbfcda5640 1 -- 172.21.15.67:0/1678526733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf8108540 msgr2=0x7fcbf819f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:10.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.603+0000 7fcbfcda5640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf8108540 0x7fcbf819f530 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000ec50 tx=0x7fcbf000c6a0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.603+0000 7fcbfcda5640 1 -- 172.21.15.67:0/1678526733 shutdown_connections 2024-07-31T15:13:10.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.603+0000 7fcbfcda5640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fcbe00b3dc0 0x7fcbe00b62d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.603+0000 7fcbfcda5640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbf8108540 0x7fcbf819f530 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.603+0000 7fcbfcda5640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbf8107bf0 0x7fcbf819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.603+0000 7fcbfcda5640 1 --2- 172.21.15.67:0/1678526733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf81022b0 0x7fcbf8199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:10.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.603+0000 7fcbfcda5640 1 -- 172.21.15.67:0/1678526733 >> 172.21.15.67:0/1678526733 conn(0x7fcbf80fdd00 msgr2=0x7fcbf8100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:10.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.604+0000 7fcbfcda5640 1 -- 172.21.15.67:0/1678526733 shutdown_connections 2024-07-31T15:13:10.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:10.604+0000 7fcbfcda5640 1 -- 172.21.15.67:0/1678526733 wait complete. 2024-07-31T15:13:11.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:11 smithi077 ceph-mon[37327]: pgmap v1106: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 719 KiB/s wr, 379 op/s 2024-07-31T15:13:11.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:11 smithi067 ceph-mon[32750]: pgmap v1106: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 719 KiB/s wr, 379 op/s 2024-07-31T15:13:11.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:11 smithi073 ceph-mon[36714]: pgmap v1106: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 719 KiB/s wr, 379 op/s 2024-07-31T15:13:12.481 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T15:13:12.482 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T15:13:12.848 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:13:13.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:13 smithi067 ceph-mon[32750]: pgmap v1107: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.3 MiB/s wr, 356 op/s 2024-07-31T15:13:13.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:13 smithi077 ceph-mon[37327]: pgmap v1107: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.3 MiB/s wr, 356 op/s 2024-07-31T15:13:13.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:13 smithi073 ceph-mon[36714]: pgmap v1107: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.3 MiB/s wr, 356 op/s 2024-07-31T15:13:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.078+0000 7fba56fb6640 1 -- 172.21.15.67:0/121118396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba501023f0 msgr2=0x7fba50102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.078+0000 7fba56fb6640 1 --2- 172.21.15.67:0/121118396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba501023f0 0x7fba50102800 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fba44009900 tx=0x7fba4402a410 comp rx=0 tx=0).stop 2024-07-31T15:13:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.078+0000 7fba56fb6640 1 -- 172.21.15.67:0/121118396 shutdown_connections 2024-07-31T15:13:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.078+0000 7fba56fb6640 1 --2- 172.21.15.67:0/121118396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba50108670 0x7fba5010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.078+0000 7fba56fb6640 1 --2- 172.21.15.67:0/121118396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba50107d20 0x7fba50108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.078+0000 7fba56fb6640 1 --2- 172.21.15.67:0/121118396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba501023f0 0x7fba50102800 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.078+0000 7fba56fb6640 1 -- 172.21.15.67:0/121118396 >> 172.21.15.67:0/121118396 conn(0x7fba500fddf0 msgr2=0x7fba50100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.078+0000 7fba56fb6640 1 -- 172.21.15.67:0/121118396 shutdown_connections 2024-07-31T15:13:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.078+0000 7fba56fb6640 1 -- 172.21.15.67:0/121118396 wait complete. 2024-07-31T15:13:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba56fb6640 1 Processor -- start 2024-07-31T15:13:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba56fb6640 1 -- start start 2024-07-31T15:13:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba56fb6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba501023f0 0x7fba500fe320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba56fb6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba50107d20 0x7fba500fe860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba56fb6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba50108670 0x7fba500feda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba56fb6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fba50112340 con 0x7fba50108670 2024-07-31T15:13:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba56fb6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fba501121c0 con 0x7fba501023f0 2024-07-31T15:13:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba56fb6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba501124c0 con 0x7fba50107d20 2024-07-31T15:13:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba54d52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba501023f0 0x7fba500fe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba54d52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba501023f0 0x7fba500fe320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:33350/0 (socket says 172.21.15.67:33350) 2024-07-31T15:13:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba4ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba50107d20 0x7fba500fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba54d52640 1 -- 172.21.15.67:0/1884890008 learned_addr learned my addr 172.21.15.67:0/1884890008 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba55553640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba50108670 0x7fba500feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba54d52640 1 -- 172.21.15.67:0/1884890008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba50107d20 msgr2=0x7fba500fe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba54d52640 1 --2- 172.21.15.67:0/1884890008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba50107d20 0x7fba500fe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba54d52640 1 -- 172.21.15.67:0/1884890008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba50108670 msgr2=0x7fba500feda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba54d52640 1 --2- 172.21.15.67:0/1884890008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba50108670 0x7fba500feda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba54d52640 1 -- 172.21.15.67:0/1884890008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba501a8100 con 0x7fba501023f0 2024-07-31T15:13:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba4ffff640 1 --2- 172.21.15.67:0/1884890008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba50107d20 0x7fba500fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.079+0000 7fba55553640 1 --2- 172.21.15.67:0/1884890008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba50108670 0x7fba500feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.080+0000 7fba54d52640 1 --2- 172.21.15.67:0/1884890008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba501023f0 0x7fba500fe320 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fba44005940 tx=0x7fba440040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.080+0000 7fba4dffb640 1 -- 172.21.15.67:0/1884890008 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4403f030 con 0x7fba501023f0 2024-07-31T15:13:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.080+0000 7fba4dffb640 1 -- 172.21.15.67:0/1884890008 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba440044c0 con 0x7fba501023f0 2024-07-31T15:13:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.080+0000 7fba56fb6640 1 -- 172.21.15.67:0/1884890008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba501a8390 con 0x7fba501023f0 2024-07-31T15:13:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.080+0000 7fba4dffb640 1 -- 172.21.15.67:0/1884890008 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba44035070 con 0x7fba501023f0 2024-07-31T15:13:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.080+0000 7fba56fb6640 1 -- 172.21.15.67:0/1884890008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba501a8880 con 0x7fba501023f0 2024-07-31T15:13:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.082+0000 7fba5df86740 1 Processor -- start 2024-07-31T15:13:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.082+0000 7fba5df86740 1 -- start start 2024-07-31T15:13:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.082+0000 7fba4dffb640 1 -- 172.21.15.67:0/1884890008 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fba44042030 con 0x7fba501023f0 2024-07-31T15:13:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba5df86740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba5df86740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdddc2f0 0x5587be1a8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba5df86740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdde44d0 0x5587be1a8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba5df86740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5587be11d3b0 con 0x5587bde051b0 2024-07-31T15:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba5df86740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5587be10c8b0 con 0x5587bdde44d0 2024-07-31T15:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba5df86740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587be1108f0 con 0x5587bdddc2f0 2024-07-31T15:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba55553640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba55553640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1a7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56114/0 (socket says 172.21.15.67:56114) 2024-07-31T15:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba55553640 1 -- 172.21.15.67:0/3904484406 learned_addr learned my addr 172.21.15.67:0/3904484406 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba4ffff640 1 --2- 172.21.15.67:0/3904484406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdde44d0 0x5587be1a8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba54d52640 1 --2- 172.21.15.67:0/3904484406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdddc2f0 0x5587be1a8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba55553640 1 -- 172.21.15.67:0/3904484406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdddc2f0 msgr2=0x5587be1a8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba55553640 1 --2- 172.21.15.67:0/3904484406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdddc2f0 0x5587be1a8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba55553640 1 -- 172.21.15.67:0/3904484406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdde44d0 msgr2=0x5587be1a8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba55553640 1 --2- 172.21.15.67:0/3904484406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdde44d0 0x5587be1a8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba55553640 1 -- 172.21.15.67:0/3904484406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587be0b39a0 con 0x5587bde051b0 2024-07-31T15:13:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.083+0000 7fba55553640 1 --2- 172.21.15.67:0/3904484406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1a7ce0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fba480117e0 tx=0x7fba48011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7360f80578f440af server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.084+0000 7fba4dffb640 1 --2- 172.21.15.67:0/1884890008 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fba400b3d70 0x7fba400b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.084+0000 7fba3affd640 1 -- 172.21.15.67:0/3904484406 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48011310 con 0x5587bde051b0 2024-07-31T15:13:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.084+0000 7fba54d52640 1 --2- 172.21.15.67:0/1884890008 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fba400b3d70 0x7fba400b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.084+0000 7fba3affd640 1 -- 172.21.15.67:0/3904484406 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba48003e20 con 0x5587bde051b0 2024-07-31T15:13:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.084+0000 7fba3affd640 1 -- 172.21.15.67:0/3904484406 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba480044f0 con 0x5587bde051b0 2024-07-31T15:13:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.084+0000 7fba4dffb640 1 -- 172.21.15.67:0/1884890008 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7fba441046c0 con 0x7fba501023f0 2024-07-31T15:13:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.084+0000 7fba5df86740 1 -- 172.21.15.67:0/3904484406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 msgr2=0x5587be1a7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.084+0000 7fba54d52640 1 --2- 172.21.15.67:0/1884890008 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fba400b3d70 0x7fba400b6280 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fba44101b00 tx=0x7fba4410a550 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.084+0000 7fba5df86740 1 --2- 172.21.15.67:0/3904484406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1a7ce0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fba480117e0 tx=0x7fba48011ca0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 -- 172.21.15.67:0/3904484406 shutdown_connections 2024-07-31T15:13:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 --2- 172.21.15.67:0/3904484406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdde44d0 0x5587be1a8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 --2- 172.21.15.67:0/3904484406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdddc2f0 0x5587be1a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 --2- 172.21.15.67:0/3904484406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1a7ce0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 -- 172.21.15.67:0/3904484406 >> 172.21.15.67:0/3904484406 conn(0x5587bddf9300 msgr2=0x5587be1a5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 -- 172.21.15.67:0/3904484406 shutdown_connections 2024-07-31T15:13:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 -- 172.21.15.67:0/3904484406 wait complete. 2024-07-31T15:13:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 -- start start 2024-07-31T15:13:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 Processor -- start 2024-07-31T15:13:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdde44d0 0x5587be1b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1bf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5587be046ab0 con 0x5587bde051b0 2024-07-31T15:13:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5587be094cf0 con 0x5587bdddc2f0 2024-07-31T15:13:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.085+0000 7fba5df86740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587be1b6460 con 0x5587bdde44d0 2024-07-31T15:13:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba5df86740 10 client.0 _ll_register_callbacks cb 0x5587be1069f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba55553640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1bf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba55553640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1bf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56122/0 (socket says 172.21.15.67:56122) 2024-07-31T15:13:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba55553640 1 -- 172.21.15.67:0/2769082214 learned_addr learned my addr 172.21.15.67:0/2769082214 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba4ffff640 1 --2- 172.21.15.67:0/2769082214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdde44d0 0x5587be1b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba55553640 1 --2- 172.21.15.67:0/2769082214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba4ffff640 1 -- 172.21.15.67:0/2769082214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 msgr2=0x5587be1b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba4ffff640 1 --2- 172.21.15.67:0/2769082214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba4ffff640 1 -- 172.21.15.67:0/2769082214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 msgr2=0x5587be1bf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba4ffff640 1 --2- 172.21.15.67:0/2769082214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1bf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba4ffff640 1 -- 172.21.15.67:0/2769082214 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587bde70790 con 0x5587bdde44d0 2024-07-31T15:13:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba55553640 1 --2- 172.21.15.67:0/2769082214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1bf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba4ffff640 1 --2- 172.21.15.67:0/2769082214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdde44d0 0x5587be1b3d20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fba5006c100 tx=0x7fba3c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e21fbd4b9b24223 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:13:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba2ffff640 1 -- 172.21.15.67:0/2769082214 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c012110 con 0x5587bdde44d0 2024-07-31T15:13:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba5df86740 1 -- 172.21.15.67:0/2769082214 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba48003e20 con 0x5587bdde44d0 2024-07-31T15:13:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.086+0000 7fba2ffff640 1 -- 172.21.15.67:0/2769082214 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7fba3c004050 con 0x5587bdde44d0 2024-07-31T15:13:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.087+0000 7fba2ffff640 1 -- 172.21.15.67:0/2769082214 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fba3c025050 con 0x5587bdde44d0 2024-07-31T15:13:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.087+0000 7fba5df86740 10 client.29500 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.087+0000 7fba5df86740 1 -- 172.21.15.67:0/2769082214 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5587bdee9620 con 0x5587bdde44d0 2024-07-31T15:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.087+0000 7fba2ffff640 1 -- 172.21.15.67:0/2769082214 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba48003e20 con 0x5587bdde44d0 2024-07-31T15:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.087+0000 7fba2ffff640 1 -- 172.21.15.67:0/2769082214 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fba3c010430 con 0x5587bdde44d0 2024-07-31T15:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.087+0000 7fba5df86740 10 client.29500 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.087+0000 7fba5df86740 10 client.29500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.087+0000 7fba5df86740 20 client.29500 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.087+0000 7fba5df86740 1 --2- 172.21.15.67:0/2769082214 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5587bddc3d80 0x5587be1c5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.087+0000 7fba5df86740 4 client.29500 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.087+0000 7fba5df86740 1 -- 172.21.15.67:0/2769082214 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587be11f300 con 0x5587bddc3d80 2024-07-31T15:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.087+0000 7fba55553640 1 --2- 172.21.15.67:0/2769082214 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5587bddc3d80 0x5587be1c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.088+0000 7fba55553640 1 --2- 172.21.15.67:0/2769082214 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5587bddc3d80 0x5587be1c5880 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19c781e98b266617 server_cookie=553bc3a138638f36 in_seq=0 out_seq=0 2024-07-31T15:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.088+0000 7fba2ffff640 10 client.29500 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.136+0000 7fba2ffff640 1 -- 172.21.15.67:0/2769082214 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587be11f300 con 0x5587bddc3d80 2024-07-31T15:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.136+0000 7fba2ffff640 10 client.29500 handle_command_reply: tid=0 2024-07-31T15:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.137+0000 7fba5df86740 1 client.29500 shutdown 2024-07-31T15:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.137+0000 7fba5df86740 1 -- 172.21.15.67:0/2769082214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdde44d0 msgr2=0x5587be1b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.137+0000 7fba5df86740 1 --2- 172.21.15.67:0/2769082214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdde44d0 0x5587be1b3d20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fba5006c100 tx=0x7fba3c00f790 comp rx=0 tx=0).stop 2024-07-31T15:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.137+0000 7fba5df86740 1 -- 172.21.15.67:0/2769082214 shutdown_connections 2024-07-31T15:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.137+0000 7fba5df86740 1 --2- 172.21.15.67:0/2769082214 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5587bddc3d80 0x5587be1c5880 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.137+0000 7fba5df86740 1 --2- 172.21.15.67:0/2769082214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1bf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.137+0000 7fba5df86740 1 --2- 172.21.15.67:0/2769082214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdde44d0 0x5587be1b3d20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.137+0000 7fba5df86740 1 --2- 172.21.15.67:0/2769082214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.137+0000 7fba5df86740 1 -- 172.21.15.67:0/2769082214 >> 172.21.15.67:0/2769082214 conn(0x5587bddf9300 msgr2=0x5587be1a8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.137+0000 7fba2ffff640 0 client.29500 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.137+0000 7fba5df86740 1 -- 172.21.15.67:0/2769082214 shutdown_connections 2024-07-31T15:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.137+0000 7fba5df86740 1 -- 172.21.15.67:0/2769082214 wait complete. 2024-07-31T15:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.137+0000 7fba5df86740 20 client.29500 trim_cache size 0 max 16384 2024-07-31T15:13:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.152+0000 7fba5df86740 1 Processor -- start 2024-07-31T15:13:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.152+0000 7fba5df86740 1 -- start start 2024-07-31T15:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba5df86740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba5df86740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdde44d0 0x5587be1b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba5df86740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba5df86740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fba501b2a90 con 0x5587bde051b0 2024-07-31T15:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba5df86740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fba501b45d0 con 0x5587bdddc2f0 2024-07-31T15:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba5df86740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba501b3160 con 0x5587bdde44d0 2024-07-31T15:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba55553640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba55553640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba55553640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1a96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:33362/0 (socket says 172.21.15.67:33362) 2024-07-31T15:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba55553640 1 -- 172.21.15.67:0/705329445 learned_addr learned my addr 172.21.15.67:0/705329445 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba4ffff640 1 --2- 172.21.15.67:0/705329445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdde44d0 0x5587be1b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba55553640 1 -- 172.21.15.67:0/705329445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdde44d0 msgr2=0x5587be1b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba55553640 1 --2- 172.21.15.67:0/705329445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdde44d0 0x5587be1b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba55553640 1 -- 172.21.15.67:0/705329445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 msgr2=0x5587be1b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba55553640 1 --2- 172.21.15.67:0/705329445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.153+0000 7fba55553640 1 -- 172.21.15.67:0/705329445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587be19fd60 con 0x5587bdddc2f0 2024-07-31T15:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.154+0000 7fba55553640 1 --2- 172.21.15.67:0/705329445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1a96d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fba48014040 tx=0x7fba48009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bab763a1d123593 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.154+0000 7fba3a7fc640 1 -- 172.21.15.67:0/705329445 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48025040 con 0x5587bdddc2f0 2024-07-31T15:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.154+0000 7fba3a7fc640 1 -- 172.21.15.67:0/705329445 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba4800fce0 con 0x5587bdddc2f0 2024-07-31T15:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.154+0000 7fba3a7fc640 1 -- 172.21.15.67:0/705329445 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4802f660 con 0x5587bdddc2f0 2024-07-31T15:13:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.155+0000 7fba5df86740 1 -- 172.21.15.67:0/705329445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 msgr2=0x5587be1a96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.155+0000 7fba5df86740 1 --2- 172.21.15.67:0/705329445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1a96d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fba48014040 tx=0x7fba48009f90 comp rx=0 tx=0).stop 2024-07-31T15:13:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.155+0000 7fba5df86740 1 -- 172.21.15.67:0/705329445 shutdown_connections 2024-07-31T15:13:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.155+0000 7fba5df86740 1 --2- 172.21.15.67:0/705329445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bde051b0 0x5587be1b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.155+0000 7fba5df86740 1 --2- 172.21.15.67:0/705329445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bdde44d0 0x5587be1b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.155+0000 7fba5df86740 1 --2- 172.21.15.67:0/705329445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1a96d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.155+0000 7fba5df86740 1 -- 172.21.15.67:0/705329445 >> 172.21.15.67:0/705329445 conn(0x5587bddf9300 msgr2=0x5587be1b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.155+0000 7fba5df86740 1 -- 172.21.15.67:0/705329445 shutdown_connections 2024-07-31T15:13:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.155+0000 7fba5df86740 1 -- 172.21.15.67:0/705329445 wait complete. 2024-07-31T15:13:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.155+0000 7fba5df86740 1 -- start start 2024-07-31T15:13:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.155+0000 7fba5df86740 1 Processor -- start 2024-07-31T15:13:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.155+0000 7fba5df86740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.155+0000 7fba5df86740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bdde44d0 0x5587be1ba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba5df86740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bde051b0 0x5587be223fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba5df86740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fba501b3f00 con 0x5587bdde44d0 2024-07-31T15:13:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba5df86740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fba501b3830 con 0x5587bdddc2f0 2024-07-31T15:13:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba5df86740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba501b0a90 con 0x5587bde051b0 2024-07-31T15:13:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba5df86740 10 client.0 _ll_register_callbacks cb 0x7fba3c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba55553640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba55553640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1ba090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:33378/0 (socket says 172.21.15.67:33378) 2024-07-31T15:13:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba55553640 1 -- 172.21.15.67:0/2167320031 learned_addr learned my addr 172.21.15.67:0/2167320031 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba4ffff640 1 --2- 172.21.15.67:0/2167320031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bdde44d0 0x5587be1ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba55553640 1 --2- 172.21.15.67:0/2167320031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bde051b0 0x5587be223fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba4ffff640 1 -- 172.21.15.67:0/2167320031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bde051b0 msgr2=0x5587be223fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba4ffff640 1 --2- 172.21.15.67:0/2167320031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bde051b0 0x5587be223fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba4ffff640 1 -- 172.21.15.67:0/2167320031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 msgr2=0x5587be1ba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba4ffff640 1 --2- 172.21.15.67:0/2167320031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1ba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba4ffff640 1 -- 172.21.15.67:0/2167320031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587be224620 con 0x5587bdde44d0 2024-07-31T15:13:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba4ffff640 1 --2- 172.21.15.67:0/2167320031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bdde44d0 0x5587be1ba630 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fba3c010880 tx=0x7fba3c012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220674a7238fd379 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:13:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba2ffff640 1 -- 172.21.15.67:0/2167320031 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c022040 con 0x5587bdde44d0 2024-07-31T15:13:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba2ffff640 1 -- 172.21.15.67:0/2167320031 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7fba3c010a70 con 0x5587bdde44d0 2024-07-31T15:13:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.156+0000 7fba5df86740 1 -- 172.21.15.67:0/2167320031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba4800fce0 con 0x5587bdde44d0 2024-07-31T15:13:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.157+0000 7fba2ffff640 1 -- 172.21.15.67:0/2167320031 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fba3c03a050 con 0x5587bdde44d0 2024-07-31T15:13:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.157+0000 7fba5df86740 10 client.20142 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.157+0000 7fba5df86740 1 -- 172.21.15.67:0/2167320031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5587be22cb10 con 0x5587bdde44d0 2024-07-31T15:13:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.157+0000 7fba2ffff640 1 -- 172.21.15.67:0/2167320031 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba4800fce0 con 0x5587bdde44d0 2024-07-31T15:13:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.157+0000 7fba2ffff640 1 -- 172.21.15.67:0/2167320031 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fba3c003710 con 0x5587bdde44d0 2024-07-31T15:13:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.157+0000 7fba5df86740 10 client.20142 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.157+0000 7fba5df86740 10 client.20142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.157+0000 7fba5df86740 20 client.20142 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.157+0000 7fba5df86740 1 --2- 172.21.15.67:0/2167320031 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5587bdd991c0 0x5587be22f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.157+0000 7fba5df86740 4 client.20142 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:13:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.157+0000 7fba5df86740 1 -- 172.21.15.67:0/2167320031 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587be11ada0 con 0x5587bdd991c0 2024-07-31T15:13:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.157+0000 7fba55553640 1 --2- 172.21.15.67:0/2167320031 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5587bdd991c0 0x5587be22f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.158+0000 7fba2ffff640 10 client.20142 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:14.158+0000 7fba55553640 1 --2- 172.21.15.67:0/2167320031 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5587bdd991c0 0x5587be22f240 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccd64b185e62a12f server_cookie=516250da55cc88bd in_seq=0 out_seq=0 2024-07-31T15:13:15.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:15 smithi067 ceph-mon[32750]: pgmap v1108: 129 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 97 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.6 MiB/s wr, 281 op/s 2024-07-31T15:13:15.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:15 smithi077 ceph-mon[37327]: pgmap v1108: 129 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 97 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.6 MiB/s wr, 281 op/s 2024-07-31T15:13:15.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.865+0000 7fb19ff56640 1 -- 172.21.15.67:0/2585416791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198107dd0 msgr2=0x7fb1981081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:15.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.865+0000 7fb19ff56640 1 --2- 172.21.15.67:0/2585416791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198107dd0 0x7fb1981081e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fb188009900 tx=0x7fb18802a410 comp rx=0 tx=0).stop 2024-07-31T15:13:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.865+0000 7fb19ff56640 1 -- 172.21.15.67:0/2585416791 shutdown_connections 2024-07-31T15:13:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.865+0000 7fb19ff56640 1 --2- 172.21.15.67:0/2585416791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb198108720 0x7fb19810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.865+0000 7fb19ff56640 1 --2- 172.21.15.67:0/2585416791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198107dd0 0x7fb1981081e0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.865+0000 7fb19ff56640 1 --2- 172.21.15.67:0/2585416791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb198102490 0x7fb1981028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.865+0000 7fb19ff56640 1 -- 172.21.15.67:0/2585416791 >> 172.21.15.67:0/2585416791 conn(0x7fb198079160 msgr2=0x7fb198079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.866+0000 7fb19ff56640 1 -- 172.21.15.67:0/2585416791 shutdown_connections 2024-07-31T15:13:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.866+0000 7fb19ff56640 1 -- 172.21.15.67:0/2585416791 wait complete. 2024-07-31T15:13:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19ff56640 1 Processor -- start 2024-07-31T15:13:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19ff56640 1 -- start start 2024-07-31T15:13:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19ff56640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb198102490 0x7fb198199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19ff56640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198107dd0 0x7fb19819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19ff56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb198108720 0x7fb19819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19ff56640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb1981134b0 con 0x7fb198107dd0 2024-07-31T15:13:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19ff56640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb198113330 con 0x7fb198102490 2024-07-31T15:13:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19ff56640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb198113630 con 0x7fb198108720 2024-07-31T15:13:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19d4f1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198107dd0 0x7fb19819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19dcf2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb198102490 0x7fb198199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:15.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19d4f1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198107dd0 0x7fb19819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56148/0 (socket says 172.21.15.67:56148) 2024-07-31T15:13:15.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19d4f1640 1 -- 172.21.15.67:0/1225644155 learned_addr learned my addr 172.21.15.67:0/1225644155 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:15.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19e4f3640 1 --2- 172.21.15.67:0/1225644155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb198108720 0x7fb19819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:15.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19d4f1640 1 -- 172.21.15.67:0/1225644155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb198108720 msgr2=0x7fb19819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:15.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19d4f1640 1 --2- 172.21.15.67:0/1225644155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb198108720 0x7fb19819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19d4f1640 1 -- 172.21.15.67:0/1225644155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb198102490 msgr2=0x7fb198199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:15.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19d4f1640 1 --2- 172.21.15.67:0/1225644155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb198102490 0x7fb198199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19d4f1640 1 -- 172.21.15.67:0/1225644155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19819fe10 con 0x7fb198107dd0 2024-07-31T15:13:15.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.867+0000 7fb19dcf2640 1 --2- 172.21.15.67:0/1225644155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb198102490 0x7fb198199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:15.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.868+0000 7fb19d4f1640 1 --2- 172.21.15.67:0/1225644155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198107dd0 0x7fb19819a440 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fb188006010 tx=0x7fb188004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:15.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.868+0000 7fb18effd640 1 -- 172.21.15.67:0/1225644155 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18803f030 con 0x7fb198107dd0 2024-07-31T15:13:15.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.868+0000 7fb19ff56640 1 -- 172.21.15.67:0/1225644155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1981a00a0 con 0x7fb198107dd0 2024-07-31T15:13:15.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.868+0000 7fb18effd640 1 -- 172.21.15.67:0/1225644155 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb188003aa0 con 0x7fb198107dd0 2024-07-31T15:13:15.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.868+0000 7fb18effd640 1 -- 172.21.15.67:0/1225644155 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188035070 con 0x7fb198107dd0 2024-07-31T15:13:15.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.868+0000 7fb19ff56640 1 -- 172.21.15.67:0/1225644155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1981a6860 con 0x7fb198107dd0 2024-07-31T15:13:15.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb1a64a7740 1 Processor -- start 2024-07-31T15:13:15.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb1a64a7740 1 -- start start 2024-07-31T15:13:15.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb18effd640 1 -- 172.21.15.67:0/1225644155 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb188042030 con 0x7fb198107dd0 2024-07-31T15:13:15.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb1a64a7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 0x55b9596803b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:15.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb1a64a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592b2490 0x55b9596808f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:15.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb1a64a7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959680e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:15.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb1a64a7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b9595682f0 con 0x55b9592ba670 2024-07-31T15:13:15.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb1a64a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b9595f6a80 con 0x55b9592b2490 2024-07-31T15:13:15.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb1a64a7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b9595e5f80 con 0x55b9592db350 2024-07-31T15:13:15.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb19e4f3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 0x55b9596803b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:15.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb19d4f1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959680e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:15.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb19dcf2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592b2490 0x55b9596808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:15.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb19e4f3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 0x55b9596803b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:59744/0 (socket says 172.21.15.67:59744) 2024-07-31T15:13:15.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb19e4f3640 1 -- 172.21.15.67:0/1192288934 learned_addr learned my addr 172.21.15.67:0/1192288934 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:15.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb19dcf2640 1 -- 172.21.15.67:0/1192288934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 msgr2=0x55b9596803b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:15.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb19dcf2640 1 --2- 172.21.15.67:0/1192288934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 0x55b9596803b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.871+0000 7fb19dcf2640 1 -- 172.21.15.67:0/1192288934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 msgr2=0x55b959680e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:15.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.872+0000 7fb19dcf2640 1 --2- 172.21.15.67:0/1192288934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959680e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.872+0000 7fb19dcf2640 1 -- 172.21.15.67:0/1192288934 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b959346930 con 0x55b9592b2490 2024-07-31T15:13:15.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.872+0000 7fb18effd640 1 --2- 172.21.15.67:0/1225644155 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb1840b3d40 0x7fb1840b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:15.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.872+0000 7fb19dcf2640 1 --2- 172.21.15.67:0/1192288934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592b2490 0x55b9596808f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb194009fd0 tx=0x7fb1940118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4a22e72f4a9b0d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:15.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.872+0000 7fb19e4f3640 1 --2- 172.21.15.67:0/1225644155 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb1840b3d40 0x7fb1840b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:15.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.872+0000 7fb18effd640 1 -- 172.21.15.67:0/1225644155 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7fb1881044f0 con 0x7fb198107dd0 2024-07-31T15:13:15.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.872+0000 7fb183fff640 1 -- 172.21.15.67:0/1192288934 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194004030 con 0x55b9592b2490 2024-07-31T15:13:15.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.872+0000 7fb183fff640 1 -- 172.21.15.67:0/1192288934 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1940054d0 con 0x55b9592b2490 2024-07-31T15:13:15.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.872+0000 7fb183fff640 1 -- 172.21.15.67:0/1192288934 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194004d40 con 0x55b9592b2490 2024-07-31T15:13:15.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.873+0000 7fb1a64a7740 1 -- 172.21.15.67:0/1192288934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592b2490 msgr2=0x55b9596808f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:15.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.873+0000 7fb19e4f3640 1 --2- 172.21.15.67:0/1225644155 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb1840b3d40 0x7fb1840b6250 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb190005c40 tx=0x7fb1900097e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:15.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.873+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/1192288934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592b2490 0x55b9596808f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb194009fd0 tx=0x7fb1940118d0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.873+0000 7fb1a64a7740 1 -- 172.21.15.67:0/1192288934 shutdown_connections 2024-07-31T15:13:15.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.873+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/1192288934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959680e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.873+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/1192288934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592b2490 0x55b9596808f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.873+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/1192288934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 0x55b9596803b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.873+0000 7fb1a64a7740 1 -- 172.21.15.67:0/1192288934 >> 172.21.15.67:0/1192288934 conn(0x55b9592cf740 msgr2=0x55b95967e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:15.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.873+0000 7fb1a64a7740 1 -- 172.21.15.67:0/1192288934 shutdown_connections 2024-07-31T15:13:15.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.873+0000 7fb1a64a7740 1 -- 172.21.15.67:0/1192288934 wait complete. 2024-07-31T15:13:15.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.874+0000 7fb1a64a7740 1 -- start start 2024-07-31T15:13:15.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.874+0000 7fb1a64a7740 1 Processor -- start 2024-07-31T15:13:15.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.874+0000 7fb1a64a7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592b2490 0x55b959690320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:15.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.874+0000 7fb1a64a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592ba670 0x55b959690860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:15.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.874+0000 7fb1a64a7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 0x55b95969c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:15.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.874+0000 7fb1a64a7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b9595884f0 con 0x55b9592b2490 2024-07-31T15:13:15.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.874+0000 7fb1a64a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b9595e9fc0 con 0x55b9592ba670 2024-07-31T15:13:15.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.874+0000 7fb1a64a7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b959449890 con 0x55b9592db350 2024-07-31T15:13:15.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.874+0000 7fb1a64a7740 10 client.0 _ll_register_callbacks cb 0x55b959592500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:15.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.874+0000 7fb19dcf2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592b2490 0x55b959690320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:15.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.874+0000 7fb19d4f1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592ba670 0x55b959690860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:15.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.874+0000 7fb19dcf2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592b2490 0x55b959690320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56166/0 (socket says 172.21.15.67:56166) 2024-07-31T15:13:15.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.874+0000 7fb19dcf2640 1 -- 172.21.15.67:0/1221036629 learned_addr learned my addr 172.21.15.67:0/1221036629 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:15.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.874+0000 7fb19e4f3640 1 --2- 172.21.15.67:0/1221036629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 0x55b95969c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:15.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.875+0000 7fb19d4f1640 1 -- 172.21.15.67:0/1221036629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 msgr2=0x55b95969c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:15.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.875+0000 7fb19d4f1640 1 --2- 172.21.15.67:0/1221036629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 0x55b95969c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.875+0000 7fb19d4f1640 1 -- 172.21.15.67:0/1221036629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592b2490 msgr2=0x55b959690320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:15.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.875+0000 7fb19d4f1640 1 --2- 172.21.15.67:0/1221036629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592b2490 0x55b959690320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.875+0000 7fb19d4f1640 1 -- 172.21.15.67:0/1221036629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b959597c50 con 0x55b9592ba670 2024-07-31T15:13:15.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.875+0000 7fb19d4f1640 1 --2- 172.21.15.67:0/1221036629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592ba670 0x55b959690860 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb188101880 tx=0x7fb188009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47ad664f737b919a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:15.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.875+0000 7fb1817fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:13:15.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.875+0000 7fb1817fa640 1 -- 172.21.15.67:0/1221036629 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188049050 con 0x55b9592ba670 2024-07-31T15:13:15.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.875+0000 7fb1817fa640 1 -- 172.21.15.67:0/1221036629 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7fb1880c6fd0 con 0x55b9592ba670 2024-07-31T15:13:15.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.875+0000 7fb1a64a7740 1 -- 172.21.15.67:0/1221036629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1940054d0 con 0x55b9592ba670 2024-07-31T15:13:15.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.875+0000 7fb1817fa640 1 -- 172.21.15.67:0/1221036629 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fb1880cab10 con 0x55b9592ba670 2024-07-31T15:13:15.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.875+0000 7fb1a64a7740 10 client.29675 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:15.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.875+0000 7fb1a64a7740 1 -- 172.21.15.67:0/1221036629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55b95969ecb0 con 0x55b9592ba670 2024-07-31T15:13:15.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.875+0000 7fb1817fa640 1 -- 172.21.15.67:0/1221036629 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1940054d0 con 0x55b9592ba670 2024-07-31T15:13:15.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.876+0000 7fb1817fa640 1 -- 172.21.15.67:0/1221036629 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fb18804c060 con 0x55b9592ba670 2024-07-31T15:13:15.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.876+0000 7fb1a64a7740 10 client.29675 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:15.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.876+0000 7fb1a64a7740 10 client.29675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:15.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.876+0000 7fb1a64a7740 20 client.29675 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:15.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.876+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/1221036629 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b959299cd0 0x55b9596a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:15.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.876+0000 7fb1a64a7740 4 client.29675 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:13:15.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.876+0000 7fb1a64a7740 1 -- 172.21.15.67:0/1221036629 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9595f89d0 con 0x55b959299cd0 2024-07-31T15:13:15.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.876+0000 7fb19dcf2640 1 --2- 172.21.15.67:0/1221036629 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b959299cd0 0x55b9596a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:15.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.876+0000 7fb1817fa640 10 client.29675 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:15.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.876+0000 7fb19dcf2640 1 --2- 172.21.15.67:0/1221036629 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b959299cd0 0x55b9596a21f0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa55af04357ca52 server_cookie=8fd32872a0bea7ab in_seq=0 out_seq=0 2024-07-31T15:13:15.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:15 smithi073 ceph-mon[36714]: pgmap v1108: 129 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 97 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.6 MiB/s wr, 281 op/s 2024-07-31T15:13:15.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.986+0000 7fb1817fa640 1 -- 172.21.15.67:0/1221036629 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9595f89d0 con 0x55b959299cd0 2024-07-31T15:13:15.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.987+0000 7fb1817fa640 10 client.29675 handle_command_reply: tid=0 2024-07-31T15:13:15.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.987+0000 7fb1a64a7740 1 client.29675 shutdown 2024-07-31T15:13:15.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.987+0000 7fb1a64a7740 1 -- 172.21.15.67:0/1221036629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592ba670 msgr2=0x55b959690860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:15.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.987+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/1221036629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592ba670 0x55b959690860 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb188101880 tx=0x7fb188009260 comp rx=0 tx=0).stop 2024-07-31T15:13:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.987+0000 7fb1a64a7740 1 -- 172.21.15.67:0/1221036629 shutdown_connections 2024-07-31T15:13:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.987+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/1221036629 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b959299cd0 0x55b9596a21f0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.987+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/1221036629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 0x55b95969c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.987+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/1221036629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592ba670 0x55b959690860 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.987+0000 7fb1817fa640 0 client.29675 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.987+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/1221036629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592b2490 0x55b959690320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.987+0000 7fb1a64a7740 1 -- 172.21.15.67:0/1221036629 >> 172.21.15.67:0/1221036629 conn(0x55b9592cf740 msgr2=0x55b959681da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.987+0000 7fb1a64a7740 1 -- 172.21.15.67:0/1221036629 shutdown_connections 2024-07-31T15:13:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.987+0000 7fb1a64a7740 1 -- 172.21.15.67:0/1221036629 wait complete. 2024-07-31T15:13:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:15.988+0000 7fb1a64a7740 20 client.29675 trim_cache size 0 max 16384 2024-07-31T15:13:16.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.006+0000 7fb1a64a7740 1 Processor -- start 2024-07-31T15:13:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.006+0000 7fb1a64a7740 1 -- start start 2024-07-31T15:13:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.006+0000 7fb1a64a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592b2490 0x55b95968b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.006+0000 7fb1a64a7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959681af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.006+0000 7fb1a64a7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 0x55b95969be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.006+0000 7fb1a64a7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb1981ae730 con 0x55b9592ba670 2024-07-31T15:13:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.006+0000 7fb1a64a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1981b0270 con 0x55b9592b2490 2024-07-31T15:13:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.006+0000 7fb1a64a7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1981aee00 con 0x55b9592db350 2024-07-31T15:13:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.006+0000 7fb19d4f1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959681af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.006+0000 7fb19d4f1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959681af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56174/0 (socket says 172.21.15.67:56174) 2024-07-31T15:13:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.006+0000 7fb19d4f1640 1 -- 172.21.15.67:0/3579929026 learned_addr learned my addr 172.21.15.67:0/3579929026 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.006+0000 7fb19dcf2640 1 --2- 172.21.15.67:0/3579929026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592b2490 0x55b95968b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.007+0000 7fb19d4f1640 1 -- 172.21.15.67:0/3579929026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 msgr2=0x55b95969be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:16.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.007+0000 7fb19e4f3640 1 --2- 172.21.15.67:0/3579929026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 0x55b95969be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.007+0000 7fb19d4f1640 1 --2- 172.21.15.67:0/3579929026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 0x55b95969be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.007+0000 7fb19d4f1640 1 -- 172.21.15.67:0/3579929026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592b2490 msgr2=0x55b95968b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.007+0000 7fb19d4f1640 1 --2- 172.21.15.67:0/3579929026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592b2490 0x55b95968b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.007+0000 7fb19d4f1640 1 -- 172.21.15.67:0/3579929026 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b95968f080 con 0x55b9592ba670 2024-07-31T15:13:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.007+0000 7fb19dcf2640 1 --2- 172.21.15.67:0/3579929026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592b2490 0x55b95968b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.007+0000 7fb19e4f3640 1 --2- 172.21.15.67:0/3579929026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 0x55b95969be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:13:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.007+0000 7fb19d4f1640 1 --2- 172.21.15.67:0/3579929026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959681af0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb1880c6dc0 tx=0x7fb188000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f34a4ead59d44c6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.007+0000 7fb1837fe640 1 -- 172.21.15.67:0/3579929026 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188049070 con 0x55b9592ba670 2024-07-31T15:13:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.007+0000 7fb1837fe640 1 -- 172.21.15.67:0/3579929026 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1880ca850 con 0x55b9592ba670 2024-07-31T15:13:16.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.007+0000 7fb1837fe640 1 -- 172.21.15.67:0/3579929026 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1880ca4b0 con 0x55b9592ba670 2024-07-31T15:13:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.008+0000 7fb1a64a7740 1 -- 172.21.15.67:0/3579929026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 msgr2=0x55b959681af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.008+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/3579929026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959681af0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb1880c6dc0 tx=0x7fb188000bc0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.008+0000 7fb1a64a7740 1 -- 172.21.15.67:0/3579929026 shutdown_connections 2024-07-31T15:13:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.008+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/3579929026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592db350 0x55b95969be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.008+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/3579929026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959681af0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.008+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/3579929026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592b2490 0x55b95968b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.008+0000 7fb1a64a7740 1 -- 172.21.15.67:0/3579929026 >> 172.21.15.67:0/3579929026 conn(0x55b9592cf740 msgr2=0x55b9596813e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.008+0000 7fb1a64a7740 1 -- 172.21.15.67:0/3579929026 shutdown_connections 2024-07-31T15:13:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.008+0000 7fb1a64a7740 1 -- 172.21.15.67:0/3579929026 wait complete. 2024-07-31T15:13:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb1a64a7740 1 -- start start 2024-07-31T15:13:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb1a64a7740 1 Processor -- start 2024-07-31T15:13:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb1a64a7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592b2490 0x55b959694510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb1a64a7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959694a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb1a64a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592db350 0x55b959709980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb1a64a7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb1981afba0 con 0x55b9592ba670 2024-07-31T15:13:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb1a64a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1981af4d0 con 0x55b9592db350 2024-07-31T15:13:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb1a64a7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1981ac730 con 0x55b9592b2490 2024-07-31T15:13:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb1a64a7740 10 client.0 _ll_register_callbacks cb 0x7fb188004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb19d4f1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959694a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb19d4f1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959694a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56180/0 (socket says 172.21.15.67:56180) 2024-07-31T15:13:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb19d4f1640 1 -- 172.21.15.67:0/3848294243 learned_addr learned my addr 172.21.15.67:0/3848294243 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb19e4f3640 1 --2- 172.21.15.67:0/3848294243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592db350 0x55b959709980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb19dcf2640 1 --2- 172.21.15.67:0/3848294243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592b2490 0x55b959694510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb19d4f1640 1 -- 172.21.15.67:0/3848294243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592b2490 msgr2=0x55b959694510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb19d4f1640 1 --2- 172.21.15.67:0/3848294243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592b2490 0x55b959694510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb19d4f1640 1 -- 172.21.15.67:0/3848294243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592db350 msgr2=0x55b959709980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.009+0000 7fb19d4f1640 1 --2- 172.21.15.67:0/3848294243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592db350 0x55b959709980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.010+0000 7fb19d4f1640 1 -- 172.21.15.67:0/3848294243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b959709ec0 con 0x55b9592ba670 2024-07-31T15:13:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.010+0000 7fb19dcf2640 1 --2- 172.21.15.67:0/3848294243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592b2490 0x55b959694510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:13:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.010+0000 7fb19d4f1640 1 --2- 172.21.15.67:0/3848294243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959694a50 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb1880cad60 tx=0x7fb188009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bce1150dc6ec8c5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.010+0000 7fb1817fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:13:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.010+0000 7fb1817fa640 1 -- 172.21.15.67:0/3848294243 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1880e9030 con 0x55b9592ba670 2024-07-31T15:13:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.010+0000 7fb1817fa640 1 -- 172.21.15.67:0/3848294243 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7fb1880c7330 con 0x55b9592ba670 2024-07-31T15:13:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.010+0000 7fb1a64a7740 1 -- 172.21.15.67:0/3848294243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1880ca850 con 0x55b9592ba670 2024-07-31T15:13:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.010+0000 7fb1817fa640 1 -- 172.21.15.67:0/3848294243 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fb188049040 con 0x55b9592ba670 2024-07-31T15:13:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.010+0000 7fb1a64a7740 10 client.20160 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.010+0000 7fb1a64a7740 1 -- 172.21.15.67:0/3848294243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55b959709ec0 con 0x55b9592ba670 2024-07-31T15:13:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.010+0000 7fb1817fa640 1 -- 172.21.15.67:0/3848294243 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1880ca850 con 0x55b9592ba670 2024-07-31T15:13:16.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.011+0000 7fb1817fa640 1 -- 172.21.15.67:0/3848294243 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fb1880cafa0 con 0x55b9592ba670 2024-07-31T15:13:16.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.011+0000 7fb1a64a7740 10 client.20160 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:16.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.011+0000 7fb1a64a7740 10 client.20160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:16.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.011+0000 7fb1a64a7740 20 client.20160 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:16.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.011+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/3848294243 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b95926f080 0x55b95970d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:16.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.011+0000 7fb1a64a7740 4 client.20160 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T15:13:16.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.011+0000 7fb1a64a7740 1 -- 172.21.15.67:0/3848294243 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b9595f4470 con 0x55b95926f080 2024-07-31T15:13:16.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.011+0000 7fb19dcf2640 1 --2- 172.21.15.67:0/3848294243 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b95926f080 0x55b95970d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:16.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.011+0000 7fb19dcf2640 1 --2- 172.21.15.67:0/3848294243 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b95926f080 0x55b95970d3f0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d4f8f216f3f4cf7 server_cookie=22335479a7113e00 in_seq=0 out_seq=0 2024-07-31T15:13:16.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.011+0000 7fb1817fa640 10 client.20160 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:16.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.706+0000 7fb1817fa640 1 -- 172.21.15.67:0/3848294243 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b9595f4470 con 0x55b95926f080 2024-07-31T15:13:16.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.706+0000 7fb1817fa640 10 client.20160 handle_command_reply: tid=0 2024-07-31T15:13:16.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.706+0000 7fb1a64a7740 1 client.20160 shutdown 2024-07-31T15:13:16.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.706+0000 7fb1a64a7740 1 -- 172.21.15.67:0/3848294243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 msgr2=0x55b959694a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:16.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.706+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/3848294243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959694a50 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb1880cad60 tx=0x7fb188009680 comp rx=0 tx=0).stop 2024-07-31T15:13:16.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.706+0000 7fb1a64a7740 1 -- 172.21.15.67:0/3848294243 shutdown_connections 2024-07-31T15:13:16.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.706+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/3848294243 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55b95926f080 0x55b95970d3f0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.706+0000 7fb1817fa640 0 client.20160 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:16.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.706+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/3848294243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9592db350 0x55b959709980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.706+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/3848294243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b9592ba670 0x55b959694a50 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.706+0000 7fb1a64a7740 1 --2- 172.21.15.67:0/3848294243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9592b2490 0x55b959694510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.706+0000 7fb1a64a7740 1 -- 172.21.15.67:0/3848294243 >> 172.21.15.67:0/3848294243 conn(0x55b9592cf740 msgr2=0x55b95967e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:16.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.706+0000 7fb1a64a7740 1 -- 172.21.15.67:0/3848294243 shutdown_connections 2024-07-31T15:13:16.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.706+0000 7fb1a64a7740 1 -- 172.21.15.67:0/3848294243 wait complete. 2024-07-31T15:13:16.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.707+0000 7fb1a64a7740 20 client.20160 trim_cache size 0 max 16384 2024-07-31T15:13:16.709 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:13:16.710 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T15:13:16.710 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "d5a4ad48-412b-4b1e-a8b7-c6af766e4c13", 2024-07-31T15:13:16.710 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T15:13:16.710 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:13:16.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.707+0000 7fb19ff56640 1 -- 172.21.15.67:0/1225644155 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb1840b3d40 msgr2=0x7fb1840b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:16.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.707+0000 7fb19ff56640 1 --2- 172.21.15.67:0/1225644155 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb1840b3d40 0x7fb1840b6250 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb190005c40 tx=0x7fb1900097e0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.707+0000 7fb19ff56640 1 -- 172.21.15.67:0/1225644155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198107dd0 msgr2=0x7fb19819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:16.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.707+0000 7fb19ff56640 1 --2- 172.21.15.67:0/1225644155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198107dd0 0x7fb19819a440 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fb188006010 tx=0x7fb188004060 comp rx=0 tx=0).stop 2024-07-31T15:13:16.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.707+0000 7fb19ff56640 1 -- 172.21.15.67:0/1225644155 shutdown_connections 2024-07-31T15:13:16.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.707+0000 7fb19ff56640 1 --2- 172.21.15.67:0/1225644155 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fb1840b3d40 0x7fb1840b6250 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.707+0000 7fb19ff56640 1 --2- 172.21.15.67:0/1225644155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb198108720 0x7fb19819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.707+0000 7fb19ff56640 1 --2- 172.21.15.67:0/1225644155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198107dd0 0x7fb19819a440 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.707+0000 7fb19ff56640 1 --2- 172.21.15.67:0/1225644155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb198102490 0x7fb198199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:16.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.707+0000 7fb19ff56640 1 -- 172.21.15.67:0/1225644155 >> 172.21.15.67:0/1225644155 conn(0x7fb198079160 msgr2=0x7fb198104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:16.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.707+0000 7fb19ff56640 1 -- 172.21.15.67:0/1225644155 shutdown_connections 2024-07-31T15:13:16.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:16.707+0000 7fb19ff56640 1 -- 172.21.15.67:0/1225644155 wait complete. 2024-07-31T15:13:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.079+0000 7fba2ffff640 1 -- 172.21.15.67:0/2167320031 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5587be11ada0 con 0x5587bdd991c0 2024-07-31T15:13:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.079+0000 7fba2ffff640 10 client.20142 handle_command_reply: tid=0 2024-07-31T15:13:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.079+0000 7fba5df86740 1 client.20142 shutdown 2024-07-31T15:13:17.081 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.079+0000 7fba5df86740 1 -- 172.21.15.67:0/2167320031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bdde44d0 msgr2=0x5587be1ba630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.079+0000 7fba5df86740 1 --2- 172.21.15.67:0/2167320031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bdde44d0 0x5587be1ba630 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fba3c010880 tx=0x7fba3c012a00 comp rx=0 tx=0).stop 2024-07-31T15:13:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.079+0000 7fba5df86740 1 -- 172.21.15.67:0/2167320031 shutdown_connections 2024-07-31T15:13:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.079+0000 7fba5df86740 1 --2- 172.21.15.67:0/2167320031 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5587bdd991c0 0x5587be22f240 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.079+0000 7fba5df86740 1 --2- 172.21.15.67:0/2167320031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587bde051b0 0x5587be223fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.079+0000 7fba5df86740 1 --2- 172.21.15.67:0/2167320031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587bdde44d0 0x5587be1ba630 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.079+0000 7fba2ffff640 0 client.20142 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.079+0000 7fba5df86740 1 --2- 172.21.15.67:0/2167320031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587bdddc2f0 0x5587be1ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.079+0000 7fba5df86740 1 -- 172.21.15.67:0/2167320031 >> 172.21.15.67:0/2167320031 conn(0x5587bddf9300 msgr2=0x5587be1b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.080+0000 7fba5df86740 1 -- 172.21.15.67:0/2167320031 shutdown_connections 2024-07-31T15:13:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.080+0000 7fba5df86740 1 -- 172.21.15.67:0/2167320031 wait complete. 2024-07-31T15:13:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.080+0000 7fba5df86740 20 client.20142 trim_cache size 0 max 16384 2024-07-31T15:13:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:13:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:13:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:13:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:13:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.080+0000 7fba56fb6640 1 -- 172.21.15.67:0/1884890008 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fba400b3d70 msgr2=0x7fba400b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.080+0000 7fba56fb6640 1 --2- 172.21.15.67:0/1884890008 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fba400b3d70 0x7fba400b6280 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fba44101b00 tx=0x7fba4410a550 comp rx=0 tx=0).stop 2024-07-31T15:13:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.080+0000 7fba56fb6640 1 -- 172.21.15.67:0/1884890008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba501023f0 msgr2=0x7fba500fe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.080+0000 7fba56fb6640 1 --2- 172.21.15.67:0/1884890008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba501023f0 0x7fba500fe320 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fba44005940 tx=0x7fba440040c0 comp rx=0 tx=0).stop 2024-07-31T15:13:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.080+0000 7fba56fb6640 1 -- 172.21.15.67:0/1884890008 shutdown_connections 2024-07-31T15:13:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.080+0000 7fba56fb6640 1 --2- 172.21.15.67:0/1884890008 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fba400b3d70 0x7fba400b6280 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.080+0000 7fba56fb6640 1 --2- 172.21.15.67:0/1884890008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba50108670 0x7fba500feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.080+0000 7fba56fb6640 1 --2- 172.21.15.67:0/1884890008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba50107d20 0x7fba500fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.080+0000 7fba56fb6640 1 --2- 172.21.15.67:0/1884890008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba501023f0 0x7fba500fe320 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.080+0000 7fba56fb6640 1 -- 172.21.15.67:0/1884890008 >> 172.21.15.67:0/1884890008 conn(0x7fba500fddf0 msgr2=0x7fba501073b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.081+0000 7fba56fb6640 1 -- 172.21.15.67:0/1884890008 shutdown_connections 2024-07-31T15:13:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:17.081+0000 7fba56fb6640 1 -- 172.21.15.67:0/1884890008 wait complete. 2024-07-31T15:13:17.503 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:13:17.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:17 smithi077 ceph-mon[37327]: pgmap v1109: 129 pgs: 7 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 108 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 248 op/s 2024-07-31T15:13:17.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:17 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T15:13:17.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:17.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:17.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:17 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:17.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:17 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:17.781 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:13:17.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:17 smithi067 ceph-mon[32750]: pgmap v1109: 129 pgs: 7 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 108 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 248 op/s 2024-07-31T15:13:17.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:17 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T15:13:17.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:17.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:17.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:17 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:17.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:17 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:17.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:17 smithi073 ceph-mon[36714]: pgmap v1109: 129 pgs: 7 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 108 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 248 op/s 2024-07-31T15:13:17.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:17 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T15:13:17.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:17.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:17.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:17 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:17.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:17 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:18.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:18.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:18.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:18.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:18.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:18 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:18.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:18 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.741 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:18.741 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.741 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:18.741 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.742 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:18.742 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:18.742 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:18.742 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.742 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:18.743 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.743 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:18.743 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.743 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:18.743 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.743 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:18.743 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.744 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:18.744 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.744 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:18.744 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:18.744 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.058 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:18 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:18.738+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000162a [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch/ auth{1=1,2=1,3=1,4=1} v21244 ap=3 RANDEPHEMERALPIN f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22) n(v53 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939) old_inodes=7 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:593920 nref=4)) caps={15489=pAsLsXsFs/-@68} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7c26ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000162a:[<0x10000000202/arch v378>,<0x10000000201/linux-6.5.11 v96>,<0x10000000006/t v68>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v57>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v60>,<0x10000000001/sv_1 v84>,<0x10000000000/qa v70>,<0x1/volumes v70>]//[]","memoryvalue":"(2)0x1000000162a:[<0x10000000202/arch v21244>,<0x10000000201/linux-6.5.11 v2854>,<0x10000000006/t v1832>,<0x10000000005/tmp v1756>,<0x100000001fb/client.0 v1724>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1688>,<0x10000000001/sv_1 v1686>,<0x10000000000/qa v1672>,<0x1/volumes v1621>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:54:49.652308+0000 23=2+21)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939)","memory_value.dirstat":"f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22)","memory_value.rstat":"n(v53 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:19.326 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:19.056+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c28d [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/9p/ auth{3=1} v15462 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:10:57.696418+0000 31=31+0) n(v3 rc2024-07-31T15:10:57.699418+0000 b700067 32=31+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:595533 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@10} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8a0eb700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c28d:[<0x1000000c25e/9p v120>,<0x10000000202/net v7765>,<0x10000000201/linux-6.5.11 v689>,<0x10000000006/t v498>,<0x10000000005/tmp v466>,<0x100000001fb/client.0 v452>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v438>,<0x10000000001/sv_1 v453>,<0x10000000000/qa v441>,<0x1/volumes v433>]//[]","memoryvalue":"(2)0x1000000c28d:[<0x1000000c25e/9p v15462>,<0x10000000202/net v21981>,<0x10000000201/linux-6.5.11 v2854>,<0x10000000006/t v1832>,<0x10000000005/tmp v1756>,<0x100000001fb/client.0 v1724>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1688>,<0x10000000001/sv_1 v1686>,<0x10000000000/qa v1672>,<0x1/volumes v1621>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:49.314316+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:10:57.699418+0000 b700067 32=31+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:10:57.696418+0000 31=31+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:10:57.699418+0000 b700067 32=31+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:19.327 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:19.151+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c276 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802/ auth{3=1} v10568 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:29.002671+0000 14=14+0) n(v3 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:595567 nref=3)) caps={15489=pAsLsXs/-@18} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8fd23180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c276:[<0x1000000c25e/802 v68>,<0x10000000202/net v7765>,<0x10000000201/linux-6.5.11 v689>,<0x10000000006/t v498>,<0x10000000005/tmp v466>,<0x100000001fb/client.0 v452>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v438>,<0x10000000001/sv_1 v453>,<0x10000000000/qa v441>,<0x1/volumes v433>]//[]","memoryvalue":"(2)0x1000000c276:[<0x1000000c25e/802 v10568>,<0x10000000202/net v21981>,<0x10000000201/linux-6.5.11 v2854>,<0x10000000006/t v1832>,<0x10000000005/tmp v1756>,<0x100000001fb/client.0 v1724>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1688>,<0x10000000001/sv_1 v1686>,<0x10000000000/qa v1672>,<0x1/volumes v1621>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:49.293317+0000 10=10+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:29.002671+0000 14=14+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:19.327 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:19.227+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c836 [...1f,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet/ auth{4=1} v12171 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:57.081877+0000 11=11+0) n(v2 rc2024-07-31T15:08:57.083877+0000 b263808 12=11+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:595581 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@15} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8c20b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c836:[<0x1000000c25e/packet v6437>,<0x10000000202/net v8736>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000c836:[<0x1000000c25e/packet v12171>,<0x10000000202/net v21981>,<0x10000000201/linux-6.5.11 v2854>,<0x10000000006/t v1832>,<0x10000000005/tmp v1756>,<0x100000001fb/client.0 v1724>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1688>,<0x10000000001/sv_1 v1686>,<0x10000000000/qa v1672>,<0x1/volumes v1621>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:56.791162+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:57.083877+0000 b263808 12=11+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:57.081877+0000 11=11+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:08:57.083877+0000 b263808 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:19.327 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:19.230+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c9e2 [...1f,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix/ auth{2=1} v12135 ap=3 RANDEPHEMERALPIN f(v2 m2024-07-31T15:08:53.494950+0000 21=21+0) n(v5 rc2024-07-31T15:08:53.495950+0000 b445955 22=21+1) old_inodes=3 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:595583 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@18} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8591ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c9e2:[<0x1000000c25e/unix v6743>,<0x10000000202/net v8736>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000c9e2:[<0x1000000c25e/unix v12135>,<0x10000000202/net v21981>,<0x10000000201/linux-6.5.11 v2854>,<0x10000000006/t v1832>,<0x10000000005/tmp v1756>,<0x100000001fb/client.0 v1724>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1688>,<0x10000000001/sv_1 v1686>,<0x10000000000/qa v1672>,<0x1/volumes v1623>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:08:21.655599+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:53.495950+0000 b445955 22=21+1)","memory_value.dirstat":"f(v2 m2024-07-31T15:08:53.494950+0000 21=21+0)","memory_value.rstat":"n(v5 rc2024-07-31T15:08:53.495950+0000 b445955 22=21+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:19.327 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:19.233+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c7cb [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/netlabel/ auth{4=1} v15466 ap=3 RANDEPHEMERALPIN f(v2 m2024-07-31T15:10:46.034656+0000 37=37+0) n(v4 rc2024-07-31T15:10:46.036656+0000 b816547 38=37+1) old_inodes=3 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:595582 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@14} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce84a58580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c7cb:[<0x1000000c25e/netlabel v6433>,<0x10000000202/net v8736>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000c7cb:[<0x1000000c25e/netlabel v15466>,<0x10000000202/net v21981>,<0x10000000201/linux-6.5.11 v2854>,<0x10000000006/t v1832>,<0x10000000005/tmp v1756>,<0x100000001fb/client.0 v1724>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1688>,<0x10000000001/sv_1 v1686>,<0x10000000000/qa v1672>,<0x1/volumes v1621>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:10:43.762703+0000 29=29+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:10:46.036656+0000 b816547 38=37+1)","memory_value.dirstat":"f(v2 m2024-07-31T15:10:46.034656+0000 37=37+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:10:46.036656+0000 b816547 38=37+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:19.327 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:19.324+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d9b4 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb/ auth{3=1} v3876 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7) n(v5 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8) old_inodes=3 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:596364 nref=3)) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce92cb2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v2020>,<0x10000000202/sound v9936>,<0x10000000201/linux-6.5.11 v830>,<0x10000000006/t v586>,<0x10000000005/tmp v553>,<0x100000001fb/client.0 v537>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v519>,<0x10000000001/sv_1 v525>,<0x10000000000/qa v522>,<0x1/volumes v503>]//[]","memoryvalue":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v3876>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2854>,<0x10000000006/t v1832>,<0x10000000005/tmp v1756>,<0x100000001fb/client.0 v1724>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1688>,<0x10000000001/sv_1 v1686>,<0x10000000000/qa v1672>,<0x1/volumes v1623>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:46.317139+0000 53=46+7)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7)","memory_value.rstat":"n(v5 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: pgmap v1110: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2024-07-31T15:13:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c28d (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/9p) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c836 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c9e2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:19 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c7cb (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/netlabel) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:19.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: pgmap v1110: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2024-07-31T15:13:19.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:19.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:19.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:19.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:19.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:19.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:19.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:19.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c28d (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/9p) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c836 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c9e2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:19 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c7cb (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/netlabel) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:19.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: pgmap v1110: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2024-07-31T15:13:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c28d (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/9p) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c836 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c9e2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:19 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c7cb (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/netlabel) see mds.b log and `damage ls` output for details 2024-07-31T15:13:19.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.986+0000 7f45ef7d1640 1 -- 172.21.15.67:0/2008216378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45e8106610 msgr2=0x7f45e8110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:19.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.986+0000 7f45ef7d1640 1 --2- 172.21.15.67:0/2008216378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45e8106610 0x7f45e8110060 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009ca0 tx=0x7f45e002a8c0 comp rx=0 tx=0).stop 2024-07-31T15:13:19.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.986+0000 7f45ef7d1640 1 -- 172.21.15.67:0/2008216378 shutdown_connections 2024-07-31T15:13:19.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.986+0000 7f45ef7d1640 1 --2- 172.21.15.67:0/2008216378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45e8106610 0x7f45e8110060 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:19.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.986+0000 7f45ef7d1640 1 --2- 172.21.15.67:0/2008216378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45e8105cc0 0x7f45e81060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:19.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.986+0000 7f45ef7d1640 1 --2- 172.21.15.67:0/2008216378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45e81019e0 0x7f45e8101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:19.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.986+0000 7f45ef7d1640 1 -- 172.21.15.67:0/2008216378 >> 172.21.15.67:0/2008216378 conn(0x7f45e8077690 msgr2=0x7f45e8077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:19.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.986+0000 7f45ef7d1640 1 -- 172.21.15.67:0/2008216378 shutdown_connections 2024-07-31T15:13:19.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.986+0000 7f45ef7d1640 1 -- 172.21.15.67:0/2008216378 wait complete. 2024-07-31T15:13:19.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.987+0000 7f45ef7d1640 1 Processor -- start 2024-07-31T15:13:19.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.987+0000 7f45ef7d1640 1 -- start start 2024-07-31T15:13:19.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.987+0000 7f45ef7d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45e81019e0 0x7f45e8102980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:19.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.987+0000 7f45ef7d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45e8105cc0 0x7f45e8102ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:19.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.987+0000 7f45ef7d1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45e8106610 0x7f45e8103400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:19.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45ed56d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45e81019e0 0x7f45e8102980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:19.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45ef7d1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f45e81134d0 con 0x7f45e81019e0 2024-07-31T15:13:19.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45ecd6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45e8105cc0 0x7f45e8102ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:19.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45ed56d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45e81019e0 0x7f45e8102980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56200/0 (socket says 172.21.15.67:56200) 2024-07-31T15:13:19.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45ed56d640 1 -- 172.21.15.67:0/100492332 learned_addr learned my addr 172.21.15.67:0/100492332 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:19.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45ef7d1640 1 -- 172.21.15.67:0/100492332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f45e8113350 con 0x7f45e8106610 2024-07-31T15:13:19.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45ef7d1640 1 -- 172.21.15.67:0/100492332 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f45e8113650 con 0x7f45e8105cc0 2024-07-31T15:13:19.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45edd6e640 1 --2- 172.21.15.67:0/100492332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45e8106610 0x7f45e8103400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:19.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45edd6e640 1 -- 172.21.15.67:0/100492332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45e8105cc0 msgr2=0x7f45e8102ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:19.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45edd6e640 1 --2- 172.21.15.67:0/100492332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45e8105cc0 0x7f45e8102ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:19.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45edd6e640 1 -- 172.21.15.67:0/100492332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45e81019e0 msgr2=0x7f45e8102980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:19.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45edd6e640 1 --2- 172.21.15.67:0/100492332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45e81019e0 0x7f45e8102980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:19.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45edd6e640 1 -- 172.21.15.67:0/100492332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e81a8160 con 0x7f45e8106610 2024-07-31T15:13:19.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45ed56d640 1 --2- 172.21.15.67:0/100492332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45e81019e0 0x7f45e8102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:19.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.988+0000 7f45edd6e640 1 --2- 172.21.15.67:0/100492332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45e8106610 0x7f45e8103400 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f45e002ade0 tx=0x7f45e0031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:19.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.989+0000 7f45de7fc640 1 -- 172.21.15.67:0/100492332 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0003b70 con 0x7f45e8106610 2024-07-31T15:13:19.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.989+0000 7f45de7fc640 1 -- 172.21.15.67:0/100492332 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e0031910 con 0x7f45e8106610 2024-07-31T15:13:19.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.989+0000 7f45de7fc640 1 -- 172.21.15.67:0/100492332 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e002f950 con 0x7f45e8106610 2024-07-31T15:13:19.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.989+0000 7f45ef7d1640 1 -- 172.21.15.67:0/100492332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e81a83f0 con 0x7f45e8106610 2024-07-31T15:13:19.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.989+0000 7f45ef7d1640 1 -- 172.21.15.67:0/100492332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e81a88e0 con 0x7f45e8106610 2024-07-31T15:13:19.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.992+0000 7f45de7fc640 1 -- 172.21.15.67:0/100492332 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f45e002faf0 con 0x7f45e8106610 2024-07-31T15:13:19.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.992+0000 7f45f5d22740 1 Processor -- start 2024-07-31T15:13:19.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.992+0000 7f45f5d22740 1 -- start start 2024-07-31T15:13:19.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45f5d22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e7834070 0x5599e7bd9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:19.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45f5d22740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e780b1b0 0x5599e7bd9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:19.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45f5d22740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 0x5599e7bd9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:19.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45f5d22740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5599e7b4fb90 con 0x5599e780b1b0 2024-07-31T15:13:19.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45f5d22740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5599e7b3f090 con 0x5599e7813390 2024-07-31T15:13:19.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45f5d22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5599e7b430d0 con 0x5599e7834070 2024-07-31T15:13:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45ed56d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e780b1b0 0x5599e7bd9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45ecd6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e7834070 0x5599e7bd9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45ed56d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e780b1b0 0x5599e7bd9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56204/0 (socket says 172.21.15.67:56204) 2024-07-31T15:13:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45ed56d640 1 -- 172.21.15.67:0/1829200028 learned_addr learned my addr 172.21.15.67:0/1829200028 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45de7fc640 1 --2- 172.21.15.67:0/100492332 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f45d40b3d40 0x7f45d40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45ed56d640 1 -- 172.21.15.67:0/1829200028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e7834070 msgr2=0x5599e7bd9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45ed56d640 1 --2- 172.21.15.67:0/1829200028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e7834070 0x5599e7bd9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45ed56d640 1 -- 172.21.15.67:0/1829200028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 msgr2=0x5599e7bd9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45ecd6c640 1 --2- 172.21.15.67:0/1829200028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 0x5599e7bd9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45edd6e640 1 --2- 172.21.15.67:0/100492332 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f45d40b3d40 0x7f45d40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45ed56d640 1 --2- 172.21.15.67:0/1829200028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 0x5599e7bd9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45ed56d640 1 -- 172.21.15.67:0/1829200028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599e7ae5b70 con 0x5599e780b1b0 2024-07-31T15:13:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45de7fc640 1 -- 172.21.15.67:0/100492332 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7f45e0105580 con 0x7f45e8106610 2024-07-31T15:13:19.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.993+0000 7f45ecd6c640 1 --2- 172.21.15.67:0/1829200028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 0x5599e7bd9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:13:19.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.994+0000 7f45ed56d640 1 --2- 172.21.15.67:0/1829200028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e780b1b0 0x5599e7bd9a90 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f45e40115f0 tx=0x7f45e4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bc440ca9411b64 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:19.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.994+0000 7f45edd6e640 1 --2- 172.21.15.67:0/100492332 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f45d40b3d40 0x7f45d40b6250 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f45e0102ad0 tx=0x7f45e00c0760 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:19.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.994+0000 7f45d37fe640 1 -- 172.21.15.67:0/1829200028 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4004020 con 0x5599e780b1b0 2024-07-31T15:13:19.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.994+0000 7f45d37fe640 1 -- 172.21.15.67:0/1829200028 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e40054d0 con 0x5599e780b1b0 2024-07-31T15:13:19.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.995+0000 7f45d37fe640 1 -- 172.21.15.67:0/1829200028 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4004ca0 con 0x5599e780b1b0 2024-07-31T15:13:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.995+0000 7f45f5d22740 1 -- 172.21.15.67:0/1829200028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e780b1b0 msgr2=0x5599e7bd9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.995+0000 7f45f5d22740 1 --2- 172.21.15.67:0/1829200028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e780b1b0 0x5599e7bd9a90 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f45e40115f0 tx=0x7f45e4011ab0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.996+0000 7f45f5d22740 1 -- 172.21.15.67:0/1829200028 shutdown_connections 2024-07-31T15:13:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.996+0000 7f45f5d22740 1 --2- 172.21.15.67:0/1829200028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 0x5599e7bd9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.996+0000 7f45f5d22740 1 --2- 172.21.15.67:0/1829200028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e780b1b0 0x5599e7bd9a90 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.996+0000 7f45f5d22740 1 --2- 172.21.15.67:0/1829200028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e7834070 0x5599e7bd9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.996+0000 7f45f5d22740 1 -- 172.21.15.67:0/1829200028 >> 172.21.15.67:0/1829200028 conn(0x5599e7828460 msgr2=0x5599e7bd7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.996+0000 7f45f5d22740 1 -- 172.21.15.67:0/1829200028 shutdown_connections 2024-07-31T15:13:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.996+0000 7f45f5d22740 1 -- 172.21.15.67:0/1829200028 wait complete. 2024-07-31T15:13:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.996+0000 7f45f5d22740 1 -- start start 2024-07-31T15:13:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.996+0000 7f45f5d22740 1 Processor -- start 2024-07-31T15:13:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.997+0000 7f45f5d22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e7813390 0x5599e7be51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.997+0000 7f45f5d22740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7be57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.997+0000 7f45ecd6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e7813390 0x5599e7be51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.997+0000 7f45f5d22740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e77f29f0 0x5599e7bf1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.997+0000 7f45f5d22740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5599e79a2700 con 0x5599e7834070 2024-07-31T15:13:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.997+0000 7f45ecd6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e7813390 0x5599e7be51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:59810/0 (socket says 172.21.15.67:59810) 2024-07-31T15:13:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.997+0000 7f45f5d22740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5599e7ae19c0 con 0x5599e77f29f0 2024-07-31T15:13:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.997+0000 7f45f5d22740 1 -- 172.21.15.67:0/1429898766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5599e7be7d30 con 0x5599e7813390 2024-07-31T15:13:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.997+0000 7f45ecd6c640 1 -- 172.21.15.67:0/1429898766 learned_addr learned my addr 172.21.15.67:0/1429898766 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.997+0000 7f45ed56d640 1 --2- 172.21.15.67:0/1429898766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7be57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.997+0000 7f45f5d22740 10 client.0 _ll_register_callbacks cb 0x5599e7afc4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.997+0000 7f45ecd6c640 1 --2- 172.21.15.67:0/1429898766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e77f29f0 0x5599e7bf1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.998+0000 7f45ed56d640 1 -- 172.21.15.67:0/1429898766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e7813390 msgr2=0x5599e7be51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.998+0000 7f45ed56d640 1 --2- 172.21.15.67:0/1429898766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e7813390 0x5599e7be51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.998+0000 7f45ed56d640 1 -- 172.21.15.67:0/1429898766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e77f29f0 msgr2=0x5599e7bf1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:20.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.998+0000 7f45ed56d640 1 --2- 172.21.15.67:0/1429898766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e77f29f0 0x5599e7bf1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.998+0000 7f45ed56d640 1 -- 172.21.15.67:0/1429898766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599e789f650 con 0x5599e7834070 2024-07-31T15:13:20.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.998+0000 7f45ecd6c640 1 --2- 172.21.15.67:0/1429898766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e7813390 0x5599e7be51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:20.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.998+0000 7f45ed56d640 1 --2- 172.21.15.67:0/1429898766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7be57e0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x5599e7bdc250 tx=0x7f45e400ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82729aabda48c96c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:20.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.998+0000 7f45d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:13:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.998+0000 7f45d0ff9640 1 -- 172.21.15.67:0/1429898766 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4028020 con 0x5599e7834070 2024-07-31T15:13:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.999+0000 7f45d0ff9640 1 -- 172.21.15.67:0/1429898766 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7f45e40049e0 con 0x5599e7834070 2024-07-31T15:13:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.999+0000 7f45f5d22740 1 -- 172.21.15.67:0/1429898766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45e40054d0 con 0x5599e7834070 2024-07-31T15:13:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.999+0000 7f45d0ff9640 1 -- 172.21.15.67:0/1429898766 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f45e4033050 con 0x5599e7834070 2024-07-31T15:13:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.999+0000 7f45f5d22740 10 client.20172 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.999+0000 7f45f5d22740 1 -- 172.21.15.67:0/1429898766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5599e7bf3da0 con 0x5599e7834070 2024-07-31T15:13:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:19.999+0000 7f45d0ff9640 1 -- 172.21.15.67:0/1429898766 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e40054d0 con 0x5599e7834070 2024-07-31T15:13:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.000+0000 7f45d0ff9640 1 -- 172.21.15.67:0/1429898766 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f45e4023470 con 0x5599e7834070 2024-07-31T15:13:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.000+0000 7f45f5d22740 10 client.20172 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.000+0000 7f45f5d22740 10 client.20172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.000+0000 7f45f5d22740 20 client.20172 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.000+0000 7f45f5d22740 1 --2- 172.21.15.67:0/1429898766 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5599e77c7da0 0x5599e7bf7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.000+0000 7f45f5d22740 4 client.20172 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:13:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.000+0000 7f45f5d22740 1 -- 172.21.15.67:0/1429898766 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599e7b51ae0 con 0x5599e77c7da0 2024-07-31T15:13:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.000+0000 7f45ecd6c640 1 --2- 172.21.15.67:0/1429898766 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5599e77c7da0 0x5599e7bf7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.001+0000 7f45d0ff9640 10 client.20172 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.001+0000 7f45ecd6c640 1 --2- 172.21.15.67:0/1429898766 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5599e77c7da0 0x5599e7bf7210 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=836c406a62432efd server_cookie=45c8b601e128af1c in_seq=0 out_seq=0 2024-07-31T15:13:20.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.009+0000 7f45d0ff9640 1 -- 172.21.15.67:0/1429898766 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599e7b51ae0 con 0x5599e77c7da0 2024-07-31T15:13:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.009+0000 7f45d0ff9640 10 client.20172 handle_command_reply: tid=0 2024-07-31T15:13:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.009+0000 7f45f5d22740 1 client.20172 shutdown 2024-07-31T15:13:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.009+0000 7f45f5d22740 1 -- 172.21.15.67:0/1429898766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 msgr2=0x5599e7be57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.009+0000 7f45f5d22740 1 --2- 172.21.15.67:0/1429898766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7be57e0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x5599e7bdc250 tx=0x7f45e400ff20 comp rx=0 tx=0).stop 2024-07-31T15:13:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.009+0000 7f45f5d22740 1 -- 172.21.15.67:0/1429898766 shutdown_connections 2024-07-31T15:13:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.009+0000 7f45f5d22740 1 --2- 172.21.15.67:0/1429898766 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5599e77c7da0 0x5599e7bf7210 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.009+0000 7f45f5d22740 1 --2- 172.21.15.67:0/1429898766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e77f29f0 0x5599e7bf1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.009+0000 7f45d0ff9640 0 client.20172 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.009+0000 7f45f5d22740 1 --2- 172.21.15.67:0/1429898766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7be57e0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.009+0000 7f45f5d22740 1 --2- 172.21.15.67:0/1429898766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e7813390 0x5599e7be51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.009+0000 7f45f5d22740 1 -- 172.21.15.67:0/1429898766 >> 172.21.15.67:0/1429898766 conn(0x5599e7828460 msgr2=0x5599e7bd5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:20.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.009+0000 7f45f5d22740 1 -- 172.21.15.67:0/1429898766 shutdown_connections 2024-07-31T15:13:20.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.009+0000 7f45f5d22740 1 -- 172.21.15.67:0/1429898766 wait complete. 2024-07-31T15:13:20.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.010+0000 7f45f5d22740 20 client.20172 trim_cache size 0 max 16384 2024-07-31T15:13:20.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.034+0000 7f45f5d22740 1 Processor -- start 2024-07-31T15:13:20.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.034+0000 7f45f5d22740 1 -- start start 2024-07-31T15:13:20.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45f5d22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e77f29f0 0x5599e7bd6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:20.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45f5d22740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 0x5599e7bded70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:20.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45f5d22740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7bf0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:20.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45f5d22740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f45e81ae680 con 0x5599e7834070 2024-07-31T15:13:20.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45f5d22740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f45e81b01c0 con 0x5599e7813390 2024-07-31T15:13:20.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45f5d22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f45e81aed50 con 0x5599e77f29f0 2024-07-31T15:13:20.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45ecd6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e77f29f0 0x5599e7bd6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:20.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45ecd6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e77f29f0 0x5599e7bd6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:59826/0 (socket says 172.21.15.67:59826) 2024-07-31T15:13:20.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45ed56d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 0x5599e7bded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:20.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45ecd6c640 1 -- 172.21.15.67:0/2745557378 learned_addr learned my addr 172.21.15.67:0/2745557378 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:20.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45ecd6c640 1 --2- 172.21.15.67:0/2745557378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7bf0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:20.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45ed56d640 1 -- 172.21.15.67:0/2745557378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e77f29f0 msgr2=0x5599e7bd6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:20.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45ed56d640 1 --2- 172.21.15.67:0/2745557378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e77f29f0 0x5599e7bd6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45ed56d640 1 -- 172.21.15.67:0/2745557378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 msgr2=0x5599e7bf0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:20.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45ed56d640 1 --2- 172.21.15.67:0/2745557378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7bf0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.035+0000 7f45ed56d640 1 -- 172.21.15.67:0/2745557378 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599e7bf7580 con 0x5599e7813390 2024-07-31T15:13:20.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.036+0000 7f45ed56d640 1 --2- 172.21.15.67:0/2745557378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 0x5599e7bded70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f45e4011510 tx=0x7f45e4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21ebece3886b8a5b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:20.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.036+0000 7f45d2ffd640 1 -- 172.21.15.67:0/2745557378 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e403b020 con 0x5599e7813390 2024-07-31T15:13:20.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.036+0000 7f45d2ffd640 1 -- 172.21.15.67:0/2745557378 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e4004600 con 0x5599e7813390 2024-07-31T15:13:20.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.036+0000 7f45d2ffd640 1 -- 172.21.15.67:0/2745557378 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4028070 con 0x5599e7813390 2024-07-31T15:13:20.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.037+0000 7f45f5d22740 1 -- 172.21.15.67:0/2745557378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 msgr2=0x5599e7bded70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:20.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.037+0000 7f45f5d22740 1 --2- 172.21.15.67:0/2745557378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 0x5599e7bded70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f45e4011510 tx=0x7f45e4000bc0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.037+0000 7f45f5d22740 1 -- 172.21.15.67:0/2745557378 shutdown_connections 2024-07-31T15:13:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.037+0000 7f45f5d22740 1 --2- 172.21.15.67:0/2745557378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7bf0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.037+0000 7f45f5d22740 1 --2- 172.21.15.67:0/2745557378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 0x5599e7bded70 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.037+0000 7f45f5d22740 1 --2- 172.21.15.67:0/2745557378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e77f29f0 0x5599e7bd6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.037+0000 7f45f5d22740 1 -- 172.21.15.67:0/2745557378 >> 172.21.15.67:0/2745557378 conn(0x5599e7828460 msgr2=0x5599e7bde660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.037+0000 7f45f5d22740 1 -- 172.21.15.67:0/2745557378 shutdown_connections 2024-07-31T15:13:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.037+0000 7f45f5d22740 1 -- 172.21.15.67:0/2745557378 wait complete. 2024-07-31T15:13:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.038+0000 7f45f5d22740 1 -- start start 2024-07-31T15:13:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.038+0000 7f45f5d22740 1 Processor -- start 2024-07-31T15:13:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.038+0000 7f45f5d22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e77f29f0 0x5599e7be85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.038+0000 7f45f5d22740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 0x5599e7be8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.038+0000 7f45f5d22740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7c5d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.038+0000 7f45f5d22740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f45e81afaf0 con 0x5599e7834070 2024-07-31T15:13:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.038+0000 7f45f5d22740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f45e81af420 con 0x5599e7813390 2024-07-31T15:13:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.038+0000 7f45f5d22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f45e81ac680 con 0x5599e77f29f0 2024-07-31T15:13:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.038+0000 7f45f5d22740 10 client.0 _ll_register_callbacks cb 0x7f45e4011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.039+0000 7f45ecd6c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7c5d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.039+0000 7f45ecd6c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7c5d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56230/0 (socket says 172.21.15.67:56230) 2024-07-31T15:13:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.039+0000 7f45ecd6c640 1 -- 172.21.15.67:0/4073738023 learned_addr learned my addr 172.21.15.67:0/4073738023 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.039+0000 7f45ecd6c640 1 --2- 172.21.15.67:0/4073738023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e77f29f0 0x5599e7be85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.039+0000 7f45ecd6c640 1 -- 172.21.15.67:0/4073738023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e77f29f0 msgr2=0x5599e7be85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.039+0000 7f45ecd6c640 1 --2- 172.21.15.67:0/4073738023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e77f29f0 0x5599e7be85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.039+0000 7f45ecd6c640 1 -- 172.21.15.67:0/4073738023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 msgr2=0x5599e7be8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.039+0000 7f45ecd6c640 1 --2- 172.21.15.67:0/4073738023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 0x5599e7be8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.039+0000 7f45ecd6c640 1 -- 172.21.15.67:0/4073738023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599e7c5ded0 con 0x5599e7834070 2024-07-31T15:13:20.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.039+0000 7f45ecd6c640 1 --2- 172.21.15.67:0/4073738023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7c5d990 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f45d80045f0 tx=0x7f45d801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5de50da95cdd6c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:20.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.039+0000 7f45d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:13:20.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.040+0000 7f45d0ff9640 1 -- 172.21.15.67:0/4073738023 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8024040 con 0x5599e7834070 2024-07-31T15:13:20.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.040+0000 7f45d0ff9640 1 -- 172.21.15.67:0/4073738023 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9483+0+0 (secure 0 0 0) 0x7f45d80295b0 con 0x5599e7834070 2024-07-31T15:13:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.040+0000 7f45f5d22740 1 -- 172.21.15.67:0/4073738023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45e4004600 con 0x5599e7834070 2024-07-31T15:13:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.040+0000 7f45d0ff9640 1 -- 172.21.15.67:0/4073738023 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f45d8029e80 con 0x5599e7834070 2024-07-31T15:13:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.040+0000 7f45f5d22740 10 client.20178 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.040+0000 7f45f5d22740 1 -- 172.21.15.67:0/4073738023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5599e7c5f150 con 0x5599e7834070 2024-07-31T15:13:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.040+0000 7f45d0ff9640 1 -- 172.21.15.67:0/4073738023 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e4004600 con 0x5599e7834070 2024-07-31T15:13:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.041+0000 7f45d0ff9640 1 -- 172.21.15.67:0/4073738023 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f45d8010950 con 0x5599e7834070 2024-07-31T15:13:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.041+0000 7f45f5d22740 10 client.20178 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.041+0000 7f45f5d22740 10 client.20178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.041+0000 7f45f5d22740 20 client.20178 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.041+0000 7f45f5d22740 1 --2- 172.21.15.67:0/4073738023 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5599e7c5f8b0 0x5599e7c61cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.041+0000 7f45f5d22740 4 client.20178 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:13:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.041+0000 7f45f5d22740 1 -- 172.21.15.67:0/4073738023 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5599e7b4d580 con 0x5599e7c5f8b0 2024-07-31T15:13:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.042+0000 7f45ed56d640 1 --2- 172.21.15.67:0/4073738023 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5599e7c5f8b0 0x5599e7c61cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:20.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.042+0000 7f45d0ff9640 10 client.20178 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:20.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.042+0000 7f45ed56d640 1 --2- 172.21.15.67:0/4073738023 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5599e7c5f8b0 0x5599e7c61cd0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbf4a64d85d58f7b server_cookie=62a7cea7e343205b in_seq=0 out_seq=0 2024-07-31T15:13:20.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.049+0000 7f45d0ff9640 1 -- 172.21.15.67:0/4073738023 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5599e7b4d580 con 0x5599e7c5f8b0 2024-07-31T15:13:20.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.049+0000 7f45d0ff9640 10 client.20178 handle_command_reply: tid=0 2024-07-31T15:13:20.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.049+0000 7f45f5d22740 1 client.20178 shutdown 2024-07-31T15:13:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.049+0000 7f45f5d22740 1 -- 172.21.15.67:0/4073738023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 msgr2=0x5599e7c5d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.049+0000 7f45f5d22740 1 --2- 172.21.15.67:0/4073738023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7c5d990 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f45d80045f0 tx=0x7f45d801d040 comp rx=0 tx=0).stop 2024-07-31T15:13:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.049+0000 7f45f5d22740 1 -- 172.21.15.67:0/4073738023 shutdown_connections 2024-07-31T15:13:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.049+0000 7f45f5d22740 1 --2- 172.21.15.67:0/4073738023 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5599e7c5f8b0 0x5599e7c61cd0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.049+0000 7f45f5d22740 1 --2- 172.21.15.67:0/4073738023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599e7834070 0x5599e7c5d990 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.049+0000 7f45f5d22740 1 --2- 172.21.15.67:0/4073738023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599e7813390 0x5599e7be8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.049+0000 7f45d0ff9640 0 client.20178 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.049+0000 7f45f5d22740 1 --2- 172.21.15.67:0/4073738023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5599e77f29f0 0x5599e7be85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.049+0000 7f45f5d22740 1 -- 172.21.15.67:0/4073738023 >> 172.21.15.67:0/4073738023 conn(0x5599e7828460 msgr2=0x5599e7bd6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.050+0000 7f45f5d22740 1 -- 172.21.15.67:0/4073738023 shutdown_connections 2024-07-31T15:13:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.050+0000 7f45f5d22740 1 -- 172.21.15.67:0/4073738023 wait complete. 2024-07-31T15:13:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.050+0000 7f45f5d22740 20 client.20178 trim_cache size 0 max 16384 2024-07-31T15:13:20.053 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:13:20.053 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (5349 inodes in the stack)", 2024-07-31T15:13:20.053 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T15:13:20.053 INFO:teuthology.orchestra.run.smithi067.stdout: "d5a4ad48-412b-4b1e-a8b7-c6af766e4c13": { 2024-07-31T15:13:20.053 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T15:13:20.053 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "d5a4ad48-412b-4b1e-a8b7-c6af766e4c13", 2024-07-31T15:13:20.053 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T15:13:20.053 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:13:20.054 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:13:20.054 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:13:20.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.051+0000 7f45ef7d1640 1 -- 172.21.15.67:0/100492332 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f45d40b3d40 msgr2=0x7f45d40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:20.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.051+0000 7f45ef7d1640 1 --2- 172.21.15.67:0/100492332 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f45d40b3d40 0x7f45d40b6250 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f45e0102ad0 tx=0x7f45e00c0760 comp rx=0 tx=0).stop 2024-07-31T15:13:20.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.051+0000 7f45ef7d1640 1 -- 172.21.15.67:0/100492332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45e8106610 msgr2=0x7f45e8103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:20.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.051+0000 7f45ef7d1640 1 --2- 172.21.15.67:0/100492332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45e8106610 0x7f45e8103400 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f45e002ade0 tx=0x7f45e0031660 comp rx=0 tx=0).stop 2024-07-31T15:13:20.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.051+0000 7f45ef7d1640 1 -- 172.21.15.67:0/100492332 shutdown_connections 2024-07-31T15:13:20.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.051+0000 7f45ef7d1640 1 --2- 172.21.15.67:0/100492332 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f45d40b3d40 0x7f45d40b6250 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.051+0000 7f45ef7d1640 1 --2- 172.21.15.67:0/100492332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45e8106610 0x7f45e8103400 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.051+0000 7f45ef7d1640 1 --2- 172.21.15.67:0/100492332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45e8105cc0 0x7f45e8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.051+0000 7f45ef7d1640 1 --2- 172.21.15.67:0/100492332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45e81019e0 0x7f45e8102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:20.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.051+0000 7f45ef7d1640 1 -- 172.21.15.67:0/100492332 >> 172.21.15.67:0/100492332 conn(0x7f45e8077690 msgr2=0x7f45e8106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:20.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.051+0000 7f45ef7d1640 1 -- 172.21.15.67:0/100492332 shutdown_connections 2024-07-31T15:13:20.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:20.051+0000 7f45ef7d1640 1 -- 172.21.15.67:0/100492332 wait complete. 2024-07-31T15:13:20.642 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:20.284+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c061 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math/ auth{1=1} v1164 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:10.213499+0000 31=31+0) n(v2 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1) old_inodes=2 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:597213 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@13} mcw={1=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8e92fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c061:[<0x1000000bf5f/math v634>,<0x10000000202/lib v5688>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000c061:[<0x1000000bf5f/math v1164>,<0x10000000202/lib v21040>,<0x10000000201/linux-6.5.11 v2856>,<0x10000000006/t v1834>,<0x10000000005/tmp v1758>,<0x100000001fb/client.0 v1726>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1690>,<0x10000000001/sv_1 v1688>,<0x10000000000/qa v1674>,<0x1/volumes v1625>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:10.988109+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:10.213499+0000 31=31+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:20 smithi073 ceph-mon[36714]: osdmap e134: 12 total, 12 up, 12 in 2024-07-31T15:13:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:13:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:20 smithi077 ceph-mon[37327]: osdmap e134: 12 total, 12 up, 12 in 2024-07-31T15:13:20.787 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.787 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.787 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.787 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.788 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.789 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.789 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.789 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.789 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.789 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.789 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.789 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.789 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.789 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.789 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.789 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.789 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:13:20.789 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.790 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:20.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:20.791 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:20 smithi067 ceph-mon[32750]: osdmap e134: 12 total, 12 up, 12 in 2024-07-31T15:13:20.818 INFO:tasks.cephfs.filesystem:scrub status for tag:d5a4ad48-412b-4b1e-a8b7-c6af766e4c13 - {'path': '/', 'tag': 'd5a4ad48-412b-4b1e-a8b7-c6af766e4c13', 'options': 'recursive,force'} 2024-07-31T15:13:21.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: pgmap v1111: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-07-31T15:13:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:13:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:21.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:21 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:21.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:21.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:21.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:21.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:21.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:21.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: pgmap v1111: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-07-31T15:13:21.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:13:21.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:21.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:21.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:21 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:21.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:21.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:21.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:21.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:21.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:21.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: pgmap v1111: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-07-31T15:13:21.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:13:21.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:21.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:21.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:21.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:21 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:22.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:22.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:22.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:22 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:22.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:22.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:22 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:22.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:22.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:22.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:22 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:22.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:22.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:22 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:22.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:22.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:22.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:22 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:22.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:22.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:22 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:23.409 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:23 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:23.153+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000ce48 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity/ auth{1=1,2=1} v1695 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3) n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4) old_inodes=3 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:597565 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@13} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8aaac580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000ce48:[<0x1000000ce07/integrity v345>,<0x10000000202/security v8766>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000ce48:[<0x1000000ce07/integrity v1695>,<0x10000000202/security v15110>,<0x10000000201/linux-6.5.11 v2858>,<0x10000000006/t v1836>,<0x10000000005/tmp v1760>,<0x100000001fb/client.0 v1728>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1692>,<0x10000000001/sv_1 v1690>,<0x10000000000/qa v1676>,<0x1/volumes v1627>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:59.470106+0000 10=7+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","memory_value.dirstat":"f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3)","memory_value.rstat":"n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:23.409 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:23 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:23.292+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009eb8 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs/ auth{4=1} v4394 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:25.901734+0000 12=12+0) n(v3 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:597908 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@10} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce809d7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009eb8:[<0x10000009d36/debugfs v3846>,<0x10000000202/fs v3830>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009eb8:[<0x10000009d36/debugfs v4394>,<0x10000000202/fs v21227>,<0x10000000201/linux-6.5.11 v2858>,<0x10000000006/t v1836>,<0x10000000005/tmp v1760>,<0x100000001fb/client.0 v1728>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1692>,<0x10000000001/sv_1 v1690>,<0x10000000000/qa v1676>,<0x1/volumes v1627>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:39.315767+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:25.901734+0000 12=12+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:23.409 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:23 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:23.355+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000a492 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs/ auth{1=1} v5396 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:30.806634+0000 8=8+0) n(v3 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1) old_inodes=2 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:597981 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@10} mcw={1=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8aca9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000a492:[<0x10000009d36/tracefs v3850>,<0x10000000202/fs v3830>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x1000000a492:[<0x10000009d36/tracefs v5396>,<0x10000000202/fs v21227>,<0x10000000201/linux-6.5.11 v2858>,<0x10000000006/t v1836>,<0x10000000005/tmp v1760>,<0x100000001fb/client.0 v1728>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1692>,<0x10000000001/sv_1 v1690>,<0x10000000000/qa v1676>,<0x1/volumes v1627>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:41.554720+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:30.806634+0000 8=8+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:23.409 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:23 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:23.407+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002351 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char/ auth{2=1} v18176 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:34.793553+0000 56=49+7) n(v10 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10) old_inodes=4 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:604924 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@15} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7ef94100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000002351:[<0x10000001d8f/char v2490>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x10000002351:[<0x10000001d8f/char v18176>,<0x10000000202/drivers v22224>,<0x10000000201/linux-6.5.11 v2858>,<0x10000000006/t v1836>,<0x10000000005/tmp v1760>,<0x100000001fb/client.0 v1728>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1692>,<0x10000000001/sv_1 v1690>,<0x10000000000/qa v1676>,<0x1/volumes v1627>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:02.131047+0000 40=33+7)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:34.793553+0000 56=49+7)","memory_value.rstat":"n(v10 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:23.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:23 smithi077 ceph-mon[37327]: pgmap v1113: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 830 KiB/s wr, 144 op/s 2024-07-31T15:13:23.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:23 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:13:23.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:23 smithi067 ceph-mon[32750]: pgmap v1113: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 830 KiB/s wr, 144 op/s 2024-07-31T15:13:23.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:23 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:13:23.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:23 smithi073 ceph-mon[36714]: pgmap v1113: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 830 KiB/s wr, 144 op/s 2024-07-31T15:13:23.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:23 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:13:23.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:23 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:23.544+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000243a [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk/ auth{2=1,4=1} fragtree_t(*^3) v18679 ap=3 RANDEPHEMERALPIN f(v3 m2024-07-31T15:07:07.059118+0000 137=88+49) n(v9 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50) old_inodes=2 (inest sync->mix g=2,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:605009 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@12} mcw={2=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7fa16680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000243a:[<0x10000001d8f/clk v2484>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x1000000243a:[<0x10000001d8f/clk v18679>,<0x10000000202/drivers v22224>,<0x10000000201/linux-6.5.11 v2858>,<0x10000000006/t v1836>,<0x10000000005/tmp v1760>,<0x100000001fb/client.0 v1728>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1692>,<0x10000000001/sv_1 v1690>,<0x10000000000/qa v1676>,<0x1/volumes v1627>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 133=84+49)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50)","memory_value.dirstat":"f(v3 m2024-07-31T15:07:07.059118+0000 137=88+49)","memory_value.rstat":"n(v9 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:23.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:23 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:23.557+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000048e8 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/i2c/ auth{3=1} v19350 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:33.581686+0000 32=29+3) n(v6 rc2024-07-31T15:11:33.583686+0000 b3393592 213=209+4) old_inodes=1 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:605018 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@9} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7d07e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000048e8:[<0x10000001d8f/i2c v9487>,<0x10000000202/drivers v2799>,<0x10000000201/linux-6.5.11 v334>,<0x10000000006/t v232>,<0x10000000005/tmp v202>,<0x100000001fb/client.0 v201>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v184>,<0x10000000001/sv_1 v213>,<0x10000000000/qa v197>,<0x1/volumes v198>]//[]","memoryvalue":"(2)0x100000048e8:[<0x10000001d8f/i2c v19350>,<0x10000000202/drivers v22224>,<0x10000000201/linux-6.5.11 v2858>,<0x10000000006/t v1836>,<0x10000000005/tmp v1760>,<0x100000001fb/client.0 v1728>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1692>,<0x10000000001/sv_1 v1690>,<0x10000000000/qa v1676>,<0x1/volumes v1627>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:40.705240+0000 18=15+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:33.583686+0000 b3393592 213=209+4)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:33.581686+0000 32=29+3)","memory_value.rstat":"n(v6 rc2024-07-31T15:11:33.583686+0000 b3393592 213=209+4)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:23.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:23 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:23.613+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000008596 [...24,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/scsi/ auth{1=6,2=1,3=1,4=1} fragtree_t(*^3) v18676 ap=3 RANDEPHEMERALPIN f(v4 m2024-07-31T15:10:01.775559+0000 269=232+37) n(v12 rc2024-07-31T15:10:01.777559+0000 b31477139 971=924+47) old_inodes=3 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:605041 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@17} mcw={1=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce864ab600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000008596:[<0x10000001d8f/scsi v16994>,<0x10000000202/drivers v3182>,<0x10000000201/linux-6.5.11 v445>,<0x10000000006/t v329>,<0x10000000005/tmp v299>,<0x100000001fb/client.0 v296>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v274>,<0x10000000001/sv_1 v299>,<0x10000000000/qa v280>,<0x1/volumes v280>]//[]","memoryvalue":"(2)0x10000008596:[<0x10000001d8f/scsi v18676>,<0x10000000202/drivers v22224>,<0x10000000201/linux-6.5.11 v2858>,<0x10000000006/t v1836>,<0x10000000005/tmp v1760>,<0x100000001fb/client.0 v1728>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1692>,<0x10000000001/sv_1 v1690>,<0x10000000000/qa v1676>,<0x1/volumes v1627>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:10:00.625582+0000 265=228+37)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:10:01.777559+0000 b31477139 971=924+47)","memory_value.dirstat":"f(v4 m2024-07-31T15:10:01.775559+0000 269=232+37)","memory_value.rstat":"n(v12 rc2024-07-31T15:10:01.777559+0000 b31477139 971=924+47)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:23.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:23 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:23.614+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009980 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video/ auth{1=1,4=1} v18105 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:16.340374+0000 28=24+4) n(v11 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28) old_inodes=2 (inest sync->mix g=1,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:605051 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@10} mcw={1=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce87325b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009980:[<0x10000001d8f/video v17889>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009980:[<0x10000001d8f/video v18105>,<0x10000000202/drivers v22224>,<0x10000000201/linux-6.5.11 v2858>,<0x10000000006/t v1836>,<0x10000000005/tmp v1760>,<0x100000001fb/client.0 v1728>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1692>,<0x10000000001/sv_1 v1690>,<0x10000000000/qa v1676>,<0x1/volumes v1627>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:35.535846+0000 16=12+4)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:16.340374+0000 28=24+4)","memory_value.rstat":"n(v11 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:23.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:23 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:23.615+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005460 [...25,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media/ auth{2=1,3=1,4=1} v18772 ap=3 RANDEPHEMERALPIN f(v2 m2024-07-31T15:11:38.494586+0000 23=6+17) n(v13 rc2024-07-31T15:11:38.496586+0000 b38474543 3300=3123+177) old_inodes=2 (inest sync->mix g=2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:605052 nref=4)) caps={15489=pAsLsXsFs/pAsLsXsFs@11} mcw={2=AsLsXsFs,3=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce82ee2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000005460:[<0x10000001d8f/media v12887>,<0x10000000202/drivers v2910>,<0x10000000201/linux-6.5.11 v379>,<0x10000000006/t v275>,<0x10000000005/tmp v244>,<0x100000001fb/client.0 v243>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v221>,<0x10000000001/sv_1 v249>,<0x10000000000/qa v233>,<0x1/volumes v234>]//[]","memoryvalue":"(2)0x10000005460:[<0x10000001d8f/media v18772>,<0x10000000202/drivers v22224>,<0x10000000201/linux-6.5.11 v2858>,<0x10000000006/t v1836>,<0x10000000005/tmp v1760>,<0x100000001fb/client.0 v1728>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1692>,<0x10000000001/sv_1 v1690>,<0x10000000000/qa v1676>,<0x1/volumes v1627>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:18.811444+0000 19=2+17)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:38.496586+0000 b38474543 3300=3123+177)","memory_value.dirstat":"f(v2 m2024-07-31T15:11:38.494586+0000 23=6+17)","memory_value.rstat":"n(v13 rc2024-07-31T15:11:38.496586+0000 b38474543 3300=3123+177)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000048e8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/i2c) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000008596 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/scsi) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000005460 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:24 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:24.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000048e8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/i2c) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000008596 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/scsi) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000005460 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:24 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:24.892 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:24.651+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000be32 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module/ auth{2=1} v2029 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0) n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:603773 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@16} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce826b7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000be32:[<0x1000000bd0a/module v1134>,<0x10000000202/kernel v5648>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000be32:[<0x1000000bd0a/module v2029>,<0x10000000202/kernel v21206>,<0x10000000201/linux-6.5.11 v2860>,<0x10000000006/t v1836>,<0x10000000005/tmp v1760>,<0x100000001fb/client.0 v1728>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1692>,<0x10000000001/sv_1 v1690>,<0x10000000000/qa v1676>,<0x1/volumes v1627>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:07.168188+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000048e8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/i2c) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:24.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000008596 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/scsi) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000005460 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media) see mds.b log and `damage ls` output for details 2024-07-31T15:13:24.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:24.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:24 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: pgmap v1114: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 205 KiB/s wr, 191 op/s 2024-07-31T15:13:25.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:13:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:25.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:25 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: pgmap v1114: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 205 KiB/s wr, 191 op/s 2024-07-31T15:13:25.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:13:25.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:25.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:25 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: pgmap v1114: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 205 KiB/s wr, 191 op/s 2024-07-31T15:13:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:13:25.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:25.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:25.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:25 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:26 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:26 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:26 smithi077 ceph-mon[37327]: pgmap v1115: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 193 KiB/s wr, 292 op/s 2024-07-31T15:13:26.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:26 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:13:26.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:26.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:26.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:26.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:26.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:26 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:26.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:26 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:26 smithi067 ceph-mon[32750]: pgmap v1115: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 193 KiB/s wr, 292 op/s 2024-07-31T15:13:26.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:26 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:13:26.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:26 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:26 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:26 smithi073 ceph-mon[36714]: pgmap v1115: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 193 KiB/s wr, 292 op/s 2024-07-31T15:13:26.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:26 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:13:27.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:13:27.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:13:27.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:13:27.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:13:27.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:13:27.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:13:27.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:13:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:13:27.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:13:28.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:28 smithi067 ceph-mon[32750]: pgmap v1116: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 176 KiB/s wr, 661 op/s 2024-07-31T15:13:28.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:28 smithi073 ceph-mon[36714]: pgmap v1116: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 176 KiB/s wr, 661 op/s 2024-07-31T15:13:28.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:28 smithi077 ceph-mon[37327]: pgmap v1116: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 176 KiB/s wr, 661 op/s 2024-07-31T15:13:29.393 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:29.046+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d187 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi/ auth{1=1} v3040 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0) n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1) old_inodes=1 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:618242 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7c6c1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d187:[<0x1000000d116/ctxfi v1858>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d187:[<0x1000000d116/ctxfi v3040>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2868>,<0x10000000006/t v1842>,<0x10000000005/tmp v1767>,<0x100000001fb/client.0 v1736>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1700>,<0x10000000001/sv_1 v1696>,<0x10000000000/qa v1682>,<0x1/volumes v1633>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.452342+0000 30=30+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:29.393 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:29.048+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d119 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97/ auth{2=1} v3208 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0) n(v3 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1) old_inodes=3 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:618245 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce9047e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d119:[<0x1000000d116/ac97 v22>,<0x1000000cf1b/pci v463>,<0x10000000202/sound v8728>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000d119:[<0x1000000d116/ac97 v3208>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2868>,<0x10000000006/t v1842>,<0x10000000005/tmp v1767>,<0x100000001fb/client.0 v1736>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1700>,<0x10000000001/sv_1 v1696>,<0x10000000000/qa v1682>,<0x1/volumes v1633>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:00.491085+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:29.393 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:29.055+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d1cc [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1/ auth{2=1} v3042 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0) n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:618252 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8f176c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v1862>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v3042>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2868>,<0x10000000006/t v1842>,<0x10000000005/tmp v1767>,<0x100000001fb/client.0 v1736>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1700>,<0x10000000001/sv_1 v1696>,<0x10000000000/qa v1682>,<0x1/volumes v1633>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.549340+0000 22=22+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:29.393 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:29.082+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d180 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio/ auth{1=1} v3038 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0) n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1) old_inodes=1 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:618291 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8ebd2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v1860>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v3038>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v2868>,<0x10000000006/t v1842>,<0x10000000005/tmp v1767>,<0x100000001fb/client.0 v1736>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1700>,<0x10000000001/sv_1 v1696>,<0x10000000000/qa v1682>,<0x1/volumes v1633>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.414343+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:29.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.595+0000 7fa25bfff640 1 -- 172.21.15.67:0/3595834195 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555e587d51a0 con 0x555e58450570 2024-07-31T15:13:29.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.595+0000 7fa25bfff640 10 client.20109 handle_command_reply: tid=0 2024-07-31T15:13:29.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.595+0000 7fa2890bd740 1 client.20109 shutdown 2024-07-31T15:13:29.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.595+0000 7fa2890bd740 1 -- 172.21.15.67:0/3595834195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e58493980 msgr2=0x555e5887a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:29.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.595+0000 7fa2890bd740 1 --2- 172.21.15.67:0/3595834195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e58493980 0x555e5887a630 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fa268000f10 tx=0x7fa268039440 comp rx=0 tx=0).stop 2024-07-31T15:13:29.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.595+0000 7fa2890bd740 1 -- 172.21.15.67:0/3595834195 shutdown_connections 2024-07-31T15:13:29.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.595+0000 7fa2890bd740 1 --2- 172.21.15.67:0/3595834195 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555e58450570 0x555e588eaf70 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:29.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.595+0000 7fa2890bd740 1 --2- 172.21.15.67:0/3595834195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e584bc840 0x555e58875c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:29.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.595+0000 7fa2890bd740 1 --2- 172.21.15.67:0/3595834195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e5849bb60 0x555e5887abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:29.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.595+0000 7fa2890bd740 1 --2- 172.21.15.67:0/3595834195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555e58493980 0x555e5887a630 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:29.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.595+0000 7fa2890bd740 1 -- 172.21.15.67:0/3595834195 >> 172.21.15.67:0/3595834195 conn(0x555e584b0990 msgr2=0x555e58872ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:29.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.595+0000 7fa25bfff640 0 client.20109 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:29.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.596+0000 7fa2890bd740 1 -- 172.21.15.67:0/3595834195 shutdown_connections 2024-07-31T15:13:29.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.596+0000 7fa2890bd740 1 -- 172.21.15.67:0/3595834195 wait complete. 2024-07-31T15:13:29.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.596+0000 7fa2890bd740 20 client.20109 trim_cache size 0 max 16384 2024-07-31T15:13:29.599 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:13:29.599 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 66, 2024-07-31T15:13:29.599 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:13:29.599 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 62, 2024-07-31T15:13:29.599 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:13:29.599 INFO:teuthology.orchestra.run.smithi067.stdout: "847f6b4e": { 2024-07-31T15:13:29.599 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 62, 2024-07-31T15:13:29.600 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:13:29.600 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:13:29.600 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:13:29.600 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:13:29.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:13:29.600 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:13:29.600 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 131.7, 2024-07-31T15:13:29.600 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:13:29.600 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:13:29.600 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:13:29.600 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:13:29.601 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:13:29.601 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:13:29.601 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:13:29.601 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:13:29.601 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:13:29.601 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:13:29.601 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:13:29.601 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:13:29.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.597+0000 7fa28190a640 1 -- 172.21.15.67:0/987175375 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa26c0b3df0 msgr2=0x7fa26c0b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:29.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.597+0000 7fa28190a640 1 --2- 172.21.15.67:0/987175375 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa26c0b3df0 0x7fa26c0b6300 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa27402ae60 tx=0x7fa274004940 comp rx=0 tx=0).stop 2024-07-31T15:13:29.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.597+0000 7fa28190a640 1 -- 172.21.15.67:0/987175375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa27c10c530 msgr2=0x7fa27c1a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:29.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.597+0000 7fa28190a640 1 --2- 172.21.15.67:0/987175375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa27c10c530 0x7fa27c1a0980 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa27000b450 tx=0x7fa27000b910 comp rx=0 tx=0).stop 2024-07-31T15:13:29.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.597+0000 7fa28190a640 1 -- 172.21.15.67:0/987175375 shutdown_connections 2024-07-31T15:13:29.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.597+0000 7fa28190a640 1 --2- 172.21.15.67:0/987175375 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa26c0b3df0 0x7fa26c0b6300 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:29.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.597+0000 7fa28190a640 1 --2- 172.21.15.67:0/987175375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa27c10ce80 0x7fa27c1a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:29.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.597+0000 7fa28190a640 1 --2- 172.21.15.67:0/987175375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa27c10c530 0x7fa27c1a0980 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:29.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.597+0000 7fa28190a640 1 --2- 172.21.15.67:0/987175375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa27c10a080 0x7fa27c1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:29.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.597+0000 7fa28190a640 1 -- 172.21.15.67:0/987175375 >> 172.21.15.67:0/987175375 conn(0x7fa27c06e8c0 msgr2=0x7fa27c10d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:29.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.597+0000 7fa28190a640 1 -- 172.21.15.67:0/987175375 shutdown_connections 2024-07-31T15:13:29.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:29.597+0000 7fa28190a640 1 -- 172.21.15.67:0/987175375 wait complete. 2024-07-31T15:13:29.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:29.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:29.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:29.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:29.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:29.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:29.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:29 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:13:29.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:29 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:13:29.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:29 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:13:29.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:29 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:13:29.892 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:29.532+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000cde2 [...10,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders/ auth{4=1} v112 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0) n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:594988 nref=3)) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8dc52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v29>,<0x1000000cbec/selinux v646>,<0x10000000202/scripts v8820>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v112>,<0x1000000cbec/selinux v749>,<0x10000000202/scripts v11639>,<0x10000000201/linux-6.5.11 v2868>,<0x10000000006/t v1842>,<0x10000000005/tmp v1767>,<0x100000001fb/client.0 v1736>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1700>,<0x10000000001/sv_1 v1696>,<0x10000000000/qa v1682>,<0x1/volumes v1633>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:58.999116+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:13:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:13:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:13:29.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:29 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:13:29.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:29.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:29 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:13:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:29 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:13:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:29 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:13:29.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:29 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:13:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.196+0000 7faa6567f640 1 -- 172.21.15.67:0/2073907088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa60109780 msgr2=0x7faa6010ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.196+0000 7faa6567f640 1 --2- 172.21.15.67:0/2073907088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa60109780 0x7faa6010ffb0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7faa58009ca0 tx=0x7faa5802a8c0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.197+0000 7faa6567f640 1 -- 172.21.15.67:0/2073907088 shutdown_connections 2024-07-31T15:13:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.197+0000 7faa6567f640 1 --2- 172.21.15.67:0/2073907088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa60109780 0x7faa6010ffb0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.197+0000 7faa6567f640 1 --2- 172.21.15.67:0/2073907088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa60108e30 0x7faa60109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.197+0000 7faa6567f640 1 --2- 172.21.15.67:0/2073907088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa60103500 0x7faa60103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.197+0000 7faa6567f640 1 -- 172.21.15.67:0/2073907088 >> 172.21.15.67:0/2073907088 conn(0x7faa600fef00 msgr2=0x7faa60101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.197+0000 7faa6567f640 1 -- 172.21.15.67:0/2073907088 shutdown_connections 2024-07-31T15:13:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.197+0000 7faa6567f640 1 -- 172.21.15.67:0/2073907088 wait complete. 2024-07-31T15:13:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.197+0000 7faa6567f640 1 Processor -- start 2024-07-31T15:13:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.197+0000 7faa6567f640 1 -- start start 2024-07-31T15:13:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa6567f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa60103500 0x7faa60075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa6567f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa60108e30 0x7faa60075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa6567f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa60109780 0x7faa60075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa6567f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faa60113450 con 0x7faa60108e30 2024-07-31T15:13:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa6567f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faa601132d0 con 0x7faa60103500 2024-07-31T15:13:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa6567f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa601135d0 con 0x7faa60109780 2024-07-31T15:13:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa5e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa60108e30 0x7faa60075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa5e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa60108e30 0x7faa60075680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48436/0 (socket says 172.21.15.67:48436) 2024-07-31T15:13:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa5e7fc640 1 -- 172.21.15.67:0/352159754 learned_addr learned my addr 172.21.15.67:0/352159754 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa5effd640 1 --2- 172.21.15.67:0/352159754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa60103500 0x7faa60075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa5f7fe640 1 --2- 172.21.15.67:0/352159754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa60109780 0x7faa60075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa5e7fc640 1 -- 172.21.15.67:0/352159754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa60109780 msgr2=0x7faa60075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa5e7fc640 1 --2- 172.21.15.67:0/352159754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa60109780 0x7faa60075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa5e7fc640 1 -- 172.21.15.67:0/352159754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa60103500 msgr2=0x7faa60075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa5e7fc640 1 --2- 172.21.15.67:0/352159754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa60103500 0x7faa60075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa5e7fc640 1 -- 172.21.15.67:0/352159754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa601a80c0 con 0x7faa60108e30 2024-07-31T15:13:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.198+0000 7faa5e7fc640 1 --2- 172.21.15.67:0/352159754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa60108e30 0x7faa60075680 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7faa5400e9f0 tx=0x7faa5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.199+0000 7faa47fff640 1 -- 172.21.15.67:0/352159754 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5400ccc0 con 0x7faa60108e30 2024-07-31T15:13:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.199+0000 7faa6567f640 1 -- 172.21.15.67:0/352159754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa601a83b0 con 0x7faa60108e30 2024-07-31T15:13:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.199+0000 7faa47fff640 1 -- 172.21.15.67:0/352159754 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa540108e0 con 0x7faa60108e30 2024-07-31T15:13:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.199+0000 7faa47fff640 1 -- 172.21.15.67:0/352159754 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54018860 con 0x7faa60108e30 2024-07-31T15:13:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.199+0000 7faa6567f640 1 -- 172.21.15.67:0/352159754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa601a8900 con 0x7faa60108e30 2024-07-31T15:13:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa6c570740 1 Processor -- start 2024-07-31T15:13:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa6c570740 1 -- start start 2024-07-31T15:13:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa47fff640 1 -- 172.21.15.67:0/352159754 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7faa54010430 con 0x7faa60108e30 2024-07-31T15:13:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa6c570740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acca21b0 0x5563ad044ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa6c570740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc792f0 0x5563ad045220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa6c570740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc814d0 0x5563ad045760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa6c570740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5563acfba3b0 con 0x5563acc792f0 2024-07-31T15:13:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa6c570740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5563acfa98b0 con 0x5563acc814d0 2024-07-31T15:13:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa6c570740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5563acfad8f0 con 0x5563acca21b0 2024-07-31T15:13:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa5f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acca21b0 0x5563ad044ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa5effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc792f0 0x5563ad045220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa5e7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc814d0 0x5563ad045760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa5f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acca21b0 0x5563ad044ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37072/0 (socket says 172.21.15.67:37072) 2024-07-31T15:13:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa5effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc792f0 0x5563ad045220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48448/0 (socket says 172.21.15.67:48448) 2024-07-31T15:13:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.202+0000 7faa5f7fe640 1 -- 172.21.15.67:0/3336655845 learned_addr learned my addr 172.21.15.67:0/3336655845 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.203+0000 7faa5f7fe640 1 -- 172.21.15.67:0/3336655845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc814d0 msgr2=0x5563ad045760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.203+0000 7faa5f7fe640 1 --2- 172.21.15.67:0/3336655845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc814d0 0x5563ad045760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.203+0000 7faa5f7fe640 1 -- 172.21.15.67:0/3336655845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc792f0 msgr2=0x5563ad045220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.203+0000 7faa5f7fe640 1 --2- 172.21.15.67:0/3336655845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc792f0 0x5563ad045220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.203+0000 7faa5f7fe640 1 -- 172.21.15.67:0/3336655845 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563acf509a0 con 0x5563acca21b0 2024-07-31T15:13:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.203+0000 7faa5f7fe640 1 --2- 172.21.15.67:0/3336655845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acca21b0 0x5563ad044ce0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faa5802f4f0 tx=0x7faa58009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b08346923311de73 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.203+0000 7faa47fff640 1 --2- 172.21.15.67:0/352159754 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faa480b3d70 0x7faa480b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.203+0000 7faa457fa640 1 -- 172.21.15.67:0/3336655845 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58031cf0 con 0x5563acca21b0 2024-07-31T15:13:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.203+0000 7faa457fa640 1 -- 172.21.15.67:0/3336655845 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa58036070 con 0x5563acca21b0 2024-07-31T15:13:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.203+0000 7faa457fa640 1 -- 172.21.15.67:0/3336655845 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa580042a0 con 0x5563acca21b0 2024-07-31T15:13:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.203+0000 7faa5effd640 1 --2- 172.21.15.67:0/352159754 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faa480b3d70 0x7faa480b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.203+0000 7faa47fff640 1 -- 172.21.15.67:0/352159754 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7faa540e45c0 con 0x7faa60108e30 2024-07-31T15:13:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.204+0000 7faa6c570740 1 -- 172.21.15.67:0/3336655845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acca21b0 msgr2=0x5563ad044ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.204+0000 7faa5effd640 1 --2- 172.21.15.67:0/352159754 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faa480b3d70 0x7faa480b6280 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x5563ad043960 tx=0x7faa5000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.204+0000 7faa6c570740 1 --2- 172.21.15.67:0/3336655845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acca21b0 0x5563ad044ce0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faa5802f4f0 tx=0x7faa58009800 comp rx=0 tx=0).stop 2024-07-31T15:13:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.204+0000 7faa6c570740 1 -- 172.21.15.67:0/3336655845 shutdown_connections 2024-07-31T15:13:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.204+0000 7faa6c570740 1 --2- 172.21.15.67:0/3336655845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc814d0 0x5563ad045760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.204+0000 7faa6c570740 1 --2- 172.21.15.67:0/3336655845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc792f0 0x5563ad045220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.204+0000 7faa6c570740 1 --2- 172.21.15.67:0/3336655845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acca21b0 0x5563ad044ce0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.204+0000 7faa6c570740 1 -- 172.21.15.67:0/3336655845 >> 172.21.15.67:0/3336655845 conn(0x5563acc96300 msgr2=0x5563ad042bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.205+0000 7faa6c570740 1 -- 172.21.15.67:0/3336655845 shutdown_connections 2024-07-31T15:13:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.205+0000 7faa6c570740 1 -- 172.21.15.67:0/3336655845 wait complete. 2024-07-31T15:13:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.205+0000 7faa6c570740 1 -- start start 2024-07-31T15:13:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.205+0000 7faa6c570740 1 Processor -- start 2024-07-31T15:13:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.205+0000 7faa6c570740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc792f0 0x5563ad050870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.205+0000 7faa6c570740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acc814d0 0x5563ad050db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa6c570740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acca21b0 0x5563ad05c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa6c570740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5563acee3ab0 con 0x5563acc792f0 2024-07-31T15:13:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa6c570740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5563acf31cf0 con 0x5563acca21b0 2024-07-31T15:13:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa6c570740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5563ad053280 con 0x5563acc814d0 2024-07-31T15:13:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa6c570740 10 client.0 _ll_register_callbacks cb 0x5563acfa39f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa5effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc792f0 0x5563ad050870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa5e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acc814d0 0x5563ad050db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa5effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc792f0 0x5563ad050870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48462/0 (socket says 172.21.15.67:48462) 2024-07-31T15:13:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa5f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acca21b0 0x5563ad05c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa5effd640 1 -- 172.21.15.67:0/735749933 learned_addr learned my addr 172.21.15.67:0/735749933 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa5e7fc640 1 --2- 172.21.15.67:0/735749933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acc814d0 0x5563ad050db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37088/0 (socket says 172.21.15.67:37088) 2024-07-31T15:13:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa5effd640 1 -- 172.21.15.67:0/735749933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acc814d0 msgr2=0x5563ad050db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa5effd640 1 --2- 172.21.15.67:0/735749933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acc814d0 0x5563ad050db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa5effd640 1 -- 172.21.15.67:0/735749933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acca21b0 msgr2=0x5563ad05c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa5effd640 1 --2- 172.21.15.67:0/735749933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acca21b0 0x5563ad05c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa5effd640 1 -- 172.21.15.67:0/735749933 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563acd0d790 con 0x5563acc792f0 2024-07-31T15:13:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa5effd640 1 --2- 172.21.15.67:0/735749933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc792f0 0x5563ad050870 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7faa50003d00 tx=0x7faa50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1294d8b909eda530 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:13:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa42ff5640 1 -- 172.21.15.67:0/735749933 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5002c060 con 0x5563acc792f0 2024-07-31T15:13:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa42ff5640 1 -- 172.21.15.67:0/735749933 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7faa50017030 con 0x5563acc792f0 2024-07-31T15:13:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.206+0000 7faa6c570740 1 -- 172.21.15.67:0/735749933 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa58036070 con 0x5563acc792f0 2024-07-31T15:13:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.207+0000 7faa42ff5640 1 -- 172.21.15.67:0/735749933 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7faa50033050 con 0x5563acc792f0 2024-07-31T15:13:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.207+0000 7faa6c570740 10 client.20190 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.207+0000 7faa6c570740 1 -- 172.21.15.67:0/735749933 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5563acd86620 con 0x5563acc792f0 2024-07-31T15:13:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.207+0000 7faa42ff5640 1 -- 172.21.15.67:0/735749933 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa58036070 con 0x5563acc792f0 2024-07-31T15:13:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.207+0000 7faa42ff5640 1 -- 172.21.15.67:0/735749933 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7faa50011440 con 0x5563acc792f0 2024-07-31T15:13:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.207+0000 7faa6c570740 10 client.20190 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.207+0000 7faa6c570740 10 client.20190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.207+0000 7faa6c570740 20 client.20190 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.207+0000 7faa6c570740 1 --2- 172.21.15.67:0/735749933 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5563acc60d80 0x5563ad062680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.207+0000 7faa6c570740 4 client.20190 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:13:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.207+0000 7faa6c570740 1 -- 172.21.15.67:0/735749933 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563acfbc300 con 0x5563acc60d80 2024-07-31T15:13:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.207+0000 7faa5e7fc640 1 --2- 172.21.15.67:0/735749933 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5563acc60d80 0x5563ad062680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.208+0000 7faa42ff5640 10 client.20190 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.208+0000 7faa5e7fc640 1 --2- 172.21.15.67:0/735749933 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5563acc60d80 0x5563ad062680 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73bbcd4c8e593c92 server_cookie=a7adfcbf00a38566 in_seq=0 out_seq=0 2024-07-31T15:13:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa42ff5640 1 -- 172.21.15.67:0/735749933 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563acfbc300 con 0x5563acc60d80 2024-07-31T15:13:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa42ff5640 10 client.20190 handle_command_reply: tid=0 2024-07-31T15:13:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa6c570740 1 client.20190 shutdown 2024-07-31T15:13:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa6c570740 1 -- 172.21.15.67:0/735749933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc792f0 msgr2=0x5563ad050870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa6c570740 1 --2- 172.21.15.67:0/735749933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc792f0 0x5563ad050870 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7faa50003d00 tx=0x7faa50004730 comp rx=0 tx=0).stop 2024-07-31T15:13:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa6c570740 1 -- 172.21.15.67:0/735749933 shutdown_connections 2024-07-31T15:13:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa6c570740 1 --2- 172.21.15.67:0/735749933 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5563acc60d80 0x5563ad062680 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa6c570740 1 --2- 172.21.15.67:0/735749933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acca21b0 0x5563ad05c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa6c570740 1 --2- 172.21.15.67:0/735749933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acc814d0 0x5563ad050db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa6c570740 1 --2- 172.21.15.67:0/735749933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc792f0 0x5563ad050870 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa42ff5640 0 client.20190 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa6c570740 1 -- 172.21.15.67:0/735749933 >> 172.21.15.67:0/735749933 conn(0x5563acc96300 msgr2=0x5563ad045d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa6c570740 1 -- 172.21.15.67:0/735749933 shutdown_connections 2024-07-31T15:13:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa6c570740 1 -- 172.21.15.67:0/735749933 wait complete. 2024-07-31T15:13:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.209+0000 7faa6c570740 20 client.20190 trim_cache size 0 max 16384 2024-07-31T15:13:30.230 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.228+0000 7faa6c570740 1 Processor -- start 2024-07-31T15:13:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa6c570740 1 -- start start 2024-07-31T15:13:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa6c570740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc792f0 0x5563ad050ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa6c570740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc814d0 0x5563ad05c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa6c570740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acca21b0 0x5563ad05c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa6c570740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faa601b2980 con 0x5563acc814d0 2024-07-31T15:13:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa6c570740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faa601b44c0 con 0x5563acc792f0 2024-07-31T15:13:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa6c570740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa601b3050 con 0x5563acca21b0 2024-07-31T15:13:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa5e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc814d0 0x5563ad05c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa5f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acca21b0 0x5563ad05c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa5e7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc814d0 0x5563ad05c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48468/0 (socket says 172.21.15.67:48468) 2024-07-31T15:13:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa5e7fc640 1 -- 172.21.15.67:0/654176162 learned_addr learned my addr 172.21.15.67:0/654176162 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa5effd640 1 --2- 172.21.15.67:0/654176162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc792f0 0x5563ad050ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa5f7fe640 1 -- 172.21.15.67:0/654176162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc792f0 msgr2=0x5563ad050ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa5f7fe640 1 --2- 172.21.15.67:0/654176162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc792f0 0x5563ad050ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa5f7fe640 1 -- 172.21.15.67:0/654176162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc814d0 msgr2=0x5563ad05c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa5f7fe640 1 --2- 172.21.15.67:0/654176162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc814d0 0x5563ad05c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.229+0000 7faa5f7fe640 1 -- 172.21.15.67:0/654176162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ad0503e0 con 0x5563acca21b0 2024-07-31T15:13:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.230+0000 7faa5f7fe640 1 --2- 172.21.15.67:0/654176162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acca21b0 0x5563ad05c8d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7faa58009bf0 tx=0x7faa5802a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55ec356f35a0df77 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.230+0000 7faa44ff9640 1 -- 172.21.15.67:0/654176162 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58003970 con 0x5563acca21b0 2024-07-31T15:13:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.230+0000 7faa44ff9640 1 -- 172.21.15.67:0/654176162 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa58036040 con 0x5563acca21b0 2024-07-31T15:13:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.230+0000 7faa44ff9640 1 -- 172.21.15.67:0/654176162 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5802f8c0 con 0x5563acca21b0 2024-07-31T15:13:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.231+0000 7faa6c570740 1 -- 172.21.15.67:0/654176162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acca21b0 msgr2=0x5563ad05c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.231+0000 7faa6c570740 1 --2- 172.21.15.67:0/654176162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acca21b0 0x5563ad05c8d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7faa58009bf0 tx=0x7faa5802a8c0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.231+0000 7faa6c570740 1 -- 172.21.15.67:0/654176162 shutdown_connections 2024-07-31T15:13:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.231+0000 7faa6c570740 1 --2- 172.21.15.67:0/654176162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acca21b0 0x5563ad05c8d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.231+0000 7faa6c570740 1 --2- 172.21.15.67:0/654176162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acc814d0 0x5563ad05c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.231+0000 7faa6c570740 1 --2- 172.21.15.67:0/654176162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc792f0 0x5563ad050ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.231+0000 7faa6c570740 1 -- 172.21.15.67:0/654176162 >> 172.21.15.67:0/654176162 conn(0x5563acc96300 msgr2=0x5563ad0505d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.231+0000 7faa6c570740 1 -- 172.21.15.67:0/654176162 shutdown_connections 2024-07-31T15:13:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.231+0000 7faa6c570740 1 -- 172.21.15.67:0/654176162 wait complete. 2024-07-31T15:13:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.231+0000 7faa6c570740 1 -- start start 2024-07-31T15:13:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa6c570740 1 Processor -- start 2024-07-31T15:13:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa6c570740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acc792f0 0x5563ad058c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa6c570740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc814d0 0x5563ad0550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa6c570740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acca21b0 0x5563ad055690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa6c570740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faa601b3df0 con 0x5563acca21b0 2024-07-31T15:13:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa6c570740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faa601b3720 con 0x5563acc814d0 2024-07-31T15:13:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa6c570740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa601b0980 con 0x5563acc792f0 2024-07-31T15:13:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa6c570740 10 client.0 _ll_register_callbacks cb 0x7faa50003bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa5effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acc792f0 0x5563ad058c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa5f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acca21b0 0x5563ad055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa5e7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc814d0 0x5563ad0550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa5effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acc792f0 0x5563ad058c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:37106/0 (socket says 172.21.15.67:37106) 2024-07-31T15:13:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa5effd640 1 -- 172.21.15.67:0/272061095 learned_addr learned my addr 172.21.15.67:0/272061095 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa5e7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc814d0 0x5563ad0550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:52230/0 (socket says 172.21.15.67:52230) 2024-07-31T15:13:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.232+0000 7faa5e7fc640 1 -- 172.21.15.67:0/272061095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acc792f0 msgr2=0x5563ad058c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa5e7fc640 1 --2- 172.21.15.67:0/272061095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acc792f0 0x5563ad058c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa5e7fc640 1 -- 172.21.15.67:0/272061095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acca21b0 msgr2=0x5563ad055690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa5e7fc640 1 --2- 172.21.15.67:0/272061095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acca21b0 0x5563ad055690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa5e7fc640 1 -- 172.21.15.67:0/272061095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ad047890 con 0x5563acc814d0 2024-07-31T15:13:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa5f7fe640 1 --2- 172.21.15.67:0/272061095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acca21b0 0x5563ad055690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa5effd640 1 --2- 172.21.15.67:0/272061095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acc792f0 0x5563ad058c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa5e7fc640 1 --2- 172.21.15.67:0/272061095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc814d0 0x5563ad0550f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7faa540a6f90 tx=0x7faa540e6ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff734bda572894d3 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:13:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa42ff5640 1 -- 172.21.15.67:0/272061095 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa540aaa20 con 0x5563acc814d0 2024-07-31T15:13:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa42ff5640 1 -- 172.21.15.67:0/272061095 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7faa540a71a0 con 0x5563acc814d0 2024-07-31T15:13:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa6c570740 1 -- 172.21.15.67:0/272061095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa58036040 con 0x5563acc814d0 2024-07-31T15:13:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa42ff5640 1 -- 172.21.15.67:0/272061095 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7faa540aaf10 con 0x5563acc814d0 2024-07-31T15:13:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa6c570740 10 client.29711 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa6c570740 1 -- 172.21.15.67:0/272061095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5563ad0c9ca0 con 0x5563acc814d0 2024-07-31T15:13:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.233+0000 7faa42ff5640 1 -- 172.21.15.67:0/272061095 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa58036040 con 0x5563acc814d0 2024-07-31T15:13:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.234+0000 7faa42ff5640 1 -- 172.21.15.67:0/272061095 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7faa540e4230 con 0x5563acc814d0 2024-07-31T15:13:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.234+0000 7faa6c570740 10 client.29711 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.234+0000 7faa6c570740 10 client.29711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.234+0000 7faa6c570740 20 client.29711 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.234+0000 7faa6c570740 1 --2- 172.21.15.67:0/272061095 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5563acc361c0 0x5563ad0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.234+0000 7faa6c570740 4 client.29711 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:13:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.234+0000 7faa6c570740 1 -- 172.21.15.67:0/272061095 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563acfb7da0 con 0x5563acc361c0 2024-07-31T15:13:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.234+0000 7faa5effd640 1 --2- 172.21.15.67:0/272061095 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5563acc361c0 0x5563ad0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.234+0000 7faa42ff5640 10 client.29711 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.235+0000 7faa5effd640 1 --2- 172.21.15.67:0/272061095 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5563acc361c0 0x5563ad0cc3c0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51a014d482f01d40 server_cookie=56666008dcbdbcd3 in_seq=0 out_seq=0 2024-07-31T15:13:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa42ff5640 1 -- 172.21.15.67:0/272061095 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563acfb7da0 con 0x5563acc361c0 2024-07-31T15:13:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa42ff5640 10 client.29711 handle_command_reply: tid=0 2024-07-31T15:13:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa6c570740 1 client.29711 shutdown 2024-07-31T15:13:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa6c570740 1 -- 172.21.15.67:0/272061095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc814d0 msgr2=0x5563ad0550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa6c570740 1 --2- 172.21.15.67:0/272061095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc814d0 0x5563ad0550f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7faa540a6f90 tx=0x7faa540e6ad0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa6c570740 1 -- 172.21.15.67:0/272061095 shutdown_connections 2024-07-31T15:13:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa6c570740 1 --2- 172.21.15.67:0/272061095 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5563acc361c0 0x5563ad0cc3c0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa6c570740 1 --2- 172.21.15.67:0/272061095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563acca21b0 0x5563ad055690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa6c570740 1 --2- 172.21.15.67:0/272061095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563acc814d0 0x5563ad0550f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa42ff5640 0 client.29711 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa6c570740 1 --2- 172.21.15.67:0/272061095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563acc792f0 0x5563ad058c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa6c570740 1 -- 172.21.15.67:0/272061095 >> 172.21.15.67:0/272061095 conn(0x5563acc96300 msgr2=0x5563ad051920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa6c570740 1 -- 172.21.15.67:0/272061095 shutdown_connections 2024-07-31T15:13:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa6c570740 1 -- 172.21.15.67:0/272061095 wait complete. 2024-07-31T15:13:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.336+0000 7faa6c570740 20 client.29711 trim_cache size 0 max 16384 2024-07-31T15:13:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:13:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:13:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:13:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:13:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.337+0000 7faa6567f640 1 -- 172.21.15.67:0/352159754 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faa480b3d70 msgr2=0x7faa480b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.337+0000 7faa6567f640 1 --2- 172.21.15.67:0/352159754 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faa480b3d70 0x7faa480b6280 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x5563ad043960 tx=0x7faa5000b390 comp rx=0 tx=0).stop 2024-07-31T15:13:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.337+0000 7faa6567f640 1 -- 172.21.15.67:0/352159754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa60108e30 msgr2=0x7faa60075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.337+0000 7faa6567f640 1 --2- 172.21.15.67:0/352159754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa60108e30 0x7faa60075680 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7faa5400e9f0 tx=0x7faa5400eeb0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.337+0000 7faa6567f640 1 -- 172.21.15.67:0/352159754 shutdown_connections 2024-07-31T15:13:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.337+0000 7faa6567f640 1 --2- 172.21.15.67:0/352159754 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7faa480b3d70 0x7faa480b6280 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.337+0000 7faa6567f640 1 --2- 172.21.15.67:0/352159754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa60109780 0x7faa60075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.337+0000 7faa6567f640 1 --2- 172.21.15.67:0/352159754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa60108e30 0x7faa60075680 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.337+0000 7faa6567f640 1 --2- 172.21.15.67:0/352159754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa60103500 0x7faa60075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.337+0000 7faa6567f640 1 -- 172.21.15.67:0/352159754 >> 172.21.15.67:0/352159754 conn(0x7faa600fef00 msgr2=0x7faa60106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.337+0000 7faa6567f640 1 -- 172.21.15.67:0/352159754 shutdown_connections 2024-07-31T15:13:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:30.337+0000 7faa6567f640 1 -- 172.21.15.67:0/352159754 wait complete. 2024-07-31T15:13:30.755 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:30.536+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000008171 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/power/supply/ auth{3=1} fragtree_t(*^3) v450 ap=3 RANDEPHEMERALPIN f(v2 m2024-07-31T15:11:41.904516+0000 130=130+0) n(v4 rc2024-07-31T15:11:41.905516+0000 b2648100 131=130+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:626730 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@9} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce89c56c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000008171:[<0x10000008145/supply v416>,<0x10000001d8f/power v16677>,<0x10000000202/drivers v3182>,<0x10000000201/linux-6.5.11 v445>,<0x10000000006/t v329>,<0x10000000005/tmp v299>,<0x100000001fb/client.0 v296>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v274>,<0x10000000001/sv_1 v299>,<0x10000000000/qa v280>,<0x1/volumes v280>]//[]","memoryvalue":"(2)0x10000008171:[<0x10000008145/supply v450>,<0x10000001d8f/power v19017>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 118=118+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:41.905516+0000 b2648100 131=130+1)","memory_value.dirstat":"f(v2 m2024-07-31T15:11:41.904516+0000 130=130+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:11:41.905516+0000 b2648100 131=130+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:30.756 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:30.549+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000021f0 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader/ auth{2=1} v893 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:04.418950+0000 19=18+1) n(v5 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:626856 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@12} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7dc21b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000021f0:[<0x100000021db/firmware_loader v414>,<0x10000001d8f/base v2486>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x100000021f0:[<0x100000021db/firmware_loader v893>,<0x10000001d8f/base v18747>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:01.469061+0000 13=12+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:04.418950+0000 19=18+1)","memory_value.rstat":"n(v5 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:30.756 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:30.611+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000006f7c [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/phy/ auth{3=1} fragtree_t(*^3) v3665 ap=3 RANDEPHEMERALPIN f(v3 m2024-07-31T15:10:56.134450+0000 111=110+1) n(v7 rc2024-07-31T15:10:57.456423+0000 b2820277 123=121+2) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:628354 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@14} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce86f32b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000006f7c:[<0x100000060b2/phy v1160>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x10000006f7c:[<0x100000060b2/phy v3665>,<0x10000001d8f/net v18833>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:10:56.134450+0000 103=102+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:10:57.456423+0000 b2820277 123=121+2)","memory_value.dirstat":"f(v3 m2024-07-31T15:10:56.134450+0000 111=110+1)","memory_value.rstat":"n(v7 rc2024-07-31T15:10:57.456423+0000 b2820277 123=121+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:30.756 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:30.756 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.756 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:30.756 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.756 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:30.756 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.757 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:30.757 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.757 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:13:30.757 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:30.757 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.757 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:30.757 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:30.757 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.757 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:30.757 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.757 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:30.757 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.757 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-mon[36714]: pgmap v1117: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 156 KiB/s wr, 696 op/s 2024-07-31T15:13:30.758 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:30.648+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000093bf [...25,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/thermal/intel/ auth{4=1} v992 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:45.716439+0000 29=28+1) n(v6 rc2024-07-31T15:11:45.722439+0000 b416255 48=46+2) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:627618 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@9} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce81e7d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000093bf:[<0x100000093a3/intel v467>,<0x10000001d8f/thermal v17380>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x100000093bf:[<0x100000093a3/intel v992>,<0x10000001d8f/thermal v18812>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:32.789903+0000 17=16+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:45.722439+0000 b416255 48=46+2)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:45.716439+0000 29=28+1)","memory_value.rstat":"n(v6 rc2024-07-31T15:11:45.722439+0000 b416255 48=46+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:30.758 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:30.753+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000048c5 [...27,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/hwtracing/intel_th/ auth{4=1} v514 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:12:40.723316+0000 21=21+0) n(v3 rc2024-07-31T15:12:40.725316+0000 b142954 22=21+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:628882 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@5} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce819e7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000048c5:[<0x1000000488c/intel_th v226>,<0x10000001d8f/hwtracing v9482>,<0x10000000202/drivers v2799>,<0x10000000201/linux-6.5.11 v334>,<0x10000000006/t v232>,<0x10000000005/tmp v202>,<0x100000001fb/client.0 v201>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v184>,<0x10000000001/sv_1 v213>,<0x10000000000/qa v197>,<0x1/volumes v198>]//[]","memoryvalue":"(2)0x100000048c5:[<0x1000000488c/intel_th v514>,<0x10000001d8f/hwtracing v20593>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:40.481244+0000 17=17+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:12:40.725316+0000 b142954 22=21+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:12:40.723316+0000 21=21+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:12:40.725316+0000 b142954 22=21+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:13:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:30 smithi067 ceph-mon[32750]: pgmap v1117: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 156 KiB/s wr, 696 op/s 2024-07-31T15:13:30.843 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '847f6b4e', quiesce took 33.5 seconds. Will release after: -21.8 2024-07-31T15:13:30.843 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:13:30.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:13:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:30.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:30.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:30.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:30.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:30.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:30.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:30 smithi077 ceph-mon[37327]: pgmap v1117: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 156 KiB/s wr, 696 op/s 2024-07-31T15:13:31.035 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:30.758+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002c0e [...27,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/xilinx/ auth{3=1} v1035 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:12:39.671338+0000 9=9+0) n(v3 rc2024-07-31T15:12:39.672338+0000 b60124 10=9+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:629042 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@6} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce80bf8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000002c0e:[<0x10000002b45/xilinx v328>,<0x10000001d8f/firmware v3927>,<0x10000000202/drivers v2561>,<0x10000000201/linux-6.5.11 v275>,<0x10000000006/t v183>,<0x10000000005/tmp v156>,<0x100000001fb/client.0 v155>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v140>,<0x10000000001/sv_1 v171>,<0x10000000000/qa v157>,<0x1/volumes v157>]//[]","memoryvalue":"(2)0x10000002c0e:[<0x10000002b45/xilinx v1035>,<0x10000001d8f/firmware v20638>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:16.051756+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:12:39.672338+0000 b60124 10=9+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:12:39.671338+0000 9=9+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:12:39.672338+0000 b60124 10=9+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:31.036 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:30.760+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002b48 [...27,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/arm_ffa/ auth{2=1} v1033 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:12:25.892619+0000 10=10+0) n(v3 rc2024-07-31T15:12:25.894619+0000 b28005 11=10+1) old_inodes=2 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:629054 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@6} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7ce0f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000002b48:[<0x10000002b45/arm_ffa v18>,<0x10000001d8f/firmware v3927>,<0x10000000202/drivers v2561>,<0x10000000201/linux-6.5.11 v275>,<0x10000000006/t v183>,<0x10000000005/tmp v156>,<0x100000001fb/client.0 v155>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v140>,<0x10000000001/sv_1 v171>,<0x10000000000/qa v157>,<0x1/volumes v157>]//[]","memoryvalue":"(2)0x10000002b48:[<0x10000002b45/arm_ffa v1033>,<0x10000001d8f/firmware v20638>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:15.738762+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:12:25.894619+0000 b28005 11=10+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:12:25.892619+0000 10=10+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:12:25.894619+0000 b28005 11=10+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:31.036 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:30 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:30.898+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000008a08 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip/ auth{3=1} v2681 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:09.647065+0000 7=7+0) n(v2 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:630186 nref=3)) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8c1aa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000008a08:[<0x1000000894f/microchip v613>,<0x10000001d8f/soc v16560>,<0x10000000202/drivers v3182>,<0x10000000201/linux-6.5.11 v445>,<0x10000000006/t v329>,<0x10000000005/tmp v299>,<0x100000001fb/client.0 v296>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v274>,<0x10000000001/sv_1 v299>,<0x10000000000/qa v280>,<0x1/volumes v280>]//[]","memoryvalue":"(2)0x10000008a08:[<0x1000000894f/microchip v2681>,<0x10000001d8f/soc v17222>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:51.041772+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:09.647065+0000 7=7+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:31.036 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:31.033+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001fc7 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica/ auth{2=1} fragtree_t(*^3) v1950 ap=3 RANDEPHEMERALPIN f(v10 m2024-07-31T15:07:39.153464+0000 516=516+0) n(v16 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1) old_inodes=4 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:631274 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@31} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7f0ba680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000001fc7:[<0x10000001fad/acpica v1734>,<0x10000001d8f/acpi v2434>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x10000001fc7:[<0x10000001fad/acpica v1950>,<0x10000001d8f/acpi v18672>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:07:37.321501+0000 494=494+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1)","memory_value.dirstat":"f(v10 m2024-07-31T15:07:39.153464+0000 516=516+0)","memory_value.rstat":"n(v16 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:31.155 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:13:31.293 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:31.090+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009462 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc/ auth{3=1} v1424 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:28.223687+0000 22=22+0) n(v4 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:631604 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@8} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce85c96b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009462:[<0x1000000945c/hvc v352>,<0x10000001d8f/tty v18356>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009462:[<0x1000000945c/hvc v1424>,<0x10000001d8f/tty v18901>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:32.992899+0000 16=16+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:28.223687+0000 22=22+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:31.293 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:31.166+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005d80 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader/ auth{4=1} v1324 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:10.777820+0000 21=21+0) n(v3 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1) old_inodes=1 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:632452 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7f8f5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000005d80:[<0x10000005d5d/cardreader v104>,<0x10000001d8f/misc v13245>,<0x10000000202/drivers v2910>,<0x10000000201/linux-6.5.11 v379>,<0x10000000006/t v275>,<0x10000000005/tmp v244>,<0x100000001fb/client.0 v243>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v221>,<0x10000000001/sv_1 v249>,<0x10000000000/qa v233>,<0x1/volumes v234>]//[]","memoryvalue":"(2)0x10000005d80:[<0x10000005d5d/cardreader v1324>,<0x10000001d8f/misc v18516>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:21.430390+0000 17=17+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:10.777820+0000 21=21+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:31.293 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:31.253+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000283c [...27,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/crypto/intel/ auth{1=1,4=1} v1335 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:12:27.079595+0000 9=6+3) n(v5 rc2024-07-31T15:12:27.080595+0000 b917229 146=134+12) old_inodes=2 (inest sync->mix g=1,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:633318 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@6} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7daf8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000283c:[<0x10000002719/intel v642>,<0x10000001d8f/crypto v3871>,<0x10000000202/drivers v2561>,<0x10000000201/linux-6.5.11 v275>,<0x10000000006/t v183>,<0x10000000005/tmp v156>,<0x100000001fb/client.0 v155>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v140>,<0x10000000001/sv_1 v171>,<0x10000000000/qa v157>,<0x1/volumes v157>]//[]","memoryvalue":"(2)0x1000000283c:[<0x10000002719/intel v1335>,<0x10000001d8f/crypto v19023>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:14.708784+0000 5=2+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:12:27.080595+0000 b917229 146=134+12)","memory_value.dirstat":"f(v1 m2024-07-31T15:12:27.079595+0000 9=6+3)","memory_value.rstat":"n(v5 rc2024-07-31T15:12:27.080595+0000 b917229 146=134+12)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:31.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.542 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.543 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000008171 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/power/supply) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000006f7c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/phy) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000093bf (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/thermal/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000048c5 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/hwtracing/intel_th) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000002c0e (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/xilinx) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000002b48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/arm_ffa) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:31.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.546 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:31 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000283c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/crypto/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.642 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:31.291+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000054d1 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media/common/v4l2-tpg/ auth{4=1} v342 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:36.520626+0000 8=8+0) n(v3 rc2024-07-31T15:11:36.521626+0000 b175839 9=8+1) old_inodes=1 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:633693 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7d165080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000054d1:[<0x100000054a1/v4l2-tpg v130>,<0x10000005460/common v1799>,<0x10000001d8f/media v12887>,<0x10000000202/drivers v2910>,<0x10000000201/linux-6.5.11 v379>,<0x10000000006/t v275>,<0x10000000005/tmp v244>,<0x100000001fb/client.0 v243>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v221>,<0x10000000001/sv_1 v249>,<0x10000000000/qa v233>,<0x1/volumes v234>]//[]","memoryvalue":"(2)0x100000054d1:[<0x100000054a1/v4l2-tpg v342>,<0x10000005460/common v3457>,<0x10000001d8f/media v18772>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:12.842569+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:36.521626+0000 b175839 9=8+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:36.520626+0000 8=8+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:11:36.521626+0000 b175839 9=8+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000008171 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/power/supply) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000006f7c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/phy) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000093bf (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/thermal/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000048c5 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/hwtracing/intel_th) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000002c0e (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/xilinx) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000002b48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/arm_ffa) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:31.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:31 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000283c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/crypto/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000008171 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/power/supply) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000006f7c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/phy) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000093bf (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/thermal/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000048c5 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/hwtracing/intel_th) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000002c0e (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/xilinx) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000002b48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/arm_ffa) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:31.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:31.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:13:31.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:31 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000283c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/crypto/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:13:32.607 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.607 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.607 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.607 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.607 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.607 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.607 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.607 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.607 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000054d1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media/common/v4l2-tpg) see mds.b log and `damage ls` output for details 2024-07-31T15:13:32.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: pgmap v1118: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 166 KiB/s wr, 1.14k op/s 2024-07-31T15:13:32.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:32.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:32.445+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000074a1 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/wireless/marvell/ auth{2=1} v1362 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:00.438362+0000 10=7+3) n(v6 rc2024-07-31T15:11:00.441362+0000 b1951078 100=96+4) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:635696 nref=3)) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8700c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000074a1:[<0x1000000708f/marvell v338>,<0x100000060b2/wireless v1686>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x100000074a1:[<0x1000000708f/marvell v1362>,<0x100000060b2/wireless v3767>,<0x10000001d8f/net v18833>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:33.434139+0000 6=3+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:00.441362+0000 b1951078 100=96+4)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:00.438362+0000 10=7+3)","memory_value.rstat":"n(v6 rc2024-07-31T15:11:00.441362+0000 b1951078 100=96+4)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:32.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:32.488+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000006b2f [...25,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/neterion/ auth{4=1} v2767 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:42.363507+0000 9=9+0) n(v4 rc2024-07-31T15:11:42.364507+0000 b310236 10=9+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:637400 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce81ca3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000006b2f:[<0x1000000622a/neterion v2003>,<0x100000060b2/ethernet v1684>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x10000006b2f:[<0x1000000622a/neterion v2767>,<0x100000060b2/ethernet v3787>,<0x10000001d8f/net v18833>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:28.646239+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:42.364507+0000 b310236 10=9+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:42.363507+0000 9=9+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:11:42.364507+0000 b310236 10=9+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:32.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:32.491+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000006af6 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/microsoft/ auth{4=1} v2613 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:38.452587+0000 7=6+1) n(v5 rc2024-07-31T15:11:38.453587+0000 b144243 15=13+2) old_inodes=1 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:637410 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce81722680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000006af6:[<0x1000000622a/microsoft v2181>,<0x100000060b2/ethernet v1684>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x10000006af6:[<0x1000000622a/microsoft v2613>,<0x100000060b2/ethernet v3787>,<0x10000001d8f/net v18833>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:28.579240+0000 3=2+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:38.453587+0000 b144243 15=13+2)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:38.452587+0000 7=6+1)","memory_value.rstat":"n(v5 rc2024-07-31T15:11:38.453587+0000 b144243 15=13+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:32.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:32.503+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000006bc8 [...27,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/pensando/ auth{2=1} v2570 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:48.960373+0000 7=6+1) n(v6 rc2024-07-31T15:11:48.962373+0000 b410323 33=31+2) old_inodes=3 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:637437 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce80ef8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000006bc8:[<0x1000000622a/pensando v2175>,<0x100000060b2/ethernet v1684>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x10000006bc8:[<0x1000000622a/pensando v2570>,<0x100000060b2/ethernet v3787>,<0x10000001d8f/net v18833>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:29.021231+0000 3=2+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:48.962373+0000 b410323 33=31+2)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:48.960373+0000 7=6+1)","memory_value.rstat":"n(v6 rc2024-07-31T15:11:48.962373+0000 b410323 33=31+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:32.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000054d1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media/common/v4l2-tpg) see mds.b log and `damage ls` output for details 2024-07-31T15:13:32.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: pgmap v1118: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 166 KiB/s wr, 1.14k op/s 2024-07-31T15:13:32.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:32.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000054d1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media/common/v4l2-tpg) see mds.b log and `damage ls` output for details 2024-07-31T15:13:32.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:32.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: pgmap v1118: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 166 KiB/s wr, 1.14k op/s 2024-07-31T15:13:32.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:32.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.066+0000 7ff48753d640 1 -- 172.21.15.67:0/4021446239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480108ed0 msgr2=0x7ff4801092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.066+0000 7ff48753d640 1 --2- 172.21.15.67:0/4021446239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480108ed0 0x7ff4801092e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff470009900 tx=0x7ff47002a410 comp rx=0 tx=0).stop 2024-07-31T15:13:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.066+0000 7ff48753d640 1 -- 172.21.15.67:0/4021446239 shutdown_connections 2024-07-31T15:13:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.066+0000 7ff48753d640 1 --2- 172.21.15.67:0/4021446239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff480109820 0x7ff480110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.066+0000 7ff48753d640 1 --2- 172.21.15.67:0/4021446239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480108ed0 0x7ff4801092e0 secure :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff470009900 tx=0x7ff47002a410 comp rx=0 tx=0).stop 2024-07-31T15:13:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.066+0000 7ff48753d640 1 --2- 172.21.15.67:0/4021446239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff480103590 0x7ff4801039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.066+0000 7ff48753d640 1 -- 172.21.15.67:0/4021446239 >> 172.21.15.67:0/4021446239 conn(0x7ff4800ff000 msgr2=0x7ff480101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.067+0000 7ff48753d640 1 -- 172.21.15.67:0/4021446239 shutdown_connections 2024-07-31T15:13:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.067+0000 7ff48753d640 1 -- 172.21.15.67:0/4021446239 wait complete. 2024-07-31T15:13:33.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.067+0000 7ff48753d640 1 Processor -- start 2024-07-31T15:13:33.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.067+0000 7ff48753d640 1 -- start start 2024-07-31T15:13:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff48753d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff480103590 0x7ff480075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff48753d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff480109820 0x7ff480075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff48753d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480075bc0 0x7ff4801a80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff48753d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff480113490 con 0x7ff480075bc0 2024-07-31T15:13:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff48753d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff480113310 con 0x7ff480103590 2024-07-31T15:13:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff48753d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff480113610 con 0x7ff480109820 2024-07-31T15:13:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff485ada640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480075bc0 0x7ff4801a80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff4852d9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff480103590 0x7ff480075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff484ad8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff480109820 0x7ff480075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff4852d9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff480103590 0x7ff480075140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56882/0 (socket says 172.21.15.67:56882) 2024-07-31T15:13:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff485ada640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480075bc0 0x7ff4801a80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39572/0 (socket says 172.21.15.67:39572) 2024-07-31T15:13:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff4852d9640 1 -- 172.21.15.67:0/1761714716 learned_addr learned my addr 172.21.15.67:0/1761714716 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff484ad8640 1 -- 172.21.15.67:0/1761714716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff480103590 msgr2=0x7ff480075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff484ad8640 1 --2- 172.21.15.67:0/1761714716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff480103590 0x7ff480075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff484ad8640 1 -- 172.21.15.67:0/1761714716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480075bc0 msgr2=0x7ff4801a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff484ad8640 1 --2- 172.21.15.67:0/1761714716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480075bc0 0x7ff4801a80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.068+0000 7ff484ad8640 1 -- 172.21.15.67:0/1761714716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4801a86e0 con 0x7ff480109820 2024-07-31T15:13:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.069+0000 7ff484ad8640 1 --2- 172.21.15.67:0/1761714716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff480109820 0x7ff480075680 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff48006c3c0 tx=0x7ff470004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.069+0000 7ff4767fc640 1 -- 172.21.15.67:0/1761714716 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47003f070 con 0x7ff480109820 2024-07-31T15:13:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.069+0000 7ff4767fc640 1 -- 172.21.15.67:0/1761714716 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff470003aa0 con 0x7ff480109820 2024-07-31T15:13:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.069+0000 7ff4767fc640 1 -- 172.21.15.67:0/1761714716 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470035070 con 0x7ff480109820 2024-07-31T15:13:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.069+0000 7ff48753d640 1 -- 172.21.15.67:0/1761714716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4801a8970 con 0x7ff480109820 2024-07-31T15:13:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.069+0000 7ff48753d640 1 -- 172.21.15.67:0/1761714716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4801a8dd0 con 0x7ff480109820 2024-07-31T15:13:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.070+0000 7ff48753d640 1 -- 172.21.15.67:0/1761714716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff480104810 con 0x7ff480109820 2024-07-31T15:13:33.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.072+0000 7ff4767fc640 1 -- 172.21.15.67:0/1761714716 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff470003c50 con 0x7ff480109820 2024-07-31T15:13:33.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.073+0000 7ff4767fc640 1 --2- 172.21.15.67:0/1761714716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff46c0b3dc0 0x7ff46c0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:33.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.073+0000 7ff4852d9640 1 --2- 172.21.15.67:0/1761714716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff46c0b3dc0 0x7ff46c0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:33.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.073+0000 7ff4767fc640 1 -- 172.21.15.67:0/1761714716 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7ff470104a90 con 0x7ff480109820 2024-07-31T15:13:33.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.074+0000 7ff4852d9640 1 --2- 172.21.15.67:0/1761714716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff46c0b3dc0 0x7ff46c0b62d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff480076b90 tx=0x7ff47c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:33.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.076+0000 7ff4767fc640 1 -- 172.21.15.67:0/1761714716 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff4700cb9d0 con 0x7ff480109820 2024-07-31T15:13:33.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.356+0000 7ff48753d640 1 -- 172.21.15.67:0/1761714716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4801092e0 con 0x7ff480109820 2024-07-31T15:13:33.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.356+0000 7ff4767fc640 1 -- 172.21.15.67:0/1761714716 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11959 (secure 0 0 0) 0x7ff4700cb2d0 con 0x7ff480109820 2024-07-31T15:13:33.358 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:13:33.358 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":66,"btime":"2024-07-31T15:10:30:235518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:10:30.235494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29129":{"gid":29129,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/631186801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":631186801},{"type":"v1","addr":"172.21.15.73:6835","nonce":631186801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:13:33.359 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 66 2024-07-31T15:13:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.359+0000 7ff48753d640 1 -- 172.21.15.67:0/1761714716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff46c0b3dc0 msgr2=0x7ff46c0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.359+0000 7ff48753d640 1 --2- 172.21.15.67:0/1761714716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff46c0b3dc0 0x7ff46c0b62d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff480076b90 tx=0x7ff47c005f90 comp rx=0 tx=0).stop 2024-07-31T15:13:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.359+0000 7ff48753d640 1 -- 172.21.15.67:0/1761714716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff480109820 msgr2=0x7ff480075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.359+0000 7ff48753d640 1 --2- 172.21.15.67:0/1761714716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff480109820 0x7ff480075680 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff48006c3c0 tx=0x7ff470004040 comp rx=0 tx=0).stop 2024-07-31T15:13:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.359+0000 7ff48753d640 1 -- 172.21.15.67:0/1761714716 shutdown_connections 2024-07-31T15:13:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.359+0000 7ff48753d640 1 --2- 172.21.15.67:0/1761714716 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff46c0b3dc0 0x7ff46c0b62d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.359+0000 7ff48753d640 1 --2- 172.21.15.67:0/1761714716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480075bc0 0x7ff4801a80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.359+0000 7ff48753d640 1 --2- 172.21.15.67:0/1761714716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff480109820 0x7ff480075680 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.359+0000 7ff48753d640 1 --2- 172.21.15.67:0/1761714716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff480103590 0x7ff480075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.359+0000 7ff48753d640 1 -- 172.21.15.67:0/1761714716 >> 172.21.15.67:0/1761714716 conn(0x7ff4800ff000 msgr2=0x7ff480101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.359+0000 7ff48753d640 1 -- 172.21.15.67:0/1761714716 shutdown_connections 2024-07-31T15:13:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:33.359+0000 7ff48753d640 1 -- 172.21.15.67:0/1761714716 wait complete. 2024-07-31T15:13:33.392 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:33.037+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000948f [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250/ auth{4=1} v716 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:35.607536+0000 75=75+0) n(v5 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:642166 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@8} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce836e4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000948f:[<0x1000000948d/8250 v533>,<0x1000000945c/serial v657>,<0x10000001d8f/tty v18356>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x1000000948f:[<0x1000000948d/8250 v716>,<0x1000000945c/serial v1492>,<0x10000001d8f/tty v18901>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:33.063897+0000 47=47+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:35.607536+0000 75=75+0)","memory_value.rstat":"n(v5 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:33.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000074a1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/wireless/marvell) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000006b2f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/neterion) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000006af6 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/microsoft) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000006bc8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/pensando) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:33.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:33 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1761714716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:13:33.892 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:33.702+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000006803 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/marvell/octeontx2/ auth{2=1} v383 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:38.527585+0000 8=6+2) n(v5 rc2024-07-31T15:11:38.528585+0000 b1832909 74=71+3) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:644305 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8265c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000006803:[<0x100000067e0/octeontx2 v159>,<0x1000000622a/marvell v2175>,<0x100000060b2/ethernet v1684>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x10000006803:[<0x100000067e0/octeontx2 v383>,<0x1000000622a/marvell v2657>,<0x100000060b2/ethernet v3787>,<0x10000001d8f/net v18833>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:27.598261+0000 4=2+2)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:38.528585+0000 b1832909 74=71+3)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:38.527585+0000 8=6+2)","memory_value.rstat":"n(v5 rc2024-07-31T15:11:38.528585+0000 b1832909 74=71+3)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:33.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000074a1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/wireless/marvell) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000006b2f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/neterion) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000006af6 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/microsoft) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000006bc8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/pensando) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:33.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:33 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1761714716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:13:33.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000074a1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/wireless/marvell) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000006b2f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/neterion) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000006af6 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/microsoft) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000006bc8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/pensando) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:33.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:33.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:13:33.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:33 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1761714716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:13:34.117 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --set-id 847f6b4e --release --await' 2024-07-31T15:13:34.117 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --set-id 847f6b4e --release --await 2024-07-31T15:13:34.392 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:34.077+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000439c [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du/ auth{2=1} v303 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:01.164016+0000 34=34+0) n(v3 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:647471 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce83f19600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000439c:[<0x10000004399/rcar-du v163>,<0x10000002d23/renesas v2988>,<0x10000002d21/drm v1196>,<0x10000001d8f/gpu v9523>,<0x10000000202/drivers v2799>,<0x10000000201/linux-6.5.11 v334>,<0x10000000006/t v232>,<0x10000000005/tmp v202>,<0x100000001fb/client.0 v201>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v184>,<0x10000000001/sv_1 v213>,<0x10000000000/qa v197>,<0x1/volumes v198>]//[]","memoryvalue":"(2)0x1000000439c:[<0x10000004399/rcar-du v303>,<0x10000002d23/renesas v3147>,<0x10000002d21/drm v3141>,<0x10000001d8f/gpu v19140>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:38.082295+0000 30=30+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:01.164016+0000 34=34+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:34.427 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:13:34.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000006803 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/marvell/octeontx2) see mds.b log and `damage ls` output for details 2024-07-31T15:13:34.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:34.802 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.802 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: pgmap v1119: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 316 KiB/s wr, 2.31k op/s 2024-07-31T15:13:34.802 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:34 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:13:34.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000006803 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/marvell/octeontx2) see mds.b log and `damage ls` output for details 2024-07-31T15:13:34.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:34.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: pgmap v1119: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 316 KiB/s wr, 2.31k op/s 2024-07-31T15:13:34.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:13:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000006803 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/marvell/octeontx2) see mds.b log and `damage ls` output for details 2024-07-31T15:13:34.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:34.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:34.995 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: pgmap v1119: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 316 KiB/s wr, 2.31k op/s 2024-07-31T15:13:34.995 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:34 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:13:35.392 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:13:34 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:13:34.978+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000003a1a [...25,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/i915/gt/uc/ auth{4=1} v637 ap=3 RANDEPHEMERALPIN f(v2 m2024-07-31T15:11:46.662419+0000 99=98+1) n(v5 rc2024-07-31T15:11:46.662419+0000 b2391460 107=105+2) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:648620 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@19} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7f0e3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000003a1a:[<0x10000003975/uc v416>,<0x10000003833/gt v1435>,<0x10000002d23/i915 v1759>,<0x10000002d21/drm v303>,<0x10000001d8f/gpu v6343>,<0x10000000202/drivers v2617>,<0x10000000201/linux-6.5.11 v300>,<0x10000000006/t v206>,<0x10000000005/tmp v176>,<0x100000001fb/client.0 v176>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v156>,<0x10000000001/sv_1 v190>,<0x10000000000/qa v173>,<0x1/volumes v174>]//[]","memoryvalue":"(2)0x10000003a1a:[<0x10000003975/uc v637>,<0x10000003833/gt v3185>,<0x10000002d23/i915 v4641>,<0x10000002d21/drm v3141>,<0x10000001d8f/gpu v19140>,<0x10000000202/drivers v22247>,<0x10000000201/linux-6.5.11 v2876>,<0x10000000006/t v1846>,<0x10000000005/tmp v1769>,<0x100000001fb/client.0 v1738>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1702>,<0x10000000001/sv_1 v1700>,<0x10000000000/qa v1686>,<0x1/volumes v1635>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:11:44.949454+0000 91=90+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:46.662419+0000 b2391460 107=105+2)","memory_value.dirstat":"f(v2 m2024-07-31T15:11:46.662419+0000 99=98+1)","memory_value.rstat":"n(v5 rc2024-07-31T15:11:46.662419+0000 b2391460 107=105+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:13:35.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:35.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:35 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000003a1a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/i915/gt/uc) see mds.b log and `damage ls` output for details 2024-07-31T15:13:35.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:35.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:35 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000003a1a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/i915/gt/uc) see mds.b log and `damage ls` output for details 2024-07-31T15:13:35.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:35.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:35.993 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:35 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000003a1a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/i915/gt/uc) see mds.b log and `damage ls` output for details 2024-07-31T15:13:36.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.610 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.611 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.612 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.614 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:13:36.614 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.614 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.614 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:36 smithi067 ceph-mon[32750]: pgmap v1120: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 316 KiB/s wr, 2.94k op/s 2024-07-31T15:13:36.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.610+0000 7fe99373e640 1 -- 172.21.15.67:0/4159354672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c1024b0 msgr2=0x7fe98c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:36.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.610+0000 7fe99373e640 1 --2- 172.21.15.67:0/4159354672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c1024b0 0x7fe98c1028c0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe988009900 tx=0x7fe98802a410 comp rx=0 tx=0).stop 2024-07-31T15:13:36.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.610+0000 7fe99373e640 1 -- 172.21.15.67:0/4159354672 shutdown_connections 2024-07-31T15:13:36.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.610+0000 7fe99373e640 1 --2- 172.21.15.67:0/4159354672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c108740 0x7fe98c10ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.610+0000 7fe99373e640 1 --2- 172.21.15.67:0/4159354672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98c107df0 0x7fe98c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.610+0000 7fe99373e640 1 --2- 172.21.15.67:0/4159354672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c1024b0 0x7fe98c1028c0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.610+0000 7fe99373e640 1 -- 172.21.15.67:0/4159354672 >> 172.21.15.67:0/4159354672 conn(0x7fe98c079170 msgr2=0x7fe98c079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:36.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.610+0000 7fe99373e640 1 -- 172.21.15.67:0/4159354672 shutdown_connections 2024-07-31T15:13:36.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.610+0000 7fe99373e640 1 -- 172.21.15.67:0/4159354672 wait complete. 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.611+0000 7fe99373e640 1 Processor -- start 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.611+0000 7fe99373e640 1 -- start start 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.611+0000 7fe99373e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c1024b0 0x7fe98c199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.611+0000 7fe99373e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98c107df0 0x7fe98c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.611+0000 7fe99373e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c108740 0x7fe98c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.611+0000 7fe99373e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1123c0 con 0x7fe98c108740 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.611+0000 7fe99373e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c112240 con 0x7fe98c1024b0 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.611+0000 7fe99373e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c112540 con 0x7fe98c107df0 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe991cdb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c108740 0x7fe98c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe991cdb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c108740 0x7fe98c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39598/0 (socket says 172.21.15.67:39598) 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe991cdb640 1 -- 172.21.15.67:0/4279318343 learned_addr learned my addr 172.21.15.67:0/4279318343 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe9914da640 1 --2- 172.21.15.67:0/4279318343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c1024b0 0x7fe98c199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe990cd9640 1 --2- 172.21.15.67:0/4279318343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98c107df0 0x7fe98c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe991cdb640 1 -- 172.21.15.67:0/4279318343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98c107df0 msgr2=0x7fe98c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe991cdb640 1 --2- 172.21.15.67:0/4279318343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98c107df0 0x7fe98c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe991cdb640 1 -- 172.21.15.67:0/4279318343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c1024b0 msgr2=0x7fe98c199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe991cdb640 1 --2- 172.21.15.67:0/4279318343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c1024b0 0x7fe98c199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe991cdb640 1 -- 172.21.15.67:0/4279318343 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c19fe50 con 0x7fe98c108740 2024-07-31T15:13:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe991cdb640 1 --2- 172.21.15.67:0/4279318343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c108740 0x7fe98c19f720 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe98400eb10 tx=0x7fe98400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe9827fc640 1 -- 172.21.15.67:0/4279318343 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984004b20 con 0x7fe98c108740 2024-07-31T15:13:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe9827fc640 1 -- 172.21.15.67:0/4279318343 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe984004cc0 con 0x7fe98c108740 2024-07-31T15:13:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe99373e640 1 -- 172.21.15.67:0/4279318343 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98c1a0140 con 0x7fe98c108740 2024-07-31T15:13:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.612+0000 7fe9827fc640 1 -- 172.21.15.67:0/4279318343 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984010670 con 0x7fe98c108740 2024-07-31T15:13:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.613+0000 7fe99373e640 1 -- 172.21.15.67:0/4279318343 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98c1a68a0 con 0x7fe98c108740 2024-07-31T15:13:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.615+0000 7fe999c8f740 1 Processor -- start 2024-07-31T15:13:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.615+0000 7fe9827fc640 1 -- 172.21.15.67:0/4279318343 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe9840108f0 con 0x7fe98c108740 2024-07-31T15:13:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe999c8f740 1 -- start start 2024-07-31T15:13:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe999c8f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f11bd520 0x5609f1563070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe999c8f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f1194660 0x5609f15635b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe999c8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f119c840 0x5609f1563af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe999c8f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5609f14d8e70 con 0x5609f11bd520 2024-07-31T15:13:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe999c8f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5609f14c8370 con 0x5609f1194660 2024-07-31T15:13:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe999c8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5609f14cc3b0 con 0x5609f119c840 2024-07-31T15:13:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe990cd9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f11bd520 0x5609f1563070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe990cd9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f11bd520 0x5609f1563070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39600/0 (socket says 172.21.15.67:39600) 2024-07-31T15:13:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe990cd9640 1 -- 172.21.15.67:0/835684577 learned_addr learned my addr 172.21.15.67:0/835684577 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe990cd9640 1 --2- 172.21.15.67:0/835684577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f119c840 0x5609f1563af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe9914da640 1 --2- 172.21.15.67:0/835684577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f1194660 0x5609f15635b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe990cd9640 1 -- 172.21.15.67:0/835684577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f119c840 msgr2=0x5609f1563af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe990cd9640 1 --2- 172.21.15.67:0/835684577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f119c840 0x5609f1563af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe990cd9640 1 -- 172.21.15.67:0/835684577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f1194660 msgr2=0x5609f15635b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe990cd9640 1 --2- 172.21.15.67:0/835684577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f1194660 0x5609f15635b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.616+0000 7fe990cd9640 1 -- 172.21.15.67:0/835684577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609f1495040 con 0x5609f11bd520 2024-07-31T15:13:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.617+0000 7fe9827fc640 1 --2- 172.21.15.67:0/4279318343 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe9780b3d40 0x7fe9780b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.617+0000 7fe990cd9640 1 --2- 172.21.15.67:0/835684577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f11bd520 0x5609f1563070 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe97c013780 tx=0x7fe97c009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823ab72330b22a3e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.617+0000 7fe9914da640 1 --2- 172.21.15.67:0/4279318343 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe9780b3d40 0x7fe9780b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.617+0000 7fe9827fc640 1 -- 172.21.15.67:0/4279318343 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7fe9840e4940 con 0x7fe98c108740 2024-07-31T15:13:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.617+0000 7fe9777fe640 1 -- 172.21.15.67:0/835684577 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c013130 con 0x5609f11bd520 2024-07-31T15:13:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.617+0000 7fe9777fe640 1 -- 172.21.15.67:0/835684577 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe97c005020 con 0x5609f11bd520 2024-07-31T15:13:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.617+0000 7fe9777fe640 1 -- 172.21.15.67:0/835684577 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c013130 con 0x5609f11bd520 2024-07-31T15:13:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.618+0000 7fe999c8f740 1 -- 172.21.15.67:0/835684577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f11bd520 msgr2=0x5609f1563070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.618+0000 7fe999c8f740 1 --2- 172.21.15.67:0/835684577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f11bd520 0x5609f1563070 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe97c013780 tx=0x7fe97c009830 comp rx=0 tx=0).stop 2024-07-31T15:13:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.618+0000 7fe9914da640 1 --2- 172.21.15.67:0/4279318343 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe9780b3d40 0x7fe9780b6250 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe988031440 tx=0x7fe988009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.618+0000 7fe999c8f740 1 -- 172.21.15.67:0/835684577 shutdown_connections 2024-07-31T15:13:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.618+0000 7fe999c8f740 1 --2- 172.21.15.67:0/835684577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f119c840 0x5609f1563af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.618+0000 7fe999c8f740 1 --2- 172.21.15.67:0/835684577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f1194660 0x5609f15635b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.618+0000 7fe999c8f740 1 --2- 172.21.15.67:0/835684577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f11bd520 0x5609f1563070 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.618+0000 7fe999c8f740 1 -- 172.21.15.67:0/835684577 >> 172.21.15.67:0/835684577 conn(0x5609f11b1910 msgr2=0x5609f1560f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.618+0000 7fe999c8f740 1 -- 172.21.15.67:0/835684577 shutdown_connections 2024-07-31T15:13:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.618+0000 7fe999c8f740 1 -- 172.21.15.67:0/835684577 wait complete. 2024-07-31T15:13:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.618+0000 7fe999c8f740 1 -- start start 2024-07-31T15:13:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.619+0000 7fe999c8f740 1 Processor -- start 2024-07-31T15:13:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.619+0000 7fe999c8f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f1194660 0x5609f156ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.619+0000 7fe999c8f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f119c840 0x5609f156f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.619+0000 7fe999c8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f11bd520 0x5609f157ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.619+0000 7fe999c8f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5609f132bbb0 con 0x5609f119c840 2024-07-31T15:13:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.619+0000 7fe999c8f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5609f146aac0 con 0x5609f1194660 2024-07-31T15:13:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.619+0000 7fe999c8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5609f15717f0 con 0x5609f11bd520 2024-07-31T15:13:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.619+0000 7fe999c8f740 10 client.0 _ll_register_callbacks cb 0x5609f14733b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.619+0000 7fe990cd9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f119c840 0x5609f156f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.619+0000 7fe990cd9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f119c840 0x5609f156f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39614/0 (socket says 172.21.15.67:39614) 2024-07-31T15:13:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.619+0000 7fe990cd9640 1 -- 172.21.15.67:0/600175388 learned_addr learned my addr 172.21.15.67:0/600175388 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:36.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.619+0000 7fe991cdb640 1 --2- 172.21.15.67:0/600175388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f11bd520 0x5609f157ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.619+0000 7fe9914da640 1 --2- 172.21.15.67:0/600175388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f1194660 0x5609f156ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe991cdb640 1 -- 172.21.15.67:0/600175388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f1194660 msgr2=0x5609f156ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:36.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe991cdb640 1 --2- 172.21.15.67:0/600175388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f1194660 0x5609f156ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe991cdb640 1 -- 172.21.15.67:0/600175388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f119c840 msgr2=0x5609f156f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:36.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe991cdb640 1 --2- 172.21.15.67:0/600175388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f119c840 0x5609f156f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe991cdb640 1 -- 172.21.15.67:0/600175388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609f1228b00 con 0x5609f11bd520 2024-07-31T15:13:36.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe990cd9640 1 --2- 172.21.15.67:0/600175388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f119c840 0x5609f156f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:36.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe991cdb640 1 --2- 172.21.15.67:0/600175388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f11bd520 0x5609f157ab10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe9840e1d20 tx=0x7fe9840aab30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=255f46d28ec8d9fb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe974ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:13:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe974ff9640 1 -- 172.21.15.67:0/600175388 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9840ab8e0 con 0x5609f11bd520 2024-07-31T15:13:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe999c8f740 1 -- 172.21.15.67:0/600175388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe97c005020 con 0x5609f11bd520 2024-07-31T15:13:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe974ff9640 1 -- 172.21.15.67:0/600175388 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7fe9840a77d0 con 0x5609f11bd520 2024-07-31T15:13:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe974ff9640 1 -- 172.21.15.67:0/600175388 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fe9840ab1e0 con 0x5609f11bd520 2024-07-31T15:13:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe999c8f740 10 client.29575 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe999c8f740 1 -- 172.21.15.67:0/600175388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5609f129f960 con 0x5609f11bd520 2024-07-31T15:13:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.620+0000 7fe974ff9640 1 -- 172.21.15.67:0/600175388 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe97c005020 con 0x5609f11bd520 2024-07-31T15:13:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.621+0000 7fe974ff9640 1 -- 172.21.15.67:0/600175388 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fe9840b3e00 con 0x5609f11bd520 2024-07-31T15:13:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.621+0000 7fe999c8f740 10 client.29575 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.621+0000 7fe999c8f740 10 client.29575 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:13:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.621+0000 7fe999c8f740 20 client.29575 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.621+0000 7fe999c8f740 1 --2- 172.21.15.67:0/600175388 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5609f117bea0 0x5609f1580b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.621+0000 7fe999c8f740 4 client.29575 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:13:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.621+0000 7fe999c8f740 1 -- 172.21.15.67:0/600175388 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609f14dadc0 con 0x5609f117bea0 2024-07-31T15:13:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.621+0000 7fe9914da640 1 --2- 172.21.15.67:0/600175388 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5609f117bea0 0x5609f1580b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.621+0000 7fe9914da640 1 --2- 172.21.15.67:0/600175388 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5609f117bea0 0x5609f1580b40 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51878b116d5981c9 server_cookie=e74906453957b08 in_seq=0 out_seq=0 2024-07-31T15:13:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.621+0000 7fe974ff9640 10 client.29575 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:36.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.622+0000 7fe974ff9640 1 -- 172.21.15.67:0/600175388 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609f14dadc0 con 0x5609f117bea0 2024-07-31T15:13:36.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.622+0000 7fe974ff9640 10 client.29575 handle_command_reply: tid=0 2024-07-31T15:13:36.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.622+0000 7fe999c8f740 1 client.29575 shutdown 2024-07-31T15:13:36.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.622+0000 7fe999c8f740 1 -- 172.21.15.67:0/600175388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f11bd520 msgr2=0x5609f157ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.622+0000 7fe999c8f740 1 --2- 172.21.15.67:0/600175388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f11bd520 0x5609f157ab10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe9840e1d20 tx=0x7fe9840aab30 comp rx=0 tx=0).stop 2024-07-31T15:13:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.622+0000 7fe999c8f740 1 -- 172.21.15.67:0/600175388 shutdown_connections 2024-07-31T15:13:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.622+0000 7fe999c8f740 1 --2- 172.21.15.67:0/600175388 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5609f117bea0 0x5609f1580b40 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.622+0000 7fe999c8f740 1 --2- 172.21.15.67:0/600175388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f11bd520 0x5609f157ab10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.622+0000 7fe999c8f740 1 --2- 172.21.15.67:0/600175388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f119c840 0x5609f156f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.622+0000 7fe999c8f740 1 --2- 172.21.15.67:0/600175388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f1194660 0x5609f156ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.622+0000 7fe974ff9640 0 client.29575 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.622+0000 7fe999c8f740 1 -- 172.21.15.67:0/600175388 >> 172.21.15.67:0/600175388 conn(0x5609f11b1910 msgr2=0x5609f1566220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.623+0000 7fe999c8f740 1 -- 172.21.15.67:0/600175388 shutdown_connections 2024-07-31T15:13:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.623+0000 7fe999c8f740 1 -- 172.21.15.67:0/600175388 wait complete. 2024-07-31T15:13:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.623+0000 7fe999c8f740 20 client.29575 trim_cache size 0 max 16384 2024-07-31T15:13:36.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe999c8f740 1 Processor -- start 2024-07-31T15:13:36.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe999c8f740 1 -- start start 2024-07-31T15:13:36.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe999c8f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f1194660 0x5609f1569c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe999c8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f119c840 0x5609f157a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe999c8f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f11bd520 0x5609f15862e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe999c8f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1b0730 con 0x5609f1194660 2024-07-31T15:13:36.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe999c8f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1b2270 con 0x5609f11bd520 2024-07-31T15:13:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe999c8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1b0e00 con 0x5609f119c840 2024-07-31T15:13:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe9914da640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f1194660 0x5609f1569c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe9914da640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f1194660 0x5609f1569c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39622/0 (socket says 172.21.15.67:39622) 2024-07-31T15:13:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe9914da640 1 -- 172.21.15.67:0/2924877688 learned_addr learned my addr 172.21.15.67:0/2924877688 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe990cd9640 1 --2- 172.21.15.67:0/2924877688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f119c840 0x5609f157a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe991cdb640 1 --2- 172.21.15.67:0/2924877688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f11bd520 0x5609f15862e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe9914da640 1 -- 172.21.15.67:0/2924877688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f119c840 msgr2=0x5609f157a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe9914da640 1 --2- 172.21.15.67:0/2924877688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f119c840 0x5609f157a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe9914da640 1 -- 172.21.15.67:0/2924877688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f11bd520 msgr2=0x5609f15862e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe9914da640 1 --2- 172.21.15.67:0/2924877688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f11bd520 0x5609f15862e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe9914da640 1 -- 172.21.15.67:0/2924877688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609f156d9f0 con 0x5609f1194660 2024-07-31T15:13:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.641+0000 7fe991cdb640 1 --2- 172.21.15.67:0/2924877688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f11bd520 0x5609f15862e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:13:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.642+0000 7fe9914da640 1 --2- 172.21.15.67:0/2924877688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f1194660 0x5609f1569c40 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe9880315b0 tx=0x7fe98803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52a211fad245e65 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.642+0000 7fe976ffd640 1 -- 172.21.15.67:0/2924877688 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988035040 con 0x5609f1194660 2024-07-31T15:13:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.642+0000 7fe976ffd640 1 -- 172.21.15.67:0/2924877688 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe988059070 con 0x5609f1194660 2024-07-31T15:13:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.642+0000 7fe976ffd640 1 -- 172.21.15.67:0/2924877688 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988004ad0 con 0x5609f1194660 2024-07-31T15:13:36.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.643+0000 7fe999c8f740 1 -- 172.21.15.67:0/2924877688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f1194660 msgr2=0x5609f1569c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:36.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.643+0000 7fe999c8f740 1 --2- 172.21.15.67:0/2924877688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f1194660 0x5609f1569c40 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe9880315b0 tx=0x7fe98803f000 comp rx=0 tx=0).stop 2024-07-31T15:13:36.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.643+0000 7fe999c8f740 1 -- 172.21.15.67:0/2924877688 shutdown_connections 2024-07-31T15:13:36.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.643+0000 7fe999c8f740 1 --2- 172.21.15.67:0/2924877688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f11bd520 0x5609f15862e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.643+0000 7fe999c8f740 1 --2- 172.21.15.67:0/2924877688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f119c840 0x5609f157a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.643+0000 7fe999c8f740 1 --2- 172.21.15.67:0/2924877688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f1194660 0x5609f1569c40 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.643+0000 7fe999c8f740 1 -- 172.21.15.67:0/2924877688 >> 172.21.15.67:0/2924877688 conn(0x5609f11b1910 msgr2=0x5609f1560880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:36.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.643+0000 7fe999c8f740 1 -- 172.21.15.67:0/2924877688 shutdown_connections 2024-07-31T15:13:36.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.643+0000 7fe999c8f740 1 -- 172.21.15.67:0/2924877688 wait complete. 2024-07-31T15:13:36.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.643+0000 7fe999c8f740 1 -- start start 2024-07-31T15:13:36.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.643+0000 7fe999c8f740 1 Processor -- start 2024-07-31T15:13:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe999c8f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f1194660 0x5609f1577170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe999c8f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f119c840 0x5609f1572670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe999c8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f11bd520 0x5609f1572bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe999c8f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1b1ba0 con 0x5609f1194660 2024-07-31T15:13:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe999c8f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1b14d0 con 0x5609f119c840 2024-07-31T15:13:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe999c8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1ae730 con 0x5609f11bd520 2024-07-31T15:13:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe9914da640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f1194660 0x5609f1577170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe999c8f740 10 client.0 _ll_register_callbacks cb 0x7fe9840aafb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe9914da640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f1194660 0x5609f1577170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39636/0 (socket says 172.21.15.67:39636) 2024-07-31T15:13:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe9914da640 1 -- 172.21.15.67:0/3463341482 learned_addr learned my addr 172.21.15.67:0/3463341482 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe990cd9640 1 --2- 172.21.15.67:0/3463341482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f119c840 0x5609f1572670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe991cdb640 1 --2- 172.21.15.67:0/3463341482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f11bd520 0x5609f1572bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe9914da640 1 -- 172.21.15.67:0/3463341482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f11bd520 msgr2=0x5609f1572bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe9914da640 1 --2- 172.21.15.67:0/3463341482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f11bd520 0x5609f1572bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe9914da640 1 -- 172.21.15.67:0/3463341482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f119c840 msgr2=0x5609f1572670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe9914da640 1 --2- 172.21.15.67:0/3463341482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f119c840 0x5609f1572670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe9914da640 1 -- 172.21.15.67:0/3463341482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609f156a320 con 0x5609f1194660 2024-07-31T15:13:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe9914da640 1 --2- 172.21.15.67:0/3463341482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f1194660 0x5609f1577170 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fe988004430 tx=0x7fe98802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc5ebfe76f9d54f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe974ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:13:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe974ff9640 1 -- 172.21.15.67:0/3463341482 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988003a00 con 0x5609f1194660 2024-07-31T15:13:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.644+0000 7fe974ff9640 1 -- 172.21.15.67:0/3463341482 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7fe988003ba0 con 0x5609f1194660 2024-07-31T15:13:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.645+0000 7fe999c8f740 1 -- 172.21.15.67:0/3463341482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe988059070 con 0x5609f1194660 2024-07-31T15:13:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.645+0000 7fe974ff9640 1 -- 172.21.15.67:0/3463341482 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fe988003da0 con 0x5609f1194660 2024-07-31T15:13:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.645+0000 7fe999c8f740 10 client.20220 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.645+0000 7fe999c8f740 1 -- 172.21.15.67:0/3463341482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5609f15e8ff0 con 0x5609f1194660 2024-07-31T15:13:36.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.645+0000 7fe974ff9640 1 -- 172.21.15.67:0/3463341482 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe988059070 con 0x5609f1194660 2024-07-31T15:13:36.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.645+0000 7fe974ff9640 1 -- 172.21.15.67:0/3463341482 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fe98803d6f0 con 0x5609f1194660 2024-07-31T15:13:36.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.645+0000 7fe999c8f740 10 client.20220 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:36.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.645+0000 7fe999c8f740 10 client.20220 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:13:36.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.645+0000 7fe999c8f740 20 client.20220 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:36.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.645+0000 7fe999c8f740 1 --2- 172.21.15.67:0/3463341482 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5609f1151250 0x5609f15eb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:36.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.645+0000 7fe999c8f740 4 client.20220 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "847f6b4e", "release": true, "await": true}] 2024-07-31T15:13:36.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.645+0000 7fe999c8f740 1 -- 172.21.15.67:0/3463341482 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "set_id": "847f6b4e", "release": true, "await": true}) -- 0x5609f14d6860 con 0x5609f1151250 2024-07-31T15:13:36.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.645+0000 7fe990cd9640 1 --2- 172.21.15.67:0/3463341482 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5609f1151250 0x5609f15eb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:36.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.646+0000 7fe990cd9640 1 --2- 172.21.15.67:0/3463341482 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5609f1151250 0x5609f15eb720 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74a9282cc87663eb server_cookie=34f16c12d817b019 in_seq=0 out_seq=0 2024-07-31T15:13:36.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:36.646+0000 7fe974ff9640 10 client.20220 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:36.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:13:36.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:36 smithi073 ceph-mon[36714]: pgmap v1120: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 316 KiB/s wr, 2.94k op/s 2024-07-31T15:13:36.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.989 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.992 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:13:36.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:13:36.994 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:36 smithi077 ceph-mon[37327]: pgmap v1120: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 316 KiB/s wr, 2.94k op/s 2024-07-31T15:13:39.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.171+0000 7fe974ff9640 1 -- 172.21.15.67:0/3463341482 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5609f14d6860 con 0x5609f1151250 2024-07-31T15:13:39.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.171+0000 7fe974ff9640 10 client.20220 handle_command_reply: tid=0 2024-07-31T15:13:39.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.171+0000 7fe999c8f740 1 client.20220 shutdown 2024-07-31T15:13:39.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.171+0000 7fe999c8f740 1 -- 172.21.15.67:0/3463341482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f1194660 msgr2=0x5609f1577170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:39.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.171+0000 7fe999c8f740 1 --2- 172.21.15.67:0/3463341482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f1194660 0x5609f1577170 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fe988004430 tx=0x7fe98802a410 comp rx=0 tx=0).stop 2024-07-31T15:13:39.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.171+0000 7fe999c8f740 1 -- 172.21.15.67:0/3463341482 shutdown_connections 2024-07-31T15:13:39.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.171+0000 7fe999c8f740 1 --2- 172.21.15.67:0/3463341482 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5609f1151250 0x5609f15eb720 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:39.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.171+0000 7fe999c8f740 1 --2- 172.21.15.67:0/3463341482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5609f11bd520 0x5609f1572bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:39.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.171+0000 7fe999c8f740 1 --2- 172.21.15.67:0/3463341482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609f119c840 0x5609f1572670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:39.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.171+0000 7fe974ff9640 0 client.20220 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:39.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.171+0000 7fe999c8f740 1 --2- 172.21.15.67:0/3463341482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609f1194660 0x5609f1577170 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:39.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.172+0000 7fe999c8f740 1 -- 172.21.15.67:0/3463341482 >> 172.21.15.67:0/3463341482 conn(0x5609f11b1910 msgr2=0x5609f156fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:39.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.172+0000 7fe999c8f740 1 -- 172.21.15.67:0/3463341482 shutdown_connections 2024-07-31T15:13:39.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.172+0000 7fe999c8f740 1 -- 172.21.15.67:0/3463341482 wait complete. 2024-07-31T15:13:39.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.172+0000 7fe999c8f740 20 client.20220 trim_cache size 0 max 16384 2024-07-31T15:13:39.176 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:13:39.176 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 66, 2024-07-31T15:13:39.176 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:13:39.176 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 64, 2024-07-31T15:13:39.176 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:13:39.176 INFO:teuthology.orchestra.run.smithi067.stdout: "847f6b4e": { 2024-07-31T15:13:39.177 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 64, 2024-07-31T15:13:39.177 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:13:39.177 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:13:39.177 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:13:39.177 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:13:39.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:13:39.177 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:13:39.177 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 131.7, 2024-07-31T15:13:39.178 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:13:39.178 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:13:39.178 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:13:39.178 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:13:39.178 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-07-31T15:13:39.178 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:13:39.178 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:13:39.178 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:13:39.178 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:13:39.178 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:13:39.178 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:13:39.179 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:13:39.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.173+0000 7fe99373e640 1 -- 172.21.15.67:0/4279318343 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe9780b3d40 msgr2=0x7fe9780b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:39.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.173+0000 7fe99373e640 1 --2- 172.21.15.67:0/4279318343 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe9780b3d40 0x7fe9780b6250 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe988031440 tx=0x7fe988009560 comp rx=0 tx=0).stop 2024-07-31T15:13:39.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.173+0000 7fe99373e640 1 -- 172.21.15.67:0/4279318343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c108740 msgr2=0x7fe98c19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:39.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.173+0000 7fe99373e640 1 --2- 172.21.15.67:0/4279318343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c108740 0x7fe98c19f720 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe98400eb10 tx=0x7fe98400c2d0 comp rx=0 tx=0).stop 2024-07-31T15:13:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.173+0000 7fe99373e640 1 -- 172.21.15.67:0/4279318343 shutdown_connections 2024-07-31T15:13:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.173+0000 7fe99373e640 1 --2- 172.21.15.67:0/4279318343 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fe9780b3d40 0x7fe9780b6250 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.173+0000 7fe99373e640 1 --2- 172.21.15.67:0/4279318343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe98c108740 0x7fe98c19f720 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.173+0000 7fe99373e640 1 --2- 172.21.15.67:0/4279318343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98c107df0 0x7fe98c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.173+0000 7fe99373e640 1 --2- 172.21.15.67:0/4279318343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c1024b0 0x7fe98c199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.173+0000 7fe99373e640 1 -- 172.21.15.67:0/4279318343 >> 172.21.15.67:0/4279318343 conn(0x7fe98c079170 msgr2=0x7fe98c101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.173+0000 7fe99373e640 1 -- 172.21.15.67:0/4279318343 shutdown_connections 2024-07-31T15:13:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:39.173+0000 7fe99373e640 1 -- 172.21.15.67:0/4279318343 wait complete. 2024-07-31T15:13:39.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:39 smithi073 ceph-mon[36714]: pgmap v1121: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 215 KiB/s wr, 4.59k op/s 2024-07-31T15:13:39.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:39 smithi077 ceph-mon[37327]: pgmap v1121: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 215 KiB/s wr, 4.59k op/s 2024-07-31T15:13:39.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:39 smithi067 ceph-mon[32750]: pgmap v1121: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 215 KiB/s wr, 4.59k op/s 2024-07-31T15:13:40.496 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '847f6b4e', total seconds elapsed: 43.2 2024-07-31T15:13:41.012 INFO:tasks.workunit.client.0.smithi067.stdout: LD vmlinux.o 2024-07-31T15:13:41.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:41 smithi073 ceph-mon[36714]: pgmap v1122: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 211 KiB/s wr, 4.37k op/s 2024-07-31T15:13:41.490 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:41 smithi077 ceph-mon[37327]: pgmap v1122: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 211 KiB/s wr, 4.37k op/s 2024-07-31T15:13:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:41 smithi067 ceph-mon[32750]: pgmap v1122: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 211 KiB/s wr, 4.37k op/s 2024-07-31T15:13:43.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:43 smithi073 ceph-mon[36714]: pgmap v1123: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 211 KiB/s wr, 4.37k op/s 2024-07-31T15:13:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.444+0000 7f19678c9640 1 -- 172.21.15.67:0/2858269959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19601023f0 msgr2=0x7f1960102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.444+0000 7f19678c9640 1 --2- 172.21.15.67:0/2858269959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19601023f0 0x7f1960102800 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f195c009900 tx=0x7f195c02a410 comp rx=0 tx=0).stop 2024-07-31T15:13:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.444+0000 7f19678c9640 1 -- 172.21.15.67:0/2858269959 shutdown_connections 2024-07-31T15:13:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.444+0000 7f19678c9640 1 --2- 172.21.15.67:0/2858269959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1960108670 0x7f196010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.444+0000 7f19678c9640 1 --2- 172.21.15.67:0/2858269959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1960107d20 0x7f1960108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.444+0000 7f19678c9640 1 --2- 172.21.15.67:0/2858269959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19601023f0 0x7f1960102800 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.444+0000 7f19678c9640 1 -- 172.21.15.67:0/2858269959 >> 172.21.15.67:0/2858269959 conn(0x7f19600fddf0 msgr2=0x7f1960100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.444+0000 7f19678c9640 1 -- 172.21.15.67:0/2858269959 shutdown_connections 2024-07-31T15:13:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.445+0000 7f19678c9640 1 -- 172.21.15.67:0/2858269959 wait complete. 2024-07-31T15:13:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.445+0000 7f19678c9640 1 Processor -- start 2024-07-31T15:13:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.445+0000 7f19678c9640 1 -- start start 2024-07-31T15:13:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f19678c9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19601023f0 0x7f19600fe320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f19678c9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1960107d20 0x7f19600fe860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f19678c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1960108670 0x7f19600feda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f19678c9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1960112340 con 0x7f19601023f0 2024-07-31T15:13:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f19678c9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f19601121c0 con 0x7f1960107d20 2024-07-31T15:13:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f19678c9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19601124c0 con 0x7f1960108670 2024-07-31T15:13:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f1965665640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19601023f0 0x7f19600fe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f1964e64640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1960107d20 0x7f19600fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f1965665640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19601023f0 0x7f19600fe320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54142/0 (socket says 172.21.15.67:54142) 2024-07-31T15:13:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f1965665640 1 -- 172.21.15.67:0/2578900782 learned_addr learned my addr 172.21.15.67:0/2578900782 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f1965e66640 1 --2- 172.21.15.67:0/2578900782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1960108670 0x7f19600feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f1965665640 1 -- 172.21.15.67:0/2578900782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1960108670 msgr2=0x7f19600feda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f1965665640 1 --2- 172.21.15.67:0/2578900782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1960108670 0x7f19600feda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f1965665640 1 -- 172.21.15.67:0/2578900782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1960107d20 msgr2=0x7f19600fe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f1965665640 1 --2- 172.21.15.67:0/2578900782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1960107d20 0x7f19600fe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f1965665640 1 -- 172.21.15.67:0/2578900782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19601a8140 con 0x7f19601023f0 2024-07-31T15:13:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.446+0000 7f1965665640 1 --2- 172.21.15.67:0/2578900782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19601023f0 0x7f19600fe320 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f195c005940 tx=0x7f195c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.447+0000 7f19567fc640 1 -- 172.21.15.67:0/2578900782 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c03f030 con 0x7f19601023f0 2024-07-31T15:13:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.447+0000 7f19567fc640 1 -- 172.21.15.67:0/2578900782 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c003aa0 con 0x7f19601023f0 2024-07-31T15:13:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.447+0000 7f19567fc640 1 -- 172.21.15.67:0/2578900782 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c035070 con 0x7f19601023f0 2024-07-31T15:13:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.447+0000 7f19678c9640 1 -- 172.21.15.67:0/2578900782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19601a83d0 con 0x7f19601023f0 2024-07-31T15:13:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.447+0000 7f19678c9640 1 -- 172.21.15.67:0/2578900782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19601a88c0 con 0x7f19601023f0 2024-07-31T15:13:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.450+0000 7f19567fc640 1 -- 172.21.15.67:0/2578900782 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f195c042030 con 0x7f19601023f0 2024-07-31T15:13:43.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.450+0000 7f196e7c7740 1 Processor -- start 2024-07-31T15:13:43.452 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.450+0000 7f196e7c7740 1 -- start start 2024-07-31T15:13:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f196e7c7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafdd1b0 0x5653fb37fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f196e7c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafb42f0 0x5653fb380220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f196e7c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafbc4d0 0x5653fb380760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f196e7c7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5653fb2f53b0 con 0x5653fafdd1b0 2024-07-31T15:13:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f196e7c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5653fb2e48b0 con 0x5653fafbc4d0 2024-07-31T15:13:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f196e7c7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5653fb2e88f0 con 0x5653fafb42f0 2024-07-31T15:13:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f19567fc640 1 --2- 172.21.15.67:0/2578900782 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f194c0b3d70 0x7f194c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f1965665640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafb42f0 0x5653fb380220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f1965665640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafb42f0 0x5653fb380220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:39374/0 (socket says 172.21.15.67:39374) 2024-07-31T15:13:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f1965665640 1 -- 172.21.15.67:0/2281782347 learned_addr learned my addr 172.21.15.67:0/2281782347 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f1965e66640 1 --2- 172.21.15.67:0/2281782347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafdd1b0 0x5653fb37fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f1964e64640 1 --2- 172.21.15.67:0/2281782347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafbc4d0 0x5653fb380760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f1965e66640 1 --2- 172.21.15.67:0/2578900782 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f194c0b3d70 0x7f194c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f19567fc640 1 -- 172.21.15.67:0/2578900782 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7f195c1049c0 con 0x7f19601023f0 2024-07-31T15:13:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f1965665640 1 -- 172.21.15.67:0/2281782347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafbc4d0 msgr2=0x5653fb380760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f1965665640 1 --2- 172.21.15.67:0/2281782347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafbc4d0 0x5653fb380760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f1965665640 1 -- 172.21.15.67:0/2281782347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafdd1b0 msgr2=0x5653fb37fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f1965665640 1 --2- 172.21.15.67:0/2281782347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafdd1b0 0x5653fb37fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f1965665640 1 -- 172.21.15.67:0/2281782347 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653fb28b9a0 con 0x5653fafb42f0 2024-07-31T15:13:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.451+0000 7f1965e66640 1 --2- 172.21.15.67:0/2281782347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafdd1b0 0x5653fb37fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.452+0000 7f1965665640 1 --2- 172.21.15.67:0/2281782347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafb42f0 0x5653fb380220 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f195c004770 tx=0x7f195c0c7300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3fc95e167b8cf8f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.452+0000 7f1965e66640 1 --2- 172.21.15.67:0/2578900782 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f194c0b3d70 0x7f194c0b6280 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1958005c40 tx=0x7f1958009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.452+0000 7f194b7fe640 1 -- 172.21.15.67:0/2281782347 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c049070 con 0x5653fafb42f0 2024-07-31T15:13:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.452+0000 7f194b7fe640 1 -- 172.21.15.67:0/2281782347 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c0cae20 con 0x5653fafb42f0 2024-07-31T15:13:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.452+0000 7f194b7fe640 1 -- 172.21.15.67:0/2281782347 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c03bc80 con 0x5653fafb42f0 2024-07-31T15:13:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.453+0000 7f196e7c7740 1 -- 172.21.15.67:0/2281782347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafb42f0 msgr2=0x5653fb380220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.453+0000 7f196e7c7740 1 --2- 172.21.15.67:0/2281782347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafb42f0 0x5653fb380220 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f195c004770 tx=0x7f195c0c7300 comp rx=0 tx=0).stop 2024-07-31T15:13:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.453+0000 7f196e7c7740 1 -- 172.21.15.67:0/2281782347 shutdown_connections 2024-07-31T15:13:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.453+0000 7f196e7c7740 1 --2- 172.21.15.67:0/2281782347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafbc4d0 0x5653fb380760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.453+0000 7f196e7c7740 1 --2- 172.21.15.67:0/2281782347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafb42f0 0x5653fb380220 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.453+0000 7f196e7c7740 1 --2- 172.21.15.67:0/2281782347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafdd1b0 0x5653fb37fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.453+0000 7f196e7c7740 1 -- 172.21.15.67:0/2281782347 >> 172.21.15.67:0/2281782347 conn(0x5653fafd1300 msgr2=0x5653fb37dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.453+0000 7f196e7c7740 1 -- 172.21.15.67:0/2281782347 shutdown_connections 2024-07-31T15:13:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.453+0000 7f196e7c7740 1 -- 172.21.15.67:0/2281782347 wait complete. 2024-07-31T15:13:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.453+0000 7f196e7c7740 1 -- start start 2024-07-31T15:13:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f196e7c7740 1 Processor -- start 2024-07-31T15:13:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f196e7c7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb38b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f196e7c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb38bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f196e7c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 0x5653fb397480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f196e7c7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5653fb21eab0 con 0x5653fafb42f0 2024-07-31T15:13:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f196e7c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5653fb26ccf0 con 0x5653fafdd1b0 2024-07-31T15:13:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f196e7c7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5653fb38e220 con 0x5653fafbc4d0 2024-07-31T15:13:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f196e7c7740 10 client.0 _ll_register_callbacks cb 0x5653fb2de9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f1965665640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb38b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f1964e64640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb38bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f1964e64640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb38bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:39386/0 (socket says 172.21.15.67:39386) 2024-07-31T15:13:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f1965665640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb38b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54166/0 (socket says 172.21.15.67:54166) 2024-07-31T15:13:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f1964e64640 1 -- 172.21.15.67:0/408971665 learned_addr learned my addr 172.21.15.67:0/408971665 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f1965e66640 1 --2- 172.21.15.67:0/408971665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 0x5653fb397480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f1964e64640 1 -- 172.21.15.67:0/408971665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 msgr2=0x5653fb397480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f1964e64640 1 --2- 172.21.15.67:0/408971665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 0x5653fb397480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f1964e64640 1 -- 172.21.15.67:0/408971665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 msgr2=0x5653fb38b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.454+0000 7f1964e64640 1 --2- 172.21.15.67:0/408971665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb38b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.455+0000 7f1964e64640 1 -- 172.21.15.67:0/408971665 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653fb048790 con 0x5653fafbc4d0 2024-07-31T15:13:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.455+0000 7f1965665640 1 --2- 172.21.15.67:0/408971665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb38b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.455+0000 7f1964e64640 1 --2- 172.21.15.67:0/408971665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb38bd00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f195001bb10 tx=0x7f195001bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f715ad2583ebcb9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.455+0000 7f1948ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:13:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.455+0000 7f1948ff9640 1 -- 172.21.15.67:0/408971665 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950028060 con 0x5653fafbc4d0 2024-07-31T15:13:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.455+0000 7f1948ff9640 1 -- 172.21.15.67:0/408971665 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7f1950003a00 con 0x5653fafbc4d0 2024-07-31T15:13:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.455+0000 7f196e7c7740 1 -- 172.21.15.67:0/408971665 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f195c0cae20 con 0x5653fafbc4d0 2024-07-31T15:13:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.455+0000 7f1948ff9640 1 -- 172.21.15.67:0/408971665 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f195002f050 con 0x5653fafbc4d0 2024-07-31T15:13:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.455+0000 7f196e7c7740 10 client.29593 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.455+0000 7f196e7c7740 1 -- 172.21.15.67:0/408971665 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5653fb0c1620 con 0x5653fafbc4d0 2024-07-31T15:13:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.456+0000 7f1948ff9640 1 -- 172.21.15.67:0/408971665 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c0cae20 con 0x5653fafbc4d0 2024-07-31T15:13:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.456+0000 7f1948ff9640 1 -- 172.21.15.67:0/408971665 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f1950022030 con 0x5653fafbc4d0 2024-07-31T15:13:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.456+0000 7f196e7c7740 10 client.29593 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.456+0000 7f196e7c7740 10 client.29593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.456+0000 7f196e7c7740 20 client.29593 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.456+0000 7f196e7c7740 1 --2- 172.21.15.67:0/408971665 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653faf9bd80 0x5653fb39d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.456+0000 7f196e7c7740 4 client.29593 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:13:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.456+0000 7f196e7c7740 1 -- 172.21.15.67:0/408971665 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653fb2f7300 con 0x5653faf9bd80 2024-07-31T15:13:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.456+0000 7f1965665640 1 --2- 172.21.15.67:0/408971665 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653faf9bd80 0x5653fb39d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.457+0000 7f1948ff9640 10 client.29593 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.457+0000 7f1965665640 1 --2- 172.21.15.67:0/408971665 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653faf9bd80 0x5653fb39d530 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b48b74d78d1f221 server_cookie=ff8f16a217f58ac0 in_seq=0 out_seq=0 2024-07-31T15:13:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.457+0000 7f1948ff9640 1 -- 172.21.15.67:0/408971665 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653fb2f7300 con 0x5653faf9bd80 2024-07-31T15:13:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.457+0000 7f1948ff9640 10 client.29593 handle_command_reply: tid=0 2024-07-31T15:13:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.457+0000 7f196e7c7740 1 client.29593 shutdown 2024-07-31T15:13:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.458+0000 7f196e7c7740 1 -- 172.21.15.67:0/408971665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 msgr2=0x5653fb38bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.458+0000 7f196e7c7740 1 --2- 172.21.15.67:0/408971665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb38bd00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f195001bb10 tx=0x7f195001bf10 comp rx=0 tx=0).stop 2024-07-31T15:13:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.458+0000 7f196e7c7740 1 -- 172.21.15.67:0/408971665 shutdown_connections 2024-07-31T15:13:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.458+0000 7f196e7c7740 1 --2- 172.21.15.67:0/408971665 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653faf9bd80 0x5653fb39d530 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.458+0000 7f196e7c7740 1 --2- 172.21.15.67:0/408971665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 0x5653fb397480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.458+0000 7f196e7c7740 1 --2- 172.21.15.67:0/408971665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb38bd00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.458+0000 7f196e7c7740 1 --2- 172.21.15.67:0/408971665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb38b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.458+0000 7f1948ff9640 0 client.29593 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.458+0000 7f196e7c7740 1 -- 172.21.15.67:0/408971665 >> 172.21.15.67:0/408971665 conn(0x5653fafd1300 msgr2=0x5653fb380d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.458+0000 7f196e7c7740 1 -- 172.21.15.67:0/408971665 shutdown_connections 2024-07-31T15:13:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.458+0000 7f196e7c7740 1 -- 172.21.15.67:0/408971665 wait complete. 2024-07-31T15:13:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.458+0000 7f196e7c7740 20 client.29593 trim_cache size 0 max 16384 2024-07-31T15:13:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.475+0000 7f196e7c7740 1 Processor -- start 2024-07-31T15:13:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f196e7c7740 1 -- start start 2024-07-31T15:13:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f196e7c7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb385db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f196e7c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb38d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f196e7c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 0x5653fb397270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f196e7c7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f19601ae580 con 0x5653fafb42f0 2024-07-31T15:13:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f196e7c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f19601b00c0 con 0x5653fafdd1b0 2024-07-31T15:13:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f196e7c7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19601aec50 con 0x5653fafbc4d0 2024-07-31T15:13:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f1964e64640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb38d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f1965665640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb385db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f1964e64640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb38d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:39402/0 (socket says 172.21.15.67:39402) 2024-07-31T15:13:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f1964e64640 1 -- 172.21.15.67:0/4033178318 learned_addr learned my addr 172.21.15.67:0/4033178318 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f1965665640 1 --2- 172.21.15.67:0/4033178318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb385db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54170/0 (socket says 172.21.15.67:54170) 2024-07-31T15:13:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f1965e66640 1 --2- 172.21.15.67:0/4033178318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 0x5653fb397270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f1964e64640 1 -- 172.21.15.67:0/4033178318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 msgr2=0x5653fb397270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f1964e64640 1 --2- 172.21.15.67:0/4033178318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 0x5653fb397270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f1964e64640 1 -- 172.21.15.67:0/4033178318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 msgr2=0x5653fb385db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f1964e64640 1 --2- 172.21.15.67:0/4033178318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb385db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.476+0000 7f1964e64640 1 -- 172.21.15.67:0/4033178318 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653fb386e00 con 0x5653fafbc4d0 2024-07-31T15:13:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.477+0000 7f1964e64640 1 --2- 172.21.15.67:0/4033178318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb38d1e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1950004340 tx=0x7f1950005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8c041d3a9460284 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.477+0000 7f194affd640 1 -- 172.21.15.67:0/4033178318 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950037020 con 0x5653fafbc4d0 2024-07-31T15:13:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.477+0000 7f194affd640 1 -- 172.21.15.67:0/4033178318 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1950011440 con 0x5653fafbc4d0 2024-07-31T15:13:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.477+0000 7f194affd640 1 -- 172.21.15.67:0/4033178318 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19500109f0 con 0x5653fafbc4d0 2024-07-31T15:13:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.478+0000 7f196e7c7740 1 -- 172.21.15.67:0/4033178318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 msgr2=0x5653fb38d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.478+0000 7f196e7c7740 1 --2- 172.21.15.67:0/4033178318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb38d1e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1950004340 tx=0x7f1950005850 comp rx=0 tx=0).stop 2024-07-31T15:13:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.478+0000 7f196e7c7740 1 -- 172.21.15.67:0/4033178318 shutdown_connections 2024-07-31T15:13:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.478+0000 7f196e7c7740 1 --2- 172.21.15.67:0/4033178318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 0x5653fb397270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.478+0000 7f196e7c7740 1 --2- 172.21.15.67:0/4033178318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb38d1e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.478+0000 7f196e7c7740 1 --2- 172.21.15.67:0/4033178318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb385db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.478+0000 7f196e7c7740 1 -- 172.21.15.67:0/4033178318 >> 172.21.15.67:0/4033178318 conn(0x5653fafd1300 msgr2=0x5653fb38cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.478+0000 7f196e7c7740 1 -- 172.21.15.67:0/4033178318 shutdown_connections 2024-07-31T15:13:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.478+0000 7f196e7c7740 1 -- 172.21.15.67:0/4033178318 wait complete. 2024-07-31T15:13:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.478+0000 7f196e7c7740 1 -- start start 2024-07-31T15:13:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.478+0000 7f196e7c7740 1 Processor -- start 2024-07-31T15:13:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f196e7c7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb38fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f196e7c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb3902e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f196e7c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 0x5653fb3fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f196e7c7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f19601af9f0 con 0x5653fafb42f0 2024-07-31T15:13:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f196e7c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f19601af320 con 0x5653fafdd1b0 2024-07-31T15:13:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f196e7c7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19601ac5e0 con 0x5653fafbc4d0 2024-07-31T15:13:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f196e7c7740 10 client.0 _ll_register_callbacks cb 0x7f1950004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f1965665640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb38fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f1964e64640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb3902e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f1965665640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb38fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54180/0 (socket says 172.21.15.67:54180) 2024-07-31T15:13:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f1964e64640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb3902e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:39410/0 (socket says 172.21.15.67:39410) 2024-07-31T15:13:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f1965e66640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 0x5653fb3fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f1965665640 1 -- 172.21.15.67:0/3741235783 learned_addr learned my addr 172.21.15.67:0/3741235783 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f1964e64640 1 -- 172.21.15.67:0/3741235783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 msgr2=0x5653fb3fcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f1964e64640 1 --2- 172.21.15.67:0/3741235783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 0x5653fb3fcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f1964e64640 1 -- 172.21.15.67:0/3741235783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 msgr2=0x5653fb38fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f1964e64640 1 --2- 172.21.15.67:0/3741235783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb38fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f1964e64640 1 -- 172.21.15.67:0/3741235783 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653fb3fd3b0 con 0x5653fafbc4d0 2024-07-31T15:13:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.479+0000 7f1965665640 1 --2- 172.21.15.67:0/3741235783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb38fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f1964e64640 1 --2- 172.21.15.67:0/3741235783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb3902e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1950010d70 tx=0x7f19500115f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8852c0c1c09113cf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f1948ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:13:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f1948ff9640 1 -- 172.21.15.67:0/3741235783 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950028040 con 0x5653fafbc4d0 2024-07-31T15:13:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f1948ff9640 1 -- 172.21.15.67:0/3741235783 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7f1950003800 con 0x5653fafbc4d0 2024-07-31T15:13:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f196e7c7740 1 -- 172.21.15.67:0/3741235783 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1950011440 con 0x5653fafbc4d0 2024-07-31T15:13:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f1948ff9640 1 -- 172.21.15.67:0/3741235783 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f195004c050 con 0x5653fafbc4d0 2024-07-31T15:13:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f196e7c7740 10 client.29599 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f196e7c7740 1 -- 172.21.15.67:0/3741235783 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5653fb4047a0 con 0x5653fafbc4d0 2024-07-31T15:13:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f1948ff9640 1 -- 172.21.15.67:0/3741235783 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1950011440 con 0x5653fafbc4d0 2024-07-31T15:13:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f1948ff9640 1 -- 172.21.15.67:0/3741235783 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f1950022040 con 0x5653fafbc4d0 2024-07-31T15:13:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f196e7c7740 10 client.29599 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f196e7c7740 10 client.29599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f196e7c7740 20 client.29599 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f196e7c7740 1 --2- 172.21.15.67:0/3741235783 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653faf711c0 0x5653fb406f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f196e7c7740 4 client.29599 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:13:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.480+0000 7f196e7c7740 1 -- 172.21.15.67:0/3741235783 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653fb2f2da0 con 0x5653faf711c0 2024-07-31T15:13:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.481+0000 7f1965665640 1 --2- 172.21.15.67:0/3741235783 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653faf711c0 0x5653fb406f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.481+0000 7f1948ff9640 10 client.29599 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.481+0000 7f1965665640 1 --2- 172.21.15.67:0/3741235783 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653faf711c0 0x5653fb406f10 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=130afa6e9f5344a0 server_cookie=76b51d0ffe3b58a2 in_seq=0 out_seq=0 2024-07-31T15:13:43.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:43 smithi077 ceph-mon[37327]: pgmap v1123: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 211 KiB/s wr, 4.37k op/s 2024-07-31T15:13:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f1948ff9640 1 -- 172.21.15.67:0/3741235783 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653fb2f2da0 con 0x5653faf711c0 2024-07-31T15:13:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f1948ff9640 10 client.29599 handle_command_reply: tid=0 2024-07-31T15:13:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f196e7c7740 1 client.29599 shutdown 2024-07-31T15:13:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f196e7c7740 1 -- 172.21.15.67:0/3741235783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 msgr2=0x5653fb3902e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f196e7c7740 1 --2- 172.21.15.67:0/3741235783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb3902e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1950010d70 tx=0x7f19500115f0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f196e7c7740 1 -- 172.21.15.67:0/3741235783 shutdown_connections 2024-07-31T15:13:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f196e7c7740 1 --2- 172.21.15.67:0/3741235783 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5653faf711c0 0x5653fb406f10 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f196e7c7740 1 --2- 172.21.15.67:0/3741235783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653fafdd1b0 0x5653fb3fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f196e7c7740 1 --2- 172.21.15.67:0/3741235783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5653fafbc4d0 0x5653fb3902e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f196e7c7740 1 --2- 172.21.15.67:0/3741235783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653fafb42f0 0x5653fb38fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f1948ff9640 0 client.29599 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f196e7c7740 1 -- 172.21.15.67:0/3741235783 >> 172.21.15.67:0/3741235783 conn(0x5653fafd1300 msgr2=0x5653fb384da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f196e7c7740 1 -- 172.21.15.67:0/3741235783 shutdown_connections 2024-07-31T15:13:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f196e7c7740 1 -- 172.21.15.67:0/3741235783 wait complete. 2024-07-31T15:13:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.530+0000 7f196e7c7740 20 client.29599 trim_cache size 0 max 16384 2024-07-31T15:13:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:13:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:13:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:13:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:13:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.531+0000 7f19678c9640 1 -- 172.21.15.67:0/2578900782 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f194c0b3d70 msgr2=0x7f194c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.531+0000 7f19678c9640 1 --2- 172.21.15.67:0/2578900782 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f194c0b3d70 0x7f194c0b6280 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1958005c40 tx=0x7f1958009500 comp rx=0 tx=0).stop 2024-07-31T15:13:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.531+0000 7f19678c9640 1 -- 172.21.15.67:0/2578900782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19601023f0 msgr2=0x7f19600fe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.531+0000 7f19678c9640 1 --2- 172.21.15.67:0/2578900782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19601023f0 0x7f19600fe320 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f195c005940 tx=0x7f195c004270 comp rx=0 tx=0).stop 2024-07-31T15:13:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.531+0000 7f19678c9640 1 -- 172.21.15.67:0/2578900782 shutdown_connections 2024-07-31T15:13:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.531+0000 7f19678c9640 1 --2- 172.21.15.67:0/2578900782 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f194c0b3d70 0x7f194c0b6280 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.531+0000 7f19678c9640 1 --2- 172.21.15.67:0/2578900782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1960108670 0x7f19600feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.531+0000 7f19678c9640 1 --2- 172.21.15.67:0/2578900782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1960107d20 0x7f19600fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.531+0000 7f19678c9640 1 --2- 172.21.15.67:0/2578900782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19601023f0 0x7f19600fe320 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.531+0000 7f19678c9640 1 -- 172.21.15.67:0/2578900782 >> 172.21.15.67:0/2578900782 conn(0x7f19600fddf0 msgr2=0x7f196006ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.531+0000 7f19678c9640 1 -- 172.21.15.67:0/2578900782 shutdown_connections 2024-07-31T15:13:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:43.531+0000 7f19678c9640 1 -- 172.21.15.67:0/2578900782 wait complete. 2024-07-31T15:13:43.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:43 smithi067 ceph-mon[32750]: pgmap v1123: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 211 KiB/s wr, 4.37k op/s 2024-07-31T15:13:45.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:45 smithi073 ceph-mon[36714]: pgmap v1124: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 194 KiB/s wr, 3.92k op/s 2024-07-31T15:13:45.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:45 smithi077 ceph-mon[37327]: pgmap v1124: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 194 KiB/s wr, 3.92k op/s 2024-07-31T15:13:45.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:45 smithi067 ceph-mon[32750]: pgmap v1124: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 194 KiB/s wr, 3.92k op/s 2024-07-31T15:13:47.193 INFO:tasks.workunit.client.0.smithi067.stdout: OBJCOPY modules.builtin.modinfo 2024-07-31T15:13:47.254 INFO:tasks.workunit.client.0.smithi067.stdout: GEN modules.builtin 2024-07-31T15:13:47.271 INFO:tasks.workunit.client.0.smithi067.stdout: MODPOST Module.symvers 2024-07-31T15:13:47.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:47 smithi073 ceph-mon[36714]: pgmap v1125: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.0 MiB/s wr, 2.72k op/s 2024-07-31T15:13:47.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:47 smithi077 ceph-mon[37327]: pgmap v1125: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.0 MiB/s wr, 2.72k op/s 2024-07-31T15:13:47.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:47 smithi067 ceph-mon[32750]: pgmap v1125: 129 pgs: 129 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.0 MiB/s wr, 2.72k op/s 2024-07-31T15:13:49.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:49 smithi073 ceph-mon[36714]: pgmap v1126: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.4 MiB/s wr, 2.15k op/s 2024-07-31T15:13:49.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:49 smithi077 ceph-mon[37327]: pgmap v1126: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.4 MiB/s wr, 2.15k op/s 2024-07-31T15:13:49.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:49 smithi067 ceph-mon[32750]: pgmap v1126: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.4 MiB/s wr, 2.15k op/s 2024-07-31T15:13:50.819 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:13:50.974 INFO:tasks.workunit.client.0.smithi067.stdout: CC .vmlinux.export.o 2024-07-31T15:13:51.009 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] fs/efivarfs/efivarfs.mod.o 2024-07-31T15:13:51.009 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] drivers/thermal/intel/x86_pkg_temp_thermal.mod.o 2024-07-31T15:13:51.011 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] net/netfilter/nf_log_syslog.mod.o 2024-07-31T15:13:51.011 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] net/netfilter/xt_mark.mod.o 2024-07-31T15:13:51.012 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] net/netfilter/xt_nat.mod.o 2024-07-31T15:13:51.013 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] net/netfilter/xt_LOG.mod.o 2024-07-31T15:13:51.015 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] net/netfilter/xt_MASQUERADE.mod.o 2024-07-31T15:13:51.119 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:13:51.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:51 smithi073 ceph-mon[36714]: pgmap v1127: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 423 op/s 2024-07-31T15:13:51.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:51 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:13:51.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:51 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:13:51.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:51 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:13:51.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:51 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:13:51.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:51 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:13:51.433 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:51 smithi067 ceph-mon[32750]: pgmap v1127: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 423 op/s 2024-07-31T15:13:51.434 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:51 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:13:51.434 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:51 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:13:51.434 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:51 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:13:51.434 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:51 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:13:51.434 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:51 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:13:51.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:51 smithi077 ceph-mon[37327]: pgmap v1127: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 423 op/s 2024-07-31T15:13:51.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:51 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:13:51.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:51 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:13:51.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:51 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:13:51.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:51 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:13:51.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:51 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:13:52.176 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] net/netfilter/xt_addrtype.mod.o 2024-07-31T15:13:52.176 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] net/ipv4/netfilter/iptable_nat.mod.o 2024-07-31T15:13:52.179 INFO:tasks.workunit.client.0.smithi067.stdout: LD [M] fs/efivarfs/efivarfs.ko 2024-07-31T15:13:52.180 INFO:tasks.workunit.client.0.smithi067.stdout: LD [M] net/netfilter/xt_LOG.ko 2024-07-31T15:13:52.182 INFO:tasks.workunit.client.0.smithi067.stdout: LD [M] net/netfilter/xt_MASQUERADE.ko 2024-07-31T15:13:52.182 INFO:tasks.workunit.client.0.smithi067.stdout: LD [M] drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-07-31T15:13:52.184 INFO:tasks.workunit.client.0.smithi067.stdout: LD [M] net/netfilter/xt_mark.ko 2024-07-31T15:13:52.215 INFO:tasks.workunit.client.0.smithi067.stdout: LD [M] net/netfilter/xt_nat.ko 2024-07-31T15:13:52.216 INFO:tasks.workunit.client.0.smithi067.stdout: LD [M] net/netfilter/nf_log_syslog.ko 2024-07-31T15:13:52.254 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/generated/utsversion.h 2024-07-31T15:13:52.278 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/version-timestamp.o 2024-07-31T15:13:52.388 INFO:tasks.workunit.client.0.smithi067.stdout: LD [M] net/netfilter/xt_addrtype.ko 2024-07-31T15:13:52.396 INFO:tasks.workunit.client.0.smithi067.stdout: LD [M] net/ipv4/netfilter/iptable_nat.ko 2024-07-31T15:13:52.429 INFO:tasks.workunit.client.0.smithi067.stdout: LD .tmp_vmlinux.kallsyms1 2024-07-31T15:13:53.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.127+0000 7fadb0e5f640 1 -- 172.21.15.67:0/1578176741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadac102490 msgr2=0x7fadac1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.127+0000 7fadb0e5f640 1 --2- 172.21.15.67:0/1578176741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadac102490 0x7fadac1028a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009900 tx=0x7fad9c02a410 comp rx=0 tx=0).stop 2024-07-31T15:13:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.127+0000 7fadb0e5f640 1 -- 172.21.15.67:0/1578176741 shutdown_connections 2024-07-31T15:13:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.127+0000 7fadb0e5f640 1 --2- 172.21.15.67:0/1578176741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadac108720 0x7fadac10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.127+0000 7fadb0e5f640 1 --2- 172.21.15.67:0/1578176741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadac107dd0 0x7fadac1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.127+0000 7fadb0e5f640 1 --2- 172.21.15.67:0/1578176741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadac102490 0x7fadac1028a0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.127+0000 7fadb0e5f640 1 -- 172.21.15.67:0/1578176741 >> 172.21.15.67:0/1578176741 conn(0x7fadac079160 msgr2=0x7fadac079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.127+0000 7fadb0e5f640 1 -- 172.21.15.67:0/1578176741 shutdown_connections 2024-07-31T15:13:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.127+0000 7fadb0e5f640 1 -- 172.21.15.67:0/1578176741 wait complete. 2024-07-31T15:13:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.128+0000 7fadb0e5f640 1 Processor -- start 2024-07-31T15:13:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.128+0000 7fadb0e5f640 1 -- start start 2024-07-31T15:13:53.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.128+0000 7fadb0e5f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadac102490 0x7fadac19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.128+0000 7fadb0e5f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadac107dd0 0x7fadac19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.128+0000 7fadb0e5f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadac108720 0x7fadac19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.128+0000 7fadb0e5f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1123a0 con 0x7fadac107dd0 2024-07-31T15:13:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.128+0000 7fadb0e5f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fadac112220 con 0x7fadac102490 2024-07-31T15:13:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.128+0000 7fadb0e5f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fadac112520 con 0x7fadac108720 2024-07-31T15:13:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.128+0000 7fadaa59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadac102490 0x7fadac19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.128+0000 7fada9d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadac107dd0 0x7fadac19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.128+0000 7fadaa59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadac102490 0x7fadac19ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:38886/0 (socket says 172.21.15.67:38886) 2024-07-31T15:13:53.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.128+0000 7fada9d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadac107dd0 0x7fadac19b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46764/0 (socket says 172.21.15.67:46764) 2024-07-31T15:13:53.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.129+0000 7fadaa59c640 1 -- 172.21.15.67:0/2905119881 learned_addr learned my addr 172.21.15.67:0/2905119881 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:53.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.129+0000 7fadaad9d640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadac108720 0x7fadac19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.129+0000 7fadaa59c640 1 -- 172.21.15.67:0/2905119881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadac108720 msgr2=0x7fadac19b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:53.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.129+0000 7fadaa59c640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadac108720 0x7fadac19b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.129+0000 7fadaa59c640 1 -- 172.21.15.67:0/2905119881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadac107dd0 msgr2=0x7fadac19b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:53.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.129+0000 7fadaa59c640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadac107dd0 0x7fadac19b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.129+0000 7fadaa59c640 1 -- 172.21.15.67:0/2905119881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac1a6020 con 0x7fadac102490 2024-07-31T15:13:53.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.129+0000 7fadaad9d640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadac108720 0x7fadac19b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:53.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.129+0000 7fada9d9b640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadac107dd0 0x7fadac19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:53.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.129+0000 7fadaa59c640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadac102490 0x7fadac19ad90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fad9c005940 tx=0x7fad9c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:53.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.129+0000 7fad9b7fe640 1 -- 172.21.15.67:0/2905119881 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c03f030 con 0x7fadac102490 2024-07-31T15:13:53.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.130+0000 7fadb0e5f640 1 -- 172.21.15.67:0/2905119881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac1a62b0 con 0x7fadac102490 2024-07-31T15:13:53.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.130+0000 7fad9b7fe640 1 -- 172.21.15.67:0/2905119881 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad9c0044c0 con 0x7fadac102490 2024-07-31T15:13:53.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.130+0000 7fad9b7fe640 1 -- 172.21.15.67:0/2905119881 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c035070 con 0x7fadac102490 2024-07-31T15:13:53.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.130+0000 7fadb0e5f640 1 -- 172.21.15.67:0/2905119881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac1a67e0 con 0x7fadac102490 2024-07-31T15:13:53.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.132+0000 7fad9b7fe640 1 -- 172.21.15.67:0/2905119881 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fad9c042030 con 0x7fadac102490 2024-07-31T15:13:53.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.133+0000 7fad9b7fe640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fad940b3df0 0x7fad940b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.133+0000 7fada9d9b640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fad940b3df0 0x7fad940b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.133+0000 7fad9b7fe640 1 -- 172.21.15.67:0/2905119881 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7fad9c104620 con 0x7fadac102490 2024-07-31T15:13:53.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.133+0000 7fada9d9b640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fad940b3df0 0x7fad940b6300 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fadac1005a0 tx=0x7fada0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:53.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.134+0000 7fadb73b0740 1 Processor -- start 2024-07-31T15:13:53.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.134+0000 7fadb73b0740 1 -- start start 2024-07-31T15:13:53.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.134+0000 7fadb73b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd60070 0x5586ad105550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.134+0000 7fadb73b0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd371b0 0x5586ad105a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.134+0000 7fadb73b0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad105fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.134+0000 7fadb73b0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5586ad07bb90 con 0x5586acd3f390 2024-07-31T15:13:53.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.134+0000 7fadb73b0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5586ad06b090 con 0x5586acd371b0 2024-07-31T15:13:53.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.134+0000 7fadb73b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5586ad06f0d0 con 0x5586acd60070 2024-07-31T15:13:53.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.134+0000 7fadaa59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd60070 0x5586ad105550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.134+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad105fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.134+0000 7fadaa59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd60070 0x5586ad105550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:45822/0 (socket says 172.21.15.67:45822) 2024-07-31T15:13:53.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.134+0000 7fadaa59c640 1 -- 172.21.15.67:0/3059760119 learned_addr learned my addr 172.21.15.67:0/3059760119 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:53.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.134+0000 7fada9d9b640 1 --2- 172.21.15.67:0/3059760119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd371b0 0x5586ad105a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.135+0000 7fadaa59c640 1 -- 172.21.15.67:0/3059760119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd371b0 msgr2=0x5586ad105a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:53.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.135+0000 7fadaa59c640 1 --2- 172.21.15.67:0/3059760119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd371b0 0x5586ad105a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.135+0000 7fadaa59c640 1 -- 172.21.15.67:0/3059760119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 msgr2=0x5586ad105fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:53.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.135+0000 7fadaa59c640 1 --2- 172.21.15.67:0/3059760119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad105fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.135+0000 7fadaa59c640 1 -- 172.21.15.67:0/3059760119 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ad011b70 con 0x5586acd60070 2024-07-31T15:13:53.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.135+0000 7fada9d9b640 1 --2- 172.21.15.67:0/3059760119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd371b0 0x5586ad105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:13:53.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.135+0000 7fadaad9d640 1 --2- 172.21.15.67:0/3059760119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:53.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.135+0000 7fadaa59c640 1 --2- 172.21.15.67:0/3059760119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd60070 0x5586ad105550 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0cabc0 tx=0x7fad9c0caa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce7f8b6e753b3dbf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:53.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.135+0000 7fad98ff9640 1 -- 172.21.15.67:0/3059760119 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c049040 con 0x5586acd60070 2024-07-31T15:13:53.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.135+0000 7fad98ff9640 1 -- 172.21.15.67:0/3059760119 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad9c0cf4c0 con 0x5586acd60070 2024-07-31T15:13:53.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.135+0000 7fad98ff9640 1 -- 172.21.15.67:0/3059760119 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0d3ae0 con 0x5586acd60070 2024-07-31T15:13:53.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.136+0000 7fadb73b0740 1 -- 172.21.15.67:0/3059760119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd60070 msgr2=0x5586ad105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:53.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.136+0000 7fadb73b0740 1 --2- 172.21.15.67:0/3059760119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd60070 0x5586ad105550 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0cabc0 tx=0x7fad9c0caa90 comp rx=0 tx=0).stop 2024-07-31T15:13:53.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.136+0000 7fadb73b0740 1 -- 172.21.15.67:0/3059760119 shutdown_connections 2024-07-31T15:13:53.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.136+0000 7fadb73b0740 1 --2- 172.21.15.67:0/3059760119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.136+0000 7fadb73b0740 1 --2- 172.21.15.67:0/3059760119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd371b0 0x5586ad105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.136+0000 7fadb73b0740 1 --2- 172.21.15.67:0/3059760119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd60070 0x5586ad105550 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.136+0000 7fadb73b0740 1 -- 172.21.15.67:0/3059760119 >> 172.21.15.67:0/3059760119 conn(0x5586acd54460 msgr2=0x5586ad103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:53.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.136+0000 7fadb73b0740 1 -- 172.21.15.67:0/3059760119 shutdown_connections 2024-07-31T15:13:53.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.136+0000 7fadb73b0740 1 -- 172.21.15.67:0/3059760119 wait complete. 2024-07-31T15:13:53.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.137+0000 7fadb73b0740 1 -- start start 2024-07-31T15:13:53.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.137+0000 7fadb73b0740 1 Processor -- start 2024-07-31T15:13:53.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.137+0000 7fadb73b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd371b0 0x5586ad1110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.137+0000 7fadb73b0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad111630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fadb73b0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd60070 0x5586ad11cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fadb73b0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5586acece700 con 0x5586acd3f390 2024-07-31T15:13:53.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fadb73b0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5586ad00d9c0 con 0x5586acd60070 2024-07-31T15:13:53.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fadb73b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5586ad113af0 con 0x5586acd371b0 2024-07-31T15:13:53.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fada9d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad111630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fada9d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad111630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46786/0 (socket says 172.21.15.67:46786) 2024-07-31T15:13:53.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fada9d9b640 1 -- 172.21.15.67:0/2625144948 learned_addr learned my addr 172.21.15.67:0/2625144948 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:53.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fadaa59c640 1 --2- 172.21.15.67:0/2625144948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd371b0 0x5586ad1110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fadaad9d640 1 --2- 172.21.15.67:0/2625144948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd60070 0x5586ad11cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fadb73b0740 10 client.0 _ll_register_callbacks cb 0x5586ad0284d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:53.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fadaa59c640 1 -- 172.21.15.67:0/2625144948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd60070 msgr2=0x5586ad11cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:53.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fadaa59c640 1 --2- 172.21.15.67:0/2625144948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd60070 0x5586ad11cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fadaa59c640 1 -- 172.21.15.67:0/2625144948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 msgr2=0x5586ad111630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fadaa59c640 1 --2- 172.21.15.67:0/2625144948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad111630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fadaa59c640 1 -- 172.21.15.67:0/2625144948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586acdcb650 con 0x5586acd371b0 2024-07-31T15:13:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fada9d9b640 1 --2- 172.21.15.67:0/2625144948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fadaa59c640 1 --2- 172.21.15.67:0/2625144948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd371b0 0x5586ad1110f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0096c0 tx=0x7fad9c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=445142a9d49b4e2b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fad91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:13:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.138+0000 7fad91ffb640 1 -- 172.21.15.67:0/2625144948 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0e9020 con 0x5586acd371b0 2024-07-31T15:13:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.139+0000 7fad91ffb640 1 -- 172.21.15.67:0/2625144948 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7fad9c0d3ae0 con 0x5586acd371b0 2024-07-31T15:13:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.139+0000 7fadb73b0740 1 -- 172.21.15.67:0/2625144948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad9c0cf4c0 con 0x5586acd371b0 2024-07-31T15:13:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.139+0000 7fad91ffb640 1 -- 172.21.15.67:0/2625144948 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fad9c0e4070 con 0x5586acd371b0 2024-07-31T15:13:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.139+0000 7fadb73b0740 10 client.29611 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.139+0000 7fadb73b0740 1 -- 172.21.15.67:0/2625144948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5586ad11fa30 con 0x5586acd371b0 2024-07-31T15:13:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.139+0000 7fad91ffb640 1 -- 172.21.15.67:0/2625144948 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad9c0cf4c0 con 0x5586acd371b0 2024-07-31T15:13:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.139+0000 7fad91ffb640 1 -- 172.21.15.67:0/2625144948 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fad9c0cf020 con 0x5586acd371b0 2024-07-31T15:13:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.139+0000 7fadb73b0740 10 client.29611 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.139+0000 7fadb73b0740 10 client.29611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.139+0000 7fadb73b0740 20 client.29611 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.139+0000 7fadb73b0740 1 --2- 172.21.15.67:0/2625144948 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5586acd1e9f0 0x5586ad122d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.139+0000 7fadb73b0740 4 client.29611 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:13:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.139+0000 7fadb73b0740 1 -- 172.21.15.67:0/2625144948 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586ad07dae0 con 0x5586acd1e9f0 2024-07-31T15:13:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.139+0000 7fada9d9b640 1 --2- 172.21.15.67:0/2625144948 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5586acd1e9f0 0x5586ad122d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.140+0000 7fad91ffb640 10 client.29611 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.140+0000 7fada9d9b640 1 --2- 172.21.15.67:0/2625144948 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5586acd1e9f0 0x5586ad122d90 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eb453db070bac3a server_cookie=60f9bfb6554d3853 in_seq=0 out_seq=0 2024-07-31T15:13:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fad91ffb640 1 -- 172.21.15.67:0/2625144948 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586ad07dae0 con 0x5586acd1e9f0 2024-07-31T15:13:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fad91ffb640 10 client.29611 handle_command_reply: tid=0 2024-07-31T15:13:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fadb73b0740 1 client.29611 shutdown 2024-07-31T15:13:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fadb73b0740 1 -- 172.21.15.67:0/2625144948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd371b0 msgr2=0x5586ad1110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fadb73b0740 1 --2- 172.21.15.67:0/2625144948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd371b0 0x5586ad1110f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0096c0 tx=0x7fad9c02a410 comp rx=0 tx=0).stop 2024-07-31T15:13:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fadb73b0740 1 -- 172.21.15.67:0/2625144948 shutdown_connections 2024-07-31T15:13:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fadb73b0740 1 --2- 172.21.15.67:0/2625144948 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5586acd1e9f0 0x5586ad122d90 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fadb73b0740 1 --2- 172.21.15.67:0/2625144948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd60070 0x5586ad11cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fad91ffb640 0 client.29611 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fadb73b0740 1 --2- 172.21.15.67:0/2625144948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fadb73b0740 1 --2- 172.21.15.67:0/2625144948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd371b0 0x5586ad1110f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fadb73b0740 1 -- 172.21.15.67:0/2625144948 >> 172.21.15.67:0/2625144948 conn(0x5586acd54460 msgr2=0x5586ad101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fadb73b0740 1 -- 172.21.15.67:0/2625144948 shutdown_connections 2024-07-31T15:13:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fadb73b0740 1 -- 172.21.15.67:0/2625144948 wait complete. 2024-07-31T15:13:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.141+0000 7fadb73b0740 20 client.29611 trim_cache size 0 max 16384 2024-07-31T15:13:53.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.165+0000 7fadb73b0740 1 Processor -- start 2024-07-31T15:13:53.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fadb73b0740 1 -- start start 2024-07-31T15:13:53.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fadb73b0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd1e9f0 0x5586ad11ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fadb73b0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd371b0 0x5586ad10a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fadb73b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd3f390 0x5586ad10aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fadb73b0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1b0880 con 0x5586acd371b0 2024-07-31T15:13:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fadb73b0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1b23c0 con 0x5586acd1e9f0 2024-07-31T15:13:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fadb73b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1b0f50 con 0x5586acd3f390 2024-07-31T15:13:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fada9d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd371b0 0x5586ad10a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fada9d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd371b0 0x5586ad10a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46802/0 (socket says 172.21.15.67:46802) 2024-07-31T15:13:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fada9d9b640 1 -- 172.21.15.67:0/1569795740 learned_addr learned my addr 172.21.15.67:0/1569795740 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fadaad9d640 1 --2- 172.21.15.67:0/1569795740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd3f390 0x5586ad10aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fadaa59c640 1 --2- 172.21.15.67:0/1569795740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd1e9f0 0x5586ad11ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fada9d9b640 1 -- 172.21.15.67:0/1569795740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd3f390 msgr2=0x5586ad10aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fada9d9b640 1 --2- 172.21.15.67:0/1569795740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd3f390 0x5586ad10aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fada9d9b640 1 -- 172.21.15.67:0/1569795740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd1e9f0 msgr2=0x5586ad11ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fada9d9b640 1 --2- 172.21.15.67:0/1569795740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd1e9f0 0x5586ad11ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.166+0000 7fada9d9b640 1 -- 172.21.15.67:0/1569795740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ad1083b0 con 0x5586acd371b0 2024-07-31T15:13:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.167+0000 7fada9d9b640 1 --2- 172.21.15.67:0/1569795740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd371b0 0x5586ad10a660 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fada0004620 tx=0x7fada0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c315873c6a53c9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:53.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.167+0000 7fad93fff640 1 -- 172.21.15.67:0/1569795740 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada000f8d0 con 0x5586acd371b0 2024-07-31T15:13:53.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.167+0000 7fad93fff640 1 -- 172.21.15.67:0/1569795740 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fada002a430 con 0x5586acd371b0 2024-07-31T15:13:53.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.167+0000 7fad93fff640 1 -- 172.21.15.67:0/1569795740 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada001f660 con 0x5586acd371b0 2024-07-31T15:13:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.168+0000 7fadb73b0740 1 -- 172.21.15.67:0/1569795740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd371b0 msgr2=0x5586ad10a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.168+0000 7fadb73b0740 1 --2- 172.21.15.67:0/1569795740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd371b0 0x5586ad10a660 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fada0004620 tx=0x7fada0014000 comp rx=0 tx=0).stop 2024-07-31T15:13:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.168+0000 7fadb73b0740 1 -- 172.21.15.67:0/1569795740 shutdown_connections 2024-07-31T15:13:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.168+0000 7fadb73b0740 1 --2- 172.21.15.67:0/1569795740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd3f390 0x5586ad10aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.168+0000 7fadb73b0740 1 --2- 172.21.15.67:0/1569795740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd371b0 0x5586ad10a660 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.168+0000 7fadb73b0740 1 --2- 172.21.15.67:0/1569795740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd1e9f0 0x5586ad11ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.168+0000 7fadb73b0740 1 -- 172.21.15.67:0/1569795740 >> 172.21.15.67:0/1569795740 conn(0x5586acd54460 msgr2=0x5586ad10f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.168+0000 7fadb73b0740 1 -- 172.21.15.67:0/1569795740 shutdown_connections 2024-07-31T15:13:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.168+0000 7fadb73b0740 1 -- 172.21.15.67:0/1569795740 wait complete. 2024-07-31T15:13:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.168+0000 7fadb73b0740 1 -- start start 2024-07-31T15:13:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadb73b0740 1 Processor -- start 2024-07-31T15:13:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadb73b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd1e9f0 0x5586ad186c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadb73b0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd371b0 0x5586ad118c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadb73b0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad119160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadb73b0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1b1cf0 con 0x5586acd3f390 2024-07-31T15:13:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadb73b0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1b1620 con 0x5586acd371b0 2024-07-31T15:13:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadb73b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1ae880 con 0x5586acd1e9f0 2024-07-31T15:13:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadaa59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd1e9f0 0x5586ad186c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadb73b0740 10 client.0 _ll_register_callbacks cb 0x7fad9c1019a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fada9d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd371b0 0x5586ad118c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadaa59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd1e9f0 0x5586ad186c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:45852/0 (socket says 172.21.15.67:45852) 2024-07-31T15:13:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadaa59c640 1 -- 172.21.15.67:0/793406470 learned_addr learned my addr 172.21.15.67:0/793406470 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadaad9d640 1 --2- 172.21.15.67:0/793406470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad119160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadaa59c640 1 -- 172.21.15.67:0/793406470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd371b0 msgr2=0x5586ad118c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadaa59c640 1 --2- 172.21.15.67:0/793406470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd371b0 0x5586ad118c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadaa59c640 1 -- 172.21.15.67:0/793406470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 msgr2=0x5586ad119160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadaa59c640 1 --2- 172.21.15.67:0/793406470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad119160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.169+0000 7fadaa59c640 1 -- 172.21.15.67:0/793406470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ad184140 con 0x5586acd1e9f0 2024-07-31T15:13:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.170+0000 7fadaa59c640 1 --2- 172.21.15.67:0/793406470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd1e9f0 0x5586ad186c70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fad9c006010 tx=0x7fad9c0c6a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8732fcecac2c7816 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.170+0000 7fad91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:13:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.170+0000 7fad91ffb640 1 -- 172.21.15.67:0/793406470 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0e9030 con 0x5586acd1e9f0 2024-07-31T15:13:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.170+0000 7fadb73b0740 1 -- 172.21.15.67:0/793406470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada002a430 con 0x5586acd1e9f0 2024-07-31T15:13:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.170+0000 7fad91ffb640 1 -- 172.21.15.67:0/793406470 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7fad9c0ca120 con 0x5586acd1e9f0 2024-07-31T15:13:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.170+0000 7fad91ffb640 1 -- 172.21.15.67:0/793406470 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fad9c049040 con 0x5586acd1e9f0 2024-07-31T15:13:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.170+0000 7fadb73b0740 10 client.29617 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.170+0000 7fadb73b0740 1 -- 172.21.15.67:0/793406470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5586ad184140 con 0x5586acd1e9f0 2024-07-31T15:13:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.171+0000 7fad91ffb640 1 -- 172.21.15.67:0/793406470 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fada002a430 con 0x5586acd1e9f0 2024-07-31T15:13:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.171+0000 7fad91ffb640 1 -- 172.21.15.67:0/793406470 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fad9c0d0650 con 0x5586acd1e9f0 2024-07-31T15:13:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.171+0000 7fadb73b0740 10 client.29617 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.171+0000 7fadb73b0740 10 client.29617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.171+0000 7fadb73b0740 20 client.29617 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.171+0000 7fadb73b0740 1 --2- 172.21.15.67:0/793406470 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5586acd60070 0x5586ad191d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.171+0000 7fadb73b0740 4 client.29617 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:13:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.171+0000 7fadb73b0740 1 -- 172.21.15.67:0/793406470 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586ad079580 con 0x5586acd60070 2024-07-31T15:13:53.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.171+0000 7fada9d9b640 1 --2- 172.21.15.67:0/793406470 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5586acd60070 0x5586ad191d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:53.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.172+0000 7fada9d9b640 1 --2- 172.21.15.67:0/793406470 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5586acd60070 0x5586ad191d10 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c6d7c10de244761 server_cookie=aedb17a928731266 in_seq=0 out_seq=0 2024-07-31T15:13:53.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.172+0000 7fad91ffb640 10 client.29617 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:53.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.172+0000 7fad91ffb640 1 -- 172.21.15.67:0/793406470 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5586ad079580 con 0x5586acd60070 2024-07-31T15:13:53.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.172+0000 7fad91ffb640 10 client.29617 handle_command_reply: tid=0 2024-07-31T15:13:53.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.172+0000 7fadb73b0740 1 client.29617 shutdown 2024-07-31T15:13:53.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.173+0000 7fadb73b0740 1 -- 172.21.15.67:0/793406470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd1e9f0 msgr2=0x5586ad186c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:53.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.173+0000 7fadb73b0740 1 --2- 172.21.15.67:0/793406470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd1e9f0 0x5586ad186c70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fad9c006010 tx=0x7fad9c0c6a60 comp rx=0 tx=0).stop 2024-07-31T15:13:53.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.173+0000 7fadb73b0740 1 -- 172.21.15.67:0/793406470 shutdown_connections 2024-07-31T15:13:53.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.173+0000 7fadb73b0740 1 --2- 172.21.15.67:0/793406470 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x5586acd60070 0x5586ad191d10 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.173+0000 7fadb73b0740 1 --2- 172.21.15.67:0/793406470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5586acd3f390 0x5586ad119160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.173+0000 7fadb73b0740 1 --2- 172.21.15.67:0/793406470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586acd371b0 0x5586ad118c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.173+0000 7fadb73b0740 1 --2- 172.21.15.67:0/793406470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586acd1e9f0 0x5586ad186c70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.173+0000 7fadb73b0740 1 -- 172.21.15.67:0/793406470 >> 172.21.15.67:0/793406470 conn(0x5586acd54460 msgr2=0x5586ad10a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:53.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.173+0000 7fad91ffb640 0 client.29617 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:53.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.174+0000 7fadb73b0740 1 -- 172.21.15.67:0/793406470 shutdown_connections 2024-07-31T15:13:53.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.174+0000 7fadb73b0740 1 -- 172.21.15.67:0/793406470 wait complete. 2024-07-31T15:13:53.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.174+0000 7fadb73b0740 20 client.29617 trim_cache size 0 max 16384 2024-07-31T15:13:53.179 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:13:53.179 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T15:13:53.179 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T15:13:53.180 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:13:53.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.175+0000 7fadb0e5f640 1 -- 172.21.15.67:0/2905119881 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fad940b3df0 msgr2=0x7fad940b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:53.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.175+0000 7fadb0e5f640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fad940b3df0 0x7fad940b6300 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fadac1005a0 tx=0x7fada0005f90 comp rx=0 tx=0).stop 2024-07-31T15:13:53.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.175+0000 7fadb0e5f640 1 -- 172.21.15.67:0/2905119881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadac102490 msgr2=0x7fadac19ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:53.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.175+0000 7fadb0e5f640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadac102490 0x7fadac19ad90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fad9c005940 tx=0x7fad9c0040c0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.175+0000 7fadb0e5f640 1 -- 172.21.15.67:0/2905119881 shutdown_connections 2024-07-31T15:13:53.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.175+0000 7fadb0e5f640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fad940b3df0 0x7fad940b6300 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.175+0000 7fadb0e5f640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadac108720 0x7fadac19b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.175+0000 7fadb0e5f640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadac107dd0 0x7fadac19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.175+0000 7fadb0e5f640 1 --2- 172.21.15.67:0/2905119881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadac102490 0x7fadac19ad90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:53.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.175+0000 7fadb0e5f640 1 -- 172.21.15.67:0/2905119881 >> 172.21.15.67:0/2905119881 conn(0x7fadac079160 msgr2=0x7fadac100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:53.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.175+0000 7fadb0e5f640 1 -- 172.21.15.67:0/2905119881 shutdown_connections 2024-07-31T15:13:53.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:53.175+0000 7fadb0e5f640 1 -- 172.21.15.67:0/2905119881 wait complete. 2024-07-31T15:13:53.217 INFO:tasks.workunit.client.0.smithi067.stdout: NM .tmp_vmlinux.kallsyms1.syms 2024-07-31T15:13:53.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:53 smithi073 ceph-mon[36714]: pgmap v1128: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 312 op/s 2024-07-31T15:13:53.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:53 smithi067 ceph-mon[32750]: pgmap v1128: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 312 op/s 2024-07-31T15:13:53.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:53 smithi077 ceph-mon[37327]: pgmap v1128: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 312 op/s 2024-07-31T15:13:53.699 INFO:tasks.workunit.client.0.smithi067.stdout: KSYMS .tmp_vmlinux.kallsyms1.S 2024-07-31T15:13:53.957 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T15:13:53.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:13:54.244 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:13:54.752 INFO:tasks.workunit.client.0.smithi067.stdout: AS .tmp_vmlinux.kallsyms1.S 2024-07-31T15:13:55.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:55 smithi073 ceph-mon[36714]: pgmap v1129: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 301 op/s 2024-07-31T15:13:55.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:55 smithi067 ceph-mon[32750]: pgmap v1129: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 301 op/s 2024-07-31T15:13:55.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:55 smithi077 ceph-mon[37327]: pgmap v1129: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 301 op/s 2024-07-31T15:13:55.954 INFO:tasks.workunit.client.0.smithi067.stdout: LD .tmp_vmlinux.kallsyms2 2024-07-31T15:13:56.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.150+0000 7f658ccfb640 1 -- 172.21.15.67:0/2802235988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f658810d840 msgr2=0x7f658810fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:56.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.150+0000 7f658ccfb640 1 --2- 172.21.15.67:0/2802235988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f658810d840 0x7f658810fd30 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6580009ca0 tx=0x7f658002a8c0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.150+0000 7f658ccfb640 1 -- 172.21.15.67:0/2802235988 shutdown_connections 2024-07-31T15:13:56.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.150+0000 7f658ccfb640 1 --2- 172.21.15.67:0/2802235988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f658810d840 0x7f658810fd30 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.150+0000 7f658ccfb640 1 --2- 172.21.15.67:0/2802235988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6588104d30 0x7f658810d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.150+0000 7f658ccfb640 1 --2- 172.21.15.67:0/2802235988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65881043e0 0x7f65881047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.150+0000 7f658ccfb640 1 -- 172.21.15.67:0/2802235988 >> 172.21.15.67:0/2802235988 conn(0x7f6588079160 msgr2=0x7f6588079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:56.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.150+0000 7f658ccfb640 1 -- 172.21.15.67:0/2802235988 shutdown_connections 2024-07-31T15:13:56.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.150+0000 7f658ccfb640 1 -- 172.21.15.67:0/2802235988 wait complete. 2024-07-31T15:13:56.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.151+0000 7f658ccfb640 1 Processor -- start 2024-07-31T15:13:56.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.151+0000 7f658ccfb640 1 -- start start 2024-07-31T15:13:56.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.151+0000 7f658ccfb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65881043e0 0x7f6588199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.151+0000 7f658ccfb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6588104d30 0x7f658819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.151+0000 7f658ccfb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f658810d840 0x7f658819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.151+0000 7f658ccfb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f65881134a0 con 0x7f658810d840 2024-07-31T15:13:56.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.151+0000 7f658ccfb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6588113320 con 0x7f65881043e0 2024-07-31T15:13:56.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.151+0000 7f658ccfb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6588113620 con 0x7f6588104d30 2024-07-31T15:13:56.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.152+0000 7f6586d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f658810d840 0x7f658819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.152+0000 7f658659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65881043e0 0x7f6588199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.152+0000 7f6586d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f658810d840 0x7f658819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46822/0 (socket says 172.21.15.67:46822) 2024-07-31T15:13:56.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.152+0000 7f6586d9d640 1 -- 172.21.15.67:0/1968208862 learned_addr learned my addr 172.21.15.67:0/1968208862 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:56.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.152+0000 7f6585d9b640 1 --2- 172.21.15.67:0/1968208862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6588104d30 0x7f658819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.152+0000 7f658659c640 1 -- 172.21.15.67:0/1968208862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6588104d30 msgr2=0x7f658819a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:56.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.152+0000 7f658659c640 1 --2- 172.21.15.67:0/1968208862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6588104d30 0x7f658819a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.152+0000 7f658659c640 1 -- 172.21.15.67:0/1968208862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f658810d840 msgr2=0x7f658819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:56.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.152+0000 7f658659c640 1 --2- 172.21.15.67:0/1968208862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f658810d840 0x7f658819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.152+0000 7f658659c640 1 -- 172.21.15.67:0/1968208862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658819fe20 con 0x7f65881043e0 2024-07-31T15:13:56.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.152+0000 7f658659c640 1 --2- 172.21.15.67:0/1968208862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65881043e0 0x7f6588199eb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f657800e9c0 tx=0x7f657800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:56.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.152+0000 7f65777fe640 1 -- 172.21.15.67:0/1968208862 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657800ccc0 con 0x7f65881043e0 2024-07-31T15:13:56.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.153+0000 7f65777fe640 1 -- 172.21.15.67:0/1968208862 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65780035f0 con 0x7f65881043e0 2024-07-31T15:13:56.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.153+0000 7f65777fe640 1 -- 172.21.15.67:0/1968208862 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657800ce60 con 0x7f65881043e0 2024-07-31T15:13:56.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.153+0000 7f658ccfb640 1 -- 172.21.15.67:0/1968208862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65881a0110 con 0x7f65881043e0 2024-07-31T15:13:56.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.153+0000 7f658ccfb640 1 -- 172.21.15.67:0/1968208862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65881a6870 con 0x7f65881043e0 2024-07-31T15:13:56.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.154+0000 7f658ccfb640 1 -- 172.21.15.67:0/1968208862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65881047f0 con 0x7f65881043e0 2024-07-31T15:13:56.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.155+0000 7f65777fe640 1 -- 172.21.15.67:0/1968208862 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6578004000 con 0x7f65881043e0 2024-07-31T15:13:56.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.156+0000 7f65777fe640 1 --2- 172.21.15.67:0/1968208862 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65700b3dc0 0x7f65700b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.156+0000 7f6585d9b640 1 --2- 172.21.15.67:0/1968208862 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65700b3dc0 0x7f65700b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.156+0000 7f65777fe640 1 -- 172.21.15.67:0/1968208862 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7f65780e47b0 con 0x7f65881043e0 2024-07-31T15:13:56.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.157+0000 7f6585d9b640 1 --2- 172.21.15.67:0/1968208862 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65700b3dc0 0x7f65700b62d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6588079ae0 tx=0x7f657c00b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:56.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.159+0000 7f65777fe640 1 -- 172.21.15.67:0/1968208862 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f65780ab7e0 con 0x7f65881043e0 2024-07-31T15:13:56.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.401+0000 7f658ccfb640 1 -- 172.21.15.67:0/1968208862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f658819afe0 con 0x7f65881043e0 2024-07-31T15:13:56.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.402+0000 7f65777fe640 1 -- 172.21.15.67:0/1968208862 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11959 (secure 0 0 0) 0x7f65780ab0e0 con 0x7f65881043e0 2024-07-31T15:13:56.404 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:13:56.404 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":66,"btime":"2024-07-31T15:10:30:235518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:10:30.235494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29129":{"gid":29129,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/631186801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":631186801},{"type":"v1","addr":"172.21.15.73:6835","nonce":631186801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:13:56.404 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 66 2024-07-31T15:13:56.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.404+0000 7f658ccfb640 1 -- 172.21.15.67:0/1968208862 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65700b3dc0 msgr2=0x7f65700b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:56.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.404+0000 7f658ccfb640 1 --2- 172.21.15.67:0/1968208862 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65700b3dc0 0x7f65700b62d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6588079ae0 tx=0x7f657c00b410 comp rx=0 tx=0).stop 2024-07-31T15:13:56.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.404+0000 7f658ccfb640 1 -- 172.21.15.67:0/1968208862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65881043e0 msgr2=0x7f6588199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:56.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.404+0000 7f658ccfb640 1 --2- 172.21.15.67:0/1968208862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65881043e0 0x7f6588199eb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f657800e9c0 tx=0x7f657800ee80 comp rx=0 tx=0).stop 2024-07-31T15:13:56.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.404+0000 7f658ccfb640 1 -- 172.21.15.67:0/1968208862 shutdown_connections 2024-07-31T15:13:56.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.404+0000 7f658ccfb640 1 --2- 172.21.15.67:0/1968208862 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65700b3dc0 0x7f65700b62d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.404+0000 7f658ccfb640 1 --2- 172.21.15.67:0/1968208862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f658810d840 0x7f658819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.404+0000 7f658ccfb640 1 --2- 172.21.15.67:0/1968208862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6588104d30 0x7f658819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.404+0000 7f658ccfb640 1 --2- 172.21.15.67:0/1968208862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65881043e0 0x7f6588199eb0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.404+0000 7f658ccfb640 1 -- 172.21.15.67:0/1968208862 >> 172.21.15.67:0/1968208862 conn(0x7f6588079160 msgr2=0x7f65881001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:56.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.404+0000 7f658ccfb640 1 -- 172.21.15.67:0/1968208862 shutdown_connections 2024-07-31T15:13:56.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:56.405+0000 7f658ccfb640 1 -- 172.21.15.67:0/1968208862 wait complete. 2024-07-31T15:13:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.651+0000 7f07cdba9640 1 -- 172.21.15.67:0/671842191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8108670 msgr2=0x7f07c810eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.651+0000 7f07cdba9640 1 --2- 172.21.15.67:0/671842191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8108670 0x7f07c810eea0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009ca0 tx=0x7f07c002a8c0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.651+0000 7f07cdba9640 1 -- 172.21.15.67:0/671842191 shutdown_connections 2024-07-31T15:13:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.651+0000 7f07cdba9640 1 --2- 172.21.15.67:0/671842191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8108670 0x7f07c810eea0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.651+0000 7f07cdba9640 1 --2- 172.21.15.67:0/671842191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07c8107d20 0x7f07c8108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.651+0000 7f07cdba9640 1 --2- 172.21.15.67:0/671842191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c81023f0 0x7f07c8102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.651+0000 7f07cdba9640 1 -- 172.21.15.67:0/671842191 >> 172.21.15.67:0/671842191 conn(0x7f07c80fddf0 msgr2=0x7f07c8100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.651+0000 7f07cdba9640 1 -- 172.21.15.67:0/671842191 shutdown_connections 2024-07-31T15:13:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.651+0000 7f07cdba9640 1 -- 172.21.15.67:0/671842191 wait complete. 2024-07-31T15:13:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.651+0000 7f07cdba9640 1 Processor -- start 2024-07-31T15:13:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07cdba9640 1 -- start start 2024-07-31T15:13:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07cdba9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c81023f0 0x7f07c8199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07cdba9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8107d20 0x7f07c819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07cdba9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07c8108670 0x7f07c819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07cdba9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8113450 con 0x7f07c8107d20 2024-07-31T15:13:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07cdba9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81132d0 con 0x7f07c8108670 2024-07-31T15:13:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07cdba9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81135d0 con 0x7f07c81023f0 2024-07-31T15:13:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07c6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8107d20 0x7f07c819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07c6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8107d20 0x7f07c819a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46838/0 (socket says 172.21.15.67:46838) 2024-07-31T15:13:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07c6ffd640 1 -- 172.21.15.67:0/894954035 learned_addr learned my addr 172.21.15.67:0/894954035 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07c77fe640 1 --2- 172.21.15.67:0/894954035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c81023f0 0x7f07c8199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07c7fff640 1 --2- 172.21.15.67:0/894954035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07c8108670 0x7f07c819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07c6ffd640 1 -- 172.21.15.67:0/894954035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c81023f0 msgr2=0x7f07c8199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07c6ffd640 1 --2- 172.21.15.67:0/894954035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c81023f0 0x7f07c8199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07c6ffd640 1 -- 172.21.15.67:0/894954035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07c8108670 msgr2=0x7f07c819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07c6ffd640 1 --2- 172.21.15.67:0/894954035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07c8108670 0x7f07c819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.652+0000 7f07c6ffd640 1 -- 172.21.15.67:0/894954035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c819fcb0 con 0x7f07c8107d20 2024-07-31T15:13:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.653+0000 7f07c6ffd640 1 --2- 172.21.15.67:0/894954035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8107d20 0x7f07c819a3b0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00d740 tx=0x7f07bc00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.653+0000 7f07c4ff9640 1 -- 172.21.15.67:0/894954035 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc017070 con 0x7f07c8107d20 2024-07-31T15:13:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.653+0000 7f07cdba9640 1 -- 172.21.15.67:0/894954035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c819ffa0 con 0x7f07c8107d20 2024-07-31T15:13:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.653+0000 7f07c4ff9640 1 -- 172.21.15.67:0/894954035 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07bc010460 con 0x7f07c8107d20 2024-07-31T15:13:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.653+0000 7f07c4ff9640 1 -- 172.21.15.67:0/894954035 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc00f600 con 0x7f07c8107d20 2024-07-31T15:13:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.653+0000 7f07cdba9640 1 -- 172.21.15.67:0/894954035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c81a67c0 con 0x7f07c8107d20 2024-07-31T15:13:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.656+0000 7f07d4b77740 1 Processor -- start 2024-07-31T15:13:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.656+0000 7f07d4b77740 1 -- start start 2024-07-31T15:13:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.656+0000 7f07d4b77740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7521b0 0x557f1aaf4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.656+0000 7f07d4b77740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7292f0 0x557f1aaf5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.656+0000 7f07d4b77740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7314d0 0x557f1aaf5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.656+0000 7f07d4b77740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557f1aa6a3b0 con 0x557f1a7314d0 2024-07-31T15:13:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.656+0000 7f07d4b77740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557f1aa598b0 con 0x557f1a7292f0 2024-07-31T15:13:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.656+0000 7f07d4b77740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557f1aa5d8f0 con 0x557f1a7521b0 2024-07-31T15:13:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.656+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7292f0 0x557f1aaf5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.656+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7292f0 0x557f1aaf5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:38966/0 (socket says 172.21.15.67:38966) 2024-07-31T15:13:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.656+0000 7f07c77fe640 1 -- 172.21.15.67:0/2053381255 learned_addr learned my addr 172.21.15.67:0/2053381255 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.656+0000 7f07c7fff640 1 --2- 172.21.15.67:0/2053381255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7521b0 0x557f1aaf4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.657+0000 7f07c77fe640 1 -- 172.21.15.67:0/2053381255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7521b0 msgr2=0x557f1aaf4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.657+0000 7f07c77fe640 1 --2- 172.21.15.67:0/2053381255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7521b0 0x557f1aaf4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.657+0000 7f07c77fe640 1 -- 172.21.15.67:0/2053381255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7314d0 msgr2=0x557f1aaf5760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:13:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.657+0000 7f07c77fe640 1 --2- 172.21.15.67:0/2053381255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7314d0 0x557f1aaf5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.657+0000 7f07c77fe640 1 -- 172.21.15.67:0/2053381255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f1aa009a0 con 0x557f1a7292f0 2024-07-31T15:13:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.657+0000 7f07c7fff640 1 --2- 172.21.15.67:0/2053381255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7521b0 0x557f1aaf4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:13:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.657+0000 7f07c4ff9640 1 -- 172.21.15.67:0/894954035 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f07bc01a070 con 0x7f07c8107d20 2024-07-31T15:13:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.657+0000 7f07c77fe640 1 --2- 172.21.15.67:0/2053381255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7292f0 0x557f1aaf5220 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f07b800ad30 tx=0x7f07b800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76801f49510d698b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.657+0000 7f07adffb640 1 -- 172.21.15.67:0/2053381255 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8010840 con 0x557f1a7292f0 2024-07-31T15:13:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.657+0000 7f07adffb640 1 -- 172.21.15.67:0/2053381255 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b8014070 con 0x557f1a7292f0 2024-07-31T15:13:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.657+0000 7f07adffb640 1 -- 172.21.15.67:0/2053381255 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8018a50 con 0x557f1a7292f0 2024-07-31T15:13:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.658+0000 7f07c4ff9640 1 --2- 172.21.15.67:0/894954035 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07b00b3d40 0x7f07b00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.658+0000 7f07c7fff640 1 --2- 172.21.15.67:0/894954035 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07b00b3d40 0x7f07b00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.658+0000 7f07c4ff9640 1 -- 172.21.15.67:0/894954035 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7f07bc0e3ce0 con 0x7f07c8107d20 2024-07-31T15:13:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.658+0000 7f07d4b77740 1 -- 172.21.15.67:0/2053381255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7292f0 msgr2=0x557f1aaf5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.658+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2053381255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7292f0 0x557f1aaf5220 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f07b800ad30 tx=0x7f07b800f790 comp rx=0 tx=0).stop 2024-07-31T15:13:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.658+0000 7f07d4b77740 1 -- 172.21.15.67:0/2053381255 shutdown_connections 2024-07-31T15:13:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.658+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2053381255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7314d0 0x557f1aaf5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.658+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2053381255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7292f0 0x557f1aaf5220 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.658+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2053381255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7521b0 0x557f1aaf4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.658+0000 7f07d4b77740 1 -- 172.21.15.67:0/2053381255 >> 172.21.15.67:0/2053381255 conn(0x557f1a746300 msgr2=0x557f1aaf2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.658+0000 7f07d4b77740 1 -- 172.21.15.67:0/2053381255 shutdown_connections 2024-07-31T15:13:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.658+0000 7f07c7fff640 1 --2- 172.21.15.67:0/894954035 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07b00b3d40 0x7f07b00b6250 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f07c002f500 tx=0x7f07c002f370 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.658+0000 7f07d4b77740 1 -- 172.21.15.67:0/2053381255 wait complete. 2024-07-31T15:13:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.659+0000 7f07d4b77740 1 -- start start 2024-07-31T15:13:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.659+0000 7f07d4b77740 1 Processor -- start 2024-07-31T15:13:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.659+0000 7f07d4b77740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7292f0 0x557f1ab00840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.659+0000 7f07d4b77740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7314d0 0x557f1ab00d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.659+0000 7f07d4b77740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7521b0 0x557f1ab0c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.659+0000 7f07d4b77740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557f1a993ab0 con 0x557f1a7292f0 2024-07-31T15:13:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.659+0000 7f07d4b77740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557f1a9e1cf0 con 0x557f1a7314d0 2024-07-31T15:13:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.659+0000 7f07d4b77740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557f1ab03460 con 0x557f1a7521b0 2024-07-31T15:13:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.659+0000 7f07d4b77740 10 client.0 _ll_register_callbacks cb 0x557f1aa539f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.659+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7292f0 0x557f1ab00840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.659+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7292f0 0x557f1ab00840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46852/0 (socket says 172.21.15.67:46852) 2024-07-31T15:13:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.659+0000 7f07c77fe640 1 -- 172.21.15.67:0/2662632235 learned_addr learned my addr 172.21.15.67:0/2662632235 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.659+0000 7f07c6ffd640 1 --2- 172.21.15.67:0/2662632235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7314d0 0x557f1ab00d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.659+0000 7f07c7fff640 1 --2- 172.21.15.67:0/2662632235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7521b0 0x557f1ab0c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.660+0000 7f07c77fe640 1 -- 172.21.15.67:0/2662632235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7521b0 msgr2=0x557f1ab0c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.660+0000 7f07c77fe640 1 --2- 172.21.15.67:0/2662632235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7521b0 0x557f1ab0c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.660+0000 7f07c77fe640 1 -- 172.21.15.67:0/2662632235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7314d0 msgr2=0x557f1ab00d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.660+0000 7f07c77fe640 1 --2- 172.21.15.67:0/2662632235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7314d0 0x557f1ab00d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.660+0000 7f07c77fe640 1 -- 172.21.15.67:0/2662632235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f1a7bd790 con 0x557f1a7292f0 2024-07-31T15:13:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.660+0000 7f07c77fe640 1 --2- 172.21.15.67:0/2662632235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7292f0 0x557f1ab00840 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f07b801a040 tx=0x7f07b8018410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4798abf1d2542a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.660+0000 7f07a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:13:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.660+0000 7f07a6ffd640 1 -- 172.21.15.67:0/2662632235 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80189b0 con 0x557f1a7292f0 2024-07-31T15:13:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.660+0000 7f07a6ffd640 1 -- 172.21.15.67:0/2662632235 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7f07b8018b50 con 0x557f1a7292f0 2024-07-31T15:13:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.660+0000 7f07d4b77740 1 -- 172.21.15.67:0/2662632235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b8014070 con 0x557f1a7292f0 2024-07-31T15:13:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.660+0000 7f07a6ffd640 1 -- 172.21.15.67:0/2662632235 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f07b8018d50 con 0x557f1a7292f0 2024-07-31T15:13:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.660+0000 7f07d4b77740 10 client.20271 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.660+0000 7f07d4b77740 1 -- 172.21.15.67:0/2662632235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x557f1a836620 con 0x557f1a7292f0 2024-07-31T15:13:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.660+0000 7f07a6ffd640 1 -- 172.21.15.67:0/2662632235 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b8014070 con 0x557f1a7292f0 2024-07-31T15:13:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.661+0000 7f07a6ffd640 1 -- 172.21.15.67:0/2662632235 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f07b8010ba0 con 0x557f1a7292f0 2024-07-31T15:13:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.661+0000 7f07d4b77740 10 client.20271 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.661+0000 7f07d4b77740 10 client.20271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.661+0000 7f07d4b77740 20 client.20271 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.661+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2662632235 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f1a710d80 0x557f1ab126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.661+0000 7f07d4b77740 4 client.20271 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:13:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.661+0000 7f07d4b77740 1 -- 172.21.15.67:0/2662632235 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f1aa6c300 con 0x557f1a710d80 2024-07-31T15:13:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.661+0000 7f07c6ffd640 1 --2- 172.21.15.67:0/2662632235 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f1a710d80 0x557f1ab126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.662+0000 7f07c6ffd640 1 --2- 172.21.15.67:0/2662632235 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f1a710d80 0x557f1ab126a0 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a4edf51f74e169 server_cookie=fc6988baf77d0c1a in_seq=0 out_seq=0 2024-07-31T15:13:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.662+0000 7f07a6ffd640 10 client.20271 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.662+0000 7f07a6ffd640 1 -- 172.21.15.67:0/2662632235 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f1aa6c300 con 0x557f1a710d80 2024-07-31T15:13:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.662+0000 7f07a6ffd640 10 client.20271 handle_command_reply: tid=0 2024-07-31T15:13:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.662+0000 7f07d4b77740 1 client.20271 shutdown 2024-07-31T15:13:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.662+0000 7f07d4b77740 1 -- 172.21.15.67:0/2662632235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7292f0 msgr2=0x557f1ab00840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.662+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2662632235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7292f0 0x557f1ab00840 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f07b801a040 tx=0x7f07b8018410 comp rx=0 tx=0).stop 2024-07-31T15:13:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.662+0000 7f07d4b77740 1 -- 172.21.15.67:0/2662632235 shutdown_connections 2024-07-31T15:13:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.662+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2662632235 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f1a710d80 0x557f1ab126a0 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.662+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2662632235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7521b0 0x557f1ab0c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.663+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2662632235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7314d0 0x557f1ab00d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.663+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2662632235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7292f0 0x557f1ab00840 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.663+0000 7f07a6ffd640 0 client.20271 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.663+0000 7f07d4b77740 1 -- 172.21.15.67:0/2662632235 >> 172.21.15.67:0/2662632235 conn(0x557f1a746300 msgr2=0x557f1aaf12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.663+0000 7f07d4b77740 1 -- 172.21.15.67:0/2662632235 shutdown_connections 2024-07-31T15:13:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.663+0000 7f07d4b77740 1 -- 172.21.15.67:0/2662632235 wait complete. 2024-07-31T15:13:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.663+0000 7f07d4b77740 20 client.20271 trim_cache size 0 max 16384 2024-07-31T15:13:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.683+0000 7f07d4b77740 1 Processor -- start 2024-07-31T15:13:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.683+0000 7f07d4b77740 1 -- start start 2024-07-31T15:13:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.683+0000 7f07d4b77740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7292f0 0x557f1aaf66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.683+0000 7f07d4b77740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7314d0 0x557f1aaffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.683+0000 7f07d4b77740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7521b0 0x557f1aaf28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.683+0000 7f07d4b77740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b0740 con 0x557f1a7292f0 2024-07-31T15:13:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.683+0000 7f07d4b77740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b2280 con 0x557f1a7521b0 2024-07-31T15:13:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.683+0000 7f07d4b77740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b0e10 con 0x557f1a7314d0 2024-07-31T15:13:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.683+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7292f0 0x557f1aaf66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.684+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7292f0 0x557f1aaf66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46864/0 (socket says 172.21.15.67:46864) 2024-07-31T15:13:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.684+0000 7f07c77fe640 1 -- 172.21.15.67:0/3226374889 learned_addr learned my addr 172.21.15.67:0/3226374889 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.684+0000 7f07c7fff640 1 --2- 172.21.15.67:0/3226374889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7521b0 0x557f1aaf28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.684+0000 7f07c6ffd640 1 --2- 172.21.15.67:0/3226374889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7314d0 0x557f1aaffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.684+0000 7f07c77fe640 1 -- 172.21.15.67:0/3226374889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7314d0 msgr2=0x557f1aaffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.684+0000 7f07c77fe640 1 --2- 172.21.15.67:0/3226374889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7314d0 0x557f1aaffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.684+0000 7f07c77fe640 1 -- 172.21.15.67:0/3226374889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7521b0 msgr2=0x557f1aaf28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.684+0000 7f07c77fe640 1 --2- 172.21.15.67:0/3226374889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7521b0 0x557f1aaf28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.684+0000 7f07c77fe640 1 -- 172.21.15.67:0/3226374889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f1aaf7210 con 0x557f1a7292f0 2024-07-31T15:13:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.684+0000 7f07c77fe640 1 --2- 172.21.15.67:0/3226374889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7292f0 0x557f1aaf66d0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f07b800fc90 tx=0x7f07b8005390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3d485599bc5220 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.684+0000 7f07ad7fa640 1 -- 172.21.15.67:0/3226374889 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8018480 con 0x557f1a7292f0 2024-07-31T15:13:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.684+0000 7f07ad7fa640 1 -- 172.21.15.67:0/3226374889 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b8014070 con 0x557f1a7292f0 2024-07-31T15:13:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.684+0000 7f07ad7fa640 1 -- 172.21.15.67:0/3226374889 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8004b30 con 0x557f1a7292f0 2024-07-31T15:13:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.685+0000 7f07d4b77740 1 -- 172.21.15.67:0/3226374889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7292f0 msgr2=0x557f1aaf66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.685+0000 7f07d4b77740 1 --2- 172.21.15.67:0/3226374889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7292f0 0x557f1aaf66d0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f07b800fc90 tx=0x7f07b8005390 comp rx=0 tx=0).stop 2024-07-31T15:13:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.685+0000 7f07d4b77740 1 -- 172.21.15.67:0/3226374889 shutdown_connections 2024-07-31T15:13:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.685+0000 7f07d4b77740 1 --2- 172.21.15.67:0/3226374889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7521b0 0x557f1aaf28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.685+0000 7f07d4b77740 1 --2- 172.21.15.67:0/3226374889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7314d0 0x557f1aaffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.685+0000 7f07d4b77740 1 --2- 172.21.15.67:0/3226374889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7292f0 0x557f1aaf66d0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.685+0000 7f07d4b77740 1 -- 172.21.15.67:0/3226374889 >> 172.21.15.67:0/3226374889 conn(0x557f1a746300 msgr2=0x557f1aaf21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.685+0000 7f07d4b77740 1 -- 172.21.15.67:0/3226374889 shutdown_connections 2024-07-31T15:13:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.685+0000 7f07d4b77740 1 -- 172.21.15.67:0/3226374889 wait complete. 2024-07-31T15:13:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.686+0000 7f07d4b77740 1 -- start start 2024-07-31T15:13:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.686+0000 7f07d4b77740 1 Processor -- start 2024-07-31T15:13:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.686+0000 7f07d4b77740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7292f0 0x557f1ab15e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.686+0000 7f07d4b77740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7314d0 0x557f1ab14360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.686+0000 7f07d4b77740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7521b0 0x557f1ab149b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.686+0000 7f07d4b77740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b1bb0 con 0x557f1a7521b0 2024-07-31T15:13:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.686+0000 7f07d4b77740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b14e0 con 0x557f1a7292f0 2024-07-31T15:13:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.686+0000 7f07d4b77740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81ae740 con 0x557f1a7314d0 2024-07-31T15:13:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.686+0000 7f07d4b77740 10 client.0 _ll_register_callbacks cb 0x7f07b800ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.686+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7521b0 0x557f1ab149b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.686+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7521b0 0x557f1ab149b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46872/0 (socket says 172.21.15.67:46872) 2024-07-31T15:13:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.686+0000 7f07c7fff640 1 -- 172.21.15.67:0/2701520695 learned_addr learned my addr 172.21.15.67:0/2701520695 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07c6ffd640 1 --2- 172.21.15.67:0/2701520695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7314d0 0x557f1ab14360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07c77fe640 1 --2- 172.21.15.67:0/2701520695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7292f0 0x557f1ab15e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07c7fff640 1 -- 172.21.15.67:0/2701520695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7314d0 msgr2=0x557f1ab14360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07c7fff640 1 --2- 172.21.15.67:0/2701520695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7314d0 0x557f1ab14360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07c7fff640 1 -- 172.21.15.67:0/2701520695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7292f0 msgr2=0x557f1ab15e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07c7fff640 1 --2- 172.21.15.67:0/2701520695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7292f0 0x557f1ab15e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07c7fff640 1 -- 172.21.15.67:0/2701520695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f1ab15120 con 0x557f1a7521b0 2024-07-31T15:13:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07c6ffd640 1 --2- 172.21.15.67:0/2701520695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7314d0 0x557f1ab14360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:13:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07c7fff640 1 --2- 172.21.15.67:0/2701520695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7521b0 0x557f1ab149b0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f07c0035000 tx=0x7f07c00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bfcf9298ad67b1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:13:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07a6ffd640 1 -- 172.21.15.67:0/2701520695 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0059060 con 0x557f1a7521b0 2024-07-31T15:13:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07a6ffd640 1 -- 172.21.15.67:0/2701520695 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7f07c0004250 con 0x557f1a7521b0 2024-07-31T15:13:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07d4b77740 1 -- 172.21.15.67:0/2701520695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b8014070 con 0x557f1a7521b0 2024-07-31T15:13:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07a6ffd640 1 -- 172.21.15.67:0/2701520695 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f07c0060050 con 0x557f1a7521b0 2024-07-31T15:13:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07d4b77740 10 client.20277 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07d4b77740 1 -- 172.21.15.67:0/2701520695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x557f1ab79c70 con 0x557f1a7521b0 2024-07-31T15:13:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.687+0000 7f07a6ffd640 1 -- 172.21.15.67:0/2701520695 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b8014070 con 0x557f1a7521b0 2024-07-31T15:13:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.688+0000 7f07a6ffd640 1 -- 172.21.15.67:0/2701520695 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f07c00305c0 con 0x557f1a7521b0 2024-07-31T15:13:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.688+0000 7f07d4b77740 10 client.20277 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.688+0000 7f07d4b77740 10 client.20277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:13:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.688+0000 7f07d4b77740 20 client.20277 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.688+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2701520695 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f1a6e61c0 0x557f1ab7c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.688+0000 7f07d4b77740 4 client.20277 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:13:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.688+0000 7f07d4b77740 1 -- 172.21.15.67:0/2701520695 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f1aa67da0 con 0x557f1a6e61c0 2024-07-31T15:13:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.688+0000 7f07c77fe640 1 --2- 172.21.15.67:0/2701520695 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f1a6e61c0 0x557f1ab7c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.689+0000 7f07a6ffd640 10 client.20277 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.689+0000 7f07c77fe640 1 --2- 172.21.15.67:0/2701520695 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f1a6e61c0 0x557f1ab7c490 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8caaa3b57b1b8ecc server_cookie=dab1b7902852ba6 in_seq=0 out_seq=0 2024-07-31T15:13:56.836 INFO:tasks.workunit.client.0.smithi067.stdout: NM .tmp_vmlinux.kallsyms2.syms 2024-07-31T15:13:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.898+0000 7f07a6ffd640 1 -- 172.21.15.67:0/2701520695 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f1aa67da0 con 0x557f1a6e61c0 2024-07-31T15:13:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.898+0000 7f07a6ffd640 10 client.20277 handle_command_reply: tid=0 2024-07-31T15:13:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.898+0000 7f07d4b77740 1 client.20277 shutdown 2024-07-31T15:13:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.898+0000 7f07d4b77740 1 -- 172.21.15.67:0/2701520695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7521b0 msgr2=0x557f1ab149b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.898+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2701520695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7521b0 0x557f1ab149b0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f07c0035000 tx=0x7f07c00095c0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.898+0000 7f07d4b77740 1 -- 172.21.15.67:0/2701520695 shutdown_connections 2024-07-31T15:13:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.898+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2701520695 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557f1a6e61c0 0x557f1ab7c490 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.898+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2701520695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f1a7521b0 0x557f1ab149b0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.898+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2701520695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f1a7314d0 0x557f1ab14360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.898+0000 7f07d4b77740 1 --2- 172.21.15.67:0/2701520695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f1a7292f0 0x557f1ab15e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.898+0000 7f07d4b77740 1 -- 172.21.15.67:0/2701520695 >> 172.21.15.67:0/2701520695 conn(0x557f1a746300 msgr2=0x557f1aaf64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.898+0000 7f07a6ffd640 0 client.20277 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:13:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.898+0000 7f07d4b77740 1 -- 172.21.15.67:0/2701520695 shutdown_connections 2024-07-31T15:13:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.898+0000 7f07d4b77740 1 -- 172.21.15.67:0/2701520695 wait complete. 2024-07-31T15:13:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.899+0000 7f07d4b77740 20 client.20277 trim_cache size 0 max 16384 2024-07-31T15:13:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:13:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:13:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:13:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:13:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.899+0000 7f07cdba9640 1 -- 172.21.15.67:0/894954035 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07b00b3d40 msgr2=0x7f07b00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.899+0000 7f07cdba9640 1 --2- 172.21.15.67:0/894954035 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07b00b3d40 0x7f07b00b6250 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f07c002f500 tx=0x7f07c002f370 comp rx=0 tx=0).stop 2024-07-31T15:13:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.899+0000 7f07cdba9640 1 -- 172.21.15.67:0/894954035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8107d20 msgr2=0x7f07c819a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.899+0000 7f07cdba9640 1 --2- 172.21.15.67:0/894954035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8107d20 0x7f07c819a3b0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00d740 tx=0x7f07bc00db40 comp rx=0 tx=0).stop 2024-07-31T15:13:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.899+0000 7f07cdba9640 1 -- 172.21.15.67:0/894954035 shutdown_connections 2024-07-31T15:13:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.899+0000 7f07cdba9640 1 --2- 172.21.15.67:0/894954035 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f07b00b3d40 0x7f07b00b6250 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.899+0000 7f07cdba9640 1 --2- 172.21.15.67:0/894954035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07c8108670 0x7f07c819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.899+0000 7f07cdba9640 1 --2- 172.21.15.67:0/894954035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8107d20 0x7f07c819a3b0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.899+0000 7f07cdba9640 1 --2- 172.21.15.67:0/894954035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c81023f0 0x7f07c8199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.899+0000 7f07cdba9640 1 -- 172.21.15.67:0/894954035 >> 172.21.15.67:0/894954035 conn(0x7f07c80fddf0 msgr2=0x7f07c8107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.900+0000 7f07cdba9640 1 -- 172.21.15.67:0/894954035 shutdown_connections 2024-07-31T15:13:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:13:56.900+0000 7f07cdba9640 1 -- 172.21.15.67:0/894954035 wait complete. 2024-07-31T15:13:57.166 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T15:13:57.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:57 smithi073 ceph-mon[36714]: pgmap v1130: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 234 op/s 2024-07-31T15:13:57.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:57 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1968208862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:13:57.440 INFO:tasks.workunit.client.0.smithi067.stdout: KSYMS .tmp_vmlinux.kallsyms2.S 2024-07-31T15:13:57.442 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:13:57.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:57 smithi077 ceph-mon[37327]: pgmap v1130: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 234 op/s 2024-07-31T15:13:57.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:57 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1968208862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:13:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:57 smithi067 ceph-mon[32750]: pgmap v1130: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 234 op/s 2024-07-31T15:13:57.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:57 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1968208862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:13:58.568 INFO:tasks.workunit.client.0.smithi067.stdout: AS .tmp_vmlinux.kallsyms2.S 2024-07-31T15:13:59.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:13:59 smithi067 ceph-mon[32750]: pgmap v1131: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-07-31T15:13:59.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.319+0000 7f26ced66640 1 -- 172.21.15.67:0/61445330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c8107e50 msgr2=0x7f26c8108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:59.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.319+0000 7f26ced66640 1 --2- 172.21.15.67:0/61445330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c8107e50 0x7f26c8108260 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009900 tx=0x7f26b002a410 comp rx=0 tx=0).stop 2024-07-31T15:13:59.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.320+0000 7f26ced66640 1 -- 172.21.15.67:0/61445330 shutdown_connections 2024-07-31T15:13:59.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.320+0000 7f26ced66640 1 --2- 172.21.15.67:0/61445330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26c81087a0 0x7f26c810efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.320+0000 7f26ced66640 1 --2- 172.21.15.67:0/61445330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c8107e50 0x7f26c8108260 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.320+0000 7f26ced66640 1 --2- 172.21.15.67:0/61445330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26c8102b70 0x7f26c8102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.320+0000 7f26ced66640 1 -- 172.21.15.67:0/61445330 >> 172.21.15.67:0/61445330 conn(0x7f26c8077690 msgr2=0x7f26c8077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:59.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.320+0000 7f26ced66640 1 -- 172.21.15.67:0/61445330 shutdown_connections 2024-07-31T15:13:59.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.320+0000 7f26ced66640 1 -- 172.21.15.67:0/61445330 wait complete. 2024-07-31T15:13:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.321+0000 7f26ced66640 1 Processor -- start 2024-07-31T15:13:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.321+0000 7f26ced66640 1 -- start start 2024-07-31T15:13:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.321+0000 7f26ced66640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c8102b70 0x7f26c819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.321+0000 7f26ced66640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26c8107e50 0x7f26c819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.321+0000 7f26ced66640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26c81087a0 0x7f26c819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.321+0000 7f26ced66640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f26c8112430 con 0x7f26c8102b70 2024-07-31T15:13:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.321+0000 7f26ced66640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f26c81122b0 con 0x7f26c8107e50 2024-07-31T15:13:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.321+0000 7f26ced66640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26c81125b0 con 0x7f26c81087a0 2024-07-31T15:13:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.321+0000 7f26ccb02640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c8102b70 0x7f26c819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.321+0000 7f26ccb02640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c8102b70 0x7f26c819ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46896/0 (socket says 172.21.15.67:46896) 2024-07-31T15:13:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.321+0000 7f26ccb02640 1 -- 172.21.15.67:0/393444446 learned_addr learned my addr 172.21.15.67:0/393444446 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.322+0000 7f26c7fff640 1 --2- 172.21.15.67:0/393444446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26c8107e50 0x7f26c819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.322+0000 7f26cd303640 1 --2- 172.21.15.67:0/393444446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26c81087a0 0x7f26c819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.322+0000 7f26ccb02640 1 -- 172.21.15.67:0/393444446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26c81087a0 msgr2=0x7f26c819b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.322+0000 7f26ccb02640 1 --2- 172.21.15.67:0/393444446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26c81087a0 0x7f26c819b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.322+0000 7f26ccb02640 1 -- 172.21.15.67:0/393444446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26c8107e50 msgr2=0x7f26c819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.322+0000 7f26ccb02640 1 --2- 172.21.15.67:0/393444446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26c8107e50 0x7f26c819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.322+0000 7f26ccb02640 1 -- 172.21.15.67:0/393444446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c81a6020 con 0x7f26c8102b70 2024-07-31T15:13:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.322+0000 7f26ccb02640 1 --2- 172.21.15.67:0/393444446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c8102b70 0x7f26c819ae20 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f26b800eb90 tx=0x7f26b800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.322+0000 7f26c5ffb640 1 -- 172.21.15.67:0/393444446 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8004d20 con 0x7f26c8102b70 2024-07-31T15:13:59.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.322+0000 7f26ced66640 1 -- 172.21.15.67:0/393444446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c81a6310 con 0x7f26c8102b70 2024-07-31T15:13:59.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.323+0000 7f26c5ffb640 1 -- 172.21.15.67:0/393444446 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26b800ee90 con 0x7f26c8102b70 2024-07-31T15:13:59.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.323+0000 7f26c5ffb640 1 -- 172.21.15.67:0/393444446 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8010670 con 0x7f26c8102b70 2024-07-31T15:13:59.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.323+0000 7f26ced66640 1 -- 172.21.15.67:0/393444446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c81a6820 con 0x7f26c8102b70 2024-07-31T15:13:59.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.325+0000 7f26d52b7740 1 Processor -- start 2024-07-31T15:13:59.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.326+0000 7f26d52b7740 1 -- start start 2024-07-31T15:13:59.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.326+0000 7f26d52b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ae7070 0x563609e8c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.326+0000 7f26d52b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609abe1b0 0x563609e8cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.326+0000 7f26d52b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609ac6390 0x563609e8cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.326+0000 7f26d52b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563609e02bb0 con 0x563609ae7070 2024-07-31T15:13:59.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.326+0000 7f26d52b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563609df20b0 con 0x563609ac6390 2024-07-31T15:13:59.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.326+0000 7f26d52b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563609df60f0 con 0x563609abe1b0 2024-07-31T15:13:59.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.326+0000 7f26cd303640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ae7070 0x563609e8c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.326+0000 7f26cd303640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ae7070 0x563609e8c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46902/0 (socket says 172.21.15.67:46902) 2024-07-31T15:13:59.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.326+0000 7f26cd303640 1 -- 172.21.15.67:0/1638419977 learned_addr learned my addr 172.21.15.67:0/1638419977 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:59.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.327+0000 7f26cd303640 1 -- 172.21.15.67:0/1638419977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609abe1b0 msgr2=0x563609e8cab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:13:59.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.327+0000 7f26cd303640 1 --2- 172.21.15.67:0/1638419977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609abe1b0 0x563609e8cab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.327+0000 7f26cd303640 1 -- 172.21.15.67:0/1638419977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609ac6390 msgr2=0x563609e8cff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:13:59.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.327+0000 7f26cd303640 1 --2- 172.21.15.67:0/1638419977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609ac6390 0x563609e8cff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.327+0000 7f26cd303640 1 -- 172.21.15.67:0/1638419977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563609d98b90 con 0x563609ae7070 2024-07-31T15:13:59.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.327+0000 7f26cd303640 1 --2- 172.21.15.67:0/1638419977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ae7070 0x563609e8c570 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f26c000ab90 tx=0x7f26c000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01bdac1c90195fa server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:59.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.327+0000 7f26beffd640 1 -- 172.21.15.67:0/1638419977 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0005220 con 0x563609ae7070 2024-07-31T15:13:59.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.327+0000 7f26c5ffb640 1 -- 172.21.15.67:0/393444446 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f26b8005020 con 0x7f26c8102b70 2024-07-31T15:13:59.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.327+0000 7f26beffd640 1 -- 172.21.15.67:0/1638419977 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c00047a0 con 0x563609ae7070 2024-07-31T15:13:59.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.327+0000 7f26beffd640 1 -- 172.21.15.67:0/1638419977 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0018680 con 0x563609ae7070 2024-07-31T15:13:59.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.328+0000 7f26c5ffb640 1 --2- 172.21.15.67:0/393444446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f26b40b3d70 0x7f26b40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.328+0000 7f26ccb02640 1 --2- 172.21.15.67:0/393444446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f26b40b3d70 0x7f26b40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.328+0000 7f26d52b7740 1 -- 172.21.15.67:0/1638419977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ae7070 msgr2=0x563609e8c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.328+0000 7f26d52b7740 1 --2- 172.21.15.67:0/1638419977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ae7070 0x563609e8c570 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f26c000ab90 tx=0x7f26c000f790 comp rx=0 tx=0).stop 2024-07-31T15:13:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.328+0000 7f26c5ffb640 1 -- 172.21.15.67:0/393444446 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7f26b80e4950 con 0x7f26c8102b70 2024-07-31T15:13:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.328+0000 7f26d52b7740 1 -- 172.21.15.67:0/1638419977 shutdown_connections 2024-07-31T15:13:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.328+0000 7f26d52b7740 1 --2- 172.21.15.67:0/1638419977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609ac6390 0x563609e8cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.328+0000 7f26d52b7740 1 --2- 172.21.15.67:0/1638419977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609abe1b0 0x563609e8cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.328+0000 7f26d52b7740 1 --2- 172.21.15.67:0/1638419977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ae7070 0x563609e8c570 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.328+0000 7f26d52b7740 1 -- 172.21.15.67:0/1638419977 >> 172.21.15.67:0/1638419977 conn(0x563609adb460 msgr2=0x563609e8a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.329+0000 7f26d52b7740 1 -- 172.21.15.67:0/1638419977 shutdown_connections 2024-07-31T15:13:59.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.329+0000 7f26d52b7740 1 -- 172.21.15.67:0/1638419977 wait complete. 2024-07-31T15:13:59.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.329+0000 7f26ccb02640 1 --2- 172.21.15.67:0/393444446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f26b40b3d70 0x7f26b40b6280 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x563609e8b1f0 tx=0x7f26b8005870 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:59.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.329+0000 7f26d52b7740 1 -- start start 2024-07-31T15:13:59.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.329+0000 7f26d52b7740 1 Processor -- start 2024-07-31T15:13:59.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.330+0000 7f26d52b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609abe1b0 0x563609e9c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.330+0000 7f26d52b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ac6390 0x563609e9c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.330+0000 7f26d52b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609ae7070 0x563609ea8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.330+0000 7f26d52b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563609c55700 con 0x563609ac6390 2024-07-31T15:13:59.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.330+0000 7f26d52b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563609d949e0 con 0x563609abe1b0 2024-07-31T15:13:59.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.330+0000 7f26d52b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563609e9f0b0 con 0x563609ae7070 2024-07-31T15:13:59.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.330+0000 7f26d52b7740 10 client.0 _ll_register_callbacks cb 0x563609daf4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:59.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.330+0000 7f26cd303640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609ae7070 0x563609ea8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.330+0000 7f26c7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ac6390 0x563609e9c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.330+0000 7f26cd303640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609ae7070 0x563609ea8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:45956/0 (socket says 172.21.15.67:45956) 2024-07-31T15:13:59.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.330+0000 7f26cd303640 1 -- 172.21.15.67:0/2162001120 learned_addr learned my addr 172.21.15.67:0/2162001120 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:59.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.330+0000 7f26cd303640 1 --2- 172.21.15.67:0/2162001120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609abe1b0 0x563609e9c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.331+0000 7f26c7fff640 1 -- 172.21.15.67:0/2162001120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609ae7070 msgr2=0x563609ea8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:59.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.331+0000 7f26c7fff640 1 --2- 172.21.15.67:0/2162001120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609ae7070 0x563609ea8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.331+0000 7f26c7fff640 1 -- 172.21.15.67:0/2162001120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609abe1b0 msgr2=0x563609e9c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:59.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.331+0000 7f26c7fff640 1 --2- 172.21.15.67:0/2162001120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609abe1b0 0x563609e9c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.331+0000 7f26c7fff640 1 -- 172.21.15.67:0/2162001120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563609b52650 con 0x563609ac6390 2024-07-31T15:13:59.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.331+0000 7f26cd303640 1 --2- 172.21.15.67:0/2162001120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609ae7070 0x563609ea8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:13:59.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.331+0000 7f26c7fff640 1 --2- 172.21.15.67:0/2162001120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ac6390 0x563609e9c9f0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f26b002a930 tx=0x7f26b0032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1676772b1d39cc1f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:59.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.331+0000 7f26a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:13:59.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.331+0000 7f26a7fff640 1 -- 172.21.15.67:0/2162001120 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0040070 con 0x563609ac6390 2024-07-31T15:13:59.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.331+0000 7f26a7fff640 1 -- 172.21.15.67:0/2162001120 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7f26b00318b0 con 0x563609ac6390 2024-07-31T15:13:59.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.331+0000 7f26d52b7740 1 -- 172.21.15.67:0/2162001120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c00047a0 con 0x563609ac6390 2024-07-31T15:13:59.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.331+0000 7f26a7fff640 1 -- 172.21.15.67:0/2162001120 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f26b0035040 con 0x563609ac6390 2024-07-31T15:13:59.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.331+0000 7f26d52b7740 10 client.20289 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:59.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.331+0000 7f26d52b7740 1 -- 172.21.15.67:0/2162001120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x563609eab0b0 con 0x563609ac6390 2024-07-31T15:13:59.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.332+0000 7f26a7fff640 1 -- 172.21.15.67:0/2162001120 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c00047a0 con 0x563609ac6390 2024-07-31T15:13:59.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.332+0000 7f26a7fff640 1 -- 172.21.15.67:0/2162001120 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f26b0004750 con 0x563609ac6390 2024-07-31T15:13:59.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.332+0000 7f26d52b7740 10 client.20289 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:59.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.332+0000 7f26d52b7740 10 client.20289 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:13:59.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.332+0000 7f26d52b7740 20 client.20289 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:59.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.332+0000 7f26d52b7740 1 --2- 172.21.15.67:0/2162001120 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563609aa59f0 0x563609eae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.332+0000 7f26d52b7740 4 client.20289 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:13:59.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.332+0000 7f26d52b7740 1 -- 172.21.15.67:0/2162001120 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563609e04b00 con 0x563609aa59f0 2024-07-31T15:13:59.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.332+0000 7f26cd303640 1 --2- 172.21.15.67:0/2162001120 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563609aa59f0 0x563609eae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.333+0000 7f26cd303640 1 --2- 172.21.15.67:0/2162001120 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563609aa59f0 0x563609eae5a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6fab5567f43ca2da server_cookie=b445874eb0594ff1 in_seq=0 out_seq=0 2024-07-31T15:13:59.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.333+0000 7f26a7fff640 10 client.20289 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:13:59.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26a7fff640 1 -- 172.21.15.67:0/2162001120 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563609e04b00 con 0x563609aa59f0 2024-07-31T15:13:59.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26a7fff640 10 client.20289 handle_command_reply: tid=0 2024-07-31T15:13:59.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26d52b7740 1 client.20289 shutdown 2024-07-31T15:13:59.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26d52b7740 1 -- 172.21.15.67:0/2162001120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ac6390 msgr2=0x563609e9c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:59.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26d52b7740 1 --2- 172.21.15.67:0/2162001120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ac6390 0x563609e9c9f0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f26b002a930 tx=0x7f26b0032e90 comp rx=0 tx=0).stop 2024-07-31T15:13:59.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26d52b7740 1 -- 172.21.15.67:0/2162001120 shutdown_connections 2024-07-31T15:13:59.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26d52b7740 1 --2- 172.21.15.67:0/2162001120 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563609aa59f0 0x563609eae5a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26a7fff640 0 client.20289 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:13:59.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26d52b7740 1 --2- 172.21.15.67:0/2162001120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609ae7070 0x563609ea8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26d52b7740 1 --2- 172.21.15.67:0/2162001120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ac6390 0x563609e9c9f0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26d52b7740 1 --2- 172.21.15.67:0/2162001120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609abe1b0 0x563609e9c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26d52b7740 1 -- 172.21.15.67:0/2162001120 >> 172.21.15.67:0/2162001120 conn(0x563609adb460 msgr2=0x563609e88c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:59.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26d52b7740 1 -- 172.21.15.67:0/2162001120 shutdown_connections 2024-07-31T15:13:59.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26d52b7740 1 -- 172.21.15.67:0/2162001120 wait complete. 2024-07-31T15:13:59.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.334+0000 7f26d52b7740 20 client.20289 trim_cache size 0 max 16384 2024-07-31T15:13:59.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.348+0000 7f26d52b7740 1 Processor -- start 2024-07-31T15:13:59.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.348+0000 7f26d52b7740 1 -- start start 2024-07-31T15:13:59.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.348+0000 7f26d52b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609aa59f0 0x563609e89350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.348+0000 7f26d52b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609abe1b0 0x563609ea8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.348+0000 7f26d52b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ac6390 0x563609ea8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.348+0000 7f26d52b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f26c81ae640 con 0x563609ac6390 2024-07-31T15:13:59.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.348+0000 7f26d52b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f26c81b0180 con 0x563609aa59f0 2024-07-31T15:13:59.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.348+0000 7f26d52b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26c81aed10 con 0x563609abe1b0 2024-07-31T15:13:59.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.349+0000 7f26cd303640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ac6390 0x563609ea8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.349+0000 7f26cd303640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ac6390 0x563609ea8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46928/0 (socket says 172.21.15.67:46928) 2024-07-31T15:13:59.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.349+0000 7f26cd303640 1 -- 172.21.15.67:0/3839297755 learned_addr learned my addr 172.21.15.67:0/3839297755 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:59.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.349+0000 7f26c7fff640 1 --2- 172.21.15.67:0/3839297755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609abe1b0 0x563609ea8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.349+0000 7f26cd303640 1 --2- 172.21.15.67:0/3839297755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609aa59f0 0x563609e89350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.349+0000 7f26cd303640 1 -- 172.21.15.67:0/3839297755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609abe1b0 msgr2=0x563609ea8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:59.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.349+0000 7f26cd303640 1 --2- 172.21.15.67:0/3839297755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609abe1b0 0x563609ea8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.349+0000 7f26cd303640 1 -- 172.21.15.67:0/3839297755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609aa59f0 msgr2=0x563609e89350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:59.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.349+0000 7f26cd303640 1 --2- 172.21.15.67:0/3839297755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609aa59f0 0x563609e89350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.349+0000 7f26cd303640 1 -- 172.21.15.67:0/3839297755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563609e9cd60 con 0x563609ac6390 2024-07-31T15:13:59.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.349+0000 7f26cd303640 1 --2- 172.21.15.67:0/3839297755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ac6390 0x563609ea8770 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f26c000a920 tx=0x7f26c000a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6389e1c6d4658539 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:59.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.349+0000 7f26be7fc640 1 -- 172.21.15.67:0/3839297755 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c00043a0 con 0x563609ac6390 2024-07-31T15:13:59.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.349+0000 7f26be7fc640 1 -- 172.21.15.67:0/3839297755 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c0018c70 con 0x563609ac6390 2024-07-31T15:13:59.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.349+0000 7f26be7fc640 1 -- 172.21.15.67:0/3839297755 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0005530 con 0x563609ac6390 2024-07-31T15:13:59.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.350+0000 7f26d52b7740 1 -- 172.21.15.67:0/3839297755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ac6390 msgr2=0x563609ea8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:59.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.350+0000 7f26d52b7740 1 --2- 172.21.15.67:0/3839297755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ac6390 0x563609ea8770 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f26c000a920 tx=0x7f26c000a250 comp rx=0 tx=0).stop 2024-07-31T15:13:59.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.350+0000 7f26d52b7740 1 -- 172.21.15.67:0/3839297755 shutdown_connections 2024-07-31T15:13:59.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.350+0000 7f26d52b7740 1 --2- 172.21.15.67:0/3839297755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609ac6390 0x563609ea8770 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.350+0000 7f26d52b7740 1 --2- 172.21.15.67:0/3839297755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609abe1b0 0x563609ea8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.350+0000 7f26d52b7740 1 --2- 172.21.15.67:0/3839297755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609aa59f0 0x563609e89350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.350+0000 7f26d52b7740 1 -- 172.21.15.67:0/3839297755 >> 172.21.15.67:0/3839297755 conn(0x563609adb460 msgr2=0x563609e88c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:59.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.350+0000 7f26d52b7740 1 -- 172.21.15.67:0/3839297755 shutdown_connections 2024-07-31T15:13:59.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.350+0000 7f26d52b7740 1 -- 172.21.15.67:0/3839297755 wait complete. 2024-07-31T15:13:59.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.351+0000 7f26d52b7740 1 -- start start 2024-07-31T15:13:59.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.351+0000 7f26d52b7740 1 Processor -- start 2024-07-31T15:13:59.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.351+0000 7f26d52b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609aa59f0 0x563609ea4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.351+0000 7f26d52b7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609abe1b0 0x563609e9fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.351+0000 7f26d52b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609ac6390 0x563609ea0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.351+0000 7f26d52b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f26c81afab0 con 0x563609aa59f0 2024-07-31T15:13:59.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.351+0000 7f26d52b7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f26c81af3e0 con 0x563609abe1b0 2024-07-31T15:13:59.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.351+0000 7f26d52b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26c81ac640 con 0x563609ac6390 2024-07-31T15:13:59.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.351+0000 7f26d52b7740 10 client.0 _ll_register_callbacks cb 0x7f26b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:13:59.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.351+0000 7f26cd303640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609aa59f0 0x563609ea4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.351+0000 7f26cd303640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609aa59f0 0x563609ea4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46940/0 (socket says 172.21.15.67:46940) 2024-07-31T15:13:59.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.351+0000 7f26cd303640 1 -- 172.21.15.67:0/2415060472 learned_addr learned my addr 172.21.15.67:0/2415060472 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:13:59.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26c7fff640 1 --2- 172.21.15.67:0/2415060472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609abe1b0 0x563609e9fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26cd303640 1 --2- 172.21.15.67:0/2415060472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609ac6390 0x563609ea0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26cd303640 1 -- 172.21.15.67:0/2415060472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609ac6390 msgr2=0x563609ea0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:59.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26cd303640 1 --2- 172.21.15.67:0/2415060472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609ac6390 0x563609ea0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26cd303640 1 -- 172.21.15.67:0/2415060472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609abe1b0 msgr2=0x563609e9fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:13:59.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26cd303640 1 --2- 172.21.15.67:0/2415060472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609abe1b0 0x563609e9fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26cd303640 1 -- 172.21.15.67:0/2415060472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563609e96ee0 con 0x563609aa59f0 2024-07-31T15:13:59.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26cd303640 1 --2- 172.21.15.67:0/2415060472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609aa59f0 0x563609ea4940 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f26c0018840 tx=0x7f26c0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7f0cfe8c635d44 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:13:59.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:13:59.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26a7fff640 1 -- 172.21.15.67:0/2415060472 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c003c020 con 0x563609aa59f0 2024-07-31T15:13:59.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26a7fff640 1 -- 172.21.15.67:0/2415060472 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9395+0+0 (secure 0 0 0) 0x7f26c0004460 con 0x563609aa59f0 2024-07-31T15:13:59.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26d52b7740 1 -- 172.21.15.67:0/2415060472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c0018c70 con 0x563609aa59f0 2024-07-31T15:13:59.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26a7fff640 1 -- 172.21.15.67:0/2415060472 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f26c0004fb0 con 0x563609aa59f0 2024-07-31T15:13:59.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26d52b7740 10 client.20295 fetch_fsmap learned FSMap version 66 2024-07-31T15:13:59.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.352+0000 7f26d52b7740 1 -- 172.21.15.67:0/2415060472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x563609e8ea90 con 0x563609aa59f0 2024-07-31T15:13:59.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.353+0000 7f26a7fff640 1 -- 172.21.15.67:0/2415060472 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c0018c70 con 0x563609aa59f0 2024-07-31T15:13:59.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.353+0000 7f26a7fff640 1 -- 172.21.15.67:0/2415060472 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f26c0003c00 con 0x563609aa59f0 2024-07-31T15:13:59.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.353+0000 7f26d52b7740 10 client.20295 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:13:59.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.353+0000 7f26d52b7740 10 client.20295 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:13:59.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.353+0000 7f26d52b7740 20 client.20295 populate_metadata read hostname 'smithi067' 2024-07-31T15:13:59.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.353+0000 7f26d52b7740 1 --2- 172.21.15.67:0/2415060472 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563609ae7070 0x563609f18db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:13:59.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.353+0000 7f26d52b7740 4 client.20295 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:13:59.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.353+0000 7f26d52b7740 1 -- 172.21.15.67:0/2415060472 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563609e005a0 con 0x563609ae7070 2024-07-31T15:13:59.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.353+0000 7f26c7fff640 1 --2- 172.21.15.67:0/2415060472 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563609ae7070 0x563609f18db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:13:59.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26a7fff640 10 client.20295 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:13:59.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26c7fff640 1 --2- 172.21.15.67:0/2415060472 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563609ae7070 0x563609f18db0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6f2e6964e35b223 server_cookie=aa674dab9e9df20e in_seq=0 out_seq=0 2024-07-31T15:13:59.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26a7fff640 1 -- 172.21.15.67:0/2415060472 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563609e005a0 con 0x563609ae7070 2024-07-31T15:13:59.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26a7fff640 10 client.20295 handle_command_reply: tid=0 2024-07-31T15:13:59.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26d52b7740 1 client.20295 shutdown 2024-07-31T15:13:59.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26d52b7740 1 -- 172.21.15.67:0/2415060472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609aa59f0 msgr2=0x563609ea4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:59.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26d52b7740 1 --2- 172.21.15.67:0/2415060472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609aa59f0 0x563609ea4940 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f26c0018840 tx=0x7f26c0001e50 comp rx=0 tx=0).stop 2024-07-31T15:13:59.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26d52b7740 1 -- 172.21.15.67:0/2415060472 shutdown_connections 2024-07-31T15:13:59.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26d52b7740 1 --2- 172.21.15.67:0/2415060472 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x563609ae7070 0x563609f18db0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26d52b7740 1 --2- 172.21.15.67:0/2415060472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563609ac6390 0x563609ea0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26d52b7740 1 --2- 172.21.15.67:0/2415060472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563609abe1b0 0x563609e9fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26d52b7740 1 --2- 172.21.15.67:0/2415060472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563609aa59f0 0x563609ea4940 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26d52b7740 1 -- 172.21.15.67:0/2415060472 >> 172.21.15.67:0/2415060472 conn(0x563609adb460 msgr2=0x563609e9d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:59.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26a7fff640 0 client.20295 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:13:59.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26d52b7740 1 -- 172.21.15.67:0/2415060472 shutdown_connections 2024-07-31T15:13:59.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.354+0000 7f26d52b7740 1 -- 172.21.15.67:0/2415060472 wait complete. 2024-07-31T15:13:59.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.355+0000 7f26d52b7740 20 client.20295 trim_cache size 0 max 16384 2024-07-31T15:13:59.363 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:13:59.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.355+0000 7f26ced66640 1 -- 172.21.15.67:0/393444446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f26b40b3d70 msgr2=0x7f26b40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:59.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.355+0000 7f26ced66640 1 --2- 172.21.15.67:0/393444446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f26b40b3d70 0x7f26b40b6280 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x563609e8b1f0 tx=0x7f26b8005870 comp rx=0 tx=0).stop 2024-07-31T15:13:59.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.355+0000 7f26ced66640 1 -- 172.21.15.67:0/393444446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c8102b70 msgr2=0x7f26c819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:13:59.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.355+0000 7f26ced66640 1 --2- 172.21.15.67:0/393444446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c8102b70 0x7f26c819ae20 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f26b800eb90 tx=0x7f26b800c6a0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.355+0000 7f26ced66640 1 -- 172.21.15.67:0/393444446 shutdown_connections 2024-07-31T15:13:59.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.355+0000 7f26ced66640 1 --2- 172.21.15.67:0/393444446 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f26b40b3d70 0x7f26b40b6280 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.355+0000 7f26ced66640 1 --2- 172.21.15.67:0/393444446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26c81087a0 0x7f26c819b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.355+0000 7f26ced66640 1 --2- 172.21.15.67:0/393444446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26c8107e50 0x7f26c819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.355+0000 7f26ced66640 1 --2- 172.21.15.67:0/393444446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c8102b70 0x7f26c819ae20 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:13:59.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.355+0000 7f26ced66640 1 -- 172.21.15.67:0/393444446 >> 172.21.15.67:0/393444446 conn(0x7f26c8077690 msgr2=0x7f26c8103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:13:59.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.356+0000 7f26ced66640 1 -- 172.21.15.67:0/393444446 shutdown_connections 2024-07-31T15:13:59.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:13:59.356+0000 7f26ced66640 1 -- 172.21.15.67:0/393444446 wait complete. 2024-07-31T15:13:59.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:13:59 smithi073 ceph-mon[36714]: pgmap v1131: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-07-31T15:13:59.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:13:59 smithi077 ceph-mon[37327]: pgmap v1131: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-07-31T15:13:59.765 INFO:tasks.workunit.client.0.smithi067.stdout: LD vmlinux 2024-07-31T15:14:00.166 INFO:tasks.workunit.client.0.smithi067.stderr:ld: final link failed: Resource temporarily unavailable 2024-07-31T15:14:00.204 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-07-31T15:14:00.204 INFO:tasks.workunit.client.0.smithi067.stderr:make[2]: *** [scripts/Makefile.vmlinux:36: vmlinux] Error 1 2024-07-31T15:14:00.204 INFO:tasks.workunit.client.0.smithi067.stderr:make[1]: *** [/home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/Makefile:1250: vmlinux] Error 2 2024-07-31T15:14:00.205 INFO:tasks.workunit.client.0.smithi067.stderr:make: *** [Makefile:234: __sub-make] Error 2 2024-07-31T15:14:00.205 INFO:tasks.workunit:Stopping ['kernel_untar_build.sh'] on client.0... 2024-07-31T15:14:00.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-07-31T15:14:00.245 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T15:14:00.559 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:14:00.590 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_8c796fca5ff5960f7542e7227bad4a784ee2c804/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_8c796fca5ff5960f7542e7227bad4a784ee2c804/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_8c796fca5ff5960f7542e7227bad4a784ee2c804/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test kernel_untar_build.sh) on smithi067 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8c796fca5ff5960f7542e7227bad4a784ee2c804 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2024-07-31T15:14:00.770 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=631ab2ff4488427db1b0d3b92e8c3583 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_8c796fca5ff5960f7542e7227bad4a784ee2c804/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_8c796fca5ff5960f7542e7227bad4a784ee2c804/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_8c796fca5ff5960f7542e7227bad4a784ee2c804/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test kernel_untar_build.sh) on smithi067 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8c796fca5ff5960f7542e7227bad4a784ee2c804 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2024-07-31T15:14:00.773 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-07-31T15:14:00.786 INFO:tasks.quiescer:joining Quiescers 2024-07-31T15:14:00.787 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-07-31T15:14:00.787 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:14:01.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:01 smithi073 ceph-mon[36714]: pgmap v1132: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.6 MiB/s wr, 90 op/s 2024-07-31T15:14:01.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:01 smithi067 ceph-mon[32750]: pgmap v1132: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.6 MiB/s wr, 90 op/s 2024-07-31T15:14:01.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:01 smithi077 ceph-mon[37327]: pgmap v1132: 129 pgs: 129 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.6 MiB/s wr, 90 op/s 2024-07-31T15:14:01.917 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:14:02.225 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-31T15:14:02.225 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.95 2024-07-31T15:14:02.225 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:14:02.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:02 smithi067 ceph-mon[32750]: osdmap e135: 12 total, 12 up, 12 in 2024-07-31T15:14:02.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:02 smithi073 ceph-mon[36714]: osdmap e135: 12 total, 12 up, 12 in 2024-07-31T15:14:02.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:02 smithi077 ceph-mon[37327]: osdmap e135: 12 total, 12 up, 12 in 2024-07-31T15:14:02.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.626+0000 7f65b26f2640 1 -- 172.21.15.67:0/2101002600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65ac107550 msgr2=0x7f65ac107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.626+0000 7f65b26f2640 1 --2- 172.21.15.67:0/2101002600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65ac107550 0x7f65ac107960 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f65a001a910 tx=0x7f65a003d760 comp rx=0 tx=0).stop 2024-07-31T15:14:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.627+0000 7f65b26f2640 1 -- 172.21.15.67:0/2101002600 shutdown_connections 2024-07-31T15:14:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.627+0000 7f65b26f2640 1 --2- 172.21.15.67:0/2101002600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac107ea0 0x7f65ac10e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.627+0000 7f65b26f2640 1 --2- 172.21.15.67:0/2101002600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65ac107550 0x7f65ac107960 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.627+0000 7f65b26f2640 1 --2- 172.21.15.67:0/2101002600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65ac101c60 0x7f65ac102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.627+0000 7f65b26f2640 1 -- 172.21.15.67:0/2101002600 >> 172.21.15.67:0/2101002600 conn(0x7f65ac077690 msgr2=0x7f65ac077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:02.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.627+0000 7f65b26f2640 1 -- 172.21.15.67:0/2101002600 shutdown_connections 2024-07-31T15:14:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.627+0000 7f65b26f2640 1 -- 172.21.15.67:0/2101002600 wait complete. 2024-07-31T15:14:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.628+0000 7f65b26f2640 1 Processor -- start 2024-07-31T15:14:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.628+0000 7f65b26f2640 1 -- start start 2024-07-31T15:14:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.629+0000 7f65b26f2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac101c60 0x7f65ac10a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.629+0000 7f65b26f2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65ac107550 0x7f65ac10a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.629+0000 7f65b26f2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65ac107ea0 0x7f65ac116360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.629+0000 7f65b26f2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac072c20 con 0x7f65ac107ea0 2024-07-31T15:14:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.629+0000 7f65b26f2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac072aa0 con 0x7f65ac101c60 2024-07-31T15:14:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.629+0000 7f65b26f2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac072da0 con 0x7f65ac107550 2024-07-31T15:14:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.629+0000 7f65b0c8f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65ac107ea0 0x7f65ac116360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.629+0000 7f65b0c8f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65ac107ea0 0x7f65ac116360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51256/0 (socket says 172.21.15.67:51256) 2024-07-31T15:14:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.629+0000 7f65b0c8f640 1 -- 172.21.15.67:0/4138475896 learned_addr learned my addr 172.21.15.67:0/4138475896 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.629+0000 7f65abfff640 1 --2- 172.21.15.67:0/4138475896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac101c60 0x7f65ac10a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.631+0000 7f65b0c8f640 1 -- 172.21.15.67:0/4138475896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65ac107550 msgr2=0x7f65ac10a6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-31T15:14:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.631+0000 7f65b0c8f640 1 --2- 172.21.15.67:0/4138475896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65ac107550 0x7f65ac10a6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.631+0000 7f65b0c8f640 1 -- 172.21.15.67:0/4138475896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac101c60 msgr2=0x7f65ac10a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.631+0000 7f65b0c8f640 1 --2- 172.21.15.67:0/4138475896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac101c60 0x7f65ac10a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.631+0000 7f65b0c8f640 1 -- 172.21.15.67:0/4138475896 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac116950 con 0x7f65ac107ea0 2024-07-31T15:14:02.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.650+0000 7f65b0c8f640 1 --2- 172.21.15.67:0/4138475896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65ac107ea0 0x7f65ac116360 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f65a400cad0 tx=0x7f65a400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:02.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.654+0000 7f65a97fa640 1 -- 172.21.15.67:0/4138475896 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a400ae20 con 0x7f65ac107ea0 2024-07-31T15:14:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.654+0000 7f65a97fa640 1 -- 172.21.15.67:0/4138475896 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a4005020 con 0x7f65ac107ea0 2024-07-31T15:14:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.656+0000 7f65b26f2640 1 -- 172.21.15.67:0/4138475896 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ac116c40 con 0x7f65ac107ea0 2024-07-31T15:14:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.656+0000 7f65a97fa640 1 -- 172.21.15.67:0/4138475896 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a400ae20 con 0x7f65ac107ea0 2024-07-31T15:14:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.656+0000 7f65b26f2640 1 -- 172.21.15.67:0/4138475896 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ac1171b0 con 0x7f65ac107ea0 2024-07-31T15:14:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.660+0000 7f65b8c43740 1 Processor -- start 2024-07-31T15:14:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.665+0000 7f65b8c43740 1 -- start start 2024-07-31T15:14:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.665+0000 7f65a97fa640 1 -- 172.21.15.67:0/4138475896 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f65a40051d0 con 0x7f65ac107ea0 2024-07-31T15:14:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.667+0000 7f65a97fa640 1 --2- 172.21.15.67:0/4138475896 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65940b41c0 0x7f65940b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.667+0000 7f65a97fa640 1 -- 172.21.15.67:0/4138475896 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9483+0+0 (secure 0 0 0) 0x7f65a400ac20 con 0x7f65ac107ea0 2024-07-31T15:14:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.668+0000 7f65b8c43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8af6070 0x55d3f8e9b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.668+0000 7f65b8c43740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8acd1b0 0x55d3f8e9bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.668+0000 7f65b8c43740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8ad5390 0x55d3f8e9bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.669+0000 7f65b8c43740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d3f8e11bb0 con 0x55d3f8ad5390 2024-07-31T15:14:02.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.669+0000 7f65b8c43740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d3f8e010b0 con 0x55d3f8acd1b0 2024-07-31T15:14:02.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.669+0000 7f65b8c43740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d3f8e050f0 con 0x55d3f8af6070 2024-07-31T15:14:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.669+0000 7f65abfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8af6070 0x55d3f8e9b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.669+0000 7f65abfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8af6070 0x55d3f8e9b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:52358/0 (socket says 172.21.15.67:52358) 2024-07-31T15:14:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.669+0000 7f65abfff640 1 -- 172.21.15.67:0/3861985140 learned_addr learned my addr 172.21.15.67:0/3861985140 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.669+0000 7f65ab7fe640 1 --2- 172.21.15.67:0/4138475896 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65940b41c0 0x7f65940b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.669+0000 7f65ab7fe640 1 --2- 172.21.15.67:0/3861985140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8acd1b0 0x55d3f8e9bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.670+0000 7f65abfff640 1 -- 172.21.15.67:0/3861985140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8acd1b0 msgr2=0x55d3f8e9bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.670+0000 7f65abfff640 1 --2- 172.21.15.67:0/3861985140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8acd1b0 0x55d3f8e9bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.670+0000 7f65abfff640 1 -- 172.21.15.67:0/3861985140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8ad5390 msgr2=0x55d3f8e9bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.670+0000 7f65abfff640 1 --2- 172.21.15.67:0/3861985140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8ad5390 0x55d3f8e9bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.670+0000 7f65abfff640 1 -- 172.21.15.67:0/3861985140 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f8da7b90 con 0x55d3f8af6070 2024-07-31T15:14:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.670+0000 7f65ab7fe640 1 --2- 172.21.15.67:0/4138475896 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65940b41c0 0x7f65940b66d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f65ac1066f0 tx=0x7f65a00200c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.670+0000 7f65abfff640 1 --2- 172.21.15.67:0/3861985140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8af6070 0x55d3f8e9b570 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f659c009a10 tx=0x7f659c009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3b699198323a71d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:02.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.674+0000 7f65927fc640 1 -- 172.21.15.67:0/3861985140 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c017070 con 0x55d3f8af6070 2024-07-31T15:14:02.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.674+0000 7f65927fc640 1 -- 172.21.15.67:0/3861985140 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f659c013470 con 0x55d3f8af6070 2024-07-31T15:14:02.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.674+0000 7f65927fc640 1 -- 172.21.15.67:0/3861985140 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c012630 con 0x55d3f8af6070 2024-07-31T15:14:02.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.676+0000 7f65b8c43740 1 -- 172.21.15.67:0/3861985140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8af6070 msgr2=0x55d3f8e9b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:02.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.676+0000 7f65b8c43740 1 --2- 172.21.15.67:0/3861985140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8af6070 0x55d3f8e9b570 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f659c009a10 tx=0x7f659c009ed0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.685+0000 7f65b8c43740 1 -- 172.21.15.67:0/3861985140 shutdown_connections 2024-07-31T15:14:02.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.685+0000 7f65b8c43740 1 --2- 172.21.15.67:0/3861985140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8ad5390 0x55d3f8e9bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.685+0000 7f65b8c43740 1 --2- 172.21.15.67:0/3861985140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8acd1b0 0x55d3f8e9bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.685+0000 7f65b8c43740 1 --2- 172.21.15.67:0/3861985140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8af6070 0x55d3f8e9b570 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.685+0000 7f65b8c43740 1 -- 172.21.15.67:0/3861985140 >> 172.21.15.67:0/3861985140 conn(0x55d3f8aea460 msgr2=0x55d3f8e99460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:02.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.689+0000 7f65b8c43740 1 -- 172.21.15.67:0/3861985140 shutdown_connections 2024-07-31T15:14:02.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.690+0000 7f65b8c43740 1 -- 172.21.15.67:0/3861985140 wait complete. 2024-07-31T15:14:02.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.691+0000 7f65b8c43740 1 -- start start 2024-07-31T15:14:02.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.692+0000 7f65b8c43740 1 Processor -- start 2024-07-31T15:14:02.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.692+0000 7f65b8c43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8acd1b0 0x55d3f8ea7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.693+0000 7f65b8c43740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8ad5390 0x55d3f8ea76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.693+0000 7f65b8c43740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8af6070 0x55d3f8eb30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.693+0000 7f65b8c43740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d3f8c64700 con 0x55d3f8ad5390 2024-07-31T15:14:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.693+0000 7f65b8c43740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d3f8da39e0 con 0x55d3f8af6070 2024-07-31T15:14:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.693+0000 7f65b8c43740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d3f8ea9d50 con 0x55d3f8acd1b0 2024-07-31T15:14:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.693+0000 7f65b8c43740 10 client.0 _ll_register_callbacks cb 0x55d3f8dbe4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:02.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.697+0000 7f65abfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8acd1b0 0x55d3f8ea7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.697+0000 7f65ab7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8ad5390 0x55d3f8ea76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.697+0000 7f65ab7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8ad5390 0x55d3f8ea76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51274/0 (socket says 172.21.15.67:51274) 2024-07-31T15:14:02.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.697+0000 7f65ab7fe640 1 -- 172.21.15.67:0/1227163946 learned_addr learned my addr 172.21.15.67:0/1227163946 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:02.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.698+0000 7f65b0c8f640 1 --2- 172.21.15.67:0/1227163946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8af6070 0x55d3f8eb30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.706+0000 7f65b0c8f640 1 -- 172.21.15.67:0/1227163946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8acd1b0 msgr2=0x55d3f8ea7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:02.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.706+0000 7f65b0c8f640 1 --2- 172.21.15.67:0/1227163946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8acd1b0 0x55d3f8ea7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.706+0000 7f65b0c8f640 1 -- 172.21.15.67:0/1227163946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8ad5390 msgr2=0x55d3f8ea76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:02.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.706+0000 7f65b0c8f640 1 --2- 172.21.15.67:0/1227163946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8ad5390 0x55d3f8ea76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.706+0000 7f65b0c8f640 1 -- 172.21.15.67:0/1227163946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f8b61650 con 0x55d3f8af6070 2024-07-31T15:14:02.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.709+0000 7f65b0c8f640 1 --2- 172.21.15.67:0/1227163946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8af6070 0x55d3f8eb30a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f65a40e0a50 tx=0x7f65a40e0fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e38e4daf98eeb04 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:02.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.710+0000 7f658b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:14:02.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.714+0000 7f658b7fe640 1 -- 172.21.15.67:0/1227163946 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40c4060 con 0x55d3f8af6070 2024-07-31T15:14:02.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.714+0000 7f658b7fe640 1 -- 172.21.15.67:0/1227163946 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9483+0+0 (secure 0 0 0) 0x7f65a40e3490 con 0x55d3f8af6070 2024-07-31T15:14:02.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.715+0000 7f65b8c43740 1 -- 172.21.15.67:0/1227163946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f659c013470 con 0x55d3f8af6070 2024-07-31T15:14:02.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.717+0000 7f658b7fe640 1 -- 172.21.15.67:0/1227163946 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f65a40ae360 con 0x55d3f8af6070 2024-07-31T15:14:02.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.717+0000 7f65b8c43740 10 client.29810 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:02.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.717+0000 7f65b8c43740 1 -- 172.21.15.67:0/1227163946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55d3f8eb5e50 con 0x55d3f8af6070 2024-07-31T15:14:02.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.719+0000 7f658b7fe640 1 -- 172.21.15.67:0/1227163946 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f659c013470 con 0x55d3f8af6070 2024-07-31T15:14:02.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.719+0000 7f658b7fe640 1 -- 172.21.15.67:0/1227163946 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f65a40a8c70 con 0x55d3f8af6070 2024-07-31T15:14:02.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.720+0000 7f65b8c43740 10 client.29810 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.720+0000 7f65b8c43740 10 client.29810 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:14:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.720+0000 7f65b8c43740 20 client.29810 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.720+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1227163946 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55d3f8ab49f0 0x55d3f8eb9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.720+0000 7f65b8c43740 4 client.29810 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:14:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.720+0000 7f65b8c43740 1 -- 172.21.15.67:0/1227163946 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3f8e13b00 con 0x55d3f8ab49f0 2024-07-31T15:14:02.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.728+0000 7f65abfff640 1 --2- 172.21.15.67:0/1227163946 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55d3f8ab49f0 0x55d3f8eb9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.740+0000 7f65abfff640 1 --2- 172.21.15.67:0/1227163946 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55d3f8ab49f0 0x55d3f8eb9680 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8d5793062b8eee42 server_cookie=60a945730f7997a3 in_seq=0 out_seq=0 2024-07-31T15:14:02.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.741+0000 7f658b7fe640 10 client.29810 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:14:02.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.745+0000 7f658b7fe640 1 -- 172.21.15.67:0/1227163946 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3f8e13b00 con 0x55d3f8ab49f0 2024-07-31T15:14:02.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.745+0000 7f658b7fe640 10 client.29810 handle_command_reply: tid=0 2024-07-31T15:14:02.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.745+0000 7f65b8c43740 1 client.29810 shutdown 2024-07-31T15:14:02.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.746+0000 7f65b8c43740 1 -- 172.21.15.67:0/1227163946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8af6070 msgr2=0x55d3f8eb30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:02.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.746+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1227163946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8af6070 0x55d3f8eb30a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f65a40e0a50 tx=0x7f65a40e0fd0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.747+0000 7f65b8c43740 1 -- 172.21.15.67:0/1227163946 shutdown_connections 2024-07-31T15:14:02.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.747+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1227163946 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55d3f8ab49f0 0x55d3f8eb9680 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.747+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1227163946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8af6070 0x55d3f8eb30a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.747+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1227163946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8ad5390 0x55d3f8ea76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.747+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1227163946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8acd1b0 0x55d3f8ea7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.747+0000 7f65b8c43740 1 -- 172.21.15.67:0/1227163946 >> 172.21.15.67:0/1227163946 conn(0x55d3f8aea460 msgr2=0x55d3f8e97c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:02.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.747+0000 7f658b7fe640 0 client.29810 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:14:02.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.749+0000 7f65b8c43740 1 -- 172.21.15.67:0/1227163946 shutdown_connections 2024-07-31T15:14:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.749+0000 7f65b8c43740 1 -- 172.21.15.67:0/1227163946 wait complete. 2024-07-31T15:14:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.750+0000 7f65b8c43740 20 client.29810 trim_cache size 0 max 16384 2024-07-31T15:14:02.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.780+0000 7f65b8c43740 1 Processor -- start 2024-07-31T15:14:02.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.781+0000 7f65b8c43740 1 -- start start 2024-07-31T15:14:02.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.781+0000 7f65b8c43740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8acd1b0 0x55d3f8e98350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.781+0000 7f65b8c43740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8ad5390 0x55d3f8eb2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.781+0000 7f65b8c43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8af6070 0x55d3f8ea0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.781+0000 7f65b8c43740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1b7120 con 0x55d3f8acd1b0 2024-07-31T15:14:02.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.781+0000 7f65b8c43740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1b8c60 con 0x55d3f8ad5390 2024-07-31T15:14:02.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.781+0000 7f65b8c43740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1b77f0 con 0x55d3f8af6070 2024-07-31T15:14:02.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.783+0000 7f65b0c8f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8af6070 0x55d3f8ea0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.783+0000 7f65ab7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8ad5390 0x55d3f8eb2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.784+0000 7f65b0c8f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8af6070 0x55d3f8ea0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:52388/0 (socket says 172.21.15.67:52388) 2024-07-31T15:14:02.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.784+0000 7f65b0c8f640 1 -- 172.21.15.67:0/1007982706 learned_addr learned my addr 172.21.15.67:0/1007982706 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:02.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.785+0000 7f65abfff640 1 --2- 172.21.15.67:0/1007982706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8acd1b0 0x55d3f8e98350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.785+0000 7f65abfff640 1 -- 172.21.15.67:0/1007982706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8af6070 msgr2=0x55d3f8ea0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:02.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.785+0000 7f65abfff640 1 --2- 172.21.15.67:0/1007982706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8af6070 0x55d3f8ea0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.785+0000 7f65abfff640 1 -- 172.21.15.67:0/1007982706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8ad5390 msgr2=0x55d3f8eb2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:02.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.785+0000 7f65abfff640 1 --2- 172.21.15.67:0/1007982706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8ad5390 0x55d3f8eb2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.785+0000 7f65abfff640 1 -- 172.21.15.67:0/1007982706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f8e9d860 con 0x55d3f8acd1b0 2024-07-31T15:14:02.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.786+0000 7f65abfff640 1 --2- 172.21.15.67:0/1007982706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8acd1b0 0x55d3f8e98350 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f659c013dd0 tx=0x7f659c009270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698e636ae00a3273 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:02.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.787+0000 7f6591ffb640 1 -- 172.21.15.67:0/1007982706 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c017040 con 0x55d3f8acd1b0 2024-07-31T15:14:02.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.787+0000 7f6591ffb640 1 -- 172.21.15.67:0/1007982706 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f659c00fdd0 con 0x55d3f8acd1b0 2024-07-31T15:14:02.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.787+0000 7f6591ffb640 1 -- 172.21.15.67:0/1007982706 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c0136d0 con 0x55d3f8acd1b0 2024-07-31T15:14:02.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.788+0000 7f65b8c43740 1 -- 172.21.15.67:0/1007982706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8acd1b0 msgr2=0x55d3f8e98350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:02.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.788+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1007982706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8acd1b0 0x55d3f8e98350 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f659c013dd0 tx=0x7f659c009270 comp rx=0 tx=0).stop 2024-07-31T15:14:02.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.792+0000 7f65b8c43740 1 -- 172.21.15.67:0/1007982706 shutdown_connections 2024-07-31T15:14:02.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.792+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1007982706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8af6070 0x55d3f8ea0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.792+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1007982706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8ad5390 0x55d3f8eb2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.792+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1007982706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8acd1b0 0x55d3f8e98350 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.792+0000 7f65b8c43740 1 -- 172.21.15.67:0/1007982706 >> 172.21.15.67:0/1007982706 conn(0x55d3f8aea460 msgr2=0x55d3f8e97c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:02.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.794+0000 7f65b8c43740 1 -- 172.21.15.67:0/1007982706 shutdown_connections 2024-07-31T15:14:02.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.794+0000 7f65b8c43740 1 -- 172.21.15.67:0/1007982706 wait complete. 2024-07-31T15:14:02.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.795+0000 7f65b8c43740 1 -- start start 2024-07-31T15:14:02.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.795+0000 7f65b8c43740 1 Processor -- start 2024-07-31T15:14:02.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.796+0000 7f65b8c43740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8acd1b0 0x55d3f8eaf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.796+0000 7f65b8c43740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8ad5390 0x55d3f8eaace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.796+0000 7f65b8c43740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8af6070 0x55d3f8eab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.797+0000 7f65b8c43740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1b8590 con 0x55d3f8af6070 2024-07-31T15:14:02.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.797+0000 7f65abfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8acd1b0 0x55d3f8eaf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.797+0000 7f65abfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8acd1b0 0x55d3f8eaf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:56930/0 (socket says 172.21.15.67:56930) 2024-07-31T15:14:02.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.797+0000 7f65abfff640 1 -- 172.21.15.67:0/1767251934 learned_addr learned my addr 172.21.15.67:0/1767251934 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:02.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.797+0000 7f65b8c43740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1b7ec0 con 0x55d3f8acd1b0 2024-07-31T15:14:02.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.797+0000 7f65b8c43740 1 -- 172.21.15.67:0/1767251934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1b5120 con 0x55d3f8ad5390 2024-07-31T15:14:02.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.797+0000 7f65b8c43740 10 client.0 _ll_register_callbacks cb 0x7f65a40a5f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:02.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.797+0000 7f65ab7fe640 1 --2- 172.21.15.67:0/1767251934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8ad5390 0x55d3f8eaace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.803+0000 7f65b0c8f640 1 --2- 172.21.15.67:0/1767251934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8af6070 0x55d3f8eab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.804+0000 7f65abfff640 1 -- 172.21.15.67:0/1767251934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8ad5390 msgr2=0x55d3f8eaace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:02.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.804+0000 7f65abfff640 1 --2- 172.21.15.67:0/1767251934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8ad5390 0x55d3f8eaace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.804+0000 7f65abfff640 1 -- 172.21.15.67:0/1767251934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8af6070 msgr2=0x55d3f8eab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:02.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.804+0000 7f65abfff640 1 --2- 172.21.15.67:0/1767251934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8af6070 0x55d3f8eab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.804+0000 7f65abfff640 1 -- 172.21.15.67:0/1767251934 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f8eabb10 con 0x55d3f8acd1b0 2024-07-31T15:14:02.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.811+0000 7f65abfff640 1 --2- 172.21.15.67:0/1767251934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8acd1b0 0x55d3f8eaf830 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f659c004280 tx=0x7f659c0042b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e618beaf2cd7ccf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:02.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.814+0000 7f658b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:14:02.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.814+0000 7f658b7fe640 1 -- 172.21.15.67:0/1767251934 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c034060 con 0x55d3f8acd1b0 2024-07-31T15:14:02.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.814+0000 7f65b8c43740 1 -- 172.21.15.67:0/1767251934 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f659c00fdd0 con 0x55d3f8acd1b0 2024-07-31T15:14:02.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.814+0000 7f658b7fe640 1 -- 172.21.15.67:0/1767251934 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9483+0+0 (secure 0 0 0) 0x7f659c004450 con 0x55d3f8acd1b0 2024-07-31T15:14:02.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.821+0000 7f658b7fe640 1 -- 172.21.15.67:0/1767251934 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f659c017040 con 0x55d3f8acd1b0 2024-07-31T15:14:02.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.824+0000 7f65b8c43740 10 client.29816 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:02.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.824+0000 7f65b8c43740 1 -- 172.21.15.67:0/1767251934 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55d3f8f21ee0 con 0x55d3f8acd1b0 2024-07-31T15:14:02.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.827+0000 7f658b7fe640 1 -- 172.21.15.67:0/1767251934 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f659c00fdd0 con 0x55d3f8acd1b0 2024-07-31T15:14:02.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.827+0000 7f658b7fe640 1 -- 172.21.15.67:0/1767251934 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f659c0038d0 con 0x55d3f8acd1b0 2024-07-31T15:14:02.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.828+0000 7f65b8c43740 10 client.29816 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:02.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.828+0000 7f65b8c43740 10 client.29816 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:14:02.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.828+0000 7f65b8c43740 20 client.29816 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:02.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.828+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1767251934 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55d3f8a89da0 0x55d3f8f244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:02.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.828+0000 7f65b8c43740 4 client.29816 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:14:02.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.828+0000 7f65b8c43740 1 -- 172.21.15.67:0/1767251934 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3f8e0f5a0 con 0x55d3f8a89da0 2024-07-31T15:14:02.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.832+0000 7f65ab7fe640 1 --2- 172.21.15.67:0/1767251934 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55d3f8a89da0 0x55d3f8f244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:02.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.836+0000 7f65ab7fe640 1 --2- 172.21.15.67:0/1767251934 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55d3f8a89da0 0x55d3f8f244c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8438bc1740a876d1 server_cookie=6a2efb37bc23b9b4 in_seq=0 out_seq=0 2024-07-31T15:14:02.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.837+0000 7f658b7fe640 10 client.29816 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:14:02.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.837+0000 7f658b7fe640 1 -- 172.21.15.67:0/1767251934 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3f8e0f5a0 con 0x55d3f8a89da0 2024-07-31T15:14:02.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.837+0000 7f658b7fe640 10 client.29816 handle_command_reply: tid=0 2024-07-31T15:14:02.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.837+0000 7f65b8c43740 1 client.29816 shutdown 2024-07-31T15:14:02.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.841+0000 7f65b8c43740 1 -- 172.21.15.67:0/1767251934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8acd1b0 msgr2=0x55d3f8eaf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.841+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1767251934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8acd1b0 0x55d3f8eaf830 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f659c004280 tx=0x7f659c0042b0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.841+0000 7f65b8c43740 1 -- 172.21.15.67:0/1767251934 shutdown_connections 2024-07-31T15:14:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.841+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1767251934 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x55d3f8a89da0 0x55d3f8f244c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.841+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1767251934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d3f8af6070 0x55d3f8eab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.841+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1767251934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3f8ad5390 0x55d3f8eaace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.841+0000 7f65b8c43740 1 --2- 172.21.15.67:0/1767251934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3f8acd1b0 0x55d3f8eaf830 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.841+0000 7f65b8c43740 1 -- 172.21.15.67:0/1767251934 >> 172.21.15.67:0/1767251934 conn(0x55d3f8aea460 msgr2=0x55d3f8e98cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:02.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.841+0000 7f658b7fe640 0 client.29816 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:14:02.846 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:14:02.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.843+0000 7f65b8c43740 1 -- 172.21.15.67:0/1767251934 shutdown_connections 2024-07-31T15:14:02.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.843+0000 7f65b8c43740 1 -- 172.21.15.67:0/1767251934 wait complete. 2024-07-31T15:14:02.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.843+0000 7f65b8c43740 20 client.29816 trim_cache size 0 max 16384 2024-07-31T15:14:02.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.846+0000 7f65b26f2640 1 -- 172.21.15.67:0/4138475896 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65940b41c0 msgr2=0x7f65940b66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:02.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.846+0000 7f65b26f2640 1 --2- 172.21.15.67:0/4138475896 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65940b41c0 0x7f65940b66d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f65ac1066f0 tx=0x7f65a00200c0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.846+0000 7f65b26f2640 1 -- 172.21.15.67:0/4138475896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65ac107ea0 msgr2=0x7f65ac116360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:02.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.846+0000 7f65b26f2640 1 --2- 172.21.15.67:0/4138475896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65ac107ea0 0x7f65ac116360 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f65a400cad0 tx=0x7f65a400cf90 comp rx=0 tx=0).stop 2024-07-31T15:14:02.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.848+0000 7f65b26f2640 1 -- 172.21.15.67:0/4138475896 shutdown_connections 2024-07-31T15:14:02.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.848+0000 7f65b26f2640 1 --2- 172.21.15.67:0/4138475896 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f65940b41c0 0x7f65940b66d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.848+0000 7f65b26f2640 1 --2- 172.21.15.67:0/4138475896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65ac107ea0 0x7f65ac116360 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.848+0000 7f65b26f2640 1 --2- 172.21.15.67:0/4138475896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65ac107550 0x7f65ac10a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.848+0000 7f65b26f2640 1 --2- 172.21.15.67:0/4138475896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac101c60 0x7f65ac10a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:02.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.848+0000 7f65b26f2640 1 -- 172.21.15.67:0/4138475896 >> 172.21.15.67:0/4138475896 conn(0x7f65ac077690 msgr2=0x7f65ac1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:02.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.848+0000 7f65b26f2640 1 -- 172.21.15.67:0/4138475896 shutdown_connections 2024-07-31T15:14:02.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:02.848+0000 7f65b26f2640 1 -- 172.21.15.67:0/4138475896 wait complete. 2024-07-31T15:14:03.190 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:03 smithi067 ceph-mon[32750]: pgmap v1134: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.8 MiB/s wr, 128 op/s 2024-07-31T15:14:03.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:03 smithi077 ceph-mon[37327]: pgmap v1134: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.8 MiB/s wr, 128 op/s 2024-07-31T15:14:03.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:03 smithi073 ceph-mon[36714]: pgmap v1134: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.8 MiB/s wr, 128 op/s 2024-07-31T15:14:04.953 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T15:14:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.211+0000 7f05aadcd640 1 -- 172.21.15.67:0/2031546413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a4107dd0 msgr2=0x7f05a41081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.211+0000 7f05aadcd640 1 --2- 172.21.15.67:0/2031546413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a4107dd0 0x7f05a41081e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0594009900 tx=0x7f059402a3f0 comp rx=0 tx=0).stop 2024-07-31T15:14:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.211+0000 7f05aadcd640 1 -- 172.21.15.67:0/2031546413 shutdown_connections 2024-07-31T15:14:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.211+0000 7f05aadcd640 1 --2- 172.21.15.67:0/2031546413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a4108720 0x7f05a410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.211+0000 7f05aadcd640 1 --2- 172.21.15.67:0/2031546413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a4107dd0 0x7f05a41081e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.211+0000 7f05aadcd640 1 --2- 172.21.15.67:0/2031546413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05a4102490 0x7f05a41028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.211+0000 7f05aadcd640 1 -- 172.21.15.67:0/2031546413 >> 172.21.15.67:0/2031546413 conn(0x7f05a4079160 msgr2=0x7f05a4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.212+0000 7f05aadcd640 1 -- 172.21.15.67:0/2031546413 shutdown_connections 2024-07-31T15:14:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.212+0000 7f05aadcd640 1 -- 172.21.15.67:0/2031546413 wait complete. 2024-07-31T15:14:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.213+0000 7f05aadcd640 1 Processor -- start 2024-07-31T15:14:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.213+0000 7f05aadcd640 1 -- start start 2024-07-31T15:14:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.213+0000 7f05aadcd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a4102490 0x7f05a410f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.213+0000 7f05aadcd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05a4107dd0 0x7f05a410fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.213+0000 7f05aadcd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a4108720 0x7f05a410ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.213+0000 7f05aadcd640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41134f0 con 0x7f05a4107dd0 2024-07-31T15:14:05.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.213+0000 7f05aadcd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4113370 con 0x7f05a4102490 2024-07-31T15:14:05.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.213+0000 7f05aadcd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4113670 con 0x7f05a4108720 2024-07-31T15:14:05.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.213+0000 7f05a936a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a4108720 0x7f05a410ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:05.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.213+0000 7f05a936a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a4108720 0x7f05a410ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:52416/0 (socket says 172.21.15.67:52416) 2024-07-31T15:14:05.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.213+0000 7f05a936a640 1 -- 172.21.15.67:0/1465041424 learned_addr learned my addr 172.21.15.67:0/1465041424 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:05.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.213+0000 7f05a8b69640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a4102490 0x7f05a410f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:05.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.214+0000 7f05a3fff640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05a4107dd0 0x7f05a410fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:05.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.214+0000 7f05a8b69640 1 -- 172.21.15.67:0/1465041424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a4108720 msgr2=0x7f05a410ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:05.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.214+0000 7f05a8b69640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a4108720 0x7f05a410ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:05.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.214+0000 7f05a8b69640 1 -- 172.21.15.67:0/1465041424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05a4107dd0 msgr2=0x7f05a410fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:05.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.214+0000 7f05a8b69640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05a4107dd0 0x7f05a410fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:05.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.214+0000 7f05a8b69640 1 -- 172.21.15.67:0/1465041424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a4073c80 con 0x7f05a4102490 2024-07-31T15:14:05.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.214+0000 7f05a3fff640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05a4107dd0 0x7f05a410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:14:05.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.214+0000 7f05a8b69640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a4102490 0x7f05a410f4c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f059c00dae0 tx=0x7f059c00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.214+0000 7f05a1ffb640 1 -- 172.21.15.67:0/1465041424 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c017030 con 0x7f05a4102490 2024-07-31T15:14:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.215+0000 7f05a1ffb640 1 -- 172.21.15.67:0/1465041424 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f059c003e20 con 0x7f05a4102490 2024-07-31T15:14:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.215+0000 7f05a1ffb640 1 -- 172.21.15.67:0/1465041424 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c010460 con 0x7f05a4102490 2024-07-31T15:14:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.215+0000 7f05aadcd640 1 -- 172.21.15.67:0/1465041424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a4073f70 con 0x7f05a4102490 2024-07-31T15:14:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.215+0000 7f05aadcd640 1 -- 172.21.15.67:0/1465041424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a4074460 con 0x7f05a4102490 2024-07-31T15:14:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.217+0000 7f05aadcd640 1 -- 172.21.15.67:0/1465041424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a41028a0 con 0x7f05a4102490 2024-07-31T15:14:05.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.219+0000 7f05a1ffb640 1 -- 172.21.15.67:0/1465041424 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f059c01a070 con 0x7f05a4102490 2024-07-31T15:14:05.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.220+0000 7f05a1ffb640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f058c0b3e90 0x7f058c0b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:05.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.220+0000 7f05a1ffb640 1 -- 172.21.15.67:0/1465041424 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9483+0+0 (secure 0 0 0) 0x7f059c0e40e0 con 0x7f05a4102490 2024-07-31T15:14:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.221+0000 7f05a3fff640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f058c0b3e90 0x7f058c0b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.222+0000 7f05a3fff640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f058c0b3e90 0x7f058c0b63a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f059402a910 tx=0x7f0594005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.222+0000 7f05a1ffb640 1 -- 172.21.15.67:0/1465041424 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f059c0ab130 con 0x7f05a4102490 2024-07-31T15:14:05.299 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:14:05.444 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:05 smithi073 ceph-mon[36714]: pgmap v1135: 129 pgs: 16 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 74 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 12 MiB/s wr, 107 op/s 2024-07-31T15:14:05.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:05 smithi067 ceph-mon[32750]: pgmap v1135: 129 pgs: 16 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 74 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 12 MiB/s wr, 107 op/s 2024-07-31T15:14:05.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:05 smithi077 ceph-mon[37327]: pgmap v1135: 129 pgs: 16 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 74 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 12 MiB/s wr, 107 op/s 2024-07-31T15:14:05.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.672+0000 7f05aadcd640 1 -- 172.21.15.67:0/1465041424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05a41107c0 con 0x7f05a4102490 2024-07-31T15:14:05.677 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:14:05.677 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":66,"btime":"2024-07-31T15:10:30:235518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:10:30.235494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29129":{"gid":29129,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/631186801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":631186801},{"type":"v1","addr":"172.21.15.73:6835","nonce":631186801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:14:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.674+0000 7f05a1ffb640 1 -- 172.21.15.67:0/1465041424 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11959 (secure 0 0 0) 0x7f05a41107c0 con 0x7f05a4102490 2024-07-31T15:14:05.678 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 66 2024-07-31T15:14:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.677+0000 7f058b7fe640 1 -- 172.21.15.67:0/1465041424 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f058c0b3e90 msgr2=0x7f058c0b63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.677+0000 7f058b7fe640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f058c0b3e90 0x7f058c0b63a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f059402a910 tx=0x7f0594005fd0 comp rx=0 tx=0).stop 2024-07-31T15:14:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.677+0000 7f058b7fe640 1 -- 172.21.15.67:0/1465041424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a4102490 msgr2=0x7f05a410f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.677+0000 7f058b7fe640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a4102490 0x7f05a410f4c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f059c00dae0 tx=0x7f059c00b260 comp rx=0 tx=0).stop 2024-07-31T15:14:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.678+0000 7f058b7fe640 1 -- 172.21.15.67:0/1465041424 shutdown_connections 2024-07-31T15:14:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.678+0000 7f058b7fe640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f058c0b3e90 0x7f058c0b63a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.678+0000 7f058b7fe640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a4108720 0x7f05a410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.678+0000 7f058b7fe640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05a4107dd0 0x7f05a410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.678+0000 7f058b7fe640 1 --2- 172.21.15.67:0/1465041424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a4102490 0x7f05a410f4c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.678+0000 7f058b7fe640 1 -- 172.21.15.67:0/1465041424 >> 172.21.15.67:0/1465041424 conn(0x7f05a4079160 msgr2=0x7f05a4103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.678+0000 7f058b7fe640 1 -- 172.21.15.67:0/1465041424 shutdown_connections 2024-07-31T15:14:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:05.678+0000 7f058b7fe640 1 -- 172.21.15.67:0/1465041424 wait complete. 2024-07-31T15:14:06.336 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:14:06.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:06 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1465041424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:14:06.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:06 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1465041424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:14:06.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:06 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1465041424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:14:06.791 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db --cancel --all' 2024-07-31T15:14:06.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db --cancel --all 2024-07-31T15:14:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:07 smithi077 ceph-mon[37327]: pgmap v1136: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 17 MiB/s wr, 99 op/s 2024-07-31T15:14:07.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:07 smithi067 ceph-mon[32750]: pgmap v1136: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 17 MiB/s wr, 99 op/s 2024-07-31T15:14:07.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:07 smithi073 ceph-mon[36714]: pgmap v1136: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 17 MiB/s wr, 99 op/s 2024-07-31T15:14:09.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:09 smithi067 ceph-mon[32750]: pgmap v1137: 129 pgs: 13 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 83 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 23 MiB/s wr, 83 op/s 2024-07-31T15:14:09.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.418+0000 7f7ce9de0640 1 -- 172.21.15.67:0/1302074060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ce4102490 msgr2=0x7f7ce41028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:09.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.418+0000 7f7ce9de0640 1 --2- 172.21.15.67:0/1302074060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ce4102490 0x7f7ce41028a0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009900 tx=0x7f7cd402a410 comp rx=0 tx=0).stop 2024-07-31T15:14:09.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.418+0000 7f7ce9de0640 1 -- 172.21.15.67:0/1302074060 shutdown_connections 2024-07-31T15:14:09.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.418+0000 7f7ce9de0640 1 --2- 172.21.15.67:0/1302074060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ce4108720 0x7f7ce410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:09.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.418+0000 7f7ce9de0640 1 --2- 172.21.15.67:0/1302074060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ce4107dd0 0x7f7ce41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:09.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.418+0000 7f7ce9de0640 1 --2- 172.21.15.67:0/1302074060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ce4102490 0x7f7ce41028a0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:09.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.418+0000 7f7ce9de0640 1 -- 172.21.15.67:0/1302074060 >> 172.21.15.67:0/1302074060 conn(0x7f7ce4079160 msgr2=0x7f7ce4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:09.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.418+0000 7f7ce9de0640 1 -- 172.21.15.67:0/1302074060 shutdown_connections 2024-07-31T15:14:09.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.418+0000 7f7ce9de0640 1 -- 172.21.15.67:0/1302074060 wait complete. 2024-07-31T15:14:09.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.419+0000 7f7ce9de0640 1 Processor -- start 2024-07-31T15:14:09.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.419+0000 7f7ce9de0640 1 -- start start 2024-07-31T15:14:09.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce9de0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ce4102490 0x7f7ce4199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:09.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce9de0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ce4107dd0 0x7f7ce419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:09.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce9de0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ce4108720 0x7f7ce419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:09.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce9de0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce41134b0 con 0x7f7ce4102490 2024-07-31T15:14:09.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce9de0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce4113330 con 0x7f7ce4108720 2024-07-31T15:14:09.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce9de0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce4113630 con 0x7f7ce4107dd0 2024-07-31T15:14:09.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ce4107dd0 0x7f7ce419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:09.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ce4107dd0 0x7f7ce419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:52436/0 (socket says 172.21.15.67:52436) 2024-07-31T15:14:09.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce2ffd640 1 -- 172.21.15.67:0/2398490948 learned_addr learned my addr 172.21.15.67:0/2398490948 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:09.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce37fe640 1 --2- 172.21.15.67:0/2398490948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ce4102490 0x7f7ce4199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:09.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce3fff640 1 --2- 172.21.15.67:0/2398490948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ce4108720 0x7f7ce419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:09.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce2ffd640 1 -- 172.21.15.67:0/2398490948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ce4108720 msgr2=0x7f7ce419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:09.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce2ffd640 1 --2- 172.21.15.67:0/2398490948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ce4108720 0x7f7ce419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:09.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce2ffd640 1 -- 172.21.15.67:0/2398490948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ce4102490 msgr2=0x7f7ce4199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:09.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce2ffd640 1 --2- 172.21.15.67:0/2398490948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ce4102490 0x7f7ce4199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:09.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce2ffd640 1 -- 172.21.15.67:0/2398490948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce419feb0 con 0x7f7ce4107dd0 2024-07-31T15:14:09.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.420+0000 7f7ce2ffd640 1 --2- 172.21.15.67:0/2398490948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ce4107dd0 0x7f7ce419a4e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800e9c0 tx=0x7f7cd800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:09.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.421+0000 7f7ce0ff9640 1 -- 172.21.15.67:0/2398490948 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd800cd80 con 0x7f7ce4107dd0 2024-07-31T15:14:09.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.421+0000 7f7ce0ff9640 1 -- 172.21.15.67:0/2398490948 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd80054d0 con 0x7f7ce4107dd0 2024-07-31T15:14:09.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.421+0000 7f7ce9de0640 1 -- 172.21.15.67:0/2398490948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce41a01a0 con 0x7f7ce4107dd0 2024-07-31T15:14:09.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.421+0000 7f7ce0ff9640 1 -- 172.21.15.67:0/2398490948 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8010430 con 0x7f7ce4107dd0 2024-07-31T15:14:09.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.421+0000 7f7ce9de0640 1 -- 172.21.15.67:0/2398490948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce41a6900 con 0x7f7ce4107dd0 2024-07-31T15:14:09.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.422+0000 7f7ce9de0640 1 -- 172.21.15.67:0/2398490948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce41028a0 con 0x7f7ce4107dd0 2024-07-31T15:14:09.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.423+0000 7f7ce0ff9640 1 -- 172.21.15.67:0/2398490948 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7cd8004fb0 con 0x7f7ce4107dd0 2024-07-31T15:14:09.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.424+0000 7f7ce0ff9640 1 --2- 172.21.15.67:0/2398490948 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7ccc0b3d40 0x7f7ccc0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:09.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.424+0000 7f7ce37fe640 1 --2- 172.21.15.67:0/2398490948 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7ccc0b3d40 0x7f7ccc0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:09.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.424+0000 7f7ce0ff9640 1 -- 172.21.15.67:0/2398490948 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9483+0+0 (secure 0 0 0) 0x7f7cd80e4a00 con 0x7f7ce4107dd0 2024-07-31T15:14:09.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.425+0000 7f7ce37fe640 1 --2- 172.21.15.67:0/2398490948 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7ccc0b3d40 0x7f7ccc0b6250 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4031440 tx=0x7f7cd4009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:09.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.427+0000 7f7ce0ff9640 1 -- 172.21.15.67:0/2398490948 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f7cd80555e0 con 0x7f7ce4107dd0 2024-07-31T15:14:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:09 smithi077 ceph-mon[37327]: pgmap v1137: 129 pgs: 13 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 83 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 23 MiB/s wr, 83 op/s 2024-07-31T15:14:09.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:09 smithi073 ceph-mon[36714]: pgmap v1137: 129 pgs: 13 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 83 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 23 MiB/s wr, 83 op/s 2024-07-31T15:14:09.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.663+0000 7f7ce9de0640 1 -- 172.21.15.67:0/2398490948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ce419b070 con 0x7f7ce4107dd0 2024-07-31T15:14:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.665+0000 7f7ce0ff9640 1 -- 172.21.15.67:0/2398490948 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11959 (secure 0 0 0) 0x7f7ce419b070 con 0x7f7ce4107dd0 2024-07-31T15:14:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:14:09.667 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":66,"btime":"2024-07-31T15:10:30:235518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:10:30.235494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29129":{"gid":29129,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/631186801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":631186801},{"type":"v1","addr":"172.21.15.73:6835","nonce":631186801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:14:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 66 2024-07-31T15:14:09.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.667+0000 7f7ce9de0640 1 -- 172.21.15.67:0/2398490948 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7ccc0b3d40 msgr2=0x7f7ccc0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:09.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.667+0000 7f7ce9de0640 1 --2- 172.21.15.67:0/2398490948 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7ccc0b3d40 0x7f7ccc0b6250 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4031440 tx=0x7f7cd4009560 comp rx=0 tx=0).stop 2024-07-31T15:14:09.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.667+0000 7f7ce9de0640 1 -- 172.21.15.67:0/2398490948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ce4107dd0 msgr2=0x7f7ce419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:09.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.667+0000 7f7ce9de0640 1 --2- 172.21.15.67:0/2398490948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ce4107dd0 0x7f7ce419a4e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800e9c0 tx=0x7f7cd800ee80 comp rx=0 tx=0).stop 2024-07-31T15:14:09.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.667+0000 7f7ce9de0640 1 -- 172.21.15.67:0/2398490948 shutdown_connections 2024-07-31T15:14:09.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.667+0000 7f7ce9de0640 1 --2- 172.21.15.67:0/2398490948 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7ccc0b3d40 0x7f7ccc0b6250 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:09.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.667+0000 7f7ce9de0640 1 --2- 172.21.15.67:0/2398490948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ce4108720 0x7f7ce419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:09.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.667+0000 7f7ce9de0640 1 --2- 172.21.15.67:0/2398490948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ce4107dd0 0x7f7ce419a4e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:09.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.667+0000 7f7ce9de0640 1 --2- 172.21.15.67:0/2398490948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ce4102490 0x7f7ce4199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:09.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.667+0000 7f7ce9de0640 1 -- 172.21.15.67:0/2398490948 >> 172.21.15.67:0/2398490948 conn(0x7f7ce4079160 msgr2=0x7f7ce4100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:09.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.667+0000 7f7ce9de0640 1 -- 172.21.15.67:0/2398490948 shutdown_connections 2024-07-31T15:14:09.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:09.667+0000 7f7ce9de0640 1 -- 172.21.15.67:0/2398490948 wait complete. 2024-07-31T15:14:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.007+0000 7f5bec50e640 1 -- 172.21.15.67:0/4152577047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be41023f0 msgr2=0x7f5be4102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.007+0000 7f5bec50e640 1 --2- 172.21.15.67:0/4152577047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be41023f0 0x7f5be4102800 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009900 tx=0x7f5bd802a410 comp rx=0 tx=0).stop 2024-07-31T15:14:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.007+0000 7f5bec50e640 1 -- 172.21.15.67:0/4152577047 shutdown_connections 2024-07-31T15:14:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.007+0000 7f5bec50e640 1 --2- 172.21.15.67:0/4152577047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be4108670 0x7f5be410eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.007+0000 7f5bec50e640 1 --2- 172.21.15.67:0/4152577047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5be4107d20 0x7f5be4108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.007+0000 7f5bec50e640 1 --2- 172.21.15.67:0/4152577047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be41023f0 0x7f5be4102800 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.007+0000 7f5bec50e640 1 -- 172.21.15.67:0/4152577047 >> 172.21.15.67:0/4152577047 conn(0x7f5be40fddf0 msgr2=0x7f5be4100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.007+0000 7f5bec50e640 1 -- 172.21.15.67:0/4152577047 shutdown_connections 2024-07-31T15:14:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.007+0000 7f5bec50e640 1 -- 172.21.15.67:0/4152577047 wait complete. 2024-07-31T15:14:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.008+0000 7f5bec50e640 1 Processor -- start 2024-07-31T15:14:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.008+0000 7f5bec50e640 1 -- start start 2024-07-31T15:14:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.008+0000 7f5bec50e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be41023f0 0x7f5be419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.008+0000 7f5bec50e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5be4107d20 0x7f5be419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.008+0000 7f5bec50e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be4108670 0x7f5be419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.008+0000 7f5bec50e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4112320 con 0x7f5be41023f0 2024-07-31T15:14:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.008+0000 7f5bec50e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41121a0 con 0x7f5be4108670 2024-07-31T15:14:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.008+0000 7f5bec50e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41124a0 con 0x7f5be4107d20 2024-07-31T15:14:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.008+0000 7f5bea2aa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be41023f0 0x7f5be419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.008+0000 7f5bea2aa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be41023f0 0x7f5be419acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51342/0 (socket says 172.21.15.67:51342) 2024-07-31T15:14:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.008+0000 7f5bea2aa640 1 -- 172.21.15.67:0/963037929 learned_addr learned my addr 172.21.15.67:0/963037929 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.008+0000 7f5be9aa9640 1 --2- 172.21.15.67:0/963037929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5be4107d20 0x7f5be419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.008+0000 7f5beaaab640 1 --2- 172.21.15.67:0/963037929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be4108670 0x7f5be419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.009+0000 7f5bea2aa640 1 -- 172.21.15.67:0/963037929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5be4107d20 msgr2=0x7f5be419b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.009+0000 7f5bea2aa640 1 --2- 172.21.15.67:0/963037929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5be4107d20 0x7f5be419b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.009+0000 7f5bea2aa640 1 -- 172.21.15.67:0/963037929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be4108670 msgr2=0x7f5be419b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.009+0000 7f5bea2aa640 1 --2- 172.21.15.67:0/963037929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be4108670 0x7f5be419b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.009+0000 7f5bea2aa640 1 -- 172.21.15.67:0/963037929 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be41a5f80 con 0x7f5be41023f0 2024-07-31T15:14:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.009+0000 7f5bea2aa640 1 --2- 172.21.15.67:0/963037929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be41023f0 0x7f5be419acf0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8005940 tx=0x7f5bd8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.009+0000 7f5bd77fe640 1 -- 172.21.15.67:0/963037929 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd803f030 con 0x7f5be41023f0 2024-07-31T15:14:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.009+0000 7f5bd77fe640 1 -- 172.21.15.67:0/963037929 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bd8003aa0 con 0x7f5be41023f0 2024-07-31T15:14:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.009+0000 7f5bec50e640 1 -- 172.21.15.67:0/963037929 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be41a6210 con 0x7f5be41023f0 2024-07-31T15:14:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.009+0000 7f5bd77fe640 1 -- 172.21.15.67:0/963037929 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8035070 con 0x7f5be41023f0 2024-07-31T15:14:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.009+0000 7f5bec50e640 1 -- 172.21.15.67:0/963037929 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be41a6750 con 0x7f5be41023f0 2024-07-31T15:14:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.013+0000 7f5bf3402740 1 Processor -- start 2024-07-31T15:14:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.013+0000 7f5bf3402740 1 -- start start 2024-07-31T15:14:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.013+0000 7f5bf3402740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c831b0 0x55a121025ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.013+0000 7f5bf3402740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c5a2f0 0x55a121026220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.013+0000 7f5bf3402740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 0x55a121026760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.013+0000 7f5bf3402740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a120f9b3b0 con 0x55a120c831b0 2024-07-31T15:14:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.013+0000 7f5bf3402740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a120f8a8b0 con 0x55a120c624d0 2024-07-31T15:14:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.013+0000 7f5bf3402740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a120f8e8f0 con 0x55a120c5a2f0 2024-07-31T15:14:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.013+0000 7f5bd77fe640 1 -- 172.21.15.67:0/963037929 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5bd8042030 con 0x7f5be41023f0 2024-07-31T15:14:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.013+0000 7f5beaaab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c831b0 0x55a121025ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.013+0000 7f5beaaab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c831b0 0x55a121025ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51346/0 (socket says 172.21.15.67:51346) 2024-07-31T15:14:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.013+0000 7f5beaaab640 1 -- 172.21.15.67:0/2108504493 learned_addr learned my addr 172.21.15.67:0/2108504493 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.013+0000 7f5be9aa9640 1 --2- 172.21.15.67:0/2108504493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 0x55a121026760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.014+0000 7f5bea2aa640 1 --2- 172.21.15.67:0/2108504493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c5a2f0 0x55a121026220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.014+0000 7f5beaaab640 1 -- 172.21.15.67:0/2108504493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c5a2f0 msgr2=0x55a121026220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.014+0000 7f5beaaab640 1 --2- 172.21.15.67:0/2108504493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c5a2f0 0x55a121026220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.014+0000 7f5beaaab640 1 -- 172.21.15.67:0/2108504493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 msgr2=0x55a121026760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.014+0000 7f5beaaab640 1 --2- 172.21.15.67:0/2108504493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 0x55a121026760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.014+0000 7f5beaaab640 1 -- 172.21.15.67:0/2108504493 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a120f319a0 con 0x55a120c831b0 2024-07-31T15:14:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.014+0000 7f5be9aa9640 1 --2- 172.21.15.67:0/2108504493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 0x55a121026760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.014+0000 7f5bea2aa640 1 --2- 172.21.15.67:0/2108504493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c5a2f0 0x55a121026220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:14:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.014+0000 7f5beaaab640 1 --2- 172.21.15.67:0/2108504493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c831b0 0x55a121025ce0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009f50 tx=0x7f5bdc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cb0fb577be188c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.014+0000 7f5bd4bfe640 1 -- 172.21.15.67:0/2108504493 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc004d20 con 0x55a120c831b0 2024-07-31T15:14:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.014+0000 7f5bd4bfe640 1 -- 172.21.15.67:0/2108504493 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bdc004050 con 0x55a120c831b0 2024-07-31T15:14:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.014+0000 7f5bd4bfe640 1 -- 172.21.15.67:0/2108504493 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc005280 con 0x55a120c831b0 2024-07-31T15:14:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.014+0000 7f5bd77fe640 1 --2- 172.21.15.67:0/963037929 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5bd00b3dc0 0x7f5bd00b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.015+0000 7f5bea2aa640 1 --2- 172.21.15.67:0/963037929 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5bd00b3dc0 0x7f5bd00b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.015+0000 7f5bd77fe640 1 -- 172.21.15.67:0/963037929 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9483+0+0 (secure 0 0 0) 0x7f5bd81049c0 con 0x7f5be41023f0 2024-07-31T15:14:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.015+0000 7f5bf3402740 1 -- 172.21.15.67:0/2108504493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c831b0 msgr2=0x55a121025ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.015+0000 7f5bf3402740 1 --2- 172.21.15.67:0/2108504493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c831b0 0x55a121025ce0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009f50 tx=0x7f5bdc0118d0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.015+0000 7f5bea2aa640 1 --2- 172.21.15.67:0/963037929 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5bd00b3dc0 0x7f5bd00b62d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8101e00 tx=0x7f5bd802a410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.015+0000 7f5bf3402740 1 -- 172.21.15.67:0/2108504493 shutdown_connections 2024-07-31T15:14:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.015+0000 7f5bf3402740 1 --2- 172.21.15.67:0/2108504493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 0x55a121026760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.015+0000 7f5bf3402740 1 --2- 172.21.15.67:0/2108504493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c5a2f0 0x55a121026220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.015+0000 7f5bf3402740 1 --2- 172.21.15.67:0/2108504493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c831b0 0x55a121025ce0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.015+0000 7f5bf3402740 1 -- 172.21.15.67:0/2108504493 >> 172.21.15.67:0/2108504493 conn(0x55a120c77300 msgr2=0x55a121023bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.015+0000 7f5bf3402740 1 -- 172.21.15.67:0/2108504493 shutdown_connections 2024-07-31T15:14:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.015+0000 7f5bf3402740 1 -- 172.21.15.67:0/2108504493 wait complete. 2024-07-31T15:14:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.016+0000 7f5bf3402740 1 -- start start 2024-07-31T15:14:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.016+0000 7f5bf3402740 1 Processor -- start 2024-07-31T15:14:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.016+0000 7f5bf3402740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c5a2f0 0x55a1210317c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.016+0000 7f5bf3402740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 0x55a121031d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.016+0000 7f5bf3402740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c831b0 0x55a12103d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.016+0000 7f5bf3402740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a120ec4ab0 con 0x55a120c5a2f0 2024-07-31T15:14:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.016+0000 7f5bf3402740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a120f12cf0 con 0x55a120c624d0 2024-07-31T15:14:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.016+0000 7f5bf3402740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a121034460 con 0x55a120c831b0 2024-07-31T15:14:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.016+0000 7f5bf3402740 10 client.0 _ll_register_callbacks cb 0x55a120f849f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.016+0000 7f5beaaab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c5a2f0 0x55a1210317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.016+0000 7f5beaaab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c5a2f0 0x55a1210317c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51358/0 (socket says 172.21.15.67:51358) 2024-07-31T15:14:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.016+0000 7f5beaaab640 1 -- 172.21.15.67:0/487236037 learned_addr learned my addr 172.21.15.67:0/487236037 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.016+0000 7f5be9aa9640 1 --2- 172.21.15.67:0/487236037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 0x55a121031d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.017+0000 7f5beaaab640 1 -- 172.21.15.67:0/487236037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c831b0 msgr2=0x55a12103d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.017+0000 7f5beaaab640 1 --2- 172.21.15.67:0/487236037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c831b0 0x55a12103d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.017+0000 7f5beaaab640 1 -- 172.21.15.67:0/487236037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 msgr2=0x55a121031d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.017+0000 7f5beaaab640 1 --2- 172.21.15.67:0/487236037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 0x55a121031d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.017+0000 7f5beaaab640 1 -- 172.21.15.67:0/487236037 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a120cee790 con 0x55a120c5a2f0 2024-07-31T15:14:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.017+0000 7f5beaaab640 1 --2- 172.21.15.67:0/487236037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c5a2f0 0x55a1210317c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009250 tx=0x7f5bdc009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9989fa6efd0b9a1f server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.017+0000 7f5bcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:14:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.017+0000 7f5bcdffb640 1 -- 172.21.15.67:0/487236037 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc028070 con 0x55a120c5a2f0 2024-07-31T15:14:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.017+0000 7f5bcdffb640 1 -- 172.21.15.67:0/487236037 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9483+0+0 (secure 0 0 0) 0x7f5bdc005040 con 0x55a120c5a2f0 2024-07-31T15:14:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.017+0000 7f5bf3402740 1 -- 172.21.15.67:0/487236037 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bdc004050 con 0x55a120c5a2f0 2024-07-31T15:14:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.017+0000 7f5bcdffb640 1 -- 172.21.15.67:0/487236037 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f5bdc004e50 con 0x55a120c5a2f0 2024-07-31T15:14:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.017+0000 7f5bf3402740 10 client.20331 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.017+0000 7f5bf3402740 1 -- 172.21.15.67:0/487236037 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55a120d67620 con 0x55a120c5a2f0 2024-07-31T15:14:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.017+0000 7f5bcdffb640 1 -- 172.21.15.67:0/487236037 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bdc004050 con 0x55a120c5a2f0 2024-07-31T15:14:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.018+0000 7f5bcdffb640 1 -- 172.21.15.67:0/487236037 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f5bdc024750 con 0x55a120c5a2f0 2024-07-31T15:14:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.018+0000 7f5bf3402740 10 client.20331 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.018+0000 7f5bf3402740 10 client.20331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:14:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.018+0000 7f5bf3402740 20 client.20331 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.018+0000 7f5bf3402740 1 --2- 172.21.15.67:0/487236037 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a120c41d80 0x55a1210435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.018+0000 7f5bf3402740 4 client.20331 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:14:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.018+0000 7f5bf3402740 1 -- 172.21.15.67:0/487236037 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a120f9d300 con 0x55a120c41d80 2024-07-31T15:14:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.018+0000 7f5be9aa9640 1 --2- 172.21.15.67:0/487236037 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a120c41d80 0x55a1210435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.019+0000 7f5bcdffb640 10 client.20331 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.019+0000 7f5be9aa9640 1 --2- 172.21.15.67:0/487236037 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a120c41d80 0x55a1210435c0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2959d810edc44d8 server_cookie=54a0b0bab0e42476 in_seq=0 out_seq=0 2024-07-31T15:14:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.019+0000 7f5bcdffb640 1 -- 172.21.15.67:0/487236037 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a120f9d300 con 0x55a120c41d80 2024-07-31T15:14:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.019+0000 7f5bcdffb640 10 client.20331 handle_command_reply: tid=0 2024-07-31T15:14:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.019+0000 7f5bf3402740 1 client.20331 shutdown 2024-07-31T15:14:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.019+0000 7f5bf3402740 1 -- 172.21.15.67:0/487236037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c5a2f0 msgr2=0x55a1210317c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.019+0000 7f5bf3402740 1 --2- 172.21.15.67:0/487236037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c5a2f0 0x55a1210317c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009250 tx=0x7f5bdc009cd0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.019+0000 7f5bf3402740 1 -- 172.21.15.67:0/487236037 shutdown_connections 2024-07-31T15:14:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.019+0000 7f5bf3402740 1 --2- 172.21.15.67:0/487236037 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a120c41d80 0x55a1210435c0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.019+0000 7f5bf3402740 1 --2- 172.21.15.67:0/487236037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c831b0 0x55a12103d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.019+0000 7f5bcdffb640 0 client.20331 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.019+0000 7f5bf3402740 1 --2- 172.21.15.67:0/487236037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 0x55a121031d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.019+0000 7f5bf3402740 1 --2- 172.21.15.67:0/487236037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c5a2f0 0x55a1210317c0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.019+0000 7f5bf3402740 1 -- 172.21.15.67:0/487236037 >> 172.21.15.67:0/487236037 conn(0x55a120c77300 msgr2=0x55a1210222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.020+0000 7f5bf3402740 1 -- 172.21.15.67:0/487236037 shutdown_connections 2024-07-31T15:14:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.020+0000 7f5bf3402740 1 -- 172.21.15.67:0/487236037 wait complete. 2024-07-31T15:14:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.020+0000 7f5bf3402740 20 client.20331 trim_cache size 0 max 16384 2024-07-31T15:14:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.035+0000 7f5bf3402740 1 Processor -- start 2024-07-31T15:14:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.035+0000 7f5bf3402740 1 -- start start 2024-07-31T15:14:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.035+0000 7f5bf3402740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c41d80 0x55a121030e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.035+0000 7f5bf3402740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c5a2f0 0x55a121027420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.035+0000 7f5bf3402740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 0x55a12103d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.035+0000 7f5bf3402740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41b0970 con 0x55a120c5a2f0 2024-07-31T15:14:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.035+0000 7f5bf3402740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41b24b0 con 0x55a120c624d0 2024-07-31T15:14:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.035+0000 7f5bf3402740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41b1040 con 0x55a120c41d80 2024-07-31T15:14:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.035+0000 7f5be9aa9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c5a2f0 0x55a121027420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.035+0000 7f5be9aa9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c5a2f0 0x55a121027420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51372/0 (socket says 172.21.15.67:51372) 2024-07-31T15:14:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.035+0000 7f5be9aa9640 1 -- 172.21.15.67:0/4223694056 learned_addr learned my addr 172.21.15.67:0/4223694056 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.035+0000 7f5beaaab640 1 --2- 172.21.15.67:0/4223694056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c41d80 0x55a121030e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.036+0000 7f5beaaab640 1 --2- 172.21.15.67:0/4223694056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 0x55a12103d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.036+0000 7f5be9aa9640 1 -- 172.21.15.67:0/4223694056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c41d80 msgr2=0x55a121030e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.036+0000 7f5be9aa9640 1 --2- 172.21.15.67:0/4223694056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c41d80 0x55a121030e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.036+0000 7f5be9aa9640 1 -- 172.21.15.67:0/4223694056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 msgr2=0x55a12103d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.036+0000 7f5be9aa9640 1 --2- 172.21.15.67:0/4223694056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 0x55a12103d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.036+0000 7f5be9aa9640 1 -- 172.21.15.67:0/4223694056 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a121026b10 con 0x55a120c5a2f0 2024-07-31T15:14:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.036+0000 7f5be9aa9640 1 --2- 172.21.15.67:0/4223694056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c5a2f0 0x55a121027420 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f5be0000c00 tx=0x7f5be000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cb76d1ca9fe1d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.036+0000 7f5bcffff640 1 -- 172.21.15.67:0/4223694056 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0005020 con 0x55a120c5a2f0 2024-07-31T15:14:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.036+0000 7f5bcffff640 1 -- 172.21.15.67:0/4223694056 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be00047a0 con 0x55a120c5a2f0 2024-07-31T15:14:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.036+0000 7f5bcffff640 1 -- 172.21.15.67:0/4223694056 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be00266a0 con 0x55a120c5a2f0 2024-07-31T15:14:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.037+0000 7f5bf3402740 1 -- 172.21.15.67:0/4223694056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c5a2f0 msgr2=0x55a121027420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.037+0000 7f5bf3402740 1 --2- 172.21.15.67:0/4223694056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c5a2f0 0x55a121027420 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f5be0000c00 tx=0x7f5be000b390 comp rx=0 tx=0).stop 2024-07-31T15:14:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.037+0000 7f5bf3402740 1 -- 172.21.15.67:0/4223694056 shutdown_connections 2024-07-31T15:14:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.037+0000 7f5bf3402740 1 --2- 172.21.15.67:0/4223694056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c624d0 0x55a12103d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.037+0000 7f5bf3402740 1 --2- 172.21.15.67:0/4223694056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c5a2f0 0x55a121027420 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.037+0000 7f5bf3402740 1 --2- 172.21.15.67:0/4223694056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c41d80 0x55a121030e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.037+0000 7f5bf3402740 1 -- 172.21.15.67:0/4223694056 >> 172.21.15.67:0/4223694056 conn(0x55a120c77300 msgr2=0x55a121026d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.037+0000 7f5bf3402740 1 -- 172.21.15.67:0/4223694056 shutdown_connections 2024-07-31T15:14:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.037+0000 7f5bf3402740 1 -- 172.21.15.67:0/4223694056 wait complete. 2024-07-31T15:14:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.038+0000 7f5bf3402740 1 -- start start 2024-07-31T15:14:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.038+0000 7f5bf3402740 1 Processor -- start 2024-07-31T15:14:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.038+0000 7f5bf3402740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c41d80 0x55a121039fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.038+0000 7f5bf3402740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c5a2f0 0x55a1210366e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.038+0000 7f5bf3402740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c624d0 0x55a121036df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.038+0000 7f5bf3402740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41b1de0 con 0x55a120c41d80 2024-07-31T15:14:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.038+0000 7f5bf3402740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41b1710 con 0x55a120c5a2f0 2024-07-31T15:14:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.038+0000 7f5bf3402740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41ae970 con 0x55a120c624d0 2024-07-31T15:14:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.038+0000 7f5bf3402740 10 client.0 _ll_register_callbacks cb 0x7f5be000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.038+0000 7f5beaaab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c41d80 0x55a121039fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.038+0000 7f5beaaab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c41d80 0x55a121039fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51388/0 (socket says 172.21.15.67:51388) 2024-07-31T15:14:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.038+0000 7f5beaaab640 1 -- 172.21.15.67:0/2130913971 learned_addr learned my addr 172.21.15.67:0/2130913971 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.038+0000 7f5be9aa9640 1 --2- 172.21.15.67:0/2130913971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c5a2f0 0x55a1210366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.038+0000 7f5beaaab640 1 --2- 172.21.15.67:0/2130913971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c624d0 0x55a121036df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.039+0000 7f5beaaab640 1 -- 172.21.15.67:0/2130913971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c624d0 msgr2=0x55a121036df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.039+0000 7f5beaaab640 1 --2- 172.21.15.67:0/2130913971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c624d0 0x55a121036df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.039+0000 7f5beaaab640 1 -- 172.21.15.67:0/2130913971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c5a2f0 msgr2=0x55a1210366e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.039+0000 7f5beaaab640 1 --2- 172.21.15.67:0/2130913971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c5a2f0 0x55a1210366e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.039+0000 7f5beaaab640 1 -- 172.21.15.67:0/2130913971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1210a9bc0 con 0x55a120c41d80 2024-07-31T15:14:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.039+0000 7f5beaaab640 1 --2- 172.21.15.67:0/2130913971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c41d80 0x55a121039fc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc024ac0 tx=0x7f5bdc009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e137a5ca9405cfb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.039+0000 7f5bcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:14:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.039+0000 7f5bcdffb640 1 -- 172.21.15.67:0/2130913971 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc041060 con 0x55a120c41d80 2024-07-31T15:14:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.039+0000 7f5bcdffb640 1 -- 172.21.15.67:0/2130913971 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9483+0+0 (secure 0 0 0) 0x7f5bdc003fc0 con 0x55a120c41d80 2024-07-31T15:14:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.039+0000 7f5bf3402740 1 -- 172.21.15.67:0/2130913971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be00047a0 con 0x55a120c41d80 2024-07-31T15:14:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.039+0000 7f5bcdffb640 1 -- 172.21.15.67:0/2130913971 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f5bdc048050 con 0x55a120c41d80 2024-07-31T15:14:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.039+0000 7f5bf3402740 10 client.20337 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.039+0000 7f5bf3402740 1 -- 172.21.15.67:0/2130913971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55a1210aacf0 con 0x55a120c41d80 2024-07-31T15:14:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.040+0000 7f5bcdffb640 1 -- 172.21.15.67:0/2130913971 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be00047a0 con 0x55a120c41d80 2024-07-31T15:14:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.040+0000 7f5bcdffb640 1 -- 172.21.15.67:0/2130913971 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f5bdc00f510 con 0x55a120c41d80 2024-07-31T15:14:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.040+0000 7f5bf3402740 10 client.20337 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.040+0000 7f5bf3402740 10 client.20337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:14:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.040+0000 7f5bf3402740 20 client.20337 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.040+0000 7f5bf3402740 1 --2- 172.21.15.67:0/2130913971 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a120c831b0 0x55a1210ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.040+0000 7f5bf3402740 4 client.20337 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:14:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.040+0000 7f5bf3402740 1 -- 172.21.15.67:0/2130913971 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a120f98da0 con 0x55a120c831b0 2024-07-31T15:14:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.040+0000 7f5be9aa9640 1 --2- 172.21.15.67:0/2130913971 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a120c831b0 0x55a1210ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.040+0000 7f5be9aa9640 1 --2- 172.21.15.67:0/2130913971 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a120c831b0 0x55a1210ad450 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c6a809f369c3200 server_cookie=ea01c0c00a27b9ab in_seq=0 out_seq=0 2024-07-31T15:14:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.040+0000 7f5bcdffb640 10 client.20337 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bcdffb640 1 -- 172.21.15.67:0/2130913971 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a120f98da0 con 0x55a120c831b0 2024-07-31T15:14:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bcdffb640 10 client.20337 handle_command_reply: tid=0 2024-07-31T15:14:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bf3402740 1 client.20337 shutdown 2024-07-31T15:14:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bf3402740 1 -- 172.21.15.67:0/2130913971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c41d80 msgr2=0x55a121039fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bf3402740 1 --2- 172.21.15.67:0/2130913971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c41d80 0x55a121039fc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc024ac0 tx=0x7f5bdc009cd0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bf3402740 1 -- 172.21.15.67:0/2130913971 shutdown_connections 2024-07-31T15:14:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bf3402740 1 --2- 172.21.15.67:0/2130913971 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55a120c831b0 0x55a1210ad450 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bf3402740 1 --2- 172.21.15.67:0/2130913971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a120c624d0 0x55a121036df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bf3402740 1 --2- 172.21.15.67:0/2130913971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a120c5a2f0 0x55a1210366e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bf3402740 1 --2- 172.21.15.67:0/2130913971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a120c41d80 0x55a121039fc0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bcdffb640 0 client.20337 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bf3402740 1 -- 172.21.15.67:0/2130913971 >> 172.21.15.67:0/2130913971 conn(0x55a120c77300 msgr2=0x55a121032a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bf3402740 1 -- 172.21.15.67:0/2130913971 shutdown_connections 2024-07-31T15:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bf3402740 1 -- 172.21.15.67:0/2130913971 wait complete. 2024-07-31T15:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.105+0000 7f5bf3402740 20 client.20337 trim_cache size 0 max 16384 2024-07-31T15:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:14:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:14:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.106+0000 7f5bec50e640 1 -- 172.21.15.67:0/963037929 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5bd00b3dc0 msgr2=0x7f5bd00b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.106+0000 7f5bec50e640 1 --2- 172.21.15.67:0/963037929 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5bd00b3dc0 0x7f5bd00b62d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8101e00 tx=0x7f5bd802a410 comp rx=0 tx=0).stop 2024-07-31T15:14:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.106+0000 7f5bec50e640 1 -- 172.21.15.67:0/963037929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be41023f0 msgr2=0x7f5be419acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.106+0000 7f5bec50e640 1 --2- 172.21.15.67:0/963037929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be41023f0 0x7f5be419acf0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8005940 tx=0x7f5bd8004270 comp rx=0 tx=0).stop 2024-07-31T15:14:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.106+0000 7f5bec50e640 1 -- 172.21.15.67:0/963037929 shutdown_connections 2024-07-31T15:14:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.106+0000 7f5bec50e640 1 --2- 172.21.15.67:0/963037929 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5bd00b3dc0 0x7f5bd00b62d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.106+0000 7f5bec50e640 1 --2- 172.21.15.67:0/963037929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be4108670 0x7f5be419b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.106+0000 7f5bec50e640 1 --2- 172.21.15.67:0/963037929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5be4107d20 0x7f5be419b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.106+0000 7f5bec50e640 1 --2- 172.21.15.67:0/963037929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be41023f0 0x7f5be419acf0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.106+0000 7f5bec50e640 1 -- 172.21.15.67:0/963037929 >> 172.21.15.67:0/963037929 conn(0x7f5be40fddf0 msgr2=0x7f5be4107410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.106+0000 7f5bec50e640 1 -- 172.21.15.67:0/963037929 shutdown_connections 2024-07-31T15:14:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:10.106+0000 7f5bec50e640 1 -- 172.21.15.67:0/963037929 wait complete. 2024-07-31T15:14:10.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:10 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2398490948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:14:10.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:10 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2398490948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:14:10.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:10 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2398490948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:14:10.663 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24671 quiesce db / --timeout 90 --expiration 137.95 --await-for 45.0' 2024-07-31T15:14:10.664 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24671 quiesce db / --timeout 90 --expiration 137.95 --await-for 45.0 2024-07-31T15:14:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:14:11.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:11 smithi067 ceph-mon[32750]: pgmap v1138: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 23 MiB/s wr, 71 op/s 2024-07-31T15:14:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:11 smithi077 ceph-mon[37327]: pgmap v1138: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 23 MiB/s wr, 71 op/s 2024-07-31T15:14:11.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:11 smithi073 ceph-mon[36714]: pgmap v1138: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 23 MiB/s wr, 71 op/s 2024-07-31T15:14:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.826+0000 7ff0328bd640 1 -- 172.21.15.67:0/1966541614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108ed0 msgr2=0x7ff02c1092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.826+0000 7ff0328bd640 1 --2- 172.21.15.67:0/1966541614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108ed0 0x7ff02c1092e0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7ff020009900 tx=0x7ff02002a410 comp rx=0 tx=0).stop 2024-07-31T15:14:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.826+0000 7ff0328bd640 1 -- 172.21.15.67:0/1966541614 shutdown_connections 2024-07-31T15:14:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.826+0000 7ff0328bd640 1 --2- 172.21.15.67:0/1966541614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff02c109820 0x7ff02c110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.826+0000 7ff0328bd640 1 --2- 172.21.15.67:0/1966541614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108ed0 0x7ff02c1092e0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.826+0000 7ff0328bd640 1 --2- 172.21.15.67:0/1966541614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff02c103590 0x7ff02c1039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.826+0000 7ff0328bd640 1 -- 172.21.15.67:0/1966541614 >> 172.21.15.67:0/1966541614 conn(0x7ff02c07b280 msgr2=0x7ff02c07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.826+0000 7ff0328bd640 1 -- 172.21.15.67:0/1966541614 shutdown_connections 2024-07-31T15:14:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.826+0000 7ff0328bd640 1 -- 172.21.15.67:0/1966541614 wait complete. 2024-07-31T15:14:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.827+0000 7ff0328bd640 1 Processor -- start 2024-07-31T15:14:11.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.827+0000 7ff0328bd640 1 -- start start 2024-07-31T15:14:11.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.827+0000 7ff0328bd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff02c103590 0x7ff02c199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.827+0000 7ff0328bd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108ed0 0x7ff02c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.827+0000 7ff0328bd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff02c109820 0x7ff02c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.827+0000 7ff0328bd640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c113490 con 0x7ff02c108ed0 2024-07-31T15:14:11.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.827+0000 7ff0328bd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c113310 con 0x7ff02c103590 2024-07-31T15:14:11.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.827+0000 7ff0328bd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c113610 con 0x7ff02c109820 2024-07-31T15:14:11.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.827+0000 7ff02b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108ed0 0x7ff02c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.827+0000 7ff02b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108ed0 0x7ff02c19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34342/0 (socket says 172.21.15.67:34342) 2024-07-31T15:14:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.827+0000 7ff02b7fe640 1 -- 172.21.15.67:0/741311413 learned_addr learned my addr 172.21.15.67:0/741311413 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.827+0000 7ff02bfff640 1 --2- 172.21.15.67:0/741311413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff02c103590 0x7ff02c199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.827+0000 7ff030e5a640 1 --2- 172.21.15.67:0/741311413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff02c109820 0x7ff02c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.828+0000 7ff02b7fe640 1 -- 172.21.15.67:0/741311413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff02c109820 msgr2=0x7ff02c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.828+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/741311413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff02c109820 0x7ff02c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.828+0000 7ff02b7fe640 1 -- 172.21.15.67:0/741311413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff02c103590 msgr2=0x7ff02c199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.828+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/741311413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff02c103590 0x7ff02c199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.828+0000 7ff02b7fe640 1 -- 172.21.15.67:0/741311413 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c19fe50 con 0x7ff02c108ed0 2024-07-31T15:14:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.828+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/741311413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108ed0 0x7ff02c19a480 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7ff02002a930 tx=0x7ff020004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.828+0000 7ff0297fa640 1 -- 172.21.15.67:0/741311413 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02003f030 con 0x7ff02c108ed0 2024-07-31T15:14:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.828+0000 7ff0297fa640 1 -- 172.21.15.67:0/741311413 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff020003aa0 con 0x7ff02c108ed0 2024-07-31T15:14:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.828+0000 7ff0328bd640 1 -- 172.21.15.67:0/741311413 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02c1a00e0 con 0x7ff02c108ed0 2024-07-31T15:14:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.828+0000 7ff0297fa640 1 -- 172.21.15.67:0/741311413 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020035070 con 0x7ff02c108ed0 2024-07-31T15:14:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.828+0000 7ff0328bd640 1 -- 172.21.15.67:0/741311413 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02c1a68a0 con 0x7ff02c108ed0 2024-07-31T15:14:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.831+0000 7ff038e0e740 1 Processor -- start 2024-07-31T15:14:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.831+0000 7ff038e0e740 1 -- start start 2024-07-31T15:14:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff038e0e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c2735690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff038e0e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c23671b0 0x5587c2735bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff038e0e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c236f390 0x5587c2736110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff038e0e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5587c26abbb0 con 0x5587c236f390 2024-07-31T15:14:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff038e0e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5587c269b0b0 con 0x5587c2390070 2024-07-31T15:14:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff038e0e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587c269f0f0 con 0x5587c23671b0 2024-07-31T15:14:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff030e5a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c2735690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff0297fa640 1 -- 172.21.15.67:0/741311413 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff020042030 con 0x7ff02c108ed0 2024-07-31T15:14:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff030e5a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c2735690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:54264/0 (socket says 172.21.15.67:54264) 2024-07-31T15:14:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff030e5a640 1 -- 172.21.15.67:0/2735979825 learned_addr learned my addr 172.21.15.67:0/2735979825 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff02bfff640 1 --2- 172.21.15.67:0/2735979825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c23671b0 0x5587c2735bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/2735979825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c236f390 0x5587c2736110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff030e5a640 1 -- 172.21.15.67:0/2735979825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c23671b0 msgr2=0x5587c2735bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff030e5a640 1 --2- 172.21.15.67:0/2735979825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c23671b0 0x5587c2735bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff030e5a640 1 -- 172.21.15.67:0/2735979825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c236f390 msgr2=0x5587c2736110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff030e5a640 1 --2- 172.21.15.67:0/2735979825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c236f390 0x5587c2736110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff030e5a640 1 -- 172.21.15.67:0/2735979825 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587c2641b90 con 0x5587c2390070 2024-07-31T15:14:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.832+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/2735979825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c236f390 0x5587c2736110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:14:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.833+0000 7ff030e5a640 1 --2- 172.21.15.67:0/2735979825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c2735690 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff024009ee0 tx=0x7ff024010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49374ef77766cbbe server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.833+0000 7ff0297fa640 1 --2- 172.21.15.67:0/741311413 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff0140b3d40 0x7ff0140b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.833+0000 7ff0127fc640 1 -- 172.21.15.67:0/2735979825 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024016070 con 0x5587c2390070 2024-07-31T15:14:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.833+0000 7ff0127fc640 1 -- 172.21.15.67:0/2735979825 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff024004d20 con 0x5587c2390070 2024-07-31T15:14:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.833+0000 7ff0127fc640 1 -- 172.21.15.67:0/2735979825 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240126b0 con 0x5587c2390070 2024-07-31T15:14:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.833+0000 7ff02bfff640 1 --2- 172.21.15.67:0/741311413 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff0140b3d40 0x7ff0140b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.833+0000 7ff0297fa640 1 -- 172.21.15.67:0/741311413 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9483+0+0 (secure 0 0 0) 0x7ff020104bc0 con 0x7ff02c108ed0 2024-07-31T15:14:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.834+0000 7ff038e0e740 1 -- 172.21.15.67:0/2735979825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 msgr2=0x5587c2735690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.834+0000 7ff02bfff640 1 --2- 172.21.15.67:0/741311413 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff0140b3d40 0x7ff0140b6250 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x5587c2734310 tx=0x7ff01c00c430 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.834+0000 7ff038e0e740 1 --2- 172.21.15.67:0/2735979825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c2735690 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff024009ee0 tx=0x7ff024010740 comp rx=0 tx=0).stop 2024-07-31T15:14:11.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.834+0000 7ff038e0e740 1 -- 172.21.15.67:0/2735979825 shutdown_connections 2024-07-31T15:14:11.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.834+0000 7ff038e0e740 1 --2- 172.21.15.67:0/2735979825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c236f390 0x5587c2736110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.834+0000 7ff038e0e740 1 --2- 172.21.15.67:0/2735979825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c23671b0 0x5587c2735bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.834+0000 7ff038e0e740 1 --2- 172.21.15.67:0/2735979825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c2735690 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.834+0000 7ff038e0e740 1 -- 172.21.15.67:0/2735979825 >> 172.21.15.67:0/2735979825 conn(0x5587c2384460 msgr2=0x5587c2733580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:11.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.834+0000 7ff038e0e740 1 -- 172.21.15.67:0/2735979825 shutdown_connections 2024-07-31T15:14:11.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.834+0000 7ff038e0e740 1 -- 172.21.15.67:0/2735979825 wait complete. 2024-07-31T15:14:11.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.834+0000 7ff038e0e740 1 -- start start 2024-07-31T15:14:11.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff038e0e740 1 Processor -- start 2024-07-31T15:14:11.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff038e0e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c23671b0 0x5587c2741260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff038e0e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c236f390 0x5587c27417a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff038e0e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c274d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff038e0e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5587c24fe700 con 0x5587c23671b0 2024-07-31T15:14:11.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff038e0e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5587c263d9e0 con 0x5587c2390070 2024-07-31T15:14:11.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff038e0e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587c2743e70 con 0x5587c236f390 2024-07-31T15:14:11.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c23671b0 0x5587c2741260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff038e0e740 10 client.0 _ll_register_callbacks cb 0x5587c26584f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:11.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c23671b0 0x5587c2741260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34360/0 (socket says 172.21.15.67:34360) 2024-07-31T15:14:11.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff02bfff640 1 -- 172.21.15.67:0/2687166775 learned_addr learned my addr 172.21.15.67:0/2687166775 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:11.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c236f390 0x5587c27417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff030e5a640 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c274d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff02bfff640 1 -- 172.21.15.67:0/2687166775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c236f390 msgr2=0x5587c27417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:11.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff02bfff640 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c236f390 0x5587c27417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff02bfff640 1 -- 172.21.15.67:0/2687166775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 msgr2=0x5587c274d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:11.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.835+0000 7ff02bfff640 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c274d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.836+0000 7ff02bfff640 1 -- 172.21.15.67:0/2687166775 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587c23fb650 con 0x5587c23671b0 2024-07-31T15:14:11.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.836+0000 7ff030e5a640 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c274d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:11.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.836+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c236f390 0x5587c27417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:11.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.836+0000 7ff02bfff640 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c23671b0 0x5587c2741260 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7ff01c005c50 tx=0x7ff01c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe14996f7c7c3f7d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:11.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.836+0000 7ff00b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:14:11.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.836+0000 7ff00b7fe640 1 -- 172.21.15.67:0/2687166775 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c00cd40 con 0x5587c23671b0 2024-07-31T15:14:11.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.836+0000 7ff00b7fe640 1 -- 172.21.15.67:0/2687166775 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9483+0+0 (secure 0 0 0) 0x7ff01c004020 con 0x5587c23671b0 2024-07-31T15:14:11.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.836+0000 7ff038e0e740 1 -- 172.21.15.67:0/2687166775 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff024004d20 con 0x5587c23671b0 2024-07-31T15:14:11.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.836+0000 7ff00b7fe640 1 -- 172.21.15.67:0/2687166775 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7ff01c02f050 con 0x5587c23671b0 2024-07-31T15:14:11.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.836+0000 7ff038e0e740 10 client.20349 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:11.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.836+0000 7ff038e0e740 1 -- 172.21.15.67:0/2687166775 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5587c274ff50 con 0x5587c23671b0 2024-07-31T15:14:11.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.836+0000 7ff00b7fe640 1 -- 172.21.15.67:0/2687166775 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff024004d20 con 0x5587c23671b0 2024-07-31T15:14:11.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.837+0000 7ff00b7fe640 1 -- 172.21.15.67:0/2687166775 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7ff01c018040 con 0x5587c23671b0 2024-07-31T15:14:11.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.837+0000 7ff038e0e740 10 client.20349 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:11.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.837+0000 7ff038e0e740 10 client.20349 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:14:11.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.837+0000 7ff038e0e740 20 client.20349 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:11.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.837+0000 7ff038e0e740 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5587c234e9f0 0x5587c27535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.837+0000 7ff038e0e740 4 client.20349 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:14:11.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.837+0000 7ff038e0e740 1 -- 172.21.15.67:0/2687166775 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587c26adb00 con 0x5587c234e9f0 2024-07-31T15:14:11.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.837+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5587c234e9f0 0x5587c27535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.837+0000 7ff00b7fe640 10 client.20349 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:14:11.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.838+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5587c234e9f0 0x5587c27535a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76044c5850511c62 server_cookie=1e40f6f47d26e0dc in_seq=0 out_seq=0 2024-07-31T15:14:11.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.838+0000 7ff00b7fe640 1 -- 172.21.15.67:0/2687166775 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587c26adb00 con 0x5587c234e9f0 2024-07-31T15:14:11.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.838+0000 7ff00b7fe640 10 client.20349 handle_command_reply: tid=0 2024-07-31T15:14:11.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.838+0000 7ff038e0e740 1 client.20349 shutdown 2024-07-31T15:14:11.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.839+0000 7ff038e0e740 1 -- 172.21.15.67:0/2687166775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c23671b0 msgr2=0x5587c2741260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:11.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.839+0000 7ff038e0e740 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c23671b0 0x5587c2741260 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7ff01c005c50 tx=0x7ff01c0046c0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.839+0000 7ff038e0e740 1 -- 172.21.15.67:0/2687166775 shutdown_connections 2024-07-31T15:14:11.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.839+0000 7ff038e0e740 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5587c234e9f0 0x5587c27535a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.839+0000 7ff038e0e740 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c274d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.839+0000 7ff038e0e740 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c236f390 0x5587c27417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.839+0000 7ff038e0e740 1 --2- 172.21.15.67:0/2687166775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c23671b0 0x5587c2741260 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.839+0000 7ff038e0e740 1 -- 172.21.15.67:0/2687166775 >> 172.21.15.67:0/2687166775 conn(0x5587c2384460 msgr2=0x5587c2731d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:11.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.839+0000 7ff00b7fe640 0 client.20349 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:14:11.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.839+0000 7ff038e0e740 1 -- 172.21.15.67:0/2687166775 shutdown_connections 2024-07-31T15:14:11.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.839+0000 7ff038e0e740 1 -- 172.21.15.67:0/2687166775 wait complete. 2024-07-31T15:14:11.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.839+0000 7ff038e0e740 20 client.20349 trim_cache size 0 max 16384 2024-07-31T15:14:11.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.853+0000 7ff038e0e740 1 Processor -- start 2024-07-31T15:14:11.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.853+0000 7ff038e0e740 1 -- start start 2024-07-31T15:14:11.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff038e0e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c23671b0 0x5587c274cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff038e0e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c236f390 0x5587c2731d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff038e0e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c2732240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff038e0e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1ae690 con 0x5587c236f390 2024-07-31T15:14:11.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff038e0e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b01d0 con 0x5587c2390070 2024-07-31T15:14:11.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff038e0e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1aed60 con 0x5587c23671b0 2024-07-31T15:14:11.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff02b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c236f390 0x5587c2731d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff02b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c236f390 0x5587c2731d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34364/0 (socket says 172.21.15.67:34364) 2024-07-31T15:14:11.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff02b7fe640 1 -- 172.21.15.67:0/691658311 learned_addr learned my addr 172.21.15.67:0/691658311 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:11.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff02bfff640 1 --2- 172.21.15.67:0/691658311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c23671b0 0x5587c274cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff030e5a640 1 --2- 172.21.15.67:0/691658311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c2732240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff02b7fe640 1 -- 172.21.15.67:0/691658311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c23671b0 msgr2=0x5587c274cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:11.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/691658311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c23671b0 0x5587c274cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff02b7fe640 1 -- 172.21.15.67:0/691658311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 msgr2=0x5587c2732240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:11.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/691658311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c2732240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.854+0000 7ff02b7fe640 1 -- 172.21.15.67:0/691658311 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587c27384f0 con 0x5587c236f390 2024-07-31T15:14:11.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.855+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/691658311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c236f390 0x5587c2731d00 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7ff0200c8680 tx=0x7ff020000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345a67b5e534d59c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:11.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.855+0000 7ff011ffb640 1 -- 172.21.15.67:0/691658311 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020049040 con 0x5587c236f390 2024-07-31T15:14:11.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.855+0000 7ff011ffb640 1 -- 172.21.15.67:0/691658311 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0200cb410 con 0x5587c236f390 2024-07-31T15:14:11.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.855+0000 7ff011ffb640 1 -- 172.21.15.67:0/691658311 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0201021d0 con 0x5587c236f390 2024-07-31T15:14:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.856+0000 7ff038e0e740 1 -- 172.21.15.67:0/691658311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c236f390 msgr2=0x5587c2731d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.856+0000 7ff038e0e740 1 --2- 172.21.15.67:0/691658311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c236f390 0x5587c2731d00 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7ff0200c8680 tx=0x7ff020000bc0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.856+0000 7ff038e0e740 1 -- 172.21.15.67:0/691658311 shutdown_connections 2024-07-31T15:14:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.856+0000 7ff038e0e740 1 --2- 172.21.15.67:0/691658311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c2390070 0x5587c2732240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.856+0000 7ff038e0e740 1 --2- 172.21.15.67:0/691658311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c236f390 0x5587c2731d00 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.856+0000 7ff038e0e740 1 --2- 172.21.15.67:0/691658311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c23671b0 0x5587c274cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.856+0000 7ff038e0e740 1 -- 172.21.15.67:0/691658311 >> 172.21.15.67:0/691658311 conn(0x5587c2384460 msgr2=0x5587c273c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.856+0000 7ff038e0e740 1 -- 172.21.15.67:0/691658311 shutdown_connections 2024-07-31T15:14:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.856+0000 7ff038e0e740 1 -- 172.21.15.67:0/691658311 wait complete. 2024-07-31T15:14:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.856+0000 7ff038e0e740 1 -- start start 2024-07-31T15:14:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff038e0e740 1 Processor -- start 2024-07-31T15:14:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff038e0e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c23671b0 0x5587c2744900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff038e0e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c236f390 0x5587c2744e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff038e0e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c2390070 0x5587c27b9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff038e0e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1afb00 con 0x5587c2390070 2024-07-31T15:14:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff038e0e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1af430 con 0x5587c23671b0 2024-07-31T15:14:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff038e0e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1ac690 con 0x5587c236f390 2024-07-31T15:14:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff038e0e740 10 client.0 _ll_register_callbacks cb 0x7ff01c00c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff02b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c236f390 0x5587c2744e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff030e5a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c2390070 0x5587c27b9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff02b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c236f390 0x5587c2744e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:55142/0 (socket says 172.21.15.67:55142) 2024-07-31T15:14:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff02b7fe640 1 -- 172.21.15.67:0/3051180954 learned_addr learned my addr 172.21.15.67:0/3051180954 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff02bfff640 1 --2- 172.21.15.67:0/3051180954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c23671b0 0x5587c2744900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff02b7fe640 1 -- 172.21.15.67:0/3051180954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c23671b0 msgr2=0x5587c2744900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.857+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/3051180954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c23671b0 0x5587c2744900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.858+0000 7ff02b7fe640 1 -- 172.21.15.67:0/3051180954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c2390070 msgr2=0x5587c27b9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.858+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/3051180954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c2390070 0x5587c27b9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.858+0000 7ff02b7fe640 1 -- 172.21.15.67:0/3051180954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587c2741c00 con 0x5587c236f390 2024-07-31T15:14:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.858+0000 7ff02bfff640 1 --2- 172.21.15.67:0/3051180954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c23671b0 0x5587c2744900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:14:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.858+0000 7ff02b7fe640 1 --2- 172.21.15.67:0/3051180954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c236f390 0x5587c2744e40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff020102000 tx=0x7ff020009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20b3715fc6e348f2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.858+0000 7ff00b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:14:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.858+0000 7ff00b7fe640 1 -- 172.21.15.67:0/3051180954 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02008f020 con 0x5587c236f390 2024-07-31T15:14:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.858+0000 7ff00b7fe640 1 -- 172.21.15.67:0/3051180954 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9483+0+0 (secure 0 0 0) 0x7ff0201021d0 con 0x5587c236f390 2024-07-31T15:14:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.858+0000 7ff038e0e740 1 -- 172.21.15.67:0/3051180954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0200cb410 con 0x5587c236f390 2024-07-31T15:14:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.858+0000 7ff00b7fe640 1 -- 172.21.15.67:0/3051180954 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7ff020049070 con 0x5587c236f390 2024-07-31T15:14:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.858+0000 7ff038e0e740 10 client.29698 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.858+0000 7ff038e0e740 1 -- 172.21.15.67:0/3051180954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5587c27bb2f0 con 0x5587c236f390 2024-07-31T15:14:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.859+0000 7ff00b7fe640 1 -- 172.21.15.67:0/3051180954 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0200cb410 con 0x5587c236f390 2024-07-31T15:14:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.859+0000 7ff00b7fe640 1 -- 172.21.15.67:0/3051180954 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7ff0200d4030 con 0x5587c236f390 2024-07-31T15:14:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.859+0000 7ff038e0e740 10 client.29698 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.859+0000 7ff038e0e740 10 client.29698 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:14:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.859+0000 7ff038e0e740 20 client.29698 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.859+0000 7ff038e0e740 1 --2- 172.21.15.67:0/3051180954 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5587c2323da0 0x5587c27bda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.859+0000 7ff038e0e740 4 client.29698 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:14:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.859+0000 7ff038e0e740 1 -- 172.21.15.67:0/3051180954 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587c26a95a0 con 0x5587c2323da0 2024-07-31T15:14:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.859+0000 7ff02bfff640 1 --2- 172.21.15.67:0/3051180954 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5587c2323da0 0x5587c27bda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.860+0000 7ff00b7fe640 10 client.29698 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:14:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.860+0000 7ff02bfff640 1 --2- 172.21.15.67:0/3051180954 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5587c2323da0 0x5587c27bda20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51fadadf9084fdcb server_cookie=f689ced2c8af9104 in_seq=0 out_seq=0 2024-07-31T15:14:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.860+0000 7ff00b7fe640 1 -- 172.21.15.67:0/3051180954 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587c26a95a0 con 0x5587c2323da0 2024-07-31T15:14:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.860+0000 7ff00b7fe640 10 client.29698 handle_command_reply: tid=0 2024-07-31T15:14:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.860+0000 7ff038e0e740 1 client.29698 shutdown 2024-07-31T15:14:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.860+0000 7ff038e0e740 1 -- 172.21.15.67:0/3051180954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c236f390 msgr2=0x5587c2744e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.860+0000 7ff038e0e740 1 --2- 172.21.15.67:0/3051180954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c236f390 0x5587c2744e40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff020102000 tx=0x7ff020009680 comp rx=0 tx=0).stop 2024-07-31T15:14:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.860+0000 7ff038e0e740 1 -- 172.21.15.67:0/3051180954 shutdown_connections 2024-07-31T15:14:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.860+0000 7ff038e0e740 1 --2- 172.21.15.67:0/3051180954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5587c2390070 0x5587c27b9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.860+0000 7ff038e0e740 1 --2- 172.21.15.67:0/3051180954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587c236f390 0x5587c2744e40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.860+0000 7ff038e0e740 1 --2- 172.21.15.67:0/3051180954 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5587c2323da0 0x5587c27bda20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.860+0000 7ff038e0e740 1 --2- 172.21.15.67:0/3051180954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5587c23671b0 0x5587c2744900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.860+0000 7ff00b7fe640 0 client.29698 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:14:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.860+0000 7ff038e0e740 1 -- 172.21.15.67:0/3051180954 >> 172.21.15.67:0/3051180954 conn(0x5587c2384460 msgr2=0x5587c2731d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.861+0000 7ff038e0e740 1 -- 172.21.15.67:0/3051180954 shutdown_connections 2024-07-31T15:14:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.861+0000 7ff038e0e740 1 -- 172.21.15.67:0/3051180954 wait complete. 2024-07-31T15:14:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.861+0000 7ff038e0e740 20 client.29698 trim_cache size 0 max 16384 2024-07-31T15:14:11.874 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:14:11.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.861+0000 7ff0328bd640 1 -- 172.21.15.67:0/741311413 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff0140b3d40 msgr2=0x7ff0140b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:11.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.861+0000 7ff0328bd640 1 --2- 172.21.15.67:0/741311413 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff0140b3d40 0x7ff0140b6250 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x5587c2734310 tx=0x7ff01c00c430 comp rx=0 tx=0).stop 2024-07-31T15:14:11.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.861+0000 7ff0328bd640 1 -- 172.21.15.67:0/741311413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108ed0 msgr2=0x7ff02c19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:11.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.861+0000 7ff0328bd640 1 --2- 172.21.15.67:0/741311413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108ed0 0x7ff02c19a480 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7ff02002a930 tx=0x7ff020004270 comp rx=0 tx=0).stop 2024-07-31T15:14:11.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.862+0000 7ff0328bd640 1 -- 172.21.15.67:0/741311413 shutdown_connections 2024-07-31T15:14:11.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.862+0000 7ff0328bd640 1 --2- 172.21.15.67:0/741311413 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff0140b3d40 0x7ff0140b6250 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.862+0000 7ff0328bd640 1 --2- 172.21.15.67:0/741311413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff02c109820 0x7ff02c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.862+0000 7ff0328bd640 1 --2- 172.21.15.67:0/741311413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff02c108ed0 0x7ff02c19a480 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.862+0000 7ff0328bd640 1 --2- 172.21.15.67:0/741311413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff02c103590 0x7ff02c199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.862+0000 7ff0328bd640 1 -- 172.21.15.67:0/741311413 >> 172.21.15.67:0/741311413 conn(0x7ff02c07b280 msgr2=0x7ff02c107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.862+0000 7ff0328bd640 1 -- 172.21.15.67:0/741311413 shutdown_connections 2024-07-31T15:14:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:11.862+0000 7ff0328bd640 1 -- 172.21.15.67:0/741311413 wait complete. 2024-07-31T15:14:13.154 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:14:13.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T15:14:13.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:13 smithi067 ceph-mon[32750]: pgmap v1139: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 20 MiB/s wr, 43 op/s 2024-07-31T15:14:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:13 smithi077 ceph-mon[37327]: pgmap v1139: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 20 MiB/s wr, 43 op/s 2024-07-31T15:14:13.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:13 smithi073 ceph-mon[36714]: pgmap v1139: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 20 MiB/s wr, 43 op/s 2024-07-31T15:14:13.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.674+0000 7f7dddd64640 1 -- 172.21.15.67:0/2992999285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd8107dd0 msgr2=0x7f7dd81081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:13.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.674+0000 7f7dddd64640 1 --2- 172.21.15.67:0/2992999285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd8107dd0 0x7f7dd81081e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009900 tx=0x7f7dc802a410 comp rx=0 tx=0).stop 2024-07-31T15:14:13.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.674+0000 7f7dddd64640 1 -- 172.21.15.67:0/2992999285 shutdown_connections 2024-07-31T15:14:13.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.674+0000 7f7dddd64640 1 --2- 172.21.15.67:0/2992999285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dd8108720 0x7f7dd810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.674+0000 7f7dddd64640 1 --2- 172.21.15.67:0/2992999285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd8107dd0 0x7f7dd81081e0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.674+0000 7f7dddd64640 1 --2- 172.21.15.67:0/2992999285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dd8102490 0x7f7dd81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.674+0000 7f7dddd64640 1 -- 172.21.15.67:0/2992999285 >> 172.21.15.67:0/2992999285 conn(0x7f7dd8079160 msgr2=0x7f7dd8079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:13.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.674+0000 7f7dddd64640 1 -- 172.21.15.67:0/2992999285 shutdown_connections 2024-07-31T15:14:13.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.674+0000 7f7dddd64640 1 -- 172.21.15.67:0/2992999285 wait complete. 2024-07-31T15:14:13.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.675+0000 7f7dddd64640 1 Processor -- start 2024-07-31T15:14:13.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.675+0000 7f7dddd64640 1 -- start start 2024-07-31T15:14:13.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dddd64640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd8102490 0x7f7dd8199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dddd64640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dd8107dd0 0x7f7dd819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dddd64640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dd8108720 0x7f7dd819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dddd64640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd8112370 con 0x7f7dd8102490 2024-07-31T15:14:13.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dddd64640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81121f0 con 0x7f7dd8107dd0 2024-07-31T15:14:13.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dddd64640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81124f0 con 0x7f7dd8108720 2024-07-31T15:14:13.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dd77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd8102490 0x7f7dd8199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dd77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd8102490 0x7f7dd8199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34396/0 (socket says 172.21.15.67:34396) 2024-07-31T15:14:13.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dd77fe640 1 -- 172.21.15.67:0/1204133672 learned_addr learned my addr 172.21.15.67:0/1204133672 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:13.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dd6ffd640 1 --2- 172.21.15.67:0/1204133672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dd8107dd0 0x7f7dd819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dd7fff640 1 --2- 172.21.15.67:0/1204133672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dd8108720 0x7f7dd819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dd77fe640 1 -- 172.21.15.67:0/1204133672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dd8108720 msgr2=0x7f7dd819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:13.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dd77fe640 1 --2- 172.21.15.67:0/1204133672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dd8108720 0x7f7dd819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dd77fe640 1 -- 172.21.15.67:0/1204133672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dd8107dd0 msgr2=0x7f7dd819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:13.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dd77fe640 1 --2- 172.21.15.67:0/1204133672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dd8107dd0 0x7f7dd819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.676+0000 7f7dd77fe640 1 -- 172.21.15.67:0/1204133672 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd819fea0 con 0x7f7dd8102490 2024-07-31T15:14:13.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.677+0000 7f7dd77fe640 1 --2- 172.21.15.67:0/1204133672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd8102490 0x7f7dd8199f30 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000d950 tx=0x7f7dc000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:13.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.677+0000 7f7dd4ff9640 1 -- 172.21.15.67:0/1204133672 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0017070 con 0x7f7dd8102490 2024-07-31T15:14:13.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.677+0000 7f7dddd64640 1 -- 172.21.15.67:0/1204133672 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd81a0190 con 0x7f7dd8102490 2024-07-31T15:14:13.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.677+0000 7f7dd4ff9640 1 -- 172.21.15.67:0/1204133672 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dc0009c30 con 0x7f7dd8102490 2024-07-31T15:14:13.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.677+0000 7f7dd4ff9640 1 -- 172.21.15.67:0/1204133672 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc000f620 con 0x7f7dd8102490 2024-07-31T15:14:13.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.677+0000 7f7dddd64640 1 -- 172.21.15.67:0/1204133672 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd81a68f0 con 0x7f7dd8102490 2024-07-31T15:14:13.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.680+0000 7f7dd4ff9640 1 -- 172.21.15.67:0/1204133672 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7dc000f7c0 con 0x7f7dd8102490 2024-07-31T15:14:13.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.680+0000 7f7de42b5740 1 Processor -- start 2024-07-31T15:14:13.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.680+0000 7f7de42b5740 1 -- start start 2024-07-31T15:14:13.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.680+0000 7f7de42b5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f792d0 0x55701731e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.680+0000 7f7de42b5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f50410 0x55701731eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.680+0000 7f7de42b5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f585f0 0x55701731f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.680+0000 7f7de42b5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557017294ac0 con 0x557016f792d0 2024-07-31T15:14:13.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.680+0000 7f7de42b5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557017283fc0 con 0x557016f50410 2024-07-31T15:14:13.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.680+0000 7f7de42b5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557017288000 con 0x557016f585f0 2024-07-31T15:14:13.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.681+0000 7f7dd7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f792d0 0x55701731e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.681+0000 7f7dd7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f792d0 0x55701731e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34404/0 (socket says 172.21.15.67:34404) 2024-07-31T15:14:13.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.681+0000 7f7dd7fff640 1 -- 172.21.15.67:0/699682644 learned_addr learned my addr 172.21.15.67:0/699682644 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:13.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.681+0000 7f7dd77fe640 1 --2- 172.21.15.67:0/699682644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f50410 0x55701731eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.681+0000 7f7dd6ffd640 1 --2- 172.21.15.67:0/699682644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f585f0 0x55701731f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.681+0000 7f7dd7fff640 1 -- 172.21.15.67:0/699682644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f585f0 msgr2=0x55701731f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:13.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.681+0000 7f7dd7fff640 1 --2- 172.21.15.67:0/699682644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f585f0 0x55701731f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.681+0000 7f7dd4ff9640 1 --2- 172.21.15.67:0/1204133672 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7dc40b3dc0 0x7f7dc40b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.681+0000 7f7dd7fff640 1 -- 172.21.15.67:0/699682644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f50410 msgr2=0x55701731eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:13.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.681+0000 7f7dd7fff640 1 --2- 172.21.15.67:0/699682644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f50410 0x55701731eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.681+0000 7f7dd7fff640 1 -- 172.21.15.67:0/699682644 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570172be440 con 0x557016f792d0 2024-07-31T15:14:13.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.681+0000 7f7dd7fff640 1 --2- 172.21.15.67:0/1204133672 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7dc40b3dc0 0x7f7dc40b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.681+0000 7f7dd4ff9640 1 -- 172.21.15.67:0/1204133672 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9483+0+0 (secure 0 0 0) 0x7f7dc00e3f20 con 0x7f7dd8102490 2024-07-31T15:14:13.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.682+0000 7f7dd7fff640 1 --2- 172.21.15.67:0/699682644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f792d0 0x55701731e9b0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000ae10 tx=0x7f7dd000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50f8cd80b94a979 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:13.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.682+0000 7f7dcdffb640 1 -- 172.21.15.67:0/699682644 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0018630 con 0x557016f792d0 2024-07-31T15:14:13.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.682+0000 7f7dcdffb640 1 -- 172.21.15.67:0/699682644 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd00187d0 con 0x557016f792d0 2024-07-31T15:14:13.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.682+0000 7f7dd7fff640 1 --2- 172.21.15.67:0/1204133672 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7dc40b3dc0 0x7f7dc40b62d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0021000 tx=0x7f7dd0022000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:13.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.682+0000 7f7dcdffb640 1 -- 172.21.15.67:0/699682644 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0005290 con 0x557016f792d0 2024-07-31T15:14:13.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.683+0000 7f7de42b5740 1 -- 172.21.15.67:0/699682644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f792d0 msgr2=0x55701731e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:13.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.683+0000 7f7de42b5740 1 --2- 172.21.15.67:0/699682644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f792d0 0x55701731e9b0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000ae10 tx=0x7f7dd000f790 comp rx=0 tx=0).stop 2024-07-31T15:14:13.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.683+0000 7f7de42b5740 1 -- 172.21.15.67:0/699682644 shutdown_connections 2024-07-31T15:14:13.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.683+0000 7f7de42b5740 1 --2- 172.21.15.67:0/699682644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f585f0 0x55701731f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.683+0000 7f7de42b5740 1 --2- 172.21.15.67:0/699682644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f50410 0x55701731eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.683+0000 7f7de42b5740 1 --2- 172.21.15.67:0/699682644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f792d0 0x55701731e9b0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.683+0000 7f7de42b5740 1 -- 172.21.15.67:0/699682644 >> 172.21.15.67:0/699682644 conn(0x557016f6d6c0 msgr2=0x55701731c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:13.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.683+0000 7f7de42b5740 1 -- 172.21.15.67:0/699682644 shutdown_connections 2024-07-31T15:14:13.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.683+0000 7f7de42b5740 1 -- 172.21.15.67:0/699682644 wait complete. 2024-07-31T15:14:13.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.683+0000 7f7de42b5740 1 -- start start 2024-07-31T15:14:13.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.684+0000 7f7de42b5740 1 Processor -- start 2024-07-31T15:14:13.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.684+0000 7f7de42b5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f50410 0x55701732a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.684+0000 7f7de42b5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f585f0 0x55701732aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.684+0000 7f7de42b5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f792d0 0x557017336480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.684+0000 7f7de42b5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5570170e7810 con 0x557016f50410 2024-07-31T15:14:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.684+0000 7f7de42b5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5570172269a0 con 0x557016f585f0 2024-07-31T15:14:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.684+0000 7f7de42b5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55701732d190 con 0x557016f792d0 2024-07-31T15:14:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.684+0000 7f7de42b5740 10 client.0 _ll_register_callbacks cb 0x55701722e540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.684+0000 7f7dd77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f50410 0x55701732a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.684+0000 7f7dd77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f50410 0x55701732a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34420/0 (socket says 172.21.15.67:34420) 2024-07-31T15:14:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.684+0000 7f7dd77fe640 1 -- 172.21.15.67:0/783091683 learned_addr learned my addr 172.21.15.67:0/783091683 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.684+0000 7f7dd6ffd640 1 --2- 172.21.15.67:0/783091683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f585f0 0x55701732aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.685+0000 7f7dd7fff640 1 --2- 172.21.15.67:0/783091683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f792d0 0x557017336480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.685+0000 7f7dd77fe640 1 -- 172.21.15.67:0/783091683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f792d0 msgr2=0x557017336480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.685+0000 7f7dd77fe640 1 --2- 172.21.15.67:0/783091683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f792d0 0x557017336480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.685+0000 7f7dd77fe640 1 -- 172.21.15.67:0/783091683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f585f0 msgr2=0x55701732aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.685+0000 7f7dd77fe640 1 --2- 172.21.15.67:0/783091683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f585f0 0x55701732aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.685+0000 7f7dd77fe640 1 -- 172.21.15.67:0/783091683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55701722a9f0 con 0x557016f50410 2024-07-31T15:14:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.685+0000 7f7dd6ffd640 1 --2- 172.21.15.67:0/783091683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f585f0 0x55701732aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:13.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.685+0000 7f7dd77fe640 1 --2- 172.21.15.67:0/783091683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f50410 0x55701732a570 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0005fd0 tx=0x7f7dc00e7660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ade73a2397b08b1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:13.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.685+0000 7f7db6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:14:13.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.685+0000 7f7db6ffd640 1 -- 172.21.15.67:0/783091683 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0020030 con 0x557016f50410 2024-07-31T15:14:13.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.685+0000 7f7db6ffd640 1 -- 172.21.15.67:0/783091683 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9483+0+0 (secure 0 0 0) 0x7f7dc00b3390 con 0x557016f50410 2024-07-31T15:14:13.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.685+0000 7f7de42b5740 1 -- 172.21.15.67:0/783091683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dd00187d0 con 0x557016f50410 2024-07-31T15:14:13.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.686+0000 7f7db6ffd640 1 -- 172.21.15.67:0/783091683 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f7dc00aa5f0 con 0x557016f50410 2024-07-31T15:14:13.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.686+0000 7f7de42b5740 10 client.20367 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:13.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.686+0000 7f7de42b5740 1 -- 172.21.15.67:0/783091683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x557016fe48b0 con 0x557016f50410 2024-07-31T15:14:13.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.686+0000 7f7db6ffd640 1 -- 172.21.15.67:0/783091683 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd00187d0 con 0x557016f50410 2024-07-31T15:14:13.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.686+0000 7f7db6ffd640 1 -- 172.21.15.67:0/783091683 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f7dc00a6190 con 0x557016f50410 2024-07-31T15:14:13.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.686+0000 7f7de42b5740 10 client.20367 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:13.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.686+0000 7f7de42b5740 10 client.20367 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:14:13.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.686+0000 7f7de42b5740 20 client.20367 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:13.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.686+0000 7f7de42b5740 1 --2- 172.21.15.67:0/783091683 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557016f37c50 0x55701733cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.686+0000 7f7de42b5740 4 client.20367 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:14:13.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.686+0000 7f7de42b5740 1 -- 172.21.15.67:0/783091683 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557017296a10 con 0x557016f37c50 2024-07-31T15:14:13.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.687+0000 7f7dd6ffd640 1 --2- 172.21.15.67:0/783091683 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557016f37c50 0x55701733cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.687+0000 7f7db6ffd640 10 client.20367 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:13.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.687+0000 7f7dd6ffd640 1 --2- 172.21.15.67:0/783091683 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557016f37c50 0x55701733cd30 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d2f650330666a4a server_cookie=5700b47d029182f9 in_seq=0 out_seq=0 2024-07-31T15:14:13.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.688+0000 7f7db6ffd640 1 -- 172.21.15.67:0/783091683 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557017296a10 con 0x557016f37c50 2024-07-31T15:14:13.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.688+0000 7f7db6ffd640 10 client.20367 handle_command_reply: tid=0 2024-07-31T15:14:13.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.688+0000 7f7de42b5740 1 client.20367 shutdown 2024-07-31T15:14:13.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.688+0000 7f7de42b5740 1 -- 172.21.15.67:0/783091683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f50410 msgr2=0x55701732a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:13.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.688+0000 7f7de42b5740 1 --2- 172.21.15.67:0/783091683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f50410 0x55701732a570 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0005fd0 tx=0x7f7dc00e7660 comp rx=0 tx=0).stop 2024-07-31T15:14:13.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.688+0000 7f7de42b5740 1 -- 172.21.15.67:0/783091683 shutdown_connections 2024-07-31T15:14:13.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.688+0000 7f7de42b5740 1 --2- 172.21.15.67:0/783091683 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557016f37c50 0x55701733cd30 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.688+0000 7f7de42b5740 1 --2- 172.21.15.67:0/783091683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f792d0 0x557017336480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.688+0000 7f7de42b5740 1 --2- 172.21.15.67:0/783091683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f585f0 0x55701732aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.688+0000 7f7de42b5740 1 --2- 172.21.15.67:0/783091683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f50410 0x55701732a570 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.688+0000 7f7de42b5740 1 -- 172.21.15.67:0/783091683 >> 172.21.15.67:0/783091683 conn(0x557016f6d6c0 msgr2=0x557017321b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:13.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.688+0000 7f7db6ffd640 0 client.20367 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:13.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.688+0000 7f7de42b5740 1 -- 172.21.15.67:0/783091683 shutdown_connections 2024-07-31T15:14:13.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.689+0000 7f7de42b5740 1 -- 172.21.15.67:0/783091683 wait complete. 2024-07-31T15:14:13.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.689+0000 7f7de42b5740 20 client.20367 trim_cache size 0 max 16384 2024-07-31T15:14:13.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.712+0000 7f7de42b5740 1 Processor -- start 2024-07-31T15:14:13.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.712+0000 7f7de42b5740 1 -- start start 2024-07-31T15:14:13.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7de42b5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f50410 0x55701732c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7de42b5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f585f0 0x5570173256e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7de42b5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f792d0 0x557017342530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7de42b5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81b09c0 con 0x557016f50410 2024-07-31T15:14:13.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7de42b5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81b2500 con 0x557016f585f0 2024-07-31T15:14:13.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7de42b5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81b1090 con 0x557016f792d0 2024-07-31T15:14:13.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7dd77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f50410 0x55701732c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7dd77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f50410 0x55701732c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34434/0 (socket says 172.21.15.67:34434) 2024-07-31T15:14:13.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7dd77fe640 1 -- 172.21.15.67:0/4048892746 learned_addr learned my addr 172.21.15.67:0/4048892746 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:13.716 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:14:13.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7dd6ffd640 1 --2- 172.21.15.67:0/4048892746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f585f0 0x5570173256e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7dd7fff640 1 --2- 172.21.15.67:0/4048892746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f792d0 0x557017342530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7dd77fe640 1 -- 172.21.15.67:0/4048892746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f792d0 msgr2=0x557017342530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:13.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7dd77fe640 1 --2- 172.21.15.67:0/4048892746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f792d0 0x557017342530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7dd77fe640 1 -- 172.21.15.67:0/4048892746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f585f0 msgr2=0x5570173256e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:13.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7dd77fe640 1 --2- 172.21.15.67:0/4048892746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f585f0 0x5570173256e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7dd77fe640 1 -- 172.21.15.67:0/4048892746 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557017322190 con 0x557016f50410 2024-07-31T15:14:13.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7dd7fff640 1 --2- 172.21.15.67:0/4048892746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f792d0 0x557017342530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:14:13.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.713+0000 7f7dd77fe640 1 --2- 172.21.15.67:0/4048892746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f50410 0x55701732c250 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00a7a90 tx=0x7f7dc000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7785cfe4061186c0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:13.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.714+0000 7f7dcd7fa640 1 -- 172.21.15.67:0/4048892746 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0020070 con 0x557016f50410 2024-07-31T15:14:13.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.714+0000 7f7dcd7fa640 1 -- 172.21.15.67:0/4048892746 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dc00aa330 con 0x557016f50410 2024-07-31T15:14:13.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.714+0000 7f7dcd7fa640 1 -- 172.21.15.67:0/4048892746 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00a9f90 con 0x557016f50410 2024-07-31T15:14:13.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.715+0000 7f7de42b5740 1 -- 172.21.15.67:0/4048892746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f50410 msgr2=0x55701732c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:13.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.715+0000 7f7de42b5740 1 --2- 172.21.15.67:0/4048892746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f50410 0x55701732c250 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00a7a90 tx=0x7f7dc000bf60 comp rx=0 tx=0).stop 2024-07-31T15:14:13.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.715+0000 7f7de42b5740 1 -- 172.21.15.67:0/4048892746 shutdown_connections 2024-07-31T15:14:13.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.715+0000 7f7de42b5740 1 --2- 172.21.15.67:0/4048892746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f792d0 0x557017342530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.715+0000 7f7de42b5740 1 --2- 172.21.15.67:0/4048892746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f585f0 0x5570173256e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.715+0000 7f7de42b5740 1 --2- 172.21.15.67:0/4048892746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f50410 0x55701732c250 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.715+0000 7f7de42b5740 1 -- 172.21.15.67:0/4048892746 >> 172.21.15.67:0/4048892746 conn(0x557016f6d6c0 msgr2=0x557017328bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:13.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.715+0000 7f7de42b5740 1 -- 172.21.15.67:0/4048892746 shutdown_connections 2024-07-31T15:14:13.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.715+0000 7f7de42b5740 1 -- 172.21.15.67:0/4048892746 wait complete. 2024-07-31T15:14:13.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.716+0000 7f7de42b5740 1 -- start start 2024-07-31T15:14:13.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.716+0000 7f7de42b5740 1 Processor -- start 2024-07-31T15:14:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7de42b5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f50410 0x557017332ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7de42b5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f585f0 0x55701732e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7de42b5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f792d0 0x55701732e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7de42b5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81b1e30 con 0x557016f792d0 2024-07-31T15:14:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7de42b5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81b1760 con 0x557016f50410 2024-07-31T15:14:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7de42b5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81ae9c0 con 0x557016f585f0 2024-07-31T15:14:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7de42b5740 10 client.0 _ll_register_callbacks cb 0x7f7dc00e1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7dd7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f792d0 0x55701732e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7dd7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f792d0 0x55701732e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34440/0 (socket says 172.21.15.67:34440) 2024-07-31T15:14:13.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7dd7fff640 1 -- 172.21.15.67:0/3307947308 learned_addr learned my addr 172.21.15.67:0/3307947308 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:13.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7dd77fe640 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f50410 0x557017332ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7dd6ffd640 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f585f0 0x55701732e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7dd6ffd640 1 -- 172.21.15.67:0/3307947308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f50410 msgr2=0x557017332ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:13.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7dd6ffd640 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f50410 0x557017332ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7dd6ffd640 1 -- 172.21.15.67:0/3307947308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f792d0 msgr2=0x55701732e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:13.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7dd6ffd640 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f792d0 0x55701732e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7dd6ffd640 1 -- 172.21.15.67:0/3307947308 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570173a4500 con 0x557016f585f0 2024-07-31T15:14:13.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7dd77fe640 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f50410 0x557017332ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:13.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.717+0000 7f7dd7fff640 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f792d0 0x55701732e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:13.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.718+0000 7f7dd6ffd640 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f585f0 0x55701732e170 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8006010 tx=0x7f7dc8004210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a020859cede3d69 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:13.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.718+0000 7f7db6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:14:13.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.718+0000 7f7db6ffd640 1 -- 172.21.15.67:0/3307947308 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8035040 con 0x557016f585f0 2024-07-31T15:14:13.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.718+0000 7f7db6ffd640 1 -- 172.21.15.67:0/3307947308 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9483+0+0 (secure 0 0 0) 0x7f7dc8005240 con 0x557016f585f0 2024-07-31T15:14:13.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.718+0000 7f7de42b5740 1 -- 172.21.15.67:0/3307947308 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dc00aa330 con 0x557016f585f0 2024-07-31T15:14:13.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.718+0000 7f7db6ffd640 1 -- 172.21.15.67:0/3307947308 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f7dc8005440 con 0x557016f585f0 2024-07-31T15:14:13.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.719+0000 7f7de42b5740 10 client.29710 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:13.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.719+0000 7f7de42b5740 1 -- 172.21.15.67:0/3307947308 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5570173a99d0 con 0x557016f585f0 2024-07-31T15:14:13.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.719+0000 7f7db6ffd640 1 -- 172.21.15.67:0/3307947308 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dc80055e0 con 0x557016f585f0 2024-07-31T15:14:13.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.719+0000 7f7db6ffd640 1 -- 172.21.15.67:0/3307947308 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f7dc80039e0 con 0x557016f585f0 2024-07-31T15:14:13.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.719+0000 7f7de42b5740 10 client.29710 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:13.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.719+0000 7f7de42b5740 10 client.29710 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:14:13.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.719+0000 7f7de42b5740 20 client.29710 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:13.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.719+0000 7f7de42b5740 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557016f0d000 0x5570173ac100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.719+0000 7f7de42b5740 4 client.29710 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-07-31T15:14:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.719+0000 7f7de42b5740 1 -- 172.21.15.67:0/3307947308 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x7f7dc00aa330 con 0x557016f0d000 2024-07-31T15:14:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.720+0000 7f7dd77fe640 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557016f0d000 0x5570173ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.720+0000 7f7dd77fe640 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557016f0d000 0x5570173ac100 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8327611a9014f33c server_cookie=e164b37e2ef915c1 in_seq=0 out_seq=0 2024-07-31T15:14:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.720+0000 7f7db6ffd640 10 client.29710 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.721+0000 7f7db6ffd640 1 -- 172.21.15.67:0/3307947308 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x7f7dc00aa330 con 0x557016f0d000 2024-07-31T15:14:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.721+0000 7f7db6ffd640 10 client.29710 handle_command_reply: tid=0 2024-07-31T15:14:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.721+0000 7f7de42b5740 1 client.29710 shutdown 2024-07-31T15:14:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.721+0000 7f7de42b5740 1 -- 172.21.15.67:0/3307947308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f585f0 msgr2=0x55701732e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.721+0000 7f7de42b5740 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f585f0 0x55701732e170 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8006010 tx=0x7f7dc8004210 comp rx=0 tx=0).stop 2024-07-31T15:14:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.721+0000 7f7de42b5740 1 -- 172.21.15.67:0/3307947308 shutdown_connections 2024-07-31T15:14:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.721+0000 7f7de42b5740 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557016f0d000 0x5570173ac100 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.721+0000 7f7de42b5740 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557016f792d0 0x55701732e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.721+0000 7f7de42b5740 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557016f585f0 0x55701732e170 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.721+0000 7f7de42b5740 1 --2- 172.21.15.67:0/3307947308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557016f50410 0x557017332ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.721+0000 7f7de42b5740 1 -- 172.21.15.67:0/3307947308 >> 172.21.15.67:0/3307947308 conn(0x557016f6d6c0 msgr2=0x55701732b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:13.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.721+0000 7f7db6ffd640 0 client.29710 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:13.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.721+0000 7f7de42b5740 1 -- 172.21.15.67:0/3307947308 shutdown_connections 2024-07-31T15:14:13.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.721+0000 7f7de42b5740 1 -- 172.21.15.67:0/3307947308 wait complete. 2024-07-31T15:14:13.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.722+0000 7f7de42b5740 20 client.29710 trim_cache size 0 max 16384 2024-07-31T15:14:13.730 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:14:13.731 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 66, 2024-07-31T15:14:13.731 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:14:13.731 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 64, 2024-07-31T15:14:13.731 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": {} 2024-07-31T15:14:13.731 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:14:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.722+0000 7f7dddd64640 1 -- 172.21.15.67:0/1204133672 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7dc40b3dc0 msgr2=0x7f7dc40b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.722+0000 7f7dddd64640 1 --2- 172.21.15.67:0/1204133672 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7dc40b3dc0 0x7f7dc40b62d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0021000 tx=0x7f7dd0022000 comp rx=0 tx=0).stop 2024-07-31T15:14:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.722+0000 7f7dddd64640 1 -- 172.21.15.67:0/1204133672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd8102490 msgr2=0x7f7dd8199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.722+0000 7f7dddd64640 1 --2- 172.21.15.67:0/1204133672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd8102490 0x7f7dd8199f30 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000d950 tx=0x7f7dc000de10 comp rx=0 tx=0).stop 2024-07-31T15:14:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.722+0000 7f7dddd64640 1 -- 172.21.15.67:0/1204133672 shutdown_connections 2024-07-31T15:14:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.722+0000 7f7dddd64640 1 --2- 172.21.15.67:0/1204133672 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7dc40b3dc0 0x7f7dc40b62d0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.722+0000 7f7dddd64640 1 --2- 172.21.15.67:0/1204133672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dd8108720 0x7f7dd819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.722+0000 7f7dddd64640 1 --2- 172.21.15.67:0/1204133672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dd8107dd0 0x7f7dd819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.722+0000 7f7dddd64640 1 --2- 172.21.15.67:0/1204133672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dd8102490 0x7f7dd8199f30 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:13.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.722+0000 7f7dddd64640 1 -- 172.21.15.67:0/1204133672 >> 172.21.15.67:0/1204133672 conn(0x7f7dd8079160 msgr2=0x7f7dd80ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:13.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.723+0000 7f7dddd64640 1 -- 172.21.15.67:0/1204133672 shutdown_connections 2024-07-31T15:14:13.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:13.723+0000 7f7dddd64640 1 -- 172.21.15.67:0/1204133672 wait complete. 2024-07-31T15:14:15.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:15 smithi077 ceph-mon[37327]: pgmap v1140: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 17 MiB/s wr, 38 op/s 2024-07-31T15:14:15.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:15 smithi067 ceph-mon[32750]: pgmap v1140: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 17 MiB/s wr, 38 op/s 2024-07-31T15:14:15.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:15 smithi073 ceph-mon[36714]: pgmap v1140: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 17 MiB/s wr, 38 op/s 2024-07-31T15:14:16.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:16 smithi067 ceph-mon[32750]: osdmap e136: 12 total, 12 up, 12 in 2024-07-31T15:14:16.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:16 smithi073 ceph-mon[36714]: osdmap e136: 12 total, 12 up, 12 in 2024-07-31T15:14:16.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:16 smithi077 ceph-mon[37327]: osdmap e136: 12 total, 12 up, 12 in 2024-07-31T15:14:17.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:17 smithi067 ceph-mon[32750]: pgmap v1142: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 7.5 MiB/s wr, 31 op/s 2024-07-31T15:14:17.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:17 smithi073 ceph-mon[36714]: pgmap v1142: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 7.5 MiB/s wr, 31 op/s 2024-07-31T15:14:17.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:17 smithi077 ceph-mon[37327]: pgmap v1142: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 7.5 MiB/s wr, 31 op/s 2024-07-31T15:14:17.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.258+0000 7f942dd98640 1 -- 172.21.15.67:0/866180188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94281024b0 msgr2=0x7f94281028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:17.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.258+0000 7f942dd98640 1 --2- 172.21.15.67:0/866180188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94281024b0 0x7f94281028c0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f9418009900 tx=0x7f941802a410 comp rx=0 tx=0).stop 2024-07-31T15:14:17.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.258+0000 7f942dd98640 1 -- 172.21.15.67:0/866180188 shutdown_connections 2024-07-31T15:14:17.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.258+0000 7f942dd98640 1 --2- 172.21.15.67:0/866180188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9428108740 0x7f942810ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.258+0000 7f942dd98640 1 --2- 172.21.15.67:0/866180188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9428107df0 0x7f9428108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.258+0000 7f942dd98640 1 --2- 172.21.15.67:0/866180188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94281024b0 0x7f94281028c0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.258+0000 7f942dd98640 1 -- 172.21.15.67:0/866180188 >> 172.21.15.67:0/866180188 conn(0x7f9428079190 msgr2=0x7f94280795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.258+0000 7f942dd98640 1 -- 172.21.15.67:0/866180188 shutdown_connections 2024-07-31T15:14:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.258+0000 7f942dd98640 1 -- 172.21.15.67:0/866180188 wait complete. 2024-07-31T15:14:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.259+0000 7f942dd98640 1 Processor -- start 2024-07-31T15:14:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.259+0000 7f942dd98640 1 -- start start 2024-07-31T15:14:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f942dd98640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94281024b0 0x7f942819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f942dd98640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9428107df0 0x7f942819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f942dd98640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9428108740 0x7f942819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f942dd98640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f94281123c0 con 0x7f94281024b0 2024-07-31T15:14:17.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f942dd98640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9428112240 con 0x7f9428107df0 2024-07-31T15:14:17.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f942dd98640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9428112540 con 0x7f9428108740 2024-07-31T15:14:17.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f94277fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94281024b0 0x7f942819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f94277fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94281024b0 0x7f942819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34448/0 (socket says 172.21.15.67:34448) 2024-07-31T15:14:17.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f94277fe640 1 -- 172.21.15.67:0/3219212251 learned_addr learned my addr 172.21.15.67:0/3219212251 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:17.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f9427fff640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9428108740 0x7f942819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f9426ffd640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9428107df0 0x7f942819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f94277fe640 1 -- 172.21.15.67:0/3219212251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9428108740 msgr2=0x7f942819b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:17.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f94277fe640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9428108740 0x7f942819b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f94277fe640 1 -- 172.21.15.67:0/3219212251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9428107df0 msgr2=0x7f942819b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:17.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.260+0000 7f94277fe640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9428107df0 0x7f942819b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.261+0000 7f94277fe640 1 -- 172.21.15.67:0/3219212251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94281a5fa0 con 0x7f94281024b0 2024-07-31T15:14:17.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.261+0000 7f9427fff640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9428108740 0x7f942819b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.261+0000 7f94277fe640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94281024b0 0x7f942819ad10 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f9418005940 tx=0x7f94180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.261+0000 7f9424ff9640 1 -- 172.21.15.67:0/3219212251 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941803f030 con 0x7f94281024b0 2024-07-31T15:14:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.261+0000 7f9424ff9640 1 -- 172.21.15.67:0/3219212251 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9418003cb0 con 0x7f94281024b0 2024-07-31T15:14:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.261+0000 7f942dd98640 1 -- 172.21.15.67:0/3219212251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94281a6230 con 0x7f94281024b0 2024-07-31T15:14:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.261+0000 7f9424ff9640 1 -- 172.21.15.67:0/3219212251 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418035070 con 0x7f94281024b0 2024-07-31T15:14:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.261+0000 7f942dd98640 1 -- 172.21.15.67:0/3219212251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94281a6760 con 0x7f94281024b0 2024-07-31T15:14:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.264+0000 7f9424ff9640 1 -- 172.21.15.67:0/3219212251 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f9418042030 con 0x7f94281024b0 2024-07-31T15:14:17.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.265+0000 7f9424ff9640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94100b3df0 0x7f94100b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.265+0000 7f9426ffd640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94100b3df0 0x7f94100b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.265+0000 7f9424ff9640 1 -- 172.21.15.67:0/3219212251 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f9418104760 con 0x7f94281024b0 2024-07-31T15:14:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.265+0000 7f94342e9740 1 Processor -- start 2024-07-31T15:14:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.265+0000 7f9426ffd640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94100b3df0 0x7f94100b6300 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9428100640 tx=0x7f941c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.266+0000 7f94342e9740 1 -- start start 2024-07-31T15:14:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.266+0000 7f94342e9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6b01a850 0x557e6b3c0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.266+0000 7f94342e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6aff1990 0x557e6b3c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.266+0000 7f94342e9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff9b70 0x557e6b3c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.266+0000 7f94342e9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557e6b335690 con 0x557e6aff9b70 2024-07-31T15:14:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.266+0000 7f94342e9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557e6b324b90 con 0x557e6b01a850 2024-07-31T15:14:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.266+0000 7f94342e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557e6b328bd0 con 0x557e6aff1990 2024-07-31T15:14:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.266+0000 7f9427fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff9b70 0x557e6b3c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.266+0000 7f9427fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff9b70 0x557e6b3c0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34454/0 (socket says 172.21.15.67:34454) 2024-07-31T15:14:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.266+0000 7f9427fff640 1 -- 172.21.15.67:0/3565047479 learned_addr learned my addr 172.21.15.67:0/3565047479 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.266+0000 7f9426ffd640 1 --2- 172.21.15.67:0/3565047479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6aff1990 0x557e6b3c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.266+0000 7f94277fe640 1 --2- 172.21.15.67:0/3565047479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6b01a850 0x557e6b3c0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.267+0000 7f9426ffd640 1 -- 172.21.15.67:0/3565047479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6b01a850 msgr2=0x557e6b3c0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:17.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.267+0000 7f9426ffd640 1 --2- 172.21.15.67:0/3565047479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6b01a850 0x557e6b3c0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.267+0000 7f9426ffd640 1 -- 172.21.15.67:0/3565047479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff9b70 msgr2=0x557e6b3c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:17.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.267+0000 7f9426ffd640 1 --2- 172.21.15.67:0/3565047479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff9b70 0x557e6b3c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.267+0000 7f9426ffd640 1 -- 172.21.15.67:0/3565047479 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6b2b0a10 con 0x557e6aff1990 2024-07-31T15:14:17.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.267+0000 7f9427fff640 1 --2- 172.21.15.67:0/3565047479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff9b70 0x557e6b3c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:17.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.267+0000 7f9426ffd640 1 --2- 172.21.15.67:0/3565047479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6aff1990 0x557e6b3c0a50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f942806a2c0 tx=0x7f941c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a57e90d81ab71d62 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:17.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.267+0000 7f940dffb640 1 -- 172.21.15.67:0/3565047479 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c011ce0 con 0x557e6aff1990 2024-07-31T15:14:17.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.268+0000 7f940dffb640 1 -- 172.21.15.67:0/3565047479 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f941c0054d0 con 0x557e6aff1990 2024-07-31T15:14:17.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.268+0000 7f940dffb640 1 -- 172.21.15.67:0/3565047479 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c01e420 con 0x557e6aff1990 2024-07-31T15:14:17.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.269+0000 7f94342e9740 1 -- 172.21.15.67:0/3565047479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6aff1990 msgr2=0x557e6b3c0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:17.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.269+0000 7f94342e9740 1 --2- 172.21.15.67:0/3565047479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6aff1990 0x557e6b3c0a50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f942806a2c0 tx=0x7f941c013000 comp rx=0 tx=0).stop 2024-07-31T15:14:17.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.269+0000 7f94342e9740 1 -- 172.21.15.67:0/3565047479 shutdown_connections 2024-07-31T15:14:17.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.269+0000 7f94342e9740 1 --2- 172.21.15.67:0/3565047479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff9b70 0x557e6b3c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.269+0000 7f94342e9740 1 --2- 172.21.15.67:0/3565047479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6aff1990 0x557e6b3c0a50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.269+0000 7f94342e9740 1 --2- 172.21.15.67:0/3565047479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6b01a850 0x557e6b3c0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.269+0000 7f94342e9740 1 -- 172.21.15.67:0/3565047479 >> 172.21.15.67:0/3565047479 conn(0x557e6b00ec40 msgr2=0x557e6b3be400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:17.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.269+0000 7f94342e9740 1 -- 172.21.15.67:0/3565047479 shutdown_connections 2024-07-31T15:14:17.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.269+0000 7f94342e9740 1 -- 172.21.15.67:0/3565047479 wait complete. 2024-07-31T15:14:17.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.269+0000 7f94342e9740 1 -- start start 2024-07-31T15:14:17.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.270+0000 7f94342e9740 1 Processor -- start 2024-07-31T15:14:17.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.270+0000 7f94342e9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff1990 0x557e6b3cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.270+0000 7f94342e9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 0x557e6b3cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.270+0000 7f94342e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6b01a850 0x557e6b3d7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.270+0000 7f94342e9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557e6b1887b0 con 0x557e6aff1990 2024-07-31T15:14:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.270+0000 7f94342e9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557e6b2c7210 con 0x557e6aff9b70 2024-07-31T15:14:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.270+0000 7f94342e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557e6b3cec60 con 0x557e6b01a850 2024-07-31T15:14:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.270+0000 7f94342e9740 10 client.0 _ll_register_callbacks cb 0x557e6b2a82c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.270+0000 7f94277fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff1990 0x557e6b3cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.270+0000 7f94277fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff1990 0x557e6b3cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34470/0 (socket says 172.21.15.67:34470) 2024-07-31T15:14:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.270+0000 7f94277fe640 1 -- 172.21.15.67:0/2374628504 learned_addr learned my addr 172.21.15.67:0/2374628504 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.270+0000 7f9427fff640 1 --2- 172.21.15.67:0/2374628504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6b01a850 0x557e6b3d7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.270+0000 7f9426ffd640 1 --2- 172.21.15.67:0/2374628504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 0x557e6b3cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.271+0000 7f94277fe640 1 -- 172.21.15.67:0/2374628504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6b01a850 msgr2=0x557e6b3d7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.271+0000 7f94277fe640 1 --2- 172.21.15.67:0/2374628504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6b01a850 0x557e6b3d7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.271+0000 7f94277fe640 1 -- 172.21.15.67:0/2374628504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 msgr2=0x557e6b3cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.271+0000 7f94277fe640 1 --2- 172.21.15.67:0/2374628504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 0x557e6b3cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.271+0000 7f94277fe640 1 -- 172.21.15.67:0/2374628504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6b2d8710 con 0x557e6aff1990 2024-07-31T15:14:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.271+0000 7f9426ffd640 1 --2- 172.21.15.67:0/2374628504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 0x557e6b3cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:14:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.271+0000 7f94277fe640 1 --2- 172.21.15.67:0/2374628504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff1990 0x557e6b3cbbe0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f9418101bf0 tx=0x7f94180caa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118384b9dcefdf62 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.271+0000 7f9406ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:14:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.271+0000 7f9406ffd640 1 -- 172.21.15.67:0/2374628504 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418049040 con 0x557e6aff1990 2024-07-31T15:14:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.271+0000 7f9406ffd640 1 -- 172.21.15.67:0/2374628504 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f94180c76a0 con 0x557e6aff1990 2024-07-31T15:14:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.271+0000 7f94342e9740 1 -- 172.21.15.67:0/2374628504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557e6b3375e0 con 0x557e6aff1990 2024-07-31T15:14:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.271+0000 7f9406ffd640 1 -- 172.21.15.67:0/2374628504 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f941803b460 con 0x557e6aff1990 2024-07-31T15:14:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.271+0000 7f94342e9740 10 client.20385 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.271+0000 7f94342e9740 1 -- 172.21.15.67:0/2374628504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x557e6b094ba0 con 0x557e6aff1990 2024-07-31T15:14:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.272+0000 7f9406ffd640 1 -- 172.21.15.67:0/2374628504 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557e6b3375e0 con 0x557e6aff1990 2024-07-31T15:14:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.272+0000 7f9406ffd640 1 -- 172.21.15.67:0/2374628504 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f94180a6070 con 0x557e6aff1990 2024-07-31T15:14:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.272+0000 7f94342e9740 10 client.20385 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.272+0000 7f94342e9740 10 client.20385 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:14:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.272+0000 7f94342e9740 20 client.20385 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.272+0000 7f94342e9740 1 --2- 172.21.15.67:0/2374628504 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e6afd91d0 0x557e6b3ddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.272+0000 7f94342e9740 4 client.20385 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:14:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.272+0000 7f94342e9740 1 -- 172.21.15.67:0/2374628504 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e6b333080 con 0x557e6afd91d0 2024-07-31T15:14:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.272+0000 7f9426ffd640 1 --2- 172.21.15.67:0/2374628504 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e6afd91d0 0x557e6b3ddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.273+0000 7f9406ffd640 10 client.20385 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.273+0000 7f9426ffd640 1 --2- 172.21.15.67:0/2374628504 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e6afd91d0 0x557e6b3ddcb0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbed0188da4efeae server_cookie=8447f96d117333bd in_seq=0 out_seq=0 2024-07-31T15:14:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f9406ffd640 1 -- 172.21.15.67:0/2374628504 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e6b333080 con 0x557e6afd91d0 2024-07-31T15:14:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f9406ffd640 10 client.20385 handle_command_reply: tid=0 2024-07-31T15:14:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f94342e9740 1 client.20385 shutdown 2024-07-31T15:14:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f94342e9740 1 -- 172.21.15.67:0/2374628504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff1990 msgr2=0x557e6b3cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:17.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f94342e9740 1 --2- 172.21.15.67:0/2374628504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff1990 0x557e6b3cbbe0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f9418101bf0 tx=0x7f94180caa00 comp rx=0 tx=0).stop 2024-07-31T15:14:17.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f94342e9740 1 -- 172.21.15.67:0/2374628504 shutdown_connections 2024-07-31T15:14:17.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f94342e9740 1 --2- 172.21.15.67:0/2374628504 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e6afd91d0 0x557e6b3ddcb0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f94342e9740 1 --2- 172.21.15.67:0/2374628504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6b01a850 0x557e6b3d7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f94342e9740 1 --2- 172.21.15.67:0/2374628504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 0x557e6b3cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f94342e9740 1 --2- 172.21.15.67:0/2374628504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff1990 0x557e6b3cbbe0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f9406ffd640 0 client.20385 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f94342e9740 1 -- 172.21.15.67:0/2374628504 >> 172.21.15.67:0/2374628504 conn(0x557e6b00ec40 msgr2=0x557e6b3c36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f94342e9740 1 -- 172.21.15.67:0/2374628504 shutdown_connections 2024-07-31T15:14:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f94342e9740 1 -- 172.21.15.67:0/2374628504 wait complete. 2024-07-31T15:14:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.274+0000 7f94342e9740 20 client.20385 trim_cache size 0 max 16384 2024-07-31T15:14:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.292+0000 7f94342e9740 1 Processor -- start 2024-07-31T15:14:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.292+0000 7f94342e9740 1 -- start start 2024-07-31T15:14:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f94342e9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff1990 0x557e6b3c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f94342e9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 0x557e6b3c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f94342e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6b01a850 0x557e6b3d7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f94342e9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f94281b0850 con 0x557e6aff1990 2024-07-31T15:14:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f94342e9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f94281b2390 con 0x557e6aff9b70 2024-07-31T15:14:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f94277fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff1990 0x557e6b3c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f94342e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f94281b0f20 con 0x557e6b01a850 2024-07-31T15:14:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f94277fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff1990 0x557e6b3c1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34482/0 (socket says 172.21.15.67:34482) 2024-07-31T15:14:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f94277fe640 1 -- 172.21.15.67:0/1270480928 learned_addr learned my addr 172.21.15.67:0/1270480928 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f9427fff640 1 --2- 172.21.15.67:0/1270480928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6b01a850 0x557e6b3d7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f9426ffd640 1 --2- 172.21.15.67:0/1270480928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 0x557e6b3c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f94277fe640 1 -- 172.21.15.67:0/1270480928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6b01a850 msgr2=0x557e6b3d7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f94277fe640 1 --2- 172.21.15.67:0/1270480928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6b01a850 0x557e6b3d7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f94277fe640 1 -- 172.21.15.67:0/1270480928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 msgr2=0x557e6b3c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f94277fe640 1 --2- 172.21.15.67:0/1270480928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 0x557e6b3c3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f94277fe640 1 -- 172.21.15.67:0/1270480928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6b3ce790 con 0x557e6aff1990 2024-07-31T15:14:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.293+0000 7f9427fff640 1 --2- 172.21.15.67:0/1270480928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6b01a850 0x557e6b3d7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.294+0000 7f94277fe640 1 --2- 172.21.15.67:0/1270480928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff1990 0x557e6b3c1f00 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f94180cadb0 tx=0x7f941803b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fb37d11430db38 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:17.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.294+0000 7f940d7fa640 1 -- 172.21.15.67:0/1270480928 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418049040 con 0x557e6aff1990 2024-07-31T15:14:17.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.294+0000 7f940d7fa640 1 -- 172.21.15.67:0/1270480928 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94180cabc0 con 0x557e6aff1990 2024-07-31T15:14:17.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.294+0000 7f940d7fa640 1 -- 172.21.15.67:0/1270480928 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94180d3e60 con 0x557e6aff1990 2024-07-31T15:14:17.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.295+0000 7f94342e9740 1 -- 172.21.15.67:0/1270480928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff1990 msgr2=0x557e6b3c1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:17.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.295+0000 7f94342e9740 1 --2- 172.21.15.67:0/1270480928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff1990 0x557e6b3c1f00 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f94180cadb0 tx=0x7f941803b640 comp rx=0 tx=0).stop 2024-07-31T15:14:17.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.295+0000 7f94342e9740 1 -- 172.21.15.67:0/1270480928 shutdown_connections 2024-07-31T15:14:17.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.295+0000 7f94342e9740 1 --2- 172.21.15.67:0/1270480928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6b01a850 0x557e6b3d7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.295+0000 7f94342e9740 1 --2- 172.21.15.67:0/1270480928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 0x557e6b3c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.295+0000 7f94342e9740 1 --2- 172.21.15.67:0/1270480928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6aff1990 0x557e6b3c1f00 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.295+0000 7f94342e9740 1 -- 172.21.15.67:0/1270480928 >> 172.21.15.67:0/1270480928 conn(0x557e6b00ec40 msgr2=0x557e6b3c3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:17.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.295+0000 7f94342e9740 1 -- 172.21.15.67:0/1270480928 shutdown_connections 2024-07-31T15:14:17.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.295+0000 7f94342e9740 1 -- 172.21.15.67:0/1270480928 wait complete. 2024-07-31T15:14:17.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.295+0000 7f94342e9740 1 -- start start 2024-07-31T15:14:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94342e9740 1 Processor -- start 2024-07-31T15:14:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94342e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6aff1990 0x557e6b3d0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94342e9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 0x557e6b3d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94342e9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6b01a850 0x557e6b445410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94342e9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f94281b1cc0 con 0x557e6b01a850 2024-07-31T15:14:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94342e9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f94281b15f0 con 0x557e6aff9b70 2024-07-31T15:14:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94342e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f94281ae850 con 0x557e6aff1990 2024-07-31T15:14:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94342e9740 10 client.0 _ll_register_callbacks cb 0x7f9418009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94277fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6aff1990 0x557e6b3d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94277fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6aff1990 0x557e6b3d0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:55250/0 (socket says 172.21.15.67:55250) 2024-07-31T15:14:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94277fe640 1 -- 172.21.15.67:0/881440846 learned_addr learned my addr 172.21.15.67:0/881440846 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f9426ffd640 1 --2- 172.21.15.67:0/881440846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 0x557e6b3d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f9427fff640 1 --2- 172.21.15.67:0/881440846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6b01a850 0x557e6b445410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94277fe640 1 -- 172.21.15.67:0/881440846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 msgr2=0x557e6b3d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94277fe640 1 --2- 172.21.15.67:0/881440846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 0x557e6b3d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94277fe640 1 -- 172.21.15.67:0/881440846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6b01a850 msgr2=0x557e6b445410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94277fe640 1 --2- 172.21.15.67:0/881440846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6b01a850 0x557e6b445410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.296+0000 7f94277fe640 1 -- 172.21.15.67:0/881440846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6b4459d0 con 0x557e6aff1990 2024-07-31T15:14:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.297+0000 7f94277fe640 1 --2- 172.21.15.67:0/881440846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6aff1990 0x557e6b3d0520 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f94180d4150 tx=0x7f94180cb540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c359801f44951f1 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.297+0000 7f9406ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:14:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.297+0000 7f9406ffd640 1 -- 172.21.15.67:0/881440846 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94180ac070 con 0x557e6aff1990 2024-07-31T15:14:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.297+0000 7f9406ffd640 1 -- 172.21.15.67:0/881440846 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f9418104520 con 0x557e6aff1990 2024-07-31T15:14:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.297+0000 7f94342e9740 1 -- 172.21.15.67:0/881440846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94180cabc0 con 0x557e6aff1990 2024-07-31T15:14:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.297+0000 7f9406ffd640 1 -- 172.21.15.67:0/881440846 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f9418049040 con 0x557e6aff1990 2024-07-31T15:14:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.297+0000 7f94342e9740 10 client.29728 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.297+0000 7f94342e9740 1 -- 172.21.15.67:0/881440846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x557e6b4459d0 con 0x557e6aff1990 2024-07-31T15:14:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.297+0000 7f9406ffd640 1 -- 172.21.15.67:0/881440846 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94180cabc0 con 0x557e6aff1990 2024-07-31T15:14:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.298+0000 7f9406ffd640 1 -- 172.21.15.67:0/881440846 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f94180af030 con 0x557e6aff1990 2024-07-31T15:14:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.298+0000 7f94342e9740 10 client.29728 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.298+0000 7f94342e9740 10 client.29728 resolve_mds: validated gid 24671 aka daemon mds.b 2024-07-31T15:14:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.298+0000 7f94342e9740 20 client.29728 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.298+0000 7f94342e9740 1 --2- 172.21.15.67:0/881440846 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e6afae580 0x557e6b448fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.298+0000 7f94342e9740 4 client.29728 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.95, "await_for": 45.0}] 2024-07-31T15:14:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.298+0000 7f94342e9740 1 -- 172.21.15.67:0/881440846 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.95, "await_for": 45.0}) -- 0x557e6b3307c0 con 0x557e6afae580 2024-07-31T15:14:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.298+0000 7f9426ffd640 1 --2- 172.21.15.67:0/881440846 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e6afae580 0x557e6b448fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.298+0000 7f9406ffd640 10 client.29728 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:17.299+0000 7f9426ffd640 1 --2- 172.21.15.67:0/881440846 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e6afae580 0x557e6b448fc0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f230c9cc7961c52d server_cookie=c6de3451b1043089 in_seq=0 out_seq=0 2024-07-31T15:14:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.915+0000 7f47badd6640 1 -- 172.21.15.67:0/2602576270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47b41042d0 msgr2=0x7f47b41046e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.915+0000 7f47badd6640 1 --2- 172.21.15.67:0/2602576270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47b41042d0 0x7f47b41046e0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009900 tx=0x7f47a402a410 comp rx=0 tx=0).stop 2024-07-31T15:14:18.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.915+0000 7f47badd6640 1 -- 172.21.15.67:0/2602576270 shutdown_connections 2024-07-31T15:14:18.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.915+0000 7f47badd6640 1 --2- 172.21.15.67:0/2602576270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47b410d880 0x7f47b410fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:18.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.915+0000 7f47badd6640 1 --2- 172.21.15.67:0/2602576270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47b4104d00 0x7f47b410d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:18.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.915+0000 7f47badd6640 1 --2- 172.21.15.67:0/2602576270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47b41042d0 0x7f47b41046e0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:18.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.915+0000 7f47badd6640 1 -- 172.21.15.67:0/2602576270 >> 172.21.15.67:0/2602576270 conn(0x7f47b4079160 msgr2=0x7f47b4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:18.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.915+0000 7f47badd6640 1 -- 172.21.15.67:0/2602576270 shutdown_connections 2024-07-31T15:14:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.915+0000 7f47badd6640 1 -- 172.21.15.67:0/2602576270 wait complete. 2024-07-31T15:14:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.916+0000 7f47badd6640 1 Processor -- start 2024-07-31T15:14:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.916+0000 7f47badd6640 1 -- start start 2024-07-31T15:14:18.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.916+0000 7f47badd6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47b41042d0 0x7f47b419c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:18.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.916+0000 7f47badd6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47b4104d00 0x7f47b419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:18.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.916+0000 7f47badd6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47b410d880 0x7f47b41a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:18.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.916+0000 7f47badd6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41134a0 con 0x7f47b4104d00 2024-07-31T15:14:18.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.916+0000 7f47badd6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4113320 con 0x7f47b41042d0 2024-07-31T15:14:18.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.916+0000 7f47badd6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4113620 con 0x7f47b410d880 2024-07-31T15:14:18.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b8b72640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47b41042d0 0x7f47b419c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:18.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47b4104d00 0x7f47b419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:18.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b8b72640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47b41042d0 0x7f47b419c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:54418/0 (socket says 172.21.15.67:54418) 2024-07-31T15:14:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b8b72640 1 -- 172.21.15.67:0/2068529112 learned_addr learned my addr 172.21.15.67:0/2068529112 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b9373640 1 --2- 172.21.15.67:0/2068529112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47b410d880 0x7f47b41a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b3fff640 1 -- 172.21.15.67:0/2068529112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47b410d880 msgr2=0x7f47b41a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b3fff640 1 --2- 172.21.15.67:0/2068529112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47b410d880 0x7f47b41a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b3fff640 1 -- 172.21.15.67:0/2068529112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47b41042d0 msgr2=0x7f47b419c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b3fff640 1 --2- 172.21.15.67:0/2068529112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47b41042d0 0x7f47b419c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b3fff640 1 -- 172.21.15.67:0/2068529112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b41a2090 con 0x7f47b4104d00 2024-07-31T15:14:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b8b72640 1 --2- 172.21.15.67:0/2068529112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47b41042d0 0x7f47b419c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b3fff640 1 --2- 172.21.15.67:0/2068529112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47b4104d00 0x7f47b419c6c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f479c00e9c0 tx=0x7f479c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b1ffb640 1 -- 172.21.15.67:0/2068529112 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c00ccc0 con 0x7f47b4104d00 2024-07-31T15:14:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b1ffb640 1 -- 172.21.15.67:0/2068529112 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f479c0108e0 con 0x7f47b4104d00 2024-07-31T15:14:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47b1ffb640 1 -- 172.21.15.67:0/2068529112 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c0187c0 con 0x7f47b4104d00 2024-07-31T15:14:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.917+0000 7f47badd6640 1 -- 172.21.15.67:0/2068529112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b41a2380 con 0x7f47b4104d00 2024-07-31T15:14:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.918+0000 7f47badd6640 1 -- 172.21.15.67:0/2068529112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b41a8ae0 con 0x7f47b4104d00 2024-07-31T15:14:18.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.920+0000 7f47c1327740 1 Processor -- start 2024-07-31T15:14:18.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.920+0000 7f47c1327740 1 -- start start 2024-07-31T15:14:18.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.920+0000 7f47b1ffb640 1 -- 172.21.15.67:0/2068529112 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f479c010430 con 0x7f47b4104d00 2024-07-31T15:14:18.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47c1327740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 0x55618f522690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:18.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47c1327740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f1541b0 0x55618f522bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:18.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47c1327740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f15c390 0x55618f523110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:18.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47c1327740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55618f498bb0 con 0x55618f1541b0 2024-07-31T15:14:18.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47c1327740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55618f4880b0 con 0x55618f17d070 2024-07-31T15:14:18.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47c1327740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55618f48c0f0 con 0x55618f15c390 2024-07-31T15:14:18.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47b9373640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 0x55618f522690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:18.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47b8b72640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f1541b0 0x55618f522bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:18.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47b9373640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 0x55618f522690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:54420/0 (socket says 172.21.15.67:54420) 2024-07-31T15:14:18.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47b9373640 1 -- 172.21.15.67:0/2246739318 learned_addr learned my addr 172.21.15.67:0/2246739318 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:18.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47b3fff640 1 --2- 172.21.15.67:0/2246739318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f15c390 0x55618f523110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:18.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47b9373640 1 -- 172.21.15.67:0/2246739318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f15c390 msgr2=0x55618f523110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:18.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47b9373640 1 --2- 172.21.15.67:0/2246739318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f15c390 0x55618f523110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:18.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47b9373640 1 -- 172.21.15.67:0/2246739318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f1541b0 msgr2=0x55618f522bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:18.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47b9373640 1 --2- 172.21.15.67:0/2246739318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f1541b0 0x55618f522bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:18.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47b9373640 1 -- 172.21.15.67:0/2246739318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618f42eb90 con 0x55618f17d070 2024-07-31T15:14:18.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47b9373640 1 --2- 172.21.15.67:0/2246739318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 0x55618f522690 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009f50 tx=0x7f47ac0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a3edf8ee2948190 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:18.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.921+0000 7f47b1ffb640 1 --2- 172.21.15.67:0/2068529112 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f47a00b3d40 0x7f47a00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:18.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.922+0000 7f47aaffd640 1 -- 172.21.15.67:0/2246739318 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac004d20 con 0x55618f17d070 2024-07-31T15:14:18.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.922+0000 7f47b8b72640 1 --2- 172.21.15.67:0/2068529112 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f47a00b3d40 0x7f47a00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:18.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.922+0000 7f47aaffd640 1 -- 172.21.15.67:0/2246739318 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ac004050 con 0x55618f17d070 2024-07-31T15:14:18.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.922+0000 7f47aaffd640 1 -- 172.21.15.67:0/2246739318 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac005280 con 0x55618f17d070 2024-07-31T15:14:18.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.922+0000 7f47b1ffb640 1 -- 172.21.15.67:0/2068529112 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f479c0e46b0 con 0x7f47b4104d00 2024-07-31T15:14:18.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.923+0000 7f47c1327740 1 -- 172.21.15.67:0/2246739318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 msgr2=0x55618f522690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:18.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.923+0000 7f47b8b72640 1 --2- 172.21.15.67:0/2068529112 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f47a00b3d40 0x7f47a00b6250 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f47a40096c0 tx=0x7f47a4032ed0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:18.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.923+0000 7f47c1327740 1 --2- 172.21.15.67:0/2246739318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 0x55618f522690 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009f50 tx=0x7f47ac0118d0 comp rx=0 tx=0).stop 2024-07-31T15:14:18.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.923+0000 7f47c1327740 1 -- 172.21.15.67:0/2246739318 shutdown_connections 2024-07-31T15:14:18.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.923+0000 7f47c1327740 1 --2- 172.21.15.67:0/2246739318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f15c390 0x55618f523110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:18.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.923+0000 7f47c1327740 1 --2- 172.21.15.67:0/2246739318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f1541b0 0x55618f522bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:18.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.923+0000 7f47c1327740 1 --2- 172.21.15.67:0/2246739318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 0x55618f522690 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:18.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.923+0000 7f47c1327740 1 -- 172.21.15.67:0/2246739318 >> 172.21.15.67:0/2246739318 conn(0x55618f171460 msgr2=0x55618f520580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:18.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.923+0000 7f47c1327740 1 -- 172.21.15.67:0/2246739318 shutdown_connections 2024-07-31T15:14:18.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.923+0000 7f47c1327740 1 -- 172.21.15.67:0/2246739318 wait complete. 2024-07-31T15:14:18.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.923+0000 7f47c1327740 1 -- start start 2024-07-31T15:14:18.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.923+0000 7f47c1327740 1 Processor -- start 2024-07-31T15:14:18.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47c1327740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f1541b0 0x55618f52e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:18.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47c1327740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f15c390 0x55618f52e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:18.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47c1327740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 0x55618f53a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:18.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47c1327740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55618f2eb700 con 0x55618f15c390 2024-07-31T15:14:18.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47c1327740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55618f42a9e0 con 0x55618f17d070 2024-07-31T15:14:18.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47c1327740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55618f530e70 con 0x55618f1541b0 2024-07-31T15:14:18.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47c1327740 10 client.0 _ll_register_callbacks cb 0x55618f4454f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:18.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47b8b72640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f1541b0 0x55618f52e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:18.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f15c390 0x55618f52e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:18.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47b8b72640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f1541b0 0x55618f52e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:55284/0 (socket says 172.21.15.67:55284) 2024-07-31T15:14:18.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f15c390 0x55618f52e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34520/0 (socket says 172.21.15.67:34520) 2024-07-31T15:14:18.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47b8b72640 1 -- 172.21.15.67:0/4147992204 learned_addr learned my addr 172.21.15.67:0/4147992204 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:18.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47b9373640 1 --2- 172.21.15.67:0/4147992204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 0x55618f53a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:18.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47b8b72640 1 -- 172.21.15.67:0/4147992204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 msgr2=0x55618f53a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:18.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47b8b72640 1 --2- 172.21.15.67:0/4147992204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 0x55618f53a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:18.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47b8b72640 1 -- 172.21.15.67:0/4147992204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f15c390 msgr2=0x55618f52e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:18.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47b8b72640 1 --2- 172.21.15.67:0/4147992204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f15c390 0x55618f52e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:18.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.924+0000 7f47b8b72640 1 -- 172.21.15.67:0/4147992204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618f1e8650 con 0x55618f1541b0 2024-07-31T15:14:18.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.925+0000 7f47b8b72640 1 --2- 172.21.15.67:0/4147992204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f1541b0 0x55618f52e320 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f47a40042c0 tx=0x7f47a4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc21909fceb9208a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:18.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.925+0000 7f4793fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:14:18.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.925+0000 7f4793fff640 1 -- 172.21.15.67:0/4147992204 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a404d070 con 0x55618f1541b0 2024-07-31T15:14:18.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.925+0000 7f4793fff640 1 -- 172.21.15.67:0/4147992204 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f47a4005220 con 0x55618f1541b0 2024-07-31T15:14:18.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.925+0000 7f47c1327740 1 -- 172.21.15.67:0/4147992204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47ac004050 con 0x55618f1541b0 2024-07-31T15:14:18.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.925+0000 7f4793fff640 1 -- 172.21.15.67:0/4147992204 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f47a4043070 con 0x55618f1541b0 2024-07-31T15:14:18.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.925+0000 7f47c1327740 10 client.29740 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:18.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.925+0000 7f47c1327740 1 -- 172.21.15.67:0/4147992204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55618f53cf00 con 0x55618f1541b0 2024-07-31T15:14:18.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.925+0000 7f4793fff640 1 -- 172.21.15.67:0/4147992204 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ac004050 con 0x55618f1541b0 2024-07-31T15:14:18.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.926+0000 7f4793fff640 1 -- 172.21.15.67:0/4147992204 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f47a40047a0 con 0x55618f1541b0 2024-07-31T15:14:18.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.926+0000 7f47c1327740 10 client.29740 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:18.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.926+0000 7f47c1327740 10 client.29740 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:14:18.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.926+0000 7f47c1327740 20 client.29740 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:18.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.926+0000 7f47c1327740 1 --2- 172.21.15.67:0/4147992204 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55618f13b9f0 0x55618f540430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:18.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.926+0000 7f47c1327740 4 client.29740 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:14:18.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.926+0000 7f47c1327740 1 -- 172.21.15.67:0/4147992204 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55618f49ab00 con 0x55618f13b9f0 2024-07-31T15:14:18.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.926+0000 7f47b3fff640 1 --2- 172.21.15.67:0/4147992204 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55618f13b9f0 0x55618f540430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:18.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.926+0000 7f47b3fff640 1 --2- 172.21.15.67:0/4147992204 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55618f13b9f0 0x55618f540430 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=193925078877e64 server_cookie=483d85fe5d5b061e in_seq=0 out_seq=0 2024-07-31T15:14:18.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:18.926+0000 7f4793fff640 10 client.29740 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:14:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f4793fff640 1 -- 172.21.15.67:0/4147992204 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55618f49ab00 con 0x55618f13b9f0 2024-07-31T15:14:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f4793fff640 10 client.29740 handle_command_reply: tid=0 2024-07-31T15:14:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f47c1327740 1 client.29740 shutdown 2024-07-31T15:14:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f47c1327740 1 -- 172.21.15.67:0/4147992204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f1541b0 msgr2=0x55618f52e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f47c1327740 1 --2- 172.21.15.67:0/4147992204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f1541b0 0x55618f52e320 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f47a40042c0 tx=0x7f47a4009260 comp rx=0 tx=0).stop 2024-07-31T15:14:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f47c1327740 1 -- 172.21.15.67:0/4147992204 shutdown_connections 2024-07-31T15:14:19.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f47c1327740 1 --2- 172.21.15.67:0/4147992204 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55618f13b9f0 0x55618f540430 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f47c1327740 1 --2- 172.21.15.67:0/4147992204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 0x55618f53a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f47c1327740 1 --2- 172.21.15.67:0/4147992204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f15c390 0x55618f52e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f4793fff640 0 client.29740 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:14:19.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f47c1327740 1 --2- 172.21.15.67:0/4147992204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f1541b0 0x55618f52e320 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f47c1327740 1 -- 172.21.15.67:0/4147992204 >> 172.21.15.67:0/4147992204 conn(0x55618f171460 msgr2=0x55618f51ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:19.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f47c1327740 1 -- 172.21.15.67:0/4147992204 shutdown_connections 2024-07-31T15:14:19.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f47c1327740 1 -- 172.21.15.67:0/4147992204 wait complete. 2024-07-31T15:14:19.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.043+0000 7f47c1327740 20 client.29740 trim_cache size 0 max 16384 2024-07-31T15:14:19.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.058+0000 7f47c1327740 1 Processor -- start 2024-07-31T15:14:19.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.058+0000 7f47c1327740 1 -- start start 2024-07-31T15:14:19.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.058+0000 7f47c1327740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f1541b0 0x55618f51f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:19.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.058+0000 7f47c1327740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f15c390 0x55618f539f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:19.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.058+0000 7f47c1327740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 0x55618f527910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:19.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.058+0000 7f47c1327740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b29c0 con 0x55618f1541b0 2024-07-31T15:14:19.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.058+0000 7f47c1327740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b4500 con 0x55618f17d070 2024-07-31T15:14:19.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.058+0000 7f47c1327740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b3090 con 0x55618f15c390 2024-07-31T15:14:19.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.058+0000 7f47b8b72640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f1541b0 0x55618f51f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:19.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.058+0000 7f47b8b72640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f1541b0 0x55618f51f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34526/0 (socket says 172.21.15.67:34526) 2024-07-31T15:14:19.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.058+0000 7f47b8b72640 1 -- 172.21.15.67:0/3882691809 learned_addr learned my addr 172.21.15.67:0/3882691809 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:19.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.058+0000 7f47b3fff640 1 --2- 172.21.15.67:0/3882691809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f15c390 0x55618f539f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:19.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.058+0000 7f47b9373640 1 --2- 172.21.15.67:0/3882691809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 0x55618f527910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:19.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.059+0000 7f47b8b72640 1 -- 172.21.15.67:0/3882691809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f15c390 msgr2=0x55618f539f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:19.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.059+0000 7f47b8b72640 1 --2- 172.21.15.67:0/3882691809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f15c390 0x55618f539f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.059+0000 7f47b8b72640 1 -- 172.21.15.67:0/3882691809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 msgr2=0x55618f527910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:19.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.059+0000 7f47b8b72640 1 --2- 172.21.15.67:0/3882691809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 0x55618f527910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.059+0000 7f47b8b72640 1 -- 172.21.15.67:0/3882691809 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618f5234a0 con 0x55618f1541b0 2024-07-31T15:14:19.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.059+0000 7f47b8b72640 1 --2- 172.21.15.67:0/3882691809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f1541b0 0x55618f51f410 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f47a4031440 tx=0x7f47a4031710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f76a45eee68d472 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:19.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.059+0000 7f47aa7fc640 1 -- 172.21.15.67:0/3882691809 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a404d070 con 0x55618f1541b0 2024-07-31T15:14:19.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.059+0000 7f47aa7fc640 1 -- 172.21.15.67:0/3882691809 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a4003cb0 con 0x55618f1541b0 2024-07-31T15:14:19.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.059+0000 7f47aa7fc640 1 -- 172.21.15.67:0/3882691809 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4035040 con 0x55618f1541b0 2024-07-31T15:14:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.060+0000 7f47c1327740 1 -- 172.21.15.67:0/3882691809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f1541b0 msgr2=0x55618f51f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.060+0000 7f47c1327740 1 --2- 172.21.15.67:0/3882691809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f1541b0 0x55618f51f410 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f47a4031440 tx=0x7f47a4031710 comp rx=0 tx=0).stop 2024-07-31T15:14:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.060+0000 7f47c1327740 1 -- 172.21.15.67:0/3882691809 shutdown_connections 2024-07-31T15:14:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.060+0000 7f47c1327740 1 --2- 172.21.15.67:0/3882691809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f17d070 0x55618f527910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.060+0000 7f47c1327740 1 --2- 172.21.15.67:0/3882691809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f15c390 0x55618f539f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.060+0000 7f47c1327740 1 --2- 172.21.15.67:0/3882691809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f1541b0 0x55618f51f410 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.060+0000 7f47c1327740 1 -- 172.21.15.67:0/3882691809 >> 172.21.15.67:0/3882691809 conn(0x55618f171460 msgr2=0x55618f51ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:19.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.060+0000 7f47c1327740 1 -- 172.21.15.67:0/3882691809 shutdown_connections 2024-07-31T15:14:19.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.060+0000 7f47c1327740 1 -- 172.21.15.67:0/3882691809 wait complete. 2024-07-31T15:14:19.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.060+0000 7f47c1327740 1 -- start start 2024-07-31T15:14:19.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.061+0000 7f47c1327740 1 Processor -- start 2024-07-31T15:14:19.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.061+0000 7f47c1327740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f1541b0 0x55618f5360d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:19.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.061+0000 7f47c1327740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f15c390 0x55618f536610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:19.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.061+0000 7f47c1327740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f17d070 0x55618f531080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:19.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.061+0000 7f47c1327740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b3e30 con 0x55618f17d070 2024-07-31T15:14:19.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.061+0000 7f47c1327740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b3760 con 0x55618f15c390 2024-07-31T15:14:19.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.061+0000 7f47c1327740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b09c0 con 0x55618f1541b0 2024-07-31T15:14:19.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.061+0000 7f47c1327740 10 client.0 _ll_register_callbacks cb 0x7f47a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:19.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.061+0000 7f47b8b72640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f1541b0 0x55618f5360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:19.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.061+0000 7f47b9373640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f17d070 0x55618f531080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:19.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.061+0000 7f47b8b72640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f1541b0 0x55618f5360d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:55306/0 (socket says 172.21.15.67:55306) 2024-07-31T15:14:19.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.061+0000 7f47b8b72640 1 -- 172.21.15.67:0/598329395 learned_addr learned my addr 172.21.15.67:0/598329395 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:19.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.061+0000 7f47b3fff640 1 --2- 172.21.15.67:0/598329395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f15c390 0x55618f536610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:19.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.062+0000 7f47b8b72640 1 -- 172.21.15.67:0/598329395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f15c390 msgr2=0x55618f536610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:19.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.062+0000 7f47b8b72640 1 --2- 172.21.15.67:0/598329395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f15c390 0x55618f536610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.062+0000 7f47b8b72640 1 -- 172.21.15.67:0/598329395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f17d070 msgr2=0x55618f531080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:19.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.062+0000 7f47b8b72640 1 --2- 172.21.15.67:0/598329395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f17d070 0x55618f531080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.062+0000 7f47b8b72640 1 -- 172.21.15.67:0/598329395 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618f5466c0 con 0x55618f1541b0 2024-07-31T15:14:19.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.062+0000 7f47b3fff640 1 --2- 172.21.15.67:0/598329395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f15c390 0x55618f536610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:19.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.062+0000 7f47b8b72640 1 --2- 172.21.15.67:0/598329395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f1541b0 0x55618f5360d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f47a40042c0 tx=0x7f47a4004fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d971ba400d4a210 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:19.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.062+0000 7f4793fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:14:19.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.062+0000 7f4793fff640 1 -- 172.21.15.67:0/598329395 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a404d070 con 0x55618f1541b0 2024-07-31T15:14:19.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.062+0000 7f47c1327740 1 -- 172.21.15.67:0/598329395 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a4003cb0 con 0x55618f1541b0 2024-07-31T15:14:19.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.062+0000 7f4793fff640 1 -- 172.21.15.67:0/598329395 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f47a40050e0 con 0x55618f1541b0 2024-07-31T15:14:19.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.062+0000 7f4793fff640 1 -- 172.21.15.67:0/598329395 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f47a4075050 con 0x55618f1541b0 2024-07-31T15:14:19.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.062+0000 7f47c1327740 10 client.29746 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:19.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.062+0000 7f47c1327740 1 -- 172.21.15.67:0/598329395 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55618f5a8fb0 con 0x55618f1541b0 2024-07-31T15:14:19.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.063+0000 7f4793fff640 1 -- 172.21.15.67:0/598329395 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a4003cb0 con 0x55618f1541b0 2024-07-31T15:14:19.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.063+0000 7f4793fff640 1 -- 172.21.15.67:0/598329395 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f47a4052b60 con 0x55618f1541b0 2024-07-31T15:14:19.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.063+0000 7f47c1327740 10 client.29746 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:19.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.063+0000 7f47c1327740 10 client.29746 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:14:19.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.063+0000 7f47c1327740 20 client.29746 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:19.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.063+0000 7f47c1327740 1 --2- 172.21.15.67:0/598329395 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55618f110da0 0x55618f5ab700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:19.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.063+0000 7f47c1327740 4 client.29746 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:14:19.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.063+0000 7f47c1327740 1 -- 172.21.15.67:0/598329395 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55618f4965a0 con 0x55618f110da0 2024-07-31T15:14:19.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.063+0000 7f47b3fff640 1 --2- 172.21.15.67:0/598329395 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55618f110da0 0x55618f5ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:19.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.063+0000 7f47b3fff640 1 --2- 172.21.15.67:0/598329395 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55618f110da0 0x55618f5ab700 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eff7b7952090185a server_cookie=b684ca4935e8de28 in_seq=0 out_seq=0 2024-07-31T15:14:19.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.063+0000 7f4793fff640 10 client.29746 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:14:19.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f4793fff640 1 -- 172.21.15.67:0/598329395 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55618f4965a0 con 0x55618f110da0 2024-07-31T15:14:19.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f4793fff640 10 client.29746 handle_command_reply: tid=0 2024-07-31T15:14:19.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f47c1327740 1 client.29746 shutdown 2024-07-31T15:14:19.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f47c1327740 1 -- 172.21.15.67:0/598329395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f1541b0 msgr2=0x55618f5360d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:19.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f47c1327740 1 --2- 172.21.15.67:0/598329395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f1541b0 0x55618f5360d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f47a40042c0 tx=0x7f47a4004fb0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f47c1327740 1 -- 172.21.15.67:0/598329395 shutdown_connections 2024-07-31T15:14:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f47c1327740 1 --2- 172.21.15.67:0/598329395 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55618f110da0 0x55618f5ab700 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f47c1327740 1 --2- 172.21.15.67:0/598329395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55618f17d070 0x55618f531080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f47c1327740 1 --2- 172.21.15.67:0/598329395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55618f15c390 0x55618f536610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f4793fff640 0 client.29746 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:14:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f47c1327740 1 --2- 172.21.15.67:0/598329395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55618f1541b0 0x55618f5360d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f47c1327740 1 -- 172.21.15.67:0/598329395 >> 172.21.15.67:0/598329395 conn(0x55618f171460 msgr2=0x55618f51fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f47c1327740 1 -- 172.21.15.67:0/598329395 shutdown_connections 2024-07-31T15:14:19.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f47c1327740 1 -- 172.21.15.67:0/598329395 wait complete. 2024-07-31T15:14:19.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.099+0000 7f47c1327740 20 client.29746 trim_cache size 0 max 16384 2024-07-31T15:14:19.103 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:14:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.100+0000 7f47badd6640 1 -- 172.21.15.67:0/2068529112 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f47a00b3d40 msgr2=0x7f47a00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.100+0000 7f47badd6640 1 --2- 172.21.15.67:0/2068529112 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f47a00b3d40 0x7f47a00b6250 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f47a40096c0 tx=0x7f47a4032ed0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.100+0000 7f47badd6640 1 -- 172.21.15.67:0/2068529112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47b4104d00 msgr2=0x7f47b419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:19.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.100+0000 7f47badd6640 1 --2- 172.21.15.67:0/2068529112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47b4104d00 0x7f47b419c6c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f479c00e9c0 tx=0x7f479c00ee80 comp rx=0 tx=0).stop 2024-07-31T15:14:19.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.100+0000 7f47badd6640 1 -- 172.21.15.67:0/2068529112 shutdown_connections 2024-07-31T15:14:19.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.100+0000 7f47badd6640 1 --2- 172.21.15.67:0/2068529112 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f47a00b3d40 0x7f47a00b6250 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.100+0000 7f47badd6640 1 --2- 172.21.15.67:0/2068529112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47b410d880 0x7f47b41a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.100+0000 7f47badd6640 1 --2- 172.21.15.67:0/2068529112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47b4104d00 0x7f47b419c6c0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.100+0000 7f47badd6640 1 --2- 172.21.15.67:0/2068529112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47b41042d0 0x7f47b419c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:19.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.100+0000 7f47badd6640 1 -- 172.21.15.67:0/2068529112 >> 172.21.15.67:0/2068529112 conn(0x7f47b4079160 msgr2=0x7f47b4100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:19.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.100+0000 7f47badd6640 1 -- 172.21.15.67:0/2068529112 shutdown_connections 2024-07-31T15:14:19.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:19.100+0000 7f47badd6640 1 -- 172.21.15.67:0/2068529112 wait complete. 2024-07-31T15:14:19.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:19 smithi073 ceph-mon[36714]: pgmap v1143: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 135 KiB/s wr, 27 op/s 2024-07-31T15:14:19.717 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:19 smithi067 ceph-mon[32750]: pgmap v1143: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 135 KiB/s wr, 27 op/s 2024-07-31T15:14:19.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:19 smithi077 ceph-mon[37327]: pgmap v1143: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 135 KiB/s wr, 27 op/s 2024-07-31T15:14:19.750 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T15:14:20.021 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:14:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:21 smithi067 ceph-mon[32750]: pgmap v1144: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 135 KiB/s wr, 26 op/s 2024-07-31T15:14:21.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:21 smithi073 ceph-mon[36714]: pgmap v1144: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 135 KiB/s wr, 26 op/s 2024-07-31T15:14:21.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:21 smithi077 ceph-mon[37327]: pgmap v1144: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 135 KiB/s wr, 26 op/s 2024-07-31T15:14:22.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.043+0000 7f661906f640 1 -- 172.21.15.67:0/2311734362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6614102490 msgr2=0x7f66141028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:22.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.043+0000 7f661906f640 1 --2- 172.21.15.67:0/2311734362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6614102490 0x7f66141028a0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f660c009900 tx=0x7f660c02a410 comp rx=0 tx=0).stop 2024-07-31T15:14:22.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.043+0000 7f661906f640 1 -- 172.21.15.67:0/2311734362 shutdown_connections 2024-07-31T15:14:22.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.043+0000 7f661906f640 1 --2- 172.21.15.67:0/2311734362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6614108720 0x7f661410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.043+0000 7f661906f640 1 --2- 172.21.15.67:0/2311734362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6614107dd0 0x7f66141081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.043+0000 7f661906f640 1 --2- 172.21.15.67:0/2311734362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6614102490 0x7f66141028a0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.043+0000 7f661906f640 1 -- 172.21.15.67:0/2311734362 >> 172.21.15.67:0/2311734362 conn(0x7f6614079160 msgr2=0x7f6614079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:22.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.043+0000 7f661906f640 1 -- 172.21.15.67:0/2311734362 shutdown_connections 2024-07-31T15:14:22.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.043+0000 7f661906f640 1 -- 172.21.15.67:0/2311734362 wait complete. 2024-07-31T15:14:22.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.044+0000 7f661906f640 1 Processor -- start 2024-07-31T15:14:22.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.044+0000 7f661906f640 1 -- start start 2024-07-31T15:14:22.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.044+0000 7f661906f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6614102490 0x7f6614199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.044+0000 7f661906f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6614107dd0 0x7f661419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.044+0000 7f661906f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6614108720 0x7f661419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.044+0000 7f661906f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f66141123a0 con 0x7f6614102490 2024-07-31T15:14:22.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.044+0000 7f661906f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6614112220 con 0x7f6614108720 2024-07-31T15:14:22.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.044+0000 7f661906f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6614112520 con 0x7f6614107dd0 2024-07-31T15:14:22.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f6612d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6614102490 0x7f6614199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f661259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6614107dd0 0x7f661419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f6612d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6614102490 0x7f6614199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56146/0 (socket says 172.21.15.67:56146) 2024-07-31T15:14:22.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f6612d9d640 1 -- 172.21.15.67:0/250570619 learned_addr learned my addr 172.21.15.67:0/250570619 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:22.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f661359e640 1 --2- 172.21.15.67:0/250570619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6614108720 0x7f661419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f661259c640 1 -- 172.21.15.67:0/250570619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6614108720 msgr2=0x7f661419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:22.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f661259c640 1 --2- 172.21.15.67:0/250570619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6614108720 0x7f661419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f661259c640 1 -- 172.21.15.67:0/250570619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6614102490 msgr2=0x7f6614199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:22.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f661259c640 1 --2- 172.21.15.67:0/250570619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6614102490 0x7f6614199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f661259c640 1 -- 172.21.15.67:0/250570619 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661419fe10 con 0x7f6614107dd0 2024-07-31T15:14:22.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f6612d9d640 1 --2- 172.21.15.67:0/250570619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6614102490 0x7f6614199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:22.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f661259c640 1 --2- 172.21.15.67:0/250570619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6614107dd0 0x7f661419a440 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00e9c0 tx=0x7f65fc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:22.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f660bfff640 1 -- 172.21.15.67:0/250570619 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00ccc0 con 0x7f6614107dd0 2024-07-31T15:14:22.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f661906f640 1 -- 172.21.15.67:0/250570619 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66141a0100 con 0x7f6614107dd0 2024-07-31T15:14:22.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.045+0000 7f660bfff640 1 -- 172.21.15.67:0/250570619 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65fc005020 con 0x7f6614107dd0 2024-07-31T15:14:22.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.046+0000 7f660bfff640 1 -- 172.21.15.67:0/250570619 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc004a00 con 0x7f6614107dd0 2024-07-31T15:14:22.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.046+0000 7f661906f640 1 -- 172.21.15.67:0/250570619 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6614103b60 con 0x7f6614107dd0 2024-07-31T15:14:22.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.048+0000 7f661f5c0740 1 Processor -- start 2024-07-31T15:14:22.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.048+0000 7f661f5c0740 1 -- start start 2024-07-31T15:14:22.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f661f5c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5532c070 0x561e556d1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f661f5c0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e553031b0 0x561e556d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f661f5c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5530b390 0x561e556d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f661f5c0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561e55647bb0 con 0x561e553031b0 2024-07-31T15:14:22.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f661f5c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561e556370b0 con 0x561e5532c070 2024-07-31T15:14:22.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f661f5c0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561e5563b0f0 con 0x561e5530b390 2024-07-31T15:14:22.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f6612d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e553031b0 0x561e556d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f6612d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e553031b0 0x561e556d1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56152/0 (socket says 172.21.15.67:56152) 2024-07-31T15:14:22.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f6612d9d640 1 -- 172.21.15.67:0/68661576 learned_addr learned my addr 172.21.15.67:0/68661576 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:22.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f661359e640 1 --2- 172.21.15.67:0/68661576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5532c070 0x561e556d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f6612d9d640 1 -- 172.21.15.67:0/68661576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5530b390 msgr2=0x561e556d1ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-31T15:14:22.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f6612d9d640 1 --2- 172.21.15.67:0/68661576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5530b390 0x561e556d1ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f6612d9d640 1 -- 172.21.15.67:0/68661576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5532c070 msgr2=0x561e556d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:22.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f6612d9d640 1 --2- 172.21.15.67:0/68661576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5532c070 0x561e556d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f6612d9d640 1 -- 172.21.15.67:0/68661576 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e555ddb90 con 0x561e553031b0 2024-07-31T15:14:22.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f660bfff640 1 -- 172.21.15.67:0/250570619 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f65fc004ba0 con 0x7f6614107dd0 2024-07-31T15:14:22.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.049+0000 7f661359e640 1 --2- 172.21.15.67:0/68661576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5532c070 0x561e556d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.050+0000 7f6612d9d640 1 --2- 172.21.15.67:0/68661576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e553031b0 0x561e556d1ab0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f660c000c00 tx=0x7f660c005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a94a299fe2c8de7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.050+0000 7f66097fa640 1 -- 172.21.15.67:0/68661576 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c035040 con 0x561e553031b0 2024-07-31T15:14:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.050+0000 7f66097fa640 1 -- 172.21.15.67:0/68661576 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f660c031820 con 0x561e553031b0 2024-07-31T15:14:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.050+0000 7f66097fa640 1 -- 172.21.15.67:0/68661576 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c0049e0 con 0x561e553031b0 2024-07-31T15:14:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.050+0000 7f660bfff640 1 --2- 172.21.15.67:0/250570619 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f66000b3d40 0x7f66000b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.050+0000 7f661359e640 1 --2- 172.21.15.67:0/250570619 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f66000b3d40 0x7f66000b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.050+0000 7f660bfff640 1 -- 172.21.15.67:0/250570619 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f65fc0e4310 con 0x7f6614107dd0 2024-07-31T15:14:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.051+0000 7f661f5c0740 1 -- 172.21.15.67:0/68661576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e553031b0 msgr2=0x561e556d1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:22.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.051+0000 7f661f5c0740 1 --2- 172.21.15.67:0/68661576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e553031b0 0x561e556d1ab0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f660c000c00 tx=0x7f660c005ed0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.051+0000 7f661359e640 1 --2- 172.21.15.67:0/250570619 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f66000b3d40 0x7f66000b6250 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6604009fd0 tx=0x7f6604009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:22.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.051+0000 7f661f5c0740 1 -- 172.21.15.67:0/68661576 shutdown_connections 2024-07-31T15:14:22.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.051+0000 7f661f5c0740 1 --2- 172.21.15.67:0/68661576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5530b390 0x561e556d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.051+0000 7f661f5c0740 1 --2- 172.21.15.67:0/68661576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e553031b0 0x561e556d1ab0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.051+0000 7f661f5c0740 1 --2- 172.21.15.67:0/68661576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5532c070 0x561e556d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.051+0000 7f661f5c0740 1 -- 172.21.15.67:0/68661576 >> 172.21.15.67:0/68661576 conn(0x561e55320460 msgr2=0x561e556cf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:22.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.051+0000 7f661f5c0740 1 -- 172.21.15.67:0/68661576 shutdown_connections 2024-07-31T15:14:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.051+0000 7f661f5c0740 1 -- 172.21.15.67:0/68661576 wait complete. 2024-07-31T15:14:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.051+0000 7f661f5c0740 1 -- start start 2024-07-31T15:14:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.052+0000 7f661f5c0740 1 Processor -- start 2024-07-31T15:14:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.052+0000 7f661f5c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e553031b0 0x561e556dd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.052+0000 7f661f5c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 0x561e556dd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.052+0000 7f661f5c0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e5532c070 0x561e556e8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.052+0000 7f661f5c0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561e5549a700 con 0x561e5532c070 2024-07-31T15:14:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.052+0000 7f661f5c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561e555d99e0 con 0x561e5530b390 2024-07-31T15:14:22.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.052+0000 7f661f5c0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561e556dfcf0 con 0x561e553031b0 2024-07-31T15:14:22.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.052+0000 7f661f5c0740 10 client.0 _ll_register_callbacks cb 0x561e555f44f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:22.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.052+0000 7f6612d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e553031b0 0x561e556dd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.052+0000 7f6612d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e553031b0 0x561e556dd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:43492/0 (socket says 172.21.15.67:43492) 2024-07-31T15:14:22.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.052+0000 7f6612d9d640 1 -- 172.21.15.67:0/3125556925 learned_addr learned my addr 172.21.15.67:0/3125556925 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:22.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.052+0000 7f661359e640 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e5532c070 0x561e556e8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.052+0000 7f6612d9d640 1 -- 172.21.15.67:0/3125556925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 msgr2=0x561e556dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:22.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f661259c640 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 0x561e556dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f6612d9d640 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 0x561e556dd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f6612d9d640 1 -- 172.21.15.67:0/3125556925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e5532c070 msgr2=0x561e556e8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:22.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f6612d9d640 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e5532c070 0x561e556e8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f6612d9d640 1 -- 172.21.15.67:0/3125556925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e55397650 con 0x561e553031b0 2024-07-31T15:14:22.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f661359e640 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e5532c070 0x561e556e8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:22.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f661259c640 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 0x561e556dd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:14:22.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f6612d9d640 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e553031b0 0x561e556dd1f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f660c004350 tx=0x7f660c0054e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c79c7881c4e8772c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:22.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f65fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:14:22.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f65fa7fc640 1 -- 172.21.15.67:0/3125556925 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c04f060 con 0x561e553031b0 2024-07-31T15:14:22.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f661f5c0740 1 -- 172.21.15.67:0/3125556925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f660c031820 con 0x561e553031b0 2024-07-31T15:14:22.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f65fa7fc640 1 -- 172.21.15.67:0/3125556925 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f660c003800 con 0x561e553031b0 2024-07-31T15:14:22.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f65fa7fc640 1 -- 172.21.15.67:0/3125556925 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f660c056050 con 0x561e553031b0 2024-07-31T15:14:22.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f661f5c0740 10 client.29755 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:22.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.053+0000 7f661f5c0740 1 -- 172.21.15.67:0/3125556925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x561e556ebd80 con 0x561e553031b0 2024-07-31T15:14:22.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.054+0000 7f65fa7fc640 1 -- 172.21.15.67:0/3125556925 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f660c031820 con 0x561e553031b0 2024-07-31T15:14:22.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.054+0000 7f65fa7fc640 1 -- 172.21.15.67:0/3125556925 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f660c0467c0 con 0x561e553031b0 2024-07-31T15:14:22.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.054+0000 7f661f5c0740 10 client.29755 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:22.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.054+0000 7f661f5c0740 10 client.29755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:14:22.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.054+0000 7f661f5c0740 20 client.29755 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:22.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.054+0000 7f661f5c0740 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e552ea9f0 0x561e556ef0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.054+0000 7f661f5c0740 4 client.29755 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:14:22.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.054+0000 7f661f5c0740 1 -- 172.21.15.67:0/3125556925 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e55649b00 con 0x561e552ea9f0 2024-07-31T15:14:22.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.054+0000 7f661259c640 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e552ea9f0 0x561e556ef0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.054+0000 7f65fa7fc640 10 client.29755 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:22.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.054+0000 7f661259c640 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e552ea9f0 0x561e556ef0e0 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64b0ed27248131cd server_cookie=6529d23924ab9f7d in_seq=0 out_seq=0 2024-07-31T15:14:22.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.298+0000 7f65fa7fc640 1 -- 172.21.15.67:0/3125556925 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e55649b00 con 0x561e552ea9f0 2024-07-31T15:14:22.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.298+0000 7f65fa7fc640 10 client.29755 handle_command_reply: tid=0 2024-07-31T15:14:22.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.298+0000 7f661f5c0740 1 client.29755 shutdown 2024-07-31T15:14:22.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.298+0000 7f661f5c0740 1 -- 172.21.15.67:0/3125556925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e553031b0 msgr2=0x561e556dd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:22.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.298+0000 7f661f5c0740 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e553031b0 0x561e556dd1f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f660c004350 tx=0x7f660c0054e0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.298+0000 7f661f5c0740 1 -- 172.21.15.67:0/3125556925 shutdown_connections 2024-07-31T15:14:22.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.298+0000 7f661f5c0740 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e552ea9f0 0x561e556ef0e0 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.298+0000 7f661f5c0740 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e5532c070 0x561e556e8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.298+0000 7f65fa7fc640 0 client.29755 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:22.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.298+0000 7f661f5c0740 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 0x561e556dd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.298+0000 7f661f5c0740 1 --2- 172.21.15.67:0/3125556925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e553031b0 0x561e556dd1f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.298+0000 7f661f5c0740 1 -- 172.21.15.67:0/3125556925 >> 172.21.15.67:0/3125556925 conn(0x561e55320460 msgr2=0x561e556cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:22.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.298+0000 7f661f5c0740 1 -- 172.21.15.67:0/3125556925 shutdown_connections 2024-07-31T15:14:22.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.298+0000 7f661f5c0740 1 -- 172.21.15.67:0/3125556925 wait complete. 2024-07-31T15:14:22.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.299+0000 7f661f5c0740 20 client.29755 trim_cache size 0 max 16384 2024-07-31T15:14:22.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.313+0000 7f661f5c0740 1 Processor -- start 2024-07-31T15:14:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.313+0000 7f661f5c0740 1 -- start start 2024-07-31T15:14:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.313+0000 7f661f5c0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e553031b0 0x561e556dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.313+0000 7f661f5c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 0x561e556d6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.313+0000 7f661f5c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5532c070 0x561e556e8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.313+0000 7f661f5c0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f66141b2940 con 0x561e553031b0 2024-07-31T15:14:22.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.313+0000 7f661f5c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f66141b4480 con 0x561e5530b390 2024-07-31T15:14:22.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.313+0000 7f661f5c0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f66141b3010 con 0x561e5532c070 2024-07-31T15:14:22.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.313+0000 7f6612d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e553031b0 0x561e556dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.313+0000 7f6612d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e553031b0 0x561e556dbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56156/0 (socket says 172.21.15.67:56156) 2024-07-31T15:14:22.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.313+0000 7f6612d9d640 1 -- 172.21.15.67:0/1930727752 learned_addr learned my addr 172.21.15.67:0/1930727752 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:22.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.313+0000 7f661359e640 1 --2- 172.21.15.67:0/1930727752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5532c070 0x561e556e8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.313+0000 7f661259c640 1 --2- 172.21.15.67:0/1930727752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 0x561e556d6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.314+0000 7f6612d9d640 1 -- 172.21.15.67:0/1930727752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5532c070 msgr2=0x561e556e8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:22.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.314+0000 7f6612d9d640 1 --2- 172.21.15.67:0/1930727752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5532c070 0x561e556e8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.314+0000 7f6612d9d640 1 -- 172.21.15.67:0/1930727752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 msgr2=0x561e556d6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:22.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.314+0000 7f6612d9d640 1 --2- 172.21.15.67:0/1930727752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 0x561e556d6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.314+0000 7f6612d9d640 1 -- 172.21.15.67:0/1930727752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e556f52c0 con 0x561e553031b0 2024-07-31T15:14:22.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.314+0000 7f6612d9d640 1 --2- 172.21.15.67:0/1930727752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e553031b0 0x561e556dbee0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f660c03c460 tx=0x7f660c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bc9cee6cd7263b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:22.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.314+0000 7f6608ff9640 1 -- 172.21.15.67:0/1930727752 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c04f040 con 0x561e553031b0 2024-07-31T15:14:22.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.314+0000 7f6608ff9640 1 -- 172.21.15.67:0/1930727752 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f660c046450 con 0x561e553031b0 2024-07-31T15:14:22.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.314+0000 7f6608ff9640 1 -- 172.21.15.67:0/1930727752 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c035040 con 0x561e553031b0 2024-07-31T15:14:22.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.315+0000 7f661f5c0740 1 -- 172.21.15.67:0/1930727752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e553031b0 msgr2=0x561e556dbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:22.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.315+0000 7f661f5c0740 1 --2- 172.21.15.67:0/1930727752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e553031b0 0x561e556dbee0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f660c03c460 tx=0x7f660c001e70 comp rx=0 tx=0).stop 2024-07-31T15:14:22.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.315+0000 7f661f5c0740 1 -- 172.21.15.67:0/1930727752 shutdown_connections 2024-07-31T15:14:22.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.315+0000 7f661f5c0740 1 --2- 172.21.15.67:0/1930727752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5532c070 0x561e556e8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.315+0000 7f661f5c0740 1 --2- 172.21.15.67:0/1930727752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 0x561e556d6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.315+0000 7f661f5c0740 1 --2- 172.21.15.67:0/1930727752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e553031b0 0x561e556dbee0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.315+0000 7f661f5c0740 1 -- 172.21.15.67:0/1930727752 >> 172.21.15.67:0/1930727752 conn(0x561e55320460 msgr2=0x561e556d6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:22.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.315+0000 7f661f5c0740 1 -- 172.21.15.67:0/1930727752 shutdown_connections 2024-07-31T15:14:22.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.315+0000 7f661f5c0740 1 -- 172.21.15.67:0/1930727752 wait complete. 2024-07-31T15:14:22.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.316+0000 7f661f5c0740 1 -- start start 2024-07-31T15:14:22.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.316+0000 7f661f5c0740 1 Processor -- start 2024-07-31T15:14:22.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.316+0000 7f661f5c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e553031b0 0x561e556efb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.316+0000 7f661f5c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 0x561e556f0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.316+0000 7f661f5c0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e5532c070 0x561e5574e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.316+0000 7f661f5c0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f66141b3db0 con 0x561e5532c070 2024-07-31T15:14:22.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.316+0000 7f661f5c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f66141b36e0 con 0x561e5530b390 2024-07-31T15:14:22.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.316+0000 7f661f5c0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f66141b0940 con 0x561e553031b0 2024-07-31T15:14:22.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.316+0000 7f661f5c0740 10 client.0 _ll_register_callbacks cb 0x7f660c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:22.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.316+0000 7f661359e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e5532c070 0x561e5574e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.316+0000 7f661259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 0x561e556f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.316+0000 7f661259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 0x561e556f0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37030/0 (socket says 172.21.15.67:37030) 2024-07-31T15:14:22.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.316+0000 7f661259c640 1 -- 172.21.15.67:0/3837064464 learned_addr learned my addr 172.21.15.67:0/3837064464 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:22.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.316+0000 7f6612d9d640 1 --2- 172.21.15.67:0/3837064464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e553031b0 0x561e556efb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.317+0000 7f6612d9d640 1 -- 172.21.15.67:0/3837064464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 msgr2=0x561e556f0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:22.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.317+0000 7f6612d9d640 1 --2- 172.21.15.67:0/3837064464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 0x561e556f0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.317+0000 7f6612d9d640 1 -- 172.21.15.67:0/3837064464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e5532c070 msgr2=0x561e5574e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:22.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.317+0000 7f6612d9d640 1 --2- 172.21.15.67:0/3837064464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e5532c070 0x561e5574e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.317+0000 7f6612d9d640 1 -- 172.21.15.67:0/3837064464 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5574ecc0 con 0x561e553031b0 2024-07-31T15:14:22.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.317+0000 7f6612d9d640 1 --2- 172.21.15.67:0/3837064464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e553031b0 0x561e556efb30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f660c046d00 tx=0x7f660c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6907f675ef6d5417 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:22.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.317+0000 7f65fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-07-31T15:14:22.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.317+0000 7f65fa7fc640 1 -- 172.21.15.67:0/3837064464 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c068020 con 0x561e553031b0 2024-07-31T15:14:22.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.317+0000 7f65fa7fc640 1 -- 172.21.15.67:0/3837064464 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f660c003a40 con 0x561e553031b0 2024-07-31T15:14:22.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.317+0000 7f661f5c0740 1 -- 172.21.15.67:0/3837064464 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f660c046450 con 0x561e553031b0 2024-07-31T15:14:22.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.317+0000 7f65fa7fc640 1 -- 172.21.15.67:0/3837064464 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f660c073050 con 0x561e553031b0 2024-07-31T15:14:22.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.317+0000 7f661f5c0740 10 client.29761 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:22.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.317+0000 7f661f5c0740 1 -- 172.21.15.67:0/3837064464 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x561e557571e0 con 0x561e553031b0 2024-07-31T15:14:22.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.318+0000 7f65fa7fc640 1 -- 172.21.15.67:0/3837064464 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f660c046450 con 0x561e553031b0 2024-07-31T15:14:22.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.318+0000 7f65fa7fc640 1 -- 172.21.15.67:0/3837064464 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f660c03c490 con 0x561e553031b0 2024-07-31T15:14:22.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.318+0000 7f661f5c0740 10 client.29761 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:22.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.318+0000 7f661f5c0740 10 client.29761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:14:22.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.318+0000 7f661f5c0740 20 client.29761 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:22.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.318+0000 7f661f5c0740 1 --2- 172.21.15.67:0/3837064464 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e552bfda0 0x561e55759940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:22.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.318+0000 7f661f5c0740 4 client.29761 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:14:22.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.318+0000 7f661f5c0740 1 -- 172.21.15.67:0/3837064464 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e556455a0 con 0x561e552bfda0 2024-07-31T15:14:22.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.318+0000 7f661259c640 1 --2- 172.21.15.67:0/3837064464 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e552bfda0 0x561e55759940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:22.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.319+0000 7f65fa7fc640 10 client.29761 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:22.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.319+0000 7f661259c640 1 --2- 172.21.15.67:0/3837064464 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e552bfda0 0x561e55759940 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec2dd9353b6aacb7 server_cookie=85ce4d6dbf43bef2 in_seq=0 out_seq=0 2024-07-31T15:14:22.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f65fa7fc640 1 -- 172.21.15.67:0/3837064464 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e556455a0 con 0x561e552bfda0 2024-07-31T15:14:22.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f65fa7fc640 10 client.29761 handle_command_reply: tid=0 2024-07-31T15:14:22.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f661f5c0740 1 client.29761 shutdown 2024-07-31T15:14:22.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f661f5c0740 1 -- 172.21.15.67:0/3837064464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e553031b0 msgr2=0x561e556efb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:22.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f661f5c0740 1 --2- 172.21.15.67:0/3837064464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e553031b0 0x561e556efb30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f660c046d00 tx=0x7f660c02a410 comp rx=0 tx=0).stop 2024-07-31T15:14:22.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f661f5c0740 1 -- 172.21.15.67:0/3837064464 shutdown_connections 2024-07-31T15:14:22.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f661f5c0740 1 --2- 172.21.15.67:0/3837064464 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561e552bfda0 0x561e55759940 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f661f5c0740 1 --2- 172.21.15.67:0/3837064464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561e5532c070 0x561e5574e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f661f5c0740 1 --2- 172.21.15.67:0/3837064464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e5530b390 0x561e556f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f65fa7fc640 0 client.29761 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f661f5c0740 1 --2- 172.21.15.67:0/3837064464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e553031b0 0x561e556efb30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f661f5c0740 1 -- 172.21.15.67:0/3837064464 >> 172.21.15.67:0/3837064464 conn(0x561e55320460 msgr2=0x561e556ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f661f5c0740 1 -- 172.21.15.67:0/3837064464 shutdown_connections 2024-07-31T15:14:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f661f5c0740 1 -- 172.21.15.67:0/3837064464 wait complete. 2024-07-31T15:14:22.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.320+0000 7f661f5c0740 20 client.29761 trim_cache size 0 max 16384 2024-07-31T15:14:22.331 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:14:22.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.321+0000 7f661906f640 1 -- 172.21.15.67:0/250570619 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f66000b3d40 msgr2=0x7f66000b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:22.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.321+0000 7f661906f640 1 --2- 172.21.15.67:0/250570619 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f66000b3d40 0x7f66000b6250 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6604009fd0 tx=0x7f6604009500 comp rx=0 tx=0).stop 2024-07-31T15:14:22.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.321+0000 7f661906f640 1 -- 172.21.15.67:0/250570619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6614107dd0 msgr2=0x7f661419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:22.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.321+0000 7f661906f640 1 --2- 172.21.15.67:0/250570619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6614107dd0 0x7f661419a440 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00e9c0 tx=0x7f65fc00ee80 comp rx=0 tx=0).stop 2024-07-31T15:14:22.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.321+0000 7f661906f640 1 -- 172.21.15.67:0/250570619 shutdown_connections 2024-07-31T15:14:22.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.321+0000 7f661906f640 1 --2- 172.21.15.67:0/250570619 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f66000b3d40 0x7f66000b6250 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.321+0000 7f661906f640 1 --2- 172.21.15.67:0/250570619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6614108720 0x7f661419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.321+0000 7f661906f640 1 --2- 172.21.15.67:0/250570619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6614107dd0 0x7f661419a440 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.321+0000 7f661906f640 1 --2- 172.21.15.67:0/250570619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6614102490 0x7f6614199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:22.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.321+0000 7f661906f640 1 -- 172.21.15.67:0/250570619 >> 172.21.15.67:0/250570619 conn(0x7f6614079160 msgr2=0x7f66141014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:22.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.321+0000 7f661906f640 1 -- 172.21.15.67:0/250570619 shutdown_connections 2024-07-31T15:14:22.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:22.321+0000 7f661906f640 1 -- 172.21.15.67:0/250570619 wait complete. 2024-07-31T15:14:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.210+0000 7f74da9b6640 1 -- 172.21.15.67:0/3047539088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d41086a0 msgr2=0x7f74d410eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.210+0000 7f74da9b6640 1 --2- 172.21.15.67:0/3047539088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d41086a0 0x7f74d410eed0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009ca0 tx=0x7f74cc02a8c0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.210+0000 7f74da9b6640 1 -- 172.21.15.67:0/3047539088 shutdown_connections 2024-07-31T15:14:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.210+0000 7f74da9b6640 1 --2- 172.21.15.67:0/3047539088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d41086a0 0x7f74d410eed0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.210+0000 7f74da9b6640 1 --2- 172.21.15.67:0/3047539088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d4107d50 0x7f74d4108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.210+0000 7f74da9b6640 1 --2- 172.21.15.67:0/3047539088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d4102470 0x7f74d4102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.210+0000 7f74da9b6640 1 -- 172.21.15.67:0/3047539088 >> 172.21.15.67:0/3047539088 conn(0x7f74d407a140 msgr2=0x7f74d407a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.210+0000 7f74da9b6640 1 -- 172.21.15.67:0/3047539088 shutdown_connections 2024-07-31T15:14:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.210+0000 7f74da9b6640 1 -- 172.21.15.67:0/3047539088 wait complete. 2024-07-31T15:14:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.211+0000 7f74da9b6640 1 Processor -- start 2024-07-31T15:14:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.211+0000 7f74da9b6640 1 -- start start 2024-07-31T15:14:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74da9b6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d4102470 0x7f74d4199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74da9b6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d4107d50 0x7f74d419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74da9b6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d41086a0 0x7f74d419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74da9b6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4112350 con 0x7f74d41086a0 2024-07-31T15:14:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74da9b6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41121d0 con 0x7f74d4107d50 2024-07-31T15:14:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74da9b6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41124d0 con 0x7f74d4102470 2024-07-31T15:14:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74d37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d4107d50 0x7f74d419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74d8f53640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d41086a0 0x7f74d419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74d37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d4107d50 0x7f74d419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37048/0 (socket says 172.21.15.67:37048) 2024-07-31T15:14:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74d37fe640 1 -- 172.21.15.67:0/3766315229 learned_addr learned my addr 172.21.15.67:0/3766315229 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74d3fff640 1 --2- 172.21.15.67:0/3766315229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d4102470 0x7f74d4199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74d37fe640 1 -- 172.21.15.67:0/3766315229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d4102470 msgr2=0x7f74d4199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74d37fe640 1 --2- 172.21.15.67:0/3766315229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d4102470 0x7f74d4199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74d37fe640 1 -- 172.21.15.67:0/3766315229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d41086a0 msgr2=0x7f74d419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74d37fe640 1 --2- 172.21.15.67:0/3766315229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d41086a0 0x7f74d419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74d37fe640 1 -- 172.21.15.67:0/3766315229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d419fdb0 con 0x7f74d4107d50 2024-07-31T15:14:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74d3fff640 1 --2- 172.21.15.67:0/3766315229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d4102470 0x7f74d4199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:14:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74d37fe640 1 --2- 172.21.15.67:0/3766315229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d4107d50 0x7f74d419a420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f74c800e9f0 tx=0x7f74c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.212+0000 7f74d17fa640 1 -- 172.21.15.67:0/3766315229 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c800ccc0 con 0x7f74d4107d50 2024-07-31T15:14:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.213+0000 7f74da9b6640 1 -- 172.21.15.67:0/3766315229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d41a00a0 con 0x7f74d4107d50 2024-07-31T15:14:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.213+0000 7f74d17fa640 1 -- 172.21.15.67:0/3766315229 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c80108e0 con 0x7f74d4107d50 2024-07-31T15:14:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.213+0000 7f74d17fa640 1 -- 172.21.15.67:0/3766315229 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8018860 con 0x7f74d4107d50 2024-07-31T15:14:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.213+0000 7f74da9b6640 1 -- 172.21.15.67:0/3766315229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d41a6830 con 0x7f74d4107d50 2024-07-31T15:14:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.215+0000 7f74d17fa640 1 -- 172.21.15.67:0/3766315229 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f74c8010430 con 0x7f74d4107d50 2024-07-31T15:14:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.215+0000 7f74e1987740 1 Processor -- start 2024-07-31T15:14:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.215+0000 7f74e1987740 1 -- start start 2024-07-31T15:14:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74e1987740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3531b0 0x56162c6f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74e1987740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c32a2f0 0x56162c6f6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74e1987740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c3324d0 0x56162c6f6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74e1987740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56162c66b3b0 con 0x56162c3324d0 2024-07-31T15:14:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74e1987740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56162c65a8b0 con 0x56162c3531b0 2024-07-31T15:14:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74e1987740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56162c65e8f0 con 0x56162c32a2f0 2024-07-31T15:14:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74d8f53640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3531b0 0x56162c6f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74d37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c3324d0 0x56162c6f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74d8f53640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3531b0 0x56162c6f5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37062/0 (socket says 172.21.15.67:37062) 2024-07-31T15:14:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74d3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c32a2f0 0x56162c6f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74d8f53640 1 -- 172.21.15.67:0/684714675 learned_addr learned my addr 172.21.15.67:0/684714675 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74d8f53640 1 -- 172.21.15.67:0/684714675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c32a2f0 msgr2=0x56162c6f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74d8f53640 1 --2- 172.21.15.67:0/684714675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c32a2f0 0x56162c6f6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74d8f53640 1 -- 172.21.15.67:0/684714675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c3324d0 msgr2=0x56162c6f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74d8f53640 1 --2- 172.21.15.67:0/684714675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c3324d0 0x56162c6f6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74d8f53640 1 -- 172.21.15.67:0/684714675 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162c6019a0 con 0x56162c3531b0 2024-07-31T15:14:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74d17fa640 1 --2- 172.21.15.67:0/3766315229 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f74bc0b3d40 0x7f74bc0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74d8f53640 1 --2- 172.21.15.67:0/684714675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3531b0 0x56162c6f5ce0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f74cc02f4f0 tx=0x7f74cc009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=324c1e0bfd618a5c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.216+0000 7f74d3fff640 1 --2- 172.21.15.67:0/3766315229 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f74bc0b3d40 0x7f74bc0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.217+0000 7f74ba7fc640 1 -- 172.21.15.67:0/684714675 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc031cf0 con 0x56162c3531b0 2024-07-31T15:14:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.217+0000 7f74d17fa640 1 -- 172.21.15.67:0/3766315229 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f74c80e4670 con 0x7f74d4107d50 2024-07-31T15:14:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.217+0000 7f74ba7fc640 1 -- 172.21.15.67:0/684714675 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74cc036070 con 0x56162c3531b0 2024-07-31T15:14:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.217+0000 7f74ba7fc640 1 -- 172.21.15.67:0/684714675 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc0042a0 con 0x56162c3531b0 2024-07-31T15:14:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.217+0000 7f74e1987740 1 -- 172.21.15.67:0/684714675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3531b0 msgr2=0x56162c6f5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.217+0000 7f74e1987740 1 --2- 172.21.15.67:0/684714675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3531b0 0x56162c6f5ce0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f74cc02f4f0 tx=0x7f74cc009800 comp rx=0 tx=0).stop 2024-07-31T15:14:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.217+0000 7f74d3fff640 1 --2- 172.21.15.67:0/3766315229 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f74bc0b3d40 0x7f74bc0b6250 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x56162c6f4960 tx=0x7f74c400b570 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.217+0000 7f74e1987740 1 -- 172.21.15.67:0/684714675 shutdown_connections 2024-07-31T15:14:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.217+0000 7f74e1987740 1 --2- 172.21.15.67:0/684714675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c3324d0 0x56162c6f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.217+0000 7f74e1987740 1 --2- 172.21.15.67:0/684714675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c32a2f0 0x56162c6f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.217+0000 7f74e1987740 1 --2- 172.21.15.67:0/684714675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3531b0 0x56162c6f5ce0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.217+0000 7f74e1987740 1 -- 172.21.15.67:0/684714675 >> 172.21.15.67:0/684714675 conn(0x56162c347300 msgr2=0x56162c6f3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.217+0000 7f74e1987740 1 -- 172.21.15.67:0/684714675 shutdown_connections 2024-07-31T15:14:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.217+0000 7f74e1987740 1 -- 172.21.15.67:0/684714675 wait complete. 2024-07-31T15:14:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.218+0000 7f74e1987740 1 -- start start 2024-07-31T15:14:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.218+0000 7f74e1987740 1 Processor -- start 2024-07-31T15:14:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.218+0000 7f74e1987740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c32a2f0 0x56162c701860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.218+0000 7f74e1987740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c3324d0 0x56162c701da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.218+0000 7f74e1987740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c3531b0 0x56162c70d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.218+0000 7f74e1987740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56162c594ab0 con 0x56162c3531b0 2024-07-31T15:14:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.218+0000 7f74e1987740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56162c5e2cf0 con 0x56162c32a2f0 2024-07-31T15:14:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.218+0000 7f74e1987740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56162c704280 con 0x56162c3324d0 2024-07-31T15:14:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74e1987740 10 client.0 _ll_register_callbacks cb 0x56162c6549f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74d3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c32a2f0 0x56162c701860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74d8f53640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c3531b0 0x56162c70d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74d3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c32a2f0 0x56162c701860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37070/0 (socket says 172.21.15.67:37070) 2024-07-31T15:14:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74d3fff640 1 -- 172.21.15.67:0/43843359 learned_addr learned my addr 172.21.15.67:0/43843359 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74d37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c3324d0 0x56162c701da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74d8f53640 1 -- 172.21.15.67:0/43843359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c3324d0 msgr2=0x56162c701da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74d8f53640 1 --2- 172.21.15.67:0/43843359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c3324d0 0x56162c701da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74d8f53640 1 -- 172.21.15.67:0/43843359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c32a2f0 msgr2=0x56162c701860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74d8f53640 1 --2- 172.21.15.67:0/43843359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c32a2f0 0x56162c701860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74d8f53640 1 -- 172.21.15.67:0/43843359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162c3be790 con 0x56162c3531b0 2024-07-31T15:14:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74d8f53640 1 --2- 172.21.15.67:0/43843359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c3531b0 0x56162c70d5d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f74cc006010 tx=0x7f74cc003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fe443fd8711e6 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:14:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74b37fe640 1 -- 172.21.15.67:0/43843359 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc009360 con 0x56162c3531b0 2024-07-31T15:14:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74b37fe640 1 -- 172.21.15.67:0/43843359 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f74cc031660 con 0x56162c3531b0 2024-07-31T15:14:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74e1987740 1 -- 172.21.15.67:0/43843359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74cc036070 con 0x56162c3531b0 2024-07-31T15:14:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74b37fe640 1 -- 172.21.15.67:0/43843359 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f74cc031860 con 0x56162c3531b0 2024-07-31T15:14:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74e1987740 10 client.20439 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.219+0000 7f74e1987740 1 -- 172.21.15.67:0/43843359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56162c437620 con 0x56162c3531b0 2024-07-31T15:14:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.220+0000 7f74b37fe640 1 -- 172.21.15.67:0/43843359 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74cc036070 con 0x56162c3531b0 2024-07-31T15:14:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.220+0000 7f74b37fe640 1 -- 172.21.15.67:0/43843359 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f74cc043020 con 0x56162c3531b0 2024-07-31T15:14:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.220+0000 7f74e1987740 10 client.20439 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.220+0000 7f74e1987740 10 client.20439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:14:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.220+0000 7f74e1987740 20 client.20439 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.220+0000 7f74e1987740 1 --2- 172.21.15.67:0/43843359 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56162c311d80 0x56162c713440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.220+0000 7f74e1987740 4 client.20439 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:14:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.220+0000 7f74e1987740 1 -- 172.21.15.67:0/43843359 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56162c66d300 con 0x56162c311d80 2024-07-31T15:14:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.220+0000 7f74d3fff640 1 --2- 172.21.15.67:0/43843359 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56162c311d80 0x56162c713440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.221+0000 7f74d3fff640 1 --2- 172.21.15.67:0/43843359 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56162c311d80 0x56162c713440 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a651b2e3e24becc server_cookie=68c94c7e4c564a26 in_seq=0 out_seq=0 2024-07-31T15:14:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.221+0000 7f74b37fe640 10 client.20439 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:23 smithi073 ceph-mon[36714]: pgmap v1145: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.6 MiB/s wr, 22 op/s 2024-07-31T15:14:23.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:23 smithi077 ceph-mon[37327]: pgmap v1145: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.6 MiB/s wr, 22 op/s 2024-07-31T15:14:23.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:23 smithi067 ceph-mon[32750]: pgmap v1145: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.6 MiB/s wr, 22 op/s 2024-07-31T15:14:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.876+0000 7f74b37fe640 1 -- 172.21.15.67:0/43843359 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56162c66d300 con 0x56162c311d80 2024-07-31T15:14:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.876+0000 7f74b37fe640 10 client.20439 handle_command_reply: tid=0 2024-07-31T15:14:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.876+0000 7f74e1987740 1 client.20439 shutdown 2024-07-31T15:14:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.877+0000 7f74e1987740 1 -- 172.21.15.67:0/43843359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c3531b0 msgr2=0x56162c70d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.877+0000 7f74e1987740 1 --2- 172.21.15.67:0/43843359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c3531b0 0x56162c70d5d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f74cc006010 tx=0x7f74cc003cd0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.877+0000 7f74e1987740 1 -- 172.21.15.67:0/43843359 shutdown_connections 2024-07-31T15:14:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.877+0000 7f74e1987740 1 --2- 172.21.15.67:0/43843359 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56162c311d80 0x56162c713440 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.877+0000 7f74e1987740 1 --2- 172.21.15.67:0/43843359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c3531b0 0x56162c70d5d0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.877+0000 7f74e1987740 1 --2- 172.21.15.67:0/43843359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c3324d0 0x56162c701da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.877+0000 7f74e1987740 1 --2- 172.21.15.67:0/43843359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c32a2f0 0x56162c701860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.877+0000 7f74b37fe640 0 client.20439 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.877+0000 7f74e1987740 1 -- 172.21.15.67:0/43843359 >> 172.21.15.67:0/43843359 conn(0x56162c347300 msgr2=0x56162c6f22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.877+0000 7f74e1987740 1 -- 172.21.15.67:0/43843359 shutdown_connections 2024-07-31T15:14:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.877+0000 7f74e1987740 1 -- 172.21.15.67:0/43843359 wait complete. 2024-07-31T15:14:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.877+0000 7f74e1987740 20 client.20439 trim_cache size 0 max 16384 2024-07-31T15:14:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.893+0000 7f74e1987740 1 Processor -- start 2024-07-31T15:14:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.893+0000 7f74e1987740 1 -- start start 2024-07-31T15:14:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.893+0000 7f74e1987740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c311d80 0x56162c701f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.893+0000 7f74e1987740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c32a2f0 0x56162c6fbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.893+0000 7f74e1987740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3324d0 0x56162c70d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.893+0000 7f74e1987740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41ae5a0 con 0x56162c32a2f0 2024-07-31T15:14:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.893+0000 7f74e1987740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41b00e0 con 0x56162c3324d0 2024-07-31T15:14:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.893+0000 7f74e1987740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41aec70 con 0x56162c311d80 2024-07-31T15:14:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.893+0000 7f74d37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c32a2f0 0x56162c6fbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.893+0000 7f74d37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c32a2f0 0x56162c6fbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56200/0 (socket says 172.21.15.67:56200) 2024-07-31T15:14:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.894+0000 7f74d8f53640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3324d0 0x56162c70d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.894+0000 7f74d37fe640 1 -- 172.21.15.67:0/1867614094 learned_addr learned my addr 172.21.15.67:0/1867614094 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.894+0000 7f74d3fff640 1 --2- 172.21.15.67:0/1867614094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c311d80 0x56162c701f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.894+0000 7f74d37fe640 1 -- 172.21.15.67:0/1867614094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c311d80 msgr2=0x56162c701f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.894+0000 7f74d37fe640 1 --2- 172.21.15.67:0/1867614094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c311d80 0x56162c701f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.894+0000 7f74d37fe640 1 -- 172.21.15.67:0/1867614094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3324d0 msgr2=0x56162c70d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.894+0000 7f74d37fe640 1 --2- 172.21.15.67:0/1867614094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3324d0 0x56162c70d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.894+0000 7f74d37fe640 1 -- 172.21.15.67:0/1867614094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162c6ffb30 con 0x56162c32a2f0 2024-07-31T15:14:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.894+0000 7f74d8f53640 1 --2- 172.21.15.67:0/1867614094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3324d0 0x56162c70d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.894+0000 7f74d37fe640 1 --2- 172.21.15.67:0/1867614094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c32a2f0 0x56162c6fbe20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f74c80e1ab0 tx=0x7f74c80096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc82fc1cc4ed6c0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.894+0000 7f74b9ffb640 1 -- 172.21.15.67:0/1867614094 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c80aa680 con 0x56162c32a2f0 2024-07-31T15:14:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.894+0000 7f74b9ffb640 1 -- 172.21.15.67:0/1867614094 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c8010720 con 0x56162c32a2f0 2024-07-31T15:14:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.894+0000 7f74b9ffb640 1 -- 172.21.15.67:0/1867614094 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c80a71a0 con 0x56162c32a2f0 2024-07-31T15:14:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.895+0000 7f74e1987740 1 -- 172.21.15.67:0/1867614094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c32a2f0 msgr2=0x56162c6fbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.895+0000 7f74e1987740 1 --2- 172.21.15.67:0/1867614094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c32a2f0 0x56162c6fbe20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f74c80e1ab0 tx=0x7f74c80096a0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.895+0000 7f74e1987740 1 -- 172.21.15.67:0/1867614094 shutdown_connections 2024-07-31T15:14:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.895+0000 7f74e1987740 1 --2- 172.21.15.67:0/1867614094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3324d0 0x56162c70d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.895+0000 7f74e1987740 1 --2- 172.21.15.67:0/1867614094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c32a2f0 0x56162c6fbe20 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.895+0000 7f74e1987740 1 --2- 172.21.15.67:0/1867614094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c311d80 0x56162c701f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.895+0000 7f74e1987740 1 -- 172.21.15.67:0/1867614094 >> 172.21.15.67:0/1867614094 conn(0x56162c347300 msgr2=0x56162c701860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.895+0000 7f74e1987740 1 -- 172.21.15.67:0/1867614094 shutdown_connections 2024-07-31T15:14:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.895+0000 7f74e1987740 1 -- 172.21.15.67:0/1867614094 wait complete. 2024-07-31T15:14:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.896+0000 7f74e1987740 1 -- start start 2024-07-31T15:14:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.896+0000 7f74e1987740 1 Processor -- start 2024-07-31T15:14:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.896+0000 7f74e1987740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c311d80 0x56162c706500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.896+0000 7f74e1987740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c32a2f0 0x56162c706a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.896+0000 7f74e1987740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3324d0 0x56162c706f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.896+0000 7f74e1987740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41afa10 con 0x56162c32a2f0 2024-07-31T15:14:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.896+0000 7f74e1987740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41af340 con 0x56162c3324d0 2024-07-31T15:14:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.896+0000 7f74e1987740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41ac5a0 con 0x56162c311d80 2024-07-31T15:14:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.896+0000 7f74e1987740 10 client.0 _ll_register_callbacks cb 0x7f74c80ea420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74d37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c32a2f0 0x56162c706a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74d8f53640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3324d0 0x56162c706f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74d37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c32a2f0 0x56162c706a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56208/0 (socket says 172.21.15.67:56208) 2024-07-31T15:14:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74d8f53640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3324d0 0x56162c706f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37080/0 (socket says 172.21.15.67:37080) 2024-07-31T15:14:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74d37fe640 1 -- 172.21.15.67:0/4077588783 learned_addr learned my addr 172.21.15.67:0/4077588783 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74d3fff640 1 --2- 172.21.15.67:0/4077588783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c311d80 0x56162c706500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74d37fe640 1 -- 172.21.15.67:0/4077588783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c311d80 msgr2=0x56162c706500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74d37fe640 1 --2- 172.21.15.67:0/4077588783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c311d80 0x56162c706500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74d37fe640 1 -- 172.21.15.67:0/4077588783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3324d0 msgr2=0x56162c706f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74d37fe640 1 --2- 172.21.15.67:0/4077588783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3324d0 0x56162c706f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74d37fe640 1 -- 172.21.15.67:0/4077588783 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162c779ac0 con 0x56162c32a2f0 2024-07-31T15:14:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74d8f53640 1 --2- 172.21.15.67:0/4077588783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3324d0 0x56162c706f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74d37fe640 1 --2- 172.21.15.67:0/4077588783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c32a2f0 0x56162c706a40 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f74c80a8180 tx=0x7f74c80a7430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2a77c4e9912ffc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:14:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74b37fe640 1 -- 172.21.15.67:0/4077588783 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c80aa680 con 0x56162c32a2f0 2024-07-31T15:14:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74b37fe640 1 -- 172.21.15.67:0/4077588783 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f74c80c5620 con 0x56162c32a2f0 2024-07-31T15:14:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.897+0000 7f74e1987740 1 -- 172.21.15.67:0/4077588783 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56162c668da0 con 0x56162c32a2f0 2024-07-31T15:14:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.898+0000 7f74b37fe640 1 -- 172.21.15.67:0/4077588783 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f74c80aaf10 con 0x56162c32a2f0 2024-07-31T15:14:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.898+0000 7f74e1987740 10 client.20445 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.898+0000 7f74e1987740 1 -- 172.21.15.67:0/4077588783 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56162c77ad50 con 0x56162c32a2f0 2024-07-31T15:14:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.898+0000 7f74b37fe640 1 -- 172.21.15.67:0/4077588783 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56162c668da0 con 0x56162c32a2f0 2024-07-31T15:14:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.898+0000 7f74b37fe640 1 -- 172.21.15.67:0/4077588783 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f74c80e6b80 con 0x56162c32a2f0 2024-07-31T15:14:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.898+0000 7f74e1987740 10 client.20445 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.898+0000 7f74e1987740 10 client.20445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-07-31T15:14:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.898+0000 7f74e1987740 20 client.20445 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.898+0000 7f74e1987740 1 --2- 172.21.15.67:0/4077588783 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56162c3531b0 0x56162c77d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.898+0000 7f74e1987740 4 client.20445 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-31T15:14:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.898+0000 7f74e1987740 1 -- 172.21.15.67:0/4077588783 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56162c6664e0 con 0x56162c3531b0 2024-07-31T15:14:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.898+0000 7f74d3fff640 1 --2- 172.21.15.67:0/4077588783 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56162c3531b0 0x56162c77d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.899+0000 7f74b37fe640 10 client.20445 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:23.899+0000 7f74d3fff640 1 --2- 172.21.15.67:0/4077588783 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56162c3531b0 0x56162c77d2f0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d77391beeb004bb3 server_cookie=111059884c8ea54 in_seq=0 out_seq=0 2024-07-31T15:14:24.102 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-31T15:14:24.102 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-31T15:14:24.372 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:14:25.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:25 smithi067 ceph-mon[32750]: pgmap v1146: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 14 op/s 2024-07-31T15:14:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74b37fe640 1 -- 172.21.15.67:0/4077588783 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56162c6664e0 con 0x56162c3531b0 2024-07-31T15:14:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74b37fe640 10 client.20445 handle_command_reply: tid=0 2024-07-31T15:14:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74e1987740 1 client.20445 shutdown 2024-07-31T15:14:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74e1987740 1 -- 172.21.15.67:0/4077588783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c32a2f0 msgr2=0x56162c706a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74e1987740 1 --2- 172.21.15.67:0/4077588783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c32a2f0 0x56162c706a40 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f74c80a8180 tx=0x7f74c80a7430 comp rx=0 tx=0).stop 2024-07-31T15:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74e1987740 1 -- 172.21.15.67:0/4077588783 shutdown_connections 2024-07-31T15:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74e1987740 1 --2- 172.21.15.67:0/4077588783 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56162c3531b0 0x56162c77d2f0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74e1987740 1 --2- 172.21.15.67:0/4077588783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56162c3324d0 0x56162c706f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74e1987740 1 --2- 172.21.15.67:0/4077588783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56162c32a2f0 0x56162c706a40 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74b37fe640 0 client.20445 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74e1987740 1 --2- 172.21.15.67:0/4077588783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56162c311d80 0x56162c706500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74e1987740 1 -- 172.21.15.67:0/4077588783 >> 172.21.15.67:0/4077588783 conn(0x56162c347300 msgr2=0x56162c700b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74e1987740 1 -- 172.21.15.67:0/4077588783 shutdown_connections 2024-07-31T15:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74e1987740 1 -- 172.21.15.67:0/4077588783 wait complete. 2024-07-31T15:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.598+0000 7f74e1987740 20 client.20445 trim_cache size 0 max 16384 2024-07-31T15:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:{ 2024-07-31T15:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "message": "", 2024-07-31T15:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout: "return_code": 0 2024-07-31T15:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stdout:} 2024-07-31T15:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.599+0000 7f74da9b6640 1 -- 172.21.15.67:0/3766315229 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f74bc0b3d40 msgr2=0x7f74bc0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.599+0000 7f74da9b6640 1 --2- 172.21.15.67:0/3766315229 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f74bc0b3d40 0x7f74bc0b6250 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x56162c6f4960 tx=0x7f74c400b570 comp rx=0 tx=0).stop 2024-07-31T15:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.599+0000 7f74da9b6640 1 -- 172.21.15.67:0/3766315229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d4107d50 msgr2=0x7f74d419a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.599+0000 7f74da9b6640 1 --2- 172.21.15.67:0/3766315229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d4107d50 0x7f74d419a420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f74c800e9f0 tx=0x7f74c800eeb0 comp rx=0 tx=0).stop 2024-07-31T15:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.599+0000 7f74da9b6640 1 -- 172.21.15.67:0/3766315229 shutdown_connections 2024-07-31T15:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.599+0000 7f74da9b6640 1 --2- 172.21.15.67:0/3766315229 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f74bc0b3d40 0x7f74bc0b6250 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.599+0000 7f74da9b6640 1 --2- 172.21.15.67:0/3766315229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d41086a0 0x7f74d419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.599+0000 7f74da9b6640 1 --2- 172.21.15.67:0/3766315229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d4107d50 0x7f74d419a420 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.599+0000 7f74da9b6640 1 --2- 172.21.15.67:0/3766315229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d4102470 0x7f74d4199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.599+0000 7f74da9b6640 1 -- 172.21.15.67:0/3766315229 >> 172.21.15.67:0/3766315229 conn(0x7f74d407a140 msgr2=0x7f74d4101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.599+0000 7f74da9b6640 1 -- 172.21.15.67:0/3766315229 shutdown_connections 2024-07-31T15:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi067.front.sepia.ceph.com.smithi067.stderr:2024-07-31T15:14:25.599+0000 7f74da9b6640 1 -- 172.21.15.67:0/3766315229 wait complete. 2024-07-31T15:14:25.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:25 smithi073 ceph-mon[36714]: pgmap v1146: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 14 op/s 2024-07-31T15:14:25.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:25 smithi077 ceph-mon[37327]: pgmap v1146: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 14 op/s 2024-07-31T15:14:26.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.224+0000 7fa224c75640 1 -- 172.21.15.67:0/1600054321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa220108ed0 msgr2=0x7fa2201092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:26.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.224+0000 7fa224c75640 1 --2- 172.21.15.67:0/1600054321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa220108ed0 0x7fa2201092e0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fa208009900 tx=0x7fa20802a3f0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.224+0000 7fa224c75640 1 -- 172.21.15.67:0/1600054321 shutdown_connections 2024-07-31T15:14:26.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.224+0000 7fa224c75640 1 --2- 172.21.15.67:0/1600054321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa220109820 0x7fa220110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.224+0000 7fa224c75640 1 --2- 172.21.15.67:0/1600054321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa220108ed0 0x7fa2201092e0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.224+0000 7fa224c75640 1 --2- 172.21.15.67:0/1600054321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa220103590 0x7fa2201039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.224+0000 7fa224c75640 1 -- 172.21.15.67:0/1600054321 >> 172.21.15.67:0/1600054321 conn(0x7fa22007b280 msgr2=0x7fa22007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:26.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.224+0000 7fa224c75640 1 -- 172.21.15.67:0/1600054321 shutdown_connections 2024-07-31T15:14:26.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.224+0000 7fa224c75640 1 -- 172.21.15.67:0/1600054321 wait complete. 2024-07-31T15:14:26.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.225+0000 7fa224c75640 1 Processor -- start 2024-07-31T15:14:26.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.225+0000 7fa224c75640 1 -- start start 2024-07-31T15:14:26.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.225+0000 7fa224c75640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa220103590 0x7fa220199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.225+0000 7fa224c75640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa220108ed0 0x7fa22019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.225+0000 7fa224c75640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa220109820 0x7fa22019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.225+0000 7fa224c75640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa220113490 con 0x7fa220109820 2024-07-31T15:14:26.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.225+0000 7fa224c75640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa220113310 con 0x7fa220103590 2024-07-31T15:14:26.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.225+0000 7fa224c75640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa220113610 con 0x7fa220108ed0 2024-07-31T15:14:26.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.225+0000 7fa21ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa220109820 0x7fa22019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.225+0000 7fa21e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa220103590 0x7fa220199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.225+0000 7fa21e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa220103590 0x7fa220199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37106/0 (socket says 172.21.15.67:37106) 2024-07-31T15:14:26.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.225+0000 7fa21ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa220109820 0x7fa22019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56214/0 (socket says 172.21.15.67:56214) 2024-07-31T15:14:26.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.225+0000 7fa21e59c640 1 -- 172.21.15.67:0/413834921 learned_addr learned my addr 172.21.15.67:0/413834921 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:26.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.225+0000 7fa21dd9b640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa220108ed0 0x7fa22019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.226+0000 7fa21e59c640 1 -- 172.21.15.67:0/413834921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa220108ed0 msgr2=0x7fa22019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:26.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.226+0000 7fa21e59c640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa220108ed0 0x7fa22019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.226+0000 7fa21e59c640 1 -- 172.21.15.67:0/413834921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa220109820 msgr2=0x7fa22019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:26.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.226+0000 7fa21e59c640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa220109820 0x7fa22019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.226+0000 7fa21e59c640 1 -- 172.21.15.67:0/413834921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22019fe10 con 0x7fa220103590 2024-07-31T15:14:26.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.226+0000 7fa21ed9d640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa220109820 0x7fa22019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:26.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.226+0000 7fa21dd9b640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa220108ed0 0x7fa22019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:14:26.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.226+0000 7fa21e59c640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa220103590 0x7fa220199f60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa210009d50 tx=0x7fa21000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:26.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.226+0000 7fa2177fe640 1 -- 172.21.15.67:0/413834921 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210003b70 con 0x7fa220103590 2024-07-31T15:14:26.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.226+0000 7fa224c75640 1 -- 172.21.15.67:0/413834921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2201a0100 con 0x7fa220103590 2024-07-31T15:14:26.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.226+0000 7fa2177fe640 1 -- 172.21.15.67:0/413834921 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa210003d10 con 0x7fa220103590 2024-07-31T15:14:26.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.226+0000 7fa2177fe640 1 -- 172.21.15.67:0/413834921 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2100108b0 con 0x7fa220103590 2024-07-31T15:14:26.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.226+0000 7fa224c75640 1 -- 172.21.15.67:0/413834921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2201a6920 con 0x7fa220103590 2024-07-31T15:14:26.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.229+0000 7fa22b1c6740 1 Processor -- start 2024-07-31T15:14:26.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.229+0000 7fa22b1c6740 1 -- start start 2024-07-31T15:14:26.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.229+0000 7fa2177fe640 1 -- 172.21.15.67:0/413834921 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fa210003ec0 con 0x7fa220103590 2024-07-31T15:14:26.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.229+0000 7fa22b1c6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9d63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.229+0000 7fa22b1c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba608490 0x561eba9d68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.229+0000 7fa22b1c6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba610670 0x561eba9d6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.229+0000 7fa22b1c6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561eba8be2f0 con 0x561eba631350 2024-07-31T15:14:26.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.229+0000 7fa22b1c6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561eba94ca80 con 0x561eba610670 2024-07-31T15:14:26.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.229+0000 7fa22b1c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561eba93bf80 con 0x561eba608490 2024-07-31T15:14:26.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.229+0000 7fa21ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9d63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.229+0000 7fa21ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9d63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56218/0 (socket says 172.21.15.67:56218) 2024-07-31T15:14:26.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.229+0000 7fa21ed9d640 1 -- 172.21.15.67:0/3453346546 learned_addr learned my addr 172.21.15.67:0/3453346546 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:26.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.229+0000 7fa21dd9b640 1 --2- 172.21.15.67:0/3453346546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba610670 0x561eba9d6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.230+0000 7fa21e59c640 1 --2- 172.21.15.67:0/3453346546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba608490 0x561eba9d68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.230+0000 7fa21ed9d640 1 -- 172.21.15.67:0/3453346546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba608490 msgr2=0x561eba9d68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:26.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.230+0000 7fa21ed9d640 1 --2- 172.21.15.67:0/3453346546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba608490 0x561eba9d68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.230+0000 7fa21ed9d640 1 -- 172.21.15.67:0/3453346546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba610670 msgr2=0x561eba9d6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:26.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.230+0000 7fa21ed9d640 1 --2- 172.21.15.67:0/3453346546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba610670 0x561eba9d6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.230+0000 7fa21ed9d640 1 -- 172.21.15.67:0/3453346546 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eba69c930 con 0x561eba631350 2024-07-31T15:14:26.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.230+0000 7fa21dd9b640 1 --2- 172.21.15.67:0/3453346546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba610670 0x561eba9d6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:26.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.230+0000 7fa2177fe640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa20c0b3d70 0x7fa20c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.230+0000 7fa21ed9d640 1 --2- 172.21.15.67:0/3453346546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9d63b0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fa21800af90 tx=0x7fa21800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893255cfea789666 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:26.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.230+0000 7fa21e59c640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa20c0b3d70 0x7fa20c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.230+0000 7fa2177fe640 1 -- 172.21.15.67:0/413834921 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7fa2100e4af0 con 0x7fa220103590 2024-07-31T15:14:26.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.230+0000 7fa214ff9640 1 -- 172.21.15.67:0/3453346546 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218005220 con 0x561eba631350 2024-07-31T15:14:26.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.230+0000 7fa214ff9640 1 -- 172.21.15.67:0/3453346546 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2180047a0 con 0x561eba631350 2024-07-31T15:14:26.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.230+0000 7fa214ff9640 1 -- 172.21.15.67:0/3453346546 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218010aa0 con 0x561eba631350 2024-07-31T15:14:26.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.231+0000 7fa22b1c6740 1 -- 172.21.15.67:0/3453346546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 msgr2=0x561eba9d63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:26.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.231+0000 7fa21e59c640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa20c0b3d70 0x7fa20c0b6280 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa2100e1f80 tx=0x7fa2100e8640 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:26.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.231+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/3453346546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9d63b0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fa21800af90 tx=0x7fa21800f850 comp rx=0 tx=0).stop 2024-07-31T15:14:26.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.231+0000 7fa22b1c6740 1 -- 172.21.15.67:0/3453346546 shutdown_connections 2024-07-31T15:14:26.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.231+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/3453346546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba610670 0x561eba9d6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.231+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/3453346546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba608490 0x561eba9d68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.231+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/3453346546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9d63b0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.231+0000 7fa22b1c6740 1 -- 172.21.15.67:0/3453346546 >> 172.21.15.67:0/3453346546 conn(0x561eba625740 msgr2=0x561eba9d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:26.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.231+0000 7fa22b1c6740 1 -- 172.21.15.67:0/3453346546 shutdown_connections 2024-07-31T15:14:26.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.231+0000 7fa22b1c6740 1 -- 172.21.15.67:0/3453346546 wait complete. 2024-07-31T15:14:26.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.232+0000 7fa22b1c6740 1 -- start start 2024-07-31T15:14:26.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.232+0000 7fa22b1c6740 1 Processor -- start 2024-07-31T15:14:26.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.232+0000 7fa22b1c6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.232+0000 7fa22b1c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba610670 0x561eba9e23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.232+0000 7fa22b1c6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9edd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.232+0000 7fa22b1c6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561eba8de4f0 con 0x561eba631350 2024-07-31T15:14:26.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.232+0000 7fa22b1c6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561eba93ffc0 con 0x561eba608490 2024-07-31T15:14:26.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.232+0000 7fa22b1c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561eba79f890 con 0x561eba610670 2024-07-31T15:14:26.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.232+0000 7fa22b1c6740 10 client.0 _ll_register_callbacks cb 0x561eba8e8500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:26.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa21ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9edd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa21ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9edd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56230/0 (socket says 172.21.15.67:56230) 2024-07-31T15:14:26.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa21ed9d640 1 -- 172.21.15.67:0/3117609346 learned_addr learned my addr 172.21.15.67:0/3117609346 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:26.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa21ed9d640 1 --2- 172.21.15.67:0/3117609346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa21dd9b640 1 --2- 172.21.15.67:0/3117609346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba610670 0x561eba9e23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa21ed9d640 1 -- 172.21.15.67:0/3117609346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba610670 msgr2=0x561eba9e23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:26.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa21ed9d640 1 --2- 172.21.15.67:0/3117609346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba610670 0x561eba9e23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa21ed9d640 1 -- 172.21.15.67:0/3117609346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 msgr2=0x561eba9e1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:26.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa21ed9d640 1 --2- 172.21.15.67:0/3117609346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9e1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa21ed9d640 1 -- 172.21.15.67:0/3117609346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eba8edc50 con 0x561eba631350 2024-07-31T15:14:26.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa21ed9d640 1 --2- 172.21.15.67:0/3117609346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9edd00 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fa218005e50 tx=0x7fa218010700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e344ba13de5c1a server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:26.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:14:26.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa205ffb640 1 -- 172.21.15.67:0/3117609346 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21802c060 con 0x561eba631350 2024-07-31T15:14:26.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa205ffb640 1 -- 172.21.15.67:0/3117609346 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7fa218004230 con 0x561eba631350 2024-07-31T15:14:26.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.233+0000 7fa22b1c6740 1 -- 172.21.15.67:0/3117609346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2180047a0 con 0x561eba631350 2024-07-31T15:14:26.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.234+0000 7fa205ffb640 1 -- 172.21.15.67:0/3117609346 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fa218033050 con 0x561eba631350 2024-07-31T15:14:26.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.234+0000 7fa22b1c6740 10 client.20457 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:26.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.234+0000 7fa22b1c6740 1 -- 172.21.15.67:0/3117609346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x561eba9f0930 con 0x561eba631350 2024-07-31T15:14:26.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.234+0000 7fa205ffb640 1 -- 172.21.15.67:0/3117609346 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2180047a0 con 0x561eba631350 2024-07-31T15:14:26.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.234+0000 7fa205ffb640 1 -- 172.21.15.67:0/3117609346 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fa218003710 con 0x561eba631350 2024-07-31T15:14:26.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.234+0000 7fa22b1c6740 10 client.20457 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:26.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.234+0000 7fa22b1c6740 10 client.20457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:14:26.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.234+0000 7fa22b1c6740 20 client.20457 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:26.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.234+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/3117609346 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561eba5efcd0 0x561eba9f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.234+0000 7fa22b1c6740 4 client.20457 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:14:26.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.234+0000 7fa22b1c6740 1 -- 172.21.15.67:0/3117609346 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561eba94e9d0 con 0x561eba5efcd0 2024-07-31T15:14:26.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.234+0000 7fa21dd9b640 1 --2- 172.21.15.67:0/3117609346 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561eba5efcd0 0x561eba9f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.235+0000 7fa21dd9b640 1 --2- 172.21.15.67:0/3117609346 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561eba5efcd0 0x561eba9f3da0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5f209be72f71afb server_cookie=b85dd6a4cff94500 in_seq=0 out_seq=0 2024-07-31T15:14:26.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.235+0000 7fa205ffb640 10 client.20457 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:26.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.256+0000 7fa205ffb640 1 -- 172.21.15.67:0/3117609346 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561eba94e9d0 con 0x561eba5efcd0 2024-07-31T15:14:26.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.256+0000 7fa205ffb640 10 client.20457 handle_command_reply: tid=0 2024-07-31T15:14:26.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.256+0000 7fa22b1c6740 1 client.20457 shutdown 2024-07-31T15:14:26.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.256+0000 7fa22b1c6740 1 -- 172.21.15.67:0/3117609346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 msgr2=0x561eba9edd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:26.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.256+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/3117609346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9edd00 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fa218005e50 tx=0x7fa218010700 comp rx=0 tx=0).stop 2024-07-31T15:14:26.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.256+0000 7fa22b1c6740 1 -- 172.21.15.67:0/3117609346 shutdown_connections 2024-07-31T15:14:26.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.256+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/3117609346 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561eba5efcd0 0x561eba9f3da0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.256+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/3117609346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9edd00 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.256+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/3117609346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba610670 0x561eba9e23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.256+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/3117609346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9e1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.256+0000 7fa22b1c6740 1 -- 172.21.15.67:0/3117609346 >> 172.21.15.67:0/3117609346 conn(0x561eba625740 msgr2=0x561eba9d9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:26.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.256+0000 7fa205ffb640 0 client.20457 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:26.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.257+0000 7fa22b1c6740 1 -- 172.21.15.67:0/3117609346 shutdown_connections 2024-07-31T15:14:26.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.257+0000 7fa22b1c6740 1 -- 172.21.15.67:0/3117609346 wait complete. 2024-07-31T15:14:26.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.257+0000 7fa22b1c6740 20 client.20457 trim_cache size 0 max 16384 2024-07-31T15:14:26.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.276+0000 7fa22b1c6740 1 Processor -- start 2024-07-31T15:14:26.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.276+0000 7fa22b1c6740 1 -- start start 2024-07-31T15:14:26.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.276+0000 7fa22b1c6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9d9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.276+0000 7fa22b1c6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba610670 0x561eba9edc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.276+0000 7fa22b1c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba631350 0x561eba9ee1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.276+0000 7fa22b1c6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201b08f0 con 0x561eba610670 2024-07-31T15:14:26.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.276+0000 7fa22b1c6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201b2430 con 0x561eba608490 2024-07-31T15:14:26.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.276+0000 7fa22b1c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201b0fc0 con 0x561eba631350 2024-07-31T15:14:26.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.276+0000 7fa21dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba610670 0x561eba9edc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.276+0000 7fa21dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba610670 0x561eba9edc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56240/0 (socket says 172.21.15.67:56240) 2024-07-31T15:14:26.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.276+0000 7fa21dd9b640 1 -- 172.21.15.67:0/682329103 learned_addr learned my addr 172.21.15.67:0/682329103 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:26.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.276+0000 7fa21ed9d640 1 --2- 172.21.15.67:0/682329103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9d9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.276+0000 7fa21ed9d640 1 --2- 172.21.15.67:0/682329103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba631350 0x561eba9ee1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.277+0000 7fa21dd9b640 1 -- 172.21.15.67:0/682329103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba631350 msgr2=0x561eba9ee1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:26.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.277+0000 7fa21dd9b640 1 --2- 172.21.15.67:0/682329103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba631350 0x561eba9ee1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.277+0000 7fa21dd9b640 1 -- 172.21.15.67:0/682329103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 msgr2=0x561eba9d9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:26.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.277+0000 7fa21dd9b640 1 --2- 172.21.15.67:0/682329103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9d9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.277+0000 7fa21dd9b640 1 -- 172.21.15.67:0/682329103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eba9d9210 con 0x561eba610670 2024-07-31T15:14:26.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.277+0000 7fa21ed9d640 1 --2- 172.21.15.67:0/682329103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9d9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:26.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.277+0000 7fa21dd9b640 1 --2- 172.21.15.67:0/682329103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba610670 0x561eba9edc70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fa208009230 tx=0x7fa208005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1daa6f72ca75b9c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:26.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.277+0000 7fa207fff640 1 -- 172.21.15.67:0/682329103 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208049070 con 0x561eba610670 2024-07-31T15:14:26.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.277+0000 7fa207fff640 1 -- 172.21.15.67:0/682329103 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa208004450 con 0x561eba610670 2024-07-31T15:14:26.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.277+0000 7fa207fff640 1 -- 172.21.15.67:0/682329103 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208035040 con 0x561eba610670 2024-07-31T15:14:26.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.278+0000 7fa22b1c6740 1 -- 172.21.15.67:0/682329103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba610670 msgr2=0x561eba9edc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:26.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.278+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/682329103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba610670 0x561eba9edc70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fa208009230 tx=0x7fa208005a20 comp rx=0 tx=0).stop 2024-07-31T15:14:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.278+0000 7fa22b1c6740 1 -- 172.21.15.67:0/682329103 shutdown_connections 2024-07-31T15:14:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.278+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/682329103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba631350 0x561eba9ee1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.278+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/682329103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba610670 0x561eba9edc70 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.278+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/682329103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9d9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.278+0000 7fa22b1c6740 1 -- 172.21.15.67:0/682329103 >> 172.21.15.67:0/682329103 conn(0x561eba625740 msgr2=0x561eba9d4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.278+0000 7fa22b1c6740 1 -- 172.21.15.67:0/682329103 shutdown_connections 2024-07-31T15:14:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.278+0000 7fa22b1c6740 1 -- 172.21.15.67:0/682329103 wait complete. 2024-07-31T15:14:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.278+0000 7fa22b1c6740 1 -- start start 2024-07-31T15:14:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.279+0000 7fa22b1c6740 1 Processor -- start 2024-07-31T15:14:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.279+0000 7fa22b1c6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9f7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.279+0000 7fa22b1c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba610670 0x561eba9f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.279+0000 7fa22b1c6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.279+0000 7fa22b1c6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201b1d60 con 0x561eba631350 2024-07-31T15:14:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.279+0000 7fa22b1c6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201b1690 con 0x561eba608490 2024-07-31T15:14:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.279+0000 7fa22b1c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201ae8f0 con 0x561eba610670 2024-07-31T15:14:26.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.279+0000 7fa22b1c6740 10 client.0 _ll_register_callbacks cb 0x7fa218010be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:26.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.279+0000 7fa21ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.279+0000 7fa21ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.279+0000 7fa21ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9f7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37132/0 (socket says 172.21.15.67:37132) 2024-07-31T15:14:26.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.279+0000 7fa21ed9d640 1 -- 172.21.15.67:0/2012914597 learned_addr learned my addr 172.21.15.67:0/2012914597 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:26.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.279+0000 7fa21dd9b640 1 --2- 172.21.15.67:0/2012914597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba610670 0x561eba9f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.280+0000 7fa21ed9d640 1 -- 172.21.15.67:0/2012914597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba610670 msgr2=0x561eba9f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:26.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.280+0000 7fa21ed9d640 1 --2- 172.21.15.67:0/2012914597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba610670 0x561eba9f47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.280+0000 7fa21ed9d640 1 -- 172.21.15.67:0/2012914597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 msgr2=0x561eba9f4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.280+0000 7fa21ed9d640 1 --2- 172.21.15.67:0/2012914597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9f4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.280+0000 7fa21ed9d640 1 -- 172.21.15.67:0/2012914597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eba9f5300 con 0x561eba608490 2024-07-31T15:14:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.280+0000 7fa21ed9d640 1 --2- 172.21.15.67:0/2012914597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9f7300 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa218005b70 tx=0x7fa21800f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a86cab214afe147d server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.280+0000 7fa205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:14:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.280+0000 7fa205ffb640 1 -- 172.21.15.67:0/2012914597 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21802c070 con 0x561eba608490 2024-07-31T15:14:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.280+0000 7fa22b1c6740 1 -- 172.21.15.67:0/2012914597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa208004450 con 0x561eba608490 2024-07-31T15:14:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.280+0000 7fa205ffb640 1 -- 172.21.15.67:0/2012914597 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7fa218004a50 con 0x561eba608490 2024-07-31T15:14:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.280+0000 7fa205ffb640 1 -- 172.21.15.67:0/2012914597 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fa218004c50 con 0x561eba608490 2024-07-31T15:14:26.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.280+0000 7fa22b1c6740 10 client.29969 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:26.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.280+0000 7fa22b1c6740 1 -- 172.21.15.67:0/2012914597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x561ebaa5c0d0 con 0x561eba608490 2024-07-31T15:14:26.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.280+0000 7fa205ffb640 1 -- 172.21.15.67:0/2012914597 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa208004450 con 0x561eba608490 2024-07-31T15:14:26.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.281+0000 7fa205ffb640 1 -- 172.21.15.67:0/2012914597 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fa218044070 con 0x561eba608490 2024-07-31T15:14:26.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.281+0000 7fa22b1c6740 10 client.29969 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:26.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.281+0000 7fa22b1c6740 10 client.29969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:14:26.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.281+0000 7fa22b1c6740 20 client.29969 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:26.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.281+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/2012914597 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561eba5c5080 0x561ebaa5e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:26.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.281+0000 7fa22b1c6740 4 client.29969 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-31T15:14:26.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.281+0000 7fa22b1c6740 1 -- 172.21.15.67:0/2012914597 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561eba94a470 con 0x561eba5c5080 2024-07-31T15:14:26.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.281+0000 7fa21dd9b640 1 --2- 172.21.15.67:0/2012914597 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561eba5c5080 0x561ebaa5e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:26.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.281+0000 7fa21dd9b640 1 --2- 172.21.15.67:0/2012914597 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561eba5c5080 0x561ebaa5e650 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2538060b6d4a4b4d server_cookie=f1e27158cb7918fa in_seq=0 out_seq=0 2024-07-31T15:14:26.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.281+0000 7fa205ffb640 10 client.29969 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:26.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa205ffb640 1 -- 172.21.15.67:0/2012914597 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561eba94a470 con 0x561eba5c5080 2024-07-31T15:14:26.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa205ffb640 10 client.29969 handle_command_reply: tid=0 2024-07-31T15:14:26.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa22b1c6740 1 client.29969 shutdown 2024-07-31T15:14:26.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa22b1c6740 1 -- 172.21.15.67:0/2012914597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 msgr2=0x561eba9f7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:26.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/2012914597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9f7300 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa218005b70 tx=0x7fa21800f850 comp rx=0 tx=0).stop 2024-07-31T15:14:26.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa22b1c6740 1 -- 172.21.15.67:0/2012914597 shutdown_connections 2024-07-31T15:14:26.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/2012914597 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x561eba5c5080 0x561ebaa5e650 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/2012914597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eba631350 0x561eba9f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/2012914597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eba610670 0x561eba9f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa22b1c6740 1 --2- 172.21.15.67:0/2012914597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561eba608490 0x561eba9f7300 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa22b1c6740 1 -- 172.21.15.67:0/2012914597 >> 172.21.15.67:0/2012914597 conn(0x561eba625740 msgr2=0x561eba9d93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:26.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa205ffb640 0 client.29969 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:26.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa22b1c6740 1 -- 172.21.15.67:0/2012914597 shutdown_connections 2024-07-31T15:14:26.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa22b1c6740 1 -- 172.21.15.67:0/2012914597 wait complete. 2024-07-31T15:14:26.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.289+0000 7fa22b1c6740 20 client.29969 trim_cache size 0 max 16384 2024-07-31T15:14:26.293 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:14:26.294 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-31T15:14:26.294 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "e297981a-9036-463f-957e-e32c8e80d535", 2024-07-31T15:14:26.294 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-31T15:14:26.294 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:14:26.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.290+0000 7fa224c75640 1 -- 172.21.15.67:0/413834921 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa20c0b3d70 msgr2=0x7fa20c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:26.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.290+0000 7fa224c75640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa20c0b3d70 0x7fa20c0b6280 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa2100e1f80 tx=0x7fa2100e8640 comp rx=0 tx=0).stop 2024-07-31T15:14:26.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.290+0000 7fa224c75640 1 -- 172.21.15.67:0/413834921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa220103590 msgr2=0x7fa220199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:26.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.290+0000 7fa224c75640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa220103590 0x7fa220199f60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa210009d50 tx=0x7fa21000c6a0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.290+0000 7fa224c75640 1 -- 172.21.15.67:0/413834921 shutdown_connections 2024-07-31T15:14:26.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.290+0000 7fa224c75640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa20c0b3d70 0x7fa20c0b6280 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.290+0000 7fa224c75640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa220109820 0x7fa22019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.290+0000 7fa224c75640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa220108ed0 0x7fa22019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.290+0000 7fa224c75640 1 --2- 172.21.15.67:0/413834921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa220103590 0x7fa220199f60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:26.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.290+0000 7fa224c75640 1 -- 172.21.15.67:0/413834921 >> 172.21.15.67:0/413834921 conn(0x7fa22007b280 msgr2=0x7fa220107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:26.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.290+0000 7fa224c75640 1 -- 172.21.15.67:0/413834921 shutdown_connections 2024-07-31T15:14:26.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:26.290+0000 7fa224c75640 1 -- 172.21.15.67:0/413834921 wait complete. 2024-07-31T15:14:27.015 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:14:27.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:26 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:26.641+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000162a [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch/ auth{1=1,2=1,3=1,4=1} v21244 ap=3 RANDEPHEMERALPIN f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22) n(v53 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939) old_inodes=7 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:649531 nref=4)) caps={15489=pAsLsXs/-@71} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7c26ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000162a:[<0x10000000202/arch v378>,<0x10000000201/linux-6.5.11 v96>,<0x10000000006/t v68>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v57>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v60>,<0x10000000001/sv_1 v84>,<0x10000000000/qa v70>,<0x1/volumes v70>]//[]","memoryvalue":"(2)0x1000000162a:[<0x10000000202/arch v21244>,<0x10000000201/linux-6.5.11 v3098>,<0x10000000006/t v1906>,<0x10000000005/tmp v1827>,<0x100000001fb/client.0 v1793>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1756>,<0x10000000001/sv_1 v1744>,<0x10000000000/qa v1738>,<0x1/volumes v1677>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:54:49.652308+0000 23=2+21)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939)","memory_value.dirstat":"f(v4 m2024-07-31T14:54:53.297232+0000 24=2+22)","memory_value.rstat":"n(v53 rc2024-07-31T15:07:51.036222+0000 b124347620 18694=17755+939)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:27.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:26 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:26.661+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c28d [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/9p/ auth{3=1} v15462 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:10:57.696418+0000 31=31+0) n(v3 rc2024-07-31T15:10:57.699418+0000 b700067 32=31+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:650169 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@13} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8a0eb700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c28d:[<0x1000000c25e/9p v120>,<0x10000000202/net v7765>,<0x10000000201/linux-6.5.11 v689>,<0x10000000006/t v498>,<0x10000000005/tmp v466>,<0x100000001fb/client.0 v452>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v438>,<0x10000000001/sv_1 v453>,<0x10000000000/qa v441>,<0x1/volumes v433>]//[]","memoryvalue":"(2)0x1000000c28d:[<0x1000000c25e/9p v15462>,<0x10000000202/net v22488>,<0x10000000201/linux-6.5.11 v3098>,<0x10000000006/t v1906>,<0x10000000005/tmp v1827>,<0x100000001fb/client.0 v1793>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1756>,<0x10000000001/sv_1 v1744>,<0x10000000000/qa v1738>,<0x1/volumes v1677>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:49.314316+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:10:57.699418+0000 b700067 32=31+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:10:57.696418+0000 31=31+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:10:57.699418+0000 b700067 32=31+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:27.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:26 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:26.742+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c276 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802/ auth{3=1} v10568 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:29.002671+0000 14=14+0) n(v3 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:650203 nref=3)) caps={15489=pAsLsXs/-@18} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8fd23180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c276:[<0x1000000c25e/802 v68>,<0x10000000202/net v7765>,<0x10000000201/linux-6.5.11 v689>,<0x10000000006/t v498>,<0x10000000005/tmp v466>,<0x100000001fb/client.0 v452>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v438>,<0x10000000001/sv_1 v453>,<0x10000000000/qa v441>,<0x1/volumes v433>]//[]","memoryvalue":"(2)0x1000000c276:[<0x1000000c25e/802 v10568>,<0x10000000202/net v22488>,<0x10000000201/linux-6.5.11 v3098>,<0x10000000006/t v1906>,<0x10000000005/tmp v1827>,<0x100000001fb/client.0 v1793>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1756>,<0x10000000001/sv_1 v1744>,<0x10000000000/qa v1738>,<0x1/volumes v1677>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:49.293317+0000 10=10+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:29.002671+0000 14=14+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:29.003671+0000 b64549 15=14+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:27.144 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:26 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:26.770+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c9e2 [...1f,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix/ auth{2=1} v12135 ap=3 RANDEPHEMERALPIN f(v2 m2024-07-31T15:08:53.494950+0000 21=21+0) n(v5 rc2024-07-31T15:08:53.495950+0000 b445955 22=21+1) old_inodes=3 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:650219 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@21} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8591ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c9e2:[<0x1000000c25e/unix v6743>,<0x10000000202/net v8736>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000c9e2:[<0x1000000c25e/unix v12135>,<0x10000000202/net v22488>,<0x10000000201/linux-6.5.11 v3098>,<0x10000000006/t v1906>,<0x10000000005/tmp v1827>,<0x100000001fb/client.0 v1793>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1756>,<0x10000000001/sv_1 v1744>,<0x10000000000/qa v1738>,<0x1/volumes v1677>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:08:21.655599+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:53.495950+0000 b445955 22=21+1)","memory_value.dirstat":"f(v2 m2024-07-31T15:08:53.494950+0000 21=21+0)","memory_value.rstat":"n(v5 rc2024-07-31T15:08:53.495950+0000 b445955 22=21+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:27.144 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:26 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:26.771+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c836 [...1f,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet/ auth{4=1} v12171 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:57.081877+0000 11=11+0) n(v2 rc2024-07-31T15:08:57.083877+0000 b263808 12=11+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:650217 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@18} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8c20b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c836:[<0x1000000c25e/packet v6437>,<0x10000000202/net v8736>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000c836:[<0x1000000c25e/packet v12171>,<0x10000000202/net v22488>,<0x10000000201/linux-6.5.11 v3098>,<0x10000000006/t v1906>,<0x10000000005/tmp v1827>,<0x100000001fb/client.0 v1793>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1756>,<0x10000000001/sv_1 v1744>,<0x10000000000/qa v1738>,<0x1/volumes v1677>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:56.791162+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:57.083877+0000 b263808 12=11+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:57.081877+0000 11=11+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:08:57.083877+0000 b263808 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:27.144 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:26 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:26.776+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c7cb [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/netlabel/ auth{4=1} v15466 ap=3 RANDEPHEMERALPIN f(v2 m2024-07-31T15:10:46.034656+0000 37=37+0) n(v4 rc2024-07-31T15:10:46.036656+0000 b816547 38=37+1) old_inodes=3 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:650218 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@17} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce84a58580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c7cb:[<0x1000000c25e/netlabel v6433>,<0x10000000202/net v8736>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000c7cb:[<0x1000000c25e/netlabel v15466>,<0x10000000202/net v22488>,<0x10000000201/linux-6.5.11 v3098>,<0x10000000006/t v1906>,<0x10000000005/tmp v1827>,<0x100000001fb/client.0 v1793>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1756>,<0x10000000001/sv_1 v1744>,<0x10000000000/qa v1738>,<0x1/volumes v1677>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:10:43.762703+0000 29=29+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:10:46.036656+0000 b816547 38=37+1)","memory_value.dirstat":"f(v2 m2024-07-31T15:10:46.034656+0000 37=37+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:10:46.036656+0000 b816547 38=37+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:27.144 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:26 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:26.816+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d9b4 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb/ auth{3=1} v3876 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7) n(v5 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8) old_inodes=3 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:649568 nref=3)) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce92cb2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v2020>,<0x10000000202/sound v9936>,<0x10000000201/linux-6.5.11 v830>,<0x10000000006/t v586>,<0x10000000005/tmp v553>,<0x100000001fb/client.0 v537>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v519>,<0x10000000001/sv_1 v525>,<0x10000000000/qa v522>,<0x1/volumes v503>]//[]","memoryvalue":"(2)0x1000000d9b4:[<0x1000000cf1b/usb v3876>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v3098>,<0x10000000006/t v1906>,<0x10000000005/tmp v1827>,<0x100000001fb/client.0 v1793>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1756>,<0x10000000001/sv_1 v1744>,<0x10000000000/qa v1738>,<0x1/volumes v1677>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:46.317139+0000 53=46+7)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:52.062646+0000 57=50+7)","memory_value.rstat":"n(v5 rc2024-07-31T15:05:52.063646+0000 b1486381 148=140+8)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:27.394 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:14:27.568 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: pgmap v1147: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.3 MiB/s wr, 15 op/s 2024-07-31T15:14:27.569 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub queued for path: / 2024-07-31T15:14:27.569 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.569 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:27.569 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:27.569 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:27.569 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:27.570 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.570 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:14:27.570 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.570 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c28d (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/9p) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.570 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:27.570 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.570 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.571 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:27.571 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c9e2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.571 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c836 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.571 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.571 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c7cb (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/netlabel) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.571 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:27.571 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.571 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:27.572 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.572 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.572 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.572 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:14:27.572 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:14:27.572 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:14:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: pgmap v1147: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.3 MiB/s wr, 15 op/s 2024-07-31T15:14:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub queued for path: / 2024-07-31T15:14:27.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:14:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c28d (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/9p) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c9e2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c836 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c7cb (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/netlabel) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:14:27.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:14:27.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:14:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: pgmap v1147: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.3 MiB/s wr, 15 op/s 2024-07-31T15:14:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub queued for path: / 2024-07-31T15:14:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:14:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c28d (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/9p) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c276 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/802) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c9e2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/unix) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c836 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/packet) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c7cb (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/net/netlabel) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:27.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:27.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d9b4 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/usb) see mds.b log and `damage ls` output for details 2024-07-31T15:14:27.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:27.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:14:27.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:14:27.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:14:28.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:28.561 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:28 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:28.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:28 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:28.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:28.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:28 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.277+0000 7fc50039f640 1 -- 172.21.15.67:0/49922185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f8107e10 msgr2=0x7fc4f8108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.277+0000 7fc50039f640 1 --2- 172.21.15.67:0/49922185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f8107e10 0x7fc4f8108220 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec009900 tx=0x7fc4ec02a410 comp rx=0 tx=0).stop 2024-07-31T15:14:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.278+0000 7fc50039f640 1 -- 172.21.15.67:0/49922185 shutdown_connections 2024-07-31T15:14:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.278+0000 7fc50039f640 1 --2- 172.21.15.67:0/49922185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f8108760 0x7fc4f810ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.278+0000 7fc50039f640 1 --2- 172.21.15.67:0/49922185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f8107e10 0x7fc4f8108220 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.278+0000 7fc50039f640 1 --2- 172.21.15.67:0/49922185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f81024d0 0x7fc4f81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.278+0000 7fc50039f640 1 -- 172.21.15.67:0/49922185 >> 172.21.15.67:0/49922185 conn(0x7fc4f8079180 msgr2=0x7fc4f8079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.278+0000 7fc50039f640 1 -- 172.21.15.67:0/49922185 shutdown_connections 2024-07-31T15:14:29.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.278+0000 7fc50039f640 1 -- 172.21.15.67:0/49922185 wait complete. 2024-07-31T15:14:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.278+0000 7fc50039f640 1 Processor -- start 2024-07-31T15:14:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.278+0000 7fc50039f640 1 -- start start 2024-07-31T15:14:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc50039f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f81024d0 0x7fc4f8198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc50039f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f8107e10 0x7fc4f81993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc50039f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f8108760 0x7fc4f819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc50039f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f8112390 con 0x7fc4f8108760 2024-07-31T15:14:29.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc50039f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f8112210 con 0x7fc4f81024d0 2024-07-31T15:14:29.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc50039f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f8112510 con 0x7fc4f8107e10 2024-07-31T15:14:29.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc4fe13b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f81024d0 0x7fc4f8198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc4fe13b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f81024d0 0x7fc4f8198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37148/0 (socket says 172.21.15.67:37148) 2024-07-31T15:14:29.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc4fe13b640 1 -- 172.21.15.67:0/2417104165 learned_addr learned my addr 172.21.15.67:0/2417104165 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:29.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc4fd93a640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f8107e10 0x7fc4f81993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc4fe93c640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f8108760 0x7fc4f819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc4fe93c640 1 -- 172.21.15.67:0/2417104165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f8107e10 msgr2=0x7fc4f81993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:29.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc4fe93c640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f8107e10 0x7fc4f81993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc4fe93c640 1 -- 172.21.15.67:0/2417104165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f81024d0 msgr2=0x7fc4f8198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:29.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc4fe93c640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f81024d0 0x7fc4f8198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc4fe93c640 1 -- 172.21.15.67:0/2417104165 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f819ed70 con 0x7fc4f8108760 2024-07-31T15:14:29.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.279+0000 7fc4fe13b640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f81024d0 0x7fc4f8198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:29.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.280+0000 7fc4fe93c640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f8108760 0x7fc4f819e640 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009fd0 tx=0x7fc4e800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:29.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.280+0000 7fc4f77fe640 1 -- 172.21.15.67:0/2417104165 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e800ce20 con 0x7fc4f8108760 2024-07-31T15:14:29.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.280+0000 7fc4f77fe640 1 -- 172.21.15.67:0/2417104165 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4e8005020 con 0x7fc4f8108760 2024-07-31T15:14:29.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.280+0000 7fc4f77fe640 1 -- 172.21.15.67:0/2417104165 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8010690 con 0x7fc4f8108760 2024-07-31T15:14:29.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.280+0000 7fc50039f640 1 -- 172.21.15.67:0/2417104165 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f819f060 con 0x7fc4f8108760 2024-07-31T15:14:29.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.280+0000 7fc50039f640 1 -- 172.21.15.67:0/2417104165 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f806b5b0 con 0x7fc4f8108760 2024-07-31T15:14:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.283+0000 7fc5068f0740 1 Processor -- start 2024-07-31T15:14:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.283+0000 7fc4f77fe640 1 -- 172.21.15.67:0/2417104165 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc4e80054d0 con 0x7fc4f8108760 2024-07-31T15:14:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.283+0000 7fc5068f0740 1 -- start start 2024-07-31T15:14:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.283+0000 7fc5068f0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41eb3070 0x555c42258640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.283+0000 7fc5068f0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41e8a1b0 0x555c42258b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.283+0000 7fc5068f0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e92390 0x555c422590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.283+0000 7fc5068f0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555c421ceb60 con 0x555c41e8a1b0 2024-07-31T15:14:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.283+0000 7fc5068f0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555c421be060 con 0x555c41eb3070 2024-07-31T15:14:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.283+0000 7fc5068f0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555c421c20a0 con 0x555c41e92390 2024-07-31T15:14:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.284+0000 7fc4fe13b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41e8a1b0 0x555c42258b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.284+0000 7fc4fe13b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41e8a1b0 0x555c42258b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56260/0 (socket says 172.21.15.67:56260) 2024-07-31T15:14:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.284+0000 7fc4fd93a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41eb3070 0x555c42258640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.284+0000 7fc4fe13b640 1 -- 172.21.15.67:0/9063741 learned_addr learned my addr 172.21.15.67:0/9063741 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:29.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.284+0000 7fc4fd93a640 1 --2- 172.21.15.67:0/9063741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e92390 0x555c422590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.284+0000 7fc4fe13b640 1 -- 172.21.15.67:0/9063741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e92390 msgr2=0x555c422590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:29.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.284+0000 7fc4fe13b640 1 --2- 172.21.15.67:0/9063741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e92390 0x555c422590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.284+0000 7fc4fe13b640 1 -- 172.21.15.67:0/9063741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41eb3070 msgr2=0x555c42258640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:29.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.284+0000 7fc4fe13b640 1 --2- 172.21.15.67:0/9063741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41eb3070 0x555c42258640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.284+0000 7fc4fe13b640 1 -- 172.21.15.67:0/9063741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c42164b40 con 0x555c41e8a1b0 2024-07-31T15:14:29.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.284+0000 7fc4fd93a640 1 --2- 172.21.15.67:0/9063741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e92390 0x555c422590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:14:29.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.284+0000 7fc4fe13b640 1 --2- 172.21.15.67:0/9063741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41e8a1b0 0x555c42258b80 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0010580 tx=0x7fc4f0010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf8b7f2c60a08ce server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:29.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.284+0000 7fc4f77fe640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc4e00b3d40 0x7fc4e00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.284+0000 7fc4f4d99640 1 -- 172.21.15.67:0/9063741 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000e890 con 0x555c41e8a1b0 2024-07-31T15:14:29.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.285+0000 7fc4f4d99640 1 -- 172.21.15.67:0/9063741 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f001c480 con 0x555c41e8a1b0 2024-07-31T15:14:29.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.285+0000 7fc4f4d99640 1 -- 172.21.15.67:0/9063741 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f00116a0 con 0x555c41e8a1b0 2024-07-31T15:14:29.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.285+0000 7fc4fd93a640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc4e00b3d40 0x7fc4e00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.285+0000 7fc4f77fe640 1 -- 172.21.15.67:0/2417104165 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7fc4e80e4a10 con 0x7fc4f8108760 2024-07-31T15:14:29.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.285+0000 7fc5068f0740 1 -- 172.21.15.67:0/9063741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41e8a1b0 msgr2=0x555c42258b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:29.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.285+0000 7fc4fd93a640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc4e00b3d40 0x7fc4e00b6250 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec000c00 tx=0x7fc4ec005780 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:29.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.285+0000 7fc5068f0740 1 --2- 172.21.15.67:0/9063741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41e8a1b0 0x555c42258b80 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0010580 tx=0x7fc4f0010a40 comp rx=0 tx=0).stop 2024-07-31T15:14:29.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.285+0000 7fc5068f0740 1 -- 172.21.15.67:0/9063741 shutdown_connections 2024-07-31T15:14:29.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.285+0000 7fc5068f0740 1 --2- 172.21.15.67:0/9063741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e92390 0x555c422590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.286+0000 7fc5068f0740 1 --2- 172.21.15.67:0/9063741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41e8a1b0 0x555c42258b80 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.286+0000 7fc5068f0740 1 --2- 172.21.15.67:0/9063741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41eb3070 0x555c42258640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.286+0000 7fc5068f0740 1 -- 172.21.15.67:0/9063741 >> 172.21.15.67:0/9063741 conn(0x555c41ea7460 msgr2=0x555c42256530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:29.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.286+0000 7fc5068f0740 1 -- 172.21.15.67:0/9063741 shutdown_connections 2024-07-31T15:14:29.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.286+0000 7fc5068f0740 1 -- 172.21.15.67:0/9063741 wait complete. 2024-07-31T15:14:29.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.286+0000 7fc5068f0740 1 -- start start 2024-07-31T15:14:29.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.286+0000 7fc5068f0740 1 Processor -- start 2024-07-31T15:14:29.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc5068f0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e8a1b0 0x555c42264260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc5068f0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e92390 0x555c422647a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc5068f0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41eb3070 0x555c42270140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc5068f0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x555c42021700 con 0x555c41eb3070 2024-07-31T15:14:29.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc5068f0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555c42160990 con 0x555c41e92390 2024-07-31T15:14:29.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc5068f0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555c42266e20 con 0x555c41e8a1b0 2024-07-31T15:14:29.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc5068f0740 10 client.0 _ll_register_callbacks cb 0x555c4217b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc4fe93c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41eb3070 0x555c42270140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc4fd93a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e92390 0x555c422647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc4fe93c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41eb3070 0x555c42270140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56270/0 (socket says 172.21.15.67:56270) 2024-07-31T15:14:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc4fe93c640 1 -- 172.21.15.67:0/2444157302 learned_addr learned my addr 172.21.15.67:0/2444157302 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc4fe13b640 1 --2- 172.21.15.67:0/2444157302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e8a1b0 0x555c42264260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc4fe93c640 1 -- 172.21.15.67:0/2444157302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e8a1b0 msgr2=0x555c42264260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:29.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc4fe93c640 1 --2- 172.21.15.67:0/2444157302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e8a1b0 0x555c42264260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc4fe93c640 1 -- 172.21.15.67:0/2444157302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e92390 msgr2=0x555c422647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:29.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc4fe93c640 1 --2- 172.21.15.67:0/2444157302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e92390 0x555c422647a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc4fe93c640 1 -- 172.21.15.67:0/2444157302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c41f1e650 con 0x555c41eb3070 2024-07-31T15:14:29.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc4fe93c640 1 --2- 172.21.15.67:0/2444157302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41eb3070 0x555c42270140 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80aafb0 tx=0x7fc4e80aba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d62d413b1190095 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:29.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.287+0000 7fc4ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:14:29.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc4ddffb640 1 -- 172.21.15.67:0/2444157302 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80b4360 con 0x555c41eb3070 2024-07-31T15:14:29.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc4ddffb640 1 -- 172.21.15.67:0/2444157302 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7fc4e80b3ed0 con 0x555c41eb3070 2024-07-31T15:14:29.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc5068f0740 1 -- 172.21.15.67:0/2444157302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f001c480 con 0x555c41eb3070 2024-07-31T15:14:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc4ddffb640 1 -- 172.21.15.67:0/2444157302 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fc4e80b4110 con 0x555c41eb3070 2024-07-31T15:14:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc5068f0740 10 client.20475 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc5068f0740 1 -- 172.21.15.67:0/2444157302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x555c42272ed0 con 0x555c41eb3070 2024-07-31T15:14:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc4ddffb640 1 -- 172.21.15.67:0/2444157302 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f001c480 con 0x555c41eb3070 2024-07-31T15:14:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc4ddffb640 1 -- 172.21.15.67:0/2444157302 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fc4e80aa010 con 0x555c41eb3070 2024-07-31T15:14:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc5068f0740 10 client.20475 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc5068f0740 10 client.20475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:14:29.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc5068f0740 20 client.20475 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:29.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc5068f0740 1 --2- 172.21.15.67:0/2444157302 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c41e719f0 0x555c42276410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc5068f0740 4 client.20475 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:14:29.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc5068f0740 1 -- 172.21.15.67:0/2444157302 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c421d0ab0 con 0x555c41e719f0 2024-07-31T15:14:29.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.288+0000 7fc4fe13b640 1 --2- 172.21.15.67:0/2444157302 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c41e719f0 0x555c42276410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.289+0000 7fc4fe13b640 1 --2- 172.21.15.67:0/2444157302 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c41e719f0 0x555c42276410 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef5631f84aa75d9 server_cookie=5e7004801eb0761d in_seq=0 out_seq=0 2024-07-31T15:14:29.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.289+0000 7fc4ddffb640 10 client.20475 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc4ddffb640 1 -- 172.21.15.67:0/2444157302 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c421d0ab0 con 0x555c41e719f0 2024-07-31T15:14:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc4ddffb640 10 client.20475 handle_command_reply: tid=0 2024-07-31T15:14:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc5068f0740 1 client.20475 shutdown 2024-07-31T15:14:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc5068f0740 1 -- 172.21.15.67:0/2444157302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41eb3070 msgr2=0x555c42270140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc5068f0740 1 --2- 172.21.15.67:0/2444157302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41eb3070 0x555c42270140 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80aafb0 tx=0x7fc4e80aba70 comp rx=0 tx=0).stop 2024-07-31T15:14:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc5068f0740 1 -- 172.21.15.67:0/2444157302 shutdown_connections 2024-07-31T15:14:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc5068f0740 1 --2- 172.21.15.67:0/2444157302 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c41e719f0 0x555c42276410 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc5068f0740 1 --2- 172.21.15.67:0/2444157302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41eb3070 0x555c42270140 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc5068f0740 1 --2- 172.21.15.67:0/2444157302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e92390 0x555c422647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc5068f0740 1 --2- 172.21.15.67:0/2444157302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e8a1b0 0x555c42264260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc5068f0740 1 -- 172.21.15.67:0/2444157302 >> 172.21.15.67:0/2444157302 conn(0x555c41ea7460 msgr2=0x555c42254cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:29.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc4ddffb640 0 client.20475 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:29.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc5068f0740 1 -- 172.21.15.67:0/2444157302 shutdown_connections 2024-07-31T15:14:29.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc5068f0740 1 -- 172.21.15.67:0/2444157302 wait complete. 2024-07-31T15:14:29.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.315+0000 7fc5068f0740 20 client.20475 trim_cache size 0 max 16384 2024-07-31T15:14:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.334+0000 7fc5068f0740 1 Processor -- start 2024-07-31T15:14:29.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.334+0000 7fc5068f0740 1 -- start start 2024-07-31T15:14:29.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.334+0000 7fc5068f0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e8a1b0 0x555c42256560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.334+0000 7fc5068f0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e92390 0x555c4226ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.334+0000 7fc5068f0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41eb3070 0x555c4227bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.334+0000 7fc5068f0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f81ae510 con 0x555c41eb3070 2024-07-31T15:14:29.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.334+0000 7fc5068f0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f81b0050 con 0x555c41e8a1b0 2024-07-31T15:14:29.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.334+0000 7fc5068f0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f81aebe0 con 0x555c41e92390 2024-07-31T15:14:29.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.334+0000 7fc4fe93c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41eb3070 0x555c4227bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.334+0000 7fc4fe13b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e8a1b0 0x555c42256560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.334+0000 7fc4fe93c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41eb3070 0x555c4227bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56284/0 (socket says 172.21.15.67:56284) 2024-07-31T15:14:29.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.334+0000 7fc4fe93c640 1 -- 172.21.15.67:0/3989917949 learned_addr learned my addr 172.21.15.67:0/3989917949 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:29.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.334+0000 7fc4fd93a640 1 --2- 172.21.15.67:0/3989917949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e92390 0x555c4226ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.335+0000 7fc4fe13b640 1 -- 172.21.15.67:0/3989917949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e92390 msgr2=0x555c4226ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:29.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.335+0000 7fc4fe13b640 1 --2- 172.21.15.67:0/3989917949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e92390 0x555c4226ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.335+0000 7fc4fe13b640 1 -- 172.21.15.67:0/3989917949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41eb3070 msgr2=0x555c4227bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:29.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.335+0000 7fc4fe13b640 1 --2- 172.21.15.67:0/3989917949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41eb3070 0x555c4227bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.335+0000 7fc4fe13b640 1 -- 172.21.15.67:0/3989917949 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c42265320 con 0x555c41e8a1b0 2024-07-31T15:14:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.335+0000 7fc4fe13b640 1 --2- 172.21.15.67:0/3989917949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e8a1b0 0x555c42256560 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc4f001cf60 tx=0x7fc4f0013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdc4df991956feb8 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.335+0000 7fc4dffff640 1 -- 172.21.15.67:0/3989917949 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f002b030 con 0x555c41e8a1b0 2024-07-31T15:14:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.335+0000 7fc4dffff640 1 -- 172.21.15.67:0/3989917949 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f0005570 con 0x555c41e8a1b0 2024-07-31T15:14:29.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.335+0000 7fc4dffff640 1 -- 172.21.15.67:0/3989917949 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f001c8e0 con 0x555c41e8a1b0 2024-07-31T15:14:29.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.336+0000 7fc5068f0740 1 -- 172.21.15.67:0/3989917949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e8a1b0 msgr2=0x555c42256560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:29.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.336+0000 7fc5068f0740 1 --2- 172.21.15.67:0/3989917949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e8a1b0 0x555c42256560 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc4f001cf60 tx=0x7fc4f0013040 comp rx=0 tx=0).stop 2024-07-31T15:14:29.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.336+0000 7fc5068f0740 1 -- 172.21.15.67:0/3989917949 shutdown_connections 2024-07-31T15:14:29.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.336+0000 7fc5068f0740 1 --2- 172.21.15.67:0/3989917949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41eb3070 0x555c4227bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.336+0000 7fc5068f0740 1 --2- 172.21.15.67:0/3989917949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41e92390 0x555c4226ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.336+0000 7fc5068f0740 1 --2- 172.21.15.67:0/3989917949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e8a1b0 0x555c42256560 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.336+0000 7fc5068f0740 1 -- 172.21.15.67:0/3989917949 >> 172.21.15.67:0/3989917949 conn(0x555c41ea7460 msgr2=0x555c42255e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:29.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.336+0000 7fc5068f0740 1 -- 172.21.15.67:0/3989917949 shutdown_connections 2024-07-31T15:14:29.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.336+0000 7fc5068f0740 1 -- 172.21.15.67:0/3989917949 wait complete. 2024-07-31T15:14:29.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.336+0000 7fc5068f0740 1 -- start start 2024-07-31T15:14:29.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc5068f0740 1 Processor -- start 2024-07-31T15:14:29.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc5068f0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41e8a1b0 0x555c42267fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc5068f0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e92390 0x555c422684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc5068f0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41eb3070 0x555c42268a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc5068f0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f81af980 con 0x555c41e8a1b0 2024-07-31T15:14:29.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc5068f0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f81af2b0 con 0x555c41e92390 2024-07-31T15:14:29.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc5068f0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f81ac510 con 0x555c41eb3070 2024-07-31T15:14:29.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc5068f0740 10 client.0 _ll_register_callbacks cb 0x7fc4e80a7170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:14:29.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc4fe13b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41e8a1b0 0x555c42267fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc4fd93a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e92390 0x555c422684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc4fe13b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41e8a1b0 0x555c42267fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56300/0 (socket says 172.21.15.67:56300) 2024-07-31T15:14:29.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc4fe93c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41eb3070 0x555c42268a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc4fe13b640 1 -- 172.21.15.67:0/2258747610 learned_addr learned my addr 172.21.15.67:0/2258747610 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:29.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc4fd93a640 1 --2- 172.21.15.67:0/2258747610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e92390 0x555c422684e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37186/0 (socket says 172.21.15.67:37186) 2024-07-31T15:14:29.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc4fd93a640 1 -- 172.21.15.67:0/2258747610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41eb3070 msgr2=0x555c42268a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:29.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc4fd93a640 1 --2- 172.21.15.67:0/2258747610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41eb3070 0x555c42268a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc4fd93a640 1 -- 172.21.15.67:0/2258747610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41e8a1b0 msgr2=0x555c42267fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:14:29.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc4fd93a640 1 --2- 172.21.15.67:0/2258747610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41e8a1b0 0x555c42267fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc4fd93a640 1 -- 172.21.15.67:0/2258747610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c422e1fc0 con 0x555c41e92390 2024-07-31T15:14:29.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc4fe93c640 1 --2- 172.21.15.67:0/2258747610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41eb3070 0x555c42268a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:29.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.337+0000 7fc4fe13b640 1 --2- 172.21.15.67:0/2258747610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41e8a1b0 0x555c42267fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:29.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.338+0000 7fc4fd93a640 1 --2- 172.21.15.67:0/2258747610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e92390 0x555c422684e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec006010 tx=0x7fc4ec02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f621f9b688b6ff2 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:29.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.338+0000 7fc4ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:14:29.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.338+0000 7fc4ddffb640 1 -- 172.21.15.67:0/2258747610 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec057060 con 0x555c41e92390 2024-07-31T15:14:29.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.338+0000 7fc5068f0740 1 -- 172.21.15.67:0/2258747610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f0005570 con 0x555c41e92390 2024-07-31T15:14:29.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.338+0000 7fc4ddffb640 1 -- 172.21.15.67:0/2258747610 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7fc4ec003a00 con 0x555c41e92390 2024-07-31T15:14:29.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.338+0000 7fc4ddffb640 1 -- 172.21.15.67:0/2258747610 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fc4ec05e050 con 0x555c41e92390 2024-07-31T15:14:29.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.338+0000 7fc5068f0740 10 client.29987 fetch_fsmap learned FSMap version 66 2024-07-31T15:14:29.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.338+0000 7fc5068f0740 1 -- 172.21.15.67:0/2258747610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x555c422e3250 con 0x555c41e92390 2024-07-31T15:14:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.338+0000 7fc4ddffb640 1 -- 172.21.15.67:0/2258747610 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f0005570 con 0x555c41e92390 2024-07-31T15:14:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.339+0000 7fc4ddffb640 1 -- 172.21.15.67:0/2258747610 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fc4ec052cd0 con 0x555c41e92390 2024-07-31T15:14:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.339+0000 7fc5068f0740 10 client.29987 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:14:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.339+0000 7fc5068f0740 10 client.29987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:14:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.339+0000 7fc5068f0740 20 client.29987 populate_metadata read hostname 'smithi067' 2024-07-31T15:14:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.339+0000 7fc5068f0740 1 --2- 172.21.15.67:0/2258747610 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c41e46da0 0x555c422e5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.339+0000 7fc5068f0740 4 client.29987 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:14:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.339+0000 7fc5068f0740 1 -- 172.21.15.67:0/2258747610 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555c421cc550 con 0x555c41e46da0 2024-07-31T15:14:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.339+0000 7fc4fe13b640 1 --2- 172.21.15.67:0/2258747610 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c41e46da0 0x555c422e5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.339+0000 7fc4ddffb640 10 client.29987 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.339+0000 7fc4fe13b640 1 --2- 172.21.15.67:0/2258747610 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c41e46da0 0x555c422e5a40 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89790c88ae69d183 server_cookie=a087add3d542f2 in_seq=0 out_seq=0 2024-07-31T15:14:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc4ddffb640 1 -- 172.21.15.67:0/2258747610 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x555c421cc550 con 0x555c41e46da0 2024-07-31T15:14:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc4ddffb640 10 client.29987 handle_command_reply: tid=0 2024-07-31T15:14:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc5068f0740 1 client.29987 shutdown 2024-07-31T15:14:29.619 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:29 smithi067 ceph-mon[32750]: pgmap v1148: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.2 MiB/s wr, 41 op/s 2024-07-31T15:14:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:29 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:29.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:29 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:29.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc5068f0740 1 -- 172.21.15.67:0/2258747610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e92390 msgr2=0x555c422684e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc5068f0740 1 --2- 172.21.15.67:0/2258747610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e92390 0x555c422684e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec006010 tx=0x7fc4ec02a410 comp rx=0 tx=0).stop 2024-07-31T15:14:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc5068f0740 1 -- 172.21.15.67:0/2258747610 shutdown_connections 2024-07-31T15:14:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc5068f0740 1 --2- 172.21.15.67:0/2258747610 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x555c41e46da0 0x555c422e5a40 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc5068f0740 1 --2- 172.21.15.67:0/2258747610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c41eb3070 0x555c42268a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc5068f0740 1 --2- 172.21.15.67:0/2258747610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c41e92390 0x555c422684e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc4ddffb640 0 client.29987 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc5068f0740 1 --2- 172.21.15.67:0/2258747610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c41e8a1b0 0x555c42267fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc5068f0740 1 -- 172.21.15.67:0/2258747610 >> 172.21.15.67:0/2258747610 conn(0x555c41ea7460 msgr2=0x555c42259670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:29.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc5068f0740 1 -- 172.21.15.67:0/2258747610 shutdown_connections 2024-07-31T15:14:29.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc5068f0740 1 -- 172.21.15.67:0/2258747610 wait complete. 2024-07-31T15:14:29.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.618+0000 7fc5068f0740 20 client.29987 trim_cache size 0 max 16384 2024-07-31T15:14:29.623 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:14:29.623 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (5357 inodes in the stack)", 2024-07-31T15:14:29.623 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-31T15:14:29.624 INFO:teuthology.orchestra.run.smithi067.stdout: "e297981a-9036-463f-957e-e32c8e80d535": { 2024-07-31T15:14:29.624 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-31T15:14:29.624 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "e297981a-9036-463f-957e-e32c8e80d535", 2024-07-31T15:14:29.624 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-31T15:14:29.624 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:14:29.624 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:14:29.624 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:14:29.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.619+0000 7fc50039f640 1 -- 172.21.15.67:0/2417104165 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc4e00b3d40 msgr2=0x7fc4e00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:29.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.619+0000 7fc50039f640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc4e00b3d40 0x7fc4e00b6250 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec000c00 tx=0x7fc4ec005780 comp rx=0 tx=0).stop 2024-07-31T15:14:29.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.619+0000 7fc50039f640 1 -- 172.21.15.67:0/2417104165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f8108760 msgr2=0x7fc4f819e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:29.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.619+0000 7fc50039f640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f8108760 0x7fc4f819e640 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009fd0 tx=0x7fc4e800ef10 comp rx=0 tx=0).stop 2024-07-31T15:14:29.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.619+0000 7fc50039f640 1 -- 172.21.15.67:0/2417104165 shutdown_connections 2024-07-31T15:14:29.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.619+0000 7fc50039f640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc4e00b3d40 0x7fc4e00b6250 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.619+0000 7fc50039f640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f8108760 0x7fc4f819e640 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.619+0000 7fc50039f640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f8107e10 0x7fc4f81993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.619+0000 7fc50039f640 1 --2- 172.21.15.67:0/2417104165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f81024d0 0x7fc4f8198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:29.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.619+0000 7fc50039f640 1 -- 172.21.15.67:0/2417104165 >> 172.21.15.67:0/2417104165 conn(0x7fc4f8079180 msgr2=0x7fc4f8104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:29.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.619+0000 7fc50039f640 1 -- 172.21.15.67:0/2417104165 shutdown_connections 2024-07-31T15:14:29.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:29.619+0000 7fc50039f640 1 -- 172.21.15.67:0/2417104165 wait complete. 2024-07-31T15:14:29.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:29.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:29 smithi073 ceph-mon[36714]: pgmap v1148: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.2 MiB/s wr, 41 op/s 2024-07-31T15:14:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:29 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:29.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:29 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:29.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:29.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:29.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:29 smithi077 ceph-mon[37327]: pgmap v1148: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.2 MiB/s wr, 41 op/s 2024-07-31T15:14:29.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:29.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:29.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:29.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:29.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:29 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:29.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:29 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:30.384 INFO:tasks.cephfs.filesystem:scrub status for tag:e297981a-9036-463f-957e-e32c8e80d535 - {'path': '/', 'tag': 'e297981a-9036-463f-957e-e32c8e80d535', 'options': 'recursive,force'} 2024-07-31T15:14:30.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:30 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:30 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:30.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:30 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:14:30.644 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:30 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:30.148+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000c061 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math/ auth{1=1} v1164 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:10.213499+0000 31=31+0) n(v2 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1) old_inodes=2 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:653488 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@16} mcw={1=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8e92fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000c061:[<0x1000000bf5f/math v634>,<0x10000000202/lib v5688>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000c061:[<0x1000000bf5f/math v1164>,<0x10000000202/lib v21040>,<0x10000000201/linux-6.5.11 v3100>,<0x10000000006/t v1908>,<0x10000000005/tmp v1829>,<0x100000001fb/client.0 v1795>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1758>,<0x10000000001/sv_1 v1746>,<0x10000000000/qa v1740>,<0x1/volumes v1679>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:10.988109+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:10.213499+0000 31=31+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:05:10.215499+0000 b133264 32=31+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:30.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:30.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:30.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:30.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:30 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:30.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:30 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:30.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:30 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:14:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:30.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:30 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:30 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:30.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:30 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000c061 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/lib/math) see mds.b log and `damage ls` output for details 2024-07-31T15:14:31.540 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:31 smithi073 ceph-mon[36714]: pgmap v1149: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 7.2 MiB/s wr, 65 op/s 2024-07-31T15:14:31.541 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:31.284+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000ce48 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity/ auth{1=1,2=1} v1695 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3) n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4) old_inodes=3 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:654013 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@16} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8aaac580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000ce48:[<0x1000000ce07/integrity v345>,<0x10000000202/security v8766>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000ce48:[<0x1000000ce07/integrity v1695>,<0x10000000202/security v15110>,<0x10000000201/linux-6.5.11 v3104>,<0x10000000006/t v1908>,<0x10000000005/tmp v1829>,<0x100000001fb/client.0 v1795>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1758>,<0x10000000001/sv_1 v1746>,<0x10000000000/qa v1740>,<0x1/volumes v1679>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:59.470106+0000 10=7+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","memory_value.dirstat":"f(v1 m2024-07-31T15:03:50.592123+0000 18=15+3)","memory_value.rstat":"n(v6 rc2024-07-31T15:03:50.594123+0000 b454474 54=50+4)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:31.541 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:31.430+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009eb8 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs/ auth{4=1} v4394 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:25.901734+0000 12=12+0) n(v3 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:654327 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@13} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce809d7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009eb8:[<0x10000009d36/debugfs v3846>,<0x10000000202/fs v3830>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009eb8:[<0x10000009d36/debugfs v4394>,<0x10000000202/fs v22489>,<0x10000000201/linux-6.5.11 v3104>,<0x10000000006/t v1908>,<0x10000000005/tmp v1829>,<0x100000001fb/client.0 v1795>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1758>,<0x10000000001/sv_1 v1746>,<0x10000000000/qa v1740>,<0x1/volumes v1679>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:39.315767+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:25.901734+0000 12=12+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:25.902734+0000 b241227 13=12+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:31.542 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:31.479+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000a492 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs/ auth{1=1} v5396 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:30.806634+0000 8=8+0) n(v3 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1) old_inodes=2 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:654400 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@13} mcw={1=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8aca9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000a492:[<0x10000009d36/tracefs v3850>,<0x10000000202/fs v3830>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x1000000a492:[<0x10000009d36/tracefs v5396>,<0x10000000202/fs v22489>,<0x10000000201/linux-6.5.11 v3104>,<0x10000000006/t v1908>,<0x10000000005/tmp v1829>,<0x100000001fb/client.0 v1795>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1758>,<0x10000000001/sv_1 v1746>,<0x10000000000/qa v1740>,<0x1/volumes v1679>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:41.554720+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:30.806634+0000 8=8+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:07:30.813634+0000 b78388 9=8+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:31.542 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:31.538+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002351 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char/ auth{2=1} v18176 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:34.793553+0000 56=49+7) n(v10 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10) old_inodes=4 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:659951 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@18} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7ef94100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000002351:[<0x10000001d8f/char v2490>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x10000002351:[<0x10000001d8f/char v18176>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3104>,<0x10000000006/t v1908>,<0x10000000005/tmp v1829>,<0x100000001fb/client.0 v1795>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1758>,<0x10000000001/sv_1 v1746>,<0x10000000000/qa v1740>,<0x1/volumes v1679>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:02.131047+0000 40=33+7)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:34.793553+0000 56=49+7)","memory_value.rstat":"n(v10 rc2024-07-31T15:07:34.794553+0000 b3465242 277=267+10)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:31.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:31 smithi077 ceph-mon[37327]: pgmap v1149: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 7.2 MiB/s wr, 65 op/s 2024-07-31T15:14:31.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:31 smithi067 ceph-mon[32750]: pgmap v1149: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 7.2 MiB/s wr, 65 op/s 2024-07-31T15:14:31.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:31.613+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009548 [...27,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/ufs/ auth{1=1} v17620 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:49.328365+0000 8=6+2) n(v6 rc2024-07-31T15:11:49.331365+0000 b747223 53=50+3) old_inodes=3 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:660000 nref=3)) | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=0 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8b8e5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009548:[<0x10000001d8f/ufs v16979>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009548:[<0x10000001d8f/ufs v17620>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3104>,<0x10000000006/t v1908>,<0x10000000005/tmp v1829>,<0x100000001fb/client.0 v1795>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1758>,<0x10000000001/sv_1 v1746>,<0x10000000000/qa v1740>,<0x1/volumes v1679>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:33.222894+0000 4=2+2)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:49.331365+0000 b747223 53=50+3)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:49.328365+0000 8=6+2)","memory_value.rstat":"n(v6 rc2024-07-31T15:11:49.331365+0000 b747223 53=50+3)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:31.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:31.661+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000243a [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk/ auth{2=1,4=1} fragtree_t(*^3) v18679 ap=3 RANDEPHEMERALPIN f(v3 m2024-07-31T15:07:07.059118+0000 137=88+49) n(v9 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50) old_inodes=2 (inest sync->mix g=2,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:660036 nref=3)) caps={15489=pAsLsXs/-@13} | dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce7fa16680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000243a:[<0x10000001d8f/clk v2484>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x1000000243a:[<0x10000001d8f/clk v18679>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3104>,<0x10000000006/t v1908>,<0x10000000005/tmp v1829>,<0x100000001fb/client.0 v1795>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1758>,<0x10000000001/sv_1 v1746>,<0x10000000000/qa v1740>,<0x1/volumes v1679>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 133=84+49)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50)","memory_value.dirstat":"f(v3 m2024-07-31T15:07:07.059118+0000 137=88+49)","memory_value.rstat":"n(v9 rc2024-07-31T15:07:07.061118+0000 b17348983 1333=1283+50)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:31.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:31.672+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000048e8 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/i2c/ auth{3=1} v19350 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:33.581686+0000 32=29+3) n(v6 rc2024-07-31T15:11:33.583686+0000 b3393592 213=209+4) old_inodes=1 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:660045 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@12} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7d07e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000048e8:[<0x10000001d8f/i2c v9487>,<0x10000000202/drivers v2799>,<0x10000000201/linux-6.5.11 v334>,<0x10000000006/t v232>,<0x10000000005/tmp v202>,<0x100000001fb/client.0 v201>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v184>,<0x10000000001/sv_1 v213>,<0x10000000000/qa v197>,<0x1/volumes v198>]//[]","memoryvalue":"(2)0x100000048e8:[<0x10000001d8f/i2c v19350>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3104>,<0x10000000006/t v1908>,<0x10000000005/tmp v1829>,<0x100000001fb/client.0 v1795>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1758>,<0x10000000001/sv_1 v1746>,<0x10000000000/qa v1740>,<0x1/volumes v1679>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:40.705240+0000 18=15+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:33.583686+0000 b3393592 213=209+4)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:33.581686+0000 32=29+3)","memory_value.rstat":"n(v6 rc2024-07-31T15:11:33.583686+0000 b3393592 213=209+4)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:31.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:31.718+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000008596 [...24,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/scsi/ auth{1=6,2=1,3=1,4=1} fragtree_t(*^3) v18676 ap=3 RANDEPHEMERALPIN f(v4 m2024-07-31T15:10:01.775559+0000 269=232+37) n(v12 rc2024-07-31T15:10:01.777559+0000 b31477139 971=924+47) old_inodes=3 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:660068 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@20} mcw={1=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce864ab600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000008596:[<0x10000001d8f/scsi v16994>,<0x10000000202/drivers v3182>,<0x10000000201/linux-6.5.11 v445>,<0x10000000006/t v329>,<0x10000000005/tmp v299>,<0x100000001fb/client.0 v296>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v274>,<0x10000000001/sv_1 v299>,<0x10000000000/qa v280>,<0x1/volumes v280>]//[]","memoryvalue":"(2)0x10000008596:[<0x10000001d8f/scsi v18676>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3104>,<0x10000000006/t v1908>,<0x10000000005/tmp v1829>,<0x100000001fb/client.0 v1795>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1758>,<0x10000000001/sv_1 v1746>,<0x10000000000/qa v1740>,<0x1/volumes v1679>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:10:00.625582+0000 265=228+37)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:10:01.777559+0000 b31477139 971=924+47)","memory_value.dirstat":"f(v4 m2024-07-31T15:10:01.775559+0000 269=232+37)","memory_value.rstat":"n(v12 rc2024-07-31T15:10:01.777559+0000 b31477139 971=924+47)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:31.894 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:31.724+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009980 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video/ auth{1=1,4=1} v18105 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:16.340374+0000 28=24+4) n(v11 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28) old_inodes=2 (inest sync->mix g=1,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:660078 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@13} mcw={1=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce87325b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009980:[<0x10000001d8f/video v17889>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009980:[<0x10000001d8f/video v18105>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3104>,<0x10000000006/t v1908>,<0x10000000005/tmp v1829>,<0x100000001fb/client.0 v1795>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1758>,<0x10000000001/sv_1 v1746>,<0x10000000000/qa v1740>,<0x1/volumes v1679>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:35.535846+0000 16=12+4)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:16.340374+0000 28=24+4)","memory_value.rstat":"n(v11 rc2024-07-31T15:05:16.341374+0000 b9537183 615=587+28)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:31.894 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:31.731+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005460 [...25,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media/ auth{2=1,3=1,4=1} v18772 ap=3 RANDEPHEMERALPIN f(v2 m2024-07-31T15:11:38.494586+0000 23=6+17) n(v13 rc2024-07-31T15:11:38.496586+0000 b38474543 3300=3123+177) old_inodes=2 (inest sync->mix g=2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:660079 nref=3)) caps={15489=pAsLsXs/-@12} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55ce82ee2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000005460:[<0x10000001d8f/media v12887>,<0x10000000202/drivers v2910>,<0x10000000201/linux-6.5.11 v379>,<0x10000000006/t v275>,<0x10000000005/tmp v244>,<0x100000001fb/client.0 v243>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v221>,<0x10000000001/sv_1 v249>,<0x10000000000/qa v233>,<0x1/volumes v234>]//[]","memoryvalue":"(2)0x10000005460:[<0x10000001d8f/media v18772>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3104>,<0x10000000006/t v1908>,<0x10000000005/tmp v1829>,<0x100000001fb/client.0 v1795>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1758>,<0x10000000001/sv_1 v1746>,<0x10000000000/qa v1740>,<0x1/volumes v1679>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:18.811444+0000 19=2+17)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:38.496586+0000 b38474543 3300=3123+177)","memory_value.dirstat":"f(v2 m2024-07-31T15:11:38.494586+0000 23=6+17)","memory_value.rstat":"n(v13 rc2024-07-31T15:11:38.496586+0000 b38474543 3300=3123+177)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:32.642 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:32.489+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000be32 [...16,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module/ auth{2=1} v2029 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0) n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:663135 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@19} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce826b7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000be32:[<0x1000000bd0a/module v1134>,<0x10000000202/kernel v5648>,<0x10000000201/linux-6.5.11 v623>,<0x10000000006/t v458>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v414>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v399>,<0x10000000001/sv_1 v419>,<0x10000000000/qa v405>,<0x1/volumes v398>]//[]","memoryvalue":"(2)0x1000000be32:[<0x1000000bd0a/module v2029>,<0x10000000202/kernel v21206>,<0x10000000201/linux-6.5.11 v3104>,<0x10000000006/t v1908>,<0x10000000005/tmp v1829>,<0x100000001fb/client.0 v1795>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1758>,<0x10000000001/sv_1 v1746>,<0x10000000000/qa v1740>,<0x1/volumes v1679>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:07.168188+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:04:02.091888+0000 37=37+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:04:02.098888+0000 b660236 38=37+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009548 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/ufs) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000048e8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/i2c) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:32.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000008596 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/scsi) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:32 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000005460 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009548 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/ufs) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000048e8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/i2c) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:32.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000008596 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/scsi) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:32 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000005460 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000ce48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/security/integrity) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009eb8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/debugfs) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000a492 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/fs/tracefs) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000002351 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/char) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009548 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/ufs) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:32.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000243a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/clk) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000048e8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/i2c) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:32.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000008596 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/scsi) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009980 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/video) see mds.b log and `damage ls` output for details 2024-07-31T15:14:32.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:32 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000005460 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media) see mds.b log and `damage ls` output for details 2024-07-31T15:14:33.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: pgmap v1150: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 7.2 MiB/s wr, 141 op/s 2024-07-31T15:14:33.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:14:33.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:33.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:33 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: pgmap v1150: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 7.2 MiB/s wr, 141 op/s 2024-07-31T15:14:33.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:14:33.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:33.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:33 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: pgmap v1150: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 7.2 MiB/s wr, 141 op/s 2024-07-31T15:14:33.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000be32 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/kernel/module) see mds.b log and `damage ls` output for details 2024-07-31T15:14:33.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:33.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:33.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:33 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:34 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:34.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:34 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:34.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:34.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:34.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:34.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:34 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:34.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:34 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:34.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:34.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:34.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:34.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:34.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:34 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:34.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:34 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:35.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:35 smithi073 ceph-mon[36714]: pgmap v1151: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 5.9 MiB/s wr, 361 op/s 2024-07-31T15:14:35.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:35 smithi077 ceph-mon[37327]: pgmap v1151: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 5.9 MiB/s wr, 361 op/s 2024-07-31T15:14:35.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:35 smithi067 ceph-mon[32750]: pgmap v1151: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 5.9 MiB/s wr, 361 op/s 2024-07-31T15:14:35.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.808+0000 7f9406ffd640 1 -- 172.21.15.67:0/881440846 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557e6b3307c0 con 0x557e6afae580 2024-07-31T15:14:35.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.808+0000 7f9406ffd640 10 client.29728 handle_command_reply: tid=0 2024-07-31T15:14:35.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.808+0000 7f94342e9740 1 client.29728 shutdown 2024-07-31T15:14:35.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.809+0000 7f94342e9740 1 -- 172.21.15.67:0/881440846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6aff1990 msgr2=0x557e6b3d0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:35.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.809+0000 7f94342e9740 1 --2- 172.21.15.67:0/881440846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6aff1990 0x557e6b3d0520 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f94180d4150 tx=0x7f94180cb540 comp rx=0 tx=0).stop 2024-07-31T15:14:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.809+0000 7f94342e9740 1 -- 172.21.15.67:0/881440846 shutdown_connections 2024-07-31T15:14:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.809+0000 7f94342e9740 1 --2- 172.21.15.67:0/881440846 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x557e6afae580 0x557e6b448fc0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.809+0000 7f94342e9740 1 --2- 172.21.15.67:0/881440846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557e6b01a850 0x557e6b445410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.809+0000 7f9406ffd640 0 client.29728 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:14:35.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.809+0000 7f94342e9740 1 --2- 172.21.15.67:0/881440846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e6aff9b70 0x557e6b3d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:35.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.809+0000 7f94342e9740 1 --2- 172.21.15.67:0/881440846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557e6aff1990 0x557e6b3d0520 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:35.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.809+0000 7f94342e9740 1 -- 172.21.15.67:0/881440846 >> 172.21.15.67:0/881440846 conn(0x557e6b00ec40 msgr2=0x557e6b3c1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:35.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.809+0000 7f94342e9740 1 -- 172.21.15.67:0/881440846 shutdown_connections 2024-07-31T15:14:35.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.809+0000 7f94342e9740 1 -- 172.21.15.67:0/881440846 wait complete. 2024-07-31T15:14:35.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.809+0000 7f94342e9740 20 client.29728 trim_cache size 0 max 16384 2024-07-31T15:14:35.813 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:14:35.814 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 66, 2024-07-31T15:14:35.814 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 24671, 2024-07-31T15:14:35.814 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 66, 2024-07-31T15:14:35.814 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-07-31T15:14:35.814 INFO:teuthology.orchestra.run.smithi067.stdout: "3f258781": { 2024-07-31T15:14:35.814 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 66, 2024-07-31T15:14:35.815 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-07-31T15:14:35.815 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:14:35.815 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:14:35.815 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:14:35.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:14:35.815 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-07-31T15:14:35.815 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 137.9, 2024-07-31T15:14:35.815 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-07-31T15:14:35.816 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-07-31T15:14:35.816 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-07-31T15:14:35.816 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-07-31T15:14:35.816 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-07-31T15:14:35.816 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-07-31T15:14:35.816 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:14:35.816 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:14:35.816 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:14:35.817 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:14:35.817 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:14:35.817 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:14:35.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.810+0000 7f942dd98640 1 -- 172.21.15.67:0/3219212251 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94100b3df0 msgr2=0x7f94100b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:35.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.810+0000 7f942dd98640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94100b3df0 0x7f94100b6300 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9428100640 tx=0x7f941c00b390 comp rx=0 tx=0).stop 2024-07-31T15:14:35.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.810+0000 7f942dd98640 1 -- 172.21.15.67:0/3219212251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94281024b0 msgr2=0x7f942819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:35.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.810+0000 7f942dd98640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94281024b0 0x7f942819ad10 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f9418005940 tx=0x7f94180042f0 comp rx=0 tx=0).stop 2024-07-31T15:14:35.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.810+0000 7f942dd98640 1 -- 172.21.15.67:0/3219212251 shutdown_connections 2024-07-31T15:14:35.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.810+0000 7f942dd98640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f94100b3df0 0x7f94100b6300 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:35.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.810+0000 7f942dd98640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9428108740 0x7f942819b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:35.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.810+0000 7f942dd98640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9428107df0 0x7f942819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:35.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.810+0000 7f942dd98640 1 --2- 172.21.15.67:0/3219212251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94281024b0 0x7f942819ad10 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:35.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.810+0000 7f942dd98640 1 -- 172.21.15.67:0/3219212251 >> 172.21.15.67:0/3219212251 conn(0x7f9428079190 msgr2=0x7f9428100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:35.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.810+0000 7f942dd98640 1 -- 172.21.15.67:0/3219212251 shutdown_connections 2024-07-31T15:14:35.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:35.810+0000 7f942dd98640 1 -- 172.21.15.67:0/3219212251 wait complete. 2024-07-31T15:14:36.382 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:35 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:35.901+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d187 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi/ auth{1=1} v3040 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0) n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1) old_inodes=1 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:665271 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7c6c1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d187:[<0x1000000d116/ctxfi v1858>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d187:[<0x1000000d116/ctxfi v3040>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.452342+0000 30=30+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.972872+0000 34=34+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.973872+0000 b319358 35=34+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:36.383 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:35 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:35.905+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d119 [...19,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97/ auth{2=1} v3208 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0) n(v3 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1) old_inodes=3 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:665274 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce9047e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d119:[<0x1000000d116/ac97 v22>,<0x1000000cf1b/pci v463>,<0x10000000202/sound v8728>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000d119:[<0x1000000d116/ac97 v3208>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:00.491085+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:40.833875+0000 12=12+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:40.834875+0000 b267165 13=12+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:36.383 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:35 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:35.906+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d1cc [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1/ auth{2=1} v3042 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0) n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:665281 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8f176c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v1862>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d1cc:[<0x1000000d116/emu10k1 v3042>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.549340+0000 22=22+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.161868+0000 26=26+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:05:41.162868+0000 b488856 27=26+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:36.383 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:35 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:35.913+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000d180 [...17,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio/ auth{1=1} v3038 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0) n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1) old_inodes=1 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:665320 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8ebd2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v1860>,<0x1000000cf1b/pci v1685>,<0x10000000202/sound v9784>,<0x10000000201/linux-6.5.11 v796>,<0x10000000006/t v562>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v513>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v494>,<0x10000000001/sv_1 v503>,<0x10000000000/qa v498>,<0x1/volumes v481>]//[]","memoryvalue":"(2)0x1000000d180:[<0x1000000d116/cs5535audio v3038>,<0x1000000cf1b/pci v3874>,<0x10000000202/sound v19478>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:59:36.414343+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:05:41.056870+0000 10=10+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:05:41.058870+0000 b33013 11=10+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:36.383 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:36.018+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000cde2 [...10,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders/ auth{4=1} v112 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0) n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:675466 nref=3)) caps={15489=pAsLsXs/-@9} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8dc52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v29>,<0x1000000cbec/selinux v646>,<0x10000000202/scripts v8820>,<0x10000000201/linux-6.5.11 v749>,<0x10000000006/t v529>,<0x10000000005/tmp v497>,<0x100000001fb/client.0 v482>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v464>,<0x10000000001/sv_1 v477>,<0x10000000000/qa v467>,<0x1/volumes v457>]//[]","memoryvalue":"(2)0x1000000cde2:[<0x1000000cddf/genheaders v112>,<0x1000000cbec/selinux v749>,<0x10000000202/scripts v11639>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:58:58.999116+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:01:02.667566+0000 5=5+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:01:02.667566+0000 b52056 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:36.616 INFO:tasks.quiescer:done joining Quiescers 2024-07-31T15:14:36.616 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-07-31T15:14:36.639 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi067.front.sepia.ceph.com 2024-07-31T15:14:36.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:36.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:36.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:36.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.648 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:36.648 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:36.648 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.649 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.649 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.649 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:36.649 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.650 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.650 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.650 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:36.650 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.650 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.651 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:36.651 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.651 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:36.651 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.651 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:36 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.653 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-07-31T15:14:36.668 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-07-31T15:14:36.669 INFO:teuthology.task.exec:Executing custom commands... 2024-07-31T15:14:36.669 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-07-31T15:14:36.669 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-07-31T15:14:36.730 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-07-31T15:14:36.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:36.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:36.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d187 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ctxfi) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:36.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d119 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/ac97) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d1cc (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/emu10k1) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:36.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000d180 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/sound/pci/cs5535audio) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:36.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:36.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:36.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:36 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000cde2 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/scripts/selinux/genheaders) see mds.b log and `damage ls` output for details 2024-07-31T15:14:36.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-07-31T15:14:36.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.972+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/570780283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3c81023f0 msgr2=0x7fa3c8102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:36.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.972+0000 7fa3cf0c9640 1 --2- 172.21.15.67:0/570780283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3c81023f0 0x7fa3c8102800 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009900 tx=0x7fa3bc02a410 comp rx=0 tx=0).stop 2024-07-31T15:14:36.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.972+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/570780283 shutdown_connections 2024-07-31T15:14:36.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.972+0000 7fa3cf0c9640 1 --2- 172.21.15.67:0/570780283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c8108670 0x7fa3c810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:36.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.972+0000 7fa3cf0c9640 1 --2- 172.21.15.67:0/570780283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3c8107d20 0x7fa3c8108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:36.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.972+0000 7fa3cf0c9640 1 --2- 172.21.15.67:0/570780283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3c81023f0 0x7fa3c8102800 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:36.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.972+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/570780283 >> 172.21.15.67:0/570780283 conn(0x7fa3c80fddf0 msgr2=0x7fa3c8100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:36.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.972+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/570780283 shutdown_connections 2024-07-31T15:14:36.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.972+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/570780283 wait complete. 2024-07-31T15:14:36.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.973+0000 7fa3cf0c9640 1 Processor -- start 2024-07-31T15:14:36.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.973+0000 7fa3cf0c9640 1 -- start start 2024-07-31T15:14:36.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.973+0000 7fa3cf0c9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3c81023f0 0x7fa3c8199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:36.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.973+0000 7fa3cf0c9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3c8107d20 0x7fa3c819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:36.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.973+0000 7fa3cf0c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c8108670 0x7fa3c819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:36.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.973+0000 7fa3cf0c9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8113480 con 0x7fa3c8107d20 2024-07-31T15:14:36.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.973+0000 7fa3cf0c9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8113300 con 0x7fa3c81023f0 2024-07-31T15:14:36.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.973+0000 7fa3cf0c9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8113600 con 0x7fa3c8108670 2024-07-31T15:14:36.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.973+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3c8107d20 0x7fa3c819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:36.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.973+0000 7fa3cce65640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3c81023f0 0x7fa3c8199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:36.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.973+0000 7fa3cd666640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c8108670 0x7fa3c819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:36.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.973+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3c8107d20 0x7fa3c819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46552/0 (socket says 172.21.15.67:46552) 2024-07-31T15:14:36.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.973+0000 7fa3cce65640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3c81023f0 0x7fa3c8199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:37780/0 (socket says 172.21.15.67:37780) 2024-07-31T15:14:36.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.973+0000 7fa3c7fff640 1 -- 172.21.15.67:0/1848469399 learned_addr learned my addr 172.21.15.67:0/1848469399 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:36.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.974+0000 7fa3cd666640 1 -- 172.21.15.67:0/1848469399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3c81023f0 msgr2=0x7fa3c8199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:36.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.974+0000 7fa3cd666640 1 --2- 172.21.15.67:0/1848469399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3c81023f0 0x7fa3c8199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:36.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.974+0000 7fa3cd666640 1 -- 172.21.15.67:0/1848469399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3c8107d20 msgr2=0x7fa3c819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:36.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.974+0000 7fa3cd666640 1 --2- 172.21.15.67:0/1848469399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3c8107d20 0x7fa3c819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:36.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.974+0000 7fa3cd666640 1 -- 172.21.15.67:0/1848469399 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c819fdf0 con 0x7fa3c8108670 2024-07-31T15:14:36.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.974+0000 7fa3c7fff640 1 --2- 172.21.15.67:0/1848469399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3c8107d20 0x7fa3c819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:14:36.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.974+0000 7fa3cd666640 1 --2- 172.21.15.67:0/1848469399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c8108670 0x7fa3c819f6f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000ec50 tx=0x7fa3c000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:36.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.974+0000 7fa3c5ffb640 1 -- 172.21.15.67:0/1848469399 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0003b70 con 0x7fa3c8108670 2024-07-31T15:14:36.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.974+0000 7fa3c5ffb640 1 -- 172.21.15.67:0/1848469399 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c0003d10 con 0x7fa3c8108670 2024-07-31T15:14:36.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.974+0000 7fa3c5ffb640 1 -- 172.21.15.67:0/1848469399 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0010830 con 0x7fa3c8108670 2024-07-31T15:14:36.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.974+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/1848469399 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c81a00e0 con 0x7fa3c8108670 2024-07-31T15:14:36.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.975+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/1848469399 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c81a6870 con 0x7fa3c8108670 2024-07-31T15:14:36.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.976+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/1848469399 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c8102800 con 0x7fa3c8108670 2024-07-31T15:14:36.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.977+0000 7fa3c5ffb640 1 -- 172.21.15.67:0/1848469399 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fa3c0004b40 con 0x7fa3c8108670 2024-07-31T15:14:36.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.978+0000 7fa3c5ffb640 1 --2- 172.21.15.67:0/1848469399 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa3b80b3dc0 0x7fa3b80b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:36.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.978+0000 7fa3cce65640 1 --2- 172.21.15.67:0/1848469399 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa3b80b3dc0 0x7fa3b80b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:36.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.978+0000 7fa3c5ffb640 1 -- 172.21.15.67:0/1848469399 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7fa3c00e4900 con 0x7fa3c8108670 2024-07-31T15:14:36.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.979+0000 7fa3cce65640 1 --2- 172.21.15.67:0/1848469399 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa3b80b3dc0 0x7fa3b80b62d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc005ab0 tx=0x7fa3bc005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:36.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:36.981+0000 7fa3c5ffb640 1 -- 172.21.15.67:0/1848469399 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fa3c00ab8a0 con 0x7fa3c8108670 2024-07-31T15:14:37.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:36.867+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000008171 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/power/supply/ auth{3=1} fragtree_t(*^3) v450 ap=3 RANDEPHEMERALPIN f(v2 m2024-07-31T15:11:41.904516+0000 130=130+0) n(v4 rc2024-07-31T15:11:41.905516+0000 b2648100 131=130+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:682322 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@12} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce89c56c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000008171:[<0x10000008145/supply v416>,<0x10000001d8f/power v16677>,<0x10000000202/drivers v3182>,<0x10000000201/linux-6.5.11 v445>,<0x10000000006/t v329>,<0x10000000005/tmp v299>,<0x100000001fb/client.0 v296>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v274>,<0x10000000001/sv_1 v299>,<0x10000000000/qa v280>,<0x1/volumes v280>]//[]","memoryvalue":"(2)0x10000008171:[<0x10000008145/supply v450>,<0x10000001d8f/power v19017>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 118=118+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:41.905516+0000 b2648100 131=130+1)","memory_value.dirstat":"f(v2 m2024-07-31T15:11:41.904516+0000 130=130+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:11:41.905516+0000 b2648100 131=130+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:37.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:36.879+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000021f0 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader/ auth{2=1} v893 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:04.418950+0000 19=18+1) n(v5 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:682275 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@15} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7dc21b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000021f0:[<0x100000021db/firmware_loader v414>,<0x10000001d8f/base v2486>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x100000021f0:[<0x100000021db/firmware_loader v893>,<0x10000001d8f/base v18747>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:01.469061+0000 13=12+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:04.418950+0000 19=18+1)","memory_value.rstat":"n(v5 rc2024-07-31T15:08:04.420949+0000 b208922 29=27+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:37.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:36.947+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000006f7c [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/phy/ auth{3=1} fragtree_t(*^3) v3665 ap=3 RANDEPHEMERALPIN f(v3 m2024-07-31T15:10:56.134450+0000 111=110+1) n(v7 rc2024-07-31T15:10:57.456423+0000 b2820277 123=121+2) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:683529 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@17} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce86f32b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000006f7c:[<0x100000060b2/phy v1160>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x10000006f7c:[<0x100000060b2/phy v3665>,<0x10000001d8f/net v18833>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:10:56.134450+0000 103=102+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:10:57.456423+0000 b2820277 123=121+2)","memory_value.dirstat":"f(v3 m2024-07-31T15:10:56.134450+0000 111=110+1)","memory_value.rstat":"n(v7 rc2024-07-31T15:10:57.456423+0000 b2820277 123=121+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:37.144 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:36 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:36.984+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000093bf [...27,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/thermal/intel/ auth{4=1} v1029 ap=3 RANDEPHEMERALPIN f(v3 m2024-07-31T15:13:52.212857+0000 34=33+1) n(v7 rc2024-07-31T15:13:52.214857+0000 b483627 53=51+2) old_inodes=3 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:683427 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFsx@16} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce81e7d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000093bf:[<0x100000093a3/intel v467>,<0x10000001d8f/thermal v17380>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x100000093bf:[<0x100000093a3/intel v1029>,<0x10000001d8f/thermal v18844>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:13:50.923884+0000 30=29+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:13:52.214857+0000 b483627 53=51+2)","memory_value.dirstat":"f(v3 m2024-07-31T15:13:52.212857+0000 34=33+1)","memory_value.rstat":"n(v7 rc2024-07-31T15:13:52.214857+0000 b483627 53=51+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:37.144 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:37.093+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000048c5 [...27,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/hwtracing/intel_th/ auth{4=1} v514 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:12:40.723316+0000 21=21+0) n(v3 rc2024-07-31T15:12:40.725316+0000 b142954 22=21+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:684705 nref=3)) caps={15489=pAsLsXs/-@6} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce819e7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000048c5:[<0x1000000488c/intel_th v226>,<0x10000001d8f/hwtracing v9482>,<0x10000000202/drivers v2799>,<0x10000000201/linux-6.5.11 v334>,<0x10000000006/t v232>,<0x10000000005/tmp v202>,<0x100000001fb/client.0 v201>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v184>,<0x10000000001/sv_1 v213>,<0x10000000000/qa v197>,<0x1/volumes v198>]//[]","memoryvalue":"(2)0x100000048c5:[<0x1000000488c/intel_th v514>,<0x10000001d8f/hwtracing v20593>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:40.481244+0000 17=17+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:12:40.725316+0000 b142954 22=21+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:12:40.723316+0000 21=21+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:12:40.725316+0000 b142954 22=21+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:37.144 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:37.097+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002c0e [...27,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/xilinx/ auth{3=1} v1035 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:12:39.671338+0000 9=9+0) n(v3 rc2024-07-31T15:12:39.672338+0000 b60124 10=9+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:684984 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce80bf8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000002c0e:[<0x10000002b45/xilinx v328>,<0x10000001d8f/firmware v3927>,<0x10000000202/drivers v2561>,<0x10000000201/linux-6.5.11 v275>,<0x10000000006/t v183>,<0x10000000005/tmp v156>,<0x100000001fb/client.0 v155>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v140>,<0x10000000001/sv_1 v171>,<0x10000000000/qa v157>,<0x1/volumes v157>]//[]","memoryvalue":"(2)0x10000002c0e:[<0x10000002b45/xilinx v1035>,<0x10000001d8f/firmware v20643>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:16.051756+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:12:39.672338+0000 b60124 10=9+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:12:39.671338+0000 9=9+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:12:39.672338+0000 b60124 10=9+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:37.144 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:37.100+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002b48 [...27,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/arm_ffa/ auth{2=1} v1033 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:12:25.892619+0000 10=10+0) n(v3 rc2024-07-31T15:12:25.894619+0000 b28005 11=10+1) old_inodes=2 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:684996 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7ce0f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000002b48:[<0x10000002b45/arm_ffa v18>,<0x10000001d8f/firmware v3927>,<0x10000000202/drivers v2561>,<0x10000000201/linux-6.5.11 v275>,<0x10000000006/t v183>,<0x10000000005/tmp v156>,<0x100000001fb/client.0 v155>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v140>,<0x10000000001/sv_1 v171>,<0x10000000000/qa v157>,<0x1/volumes v157>]//[]","memoryvalue":"(2)0x10000002b48:[<0x10000002b45/arm_ffa v1033>,<0x10000001d8f/firmware v20643>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:15.738762+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:12:25.894619+0000 b28005 11=10+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:12:25.892619+0000 10=10+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:12:25.894619+0000 b28005 11=10+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:37.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.247+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/1848469399 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa3c8063230 con 0x7fa3b80b3dc0 2024-07-31T15:14:37.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.256+0000 7fa3c5ffb640 1 -- 172.21.15.67:0/1848469399 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7fa3c8063230 con 0x7fa3b80b3dc0 2024-07-31T15:14:37.259 INFO:teuthology.orchestra.run.smithi067.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-07-31T00:00:00", "created": "2024-07-31T14:51:21", "first": "2024-07-31T14:52:00", "last": "2024-07-31T15:14:00", "last_pruned": "2024-07-31T15:14:00", "created_count": 23, "pruned_count": 16, "active": true} 2024-07-31T15:14:37.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.259+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/1848469399 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa3b80b3dc0 msgr2=0x7fa3b80b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:37.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.259+0000 7fa3cf0c9640 1 --2- 172.21.15.67:0/1848469399 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa3b80b3dc0 0x7fa3b80b62d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc005ab0 tx=0x7fa3bc005a20 comp rx=0 tx=0).stop 2024-07-31T15:14:37.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.259+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/1848469399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c8108670 msgr2=0x7fa3c819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:37.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.259+0000 7fa3cf0c9640 1 --2- 172.21.15.67:0/1848469399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c8108670 0x7fa3c819f6f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000ec50 tx=0x7fa3c000c6a0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.259+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/1848469399 shutdown_connections 2024-07-31T15:14:37.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.259+0000 7fa3cf0c9640 1 --2- 172.21.15.67:0/1848469399 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa3b80b3dc0 0x7fa3b80b62d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.259+0000 7fa3cf0c9640 1 --2- 172.21.15.67:0/1848469399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c8108670 0x7fa3c819f6f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.259+0000 7fa3cf0c9640 1 --2- 172.21.15.67:0/1848469399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3c8107d20 0x7fa3c819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.259+0000 7fa3cf0c9640 1 --2- 172.21.15.67:0/1848469399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3c81023f0 0x7fa3c8199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.259+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/1848469399 >> 172.21.15.67:0/1848469399 conn(0x7fa3c80fddf0 msgr2=0x7fa3c81073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:37.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.259+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/1848469399 shutdown_connections 2024-07-31T15:14:37.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.259+0000 7fa3cf0c9640 1 -- 172.21.15.67:0/1848469399 wait complete. 2024-07-31T15:14:37.275 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-07-31T15:14:37.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.411+0000 7fc11caa9640 1 -- 172.21.15.67:0/3234064013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc118101b90 msgr2=0x7fc118101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:37.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.411+0000 7fc11caa9640 1 --2- 172.21.15.67:0/3234064013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc118101b90 0x7fc118101fa0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc108009900 tx=0x7fc10802a410 comp rx=0 tx=0).stop 2024-07-31T15:14:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.412+0000 7fc11caa9640 1 -- 172.21.15.67:0/3234064013 shutdown_connections 2024-07-31T15:14:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.412+0000 7fc11caa9640 1 --2- 172.21.15.67:0/3234064013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc118107f10 0x7fc11810e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.412+0000 7fc11caa9640 1 --2- 172.21.15.67:0/3234064013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1181075c0 0x7fc1181079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.412+0000 7fc11caa9640 1 --2- 172.21.15.67:0/3234064013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc118101b90 0x7fc118101fa0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.412+0000 7fc11caa9640 1 -- 172.21.15.67:0/3234064013 >> 172.21.15.67:0/3234064013 conn(0x7fc118079880 msgr2=0x7fc118079c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.412+0000 7fc11caa9640 1 -- 172.21.15.67:0/3234064013 shutdown_connections 2024-07-31T15:14:37.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.412+0000 7fc11caa9640 1 -- 172.21.15.67:0/3234064013 wait complete. 2024-07-31T15:14:37.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.413+0000 7fc11caa9640 1 Processor -- start 2024-07-31T15:14:37.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.413+0000 7fc11caa9640 1 -- start start 2024-07-31T15:14:37.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc11caa9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc118101b90 0x7fc118199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:37.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc11caa9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1181075c0 0x7fc11819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:37.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc11caa9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc118107f10 0x7fc11819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:37.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc11caa9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc118111bd0 con 0x7fc1181075c0 2024-07-31T15:14:37.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc11caa9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc118111a50 con 0x7fc118107f10 2024-07-31T15:14:37.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc11caa9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc118111d50 con 0x7fc118101b90 2024-07-31T15:14:37.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc115d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1181075c0 0x7fc11819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:37.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc11659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc118101b90 0x7fc118199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:37.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc11659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc118101b90 0x7fc118199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:50462/0 (socket says 172.21.15.67:50462) 2024-07-31T15:14:37.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc115d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1181075c0 0x7fc11819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46572/0 (socket says 172.21.15.67:46572) 2024-07-31T15:14:37.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc11659c640 1 -- 172.21.15.67:0/1116274548 learned_addr learned my addr 172.21.15.67:0/1116274548 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:14:37.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc116d9d640 1 --2- 172.21.15.67:0/1116274548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc118107f10 0x7fc11819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:37.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc115d9b640 1 -- 172.21.15.67:0/1116274548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc118101b90 msgr2=0x7fc118199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:37.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc115d9b640 1 --2- 172.21.15.67:0/1116274548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc118101b90 0x7fc118199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc115d9b640 1 -- 172.21.15.67:0/1116274548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc118107f10 msgr2=0x7fc11819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:37.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc115d9b640 1 --2- 172.21.15.67:0/1116274548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc118107f10 0x7fc11819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc115d9b640 1 -- 172.21.15.67:0/1116274548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11819fe90 con 0x7fc1181075c0 2024-07-31T15:14:37.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.414+0000 7fc115d9b640 1 --2- 172.21.15.67:0/1116274548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1181075c0 0x7fc11819a500 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fc118075360 tx=0x7fc10c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:37.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.415+0000 7fc1077fe640 1 -- 172.21.15.67:0/1116274548 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c00cd00 con 0x7fc1181075c0 2024-07-31T15:14:37.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.415+0000 7fc1077fe640 1 -- 172.21.15.67:0/1116274548 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc10c0035f0 con 0x7fc1181075c0 2024-07-31T15:14:37.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.415+0000 7fc11caa9640 1 -- 172.21.15.67:0/1116274548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1181a0180 con 0x7fc1181075c0 2024-07-31T15:14:37.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.415+0000 7fc1077fe640 1 -- 172.21.15.67:0/1116274548 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c010690 con 0x7fc1181075c0 2024-07-31T15:14:37.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.415+0000 7fc11caa9640 1 -- 172.21.15.67:0/1116274548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1181a6910 con 0x7fc1181075c0 2024-07-31T15:14:37.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.416+0000 7fc11caa9640 1 -- 172.21.15.67:0/1116274548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc118102e00 con 0x7fc1181075c0 2024-07-31T15:14:37.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.418+0000 7fc1077fe640 1 -- 172.21.15.67:0/1116274548 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc10c0037a0 con 0x7fc1181075c0 2024-07-31T15:14:37.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.419+0000 7fc1077fe640 1 --2- 172.21.15.67:0/1116274548 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc1000b3d40 0x7fc1000b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:14:37.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.419+0000 7fc1077fe640 1 -- 172.21.15.67:0/1116274548 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7fc10c0e4830 con 0x7fc1181075c0 2024-07-31T15:14:37.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.420+0000 7fc11659c640 1 --2- 172.21.15.67:0/1116274548 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc1000b3d40 0x7fc1000b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:14:37.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.420+0000 7fc11659c640 1 --2- 172.21.15.67:0/1116274548 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc1000b3d40 0x7fc1000b6250 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc108031440 tx=0x7fc1080058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:14:37.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.423+0000 7fc1077fe640 1 -- 172.21.15.67:0/1116274548 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc10c0555e0 con 0x7fc1181075c0 2024-07-31T15:14:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: pgmap v1152: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 3.1 MiB/s wr, 473 op/s 2024-07-31T15:14:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000008171 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/power/supply) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000006f7c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/phy) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:37.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000093bf (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/thermal/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000048c5 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/hwtracing/intel_th) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000002c0e (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/xilinx) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000002b48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/arm_ffa) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.560 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:37 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.573 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:37.237+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000008a08 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip/ auth{3=1} v2681 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:09.647065+0000 7=7+0) n(v2 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:687731 nref=3)) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8c1aa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000008a08:[<0x1000000894f/microchip v613>,<0x10000001d8f/soc v16560>,<0x10000000202/drivers v3182>,<0x10000000201/linux-6.5.11 v445>,<0x10000000006/t v329>,<0x10000000005/tmp v299>,<0x100000001fb/client.0 v296>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v274>,<0x10000000001/sv_1 v299>,<0x10000000000/qa v280>,<0x1/volumes v280>]//[]","memoryvalue":"(2)0x10000008a08:[<0x1000000894f/microchip v2681>,<0x10000001d8f/soc v17222>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:51.041772+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:09.647065+0000 7=7+0)","memory_value.rstat":"n(v2 rc2024-07-31T15:07:09.649065+0000 b6433 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:37.574 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:37.352+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001fc7 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica/ auth{2=1} fragtree_t(*^3) v1950 ap=3 RANDEPHEMERALPIN f(v10 m2024-07-31T15:07:39.153464+0000 516=516+0) n(v16 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1) old_inodes=4 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:689121 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@34} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7f0ba680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000001fc7:[<0x10000001fad/acpica v1734>,<0x10000001d8f/acpi v2434>,<0x10000000202/drivers v2461>,<0x10000000201/linux-6.5.11 v252>,<0x10000000006/t v164>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v140>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v126>,<0x10000000001/sv_1 v157>,<0x10000000000/qa v144>,<0x1/volumes v143>]//[]","memoryvalue":"(2)0x10000001fc7:[<0x10000001fad/acpica v1950>,<0x10000001d8f/acpi v18672>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:07:37.321501+0000 494=494+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1)","memory_value.dirstat":"f(v10 m2024-07-31T15:07:39.153464+0000 516=516+0)","memory_value.rstat":"n(v16 rc2024-07-31T15:07:39.154464+0000 b9174466 517=516+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:37.574 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:37.413+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000009462 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc/ auth{3=1} v1424 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:28.223687+0000 22=22+0) n(v4 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1) old_inodes=2 (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:689444 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@11} mcw={3=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce85c96b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000009462:[<0x1000000945c/hvc v352>,<0x10000001d8f/tty v18356>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x10000009462:[<0x1000000945c/hvc v1424>,<0x10000001d8f/tty v18901>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:32.992899+0000 16=16+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:28.223687+0000 22=22+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:07:28.231686+0000 b285324 23=22+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:37.574 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:37.485+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005d80 [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader/ auth{4=1} v1324 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:10.777820+0000 21=21+0) n(v3 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1) old_inodes=1 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:690339 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7f8f5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000005d80:[<0x10000005d5d/cardreader v104>,<0x10000001d8f/misc v13245>,<0x10000000202/drivers v2910>,<0x10000000201/linux-6.5.11 v379>,<0x10000000006/t v275>,<0x10000000005/tmp v244>,<0x100000001fb/client.0 v243>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v221>,<0x10000000001/sv_1 v249>,<0x10000000000/qa v233>,<0x1/volumes v234>]//[]","memoryvalue":"(2)0x10000005d80:[<0x10000005d5d/cardreader v1324>,<0x10000001d8f/misc v18516>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:21.430390+0000 17=17+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:10.777820+0000 21=21+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:08:10.781820+0000 b217931 22=21+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:37.575 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.575 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.575 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: pgmap v1152: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 3.1 MiB/s wr, 473 op/s 2024-07-31T15:14:37.575 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.575 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.575 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.575 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.575 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.576 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.576 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.576 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.576 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.576 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.576 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.576 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.576 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.577 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.577 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.577 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.577 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.577 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.577 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.577 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.578 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.578 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.578 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.578 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.578 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.578 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.578 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.578 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.579 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.579 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.579 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.579 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.579 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.579 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.579 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000008171 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/power/supply) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.579 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.579 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.580 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.580 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.580 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.580 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000006f7c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/phy) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.580 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:37.580 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.580 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000093bf (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/thermal/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.581 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000048c5 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/hwtracing/intel_th) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.581 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000002c0e (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/xilinx) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.581 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000002b48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/arm_ffa) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.581 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.687+0000 7fc11caa9640 1 -- 172.21.15.67:0/1116274548 --> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fc118063230 con 0x7fc1000b3d40 2024-07-31T15:14:37.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.696+0000 7fc1077fe640 1 -- 172.21.15.67:0/1116274548 <== mgr.14214 v2:172.21.15.67:6800/2637562983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fc118063230 con 0x7fc1000b3d40 2024-07-31T15:14:37.699 INFO:teuthology.orchestra.run.smithi067.stdout:/ 1m 6m3h 2024-07-31T15:14:37.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.699+0000 7fc11caa9640 1 -- 172.21.15.67:0/1116274548 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc1000b3d40 msgr2=0x7fc1000b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.699+0000 7fc11caa9640 1 --2- 172.21.15.67:0/1116274548 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc1000b3d40 0x7fc1000b6250 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc108031440 tx=0x7fc1080058d0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.699+0000 7fc11caa9640 1 -- 172.21.15.67:0/1116274548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1181075c0 msgr2=0x7fc11819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:14:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.699+0000 7fc11caa9640 1 --2- 172.21.15.67:0/1116274548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1181075c0 0x7fc11819a500 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fc118075360 tx=0x7fc10c00ee50 comp rx=0 tx=0).stop 2024-07-31T15:14:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.699+0000 7fc11caa9640 1 -- 172.21.15.67:0/1116274548 shutdown_connections 2024-07-31T15:14:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.699+0000 7fc11caa9640 1 --2- 172.21.15.67:0/1116274548 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fc1000b3d40 0x7fc1000b6250 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.699+0000 7fc11caa9640 1 --2- 172.21.15.67:0/1116274548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc118107f10 0x7fc11819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.699+0000 7fc11caa9640 1 --2- 172.21.15.67:0/1116274548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1181075c0 0x7fc11819a500 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.699+0000 7fc11caa9640 1 --2- 172.21.15.67:0/1116274548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc118101b90 0x7fc118199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:14:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.699+0000 7fc11caa9640 1 -- 172.21.15.67:0/1116274548 >> 172.21.15.67:0/1116274548 conn(0x7fc118079880 msgr2=0x7fc1181007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:14:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.699+0000 7fc11caa9640 1 -- 172.21.15.67:0/1116274548 shutdown_connections 2024-07-31T15:14:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:14:37.699+0000 7fc11caa9640 1 -- 172.21.15.67:0/1116274548 wait complete. 2024-07-31T15:14:37.715 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-07-31T15:14:37.726 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-07-31T15:14:37.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: pgmap v1152: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 3.1 MiB/s wr, 473 op/s 2024-07-31T15:14:37.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000008171 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/power/supply) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000021f0 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/base/firmware_loader) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000006f7c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/phy) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:37.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:37.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000093bf (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/thermal/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000048c5 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/hwtracing/intel_th) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000002c0e (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/xilinx) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000002b48 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/firmware/arm_ffa) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:37 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000008a08 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/soc/microchip) see mds.b log and `damage ls` output for details 2024-07-31T15:14:37.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:37.572+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000283c [...27,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/crypto/intel/ auth{1=1,4=1} v1335 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:12:27.079595+0000 9=6+3) n(v5 rc2024-07-31T15:12:27.080595+0000 b917229 146=134+12) old_inodes=2 (inest sync->mix g=1,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:691247 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7daf8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000283c:[<0x10000002719/intel v642>,<0x10000001d8f/crypto v3871>,<0x10000000202/drivers v2561>,<0x10000000201/linux-6.5.11 v275>,<0x10000000006/t v183>,<0x10000000005/tmp v156>,<0x100000001fb/client.0 v155>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v140>,<0x10000000001/sv_1 v171>,<0x10000000000/qa v157>,<0x1/volumes v157>]//[]","memoryvalue":"(2)0x1000000283c:[<0x10000002719/intel v1335>,<0x10000001d8f/crypto v19023>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:14.708784+0000 5=2+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:12:27.080595+0000 b917229 146=134+12)","memory_value.dirstat":"f(v1 m2024-07-31T15:12:27.079595+0000 9=6+3)","memory_value.rstat":"n(v5 rc2024-07-31T15:12:27.080595+0000 b917229 146=134+12)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:37.893 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:37.605+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000054d1 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media/common/v4l2-tpg/ auth{4=1} v342 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:36.520626+0000 8=8+0) n(v3 rc2024-07-31T15:11:36.521626+0000 b175839 9=8+1) old_inodes=1 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:691911 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7d165080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000054d1:[<0x100000054a1/v4l2-tpg v130>,<0x10000005460/common v1799>,<0x10000001d8f/media v12887>,<0x10000000202/drivers v2910>,<0x10000000201/linux-6.5.11 v379>,<0x10000000006/t v275>,<0x10000000005/tmp v244>,<0x100000001fb/client.0 v243>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v221>,<0x10000000001/sv_1 v249>,<0x10000000000/qa v233>,<0x1/volumes v234>]//[]","memoryvalue":"(2)0x100000054d1:[<0x100000054a1/v4l2-tpg v342>,<0x10000005460/common v3457>,<0x10000001d8f/media v18772>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:12.842569+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:36.521626+0000 b175839 9=8+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:36.520626+0000 8=8+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:11:36.521626+0000 b175839 9=8+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: from='client.29818 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T15:14:38.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000283c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/crypto/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000054d1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media/common/v4l2-tpg) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:38.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: from='client.29818 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T15:14:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000283c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/crypto/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000054d1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media/common/v4l2-tpg) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:38.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:38 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: from='client.29818 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T15:14:38.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000001fc7 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/acpi/acpica) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000009462 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/hvc) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000005d80 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/misc/cardreader) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000283c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/crypto/intel) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000054d1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/media/common/v4l2-tpg) see mds.b log and `damage ls` output for details 2024-07-31T15:14:38.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:38.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:38.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:38 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.142 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:38.769+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000074a1 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/wireless/marvell/ auth{2=1} v1362 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:00.438362+0000 10=7+3) n(v6 rc2024-07-31T15:11:00.441362+0000 b1951078 100=96+4) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:685042 nref=3)) caps={15489=pAsLsXs/-@5} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8700c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000074a1:[<0x1000000708f/marvell v338>,<0x100000060b2/wireless v1686>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x100000074a1:[<0x1000000708f/marvell v1362>,<0x100000060b2/wireless v3767>,<0x10000001d8f/net v18833>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:33.434139+0000 6=3+3)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:00.441362+0000 b1951078 100=96+4)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:00.438362+0000 10=7+3)","memory_value.rstat":"n(v6 rc2024-07-31T15:11:00.441362+0000 b1951078 100=96+4)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:39.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:38.811+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000006b2f [...25,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/neterion/ auth{4=1} v2767 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:42.363507+0000 9=9+0) n(v4 rc2024-07-31T15:11:42.364507+0000 b310236 10=9+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:694118 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce81ca3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000006b2f:[<0x1000000622a/neterion v2003>,<0x100000060b2/ethernet v1684>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x10000006b2f:[<0x1000000622a/neterion v2767>,<0x100000060b2/ethernet v3787>,<0x10000001d8f/net v18833>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:28.646239+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:42.364507+0000 b310236 10=9+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:42.363507+0000 9=9+0)","memory_value.rstat":"n(v4 rc2024-07-31T15:11:42.364507+0000 b310236 10=9+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:39.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:38.814+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000006af6 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/microsoft/ auth{4=1} v2613 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:38.452587+0000 7=6+1) n(v5 rc2024-07-31T15:11:38.453587+0000 b144243 15=13+2) old_inodes=1 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:694128 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce81722680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000006af6:[<0x1000000622a/microsoft v2181>,<0x100000060b2/ethernet v1684>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x10000006af6:[<0x1000000622a/microsoft v2613>,<0x100000060b2/ethernet v3787>,<0x10000001d8f/net v18833>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:28.579240+0000 3=2+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:38.453587+0000 b144243 15=13+2)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:38.452587+0000 7=6+1)","memory_value.rstat":"n(v5 rc2024-07-31T15:11:38.453587+0000 b144243 15=13+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:39.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:38 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:38.825+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000006bc8 [...27,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/pensando/ auth{2=1} v2570 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:48.960373+0000 7=6+1) n(v6 rc2024-07-31T15:11:48.962373+0000 b410323 33=31+2) old_inodes=3 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:694155 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce80ef8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000006bc8:[<0x1000000622a/pensando v2175>,<0x100000060b2/ethernet v1684>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x10000006bc8:[<0x1000000622a/pensando v2570>,<0x100000060b2/ethernet v3787>,<0x10000001d8f/net v18833>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:29.021231+0000 3=2+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:48.962373+0000 b410323 33=31+2)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:48.960373+0000 7=6+1)","memory_value.rstat":"n(v6 rc2024-07-31T15:11:48.962373+0000 b410323 33=31+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:39.642 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:39.303+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000948f [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250/ auth{4=1} v716 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:07:35.607536+0000 75=75+0) n(v5 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:697144 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@11} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce836e4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000948f:[<0x1000000948d/8250 v533>,<0x1000000945c/serial v657>,<0x10000001d8f/tty v18356>,<0x10000000202/drivers v3826>,<0x10000000201/linux-6.5.11 v521>,<0x10000000006/t v396>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v360>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v340>,<0x10000000001/sv_1 v365>,<0x10000000000/qa v346>,<0x1/volumes v340>]//[]","memoryvalue":"(2)0x1000000948f:[<0x1000000948d/8250 v716>,<0x1000000945c/serial v1492>,<0x10000001d8f/tty v18901>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:57:33.063897+0000 47=47+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:07:35.607536+0000 75=75+0)","memory_value.rstat":"n(v5 rc2024-07-31T15:07:35.609536+0000 b1577733 76=75+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:39.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: from='client.20493 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T15:14:39.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: pgmap v1153: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 1.04k op/s 2024-07-31T15:14:39.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: Scrub error on inode 0x100000074a1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/wireless/marvell) see mds.b log and `damage ls` output for details 2024-07-31T15:14:39.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000006b2f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/neterion) see mds.b log and `damage ls` output for details 2024-07-31T15:14:39.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000006af6 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/microsoft) see mds.b log and `damage ls` output for details 2024-07-31T15:14:39.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:39.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000006bc8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/pensando) see mds.b log and `damage ls` output for details 2024-07-31T15:14:39.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: from='client.20493 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T15:14:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: pgmap v1153: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 1.04k op/s 2024-07-31T15:14:39.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: Scrub error on inode 0x100000074a1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/wireless/marvell) see mds.b log and `damage ls` output for details 2024-07-31T15:14:39.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000006b2f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/neterion) see mds.b log and `damage ls` output for details 2024-07-31T15:14:39.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000006af6 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/microsoft) see mds.b log and `damage ls` output for details 2024-07-31T15:14:39.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:39.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000006bc8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/pensando) see mds.b log and `damage ls` output for details 2024-07-31T15:14:39.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.748 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:39 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: from='client.20493 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-31T15:14:39.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: pgmap v1153: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 1.04k op/s 2024-07-31T15:14:39.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.814 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: Scrub error on inode 0x100000074a1 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/wireless/marvell) see mds.b log and `damage ls` output for details 2024-07-31T15:14:39.814 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.814 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.815 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.815 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.815 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.815 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.816 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.816 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000006b2f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/neterion) see mds.b log and `damage ls` output for details 2024-07-31T15:14:39.816 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.816 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.817 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000006af6 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/microsoft) see mds.b log and `damage ls` output for details 2024-07-31T15:14:39.817 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.817 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.817 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.818 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:39.818 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000006bc8 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/pensando) see mds.b log and `damage ls` output for details 2024-07-31T15:14:39.818 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:39.818 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:39 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.311 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:39.940+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000006803 [...23,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/marvell/octeontx2/ auth{2=1} v383 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:11:38.527585+0000 8=6+2) n(v5 rc2024-07-31T15:11:38.528585+0000 b1832909 74=71+3) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:702225 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce8265c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000006803:[<0x100000067e0/octeontx2 v159>,<0x1000000622a/marvell v2175>,<0x100000060b2/ethernet v1684>,<0x10000001d8f/net v15665>,<0x10000000202/drivers v3018>,<0x10000000201/linux-6.5.11 v410>,<0x10000000006/t v303>,<0x10000000005/tmp v274>,<0x100000001fb/client.0 v271>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v249>,<0x10000000001/sv_1 v275>,<0x10000000000/qa v258>,<0x1/volumes v258>]//[]","memoryvalue":"(2)0x10000006803:[<0x100000067e0/octeontx2 v383>,<0x1000000622a/marvell v2657>,<0x100000060b2/ethernet v3787>,<0x10000001d8f/net v18833>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:56:27.598261+0000 4=2+2)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:38.528585+0000 b1832909 74=71+3)","memory_value.dirstat":"f(v1 m2024-07-31T15:11:38.527585+0000 8=6+2)","memory_value.rstat":"n(v5 rc2024-07-31T15:11:38.528585+0000 b1832909 74=71+3)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:40.642 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:40.309+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000439c [...1d,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du/ auth{2=1} v303 ap=3 RANDEPHEMERALPIN f(v1 m2024-07-31T15:08:01.164016+0000 34=34+0) n(v3 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1) old_inodes=1 (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:703301 nref=3)) caps={15489=pAsLsXs/-@7} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce83f19600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000439c:[<0x10000004399/rcar-du v163>,<0x10000002d23/renesas v2988>,<0x10000002d21/drm v1196>,<0x10000001d8f/gpu v9523>,<0x10000000202/drivers v2799>,<0x10000000201/linux-6.5.11 v334>,<0x10000000006/t v232>,<0x10000000005/tmp v202>,<0x100000001fb/client.0 v201>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v184>,<0x10000000001/sv_1 v213>,<0x10000000000/qa v197>,<0x1/volumes v198>]//[]","memoryvalue":"(2)0x1000000439c:[<0x10000004399/rcar-du v303>,<0x10000002d23/renesas v3147>,<0x10000002d21/drm v3145>,<0x10000001d8f/gpu v19149>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T14:55:38.082295+0000 30=30+0)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1)","memory_value.dirstat":"f(v1 m2024-07-31T15:08:01.164016+0000 34=34+0)","memory_value.rstat":"n(v3 rc2024-07-31T15:08:01.165016+0000 b277631 35=34+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:14:40.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000006803 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/marvell/octeontx2) see mds.b log and `damage ls` output for details 2024-07-31T15:14:40.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:40.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:40 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:14:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000006803 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/marvell/octeontx2) see mds.b log and `damage ls` output for details 2024-07-31T15:14:40.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:40.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:40 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000948f (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/tty/serial/8250) see mds.b log and `damage ls` output for details 2024-07-31T15:14:40.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000006803 (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/net/ethernet/marvell/octeontx2) see mds.b log and `damage ls` output for details 2024-07-31T15:14:40.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:40.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:40.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:40 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.642 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:14:41.233+0000 7f2e9beab640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000003a1a [...25,head] /volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/i915/gt/uc/ auth{4=1} v637 ap=3 RANDEPHEMERALPIN f(v2 m2024-07-31T15:11:46.662419+0000 99=98+1) n(v5 rc2024-07-31T15:11:46.662419+0000 b2391460 107=105+2) old_inodes=2 (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:704343 nref=3)) caps={15489=pAsLsXsFs/pAsLsXsFs@22} mcw={4=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ce7f0e3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000003a1a:[<0x10000003975/uc v416>,<0x10000003833/gt v1435>,<0x10000002d23/i915 v1759>,<0x10000002d21/drm v303>,<0x10000001d8f/gpu v6343>,<0x10000000202/drivers v2617>,<0x10000000201/linux-6.5.11 v300>,<0x10000000006/t v206>,<0x10000000005/tmp v176>,<0x100000001fb/client.0 v176>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v156>,<0x10000000001/sv_1 v190>,<0x10000000000/qa v173>,<0x1/volumes v174>]//[]","memoryvalue":"(2)0x10000003a1a:[<0x10000003975/uc v637>,<0x10000003833/gt v3185>,<0x10000002d23/i915 v4641>,<0x10000002d21/drm v3145>,<0x10000001d8f/gpu v19149>,<0x10000000202/drivers v22490>,<0x10000000201/linux-6.5.11 v3108>,<0x10000000006/t v1911>,<0x10000000005/tmp v1831>,<0x100000001fb/client.0 v1798>,<0x100000001fa/67428b9b-e567-4ac7-8a12-63fc5983a051 v1760>,<0x10000000001/sv_1 v1749>,<0x10000000000/qa v1743>,<0x1/volumes v1681>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-31T15:11:44.949454+0000 91=90+1)","ondisk_value.rstat":"n(v0 rc2024-07-31T15:11:46.662419+0000 b2391460 107=105+2)","memory_value.dirstat":"f(v2 m2024-07-31T15:11:46.662419+0000 99=98+1)","memory_value.rstat":"n(v5 rc2024-07-31T15:11:46.662419+0000 b2391460 107=105+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-31T15:14:41.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: pgmap v1154: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 10 KiB/s wr, 1.50k op/s 2024-07-31T15:14:41.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.647 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:14:41.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.648 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.649 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.650 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.651 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.652 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.653 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.654 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:41.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:41 smithi073 ceph-mon[36714]: Scrub error on inode 0x10000003a1a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/i915/gt/uc) see mds.b log and `damage ls` output for details 2024-07-31T15:14:41.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: pgmap v1154: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 10 KiB/s wr, 1.50k op/s 2024-07-31T15:14:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:14:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.746 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:41.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.747 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:41 smithi077 ceph-mon[37327]: Scrub error on inode 0x10000003a1a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/i915/gt/uc) see mds.b log and `damage ls` output for details 2024-07-31T15:14:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: pgmap v1154: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 10 KiB/s wr, 1.50k op/s 2024-07-31T15:14:41.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: Scrub error on inode 0x1000000439c (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/renesas/rcar-du) see mds.b log and `damage ls` output for details 2024-07-31T15:14:41.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.812 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.813 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.814 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.814 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.814 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.814 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.814 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:41.814 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:41.815 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:41 smithi067 ceph-mon[32750]: Scrub error on inode 0x10000003a1a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/drivers/gpu/drm/i915/gt/uc) see mds.b log and `damage ls` output for details 2024-07-31T15:14:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.740 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.741 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.742 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.743 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.744 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:42.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.745 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:42 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:42.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:42 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.895 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.897 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.898 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.901 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.901 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.901 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:42.901 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:42.901 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:42 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:43.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:43 smithi077 ceph-mon[37327]: scrub summary: active paths [/] 2024-07-31T15:14:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:43 smithi077 ceph-mon[37327]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.739 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:43 smithi077 ceph-mon[37327]: pgmap v1155: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.2 KiB/s wr, 3.06k op/s 2024-07-31T15:14:43.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:43.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:43.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:43.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:43.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:43 smithi067 ceph-mon[32750]: scrub summary: active paths [/] 2024-07-31T15:14:43.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:43 smithi067 ceph-mon[32750]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:43 smithi067 ceph-mon[32750]: pgmap v1155: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.2 KiB/s wr, 3.06k op/s 2024-07-31T15:14:43.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:43.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:43.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:43.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:43.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:43 smithi073 ceph-mon[36714]: scrub summary: active paths [/] 2024-07-31T15:14:43.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:43 smithi073 ceph-mon[36714]: scrub summary: idle+waiting paths [/] 2024-07-31T15:14:43.894 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:43 smithi073 ceph-mon[36714]: pgmap v1155: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.2 KiB/s wr, 3.06k op/s 2024-07-31T15:14:45.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:45 smithi077 ceph-mon[37327]: pgmap v1156: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 KiB/s wr, 4.50k op/s 2024-07-31T15:14:45.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:45 smithi067 ceph-mon[32750]: pgmap v1156: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 KiB/s wr, 4.50k op/s 2024-07-31T15:14:45.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:45 smithi073 ceph-mon[36714]: pgmap v1156: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 KiB/s wr, 4.50k op/s 2024-07-31T15:14:47.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:47 smithi077 ceph-mon[37327]: pgmap v1157: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 KiB/s wr, 4.47k op/s 2024-07-31T15:14:47.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:47 smithi067 ceph-mon[32750]: pgmap v1157: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 KiB/s wr, 4.47k op/s 2024-07-31T15:14:47.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:47 smithi073 ceph-mon[36714]: pgmap v1157: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 KiB/s wr, 4.47k op/s 2024-07-31T15:14:49.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:49 smithi077 ceph-mon[37327]: pgmap v1158: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.8 KiB/s wr, 4.45k op/s 2024-07-31T15:14:49.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:49 smithi067 ceph-mon[32750]: pgmap v1158: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.8 KiB/s wr, 4.45k op/s 2024-07-31T15:14:49.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:49 smithi073 ceph-mon[36714]: pgmap v1158: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.8 KiB/s wr, 4.45k op/s 2024-07-31T15:14:51.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:51 smithi077 ceph-mon[37327]: pgmap v1159: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 KiB/s wr, 3.88k op/s 2024-07-31T15:14:51.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:51 smithi067 ceph-mon[32750]: pgmap v1159: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 KiB/s wr, 3.88k op/s 2024-07-31T15:14:51.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:51 smithi073 ceph-mon[36714]: pgmap v1159: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 KiB/s wr, 3.88k op/s 2024-07-31T15:14:53.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:53 smithi077 ceph-mon[37327]: pgmap v1160: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 KiB/s wr, 3.39k op/s 2024-07-31T15:14:53.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:53 smithi067 ceph-mon[32750]: pgmap v1160: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 KiB/s wr, 3.39k op/s 2024-07-31T15:14:53.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:53 smithi073 ceph-mon[36714]: pgmap v1160: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 KiB/s wr, 3.39k op/s 2024-07-31T15:14:55.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:55 smithi077 ceph-mon[37327]: pgmap v1161: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 KiB/s wr, 1.81k op/s 2024-07-31T15:14:55.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:55 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:14:55.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:55 smithi067 ceph-mon[32750]: pgmap v1161: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 KiB/s wr, 1.81k op/s 2024-07-31T15:14:55.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:55 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:14:55.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:55 smithi073 ceph-mon[36714]: pgmap v1161: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 KiB/s wr, 1.81k op/s 2024-07-31T15:14:55.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:55 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:14:56.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:56 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:14:56.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:56 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:14:56.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:56 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:14:56.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:56 smithi077 ceph-mon[37327]: scrub summary: idle 2024-07-31T15:14:56.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:56 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:14:56.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:56 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:14:56.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:56 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:14:56.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:56 smithi067 ceph-mon[32750]: scrub summary: idle 2024-07-31T15:14:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:56 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:14:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:56 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:14:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:56 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:14:56.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:56 smithi073 ceph-mon[36714]: scrub summary: idle 2024-07-31T15:14:57.705 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:57 smithi067 ceph-mon[32750]: pgmap v1162: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 4.5 KiB/s wr, 297 op/s 2024-07-31T15:14:57.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:57 smithi077 ceph-mon[37327]: pgmap v1162: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 4.5 KiB/s wr, 297 op/s 2024-07-31T15:14:57.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:57 smithi073 ceph-mon[36714]: pgmap v1162: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 4.5 KiB/s wr, 297 op/s 2024-07-31T15:14:59.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:14:59 smithi077 ceph-mon[37327]: pgmap v1163: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.2 KiB/s wr, 101 op/s 2024-07-31T15:14:59.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:14:59 smithi067 ceph-mon[32750]: pgmap v1163: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.2 KiB/s wr, 101 op/s 2024-07-31T15:14:59.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:14:59 smithi073 ceph-mon[36714]: pgmap v1163: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.2 KiB/s wr, 101 op/s 2024-07-31T15:15:00.385 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-31T15:15:00.654 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:15:01.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:01 smithi077 ceph-mon[37327]: pgmap v1164: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-07-31T15:15:01.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:01 smithi067 ceph-mon[32750]: pgmap v1164: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-07-31T15:15:01.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:01 smithi073 ceph-mon[36714]: pgmap v1164: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-07-31T15:15:02.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.460+0000 7f3488e9e640 1 -- 172.21.15.67:0/2234623513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3484107de0 msgr2=0x7f34841081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:02.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.460+0000 7f3488e9e640 1 --2- 172.21.15.67:0/2234623513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3484107de0 0x7f34841081f0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f3478009900 tx=0x7f347802a410 comp rx=0 tx=0).stop 2024-07-31T15:15:02.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.460+0000 7f3488e9e640 1 -- 172.21.15.67:0/2234623513 shutdown_connections 2024-07-31T15:15:02.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.460+0000 7f3488e9e640 1 --2- 172.21.15.67:0/2234623513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3484108730 0x7f348410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.460+0000 7f3488e9e640 1 --2- 172.21.15.67:0/2234623513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3484107de0 0x7f34841081f0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.460+0000 7f3488e9e640 1 --2- 172.21.15.67:0/2234623513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34841024a0 0x7f34841028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.460+0000 7f3488e9e640 1 -- 172.21.15.67:0/2234623513 >> 172.21.15.67:0/2234623513 conn(0x7f3484079140 msgr2=0x7f3484079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.460+0000 7f3488e9e640 1 -- 172.21.15.67:0/2234623513 shutdown_connections 2024-07-31T15:15:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.460+0000 7f3488e9e640 1 -- 172.21.15.67:0/2234623513 wait complete. 2024-07-31T15:15:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.461+0000 7f3488e9e640 1 Processor -- start 2024-07-31T15:15:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.461+0000 7f3488e9e640 1 -- start start 2024-07-31T15:15:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.461+0000 7f3488e9e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34841024a0 0x7f3484198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.461+0000 7f3488e9e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3484107de0 0x7f3484199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.461+0000 7f3488e9e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3484108730 0x7f348419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.461+0000 7f3488e9e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f34841123a0 con 0x7f34841024a0 2024-07-31T15:15:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.461+0000 7f3488e9e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3484112220 con 0x7f3484108730 2024-07-31T15:15:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.461+0000 7f3488e9e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3484112520 con 0x7f3484107de0 2024-07-31T15:15:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.461+0000 7f348259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34841024a0 0x7f3484198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.461+0000 7f348259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34841024a0 0x7f3484198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58904/0 (socket says 172.21.15.67:58904) 2024-07-31T15:15:02.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.461+0000 7f348259c640 1 -- 172.21.15.67:0/4228199457 learned_addr learned my addr 172.21.15.67:0/4228199457 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:02.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.462+0000 7f3481d9b640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3484107de0 0x7f3484199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.462+0000 7f348259c640 1 -- 172.21.15.67:0/4228199457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3484107de0 msgr2=0x7f3484199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:02.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.462+0000 7f348259c640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3484107de0 0x7f3484199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.462+0000 7f348259c640 1 -- 172.21.15.67:0/4228199457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3484108730 msgr2=0x7f348419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:02.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.462+0000 7f3482d9d640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3484108730 0x7f348419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.462+0000 7f348259c640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3484108730 0x7f348419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.462+0000 7f348259c640 1 -- 172.21.15.67:0/4228199457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348419ed40 con 0x7f34841024a0 2024-07-31T15:15:02.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.462+0000 7f3481d9b640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3484107de0 0x7f3484199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:15:02.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.462+0000 7f348259c640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34841024a0 0x7f3484198e90 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f347400d950 tx=0x7f347400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:02.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.462+0000 7f34737fe640 1 -- 172.21.15.67:0/4228199457 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474017070 con 0x7f34841024a0 2024-07-31T15:15:02.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.462+0000 7f34737fe640 1 -- 172.21.15.67:0/4228199457 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3474009c30 con 0x7f34841024a0 2024-07-31T15:15:02.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.462+0000 7f34737fe640 1 -- 172.21.15.67:0/4228199457 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347400f620 con 0x7f34841024a0 2024-07-31T15:15:02.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.462+0000 7f3488e9e640 1 -- 172.21.15.67:0/4228199457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348419f030 con 0x7f34841024a0 2024-07-31T15:15:02.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.462+0000 7f3488e9e640 1 -- 172.21.15.67:0/4228199457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34841a5850 con 0x7f34841024a0 2024-07-31T15:15:02.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.465+0000 7f348f3ef740 1 Processor -- start 2024-07-31T15:15:02.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.465+0000 7f34737fe640 1 -- 172.21.15.67:0/4228199457 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3474010460 con 0x7f34841024a0 2024-07-31T15:15:02.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.465+0000 7f348f3ef740 1 -- start start 2024-07-31T15:15:02.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.465+0000 7f348f3ef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0ee070 0x56457a493670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.465+0000 7f348f3ef740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0c51b0 0x56457a493bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.465+0000 7f348f3ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0cd390 0x56457a4940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.465+0000 7f348f3ef740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56457a409b90 con 0x56457a0c51b0 2024-07-31T15:15:02.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.465+0000 7f348f3ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56457a3f9090 con 0x56457a0cd390 2024-07-31T15:15:02.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.465+0000 7f348f3ef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56457a3fd0d0 con 0x56457a0ee070 2024-07-31T15:15:02.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.466+0000 7f348259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0c51b0 0x56457a493bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.466+0000 7f348259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0c51b0 0x56457a493bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58916/0 (socket says 172.21.15.67:58916) 2024-07-31T15:15:02.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.466+0000 7f348259c640 1 -- 172.21.15.67:0/1620179629 learned_addr learned my addr 172.21.15.67:0/1620179629 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:02.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.466+0000 7f3481d9b640 1 --2- 172.21.15.67:0/1620179629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0cd390 0x56457a4940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.466+0000 7f3482d9d640 1 --2- 172.21.15.67:0/1620179629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0ee070 0x56457a493670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.466+0000 7f34737fe640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f346c0b3d70 0x7f346c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.466+0000 7f348259c640 1 -- 172.21.15.67:0/1620179629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0ee070 msgr2=0x56457a493670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:02.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.466+0000 7f348259c640 1 --2- 172.21.15.67:0/1620179629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0ee070 0x56457a493670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.466+0000 7f348259c640 1 -- 172.21.15.67:0/1620179629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0cd390 msgr2=0x56457a4940f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:02.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.466+0000 7f348259c640 1 --2- 172.21.15.67:0/1620179629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0cd390 0x56457a4940f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.466+0000 7f348259c640 1 -- 172.21.15.67:0/1620179629 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56457a39fb70 con 0x56457a0c51b0 2024-07-31T15:15:02.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.466+0000 7f3482d9d640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f346c0b3d70 0x7f346c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.466+0000 7f348259c640 1 --2- 172.21.15.67:0/1620179629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0c51b0 0x56457a493bb0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f34740e1360 tx=0x7f34740bc1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c7305d59543bee server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:02.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.466+0000 7f34737fe640 1 -- 172.21.15.67:0/4228199457 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f34740e4010 con 0x7f34841024a0 2024-07-31T15:15:02.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.467+0000 7f3470ff9640 1 -- 172.21.15.67:0/1620179629 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474020020 con 0x56457a0c51b0 2024-07-31T15:15:02.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.467+0000 7f3470ff9640 1 -- 172.21.15.67:0/1620179629 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34740b32a0 con 0x56457a0c51b0 2024-07-31T15:15:02.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.467+0000 7f3482d9d640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f346c0b3d70 0x7f346c0b6280 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f347c005cc0 tx=0x7f347c004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:02.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.467+0000 7f3470ff9640 1 -- 172.21.15.67:0/1620179629 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34740aa200 con 0x56457a0c51b0 2024-07-31T15:15:02.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.468+0000 7f348f3ef740 1 -- 172.21.15.67:0/1620179629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0c51b0 msgr2=0x56457a493bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:02.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.468+0000 7f348f3ef740 1 --2- 172.21.15.67:0/1620179629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0c51b0 0x56457a493bb0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f34740e1360 tx=0x7f34740bc1b0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.468+0000 7f348f3ef740 1 -- 172.21.15.67:0/1620179629 shutdown_connections 2024-07-31T15:15:02.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.468+0000 7f348f3ef740 1 --2- 172.21.15.67:0/1620179629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0cd390 0x56457a4940f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.468+0000 7f348f3ef740 1 --2- 172.21.15.67:0/1620179629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0c51b0 0x56457a493bb0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.468+0000 7f348f3ef740 1 --2- 172.21.15.67:0/1620179629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0ee070 0x56457a493670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.468+0000 7f348f3ef740 1 -- 172.21.15.67:0/1620179629 >> 172.21.15.67:0/1620179629 conn(0x56457a0e2460 msgr2=0x56457a491560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:02.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.468+0000 7f348f3ef740 1 -- 172.21.15.67:0/1620179629 shutdown_connections 2024-07-31T15:15:02.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.468+0000 7f348f3ef740 1 -- 172.21.15.67:0/1620179629 wait complete. 2024-07-31T15:15:02.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.468+0000 7f348f3ef740 1 -- start start 2024-07-31T15:15:02.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.468+0000 7f348f3ef740 1 Processor -- start 2024-07-31T15:15:02.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f348f3ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0c51b0 0x56457a49f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f348f3ef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0cd390 0x56457a49f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f348f3ef740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0ee070 0x56457a4ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f348f3ef740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56457a25c700 con 0x56457a0ee070 2024-07-31T15:15:02.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f348f3ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56457a39b9c0 con 0x56457a0c51b0 2024-07-31T15:15:02.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f348f3ef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56457a4a3eb0 con 0x56457a0cd390 2024-07-31T15:15:02.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f348f3ef740 10 client.0 _ll_register_callbacks cb 0x56457a3b64d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:15:02.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f3482d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0ee070 0x56457a4ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f3482d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0ee070 0x56457a4ad1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58930/0 (socket says 172.21.15.67:58930) 2024-07-31T15:15:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f3482d9d640 1 -- 172.21.15.67:0/681951557 learned_addr learned my addr 172.21.15.67:0/681951557 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f348259c640 1 --2- 172.21.15.67:0/681951557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0c51b0 0x56457a49f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f3481d9b640 1 --2- 172.21.15.67:0/681951557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0cd390 0x56457a49f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f3482d9d640 1 -- 172.21.15.67:0/681951557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0cd390 msgr2=0x56457a49f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f3482d9d640 1 --2- 172.21.15.67:0/681951557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0cd390 0x56457a49f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f3482d9d640 1 -- 172.21.15.67:0/681951557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0c51b0 msgr2=0x56457a49f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f3482d9d640 1 --2- 172.21.15.67:0/681951557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0c51b0 0x56457a49f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f3482d9d640 1 -- 172.21.15.67:0/681951557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56457a159650 con 0x56457a0ee070 2024-07-31T15:15:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f3481d9b640 1 --2- 172.21.15.67:0/681951557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0cd390 0x56457a49f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:15:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f3482d9d640 1 --2- 172.21.15.67:0/681951557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0ee070 0x56457a4ad1f0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f347c01ce40 tx=0x7f347c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5072af3ff111401 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f3469ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:15:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f3469ffb640 1 -- 172.21.15.67:0/681951557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c029060 con 0x56457a0ee070 2024-07-31T15:15:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f3469ffb640 1 -- 172.21.15.67:0/681951557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f347c019620 con 0x56457a0ee070 2024-07-31T15:15:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.469+0000 7f348f3ef740 1 -- 172.21.15.67:0/681951557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56457a40bae0 con 0x56457a0ee070 2024-07-31T15:15:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.470+0000 7f3469ffb640 1 -- 172.21.15.67:0/681951557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f347c030050 con 0x56457a0ee070 2024-07-31T15:15:02.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.470+0000 7f348f3ef740 10 client.20505 fetch_fsmap learned FSMap version 66 2024-07-31T15:15:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.470+0000 7f348f3ef740 1 -- 172.21.15.67:0/681951557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56457a4b0040 con 0x56457a0ee070 2024-07-31T15:15:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.470+0000 7f3469ffb640 1 -- 172.21.15.67:0/681951557 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56457a40bae0 con 0x56457a0ee070 2024-07-31T15:15:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.470+0000 7f3469ffb640 1 -- 172.21.15.67:0/681951557 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f347c0247e0 con 0x56457a0ee070 2024-07-31T15:15:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.470+0000 7f348f3ef740 10 client.20505 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:15:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.470+0000 7f348f3ef740 10 client.20505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:15:02.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.470+0000 7f348f3ef740 20 client.20505 populate_metadata read hostname 'smithi067' 2024-07-31T15:15:02.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.470+0000 7f348f3ef740 1 --2- 172.21.15.67:0/681951557 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56457a0ac9f0 0x56457a4b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.470+0000 7f348f3ef740 4 client.20505 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:15:02.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.470+0000 7f348f3ef740 1 -- 172.21.15.67:0/681951557 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56457a407580 con 0x56457a0ac9f0 2024-07-31T15:15:02.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.470+0000 7f348259c640 1 --2- 172.21.15.67:0/681951557 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56457a0ac9f0 0x56457a4b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.471+0000 7f348259c640 1 --2- 172.21.15.67:0/681951557 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56457a0ac9f0 0x56457a4b35b0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a14c1bf09250a34 server_cookie=4d9e00cc4a1636d4 in_seq=0 out_seq=0 2024-07-31T15:15:02.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.471+0000 7f3469ffb640 10 client.20505 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:15:02.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f3469ffb640 1 -- 172.21.15.67:0/681951557 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56457a407580 con 0x56457a0ac9f0 2024-07-31T15:15:02.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f3469ffb640 10 client.20505 handle_command_reply: tid=0 2024-07-31T15:15:02.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f348f3ef740 1 client.20505 shutdown 2024-07-31T15:15:02.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f348f3ef740 1 -- 172.21.15.67:0/681951557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0ee070 msgr2=0x56457a4ad1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:02.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f348f3ef740 1 --2- 172.21.15.67:0/681951557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0ee070 0x56457a4ad1f0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f347c01ce40 tx=0x7f347c004ab0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f348f3ef740 1 -- 172.21.15.67:0/681951557 shutdown_connections 2024-07-31T15:15:02.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f348f3ef740 1 --2- 172.21.15.67:0/681951557 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56457a0ac9f0 0x56457a4b35b0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f348f3ef740 1 --2- 172.21.15.67:0/681951557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0ee070 0x56457a4ad1f0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f348f3ef740 1 --2- 172.21.15.67:0/681951557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0cd390 0x56457a49f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f3469ffb640 0 client.20505 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:15:02.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f348f3ef740 1 --2- 172.21.15.67:0/681951557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0c51b0 0x56457a49f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f348f3ef740 1 -- 172.21.15.67:0/681951557 >> 172.21.15.67:0/681951557 conn(0x56457a0e2460 msgr2=0x56457a48fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:02.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f348f3ef740 1 -- 172.21.15.67:0/681951557 shutdown_connections 2024-07-31T15:15:02.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f348f3ef740 1 -- 172.21.15.67:0/681951557 wait complete. 2024-07-31T15:15:02.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.472+0000 7f348f3ef740 20 client.20505 trim_cache size 0 max 16384 2024-07-31T15:15:02.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.490+0000 7f348f3ef740 1 Processor -- start 2024-07-31T15:15:02.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348f3ef740 1 -- start start 2024-07-31T15:15:02.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348f3ef740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0ac9f0 0x56457a4903f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348f3ef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0c51b0 0x56457a4988d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348f3ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0cd390 0x56457a498e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348f3ef740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f34841ae6a0 con 0x56457a0ac9f0 2024-07-31T15:15:02.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348f3ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f34841b01e0 con 0x56457a0cd390 2024-07-31T15:15:02.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348f3ef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f34841aed70 con 0x56457a0c51b0 2024-07-31T15:15:02.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0ac9f0 0x56457a4903f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0ac9f0 0x56457a4903f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58944/0 (socket says 172.21.15.67:58944) 2024-07-31T15:15:02.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348259c640 1 -- 172.21.15.67:0/3372122753 learned_addr learned my addr 172.21.15.67:0/3372122753 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:02.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f3481d9b640 1 --2- 172.21.15.67:0/3372122753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0c51b0 0x56457a4988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f3482d9d640 1 --2- 172.21.15.67:0/3372122753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0cd390 0x56457a498e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348259c640 1 -- 172.21.15.67:0/3372122753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0c51b0 msgr2=0x56457a4988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:02.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348259c640 1 --2- 172.21.15.67:0/3372122753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0c51b0 0x56457a4988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348259c640 1 -- 172.21.15.67:0/3372122753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0cd390 msgr2=0x56457a498e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:02.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348259c640 1 --2- 172.21.15.67:0/3372122753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0cd390 0x56457a498e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348259c640 1 -- 172.21.15.67:0/3372122753 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56457a499350 con 0x56457a0ac9f0 2024-07-31T15:15:02.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f3482d9d640 1 --2- 172.21.15.67:0/3372122753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0cd390 0x56457a498e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:15:02.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f3481d9b640 1 --2- 172.21.15.67:0/3372122753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0c51b0 0x56457a4988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:15:02.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.491+0000 7f348259c640 1 --2- 172.21.15.67:0/3372122753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0ac9f0 0x56457a4903f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f34740e14a0 tx=0x7f3474000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe6852b66cb22b7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:02.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.492+0000 7f346bfff640 1 -- 172.21.15.67:0/3372122753 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474020040 con 0x56457a0ac9f0 2024-07-31T15:15:02.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.492+0000 7f346bfff640 1 -- 172.21.15.67:0/3372122753 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34740aafb0 con 0x56457a0ac9f0 2024-07-31T15:15:02.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.492+0000 7f346bfff640 1 -- 172.21.15.67:0/3372122753 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34740aab10 con 0x56457a0ac9f0 2024-07-31T15:15:02.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.492+0000 7f348f3ef740 1 -- 172.21.15.67:0/3372122753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0ac9f0 msgr2=0x56457a4903f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:02.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.492+0000 7f348f3ef740 1 --2- 172.21.15.67:0/3372122753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0ac9f0 0x56457a4903f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f34740e14a0 tx=0x7f3474000bc0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.492+0000 7f348f3ef740 1 -- 172.21.15.67:0/3372122753 shutdown_connections 2024-07-31T15:15:02.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.492+0000 7f348f3ef740 1 --2- 172.21.15.67:0/3372122753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0cd390 0x56457a498e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.492+0000 7f348f3ef740 1 --2- 172.21.15.67:0/3372122753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0c51b0 0x56457a4988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.492+0000 7f348f3ef740 1 --2- 172.21.15.67:0/3372122753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0ac9f0 0x56457a4903f0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.492+0000 7f348f3ef740 1 -- 172.21.15.67:0/3372122753 >> 172.21.15.67:0/3372122753 conn(0x56457a0e2460 msgr2=0x56457a48fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:02.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.493+0000 7f348f3ef740 1 -- 172.21.15.67:0/3372122753 shutdown_connections 2024-07-31T15:15:02.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.493+0000 7f348f3ef740 1 -- 172.21.15.67:0/3372122753 wait complete. 2024-07-31T15:15:02.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.493+0000 7f348f3ef740 1 -- start start 2024-07-31T15:15:02.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.493+0000 7f348f3ef740 1 Processor -- start 2024-07-31T15:15:02.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.493+0000 7f348f3ef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0ac9f0 0x56457a4ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.493+0000 7f348f3ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0c51b0 0x56457a4acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.493+0000 7f348f3ef740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0cd390 0x56457a4a8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.493+0000 7f348f3ef740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f34841afb10 con 0x56457a0cd390 2024-07-31T15:15:02.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.493+0000 7f348f3ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f34841af440 con 0x56457a0c51b0 2024-07-31T15:15:02.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.493+0000 7f348f3ef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f34841ac6a0 con 0x56457a0ac9f0 2024-07-31T15:15:02.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.493+0000 7f348f3ef740 10 client.0 _ll_register_callbacks cb 0x7f347c00bc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:15:02.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.493+0000 7f3482d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0cd390 0x56457a4a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.494+0000 7f3482d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0cd390 0x56457a4a8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58956/0 (socket says 172.21.15.67:58956) 2024-07-31T15:15:02.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.494+0000 7f3482d9d640 1 -- 172.21.15.67:0/1467360418 learned_addr learned my addr 172.21.15.67:0/1467360418 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:02.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.494+0000 7f348259c640 1 --2- 172.21.15.67:0/1467360418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0ac9f0 0x56457a4ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.494+0000 7f3481d9b640 1 --2- 172.21.15.67:0/1467360418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0c51b0 0x56457a4acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.494+0000 7f3482d9d640 1 -- 172.21.15.67:0/1467360418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0ac9f0 msgr2=0x56457a4ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:02.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.494+0000 7f3482d9d640 1 --2- 172.21.15.67:0/1467360418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0ac9f0 0x56457a4ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.494+0000 7f3482d9d640 1 -- 172.21.15.67:0/1467360418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0c51b0 msgr2=0x56457a4acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:02.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.494+0000 7f3482d9d640 1 --2- 172.21.15.67:0/1467360418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0c51b0 0x56457a4acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.494+0000 7f3482d9d640 1 -- 172.21.15.67:0/1467360418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56457a4a8750 con 0x56457a0cd390 2024-07-31T15:15:02.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.494+0000 7f3482d9d640 1 --2- 172.21.15.67:0/1467360418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0cd390 0x56457a4a8190 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f347c00b730 tx=0x7f347c024690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b1ba75bfcb58a7 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:02.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.494+0000 7f3469ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:15:02.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.494+0000 7f3469ffb640 1 -- 172.21.15.67:0/1467360418 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c039060 con 0x56457a0cd390 2024-07-31T15:15:02.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.494+0000 7f3469ffb640 1 -- 172.21.15.67:0/1467360418 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f347c019820 con 0x56457a0cd390 2024-07-31T15:15:02.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.494+0000 7f348f3ef740 1 -- 172.21.15.67:0/1467360418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34740aafb0 con 0x56457a0cd390 2024-07-31T15:15:02.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.495+0000 7f3469ffb640 1 -- 172.21.15.67:0/1467360418 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f347c040050 con 0x56457a0cd390 2024-07-31T15:15:02.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.495+0000 7f348f3ef740 10 client.20511 fetch_fsmap learned FSMap version 66 2024-07-31T15:15:02.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.495+0000 7f348f3ef740 1 -- 172.21.15.67:0/1467360418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56457a4a9950 con 0x56457a0cd390 2024-07-31T15:15:02.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.495+0000 7f3469ffb640 1 -- 172.21.15.67:0/1467360418 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34740aafb0 con 0x56457a0cd390 2024-07-31T15:15:02.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.495+0000 7f3469ffb640 1 -- 172.21.15.67:0/1467360418 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f347c0345d0 con 0x56457a0cd390 2024-07-31T15:15:02.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.495+0000 7f348f3ef740 10 client.20511 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:15:02.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.495+0000 7f348f3ef740 10 client.20511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:15:02.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.495+0000 7f348f3ef740 20 client.20511 populate_metadata read hostname 'smithi067' 2024-07-31T15:15:02.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.495+0000 7f348f3ef740 1 --2- 172.21.15.67:0/1467360418 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56457a0ee070 0x56457a51dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:02.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.495+0000 7f348f3ef740 4 client.20511 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-31T15:15:02.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.495+0000 7f348f3ef740 1 -- 172.21.15.67:0/1467360418 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56457a404cc0 con 0x56457a0ee070 2024-07-31T15:15:02.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.495+0000 7f348259c640 1 --2- 172.21.15.67:0/1467360418 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56457a0ee070 0x56457a51dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:02.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f3469ffb640 10 client.20511 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:15:02.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f348259c640 1 --2- 172.21.15.67:0/1467360418 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56457a0ee070 0x56457a51dec0 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b34bb35310a81418 server_cookie=87b09c899929f2f9 in_seq=0 out_seq=0 2024-07-31T15:15:02.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f3469ffb640 1 -- 172.21.15.67:0/1467360418 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56457a404cc0 con 0x56457a0ee070 2024-07-31T15:15:02.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f3469ffb640 10 client.20511 handle_command_reply: tid=0 2024-07-31T15:15:02.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f348f3ef740 1 client.20511 shutdown 2024-07-31T15:15:02.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f348f3ef740 1 -- 172.21.15.67:0/1467360418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0cd390 msgr2=0x56457a4a8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:02.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f348f3ef740 1 --2- 172.21.15.67:0/1467360418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0cd390 0x56457a4a8190 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f347c00b730 tx=0x7f347c024690 comp rx=0 tx=0).stop 2024-07-31T15:15:02.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f348f3ef740 1 -- 172.21.15.67:0/1467360418 shutdown_connections 2024-07-31T15:15:02.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f348f3ef740 1 --2- 172.21.15.67:0/1467360418 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x56457a0ee070 0x56457a51dec0 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f348f3ef740 1 --2- 172.21.15.67:0/1467360418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56457a0cd390 0x56457a4a8190 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f348f3ef740 1 --2- 172.21.15.67:0/1467360418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56457a0c51b0 0x56457a4acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f348f3ef740 1 --2- 172.21.15.67:0/1467360418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56457a0ac9f0 0x56457a4ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f3469ffb640 0 client.20511 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:15:02.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f348f3ef740 1 -- 172.21.15.67:0/1467360418 >> 172.21.15.67:0/1467360418 conn(0x56457a0e2460 msgr2=0x56457a4a05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:02.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f348f3ef740 1 -- 172.21.15.67:0/1467360418 shutdown_connections 2024-07-31T15:15:02.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f348f3ef740 1 -- 172.21.15.67:0/1467360418 wait complete. 2024-07-31T15:15:02.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.496+0000 7f348f3ef740 20 client.20511 trim_cache size 0 max 16384 2024-07-31T15:15:02.509 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:15:02.509 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-31T15:15:02.509 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-31T15:15:02.509 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:15:02.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.497+0000 7f3488e9e640 1 -- 172.21.15.67:0/4228199457 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f346c0b3d70 msgr2=0x7f346c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:02.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.497+0000 7f3488e9e640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f346c0b3d70 0x7f346c0b6280 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f347c005cc0 tx=0x7f347c004a40 comp rx=0 tx=0).stop 2024-07-31T15:15:02.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.497+0000 7f3488e9e640 1 -- 172.21.15.67:0/4228199457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34841024a0 msgr2=0x7f3484198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:02.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.497+0000 7f3488e9e640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34841024a0 0x7f3484198e90 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f347400d950 tx=0x7f347400de10 comp rx=0 tx=0).stop 2024-07-31T15:15:02.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.497+0000 7f3488e9e640 1 -- 172.21.15.67:0/4228199457 shutdown_connections 2024-07-31T15:15:02.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.497+0000 7f3488e9e640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f346c0b3d70 0x7f346c0b6280 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.497+0000 7f3488e9e640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3484108730 0x7f348419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.497+0000 7f3488e9e640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3484107de0 0x7f3484199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.497+0000 7f3488e9e640 1 --2- 172.21.15.67:0/4228199457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34841024a0 0x7f3484198e90 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:02.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.497+0000 7f3488e9e640 1 -- 172.21.15.67:0/4228199457 >> 172.21.15.67:0/4228199457 conn(0x7f3484079140 msgr2=0x7f3484105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:02.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.497+0000 7f3488e9e640 1 -- 172.21.15.67:0/4228199457 shutdown_connections 2024-07-31T15:15:02.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:02.497+0000 7f3488e9e640 1 -- 172.21.15.67:0/4228199457 wait complete. 2024-07-31T15:15:03.301 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-31T15:15:03.302 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:15:03.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:03 smithi067 ceph-mon[32750]: pgmap v1165: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-07-31T15:15:03.587 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:15:03.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:03 smithi077 ceph-mon[37327]: pgmap v1165: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-07-31T15:15:03.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:03 smithi073 ceph-mon[36714]: pgmap v1165: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-07-31T15:15:05.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.345+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2983360657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bbc109870 msgr2=0x7f4bbc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:05.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.345+0000 7f4bc16dc640 1 --2- 172.21.15.67:0/2983360657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bbc109870 0x7f4bbc1100a0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009900 tx=0x7f4bb402a410 comp rx=0 tx=0).stop 2024-07-31T15:15:05.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.345+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2983360657 shutdown_connections 2024-07-31T15:15:05.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.345+0000 7f4bc16dc640 1 --2- 172.21.15.67:0/2983360657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bbc109870 0x7f4bbc1100a0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:05.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.345+0000 7f4bc16dc640 1 --2- 172.21.15.67:0/2983360657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bbc108f20 0x7f4bbc109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:05.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.345+0000 7f4bc16dc640 1 --2- 172.21.15.67:0/2983360657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bbc1035e0 0x7f4bbc1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:05.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.345+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2983360657 >> 172.21.15.67:0/2983360657 conn(0x7f4bbc0ff030 msgr2=0x7f4bbc101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:05.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.345+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2983360657 shutdown_connections 2024-07-31T15:15:05.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.345+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2983360657 wait complete. 2024-07-31T15:15:05.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.346+0000 7f4bc16dc640 1 Processor -- start 2024-07-31T15:15:05.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.346+0000 7f4bc16dc640 1 -- start start 2024-07-31T15:15:05.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bc16dc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bbc1035e0 0x7f4bbc199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:05.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bc16dc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bbc108f20 0x7f4bbc19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:05.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bc16dc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bbc109870 0x7f4bbc19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:05.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bc16dc640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc113520 con 0x7f4bbc108f20 2024-07-31T15:15:05.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bc16dc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1133a0 con 0x7f4bbc1035e0 2024-07-31T15:15:05.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bc16dc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1136a0 con 0x7f4bbc109870 2024-07-31T15:15:05.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bba7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bbc108f20 0x7f4bbc19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:05.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bba7fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bbc108f20 0x7f4bbc19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58962/0 (socket says 172.21.15.67:58962) 2024-07-31T15:15:05.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bba7fc640 1 -- 172.21.15.67:0/2410316702 learned_addr learned my addr 172.21.15.67:0/2410316702 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:05.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bbaffd640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bbc1035e0 0x7f4bbc199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:05.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bbb7fe640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bbc109870 0x7f4bbc19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:05.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bba7fc640 1 -- 172.21.15.67:0/2410316702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bbc109870 msgr2=0x7f4bbc19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:05.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bba7fc640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bbc109870 0x7f4bbc19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:05.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bba7fc640 1 -- 172.21.15.67:0/2410316702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bbc1035e0 msgr2=0x7f4bbc199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:05.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bba7fc640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bbc1035e0 0x7f4bbc199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:05.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bba7fc640 1 -- 172.21.15.67:0/2410316702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bbc19fe80 con 0x7f4bbc108f20 2024-07-31T15:15:05.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bbaffd640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bbc1035e0 0x7f4bbc199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:15:05.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.347+0000 7f4bba7fc640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bbc108f20 0x7f4bbc19a4b0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000d740 tx=0x7f4bb000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:05.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.348+0000 7f4ba3fff640 1 -- 172.21.15.67:0/2410316702 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0017070 con 0x7f4bbc108f20 2024-07-31T15:15:05.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.348+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2410316702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bbc1a0170 con 0x7f4bbc108f20 2024-07-31T15:15:05.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.348+0000 7f4ba3fff640 1 -- 172.21.15.67:0/2410316702 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bb000de50 con 0x7f4bbc108f20 2024-07-31T15:15:05.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.348+0000 7f4ba3fff640 1 -- 172.21.15.67:0/2410316702 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000f620 con 0x7f4bbc108f20 2024-07-31T15:15:05.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.348+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2410316702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bbc1a68d0 con 0x7f4bbc108f20 2024-07-31T15:15:05.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.349+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2410316702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bbc1039f0 con 0x7f4bbc108f20 2024-07-31T15:15:05.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.350+0000 7f4ba3fff640 1 -- 172.21.15.67:0/2410316702 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4bb001a070 con 0x7f4bbc108f20 2024-07-31T15:15:05.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.351+0000 7f4ba3fff640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4ba40b3d40 0x7f4ba40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:05.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.351+0000 7f4bbaffd640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4ba40b3d40 0x7f4ba40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:05.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.351+0000 7f4ba3fff640 1 -- 172.21.15.67:0/2410316702 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7f4bb00e3cd0 con 0x7f4bbc108f20 2024-07-31T15:15:05.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.352+0000 7f4bbaffd640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4ba40b3d40 0x7f4ba40b6250 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009fd0 tx=0x7f4bac009760 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:05.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.354+0000 7f4ba3fff640 1 -- 172.21.15.67:0/2410316702 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f4bb00aac70 con 0x7f4bbc108f20 2024-07-31T15:15:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.587+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2410316702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bbc19b040 con 0x7f4bbc108f20 2024-07-31T15:15:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.588+0000 7f4ba3fff640 1 -- 172.21.15.67:0/2410316702 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11959 (secure 0 0 0) 0x7f4bbc19b040 con 0x7f4bbc108f20 2024-07-31T15:15:05.590 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:15:05.590 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":66,"btime":"2024-07-31T15:10:30:235518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:10:30.235494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29129":{"gid":29129,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/631186801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":631186801},{"type":"v1","addr":"172.21.15.73:6835","nonce":631186801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:15:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 66 2024-07-31T15:15:05.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.590+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2410316702 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4ba40b3d40 msgr2=0x7f4ba40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:05.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.590+0000 7f4bc16dc640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4ba40b3d40 0x7f4ba40b6250 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009fd0 tx=0x7f4bac009760 comp rx=0 tx=0).stop 2024-07-31T15:15:05.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.590+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2410316702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bbc108f20 msgr2=0x7f4bbc19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:05.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.590+0000 7f4bc16dc640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bbc108f20 0x7f4bbc19a4b0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000d740 tx=0x7f4bb000dc00 comp rx=0 tx=0).stop 2024-07-31T15:15:05.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.590+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2410316702 shutdown_connections 2024-07-31T15:15:05.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.590+0000 7f4bc16dc640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f4ba40b3d40 0x7f4ba40b6250 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.590+0000 7f4bc16dc640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bbc109870 0x7f4bbc19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.590+0000 7f4bc16dc640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bbc108f20 0x7f4bbc19a4b0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.590+0000 7f4bc16dc640 1 --2- 172.21.15.67:0/2410316702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bbc1035e0 0x7f4bbc199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.590+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2410316702 >> 172.21.15.67:0/2410316702 conn(0x7f4bbc0ff030 msgr2=0x7f4bbc101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.590+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2410316702 shutdown_connections 2024-07-31T15:15:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:05.590+0000 7f4bc16dc640 1 -- 172.21.15.67:0/2410316702 wait complete. 2024-07-31T15:15:05.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:05 smithi077 ceph-mon[37327]: pgmap v1166: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-07-31T15:15:05.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:05 smithi067 ceph-mon[32750]: pgmap v1166: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-07-31T15:15:05.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:05 smithi073 ceph-mon[36714]: pgmap v1166: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-07-31T15:15:06.278 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-31T15:15:06.546 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:15:06.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:06 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2410316702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:15:06.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:06 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2410316702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:15:06.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:06 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2410316702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:15:07.724 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:07 smithi067 ceph-mon[32750]: pgmap v1167: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 26 op/s 2024-07-31T15:15:07.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:07 smithi077 ceph-mon[37327]: pgmap v1167: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 26 op/s 2024-07-31T15:15:07.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:07 smithi073 ceph-mon[36714]: pgmap v1167: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 26 op/s 2024-07-31T15:15:08.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.315+0000 7fbe60faf640 1 -- 172.21.15.67:0/2670680456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe5c107e10 msgr2=0x7fbe5c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:08.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.315+0000 7fbe60faf640 1 --2- 172.21.15.67:0/2670680456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe5c107e10 0x7fbe5c108220 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009900 tx=0x7fbe5002a410 comp rx=0 tx=0).stop 2024-07-31T15:15:08.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.315+0000 7fbe60faf640 1 -- 172.21.15.67:0/2670680456 shutdown_connections 2024-07-31T15:15:08.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.315+0000 7fbe60faf640 1 --2- 172.21.15.67:0/2670680456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c108760 0x7fbe5c10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.315+0000 7fbe60faf640 1 --2- 172.21.15.67:0/2670680456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe5c107e10 0x7fbe5c108220 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.315+0000 7fbe60faf640 1 --2- 172.21.15.67:0/2670680456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe5c1024d0 0x7fbe5c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.315+0000 7fbe60faf640 1 -- 172.21.15.67:0/2670680456 >> 172.21.15.67:0/2670680456 conn(0x7fbe5c079190 msgr2=0x7fbe5c0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:08.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.315+0000 7fbe60faf640 1 -- 172.21.15.67:0/2670680456 shutdown_connections 2024-07-31T15:15:08.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.315+0000 7fbe60faf640 1 -- 172.21.15.67:0/2670680456 wait complete. 2024-07-31T15:15:08.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.316+0000 7fbe60faf640 1 Processor -- start 2024-07-31T15:15:08.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.316+0000 7fbe60faf640 1 -- start start 2024-07-31T15:15:08.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.316+0000 7fbe60faf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c1024d0 0x7fbe5c10fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe60faf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe5c107e10 0x7fbe5c10ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe60faf640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe5c108760 0x7fbe5c1104a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe60faf640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c1134f0 con 0x7fbe5c108760 2024-07-31T15:15:08.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe60faf640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c113370 con 0x7fbe5c1024d0 2024-07-31T15:15:08.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe60faf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c113670 con 0x7fbe5c107e10 2024-07-31T15:15:08.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe5ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe5c108760 0x7fbe5c1104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe5ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe5c108760 0x7fbe5c1104a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58986/0 (socket says 172.21.15.67:58986) 2024-07-31T15:15:08.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe5ad9d640 1 -- 172.21.15.67:0/218567509 learned_addr learned my addr 172.21.15.67:0/218567509 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:08.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe59d9b640 1 --2- 172.21.15.67:0/218567509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe5c107e10 0x7fbe5c10ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe5a59c640 1 --2- 172.21.15.67:0/218567509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c1024d0 0x7fbe5c10fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe5ad9d640 1 -- 172.21.15.67:0/218567509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe5c107e10 msgr2=0x7fbe5c10ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:08.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe5ad9d640 1 --2- 172.21.15.67:0/218567509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe5c107e10 0x7fbe5c10ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe5ad9d640 1 -- 172.21.15.67:0/218567509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c1024d0 msgr2=0x7fbe5c10fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:08.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe5ad9d640 1 --2- 172.21.15.67:0/218567509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c1024d0 0x7fbe5c10fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe5ad9d640 1 -- 172.21.15.67:0/218567509 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c1a80b0 con 0x7fbe5c108760 2024-07-31T15:15:08.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.317+0000 7fbe5ad9d640 1 --2- 172.21.15.67:0/218567509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe5c108760 0x7fbe5c1104a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400eaa0 tx=0x7fbe5400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:08.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.318+0000 7fbe4b7fe640 1 -- 172.21.15.67:0/218567509 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54004d20 con 0x7fbe5c108760 2024-07-31T15:15:08.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.318+0000 7fbe60faf640 1 -- 172.21.15.67:0/218567509 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5c1a83a0 con 0x7fbe5c108760 2024-07-31T15:15:08.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.318+0000 7fbe4b7fe640 1 -- 172.21.15.67:0/218567509 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe54005020 con 0x7fbe5c108760 2024-07-31T15:15:08.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.318+0000 7fbe4b7fe640 1 -- 172.21.15.67:0/218567509 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540116b0 con 0x7fbe5c108760 2024-07-31T15:15:08.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.318+0000 7fbe60faf640 1 -- 172.21.15.67:0/218567509 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe5c1a8860 con 0x7fbe5c108760 2024-07-31T15:15:08.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.320+0000 7fbe67500740 1 Processor -- start 2024-07-31T15:15:08.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.320+0000 7fbe4b7fe640 1 -- 172.21.15.67:0/218567509 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbe540051d0 con 0x7fbe5c108760 2024-07-31T15:15:08.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe67500740 1 -- start start 2024-07-31T15:15:08.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe67500740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 0x5639f3398570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe67500740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fca1b0 0x5639f3398ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe67500740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fd2390 0x5639f3398ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe67500740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5639f330ebb0 con 0x5639f2fd2390 2024-07-31T15:15:08.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe67500740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5639f32fe0b0 con 0x5639f2ff3070 2024-07-31T15:15:08.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe67500740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5639f33020f0 con 0x5639f2fca1b0 2024-07-31T15:15:08.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe59d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fd2390 0x5639f3398ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe5a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fca1b0 0x5639f3398ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe59d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fd2390 0x5639f3398ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58990/0 (socket says 172.21.15.67:58990) 2024-07-31T15:15:08.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe59d9b640 1 -- 172.21.15.67:0/819845792 learned_addr learned my addr 172.21.15.67:0/819845792 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:08.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe59d9b640 1 --2- 172.21.15.67:0/819845792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 0x5639f3398570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe5a59c640 1 -- 172.21.15.67:0/819845792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 msgr2=0x5639f3398570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:08.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe5a59c640 1 --2- 172.21.15.67:0/819845792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 0x5639f3398570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe5a59c640 1 -- 172.21.15.67:0/819845792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fd2390 msgr2=0x5639f3398ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:08.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.321+0000 7fbe5a59c640 1 --2- 172.21.15.67:0/819845792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fd2390 0x5639f3398ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.322+0000 7fbe5a59c640 1 -- 172.21.15.67:0/819845792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f32a4b90 con 0x5639f2fca1b0 2024-07-31T15:15:08.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.322+0000 7fbe59d9b640 1 --2- 172.21.15.67:0/819845792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 0x5639f3398570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:15:08.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.322+0000 7fbe4b7fe640 1 --2- 172.21.15.67:0/218567509 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbe440b3d40 0x7fbe440b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.322+0000 7fbe59d9b640 1 --2- 172.21.15.67:0/218567509 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbe440b3d40 0x7fbe440b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.322+0000 7fbe5a59c640 1 --2- 172.21.15.67:0/819845792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fca1b0 0x5639f3398ab0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00aba0 tx=0x7fbe4c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=942572bbfc8bc1d9 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:08.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.322+0000 7fbe4b7fe640 1 -- 172.21.15.67:0/218567509 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7fbe540e4540 con 0x7fbe5c108760 2024-07-31T15:15:08.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.322+0000 7fbe48ff9640 1 -- 172.21.15.67:0/819845792 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c005220 con 0x5639f2fca1b0 2024-07-31T15:15:08.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.322+0000 7fbe48ff9640 1 -- 172.21.15.67:0/819845792 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe4c004c50 con 0x5639f2fca1b0 2024-07-31T15:15:08.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.322+0000 7fbe48ff9640 1 -- 172.21.15.67:0/819845792 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c018800 con 0x5639f2fca1b0 2024-07-31T15:15:08.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.323+0000 7fbe67500740 1 -- 172.21.15.67:0/819845792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fca1b0 msgr2=0x5639f3398ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:08.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.323+0000 7fbe67500740 1 --2- 172.21.15.67:0/819845792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fca1b0 0x5639f3398ab0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00aba0 tx=0x7fbe4c00f790 comp rx=0 tx=0).stop 2024-07-31T15:15:08.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.323+0000 7fbe59d9b640 1 --2- 172.21.15.67:0/218567509 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbe440b3d40 0x7fbe440b6250 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbe500096c0 tx=0x7fbe50041040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:08.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.323+0000 7fbe67500740 1 -- 172.21.15.67:0/819845792 shutdown_connections 2024-07-31T15:15:08.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.323+0000 7fbe67500740 1 --2- 172.21.15.67:0/819845792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fd2390 0x5639f3398ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.323+0000 7fbe67500740 1 --2- 172.21.15.67:0/819845792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fca1b0 0x5639f3398ab0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.323+0000 7fbe67500740 1 --2- 172.21.15.67:0/819845792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 0x5639f3398570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.323+0000 7fbe67500740 1 -- 172.21.15.67:0/819845792 >> 172.21.15.67:0/819845792 conn(0x5639f2fe7460 msgr2=0x5639f3396460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:08.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.323+0000 7fbe67500740 1 -- 172.21.15.67:0/819845792 shutdown_connections 2024-07-31T15:15:08.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.323+0000 7fbe67500740 1 -- 172.21.15.67:0/819845792 wait complete. 2024-07-31T15:15:08.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.324+0000 7fbe67500740 1 -- start start 2024-07-31T15:15:08.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.324+0000 7fbe67500740 1 Processor -- start 2024-07-31T15:15:08.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.324+0000 7fbe67500740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fca1b0 0x5639f33a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.324+0000 7fbe67500740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fd2390 0x5639f33a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.324+0000 7fbe67500740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 0x5639f33b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.324+0000 7fbe67500740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5639f3161700 con 0x5639f2fd2390 2024-07-31T15:15:08.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.324+0000 7fbe67500740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5639f32a09e0 con 0x5639f2ff3070 2024-07-31T15:15:08.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.324+0000 7fbe67500740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5639f33a6d50 con 0x5639f2fca1b0 2024-07-31T15:15:08.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.324+0000 7fbe67500740 10 client.0 _ll_register_callbacks cb 0x5639f32bb4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:15:08.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.324+0000 7fbe59d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fd2390 0x5639f33a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.324+0000 7fbe59d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fd2390 0x5639f33a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59006/0 (socket says 172.21.15.67:59006) 2024-07-31T15:15:08.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.324+0000 7fbe59d9b640 1 -- 172.21.15.67:0/78115980 learned_addr learned my addr 172.21.15.67:0/78115980 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:08.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.324+0000 7fbe5a59c640 1 --2- 172.21.15.67:0/78115980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fca1b0 0x5639f33a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.325+0000 7fbe5ad9d640 1 --2- 172.21.15.67:0/78115980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 0x5639f33b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.325+0000 7fbe59d9b640 1 -- 172.21.15.67:0/78115980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fca1b0 msgr2=0x5639f33a40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:08.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.325+0000 7fbe59d9b640 1 --2- 172.21.15.67:0/78115980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fca1b0 0x5639f33a40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.325+0000 7fbe59d9b640 1 -- 172.21.15.67:0/78115980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 msgr2=0x5639f33b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:08.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.325+0000 7fbe59d9b640 1 --2- 172.21.15.67:0/78115980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 0x5639f33b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.325+0000 7fbe59d9b640 1 -- 172.21.15.67:0/78115980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f305e650 con 0x5639f2fd2390 2024-07-31T15:15:08.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.325+0000 7fbe59d9b640 1 --2- 172.21.15.67:0/78115980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fd2390 0x5639f33a45f0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fbe5002a930 tx=0x7fbe5002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f47903f49a177 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:08.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.325+0000 7fbe41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:15:08.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.325+0000 7fbe41ffb640 1 -- 172.21.15.67:0/78115980 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe50050070 con 0x5639f2fd2390 2024-07-31T15:15:08.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.325+0000 7fbe67500740 1 -- 172.21.15.67:0/78115980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe4c004c50 con 0x5639f2fd2390 2024-07-31T15:15:08.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.325+0000 7fbe41ffb640 1 -- 172.21.15.67:0/78115980 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7fbe500037f0 con 0x5639f2fd2390 2024-07-31T15:15:08.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.325+0000 7fbe41ffb640 1 -- 172.21.15.67:0/78115980 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fbe5005e050 con 0x5639f2fd2390 2024-07-31T15:15:08.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.325+0000 7fbe67500740 10 client.20529 fetch_fsmap learned FSMap version 66 2024-07-31T15:15:08.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.325+0000 7fbe67500740 1 -- 172.21.15.67:0/78115980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5639f33b2d10 con 0x5639f2fd2390 2024-07-31T15:15:08.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.326+0000 7fbe41ffb640 1 -- 172.21.15.67:0/78115980 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe4c004c50 con 0x5639f2fd2390 2024-07-31T15:15:08.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.326+0000 7fbe41ffb640 1 -- 172.21.15.67:0/78115980 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fbe50055a80 con 0x5639f2fd2390 2024-07-31T15:15:08.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.326+0000 7fbe67500740 10 client.20529 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:15:08.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.326+0000 7fbe67500740 10 client.20529 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:15:08.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.326+0000 7fbe67500740 20 client.20529 populate_metadata read hostname 'smithi067' 2024-07-31T15:15:08.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.326+0000 7fbe67500740 1 --2- 172.21.15.67:0/78115980 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5639f2fb19f0 0x5639f33b6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.326+0000 7fbe67500740 4 client.20529 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:15:08.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.326+0000 7fbe67500740 1 -- 172.21.15.67:0/78115980 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639f3310b00 con 0x5639f2fb19f0 2024-07-31T15:15:08.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.326+0000 7fbe5a59c640 1 --2- 172.21.15.67:0/78115980 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5639f2fb19f0 0x5639f33b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.327+0000 7fbe41ffb640 10 client.20529 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:15:08.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.327+0000 7fbe5a59c640 1 --2- 172.21.15.67:0/78115980 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5639f2fb19f0 0x5639f33b6230 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3e5d8a97e5212ce server_cookie=72f943d149c38d3d in_seq=0 out_seq=0 2024-07-31T15:15:08.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.327+0000 7fbe41ffb640 1 -- 172.21.15.67:0/78115980 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639f3310b00 con 0x5639f2fb19f0 2024-07-31T15:15:08.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.327+0000 7fbe41ffb640 10 client.20529 handle_command_reply: tid=0 2024-07-31T15:15:08.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.327+0000 7fbe67500740 1 client.20529 shutdown 2024-07-31T15:15:08.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.327+0000 7fbe67500740 1 -- 172.21.15.67:0/78115980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fd2390 msgr2=0x5639f33a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:08.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.327+0000 7fbe67500740 1 --2- 172.21.15.67:0/78115980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fd2390 0x5639f33a45f0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fbe5002a930 tx=0x7fbe5002a410 comp rx=0 tx=0).stop 2024-07-31T15:15:08.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.327+0000 7fbe67500740 1 -- 172.21.15.67:0/78115980 shutdown_connections 2024-07-31T15:15:08.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.328+0000 7fbe67500740 1 --2- 172.21.15.67:0/78115980 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5639f2fb19f0 0x5639f33b6230 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.328+0000 7fbe41ffb640 0 client.20529 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:15:08.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.328+0000 7fbe67500740 1 --2- 172.21.15.67:0/78115980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 0x5639f33b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.328+0000 7fbe67500740 1 --2- 172.21.15.67:0/78115980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fd2390 0x5639f33a45f0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.328+0000 7fbe67500740 1 --2- 172.21.15.67:0/78115980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fca1b0 0x5639f33a40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.328+0000 7fbe67500740 1 -- 172.21.15.67:0/78115980 >> 172.21.15.67:0/78115980 conn(0x5639f2fe7460 msgr2=0x5639f339b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:08.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.328+0000 7fbe67500740 1 -- 172.21.15.67:0/78115980 shutdown_connections 2024-07-31T15:15:08.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.328+0000 7fbe67500740 1 -- 172.21.15.67:0/78115980 wait complete. 2024-07-31T15:15:08.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.328+0000 7fbe67500740 20 client.20529 trim_cache size 0 max 16384 2024-07-31T15:15:08.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe67500740 1 Processor -- start 2024-07-31T15:15:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe67500740 1 -- start start 2024-07-31T15:15:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe67500740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fca1b0 0x5639f3396490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe67500740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fd2390 0x5639f33afe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe67500740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 0x5639f33b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe67500740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c1b2860 con 0x5639f2fca1b0 2024-07-31T15:15:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe67500740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c1b43a0 con 0x5639f2ff3070 2024-07-31T15:15:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe67500740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c1b2f30 con 0x5639f2fd2390 2024-07-31T15:15:08.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe5a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fca1b0 0x5639f3396490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe5a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fca1b0 0x5639f3396490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59018/0 (socket says 172.21.15.67:59018) 2024-07-31T15:15:08.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe5a59c640 1 -- 172.21.15.67:0/3083857116 learned_addr learned my addr 172.21.15.67:0/3083857116 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:08.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe59d9b640 1 --2- 172.21.15.67:0/3083857116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fd2390 0x5639f33afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe5ad9d640 1 --2- 172.21.15.67:0/3083857116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 0x5639f33b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe5a59c640 1 -- 172.21.15.67:0/3083857116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fd2390 msgr2=0x5639f33afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:08.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe5a59c640 1 --2- 172.21.15.67:0/3083857116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fd2390 0x5639f33afe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe5a59c640 1 -- 172.21.15.67:0/3083857116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 msgr2=0x5639f33b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:08.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.342+0000 7fbe5a59c640 1 --2- 172.21.15.67:0/3083857116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 0x5639f33b03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.343+0000 7fbe5a59c640 1 -- 172.21.15.67:0/3083857116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f339a860 con 0x5639f2fca1b0 2024-07-31T15:15:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.343+0000 7fbe5a59c640 1 --2- 172.21.15.67:0/3083857116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fca1b0 0x5639f3396490 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00fc90 tx=0x7fbe4c00a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80914d4584c937c server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.343+0000 7fbe43fff640 1 -- 172.21.15.67:0/3083857116 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c032070 con 0x5639f2fca1b0 2024-07-31T15:15:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.343+0000 7fbe43fff640 1 -- 172.21.15.67:0/3083857116 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe4c004bf0 con 0x5639f2fca1b0 2024-07-31T15:15:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.343+0000 7fbe43fff640 1 -- 172.21.15.67:0/3083857116 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0184d0 con 0x5639f2fca1b0 2024-07-31T15:15:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.344+0000 7fbe67500740 1 -- 172.21.15.67:0/3083857116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fca1b0 msgr2=0x5639f3396490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.344+0000 7fbe67500740 1 --2- 172.21.15.67:0/3083857116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fca1b0 0x5639f3396490 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00fc90 tx=0x7fbe4c00a530 comp rx=0 tx=0).stop 2024-07-31T15:15:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.344+0000 7fbe67500740 1 -- 172.21.15.67:0/3083857116 shutdown_connections 2024-07-31T15:15:08.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.344+0000 7fbe67500740 1 --2- 172.21.15.67:0/3083857116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2ff3070 0x5639f33b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.344+0000 7fbe67500740 1 --2- 172.21.15.67:0/3083857116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fd2390 0x5639f33afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.344+0000 7fbe67500740 1 --2- 172.21.15.67:0/3083857116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2fca1b0 0x5639f3396490 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.344+0000 7fbe67500740 1 -- 172.21.15.67:0/3083857116 >> 172.21.15.67:0/3083857116 conn(0x5639f2fe7460 msgr2=0x5639f3395d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:08.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.344+0000 7fbe67500740 1 -- 172.21.15.67:0/3083857116 shutdown_connections 2024-07-31T15:15:08.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.344+0000 7fbe67500740 1 -- 172.21.15.67:0/3083857116 wait complete. 2024-07-31T15:15:08.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.344+0000 7fbe67500740 1 -- start start 2024-07-31T15:15:08.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.344+0000 7fbe67500740 1 Processor -- start 2024-07-31T15:15:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe67500740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2fca1b0 0x5639f33ac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe67500740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fd2390 0x5639f33a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe67500740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2ff3070 0x5639f33a8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe67500740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c1b3cd0 con 0x5639f2ff3070 2024-07-31T15:15:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe67500740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c1b3600 con 0x5639f2fca1b0 2024-07-31T15:15:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe67500740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c1b0860 con 0x5639f2fd2390 2024-07-31T15:15:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe67500740 10 client.0 _ll_register_callbacks cb 0x7fbe50032d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:15:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe5ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2ff3070 0x5639f33a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe5ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2ff3070 0x5639f33a8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59032/0 (socket says 172.21.15.67:59032) 2024-07-31T15:15:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe5ad9d640 1 -- 172.21.15.67:0/3112538551 learned_addr learned my addr 172.21.15.67:0/3112538551 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe5a59c640 1 --2- 172.21.15.67:0/3112538551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2fca1b0 0x5639f33ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe59d9b640 1 --2- 172.21.15.67:0/3112538551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fd2390 0x5639f33a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe5ad9d640 1 -- 172.21.15.67:0/3112538551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fd2390 msgr2=0x5639f33a7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe5ad9d640 1 --2- 172.21.15.67:0/3112538551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fd2390 0x5639f33a7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe5ad9d640 1 -- 172.21.15.67:0/3112538551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2fca1b0 msgr2=0x5639f33ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe5ad9d640 1 --2- 172.21.15.67:0/3112538551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2fca1b0 0x5639f33ac650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe5ad9d640 1 -- 172.21.15.67:0/3112538551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f33a8780 con 0x5639f2ff3070 2024-07-31T15:15:08.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe5ad9d640 1 --2- 172.21.15.67:0/3112538551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2ff3070 0x5639f33a8040 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fbe540a8050 tx=0x7fbe540aa8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8dab8b2bbee64 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:08.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:15:08.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe41ffb640 1 -- 172.21.15.67:0/3112538551 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5401f040 con 0x5639f2ff3070 2024-07-31T15:15:08.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe41ffb640 1 -- 172.21.15.67:0/3112538551 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9395+0+0 (secure 0 0 0) 0x7fbe540e4300 con 0x5639f2ff3070 2024-07-31T15:15:08.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.345+0000 7fbe67500740 1 -- 172.21.15.67:0/3112538551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe4c004bf0 con 0x5639f2ff3070 2024-07-31T15:15:08.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.346+0000 7fbe41ffb640 1 -- 172.21.15.67:0/3112538551 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fbe540af3e0 con 0x5639f2ff3070 2024-07-31T15:15:08.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.346+0000 7fbe67500740 10 client.20535 fetch_fsmap learned FSMap version 66 2024-07-31T15:15:08.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.346+0000 7fbe67500740 1 -- 172.21.15.67:0/3112538551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5639f341e8d0 con 0x5639f2ff3070 2024-07-31T15:15:08.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.346+0000 7fbe41ffb640 1 -- 172.21.15.67:0/3112538551 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe4c004bf0 con 0x5639f2ff3070 2024-07-31T15:15:08.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.346+0000 7fbe41ffb640 1 -- 172.21.15.67:0/3112538551 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fbe5400f040 con 0x5639f2ff3070 2024-07-31T15:15:08.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.346+0000 7fbe67500740 10 client.20535 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:15:08.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.346+0000 7fbe67500740 10 client.20535 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-07-31T15:15:08.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.346+0000 7fbe67500740 20 client.20535 populate_metadata read hostname 'smithi067' 2024-07-31T15:15:08.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.346+0000 7fbe67500740 1 --2- 172.21.15.67:0/3112538551 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5639f2f86da0 0x5639f34210b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:08.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.346+0000 7fbe67500740 4 client.20535 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:15:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.346+0000 7fbe67500740 1 -- 172.21.15.67:0/3112538551 --> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5639f330c5a0 con 0x5639f2f86da0 2024-07-31T15:15:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.346+0000 7fbe5a59c640 1 --2- 172.21.15.67:0/3112538551 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5639f2f86da0 0x5639f34210b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe5a59c640 1 --2- 172.21.15.67:0/3112538551 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5639f2f86da0 0x5639f34210b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b62247e630b68425 server_cookie=30695b93e85373b in_seq=0 out_seq=0 2024-07-31T15:15:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe41ffb640 10 client.20535 ms_handle_connect on v2:172.21.15.77:6834/1866134372 2024-07-31T15:15:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe41ffb640 1 -- 172.21.15.67:0/3112538551 <== mds.4 v2:172.21.15.77:6834/1866134372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5639f330c5a0 con 0x5639f2f86da0 2024-07-31T15:15:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe41ffb640 10 client.20535 handle_command_reply: tid=0 2024-07-31T15:15:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe67500740 1 client.20535 shutdown 2024-07-31T15:15:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe67500740 1 -- 172.21.15.67:0/3112538551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2ff3070 msgr2=0x5639f33a8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe67500740 1 --2- 172.21.15.67:0/3112538551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2ff3070 0x5639f33a8040 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fbe540a8050 tx=0x7fbe540aa8f0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe67500740 1 -- 172.21.15.67:0/3112538551 shutdown_connections 2024-07-31T15:15:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe67500740 1 --2- 172.21.15.67:0/3112538551 >> [v2:172.21.15.77:6834/1866134372,v1:172.21.15.77:6835/1866134372] conn(0x5639f2f86da0 0x5639f34210b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe67500740 1 --2- 172.21.15.67:0/3112538551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5639f2ff3070 0x5639f33a8040 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe67500740 1 --2- 172.21.15.67:0/3112538551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639f2fd2390 0x5639f33a7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe67500740 1 --2- 172.21.15.67:0/3112538551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639f2fca1b0 0x5639f33ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe41ffb640 0 client.20535 ms_handle_reset on v2:172.21.15.77:6834/1866134372 2024-07-31T15:15:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe67500740 1 -- 172.21.15.67:0/3112538551 >> 172.21.15.67:0/3112538551 conn(0x5639f2fe7460 msgr2=0x5639f339b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:08.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe67500740 1 -- 172.21.15.67:0/3112538551 shutdown_connections 2024-07-31T15:15:08.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe67500740 1 -- 172.21.15.67:0/3112538551 wait complete. 2024-07-31T15:15:08.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.347+0000 7fbe67500740 20 client.20535 trim_cache size 0 max 16384 2024-07-31T15:15:08.358 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:15:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.348+0000 7fbe60faf640 1 -- 172.21.15.67:0/218567509 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbe440b3d40 msgr2=0x7fbe440b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.348+0000 7fbe60faf640 1 --2- 172.21.15.67:0/218567509 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbe440b3d40 0x7fbe440b6250 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbe500096c0 tx=0x7fbe50041040 comp rx=0 tx=0).stop 2024-07-31T15:15:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.348+0000 7fbe60faf640 1 -- 172.21.15.67:0/218567509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe5c108760 msgr2=0x7fbe5c1104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.348+0000 7fbe60faf640 1 --2- 172.21.15.67:0/218567509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe5c108760 0x7fbe5c1104a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400eaa0 tx=0x7fbe5400eea0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.348+0000 7fbe60faf640 1 -- 172.21.15.67:0/218567509 shutdown_connections 2024-07-31T15:15:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.348+0000 7fbe60faf640 1 --2- 172.21.15.67:0/218567509 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fbe440b3d40 0x7fbe440b6250 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.348+0000 7fbe60faf640 1 --2- 172.21.15.67:0/218567509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe5c108760 0x7fbe5c1104a0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.348+0000 7fbe60faf640 1 --2- 172.21.15.67:0/218567509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe5c107e10 0x7fbe5c10ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.348+0000 7fbe60faf640 1 --2- 172.21.15.67:0/218567509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c1024d0 0x7fbe5c10fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.348+0000 7fbe60faf640 1 -- 172.21.15.67:0/218567509 >> 172.21.15.67:0/218567509 conn(0x7fbe5c079190 msgr2=0x7fbe5c104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.348+0000 7fbe60faf640 1 -- 172.21.15.67:0/218567509 shutdown_connections 2024-07-31T15:15:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:08.348+0000 7fbe60faf640 1 -- 172.21.15.67:0/218567509 wait complete. 2024-07-31T15:15:09.063 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-31T15:15:09.329 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:15:09.714 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:09 smithi067 ceph-mon[32750]: pgmap v1168: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 50 KiB/s wr, 28 op/s 2024-07-31T15:15:09.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:09 smithi077 ceph-mon[37327]: pgmap v1168: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 50 KiB/s wr, 28 op/s 2024-07-31T15:15:09.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:09 smithi073 ceph-mon[36714]: pgmap v1168: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 50 KiB/s wr, 28 op/s 2024-07-31T15:15:11.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.184+0000 7f8ea307c640 1 -- 172.21.15.67:0/3888990146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c11c660 msgr2=0x7f8e9c11eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.184+0000 7f8ea307c640 1 --2- 172.21.15.67:0/3888990146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c11c660 0x7f8e9c11eb70 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009be0 tx=0x7f8e9402a6f0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.185+0000 7f8ea307c640 1 -- 172.21.15.67:0/3888990146 shutdown_connections 2024-07-31T15:15:11.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.185+0000 7f8ea307c640 1 --2- 172.21.15.67:0/3888990146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c11c660 0x7f8e9c11eb70 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.185+0000 7f8ea307c640 1 --2- 172.21.15.67:0/3888990146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c10a410 0x7f8e9c10a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.185+0000 7f8ea307c640 1 --2- 172.21.15.67:0/3888990146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c111970 0x7f8e9c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.185+0000 7f8ea307c640 1 -- 172.21.15.67:0/3888990146 >> 172.21.15.67:0/3888990146 conn(0x7f8e9c0708f0 msgr2=0x7f8e9c070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:11.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.188+0000 7f8ea307c640 1 -- 172.21.15.67:0/3888990146 shutdown_connections 2024-07-31T15:15:11.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.188+0000 7f8ea307c640 1 -- 172.21.15.67:0/3888990146 wait complete. 2024-07-31T15:15:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.189+0000 7f8ea307c640 1 Processor -- start 2024-07-31T15:15:11.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.190+0000 7f8ea307c640 1 -- start start 2024-07-31T15:15:11.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.190+0000 7f8ea307c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c10a410 0x7f8e9c1a8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.190+0000 7f8ea307c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c111970 0x7f8e9c1a9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.190+0000 7f8ea307c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c11c660 0x7f8e9c1ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.190+0000 7f8ea307c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c122340 con 0x7f8e9c10a410 2024-07-31T15:15:11.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.190+0000 7f8ea307c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c1221c0 con 0x7f8e9c11c660 2024-07-31T15:15:11.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.190+0000 7f8ea307c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c1224c0 con 0x7f8e9c111970 2024-07-31T15:15:11.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.191+0000 7f8ea1619640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c11c660 0x7f8e9c1ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:11.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.191+0000 7f8ea1619640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c11c660 0x7f8e9c1ae5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46848/0 (socket says 172.21.15.67:46848) 2024-07-31T15:15:11.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.191+0000 7f8ea1619640 1 -- 172.21.15.67:0/4044340589 learned_addr learned my addr 172.21.15.67:0/4044340589 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:11.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.191+0000 7f8ea1619640 1 -- 172.21.15.67:0/4044340589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c111970 msgr2=0x7f8e9c1a9310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-31T15:15:11.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.191+0000 7f8ea1619640 1 --2- 172.21.15.67:0/4044340589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c111970 0x7f8e9c1a9310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.191+0000 7f8ea1619640 1 -- 172.21.15.67:0/4044340589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c10a410 msgr2=0x7f8e9c1a8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:11.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.191+0000 7f8ea1619640 1 --2- 172.21.15.67:0/4044340589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c10a410 0x7f8e9c1a8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.192+0000 7f8ea1619640 1 -- 172.21.15.67:0/4044340589 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9c1aece0 con 0x7f8e9c11c660 2024-07-31T15:15:11.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.192+0000 7f8ea1619640 1 --2- 172.21.15.67:0/4044340589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c11c660 0x7f8e9c1ae5b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8e94003f10 tx=0x7f8e94003f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.195+0000 7f8e99ffb640 1 -- 172.21.15.67:0/4044340589 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94040020 con 0x7f8e9c11c660 2024-07-31T15:15:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.195+0000 7f8ea307c640 1 -- 172.21.15.67:0/4044340589 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9c1aef70 con 0x7f8e9c11c660 2024-07-31T15:15:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.196+0000 7f8ea307c640 1 -- 172.21.15.67:0/4044340589 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9c1b5730 con 0x7f8e9c11c660 2024-07-31T15:15:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.196+0000 7f8e99ffb640 1 -- 172.21.15.67:0/4044340589 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e94030970 con 0x7f8e9c11c660 2024-07-31T15:15:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.196+0000 7f8e99ffb640 1 -- 172.21.15.67:0/4044340589 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94036040 con 0x7f8e9c11c660 2024-07-31T15:15:11.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.205+0000 7f8ea95cd740 1 Processor -- start 2024-07-31T15:15:11.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.206+0000 7f8e99ffb640 1 -- 172.21.15.67:0/4044340589 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8e94043020 con 0x7f8e9c11c660 2024-07-31T15:15:11.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.207+0000 7f8e99ffb640 1 --2- 172.21.15.67:0/4044340589 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8e8c0b3f50 0x7f8e8c0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.208+0000 7f8ea95cd740 1 -- start start 2024-07-31T15:15:11.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.208+0000 7f8e99ffb640 1 -- 172.21.15.67:0/4044340589 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f8e941051a0 con 0x7f8e9c11c660 2024-07-31T15:15:11.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.208+0000 7f8ea95cd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da324070 0x5568da6c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.208+0000 7f8ea95cd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da2fb1b0 0x5568da6c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.208+0000 7f8ea95cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da303390 0x5568da6c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.208+0000 7f8ea95cd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5568da63fbb0 con 0x5568da2fb1b0 2024-07-31T15:15:11.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.208+0000 7f8ea95cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5568da62f0b0 con 0x5568da303390 2024-07-31T15:15:11.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.208+0000 7f8ea95cd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5568da6330f0 con 0x5568da324070 2024-07-31T15:15:11.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.209+0000 7f8ea0e18640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da324070 0x5568da6c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:11.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.209+0000 7f8ea0e18640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da324070 0x5568da6c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:51040/0 (socket says 172.21.15.67:51040) 2024-07-31T15:15:11.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.209+0000 7f8ea0e18640 1 -- 172.21.15.67:0/3011089598 learned_addr learned my addr 172.21.15.67:0/3011089598 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:11.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.209+0000 7f8ea1619640 1 --2- 172.21.15.67:0/3011089598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da303390 0x5568da6c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:11.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.209+0000 7f8ea0e18640 1 -- 172.21.15.67:0/3011089598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da303390 msgr2=0x5568da6c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:11.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.209+0000 7f8ea0e18640 1 --2- 172.21.15.67:0/3011089598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da303390 0x5568da6c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.209+0000 7f8ea0e18640 1 -- 172.21.15.67:0/3011089598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da2fb1b0 msgr2=0x5568da6c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:11.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.209+0000 7f8ea0e18640 1 --2- 172.21.15.67:0/3011089598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da2fb1b0 0x5568da6c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.209+0000 7f8ea0e18640 1 -- 172.21.15.67:0/3011089598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568da5d5b90 con 0x5568da324070 2024-07-31T15:15:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.212+0000 7f8ea0e18640 1 --2- 172.21.15.67:0/3011089598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da324070 0x5568da6c9570 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000bae0 tx=0x7f8e9000d2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ece2765ddc0c164b server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.214+0000 7f8e9bfff640 1 --2- 172.21.15.67:0/4044340589 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8e8c0b3f50 0x7f8e8c0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.215+0000 7f8e86ffd640 1 -- 172.21.15.67:0/3011089598 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90016070 con 0x5568da324070 2024-07-31T15:15:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.215+0000 7f8e86ffd640 1 -- 172.21.15.67:0/3011089598 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e900054d0 con 0x5568da324070 2024-07-31T15:15:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.215+0000 7f8e86ffd640 1 -- 172.21.15.67:0/3011089598 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90010460 con 0x5568da324070 2024-07-31T15:15:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.216+0000 7f8ea95cd740 1 -- 172.21.15.67:0/3011089598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da324070 msgr2=0x5568da6c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:11.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.216+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/3011089598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da324070 0x5568da6c9570 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000bae0 tx=0x7f8e9000d2e0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.217+0000 7f8e9bfff640 1 --2- 172.21.15.67:0/4044340589 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8e8c0b3f50 0x7f8e8c0b6460 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x5568da6c81f0 tx=0x7f8e88007ab0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:11.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.217+0000 7f8ea95cd740 1 -- 172.21.15.67:0/3011089598 shutdown_connections 2024-07-31T15:15:11.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.217+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/3011089598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da303390 0x5568da6c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.217+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/3011089598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da2fb1b0 0x5568da6c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.217+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/3011089598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da324070 0x5568da6c9570 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.217+0000 7f8ea95cd740 1 -- 172.21.15.67:0/3011089598 >> 172.21.15.67:0/3011089598 conn(0x5568da318460 msgr2=0x5568da6c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.217+0000 7f8ea95cd740 1 -- 172.21.15.67:0/3011089598 shutdown_connections 2024-07-31T15:15:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.217+0000 7f8ea95cd740 1 -- 172.21.15.67:0/3011089598 wait complete. 2024-07-31T15:15:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.218+0000 7f8ea95cd740 1 -- start start 2024-07-31T15:15:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.218+0000 7f8ea95cd740 1 Processor -- start 2024-07-31T15:15:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.219+0000 7f8ea95cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da2fb1b0 0x5568da6d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.219+0000 7f8ea95cd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da303390 0x5568da6d56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.219+0000 7f8ea95cd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da324070 0x5568da7030e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.219+0000 7f8ea95cd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5568da492700 con 0x5568da324070 2024-07-31T15:15:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.219+0000 7f8ea95cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5568da5d19e0 con 0x5568da2fb1b0 2024-07-31T15:15:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.219+0000 7f8ea95cd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5568da6f9d60 con 0x5568da303390 2024-07-31T15:15:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.219+0000 7f8ea95cd740 10 client.0 _ll_register_callbacks cb 0x5568da5ec4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:15:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.219+0000 7f8ea1619640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da324070 0x5568da7030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.219+0000 7f8ea1619640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da324070 0x5568da7030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44380/0 (socket says 172.21.15.67:44380) 2024-07-31T15:15:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.219+0000 7f8ea1619640 1 -- 172.21.15.67:0/1214758116 learned_addr learned my addr 172.21.15.67:0/1214758116 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:11.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.220+0000 7f8ea0e18640 1 --2- 172.21.15.67:0/1214758116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da2fb1b0 0x5568da6d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:11.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.222+0000 7f8e9bfff640 1 --2- 172.21.15.67:0/1214758116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da303390 0x5568da6d56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.224+0000 7f8ea1619640 1 -- 172.21.15.67:0/1214758116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da303390 msgr2=0x5568da6d56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.224+0000 7f8ea1619640 1 --2- 172.21.15.67:0/1214758116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da303390 0x5568da6d56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.225+0000 7f8ea1619640 1 -- 172.21.15.67:0/1214758116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da2fb1b0 msgr2=0x5568da6d51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.225+0000 7f8ea1619640 1 --2- 172.21.15.67:0/1214758116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da2fb1b0 0x5568da6d51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.225+0000 7f8ea1619640 1 -- 172.21.15.67:0/1214758116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568da38f650 con 0x5568da324070 2024-07-31T15:15:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.226+0000 7f8ea1619640 1 --2- 172.21.15.67:0/1214758116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da324070 0x5568da7030e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f8e940099a0 tx=0x7f8e940cb230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432121d3dbe44d17 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.226+0000 7f8e7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:15:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.229+0000 7f8e7bfff640 1 -- 172.21.15.67:0/1214758116 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e940e4030 con 0x5568da324070 2024-07-31T15:15:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.229+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1214758116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e900054d0 con 0x5568da324070 2024-07-31T15:15:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.229+0000 7f8e7bfff640 1 -- 172.21.15.67:0/1214758116 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f8e940c7f80 con 0x5568da324070 2024-07-31T15:15:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.233+0000 7f8e7bfff640 1 -- 172.21.15.67:0/1214758116 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f8e94102960 con 0x5568da324070 2024-07-31T15:15:11.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.234+0000 7f8ea95cd740 10 client.20541 fetch_fsmap learned FSMap version 66 2024-07-31T15:15:11.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.234+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1214758116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5568da705e10 con 0x5568da324070 2024-07-31T15:15:11.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.240+0000 7f8e7bfff640 1 -- 172.21.15.67:0/1214758116 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e900054d0 con 0x5568da324070 2024-07-31T15:15:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.240+0000 7f8e7bfff640 1 -- 172.21.15.67:0/1214758116 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f8e9402f3a0 con 0x5568da324070 2024-07-31T15:15:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.241+0000 7f8ea95cd740 10 client.20541 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:15:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.241+0000 7f8ea95cd740 10 client.20541 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:15:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.241+0000 7f8ea95cd740 20 client.20541 populate_metadata read hostname 'smithi067' 2024-07-31T15:15:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.241+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1214758116 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5568da2e29f0 0x5568da7092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.241+0000 7f8ea95cd740 4 client.20541 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:15:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.241+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1214758116 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568da641b00 con 0x5568da2e29f0 2024-07-31T15:15:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.241+0000 7f8ea0e18640 1 --2- 172.21.15.67:0/1214758116 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5568da2e29f0 0x5568da7092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:11.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.244+0000 7f8e7bfff640 10 client.20541 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:15:11.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.244+0000 7f8ea0e18640 1 --2- 172.21.15.67:0/1214758116 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5568da2e29f0 0x5568da7092d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f0722341e92edc server_cookie=394f8722c324b734 in_seq=0 out_seq=0 2024-07-31T15:15:11.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.248+0000 7f8e7bfff640 1 -- 172.21.15.67:0/1214758116 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5568da641b00 con 0x5568da2e29f0 2024-07-31T15:15:11.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.248+0000 7f8e7bfff640 10 client.20541 handle_command_reply: tid=0 2024-07-31T15:15:11.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.249+0000 7f8ea95cd740 1 client.20541 shutdown 2024-07-31T15:15:11.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.249+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1214758116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da324070 msgr2=0x5568da7030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:11.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.249+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1214758116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da324070 0x5568da7030e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f8e940099a0 tx=0x7f8e940cb230 comp rx=0 tx=0).stop 2024-07-31T15:15:11.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.251+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1214758116 shutdown_connections 2024-07-31T15:15:11.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.251+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1214758116 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5568da2e29f0 0x5568da7092d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.251+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1214758116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da324070 0x5568da7030e0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.251+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1214758116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da303390 0x5568da6d56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.251+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1214758116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da2fb1b0 0x5568da6d51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.251+0000 7f8e7bfff640 0 client.20541 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:15:11.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.251+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1214758116 >> 172.21.15.67:0/1214758116 conn(0x5568da318460 msgr2=0x5568da6cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:11.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.252+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1214758116 shutdown_connections 2024-07-31T15:15:11.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.255+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1214758116 wait complete. 2024-07-31T15:15:11.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.256+0000 7f8ea95cd740 20 client.20541 trim_cache size 0 max 16384 2024-07-31T15:15:11.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.283+0000 7f8ea95cd740 1 Processor -- start 2024-07-31T15:15:11.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.283+0000 7f8ea95cd740 1 -- start start 2024-07-31T15:15:11.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.284+0000 7f8ea95cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da2fb1b0 0x5568da6ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.284+0000 7f8ea95cd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da303390 0x5568da702ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.284+0000 7f8ea95cd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da324070 0x5568da6ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.284+0000 7f8e9bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da303390 0x5568da702ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.284+0000 7f8e9bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da303390 0x5568da702ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:51060/0 (socket says 172.21.15.67:51060) 2024-07-31T15:15:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.285+0000 7f8e9bfff640 1 -- 172.21.15.67:0/1731607041 learned_addr learned my addr 172.21.15.67:0/1731607041 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.285+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1731607041 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80010030 con 0x5568da324070 2024-07-31T15:15:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.285+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1731607041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80011b70 con 0x5568da2fb1b0 2024-07-31T15:15:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.285+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1731607041 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80010700 con 0x5568da303390 2024-07-31T15:15:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.286+0000 7f8ea1619640 1 --2- 172.21.15.67:0/1731607041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da324070 0x5568da6ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.286+0000 7f8ea0e18640 1 --2- 172.21.15.67:0/1731607041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da2fb1b0 0x5568da6ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:11.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.287+0000 7f8e9bfff640 1 -- 172.21.15.67:0/1731607041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da2fb1b0 msgr2=0x5568da6ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:11.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.287+0000 7f8e9bfff640 1 --2- 172.21.15.67:0/1731607041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da2fb1b0 0x5568da6ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.287+0000 7f8e9bfff640 1 -- 172.21.15.67:0/1731607041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da324070 msgr2=0x5568da6ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:11.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.287+0000 7f8e9bfff640 1 --2- 172.21.15.67:0/1731607041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da324070 0x5568da6ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.287+0000 7f8e9bfff640 1 -- 172.21.15.67:0/1731607041 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568da6cb860 con 0x5568da303390 2024-07-31T15:15:11.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.291+0000 7f8e9bfff640 1 --2- 172.21.15.67:0/1731607041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da303390 0x5568da702ed0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8e880147c0 tx=0x7f8e8800f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bd3c6aa09e3abf server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.295+0000 7f8e867fc640 1 -- 172.21.15.67:0/1731607041 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88024070 con 0x5568da303390 2024-07-31T15:15:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.295+0000 7f8e867fc640 1 -- 172.21.15.67:0/1731607041 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e8801f400 con 0x5568da303390 2024-07-31T15:15:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.295+0000 7f8e867fc640 1 -- 172.21.15.67:0/1731607041 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8801e600 con 0x5568da303390 2024-07-31T15:15:11.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.299+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1731607041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da303390 msgr2=0x5568da702ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:11.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.299+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1731607041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da303390 0x5568da702ed0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8e880147c0 tx=0x7f8e8800f000 comp rx=0 tx=0).stop 2024-07-31T15:15:11.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.302+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1731607041 shutdown_connections 2024-07-31T15:15:11.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.302+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1731607041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da324070 0x5568da6ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.302+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1731607041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da303390 0x5568da702ed0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.302+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1731607041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da2fb1b0 0x5568da6ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.302+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1731607041 >> 172.21.15.67:0/1731607041 conn(0x5568da318460 msgr2=0x5568da6cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:11.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.304+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1731607041 shutdown_connections 2024-07-31T15:15:11.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.304+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1731607041 wait complete. 2024-07-31T15:15:11.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.305+0000 7f8ea95cd740 1 -- start start 2024-07-31T15:15:11.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.305+0000 7f8ea95cd740 1 Processor -- start 2024-07-31T15:15:11.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.308+0000 7f8ea95cd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da2fb1b0 0x5568da6ff050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.308+0000 7f8ea95cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da303390 0x5568da6ff590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.308+0000 7f8ea95cd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da324070 0x5568da6fa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.308+0000 7f8ea95cd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8e800114a0 con 0x5568da2fb1b0 2024-07-31T15:15:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.308+0000 7f8ea95cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80010dd0 con 0x5568da303390 2024-07-31T15:15:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.308+0000 7f8ea95cd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8000e030 con 0x5568da324070 2024-07-31T15:15:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.308+0000 7f8ea95cd740 10 client.0 _ll_register_callbacks cb 0x7f8e940c7900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:15:11.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.310+0000 7f8e9bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da303390 0x5568da6ff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:11.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.310+0000 7f8e9bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da303390 0x5568da6ff590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46884/0 (socket says 172.21.15.67:46884) 2024-07-31T15:15:11.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.310+0000 7f8e9bfff640 1 -- 172.21.15.67:0/1757360112 learned_addr learned my addr 172.21.15.67:0/1757360112 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:11.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.312+0000 7f8e9bfff640 1 -- 172.21.15.67:0/1757360112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da324070 msgr2=0x5568da6fa000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-31T15:15:11.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.312+0000 7f8e9bfff640 1 --2- 172.21.15.67:0/1757360112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da324070 0x5568da6fa000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.312+0000 7f8e9bfff640 1 -- 172.21.15.67:0/1757360112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da2fb1b0 msgr2=0x5568da6ff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:11.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.312+0000 7f8e9bfff640 1 --2- 172.21.15.67:0/1757360112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da2fb1b0 0x5568da6ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.312+0000 7f8e9bfff640 1 -- 172.21.15.67:0/1757360112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568da6c77b0 con 0x5568da303390 2024-07-31T15:15:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.328+0000 7f8e9bfff640 1 --2- 172.21.15.67:0/1757360112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da303390 0x5568da6ff590 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8e8801fd20 tx=0x7f8e88007890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e865dc55cd3531bb server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.328+0000 7f8e7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-07-31T15:15:11.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.329+0000 7f8e7bfff640 1 -- 172.21.15.67:0/1757360112 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8802c030 con 0x5568da303390 2024-07-31T15:15:11.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.329+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1757360112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e8801f400 con 0x5568da303390 2024-07-31T15:15:11.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.329+0000 7f8e7bfff640 1 -- 172.21.15.67:0/1757360112 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f8e88013b40 con 0x5568da303390 2024-07-31T15:15:11.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.329+0000 7f8e7bfff640 1 -- 172.21.15.67:0/1757360112 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f8e88036050 con 0x5568da303390 2024-07-31T15:15:11.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.329+0000 7f8ea95cd740 10 client.30050 fetch_fsmap learned FSMap version 66 2024-07-31T15:15:11.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.329+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1757360112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5568da771e10 con 0x5568da303390 2024-07-31T15:15:11.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.330+0000 7f8e7bfff640 1 -- 172.21.15.67:0/1757360112 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e8801f400 con 0x5568da303390 2024-07-31T15:15:11.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.330+0000 7f8e7bfff640 1 -- 172.21.15.67:0/1757360112 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f8e88003920 con 0x5568da303390 2024-07-31T15:15:11.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.330+0000 7f8ea95cd740 10 client.30050 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:15:11.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.330+0000 7f8ea95cd740 10 client.30050 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-31T15:15:11.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.330+0000 7f8ea95cd740 20 client.30050 populate_metadata read hostname 'smithi067' 2024-07-31T15:15:11.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.330+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1757360112 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5568da2b7da0 0x5568da7744c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:11.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.330+0000 7f8ea95cd740 4 client.30050 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:15:11.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.330+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1757360112 --> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568da63d5a0 con 0x5568da2b7da0 2024-07-31T15:15:11.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.331+0000 7f8ea0e18640 1 --2- 172.21.15.67:0/1757360112 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5568da2b7da0 0x5568da7744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.335+0000 7f8ea0e18640 1 --2- 172.21.15.67:0/1757360112 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5568da2b7da0 0x5568da7744c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bb9db91ef1552d99 server_cookie=c069dea8cb8dec10 in_seq=0 out_seq=0 2024-07-31T15:15:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.335+0000 7f8e7bfff640 10 client.30050 ms_handle_connect on v2:172.21.15.77:6836/3460575274 2024-07-31T15:15:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.335+0000 7f8e7bfff640 1 -- 172.21.15.67:0/1757360112 <== mds.3 v2:172.21.15.77:6836/3460575274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568da63d5a0 con 0x5568da2b7da0 2024-07-31T15:15:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.335+0000 7f8e7bfff640 10 client.30050 handle_command_reply: tid=0 2024-07-31T15:15:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.335+0000 7f8ea95cd740 1 client.30050 shutdown 2024-07-31T15:15:11.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.339+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1757360112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da303390 msgr2=0x5568da6ff590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:11.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.339+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1757360112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da303390 0x5568da6ff590 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8e8801fd20 tx=0x7f8e88007890 comp rx=0 tx=0).stop 2024-07-31T15:15:11.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.339+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1757360112 shutdown_connections 2024-07-31T15:15:11.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.339+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1757360112 >> [v2:172.21.15.77:6836/3460575274,v1:172.21.15.77:6837/3460575274] conn(0x5568da2b7da0 0x5568da7744c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.339+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1757360112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5568da324070 0x5568da6fa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.339+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1757360112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568da303390 0x5568da6ff590 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.339+0000 7f8ea95cd740 1 --2- 172.21.15.67:0/1757360112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5568da2fb1b0 0x5568da6ff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.339+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1757360112 >> 172.21.15.67:0/1757360112 conn(0x5568da318460 msgr2=0x5568da6ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:11.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.339+0000 7f8e7bfff640 0 client.30050 ms_handle_reset on v2:172.21.15.77:6836/3460575274 2024-07-31T15:15:11.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.339+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1757360112 shutdown_connections 2024-07-31T15:15:11.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.339+0000 7f8ea95cd740 1 -- 172.21.15.67:0/1757360112 wait complete. 2024-07-31T15:15:11.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.340+0000 7f8ea95cd740 20 client.30050 trim_cache size 0 max 16384 2024-07-31T15:15:11.343 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:15:11.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.340+0000 7f8ea307c640 1 -- 172.21.15.67:0/4044340589 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8e8c0b3f50 msgr2=0x7f8e8c0b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:11.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.340+0000 7f8ea307c640 1 --2- 172.21.15.67:0/4044340589 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8e8c0b3f50 0x7f8e8c0b6460 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x5568da6c81f0 tx=0x7f8e88007ab0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.340+0000 7f8ea307c640 1 -- 172.21.15.67:0/4044340589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c11c660 msgr2=0x7f8e9c1ae5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:11.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.340+0000 7f8ea307c640 1 --2- 172.21.15.67:0/4044340589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c11c660 0x7f8e9c1ae5b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8e94003f10 tx=0x7f8e94003f40 comp rx=0 tx=0).stop 2024-07-31T15:15:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.341+0000 7f8ea307c640 1 -- 172.21.15.67:0/4044340589 shutdown_connections 2024-07-31T15:15:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.341+0000 7f8ea307c640 1 --2- 172.21.15.67:0/4044340589 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f8e8c0b3f50 0x7f8e8c0b6460 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.341+0000 7f8ea307c640 1 --2- 172.21.15.67:0/4044340589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e9c11c660 0x7f8e9c1ae5b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.341+0000 7f8ea307c640 1 --2- 172.21.15.67:0/4044340589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c111970 0x7f8e9c1a9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.341+0000 7f8ea307c640 1 --2- 172.21.15.67:0/4044340589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c10a410 0x7f8e9c1a8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.341+0000 7f8ea307c640 1 -- 172.21.15.67:0/4044340589 >> 172.21.15.67:0/4044340589 conn(0x7f8e9c0708f0 msgr2=0x7f8e9c11ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.341+0000 7f8ea307c640 1 -- 172.21.15.67:0/4044340589 shutdown_connections 2024-07-31T15:15:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:11.341+0000 7f8ea307c640 1 -- 172.21.15.67:0/4044340589 wait complete. 2024-07-31T15:15:11.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:11 smithi067 ceph-mon[32750]: pgmap v1169: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 27 op/s 2024-07-31T15:15:11.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:11 smithi067 ceph-mon[32750]: osdmap e137: 12 total, 12 up, 12 in 2024-07-31T15:15:11.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:11 smithi077 ceph-mon[37327]: pgmap v1169: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 27 op/s 2024-07-31T15:15:11.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:11 smithi077 ceph-mon[37327]: osdmap e137: 12 total, 12 up, 12 in 2024-07-31T15:15:11.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:11 smithi073 ceph-mon[36714]: pgmap v1169: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 27 op/s 2024-07-31T15:15:11.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:11 smithi073 ceph-mon[36714]: osdmap e137: 12 total, 12 up, 12 in 2024-07-31T15:15:12.065 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-31T15:15:12.584 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:15:13.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:13 smithi077 ceph-mon[37327]: pgmap v1171: 129 pgs: 9 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 93 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-07-31T15:15:13.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:13 smithi067 ceph-mon[32750]: pgmap v1171: 129 pgs: 9 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 93 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-07-31T15:15:13.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:13 smithi073 ceph-mon[36714]: pgmap v1171: 129 pgs: 9 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 93 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-07-31T15:15:15.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.161+0000 7f4897548640 1 -- 172.21.15.67:0/379650054 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890106620 msgr2=0x7f4890110040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:15.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.161+0000 7f4897548640 1 --2- 172.21.15.67:0/379650054 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890106620 0x7f4890110040 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f488800f0a0 tx=0x7f488802e6f0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.161+0000 7f4897548640 1 -- 172.21.15.67:0/379650054 shutdown_connections 2024-07-31T15:15:15.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.161+0000 7f4897548640 1 --2- 172.21.15.67:0/379650054 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890106620 0x7f4890110040 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.161+0000 7f4897548640 1 --2- 172.21.15.67:0/379650054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4890105cd0 0x7f48901060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.161+0000 7f4897548640 1 --2- 172.21.15.67:0/379650054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4890101ae0 0x7f4890101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.161+0000 7f4897548640 1 -- 172.21.15.67:0/379650054 >> 172.21.15.67:0/379650054 conn(0x7f4890077690 msgr2=0x7f4890077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:15.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.162+0000 7f4897548640 1 -- 172.21.15.67:0/379650054 shutdown_connections 2024-07-31T15:15:15.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.162+0000 7f4897548640 1 -- 172.21.15.67:0/379650054 wait complete. 2024-07-31T15:15:15.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.163+0000 7f4897548640 1 Processor -- start 2024-07-31T15:15:15.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.163+0000 7f4897548640 1 -- start start 2024-07-31T15:15:15.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.163+0000 7f4897548640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890101ae0 0x7f4890197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.163+0000 7f4897548640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4890105cd0 0x7f4890198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.163+0000 7f4897548640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4890106620 0x7f489019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.163+0000 7f4897548640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f48901112b0 con 0x7f4890101ae0 2024-07-31T15:15:15.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.163+0000 7f4897548640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4890111130 con 0x7f4890106620 2024-07-31T15:15:15.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.163+0000 7f4897548640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4890111430 con 0x7f4890105cd0 2024-07-31T15:15:15.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.163+0000 7f4894ae3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4890105cd0 0x7f4890198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.163+0000 7f4894ae3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4890105cd0 0x7f4890198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:51094/0 (socket says 172.21.15.67:51094) 2024-07-31T15:15:15.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.163+0000 7f48952e4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890101ae0 0x7f4890197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.163+0000 7f4894ae3640 1 -- 172.21.15.67:0/1231761973 learned_addr learned my addr 172.21.15.67:0/1231761973 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.164+0000 7f4895ae5640 1 --2- 172.21.15.67:0/1231761973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4890106620 0x7f489019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.164+0000 7f4894ae3640 1 -- 172.21.15.67:0/1231761973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4890106620 msgr2=0x7f489019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.164+0000 7f4894ae3640 1 --2- 172.21.15.67:0/1231761973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4890106620 0x7f489019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.164+0000 7f4894ae3640 1 -- 172.21.15.67:0/1231761973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890101ae0 msgr2=0x7f4890197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.164+0000 7f4894ae3640 1 --2- 172.21.15.67:0/1231761973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890101ae0 0x7f4890197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.164+0000 7f4894ae3640 1 -- 172.21.15.67:0/1231761973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489019db40 con 0x7f4890105cd0 2024-07-31T15:15:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.164+0000 7f4895ae5640 1 --2- 172.21.15.67:0/1231761973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4890106620 0x7f489019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:15:15.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.164+0000 7f4894ae3640 1 --2- 172.21.15.67:0/1231761973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4890105cd0 0x7f4890198170 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f488000e9c0 tx=0x7f488000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:15.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.164+0000 7f48867fc640 1 -- 172.21.15.67:0/1231761973 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488000cd80 con 0x7f4890105cd0 2024-07-31T15:15:15.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.164+0000 7f48867fc640 1 -- 172.21.15.67:0/1231761973 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48800054d0 con 0x7f4890105cd0 2024-07-31T15:15:15.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.164+0000 7f48867fc640 1 -- 172.21.15.67:0/1231761973 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880010430 con 0x7f4890105cd0 2024-07-31T15:15:15.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.164+0000 7f4897548640 1 -- 172.21.15.67:0/1231761973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489019de30 con 0x7f4890105cd0 2024-07-31T15:15:15.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.164+0000 7f4897548640 1 -- 172.21.15.67:0/1231761973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48901a4590 con 0x7f4890105cd0 2024-07-31T15:15:15.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.167+0000 7f48867fc640 1 -- 172.21.15.67:0/1231761973 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4880004fd0 con 0x7f4890105cd0 2024-07-31T15:15:15.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.167+0000 7f489da99740 1 Processor -- start 2024-07-31T15:15:15.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.167+0000 7f489da99740 1 -- start start 2024-07-31T15:15:15.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f489da99740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dfe070 0x5563b11abea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f489da99740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dd51b0 0x5563b11ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f489da99740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 0x5563b11ac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f489da99740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5563b1119bb0 con 0x5563b0dfe070 2024-07-31T15:15:15.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f489da99740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5563b11090b0 con 0x5563b0dd51b0 2024-07-31T15:15:15.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f489da99740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5563b110d0f0 con 0x5563b0ddd390 2024-07-31T15:15:15.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f4894ae3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 0x5563b11ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f48952e4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dd51b0 0x5563b11ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f48952e4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dd51b0 0x5563b11ac3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:46916/0 (socket says 172.21.15.67:46916) 2024-07-31T15:15:15.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f4895ae5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dfe070 0x5563b11abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f48952e4640 1 -- 172.21.15.67:0/2232102757 learned_addr learned my addr 172.21.15.67:0/2232102757 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:15.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f4895ae5640 1 -- 172.21.15.67:0/2232102757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 msgr2=0x5563b11ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:15.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f4895ae5640 1 --2- 172.21.15.67:0/2232102757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 0x5563b11ac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f4895ae5640 1 -- 172.21.15.67:0/2232102757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dd51b0 msgr2=0x5563b11ac3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:15.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.168+0000 7f4895ae5640 1 --2- 172.21.15.67:0/2232102757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dd51b0 0x5563b11ac3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.169+0000 7f4895ae5640 1 -- 172.21.15.67:0/2232102757 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563b10afb90 con 0x5563b0dfe070 2024-07-31T15:15:15.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.169+0000 7f4895ae5640 1 --2- 172.21.15.67:0/2232102757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dfe070 0x5563b11abea0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f488800f0a0 tx=0x7f488800d850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b349df9820c82f08 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:15.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.169+0000 7f48867fc640 1 --2- 172.21.15.67:0/1231761973 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f487c0b3d40 0x7f487c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.169+0000 7f487b7fe640 1 -- 172.21.15.67:0/2232102757 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488803a070 con 0x5563b0dfe070 2024-07-31T15:15:15.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.169+0000 7f487b7fe640 1 -- 172.21.15.67:0/2232102757 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4888035c50 con 0x5563b0dfe070 2024-07-31T15:15:15.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.169+0000 7f487b7fe640 1 -- 172.21.15.67:0/2232102757 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888004040 con 0x5563b0dfe070 2024-07-31T15:15:15.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.169+0000 7f48952e4640 1 --2- 172.21.15.67:0/1231761973 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f487c0b3d40 0x7f487c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.170+0000 7f48867fc640 1 -- 172.21.15.67:0/1231761973 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f48800e4980 con 0x7f4890105cd0 2024-07-31T15:15:15.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.171+0000 7f489da99740 1 -- 172.21.15.67:0/2232102757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dfe070 msgr2=0x5563b11abea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:15.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.171+0000 7f48952e4640 1 --2- 172.21.15.67:0/1231761973 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f487c0b3d40 0x7f487c0b6250 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x5563b11aab20 tx=0x7f488c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:15.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.171+0000 7f489da99740 1 --2- 172.21.15.67:0/2232102757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dfe070 0x5563b11abea0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f488800f0a0 tx=0x7f488800d850 comp rx=0 tx=0).stop 2024-07-31T15:15:15.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.171+0000 7f489da99740 1 -- 172.21.15.67:0/2232102757 shutdown_connections 2024-07-31T15:15:15.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.171+0000 7f489da99740 1 --2- 172.21.15.67:0/2232102757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 0x5563b11ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.171+0000 7f489da99740 1 --2- 172.21.15.67:0/2232102757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dd51b0 0x5563b11ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.171+0000 7f489da99740 1 --2- 172.21.15.67:0/2232102757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dfe070 0x5563b11abea0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.171+0000 7f489da99740 1 -- 172.21.15.67:0/2232102757 >> 172.21.15.67:0/2232102757 conn(0x5563b0df2460 msgr2=0x5563b11a9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:15.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.171+0000 7f489da99740 1 -- 172.21.15.67:0/2232102757 shutdown_connections 2024-07-31T15:15:15.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.171+0000 7f489da99740 1 -- 172.21.15.67:0/2232102757 wait complete. 2024-07-31T15:15:15.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.171+0000 7f489da99740 1 -- start start 2024-07-31T15:15:15.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.171+0000 7f489da99740 1 Processor -- start 2024-07-31T15:15:15.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.172+0000 7f489da99740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dd51b0 0x5563b11b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.172+0000 7f489da99740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 0x5563b11b8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.172+0000 7f489da99740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dfe070 0x5563b11c3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.172+0000 7f489da99740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5563b0f6c700 con 0x5563b0dd51b0 2024-07-31T15:15:15.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.172+0000 7f489da99740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5563b10ab9e0 con 0x5563b0dfe070 2024-07-31T15:15:15.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.172+0000 7f489da99740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5563b11ba620 con 0x5563b0ddd390 2024-07-31T15:15:15.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.172+0000 7f48952e4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dd51b0 0x5563b11b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.172+0000 7f48952e4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dd51b0 0x5563b11b7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44420/0 (socket says 172.21.15.67:44420) 2024-07-31T15:15:15.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.172+0000 7f489da99740 10 client.0 _ll_register_callbacks cb 0x5563b10c64f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:15:15.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.172+0000 7f48952e4640 1 -- 172.21.15.67:0/1021059553 learned_addr learned my addr 172.21.15.67:0/1021059553 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:15.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.172+0000 7f4894ae3640 1 --2- 172.21.15.67:0/1021059553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 0x5563b11b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.173+0000 7f48952e4640 1 -- 172.21.15.67:0/1021059553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 msgr2=0x5563b11b8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:15.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.173+0000 7f48952e4640 1 --2- 172.21.15.67:0/1021059553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 0x5563b11b8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.173+0000 7f48952e4640 1 -- 172.21.15.67:0/1021059553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dfe070 msgr2=0x5563b11c3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:15.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.173+0000 7f48952e4640 1 --2- 172.21.15.67:0/1021059553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dfe070 0x5563b11c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.173+0000 7f48952e4640 1 -- 172.21.15.67:0/1021059553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563b0e69650 con 0x5563b0dd51b0 2024-07-31T15:15:15.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.173+0000 7f48952e4640 1 --2- 172.21.15.67:0/1021059553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dd51b0 0x5563b11b7ae0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f488c013010 tx=0x7f488c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3126a98afdefd8e5 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:15.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.173+0000 7f4878ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:15:15.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.173+0000 7f4878ff9640 1 -- 172.21.15.67:0/1021059553 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c02d060 con 0x5563b0dd51b0 2024-07-31T15:15:15.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.173+0000 7f489da99740 1 -- 172.21.15.67:0/1021059553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4888035c50 con 0x5563b0dd51b0 2024-07-31T15:15:15.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.173+0000 7f4878ff9640 1 -- 172.21.15.67:0/1021059553 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f488c018030 con 0x5563b0dd51b0 2024-07-31T15:15:15.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.173+0000 7f4878ff9640 1 -- 172.21.15.67:0/1021059553 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f488c034050 con 0x5563b0dd51b0 2024-07-31T15:15:15.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.173+0000 7f489da99740 10 client.20556 fetch_fsmap learned FSMap version 66 2024-07-31T15:15:15.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.173+0000 7f489da99740 1 -- 172.21.15.67:0/1021059553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5563b11c6710 con 0x5563b0dd51b0 2024-07-31T15:15:15.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.174+0000 7f4878ff9640 1 -- 172.21.15.67:0/1021059553 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4888035c50 con 0x5563b0dd51b0 2024-07-31T15:15:15.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.174+0000 7f4878ff9640 1 -- 172.21.15.67:0/1021059553 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f488c01daf0 con 0x5563b0dd51b0 2024-07-31T15:15:15.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.174+0000 7f489da99740 10 client.20556 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:15:15.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.174+0000 7f489da99740 10 client.20556 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:15:15.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.174+0000 7f489da99740 20 client.20556 populate_metadata read hostname 'smithi067' 2024-07-31T15:15:15.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.174+0000 7f489da99740 1 --2- 172.21.15.67:0/1021059553 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5563b0dbc9f0 0x5563b11c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.174+0000 7f489da99740 4 client.20556 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:15:15.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.174+0000 7f489da99740 1 -- 172.21.15.67:0/1021059553 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563b111bb00 con 0x5563b0dbc9f0 2024-07-31T15:15:15.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.174+0000 7f4894ae3640 1 --2- 172.21.15.67:0/1021059553 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5563b0dbc9f0 0x5563b11c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.175+0000 7f4878ff9640 10 client.20556 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:15:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.175+0000 7f4894ae3640 1 --2- 172.21.15.67:0/1021059553 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5563b0dbc9f0 0x5563b11c9bc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd95f0360745e144 server_cookie=98d66c94678dc8be in_seq=0 out_seq=0 2024-07-31T15:15:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f4878ff9640 1 -- 172.21.15.67:0/1021059553 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563b111bb00 con 0x5563b0dbc9f0 2024-07-31T15:15:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f4878ff9640 10 client.20556 handle_command_reply: tid=0 2024-07-31T15:15:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f489da99740 1 client.20556 shutdown 2024-07-31T15:15:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f489da99740 1 -- 172.21.15.67:0/1021059553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dd51b0 msgr2=0x5563b11b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f489da99740 1 --2- 172.21.15.67:0/1021059553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dd51b0 0x5563b11b7ae0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f488c013010 tx=0x7f488c004730 comp rx=0 tx=0).stop 2024-07-31T15:15:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f489da99740 1 -- 172.21.15.67:0/1021059553 shutdown_connections 2024-07-31T15:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f489da99740 1 --2- 172.21.15.67:0/1021059553 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5563b0dbc9f0 0x5563b11c9bc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f4878ff9640 0 client.20556 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f489da99740 1 --2- 172.21.15.67:0/1021059553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dfe070 0x5563b11c3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f489da99740 1 --2- 172.21.15.67:0/1021059553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 0x5563b11b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f489da99740 1 --2- 172.21.15.67:0/1021059553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dd51b0 0x5563b11b7ae0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f489da99740 1 -- 172.21.15.67:0/1021059553 >> 172.21.15.67:0/1021059553 conn(0x5563b0df2460 msgr2=0x5563b11af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f489da99740 1 -- 172.21.15.67:0/1021059553 shutdown_connections 2024-07-31T15:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f489da99740 1 -- 172.21.15.67:0/1021059553 wait complete. 2024-07-31T15:15:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.176+0000 7f489da99740 20 client.20556 trim_cache size 0 max 16384 2024-07-31T15:15:15.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.191+0000 7f489da99740 1 Processor -- start 2024-07-31T15:15:15.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.191+0000 7f489da99740 1 -- start start 2024-07-31T15:15:15.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.191+0000 7f489da99740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dbc9f0 0x5563b11a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.191+0000 7f489da99740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dd51b0 0x5563b11b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.191+0000 7f489da99740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 0x5563b11b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.191+0000 7f489da99740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f48901ae750 con 0x5563b0dbc9f0 2024-07-31T15:15:15.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.191+0000 7f489da99740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f48901b0290 con 0x5563b0dd51b0 2024-07-31T15:15:15.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.191+0000 7f489da99740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48901aee20 con 0x5563b0ddd390 2024-07-31T15:15:15.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.191+0000 7f48952e4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dbc9f0 0x5563b11a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.191+0000 7f48952e4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dbc9f0 0x5563b11a9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44424/0 (socket says 172.21.15.67:44424) 2024-07-31T15:15:15.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.191+0000 7f48952e4640 1 -- 172.21.15.67:0/4228026761 learned_addr learned my addr 172.21.15.67:0/4228026761 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:15.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.191+0000 7f4894ae3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dd51b0 0x5563b11b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.191+0000 7f4895ae5640 1 --2- 172.21.15.67:0/4228026761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 0x5563b11b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.192+0000 7f4894ae3640 1 -- 172.21.15.67:0/4228026761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 msgr2=0x5563b11b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:15.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.192+0000 7f4894ae3640 1 --2- 172.21.15.67:0/4228026761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 0x5563b11b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.192+0000 7f4894ae3640 1 -- 172.21.15.67:0/4228026761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dbc9f0 msgr2=0x5563b11a9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:15.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.192+0000 7f4894ae3640 1 --2- 172.21.15.67:0/4228026761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dbc9f0 0x5563b11a9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.192+0000 7f4894ae3640 1 -- 172.21.15.67:0/4228026761 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563b11b3170 con 0x5563b0dd51b0 2024-07-31T15:15:15.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.192+0000 7f4894ae3640 1 --2- 172.21.15.67:0/4228026761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dd51b0 0x5563b11b0f80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f48800e1cb0 tx=0x7f48800e6dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea367d3c95d9eb5e server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:15.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.192+0000 7f487affd640 1 -- 172.21.15.67:0/4228026761 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48800aa8e0 con 0x5563b0dd51b0 2024-07-31T15:15:15.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.192+0000 7f487affd640 1 -- 172.21.15.67:0/4228026761 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48800046c0 con 0x5563b0dd51b0 2024-07-31T15:15:15.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.192+0000 7f487affd640 1 -- 172.21.15.67:0/4228026761 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48800a7400 con 0x5563b0dd51b0 2024-07-31T15:15:15.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.194+0000 7f489da99740 1 -- 172.21.15.67:0/4228026761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dd51b0 msgr2=0x5563b11b0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:15.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.194+0000 7f489da99740 1 --2- 172.21.15.67:0/4228026761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dd51b0 0x5563b11b0f80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f48800e1cb0 tx=0x7f48800e6dd0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.194+0000 7f489da99740 1 -- 172.21.15.67:0/4228026761 shutdown_connections 2024-07-31T15:15:15.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.194+0000 7f489da99740 1 --2- 172.21.15.67:0/4228026761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0ddd390 0x5563b11b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.194+0000 7f489da99740 1 --2- 172.21.15.67:0/4228026761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0dd51b0 0x5563b11b0f80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.194+0000 7f489da99740 1 --2- 172.21.15.67:0/4228026761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dbc9f0 0x5563b11a9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.194+0000 7f489da99740 1 -- 172.21.15.67:0/4228026761 >> 172.21.15.67:0/4228026761 conn(0x5563b0df2460 msgr2=0x5563b11a9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:15.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.194+0000 7f489da99740 1 -- 172.21.15.67:0/4228026761 shutdown_connections 2024-07-31T15:15:15.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.194+0000 7f489da99740 1 -- 172.21.15.67:0/4228026761 wait complete. 2024-07-31T15:15:15.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.194+0000 7f489da99740 1 -- start start 2024-07-31T15:15:15.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.194+0000 7f489da99740 1 Processor -- start 2024-07-31T15:15:15.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.195+0000 7f489da99740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dbc9f0 0x5563b11bfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.195+0000 7f489da99740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0dd51b0 0x5563b11c3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.195+0000 7f489da99740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0ddd390 0x5563b11c3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.195+0000 7f489da99740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f48901afbc0 con 0x5563b0dbc9f0 2024-07-31T15:15:15.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.195+0000 7f489da99740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f48901af4f0 con 0x5563b0ddd390 2024-07-31T15:15:15.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.195+0000 7f489da99740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48901ac750 con 0x5563b0dd51b0 2024-07-31T15:15:15.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.195+0000 7f489da99740 10 client.0 _ll_register_callbacks cb 0x7f488c009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:15:15.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.195+0000 7f48952e4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dbc9f0 0x5563b11bfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.195+0000 7f48952e4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dbc9f0 0x5563b11bfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44426/0 (socket says 172.21.15.67:44426) 2024-07-31T15:15:15.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.195+0000 7f48952e4640 1 -- 172.21.15.67:0/1562963897 learned_addr learned my addr 172.21.15.67:0/1562963897 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:15.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.196+0000 7f48952e4640 1 -- 172.21.15.67:0/1562963897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0dd51b0 msgr2=0x5563b11c3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:15.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.196+0000 7f4894ae3640 1 --2- 172.21.15.67:0/1562963897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0dd51b0 0x5563b11c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.196+0000 7f4895ae5640 1 --2- 172.21.15.67:0/1562963897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0ddd390 0x5563b11c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.196+0000 7f48952e4640 1 --2- 172.21.15.67:0/1562963897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0dd51b0 0x5563b11c3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.196+0000 7f48952e4640 1 -- 172.21.15.67:0/1562963897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0ddd390 msgr2=0x5563b11c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:15.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.196+0000 7f48952e4640 1 --2- 172.21.15.67:0/1562963897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0ddd390 0x5563b11c3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.196+0000 7f48952e4640 1 -- 172.21.15.67:0/1562963897 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563b11c3ce0 con 0x5563b0dbc9f0 2024-07-31T15:15:15.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.196+0000 7f48952e4640 1 --2- 172.21.15.67:0/1562963897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dbc9f0 0x5563b11bfe80 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f488c009790 tx=0x7f488c010750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1481454493d069fc server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:15.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.196+0000 7f4878ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:15:15.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.196+0000 7f4878ff9640 1 -- 172.21.15.67:0/1562963897 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c03c020 con 0x5563b0dbc9f0 2024-07-31T15:15:15.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.196+0000 7f4878ff9640 1 -- 172.21.15.67:0/1562963897 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f488c010920 con 0x5563b0dbc9f0 2024-07-31T15:15:15.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.196+0000 7f489da99740 1 -- 172.21.15.67:0/1562963897 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5563b11175a0 con 0x5563b0dbc9f0 2024-07-31T15:15:15.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.196+0000 7f4878ff9640 1 -- 172.21.15.67:0/1562963897 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f488c047050 con 0x5563b0dbc9f0 2024-07-31T15:15:15.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.197+0000 7f489da99740 10 client.20562 fetch_fsmap learned FSMap version 66 2024-07-31T15:15:15.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.197+0000 7f489da99740 1 -- 172.21.15.67:0/1562963897 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5563b1232390 con 0x5563b0dbc9f0 2024-07-31T15:15:15.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.197+0000 7f4878ff9640 1 -- 172.21.15.67:0/1562963897 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5563b11175a0 con 0x5563b0dbc9f0 2024-07-31T15:15:15.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.197+0000 7f4878ff9640 1 -- 172.21.15.67:0/1562963897 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f488c01d940 con 0x5563b0dbc9f0 2024-07-31T15:15:15.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.197+0000 7f489da99740 10 client.20562 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:15:15.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.197+0000 7f489da99740 10 client.20562 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-07-31T15:15:15.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.197+0000 7f489da99740 20 client.20562 populate_metadata read hostname 'smithi067' 2024-07-31T15:15:15.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.197+0000 7f489da99740 1 --2- 172.21.15.67:0/1562963897 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5563b0dfe070 0x5563b1234960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:15.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.197+0000 7f489da99740 4 client.20562 mds_command: new command op to 24650 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:15:15.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.197+0000 7f489da99740 1 -- 172.21.15.67:0/1562963897 --> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563b1114ce0 con 0x5563b0dfe070 2024-07-31T15:15:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.198+0000 7f4894ae3640 1 --2- 172.21.15.67:0/1562963897 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5563b0dfe070 0x5563b1234960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.198+0000 7f4878ff9640 10 client.20562 ms_handle_connect on v2:172.21.15.73:6836/3592232039 2024-07-31T15:15:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.198+0000 7f4894ae3640 1 --2- 172.21.15.67:0/1562963897 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5563b0dfe070 0x5563b1234960 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f7f791155cd5585 server_cookie=72dc4f39f03b52d4 in_seq=0 out_seq=0 2024-07-31T15:15:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.198+0000 7f4878ff9640 1 -- 172.21.15.67:0/1562963897 <== mds.2 v2:172.21.15.73:6836/3592232039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563b1114ce0 con 0x5563b0dfe070 2024-07-31T15:15:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.198+0000 7f4878ff9640 10 client.20562 handle_command_reply: tid=0 2024-07-31T15:15:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.198+0000 7f489da99740 1 client.20562 shutdown 2024-07-31T15:15:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.199+0000 7f489da99740 1 -- 172.21.15.67:0/1562963897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dbc9f0 msgr2=0x5563b11bfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.199+0000 7f489da99740 1 --2- 172.21.15.67:0/1562963897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dbc9f0 0x5563b11bfe80 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f488c009790 tx=0x7f488c010750 comp rx=0 tx=0).stop 2024-07-31T15:15:15.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.199+0000 7f489da99740 1 -- 172.21.15.67:0/1562963897 shutdown_connections 2024-07-31T15:15:15.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.199+0000 7f489da99740 1 --2- 172.21.15.67:0/1562963897 >> [v2:172.21.15.73:6836/3592232039,v1:172.21.15.73:6837/3592232039] conn(0x5563b0dfe070 0x5563b1234960 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.199+0000 7f489da99740 1 --2- 172.21.15.67:0/1562963897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563b0ddd390 0x5563b11c3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.199+0000 7f4878ff9640 0 client.20562 ms_handle_reset on v2:172.21.15.73:6836/3592232039 2024-07-31T15:15:15.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.199+0000 7f489da99740 1 --2- 172.21.15.67:0/1562963897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563b0dd51b0 0x5563b11c3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.199+0000 7f489da99740 1 --2- 172.21.15.67:0/1562963897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5563b0dbc9f0 0x5563b11bfe80 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.199+0000 7f489da99740 1 -- 172.21.15.67:0/1562963897 >> 172.21.15.67:0/1562963897 conn(0x5563b0df2460 msgr2=0x5563b11aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:15.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.199+0000 7f489da99740 1 -- 172.21.15.67:0/1562963897 shutdown_connections 2024-07-31T15:15:15.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.199+0000 7f489da99740 1 -- 172.21.15.67:0/1562963897 wait complete. 2024-07-31T15:15:15.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.199+0000 7f489da99740 20 client.20562 trim_cache size 0 max 16384 2024-07-31T15:15:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:15:15.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.200+0000 7f4897548640 1 -- 172.21.15.67:0/1231761973 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f487c0b3d40 msgr2=0x7f487c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:15.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.200+0000 7f4897548640 1 --2- 172.21.15.67:0/1231761973 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f487c0b3d40 0x7f487c0b6250 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x5563b11aab20 tx=0x7f488c005f90 comp rx=0 tx=0).stop 2024-07-31T15:15:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.200+0000 7f4897548640 1 -- 172.21.15.67:0/1231761973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4890105cd0 msgr2=0x7f4890198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.200+0000 7f4897548640 1 --2- 172.21.15.67:0/1231761973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4890105cd0 0x7f4890198170 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f488000e9c0 tx=0x7f488000ee80 comp rx=0 tx=0).stop 2024-07-31T15:15:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.200+0000 7f4897548640 1 -- 172.21.15.67:0/1231761973 shutdown_connections 2024-07-31T15:15:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.200+0000 7f4897548640 1 --2- 172.21.15.67:0/1231761973 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f487c0b3d40 0x7f487c0b6250 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.200+0000 7f4897548640 1 --2- 172.21.15.67:0/1231761973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4890106620 0x7f489019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.200+0000 7f4897548640 1 --2- 172.21.15.67:0/1231761973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4890105cd0 0x7f4890198170 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.200+0000 7f4897548640 1 --2- 172.21.15.67:0/1231761973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890101ae0 0x7f4890197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.200+0000 7f4897548640 1 -- 172.21.15.67:0/1231761973 >> 172.21.15.67:0/1231761973 conn(0x7f4890077690 msgr2=0x7f4890107eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:15.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.200+0000 7f4897548640 1 -- 172.21.15.67:0/1231761973 shutdown_connections 2024-07-31T15:15:15.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:15.200+0000 7f4897548640 1 -- 172.21.15.67:0/1231761973 wait complete. 2024-07-31T15:15:15.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:15 smithi077 ceph-mon[37327]: pgmap v1172: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-07-31T15:15:15.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:15 smithi067 ceph-mon[32750]: pgmap v1172: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-07-31T15:15:15.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:15 smithi073 ceph-mon[36714]: pgmap v1172: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-07-31T15:15:15.986 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-31T15:15:16.332 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:15:17.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:17 smithi077 ceph-mon[37327]: pgmap v1173: 129 pgs: 17 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 61 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-07-31T15:15:17.768 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:17 smithi067 ceph-mon[32750]: pgmap v1173: 129 pgs: 17 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 61 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-07-31T15:15:17.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:17 smithi073 ceph-mon[36714]: pgmap v1173: 129 pgs: 17 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 61 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-07-31T15:15:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.138+0000 7f97a769d640 1 -- 172.21.15.67:0/1360070297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97a0108740 msgr2=0x7f97a010ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:18.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.138+0000 7f97a769d640 1 --2- 172.21.15.67:0/1360070297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97a0108740 0x7f97a010ef70 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f9798009ca0 tx=0x7f979802a8c0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.138+0000 7f97a769d640 1 -- 172.21.15.67:0/1360070297 shutdown_connections 2024-07-31T15:15:18.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.138+0000 7f97a769d640 1 --2- 172.21.15.67:0/1360070297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97a0108740 0x7f97a010ef70 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.138+0000 7f97a769d640 1 --2- 172.21.15.67:0/1360070297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97a0107df0 0x7f97a0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.138+0000 7f97a769d640 1 --2- 172.21.15.67:0/1360070297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97a01024b0 0x7f97a01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.139+0000 7f97a769d640 1 -- 172.21.15.67:0/1360070297 >> 172.21.15.67:0/1360070297 conn(0x7f97a0079170 msgr2=0x7f97a0079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.139+0000 7f97a769d640 1 -- 172.21.15.67:0/1360070297 shutdown_connections 2024-07-31T15:15:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.139+0000 7f97a769d640 1 -- 172.21.15.67:0/1360070297 wait complete. 2024-07-31T15:15:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.139+0000 7f97a769d640 1 Processor -- start 2024-07-31T15:15:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.139+0000 7f97a769d640 1 -- start start 2024-07-31T15:15:18.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a769d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97a01024b0 0x7f97a0199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a769d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97a0107df0 0x7f97a019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a769d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97a0108740 0x7f97a019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a769d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01123a0 con 0x7f97a0107df0 2024-07-31T15:15:18.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a769d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f97a0112220 con 0x7f97a01024b0 2024-07-31T15:15:18.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a769d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97a0112520 con 0x7f97a0108740 2024-07-31T15:15:18.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a4c38640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97a0107df0 0x7f97a019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a5439640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97a01024b0 0x7f97a0199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a4c38640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97a0107df0 0x7f97a019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44440/0 (socket says 172.21.15.67:44440) 2024-07-31T15:15:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a4c38640 1 -- 172.21.15.67:0/2993817740 learned_addr learned my addr 172.21.15.67:0/2993817740 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a5c3a640 1 --2- 172.21.15.67:0/2993817740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97a0108740 0x7f97a019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a4c38640 1 -- 172.21.15.67:0/2993817740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97a0108740 msgr2=0x7f97a019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a4c38640 1 --2- 172.21.15.67:0/2993817740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97a0108740 0x7f97a019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a4c38640 1 -- 172.21.15.67:0/2993817740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97a01024b0 msgr2=0x7f97a0199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a4c38640 1 --2- 172.21.15.67:0/2993817740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97a01024b0 0x7f97a0199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a4c38640 1 -- 172.21.15.67:0/2993817740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a019fdd0 con 0x7f97a0107df0 2024-07-31T15:15:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97a4c38640 1 --2- 172.21.15.67:0/2993817740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97a0107df0 0x7f97a019a4c0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f979c00c970 tx=0x7f979c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.140+0000 7f97927fc640 1 -- 172.21.15.67:0/2993817740 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c014070 con 0x7f97a0107df0 2024-07-31T15:15:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.141+0000 7f97927fc640 1 -- 172.21.15.67:0/2993817740 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f979c004440 con 0x7f97a0107df0 2024-07-31T15:15:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.141+0000 7f97927fc640 1 -- 172.21.15.67:0/2993817740 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c003e50 con 0x7f97a0107df0 2024-07-31T15:15:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.141+0000 7f97a769d640 1 -- 172.21.15.67:0/2993817740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a01a00c0 con 0x7f97a0107df0 2024-07-31T15:15:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.141+0000 7f97a769d640 1 -- 172.21.15.67:0/2993817740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a00fefa0 con 0x7f97a0107df0 2024-07-31T15:15:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.144+0000 7f97adbee740 1 Processor -- start 2024-07-31T15:15:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.144+0000 7f97927fc640 1 -- 172.21.15.67:0/2993817740 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f979c017070 con 0x7f97a0107df0 2024-07-31T15:15:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.144+0000 7f97adbee740 1 -- start start 2024-07-31T15:15:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.144+0000 7f97adbee740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3e7070 0x55762b78c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.144+0000 7f97adbee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3be1b0 0x55762b78cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.144+0000 7f97adbee740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b78cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.144+0000 7f97adbee740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55762b702bb0 con 0x55762b3c6390 2024-07-31T15:15:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.144+0000 7f97adbee740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55762b6f20b0 con 0x55762b3e7070 2024-07-31T15:15:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.144+0000 7f97adbee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55762b6f60f0 con 0x55762b3be1b0 2024-07-31T15:15:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.144+0000 7f97a5439640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3be1b0 0x55762b78cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.144+0000 7f97a5439640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3be1b0 0x55762b78cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.67:51152/0 (socket says 172.21.15.67:51152) 2024-07-31T15:15:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.144+0000 7f97a5439640 1 -- 172.21.15.67:0/4022723520 learned_addr learned my addr 172.21.15.67:0/4022723520 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.144+0000 7f97a5c3a640 1 --2- 172.21.15.67:0/4022723520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3e7070 0x55762b78c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f97a4c38640 1 --2- 172.21.15.67:0/4022723520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b78cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f97a5439640 1 -- 172.21.15.67:0/4022723520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3e7070 msgr2=0x55762b78c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f97927fc640 1 --2- 172.21.15.67:0/2993817740 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f978c0b3f90 0x7f978c0b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f97a5439640 1 --2- 172.21.15.67:0/4022723520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3e7070 0x55762b78c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f97a5439640 1 -- 172.21.15.67:0/4022723520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 msgr2=0x55762b78cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f97a5439640 1 --2- 172.21.15.67:0/4022723520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b78cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f97a5439640 1 -- 172.21.15.67:0/4022723520 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55762b698b90 con 0x55762b3be1b0 2024-07-31T15:15:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f97a5c3a640 1 --2- 172.21.15.67:0/4022723520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3e7070 0x55762b78c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:15:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f97a4c38640 1 --2- 172.21.15.67:0/4022723520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b78cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:15:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f97a5c3a640 1 --2- 172.21.15.67:0/2993817740 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f978c0b3f90 0x7f978c0b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f97927fc640 1 -- 172.21.15.67:0/2993817740 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f979c0e3de0 con 0x7f97a0107df0 2024-07-31T15:15:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f97a5439640 1 --2- 172.21.15.67:0/4022723520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3be1b0 0x55762b78cab0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f97940093d0 tx=0x7f9794009890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6854f328a0fdb678 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f978b7fe640 1 -- 172.21.15.67:0/4022723520 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979401f070 con 0x55762b3be1b0 2024-07-31T15:15:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f978b7fe640 1 -- 172.21.15.67:0/4022723520 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9794004230 con 0x55762b3be1b0 2024-07-31T15:15:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.145+0000 7f978b7fe640 1 -- 172.21.15.67:0/4022723520 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979401a7d0 con 0x55762b3be1b0 2024-07-31T15:15:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.146+0000 7f97adbee740 1 -- 172.21.15.67:0/4022723520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3be1b0 msgr2=0x55762b78cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.146+0000 7f97adbee740 1 --2- 172.21.15.67:0/4022723520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3be1b0 0x55762b78cab0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f97940093d0 tx=0x7f9794009890 comp rx=0 tx=0).stop 2024-07-31T15:15:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.146+0000 7f97a5c3a640 1 --2- 172.21.15.67:0/2993817740 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f978c0b3f90 0x7f978c0b64a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9798005cc0 tx=0x7f97980095c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:18.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.146+0000 7f97adbee740 1 -- 172.21.15.67:0/4022723520 shutdown_connections 2024-07-31T15:15:18.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.146+0000 7f97adbee740 1 --2- 172.21.15.67:0/4022723520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b78cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.146+0000 7f97adbee740 1 --2- 172.21.15.67:0/4022723520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3be1b0 0x55762b78cab0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.146+0000 7f97adbee740 1 --2- 172.21.15.67:0/4022723520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3e7070 0x55762b78c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.146+0000 7f97adbee740 1 -- 172.21.15.67:0/4022723520 >> 172.21.15.67:0/4022723520 conn(0x55762b3db460 msgr2=0x55762b78a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:18.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.146+0000 7f97adbee740 1 -- 172.21.15.67:0/4022723520 shutdown_connections 2024-07-31T15:15:18.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.146+0000 7f97adbee740 1 -- 172.21.15.67:0/4022723520 wait complete. 2024-07-31T15:15:18.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.147+0000 7f97adbee740 1 -- start start 2024-07-31T15:15:18.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.147+0000 7f97adbee740 1 Processor -- start 2024-07-31T15:15:18.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.147+0000 7f97adbee740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3be1b0 0x55762b798110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.147+0000 7f97adbee740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b798650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.147+0000 7f97adbee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3e7070 0x55762b7a4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.147+0000 7f97adbee740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55762b555700 con 0x55762b3c6390 2024-07-31T15:15:18.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.147+0000 7f97adbee740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55762b6949e0 con 0x55762b3be1b0 2024-07-31T15:15:18.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.147+0000 7f97adbee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55762b79ad50 con 0x55762b3e7070 2024-07-31T15:15:18.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f97adbee740 10 client.0 _ll_register_callbacks cb 0x55762b6af4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:15:18.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f97a4c38640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b798650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f97a4c38640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b798650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44462/0 (socket says 172.21.15.67:44462) 2024-07-31T15:15:18.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f97a4c38640 1 -- 172.21.15.67:0/2880748471 learned_addr learned my addr 172.21.15.67:0/2880748471 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:18.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f97a5439640 1 --2- 172.21.15.67:0/2880748471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3be1b0 0x55762b798110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f97a5c3a640 1 --2- 172.21.15.67:0/2880748471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3e7070 0x55762b7a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f97a4c38640 1 -- 172.21.15.67:0/2880748471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3e7070 msgr2=0x55762b7a4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f97a4c38640 1 --2- 172.21.15.67:0/2880748471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3e7070 0x55762b7a4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f97a4c38640 1 -- 172.21.15.67:0/2880748471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3be1b0 msgr2=0x55762b798110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f97a4c38640 1 --2- 172.21.15.67:0/2880748471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3be1b0 0x55762b798110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f97a4c38640 1 -- 172.21.15.67:0/2880748471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55762b452650 con 0x55762b3c6390 2024-07-31T15:15:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f97a4c38640 1 --2- 172.21.15.67:0/2880748471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b798650 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f979c0aa330 tx=0x7f979c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b78f21b74ecf37 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f9788ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:15:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f9788ff9640 1 -- 172.21.15.67:0/2880748471 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0aa140 con 0x55762b3c6390 2024-07-31T15:15:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f9788ff9640 1 -- 172.21.15.67:0/2880748471 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f979c0a68c0 con 0x55762b3c6390 2024-07-31T15:15:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.148+0000 7f97adbee740 1 -- 172.21.15.67:0/2880748471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9794004230 con 0x55762b3c6390 2024-07-31T15:15:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.149+0000 7f9788ff9640 1 -- 172.21.15.67:0/2880748471 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f979c0c6020 con 0x55762b3c6390 2024-07-31T15:15:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.149+0000 7f97adbee740 10 client.20574 fetch_fsmap learned FSMap version 66 2024-07-31T15:15:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.149+0000 7f97adbee740 1 -- 172.21.15.67:0/2880748471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55762b7a6dc0 con 0x55762b3c6390 2024-07-31T15:15:18.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.149+0000 7f9788ff9640 1 -- 172.21.15.67:0/2880748471 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9794004230 con 0x55762b3c6390 2024-07-31T15:15:18.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.149+0000 7f9788ff9640 1 -- 172.21.15.67:0/2880748471 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f979c0a9890 con 0x55762b3c6390 2024-07-31T15:15:18.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.149+0000 7f97adbee740 10 client.20574 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:15:18.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.149+0000 7f97adbee740 10 client.20574 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:15:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.149+0000 7f97adbee740 20 client.20574 populate_metadata read hostname 'smithi067' 2024-07-31T15:15:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.149+0000 7f97adbee740 1 --2- 172.21.15.67:0/2880748471 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55762b3a59f0 0x55762b7aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.149+0000 7f97adbee740 4 client.20574 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:15:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.149+0000 7f97adbee740 1 -- 172.21.15.67:0/2880748471 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55762b704b00 con 0x55762b3a59f0 2024-07-31T15:15:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.150+0000 7f97a5439640 1 --2- 172.21.15.67:0/2880748471 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55762b3a59f0 0x55762b7aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.150+0000 7f97a5439640 1 --2- 172.21.15.67:0/2880748471 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55762b3a59f0 0x55762b7aa2b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db6a7a53e963c9d6 server_cookie=4c1f11d2e71b74ac in_seq=0 out_seq=0 2024-07-31T15:15:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.150+0000 7f9788ff9640 10 client.20574 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:15:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f9788ff9640 1 -- 172.21.15.67:0/2880748471 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55762b704b00 con 0x55762b3a59f0 2024-07-31T15:15:18.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f9788ff9640 10 client.20574 handle_command_reply: tid=0 2024-07-31T15:15:18.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f97adbee740 1 client.20574 shutdown 2024-07-31T15:15:18.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f97adbee740 1 -- 172.21.15.67:0/2880748471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 msgr2=0x55762b798650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:18.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f97adbee740 1 --2- 172.21.15.67:0/2880748471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b798650 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f979c0aa330 tx=0x7f979c003820 comp rx=0 tx=0).stop 2024-07-31T15:15:18.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f97adbee740 1 -- 172.21.15.67:0/2880748471 shutdown_connections 2024-07-31T15:15:18.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f97adbee740 1 --2- 172.21.15.67:0/2880748471 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55762b3a59f0 0x55762b7aa2b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f97adbee740 1 --2- 172.21.15.67:0/2880748471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3e7070 0x55762b7a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f9788ff9640 0 client.20574 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:15:18.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f97adbee740 1 --2- 172.21.15.67:0/2880748471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b798650 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f97adbee740 1 --2- 172.21.15.67:0/2880748471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3be1b0 0x55762b798110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f97adbee740 1 -- 172.21.15.67:0/2880748471 >> 172.21.15.67:0/2880748471 conn(0x55762b3db460 msgr2=0x55762b788c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:18.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f97adbee740 1 -- 172.21.15.67:0/2880748471 shutdown_connections 2024-07-31T15:15:18.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f97adbee740 1 -- 172.21.15.67:0/2880748471 wait complete. 2024-07-31T15:15:18.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.151+0000 7f97adbee740 20 client.20574 trim_cache size 0 max 16384 2024-07-31T15:15:18.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.165+0000 7f97adbee740 1 Processor -- start 2024-07-31T15:15:18.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.165+0000 7f97adbee740 1 -- start start 2024-07-31T15:15:18.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97adbee740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3a59f0 0x55762b78df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97adbee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3be1b0 0x55762b7a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97adbee740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b799b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97adbee740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01ae710 con 0x55762b3c6390 2024-07-31T15:15:18.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97adbee740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01b0250 con 0x55762b3a59f0 2024-07-31T15:15:18.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97adbee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01aede0 con 0x55762b3be1b0 2024-07-31T15:15:18.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97a5c3a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b799b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97a5c3a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b799b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44466/0 (socket says 172.21.15.67:44466) 2024-07-31T15:15:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97a5c3a640 1 -- 172.21.15.67:0/555606639 learned_addr learned my addr 172.21.15.67:0/555606639 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97a4c38640 1 --2- 172.21.15.67:0/555606639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3be1b0 0x55762b7a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97a5439640 1 --2- 172.21.15.67:0/555606639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3a59f0 0x55762b78df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97a5c3a640 1 -- 172.21.15.67:0/555606639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3be1b0 msgr2=0x55762b7a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97a5c3a640 1 --2- 172.21.15.67:0/555606639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3be1b0 0x55762b7a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97a5c3a640 1 -- 172.21.15.67:0/555606639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3a59f0 msgr2=0x55762b78df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:18.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97a5c3a640 1 --2- 172.21.15.67:0/555606639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3a59f0 0x55762b78df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97a5c3a640 1 -- 172.21.15.67:0/555606639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55762b78f3d0 con 0x55762b3c6390 2024-07-31T15:15:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97a4c38640 1 --2- 172.21.15.67:0/555606639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3be1b0 0x55762b7a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:15:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97a5439640 1 --2- 172.21.15.67:0/555606639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3a59f0 0x55762b78df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:15:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.166+0000 7f97a5c3a640 1 --2- 172.21.15.67:0/555606639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b799b40 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f9798009ca0 tx=0x7f9798041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef2d0dcf3134158 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.167+0000 7f978affd640 1 -- 172.21.15.67:0/555606639 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798003970 con 0x55762b3c6390 2024-07-31T15:15:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.167+0000 7f978affd640 1 -- 172.21.15.67:0/555606639 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9798036040 con 0x55762b3c6390 2024-07-31T15:15:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.167+0000 7f978affd640 1 -- 172.21.15.67:0/555606639 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979802f8c0 con 0x55762b3c6390 2024-07-31T15:15:18.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.167+0000 7f97adbee740 1 -- 172.21.15.67:0/555606639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 msgr2=0x55762b799b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.167+0000 7f97adbee740 1 --2- 172.21.15.67:0/555606639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b799b40 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f9798009ca0 tx=0x7f9798041000 comp rx=0 tx=0).stop 2024-07-31T15:15:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.167+0000 7f97adbee740 1 -- 172.21.15.67:0/555606639 shutdown_connections 2024-07-31T15:15:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.167+0000 7f97adbee740 1 --2- 172.21.15.67:0/555606639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b799b40 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.167+0000 7f97adbee740 1 --2- 172.21.15.67:0/555606639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3be1b0 0x55762b7a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.167+0000 7f97adbee740 1 --2- 172.21.15.67:0/555606639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3a59f0 0x55762b78df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.167+0000 7f97adbee740 1 -- 172.21.15.67:0/555606639 >> 172.21.15.67:0/555606639 conn(0x55762b3db460 msgr2=0x55762b799430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.167+0000 7f97adbee740 1 -- 172.21.15.67:0/555606639 shutdown_connections 2024-07-31T15:15:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.167+0000 7f97adbee740 1 -- 172.21.15.67:0/555606639 wait complete. 2024-07-31T15:15:18.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.168+0000 7f97adbee740 1 -- start start 2024-07-31T15:15:18.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.168+0000 7f97adbee740 1 Processor -- start 2024-07-31T15:15:18.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.168+0000 7f97adbee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3a59f0 0x55762b79bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.168+0000 7f97adbee740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3be1b0 0x55762b79c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.168+0000 7f97adbee740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b79c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.168+0000 7f97adbee740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01afb80 con 0x55762b3c6390 2024-07-31T15:15:18.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.168+0000 7f97adbee740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01af4b0 con 0x55762b3be1b0 2024-07-31T15:15:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.168+0000 7f97adbee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01ac710 con 0x55762b3a59f0 2024-07-31T15:15:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.168+0000 7f97adbee740 10 client.0 _ll_register_callbacks cb 0x7f979c00c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:15:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f97a5c3a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b79c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f97a5c3a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b79c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44470/0 (socket says 172.21.15.67:44470) 2024-07-31T15:15:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f97a5c3a640 1 -- 172.21.15.67:0/4169688647 learned_addr learned my addr 172.21.15.67:0/4169688647 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f97a4c38640 1 --2- 172.21.15.67:0/4169688647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3be1b0 0x55762b79c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f97a5439640 1 --2- 172.21.15.67:0/4169688647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3a59f0 0x55762b79bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f97a5c3a640 1 -- 172.21.15.67:0/4169688647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3a59f0 msgr2=0x55762b79bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f97a5c3a640 1 --2- 172.21.15.67:0/4169688647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3a59f0 0x55762b79bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f97a5c3a640 1 -- 172.21.15.67:0/4169688647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3be1b0 msgr2=0x55762b79c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f97a5c3a640 1 --2- 172.21.15.67:0/4169688647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3be1b0 0x55762b79c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f97a5c3a640 1 -- 172.21.15.67:0/4169688647 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55762b811b10 con 0x55762b3c6390 2024-07-31T15:15:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f97a5c3a640 1 --2- 172.21.15.67:0/4169688647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b79c9b0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f979802ade0 tx=0x7f9798004880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69a6ba3adbb8670 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f9788ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:15:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f9788ff9640 1 -- 172.21.15.67:0/4169688647 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979805c020 con 0x55762b3c6390 2024-07-31T15:15:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f9788ff9640 1 -- 172.21.15.67:0/4169688647 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f9798003c00 con 0x55762b3c6390 2024-07-31T15:15:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f97adbee740 1 -- 172.21.15.67:0/4169688647 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9798036040 con 0x55762b3c6390 2024-07-31T15:15:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f9788ff9640 1 -- 172.21.15.67:0/4169688647 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f9798067050 con 0x55762b3c6390 2024-07-31T15:15:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.169+0000 7f97adbee740 10 client.20580 fetch_fsmap learned FSMap version 66 2024-07-31T15:15:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.170+0000 7f97adbee740 1 -- 172.21.15.67:0/4169688647 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55762b812d90 con 0x55762b3c6390 2024-07-31T15:15:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.170+0000 7f9788ff9640 1 -- 172.21.15.67:0/4169688647 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9798036040 con 0x55762b3c6390 2024-07-31T15:15:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.170+0000 7f9788ff9640 1 -- 172.21.15.67:0/4169688647 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f9798030760 con 0x55762b3c6390 2024-07-31T15:15:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.170+0000 7f97adbee740 10 client.20580 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:15:18.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.170+0000 7f97adbee740 10 client.20580 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-07-31T15:15:18.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.170+0000 7f97adbee740 20 client.20580 populate_metadata read hostname 'smithi067' 2024-07-31T15:15:18.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.170+0000 7f97adbee740 1 --2- 172.21.15.67:0/4169688647 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55762b3e7070 0x55762b815540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:18.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.170+0000 7f97adbee740 4 client.20580 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:15:18.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.170+0000 7f97adbee740 1 -- 172.21.15.67:0/4169688647 --> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55762b7005a0 con 0x55762b3e7070 2024-07-31T15:15:18.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.170+0000 7f97a5439640 1 --2- 172.21.15.67:0/4169688647 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55762b3e7070 0x55762b815540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:18.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f9788ff9640 10 client.20580 ms_handle_connect on v2:172.21.15.77:6838/2802806156 2024-07-31T15:15:18.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f97a5439640 1 --2- 172.21.15.67:0/4169688647 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55762b3e7070 0x55762b815540 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61806a4ee5869272 server_cookie=c3d963a3dad6d124 in_seq=0 out_seq=0 2024-07-31T15:15:18.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f9788ff9640 1 -- 172.21.15.67:0/4169688647 <== mds.1 v2:172.21.15.77:6838/2802806156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55762b7005a0 con 0x55762b3e7070 2024-07-31T15:15:18.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f9788ff9640 10 client.20580 handle_command_reply: tid=0 2024-07-31T15:15:18.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f97adbee740 1 client.20580 shutdown 2024-07-31T15:15:18.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f97adbee740 1 -- 172.21.15.67:0/4169688647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 msgr2=0x55762b79c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:18.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f97adbee740 1 --2- 172.21.15.67:0/4169688647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b79c9b0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f979802ade0 tx=0x7f9798004880 comp rx=0 tx=0).stop 2024-07-31T15:15:18.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f97adbee740 1 -- 172.21.15.67:0/4169688647 shutdown_connections 2024-07-31T15:15:18.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f97adbee740 1 --2- 172.21.15.67:0/4169688647 >> [v2:172.21.15.77:6838/2802806156,v1:172.21.15.77:6839/2802806156] conn(0x55762b3e7070 0x55762b815540 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f97adbee740 1 --2- 172.21.15.67:0/4169688647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55762b3c6390 0x55762b79c9b0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f97adbee740 1 --2- 172.21.15.67:0/4169688647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55762b3be1b0 0x55762b79c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f97adbee740 1 --2- 172.21.15.67:0/4169688647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55762b3a59f0 0x55762b79bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f97adbee740 1 -- 172.21.15.67:0/4169688647 >> 172.21.15.67:0/4169688647 conn(0x55762b3db460 msgr2=0x55762b799b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:18.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f9788ff9640 0 client.20580 ms_handle_reset on v2:172.21.15.77:6838/2802806156 2024-07-31T15:15:18.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f97adbee740 1 -- 172.21.15.67:0/4169688647 shutdown_connections 2024-07-31T15:15:18.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f97adbee740 1 -- 172.21.15.67:0/4169688647 wait complete. 2024-07-31T15:15:18.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.171+0000 7f97adbee740 20 client.20580 trim_cache size 0 max 16384 2024-07-31T15:15:18.183 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:15:18.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.172+0000 7f97a769d640 1 -- 172.21.15.67:0/2993817740 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f978c0b3f90 msgr2=0x7f978c0b64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:18.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.172+0000 7f97a769d640 1 --2- 172.21.15.67:0/2993817740 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f978c0b3f90 0x7f978c0b64a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9798005cc0 tx=0x7f97980095c0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.172+0000 7f97a769d640 1 -- 172.21.15.67:0/2993817740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97a0107df0 msgr2=0x7f97a019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:18.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.172+0000 7f97a769d640 1 --2- 172.21.15.67:0/2993817740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97a0107df0 0x7f97a019a4c0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f979c00c970 tx=0x7f979c00ce30 comp rx=0 tx=0).stop 2024-07-31T15:15:18.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.172+0000 7f97a769d640 1 -- 172.21.15.67:0/2993817740 shutdown_connections 2024-07-31T15:15:18.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.172+0000 7f97a769d640 1 --2- 172.21.15.67:0/2993817740 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f978c0b3f90 0x7f978c0b64a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.172+0000 7f97a769d640 1 --2- 172.21.15.67:0/2993817740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97a0108740 0x7f97a019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.172+0000 7f97a769d640 1 --2- 172.21.15.67:0/2993817740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97a0107df0 0x7f97a019a4c0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.172+0000 7f97a769d640 1 --2- 172.21.15.67:0/2993817740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97a01024b0 0x7f97a0199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.172+0000 7f97a769d640 1 -- 172.21.15.67:0/2993817740 >> 172.21.15.67:0/2993817740 conn(0x7f97a0079170 msgr2=0x7f97a0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.172+0000 7f97a769d640 1 -- 172.21.15.67:0/2993817740 shutdown_connections 2024-07-31T15:15:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:18.172+0000 7f97a769d640 1 -- 172.21.15.67:0/2993817740 wait complete. 2024-07-31T15:15:19.073 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-31T15:15:19.343 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:15:19.725 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:19 smithi067 ceph-mon[32750]: pgmap v1174: 129 pgs: 5 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 106 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-07-31T15:15:19.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:19 smithi077 ceph-mon[37327]: pgmap v1174: 129 pgs: 5 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 106 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-07-31T15:15:19.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:19 smithi073 ceph-mon[36714]: pgmap v1174: 129 pgs: 5 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 106 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-07-31T15:15:21.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.359+0000 7f7d33857640 1 -- 172.21.15.67:0/1703127683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d2c108760 msgr2=0x7f7d2c10ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:21.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.359+0000 7f7d33857640 1 --2- 172.21.15.67:0/1703127683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d2c108760 0x7f7d2c10ef90 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009ca0 tx=0x7f7d2402a8c0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.359+0000 7f7d33857640 1 -- 172.21.15.67:0/1703127683 shutdown_connections 2024-07-31T15:15:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.359+0000 7f7d33857640 1 --2- 172.21.15.67:0/1703127683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d2c108760 0x7f7d2c10ef90 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.359+0000 7f7d33857640 1 --2- 172.21.15.67:0/1703127683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d2c107e10 0x7f7d2c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.359+0000 7f7d33857640 1 --2- 172.21.15.67:0/1703127683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d2c1024d0 0x7f7d2c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.359+0000 7f7d33857640 1 -- 172.21.15.67:0/1703127683 >> 172.21.15.67:0/1703127683 conn(0x7f7d2c079190 msgr2=0x7f7d2c0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.360+0000 7f7d33857640 1 -- 172.21.15.67:0/1703127683 shutdown_connections 2024-07-31T15:15:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.360+0000 7f7d33857640 1 -- 172.21.15.67:0/1703127683 wait complete. 2024-07-31T15:15:21.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.360+0000 7f7d33857640 1 Processor -- start 2024-07-31T15:15:21.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.360+0000 7f7d33857640 1 -- start start 2024-07-31T15:15:21.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.360+0000 7f7d33857640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d2c1024d0 0x7f7d2c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.360+0000 7f7d33857640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d2c107e10 0x7f7d2c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.360+0000 7f7d33857640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d2c108760 0x7f7d2c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.360+0000 7f7d33857640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1123e0 con 0x7f7d2c108760 2024-07-31T15:15:21.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.360+0000 7f7d33857640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c112260 con 0x7f7d2c107e10 2024-07-31T15:15:21.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d33857640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c112560 con 0x7f7d2c1024d0 2024-07-31T15:15:21.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d31df4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d2c108760 0x7f7d2c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d31df4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d2c108760 0x7f7d2c19f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36624/0 (socket says 172.21.15.67:36624) 2024-07-31T15:15:21.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d31df4640 1 -- 172.21.15.67:0/2402741040 learned_addr learned my addr 172.21.15.67:0/2402741040 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:21.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d30df2640 1 --2- 172.21.15.67:0/2402741040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d2c107e10 0x7f7d2c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d315f3640 1 --2- 172.21.15.67:0/2402741040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d2c1024d0 0x7f7d2c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d31df4640 1 -- 172.21.15.67:0/2402741040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d2c1024d0 msgr2=0x7f7d2c199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:21.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d31df4640 1 --2- 172.21.15.67:0/2402741040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d2c1024d0 0x7f7d2c199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d31df4640 1 -- 172.21.15.67:0/2402741040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d2c107e10 msgr2=0x7f7d2c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:21.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d31df4640 1 --2- 172.21.15.67:0/2402741040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d2c107e10 0x7f7d2c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d31df4640 1 -- 172.21.15.67:0/2402741040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2c19fe10 con 0x7f7d2c108760 2024-07-31T15:15:21.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d31df4640 1 --2- 172.21.15.67:0/2402741040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d2c108760 0x7f7d2c19f6e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f7d24005cc0 tx=0x7f7d2402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:21.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d227fc640 1 -- 172.21.15.67:0/2402741040 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24003bf0 con 0x7f7d2c108760 2024-07-31T15:15:21.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d227fc640 1 -- 172.21.15.67:0/2402741040 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d24003d90 con 0x7f7d2c108760 2024-07-31T15:15:21.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d227fc640 1 -- 172.21.15.67:0/2402741040 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2402fbc0 con 0x7f7d2c108760 2024-07-31T15:15:21.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d33857640 1 -- 172.21.15.67:0/2402741040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2c1a00a0 con 0x7f7d2c108760 2024-07-31T15:15:21.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.361+0000 7f7d33857640 1 -- 172.21.15.67:0/2402741040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2c1a6860 con 0x7f7d2c108760 2024-07-31T15:15:21.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.364+0000 7f7d39da8740 1 Processor -- start 2024-07-31T15:15:21.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.364+0000 7f7d227fc640 1 -- 172.21.15.67:0/2402741040 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7d2402fd60 con 0x7f7d2c108760 2024-07-31T15:15:21.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.364+0000 7f7d39da8740 1 -- start start 2024-07-31T15:15:21.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d39da8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b3f070 0x55bc58ee4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d39da8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b161b0 0x55bc58ee4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d39da8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b1e390 0x55bc58ee4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d39da8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bc58e5abb0 con 0x55bc58b3f070 2024-07-31T15:15:21.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d39da8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bc58e4a0b0 con 0x55bc58b161b0 2024-07-31T15:15:21.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d39da8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bc58e4e0f0 con 0x55bc58b1e390 2024-07-31T15:15:21.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d30df2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b3f070 0x55bc58ee4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d30df2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b3f070 0x55bc58ee4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36632/0 (socket says 172.21.15.67:36632) 2024-07-31T15:15:21.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d30df2640 1 -- 172.21.15.67:0/1963018283 learned_addr learned my addr 172.21.15.67:0/1963018283 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:21.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d315f3640 1 --2- 172.21.15.67:0/1963018283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b161b0 0x55bc58ee4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d30df2640 1 --2- 172.21.15.67:0/1963018283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b1e390 0x55bc58ee4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d30df2640 1 -- 172.21.15.67:0/1963018283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b1e390 msgr2=0x55bc58ee4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:21.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d30df2640 1 --2- 172.21.15.67:0/1963018283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b1e390 0x55bc58ee4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d30df2640 1 -- 172.21.15.67:0/1963018283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b161b0 msgr2=0x55bc58ee4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:21.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d30df2640 1 --2- 172.21.15.67:0/1963018283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b161b0 0x55bc58ee4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d30df2640 1 -- 172.21.15.67:0/1963018283 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc58df0b90 con 0x55bc58b3f070 2024-07-31T15:15:21.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d227fc640 1 --2- 172.21.15.67:0/2402741040 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7d180b3dc0 0x7f7d180b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.365+0000 7f7d227fc640 1 -- 172.21.15.67:0/2402741040 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f7d241059b0 con 0x7f7d2c108760 2024-07-31T15:15:21.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.366+0000 7f7d30df2640 1 --2- 172.21.15.67:0/2402741040 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7d180b3dc0 0x7f7d180b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.366+0000 7f7d30df2640 1 --2- 172.21.15.67:0/1963018283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b3f070 0x55bc58ee4570 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00afd0 tx=0x7f7d1c00a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129a6133e05e09c4 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:21.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.366+0000 7f7d177fe640 1 -- 172.21.15.67:0/1963018283 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00f6f0 con 0x55bc58b3f070 2024-07-31T15:15:21.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.366+0000 7f7d177fe640 1 -- 172.21.15.67:0/1963018283 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d1c0129c0 con 0x55bc58b3f070 2024-07-31T15:15:21.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.366+0000 7f7d177fe640 1 -- 172.21.15.67:0/1963018283 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c022710 con 0x55bc58b3f070 2024-07-31T15:15:21.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.367+0000 7f7d39da8740 1 -- 172.21.15.67:0/1963018283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b3f070 msgr2=0x55bc58ee4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:21.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.367+0000 7f7d39da8740 1 --2- 172.21.15.67:0/1963018283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b3f070 0x55bc58ee4570 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00afd0 tx=0x7f7d1c00a6a0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.367+0000 7f7d30df2640 1 --2- 172.21.15.67:0/2402741040 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7d180b3dc0 0x7f7d180b62d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c021000 tx=0x7f7d1c01e000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:21.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.367+0000 7f7d39da8740 1 -- 172.21.15.67:0/1963018283 shutdown_connections 2024-07-31T15:15:21.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.367+0000 7f7d39da8740 1 --2- 172.21.15.67:0/1963018283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b1e390 0x55bc58ee4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.367+0000 7f7d39da8740 1 --2- 172.21.15.67:0/1963018283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b161b0 0x55bc58ee4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.367+0000 7f7d39da8740 1 --2- 172.21.15.67:0/1963018283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b3f070 0x55bc58ee4570 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.367+0000 7f7d39da8740 1 -- 172.21.15.67:0/1963018283 >> 172.21.15.67:0/1963018283 conn(0x55bc58b33460 msgr2=0x55bc58ee2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:21.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.367+0000 7f7d39da8740 1 -- 172.21.15.67:0/1963018283 shutdown_connections 2024-07-31T15:15:21.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.367+0000 7f7d39da8740 1 -- 172.21.15.67:0/1963018283 wait complete. 2024-07-31T15:15:21.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.367+0000 7f7d39da8740 1 -- start start 2024-07-31T15:15:21.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.367+0000 7f7d39da8740 1 Processor -- start 2024-07-31T15:15:21.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d39da8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b161b0 0x55bc58ef01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d39da8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b1e390 0x55bc58ef0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d39da8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b3f070 0x55bc58efc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d39da8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bc58cad700 con 0x55bc58b161b0 2024-07-31T15:15:21.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d39da8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bc58dec9e0 con 0x55bc58b3f070 2024-07-31T15:15:21.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d39da8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bc58ef2d50 con 0x55bc58b1e390 2024-07-31T15:15:21.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d39da8740 10 client.0 _ll_register_callbacks cb 0x55bc58e074f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:15:21.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d315f3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b161b0 0x55bc58ef01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d315f3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b161b0 0x55bc58ef01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36642/0 (socket says 172.21.15.67:36642) 2024-07-31T15:15:21.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d315f3640 1 -- 172.21.15.67:0/3580551005 learned_addr learned my addr 172.21.15.67:0/3580551005 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:21.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d30df2640 1 --2- 172.21.15.67:0/3580551005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b1e390 0x55bc58ef0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d31df4640 1 --2- 172.21.15.67:0/3580551005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b3f070 0x55bc58efc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d315f3640 1 -- 172.21.15.67:0/3580551005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b1e390 msgr2=0x55bc58ef0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:21.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d315f3640 1 --2- 172.21.15.67:0/3580551005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b1e390 0x55bc58ef0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d315f3640 1 -- 172.21.15.67:0/3580551005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b3f070 msgr2=0x55bc58efc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:21.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d315f3640 1 --2- 172.21.15.67:0/3580551005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b3f070 0x55bc58efc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.368+0000 7f7d315f3640 1 -- 172.21.15.67:0/3580551005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc58baa650 con 0x55bc58b161b0 2024-07-31T15:15:21.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.369+0000 7f7d315f3640 1 --2- 172.21.15.67:0/3580551005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b161b0 0x55bc58ef01c0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009e00 tx=0x7f7d280117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefb3d73651542b0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:21.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.369+0000 7f7d14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:15:21.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.369+0000 7f7d14ff9640 1 -- 172.21.15.67:0/3580551005 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28012860 con 0x55bc58b161b0 2024-07-31T15:15:21.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.369+0000 7f7d14ff9640 1 -- 172.21.15.67:0/3580551005 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f7d28012a00 con 0x55bc58b161b0 2024-07-31T15:15:21.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.369+0000 7f7d39da8740 1 -- 172.21.15.67:0/3580551005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d1c0129c0 con 0x55bc58b161b0 2024-07-31T15:15:21.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.369+0000 7f7d14ff9640 1 -- 172.21.15.67:0/3580551005 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f7d28025050 con 0x55bc58b161b0 2024-07-31T15:15:21.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.369+0000 7f7d39da8740 10 client.20592 fetch_fsmap learned FSMap version 66 2024-07-31T15:15:21.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.369+0000 7f7d39da8740 1 -- 172.21.15.67:0/3580551005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55bc58efedd0 con 0x55bc58b161b0 2024-07-31T15:15:21.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.369+0000 7f7d14ff9640 1 -- 172.21.15.67:0/3580551005 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d1c0129c0 con 0x55bc58b161b0 2024-07-31T15:15:21.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.369+0000 7f7d14ff9640 1 -- 172.21.15.67:0/3580551005 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f7d2800f040 con 0x55bc58b161b0 2024-07-31T15:15:21.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.369+0000 7f7d39da8740 10 client.20592 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:15:21.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.369+0000 7f7d39da8740 10 client.20592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:15:21.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.369+0000 7f7d39da8740 20 client.20592 populate_metadata read hostname 'smithi067' 2024-07-31T15:15:21.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.370+0000 7f7d39da8740 1 --2- 172.21.15.67:0/3580551005 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc58afd9f0 0x55bc58f02350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.370+0000 7f7d39da8740 4 client.20592 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-31T15:15:21.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.370+0000 7f7d39da8740 1 -- 172.21.15.67:0/3580551005 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc58e5cb00 con 0x55bc58afd9f0 2024-07-31T15:15:21.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.370+0000 7f7d30df2640 1 --2- 172.21.15.67:0/3580551005 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc58afd9f0 0x55bc58f02350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.371+0000 7f7d30df2640 1 --2- 172.21.15.67:0/3580551005 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc58afd9f0 0x55bc58f02350 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b06a4bc965b413 server_cookie=1d359aa9dfa412c1 in_seq=0 out_seq=0 2024-07-31T15:15:21.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.371+0000 7f7d14ff9640 10 client.20592 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:15:21.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.371+0000 7f7d14ff9640 1 -- 172.21.15.67:0/3580551005 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc58e5cb00 con 0x55bc58afd9f0 2024-07-31T15:15:21.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.371+0000 7f7d14ff9640 10 client.20592 handle_command_reply: tid=0 2024-07-31T15:15:21.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.371+0000 7f7d39da8740 1 client.20592 shutdown 2024-07-31T15:15:21.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.371+0000 7f7d39da8740 1 -- 172.21.15.67:0/3580551005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b161b0 msgr2=0x55bc58ef01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:21.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.371+0000 7f7d39da8740 1 --2- 172.21.15.67:0/3580551005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b161b0 0x55bc58ef01c0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009e00 tx=0x7f7d280117b0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.372+0000 7f7d39da8740 1 -- 172.21.15.67:0/3580551005 shutdown_connections 2024-07-31T15:15:21.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.372+0000 7f7d39da8740 1 --2- 172.21.15.67:0/3580551005 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc58afd9f0 0x55bc58f02350 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.372+0000 7f7d39da8740 1 --2- 172.21.15.67:0/3580551005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b3f070 0x55bc58efc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.372+0000 7f7d39da8740 1 --2- 172.21.15.67:0/3580551005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b1e390 0x55bc58ef0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.372+0000 7f7d39da8740 1 --2- 172.21.15.67:0/3580551005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b161b0 0x55bc58ef01c0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.372+0000 7f7d39da8740 1 -- 172.21.15.67:0/3580551005 >> 172.21.15.67:0/3580551005 conn(0x55bc58b33460 msgr2=0x55bc58ee0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:21.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.372+0000 7f7d14ff9640 0 client.20592 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:15:21.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.372+0000 7f7d39da8740 1 -- 172.21.15.67:0/3580551005 shutdown_connections 2024-07-31T15:15:21.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.372+0000 7f7d39da8740 1 -- 172.21.15.67:0/3580551005 wait complete. 2024-07-31T15:15:21.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.372+0000 7f7d39da8740 20 client.20592 trim_cache size 0 max 16384 2024-07-31T15:15:21.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.386+0000 7f7d39da8740 1 Processor -- start 2024-07-31T15:15:21.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.386+0000 7f7d39da8740 1 -- start start 2024-07-31T15:15:21.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.386+0000 7f7d39da8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b1e390 0x55bc58ee1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.386+0000 7f7d39da8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b3f070 0x55bc58efbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.386+0000 7f7d39da8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58ad2da0 0x55bc58ee9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.386+0000 7f7d39da8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1b0880 con 0x55bc58b3f070 2024-07-31T15:15:21.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.386+0000 7f7d39da8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1b23c0 con 0x55bc58b1e390 2024-07-31T15:15:21.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.386+0000 7f7d39da8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1b0f50 con 0x55bc58ad2da0 2024-07-31T15:15:21.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.386+0000 7f7d30df2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b3f070 0x55bc58efbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.386+0000 7f7d315f3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b1e390 0x55bc58ee1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.386+0000 7f7d30df2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b3f070 0x55bc58efbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36654/0 (socket says 172.21.15.67:36654) 2024-07-31T15:15:21.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.386+0000 7f7d30df2640 1 -- 172.21.15.67:0/1767250854 learned_addr learned my addr 172.21.15.67:0/1767250854 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:21.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.386+0000 7f7d31df4640 1 --2- 172.21.15.67:0/1767250854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58ad2da0 0x55bc58ee9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.387+0000 7f7d315f3640 1 -- 172.21.15.67:0/1767250854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58ad2da0 msgr2=0x55bc58ee9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:21.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.387+0000 7f7d315f3640 1 --2- 172.21.15.67:0/1767250854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58ad2da0 0x55bc58ee9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.387+0000 7f7d315f3640 1 -- 172.21.15.67:0/1767250854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b3f070 msgr2=0x55bc58efbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:21.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.387+0000 7f7d315f3640 1 --2- 172.21.15.67:0/1767250854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b3f070 0x55bc58efbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.387+0000 7f7d315f3640 1 -- 172.21.15.67:0/1767250854 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc58ee6860 con 0x55bc58b1e390 2024-07-31T15:15:21.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.387+0000 7f7d30df2640 1 --2- 172.21.15.67:0/1767250854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b3f070 0x55bc58efbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:15:21.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.387+0000 7f7d315f3640 1 --2- 172.21.15.67:0/1767250854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b1e390 0x55bc58ee1350 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800b000 tx=0x7f7d280099f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2c5ae1d55413830 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:21.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.387+0000 7f7d16ffd640 1 -- 172.21.15.67:0/1767250854 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28005850 con 0x55bc58b1e390 2024-07-31T15:15:21.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.387+0000 7f7d16ffd640 1 -- 172.21.15.67:0/1767250854 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d28011cb0 con 0x55bc58b1e390 2024-07-31T15:15:21.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.387+0000 7f7d16ffd640 1 -- 172.21.15.67:0/1767250854 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d280196c0 con 0x55bc58b1e390 2024-07-31T15:15:21.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.388+0000 7f7d39da8740 1 -- 172.21.15.67:0/1767250854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b1e390 msgr2=0x55bc58ee1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:21.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.388+0000 7f7d39da8740 1 --2- 172.21.15.67:0/1767250854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b1e390 0x55bc58ee1350 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800b000 tx=0x7f7d280099f0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.388+0000 7f7d39da8740 1 -- 172.21.15.67:0/1767250854 shutdown_connections 2024-07-31T15:15:21.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.388+0000 7f7d39da8740 1 --2- 172.21.15.67:0/1767250854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58ad2da0 0x55bc58ee9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.388+0000 7f7d39da8740 1 --2- 172.21.15.67:0/1767250854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58b3f070 0x55bc58efbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.388+0000 7f7d39da8740 1 --2- 172.21.15.67:0/1767250854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b1e390 0x55bc58ee1350 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.388+0000 7f7d39da8740 1 -- 172.21.15.67:0/1767250854 >> 172.21.15.67:0/1767250854 conn(0x55bc58b33460 msgr2=0x55bc58ee0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:21.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.388+0000 7f7d39da8740 1 -- 172.21.15.67:0/1767250854 shutdown_connections 2024-07-31T15:15:21.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.388+0000 7f7d39da8740 1 -- 172.21.15.67:0/1767250854 wait complete. 2024-07-31T15:15:21.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.389+0000 7f7d39da8740 1 -- start start 2024-07-31T15:15:21.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.389+0000 7f7d39da8740 1 Processor -- start 2024-07-31T15:15:21.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.389+0000 7f7d39da8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58ad2da0 0x55bc58ef41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.389+0000 7f7d39da8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b1e390 0x55bc58ef4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.389+0000 7f7d39da8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b3f070 0x55bc58f69b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.389+0000 7f7d39da8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1b1cf0 con 0x55bc58ad2da0 2024-07-31T15:15:21.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.389+0000 7f7d39da8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1b1620 con 0x55bc58b1e390 2024-07-31T15:15:21.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.389+0000 7f7d39da8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1ae880 con 0x55bc58b3f070 2024-07-31T15:15:21.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.389+0000 7f7d39da8740 10 client.0 _ll_register_callbacks cb 0x7f7d28009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-31T15:15:21.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.389+0000 7f7d315f3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58ad2da0 0x55bc58ef41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d31df4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b3f070 0x55bc58f69b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d315f3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58ad2da0 0x55bc58ef41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36666/0 (socket says 172.21.15.67:36666) 2024-07-31T15:15:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d315f3640 1 -- 172.21.15.67:0/2963665745 learned_addr learned my addr 172.21.15.67:0/2963665745 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d30df2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b1e390 0x55bc58ef4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d315f3640 1 -- 172.21.15.67:0/2963665745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b3f070 msgr2=0x55bc58f69b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d315f3640 1 --2- 172.21.15.67:0/2963665745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b3f070 0x55bc58f69b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d315f3640 1 -- 172.21.15.67:0/2963665745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b1e390 msgr2=0x55bc58ef4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-31T15:15:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d315f3640 1 --2- 172.21.15.67:0/2963665745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b1e390 0x55bc58ef4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d315f3640 1 -- 172.21.15.67:0/2963665745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc58ef4fb0 con 0x55bc58ad2da0 2024-07-31T15:15:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d31df4640 1 --2- 172.21.15.67:0/2963665745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b3f070 0x55bc58f69b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-31T15:15:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d315f3640 1 --2- 172.21.15.67:0/2963665745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58ad2da0 0x55bc58ef41f0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7d2801af20 tx=0x7f7d2801a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f603b56102f2ed0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-31T15:15:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d14ff9640 1 -- 172.21.15.67:0/2963665745 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28035030 con 0x55bc58ad2da0 2024-07-31T15:15:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d14ff9640 1 -- 172.21.15.67:0/2963665745 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f7d2801aa60 con 0x55bc58ad2da0 2024-07-31T15:15:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d39da8740 1 -- 172.21.15.67:0/2963665745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d28011cb0 con 0x55bc58ad2da0 2024-07-31T15:15:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d14ff9640 1 -- 172.21.15.67:0/2963665745 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f7d28042050 con 0x55bc58ad2da0 2024-07-31T15:15:21.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.390+0000 7f7d39da8740 10 client.20598 fetch_fsmap learned FSMap version 66 2024-07-31T15:15:21.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.391+0000 7f7d39da8740 1 -- 172.21.15.67:0/2963665745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55bc58f6b070 con 0x55bc58ad2da0 2024-07-31T15:15:21.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.391+0000 7f7d14ff9640 1 -- 172.21.15.67:0/2963665745 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d28011cb0 con 0x55bc58ad2da0 2024-07-31T15:15:21.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.391+0000 7f7d14ff9640 1 -- 172.21.15.67:0/2963665745 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f7d28030040 con 0x55bc58ad2da0 2024-07-31T15:15:21.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.391+0000 7f7d39da8740 10 client.20598 fetch_fsmap finished waiting for FSMap version 66 2024-07-31T15:15:21.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.391+0000 7f7d39da8740 10 client.20598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-31T15:15:21.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.391+0000 7f7d39da8740 20 client.20598 populate_metadata read hostname 'smithi067' 2024-07-31T15:15:21.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.391+0000 7f7d39da8740 1 --2- 172.21.15.67:0/2963665745 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc58f6b6d0 0x55bc58f6daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:21.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.391+0000 7f7d39da8740 4 client.20598 mds_command: new command op to 24671 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-31T15:15:21.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.391+0000 7f7d39da8740 1 -- 172.21.15.67:0/2963665745 --> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc58eebb70 con 0x55bc58f6b6d0 2024-07-31T15:15:21.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.391+0000 7f7d30df2640 1 --2- 172.21.15.67:0/2963665745 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc58f6b6d0 0x55bc58f6daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:21.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d14ff9640 10 client.20598 ms_handle_connect on v2:172.21.15.73:6838/4131046421 2024-07-31T15:15:21.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d30df2640 1 --2- 172.21.15.67:0/2963665745 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc58f6b6d0 0x55bc58f6daf0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79878c797e128c86 server_cookie=9c4e46963cc1d6e1 in_seq=0 out_seq=0 2024-07-31T15:15:21.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d14ff9640 1 -- 172.21.15.67:0/2963665745 <== mds.0 v2:172.21.15.73:6838/4131046421 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc58eebb70 con 0x55bc58f6b6d0 2024-07-31T15:15:21.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d14ff9640 10 client.20598 handle_command_reply: tid=0 2024-07-31T15:15:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d39da8740 1 client.20598 shutdown 2024-07-31T15:15:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d39da8740 1 -- 172.21.15.67:0/2963665745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58ad2da0 msgr2=0x55bc58ef41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d39da8740 1 --2- 172.21.15.67:0/2963665745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58ad2da0 0x55bc58ef41f0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7d2801af20 tx=0x7f7d2801a3f0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d39da8740 1 -- 172.21.15.67:0/2963665745 shutdown_connections 2024-07-31T15:15:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d39da8740 1 --2- 172.21.15.67:0/2963665745 >> [v2:172.21.15.73:6838/4131046421,v1:172.21.15.73:6839/4131046421] conn(0x55bc58f6b6d0 0x55bc58f6daf0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d39da8740 1 --2- 172.21.15.67:0/2963665745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc58b3f070 0x55bc58f69b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d39da8740 1 --2- 172.21.15.67:0/2963665745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc58b1e390 0x55bc58ef4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d39da8740 1 --2- 172.21.15.67:0/2963665745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc58ad2da0 0x55bc58ef41f0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d39da8740 1 -- 172.21.15.67:0/2963665745 >> 172.21.15.67:0/2963665745 conn(0x55bc58b33460 msgr2=0x55bc58ee1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:21.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d14ff9640 0 client.20598 ms_handle_reset on v2:172.21.15.73:6838/4131046421 2024-07-31T15:15:21.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d39da8740 1 -- 172.21.15.67:0/2963665745 shutdown_connections 2024-07-31T15:15:21.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d39da8740 1 -- 172.21.15.67:0/2963665745 wait complete. 2024-07-31T15:15:21.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.392+0000 7f7d39da8740 20 client.20598 trim_cache size 0 max 16384 2024-07-31T15:15:21.409 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-31T15:15:21.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.393+0000 7f7d33857640 1 -- 172.21.15.67:0/2402741040 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7d180b3dc0 msgr2=0x7f7d180b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:21.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.393+0000 7f7d33857640 1 --2- 172.21.15.67:0/2402741040 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7d180b3dc0 0x7f7d180b62d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c021000 tx=0x7f7d1c01e000 comp rx=0 tx=0).stop 2024-07-31T15:15:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.393+0000 7f7d33857640 1 -- 172.21.15.67:0/2402741040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d2c108760 msgr2=0x7f7d2c19f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.393+0000 7f7d33857640 1 --2- 172.21.15.67:0/2402741040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d2c108760 0x7f7d2c19f6e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f7d24005cc0 tx=0x7f7d2402f380 comp rx=0 tx=0).stop 2024-07-31T15:15:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.393+0000 7f7d33857640 1 -- 172.21.15.67:0/2402741040 shutdown_connections 2024-07-31T15:15:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.393+0000 7f7d33857640 1 --2- 172.21.15.67:0/2402741040 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f7d180b3dc0 0x7f7d180b62d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.393+0000 7f7d33857640 1 --2- 172.21.15.67:0/2402741040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d2c108760 0x7f7d2c19f6e0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.393+0000 7f7d33857640 1 --2- 172.21.15.67:0/2402741040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d2c107e10 0x7f7d2c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.393+0000 7f7d33857640 1 --2- 172.21.15.67:0/2402741040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d2c1024d0 0x7f7d2c199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.393+0000 7f7d33857640 1 -- 172.21.15.67:0/2402741040 >> 172.21.15.67:0/2402741040 conn(0x7f7d2c079190 msgr2=0x7f7d2c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.393+0000 7f7d33857640 1 -- 172.21.15.67:0/2402741040 shutdown_connections 2024-07-31T15:15:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:21.393+0000 7f7d33857640 1 -- 172.21.15.67:0/2402741040 wait complete. 2024-07-31T15:15:21.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:21 smithi067 ceph-mon[32750]: pgmap v1175: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-07-31T15:15:21.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:21 smithi077 ceph-mon[37327]: pgmap v1175: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-07-31T15:15:21.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:21 smithi073 ceph-mon[36714]: pgmap v1175: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-07-31T15:15:22.290 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_8c796fca5ff5960f7542e7227bad4a784ee2c804/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_8c796fca5ff5960f7542e7227bad4a784ee2c804/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_8c796fca5ff5960f7542e7227bad4a784ee2c804/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-07-31T15:15:22.296 INFO:tasks.fwd_scrub:done joining 2024-07-31T15:15:22.297 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-07-31T15:15:22.311 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-31T15:15:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.430+0000 7f5864aa9640 1 -- 172.21.15.67:0/1330139780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5860104d00 msgr2=0x7f586010d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.430+0000 7f5864aa9640 1 --2- 172.21.15.67:0/1330139780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5860104d00 0x7f586010d240 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f5850009900 tx=0x7f585002a410 comp rx=0 tx=0).stop 2024-07-31T15:15:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.430+0000 7f5864aa9640 1 -- 172.21.15.67:0/1330139780 shutdown_connections 2024-07-31T15:15:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.430+0000 7f5864aa9640 1 --2- 172.21.15.67:0/1330139780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f586010d7d0 0x7f586010fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.430+0000 7f5864aa9640 1 --2- 172.21.15.67:0/1330139780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5860104d00 0x7f586010d240 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.430+0000 7f5864aa9640 1 --2- 172.21.15.67:0/1330139780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58601043b0 0x7f58601047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.430+0000 7f5864aa9640 1 -- 172.21.15.67:0/1330139780 >> 172.21.15.67:0/1330139780 conn(0x7f58600fdde0 msgr2=0x7f5860100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:22.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.430+0000 7f5864aa9640 1 -- 172.21.15.67:0/1330139780 shutdown_connections 2024-07-31T15:15:22.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.430+0000 7f5864aa9640 1 -- 172.21.15.67:0/1330139780 wait complete. 2024-07-31T15:15:22.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.431+0000 7f5864aa9640 1 Processor -- start 2024-07-31T15:15:22.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.431+0000 7f5864aa9640 1 -- start start 2024-07-31T15:15:22.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.431+0000 7f5864aa9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58601043b0 0x7f58601020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:22.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.431+0000 7f5864aa9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5860104d00 0x7f5860102620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:22.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.431+0000 7f5864aa9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f586010d7d0 0x7f5860102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:22.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.431+0000 7f5864aa9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5860113400 con 0x7f5860104d00 2024-07-31T15:15:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.431+0000 7f5864aa9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5860113280 con 0x7f58601043b0 2024-07-31T15:15:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.431+0000 7f5864aa9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5860113580 con 0x7f586010d7d0 2024-07-31T15:15:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.431+0000 7f585dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5860104d00 0x7f5860102620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.431+0000 7f585dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5860104d00 0x7f5860102620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36680/0 (socket says 172.21.15.67:36680) 2024-07-31T15:15:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.431+0000 7f585dd9b640 1 -- 172.21.15.67:0/543966205 learned_addr learned my addr 172.21.15.67:0/543966205 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.431+0000 7f585ed9d640 1 --2- 172.21.15.67:0/543966205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f586010d7d0 0x7f5860102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.431+0000 7f585e59c640 1 --2- 172.21.15.67:0/543966205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58601043b0 0x7f58601020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.432+0000 7f585dd9b640 1 -- 172.21.15.67:0/543966205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f586010d7d0 msgr2=0x7f5860102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.432+0000 7f585dd9b640 1 --2- 172.21.15.67:0/543966205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f586010d7d0 0x7f5860102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.432+0000 7f585dd9b640 1 -- 172.21.15.67:0/543966205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58601043b0 msgr2=0x7f58601020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.432+0000 7f585dd9b640 1 --2- 172.21.15.67:0/543966205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58601043b0 0x7f58601020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.432+0000 7f585dd9b640 1 -- 172.21.15.67:0/543966205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58601a8110 con 0x7f5860104d00 2024-07-31T15:15:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.432+0000 7f585dd9b640 1 --2- 172.21.15.67:0/543966205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5860104d00 0x7f5860102620 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f5850006010 tx=0x7f5850004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.432+0000 7f58577fe640 1 -- 172.21.15.67:0/543966205 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585003f030 con 0x7f5860104d00 2024-07-31T15:15:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.432+0000 7f58577fe640 1 -- 172.21.15.67:0/543966205 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5850003aa0 con 0x7f5860104d00 2024-07-31T15:15:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.432+0000 7f5864aa9640 1 -- 172.21.15.67:0/543966205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58601a83a0 con 0x7f5860104d00 2024-07-31T15:15:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.432+0000 7f58577fe640 1 -- 172.21.15.67:0/543966205 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850035070 con 0x7f5860104d00 2024-07-31T15:15:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.432+0000 7f5864aa9640 1 -- 172.21.15.67:0/543966205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58601a8890 con 0x7f5860104d00 2024-07-31T15:15:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.433+0000 7f5864aa9640 1 -- 172.21.15.67:0/543966205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58601047c0 con 0x7f5860104d00 2024-07-31T15:15:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.435+0000 7f58577fe640 1 -- 172.21.15.67:0/543966205 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5850042030 con 0x7f5860104d00 2024-07-31T15:15:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.435+0000 7f58577fe640 1 --2- 172.21.15.67:0/543966205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f584c0b3d70 0x7f584c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.436+0000 7f585e59c640 1 --2- 172.21.15.67:0/543966205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f584c0b3d70 0x7f584c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.436+0000 7f58577fe640 1 -- 172.21.15.67:0/543966205 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f5850104cb0 con 0x7f5860104d00 2024-07-31T15:15:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.436+0000 7f585e59c640 1 --2- 172.21.15.67:0/543966205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f584c0b3d70 0x7f584c0b6280 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5860103b30 tx=0x7f5848005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.439+0000 7f58577fe640 1 -- 172.21.15.67:0/543966205 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f58500cbc00 con 0x7f5860104d00 2024-07-31T15:15:22.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.637+0000 7f5864aa9640 1 -- 172.21.15.67:0/543966205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58601a8c00 con 0x7f5860104d00 2024-07-31T15:15:22.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.638+0000 7f58577fe640 1 -- 172.21.15.67:0/543966205 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+25199 (secure 0 0 0) 0x7f58500cb500 con 0x7f5860104d00 2024-07-31T15:15:22.640 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:15:22.640 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":137,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T15:15:10.457463+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":42,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":42,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":29,"length":1},{"begin":42,"length":1}]},{"pool":4,"snaps":[{"begin":29,"length":1},{"begin":42,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":29,"length":1},{"begin":42,"length":1}]},{"pool":4,"snaps":[{"begin":29,"length":1},{"begin":42,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T15:15:22.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.641+0000 7f5864aa9640 1 -- 172.21.15.67:0/543966205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f584c0b3d70 msgr2=0x7f584c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:22.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.641+0000 7f5864aa9640 1 --2- 172.21.15.67:0/543966205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f584c0b3d70 0x7f584c0b6280 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5860103b30 tx=0x7f5848005f90 comp rx=0 tx=0).stop 2024-07-31T15:15:22.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.641+0000 7f5864aa9640 1 -- 172.21.15.67:0/543966205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5860104d00 msgr2=0x7f5860102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:22.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.641+0000 7f5864aa9640 1 --2- 172.21.15.67:0/543966205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5860104d00 0x7f5860102620 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f5850006010 tx=0x7f5850004060 comp rx=0 tx=0).stop 2024-07-31T15:15:22.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.641+0000 7f5864aa9640 1 -- 172.21.15.67:0/543966205 shutdown_connections 2024-07-31T15:15:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.641+0000 7f5864aa9640 1 --2- 172.21.15.67:0/543966205 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f584c0b3d70 0x7f584c0b6280 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.641+0000 7f5864aa9640 1 --2- 172.21.15.67:0/543966205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f586010d7d0 0x7f5860102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.641+0000 7f5864aa9640 1 --2- 172.21.15.67:0/543966205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5860104d00 0x7f5860102620 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.641+0000 7f5864aa9640 1 --2- 172.21.15.67:0/543966205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58601043b0 0x7f58601020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.641+0000 7f5864aa9640 1 -- 172.21.15.67:0/543966205 >> 172.21.15.67:0/543966205 conn(0x7f58600fdde0 msgr2=0x7f5860107300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.641+0000 7f5864aa9640 1 -- 172.21.15.67:0/543966205 shutdown_connections 2024-07-31T15:15:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.641+0000 7f5864aa9640 1 -- 172.21.15.67:0/543966205 wait complete. 2024-07-31T15:15:22.654 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-31T14:42:27.359626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-07-31T14:46:58.459751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-07-31T14:47:01.783728+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 42, 'snap_epoch': 137, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-07-31T14:47:11.080224+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 42, 'snap_epoch': 137, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-07-31T15:15:22.655 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-31T15:15:22.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.772+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/3514877388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b541034e0 msgr2=0x7f6b541038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:22.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.772+0000 7f6b5a5c8640 1 --2- 172.21.15.67:0/3514877388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b541034e0 0x7f6b541038f0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009900 tx=0x7f6b4402a3f0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.772+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/3514877388 shutdown_connections 2024-07-31T15:15:22.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.772+0000 7f6b5a5c8640 1 --2- 172.21.15.67:0/3514877388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b54109760 0x7f6b5410ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.772+0000 7f6b5a5c8640 1 --2- 172.21.15.67:0/3514877388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b54108e10 0x7f6b54109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.772+0000 7f6b5a5c8640 1 --2- 172.21.15.67:0/3514877388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b541034e0 0x7f6b541038f0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.772+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/3514877388 >> 172.21.15.67:0/3514877388 conn(0x7f6b540fef20 msgr2=0x7f6b54101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.772+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/3514877388 shutdown_connections 2024-07-31T15:15:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.772+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/3514877388 wait complete. 2024-07-31T15:15:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.772+0000 7f6b5a5c8640 1 Processor -- start 2024-07-31T15:15:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.772+0000 7f6b5a5c8640 1 -- start start 2024-07-31T15:15:22.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b5a5c8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b541034e0 0x7f6b54199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:22.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b5a5c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b54108e10 0x7f6b5419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:22.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b5a5c8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b54109760 0x7f6b5419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:22.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b5a5c8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6b54113410 con 0x7f6b54109760 2024-07-31T15:15:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b5a5c8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6b54113290 con 0x7f6b54108e10 2024-07-31T15:15:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b5a5c8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6b54113590 con 0x7f6b541034e0 2024-07-31T15:15:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b58b65640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b54109760 0x7f6b5419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b58b65640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b54109760 0x7f6b5419f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36696/0 (socket says 172.21.15.67:36696) 2024-07-31T15:15:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b58b65640 1 -- 172.21.15.67:0/534636494 learned_addr learned my addr 172.21.15.67:0/534636494 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b53fff640 1 --2- 172.21.15.67:0/534636494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b541034e0 0x7f6b54199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b537fe640 1 --2- 172.21.15.67:0/534636494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b54108e10 0x7f6b5419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b58b65640 1 -- 172.21.15.67:0/534636494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b541034e0 msgr2=0x7f6b54199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b58b65640 1 --2- 172.21.15.67:0/534636494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b541034e0 0x7f6b54199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b58b65640 1 -- 172.21.15.67:0/534636494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b54108e10 msgr2=0x7f6b5419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b58b65640 1 --2- 172.21.15.67:0/534636494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b54108e10 0x7f6b5419a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.773+0000 7f6b58b65640 1 -- 172.21.15.67:0/534636494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5419fcb0 con 0x7f6b54109760 2024-07-31T15:15:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.774+0000 7f6b58b65640 1 --2- 172.21.15.67:0/534636494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b54109760 0x7f6b5419f640 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00cce0 tx=0x7f6b4c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.774+0000 7f6b517fa640 1 -- 172.21.15.67:0/534636494 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c00f450 con 0x7f6b54109760 2024-07-31T15:15:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.774+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/534636494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5419ffa0 con 0x7f6b54109760 2024-07-31T15:15:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.774+0000 7f6b517fa640 1 -- 172.21.15.67:0/534636494 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b4c00faa0 con 0x7f6b54109760 2024-07-31T15:15:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.774+0000 7f6b517fa640 1 -- 172.21.15.67:0/534636494 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c018a80 con 0x7f6b54109760 2024-07-31T15:15:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.774+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/534636494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b541a67c0 con 0x7f6b54109760 2024-07-31T15:15:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.775+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/534636494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b54104750 con 0x7f6b54109760 2024-07-31T15:15:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.777+0000 7f6b517fa640 1 -- 172.21.15.67:0/534636494 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6b4c00fc50 con 0x7f6b54109760 2024-07-31T15:15:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.777+0000 7f6b517fa640 1 --2- 172.21.15.67:0/534636494 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6b3c0b3d40 0x7f6b3c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.777+0000 7f6b53fff640 1 --2- 172.21.15.67:0/534636494 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6b3c0b3d40 0x7f6b3c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.777+0000 7f6b517fa640 1 -- 172.21.15.67:0/534636494 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f6b4c0e4bf0 con 0x7f6b54109760 2024-07-31T15:15:22.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.778+0000 7f6b53fff640 1 --2- 172.21.15.67:0/534636494 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6b3c0b3d40 0x7f6b3c0b6250 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6b44031440 tx=0x7f6b44005e80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:22.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.780+0000 7f6b517fa640 1 -- 172.21.15.67:0/534636494 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f6b4c014030 con 0x7f6b54109760 2024-07-31T15:15:22.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.972+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/534636494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6b5410b4e0 con 0x7f6b54109760 2024-07-31T15:15:22.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.973+0000 7f6b517fa640 1 -- 172.21.15.67:0/534636494 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) ==== 93+0+10 (secure 0 0 0) 0x7f6b4c0abb40 con 0x7f6b54109760 2024-07-31T15:15:22.975 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-07-31T15:15:22.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.975+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/534636494 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6b3c0b3d40 msgr2=0x7f6b3c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:22.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.975+0000 7f6b5a5c8640 1 --2- 172.21.15.67:0/534636494 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6b3c0b3d40 0x7f6b3c0b6250 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6b44031440 tx=0x7f6b44005e80 comp rx=0 tx=0).stop 2024-07-31T15:15:22.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.975+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/534636494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b54109760 msgr2=0x7f6b5419f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:22.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.975+0000 7f6b5a5c8640 1 --2- 172.21.15.67:0/534636494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b54109760 0x7f6b5419f640 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00cce0 tx=0x7f6b4c00a620 comp rx=0 tx=0).stop 2024-07-31T15:15:22.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.975+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/534636494 shutdown_connections 2024-07-31T15:15:22.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.975+0000 7f6b5a5c8640 1 --2- 172.21.15.67:0/534636494 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6b3c0b3d40 0x7f6b3c0b6250 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.975+0000 7f6b5a5c8640 1 --2- 172.21.15.67:0/534636494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b54109760 0x7f6b5419f640 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.975+0000 7f6b5a5c8640 1 --2- 172.21.15.67:0/534636494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b54108e10 0x7f6b5419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.975+0000 7f6b5a5c8640 1 --2- 172.21.15.67:0/534636494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b541034e0 0x7f6b54199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:22.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.975+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/534636494 >> 172.21.15.67:0/534636494 conn(0x7f6b540fef20 msgr2=0x7f6b54101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:22.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.975+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/534636494 shutdown_connections 2024-07-31T15:15:22.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:22.975+0000 7f6b5a5c8640 1 -- 172.21.15.67:0/534636494 wait complete. 2024-07-31T15:15:22.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-07-31T15:15:23.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.105+0000 7fa66bb4f640 1 -- 172.21.15.67:0/4170181831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6641023d0 msgr2=0x7fa6641027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.105+0000 7fa66bb4f640 1 --2- 172.21.15.67:0/4170181831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6641023d0 0x7fa6641027e0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009900 tx=0x7fa65c02a410 comp rx=0 tx=0).stop 2024-07-31T15:15:23.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.105+0000 7fa66bb4f640 1 -- 172.21.15.67:0/4170181831 shutdown_connections 2024-07-31T15:15:23.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.105+0000 7fa66bb4f640 1 --2- 172.21.15.67:0/4170181831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa664108650 0x7fa66410ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.105+0000 7fa66bb4f640 1 --2- 172.21.15.67:0/4170181831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa664107d00 0x7fa664108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.105+0000 7fa66bb4f640 1 --2- 172.21.15.67:0/4170181831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6641023d0 0x7fa6641027e0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.105+0000 7fa66bb4f640 1 -- 172.21.15.67:0/4170181831 >> 172.21.15.67:0/4170181831 conn(0x7fa6640fde10 msgr2=0x7fa664100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:23.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.105+0000 7fa66bb4f640 1 -- 172.21.15.67:0/4170181831 shutdown_connections 2024-07-31T15:15:23.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.105+0000 7fa66bb4f640 1 -- 172.21.15.67:0/4170181831 wait complete. 2024-07-31T15:15:23.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.106+0000 7fa66bb4f640 1 Processor -- start 2024-07-31T15:15:23.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.106+0000 7fa66bb4f640 1 -- start start 2024-07-31T15:15:23.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.106+0000 7fa66bb4f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6641023d0 0x7fa664199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:23.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.106+0000 7fa66bb4f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa664107d00 0x7fa66419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:23.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.106+0000 7fa66bb4f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa664108650 0x7fa66419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:23.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.106+0000 7fa66bb4f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa664112300 con 0x7fa6641023d0 2024-07-31T15:15:23.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.106+0000 7fa66bb4f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa664112180 con 0x7fa664108650 2024-07-31T15:15:23.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.106+0000 7fa66bb4f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa664112480 con 0x7fa664107d00 2024-07-31T15:15:23.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.106+0000 7fa6698eb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6641023d0 0x7fa664199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:23.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.106+0000 7fa6698eb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6641023d0 0x7fa664199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36712/0 (socket says 172.21.15.67:36712) 2024-07-31T15:15:23.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.106+0000 7fa6698eb640 1 -- 172.21.15.67:0/3678575195 learned_addr learned my addr 172.21.15.67:0/3678575195 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:23.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.106+0000 7fa66a0ec640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa664108650 0x7fa66419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:23.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.106+0000 7fa6690ea640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa664107d00 0x7fa66419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:23.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.107+0000 7fa6698eb640 1 -- 172.21.15.67:0/3678575195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa664107d00 msgr2=0x7fa66419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.107+0000 7fa6698eb640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa664107d00 0x7fa66419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.107+0000 7fa6698eb640 1 -- 172.21.15.67:0/3678575195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa664108650 msgr2=0x7fa66419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.107+0000 7fa6698eb640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa664108650 0x7fa66419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.107+0000 7fa6698eb640 1 -- 172.21.15.67:0/3678575195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66419fd80 con 0x7fa6641023d0 2024-07-31T15:15:23.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.107+0000 7fa6690ea640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa664107d00 0x7fa66419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-31T15:15:23.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.107+0000 7fa6698eb640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6641023d0 0x7fa664199e50 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa65c005940 tx=0x7fa65c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:23.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.107+0000 7fa662ffd640 1 -- 172.21.15.67:0/3678575195 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c03f030 con 0x7fa6641023d0 2024-07-31T15:15:23.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.107+0000 7fa662ffd640 1 -- 172.21.15.67:0/3678575195 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa65c003aa0 con 0x7fa6641023d0 2024-07-31T15:15:23.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.107+0000 7fa66bb4f640 1 -- 172.21.15.67:0/3678575195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6641a0010 con 0x7fa6641023d0 2024-07-31T15:15:23.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.107+0000 7fa662ffd640 1 -- 172.21.15.67:0/3678575195 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c035070 con 0x7fa6641023d0 2024-07-31T15:15:23.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.107+0000 7fa66bb4f640 1 -- 172.21.15.67:0/3678575195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6641a6800 con 0x7fa6641023d0 2024-07-31T15:15:23.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.108+0000 7fa66bb4f640 1 -- 172.21.15.67:0/3678575195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa664103640 con 0x7fa6641023d0 2024-07-31T15:15:23.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.110+0000 7fa662ffd640 1 -- 172.21.15.67:0/3678575195 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fa65c042030 con 0x7fa6641023d0 2024-07-31T15:15:23.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.110+0000 7fa662ffd640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa64c0b3d40 0x7fa64c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:23.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.110+0000 7fa6690ea640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa64c0b3d40 0x7fa64c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:23.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.111+0000 7fa662ffd640 1 -- 172.21.15.67:0/3678575195 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7fa65c104ca0 con 0x7fa6641023d0 2024-07-31T15:15:23.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.111+0000 7fa6690ea640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa64c0b3d40 0x7fa64c0b6250 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa664107980 tx=0x7fa658005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:23.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.113+0000 7fa662ffd640 1 -- 172.21.15.67:0/3678575195 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fa65c0cbbf0 con 0x7fa6641023d0 2024-07-31T15:15:23.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.305+0000 7fa66bb4f640 1 -- 172.21.15.67:0/3678575195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa6641a6a60 con 0x7fa6641023d0 2024-07-31T15:15:23.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.305+0000 7fa662ffd640 1 -- 172.21.15.67:0/3678575195 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) ==== 104+0+11 (secure 0 0 0) 0x7fa65c0cb4f0 con 0x7fa6641023d0 2024-07-31T15:15:23.308 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-07-31T15:15:23.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.307+0000 7fa66bb4f640 1 -- 172.21.15.67:0/3678575195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa64c0b3d40 msgr2=0x7fa64c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.307+0000 7fa66bb4f640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa64c0b3d40 0x7fa64c0b6250 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa664107980 tx=0x7fa658005f90 comp rx=0 tx=0).stop 2024-07-31T15:15:23.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.307+0000 7fa66bb4f640 1 -- 172.21.15.67:0/3678575195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6641023d0 msgr2=0x7fa664199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.307+0000 7fa66bb4f640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6641023d0 0x7fa664199e50 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa65c005940 tx=0x7fa65c004270 comp rx=0 tx=0).stop 2024-07-31T15:15:23.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.307+0000 7fa66bb4f640 1 -- 172.21.15.67:0/3678575195 shutdown_connections 2024-07-31T15:15:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.307+0000 7fa66bb4f640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fa64c0b3d40 0x7fa64c0b6250 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.307+0000 7fa66bb4f640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa664108650 0x7fa66419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.307+0000 7fa66bb4f640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa664107d00 0x7fa66419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.307+0000 7fa66bb4f640 1 --2- 172.21.15.67:0/3678575195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6641023d0 0x7fa664199e50 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.307+0000 7fa66bb4f640 1 -- 172.21.15.67:0/3678575195 >> 172.21.15.67:0/3678575195 conn(0x7fa6640fde10 msgr2=0x7fa664100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.308+0000 7fa66bb4f640 1 -- 172.21.15.67:0/3678575195 shutdown_connections 2024-07-31T15:15:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.308+0000 7fa66bb4f640 1 -- 172.21.15.67:0/3678575195 wait complete. 2024-07-31T15:15:23.322 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-07-31T15:15:23.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.438+0000 7f17a9b00640 1 -- 172.21.15.67:0/3470063041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17a4107d10 msgr2=0x7f17a4108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.438+0000 7f17a9b00640 1 --2- 172.21.15.67:0/3470063041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17a4107d10 0x7f17a4108120 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f1798009900 tx=0x7f179802a410 comp rx=0 tx=0).stop 2024-07-31T15:15:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.438+0000 7f17a9b00640 1 -- 172.21.15.67:0/3470063041 shutdown_connections 2024-07-31T15:15:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.438+0000 7f17a9b00640 1 --2- 172.21.15.67:0/3470063041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a4108660 0x7f17a410ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.438+0000 7f17a9b00640 1 --2- 172.21.15.67:0/3470063041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17a4107d10 0x7f17a4108120 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.438+0000 7f17a9b00640 1 --2- 172.21.15.67:0/3470063041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17a41023e0 0x7f17a41027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.439+0000 7f17a9b00640 1 -- 172.21.15.67:0/3470063041 >> 172.21.15.67:0/3470063041 conn(0x7f17a40fddc0 msgr2=0x7f17a4100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.439+0000 7f17a9b00640 1 -- 172.21.15.67:0/3470063041 shutdown_connections 2024-07-31T15:15:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.439+0000 7f17a9b00640 1 -- 172.21.15.67:0/3470063041 wait complete. 2024-07-31T15:15:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.439+0000 7f17a9b00640 1 Processor -- start 2024-07-31T15:15:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.439+0000 7f17a9b00640 1 -- start start 2024-07-31T15:15:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a9b00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17a41023e0 0x7f17a419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a9b00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17a4107d10 0x7f17a419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a9b00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a4108660 0x7f17a419b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a9b00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41122f0 con 0x7f17a4107d10 2024-07-31T15:15:23.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a9b00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4112170 con 0x7f17a4108660 2024-07-31T15:15:23.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a9b00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4112470 con 0x7f17a41023e0 2024-07-31T15:15:23.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17a4107d10 0x7f17a419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:23.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17a4107d10 0x7f17a419b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36736/0 (socket says 172.21.15.67:36736) 2024-07-31T15:15:23.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a2ffd640 1 -- 172.21.15.67:0/3192077369 learned_addr learned my addr 172.21.15.67:0/3192077369 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:23.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a37fe640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17a41023e0 0x7f17a419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:23.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a3fff640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a4108660 0x7f17a419b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a2ffd640 1 -- 172.21.15.67:0/3192077369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17a41023e0 msgr2=0x7f17a419aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a2ffd640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17a41023e0 0x7f17a419aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a2ffd640 1 -- 172.21.15.67:0/3192077369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a4108660 msgr2=0x7f17a419b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a2ffd640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a4108660 0x7f17a419b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a2ffd640 1 -- 172.21.15.67:0/3192077369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a41a5f30 con 0x7f17a4107d10 2024-07-31T15:15:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.440+0000 7f17a3fff640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a4108660 0x7f17a419b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:15:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.441+0000 7f17a2ffd640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17a4107d10 0x7f17a419b1e0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f1798006010 tx=0x7f1798004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.441+0000 7f17a0ff9640 1 -- 172.21.15.67:0/3192077369 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179803f030 con 0x7f17a4107d10 2024-07-31T15:15:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.441+0000 7f17a0ff9640 1 -- 172.21.15.67:0/3192077369 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1798003aa0 con 0x7f17a4107d10 2024-07-31T15:15:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.441+0000 7f17a9b00640 1 -- 172.21.15.67:0/3192077369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a41a61c0 con 0x7f17a4107d10 2024-07-31T15:15:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.441+0000 7f17a0ff9640 1 -- 172.21.15.67:0/3192077369 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798035070 con 0x7f17a4107d10 2024-07-31T15:15:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.441+0000 7f17a9b00640 1 -- 172.21.15.67:0/3192077369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a41a66b0 con 0x7f17a4107d10 2024-07-31T15:15:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.442+0000 7f17a9b00640 1 -- 172.21.15.67:0/3192077369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a41027f0 con 0x7f17a4107d10 2024-07-31T15:15:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.444+0000 7f17a0ff9640 1 -- 172.21.15.67:0/3192077369 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1798042030 con 0x7f17a4107d10 2024-07-31T15:15:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.444+0000 7f17a0ff9640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f178c0b3d40 0x7f178c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.445+0000 7f17a37fe640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f178c0b3d40 0x7f178c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.445+0000 7f17a0ff9640 1 -- 172.21.15.67:0/3192077369 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f1798104c50 con 0x7f17a4107d10 2024-07-31T15:15:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.445+0000 7f17a37fe640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f178c0b3d40 0x7f178c0b6250 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f17a406bfc0 tx=0x7f1794005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.448+0000 7f17a0ff9640 1 -- 172.21.15.67:0/3192077369 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f17980cbba0 con 0x7f17a4107d10 2024-07-31T15:15:23.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.642+0000 7f17a9b00640 1 -- 172.21.15.67:0/3192077369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f17a402e0c0 con 0x7f17a4107d10 2024-07-31T15:15:23.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.642+0000 7f17a0ff9640 1 -- 172.21.15.67:0/3192077369 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) ==== 100+0+11 (secure 0 0 0) 0x7f17980d2e20 con 0x7f17a4107d10 2024-07-31T15:15:23.645 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-07-31T15:15:23.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.644+0000 7f17a9b00640 1 -- 172.21.15.67:0/3192077369 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f178c0b3d40 msgr2=0x7f178c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.644+0000 7f17a9b00640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f178c0b3d40 0x7f178c0b6250 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f17a406bfc0 tx=0x7f1794005f90 comp rx=0 tx=0).stop 2024-07-31T15:15:23.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.644+0000 7f17a9b00640 1 -- 172.21.15.67:0/3192077369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17a4107d10 msgr2=0x7f17a419b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.644+0000 7f17a9b00640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17a4107d10 0x7f17a419b1e0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f1798006010 tx=0x7f1798004060 comp rx=0 tx=0).stop 2024-07-31T15:15:23.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.644+0000 7f17a9b00640 1 -- 172.21.15.67:0/3192077369 shutdown_connections 2024-07-31T15:15:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.644+0000 7f17a9b00640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f178c0b3d40 0x7f178c0b6250 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.644+0000 7f17a9b00640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17a4108660 0x7f17a419b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.644+0000 7f17a9b00640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17a4107d10 0x7f17a419b1e0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.644+0000 7f17a9b00640 1 --2- 172.21.15.67:0/3192077369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17a41023e0 0x7f17a419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.644+0000 7f17a9b00640 1 -- 172.21.15.67:0/3192077369 >> 172.21.15.67:0/3192077369 conn(0x7f17a40fddc0 msgr2=0x7f17a41001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.645+0000 7f17a9b00640 1 -- 172.21.15.67:0/3192077369 shutdown_connections 2024-07-31T15:15:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.645+0000 7f17a9b00640 1 -- 172.21.15.67:0/3192077369 wait complete. 2024-07-31T15:15:23.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:23 smithi067 ceph-mon[32750]: pgmap v1176: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-07-31T15:15:23.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:23 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/543966205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T15:15:23.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:23 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/534636494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-31T15:15:23.658 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:23 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3678575195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-31T15:15:23.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-07-31T15:15:23.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.774+0000 7eff948e7640 1 -- 172.21.15.67:0/424419069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff90109790 msgr2=0x7eff9010ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.774+0000 7eff948e7640 1 --2- 172.21.15.67:0/424419069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff90109790 0x7eff9010ffc0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7eff88009ca0 tx=0x7eff8802a8c0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.774+0000 7eff948e7640 1 -- 172.21.15.67:0/424419069 shutdown_connections 2024-07-31T15:15:23.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.774+0000 7eff948e7640 1 --2- 172.21.15.67:0/424419069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff90109790 0x7eff9010ffc0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.774+0000 7eff948e7640 1 --2- 172.21.15.67:0/424419069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff90108e40 0x7eff90109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.774+0000 7eff948e7640 1 --2- 172.21.15.67:0/424419069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff90101910 0x7eff90101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.775+0000 7eff948e7640 1 -- 172.21.15.67:0/424419069 >> 172.21.15.67:0/424419069 conn(0x7eff900775e0 msgr2=0x7eff90077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:23.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.775+0000 7eff948e7640 1 -- 172.21.15.67:0/424419069 shutdown_connections 2024-07-31T15:15:23.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.775+0000 7eff948e7640 1 -- 172.21.15.67:0/424419069 wait complete. 2024-07-31T15:15:23.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.775+0000 7eff948e7640 1 Processor -- start 2024-07-31T15:15:23.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.775+0000 7eff948e7640 1 -- start start 2024-07-31T15:15:23.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff948e7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff90101910 0x7eff90199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:23.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff948e7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff90108e40 0x7eff9019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:23.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff948e7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff90109790 0x7eff9019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:23.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff948e7640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7eff90113460 con 0x7eff90108e40 2024-07-31T15:15:23.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff948e7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7eff901132e0 con 0x7eff90101910 2024-07-31T15:15:23.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff948e7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7eff901135e0 con 0x7eff90109790 2024-07-31T15:15:23.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff87fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff90108e40 0x7eff9019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:23.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff87fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff90108e40 0x7eff9019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36752/0 (socket says 172.21.15.67:36752) 2024-07-31T15:15:23.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff87fff640 1 -- 172.21.15.67:0/3434759285 learned_addr learned my addr 172.21.15.67:0/3434759285 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:23.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff8e59c640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff90101910 0x7eff90199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:23.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff8ed9d640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff90109790 0x7eff9019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:23.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff87fff640 1 -- 172.21.15.67:0/3434759285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff90109790 msgr2=0x7eff9019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff87fff640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff90109790 0x7eff9019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff87fff640 1 -- 172.21.15.67:0/3434759285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff90101910 msgr2=0x7eff90199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff87fff640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff90101910 0x7eff90199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff87fff640 1 -- 172.21.15.67:0/3434759285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff9019fcf0 con 0x7eff90108e40 2024-07-31T15:15:23.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff8ed9d640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff90109790 0x7eff9019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:15:23.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.776+0000 7eff87fff640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff90108e40 0x7eff9019a360 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7eff7800c970 tx=0x7eff7800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:23.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.777+0000 7eff877fe640 1 -- 172.21.15.67:0/3434759285 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78004220 con 0x7eff90108e40 2024-07-31T15:15:23.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.777+0000 7eff877fe640 1 -- 172.21.15.67:0/3434759285 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff780043c0 con 0x7eff90108e40 2024-07-31T15:15:23.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.777+0000 7eff877fe640 1 -- 172.21.15.67:0/3434759285 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78004a00 con 0x7eff90108e40 2024-07-31T15:15:23.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.777+0000 7eff948e7640 1 -- 172.21.15.67:0/3434759285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff9019ffe0 con 0x7eff90108e40 2024-07-31T15:15:23.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.777+0000 7eff948e7640 1 -- 172.21.15.67:0/3434759285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff901a6770 con 0x7eff90108e40 2024-07-31T15:15:23.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.778+0000 7eff948e7640 1 -- 172.21.15.67:0/3434759285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff90101d20 con 0x7eff90108e40 2024-07-31T15:15:23.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.779+0000 7eff877fe640 1 -- 172.21.15.67:0/3434759285 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7eff780079b0 con 0x7eff90108e40 2024-07-31T15:15:23.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.780+0000 7eff877fe640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7eff7c0b3d40 0x7eff7c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:23.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.780+0000 7eff8e59c640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7eff7c0b3d40 0x7eff7c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:23.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.780+0000 7eff877fe640 1 -- 172.21.15.67:0/3434759285 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7eff780e4140 con 0x7eff90108e40 2024-07-31T15:15:23.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.780+0000 7eff8e59c640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7eff7c0b3d40 0x7eff7c0b6250 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7eff800058c0 tx=0x7eff8000a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:23.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.784+0000 7eff877fe640 1 -- 172.21.15.67:0/3434759285 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7eff78014030 con 0x7eff90108e40 2024-07-31T15:15:23.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:23 smithi073 ceph-mon[36714]: pgmap v1176: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-07-31T15:15:23.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:23 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/543966205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T15:15:23.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:23 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/534636494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-31T15:15:23.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:23 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3678575195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-31T15:15:23.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.976+0000 7eff948e7640 1 -- 172.21.15.67:0/3434759285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7eff9019aee0 con 0x7eff90108e40 2024-07-31T15:15:23.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.976+0000 7eff877fe640 1 -- 172.21.15.67:0/3434759285 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v137) ==== 103+0+11 (secure 0 0 0) 0x7eff780ab090 con 0x7eff90108e40 2024-07-31T15:15:23.979 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-07-31T15:15:23.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.978+0000 7eff948e7640 1 -- 172.21.15.67:0/3434759285 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7eff7c0b3d40 msgr2=0x7eff7c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.978+0000 7eff948e7640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7eff7c0b3d40 0x7eff7c0b6250 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7eff800058c0 tx=0x7eff8000a2d0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.978+0000 7eff948e7640 1 -- 172.21.15.67:0/3434759285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff90108e40 msgr2=0x7eff9019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:23.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.978+0000 7eff948e7640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff90108e40 0x7eff9019a360 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7eff7800c970 tx=0x7eff7800ce30 comp rx=0 tx=0).stop 2024-07-31T15:15:23.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.978+0000 7eff948e7640 1 -- 172.21.15.67:0/3434759285 shutdown_connections 2024-07-31T15:15:23.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.978+0000 7eff948e7640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7eff7c0b3d40 0x7eff7c0b6250 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.978+0000 7eff948e7640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff90109790 0x7eff9019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.978+0000 7eff948e7640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff90108e40 0x7eff9019a360 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.978+0000 7eff948e7640 1 --2- 172.21.15.67:0/3434759285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff90101910 0x7eff90199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:23.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.978+0000 7eff948e7640 1 -- 172.21.15.67:0/3434759285 >> 172.21.15.67:0/3434759285 conn(0x7eff900775e0 msgr2=0x7eff90106e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:23.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.978+0000 7eff948e7640 1 -- 172.21.15.67:0/3434759285 shutdown_connections 2024-07-31T15:15:23.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:23.978+0000 7eff948e7640 1 -- 172.21.15.67:0/3434759285 wait complete. 2024-07-31T15:15:23.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:23 smithi077 ceph-mon[37327]: pgmap v1176: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-07-31T15:15:23.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:23 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/543966205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T15:15:23.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:23 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/534636494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-31T15:15:23.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:23 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3678575195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-31T15:15:23.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-07-31T15:15:24.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.105+0000 7f5d70a77640 1 -- 172.21.15.67:0/2518491323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d6c107d30 msgr2=0x7f5d6c108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:24.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.105+0000 7f5d70a77640 1 --2- 172.21.15.67:0/2518491323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d6c107d30 0x7f5d6c108140 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009900 tx=0x7f5d6002a410 comp rx=0 tx=0).stop 2024-07-31T15:15:24.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.105+0000 7f5d70a77640 1 -- 172.21.15.67:0/2518491323 shutdown_connections 2024-07-31T15:15:24.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.105+0000 7f5d70a77640 1 --2- 172.21.15.67:0/2518491323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d6c108680 0x7f5d6c10eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:24.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.105+0000 7f5d70a77640 1 --2- 172.21.15.67:0/2518491323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d6c107d30 0x7f5d6c108140 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:24.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.105+0000 7f5d70a77640 1 --2- 172.21.15.67:0/2518491323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d6c102400 0x7f5d6c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:24.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.105+0000 7f5d70a77640 1 -- 172.21.15.67:0/2518491323 >> 172.21.15.67:0/2518491323 conn(0x7f5d6c0fdde0 msgr2=0x7f5d6c100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:24.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.105+0000 7f5d70a77640 1 -- 172.21.15.67:0/2518491323 shutdown_connections 2024-07-31T15:15:24.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.105+0000 7f5d70a77640 1 -- 172.21.15.67:0/2518491323 wait complete. 2024-07-31T15:15:24.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d70a77640 1 Processor -- start 2024-07-31T15:15:24.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d70a77640 1 -- start start 2024-07-31T15:15:24.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d70a77640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d6c102400 0x7f5d6c199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:24.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d70a77640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d6c107d30 0x7f5d6c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:24.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d70a77640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d6c108680 0x7f5d6c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:24.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d70a77640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c112360 con 0x7f5d6c108680 2024-07-31T15:15:24.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d70a77640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1121e0 con 0x7f5d6c107d30 2024-07-31T15:15:24.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d70a77640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1124e0 con 0x7f5d6c102400 2024-07-31T15:15:24.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d6ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d6c108680 0x7f5d6c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:24.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d6ad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d6c108680 0x7f5d6c19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36764/0 (socket says 172.21.15.67:36764) 2024-07-31T15:15:24.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d6ad9d640 1 -- 172.21.15.67:0/2778832098 learned_addr learned my addr 172.21.15.67:0/2778832098 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:15:24.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d6a59c640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d6c102400 0x7f5d6c199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:24.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d69d9b640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d6c107d30 0x7f5d6c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:24.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d6ad9d640 1 -- 172.21.15.67:0/2778832098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d6c102400 msgr2=0x7f5d6c199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:24.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d6ad9d640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d6c102400 0x7f5d6c199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:24.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d6ad9d640 1 -- 172.21.15.67:0/2778832098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d6c107d30 msgr2=0x7f5d6c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:24.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d6ad9d640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d6c107d30 0x7f5d6c19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:24.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d6ad9d640 1 -- 172.21.15.67:0/2778832098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c19fd80 con 0x7f5d6c108680 2024-07-31T15:15:24.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d6a59c640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d6c102400 0x7f5d6c199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:15:24.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.106+0000 7f5d69d9b640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d6c107d30 0x7f5d6c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:15:24.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.107+0000 7f5d6ad9d640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d6c108680 0x7f5d6c19f680 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400eb10 tx=0x7f5d6400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:24.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.107+0000 7f5d5b7fe640 1 -- 172.21.15.67:0/2778832098 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d64004b20 con 0x7f5d6c108680 2024-07-31T15:15:24.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.107+0000 7f5d5b7fe640 1 -- 172.21.15.67:0/2778832098 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d64004cc0 con 0x7f5d6c108680 2024-07-31T15:15:24.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.107+0000 7f5d70a77640 1 -- 172.21.15.67:0/2778832098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6c1a0070 con 0x7f5d6c108680 2024-07-31T15:15:24.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.107+0000 7f5d5b7fe640 1 -- 172.21.15.67:0/2778832098 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d64010670 con 0x7f5d6c108680 2024-07-31T15:15:24.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.107+0000 7f5d70a77640 1 -- 172.21.15.67:0/2778832098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6c1a6800 con 0x7f5d6c108680 2024-07-31T15:15:24.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.108+0000 7f5d70a77640 1 -- 172.21.15.67:0/2778832098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d6c102810 con 0x7f5d6c108680 2024-07-31T15:15:24.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.110+0000 7f5d5b7fe640 1 -- 172.21.15.67:0/2778832098 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5d64005000 con 0x7f5d6c108680 2024-07-31T15:15:24.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.110+0000 7f5d5b7fe640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5d540b3d40 0x7f5d540b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:15:24.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.110+0000 7f5d5b7fe640 1 -- 172.21.15.67:0/2778832098 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9483+0+0 (secure 0 0 0) 0x7f5d640e4cc0 con 0x7f5d6c108680 2024-07-31T15:15:24.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.110+0000 7f5d6a59c640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5d540b3d40 0x7f5d540b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:15:24.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.111+0000 7f5d6a59c640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5d540b3d40 0x7f5d540b6250 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009c20 tx=0x7f5d5c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:15:24.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.113+0000 7f5d5b7fe640 1 -- 172.21.15.67:0/2778832098 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5d640555e0 con 0x7f5d6c108680 2024-07-31T15:15:24.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.362+0000 7f5d70a77640 1 -- 172.21.15.67:0/2778832098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f5d6c10b4b0 con 0x7f5d6c108680 2024-07-31T15:15:24.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.371+0000 7f5d5b7fe640 1 -- 172.21.15.67:0/2778832098 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+821908 (secure 0 0 0) 0x7f5d640abc10 con 0x7f5d6c108680 2024-07-31T15:15:24.384 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:15:24.384 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:15:24.384 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 27, 2024-07-31T15:15:24.384 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.384 INFO:teuthology.orchestra.run.smithi067.stdout: "active_gid": 14214, 2024-07-31T15:15:24.384 INFO:teuthology.orchestra.run.smithi067.stdout: "active_name": "x", 2024-07-31T15:15:24.385 INFO:teuthology.orchestra.run.smithi067.stdout: "active_addrs": { 2024-07-31T15:15:24.385 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-07-31T15:15:24.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.385 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-07-31T15:15:24.385 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6800", 2024-07-31T15:15:24.385 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2637562983 2024-07-31T15:15:24.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.386 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-07-31T15:15:24.386 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6801", 2024-07-31T15:15:24.386 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2637562983 2024-07-31T15:15:24.386 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.386 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-31T15:15:24.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.386 INFO:teuthology.orchestra.run.smithi067.stdout: "active_addr": "172.21.15.67:6801/2637562983", 2024-07-31T15:15:24.387 INFO:teuthology.orchestra.run.smithi067.stdout: "active_change": "2024-07-31T14:40:19.281908+0000", 2024-07-31T15:15:24.387 INFO:teuthology.orchestra.run.smithi067.stdout: "active_mgr_features": 4540701547738038271, 2024-07-31T15:15:24.387 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-07-31T15:15:24.387 INFO:teuthology.orchestra.run.smithi067.stdout: "standbys": [ 2024-07-31T15:15:24.387 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.387 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 24145, 2024-07-31T15:15:24.387 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "z", 2024-07-31T15:15:24.387 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_features": 4540701547738038271, 2024-07-31T15:15:24.388 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-07-31T15:15:24.388 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.388 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-07-31T15:15:24.388 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.388 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.388 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.388 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:24.388 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:24.389 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.389 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.389 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.389 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:24.389 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.389 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.389 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.389 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-07-31T15:15:24.390 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.390 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.390 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.390 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.390 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.390 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.390 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.391 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.391 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.391 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.391 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.391 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.391 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.391 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.392 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.392 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.392 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.392 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.392 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.392 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.392 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.392 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.392 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.393 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.393 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.393 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.393 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.393 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.393 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.393 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.394 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.394 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.394 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.394 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.394 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.394 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.395 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.395 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.395 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.395 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.396 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.396 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.396 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.396 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.396 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.396 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.396 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.396 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.397 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.397 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.397 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.397 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.397 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.397 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.398 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.398 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.398 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.398 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.398 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.398 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.398 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.399 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.399 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.399 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.399 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.399 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.399 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.399 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.399 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.400 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-07-31T15:15:24.400 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-07-31T15:15:24.400 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.400 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.400 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.400 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.401 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.401 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.401 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.401 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to", 2024-07-31T15:15:24.401 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.401 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.401 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.402 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-07-31T15:15:24.402 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-07-31T15:15:24.402 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.402 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.402 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.402 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-07-31T15:15:24.402 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.402 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.403 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.403 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-07-31T15:15:24.403 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.403 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.403 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.403 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.403 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-07-31T15:15:24.403 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-07-31T15:15:24.403 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.404 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.404 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.404 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.404 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.404 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.404 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.404 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-07-31T15:15:24.405 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.405 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.405 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.405 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-07-31T15:15:24.405 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-07-31T15:15:24.405 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.405 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.406 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.406 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.406 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.406 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.406 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.406 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-07-31T15:15:24.406 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.407 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.407 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.407 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-07-31T15:15:24.407 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-07-31T15:15:24.407 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.407 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.407 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.408 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-07-31T15:15:24.408 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.408 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.408 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.408 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-07-31T15:15:24.408 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.408 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.408 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.409 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-07-31T15:15:24.409 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-07-31T15:15:24.409 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.409 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.409 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.409 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.410 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.410 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.410 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.410 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-07-31T15:15:24.410 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.410 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.410 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.411 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-07-31T15:15:24.411 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-07-31T15:15:24.411 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.411 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.411 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.411 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.411 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.412 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.412 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.412 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-31T15:15:24.412 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.412 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.412 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.413 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-07-31T15:15:24.413 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-07-31T15:15:24.413 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.413 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.413 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.413 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.413 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.414 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.414 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.414 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-07-31T15:15:24.414 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.414 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.414 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.414 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.415 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.415 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.415 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.415 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.415 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.415 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.415 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.415 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.416 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.416 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.416 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.416 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.416 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.416 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.417 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-07-31T15:15:24.417 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.417 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.417 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.417 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-07-31T15:15:24.417 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-07-31T15:15:24.417 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.418 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.418 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.418 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.418 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.418 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.418 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.418 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-07-31T15:15:24.419 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.419 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.419 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.419 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-07-31T15:15:24.419 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-07-31T15:15:24.419 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.420 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.420 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.420 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-07-31T15:15:24.420 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.420 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.420 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.420 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-07-31T15:15:24.421 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-31T15:15:24.421 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.421 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.421 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-07-31T15:15:24.421 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-07-31T15:15:24.421 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:24.421 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.422 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.422 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.422 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-07-31T15:15:24.422 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-07-31T15:15:24.422 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.422 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-31T15:15:24.422 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-31T15:15:24.422 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.423 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.423 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-07-31T15:15:24.423 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-07-31T15:15:24.423 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:24.423 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.423 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.423 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-07-31T15:15:24.424 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-31T15:15:24.424 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-07-31T15:15:24.424 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.424 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-31T15:15:24.424 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.424 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.424 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.425 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-07-31T15:15:24.425 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-07-31T15:15:24.425 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.425 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.425 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.425 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-07-31T15:15:24.425 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.426 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.426 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.426 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-31T15:15:24.426 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-31T15:15:24.426 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.426 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.426 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-07-31T15:15:24.427 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-07-31T15:15:24.427 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:24.427 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.427 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.427 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-07-31T15:15:24.427 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-07-31T15:15:24.427 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-07-31T15:15:24.427 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.428 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-07-31T15:15:24.428 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-31T15:15:24.428 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.428 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.428 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-07-31T15:15:24.428 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-07-31T15:15:24.428 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.429 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.429 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.429 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-07-31T15:15:24.429 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.429 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.429 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.429 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-07-31T15:15:24.430 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-31T15:15:24.430 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.430 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.430 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-07-31T15:15:24.430 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-07-31T15:15:24.430 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:24.430 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.431 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.431 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.431 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-07-31T15:15:24.431 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-07-31T15:15:24.431 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.431 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-31T15:15:24.431 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-31T15:15:24.431 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.432 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.432 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.432 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.432 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.432 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.432 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.433 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.433 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.433 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.433 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.433 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.433 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.433 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.433 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.434 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.434 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.434 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.434 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.434 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.434 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.434 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.435 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.435 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.435 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.435 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.435 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.435 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.436 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.436 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.436 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.436 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.436 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.436 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.436 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.437 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.437 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.437 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.437 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.437 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.437 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.437 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.438 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.438 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.438 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.438 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.438 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.438 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.438 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.439 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.439 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.439 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.439 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.439 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.439 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.439 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.440 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.440 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.440 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.440 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.440 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.440 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.440 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.440 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.441 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.441 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.441 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.441 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.441 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-07-31T15:15:24.441 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-07-31T15:15:24.442 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:24.442 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.442 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.442 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.442 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.442 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.442 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.442 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-31T15:15:24.443 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.443 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.443 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.443 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-07-31T15:15:24.443 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-07-31T15:15:24.443 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.443 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.444 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.444 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-07-31T15:15:24.444 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.444 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.444 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.444 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-07-31T15:15:24.444 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-07-31T15:15:24.444 INFO:teuthology.orchestra.run.smithi067.stdout: "read", 2024-07-31T15:15:24.445 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap", 2024-07-31T15:15:24.445 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap-read" 2024-07-31T15:15:24.445 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.445 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-07-31T15:15:24.445 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.445 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.445 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.446 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-07-31T15:15:24.446 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-07-31T15:15:24.446 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.446 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.446 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.446 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.446 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.446 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.447 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.447 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-31T15:15:24.447 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.447 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.447 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.447 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:24.448 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:24.448 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.448 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.448 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.448 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:24.448 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.448 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.448 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.449 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-31T15:15:24.449 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.449 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.449 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.449 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.449 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.449 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.450 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.450 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.450 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.450 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.450 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.450 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.450 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.451 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.451 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.451 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.451 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-07-31T15:15:24.451 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-07-31T15:15:24.451 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.451 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.451 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.452 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:24.452 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-31T15:15:24.452 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.452 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.452 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-31T15:15:24.452 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-31T15:15:24.452 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.452 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.453 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-07-31T15:15:24.453 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-07-31T15:15:24.453 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:24.453 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.453 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.453 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:24.453 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.453 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.454 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.454 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-31T15:15:24.454 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.454 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.454 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.454 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.454 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.455 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-07-31T15:15:24.455 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.455 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.455 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.455 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-07-31T15:15:24.455 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-07-31T15:15:24.456 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:24.456 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.456 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.456 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-07-31T15:15:24.456 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.456 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.456 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.456 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-31T15:15:24.457 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.457 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.457 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.457 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-07-31T15:15:24.457 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-07-31T15:15:24.458 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.458 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.458 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.458 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-07-31T15:15:24.458 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.458 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.458 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.458 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-31T15:15:24.459 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.459 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.459 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.459 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-07-31T15:15:24.459 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-07-31T15:15:24.459 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.459 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.460 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.460 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-07-31T15:15:24.460 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.460 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.460 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.460 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-31T15:15:24.460 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.460 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.461 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.461 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-07-31T15:15:24.461 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-07-31T15:15:24.461 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.461 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.461 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.461 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.462 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.462 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.462 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.462 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-31T15:15:24.462 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-31T15:15:24.462 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.462 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.463 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-07-31T15:15:24.463 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-07-31T15:15:24.463 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.463 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.463 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.463 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:24.463 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.464 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.464 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.464 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-31T15:15:24.464 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.464 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.464 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.464 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-07-31T15:15:24.465 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-07-31T15:15:24.465 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:24.465 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.465 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.465 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-07-31T15:15:24.465 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.465 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.465 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.465 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-31T15:15:24.466 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.466 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.466 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.466 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm_log_destination": { 2024-07-31T15:15:24.466 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm_log_destination", 2024-07-31T15:15:24.466 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "file", 2024-07-31T15:15:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "file,syslog", 2024-07-31T15:15:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "syslog" 2024-07-31T15:15:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-31T15:15:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-07-31T15:15:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-07-31T15:15:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-31T15:15:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-07-31T15:15:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-07-31T15:15:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-31T15:15:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-07-31T15:15:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-07-31T15:15:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.473 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.473 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.473 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.473 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.473 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.473 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.473 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-31T15:15:24.473 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.474 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.474 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.474 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-07-31T15:15:24.474 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-07-31T15:15:24.474 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.474 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.474 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.475 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-07-31T15:15:24.475 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.475 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.475 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.475 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-31T15:15:24.475 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.475 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.475 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.476 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-07-31T15:15:24.476 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-07-31T15:15:24.476 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.476 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.476 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.476 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-31T15:15:24.476 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.477 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.477 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.477 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-07-31T15:15:24.477 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.477 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.477 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.477 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-07-31T15:15:24.478 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-07-31T15:15:24.478 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.478 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.478 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.478 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-31T15:15:24.478 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.478 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.479 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.479 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-07-31T15:15:24.479 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.479 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.479 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.479 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-07-31T15:15:24.480 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-07-31T15:15:24.480 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.480 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.480 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.480 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-07-31T15:15:24.480 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.480 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.480 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.480 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-31T15:15:24.481 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.481 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.481 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.481 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-07-31T15:15:24.481 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-07-31T15:15:24.481 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.482 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.482 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.482 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-31T15:15:24.482 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.482 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.482 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.482 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-07-31T15:15:24.482 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-07-31T15:15:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-07-31T15:15:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-31T15:15:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-07-31T15:15:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-07-31T15:15:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-07-31T15:15:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-31T15:15:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-07-31T15:15:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-07-31T15:15:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-07-31T15:15:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-31T15:15:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-07-31T15:15:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-07-31T15:15:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-07-31T15:15:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-31T15:15:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-07-31T15:15:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-07-31T15:15:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-07-31T15:15:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-31T15:15:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-07-31T15:15:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nginx": { 2024-07-31T15:15:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nginx", 2024-07-31T15:15:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-07-31T15:15:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nginx container image", 2024-07-31T15:15:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-07-31T15:15:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-07-31T15:15:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-07-31T15:15:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-31T15:15:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-07-31T15:15:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-07-31T15:15:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-31T15:15:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-07-31T15:15:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-07-31T15:15:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-07-31T15:15:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-07-31T15:15:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-31T15:15:24.499 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.499 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.499 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.499 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-07-31T15:15:24.499 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-07-31T15:15:24.499 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.499 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.500 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.500 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-31T15:15:24.500 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.500 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.500 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.500 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-07-31T15:15:24.500 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.501 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.501 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.501 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba": { 2024-07-31T15:15:24.501 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba", 2024-07-31T15:15:24.501 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.501 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.501 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.502 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-31T15:15:24.502 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.502 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.502 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.502 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB container image", 2024-07-31T15:15:24.502 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.502 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.503 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.503 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-07-31T15:15:24.503 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-07-31T15:15:24.503 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.503 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.503 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.503 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-31T15:15:24.503 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.504 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.504 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.504 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-07-31T15:15:24.504 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.504 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.504 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.504 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-07-31T15:15:24.505 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-07-31T15:15:24.505 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.505 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.505 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.505 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.505 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.505 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.505 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.506 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-07-31T15:15:24.506 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.506 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.506 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.506 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-07-31T15:15:24.506 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-07-31T15:15:24.506 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.507 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.507 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.507 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:24.507 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.507 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.507 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.507 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-31T15:15:24.507 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.508 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.508 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.508 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-07-31T15:15:24.508 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-07-31T15:15:24.508 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.508 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.508 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.508 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-07-31T15:15:24.509 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.509 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.509 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.509 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-31T15:15:24.509 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.509 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.509 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.510 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-07-31T15:15:24.510 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-07-31T15:15:24.510 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.510 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.510 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.510 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-07-31T15:15:24.510 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.511 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.511 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.511 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-31T15:15:24.511 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.511 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.511 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.512 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-07-31T15:15:24.512 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-07-31T15:15:24.512 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.512 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.512 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.512 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-07-31T15:15:24.512 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.512 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.512 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.513 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-07-31T15:15:24.513 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.513 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.513 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.513 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-07-31T15:15:24.513 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-07-31T15:15:24.513 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.514 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.514 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.514 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.514 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.514 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.514 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.514 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-31T15:15:24.514 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.515 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.515 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.515 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-07-31T15:15:24.515 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-07-31T15:15:24.515 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.515 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.515 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.516 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:24.516 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.516 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.516 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.516 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-07-31T15:15:24.516 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.516 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.516 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.517 INFO:teuthology.orchestra.run.smithi067.stdout: "grafana_dashboards_path": { 2024-07-31T15:15:24.517 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "grafana_dashboards_path", 2024-07-31T15:15:24.517 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.517 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.517 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.517 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-31T15:15:24.518 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.518 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.518 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.518 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-31T15:15:24.518 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.518 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.518 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.519 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-07-31T15:15:24.519 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-07-31T15:15:24.519 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.519 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.519 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.519 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:24.519 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.519 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.520 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.520 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-07-31T15:15:24.520 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.520 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.520 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.520 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-07-31T15:15:24.520 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-07-31T15:15:24.521 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.521 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.521 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.521 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.521 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.521 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.521 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.522 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-31T15:15:24.522 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.522 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.522 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.522 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-07-31T15:15:24.522 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-07-31T15:15:24.523 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.523 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.523 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.523 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.523 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.523 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.523 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.523 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-31T15:15:24.523 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.524 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.524 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.524 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.524 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.524 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.524 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.524 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.525 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.525 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.525 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.525 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.525 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.525 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.525 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.526 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.526 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.526 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.526 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.526 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.526 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.526 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.526 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.527 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-07-31T15:15:24.527 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-07-31T15:15:24.527 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.527 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.527 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.527 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.527 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.528 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.528 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.528 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-31T15:15:24.528 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.528 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.528 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.528 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.529 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.529 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.529 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.529 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.529 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.529 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.529 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.529 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.529 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-31T15:15:24.530 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.530 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.530 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.530 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.530 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.530 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.530 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.531 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.531 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.531 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.531 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.531 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.531 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.531 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.531 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.531 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.532 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.532 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.532 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.532 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.532 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.532 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.532 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.533 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.533 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.533 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.533 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.533 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.533 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.533 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.533 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.534 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.534 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.534 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.534 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.534 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.534 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-31T15:15:24.535 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-31T15:15:24.535 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.535 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.535 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.535 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.535 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.535 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.535 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.536 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-31T15:15:24.536 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.536 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.536 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.536 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-31T15:15:24.536 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-31T15:15:24.537 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.537 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.537 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.537 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-07-31T15:15:24.537 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.537 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.537 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.537 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-31T15:15:24.537 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.538 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.538 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.538 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-07-31T15:15:24.538 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-07-31T15:15:24.538 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.538 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.539 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.539 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:24.539 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.539 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.539 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.539 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-07-31T15:15:24.539 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.539 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.540 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.540 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-07-31T15:15:24.540 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-07-31T15:15:24.540 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.540 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.540 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.541 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:24.541 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.541 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.541 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.541 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-31T15:15:24.541 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.541 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.541 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.542 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-07-31T15:15:24.542 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-07-31T15:15:24.542 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.542 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.542 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.542 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.543 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.543 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.543 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.543 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-07-31T15:15:24.543 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.543 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.543 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-07-31T15:15:24.544 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-07-31T15:15:24.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.544 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.544 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.544 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-07-31T15:15:24.544 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.545 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.545 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.545 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-07-31T15:15:24.545 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-07-31T15:15:24.545 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.545 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-07-31T15:15:24.545 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.545 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.545 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.546 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-07-31T15:15:24.546 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-07-31T15:15:24.546 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.546 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.546 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.546 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-07-31T15:15:24.546 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.547 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.547 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.547 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-31T15:15:24.547 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.547 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.547 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.548 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-07-31T15:15:24.548 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-07-31T15:15:24.548 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.548 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.548 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.548 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-31T15:15:24.548 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.548 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.549 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.549 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-31T15:15:24.549 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.549 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.549 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.549 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-07-31T15:15:24.549 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-07-31T15:15:24.550 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.550 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.550 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.550 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.550 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.550 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.550 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.551 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-31T15:15:24.551 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.551 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.551 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.551 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-07-31T15:15:24.551 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-07-31T15:15:24.551 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.552 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.552 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.552 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.552 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.552 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.552 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.552 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-31T15:15:24.552 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.552 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.553 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.553 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-07-31T15:15:24.553 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-07-31T15:15:24.553 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.553 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.553 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.554 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.554 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.554 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.554 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.554 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-31T15:15:24.554 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.554 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.554 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.555 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-07-31T15:15:24.555 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-07-31T15:15:24.555 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.555 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.555 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.555 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.556 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.556 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.556 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.556 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-31T15:15:24.556 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.556 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.556 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.557 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-07-31T15:15:24.557 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-07-31T15:15:24.557 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.557 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.557 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.557 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.557 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.558 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.558 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.558 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-31T15:15:24.558 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.558 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.558 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.558 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-07-31T15:15:24.559 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-07-31T15:15:24.559 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.559 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.559 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.559 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-07-31T15:15:24.559 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.559 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.560 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.560 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-07-31T15:15:24.560 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.560 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.560 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.560 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.560 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.561 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.561 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.561 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.561 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.561 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.561 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.561 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.562 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.562 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.562 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.562 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-07-31T15:15:24.562 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-07-31T15:15:24.562 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.563 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.563 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.563 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.563 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.563 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.563 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.563 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-31T15:15:24.563 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.564 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.564 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.564 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-07-31T15:15:24.564 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-07-31T15:15:24.564 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.564 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.564 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.565 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.565 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.565 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.565 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.565 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-31T15:15:24.565 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.565 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.566 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.566 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-07-31T15:15:24.566 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-07-31T15:15:24.566 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.566 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.566 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.567 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.567 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.567 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.567 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.567 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-31T15:15:24.567 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.567 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.568 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.568 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-07-31T15:15:24.568 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-07-31T15:15:24.568 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.568 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.568 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.568 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.569 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.569 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.569 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.569 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-07-31T15:15:24.569 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.569 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.569 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.569 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-07-31T15:15:24.570 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-07-31T15:15:24.570 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.570 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.570 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.570 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.570 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.570 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.571 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.571 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-31T15:15:24.571 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.571 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.571 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.571 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-07-31T15:15:24.571 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-07-31T15:15:24.571 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.572 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.572 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.572 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.572 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.572 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.572 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.572 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-31T15:15:24.572 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.573 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.573 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.573 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.573 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.573 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-07-31T15:15:24.573 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.574 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.574 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.574 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.574 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.574 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.574 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.574 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.575 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.575 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.575 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.575 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.575 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.575 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.575 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.575 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.575 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.576 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.576 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.576 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.576 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.576 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.576 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.576 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.577 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.577 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.577 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.577 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.577 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.577 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.577 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.577 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.578 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.578 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.578 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.578 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.578 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.578 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.579 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.579 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.579 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.579 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.579 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.579 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.579 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.579 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.580 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.580 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.580 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.580 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.580 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.580 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.580 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.580 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.580 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.581 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.581 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.581 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.581 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.581 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.581 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.581 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.582 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.582 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.582 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.582 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.582 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.582 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.582 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.583 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-07-31T15:15:24.583 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-07-31T15:15:24.583 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.583 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.583 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.583 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-07-31T15:15:24.584 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.584 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.584 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.584 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-31T15:15:24.584 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.584 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.584 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.585 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.585 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.585 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.585 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.585 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.585 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.585 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.586 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.586 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.586 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.586 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.586 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.586 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.586 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-07-31T15:15:24.587 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-07-31T15:15:24.587 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.587 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.587 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.587 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-07-31T15:15:24.587 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.587 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.587 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.588 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-31T15:15:24.588 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.588 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.588 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.588 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.588 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.589 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-07-31T15:15:24.589 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.589 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.589 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.589 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-31T15:15:24.589 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-31T15:15:24.589 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.590 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.590 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.590 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:24.590 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.590 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.590 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.590 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.590 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.591 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.591 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.591 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-07-31T15:15:24.591 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-31T15:15:24.591 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.591 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.592 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.592 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.592 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.592 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.592 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.592 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.592 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.592 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.593 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.593 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-31T15:15:24.593 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-31T15:15:24.593 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.593 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.593 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.593 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.594 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.594 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.594 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.594 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.594 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.594 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.594 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.595 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-07-31T15:15:24.595 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-07-31T15:15:24.595 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.595 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.595 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.595 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.595 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.595 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.596 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.596 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.596 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.596 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.596 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.596 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-31T15:15:24.596 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-31T15:15:24.596 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.597 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.597 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.597 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.597 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.597 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.597 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.597 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.598 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.598 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.598 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.598 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-07-31T15:15:24.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-31T15:15:24.598 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.598 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.599 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.599 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.599 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.599 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.599 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.599 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.599 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.599 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.600 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.600 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-31T15:15:24.600 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-31T15:15:24.600 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.600 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.600 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.600 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.601 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.601 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.601 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.601 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.601 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.601 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.601 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.602 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-31T15:15:24.602 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-31T15:15:24.602 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.602 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.602 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.602 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.602 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.602 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.603 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.603 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.603 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.603 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.603 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.603 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-31T15:15:24.603 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-31T15:15:24.603 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.604 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.604 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.604 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.604 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.604 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.604 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.604 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.604 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.604 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.605 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.605 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-31T15:15:24.605 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-31T15:15:24.605 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.605 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.605 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.605 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.606 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.606 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.606 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.606 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.606 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.606 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.607 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.607 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-31T15:15:24.607 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-31T15:15:24.607 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.607 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.607 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.607 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.607 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.609 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.609 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.609 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.610 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.610 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.610 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.610 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-31T15:15:24.610 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-31T15:15:24.610 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.611 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.611 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.611 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.611 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.611 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.611 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.611 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.611 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.611 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.612 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.612 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-31T15:15:24.612 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-31T15:15:24.612 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.612 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.612 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.613 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.613 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.613 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.613 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.613 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.613 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.613 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.613 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.614 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-31T15:15:24.614 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-31T15:15:24.614 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.614 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.614 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.614 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.614 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.614 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.614 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.615 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.615 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.615 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.615 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.615 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-07-31T15:15:24.615 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-31T15:15:24.615 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.616 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.616 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.616 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-31T15:15:24.616 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.616 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.616 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.616 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.616 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.617 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.617 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.617 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-31T15:15:24.617 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-31T15:15:24.617 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.617 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.617 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.618 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.618 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.618 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.618 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.618 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.618 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.618 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.618 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.619 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-07-31T15:15:24.619 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-07-31T15:15:24.619 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.619 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.619 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.619 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.619 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.620 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.620 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.620 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.620 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.620 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.620 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.620 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-07-31T15:15:24.620 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-31T15:15:24.621 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.621 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.621 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.621 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-31T15:15:24.621 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.621 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.621 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.621 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.622 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.622 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.622 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.622 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-31T15:15:24.622 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-31T15:15:24.622 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.622 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.622 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.623 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.623 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.623 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.623 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.623 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.623 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.623 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.623 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.624 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-31T15:15:24.624 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-31T15:15:24.624 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.624 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.624 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.624 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.624 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.624 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.625 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.625 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.625 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.625 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.625 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.625 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-31T15:15:24.625 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-31T15:15:24.626 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.626 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.626 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.626 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.626 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.626 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.626 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.627 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.627 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.627 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.627 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.627 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.627 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-31T15:15:24.627 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-31T15:15:24.627 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.628 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.628 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.628 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.628 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.628 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.628 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.628 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.628 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.629 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.629 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.629 INFO:teuthology.orchestra.run.smithi067.stdout: "MANAGED_BY_CLUSTERS": { 2024-07-31T15:15:24.629 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-07-31T15:15:24.629 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.629 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.629 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.629 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "[]", 2024-07-31T15:15:24.630 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.630 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.630 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.630 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.630 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.630 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.630 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.631 INFO:teuthology.orchestra.run.smithi067.stdout: "MULTICLUSTER_CONFIG": { 2024-07-31T15:15:24.631 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MULTICLUSTER_CONFIG", 2024-07-31T15:15:24.631 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.631 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.631 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.631 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "{}", 2024-07-31T15:15:24.631 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.631 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.631 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.632 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.632 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.632 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.632 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.632 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.632 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-07-31T15:15:24.632 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-31T15:15:24.632 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.632 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.633 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.633 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.633 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.633 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.633 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.633 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.633 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.633 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.634 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.634 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-31T15:15:24.634 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-31T15:15:24.634 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.634 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.634 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.634 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.635 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.635 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.635 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.635 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.635 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.635 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.635 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.636 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-31T15:15:24.636 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-31T15:15:24.636 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.636 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.636 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.636 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.636 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.637 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.637 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.637 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.637 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.637 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.637 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.637 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.637 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-31T15:15:24.637 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-31T15:15:24.638 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.638 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.638 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.638 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.638 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.638 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.639 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.639 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.639 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.639 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.639 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.639 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.639 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-31T15:15:24.639 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-31T15:15:24.640 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.640 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.640 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.640 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.640 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.640 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.640 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.640 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.641 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.641 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.641 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.641 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-31T15:15:24.641 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-31T15:15:24.641 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.642 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.642 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.642 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.642 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.642 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.642 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.642 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.642 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.643 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.643 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.643 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.643 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-31T15:15:24.643 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-31T15:15:24.643 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.643 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.644 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.644 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.644 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.644 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.644 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.644 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.644 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.644 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.645 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.645 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-31T15:15:24.645 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-31T15:15:24.645 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.645 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.645 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.645 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.646 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.646 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.646 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.646 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.646 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.646 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.646 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.647 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-31T15:15:24.647 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-31T15:15:24.647 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.647 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.647 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.647 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.647 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.648 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.648 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.648 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.648 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.648 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.648 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.649 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-07-31T15:15:24.649 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-31T15:15:24.649 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.649 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.649 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.649 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.649 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.649 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.649 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.650 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.650 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.650 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.650 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.650 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-31T15:15:24.650 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-31T15:15:24.651 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.651 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.651 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.651 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-31T15:15:24.651 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.651 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.651 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.651 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.652 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.652 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.652 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.652 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.652 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-31T15:15:24.652 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-31T15:15:24.652 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.653 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.653 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.653 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:24.653 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.653 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.653 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.653 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.654 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.654 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.654 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.654 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.654 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-31T15:15:24.654 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-31T15:15:24.654 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.655 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.655 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.655 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-07-31T15:15:24.655 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.655 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.655 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.655 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.655 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.656 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.656 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.656 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-31T15:15:24.656 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-31T15:15:24.656 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.656 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.656 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.657 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-07-31T15:15:24.657 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.657 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.657 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.657 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.657 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.657 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.658 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.658 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.658 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-07-31T15:15:24.658 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-31T15:15:24.658 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.658 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.658 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.658 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.659 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.659 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.659 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.659 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.659 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.659 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.659 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.660 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-31T15:15:24.660 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-31T15:15:24.660 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.660 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.660 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.660 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-31T15:15:24.660 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.660 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.661 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.661 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.661 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.661 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.661 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.661 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.661 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-07-31T15:15:24.662 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-31T15:15:24.662 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.662 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.662 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.662 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.662 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.662 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.663 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.663 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.663 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.663 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.663 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.663 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-07-31T15:15:24.663 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-31T15:15:24.664 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.664 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.664 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.664 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.664 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.664 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.664 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.665 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.665 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.665 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.665 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.665 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.665 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-07-31T15:15:24.665 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-07-31T15:15:24.665 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.666 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.666 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.666 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.666 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.666 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.666 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.666 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.666 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.667 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.667 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.667 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.667 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-31T15:15:24.667 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-31T15:15:24.667 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.667 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.668 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.668 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.668 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.668 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.668 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.668 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.668 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.668 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.668 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.669 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.669 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-31T15:15:24.669 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-31T15:15:24.669 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.669 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.669 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.669 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:24.669 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.670 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.670 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.670 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.670 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.670 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.670 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.670 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.671 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-31T15:15:24.671 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-31T15:15:24.671 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.671 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.671 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.671 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:24.671 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.671 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.672 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.672 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.672 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.672 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.672 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.672 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.672 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-07-31T15:15:24.672 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-07-31T15:15:24.672 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.673 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.673 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.673 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.673 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.673 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.673 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.673 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.674 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.674 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.674 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.674 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.674 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-07-31T15:15:24.674 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-07-31T15:15:24.674 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.674 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.674 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.675 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.675 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.675 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.675 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.675 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.675 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.675 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.676 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.676 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.676 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-07-31T15:15:24.676 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-07-31T15:15:24.676 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.676 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.676 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.676 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.676 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.677 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.677 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.677 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-07-31T15:15:24.677 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.677 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.677 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.677 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.678 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-07-31T15:15:24.678 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-07-31T15:15:24.678 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.678 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.678 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.678 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-07-31T15:15:24.678 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.678 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.679 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.679 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.679 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.679 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.679 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.679 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.679 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-07-31T15:15:24.679 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-07-31T15:15:24.680 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.680 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.680 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.680 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.680 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.680 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.680 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.680 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.681 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.681 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.681 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.681 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.681 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.681 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.681 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.681 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.682 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.682 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.682 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.682 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.682 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.682 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.682 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.683 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.683 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.683 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.683 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.683 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.683 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.683 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.683 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-07-31T15:15:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-07-31T15:15:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-07-31T15:15:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-07-31T15:15:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-07-31T15:15:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-31T15:15:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-31T15:15:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-07-31T15:15:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-31T15:15:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-31T15:15:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.696 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.696 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.696 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-07-31T15:15:24.696 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.696 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.696 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.696 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.697 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.697 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.697 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.697 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.697 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.697 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.697 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.697 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.698 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.698 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.698 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.698 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.698 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.698 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.698 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.698 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.699 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.699 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.699 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-07-31T15:15:24.699 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-07-31T15:15:24.699 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.699 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.699 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.699 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.700 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.700 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.700 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.700 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.700 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.700 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.700 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.700 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.701 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-07-31T15:15:24.701 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-07-31T15:15:24.701 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.701 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.701 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.701 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-07-31T15:15:24.701 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.701 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.702 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.702 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.702 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.702 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.702 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.702 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.702 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-07-31T15:15:24.702 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-07-31T15:15:24.703 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.703 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.703 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.703 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-07-31T15:15:24.703 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.703 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.703 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.703 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.704 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-07-31T15:15:24.704 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.704 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.704 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.704 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.704 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.704 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-07-31T15:15:24.705 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-07-31T15:15:24.705 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.705 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.705 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.705 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-07-31T15:15:24.705 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-07-31T15:15:24.705 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-07-31T15:15:24.705 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.705 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.706 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.706 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.706 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.706 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-07-31T15:15:24.706 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-07-31T15:15:24.706 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.707 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.707 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.707 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.707 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.707 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.707 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.707 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.707 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.708 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.708 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.708 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.708 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.708 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.708 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.708 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-07-31T15:15:24.708 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.708 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.709 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.709 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-07-31T15:15:24.709 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-07-31T15:15:24.709 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.709 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.709 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.709 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.710 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.710 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.710 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.710 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-07-31T15:15:24.710 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.710 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.710 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.711 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.711 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.711 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.711 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.711 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.711 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.711 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.712 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.712 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.712 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.712 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.712 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.712 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.712 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.712 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.713 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.713 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.713 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.713 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.713 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.713 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.713 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.713 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.714 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.714 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.714 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.714 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.714 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.714 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.714 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.714 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.715 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.715 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.715 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.715 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.715 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.715 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.715 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.715 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.715 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.716 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.716 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.716 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.716 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.716 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.716 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.716 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.716 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.717 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.717 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.717 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.717 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.717 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.717 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.718 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.718 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.718 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.718 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.718 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.718 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.718 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.718 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.719 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.719 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.719 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.719 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.719 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.719 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-07-31T15:15:24.719 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-07-31T15:15:24.719 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.720 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.720 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.720 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-07-31T15:15:24.720 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.720 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.720 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.720 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-31T15:15:24.720 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.720 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.721 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.721 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-07-31T15:15:24.721 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-07-31T15:15:24.721 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.721 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.721 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.721 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-07-31T15:15:24.722 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.722 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.722 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.722 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-31T15:15:24.722 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.722 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.722 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.722 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.723 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-07-31T15:15:24.723 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-07-31T15:15:24.723 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.723 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.723 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.723 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-07-31T15:15:24.723 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.723 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.724 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.724 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-07-31T15:15:24.724 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.724 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.724 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.724 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-07-31T15:15:24.724 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-07-31T15:15:24.725 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.725 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.725 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.725 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-07-31T15:15:24.725 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.725 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.725 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.725 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-31T15:15:24.725 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.726 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.726 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.726 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.726 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-07-31T15:15:24.726 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-07-31T15:15:24.726 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.726 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.726 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.727 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.727 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.727 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.727 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.727 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-31T15:15:24.727 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.727 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.727 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.728 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:24.728 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:24.728 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.728 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.728 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.728 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:24.728 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.728 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.729 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.729 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-07-31T15:15:24.729 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.729 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.729 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.729 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.729 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.730 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.730 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.730 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.730 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.730 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.730 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.730 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.730 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.731 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.731 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.731 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.731 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-07-31T15:15:24.731 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-07-31T15:15:24.731 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.731 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.732 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.732 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-07-31T15:15:24.732 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.732 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.732 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.732 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-31T15:15:24.732 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.732 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.732 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-07-31T15:15:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.740 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.741 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.741 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.741 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.741 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.741 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.741 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.742 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.742 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.742 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.742 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.742 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.742 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.742 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.742 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-07-31T15:15:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-07-31T15:15:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-07-31T15:15:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-07-31T15:15:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-07-31T15:15:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-07-31T15:15:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-07-31T15:15:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-07-31T15:15:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-07-31T15:15:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-07-31T15:15:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-07-31T15:15:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-07-31T15:15:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-31T15:15:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-07-31T15:15:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-07-31T15:15:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-07-31T15:15:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-31T15:15:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-07-31T15:15:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-07-31T15:15:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-07-31T15:15:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.757 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:24.757 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:24.757 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.757 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.757 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.757 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-07-31T15:15:24.757 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-07-31T15:15:24.758 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.758 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.758 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-31T15:15:24.758 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.758 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.758 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.759 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.759 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.759 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.759 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.759 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.759 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.759 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.759 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.760 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.760 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.760 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.760 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.760 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.760 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.760 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.760 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.761 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.761 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.761 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.761 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.761 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.761 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.761 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.761 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.762 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.762 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.762 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.762 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.762 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.762 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.762 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.763 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.763 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.763 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.763 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.763 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.763 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.763 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.763 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.764 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.764 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.764 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.764 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.764 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.764 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.764 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.764 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.765 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.765 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.765 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.765 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.765 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.765 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.765 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.766 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.766 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.766 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.766 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.766 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.766 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.766 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.767 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.767 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.767 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.767 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.767 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.767 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.768 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-07-31T15:15:24.768 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-07-31T15:15:24.768 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.768 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.768 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.768 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.768 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.768 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.768 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.769 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-07-31T15:15:24.769 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.769 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.769 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.769 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-07-31T15:15:24.769 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-07-31T15:15:24.769 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.770 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.770 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.770 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-07-31T15:15:24.770 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.770 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.770 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.770 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-07-31T15:15:24.771 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.771 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.771 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.771 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.771 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.771 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.771 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.772 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.772 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.772 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.772 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.772 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.772 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.772 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.772 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.773 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.773 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-07-31T15:15:24.773 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-07-31T15:15:24.773 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.773 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.773 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.773 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-07-31T15:15:24.774 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.774 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.774 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.774 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-31T15:15:24.774 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.774 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.774 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.775 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-07-31T15:15:24.775 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-07-31T15:15:24.775 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.775 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.775 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.775 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:24.775 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-31T15:15:24.775 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-07-31T15:15:24.775 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.776 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-31T15:15:24.776 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.776 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.776 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.776 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-07-31T15:15:24.776 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-07-31T15:15:24.777 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.777 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.777 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.777 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.777 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.777 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.777 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.777 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-07-31T15:15:24.778 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.778 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.778 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.778 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-07-31T15:15:24.778 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-07-31T15:15:24.778 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.778 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.779 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.779 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-07-31T15:15:24.779 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.779 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.779 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.779 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-31T15:15:24.779 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.780 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.780 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.780 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.780 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.780 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.780 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-07-31T15:15:24.780 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.780 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.781 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.781 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.781 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.781 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.781 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.781 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.781 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.782 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.782 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.782 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.782 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.782 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.782 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.782 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.782 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.782 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.783 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.783 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.783 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.783 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.783 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.783 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.783 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.784 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.784 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.784 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.784 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.784 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.784 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.784 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.784 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.785 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.785 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.785 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.785 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.785 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.785 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.785 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.786 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.786 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.786 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.786 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.786 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.786 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.786 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.787 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.787 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.787 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.787 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.787 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.787 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.787 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.788 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.788 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.788 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.788 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.788 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.788 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.789 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.789 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.789 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.789 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.789 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.789 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.789 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.789 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.790 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.790 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.790 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.790 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.790 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.790 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.791 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.791 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.791 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.791 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.791 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.791 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.791 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.791 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.791 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.792 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.792 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.792 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-07-31T15:15:24.792 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.792 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.792 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.792 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.793 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.793 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.793 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.793 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.793 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.793 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.794 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.794 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.794 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.794 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.794 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.794 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.794 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.794 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.795 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.795 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.795 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.795 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.795 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.795 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.795 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.796 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.796 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.796 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.796 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.796 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.796 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.796 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.796 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.797 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.797 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.797 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.797 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.797 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.797 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.797 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.797 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.798 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.798 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.798 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.798 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.798 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.798 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.798 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.798 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.799 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.799 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.799 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.799 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.799 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.799 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.799 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.800 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.800 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.800 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.800 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.800 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.800 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.800 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.800 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.801 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.801 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.801 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.801 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.801 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.801 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.802 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.802 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.802 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.802 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.802 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.802 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.802 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.803 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.803 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.803 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.803 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.803 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.803 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.803 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-07-31T15:15:24.804 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.804 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.804 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.804 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-07-31T15:15:24.804 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-07-31T15:15:24.804 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.804 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.804 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.805 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-07-31T15:15:24.805 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.805 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.805 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.805 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-31T15:15:24.805 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.805 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.805 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.806 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-07-31T15:15:24.806 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-07-31T15:15:24.806 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.806 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.806 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.806 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:24.807 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-07-31T15:15:24.807 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.807 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.807 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-31T15:15:24.807 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.807 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.807 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.808 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.808 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.808 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.808 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.808 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.808 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.808 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.808 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.809 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.809 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.809 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.809 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.809 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.809 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.809 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.809 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.810 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.810 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.810 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.810 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.810 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.810 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.811 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.811 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.811 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.811 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.811 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.811 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.811 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.811 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.812 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.812 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.812 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.812 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.812 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.812 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.813 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.813 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.813 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.813 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.813 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.813 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.813 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.813 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.814 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.814 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.814 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.814 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.814 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.814 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.814 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.814 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.814 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.815 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.815 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.815 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.815 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.815 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.815 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.816 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.816 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.816 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.816 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.816 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.816 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.816 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.817 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.817 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.817 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.817 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.817 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.817 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.817 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.818 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.818 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.818 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.818 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.818 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.818 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.818 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.819 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-07-31T15:15:24.819 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.819 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.819 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.819 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-07-31T15:15:24.819 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-07-31T15:15:24.820 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.820 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.820 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.820 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-07-31T15:15:24.820 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.820 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.820 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.820 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-07-31T15:15:24.821 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-31T15:15:24.821 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.821 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.821 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.821 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.821 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.821 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.822 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.822 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.822 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.822 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.822 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.822 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.822 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.822 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.823 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.823 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.823 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.823 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.823 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.823 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.823 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.823 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.824 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.824 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.824 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.824 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.824 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.824 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.824 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.824 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.825 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.825 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.825 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.825 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.825 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.825 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.825 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.826 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.826 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.826 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.826 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.826 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.826 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.826 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.827 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.827 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.827 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.827 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.827 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.827 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.827 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.828 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.828 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.828 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.828 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.828 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.828 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.828 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.828 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.829 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.829 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.829 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.829 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.829 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.829 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.829 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.830 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.830 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.830 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-07-31T15:15:24.830 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-07-31T15:15:24.830 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.830 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.830 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.831 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.831 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.831 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.831 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.831 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-07-31T15:15:24.831 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-31T15:15:24.831 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.831 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.832 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-07-31T15:15:24.832 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-07-31T15:15:24.832 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.832 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.832 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.832 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-07-31T15:15:24.832 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.832 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.833 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.833 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-07-31T15:15:24.833 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.833 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.833 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.833 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-07-31T15:15:24.833 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-07-31T15:15:24.834 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.834 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.834 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.834 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-07-31T15:15:24.834 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.834 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.834 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.835 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-07-31T15:15:24.835 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.835 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.835 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.835 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-07-31T15:15:24.835 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-07-31T15:15:24.835 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.835 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.836 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.836 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.836 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.836 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.836 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.836 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-07-31T15:15:24.836 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.837 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.837 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.837 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.837 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.837 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.837 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.837 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.838 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.838 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.838 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.838 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.838 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.838 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.838 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.839 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.839 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-07-31T15:15:24.839 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-07-31T15:15:24.839 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.839 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.839 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.839 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-07-31T15:15:24.840 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.840 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.840 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.840 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-31T15:15:24.840 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-31T15:15:24.840 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.840 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.840 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.841 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.841 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.841 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-07-31T15:15:24.841 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.841 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.841 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.842 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.842 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.842 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.842 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.842 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.842 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.842 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.842 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.843 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.843 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.843 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.843 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.843 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.843 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.843 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.844 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.844 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.844 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.844 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.844 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.844 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.844 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.844 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.844 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.845 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.845 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.845 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.845 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.845 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.845 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.845 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.845 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.845 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.846 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.846 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.846 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.846 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.846 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.846 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.846 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.847 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.847 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.847 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.847 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.847 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.847 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.847 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.847 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.848 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.848 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.848 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.848 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.848 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.848 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.848 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.849 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.849 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.849 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.849 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.849 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.849 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.849 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.849 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.850 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.850 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.850 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.850 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.850 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.850 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.850 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.851 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.851 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.851 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.851 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.851 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.851 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.851 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.851 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.852 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.852 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.852 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.852 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-07-31T15:15:24.852 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.853 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.853 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.853 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.853 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.853 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.853 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.853 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.853 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.854 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.854 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.854 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.854 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.854 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.854 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.854 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.855 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.855 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.855 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.855 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.855 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.855 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.855 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.855 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.856 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.856 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.856 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.856 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.856 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.856 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.856 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.856 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.857 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.857 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.857 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.857 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.857 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.857 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.858 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.858 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.858 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.858 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.858 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.858 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.858 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.859 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.859 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.859 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.859 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.859 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.859 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.859 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.859 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.860 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.860 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.860 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.860 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.860 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.860 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.861 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.861 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.861 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.861 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.861 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.861 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.861 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.862 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.862 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.862 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.862 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.862 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.862 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.863 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.863 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.863 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.863 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.863 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.863 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.863 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.863 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.864 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.864 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.864 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.864 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.864 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.864 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-07-31T15:15:24.864 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.865 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.865 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.865 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.865 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.865 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.865 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.865 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.865 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.866 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.866 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.866 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.866 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.866 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.866 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.866 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.866 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.867 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.867 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.867 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.867 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.867 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.867 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.868 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.868 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.868 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.868 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.868 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.868 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.868 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.868 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.869 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.870 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.871 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.871 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.871 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.871 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.871 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.871 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.871 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.872 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.872 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.872 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.872 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.872 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.872 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.872 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.873 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.873 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.873 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.873 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.873 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.873 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.873 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.874 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.874 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.874 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.874 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.874 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.874 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.874 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.875 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.875 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.875 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.875 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.875 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.875 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.875 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.876 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.876 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.876 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.876 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.876 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.876 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.876 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-07-31T15:15:24.877 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.877 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.877 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.877 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-07-31T15:15:24.877 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-07-31T15:15:24.877 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.877 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.877 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.878 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.878 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.878 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.878 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.878 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-31T15:15:24.878 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.878 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.879 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.879 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.879 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.879 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.879 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.879 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.879 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.879 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.879 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.880 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.880 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.880 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.880 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.880 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.880 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.880 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.881 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.881 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.881 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.881 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.881 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.881 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.882 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.882 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.882 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.882 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.882 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.882 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.882 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.882 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.883 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.883 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.883 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.883 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.883 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.883 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.883 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.884 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.884 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.884 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.884 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.884 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.884 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.884 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.884 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.885 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.885 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.885 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.885 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.885 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.885 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.885 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.886 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.886 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.886 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.886 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.886 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.886 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.887 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.887 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.887 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.887 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.887 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.887 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.887 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.887 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.888 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.888 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.888 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.888 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-07-31T15:15:24.888 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-07-31T15:15:24.888 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.888 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.888 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.889 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.889 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.889 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.889 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.889 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-07-31T15:15:24.889 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-07-31T15:15:24.889 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-07-31T15:15:24.889 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.890 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-07-31T15:15:24.890 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.890 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.890 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.890 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.890 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.890 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.890 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.890 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.891 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.891 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.891 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.891 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.891 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.891 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.891 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.891 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.891 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.892 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.892 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.892 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.892 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.892 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-07-31T15:15:24.892 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.892 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.893 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.893 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.893 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.893 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.893 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.893 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.893 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.893 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.894 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.894 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.894 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.894 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.894 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.894 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.894 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.894 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.895 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.895 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.895 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.895 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.895 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.895 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.896 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.896 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.896 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.896 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.896 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.896 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.896 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.896 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.897 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.897 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.897 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.897 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.897 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.897 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.897 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.898 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.898 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.898 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.898 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.898 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.898 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.898 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.898 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.899 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.899 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.899 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.899 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.899 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.899 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.899 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.899 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.900 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.900 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.900 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.900 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.900 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.900 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.900 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.901 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.901 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.901 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.901 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.901 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.901 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.901 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.901 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.902 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.902 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.902 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.902 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.902 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.902 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.902 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.902 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.903 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.903 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.903 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.903 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.903 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.903 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.903 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.903 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.904 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.904 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-07-31T15:15:24.904 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.904 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.904 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.904 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.904 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.904 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.905 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.905 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.905 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.905 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.905 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.905 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.905 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.905 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.906 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.906 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.906 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.906 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.906 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.906 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.907 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.907 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.907 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.907 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.907 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.907 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.908 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.908 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.908 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.908 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.908 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.908 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.908 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.908 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.908 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.909 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.909 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.909 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.909 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.909 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.909 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.909 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.909 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.910 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.910 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.910 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.910 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.910 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.910 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.910 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.910 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.911 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.911 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.911 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.911 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.911 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.911 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.911 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.912 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.912 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.912 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.912 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.912 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.912 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.912 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.912 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.912 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.913 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.913 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.913 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.913 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.913 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.913 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.913 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.914 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.914 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.914 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.914 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.914 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.914 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.914 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.914 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.915 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.915 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.915 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-07-31T15:15:24.915 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.915 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.915 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.915 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.916 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.916 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.916 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.916 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.916 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.916 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.916 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.916 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.916 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.917 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.917 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.917 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.917 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.917 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.917 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.917 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.917 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.918 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.918 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.918 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.918 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.918 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.918 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.918 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.919 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.919 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.919 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.919 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.919 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.919 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.919 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.919 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.920 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.920 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.920 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.920 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.920 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.920 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.920 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.921 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.921 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.921 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.921 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.921 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.921 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.921 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.922 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.922 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.922 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.922 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.922 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.922 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.922 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.922 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.923 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.923 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.923 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.923 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.923 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.923 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.923 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.923 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.924 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.924 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.924 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.924 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:24.924 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:24.924 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.924 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.925 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.925 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:24.925 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.925 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.925 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.925 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.925 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.926 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.926 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.926 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.926 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.926 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.926 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.926 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.926 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.927 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.927 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.927 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.927 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.927 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.927 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.927 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.927 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.928 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-07-31T15:15:24.928 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-07-31T15:15:24.928 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:24.928 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.928 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.928 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-07-31T15:15:24.928 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-07-31T15:15:24.929 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.929 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.929 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-07-31T15:15:24.929 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-31T15:15:24.929 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.929 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.929 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.930 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.930 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.930 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-07-31T15:15:24.930 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.930 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.930 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.930 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-07-31T15:15:24.931 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-07-31T15:15:24.931 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.931 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.931 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.931 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.931 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.931 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.932 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.932 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-31T15:15:24.932 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.932 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.932 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.932 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-07-31T15:15:24.932 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-07-31T15:15:24.932 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.933 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.933 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.933 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.933 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.933 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.933 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.933 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.933 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.934 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.934 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.934 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.934 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.934 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.934 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.935 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.935 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.935 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.935 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.935 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.935 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.935 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.935 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.936 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.936 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.936 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.936 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.936 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.936 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.936 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.936 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.937 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.937 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.937 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.937 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.937 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.937 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.937 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.938 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.938 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.938 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.938 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.938 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.938 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.938 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.939 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.939 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.939 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.939 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.939 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.939 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.939 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.939 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.940 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.940 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.940 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.940 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.940 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.940 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.940 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.940 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.940 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.941 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.941 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.941 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.941 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.941 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.941 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.941 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.942 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.942 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.942 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.942 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.942 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.942 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.942 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.942 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.943 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-07-31T15:15:24.943 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-07-31T15:15:24.943 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.943 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.943 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.943 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-07-31T15:15:24.943 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.943 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.944 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.944 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-07-31T15:15:24.944 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.944 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.944 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.944 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:24.944 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:24.945 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:24.945 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.945 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.945 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:24.945 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.945 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.945 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.945 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-07-31T15:15:24.946 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.946 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.946 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.946 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.946 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.946 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.947 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.947 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.947 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.947 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.947 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.947 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.947 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.948 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.948 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.948 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.948 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.948 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.948 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.948 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-07-31T15:15:24.948 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.949 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.949 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.949 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-07-31T15:15:24.949 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-07-31T15:15:24.949 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.949 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.949 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.950 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.950 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.950 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.950 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.950 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.950 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.950 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.950 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.951 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-07-31T15:15:24.951 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-07-31T15:15:24.951 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.951 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.951 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.951 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.951 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.951 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.952 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.952 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-31T15:15:24.952 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-31T15:15:24.952 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.952 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.953 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.953 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.953 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.953 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.953 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.953 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.953 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.954 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.954 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.954 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.954 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.954 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.954 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.954 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.955 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.955 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.955 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.955 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.955 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.955 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.955 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.955 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.956 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.956 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.956 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.956 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.956 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.956 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.956 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.957 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.957 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.957 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.957 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.957 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.957 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.957 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.957 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.958 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.958 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.958 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.958 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.958 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.958 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.958 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.958 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.959 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.959 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.959 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.959 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.959 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.959 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.959 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.959 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.959 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.960 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.960 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.960 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.960 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.960 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.960 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.960 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.960 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.961 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.961 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.961 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.961 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.961 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.961 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-07-31T15:15:24.961 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-07-31T15:15:24.962 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.962 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.962 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.962 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.962 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.962 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.962 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.962 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.963 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.963 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.963 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.963 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-31T15:15:24.963 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-31T15:15:24.963 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.963 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.963 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.964 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-07-31T15:15:24.964 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.964 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.964 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.964 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.964 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.964 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.964 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.965 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-07-31T15:15:24.965 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-07-31T15:15:24.965 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:24.965 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.965 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.965 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-07-31T15:15:24.965 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.965 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.966 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.966 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.966 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.966 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.966 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.966 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.966 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-31T15:15:24.966 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-31T15:15:24.967 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.967 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.967 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.967 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-07-31T15:15:24.967 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.967 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.967 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.968 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-31T15:15:24.968 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.968 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.968 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.968 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-31T15:15:24.968 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-31T15:15:24.968 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-07-31T15:15:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-31T15:15:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-07-31T15:15:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-07-31T15:15:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-07-31T15:15:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-07-31T15:15:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-07-31T15:15:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-07-31T15:15:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-07-31T15:15:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-07-31T15:15:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-07-31T15:15:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-07-31T15:15:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-07-31T15:15:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-07-31T15:15:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-07-31T15:15:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-07-31T15:15:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.980 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.980 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.980 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.980 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.980 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.980 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.980 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.981 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.981 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.981 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.981 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:24.981 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:24.981 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.981 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.981 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.982 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.982 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.982 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.982 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.982 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.982 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.982 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.982 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.983 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:24.983 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:24.983 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.983 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.983 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.983 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:24.984 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.984 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.984 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:24.984 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:24.984 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:24.984 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:24.984 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:24.984 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:24.984 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:24.985 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:24.985 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.985 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.985 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.985 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.985 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:24.985 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:24.986 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.986 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.986 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.986 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:24.986 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.986 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.986 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.986 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.987 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.987 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.987 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.987 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-07-31T15:15:24.987 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-07-31T15:15:24.987 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.987 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.988 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.988 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:24.988 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.988 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.988 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.988 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.988 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.988 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.989 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.989 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-07-31T15:15:24.989 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-07-31T15:15:24.989 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.989 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.989 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.989 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.990 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.990 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.990 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.990 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.990 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.990 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.990 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.991 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:24.991 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:24.991 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:24.991 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:24.991 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.991 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:24.991 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.991 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.992 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.992 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.992 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.992 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.992 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.992 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-07-31T15:15:24.992 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-07-31T15:15:24.993 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.993 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.993 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.993 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.993 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.993 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.993 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.993 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.994 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.994 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.994 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.994 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.994 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:24.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.994 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:24.994 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-07-31T15:15:24.995 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:24.995 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:24.995 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:24.995 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-07-31T15:15:24.995 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-07-31T15:15:24.995 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:24.995 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.996 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.996 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:24.996 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.996 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.996 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.996 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.996 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.996 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.996 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.997 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-07-31T15:15:24.997 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-07-31T15:15:24.997 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.997 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.997 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:24.997 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.997 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.998 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:24.998 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:24.998 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:24.998 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:24.998 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:24.998 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:24.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:24.999 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:24.999 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:24.999 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:24.999 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:24.999 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:24.999 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:24.999 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:24.999 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.000 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.000 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.000 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.000 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.000 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.000 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.000 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.001 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.001 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.001 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.001 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.001 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.001 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.001 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.002 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.002 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.002 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.002 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.002 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.002 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.002 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.002 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.003 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.003 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.003 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.003 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.003 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.003 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.004 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.004 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.004 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.004 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.004 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.004 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.004 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.004 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.005 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.005 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.005 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.005 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.005 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.005 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.005 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.006 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.006 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.006 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.006 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.006 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.006 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.007 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.007 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.007 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.007 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.007 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.007 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.007 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.007 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.008 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.008 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.008 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-31T15:15:25.008 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-31T15:15:25.008 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.008 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.008 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.008 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.009 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.009 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.009 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.009 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.009 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.009 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.009 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.010 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-31T15:15:25.010 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-31T15:15:25.010 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.010 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.010 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.010 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.010 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.010 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.011 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.011 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.011 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.011 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.011 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.011 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.011 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.011 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.012 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.012 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.012 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.012 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.012 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.012 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.012 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.012 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.013 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.013 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.013 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.013 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.013 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.013 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.014 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-07-31T15:15:25.014 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.014 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.014 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.014 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.014 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.014 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.015 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.015 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.015 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.015 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.015 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.015 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.015 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.015 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.016 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.016 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.016 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.016 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.016 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.016 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.016 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.016 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.017 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.017 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.017 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.017 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.017 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.017 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.017 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.018 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.018 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.018 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.018 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.018 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.018 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.018 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.019 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.019 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.019 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.019 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.019 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.019 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.019 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.019 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.020 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.020 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.020 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.020 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.020 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.020 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.020 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.020 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.021 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.021 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.021 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.021 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.021 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.021 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.021 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.022 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.022 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.022 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.022 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.022 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.022 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.023 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.023 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.023 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.023 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.023 INFO:teuthology.orchestra.run.smithi067.stdout: "secondary_zone_period_retry_limit": { 2024-07-31T15:15:25.023 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secondary_zone_period_retry_limit", 2024-07-31T15:15:25.023 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.023 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.024 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.024 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:25.024 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.024 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.024 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.024 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-07-31T15:15:25.024 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.024 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.025 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.025 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.025 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.025 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.025 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.025 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.026 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.026 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.026 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.026 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.026 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.026 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.026 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.027 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.027 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.027 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.027 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.027 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-07-31T15:15:25.027 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.028 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.028 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.028 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.028 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.028 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.028 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.028 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.028 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.028 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.029 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.029 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.029 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.029 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.029 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.029 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.029 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.029 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.030 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.030 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.030 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.030 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.030 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.030 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.030 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.031 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.031 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.031 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.031 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.031 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.031 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.031 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.031 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.032 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.032 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.032 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.032 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.032 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.032 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.032 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.033 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.033 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.033 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.033 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.033 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.033 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.033 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.033 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.033 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.034 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.034 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.034 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.034 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.034 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.034 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.034 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.035 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.035 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.035 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.035 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.035 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.035 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.035 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.035 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.036 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.036 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.036 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.036 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.036 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.036 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.036 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.037 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.038 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.038 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.038 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.038 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.038 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-07-31T15:15:25.038 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-07-31T15:15:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-07-31T15:15:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-31T15:15:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.040 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.040 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.040 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.040 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.040 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.040 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.040 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-07-31T15:15:25.041 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.041 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.041 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.041 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.041 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.041 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.041 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.041 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.042 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.042 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.042 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.042 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.042 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.042 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.042 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.043 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.043 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.043 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.043 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.043 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.043 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.043 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.043 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.044 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.044 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.044 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.044 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.044 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.044 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.044 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.044 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.045 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.045 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.045 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.045 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.045 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.045 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.045 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.045 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.046 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.046 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.046 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.046 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.046 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.046 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.046 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.047 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.047 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.047 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.047 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.047 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.047 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.047 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.048 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.048 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.048 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.048 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.048 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.048 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.049 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.049 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.049 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.049 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.049 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.049 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.049 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.049 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.050 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.050 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.050 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.050 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-07-31T15:15:25.050 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-07-31T15:15:25.050 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.050 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.051 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.051 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.051 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.051 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.051 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.051 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.051 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.051 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.052 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.052 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-07-31T15:15:25.052 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-07-31T15:15:25.052 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.052 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.052 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.052 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-07-31T15:15:25.053 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.053 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.053 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.053 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.053 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.053 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.053 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.053 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-07-31T15:15:25.054 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-07-31T15:15:25.054 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.054 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.054 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.054 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.054 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.054 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.054 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.055 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.055 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.055 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.055 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.055 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-07-31T15:15:25.055 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-07-31T15:15:25.055 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.056 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.056 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.056 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.056 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.056 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.056 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.056 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.056 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.057 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.057 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.057 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-07-31T15:15:25.057 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-07-31T15:15:25.057 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:25.057 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.057 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.058 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.058 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.058 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.058 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.058 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.058 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.058 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.058 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-07-31T15:15:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-07-31T15:15:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-07-31T15:15:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-07-31T15:15:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-07-31T15:15:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-07-31T15:15:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-07-31T15:15:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-07-31T15:15:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-31T15:15:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-07-31T15:15:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-07-31T15:15:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-07-31T15:15:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-07-31T15:15:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-07-31T15:15:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-07-31T15:15:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-07-31T15:15:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smb", 2024-07-31T15:15:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "internal_store_backend": { 2024-07-31T15:15:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "internal_store_backend", 2024-07-31T15:15:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-07-31T15:15:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "update_orchestration": { 2024-07-31T15:15:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "update_orchestration", 2024-07-31T15:15:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-07-31T15:15:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-07-31T15:15:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-07-31T15:15:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-07-31T15:15:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-07-31T15:15:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-07-31T15:15:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-07-31T15:15:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-07-31T15:15:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-07-31T15:15:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-07-31T15:15:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-07-31T15:15:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-07-31T15:15:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-07-31T15:15:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-31T15:15:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-07-31T15:15:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-07-31T15:15:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-07-31T15:15:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-07-31T15:15:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-31T15:15:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-07-31T15:15:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-07-31T15:15:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-31T15:15:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-07-31T15:15:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-07-31T15:15:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-31T15:15:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-07-31T15:15:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-07-31T15:15:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-31T15:15:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-07-31T15:15:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-07-31T15:15:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-31T15:15:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-07-31T15:15:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-07-31T15:15:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.152 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.152 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.152 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.152 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.152 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.152 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.152 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.153 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-07-31T15:15:25.153 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-07-31T15:15:25.153 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.153 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.153 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.153 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.153 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.153 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-07-31T15:15:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-07-31T15:15:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-31T15:15:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-07-31T15:15:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-07-31T15:15:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-07-31T15:15:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-07-31T15:15:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-07-31T15:15:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-07-31T15:15:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-07-31T15:15:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-07-31T15:15:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-07-31T15:15:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-07-31T15:15:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-07-31T15:15:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.172 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.172 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.172 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.172 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.172 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.172 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.172 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.172 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.173 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.173 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.173 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.173 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.173 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.173 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.173 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-07-31T15:15:25.174 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-07-31T15:15:25.174 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.174 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.174 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.174 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.174 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.174 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.175 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.175 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.175 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.175 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.175 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.175 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-07-31T15:15:25.175 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-07-31T15:15:25.176 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.176 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.176 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.176 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.176 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.176 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.176 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.176 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.177 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.177 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.177 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.177 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.177 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.177 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.177 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.178 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.178 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.178 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.178 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.178 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.178 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.178 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.178 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.179 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.179 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-07-31T15:15:25.179 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-07-31T15:15:25.179 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.179 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.179 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.179 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-31T15:15:25.179 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.180 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.180 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.180 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.180 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.180 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.180 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.180 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.180 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.181 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.181 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-07-31T15:15:25.181 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.181 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.181 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.181 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.181 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.181 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.182 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.182 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.182 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.182 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.182 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.182 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.182 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.183 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.183 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.183 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.183 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.183 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.183 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.183 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.183 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.183 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.184 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.184 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.184 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.184 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.184 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.184 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.184 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.184 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.185 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.185 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.185 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.185 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.185 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.185 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.185 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.185 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.186 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.186 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.186 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.186 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.186 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.186 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.186 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.186 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.187 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.187 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.187 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.187 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.187 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.187 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.187 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.187 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.188 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.188 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.188 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.188 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.188 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.188 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.188 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.188 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.189 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.189 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.189 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.189 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.189 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.189 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.189 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.189 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.189 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.190 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.190 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.190 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.190 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.190 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.190 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.190 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.190 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.190 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.191 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.191 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.191 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.191 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.191 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.191 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.191 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.191 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.191 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.192 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-07-31T15:15:25.192 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.192 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.192 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.192 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.192 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.192 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.192 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.193 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.193 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.193 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.193 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.193 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.193 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.193 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.193 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.193 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.193 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.194 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.194 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.194 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.194 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.194 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.194 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.194 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.195 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.195 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.195 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.195 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.195 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.195 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.195 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.196 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.196 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.196 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.196 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.196 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.196 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.196 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.197 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.197 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.197 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.197 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.197 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.197 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.197 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.197 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-07-31T15:15:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-07-31T15:15:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-07-31T15:15:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-31T15:15:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: "periodic_async_work": { 2024-07-31T15:15:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "periodic_async_work", 2024-07-31T15:15:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Periodically check for async work", 2024-07-31T15:15:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-07-31T15:15:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-07-31T15:15:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-31T15:15:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-07-31T15:15:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-07-31T15:15:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-31T15:15:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-07-31T15:15:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-07-31T15:15:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-07-31T15:15:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-07-31T15:15:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-07-31T15:15:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-07-31T15:15:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.216 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.216 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.216 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.216 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.216 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.216 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.216 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.217 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.217 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.217 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.217 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.217 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.217 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.217 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.217 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.218 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.218 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.218 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.218 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.218 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.218 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.218 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.218 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.219 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.219 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.219 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.219 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.219 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.219 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.219 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.220 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.220 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.220 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.220 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.220 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.220 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.220 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.221 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.221 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.221 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.221 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.221 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.221 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.221 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.221 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.221 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.222 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.222 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.222 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.222 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.222 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.222 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.222 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.223 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.223 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.223 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.223 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.223 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.223 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.223 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.224 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.224 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.224 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.224 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.224 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.224 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.224 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.224 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.224 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.225 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.225 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.225 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.225 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.225 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.225 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.225 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.225 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.226 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.226 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.226 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.226 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.226 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.226 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-07-31T15:15:25.227 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-07-31T15:15:25.227 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.227 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.227 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.227 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.227 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.227 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.227 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.228 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.228 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.228 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.228 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.228 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-07-31T15:15:25.228 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-07-31T15:15:25.229 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.229 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.229 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.229 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-07-31T15:15:25.229 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.229 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.229 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.229 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.230 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.230 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.230 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.230 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-07-31T15:15:25.230 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-07-31T15:15:25.230 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.230 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.231 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.231 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-31T15:15:25.231 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.231 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.231 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.231 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.231 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.231 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.232 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.232 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.232 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.232 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.232 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-31T15:15:25.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.232 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.232 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 24146, 2024-07-31T15:15:25.233 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "y", 2024-07-31T15:15:25.233 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_features": 4540701547738038271, 2024-07-31T15:15:25.233 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-07-31T15:15:25.233 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.233 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-07-31T15:15:25.233 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.233 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.234 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.234 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:25.234 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:25.234 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.234 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.234 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.234 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:25.235 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.235 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.235 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.235 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-07-31T15:15:25.235 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.235 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.235 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.236 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.236 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.236 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.236 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.236 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.236 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.236 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.236 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.237 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.237 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.237 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.237 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.237 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.237 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.237 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.238 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.238 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.238 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.238 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.238 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.238 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.238 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.239 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.239 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.239 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.239 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.239 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.239 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.239 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.239 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.240 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.240 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.240 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.240 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.240 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.240 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.240 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.241 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.241 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.241 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.241 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.241 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.241 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.241 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.242 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.242 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.242 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.242 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.242 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.242 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.242 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.242 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.243 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.243 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.243 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.243 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.243 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.243 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.243 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.243 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.244 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.244 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.244 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.244 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.244 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.244 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.244 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.245 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.245 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-07-31T15:15:25.245 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-07-31T15:15:25.245 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.245 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.245 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.245 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.245 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.246 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.246 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.246 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to", 2024-07-31T15:15:25.246 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.246 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.246 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.247 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-07-31T15:15:25.247 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-07-31T15:15:25.247 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.247 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.247 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.247 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-07-31T15:15:25.247 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.247 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.248 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.248 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-07-31T15:15:25.248 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.248 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.248 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.248 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-07-31T15:15:25.248 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-07-31T15:15:25.249 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.249 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.249 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.249 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.249 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.249 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.249 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.249 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-07-31T15:15:25.250 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.250 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.250 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.250 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-07-31T15:15:25.250 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-07-31T15:15:25.250 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.250 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.251 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.251 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.251 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.251 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.251 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.251 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-07-31T15:15:25.251 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.251 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.251 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.252 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-07-31T15:15:25.252 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-07-31T15:15:25.252 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.252 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.252 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.252 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-07-31T15:15:25.253 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.253 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.253 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.253 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-07-31T15:15:25.253 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.253 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.253 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.254 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-07-31T15:15:25.254 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-07-31T15:15:25.254 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.254 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.254 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.254 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.254 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.255 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.255 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.255 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-07-31T15:15:25.255 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.255 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.255 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.255 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-07-31T15:15:25.256 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-07-31T15:15:25.256 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.256 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.256 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.256 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.256 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.256 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.256 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.256 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-31T15:15:25.257 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.257 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.257 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.257 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-07-31T15:15:25.257 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-07-31T15:15:25.257 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.257 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.258 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.258 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.258 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.258 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.258 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.258 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-07-31T15:15:25.258 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.259 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.259 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.259 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.259 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.259 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.259 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.259 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.259 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.260 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.260 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.260 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.260 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.260 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.260 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.260 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.261 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.261 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.261 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.261 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-07-31T15:15:25.261 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.261 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.261 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.262 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-07-31T15:15:25.262 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-07-31T15:15:25.262 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.262 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.262 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.262 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.262 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.262 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.263 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.263 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-07-31T15:15:25.263 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.263 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.263 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.263 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-07-31T15:15:25.263 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-07-31T15:15:25.264 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.264 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.264 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.264 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-07-31T15:15:25.264 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.264 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.264 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.264 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-07-31T15:15:25.264 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-31T15:15:25.265 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.265 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.265 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-07-31T15:15:25.265 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-07-31T15:15:25.265 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:25.265 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.265 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.266 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.266 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-07-31T15:15:25.266 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-07-31T15:15:25.266 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.266 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-31T15:15:25.266 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-31T15:15:25.266 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.267 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.267 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.267 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-07-31T15:15:25.267 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-07-31T15:15:25.267 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:25.267 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.267 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.267 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-07-31T15:15:25.268 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-31T15:15:25.268 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-07-31T15:15:25.268 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.268 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-31T15:15:25.268 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.268 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.268 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.269 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-07-31T15:15:25.269 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-07-31T15:15:25.269 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.269 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.269 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.269 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-07-31T15:15:25.269 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.270 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.270 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.270 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-31T15:15:25.270 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-31T15:15:25.270 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.270 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.270 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.271 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-07-31T15:15:25.271 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-07-31T15:15:25.271 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:25.271 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.271 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.271 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-07-31T15:15:25.271 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-07-31T15:15:25.271 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-07-31T15:15:25.271 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.272 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-07-31T15:15:25.272 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-31T15:15:25.272 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.272 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.272 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-07-31T15:15:25.272 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-07-31T15:15:25.272 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.273 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.273 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.273 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-07-31T15:15:25.273 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.273 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.273 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.273 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-07-31T15:15:25.273 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-31T15:15:25.274 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.274 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.274 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-07-31T15:15:25.274 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-07-31T15:15:25.274 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:25.274 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.275 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.275 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.275 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-07-31T15:15:25.275 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-07-31T15:15:25.275 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.275 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-31T15:15:25.275 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-31T15:15:25.275 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.276 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.276 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.276 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.276 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.276 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.276 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.276 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.277 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.277 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.277 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.277 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.277 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.277 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.277 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.277 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.278 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.278 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.278 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.278 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.278 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.278 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.279 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.279 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.279 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.279 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.279 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.279 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.279 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.279 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.280 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.280 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.280 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.280 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.280 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.280 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.280 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.281 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.281 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.281 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.281 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.281 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.281 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.281 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.281 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.282 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.282 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.282 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.282 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.282 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.282 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.282 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.282 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.283 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.283 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.283 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.283 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.283 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.283 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.283 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.283 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.284 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.284 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.284 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.284 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.284 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.284 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.284 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.285 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.285 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-07-31T15:15:25.285 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-07-31T15:15:25.285 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:25.285 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.285 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.285 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.286 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.286 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.286 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.286 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-31T15:15:25.286 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.286 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.286 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.287 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-07-31T15:15:25.287 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-07-31T15:15:25.287 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.287 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.287 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.287 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-07-31T15:15:25.287 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.287 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.288 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.288 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-07-31T15:15:25.288 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-07-31T15:15:25.288 INFO:teuthology.orchestra.run.smithi067.stdout: "read", 2024-07-31T15:15:25.288 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap", 2024-07-31T15:15:25.288 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap-read" 2024-07-31T15:15:25.288 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.288 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-07-31T15:15:25.289 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.289 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.289 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.289 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-07-31T15:15:25.289 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-07-31T15:15:25.289 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.289 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.289 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.290 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.290 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.290 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.290 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.290 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-31T15:15:25.290 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.290 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.290 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.291 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:25.291 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:25.291 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.291 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.291 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.291 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:25.291 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.292 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.292 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.292 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-31T15:15:25.292 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.292 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.292 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.292 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.292 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.293 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.293 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.293 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.293 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.293 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.293 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.293 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.293 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.293 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.294 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.294 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.294 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.294 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-07-31T15:15:25.294 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-07-31T15:15:25.294 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.294 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.295 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.295 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:25.295 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-31T15:15:25.295 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.295 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.295 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-31T15:15:25.295 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-31T15:15:25.295 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.296 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.296 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-07-31T15:15:25.296 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-07-31T15:15:25.296 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:25.296 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.296 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.297 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:25.297 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.297 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.297 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.297 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-31T15:15:25.297 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.297 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.297 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.298 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.298 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.298 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.298 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-07-31T15:15:25.298 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.298 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.298 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.299 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-07-31T15:15:25.299 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-07-31T15:15:25.299 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:25.299 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.299 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.299 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-07-31T15:15:25.299 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.299 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.300 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.300 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-31T15:15:25.300 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.300 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.300 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.300 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-07-31T15:15:25.301 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-07-31T15:15:25.301 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.301 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.301 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.301 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-07-31T15:15:25.301 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.301 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.301 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.301 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-31T15:15:25.302 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.302 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.302 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.302 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-07-31T15:15:25.302 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-07-31T15:15:25.302 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.302 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.303 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.303 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-07-31T15:15:25.303 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.303 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.303 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.303 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-31T15:15:25.303 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.304 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.304 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.304 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-07-31T15:15:25.304 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-07-31T15:15:25.304 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.304 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.304 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.305 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.305 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.305 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.305 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.305 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-31T15:15:25.305 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-31T15:15:25.305 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.305 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.306 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-07-31T15:15:25.306 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-07-31T15:15:25.306 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.306 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.306 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.306 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:25.306 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.307 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.307 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.307 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-31T15:15:25.307 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.307 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.307 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.307 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.307 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-07-31T15:15:25.308 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-07-31T15:15:25.308 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:25.308 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.308 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.308 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-07-31T15:15:25.308 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.308 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.308 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.309 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-31T15:15:25.309 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.309 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.309 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.309 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm_log_destination": { 2024-07-31T15:15:25.309 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm_log_destination", 2024-07-31T15:15:25.309 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.310 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.310 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.310 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.310 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.310 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.310 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.310 INFO:teuthology.orchestra.run.smithi067.stdout: "file", 2024-07-31T15:15:25.310 INFO:teuthology.orchestra.run.smithi067.stdout: "file,syslog", 2024-07-31T15:15:25.311 INFO:teuthology.orchestra.run.smithi067.stdout: "syslog" 2024-07-31T15:15:25.311 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.311 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-31T15:15:25.311 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.311 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.311 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.311 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.312 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-07-31T15:15:25.312 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-07-31T15:15:25.312 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.312 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.312 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.312 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.312 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.313 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.313 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.313 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-31T15:15:25.313 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.313 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.313 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.313 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-07-31T15:15:25.314 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-07-31T15:15:25.314 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.314 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.314 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.314 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.314 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.314 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.314 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.315 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-31T15:15:25.315 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.315 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.315 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.315 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.315 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-07-31T15:15:25.315 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-07-31T15:15:25.315 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.316 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.316 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.316 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.316 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.316 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.316 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.316 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-31T15:15:25.316 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.317 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.317 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.317 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-07-31T15:15:25.317 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-07-31T15:15:25.317 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.317 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.318 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.318 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-07-31T15:15:25.318 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.318 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.318 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.318 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-31T15:15:25.318 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.319 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.319 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.319 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-07-31T15:15:25.319 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-07-31T15:15:25.319 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.319 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.319 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.320 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-31T15:15:25.320 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.320 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.320 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.320 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-07-31T15:15:25.320 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.320 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.320 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.321 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-07-31T15:15:25.321 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-07-31T15:15:25.321 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.321 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.321 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.321 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-31T15:15:25.321 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.321 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.321 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.322 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-07-31T15:15:25.322 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.322 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.322 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.322 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-07-31T15:15:25.322 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-07-31T15:15:25.322 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.323 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.323 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.323 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-07-31T15:15:25.323 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.323 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.323 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.323 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-31T15:15:25.323 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.324 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.324 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.324 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-07-31T15:15:25.324 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-07-31T15:15:25.324 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.324 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.324 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.325 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-31T15:15:25.325 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.325 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.325 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.325 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-07-31T15:15:25.325 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.325 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.325 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.326 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-07-31T15:15:25.326 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-07-31T15:15:25.326 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.326 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.326 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.326 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-31T15:15:25.326 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.326 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.327 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.327 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-07-31T15:15:25.327 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.327 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.327 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.327 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-07-31T15:15:25.328 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-07-31T15:15:25.328 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.328 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.328 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.328 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-31T15:15:25.328 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.328 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.328 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.328 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-07-31T15:15:25.329 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.329 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.329 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.329 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-07-31T15:15:25.329 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-07-31T15:15:25.329 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.329 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.330 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.330 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-31T15:15:25.330 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.330 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.330 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.330 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-07-31T15:15:25.330 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.330 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.331 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.331 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-07-31T15:15:25.331 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-07-31T15:15:25.331 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.331 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.331 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.331 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-31T15:15:25.331 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.332 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.332 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.332 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-07-31T15:15:25.332 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.332 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.332 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.333 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-07-31T15:15:25.333 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-07-31T15:15:25.333 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.333 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.333 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.333 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-31T15:15:25.333 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.333 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.334 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.334 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-07-31T15:15:25.334 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.334 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.334 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.334 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nginx": { 2024-07-31T15:15:25.334 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nginx", 2024-07-31T15:15:25.335 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.335 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.335 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.335 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-07-31T15:15:25.335 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.335 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.335 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.335 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nginx container image", 2024-07-31T15:15:25.336 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.336 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.336 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.336 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-07-31T15:15:25.336 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-07-31T15:15:25.336 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.337 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.337 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.337 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-07-31T15:15:25.337 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.337 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.337 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.337 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-31T15:15:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-07-31T15:15:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-07-31T15:15:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-31T15:15:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-07-31T15:15:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-07-31T15:15:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-07-31T15:15:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-07-31T15:15:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-31T15:15:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-07-31T15:15:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-07-31T15:15:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-31T15:15:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-07-31T15:15:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba": { 2024-07-31T15:15:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba", 2024-07-31T15:15:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-31T15:15:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB container image", 2024-07-31T15:15:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-07-31T15:15:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-07-31T15:15:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-31T15:15:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-07-31T15:15:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-07-31T15:15:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-07-31T15:15:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-07-31T15:15:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-07-31T15:15:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-07-31T15:15:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-31T15:15:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.351 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.351 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.351 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-07-31T15:15:25.351 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-07-31T15:15:25.351 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.351 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.352 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.352 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-07-31T15:15:25.352 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.352 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.352 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.352 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-31T15:15:25.352 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.352 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.353 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.353 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-07-31T15:15:25.353 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-07-31T15:15:25.353 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.353 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.353 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.353 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-07-31T15:15:25.354 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.354 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.354 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.354 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-31T15:15:25.354 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.354 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.354 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.355 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-07-31T15:15:25.355 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-07-31T15:15:25.355 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.355 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.355 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.355 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-07-31T15:15:25.356 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.356 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.356 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.356 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-07-31T15:15:25.356 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.356 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.356 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.357 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-07-31T15:15:25.357 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-07-31T15:15:25.357 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.357 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.357 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.357 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.357 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.358 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.358 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.358 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-31T15:15:25.358 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.358 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.358 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.359 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-07-31T15:15:25.359 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-07-31T15:15:25.359 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.359 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.359 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.359 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:25.359 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.359 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.359 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.360 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-07-31T15:15:25.360 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.360 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.360 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.360 INFO:teuthology.orchestra.run.smithi067.stdout: "grafana_dashboards_path": { 2024-07-31T15:15:25.360 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "grafana_dashboards_path", 2024-07-31T15:15:25.361 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.361 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.361 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.361 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-31T15:15:25.361 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.361 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.361 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.361 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-31T15:15:25.361 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.362 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.362 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.362 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-07-31T15:15:25.362 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-07-31T15:15:25.362 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.362 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.363 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.363 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:25.363 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.363 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.363 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.363 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-07-31T15:15:25.363 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.363 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.363 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.364 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-07-31T15:15:25.364 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-07-31T15:15:25.364 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.364 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.364 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.364 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.364 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.365 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.365 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.365 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-31T15:15:25.365 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.365 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.365 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.365 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-07-31T15:15:25.366 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-07-31T15:15:25.366 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.366 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.366 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.366 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.366 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.366 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.367 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.367 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-31T15:15:25.367 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.367 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.367 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.367 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.367 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.368 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.368 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.368 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.368 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.368 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.368 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.368 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.368 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.368 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.369 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.369 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.369 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.369 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.369 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.369 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.369 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.370 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.370 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.370 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-07-31T15:15:25.370 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-07-31T15:15:25.370 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.370 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.371 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.371 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.371 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.371 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.371 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.371 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-31T15:15:25.371 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.371 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.371 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.372 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.372 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.372 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.372 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.372 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.372 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.373 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.373 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.373 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.373 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-31T15:15:25.373 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.373 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.373 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.374 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.374 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.374 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.374 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.374 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.374 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.374 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.374 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.375 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.375 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.375 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.375 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.375 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.375 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.375 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.375 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.376 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.376 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.376 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.376 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.376 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.376 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.376 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.377 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.377 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.377 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.377 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.377 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.377 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.377 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.377 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.378 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.378 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.378 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-31T15:15:25.378 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-31T15:15:25.378 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.378 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.379 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.379 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.379 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.379 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.379 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.379 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-31T15:15:25.379 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.380 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.380 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.380 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-31T15:15:25.380 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-31T15:15:25.380 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.380 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.380 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.381 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-07-31T15:15:25.381 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.381 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.381 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.381 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-31T15:15:25.381 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.381 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.381 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.382 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-07-31T15:15:25.382 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-07-31T15:15:25.382 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.382 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.382 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.382 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:25.383 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.383 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.383 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.383 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-07-31T15:15:25.383 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.383 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.383 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.384 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-07-31T15:15:25.384 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-07-31T15:15:25.384 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.384 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.384 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.384 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:25.384 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.385 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.385 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.385 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-31T15:15:25.385 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.385 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.385 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.385 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-07-31T15:15:25.386 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-07-31T15:15:25.386 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.386 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.386 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.386 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.386 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.386 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.387 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.387 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-07-31T15:15:25.387 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.387 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.387 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.387 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-07-31T15:15:25.387 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-07-31T15:15:25.388 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.388 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.388 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.388 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-07-31T15:15:25.388 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.388 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.388 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.388 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-07-31T15:15:25.388 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-07-31T15:15:25.389 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.389 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-07-31T15:15:25.389 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.389 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.389 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.389 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-07-31T15:15:25.389 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-07-31T15:15:25.390 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.390 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.390 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.390 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-07-31T15:15:25.390 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.390 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.390 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.390 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-31T15:15:25.391 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.391 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.391 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.391 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-07-31T15:15:25.391 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-07-31T15:15:25.391 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.391 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.391 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.392 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-31T15:15:25.392 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.392 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.392 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.392 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-31T15:15:25.392 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.392 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.392 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.393 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-07-31T15:15:25.393 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-07-31T15:15:25.393 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.393 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.393 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.393 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.393 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.393 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.394 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.394 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-31T15:15:25.394 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.394 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.394 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.394 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-07-31T15:15:25.395 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-07-31T15:15:25.395 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.395 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.395 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.395 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.395 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.395 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.396 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.396 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-31T15:15:25.396 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.396 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.396 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.396 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-07-31T15:15:25.396 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-07-31T15:15:25.397 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.397 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.397 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.397 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.397 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.397 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.397 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.397 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-31T15:15:25.398 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.398 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.398 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.398 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-07-31T15:15:25.398 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-07-31T15:15:25.398 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.399 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.399 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.399 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.399 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.399 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.399 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.399 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-31T15:15:25.399 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.400 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.400 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.400 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-07-31T15:15:25.400 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-07-31T15:15:25.400 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.400 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.400 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.401 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.401 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.401 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.401 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.401 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-31T15:15:25.401 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.401 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.402 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.402 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-07-31T15:15:25.402 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-07-31T15:15:25.402 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.402 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.402 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.402 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-07-31T15:15:25.403 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.403 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.403 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.403 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-07-31T15:15:25.403 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.403 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.403 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.404 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.404 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.404 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.404 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.404 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.404 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.404 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.405 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.405 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.405 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.405 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.405 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.405 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.405 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-07-31T15:15:25.406 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-07-31T15:15:25.406 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.406 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.406 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.406 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.406 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.406 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.407 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.407 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-31T15:15:25.407 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.407 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.407 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.407 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-07-31T15:15:25.408 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-07-31T15:15:25.408 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.408 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.408 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.408 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.408 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.408 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.408 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.408 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-31T15:15:25.409 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.409 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.409 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.409 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-07-31T15:15:25.409 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-07-31T15:15:25.409 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.409 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.410 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.410 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.410 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.410 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.410 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.410 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-31T15:15:25.410 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.411 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.411 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.411 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-07-31T15:15:25.411 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-07-31T15:15:25.411 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.411 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.411 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.412 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.412 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.412 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.412 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.412 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-07-31T15:15:25.412 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.412 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.413 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.413 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-07-31T15:15:25.413 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-07-31T15:15:25.413 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.413 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.413 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.413 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.414 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.414 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.414 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.414 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-31T15:15:25.414 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.414 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.414 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.415 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-07-31T15:15:25.415 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-07-31T15:15:25.415 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.415 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.415 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.415 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.415 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.415 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.416 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.416 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-31T15:15:25.416 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.416 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.416 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.416 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.416 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.417 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-07-31T15:15:25.417 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.417 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.417 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.417 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.417 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.417 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.418 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.418 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.418 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.418 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.418 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.418 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.418 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.418 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.419 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.419 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.419 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.419 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.419 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.419 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.419 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.419 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.420 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.420 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.420 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.420 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.420 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.420 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.421 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.421 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.421 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.421 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.421 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.421 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.421 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.421 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.422 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.422 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.422 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.422 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.422 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.422 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.422 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.423 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.423 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.423 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.423 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.423 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.423 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.423 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.424 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.424 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.424 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.424 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.424 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.424 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.424 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.425 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.425 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.425 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.425 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.425 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.425 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.426 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.426 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.426 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.426 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.426 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.426 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.427 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-07-31T15:15:25.427 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-07-31T15:15:25.427 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.427 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.427 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.427 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-07-31T15:15:25.427 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.428 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.428 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.428 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-31T15:15:25.428 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.428 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.428 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.428 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.429 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.429 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.429 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.429 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.429 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.429 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.429 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.429 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.430 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.430 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.430 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.430 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.430 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-07-31T15:15:25.430 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-07-31T15:15:25.431 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.431 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.431 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.431 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-07-31T15:15:25.431 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.431 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.431 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.431 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-31T15:15:25.432 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.432 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.432 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.432 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.432 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.432 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-07-31T15:15:25.433 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.433 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.433 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.433 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-31T15:15:25.433 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-31T15:15:25.433 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.433 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.433 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.434 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:25.434 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.434 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.434 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.434 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.434 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.434 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-07-31T15:15:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-31T15:15:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.437 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-31T15:15:25.437 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-31T15:15:25.437 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.437 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.437 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.437 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.437 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.437 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.438 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.438 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.438 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.438 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.438 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.438 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-07-31T15:15:25.438 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-07-31T15:15:25.439 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.439 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.439 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.439 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.439 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.439 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.439 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.439 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.440 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.440 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.440 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.440 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-31T15:15:25.440 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-31T15:15:25.440 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.440 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.440 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.441 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.441 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.441 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.441 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.441 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.441 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.441 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.442 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.442 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-07-31T15:15:25.442 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-31T15:15:25.442 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.442 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.442 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.442 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.443 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.443 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.443 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.443 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.443 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.443 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.443 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.444 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-31T15:15:25.444 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-31T15:15:25.444 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.444 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.444 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.444 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.444 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.444 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.444 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.445 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.445 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.445 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.445 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.445 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-31T15:15:25.445 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-31T15:15:25.445 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.445 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.446 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.446 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.446 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.446 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.446 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.446 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.446 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.447 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.447 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.447 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-31T15:15:25.447 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-31T15:15:25.447 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.447 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.447 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.448 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.448 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.448 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.448 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.448 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.448 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.448 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.448 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.449 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-31T15:15:25.449 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-31T15:15:25.449 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.449 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.449 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.449 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.449 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.449 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.450 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.450 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.450 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.450 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.450 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.450 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-31T15:15:25.450 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-31T15:15:25.451 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.451 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.451 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.451 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.451 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.451 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.451 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.452 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.452 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.452 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.452 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.452 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-31T15:15:25.452 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-31T15:15:25.453 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.453 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.453 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.453 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.453 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.453 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.453 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.453 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.453 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.454 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.454 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.454 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-31T15:15:25.454 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-31T15:15:25.454 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.454 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.454 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.455 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.455 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.455 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.455 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.455 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.455 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.455 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.455 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.456 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-31T15:15:25.456 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-31T15:15:25.456 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.456 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.456 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.456 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.456 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.457 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.457 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.457 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.457 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.457 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.457 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.458 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-07-31T15:15:25.458 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-31T15:15:25.458 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.458 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.458 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.458 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-31T15:15:25.458 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.458 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.459 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.459 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.459 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.459 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.459 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.459 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-31T15:15:25.459 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-31T15:15:25.459 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.460 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.460 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.460 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.460 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.460 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.460 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.460 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.460 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.460 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.461 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.461 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-07-31T15:15:25.461 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-07-31T15:15:25.461 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.461 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.461 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.461 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.462 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.462 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.462 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.462 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.462 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.462 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.462 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.463 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-07-31T15:15:25.463 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-31T15:15:25.463 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.463 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.463 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.463 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-31T15:15:25.463 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.464 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.464 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.464 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.464 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.464 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.464 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.464 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-31T15:15:25.464 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-31T15:15:25.465 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.465 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.465 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.465 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.465 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.465 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.465 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.466 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.466 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.466 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.466 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.466 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-31T15:15:25.466 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-31T15:15:25.467 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.467 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.467 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.467 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.467 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.467 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.467 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.467 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.468 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.468 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.468 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.468 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-31T15:15:25.468 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-31T15:15:25.468 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.468 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.469 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.469 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.469 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.469 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.469 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.469 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.469 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.469 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.469 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.470 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-31T15:15:25.470 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-31T15:15:25.470 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.470 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.470 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.470 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.470 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.471 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.471 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.471 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.471 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.471 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.471 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.471 INFO:teuthology.orchestra.run.smithi067.stdout: "MANAGED_BY_CLUSTERS": { 2024-07-31T15:15:25.472 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-07-31T15:15:25.472 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.472 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.472 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.472 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "[]", 2024-07-31T15:15:25.472 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.472 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.472 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.473 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.473 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.473 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.473 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.473 INFO:teuthology.orchestra.run.smithi067.stdout: "MULTICLUSTER_CONFIG": { 2024-07-31T15:15:25.473 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MULTICLUSTER_CONFIG", 2024-07-31T15:15:25.473 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.474 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.474 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.474 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "{}", 2024-07-31T15:15:25.474 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.474 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.474 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.474 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.474 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.475 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.475 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.475 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-07-31T15:15:25.475 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-31T15:15:25.475 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.475 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.475 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.476 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.476 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.476 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.476 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.476 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.476 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.476 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.476 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.477 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-31T15:15:25.477 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-31T15:15:25.477 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.477 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.477 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.477 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.478 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.478 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.478 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.478 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.478 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.478 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.478 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.479 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-31T15:15:25.479 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-31T15:15:25.479 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.479 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.479 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.479 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.479 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.479 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.480 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.480 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.480 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.480 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.480 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.480 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-31T15:15:25.480 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-31T15:15:25.481 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.481 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.481 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.481 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.481 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.481 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.481 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.481 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.482 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.482 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.482 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.482 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-31T15:15:25.482 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-31T15:15:25.482 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.482 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.483 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.483 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.483 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.483 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.483 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.483 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.483 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.483 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.484 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.484 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-31T15:15:25.484 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-31T15:15:25.484 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.484 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.484 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.484 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.484 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.485 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.485 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.485 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.485 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.485 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.485 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.485 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-31T15:15:25.486 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-31T15:15:25.486 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.486 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.486 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.486 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.486 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.486 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.486 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.487 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.487 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.487 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.487 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.487 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-31T15:15:25.487 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-31T15:15:25.487 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.488 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.488 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.488 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.488 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.488 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.488 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.488 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.488 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.489 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.489 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.489 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-31T15:15:25.489 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-31T15:15:25.489 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.489 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.490 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.490 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.490 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.490 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.490 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.490 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.490 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.490 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.491 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.491 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-07-31T15:15:25.491 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-31T15:15:25.491 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.491 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.491 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.491 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.492 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.492 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.492 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.492 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.492 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.492 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.492 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.493 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-31T15:15:25.493 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-31T15:15:25.493 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.493 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.493 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.493 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-31T15:15:25.493 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.493 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.494 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.494 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.494 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.494 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.494 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.494 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-31T15:15:25.494 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-31T15:15:25.495 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.495 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.495 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.495 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:25.495 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.495 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.495 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.495 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.496 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.496 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.496 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.496 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-31T15:15:25.496 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-31T15:15:25.496 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.496 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.497 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.497 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-07-31T15:15:25.497 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.497 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.497 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.497 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.497 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.497 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.498 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.498 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-31T15:15:25.498 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-31T15:15:25.498 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.498 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.498 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.499 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-07-31T15:15:25.499 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.499 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.499 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.499 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.499 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.499 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.499 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.500 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-07-31T15:15:25.500 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-31T15:15:25.500 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.500 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.500 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.500 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.500 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.500 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.501 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.501 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.501 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.501 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.501 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.501 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-31T15:15:25.501 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-31T15:15:25.502 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.502 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.502 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.502 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-31T15:15:25.502 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.502 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.502 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.502 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.503 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.503 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.503 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.503 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-07-31T15:15:25.503 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-31T15:15:25.503 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.503 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.504 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.504 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.504 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.504 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.504 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.504 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.504 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.504 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.504 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.505 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-07-31T15:15:25.505 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-31T15:15:25.505 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.505 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.505 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.505 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.505 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.505 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.506 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.506 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.506 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.506 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.506 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.506 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-07-31T15:15:25.506 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-07-31T15:15:25.507 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.507 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.507 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.507 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.507 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.507 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.507 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.507 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.508 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.508 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.508 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.508 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-31T15:15:25.508 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-31T15:15:25.508 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.508 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.509 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.509 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.509 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.509 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.509 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.509 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.509 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.509 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.510 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.510 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-31T15:15:25.510 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-31T15:15:25.510 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.510 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.510 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.511 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:25.511 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.511 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.511 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.511 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.511 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.511 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.512 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.512 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-31T15:15:25.512 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-31T15:15:25.512 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.512 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.512 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.512 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:25.513 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.513 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.513 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.513 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.513 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.513 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.513 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.514 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-07-31T15:15:25.514 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-07-31T15:15:25.514 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.514 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.514 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.514 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.514 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.515 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.515 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.515 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.515 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.515 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.515 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.515 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-07-31T15:15:25.516 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-07-31T15:15:25.516 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.516 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.516 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.516 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.516 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.516 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.516 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.517 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.517 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.517 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.517 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-07-31T15:15:25.517 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-07-31T15:15:25.517 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.518 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.518 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.518 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.518 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.518 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.518 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.518 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-07-31T15:15:25.519 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.519 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.519 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.519 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-07-31T15:15:25.519 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-07-31T15:15:25.519 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.519 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.520 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.520 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-07-31T15:15:25.520 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.520 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.520 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.520 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.520 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.520 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.520 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.521 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-07-31T15:15:25.521 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-07-31T15:15:25.521 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.521 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.521 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.521 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.521 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.522 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.522 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.522 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.522 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.522 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.522 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.522 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.523 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.523 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.523 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.523 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.523 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.523 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.523 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.523 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.523 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.524 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.524 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.524 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.524 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.524 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.524 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.524 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.524 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.524 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.525 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.525 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.525 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.525 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.525 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.525 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.525 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.526 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.526 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.526 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.526 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.526 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.526 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.526 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.527 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.527 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.527 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.527 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.527 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.527 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.528 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.528 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.528 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.528 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.528 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.528 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.528 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.528 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.529 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.529 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.529 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.529 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.529 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.529 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.530 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.530 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.530 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.530 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.530 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.530 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.530 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.531 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.531 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.531 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.531 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.531 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.531 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.531 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-07-31T15:15:25.532 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-07-31T15:15:25.532 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.532 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.532 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.532 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.532 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.532 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.533 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.533 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-07-31T15:15:25.533 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.533 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.533 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.533 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-07-31T15:15:25.533 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-07-31T15:15:25.534 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.534 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.534 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.534 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.534 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.534 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.534 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.535 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.535 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.535 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.535 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.535 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-31T15:15:25.535 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-31T15:15:25.535 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.535 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.536 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.536 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-07-31T15:15:25.536 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.536 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.536 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.536 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.536 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.536 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.537 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.537 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-31T15:15:25.537 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-31T15:15:25.537 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.537 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.537 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.537 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-07-31T15:15:25.538 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.538 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.538 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.538 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.538 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.538 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.538 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.539 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.539 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.539 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.539 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.539 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.539 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.539 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.539 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.539 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.540 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.540 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.540 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.540 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.540 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-07-31T15:15:25.540 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-07-31T15:15:25.540 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.541 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.541 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.541 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.541 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.541 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.541 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.542 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.542 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.542 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.542 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.542 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-07-31T15:15:25.542 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-07-31T15:15:25.542 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.543 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.543 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.543 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-07-31T15:15:25.543 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.543 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.543 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.543 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.543 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.544 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.544 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.544 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-07-31T15:15:25.544 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-07-31T15:15:25.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.544 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.545 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.545 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-07-31T15:15:25.545 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.545 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.545 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.545 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.545 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-07-31T15:15:25.545 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.546 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.546 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.546 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.546 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.546 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-07-31T15:15:25.546 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-07-31T15:15:25.546 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.547 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.547 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.547 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-07-31T15:15:25.547 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-07-31T15:15:25.547 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-07-31T15:15:25.547 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.547 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.547 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.548 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.548 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.548 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-07-31T15:15:25.548 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-07-31T15:15:25.548 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.548 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.548 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.548 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.549 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.549 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.549 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.549 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.549 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.549 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.549 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.549 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.550 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.550 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-07-31T15:15:25.550 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.550 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.550 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.550 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-07-31T15:15:25.551 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-07-31T15:15:25.551 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.551 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.551 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.551 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.551 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.551 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.551 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.551 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-07-31T15:15:25.552 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.552 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.552 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.552 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.552 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.552 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.552 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.553 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.553 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.553 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.553 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.553 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.553 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.553 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.554 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.554 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.554 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.554 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.554 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.554 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.554 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.554 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.555 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.555 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.555 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.555 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.555 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.555 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.555 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.555 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.556 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.556 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.556 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.556 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.556 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.556 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.556 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.557 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.557 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.557 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.557 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.557 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.557 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.557 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.557 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.557 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.558 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.558 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.558 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.558 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.558 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.558 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.558 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.559 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.559 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.559 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.559 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.559 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.559 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.560 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.560 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.560 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.560 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.560 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.560 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.560 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.561 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.561 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.561 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-07-31T15:15:25.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-07-31T15:15:25.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.561 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.561 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.562 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-07-31T15:15:25.562 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.562 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.562 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.562 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-31T15:15:25.562 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.562 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.562 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.563 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-07-31T15:15:25.563 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-07-31T15:15:25.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.563 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.563 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.563 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-07-31T15:15:25.563 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.564 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.564 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.564 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-31T15:15:25.564 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.564 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.564 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.565 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-07-31T15:15:25.565 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-07-31T15:15:25.565 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.565 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.565 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.566 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-07-31T15:15:25.566 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.566 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.566 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.566 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-07-31T15:15:25.566 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.566 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.566 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.567 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-07-31T15:15:25.567 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-07-31T15:15:25.567 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.567 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.567 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.567 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-07-31T15:15:25.568 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.568 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.568 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.568 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-31T15:15:25.568 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.568 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.568 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.569 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-07-31T15:15:25.569 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-07-31T15:15:25.569 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.569 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.569 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.569 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.569 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.569 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.570 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.570 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-31T15:15:25.570 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.570 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.570 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.570 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:25.570 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:25.571 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.571 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.571 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.571 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:25.571 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.571 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.571 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.572 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-07-31T15:15:25.572 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.572 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.572 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.572 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.572 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.572 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.573 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.573 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.573 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.573 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.573 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.573 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.573 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.574 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.574 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.574 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.574 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-07-31T15:15:25.574 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-07-31T15:15:25.574 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.574 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.575 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.575 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-07-31T15:15:25.575 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.575 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.575 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.575 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-31T15:15:25.575 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.576 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.576 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.576 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.576 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.576 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.576 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-07-31T15:15:25.576 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.577 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.577 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.577 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.577 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.577 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.577 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.577 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.578 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.578 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.578 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.578 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.578 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.578 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.578 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.578 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.578 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.579 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.579 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.579 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.579 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.579 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.579 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.579 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-07-31T15:15:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-07-31T15:15:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-07-31T15:15:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-07-31T15:15:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-07-31T15:15:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-07-31T15:15:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-07-31T15:15:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-07-31T15:15:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-07-31T15:15:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-07-31T15:15:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-07-31T15:15:25.594 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.594 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.594 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.594 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-07-31T15:15:25.594 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.594 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.594 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.594 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-31T15:15:25.594 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-07-31T15:15:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-07-31T15:15:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-07-31T15:15:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-31T15:15:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.597 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-07-31T15:15:25.597 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-07-31T15:15:25.597 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.597 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.597 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.597 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.597 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.597 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.598 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.598 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-07-31T15:15:25.598 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.598 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.598 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.598 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-07-31T15:15:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-07-31T15:15:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-31T15:15:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.603 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.603 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.603 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.603 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.603 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.603 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.603 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.603 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.604 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.604 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.604 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.604 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.604 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.604 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.604 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.604 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-07-31T15:15:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-07-31T15:15:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-07-31T15:15:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-07-31T15:15:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-07-31T15:15:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.611 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.611 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.611 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-07-31T15:15:25.611 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.611 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.611 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.611 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-07-31T15:15:25.611 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.612 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.612 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.612 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.612 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.612 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.612 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.612 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.613 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.613 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.613 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.613 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.613 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.613 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.613 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.613 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.614 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-07-31T15:15:25.614 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-07-31T15:15:25.614 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.614 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.614 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.614 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-07-31T15:15:25.614 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.615 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.615 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.615 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-31T15:15:25.615 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.615 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.615 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.615 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-07-31T15:15:25.615 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-07-31T15:15:25.616 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.616 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.616 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.616 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:25.616 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-31T15:15:25.616 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-07-31T15:15:25.616 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.616 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-31T15:15:25.616 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.617 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.617 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.617 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-07-31T15:15:25.617 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-07-31T15:15:25.617 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.617 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.618 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.618 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.618 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.618 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.618 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.618 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-07-31T15:15:25.618 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.619 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.619 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.619 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-07-31T15:15:25.619 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-07-31T15:15:25.619 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.619 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.619 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.620 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-07-31T15:15:25.620 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.620 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.620 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.620 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-31T15:15:25.620 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.620 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.620 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.621 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.621 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.621 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.621 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-07-31T15:15:25.621 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.621 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.621 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.621 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.622 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.622 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.622 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.622 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.622 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.622 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.622 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.622 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.623 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.623 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.623 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.623 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.623 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.623 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.623 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.623 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.624 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.624 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.624 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.624 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.624 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.624 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.624 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.624 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.625 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.625 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.625 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.625 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.625 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.625 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.625 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.625 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.626 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.626 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.626 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.626 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.626 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.626 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.626 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.627 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.627 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.627 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.627 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.627 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.627 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.627 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.627 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.628 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.628 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.628 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.628 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.628 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.628 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.628 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.629 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.629 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.629 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.629 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.629 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.629 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.629 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.629 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.629 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.630 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.630 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.630 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.630 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.630 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.630 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.630 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.631 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.631 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.631 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.631 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.631 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.631 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.631 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.631 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.632 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.632 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.632 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.632 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.632 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-07-31T15:15:25.632 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.632 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.632 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.632 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.633 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.633 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.633 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.633 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.633 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.633 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.633 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.633 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.634 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.634 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.634 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.634 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.634 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.634 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.634 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.634 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.635 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.635 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.635 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.635 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.635 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.635 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.636 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.636 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.636 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.636 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.636 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.636 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.636 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.636 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.637 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.637 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.637 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.637 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.637 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.637 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.637 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.637 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.638 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.638 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.638 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.638 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.638 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.638 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.638 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.639 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.639 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.639 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.639 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.639 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.639 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.639 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.639 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.640 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.640 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.640 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.640 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.640 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.640 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.640 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.641 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.641 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.641 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.641 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.641 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.641 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.641 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.642 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.642 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.642 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.642 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.642 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.642 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.642 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.643 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.643 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.643 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.643 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.643 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.643 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.643 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.643 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.643 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.644 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-07-31T15:15:25.644 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.644 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.644 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.644 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-07-31T15:15:25.644 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-07-31T15:15:25.644 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.644 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.645 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.645 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-07-31T15:15:25.645 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.645 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.645 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.645 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-31T15:15:25.645 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.645 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.645 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.646 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-07-31T15:15:25.646 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-07-31T15:15:25.646 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.646 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.646 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.646 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:25.647 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-07-31T15:15:25.647 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.647 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.647 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-31T15:15:25.647 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.647 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.647 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.648 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.648 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.648 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.648 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.648 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.648 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.648 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.648 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.648 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.649 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.649 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.649 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.649 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.649 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.649 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.649 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.649 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.649 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.650 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.650 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.650 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.650 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.650 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.650 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.650 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.651 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.651 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.651 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.651 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.651 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.651 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.651 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.652 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.652 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.652 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.652 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.652 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.652 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.652 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.652 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.653 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.653 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.653 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.653 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.653 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.653 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.653 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.653 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.655 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.655 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.655 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.655 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.655 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.655 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.655 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.656 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.656 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.656 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.656 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.656 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.656 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.656 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.657 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.657 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.657 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.657 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.657 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.657 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.657 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.658 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.658 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.658 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.658 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.658 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.658 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.658 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.658 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.659 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-07-31T15:15:25.659 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.659 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.659 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.659 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-07-31T15:15:25.659 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-07-31T15:15:25.660 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.660 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.660 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.660 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-07-31T15:15:25.660 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.660 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.660 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.660 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-07-31T15:15:25.661 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-31T15:15:25.661 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.661 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.661 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.661 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.661 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.661 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.662 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.662 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.662 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.662 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.662 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.662 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.662 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.663 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.663 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.663 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.663 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.663 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.663 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.663 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.663 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.664 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.664 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.664 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.664 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.664 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.664 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.664 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.664 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.665 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.665 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.665 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.665 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.665 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.665 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.665 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.665 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.666 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.666 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.666 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.666 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.666 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.666 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.666 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.667 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.667 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.667 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.667 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.667 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.667 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.667 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.667 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.668 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.668 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.668 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.668 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.668 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.668 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.668 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.668 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.669 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.669 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.669 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.669 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.669 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.669 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.669 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.669 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.669 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.670 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.670 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.670 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.670 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-07-31T15:15:25.670 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-07-31T15:15:25.670 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.670 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.670 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.671 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.671 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.671 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.671 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.671 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-07-31T15:15:25.671 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-31T15:15:25.671 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.671 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.671 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.672 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-07-31T15:15:25.672 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-07-31T15:15:25.672 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.672 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.672 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.672 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-07-31T15:15:25.672 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.672 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.673 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.673 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-07-31T15:15:25.673 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.673 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.673 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.673 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.673 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-07-31T15:15:25.673 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-07-31T15:15:25.674 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.674 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.674 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.674 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-07-31T15:15:25.674 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.674 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.674 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.675 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-07-31T15:15:25.675 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.675 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.675 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.675 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.675 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-07-31T15:15:25.675 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-07-31T15:15:25.675 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.676 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.676 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.676 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.676 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.676 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.676 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.676 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-07-31T15:15:25.676 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.677 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.677 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.677 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.677 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.677 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.677 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.677 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.677 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.678 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.678 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.678 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.678 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.678 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.678 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.678 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.678 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.679 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.679 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-07-31T15:15:25.679 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-07-31T15:15:25.679 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.679 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.679 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.679 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-07-31T15:15:25.680 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.680 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.680 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.680 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-31T15:15:25.680 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-31T15:15:25.680 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.680 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.680 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.681 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.681 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.681 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.681 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-07-31T15:15:25.681 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.681 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.681 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.681 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.682 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.682 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.682 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.682 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.682 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.682 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.682 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.683 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.683 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.683 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.683 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.683 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.683 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.683 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.684 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.684 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.684 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.684 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.684 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.684 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.684 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.685 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.685 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.685 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.685 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.685 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.685 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.685 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.685 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.686 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.686 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.686 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.686 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.686 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.686 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.686 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.687 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.687 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.687 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.687 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.687 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.687 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.687 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.687 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.688 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.688 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.688 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.688 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.688 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.688 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.688 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.688 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.689 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.689 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.689 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.689 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.689 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.689 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.690 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.690 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.690 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.690 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.690 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.690 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.690 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.691 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.691 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.691 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.691 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.691 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.691 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.691 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.691 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.692 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.692 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.692 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.692 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.692 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.692 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.693 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.693 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-07-31T15:15:25.693 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.693 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.693 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.693 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.693 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.693 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.694 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.694 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.694 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.694 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.694 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.694 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.694 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.695 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.695 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.695 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.695 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.695 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.696 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.696 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.696 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.696 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.696 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.696 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.697 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.697 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.697 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.697 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.697 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.697 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.697 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.697 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.698 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.698 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.698 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.698 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.698 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.698 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.699 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.699 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.699 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.699 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.699 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.699 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.700 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.700 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.700 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.700 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.700 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.700 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.700 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.700 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.701 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.701 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.701 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.701 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.701 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.701 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.701 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.702 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.702 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.702 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.702 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.702 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.702 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.703 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.703 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.703 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.703 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.703 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.703 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.703 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.704 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.704 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.704 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.704 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.704 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.704 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.704 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.705 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.705 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.705 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.705 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.705 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.705 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.706 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.706 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-07-31T15:15:25.706 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.706 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.706 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.706 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.706 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.706 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.707 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.707 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.707 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.707 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.707 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.707 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.707 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.708 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.708 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.708 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.708 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.708 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.708 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.708 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.708 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.709 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.709 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.709 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.709 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.709 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.709 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.709 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.710 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.710 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.710 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.710 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.710 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.710 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.710 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.711 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.711 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.711 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.711 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.711 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.711 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.711 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.712 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.712 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.712 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.712 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.712 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.712 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.712 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.713 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.713 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.713 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.713 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.713 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.713 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.713 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.714 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.714 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.714 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.714 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.714 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.714 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.714 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.715 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.715 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.715 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.715 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.715 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.715 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.715 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.715 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.716 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.716 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.716 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.716 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.716 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.716 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.716 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.716 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.716 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.717 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.717 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.717 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.717 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.717 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.717 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.717 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-07-31T15:15:25.718 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.718 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.718 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.718 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-07-31T15:15:25.718 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-07-31T15:15:25.718 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.718 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.718 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.719 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.719 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.719 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.719 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.719 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-31T15:15:25.719 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.719 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.719 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.720 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.720 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.720 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.720 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.720 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.720 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.720 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.720 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.721 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.721 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.721 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.721 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.721 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.721 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.721 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.721 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.721 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.722 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.722 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.722 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.722 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.722 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.722 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.722 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.722 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.723 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.723 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.723 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.723 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.723 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.723 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.723 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.723 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.724 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.724 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.724 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.724 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.724 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.724 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.725 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.725 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.725 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.725 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.725 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.725 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.725 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.725 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.726 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.726 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.726 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.726 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.726 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.726 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.726 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.727 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.727 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.727 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.727 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.727 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.727 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.727 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.728 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.728 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.728 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.728 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.728 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.728 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.728 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-07-31T15:15:25.729 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-07-31T15:15:25.729 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.729 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.729 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.729 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.729 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.729 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.730 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.730 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-07-31T15:15:25.730 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-07-31T15:15:25.730 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-07-31T15:15:25.730 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.730 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-07-31T15:15:25.730 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.730 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.731 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.731 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.731 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.731 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.731 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.731 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.732 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.732 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.732 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.732 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.732 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.732 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.732 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.732 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.733 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.733 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.733 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.733 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-07-31T15:15:25.733 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.733 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.733 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.733 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.734 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.734 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.734 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.734 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.734 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.734 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.734 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.735 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.735 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.735 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.735 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.735 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.735 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.735 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.735 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.736 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.736 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.736 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.736 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.736 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.737 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.737 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.737 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.737 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.737 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.737 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.737 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.737 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.738 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.738 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.738 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.738 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.738 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.738 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.739 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.739 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.739 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.739 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.739 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.739 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.739 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.739 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.740 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.740 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.740 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.740 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.740 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.740 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.740 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.740 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.741 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.741 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.741 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.741 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.741 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.741 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.742 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.742 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.742 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.742 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.742 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.742 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.742 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.742 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.743 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.743 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.743 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.743 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.743 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.743 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.744 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.744 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.744 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.744 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.744 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.744 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.744 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.744 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.745 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.745 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.745 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.745 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.745 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-07-31T15:15:25.745 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.745 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.745 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.746 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.746 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.746 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.746 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.746 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.746 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.746 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.747 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.747 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.747 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.747 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.747 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.747 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.747 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.748 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.748 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.748 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.748 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.748 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.748 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.748 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.749 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.749 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.749 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.749 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.749 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.749 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.749 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.749 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.749 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.750 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.750 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.750 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.750 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.750 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.750 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.750 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.751 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.751 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.751 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.751 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.751 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.751 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.751 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.751 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.752 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.752 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.752 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.752 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.752 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.752 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.752 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.752 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.753 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.753 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.753 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.753 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.753 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.753 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.754 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.754 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.754 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.754 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.754 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.754 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.754 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.755 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.755 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.755 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.755 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.755 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.755 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.755 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.755 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.756 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.756 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.756 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.756 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.756 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.756 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.756 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.757 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.757 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-07-31T15:15:25.757 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.757 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.757 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.757 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.757 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.758 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.758 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.758 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.758 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.758 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.758 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.758 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.758 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.759 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.759 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.759 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.759 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.759 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.759 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.759 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.760 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.760 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.760 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.760 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.760 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.760 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.760 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.760 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.761 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.761 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.761 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.761 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.761 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.761 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.761 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.761 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.761 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.762 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.762 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.762 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.762 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.762 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.762 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.762 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.763 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.763 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.763 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.763 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.763 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.763 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.763 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.764 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.764 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.764 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.764 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.764 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.764 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.765 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.765 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.765 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.765 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.765 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.765 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.765 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.765 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.765 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.766 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.766 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.766 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.766 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.766 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:25.766 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:25.767 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.767 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.767 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.767 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:25.767 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.767 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.767 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.767 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.768 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.768 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.768 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.768 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.768 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.768 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.768 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.769 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.769 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.769 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.769 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.769 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.769 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.769 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.770 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.770 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.770 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-07-31T15:15:25.770 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-07-31T15:15:25.770 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:25.770 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.770 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.771 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-07-31T15:15:25.771 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-07-31T15:15:25.771 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.771 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.771 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-07-31T15:15:25.771 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-31T15:15:25.771 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.771 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.772 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.772 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.772 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.772 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-07-31T15:15:25.772 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.772 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.772 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.773 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-07-31T15:15:25.773 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-07-31T15:15:25.773 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.773 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.773 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.773 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.773 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.773 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.774 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.774 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-31T15:15:25.774 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.774 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.774 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.774 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-07-31T15:15:25.775 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-07-31T15:15:25.775 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.775 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.775 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.775 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.775 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.775 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.775 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.776 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.776 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.776 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.776 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.776 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.776 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.777 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.777 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.777 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.777 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.777 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.777 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.777 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.778 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.778 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.778 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.778 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.778 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.778 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.778 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.778 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.779 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.779 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.779 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.779 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.779 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.779 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.780 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.780 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.780 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.780 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.780 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.780 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.780 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.780 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.781 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.781 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.781 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.781 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.781 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.781 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.781 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.782 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.782 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.782 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.782 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.782 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.782 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.782 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.783 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.783 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.783 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.783 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.783 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.783 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.783 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.783 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.784 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.784 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.784 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.784 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.784 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.784 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.784 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.785 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.785 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.785 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.785 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.785 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.785 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.785 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-07-31T15:15:25.786 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-07-31T15:15:25.786 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.786 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.786 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.786 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-07-31T15:15:25.786 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.786 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.787 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.787 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-07-31T15:15:25.787 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.787 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.787 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.787 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:25.787 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:25.787 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.788 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.788 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.788 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:25.788 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.788 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.788 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.788 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-07-31T15:15:25.788 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.789 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.789 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.789 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.789 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.789 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.789 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.789 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.790 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.790 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.790 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.790 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.790 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.790 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.790 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.791 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.791 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.791 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.791 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-07-31T15:15:25.791 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.791 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.791 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.792 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-07-31T15:15:25.792 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-07-31T15:15:25.792 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.792 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.792 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.792 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.792 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.792 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.792 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.793 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.793 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.793 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.793 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.793 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-07-31T15:15:25.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-07-31T15:15:25.793 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.794 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.794 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.794 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.794 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.794 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.794 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.794 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-31T15:15:25.794 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-31T15:15:25.795 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.795 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.795 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.795 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.795 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.795 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.795 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.796 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.796 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.796 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.796 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.796 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.796 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.796 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.796 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.797 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.797 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.797 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.797 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.797 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.797 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.797 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.797 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.798 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.798 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.798 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.798 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.798 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.798 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.798 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.799 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.799 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.799 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.799 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.799 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.799 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.799 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.800 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.800 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.800 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.800 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.800 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.800 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.800 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.800 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.801 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.801 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.801 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.801 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.801 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.801 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.801 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.801 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.802 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.802 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.802 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.802 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.802 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.802 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.802 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.803 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.803 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.803 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.803 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.803 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.803 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.803 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.803 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.804 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-07-31T15:15:25.804 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-07-31T15:15:25.804 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.804 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.804 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.804 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.804 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.804 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.805 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.805 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.805 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.805 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.805 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.805 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-31T15:15:25.805 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-31T15:15:25.806 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.806 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.806 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.806 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-07-31T15:15:25.806 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.806 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.806 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.806 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.807 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.807 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.807 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.807 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-07-31T15:15:25.807 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-07-31T15:15:25.807 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:25.807 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.808 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.808 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-07-31T15:15:25.808 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.808 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.808 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.808 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.808 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.808 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.809 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.809 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-31T15:15:25.809 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-31T15:15:25.809 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.809 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.809 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.809 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-07-31T15:15:25.810 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.810 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.810 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.810 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-31T15:15:25.810 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.810 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.810 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.811 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-31T15:15:25.811 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-31T15:15:25.811 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.811 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.811 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.811 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-07-31T15:15:25.811 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.811 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.812 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.812 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-31T15:15:25.812 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.812 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.812 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.812 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.812 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.813 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.813 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.813 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.813 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.813 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.813 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.813 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.813 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.814 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.814 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.814 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.814 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-07-31T15:15:25.814 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-07-31T15:15:25.814 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-07-31T15:15:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.815 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-07-31T15:15:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-07-31T15:15:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.816 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-07-31T15:15:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-07-31T15:15:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-07-31T15:15:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.817 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-07-31T15:15:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-07-31T15:15:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-07-31T15:15:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-07-31T15:15:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-07-31T15:15:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.819 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-07-31T15:15:25.820 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.821 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.822 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.824 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.825 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.826 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.827 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.827 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.827 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.827 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.827 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.827 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.827 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.828 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.828 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.828 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.828 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.828 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.828 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.828 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.829 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.829 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.829 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.829 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.829 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.829 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.829 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.829 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.830 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.830 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-07-31T15:15:25.830 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-07-31T15:15:25.830 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.830 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.830 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.831 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:25.831 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.831 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.831 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.831 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.831 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.831 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.831 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.832 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-07-31T15:15:25.832 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-07-31T15:15:25.832 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.832 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.832 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.832 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.833 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.833 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.833 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.833 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.833 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.833 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.833 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.834 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.834 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.834 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.834 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.834 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.834 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.834 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.834 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.835 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.835 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.835 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.835 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.835 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.835 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-07-31T15:15:25.835 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-07-31T15:15:25.836 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.836 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.836 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.836 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.836 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.836 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.836 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.837 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.837 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.837 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.837 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.837 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.837 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.837 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.838 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-07-31T15:15:25.838 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.838 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.838 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.838 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-07-31T15:15:25.838 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-07-31T15:15:25.838 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.838 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.839 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.839 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.839 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.839 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.839 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.839 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.839 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.839 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.840 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.840 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-07-31T15:15:25.840 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-07-31T15:15:25.840 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.840 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.840 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.840 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.841 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.841 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.841 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.841 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.841 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.841 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.841 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.841 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.842 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.842 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.842 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.842 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.842 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.842 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.842 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.843 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.843 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.843 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.843 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.843 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.843 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.843 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.843 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.844 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.844 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.844 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.844 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.844 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.844 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.844 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.844 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.845 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.845 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.845 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.845 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.845 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.845 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.845 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.845 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.846 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.846 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.846 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.846 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.846 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.846 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.846 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.847 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.847 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.847 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.847 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.847 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.847 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.847 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.848 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.848 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.848 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.848 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.848 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.848 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.848 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.848 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.849 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.849 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.849 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.849 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.849 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.849 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.849 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.850 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.850 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.850 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.850 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.850 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.850 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.850 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-31T15:15:25.851 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-31T15:15:25.851 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.851 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.851 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.851 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.851 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.851 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.851 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.851 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.852 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.852 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.852 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.852 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-31T15:15:25.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-31T15:15:25.852 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.853 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.853 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.853 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.853 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.853 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.853 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.853 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.853 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.854 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.854 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.854 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.854 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.854 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.854 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.854 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.855 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.855 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.855 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.855 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.855 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.855 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.855 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.856 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.856 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.856 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.856 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.856 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-07-31T15:15:25.856 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.856 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.857 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.857 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.857 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.857 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.857 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.857 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.857 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.858 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.858 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.858 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.858 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.858 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.858 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.858 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.858 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.859 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.859 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.859 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.859 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.859 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.859 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.860 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.860 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.860 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.860 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.860 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.860 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.860 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.860 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.861 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.861 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.861 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.861 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.861 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.861 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.861 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.861 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.862 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.862 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.862 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.862 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.862 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.862 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.862 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.863 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.863 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.863 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.863 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.863 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.863 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.863 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.863 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.863 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.864 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.864 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.864 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.864 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.864 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.864 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.865 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.865 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.865 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.865 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.865 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.865 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.865 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.865 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.866 INFO:teuthology.orchestra.run.smithi067.stdout: "secondary_zone_period_retry_limit": { 2024-07-31T15:15:25.866 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secondary_zone_period_retry_limit", 2024-07-31T15:15:25.866 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.866 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.866 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.866 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:25.867 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.867 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.867 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.867 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-07-31T15:15:25.867 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.867 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.867 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.868 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.868 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.868 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.868 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.868 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.868 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.868 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.868 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.869 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.870 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.870 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.870 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.870 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.870 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.870 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.870 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.871 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-07-31T15:15:25.871 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.871 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.871 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.871 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.871 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.871 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.872 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.872 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.872 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.872 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.872 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.872 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.872 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.873 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.873 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.873 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.873 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.873 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.873 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.873 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.873 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.874 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.874 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.874 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.874 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.874 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.874 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.875 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.875 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.875 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.875 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.875 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.875 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.875 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.875 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.876 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.876 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.876 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.876 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.876 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.876 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.876 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.876 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.877 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.877 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.877 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.877 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.877 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.877 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.877 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.877 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.878 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.878 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.878 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.878 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.878 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.878 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.878 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.878 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.879 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.879 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.879 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.879 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.879 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.879 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.879 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.879 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.880 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.880 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.880 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.880 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.880 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.880 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.880 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.881 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.881 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.881 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.881 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.881 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.881 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.881 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.881 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.882 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.882 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.882 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.882 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-07-31T15:15:25.882 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-07-31T15:15:25.882 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.882 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.882 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.883 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-07-31T15:15:25.883 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.883 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.883 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.883 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-31T15:15:25.883 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.883 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.884 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.884 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.884 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.884 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.884 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-07-31T15:15:25.884 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.884 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.885 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.885 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.885 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.885 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.885 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.885 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.885 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.886 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.886 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.886 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.886 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.886 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.886 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.886 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.886 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.887 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.887 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.887 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.887 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.887 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.887 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.887 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.887 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.888 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.888 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.888 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.888 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.888 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.888 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.888 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.888 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.889 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.889 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.889 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.889 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.889 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.889 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.889 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.889 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.890 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.890 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.890 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.890 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.890 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.890 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.890 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.890 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.891 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.891 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.891 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.891 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.891 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.891 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.891 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.891 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.892 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.892 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.892 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.892 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.892 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.892 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.892 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.892 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.893 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.893 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.893 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.893 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.893 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.893 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-07-31T15:15:25.893 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-07-31T15:15:25.894 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.894 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.894 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.894 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.894 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.894 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.894 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.894 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.894 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.895 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.895 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.895 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-07-31T15:15:25.895 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-07-31T15:15:25.895 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.895 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.895 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.895 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-07-31T15:15:25.896 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.896 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.896 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.896 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.896 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.896 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.896 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.897 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-07-31T15:15:25.897 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-07-31T15:15:25.897 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.897 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.897 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.897 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.897 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.898 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.898 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.898 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.898 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.898 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.898 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.898 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.898 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-07-31T15:15:25.898 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-07-31T15:15:25.899 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.899 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.899 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.899 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.899 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.899 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.899 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.900 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.900 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.900 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.900 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.900 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-07-31T15:15:25.900 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-07-31T15:15:25.900 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:25.901 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.901 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.901 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.901 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.901 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.901 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.901 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.901 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.902 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.902 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.902 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-07-31T15:15:25.902 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-07-31T15:15:25.902 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.902 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.902 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.903 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.903 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.903 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.903 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.903 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.903 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.903 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.903 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.904 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.904 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-07-31T15:15:25.904 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-07-31T15:15:25.904 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.904 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.904 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.904 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.904 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.905 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.905 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.905 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.905 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.905 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.905 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.905 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-07-31T15:15:25.906 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-07-31T15:15:25.906 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.906 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.906 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.906 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.906 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.906 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.906 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.907 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.907 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.907 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.907 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.907 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-07-31T15:15:25.907 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-07-31T15:15:25.908 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.908 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.908 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.908 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.908 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-31T15:15:25.908 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-07-31T15:15:25.908 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.908 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.908 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.909 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.909 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.909 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.909 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.909 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.909 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.909 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.909 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.910 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.910 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.910 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.910 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.910 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.910 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.910 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.911 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-07-31T15:15:25.911 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-07-31T15:15:25.911 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.911 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.911 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.911 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.911 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.911 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.912 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.912 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.912 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.912 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.912 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.912 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-07-31T15:15:25.912 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-07-31T15:15:25.913 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.913 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.913 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.913 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.913 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.913 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.913 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.913 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.914 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.914 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.914 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.914 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-07-31T15:15:25.914 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-07-31T15:15:25.914 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.914 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.915 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.915 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.915 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.915 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.915 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.915 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.915 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.915 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.916 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.916 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.916 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.916 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.916 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smb", 2024-07-31T15:15:25.916 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.916 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.916 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.917 INFO:teuthology.orchestra.run.smithi067.stdout: "internal_store_backend": { 2024-07-31T15:15:25.917 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "internal_store_backend", 2024-07-31T15:15:25.917 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.917 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.917 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.917 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.917 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.917 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.917 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.918 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-07-31T15:15:25.918 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.918 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.918 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.918 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.918 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.918 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.918 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.919 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.919 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.919 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.919 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.919 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.919 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.919 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.919 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.919 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.920 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.920 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.920 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.920 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.920 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.920 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.920 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.921 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.921 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.921 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.921 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.921 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.921 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.921 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.922 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.922 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.922 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.922 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.922 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.922 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.922 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.923 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.923 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.923 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.923 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.923 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.923 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.923 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.923 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.923 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.924 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.924 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.924 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.924 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.924 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.924 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.924 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.924 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.925 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.925 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.925 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.925 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.925 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.925 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.925 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.926 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.926 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.926 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.926 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.926 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.926 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.926 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.927 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.927 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.927 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.927 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.927 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.927 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.927 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.927 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.928 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.928 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.928 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.928 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.928 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.928 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.928 INFO:teuthology.orchestra.run.smithi067.stdout: "update_orchestration": { 2024-07-31T15:15:25.928 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "update_orchestration", 2024-07-31T15:15:25.929 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.929 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.929 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.929 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.929 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.929 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.929 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.929 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-07-31T15:15:25.929 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.930 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.930 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.930 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.930 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.930 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.930 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-07-31T15:15:25.931 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.931 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.931 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.931 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-07-31T15:15:25.931 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-07-31T15:15:25.931 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.931 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.931 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.932 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.932 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.932 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.932 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.932 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-07-31T15:15:25.932 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.932 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.932 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.933 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.933 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-07-31T15:15:25.933 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-07-31T15:15:25.933 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.933 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.933 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.933 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.934 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.934 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.934 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.934 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-07-31T15:15:25.934 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.934 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.934 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.935 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.935 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.935 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.935 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.935 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.935 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.935 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.935 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.936 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.936 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.936 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.936 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.936 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.936 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.936 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.936 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.936 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.937 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.937 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.937 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.937 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.937 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.937 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.938 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.938 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.938 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.938 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.938 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.938 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.938 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.939 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.939 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.939 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.939 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.939 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.939 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.939 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.939 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.939 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.940 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.940 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.940 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.940 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.940 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.940 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.940 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.940 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.941 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.941 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.941 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.941 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.941 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.941 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.941 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.942 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.942 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.942 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.942 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.942 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.942 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.942 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.943 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.943 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.943 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.943 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.943 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.943 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.943 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.944 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.944 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.944 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.944 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.944 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.944 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.944 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.944 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.945 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.945 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.945 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.945 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.945 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.945 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.945 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.945 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-07-31T15:15:25.946 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.946 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.946 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.946 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.946 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.946 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.946 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.946 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.947 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.947 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.947 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.947 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.947 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.947 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.947 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.947 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.948 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.948 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.948 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.948 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.948 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.948 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.948 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.949 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.949 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.949 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.949 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.949 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.949 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.949 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.949 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.950 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.950 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.950 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.950 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.950 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.950 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.951 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.951 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.951 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.951 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.951 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.951 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.951 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.951 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.951 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.952 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.952 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.952 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.952 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.952 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.952 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.952 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.952 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.953 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.953 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.953 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.953 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.953 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.953 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.953 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.954 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.954 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.954 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.954 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.954 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.954 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.954 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.954 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.955 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.955 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.955 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.955 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.955 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.955 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.956 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.956 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.956 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.956 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.956 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.956 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.956 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.956 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.957 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.957 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.957 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.957 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-07-31T15:15:25.957 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.957 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.957 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.958 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.958 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.958 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.958 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.958 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.958 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.958 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.959 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.959 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.959 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.959 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.959 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.959 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.959 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.959 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.959 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.960 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.960 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.960 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.960 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.960 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.960 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.960 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.961 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.961 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.961 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.961 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.961 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.961 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.961 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.961 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.961 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.962 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.962 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.962 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.962 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.962 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.962 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.962 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.962 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.963 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.963 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.963 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.963 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.963 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.963 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.963 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.964 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.964 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.964 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.964 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.964 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.964 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.964 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.965 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.965 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.965 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.965 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.965 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.965 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.965 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.965 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.966 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.966 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.966 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.966 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.966 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.966 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.966 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.966 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.967 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.967 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.967 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.967 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.967 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.967 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.967 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.967 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.968 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.968 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.968 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.968 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.968 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.968 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.968 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-07-31T15:15:25.969 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.969 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.969 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.969 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-07-31T15:15:25.969 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-07-31T15:15:25.969 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.969 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.969 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.969 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-31T15:15:25.970 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.970 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.970 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.970 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.970 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.970 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.970 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.971 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:25.971 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:25.971 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:25.971 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.971 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.971 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-07-31T15:15:25.971 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.972 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.972 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.972 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.972 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.972 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.972 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.972 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:25.973 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:25.973 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.973 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.973 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.973 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.973 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.973 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.973 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.974 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.974 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.974 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.974 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.974 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.974 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.974 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.974 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.975 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.975 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.975 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.975 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.975 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:25.975 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:25.975 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.975 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.976 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.976 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.976 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.976 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.976 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.976 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.976 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.976 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.976 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.977 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:25.977 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:25.977 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.977 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.977 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.977 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:25.977 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.978 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.978 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:25.978 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:25.978 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:25.978 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:25.978 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:25.978 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:25.978 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:25.979 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:25.979 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.979 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.979 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.979 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.979 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:25.979 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:25.979 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.980 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.980 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.980 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.980 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.980 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.980 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.980 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.980 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.981 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.981 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.981 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.981 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:25.981 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:25.981 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:25.981 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:25.982 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:25.982 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:25.982 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.982 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.982 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.982 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.982 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.982 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.983 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.983 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.983 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:25.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.983 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:25.983 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-07-31T15:15:25.983 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:25.983 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:25.984 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:25.984 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-07-31T15:15:25.984 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-07-31T15:15:25.984 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.984 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.984 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.984 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.984 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.985 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.985 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.985 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-31T15:15:25.985 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.985 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.985 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.985 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-07-31T15:15:25.986 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-07-31T15:15:25.986 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.986 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.986 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.986 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.986 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.986 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.986 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.987 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-31T15:15:25.987 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.987 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.987 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.987 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-07-31T15:15:25.987 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-07-31T15:15:25.987 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.988 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.988 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.988 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:25.988 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.988 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.988 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.988 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-31T15:15:25.988 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.989 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.989 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.989 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-07-31T15:15:25.989 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-07-31T15:15:25.989 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.989 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.990 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.990 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.990 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.990 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.990 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.990 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-31T15:15:25.990 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.990 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.991 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.991 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-07-31T15:15:25.991 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-07-31T15:15:25.991 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.991 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.991 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.992 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.992 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.992 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.992 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.992 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-31T15:15:25.992 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.992 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.992 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.993 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-07-31T15:15:25.993 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-07-31T15:15:25.993 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.993 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.993 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.993 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.993 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.994 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.994 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.994 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.994 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.994 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.994 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.995 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-07-31T15:15:25.995 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-07-31T15:15:25.995 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.995 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.995 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.995 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:25.995 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.996 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.996 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.996 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.996 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.996 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.996 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.996 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-07-31T15:15:25.997 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-07-31T15:15:25.997 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:25.997 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.997 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.997 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-31T15:15:25.997 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.997 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.998 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:25.998 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:25.998 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:25.998 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:25.998 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:25.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:25.998 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-07-31T15:15:25.998 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-07-31T15:15:25.999 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:25.999 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:25.999 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:25.999 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:25.999 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:25.999 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:25.999 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.000 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.000 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.000 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.000 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.000 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.000 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:26.000 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:26.000 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.001 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.001 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.001 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-07-31T15:15:26.001 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-07-31T15:15:26.001 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.001 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.001 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.001 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.001 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.002 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.002 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-07-31T15:15:26.002 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-07-31T15:15:26.002 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.002 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.002 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.002 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-07-31T15:15:26.003 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.003 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.003 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.003 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.003 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.003 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.003 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.004 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-07-31T15:15:26.004 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-07-31T15:15:26.004 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.004 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.004 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.004 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.004 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.004 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.005 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.005 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.005 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.005 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.005 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.005 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-07-31T15:15:26.005 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-07-31T15:15:26.006 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.006 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.006 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.006 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.006 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.006 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.006 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.006 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.007 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.007 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.007 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.007 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.007 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.007 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.007 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.007 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.008 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.008 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.008 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.008 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.008 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.008 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.008 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.008 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.008 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.009 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.009 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.009 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.009 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.009 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.009 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.009 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.009 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.010 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.010 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.010 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.010 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.010 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.010 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.010 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.011 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.011 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.011 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.011 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.011 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.011 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.011 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.011 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.011 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.012 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.012 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.012 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.012 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.012 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.012 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.012 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.012 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.013 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.013 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.013 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.013 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.013 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.013 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.013 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.013 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.014 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.014 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.014 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.014 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.014 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.014 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.014 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.014 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.015 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.015 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.015 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.015 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.015 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.015 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.015 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.015 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-07-31T15:15:26.016 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-07-31T15:15:26.016 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.016 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.016 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.016 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.016 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.016 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.017 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.017 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.017 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.017 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.017 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.017 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-07-31T15:15:26.017 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-07-31T15:15:26.018 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.018 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.018 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.018 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.018 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.018 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.018 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.018 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.019 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.019 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.019 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.019 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.019 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.019 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.019 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.019 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.020 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.020 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.020 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.020 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.020 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.020 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.020 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.020 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.021 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-07-31T15:15:26.021 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-07-31T15:15:26.021 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.021 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.021 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.021 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-31T15:15:26.021 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.021 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.022 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.022 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.022 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.022 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.022 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.022 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.022 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.023 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-07-31T15:15:26.023 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.023 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.023 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.023 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.023 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.024 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.024 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.024 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.024 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.024 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.024 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.024 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.024 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.025 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.025 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.025 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.025 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.025 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.025 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.025 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.026 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.026 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.026 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.026 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.026 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.026 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.026 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.026 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.027 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.027 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.027 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.027 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.027 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.027 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.027 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.027 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.028 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.028 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.028 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.028 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.028 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.028 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.028 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.029 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.029 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.029 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.029 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.029 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.029 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.029 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.029 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.030 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.030 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.030 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.030 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.030 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.030 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.030 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.031 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.031 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.031 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.031 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.031 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.031 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.031 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.031 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.032 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.032 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.032 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.032 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.032 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.032 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.032 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.033 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.033 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.033 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.033 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.033 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.033 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.033 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.033 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.034 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.034 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.034 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.034 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.034 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.034 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.034 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-07-31T15:15:26.035 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.035 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.035 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.035 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.035 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.035 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.035 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.035 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.035 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.036 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.036 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.036 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.036 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.036 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.036 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.036 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.036 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.037 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.037 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.037 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.037 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.037 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.037 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.038 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.038 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.038 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.038 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.038 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.038 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.038 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.038 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.039 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.039 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.039 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.039 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.039 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.039 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.039 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.040 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.040 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.040 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.040 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.040 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.040 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.040 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.040 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.041 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.041 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.041 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.041 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.041 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.041 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.041 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.041 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.042 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.042 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.042 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.042 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.042 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.042 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.042 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.043 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.043 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.043 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.043 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.043 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.043 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.043 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.043 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.044 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.044 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-07-31T15:15:26.044 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-07-31T15:15:26.044 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.044 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.044 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.044 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-07-31T15:15:26.044 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.045 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.045 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.045 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-31T15:15:26.045 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.045 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.045 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.045 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.045 INFO:teuthology.orchestra.run.smithi067.stdout: "periodic_async_work": { 2024-07-31T15:15:26.046 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "periodic_async_work", 2024-07-31T15:15:26.046 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.046 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.046 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.046 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.046 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.046 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.046 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.047 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Periodically check for async work", 2024-07-31T15:15:26.047 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.047 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.047 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.047 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-07-31T15:15:26.047 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-07-31T15:15:26.048 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.048 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.048 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.048 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.048 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.048 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.048 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.048 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-31T15:15:26.049 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.049 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.049 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.049 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-07-31T15:15:26.049 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-07-31T15:15:26.049 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.049 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.050 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.050 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.050 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.050 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.050 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.050 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-31T15:15:26.050 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.051 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.051 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.051 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.051 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.051 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.051 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.052 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.052 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.052 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.052 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.052 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.052 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.052 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.052 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.053 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.053 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.053 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.053 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-07-31T15:15:26.053 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.053 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.054 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.054 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-07-31T15:15:26.054 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-07-31T15:15:26.054 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:26.054 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.054 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.054 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-07-31T15:15:26.055 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.055 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.055 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.055 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.055 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.055 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.055 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.056 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-07-31T15:15:26.056 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-07-31T15:15:26.056 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.056 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.056 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.056 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.056 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.056 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.057 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.057 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.057 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.057 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.057 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.057 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:26.058 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:26.058 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.058 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.058 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.058 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:26.058 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.058 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.059 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.059 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.059 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.059 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.059 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.059 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.059 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.059 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.060 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.060 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.060 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.060 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.060 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.060 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.060 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.061 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.061 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.061 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.061 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.061 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.061 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.061 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.061 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.062 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.062 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.062 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.062 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.062 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.062 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.063 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.063 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.063 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.063 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.063 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.063 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.063 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.063 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.064 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.064 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.064 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.064 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.064 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.064 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.064 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.064 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.065 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.065 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.065 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.065 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.065 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.065 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.065 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.066 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.066 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.066 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.066 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.066 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.066 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.066 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.067 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.067 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.067 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.067 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.067 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.067 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.067 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.067 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.068 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.068 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.068 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.068 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.068 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.068 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.068 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.069 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.069 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.069 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.069 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.069 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.069 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.069 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.069 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.070 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.070 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.070 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-07-31T15:15:26.070 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-07-31T15:15:26.070 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.073 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.074 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.074 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.074 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.074 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.074 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.074 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.074 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.074 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.075 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.075 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-07-31T15:15:26.075 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-07-31T15:15:26.075 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.075 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.075 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.075 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-07-31T15:15:26.076 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.076 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.076 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.076 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.076 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.076 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.076 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.076 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-07-31T15:15:26.077 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-07-31T15:15:26.077 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.077 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.077 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.077 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-31T15:15:26.077 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.077 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.077 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.078 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.078 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.078 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.078 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.078 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.078 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.078 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.078 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-31T15:15:26.079 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.079 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.079 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-07-31T15:15:26.079 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-07-31T15:15:26.079 INFO:teuthology.orchestra.run.smithi067.stdout: "dashboard", 2024-07-31T15:15:26.079 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-07-31T15:15:26.079 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-07-31T15:15:26.079 INFO:teuthology.orchestra.run.smithi067.stdout: "restful", 2024-07-31T15:15:26.080 INFO:teuthology.orchestra.run.smithi067.stdout: "snap_schedule" 2024-07-31T15:15:26.080 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.080 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-07-31T15:15:26.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.080 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-07-31T15:15:26.080 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.080 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.080 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.081 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:26.081 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:26.081 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.081 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.081 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.081 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:26.081 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.081 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.082 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.082 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-07-31T15:15:26.082 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.082 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.082 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.082 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.082 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.083 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.083 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.083 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.083 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.083 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.083 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.083 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.084 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.084 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.084 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.084 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.084 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.084 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.084 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.084 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.085 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.085 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.085 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.085 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.085 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.085 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.085 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.086 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.086 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.086 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.086 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.086 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.086 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.086 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.087 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.087 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.087 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.087 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.087 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.087 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.087 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.088 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.088 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.088 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.088 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.088 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.088 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.088 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.089 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.089 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.089 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.089 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.089 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.089 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.089 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.089 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.090 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.090 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.090 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.090 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.090 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.090 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.090 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.091 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.091 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.091 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.091 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.091 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.091 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.091 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-07-31T15:15:26.092 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-07-31T15:15:26.092 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.092 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.092 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.092 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.092 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.092 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.093 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.093 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to", 2024-07-31T15:15:26.093 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.093 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.093 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.093 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-07-31T15:15:26.094 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-07-31T15:15:26.094 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.094 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.094 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.094 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-07-31T15:15:26.094 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.094 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.094 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.095 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-07-31T15:15:26.095 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.095 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.095 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.095 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-07-31T15:15:26.095 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-07-31T15:15:26.095 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.096 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.096 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.096 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.096 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.096 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.096 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.096 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-07-31T15:15:26.097 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.097 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.097 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.097 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-07-31T15:15:26.097 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-07-31T15:15:26.097 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.097 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.098 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.098 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.098 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.098 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.098 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.098 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-07-31T15:15:26.098 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.099 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.099 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.099 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-07-31T15:15:26.099 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-07-31T15:15:26.099 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.099 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.099 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.100 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-07-31T15:15:26.100 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.100 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.100 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.100 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-07-31T15:15:26.100 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.100 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.100 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.101 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-07-31T15:15:26.101 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-07-31T15:15:26.101 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.101 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.101 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.101 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.101 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.102 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.102 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.102 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-07-31T15:15:26.102 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.102 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.102 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.102 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-07-31T15:15:26.103 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-07-31T15:15:26.103 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.103 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.103 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.103 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.103 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.103 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.104 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.104 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-31T15:15:26.104 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.104 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.104 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.104 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-07-31T15:15:26.104 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-07-31T15:15:26.105 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.105 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.105 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.105 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.105 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.105 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.105 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.105 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-07-31T15:15:26.106 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.106 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.106 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.106 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.106 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.106 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.107 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.107 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.107 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.107 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.107 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.107 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.107 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.108 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.108 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.108 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.108 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.108 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.108 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-07-31T15:15:26.109 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.109 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.109 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.109 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-07-31T15:15:26.109 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-07-31T15:15:26.109 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.109 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.109 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-07-31T15:15:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-07-31T15:15:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-07-31T15:15:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-07-31T15:15:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-07-31T15:15:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-31T15:15:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-07-31T15:15:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-07-31T15:15:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-07-31T15:15:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-07-31T15:15:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-31T15:15:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-31T15:15:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-07-31T15:15:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-07-31T15:15:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-07-31T15:15:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-31T15:15:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-07-31T15:15:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-31T15:15:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-07-31T15:15:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-07-31T15:15:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-07-31T15:15:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-31T15:15:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-31T15:15:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.118 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.118 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-07-31T15:15:26.118 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-07-31T15:15:26.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:26.118 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.118 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.118 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-07-31T15:15:26.119 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-07-31T15:15:26.119 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-07-31T15:15:26.119 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.119 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-07-31T15:15:26.119 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-31T15:15:26.119 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.119 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.120 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-07-31T15:15:26.120 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-07-31T15:15:26.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.120 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.120 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.120 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-07-31T15:15:26.120 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.120 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.120 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.121 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-07-31T15:15:26.121 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-31T15:15:26.121 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.121 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.121 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-07-31T15:15:26.121 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-07-31T15:15:26.122 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:26.122 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.122 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.122 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.122 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-07-31T15:15:26.122 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-07-31T15:15:26.122 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.122 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-31T15:15:26.123 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-31T15:15:26.123 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.123 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.123 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.123 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.123 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.123 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.123 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.124 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.124 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.124 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.124 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.124 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.124 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.124 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.124 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.125 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.125 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.125 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.125 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.125 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.125 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.125 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.126 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.126 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.126 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.126 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.126 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.126 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.126 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.127 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.127 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.127 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.127 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.127 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.127 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.127 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.128 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.128 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.128 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.128 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.128 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.128 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.128 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.128 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.128 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.129 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.129 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.129 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.129 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.129 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.129 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.129 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.129 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.130 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.130 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.130 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.130 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.130 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.130 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.130 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.131 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.131 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.131 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.131 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.131 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.131 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.131 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.131 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.132 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-07-31T15:15:26.132 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-07-31T15:15:26.132 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:26.132 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.132 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.132 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.132 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.133 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.133 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.133 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-31T15:15:26.133 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.133 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.133 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.134 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-07-31T15:15:26.134 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-07-31T15:15:26.134 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.134 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.134 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.134 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-07-31T15:15:26.134 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.134 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.134 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.135 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-07-31T15:15:26.135 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-07-31T15:15:26.135 INFO:teuthology.orchestra.run.smithi067.stdout: "read", 2024-07-31T15:15:26.135 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap", 2024-07-31T15:15:26.135 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap-read" 2024-07-31T15:15:26.135 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.135 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-07-31T15:15:26.135 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.136 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.136 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.136 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-07-31T15:15:26.136 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-07-31T15:15:26.136 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.136 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.136 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.136 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.137 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.137 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.137 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.137 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-31T15:15:26.137 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.137 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.137 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.138 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:26.138 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:26.138 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.138 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.138 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.138 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:26.139 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.139 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.139 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.139 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-31T15:15:26.139 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.139 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.139 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.140 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.140 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.140 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.140 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.140 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.140 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.140 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.140 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.141 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.141 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.141 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.141 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.141 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.141 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-07-31T15:15:26.142 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-07-31T15:15:26.142 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.142 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.142 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.142 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:26.142 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-31T15:15:26.142 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.142 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.143 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-31T15:15:26.143 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-31T15:15:26.143 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.143 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.143 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-07-31T15:15:26.143 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-07-31T15:15:26.144 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:26.144 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.144 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.144 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:26.144 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.144 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.144 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.145 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-31T15:15:26.145 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.145 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.145 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.145 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.145 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.146 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-07-31T15:15:26.146 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.146 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.146 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.146 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-07-31T15:15:26.146 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-07-31T15:15:26.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:26.147 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.147 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.147 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-07-31T15:15:26.147 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.147 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.147 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.147 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-31T15:15:26.147 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.148 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.148 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.148 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-07-31T15:15:26.148 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-07-31T15:15:26.148 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.148 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.148 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.149 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-07-31T15:15:26.149 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.149 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.149 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.149 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-31T15:15:26.149 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.149 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.149 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.150 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-07-31T15:15:26.150 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-07-31T15:15:26.150 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.150 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.150 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.150 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-07-31T15:15:26.151 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.151 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.151 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.151 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-31T15:15:26.151 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.151 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.151 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.152 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-07-31T15:15:26.152 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-07-31T15:15:26.152 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.152 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.152 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.152 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.152 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.152 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.153 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.153 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-31T15:15:26.153 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-31T15:15:26.153 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.153 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.153 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-07-31T15:15:26.153 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-07-31T15:15:26.154 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.154 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.154 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.154 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:26.154 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.154 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.154 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.154 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-31T15:15:26.155 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.155 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.155 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.155 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-07-31T15:15:26.155 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-07-31T15:15:26.155 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:26.156 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.156 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.156 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-07-31T15:15:26.156 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.156 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.156 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.156 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-31T15:15:26.156 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.157 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.157 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.157 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm_log_destination": { 2024-07-31T15:15:26.157 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm_log_destination", 2024-07-31T15:15:26.157 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.157 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.158 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.158 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.158 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.158 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.158 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.158 INFO:teuthology.orchestra.run.smithi067.stdout: "file", 2024-07-31T15:15:26.158 INFO:teuthology.orchestra.run.smithi067.stdout: "file,syslog", 2024-07-31T15:15:26.158 INFO:teuthology.orchestra.run.smithi067.stdout: "syslog" 2024-07-31T15:15:26.159 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.159 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-31T15:15:26.159 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.159 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.159 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.159 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-07-31T15:15:26.159 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-07-31T15:15:26.160 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.160 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.160 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.160 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.160 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.160 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.160 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.160 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-31T15:15:26.161 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.161 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.161 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.161 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-07-31T15:15:26.161 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-07-31T15:15:26.161 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.161 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.161 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.162 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.162 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.162 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.162 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.162 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-31T15:15:26.162 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.162 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.162 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.163 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-07-31T15:15:26.163 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-07-31T15:15:26.163 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.163 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.163 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.163 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.163 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.164 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.164 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.164 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-31T15:15:26.164 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.164 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.164 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.164 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-07-31T15:15:26.165 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-07-31T15:15:26.165 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.165 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.165 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.165 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-07-31T15:15:26.165 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.165 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.165 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.166 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-31T15:15:26.166 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.166 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.166 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.166 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-07-31T15:15:26.166 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-07-31T15:15:26.166 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.167 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.167 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.167 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-31T15:15:26.167 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.167 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.167 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.167 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-07-31T15:15:26.168 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.168 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.168 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.168 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-07-31T15:15:26.168 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-07-31T15:15:26.168 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.168 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.169 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.169 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-31T15:15:26.169 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.169 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.169 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.169 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-07-31T15:15:26.169 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.169 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.169 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.170 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-07-31T15:15:26.170 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-07-31T15:15:26.170 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.170 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.170 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.170 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-07-31T15:15:26.171 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.171 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.171 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.171 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-31T15:15:26.171 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.171 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.171 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.172 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-07-31T15:15:26.172 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-07-31T15:15:26.172 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.172 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.172 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.172 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-31T15:15:26.172 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.172 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.173 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.173 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-07-31T15:15:26.173 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.173 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.173 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.173 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-07-31T15:15:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-07-31T15:15:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-31T15:15:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.174 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.175 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-07-31T15:15:26.175 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.175 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.175 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.175 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-07-31T15:15:26.175 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-07-31T15:15:26.175 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.176 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.176 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.176 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-31T15:15:26.176 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.176 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.176 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.176 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-07-31T15:15:26.176 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.177 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.177 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.177 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-07-31T15:15:26.177 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-07-31T15:15:26.177 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.177 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.178 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.178 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-31T15:15:26.178 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.178 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.178 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.178 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-07-31T15:15:26.178 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.178 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.179 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.179 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-07-31T15:15:26.179 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-07-31T15:15:26.179 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.179 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.179 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.180 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-31T15:15:26.180 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.180 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.180 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.180 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-07-31T15:15:26.180 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.180 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.180 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.181 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-07-31T15:15:26.181 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-07-31T15:15:26.181 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.181 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.181 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.181 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-31T15:15:26.182 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.182 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.182 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.182 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-07-31T15:15:26.182 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.182 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.182 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.183 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nginx": { 2024-07-31T15:15:26.183 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nginx", 2024-07-31T15:15:26.183 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.183 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.183 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.183 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-07-31T15:15:26.183 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.184 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.184 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.184 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nginx container image", 2024-07-31T15:15:26.184 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.184 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.184 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.184 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.185 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-07-31T15:15:26.185 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-07-31T15:15:26.185 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.185 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.185 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.185 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-07-31T15:15:26.185 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.185 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.186 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.186 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-31T15:15:26.186 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.186 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.186 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.186 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.186 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-07-31T15:15:26.186 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-07-31T15:15:26.187 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.187 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.187 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.187 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-31T15:15:26.187 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.187 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.187 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.187 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-07-31T15:15:26.188 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.188 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.188 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.188 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-07-31T15:15:26.188 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-07-31T15:15:26.188 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.188 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.188 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.189 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-07-31T15:15:26.189 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.189 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.189 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.189 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-31T15:15:26.189 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.189 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.189 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.190 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.190 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-07-31T15:15:26.190 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-07-31T15:15:26.190 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.190 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.190 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.190 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-31T15:15:26.191 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.191 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.191 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.191 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-07-31T15:15:26.191 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.191 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.191 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.191 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.191 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba": { 2024-07-31T15:15:26.192 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba", 2024-07-31T15:15:26.192 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.192 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.192 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.192 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-31T15:15:26.192 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.192 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.192 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.192 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB container image", 2024-07-31T15:15:26.193 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.193 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.193 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.193 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.193 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-07-31T15:15:26.193 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-07-31T15:15:26.193 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.194 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.194 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.194 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-31T15:15:26.194 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.194 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.194 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.194 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-07-31T15:15:26.194 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.195 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.195 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.195 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.195 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-07-31T15:15:26.195 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-07-31T15:15:26.195 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.195 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.196 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.196 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.196 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.196 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.196 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.196 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-07-31T15:15:26.196 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.196 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.197 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.197 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-07-31T15:15:26.197 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-07-31T15:15:26.197 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.197 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.197 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.197 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:26.198 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.198 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.198 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.198 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-31T15:15:26.198 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.198 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.198 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.199 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-07-31T15:15:26.199 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-07-31T15:15:26.199 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.199 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.199 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.199 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-07-31T15:15:26.199 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.199 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.200 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.200 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-31T15:15:26.200 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.200 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.200 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.200 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-07-31T15:15:26.200 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-07-31T15:15:26.200 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.201 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.201 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.201 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-07-31T15:15:26.201 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.201 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.201 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.201 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-31T15:15:26.201 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.202 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.202 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.202 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.202 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-07-31T15:15:26.202 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-07-31T15:15:26.202 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.202 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.202 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.203 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-07-31T15:15:26.203 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.203 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.203 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.203 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-07-31T15:15:26.203 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.203 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.203 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.204 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.204 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-07-31T15:15:26.204 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-07-31T15:15:26.204 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.204 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.204 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.204 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.204 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.204 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.205 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.205 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-31T15:15:26.205 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.205 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.205 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.205 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.205 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-07-31T15:15:26.205 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-07-31T15:15:26.206 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.206 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.206 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.206 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:26.206 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.206 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.206 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.206 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-07-31T15:15:26.206 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.207 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.207 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.207 INFO:teuthology.orchestra.run.smithi067.stdout: "grafana_dashboards_path": { 2024-07-31T15:15:26.207 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "grafana_dashboards_path", 2024-07-31T15:15:26.207 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.207 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.207 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.208 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-31T15:15:26.208 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.208 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.208 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.208 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-31T15:15:26.208 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.208 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.208 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.209 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-07-31T15:15:26.209 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-07-31T15:15:26.209 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.209 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.209 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.209 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:26.209 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.209 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.210 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.210 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-07-31T15:15:26.210 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.210 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.210 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.210 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.210 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-07-31T15:15:26.210 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-07-31T15:15:26.210 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.211 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.211 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.211 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.211 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.211 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.211 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.211 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-31T15:15:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-07-31T15:15:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-07-31T15:15:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.212 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-31T15:15:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.213 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.214 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.214 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.214 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.214 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.214 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.214 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.214 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.215 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.215 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.215 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.215 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.215 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.216 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.216 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.216 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.216 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.216 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.216 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.216 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.216 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.217 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-07-31T15:15:26.217 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-07-31T15:15:26.217 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.217 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.217 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.217 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.217 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.218 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.218 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.218 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-31T15:15:26.218 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.218 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.218 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.218 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.219 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.219 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.219 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.219 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.219 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.219 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.219 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.220 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.220 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-31T15:15:26.220 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.220 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.220 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.220 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.220 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.220 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.221 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.221 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.221 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.221 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.221 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.221 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.221 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.221 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.222 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.222 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.222 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.222 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.222 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.222 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.222 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.222 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.223 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.223 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.223 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.223 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.223 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.223 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.224 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.224 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.224 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.224 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.224 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.224 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.224 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.224 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.225 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-31T15:15:26.225 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-31T15:15:26.225 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.225 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.225 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.225 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.225 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.226 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.226 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.226 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-31T15:15:26.226 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.226 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.226 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.227 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-31T15:15:26.227 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-31T15:15:26.227 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.227 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.227 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.227 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-07-31T15:15:26.227 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.228 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.228 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.228 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-31T15:15:26.228 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.228 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.228 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.228 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-07-31T15:15:26.229 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-07-31T15:15:26.229 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.229 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.229 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.229 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:26.229 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.229 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.229 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.229 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-07-31T15:15:26.230 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.230 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.230 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.230 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-07-31T15:15:26.230 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-07-31T15:15:26.230 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.231 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.231 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.231 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:26.231 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.231 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.231 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.231 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-31T15:15:26.231 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.232 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.232 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.232 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-07-31T15:15:26.232 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-07-31T15:15:26.232 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.232 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.232 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.233 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.233 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.233 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.233 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.233 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-07-31T15:15:26.233 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.233 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.233 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.234 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-07-31T15:15:26.234 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-07-31T15:15:26.234 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.234 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.234 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.234 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-07-31T15:15:26.234 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.235 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.235 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.235 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-07-31T15:15:26.235 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-07-31T15:15:26.235 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.235 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-07-31T15:15:26.235 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.235 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.236 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.236 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-07-31T15:15:26.236 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-07-31T15:15:26.236 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.236 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.236 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.237 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-07-31T15:15:26.237 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.237 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.237 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.237 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-31T15:15:26.237 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.237 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.237 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.238 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-07-31T15:15:26.238 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-07-31T15:15:26.238 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.238 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.238 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.238 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-31T15:15:26.239 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.239 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.239 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.239 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-31T15:15:26.239 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.239 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.239 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.240 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-07-31T15:15:26.240 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-07-31T15:15:26.240 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.240 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.240 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.240 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.240 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.240 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.240 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.241 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-31T15:15:26.241 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.241 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.241 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.241 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-07-31T15:15:26.241 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-07-31T15:15:26.241 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.242 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.242 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.242 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.242 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.242 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.242 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.243 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-31T15:15:26.243 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.243 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.243 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.243 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.243 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-07-31T15:15:26.243 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-07-31T15:15:26.243 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.243 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.244 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.244 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.244 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.244 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.244 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.244 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-31T15:15:26.244 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.244 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.245 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.245 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.245 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-07-31T15:15:26.245 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-07-31T15:15:26.245 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.245 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.245 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.245 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.246 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.246 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.246 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.246 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-31T15:15:26.246 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.246 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.246 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.247 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-07-31T15:15:26.247 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-07-31T15:15:26.247 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.247 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.247 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.247 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.247 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.247 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.248 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.248 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-31T15:15:26.248 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.248 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.248 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.248 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-07-31T15:15:26.248 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-07-31T15:15:26.248 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.249 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.249 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.249 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-07-31T15:15:26.249 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.249 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.249 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.249 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-07-31T15:15:26.249 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.250 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.250 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.250 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.250 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.250 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.250 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.250 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.251 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.251 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.251 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.251 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.251 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.251 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.251 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.252 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.252 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-07-31T15:15:26.252 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-07-31T15:15:26.252 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.252 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.252 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.252 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.253 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.253 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.253 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.253 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-31T15:15:26.253 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.253 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.253 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.254 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-07-31T15:15:26.254 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-07-31T15:15:26.254 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.254 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.254 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.254 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.254 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.255 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.255 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.255 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-31T15:15:26.255 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.255 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.255 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.255 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-07-31T15:15:26.256 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-07-31T15:15:26.256 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.256 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.256 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.256 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.256 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.256 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.256 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.257 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-31T15:15:26.257 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.257 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.257 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.257 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-07-31T15:15:26.257 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-07-31T15:15:26.258 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.258 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.258 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.258 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.258 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.258 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.258 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.258 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-07-31T15:15:26.259 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.259 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.259 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.259 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-07-31T15:15:26.259 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-07-31T15:15:26.259 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.260 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.260 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.260 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.260 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.260 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.260 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.260 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-31T15:15:26.260 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.261 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.261 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.261 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-07-31T15:15:26.261 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-07-31T15:15:26.261 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.261 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.261 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.262 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.262 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.262 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.262 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.262 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-31T15:15:26.262 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.262 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.263 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.263 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.263 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.263 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.263 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-07-31T15:15:26.263 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.263 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.264 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.264 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.264 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.264 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.264 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.264 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.264 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.264 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.265 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.265 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.265 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.265 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.265 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.265 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.265 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.265 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.266 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.266 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.266 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.266 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.266 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.266 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.267 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.267 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.267 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.267 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.267 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.267 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.267 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.267 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.268 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.268 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.268 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.268 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.268 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.268 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.268 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.269 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.269 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.269 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.269 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.269 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.269 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.269 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.270 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.270 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.270 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.270 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.270 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.270 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.270 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.271 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.271 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.271 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.271 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.271 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.271 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.271 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.271 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.272 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.272 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.272 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.272 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.272 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.272 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.272 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.273 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.273 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.273 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-07-31T15:15:26.273 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-07-31T15:15:26.273 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.273 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.273 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.274 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-07-31T15:15:26.274 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.274 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.274 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.274 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-31T15:15:26.274 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.274 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.274 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.275 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.275 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.275 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.275 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.275 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.275 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.275 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.276 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.277 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.277 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.277 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.278 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.278 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.278 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-07-31T15:15:26.278 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-07-31T15:15:26.278 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.278 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.279 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.279 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-07-31T15:15:26.279 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.279 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.279 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.279 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-31T15:15:26.279 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.280 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.280 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.280 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.280 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.280 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-07-31T15:15:26.280 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.281 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.281 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.281 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-31T15:15:26.281 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-31T15:15:26.281 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.281 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.281 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.281 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:26.282 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.282 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.282 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.282 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.282 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.282 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.282 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.283 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-07-31T15:15:26.283 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-31T15:15:26.283 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.283 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.283 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.283 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.283 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.284 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.284 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.284 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.284 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.284 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.284 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.284 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.284 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-31T15:15:26.285 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-31T15:15:26.285 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.285 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.285 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.285 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.285 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.285 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.285 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.286 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.286 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.286 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.286 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.286 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-07-31T15:15:26.286 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-07-31T15:15:26.286 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.287 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.287 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.287 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.287 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.287 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.287 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.287 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.287 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.288 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.288 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.288 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-31T15:15:26.288 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-31T15:15:26.288 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.288 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.288 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.288 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.289 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.289 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.289 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.289 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.289 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.289 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.289 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.290 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-07-31T15:15:26.290 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-31T15:15:26.290 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.290 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.290 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.290 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.290 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.291 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.291 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.291 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.291 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.291 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.291 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.291 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-31T15:15:26.292 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-31T15:15:26.292 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.292 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.292 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.292 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.292 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.292 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.292 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.293 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.293 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.293 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.293 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.293 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.293 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-31T15:15:26.293 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-31T15:15:26.293 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.294 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.294 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.294 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.294 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.294 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.294 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.294 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.294 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.295 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.295 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.295 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-31T15:15:26.295 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-31T15:15:26.295 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.295 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.295 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.296 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.296 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.296 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.296 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.296 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.296 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.296 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.296 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.297 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-31T15:15:26.297 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-31T15:15:26.297 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.297 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.297 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.297 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.297 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.298 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.298 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.298 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.298 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.298 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.298 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.298 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-31T15:15:26.299 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-31T15:15:26.299 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.299 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.299 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.299 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.299 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.299 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.299 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.300 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.300 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.300 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.300 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.300 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-31T15:15:26.300 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-31T15:15:26.300 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.301 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.301 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.301 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.301 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.301 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.301 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.301 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.301 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.301 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.302 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.302 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-31T15:15:26.302 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-31T15:15:26.302 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.302 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.302 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.303 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.303 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.303 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.303 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.303 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.303 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.303 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.303 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.304 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-31T15:15:26.304 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-31T15:15:26.304 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.304 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.304 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.304 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.304 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.304 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.305 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.305 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.305 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.305 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.305 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.305 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-07-31T15:15:26.305 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-31T15:15:26.306 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.306 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.306 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.306 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-31T15:15:26.306 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.306 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.306 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.306 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.307 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.307 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.307 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.307 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.307 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-31T15:15:26.307 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-31T15:15:26.307 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.308 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.308 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.308 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.308 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.308 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.308 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.308 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.308 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.309 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.309 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.309 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-07-31T15:15:26.309 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-07-31T15:15:26.309 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.309 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.309 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.310 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.310 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.310 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.310 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.310 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.310 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.310 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.310 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.311 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.311 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-07-31T15:15:26.311 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-31T15:15:26.311 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.311 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.311 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.311 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-31T15:15:26.311 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.312 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.312 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.312 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.312 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.312 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.312 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.312 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-31T15:15:26.313 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-31T15:15:26.313 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.313 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.313 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.313 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.313 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.313 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.313 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.313 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.314 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.314 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.314 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.314 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-31T15:15:26.314 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-31T15:15:26.314 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.315 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.315 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.315 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.315 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.315 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.315 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.315 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.315 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.316 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.316 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.316 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-31T15:15:26.316 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-31T15:15:26.316 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.316 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.316 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.317 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.317 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.317 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.317 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.317 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.317 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.317 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.318 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.318 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-31T15:15:26.318 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-31T15:15:26.318 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.318 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.318 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.318 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.319 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.319 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.319 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.319 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.319 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.319 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.319 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.319 INFO:teuthology.orchestra.run.smithi067.stdout: "MANAGED_BY_CLUSTERS": { 2024-07-31T15:15:26.320 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-07-31T15:15:26.320 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.320 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.320 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.320 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "[]", 2024-07-31T15:15:26.320 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.320 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.320 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.321 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.321 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.321 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.321 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.321 INFO:teuthology.orchestra.run.smithi067.stdout: "MULTICLUSTER_CONFIG": { 2024-07-31T15:15:26.321 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MULTICLUSTER_CONFIG", 2024-07-31T15:15:26.321 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.322 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.322 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.322 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "{}", 2024-07-31T15:15:26.322 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.322 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.322 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.322 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.322 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.323 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.323 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.323 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-07-31T15:15:26.323 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-31T15:15:26.323 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.323 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.323 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.324 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.324 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.324 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.324 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.324 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.324 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.324 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.325 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.325 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-31T15:15:26.325 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-31T15:15:26.325 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.325 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.325 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.325 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.326 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.326 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.326 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.326 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.326 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.326 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.326 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.327 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-31T15:15:26.327 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-31T15:15:26.327 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.327 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.327 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.327 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.327 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.328 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.328 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.328 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.328 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.328 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.328 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.329 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-31T15:15:26.329 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-31T15:15:26.329 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.329 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.329 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.329 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.329 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.329 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.330 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.330 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.330 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.330 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.330 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.330 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-31T15:15:26.330 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-31T15:15:26.331 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.331 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.331 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.331 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.331 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.331 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.331 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.331 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.332 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.332 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.332 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.332 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-31T15:15:26.332 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-31T15:15:26.332 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.333 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.333 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.333 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.333 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.333 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.333 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.333 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.333 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.333 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.334 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.334 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-31T15:15:26.334 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-31T15:15:26.334 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.334 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.334 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.335 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.335 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.335 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.335 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.335 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.335 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.335 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.335 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.336 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-31T15:15:26.336 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-31T15:15:26.336 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.336 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.336 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.336 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.336 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.337 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.337 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.337 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.337 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.337 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.337 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.338 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-31T15:15:26.338 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-31T15:15:26.338 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.338 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.338 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.338 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.338 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.339 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.339 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.339 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.339 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.339 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.339 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.339 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-07-31T15:15:26.340 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-31T15:15:26.340 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.340 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.340 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.340 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.340 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.340 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.340 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.340 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.341 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.341 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.341 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.341 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-31T15:15:26.341 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-31T15:15:26.341 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.342 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.342 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.342 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-31T15:15:26.342 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.342 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.342 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.342 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.343 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.343 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.343 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.343 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-31T15:15:26.343 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-31T15:15:26.343 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.343 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.344 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.344 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:26.344 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.344 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.344 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.344 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.344 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.345 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.345 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.345 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-31T15:15:26.345 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-31T15:15:26.345 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.345 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.345 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.345 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-07-31T15:15:26.346 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.346 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.346 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.346 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.346 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.346 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.346 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.347 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-31T15:15:26.347 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-31T15:15:26.347 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.347 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.347 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.347 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-07-31T15:15:26.347 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.348 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.348 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.348 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.348 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.348 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.348 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.348 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-07-31T15:15:26.349 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-31T15:15:26.349 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.349 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.349 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.349 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.349 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.349 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.350 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.350 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.350 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.350 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.350 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.350 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-31T15:15:26.350 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-31T15:15:26.351 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.351 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.351 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.351 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-31T15:15:26.351 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.351 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.351 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.351 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.352 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.352 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.352 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.352 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-07-31T15:15:26.352 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-31T15:15:26.352 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.352 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.353 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.353 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.353 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.353 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.353 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.353 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.353 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.353 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.354 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.354 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-07-31T15:15:26.354 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-31T15:15:26.354 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.354 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.354 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.355 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.355 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.355 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.355 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.355 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.355 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.355 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.355 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.356 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-07-31T15:15:26.356 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-07-31T15:15:26.356 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.356 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.356 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.356 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.356 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.357 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.357 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.357 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.357 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.357 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.357 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.357 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-31T15:15:26.358 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-31T15:15:26.358 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.358 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.358 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.358 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.358 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.358 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.359 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.359 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.359 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.359 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.359 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.359 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-31T15:15:26.359 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-31T15:15:26.359 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.360 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.360 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.360 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:26.360 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.360 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.360 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.360 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.360 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.361 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.361 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.361 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-31T15:15:26.361 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-31T15:15:26.361 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.361 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.361 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.361 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:26.362 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.362 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.362 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.362 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.362 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.362 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.362 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.363 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-07-31T15:15:26.363 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-07-31T15:15:26.363 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.363 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.363 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.363 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.363 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.364 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.364 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.364 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.364 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.364 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.364 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.364 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-07-31T15:15:26.365 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-07-31T15:15:26.365 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.365 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.365 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.365 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.365 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.365 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.365 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.366 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.366 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.366 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.366 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.366 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-07-31T15:15:26.366 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-07-31T15:15:26.367 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.367 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.367 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.367 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.367 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.367 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.367 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.367 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-07-31T15:15:26.367 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.368 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.368 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.368 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-07-31T15:15:26.368 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-07-31T15:15:26.368 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.368 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.368 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.369 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-07-31T15:15:26.369 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.369 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.369 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.369 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.369 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.369 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.369 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.370 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-07-31T15:15:26.370 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-07-31T15:15:26.370 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.370 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.370 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.370 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.370 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.371 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.371 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.371 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.371 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.371 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.371 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.372 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.372 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.372 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.372 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.372 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.372 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.372 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.372 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.373 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.373 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.373 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.373 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.373 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.373 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.373 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.373 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.374 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.374 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.374 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.374 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.374 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.374 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.374 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.375 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.375 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.375 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.375 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.375 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.375 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.375 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.376 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.376 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.376 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.376 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.376 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.376 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.376 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.377 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.377 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.377 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.377 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.377 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.377 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.377 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.378 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.378 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.378 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.378 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.378 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.378 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.378 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.378 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.379 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.379 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.379 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.379 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.379 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.379 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.379 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.380 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.380 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.380 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.380 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.380 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.380 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.380 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.381 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-07-31T15:15:26.381 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-07-31T15:15:26.381 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.381 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.381 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.381 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.381 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.382 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.382 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.382 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-07-31T15:15:26.382 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.382 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.382 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.382 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-07-31T15:15:26.383 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-07-31T15:15:26.383 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.383 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.383 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.383 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.383 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.383 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.384 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.384 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.384 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.384 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.384 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.384 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-31T15:15:26.384 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-31T15:15:26.385 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.385 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.385 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.385 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-07-31T15:15:26.385 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.385 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.385 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.385 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.386 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.386 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.386 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.386 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-31T15:15:26.386 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-31T15:15:26.386 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.386 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.387 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.387 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-07-31T15:15:26.387 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.387 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.387 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.387 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.387 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.388 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.388 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.388 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.388 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.388 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.388 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.388 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.389 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.389 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.389 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.389 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.389 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.389 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.389 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.389 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.390 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-07-31T15:15:26.390 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-07-31T15:15:26.390 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.390 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.390 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.390 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.391 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.391 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.391 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.391 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.391 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.391 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.391 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.391 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-07-31T15:15:26.392 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-07-31T15:15:26.392 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.392 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.392 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.392 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-07-31T15:15:26.392 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.392 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.392 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.393 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.393 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.393 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.393 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.393 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-07-31T15:15:26.393 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-07-31T15:15:26.393 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.394 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.394 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.394 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-07-31T15:15:26.394 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.394 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.394 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.394 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.395 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-07-31T15:15:26.395 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.395 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.395 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.395 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.395 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.395 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-07-31T15:15:26.395 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-07-31T15:15:26.396 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.396 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.396 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.396 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-07-31T15:15:26.396 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-07-31T15:15:26.396 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-07-31T15:15:26.396 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.396 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.397 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.397 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.397 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.397 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-07-31T15:15:26.397 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-07-31T15:15:26.397 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.397 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.397 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.398 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.398 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.398 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.398 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.398 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.398 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.398 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.398 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.399 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.399 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.399 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-07-31T15:15:26.399 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.399 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.399 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.399 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-07-31T15:15:26.400 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-07-31T15:15:26.400 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.400 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.400 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.400 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.400 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.400 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.400 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.401 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-07-31T15:15:26.401 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.401 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.401 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.401 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.401 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.401 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.402 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.402 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.402 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.402 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.402 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.402 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.402 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.402 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.403 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.403 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.403 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.403 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.403 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.403 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.403 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.403 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.403 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.404 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.404 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.404 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.404 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.404 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.404 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.404 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.405 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.405 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.405 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.405 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.405 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.405 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.405 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.406 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.406 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.406 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.406 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.406 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.406 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.406 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.407 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.407 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.407 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.407 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.407 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.407 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.407 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.407 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.408 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.408 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.408 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.408 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.408 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.408 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.408 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.409 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.409 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.409 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.409 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.409 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.409 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.409 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.410 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.410 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.410 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.410 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-07-31T15:15:26.410 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-07-31T15:15:26.410 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.410 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.411 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.411 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-07-31T15:15:26.411 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.411 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.411 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.411 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-31T15:15:26.411 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.412 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.412 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.412 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-07-31T15:15:26.412 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-07-31T15:15:26.412 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.412 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.412 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.413 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-07-31T15:15:26.413 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.413 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.413 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.413 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-31T15:15:26.413 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.413 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.413 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.414 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-07-31T15:15:26.414 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-07-31T15:15:26.414 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.414 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.414 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.414 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-07-31T15:15:26.414 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.415 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.415 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.415 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-07-31T15:15:26.415 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.415 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.415 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.415 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-07-31T15:15:26.416 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-07-31T15:15:26.416 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.416 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.416 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.416 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-07-31T15:15:26.416 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.416 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.416 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.417 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-31T15:15:26.417 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.417 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.417 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.417 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-07-31T15:15:26.417 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-07-31T15:15:26.417 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.418 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.418 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.418 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.418 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.418 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.418 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.418 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-31T15:15:26.419 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.419 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.419 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.419 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:26.419 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:26.419 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.419 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.420 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.420 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:26.420 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.420 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.420 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.420 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-07-31T15:15:26.420 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.420 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.421 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.421 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.421 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.421 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.421 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.421 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.421 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.422 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.422 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.422 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.422 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.422 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.422 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.422 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.423 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-07-31T15:15:26.423 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-07-31T15:15:26.423 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.423 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.423 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.423 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-07-31T15:15:26.423 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.424 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.424 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.424 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-31T15:15:26.424 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.424 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.424 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.424 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.425 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.425 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-07-31T15:15:26.425 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.425 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.425 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.425 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.426 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.426 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.426 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.426 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.426 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.426 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.426 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.427 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.427 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.427 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.427 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.427 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.427 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.427 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.427 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.428 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.428 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.428 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.428 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.428 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.428 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.429 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.429 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.429 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.429 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.429 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.429 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.429 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.430 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.430 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.430 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.430 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.430 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.430 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.430 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.431 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.431 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.431 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.431 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.431 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.431 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.431 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.431 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.432 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.432 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.432 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.432 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.432 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.432 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.432 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.432 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.433 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.433 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.433 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.433 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.433 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.433 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.434 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.434 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.434 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.434 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.434 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.434 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.434 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.435 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.435 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-07-31T15:15:26.435 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-07-31T15:15:26.435 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.435 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.435 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.435 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-07-31T15:15:26.436 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.436 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.436 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.436 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.436 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.436 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.436 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.437 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-07-31T15:15:26.437 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-07-31T15:15:26.437 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.437 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.437 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.437 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-07-31T15:15:26.437 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.438 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.438 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.438 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.438 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.438 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.438 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.439 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:26.439 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:26.439 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.439 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.439 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.439 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-31T15:15:26.439 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.439 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.440 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.440 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.440 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.440 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.440 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.440 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.440 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.440 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.441 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.441 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.441 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.441 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.441 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.441 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.441 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.441 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.441 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.442 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.442 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.442 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.442 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-07-31T15:15:26.442 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-07-31T15:15:26.442 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-07-31T15:15:26.443 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.443 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-07-31T15:15:26.443 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-07-31T15:15:26.443 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.443 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.443 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.443 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-07-31T15:15:26.443 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.444 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.444 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.444 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-31T15:15:26.444 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.444 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.444 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.444 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-07-31T15:15:26.445 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-07-31T15:15:26.445 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.445 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.445 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.445 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-07-31T15:15:26.445 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.445 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-31T15:15:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-07-31T15:15:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-07-31T15:15:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-07-31T15:15:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-07-31T15:15:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-07-31T15:15:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-31T15:15:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-07-31T15:15:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-07-31T15:15:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-07-31T15:15:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-07-31T15:15:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-07-31T15:15:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-07-31T15:15:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-07-31T15:15:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.463 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.463 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.463 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.463 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.463 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.463 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.463 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.463 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.463 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.464 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.464 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.464 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.464 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-07-31T15:15:26.464 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-07-31T15:15:26.464 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.464 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.465 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.465 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-07-31T15:15:26.465 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.465 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.465 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.465 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-31T15:15:26.465 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.466 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.466 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.466 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-07-31T15:15:26.466 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-07-31T15:15:26.466 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.466 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.467 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.467 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:26.467 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-31T15:15:26.467 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-07-31T15:15:26.467 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.467 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-31T15:15:26.467 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.467 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.468 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.468 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-07-31T15:15:26.468 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-07-31T15:15:26.468 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.468 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.468 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.468 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.469 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.469 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.469 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.469 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-07-31T15:15:26.469 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.469 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.469 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.470 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-07-31T15:15:26.470 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-07-31T15:15:26.470 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.470 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.470 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.470 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-07-31T15:15:26.470 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.471 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.471 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.471 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-31T15:15:26.471 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.471 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.471 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.471 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.471 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.472 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-07-31T15:15:26.472 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.472 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.472 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.472 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.472 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.473 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.473 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.473 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.473 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.473 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.473 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.473 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.473 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.474 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.474 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.474 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.474 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.474 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.474 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.474 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.475 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.475 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.475 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.475 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.475 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.475 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.475 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.476 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.476 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.476 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.476 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.476 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.476 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.476 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.477 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.477 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.477 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.477 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.477 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.477 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.477 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.478 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.478 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.478 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.478 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.478 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.478 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.478 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.478 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.479 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.479 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.479 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.479 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.479 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.479 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.479 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.480 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.481 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.482 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.482 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.482 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.482 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.482 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.483 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.483 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.483 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.483 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.484 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.484 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.484 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.484 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.484 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.485 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.485 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.485 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.485 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.485 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.486 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.486 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.486 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.486 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.486 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.486 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.486 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.487 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-07-31T15:15:26.487 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.487 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.487 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.487 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.488 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.488 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.488 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.488 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.488 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.488 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.488 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.488 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.489 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.489 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.489 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.489 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.489 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.489 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.489 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.489 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.489 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.490 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.490 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.490 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.490 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.490 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.490 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.490 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.490 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.491 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.491 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.491 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.491 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.491 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.491 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.491 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.492 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.492 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.492 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.492 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.492 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.492 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.492 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.492 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.492 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.493 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.493 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.493 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.493 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.493 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.493 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.493 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.493 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.493 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.494 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.494 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.494 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.494 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.494 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.494 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.495 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.495 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.495 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.495 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.495 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.495 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.495 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.495 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.496 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.496 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.496 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.496 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.496 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.496 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.496 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.497 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.497 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.497 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.497 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.497 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.497 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.497 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.497 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.498 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.498 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-07-31T15:15:26.498 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.498 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.498 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.498 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-07-31T15:15:26.498 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-07-31T15:15:26.499 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.499 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.499 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.499 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-07-31T15:15:26.499 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.499 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.499 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.499 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-31T15:15:26.500 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.500 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.500 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.500 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-07-31T15:15:26.500 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-07-31T15:15:26.500 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.500 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.500 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.501 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:26.501 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-07-31T15:15:26.501 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.501 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.501 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-31T15:15:26.501 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.501 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.501 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.502 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.502 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.502 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.502 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.502 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.502 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.502 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.503 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.503 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.503 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.503 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.503 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.503 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.503 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.503 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.504 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.504 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.504 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.504 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.504 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.504 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.504 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.505 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.505 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.505 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.505 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.505 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.505 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.505 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.505 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.506 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.506 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.506 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.506 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.506 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.506 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.507 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.507 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.507 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.507 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.507 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.507 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.507 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.507 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.508 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.508 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.508 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.508 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.508 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.508 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.508 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.509 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.509 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.509 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.509 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.509 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.509 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.509 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.510 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.510 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.510 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.510 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.510 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.510 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.510 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.511 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.511 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.511 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.511 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.511 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.511 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.511 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.512 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.512 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.512 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.512 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.512 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.512 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.512 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.513 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.513 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.513 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.513 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-07-31T15:15:26.513 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.513 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.513 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.514 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-07-31T15:15:26.514 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-07-31T15:15:26.514 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.514 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.514 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.514 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-07-31T15:15:26.514 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.515 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.515 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.515 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-07-31T15:15:26.515 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-31T15:15:26.515 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.515 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.516 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.516 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.516 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.516 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.516 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.516 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.516 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.516 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.517 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.517 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.517 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.517 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.517 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.517 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.517 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.518 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.518 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.518 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.518 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.518 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.518 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.519 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.519 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.519 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.519 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.519 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.519 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.519 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.520 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.520 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.520 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.520 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.520 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.520 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.521 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.521 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.521 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.521 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.521 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.521 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.521 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.521 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.522 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.522 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.522 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.522 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.522 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.522 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.522 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.523 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.523 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.523 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.523 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.523 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.523 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.523 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.524 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.524 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.524 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.524 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.524 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.524 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.524 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.524 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.524 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.525 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-07-31T15:15:26.525 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-07-31T15:15:26.525 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.525 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.525 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.525 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.525 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.526 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.526 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.526 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-07-31T15:15:26.526 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-31T15:15:26.526 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.526 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.526 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-07-31T15:15:26.527 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-07-31T15:15:26.527 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.527 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.527 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.527 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-07-31T15:15:26.527 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.527 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.528 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.528 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-07-31T15:15:26.528 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.528 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.528 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.528 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-07-31T15:15:26.528 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-07-31T15:15:26.529 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.529 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.529 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.529 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-07-31T15:15:26.529 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.529 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.529 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.529 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-07-31T15:15:26.530 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.530 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.530 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.530 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-07-31T15:15:26.530 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-07-31T15:15:26.530 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.530 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.531 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.531 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.531 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.531 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.531 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.531 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-07-31T15:15:26.531 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.531 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.531 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-07-31T15:15:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-07-31T15:15:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-07-31T15:15:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-31T15:15:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-31T15:15:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-07-31T15:15:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.541 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.541 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.541 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.541 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.541 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.541 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.541 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.541 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.541 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.542 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.542 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.542 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.542 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.542 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.542 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.542 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.542 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.543 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.543 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.543 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.543 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.543 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.543 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.543 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.543 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.544 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.544 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.544 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.544 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.544 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.544 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.545 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.545 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.545 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.545 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.545 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.545 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.545 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.545 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.546 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.546 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.546 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.546 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.546 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-07-31T15:15:26.546 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.547 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.547 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.547 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.547 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.547 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.547 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.547 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.548 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.548 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.548 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.548 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.548 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.548 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.548 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.548 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.548 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.549 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.549 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.549 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.549 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.549 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.549 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.549 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.550 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.550 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.550 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.550 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.550 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.550 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.550 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.550 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.551 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.551 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.551 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.551 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.551 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.551 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.552 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.552 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.552 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.552 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.552 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.552 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.552 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.552 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.552 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.553 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.553 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.553 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.553 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.553 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.553 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.553 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.554 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.554 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.554 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.554 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.554 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.554 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.554 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.555 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.555 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.555 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.555 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.555 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.555 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.555 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.556 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.556 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.556 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.556 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.556 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.556 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.556 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.556 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.557 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.557 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.557 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.557 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.557 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.557 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.557 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.557 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.558 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-07-31T15:15:26.558 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.558 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.558 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.558 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.558 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.559 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.559 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.559 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.559 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.559 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.559 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.559 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.560 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.560 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.560 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.560 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.560 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.560 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.560 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.560 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.561 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.561 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.561 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.561 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.561 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.562 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.562 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.562 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.562 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.562 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.562 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.562 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.562 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.563 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.563 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.563 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.563 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.563 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.564 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.564 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.564 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.564 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.564 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.564 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.564 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.564 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.565 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.565 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.565 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.565 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.565 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.565 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.565 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.566 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.566 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.566 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.566 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.566 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.566 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.566 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.567 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.567 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.567 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.567 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.567 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.567 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.567 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.568 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.568 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.568 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.568 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.568 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.568 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.568 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.568 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.569 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.569 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.569 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.569 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.569 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.569 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.570 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.570 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-07-31T15:15:26.570 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.570 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.570 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.570 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-07-31T15:15:26.570 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-07-31T15:15:26.570 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.571 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.571 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.571 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.571 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.571 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.571 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.571 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-31T15:15:26.572 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.572 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.572 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.572 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.572 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.572 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.572 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.573 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.573 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.573 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.573 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.573 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.573 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.573 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.573 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.573 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.574 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.574 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.574 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.574 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.574 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.574 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.574 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.575 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.575 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.575 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.575 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.575 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.575 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.575 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.576 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.576 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.576 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.576 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.576 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.576 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.576 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.577 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.577 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.577 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.577 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.577 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.577 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.578 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.578 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.578 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.578 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.578 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.578 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.578 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.578 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.579 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.579 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.579 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.579 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.579 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.579 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.579 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.580 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.580 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.580 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.580 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.580 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.580 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.580 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.580 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.581 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.581 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.581 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.581 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-07-31T15:15:26.581 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-07-31T15:15:26.581 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.581 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.581 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.582 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.582 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.582 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.582 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.582 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-07-31T15:15:26.582 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-07-31T15:15:26.582 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-07-31T15:15:26.583 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.583 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-07-31T15:15:26.583 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.583 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.583 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.583 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.584 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.584 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.584 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.584 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.584 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.584 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.584 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.584 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.585 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.585 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.585 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.585 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.585 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.585 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.585 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.586 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-07-31T15:15:26.586 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.586 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.586 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.586 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.586 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.586 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.587 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.587 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.587 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.587 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.587 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.587 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.587 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.587 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.588 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.588 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.588 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.588 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.588 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.588 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.588 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.588 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.589 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.589 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.589 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.589 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.589 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.589 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.589 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.590 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.590 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.590 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.590 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.590 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.590 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.590 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.591 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.591 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.591 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.591 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.591 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.591 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.591 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.592 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.592 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.592 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.592 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.592 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.592 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.592 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.593 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.593 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.593 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.593 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.593 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.593 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.593 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.594 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.594 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.594 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.594 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.594 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.594 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.594 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.595 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.595 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.595 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.595 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.595 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.595 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.595 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.596 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.596 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.596 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.596 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.596 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.596 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.596 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.597 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.597 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.597 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.597 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.597 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.597 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-07-31T15:15:26.598 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.598 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.598 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.598 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.598 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.598 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.599 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.599 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.599 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.599 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.599 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.599 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.599 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.599 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.600 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.600 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.600 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.600 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.600 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.600 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.600 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.601 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.601 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.601 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.601 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.601 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.601 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.601 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.602 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.602 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.602 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.602 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.602 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.602 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.602 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.603 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.603 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.603 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.603 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.603 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.603 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.603 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.603 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.604 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.604 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.604 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.604 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.604 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.604 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.604 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.604 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.605 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.605 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.605 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.605 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.605 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.605 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.606 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.606 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.606 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.606 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.606 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.606 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.606 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.607 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.607 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.607 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.607 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.607 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.607 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.607 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.608 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.608 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.608 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.608 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.608 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.608 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.608 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.608 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.609 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.609 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.609 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.609 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.609 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.609 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-07-31T15:15:26.609 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.609 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.610 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.610 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.610 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.610 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.610 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.610 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.610 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.611 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.611 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.611 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.611 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.611 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.611 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.611 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.611 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.612 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.612 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.612 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.612 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.612 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.612 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.613 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.613 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.613 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.613 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.613 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.613 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.613 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.613 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.614 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.614 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.614 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.614 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.614 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.614 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.614 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.615 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.615 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.615 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.615 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.615 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.615 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.615 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.615 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.616 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.616 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.616 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.616 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.616 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.616 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.616 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.617 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.617 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.617 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.617 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.617 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.617 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.617 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.618 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.618 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.618 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.618 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.618 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.618 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.618 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.619 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.619 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.619 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:26.619 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:26.619 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.619 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.619 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.620 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:26.620 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.620 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.620 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.620 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.620 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.620 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.620 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.621 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.621 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.621 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.621 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.621 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.621 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.621 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.622 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.622 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.622 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.622 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.622 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.622 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.623 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-07-31T15:15:26.623 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-07-31T15:15:26.623 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:26.623 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.623 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.623 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-07-31T15:15:26.623 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-07-31T15:15:26.623 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.624 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.624 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-07-31T15:15:26.624 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-31T15:15:26.624 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.624 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.624 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.624 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.625 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.625 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-07-31T15:15:26.625 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.625 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.625 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.625 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-07-31T15:15:26.625 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-07-31T15:15:26.625 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.625 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.626 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.626 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.626 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.626 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.626 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.626 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-31T15:15:26.626 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.627 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.627 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.629 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-07-31T15:15:26.629 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-07-31T15:15:26.630 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.630 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.630 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.630 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.630 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.630 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.630 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.631 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.631 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.631 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.631 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.631 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.631 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.631 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.632 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.632 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.632 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.632 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.632 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.632 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.632 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.632 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.632 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.633 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.633 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.633 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.633 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.633 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.633 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.633 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.633 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.634 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.634 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.634 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.634 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.634 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.634 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.634 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.635 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.635 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.635 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.635 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.635 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.635 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.635 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.636 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.636 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.636 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.636 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.636 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.636 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.636 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.636 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.637 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.637 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.637 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.637 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.637 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.637 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.637 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.638 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.638 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.638 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.638 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.638 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.638 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.638 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.639 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.639 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.639 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.639 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.639 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.639 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.639 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.640 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.640 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.640 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.640 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.640 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.640 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.641 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-07-31T15:15:26.641 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-07-31T15:15:26.641 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.641 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.641 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.641 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-07-31T15:15:26.641 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.642 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.642 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.642 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-07-31T15:15:26.642 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.642 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.642 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.642 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.642 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-31T15:15:26.643 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-31T15:15:26.643 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.643 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.643 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.643 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:26.643 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.643 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.643 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.644 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-07-31T15:15:26.644 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.644 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.644 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.644 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.644 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.644 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.645 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.645 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.645 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.645 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.645 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.645 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.645 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.645 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.646 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.646 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.646 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.646 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.646 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.647 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-07-31T15:15:26.647 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.647 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.647 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.647 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-07-31T15:15:26.647 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-07-31T15:15:26.647 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.647 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.648 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.648 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.648 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.648 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.648 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.648 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.648 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.648 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.649 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.649 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-07-31T15:15:26.649 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-07-31T15:15:26.649 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.649 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.649 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.649 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.650 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.650 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.650 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.650 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-31T15:15:26.650 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-31T15:15:26.650 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.650 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.651 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.651 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.651 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.651 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.651 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.651 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.651 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.651 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.652 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.652 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.652 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.652 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.652 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.652 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.652 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.652 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.653 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.653 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.653 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.653 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.653 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.653 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.653 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.653 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.653 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.654 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.654 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.654 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.654 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.654 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.654 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.654 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.655 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.655 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.655 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.655 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.655 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.655 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.655 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.655 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.656 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.656 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.656 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.656 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.656 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.656 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.656 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.657 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.657 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.657 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.657 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.657 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.657 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.657 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.658 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.658 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.658 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.658 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.658 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.658 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.658 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.658 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.659 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.659 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.659 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.659 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.659 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.659 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-07-31T15:15:26.659 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-07-31T15:15:26.659 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.660 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.660 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.660 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.660 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.660 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.660 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.660 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.660 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.661 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.661 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.661 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.661 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-31T15:15:26.661 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-31T15:15:26.661 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.661 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.662 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.662 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-07-31T15:15:26.662 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.662 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.662 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.662 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.662 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.663 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.663 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.663 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-07-31T15:15:26.663 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-07-31T15:15:26.663 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:26.663 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.663 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.663 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-07-31T15:15:26.664 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.664 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.664 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.664 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.664 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.664 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.664 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.665 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-31T15:15:26.665 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-31T15:15:26.665 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.665 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.665 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.665 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-07-31T15:15:26.665 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.665 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.666 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.666 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-31T15:15:26.666 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.666 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.666 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.666 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.666 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-31T15:15:26.667 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-31T15:15:26.667 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.667 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.667 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.667 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-07-31T15:15:26.667 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.667 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.667 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.668 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-31T15:15:26.668 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.668 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.668 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.668 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.668 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.668 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.668 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.669 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.669 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.669 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.669 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.669 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.669 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.669 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.669 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.670 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.670 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.670 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.670 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-07-31T15:15:26.670 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-07-31T15:15:26.670 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.670 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.671 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.671 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-07-31T15:15:26.671 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.671 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.671 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.671 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.671 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.671 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.672 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.672 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.672 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-07-31T15:15:26.672 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-07-31T15:15:26.672 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.672 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.672 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.672 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-07-31T15:15:26.673 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.673 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.673 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.673 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-07-31T15:15:26.673 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-07-31T15:15:26.673 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.673 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.674 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.674 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.674 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.674 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.674 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-07-31T15:15:26.674 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-07-31T15:15:26.674 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.675 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.675 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.675 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-07-31T15:15:26.675 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-07-31T15:15:26.675 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-07-31T15:15:26.675 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.675 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.675 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.676 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.676 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.676 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.676 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.676 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.676 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.676 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-07-31T15:15:26.677 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.677 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.677 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.677 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.677 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.677 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.677 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.677 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.678 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.678 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.678 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.678 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.678 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.678 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.678 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.679 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.679 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.679 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.679 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.679 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.679 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.679 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.680 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.680 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.680 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.680 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.680 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.680 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.680 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.680 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.681 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.681 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.681 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.681 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.681 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.681 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.681 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.682 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.682 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.682 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.682 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.682 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.682 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.682 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.682 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.683 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.683 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.683 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.683 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.683 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.683 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.683 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.684 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.684 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.684 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.684 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.684 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.684 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.684 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.685 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.685 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.685 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.685 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.685 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.685 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.685 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.686 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.686 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.686 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.686 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.686 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.686 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-07-31T15:15:26.687 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-07-31T15:15:26.687 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.687 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.687 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.687 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-31T15:15:26.687 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.687 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.687 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-07-31T15:15:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-07-31T15:15:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-07-31T15:15:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-07-31T15:15:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-07-31T15:15:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-07-31T15:15:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-07-31T15:15:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-07-31T15:15:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-07-31T15:15:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.700 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.700 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.700 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.700 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.700 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.700 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.700 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.701 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.701 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.701 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.701 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.701 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.701 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.701 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.701 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.702 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.702 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.702 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.702 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.702 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.702 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.702 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.703 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.703 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.703 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.703 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.703 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.703 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.703 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.704 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.704 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.704 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.704 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.704 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.704 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.704 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.704 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.705 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.705 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.705 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.705 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.705 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.705 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.705 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.705 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.706 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.706 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.706 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.706 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.706 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.706 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.707 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.707 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.707 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.707 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.707 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.707 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.707 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.708 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-31T15:15:26.708 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-31T15:15:26.708 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.708 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.708 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.708 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.708 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.708 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.709 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.709 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.709 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.709 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.709 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.709 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-31T15:15:26.710 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-31T15:15:26.710 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.710 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.710 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.710 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.710 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.710 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.710 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.711 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.711 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.711 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.711 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.711 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.711 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.711 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.712 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.712 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.712 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.712 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.712 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.712 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.712 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.712 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.713 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.713 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.713 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.713 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.713 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.713 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-07-31T15:15:26.714 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.714 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.714 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.714 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.714 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.714 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.714 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.714 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.715 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.715 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.715 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.715 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.715 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.715 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.715 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.715 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.716 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.716 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.716 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.716 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.716 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.716 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.716 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.717 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.717 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.717 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.717 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.717 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.717 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.717 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.718 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.718 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.718 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.718 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.718 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.718 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.718 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.719 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.719 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.719 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.719 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.719 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.719 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.719 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.719 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.720 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.720 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.720 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.720 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.720 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.720 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.720 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.720 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.720 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.721 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.721 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.721 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.721 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.721 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.721 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.722 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.722 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.722 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.722 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.722 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.722 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.722 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.723 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.723 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.723 INFO:teuthology.orchestra.run.smithi067.stdout: "secondary_zone_period_retry_limit": { 2024-07-31T15:15:26.723 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secondary_zone_period_retry_limit", 2024-07-31T15:15:26.723 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.723 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.723 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.724 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-31T15:15:26.724 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.724 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.724 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.724 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-07-31T15:15:26.724 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.724 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.724 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.725 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.725 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.725 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.725 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.725 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.725 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.725 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.726 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.726 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.726 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.726 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.726 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.726 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.726 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.726 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.727 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.727 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-07-31T15:15:26.727 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.727 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.727 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.727 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.727 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.728 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.728 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.728 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.728 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.728 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.728 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.728 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.729 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.729 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.729 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.729 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.729 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.729 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.729 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.729 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.730 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.730 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.730 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.730 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.730 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.730 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.730 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.731 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.731 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.731 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.731 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.731 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.731 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.731 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.731 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.732 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.732 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.732 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.732 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.732 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.732 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.732 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.732 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.733 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.733 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.733 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.733 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.733 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.733 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.733 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.733 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.734 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.734 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.734 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.734 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.734 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.734 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.735 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.735 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.735 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.735 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.735 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.735 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.735 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.735 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.736 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.736 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.736 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.736 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.736 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.736 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.736 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.737 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.737 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.737 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.737 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.737 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.737 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.737 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.738 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.738 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.738 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.738 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.738 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-07-31T15:15:26.738 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-07-31T15:15:26.738 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.739 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.739 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.739 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-07-31T15:15:26.739 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.739 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.739 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.739 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-31T15:15:26.739 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.740 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.740 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.740 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.740 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.740 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.740 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.740 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-07-31T15:15:26.740 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.741 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.741 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.741 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.741 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.741 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.741 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.741 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.741 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.742 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.742 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.742 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.742 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.742 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.742 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.742 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.742 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.743 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.743 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.743 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.743 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.743 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.743 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.743 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.744 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.744 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.744 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.744 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.744 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.744 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.744 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.744 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.745 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.745 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.745 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.745 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.745 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.745 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.745 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.745 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.746 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.746 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.746 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.746 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.746 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.746 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.746 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.747 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.747 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.747 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.747 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.747 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.747 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.747 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.747 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.748 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.748 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.748 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.748 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.748 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.748 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.748 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.748 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.749 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.749 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.749 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.749 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.749 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.749 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.749 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.750 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-07-31T15:15:26.750 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-07-31T15:15:26.750 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.750 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.750 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.750 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.750 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.751 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.751 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.751 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.751 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.751 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.751 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.751 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-07-31T15:15:26.752 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-07-31T15:15:26.752 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.752 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.752 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.752 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-07-31T15:15:26.752 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.752 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.752 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.753 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.753 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.753 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.753 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.753 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-07-31T15:15:26.753 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-07-31T15:15:26.753 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.754 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.754 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.754 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.754 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.754 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.754 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.754 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.755 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.755 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.755 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.755 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-07-31T15:15:26.755 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-07-31T15:15:26.755 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.756 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.756 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.756 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.756 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.756 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.756 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.756 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.756 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.757 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.757 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.757 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-07-31T15:15:26.757 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-07-31T15:15:26.757 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-31T15:15:26.757 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.758 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.758 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.758 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.758 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.758 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.758 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.758 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.758 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.759 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.759 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.759 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-07-31T15:15:26.759 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-07-31T15:15:26.759 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.759 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.759 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.759 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.760 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.760 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.760 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.760 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.760 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.760 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.760 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.760 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.761 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-07-31T15:15:26.761 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-07-31T15:15:26.761 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.761 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.761 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.761 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.761 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.762 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.762 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.762 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.762 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.762 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.762 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.762 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-07-31T15:15:26.763 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-07-31T15:15:26.763 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.763 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.763 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.763 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.763 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.763 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.763 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.763 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.764 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.764 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.764 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.764 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-07-31T15:15:26.764 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-07-31T15:15:26.764 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.764 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.765 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.765 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.765 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-31T15:15:26.765 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-07-31T15:15:26.765 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.765 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.765 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.765 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.765 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.766 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.766 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.766 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.766 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.766 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.767 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.767 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.767 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.767 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.767 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.767 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.767 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.767 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.768 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-07-31T15:15:26.768 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-07-31T15:15:26.768 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.768 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.768 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.768 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.769 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.769 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.769 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.769 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.769 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.769 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.769 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.770 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-07-31T15:15:26.770 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-07-31T15:15:26.770 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.770 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.770 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.770 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.770 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.770 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.771 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.771 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.771 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.771 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.771 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.771 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-07-31T15:15:26.771 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-07-31T15:15:26.772 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.772 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.772 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.772 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.772 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.772 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.772 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.773 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.773 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.773 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.773 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.773 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.773 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.773 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.774 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smb", 2024-07-31T15:15:26.774 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.774 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.774 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.774 INFO:teuthology.orchestra.run.smithi067.stdout: "internal_store_backend": { 2024-07-31T15:15:26.774 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "internal_store_backend", 2024-07-31T15:15:26.774 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.774 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.775 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.775 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.775 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.775 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.775 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.775 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-07-31T15:15:26.775 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.775 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.776 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.776 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.776 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.776 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.776 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.776 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.776 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.777 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.777 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.777 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.777 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.777 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.777 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.777 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.778 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.778 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.778 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.778 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.778 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.778 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.778 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.779 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.779 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.779 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.779 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.779 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.779 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.779 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.779 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.780 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.780 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.780 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.780 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.780 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.780 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.780 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.781 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.781 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.781 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.781 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.781 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.781 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.781 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.783 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.783 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.783 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.783 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.783 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.783 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.783 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.784 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.784 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.784 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.784 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.784 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.784 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.784 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.785 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.785 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.785 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.785 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.785 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.785 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.785 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.786 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.786 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.786 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.786 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.786 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.786 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.786 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.787 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.787 INFO:teuthology.orchestra.run.smithi067.stdout: "update_orchestration": { 2024-07-31T15:15:26.787 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "update_orchestration", 2024-07-31T15:15:26.787 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.787 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.787 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.787 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.788 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.788 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.788 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.788 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-07-31T15:15:26.788 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.788 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.788 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.789 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.789 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.789 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-07-31T15:15:26.789 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.789 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.789 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.790 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-07-31T15:15:26.790 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-07-31T15:15:26.790 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.790 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.790 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.790 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.790 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.791 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.791 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.791 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-07-31T15:15:26.791 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.791 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.791 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.791 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-07-31T15:15:26.792 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-07-31T15:15:26.792 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.792 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.792 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.792 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.792 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.792 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.792 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.793 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-07-31T15:15:26.793 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.793 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.793 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.793 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.793 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.794 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.794 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.794 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.794 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.794 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.794 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.794 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.794 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.795 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.795 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.795 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.795 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.795 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.795 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.795 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.796 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.796 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.796 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.796 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.796 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.796 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.797 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.797 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.797 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.797 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.797 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.797 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.797 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.797 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.798 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.798 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.798 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.798 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.798 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.798 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.798 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-07-31T15:15:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.815 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.815 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.815 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.815 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.815 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.815 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.815 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-07-31T15:15:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.827 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.827 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.827 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.827 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.827 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.827 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.827 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-07-31T15:15:26.828 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.828 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.828 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.828 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-07-31T15:15:26.828 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-07-31T15:15:26.828 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.828 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.829 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.829 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-31T15:15:26.829 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.829 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.829 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.829 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.829 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.829 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.830 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.830 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:26.830 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:26.830 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.830 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.830 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.831 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-07-31T15:15:26.831 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.831 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.831 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.831 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.831 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.831 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.831 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.832 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.832 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.832 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.832 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.832 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.832 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.832 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.833 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.833 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.833 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.833 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.833 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.833 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.833 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.833 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.834 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.834 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.834 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.834 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.834 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.834 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.834 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.835 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.835 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.835 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.835 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.835 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.835 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.835 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.836 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.836 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.836 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.836 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.836 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.836 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.836 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.836 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.837 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.837 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.837 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.837 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.837 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.837 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.837 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.837 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.838 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.838 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.838 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.838 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.838 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.838 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.838 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.839 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.839 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.839 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.839 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.839 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.839 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.839 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.839 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.840 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.840 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.840 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.840 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.840 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.840 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.840 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.841 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.841 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.841 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.841 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.841 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.841 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.842 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.842 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.842 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.842 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.842 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.842 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.842 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.842 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.843 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.843 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.843 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-07-31T15:15:26.843 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.843 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.843 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.844 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-07-31T15:15:26.844 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-07-31T15:15:26.844 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.844 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.844 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.844 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.844 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.844 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.845 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.845 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-31T15:15:26.845 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.845 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.845 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.845 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-07-31T15:15:26.845 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-07-31T15:15:26.846 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.846 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.846 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.846 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.846 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.846 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.846 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.847 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-31T15:15:26.847 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.847 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.847 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.847 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.847 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-07-31T15:15:26.847 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-07-31T15:15:26.848 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.848 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.848 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.848 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.848 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.848 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.848 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.848 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-31T15:15:26.849 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.849 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.849 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.849 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-07-31T15:15:26.849 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-07-31T15:15:26.849 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.850 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.850 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.850 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.850 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.850 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.850 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.850 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-31T15:15:26.850 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.851 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.851 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.851 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.851 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-07-31T15:15:26.851 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-07-31T15:15:26.851 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.851 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.851 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.851 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.852 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.852 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.852 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.852 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-31T15:15:26.852 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.852 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.852 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.852 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-07-31T15:15:26.853 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-07-31T15:15:26.853 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.853 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.853 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.853 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.853 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.853 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.853 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.854 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.854 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.854 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.854 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.854 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-07-31T15:15:26.854 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-07-31T15:15:26.855 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.855 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.855 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.855 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.855 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.855 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.855 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.855 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.856 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.856 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.856 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.856 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-07-31T15:15:26.856 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-07-31T15:15:26.856 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.856 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.857 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.857 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-31T15:15:26.857 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.857 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.857 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.857 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.857 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.857 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.857 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.858 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-07-31T15:15:26.858 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-07-31T15:15:26.858 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.858 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.858 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.858 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.858 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.859 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.859 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.859 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.859 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.859 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.859 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.859 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:26.860 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:26.860 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.860 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.860 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.860 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-07-31T15:15:26.860 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-07-31T15:15:26.860 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.860 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.861 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.861 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.861 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.861 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.861 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-07-31T15:15:26.861 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-07-31T15:15:26.861 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.862 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.862 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.862 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-07-31T15:15:26.862 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.862 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.862 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.862 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.863 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.863 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.863 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.863 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-07-31T15:15:26.863 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-07-31T15:15:26.863 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.863 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.863 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.864 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.864 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.864 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.864 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.864 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.864 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.864 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.864 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.865 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-07-31T15:15:26.865 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-07-31T15:15:26.865 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.865 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.865 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.865 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.865 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.866 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.866 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.866 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.866 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.866 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.866 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.866 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.867 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.867 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.867 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.867 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.867 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.867 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.867 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.867 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.868 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.868 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.868 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.868 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.868 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.868 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.868 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.868 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.869 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.869 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.869 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.869 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.869 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.869 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.869 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.870 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.870 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.870 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.870 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.870 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.870 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.870 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.871 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.871 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.871 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.871 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.871 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.871 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.871 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.872 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.872 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.872 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.872 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.872 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.872 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.873 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.873 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.873 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.873 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.873 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.873 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.873 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.874 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.874 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.874 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.875 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.875 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.875 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.875 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.876 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.876 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.876 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.876 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.877 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.877 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.877 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.877 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.877 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.878 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-07-31T15:15:26.878 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-07-31T15:15:26.878 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.878 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.878 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.879 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.879 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.879 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.879 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.880 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.880 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.880 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.880 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.881 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-07-31T15:15:26.881 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-07-31T15:15:26.881 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.881 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.882 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.882 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.882 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.882 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.883 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.883 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.883 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.883 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.884 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.884 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.884 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.885 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.885 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.885 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.885 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.886 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.886 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.886 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.886 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.887 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.887 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.887 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.887 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.888 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-07-31T15:15:26.888 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-07-31T15:15:26.888 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.888 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.889 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.889 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-31T15:15:26.889 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.889 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.890 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.890 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.890 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.890 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.893 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.894 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.894 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.894 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.895 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-07-31T15:15:26.895 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.895 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.895 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.896 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.896 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.896 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.896 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.896 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.897 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.897 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.897 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.897 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.898 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.898 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.898 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.898 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.899 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.899 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.899 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.899 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.900 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.900 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.900 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.900 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.901 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.901 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.901 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.901 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.902 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.902 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.902 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.902 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.903 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.903 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.903 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.903 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.904 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.904 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.904 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.904 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.905 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.905 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.905 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.905 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.906 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.906 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.906 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.906 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.907 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.907 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.907 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.907 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.907 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.908 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.908 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.908 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.908 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.909 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.909 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.909 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.909 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.909 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.909 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.909 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.909 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.910 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.910 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.910 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.910 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.910 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.910 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.911 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.911 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.911 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.911 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.911 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.911 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.911 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.911 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.912 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.912 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.912 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.912 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.912 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.912 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.912 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.913 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-07-31T15:15:26.913 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.913 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.913 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.913 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.913 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.913 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.914 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.914 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.914 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.914 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.914 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.914 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.914 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.915 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.915 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.915 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.915 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.915 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.915 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.915 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.915 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.915 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.916 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.916 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.916 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.916 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.916 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.916 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.917 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.917 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.917 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.917 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.917 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.917 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.917 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.917 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.918 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.918 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.918 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.918 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.918 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.918 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.919 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.919 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.919 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.919 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.919 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.919 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.919 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.919 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.920 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.920 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.920 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.920 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.920 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.920 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.920 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.920 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.921 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.921 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.921 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.921 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.921 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.921 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.921 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.921 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.922 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.922 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.922 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.922 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.922 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-07-31T15:15:26.922 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-07-31T15:15:26.923 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.923 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.923 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.923 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-07-31T15:15:26.923 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.923 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.923 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.923 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-31T15:15:26.924 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.924 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.924 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.924 INFO:teuthology.orchestra.run.smithi067.stdout: "periodic_async_work": { 2024-07-31T15:15:26.924 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "periodic_async_work", 2024-07-31T15:15:26.924 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.925 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.925 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.925 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.925 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.925 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.925 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.925 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Periodically check for async work", 2024-07-31T15:15:26.926 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.926 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.926 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.926 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.926 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-07-31T15:15:26.926 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-07-31T15:15:26.926 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.927 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.927 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.927 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.927 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.927 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.927 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.927 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-31T15:15:26.928 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.928 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.928 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.928 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-07-31T15:15:26.928 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-07-31T15:15:26.929 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.929 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.929 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.929 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-31T15:15:26.929 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.929 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.929 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.930 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-31T15:15:26.930 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.930 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.930 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.930 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.931 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.931 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.931 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.931 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.931 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.931 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.931 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.932 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.932 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.932 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.932 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.932 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.932 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.932 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.933 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.933 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-07-31T15:15:26.933 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-31T15:15:26.933 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-31T15:15:26.933 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-31T15:15:26.933 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-07-31T15:15:26.933 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-07-31T15:15:26.933 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-31T15:15:26.934 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.934 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.934 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-07-31T15:15:26.934 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.934 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.934 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.934 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.935 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.935 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.935 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.935 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-07-31T15:15:26.935 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-07-31T15:15:26.935 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.935 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.936 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.936 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.936 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.936 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.936 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.936 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.936 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.937 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.937 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.937 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-31T15:15:26.937 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-31T15:15:26.937 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-31T15:15:26.937 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.937 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.938 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-31T15:15:26.938 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.938 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.938 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.938 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.938 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.938 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.939 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.939 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.939 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-31T15:15:26.939 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-31T15:15:26.939 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.939 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.939 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.939 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.940 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.940 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.940 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.940 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.940 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.940 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.940 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.940 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.941 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.941 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.941 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.941 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.941 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.941 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.941 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-31T15:15:26.942 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-31T15:15:26.942 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.942 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.942 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.942 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.942 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.942 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.943 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.943 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.943 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.943 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.943 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.943 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-31T15:15:26.943 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-31T15:15:26.944 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.944 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.944 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.944 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-31T15:15:26.944 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.944 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.944 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-31T15:15:26.944 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-31T15:15:26.945 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-31T15:15:26.945 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-31T15:15:26.945 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-31T15:15:26.945 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-31T15:15:26.945 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-31T15:15:26.945 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.945 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.945 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.946 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.946 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.946 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-31T15:15:26.946 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-31T15:15:26.946 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-31T15:15:26.946 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.946 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.947 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-31T15:15:26.947 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.947 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.947 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.947 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.947 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.947 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.948 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.948 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-31T15:15:26.948 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-31T15:15:26.948 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.948 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-31T15:15:26.948 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-31T15:15:26.948 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-31T15:15:26.949 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.949 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.949 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.949 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.949 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.949 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.949 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.950 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-07-31T15:15:26.950 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-07-31T15:15:26.950 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.950 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.950 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.950 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-31T15:15:26.950 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.951 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.951 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.951 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.951 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.951 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.951 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.951 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-07-31T15:15:26.952 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-07-31T15:15:26.952 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-31T15:15:26.952 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.952 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.952 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-07-31T15:15:26.952 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.952 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.953 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.953 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.953 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.953 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.953 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.953 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-07-31T15:15:26.953 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-07-31T15:15:26.954 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-31T15:15:26.954 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-31T15:15:26.954 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-31T15:15:26.954 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-31T15:15:26.954 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-31T15:15:26.954 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-31T15:15:26.954 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-31T15:15:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-31T15:15:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-31T15:15:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-31T15:15:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-31T15:15:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: "services": { 2024-07-31T15:15:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: "dashboard": "https://172.21.15.67:8443/" 2024-07-31T15:15:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: "always_on_modules": { 2024-07-31T15:15:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus": [ 2024-07-31T15:15:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-07-31T15:15:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-07-31T15:15:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-07-31T15:15:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-07-31T15:15:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-07-31T15:15:26.957 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-07-31T15:15:26.957 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-07-31T15:15:26.957 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-07-31T15:15:26.957 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-07-31T15:15:26.957 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-07-31T15:15:26.957 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.957 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific": [ 2024-07-31T15:15:26.957 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-07-31T15:15:26.958 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-07-31T15:15:26.958 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-07-31T15:15:26.958 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-07-31T15:15:26.958 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-07-31T15:15:26.958 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-07-31T15:15:26.958 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-07-31T15:15:26.958 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-07-31T15:15:26.959 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-07-31T15:15:26.959 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-07-31T15:15:26.959 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.959 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy": [ 2024-07-31T15:15:26.959 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-07-31T15:15:26.959 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-07-31T15:15:26.959 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-07-31T15:15:26.959 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-07-31T15:15:26.960 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-07-31T15:15:26.960 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-07-31T15:15:26.960 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-07-31T15:15:26.960 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-07-31T15:15:26.960 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-07-31T15:15:26.960 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-07-31T15:15:26.960 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.961 INFO:teuthology.orchestra.run.smithi067.stdout: "reef": [ 2024-07-31T15:15:26.961 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-07-31T15:15:26.961 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-07-31T15:15:26.961 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-07-31T15:15:26.961 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-07-31T15:15:26.961 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-07-31T15:15:26.961 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-07-31T15:15:26.961 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-07-31T15:15:26.962 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-07-31T15:15:26.962 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-07-31T15:15:26.962 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-07-31T15:15:26.962 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-31T15:15:26.962 INFO:teuthology.orchestra.run.smithi067.stdout: "squid": [ 2024-07-31T15:15:26.962 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-07-31T15:15:26.962 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-07-31T15:15:26.963 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-07-31T15:15:26.963 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-07-31T15:15:26.963 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-07-31T15:15:26.963 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-07-31T15:15:26.963 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-07-31T15:15:26.963 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-07-31T15:15:26.963 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-07-31T15:15:26.963 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-07-31T15:15:26.964 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-31T15:15:26.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.964 INFO:teuthology.orchestra.run.smithi067.stdout: "last_failure_osd_epoch": 5, 2024-07-31T15:15:26.964 INFO:teuthology.orchestra.run.smithi067.stdout: "active_clients": [ 2024-07-31T15:15:26.964 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.964 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-07-31T15:15:26.965 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-07-31T15:15:26.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.965 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-07-31T15:15:26.965 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-07-31T15:15:26.965 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 746517180 2024-07-31T15:15:26.965 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.965 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-31T15:15:26.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.966 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "libcephsqlite", 2024-07-31T15:15:26.966 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-07-31T15:15:26.966 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.966 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-07-31T15:15:26.966 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-07-31T15:15:26.966 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 370543409 2024-07-31T15:15:26.966 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.967 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-31T15:15:26.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.967 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.967 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-07-31T15:15:26.967 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-07-31T15:15:26.967 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.967 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-07-31T15:15:26.967 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-07-31T15:15:26.968 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1316333158 2024-07-31T15:15:26.968 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.968 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-31T15:15:26.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.968 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.968 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-07-31T15:15:26.968 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-07-31T15:15:26.969 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.969 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-07-31T15:15:26.969 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-07-31T15:15:26.969 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1833601243 2024-07-31T15:15:26.969 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.969 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-31T15:15:26.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.969 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.970 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-07-31T15:15:26.970 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-07-31T15:15:26.970 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.970 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-07-31T15:15:26.970 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-07-31T15:15:26.970 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 3305725611 2024-07-31T15:15:26.970 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.971 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-31T15:15:26.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:26.971 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.971 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-07-31T15:15:26.971 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-07-31T15:15:26.971 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-31T15:15:26.971 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-07-31T15:15:26.972 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-07-31T15:15:26.972 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 3221800371 2024-07-31T15:15:26.972 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.972 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-31T15:15:26.972 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:26.972 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-31T15:15:26.972 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:15:26.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.377+0000 7f5d70a77640 1 -- 172.21.15.67:0/2778832098 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5d540b3d40 msgr2=0x7f5d540b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:26.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.377+0000 7f5d70a77640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5d540b3d40 0x7f5d540b6250 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009c20 tx=0x7f5d5c005f90 comp rx=0 tx=0).stop 2024-07-31T15:15:26.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.377+0000 7f5d70a77640 1 -- 172.21.15.67:0/2778832098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d6c108680 msgr2=0x7f5d6c19f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:15:26.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.377+0000 7f5d70a77640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d6c108680 0x7f5d6c19f680 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400eb10 tx=0x7f5d6400c2d0 comp rx=0 tx=0).stop 2024-07-31T15:15:26.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.377+0000 7f5d70a77640 1 -- 172.21.15.67:0/2778832098 shutdown_connections 2024-07-31T15:15:26.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.377+0000 7f5d70a77640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f5d540b3d40 0x7f5d540b6250 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:26.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.377+0000 7f5d70a77640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d6c108680 0x7f5d6c19f680 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:26.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.377+0000 7f5d70a77640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d6c107d30 0x7f5d6c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:26.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.377+0000 7f5d70a77640 1 --2- 172.21.15.67:0/2778832098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d6c102400 0x7f5d6c199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:15:26.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.377+0000 7f5d70a77640 1 -- 172.21.15.67:0/2778832098 >> 172.21.15.67:0/2778832098 conn(0x7f5d6c0fdde0 msgr2=0x7f5d6c069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:15:26.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.377+0000 7f5d70a77640 1 -- 172.21.15.67:0/2778832098 shutdown_connections 2024-07-31T15:15:26.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:15:24.377+0000 7f5d70a77640 1 -- 172.21.15.67:0/2778832098 wait complete. 2024-07-31T15:15:26.986 DEBUG:tasks.check_counter:Getting stats from a 2024-07-31T15:15:26.986 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph daemon mds.a perf dump 2024-07-31T15:15:26.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:24 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3192077369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-31T15:15:26.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:24 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3434759285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-31T15:15:26.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:24 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2778832098' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-31T15:15:26.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:25 smithi067 ceph-mon[32750]: pgmap v1177: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-07-31T15:15:26.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:25 smithi067 ceph-mon[32750]: osdmap e138: 12 total, 12 up, 12 in 2024-07-31T15:15:26.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:26 smithi067 ceph-mon[32750]: pgmap v1179: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-07-31T15:15:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:24 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3192077369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-31T15:15:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:24 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3434759285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-31T15:15:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:24 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2778832098' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-31T15:15:26.990 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:25 smithi077 ceph-mon[37327]: pgmap v1177: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-07-31T15:15:26.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:25 smithi077 ceph-mon[37327]: osdmap e138: 12 total, 12 up, 12 in 2024-07-31T15:15:26.991 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:26 smithi077 ceph-mon[37327]: pgmap v1179: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-07-31T15:15:26.991 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:24 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3192077369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-31T15:15:26.991 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:24 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3434759285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-31T15:15:26.991 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:24 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2778832098' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-31T15:15:26.991 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:25 smithi073 ceph-mon[36714]: pgmap v1177: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-07-31T15:15:26.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:25 smithi073 ceph-mon[36714]: osdmap e138: 12 total, 12 up, 12 in 2024-07-31T15:15:26.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:26 smithi073 ceph-mon[36714]: pgmap v1179: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-07-31T15:15:27.268 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:15:27.599 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:27 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:15:27.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:27 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:15:27.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:27 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:15:28.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:28 smithi067 ceph-mon[32750]: pgmap v1180: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-31T15:15:28.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:28 smithi073 ceph-mon[36714]: pgmap v1180: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-31T15:15:28.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:28 smithi077 ceph-mon[37327]: pgmap v1180: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-31T15:15:29.340 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:15:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-07-31T15:15:29.340 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 2986, 2024-07-31T15:15:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 2986, 2024-07-31T15:15:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 13483835, 2024-07-31T15:15:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 948725, 2024-07-31T15:15:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 7, 2024-07-31T15:15:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 3, 2024-07-31T15:15:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.313353371, 2024-07-31T15:15:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.092720691, 2024-07-31T15:15:29.341 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.090902728, 2024-07-31T15:15:29.342 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.105123803, 2024-07-31T15:15:29.342 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:29.342 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 2985, 2024-07-31T15:15:29.342 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.264651292, 2024-07-31T15:15:29.342 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000088660 2024-07-31T15:15:29.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.342 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:29.343 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.343 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.343 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.343 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-07-31T15:15:29.343 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 448 2024-07-31T15:15:29.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.343 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-07-31T15:15:29.344 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 568, 2024-07-31T15:15:29.344 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 915, 2024-07-31T15:15:29.344 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 8968012, 2024-07-31T15:15:29.344 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 952424, 2024-07-31T15:15:29.344 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 8, 2024-07-31T15:15:29.344 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 5, 2024-07-31T15:15:29.344 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.087052776, 2024-07-31T15:15:29.344 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.036296752, 2024-07-31T15:15:29.345 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.024579053, 2024-07-31T15:15:29.345 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.018701295, 2024-07-31T15:15:29.345 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:29.345 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 913, 2024-07-31T15:15:29.345 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.056485005, 2024-07-31T15:15:29.345 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000061867 2024-07-31T15:15:29.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.346 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:29.346 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.346 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.346 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.346 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-31T15:15:29.346 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 772368 2024-07-31T15:15:29.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.347 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-07-31T15:15:29.347 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 1953, 2024-07-31T15:15:29.347 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 1907, 2024-07-31T15:15:29.347 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 10389924, 2024-07-31T15:15:29.347 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 736214, 2024-07-31T15:15:29.347 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-07-31T15:15:29.347 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 3, 2024-07-31T15:15:29.348 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.224849196, 2024-07-31T15:15:29.348 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.069712294, 2024-07-31T15:15:29.348 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.103999659, 2024-07-31T15:15:29.348 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.053346402, 2024-07-31T15:15:29.348 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:29.348 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 1906, 2024-07-31T15:15:29.348 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.138377613, 2024-07-31T15:15:29.348 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000072601 2024-07-31T15:15:29.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.349 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:29.349 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.349 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.349 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.349 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 1653344, 2024-07-31T15:15:29.349 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 291968 2024-07-31T15:15:29.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.350 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-07-31T15:15:29.350 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 1, 2024-07-31T15:15:29.350 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-07-31T15:15:29.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.350 INFO:teuthology.orchestra.run.smithi067.stdout: "finisher-MDSRank": { 2024-07-31T15:15:29.350 INFO:teuthology.orchestra.run.smithi067.stdout: "queue_len": 0, 2024-07-31T15:15:29.350 INFO:teuthology.orchestra.run.smithi067.stdout: "complete_latency": { 2024-07-31T15:15:29.351 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 8171, 2024-07-31T15:15:29.351 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.324261009, 2024-07-31T15:15:29.351 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000039684 2024-07-31T15:15:29.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:29.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.351 INFO:teuthology.orchestra.run.smithi067.stdout: "finisher-PurgeQueue": { 2024-07-31T15:15:29.351 INFO:teuthology.orchestra.run.smithi067.stdout: "queue_len": 0, 2024-07-31T15:15:29.351 INFO:teuthology.orchestra.run.smithi067.stdout: "complete_latency": { 2024-07-31T15:15:29.352 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.352 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.352 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.352 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:29.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.352 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": { 2024-07-31T15:15:29.352 INFO:teuthology.orchestra.run.smithi067.stdout: "request": 0, 2024-07-31T15:15:29.352 INFO:teuthology.orchestra.run.smithi067.stdout: "reply": 0, 2024-07-31T15:15:29.352 INFO:teuthology.orchestra.run.smithi067.stdout: "reply_latency": { 2024-07-31T15:15:29.353 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.353 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.353 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.353 INFO:teuthology.orchestra.run.smithi067.stdout: "slow_reply": 0, 2024-07-31T15:15:29.353 INFO:teuthology.orchestra.run.smithi067.stdout: "forward": 0, 2024-07-31T15:15:29.354 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_fetch_complete": 0, 2024-07-31T15:15:29.354 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_fetch_keys": 0, 2024-07-31T15:15:29.354 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_commit": 0, 2024-07-31T15:15:29.354 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_split": 0, 2024-07-31T15:15:29.354 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_merge": 0, 2024-07-31T15:15:29.354 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes": 7928, 2024-07-31T15:15:29.354 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_top": 62, 2024-07-31T15:15:29.354 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_bottom": 7866, 2024-07-31T15:15:29.355 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_pin_tail": 0, 2024-07-31T15:15:29.355 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_pinned": 7840, 2024-07-31T15:15:29.355 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_expired": 0, 2024-07-31T15:15:29.355 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_with_caps": 0, 2024-07-31T15:15:29.355 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": 0, 2024-07-31T15:15:29.355 INFO:teuthology.orchestra.run.smithi067.stdout: "subtrees": 175, 2024-07-31T15:15:29.355 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse": 0, 2024-07-31T15:15:29.355 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_hit": 0, 2024-07-31T15:15:29.356 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_forward": 0, 2024-07-31T15:15:29.356 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_discover": 0, 2024-07-31T15:15:29.356 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_dir_fetch": 0, 2024-07-31T15:15:29.356 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_remote_ino": 0, 2024-07-31T15:15:29.356 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_lock": 0, 2024-07-31T15:15:29.356 INFO:teuthology.orchestra.run.smithi067.stdout: "load_cent": 0, 2024-07-31T15:15:29.356 INFO:teuthology.orchestra.run.smithi067.stdout: "q": 0, 2024-07-31T15:15:29.356 INFO:teuthology.orchestra.run.smithi067.stdout: "exported": 0, 2024-07-31T15:15:29.357 INFO:teuthology.orchestra.run.smithi067.stdout: "exported_inodes": 0, 2024-07-31T15:15:29.357 INFO:teuthology.orchestra.run.smithi067.stdout: "imported": 0, 2024-07-31T15:15:29.357 INFO:teuthology.orchestra.run.smithi067.stdout: "imported_inodes": 0, 2024-07-31T15:15:29.357 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_dir_fetch": 0, 2024-07-31T15:15:29.357 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_backtrace_fetch": 0, 2024-07-31T15:15:29.357 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_peer_discover": 0, 2024-07-31T15:15:29.357 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rfiles": 0, 2024-07-31T15:15:29.357 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rbytes": 0, 2024-07-31T15:15:29.358 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rsnaps": 0, 2024-07-31T15:15:29.358 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_backtrace_fetch": 0, 2024-07-31T15:15:29.358 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_set_tag": 0, 2024-07-31T15:15:29.358 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_backtrace_repaired": 0, 2024-07-31T15:15:29.358 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_inotable_repaired": 0, 2024-07-31T15:15:29.358 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dir_inodes": 0, 2024-07-31T15:15:29.358 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dir_base_inodes": 0, 2024-07-31T15:15:29.359 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dirfrag_rstats": 0, 2024-07-31T15:15:29.359 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_file_inodes": 0, 2024-07-31T15:15:29.359 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_inode_file_caps": 0, 2024-07-31T15:15:29.359 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_revoke": 0, 2024-07-31T15:15:29.359 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_grant": 0, 2024-07-31T15:15:29.359 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_trunc": 0, 2024-07-31T15:15:29.359 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-31T15:15:29.359 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-31T15:15:29.360 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_caps": 0, 2024-07-31T15:15:29.360 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_caps_dirty": 0, 2024-07-31T15:15:29.360 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_cap_release": 0, 2024-07-31T15:15:29.360 INFO:teuthology.orchestra.run.smithi067.stdout: "process_request_cap_release": 0 2024-07-31T15:15:29.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.360 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache": { 2024-07-31T15:15:29.360 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_update": 0, 2024-07-31T15:15:29.360 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_update_receipt": 0, 2024-07-31T15:15:29.361 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_try_discover": 0, 2024-07-31T15:15:29.361 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_send_discover": 0, 2024-07-31T15:15:29.361 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_handle_discover": 0, 2024-07-31T15:15:29.361 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays": 0, 2024-07-31T15:15:29.361 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays_delayed": 0, 2024-07-31T15:15:29.361 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays_enqueuing": 0, 2024-07-31T15:15:29.361 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_created": 0, 2024-07-31T15:15:29.361 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_enqueued": 0, 2024-07-31T15:15:29.362 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_reintegrated": 0, 2024-07-31T15:15:29.362 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_migrated": 0, 2024-07-31T15:15:29.362 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_processing": 0, 2024-07-31T15:15:29.362 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_enqueued": 0, 2024-07-31T15:15:29.362 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_prioritized": 0, 2024-07-31T15:15:29.362 INFO:teuthology.orchestra.run.smithi067.stdout: "recovery_started": 0, 2024-07-31T15:15:29.362 INFO:teuthology.orchestra.run.smithi067.stdout: "recovery_completed": 0, 2024-07-31T15:15:29.362 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_quiesce_path": 0, 2024-07-31T15:15:29.363 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_quiesce_inode": 0, 2024-07-31T15:15:29.363 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_enqueue_scrub": 0, 2024-07-31T15:15:29.363 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_exportdir": 0, 2024-07-31T15:15:29.363 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_flush": 0, 2024-07-31T15:15:29.363 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_fragmentdir": 0, 2024-07-31T15:15:29.363 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_fragstats": 0, 2024-07-31T15:15:29.363 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_inodestats": 0 2024-07-31T15:15:29.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.364 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log": { 2024-07-31T15:15:29.364 INFO:teuthology.orchestra.run.smithi067.stdout: "evlrg": 0, 2024-07-31T15:15:29.364 INFO:teuthology.orchestra.run.smithi067.stdout: "evadd": 0, 2024-07-31T15:15:29.364 INFO:teuthology.orchestra.run.smithi067.stdout: "evex": 0, 2024-07-31T15:15:29.364 INFO:teuthology.orchestra.run.smithi067.stdout: "evtrm": 0, 2024-07-31T15:15:29.364 INFO:teuthology.orchestra.run.smithi067.stdout: "ev": 13171, 2024-07-31T15:15:29.364 INFO:teuthology.orchestra.run.smithi067.stdout: "evexg": 0, 2024-07-31T15:15:29.364 INFO:teuthology.orchestra.run.smithi067.stdout: "evexd": 0, 2024-07-31T15:15:29.365 INFO:teuthology.orchestra.run.smithi067.stdout: "segadd": 0, 2024-07-31T15:15:29.365 INFO:teuthology.orchestra.run.smithi067.stdout: "segex": 0, 2024-07-31T15:15:29.365 INFO:teuthology.orchestra.run.smithi067.stdout: "segtrm": 0, 2024-07-31T15:15:29.365 INFO:teuthology.orchestra.run.smithi067.stdout: "seg": 18, 2024-07-31T15:15:29.365 INFO:teuthology.orchestra.run.smithi067.stdout: "segmjr": 3, 2024-07-31T15:15:29.365 INFO:teuthology.orchestra.run.smithi067.stdout: "segexg": 0, 2024-07-31T15:15:29.365 INFO:teuthology.orchestra.run.smithi067.stdout: "segexd": 0, 2024-07-31T15:15:29.365 INFO:teuthology.orchestra.run.smithi067.stdout: "expos": 4194304, 2024-07-31T15:15:29.365 INFO:teuthology.orchestra.run.smithi067.stdout: "wrpos": 33987709, 2024-07-31T15:15:29.366 INFO:teuthology.orchestra.run.smithi067.stdout: "rdpos": 33986611, 2024-07-31T15:15:29.366 INFO:teuthology.orchestra.run.smithi067.stdout: "jlat": { 2024-07-31T15:15:29.366 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.366 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.366 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.366 INFO:teuthology.orchestra.run.smithi067.stdout: "replayed": 13171 2024-07-31T15:15:29.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.367 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_mem": { 2024-07-31T15:15:29.367 INFO:teuthology.orchestra.run.smithi067.stdout: "ino": 7830, 2024-07-31T15:15:29.367 INFO:teuthology.orchestra.run.smithi067.stdout: "ino+": 8035, 2024-07-31T15:15:29.367 INFO:teuthology.orchestra.run.smithi067.stdout: "ino-": 205, 2024-07-31T15:15:29.367 INFO:teuthology.orchestra.run.smithi067.stdout: "dir": 633, 2024-07-31T15:15:29.367 INFO:teuthology.orchestra.run.smithi067.stdout: "dir+": 655, 2024-07-31T15:15:29.367 INFO:teuthology.orchestra.run.smithi067.stdout: "dir-": 22, 2024-07-31T15:15:29.368 INFO:teuthology.orchestra.run.smithi067.stdout: "dn": 7928, 2024-07-31T15:15:29.368 INFO:teuthology.orchestra.run.smithi067.stdout: "dn+": 8081, 2024-07-31T15:15:29.368 INFO:teuthology.orchestra.run.smithi067.stdout: "dn-": 153, 2024-07-31T15:15:29.368 INFO:teuthology.orchestra.run.smithi067.stdout: "cap": 0, 2024-07-31T15:15:29.368 INFO:teuthology.orchestra.run.smithi067.stdout: "cap+": 0, 2024-07-31T15:15:29.368 INFO:teuthology.orchestra.run.smithi067.stdout: "cap-": 0, 2024-07-31T15:15:29.368 INFO:teuthology.orchestra.run.smithi067.stdout: "rss": 148144, 2024-07-31T15:15:29.368 INFO:teuthology.orchestra.run.smithi067.stdout: "heap": 215348 2024-07-31T15:15:29.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.369 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_server": { 2024-07-31T15:15:29.369 INFO:teuthology.orchestra.run.smithi067.stdout: "dispatch_client_request": 0, 2024-07-31T15:15:29.369 INFO:teuthology.orchestra.run.smithi067.stdout: "dispatch_server_request": 0, 2024-07-31T15:15:29.369 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_request": 0, 2024-07-31T15:15:29.369 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_session": 0, 2024-07-31T15:15:29.369 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_peer_request": 0, 2024-07-31T15:15:29.369 INFO:teuthology.orchestra.run.smithi067.stdout: "req_create_latency": { 2024-07-31T15:15:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getattr_latency": { 2024-07-31T15:15:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getfilelock_latency": { 2024-07-31T15:15:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "req_link_latency": { 2024-07-31T15:15:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookup_latency": { 2024-07-31T15:15:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookuphash_latency": { 2024-07-31T15:15:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupino_latency": { 2024-07-31T15:15:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupname_latency": { 2024-07-31T15:15:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupparent_latency": { 2024-07-31T15:15:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupsnap_latency": { 2024-07-31T15:15:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lssnap_latency": { 2024-07-31T15:15:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mkdir_latency": { 2024-07-31T15:15:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mknod_latency": { 2024-07-31T15:15:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mksnap_latency": { 2024-07-31T15:15:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: "req_open_latency": { 2024-07-31T15:15:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: "req_readdir_latency": { 2024-07-31T15:15:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rename_latency": { 2024-07-31T15:15:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: "req_renamesnap_latency": { 2024-07-31T15:15:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: "req_snapdiff_latency": { 2024-07-31T15:15:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmdir_latency": { 2024-07-31T15:15:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmsnap_latency": { 2024-07-31T15:15:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmxattr_latency": { 2024-07-31T15:15:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setattr_latency": { 2024-07-31T15:15:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setdirlayout_latency": { 2024-07-31T15:15:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.386 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setfilelock_latency": { 2024-07-31T15:15:29.386 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.386 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.386 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.386 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setlayout_latency": { 2024-07-31T15:15:29.387 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.387 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.387 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.387 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setxattr_latency": { 2024-07-31T15:15:29.387 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.387 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.387 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.388 INFO:teuthology.orchestra.run.smithi067.stdout: "req_symlink_latency": { 2024-07-31T15:15:29.388 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.388 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.388 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.388 INFO:teuthology.orchestra.run.smithi067.stdout: "req_unlink_latency": { 2024-07-31T15:15:29.388 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.389 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.389 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.389 INFO:teuthology.orchestra.run.smithi067.stdout: "cap_revoke_eviction": 0, 2024-07-31T15:15:29.389 INFO:teuthology.orchestra.run.smithi067.stdout: "cap_acquisition_throttle": 0, 2024-07-31T15:15:29.389 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getvxattr_latency": { 2024-07-31T15:15:29.389 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.389 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.389 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.390 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:29.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.390 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_sessions": { 2024-07-31T15:15:29.390 INFO:teuthology.orchestra.run.smithi067.stdout: "session_count": 2, 2024-07-31T15:15:29.390 INFO:teuthology.orchestra.run.smithi067.stdout: "session_add": 3, 2024-07-31T15:15:29.390 INFO:teuthology.orchestra.run.smithi067.stdout: "session_remove": 1, 2024-07-31T15:15:29.390 INFO:teuthology.orchestra.run.smithi067.stdout: "sessions_open": 3, 2024-07-31T15:15:29.391 INFO:teuthology.orchestra.run.smithi067.stdout: "sessions_stale": 0, 2024-07-31T15:15:29.391 INFO:teuthology.orchestra.run.smithi067.stdout: "total_load": 0, 2024-07-31T15:15:29.391 INFO:teuthology.orchestra.run.smithi067.stdout: "average_load": 0, 2024-07-31T15:15:29.391 INFO:teuthology.orchestra.run.smithi067.stdout: "avg_session_uptime": 3366, 2024-07-31T15:15:29.391 INFO:teuthology.orchestra.run.smithi067.stdout: "mdthresh_evicted": 0 2024-07-31T15:15:29.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.391 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-07-31T15:15:29.392 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-07-31T15:15:29.392 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-07-31T15:15:29.392 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-07-31T15:15:29.392 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-07-31T15:15:29.392 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-07-31T15:15:29.392 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-07-31T15:15:29.392 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-31T15:15:29.392 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-07-31T15:15:29.393 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-31T15:15:29.393 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-07-31T15:15:29.393 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-07-31T15:15:29.393 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-07-31T15:15:29.393 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-31T15:15:29.393 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-07-31T15:15:29.393 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-07-31T15:15:29.394 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-07-31T15:15:29.394 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-07-31T15:15:29.394 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-07-31T15:15:29.394 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-31T15:15:29.394 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-07-31T15:15:29.394 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-31T15:15:29.394 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-07-31T15:15:29.394 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-07-31T15:15:29.395 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-07-31T15:15:29.395 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-07-31T15:15:29.395 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-07-31T15:15:29.395 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-31T15:15:29.395 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-07-31T15:15:29.395 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-07-31T15:15:29.395 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-07-31T15:15:29.395 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-07-31T15:15:29.396 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-07-31T15:15:29.396 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-07-31T15:15:29.396 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-07-31T15:15:29.396 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-07-31T15:15:29.396 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-07-31T15:15:29.396 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 515053, 2024-07-31T15:15:29.396 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 8132, 2024-07-31T15:15:29.396 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-07-31T15:15:29.397 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-07-31T15:15:29.397 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-07-31T15:15:29.397 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-07-31T15:15:29.397 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-07-31T15:15:29.397 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-07-31T15:15:29.397 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-07-31T15:15:29.397 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-07-31T15:15:29.398 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 9352, 2024-07-31T15:15:29.398 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 189, 2024-07-31T15:15:29.398 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-07-31T15:15:29.398 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-07-31T15:15:29.398 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-07-31T15:15:29.398 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-07-31T15:15:29.398 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 24727542, 2024-07-31T15:15:29.398 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 264328, 2024-07-31T15:15:29.399 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-07-31T15:15:29.399 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-07-31T15:15:29.399 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-07-31T15:15:29.399 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-07-31T15:15:29.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.399 INFO:teuthology.orchestra.run.smithi067.stdout: "objecter": { 2024-07-31T15:15:29.399 INFO:teuthology.orchestra.run.smithi067.stdout: "op_active": 0, 2024-07-31T15:15:29.399 INFO:teuthology.orchestra.run.smithi067.stdout: "op_laggy": 0, 2024-07-31T15:15:29.400 INFO:teuthology.orchestra.run.smithi067.stdout: "op_send": 4945, 2024-07-31T15:15:29.400 INFO:teuthology.orchestra.run.smithi067.stdout: "op_send_bytes": 21, 2024-07-31T15:15:29.400 INFO:teuthology.orchestra.run.smithi067.stdout: "op_resend": 3, 2024-07-31T15:15:29.400 INFO:teuthology.orchestra.run.smithi067.stdout: "op_reply": 4945, 2024-07-31T15:15:29.400 INFO:teuthology.orchestra.run.smithi067.stdout: "op_latency": { 2024-07-31T15:15:29.400 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 4945, 2024-07-31T15:15:29.400 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 43.833090589, 2024-07-31T15:15:29.400 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.008864123 2024-07-31T15:15:29.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.401 INFO:teuthology.orchestra.run.smithi067.stdout: "op_inflight": 0, 2024-07-31T15:15:29.401 INFO:teuthology.orchestra.run.smithi067.stdout: "oplen_avg": { 2024-07-31T15:15:29.401 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 4945, 2024-07-31T15:15:29.401 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 4946 2024-07-31T15:15:29.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.401 INFO:teuthology.orchestra.run.smithi067.stdout: "op": 4945, 2024-07-31T15:15:29.402 INFO:teuthology.orchestra.run.smithi067.stdout: "op_r": 4945, 2024-07-31T15:15:29.402 INFO:teuthology.orchestra.run.smithi067.stdout: "op_w": 0, 2024-07-31T15:15:29.402 INFO:teuthology.orchestra.run.smithi067.stdout: "op_rmw": 0, 2024-07-31T15:15:29.402 INFO:teuthology.orchestra.run.smithi067.stdout: "op_pg": 0, 2024-07-31T15:15:29.402 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_stat": 3026, 2024-07-31T15:15:29.402 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_create": 0, 2024-07-31T15:15:29.402 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_read": 1917, 2024-07-31T15:15:29.402 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_write": 0, 2024-07-31T15:15:29.403 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_writefull": 0, 2024-07-31T15:15:29.403 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_writesame": 0, 2024-07-31T15:15:29.403 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_append": 0, 2024-07-31T15:15:29.403 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_zero": 0, 2024-07-31T15:15:29.403 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_truncate": 0, 2024-07-31T15:15:29.403 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_delete": 0, 2024-07-31T15:15:29.403 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_mapext": 0, 2024-07-31T15:15:29.403 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_sparse_read": 0, 2024-07-31T15:15:29.404 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_clonerange": 0, 2024-07-31T15:15:29.404 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_getxattr": 1, 2024-07-31T15:15:29.404 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_setxattr": 0, 2024-07-31T15:15:29.404 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_cmpxattr": 0, 2024-07-31T15:15:29.404 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_rmxattr": 0, 2024-07-31T15:15:29.404 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_resetxattrs": 0, 2024-07-31T15:15:29.404 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_call": 0, 2024-07-31T15:15:29.404 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_watch": 0, 2024-07-31T15:15:29.405 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_notify": 0, 2024-07-31T15:15:29.405 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_src_cmpxattr": 0, 2024-07-31T15:15:29.405 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_pgls": 0, 2024-07-31T15:15:29.405 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_pgls_filter": 0, 2024-07-31T15:15:29.405 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_other": 0, 2024-07-31T15:15:29.405 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_active": 0, 2024-07-31T15:15:29.405 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_send": 0, 2024-07-31T15:15:29.405 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_resend": 0, 2024-07-31T15:15:29.406 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_ping": 0, 2024-07-31T15:15:29.406 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_active": 0, 2024-07-31T15:15:29.406 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_send": 0, 2024-07-31T15:15:29.406 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_resend": 0, 2024-07-31T15:15:29.406 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_active": 0, 2024-07-31T15:15:29.406 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_send": 0, 2024-07-31T15:15:29.406 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_resend": 0, 2024-07-31T15:15:29.407 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_active": 0, 2024-07-31T15:15:29.407 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_send": 0, 2024-07-31T15:15:29.407 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_resend": 0, 2024-07-31T15:15:29.407 INFO:teuthology.orchestra.run.smithi067.stdout: "command_active": 0, 2024-07-31T15:15:29.407 INFO:teuthology.orchestra.run.smithi067.stdout: "command_send": 0, 2024-07-31T15:15:29.407 INFO:teuthology.orchestra.run.smithi067.stdout: "command_resend": 0, 2024-07-31T15:15:29.407 INFO:teuthology.orchestra.run.smithi067.stdout: "map_epoch": 138, 2024-07-31T15:15:29.407 INFO:teuthology.orchestra.run.smithi067.stdout: "map_full": 0, 2024-07-31T15:15:29.408 INFO:teuthology.orchestra.run.smithi067.stdout: "map_inc": 49, 2024-07-31T15:15:29.408 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_sessions": 8, 2024-07-31T15:15:29.408 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_session_open": 10, 2024-07-31T15:15:29.408 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_session_close": 2, 2024-07-31T15:15:29.408 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_laggy": 0, 2024-07-31T15:15:29.408 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_wr": 0, 2024-07-31T15:15:29.408 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_rd": 2, 2024-07-31T15:15:29.408 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_del": 0 2024-07-31T15:15:29.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.409 INFO:teuthology.orchestra.run.smithi067.stdout: "oft": { 2024-07-31T15:15:29.409 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_objs": 0, 2024-07-31T15:15:29.409 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_kv_pairs": 0, 2024-07-31T15:15:29.409 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_updates": 0, 2024-07-31T15:15:29.409 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_removes": 0 2024-07-31T15:15:29.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.409 INFO:teuthology.orchestra.run.smithi067.stdout: "osd-slow-ops": { 2024-07-31T15:15:29.409 INFO:teuthology.orchestra.run.smithi067.stdout: "slow_ops_count": 0 2024-07-31T15:15:29.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.410 INFO:teuthology.orchestra.run.smithi067.stdout: "purge_queue": { 2024-07-31T15:15:29.410 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_ops": 0, 2024-07-31T15:15:29.410 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_ops_high_water": 0, 2024-07-31T15:15:29.410 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing": 0, 2024-07-31T15:15:29.410 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_high_water": 0, 2024-07-31T15:15:29.410 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executed_ops": 0, 2024-07-31T15:15:29.410 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executed": 0, 2024-07-31T15:15:29.411 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_item_in_journal": 0 2024-07-31T15:15:29.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.411 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-31T15:15:29.411 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-31T15:15:29.411 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-07-31T15:15:29.411 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-31T15:15:29.411 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 5503, 2024-07-31T15:15:29.411 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 32315326, 2024-07-31T15:15:29.412 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:29.412 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 5503, 2024-07-31T15:15:29.412 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-31T15:15:29.412 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-31T15:15:29.412 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 5503, 2024-07-31T15:15:29.412 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 32315326, 2024-07-31T15:15:29.412 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-31T15:15:29.412 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.412 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.413 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.413 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:29.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.413 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-objecter_bytes": { 2024-07-31T15:15:29.413 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-31T15:15:29.413 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-07-31T15:15:29.413 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-31T15:15:29.413 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-31T15:15:29.414 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-31T15:15:29.414 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:29.414 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:29.414 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 4945, 2024-07-31T15:15:29.414 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 29796899, 2024-07-31T15:15:29.414 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 4945, 2024-07-31T15:15:29.414 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 29796899, 2024-07-31T15:15:29.414 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-31T15:15:29.415 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.415 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.415 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.415 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:29.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.415 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-objecter_ops": { 2024-07-31T15:15:29.415 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-31T15:15:29.415 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1024, 2024-07-31T15:15:29.416 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-31T15:15:29.416 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-31T15:15:29.416 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-31T15:15:29.416 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:29.416 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:29.416 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 4945, 2024-07-31T15:15:29.416 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 4945, 2024-07-31T15:15:29.417 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 4945, 2024-07-31T15:15:29.417 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 4945, 2024-07-31T15:15:29.417 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-31T15:15:29.417 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.417 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.418 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.418 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:29.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.418 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-write_buf_throttle": { 2024-07-31T15:15:29.418 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-31T15:15:29.418 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 3758096384, 2024-07-31T15:15:29.418 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-31T15:15:29.418 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-31T15:15:29.419 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-31T15:15:29.419 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:29.419 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:29.419 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-31T15:15:29.419 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-31T15:15:29.419 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 0, 2024-07-31T15:15:29.419 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 0, 2024-07-31T15:15:29.420 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-31T15:15:29.420 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.420 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.420 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.420 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:29.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:29.420 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-write_buf_throttle-0x5598210f0c30": { 2024-07-31T15:15:29.420 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-31T15:15:29.421 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 3758096384, 2024-07-31T15:15:29.421 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-31T15:15:29.421 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-31T15:15:29.421 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-31T15:15:29.421 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:29.421 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:29.421 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-31T15:15:29.421 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-31T15:15:29.421 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 0, 2024-07-31T15:15:29.422 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 0, 2024-07-31T15:15:29.422 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-31T15:15:29.422 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:29.422 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:29.422 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:29.422 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:29.422 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:29.423 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:15:31.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:31 smithi073 ceph-mon[36714]: pgmap v1181: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-31T15:15:31.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:31 smithi067 ceph-mon[32750]: pgmap v1181: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-31T15:15:31.469 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-07-31T15:15:31.469 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-07-31T15:15:31.469 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-07-31T15:15:31.469 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-07-31T15:15:31.469 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-07-31T15:15:31.470 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-07-31T15:15:31.470 DEBUG:tasks.check_counter:Getting stats from d 2024-07-31T15:15:31.470 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph daemon mds.d perf dump 2024-07-31T15:15:31.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:31 smithi077 ceph-mon[37327]: pgmap v1181: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-31T15:15:32.050 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:15:33.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:33 smithi067 ceph-mon[32750]: pgmap v1182: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-31T15:15:33.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:33 smithi073 ceph-mon[36714]: pgmap v1182: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-31T15:15:33.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:33 smithi077 ceph-mon[37327]: pgmap v1182: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-31T15:15:34.772 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:15:34.773 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-07-31T15:15:34.773 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 474, 2024-07-31T15:15:34.773 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 464, 2024-07-31T15:15:34.773 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 1347888, 2024-07-31T15:15:34.773 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 286800, 2024-07-31T15:15:34.774 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 4, 2024-07-31T15:15:34.774 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 0, 2024-07-31T15:15:34.774 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.084615107, 2024-07-31T15:15:34.774 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.032177450, 2024-07-31T15:15:34.774 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.063032325, 2024-07-31T15:15:34.774 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-31T15:15:34.774 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:34.775 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 464, 2024-07-31T15:15:34.775 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.042417587, 2024-07-31T15:15:34.775 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000091417 2024-07-31T15:15:34.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:34.775 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:34.775 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:34.775 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 1347888, 2024-07-31T15:15:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 286800 2024-07-31T15:15:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-07-31T15:15:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 2, 2024-07-31T15:15:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 354, 2024-07-31T15:15:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 320, 2024-07-31T15:15:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 221408, 2024-07-31T15:15:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 4, 2024-07-31T15:15:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 2, 2024-07-31T15:15:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.028689527, 2024-07-31T15:15:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.022123883, 2024-07-31T15:15:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.002156344, 2024-07-31T15:15:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-31T15:15:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 352, 2024-07-31T15:15:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.035520468, 2024-07-31T15:15:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000100910 2024-07-31T15:15:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-31T15:15:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 221408 2024-07-31T15:15:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-07-31T15:15:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 0, 2024-07-31T15:15:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 2, 2024-07-31T15:15:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 0, 2024-07-31T15:15:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 0, 2024-07-31T15:15:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 2, 2024-07-31T15:15:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 1, 2024-07-31T15:15:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.000531307, 2024-07-31T15:15:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.000000000, 2024-07-31T15:15:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.000431707, 2024-07-31T15:15:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-31T15:15:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-31T15:15:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 0 2024-07-31T15:15:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-07-31T15:15:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 0, 2024-07-31T15:15:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-07-31T15:15:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-07-31T15:15:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-07-31T15:15:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-07-31T15:15:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-07-31T15:15:34.784 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-07-31T15:15:34.784 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-07-31T15:15:34.784 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-07-31T15:15:34.784 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-31T15:15:34.784 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-07-31T15:15:34.784 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-31T15:15:34.784 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-07-31T15:15:34.784 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-07-31T15:15:34.785 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-07-31T15:15:34.785 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-31T15:15:34.785 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-07-31T15:15:34.785 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-07-31T15:15:34.785 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-07-31T15:15:34.785 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-07-31T15:15:34.785 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-07-31T15:15:34.785 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-31T15:15:34.786 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-07-31T15:15:34.786 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-31T15:15:34.786 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-07-31T15:15:34.786 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-07-31T15:15:34.786 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-07-31T15:15:34.786 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-07-31T15:15:34.786 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-07-31T15:15:34.787 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-31T15:15:34.787 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-07-31T15:15:34.787 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-07-31T15:15:34.787 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-07-31T15:15:34.787 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-07-31T15:15:34.787 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-07-31T15:15:34.787 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-07-31T15:15:34.787 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-07-31T15:15:34.788 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-07-31T15:15:34.788 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-07-31T15:15:34.788 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 148721, 2024-07-31T15:15:34.788 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 30, 2024-07-31T15:15:34.788 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-07-31T15:15:34.788 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-07-31T15:15:34.788 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-07-31T15:15:34.789 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-07-31T15:15:34.789 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-07-31T15:15:34.789 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-07-31T15:15:34.789 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-07-31T15:15:34.789 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-07-31T15:15:34.789 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 0, 2024-07-31T15:15:34.789 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 0, 2024-07-31T15:15:34.789 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-07-31T15:15:34.790 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-07-31T15:15:34.790 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-07-31T15:15:34.790 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-07-31T15:15:34.790 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 928, 2024-07-31T15:15:34.790 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 2, 2024-07-31T15:15:34.790 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-07-31T15:15:34.791 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-07-31T15:15:34.791 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-07-31T15:15:34.791 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-07-31T15:15:34.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:34.791 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-31T15:15:34.791 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-31T15:15:34.791 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-07-31T15:15:34.791 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-31T15:15:34.792 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 472, 2024-07-31T15:15:34.792 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 1278433, 2024-07-31T15:15:34.792 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:34.792 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 472, 2024-07-31T15:15:34.792 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-31T15:15:34.792 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-31T15:15:34.792 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 472, 2024-07-31T15:15:34.792 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 1278433, 2024-07-31T15:15:34.793 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-31T15:15:34.793 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:34.793 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:34.793 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:34.793 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:34.793 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:34.793 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:15:35.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:15:35.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:15:35.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:15:35.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:15:35.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:15:35.054 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:34 smithi067 ceph-mon[32750]: pgmap v1183: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-31T15:15:35.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:15:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:15:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:15:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:15:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:15:35.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:34 smithi073 ceph-mon[36714]: pgmap v1183: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-31T15:15:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:15:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:15:35.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:15:35.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:15:35.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:15:35.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:34 smithi077 ceph-mon[37327]: pgmap v1183: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-31T15:15:35.675 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-07-31T15:15:35.675 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-07-31T15:15:35.675 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-07-31T15:15:35.675 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-07-31T15:15:35.675 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-07-31T15:15:35.676 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-07-31T15:15:35.676 DEBUG:tasks.check_counter:Getting stats from g 2024-07-31T15:15:35.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph daemon mds.g perf dump 2024-07-31T15:15:35.945 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:15:37.392 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:37 smithi067 ceph-mon[32750]: pgmap v1184: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-31T15:15:37.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:37 smithi073 ceph-mon[36714]: pgmap v1184: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-31T15:15:37.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:37 smithi077 ceph-mon[37327]: pgmap v1184: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-31T15:15:37.795 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:15:37.795 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-07-31T15:15:37.795 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 2254, 2024-07-31T15:15:37.796 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 2205, 2024-07-31T15:15:37.796 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 6408670, 2024-07-31T15:15:37.796 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 831756, 2024-07-31T15:15:37.796 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-07-31T15:15:37.796 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 1, 2024-07-31T15:15:37.796 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.299161394, 2024-07-31T15:15:37.796 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.112605302, 2024-07-31T15:15:37.796 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.115463795, 2024-07-31T15:15:37.797 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.068794502, 2024-07-31T15:15:37.797 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:37.797 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 2205, 2024-07-31T15:15:37.797 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.264695449, 2024-07-31T15:15:37.797 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000120043 2024-07-31T15:15:37.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.797 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:37.798 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.798 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.798 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.798 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 1657152, 2024-07-31T15:15:37.798 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 294336 2024-07-31T15:15:37.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.799 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-07-31T15:15:37.799 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 2171, 2024-07-31T15:15:37.799 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 2519, 2024-07-31T15:15:37.799 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 17324916, 2024-07-31T15:15:37.799 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 1374951, 2024-07-31T15:15:37.799 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-07-31T15:15:37.799 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 4, 2024-07-31T15:15:37.799 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.217214812, 2024-07-31T15:15:37.800 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.065194568, 2024-07-31T15:15:37.800 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.060763747, 2024-07-31T15:15:37.800 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.072993710, 2024-07-31T15:15:37.800 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:37.800 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 2517, 2024-07-31T15:15:37.800 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.147691223, 2024-07-31T15:15:37.800 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000058677 2024-07-31T15:15:37.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.801 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:37.801 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.801 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.801 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.801 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-31T15:15:37.801 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 685152 2024-07-31T15:15:37.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.802 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-07-31T15:15:37.802 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 959, 2024-07-31T15:15:37.802 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 961, 2024-07-31T15:15:37.802 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 2383514, 2024-07-31T15:15:37.803 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 304974, 2024-07-31T15:15:37.803 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 6, 2024-07-31T15:15:37.803 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 4, 2024-07-31T15:15:37.803 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.088572630, 2024-07-31T15:15:37.803 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.024987177, 2024-07-31T15:15:37.803 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.025260876, 2024-07-31T15:15:37.803 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.028260451, 2024-07-31T15:15:37.803 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:37.803 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 959, 2024-07-31T15:15:37.804 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.081278644, 2024-07-31T15:15:37.804 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000084753 2024-07-31T15:15:37.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.804 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:37.804 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.804 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.804 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.805 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-31T15:15:37.805 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 0 2024-07-31T15:15:37.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.805 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-07-31T15:15:37.805 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 1, 2024-07-31T15:15:37.805 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-07-31T15:15:37.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.805 INFO:teuthology.orchestra.run.smithi067.stdout: "finisher-MDSRank": { 2024-07-31T15:15:37.806 INFO:teuthology.orchestra.run.smithi067.stdout: "queue_len": 0, 2024-07-31T15:15:37.806 INFO:teuthology.orchestra.run.smithi067.stdout: "complete_latency": { 2024-07-31T15:15:37.806 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 7961, 2024-07-31T15:15:37.806 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.299983350, 2024-07-31T15:15:37.806 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000037681 2024-07-31T15:15:37.806 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:37.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.807 INFO:teuthology.orchestra.run.smithi067.stdout: "finisher-PurgeQueue": { 2024-07-31T15:15:37.807 INFO:teuthology.orchestra.run.smithi067.stdout: "queue_len": 0, 2024-07-31T15:15:37.807 INFO:teuthology.orchestra.run.smithi067.stdout: "complete_latency": { 2024-07-31T15:15:37.807 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.807 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.807 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.807 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:37.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.808 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": { 2024-07-31T15:15:37.808 INFO:teuthology.orchestra.run.smithi067.stdout: "request": 0, 2024-07-31T15:15:37.808 INFO:teuthology.orchestra.run.smithi067.stdout: "reply": 0, 2024-07-31T15:15:37.808 INFO:teuthology.orchestra.run.smithi067.stdout: "reply_latency": { 2024-07-31T15:15:37.808 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.808 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.809 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.809 INFO:teuthology.orchestra.run.smithi067.stdout: "slow_reply": 0, 2024-07-31T15:15:37.809 INFO:teuthology.orchestra.run.smithi067.stdout: "forward": 0, 2024-07-31T15:15:37.809 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_fetch_complete": 0, 2024-07-31T15:15:37.809 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_fetch_keys": 0, 2024-07-31T15:15:37.809 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_commit": 0, 2024-07-31T15:15:37.809 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_split": 0, 2024-07-31T15:15:37.810 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_merge": 0, 2024-07-31T15:15:37.810 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes": 7778, 2024-07-31T15:15:37.810 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_top": 1168, 2024-07-31T15:15:37.810 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_bottom": 6610, 2024-07-31T15:15:37.810 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_pin_tail": 0, 2024-07-31T15:15:37.810 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_pinned": 6110, 2024-07-31T15:15:37.810 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_expired": 0, 2024-07-31T15:15:37.811 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_with_caps": 0, 2024-07-31T15:15:37.811 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": 0, 2024-07-31T15:15:37.811 INFO:teuthology.orchestra.run.smithi067.stdout: "subtrees": 155, 2024-07-31T15:15:37.811 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse": 0, 2024-07-31T15:15:37.811 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_hit": 0, 2024-07-31T15:15:37.811 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_forward": 0, 2024-07-31T15:15:37.811 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_discover": 0, 2024-07-31T15:15:37.812 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_dir_fetch": 0, 2024-07-31T15:15:37.812 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_remote_ino": 0, 2024-07-31T15:15:37.812 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_lock": 0, 2024-07-31T15:15:37.812 INFO:teuthology.orchestra.run.smithi067.stdout: "load_cent": 0, 2024-07-31T15:15:37.812 INFO:teuthology.orchestra.run.smithi067.stdout: "q": 0, 2024-07-31T15:15:37.812 INFO:teuthology.orchestra.run.smithi067.stdout: "exported": 0, 2024-07-31T15:15:37.812 INFO:teuthology.orchestra.run.smithi067.stdout: "exported_inodes": 0, 2024-07-31T15:15:37.812 INFO:teuthology.orchestra.run.smithi067.stdout: "imported": 0, 2024-07-31T15:15:37.813 INFO:teuthology.orchestra.run.smithi067.stdout: "imported_inodes": 0, 2024-07-31T15:15:37.813 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_dir_fetch": 0, 2024-07-31T15:15:37.813 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_backtrace_fetch": 0, 2024-07-31T15:15:37.813 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_peer_discover": 0, 2024-07-31T15:15:37.813 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rfiles": 0, 2024-07-31T15:15:37.813 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rbytes": 0, 2024-07-31T15:15:37.813 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rsnaps": 0, 2024-07-31T15:15:37.814 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_backtrace_fetch": 0, 2024-07-31T15:15:37.814 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_set_tag": 0, 2024-07-31T15:15:37.814 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_backtrace_repaired": 0, 2024-07-31T15:15:37.814 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_inotable_repaired": 0, 2024-07-31T15:15:37.814 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dir_inodes": 0, 2024-07-31T15:15:37.814 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dir_base_inodes": 0, 2024-07-31T15:15:37.814 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dirfrag_rstats": 0, 2024-07-31T15:15:37.814 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_file_inodes": 0, 2024-07-31T15:15:37.815 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_inode_file_caps": 0, 2024-07-31T15:15:37.815 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_revoke": 0, 2024-07-31T15:15:37.815 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_grant": 0, 2024-07-31T15:15:37.815 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_trunc": 0, 2024-07-31T15:15:37.815 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-31T15:15:37.815 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-31T15:15:37.815 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_caps": 0, 2024-07-31T15:15:37.815 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_caps_dirty": 0, 2024-07-31T15:15:37.816 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_cap_release": 0, 2024-07-31T15:15:37.816 INFO:teuthology.orchestra.run.smithi067.stdout: "process_request_cap_release": 0 2024-07-31T15:15:37.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.816 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache": { 2024-07-31T15:15:37.816 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_update": 0, 2024-07-31T15:15:37.816 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_update_receipt": 0, 2024-07-31T15:15:37.816 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_try_discover": 0, 2024-07-31T15:15:37.817 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_send_discover": 0, 2024-07-31T15:15:37.817 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_handle_discover": 0, 2024-07-31T15:15:37.817 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays": 0, 2024-07-31T15:15:37.817 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays_delayed": 0, 2024-07-31T15:15:37.817 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays_enqueuing": 0, 2024-07-31T15:15:37.817 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_created": 0, 2024-07-31T15:15:37.817 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_enqueued": 0, 2024-07-31T15:15:37.818 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_reintegrated": 0, 2024-07-31T15:15:37.818 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_migrated": 0, 2024-07-31T15:15:37.818 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_processing": 0, 2024-07-31T15:15:37.818 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_enqueued": 0, 2024-07-31T15:15:37.818 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_prioritized": 0, 2024-07-31T15:15:37.818 INFO:teuthology.orchestra.run.smithi067.stdout: "recovery_started": 0, 2024-07-31T15:15:37.818 INFO:teuthology.orchestra.run.smithi067.stdout: "recovery_completed": 0, 2024-07-31T15:15:37.819 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_quiesce_path": 0, 2024-07-31T15:15:37.819 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_quiesce_inode": 0, 2024-07-31T15:15:37.819 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_enqueue_scrub": 0, 2024-07-31T15:15:37.819 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_exportdir": 0, 2024-07-31T15:15:37.819 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_flush": 0, 2024-07-31T15:15:37.819 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_fragmentdir": 0, 2024-07-31T15:15:37.819 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_fragstats": 0, 2024-07-31T15:15:37.819 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_inodestats": 0 2024-07-31T15:15:37.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.820 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log": { 2024-07-31T15:15:37.820 INFO:teuthology.orchestra.run.smithi067.stdout: "evlrg": 0, 2024-07-31T15:15:37.820 INFO:teuthology.orchestra.run.smithi067.stdout: "evadd": 0, 2024-07-31T15:15:37.820 INFO:teuthology.orchestra.run.smithi067.stdout: "evex": 0, 2024-07-31T15:15:37.820 INFO:teuthology.orchestra.run.smithi067.stdout: "evtrm": 0, 2024-07-31T15:15:37.820 INFO:teuthology.orchestra.run.smithi067.stdout: "ev": 7090, 2024-07-31T15:15:37.820 INFO:teuthology.orchestra.run.smithi067.stdout: "evexg": 0, 2024-07-31T15:15:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "evexd": 0, 2024-07-31T15:15:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "segadd": 0, 2024-07-31T15:15:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "segex": 0, 2024-07-31T15:15:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "segtrm": 0, 2024-07-31T15:15:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "seg": 11, 2024-07-31T15:15:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "segmjr": 2, 2024-07-31T15:15:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "segexg": 0, 2024-07-31T15:15:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "segexd": 0, 2024-07-31T15:15:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "expos": 4194304, 2024-07-31T15:15:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "wrpos": 27299883, 2024-07-31T15:15:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "rdpos": 27299284, 2024-07-31T15:15:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "jlat": { 2024-07-31T15:15:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "replayed": 7090 2024-07-31T15:15:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_mem": { 2024-07-31T15:15:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "ino": 7669, 2024-07-31T15:15:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "ino+": 7802, 2024-07-31T15:15:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "ino-": 133, 2024-07-31T15:15:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "dir": 817, 2024-07-31T15:15:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "dir+": 846, 2024-07-31T15:15:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "dir-": 29, 2024-07-31T15:15:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "dn": 7778, 2024-07-31T15:15:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "dn+": 7854, 2024-07-31T15:15:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "dn-": 76, 2024-07-31T15:15:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "cap": 0, 2024-07-31T15:15:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "cap+": 0, 2024-07-31T15:15:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "cap-": 0, 2024-07-31T15:15:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "rss": 148596, 2024-07-31T15:15:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "heap": 215348 2024-07-31T15:15:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_server": { 2024-07-31T15:15:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "dispatch_client_request": 0, 2024-07-31T15:15:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "dispatch_server_request": 0, 2024-07-31T15:15:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_request": 0, 2024-07-31T15:15:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_session": 0, 2024-07-31T15:15:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_peer_request": 0, 2024-07-31T15:15:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "req_create_latency": { 2024-07-31T15:15:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getattr_latency": { 2024-07-31T15:15:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getfilelock_latency": { 2024-07-31T15:15:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.828 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.828 INFO:teuthology.orchestra.run.smithi067.stdout: "req_link_latency": { 2024-07-31T15:15:37.828 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.828 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.828 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.828 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookup_latency": { 2024-07-31T15:15:37.829 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.829 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.829 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.829 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookuphash_latency": { 2024-07-31T15:15:37.829 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.829 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.829 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.830 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupino_latency": { 2024-07-31T15:15:37.830 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.830 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.830 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.830 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupname_latency": { 2024-07-31T15:15:37.830 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.831 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.831 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.831 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupparent_latency": { 2024-07-31T15:15:37.831 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.831 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.831 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupsnap_latency": { 2024-07-31T15:15:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lssnap_latency": { 2024-07-31T15:15:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mkdir_latency": { 2024-07-31T15:15:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mknod_latency": { 2024-07-31T15:15:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mksnap_latency": { 2024-07-31T15:15:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "req_open_latency": { 2024-07-31T15:15:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "req_readdir_latency": { 2024-07-31T15:15:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rename_latency": { 2024-07-31T15:15:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "req_renamesnap_latency": { 2024-07-31T15:15:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "req_snapdiff_latency": { 2024-07-31T15:15:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmdir_latency": { 2024-07-31T15:15:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmsnap_latency": { 2024-07-31T15:15:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmxattr_latency": { 2024-07-31T15:15:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setattr_latency": { 2024-07-31T15:15:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setdirlayout_latency": { 2024-07-31T15:15:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setfilelock_latency": { 2024-07-31T15:15:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setlayout_latency": { 2024-07-31T15:15:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setxattr_latency": { 2024-07-31T15:15:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.844 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.844 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.844 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.844 INFO:teuthology.orchestra.run.smithi067.stdout: "req_symlink_latency": { 2024-07-31T15:15:37.844 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.844 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.844 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.845 INFO:teuthology.orchestra.run.smithi067.stdout: "req_unlink_latency": { 2024-07-31T15:15:37.845 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.845 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.845 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.845 INFO:teuthology.orchestra.run.smithi067.stdout: "cap_revoke_eviction": 0, 2024-07-31T15:15:37.845 INFO:teuthology.orchestra.run.smithi067.stdout: "cap_acquisition_throttle": 0, 2024-07-31T15:15:37.846 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getvxattr_latency": { 2024-07-31T15:15:37.846 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.846 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.846 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.846 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:37.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.846 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_sessions": { 2024-07-31T15:15:37.847 INFO:teuthology.orchestra.run.smithi067.stdout: "session_count": 1, 2024-07-31T15:15:37.847 INFO:teuthology.orchestra.run.smithi067.stdout: "session_add": 2, 2024-07-31T15:15:37.847 INFO:teuthology.orchestra.run.smithi067.stdout: "session_remove": 1, 2024-07-31T15:15:37.847 INFO:teuthology.orchestra.run.smithi067.stdout: "sessions_open": 1, 2024-07-31T15:15:37.847 INFO:teuthology.orchestra.run.smithi067.stdout: "sessions_stale": 0, 2024-07-31T15:15:37.847 INFO:teuthology.orchestra.run.smithi067.stdout: "total_load": 0, 2024-07-31T15:15:37.847 INFO:teuthology.orchestra.run.smithi067.stdout: "average_load": 0, 2024-07-31T15:15:37.848 INFO:teuthology.orchestra.run.smithi067.stdout: "avg_session_uptime": 2736, 2024-07-31T15:15:37.848 INFO:teuthology.orchestra.run.smithi067.stdout: "mdthresh_evicted": 0 2024-07-31T15:15:37.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.848 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-07-31T15:15:37.848 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-07-31T15:15:37.848 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-07-31T15:15:37.848 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-07-31T15:15:37.848 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-07-31T15:15:37.849 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-07-31T15:15:37.849 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-07-31T15:15:37.849 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-31T15:15:37.849 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-07-31T15:15:37.849 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-31T15:15:37.849 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-07-31T15:15:37.849 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-07-31T15:15:37.849 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-07-31T15:15:37.850 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-31T15:15:37.850 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-07-31T15:15:37.850 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-07-31T15:15:37.850 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-07-31T15:15:37.850 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-07-31T15:15:37.850 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-07-31T15:15:37.850 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-31T15:15:37.850 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-07-31T15:15:37.851 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-31T15:15:37.851 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-07-31T15:15:37.851 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-07-31T15:15:37.851 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-07-31T15:15:37.851 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-07-31T15:15:37.851 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-07-31T15:15:37.851 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-31T15:15:37.852 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-07-31T15:15:37.852 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-07-31T15:15:37.852 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-07-31T15:15:37.852 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-07-31T15:15:37.852 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-07-31T15:15:37.853 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-07-31T15:15:37.853 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-07-31T15:15:37.853 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-07-31T15:15:37.853 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-07-31T15:15:37.853 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 505179, 2024-07-31T15:15:37.853 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 8352, 2024-07-31T15:15:37.853 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-07-31T15:15:37.854 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-07-31T15:15:37.854 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-07-31T15:15:37.854 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-07-31T15:15:37.854 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-07-31T15:15:37.854 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-07-31T15:15:37.854 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-07-31T15:15:37.854 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-07-31T15:15:37.854 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 9352, 2024-07-31T15:15:37.855 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 189, 2024-07-31T15:15:37.855 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-07-31T15:15:37.855 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-07-31T15:15:37.855 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-07-31T15:15:37.855 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-07-31T15:15:37.855 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 24811846, 2024-07-31T15:15:37.855 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 274016, 2024-07-31T15:15:37.856 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-07-31T15:15:37.856 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-07-31T15:15:37.856 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-07-31T15:15:37.856 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-07-31T15:15:37.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.856 INFO:teuthology.orchestra.run.smithi067.stdout: "objecter": { 2024-07-31T15:15:37.856 INFO:teuthology.orchestra.run.smithi067.stdout: "op_active": 0, 2024-07-31T15:15:37.856 INFO:teuthology.orchestra.run.smithi067.stdout: "op_laggy": 0, 2024-07-31T15:15:37.857 INFO:teuthology.orchestra.run.smithi067.stdout: "op_send": 4818, 2024-07-31T15:15:37.857 INFO:teuthology.orchestra.run.smithi067.stdout: "op_send_bytes": 21, 2024-07-31T15:15:37.857 INFO:teuthology.orchestra.run.smithi067.stdout: "op_resend": 3, 2024-07-31T15:15:37.857 INFO:teuthology.orchestra.run.smithi067.stdout: "op_reply": 4818, 2024-07-31T15:15:37.857 INFO:teuthology.orchestra.run.smithi067.stdout: "op_latency": { 2024-07-31T15:15:37.857 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 4818, 2024-07-31T15:15:37.857 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 91.062120433, 2024-07-31T15:15:37.858 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.018900398 2024-07-31T15:15:37.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.858 INFO:teuthology.orchestra.run.smithi067.stdout: "op_inflight": 0, 2024-07-31T15:15:37.858 INFO:teuthology.orchestra.run.smithi067.stdout: "oplen_avg": { 2024-07-31T15:15:37.858 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 4818, 2024-07-31T15:15:37.858 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 4819 2024-07-31T15:15:37.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.858 INFO:teuthology.orchestra.run.smithi067.stdout: "op": 4818, 2024-07-31T15:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "op_r": 4818, 2024-07-31T15:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "op_w": 0, 2024-07-31T15:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "op_rmw": 0, 2024-07-31T15:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "op_pg": 0, 2024-07-31T15:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_stat": 3032, 2024-07-31T15:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_create": 0, 2024-07-31T15:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_read": 1784, 2024-07-31T15:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_write": 0, 2024-07-31T15:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_writefull": 0, 2024-07-31T15:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_writesame": 0, 2024-07-31T15:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_append": 0, 2024-07-31T15:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_zero": 0, 2024-07-31T15:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_truncate": 0, 2024-07-31T15:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_delete": 0, 2024-07-31T15:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_mapext": 0, 2024-07-31T15:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_sparse_read": 0, 2024-07-31T15:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_clonerange": 0, 2024-07-31T15:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_getxattr": 1, 2024-07-31T15:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_setxattr": 0, 2024-07-31T15:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_cmpxattr": 0, 2024-07-31T15:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_rmxattr": 0, 2024-07-31T15:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_resetxattrs": 0, 2024-07-31T15:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_call": 0, 2024-07-31T15:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_watch": 0, 2024-07-31T15:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_notify": 0, 2024-07-31T15:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_src_cmpxattr": 0, 2024-07-31T15:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_pgls": 0, 2024-07-31T15:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_pgls_filter": 0, 2024-07-31T15:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_other": 0, 2024-07-31T15:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_active": 0, 2024-07-31T15:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_send": 0, 2024-07-31T15:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_resend": 0, 2024-07-31T15:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_ping": 0, 2024-07-31T15:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_active": 0, 2024-07-31T15:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_send": 0, 2024-07-31T15:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_resend": 0, 2024-07-31T15:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_active": 0, 2024-07-31T15:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_send": 0, 2024-07-31T15:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_resend": 0, 2024-07-31T15:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_active": 0, 2024-07-31T15:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_send": 0, 2024-07-31T15:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_resend": 0, 2024-07-31T15:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "command_active": 0, 2024-07-31T15:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "command_send": 0, 2024-07-31T15:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "command_resend": 0, 2024-07-31T15:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "map_epoch": 138, 2024-07-31T15:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "map_full": 0, 2024-07-31T15:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "map_inc": 49, 2024-07-31T15:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_sessions": 6, 2024-07-31T15:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_session_open": 7, 2024-07-31T15:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_session_close": 1, 2024-07-31T15:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_laggy": 0, 2024-07-31T15:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_wr": 0, 2024-07-31T15:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_rd": 2, 2024-07-31T15:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_del": 0 2024-07-31T15:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: "oft": { 2024-07-31T15:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_objs": 0, 2024-07-31T15:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_kv_pairs": 0, 2024-07-31T15:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_updates": 0, 2024-07-31T15:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_removes": 0 2024-07-31T15:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "osd-slow-ops": { 2024-07-31T15:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "slow_ops_count": 0 2024-07-31T15:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "purge_queue": { 2024-07-31T15:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_ops": 0, 2024-07-31T15:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_ops_high_water": 0, 2024-07-31T15:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing": 0, 2024-07-31T15:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_high_water": 0, 2024-07-31T15:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executed_ops": 0, 2024-07-31T15:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executed": 0, 2024-07-31T15:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_item_in_journal": 0 2024-07-31T15:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-31T15:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-31T15:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-07-31T15:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-31T15:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 5380, 2024-07-31T15:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 25601612, 2024-07-31T15:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 5380, 2024-07-31T15:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-31T15:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-31T15:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 5380, 2024-07-31T15:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 25601612, 2024-07-31T15:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-31T15:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-objecter_bytes": { 2024-07-31T15:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-31T15:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-07-31T15:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-31T15:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-31T15:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-31T15:15:37.872 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:37.872 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:37.872 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 4818, 2024-07-31T15:15:37.872 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 23105584, 2024-07-31T15:15:37.872 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 4818, 2024-07-31T15:15:37.872 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 23105584, 2024-07-31T15:15:37.872 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-31T15:15:37.873 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.873 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.873 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.873 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:37.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.873 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-objecter_ops": { 2024-07-31T15:15:37.873 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-31T15:15:37.873 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1024, 2024-07-31T15:15:37.874 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-31T15:15:37.874 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-31T15:15:37.874 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-31T15:15:37.874 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:37.874 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:37.874 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 4818, 2024-07-31T15:15:37.874 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 4818, 2024-07-31T15:15:37.874 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 4818, 2024-07-31T15:15:37.875 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 4818, 2024-07-31T15:15:37.875 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-31T15:15:37.875 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.875 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.875 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.875 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:37.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.875 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-write_buf_throttle": { 2024-07-31T15:15:37.876 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-31T15:15:37.876 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 3758096384, 2024-07-31T15:15:37.876 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-31T15:15:37.876 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-31T15:15:37.876 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-31T15:15:37.876 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:37.876 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:37.877 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-31T15:15:37.877 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-31T15:15:37.877 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 0, 2024-07-31T15:15:37.877 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 0, 2024-07-31T15:15:37.877 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-31T15:15:37.877 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.877 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.877 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.877 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:37.878 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:37.878 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-write_buf_throttle-0x564db865c8f0": { 2024-07-31T15:15:37.878 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-31T15:15:37.878 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 3758096384, 2024-07-31T15:15:37.878 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-31T15:15:37.878 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-31T15:15:37.878 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-31T15:15:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-31T15:15:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-31T15:15:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 0, 2024-07-31T15:15:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 0, 2024-07-31T15:15:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-31T15:15:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:37.880 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:15:38.807 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-07-31T15:15:38.807 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-07-31T15:15:38.807 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-07-31T15:15:38.808 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-07-31T15:15:38.808 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-07-31T15:15:38.808 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-07-31T15:15:38.808 DEBUG:tasks.check_counter:Getting stats from j 2024-07-31T15:15:38.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph daemon mds.j perf dump 2024-07-31T15:15:39.076 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:15:39.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:39 smithi073 ceph-mon[36714]: pgmap v1185: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-31T15:15:39.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:39 smithi067 ceph-mon[32750]: pgmap v1185: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-31T15:15:39.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:39 smithi077 ceph-mon[37327]: pgmap v1185: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-31T15:15:41.026 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-31T15:15:41.027 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-07-31T15:15:41.027 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 0, 2024-07-31T15:15:41.027 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 2, 2024-07-31T15:15:41.027 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 0, 2024-07-31T15:15:41.027 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 0, 2024-07-31T15:15:41.027 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 4, 2024-07-31T15:15:41.028 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 0, 2024-07-31T15:15:41.028 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.001267459, 2024-07-31T15:15:41.028 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.000000000, 2024-07-31T15:15:41.028 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.000189823, 2024-07-31T15:15:41.028 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-31T15:15:41.028 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:41.028 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:41.028 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:41.029 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:41.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:41.029 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:41.029 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:41.029 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:41.029 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:41.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:41.029 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-31T15:15:41.030 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 0 2024-07-31T15:15:41.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:41.030 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-07-31T15:15:41.030 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 6, 2024-07-31T15:15:41.030 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 357, 2024-07-31T15:15:41.030 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 3312, 2024-07-31T15:15:41.030 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 222032, 2024-07-31T15:15:41.030 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 4, 2024-07-31T15:15:41.031 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 2, 2024-07-31T15:15:41.031 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.030664630, 2024-07-31T15:15:41.031 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.023429579, 2024-07-31T15:15:41.031 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.004093152, 2024-07-31T15:15:41.031 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-31T15:15:41.031 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:41.031 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 356, 2024-07-31T15:15:41.032 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.025375452, 2024-07-31T15:15:41.032 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000071279 2024-07-31T15:15:41.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:41.032 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:41.032 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:41.032 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:41.032 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:41.033 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:41.033 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-07-31T15:15:41.033 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 222032 2024-07-31T15:15:41.033 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:41.033 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-07-31T15:15:41.033 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 472, 2024-07-31T15:15:41.033 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 464, 2024-07-31T15:15:41.033 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 1345440, 2024-07-31T15:15:41.034 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 287632, 2024-07-31T15:15:41.034 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 2, 2024-07-31T15:15:41.034 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 1, 2024-07-31T15:15:41.034 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.084047484, 2024-07-31T15:15:41.034 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.033327938, 2024-07-31T15:15:41.034 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.063419516, 2024-07-31T15:15:41.034 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-31T15:15:41.035 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:41.035 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 463, 2024-07-31T15:15:41.035 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.034114477, 2024-07-31T15:15:41.035 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000073681 2024-07-31T15:15:41.035 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:41.035 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:41.035 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:41.035 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:41.036 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:41.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:41.036 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 1345440, 2024-07-31T15:15:41.036 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 287632 2024-07-31T15:15:41.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:41.036 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-07-31T15:15:41.036 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 0, 2024-07-31T15:15:41.036 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-07-31T15:15:41.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:41.037 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-07-31T15:15:41.037 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-07-31T15:15:41.037 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-07-31T15:15:41.037 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-07-31T15:15:41.037 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-07-31T15:15:41.037 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-07-31T15:15:41.037 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-07-31T15:15:41.038 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-31T15:15:41.038 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-07-31T15:15:41.038 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-31T15:15:41.038 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-07-31T15:15:41.038 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-07-31T15:15:41.038 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-07-31T15:15:41.038 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-31T15:15:41.038 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-07-31T15:15:41.039 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-07-31T15:15:41.039 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-07-31T15:15:41.039 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-07-31T15:15:41.039 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-07-31T15:15:41.039 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-31T15:15:41.039 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-07-31T15:15:41.039 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-31T15:15:41.039 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-07-31T15:15:41.040 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-07-31T15:15:41.040 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-07-31T15:15:41.040 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-07-31T15:15:41.040 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-07-31T15:15:41.040 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-31T15:15:41.040 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-07-31T15:15:41.040 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-07-31T15:15:41.041 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-07-31T15:15:41.041 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-07-31T15:15:41.041 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-07-31T15:15:41.041 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-07-31T15:15:41.041 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-07-31T15:15:41.041 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-07-31T15:15:41.041 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-07-31T15:15:41.041 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 148704, 2024-07-31T15:15:41.042 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 29, 2024-07-31T15:15:41.042 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-07-31T15:15:41.042 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-07-31T15:15:41.042 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-07-31T15:15:41.042 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-07-31T15:15:41.042 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-07-31T15:15:41.042 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-07-31T15:15:41.042 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-07-31T15:15:41.043 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-07-31T15:15:41.043 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 0, 2024-07-31T15:15:41.043 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 0, 2024-07-31T15:15:41.043 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-07-31T15:15:41.043 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-07-31T15:15:41.043 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-07-31T15:15:41.043 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-07-31T15:15:41.044 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 928, 2024-07-31T15:15:41.044 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 2, 2024-07-31T15:15:41.044 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-07-31T15:15:41.044 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-07-31T15:15:41.044 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-07-31T15:15:41.044 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-07-31T15:15:41.044 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-31T15:15:41.044 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-31T15:15:41.045 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-31T15:15:41.045 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-07-31T15:15:41.045 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-31T15:15:41.045 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 474, 2024-07-31T15:15:41.045 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 1278693, 2024-07-31T15:15:41.045 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:41.045 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 474, 2024-07-31T15:15:41.045 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-31T15:15:41.046 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-31T15:15:41.046 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 474, 2024-07-31T15:15:41.046 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 1278693, 2024-07-31T15:15:41.046 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-31T15:15:41.046 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-31T15:15:41.046 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-31T15:15:41.046 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-31T15:15:41.047 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:41.047 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-31T15:15:41.047 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-31T15:15:41.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:41 smithi067 ceph-mon[32750]: pgmap v1186: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-07-31T15:15:41.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:41 smithi073 ceph-mon[36714]: pgmap v1186: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-07-31T15:15:41.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:41 smithi077 ceph-mon[37327]: pgmap v1186: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-07-31T15:15:41.744 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-07-31T15:15:41.744 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-07-31T15:15:41.744 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-07-31T15:15:41.744 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-07-31T15:15:41.745 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-07-31T15:15:41.745 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-07-31T15:15:41.745 DEBUG:tasks.check_counter:Getting stats from b 2024-07-31T15:15:41.745 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph daemon mds.b perf dump 2024-07-31T15:15:42.178 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.b/config 2024-07-31T15:15:43.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:43 smithi073 ceph-mon[36714]: pgmap v1187: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-07-31T15:15:43.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:43 smithi077 ceph-mon[37327]: pgmap v1187: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-07-31T15:15:43.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:43 smithi067 ceph-mon[32750]: pgmap v1187: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-07-31T15:15:44.304 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-07-31T15:15:44.304 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-07-31T15:15:44.304 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 613430, 2024-07-31T15:15:44.305 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 800050, 2024-07-31T15:15:44.305 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 306405798, 2024-07-31T15:15:44.305 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 584428690, 2024-07-31T15:15:44.305 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 25, 2024-07-31T15:15:44.305 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 4, 2024-07-31T15:15:44.305 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 39.012329343, 2024-07-31T15:15:44.305 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 15.428265641, 2024-07-31T15:15:44.306 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 37.392304471, 2024-07-31T15:15:44.306 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 6.779443358, 2024-07-31T15:15:44.306 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:44.306 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 800049, 2024-07-31T15:15:44.306 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 172.245247616, 2024-07-31T15:15:44.306 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000215293 2024-07-31T15:15:44.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.307 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:44.307 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 719500, 2024-07-31T15:15:44.307 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.152614756, 2024-07-31T15:15:44.307 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000212 2024-07-31T15:15:44.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.307 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 129772432, 2024-07-31T15:15:44.307 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 284098128 2024-07-31T15:15:44.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.308 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-07-31T15:15:44.308 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 303012, 2024-07-31T15:15:44.308 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 311394, 2024-07-31T15:15:44.308 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 183452495, 2024-07-31T15:15:44.308 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 371522928, 2024-07-31T15:15:44.308 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 127, 2024-07-31T15:15:44.308 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 6, 2024-07-31T15:15:44.309 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 18.326672356, 2024-07-31T15:15:44.309 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 5.569159611, 2024-07-31T15:15:44.309 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 4.365156667, 2024-07-31T15:15:44.309 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 7.375626267, 2024-07-31T15:15:44.309 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:44.309 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 311392, 2024-07-31T15:15:44.309 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 145.119456423, 2024-07-31T15:15:44.309 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000466034 2024-07-31T15:15:44.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.310 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:44.310 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 31090, 2024-07-31T15:15:44.310 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.005948462, 2024-07-31T15:15:44.310 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000191 2024-07-31T15:15:44.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.310 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-31T15:15:44.310 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 717104 2024-07-31T15:15:44.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.311 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-07-31T15:15:44.311 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 347816, 2024-07-31T15:15:44.311 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 359712, 2024-07-31T15:15:44.311 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 212103356, 2024-07-31T15:15:44.311 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 407420563, 2024-07-31T15:15:44.311 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 304, 2024-07-31T15:15:44.311 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 6, 2024-07-31T15:15:44.312 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 22.056991096, 2024-07-31T15:15:44.312 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 7.706176922, 2024-07-31T15:15:44.312 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 5.816758748, 2024-07-31T15:15:44.312 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 7.626158858, 2024-07-31T15:15:44.312 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:44.312 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 359711, 2024-07-31T15:15:44.312 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 110.556862551, 2024-07-31T15:15:44.313 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000307349 2024-07-31T15:15:44.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.313 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:44.313 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 76947, 2024-07-31T15:15:44.313 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.014486418, 2024-07-31T15:15:44.313 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000188 2024-07-31T15:15:44.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.313 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 36736, 2024-07-31T15:15:44.314 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 2384 2024-07-31T15:15:44.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.314 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-07-31T15:15:44.314 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-07-31T15:15:44.314 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-07-31T15:15:44.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.314 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-07-31T15:15:44.314 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-07-31T15:15:44.315 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-07-31T15:15:44.315 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 543762, 2024-07-31T15:15:44.315 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 365.935019730, 2024-07-31T15:15:44.315 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000672969 2024-07-31T15:15:44.315 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:44.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.315 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-07-31T15:15:44.315 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-07-31T15:15:44.316 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-07-31T15:15:44.316 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 5822, 2024-07-31T15:15:44.316 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1.238803369, 2024-07-31T15:15:44.316 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000212779 2024-07-31T15:15:44.316 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:44.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.316 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-07-31T15:15:44.316 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 119317, 2024-07-31T15:15:44.317 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 117374, 2024-07-31T15:15:44.317 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-07-31T15:15:44.317 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 117374, 2024-07-31T15:15:44.317 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1857.830124244, 2024-07-31T15:15:44.317 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.015828293 2024-07-31T15:15:44.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.317 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-07-31T15:15:44.317 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 1943, 2024-07-31T15:15:44.318 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 52, 2024-07-31T15:15:44.318 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 22, 2024-07-31T15:15:44.318 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 13973, 2024-07-31T15:15:44.318 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 118, 2024-07-31T15:15:44.318 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 0, 2024-07-31T15:15:44.318 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 67243, 2024-07-31T15:15:44.319 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 26234, 2024-07-31T15:15:44.319 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 40952, 2024-07-31T15:15:44.319 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 57, 2024-07-31T15:15:44.319 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 55352, 2024-07-31T15:15:44.319 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 18618, 2024-07-31T15:15:44.319 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 580, 2024-07-31T15:15:44.319 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 580, 2024-07-31T15:15:44.319 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 478, 2024-07-31T15:15:44.320 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 146260, 2024-07-31T15:15:44.320 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 120945, 2024-07-31T15:15:44.320 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 1719, 2024-07-31T15:15:44.320 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 62, 2024-07-31T15:15:44.320 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-07-31T15:15:44.320 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-07-31T15:15:44.320 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 2, 2024-07-31T15:15:44.320 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-07-31T15:15:44.321 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-07-31T15:15:44.321 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 785, 2024-07-31T15:15:44.321 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 18652, 2024-07-31T15:15:44.321 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 58, 2024-07-31T15:15:44.321 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 1125, 2024-07-31T15:15:44.321 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-07-31T15:15:44.321 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 697759, 2024-07-31T15:15:44.321 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-07-31T15:15:44.321 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 91397, 2024-07-31T15:15:44.322 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 2022860651, 2024-07-31T15:15:44.322 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 7, 2024-07-31T15:15:44.322 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 697759, 2024-07-31T15:15:44.322 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-07-31T15:15:44.322 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-07-31T15:15:44.322 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-07-31T15:15:44.322 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 43121, 2024-07-31T15:15:44.323 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 20, 2024-07-31T15:15:44.323 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 43101, 2024-07-31T15:15:44.323 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 654638, 2024-07-31T15:15:44.323 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 695, 2024-07-31T15:15:44.323 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 137455, 2024-07-31T15:15:44.323 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 85448, 2024-07-31T15:15:44.323 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 173, 2024-07-31T15:15:44.323 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-31T15:15:44.324 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-31T15:15:44.324 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 191554, 2024-07-31T15:15:44.324 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 71516, 2024-07-31T15:15:44.324 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 9517, 2024-07-31T15:15:44.324 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 12872 2024-07-31T15:15:44.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.324 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-07-31T15:15:44.324 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 0, 2024-07-31T15:15:44.325 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 0, 2024-07-31T15:15:44.325 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 0, 2024-07-31T15:15:44.325 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 60, 2024-07-31T15:15:44.325 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 839, 2024-07-31T15:15:44.325 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 0, 2024-07-31T15:15:44.325 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-07-31T15:15:44.325 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-07-31T15:15:44.326 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 2709, 2024-07-31T15:15:44.326 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 2709, 2024-07-31T15:15:44.326 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-07-31T15:15:44.326 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-07-31T15:15:44.326 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-07-31T15:15:44.326 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-07-31T15:15:44.326 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-07-31T15:15:44.326 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-07-31T15:15:44.327 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-07-31T15:15:44.327 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 16, 2024-07-31T15:15:44.327 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 644261, 2024-07-31T15:15:44.327 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 20, 2024-07-31T15:15:44.327 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 789, 2024-07-31T15:15:44.327 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-07-31T15:15:44.327 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 120, 2024-07-31T15:15:44.327 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-07-31T15:15:44.328 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-07-31T15:15:44.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.328 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-07-31T15:15:44.328 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 66, 2024-07-31T15:15:44.328 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 190297, 2024-07-31T15:15:44.328 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 190253, 2024-07-31T15:15:44.328 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 190251, 2024-07-31T15:15:44.328 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 46, 2024-07-31T15:15:44.329 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-07-31T15:15:44.329 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 23, 2024-07-31T15:15:44.329 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 318, 2024-07-31T15:15:44.329 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 317, 2024-07-31T15:15:44.329 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 316, 2024-07-31T15:15:44.329 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 2, 2024-07-31T15:15:44.329 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 1, 2024-07-31T15:15:44.330 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-07-31T15:15:44.330 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 2, 2024-07-31T15:15:44.330 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 590829638, 2024-07-31T15:15:44.330 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 592286197, 2024-07-31T15:15:44.330 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 0, 2024-07-31T15:15:44.330 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-07-31T15:15:44.330 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 33103, 2024-07-31T15:15:44.330 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 8355.731155218, 2024-07-31T15:15:44.331 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.252416130 2024-07-31T15:15:44.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.331 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 0 2024-07-31T15:15:44.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.331 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-07-31T15:15:44.331 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 61918, 2024-07-31T15:15:44.331 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 83260, 2024-07-31T15:15:44.332 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 21342, 2024-07-31T15:15:44.332 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 4254, 2024-07-31T15:15:44.332 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 5407, 2024-07-31T15:15:44.332 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 1153, 2024-07-31T15:15:44.332 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 67243, 2024-07-31T15:15:44.332 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 92947, 2024-07-31T15:15:44.332 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 25704, 2024-07-31T15:15:44.332 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 580, 2024-07-31T15:15:44.333 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 138242, 2024-07-31T15:15:44.333 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 137662, 2024-07-31T15:15:44.333 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 637504, 2024-07-31T15:15:44.333 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 231732 2024-07-31T15:15:44.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.333 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-07-31T15:15:44.333 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 121909, 2024-07-31T15:15:44.333 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 1771, 2024-07-31T15:15:44.334 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 119317, 2024-07-31T15:15:44.334 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 686, 2024-07-31T15:15:44.334 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 3668, 2024-07-31T15:15:44.334 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-07-31T15:15:44.334 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 69060, 2024-07-31T15:15:44.334 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 589.166951252, 2024-07-31T15:15:44.334 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.008531233 2024-07-31T15:15:44.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.335 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-07-31T15:15:44.335 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 524, 2024-07-31T15:15:44.335 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 81.695693173, 2024-07-31T15:15:44.335 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.155907811 2024-07-31T15:15:44.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.335 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-07-31T15:15:44.335 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.336 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.336 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.336 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-07-31T15:15:44.336 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.336 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.336 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.337 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-07-31T15:15:44.337 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 25935, 2024-07-31T15:15:44.337 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 990.156412109, 2024-07-31T15:15:44.337 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.038178384 2024-07-31T15:15:44.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.337 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-07-31T15:15:44.337 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.338 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.338 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.338 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-07-31T15:15:44.338 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1, 2024-07-31T15:15:44.338 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000151688, 2024-07-31T15:15:44.338 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000151688 2024-07-31T15:15:44.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.339 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-07-31T15:15:44.339 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.339 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.339 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.339 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.339 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-07-31T15:15:44.339 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.339 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.340 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.340 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-07-31T15:15:44.341 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.341 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.341 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.341 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-07-31T15:15:44.341 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 24, 2024-07-31T15:15:44.341 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.039638953, 2024-07-31T15:15:44.341 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001651623 2024-07-31T15:15:44.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.342 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-07-31T15:15:44.342 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 4233, 2024-07-31T15:15:44.342 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 106.736609692, 2024-07-31T15:15:44.342 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.025215357 2024-07-31T15:15:44.342 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.342 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-07-31T15:15:44.342 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.343 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.343 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.343 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-07-31T15:15:44.343 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 24, 2024-07-31T15:15:44.343 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 18.400764065, 2024-07-31T15:15:44.343 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.766698502 2024-07-31T15:15:44.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.344 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-07-31T15:15:44.344 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 10189, 2024-07-31T15:15:44.344 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 40.533078015, 2024-07-31T15:15:44.344 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.003978121 2024-07-31T15:15:44.344 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.344 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-07-31T15:15:44.344 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 4320, 2024-07-31T15:15:44.344 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 15.981206968, 2024-07-31T15:15:44.345 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.003699353 2024-07-31T15:15:44.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.345 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-07-31T15:15:44.345 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 31, 2024-07-31T15:15:44.345 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.052888278, 2024-07-31T15:15:44.345 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001706073 2024-07-31T15:15:44.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.345 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-07-31T15:15:44.345 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.346 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.346 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.346 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-07-31T15:15:44.346 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.346 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.346 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.347 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-07-31T15:15:44.347 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 124, 2024-07-31T15:15:44.347 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 2.527365239, 2024-07-31T15:15:44.347 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.020381977 2024-07-31T15:15:44.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.347 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-07-31T15:15:44.348 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 17, 2024-07-31T15:15:44.348 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1.072911222, 2024-07-31T15:15:44.348 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.063112424 2024-07-31T15:15:44.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.348 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-07-31T15:15:44.348 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.348 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.348 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.349 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-07-31T15:15:44.349 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 243, 2024-07-31T15:15:44.349 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 6.085649216, 2024-07-31T15:15:44.349 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.025043823 2024-07-31T15:15:44.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.349 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-07-31T15:15:44.349 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.349 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.350 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.350 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-07-31T15:15:44.350 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.350 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.350 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.351 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-07-31T15:15:44.351 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.351 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.351 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.351 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-07-31T15:15:44.351 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 9, 2024-07-31T15:15:44.351 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.022969111, 2024-07-31T15:15:44.352 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.002552123 2024-07-31T15:15:44.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.352 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-07-31T15:15:44.352 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 55, 2024-07-31T15:15:44.352 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.077186846, 2024-07-31T15:15:44.352 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001403397 2024-07-31T15:15:44.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.352 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-07-31T15:15:44.352 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2585, 2024-07-31T15:15:44.353 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 5.280648417, 2024-07-31T15:15:44.353 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.002042804 2024-07-31T15:15:44.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.353 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-07-31T15:15:44.353 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-07-31T15:15:44.353 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-07-31T15:15:44.353 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.353 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.354 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.354 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:44.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.354 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-07-31T15:15:44.354 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 2, 2024-07-31T15:15:44.354 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 5, 2024-07-31T15:15:44.354 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 3, 2024-07-31T15:15:44.354 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 2, 2024-07-31T15:15:44.355 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-07-31T15:15:44.355 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 3773, 2024-07-31T15:15:44.355 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 1886, 2024-07-31T15:15:44.355 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 1446, 2024-07-31T15:15:44.355 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-07-31T15:15:44.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.355 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-07-31T15:15:44.355 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-07-31T15:15:44.356 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-07-31T15:15:44.356 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-07-31T15:15:44.356 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-07-31T15:15:44.356 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-07-31T15:15:44.356 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-07-31T15:15:44.356 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-31T15:15:44.356 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-07-31T15:15:44.356 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-31T15:15:44.357 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-07-31T15:15:44.357 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-07-31T15:15:44.357 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-07-31T15:15:44.357 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-31T15:15:44.357 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-07-31T15:15:44.357 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-07-31T15:15:44.357 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-07-31T15:15:44.358 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-07-31T15:15:44.358 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-07-31T15:15:44.358 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-31T15:15:44.358 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-07-31T15:15:44.358 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-31T15:15:44.358 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-07-31T15:15:44.358 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-07-31T15:15:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-07-31T15:15:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-07-31T15:15:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-07-31T15:15:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-31T15:15:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-07-31T15:15:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-07-31T15:15:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-07-31T15:15:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-07-31T15:15:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-07-31T15:15:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-07-31T15:15:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-07-31T15:15:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-07-31T15:15:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-07-31T15:15:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 2375667, 2024-07-31T15:15:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 55485, 2024-07-31T15:15:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 0, 2024-07-31T15:15:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 0, 2024-07-31T15:15:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-07-31T15:15:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-07-31T15:15:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-07-31T15:15:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-07-31T15:15:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-07-31T15:15:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-07-31T15:15:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 9352, 2024-07-31T15:15:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 189, 2024-07-31T15:15:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-07-31T15:15:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-07-31T15:15:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-07-31T15:15:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-07-31T15:15:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 203816511, 2024-07-31T15:15:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 2744989, 2024-07-31T15:15:44.363 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-07-31T15:15:44.363 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-07-31T15:15:44.363 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-07-31T15:15:44.363 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-07-31T15:15:44.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.363 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-07-31T15:15:44.363 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-07-31T15:15:44.364 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-07-31T15:15:44.364 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 870794, 2024-07-31T15:15:44.364 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 750607634, 2024-07-31T15:15:44.364 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 37, 2024-07-31T15:15:44.364 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 870794, 2024-07-31T15:15:44.364 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-07-31T15:15:44.364 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 870794, 2024-07-31T15:15:44.364 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 135918.039572171, 2024-07-31T15:15:44.365 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.156085181 2024-07-31T15:15:44.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.365 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-07-31T15:15:44.365 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-07-31T15:15:44.365 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 870794, 2024-07-31T15:15:44.365 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1073324 2024-07-31T15:15:44.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.365 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 870794, 2024-07-31T15:15:44.366 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 698173, 2024-07-31T15:15:44.366 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 172621, 2024-07-31T15:15:44.366 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-07-31T15:15:44.366 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-07-31T15:15:44.366 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 9701, 2024-07-31T15:15:44.366 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 117211, 2024-07-31T15:15:44.366 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 254, 2024-07-31T15:15:44.366 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 33338, 2024-07-31T15:15:44.367 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 558, 2024-07-31T15:15:44.367 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-07-31T15:15:44.367 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-07-31T15:15:44.367 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-07-31T15:15:44.367 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 102, 2024-07-31T15:15:44.367 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 6121, 2024-07-31T15:15:44.367 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-07-31T15:15:44.368 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-07-31T15:15:44.368 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-07-31T15:15:44.368 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 697846, 2024-07-31T15:15:44.368 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 177648, 2024-07-31T15:15:44.368 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-07-31T15:15:44.368 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-07-31T15:15:44.368 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-07-31T15:15:44.368 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-07-31T15:15:44.369 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-07-31T15:15:44.369 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-07-31T15:15:44.369 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-07-31T15:15:44.369 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-07-31T15:15:44.369 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-07-31T15:15:44.369 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 173, 2024-07-31T15:15:44.369 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-07-31T15:15:44.369 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-07-31T15:15:44.370 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-07-31T15:15:44.370 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-07-31T15:15:44.370 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-07-31T15:15:44.370 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-07-31T15:15:44.370 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-07-31T15:15:44.370 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-07-31T15:15:44.370 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-07-31T15:15:44.370 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-07-31T15:15:44.371 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-07-31T15:15:44.371 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-07-31T15:15:44.371 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-07-31T15:15:44.371 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-07-31T15:15:44.371 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-07-31T15:15:44.371 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-07-31T15:15:44.371 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 138, 2024-07-31T15:15:44.371 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-07-31T15:15:44.372 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 58, 2024-07-31T15:15:44.372 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 12, 2024-07-31T15:15:44.372 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 12, 2024-07-31T15:15:44.372 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 0, 2024-07-31T15:15:44.372 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-07-31T15:15:44.372 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 28626, 2024-07-31T15:15:44.372 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 214, 2024-07-31T15:15:44.372 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 1532 2024-07-31T15:15:44.373 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.373 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-07-31T15:15:44.373 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 1, 2024-07-31T15:15:44.373 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-07-31T15:15:44.373 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-07-31T15:15:44.373 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-07-31T15:15:44.373 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.374 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-07-31T15:15:44.374 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 631478 2024-07-31T15:15:44.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.374 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-07-31T15:15:44.374 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-07-31T15:15:44.374 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 22, 2024-07-31T15:15:44.374 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-07-31T15:15:44.375 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 9, 2024-07-31T15:15:44.375 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 5314, 2024-07-31T15:15:44.375 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 2709, 2024-07-31T15:15:44.375 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-07-31T15:15:44.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.375 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-31T15:15:44.375 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:44.375 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-07-31T15:15:44.376 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:44.376 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 1264254, 2024-07-31T15:15:44.376 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 571600401, 2024-07-31T15:15:44.376 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:44.376 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 1264254, 2024-07-31T15:15:44.376 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-07-31T15:15:44.376 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-07-31T15:15:44.377 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 1264254, 2024-07-31T15:15:44.377 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 571600401, 2024-07-31T15:15:44.377 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:44.377 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.377 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.377 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.377 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:44.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.378 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-07-31T15:15:44.378 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:44.378 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-07-31T15:15:44.378 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:44.378 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-07-31T15:15:44.378 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-07-31T15:15:44.378 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:44.379 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:44.379 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 870794, 2024-07-31T15:15:44.379 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 750899184, 2024-07-31T15:15:44.379 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 870794, 2024-07-31T15:15:44.379 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 750899184, 2024-07-31T15:15:44.379 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:44.379 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.379 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.380 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.380 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:44.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.380 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-07-31T15:15:44.380 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:44.380 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-07-31T15:15:44.380 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:44.381 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-07-31T15:15:44.381 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-07-31T15:15:44.381 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:44.381 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:44.381 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 870794, 2024-07-31T15:15:44.381 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 870794, 2024-07-31T15:15:44.381 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 870794, 2024-07-31T15:15:44.381 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 870794, 2024-07-31T15:15:44.382 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:44.382 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.382 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.382 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.382 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:44.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.382 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-07-31T15:15:44.383 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:44.383 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-07-31T15:15:44.383 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:44.383 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 2709, 2024-07-31T15:15:44.383 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 294625, 2024-07-31T15:15:44.383 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:44.383 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 2709, 2024-07-31T15:15:44.383 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-07-31T15:15:44.384 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-07-31T15:15:44.384 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 235, 2024-07-31T15:15:44.384 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 294625, 2024-07-31T15:15:44.384 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:44.384 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.384 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.384 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.384 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:44.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:44.385 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x55ce7b96c750": { 2024-07-31T15:15:44.385 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:44.385 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-07-31T15:15:44.385 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:44.385 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 190297, 2024-07-31T15:15:44.385 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 588091893, 2024-07-31T15:15:44.386 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:44.386 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 190297, 2024-07-31T15:15:44.386 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-07-31T15:15:44.386 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-07-31T15:15:44.386 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 33103, 2024-07-31T15:15:44.386 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 588091893, 2024-07-31T15:15:44.386 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:44.386 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:44.387 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:44.387 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:44.387 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:44.387 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:44.387 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-07-31T15:15:45.268 INFO:tasks.check_counter:Daemon mds.b mds.exported=785 2024-07-31T15:15:45.268 INFO:tasks.check_counter:Daemon mds.b mds.imported=58 2024-07-31T15:15:45.269 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-07-31T15:15:45.269 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=24 2024-07-31T15:15:45.269 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=17 2024-07-31T15:15:45.269 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=118 2024-07-31T15:15:45.269 DEBUG:tasks.check_counter:Getting stats from e 2024-07-31T15:15:45.269 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph daemon mds.e perf dump 2024-07-31T15:15:45.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:45 smithi077 ceph-mon[37327]: pgmap v1188: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 9 op/s 2024-07-31T15:15:45.529 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.b/config 2024-07-31T15:15:45.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:45 smithi067 ceph-mon[32750]: pgmap v1188: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 9 op/s 2024-07-31T15:15:45.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:45 smithi073 ceph-mon[36714]: pgmap v1188: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 9 op/s 2024-07-31T15:15:47.344 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:47 smithi073 ceph-mon[36714]: pgmap v1189: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-07-31T15:15:47.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:47 smithi077 ceph-mon[37327]: pgmap v1189: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-07-31T15:15:47.539 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-07-31T15:15:47.540 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-07-31T15:15:47.540 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 776, 2024-07-31T15:15:47.540 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 776, 2024-07-31T15:15:47.540 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 26318716, 2024-07-31T15:15:47.540 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 245954, 2024-07-31T15:15:47.540 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 8, 2024-07-31T15:15:47.540 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 4, 2024-07-31T15:15:47.540 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.079289911, 2024-07-31T15:15:47.541 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.020016422, 2024-07-31T15:15:47.541 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.033213931, 2024-07-31T15:15:47.541 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.020602653, 2024-07-31T15:15:47.541 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:47.541 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 775, 2024-07-31T15:15:47.541 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.031721965, 2024-07-31T15:15:47.541 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000040931 2024-07-31T15:15:47.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.542 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:47.542 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.542 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.542 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.542 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-07-31T15:15:47.542 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 448 2024-07-31T15:15:47.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.543 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-07-31T15:15:47.543 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 237, 2024-07-31T15:15:47.543 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 237, 2024-07-31T15:15:47.543 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 21190972, 2024-07-31T15:15:47.543 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 98415, 2024-07-31T15:15:47.543 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 5, 2024-07-31T15:15:47.543 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 4, 2024-07-31T15:15:47.544 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.033800777, 2024-07-31T15:15:47.544 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.008157484, 2024-07-31T15:15:47.544 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.029577697, 2024-07-31T15:15:47.544 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.003071532, 2024-07-31T15:15:47.544 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:47.544 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 236, 2024-07-31T15:15:47.544 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.010464006, 2024-07-31T15:15:47.544 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000044339 2024-07-31T15:15:47.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.545 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:47.545 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.545 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.545 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.545 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 203616, 2024-07-31T15:15:47.545 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 56064 2024-07-31T15:15:47.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.545 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-07-31T15:15:47.546 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 205, 2024-07-31T15:15:47.546 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 271, 2024-07-31T15:15:47.546 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 21176441, 2024-07-31T15:15:47.546 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 268754, 2024-07-31T15:15:47.546 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 6, 2024-07-31T15:15:47.546 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-07-31T15:15:47.546 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.034689830, 2024-07-31T15:15:47.546 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.006247902, 2024-07-31T15:15:47.547 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.021660610, 2024-07-31T15:15:47.547 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.005030413, 2024-07-31T15:15:47.547 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:47.547 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 269, 2024-07-31T15:15:47.547 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.018748148, 2024-07-31T15:15:47.547 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000069695 2024-07-31T15:15:47.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.547 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:47.548 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.548 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.548 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.548 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-31T15:15:47.548 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 203872 2024-07-31T15:15:47.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.548 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-07-31T15:15:47.548 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-07-31T15:15:47.549 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-07-31T15:15:47.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.549 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-07-31T15:15:47.549 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-07-31T15:15:47.549 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-07-31T15:15:47.549 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1858, 2024-07-31T15:15:47.549 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.102478481, 2024-07-31T15:15:47.549 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000055155 2024-07-31T15:15:47.550 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:47.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.550 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-07-31T15:15:47.550 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-07-31T15:15:47.550 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-07-31T15:15:47.550 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.550 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.550 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.551 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:47.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.551 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-07-31T15:15:47.551 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 0, 2024-07-31T15:15:47.551 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 0, 2024-07-31T15:15:47.551 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-07-31T15:15:47.551 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.551 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.551 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.552 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-07-31T15:15:47.552 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 0, 2024-07-31T15:15:47.552 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 0, 2024-07-31T15:15:47.552 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-07-31T15:15:47.552 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 0, 2024-07-31T15:15:47.552 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 0, 2024-07-31T15:15:47.552 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 0, 2024-07-31T15:15:47.552 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 7886, 2024-07-31T15:15:47.553 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 4904, 2024-07-31T15:15:47.553 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 2982, 2024-07-31T15:15:47.553 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 0, 2024-07-31T15:15:47.553 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 883, 2024-07-31T15:15:47.553 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 0, 2024-07-31T15:15:47.553 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 0, 2024-07-31T15:15:47.553 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 0, 2024-07-31T15:15:47.553 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 401, 2024-07-31T15:15:47.554 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 0, 2024-07-31T15:15:47.554 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 0, 2024-07-31T15:15:47.554 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 0, 2024-07-31T15:15:47.554 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 0, 2024-07-31T15:15:47.554 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-07-31T15:15:47.554 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-07-31T15:15:47.554 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 0, 2024-07-31T15:15:47.554 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-07-31T15:15:47.554 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-07-31T15:15:47.555 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 0, 2024-07-31T15:15:47.555 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 0, 2024-07-31T15:15:47.555 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 0, 2024-07-31T15:15:47.555 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 0, 2024-07-31T15:15:47.555 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-07-31T15:15:47.555 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 0, 2024-07-31T15:15:47.555 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-07-31T15:15:47.555 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 91397, 2024-07-31T15:15:47.556 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 2022860651, 2024-07-31T15:15:47.556 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 7, 2024-07-31T15:15:47.556 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 0, 2024-07-31T15:15:47.556 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-07-31T15:15:47.556 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-07-31T15:15:47.556 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-07-31T15:15:47.556 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 0, 2024-07-31T15:15:47.556 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-07-31T15:15:47.557 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 0, 2024-07-31T15:15:47.557 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 0, 2024-07-31T15:15:47.557 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 0, 2024-07-31T15:15:47.557 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 0, 2024-07-31T15:15:47.557 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 0, 2024-07-31T15:15:47.557 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-07-31T15:15:47.557 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-31T15:15:47.557 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-31T15:15:47.558 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 0, 2024-07-31T15:15:47.558 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 0, 2024-07-31T15:15:47.558 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 0, 2024-07-31T15:15:47.558 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 0 2024-07-31T15:15:47.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.558 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-07-31T15:15:47.558 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 0, 2024-07-31T15:15:47.559 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 0, 2024-07-31T15:15:47.559 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 0, 2024-07-31T15:15:47.559 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 0, 2024-07-31T15:15:47.559 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 0, 2024-07-31T15:15:47.559 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 0, 2024-07-31T15:15:47.559 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-07-31T15:15:47.559 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-07-31T15:15:47.559 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 0, 2024-07-31T15:15:47.559 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 0, 2024-07-31T15:15:47.560 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-07-31T15:15:47.560 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-07-31T15:15:47.560 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-07-31T15:15:47.560 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-07-31T15:15:47.560 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-07-31T15:15:47.560 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-07-31T15:15:47.560 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-07-31T15:15:47.560 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 0, 2024-07-31T15:15:47.561 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 0, 2024-07-31T15:15:47.561 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-07-31T15:15:47.561 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 0, 2024-07-31T15:15:47.561 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-07-31T15:15:47.561 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 0, 2024-07-31T15:15:47.561 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-07-31T15:15:47.561 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-07-31T15:15:47.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.562 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-07-31T15:15:47.562 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 0, 2024-07-31T15:15:47.562 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 0, 2024-07-31T15:15:47.562 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 0, 2024-07-31T15:15:47.562 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 0, 2024-07-31T15:15:47.562 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 14194, 2024-07-31T15:15:47.562 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-07-31T15:15:47.562 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 0, 2024-07-31T15:15:47.563 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 0, 2024-07-31T15:15:47.563 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 0, 2024-07-31T15:15:47.563 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 0, 2024-07-31T15:15:47.563 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 3, 2024-07-31T15:15:47.563 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 17, 2024-07-31T15:15:47.563 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-07-31T15:15:47.563 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 0, 2024-07-31T15:15:47.563 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 590829638, 2024-07-31T15:15:47.563 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 592286197, 2024-07-31T15:15:47.564 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 592272228, 2024-07-31T15:15:47.564 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-07-31T15:15:47.564 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.564 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.564 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.564 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 14194 2024-07-31T15:15:47.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.565 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-07-31T15:15:47.565 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 6398, 2024-07-31T15:15:47.565 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 7154, 2024-07-31T15:15:47.565 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 756, 2024-07-31T15:15:47.565 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 1344, 2024-07-31T15:15:47.565 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 1360, 2024-07-31T15:15:47.565 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 16, 2024-07-31T15:15:47.566 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 7886, 2024-07-31T15:15:47.566 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 7891, 2024-07-31T15:15:47.566 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 5, 2024-07-31T15:15:47.566 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 0, 2024-07-31T15:15:47.566 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 0, 2024-07-31T15:15:47.566 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 0, 2024-07-31T15:15:47.566 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 145372, 2024-07-31T15:15:47.567 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 215348 2024-07-31T15:15:47.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.567 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-07-31T15:15:47.567 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 0, 2024-07-31T15:15:47.567 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 0, 2024-07-31T15:15:47.567 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 0, 2024-07-31T15:15:47.567 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 0, 2024-07-31T15:15:47.567 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 0, 2024-07-31T15:15:47.568 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-07-31T15:15:47.568 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.568 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.568 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.568 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-07-31T15:15:47.568 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.568 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.569 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.569 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-07-31T15:15:47.569 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.569 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.569 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.569 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-07-31T15:15:47.570 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.570 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.570 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.570 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-07-31T15:15:47.570 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.570 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.570 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.571 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-07-31T15:15:47.571 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.571 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.571 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.571 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-07-31T15:15:47.571 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.571 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.572 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.572 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-07-31T15:15:47.572 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.572 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.572 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.572 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-07-31T15:15:47.572 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.573 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.573 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.573 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-07-31T15:15:47.573 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.573 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.573 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.574 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-07-31T15:15:47.574 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.574 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.574 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.575 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:47 smithi067 ceph-mon[32750]: pgmap v1189: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-07-31T15:15:47.575 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-07-31T15:15:47.575 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.575 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.575 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.576 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-07-31T15:15:47.576 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.576 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.576 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.576 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-07-31T15:15:47.576 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.577 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.577 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.577 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-07-31T15:15:47.577 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.577 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.577 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-07-31T15:15:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-07-31T15:15:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-07-31T15:15:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-07-31T15:15:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-07-31T15:15:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-07-31T15:15:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-07-31T15:15:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-07-31T15:15:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-07-31T15:15:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-07-31T15:15:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-07-31T15:15:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-07-31T15:15:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-07-31T15:15:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-07-31T15:15:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-07-31T15:15:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-07-31T15:15:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-07-31T15:15:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.587 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.587 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.587 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:47.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.587 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-07-31T15:15:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 2, 2024-07-31T15:15:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 2, 2024-07-31T15:15:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 0, 2024-07-31T15:15:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 0, 2024-07-31T15:15:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-07-31T15:15:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 0, 2024-07-31T15:15:47.589 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 0, 2024-07-31T15:15:47.589 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 2976, 2024-07-31T15:15:47.589 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-07-31T15:15:47.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.589 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-07-31T15:15:47.589 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-07-31T15:15:47.589 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-07-31T15:15:47.589 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-07-31T15:15:47.590 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-07-31T15:15:47.590 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-07-31T15:15:47.590 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-07-31T15:15:47.590 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-31T15:15:47.590 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-07-31T15:15:47.590 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-31T15:15:47.590 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-07-31T15:15:47.591 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-07-31T15:15:47.591 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-07-31T15:15:47.591 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-31T15:15:47.591 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-07-31T15:15:47.591 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-07-31T15:15:47.591 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-07-31T15:15:47.591 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-07-31T15:15:47.591 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-07-31T15:15:47.592 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-31T15:15:47.592 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-07-31T15:15:47.592 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-31T15:15:47.592 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-07-31T15:15:47.592 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-07-31T15:15:47.592 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-07-31T15:15:47.592 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-07-31T15:15:47.593 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-07-31T15:15:47.593 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-31T15:15:47.593 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-07-31T15:15:47.593 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-07-31T15:15:47.593 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-07-31T15:15:47.593 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-07-31T15:15:47.593 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-07-31T15:15:47.593 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-07-31T15:15:47.593 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-07-31T15:15:47.594 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-07-31T15:15:47.594 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-07-31T15:15:47.594 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 512166, 2024-07-31T15:15:47.594 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 7677, 2024-07-31T15:15:47.594 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 0, 2024-07-31T15:15:47.594 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 0, 2024-07-31T15:15:47.594 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-07-31T15:15:47.594 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-07-31T15:15:47.595 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-07-31T15:15:47.595 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-07-31T15:15:47.595 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-07-31T15:15:47.595 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-07-31T15:15:47.595 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 9352, 2024-07-31T15:15:47.596 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 189, 2024-07-31T15:15:47.596 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-07-31T15:15:47.596 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-07-31T15:15:47.596 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-07-31T15:15:47.596 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-07-31T15:15:47.596 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 23183674, 2024-07-31T15:15:47.596 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 237103, 2024-07-31T15:15:47.596 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-07-31T15:15:47.597 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-07-31T15:15:47.597 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-07-31T15:15:47.597 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-07-31T15:15:47.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.597 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-07-31T15:15:47.597 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-07-31T15:15:47.597 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-07-31T15:15:47.597 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 1109, 2024-07-31T15:15:47.598 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 26, 2024-07-31T15:15:47.598 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 4, 2024-07-31T15:15:47.598 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 1109, 2024-07-31T15:15:47.598 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-07-31T15:15:47.598 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1109, 2024-07-31T15:15:47.598 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 4.906893274, 2024-07-31T15:15:47.598 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.004424610 2024-07-31T15:15:47.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.599 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-07-31T15:15:47.599 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-07-31T15:15:47.599 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1109, 2024-07-31T15:15:47.599 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1110 2024-07-31T15:15:47.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.599 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 1109, 2024-07-31T15:15:47.599 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 1109, 2024-07-31T15:15:47.599 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 0, 2024-07-31T15:15:47.599 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-07-31T15:15:47.600 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-07-31T15:15:47.600 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 627, 2024-07-31T15:15:47.600 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 0, 2024-07-31T15:15:47.600 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 479, 2024-07-31T15:15:47.600 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 0, 2024-07-31T15:15:47.600 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 0, 2024-07-31T15:15:47.600 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-07-31T15:15:47.600 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-07-31T15:15:47.601 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-07-31T15:15:47.601 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 0, 2024-07-31T15:15:47.601 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 0, 2024-07-31T15:15:47.601 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-07-31T15:15:47.601 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-07-31T15:15:47.601 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-07-31T15:15:47.601 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 2, 2024-07-31T15:15:47.601 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 0, 2024-07-31T15:15:47.602 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-07-31T15:15:47.602 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-07-31T15:15:47.602 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-07-31T15:15:47.602 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-07-31T15:15:47.602 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-07-31T15:15:47.602 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-07-31T15:15:47.602 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-07-31T15:15:47.602 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-07-31T15:15:47.602 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-07-31T15:15:47.603 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-07-31T15:15:47.603 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-07-31T15:15:47.603 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-07-31T15:15:47.603 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-07-31T15:15:47.603 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-07-31T15:15:47.603 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-07-31T15:15:47.603 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-07-31T15:15:47.603 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-07-31T15:15:47.603 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-07-31T15:15:47.604 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-07-31T15:15:47.604 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-07-31T15:15:47.604 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-07-31T15:15:47.604 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-07-31T15:15:47.604 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-07-31T15:15:47.604 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-07-31T15:15:47.604 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-07-31T15:15:47.604 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-07-31T15:15:47.604 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 138, 2024-07-31T15:15:47.605 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-07-31T15:15:47.605 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 10, 2024-07-31T15:15:47.605 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 10, 2024-07-31T15:15:47.605 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 10, 2024-07-31T15:15:47.605 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 0, 2024-07-31T15:15:47.605 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-07-31T15:15:47.605 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 0, 2024-07-31T15:15:47.605 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 2, 2024-07-31T15:15:47.605 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 0 2024-07-31T15:15:47.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.606 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-07-31T15:15:47.606 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 0, 2024-07-31T15:15:47.606 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-07-31T15:15:47.606 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-07-31T15:15:47.606 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-07-31T15:15:47.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.607 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-07-31T15:15:47.607 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 0 2024-07-31T15:15:47.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.607 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-07-31T15:15:47.607 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-07-31T15:15:47.607 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 0, 2024-07-31T15:15:47.607 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-07-31T15:15:47.607 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 0, 2024-07-31T15:15:47.608 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 0, 2024-07-31T15:15:47.608 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 0, 2024-07-31T15:15:47.608 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-07-31T15:15:47.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.608 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-31T15:15:47.608 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:47.608 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-07-31T15:15:47.608 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:47.608 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 1214, 2024-07-31T15:15:47.609 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 68568571, 2024-07-31T15:15:47.609 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:47.609 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 1214, 2024-07-31T15:15:47.609 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-07-31T15:15:47.609 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-07-31T15:15:47.609 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 1214, 2024-07-31T15:15:47.609 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 68568571, 2024-07-31T15:15:47.609 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:47.609 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.610 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.610 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.610 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:47.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.610 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-07-31T15:15:47.610 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:47.610 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-07-31T15:15:47.610 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:47.610 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-07-31T15:15:47.611 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-07-31T15:15:47.611 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:47.611 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:47.611 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 1109, 2024-07-31T15:15:47.611 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 68165747, 2024-07-31T15:15:47.611 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 1109, 2024-07-31T15:15:47.611 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 68165747, 2024-07-31T15:15:47.611 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:47.611 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.612 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.612 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.612 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:47.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.612 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-07-31T15:15:47.612 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:47.612 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-07-31T15:15:47.612 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:47.613 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-07-31T15:15:47.613 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-07-31T15:15:47.613 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:47.613 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:47.613 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 1109, 2024-07-31T15:15:47.613 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 1109, 2024-07-31T15:15:47.613 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 1109, 2024-07-31T15:15:47.613 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 1109, 2024-07-31T15:15:47.613 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:47.614 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.614 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.614 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.614 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:47.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.614 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-07-31T15:15:47.614 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:47.614 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-07-31T15:15:47.615 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:47.615 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-07-31T15:15:47.615 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-07-31T15:15:47.615 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:47.615 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:47.615 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-07-31T15:15:47.615 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-07-31T15:15:47.615 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-07-31T15:15:47.615 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-07-31T15:15:47.615 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:47.616 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.616 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.616 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.616 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:47.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:47.616 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x563991a24820": { 2024-07-31T15:15:47.616 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:47.616 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-07-31T15:15:47.617 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:47.617 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-07-31T15:15:47.617 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-07-31T15:15:47.617 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:47.617 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:47.617 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-07-31T15:15:47.617 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-07-31T15:15:47.617 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-07-31T15:15:47.618 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-07-31T15:15:47.618 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:47.618 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:47.618 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:47.618 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:47.618 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:47.618 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:47.618 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-07-31T15:15:48.520 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-07-31T15:15:48.521 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-07-31T15:15:48.521 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=7 2024-07-31T15:15:48.521 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-07-31T15:15:48.521 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-07-31T15:15:48.521 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-07-31T15:15:48.521 DEBUG:tasks.check_counter:Getting stats from h 2024-07-31T15:15:48.522 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph daemon mds.h perf dump 2024-07-31T15:15:48.775 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.b/config 2024-07-31T15:15:49.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:49 smithi077 ceph-mon[37327]: pgmap v1190: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-07-31T15:15:49.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:49 smithi067 ceph-mon[32750]: pgmap v1190: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-07-31T15:15:49.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:49 smithi073 ceph-mon[36714]: pgmap v1190: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-07-31T15:15:50.456 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-07-31T15:15:50.457 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-07-31T15:15:50.457 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 114678, 2024-07-31T15:15:50.457 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 137969, 2024-07-31T15:15:50.457 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 40426177, 2024-07-31T15:15:50.457 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 71629795, 2024-07-31T15:15:50.458 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 54, 2024-07-31T15:15:50.458 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 7, 2024-07-31T15:15:50.458 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 7.387610607, 2024-07-31T15:15:50.458 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 2.608334171, 2024-07-31T15:15:50.458 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 4.396438096, 2024-07-31T15:15:50.458 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 1.744551661, 2024-07-31T15:15:50.458 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:50.459 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 137966, 2024-07-31T15:15:50.459 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 14.553912678, 2024-07-31T15:15:50.459 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000105489 2024-07-31T15:15:50.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.459 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:50.459 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 100116, 2024-07-31T15:15:50.459 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.021031717, 2024-07-31T15:15:50.459 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000210 2024-07-31T15:15:50.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.460 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 21195552, 2024-07-31T15:15:50.460 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 39278112 2024-07-31T15:15:50.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.460 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-07-31T15:15:50.461 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 105118, 2024-07-31T15:15:50.461 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 100772, 2024-07-31T15:15:50.461 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 37351140, 2024-07-31T15:15:50.461 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 54984334, 2024-07-31T15:15:50.461 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 11, 2024-07-31T15:15:50.461 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 6, 2024-07-31T15:15:50.461 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 7.162146772, 2024-07-31T15:15:50.461 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 2.635662234, 2024-07-31T15:15:50.462 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 2.522028787, 2024-07-31T15:15:50.462 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 1.956134085, 2024-07-31T15:15:50.462 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:50.462 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 100772, 2024-07-31T15:15:50.462 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 5.342203159, 2024-07-31T15:15:50.462 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000053012 2024-07-31T15:15:50.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.462 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:50.463 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 54562, 2024-07-31T15:15:50.463 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.010221252, 2024-07-31T15:15:50.463 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000187 2024-07-31T15:15:50.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.463 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-31T15:15:50.463 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 694192 2024-07-31T15:15:50.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.464 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-07-31T15:15:50.464 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 49084, 2024-07-31T15:15:50.464 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 48996, 2024-07-31T15:15:50.464 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 14572307, 2024-07-31T15:15:50.464 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 33966856, 2024-07-31T15:15:50.464 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 11, 2024-07-31T15:15:50.464 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 7, 2024-07-31T15:15:50.465 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 3.302852795, 2024-07-31T15:15:50.465 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 1.011544979, 2024-07-31T15:15:50.465 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.813177209, 2024-07-31T15:15:50.465 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 1.234203043, 2024-07-31T15:15:50.465 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:50.465 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 48995, 2024-07-31T15:15:50.465 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1.997867988, 2024-07-31T15:15:50.465 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000040776 2024-07-31T15:15:50.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.466 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:50.466 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 4069, 2024-07-31T15:15:50.466 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000763607, 2024-07-31T15:15:50.466 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000187 2024-07-31T15:15:50.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.466 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 7968, 2024-07-31T15:15:50.466 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 28656 2024-07-31T15:15:50.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.467 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-07-31T15:15:50.467 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-07-31T15:15:50.467 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-07-31T15:15:50.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.467 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-07-31T15:15:50.467 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-07-31T15:15:50.468 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-07-31T15:15:50.468 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 143676, 2024-07-31T15:15:50.468 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 45.926125945, 2024-07-31T15:15:50.468 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000319650 2024-07-31T15:15:50.468 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:50.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.468 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-07-31T15:15:50.468 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-07-31T15:15:50.469 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-07-31T15:15:50.469 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 550, 2024-07-31T15:15:50.469 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.039844769, 2024-07-31T15:15:50.469 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000072445 2024-07-31T15:15:50.469 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:50.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.469 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-07-31T15:15:50.469 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 13289, 2024-07-31T15:15:50.469 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 9703, 2024-07-31T15:15:50.470 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-07-31T15:15:50.470 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 9703, 2024-07-31T15:15:50.470 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 88.601853023, 2024-07-31T15:15:50.470 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.009131387 2024-07-31T15:15:50.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.471 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-07-31T15:15:50.471 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 3586, 2024-07-31T15:15:50.471 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 2, 2024-07-31T15:15:50.471 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-07-31T15:15:50.471 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 172, 2024-07-31T15:15:50.471 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 18, 2024-07-31T15:15:50.471 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 0, 2024-07-31T15:15:50.472 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 12536, 2024-07-31T15:15:50.472 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 299, 2024-07-31T15:15:50.472 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 12223, 2024-07-31T15:15:50.472 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 14, 2024-07-31T15:15:50.472 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 12360, 2024-07-31T15:15:50.472 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 856, 2024-07-31T15:15:50.472 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 57, 2024-07-31T15:15:50.473 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 57, 2024-07-31T15:15:50.473 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 221, 2024-07-31T15:15:50.473 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 16354, 2024-07-31T15:15:50.473 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 9906, 2024-07-31T15:15:50.473 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 3538, 2024-07-31T15:15:50.473 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 285, 2024-07-31T15:15:50.473 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-07-31T15:15:50.473 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-07-31T15:15:50.474 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 0, 2024-07-31T15:15:50.474 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-07-31T15:15:50.474 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-07-31T15:15:50.474 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 53, 2024-07-31T15:15:50.474 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 877, 2024-07-31T15:15:50.474 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 239, 2024-07-31T15:15:50.474 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 6095, 2024-07-31T15:15:50.475 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-07-31T15:15:50.475 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 177851, 2024-07-31T15:15:50.475 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-07-31T15:15:50.475 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 91397, 2024-07-31T15:15:50.475 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 2022860651, 2024-07-31T15:15:50.475 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 7, 2024-07-31T15:15:50.475 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 177851, 2024-07-31T15:15:50.475 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-07-31T15:15:50.476 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-07-31T15:15:50.476 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-07-31T15:15:50.476 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 7632, 2024-07-31T15:15:50.476 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-07-31T15:15:50.476 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 7632, 2024-07-31T15:15:50.476 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 170219, 2024-07-31T15:15:50.476 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 114, 2024-07-31T15:15:50.476 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 24265, 2024-07-31T15:15:50.477 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 9666, 2024-07-31T15:15:50.477 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 9, 2024-07-31T15:15:50.477 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-31T15:15:50.477 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-31T15:15:50.477 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 34337, 2024-07-31T15:15:50.477 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 11194, 2024-07-31T15:15:50.477 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 429, 2024-07-31T15:15:50.477 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 1632 2024-07-31T15:15:50.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.478 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-07-31T15:15:50.478 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 0, 2024-07-31T15:15:50.478 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 0, 2024-07-31T15:15:50.478 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 0, 2024-07-31T15:15:50.478 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 270, 2024-07-31T15:15:50.478 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 75, 2024-07-31T15:15:50.479 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 0, 2024-07-31T15:15:50.479 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-07-31T15:15:50.479 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-07-31T15:15:50.479 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 194, 2024-07-31T15:15:50.479 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 194, 2024-07-31T15:15:50.479 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-07-31T15:15:50.479 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-07-31T15:15:50.479 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-07-31T15:15:50.480 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-07-31T15:15:50.480 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-07-31T15:15:50.480 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-07-31T15:15:50.480 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-07-31T15:15:50.480 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 16, 2024-07-31T15:15:50.480 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 164423, 2024-07-31T15:15:50.480 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-07-31T15:15:50.480 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 53, 2024-07-31T15:15:50.481 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-07-31T15:15:50.481 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 18, 2024-07-31T15:15:50.481 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-07-31T15:15:50.481 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-07-31T15:15:50.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.481 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-07-31T15:15:50.482 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 1, 2024-07-31T15:15:50.482 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 22557, 2024-07-31T15:15:50.482 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 0, 2024-07-31T15:15:50.482 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 0, 2024-07-31T15:15:50.482 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 22557, 2024-07-31T15:15:50.482 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-07-31T15:15:50.482 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 0, 2024-07-31T15:15:50.483 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 29, 2024-07-31T15:15:50.483 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 0, 2024-07-31T15:15:50.483 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 0, 2024-07-31T15:15:50.483 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 29, 2024-07-31T15:15:50.483 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 3, 2024-07-31T15:15:50.483 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-07-31T15:15:50.483 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 0, 2024-07-31T15:15:50.483 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 4194304, 2024-07-31T15:15:50.483 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 55534834, 2024-07-31T15:15:50.484 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 0, 2024-07-31T15:15:50.484 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-07-31T15:15:50.484 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 3556, 2024-07-31T15:15:50.484 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 104.609961710, 2024-07-31T15:15:50.484 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.029417874 2024-07-31T15:15:50.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.484 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 0 2024-07-31T15:15:50.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.484 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-07-31T15:15:50.485 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 12169, 2024-07-31T15:15:50.485 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 13218, 2024-07-31T15:15:50.485 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 1049, 2024-07-31T15:15:50.485 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 1079, 2024-07-31T15:15:50.485 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 1131, 2024-07-31T15:15:50.485 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 52, 2024-07-31T15:15:50.485 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 12536, 2024-07-31T15:15:50.485 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 13887, 2024-07-31T15:15:50.486 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 1351, 2024-07-31T15:15:50.486 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 57, 2024-07-31T15:15:50.486 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 24857, 2024-07-31T15:15:50.486 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 24800, 2024-07-31T15:15:50.486 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 231980, 2024-07-31T15:15:50.486 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 223540 2024-07-31T15:15:50.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.486 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-07-31T15:15:50.487 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 14300, 2024-07-31T15:15:50.487 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 6, 2024-07-31T15:15:50.487 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 13289, 2024-07-31T15:15:50.487 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 95, 2024-07-31T15:15:50.487 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 845, 2024-07-31T15:15:50.487 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-07-31T15:15:50.487 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 6406, 2024-07-31T15:15:50.487 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 81.011199863, 2024-07-31T15:15:50.487 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.012646144 2024-07-31T15:15:50.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.488 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-07-31T15:15:50.488 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 6, 2024-07-31T15:15:50.488 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 3.053388880, 2024-07-31T15:15:50.488 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.508898146 2024-07-31T15:15:50.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.488 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-07-31T15:15:50.488 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.489 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.489 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.489 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-07-31T15:15:50.489 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.489 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.489 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.490 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-07-31T15:15:50.490 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1807, 2024-07-31T15:15:50.490 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 2.458171810, 2024-07-31T15:15:50.490 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001360360 2024-07-31T15:15:50.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.491 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-07-31T15:15:50.491 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.491 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.491 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.491 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-07-31T15:15:50.492 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.492 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.492 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.492 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-07-31T15:15:50.492 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.492 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.492 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.493 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-07-31T15:15:50.493 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.493 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.493 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.493 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-07-31T15:15:50.493 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.493 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.494 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.495 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-07-31T15:15:50.495 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.495 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.495 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.495 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-07-31T15:15:50.495 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 242, 2024-07-31T15:15:50.496 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.197456228, 2024-07-31T15:15:50.496 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000815934 2024-07-31T15:15:50.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.496 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-07-31T15:15:50.496 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.496 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.496 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.497 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-07-31T15:15:50.497 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.497 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.497 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.497 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-07-31T15:15:50.497 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 285, 2024-07-31T15:15:50.497 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.137376424, 2024-07-31T15:15:50.498 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000482022 2024-07-31T15:15:50.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.498 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-07-31T15:15:50.498 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 754, 2024-07-31T15:15:50.498 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1.110429684, 2024-07-31T15:15:50.498 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001472718 2024-07-31T15:15:50.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.499 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-07-31T15:15:50.499 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.499 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.499 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.499 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-07-31T15:15:50.499 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.499 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.499 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.500 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-07-31T15:15:50.500 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.500 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.500 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.500 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-07-31T15:15:50.500 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.501 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.501 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.501 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-07-31T15:15:50.501 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.501 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.502 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.502 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-07-31T15:15:50.502 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.502 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.503 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.503 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-07-31T15:15:50.503 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 9, 2024-07-31T15:15:50.503 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.213638611, 2024-07-31T15:15:50.503 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.023737623 2024-07-31T15:15:50.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.503 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-07-31T15:15:50.503 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.504 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.504 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.504 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-07-31T15:15:50.504 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.504 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.504 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.505 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-07-31T15:15:50.505 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.505 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.505 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.505 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-07-31T15:15:50.505 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.506 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.506 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.506 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-07-31T15:15:50.506 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.506 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.506 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.507 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-07-31T15:15:50.507 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 194, 2024-07-31T15:15:50.507 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.420191523, 2024-07-31T15:15:50.507 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.002165935 2024-07-31T15:15:50.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.507 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-07-31T15:15:50.507 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-07-31T15:15:50.507 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-07-31T15:15:50.508 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.508 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.508 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.508 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:50.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.508 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-07-31T15:15:50.508 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 2, 2024-07-31T15:15:50.508 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 4, 2024-07-31T15:15:50.509 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 2, 2024-07-31T15:15:50.509 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 2, 2024-07-31T15:15:50.509 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-07-31T15:15:50.509 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 329, 2024-07-31T15:15:50.509 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 164, 2024-07-31T15:15:50.509 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 1429, 2024-07-31T15:15:50.509 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-07-31T15:15:50.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.510 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-07-31T15:15:50.510 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-07-31T15:15:50.510 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-07-31T15:15:50.510 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-07-31T15:15:50.510 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-07-31T15:15:50.510 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-07-31T15:15:50.510 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-07-31T15:15:50.511 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-31T15:15:50.511 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-07-31T15:15:50.511 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-31T15:15:50.511 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-07-31T15:15:50.511 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-07-31T15:15:50.512 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-07-31T15:15:50.512 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-31T15:15:50.512 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-07-31T15:15:50.512 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-07-31T15:15:50.512 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-07-31T15:15:50.512 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-07-31T15:15:50.513 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-07-31T15:15:50.513 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-31T15:15:50.513 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-07-31T15:15:50.513 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-31T15:15:50.513 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-07-31T15:15:50.513 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-07-31T15:15:50.513 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-07-31T15:15:50.514 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-07-31T15:15:50.514 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-07-31T15:15:50.514 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-31T15:15:50.514 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-07-31T15:15:50.514 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-07-31T15:15:50.514 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-07-31T15:15:50.514 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-07-31T15:15:50.515 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-07-31T15:15:50.515 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-07-31T15:15:50.515 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-07-31T15:15:50.515 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-07-31T15:15:50.515 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-07-31T15:15:50.515 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 756696, 2024-07-31T15:15:50.515 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 12139, 2024-07-31T15:15:50.515 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 0, 2024-07-31T15:15:50.516 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 0, 2024-07-31T15:15:50.516 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-07-31T15:15:50.516 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-07-31T15:15:50.516 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-07-31T15:15:50.516 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-07-31T15:15:50.516 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-07-31T15:15:50.516 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-07-31T15:15:50.516 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 9352, 2024-07-31T15:15:50.517 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 189, 2024-07-31T15:15:50.517 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-07-31T15:15:50.517 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-07-31T15:15:50.517 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-07-31T15:15:50.517 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-07-31T15:15:50.517 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 39914153, 2024-07-31T15:15:50.517 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 605569, 2024-07-31T15:15:50.517 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-07-31T15:15:50.518 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-07-31T15:15:50.518 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-07-31T15:15:50.518 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-07-31T15:15:50.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.518 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-07-31T15:15:50.518 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-07-31T15:15:50.518 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-07-31T15:15:50.519 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 182642, 2024-07-31T15:15:50.519 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 56128653, 2024-07-31T15:15:50.519 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 28, 2024-07-31T15:15:50.519 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 182642, 2024-07-31T15:15:50.519 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-07-31T15:15:50.519 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 182642, 2024-07-31T15:15:50.519 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 329.601669079, 2024-07-31T15:15:50.519 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001804632 2024-07-31T15:15:50.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.520 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-07-31T15:15:50.520 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-07-31T15:15:50.520 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 182642, 2024-07-31T15:15:50.520 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 183302 2024-07-31T15:15:50.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.520 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 182642, 2024-07-31T15:15:50.521 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 177903, 2024-07-31T15:15:50.521 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 4739, 2024-07-31T15:15:50.521 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-07-31T15:15:50.521 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-07-31T15:15:50.521 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 0, 2024-07-31T15:15:50.521 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 10, 2024-07-31T15:15:50.521 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 49, 2024-07-31T15:15:50.522 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 3605, 2024-07-31T15:15:50.522 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 107, 2024-07-31T15:15:50.522 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-07-31T15:15:50.522 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-07-31T15:15:50.522 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-07-31T15:15:50.522 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 18, 2024-07-31T15:15:50.522 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 412, 2024-07-31T15:15:50.523 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-07-31T15:15:50.523 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-07-31T15:15:50.523 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-07-31T15:15:50.523 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 177853, 2024-07-31T15:15:50.523 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 20, 2024-07-31T15:15:50.523 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-07-31T15:15:50.524 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-07-31T15:15:50.524 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-07-31T15:15:50.524 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-07-31T15:15:50.524 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-07-31T15:15:50.524 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-07-31T15:15:50.524 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-07-31T15:15:50.524 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-07-31T15:15:50.524 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-07-31T15:15:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 9, 2024-07-31T15:15:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-07-31T15:15:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-07-31T15:15:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-07-31T15:15:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-07-31T15:15:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-07-31T15:15:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-07-31T15:15:50.525 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-07-31T15:15:50.526 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-07-31T15:15:50.526 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-07-31T15:15:50.526 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-07-31T15:15:50.526 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-07-31T15:15:50.526 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-07-31T15:15:50.526 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-07-31T15:15:50.527 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-07-31T15:15:50.527 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-07-31T15:15:50.527 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-07-31T15:15:50.527 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 138, 2024-07-31T15:15:50.527 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-07-31T15:15:50.527 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 49, 2024-07-31T15:15:50.527 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 12, 2024-07-31T15:15:50.527 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 12, 2024-07-31T15:15:50.528 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 0, 2024-07-31T15:15:50.528 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-07-31T15:15:50.528 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 1007, 2024-07-31T15:15:50.528 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 5, 2024-07-31T15:15:50.528 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 207 2024-07-31T15:15:50.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.528 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-07-31T15:15:50.529 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 1, 2024-07-31T15:15:50.529 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-07-31T15:15:50.529 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-07-31T15:15:50.529 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-07-31T15:15:50.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.529 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-07-31T15:15:50.529 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 164417 2024-07-31T15:15:50.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.530 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-07-31T15:15:50.530 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-07-31T15:15:50.530 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 10, 2024-07-31T15:15:50.530 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-07-31T15:15:50.530 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 5, 2024-07-31T15:15:50.530 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 388, 2024-07-31T15:15:50.530 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 194, 2024-07-31T15:15:50.531 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-07-31T15:15:50.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.531 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-31T15:15:50.531 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:50.531 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-07-31T15:15:50.531 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:50.531 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 268876, 2024-07-31T15:15:50.531 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 65660578, 2024-07-31T15:15:50.532 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:50.532 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 268876, 2024-07-31T15:15:50.532 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-07-31T15:15:50.532 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-07-31T15:15:50.532 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 268876, 2024-07-31T15:15:50.532 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 65660578, 2024-07-31T15:15:50.532 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:50.532 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.533 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.533 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.533 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:50.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.533 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-07-31T15:15:50.533 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:50.534 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-07-31T15:15:50.534 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:50.534 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-07-31T15:15:50.534 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-07-31T15:15:50.534 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:50.534 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:50.535 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 182642, 2024-07-31T15:15:50.535 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 56149717, 2024-07-31T15:15:50.535 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 182642, 2024-07-31T15:15:50.535 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 56149717, 2024-07-31T15:15:50.535 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:50.535 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.535 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.535 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.536 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:50.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.536 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-07-31T15:15:50.536 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:50.536 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-07-31T15:15:50.536 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:50.536 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-07-31T15:15:50.536 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-07-31T15:15:50.537 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:50.537 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:50.537 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 182642, 2024-07-31T15:15:50.537 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 182642, 2024-07-31T15:15:50.537 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 182642, 2024-07-31T15:15:50.537 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 182642, 2024-07-31T15:15:50.537 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:50.537 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.538 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.538 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.538 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:50.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.538 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-07-31T15:15:50.538 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:50.538 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-07-31T15:15:50.539 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:50.539 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 194, 2024-07-31T15:15:50.539 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 21146, 2024-07-31T15:15:50.539 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:50.539 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 194, 2024-07-31T15:15:50.539 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-07-31T15:15:50.539 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-07-31T15:15:50.539 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 49, 2024-07-31T15:15:50.540 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 21146, 2024-07-31T15:15:50.540 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:50.540 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.540 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.540 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.540 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:50.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:50.540 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x562928ba8820": { 2024-07-31T15:15:50.540 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:50.541 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-07-31T15:15:50.541 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:50.541 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 22557, 2024-07-31T15:15:50.541 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 51340530, 2024-07-31T15:15:50.541 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:50.541 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 22557, 2024-07-31T15:15:50.542 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-07-31T15:15:50.542 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-07-31T15:15:50.542 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 3556, 2024-07-31T15:15:50.542 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 51340530, 2024-07-31T15:15:50.542 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:50.542 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:50.542 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:50.543 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:50.543 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:50.543 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:50.543 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-07-31T15:15:51.230 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:51 smithi073 ceph-mon[36714]: pgmap v1191: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-07-31T15:15:51.259 INFO:tasks.check_counter:Daemon mds.h mds.exported=53 2024-07-31T15:15:51.259 INFO:tasks.check_counter:Daemon mds.h mds.imported=239 2024-07-31T15:15:51.259 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-07-31T15:15:51.259 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-07-31T15:15:51.260 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-07-31T15:15:51.260 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=18 2024-07-31T15:15:51.260 DEBUG:tasks.check_counter:Getting stats from k 2024-07-31T15:15:51.260 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph daemon mds.k perf dump 2024-07-31T15:15:51.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:51 smithi077 ceph-mon[37327]: pgmap v1191: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-07-31T15:15:51.512 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.b/config 2024-07-31T15:15:51.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:51 smithi067 ceph-mon[32750]: pgmap v1191: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-07-31T15:15:53.286 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:53 smithi073 ceph-mon[36714]: pgmap v1192: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-31T15:15:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-07-31T15:15:53.287 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-07-31T15:15:53.287 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 639, 2024-07-31T15:15:53.287 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 591, 2024-07-31T15:15:53.287 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 10065500, 2024-07-31T15:15:53.287 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 321249, 2024-07-31T15:15:53.287 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 6, 2024-07-31T15:15:53.287 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 2, 2024-07-31T15:15:53.288 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.084693188, 2024-07-31T15:15:53.288 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.031239913, 2024-07-31T15:15:53.288 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.069820152, 2024-07-31T15:15:53.288 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.001627768, 2024-07-31T15:15:53.288 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:53.288 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 591, 2024-07-31T15:15:53.288 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.034292118, 2024-07-31T15:15:53.289 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000058023 2024-07-31T15:15:53.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.289 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:53.289 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.289 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.289 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.290 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 1659936, 2024-07-31T15:15:53.290 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 300256 2024-07-31T15:15:53.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.290 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-07-31T15:15:53.290 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 2343, 2024-07-31T15:15:53.290 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 2700, 2024-07-31T15:15:53.290 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 18383225, 2024-07-31T15:15:53.291 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 1436170, 2024-07-31T15:15:53.291 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 7, 2024-07-31T15:15:53.291 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-07-31T15:15:53.291 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.191530334, 2024-07-31T15:15:53.291 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.054125116, 2024-07-31T15:15:53.291 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.056236271, 2024-07-31T15:15:53.291 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.066660703, 2024-07-31T15:15:53.291 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:53.291 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2698, 2024-07-31T15:15:53.292 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.119966805, 2024-07-31T15:15:53.292 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000044465 2024-07-31T15:15:53.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.292 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:53.292 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.292 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.292 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.293 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-31T15:15:53.293 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 691408 2024-07-31T15:15:53.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.293 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-07-31T15:15:53.293 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 2613, 2024-07-31T15:15:53.293 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 2615, 2024-07-31T15:15:53.293 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 25963559, 2024-07-31T15:15:53.293 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 830992, 2024-07-31T15:15:53.294 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 6, 2024-07-31T15:15:53.294 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-07-31T15:15:53.294 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.272694489, 2024-07-31T15:15:53.294 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.074887207, 2024-07-31T15:15:53.294 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.079253493, 2024-07-31T15:15:53.294 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.098505543, 2024-07-31T15:15:53.294 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:53.294 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2613, 2024-07-31T15:15:53.295 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.106933893, 2024-07-31T15:15:53.295 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000040923 2024-07-31T15:15:53.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.295 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:53.295 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.295 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.295 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.296 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-31T15:15:53.296 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 0 2024-07-31T15:15:53.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.296 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-07-31T15:15:53.296 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-07-31T15:15:53.296 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-07-31T15:15:53.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.296 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-07-31T15:15:53.297 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-07-31T15:15:53.297 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-07-31T15:15:53.297 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 8424, 2024-07-31T15:15:53.297 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.270365930, 2024-07-31T15:15:53.297 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000032094 2024-07-31T15:15:53.297 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:53.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.297 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-07-31T15:15:53.297 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-07-31T15:15:53.298 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-07-31T15:15:53.298 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.298 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.298 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.298 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:53.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.298 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-07-31T15:15:53.299 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 0, 2024-07-31T15:15:53.299 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 0, 2024-07-31T15:15:53.299 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-07-31T15:15:53.299 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.299 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.299 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.299 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-07-31T15:15:53.300 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 0, 2024-07-31T15:15:53.300 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 0, 2024-07-31T15:15:53.300 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-07-31T15:15:53.300 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 0, 2024-07-31T15:15:53.300 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 0, 2024-07-31T15:15:53.300 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 0, 2024-07-31T15:15:53.300 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 12513, 2024-07-31T15:15:53.300 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 78, 2024-07-31T15:15:53.300 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 12435, 2024-07-31T15:15:53.301 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 0, 2024-07-31T15:15:53.301 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 12402, 2024-07-31T15:15:53.301 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 0, 2024-07-31T15:15:53.301 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 0, 2024-07-31T15:15:53.301 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 0, 2024-07-31T15:15:53.301 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 193, 2024-07-31T15:15:53.301 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 0, 2024-07-31T15:15:53.301 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 0, 2024-07-31T15:15:53.302 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 0, 2024-07-31T15:15:53.302 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 0, 2024-07-31T15:15:53.302 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-07-31T15:15:53.302 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-07-31T15:15:53.302 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 0, 2024-07-31T15:15:53.302 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-07-31T15:15:53.302 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-07-31T15:15:53.303 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 0, 2024-07-31T15:15:53.303 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 0, 2024-07-31T15:15:53.303 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 0, 2024-07-31T15:15:53.303 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 0, 2024-07-31T15:15:53.303 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-07-31T15:15:53.303 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 0, 2024-07-31T15:15:53.303 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-07-31T15:15:53.304 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 0, 2024-07-31T15:15:53.304 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 0, 2024-07-31T15:15:53.304 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 0, 2024-07-31T15:15:53.304 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 0, 2024-07-31T15:15:53.304 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-07-31T15:15:53.304 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-07-31T15:15:53.304 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-07-31T15:15:53.304 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 0, 2024-07-31T15:15:53.304 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-07-31T15:15:53.305 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 0, 2024-07-31T15:15:53.305 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 0, 2024-07-31T15:15:53.305 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 0, 2024-07-31T15:15:53.305 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 0, 2024-07-31T15:15:53.305 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 0, 2024-07-31T15:15:53.305 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-07-31T15:15:53.305 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-31T15:15:53.306 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-31T15:15:53.306 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 0, 2024-07-31T15:15:53.306 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 0, 2024-07-31T15:15:53.306 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 0, 2024-07-31T15:15:53.306 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 0 2024-07-31T15:15:53.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.306 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-07-31T15:15:53.307 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 0, 2024-07-31T15:15:53.307 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 0, 2024-07-31T15:15:53.307 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 0, 2024-07-31T15:15:53.307 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 0, 2024-07-31T15:15:53.307 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 0, 2024-07-31T15:15:53.307 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 0, 2024-07-31T15:15:53.307 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-07-31T15:15:53.308 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-07-31T15:15:53.308 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 0, 2024-07-31T15:15:53.308 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 0, 2024-07-31T15:15:53.308 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-07-31T15:15:53.308 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-07-31T15:15:53.308 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-07-31T15:15:53.308 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-07-31T15:15:53.309 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-07-31T15:15:53.309 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-07-31T15:15:53.309 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-07-31T15:15:53.309 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 0, 2024-07-31T15:15:53.309 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 0, 2024-07-31T15:15:53.309 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-07-31T15:15:53.309 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 0, 2024-07-31T15:15:53.309 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-07-31T15:15:53.309 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 0, 2024-07-31T15:15:53.310 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-07-31T15:15:53.310 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-07-31T15:15:53.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.310 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-07-31T15:15:53.310 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 0, 2024-07-31T15:15:53.310 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 0, 2024-07-31T15:15:53.310 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 0, 2024-07-31T15:15:53.311 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 0, 2024-07-31T15:15:53.311 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 22557, 2024-07-31T15:15:53.311 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-07-31T15:15:53.311 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 0, 2024-07-31T15:15:53.311 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 0, 2024-07-31T15:15:53.311 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 0, 2024-07-31T15:15:53.311 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 0, 2024-07-31T15:15:53.311 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 29, 2024-07-31T15:15:53.312 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 3, 2024-07-31T15:15:53.312 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-07-31T15:15:53.312 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 0, 2024-07-31T15:15:53.312 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 4194304, 2024-07-31T15:15:53.312 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 55534834, 2024-07-31T15:15:53.312 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 55534260, 2024-07-31T15:15:53.312 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-07-31T15:15:53.313 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.313 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.313 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.313 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 22557 2024-07-31T15:15:53.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.313 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-07-31T15:15:53.313 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 12128, 2024-07-31T15:15:53.314 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 13183, 2024-07-31T15:15:53.314 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 1055, 2024-07-31T15:15:53.314 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 921, 2024-07-31T15:15:53.314 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 992, 2024-07-31T15:15:53.314 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 71, 2024-07-31T15:15:53.314 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 12513, 2024-07-31T15:15:53.314 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 13374, 2024-07-31T15:15:53.315 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 861, 2024-07-31T15:15:53.315 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 0, 2024-07-31T15:15:53.315 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 0, 2024-07-31T15:15:53.315 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 0, 2024-07-31T15:15:53.315 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 167036, 2024-07-31T15:15:53.315 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 215348 2024-07-31T15:15:53.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.315 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-07-31T15:15:53.316 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 0, 2024-07-31T15:15:53.316 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 0, 2024-07-31T15:15:53.316 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 0, 2024-07-31T15:15:53.316 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 0, 2024-07-31T15:15:53.316 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 0, 2024-07-31T15:15:53.316 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-07-31T15:15:53.316 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.316 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.316 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.317 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-07-31T15:15:53.317 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.317 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.317 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.318 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-07-31T15:15:53.318 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.318 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.318 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.318 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.318 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-07-31T15:15:53.318 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.318 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.319 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.319 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-07-31T15:15:53.319 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.319 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.319 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.319 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-07-31T15:15:53.320 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.320 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.320 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.320 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-07-31T15:15:53.320 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.320 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.321 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.321 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-07-31T15:15:53.321 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.321 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.321 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.321 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-07-31T15:15:53.321 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.322 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.322 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.322 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-07-31T15:15:53.322 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.322 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.322 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.323 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-07-31T15:15:53.323 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.323 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.323 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.324 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-07-31T15:15:53.324 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.324 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.324 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.324 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-07-31T15:15:53.324 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.325 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.325 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.325 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-07-31T15:15:53.325 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.325 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.325 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.326 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-07-31T15:15:53.326 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.326 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.326 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.326 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-07-31T15:15:53.326 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.326 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.327 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.327 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-07-31T15:15:53.327 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.327 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.327 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.327 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-07-31T15:15:53.328 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.328 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.328 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.328 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-07-31T15:15:53.328 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.328 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.328 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.329 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-07-31T15:15:53.329 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.329 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.329 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.329 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-07-31T15:15:53.329 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.329 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.330 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.330 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-07-31T15:15:53.330 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.330 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.330 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.330 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-07-31T15:15:53.331 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.331 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.331 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.331 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-07-31T15:15:53.331 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.331 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.331 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.332 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-07-31T15:15:53.332 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.332 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.332 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.332 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-07-31T15:15:53.332 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.332 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.333 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.333 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-07-31T15:15:53.333 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.333 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.333 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.334 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.334 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-07-31T15:15:53.334 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.334 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.334 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.334 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.334 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-07-31T15:15:53.334 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.335 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.335 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.335 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-07-31T15:15:53.335 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-07-31T15:15:53.335 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-07-31T15:15:53.335 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.335 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.336 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.336 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:53.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.336 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-07-31T15:15:53.336 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 2, 2024-07-31T15:15:53.336 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 4, 2024-07-31T15:15:53.336 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 2, 2024-07-31T15:15:53.336 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 2, 2024-07-31T15:15:53.337 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-07-31T15:15:53.337 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 0, 2024-07-31T15:15:53.337 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 0, 2024-07-31T15:15:53.337 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 4486, 2024-07-31T15:15:53.337 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-07-31T15:15:53.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.337 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-07-31T15:15:53.337 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-07-31T15:15:53.338 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-07-31T15:15:53.338 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-07-31T15:15:53.338 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-07-31T15:15:53.338 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-07-31T15:15:53.338 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-07-31T15:15:53.338 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-31T15:15:53.338 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-07-31T15:15:53.338 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-31T15:15:53.339 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-07-31T15:15:53.339 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-07-31T15:15:53.339 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-07-31T15:15:53.339 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-31T15:15:53.339 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-07-31T15:15:53.340 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-07-31T15:15:53.340 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-07-31T15:15:53.340 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-07-31T15:15:53.340 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-07-31T15:15:53.340 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-31T15:15:53.340 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-07-31T15:15:53.340 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-31T15:15:53.340 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-07-31T15:15:53.341 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-07-31T15:15:53.341 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-07-31T15:15:53.341 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-07-31T15:15:53.341 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-07-31T15:15:53.341 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-31T15:15:53.341 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-07-31T15:15:53.341 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-07-31T15:15:53.341 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-07-31T15:15:53.342 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-07-31T15:15:53.342 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-07-31T15:15:53.342 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-07-31T15:15:53.342 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-07-31T15:15:53.342 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-07-31T15:15:53.342 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-07-31T15:15:53.342 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 683151, 2024-07-31T15:15:53.343 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 12480, 2024-07-31T15:15:53.343 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 0, 2024-07-31T15:15:53.343 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 0, 2024-07-31T15:15:53.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-07-31T15:15:53.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-07-31T15:15:53.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-07-31T15:15:53.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-07-31T15:15:53.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-07-31T15:15:53.344 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-07-31T15:15:53.344 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 9352, 2024-07-31T15:15:53.344 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 189, 2024-07-31T15:15:53.344 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-07-31T15:15:53.344 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-07-31T15:15:53.344 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-07-31T15:15:53.344 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-07-31T15:15:53.344 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 38323069, 2024-07-31T15:15:53.345 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 457320, 2024-07-31T15:15:53.345 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-07-31T15:15:53.345 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-07-31T15:15:53.345 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-07-31T15:15:53.345 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-07-31T15:15:53.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.345 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-07-31T15:15:53.346 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-07-31T15:15:53.346 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-07-31T15:15:53.346 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 5021, 2024-07-31T15:15:53.346 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 21, 2024-07-31T15:15:53.346 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 3, 2024-07-31T15:15:53.346 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 5021, 2024-07-31T15:15:53.346 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-07-31T15:15:53.347 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 5021, 2024-07-31T15:15:53.347 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 63.582572794, 2024-07-31T15:15:53.347 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.012663328 2024-07-31T15:15:53.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.347 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-07-31T15:15:53.347 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-07-31T15:15:53.347 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 5021, 2024-07-31T15:15:53.347 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 5022 2024-07-31T15:15:53.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.348 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 5021, 2024-07-31T15:15:53.348 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 5021, 2024-07-31T15:15:53.348 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 0, 2024-07-31T15:15:53.348 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-07-31T15:15:53.348 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-07-31T15:15:53.348 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 3129, 2024-07-31T15:15:53.349 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 0, 2024-07-31T15:15:53.349 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 1890, 2024-07-31T15:15:53.349 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 0, 2024-07-31T15:15:53.349 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 0, 2024-07-31T15:15:53.349 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-07-31T15:15:53.349 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-07-31T15:15:53.349 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-07-31T15:15:53.350 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 0, 2024-07-31T15:15:53.350 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 0, 2024-07-31T15:15:53.350 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-07-31T15:15:53.350 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-07-31T15:15:53.350 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-07-31T15:15:53.350 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 1, 2024-07-31T15:15:53.350 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 0, 2024-07-31T15:15:53.351 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-07-31T15:15:53.351 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-07-31T15:15:53.351 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-07-31T15:15:53.351 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-07-31T15:15:53.351 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-07-31T15:15:53.351 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-07-31T15:15:53.351 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-07-31T15:15:53.351 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-07-31T15:15:53.352 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-07-31T15:15:53.352 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-07-31T15:15:53.352 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-07-31T15:15:53.352 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-07-31T15:15:53.352 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-07-31T15:15:53.352 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-07-31T15:15:53.352 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-07-31T15:15:53.353 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-07-31T15:15:53.353 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-07-31T15:15:53.353 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-07-31T15:15:53.353 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-07-31T15:15:53.353 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-07-31T15:15:53.353 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-07-31T15:15:53.353 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-07-31T15:15:53.353 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-07-31T15:15:53.354 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-07-31T15:15:53.354 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-07-31T15:15:53.354 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-07-31T15:15:53.354 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 138, 2024-07-31T15:15:53.354 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-07-31T15:15:53.354 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 49, 2024-07-31T15:15:53.354 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 9, 2024-07-31T15:15:53.354 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 10, 2024-07-31T15:15:53.355 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 1, 2024-07-31T15:15:53.355 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-07-31T15:15:53.355 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 0, 2024-07-31T15:15:53.355 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 2, 2024-07-31T15:15:53.355 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 0 2024-07-31T15:15:53.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.355 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-07-31T15:15:53.355 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 0, 2024-07-31T15:15:53.355 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-07-31T15:15:53.356 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-07-31T15:15:53.356 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-07-31T15:15:53.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.356 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-07-31T15:15:53.356 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 0 2024-07-31T15:15:53.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.356 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-07-31T15:15:53.356 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-07-31T15:15:53.357 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 0, 2024-07-31T15:15:53.357 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-07-31T15:15:53.357 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 0, 2024-07-31T15:15:53.357 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 0, 2024-07-31T15:15:53.357 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 0, 2024-07-31T15:15:53.357 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-07-31T15:15:53.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.357 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-31T15:15:53.357 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:53.358 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-07-31T15:15:53.358 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:53.358 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 5591, 2024-07-31T15:15:53.358 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 53877286, 2024-07-31T15:15:53.358 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:53.358 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 5591, 2024-07-31T15:15:53.358 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-07-31T15:15:53.358 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-07-31T15:15:53.359 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 5591, 2024-07-31T15:15:53.359 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 53877286, 2024-07-31T15:15:53.359 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:53.359 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.359 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.359 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.359 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:53.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.360 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-07-31T15:15:53.360 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:53.360 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-07-31T15:15:53.360 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:53.360 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-07-31T15:15:53.360 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-07-31T15:15:53.360 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:53.361 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:53.361 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 5021, 2024-07-31T15:15:53.361 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 51344120, 2024-07-31T15:15:53.361 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 5021, 2024-07-31T15:15:53.361 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 51344120, 2024-07-31T15:15:53.361 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:53.361 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.362 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.362 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.362 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:53.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.362 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-07-31T15:15:53.362 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:53.362 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-07-31T15:15:53.362 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:53.363 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-07-31T15:15:53.363 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-07-31T15:15:53.363 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:53.363 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:53.363 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 5021, 2024-07-31T15:15:53.363 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 5021, 2024-07-31T15:15:53.363 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 5021, 2024-07-31T15:15:53.363 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 5021, 2024-07-31T15:15:53.363 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:53.364 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.364 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.364 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.364 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:53.364 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.364 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-07-31T15:15:53.364 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:53.364 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-07-31T15:15:53.365 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:53.365 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-07-31T15:15:53.365 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-07-31T15:15:53.365 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:53.365 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:53.365 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-07-31T15:15:53.365 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-07-31T15:15:53.365 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-07-31T15:15:53.365 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-07-31T15:15:53.366 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:53.366 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.366 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.366 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.366 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:53.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-07-31T15:15:53.366 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x55af1eff28f0": { 2024-07-31T15:15:53.367 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-07-31T15:15:53.367 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-07-31T15:15:53.367 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-07-31T15:15:53.367 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-07-31T15:15:53.367 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-07-31T15:15:53.367 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:53.367 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:53.367 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-07-31T15:15:53.367 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-07-31T15:15:53.368 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-07-31T15:15:53.368 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-07-31T15:15:53.368 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-07-31T15:15:53.368 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-07-31T15:15:53.368 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-07-31T15:15:53.368 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-07-31T15:15:53.368 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:53.369 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-07-31T15:15:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-07-31T15:15:53.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:53 smithi077 ceph-mon[37327]: pgmap v1192: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-31T15:15:53.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:53 smithi067 ceph-mon[32750]: pgmap v1192: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-31T15:15:54.104 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-07-31T15:15:54.104 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-07-31T15:15:54.104 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-07-31T15:15:54.104 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-07-31T15:15:54.104 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-07-31T15:15:54.104 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-07-31T15:15:54.105 DEBUG:tasks.check_counter:Getting stats from c 2024-07-31T15:15:54.105 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph daemon mds.c perf dump 2024-07-31T15:15:54.428 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T15:15:55.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:55 smithi077 ceph-mon[37327]: pgmap v1193: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-07-31T15:15:55.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:55 smithi067 ceph-mon[32750]: pgmap v1193: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-07-31T15:15:55.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:55 smithi073 ceph-mon[36714]: pgmap v1193: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-07-31T15:15:56.522 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-07-31T15:15:56.522 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-07-31T15:15:56.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 66516, 2024-07-31T15:15:56.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 79782, 2024-07-31T15:15:56.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 24161153, 2024-07-31T15:15:56.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 47333453, 2024-07-31T15:15:56.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 47, 2024-07-31T15:15:56.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 6, 2024-07-31T15:15:56.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 4.723860782, 2024-07-31T15:15:56.523 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 1.925605457, 2024-07-31T15:15:56.524 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 3.128537732, 2024-07-31T15:15:56.524 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.675963647, 2024-07-31T15:15:56.524 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:56.524 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 79779, 2024-07-31T15:15:56.524 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 16.157973109, 2024-07-31T15:15:56.524 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000202534 2024-07-31T15:15:56.524 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.524 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:56.525 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 77143, 2024-07-31T15:15:56.525 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.015559488, 2024-07-31T15:15:56.525 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000201 2024-07-31T15:15:56.525 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.525 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 14869872, 2024-07-31T15:15:56.525 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 29254688 2024-07-31T15:15:56.525 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.525 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-07-31T15:15:56.526 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 63192, 2024-07-31T15:15:56.526 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 55508, 2024-07-31T15:15:56.526 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 24260971, 2024-07-31T15:15:56.526 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 36340986, 2024-07-31T15:15:56.526 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 13, 2024-07-31T15:15:56.526 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 6, 2024-07-31T15:15:56.526 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 4.395078944, 2024-07-31T15:15:56.527 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 1.517569381, 2024-07-31T15:15:56.527 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 1.679112356, 2024-07-31T15:15:56.527 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 1.136641020, 2024-07-31T15:15:56.527 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:56.527 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 55508, 2024-07-31T15:15:56.527 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 3.813933095, 2024-07-31T15:15:56.527 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000068709 2024-07-31T15:15:56.527 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.528 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:56.528 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 31680, 2024-07-31T15:15:56.528 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.006922948, 2024-07-31T15:15:56.528 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000218 2024-07-31T15:15:56.528 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.528 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-31T15:15:56.528 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 695952 2024-07-31T15:15:56.528 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.528 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-07-31T15:15:56.529 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 29669, 2024-07-31T15:15:56.529 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 29622, 2024-07-31T15:15:56.529 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 10649944, 2024-07-31T15:15:56.529 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 14053375, 2024-07-31T15:15:56.529 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 15, 2024-07-31T15:15:56.529 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 7, 2024-07-31T15:15:56.529 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 2.357149251, 2024-07-31T15:15:56.529 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.831327376, 2024-07-31T15:15:56.529 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.696222517, 2024-07-31T15:15:56.530 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.675284831, 2024-07-31T15:15:56.530 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:56.530 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 29621, 2024-07-31T15:15:56.530 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 2.510384522, 2024-07-31T15:15:56.530 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000084750 2024-07-31T15:15:56.530 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.530 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:56.530 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 7013, 2024-07-31T15:15:56.531 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.001311042, 2024-07-31T15:15:56.531 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000186 2024-07-31T15:15:56.531 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.531 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1717312, 2024-07-31T15:15:56.531 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 655296 2024-07-31T15:15:56.531 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.531 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-07-31T15:15:56.531 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-07-31T15:15:56.532 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-07-31T15:15:56.532 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.532 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-07-31T15:15:56.532 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-07-31T15:15:56.532 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-07-31T15:15:56.532 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 69943, 2024-07-31T15:15:56.532 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 22.043421438, 2024-07-31T15:15:56.532 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000315162 2024-07-31T15:15:56.533 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:56.533 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.533 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-07-31T15:15:56.533 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-07-31T15:15:56.533 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-07-31T15:15:56.533 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 183, 2024-07-31T15:15:56.533 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.009420603, 2024-07-31T15:15:56.533 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000051478 2024-07-31T15:15:56.534 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:56.534 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.534 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-07-31T15:15:56.534 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 10925, 2024-07-31T15:15:56.534 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 9241, 2024-07-31T15:15:56.534 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-07-31T15:15:56.534 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 9241, 2024-07-31T15:15:56.535 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 43.104929290, 2024-07-31T15:15:56.535 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.004664530 2024-07-31T15:15:56.535 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.535 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-07-31T15:15:56.535 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 1684, 2024-07-31T15:15:56.535 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-07-31T15:15:56.535 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-07-31T15:15:56.536 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 80, 2024-07-31T15:15:56.536 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 8, 2024-07-31T15:15:56.536 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-07-31T15:15:56.536 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 8333, 2024-07-31T15:15:56.536 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 628, 2024-07-31T15:15:56.536 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 7705, 2024-07-31T15:15:56.536 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-07-31T15:15:56.536 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 7663, 2024-07-31T15:15:56.537 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 153, 2024-07-31T15:15:56.537 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 135, 2024-07-31T15:15:56.537 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 135, 2024-07-31T15:15:56.537 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 191, 2024-07-31T15:15:56.537 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 16239, 2024-07-31T15:15:56.537 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 8977, 2024-07-31T15:15:56.537 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 1667, 2024-07-31T15:15:56.537 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 272, 2024-07-31T15:15:56.538 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-07-31T15:15:56.538 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-07-31T15:15:56.538 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-07-31T15:15:56.538 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-07-31T15:15:56.538 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-07-31T15:15:56.538 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 4, 2024-07-31T15:15:56.538 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 153, 2024-07-31T15:15:56.538 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 224, 2024-07-31T15:15:56.539 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 5311, 2024-07-31T15:15:56.539 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-07-31T15:15:56.539 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 87931, 2024-07-31T15:15:56.539 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-07-31T15:15:56.539 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 91397, 2024-07-31T15:15:56.539 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 2022860651, 2024-07-31T15:15:56.539 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 7, 2024-07-31T15:15:56.540 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 87931, 2024-07-31T15:15:56.540 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-07-31T15:15:56.540 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-07-31T15:15:56.540 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-07-31T15:15:56.540 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 3943, 2024-07-31T15:15:56.540 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-07-31T15:15:56.540 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 3943, 2024-07-31T15:15:56.540 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 83988, 2024-07-31T15:15:56.540 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 135, 2024-07-31T15:15:56.541 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 18764, 2024-07-31T15:15:56.541 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 5565, 2024-07-31T15:15:56.541 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 8, 2024-07-31T15:15:56.541 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-31T15:15:56.541 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-31T15:15:56.541 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 25593, 2024-07-31T15:15:56.541 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 7437, 2024-07-31T15:15:56.541 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 441, 2024-07-31T15:15:56.542 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 910 2024-07-31T15:15:56.542 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.542 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-07-31T15:15:56.542 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-07-31T15:15:56.542 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-07-31T15:15:56.542 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-07-31T15:15:56.542 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 253, 2024-07-31T15:15:56.543 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 41, 2024-07-31T15:15:56.543 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-07-31T15:15:56.543 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-07-31T15:15:56.543 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-07-31T15:15:56.543 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 52, 2024-07-31T15:15:56.543 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 52, 2024-07-31T15:15:56.543 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-07-31T15:15:56.543 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-07-31T15:15:56.544 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-07-31T15:15:56.544 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-07-31T15:15:56.544 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-07-31T15:15:56.544 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-07-31T15:15:56.544 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-07-31T15:15:56.544 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 16, 2024-07-31T15:15:56.544 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 83914, 2024-07-31T15:15:56.544 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-07-31T15:15:56.545 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 4, 2024-07-31T15:15:56.545 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-07-31T15:15:56.545 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 8, 2024-07-31T15:15:56.545 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-07-31T15:15:56.545 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-07-31T15:15:56.545 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.545 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-07-31T15:15:56.545 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-07-31T15:15:56.546 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 13171, 2024-07-31T15:15:56.546 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-07-31T15:15:56.546 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-07-31T15:15:56.546 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 13171, 2024-07-31T15:15:56.546 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-07-31T15:15:56.546 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-07-31T15:15:56.547 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 18, 2024-07-31T15:15:56.547 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-07-31T15:15:56.547 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-07-31T15:15:56.549 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 18, 2024-07-31T15:15:56.549 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 3, 2024-07-31T15:15:56.550 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-07-31T15:15:56.550 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-07-31T15:15:56.550 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2024-07-31T15:15:56.550 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 33987709, 2024-07-31T15:15:56.550 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 0, 2024-07-31T15:15:56.550 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-07-31T15:15:56.550 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1989, 2024-07-31T15:15:56.550 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 239.615191755, 2024-07-31T15:15:56.551 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.120470181 2024-07-31T15:15:56.551 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.551 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 0 2024-07-31T15:15:56.551 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.551 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-07-31T15:15:56.551 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 7835, 2024-07-31T15:15:56.551 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 8040, 2024-07-31T15:15:56.551 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 205, 2024-07-31T15:15:56.552 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 715, 2024-07-31T15:15:56.552 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 740, 2024-07-31T15:15:56.552 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 25, 2024-07-31T15:15:56.552 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 8333, 2024-07-31T15:15:56.552 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 8965, 2024-07-31T15:15:56.552 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 632, 2024-07-31T15:15:56.552 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 135, 2024-07-31T15:15:56.552 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 16583, 2024-07-31T15:15:56.553 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 16448, 2024-07-31T15:15:56.553 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 192412, 2024-07-31T15:15:56.553 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 223540 2024-07-31T15:15:56.553 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.553 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-07-31T15:15:56.553 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 11271, 2024-07-31T15:15:56.553 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 70, 2024-07-31T15:15:56.554 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 10925, 2024-07-31T15:15:56.554 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 45, 2024-07-31T15:15:56.554 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 377, 2024-07-31T15:15:56.554 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-07-31T15:15:56.554 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1516, 2024-07-31T15:15:56.554 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1.336139901, 2024-07-31T15:15:56.554 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000881358 2024-07-31T15:15:56.554 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.555 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-07-31T15:15:56.555 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 21, 2024-07-31T15:15:56.555 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 14.387688228, 2024-07-31T15:15:56.555 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.685128010 2024-07-31T15:15:56.555 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.555 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-07-31T15:15:56.555 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.555 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.555 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.556 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.556 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-07-31T15:15:56.556 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.556 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.556 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.556 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.556 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-07-31T15:15:56.556 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 4708, 2024-07-31T15:15:56.557 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 25.056343262, 2024-07-31T15:15:56.557 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.005322078 2024-07-31T15:15:56.557 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.557 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-07-31T15:15:56.557 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.557 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.557 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.558 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.558 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-07-31T15:15:56.558 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1, 2024-07-31T15:15:56.558 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000169435, 2024-07-31T15:15:56.558 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000169435 2024-07-31T15:15:56.558 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.558 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-07-31T15:15:56.558 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.559 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.559 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.559 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.559 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-07-31T15:15:56.559 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.559 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.559 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.559 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.560 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-07-31T15:15:56.560 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.560 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.561 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.561 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.561 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-07-31T15:15:56.561 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.561 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.561 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.562 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.562 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-07-31T15:15:56.562 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 48, 2024-07-31T15:15:56.562 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.229234891, 2024-07-31T15:15:56.562 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.004775726 2024-07-31T15:15:56.562 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.562 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-07-31T15:15:56.562 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.563 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.563 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.563 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.563 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-07-31T15:15:56.563 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.563 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.563 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.563 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.564 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-07-31T15:15:56.564 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2260, 2024-07-31T15:15:56.564 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1.155472645, 2024-07-31T15:15:56.564 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000511271 2024-07-31T15:15:56.564 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.564 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-07-31T15:15:56.564 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 620, 2024-07-31T15:15:56.564 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.643902633, 2024-07-31T15:15:56.565 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001038552 2024-07-31T15:15:56.565 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.565 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-07-31T15:15:56.565 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.565 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.565 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.565 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.566 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-07-31T15:15:56.566 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.566 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.566 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.566 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.566 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-07-31T15:15:56.566 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.566 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.567 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.567 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.567 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-07-31T15:15:56.567 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.567 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.567 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.567 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.567 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-07-31T15:15:56.568 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.568 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.568 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.568 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.568 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-07-31T15:15:56.568 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.568 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.568 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.569 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.569 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-07-31T15:15:56.569 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 12, 2024-07-31T15:15:56.569 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.154501632, 2024-07-31T15:15:56.569 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.012875136 2024-07-31T15:15:56.569 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.569 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-07-31T15:15:56.569 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.570 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.570 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.570 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.570 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-07-31T15:15:56.570 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.570 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.570 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.571 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.571 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-07-31T15:15:56.571 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.571 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.571 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.571 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.571 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-07-31T15:15:56.571 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.572 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.572 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.572 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.572 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-07-31T15:15:56.572 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 3, 2024-07-31T15:15:56.572 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.023904449, 2024-07-31T15:15:56.572 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.007968149 2024-07-31T15:15:56.572 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.573 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-07-31T15:15:56.573 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 52, 2024-07-31T15:15:56.573 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.117572214, 2024-07-31T15:15:56.573 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002261004 2024-07-31T15:15:56.573 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.573 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-07-31T15:15:56.573 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-07-31T15:15:56.574 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-07-31T15:15:56.574 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.574 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.574 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.574 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:56.574 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.574 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-07-31T15:15:56.574 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 2, 2024-07-31T15:15:56.575 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 3, 2024-07-31T15:15:56.575 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 1, 2024-07-31T15:15:56.575 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 2, 2024-07-31T15:15:56.575 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-07-31T15:15:56.575 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 1129, 2024-07-31T15:15:56.575 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 564, 2024-07-31T15:15:56.575 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 1461, 2024-07-31T15:15:56.575 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-07-31T15:15:56.576 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.576 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-07-31T15:15:56.576 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-07-31T15:15:56.576 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-07-31T15:15:56.576 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-07-31T15:15:56.576 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-07-31T15:15:56.576 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-07-31T15:15:56.576 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-07-31T15:15:56.577 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-31T15:15:56.577 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-07-31T15:15:56.577 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-31T15:15:56.577 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-07-31T15:15:56.577 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-07-31T15:15:56.577 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-07-31T15:15:56.577 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-31T15:15:56.577 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-07-31T15:15:56.578 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-07-31T15:15:56.578 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-07-31T15:15:56.578 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-07-31T15:15:56.578 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-07-31T15:15:56.578 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-31T15:15:56.578 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-07-31T15:15:56.578 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-31T15:15:56.578 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-07-31T15:15:56.579 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-07-31T15:15:56.579 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-07-31T15:15:56.579 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-07-31T15:15:56.579 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-07-31T15:15:56.579 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-31T15:15:56.579 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-07-31T15:15:56.579 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-07-31T15:15:56.579 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-07-31T15:15:56.580 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-07-31T15:15:56.580 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-07-31T15:15:56.580 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-07-31T15:15:56.580 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-07-31T15:15:56.580 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-07-31T15:15:56.580 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-07-31T15:15:56.580 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 553113, 2024-07-31T15:15:56.580 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 7045, 2024-07-31T15:15:56.581 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-07-31T15:15:56.581 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-07-31T15:15:56.581 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-07-31T15:15:56.581 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-07-31T15:15:56.581 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-07-31T15:15:56.581 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-07-31T15:15:56.581 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-07-31T15:15:56.581 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-07-31T15:15:56.582 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 9352, 2024-07-31T15:15:56.582 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 189, 2024-07-31T15:15:56.582 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-07-31T15:15:56.582 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-07-31T15:15:56.582 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-07-31T15:15:56.582 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-07-31T15:15:56.583 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 25803401, 2024-07-31T15:15:56.583 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 347817, 2024-07-31T15:15:56.583 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-07-31T15:15:56.583 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-07-31T15:15:56.583 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-07-31T15:15:56.583 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-07-31T15:15:56.583 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.584 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-07-31T15:15:56.584 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-07-31T15:15:56.584 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-07-31T15:15:56.584 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 90805, 2024-07-31T15:15:56.584 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 32247851, 2024-07-31T15:15:56.584 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 33, 2024-07-31T15:15:56.584 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 90805, 2024-07-31T15:15:56.584 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-07-31T15:15:56.585 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 90805, 2024-07-31T15:15:56.585 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 386.078771500, 2024-07-31T15:15:56.585 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.004251734 2024-07-31T15:15:56.585 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.585 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-07-31T15:15:56.585 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-07-31T15:15:56.585 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 90805, 2024-07-31T15:15:56.585 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 91489 2024-07-31T15:15:56.586 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.586 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 90805, 2024-07-31T15:15:56.586 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 87950, 2024-07-31T15:15:56.586 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 2855, 2024-07-31T15:15:56.586 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-07-31T15:15:56.586 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-07-31T15:15:56.586 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 0, 2024-07-31T15:15:56.587 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 10, 2024-07-31T15:15:56.587 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 19, 2024-07-31T15:15:56.587 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 2008, 2024-07-31T15:15:56.587 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 96, 2024-07-31T15:15:56.587 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-07-31T15:15:56.587 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-07-31T15:15:56.587 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-07-31T15:15:56.587 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 6, 2024-07-31T15:15:56.588 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 125, 2024-07-31T15:15:56.588 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-07-31T15:15:56.588 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-07-31T15:15:56.588 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-07-31T15:15:56.588 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 87931, 2024-07-31T15:15:56.588 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 20, 2024-07-31T15:15:56.588 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-07-31T15:15:56.588 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-07-31T15:15:56.589 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-07-31T15:15:56.589 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-07-31T15:15:56.589 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-07-31T15:15:56.589 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-07-31T15:15:56.589 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-07-31T15:15:56.589 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-07-31T15:15:56.589 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-07-31T15:15:56.589 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 8, 2024-07-31T15:15:56.590 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-07-31T15:15:56.590 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-07-31T15:15:56.590 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-07-31T15:15:56.590 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-07-31T15:15:56.590 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-07-31T15:15:56.590 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-07-31T15:15:56.590 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-07-31T15:15:56.591 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-07-31T15:15:56.591 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-07-31T15:15:56.591 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-07-31T15:15:56.591 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-07-31T15:15:56.591 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-07-31T15:15:56.591 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-07-31T15:15:56.591 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-07-31T15:15:56.591 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-07-31T15:15:56.591 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-07-31T15:15:56.592 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 138, 2024-07-31T15:15:56.592 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-07-31T15:15:56.592 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 49, 2024-07-31T15:15:56.592 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 12, 2024-07-31T15:15:56.592 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 12, 2024-07-31T15:15:56.592 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-07-31T15:15:56.592 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-07-31T15:15:56.593 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 1066, 2024-07-31T15:15:56.593 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 0, 2024-07-31T15:15:56.593 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 200 2024-07-31T15:15:56.593 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.593 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-07-31T15:15:56.593 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 1, 2024-07-31T15:15:56.593 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-07-31T15:15:56.593 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-07-31T15:15:56.594 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-07-31T15:15:56.594 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.594 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-07-31T15:15:56.594 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 83930 2024-07-31T15:15:56.594 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.594 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-07-31T15:15:56.594 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-07-31T15:15:56.594 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 10, 2024-07-31T15:15:56.595 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-07-31T15:15:56.595 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 5, 2024-07-31T15:15:56.595 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 104, 2024-07-31T15:15:56.595 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 52, 2024-07-31T15:15:56.595 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-07-31T15:15:56.595 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.595 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-31T15:15:56.595 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:15:56.595 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-07-31T15:15:56.596 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:15:56.596 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 159373, 2024-07-31T15:15:56.596 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 42827476, 2024-07-31T15:15:56.596 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:56.596 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 159373, 2024-07-31T15:15:56.596 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-07-31T15:15:56.596 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-07-31T15:15:56.596 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 159373, 2024-07-31T15:15:56.597 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 42827476, 2024-07-31T15:15:56.597 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:15:56.597 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.597 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.597 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.597 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:56.597 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.597 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-07-31T15:15:56.598 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:15:56.598 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-07-31T15:15:56.598 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:15:56.598 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-07-31T15:15:56.598 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-07-31T15:15:56.598 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:56.599 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:56.599 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 90805, 2024-07-31T15:15:56.599 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 32253519, 2024-07-31T15:15:56.599 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 90805, 2024-07-31T15:15:56.599 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 32253519, 2024-07-31T15:15:56.599 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:15:56.599 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.599 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.600 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.600 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:56.600 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.600 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-07-31T15:15:56.600 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:15:56.600 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-07-31T15:15:56.600 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:15:56.600 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-07-31T15:15:56.600 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-07-31T15:15:56.601 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:56.601 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:56.601 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 90805, 2024-07-31T15:15:56.601 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 90805, 2024-07-31T15:15:56.601 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 90805, 2024-07-31T15:15:56.601 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 90805, 2024-07-31T15:15:56.601 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:15:56.601 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.602 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.602 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.602 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:56.602 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.602 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-07-31T15:15:56.602 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:15:56.602 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-07-31T15:15:56.602 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:15:56.603 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 52, 2024-07-31T15:15:56.603 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 5668, 2024-07-31T15:15:56.603 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:56.603 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 52, 2024-07-31T15:15:56.603 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-07-31T15:15:56.603 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-07-31T15:15:56.603 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 19, 2024-07-31T15:15:56.603 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 5668, 2024-07-31T15:15:56.604 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:15:56.604 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.604 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.604 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.604 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:56.604 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:56.604 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x561f13734750": { 2024-07-31T15:15:56.604 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:15:56.605 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-07-31T15:15:56.605 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:15:56.605 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 13171, 2024-07-31T15:15:56.605 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 29793405, 2024-07-31T15:15:56.605 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:56.605 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 13171, 2024-07-31T15:15:56.605 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-07-31T15:15:56.605 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-07-31T15:15:56.606 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1989, 2024-07-31T15:15:56.606 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 29793405, 2024-07-31T15:15:56.606 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:15:56.606 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:56.606 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:56.606 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:56.606 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:56.607 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:56.607 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-07-31T15:15:57.395 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:57 smithi077 ceph-mon[37327]: pgmap v1194: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-31T15:15:57.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:57 smithi073 ceph-mon[36714]: pgmap v1194: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-31T15:15:57.438 INFO:tasks.check_counter:Daemon mds.c mds.exported=4 2024-07-31T15:15:57.439 INFO:tasks.check_counter:Daemon mds.c mds.imported=224 2024-07-31T15:15:57.439 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-07-31T15:15:57.439 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-07-31T15:15:57.439 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-07-31T15:15:57.439 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=8 2024-07-31T15:15:57.439 DEBUG:tasks.check_counter:Getting stats from f 2024-07-31T15:15:57.439 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph daemon mds.f perf dump 2024-07-31T15:15:57.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:57 smithi067 ceph-mon[32750]: pgmap v1194: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-31T15:15:57.697 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T15:15:59.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:15:59 smithi077 ceph-mon[37327]: pgmap v1195: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-31T15:15:59.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:15:59 smithi067 ceph-mon[32750]: pgmap v1195: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-31T15:15:59.608 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-07-31T15:15:59.608 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-07-31T15:15:59.608 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 28668, 2024-07-31T15:15:59.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 28714, 2024-07-31T15:15:59.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 13317208, 2024-07-31T15:15:59.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 26462373, 2024-07-31T15:15:59.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 17, 2024-07-31T15:15:59.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2024-07-31T15:15:59.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 2.550538047, 2024-07-31T15:15:59.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 1.103011046, 2024-07-31T15:15:59.610 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.839824384, 2024-07-31T15:15:59.610 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.541043255, 2024-07-31T15:15:59.610 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:59.610 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 28714, 2024-07-31T15:15:59.610 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 2.910081896, 2024-07-31T15:15:59.610 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000101347 2024-07-31T15:15:59.610 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.611 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:59.611 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 14943, 2024-07-31T15:15:59.611 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.003680541, 2024-07-31T15:15:59.611 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000246 2024-07-31T15:15:59.611 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.611 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1713184, 2024-07-31T15:15:59.611 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 682144 2024-07-31T15:15:59.611 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.612 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-07-31T15:15:59.612 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 33427, 2024-07-31T15:15:59.612 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 40402, 2024-07-31T15:15:59.612 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 13700765, 2024-07-31T15:15:59.612 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 19963373, 2024-07-31T15:15:59.612 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 20, 2024-07-31T15:15:59.612 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 8, 2024-07-31T15:15:59.613 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 2.401994858, 2024-07-31T15:15:59.613 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.878788128, 2024-07-31T15:15:59.613 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 1.698793845, 2024-07-31T15:15:59.613 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.416276318, 2024-07-31T15:15:59.613 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:59.613 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 40398, 2024-07-31T15:15:59.613 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 12.595605790, 2024-07-31T15:15:59.613 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000311787 2024-07-31T15:15:59.614 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.614 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:59.614 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 28903, 2024-07-31T15:15:59.614 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.007632260, 2024-07-31T15:15:59.614 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000264 2024-07-31T15:15:59.614 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.614 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 6290512, 2024-07-31T15:15:59.614 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 13614944 2024-07-31T15:15:59.615 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.615 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-07-31T15:15:59.615 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 61660, 2024-07-31T15:15:59.615 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 54836, 2024-07-31T15:15:59.615 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 30141027, 2024-07-31T15:15:59.615 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 23813674, 2024-07-31T15:15:59.615 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 38, 2024-07-31T15:15:59.615 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2024-07-31T15:15:59.616 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 4.165422944, 2024-07-31T15:15:59.616 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 1.452281161, 2024-07-31T15:15:59.616 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 1.759986511, 2024-07-31T15:15:59.616 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.976990710, 2024-07-31T15:15:59.616 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:15:59.616 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 54836, 2024-07-31T15:15:59.616 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 4.380940182, 2024-07-31T15:15:59.617 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000079891 2024-07-31T15:15:59.617 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.617 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:15:59.617 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 37829, 2024-07-31T15:15:59.617 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.007333213, 2024-07-31T15:15:59.617 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000193 2024-07-31T15:15:59.617 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.618 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-31T15:15:59.618 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-07-31T15:15:59.618 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.618 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-07-31T15:15:59.618 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-07-31T15:15:59.618 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-07-31T15:15:59.618 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.618 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-07-31T15:15:59.619 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-07-31T15:15:59.619 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-07-31T15:15:59.619 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 57029, 2024-07-31T15:15:59.619 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 16.319670204, 2024-07-31T15:15:59.619 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000286164 2024-07-31T15:15:59.619 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:59.619 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.619 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-07-31T15:15:59.620 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-07-31T15:15:59.620 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-07-31T15:15:59.620 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 201, 2024-07-31T15:15:59.620 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.010632176, 2024-07-31T15:15:59.620 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000052896 2024-07-31T15:15:59.620 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:59.620 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.620 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-07-31T15:15:59.621 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 2783, 2024-07-31T15:15:59.621 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 2114, 2024-07-31T15:15:59.621 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-07-31T15:15:59.621 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2114, 2024-07-31T15:15:59.621 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 4.471389129, 2024-07-31T15:15:59.621 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002115132 2024-07-31T15:15:59.621 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.622 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-07-31T15:15:59.622 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 669, 2024-07-31T15:15:59.622 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 1, 2024-07-31T15:15:59.622 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-07-31T15:15:59.622 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 70, 2024-07-31T15:15:59.622 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 7, 2024-07-31T15:15:59.622 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-07-31T15:15:59.622 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 7802, 2024-07-31T15:15:59.623 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 1303, 2024-07-31T15:15:59.623 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 6306, 2024-07-31T15:15:59.623 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 193, 2024-07-31T15:15:59.623 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 6858, 2024-07-31T15:15:59.623 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 76, 2024-07-31T15:15:59.623 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 21, 2024-07-31T15:15:59.623 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 21, 2024-07-31T15:15:59.623 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 181, 2024-07-31T15:15:59.624 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 4479, 2024-07-31T15:15:59.624 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 2243, 2024-07-31T15:15:59.624 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 632, 2024-07-31T15:15:59.624 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 545, 2024-07-31T15:15:59.624 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 1, 2024-07-31T15:15:59.624 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-07-31T15:15:59.624 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-07-31T15:15:59.624 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-07-31T15:15:59.625 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-07-31T15:15:59.625 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 8, 2024-07-31T15:15:59.625 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 76, 2024-07-31T15:15:59.625 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 308, 2024-07-31T15:15:59.625 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 6840, 2024-07-31T15:15:59.625 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-07-31T15:15:59.625 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 72741, 2024-07-31T15:15:59.626 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 1, 2024-07-31T15:15:59.626 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 91397, 2024-07-31T15:15:59.626 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 2022860651, 2024-07-31T15:15:59.626 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 7, 2024-07-31T15:15:59.626 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 72741, 2024-07-31T15:15:59.626 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-07-31T15:15:59.626 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-07-31T15:15:59.627 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-07-31T15:15:59.627 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 6489, 2024-07-31T15:15:59.627 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-07-31T15:15:59.627 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 6489, 2024-07-31T15:15:59.627 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 66252, 2024-07-31T15:15:59.627 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 102, 2024-07-31T15:15:59.627 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 8694, 2024-07-31T15:15:59.627 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 2048, 2024-07-31T15:15:59.628 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 5, 2024-07-31T15:15:59.628 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-31T15:15:59.628 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-31T15:15:59.628 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 12672, 2024-07-31T15:15:59.628 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 4227, 2024-07-31T15:15:59.628 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 153, 2024-07-31T15:15:59.628 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 849 2024-07-31T15:15:59.628 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.629 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-07-31T15:15:59.629 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-07-31T15:15:59.629 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-07-31T15:15:59.629 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-07-31T15:15:59.629 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 372, 2024-07-31T15:15:59.629 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 32, 2024-07-31T15:15:59.629 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-07-31T15:15:59.630 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-07-31T15:15:59.630 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-07-31T15:15:59.630 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 57, 2024-07-31T15:15:59.630 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 57, 2024-07-31T15:15:59.630 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-07-31T15:15:59.630 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-07-31T15:15:59.630 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-07-31T15:15:59.631 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-07-31T15:15:59.631 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-07-31T15:15:59.631 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-07-31T15:15:59.631 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-07-31T15:15:59.631 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 16, 2024-07-31T15:15:59.631 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 69334, 2024-07-31T15:15:59.631 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-07-31T15:15:59.631 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 9, 2024-07-31T15:15:59.631 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-07-31T15:15:59.632 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 8, 2024-07-31T15:15:59.632 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-07-31T15:15:59.632 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-07-31T15:15:59.632 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.632 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-07-31T15:15:59.632 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-07-31T15:15:59.632 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 7090, 2024-07-31T15:15:59.632 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-07-31T15:15:59.632 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-07-31T15:15:59.633 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 7090, 2024-07-31T15:15:59.633 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-07-31T15:15:59.633 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-07-31T15:15:59.633 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 11, 2024-07-31T15:15:59.633 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-07-31T15:15:59.633 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-07-31T15:15:59.633 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 11, 2024-07-31T15:15:59.633 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 2, 2024-07-31T15:15:59.633 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-07-31T15:15:59.634 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-07-31T15:15:59.634 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2024-07-31T15:15:59.634 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 27299883, 2024-07-31T15:15:59.634 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 0, 2024-07-31T15:15:59.634 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-07-31T15:15:59.634 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1743, 2024-07-31T15:15:59.634 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 250.618452344, 2024-07-31T15:15:59.634 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.143785686 2024-07-31T15:15:59.635 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.635 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 0 2024-07-31T15:15:59.635 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.635 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-07-31T15:15:59.635 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 7681, 2024-07-31T15:15:59.635 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 7814, 2024-07-31T15:15:59.635 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 133, 2024-07-31T15:15:59.635 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 897, 2024-07-31T15:15:59.635 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 924, 2024-07-31T15:15:59.636 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 27, 2024-07-31T15:15:59.636 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 7802, 2024-07-31T15:15:59.636 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 8067, 2024-07-31T15:15:59.636 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 265, 2024-07-31T15:15:59.636 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 21, 2024-07-31T15:15:59.636 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 8957, 2024-07-31T15:15:59.636 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 8936, 2024-07-31T15:15:59.636 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 190896, 2024-07-31T15:15:59.637 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 223540 2024-07-31T15:15:59.637 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.637 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-07-31T15:15:59.637 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 3337, 2024-07-31T15:15:59.637 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 3, 2024-07-31T15:15:59.637 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 2783, 2024-07-31T15:15:59.637 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 32, 2024-07-31T15:15:59.637 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 288, 2024-07-31T15:15:59.638 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-07-31T15:15:59.638 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 522, 2024-07-31T15:15:59.638 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.832843367, 2024-07-31T15:15:59.638 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001595485 2024-07-31T15:15:59.638 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.638 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-07-31T15:15:59.638 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 19, 2024-07-31T15:15:59.638 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.023552051, 2024-07-31T15:15:59.638 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001239581 2024-07-31T15:15:59.639 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.639 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-07-31T15:15:59.639 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.639 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.639 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.639 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.639 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-07-31T15:15:59.639 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.639 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.640 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.640 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.640 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-07-31T15:15:59.640 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 658, 2024-07-31T15:15:59.640 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.360842048, 2024-07-31T15:15:59.640 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000548392 2024-07-31T15:15:59.640 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.640 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-07-31T15:15:59.640 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.641 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.641 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.641 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.641 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-07-31T15:15:59.641 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.641 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.641 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.641 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.642 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-07-31T15:15:59.642 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.642 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.642 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.642 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.642 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-07-31T15:15:59.642 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.643 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.643 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.643 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.643 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-07-31T15:15:59.643 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.643 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.643 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.643 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:15:59 smithi073 ceph-mon[36714]: pgmap v1195: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-31T15:15:59.644 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-07-31T15:15:59.645 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.645 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.645 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.645 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.645 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-07-31T15:15:59.645 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 8, 2024-07-31T15:15:59.645 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.009910229, 2024-07-31T15:15:59.645 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001238778 2024-07-31T15:15:59.646 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.646 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-07-31T15:15:59.646 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.646 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.646 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.646 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.646 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-07-31T15:15:59.646 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.647 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.647 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.647 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.647 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-07-31T15:15:59.647 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 189, 2024-07-31T15:15:59.647 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.088995170, 2024-07-31T15:15:59.647 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000470873 2024-07-31T15:15:59.648 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.648 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-07-31T15:15:59.648 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 652, 2024-07-31T15:15:59.648 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 2.973164526, 2024-07-31T15:15:59.648 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.004560068 2024-07-31T15:15:59.648 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.648 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-07-31T15:15:59.648 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.649 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.649 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.649 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.649 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-07-31T15:15:59.649 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.649 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.649 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.649 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.649 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-07-31T15:15:59.650 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.650 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.650 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.650 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.650 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-07-31T15:15:59.650 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.650 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.650 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.651 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.651 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-07-31T15:15:59.651 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.651 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.651 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.651 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.651 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-07-31T15:15:59.651 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.652 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.652 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.652 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.652 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-07-31T15:15:59.652 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 7, 2024-07-31T15:15:59.652 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.058418643, 2024-07-31T15:15:59.652 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.008345520 2024-07-31T15:15:59.652 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.652 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-07-31T15:15:59.653 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.653 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.653 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.653 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.653 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-07-31T15:15:59.653 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.653 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.653 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.653 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.654 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-07-31T15:15:59.654 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.654 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.654 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.654 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.654 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-07-31T15:15:59.654 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.654 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.655 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.655 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.655 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-07-31T15:15:59.655 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2, 2024-07-31T15:15:59.655 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.002605188, 2024-07-31T15:15:59.655 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001302594 2024-07-31T15:15:59.655 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.655 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-07-31T15:15:59.655 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 57, 2024-07-31T15:15:59.656 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.121057907, 2024-07-31T15:15:59.656 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002123822 2024-07-31T15:15:59.656 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.656 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-07-31T15:15:59.656 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-07-31T15:15:59.656 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-07-31T15:15:59.656 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.656 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.656 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.657 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:59.657 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.657 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-07-31T15:15:59.657 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 1, 2024-07-31T15:15:59.657 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 2, 2024-07-31T15:15:59.657 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 1, 2024-07-31T15:15:59.657 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 1, 2024-07-31T15:15:59.657 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-07-31T15:15:59.658 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 197, 2024-07-31T15:15:59.658 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 197, 2024-07-31T15:15:59.658 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 1448, 2024-07-31T15:15:59.658 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-07-31T15:15:59.658 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.658 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-07-31T15:15:59.658 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-07-31T15:15:59.659 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-07-31T15:15:59.659 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-07-31T15:15:59.659 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-07-31T15:15:59.659 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-07-31T15:15:59.659 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-07-31T15:15:59.659 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-31T15:15:59.659 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-07-31T15:15:59.659 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-31T15:15:59.659 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-07-31T15:15:59.660 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-07-31T15:15:59.660 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-07-31T15:15:59.660 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-31T15:15:59.660 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-07-31T15:15:59.660 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-07-31T15:15:59.660 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-07-31T15:15:59.660 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-07-31T15:15:59.660 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-07-31T15:15:59.661 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-31T15:15:59.661 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-07-31T15:15:59.661 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-31T15:15:59.661 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-07-31T15:15:59.661 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-07-31T15:15:59.661 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-07-31T15:15:59.661 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-07-31T15:15:59.661 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-07-31T15:15:59.661 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-31T15:15:59.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-07-31T15:15:59.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-07-31T15:15:59.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-07-31T15:15:59.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-07-31T15:15:59.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-07-31T15:15:59.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-07-31T15:15:59.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-07-31T15:15:59.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-07-31T15:15:59.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-07-31T15:15:59.663 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 574257, 2024-07-31T15:15:59.663 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 7832, 2024-07-31T15:15:59.663 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-07-31T15:15:59.663 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-07-31T15:15:59.663 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-07-31T15:15:59.663 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-07-31T15:15:59.663 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-07-31T15:15:59.663 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-07-31T15:15:59.664 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-07-31T15:15:59.664 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-07-31T15:15:59.664 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 9352, 2024-07-31T15:15:59.664 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 189, 2024-07-31T15:15:59.664 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-07-31T15:15:59.664 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-07-31T15:15:59.664 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-07-31T15:15:59.664 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-07-31T15:15:59.665 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 25652384, 2024-07-31T15:15:59.665 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 353555, 2024-07-31T15:15:59.665 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-07-31T15:15:59.665 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-07-31T15:15:59.665 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-07-31T15:15:59.665 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-07-31T15:15:59.665 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.665 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-07-31T15:15:59.665 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-07-31T15:15:59.666 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-07-31T15:15:59.666 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 75081, 2024-07-31T15:15:59.666 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 24849249, 2024-07-31T15:15:59.666 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 33, 2024-07-31T15:15:59.666 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 75081, 2024-07-31T15:15:59.666 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-07-31T15:15:59.666 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 75081, 2024-07-31T15:15:59.666 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 374.353048388, 2024-07-31T15:15:59.667 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.004985989 2024-07-31T15:15:59.667 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.667 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-07-31T15:15:59.667 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-07-31T15:15:59.667 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 75081, 2024-07-31T15:15:59.667 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 75463 2024-07-31T15:15:59.667 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.667 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 75081, 2024-07-31T15:15:59.667 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 72763, 2024-07-31T15:15:59.668 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 2318, 2024-07-31T15:15:59.668 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-07-31T15:15:59.668 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-07-31T15:15:59.668 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 0, 2024-07-31T15:15:59.668 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 10, 2024-07-31T15:15:59.668 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 20, 2024-07-31T15:15:59.668 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 1763, 2024-07-31T15:15:59.668 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 75, 2024-07-31T15:15:59.669 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-07-31T15:15:59.669 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-07-31T15:15:59.669 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-07-31T15:15:59.669 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 7, 2024-07-31T15:15:59.669 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 131, 2024-07-31T15:15:59.669 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-07-31T15:15:59.669 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-07-31T15:15:59.670 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-07-31T15:15:59.670 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 72742, 2024-07-31T15:15:59.670 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 20, 2024-07-31T15:15:59.670 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-07-31T15:15:59.670 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-07-31T15:15:59.670 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-07-31T15:15:59.670 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-07-31T15:15:59.671 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-07-31T15:15:59.671 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-07-31T15:15:59.671 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-07-31T15:15:59.671 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-07-31T15:15:59.671 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-07-31T15:15:59.671 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 5, 2024-07-31T15:15:59.671 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-07-31T15:15:59.671 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-07-31T15:15:59.672 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-07-31T15:15:59.672 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-07-31T15:15:59.672 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-07-31T15:15:59.672 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-07-31T15:15:59.672 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-07-31T15:15:59.672 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-07-31T15:15:59.672 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-07-31T15:15:59.673 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-07-31T15:15:59.673 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-07-31T15:15:59.673 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-07-31T15:15:59.673 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-07-31T15:15:59.673 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-07-31T15:15:59.673 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-07-31T15:15:59.673 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-07-31T15:15:59.673 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 138, 2024-07-31T15:15:59.674 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-07-31T15:15:59.674 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 49, 2024-07-31T15:15:59.674 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 12, 2024-07-31T15:15:59.674 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 12, 2024-07-31T15:15:59.674 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-07-31T15:15:59.674 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-07-31T15:15:59.674 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 548, 2024-07-31T15:15:59.674 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 3, 2024-07-31T15:15:59.675 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 139 2024-07-31T15:15:59.675 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.675 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-07-31T15:15:59.675 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 1, 2024-07-31T15:15:59.675 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-07-31T15:15:59.675 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-07-31T15:15:59.675 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-07-31T15:15:59.675 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.676 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-07-31T15:15:59.676 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 69350 2024-07-31T15:15:59.676 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.676 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-07-31T15:15:59.676 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-07-31T15:15:59.676 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 10, 2024-07-31T15:15:59.676 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-07-31T15:15:59.676 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 5, 2024-07-31T15:15:59.677 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 114, 2024-07-31T15:15:59.677 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 57, 2024-07-31T15:15:59.677 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-07-31T15:15:59.677 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.677 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-31T15:15:59.677 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:15:59.677 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-07-31T15:15:59.678 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:15:59.678 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 123751, 2024-07-31T15:15:59.678 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 45206069, 2024-07-31T15:15:59.678 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:59.678 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 123751, 2024-07-31T15:15:59.678 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-07-31T15:15:59.678 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-07-31T15:15:59.679 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 123751, 2024-07-31T15:15:59.679 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 45206069, 2024-07-31T15:15:59.679 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:15:59.679 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.679 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.679 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.679 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:59.679 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.680 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-07-31T15:15:59.680 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:15:59.680 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-07-31T15:15:59.680 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:15:59.680 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-07-31T15:15:59.680 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-07-31T15:15:59.680 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:59.681 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:59.681 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 75081, 2024-07-31T15:15:59.681 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 24855412, 2024-07-31T15:15:59.681 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 75081, 2024-07-31T15:15:59.681 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 24855412, 2024-07-31T15:15:59.681 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:15:59.681 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.681 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.682 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.682 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:59.682 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.682 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-07-31T15:15:59.682 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:15:59.682 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-07-31T15:15:59.682 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:15:59.683 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-07-31T15:15:59.683 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-07-31T15:15:59.683 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:59.683 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-07-31T15:15:59.683 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 75081, 2024-07-31T15:15:59.683 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 75081, 2024-07-31T15:15:59.683 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 75081, 2024-07-31T15:15:59.683 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 75081, 2024-07-31T15:15:59.684 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:15:59.684 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.684 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.684 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.684 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:59.684 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.684 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-07-31T15:15:59.684 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:15:59.685 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-07-31T15:15:59.685 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:15:59.685 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 57, 2024-07-31T15:15:59.685 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 6213, 2024-07-31T15:15:59.685 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:59.685 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 57, 2024-07-31T15:15:59.685 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-07-31T15:15:59.686 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-07-31T15:15:59.686 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 20, 2024-07-31T15:15:59.686 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 6213, 2024-07-31T15:15:59.686 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:15:59.686 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.686 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.686 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.686 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:59.686 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:15:59.687 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x56508405c750": { 2024-07-31T15:15:59.687 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:15:59.687 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-07-31T15:15:59.687 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:15:59.687 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 7090, 2024-07-31T15:15:59.687 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 23105579, 2024-07-31T15:15:59.687 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:15:59.687 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 7090, 2024-07-31T15:15:59.688 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-07-31T15:15:59.688 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-07-31T15:15:59.688 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1743, 2024-07-31T15:15:59.688 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 23105579, 2024-07-31T15:15:59.688 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:15:59.688 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:15:59.688 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:15:59.688 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:15:59.688 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:59.689 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:15:59.689 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-07-31T15:16:00.443 INFO:tasks.check_counter:Daemon mds.f mds.exported=8 2024-07-31T15:16:00.444 INFO:tasks.check_counter:Daemon mds.f mds.imported=308 2024-07-31T15:16:00.444 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=7 2024-07-31T15:16:00.444 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-07-31T15:16:00.444 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-07-31T15:16:00.444 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=7 2024-07-31T15:16:00.444 DEBUG:tasks.check_counter:Getting stats from i 2024-07-31T15:16:00.445 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph daemon mds.i perf dump 2024-07-31T15:16:00.699 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T15:16:01.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:01 smithi077 ceph-mon[37327]: pgmap v1196: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-07-31T15:16:01.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:01 smithi067 ceph-mon[32750]: pgmap v1196: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-07-31T15:16:01.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:01 smithi073 ceph-mon[36714]: pgmap v1196: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-07-31T15:16:02.539 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-07-31T15:16:02.539 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-07-31T15:16:02.539 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 107121, 2024-07-31T15:16:02.539 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 101499, 2024-07-31T15:16:02.539 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 35027060, 2024-07-31T15:16:02.540 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 68034357, 2024-07-31T15:16:02.540 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 11, 2024-07-31T15:16:02.540 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2024-07-31T15:16:02.540 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 8.161983159, 2024-07-31T15:16:02.540 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 3.077277234, 2024-07-31T15:16:02.540 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 2.762190693, 2024-07-31T15:16:02.540 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 2.239930715, 2024-07-31T15:16:02.540 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:16:02.541 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 101500, 2024-07-31T15:16:02.541 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 7.110079979, 2024-07-31T15:16:02.541 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000070050 2024-07-31T15:16:02.541 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.541 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:16:02.541 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 51462, 2024-07-31T15:16:02.541 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.010559101, 2024-07-31T15:16:02.541 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000205 2024-07-31T15:16:02.542 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.542 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1714416, 2024-07-31T15:16:02.542 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 558496 2024-07-31T15:16:02.542 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.542 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-07-31T15:16:02.542 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 54410, 2024-07-31T15:16:02.542 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 55717, 2024-07-31T15:16:02.543 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 14729809, 2024-07-31T15:16:02.543 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 56537753, 2024-07-31T15:16:02.543 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 21, 2024-07-31T15:16:02.543 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 8, 2024-07-31T15:16:02.543 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 4.039976264, 2024-07-31T15:16:02.543 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 1.463277073, 2024-07-31T15:16:02.543 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 1.150355179, 2024-07-31T15:16:02.543 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 1.200823644, 2024-07-31T15:16:02.544 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:16:02.544 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 55713, 2024-07-31T15:16:02.544 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 3.657742417, 2024-07-31T15:16:02.544 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000065653 2024-07-31T15:16:02.544 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.544 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:16:02.544 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 17245, 2024-07-31T15:16:02.544 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.003498920, 2024-07-31T15:16:02.545 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000202 2024-07-31T15:16:02.545 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.545 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-07-31T15:16:02.545 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 700096 2024-07-31T15:16:02.545 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.545 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-07-31T15:16:02.545 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 249977, 2024-07-31T15:16:02.545 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 302091, 2024-07-31T15:16:02.546 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 78325490, 2024-07-31T15:16:02.546 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 201323294, 2024-07-31T15:16:02.546 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 42, 2024-07-31T15:16:02.546 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 6, 2024-07-31T15:16:02.546 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 15.853146497, 2024-07-31T15:16:02.546 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 5.894085603, 2024-07-31T15:16:02.546 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 44.440121629, 2024-07-31T15:16:02.547 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 3.824076477, 2024-07-31T15:16:02.547 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:16:02.547 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 302090, 2024-07-31T15:16:02.547 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 33.258013212, 2024-07-31T15:16:02.547 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000110093 2024-07-31T15:16:02.547 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.547 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:16:02.548 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 192394, 2024-07-31T15:16:02.548 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.037116486, 2024-07-31T15:16:02.548 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000192 2024-07-31T15:16:02.548 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.548 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 37795152, 2024-07-31T15:16:02.548 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 77666552 2024-07-31T15:16:02.548 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.548 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-07-31T15:16:02.548 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-07-31T15:16:02.549 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-07-31T15:16:02.549 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.549 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-07-31T15:16:02.549 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-07-31T15:16:02.549 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-07-31T15:16:02.549 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 222092, 2024-07-31T15:16:02.549 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 91.761819313, 2024-07-31T15:16:02.549 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000413170 2024-07-31T15:16:02.550 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:02.550 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.550 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-07-31T15:16:02.550 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-07-31T15:16:02.550 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-07-31T15:16:02.550 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 959, 2024-07-31T15:16:02.550 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.072441242, 2024-07-31T15:16:02.550 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000075538 2024-07-31T15:16:02.550 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:02.551 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.551 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-07-31T15:16:02.551 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 25585, 2024-07-31T15:16:02.551 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 24136, 2024-07-31T15:16:02.551 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-07-31T15:16:02.551 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 24136, 2024-07-31T15:16:02.551 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 449.281303682, 2024-07-31T15:16:02.551 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.018614571 2024-07-31T15:16:02.552 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.552 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-07-31T15:16:02.552 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 1449, 2024-07-31T15:16:02.552 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 36, 2024-07-31T15:16:02.552 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-07-31T15:16:02.552 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 406, 2024-07-31T15:16:02.552 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 44, 2024-07-31T15:16:02.552 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-07-31T15:16:02.552 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 24947, 2024-07-31T15:16:02.553 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 1127, 2024-07-31T15:16:02.553 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 22694, 2024-07-31T15:16:02.553 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 1126, 2024-07-31T15:16:02.553 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 23570, 2024-07-31T15:16:02.553 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 4163, 2024-07-31T15:16:02.553 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 77, 2024-07-31T15:16:02.553 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 77, 2024-07-31T15:16:02.553 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 262, 2024-07-31T15:16:02.553 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 29639, 2024-07-31T15:16:02.554 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 25326, 2024-07-31T15:16:02.554 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 1339, 2024-07-31T15:16:02.554 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 201, 2024-07-31T15:16:02.554 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-07-31T15:16:02.554 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-07-31T15:16:02.554 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-07-31T15:16:02.554 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-07-31T15:16:02.554 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-07-31T15:16:02.555 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 160, 2024-07-31T15:16:02.555 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 4163, 2024-07-31T15:16:02.555 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 181, 2024-07-31T15:16:02.555 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 4550, 2024-07-31T15:16:02.555 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-07-31T15:16:02.555 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 268576, 2024-07-31T15:16:02.555 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-07-31T15:16:02.555 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 91397, 2024-07-31T15:16:02.556 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 2022860651, 2024-07-31T15:16:02.556 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 7, 2024-07-31T15:16:02.556 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 268576, 2024-07-31T15:16:02.556 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-07-31T15:16:02.556 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-07-31T15:16:02.556 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-07-31T15:16:02.556 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 13349, 2024-07-31T15:16:02.556 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-07-31T15:16:02.557 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 13349, 2024-07-31T15:16:02.557 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 255227, 2024-07-31T15:16:02.557 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 169, 2024-07-31T15:16:02.557 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 42254, 2024-07-31T15:16:02.557 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 22429, 2024-07-31T15:16:02.557 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 32, 2024-07-31T15:16:02.557 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-31T15:16:02.557 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-31T15:16:02.558 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 67025, 2024-07-31T15:16:02.558 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 28490, 2024-07-31T15:16:02.558 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 819, 2024-07-31T15:16:02.558 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 2780 2024-07-31T15:16:02.558 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.558 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-07-31T15:16:02.558 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-07-31T15:16:02.558 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-07-31T15:16:02.558 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-07-31T15:16:02.559 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 227, 2024-07-31T15:16:02.559 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 195, 2024-07-31T15:16:02.559 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-07-31T15:16:02.559 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-07-31T15:16:02.559 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-07-31T15:16:02.559 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 354, 2024-07-31T15:16:02.559 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 354, 2024-07-31T15:16:02.559 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-07-31T15:16:02.559 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-07-31T15:16:02.560 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-07-31T15:16:02.560 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-07-31T15:16:02.560 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-07-31T15:16:02.560 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-07-31T15:16:02.560 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-07-31T15:16:02.560 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 16, 2024-07-31T15:16:02.560 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 253646, 2024-07-31T15:16:02.560 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-07-31T15:16:02.560 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 168, 2024-07-31T15:16:02.561 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-07-31T15:16:02.561 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 44, 2024-07-31T15:16:02.561 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-07-31T15:16:02.561 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-07-31T15:16:02.561 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.561 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-07-31T15:16:02.561 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-07-31T15:16:02.561 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 56732, 2024-07-31T15:16:02.561 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-07-31T15:16:02.562 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-07-31T15:16:02.562 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 56732, 2024-07-31T15:16:02.562 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-07-31T15:16:02.562 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-07-31T15:16:02.562 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 74, 2024-07-31T15:16:02.562 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-07-31T15:16:02.562 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-07-31T15:16:02.562 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 74, 2024-07-31T15:16:02.563 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 6, 2024-07-31T15:16:02.563 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-07-31T15:16:02.563 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-07-31T15:16:02.563 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2024-07-31T15:16:02.563 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 149738264, 2024-07-31T15:16:02.563 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 0, 2024-07-31T15:16:02.563 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-07-31T15:16:02.564 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 8375, 2024-07-31T15:16:02.564 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1563.662253121, 2024-07-31T15:16:02.564 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.186705940 2024-07-31T15:16:02.564 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.564 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 0 2024-07-31T15:16:02.564 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.564 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-07-31T15:16:02.564 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 24395, 2024-07-31T15:16:02.565 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 28912, 2024-07-31T15:16:02.565 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 4517, 2024-07-31T15:16:02.565 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 1569, 2024-07-31T15:16:02.565 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 1846, 2024-07-31T15:16:02.565 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 277, 2024-07-31T15:16:02.565 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 24947, 2024-07-31T15:16:02.565 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 29943, 2024-07-31T15:16:02.565 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 4996, 2024-07-31T15:16:02.565 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 77, 2024-07-31T15:16:02.566 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 41925, 2024-07-31T15:16:02.566 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 41848, 2024-07-31T15:16:02.566 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 300752, 2024-07-31T15:16:02.566 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 223540 2024-07-31T15:16:02.566 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.566 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-07-31T15:16:02.566 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 26423, 2024-07-31T15:16:02.567 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 126, 2024-07-31T15:16:02.567 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 25585, 2024-07-31T15:16:02.567 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 187, 2024-07-31T15:16:02.567 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 750, 2024-07-31T15:16:02.567 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-07-31T15:16:02.567 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 17073, 2024-07-31T15:16:02.567 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 190.188712307, 2024-07-31T15:16:02.567 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.011139735 2024-07-31T15:16:02.568 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.568 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-07-31T15:16:02.568 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 102, 2024-07-31T15:16:02.568 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 205.980899960, 2024-07-31T15:16:02.568 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 2.019420587 2024-07-31T15:16:02.568 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.568 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-07-31T15:16:02.568 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.569 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.569 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.569 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.569 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-07-31T15:16:02.569 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.569 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.569 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.569 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.570 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-07-31T15:16:02.570 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 3099, 2024-07-31T15:16:02.570 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 43.564087674, 2024-07-31T15:16:02.570 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.014057466 2024-07-31T15:16:02.570 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.570 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-07-31T15:16:02.570 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.570 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.571 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.571 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.571 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-07-31T15:16:02.571 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.571 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.571 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.571 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.572 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-07-31T15:16:02.572 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.572 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.572 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.572 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.572 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-07-31T15:16:02.572 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.572 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.573 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.573 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.573 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-07-31T15:16:02.574 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.574 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.574 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.574 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.574 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-07-31T15:16:02.574 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.575 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.575 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.575 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.575 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-07-31T15:16:02.575 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 892, 2024-07-31T15:16:02.575 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.922950612, 2024-07-31T15:16:02.575 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001034697 2024-07-31T15:16:02.576 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.576 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-07-31T15:16:02.576 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.576 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.576 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.576 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.577 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-07-31T15:16:02.577 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.577 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.577 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.577 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.577 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-07-31T15:16:02.577 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1311, 2024-07-31T15:16:02.577 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 4.286892449, 2024-07-31T15:16:02.578 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.003269940 2024-07-31T15:16:02.578 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.578 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-07-31T15:16:02.578 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1267, 2024-07-31T15:16:02.578 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 2.367329081, 2024-07-31T15:16:02.578 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001868452 2024-07-31T15:16:02.578 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.578 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-07-31T15:16:02.579 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.579 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.579 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.579 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.579 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-07-31T15:16:02.579 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.579 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.580 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.580 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.580 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-07-31T15:16:02.580 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.580 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.580 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.580 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.580 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-07-31T15:16:02.580 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.581 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.581 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.581 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.581 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-07-31T15:16:02.581 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.581 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.581 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.581 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.581 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-07-31T15:16:02.582 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.582 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.582 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.582 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.582 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-07-31T15:16:02.582 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 36, 2024-07-31T15:16:02.582 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.952980513, 2024-07-31T15:16:02.583 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.026471680 2024-07-31T15:16:02.583 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.583 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-07-31T15:16:02.583 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.583 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.583 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.583 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.583 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-07-31T15:16:02.584 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.584 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.584 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.584 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.584 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-07-31T15:16:02.584 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.584 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.585 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.585 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.585 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-07-31T15:16:02.585 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.585 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.585 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.585 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.586 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-07-31T15:16:02.586 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2, 2024-07-31T15:16:02.586 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.002227966, 2024-07-31T15:16:02.586 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001113983 2024-07-31T15:16:02.586 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.586 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-07-31T15:16:02.586 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 354, 2024-07-31T15:16:02.587 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1.015223120, 2024-07-31T15:16:02.587 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002867861 2024-07-31T15:16:02.587 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.587 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-07-31T15:16:02.587 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-07-31T15:16:02.587 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-07-31T15:16:02.587 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.587 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.588 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.588 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:02.588 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.588 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-07-31T15:16:02.588 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 1, 2024-07-31T15:16:02.588 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 2, 2024-07-31T15:16:02.588 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 1, 2024-07-31T15:16:02.588 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 1, 2024-07-31T15:16:02.588 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-07-31T15:16:02.589 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 650, 2024-07-31T15:16:02.589 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 650, 2024-07-31T15:16:02.589 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 2995, 2024-07-31T15:16:02.589 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-07-31T15:16:02.589 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.589 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-07-31T15:16:02.589 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-07-31T15:16:02.589 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-07-31T15:16:02.590 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-07-31T15:16:02.590 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-07-31T15:16:02.590 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-07-31T15:16:02.590 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-07-31T15:16:02.590 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-31T15:16:02.590 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-07-31T15:16:02.590 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-31T15:16:02.590 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-07-31T15:16:02.591 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-07-31T15:16:02.591 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-07-31T15:16:02.591 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-31T15:16:02.591 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-07-31T15:16:02.591 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-07-31T15:16:02.591 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-07-31T15:16:02.591 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-07-31T15:16:02.591 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-07-31T15:16:02.592 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-31T15:16:02.592 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-07-31T15:16:02.592 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-31T15:16:02.592 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-07-31T15:16:02.592 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-07-31T15:16:02.592 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-07-31T15:16:02.592 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-07-31T15:16:02.592 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-07-31T15:16:02.593 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-31T15:16:02.593 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-07-31T15:16:02.593 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-07-31T15:16:02.593 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-07-31T15:16:02.593 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-07-31T15:16:02.593 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-07-31T15:16:02.593 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-07-31T15:16:02.593 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-07-31T15:16:02.593 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-07-31T15:16:02.594 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-07-31T15:16:02.594 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 987761, 2024-07-31T15:16:02.594 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 18602, 2024-07-31T15:16:02.594 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-07-31T15:16:02.594 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-07-31T15:16:02.594 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-07-31T15:16:02.594 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-07-31T15:16:02.595 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-07-31T15:16:02.595 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-07-31T15:16:02.595 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-07-31T15:16:02.595 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-07-31T15:16:02.595 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 9352, 2024-07-31T15:16:02.595 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 189, 2024-07-31T15:16:02.595 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-07-31T15:16:02.595 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-07-31T15:16:02.596 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-07-31T15:16:02.596 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-07-31T15:16:02.596 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 77410193, 2024-07-31T15:16:02.596 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 978880, 2024-07-31T15:16:02.596 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-07-31T15:16:02.596 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-07-31T15:16:02.596 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-07-31T15:16:02.596 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-07-31T15:16:02.597 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.597 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-07-31T15:16:02.597 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-07-31T15:16:02.597 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-07-31T15:16:02.597 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 279197, 2024-07-31T15:16:02.597 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 155974072, 2024-07-31T15:16:02.597 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 29, 2024-07-31T15:16:02.597 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 279197, 2024-07-31T15:16:02.597 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-07-31T15:16:02.598 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 279197, 2024-07-31T15:16:02.598 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1665.513356122, 2024-07-31T15:16:02.598 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.005965369 2024-07-31T15:16:02.598 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.598 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-07-31T15:16:02.598 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-07-31T15:16:02.598 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 279197, 2024-07-31T15:16:02.599 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 280441 2024-07-31T15:16:02.599 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.599 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 279197, 2024-07-31T15:16:02.599 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 268735, 2024-07-31T15:16:02.599 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 10462, 2024-07-31T15:16:02.599 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-07-31T15:16:02.599 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-07-31T15:16:02.599 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 0, 2024-07-31T15:16:02.600 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 10, 2024-07-31T15:16:02.600 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 80, 2024-07-31T15:16:02.600 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 8455, 2024-07-31T15:16:02.600 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 133, 2024-07-31T15:16:02.600 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-07-31T15:16:02.600 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-07-31T15:16:02.600 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-07-31T15:16:02.600 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 43, 2024-07-31T15:16:02.600 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 755, 2024-07-31T15:16:02.601 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-07-31T15:16:02.601 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-07-31T15:16:02.601 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-07-31T15:16:02.601 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 268576, 2024-07-31T15:16:02.601 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 20, 2024-07-31T15:16:02.601 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-07-31T15:16:02.601 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-07-31T15:16:02.601 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-07-31T15:16:02.602 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-07-31T15:16:02.602 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-07-31T15:16:02.602 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-07-31T15:16:02.602 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-07-31T15:16:02.602 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-07-31T15:16:02.602 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-07-31T15:16:02.602 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 32, 2024-07-31T15:16:02.602 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-07-31T15:16:02.602 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-07-31T15:16:02.603 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-07-31T15:16:02.603 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-07-31T15:16:02.603 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-07-31T15:16:02.603 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-07-31T15:16:02.603 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-07-31T15:16:02.603 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-07-31T15:16:02.603 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-07-31T15:16:02.603 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-07-31T15:16:02.604 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-07-31T15:16:02.604 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-07-31T15:16:02.604 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-07-31T15:16:02.604 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-07-31T15:16:02.604 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-07-31T15:16:02.604 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-07-31T15:16:02.604 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 138, 2024-07-31T15:16:02.604 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-07-31T15:16:02.604 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 49, 2024-07-31T15:16:02.605 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 12, 2024-07-31T15:16:02.605 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 12, 2024-07-31T15:16:02.605 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-07-31T15:16:02.605 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-07-31T15:16:02.605 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 1895, 2024-07-31T15:16:02.605 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 115, 2024-07-31T15:16:02.605 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 327 2024-07-31T15:16:02.605 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.606 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-07-31T15:16:02.606 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 1, 2024-07-31T15:16:02.606 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-07-31T15:16:02.606 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-07-31T15:16:02.606 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-07-31T15:16:02.606 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.606 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-07-31T15:16:02.607 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 253674 2024-07-31T15:16:02.607 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.607 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-07-31T15:16:02.607 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-07-31T15:16:02.607 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 10, 2024-07-31T15:16:02.607 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-07-31T15:16:02.607 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 5, 2024-07-31T15:16:02.607 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 708, 2024-07-31T15:16:02.608 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 354, 2024-07-31T15:16:02.608 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-07-31T15:16:02.608 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.608 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-31T15:16:02.608 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:16:02.608 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-07-31T15:16:02.608 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:16:02.608 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 411504, 2024-07-31T15:16:02.609 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 86209347, 2024-07-31T15:16:02.609 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:16:02.609 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 411504, 2024-07-31T15:16:02.609 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-07-31T15:16:02.609 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-07-31T15:16:02.609 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 411504, 2024-07-31T15:16:02.609 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 86209347, 2024-07-31T15:16:02.609 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:16:02.610 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.610 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.610 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.610 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:02.610 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.610 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-07-31T15:16:02.610 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:16:02.611 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-07-31T15:16:02.611 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:16:02.611 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-07-31T15:16:02.611 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-07-31T15:16:02.611 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:16:02.611 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-07-31T15:16:02.611 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 279197, 2024-07-31T15:16:02.611 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 156010457, 2024-07-31T15:16:02.612 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 279197, 2024-07-31T15:16:02.612 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 156010457, 2024-07-31T15:16:02.612 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:16:02.612 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.612 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.612 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.612 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:02.612 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.613 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-07-31T15:16:02.613 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:16:02.613 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-07-31T15:16:02.613 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:16:02.613 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-07-31T15:16:02.613 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-07-31T15:16:02.613 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:16:02.613 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-07-31T15:16:02.613 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 279197, 2024-07-31T15:16:02.614 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 279197, 2024-07-31T15:16:02.614 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 279197, 2024-07-31T15:16:02.614 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 279197, 2024-07-31T15:16:02.614 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:16:02.614 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.614 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.614 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.614 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:02.614 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.615 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-07-31T15:16:02.615 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:16:02.615 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-07-31T15:16:02.615 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:16:02.615 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 354, 2024-07-31T15:16:02.615 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 38650, 2024-07-31T15:16:02.615 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:16:02.615 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 354, 2024-07-31T15:16:02.615 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-07-31T15:16:02.616 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-07-31T15:16:02.616 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 80, 2024-07-31T15:16:02.616 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 38650, 2024-07-31T15:16:02.616 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:16:02.616 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.616 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.616 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.616 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:02.616 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:02.617 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x55fecb84e820": { 2024-07-31T15:16:02.617 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:16:02.617 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-07-31T15:16:02.617 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:16:02.617 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 56732, 2024-07-31T15:16:02.617 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 145543960, 2024-07-31T15:16:02.617 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:16:02.617 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 56732, 2024-07-31T15:16:02.618 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-07-31T15:16:02.618 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-07-31T15:16:02.618 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 8375, 2024-07-31T15:16:02.618 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 145543960, 2024-07-31T15:16:02.618 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:16:02.618 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:02.618 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:02.618 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:02.619 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:02.619 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:02.619 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-07-31T15:16:03.413 INFO:tasks.check_counter:Daemon mds.i mds.exported=160 2024-07-31T15:16:03.413 INFO:tasks.check_counter:Daemon mds.i mds.imported=181 2024-07-31T15:16:03.413 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-07-31T15:16:03.414 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-07-31T15:16:03.414 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-07-31T15:16:03.414 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=44 2024-07-31T15:16:03.414 DEBUG:tasks.check_counter:Getting stats from l 2024-07-31T15:16:03.414 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph daemon mds.l perf dump 2024-07-31T15:16:03.437 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:03 smithi077 ceph-mon[37327]: pgmap v1197: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-07-31T15:16:03.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:03 smithi067 ceph-mon[32750]: pgmap v1197: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-07-31T15:16:03.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:03 smithi073 ceph-mon[36714]: pgmap v1197: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-07-31T15:16:03.671 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.c/config 2024-07-31T15:16:05.405 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:05 smithi077 ceph-mon[37327]: pgmap v1198: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-07-31T15:16:05.445 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:05 smithi073 ceph-mon[36714]: pgmap v1198: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-07-31T15:16:05.544 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-07-31T15:16:05.544 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-07-31T15:16:05.544 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 3837, 2024-07-31T15:16:05.545 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 3791, 2024-07-31T15:16:05.545 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 30724227, 2024-07-31T15:16:05.545 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 1340320, 2024-07-31T15:16:05.545 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 7, 2024-07-31T15:16:05.545 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-07-31T15:16:05.545 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.391766596, 2024-07-31T15:16:05.545 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.121970699, 2024-07-31T15:16:05.546 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.156970732, 2024-07-31T15:16:05.546 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.106274913, 2024-07-31T15:16:05.546 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:16:05.546 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 3790, 2024-07-31T15:16:05.546 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.167009622, 2024-07-31T15:16:05.546 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000044065 2024-07-31T15:16:05.546 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.547 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:16:05.547 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.547 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.547 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.547 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.547 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1658032, 2024-07-31T15:16:05.547 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 302368 2024-07-31T15:16:05.547 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.548 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-07-31T15:16:05.548 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1054, 2024-07-31T15:16:05.548 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 1412, 2024-07-31T15:16:05.548 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 54800133, 2024-07-31T15:16:05.548 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 1119501, 2024-07-31T15:16:05.548 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 8, 2024-07-31T15:16:05.548 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 6, 2024-07-31T15:16:05.548 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.139126617, 2024-07-31T15:16:05.549 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.034762254, 2024-07-31T15:16:05.549 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.061220086, 2024-07-31T15:16:05.549 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.033900120, 2024-07-31T15:16:05.549 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:16:05.549 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1411, 2024-07-31T15:16:05.549 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.070632441, 2024-07-31T15:16:05.549 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000050058 2024-07-31T15:16:05.550 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.550 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:16:05.550 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.550 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.550 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.550 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.550 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-07-31T15:16:05.554 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 786416 2024-07-31T15:16:05.554 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.554 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-07-31T15:16:05.554 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 953, 2024-07-31T15:16:05.554 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 955, 2024-07-31T15:16:05.554 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 63164972, 2024-07-31T15:16:05.555 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 303061, 2024-07-31T15:16:05.555 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 6, 2024-07-31T15:16:05.555 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2024-07-31T15:16:05.555 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.109053329, 2024-07-31T15:16:05.555 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.022345685, 2024-07-31T15:16:05.555 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.054123488, 2024-07-31T15:16:05.555 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.026751317, 2024-07-31T15:16:05.555 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-07-31T15:16:05.556 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 953, 2024-07-31T15:16:05.556 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.033932449, 2024-07-31T15:16:05.556 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000035605 2024-07-31T15:16:05.556 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.556 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-07-31T15:16:05.556 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.556 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.556 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.556 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.557 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-31T15:16:05.557 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-07-31T15:16:05.557 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.557 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-07-31T15:16:05.557 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-07-31T15:16:05.557 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-07-31T15:16:05.557 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.558 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-07-31T15:16:05.558 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-07-31T15:16:05.558 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-07-31T15:16:05.558 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 8781, 2024-07-31T15:16:05.558 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.280083915, 2024-07-31T15:16:05.558 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000031896 2024-07-31T15:16:05.558 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:05.558 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.559 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-07-31T15:16:05.559 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-07-31T15:16:05.559 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-07-31T15:16:05.559 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.559 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.559 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.559 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:05.559 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.560 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-07-31T15:16:05.560 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 0, 2024-07-31T15:16:05.560 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 0, 2024-07-31T15:16:05.560 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-07-31T15:16:05.560 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.560 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.560 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.561 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.561 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-07-31T15:16:05.561 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2024-07-31T15:16:05.561 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-07-31T15:16:05.561 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-07-31T15:16:05.561 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 0, 2024-07-31T15:16:05.561 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-07-31T15:16:05.561 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-07-31T15:16:05.562 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 24975, 2024-07-31T15:16:05.562 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 130, 2024-07-31T15:16:05.562 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 24845, 2024-07-31T15:16:05.562 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-07-31T15:16:05.562 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 24789, 2024-07-31T15:16:05.562 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-07-31T15:16:05.562 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 0, 2024-07-31T15:16:05.562 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 0, 2024-07-31T15:16:05.563 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 230, 2024-07-31T15:16:05.563 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 0, 2024-07-31T15:16:05.563 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 0, 2024-07-31T15:16:05.563 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-07-31T15:16:05.563 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2024-07-31T15:16:05.563 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-07-31T15:16:05.563 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-07-31T15:16:05.563 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-07-31T15:16:05.564 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-07-31T15:16:05.564 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-07-31T15:16:05.564 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-07-31T15:16:05.564 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-07-31T15:16:05.564 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-07-31T15:16:05.564 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-07-31T15:16:05.564 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-07-31T15:16:05.565 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2024-07-31T15:16:05.565 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-07-31T15:16:05.565 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 0, 2024-07-31T15:16:05.565 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 0, 2024-07-31T15:16:05.565 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 0, 2024-07-31T15:16:05.565 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2024-07-31T15:16:05.565 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-07-31T15:16:05.565 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-07-31T15:16:05.566 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-07-31T15:16:05.566 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2024-07-31T15:16:05.566 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-07-31T15:16:05.566 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2024-07-31T15:16:05.566 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2024-07-31T15:16:05.566 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-07-31T15:16:05.566 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 0, 2024-07-31T15:16:05.567 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 0, 2024-07-31T15:16:05.567 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-07-31T15:16:05.567 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-31T15:16:05.567 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-31T15:16:05.567 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 0, 2024-07-31T15:16:05.567 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 0, 2024-07-31T15:16:05.567 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 0, 2024-07-31T15:16:05.567 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 0 2024-07-31T15:16:05.568 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.568 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-07-31T15:16:05.568 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-07-31T15:16:05.568 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-07-31T15:16:05.568 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-07-31T15:16:05.568 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2024-07-31T15:16:05.568 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2024-07-31T15:16:05.568 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-07-31T15:16:05.569 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-07-31T15:16:05.569 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-07-31T15:16:05.569 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 0, 2024-07-31T15:16:05.569 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 0, 2024-07-31T15:16:05.569 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-07-31T15:16:05.569 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-07-31T15:16:05.569 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-07-31T15:16:05.569 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-07-31T15:16:05.570 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-07-31T15:16:05.570 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-07-31T15:16:05.570 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-07-31T15:16:05.570 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 0, 2024-07-31T15:16:05.570 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 0, 2024-07-31T15:16:05.570 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-07-31T15:16:05.570 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-07-31T15:16:05.570 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-07-31T15:16:05.570 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-07-31T15:16:05.571 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-07-31T15:16:05.571 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-07-31T15:16:05.571 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.571 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-07-31T15:16:05.571 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-07-31T15:16:05.571 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 0, 2024-07-31T15:16:05.571 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-07-31T15:16:05.571 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-07-31T15:16:05.572 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 56732, 2024-07-31T15:16:05.572 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-07-31T15:16:05.572 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-07-31T15:16:05.572 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 0, 2024-07-31T15:16:05.572 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-07-31T15:16:05.572 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-07-31T15:16:05.572 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 74, 2024-07-31T15:16:05.572 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 6, 2024-07-31T15:16:05.573 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-07-31T15:16:05.573 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-07-31T15:16:05.573 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2024-07-31T15:16:05.573 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 149738264, 2024-07-31T15:16:05.573 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 149737166, 2024-07-31T15:16:05.573 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-07-31T15:16:05.573 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.573 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.574 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.574 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.574 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 56732 2024-07-31T15:16:05.574 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.574 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-07-31T15:16:05.574 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 24271, 2024-07-31T15:16:05.574 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 28777, 2024-07-31T15:16:05.574 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 4506, 2024-07-31T15:16:05.575 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 1461, 2024-07-31T15:16:05.575 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 1775, 2024-07-31T15:16:05.575 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 314, 2024-07-31T15:16:05.575 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 24975, 2024-07-31T15:16:05.575 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 29127, 2024-07-31T15:16:05.575 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 4152, 2024-07-31T15:16:05.575 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 0, 2024-07-31T15:16:05.575 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 0, 2024-07-31T15:16:05.576 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 0, 2024-07-31T15:16:05.576 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 210464, 2024-07-31T15:16:05.576 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 215348 2024-07-31T15:16:05.576 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.576 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-07-31T15:16:05.576 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 0, 2024-07-31T15:16:05.576 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 0, 2024-07-31T15:16:05.576 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 0, 2024-07-31T15:16:05.576 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 0, 2024-07-31T15:16:05.577 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 0, 2024-07-31T15:16:05.577 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-07-31T15:16:05.577 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.577 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.577 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.577 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.577 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-07-31T15:16:05.577 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.578 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.578 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.578 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.578 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-07-31T15:16:05.578 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.578 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.578 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.579 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.579 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-07-31T15:16:05.579 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.579 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.579 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.579 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.579 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-07-31T15:16:05.579 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.579 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.580 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.580 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.580 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-07-31T15:16:05.580 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.580 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.580 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.580 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.581 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-07-31T15:16:05.581 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.581 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.581 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.581 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.581 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-07-31T15:16:05.581 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.582 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.582 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.582 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.582 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-07-31T15:16:05.582 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.582 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.582 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.583 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.583 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-07-31T15:16:05.583 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.583 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.583 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.583 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.583 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-07-31T15:16:05.584 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.584 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.584 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.584 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.585 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:05 smithi067 ceph-mon[32750]: pgmap v1198: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-07-31T15:16:05.585 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-07-31T15:16:05.585 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.586 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.586 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.586 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.586 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-07-31T15:16:05.586 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.586 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.586 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.587 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.587 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-07-31T15:16:05.587 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.587 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.587 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.587 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.587 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-07-31T15:16:05.587 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.588 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.588 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.588 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.588 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-07-31T15:16:05.588 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.588 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.588 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.588 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.589 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-07-31T15:16:05.589 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.589 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.589 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.589 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.589 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-07-31T15:16:05.589 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.590 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.590 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.590 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.590 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-07-31T15:16:05.590 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.590 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.590 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.590 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.591 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-07-31T15:16:05.591 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.591 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.591 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.591 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.591 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-07-31T15:16:05.591 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.591 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.592 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.592 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.592 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-07-31T15:16:05.592 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.592 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.592 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.592 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.593 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-07-31T15:16:05.593 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.593 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.593 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.593 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.593 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-07-31T15:16:05.593 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.593 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.594 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.594 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.594 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-07-31T15:16:05.594 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.594 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.594 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.594 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.595 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-07-31T15:16:05.595 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.595 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.595 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.595 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.595 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-07-31T15:16:05.595 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.595 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.596 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.596 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.596 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-07-31T15:16:05.596 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.596 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.596 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.596 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.596 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-07-31T15:16:05.597 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.597 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.597 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.597 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.597 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-07-31T15:16:05.597 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-07-31T15:16:05.597 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-07-31T15:16:05.597 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.598 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.598 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.598 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:05.598 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.598 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-07-31T15:16:05.598 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 1, 2024-07-31T15:16:05.598 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 2, 2024-07-31T15:16:05.599 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 1, 2024-07-31T15:16:05.599 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 1, 2024-07-31T15:16:05.599 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-07-31T15:16:05.599 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 0, 2024-07-31T15:16:05.599 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 0, 2024-07-31T15:16:05.599 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 2999, 2024-07-31T15:16:05.599 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-07-31T15:16:05.600 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.600 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-07-31T15:16:05.600 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-07-31T15:16:05.600 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-07-31T15:16:05.600 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-07-31T15:16:05.600 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-07-31T15:16:05.600 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-07-31T15:16:05.600 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-07-31T15:16:05.600 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-31T15:16:05.601 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-07-31T15:16:05.601 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-31T15:16:05.601 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-07-31T15:16:05.601 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-07-31T15:16:05.601 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-07-31T15:16:05.601 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-31T15:16:05.601 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-07-31T15:16:05.601 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-07-31T15:16:05.602 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-07-31T15:16:05.602 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-07-31T15:16:05.602 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-07-31T15:16:05.602 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-31T15:16:05.602 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-07-31T15:16:05.602 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-31T15:16:05.602 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-07-31T15:16:05.603 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-07-31T15:16:05.603 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-07-31T15:16:05.603 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-07-31T15:16:05.603 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-07-31T15:16:05.603 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-31T15:16:05.603 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-07-31T15:16:05.603 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-07-31T15:16:05.603 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-07-31T15:16:05.604 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-07-31T15:16:05.604 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-07-31T15:16:05.604 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-07-31T15:16:05.604 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-07-31T15:16:05.604 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-07-31T15:16:05.604 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-07-31T15:16:05.604 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 1165044, 2024-07-31T15:16:05.604 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 25094, 2024-07-31T15:16:05.605 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-07-31T15:16:05.605 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-07-31T15:16:05.605 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-07-31T15:16:05.605 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-07-31T15:16:05.605 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-07-31T15:16:05.605 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-07-31T15:16:05.605 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-07-31T15:16:05.605 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-07-31T15:16:05.606 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 9352, 2024-07-31T15:16:05.606 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 189, 2024-07-31T15:16:05.606 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-07-31T15:16:05.606 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-07-31T15:16:05.606 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-07-31T15:16:05.606 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-07-31T15:16:05.606 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 76210823, 2024-07-31T15:16:05.607 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 850784, 2024-07-31T15:16:05.607 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-07-31T15:16:05.607 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-07-31T15:16:05.607 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-07-31T15:16:05.607 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-07-31T15:16:05.607 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.607 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-07-31T15:16:05.607 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-07-31T15:16:05.607 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-07-31T15:16:05.608 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 5264, 2024-07-31T15:16:05.608 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 21, 2024-07-31T15:16:05.608 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 3, 2024-07-31T15:16:05.608 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 5264, 2024-07-31T15:16:05.608 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-07-31T15:16:05.608 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 5264, 2024-07-31T15:16:05.608 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 60.821700829, 2024-07-31T15:16:05.608 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.011554274 2024-07-31T15:16:05.609 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.609 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-07-31T15:16:05.609 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-07-31T15:16:05.609 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 5264, 2024-07-31T15:16:05.609 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 5265 2024-07-31T15:16:05.609 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.609 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 5264, 2024-07-31T15:16:05.609 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 5264, 2024-07-31T15:16:05.610 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 0, 2024-07-31T15:16:05.610 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-07-31T15:16:05.610 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-07-31T15:16:05.610 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 3239, 2024-07-31T15:16:05.610 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 0, 2024-07-31T15:16:05.610 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 2023, 2024-07-31T15:16:05.610 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 0, 2024-07-31T15:16:05.610 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 0, 2024-07-31T15:16:05.611 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-07-31T15:16:05.611 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-07-31T15:16:05.611 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-07-31T15:16:05.611 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-07-31T15:16:05.611 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 0, 2024-07-31T15:16:05.611 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-07-31T15:16:05.611 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-07-31T15:16:05.611 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-07-31T15:16:05.612 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 1, 2024-07-31T15:16:05.612 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 0, 2024-07-31T15:16:05.612 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-07-31T15:16:05.612 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-07-31T15:16:05.612 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-07-31T15:16:05.612 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-07-31T15:16:05.612 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-07-31T15:16:05.612 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-07-31T15:16:05.613 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-07-31T15:16:05.613 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-07-31T15:16:05.613 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-07-31T15:16:05.613 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-07-31T15:16:05.613 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-07-31T15:16:05.613 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-07-31T15:16:05.613 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-07-31T15:16:05.613 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-07-31T15:16:05.613 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-07-31T15:16:05.614 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-07-31T15:16:05.614 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-07-31T15:16:05.614 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-07-31T15:16:05.614 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-07-31T15:16:05.614 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-07-31T15:16:05.614 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-07-31T15:16:05.614 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-07-31T15:16:05.615 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-07-31T15:16:05.615 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-07-31T15:16:05.615 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-07-31T15:16:05.615 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-07-31T15:16:05.615 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 138, 2024-07-31T15:16:05.615 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-07-31T15:16:05.615 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 49, 2024-07-31T15:16:05.615 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 11, 2024-07-31T15:16:05.615 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 11, 2024-07-31T15:16:05.616 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-07-31T15:16:05.616 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-07-31T15:16:05.616 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 0, 2024-07-31T15:16:05.616 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 2, 2024-07-31T15:16:05.616 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 0 2024-07-31T15:16:05.616 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.616 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-07-31T15:16:05.616 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 0, 2024-07-31T15:16:05.617 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-07-31T15:16:05.617 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-07-31T15:16:05.617 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-07-31T15:16:05.617 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.617 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-07-31T15:16:05.617 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 0 2024-07-31T15:16:05.617 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.617 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-07-31T15:16:05.617 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-07-31T15:16:05.618 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 0, 2024-07-31T15:16:05.618 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-07-31T15:16:05.618 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 0, 2024-07-31T15:16:05.618 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 0, 2024-07-31T15:16:05.618 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 0, 2024-07-31T15:16:05.618 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-07-31T15:16:05.618 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.619 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-31T15:16:05.619 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:16:05.619 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-07-31T15:16:05.619 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:16:05.619 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 5840, 2024-07-31T15:16:05.619 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 148131673, 2024-07-31T15:16:05.619 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:16:05.619 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 5840, 2024-07-31T15:16:05.619 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-07-31T15:16:05.620 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-07-31T15:16:05.620 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 5840, 2024-07-31T15:16:05.620 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 148131673, 2024-07-31T15:16:05.620 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:16:05.620 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.620 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.620 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.620 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:05.621 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.621 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-07-31T15:16:05.621 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:16:05.621 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-07-31T15:16:05.621 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:16:05.621 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-07-31T15:16:05.621 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-07-31T15:16:05.621 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:16:05.621 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-07-31T15:16:05.622 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 5264, 2024-07-31T15:16:05.622 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 145556560, 2024-07-31T15:16:05.622 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 5264, 2024-07-31T15:16:05.622 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 145556560, 2024-07-31T15:16:05.622 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:16:05.622 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.622 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.622 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.623 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:05.623 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.623 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-07-31T15:16:05.623 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:16:05.623 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-07-31T15:16:05.623 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:16:05.623 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-07-31T15:16:05.623 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-07-31T15:16:05.624 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:16:05.624 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-07-31T15:16:05.624 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 5264, 2024-07-31T15:16:05.624 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 5264, 2024-07-31T15:16:05.624 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 5264, 2024-07-31T15:16:05.624 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 5264, 2024-07-31T15:16:05.624 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:16:05.625 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.625 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.625 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.625 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:05.625 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.625 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-07-31T15:16:05.625 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:16:05.625 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-07-31T15:16:05.626 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:16:05.626 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-07-31T15:16:05.626 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-07-31T15:16:05.626 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:16:05.626 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-07-31T15:16:05.626 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-07-31T15:16:05.626 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-07-31T15:16:05.627 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-07-31T15:16:05.627 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-07-31T15:16:05.627 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:16:05.627 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.627 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.627 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.627 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:05.627 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-07-31T15:16:05.627 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x55d1dbc648f0": { 2024-07-31T15:16:05.628 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-07-31T15:16:05.628 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-07-31T15:16:05.628 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-07-31T15:16:05.628 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-07-31T15:16:05.628 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-07-31T15:16:05.628 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-07-31T15:16:05.628 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-07-31T15:16:05.629 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-07-31T15:16:05.629 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-07-31T15:16:05.629 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-07-31T15:16:05.629 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-07-31T15:16:05.629 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-07-31T15:16:05.629 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-07-31T15:16:05.629 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-07-31T15:16:05.629 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-07-31T15:16:05.630 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:05.630 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-07-31T15:16:05.630 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-07-31T15:16:06.298 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-07-31T15:16:06.298 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-07-31T15:16:06.298 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-07-31T15:16:06.298 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-07-31T15:16:06.298 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-07-31T15:16:06.299 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-07-31T15:16:06.299 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-07-31T15:16:06.311 INFO:tasks.kclient:Unmounting kernel clients... 2024-07-31T15:16:06.311 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T15:16:06.311 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-31T15:16:06.342 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-31T15:16:06.343 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-31T15:16:06.403 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd blocklist ls 2024-07-31T15:16:06.719 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:16:07.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:07 smithi077 ceph-mon[37327]: pgmap v1199: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-07-31T15:16:07.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:07 smithi067 ceph-mon[32750]: pgmap v1199: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-07-31T15:16:07.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:07 smithi073 ceph-mon[36714]: pgmap v1199: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-07-31T15:16:08.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.563+0000 7f6b53089640 1 -- 172.21.15.67:0/3339467854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b4c1019f0 msgr2=0x7f6b4c101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.563+0000 7f6b53089640 1 --2- 172.21.15.67:0/3339467854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b4c1019f0 0x7f6b4c101e00 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009ca0 tx=0x7f6b3c02a8c0 comp rx=0 tx=0).stop 2024-07-31T15:16:08.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.563+0000 7f6b53089640 1 -- 172.21.15.67:0/3339467854 shutdown_connections 2024-07-31T15:16:08.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.563+0000 7f6b53089640 1 --2- 172.21.15.67:0/3339467854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c109880 0x7f6b4c1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:08.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.563+0000 7f6b53089640 1 --2- 172.21.15.67:0/3339467854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b4c108f30 0x7f6b4c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:08.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.563+0000 7f6b53089640 1 --2- 172.21.15.67:0/3339467854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b4c1019f0 0x7f6b4c101e00 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:08.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.563+0000 7f6b53089640 1 -- 172.21.15.67:0/3339467854 >> 172.21.15.67:0/3339467854 conn(0x7f6b4c0799c0 msgr2=0x7f6b4c079dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:08.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.563+0000 7f6b53089640 1 -- 172.21.15.67:0/3339467854 shutdown_connections 2024-07-31T15:16:08.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.563+0000 7f6b53089640 1 -- 172.21.15.67:0/3339467854 wait complete. 2024-07-31T15:16:08.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.564+0000 7f6b53089640 1 Processor -- start 2024-07-31T15:16:08.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.564+0000 7f6b53089640 1 -- start start 2024-07-31T15:16:08.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.564+0000 7f6b53089640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c1019f0 0x7f6b4c199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:08.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.564+0000 7f6b53089640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b4c108f30 0x7f6b4c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:08.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.564+0000 7f6b53089640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b4c109880 0x7f6b4c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:08.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.564+0000 7f6b53089640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c113550 con 0x7f6b4c108f30 2024-07-31T15:16:08.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.564+0000 7f6b53089640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1133d0 con 0x7f6b4c109880 2024-07-31T15:16:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.564+0000 7f6b53089640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1136d0 con 0x7f6b4c1019f0 2024-07-31T15:16:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b4bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b4c108f30 0x7f6b4c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b4bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b4c108f30 0x7f6b4c19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35930/0 (socket says 172.21.15.67:35930) 2024-07-31T15:16:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b4bfff640 1 -- 172.21.15.67:0/3204880822 learned_addr learned my addr 172.21.15.67:0/3204880822 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:16:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b50e25640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c1019f0 0x7f6b4c199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b51626640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b4c109880 0x7f6b4c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b4bfff640 1 -- 172.21.15.67:0/3204880822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c1019f0 msgr2=0x7f6b4c199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b4bfff640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c1019f0 0x7f6b4c199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b4bfff640 1 -- 172.21.15.67:0/3204880822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b4c109880 msgr2=0x7f6b4c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b4bfff640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b4c109880 0x7f6b4c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b4bfff640 1 -- 172.21.15.67:0/3204880822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4c19feb0 con 0x7f6b4c108f30 2024-07-31T15:16:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b51626640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b4c109880 0x7f6b4c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-31T15:16:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b4bfff640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b4c108f30 0x7f6b4c19a4e0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400d740 tx=0x7f6b3400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b49ffb640 1 -- 172.21.15.67:0/3204880822 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34017070 con 0x7f6b4c108f30 2024-07-31T15:16:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b49ffb640 1 -- 172.21.15.67:0/3204880822 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b3400de50 con 0x7f6b4c108f30 2024-07-31T15:16:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b49ffb640 1 -- 172.21.15.67:0/3204880822 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3400f620 con 0x7f6b4c108f30 2024-07-31T15:16:08.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.565+0000 7f6b53089640 1 -- 172.21.15.67:0/3204880822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4c1a01a0 con 0x7f6b4c108f30 2024-07-31T15:16:08.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.566+0000 7f6b53089640 1 -- 172.21.15.67:0/3204880822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b4c1a6900 con 0x7f6b4c108f30 2024-07-31T15:16:08.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.567+0000 7f6b53089640 1 -- 172.21.15.67:0/3204880822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b4c101e00 con 0x7f6b4c108f30 2024-07-31T15:16:08.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.568+0000 7f6b49ffb640 1 -- 172.21.15.67:0/3204880822 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6b3401a070 con 0x7f6b4c108f30 2024-07-31T15:16:08.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.569+0000 7f6b49ffb640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6b380b3d40 0x7f6b380b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:08.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.569+0000 7f6b50e25640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6b380b3d40 0x7f6b380b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:08.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.569+0000 7f6b49ffb640 1 -- 172.21.15.67:0/3204880822 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9395+0+0 (secure 0 0 0) 0x7f6b340e3d50 con 0x7f6b4c108f30 2024-07-31T15:16:08.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.569+0000 7f6b50e25640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6b380b3d40 0x7f6b380b6250 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c005e30 tx=0x7f6b3c009360 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:08.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.573+0000 7f6b49ffb640 1 -- 172.21.15.67:0/3204880822 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f6b340e7030 con 0x7f6b4c108f30 2024-07-31T15:16:08.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.782+0000 7f6b53089640 1 -- 172.21.15.67:0/3204880822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6b4c02e0c0 con 0x7f6b4c108f30 2024-07-31T15:16:08.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.782+0000 7f6b49ffb640 1 -- 172.21.15.67:0/3204880822 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v138) ==== 81+0+884 (secure 0 0 0) 0x7f6b340aacf0 con 0x7f6b4c108f30 2024-07-31T15:16:08.784 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/3601135096 2024-08-01T14:40:19.281779+0000 2024-07-31T15:16:08.784 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/3601135096 2024-08-01T14:40:19.281779+0000 2024-07-31T15:16:08.785 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/4249598263 2024-08-01T14:39:07.942187+0000 2024-07-31T15:16:08.785 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/927641174 2024-08-01T14:38:41.486084+0000 2024-07-31T15:16:08.785 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3901988655 2024-08-01T14:38:41.486084+0000 2024-07-31T15:16:08.785 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/1712042445 2024-08-01T14:38:41.486084+0000 2024-07-31T15:16:08.785 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1243394894 2024-08-01T14:39:07.942187+0000 2024-07-31T15:16:08.785 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1563471445 2024-08-01T14:39:07.942187+0000 2024-07-31T15:16:08.785 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/1712042445 2024-08-01T14:38:41.486084+0000 2024-07-31T15:16:08.786 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/76992902 2024-08-01T14:39:07.942187+0000 2024-07-31T15:16:08.786 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3065587341 2024-08-01T14:38:41.486084+0000 2024-07-31T15:16:08.786 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/4249598263 2024-08-01T14:39:07.942187+0000 2024-07-31T15:16:08.786 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3411853619 2024-08-01T14:40:19.281779+0000 2024-07-31T15:16:08.786 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/618685030 2024-08-01T14:40:19.281779+0000 2024-07-31T15:16:08.786 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2614585786 2024-08-01T14:40:19.281779+0000 2024-07-31T15:16:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:listed 15 entries 2024-07-31T15:16:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.785+0000 7f6b53089640 1 -- 172.21.15.67:0/3204880822 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6b380b3d40 msgr2=0x7f6b380b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.785+0000 7f6b53089640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6b380b3d40 0x7f6b380b6250 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c005e30 tx=0x7f6b3c009360 comp rx=0 tx=0).stop 2024-07-31T15:16:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.785+0000 7f6b53089640 1 -- 172.21.15.67:0/3204880822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b4c108f30 msgr2=0x7f6b4c19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.785+0000 7f6b53089640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b4c108f30 0x7f6b4c19a4e0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400d740 tx=0x7f6b3400dc00 comp rx=0 tx=0).stop 2024-07-31T15:16:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.785+0000 7f6b53089640 1 -- 172.21.15.67:0/3204880822 shutdown_connections 2024-07-31T15:16:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.785+0000 7f6b53089640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f6b380b3d40 0x7f6b380b6250 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.785+0000 7f6b53089640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b4c109880 0x7f6b4c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.785+0000 7f6b53089640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b4c108f30 0x7f6b4c19a4e0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.785+0000 7f6b53089640 1 --2- 172.21.15.67:0/3204880822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c1019f0 0x7f6b4c199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.785+0000 7f6b53089640 1 -- 172.21.15.67:0/3204880822 >> 172.21.15.67:0/3204880822 conn(0x7f6b4c0799c0 msgr2=0x7f6b4c106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.785+0000 7f6b53089640 1 -- 172.21.15.67:0/3204880822 shutdown_connections 2024-07-31T15:16:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:08.785+0000 7f6b53089640 1 -- 172.21.15.67:0/3204880822 wait complete. 2024-07-31T15:16:09.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:09 smithi067 ceph-mon[32750]: pgmap v1200: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-07-31T15:16:09.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:09 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3204880822' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-31T15:16:09.441 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-31T15:16:09.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T15:16:09.441 DEBUG:teuthology.orchestra.run.smithi067:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-31T15:16:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:09 smithi077 ceph-mon[37327]: pgmap v1200: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-07-31T15:16:09.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:09 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3204880822' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-31T15:16:09.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:09 smithi073 ceph-mon[36714]: pgmap v1200: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-07-31T15:16:09.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:09 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3204880822' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-31T15:16:09.685 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-07-31T15:16:09.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T15:16:09.685 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-31T15:16:09.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T15:16:09.727 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-07-31T15:16:09.816 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-31T15:16:09.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T15:16:09.817 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-31T15:16:09.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-31T15:16:09.869 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-07-31T15:16:09.945 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-07-31T15:16:09.956 INFO:tasks.cephadm:Teardown begin 2024-07-31T15:16:09.956 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-31T15:16:09.957 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:16:10.238 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:16:11.479 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:11 smithi067 ceph-mon[32750]: pgmap v1201: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-07-31T15:16:11.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:11 smithi077 ceph-mon[37327]: pgmap v1201: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-07-31T15:16:11.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:11 smithi073 ceph-mon[36714]: pgmap v1201: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-07-31T15:16:12.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.235+0000 7ff83d0eb640 1 -- 172.21.15.67:0/3022097345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83810fe50 msgr2=0x7ff838116680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:12.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.235+0000 7ff83d0eb640 1 --2- 172.21.15.67:0/3022097345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83810fe50 0x7ff838116680 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ff830009be0 tx=0x7ff83002a710 comp rx=0 tx=0).stop 2024-07-31T15:16:12.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.240+0000 7ff83d0eb640 1 -- 172.21.15.67:0/3022097345 shutdown_connections 2024-07-31T15:16:12.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.240+0000 7ff83d0eb640 1 --2- 172.21.15.67:0/3022097345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83810fe50 0x7ff838116680 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:12.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.240+0000 7ff83d0eb640 1 --2- 172.21.15.67:0/3022097345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff83810f500 0x7ff83810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:12.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.240+0000 7ff83d0eb640 1 --2- 172.21.15.67:0/3022097345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff838109bc0 0x7ff838109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:12.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.240+0000 7ff83d0eb640 1 -- 172.21.15.67:0/3022097345 >> 172.21.15.67:0/3022097345 conn(0x7ff838077b20 msgr2=0x7ff838107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:12.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.241+0000 7ff83d0eb640 1 -- 172.21.15.67:0/3022097345 shutdown_connections 2024-07-31T15:16:12.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.242+0000 7ff83d0eb640 1 -- 172.21.15.67:0/3022097345 wait complete. 2024-07-31T15:16:12.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.243+0000 7ff83d0eb640 1 Processor -- start 2024-07-31T15:16:12.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.243+0000 7ff83d0eb640 1 -- start start 2024-07-31T15:16:12.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.243+0000 7ff83d0eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff838109bc0 0x7ff838083790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:12.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.243+0000 7ff83d0eb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83810f500 0x7ff83807da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:12.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.243+0000 7ff83d0eb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff83810fe50 0x7ff83807e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:12.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.244+0000 7ff83d0eb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff83807bf00 con 0x7ff83810f500 2024-07-31T15:16:12.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.244+0000 7ff83d0eb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff83807bd80 con 0x7ff83810fe50 2024-07-31T15:16:12.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.244+0000 7ff83d0eb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff83807c080 con 0x7ff838109bc0 2024-07-31T15:16:12.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.245+0000 7ff83759e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff83810fe50 0x7ff83807e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:12.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.245+0000 7ff836d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff838109bc0 0x7ff838083790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:12.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.245+0000 7ff83759e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff83810fe50 0x7ff83807e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:50134/0 (socket says 172.21.15.67:50134) 2024-07-31T15:16:12.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.245+0000 7ff83759e640 1 -- 172.21.15.67:0/2208980959 learned_addr learned my addr 172.21.15.67:0/2208980959 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:16:12.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.245+0000 7ff83659c640 1 --2- 172.21.15.67:0/2208980959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83810f500 0x7ff83807da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:12.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.247+0000 7ff83659c640 1 -- 172.21.15.67:0/2208980959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff838109bc0 msgr2=0x7ff838083790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:12.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.247+0000 7ff83659c640 1 --2- 172.21.15.67:0/2208980959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff838109bc0 0x7ff838083790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:12.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.247+0000 7ff83659c640 1 -- 172.21.15.67:0/2208980959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff83810fe50 msgr2=0x7ff83807e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:12.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.247+0000 7ff83659c640 1 --2- 172.21.15.67:0/2208980959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff83810fe50 0x7ff83807e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:12.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.247+0000 7ff83659c640 1 -- 172.21.15.67:0/2208980959 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8381b4fa0 con 0x7ff83810f500 2024-07-31T15:16:12.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.247+0000 7ff83659c640 1 --2- 172.21.15.67:0/2208980959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83810f500 0x7ff83807da50 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ff83806b560 tx=0x7ff82c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:12.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.248+0000 7ff81ffff640 1 -- 172.21.15.67:0/2208980959 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c00cc40 con 0x7ff83810f500 2024-07-31T15:16:12.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.249+0000 7ff81ffff640 1 -- 172.21.15.67:0/2208980959 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff82c00cde0 con 0x7ff83810f500 2024-07-31T15:16:12.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.249+0000 7ff81ffff640 1 -- 172.21.15.67:0/2208980959 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0187f0 con 0x7ff83810f500 2024-07-31T15:16:12.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.249+0000 7ff83d0eb640 1 -- 172.21.15.67:0/2208980959 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8381b5170 con 0x7ff83810f500 2024-07-31T15:16:12.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.249+0000 7ff83d0eb640 1 -- 172.21.15.67:0/2208980959 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8381b55d0 con 0x7ff83810f500 2024-07-31T15:16:12.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.254+0000 7ff83d0eb640 1 -- 172.21.15.67:0/2208980959 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8240052c0 con 0x7ff83810f500 2024-07-31T15:16:12.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.254+0000 7ff81ffff640 1 -- 172.21.15.67:0/2208980959 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff82c018990 con 0x7ff83810f500 2024-07-31T15:16:12.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.256+0000 7ff81ffff640 1 --2- 172.21.15.67:0/2208980959 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8200b3f50 0x7ff8200b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:12.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.256+0000 7ff81ffff640 1 -- 172.21.15.67:0/2208980959 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9483+0+0 (secure 0 0 0) 0x7ff82c0e44d0 con 0x7ff83810f500 2024-07-31T15:16:12.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.257+0000 7ff836d9d640 1 --2- 172.21.15.67:0/2208980959 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8200b3f50 0x7ff8200b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:12.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.258+0000 7ff836d9d640 1 --2- 172.21.15.67:0/2208980959 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8200b3f50 0x7ff8200b6460 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff8380834e0 tx=0x7ff828009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.266+0000 7ff81ffff640 1 -- 172.21.15.67:0/2208980959 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff82c0ab3c0 con 0x7ff83810f500 2024-07-31T15:16:12.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:12 smithi077 ceph-mon[37327]: osdmap e139: 12 total, 12 up, 12 in 2024-07-31T15:16:12.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:12 smithi067 ceph-mon[32750]: osdmap e139: 12 total, 12 up, 12 in 2024-07-31T15:16:12.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:12 smithi073 ceph-mon[36714]: osdmap e139: 12 total, 12 up, 12 in 2024-07-31T15:16:12.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.720+0000 7ff83d0eb640 1 -- 172.21.15.67:0/2208980959 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff824005880 con 0x7ff83810f500 2024-07-31T15:16:12.725 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:16:12.726 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":66,"btime":"2024-07-31T15:10:30:235518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:10:30.235494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29129":{"gid":29129,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/631186801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":631186801},{"type":"v1","addr":"172.21.15.73:6835","nonce":631186801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:16:12.726 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 66 2024-07-31T15:16:12.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.723+0000 7ff81ffff640 1 -- 172.21.15.67:0/2208980959 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11959 (secure 0 0 0) 0x7ff82c0aacc0 con 0x7ff83810f500 2024-07-31T15:16:12.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.730+0000 7ff81dffb640 1 -- 172.21.15.67:0/2208980959 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8200b3f50 msgr2=0x7ff8200b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:12.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.730+0000 7ff81dffb640 1 --2- 172.21.15.67:0/2208980959 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8200b3f50 0x7ff8200b6460 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff8380834e0 tx=0x7ff828009260 comp rx=0 tx=0).stop 2024-07-31T15:16:12.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.730+0000 7ff81dffb640 1 -- 172.21.15.67:0/2208980959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83810f500 msgr2=0x7ff83807da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:12.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.730+0000 7ff81dffb640 1 --2- 172.21.15.67:0/2208980959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83810f500 0x7ff83807da50 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ff83806b560 tx=0x7ff82c00ee50 comp rx=0 tx=0).stop 2024-07-31T15:16:12.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.734+0000 7ff81dffb640 1 -- 172.21.15.67:0/2208980959 shutdown_connections 2024-07-31T15:16:12.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.734+0000 7ff81dffb640 1 --2- 172.21.15.67:0/2208980959 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff8200b3f50 0x7ff8200b6460 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:12.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.734+0000 7ff81dffb640 1 --2- 172.21.15.67:0/2208980959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff83810fe50 0x7ff83807e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:12.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.734+0000 7ff81dffb640 1 --2- 172.21.15.67:0/2208980959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83810f500 0x7ff83807da50 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:12.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.734+0000 7ff81dffb640 1 --2- 172.21.15.67:0/2208980959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff838109bc0 0x7ff838083790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:12.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.734+0000 7ff81dffb640 1 -- 172.21.15.67:0/2208980959 >> 172.21.15.67:0/2208980959 conn(0x7ff838077b20 msgr2=0x7ff83810d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:12.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.738+0000 7ff81dffb640 1 -- 172.21.15.67:0/2208980959 shutdown_connections 2024-07-31T15:16:12.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:12.738+0000 7ff81dffb640 1 -- 172.21.15.67:0/2208980959 wait complete. 2024-07-31T15:16:13.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:13 smithi067 ceph-mon[32750]: pgmap v1203: 129 pgs: 6 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 103 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-07-31T15:16:13.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:13 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2208980959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:16:13.464 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-07-31T15:16:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:13 smithi077 ceph-mon[37327]: pgmap v1203: 129 pgs: 6 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 103 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-07-31T15:16:13.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:13 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2208980959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:16:13.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:13 smithi073 ceph-mon[36714]: pgmap v1203: 129 pgs: 6 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 103 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-07-31T15:16:13.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:13 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2208980959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:16:13.941 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:16:15.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:15 smithi067 ceph-mon[32750]: pgmap v1204: 129 pgs: 14 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 75 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-07-31T15:16:15.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:15 smithi077 ceph-mon[37327]: pgmap v1204: 129 pgs: 14 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 75 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-07-31T15:16:15.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:15 smithi073 ceph-mon[36714]: pgmap v1204: 129 pgs: 14 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 75 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-07-31T15:16:15.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.886+0000 7f0a81b77640 1 -- 172.21.15.67:0/3887744219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a7c107dd0 msgr2=0x7f0a7c1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:15.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.886+0000 7f0a81b77640 1 --2- 172.21.15.67:0/3887744219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a7c107dd0 0x7f0a7c1081e0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009900 tx=0x7f0a7002a410 comp rx=0 tx=0).stop 2024-07-31T15:16:15.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.886+0000 7f0a81b77640 1 -- 172.21.15.67:0/3887744219 shutdown_connections 2024-07-31T15:16:15.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.886+0000 7f0a81b77640 1 --2- 172.21.15.67:0/3887744219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a7c108720 0x7f0a7c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:15.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.886+0000 7f0a81b77640 1 --2- 172.21.15.67:0/3887744219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a7c107dd0 0x7f0a7c1081e0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:15.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.886+0000 7f0a81b77640 1 --2- 172.21.15.67:0/3887744219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a7c102490 0x7f0a7c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:15.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.886+0000 7f0a81b77640 1 -- 172.21.15.67:0/3887744219 >> 172.21.15.67:0/3887744219 conn(0x7f0a7c079160 msgr2=0x7f0a7c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:15.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.886+0000 7f0a81b77640 1 -- 172.21.15.67:0/3887744219 shutdown_connections 2024-07-31T15:16:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.886+0000 7f0a81b77640 1 -- 172.21.15.67:0/3887744219 wait complete. 2024-07-31T15:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.888+0000 7f0a81b77640 1 Processor -- start 2024-07-31T15:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.888+0000 7f0a81b77640 1 -- start start 2024-07-31T15:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.888+0000 7f0a81b77640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a7c102490 0x7f0a7c06b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.888+0000 7f0a81b77640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a7c107dd0 0x7f0a7c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.888+0000 7f0a81b77640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a7c108720 0x7f0a7c198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.888+0000 7f0a81b77640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c112370 con 0x7f0a7c108720 2024-07-31T15:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.888+0000 7f0a81b77640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1121f0 con 0x7f0a7c107dd0 2024-07-31T15:16:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.888+0000 7f0a81b77640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1124f0 con 0x7f0a7c102490 2024-07-31T15:16:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.889+0000 7f0a7affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a7c107dd0 0x7f0a7c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.889+0000 7f0a7affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a7c107dd0 0x7f0a7c198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.67:50154/0 (socket says 172.21.15.67:50154) 2024-07-31T15:16:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.889+0000 7f0a7affd640 1 -- 172.21.15.67:0/2500809657 learned_addr learned my addr 172.21.15.67:0/2500809657 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:16:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.889+0000 7f0a7b7fe640 1 --2- 172.21.15.67:0/2500809657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a7c102490 0x7f0a7c06b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.889+0000 7f0a7affd640 1 -- 172.21.15.67:0/2500809657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a7c102490 msgr2=0x7f0a7c06b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.889+0000 7f0a7affd640 1 --2- 172.21.15.67:0/2500809657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a7c102490 0x7f0a7c06b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.889+0000 7f0a7affd640 1 -- 172.21.15.67:0/2500809657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a7c108720 msgr2=0x7f0a7c198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.889+0000 7f0a7affd640 1 --2- 172.21.15.67:0/2500809657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a7c108720 0x7f0a7c198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.889+0000 7f0a7affd640 1 -- 172.21.15.67:0/2500809657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c199740 con 0x7f0a7c107dd0 2024-07-31T15:16:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.890+0000 7f0a7affd640 1 --2- 172.21.15.67:0/2500809657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a7c107dd0 0x7f0a7c198ab0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0a70006010 tx=0x7f0a70004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.894+0000 7f0a78ff9640 1 -- 172.21.15.67:0/2500809657 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7003f030 con 0x7f0a7c107dd0 2024-07-31T15:16:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.894+0000 7f0a78ff9640 1 -- 172.21.15.67:0/2500809657 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a70003aa0 con 0x7f0a7c107dd0 2024-07-31T15:16:15.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.894+0000 7f0a81b77640 1 -- 172.21.15.67:0/2500809657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7c19db00 con 0x7f0a7c107dd0 2024-07-31T15:16:15.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.894+0000 7f0a81b77640 1 -- 172.21.15.67:0/2500809657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7c19dfe0 con 0x7f0a7c107dd0 2024-07-31T15:16:15.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.894+0000 7f0a78ff9640 1 -- 172.21.15.67:0/2500809657 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70035070 con 0x7f0a7c107dd0 2024-07-31T15:16:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.899+0000 7f0a627fc640 1 -- 172.21.15.67:0/2500809657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a7c1028a0 con 0x7f0a7c107dd0 2024-07-31T15:16:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.899+0000 7f0a78ff9640 1 -- 172.21.15.67:0/2500809657 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0a70042030 con 0x7f0a7c107dd0 2024-07-31T15:16:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.900+0000 7f0a78ff9640 1 --2- 172.21.15.67:0/2500809657 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0a640b3f10 0x7f0a640b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.901+0000 7f0a78ff9640 1 -- 172.21.15.67:0/2500809657 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9483+0+0 (secure 0 0 0) 0x7f0a701047a0 con 0x7f0a7c107dd0 2024-07-31T15:16:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.901+0000 7f0a7b7fe640 1 --2- 172.21.15.67:0/2500809657 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0a640b3f10 0x7f0a640b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.902+0000 7f0a7b7fe640 1 --2- 172.21.15.67:0/2500809657 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0a640b3f10 0x7f0a640b6420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c19eb70 tx=0x7f0a6c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:15.905+0000 7f0a78ff9640 1 -- 172.21.15.67:0/2500809657 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f0a700cb690 con 0x7f0a7c107dd0 2024-07-31T15:16:16.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:16.248+0000 7f0a627fc640 1 -- 172.21.15.67:0/2500809657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a7c1af020 con 0x7f0a7c107dd0 2024-07-31T15:16:16.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:16.250+0000 7f0a78ff9640 1 -- 172.21.15.67:0/2500809657 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11959 (secure 0 0 0) 0x7f0a700caf90 con 0x7f0a7c107dd0 2024-07-31T15:16:16.252 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:16:16.253 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":66,"btime":"2024-07-31T15:10:30:235518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/952957104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":952957104},{"type":"v1","addr":"172.21.15.67:6835","nonce":952957104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/505336439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":505336439},{"type":"v1","addr":"172.21.15.67:6837","nonce":505336439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-31T14:47:04.833142+0000","modified":"2024-07-31T15:10:30.235494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24671,"mds_1":24655,"mds_2":24650,"mds_3":24646,"mds_4":24634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14778":{"gid":14778,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/3532250223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3532250223},{"type":"v1","addr":"172.21.15.67:6839","nonce":3532250223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14790":{"gid":14790,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/1797875282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":1797875282},{"type":"v1","addr":"172.21.15.67:6841","nonce":1797875282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1184537619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1184537619},{"type":"v1","addr":"172.21.15.73:6833","nonce":1184537619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2710994774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2710994774},{"type":"v1","addr":"172.21.15.77:6833","nonce":2710994774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24634":{"gid":24634,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.77:6835/1866134372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1866134372},{"type":"v1","addr":"172.21.15.77:6835","nonce":1866134372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24646":{"gid":24646,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.77:6837/3460575274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3460575274},{"type":"v1","addr":"172.21.15.77:6837","nonce":3460575274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24650":{"gid":24650,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.73:6837/3592232039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3592232039},{"type":"v1","addr":"172.21.15.73:6837","nonce":3592232039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.77:6839/2802806156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2802806156},{"type":"v1","addr":"172.21.15.77:6839","nonce":2802806156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24671":{"gid":24671,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/4131046421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":4131046421},{"type":"v1","addr":"172.21.15.73:6839","nonce":4131046421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29129":{"gid":29129,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/631186801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":631186801},{"type":"v1","addr":"172.21.15.73:6835","nonce":631186801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24671,"qdb_cluster":[24671,24655,24650,24646,24634]},"id":1}]} 2024-07-31T15:16:16.253 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 66 2024-07-31T15:16:16.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:16.253+0000 7f0a627fc640 1 -- 172.21.15.67:0/2500809657 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0a640b3f10 msgr2=0x7f0a640b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:16.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:16.253+0000 7f0a627fc640 1 --2- 172.21.15.67:0/2500809657 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0a640b3f10 0x7f0a640b6420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c19eb70 tx=0x7f0a6c005f90 comp rx=0 tx=0).stop 2024-07-31T15:16:16.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:16.253+0000 7f0a627fc640 1 -- 172.21.15.67:0/2500809657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a7c107dd0 msgr2=0x7f0a7c198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:16.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:16.253+0000 7f0a627fc640 1 --2- 172.21.15.67:0/2500809657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a7c107dd0 0x7f0a7c198ab0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0a70006010 tx=0x7f0a70004060 comp rx=0 tx=0).stop 2024-07-31T15:16:16.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:16.253+0000 7f0a627fc640 1 -- 172.21.15.67:0/2500809657 shutdown_connections 2024-07-31T15:16:16.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:16.253+0000 7f0a627fc640 1 --2- 172.21.15.67:0/2500809657 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f0a640b3f10 0x7f0a640b6420 secure :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c19eb70 tx=0x7f0a6c005f90 comp rx=0 tx=0).stop 2024-07-31T15:16:16.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:16.253+0000 7f0a627fc640 1 --2- 172.21.15.67:0/2500809657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a7c108720 0x7f0a7c198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:16.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:16.253+0000 7f0a627fc640 1 --2- 172.21.15.67:0/2500809657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a7c107dd0 0x7f0a7c198ab0 secure :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0a70006010 tx=0x7f0a70004060 comp rx=0 tx=0).stop 2024-07-31T15:16:16.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:16.253+0000 7f0a627fc640 1 --2- 172.21.15.67:0/2500809657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a7c102490 0x7f0a7c06b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:16.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:16.253+0000 7f0a627fc640 1 -- 172.21.15.67:0/2500809657 >> 172.21.15.67:0/2500809657 conn(0x7f0a7c079160 msgr2=0x7f0a7c103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:16.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:16.254+0000 7f0a627fc640 1 -- 172.21.15.67:0/2500809657 shutdown_connections 2024-07-31T15:16:16.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:16.254+0000 7f0a627fc640 1 -- 172.21.15.67:0/2500809657 wait complete. 2024-07-31T15:16:17.256 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-07-31T15:16:17.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:17 smithi077 ceph-mon[37327]: pgmap v1205: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-07-31T15:16:17.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:17 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2500809657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:16:17.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:17 smithi067 ceph-mon[32750]: pgmap v1205: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-07-31T15:16:17.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:17 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2500809657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:16:17.597 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:16:17.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:17 smithi073 ceph-mon[36714]: pgmap v1205: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-07-31T15:16:17.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:17 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2500809657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-31T15:16:19.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.316+0000 7ff7613a0640 1 -- 172.21.15.67:0/1474961805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff75c109650 msgr2=0x7ff75c10fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:19.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.316+0000 7ff7613a0640 1 --2- 172.21.15.67:0/1474961805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff75c109650 0x7ff75c10fe80 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7ff754009ca0 tx=0x7ff75402a8c0 comp rx=0 tx=0).stop 2024-07-31T15:16:19.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.316+0000 7ff7613a0640 1 -- 172.21.15.67:0/1474961805 shutdown_connections 2024-07-31T15:16:19.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.316+0000 7ff7613a0640 1 --2- 172.21.15.67:0/1474961805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff75c109650 0x7ff75c10fe80 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.316+0000 7ff7613a0640 1 --2- 172.21.15.67:0/1474961805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff75c108d00 0x7ff75c109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.316+0000 7ff7613a0640 1 --2- 172.21.15.67:0/1474961805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c1033c0 0x7ff75c1037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.316+0000 7ff7613a0640 1 -- 172.21.15.67:0/1474961805 >> 172.21.15.67:0/1474961805 conn(0x7ff75c0fee10 msgr2=0x7ff75c101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.316+0000 7ff7613a0640 1 -- 172.21.15.67:0/1474961805 shutdown_connections 2024-07-31T15:16:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.316+0000 7ff7613a0640 1 -- 172.21.15.67:0/1474961805 wait complete. 2024-07-31T15:16:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.317+0000 7ff7613a0640 1 Processor -- start 2024-07-31T15:16:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.317+0000 7ff7613a0640 1 -- start start 2024-07-31T15:16:19.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff7613a0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff75c1033c0 0x7ff75c199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff7613a0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c108d00 0x7ff75c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff7613a0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff75c109650 0x7ff75c19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff7613a0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c113320 con 0x7ff75c109650 2024-07-31T15:16:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff7613a0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c1131a0 con 0x7ff75c1033c0 2024-07-31T15:16:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff7613a0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c1134a0 con 0x7ff75c108d00 2024-07-31T15:16:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff760b9f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff75c109650 0x7ff75c19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff760b9f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff75c109650 0x7ff75c19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37620/0 (socket says 172.21.15.67:37620) 2024-07-31T15:16:19.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff760b9f640 1 -- 172.21.15.67:0/1198967911 learned_addr learned my addr 172.21.15.67:0/1198967911 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:16:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff75bfff640 1 --2- 172.21.15.67:0/1198967911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff75c1033c0 0x7ff75c199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff75b7fe640 1 --2- 172.21.15.67:0/1198967911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c108d00 0x7ff75c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff760b9f640 1 -- 172.21.15.67:0/1198967911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c108d00 msgr2=0x7ff75c19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff760b9f640 1 --2- 172.21.15.67:0/1198967911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c108d00 0x7ff75c19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff760b9f640 1 -- 172.21.15.67:0/1198967911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff75c1033c0 msgr2=0x7ff75c199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff760b9f640 1 --2- 172.21.15.67:0/1198967911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff75c1033c0 0x7ff75c199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff760b9f640 1 -- 172.21.15.67:0/1198967911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c19fc20 con 0x7ff75c109650 2024-07-31T15:16:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff760b9f640 1 --2- 172.21.15.67:0/1198967911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff75c109650 0x7ff75c19f4f0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7ff754005cc0 tx=0x7ff75402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:19.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.318+0000 7ff7597fa640 1 -- 172.21.15.67:0/1198967911 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754003cf0 con 0x7ff75c109650 2024-07-31T15:16:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.319+0000 7ff7597fa640 1 -- 172.21.15.67:0/1198967911 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7540044f0 con 0x7ff75c109650 2024-07-31T15:16:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.319+0000 7ff7597fa640 1 -- 172.21.15.67:0/1198967911 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75402fd10 con 0x7ff75c109650 2024-07-31T15:16:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.319+0000 7ff7613a0640 1 -- 172.21.15.67:0/1198967911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75c19feb0 con 0x7ff75c109650 2024-07-31T15:16:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.319+0000 7ff7613a0640 1 -- 172.21.15.67:0/1198967911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff75c1a6670 con 0x7ff75c109650 2024-07-31T15:16:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.320+0000 7ff7613a0640 1 -- 172.21.15.67:0/1198967911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff75c104640 con 0x7ff75c109650 2024-07-31T15:16:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.322+0000 7ff7597fa640 1 -- 172.21.15.67:0/1198967911 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff754004040 con 0x7ff75c109650 2024-07-31T15:16:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.323+0000 7ff7597fa640 1 --2- 172.21.15.67:0/1198967911 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff7440b3dc0 0x7ff7440b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:19.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.323+0000 7ff75bfff640 1 --2- 172.21.15.67:0/1198967911 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff7440b3dc0 0x7ff7440b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:19.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.323+0000 7ff7597fa640 1 -- 172.21.15.67:0/1198967911 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9483+0+0 (secure 0 0 0) 0x7ff754105b40 con 0x7ff75c109650 2024-07-31T15:16:19.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.323+0000 7ff75bfff640 1 --2- 172.21.15.67:0/1198967911 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff7440b3dc0 0x7ff7440b62d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0058c0 tx=0x7ff74c00a250 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:19.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.326+0000 7ff7597fa640 1 -- 172.21.15.67:0/1198967911 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff7540ccba0 con 0x7ff75c109650 2024-07-31T15:16:19.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:19 smithi077 ceph-mon[37327]: pgmap v1206: 129 pgs: 9 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 88 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-07-31T15:16:19.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.519+0000 7ff7613a0640 1 -- 172.21.15.67:0/1198967911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff75c10b440 con 0x7ff75c109650 2024-07-31T15:16:19.521 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:19 smithi067 ceph-mon[32750]: pgmap v1206: 129 pgs: 9 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 88 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-07-31T15:16:19.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.521+0000 7ff7597fa640 1 -- 172.21.15.67:0/1198967911 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+25199 (secure 0 0 0) 0x7ff7540d3e20 con 0x7ff75c109650 2024-07-31T15:16:19.523 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-31T15:16:19.523 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":139,"fsid":"5ca2b71e-4f4a-11ef-bcc9-c7b262605968","created":"2024-07-31T14:38:19.925705+0000","modified":"2024-07-31T15:16:11.188935+0000","last_up_change":"2024-07-31T14:46:04.152899+0000","last_in_change":"2024-07-31T14:45:45.126448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-31T14:42:27.359626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-31T14:46:58.459751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-31T14:47:01.783728+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":44,"snap_epoch":139,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-31T14:47:11.080224+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":44,"snap_epoch":139,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1469d842-7dd3-4a23-969f-b98d853dbc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6803","nonce":2442400504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6805","nonce":2442400504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6809","nonce":2442400504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2442400504},{"type":"v1","addr":"172.21.15.67:6807","nonce":2442400504}]},"public_addr":"172.21.15.67:6803/2442400504","cluster_addr":"172.21.15.67:6805/2442400504","heartbeat_back_addr":"172.21.15.67:6809/2442400504","heartbeat_front_addr":"172.21.15.67:6807/2442400504","state":["exists","up"]},{"osd":1,"uuid":"329506f6-e103-4178-bcf7-10913f292047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6801","nonce":1103939318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6803","nonce":1103939318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6807","nonce":1103939318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1103939318},{"type":"v1","addr":"172.21.15.73:6805","nonce":1103939318}]},"public_addr":"172.21.15.73:6801/1103939318","cluster_addr":"172.21.15.73:6803/1103939318","heartbeat_back_addr":"172.21.15.73:6807/1103939318","heartbeat_front_addr":"172.21.15.73:6805/1103939318","state":["exists","up"]},{"osd":2,"uuid":"3855f3c9-0985-4dcd-a1f6-97dd54d1d833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6801","nonce":1624991750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6803","nonce":1624991750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6807","nonce":1624991750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1624991750},{"type":"v1","addr":"172.21.15.77:6805","nonce":1624991750}]},"public_addr":"172.21.15.77:6801/1624991750","cluster_addr":"172.21.15.77:6803/1624991750","heartbeat_back_addr":"172.21.15.77:6807/1624991750","heartbeat_front_addr":"172.21.15.77:6805/1624991750","state":["exists","up"]},{"osd":3,"uuid":"c1693ca0-afc8-4cbd-abef-251f8f4c42f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6811","nonce":3100079409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6813","nonce":3100079409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6817","nonce":3100079409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3100079409},{"type":"v1","addr":"172.21.15.67:6815","nonce":3100079409}]},"public_addr":"172.21.15.67:6811/3100079409","cluster_addr":"172.21.15.67:6813/3100079409","heartbeat_back_addr":"172.21.15.67:6817/3100079409","heartbeat_front_addr":"172.21.15.67:6815/3100079409","state":["exists","up"]},{"osd":4,"uuid":"17975063-5f3d-4a52-84f4-6d54ce4fce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6809","nonce":3387924579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6811","nonce":3387924579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6815","nonce":3387924579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3387924579},{"type":"v1","addr":"172.21.15.73:6813","nonce":3387924579}]},"public_addr":"172.21.15.73:6809/3387924579","cluster_addr":"172.21.15.73:6811/3387924579","heartbeat_back_addr":"172.21.15.73:6815/3387924579","heartbeat_front_addr":"172.21.15.73:6813/3387924579","state":["exists","up"]},{"osd":5,"uuid":"e41e714e-62fa-4946-923b-56f8ee66fb86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6809","nonce":683909212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6811","nonce":683909212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6815","nonce":683909212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":683909212},{"type":"v1","addr":"172.21.15.77:6813","nonce":683909212}]},"public_addr":"172.21.15.77:6809/683909212","cluster_addr":"172.21.15.77:6811/683909212","heartbeat_back_addr":"172.21.15.77:6815/683909212","heartbeat_front_addr":"172.21.15.77:6813/683909212","state":["exists","up"]},{"osd":6,"uuid":"b20c9a1c-1462-4596-915a-70c38b3b8c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6819","nonce":380463157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6821","nonce":380463157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6825","nonce":380463157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":380463157},{"type":"v1","addr":"172.21.15.67:6823","nonce":380463157}]},"public_addr":"172.21.15.67:6819/380463157","cluster_addr":"172.21.15.67:6821/380463157","heartbeat_back_addr":"172.21.15.67:6825/380463157","heartbeat_front_addr":"172.21.15.67:6823/380463157","state":["exists","up"]},{"osd":7,"uuid":"8cb608dc-dd03-44a4-8e6f-1fce67757c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6817","nonce":2240196566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6819","nonce":2240196566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6823","nonce":2240196566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2240196566},{"type":"v1","addr":"172.21.15.73:6821","nonce":2240196566}]},"public_addr":"172.21.15.73:6817/2240196566","cluster_addr":"172.21.15.73:6819/2240196566","heartbeat_back_addr":"172.21.15.73:6823/2240196566","heartbeat_front_addr":"172.21.15.73:6821/2240196566","state":["exists","up"]},{"osd":8,"uuid":"2b2baf2b-fe02-4048-8adb-de4d19f545d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6817","nonce":2208692754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6819","nonce":2208692754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6823","nonce":2208692754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2208692754},{"type":"v1","addr":"172.21.15.77:6821","nonce":2208692754}]},"public_addr":"172.21.15.77:6817/2208692754","cluster_addr":"172.21.15.77:6819/2208692754","heartbeat_back_addr":"172.21.15.77:6823/2208692754","heartbeat_front_addr":"172.21.15.77:6821/2208692754","state":["exists","up"]},{"osd":9,"uuid":"246aa076-dae2-448a-a294-2b0ca5ac3368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6827","nonce":3242854060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6829","nonce":3242854060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6833","nonce":3242854060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3242854060},{"type":"v1","addr":"172.21.15.67:6831","nonce":3242854060}]},"public_addr":"172.21.15.67:6827/3242854060","cluster_addr":"172.21.15.67:6829/3242854060","heartbeat_back_addr":"172.21.15.67:6833/3242854060","heartbeat_front_addr":"172.21.15.67:6831/3242854060","state":["exists","up"]},{"osd":10,"uuid":"65a6d993-7acb-467d-abf4-92d5e443438e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6825","nonce":1494741997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6827","nonce":1494741997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6831","nonce":1494741997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1494741997},{"type":"v1","addr":"172.21.15.73:6829","nonce":1494741997}]},"public_addr":"172.21.15.73:6825/1494741997","cluster_addr":"172.21.15.73:6827/1494741997","heartbeat_back_addr":"172.21.15.73:6831/1494741997","heartbeat_front_addr":"172.21.15.73:6829/1494741997","state":["exists","up"]},{"osd":11,"uuid":"709fb536-23a5-4c14-aed6-97fad837c210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6825","nonce":764558961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6827","nonce":764558961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6831","nonce":764558961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":764558961},{"type":"v1","addr":"172.21.15.77:6829","nonce":764558961}]},"public_addr":"172.21.15.77:6825/764558961","cluster_addr":"172.21.15.77:6827/764558961","heartbeat_back_addr":"172.21.15.77:6831/764558961","heartbeat_front_addr":"172.21.15.77:6829/764558961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:41:39.210679+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:01.613139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:23.947250+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:42:47.532310+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:11.480093+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:43:35.405523+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:00.227544+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:24.605451+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:44:49.069469+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:13.502256+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:45:38.217030+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-31T14:46:02.584786+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6801/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/3601135096":"2024-08-01T14:40:19.281779+0000","172.21.15.67:6800/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/927641174":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/3901988655":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6800/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/1243394894":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/1563471445":"2024-08-01T14:39:07.942187+0000","172.21.15.67:6801/1712042445":"2024-08-01T14:38:41.486084+0000","172.21.15.67:0/76992902":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3065587341":"2024-08-01T14:38:41.486084+0000","172.21.15.67:6801/4249598263":"2024-08-01T14:39:07.942187+0000","172.21.15.67:0/3411853619":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/618685030":"2024-08-01T14:40:19.281779+0000","172.21.15.67:0/2614585786":"2024-08-01T14:40:19.281779+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":31,"length":1},{"begin":44,"length":1}]},{"pool":4,"snaps":[{"begin":31,"length":1},{"begin":44,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":31,"length":1},{"begin":44,"length":1}]},{"pool":4,"snaps":[{"begin":31,"length":1},{"begin":44,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-31T15:16:19.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.523+0000 7ff7613a0640 1 -- 172.21.15.67:0/1198967911 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff7440b3dc0 msgr2=0x7ff7440b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:19.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.523+0000 7ff7613a0640 1 --2- 172.21.15.67:0/1198967911 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff7440b3dc0 0x7ff7440b62d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0058c0 tx=0x7ff74c00a250 comp rx=0 tx=0).stop 2024-07-31T15:16:19.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.523+0000 7ff7613a0640 1 -- 172.21.15.67:0/1198967911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff75c109650 msgr2=0x7ff75c19f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:19.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.523+0000 7ff7613a0640 1 --2- 172.21.15.67:0/1198967911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff75c109650 0x7ff75c19f4f0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7ff754005cc0 tx=0x7ff75402f470 comp rx=0 tx=0).stop 2024-07-31T15:16:19.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.523+0000 7ff7613a0640 1 -- 172.21.15.67:0/1198967911 shutdown_connections 2024-07-31T15:16:19.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.523+0000 7ff7613a0640 1 --2- 172.21.15.67:0/1198967911 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff7440b3dc0 0x7ff7440b62d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:19.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.523+0000 7ff7613a0640 1 --2- 172.21.15.67:0/1198967911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff75c109650 0x7ff75c19f4f0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:19.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.523+0000 7ff7613a0640 1 --2- 172.21.15.67:0/1198967911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c108d00 0x7ff75c19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:19.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.523+0000 7ff7613a0640 1 --2- 172.21.15.67:0/1198967911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff75c1033c0 0x7ff75c199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:19.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.523+0000 7ff7613a0640 1 -- 172.21.15.67:0/1198967911 >> 172.21.15.67:0/1198967911 conn(0x7ff75c0fee10 msgr2=0x7ff75c101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:19.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.523+0000 7ff7613a0640 1 -- 172.21.15.67:0/1198967911 shutdown_connections 2024-07-31T15:16:19.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:19.523+0000 7ff7613a0640 1 -- 172.21.15.67:0/1198967911 wait complete. 2024-07-31T15:16:19.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:19 smithi073 ceph-mon[36714]: pgmap v1206: 129 pgs: 9 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 88 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-07-31T15:16:20.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs fail cephfs 2024-07-31T15:16:20.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:20 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/1198967911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T15:16:20.458 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:16:20.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:20 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/1198967911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T15:16:20.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:20 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/1198967911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-31T15:16:21.303 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:21 smithi067 ceph-mon[32750]: pgmap v1207: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-07-31T15:16:21.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:21 smithi077 ceph-mon[37327]: pgmap v1207: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-07-31T15:16:21.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:21 smithi073 ceph-mon[36714]: pgmap v1207: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-07-31T15:16:22.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.491+0000 7f3341a4c640 1 -- 172.21.15.67:0/3562283219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c106660 msgr2=0x7f333c1100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:22.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.491+0000 7f3341a4c640 1 --2- 172.21.15.67:0/3562283219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c106660 0x7f333c1100b0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f3334009ca0 tx=0x7f333402a8c0 comp rx=0 tx=0).stop 2024-07-31T15:16:22.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.491+0000 7f3341a4c640 1 -- 172.21.15.67:0/3562283219 shutdown_connections 2024-07-31T15:16:22.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.491+0000 7f3341a4c640 1 --2- 172.21.15.67:0/3562283219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c106660 0x7f333c1100b0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:22.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.491+0000 7f3341a4c640 1 --2- 172.21.15.67:0/3562283219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333c105d10 0x7f333c106120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:22.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.491+0000 7f3341a4c640 1 --2- 172.21.15.67:0/3562283219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c101a30 0x7f333c101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:22.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.491+0000 7f3341a4c640 1 -- 172.21.15.67:0/3562283219 >> 172.21.15.67:0/3562283219 conn(0x7f333c079870 msgr2=0x7f333c079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:22.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.491+0000 7f3341a4c640 1 -- 172.21.15.67:0/3562283219 shutdown_connections 2024-07-31T15:16:22.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.492+0000 7f3341a4c640 1 -- 172.21.15.67:0/3562283219 wait complete. 2024-07-31T15:16:22.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.492+0000 7f3341a4c640 1 Processor -- start 2024-07-31T15:16:22.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.492+0000 7f3341a4c640 1 -- start start 2024-07-31T15:16:22.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f3341a4c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c101a30 0x7f333c197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:22.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f3341a4c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333c105d10 0x7f333c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:22.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f3341a4c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c106660 0x7f333c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:22.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f3341a4c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f333c111320 con 0x7f333c106660 2024-07-31T15:16:22.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f3341a4c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f333c1111a0 con 0x7f333c105d10 2024-07-31T15:16:22.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f3341a4c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f333c1114a0 con 0x7f333c101a30 2024-07-31T15:16:22.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f333b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c106660 0x7f333c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:22.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f333b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c106660 0x7f333c19d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50294/0 (socket says 172.21.15.67:50294) 2024-07-31T15:16:22.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f333b7fe640 1 -- 172.21.15.67:0/157093914 learned_addr learned my addr 172.21.15.67:0/157093914 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:16:22.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f333a7fc640 1 --2- 172.21.15.67:0/157093914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333c105d10 0x7f333c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:22.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f333affd640 1 --2- 172.21.15.67:0/157093914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c101a30 0x7f333c197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:22.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f333b7fe640 1 -- 172.21.15.67:0/157093914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c101a30 msgr2=0x7f333c197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:22.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f333b7fe640 1 --2- 172.21.15.67:0/157093914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c101a30 0x7f333c197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:22.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f333b7fe640 1 -- 172.21.15.67:0/157093914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333c105d10 msgr2=0x7f333c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:22.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f333b7fe640 1 --2- 172.21.15.67:0/157093914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333c105d10 0x7f333c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:22.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f333b7fe640 1 -- 172.21.15.67:0/157093914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c19db70 con 0x7f333c106660 2024-07-31T15:16:22.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.493+0000 7f333b7fe640 1 --2- 172.21.15.67:0/157093914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c106660 0x7f333c19d440 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f33340098d0 tx=0x7f33340317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:22.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.494+0000 7f3340a4a640 1 -- 172.21.15.67:0/157093914 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334003b70 con 0x7f333c106660 2024-07-31T15:16:22.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.494+0000 7f3340a4a640 1 -- 172.21.15.67:0/157093914 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3334003d10 con 0x7f333c106660 2024-07-31T15:16:22.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.494+0000 7f3341a4c640 1 -- 172.21.15.67:0/157093914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333c19de00 con 0x7f333c106660 2024-07-31T15:16:22.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.494+0000 7f3340a4a640 1 -- 172.21.15.67:0/157093914 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333402fa20 con 0x7f333c106660 2024-07-31T15:16:22.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.494+0000 7f3341a4c640 1 -- 172.21.15.67:0/157093914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f333c1a45c0 con 0x7f333c106660 2024-07-31T15:16:22.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.495+0000 7f3341a4c640 1 -- 172.21.15.67:0/157093914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333c101e40 con 0x7f333c106660 2024-07-31T15:16:22.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.496+0000 7f3340a4a640 1 -- 172.21.15.67:0/157093914 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f333402fbe0 con 0x7f333c106660 2024-07-31T15:16:22.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.497+0000 7f3340a4a640 1 --2- 172.21.15.67:0/157093914 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33280b3d40 0x7f33280b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:22.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.497+0000 7f333affd640 1 --2- 172.21.15.67:0/157093914 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33280b3d40 0x7f33280b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:22.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.497+0000 7f3340a4a640 1 -- 172.21.15.67:0/157093914 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9483+0+0 (secure 0 0 0) 0x7f33341058b0 con 0x7f333c106660 2024-07-31T15:16:22.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.498+0000 7f333affd640 1 --2- 172.21.15.67:0/157093914 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33280b3d40 0x7f33280b6250 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f332400ff50 tx=0x7f3324005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:22.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.501+0000 7f3340a4a640 1 -- 172.21.15.67:0/157093914 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f33340cc910 con 0x7f333c106660 2024-07-31T15:16:22.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.735+0000 7f3341a4c640 1 -- 172.21.15.67:0/157093914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f333c198d30 con 0x7f333c106660 2024-07-31T15:16:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.738+0000 7f3340a4a640 1 -- 172.21.15.67:0/157093914 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) ==== 161+0+0 (secure 0 0 0) 0x7f333c198d30 con 0x7f333c106660 2024-07-31T15:16:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-31T15:16:22.743 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a[32746]: 2024-07-31T15:16:22.736+0000 7f9e61b25640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-31T15:16:22.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.741+0000 7f3341a4c640 1 -- 172.21.15.67:0/157093914 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33280b3d40 msgr2=0x7f33280b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:22.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.741+0000 7f3341a4c640 1 --2- 172.21.15.67:0/157093914 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33280b3d40 0x7f33280b6250 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f332400ff50 tx=0x7f3324005f90 comp rx=0 tx=0).stop 2024-07-31T15:16:22.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.741+0000 7f3341a4c640 1 -- 172.21.15.67:0/157093914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c106660 msgr2=0x7f333c19d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:22.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.741+0000 7f3341a4c640 1 --2- 172.21.15.67:0/157093914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c106660 0x7f333c19d440 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f33340098d0 tx=0x7f33340317b0 comp rx=0 tx=0).stop 2024-07-31T15:16:22.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.742+0000 7f3341a4c640 1 -- 172.21.15.67:0/157093914 shutdown_connections 2024-07-31T15:16:22.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.742+0000 7f3341a4c640 1 --2- 172.21.15.67:0/157093914 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f33280b3d40 0x7f33280b6250 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:22.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.742+0000 7f3341a4c640 1 --2- 172.21.15.67:0/157093914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c106660 0x7f333c19d440 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:22.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.742+0000 7f3341a4c640 1 --2- 172.21.15.67:0/157093914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333c105d10 0x7f333c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:22.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.742+0000 7f3341a4c640 1 --2- 172.21.15.67:0/157093914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c101a30 0x7f333c197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:22.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.742+0000 7f3341a4c640 1 -- 172.21.15.67:0/157093914 >> 172.21.15.67:0/157093914 conn(0x7f333c079870 msgr2=0x7f333c106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:22.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.744+0000 7f3341a4c640 1 -- 172.21.15.67:0/157093914 shutdown_connections 2024-07-31T15:16:22.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:22.744+0000 7f3341a4c640 1 -- 172.21.15.67:0/157093914 wait complete. 2024-07-31T15:16:22.845 INFO:journalctl@ceph.mds.l.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-l[68994]: ignoring --setuser ceph since I am not root 2024-07-31T15:16:22.845 INFO:journalctl@ceph.mds.l.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-l[68994]: ignoring --setgroup ceph since I am not root 2024-07-31T15:16:22.846 INFO:journalctl@ceph.mds.l.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-mds[68998]: -- 172.21.15.77:0/1572710269 <== mon.1 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eda3191860 con 0x55eda7333180 2024-07-31T15:16:22.846 INFO:journalctl@ceph.mds.l.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-l[68994]: starting mds.l at 2024-07-31T15:16:22.846 INFO:journalctl@ceph.mds.i.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-i[69903]: ignoring --setuser ceph since I am not root 2024-07-31T15:16:22.846 INFO:journalctl@ceph.mds.i.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-i[69903]: ignoring --setgroup ceph since I am not root 2024-07-31T15:16:22.846 INFO:journalctl@ceph.mds.i.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-mds[69907]: -- 172.21.15.77:0/1108107899 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55efb0d61860 con 0x55efb4f03a80 2024-07-31T15:16:22.846 INFO:journalctl@ceph.mds.i.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-i[69903]: starting mds.i at 2024-07-31T15:16:23.054 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 15:16:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-a[80778]: ignoring --setuser ceph since I am not root 2024-07-31T15:16:23.054 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 15:16:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-a[80778]: ignoring --setgroup ceph since I am not root 2024-07-31T15:16:23.054 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 15:16:22 smithi067 ceph-mds[80782]: -- 172.21.15.67:0/1341065443 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b53b5ff860 con 0x55b53f7a1600 2024-07-31T15:16:23.054 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 15:16:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-a[80778]: starting mds.a at 2024-07-31T15:16:23.055 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 31 15:16:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-g[80345]: ignoring --setuser ceph since I am not root 2024-07-31T15:16:23.055 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 31 15:16:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-g[80345]: ignoring --setgroup ceph since I am not root 2024-07-31T15:16:23.055 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 31 15:16:22 smithi067 ceph-mds[80362]: -- 172.21.15.67:0/1603231035 <== mon.1 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cac0a35860 con 0x55cac4bd7600 2024-07-31T15:16:23.055 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 31 15:16:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-g[80345]: starting mds.g at 2024-07-31T15:16:23.142 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: ignoring --setuser ceph since I am not root 2024-07-31T15:16:23.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: ignoring --setgroup ceph since I am not root 2024-07-31T15:16:23.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-mds[67894]: -- 172.21.15.73:0/960439828 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a33b8e1860 con 0x55a33fa83180 2024-07-31T15:16:23.143 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: starting mds.b at 2024-07-31T15:16:23.143 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[70341]: ignoring --setuser ceph since I am not root 2024-07-31T15:16:23.144 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[70341]: ignoring --setgroup ceph since I am not root 2024-07-31T15:16:23.144 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-mds[70345]: -- 172.21.15.73:0/3357321559 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cd7bc19860 con 0x55cd7fdbb600 2024-07-31T15:16:23.144 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[70341]: starting mds.e at 2024-07-31T15:16:23.144 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-k[66966]: ignoring --setuser ceph since I am not root 2024-07-31T15:16:23.144 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-k[66966]: ignoring --setgroup ceph since I am not root 2024-07-31T15:16:23.144 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-mds[66970]: -- 172.21.15.73:0/3633863492 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5588700c3860 con 0x558874265a80 2024-07-31T15:16:23.145 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-k[66966]: starting mds.k at 2024-07-31T15:16:23.145 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-k[66966]: 2024-07-31T15:16:22.859+0000 7f0b73b30640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6836/0: (98) Address already in use 2024-07-31T15:16:23.145 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-h[67585]: ignoring --setuser ceph since I am not root 2024-07-31T15:16:23.145 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-h[67585]: ignoring --setgroup ceph since I am not root 2024-07-31T15:16:23.145 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-mds[67589]: -- 172.21.15.73:0/1362578489 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ea70f79860 con 0x55ea7511b600 2024-07-31T15:16:23.145 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 15:16:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-h[67585]: starting mds.h at 2024-07-31T15:16:23.236 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-c[69602]: ignoring --setuser ceph since I am not root 2024-07-31T15:16:23.236 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-c[69602]: ignoring --setgroup ceph since I am not root 2024-07-31T15:16:23.237 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-mds[69606]: -- 172.21.15.77:0/3408268933 <== mon.1 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558fea27d860 con 0x558fee41f180 2024-07-31T15:16:23.237 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-c[69602]: starting mds.c at 2024-07-31T15:16:23.237 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-f[69298]: ignoring --setuser ceph since I am not root 2024-07-31T15:16:23.237 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-f[69298]: ignoring --setgroup ceph since I am not root 2024-07-31T15:16:23.237 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-mds[69302]: -- 172.21.15.77:0/1288672345 <== mon.1 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55edfa755860 con 0x55edfe8f7180 2024-07-31T15:16:23.237 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 15:16:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-f[69298]: starting mds.f at 2024-07-31T15:16:23.426 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:23 smithi067 ceph-mon[32750]: pgmap v1208: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-07-31T15:16:23.426 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:23 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/157093914' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-31T15:16:23.426 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:23 smithi067 ceph-mon[32750]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-31T15:16:23.426 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:23 smithi067 ceph-mon[32750]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-31T15:16:23.426 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:23 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/157093914' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-31T15:16:23.427 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:23 smithi067 ceph-mon[32750]: osdmap e140: 12 total, 12 up, 12 in 2024-07-31T15:16:23.427 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:23 smithi067 ceph-mon[32750]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-07-31T15:16:23.457 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-07-31T15:16:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:23 smithi077 ceph-mon[37327]: pgmap v1208: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-07-31T15:16:23.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:23 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/157093914' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-31T15:16:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:23 smithi077 ceph-mon[37327]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-31T15:16:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:23 smithi077 ceph-mon[37327]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-31T15:16:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:23 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/157093914' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-31T15:16:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:23 smithi077 ceph-mon[37327]: osdmap e140: 12 total, 12 up, 12 in 2024-07-31T15:16:23.488 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:23 smithi077 ceph-mon[37327]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-07-31T15:16:23.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:23 smithi073 ceph-mon[36714]: pgmap v1208: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-07-31T15:16:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:23 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/157093914' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-31T15:16:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:23 smithi073 ceph-mon[36714]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-31T15:16:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:23 smithi073 ceph-mon[36714]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-31T15:16:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:23 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/157093914' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-31T15:16:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:23 smithi073 ceph-mon[36714]: osdmap e140: 12 total, 12 up, 12 in 2024-07-31T15:16:23.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:23 smithi073 ceph-mon[36714]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-07-31T15:16:23.727 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:16:24.935 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.77:6832/665565093,v1:172.21.15.77:6834/665565093] up:boot 2024-07-31T15:16:24.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.77:6833/2181782693,v1:172.21.15.77:6835/2181782693] up:boot 2024-07-31T15:16:24.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.73:6832/899451907,v1:172.21.15.73:6833/899451907] up:boot 2024-07-31T15:16:24.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.73:6834/2375992631,v1:172.21.15.73:6835/2375992631] up:boot 2024-07-31T15:16:24.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.77:6836/385523247,v1:172.21.15.77:6837/385523247] up:boot 2024-07-31T15:16:24.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.77:6838/2487466309,v1:172.21.15.77:6839/2487466309] up:boot 2024-07-31T15:16:24.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.73:6837/2901999270,v1:172.21.15.73:6839/2901999270] up:boot 2024-07-31T15:16:24.937 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.73:6836/2219875946,v1:172.21.15.73:6838/2219875946] up:boot 2024-07-31T15:16:24.937 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.67:6838/2524962059,v1:172.21.15.67:6840/2524962059] up:boot 2024-07-31T15:16:24.937 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: mds.? [v2:172.21.15.67:6839/1553209756,v1:172.21.15.67:6841/1553209756] up:boot 2024-07-31T15:16:24.937 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-07-31T15:16:24.937 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-31T15:16:24.937 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-31T15:16:24.937 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-31T15:16:24.938 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-31T15:16:24.938 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T15:16:24.938 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-31T15:16:24.938 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-31T15:16:24.938 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-31T15:16:24.938 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-31T15:16:24.938 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-31T15:16:24.938 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:24 smithi067 ceph-mon[32750]: pgmap v1210: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-07-31T15:16:25.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.77:6832/665565093,v1:172.21.15.77:6834/665565093] up:boot 2024-07-31T15:16:25.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.77:6833/2181782693,v1:172.21.15.77:6835/2181782693] up:boot 2024-07-31T15:16:25.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.73:6832/899451907,v1:172.21.15.73:6833/899451907] up:boot 2024-07-31T15:16:25.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.73:6834/2375992631,v1:172.21.15.73:6835/2375992631] up:boot 2024-07-31T15:16:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.77:6836/385523247,v1:172.21.15.77:6837/385523247] up:boot 2024-07-31T15:16:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.77:6838/2487466309,v1:172.21.15.77:6839/2487466309] up:boot 2024-07-31T15:16:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.73:6837/2901999270,v1:172.21.15.73:6839/2901999270] up:boot 2024-07-31T15:16:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.73:6836/2219875946,v1:172.21.15.73:6838/2219875946] up:boot 2024-07-31T15:16:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.67:6838/2524962059,v1:172.21.15.67:6840/2524962059] up:boot 2024-07-31T15:16:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: mds.? [v2:172.21.15.67:6839/1553209756,v1:172.21.15.67:6841/1553209756] up:boot 2024-07-31T15:16:25.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-07-31T15:16:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-31T15:16:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-31T15:16:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-31T15:16:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-31T15:16:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T15:16:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-31T15:16:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-31T15:16:25.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-31T15:16:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-31T15:16:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-31T15:16:25.146 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:24 smithi073 ceph-mon[36714]: pgmap v1210: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-07-31T15:16:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.77:6832/665565093,v1:172.21.15.77:6834/665565093] up:boot 2024-07-31T15:16:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.77:6833/2181782693,v1:172.21.15.77:6835/2181782693] up:boot 2024-07-31T15:16:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.73:6832/899451907,v1:172.21.15.73:6833/899451907] up:boot 2024-07-31T15:16:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.73:6834/2375992631,v1:172.21.15.73:6835/2375992631] up:boot 2024-07-31T15:16:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.77:6836/385523247,v1:172.21.15.77:6837/385523247] up:boot 2024-07-31T15:16:25.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.77:6838/2487466309,v1:172.21.15.77:6839/2487466309] up:boot 2024-07-31T15:16:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.73:6837/2901999270,v1:172.21.15.73:6839/2901999270] up:boot 2024-07-31T15:16:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.73:6836/2219875946,v1:172.21.15.73:6838/2219875946] up:boot 2024-07-31T15:16:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.67:6838/2524962059,v1:172.21.15.67:6840/2524962059] up:boot 2024-07-31T15:16:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: mds.? [v2:172.21.15.67:6839/1553209756,v1:172.21.15.67:6841/1553209756] up:boot 2024-07-31T15:16:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-07-31T15:16:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-31T15:16:25.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-31T15:16:25.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-31T15:16:25.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-31T15:16:25.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-31T15:16:25.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-31T15:16:25.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-31T15:16:25.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-31T15:16:25.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-31T15:16:25.240 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-31T15:16:25.241 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:24 smithi077 ceph-mon[37327]: pgmap v1210: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-07-31T15:16:25.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.497+0000 7ff38e07e640 1 -- 172.21.15.67:0/414054329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff388108760 msgr2=0x7ff38810ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:25.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.497+0000 7ff38e07e640 1 --2- 172.21.15.67:0/414054329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff388108760 0x7ff38810ef90 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff380009ca0 tx=0x7ff38002a8c0 comp rx=0 tx=0).stop 2024-07-31T15:16:25.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.497+0000 7ff38e07e640 1 -- 172.21.15.67:0/414054329 shutdown_connections 2024-07-31T15:16:25.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.497+0000 7ff38e07e640 1 --2- 172.21.15.67:0/414054329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff388108760 0x7ff38810ef90 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.497+0000 7ff38e07e640 1 --2- 172.21.15.67:0/414054329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff388107e10 0x7ff388108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.497+0000 7ff38e07e640 1 --2- 172.21.15.67:0/414054329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3881024d0 0x7ff3881028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.497+0000 7ff38e07e640 1 -- 172.21.15.67:0/414054329 >> 172.21.15.67:0/414054329 conn(0x7ff388079190 msgr2=0x7ff3880795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.497+0000 7ff38e07e640 1 -- 172.21.15.67:0/414054329 shutdown_connections 2024-07-31T15:16:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.497+0000 7ff38e07e640 1 -- 172.21.15.67:0/414054329 wait complete. 2024-07-31T15:16:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.498+0000 7ff38e07e640 1 Processor -- start 2024-07-31T15:16:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.498+0000 7ff38e07e640 1 -- start start 2024-07-31T15:16:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.498+0000 7ff38e07e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3881024d0 0x7ff3880ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.498+0000 7ff38e07e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff388107e10 0x7ff3880ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.498+0000 7ff38e07e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff388108760 0x7ff3880ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.498+0000 7ff38e07e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff3881123c0 con 0x7ff388108760 2024-07-31T15:16:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.498+0000 7ff38e07e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff388112240 con 0x7ff3881024d0 2024-07-31T15:16:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.498+0000 7ff38e07e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff388112540 con 0x7ff388107e10 2024-07-31T15:16:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.498+0000 7ff387fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff388108760 0x7ff3880ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.498+0000 7ff387fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff388108760 0x7ff3880ffb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50338/0 (socket says 172.21.15.67:50338) 2024-07-31T15:16:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.498+0000 7ff387fff640 1 -- 172.21.15.67:0/3591037303 learned_addr learned my addr 172.21.15.67:0/3591037303 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:16:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.498+0000 7ff3877fe640 1 --2- 172.21.15.67:0/3591037303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3881024d0 0x7ff3880ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.499+0000 7ff386ffd640 1 --2- 172.21.15.67:0/3591037303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff388107e10 0x7ff3880ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:25.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.499+0000 7ff387fff640 1 -- 172.21.15.67:0/3591037303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff388107e10 msgr2=0x7ff3880ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:25.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.499+0000 7ff387fff640 1 --2- 172.21.15.67:0/3591037303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff388107e10 0x7ff3880ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:25.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.499+0000 7ff387fff640 1 -- 172.21.15.67:0/3591037303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3881024d0 msgr2=0x7ff3880ff0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:25.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.499+0000 7ff387fff640 1 --2- 172.21.15.67:0/3591037303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3881024d0 0x7ff3880ff0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:25.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.499+0000 7ff387fff640 1 -- 172.21.15.67:0/3591037303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3881a8130 con 0x7ff388108760 2024-07-31T15:16:25.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.499+0000 7ff387fff640 1 --2- 172.21.15.67:0/3591037303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff388108760 0x7ff3880ffb20 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff380005cc0 tx=0x7ff38002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:25.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.499+0000 7ff384ff9640 1 -- 172.21.15.67:0/3591037303 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380003bf0 con 0x7ff388108760 2024-07-31T15:16:25.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.499+0000 7ff38e07e640 1 -- 172.21.15.67:0/3591037303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3881a83c0 con 0x7ff388108760 2024-07-31T15:16:25.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.499+0000 7ff384ff9640 1 -- 172.21.15.67:0/3591037303 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff380003d90 con 0x7ff388108760 2024-07-31T15:16:25.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.499+0000 7ff384ff9640 1 -- 172.21.15.67:0/3591037303 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38002fb20 con 0x7ff388108760 2024-07-31T15:16:25.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.499+0000 7ff38e07e640 1 -- 172.21.15.67:0/3591037303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3881a8900 con 0x7ff388108760 2024-07-31T15:16:25.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.501+0000 7ff38e07e640 1 -- 172.21.15.67:0/3591037303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff388103750 con 0x7ff388108760 2024-07-31T15:16:25.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.503+0000 7ff384ff9640 1 -- 172.21.15.67:0/3591037303 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff380004580 con 0x7ff388108760 2024-07-31T15:16:25.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.504+0000 7ff384ff9640 1 --2- 172.21.15.67:0/3591037303 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff3700b3d40 0x7ff3700b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:25.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.504+0000 7ff3877fe640 1 --2- 172.21.15.67:0/3591037303 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff3700b3d40 0x7ff3700b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:25.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.504+0000 7ff384ff9640 1 -- 172.21.15.67:0/3591037303 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 9825+0+0 (secure 0 0 0) 0x7ff380105a70 con 0x7ff388108760 2024-07-31T15:16:25.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.505+0000 7ff3877fe640 1 --2- 172.21.15.67:0/3591037303 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff3700b3d40 0x7ff3700b6250 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff388100af0 tx=0x7ff37800a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:25.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.507+0000 7ff384ff9640 1 -- 172.21.15.67:0/3591037303 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff3800cc970 con 0x7ff388108760 2024-07-31T15:16:25.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:25.752+0000 7ff38e07e640 1 -- 172.21.15.67:0/3591037303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff3881003a0 con 0x7ff388108760 2024-07-31T15:16:26.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:26.363+0000 7ff384ff9640 1 -- 172.21.15.67:0/3591037303 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) ==== 104+0+0 (secure 0 0 0) 0x7ff3881003a0 con 0x7ff388108760 2024-07-31T15:16:26.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:26.365+0000 7ff38e07e640 1 -- 172.21.15.67:0/3591037303 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff3700b3d40 msgr2=0x7ff3700b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:26.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:26.365+0000 7ff38e07e640 1 --2- 172.21.15.67:0/3591037303 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff3700b3d40 0x7ff3700b6250 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff388100af0 tx=0x7ff37800a2d0 comp rx=0 tx=0).stop 2024-07-31T15:16:26.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:26.365+0000 7ff38e07e640 1 -- 172.21.15.67:0/3591037303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff388108760 msgr2=0x7ff3880ffb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:26.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:26.366+0000 7ff38e07e640 1 --2- 172.21.15.67:0/3591037303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff388108760 0x7ff3880ffb20 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff380005cc0 tx=0x7ff38002f380 comp rx=0 tx=0).stop 2024-07-31T15:16:26.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:26.366+0000 7ff38e07e640 1 -- 172.21.15.67:0/3591037303 shutdown_connections 2024-07-31T15:16:26.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:26.366+0000 7ff38e07e640 1 --2- 172.21.15.67:0/3591037303 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7ff3700b3d40 0x7ff3700b6250 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:26.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:26.366+0000 7ff38e07e640 1 --2- 172.21.15.67:0/3591037303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff388108760 0x7ff3880ffb20 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:26.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:26.366+0000 7ff38e07e640 1 --2- 172.21.15.67:0/3591037303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff388107e10 0x7ff3880ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:26.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:26.366+0000 7ff38e07e640 1 --2- 172.21.15.67:0/3591037303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3881024d0 0x7ff3880ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:26.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:26.366+0000 7ff38e07e640 1 -- 172.21.15.67:0/3591037303 >> 172.21.15.67:0/3591037303 conn(0x7ff388079190 msgr2=0x7ff388101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:26.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:26.366+0000 7ff38e07e640 1 -- 172.21.15.67:0/3591037303 shutdown_connections 2024-07-31T15:16:26.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:26.366+0000 7ff38e07e640 1 -- 172.21.15.67:0/3591037303 wait complete. 2024-07-31T15:16:26.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:26 smithi073 ceph-mon[36714]: osdmap e141: 12 total, 12 up, 12 in 2024-07-31T15:16:26.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:26 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3591037303' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-31T15:16:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:26 smithi077 ceph-mon[37327]: osdmap e141: 12 total, 12 up, 12 in 2024-07-31T15:16:26.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:26 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3591037303' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-31T15:16:26.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:26 smithi067 ceph-mon[32750]: osdmap e141: 12 total, 12 up, 12 in 2024-07-31T15:16:26.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:26 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3591037303' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-31T15:16:27.190 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-31T15:16:27.465 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:16:27.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:27 smithi067 ceph-mon[32750]: pgmap v1212: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 127 B/s wr, 9 op/s 2024-07-31T15:16:27.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:27 smithi067 ceph-mon[32750]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-31T15:16:27.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:27 smithi067 ceph-mon[32750]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-31T15:16:27.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:27 smithi067 ceph-mon[32750]: Cluster is now healthy 2024-07-31T15:16:27.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:27 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/3591037303' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-31T15:16:27.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:27 smithi067 ceph-mon[32750]: fsmap 12 up:standby 2024-07-31T15:16:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:27 smithi073 ceph-mon[36714]: pgmap v1212: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 127 B/s wr, 9 op/s 2024-07-31T15:16:27.645 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:27 smithi073 ceph-mon[36714]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-31T15:16:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:27 smithi073 ceph-mon[36714]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-31T15:16:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:27 smithi073 ceph-mon[36714]: Cluster is now healthy 2024-07-31T15:16:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:27 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/3591037303' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-31T15:16:27.646 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:27 smithi073 ceph-mon[36714]: fsmap 12 up:standby 2024-07-31T15:16:27.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:27 smithi077 ceph-mon[37327]: pgmap v1212: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 127 B/s wr, 9 op/s 2024-07-31T15:16:27.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:27 smithi077 ceph-mon[37327]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-31T15:16:27.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:27 smithi077 ceph-mon[37327]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-31T15:16:27.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:27 smithi077 ceph-mon[37327]: Cluster is now healthy 2024-07-31T15:16:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:27 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/3591037303' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-31T15:16:27.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:27 smithi077 ceph-mon[37327]: fsmap 12 up:standby 2024-07-31T15:16:29.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.268+0000 7f1914b6a640 1 -- 172.21.15.67:0/3903296401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1910108720 msgr2=0x7f191010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:29.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.268+0000 7f1914b6a640 1 --2- 172.21.15.67:0/3903296401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1910108720 0x7f191010ef50 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f1908009ca0 tx=0x7f190802a8c0 comp rx=0 tx=0).stop 2024-07-31T15:16:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.269+0000 7f1914b6a640 1 -- 172.21.15.67:0/3903296401 shutdown_connections 2024-07-31T15:16:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.269+0000 7f1914b6a640 1 --2- 172.21.15.67:0/3903296401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1910108720 0x7f191010ef50 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.269+0000 7f1914b6a640 1 --2- 172.21.15.67:0/3903296401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1910107dd0 0x7f19101081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.269+0000 7f1914b6a640 1 --2- 172.21.15.67:0/3903296401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1910102490 0x7f19101028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.269+0000 7f1914b6a640 1 -- 172.21.15.67:0/3903296401 >> 172.21.15.67:0/3903296401 conn(0x7f1910079160 msgr2=0x7f1910079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.269+0000 7f1914b6a640 1 -- 172.21.15.67:0/3903296401 shutdown_connections 2024-07-31T15:16:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.269+0000 7f1914b6a640 1 -- 172.21.15.67:0/3903296401 wait complete. 2024-07-31T15:16:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.270+0000 7f1914b6a640 1 Processor -- start 2024-07-31T15:16:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.270+0000 7f1914b6a640 1 -- start start 2024-07-31T15:16:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.270+0000 7f1914b6a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1910102490 0x7f191019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.270+0000 7f1914b6a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1910107dd0 0x7f191019b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.270+0000 7f1914b6a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1910108720 0x7f191019b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.270+0000 7f1914b6a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f19101123a0 con 0x7f1910108720 2024-07-31T15:16:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.270+0000 7f1914b6a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1910112220 con 0x7f1910102490 2024-07-31T15:16:29.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.270+0000 7f1914b6a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1910112520 con 0x7f1910107dd0 2024-07-31T15:16:29.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.270+0000 7f190ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1910108720 0x7f191019b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:29.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.270+0000 7f190ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1910108720 0x7f191019b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50356/0 (socket says 172.21.15.67:50356) 2024-07-31T15:16:29.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.271+0000 7f190ed9d640 1 -- 172.21.15.67:0/2457291232 learned_addr learned my addr 172.21.15.67:0/2457291232 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:16:29.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.271+0000 7f190e59c640 1 --2- 172.21.15.67:0/2457291232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1910102490 0x7f191019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:29.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.271+0000 7f190dd9b640 1 --2- 172.21.15.67:0/2457291232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1910107dd0 0x7f191019b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:29.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.271+0000 7f190ed9d640 1 -- 172.21.15.67:0/2457291232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1910107dd0 msgr2=0x7f191019b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:29.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.271+0000 7f190ed9d640 1 --2- 172.21.15.67:0/2457291232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1910107dd0 0x7f191019b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:29.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.271+0000 7f190ed9d640 1 -- 172.21.15.67:0/2457291232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1910102490 msgr2=0x7f191019acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.271+0000 7f190ed9d640 1 --2- 172.21.15.67:0/2457291232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1910102490 0x7f191019acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.271+0000 7f190ed9d640 1 -- 172.21.15.67:0/2457291232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19101a5f40 con 0x7f1910108720 2024-07-31T15:16:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.271+0000 7f190ed9d640 1 --2- 172.21.15.67:0/2457291232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1910108720 0x7f191019b730 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f1908005cc0 tx=0x7f1908031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.271+0000 7f18ff7fe640 1 -- 172.21.15.67:0/2457291232 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908003b70 con 0x7f1910108720 2024-07-31T15:16:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.272+0000 7f1914b6a640 1 -- 172.21.15.67:0/2457291232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1910198aa0 con 0x7f1910108720 2024-07-31T15:16:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.272+0000 7f18ff7fe640 1 -- 172.21.15.67:0/2457291232 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19080318f0 con 0x7f1910108720 2024-07-31T15:16:29.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.272+0000 7f18ff7fe640 1 -- 172.21.15.67:0/2457291232 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190802f950 con 0x7f1910108720 2024-07-31T15:16:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.272+0000 7f1914b6a640 1 -- 172.21.15.67:0/2457291232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1910199030 con 0x7f1910108720 2024-07-31T15:16:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.273+0000 7f1914b6a640 1 -- 172.21.15.67:0/2457291232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19101028a0 con 0x7f1910108720 2024-07-31T15:16:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.275+0000 7f18ff7fe640 1 -- 172.21.15.67:0/2457291232 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f190802fb10 con 0x7f1910108720 2024-07-31T15:16:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.276+0000 7f18ff7fe640 1 --2- 172.21.15.67:0/2457291232 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18f80b3df0 0x7f18f80b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.276+0000 7f190e59c640 1 --2- 172.21.15.67:0/2457291232 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18f80b3df0 0x7f18f80b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.276+0000 7f18ff7fe640 1 -- 172.21.15.67:0/2457291232 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 9825+0+0 (secure 0 0 0) 0x7f1908105790 con 0x7f1910108720 2024-07-31T15:16:29.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.276+0000 7f190e59c640 1 --2- 172.21.15.67:0/2457291232 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18f80b3df0 0x7f18f80b6300 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f191006b3f0 tx=0x7f1900012040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:29.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.281+0000 7f18ff7fe640 1 -- 172.21.15.67:0/2457291232 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f19080cc630 con 0x7f1910108720 2024-07-31T15:16:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:29.481+0000 7f1914b6a640 1 -- 172.21.15.67:0/2457291232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f191002e0c0 con 0x7f1910108720 2024-07-31T15:16:29.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:29 smithi067 ceph-mon[32750]: pgmap v1213: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-31T15:16:29.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:29 smithi073 ceph-mon[36714]: pgmap v1213: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-31T15:16:29.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:29 smithi077 ceph-mon[37327]: pgmap v1213: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-31T15:16:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:30.373+0000 7f18ff7fe640 1 -- 172.21.15.67:0/2457291232 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v142) ==== 181+0+0 (secure 0 0 0) 0x7f19080cbf30 con 0x7f1910108720 2024-07-31T15:16:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-07-31T15:16:30.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:30.376+0000 7f1914b6a640 1 -- 172.21.15.67:0/2457291232 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18f80b3df0 msgr2=0x7f18f80b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:30.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:30.376+0000 7f1914b6a640 1 --2- 172.21.15.67:0/2457291232 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18f80b3df0 0x7f18f80b6300 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f191006b3f0 tx=0x7f1900012040 comp rx=0 tx=0).stop 2024-07-31T15:16:30.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:30.376+0000 7f1914b6a640 1 -- 172.21.15.67:0/2457291232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1910108720 msgr2=0x7f191019b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:30.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:30.376+0000 7f1914b6a640 1 --2- 172.21.15.67:0/2457291232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1910108720 0x7f191019b730 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f1908005cc0 tx=0x7f1908031700 comp rx=0 tx=0).stop 2024-07-31T15:16:30.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:30.376+0000 7f1914b6a640 1 -- 172.21.15.67:0/2457291232 shutdown_connections 2024-07-31T15:16:30.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:30.376+0000 7f1914b6a640 1 --2- 172.21.15.67:0/2457291232 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f18f80b3df0 0x7f18f80b6300 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:30.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:30.376+0000 7f1914b6a640 1 --2- 172.21.15.67:0/2457291232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1910108720 0x7f191019b730 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:30.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:30.376+0000 7f1914b6a640 1 --2- 172.21.15.67:0/2457291232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1910107dd0 0x7f191019b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:30.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:30.376+0000 7f1914b6a640 1 --2- 172.21.15.67:0/2457291232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1910102490 0x7f191019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:30.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:30.376+0000 7f1914b6a640 1 -- 172.21.15.67:0/2457291232 >> 172.21.15.67:0/2457291232 conn(0x7f1910079160 msgr2=0x7f1910104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:30.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:30.376+0000 7f1914b6a640 1 -- 172.21.15.67:0/2457291232 shutdown_connections 2024-07-31T15:16:30.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:30.376+0000 7f1914b6a640 1 -- 172.21.15.67:0/2457291232 wait complete. 2024-07-31T15:16:30.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:30 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2457291232' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-31T15:16:30.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:30 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2457291232' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-31T15:16:30.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:30 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2457291232' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-31T15:16:31.352 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-31T15:16:31.632 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:16:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:31 smithi073 ceph-mon[36714]: pgmap v1214: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s 2024-07-31T15:16:31.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:31 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2457291232' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-31T15:16:31.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:31 smithi073 ceph-mon[36714]: osdmap e142: 12 total, 12 up, 12 in 2024-07-31T15:16:31.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:31 smithi077 ceph-mon[37327]: pgmap v1214: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s 2024-07-31T15:16:31.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:31 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2457291232' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-31T15:16:31.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:31 smithi077 ceph-mon[37327]: osdmap e142: 12 total, 12 up, 12 in 2024-07-31T15:16:31.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:31 smithi067 ceph-mon[32750]: pgmap v1214: 129 pgs: 129 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s 2024-07-31T15:16:31.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:31 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2457291232' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-31T15:16:31.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:31 smithi067 ceph-mon[32750]: osdmap e142: 12 total, 12 up, 12 in 2024-07-31T15:16:33.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.475+0000 7fdcab981640 1 -- 172.21.15.67:0/576804143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca41035e0 msgr2=0x7fdca41039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.475+0000 7fdcab981640 1 --2- 172.21.15.67:0/576804143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca41035e0 0x7fdca41039f0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009900 tx=0x7fdca002a410 comp rx=0 tx=0).stop 2024-07-31T15:16:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.475+0000 7fdcab981640 1 -- 172.21.15.67:0/576804143 shutdown_connections 2024-07-31T15:16:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.475+0000 7fdcab981640 1 --2- 172.21.15.67:0/576804143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdca4109870 0x7fdca41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.476+0000 7fdcab981640 1 --2- 172.21.15.67:0/576804143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca4108f20 0x7fdca4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.476+0000 7fdcab981640 1 --2- 172.21.15.67:0/576804143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca41035e0 0x7fdca41039f0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.476+0000 7fdcab981640 1 -- 172.21.15.67:0/576804143 >> 172.21.15.67:0/576804143 conn(0x7fdca40ff030 msgr2=0x7fdca4101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.476+0000 7fdcab981640 1 -- 172.21.15.67:0/576804143 shutdown_connections 2024-07-31T15:16:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.476+0000 7fdcab981640 1 -- 172.21.15.67:0/576804143 wait complete. 2024-07-31T15:16:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.476+0000 7fdcab981640 1 Processor -- start 2024-07-31T15:16:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.476+0000 7fdcab981640 1 -- start start 2024-07-31T15:16:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdcab981640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdca41035e0 0x7fdca4075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdcab981640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca4108f20 0x7fdca4075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdcab981640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca4109870 0x7fdca4075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdcab981640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdca4113540 con 0x7fdca4109870 2024-07-31T15:16:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdcab981640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41133c0 con 0x7fdca41035e0 2024-07-31T15:16:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdcab981640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41136c0 con 0x7fdca4108f20 2024-07-31T15:16:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdca9f1e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca4109870 0x7fdca4075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdca9f1e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca4109870 0x7fdca4075bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49492/0 (socket says 172.21.15.67:49492) 2024-07-31T15:16:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdca9f1e640 1 -- 172.21.15.67:0/4011563280 learned_addr learned my addr 172.21.15.67:0/4011563280 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:16:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdca8f1c640 1 --2- 172.21.15.67:0/4011563280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca4108f20 0x7fdca4075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdca971d640 1 --2- 172.21.15.67:0/4011563280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdca41035e0 0x7fdca4075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdca9f1e640 1 -- 172.21.15.67:0/4011563280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca4108f20 msgr2=0x7fdca4075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdca9f1e640 1 --2- 172.21.15.67:0/4011563280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca4108f20 0x7fdca4075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdca9f1e640 1 -- 172.21.15.67:0/4011563280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdca41035e0 msgr2=0x7fdca4075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdca9f1e640 1 --2- 172.21.15.67:0/4011563280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdca41035e0 0x7fdca4075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdca9f1e640 1 -- 172.21.15.67:0/4011563280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca41a81f0 con 0x7fdca4109870 2024-07-31T15:16:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdca9f1e640 1 --2- 172.21.15.67:0/4011563280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca4109870 0x7fdca4075bc0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00eaa0 tx=0x7fdc9c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdc9a7fc640 1 -- 172.21.15.67:0/4011563280 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00cdb0 con 0x7fdca4109870 2024-07-31T15:16:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdcab981640 1 -- 172.21.15.67:0/4011563280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca41a84e0 con 0x7fdca4109870 2024-07-31T15:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.477+0000 7fdc9a7fc640 1 -- 172.21.15.67:0/4011563280 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc9c005020 con 0x7fdca4109870 2024-07-31T15:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.478+0000 7fdc9a7fc640 1 -- 172.21.15.67:0/4011563280 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0116d0 con 0x7fdca4109870 2024-07-31T15:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.478+0000 7fdcab981640 1 -- 172.21.15.67:0/4011563280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca41a8a20 con 0x7fdca4109870 2024-07-31T15:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.478+0000 7fdcab981640 1 -- 172.21.15.67:0/4011563280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca41039f0 con 0x7fdca4109870 2024-07-31T15:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.481+0000 7fdc9a7fc640 1 -- 172.21.15.67:0/4011563280 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fdc9c0051d0 con 0x7fdca4109870 2024-07-31T15:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.482+0000 7fdc9a7fc640 1 --2- 172.21.15.67:0/4011563280 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdc900b3d40 0x7fdc900b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.482+0000 7fdca971d640 1 --2- 172.21.15.67:0/4011563280 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdc900b3d40 0x7fdc900b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.482+0000 7fdc9a7fc640 1 -- 172.21.15.67:0/4011563280 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9241+0+0 (secure 0 0 0) 0x7fdc9c0e4890 con 0x7fdca4109870 2024-07-31T15:16:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.482+0000 7fdca971d640 1 --2- 172.21.15.67:0/4011563280 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdc900b3d40 0x7fdc900b6250 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdca0000c00 tx=0x7fdca0031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.485+0000 7fdc9a7fc640 1 -- 172.21.15.67:0/4011563280 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fdc9c016030 con 0x7fdca4109870 2024-07-31T15:16:33.642 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:33 smithi073 ceph-mon[36714]: pgmap v1216: 97 pgs: 97 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T15:16:33.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:33.683+0000 7fdcab981640 1 -- 172.21.15.67:0/4011563280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdca402e0c0 con 0x7fdca4109870 2024-07-31T15:16:33.684 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:33 smithi067 ceph-mon[32750]: pgmap v1216: 97 pgs: 97 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T15:16:33.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:33 smithi077 ceph-mon[37327]: pgmap v1216: 97 pgs: 97 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T15:16:34.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:34.384+0000 7fdc9a7fc640 1 -- 172.21.15.67:0/4011563280 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v143) ==== 169+0+0 (secure 0 0 0) 0x7fdc9c0ab8d0 con 0x7fdca4109870 2024-07-31T15:16:34.386 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-07-31T15:16:34.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:34.386+0000 7fdcab981640 1 -- 172.21.15.67:0/4011563280 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdc900b3d40 msgr2=0x7fdc900b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:34.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:34.386+0000 7fdcab981640 1 --2- 172.21.15.67:0/4011563280 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdc900b3d40 0x7fdc900b6250 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdca0000c00 tx=0x7fdca0031400 comp rx=0 tx=0).stop 2024-07-31T15:16:34.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:34.387+0000 7fdcab981640 1 -- 172.21.15.67:0/4011563280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca4109870 msgr2=0x7fdca4075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:34.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:34.387+0000 7fdcab981640 1 --2- 172.21.15.67:0/4011563280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca4109870 0x7fdca4075bc0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00eaa0 tx=0x7fdc9c00ef60 comp rx=0 tx=0).stop 2024-07-31T15:16:34.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:34.387+0000 7fdcab981640 1 -- 172.21.15.67:0/4011563280 shutdown_connections 2024-07-31T15:16:34.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:34.387+0000 7fdcab981640 1 --2- 172.21.15.67:0/4011563280 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7fdc900b3d40 0x7fdc900b6250 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:34.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:34.387+0000 7fdcab981640 1 --2- 172.21.15.67:0/4011563280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca4109870 0x7fdca4075bc0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:34.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:34.387+0000 7fdcab981640 1 --2- 172.21.15.67:0/4011563280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdca4108f20 0x7fdca4075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:34.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:34.387+0000 7fdcab981640 1 --2- 172.21.15.67:0/4011563280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdca41035e0 0x7fdca4075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:34.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:34.387+0000 7fdcab981640 1 -- 172.21.15.67:0/4011563280 >> 172.21.15.67:0/4011563280 conn(0x7fdca40ff030 msgr2=0x7fdca4101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:34.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:34.387+0000 7fdcab981640 1 -- 172.21.15.67:0/4011563280 shutdown_connections 2024-07-31T15:16:34.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:34.387+0000 7fdcab981640 1 -- 172.21.15.67:0/4011563280 wait complete. 2024-07-31T15:16:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:34 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/4011563280' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-31T15:16:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:16:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:34.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:34 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:34.716 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:34 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/4011563280' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-31T15:16:34.717 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:16:34.717 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:34.717 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:34.717 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:34.717 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:34 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:34.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:34 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/4011563280' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-31T15:16:34.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-31T15:16:34.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:34.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:34.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:34.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:34 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:35.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804 shell --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-07-31T15:16:35.328 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/mon.a/config 2024-07-31T15:16:35.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:35 smithi073 ceph-mon[36714]: pgmap v1217: 97 pgs: 97 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T15:16:35.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:35 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi067 to 176.0M 2024-07-31T15:16:35.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:35 smithi073 ceph-mon[36714]: Unable to set osd_memory_target on smithi067 to 184601804: error parsing value: Value '184601804' is below minimum 939524096 2024-07-31T15:16:35.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:35 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/4011563280' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-31T15:16:35.643 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:35 smithi073 ceph-mon[36714]: osdmap e143: 12 total, 12 up, 12 in 2024-07-31T15:16:35.652 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:35 smithi067 ceph-mon[32750]: pgmap v1217: 97 pgs: 97 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T15:16:35.652 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:35 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi067 to 176.0M 2024-07-31T15:16:35.652 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:35 smithi067 ceph-mon[32750]: Unable to set osd_memory_target on smithi067 to 184601804: error parsing value: Value '184601804' is below minimum 939524096 2024-07-31T15:16:35.652 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:35 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/4011563280' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-31T15:16:35.652 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:35 smithi067 ceph-mon[32750]: osdmap e143: 12 total, 12 up, 12 in 2024-07-31T15:16:35.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:35 smithi077 ceph-mon[37327]: pgmap v1217: 97 pgs: 97 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T15:16:35.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:35 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi067 to 176.0M 2024-07-31T15:16:35.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:35 smithi077 ceph-mon[37327]: Unable to set osd_memory_target on smithi067 to 184601804: error parsing value: Value '184601804' is below minimum 939524096 2024-07-31T15:16:35.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:35 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/4011563280' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-31T15:16:35.738 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:35 smithi077 ceph-mon[37327]: osdmap e143: 12 total, 12 up, 12 in 2024-07-31T15:16:37.585 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:37 smithi073 ceph-mon[36714]: pgmap v1219: 33 pgs: 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T15:16:37.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.630+0000 7f3e92a8b640 1 -- 172.21.15.67:0/3393164491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e8c1043e0 msgr2=0x7f3e8c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:37.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.630+0000 7f3e92a8b640 1 --2- 172.21.15.67:0/3393164491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e8c1043e0 0x7f3e8c1047f0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009900 tx=0x7f3e7c02a410 comp rx=0 tx=0).stop 2024-07-31T15:16:37.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.630+0000 7f3e92a8b640 1 -- 172.21.15.67:0/3393164491 shutdown_connections 2024-07-31T15:16:37.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.630+0000 7f3e92a8b640 1 --2- 172.21.15.67:0/3393164491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e8c10d840 0x7f3e8c10fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:37.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.630+0000 7f3e92a8b640 1 --2- 172.21.15.67:0/3393164491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e8c104d30 0x7f3e8c10d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:37.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.630+0000 7f3e92a8b640 1 --2- 172.21.15.67:0/3393164491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e8c1043e0 0x7f3e8c1047f0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:37.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.631+0000 7f3e92a8b640 1 -- 172.21.15.67:0/3393164491 >> 172.21.15.67:0/3393164491 conn(0x7f3e8c079160 msgr2=0x7f3e8c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:37.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.631+0000 7f3e92a8b640 1 -- 172.21.15.67:0/3393164491 shutdown_connections 2024-07-31T15:16:37.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.631+0000 7f3e92a8b640 1 -- 172.21.15.67:0/3393164491 wait complete. 2024-07-31T15:16:37.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.631+0000 7f3e92a8b640 1 Processor -- start 2024-07-31T15:16:37.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.631+0000 7f3e92a8b640 1 -- start start 2024-07-31T15:16:37.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e92a8b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e8c1043e0 0x7f3e8c102120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:37.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e92a8b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e8c104d30 0x7f3e8c102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:37.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e92a8b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e8c10d840 0x7f3e8c102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:37.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e92a8b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1134b0 con 0x7f3e8c104d30 2024-07-31T15:16:37.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e92a8b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c113330 con 0x7f3e8c1043e0 2024-07-31T15:16:37.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e92a8b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c113630 con 0x7f3e8c10d840 2024-07-31T15:16:37.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e8bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e8c104d30 0x7f3e8c102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:37.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e8bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e8c104d30 0x7f3e8c102660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49508/0 (socket says 172.21.15.67:49508) 2024-07-31T15:16:37.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e8bfff640 1 -- 172.21.15.67:0/2515009122 learned_addr learned my addr 172.21.15.67:0/2515009122 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-31T15:16:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e90827640 1 --2- 172.21.15.67:0/2515009122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e8c1043e0 0x7f3e8c102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e91028640 1 --2- 172.21.15.67:0/2515009122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e8c10d840 0x7f3e8c102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e8bfff640 1 -- 172.21.15.67:0/2515009122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e8c10d840 msgr2=0x7f3e8c102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e8bfff640 1 --2- 172.21.15.67:0/2515009122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e8c10d840 0x7f3e8c102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e8bfff640 1 -- 172.21.15.67:0/2515009122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e8c1043e0 msgr2=0x7f3e8c102120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e8bfff640 1 --2- 172.21.15.67:0/2515009122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e8c1043e0 0x7f3e8c102120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e8bfff640 1 -- 172.21.15.67:0/2515009122 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8c1a82a0 con 0x7f3e8c104d30 2024-07-31T15:16:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e8bfff640 1 --2- 172.21.15.67:0/2515009122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e8c104d30 0x7f3e8c102660 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400e9f0 tx=0x7f3e7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:37.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.632+0000 7f3e89ffb640 1 -- 172.21.15.67:0/2515009122 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7400ccc0 con 0x7f3e8c104d30 2024-07-31T15:16:37.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.633+0000 7f3e92a8b640 1 -- 172.21.15.67:0/2515009122 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e8c1a8590 con 0x7f3e8c104d30 2024-07-31T15:16:37.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.633+0000 7f3e89ffb640 1 -- 172.21.15.67:0/2515009122 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e740108e0 con 0x7f3e8c104d30 2024-07-31T15:16:37.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.633+0000 7f3e89ffb640 1 -- 172.21.15.67:0/2515009122 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74018860 con 0x7f3e8c104d30 2024-07-31T15:16:37.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.633+0000 7f3e92a8b640 1 -- 172.21.15.67:0/2515009122 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e8c1a8b20 con 0x7f3e8c104d30 2024-07-31T15:16:37.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.634+0000 7f3e92a8b640 1 -- 172.21.15.67:0/2515009122 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e8c1047f0 con 0x7f3e8c104d30 2024-07-31T15:16:37.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.635+0000 7f3e89ffb640 1 -- 172.21.15.67:0/2515009122 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3e74010430 con 0x7f3e8c104d30 2024-07-31T15:16:37.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.636+0000 7f3e89ffb640 1 --2- 172.21.15.67:0/2515009122 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e780b3d40 0x7f3e780b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-31T15:16:37.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.636+0000 7f3e90827640 1 --2- 172.21.15.67:0/2515009122 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e780b3d40 0x7f3e780b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-31T15:16:37.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.636+0000 7f3e89ffb640 1 -- 172.21.15.67:0/2515009122 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 8711+0+0 (secure 0 0 0) 0x7f3e740e4380 con 0x7f3e8c104d30 2024-07-31T15:16:37.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.637+0000 7f3e90827640 1 --2- 172.21.15.67:0/2515009122 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e780b3d40 0x7f3e780b6250 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009230 tx=0x7f3e7c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-31T15:16:37.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.640+0000 7f3e89ffb640 1 -- 172.21.15.67:0/2515009122 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f3e740aaf80 con 0x7f3e8c104d30 2024-07-31T15:16:37.642 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:37 smithi067 ceph-mon[32750]: pgmap v1219: 33 pgs: 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T15:16:37.737 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:37 smithi077 ceph-mon[37327]: pgmap v1219: 33 pgs: 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T15:16:37.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:37.839+0000 7f3e92a8b640 1 -- 172.21.15.67:0/2515009122 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3e8c103420 con 0x7f3e8c104d30 2024-07-31T15:16:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:38.826+0000 7f3e89ffb640 1 -- 172.21.15.67:0/2515009122 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v144) ==== 178+0+0 (secure 0 0 0) 0x7f3e740af4d0 con 0x7f3e8c104d30 2024-07-31T15:16:38.828 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data_ec' removed 2024-07-31T15:16:38.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:38.829+0000 7f3e92a8b640 1 -- 172.21.15.67:0/2515009122 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e780b3d40 msgr2=0x7f3e780b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:38.829+0000 7f3e92a8b640 1 --2- 172.21.15.67:0/2515009122 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e780b3d40 0x7f3e780b6250 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009230 tx=0x7f3e7c031400 comp rx=0 tx=0).stop 2024-07-31T15:16:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:38.829+0000 7f3e92a8b640 1 -- 172.21.15.67:0/2515009122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e8c104d30 msgr2=0x7f3e8c102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-31T15:16:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:38.829+0000 7f3e92a8b640 1 --2- 172.21.15.67:0/2515009122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e8c104d30 0x7f3e8c102660 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400e9f0 tx=0x7f3e7400eeb0 comp rx=0 tx=0).stop 2024-07-31T15:16:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:38.829+0000 7f3e92a8b640 1 -- 172.21.15.67:0/2515009122 shutdown_connections 2024-07-31T15:16:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:38.829+0000 7f3e92a8b640 1 --2- 172.21.15.67:0/2515009122 >> [v2:172.21.15.67:6800/2637562983,v1:172.21.15.67:6801/2637562983] conn(0x7f3e780b3d40 0x7f3e780b6250 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:38.829+0000 7f3e92a8b640 1 --2- 172.21.15.67:0/2515009122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e8c10d840 0x7f3e8c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:38.829+0000 7f3e92a8b640 1 --2- 172.21.15.67:0/2515009122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e8c104d30 0x7f3e8c102660 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:38.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:38.829+0000 7f3e92a8b640 1 --2- 172.21.15.67:0/2515009122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e8c1043e0 0x7f3e8c102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-31T15:16:38.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:38.829+0000 7f3e92a8b640 1 -- 172.21.15.67:0/2515009122 >> 172.21.15.67:0/2515009122 conn(0x7f3e8c079160 msgr2=0x7f3e8c1002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-31T15:16:38.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:38.829+0000 7f3e92a8b640 1 -- 172.21.15.67:0/2515009122 shutdown_connections 2024-07-31T15:16:38.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-31T15:16:38.829+0000 7f3e92a8b640 1 -- 172.21.15.67:0/2515009122 wait complete. 2024-07-31T15:16:39.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2515009122' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-31T15:16:39.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:39.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:39.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:39.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:39.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: Adjusting osd_memory_target on smithi073 to 176.2M 2024-07-31T15:16:39.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: Unable to set osd_memory_target on smithi073 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-07-31T15:16:39.144 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:16:39.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:16:39.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.145 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:38 smithi073 ceph-mon[36714]: pgmap v1220: 33 pgs: 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T15:16:39.154 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.155 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.155 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2515009122' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-31T15:16:39.155 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.155 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.155 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:39.155 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:39.156 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:39.156 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:39.156 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: Adjusting osd_memory_target on smithi073 to 176.2M 2024-07-31T15:16:39.156 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: Unable to set osd_memory_target on smithi073 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-07-31T15:16:39.156 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:16:39.156 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:16:39.156 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.156 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:38 smithi067 ceph-mon[32750]: pgmap v1220: 33 pgs: 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T15:16:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2515009122' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-31T15:16:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:39.238 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-31T15:16:39.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: Adjusting osd_memory_target on smithi073 to 176.2M 2024-07-31T15:16:39.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: Unable to set osd_memory_target on smithi073 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-07-31T15:16:39.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-31T15:16:39.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-31T15:16:39.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: from='mgr.14214 172.21.15.67:0/2194757382' entity='mgr.x' 2024-07-31T15:16:39.239 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:38 smithi077 ceph-mon[37327]: pgmap v1220: 33 pgs: 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-31T15:16:39.618 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-31T15:16:39.674 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-31T15:16:39.726 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-31T15:16:39.757 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-07-31T15:16:39.757 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-31T15:16:39.782 DEBUG:teuthology.orchestra.run.smithi073:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-31T15:16:39.804 DEBUG:teuthology.orchestra.run.smithi077:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-31T15:16:39.820 INFO:tasks.cephadm:Stopping all daemons... 2024-07-31T15:16:39.821 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-07-31T15:16:39.821 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.a 2024-07-31T15:16:40.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:39 smithi073 ceph-mon[36714]: from='client.? 172.21.15.67:0/2515009122' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-07-31T15:16:40.143 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:39 smithi073 ceph-mon[36714]: osdmap e144: 12 total, 12 up, 12 in 2024-07-31T15:16:40.235 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:39 smithi067 ceph-mon[32750]: from='client.? 172.21.15.67:0/2515009122' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-07-31T15:16:40.235 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:39 smithi067 ceph-mon[32750]: osdmap e144: 12 total, 12 up, 12 in 2024-07-31T15:16:40.235 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:39 smithi067 systemd[1]: Stopping Ceph mon.a for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:16:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:39 smithi077 ceph-mon[37327]: from='client.? 172.21.15.67:0/2515009122' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-07-31T15:16:40.237 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:39 smithi077 ceph-mon[37327]: osdmap e144: 12 total, 12 up, 12 in 2024-07-31T15:16:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:40 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a[32746]: 2024-07-31T15:16:40.363+0000 7f9e67330640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:16:40.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:40 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a[32746]: 2024-07-31T15:16:40.363+0000 7f9e67330640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-07-31T15:16:40.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:40 smithi067 podman[186867]: 2024-07-31 15:16:40.455586939 +0000 UTC m=+0.244355018 container died 65e2949f2afd31939b9ba7ba8838e0ba2f9657a87212f8737651feaf22b08db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, ceph=True, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-07-31T15:16:41.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:40 smithi067 podman[186867]: 2024-07-31 15:16:40.985982891 +0000 UTC m=+0.774750959 container cleanup 65e2949f2afd31939b9ba7ba8838e0ba2f9657a87212f8737651feaf22b08db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-07-31T15:16:41.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:40 smithi067 bash[186867]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a 2024-07-31T15:16:41.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 31 15:16:41 smithi067 podman[186878]: 2024-07-31 15:16:41.271596027 +0000 UTC m=+0.811780855 container remove 65e2949f2afd31939b9ba7ba8838e0ba2f9657a87212f8737651feaf22b08db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc) 2024-07-31T15:16:41.583 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.a.service' 2024-07-31T15:16:41.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:16:41.654 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-07-31T15:16:41.654 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-07-31T15:16:41.654 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.b 2024-07-31T15:16:42.098 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:41 smithi073 systemd[1]: Stopping Ceph mon.b for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:16:42.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-b[36710]: 2024-07-31T15:16:42.096+0000 7f997fd48640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:16:42.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-b[36710]: 2024-07-31T15:16:42.096+0000 7f997fd48640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-07-31T15:16:42.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:42 smithi073 podman[79009]: 2024-07-31 15:16:42.15837382 +0000 UTC m=+0.183605850 container stop 0cd68fed9b4c8667597b83db1d7fcad537ceb13b74995b471a11f494d765b8fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-07-31T15:16:42.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:42 smithi073 podman[79009]: 2024-07-31 15:16:42.184140133 +0000 UTC m=+0.209372163 container died 0cd68fed9b4c8667597b83db1d7fcad537ceb13b74995b471a11f494d765b8fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-07-31T15:16:42.976 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:42 smithi073 podman[79009]: 2024-07-31 15:16:42.739704243 +0000 UTC m=+0.764936272 container cleanup 0cd68fed9b4c8667597b83db1d7fcad537ceb13b74995b471a11f494d765b8fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-b, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS) 2024-07-31T15:16:42.976 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:42 smithi073 bash[79009]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-b 2024-07-31T15:16:43.217 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.b.service' 2024-07-31T15:16:43.238 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:42 smithi073 podman[79021]: 2024-07-31 15:16:42.975214251 +0000 UTC m=+0.810078078 container remove 0cd68fed9b4c8667597b83db1d7fcad537ceb13b74995b471a11f494d765b8fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-b, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T15:16:43.238 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:43 smithi073 systemd[1]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.b.service: Deactivated successfully. 2024-07-31T15:16:43.238 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:43 smithi073 systemd[1]: Stopped Ceph mon.b for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T15:16:43.238 INFO:journalctl@ceph.mon.b.smithi073.stdout:Jul 31 15:16:43 smithi073 systemd[1]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.b.service: Consumed 23.458s CPU time. 2024-07-31T15:16:43.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:16:43.270 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-07-31T15:16:43.270 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-07-31T15:16:43.270 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.c 2024-07-31T15:16:43.642 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:43 smithi077 systemd[1]: Stopping Ceph mon.c for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:16:43.987 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:43 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-c[37320]: 2024-07-31T15:16:43.639+0000 7fee0474c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:16:43.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:43 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-c[37320]: 2024-07-31T15:16:43.639+0000 7fee0474c640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-07-31T15:16:43.988 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:43 smithi077 podman[79841]: 2024-07-31 15:16:43.875171736 +0000 UTC m=+0.365166504 container died 31b92f1da4cc83986f33c7864d7e23dd71ae4ca83812ca35b442f6146fd87c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-c, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-07-31T15:16:44.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:44 smithi077 podman[79841]: 2024-07-31 15:16:44.283157559 +0000 UTC m=+0.773152298 container cleanup 31b92f1da4cc83986f33c7864d7e23dd71ae4ca83812ca35b442f6146fd87c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-c, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T15:16:44.487 INFO:journalctl@ceph.mon.c.smithi077.stdout:Jul 31 15:16:44 smithi077 bash[79841]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mon-c 2024-07-31T15:16:44.709 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mon.c.service' 2024-07-31T15:16:44.757 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:16:44.757 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-07-31T15:16:44.757 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-07-31T15:16:44.757 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mgr.x 2024-07-31T15:16:45.053 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 15:16:44 smithi067 systemd[1]: Stopping Ceph mgr.x for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:16:45.419 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 15:16:45 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T15:16:45.099+0000 7f4d61c77640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:16:45.419 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 15:16:45 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x[32977]: 2024-07-31T15:16:45.099+0000 7f4d61c77640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-07-31T15:16:45.419 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 15:16:45 smithi067 podman[186979]: 2024-07-31 15:16:45.142363916 +0000 UTC m=+0.175296665 container died f86146d771dd3601cc78b3c5f3d48a880eafb75d62504e62af847b88a38882fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True) 2024-07-31T15:16:45.803 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 15:16:45 smithi067 podman[186979]: 2024-07-31 15:16:45.618232477 +0000 UTC m=+0.651165220 container cleanup f86146d771dd3601cc78b3c5f3d48a880eafb75d62504e62af847b88a38882fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8c796fca5ff5960f7542e7227bad4a784ee2c804, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716) 2024-07-31T15:16:45.803 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 31 15:16:45 smithi067 bash[186979]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-x 2024-07-31T15:16:46.042 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mgr.x.service' 2024-07-31T15:16:46.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:16:46.079 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-07-31T15:16:46.080 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-07-31T15:16:46.080 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mgr.y 2024-07-31T15:16:46.392 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Jul 31 15:16:46 smithi073 systemd[1]: Stopping Ceph mgr.y for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:16:46.734 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Jul 31 15:16:46 smithi073 podman[79133]: 2024-07-31 15:16:46.42418482 +0000 UTC m=+0.158545942 container died 023c4c6614c00fb5383f6463fb9c2011f012825c1633b63c194d87e8005174b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T15:16:47.142 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Jul 31 15:16:46 smithi073 podman[79133]: 2024-07-31 15:16:46.854315473 +0000 UTC m=+0.588676639 container cleanup 023c4c6614c00fb5383f6463fb9c2011f012825c1633b63c194d87e8005174b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-07-31T15:16:47.143 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Jul 31 15:16:46 smithi073 bash[79133]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-y 2024-07-31T15:16:48.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mgr.y.service' 2024-07-31T15:16:48.648 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:16:48.648 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-07-31T15:16:48.648 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-07-31T15:16:48.649 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mgr.z 2024-07-31T15:16:48.984 INFO:journalctl@ceph.mgr.z.smithi077.stdout:Jul 31 15:16:48 smithi077 systemd[1]: Stopping Ceph mgr.z for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:16:49.237 INFO:journalctl@ceph.mgr.z.smithi077.stdout:Jul 31 15:16:48 smithi077 podman[79961]: 2024-07-31 15:16:48.983771654 +0000 UTC m=+0.139393970 container died 550da08080994fa621f23a1f1d0d62911121b0c6b38819ddc53a1d00ca9aa235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-07-31T15:16:49.682 INFO:journalctl@ceph.mgr.z.smithi077.stdout:Jul 31 15:16:49 smithi077 podman[79961]: 2024-07-31 15:16:49.413448734 +0000 UTC m=+0.569071052 container cleanup 550da08080994fa621f23a1f1d0d62911121b0c6b38819ddc53a1d00ca9aa235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-z, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS) 2024-07-31T15:16:49.683 INFO:journalctl@ceph.mgr.z.smithi077.stdout:Jul 31 15:16:49 smithi077 bash[79961]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mgr-z 2024-07-31T15:16:49.806 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mgr.z.service' 2024-07-31T15:16:49.840 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:16:49.840 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-07-31T15:16:49.841 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-07-31T15:16:49.841 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.0 2024-07-31T15:16:50.159 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 15:16:49 smithi067 systemd[1]: Stopping Ceph osd.0 for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:16:50.553 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 15:16:50 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0[53057]: 2024-07-31T15:16:50.157+0000 7f61148c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:16:50.554 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 15:16:50 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0[53057]: 2024-07-31T15:16:50.157+0000 7f61148c1640 -1 osd.0 144 *** Got signal Terminated *** 2024-07-31T15:16:50.554 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 15:16:50 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0[53057]: 2024-07-31T15:16:50.157+0000 7f61148c1640 -1 osd.0 144 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-31T15:16:56.010 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 15:16:55 smithi067 podman[187103]: 2024-07-31 15:16:55.707213478 +0000 UTC m=+5.666118511 container died 7fb2494fd29d4eeb0e7ce61916e013040293c22db9c1d77a55342557746e4bd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-07-31T15:16:56.303 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 15:16:56 smithi067 podman[187103]: 2024-07-31 15:16:56.216966031 +0000 UTC m=+6.175871104 container cleanup 7fb2494fd29d4eeb0e7ce61916e013040293c22db9c1d77a55342557746e4bd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-07-31T15:16:56.303 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 15:16:56 smithi067 bash[187103]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0 2024-07-31T15:16:56.803 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 15:16:56 smithi067 podman[187125]: 2024-07-31 15:16:56.485498023 +0000 UTC m=+0.772681685 container remove 7fb2494fd29d4eeb0e7ce61916e013040293c22db9c1d77a55342557746e4bd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0) 2024-07-31T15:16:57.281 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 15:16:56 smithi067 podman[187214]: 2024-07-31 15:16:56.819774235 +0000 UTC m=+0.017937658 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T15:16:57.281 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 15:16:56 smithi067 podman[187214]: 2024-07-31 15:16:56.936410835 +0000 UTC m=+0.134574244 container create 2d23147a6d3526603c01070756bef20539fa022456e49791082e374be4389148 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0-deactivate, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716) 2024-07-31T15:16:57.553 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 15:16:57 smithi067 podman[187214]: 2024-07-31 15:16:57.280480981 +0000 UTC m=+0.478644393 container init 2d23147a6d3526603c01070756bef20539fa022456e49791082e374be4389148 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0-deactivate, ceph=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716) 2024-07-31T15:16:57.554 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 15:16:57 smithi067 podman[187214]: 2024-07-31 15:16:57.28446461 +0000 UTC m=+0.482628027 container start 2d23147a6d3526603c01070756bef20539fa022456e49791082e374be4389148 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=) 2024-07-31T15:16:58.053 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 15:16:57 smithi067 podman[187214]: 2024-07-31 15:16:57.907635497 +0000 UTC m=+1.105798922 container attach 2d23147a6d3526603c01070756bef20539fa022456e49791082e374be4389148 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T15:16:58.053 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 31 15:16:57 smithi067 podman[187214]: 2024-07-31 15:16:57.908130238 +0000 UTC m=+1.106293658 container died 2d23147a6d3526603c01070756bef20539fa022456e49791082e374be4389148 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T15:16:58.670 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.0.service' 2024-07-31T15:16:58.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:16:58.711 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-07-31T15:16:58.711 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-07-31T15:16:58.711 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.1 2024-07-31T15:16:59.044 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 15:16:58 smithi073 systemd[1]: Stopping Ceph osd.1 for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:16:59.392 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 15:16:59 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-1[42580]: 2024-07-31T15:16:59.041+0000 7f66df9bd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:16:59.393 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 15:16:59 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-1[42580]: 2024-07-31T15:16:59.041+0000 7f66df9bd640 -1 osd.1 144 *** Got signal Terminated *** 2024-07-31T15:16:59.393 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 15:16:59 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-1[42580]: 2024-07-31T15:16:59.041+0000 7f66df9bd640 -1 osd.1 144 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-31T15:17:04.978 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 15:17:04 smithi073 podman[79259]: 2024-07-31 15:17:04.663460244 +0000 UTC m=+5.756182661 container died 9ce8c800be4e53abd53000f87c0fecabf286b75da914e09ef6a0a7e34b483181 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T15:17:05.337 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 15:17:05 smithi073 podman[79259]: 2024-07-31 15:17:05.076452261 +0000 UTC m=+6.169174678 container cleanup 9ce8c800be4e53abd53000f87c0fecabf286b75da914e09ef6a0a7e34b483181 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-07-31T15:17:05.338 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 15:17:05 smithi073 bash[79259]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-1 2024-07-31T15:17:05.642 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 15:17:05 smithi073 podman[79275]: 2024-07-31 15:17:05.33700273 +0000 UTC m=+0.673299696 container remove 9ce8c800be4e53abd53000f87c0fecabf286b75da914e09ef6a0a7e34b483181 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-1, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T15:17:05.991 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 15:17:05 smithi073 podman[79367]: 2024-07-31 15:17:05.642611813 +0000 UTC m=+0.017269618 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T15:17:05.991 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 15:17:05 smithi073 podman[79367]: 2024-07-31 15:17:05.754269239 +0000 UTC m=+0.128927044 container create 9c4a8bbe166972a05c7642e6de1df72a288a4c4ec4fa81897abab031b54fee87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-07-31T15:17:06.392 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 15:17:05 smithi073 podman[79367]: 2024-07-31 15:17:05.990473562 +0000 UTC m=+0.365131369 container init 9c4a8bbe166972a05c7642e6de1df72a288a4c4ec4fa81897abab031b54fee87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-1-deactivate, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T15:17:06.393 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 15:17:05 smithi073 podman[79367]: 2024-07-31 15:17:05.993838273 +0000 UTC m=+0.368496077 container start 9c4a8bbe166972a05c7642e6de1df72a288a4c4ec4fa81897abab031b54fee87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T15:17:06.393 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 15:17:06 smithi073 podman[79367]: 2024-07-31 15:17:06.090618524 +0000 UTC m=+0.465276343 container attach 9c4a8bbe166972a05c7642e6de1df72a288a4c4ec4fa81897abab031b54fee87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-07-31T15:17:06.393 INFO:journalctl@ceph.osd.1.smithi073.stdout:Jul 31 15:17:06 smithi073 podman[79367]: 2024-07-31 15:17:06.205374018 +0000 UTC m=+0.580031824 container died 9c4a8bbe166972a05c7642e6de1df72a288a4c4ec4fa81897abab031b54fee87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-07-31T15:17:06.862 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.1.service' 2024-07-31T15:17:06.900 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:17:06.900 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-07-31T15:17:06.900 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-07-31T15:17:06.901 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.2 2024-07-31T15:17:06.987 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 15:17:06 smithi077 systemd[1]: Stopping Ceph osd.2 for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:17:07.487 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 15:17:07 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-2[43763]: 2024-07-31T15:17:07.210+0000 7f1b16751640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:17:07.487 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 15:17:07 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-2[43763]: 2024-07-31T15:17:07.210+0000 7f1b16751640 -1 osd.2 144 *** Got signal Terminated *** 2024-07-31T15:17:07.487 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 15:17:07 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-2[43763]: 2024-07-31T15:17:07.210+0000 7f1b16751640 -1 osd.2 144 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-31T15:17:12.987 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 15:17:12 smithi077 podman[80098]: 2024-07-31 15:17:12.677394645 +0000 UTC m=+5.578816535 container died 9de1d1ea5b348b6981e63060248c8306d867235f36002fc4c2e1044a51f79df8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T15:17:13.488 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 15:17:13 smithi077 podman[80098]: 2024-07-31 15:17:13.110692446 +0000 UTC m=+6.012114325 container cleanup 9de1d1ea5b348b6981e63060248c8306d867235f36002fc4c2e1044a51f79df8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240716) 2024-07-31T15:17:13.488 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 15:17:13 smithi077 bash[80098]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-2 2024-07-31T15:17:14.237 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 15:17:13 smithi077 podman[80114]: 2024-07-31 15:17:13.846263896 +0000 UTC m=+1.167121244 container remove 9de1d1ea5b348b6981e63060248c8306d867235f36002fc4c2e1044a51f79df8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T15:17:14.600 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 15:17:14 smithi077 podman[80202]: 2024-07-31 15:17:14.201284953 +0000 UTC m=+0.017124103 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T15:17:14.601 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 15:17:14 smithi077 podman[80202]: 2024-07-31 15:17:14.330756924 +0000 UTC m=+0.146596067 container create ffe0d5cb4007ba09799addad34dd6d67d93d5a9743dcf71d391d4224086e0e10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-2-deactivate, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-07-31T15:17:14.601 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 15:17:14 smithi077 podman[80202]: 2024-07-31 15:17:14.600078395 +0000 UTC m=+0.415917555 container init ffe0d5cb4007ba09799addad34dd6d67d93d5a9743dcf71d391d4224086e0e10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-07-31T15:17:14.987 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 15:17:14 smithi077 podman[80202]: 2024-07-31 15:17:14.603274403 +0000 UTC m=+0.419113548 container start ffe0d5cb4007ba09799addad34dd6d67d93d5a9743dcf71d391d4224086e0e10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T15:17:14.988 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 15:17:14 smithi077 podman[80202]: 2024-07-31 15:17:14.666660349 +0000 UTC m=+0.482499492 container attach ffe0d5cb4007ba09799addad34dd6d67d93d5a9743dcf71d391d4224086e0e10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716) 2024-07-31T15:17:14.988 INFO:journalctl@ceph.osd.2.smithi077.stdout:Jul 31 15:17:14 smithi077 podman[80202]: 2024-07-31 15:17:14.799033509 +0000 UTC m=+0.614872657 container died ffe0d5cb4007ba09799addad34dd6d67d93d5a9743dcf71d391d4224086e0e10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716) 2024-07-31T15:17:15.489 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.2.service' 2024-07-31T15:17:15.526 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:17:15.526 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-07-31T15:17:15.527 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-07-31T15:17:15.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.3 2024-07-31T15:17:15.859 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:15 smithi067 systemd[1]: Stopping Ceph osd.3 for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:17:16.303 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:15 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3[59816]: 2024-07-31T15:17:15.857+0000 7f05edf85640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:17:16.303 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:15 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3[59816]: 2024-07-31T15:17:15.857+0000 7f05edf85640 -1 osd.3 144 *** Got signal Terminated *** 2024-07-31T15:17:16.304 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:15 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3[59816]: 2024-07-31T15:17:15.857+0000 7f05edf85640 -1 osd.3 144 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-31T15:17:17.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:17 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:17.152+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:17.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:17 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:17.503+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:18.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:18 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:18.046+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:18.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:18 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:18.179+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:18.488 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:18 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:18.012+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:18.892 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:18 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:18.536+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:19.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:19 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:19.146+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:19.488 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:18 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:18.989+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:19.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:19 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:19.056+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:19.892 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:19 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:19.549+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:20.487 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:20 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:20.038+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:20.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:20 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:20.123+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:20.502 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:20 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:20.098+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:20.803 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:20 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3[59816]: 2024-07-31T15:17:20.501+0000 7f05ea570640 -1 osd.3 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:54.870102+0000 front 2024-07-31T15:16:54.870011+0000 (oldest deadline 2024-07-31T15:17:20.070835+0000) 2024-07-31T15:17:20.892 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:20 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:20.540+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:21.487 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:21 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:21.060+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:21.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:21 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:21.172+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:21.491 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:21 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:21.105+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:21.803 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:21 smithi067 podman[187329]: 2024-07-31 15:17:21.490004635 +0000 UTC m=+5.748967536 container died ed7383f8c7914e8a5af924d1b99f27d7e4cb36db1f0d31d1b1934e7ef54b4e85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-07-31T15:17:21.892 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:21 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:21.576+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:22.119 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:22 smithi067 podman[187329]: 2024-07-31 15:17:22.008052871 +0000 UTC m=+6.267015772 container cleanup ed7383f8c7914e8a5af924d1b99f27d7e4cb36db1f0d31d1b1934e7ef54b4e85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-07-31T15:17:22.119 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:22 smithi067 bash[187329]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3 2024-07-31T15:17:22.487 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:22.034+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:22.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:22.188+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:22.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:22 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:22.118+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:22.553 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:22 smithi067 podman[187345]: 2024-07-31 15:17:22.229090864 +0000 UTC m=+0.738767941 container remove ed7383f8c7914e8a5af924d1b99f27d7e4cb36db1f0d31d1b1934e7ef54b4e85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-07-31T15:17:22.892 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:22 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:22.549+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:22.976 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:22 smithi067 podman[187431]: 2024-07-31 15:17:22.58942131 +0000 UTC m=+0.017857242 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T15:17:22.976 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:22 smithi067 podman[187431]: 2024-07-31 15:17:22.705067524 +0000 UTC m=+0.133503487 container create 18a5631ff40dd21aa3fe61f6e04bb552e4ae787968e78049bd0f560de7a5ca3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716) 2024-07-31T15:17:23.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:23 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:23.160+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:23.303 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:22 smithi067 podman[187431]: 2024-07-31 15:17:22.97489784 +0000 UTC m=+0.403333771 container init 18a5631ff40dd21aa3fe61f6e04bb552e4ae787968e78049bd0f560de7a5ca3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716) 2024-07-31T15:17:23.304 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:22 smithi067 podman[187431]: 2024-07-31 15:17:22.979135548 +0000 UTC m=+0.407571470 container start 18a5631ff40dd21aa3fe61f6e04bb552e4ae787968e78049bd0f560de7a5ca3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2) 2024-07-31T15:17:23.304 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:23 smithi067 podman[187431]: 2024-07-31 15:17:23.058010948 +0000 UTC m=+0.486446870 container attach 18a5631ff40dd21aa3fe61f6e04bb552e4ae787968e78049bd0f560de7a5ca3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-07-31T15:17:23.304 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 31 15:17:23 smithi067 podman[187431]: 2024-07-31 15:17:23.17826636 +0000 UTC m=+0.606702281 container died 18a5631ff40dd21aa3fe61f6e04bb552e4ae787968e78049bd0f560de7a5ca3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_BRANCH=HEAD) 2024-07-31T15:17:23.487 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:23 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:23.064+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:23.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:23 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:23.142+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:23.777 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.3.service' 2024-07-31T15:17:23.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:17:23.819 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-07-31T15:17:23.819 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-07-31T15:17:23.819 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.4 2024-07-31T15:17:23.845 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:23 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:23.515+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:24.140 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:23 smithi073 systemd[1]: Stopping Ceph osd.4 for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:17:24.392 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:24 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4[49482]: 2024-07-31T15:17:24.138+0000 7f5714354640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:17:24.393 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:24 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4[49482]: 2024-07-31T15:17:24.138+0000 7f5714354640 -1 osd.4 144 *** Got signal Terminated *** 2024-07-31T15:17:24.393 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:24 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4[49482]: 2024-07-31T15:17:24.138+0000 7f5714354640 -1 osd.4 144 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-31T15:17:24.487 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:24 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:24.108+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:24.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:24 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:24.135+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:24.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:24 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:24.184+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:24.892 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:24 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:24.528+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:25.389 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:25 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:25.134+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:25.389 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:25 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:25.122+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:25.390 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:25 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:25.122+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.108240+0000 front 2024-07-31T15:17:01.108293+0000 (oldest deadline 2024-07-31T15:17:25.108516+0000) 2024-07-31T15:17:25.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:25 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:25.214+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:25.892 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:25 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:25.499+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:25.987 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:25 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:25.850+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:26.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:26 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:26.124+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:26.488 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:26 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:26.084+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:26.488 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:26 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:26.084+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.108240+0000 front 2024-07-31T15:17:01.108293+0000 (oldest deadline 2024-07-31T15:17:25.108516+0000) 2024-07-31T15:17:26.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:26 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:26.238+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:26.892 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:26 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:26.473+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:26.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:26 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:26.473+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:27.237 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:27 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:27.087+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:27.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:26 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:26.893+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:27.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:27 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:27.052+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:27.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:27 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:27.052+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.108240+0000 front 2024-07-31T15:17:01.108293+0000 (oldest deadline 2024-07-31T15:17:25.108516+0000) 2024-07-31T15:17:27.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:27 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:27.272+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:27.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:27 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:27.272+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:27.783 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:27 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:27.447+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:27.783 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:27 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:27.447+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:28.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:27 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:27.781+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:28.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:27 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:27.879+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:28.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:28 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:28.099+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:28.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:28 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:28.099+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.108240+0000 front 2024-07-31T15:17:01.108293+0000 (oldest deadline 2024-07-31T15:17:25.108516+0000) 2024-07-31T15:17:28.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:28 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:28.099+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:28.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:28 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:28.099+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:28.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:28 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:28.281+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:28.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:28 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:28.281+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:28.642 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:28 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4[49482]: 2024-07-31T15:17:28.286+0000 7f571093f640 -1 osd.4 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.931496+0000 front 2024-07-31T15:17:01.931538+0000 (oldest deadline 2024-07-31T15:17:27.732299+0000) 2024-07-31T15:17:28.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:28 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:28.418+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:28.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:28 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:28.418+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:29.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:28 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:28.754+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:29.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:28 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:28.926+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:29.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:29 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:29.149+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:29.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:29 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:29.149+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.108240+0000 front 2024-07-31T15:17:01.108293+0000 (oldest deadline 2024-07-31T15:17:25.108516+0000) 2024-07-31T15:17:29.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:29 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:29.117+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:29.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:29 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:29.117+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:29.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:29 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:29.269+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:29.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:29 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:29.269+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:29.642 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:29 smithi073 podman[79482]: 2024-07-31 15:17:29.615500805 +0000 UTC m=+5.603264151 container died 851b2ea221d29758990d1e706ae578be339e81caacdd457c49694eaa41de396c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-07-31T15:17:29.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:29 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:29.390+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:29.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:29 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:29.390+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:30.023 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:29 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:29.731+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:30.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:29 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:29.644+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:30.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:29 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:29.942+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:30.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:30 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:30.156+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:30.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:30 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:30.156+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.108240+0000 front 2024-07-31T15:17:01.108293+0000 (oldest deadline 2024-07-31T15:17:25.108516+0000) 2024-07-31T15:17:30.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:30 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:30.134+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:30.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:30 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:30.134+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:30.299 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:30 smithi073 podman[79482]: 2024-07-31 15:17:30.022502761 +0000 UTC m=+6.010266173 container cleanup 851b2ea221d29758990d1e706ae578be339e81caacdd457c49694eaa41de396c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc) 2024-07-31T15:17:30.299 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:30 smithi073 bash[79482]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4 2024-07-31T15:17:30.642 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:30 smithi073 podman[79500]: 2024-07-31 15:17:30.299006396 +0000 UTC m=+0.682442956 container remove 851b2ea221d29758990d1e706ae578be339e81caacdd457c49694eaa41de396c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-07-31T15:17:30.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:30 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:30.387+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:30.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:30 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:30.387+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:30.687 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:30 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:30.316+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:30.687 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:30 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:30.316+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:30.892 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:30 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:30.779+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:30.893 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:30 smithi073 podman[79586]: 2024-07-31 15:17:30.641506939 +0000 UTC m=+0.112613303 container create 37f09999d454e2cbadedf79919a73cc3bb9c6d8395a0502e2370407a5f4935c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=) 2024-07-31T15:17:30.893 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:30 smithi073 podman[79586]: 2024-07-31 15:17:30.546117062 +0000 UTC m=+0.017223395 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T15:17:30.893 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:30 smithi073 podman[79586]: 2024-07-31 15:17:30.885907513 +0000 UTC m=+0.357013835 container init 37f09999d454e2cbadedf79919a73cc3bb9c6d8395a0502e2370407a5f4935c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4-deactivate, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-07-31T15:17:30.893 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:30 smithi073 podman[79586]: 2024-07-31 15:17:30.888969918 +0000 UTC m=+0.360076243 container start 37f09999d454e2cbadedf79919a73cc3bb9c6d8395a0502e2370407a5f4935c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240716) 2024-07-31T15:17:31.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:30 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:30.686+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:31.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:30 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:30.910+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:31.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:31 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:31.136+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:31.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:31 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:31.136+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.108240+0000 front 2024-07-31T15:17:01.108293+0000 (oldest deadline 2024-07-31T15:17:25.108516+0000) 2024-07-31T15:17:31.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:31 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:31.105+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:31.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:31 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:31.105+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:31.363 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:30 smithi073 podman[79586]: 2024-07-31 15:17:30.994624353 +0000 UTC m=+0.465730688 container attach 37f09999d454e2cbadedf79919a73cc3bb9c6d8395a0502e2370407a5f4935c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc) 2024-07-31T15:17:31.364 INFO:journalctl@ceph.osd.4.smithi073.stdout:Jul 31 15:17:31 smithi073 podman[79586]: 2024-07-31 15:17:31.081317508 +0000 UTC m=+0.552423830 container died 37f09999d454e2cbadedf79919a73cc3bb9c6d8395a0502e2370407a5f4935c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-4-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-07-31T15:17:31.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:31.379+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:31.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:31.379+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:31.708 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:31 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:31.310+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:31.708 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:31 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:31.310+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:31.759 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.4.service' 2024-07-31T15:17:31.839 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:17:31.839 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-07-31T15:17:31.840 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-07-31T15:17:31.840 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.5 2024-07-31T15:17:32.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:31 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:31.707+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:32.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:31 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:31.745+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:32.208 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:31 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:31.895+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:32.208 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:32 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:32.097+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:32.208 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:32 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:32.097+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:32.209 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:31 smithi077 systemd[1]: Stopping Ceph osd.5 for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:17:32.209 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:32 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:32.093+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:32.209 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:32 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:32.093+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.108240+0000 front 2024-07-31T15:17:01.108293+0000 (oldest deadline 2024-07-31T15:17:25.108516+0000) 2024-07-31T15:17:32.209 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:32 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:32.206+0000 7fa2945e0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:17:32.209 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:32 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:32.206+0000 7fa2945e0640 -1 osd.5 144 *** Got signal Terminated *** 2024-07-31T15:17:32.209 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:32 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:32.206+0000 7fa2945e0640 -1 osd.5 144 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-31T15:17:32.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:32 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:32.289+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:32.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:32 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:32.289+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:32.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:32 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:32.387+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:32.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:32 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:32.387+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:33.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:32 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:32.662+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:33.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:32 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:32.758+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:33.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:32 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:32.923+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:33.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:33 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:33.110+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:33.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:33 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:33.110+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.108240+0000 front 2024-07-31T15:17:01.108293+0000 (oldest deadline 2024-07-31T15:17:25.108516+0000) 2024-07-31T15:17:33.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:33 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:33.099+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:33.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:33 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:33.099+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:33.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:33 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:33.262+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:33.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:33 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:33.262+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:33.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:33 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:33.262+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:33.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:33 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:33.423+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:33.795 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:33 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:33.423+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:34.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:33 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:33.626+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:34.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:33 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:33.793+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:34.143 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:33 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:33.793+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:34.237 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:34 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:34.115+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:34.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:34 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:34.115+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.108240+0000 front 2024-07-31T15:17:01.108293+0000 (oldest deadline 2024-07-31T15:17:25.108516+0000) 2024-07-31T15:17:34.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:33 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:33.881+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:34.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:33 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:33.881+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:34.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:34 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:34.114+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:34.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:34 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:34.114+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:34.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:34 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:34.214+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:34.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:34 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:34.214+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:34.554 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:34 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:34.214+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:34.791 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:34 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:34.405+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:34.792 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:34 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:34.405+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:35.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:34 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:34.591+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:35.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:34 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:34.790+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:35.143 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:34 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:34.790+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:35.237 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:35 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:35.099+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:35.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:35 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:35.099+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.108240+0000 front 2024-07-31T15:17:01.108293+0000 (oldest deadline 2024-07-31T15:17:25.108516+0000) 2024-07-31T15:17:35.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:34 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:34.880+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:35.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:34 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:34.880+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:35.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:35 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:35.122+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:35.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:35 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:35.122+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:35.550 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:35 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:35.233+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:35.551 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:35 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:35.233+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:35.551 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:35 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:35.233+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:35.803 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:35 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:35.549+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:35.803 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:35 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:35.549+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.631046+0000 front 2024-07-31T15:17:09.630857+0000 (oldest deadline 2024-07-31T15:17:35.530817+0000) 2024-07-31T15:17:35.841 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:35 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:35.415+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:35.842 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:35 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:35.415+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:35.842 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:35 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:35.415+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:36.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:35 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:35.840+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:36.143 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:35 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:35.840+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:36.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:35 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:35.929+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:36.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:35 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:35.929+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:36.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:36 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:36.148+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:36.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:36 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:36.148+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:36.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:36 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:36.059+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:36.239 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:36 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:36.059+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.108240+0000 front 2024-07-31T15:17:01.108293+0000 (oldest deadline 2024-07-31T15:17:25.108516+0000) 2024-07-31T15:17:36.239 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:36 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:36.059+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.311257+0000 front 2024-07-31T15:17:11.311282+0000 (oldest deadline 2024-07-31T15:17:35.411410+0000) 2024-07-31T15:17:36.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:36 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:36.190+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:36.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:36 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:36.190+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:36.554 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:36 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:36.190+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:36.792 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:36 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:36.413+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:36.793 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:36 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:36.413+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:36.793 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:36 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:36.413+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:37.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:36 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:36.586+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:37.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:36 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:36.586+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.631046+0000 front 2024-07-31T15:17:09.630857+0000 (oldest deadline 2024-07-31T15:17:35.530817+0000) 2024-07-31T15:17:37.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:36 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:36.791+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:37.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:36 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:36.791+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:37.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:36 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:36.959+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:37.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:36 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:36.959+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:37.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:37 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:37.044+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:52.907097+0000 front 2024-07-31T15:16:52.907161+0000 (oldest deadline 2024-07-31T15:17:17.607064+0000) 2024-07-31T15:17:37.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:37 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:37.044+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.108240+0000 front 2024-07-31T15:17:01.108293+0000 (oldest deadline 2024-07-31T15:17:25.108516+0000) 2024-07-31T15:17:37.238 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:37 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5[50669]: 2024-07-31T15:17:37.044+0000 7fa290bcb640 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.311257+0000 front 2024-07-31T15:17:11.311282+0000 (oldest deadline 2024-07-31T15:17:35.411410+0000) 2024-07-31T15:17:37.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:37 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:37.125+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:37.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:37 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:37.125+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:37.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:37 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:37.125+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:37.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:37 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:37.215+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:37.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:37 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:37.215+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:37.554 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:37 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:37.215+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:37.744 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:37.409+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:37.744 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:37.409+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:37.745 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:37.409+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:37.987 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:37 smithi077 podman[80315]: 2024-07-31 15:17:37.735622962 +0000 UTC m=+5.672721219 container died fa2fdfb8a06244b538828e76c22a369d27d9f3e5e22464517f722b157f965633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-07-31T15:17:37.988 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:37 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:37.910+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:37.988 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:37 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:37.911+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:38.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:37 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:37.634+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:38.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:37 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:37.634+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.631046+0000 front 2024-07-31T15:17:09.630857+0000 (oldest deadline 2024-07-31T15:17:35.530817+0000) 2024-07-31T15:17:38.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:37.743+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:38.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:37 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:37.743+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:38.487 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:38 smithi077 podman[80315]: 2024-07-31 15:17:38.299435666 +0000 UTC m=+6.236533920 container cleanup fa2fdfb8a06244b538828e76c22a369d27d9f3e5e22464517f722b157f965633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T15:17:38.488 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:38 smithi077 bash[80315]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5 2024-07-31T15:17:38.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:38 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:38.091+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:38.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:38 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:38.091+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:38.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:38 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:38.091+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:38.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:38 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:38.198+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:38.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:38 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:38.198+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:38.554 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:38 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:38.198+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:38.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:38 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:38.377+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:38.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:38 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:38.377+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:38.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:38 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:38.377+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:38.868 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:38 smithi077 podman[80332]: 2024-07-31 15:17:38.600764298 +0000 UTC m=+0.856674638 container remove fa2fdfb8a06244b538828e76c22a369d27d9f3e5e22464517f722b157f965633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0) 2024-07-31T15:17:39.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:38 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:38.677+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:39.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:38 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:38.677+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.631046+0000 front 2024-07-31T15:17:09.630857+0000 (oldest deadline 2024-07-31T15:17:35.530817+0000) 2024-07-31T15:17:39.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:38 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:38.711+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:39.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:38 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:38.711+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:39.222 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:38 smithi077 podman[80420]: 2024-07-31 15:17:38.827338207 +0000 UTC m=+0.017063468 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T15:17:39.222 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:38 smithi077 podman[80420]: 2024-07-31 15:17:38.968909989 +0000 UTC m=+0.158635248 container create 696530c660c039f653b56f7fa6931407be103757671a49c937a84daf0eb286f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS) 2024-07-31T15:17:39.222 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:39 smithi077 podman[80420]: 2024-07-31 15:17:39.221590797 +0000 UTC m=+0.411316071 container init 696530c660c039f653b56f7fa6931407be103757671a49c937a84daf0eb286f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5-deactivate, io.buildah.version=1.36.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-07-31T15:17:39.223 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:38 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:38.867+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:39.223 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:38 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:38.867+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:39.223 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:39 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:39.078+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:39.223 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:39 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:39.078+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:39.223 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:39 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:39.078+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:39.487 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:39 smithi077 podman[80420]: 2024-07-31 15:17:39.224791645 +0000 UTC m=+0.414516906 container start 696530c660c039f653b56f7fa6931407be103757671a49c937a84daf0eb286f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-07-31T15:17:39.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:39 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:39.164+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:39.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:39 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:39.164+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:39.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:39 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:39.164+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:39.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:39.384+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:39.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:39.384+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:39.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:39.384+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:39.987 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:39 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:39.909+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:39.987 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:39 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:39.909+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:39.988 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:39 smithi077 podman[80420]: 2024-07-31 15:17:39.730560402 +0000 UTC m=+0.920285663 container attach 696530c660c039f653b56f7fa6931407be103757671a49c937a84daf0eb286f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-07-31T15:17:39.988 INFO:journalctl@ceph.osd.5.smithi077.stdout:Jul 31 15:17:39 smithi077 podman[80420]: 2024-07-31 15:17:39.731056758 +0000 UTC m=+0.920782021 container died 696530c660c039f653b56f7fa6931407be103757671a49c937a84daf0eb286f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-5-deactivate, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T15:17:40.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:39 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:39.720+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:40.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:39 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:39.720+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.631046+0000 front 2024-07-31T15:17:09.630857+0000 (oldest deadline 2024-07-31T15:17:35.530817+0000) 2024-07-31T15:17:40.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:39.688+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:40.143 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:39 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:39.688+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:40.402 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:40 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:40.030+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:40.402 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:40 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:40.030+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:40.402 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:40 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:40.030+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:40.423 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.5.service' 2024-07-31T15:17:40.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:17:40.464 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-07-31T15:17:40.465 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-07-31T15:17:40.465 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.6 2024-07-31T15:17:40.493 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:40 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:40.171+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:40.493 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:40 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:40.171+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:40.493 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:40 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:40.171+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:40.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:40 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:40.385+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:40.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:40 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:40.385+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:40.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:40 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:40.385+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:40.758 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:40 smithi067 systemd[1]: Stopping Ceph osd.6 for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:17:41.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:40 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:40.757+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:41.054 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:40 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:40.757+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.631046+0000 front 2024-07-31T15:17:09.630857+0000 (oldest deadline 2024-07-31T15:17:35.530817+0000) 2024-07-31T15:17:41.054 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:40 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:40.806+0000 7fd9d978b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:17:41.054 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:40 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:40.806+0000 7fd9d978b640 -1 osd.6 144 *** Got signal Terminated *** 2024-07-31T15:17:41.054 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:40 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:40.806+0000 7fd9d978b640 -1 osd.6 144 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-31T15:17:41.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:40 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:40.652+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:41.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:40 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:40.652+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:41.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:40 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:40.938+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:41.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:40 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:40.938+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:41.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:41 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:41.064+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:41.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:41 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:41.064+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:41.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:41 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:41.064+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:41.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:41 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:41.126+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:41.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:41 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:41.126+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:41.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:41 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:41.126+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:41.804 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:41 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:41.618+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:41.805 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:41 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:41.618+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:41.805 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:41 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:41.413+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:41.805 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:41 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:41.413+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:41.805 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:41 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:41.413+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:42.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:41 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:41.763+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:42.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:41 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:41.763+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.631046+0000 front 2024-07-31T15:17:09.630857+0000 (oldest deadline 2024-07-31T15:17:35.530817+0000) 2024-07-31T15:17:42.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:41 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:41.899+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:42.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:41 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:41.899+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:42.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:42 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:42.045+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:42.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:42 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:42.045+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:42.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:42 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:42.045+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:42.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:42 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:42.099+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:42.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:42 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:42.099+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:42.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:42 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:42.099+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:42.642 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:42.612+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:42.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:42.612+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:42.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:42.612+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.782020+0000 front 2024-07-31T15:17:17.782008+0000 (oldest deadline 2024-07-31T15:17:42.481929+0000) 2024-07-31T15:17:42.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:42.386+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:42.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:42.386+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:42.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:42.386+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:43.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:42 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:42.769+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:43.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:42 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:42.769+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.631046+0000 front 2024-07-31T15:17:09.630857+0000 (oldest deadline 2024-07-31T15:17:35.530817+0000) 2024-07-31T15:17:43.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:42 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:42.945+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:43.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:42 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:42.945+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:43.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:42 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:42.945+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:43.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:43 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:43.093+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:43.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:43 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:43.093+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:43.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:43 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:43.093+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:43.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:43.103+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:43.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:43.103+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:43.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:43.103+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:43.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:43.103+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:43.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:43 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:43.591+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:43.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:43 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:43.591+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:43.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:43 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:43.591+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.782020+0000 front 2024-07-31T15:17:17.782008+0000 (oldest deadline 2024-07-31T15:17:42.481929+0000) 2024-07-31T15:17:43.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:43 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:43.379+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:43.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:43 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:43.379+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:43.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:43 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:43.379+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:44.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:43 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:43.924+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:44.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:43 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:43.924+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:44.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:43 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:43.924+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:44.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:44 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:44.077+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:44.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:44 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:44.077+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:44.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:44 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:44.077+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:44.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:44 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:44.059+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:44.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:44 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:44.059+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:44.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:44 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:44.059+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:44.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:44 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:44.059+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:44.304 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:43.815+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:44.304 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:43.815+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.631046+0000 front 2024-07-31T15:17:09.630857+0000 (oldest deadline 2024-07-31T15:17:35.530817+0000) 2024-07-31T15:17:44.304 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:43 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:43.815+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:18.332301+0000 front 2024-07-31T15:17:18.332258+0000 (oldest deadline 2024-07-31T15:17:43.632361+0000) 2024-07-31T15:17:44.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:44 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:44.544+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:44.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:44 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:44.544+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:44.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:44 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:44.544+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.782020+0000 front 2024-07-31T15:17:17.782008+0000 (oldest deadline 2024-07-31T15:17:42.481929+0000) 2024-07-31T15:17:44.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:44 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:44.357+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:44.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:44 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:44.357+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:44.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:44 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:44.357+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:44.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:44 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:44.357+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:45.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:44 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:44.798+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.730300+0000 front 2024-07-31T15:17:03.730365+0000 (oldest deadline 2024-07-31T15:17:29.630209+0000) 2024-07-31T15:17:45.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:44 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:44.798+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.631046+0000 front 2024-07-31T15:17:09.630857+0000 (oldest deadline 2024-07-31T15:17:35.530817+0000) 2024-07-31T15:17:45.053 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:44 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6[66704]: 2024-07-31T15:17:44.798+0000 7fd9d1d6e640 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:18.332301+0000 front 2024-07-31T15:17:18.332258+0000 (oldest deadline 2024-07-31T15:17:43.632361+0000) 2024-07-31T15:17:45.054 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:45 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:45.031+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:45.054 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:45 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:45.031+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:45.054 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:45 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:45.031+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:45.054 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:45 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:45.031+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:45.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:44 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:44.902+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:45.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:44 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:44.902+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:45.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:44 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:44.902+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:45.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:45 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:45.039+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:45.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:45 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:45.039+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:45.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:45 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:45.039+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:45.892 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:45 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:45.510+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:45.893 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:45 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:45.510+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:45.893 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:45 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:45.510+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.782020+0000 front 2024-07-31T15:17:17.782008+0000 (oldest deadline 2024-07-31T15:17:42.481929+0000) 2024-07-31T15:17:45.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:45 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:45.406+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:45.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:45 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:45.406+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:45.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:45 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:45.406+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:45.894 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:45 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:45.406+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:46.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:45 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:45.919+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:46.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:45 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:45.919+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:46.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:45 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:45.919+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:46.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:45 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:45.998+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:46.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:45 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:45.998+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:46.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:45 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:45.998+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:46.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:46 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:46.033+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:46.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:46 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:46.033+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:46.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:46 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:46.033+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:46.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:46 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:46.033+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:46.642 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:46 smithi067 podman[187546]: 2024-07-31 15:17:46.354528808 +0000 UTC m=+5.685787387 container died 8d81460cf6227255cc79fc1df75ca4e2bfb26164b4d5029d4e74945d5f05c55a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc) 2024-07-31T15:17:46.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:46 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:46.474+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:46.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:46 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:46.474+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:46.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:46 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:46.474+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.782020+0000 front 2024-07-31T15:17:17.782008+0000 (oldest deadline 2024-07-31T15:17:42.481929+0000) 2024-07-31T15:17:46.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:46 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:46.361+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:46.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:46 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:46.361+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:46.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:46 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:46.361+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:46.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:46 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:46.361+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:47.001 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:46 smithi067 podman[187546]: 2024-07-31 15:17:46.824184143 +0000 UTC m=+6.155442721 container cleanup 8d81460cf6227255cc79fc1df75ca4e2bfb26164b4d5029d4e74945d5f05c55a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-07-31T15:17:47.001 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:46 smithi067 bash[187546]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6 2024-07-31T15:17:47.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:46 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:46.910+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:47.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:46 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:46.910+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:47.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:46 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:46.910+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:47.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:46 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:46.977+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:47.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:46 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:46.977+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:47.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:46 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:46.977+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:47.303 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:47 smithi067 podman[187561]: 2024-07-31 15:17:47.000539063 +0000 UTC m=+0.645729488 container remove 8d81460cf6227255cc79fc1df75ca4e2bfb26164b4d5029d4e74945d5f05c55a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T15:17:47.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:47.067+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:47.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:47.067+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:47.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:47.067+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:47.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:47 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:47.067+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:47.642 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:47 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:47.448+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:47.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:47 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:47.448+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:47.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:47 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:47.448+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.782020+0000 front 2024-07-31T15:17:17.782008+0000 (oldest deadline 2024-07-31T15:17:42.481929+0000) 2024-07-31T15:17:47.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:47 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:47.338+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:47.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:47 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:47.338+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:47.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:47 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:47.338+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:47.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:47 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:47.338+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:47.738 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:47 smithi067 podman[187650]: 2024-07-31 15:17:47.451565953 +0000 UTC m=+0.114308043 container create e6f907307399092836f58a155eb85108dfebe87760fb52b5253a6bc13b71a4d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-31T15:17:47.738 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:47 smithi067 podman[187650]: 2024-07-31 15:17:47.354558841 +0000 UTC m=+0.017300933 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T15:17:47.738 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:47 smithi067 podman[187650]: 2024-07-31 15:17:47.662237004 +0000 UTC m=+0.324979110 container init e6f907307399092836f58a155eb85108dfebe87760fb52b5253a6bc13b71a4d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-07-31T15:17:47.739 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:47 smithi067 podman[187650]: 2024-07-31 15:17:47.665646981 +0000 UTC m=+0.328389073 container start e6f907307399092836f58a155eb85108dfebe87760fb52b5253a6bc13b71a4d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-07-31T15:17:48.037 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:47 smithi067 podman[187650]: 2024-07-31 15:17:47.737138033 +0000 UTC m=+0.399880136 container attach e6f907307399092836f58a155eb85108dfebe87760fb52b5253a6bc13b71a4d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc) 2024-07-31T15:17:48.038 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 31 15:17:47 smithi067 podman[187650]: 2024-07-31 15:17:47.864644143 +0000 UTC m=+0.527386237 container died e6f907307399092836f58a155eb85108dfebe87760fb52b5253a6bc13b71a4d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T15:17:48.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:47 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:47.884+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:48.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:47 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:47.884+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:48.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:47 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:47.884+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:48.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:47 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:47.948+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:48.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:47 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:47.948+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:48.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:47 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:47.948+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:48.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:48 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:48.036+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:48.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:48 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:48.036+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:48.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:48 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:48.036+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:48.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:48 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:48.036+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:48.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.6.service' 2024-07-31T15:17:48.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:17:48.625 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-07-31T15:17:48.625 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-07-31T15:17:48.625 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.7 2024-07-31T15:17:48.642 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:48 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:48.457+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:48.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:48 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:48.457+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:48.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:48 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:48.457+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.782020+0000 front 2024-07-31T15:17:17.782008+0000 (oldest deadline 2024-07-31T15:17:42.481929+0000) 2024-07-31T15:17:48.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:48 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:48.321+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:48.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:48 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:48.321+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:48.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:48 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:48.321+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:48.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:48 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:48.321+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:48.935 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:48 smithi073 systemd[1]: Stopping Ceph osd.7 for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:17:49.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:48 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:48.884+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:49.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:48 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:48.884+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:49.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:48 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:48.884+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:49.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:48 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:48.970+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:49.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:48 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:48.970+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:49.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:48 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:48.971+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:49.337 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:48 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:48.934+0000 7fbe438df640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:17:49.337 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:48 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:48.934+0000 7fbe438df640 -1 osd.7 144 *** Got signal Terminated *** 2024-07-31T15:17:49.337 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:48 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:48.934+0000 7fbe438df640 -1 osd.7 144 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-31T15:17:49.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:49 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:49.069+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:49.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:49 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:49.069+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:49.553 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:49 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:49.069+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:49.554 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:49 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:49.069+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:49.642 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:49 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:49.505+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:49.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:49 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:49.505+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:49.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:49 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:49.505+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.782020+0000 front 2024-07-31T15:17:17.782008+0000 (oldest deadline 2024-07-31T15:17:42.481929+0000) 2024-07-31T15:17:49.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:49 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:49.336+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:49.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:49 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:49.336+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:49.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:49 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:49.336+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:49.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:49 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:49.336+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:50.053 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:50 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:50.022+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:50.053 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:50 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:50.022+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:50.053 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:50 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:50.022+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:50.054 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:50 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:50.022+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:50.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:49 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:49.865+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:50.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:49 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:49.865+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:50.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:49 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:49.865+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:50.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:50 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:50.010+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:50.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:50 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:50.010+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:50.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:50 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:50.010+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:50.642 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:50 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:50.536+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:50.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:50 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:50.536+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:50.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:50 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:50.536+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.782020+0000 front 2024-07-31T15:17:17.782008+0000 (oldest deadline 2024-07-31T15:17:42.481929+0000) 2024-07-31T15:17:50.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:50 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:50.309+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:50.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:50 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:50.309+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:50.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:50 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:50.309+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:50.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:50 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:50.309+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:51.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:50 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:50.879+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:51.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:50 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:50.879+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:51.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:50 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:50.880+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:51.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:51 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:51.054+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:51.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:51 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:51.054+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:51.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:51 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:51.054+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:51.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:50 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:50.985+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:51.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:50 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:50.985+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:51.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:50 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:50.985+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:51.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:50 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:50.985+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:51.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:51.491+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:51.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:51.491+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:51.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:51.491+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.782020+0000 front 2024-07-31T15:17:17.782008+0000 (oldest deadline 2024-07-31T15:17:42.481929+0000) 2024-07-31T15:17:51.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:51.491+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.983161+0000 front 2024-07-31T15:17:25.983136+0000 (oldest deadline 2024-07-31T15:17:51.283228+0000) 2024-07-31T15:17:51.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:51.356+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:51.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:51.356+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:51.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:51.356+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:51.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:51 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:51.356+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:52.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:51 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:51.901+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:52.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:51 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:51.901+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:52.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:51 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:51.901+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:52.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:52 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:52.026+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:52.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:52 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:52.026+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:52.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:52 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:52.026+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:52.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:52 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:52.026+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:17:52.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:51 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:51.959+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:52.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:51 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:51.959+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:52.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:51 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:51.959+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:52.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:51 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:51.959+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:52.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:52 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:52.509+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:52.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:52 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:52.509+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:52.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:52 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:52.509+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.782020+0000 front 2024-07-31T15:17:17.782008+0000 (oldest deadline 2024-07-31T15:17:42.481929+0000) 2024-07-31T15:17:52.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:52 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:52.509+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.983161+0000 front 2024-07-31T15:17:25.983136+0000 (oldest deadline 2024-07-31T15:17:51.283228+0000) 2024-07-31T15:17:52.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:52 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:52.336+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:52.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:52 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:52.336+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:52.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:52 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:52.336+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:52.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:52 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:52.336+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:53.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:52 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:52.926+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:53.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:52 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:52.926+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:53.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:52 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:52.926+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:53.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:52 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:52.926+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:28.631130+0000 front 2024-07-31T15:17:28.631011+0000 (oldest deadline 2024-07-31T15:17:52.731005+0000) 2024-07-31T15:17:53.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:53 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:53.033+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:53.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:53 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:53.033+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:53.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:53 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:53.033+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:53.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:53 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:53.033+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:17:53.305 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:52 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:52.917+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:53.305 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:52 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:52.917+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:53.305 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:52 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:52.917+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:53.305 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:52 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:52.917+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:53.306 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:52 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:52.917+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:17:53.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:53 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:53.471+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:01.379541+0000 front 2024-07-31T15:17:01.379565+0000 (oldest deadline 2024-07-31T15:17:27.279644+0000) 2024-07-31T15:17:53.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:53 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:53.471+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:09.580585+0000 front 2024-07-31T15:17:09.580819+0000 (oldest deadline 2024-07-31T15:17:33.680629+0000) 2024-07-31T15:17:53.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:53 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:53.471+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.782020+0000 front 2024-07-31T15:17:17.782008+0000 (oldest deadline 2024-07-31T15:17:42.481929+0000) 2024-07-31T15:17:53.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:53 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7[55981]: 2024-07-31T15:17:53.471+0000 7fbe3bec2640 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.983161+0000 front 2024-07-31T15:17:25.983136+0000 (oldest deadline 2024-07-31T15:17:51.283228+0000) 2024-07-31T15:17:53.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:53 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:53.300+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:53.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:53 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:53.300+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:53.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:53 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:53.300+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:53.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:53 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:53.300+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:54.237 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:53 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:53.895+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:54.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:53 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:53.895+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:54.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:53 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:53.895+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:54.238 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:53 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:53.895+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:28.631130+0000 front 2024-07-31T15:17:28.631011+0000 (oldest deadline 2024-07-31T15:17:52.731005+0000) 2024-07-31T15:17:54.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:54 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:54.058+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:54.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:54 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:54.058+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:54.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:54 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:54.058+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:54.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:54 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:54.058+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:17:54.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:53 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:53.936+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:54.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:53 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:53.936+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:54.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:53 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:53.936+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:54.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:53 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:53.936+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:54.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:53 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:53.936+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:17:54.643 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:54 smithi073 podman[79701]: 2024-07-31 15:17:54.525856078 +0000 UTC m=+5.700338287 container died 0053f8eb312ad0dc4c28f3fe59b7ba37a315a12eed60c6ab4c4db0eec464a2c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240716) 2024-07-31T15:17:54.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:54 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:54.317+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:54.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:54 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:54.317+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:54.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:54 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:54.317+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:54.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:54 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:54.317+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:54.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:54 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:54.317+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:17:55.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:54 smithi073 podman[79701]: 2024-07-31 15:17:54.95145262 +0000 UTC m=+6.125934818 container cleanup 0053f8eb312ad0dc4c28f3fe59b7ba37a315a12eed60c6ab4c4db0eec464a2c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-07-31T15:17:55.143 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:54 smithi073 bash[79701]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7 2024-07-31T15:17:55.162 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:55 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:55.103+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:55.162 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:55 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:55.103+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:55.162 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:55 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:55.103+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:55.162 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:55 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:55.103+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:17:55.163 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:54 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:54.895+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:55.163 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:54 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:54.895+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:55.163 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:54 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:54.895+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:55.163 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:54 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:54.895+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:28.631130+0000 front 2024-07-31T15:17:28.631011+0000 (oldest deadline 2024-07-31T15:17:52.731005+0000) 2024-07-31T15:17:55.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:54 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:54.893+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:55.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:54 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:54.893+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:55.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:54 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:54.893+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:55.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:54 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:54.893+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:55.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:54 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:54.893+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:17:55.562 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:55 smithi073 podman[79716]: 2024-07-31 15:17:55.152979874 +0000 UTC m=+0.623132810 container remove 0053f8eb312ad0dc4c28f3fe59b7ba37a315a12eed60c6ab4c4db0eec464a2c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.schema-version=1.0) 2024-07-31T15:17:55.563 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:55 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:55.345+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:55.563 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:55 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:55.345+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:55.563 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:55 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:55.345+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:55.563 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:55 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:55.345+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:55.563 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:55 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:55.345+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:17:55.873 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:55 smithi073 podman[79804]: 2024-07-31 15:17:55.561891915 +0000 UTC m=+0.107797467 container create 3c66053bb16262171e74338def27f6eaca89eef27e27689ec3238e4594b86773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-07-31T15:17:55.874 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:55 smithi073 podman[79804]: 2024-07-31 15:17:55.471067373 +0000 UTC m=+0.016972927 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T15:17:55.874 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:55 smithi073 podman[79804]: 2024-07-31 15:17:55.781190273 +0000 UTC m=+0.327095826 container init 3c66053bb16262171e74338def27f6eaca89eef27e27689ec3238e4594b86773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux ) 2024-07-31T15:17:55.874 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:55 smithi073 podman[79804]: 2024-07-31 15:17:55.784293886 +0000 UTC m=+0.330199447 container start 3c66053bb16262171e74338def27f6eaca89eef27e27689ec3238e4594b86773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7-deactivate, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T15:17:55.874 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:55 smithi073 podman[79804]: 2024-07-31 15:17:55.87287988 +0000 UTC m=+0.418785446 container attach 3c66053bb16262171e74338def27f6eaca89eef27e27689ec3238e4594b86773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716) 2024-07-31T15:17:56.104 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:55 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:55.851+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:56.104 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:55 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:55.851+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:56.104 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:55 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:55.851+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:56.105 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:55 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:55.851+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:28.631130+0000 front 2024-07-31T15:17:28.631011+0000 (oldest deadline 2024-07-31T15:17:52.731005+0000) 2024-07-31T15:17:56.142 INFO:journalctl@ceph.osd.7.smithi073.stdout:Jul 31 15:17:55 smithi073 podman[79804]: 2024-07-31 15:17:55.970561601 +0000 UTC m=+0.516467155 container died 3c66053bb16262171e74338def27f6eaca89eef27e27689ec3238e4594b86773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True) 2024-07-31T15:17:56.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:55 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:55.870+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:56.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:55 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:55.870+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:56.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:55 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:55.870+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:56.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:55 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:55.870+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:56.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:55 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:55.870+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:17:56.361 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:56 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:56.103+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:56.361 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:56 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:56.103+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:56.361 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:56 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:56.103+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:56.361 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:56 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:56.103+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:17:56.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:56 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:56.351+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:56.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:56 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:56.351+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:56.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:56 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:56.351+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:56.545 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:56 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:56.351+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:56.545 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:56 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:56.351+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:17:56.563 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.7.service' 2024-07-31T15:17:56.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:17:56.643 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-07-31T15:17:56.643 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-07-31T15:17:56.643 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.8 2024-07-31T15:17:56.946 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:56 smithi077 systemd[1]: Stopping Ceph osd.8 for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:17:56.946 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:56 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:56.804+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:56.946 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:56 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:56.804+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:56.947 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:56 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:56.804+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:56.947 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:56 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:56.804+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:28.631130+0000 front 2024-07-31T15:17:28.631011+0000 (oldest deadline 2024-07-31T15:17:52.731005+0000) 2024-07-31T15:17:56.947 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:56 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:56.945+0000 7fc2cd9c8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:17:56.947 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:56 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:56.945+0000 7fc2cd9c8640 -1 osd.8 144 *** Got signal Terminated *** 2024-07-31T15:17:56.947 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:56 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:56.945+0000 7fc2cd9c8640 -1 osd.8 144 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-31T15:17:57.205 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:56 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:56.852+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:57.205 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:56 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:56.852+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:57.205 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:56 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:56.852+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:57.205 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:56 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:56.852+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:57.205 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:56 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:56.852+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:17:57.237 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:57 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:57.054+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:57.237 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:57 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:57.054+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:57.237 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:57 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:57.054+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:57.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:57 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:57.054+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:17:57.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:57 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:57.360+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:57.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:57 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:57.360+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:57.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:57 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:57.360+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:57.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:57 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:57.360+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:57.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:57 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:57.360+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:17:58.101 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:57 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:57.820+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:58.101 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:57 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:57.820+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:58.101 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:57 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:57.820+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:58.101 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:57 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:57.820+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:28.631130+0000 front 2024-07-31T15:17:28.631011+0000 (oldest deadline 2024-07-31T15:17:52.731005+0000) 2024-07-31T15:17:58.216 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:57 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:57.899+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:58.216 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:57 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:57.899+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:58.216 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:57 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:57.899+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:58.217 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:57 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:57.899+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:58.217 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:57 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:57.899+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:17:58.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:58 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:58.100+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:58.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:58 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:58.100+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:58.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:58 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:58.100+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:58.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:58 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:58.100+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:17:58.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:58 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:58.346+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:58.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:58 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:58.346+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:58.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:58 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:58.346+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:58.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:58 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:58.346+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:58.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:58 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:58.346+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:17:59.068 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:58 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:58.811+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:17:59.068 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:58 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:58.811+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:17:59.069 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:58 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:58.811+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:17:59.069 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:58 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:58.811+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:28.631130+0000 front 2024-07-31T15:17:28.631011+0000 (oldest deadline 2024-07-31T15:17:52.731005+0000) 2024-07-31T15:17:59.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:58 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:58.896+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:17:59.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:58 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:58.896+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:17:59.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:58 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:58.896+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:17:59.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:58 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:58.896+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:17:59.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:58 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:58.896+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:17:59.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:59 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:59.067+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:17:59.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:59 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:59.067+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:17:59.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:59 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:59.067+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:17:59.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:59 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:59.067+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:17:59.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:17:59 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:17:59.067+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:17:59.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:59 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:59.300+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:17:59.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:59 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:59.300+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:17:59.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:59 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:59.300+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:17:59.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:59 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:59.300+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:17:59.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:17:59 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:17:59.300+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:00.068 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:59 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:59.772+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:18:00.069 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:59 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:59.772+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:18:00.069 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:59 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:59.772+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:18:00.069 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:59 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:59.772+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:28.631130+0000 front 2024-07-31T15:17:28.631011+0000 (oldest deadline 2024-07-31T15:17:52.731005+0000) 2024-07-31T15:18:00.069 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:17:59 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:17:59.772+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.831983+0000 front 2024-07-31T15:17:33.831960+0000 (oldest deadline 2024-07-31T15:17:59.134779+0000) 2024-07-31T15:18:00.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:59 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:59.900+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:18:00.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:59 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:59.900+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:18:00.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:59 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:59.900+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:18:00.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:59 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:59.900+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:18:00.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:17:59 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:17:59.900+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:18:00.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:00 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:00.067+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:00.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:00 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:00.067+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:00.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:00 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:00.067+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:00.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:00 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:00.067+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:00.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:00 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:00.067+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:00.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:00 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:00.254+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:00.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:00 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:00.254+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:00.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:00 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:00.254+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:00.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:00 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:00.254+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:00.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:00 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:00.254+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:00.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:00 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:00.254+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:01.076 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:00 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:18:00.781+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:18:01.077 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:00 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:18:00.781+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:18:01.077 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:00 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:18:00.781+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:18:01.077 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:00 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:18:00.781+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:28.631130+0000 front 2024-07-31T15:17:28.631011+0000 (oldest deadline 2024-07-31T15:17:52.731005+0000) 2024-07-31T15:18:01.077 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:00 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:18:00.781+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.831983+0000 front 2024-07-31T15:17:33.831960+0000 (oldest deadline 2024-07-31T15:17:59.134779+0000) 2024-07-31T15:18:01.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:00 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:00.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:18:01.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:00 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:00.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:18:01.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:00 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:00.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:18:01.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:00 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:00.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:18:01.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:00 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:00.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:18:01.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:01 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:01.075+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:01.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:01 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:01.075+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:01.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:01 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:01.075+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:01.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:01 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:01.075+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:01.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:01 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:01.075+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:01.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:01 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:01.272+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:01.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:01 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:01.272+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:01.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:01 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:01.272+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:01.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:01 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:01.272+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:01.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:01 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:01.272+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:01.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:01 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:01.272+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:02.125 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:01 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:18:01.817+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:00.427859+0000 front 2024-07-31T15:17:00.427667+0000 (oldest deadline 2024-07-31T15:17:25.127542+0000) 2024-07-31T15:18:02.125 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:01 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:18:01.817+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:07.428236+0000 front 2024-07-31T15:17:07.428406+0000 (oldest deadline 2024-07-31T15:17:33.328456+0000) 2024-07-31T15:18:02.125 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:01 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:18:01.817+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.430157+0000 front 2024-07-31T15:17:17.430070+0000 (oldest deadline 2024-07-31T15:17:42.729862+0000) 2024-07-31T15:18:02.125 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:01 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:18:01.817+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:28.631130+0000 front 2024-07-31T15:17:28.631011+0000 (oldest deadline 2024-07-31T15:17:52.731005+0000) 2024-07-31T15:18:02.125 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:01 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8[57340]: 2024-07-31T15:18:01.817+0000 7fc2c9fb3640 -1 osd.8 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.831983+0000 front 2024-07-31T15:17:33.831960+0000 (oldest deadline 2024-07-31T15:17:59.134779+0000) 2024-07-31T15:18:02.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:01 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:01.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:18:02.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:01 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:01.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:18:02.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:01 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:01.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:18:02.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:01 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:01.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:18:02.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:01 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:01.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:18:02.487 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:02 smithi077 podman[80534]: 2024-07-31 15:18:02.357006984 +0000 UTC m=+5.532958290 container died f2833fc5abbe1c594442bff5820f5575176d53399870a51f05c27a1ad3cca401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-07-31T15:18:02.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:02 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:02.123+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:02.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:02 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:02.123+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:02.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:02 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:02.123+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:02.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:02 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:02.123+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:02.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:02 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:02.123+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:02.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:02 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:02.237+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:02.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:02 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:02.237+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:02.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:02 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:02.237+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:02.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:02 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:02.237+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:02.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:02 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:02.237+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:02.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:02 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:02.237+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:02.987 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:02 smithi077 podman[80534]: 2024-07-31 15:18:02.745522143 +0000 UTC m=+5.921473435 container cleanup f2833fc5abbe1c594442bff5820f5575176d53399870a51f05c27a1ad3cca401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8, ceph=True, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T15:18:02.988 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:02 smithi077 bash[80534]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8 2024-07-31T15:18:03.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:02 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:02.915+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:18:03.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:02 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:02.915+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:18:03.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:02 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:02.915+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:18:03.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:02 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:02.915+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:18:03.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:02 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:02.915+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:18:03.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:02 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:02.915+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:36.620273+0000 front 2024-07-31T15:17:36.620291+0000 (oldest deadline 2024-07-31T15:18:02.520112+0000) 2024-07-31T15:18:03.313 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:02 smithi077 podman[80549]: 2024-07-31 15:18:02.996600488 +0000 UTC m=+0.640066776 container remove f2833fc5abbe1c594442bff5820f5575176d53399870a51f05c27a1ad3cca401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T15:18:03.313 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:03 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:03.084+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:03.313 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:03 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:03.084+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:03.313 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:03 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:03.084+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:03.313 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:03 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:03.084+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:03.313 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:03 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:03.084+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:03.567 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:03 smithi077 podman[80640]: 2024-07-31 15:18:03.215076716 +0000 UTC m=+0.017092844 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T15:18:03.567 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:03 smithi077 podman[80640]: 2024-07-31 15:18:03.326383047 +0000 UTC m=+0.128399166 container create 793d7033427154787d3ab1487845f80e4b9e0562c12f91dd151f0f06f26243d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T15:18:03.567 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:03 smithi077 podman[80640]: 2024-07-31 15:18:03.566552994 +0000 UTC m=+0.368569118 container init 793d7033427154787d3ab1487845f80e4b9e0562c12f91dd151f0f06f26243d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-07-31T15:18:03.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:03 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:03.194+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:03.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:03 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:03.194+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:03.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:03 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:03.194+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:03.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:03 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:03.194+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:03.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:03 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:03.194+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:03.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:03 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:03.194+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:03.987 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:03 smithi077 podman[80640]: 2024-07-31 15:18:03.570041784 +0000 UTC m=+0.372057913 container start 793d7033427154787d3ab1487845f80e4b9e0562c12f91dd151f0f06f26243d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8-deactivate, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-31T15:18:03.988 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:03 smithi077 podman[80640]: 2024-07-31 15:18:03.64185627 +0000 UTC m=+0.443872404 container attach 793d7033427154787d3ab1487845f80e4b9e0562c12f91dd151f0f06f26243d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T15:18:03.988 INFO:journalctl@ceph.osd.8.smithi077.stdout:Jul 31 15:18:03 smithi077 podman[80640]: 2024-07-31 15:18:03.766727882 +0000 UTC m=+0.568744013 container died 793d7033427154787d3ab1487845f80e4b9e0562c12f91dd151f0f06f26243d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-07-31T15:18:04.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:03 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:03.939+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:18:04.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:03 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:03.939+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:18:04.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:03 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:03.939+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:18:04.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:03 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:03.939+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:18:04.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:03 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:03.939+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:18:04.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:03 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:03.939+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:36.620273+0000 front 2024-07-31T15:17:36.620291+0000 (oldest deadline 2024-07-31T15:18:02.520112+0000) 2024-07-31T15:18:04.421 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:04 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:04.068+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:04.421 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:04 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:04.068+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:04.421 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:04 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:04.069+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:04.421 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:04 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:04.069+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:04.421 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:04 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:04.069+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:04.443 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.8.service' 2024-07-31T15:18:04.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:18:04.484 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-07-31T15:18:04.484 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-07-31T15:18:04.484 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.9 2024-07-31T15:18:04.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:04 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:04.241+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:04.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:04 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:04.241+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:04.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:04 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:04.241+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:04.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:04 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:04.241+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:04.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:04 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:04.241+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:04.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:04 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:04.241+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:04.803 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:04 smithi067 systemd[1]: Stopping Ceph osd.9 for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:18:05.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:04 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:04.819+0000 7fb8961c2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:18:05.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:04 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:04.819+0000 7fb8961c2640 -1 osd.9 144 *** Got signal Terminated *** 2024-07-31T15:18:05.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:04 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:04.819+0000 7fb8961c2640 -1 osd.9 144 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-31T15:18:05.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:04 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:04.899+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:18:05.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:04 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:04.899+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:18:05.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:04 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:04.899+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:18:05.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:04 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:04.899+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:18:05.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:04 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:04.899+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:18:05.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:04 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:04.899+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:36.620273+0000 front 2024-07-31T15:17:36.620291+0000 (oldest deadline 2024-07-31T15:18:02.520112+0000) 2024-07-31T15:18:05.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:05 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:05.080+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:05.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:05 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:05.080+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:05.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:05 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:05.080+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:05.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:05 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:05.080+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:05.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:05 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:05.080+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:05.543 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:05 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:05.277+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:05.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:05 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:05.277+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:05.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:05 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:05.277+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:05.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:05 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:05.277+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:05.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:05 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:05.277+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:05.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:05 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:05.277+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:06.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:05 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:05.884+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:18:06.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:05 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:05.884+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:18:06.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:05 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:05.884+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:18:06.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:05 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:05.884+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:18:06.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:05 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:05.884+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:18:06.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:05 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:05.884+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:36.620273+0000 front 2024-07-31T15:17:36.620291+0000 (oldest deadline 2024-07-31T15:18:02.520112+0000) 2024-07-31T15:18:06.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:06 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:06.121+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:06.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:06 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:06.121+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:06.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:06 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:06.121+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:06.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:06 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:06.121+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:06.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:06 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:06.121+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:06.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:06 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:06.318+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:06.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:06 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:06.318+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:06.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:06 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:06.318+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:06.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:06 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:06.318+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:06.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:06 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:06.318+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:06.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:06 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:06.318+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:07.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:06 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:06.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:18:07.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:06 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:06.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:18:07.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:06 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:06.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:18:07.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:06 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:06.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:18:07.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:06 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:06.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:18:07.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:06 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:06.901+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:36.620273+0000 front 2024-07-31T15:17:36.620291+0000 (oldest deadline 2024-07-31T15:18:02.520112+0000) 2024-07-31T15:18:07.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:07 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:07.085+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:07.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:07 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:07.085+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:07.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:07 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:07.085+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:07.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:07 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:07.085+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:07.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:07 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:07.085+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:07.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:07 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:07.321+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:07.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:07 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:07.321+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:07.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:07 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:07.321+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:07.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:07 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:07.321+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:07.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:07 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:07.321+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:07.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:07 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:07.321+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:08.260 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:07 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:07.933+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:18:08.260 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:07 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:07.933+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:18:08.260 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:07 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:07.933+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:18:08.261 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:07 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:07.933+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:18:08.261 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:07 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:07.933+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:18:08.261 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:07 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:07.933+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:36.620273+0000 front 2024-07-31T15:17:36.620291+0000 (oldest deadline 2024-07-31T15:18:02.520112+0000) 2024-07-31T15:18:08.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:08 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:08.091+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:08.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:08 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:08.091+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:08.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:08 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:08.091+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:08.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:08 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:08.091+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:08.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:08 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:08.091+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:08.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:08 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:08.314+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:08.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:08 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:08.314+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:08.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:08 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:08.314+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:08.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:08 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:08.314+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:08.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:08 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:08.314+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:08.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:08 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:08.314+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:09.303 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:08 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:08.936+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.210690+0000 front 2024-07-31T15:16:53.210716+0000 (oldest deadline 2024-07-31T15:17:17.310865+0000) 2024-07-31T15:18:09.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:08 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:08.936+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:03.212384+0000 front 2024-07-31T15:17:03.212336+0000 (oldest deadline 2024-07-31T15:17:26.712325+0000) 2024-07-31T15:18:09.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:08 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:08.936+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.413645+0000 front 2024-07-31T15:17:08.413448+0000 (oldest deadline 2024-07-31T15:17:33.113514+0000) 2024-07-31T15:18:09.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:08 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:08.936+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:17.615655+0000 front 2024-07-31T15:17:17.615746+0000 (oldest deadline 2024-07-31T15:17:42.216278+0000) 2024-07-31T15:18:09.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:08 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:08.936+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:29.118481+0000 front 2024-07-31T15:17:29.118529+0000 (oldest deadline 2024-07-31T15:17:52.618399+0000) 2024-07-31T15:18:09.304 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:08 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9[73688]: 2024-07-31T15:18:08.936+0000 7fb88e7a5640 -1 osd.9 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:36.620273+0000 front 2024-07-31T15:17:36.620291+0000 (oldest deadline 2024-07-31T15:18:02.520112+0000) 2024-07-31T15:18:09.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:09 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:09.068+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:09.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:09 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:09.068+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:09.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:09 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:09.068+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:09.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:09 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:09.068+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:09.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:09 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:09.068+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:09.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:09 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:09.329+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:09.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:09 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:09.329+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:09.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:09 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:09.329+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:09.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:09 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:09.329+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:09.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:09 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:09.329+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:09.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:09 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:09.329+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:10.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:10 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:10.044+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:10.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:10 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:10.044+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:10.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:10 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:10.044+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:10.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:10 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:10.044+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:10.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:10 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:10.044+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:10.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:10 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:10.044+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:10.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:10 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:10.347+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:10.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:10 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:10.347+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:10.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:10 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:10.347+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:10.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:10 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:10.347+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:10.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:10 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:10.347+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:10.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:10 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:10.347+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:10.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:10 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:10.347+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:40.864632+0000 front 2024-07-31T15:17:40.864818+0000 (oldest deadline 2024-07-31T15:18:09.664887+0000) 2024-07-31T15:18:10.740 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:10 smithi067 podman[187763]: 2024-07-31 15:18:10.452440653 +0000 UTC m=+5.754878571 container died 17ef155e188884dd37ddb4af9077b067c6684e8e6f8bf2f42f2e7f04adc3f2f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-07-31T15:18:11.053 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:10 smithi067 podman[187763]: 2024-07-31 15:18:10.913957761 +0000 UTC m=+6.216395669 container cleanup 17ef155e188884dd37ddb4af9077b067c6684e8e6f8bf2f42f2e7f04adc3f2f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716) 2024-07-31T15:18:11.053 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:10 smithi067 bash[187763]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9 2024-07-31T15:18:11.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:11 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:11.043+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:11.508 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:11 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:11.043+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:11.508 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:11 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:11.044+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:11.508 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:11 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:11.044+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:11.508 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:11 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:11.044+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:11.509 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:11 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:11.044+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:11.541 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:11 smithi067 podman[187777]: 2024-07-31 15:18:11.090280243 +0000 UTC m=+0.636187768 container remove 17ef155e188884dd37ddb4af9077b067c6684e8e6f8bf2f42f2e7f04adc3f2f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-07-31T15:18:11.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:11 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:11.371+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:11.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:11 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:11.371+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:11.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:11 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:11.371+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:11.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:11 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:11.371+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:11.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:11 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:11.371+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:11.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:11 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:11.371+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:11.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:11 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:11.371+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:40.864632+0000 front 2024-07-31T15:17:40.864818+0000 (oldest deadline 2024-07-31T15:18:09.664887+0000) 2024-07-31T15:18:11.803 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:11 smithi067 podman[187867]: 2024-07-31 15:18:11.54101595 +0000 UTC m=+0.110434133 container create 7d67486849193399b0d292d3b17cff05812c69720657c37eb3799b593e414af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc) 2024-07-31T15:18:11.803 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:11 smithi067 podman[187867]: 2024-07-31 15:18:11.448685882 +0000 UTC m=+0.018104066 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T15:18:11.803 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:11 smithi067 podman[187867]: 2024-07-31 15:18:11.760501641 +0000 UTC m=+0.329919833 container init 7d67486849193399b0d292d3b17cff05812c69720657c37eb3799b593e414af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-07-31T15:18:11.804 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:11 smithi067 podman[187867]: 2024-07-31 15:18:11.76411273 +0000 UTC m=+0.333530914 container start 7d67486849193399b0d292d3b17cff05812c69720657c37eb3799b593e414af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-07-31T15:18:12.246 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:11 smithi067 podman[187867]: 2024-07-31 15:18:11.843526771 +0000 UTC m=+0.412944969 container attach 7d67486849193399b0d292d3b17cff05812c69720657c37eb3799b593e414af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716) 2024-07-31T15:18:12.247 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 31 15:18:11 smithi067 podman[187867]: 2024-07-31 15:18:11.961170851 +0000 UTC m=+0.530589038 container died 7d67486849193399b0d292d3b17cff05812c69720657c37eb3799b593e414af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-9-deactivate, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-07-31T15:18:12.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:12 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:12.013+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:12.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:12 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:12.013+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:12.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:12 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:12.013+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:12.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:12 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:12.013+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:12.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:12 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:12.013+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:12.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:12 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:12.013+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:12.529 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.9.service' 2024-07-31T15:18:12.570 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:18:12.570 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-07-31T15:18:12.570 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-07-31T15:18:12.571 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.10 2024-07-31T15:18:12.855 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:12 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:12.404+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:12.855 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:12 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:12.404+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:12.855 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:12 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:12.404+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:12.855 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:12 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:12.404+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:12.855 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:12 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:12.404+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:12.855 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:12 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:12.404+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:12.856 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:12 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:12.404+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:40.864632+0000 front 2024-07-31T15:17:40.864818+0000 (oldest deadline 2024-07-31T15:18:09.664887+0000) 2024-07-31T15:18:12.856 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:12 smithi073 systemd[1]: Stopping Ceph osd.10 for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:18:12.856 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:12 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:12.853+0000 7fbfe457b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:18:12.856 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:12 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:12.853+0000 7fbfe457b640 -1 osd.10 144 *** Got signal Terminated *** 2024-07-31T15:18:12.856 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:12 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:12.853+0000 7fbfe457b640 -1 osd.10 144 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-31T15:18:13.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:13 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:13.026+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:13.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:13 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:13.026+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:13.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:13 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:13.026+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:13.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:13 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:13.026+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:13.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:13 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:13.026+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:13.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:13 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:13.026+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:13.642 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:13.367+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:13.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:13.367+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:13.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:13.367+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:13.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:13.367+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:13.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:13.367+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:13.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:13.367+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:13.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:13 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:13.367+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:40.864632+0000 front 2024-07-31T15:17:40.864818+0000 (oldest deadline 2024-07-31T15:18:09.664887+0000) 2024-07-31T15:18:14.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:14 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:14.015+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:14.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:14 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:14.015+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:14.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:14 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:14.015+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:14.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:14 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:14.015+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:14.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:14 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:14.015+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:14.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:14 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:14.015+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:14.892 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:14 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:14.398+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:14.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:14 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:14.398+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:14.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:14 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:14.398+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:14.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:14 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:14.398+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:14.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:14 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:14.398+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:14.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:14 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:14.398+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:14.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:14 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:14.398+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:40.864632+0000 front 2024-07-31T15:17:40.864818+0000 (oldest deadline 2024-07-31T15:18:09.664887+0000) 2024-07-31T15:18:15.237 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:14 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:14.965+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:15.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:14 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:14.965+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:15.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:14 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:14.965+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:15.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:14 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:14.965+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:15.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:14 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:14.965+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:15.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:14 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:14.965+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:15.892 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:15.427+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:15.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:15.427+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:15.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:15.427+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:15.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:15.427+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:15.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:15.427+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:15.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:15.427+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:15.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:15 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:15.427+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:40.864632+0000 front 2024-07-31T15:17:40.864818+0000 (oldest deadline 2024-07-31T15:18:09.664887+0000) 2024-07-31T15:18:16.237 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:15 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:15.943+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:16.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:15 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:15.943+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:16.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:15 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:15.943+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:16.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:15 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:15.943+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:16.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:15 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:15.943+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:16.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:15 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:15.943+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:16.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:15 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:15.943+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-07-31T15:17:50.996241+0000 front 2024-07-31T15:17:50.996265+0000 (oldest deadline 2024-07-31T15:18:15.696108+0000) 2024-07-31T15:18:16.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:16.409+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:16.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:16.409+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:16.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:16.409+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:16.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:16.409+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:16.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:16.409+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:16.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:16.409+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:16.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:16 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:16.409+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:40.864632+0000 front 2024-07-31T15:17:40.864818+0000 (oldest deadline 2024-07-31T15:18:09.664887+0000) 2024-07-31T15:18:17.237 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:16 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:16.958+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:17.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:16 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:16.958+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:17.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:16 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:16.958+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:17.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:16 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:16.958+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:17.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:16 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:16.958+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:17.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:16 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:16.958+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:17.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:16 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:16.958+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-07-31T15:17:50.996241+0000 front 2024-07-31T15:17:50.996265+0000 (oldest deadline 2024-07-31T15:18:15.696108+0000) 2024-07-31T15:18:17.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:17 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:17.399+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:51.256827+0000 front 2024-07-31T15:16:51.256933+0000 (oldest deadline 2024-07-31T15:17:16.557053+0000) 2024-07-31T15:18:17.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:17 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:17.399+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:16:59.958488+0000 front 2024-07-31T15:16:59.958456+0000 (oldest deadline 2024-07-31T15:17:25.858437+0000) 2024-07-31T15:18:17.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:17 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:17.399+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:08.660080+0000 front 2024-07-31T15:17:08.659913+0000 (oldest deadline 2024-07-31T15:17:34.560076+0000) 2024-07-31T15:18:17.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:17 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:17.399+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-31T15:17:19.161854+0000 front 2024-07-31T15:17:19.161756+0000 (oldest deadline 2024-07-31T15:17:43.861832+0000) 2024-07-31T15:18:17.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:17 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:17.399+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:27.962874+0000 front 2024-07-31T15:17:27.962929+0000 (oldest deadline 2024-07-31T15:17:53.863013+0000) 2024-07-31T15:18:17.643 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:17 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:17.399+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:33.863700+0000 front 2024-07-31T15:17:33.863571+0000 (oldest deadline 2024-07-31T15:17:59.763500+0000) 2024-07-31T15:18:17.644 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:17 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10[62533]: 2024-07-31T15:18:17.399+0000 7fbfdcb5e640 -1 osd.10 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:40.864632+0000 front 2024-07-31T15:17:40.864818+0000 (oldest deadline 2024-07-31T15:18:09.664887+0000) 2024-07-31T15:18:18.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:17 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:17.955+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:18.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:17 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:17.955+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:18.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:17 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:17.955+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:18.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:17 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:17.955+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:18.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:17 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:17.955+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:18.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:17 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:17.955+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:18.239 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:17 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:17.955+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-07-31T15:17:50.996241+0000 front 2024-07-31T15:17:50.996265+0000 (oldest deadline 2024-07-31T15:18:15.696108+0000) 2024-07-31T15:18:18.621 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:18 smithi073 podman[79917]: 2024-07-31 15:18:18.365536964 +0000 UTC m=+5.597907244 container died 2acd3da468f7c3734a06799b71c05592fa94143f26a91398c3e9b9d935fcb43f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc) 2024-07-31T15:18:18.892 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:18 smithi073 podman[79917]: 2024-07-31 15:18:18.711527668 +0000 UTC m=+5.943897960 container cleanup 2acd3da468f7c3734a06799b71c05592fa94143f26a91398c3e9b9d935fcb43f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-07-31T15:18:18.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:18 smithi073 bash[79917]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10 2024-07-31T15:18:19.237 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:18 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:18.961+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:19.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:18 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:18.961+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:19.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:18 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:18.961+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:19.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:18 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:18.961+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:19.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:18 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:18.961+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:19.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:18 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:18.961+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:19.238 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:18 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:18.961+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-07-31T15:17:50.996241+0000 front 2024-07-31T15:17:50.996265+0000 (oldest deadline 2024-07-31T15:18:15.696108+0000) 2024-07-31T15:18:19.335 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:18 smithi073 podman[79934]: 2024-07-31 15:18:18.938114146 +0000 UTC m=+0.568670948 container remove 2acd3da468f7c3734a06799b71c05592fa94143f26a91398c3e9b9d935fcb43f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, io.buildah.version=1.36.0) 2024-07-31T15:18:19.592 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:19 smithi073 podman[80026]: 2024-07-31 15:18:19.237786338 +0000 UTC m=+0.017200106 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T15:18:19.592 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:19 smithi073 podman[80026]: 2024-07-31 15:18:19.372280586 +0000 UTC m=+0.151694343 container create a4eac20fdc15876d2c5eff3d5c27194a3377664b2706edacad9ee944f72b115e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10-deactivate, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-07-31T15:18:19.892 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:19 smithi073 podman[80026]: 2024-07-31 15:18:19.591288932 +0000 UTC m=+0.370702691 container init a4eac20fdc15876d2c5eff3d5c27194a3377664b2706edacad9ee944f72b115e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc) 2024-07-31T15:18:19.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:19 smithi073 podman[80026]: 2024-07-31 15:18:19.595417304 +0000 UTC m=+0.374831076 container start a4eac20fdc15876d2c5eff3d5c27194a3377664b2706edacad9ee944f72b115e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-07-31T15:18:19.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:19 smithi073 podman[80026]: 2024-07-31 15:18:19.691218622 +0000 UTC m=+0.470632396 container attach a4eac20fdc15876d2c5eff3d5c27194a3377664b2706edacad9ee944f72b115e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-07-31T15:18:19.893 INFO:journalctl@ceph.osd.10.smithi073.stdout:Jul 31 15:18:19 smithi073 podman[80026]: 2024-07-31 15:18:19.789152976 +0000 UTC m=+0.568566748 container died a4eac20fdc15876d2c5eff3d5c27194a3377664b2706edacad9ee944f72b115e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-10-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-07-31T15:18:20.326 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.10.service' 2024-07-31T15:18:20.364 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:18:20.365 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-07-31T15:18:20.365 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-07-31T15:18:20.365 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.11 2024-07-31T15:18:20.392 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:20 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:20.009+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:20.392 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:20 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:20.009+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:20.392 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:20 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:20.009+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:20.392 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:20 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:20.009+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:20.392 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:20 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:20.009+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:20.392 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:20 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:20.009+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:20.393 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:20 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:20.009+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-07-31T15:17:50.996241+0000 front 2024-07-31T15:17:50.996265+0000 (oldest deadline 2024-07-31T15:18:15.696108+0000) 2024-07-31T15:18:20.657 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:20 smithi077 systemd[1]: Stopping Ceph osd.11 for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:18:20.987 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:20 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:20.655+0000 7f6b92db9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:18:20.987 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:20 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:20.655+0000 7f6b92db9640 -1 osd.11 144 *** Got signal Terminated *** 2024-07-31T15:18:20.987 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:20 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:20.656+0000 7f6b92db9640 -1 osd.11 144 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-31T15:18:21.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:21 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:21.021+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:21.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:21 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:21.021+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:21.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:21 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:21.021+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:21.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:21 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:21.021+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:21.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:21 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:21.021+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:21.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:21 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:21.021+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:21.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:21 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:21.021+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-07-31T15:17:50.996241+0000 front 2024-07-31T15:17:50.996265+0000 (oldest deadline 2024-07-31T15:18:15.696108+0000) 2024-07-31T15:18:22.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:22.035+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:22.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:22.035+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:22.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:22.035+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:22.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:22.035+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:22.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:22.035+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:22.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:22.035+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:22.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:22 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:22.035+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-07-31T15:17:50.996241+0000 front 2024-07-31T15:17:50.996265+0000 (oldest deadline 2024-07-31T15:18:15.696108+0000) 2024-07-31T15:18:23.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:23 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:23.061+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:23.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:23 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:23.061+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:23.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:23 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:23.061+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:23.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:23 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:23.061+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:23.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:23 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:23.061+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:23.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:23 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:23.061+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:23.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:23 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:23.061+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-07-31T15:17:50.996241+0000 front 2024-07-31T15:17:50.996265+0000 (oldest deadline 2024-07-31T15:18:15.696108+0000) 2024-07-31T15:18:24.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:24 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:24.082+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:24.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:24 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:24.082+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:24.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:24 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:24.082+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:24.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:24 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:24.082+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:24.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:24 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:24.082+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:24.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:24 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:24.082+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:24.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:24 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:24.082+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-07-31T15:17:50.996241+0000 front 2024-07-31T15:17:50.996265+0000 (oldest deadline 2024-07-31T15:18:15.696108+0000) 2024-07-31T15:18:25.389 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:25 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:25.089+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-31T15:16:53.787380+0000 front 2024-07-31T15:16:53.787400+0000 (oldest deadline 2024-07-31T15:17:16.687437+0000) 2024-07-31T15:18:25.389 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:25 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:25.089+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-07-31T15:17:02.588583+0000 front 2024-07-31T15:17:02.588430+0000 (oldest deadline 2024-07-31T15:17:27.788889+0000) 2024-07-31T15:18:25.389 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:25 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:25.089+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-07-31T15:17:11.289933+0000 front 2024-07-31T15:17:11.289993+0000 (oldest deadline 2024-07-31T15:17:36.590153+0000) 2024-07-31T15:18:25.389 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:25 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:25.089+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-07-31T15:17:25.792971+0000 front 2024-07-31T15:17:25.792955+0000 (oldest deadline 2024-07-31T15:17:51.092983+0000) 2024-07-31T15:18:25.389 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:25 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:25.089+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-07-31T15:17:32.794016+0000 front 2024-07-31T15:17:32.793995+0000 (oldest deadline 2024-07-31T15:17:58.694154+0000) 2024-07-31T15:18:25.390 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:25 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:25.089+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-31T15:17:44.595354+0000 front 2024-07-31T15:17:44.595414+0000 (oldest deadline 2024-07-31T15:18:09.295223+0000) 2024-07-31T15:18:25.390 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:25 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11[64197]: 2024-07-31T15:18:25.089+0000 7f6b8f3a4640 -1 osd.11 144 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-07-31T15:17:50.996241+0000 front 2024-07-31T15:17:50.996265+0000 (oldest deadline 2024-07-31T15:18:15.696108+0000) 2024-07-31T15:18:26.569 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:26 smithi077 podman[80742]: 2024-07-31 15:18:26.277201549 +0000 UTC m=+5.712982582 container died 227d5a558e4205e3973ccb6cec8f12b938108f2c06c427eeebf35fae8ea91827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-07-31T15:18:26.908 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:26 smithi077 podman[80742]: 2024-07-31 15:18:26.67351461 +0000 UTC m=+6.109295649 container cleanup 227d5a558e4205e3973ccb6cec8f12b938108f2c06c427eeebf35fae8ea91827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0) 2024-07-31T15:18:26.908 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:26 smithi077 bash[80742]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11 2024-07-31T15:18:27.232 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:26 smithi077 podman[80768]: 2024-07-31 15:18:26.908069725 +0000 UTC m=+0.631220804 container remove 227d5a558e4205e3973ccb6cec8f12b938108f2c06c427eeebf35fae8ea91827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716) 2024-07-31T15:18:27.487 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:27 smithi077 podman[80857]: 2024-07-31 15:18:27.134939731 +0000 UTC m=+0.017163241 image pull 2d83fc2832e3e825559bea1bb55f0606347710169126b757bbfacf69bad83f86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844 2024-07-31T15:18:27.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:27 smithi077 podman[80857]: 2024-07-31 15:18:27.24232972 +0000 UTC m=+0.124553224 container create c198bd7d33d993be61fde1c8247ff92d74ea86f467b49fdffc21cc9032259ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T15:18:27.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:27 smithi077 podman[80857]: 2024-07-31 15:18:27.436107933 +0000 UTC m=+0.318331437 container init c198bd7d33d993be61fde1c8247ff92d74ea86f467b49fdffc21cc9032259ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-07-31T15:18:27.488 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:27 smithi077 podman[80857]: 2024-07-31 15:18:27.439211919 +0000 UTC m=+0.321435427 container start c198bd7d33d993be61fde1c8247ff92d74ea86f467b49fdffc21cc9032259ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-07-31T15:18:27.918 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:27 smithi077 podman[80857]: 2024-07-31 15:18:27.519450573 +0000 UTC m=+0.401674090 container attach c198bd7d33d993be61fde1c8247ff92d74ea86f467b49fdffc21cc9032259ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-07-31T15:18:27.918 INFO:journalctl@ceph.osd.11.smithi077.stdout:Jul 31 15:18:27 smithi077 podman[80857]: 2024-07-31 15:18:27.625870269 +0000 UTC m=+0.508093774 container died c198bd7d33d993be61fde1c8247ff92d74ea86f467b49fdffc21cc9032259ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=) 2024-07-31T15:18:28.204 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@osd.11.service' 2024-07-31T15:18:28.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:18:28.285 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-07-31T15:18:28.285 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-07-31T15:18:28.285 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.a 2024-07-31T15:18:28.608 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 15:18:28 smithi067 systemd[1]: Stopping Ceph mds.a for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:18:28.881 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 15:18:28 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-a[80778]: 2024-07-31T15:18:28.606+0000 7f55f579d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:18:28.881 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 15:18:28 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-a[80778]: 2024-07-31T15:18:28.606+0000 7f55f579d640 -1 mds.a *** got signal Terminated *** 2024-07-31T15:18:31.192 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 15:18:30 smithi067 podman[187971]: 2024-07-31 15:18:30.91253082 +0000 UTC m=+2.417478276 container died 143db92221e1558924f6f773c239c984d9721ba3b212d8dc65c7f907c26eb84a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.build-date=20240716) 2024-07-31T15:18:31.553 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 15:18:31 smithi067 podman[187971]: 2024-07-31 15:18:31.381972389 +0000 UTC m=+2.886919834 container cleanup 143db92221e1558924f6f773c239c984d9721ba3b212d8dc65c7f907c26eb84a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc) 2024-07-31T15:18:31.553 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 31 15:18:31 smithi067 bash[187971]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-a 2024-07-31T15:18:31.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.a.service' 2024-07-31T15:18:31.845 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:18:31.845 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-07-31T15:18:31.845 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-07-31T15:18:31.846 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.d 2024-07-31T15:18:32.188 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 31 15:18:31 smithi067 systemd[1]: Stopping Ceph mds.d for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:18:32.553 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 31 15:18:32 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-d[79527]: 2024-07-31T15:18:32.186+0000 7fef51cf3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:18:32.553 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 31 15:18:32 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-d[79527]: 2024-07-31T15:18:32.186+0000 7fef51cf3640 -1 mds.d *** got signal Terminated *** 2024-07-31T15:18:36.553 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 31 15:18:36 smithi067 podman[188104]: 2024-07-31 15:18:36.206055347 +0000 UTC m=+4.126060817 container died 7062ce67ed1463ad1dd342c447dab5d70a993da00a300e535e486f10c111c649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-07-31T15:18:37.050 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 31 15:18:36 smithi067 podman[188104]: 2024-07-31 15:18:36.62832199 +0000 UTC m=+4.548327499 container cleanup 7062ce67ed1463ad1dd342c447dab5d70a993da00a300e535e486f10c111c649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-d, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, ceph=True, CEPH_POINT_RELEASE=) 2024-07-31T15:18:37.050 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 31 15:18:36 smithi067 bash[188104]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-d 2024-07-31T15:18:37.050 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 31 15:18:36 smithi067 podman[188115]: 2024-07-31 15:18:36.838089576 +0000 UTC m=+0.627220064 container remove 7062ce67ed1463ad1dd342c447dab5d70a993da00a300e535e486f10c111c649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-d, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.build-date=20240716, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-07-31T15:18:37.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.d.service' 2024-07-31T15:18:37.099 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:18:37.099 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-07-31T15:18:37.099 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-07-31T15:18:37.099 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.g 2024-07-31T15:18:37.441 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 31 15:18:37 smithi067 systemd[1]: Stopping Ceph mds.g for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:18:37.803 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 31 15:18:37 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-g[80345]: 2024-07-31T15:18:37.440+0000 7fa4555b8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:18:37.803 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 31 15:18:37 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-g[80345]: 2024-07-31T15:18:37.440+0000 7fa4555b8640 -1 mds.g *** got signal Terminated *** 2024-07-31T15:18:39.200 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 31 15:18:38 smithi067 podman[188225]: 2024-07-31 15:18:38.896102821 +0000 UTC m=+1.570042033 container died 8117ab345ba3c1ac5e350aaa6c7281a3b35801cd8626a432e8fb63aac743ffb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-07-31T15:18:39.553 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 31 15:18:39 smithi067 podman[188225]: 2024-07-31 15:18:39.38223962 +0000 UTC m=+2.056178870 container cleanup 8117ab345ba3c1ac5e350aaa6c7281a3b35801cd8626a432e8fb63aac743ffb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-31T15:18:39.553 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 31 15:18:39 smithi067 bash[188225]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-g 2024-07-31T15:18:39.829 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.g.service' 2024-07-31T15:18:39.863 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:18:39.863 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-07-31T15:18:39.863 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-07-31T15:18:39.864 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.j 2024-07-31T15:18:40.187 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 31 15:18:39 smithi067 systemd[1]: Stopping Ceph mds.j for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:18:40.553 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 31 15:18:40 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-j[79211]: 2024-07-31T15:18:40.184+0000 7f2e8e490640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:18:40.553 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 31 15:18:40 smithi067 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-j[79211]: 2024-07-31T15:18:40.184+0000 7f2e8e490640 -1 mds.j *** got signal Terminated *** 2024-07-31T15:18:42.122 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 31 15:18:41 smithi067 podman[188347]: 2024-07-31 15:18:41.868452074 +0000 UTC m=+1.775142669 container died 956bba6a593a085fa7e4f359f9d86a6a57039df842c388cae7ecd3a18524a838 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-j, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0) 2024-07-31T15:18:42.380 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 31 15:18:42 smithi067 podman[188347]: 2024-07-31 15:18:42.211793046 +0000 UTC m=+2.118483641 container cleanup 956bba6a593a085fa7e4f359f9d86a6a57039df842c388cae7ecd3a18524a838 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-07-31T15:18:42.380 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 31 15:18:42 smithi067 bash[188347]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-j 2024-07-31T15:18:42.609 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.j.service' 2024-07-31T15:18:42.631 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 31 15:18:42 smithi067 podman[188360]: 2024-07-31 15:18:42.379309721 +0000 UTC m=+0.506051346 container remove 956bba6a593a085fa7e4f359f9d86a6a57039df842c388cae7ecd3a18524a838 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-07-31T15:18:42.631 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 31 15:18:42 smithi067 systemd[1]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.j.service: Deactivated successfully. 2024-07-31T15:18:42.631 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 31 15:18:42 smithi067 systemd[1]: Stopped Ceph mds.j for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T15:18:42.631 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 31 15:18:42 smithi067 systemd[1]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.j.service: Consumed 1.315s CPU time. 2024-07-31T15:18:42.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:18:42.642 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-07-31T15:18:42.643 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-07-31T15:18:42.643 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.b 2024-07-31T15:18:42.964 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:18:42 smithi073 systemd[1]: Stopping Ceph mds.b for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:18:43.392 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:18:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:18:42.961+0000 7ff0bf1c2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:18:43.393 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:18:42 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b[67890]: 2024-07-31T15:18:42.962+0000 7ff0bf1c2640 -1 mds.b *** got signal Terminated *** 2024-07-31T15:18:47.142 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:18:46 smithi073 podman[80143]: 2024-07-31 15:18:46.866799817 +0000 UTC m=+4.022714228 container died 121ffbbb3f06f05b9f459fe274b9c2c58ced1a5547843d840829a5250181f8e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=) 2024-07-31T15:18:47.488 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:18:47 smithi073 podman[80143]: 2024-07-31 15:18:47.22006224 +0000 UTC m=+4.375976642 container cleanup 121ffbbb3f06f05b9f459fe274b9c2c58ced1a5547843d840829a5250181f8e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux ) 2024-07-31T15:18:47.488 INFO:journalctl@ceph.mds.b.smithi073.stdout:Jul 31 15:18:47 smithi073 bash[80143]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-b 2024-07-31T15:18:47.650 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.b.service' 2024-07-31T15:18:47.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:18:47.686 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-07-31T15:18:47.686 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-07-31T15:18:47.687 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.e 2024-07-31T15:18:48.000 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:18:47 smithi073 systemd[1]: Stopping Ceph mds.e for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:18:48.392 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:18:48 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[70341]: 2024-07-31T15:18:47.998+0000 7f1a75df4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:18:48.392 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:18:48 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e[70341]: 2024-07-31T15:18:47.998+0000 7f1a75df4640 -1 mds.e *** got signal Terminated *** 2024-07-31T15:18:51.170 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:18:50 smithi073 podman[80276]: 2024-07-31 15:18:50.898971198 +0000 UTC m=+2.995739288 container died d2b5d61a47c8fcabd761759224b6df40abfddd34f326ce81debf0d79dbfc294d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716) 2024-07-31T15:18:51.543 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:18:51 smithi073 podman[80276]: 2024-07-31 15:18:51.266261744 +0000 UTC m=+3.363029825 container cleanup d2b5d61a47c8fcabd761759224b6df40abfddd34f326ce81debf0d79dbfc294d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2) 2024-07-31T15:18:51.543 INFO:journalctl@ceph.mds.e.smithi073.stdout:Jul 31 15:18:51 smithi073 bash[80276]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-e 2024-07-31T15:18:51.680 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.e.service' 2024-07-31T15:18:51.712 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:18:51.712 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-07-31T15:18:51.712 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-07-31T15:18:51.713 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.h 2024-07-31T15:18:51.892 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 15:18:51 smithi073 systemd[1]: Stopping Ceph mds.h for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:18:52.392 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 15:18:52 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-h[67585]: 2024-07-31T15:18:52.027+0000 7f6434352640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:18:52.393 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 15:18:52 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-h[67585]: 2024-07-31T15:18:52.027+0000 7f6434352640 -1 mds.h *** got signal Terminated *** 2024-07-31T15:18:55.129 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 15:18:54 smithi073 podman[80397]: 2024-07-31 15:18:54.863350113 +0000 UTC m=+2.937097805 container died 461ee206e533d2666bca08ded406b1d6f2f0e926b441b7137bca8eb6db47b68e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-h, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240716) 2024-07-31T15:18:55.392 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 15:18:55 smithi073 podman[80397]: 2024-07-31 15:18:55.219832432 +0000 UTC m=+3.293580126 container cleanup 461ee206e533d2666bca08ded406b1d6f2f0e926b441b7137bca8eb6db47b68e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-31T15:18:55.393 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 15:18:55 smithi073 bash[80397]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-h 2024-07-31T15:18:55.648 INFO:journalctl@ceph.mds.h.smithi073.stdout:Jul 31 15:18:55 smithi073 podman[80409]: 2024-07-31 15:18:55.396023862 +0000 UTC m=+0.528933792 container remove 461ee206e533d2666bca08ded406b1d6f2f0e926b441b7137bca8eb6db47b68e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.build-date=20240716) 2024-07-31T15:18:55.652 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.h.service' 2024-07-31T15:18:55.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:18:55.683 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-07-31T15:18:55.683 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-07-31T15:18:55.683 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.k 2024-07-31T15:18:55.990 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 15:18:55 smithi073 systemd[1]: Stopping Ceph mds.k for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:18:56.392 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 15:18:55 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-k[66966]: 2024-07-31T15:18:55.989+0000 7f0b71b2c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:18:56.393 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 15:18:55 smithi073 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-k[66966]: 2024-07-31T15:18:55.989+0000 7f0b71b2c640 -1 mds.k *** got signal Terminated *** 2024-07-31T15:18:59.142 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 15:18:58 smithi073 podman[80517]: 2024-07-31 15:18:58.880296689 +0000 UTC m=+2.979385870 container died 60da7f5aff932b9745617a078643a427780eb967e27ca2b498356bd54ea3637e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-k, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-31T15:18:59.616 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 15:18:59 smithi073 podman[80517]: 2024-07-31 15:18:59.332088921 +0000 UTC m=+3.431178094 container cleanup 60da7f5aff932b9745617a078643a427780eb967e27ca2b498356bd54ea3637e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-07-31T15:18:59.617 INFO:journalctl@ceph.mds.k.smithi073.stdout:Jul 31 15:18:59 smithi073 bash[80517]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-k 2024-07-31T15:18:59.759 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.k.service' 2024-07-31T15:18:59.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:18:59.789 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-07-31T15:18:59.789 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-07-31T15:18:59.790 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.c 2024-07-31T15:19:00.083 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 15:18:59 smithi077 systemd[1]: Stopping Ceph mds.c for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:19:00.346 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 15:19:00 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-c[69602]: 2024-07-31T15:19:00.081+0000 7fa13153c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:19:00.347 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 15:19:00 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-c[69602]: 2024-07-31T15:19:00.081+0000 7fa13153c640 -1 mds.c *** got signal Terminated *** 2024-07-31T15:19:03.199 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 15:19:02 smithi077 podman[80959]: 2024-07-31 15:19:02.871004756 +0000 UTC m=+2.890753749 container died 71d4c7cc57d35f3086f37369d072b051c5b6f87e0c6a524a24938fb4c8356439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-07-31T15:19:03.487 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 15:19:03 smithi077 podman[80959]: 2024-07-31 15:19:03.297997091 +0000 UTC m=+3.317746085 container cleanup 71d4c7cc57d35f3086f37369d072b051c5b6f87e0c6a524a24938fb4c8356439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-c, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-31T15:19:03.487 INFO:journalctl@ceph.mds.c.smithi077.stdout:Jul 31 15:19:03 smithi077 bash[80959]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-c 2024-07-31T15:19:03.728 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.c.service' 2024-07-31T15:19:03.762 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:19:03.763 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-07-31T15:19:03.763 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-07-31T15:19:03.763 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.f 2024-07-31T15:19:04.086 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 15:19:03 smithi077 systemd[1]: Stopping Ceph mds.f for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:19:04.487 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 15:19:04 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-f[69298]: 2024-07-31T15:19:04.084+0000 7f8a680eb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:19:04.487 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 15:19:04 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-f[69298]: 2024-07-31T15:19:04.084+0000 7f8a680eb640 -1 mds.f *** got signal Terminated *** 2024-07-31T15:19:07.128 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 15:19:06 smithi077 podman[81095]: 2024-07-31 15:19:06.871648685 +0000 UTC m=+2.895386336 container died 9519cd747ec15aafa13d69f8751dde29794255449f38c16fcbee8ed50b395ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-f, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc) 2024-07-31T15:19:07.412 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 15:19:07 smithi077 podman[81095]: 2024-07-31 15:19:07.227098723 +0000 UTC m=+3.250836420 container cleanup 9519cd747ec15aafa13d69f8751dde29794255449f38c16fcbee8ed50b395ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-f, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-07-31T15:19:07.412 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 15:19:07 smithi077 bash[81095]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-f 2024-07-31T15:19:07.650 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.f.service' 2024-07-31T15:19:07.670 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 15:19:07 smithi077 podman[81107]: 2024-07-31 15:19:07.411528198 +0000 UTC m=+0.536689759 container remove 9519cd747ec15aafa13d69f8751dde29794255449f38c16fcbee8ed50b395ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc) 2024-07-31T15:19:07.670 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 15:19:07 smithi077 systemd[1]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.f.service: Deactivated successfully. 2024-07-31T15:19:07.671 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 15:19:07 smithi077 systemd[1]: Stopped Ceph mds.f for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968. 2024-07-31T15:19:07.671 INFO:journalctl@ceph.mds.f.smithi077.stdout:Jul 31 15:19:07 smithi077 systemd[1]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.f.service: Consumed 1min 50.505s CPU time. 2024-07-31T15:19:07.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:19:07.682 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-07-31T15:19:07.682 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-07-31T15:19:07.683 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.i 2024-07-31T15:19:07.987 INFO:journalctl@ceph.mds.i.smithi077.stdout:Jul 31 15:19:07 smithi077 systemd[1]: Stopping Ceph mds.i for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:19:08.487 INFO:journalctl@ceph.mds.i.smithi077.stdout:Jul 31 15:19:07 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-i[69903]: 2024-07-31T15:19:07.996+0000 7fe445a26640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:19:08.487 INFO:journalctl@ceph.mds.i.smithi077.stdout:Jul 31 15:19:07 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-i[69903]: 2024-07-31T15:19:07.996+0000 7fe445a26640 -1 mds.i *** got signal Terminated *** 2024-07-31T15:19:11.132 INFO:journalctl@ceph.mds.i.smithi077.stdout:Jul 31 15:19:10 smithi077 podman[81218]: 2024-07-31 15:19:10.863913537 +0000 UTC m=+2.964327654 container died c5369626e9f012373be002839c4250df98f5ba365ef7c7539555c2b504b1505b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-i, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-07-31T15:19:11.487 INFO:journalctl@ceph.mds.i.smithi077.stdout:Jul 31 15:19:11 smithi077 podman[81218]: 2024-07-31 15:19:11.213992154 +0000 UTC m=+3.314406254 container cleanup c5369626e9f012373be002839c4250df98f5ba365ef7c7539555c2b504b1505b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-07-31T15:19:11.488 INFO:journalctl@ceph.mds.i.smithi077.stdout:Jul 31 15:19:11 smithi077 bash[81218]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-i 2024-07-31T15:19:11.644 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.i.service' 2024-07-31T15:19:11.676 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:19:11.676 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-07-31T15:19:11.676 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-07-31T15:19:11.676 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.l 2024-07-31T15:19:12.237 INFO:journalctl@ceph.mds.l.smithi077.stdout:Jul 31 15:19:11 smithi077 systemd[1]: Stopping Ceph mds.l for 5ca2b71e-4f4a-11ef-bcc9-c7b262605968... 2024-07-31T15:19:12.237 INFO:journalctl@ceph.mds.l.smithi077.stdout:Jul 31 15:19:12 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-l[68994]: 2024-07-31T15:19:12.000+0000 7f459ec78640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-31T15:19:12.238 INFO:journalctl@ceph.mds.l.smithi077.stdout:Jul 31 15:19:12 smithi077 ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-l[68994]: 2024-07-31T15:19:12.000+0000 7f459ec78640 -1 mds.l *** got signal Terminated *** 2024-07-31T15:19:15.127 INFO:journalctl@ceph.mds.l.smithi077.stdout:Jul 31 15:19:14 smithi077 podman[81337]: 2024-07-31 15:19:14.858528524 +0000 UTC m=+2.961026967 container died 40b7e4bfb4acde6c0fb3b6c07f99f88888f7889d6313fbdb0d2c76c1009be06d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-l, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-07-31T15:19:15.403 INFO:journalctl@ceph.mds.l.smithi077.stdout:Jul 31 15:19:15 smithi077 podman[81337]: 2024-07-31 15:19:15.201371674 +0000 UTC m=+3.303870106 container cleanup 40b7e4bfb4acde6c0fb3b6c07f99f88888f7889d6313fbdb0d2c76c1009be06d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:38f71995ceee6af1dfe9130d95269f2783d6cae1f2968d9a066b91b8db9fb844, name=ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240723.143739-debug-8c796fc, org.label-schema.schema-version=1.0) 2024-07-31T15:19:15.403 INFO:journalctl@ceph.mds.l.smithi077.stdout:Jul 31 15:19:15 smithi077 bash[81337]: ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968-mds-l 2024-07-31T15:19:15.620 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ca2b71e-4f4a-11ef-bcc9-c7b262605968@mds.l.service' 2024-07-31T15:19:15.650 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-31T15:19:15.651 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-07-31T15:19:15.652 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 --force --keep-logs 2024-07-31T15:19:15.870 INFO:teuthology.orchestra.run.smithi067.stdout:Deleting cluster with fsid: 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T15:19:18.550 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 --force --keep-logs 2024-07-31T15:19:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:Deleting cluster with fsid: 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T15:19:21.644 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 --force --keep-logs 2024-07-31T15:19:21.848 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T15:19:24.285 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-31T15:19:24.320 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-31T15:19:24.352 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-31T15:19:24.377 INFO:tasks.cephadm:Archiving crash dumps... 2024-07-31T15:19:24.379 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-07-31_13:38:24-fs-wip-vshankar-testing-20240723.143739-debug-testing-default-smithi/7828591/remote/smithi067/crash 2024-07-31T15:19:24.381 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/crash -- . 2024-07-31T15:19:24.410 INFO:teuthology.orchestra.run.smithi067.stderr:tar: /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/crash: Cannot open: No such file or directory 2024-07-31T15:19:24.411 INFO:teuthology.orchestra.run.smithi067.stderr:tar: Error is not recoverable: exiting now 2024-07-31T15:19:24.413 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-07-31_13:38:24-fs-wip-vshankar-testing-20240723.143739-debug-testing-default-smithi/7828591/remote/smithi073/crash 2024-07-31T15:19:24.414 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/crash -- . 2024-07-31T15:19:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:tar: /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/crash: Cannot open: No such file or directory 2024-07-31T15:19:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:tar: Error is not recoverable: exiting now 2024-07-31T15:19:24.490 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-07-31_13:38:24-fs-wip-vshankar-testing-20240723.143739-debug-testing-default-smithi/7828591/remote/smithi077/crash 2024-07-31T15:19:24.491 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/crash -- . 2024-07-31T15:19:24.522 INFO:teuthology.orchestra.run.smithi077.stderr:tar: /var/lib/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/crash: Cannot open: No such file or directory 2024-07-31T15:19:24.522 INFO:teuthology.orchestra.run.smithi077.stderr:tar: Error is not recoverable: exiting now 2024-07-31T15:19:24.523 INFO:tasks.cephadm:Checking cluster log for badness... 2024-07-31T15:19:24.524 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-07-31T15:19:24.666 INFO:teuthology.orchestra.run.smithi067.stdout:2024-07-31T14:57:04.288263+0000 mds.b (mds.0) 40 : cluster [WRN] Scrub error on inode 0x1000000162a (/volumes/qa/sv_1/67428b9b-e567-4ac7-8a12-63fc5983a051/client.0/tmp/t/linux-6.5.11/arch) see mds.b log and `damage ls` output for details 2024-07-31T15:19:24.670 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-07-31T15:19:24.670 INFO:tasks.cephadm:Compressing logs... 2024-07-31T15:19:24.670 DEBUG:teuthology.orchestra.run.smithi067:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-31T15:19:24.673 DEBUG:teuthology.orchestra.run.smithi073:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-31T15:19:24.676 DEBUG:teuthology.orchestra.run.smithi077:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-31T15:19:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-31T15:19:24.701 INFO:teuthology.orchestra.run.smithi067.stderr:: No such file or directory 2024-07-31T15:19:24.701 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.log 2024-07-31T15:19:24.703 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.audit.log 2024-07-31T15:19:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.g.log 2024-07-31T15:19:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.j.log 2024-07-31T15:19:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.3.log 2024-07-31T15:19:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.0.log 2024-07-31T15:19:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mgr.x.log 2024-07-31T15:19:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mon.a.log 2024-07-31T15:19:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.6.log 2024-07-31T15:19:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.cephadm.log 2024-07-31T15:19:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.a.log 2024-07-31T15:19:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.6.log: /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-volume.log 2024-07-31T15:19:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.d.log 2024-07-31T15:19:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.9.log 2024-07-31T15:19:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-07-31T15:19:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.9.log: /var/log/ceph/ceph-client.admin.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-07-31T15:19:24.713 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.log.gz 2024-07-31T15:19:24.716 INFO:teuthology.orchestra.run.smithi073.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-07-31T15:19:24.717 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-31T15:19:24.717 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.log 2024-07-31T15:19:24.717 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.audit.log 2024-07-31T15:19:24.717 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.7.log 2024-07-31T15:19:24.717 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.log: /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.b.log 2024-07-31T15:19:24.717 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.4.log 2024-07-31T15:19:24.717 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.1.log 2024-07-31T15:19:24.718 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mgr.y.log 2024-07-31T15:19:24.718 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.h.log 2024-07-31T15:19:24.718 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.10.log 2024-07-31T15:19:24.718 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.cephadm.log 2024-07-31T15:19:24.719 INFO:teuthology.orchestra.run.smithi077.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-07-31T15:19:24.721 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.e.log 2024-07-31T15:19:24.721 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-volume.log 2024-07-31T15:19:24.721 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mon.b.log 2024-07-31T15:19:24.721 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.k.log 2024-07-31T15:19:24.722 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-31T15:19:24.723 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.log 2024-07-31T15:19:24.723 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.2.log 2024-07-31T15:19:24.723 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.f.log 2024-07-31T15:19:24.723 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.audit.log 2024-07-31T15:19:24.726 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.5.log 2024-07-31T15:19:24.726 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.c.log 2024-07-31T15:19:24.726 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mon.c.log 2024-07-31T15:19:24.727 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mgr.z.log 2024-07-31T15:19:24.727 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.i.log 2024-07-31T15:19:24.727 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.cephadm.log 2024-07-31T15:19:24.727 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.11.log 2024-07-31T15:19:24.727 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.8.log 2024-07-31T15:19:24.727 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-volume.log 2024-07-31T15:19:24.727 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.l.log 2024-07-31T15:19:24.775 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mon.b.log: /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.k.log: 82.6% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.cephadm.log.gz 2024-07-31T15:19:24.808 INFO:teuthology.orchestra.run.smithi067.stderr: 90.2% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.audit.log.gz 2024-07-31T15:19:24.812 INFO:teuthology.orchestra.run.smithi067.stderr: 95.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-31T15:19:24.843 INFO:teuthology.orchestra.run.smithi073.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-31T15:19:24.858 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-volume.log: /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.l.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-31T15:19:24.890 INFO:teuthology.orchestra.run.smithi067.stderr: 94.6% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.j.log.gz 2024-07-31T15:19:24.908 INFO:teuthology.orchestra.run.smithi077.stderr: 82.5% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.cephadm.log.gz 2024-07-31T15:19:24.918 INFO:teuthology.orchestra.run.smithi077.stderr: 90.3% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.audit.log.gz 2024-07-31T15:19:24.923 INFO:teuthology.orchestra.run.smithi067.stderr: 94.6% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.d.log.gz 2024-07-31T15:19:24.948 INFO:teuthology.orchestra.run.smithi067.stderr: 91.1% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-volume.log.gz 2024-07-31T15:19:24.962 INFO:teuthology.orchestra.run.smithi067.stderr: 84.7% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.cephadm.log.gz 2024-07-31T15:19:24.973 INFO:teuthology.orchestra.run.smithi073.stderr: 91.0% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-volume.log.gz 2024-07-31T15:19:25.037 INFO:teuthology.orchestra.run.smithi073.stderr: 93.4% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mgr.y.log.gz 2024-07-31T15:19:25.059 INFO:teuthology.orchestra.run.smithi077.stderr: 91.0% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-volume.log.gz 2024-07-31T15:19:25.212 INFO:teuthology.orchestra.run.smithi073.stderr: 90.3% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.audit.log.gz 2024-07-31T15:19:26.099 INFO:teuthology.orchestra.run.smithi067.stderr: 94.2% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.g.log.gz 2024-07-31T15:19:26.398 INFO:teuthology.orchestra.run.smithi077.stderr: 93.5% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mgr.z.log.gz 2024-07-31T15:19:26.555 INFO:teuthology.orchestra.run.smithi073.stderr: 89.6% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.log.gz 2024-07-31T15:19:27.233 INFO:teuthology.orchestra.run.smithi067.stderr: 90.7% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mgr.x.log.gz 2024-07-31T15:19:27.592 INFO:teuthology.orchestra.run.smithi067.stderr: 94.6% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.a.log.gz 2024-07-31T15:19:27.916 INFO:teuthology.orchestra.run.smithi077.stderr: 89.6% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph.log.gz 2024-07-31T15:19:31.186 INFO:teuthology.orchestra.run.smithi067.stderr: 90.3% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mon.a.log.gz 2024-07-31T15:19:35.125 INFO:teuthology.orchestra.run.smithi073.stderr: 91.5% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mon.b.log.gz 2024-07-31T15:19:41.937 INFO:teuthology.orchestra.run.smithi077.stderr: 91.5% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mon.c.log.gz 2024-07-31T15:19:54.393 INFO:teuthology.orchestra.run.smithi077.stderr: 95.5% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.l.log.gz 2024-07-31T15:20:02.356 INFO:teuthology.orchestra.run.smithi073.stderr: 95.2% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.k.log.gz 2024-07-31T15:20:39.761 INFO:teuthology.orchestra.run.smithi067.stderr: 92.0% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.3.log.gz 2024-07-31T15:21:10.054 INFO:teuthology.orchestra.run.smithi067.stderr: 92.2% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.6.log.gz 2024-07-31T15:21:13.816 INFO:teuthology.orchestra.run.smithi067.stderr: 92.2% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.0.log.gz 2024-07-31T15:21:45.847 INFO:teuthology.orchestra.run.smithi067.stderr: 92.3% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.9.log.gz 2024-07-31T15:21:45.849 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-07-31T15:21:45.849 INFO:teuthology.orchestra.run.smithi067.stderr:real 2m21.160s 2024-07-31T15:21:45.849 INFO:teuthology.orchestra.run.smithi067.stderr:user 4m4.300s 2024-07-31T15:21:45.849 INFO:teuthology.orchestra.run.smithi067.stderr:sys 0m8.616s 2024-07-31T15:24:11.016 INFO:teuthology.orchestra.run.smithi073.stderr: 94.9% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.e.log.gz 2024-07-31T15:28:37.118 INFO:teuthology.orchestra.run.smithi073.stderr: 92.0% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.10.log.gz 2024-07-31T15:29:40.757 INFO:teuthology.orchestra.run.smithi077.stderr: 92.0% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.8.log.gz 2024-07-31T15:30:38.672 INFO:teuthology.orchestra.run.smithi077.stderr: 96.1% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.f.log.gz 2024-07-31T15:31:09.809 INFO:teuthology.orchestra.run.smithi073.stderr: 92.1% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.1.log.gz 2024-07-31T15:32:14.764 INFO:teuthology.orchestra.run.smithi073.stderr: 92.0% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.7.log.gz 2024-07-31T15:32:33.107 INFO:teuthology.orchestra.run.smithi077.stderr: 95.9% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.c.log.gz 2024-07-31T15:33:55.683 INFO:teuthology.orchestra.run.smithi077.stderr: 92.5% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.11.log.gz 2024-07-31T15:37:01.040 INFO:teuthology.orchestra.run.smithi077.stderr: 92.4% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.2.log.gz 2024-07-31T15:38:33.809 INFO:teuthology.orchestra.run.smithi077.stderr: 92.2% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.5.log.gz 2024-07-31T15:38:34.153 INFO:teuthology.orchestra.run.smithi073.stderr: 96.0% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.h.log.gz 2024-07-31T15:39:22.634 INFO:teuthology.orchestra.run.smithi073.stderr: 92.8% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-osd.4.log.gz 2024-07-31T15:39:27.525 INFO:teuthology.orchestra.run.smithi077.stderr: 95.6% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.i.log.gz 2024-07-31T15:39:27.526 INFO:teuthology.orchestra.run.smithi077.stderr: 2024-07-31T15:39:27.527 INFO:teuthology.orchestra.run.smithi077.stderr:real 20m2.837s 2024-07-31T15:39:27.527 INFO:teuthology.orchestra.run.smithi077.stderr:user 6m49.534s 2024-07-31T15:39:27.527 INFO:teuthology.orchestra.run.smithi077.stderr:sys 0m21.658s 2024-07-31T15:44:05.444 INFO:teuthology.orchestra.run.smithi073.stderr: 95.0% -- replaced with /var/log/ceph/5ca2b71e-4f4a-11ef-bcc9-c7b262605968/ceph-mds.b.log.gz 2024-07-31T15:44:05.447 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-07-31T15:44:05.447 INFO:teuthology.orchestra.run.smithi073.stderr:real 24m40.755s 2024-07-31T15:44:05.447 INFO:teuthology.orchestra.run.smithi073.stderr:user 9m57.541s 2024-07-31T15:44:05.448 INFO:teuthology.orchestra.run.smithi073.stderr:sys 0m29.626s 2024-07-31T15:44:05.448 INFO:tasks.cephadm:Archiving logs... 2024-07-31T15:44:05.548 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-07-31_13:38:24-fs-wip-vshankar-testing-20240723.143739-debug-testing-default-smithi/7828591/remote/smithi067/log 2024-07-31T15:44:05.549 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-31T15:44:46.168 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-07-31_13:38:24-fs-wip-vshankar-testing-20240723.143739-debug-testing-default-smithi/7828591/remote/smithi073/log 2024-07-31T15:44:46.169 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-31T15:46:10.383 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-07-31_13:38:24-fs-wip-vshankar-testing-20240723.143739-debug-testing-default-smithi/7828591/remote/smithi077/log 2024-07-31T15:46:10.384 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-31T15:47:10.390 INFO:tasks.cephadm:Removing cluster... 2024-07-31T15:47:10.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 --force 2024-07-31T15:47:10.612 INFO:teuthology.orchestra.run.smithi067.stdout:Deleting cluster with fsid: 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T15:47:11.220 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 --force 2024-07-31T15:47:11.524 INFO:teuthology.orchestra.run.smithi073.stdout:Deleting cluster with fsid: 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T15:47:12.154 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 --force 2024-07-31T15:47:12.517 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: 5ca2b71e-4f4a-11ef-bcc9-c7b262605968 2024-07-31T15:47:13.202 INFO:tasks.cephadm:Removing cephadm ... 2024-07-31T15:47:13.202 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-31T15:47:13.223 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-31T15:47:13.241 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-31T15:47:13.257 INFO:tasks.cephadm:Teardown complete 2024-07-31T15:47:13.257 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-07-31T15:47:13.272 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-07-31T15:47:13.272 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-31T15:47:13.275 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-31T15:47:13.284 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-31T15:47:13.579 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-07-31T15:47:13.590 INFO:teuthology.task.clock:Checking final clock skew... 2024-07-31T15:47:13.590 DEBUG:teuthology.orchestra.run.smithi067:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-31T15:47:13.593 DEBUG:teuthology.orchestra.run.smithi073:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-31T15:47:13.596 DEBUG:teuthology.orchestra.run.smithi077:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-31T15:47:13.609 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-07-31T15:47:13.610 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-07-31T15:47:13.610 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-07-31T15:47:13.614 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-31T15:47:13.614 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-07-31T15:47:13.614 INFO:teuthology.orchestra.run.smithi067.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +224us[ +225us] +/- 42ms 2024-07-31T15:47:13.614 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 52 -1023us[-1023us] +/- 73ms 2024-07-31T15:47:13.614 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +758us[ +758us] +/- 75ms 2024-07-31T15:47:13.614 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T15:47:13.624 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-31T15:47:13.624 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-07-31T15:47:13.624 INFO:teuthology.orchestra.run.smithi077.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 +217us[ +245us] +/- 42ms 2024-07-31T15:47:13.625 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -1042us[-1014us] +/- 73ms 2024-07-31T15:47:13.625 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +722us[ +722us] +/- 75ms 2024-07-31T15:47:13.625 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T15:47:13.642 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-31T15:47:13.643 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-07-31T15:47:13.643 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv01.front.sepia.ceph.com 3 6 377 46 +186us[ +171us] +/- 42ms 2024-07-31T15:47:13.643 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 48 -1066us[-1080us] +/- 73ms 2024-07-31T15:47:13.643 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +724us[ +709us] +/- 75ms 2024-07-31T15:47:13.643 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-07-31T15:47:13.644 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-07-31T15:47:13.692 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-07-31T15:47:13.692 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-07-31T15:47:13.744 DEBUG:teuthology.orchestra.run.smithi067:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-31T15:47:13.924 DEBUG:teuthology.orchestra.run.smithi073:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-31T15:47:14.048 DEBUG:teuthology.orchestra.run.smithi077:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-31T15:47:14.127 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-31T15:47:14.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-31T15:47:14.161 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-31T15:47:14.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-31T15:47:14.191 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-31T15:47:14.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-31T15:47:14.219 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-07-31T15:47:14.231 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-07-31T15:47:14.278 INFO:teuthology.task.internal:Duration was 4613.081241 seconds 2024-07-31T15:47:14.278 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-07-31T15:47:14.412 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-07-31T15:47:14.412 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-31T15:47:14.415 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-31T15:47:14.417 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-31T15:47:14.478 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-31T15:47:14.481 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-31T15:47:14.492 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-31T15:47:14.944 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-07-31T15:47:14.944 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi067.front.sepia.ceph.com 2024-07-31T15:47:14.944 DEBUG:teuthology.orchestra.run.smithi067:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-31T15:47:14.970 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi073.front.sepia.ceph.com 2024-07-31T15:47:14.971 DEBUG:teuthology.orchestra.run.smithi073:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-31T15:47:15.017 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi077.front.sepia.ceph.com 2024-07-31T15:47:15.018 DEBUG:teuthology.orchestra.run.smithi077:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-31T15:47:15.047 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-07-31T15:47:15.048 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-31T15:47:15.051 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-31T15:47:15.055 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-31T15:47:15.139 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-07-31T15:47:15.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-31T15:47:15.143 DEBUG:teuthology.orchestra.run.smithi073:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-31T15:47:15.146 DEBUG:teuthology.orchestra.run.smithi077:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-31T15:47:17.086 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-07-31T15:47:17.099 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-07-31T15:47:17.099 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-31T15:47:17.126 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-31T15:47:17.156 DEBUG:teuthology.orchestra.run.smithi077:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-31T15:47:17.324 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-07-31T15:47:17.336 DEBUG:teuthology.orchestra.run.smithi067:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-31T15:47:17.339 DEBUG:teuthology.orchestra.run.smithi073:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-31T15:47:17.341 DEBUG:teuthology.orchestra.run.smithi077:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-31T15:47:17.372 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = core 2024-07-31T15:47:17.375 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = core 2024-07-31T15:47:17.384 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = core 2024-07-31T15:47:17.406 DEBUG:teuthology.orchestra.run.smithi067:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-31T15:47:17.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-31T15:47:17.446 DEBUG:teuthology.orchestra.run.smithi073:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-31T15:47:17.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-31T15:47:17.462 DEBUG:teuthology.orchestra.run.smithi077:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-31T15:47:17.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-31T15:47:17.479 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-07-31T15:47:17.491 INFO:teuthology.task.internal:Transferring archived files... 2024-07-31T15:47:17.493 DEBUG:teuthology.misc:Transferring archived files from smithi067:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-07-31_13:38:24-fs-wip-vshankar-testing-20240723.143739-debug-testing-default-smithi/7828591/remote/smithi067 2024-07-31T15:47:17.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-31T15:47:17.575 DEBUG:teuthology.misc:Transferring archived files from smithi073:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-07-31_13:38:24-fs-wip-vshankar-testing-20240723.143739-debug-testing-default-smithi/7828591/remote/smithi073 2024-07-31T15:47:17.575 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-31T15:47:17.615 DEBUG:teuthology.misc:Transferring archived files from smithi077:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-07-31_13:38:24-fs-wip-vshankar-testing-20240723.143739-debug-testing-default-smithi/7828591/remote/smithi077 2024-07-31T15:47:17.615 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-31T15:47:17.656 INFO:teuthology.task.internal:Removing archive directory... 2024-07-31T15:47:17.656 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-31T15:47:17.658 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-31T15:47:17.661 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-31T15:47:17.706 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-07-31T15:47:17.719 INFO:teuthology.task.internal:Not uploading archives. 2024-07-31T15:47:17.719 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-07-31T15:47:17.744 INFO:teuthology.task.internal:Tidying up after the test... 2024-07-31T15:47:17.745 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-31T15:47:17.747 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-31T15:47:17.749 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-31T15:47:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: 265604 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 31 15:47 /home/ubuntu/cephtest 2024-07-31T15:47:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 31 15:47 /home/ubuntu/cephtest 2024-07-31T15:47:17.765 INFO:teuthology.orchestra.run.smithi077.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 31 15:47 /home/ubuntu/cephtest 2024-07-31T15:47:17.767 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-07-31T15:47:17.780 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-07-31T15:47:17.849 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} duration: 4613.08124089241 failure_reason: 'Command failed (workunit test kernel_untar_build.sh) on smithi067 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8c796fca5ff5960f7542e7227bad4a784ee2c804 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh''' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=631ab2ff4488427db1b0d3b92e8c3583 status: fail success: false 2024-07-31T15:47:17.849 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-31T15:47:17.941 INFO:teuthology.run:FAIL